2024-09-12T02:41:56.613 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-12T02:41:56.637 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T02:41:56.740 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901644 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7901644' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported dry_run: true install: ceph: flavor: default sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 481 sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc sleep_before_teardown: 0 subset: 57/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi116.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBGrxTzMNTBcm4tGhc/gT8EPfRnN4HhjQ6OvFeKr0q0BcqeLqQvniBS4GpuMJXuHCv0e34GMbBto7jf1dxYHAts= smithi177.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJswOwjNu1eIb2FtPOhaZdO2BHBR/U0MHx82bfI7/W3aBcY5s3gXeOaSqDbkmVXZjlPm+g3ZNEFiPBjyn0DJ/4Y= smithi183.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNF7PTVxyTIRAsw7WGoidQFH+Qw31h478TjHgboxkQa3WsdGGXkpIILTaP7kiswiqh+2rfIySQ+6MTNbp/4S9eI= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - quiescer: null - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-12T02:41:56.741 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa; will attempt to use it 2024-09-12T02:41:56.741 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks 2024-09-12T02:41:56.742 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-12T02:41:56.743 INFO:teuthology.task.internal:Checking packages... 2024-09-12T02:41:56.765 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc' 2024-09-12T02:41:56.765 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-12T02:41:56.766 INFO:teuthology.packaging:ref: None 2024-09-12T02:41:56.766 INFO:teuthology.packaging:tag: None 2024-09-12T02:41:56.766 INFO:teuthology.packaging:branch: squid 2024-09-12T02:41:56.766 INFO:teuthology.packaging:sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:41:56.766 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-12T02:41:56.980 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-314.g086c737f 2024-09-12T02:41:56.981 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-12T02:41:56.989 INFO:teuthology.task.internal:no buildpackages task found 2024-09-12T02:41:56.989 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-12T02:41:56.996 INFO:teuthology.task.internal:Saving configuration 2024-09-12T02:41:57.015 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-12T02:41:57.022 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-12T02:41:57.044 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi116.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901644', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 02:37:00.731130', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBGrxTzMNTBcm4tGhc/gT8EPfRnN4HhjQ6OvFeKr0q0BcqeLqQvniBS4GpuMJXuHCv0e34GMbBto7jf1dxYHAts='} 2024-09-12T02:41:57.067 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi177.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901644', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 02:37:00.730151', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJswOwjNu1eIb2FtPOhaZdO2BHBR/U0MHx82bfI7/W3aBcY5s3gXeOaSqDbkmVXZjlPm+g3ZNEFiPBjyn0DJ/4Y='} 2024-09-12T02:41:57.087 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901644', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 02:37:00.728976', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNF7PTVxyTIRAsw7WGoidQFH+Qw31h478TjHgboxkQa3WsdGGXkpIILTaP7kiswiqh+2rfIySQ+6MTNbp/4S9eI='} 2024-09-12T02:41:57.087 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-12T02:41:57.093 INFO:teuthology.task.internal:roles: ubuntu@smithi116.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-12T02:41:57.094 INFO:teuthology.task.internal:roles: ubuntu@smithi177.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-12T02:41:57.094 INFO:teuthology.task.internal:roles: ubuntu@smithi183.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-12T02:41:57.094 INFO:teuthology.run_tasks:Running task console_log... 2024-09-12T02:41:57.181 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4cce137be0>, signals=[15]) 2024-09-12T02:41:57.181 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-12T02:41:57.188 INFO:teuthology.task.internal:Opening connections... 2024-09-12T02:41:57.188 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi116.front.sepia.ceph.com 2024-09-12T02:41:57.189 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:41:57.268 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi177.front.sepia.ceph.com 2024-09-12T02:41:57.269 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:41:57.345 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi183.front.sepia.ceph.com 2024-09-12T02:41:57.345 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:41:57.419 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-12T02:41:57.426 DEBUG:teuthology.orchestra.run.smithi116:> uname -m 2024-09-12T02:41:57.444 INFO:teuthology.orchestra.run.smithi116.stdout:x86_64 2024-09-12T02:41:57.445 DEBUG:teuthology.orchestra.run.smithi116:> cat /etc/os-release 2024-09-12T02:41:57.502 INFO:teuthology.orchestra.run.smithi116.stdout:NAME="CentOS Stream" 2024-09-12T02:41:57.502 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION="9" 2024-09-12T02:41:57.502 INFO:teuthology.orchestra.run.smithi116.stdout:ID="centos" 2024-09-12T02:41:57.502 INFO:teuthology.orchestra.run.smithi116.stdout:ID_LIKE="rhel fedora" 2024-09-12T02:41:57.502 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_ID="9" 2024-09-12T02:41:57.502 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM_ID="platform:el9" 2024-09-12T02:41:57.502 INFO:teuthology.orchestra.run.smithi116.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T02:41:57.502 INFO:teuthology.orchestra.run.smithi116.stdout:ANSI_COLOR="0;31" 2024-09-12T02:41:57.502 INFO:teuthology.orchestra.run.smithi116.stdout:LOGO="fedora-logo-icon" 2024-09-12T02:41:57.503 INFO:teuthology.orchestra.run.smithi116.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T02:41:57.503 INFO:teuthology.orchestra.run.smithi116.stdout:HOME_URL="https://centos.org/" 2024-09-12T02:41:57.503 INFO:teuthology.orchestra.run.smithi116.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T02:41:57.503 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T02:41:57.503 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T02:41:57.503 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server 2024-09-12T02:41:57.523 DEBUG:teuthology.orchestra.run.smithi177:> uname -m 2024-09-12T02:41:57.541 INFO:teuthology.orchestra.run.smithi177.stdout:x86_64 2024-09-12T02:41:57.542 DEBUG:teuthology.orchestra.run.smithi177:> cat /etc/os-release 2024-09-12T02:41:57.600 INFO:teuthology.orchestra.run.smithi177.stdout:NAME="CentOS Stream" 2024-09-12T02:41:57.600 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION="9" 2024-09-12T02:41:57.600 INFO:teuthology.orchestra.run.smithi177.stdout:ID="centos" 2024-09-12T02:41:57.600 INFO:teuthology.orchestra.run.smithi177.stdout:ID_LIKE="rhel fedora" 2024-09-12T02:41:57.600 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION_ID="9" 2024-09-12T02:41:57.600 INFO:teuthology.orchestra.run.smithi177.stdout:PLATFORM_ID="platform:el9" 2024-09-12T02:41:57.600 INFO:teuthology.orchestra.run.smithi177.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T02:41:57.600 INFO:teuthology.orchestra.run.smithi177.stdout:ANSI_COLOR="0;31" 2024-09-12T02:41:57.600 INFO:teuthology.orchestra.run.smithi177.stdout:LOGO="fedora-logo-icon" 2024-09-12T02:41:57.600 INFO:teuthology.orchestra.run.smithi177.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T02:41:57.601 INFO:teuthology.orchestra.run.smithi177.stdout:HOME_URL="https://centos.org/" 2024-09-12T02:41:57.601 INFO:teuthology.orchestra.run.smithi177.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T02:41:57.601 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T02:41:57.601 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T02:41:57.601 INFO:teuthology.lock.ops:Updating smithi177.front.sepia.ceph.com on lock server 2024-09-12T02:41:57.778 DEBUG:teuthology.orchestra.run.smithi183:> uname -m 2024-09-12T02:41:57.797 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2024-09-12T02:41:57.797 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2024-09-12T02:41:57.855 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream" 2024-09-12T02:41:57.855 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="9" 2024-09-12T02:41:57.856 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos" 2024-09-12T02:41:57.856 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora" 2024-09-12T02:41:57.856 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="9" 2024-09-12T02:41:57.856 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el9" 2024-09-12T02:41:57.856 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T02:41:57.856 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31" 2024-09-12T02:41:57.856 INFO:teuthology.orchestra.run.smithi183.stdout:LOGO="fedora-logo-icon" 2024-09-12T02:41:57.856 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T02:41:57.856 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/" 2024-09-12T02:41:57.856 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T02:41:57.856 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T02:41:57.856 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T02:41:57.857 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2024-09-12T02:41:57.873 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-12T02:41:57.883 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-12T02:41:57.889 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-12T02:41:57.889 DEBUG:teuthology.orchestra.run.smithi116:> test '!' -e /home/ubuntu/cephtest 2024-09-12T02:41:57.892 DEBUG:teuthology.orchestra.run.smithi177:> test '!' -e /home/ubuntu/cephtest 2024-09-12T02:41:57.894 DEBUG:teuthology.orchestra.run.smithi183:> test '!' -e /home/ubuntu/cephtest 2024-09-12T02:41:57.912 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-12T02:41:57.919 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-12T02:41:57.919 DEBUG:teuthology.orchestra.run.smithi116:> test -z $(ls -A /var/lib/ceph) 2024-09-12T02:41:57.950 DEBUG:teuthology.orchestra.run.smithi177:> test -z $(ls -A /var/lib/ceph) 2024-09-12T02:41:57.952 DEBUG:teuthology.orchestra.run.smithi183:> test -z $(ls -A /var/lib/ceph) 2024-09-12T02:41:58.055 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-12T02:41:58.125 INFO:teuthology.run_tasks:Running task kernel... 2024-09-12T02:41:58.139 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-12T02:41:58.139 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-12T02:41:58.139 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-12T02:41:58.140 DEBUG:teuthology.orchestra.run.smithi116:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T02:41:58.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:41:58.158 DEBUG:teuthology.orchestra.run.smithi116:> uname -r 2024-09-12T02:41:58.215 INFO:teuthology.orchestra.run.smithi116.stdout:5.14.0-503.el9.x86_64 2024-09-12T02:41:58.215 INFO:teuthology.task.kernel:Running kernel on smithi116: 5.14.0-503.el9.x86_64 2024-09-12T02:41:58.215 DEBUG:teuthology.orchestra.run.smithi116:> sudo yum install -y kernel 2024-09-12T02:42:00.814 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - BaseOS 8.8 MB/s | 8.2 MB 00:00 2024-09-12T02:42:02.765 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - AppStream 24 MB/s | 20 MB 00:00 2024-09-12T02:42:07.966 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - CRB 4.6 MB/s | 6.5 MB 00:01 2024-09-12T02:42:10.083 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 19 kB 00:00 2024-09-12T02:42:10.920 INFO:teuthology.orchestra.run.smithi116.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-12T02:42:16.343 INFO:teuthology.orchestra.run.smithi116.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-12T02:42:18.357 INFO:teuthology.orchestra.run.smithi116.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-12T02:42:18.358 INFO:teuthology.orchestra.run.smithi116.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-12T02:42:18.451 INFO:teuthology.orchestra.run.smithi116.stdout:Dependencies resolved. 2024-09-12T02:42:18.459 INFO:teuthology.orchestra.run.smithi116.stdout:================================================================================ 2024-09-12T02:42:18.459 INFO:teuthology.orchestra.run.smithi116.stdout: Package Architecture Version Repository Size 2024-09-12T02:42:18.459 INFO:teuthology.orchestra.run.smithi116.stdout:================================================================================ 2024-09-12T02:42:18.460 INFO:teuthology.orchestra.run.smithi116.stdout:Installing: 2024-09-12T02:42:18.460 INFO:teuthology.orchestra.run.smithi116.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-12T02:42:18.460 INFO:teuthology.orchestra.run.smithi116.stdout:Installing dependencies: 2024-09-12T02:42:18.460 INFO:teuthology.orchestra.run.smithi116.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-12T02:42:18.460 INFO:teuthology.orchestra.run.smithi116.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-12T02:42:18.460 INFO:teuthology.orchestra.run.smithi116.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-12T02:42:18.460 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:42:18.460 INFO:teuthology.orchestra.run.smithi116.stdout:Transaction Summary 2024-09-12T02:42:18.460 INFO:teuthology.orchestra.run.smithi116.stdout:================================================================================ 2024-09-12T02:42:18.460 INFO:teuthology.orchestra.run.smithi116.stdout:Install 4 Packages 2024-09-12T02:42:18.460 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:42:18.461 INFO:teuthology.orchestra.run.smithi116.stdout:Total download size: 79 M 2024-09-12T02:42:18.461 INFO:teuthology.orchestra.run.smithi116.stdout:Installed size: 126 M 2024-09-12T02:42:18.461 INFO:teuthology.orchestra.run.smithi116.stdout:Downloading Packages: 2024-09-12T02:42:18.686 INFO:teuthology.orchestra.run.smithi116.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 248 kB/s | 23 kB 00:00 2024-09-12T02:42:19.094 INFO:teuthology.orchestra.run.smithi116.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-09-12T02:42:19.512 INFO:teuthology.orchestra.run.smithi116.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 38 MB/s | 35 MB 00:00 2024-09-12T02:42:19.912 INFO:teuthology.orchestra.run.smithi116.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 23 MB/s | 29 MB 00:01 2024-09-12T02:42:19.912 INFO:teuthology.orchestra.run.smithi116.stdout:-------------------------------------------------------------------------------- 2024-09-12T02:42:19.912 INFO:teuthology.orchestra.run.smithi116.stdout:Total 54 MB/s | 79 MB 00:01 2024-09-12T02:42:20.589 INFO:teuthology.orchestra.run.smithi116.stdout:Running transaction check 2024-09-12T02:42:20.948 INFO:teuthology.orchestra.run.smithi116.stdout:Transaction check succeeded. 2024-09-12T02:42:20.948 INFO:teuthology.orchestra.run.smithi116.stdout:Running transaction test 2024-09-12T02:42:21.931 INFO:teuthology.orchestra.run.smithi116.stdout:Transaction test succeeded. 2024-09-12T02:42:21.932 INFO:teuthology.orchestra.run.smithi116.stdout:Running transaction 2024-09-12T02:42:24.356 INFO:teuthology.orchestra.run.smithi116.stdout: Preparing : 1/1 2024-09-12T02:42:25.005 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-12T02:42:25.095 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T02:42:27.897 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T02:42:28.024 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T02:42:32.842 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T02:42:32.957 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:42:37.797 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:43:15.840 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:43:15.879 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:43:16.230 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:43:16.230 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-12T02:43:16.230 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T02:43:16.230 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T02:43:17.055 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:43:17.056 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:43:17.056 INFO:teuthology.orchestra.run.smithi116.stdout:Installed: 2024-09-12T02:43:17.056 INFO:teuthology.orchestra.run.smithi116.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-12T02:43:17.056 INFO:teuthology.orchestra.run.smithi116.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-12T02:43:17.056 INFO:teuthology.orchestra.run.smithi116.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-12T02:43:17.056 INFO:teuthology.orchestra.run.smithi116.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-12T02:43:17.056 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:43:17.056 INFO:teuthology.orchestra.run.smithi116.stdout:Complete! 2024-09-12T02:43:17.420 DEBUG:teuthology.orchestra.run.smithi116:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T02:43:17.498 INFO:teuthology.orchestra.run.smithi116.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T02:43:17.498 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T02:43:17.498 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-12T02:43:17.498 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-12T02:43:17.498 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-12T02:43:17.499 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-12T02:43:17.499 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi116.front.sepia.ceph.com, path=None, version=distro) 2024-09-12T02:43:17.499 DEBUG:teuthology.orchestra.run.smithi116:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T02:43:17.557 INFO:teuthology.orchestra.run.smithi116.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T02:43:17.557 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T02:43:17.557 DEBUG:teuthology.orchestra.run.smithi116:> sudo rpm -qi grub2-tools 2024-09-12T02:43:17.596 INFO:teuthology.orchestra.run.smithi116.stdout:Name : grub2-tools 2024-09-12T02:43:17.596 INFO:teuthology.orchestra.run.smithi116.stdout:Epoch : 1 2024-09-12T02:43:17.596 INFO:teuthology.orchestra.run.smithi116.stdout:Version : 2.06 2024-09-12T02:43:17.597 INFO:teuthology.orchestra.run.smithi116.stdout:Release : 82.el9 2024-09-12T02:43:17.597 INFO:teuthology.orchestra.run.smithi116.stdout:Architecture: x86_64 2024-09-12T02:43:17.597 INFO:teuthology.orchestra.run.smithi116.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-12T02:43:17.597 INFO:teuthology.orchestra.run.smithi116.stdout:Group : Unspecified 2024-09-12T02:43:17.597 INFO:teuthology.orchestra.run.smithi116.stdout:Size : 8274670 2024-09-12T02:43:17.597 INFO:teuthology.orchestra.run.smithi116.stdout:License : GPLv3+ 2024-09-12T02:43:17.597 INFO:teuthology.orchestra.run.smithi116.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-12T02:43:17.597 INFO:teuthology.orchestra.run.smithi116.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-12T02:43:17.597 INFO:teuthology.orchestra.run.smithi116.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-12T02:43:17.597 INFO:teuthology.orchestra.run.smithi116.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-12T02:43:17.597 INFO:teuthology.orchestra.run.smithi116.stdout:Packager : builder@centos.org 2024-09-12T02:43:17.597 INFO:teuthology.orchestra.run.smithi116.stdout:Vendor : CentOS 2024-09-12T02:43:17.597 INFO:teuthology.orchestra.run.smithi116.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-12T02:43:17.598 INFO:teuthology.orchestra.run.smithi116.stdout:Summary : Support tools for GRUB. 2024-09-12T02:43:17.598 INFO:teuthology.orchestra.run.smithi116.stdout:Description : 2024-09-12T02:43:17.598 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:43:17.598 INFO:teuthology.orchestra.run.smithi116.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-12T02:43:17.598 INFO:teuthology.orchestra.run.smithi116.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-12T02:43:17.598 INFO:teuthology.orchestra.run.smithi116.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-12T02:43:17.598 INFO:teuthology.orchestra.run.smithi116.stdout:hardware devices. 2024-09-12T02:43:17.598 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:43:17.598 INFO:teuthology.orchestra.run.smithi116.stdout:This subpackage provides tools for support of all platforms. 2024-09-12T02:43:17.599 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-12T02:43:17.599 INFO:teuthology.task.kernel:Updating grub on smithi116 to boot 5.14.0-505.el9.x86_64 2024-09-12T02:43:17.599 DEBUG:teuthology.orchestra.run.smithi116:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-12T02:43:18.305 INFO:teuthology.orchestra.run.smithi116.stderr:Generating grub configuration file ... 2024-09-12T02:43:19.635 INFO:teuthology.orchestra.run.smithi116.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-12T02:43:19.667 INFO:teuthology.orchestra.run.smithi116.stderr:done 2024-09-12T02:43:19.670 DEBUG:teuthology.orchestra.run.smithi116:> mktemp 2024-09-12T02:43:19.685 INFO:teuthology.orchestra.run.smithi116.stdout:/tmp/tmp.Blz3Nicwod 2024-09-12T02:43:19.686 DEBUG:teuthology.orchestra.run.smithi116:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Blz3Nicwod 2024-09-12T02:43:19.757 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 0666 /tmp/tmp.Blz3Nicwod 2024-09-12T02:43:19.898 DEBUG:teuthology.orchestra.remote:smithi116:/tmp/tmp.Blz3Nicwod is 6KB 2024-09-12T02:43:19.946 DEBUG:teuthology.orchestra.run.smithi116:> rm -fr /tmp/tmp.Blz3Nicwod 2024-09-12T02:43:19.960 DEBUG:teuthology.orchestra.run.smithi116:> sudo /bin/ls /boot/loader/entries || true 2024-09-12T02:43:20.023 INFO:teuthology.orchestra.run.smithi116.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-12T02:43:20.024 INFO:teuthology.orchestra.run.smithi116.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-12T02:43:20.024 INFO:teuthology.orchestra.run.smithi116.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-12T02:43:20.024 INFO:teuthology.orchestra.run.smithi116.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-12T02:43:20.025 DEBUG:teuthology.orchestra.run.smithi116:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-12T02:43:20.166 DEBUG:teuthology.orchestra.run.smithi116:> sudo shutdown -r now 2024-09-12T02:43:50.169 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-12T02:43:50.170 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-09-12T02:43:50.171 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:44:08.637 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.116 2024-09-12T02:44:17.638 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-09-12T02:44:17.639 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:44:20.701 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.116 2024-09-12T02:44:32.702 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-09-12T02:44:32.702 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:45:04.744 DEBUG:teuthology.orchestra.run.smithi116:> true 2024-09-12T02:45:05.116 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-09-12T02:45:05.117 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-12T02:45:05.117 DEBUG:teuthology.orchestra.run.smithi116:> uname -r 2024-09-12T02:45:05.135 INFO:teuthology.orchestra.run.smithi116.stdout:5.14.0-505.el9.x86_64 2024-09-12T02:45:05.135 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-12T02:45:05.136 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-12T02:45:05.136 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-12T02:45:06.137 DEBUG:teuthology.parallel:result is None 2024-09-12T02:45:06.137 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-12T02:45:06.144 INFO:teuthology.task.internal:Creating test directory... 2024-09-12T02:45:06.144 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T02:45:06.147 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T02:45:06.150 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T02:45:06.190 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-12T02:45:06.197 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-12T02:45:06.204 INFO:teuthology.task.internal:Creating archive directory... 2024-09-12T02:45:06.204 DEBUG:teuthology.orchestra.run.smithi116:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T02:45:06.209 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T02:45:06.211 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T02:45:06.256 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-12T02:45:06.313 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-12T02:45:06.313 DEBUG:teuthology.orchestra.run.smithi177:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T02:45:06.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:45:06.342 DEBUG:teuthology.orchestra.run.smithi183:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T02:45:06.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:45:06.368 DEBUG:teuthology.orchestra.run.smithi116:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T02:45:06.394 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T02:45:06.396 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T02:45:06.444 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T02:45:06.466 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T02:45:06.492 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T02:45:06.512 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T02:45:06.533 INFO:teuthology.orchestra.run.smithi116.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T02:45:06.595 INFO:teuthology.orchestra.run.smithi116.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T02:45:06.598 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-12T02:45:07.064 INFO:teuthology.task.internal:Configuring sudo... 2024-09-12T02:45:07.064 DEBUG:teuthology.orchestra.run.smithi116:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T02:45:07.067 DEBUG:teuthology.orchestra.run.smithi177:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T02:45:07.069 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T02:45:07.099 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-12T02:45:07.146 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-12T02:45:07.146 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T02:45:07.149 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T02:45:07.152 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T02:45:07.170 DEBUG:teuthology.orchestra.run.smithi116:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T02:45:07.245 DEBUG:teuthology.orchestra.run.smithi116:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T02:45:07.379 DEBUG:teuthology.orchestra.run.smithi116:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T02:45:07.422 DEBUG:teuthology.orchestra.run.smithi116:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T02:45:07.461 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-12T02:45:07.461 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T02:45:07.526 DEBUG:teuthology.orchestra.run.smithi177:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T02:45:07.570 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T02:45:07.626 DEBUG:teuthology.orchestra.run.smithi177:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T02:45:07.710 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T02:45:07.755 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-12T02:45:07.755 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T02:45:07.821 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T02:45:07.867 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T02:45:08.174 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T02:45:08.221 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T02:45:08.263 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-12T02:45:08.263 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T02:45:08.332 DEBUG:teuthology.orchestra.run.smithi116:> sudo service rsyslog restart 2024-09-12T02:45:08.334 DEBUG:teuthology.orchestra.run.smithi177:> sudo service rsyslog restart 2024-09-12T02:45:08.338 DEBUG:teuthology.orchestra.run.smithi183:> sudo service rsyslog restart 2024-09-12T02:45:08.376 INFO:teuthology.orchestra.run.smithi177.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T02:45:08.383 INFO:teuthology.orchestra.run.smithi116.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T02:45:08.415 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T02:45:08.894 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-12T02:45:08.901 INFO:teuthology.task.internal:Starting timer... 2024-09-12T02:45:08.901 INFO:teuthology.run_tasks:Running task pcp... 2024-09-12T02:45:08.911 INFO:teuthology.run_tasks:Running task selinux... 2024-09-12T02:45:08.919 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-12T02:45:08.919 DEBUG:teuthology.orchestra.run.smithi116:> sudo service auditd rotate 2024-09-12T02:45:08.998 INFO:teuthology.orchestra.run.smithi116.stdout:Rotating logs: 2024-09-12T02:45:09.000 DEBUG:teuthology.orchestra.run.smithi177:> sudo service auditd rotate 2024-09-12T02:45:09.070 INFO:teuthology.orchestra.run.smithi177.stdout:Rotating logs: 2024-09-12T02:45:09.072 DEBUG:teuthology.orchestra.run.smithi183:> sudo service auditd rotate 2024-09-12T02:45:09.140 INFO:teuthology.orchestra.run.smithi183.stdout:Rotating logs: 2024-09-12T02:45:09.142 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-12T02:45:09.142 DEBUG:teuthology.orchestra.run.smithi116:> /usr/sbin/getenforce 2024-09-12T02:45:09.175 INFO:teuthology.orchestra.run.smithi116.stdout:Permissive 2024-09-12T02:45:09.176 DEBUG:teuthology.orchestra.run.smithi177:> /usr/sbin/getenforce 2024-09-12T02:45:09.204 INFO:teuthology.orchestra.run.smithi177.stdout:Permissive 2024-09-12T02:45:09.204 DEBUG:teuthology.orchestra.run.smithi183:> /usr/sbin/getenforce 2024-09-12T02:45:09.230 INFO:teuthology.orchestra.run.smithi183.stdout:Permissive 2024-09-12T02:45:09.230 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi116.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi177.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi183.front.sepia.ceph.com': 'permissive'} 2024-09-12T02:45:09.230 DEBUG:teuthology.orchestra.run.smithi116:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T02:45:09.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:45:09.259 DEBUG:teuthology.orchestra.run.smithi177:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T02:45:09.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:45:09.288 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T02:45:09.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:45:09.314 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-12T02:45:09.315 DEBUG:teuthology.orchestra.run.smithi116:> sudo /usr/sbin/setenforce permissive 2024-09-12T02:45:09.345 DEBUG:teuthology.orchestra.run.smithi177:> sudo /usr/sbin/setenforce permissive 2024-09-12T02:45:09.384 DEBUG:teuthology.orchestra.run.smithi183:> sudo /usr/sbin/setenforce permissive 2024-09-12T02:45:09.419 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-12T02:45:09.429 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-12T02:45:09.444 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-12T02:45:09.445 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi116.front.sepia.ceph.com,smithi177.front.sepia.ceph.com,smithi183.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-12T02:49:15.641 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi116.front.sepia.ceph.com'), Remote(name='ubuntu@smithi177.front.sepia.ceph.com'), Remote(name='ubuntu@smithi183.front.sepia.ceph.com')] 2024-09-12T02:49:15.642 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-09-12T02:49:15.643 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:49:15.727 DEBUG:teuthology.orchestra.run.smithi116:> true 2024-09-12T02:49:15.800 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-09-12T02:49:15.801 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-09-12T02:49:15.801 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:49:15.874 DEBUG:teuthology.orchestra.run.smithi177:> true 2024-09-12T02:49:15.955 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-09-12T02:49:15.956 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-09-12T02:49:15.957 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:49:16.044 DEBUG:teuthology.orchestra.run.smithi183:> true 2024-09-12T02:49:16.127 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-09-12T02:49:16.129 INFO:teuthology.run_tasks:Running task clock... 2024-09-12T02:49:16.141 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-12T02:49:16.142 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T02:49:16.142 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T02:49:16.145 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T02:49:16.145 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T02:49:16.147 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T02:49:16.147 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T02:49:16.175 INFO:teuthology.orchestra.run.smithi116.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T02:49:16.178 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T02:49:16.191 INFO:teuthology.orchestra.run.smithi116.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T02:49:16.194 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T02:49:16.198 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T02:49:16.213 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T02:49:16.217 INFO:teuthology.orchestra.run.smithi116.stderr:sudo: ntpd: command not found 2024-09-12T02:49:16.220 INFO:teuthology.orchestra.run.smithi177.stderr:sudo: ntpd: command not found 2024-09-12T02:49:16.228 INFO:teuthology.orchestra.run.smithi116.stdout:506 Cannot talk to daemon 2024-09-12T02:49:16.231 INFO:teuthology.orchestra.run.smithi177.stdout:506 Cannot talk to daemon 2024-09-12T02:49:16.237 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: ntpd: command not found 2024-09-12T02:49:16.242 INFO:teuthology.orchestra.run.smithi116.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T02:49:16.246 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T02:49:16.248 INFO:teuthology.orchestra.run.smithi183.stdout:506 Cannot talk to daemon 2024-09-12T02:49:16.257 INFO:teuthology.orchestra.run.smithi116.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T02:49:16.261 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T02:49:16.262 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T02:49:16.276 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T02:49:16.313 INFO:teuthology.orchestra.run.smithi177.stderr:bash: line 1: ntpq: command not found 2024-09-12T02:49:16.317 INFO:teuthology.orchestra.run.smithi177.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T02:49:16.317 INFO:teuthology.orchestra.run.smithi177.stdout:=============================================================================== 2024-09-12T02:49:16.317 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:49:16.317 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:49:16.317 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:49:16.317 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:49:16.319 INFO:teuthology.orchestra.run.smithi116.stderr:bash: line 1: ntpq: command not found 2024-09-12T02:49:16.322 INFO:teuthology.orchestra.run.smithi116.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T02:49:16.322 INFO:teuthology.orchestra.run.smithi116.stdout:=============================================================================== 2024-09-12T02:49:16.322 INFO:teuthology.orchestra.run.smithi116.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:49:16.322 INFO:teuthology.orchestra.run.smithi116.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:49:16.322 INFO:teuthology.orchestra.run.smithi116.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:49:16.322 INFO:teuthology.orchestra.run.smithi116.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:49:16.333 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-09-12T02:49:16.336 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T02:49:16.337 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-09-12T02:49:16.337 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:49:16.337 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:49:16.337 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:49:16.337 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:49:16.338 INFO:teuthology.run_tasks:Running task install... 2024-09-12T02:49:16.349 DEBUG:teuthology.task.install:project ceph 2024-09-12T02:49:16.349 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}} 2024-09-12T02:49:16.350 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'} 2024-09-12T02:49:16.350 INFO:teuthology.task.install:Using flavor: default 2024-09-12T02:49:16.355 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-12T02:49:16.356 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-12T02:49:16.356 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T02:49:16.357 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:49:16.359 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T02:49:16.359 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:49:16.361 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T02:49:16.361 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:49:16.522 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T02:49:16.522 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T02:49:16.526 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T02:49:16.526 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T02:49:16.530 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T02:49:16.531 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T02:49:16.646 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T02:49:16.646 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-12T02:49:16.646 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T02:49:16.651 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T02:49:16.651 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-12T02:49:16.651 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T02:49:16.667 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T02:49:16.667 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-12T02:49:16.667 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T02:49:16.672 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T02:49:16.672 DEBUG:teuthology.orchestra.run.smithi116:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T02:49:16.677 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T02:49:16.677 DEBUG:teuthology.orchestra.run.smithi177:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T02:49:16.693 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T02:49:16.694 DEBUG:teuthology.orchestra.run.smithi183:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T02:49:16.739 DEBUG:teuthology.orchestra.run.smithi116:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T02:49:16.743 DEBUG:teuthology.orchestra.run.smithi177:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T02:49:16.759 DEBUG:teuthology.orchestra.run.smithi183:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T02:49:16.814 DEBUG:teuthology.orchestra.run.smithi116:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T02:49:16.818 DEBUG:teuthology.orchestra.run.smithi177:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T02:49:16.834 DEBUG:teuthology.orchestra.run.smithi183:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T02:49:16.878 INFO:teuthology.orchestra.run.smithi116.stdout:check_obsoletes = 1 2024-09-12T02:49:16.880 DEBUG:teuthology.orchestra.run.smithi116:> sudo yum clean all 2024-09-12T02:49:16.883 INFO:teuthology.orchestra.run.smithi177.stdout:check_obsoletes = 1 2024-09-12T02:49:16.884 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum clean all 2024-09-12T02:49:16.898 INFO:teuthology.orchestra.run.smithi183.stdout:check_obsoletes = 1 2024-09-12T02:49:16.900 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum clean all 2024-09-12T02:49:17.100 INFO:teuthology.orchestra.run.smithi116.stdout:45 files removed 2024-09-12T02:49:17.111 INFO:teuthology.orchestra.run.smithi177.stdout:45 files removed 2024-09-12T02:49:17.126 INFO:teuthology.orchestra.run.smithi183.stdout:45 files removed 2024-09-12T02:49:17.127 DEBUG:teuthology.orchestra.run.smithi116:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T02:49:17.139 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T02:49:17.153 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T02:49:17.676 INFO:teuthology.orchestra.run.smithi116.stdout:ceph packages for x86_64 319 kB/s | 84 kB 00:00 2024-09-12T02:49:17.688 INFO:teuthology.orchestra.run.smithi177.stdout:ceph packages for x86_64 307 kB/s | 84 kB 00:00 2024-09-12T02:49:17.701 INFO:teuthology.orchestra.run.smithi183.stdout:ceph packages for x86_64 308 kB/s | 84 kB 00:00 2024-09-12T02:49:17.943 INFO:teuthology.orchestra.run.smithi116.stdout:ceph noarch packages 52 kB/s | 12 kB 00:00 2024-09-12T02:49:17.964 INFO:teuthology.orchestra.run.smithi177.stdout:ceph noarch packages 50 kB/s | 12 kB 00:00 2024-09-12T02:49:17.968 INFO:teuthology.orchestra.run.smithi183.stdout:ceph noarch packages 52 kB/s | 12 kB 00:00 2024-09-12T02:49:18.210 INFO:teuthology.orchestra.run.smithi116.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-12T02:49:18.223 INFO:teuthology.orchestra.run.smithi177.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-12T02:49:18.235 INFO:teuthology.orchestra.run.smithi183.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-12T02:49:18.765 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.2 MB 00:00 2024-09-12T02:49:19.521 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - BaseOS 6.5 MB/s | 8.2 MB 00:01 2024-09-12T02:49:20.541 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - AppStream 29 MB/s | 20 MB 00:00 2024-09-12T02:49:20.727 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - BaseOS 3.4 MB/s | 8.2 MB 00:02 2024-09-12T02:49:21.320 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-09-12T02:49:22.578 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - AppStream 29 MB/s | 20 MB 00:00 2024-09-12T02:49:24.775 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-12T02:49:25.954 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - CRB 7.7 MB/s | 6.5 MB 00:00 2024-09-12T02:49:26.534 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 19 kB 00:00 2024-09-12T02:49:27.121 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - CRB 8.7 MB/s | 6.5 MB 00:00 2024-09-12T02:49:27.496 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - Extras packages 89 kB/s | 19 kB 00:00 2024-09-12T02:49:27.572 INFO:teuthology.orchestra.run.smithi177.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-12T02:49:28.451 INFO:teuthology.orchestra.run.smithi116.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-12T02:49:28.755 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 19 kB 00:00 2024-09-12T02:49:29.626 INFO:teuthology.orchestra.run.smithi183.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-12T02:49:32.919 INFO:teuthology.orchestra.run.smithi177.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2024-09-12T02:49:33.957 INFO:teuthology.orchestra.run.smithi116.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-09-12T02:49:35.065 INFO:teuthology.orchestra.run.smithi183.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-12T02:49:35.190 INFO:teuthology.orchestra.run.smithi177.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T02:49:35.191 INFO:teuthology.orchestra.run.smithi177.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T02:49:35.204 INFO:teuthology.orchestra.run.smithi177.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T02:49:35.205 INFO:teuthology.orchestra.run.smithi177.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T02:49:35.206 INFO:teuthology.orchestra.run.smithi177.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T02:49:35.207 INFO:teuthology.orchestra.run.smithi177.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T02:49:35.209 INFO:teuthology.orchestra.run.smithi177.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T02:49:35.212 INFO:teuthology.orchestra.run.smithi177.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T02:49:35.212 INFO:teuthology.orchestra.run.smithi177.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T02:49:35.274 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-09-12T02:49:35.293 INFO:teuthology.orchestra.run.smithi177.stdout:====================================================================================== 2024-09-12T02:49:35.293 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-09-12T02:49:35.293 INFO:teuthology.orchestra.run.smithi177.stdout:====================================================================================== 2024-09-12T02:49:35.293 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-09-12T02:49:35.294 INFO:teuthology.orchestra.run.smithi177.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T02:49:35.294 INFO:teuthology.orchestra.run.smithi177.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T02:49:35.294 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T02:49:35.294 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T02:49:35.294 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T02:49:35.294 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T02:49:35.294 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T02:49:35.294 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T02:49:35.294 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T02:49:35.294 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T02:49:35.294 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T02:49:35.295 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T02:49:35.295 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T02:49:35.295 INFO:teuthology.orchestra.run.smithi177.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T02:49:35.295 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T02:49:35.295 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T02:49:35.295 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T02:49:35.295 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T02:49:35.295 INFO:teuthology.orchestra.run.smithi177.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T02:49:35.295 INFO:teuthology.orchestra.run.smithi177.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T02:49:35.295 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T02:49:35.295 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T02:49:35.295 INFO:teuthology.orchestra.run.smithi177.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T02:49:35.296 INFO:teuthology.orchestra.run.smithi177.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T02:49:35.296 INFO:teuthology.orchestra.run.smithi177.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T02:49:35.296 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T02:49:35.296 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T02:49:35.296 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T02:49:35.296 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T02:49:35.296 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T02:49:35.296 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T02:49:35.296 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T02:49:35.296 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T02:49:35.296 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T02:49:35.296 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T02:49:35.296 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T02:49:35.297 INFO:teuthology.orchestra.run.smithi177.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T02:49:35.297 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrading: 2024-09-12T02:49:35.297 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T02:49:35.297 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T02:49:35.297 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T02:49:35.297 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T02:49:35.297 INFO:teuthology.orchestra.run.smithi177.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T02:49:35.297 INFO:teuthology.orchestra.run.smithi177.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T02:49:35.297 INFO:teuthology.orchestra.run.smithi177.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T02:49:35.297 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T02:49:35.297 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-09-12T02:49:35.297 INFO:teuthology.orchestra.run.smithi177.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T02:49:35.298 INFO:teuthology.orchestra.run.smithi177.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T02:49:35.298 INFO:teuthology.orchestra.run.smithi177.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T02:49:35.298 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T02:49:35.298 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T02:49:35.298 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T02:49:35.298 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T02:49:35.298 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T02:49:35.298 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T02:49:35.298 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T02:49:35.298 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T02:49:35.298 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T02:49:35.298 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T02:49:35.299 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T02:49:35.299 INFO:teuthology.orchestra.run.smithi177.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T02:49:35.299 INFO:teuthology.orchestra.run.smithi177.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T02:49:35.299 INFO:teuthology.orchestra.run.smithi177.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T02:49:35.299 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T02:49:35.299 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T02:49:35.299 INFO:teuthology.orchestra.run.smithi177.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T02:49:35.299 INFO:teuthology.orchestra.run.smithi177.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T02:49:35.299 INFO:teuthology.orchestra.run.smithi177.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T02:49:35.299 INFO:teuthology.orchestra.run.smithi177.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T02:49:35.299 INFO:teuthology.orchestra.run.smithi177.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T02:49:35.300 INFO:teuthology.orchestra.run.smithi177.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T02:49:35.300 INFO:teuthology.orchestra.run.smithi177.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T02:49:35.300 INFO:teuthology.orchestra.run.smithi177.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T02:49:35.300 INFO:teuthology.orchestra.run.smithi177.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T02:49:35.300 INFO:teuthology.orchestra.run.smithi177.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T02:49:35.300 INFO:teuthology.orchestra.run.smithi177.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T02:49:35.300 INFO:teuthology.orchestra.run.smithi177.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T02:49:35.300 INFO:teuthology.orchestra.run.smithi177.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T02:49:35.300 INFO:teuthology.orchestra.run.smithi177.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T02:49:35.301 INFO:teuthology.orchestra.run.smithi177.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T02:49:35.301 INFO:teuthology.orchestra.run.smithi177.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T02:49:35.301 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T02:49:35.301 INFO:teuthology.orchestra.run.smithi177.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T02:49:35.301 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T02:49:35.301 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T02:49:35.301 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T02:49:35.301 INFO:teuthology.orchestra.run.smithi177.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T02:49:35.301 INFO:teuthology.orchestra.run.smithi177.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T02:49:35.301 INFO:teuthology.orchestra.run.smithi177.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T02:49:35.301 INFO:teuthology.orchestra.run.smithi177.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T02:49:35.301 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T02:49:35.301 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T02:49:35.302 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T02:49:35.302 INFO:teuthology.orchestra.run.smithi177.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T02:49:35.302 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T02:49:35.302 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T02:49:35.302 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T02:49:35.302 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T02:49:35.302 INFO:teuthology.orchestra.run.smithi177.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T02:49:35.302 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T02:49:35.302 INFO:teuthology.orchestra.run.smithi177.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T02:49:35.302 INFO:teuthology.orchestra.run.smithi177.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T02:49:35.302 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T02:49:35.302 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T02:49:35.303 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T02:49:35.303 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T02:49:35.303 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T02:49:35.303 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T02:49:35.303 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T02:49:35.303 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T02:49:35.303 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T02:49:35.303 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T02:49:35.303 INFO:teuthology.orchestra.run.smithi177.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T02:49:35.303 INFO:teuthology.orchestra.run.smithi177.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T02:49:35.303 INFO:teuthology.orchestra.run.smithi177.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T02:49:35.303 INFO:teuthology.orchestra.run.smithi177.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T02:49:35.304 INFO:teuthology.orchestra.run.smithi177.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T02:49:35.304 INFO:teuthology.orchestra.run.smithi177.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T02:49:35.304 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T02:49:35.304 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T02:49:35.304 INFO:teuthology.orchestra.run.smithi177.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T02:49:35.304 INFO:teuthology.orchestra.run.smithi177.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T02:49:35.304 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T02:49:35.304 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T02:49:35.304 INFO:teuthology.orchestra.run.smithi177.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T02:49:35.304 INFO:teuthology.orchestra.run.smithi177.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T02:49:35.304 INFO:teuthology.orchestra.run.smithi177.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T02:49:35.304 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T02:49:35.305 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T02:49:35.305 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T02:49:35.305 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T02:49:35.305 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T02:49:35.305 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T02:49:35.305 INFO:teuthology.orchestra.run.smithi177.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T02:49:35.305 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T02:49:35.305 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T02:49:35.305 INFO:teuthology.orchestra.run.smithi177.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T02:49:35.305 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T02:49:35.305 INFO:teuthology.orchestra.run.smithi177.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T02:49:35.306 INFO:teuthology.orchestra.run.smithi177.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T02:49:35.306 INFO:teuthology.orchestra.run.smithi177.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T02:49:35.306 INFO:teuthology.orchestra.run.smithi177.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T02:49:35.306 INFO:teuthology.orchestra.run.smithi177.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T02:49:35.306 INFO:teuthology.orchestra.run.smithi177.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T02:49:35.306 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T02:49:35.306 INFO:teuthology.orchestra.run.smithi177.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T02:49:35.306 INFO:teuthology.orchestra.run.smithi177.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T02:49:35.306 INFO:teuthology.orchestra.run.smithi177.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T02:49:35.306 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T02:49:35.306 INFO:teuthology.orchestra.run.smithi177.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T02:49:35.306 INFO:teuthology.orchestra.run.smithi177.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T02:49:35.307 INFO:teuthology.orchestra.run.smithi177.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T02:49:35.307 INFO:teuthology.orchestra.run.smithi177.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T02:49:35.307 INFO:teuthology.orchestra.run.smithi177.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T02:49:35.307 INFO:teuthology.orchestra.run.smithi177.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T02:49:35.307 INFO:teuthology.orchestra.run.smithi177.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T02:49:35.307 INFO:teuthology.orchestra.run.smithi177.stdout:Installing weak dependencies: 2024-09-12T02:49:35.307 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T02:49:35.307 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-12T02:49:35.307 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-09-12T02:49:35.307 INFO:teuthology.orchestra.run.smithi177.stdout:====================================================================================== 2024-09-12T02:49:35.307 INFO:teuthology.orchestra.run.smithi177.stdout:Install 148 Packages 2024-09-12T02:49:35.307 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrade 8 Packages 2024-09-12T02:49:35.307 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-12T02:49:35.308 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 226 M 2024-09-12T02:49:35.308 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-09-12T02:49:36.145 INFO:teuthology.orchestra.run.smithi177.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 51 kB/s | 6.5 kB 00:00 2024-09-12T02:49:36.284 INFO:teuthology.orchestra.run.smithi116.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T02:49:36.285 INFO:teuthology.orchestra.run.smithi116.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T02:49:36.296 INFO:teuthology.orchestra.run.smithi116.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T02:49:36.297 INFO:teuthology.orchestra.run.smithi116.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T02:49:36.298 INFO:teuthology.orchestra.run.smithi116.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T02:49:36.299 INFO:teuthology.orchestra.run.smithi116.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T02:49:36.301 INFO:teuthology.orchestra.run.smithi116.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T02:49:36.303 INFO:teuthology.orchestra.run.smithi116.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T02:49:36.304 INFO:teuthology.orchestra.run.smithi116.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T02:49:36.367 INFO:teuthology.orchestra.run.smithi116.stdout:Dependencies resolved. 2024-09-12T02:49:36.396 INFO:teuthology.orchestra.run.smithi116.stdout:====================================================================================== 2024-09-12T02:49:36.396 INFO:teuthology.orchestra.run.smithi116.stdout: Package Arch Version Repository Size 2024-09-12T02:49:36.397 INFO:teuthology.orchestra.run.smithi116.stdout:====================================================================================== 2024-09-12T02:49:36.397 INFO:teuthology.orchestra.run.smithi116.stdout:Installing: 2024-09-12T02:49:36.397 INFO:teuthology.orchestra.run.smithi116.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T02:49:36.397 INFO:teuthology.orchestra.run.smithi116.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T02:49:36.397 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T02:49:36.397 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T02:49:36.397 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T02:49:36.397 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T02:49:36.397 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T02:49:36.397 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T02:49:36.397 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T02:49:36.398 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T02:49:36.398 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T02:49:36.398 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T02:49:36.398 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T02:49:36.398 INFO:teuthology.orchestra.run.smithi116.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T02:49:36.398 INFO:teuthology.orchestra.run.smithi116.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T02:49:36.398 INFO:teuthology.orchestra.run.smithi116.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T02:49:36.398 INFO:teuthology.orchestra.run.smithi116.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T02:49:36.398 INFO:teuthology.orchestra.run.smithi116.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T02:49:36.398 INFO:teuthology.orchestra.run.smithi116.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T02:49:36.398 INFO:teuthology.orchestra.run.smithi116.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T02:49:36.398 INFO:teuthology.orchestra.run.smithi116.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T02:49:36.398 INFO:teuthology.orchestra.run.smithi116.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T02:49:36.399 INFO:teuthology.orchestra.run.smithi116.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T02:49:36.399 INFO:teuthology.orchestra.run.smithi116.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T02:49:36.399 INFO:teuthology.orchestra.run.smithi116.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T02:49:36.399 INFO:teuthology.orchestra.run.smithi116.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T02:49:36.399 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T02:49:36.399 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T02:49:36.399 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T02:49:36.399 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T02:49:36.399 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T02:49:36.399 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T02:49:36.399 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T02:49:36.399 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T02:49:36.400 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T02:49:36.400 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T02:49:36.400 INFO:teuthology.orchestra.run.smithi116.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T02:49:36.400 INFO:teuthology.orchestra.run.smithi116.stdout:Upgrading: 2024-09-12T02:49:36.400 INFO:teuthology.orchestra.run.smithi116.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T02:49:36.400 INFO:teuthology.orchestra.run.smithi116.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T02:49:36.400 INFO:teuthology.orchestra.run.smithi116.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T02:49:36.400 INFO:teuthology.orchestra.run.smithi116.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T02:49:36.400 INFO:teuthology.orchestra.run.smithi116.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T02:49:36.400 INFO:teuthology.orchestra.run.smithi116.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T02:49:36.400 INFO:teuthology.orchestra.run.smithi116.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T02:49:36.400 INFO:teuthology.orchestra.run.smithi116.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T02:49:36.400 INFO:teuthology.orchestra.run.smithi116.stdout:Installing dependencies: 2024-09-12T02:49:36.401 INFO:teuthology.orchestra.run.smithi116.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T02:49:36.401 INFO:teuthology.orchestra.run.smithi116.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T02:49:36.401 INFO:teuthology.orchestra.run.smithi116.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T02:49:36.401 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T02:49:36.401 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T02:49:36.401 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T02:49:36.401 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T02:49:36.401 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T02:49:36.401 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T02:49:36.401 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T02:49:36.401 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T02:49:36.401 INFO:teuthology.orchestra.run.smithi116.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T02:49:36.401 INFO:teuthology.orchestra.run.smithi116.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T02:49:36.402 INFO:teuthology.orchestra.run.smithi116.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T02:49:36.402 INFO:teuthology.orchestra.run.smithi116.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T02:49:36.402 INFO:teuthology.orchestra.run.smithi116.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T02:49:36.402 INFO:teuthology.orchestra.run.smithi116.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T02:49:36.402 INFO:teuthology.orchestra.run.smithi116.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T02:49:36.402 INFO:teuthology.orchestra.run.smithi116.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T02:49:36.402 INFO:teuthology.orchestra.run.smithi116.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T02:49:36.402 INFO:teuthology.orchestra.run.smithi116.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T02:49:36.402 INFO:teuthology.orchestra.run.smithi116.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T02:49:36.402 INFO:teuthology.orchestra.run.smithi116.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T02:49:36.402 INFO:teuthology.orchestra.run.smithi116.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T02:49:36.402 INFO:teuthology.orchestra.run.smithi116.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T02:49:36.402 INFO:teuthology.orchestra.run.smithi116.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T02:49:36.403 INFO:teuthology.orchestra.run.smithi116.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T02:49:36.403 INFO:teuthology.orchestra.run.smithi116.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T02:49:36.403 INFO:teuthology.orchestra.run.smithi116.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T02:49:36.403 INFO:teuthology.orchestra.run.smithi116.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T02:49:36.403 INFO:teuthology.orchestra.run.smithi116.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T02:49:36.403 INFO:teuthology.orchestra.run.smithi116.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T02:49:36.403 INFO:teuthology.orchestra.run.smithi116.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T02:49:36.403 INFO:teuthology.orchestra.run.smithi116.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T02:49:36.403 INFO:teuthology.orchestra.run.smithi116.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T02:49:36.403 INFO:teuthology.orchestra.run.smithi116.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T02:49:36.403 INFO:teuthology.orchestra.run.smithi116.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T02:49:36.403 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T02:49:36.403 INFO:teuthology.orchestra.run.smithi116.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T02:49:36.404 INFO:teuthology.orchestra.run.smithi116.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T02:49:36.404 INFO:teuthology.orchestra.run.smithi116.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T02:49:36.404 INFO:teuthology.orchestra.run.smithi116.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T02:49:36.404 INFO:teuthology.orchestra.run.smithi116.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T02:49:36.404 INFO:teuthology.orchestra.run.smithi116.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T02:49:36.404 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T02:49:36.404 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T02:49:36.404 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T02:49:36.404 INFO:teuthology.orchestra.run.smithi116.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T02:49:36.404 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T02:49:36.404 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T02:49:36.404 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T02:49:36.405 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T02:49:36.405 INFO:teuthology.orchestra.run.smithi116.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T02:49:36.405 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T02:49:36.405 INFO:teuthology.orchestra.run.smithi116.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T02:49:36.405 INFO:teuthology.orchestra.run.smithi116.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T02:49:36.405 INFO:teuthology.orchestra.run.smithi116.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T02:49:36.405 INFO:teuthology.orchestra.run.smithi116.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T02:49:36.405 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T02:49:36.405 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T02:49:36.405 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T02:49:36.405 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T02:49:36.405 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T02:49:36.406 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T02:49:36.406 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T02:49:36.406 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T02:49:36.406 INFO:teuthology.orchestra.run.smithi116.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T02:49:36.406 INFO:teuthology.orchestra.run.smithi116.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T02:49:36.406 INFO:teuthology.orchestra.run.smithi116.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T02:49:36.406 INFO:teuthology.orchestra.run.smithi116.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T02:49:36.406 INFO:teuthology.orchestra.run.smithi116.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T02:49:36.406 INFO:teuthology.orchestra.run.smithi116.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T02:49:36.406 INFO:teuthology.orchestra.run.smithi116.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T02:49:36.407 INFO:teuthology.orchestra.run.smithi116.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T02:49:36.407 INFO:teuthology.orchestra.run.smithi116.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T02:49:36.407 INFO:teuthology.orchestra.run.smithi116.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T02:49:36.407 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T02:49:36.407 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T02:49:36.407 INFO:teuthology.orchestra.run.smithi116.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T02:49:36.407 INFO:teuthology.orchestra.run.smithi116.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T02:49:36.407 INFO:teuthology.orchestra.run.smithi116.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T02:49:36.407 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T02:49:36.407 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T02:49:36.407 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T02:49:36.407 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T02:49:36.408 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T02:49:36.408 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T02:49:36.408 INFO:teuthology.orchestra.run.smithi116.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T02:49:36.408 INFO:teuthology.orchestra.run.smithi116.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T02:49:36.408 INFO:teuthology.orchestra.run.smithi116.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T02:49:36.408 INFO:teuthology.orchestra.run.smithi116.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T02:49:36.408 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T02:49:36.408 INFO:teuthology.orchestra.run.smithi116.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T02:49:36.408 INFO:teuthology.orchestra.run.smithi116.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T02:49:36.408 INFO:teuthology.orchestra.run.smithi116.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T02:49:36.408 INFO:teuthology.orchestra.run.smithi116.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T02:49:36.408 INFO:teuthology.orchestra.run.smithi116.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T02:49:36.408 INFO:teuthology.orchestra.run.smithi116.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T02:49:36.409 INFO:teuthology.orchestra.run.smithi116.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T02:49:36.409 INFO:teuthology.orchestra.run.smithi116.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T02:49:36.409 INFO:teuthology.orchestra.run.smithi116.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T02:49:36.409 INFO:teuthology.orchestra.run.smithi116.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T02:49:36.409 INFO:teuthology.orchestra.run.smithi116.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T02:49:36.409 INFO:teuthology.orchestra.run.smithi116.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T02:49:36.409 INFO:teuthology.orchestra.run.smithi116.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T02:49:36.409 INFO:teuthology.orchestra.run.smithi116.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T02:49:36.409 INFO:teuthology.orchestra.run.smithi116.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T02:49:36.409 INFO:teuthology.orchestra.run.smithi116.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T02:49:36.409 INFO:teuthology.orchestra.run.smithi116.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T02:49:36.409 INFO:teuthology.orchestra.run.smithi116.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T02:49:36.410 INFO:teuthology.orchestra.run.smithi116.stdout:Installing weak dependencies: 2024-09-12T02:49:36.410 INFO:teuthology.orchestra.run.smithi116.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T02:49:36.410 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:49:36.410 INFO:teuthology.orchestra.run.smithi116.stdout:Transaction Summary 2024-09-12T02:49:36.410 INFO:teuthology.orchestra.run.smithi116.stdout:====================================================================================== 2024-09-12T02:49:36.410 INFO:teuthology.orchestra.run.smithi116.stdout:Install 148 Packages 2024-09-12T02:49:36.410 INFO:teuthology.orchestra.run.smithi116.stdout:Upgrade 8 Packages 2024-09-12T02:49:36.410 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:49:36.411 INFO:teuthology.orchestra.run.smithi116.stdout:Total download size: 226 M 2024-09-12T02:49:36.411 INFO:teuthology.orchestra.run.smithi116.stdout:Downloading Packages: 2024-09-12T02:49:36.537 INFO:teuthology.orchestra.run.smithi177.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 2.2 MB/s | 897 kB 00:00 2024-09-12T02:49:36.604 INFO:teuthology.orchestra.run.smithi177.stdout:(3/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 9.2 MB/s | 5.4 MB 00:00 2024-09-12T02:49:36.629 INFO:teuthology.orchestra.run.smithi177.stdout:(4/156): ceph-immutable-object-cache-19.1.1-298 1.5 MB/s | 145 kB 00:00 2024-09-12T02:49:36.791 INFO:teuthology.orchestra.run.smithi116.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 31 kB/s | 6.5 kB 00:00 2024-09-12T02:49:36.821 INFO:teuthology.orchestra.run.smithi177.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 11 MB/s | 2.4 MB 00:00 2024-09-12T02:49:37.264 INFO:teuthology.orchestra.run.smithi177.stdout:(6/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 11 MB/s | 4.7 MB 00:00 2024-09-12T02:49:37.305 INFO:teuthology.orchestra.run.smithi177.stdout:(7/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 2.3 MB/s | 1.5 MB 00:00 2024-09-12T02:49:37.395 INFO:teuthology.orchestra.run.smithi183.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T02:49:37.396 INFO:teuthology.orchestra.run.smithi183.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T02:49:37.409 INFO:teuthology.orchestra.run.smithi183.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T02:49:37.410 INFO:teuthology.orchestra.run.smithi183.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T02:49:37.411 INFO:teuthology.orchestra.run.smithi183.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T02:49:37.412 INFO:teuthology.orchestra.run.smithi183.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T02:49:37.414 INFO:teuthology.orchestra.run.smithi183.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T02:49:37.417 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T02:49:37.417 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T02:49:37.480 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-09-12T02:49:37.506 INFO:teuthology.orchestra.run.smithi183.stdout:====================================================================================== 2024-09-12T02:49:37.506 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2024-09-12T02:49:37.506 INFO:teuthology.orchestra.run.smithi183.stdout:====================================================================================== 2024-09-12T02:49:37.506 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-09-12T02:49:37.506 INFO:teuthology.orchestra.run.smithi183.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T02:49:37.506 INFO:teuthology.orchestra.run.smithi183.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T02:49:37.506 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T02:49:37.506 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T02:49:37.506 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T02:49:37.507 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T02:49:37.507 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T02:49:37.507 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T02:49:37.507 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T02:49:37.507 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T02:49:37.507 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T02:49:37.507 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T02:49:37.507 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T02:49:37.507 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T02:49:37.507 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T02:49:37.507 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T02:49:37.507 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T02:49:37.507 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T02:49:37.507 INFO:teuthology.orchestra.run.smithi183.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T02:49:37.508 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T02:49:37.508 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T02:49:37.508 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T02:49:37.508 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T02:49:37.508 INFO:teuthology.orchestra.run.smithi183.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T02:49:37.508 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T02:49:37.508 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T02:49:37.508 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T02:49:37.508 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T02:49:37.508 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T02:49:37.508 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T02:49:37.508 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T02:49:37.508 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T02:49:37.509 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T02:49:37.509 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T02:49:37.509 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T02:49:37.509 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T02:49:37.509 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T02:49:37.509 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrading: 2024-09-12T02:49:37.509 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T02:49:37.509 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T02:49:37.509 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T02:49:37.509 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T02:49:37.509 INFO:teuthology.orchestra.run.smithi183.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T02:49:37.509 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T02:49:37.509 INFO:teuthology.orchestra.run.smithi183.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T02:49:37.510 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T02:49:37.510 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2024-09-12T02:49:37.510 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T02:49:37.510 INFO:teuthology.orchestra.run.smithi183.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T02:49:37.510 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T02:49:37.510 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T02:49:37.510 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T02:49:37.510 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T02:49:37.510 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T02:49:37.510 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T02:49:37.510 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T02:49:37.510 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T02:49:37.510 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T02:49:37.510 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T02:49:37.511 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T02:49:37.511 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T02:49:37.511 INFO:teuthology.orchestra.run.smithi183.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T02:49:37.511 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T02:49:37.511 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T02:49:37.511 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T02:49:37.511 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T02:49:37.511 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T02:49:37.511 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T02:49:37.511 INFO:teuthology.orchestra.run.smithi183.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T02:49:37.511 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T02:49:37.511 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T02:49:37.511 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T02:49:37.511 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T02:49:37.512 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T02:49:37.512 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T02:49:37.512 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T02:49:37.512 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T02:49:37.512 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T02:49:37.512 INFO:teuthology.orchestra.run.smithi183.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T02:49:37.512 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T02:49:37.512 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T02:49:37.512 INFO:teuthology.orchestra.run.smithi183.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T02:49:37.512 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T02:49:37.512 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T02:49:37.512 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T02:49:37.512 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T02:49:37.513 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T02:49:37.513 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T02:49:37.513 INFO:teuthology.orchestra.run.smithi183.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T02:49:37.513 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T02:49:37.513 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T02:49:37.513 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T02:49:37.513 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T02:49:37.513 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T02:49:37.513 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T02:49:37.513 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T02:49:37.513 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T02:49:37.513 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T02:49:37.513 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T02:49:37.513 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T02:49:37.514 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T02:49:37.514 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T02:49:37.514 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T02:49:37.514 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T02:49:37.514 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T02:49:37.514 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T02:49:37.514 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T02:49:37.514 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T02:49:37.514 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T02:49:37.514 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T02:49:37.514 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T02:49:37.514 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T02:49:37.514 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T02:49:37.515 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T02:49:37.515 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T02:49:37.515 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T02:49:37.515 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T02:49:37.515 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T02:49:37.515 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T02:49:37.515 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T02:49:37.515 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T02:49:37.515 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T02:49:37.515 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T02:49:37.515 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T02:49:37.515 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T02:49:37.515 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T02:49:37.515 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T02:49:37.516 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T02:49:37.516 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T02:49:37.516 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T02:49:37.516 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T02:49:37.516 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T02:49:37.516 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T02:49:37.516 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T02:49:37.516 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T02:49:37.516 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T02:49:37.516 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T02:49:37.516 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T02:49:37.516 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T02:49:37.516 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T02:49:37.516 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T02:49:37.517 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T02:49:37.517 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T02:49:37.517 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T02:49:37.517 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T02:49:37.517 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T02:49:37.517 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T02:49:37.517 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T02:49:37.517 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T02:49:37.517 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T02:49:37.517 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T02:49:37.517 INFO:teuthology.orchestra.run.smithi183.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T02:49:37.517 INFO:teuthology.orchestra.run.smithi183.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T02:49:37.517 INFO:teuthology.orchestra.run.smithi183.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T02:49:37.517 INFO:teuthology.orchestra.run.smithi183.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T02:49:37.518 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T02:49:37.518 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T02:49:37.518 INFO:teuthology.orchestra.run.smithi183.stdout:Installing weak dependencies: 2024-09-12T02:49:37.518 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T02:49:37.518 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T02:49:37.518 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-09-12T02:49:37.518 INFO:teuthology.orchestra.run.smithi183.stdout:====================================================================================== 2024-09-12T02:49:37.518 INFO:teuthology.orchestra.run.smithi183.stdout:Install 148 Packages 2024-09-12T02:49:37.518 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrade 8 Packages 2024-09-12T02:49:37.518 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T02:49:37.519 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 226 M 2024-09-12T02:49:37.519 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-09-12T02:49:38.125 INFO:teuthology.orchestra.run.smithi183.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 23 kB/s | 6.5 kB 00:00 2024-09-12T02:49:38.150 INFO:teuthology.orchestra.run.smithi116.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 661 kB/s | 897 kB 00:01 2024-09-12T02:49:38.333 INFO:teuthology.orchestra.run.smithi116.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 792 kB/s | 145 kB 00:00 2024-09-12T02:49:38.397 INFO:teuthology.orchestra.run.smithi177.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 8.9 MB/s | 21 MB 00:02 2024-09-12T02:49:38.600 INFO:teuthology.orchestra.run.smithi177.stdout:(9/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86_ 13 MB/s | 17 MB 00:01 2024-09-12T02:49:38.614 INFO:teuthology.orchestra.run.smithi177.stdout:(10/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9 116 kB/s | 25 kB 00:00 2024-09-12T02:49:38.723 INFO:teuthology.orchestra.run.smithi177.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 2.1 MB/s | 228 kB 00:00 2024-09-12T02:49:38.815 INFO:teuthology.orchestra.run.smithi177.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 294 kB/s | 27 kB 00:00 2024-09-12T02:49:38.850 INFO:teuthology.orchestra.run.smithi183.stdout:(2/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 5.4 MB/s | 5.4 MB 00:01 2024-09-12T02:49:38.881 INFO:teuthology.orchestra.run.smithi177.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 476 kB/s | 32 kB 00:00 2024-09-12T02:49:38.917 INFO:teuthology.orchestra.run.smithi183.stdout:(3/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.1 MB/s | 897 kB 00:00 2024-09-12T02:49:38.942 INFO:teuthology.orchestra.run.smithi183.stdout:(4/156): ceph-immutable-object-cache-19.1.1-298 1.5 MB/s | 145 kB 00:00 2024-09-12T02:49:39.074 INFO:teuthology.orchestra.run.smithi177.stdout:(14/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 6.0 MB/s | 11 MB 00:01 2024-09-12T02:49:39.109 INFO:teuthology.orchestra.run.smithi183.stdout:(5/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 9.1 MB/s | 1.5 MB 00:00 2024-09-12T02:49:39.116 INFO:teuthology.orchestra.run.smithi177.stdout:(15/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 3.1 MB/s | 752 kB 00:00 2024-09-12T02:49:39.150 INFO:teuthology.orchestra.run.smithi177.stdout:(16/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.1 MB/s | 163 kB 00:00 2024-09-12T02:49:39.217 INFO:teuthology.orchestra.run.smithi177.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.2 MB/s | 127 kB 00:00 2024-09-12T02:49:39.325 INFO:teuthology.orchestra.run.smithi177.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 2.8 MB/s | 506 kB 00:00 2024-09-12T02:49:39.425 INFO:teuthology.orchestra.run.smithi177.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 451 kB/s | 45 kB 00:00 2024-09-12T02:49:39.525 INFO:teuthology.orchestra.run.smithi177.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.3 MB/s | 136 kB 00:00 2024-09-12T02:49:39.617 INFO:teuthology.orchestra.run.smithi177.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 1.7 MB/s | 162 kB 00:00 2024-09-12T02:49:39.626 INFO:teuthology.orchestra.run.smithi183.stdout:(6/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 9.1 MB/s | 4.7 MB 00:00 2024-09-12T02:49:39.718 INFO:teuthology.orchestra.run.smithi177.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 3.1 MB/s | 321 kB 00:00 2024-09-12T02:49:40.151 INFO:teuthology.orchestra.run.smithi183.stdout:(7/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 1.9 MB/s | 2.4 MB 00:01 2024-09-12T02:49:40.260 INFO:teuthology.orchestra.run.smithi177.stdout:(23/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 5.1 MB/s | 5.3 MB 00:01 2024-09-12T02:49:40.368 INFO:teuthology.orchestra.run.smithi177.stdout:(24/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 920 kB/s | 100 kB 00:00 2024-09-12T02:49:40.477 INFO:teuthology.orchestra.run.smithi177.stdout:(25/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 789 kB/s | 85 kB 00:00 2024-09-12T02:49:40.602 INFO:teuthology.orchestra.run.smithi177.stdout:(26/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 341 kB/s | 301 kB 00:00 2024-09-12T02:49:40.703 INFO:teuthology.orchestra.run.smithi177.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 1.7 MB/s | 171 kB 00:00 2024-09-12T02:49:40.811 INFO:teuthology.orchestra.run.smithi177.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 294 kB/s | 32 kB 00:00 2024-09-12T02:49:41.050 INFO:teuthology.orchestra.run.smithi116.stdout:(4/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 899 kB/s | 2.4 MB 00:02 2024-09-12T02:49:41.251 INFO:teuthology.orchestra.run.smithi183.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 6.2 MB/s | 21 MB 00:03 2024-09-12T02:49:41.325 INFO:teuthology.orchestra.run.smithi116.stdout:(5/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 1.1 MB/s | 5.4 MB 00:04 2024-09-12T02:49:41.343 INFO:teuthology.orchestra.run.smithi183.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 275 kB/s | 25 kB 00:00 2024-09-12T02:49:41.811 INFO:teuthology.orchestra.run.smithi177.stdout:(29/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 2.3 MB/s | 3.1 MB 00:01 2024-09-12T02:49:41.935 INFO:teuthology.orchestra.run.smithi183.stdout:(10/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 7.3 MB/s | 17 MB 00:02 2024-09-12T02:49:42.027 INFO:teuthology.orchestra.run.smithi183.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 2.4 MB/s | 228 kB 00:00 2024-09-12T02:49:42.036 INFO:teuthology.orchestra.run.smithi177.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 119 kB/s | 146 kB 00:01 2024-09-12T02:49:42.102 INFO:teuthology.orchestra.run.smithi183.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 361 kB/s | 27 kB 00:00 2024-09-12T02:49:42.178 INFO:teuthology.orchestra.run.smithi183.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 422 kB/s | 32 kB 00:00 2024-09-12T02:49:42.354 INFO:teuthology.orchestra.run.smithi183.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 4.2 MB/s | 752 kB 00:00 2024-09-12T02:49:42.487 INFO:teuthology.orchestra.run.smithi183.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 1.3 MB/s | 163 kB 00:00 2024-09-12T02:49:42.579 INFO:teuthology.orchestra.run.smithi183.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.2 MB/s | 127 kB 00:00 2024-09-12T02:49:43.142 INFO:teuthology.orchestra.run.smithi116.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 745 kB/s | 1.5 MB 00:02 2024-09-12T02:49:43.179 INFO:teuthology.orchestra.run.smithi183.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 844 kB/s | 506 kB 00:00 2024-09-12T02:49:43.720 INFO:teuthology.orchestra.run.smithi177.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 1.9 MB/s | 3.6 MB 00:01 2024-09-12T02:49:43.904 INFO:teuthology.orchestra.run.smithi177.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 1.3 MB/s | 249 kB 00:00 2024-09-12T02:49:44.012 INFO:teuthology.orchestra.run.smithi177.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 456 kB/s | 49 kB 00:00 2024-09-12T02:49:44.112 INFO:teuthology.orchestra.run.smithi177.stdout:(34/156): ceph-prometheus-alerts-19.1.1-298.gb5 164 kB/s | 16 kB 00:00 2024-09-12T02:49:44.296 INFO:teuthology.orchestra.run.smithi177.stdout:(35/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 1.5 MB/s | 285 kB 00:00 2024-09-12T02:49:44.412 INFO:teuthology.orchestra.run.smithi183.stdout:(18/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 4.3 MB/s | 5.3 MB 00:01 2024-09-12T02:49:44.512 INFO:teuthology.orchestra.run.smithi183.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 452 kB/s | 45 kB 00:00 2024-09-12T02:49:44.613 INFO:teuthology.orchestra.run.smithi183.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.3 MB/s | 136 kB 00:00 2024-09-12T02:49:44.705 INFO:teuthology.orchestra.run.smithi177.stdout:(36/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 1.8 MB/s | 765 kB 00:00 2024-09-12T02:49:44.721 INFO:teuthology.orchestra.run.smithi183.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 1.5 MB/s | 162 kB 00:00 2024-09-12T02:49:44.796 INFO:teuthology.orchestra.run.smithi177.stdout:(37/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 200 kB/s | 18 kB 00:00 2024-09-12T02:49:44.822 INFO:teuthology.orchestra.run.smithi183.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 3.1 MB/s | 321 kB 00:00 2024-09-12T02:49:44.897 INFO:teuthology.orchestra.run.smithi177.stdout:(38/156): fuse-2.9.9-16.el9.x86_64.rpm 799 kB/s | 80 kB 00:00 2024-09-12T02:49:44.923 INFO:teuthology.orchestra.run.smithi183.stdout:(23/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 2.9 MB/s | 301 kB 00:00 2024-09-12T02:49:45.006 INFO:teuthology.orchestra.run.smithi177.stdout:(39/156): libgfortran-11.5.0-2.el9.x86_64.rpm 7.2 MB/s | 801 kB 00:00 2024-09-12T02:49:45.007 INFO:teuthology.orchestra.run.smithi183.stdout:(24/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.2 MB/s | 100 kB 00:00 2024-09-12T02:49:45.047 INFO:teuthology.orchestra.run.smithi177.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-12T02:49:45.059 INFO:teuthology.orchestra.run.smithi116.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 1.3 MB/s | 4.7 MB 00:03 2024-09-12T02:49:45.074 INFO:teuthology.orchestra.run.smithi183.stdout:(25/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.3 MB/s | 85 kB 00:00 2024-09-12T02:49:45.089 INFO:teuthology.orchestra.run.smithi177.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 6.0 MB/s | 253 kB 00:00 2024-09-12T02:49:45.157 INFO:teuthology.orchestra.run.smithi177.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-09-12T02:49:45.199 INFO:teuthology.orchestra.run.smithi177.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-12T02:49:45.266 INFO:teuthology.orchestra.run.smithi177.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-12T02:49:45.308 INFO:teuthology.orchestra.run.smithi177.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-09-12T02:49:45.350 INFO:teuthology.orchestra.run.smithi177.stdout:(46/156): python3-urllib3-1.26.5-6.el9.noarch.r 5.0 MB/s | 215 kB 00:00 2024-09-12T02:49:45.641 INFO:teuthology.orchestra.run.smithi177.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-09-12T02:49:45.767 INFO:teuthology.orchestra.run.smithi177.stdout:(48/156): bison-3.7.4-5.el9.x86_64.rpm 7.4 MB/s | 943 kB 00:00 2024-09-12T02:49:45.817 INFO:teuthology.orchestra.run.smithi177.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-09-12T02:49:45.867 INFO:teuthology.orchestra.run.smithi177.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 433 kB/s | 22 kB 00:00 2024-09-12T02:49:45.924 INFO:teuthology.orchestra.run.smithi183.stdout:(26/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 1.8 MB/s | 11 MB 00:05 2024-09-12T02:49:45.976 INFO:teuthology.orchestra.run.smithi177.stdout:(51/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-09-12T02:49:45.991 INFO:teuthology.orchestra.run.smithi183.stdout:(27/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 3.4 MB/s | 3.1 MB 00:00 2024-09-12T02:49:46.026 INFO:teuthology.orchestra.run.smithi177.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 657 kB/s | 33 kB 00:00 2024-09-12T02:49:46.075 INFO:teuthology.orchestra.run.smithi183.stdout:(28/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 1.1 MB/s | 171 kB 00:00 2024-09-12T02:49:46.100 INFO:teuthology.orchestra.run.smithi183.stdout:(29/156): ceph-grafana-dashboards-19.1.1-298.gb 294 kB/s | 32 kB 00:00 2024-09-12T02:49:46.150 INFO:teuthology.orchestra.run.smithi183.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.9 MB/s | 146 kB 00:00 2024-09-12T02:49:46.718 INFO:teuthology.orchestra.run.smithi177.stdout:(53/156): ceph-mgr-diskprediction-local-19.1.1- 1.6 MB/s | 7.4 MB 00:04 2024-09-12T02:49:46.835 INFO:teuthology.orchestra.run.smithi177.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 156 kB/s | 18 kB 00:00 2024-09-12T02:49:46.885 INFO:teuthology.orchestra.run.smithi177.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-09-12T02:49:47.036 INFO:teuthology.orchestra.run.smithi177.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-09-12T02:49:47.086 INFO:teuthology.orchestra.run.smithi177.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 906 kB/s | 45 kB 00:00 2024-09-12T02:49:47.142 INFO:teuthology.orchestra.run.smithi183.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 3.4 MB/s | 3.6 MB 00:01 2024-09-12T02:49:47.186 INFO:teuthology.orchestra.run.smithi177.stdout:(58/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.6 MB/s | 3.0 MB 00:01 2024-09-12T02:49:47.243 INFO:teuthology.orchestra.run.smithi183.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 2.4 MB/s | 249 kB 00:00 2024-09-12T02:49:47.270 INFO:teuthology.orchestra.run.smithi177.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.5 MB/s | 662 kB 00:00 2024-09-12T02:49:47.320 INFO:teuthology.orchestra.run.smithi177.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 492 kB/s | 24 kB 00:00 2024-09-12T02:49:47.334 INFO:teuthology.orchestra.run.smithi183.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 538 kB/s | 49 kB 00:00 2024-09-12T02:49:47.370 INFO:teuthology.orchestra.run.smithi177.stdout:(61/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 950 kB/s | 47 kB 00:00 2024-09-12T02:49:47.418 INFO:teuthology.orchestra.run.smithi183.stdout:(34/156): ceph-prometheus-alerts-19.1.1-298.gb5 197 kB/s | 16 kB 00:00 2024-09-12T02:49:47.429 INFO:teuthology.orchestra.run.smithi177.stdout:(62/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-09-12T02:49:47.488 INFO:teuthology.orchestra.run.smithi177.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-09-12T02:49:47.555 INFO:teuthology.orchestra.run.smithi177.stdout:(64/156): libtool-2.4.6-46.el9.x86_64.rpm 1.5 MB/s | 581 kB 00:00 2024-09-12T02:49:47.568 INFO:teuthology.orchestra.run.smithi183.stdout:(35/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 1.9 MB/s | 285 kB 00:00 2024-09-12T02:49:47.579 INFO:teuthology.orchestra.run.smithi177.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 419 kB/s | 38 kB 00:00 2024-09-12T02:49:47.794 INFO:teuthology.orchestra.run.smithi183.stdout:(36/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 3.3 MB/s | 765 kB 00:00 2024-09-12T02:49:47.877 INFO:teuthology.orchestra.run.smithi183.stdout:(37/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 218 kB/s | 18 kB 00:00 2024-09-12T02:49:48.019 INFO:teuthology.orchestra.run.smithi183.stdout:(38/156): fuse-2.9.9-16.el9.x86_64.rpm 564 kB/s | 80 kB 00:00 2024-09-12T02:49:48.178 INFO:teuthology.orchestra.run.smithi183.stdout:(39/156): libgfortran-11.5.0-2.el9.x86_64.rpm 4.9 MB/s | 801 kB 00:00 2024-09-12T02:49:48.237 INFO:teuthology.orchestra.run.smithi183.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-12T02:49:48.295 INFO:teuthology.orchestra.run.smithi183.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-09-12T02:49:48.596 INFO:teuthology.orchestra.run.smithi183.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 4.1 MB/s | 1.2 MB 00:00 2024-09-12T02:49:48.679 INFO:teuthology.orchestra.run.smithi183.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-09-12T02:49:48.788 INFO:teuthology.orchestra.run.smithi183.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-09-12T02:49:48.872 INFO:teuthology.orchestra.run.smithi183.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-09-12T02:49:49.014 INFO:teuthology.orchestra.run.smithi183.stdout:(46/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.5 MB/s | 215 kB 00:00 2024-09-12T02:49:49.097 INFO:teuthology.orchestra.run.smithi183.stdout:(47/156): ceph-mgr-diskprediction-local-19.1.1- 2.5 MB/s | 7.4 MB 00:02 2024-09-12T02:49:49.373 INFO:teuthology.orchestra.run.smithi183.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 1.9 MB/s | 689 kB 00:00 2024-09-12T02:49:49.406 INFO:teuthology.orchestra.run.smithi183.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 3.0 MB/s | 943 kB 00:00 2024-09-12T02:49:49.457 INFO:teuthology.orchestra.run.smithi183.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.2 MB/s | 107 kB 00:00 2024-09-12T02:49:49.480 INFO:teuthology.orchestra.run.smithi177.stdout:(66/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.3 MB/s | 4.4 MB 00:01 2024-09-12T02:49:49.482 INFO:teuthology.orchestra.run.smithi183.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 288 kB/s | 22 kB 00:00 2024-09-12T02:49:49.899 INFO:teuthology.orchestra.run.smithi183.stdout:(52/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 5.7 MB/s | 49 MB 00:08 2024-09-12T02:49:49.924 INFO:teuthology.orchestra.run.smithi183.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 74 kB/s | 33 kB 00:00 2024-09-12T02:49:49.950 INFO:teuthology.orchestra.run.smithi183.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 637 kB/s | 313 kB 00:00 2024-09-12T02:49:49.975 INFO:teuthology.orchestra.run.smithi183.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-09-12T02:49:49.980 INFO:teuthology.orchestra.run.smithi177.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 2.0 MB/s | 4.9 MB 00:02 2024-09-12T02:49:50.000 INFO:teuthology.orchestra.run.smithi183.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-09-12T02:49:50.033 INFO:teuthology.orchestra.run.smithi183.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-12T02:49:50.059 INFO:teuthology.orchestra.run.smithi183.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 773 kB/s | 45 kB 00:00 2024-09-12T02:49:50.159 INFO:teuthology.orchestra.run.smithi183.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-09-12T02:49:50.217 INFO:teuthology.orchestra.run.smithi183.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 420 kB/s | 24 kB 00:00 2024-09-12T02:49:50.268 INFO:teuthology.orchestra.run.smithi183.stdout:(61/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 949 kB/s | 47 kB 00:00 2024-09-12T02:49:50.314 INFO:teuthology.orchestra.run.smithi177.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 1.9 MB/s | 1.6 MB 00:00 2024-09-12T02:49:50.327 INFO:teuthology.orchestra.run.smithi183.stdout:(62/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-09-12T02:49:50.385 INFO:teuthology.orchestra.run.smithi183.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-09-12T02:49:50.422 INFO:teuthology.orchestra.run.smithi177.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-12T02:49:50.427 INFO:teuthology.orchestra.run.smithi183.stdout:(64/156): libtool-2.4.6-46.el9.x86_64.rpm 1.5 MB/s | 581 kB 00:00 2024-09-12T02:49:50.444 INFO:teuthology.orchestra.run.smithi183.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 654 kB/s | 38 kB 00:00 2024-09-12T02:49:50.498 INFO:teuthology.orchestra.run.smithi177.stdout:(70/156): postgresql-contrib-13.16-1.el9.x86_64 1.7 MB/s | 910 kB 00:00 2024-09-12T02:49:50.936 INFO:teuthology.orchestra.run.smithi183.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.9 MB/s | 3.0 MB 00:01 2024-09-12T02:49:51.015 INFO:teuthology.orchestra.run.smithi177.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 2.0 MB/s | 1.0 MB 00:00 2024-09-12T02:49:51.098 INFO:teuthology.orchestra.run.smithi177.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-09-12T02:49:51.717 INFO:teuthology.orchestra.run.smithi116.stdout:(8/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9. 1.6 MB/s | 11 MB 00:06 2024-09-12T02:49:51.728 INFO:teuthology.orchestra.run.smithi183.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 2.0 MB/s | 1.6 MB 00:00 2024-09-12T02:49:51.942 INFO:teuthology.orchestra.run.smithi116.stdout:(9/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 1.4 MB/s | 21 MB 00:15 2024-09-12T02:49:51.967 INFO:teuthology.orchestra.run.smithi116.stdout:(10/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9 101 kB/s | 25 kB 00:00 2024-09-12T02:49:52.142 INFO:teuthology.orchestra.run.smithi116.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 1.3 MB/s | 228 kB 00:00 2024-09-12T02:49:52.187 INFO:teuthology.orchestra.run.smithi183.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 1.9 MB/s | 910 kB 00:00 2024-09-12T02:49:52.209 INFO:teuthology.orchestra.run.smithi116.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 405 kB/s | 27 kB 00:00 2024-09-12T02:49:52.293 INFO:teuthology.orchestra.run.smithi116.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 380 kB/s | 32 kB 00:00 2024-09-12T02:49:52.295 INFO:teuthology.orchestra.run.smithi183.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-12T02:49:52.462 INFO:teuthology.orchestra.run.smithi183.stdout:(70/156): openblas-openmp-0.3.26-2.el9.x86_64.r 2.4 MB/s | 4.9 MB 00:02 2024-09-12T02:49:52.710 INFO:teuthology.orchestra.run.smithi116.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 1.8 MB/s | 752 kB 00:00 2024-09-12T02:49:52.712 INFO:teuthology.orchestra.run.smithi183.stdout:(71/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.0 MB/s | 4.4 MB 00:02 2024-09-12T02:49:52.797 INFO:teuthology.orchestra.run.smithi183.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-09-12T02:49:52.844 INFO:teuthology.orchestra.run.smithi116.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 1.2 MB/s | 163 kB 00:00 2024-09-12T02:49:52.977 INFO:teuthology.orchestra.run.smithi116.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 950 kB/s | 127 kB 00:00 2024-09-12T02:49:52.979 INFO:teuthology.orchestra.run.smithi183.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 2.0 MB/s | 1.0 MB 00:00 2024-09-12T02:49:53.030 INFO:teuthology.orchestra.run.smithi183.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-12T02:49:53.180 INFO:teuthology.orchestra.run.smithi183.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.6 MB/s | 245 kB 00:00 2024-09-12T02:49:53.244 INFO:teuthology.orchestra.run.smithi116.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 1.9 MB/s | 506 kB 00:00 2024-09-12T02:49:53.265 INFO:teuthology.orchestra.run.smithi177.stdout:(73/156): postgresql-server-13.16-1.el9.x86_64. 2.0 MB/s | 5.8 MB 00:02 2024-09-12T02:49:53.315 INFO:teuthology.orchestra.run.smithi177.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-12T02:49:53.332 INFO:teuthology.orchestra.run.smithi183.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-09-12T02:49:53.381 INFO:teuthology.orchestra.run.smithi183.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-09-12T02:49:53.431 INFO:teuthology.orchestra.run.smithi183.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-12T02:49:53.457 INFO:teuthology.orchestra.run.smithi177.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.7 MB/s | 245 kB 00:00 2024-09-12T02:49:53.548 INFO:teuthology.orchestra.run.smithi183.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-12T02:49:53.598 INFO:teuthology.orchestra.run.smithi183.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-09-12T02:49:53.607 INFO:teuthology.orchestra.run.smithi177.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-09-12T02:49:53.658 INFO:teuthology.orchestra.run.smithi177.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-09-12T02:49:53.708 INFO:teuthology.orchestra.run.smithi177.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 384 kB/s | 19 kB 00:00 2024-09-12T02:49:53.825 INFO:teuthology.orchestra.run.smithi177.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-12T02:49:53.875 INFO:teuthology.orchestra.run.smithi177.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 698 kB/s | 35 kB 00:00 2024-09-12T02:49:54.067 INFO:teuthology.orchestra.run.smithi177.stdout:(81/156): python3-babel-2.9.1-2.el9.noarch.rpm 2.0 MB/s | 6.0 MB 00:02 2024-09-12T02:49:54.217 INFO:teuthology.orchestra.run.smithi177.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-09-12T02:49:54.368 INFO:teuthology.orchestra.run.smithi177.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 222 kB 00:00 2024-09-12T02:49:54.451 INFO:teuthology.orchestra.run.smithi177.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 928 kB/s | 77 kB 00:00 2024-09-12T02:49:54.502 INFO:teuthology.orchestra.run.smithi177.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-09-12T02:49:54.677 INFO:teuthology.orchestra.run.smithi177.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 1.5 MB/s | 269 kB 00:00 2024-09-12T02:49:54.794 INFO:teuthology.orchestra.run.smithi177.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-12T02:49:54.970 INFO:teuthology.orchestra.run.smithi177.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.6 MB/s | 279 kB 00:00 2024-09-12T02:49:55.027 INFO:teuthology.orchestra.run.smithi177.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 876 kB/s | 51 kB 00:00 2024-09-12T02:49:55.078 INFO:teuthology.orchestra.run.smithi177.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-12T02:49:55.115 INFO:teuthology.orchestra.run.smithi183.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 2.0 MB/s | 5.8 MB 00:02 2024-09-12T02:49:55.144 INFO:teuthology.orchestra.run.smithi116.stdout:(18/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 2.8 MB/s | 5.3 MB 00:01 2024-09-12T02:49:55.174 INFO:teuthology.orchestra.run.smithi183.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-12T02:49:55.315 INFO:teuthology.orchestra.run.smithi183.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.5 MB/s | 222 kB 00:00 2024-09-12T02:49:55.361 INFO:teuthology.orchestra.run.smithi116.stdout:(19/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 1.4 MB/s | 17 MB 00:12 2024-09-12T02:49:55.386 INFO:teuthology.orchestra.run.smithi116.stdout:(20/156): python3-ceph-argparse-19.1.1-298.gb53 187 kB/s | 45 kB 00:00 2024-09-12T02:49:55.424 INFO:teuthology.orchestra.run.smithi183.stdout:(84/156): python3-babel-2.9.1-2.el9.noarch.rpm 2.3 MB/s | 6.0 MB 00:02 2024-09-12T02:49:55.428 INFO:teuthology.orchestra.run.smithi116.stdout:(21/156): python3-ceph-common-19.1.1-298.gb53b6 2.0 MB/s | 136 kB 00:00 2024-09-12T02:49:55.449 INFO:teuthology.orchestra.run.smithi183.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 577 kB/s | 77 kB 00:00 2024-09-12T02:49:55.475 INFO:teuthology.orchestra.run.smithi183.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-09-12T02:49:55.504 INFO:teuthology.orchestra.run.smithi116.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 1.4 MB/s | 162 kB 00:00 2024-09-12T02:49:55.585 INFO:teuthology.orchestra.run.smithi183.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-12T02:49:55.604 INFO:teuthology.orchestra.run.smithi116.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 1.8 MB/s | 321 kB 00:00 2024-09-12T02:49:55.634 INFO:teuthology.orchestra.run.smithi183.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 1.4 MB/s | 269 kB 00:00 2024-09-12T02:49:55.688 INFO:teuthology.orchestra.run.smithi116.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 1.6 MB/s | 301 kB 00:00 2024-09-12T02:49:55.709 INFO:teuthology.orchestra.run.smithi183.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 682 kB/s | 51 kB 00:00 2024-09-12T02:49:55.713 INFO:teuthology.orchestra.run.smithi116.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 918 kB/s | 100 kB 00:00 2024-09-12T02:49:55.751 INFO:teuthology.orchestra.run.smithi183.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-09-12T02:49:55.754 INFO:teuthology.orchestra.run.smithi116.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.2 MB/s | 85 kB 00:00 2024-09-12T02:49:55.809 INFO:teuthology.orchestra.run.smithi183.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.2 MB/s | 279 kB 00:00 2024-09-12T02:49:55.860 INFO:teuthology.orchestra.run.smithi183.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 828 kB/s | 42 kB 00:00 2024-09-12T02:49:55.872 INFO:teuthology.orchestra.run.smithi116.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 1.4 MB/s | 171 kB 00:00 2024-09-12T02:49:55.910 INFO:teuthology.orchestra.run.smithi183.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 885 kB/s | 44 kB 00:00 2024-09-12T02:49:55.939 INFO:teuthology.orchestra.run.smithi116.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 478 kB/s | 32 kB 00:00 2024-09-12T02:49:56.014 INFO:teuthology.orchestra.run.smithi116.stdout:(29/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.9 MB/s | 146 kB 00:00 2024-09-12T02:49:56.052 INFO:teuthology.orchestra.run.smithi183.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.5 MB/s | 221 kB 00:00 2024-09-12T02:49:56.111 INFO:teuthology.orchestra.run.smithi183.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 618 kB/s | 36 kB 00:00 2024-09-12T02:49:56.161 INFO:teuthology.orchestra.run.smithi183.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 950 kB/s | 47 kB 00:00 2024-09-12T02:49:56.203 INFO:teuthology.orchestra.run.smithi177.stdout:(91/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.2 MB/s | 5.1 MB 00:02 2024-09-12T02:49:56.254 INFO:teuthology.orchestra.run.smithi177.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 836 kB/s | 42 kB 00:00 2024-09-12T02:49:56.304 INFO:teuthology.orchestra.run.smithi177.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 884 kB/s | 44 kB 00:00 2024-09-12T02:49:56.319 INFO:teuthology.orchestra.run.smithi183.stdout:(97/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.9 MB/s | 5.1 MB 00:02 2024-09-12T02:49:56.370 INFO:teuthology.orchestra.run.smithi183.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 1.4 MB/s | 304 kB 00:00 2024-09-12T02:49:56.395 INFO:teuthology.orchestra.run.smithi183.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 770 kB/s | 58 kB 00:00 2024-09-12T02:49:56.454 INFO:teuthology.orchestra.run.smithi183.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.1 MB/s | 96 kB 00:00 2024-09-12T02:49:56.462 INFO:teuthology.orchestra.run.smithi177.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.4 MB/s | 221 kB 00:00 2024-09-12T02:49:56.487 INFO:teuthology.orchestra.run.smithi183.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 692 kB/s | 64 kB 00:00 2024-09-12T02:49:56.513 INFO:teuthology.orchestra.run.smithi177.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 720 kB/s | 36 kB 00:00 2024-09-12T02:49:56.513 INFO:teuthology.orchestra.run.smithi183.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 767 kB/s | 45 kB 00:00 2024-09-12T02:49:56.623 INFO:teuthology.orchestra.run.smithi116.stdout:(30/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 3.4 MB/s | 3.1 MB 00:00 2024-09-12T02:49:56.796 INFO:teuthology.orchestra.run.smithi183.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 72 kB/s | 22 kB 00:00 2024-09-12T02:49:56.846 INFO:teuthology.orchestra.run.smithi183.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-12T02:49:56.872 INFO:teuthology.orchestra.run.smithi183.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-09-12T02:49:56.897 INFO:teuthology.orchestra.run.smithi183.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 783 kB/s | 19 kB 00:00 2024-09-12T02:49:56.962 INFO:teuthology.orchestra.run.smithi116.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 3.8 MB/s | 3.6 MB 00:00 2024-09-12T02:49:56.997 INFO:teuthology.orchestra.run.smithi183.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 44 MB/s | 4.4 MB 00:00 2024-09-12T02:49:57.004 INFO:teuthology.orchestra.run.smithi177.stdout:(96/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 2.6 MB/s | 49 MB 00:18 2024-09-12T02:49:57.022 INFO:teuthology.orchestra.run.smithi183.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-12T02:49:57.030 INFO:teuthology.orchestra.run.smithi177.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 92 kB/s | 47 kB 00:00 2024-09-12T02:49:57.048 INFO:teuthology.orchestra.run.smithi183.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-12T02:49:57.057 INFO:teuthology.orchestra.run.smithi116.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 2.6 MB/s | 249 kB 00:00 2024-09-12T02:49:57.082 INFO:teuthology.orchestra.run.smithi183.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-12T02:49:57.089 INFO:teuthology.orchestra.run.smithi177.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 990 kB/s | 58 kB 00:00 2024-09-12T02:49:57.107 INFO:teuthology.orchestra.run.smithi183.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-12T02:49:57.107 INFO:teuthology.orchestra.run.smithi116.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 990 kB/s | 49 kB 00:00 2024-09-12T02:49:57.147 INFO:teuthology.orchestra.run.smithi177.stdout:(99/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 1.6 MB/s | 96 kB 00:00 2024-09-12T02:49:57.148 INFO:teuthology.orchestra.run.smithi183.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-12T02:49:57.165 INFO:teuthology.orchestra.run.smithi116.stdout:(34/156): ceph-prometheus-alerts-19.1.1-298.gb5 283 kB/s | 16 kB 00:00 2024-09-12T02:49:57.197 INFO:teuthology.orchestra.run.smithi177.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-12T02:49:57.207 INFO:teuthology.orchestra.run.smithi183.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-09-12T02:49:57.223 INFO:teuthology.orchestra.run.smithi177.stdout:(101/156): socat-1.7.4.1-6.el9.x86_64.rpm 1.4 MB/s | 304 kB 00:00 2024-09-12T02:49:57.233 INFO:teuthology.orchestra.run.smithi183.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T02:49:57.241 INFO:teuthology.orchestra.run.smithi116.stdout:(35/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 3.7 MB/s | 285 kB 00:00 2024-09-12T02:49:57.248 INFO:teuthology.orchestra.run.smithi177.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-09-12T02:49:57.258 INFO:teuthology.orchestra.run.smithi183.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-12T02:49:57.283 INFO:teuthology.orchestra.run.smithi183.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 568 kB/s | 14 kB 00:00 2024-09-12T02:49:57.308 INFO:teuthology.orchestra.run.smithi183.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-12T02:49:57.315 INFO:teuthology.orchestra.run.smithi177.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-09-12T02:49:57.342 INFO:teuthology.orchestra.run.smithi183.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 11 MB/s | 358 kB 00:00 2024-09-12T02:49:57.365 INFO:teuthology.orchestra.run.smithi177.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-12T02:49:57.367 INFO:teuthology.orchestra.run.smithi183.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 12 MB/s | 303 kB 00:00 2024-09-12T02:49:57.383 INFO:teuthology.orchestra.run.smithi116.stdout:(36/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 5.3 MB/s | 765 kB 00:00 2024-09-12T02:49:57.393 INFO:teuthology.orchestra.run.smithi183.stdout:(120/156): python3-google-auth-2.34.0-1.el9.noa 8.6 MB/s | 220 kB 00:00 2024-09-12T02:49:57.399 INFO:teuthology.orchestra.run.smithi177.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-09-12T02:49:57.425 INFO:teuthology.orchestra.run.smithi177.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 784 kB/s | 19 kB 00:00 2024-09-12T02:49:57.433 INFO:teuthology.orchestra.run.smithi116.stdout:(37/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 365 kB/s | 18 kB 00:00 2024-09-12T02:49:57.468 INFO:teuthology.orchestra.run.smithi183.stdout:(121/156): python3-grpcio-1.46.7-10.el9.x86_64. 27 MB/s | 2.0 MB 00:00 2024-09-12T02:49:57.491 INFO:teuthology.orchestra.run.smithi177.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.5 MB/s | 863 kB 00:00 2024-09-12T02:49:57.493 INFO:teuthology.orchestra.run.smithi183.stdout:(122/156): protobuf-compiler-3.14.0-13.el9.x86_ 880 kB/s | 863 kB 00:00 2024-09-12T02:49:57.527 INFO:teuthology.orchestra.run.smithi183.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-09-12T02:49:57.553 INFO:teuthology.orchestra.run.smithi183.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 182 kB/s | 11 kB 00:00 2024-09-12T02:49:57.566 INFO:teuthology.orchestra.run.smithi177.stdout:(108/156): libarrow-9.0.0-11.el9.x86_64.rpm 31 MB/s | 4.4 MB 00:00 2024-09-12T02:49:57.569 INFO:teuthology.orchestra.run.smithi183.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 440 kB/s | 18 kB 00:00 2024-09-12T02:49:57.586 INFO:teuthology.orchestra.run.smithi183.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 695 kB/s | 23 kB 00:00 2024-09-12T02:49:57.591 INFO:teuthology.orchestra.run.smithi177.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-12T02:49:57.603 INFO:teuthology.orchestra.run.smithi183.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 581 kB/s | 19 kB 00:00 2024-09-12T02:49:57.617 INFO:teuthology.orchestra.run.smithi177.stdout:(110/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 207 kB/s | 26 kB 00:00 2024-09-12T02:49:57.642 INFO:teuthology.orchestra.run.smithi177.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-12T02:49:57.668 INFO:teuthology.orchestra.run.smithi177.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-09-12T02:49:57.704 INFO:teuthology.orchestra.run.smithi183.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 168 kB/s | 20 kB 00:00 2024-09-12T02:49:57.801 INFO:teuthology.orchestra.run.smithi177.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.2 MB/s | 839 kB 00:00 2024-09-12T02:49:57.804 INFO:teuthology.orchestra.run.smithi183.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 463 kB/s | 46 kB 00:00 2024-09-12T02:49:57.829 INFO:teuthology.orchestra.run.smithi183.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T02:49:57.855 INFO:teuthology.orchestra.run.smithi183.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-12T02:49:57.889 INFO:teuthology.orchestra.run.smithi183.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-12T02:49:57.909 INFO:teuthology.orchestra.run.smithi116.stdout:(38/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 8.2 MB/s | 49 MB 00:05 2024-09-12T02:49:57.914 INFO:teuthology.orchestra.run.smithi183.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 663 kB/s | 16 kB 00:00 2024-09-12T02:49:57.919 INFO:teuthology.orchestra.run.smithi177.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 372 kB/s | 43 kB 00:00 2024-09-12T02:49:57.948 INFO:teuthology.orchestra.run.smithi183.stdout:(134/156): python3-kubernetes-26.1.0-2.el9.noar 3.0 MB/s | 1.0 MB 00:00 2024-09-12T02:49:57.952 INFO:teuthology.orchestra.run.smithi177.stdout:(115/156): python3-asyncssh-2.13.2-5.el9.noarch 1.9 MB/s | 548 kB 00:00 2024-09-12T02:49:57.969 INFO:teuthology.orchestra.run.smithi177.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-09-12T02:49:57.972 INFO:teuthology.orchestra.run.smithi183.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-12T02:49:57.986 INFO:teuthology.orchestra.run.smithi177.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 421 kB/s | 14 kB 00:00 2024-09-12T02:49:57.998 INFO:teuthology.orchestra.run.smithi183.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 566 kB/s | 28 kB 00:00 2024-09-12T02:49:58.011 INFO:teuthology.orchestra.run.smithi177.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-09-12T02:49:58.023 INFO:teuthology.orchestra.run.smithi183.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-09-12T02:49:58.037 INFO:teuthology.orchestra.run.smithi177.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-12T02:49:58.048 INFO:teuthology.orchestra.run.smithi183.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-12T02:49:58.062 INFO:teuthology.orchestra.run.smithi177.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-12T02:49:58.082 INFO:teuthology.orchestra.run.smithi183.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-09-12T02:49:58.087 INFO:teuthology.orchestra.run.smithi177.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-12T02:49:58.107 INFO:teuthology.orchestra.run.smithi183.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 610 kB/s | 36 kB 00:00 2024-09-12T02:49:58.120 INFO:teuthology.orchestra.run.smithi177.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-12T02:49:58.132 INFO:teuthology.orchestra.run.smithi183.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-12T02:49:58.146 INFO:teuthology.orchestra.run.smithi177.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-12T02:49:58.151 INFO:teuthology.orchestra.run.smithi116.stdout:(39/156): ceph-mgr-diskprediction-local-19.1.1- 4.8 MB/s | 7.4 MB 00:01 2024-09-12T02:49:58.158 INFO:teuthology.orchestra.run.smithi183.stdout:(142/156): python3-webob-1.8.8-2.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-09-12T02:49:58.171 INFO:teuthology.orchestra.run.smithi177.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 710 kB/s | 18 kB 00:00 2024-09-12T02:49:58.183 INFO:teuthology.orchestra.run.smithi183.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-12T02:49:58.196 INFO:teuthology.orchestra.run.smithi177.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 918 kB/s | 23 kB 00:00 2024-09-12T02:49:58.208 INFO:teuthology.orchestra.run.smithi183.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 802 kB/s | 20 kB 00:00 2024-09-12T02:49:58.222 INFO:teuthology.orchestra.run.smithi177.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 779 kB/s | 19 kB 00:00 2024-09-12T02:49:58.233 INFO:teuthology.orchestra.run.smithi183.stdout:(145/156): re2-20211101-20.el9.x86_64.rpm 7.6 MB/s | 191 kB 00:00 2024-09-12T02:49:58.247 INFO:teuthology.orchestra.run.smithi177.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 793 kB/s | 20 kB 00:00 2024-09-12T02:49:58.259 INFO:teuthology.orchestra.run.smithi183.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-09-12T02:49:58.284 INFO:teuthology.orchestra.run.smithi116.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 94 kB/s | 80 kB 00:00 2024-09-12T02:49:58.298 INFO:teuthology.orchestra.run.smithi177.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-09-12T02:49:58.323 INFO:teuthology.orchestra.run.smithi177.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T02:49:58.357 INFO:teuthology.orchestra.run.smithi177.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-12T02:49:58.382 INFO:teuthology.orchestra.run.smithi177.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-12T02:49:58.384 INFO:teuthology.orchestra.run.smithi183.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-12T02:49:58.415 INFO:teuthology.orchestra.run.smithi177.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-09-12T02:49:58.441 INFO:teuthology.orchestra.run.smithi177.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-09-12T02:49:58.466 INFO:teuthology.orchestra.run.smithi177.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 326 kB/s | 16 kB 00:00 2024-09-12T02:49:58.491 INFO:teuthology.orchestra.run.smithi177.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-12T02:49:58.517 INFO:teuthology.orchestra.run.smithi177.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-12T02:49:58.542 INFO:teuthology.orchestra.run.smithi177.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-12T02:49:58.567 INFO:teuthology.orchestra.run.smithi177.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-09-12T02:49:58.592 INFO:teuthology.orchestra.run.smithi177.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-12T02:49:58.617 INFO:teuthology.orchestra.run.smithi177.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-12T02:49:58.643 INFO:teuthology.orchestra.run.smithi177.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-12T02:49:58.659 INFO:teuthology.orchestra.run.smithi116.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 675 kB/s | 253 kB 00:00 2024-09-12T02:49:58.668 INFO:teuthology.orchestra.run.smithi177.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-12T02:49:58.693 INFO:teuthology.orchestra.run.smithi177.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-12T02:49:58.701 INFO:teuthology.orchestra.run.smithi116.stdout:(42/156): libquadmath-11.5.0-2.el9.x86_64.rpm 347 kB/s | 191 kB 00:00 2024-09-12T02:49:58.727 INFO:teuthology.orchestra.run.smithi177.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 598 kB/s | 20 kB 00:00 2024-09-12T02:49:58.752 INFO:teuthology.orchestra.run.smithi177.stdout:(145/156): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-12T02:49:58.760 INFO:teuthology.orchestra.run.smithi183.stdout:(148/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 8.4 MB/s | 3.2 MB 00:00 2024-09-12T02:49:58.776 INFO:teuthology.orchestra.run.smithi116.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 924 kB/s | 801 kB 00:00 2024-09-12T02:49:58.777 INFO:teuthology.orchestra.run.smithi177.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-09-12T02:49:58.810 INFO:teuthology.orchestra.run.smithi116.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 979 kB/s | 106 kB 00:00 2024-09-12T02:49:58.828 INFO:teuthology.orchestra.run.smithi177.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-12T02:49:58.835 INFO:teuthology.orchestra.run.smithi183.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 5.9 MB/s | 3.4 MB 00:00 2024-09-12T02:49:58.935 INFO:teuthology.orchestra.run.smithi116.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 851 kB/s | 135 kB 00:00 2024-09-12T02:49:58.961 INFO:teuthology.orchestra.run.smithi116.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 829 kB/s | 125 kB 00:00 2024-09-12T02:49:59.061 INFO:teuthology.orchestra.run.smithi116.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.7 MB/s | 215 kB 00:00 2024-09-12T02:49:59.095 INFO:teuthology.orchestra.run.smithi116.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 2.8 MB/s | 1.2 MB 00:00 2024-09-12T02:49:59.144 INFO:teuthology.orchestra.run.smithi183.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 5.9 MB/s | 1.8 MB 00:00 2024-09-12T02:49:59.202 INFO:teuthology.orchestra.run.smithi183.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 701 kB/s | 41 kB 00:00 2024-09-12T02:49:59.211 INFO:teuthology.orchestra.run.smithi116.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 917 kB/s | 107 kB 00:00 2024-09-12T02:49:59.311 INFO:teuthology.orchestra.run.smithi177.stdout:(148/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 6.5 MB/s | 3.2 MB 00:00 2024-09-12T02:49:59.329 INFO:teuthology.orchestra.run.smithi116.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 186 kB/s | 22 kB 00:00 2024-09-12T02:49:59.344 INFO:teuthology.orchestra.run.smithi183.stdout:(152/156): NetworkManager-1.49.90-1.el9.x86_64. 4.0 MB/s | 2.3 MB 00:00 2024-09-12T02:49:59.362 INFO:teuthology.orchestra.run.smithi116.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 1.7 MB/s | 689 kB 00:00 2024-09-12T02:49:59.378 INFO:teuthology.orchestra.run.smithi183.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 1.4 MB/s | 249 kB 00:00 2024-09-12T02:49:59.446 INFO:teuthology.orchestra.run.smithi116.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-12T02:49:59.473 INFO:teuthology.orchestra.run.smithi116.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 301 kB/s | 33 kB 00:00 2024-09-12T02:49:59.520 INFO:teuthology.orchestra.run.smithi177.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-09-12T02:49:59.571 INFO:teuthology.orchestra.run.smithi116.stdout:(54/156): bison-3.7.4-5.el9.x86_64.rpm 1.8 MB/s | 943 kB 00:00 2024-09-12T02:49:59.596 INFO:teuthology.orchestra.run.smithi116.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 145 kB/s | 18 kB 00:00 2024-09-12T02:49:59.612 INFO:teuthology.orchestra.run.smithi177.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 20 MB/s | 1.8 MB 00:00 2024-09-12T02:49:59.662 INFO:teuthology.orchestra.run.smithi177.stdout:(151/156): librados2-19.1.1-298.gb53b6d3e.el9.x 3.9 MB/s | 3.4 MB 00:00 2024-09-12T02:49:59.679 INFO:teuthology.orchestra.run.smithi177.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 610 kB/s | 41 kB 00:00 2024-09-12T02:49:59.680 INFO:teuthology.orchestra.run.smithi116.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 11 kB 00:00 2024-09-12T02:49:59.706 INFO:teuthology.orchestra.run.smithi116.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-09-12T02:49:59.746 INFO:teuthology.orchestra.run.smithi177.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 20 MB/s | 1.4 MB 00:00 2024-09-12T02:49:59.806 INFO:teuthology.orchestra.run.smithi116.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 361 kB/s | 45 kB 00:00 2024-09-12T02:49:59.830 INFO:teuthology.orchestra.run.smithi177.stdout:(154/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 29 MB/s | 2.4 MB 00:00 2024-09-12T02:49:59.855 INFO:teuthology.orchestra.run.smithi177.stdout:(155/156): NetworkManager-tui-1.49.90-1.el9.x86 1.3 MB/s | 249 kB 00:00 2024-09-12T02:49:59.939 INFO:teuthology.orchestra.run.smithi116.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.1 MB/s | 3.0 MB 00:00 2024-09-12T02:50:00.048 INFO:teuthology.orchestra.run.smithi116.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 1.9 MB/s | 662 kB 00:00 2024-09-12T02:50:00.073 INFO:teuthology.orchestra.run.smithi116.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 183 kB/s | 24 kB 00:00 2024-09-12T02:50:00.103 INFO:teuthology.orchestra.run.smithi183.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 1.8 MB/s | 1.4 MB 00:00 2024-09-12T02:50:00.174 INFO:teuthology.orchestra.run.smithi116.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 1.5 MB/s | 581 kB 00:00 2024-09-12T02:50:00.199 INFO:teuthology.orchestra.run.smithi116.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 315 kB/s | 47 kB 00:00 2024-09-12T02:50:00.224 INFO:teuthology.orchestra.run.smithi116.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 292 kB 00:00 2024-09-12T02:50:00.283 INFO:teuthology.orchestra.run.smithi116.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-12T02:50:00.316 INFO:teuthology.orchestra.run.smithi116.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 329 kB/s | 38 kB 00:00 2024-09-12T02:50:00.533 INFO:teuthology.orchestra.run.smithi116.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-09-12T02:50:00.637 INFO:teuthology.orchestra.run.smithi183.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 1.9 MB/s | 2.4 MB 00:01 2024-09-12T02:50:00.658 INFO:teuthology.orchestra.run.smithi116.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 7.1 MB/s | 910 kB 00:00 2024-09-12T02:50:00.784 INFO:teuthology.orchestra.run.smithi116.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-12T02:50:02.042 INFO:teuthology.orchestra.run.smithi116.stdout:(70/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.5 MB/s | 4.4 MB 00:01 2024-09-12T02:50:02.334 INFO:teuthology.orchestra.run.smithi116.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 3.5 MB/s | 1.0 MB 00:00 2024-09-12T02:50:02.576 INFO:teuthology.orchestra.run.smithi116.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 361 kB/s | 87 kB 00:00 2024-09-12T02:50:04.112 INFO:teuthology.orchestra.run.smithi183.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 1.9 MB/s | 16 MB 00:08 2024-09-12T02:50:04.117 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-09-12T02:50:04.117 INFO:teuthology.orchestra.run.smithi183.stdout:Total 8.5 MB/s | 226 MB 00:26 2024-09-12T02:50:04.188 INFO:teuthology.orchestra.run.smithi177.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 1.8 MB/s | 16 MB 00:09 2024-09-12T02:50:04.193 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-09-12T02:50:04.193 INFO:teuthology.orchestra.run.smithi177.stdout:Total 7.8 MB/s | 226 MB 00:28 2024-09-12T02:50:04.218 INFO:teuthology.orchestra.run.smithi116.stdout:(73/156): python3-babel-2.9.1-2.el9.noarch.rpm 3.6 MB/s | 6.0 MB 00:01 2024-09-12T02:50:04.335 INFO:teuthology.orchestra.run.smithi116.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 525 kB/s | 61 kB 00:00 2024-09-12T02:50:04.452 INFO:teuthology.orchestra.run.smithi116.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-09-12T02:50:04.569 INFO:teuthology.orchestra.run.smithi116.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-09-12T02:50:04.669 INFO:teuthology.orchestra.run.smithi116.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-09-12T02:50:04.769 INFO:teuthology.orchestra.run.smithi116.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 191 kB/s | 19 kB 00:00 2024-09-12T02:50:04.878 INFO:teuthology.orchestra.run.smithi116.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-09-12T02:50:04.978 INFO:teuthology.orchestra.run.smithi116.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 348 kB/s | 35 kB 00:00 2024-09-12T02:50:05.137 INFO:teuthology.orchestra.run.smithi116.stdout:(81/156): openblas-openmp-0.3.26-2.el9.x86_64.r 1.0 MB/s | 4.9 MB 00:04 2024-09-12T02:50:05.353 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-09-12T02:50:05.385 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-09-12T02:50:05.503 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-09-12T02:50:05.503 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-09-12T02:50:05.529 INFO:teuthology.orchestra.run.smithi116.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 640 kB/s | 250 kB 00:00 2024-09-12T02:50:05.534 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-09-12T02:50:05.534 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-09-12T02:50:05.913 INFO:teuthology.orchestra.run.smithi116.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 580 kB/s | 222 kB 00:00 2024-09-12T02:50:06.054 INFO:teuthology.orchestra.run.smithi116.stdout:(84/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.7 MB/s | 5.1 MB 00:01 2024-09-12T02:50:06.163 INFO:teuthology.orchestra.run.smithi116.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 390 kB/s | 42 kB 00:00 2024-09-12T02:50:06.188 INFO:teuthology.orchestra.run.smithi116.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 280 kB/s | 77 kB 00:00 2024-09-12T02:50:06.297 INFO:teuthology.orchestra.run.smithi116.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-12T02:50:06.414 INFO:teuthology.orchestra.run.smithi116.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-09-12T02:50:06.556 INFO:teuthology.orchestra.run.smithi116.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 360 kB/s | 51 kB 00:00 2024-09-12T02:50:06.664 INFO:teuthology.orchestra.run.smithi116.stdout:(90/156): python3-protobuf-3.14.0-13.el9.noarch 537 kB/s | 269 kB 00:00 2024-09-12T02:50:06.690 INFO:teuthology.orchestra.run.smithi116.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 402 kB/s | 54 kB 00:00 2024-09-12T02:50:06.798 INFO:teuthology.orchestra.run.smithi116.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 384 kB/s | 42 kB 00:00 2024-09-12T02:50:06.916 INFO:teuthology.orchestra.run.smithi116.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 378 kB/s | 44 kB 00:00 2024-09-12T02:50:06.982 INFO:teuthology.orchestra.run.smithi116.stdout:(94/156): postgresql-server-13.16-1.el9.x86_64. 954 kB/s | 5.8 MB 00:06 2024-09-12T02:50:07.032 INFO:teuthology.orchestra.run.smithi116.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.8 MB/s | 221 kB 00:00 2024-09-12T02:50:07.091 INFO:teuthology.orchestra.run.smithi116.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 330 kB/s | 36 kB 00:00 2024-09-12T02:50:07.141 INFO:teuthology.orchestra.run.smithi116.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 436 kB/s | 47 kB 00:00 2024-09-12T02:50:07.250 INFO:teuthology.orchestra.run.smithi116.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 533 kB/s | 58 kB 00:00 2024-09-12T02:50:07.311 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-09-12T02:50:07.311 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-09-12T02:50:07.323 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-09-12T02:50:07.324 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-09-12T02:50:07.358 INFO:teuthology.orchestra.run.smithi116.stdout:(99/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 884 kB/s | 96 kB 00:00 2024-09-12T02:50:07.467 INFO:teuthology.orchestra.run.smithi116.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 587 kB/s | 64 kB 00:00 2024-09-12T02:50:07.559 INFO:teuthology.orchestra.run.smithi116.stdout:(101/156): socat-1.7.4.1-6.el9.x86_64.rpm 650 kB/s | 304 kB 00:00 2024-09-12T02:50:07.584 INFO:teuthology.orchestra.run.smithi116.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 383 kB/s | 45 kB 00:00 2024-09-12T02:50:07.660 INFO:teuthology.orchestra.run.smithi116.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 223 kB/s | 22 kB 00:00 2024-09-12T02:50:07.718 INFO:teuthology.orchestra.run.smithi116.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-12T02:50:07.744 INFO:teuthology.orchestra.run.smithi116.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-09-12T02:50:07.769 INFO:teuthology.orchestra.run.smithi116.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 774 kB/s | 19 kB 00:00 2024-09-12T02:50:07.794 INFO:teuthology.orchestra.run.smithi116.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 4.0 MB/s | 863 kB 00:00 2024-09-12T02:50:07.837 INFO:teuthology.orchestra.run.smithi116.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 621 kB/s | 26 kB 00:00 2024-09-12T02:50:07.862 INFO:teuthology.orchestra.run.smithi116.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-12T02:50:07.895 INFO:teuthology.orchestra.run.smithi116.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-12T02:50:07.962 INFO:teuthology.orchestra.run.smithi116.stdout:(111/156): libarrow-9.0.0-11.el9.x86_64.rpm 23 MB/s | 4.4 MB 00:00 2024-09-12T02:50:08.063 INFO:teuthology.orchestra.run.smithi116.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 906 kB/s | 151 kB 00:00 2024-09-12T02:50:08.121 INFO:teuthology.orchestra.run.smithi116.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.2 MB/s | 839 kB 00:00 2024-09-12T02:50:08.146 INFO:teuthology.orchestra.run.smithi116.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-09-12T02:50:08.172 INFO:teuthology.orchestra.run.smithi116.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 866 kB/s | 43 kB 00:00 2024-09-12T02:50:08.197 INFO:teuthology.orchestra.run.smithi116.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 640 kB/s | 32 kB 00:00 2024-09-12T02:50:08.222 INFO:teuthology.orchestra.run.smithi116.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-12T02:50:08.247 INFO:teuthology.orchestra.run.smithi116.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-09-12T02:50:08.281 INFO:teuthology.orchestra.run.smithi116.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-12T02:50:08.323 INFO:teuthology.orchestra.run.smithi116.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-09-12T02:50:08.348 INFO:teuthology.orchestra.run.smithi116.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.2 MB/s | 220 kB 00:00 2024-09-12T02:50:08.382 INFO:teuthology.orchestra.run.smithi116.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-12T02:50:08.407 INFO:teuthology.orchestra.run.smithi116.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 426 kB/s | 11 kB 00:00 2024-09-12T02:50:08.432 INFO:teuthology.orchestra.run.smithi116.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 706 kB/s | 18 kB 00:00 2024-09-12T02:50:08.466 INFO:teuthology.orchestra.run.smithi116.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 689 kB/s | 23 kB 00:00 2024-09-12T02:50:08.492 INFO:teuthology.orchestra.run.smithi116.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 781 kB/s | 19 kB 00:00 2024-09-12T02:50:08.517 INFO:teuthology.orchestra.run.smithi116.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 787 kB/s | 20 kB 00:00 2024-09-12T02:50:08.567 INFO:teuthology.orchestra.run.smithi116.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 9.3 MB/s | 2.0 MB 00:00 2024-09-12T02:50:08.592 INFO:teuthology.orchestra.run.smithi116.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T02:50:08.618 INFO:teuthology.orchestra.run.smithi116.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T02:50:08.643 INFO:teuthology.orchestra.run.smithi116.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-12T02:50:08.677 INFO:teuthology.orchestra.run.smithi116.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-09-12T02:50:08.702 INFO:teuthology.orchestra.run.smithi116.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 655 kB/s | 16 kB 00:00 2024-09-12T02:50:08.727 INFO:teuthology.orchestra.run.smithi116.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-12T02:50:08.761 INFO:teuthology.orchestra.run.smithi116.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-12T02:50:08.786 INFO:teuthology.orchestra.run.smithi116.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 480 kB/s | 28 kB 00:00 2024-09-12T02:50:08.811 INFO:teuthology.orchestra.run.smithi116.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-12T02:50:08.837 INFO:teuthology.orchestra.run.smithi116.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-12T02:50:08.862 INFO:teuthology.orchestra.run.smithi116.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-12T02:50:08.887 INFO:teuthology.orchestra.run.smithi116.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-12T02:50:08.913 INFO:teuthology.orchestra.run.smithi116.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-12T02:50:08.946 INFO:teuthology.orchestra.run.smithi116.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 2.6 MB/s | 90 kB 00:00 2024-09-12T02:50:08.972 INFO:teuthology.orchestra.run.smithi116.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-12T02:50:08.988 INFO:teuthology.orchestra.run.smithi116.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-12T02:50:09.022 INFO:teuthology.orchestra.run.smithi116.stdout:(145/156): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-12T02:50:09.049 INFO:teuthology.orchestra.run.smithi116.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-09-12T02:50:09.106 INFO:teuthology.orchestra.run.smithi116.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-12T02:50:09.548 INFO:teuthology.orchestra.run.smithi116.stdout:(148/156): librados2-19.1.1-298.gb53b6d3e.el9.x 6.8 MB/s | 3.4 MB 00:00 2024-09-12T02:50:09.623 INFO:teuthology.orchestra.run.smithi116.stdout:(149/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 6.1 MB/s | 3.2 MB 00:00 2024-09-12T02:50:10.448 INFO:teuthology.orchestra.run.smithi116.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 2.2 MB/s | 1.8 MB 00:00 2024-09-12T02:50:10.565 INFO:teuthology.orchestra.run.smithi116.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 349 kB/s | 41 kB 00:00 2024-09-12T02:50:10.682 INFO:teuthology.orchestra.run.smithi116.stdout:(152/156): NetworkManager-1.49.90-1.el9.x86_64. 2.0 MB/s | 2.3 MB 00:01 2024-09-12T02:50:10.716 INFO:teuthology.orchestra.run.smithi116.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 1.6 MB/s | 249 kB 00:00 2024-09-12T02:50:10.991 INFO:teuthology.orchestra.run.smithi116.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 4.4 MB/s | 1.4 MB 00:00 2024-09-12T02:50:11.041 INFO:teuthology.orchestra.run.smithi116.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 7.4 MB/s | 2.4 MB 00:00 2024-09-12T02:50:13.579 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-09-12T02:50:13.683 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T02:50:13.731 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T02:50:13.776 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-09-12T02:50:13.793 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T02:50:13.826 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T02:50:13.851 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T02:50:13.856 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T02:50:13.894 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T02:50:13.895 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T02:50:13.956 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T02:50:13.990 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T02:50:14.016 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T02:50:14.059 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T02:50:14.240 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T02:50:14.352 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:50:14.389 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T02:50:14.474 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:50:14.500 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:50:14.503 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:50:14.594 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:50:14.622 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:50:14.624 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T02:50:14.643 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:50:14.734 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:50:14.764 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T02:50:14.778 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T02:50:14.829 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T02:50:14.857 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:50:14.919 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T02:50:14.961 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:50:14.977 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T02:50:14.983 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T02:50:15.005 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:50:15.054 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T02:50:15.075 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T02:50:15.102 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T02:50:15.109 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:50:15.132 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T02:50:15.186 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T02:50:15.208 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T02:50:15.243 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T02:50:15.294 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T02:50:15.434 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T02:50:15.439 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T02:50:15.560 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T02:50:15.573 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T02:50:15.672 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:50:15.706 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T02:50:15.762 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:50:15.810 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:50:15.903 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:50:15.975 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:50:16.083 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T02:50:16.117 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:50:16.168 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T02:50:16.200 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T02:50:16.230 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T02:50:16.243 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T02:50:16.306 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T02:50:16.331 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T02:50:16.360 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T02:50:16.374 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T02:50:16.400 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T02:50:16.409 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T02:50:16.444 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T02:50:16.476 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T02:50:16.483 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T02:50:16.505 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T02:50:16.527 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T02:50:16.572 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T02:50:16.576 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T02:50:16.626 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T02:50:16.628 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T02:50:16.660 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T02:50:16.668 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T02:50:16.693 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T02:50:16.696 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T02:50:16.766 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T02:50:16.776 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T02:50:16.812 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T02:50:16.819 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T02:50:16.849 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T02:50:16.855 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T02:50:16.884 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T02:50:16.894 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T02:50:16.953 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T02:50:16.969 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T02:50:16.998 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T02:50:17.004 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T02:50:17.047 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T02:50:17.071 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T02:50:17.076 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T02:50:17.146 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T02:50:17.208 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T02:50:17.211 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T02:50:17.267 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T02:50:17.290 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T02:50:17.424 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T02:50:17.477 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T02:50:17.788 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T02:50:17.887 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T02:50:17.926 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T02:50:17.960 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T02:50:17.998 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T02:50:18.003 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T02:50:18.040 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T02:50:18.091 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T02:50:18.129 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T02:50:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T02:50:18.206 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T02:50:18.209 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T02:50:18.253 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T02:50:18.259 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T02:50:18.313 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T02:50:18.379 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T02:50:18.410 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T02:50:18.410 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T02:50:18.447 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T02:50:18.481 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T02:50:18.501 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T02:50:18.535 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T02:50:18.546 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T02:50:18.589 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T02:50:18.802 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T02:50:18.802 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T02:50:18.802 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T02:50:18.802 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T02:50:18.803 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T02:50:18.803 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T02:50:18.803 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T02:50:18.808 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T02:50:18.815 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T02:50:18.849 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T02:50:18.862 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T02:50:18.950 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T02:50:18.991 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T02:50:19.010 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T02:50:19.031 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T02:50:19.038 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T02:50:19.057 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T02:50:19.080 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T02:50:19.090 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T02:50:19.163 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T02:50:19.182 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T02:50:19.213 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T02:50:19.253 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T02:50:19.287 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T02:50:19.299 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T02:50:19.320 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T02:50:19.330 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T02:50:19.358 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T02:50:19.392 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T02:50:19.393 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T02:50:19.431 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T02:50:19.463 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T02:50:19.501 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T02:50:19.522 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T02:50:19.544 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T02:50:19.561 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T02:50:19.578 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T02:50:19.589 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T02:50:19.619 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T02:50:19.623 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T02:50:19.654 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T02:50:19.671 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T02:50:19.710 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T02:50:19.740 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T02:50:19.747 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T02:50:19.783 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T02:50:19.792 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T02:50:19.823 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T02:50:19.825 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T02:50:19.867 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T02:50:19.901 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T02:50:19.926 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T02:50:19.982 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T02:50:19.988 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T02:50:20.040 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T02:50:20.072 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T02:50:20.174 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T02:50:20.230 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T02:50:20.690 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T02:50:20.738 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T02:50:20.771 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T02:50:20.868 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T02:50:20.926 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T02:50:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T02:50:20.970 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T02:50:21.003 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T02:50:21.037 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T02:50:21.049 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T02:50:21.082 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T02:50:21.134 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T02:50:21.183 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T02:50:21.228 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T02:50:21.265 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T02:50:21.305 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T02:50:21.323 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T02:50:21.356 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T02:50:21.382 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T02:50:21.539 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T02:50:21.571 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T02:50:21.647 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T02:50:22.018 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T02:50:22.128 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T02:50:22.290 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T02:50:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T02:50:23.338 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T02:50:23.426 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T02:50:23.453 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T02:50:23.486 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T02:50:23.624 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T02:50:23.733 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T02:50:23.761 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T02:50:23.794 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T02:50:23.803 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T02:50:23.849 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T02:50:23.939 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T02:50:23.970 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T02:50:24.006 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T02:50:24.103 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T02:50:24.149 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T02:50:24.240 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T02:50:24.278 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T02:50:24.314 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T02:50:24.486 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T02:50:24.518 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T02:50:24.589 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T02:50:24.612 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T02:50:24.790 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T02:50:24.827 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T02:50:24.898 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T02:50:24.928 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T02:50:26.920 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:50:26.960 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:50:27.239 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:50:27.283 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:50:27.505 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:50:27.815 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T02:50:27.828 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:50:27.870 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T02:50:27.908 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T02:50:27.909 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T02:50:27.909 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T02:50:27.909 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-12T02:50:27.939 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T02:50:27.955 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T02:50:27.955 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T02:50:27.955 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T02:50:27.956 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T02:50:27.988 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T02:50:30.481 INFO:teuthology.orchestra.run.smithi116.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 699 kB/s | 16 MB 00:23 2024-09-12T02:50:30.487 INFO:teuthology.orchestra.run.smithi116.stdout:-------------------------------------------------------------------------------- 2024-09-12T02:50:30.487 INFO:teuthology.orchestra.run.smithi116.stdout:Total 4.2 MB/s | 226 MB 00:54 2024-09-12T02:50:31.656 INFO:teuthology.orchestra.run.smithi116.stdout:Running transaction check 2024-09-12T02:50:31.807 INFO:teuthology.orchestra.run.smithi116.stdout:Transaction check succeeded. 2024-09-12T02:50:31.808 INFO:teuthology.orchestra.run.smithi116.stdout:Running transaction test 2024-09-12T02:50:34.055 INFO:teuthology.orchestra.run.smithi116.stdout:Transaction test succeeded. 2024-09-12T02:50:34.055 INFO:teuthology.orchestra.run.smithi116.stdout:Running transaction 2024-09-12T02:50:39.553 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T02:50:39.553 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /sys 2024-09-12T02:50:39.553 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /proc 2024-09-12T02:50:39.554 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /mnt 2024-09-12T02:50:39.554 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /var/tmp 2024-09-12T02:50:39.554 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /home 2024-09-12T02:50:39.554 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /root 2024-09-12T02:50:39.554 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /tmp 2024-09-12T02:50:39.554 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-12T02:50:39.720 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T02:50:39.720 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /sys 2024-09-12T02:50:39.720 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /proc 2024-09-12T02:50:39.721 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /mnt 2024-09-12T02:50:39.721 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /var/tmp 2024-09-12T02:50:39.721 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /home 2024-09-12T02:50:39.721 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /root 2024-09-12T02:50:39.721 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /tmp 2024-09-12T02:50:39.721 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T02:50:39.725 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T02:50:39.741 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T02:50:39.741 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:50:39.742 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T02:50:39.742 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-12T02:50:39.900 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T02:50:39.916 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T02:50:39.916 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:50:39.916 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T02:50:39.916 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T02:50:40.388 INFO:teuthology.orchestra.run.smithi116.stdout: Preparing : 1/1 2024-09-12T02:50:40.510 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T02:50:40.560 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T02:50:40.632 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T02:50:40.669 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T02:50:40.694 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T02:50:40.744 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T02:50:41.081 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T02:50:41.187 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:50:41.211 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T02:50:41.238 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T02:50:41.238 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:50:41.238 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T02:50:41.238 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T02:50:41.238 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T02:50:41.239 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-12T02:50:41.293 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:50:41.321 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:50:41.386 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T02:50:41.396 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:50:41.413 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T02:50:41.413 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:50:41.413 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T02:50:41.413 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T02:50:41.413 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T02:50:41.414 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T02:50:41.426 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T02:50:41.470 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T02:50:41.493 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T02:50:41.494 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:50:41.494 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T02:50:41.494 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T02:50:41.494 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T02:50:41.494 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-12T02:50:41.584 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T02:50:41.631 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T02:50:41.637 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T02:50:41.660 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T02:50:41.661 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:50:41.661 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T02:50:41.661 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T02:50:41.661 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T02:50:41.661 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T02:50:41.684 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:50:41.787 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:50:41.818 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T02:50:41.881 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T02:50:41.919 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T02:50:41.929 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T02:50:41.940 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T02:50:41.952 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T02:50:41.952 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:50:41.952 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T02:50:41.952 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T02:50:41.952 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T02:50:41.952 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-12T02:50:42.051 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T02:50:42.096 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T02:50:42.107 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T02:50:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T02:50:42.120 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:50:42.120 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T02:50:42.120 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T02:50:42.120 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T02:50:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T02:50:42.131 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T02:50:42.162 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T02:50:42.226 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T02:50:42.282 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T02:50:42.309 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T02:50:42.338 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T02:50:42.355 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T02:50:42.404 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T02:50:42.451 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T02:50:42.474 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T02:50:42.532 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T02:50:42.561 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T02:50:42.567 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:50:42.580 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T02:50:42.644 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:50:42.658 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T02:50:42.700 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T02:50:42.738 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T02:50:42.746 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T02:50:42.770 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T02:50:42.798 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T02:50:42.877 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:50:42.878 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T02:50:42.922 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T02:50:42.954 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T02:50:42.990 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T02:50:42.993 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T02:50:43.095 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T02:50:43.141 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T02:50:43.181 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T02:50:43.183 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T02:50:43.213 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T02:50:43.244 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T02:50:43.257 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T02:50:43.301 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T02:50:43.305 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T02:50:43.336 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T02:50:43.342 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T02:50:43.385 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T02:50:43.418 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T02:50:43.444 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T02:50:43.445 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T02:50:43.472 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T02:50:43.512 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T02:50:43.513 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T02:50:43.567 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T02:50:43.574 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T02:50:43.604 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T02:50:43.607 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T02:50:43.659 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T02:50:43.734 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T02:50:43.769 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T02:50:43.814 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T02:50:43.841 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T02:50:43.910 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T02:50:43.956 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T02:50:44.030 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T02:50:44.171 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T02:50:44.225 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T02:50:44.369 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T02:50:44.413 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T02:50:44.655 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T02:50:44.697 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T02:50:44.742 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T02:50:44.830 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T02:50:44.867 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T02:50:44.909 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T02:50:44.947 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T02:50:44.989 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T02:50:45.165 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T02:50:45.242 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T02:50:45.296 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T02:50:45.369 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T02:50:45.372 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T02:50:45.410 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T02:50:45.413 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T02:50:45.445 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T02:50:45.492 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T02:50:45.514 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T02:50:45.528 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T02:50:45.571 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T02:50:45.625 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T02:50:45.668 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T02:50:45.683 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T02:50:45.714 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T02:50:45.715 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T02:50:45.739 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T02:50:45.739 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:50:45.739 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T02:50:45.740 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T02:50:45.740 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T02:50:45.740 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-12T02:50:45.760 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T02:50:45.791 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T02:50:45.803 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T02:50:45.810 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T02:50:45.818 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T02:50:45.851 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T02:50:45.899 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T02:50:45.947 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T02:50:45.954 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T02:50:45.976 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T02:50:45.986 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T02:50:46.007 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T02:50:46.025 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T02:50:46.032 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T02:50:46.032 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:50:46.032 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T02:50:46.032 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T02:50:46.032 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T02:50:46.032 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T02:50:46.059 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T02:50:46.083 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T02:50:46.084 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T02:50:46.095 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T02:50:46.180 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T02:50:46.194 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T02:50:46.249 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T02:50:46.308 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T02:50:46.349 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T02:50:46.378 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T02:50:46.414 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T02:50:46.451 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T02:50:46.482 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T02:50:46.512 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T02:50:46.547 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T02:50:46.588 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T02:50:46.630 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T02:50:46.673 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T02:50:46.758 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T02:50:46.811 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T02:50:46.851 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T02:50:46.866 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T02:50:46.893 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T02:50:46.917 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T02:50:46.917 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:50:46.917 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T02:50:46.917 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T02:50:46.917 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T02:50:46.917 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-12T02:50:46.954 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T02:50:46.961 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T02:50:46.983 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T02:50:46.983 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:50:46.983 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T02:50:46.983 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-12T02:50:47.002 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T02:50:47.037 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T02:50:47.060 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T02:50:47.060 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:50:47.060 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T02:50:47.060 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T02:50:47.060 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T02:50:47.060 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-12T02:50:47.178 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T02:50:47.203 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T02:50:47.227 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T02:50:47.227 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:50:47.228 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T02:50:47.228 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T02:50:47.228 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T02:50:47.228 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T02:50:47.278 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T02:50:47.302 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T02:50:47.302 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:50:47.302 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T02:50:47.302 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T02:50:47.354 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T02:50:47.362 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T02:50:47.377 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T02:50:47.377 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:50:47.378 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T02:50:47.378 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T02:50:47.378 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T02:50:47.378 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T02:50:47.385 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T02:50:47.385 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:50:47.385 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T02:50:47.385 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T02:50:47.385 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T02:50:47.385 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-12T02:50:47.680 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T02:50:47.703 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T02:50:47.703 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:50:47.703 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T02:50:47.704 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T02:50:47.704 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T02:50:47.704 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T02:50:47.721 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T02:50:47.766 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T02:50:47.800 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T02:50:47.898 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T02:50:47.954 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T02:50:47.999 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T02:50:48.086 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T02:50:48.127 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T02:50:48.310 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T02:50:48.342 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T02:50:48.426 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T02:50:49.289 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T02:50:49.407 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T02:50:50.654 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T02:50:50.747 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T02:50:50.775 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T02:50:50.808 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T02:50:51.120 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T02:50:51.163 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T02:50:51.254 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T02:50:51.284 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T02:50:51.328 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T02:50:51.814 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T02:50:51.849 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T02:50:51.907 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T02:50:51.920 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T02:50:51.948 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T02:50:51.950 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T02:50:51.989 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T02:50:52.021 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T02:50:52.049 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T02:50:52.225 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T02:50:52.226 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T02:50:52.266 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T02:50:52.279 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T02:50:52.307 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T02:50:52.309 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T02:50:52.339 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T02:50:52.339 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T02:50:52.366 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T02:50:52.385 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T02:50:52.545 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T02:50:52.588 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T02:50:52.618 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T02:50:52.648 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T02:50:52.694 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T02:50:53.008 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:50:53.049 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:50:53.069 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:50:53.288 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:50:53.334 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:50:53.354 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:50:53.906 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T02:50:53.995 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T02:50:54.036 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T02:50:54.124 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T02:50:54.165 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T02:50:54.184 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T02:50:54.191 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T02:50:54.217 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T02:50:54.217 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T02:50:54.272 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T02:50:54.301 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T02:50:54.301 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T02:50:54.307 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:50:54.312 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T02:50:54.339 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:50:54.385 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T02:50:54.393 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T02:50:54.410 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T02:50:54.415 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:50:54.415 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:50:54.433 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T02:50:54.467 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T02:50:54.502 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T02:50:54.502 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T02:50:54.577 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T02:50:54.577 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T02:50:54.595 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:50:54.595 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T02:50:54.670 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T02:50:54.688 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T02:50:54.695 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T02:50:54.700 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:50:54.700 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:50:54.750 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:50:54.880 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:50:54.880 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T02:50:54.898 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:50:54.934 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T02:50:54.964 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T02:50:55.021 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T02:50:55.021 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T02:50:55.021 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T02:50:55.021 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:50:55.035 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:50:55.061 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T02:51:00.316 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:51:00.316 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T02:51:00.316 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T02:51:00.316 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T02:51:00.316 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T02:51:00.316 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T02:51:00.316 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T02:51:00.317 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T02:51:00.317 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:51:00.317 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:51:00.317 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T02:51:00.317 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T02:51:00.317 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T02:51:00.317 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:51:00.317 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T02:51:00.317 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T02:51:00.317 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T02:51:00.317 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T02:51:00.320 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T02:51:00.321 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T02:51:00.321 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T02:51:00.321 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T02:51:00.321 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T02:51:00.321 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T02:51:00.321 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T02:51:00.321 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T02:51:00.321 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T02:51:00.321 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T02:51:00.321 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T02:51:00.321 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T02:51:00.321 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T02:51:00.322 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T02:51:00.322 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T02:51:00.322 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T02:51:00.322 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T02:51:00.322 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T02:51:00.322 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T02:51:00.322 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T02:51:00.322 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T02:51:00.323 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T02:51:00.323 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T02:51:00.323 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T02:51:00.323 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T02:51:00.323 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T02:51:00.323 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T02:51:00.323 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T02:51:00.323 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T02:51:00.323 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T02:51:00.323 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T02:51:00.323 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T02:51:00.323 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T02:51:00.324 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T02:51:00.324 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T02:51:00.324 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T02:51:00.324 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T02:51:00.324 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T02:51:00.324 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T02:51:00.324 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T02:51:00.324 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T02:51:00.324 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T02:51:00.324 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T02:51:00.324 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T02:51:00.324 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T02:51:00.325 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T02:51:00.325 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T02:51:00.325 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T02:51:00.325 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T02:51:00.325 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T02:51:00.325 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T02:51:00.325 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T02:51:00.325 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T02:51:00.325 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T02:51:00.326 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T02:51:00.326 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T02:51:00.326 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T02:51:00.326 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T02:51:00.326 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T02:51:00.326 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T02:51:00.326 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T02:51:00.326 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T02:51:00.326 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T02:51:00.326 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T02:51:00.326 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T02:51:00.326 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T02:51:00.327 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T02:51:00.327 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T02:51:00.327 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T02:51:00.327 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T02:51:00.327 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T02:51:00.327 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T02:51:00.327 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T02:51:00.327 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T02:51:00.327 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T02:51:00.327 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T02:51:00.327 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T02:51:00.328 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T02:51:00.328 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T02:51:00.328 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T02:51:00.328 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T02:51:00.328 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T02:51:00.328 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T02:51:00.328 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T02:51:00.328 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T02:51:00.328 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T02:51:00.328 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T02:51:00.329 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T02:51:00.329 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T02:51:00.329 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T02:51:00.329 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T02:51:00.329 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T02:51:00.329 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T02:51:00.329 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T02:51:00.329 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T02:51:00.329 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T02:51:00.329 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T02:51:00.329 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T02:51:00.329 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T02:51:00.330 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T02:51:00.330 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T02:51:00.330 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T02:51:00.330 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T02:51:00.330 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T02:51:00.330 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T02:51:00.330 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T02:51:00.330 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T02:51:00.330 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T02:51:00.330 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T02:51:00.330 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T02:51:00.331 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T02:51:00.331 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T02:51:00.331 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T02:51:00.331 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T02:51:00.331 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T02:51:00.331 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T02:51:00.331 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T02:51:00.331 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T02:51:00.331 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T02:51:00.331 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T02:51:00.331 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T02:51:00.331 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T02:51:00.332 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T02:51:00.332 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T02:51:00.332 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T02:51:00.332 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T02:51:00.332 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T02:51:00.332 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T02:51:00.332 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T02:51:00.332 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T02:51:00.333 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T02:51:00.333 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T02:51:00.333 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T02:51:00.333 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T02:51:00.333 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T02:51:00.333 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T02:51:00.333 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T02:51:00.334 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T02:51:00.334 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T02:51:00.334 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T02:51:00.334 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T02:51:00.334 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T02:51:00.334 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T02:51:00.335 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T02:51:00.335 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T02:51:00.335 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T02:51:00.660 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:51:00.660 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T02:51:00.660 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T02:51:00.661 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T02:51:00.661 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T02:51:00.661 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T02:51:00.661 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T02:51:00.661 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T02:51:00.661 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:51:00.661 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:51:00.661 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T02:51:00.661 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T02:51:00.662 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T02:51:00.662 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:51:00.662 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T02:51:00.662 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T02:51:00.662 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T02:51:00.662 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T02:51:00.662 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T02:51:00.662 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T02:51:00.662 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T02:51:00.662 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T02:51:00.662 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T02:51:00.662 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T02:51:00.662 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T02:51:00.663 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T02:51:00.663 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T02:51:00.663 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T02:51:00.663 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T02:51:00.663 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T02:51:00.663 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T02:51:00.663 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T02:51:00.663 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T02:51:00.663 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T02:51:00.663 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T02:51:00.663 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T02:51:00.663 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T02:51:00.663 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T02:51:00.663 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T02:51:00.664 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T02:51:00.664 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T02:51:00.664 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T02:51:00.664 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T02:51:00.664 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T02:51:00.664 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T02:51:00.664 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T02:51:00.664 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T02:51:00.665 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T02:51:00.665 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T02:51:00.665 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T02:51:00.665 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T02:51:00.665 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T02:51:00.665 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T02:51:00.665 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T02:51:00.665 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T02:51:00.665 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T02:51:00.665 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T02:51:00.665 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T02:51:00.665 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T02:51:00.666 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T02:51:00.666 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T02:51:00.666 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T02:51:00.666 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T02:51:00.666 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T02:51:00.666 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T02:51:00.666 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T02:51:00.666 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T02:51:00.666 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T02:51:00.666 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T02:51:00.666 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T02:51:00.666 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T02:51:00.666 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T02:51:00.667 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T02:51:00.667 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T02:51:00.667 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T02:51:00.667 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T02:51:00.667 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T02:51:00.667 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T02:51:00.667 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T02:51:00.667 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T02:51:00.667 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T02:51:00.667 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T02:51:00.667 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T02:51:00.667 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T02:51:00.667 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T02:51:00.668 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T02:51:00.668 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T02:51:00.668 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T02:51:00.668 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T02:51:00.668 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T02:51:00.668 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T02:51:00.668 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T02:51:00.669 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T02:51:00.669 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T02:51:00.669 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T02:51:00.669 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T02:51:00.669 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T02:51:00.669 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T02:51:00.669 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T02:51:00.669 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T02:51:00.669 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T02:51:00.669 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T02:51:00.669 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T02:51:00.669 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T02:51:00.669 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T02:51:00.670 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T02:51:00.670 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T02:51:00.670 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T02:51:00.670 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T02:51:00.670 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T02:51:00.670 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T02:51:00.670 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T02:51:00.670 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T02:51:00.670 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T02:51:00.670 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T02:51:00.671 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T02:51:00.671 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T02:51:00.671 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T02:51:00.671 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T02:51:00.671 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T02:51:00.671 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T02:51:00.671 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T02:51:00.672 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T02:51:00.672 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T02:51:00.672 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T02:51:00.672 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T02:51:00.672 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T02:51:00.672 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T02:51:00.672 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T02:51:00.672 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T02:51:00.672 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T02:51:00.672 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T02:51:00.672 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T02:51:00.672 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T02:51:00.672 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T02:51:00.673 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T02:51:00.673 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T02:51:00.673 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T02:51:00.673 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T02:51:00.673 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T02:51:00.673 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T02:51:00.673 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T02:51:00.673 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T02:51:00.673 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T02:51:00.673 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T02:51:00.673 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T02:51:00.673 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T02:51:00.673 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T02:51:00.673 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T02:51:00.674 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T02:51:00.674 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T02:51:00.674 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T02:51:00.674 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T02:51:00.674 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T02:51:00.674 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T02:51:00.674 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T02:51:00.674 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T02:51:00.674 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T02:51:00.674 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T02:51:00.674 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T02:51:00.674 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T02:51:00.674 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T02:51:00.675 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T02:51:00.675 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T02:51:02.856 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:51:02.856 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-12T02:51:02.856 INFO:teuthology.orchestra.run.smithi177.stdout:Upgraded: 2024-09-12T02:51:02.856 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T02:51:02.856 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T02:51:02.856 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T02:51:02.856 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T02:51:02.856 INFO:teuthology.orchestra.run.smithi177.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.857 INFO:teuthology.orchestra.run.smithi177.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.857 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T02:51:02.857 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T02:51:02.857 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-09-12T02:51:02.857 INFO:teuthology.orchestra.run.smithi177.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T02:51:02.857 INFO:teuthology.orchestra.run.smithi177.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T02:51:02.857 INFO:teuthology.orchestra.run.smithi177.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T02:51:02.857 INFO:teuthology.orchestra.run.smithi177.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T02:51:02.857 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.857 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.857 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.857 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.857 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:51:02.858 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.858 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.858 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.858 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:51:02.858 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:51:02.858 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:51:02.858 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:51:02.858 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:51:02.858 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.858 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.858 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:51:02.858 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.858 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.858 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.859 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:51:02.859 INFO:teuthology.orchestra.run.smithi177.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:51:02.859 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.859 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.859 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:51:02.859 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T02:51:02.859 INFO:teuthology.orchestra.run.smithi177.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T02:51:02.859 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T02:51:02.859 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T02:51:02.859 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T02:51:02.859 INFO:teuthology.orchestra.run.smithi177.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T02:51:02.859 INFO:teuthology.orchestra.run.smithi177.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T02:51:02.859 INFO:teuthology.orchestra.run.smithi177.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T02:51:02.860 INFO:teuthology.orchestra.run.smithi177.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T02:51:02.860 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T02:51:02.860 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T02:51:02.860 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.860 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.860 INFO:teuthology.orchestra.run.smithi177.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.860 INFO:teuthology.orchestra.run.smithi177.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T02:51:02.860 INFO:teuthology.orchestra.run.smithi177.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T02:51:02.860 INFO:teuthology.orchestra.run.smithi177.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T02:51:02.860 INFO:teuthology.orchestra.run.smithi177.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T02:51:02.860 INFO:teuthology.orchestra.run.smithi177.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T02:51:02.860 INFO:teuthology.orchestra.run.smithi177.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.860 INFO:teuthology.orchestra.run.smithi177.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.861 INFO:teuthology.orchestra.run.smithi177.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T02:51:02.861 INFO:teuthology.orchestra.run.smithi177.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.861 INFO:teuthology.orchestra.run.smithi177.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T02:51:02.861 INFO:teuthology.orchestra.run.smithi177.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T02:51:02.861 INFO:teuthology.orchestra.run.smithi177.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T02:51:02.861 INFO:teuthology.orchestra.run.smithi177.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T02:51:02.862 INFO:teuthology.orchestra.run.smithi177.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T02:51:02.862 INFO:teuthology.orchestra.run.smithi177.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T02:51:02.862 INFO:teuthology.orchestra.run.smithi177.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T02:51:02.862 INFO:teuthology.orchestra.run.smithi177.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T02:51:02.862 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T02:51:02.862 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T02:51:02.862 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T02:51:02.862 INFO:teuthology.orchestra.run.smithi177.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T02:51:02.862 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T02:51:02.862 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T02:51:02.862 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T02:51:02.863 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T02:51:02.863 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T02:51:02.863 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T02:51:02.863 INFO:teuthology.orchestra.run.smithi177.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T02:51:02.863 INFO:teuthology.orchestra.run.smithi177.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T02:51:02.863 INFO:teuthology.orchestra.run.smithi177.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T02:51:02.863 INFO:teuthology.orchestra.run.smithi177.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T02:51:02.863 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T02:51:02.863 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.863 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.863 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.863 INFO:teuthology.orchestra.run.smithi177.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T02:51:02.863 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T02:51:02.864 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T02:51:02.864 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T02:51:02.864 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T02:51:02.864 INFO:teuthology.orchestra.run.smithi177.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T02:51:02.864 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T02:51:02.864 INFO:teuthology.orchestra.run.smithi177.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T02:51:02.864 INFO:teuthology.orchestra.run.smithi177.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T02:51:02.864 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T02:51:02.864 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T02:51:02.864 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T02:51:02.864 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T02:51:02.864 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T02:51:02.864 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T02:51:02.865 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T02:51:02.865 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T02:51:02.865 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T02:51:02.865 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T02:51:02.868 INFO:teuthology.orchestra.run.smithi177.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T02:51:02.868 INFO:teuthology.orchestra.run.smithi177.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T02:51:02.869 INFO:teuthology.orchestra.run.smithi177.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T02:51:02.869 INFO:teuthology.orchestra.run.smithi177.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T02:51:02.869 INFO:teuthology.orchestra.run.smithi177.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T02:51:02.869 INFO:teuthology.orchestra.run.smithi177.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T02:51:02.869 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T02:51:02.869 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T02:51:02.869 INFO:teuthology.orchestra.run.smithi177.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T02:51:02.869 INFO:teuthology.orchestra.run.smithi177.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T02:51:02.869 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T02:51:02.869 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T02:51:02.869 INFO:teuthology.orchestra.run.smithi177.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T02:51:02.869 INFO:teuthology.orchestra.run.smithi177.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T02:51:02.869 INFO:teuthology.orchestra.run.smithi177.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T02:51:02.869 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T02:51:02.870 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T02:51:02.870 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T02:51:02.870 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T02:51:02.870 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T02:51:02.870 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T02:51:02.870 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.870 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.870 INFO:teuthology.orchestra.run.smithi177.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T02:51:02.870 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T02:51:02.870 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T02:51:02.870 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.870 INFO:teuthology.orchestra.run.smithi177.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T02:51:02.870 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T02:51:02.871 INFO:teuthology.orchestra.run.smithi177.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T02:51:02.871 INFO:teuthology.orchestra.run.smithi177.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T02:51:02.871 INFO:teuthology.orchestra.run.smithi177.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T02:51:02.871 INFO:teuthology.orchestra.run.smithi177.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T02:51:02.871 INFO:teuthology.orchestra.run.smithi177.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T02:51:02.871 INFO:teuthology.orchestra.run.smithi177.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T02:51:02.871 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T02:51:02.871 INFO:teuthology.orchestra.run.smithi177.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T02:51:02.871 INFO:teuthology.orchestra.run.smithi177.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T02:51:02.871 INFO:teuthology.orchestra.run.smithi177.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T02:51:02.871 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T02:51:02.871 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T02:51:02.871 INFO:teuthology.orchestra.run.smithi177.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T02:51:02.871 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.872 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.872 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:02.872 INFO:teuthology.orchestra.run.smithi177.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T02:51:02.872 INFO:teuthology.orchestra.run.smithi177.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T02:51:02.872 INFO:teuthology.orchestra.run.smithi177.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T02:51:02.872 INFO:teuthology.orchestra.run.smithi177.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T02:51:02.872 INFO:teuthology.orchestra.run.smithi177.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T02:51:02.872 INFO:teuthology.orchestra.run.smithi177.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T02:51:02.873 INFO:teuthology.orchestra.run.smithi177.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T02:51:02.873 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-12T02:51:02.873 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-09-12T02:51:03.227 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:51:03.228 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T02:51:03.228 INFO:teuthology.orchestra.run.smithi183.stdout:Upgraded: 2024-09-12T02:51:03.228 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T02:51:03.228 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T02:51:03.228 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T02:51:03.228 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T02:51:03.228 INFO:teuthology.orchestra.run.smithi183.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.228 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.228 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T02:51:03.228 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T02:51:03.228 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-09-12T02:51:03.229 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T02:51:03.229 INFO:teuthology.orchestra.run.smithi183.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T02:51:03.229 INFO:teuthology.orchestra.run.smithi183.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T02:51:03.229 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T02:51:03.229 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.229 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.229 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.229 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.229 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:51:03.229 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.229 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.229 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.229 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:51:03.230 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:51:03.230 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:51:03.230 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:51:03.230 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:51:03.230 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.230 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.230 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:51:03.230 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.230 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.230 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.230 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:51:03.230 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:51:03.230 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.230 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.231 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:51:03.231 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T02:51:03.231 INFO:teuthology.orchestra.run.smithi183.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T02:51:03.231 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T02:51:03.231 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T02:51:03.231 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T02:51:03.231 INFO:teuthology.orchestra.run.smithi183.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T02:51:03.231 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T02:51:03.231 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T02:51:03.231 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T02:51:03.231 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T02:51:03.231 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T02:51:03.231 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.231 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.232 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.232 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T02:51:03.232 INFO:teuthology.orchestra.run.smithi183.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T02:51:03.232 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T02:51:03.232 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T02:51:03.232 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T02:51:03.232 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.232 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.232 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T02:51:03.232 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.232 INFO:teuthology.orchestra.run.smithi183.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T02:51:03.232 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T02:51:03.232 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T02:51:03.233 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T02:51:03.233 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T02:51:03.233 INFO:teuthology.orchestra.run.smithi183.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T02:51:03.233 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T02:51:03.233 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T02:51:03.233 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T02:51:03.233 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T02:51:03.233 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T02:51:03.233 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T02:51:03.233 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T02:51:03.233 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T02:51:03.233 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T02:51:03.233 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T02:51:03.233 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T02:51:03.234 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T02:51:03.234 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T02:51:03.234 INFO:teuthology.orchestra.run.smithi183.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T02:51:03.234 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T02:51:03.234 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T02:51:03.234 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T02:51:03.234 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.234 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.234 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.235 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T02:51:03.235 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T02:51:03.235 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T02:51:03.235 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T02:51:03.235 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T02:51:03.235 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T02:51:03.235 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T02:51:03.235 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T02:51:03.235 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T02:51:03.235 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T02:51:03.235 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T02:51:03.235 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T02:51:03.235 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T02:51:03.235 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T02:51:03.236 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T02:51:03.236 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T02:51:03.236 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T02:51:03.236 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T02:51:03.236 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T02:51:03.236 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T02:51:03.236 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T02:51:03.236 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T02:51:03.236 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T02:51:03.237 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T02:51:03.237 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T02:51:03.237 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T02:51:03.237 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T02:51:03.237 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T02:51:03.237 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T02:51:03.237 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T02:51:03.237 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T02:51:03.237 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T02:51:03.237 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T02:51:03.237 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T02:51:03.237 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T02:51:03.237 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T02:51:03.237 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T02:51:03.238 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T02:51:03.238 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T02:51:03.238 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T02:51:03.238 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.238 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.238 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T02:51:03.238 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T02:51:03.238 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T02:51:03.238 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.238 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T02:51:03.238 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T02:51:03.238 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T02:51:03.238 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T02:51:03.238 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T02:51:03.238 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T02:51:03.239 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T02:51:03.239 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T02:51:03.239 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T02:51:03.239 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T02:51:03.239 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T02:51:03.239 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T02:51:03.240 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T02:51:03.240 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T02:51:03.240 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T02:51:03.240 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.240 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.240 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:03.240 INFO:teuthology.orchestra.run.smithi183.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T02:51:03.240 INFO:teuthology.orchestra.run.smithi183.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T02:51:03.240 INFO:teuthology.orchestra.run.smithi183.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T02:51:03.240 INFO:teuthology.orchestra.run.smithi183.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T02:51:03.240 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T02:51:03.240 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T02:51:03.241 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T02:51:03.241 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T02:51:03.241 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-09-12T02:51:03.265 DEBUG:teuthology.parallel:result is None 2024-09-12T02:51:03.646 DEBUG:teuthology.parallel:result is None 2024-09-12T02:51:06.861 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T02:51:06.861 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /sys 2024-09-12T02:51:06.861 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /proc 2024-09-12T02:51:06.861 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /mnt 2024-09-12T02:51:06.861 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /var/tmp 2024-09-12T02:51:06.861 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /home 2024-09-12T02:51:06.861 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /root 2024-09-12T02:51:06.861 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /tmp 2024-09-12T02:51:06.861 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:51:07.038 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T02:51:07.054 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T02:51:07.054 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:51:07.055 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T02:51:07.055 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:51:08.566 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T02:51:08.593 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T02:51:08.593 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:51:08.593 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T02:51:08.593 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T02:51:08.593 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T02:51:08.594 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:51:08.834 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T02:51:08.857 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T02:51:08.857 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:51:08.857 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T02:51:08.857 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T02:51:08.857 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T02:51:08.857 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:51:09.301 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T02:51:09.324 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T02:51:09.324 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:51:09.324 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T02:51:09.324 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T02:51:09.325 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T02:51:09.325 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:51:09.439 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T02:51:09.487 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T02:51:09.542 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T02:51:09.726 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T02:51:09.785 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T02:51:09.864 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T02:51:09.968 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T02:51:10.096 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T02:51:10.152 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T02:51:10.184 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T02:51:10.220 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T02:51:10.663 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T02:51:10.694 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T02:51:10.725 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T02:51:10.788 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T02:51:10.816 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T02:51:11.856 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T02:51:11.893 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T02:51:12.857 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T02:51:12.894 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T02:51:12.994 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T02:51:13.156 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T02:51:13.179 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T02:51:13.203 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T02:51:13.203 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:51:13.203 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T02:51:13.203 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T02:51:13.203 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T02:51:13.204 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:51:13.255 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T02:51:13.266 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T02:51:13.365 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T02:51:13.445 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T02:51:14.370 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T02:51:14.399 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T02:51:14.423 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T02:51:14.423 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:51:14.423 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T02:51:14.423 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T02:51:14.423 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T02:51:14.423 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:51:14.466 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T02:51:14.488 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T02:51:14.488 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:51:14.489 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T02:51:14.489 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:51:14.533 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T02:51:14.556 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T02:51:14.556 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:51:14.557 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T02:51:14.557 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T02:51:14.557 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T02:51:14.557 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:51:14.859 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T02:51:14.882 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T02:51:14.882 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:51:14.882 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T02:51:14.882 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T02:51:14.882 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T02:51:14.882 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:51:19.429 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T02:51:19.471 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T02:51:19.511 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T02:51:19.552 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T02:51:19.579 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T02:51:19.780 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T02:51:19.828 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T02:51:19.856 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T02:51:19.885 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T02:51:19.932 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T02:51:20.552 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:51:20.596 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:51:20.616 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:51:21.447 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T02:51:21.526 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T02:51:21.574 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T02:51:21.664 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T02:51:21.695 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T02:51:21.727 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T02:51:21.763 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T02:51:21.763 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T02:51:21.847 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T02:51:21.848 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T02:51:21.931 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T02:51:21.956 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T02:51:21.962 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:51:21.962 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:51:22.141 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:51:22.141 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T02:51:22.226 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T02:51:22.290 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:51:28.244 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:51:28.244 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T02:51:28.244 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T02:51:28.244 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T02:51:28.244 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T02:51:28.244 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T02:51:28.244 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T02:51:28.245 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T02:51:28.245 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:51:28.245 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:51:28.245 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T02:51:28.245 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T02:51:28.245 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T02:51:28.245 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:51:28.245 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T02:51:28.245 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T02:51:28.245 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T02:51:28.245 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T02:51:28.245 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T02:51:28.246 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T02:51:28.246 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T02:51:28.246 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T02:51:28.246 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T02:51:28.246 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T02:51:28.246 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T02:51:28.246 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T02:51:28.246 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T02:51:28.246 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T02:51:28.246 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T02:51:28.246 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T02:51:28.246 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T02:51:28.246 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T02:51:28.247 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T02:51:28.247 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T02:51:28.247 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T02:51:28.247 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T02:51:28.247 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T02:51:28.247 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T02:51:28.247 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T02:51:28.247 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T02:51:28.247 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T02:51:28.247 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T02:51:28.247 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T02:51:28.247 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T02:51:28.248 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T02:51:28.248 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T02:51:28.248 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T02:51:28.248 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T02:51:28.248 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T02:51:28.248 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T02:51:28.248 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T02:51:28.248 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T02:51:28.248 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T02:51:28.248 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T02:51:28.248 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T02:51:28.248 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T02:51:28.249 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T02:51:28.249 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T02:51:28.249 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T02:51:28.249 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T02:51:28.249 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T02:51:28.249 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T02:51:28.249 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T02:51:28.250 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T02:51:28.250 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T02:51:28.250 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T02:51:28.250 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T02:51:28.250 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T02:51:28.250 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T02:51:28.250 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T02:51:28.250 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T02:51:28.250 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T02:51:28.251 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T02:51:28.251 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T02:51:28.251 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T02:51:28.251 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T02:51:28.251 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T02:51:28.251 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T02:51:28.251 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T02:51:28.251 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T02:51:28.251 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T02:51:28.251 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T02:51:28.251 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T02:51:28.252 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T02:51:28.252 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T02:51:28.252 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T02:51:28.252 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T02:51:28.252 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T02:51:28.252 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T02:51:28.252 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T02:51:28.252 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T02:51:28.252 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T02:51:28.252 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T02:51:28.252 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T02:51:28.252 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T02:51:28.253 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T02:51:28.253 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T02:51:28.253 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T02:51:28.253 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T02:51:28.253 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T02:51:28.253 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T02:51:28.253 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T02:51:28.253 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T02:51:28.253 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T02:51:28.254 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T02:51:28.254 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T02:51:28.254 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T02:51:28.254 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T02:51:28.254 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T02:51:28.254 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T02:51:28.254 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T02:51:28.254 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T02:51:28.254 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T02:51:28.254 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T02:51:28.254 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T02:51:28.255 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T02:51:28.255 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T02:51:28.255 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T02:51:28.255 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T02:51:28.255 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T02:51:28.255 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T02:51:28.255 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T02:51:28.255 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T02:51:28.255 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T02:51:28.255 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T02:51:28.256 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T02:51:28.256 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T02:51:28.256 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T02:51:28.256 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T02:51:28.256 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T02:51:28.256 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T02:51:28.256 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T02:51:28.256 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T02:51:28.256 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T02:51:28.256 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T02:51:28.256 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T02:51:28.256 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T02:51:28.256 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T02:51:28.257 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T02:51:28.257 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T02:51:28.257 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T02:51:28.257 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T02:51:28.257 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T02:51:28.257 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T02:51:28.257 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T02:51:28.257 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T02:51:28.257 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T02:51:28.257 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T02:51:28.257 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T02:51:28.257 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T02:51:28.257 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T02:51:28.258 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T02:51:28.258 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T02:51:28.258 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T02:51:28.258 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T02:51:28.258 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T02:51:28.258 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T02:51:28.258 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T02:51:28.258 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T02:51:28.258 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T02:51:28.258 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T02:51:28.258 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T02:51:28.258 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T02:51:28.258 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T02:51:30.680 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:51:30.680 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:51:30.680 INFO:teuthology.orchestra.run.smithi116.stdout:Upgraded: 2024-09-12T02:51:30.680 INFO:teuthology.orchestra.run.smithi116.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T02:51:30.680 INFO:teuthology.orchestra.run.smithi116.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T02:51:30.680 INFO:teuthology.orchestra.run.smithi116.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T02:51:30.680 INFO:teuthology.orchestra.run.smithi116.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T02:51:30.680 INFO:teuthology.orchestra.run.smithi116.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.681 INFO:teuthology.orchestra.run.smithi116.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.681 INFO:teuthology.orchestra.run.smithi116.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T02:51:30.681 INFO:teuthology.orchestra.run.smithi116.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T02:51:30.681 INFO:teuthology.orchestra.run.smithi116.stdout:Installed: 2024-09-12T02:51:30.681 INFO:teuthology.orchestra.run.smithi116.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T02:51:30.681 INFO:teuthology.orchestra.run.smithi116.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T02:51:30.681 INFO:teuthology.orchestra.run.smithi116.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T02:51:30.681 INFO:teuthology.orchestra.run.smithi116.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T02:51:30.681 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.681 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.681 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.681 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.682 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:51:30.682 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.682 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.682 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.682 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:51:30.682 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:51:30.682 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:51:30.682 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:51:30.682 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:51:30.682 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.682 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.682 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:51:30.682 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.683 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.683 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.683 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:51:30.683 INFO:teuthology.orchestra.run.smithi116.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:51:30.683 INFO:teuthology.orchestra.run.smithi116.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.683 INFO:teuthology.orchestra.run.smithi116.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.683 INFO:teuthology.orchestra.run.smithi116.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:51:30.683 INFO:teuthology.orchestra.run.smithi116.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T02:51:30.683 INFO:teuthology.orchestra.run.smithi116.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T02:51:30.683 INFO:teuthology.orchestra.run.smithi116.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T02:51:30.683 INFO:teuthology.orchestra.run.smithi116.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T02:51:30.683 INFO:teuthology.orchestra.run.smithi116.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T02:51:30.683 INFO:teuthology.orchestra.run.smithi116.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T02:51:30.684 INFO:teuthology.orchestra.run.smithi116.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T02:51:30.684 INFO:teuthology.orchestra.run.smithi116.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T02:51:30.684 INFO:teuthology.orchestra.run.smithi116.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T02:51:30.684 INFO:teuthology.orchestra.run.smithi116.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T02:51:30.684 INFO:teuthology.orchestra.run.smithi116.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T02:51:30.684 INFO:teuthology.orchestra.run.smithi116.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.684 INFO:teuthology.orchestra.run.smithi116.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.684 INFO:teuthology.orchestra.run.smithi116.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.684 INFO:teuthology.orchestra.run.smithi116.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T02:51:30.684 INFO:teuthology.orchestra.run.smithi116.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T02:51:30.684 INFO:teuthology.orchestra.run.smithi116.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T02:51:30.684 INFO:teuthology.orchestra.run.smithi116.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T02:51:30.685 INFO:teuthology.orchestra.run.smithi116.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T02:51:30.685 INFO:teuthology.orchestra.run.smithi116.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.685 INFO:teuthology.orchestra.run.smithi116.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.685 INFO:teuthology.orchestra.run.smithi116.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T02:51:30.685 INFO:teuthology.orchestra.run.smithi116.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.685 INFO:teuthology.orchestra.run.smithi116.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T02:51:30.685 INFO:teuthology.orchestra.run.smithi116.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T02:51:30.685 INFO:teuthology.orchestra.run.smithi116.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T02:51:30.685 INFO:teuthology.orchestra.run.smithi116.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T02:51:30.685 INFO:teuthology.orchestra.run.smithi116.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T02:51:30.685 INFO:teuthology.orchestra.run.smithi116.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T02:51:30.686 INFO:teuthology.orchestra.run.smithi116.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T02:51:30.686 INFO:teuthology.orchestra.run.smithi116.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T02:51:30.686 INFO:teuthology.orchestra.run.smithi116.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T02:51:30.686 INFO:teuthology.orchestra.run.smithi116.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T02:51:30.686 INFO:teuthology.orchestra.run.smithi116.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T02:51:30.686 INFO:teuthology.orchestra.run.smithi116.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T02:51:30.686 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T02:51:30.686 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T02:51:30.686 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T02:51:30.686 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T02:51:30.686 INFO:teuthology.orchestra.run.smithi116.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T02:51:30.686 INFO:teuthology.orchestra.run.smithi116.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T02:51:30.686 INFO:teuthology.orchestra.run.smithi116.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T02:51:30.687 INFO:teuthology.orchestra.run.smithi116.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T02:51:30.687 INFO:teuthology.orchestra.run.smithi116.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T02:51:30.687 INFO:teuthology.orchestra.run.smithi116.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T02:51:30.687 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T02:51:30.687 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.687 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.687 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.687 INFO:teuthology.orchestra.run.smithi116.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T02:51:30.687 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T02:51:30.687 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T02:51:30.687 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T02:51:30.687 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T02:51:30.687 INFO:teuthology.orchestra.run.smithi116.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T02:51:30.688 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T02:51:30.688 INFO:teuthology.orchestra.run.smithi116.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T02:51:30.688 INFO:teuthology.orchestra.run.smithi116.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T02:51:30.688 INFO:teuthology.orchestra.run.smithi116.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T02:51:30.688 INFO:teuthology.orchestra.run.smithi116.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T02:51:30.688 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T02:51:30.688 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T02:51:30.688 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T02:51:30.688 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T02:51:30.688 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T02:51:30.688 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T02:51:30.689 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T02:51:30.689 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T02:51:30.689 INFO:teuthology.orchestra.run.smithi116.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T02:51:30.689 INFO:teuthology.orchestra.run.smithi116.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T02:51:30.689 INFO:teuthology.orchestra.run.smithi116.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T02:51:30.689 INFO:teuthology.orchestra.run.smithi116.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T02:51:30.689 INFO:teuthology.orchestra.run.smithi116.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T02:51:30.689 INFO:teuthology.orchestra.run.smithi116.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T02:51:30.689 INFO:teuthology.orchestra.run.smithi116.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T02:51:30.689 INFO:teuthology.orchestra.run.smithi116.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T02:51:30.690 INFO:teuthology.orchestra.run.smithi116.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T02:51:30.690 INFO:teuthology.orchestra.run.smithi116.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T02:51:30.690 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T02:51:30.690 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T02:51:30.690 INFO:teuthology.orchestra.run.smithi116.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T02:51:30.690 INFO:teuthology.orchestra.run.smithi116.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T02:51:30.690 INFO:teuthology.orchestra.run.smithi116.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T02:51:30.690 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T02:51:30.690 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T02:51:30.690 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T02:51:30.690 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T02:51:30.691 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T02:51:30.691 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T02:51:30.691 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.691 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.691 INFO:teuthology.orchestra.run.smithi116.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T02:51:30.691 INFO:teuthology.orchestra.run.smithi116.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T02:51:30.691 INFO:teuthology.orchestra.run.smithi116.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T02:51:30.691 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.691 INFO:teuthology.orchestra.run.smithi116.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T02:51:30.691 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T02:51:30.691 INFO:teuthology.orchestra.run.smithi116.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T02:51:30.691 INFO:teuthology.orchestra.run.smithi116.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T02:51:30.691 INFO:teuthology.orchestra.run.smithi116.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T02:51:30.692 INFO:teuthology.orchestra.run.smithi116.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T02:51:30.692 INFO:teuthology.orchestra.run.smithi116.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T02:51:30.692 INFO:teuthology.orchestra.run.smithi116.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T02:51:30.692 INFO:teuthology.orchestra.run.smithi116.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T02:51:30.692 INFO:teuthology.orchestra.run.smithi116.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T02:51:30.692 INFO:teuthology.orchestra.run.smithi116.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T02:51:30.692 INFO:teuthology.orchestra.run.smithi116.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T02:51:30.692 INFO:teuthology.orchestra.run.smithi116.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T02:51:30.692 INFO:teuthology.orchestra.run.smithi116.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T02:51:30.692 INFO:teuthology.orchestra.run.smithi116.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T02:51:30.693 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.693 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.693 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:51:30.693 INFO:teuthology.orchestra.run.smithi116.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T02:51:30.693 INFO:teuthology.orchestra.run.smithi116.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T02:51:30.693 INFO:teuthology.orchestra.run.smithi116.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T02:51:30.693 INFO:teuthology.orchestra.run.smithi116.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T02:51:30.693 INFO:teuthology.orchestra.run.smithi116.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T02:51:30.693 INFO:teuthology.orchestra.run.smithi116.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T02:51:30.694 INFO:teuthology.orchestra.run.smithi116.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T02:51:30.694 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:51:30.694 INFO:teuthology.orchestra.run.smithi116.stdout:Complete! 2024-09-12T02:51:31.117 DEBUG:teuthology.parallel:result is None 2024-09-12T02:51:31.118 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:51:31.263 DEBUG:teuthology.orchestra.run.smithi116:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T02:51:31.286 INFO:teuthology.orchestra.run.smithi116.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T02:51:31.287 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T02:51:31.288 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T02:51:31.291 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:51:31.432 DEBUG:teuthology.orchestra.run.smithi177:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T02:51:31.459 INFO:teuthology.orchestra.run.smithi177.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T02:51:31.459 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T02:51:31.459 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T02:51:31.461 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:51:31.600 DEBUG:teuthology.orchestra.run.smithi183:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T02:51:31.627 INFO:teuthology.orchestra.run.smithi183.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T02:51:31.627 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T02:51:31.627 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T02:51:31.629 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-12T02:51:31.629 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-12T02:51:31.630 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T02:51:31.661 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-12T02:51:31.661 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T02:51:31.694 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-12T02:51:31.694 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T02:51:31.734 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-12T02:51:31.734 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-12T02:51:31.734 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T02:51:31.767 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T02:51:31.854 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-12T02:51:31.854 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T02:51:31.897 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T02:51:31.982 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-12T02:51:31.983 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T02:51:32.016 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T02:51:32.109 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-12T02:51:32.109 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-12T02:51:32.109 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T02:51:32.142 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T02:51:32.208 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-12T02:51:32.208 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T02:51:32.235 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T02:51:32.301 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-12T02:51:32.302 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T02:51:32.328 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T02:51:32.395 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-12T02:51:32.395 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-12T02:51:32.395 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T02:51:32.420 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T02:51:32.485 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-12T02:51:32.485 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T02:51:32.510 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T02:51:32.575 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-12T02:51:32.575 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T02:51:32.600 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T02:51:32.666 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-12T02:51:32.769 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'subvols': {'create': 2}} 2024-09-12T02:51:32.769 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:51:32.769 INFO:tasks.cephadm:Cluster fsid is eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T02:51:32.769 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-12T02:51:32.770 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.116', 'mon.b': '172.21.15.177', 'mon.c': '172.21.15.183'} 2024-09-12T02:51:32.770 INFO:tasks.cephadm:First mon is mon.a on smithi116 2024-09-12T02:51:32.770 INFO:tasks.cephadm:First mgr is x 2024-09-12T02:51:32.770 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-12T02:51:32.770 DEBUG:teuthology.orchestra.run.smithi116:> sudo hostname $(hostname -s) 2024-09-12T02:51:32.796 DEBUG:teuthology.orchestra.run.smithi177:> sudo hostname $(hostname -s) 2024-09-12T02:51:32.821 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname $(hostname -s) 2024-09-12T02:51:32.846 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-12T02:51:32.846 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:51:32.988 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56038/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-298-gb53b6d3e', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-298.gb53b6d3e'}, 'url': 'https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 17:24:58.599585', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-12T02:51:33.134 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref squid, sha1 b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:51:33.135 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-12T02:51:33.136 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-12T02:51:33.136 DEBUG:teuthology.orchestra.run.smithi116:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T02:51:33.530 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 02:51 /home/ubuntu/cephtest/cephadm 2024-09-12T02:51:33.530 DEBUG:teuthology.orchestra.run.smithi177:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T02:51:33.953 INFO:teuthology.orchestra.run.smithi177.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 02:51 /home/ubuntu/cephtest/cephadm 2024-09-12T02:51:33.953 DEBUG:teuthology.orchestra.run.smithi183:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T02:51:34.377 INFO:teuthology.orchestra.run.smithi183.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 02:51 /home/ubuntu/cephtest/cephadm 2024-09-12T02:51:34.378 DEBUG:teuthology.orchestra.run.smithi116:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T02:51:34.397 DEBUG:teuthology.orchestra.run.smithi177:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T02:51:34.417 DEBUG:teuthology.orchestra.run.smithi183:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T02:51:34.451 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc on all hosts... 2024-09-12T02:51:34.452 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T02:51:34.454 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T02:51:34.460 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T02:51:36.023 INFO:teuthology.orchestra.run.smithi177.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T02:51:36.024 INFO:teuthology.orchestra.run.smithi116.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T02:51:36.083 INFO:teuthology.orchestra.run.smithi183.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T02:52:41.958 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-09-12T02:52:41.958 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T02:52:41.959 INFO:teuthology.orchestra.run.smithi177.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T02:52:41.959 INFO:teuthology.orchestra.run.smithi177.stdout: "repo_digests": [ 2024-09-12T02:52:41.959 INFO:teuthology.orchestra.run.smithi177.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T02:52:41.959 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-09-12T02:52:41.959 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-09-12T02:52:55.693 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-12T02:52:55.693 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T02:52:55.693 INFO:teuthology.orchestra.run.smithi183.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T02:52:55.693 INFO:teuthology.orchestra.run.smithi183.stdout: "repo_digests": [ 2024-09-12T02:52:55.693 INFO:teuthology.orchestra.run.smithi183.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T02:52:55.694 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-12T02:52:55.694 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-12T02:52:57.737 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T02:52:57.737 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T02:52:57.737 INFO:teuthology.orchestra.run.smithi116.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T02:52:57.737 INFO:teuthology.orchestra.run.smithi116.stdout: "repo_digests": [ 2024-09-12T02:52:57.737 INFO:teuthology.orchestra.run.smithi116.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T02:52:57.737 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2024-09-12T02:52:57.737 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T02:52:57.761 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph 2024-09-12T02:52:57.823 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /etc/ceph 2024-09-12T02:52:57.856 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /etc/ceph 2024-09-12T02:52:57.889 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 777 /etc/ceph 2024-09-12T02:52:57.914 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 777 /etc/ceph 2024-09-12T02:52:57.943 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 777 /etc/ceph 2024-09-12T02:52:57.972 INFO:tasks.cephadm:Writing seed config... 2024-09-12T02:52:57.974 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-12T02:52:57.974 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-12T02:52:57.974 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-12T02:52:57.974 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-12T02:52:57.974 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-12T02:52:57.974 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-12T02:52:57.974 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-12T02:52:57.974 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-12T02:52:57.974 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-12T02:52:57.974 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-12T02:52:57.974 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-12T02:52:57.974 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-12T02:52:57.974 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-12T02:52:57.975 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-12T02:52:57.975 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-12T02:52:57.975 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-12T02:52:57.975 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-12T02:52:57.975 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-12T02:52:57.975 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-12T02:52:57.975 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-12T02:52:57.976 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-12T02:52:57.976 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-12T02:52:57.976 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-12T02:52:57.976 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-12T02:52:57.976 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-12T02:52:57.976 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-12T02:52:57.976 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-12T02:52:57.976 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-12T02:52:57.976 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-12T02:52:57.976 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-12T02:52:57.976 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-12T02:52:57.976 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-12T02:52:57.976 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-12T02:52:57.977 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-12T02:52:57.977 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-12T02:52:57.977 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-12T02:52:57.977 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-12T02:52:57.977 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-12T02:52:57.977 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-12T02:52:57.977 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-12T02:52:57.977 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-12T02:52:57.977 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-12T02:52:57.977 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-12T02:52:57.977 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-12T02:52:57.977 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-12T02:52:57.977 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-12T02:52:57.977 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-12T02:52:57.978 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-12T02:52:57.978 DEBUG:teuthology.orchestra.run.smithi116:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-12T02:52:57.994 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = eb1ac602-70b1-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-12T02:52:57.994 DEBUG:teuthology.orchestra.run.smithi116:mon.a> sudo journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mon.a.service 2024-09-12T02:52:58.037 DEBUG:teuthology.orchestra.run.smithi116:mgr.x> sudo journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mgr.x.service 2024-09-12T02:52:58.080 INFO:tasks.cephadm:Bootstrapping... 2024-09-12T02:52:58.080 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc -v bootstrap --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.116 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:52:58.280 INFO:teuthology.orchestra.run.smithi116.stdout:-------------------------------------------------------------------------------- 2024-09-12T02:52:58.280 INFO:teuthology.orchestra.run.smithi116.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', '-v', 'bootstrap', '--fsid', 'eb1ac602-70b1-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.116', '--skip-admin-label'] 2024-09-12T02:52:58.281 INFO:teuthology.orchestra.run.smithi116.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-12T02:52:58.281 INFO:teuthology.orchestra.run.smithi116.stdout:Verifying podman|docker is present... 2024-09-12T02:52:58.305 INFO:teuthology.orchestra.run.smithi116.stdout:/bin/podman: stdout 5.2.2 2024-09-12T02:52:58.305 INFO:teuthology.orchestra.run.smithi116.stdout:Verifying lvm2 is present... 2024-09-12T02:52:58.305 INFO:teuthology.orchestra.run.smithi116.stdout:Verifying time synchronization is in place... 2024-09-12T02:52:58.313 INFO:teuthology.orchestra.run.smithi116.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-12T02:52:58.313 INFO:teuthology.orchestra.run.smithi116.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-12T02:52:58.319 INFO:teuthology.orchestra.run.smithi116.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-12T02:52:58.320 INFO:teuthology.orchestra.run.smithi116.stdout:systemctl: stdout inactive 2024-09-12T02:52:58.326 INFO:teuthology.orchestra.run.smithi116.stdout:systemctl: stdout enabled 2024-09-12T02:52:58.333 INFO:teuthology.orchestra.run.smithi116.stdout:systemctl: stdout active 2024-09-12T02:52:58.333 INFO:teuthology.orchestra.run.smithi116.stdout:Unit chronyd.service is enabled and running 2024-09-12T02:52:58.333 INFO:teuthology.orchestra.run.smithi116.stdout:Repeating the final host check... 2024-09-12T02:52:58.355 INFO:teuthology.orchestra.run.smithi116.stdout:/bin/podman: stdout 5.2.2 2024-09-12T02:52:58.355 INFO:teuthology.orchestra.run.smithi116.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-12T02:52:58.355 INFO:teuthology.orchestra.run.smithi116.stdout:systemctl is present 2024-09-12T02:52:58.355 INFO:teuthology.orchestra.run.smithi116.stdout:lvcreate is present 2024-09-12T02:52:58.362 INFO:teuthology.orchestra.run.smithi116.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-12T02:52:58.362 INFO:teuthology.orchestra.run.smithi116.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-12T02:52:58.368 INFO:teuthology.orchestra.run.smithi116.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-12T02:52:58.368 INFO:teuthology.orchestra.run.smithi116.stdout:systemctl: stdout inactive 2024-09-12T02:52:58.375 INFO:teuthology.orchestra.run.smithi116.stdout:systemctl: stdout enabled 2024-09-12T02:52:58.382 INFO:teuthology.orchestra.run.smithi116.stdout:systemctl: stdout active 2024-09-12T02:52:58.382 INFO:teuthology.orchestra.run.smithi116.stdout:Unit chronyd.service is enabled and running 2024-09-12T02:52:58.382 INFO:teuthology.orchestra.run.smithi116.stdout:Host looks OK 2024-09-12T02:52:58.382 INFO:teuthology.orchestra.run.smithi116.stdout:Cluster fsid: eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T02:52:58.382 INFO:teuthology.orchestra.run.smithi116.stdout:Acquiring lock 140413491540992 on /run/cephadm/eb1ac602-70b1-11ef-bcea-c7b262605968.lock 2024-09-12T02:52:58.382 INFO:teuthology.orchestra.run.smithi116.stdout:Lock 140413491540992 acquired on /run/cephadm/eb1ac602-70b1-11ef-bcea-c7b262605968.lock 2024-09-12T02:52:58.382 INFO:teuthology.orchestra.run.smithi116.stdout:Verifying IP 172.21.15.116 port 3300 ... 2024-09-12T02:52:58.383 INFO:teuthology.orchestra.run.smithi116.stdout:Verifying IP 172.21.15.116 port 6789 ... 2024-09-12T02:52:58.383 INFO:teuthology.orchestra.run.smithi116.stdout:Base mon IP(s) is [172.21.15.116:3300, 172.21.15.116:6789], mon addrv is [v2:172.21.15.116:3300,v1:172.21.15.116:6789] 2024-09-12T02:52:58.386 INFO:teuthology.orchestra.run.smithi116.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.116 metric 100 2024-09-12T02:52:58.386 INFO:teuthology.orchestra.run.smithi116.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.116 metric 100 2024-09-12T02:52:58.388 INFO:teuthology.orchestra.run.smithi116.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-12T02:52:58.388 INFO:teuthology.orchestra.run.smithi116.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-12T02:52:58.389 INFO:teuthology.orchestra.run.smithi116.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-12T02:52:58.391 INFO:teuthology.orchestra.run.smithi116.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-12T02:52:58.391 INFO:teuthology.orchestra.run.smithi116.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-12T02:52:58.391 INFO:teuthology.orchestra.run.smithi116.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-12T02:52:58.391 INFO:teuthology.orchestra.run.smithi116.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-12T02:52:58.391 INFO:teuthology.orchestra.run.smithi116.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link noprefixroute 2024-09-12T02:52:58.391 INFO:teuthology.orchestra.run.smithi116.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-12T02:52:58.392 INFO:teuthology.orchestra.run.smithi116.stdout:Mon IP `172.21.15.116` is in CIDR network `172.21.0.0/20` 2024-09-12T02:52:58.392 INFO:teuthology.orchestra.run.smithi116.stdout:Mon IP `172.21.15.116` is in CIDR network `172.21.0.0/20` 2024-09-12T02:52:58.392 INFO:teuthology.orchestra.run.smithi116.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-12T02:52:58.392 INFO:teuthology.orchestra.run.smithi116.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-12T02:52:58.393 INFO:teuthology.orchestra.run.smithi116.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T02:52:58.865 INFO:teuthology.orchestra.run.smithi116.stdout:/bin/podman: stdout 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-12T02:52:58.865 INFO:teuthology.orchestra.run.smithi116.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T02:52:58.865 INFO:teuthology.orchestra.run.smithi116.stdout:/bin/podman: stderr Getting image source signatures 2024-09-12T02:52:58.865 INFO:teuthology.orchestra.run.smithi116.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-12T02:52:58.865 INFO:teuthology.orchestra.run.smithi116.stdout:/bin/podman: stderr Copying blob sha256:60180723200353b4e13b39fd162aecaf1bc097c859c7c6537324229e97c78567 2024-09-12T02:52:58.865 INFO:teuthology.orchestra.run.smithi116.stdout:/bin/podman: stderr Copying config sha256:54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-12T02:52:58.865 INFO:teuthology.orchestra.run.smithi116.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-12T02:52:59.844 INFO:teuthology.orchestra.run.smithi116.stdout:ceph: stdout ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-12T02:52:59.844 INFO:teuthology.orchestra.run.smithi116.stdout:Ceph version: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-12T02:52:59.844 INFO:teuthology.orchestra.run.smithi116.stdout:Extracting ceph user uid/gid from container image... 2024-09-12T02:53:00.833 INFO:teuthology.orchestra.run.smithi116.stdout:stat: stdout 167 167 2024-09-12T02:53:00.834 INFO:teuthology.orchestra.run.smithi116.stdout:Creating initial keys... 2024-09-12T02:53:01.748 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph-authtool: stdout AQCNV+JmXbPaCRAAjpRtlsI1oXTpY8WpZk1hhQ== 2024-09-12T02:53:02.795 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph-authtool: stdout AQCOV+Jm11NkBBAACq6ikvgU2Q334TD6HNmdWw== 2024-09-12T02:53:03.818 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph-authtool: stdout AQCPV+JmBws1CBAAxBffIQwgi6O/6i21WxbPfQ== 2024-09-12T02:53:03.818 INFO:teuthology.orchestra.run.smithi116.stdout:Creating initial monmap... 2024-09-12T02:53:04.732 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-12T02:53:04.733 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-12T02:53:04.733 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T02:53:04.733 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-12T02:53:04.733 INFO:teuthology.orchestra.run.smithi116.stdout:monmaptool for a [v2:172.21.15.116:3300,v1:172.21.15.116:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-12T02:53:04.733 INFO:teuthology.orchestra.run.smithi116.stdout:setting min_mon_release = quincy 2024-09-12T02:53:04.733 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/monmaptool: set fsid to eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T02:53:04.733 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-12T02:53:04.733 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:53:04.733 INFO:teuthology.orchestra.run.smithi116.stdout:Creating mon... 2024-09-12T02:53:05.696 INFO:teuthology.orchestra.run.smithi116.stdout:create mon.a on 2024-09-12T02:53:05.906 INFO:teuthology.orchestra.run.smithi116.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-12T02:53:06.049 INFO:teuthology.orchestra.run.smithi116.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-12T02:53:06.242 INFO:teuthology.orchestra.run.smithi116.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-eb1ac602-70b1-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-eb1ac602-70b1-11ef-bcea-c7b262605968.target. 2024-09-12T02:53:06.242 INFO:teuthology.orchestra.run.smithi116.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-eb1ac602-70b1-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-eb1ac602-70b1-11ef-bcea-c7b262605968.target. 2024-09-12T02:53:06.467 INFO:teuthology.orchestra.run.smithi116.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mon.a 2024-09-12T02:53:06.468 INFO:teuthology.orchestra.run.smithi116.stdout:systemctl: stderr Failed to reset failed state of unit ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-12T02:53:06.628 INFO:teuthology.orchestra.run.smithi116.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-eb1ac602-70b1-11ef-bcea-c7b262605968.target.wants/ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@.service. 2024-09-12T02:53:07.152 INFO:teuthology.orchestra.run.smithi116.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T02:53:07.153 INFO:teuthology.orchestra.run.smithi116.stdout:systemctl: stdout disabled 2024-09-12T02:53:07.160 INFO:teuthology.orchestra.run.smithi116.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T02:53:07.160 INFO:teuthology.orchestra.run.smithi116.stdout:systemctl: stdout inactive 2024-09-12T02:53:07.160 INFO:teuthology.orchestra.run.smithi116.stdout:firewalld.service is not enabled 2024-09-12T02:53:07.160 INFO:teuthology.orchestra.run.smithi116.stdout:Not possible to enable service . firewalld.service is not available 2024-09-12T02:53:07.160 INFO:teuthology.orchestra.run.smithi116.stdout:Waiting for mon to start... 2024-09-12T02:53:07.160 INFO:teuthology.orchestra.run.smithi116.stdout:Waiting for mon... 2024-09-12T02:53:08.211 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:08 smithi116 ceph-mon[31042]: from='client.? 172.21.15.116:0/2180586562' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-12T02:53:08.214 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout cluster: 2024-09-12T02:53:08.220 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout id: eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T02:53:08.220 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-12T02:53:08.221 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout 2024-09-12T02:53:08.221 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout services: 2024-09-12T02:53:08.221 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.531799s) 2024-09-12T02:53:08.221 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-12T02:53:08.221 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-12T02:53:08.221 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout 2024-09-12T02:53:08.221 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout data: 2024-09-12T02:53:08.221 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-12T02:53:08.221 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-12T02:53:08.221 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-12T02:53:08.221 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout pgs: 2024-09-12T02:53:08.222 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout 2024-09-12T02:53:08.222 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.567+0000 7f362a067640 1 Processor -- start 2024-09-12T02:53:08.222 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.568+0000 7f362a067640 1 -- start start 2024-09-12T02:53:08.222 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.568+0000 7f362a067640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3624107680 0x7f3624107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:08.222 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.568+0000 7f362a067640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3624108050 con 0x7f3624107680 2024-09-12T02:53:08.222 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.568+0000 7f36237fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3624107680 0x7f3624107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:08.222 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.568+0000 7f36237fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3624107680 0x7f3624107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38932/0 (socket says 172.21.15.116:38932) 2024-09-12T02:53:08.222 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.568+0000 7f36237fe640 1 -- 172.21.15.116:0/3113382691 learned_addr learned my addr 172.21.15.116:0/3113382691 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:08.222 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.568+0000 7f36237fe640 1 -- 172.21.15.116:0/3113382691 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3624108830 con 0x7f3624107680 2024-09-12T02:53:08.222 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.570+0000 7f36237fe640 1 --2- 172.21.15.116:0/3113382691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3624107680 0x7f3624107a80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3614009820 tx=0x7f361402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92221bb032097250 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:08.222 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.571+0000 7f3622ffd640 1 -- 172.21.15.116:0/3113382691 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3614037070 con 0x7f3624107680 2024-09-12T02:53:08.222 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.571+0000 7f3622ffd640 1 -- 172.21.15.116:0/3113382691 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3614032440 con 0x7f3624107680 2024-09-12T02:53:08.222 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.571+0000 7f3622ffd640 1 -- 172.21.15.116:0/3113382691 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3614032740 con 0x7f3624107680 2024-09-12T02:53:08.223 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.571+0000 7f362a067640 1 -- 172.21.15.116:0/3113382691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3624107680 msgr2=0x7f3624107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:08.223 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.571+0000 7f362a067640 1 --2- 172.21.15.116:0/3113382691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3624107680 0x7f3624107a80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3614009820 tx=0x7f361402a270 comp rx=0 tx=0).stop 2024-09-12T02:53:08.223 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.571+0000 7f362a067640 1 -- 172.21.15.116:0/3113382691 shutdown_connections 2024-09-12T02:53:08.223 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.571+0000 7f362a067640 1 --2- 172.21.15.116:0/3113382691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3624107680 0x7f3624107a80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:08.223 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.571+0000 7f362a067640 1 -- 172.21.15.116:0/3113382691 >> 172.21.15.116:0/3113382691 conn(0x7f3624102e50 msgr2=0x7f3624105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:08.223 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.571+0000 7f362a067640 1 -- 172.21.15.116:0/3113382691 shutdown_connections 2024-09-12T02:53:08.223 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.571+0000 7f362a067640 1 -- 172.21.15.116:0/3113382691 wait complete. 2024-09-12T02:53:08.223 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.572+0000 7f362a067640 1 Processor -- start 2024-09-12T02:53:08.223 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.572+0000 7f362a067640 1 -- start start 2024-09-12T02:53:08.223 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.572+0000 7f362a067640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3624107680 0x7f362419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:08.223 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.572+0000 7f362a067640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3624108dd0 con 0x7f3624107680 2024-09-12T02:53:08.223 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.572+0000 7f36237fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3624107680 0x7f362419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:08.224 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.572+0000 7f36237fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3624107680 0x7f362419adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38938/0 (socket says 172.21.15.116:38938) 2024-09-12T02:53:08.224 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.572+0000 7f36237fe640 1 -- 172.21.15.116:0/2180586562 learned_addr learned my addr 172.21.15.116:0/2180586562 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:08.224 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.572+0000 7f36237fe640 1 -- 172.21.15.116:0/2180586562 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f362419b330 con 0x7f3624107680 2024-09-12T02:53:08.224 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.573+0000 7f36237fe640 1 --2- 172.21.15.116:0/2180586562 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3624107680 0x7f362419adf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f36140090b0 tx=0x7f36140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:08.224 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.573+0000 7f36217fa640 1 -- 172.21.15.116:0/2180586562 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3614041070 con 0x7f3624107680 2024-09-12T02:53:08.224 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.573+0000 7f36217fa640 1 -- 172.21.15.116:0/2180586562 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f361403c7a0 con 0x7f3624107680 2024-09-12T02:53:08.224 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.573+0000 7f362a067640 1 -- 172.21.15.116:0/2180586562 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f362419b5c0 con 0x7f3624107680 2024-09-12T02:53:08.224 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.573+0000 7f36217fa640 1 -- 172.21.15.116:0/2180586562 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3614037040 con 0x7f3624107680 2024-09-12T02:53:08.224 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.573+0000 7f362a067640 1 -- 172.21.15.116:0/2180586562 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f362419e2b0 con 0x7f3624107680 2024-09-12T02:53:08.224 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.573+0000 7f36217fa640 1 -- 172.21.15.116:0/2180586562 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3614044070 con 0x7f3624107680 2024-09-12T02:53:08.224 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.573+0000 7f36217fa640 1 -- 172.21.15.116:0/2180586562 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f361403d3f0 con 0x7f3624107680 2024-09-12T02:53:08.224 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.574+0000 7f362a067640 1 -- 172.21.15.116:0/2180586562 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3624109420 con 0x7f3624107680 2024-09-12T02:53:08.225 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.576+0000 7f36217fa640 1 -- 172.21.15.116:0/2180586562 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f3614046450 con 0x7f3624107680 2024-09-12T02:53:08.225 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.635+0000 7f362a067640 1 -- 172.21.15.116:0/2180586562 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f362419e810 con 0x7f3624107680 2024-09-12T02:53:08.225 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.635+0000 7f36217fa640 1 -- 172.21.15.116:0/2180586562 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f3614046630 con 0x7f3624107680 2024-09-12T02:53:08.225 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.636+0000 7f362a067640 1 -- 172.21.15.116:0/2180586562 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3624107680 msgr2=0x7f362419adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:08.225 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.636+0000 7f362a067640 1 --2- 172.21.15.116:0/2180586562 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3624107680 0x7f362419adf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f36140090b0 tx=0x7f36140046c0 comp rx=0 tx=0).stop 2024-09-12T02:53:08.225 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.636+0000 7f362a067640 1 -- 172.21.15.116:0/2180586562 shutdown_connections 2024-09-12T02:53:08.225 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.636+0000 7f362a067640 1 --2- 172.21.15.116:0/2180586562 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3624107680 0x7f362419adf0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:08.225 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.636+0000 7f362a067640 1 -- 172.21.15.116:0/2180586562 >> 172.21.15.116:0/2180586562 conn(0x7f3624102e50 msgr2=0x7f362418f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:08.225 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.636+0000 7f362a067640 1 -- 172.21.15.116:0/2180586562 shutdown_connections 2024-09-12T02:53:08.225 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:07.636+0000 7f362a067640 1 -- 172.21.15.116:0/2180586562 wait complete. 2024-09-12T02:53:08.225 INFO:teuthology.orchestra.run.smithi116.stdout:mon is available 2024-09-12T02:53:08.225 INFO:teuthology.orchestra.run.smithi116.stdout:Assimilating anything we can from ceph.conf... 2024-09-12T02:53:09.252 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:09 smithi116 ceph-mon[31042]: from='client.? 172.21.15.116:0/1051569697' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-12T02:53:09.252 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:09 smithi116 ceph-mon[31042]: from='client.? 172.21.15.116:0/1051569697' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-12T02:53:09.261 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout 2024-09-12T02:53:09.261 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout [global] 2024-09-12T02:53:09.261 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout fsid = eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T02:53:09.261 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.116:3300,v1:172.21.15.116:6789] 2024-09-12T02:53:09.261 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-12T02:53:09.261 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-12T02:53:09.261 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-12T02:53:09.261 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-12T02:53:09.261 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout 2024-09-12T02:53:09.261 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-12T02:53:09.261 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-12T02:53:09.261 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout 2024-09-12T02:53:09.262 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout [osd] 2024-09-12T02:53:09.262 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-12T02:53:09.262 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-12T02:53:09.262 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-12T02:53:09.262 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-12T02:53:09.262 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-12T02:53:09.262 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.634+0000 7f9af1205640 1 Processor -- start 2024-09-12T02:53:09.262 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.634+0000 7f9af1205640 1 -- start start 2024-09-12T02:53:09.262 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.634+0000 7f9af1205640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9aec106520 0x7f9aec106920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:09.262 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.634+0000 7f9af1205640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9aec106ef0 con 0x7f9aec106520 2024-09-12T02:53:09.262 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.634+0000 7f9aead9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9aec106520 0x7f9aec106920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:09.262 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.635+0000 7f9aead9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9aec106520 0x7f9aec106920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38954/0 (socket says 172.21.15.116:38954) 2024-09-12T02:53:09.262 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.635+0000 7f9aead9d640 1 -- 172.21.15.116:0/21885280 learned_addr learned my addr 172.21.15.116:0/21885280 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:09.262 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.635+0000 7f9aead9d640 1 -- 172.21.15.116:0/21885280 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aec107720 con 0x7f9aec106520 2024-09-12T02:53:09.263 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.635+0000 7f9aead9d640 1 --2- 172.21.15.116:0/21885280 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9aec106520 0x7f9aec106920 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4009e80 tx=0x7f9ae402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c8cfe79a72449c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:09.263 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.636+0000 7f9ae9d9b640 1 -- 172.21.15.116:0/21885280 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ae4036070 con 0x7f9aec106520 2024-09-12T02:53:09.263 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.636+0000 7f9ae9d9b640 1 -- 172.21.15.116:0/21885280 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ae4031400 con 0x7f9aec106520 2024-09-12T02:53:09.263 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.636+0000 7f9ae9d9b640 1 -- 172.21.15.116:0/21885280 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ae4031700 con 0x7f9aec106520 2024-09-12T02:53:09.263 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.636+0000 7f9af1205640 1 -- 172.21.15.116:0/21885280 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9aec106520 msgr2=0x7f9aec106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:09.263 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.636+0000 7f9af1205640 1 --2- 172.21.15.116:0/21885280 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9aec106520 0x7f9aec106920 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4009e80 tx=0x7f9ae402a7a0 comp rx=0 tx=0).stop 2024-09-12T02:53:09.263 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.636+0000 7f9af1205640 1 -- 172.21.15.116:0/21885280 shutdown_connections 2024-09-12T02:53:09.263 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.636+0000 7f9af1205640 1 --2- 172.21.15.116:0/21885280 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9aec106520 0x7f9aec106920 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:09.263 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.636+0000 7f9af1205640 1 -- 172.21.15.116:0/21885280 >> 172.21.15.116:0/21885280 conn(0x7f9aec101db0 msgr2=0x7f9aec1041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:09.263 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.636+0000 7f9af1205640 1 -- 172.21.15.116:0/21885280 shutdown_connections 2024-09-12T02:53:09.263 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.636+0000 7f9af1205640 1 -- 172.21.15.116:0/21885280 wait complete. 2024-09-12T02:53:09.263 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.637+0000 7f9af1205640 1 Processor -- start 2024-09-12T02:53:09.263 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.637+0000 7f9af1205640 1 -- start start 2024-09-12T02:53:09.264 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.637+0000 7f9af1205640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9aec106520 0x7f9aec18f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:09.264 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.637+0000 7f9af1205640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9aec107cc0 con 0x7f9aec106520 2024-09-12T02:53:09.264 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.637+0000 7f9aead9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9aec106520 0x7f9aec18f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:09.264 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.637+0000 7f9aead9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9aec106520 0x7f9aec18f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38958/0 (socket says 172.21.15.116:38958) 2024-09-12T02:53:09.264 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.637+0000 7f9aead9d640 1 -- 172.21.15.116:0/1051569697 learned_addr learned my addr 172.21.15.116:0/1051569697 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:09.264 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.637+0000 7f9aead9d640 1 -- 172.21.15.116:0/1051569697 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aec18f620 con 0x7f9aec106520 2024-09-12T02:53:09.264 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.638+0000 7f9aead9d640 1 --2- 172.21.15.116:0/1051569697 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9aec106520 0x7f9aec18f0e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4034040 tx=0x7f9ae4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:09.264 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.638+0000 7f9ae3fff640 1 -- 172.21.15.116:0/1051569697 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ae4041070 con 0x7f9aec106520 2024-09-12T02:53:09.264 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.638+0000 7f9ae3fff640 1 -- 172.21.15.116:0/1051569697 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ae403c650 con 0x7f9aec106520 2024-09-12T02:53:09.264 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.638+0000 7f9af1205640 1 -- 172.21.15.116:0/1051569697 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aec18d810 con 0x7f9aec106520 2024-09-12T02:53:09.264 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.638+0000 7f9ae3fff640 1 -- 172.21.15.116:0/1051569697 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ae4036040 con 0x7f9aec106520 2024-09-12T02:53:09.264 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.638+0000 7f9af1205640 1 -- 172.21.15.116:0/1051569697 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aec18dd50 con 0x7f9aec106520 2024-09-12T02:53:09.265 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.638+0000 7f9ae3fff640 1 -- 172.21.15.116:0/1051569697 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9ae4044070 con 0x7f9aec106520 2024-09-12T02:53:09.265 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.638+0000 7f9ae3fff640 1 -- 172.21.15.116:0/1051569697 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9ae4046c10 con 0x7f9aec106520 2024-09-12T02:53:09.265 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.639+0000 7f9af1205640 1 -- 172.21.15.116:0/1051569697 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aec108310 con 0x7f9aec106520 2024-09-12T02:53:09.265 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.641+0000 7f9ae3fff640 1 -- 172.21.15.116:0/1051569697 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f9ae403c950 con 0x7f9aec106520 2024-09-12T02:53:09.265 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.699+0000 7f9af1205640 1 -- 172.21.15.116:0/1051569697 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f9aec1946e0 con 0x7f9aec106520 2024-09-12T02:53:09.265 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.702+0000 7f9ae3fff640 1 -- 172.21.15.116:0/1051569697 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+431 (secure 0 0 0) 0x7f9ae403cb30 con 0x7f9aec106520 2024-09-12T02:53:09.265 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.703+0000 7f9ae3fff640 1 -- 172.21.15.116:0/1051569697 <== mon.0 v2:172.21.15.116:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9ae4056070 con 0x7f9aec106520 2024-09-12T02:53:09.265 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.704+0000 7f9af1205640 1 -- 172.21.15.116:0/1051569697 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9aec106520 msgr2=0x7f9aec18f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:09.265 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.704+0000 7f9af1205640 1 --2- 172.21.15.116:0/1051569697 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9aec106520 0x7f9aec18f0e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4034040 tx=0x7f9ae4004a40 comp rx=0 tx=0).stop 2024-09-12T02:53:09.265 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.704+0000 7f9af1205640 1 -- 172.21.15.116:0/1051569697 shutdown_connections 2024-09-12T02:53:09.265 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.704+0000 7f9af1205640 1 --2- 172.21.15.116:0/1051569697 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9aec106520 0x7f9aec18f0e0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:09.265 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.704+0000 7f9af1205640 1 -- 172.21.15.116:0/1051569697 >> 172.21.15.116:0/1051569697 conn(0x7f9aec101db0 msgr2=0x7f9aec191aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:09.266 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.704+0000 7f9af1205640 1 -- 172.21.15.116:0/1051569697 shutdown_connections 2024-09-12T02:53:09.266 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:08.704+0000 7f9af1205640 1 -- 172.21.15.116:0/1051569697 wait complete. 2024-09-12T02:53:09.266 INFO:teuthology.orchestra.run.smithi116.stdout:Generating new minimal ceph.conf... 2024-09-12T02:53:10.393 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:10 smithi116 ceph-mon[31042]: from='client.? 172.21.15.116:0/3841251706' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:10.399 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.680+0000 7fbf7f2b9640 1 Processor -- start 2024-09-12T02:53:10.399 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.680+0000 7fbf7f2b9640 1 -- start start 2024-09-12T02:53:10.399 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.680+0000 7fbf7f2b9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf7807de30 0x7fbf7807c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:10.399 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.680+0000 7fbf7f2b9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7807c820 con 0x7fbf7807de30 2024-09-12T02:53:10.399 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.680+0000 7fbf7d055640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf7807de30 0x7fbf7807c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:10.399 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.680+0000 7fbf7d055640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf7807de30 0x7fbf7807c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38970/0 (socket says 172.21.15.116:38970) 2024-09-12T02:53:10.399 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.680+0000 7fbf7d055640 1 -- 172.21.15.116:0/671427705 learned_addr learned my addr 172.21.15.116:0/671427705 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:10.399 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.680+0000 7fbf7d055640 1 -- 172.21.15.116:0/671427705 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7807c9a0 con 0x7fbf7807de30 2024-09-12T02:53:10.400 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.681+0000 7fbf7d055640 1 --2- 172.21.15.116:0/671427705 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf7807de30 0x7fbf7807c2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009820 tx=0x7fbf7402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ce6cd1855cfcd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:10.400 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.681+0000 7fbf6ffff640 1 -- 172.21.15.116:0/671427705 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf74037070 con 0x7fbf7807de30 2024-09-12T02:53:10.400 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.681+0000 7fbf6ffff640 1 -- 172.21.15.116:0/671427705 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbf74032900 con 0x7fbf7807de30 2024-09-12T02:53:10.400 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.681+0000 7fbf6ffff640 1 -- 172.21.15.116:0/671427705 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf74030850 con 0x7fbf7807de30 2024-09-12T02:53:10.400 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.682+0000 7fbf7f2b9640 1 -- 172.21.15.116:0/671427705 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf7807de30 msgr2=0x7fbf7807c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:10.400 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.682+0000 7fbf7f2b9640 1 --2- 172.21.15.116:0/671427705 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf7807de30 0x7fbf7807c2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009820 tx=0x7fbf7402a270 comp rx=0 tx=0).stop 2024-09-12T02:53:10.400 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.682+0000 7fbf7f2b9640 1 -- 172.21.15.116:0/671427705 shutdown_connections 2024-09-12T02:53:10.400 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.682+0000 7fbf7f2b9640 1 --2- 172.21.15.116:0/671427705 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf7807de30 0x7fbf7807c2e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:10.400 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.682+0000 7fbf7f2b9640 1 -- 172.21.15.116:0/671427705 >> 172.21.15.116:0/671427705 conn(0x7fbf78102e50 msgr2=0x7fbf78105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:10.400 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.682+0000 7fbf7f2b9640 1 -- 172.21.15.116:0/671427705 shutdown_connections 2024-09-12T02:53:10.400 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.682+0000 7fbf7f2b9640 1 -- 172.21.15.116:0/671427705 wait complete. 2024-09-12T02:53:10.400 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.683+0000 7fbf7f2b9640 1 Processor -- start 2024-09-12T02:53:10.401 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.683+0000 7fbf7f2b9640 1 -- start start 2024-09-12T02:53:10.401 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.683+0000 7fbf7f2b9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf7807de30 0x7fbf7819d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:10.401 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.683+0000 7fbf7f2b9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7810dcb0 con 0x7fbf7807de30 2024-09-12T02:53:10.401 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.683+0000 7fbf7d055640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf7807de30 0x7fbf7819d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:10.401 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.683+0000 7fbf7d055640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf7807de30 0x7fbf7819d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38980/0 (socket says 172.21.15.116:38980) 2024-09-12T02:53:10.401 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.683+0000 7fbf7d055640 1 -- 172.21.15.116:0/3841251706 learned_addr learned my addr 172.21.15.116:0/3841251706 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:10.401 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.683+0000 7fbf7d055640 1 -- 172.21.15.116:0/3841251706 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7819ddc0 con 0x7fbf7807de30 2024-09-12T02:53:10.401 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.683+0000 7fbf7d055640 1 --2- 172.21.15.116:0/3841251706 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf7807de30 0x7fbf7819d880 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbf74031000 tx=0x7fbf740326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:10.401 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.683+0000 7fbf6e7fc640 1 -- 172.21.15.116:0/3841251706 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf74042070 con 0x7fbf7807de30 2024-09-12T02:53:10.401 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.683+0000 7fbf6e7fc640 1 -- 172.21.15.116:0/3841251706 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbf74032be0 con 0x7fbf7807de30 2024-09-12T02:53:10.401 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.683+0000 7fbf7f2b9640 1 -- 172.21.15.116:0/3841251706 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf7819e050 con 0x7fbf7807de30 2024-09-12T02:53:10.401 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.683+0000 7fbf6e7fc640 1 -- 172.21.15.116:0/3841251706 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf74037040 con 0x7fbf7807de30 2024-09-12T02:53:10.401 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.683+0000 7fbf7f2b9640 1 -- 172.21.15.116:0/3841251706 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf781a0d40 con 0x7fbf7807de30 2024-09-12T02:53:10.401 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.684+0000 7fbf6e7fc640 1 -- 172.21.15.116:0/3841251706 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbf74045070 con 0x7fbf7807de30 2024-09-12T02:53:10.402 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.684+0000 7fbf6e7fc640 1 -- 172.21.15.116:0/3841251706 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbf74047b20 con 0x7fbf7807de30 2024-09-12T02:53:10.402 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.684+0000 7fbf7f2b9640 1 -- 172.21.15.116:0/3841251706 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf7807c360 con 0x7fbf7807de30 2024-09-12T02:53:10.402 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.687+0000 7fbf6e7fc640 1 -- 172.21.15.116:0/3841251706 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fbf7404c030 con 0x7fbf7807de30 2024-09-12T02:53:10.402 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.744+0000 7fbf7f2b9640 1 -- 172.21.15.116:0/3841251706 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fbf781a0f10 con 0x7fbf7807de30 2024-09-12T02:53:10.402 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.744+0000 7fbf6e7fc640 1 -- 172.21.15.116:0/3841251706 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7fbf740328c0 con 0x7fbf7807de30 2024-09-12T02:53:10.402 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.745+0000 7fbf7f2b9640 1 -- 172.21.15.116:0/3841251706 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf7807de30 msgr2=0x7fbf7819d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:10.402 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.745+0000 7fbf7f2b9640 1 --2- 172.21.15.116:0/3841251706 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf7807de30 0x7fbf7819d880 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbf74031000 tx=0x7fbf740326e0 comp rx=0 tx=0).stop 2024-09-12T02:53:10.402 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.745+0000 7fbf7f2b9640 1 -- 172.21.15.116:0/3841251706 shutdown_connections 2024-09-12T02:53:10.402 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.745+0000 7fbf7f2b9640 1 --2- 172.21.15.116:0/3841251706 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf7807de30 0x7fbf7819d880 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:10.402 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.745+0000 7fbf7f2b9640 1 -- 172.21.15.116:0/3841251706 >> 172.21.15.116:0/3841251706 conn(0x7fbf78102e50 msgr2=0x7fbf78103a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:10.402 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.745+0000 7fbf7f2b9640 1 -- 172.21.15.116:0/3841251706 shutdown_connections 2024-09-12T02:53:10.402 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:09.745+0000 7fbf7f2b9640 1 -- 172.21.15.116:0/3841251706 wait complete. 2024-09-12T02:53:10.402 INFO:teuthology.orchestra.run.smithi116.stdout:Restarting the monitor... 2024-09-12T02:53:10.679 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:10 smithi116 systemd[1]: Stopping Ceph mon.a for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T02:53:11.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:10 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mon-a[31038]: 2024-09-12T02:53:10.678+0000 7f7daa3d0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:53:11.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:10 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mon-a[31038]: 2024-09-12T02:53:10.678+0000 7f7daa3d0640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-12T02:53:11.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:10 smithi116 podman[31282]: 2024-09-12 02:53:10.88294137 +0000 UTC m=+0.327753335 container died a1b9eec41df654c04f61fa6f958a15923507e12728d168d91a86482900d21a6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:53:11.412 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:11 smithi116 podman[31282]: 2024-09-12 02:53:11.194507525 +0000 UTC m=+0.639319488 container cleanup a1b9eec41df654c04f61fa6f958a15923507e12728d168d91a86482900d21a6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-12T02:53:11.412 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:11 smithi116 bash[31282]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mon-a 2024-09-12T02:53:11.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:11 smithi116 podman[31297]: 2024-09-12 02:53:11.412120479 +0000 UTC m=+0.522510279 container remove a1b9eec41df654c04f61fa6f958a15923507e12728d168d91a86482900d21a6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:53:11.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:11 smithi116 systemd[1]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-12T02:53:11.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:11 smithi116 systemd[1]: Stopped Ceph mon.a for eb1ac602-70b1-11ef-bcea-c7b262605968. 2024-09-12T02:53:11.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:11 smithi116 systemd[1]: Starting Ceph mon.a for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T02:53:12.082 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:11 smithi116 podman[31390]: 2024-09-12 02:53:11.731737524 +0000 UTC m=+0.015470074 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:53:12.082 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:11 smithi116 podman[31390]: 2024-09-12 02:53:11.888341619 +0000 UTC m=+0.172074158 container create 3479f0eb634584799d95b5090fc6e44659aea28a4a22f04fc967e4c3ba35010a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-12T02:53:12.082 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 podman[31390]: 2024-09-12 02:53:12.082280464 +0000 UTC m=+0.366013000 container init 3479f0eb634584799d95b5090fc6e44659aea28a4a22f04fc967e4c3ba35010a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-12T02:53:12.169 INFO:teuthology.orchestra.run.smithi116.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-12T02:53:12.418 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 podman[31390]: 2024-09-12 02:53:12.085512774 +0000 UTC m=+0.369245321 container start 3479f0eb634584799d95b5090fc6e44659aea28a4a22f04fc967e4c3ba35010a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True) 2024-09-12T02:53:12.418 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: set uid:gid to 167:167 (ceph:ceph) 2024-09-12T02:53:12.418 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-12T02:53:12.418 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: pidfile_write: ignore empty --pid-file 2024-09-12T02:53:12.418 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: load: jerasure load: lrc 2024-09-12T02:53:12.418 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: RocksDB version: 7.9.2 2024-09-12T02:53:12.418 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Git sha 0 2024-09-12T02:53:12.419 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-12T02:53:12.419 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: DB SUMMARY 2024-09-12T02:53:12.419 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: DB Session ID: OWSRJUI4H4FLQ2NNVFL7 2024-09-12T02:53:12.419 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: CURRENT file: CURRENT 2024-09-12T02:53:12.419 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: IDENTITY file: IDENTITY 2024-09-12T02:53:12.419 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-12T02:53:12.419 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-12T02:53:12.419 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105442 ; 2024-09-12T02:53:12.419 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.error_if_exists: 0 2024-09-12T02:53:12.419 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.create_if_missing: 0 2024-09-12T02:53:12.419 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.paranoid_checks: 1 2024-09-12T02:53:12.419 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-12T02:53:12.420 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-12T02:53:12.420 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-12T02:53:12.420 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.env: 0x555f8b063940 2024-09-12T02:53:12.420 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.fs: PosixFileSystem 2024-09-12T02:53:12.420 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.info_log: 0x555f8c3fb6e0 2024-09-12T02:53:12.420 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.max_file_opening_threads: 16 2024-09-12T02:53:12.420 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.statistics: (nil) 2024-09-12T02:53:12.420 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.use_fsync: 0 2024-09-12T02:53:12.420 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.max_log_file_size: 0 2024-09-12T02:53:12.420 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-12T02:53:12.420 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-12T02:53:12.420 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.keep_log_file_num: 1000 2024-09-12T02:53:12.420 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.recycle_log_file_num: 0 2024-09-12T02:53:12.421 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.allow_fallocate: 1 2024-09-12T02:53:12.422 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.allow_mmap_reads: 0 2024-09-12T02:53:12.422 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.allow_mmap_writes: 0 2024-09-12T02:53:12.422 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.use_direct_reads: 0 2024-09-12T02:53:12.422 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-12T02:53:12.422 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.create_missing_column_families: 0 2024-09-12T02:53:12.422 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.db_log_dir: 2024-09-12T02:53:12.422 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.wal_dir: 2024-09-12T02:53:12.422 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-12T02:53:12.422 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-12T02:53:12.423 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-12T02:53:12.423 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-12T02:53:12.423 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-12T02:53:12.423 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-12T02:53:12.423 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.advise_random_on_open: 1 2024-09-12T02:53:12.423 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.db_write_buffer_size: 0 2024-09-12T02:53:12.423 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.write_buffer_manager: 0x555f8c40adc0 2024-09-12T02:53:12.423 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-12T02:53:12.423 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-12T02:53:12.423 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-12T02:53:12.423 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.rate_limiter: (nil) 2024-09-12T02:53:12.423 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-12T02:53:12.424 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.wal_recovery_mode: 2 2024-09-12T02:53:12.424 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.enable_thread_tracking: 0 2024-09-12T02:53:12.424 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.enable_pipelined_write: 0 2024-09-12T02:53:12.424 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.unordered_write: 0 2024-09-12T02:53:12.424 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-12T02:53:12.424 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-12T02:53:12.424 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-12T02:53:12.424 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-12T02:53:12.424 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.row_cache: None 2024-09-12T02:53:12.424 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.wal_filter: None 2024-09-12T02:53:12.424 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-12T02:53:12.424 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.allow_ingest_behind: 0 2024-09-12T02:53:12.424 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.two_write_queues: 0 2024-09-12T02:53:12.425 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.manual_wal_flush: 0 2024-09-12T02:53:12.425 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.wal_compression: 0 2024-09-12T02:53:12.425 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.atomic_flush: 0 2024-09-12T02:53:12.425 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-12T02:53:12.425 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-12T02:53:12.425 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-12T02:53:12.425 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.log_readahead_size: 0 2024-09-12T02:53:12.425 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-12T02:53:12.425 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.best_efforts_recovery: 0 2024-09-12T02:53:12.425 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-12T02:53:12.425 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-12T02:53:12.425 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.allow_data_in_errors: 0 2024-09-12T02:53:12.425 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.db_host_id: __hostname__ 2024-09-12T02:53:12.426 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-12T02:53:12.426 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.max_background_jobs: 2 2024-09-12T02:53:12.426 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.max_background_compactions: -1 2024-09-12T02:53:12.426 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.max_subcompactions: 1 2024-09-12T02:53:12.426 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-12T02:53:12.426 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-12T02:53:12.426 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-12T02:53:12.426 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.max_total_wal_size: 0 2024-09-12T02:53:12.426 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-12T02:53:12.426 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-12T02:53:12.426 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-12T02:53:12.426 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-12T02:53:12.426 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.max_open_files: -1 2024-09-12T02:53:12.426 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.bytes_per_sync: 0 2024-09-12T02:53:12.427 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-12T02:53:12.427 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-12T02:53:12.427 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.compaction_readahead_size: 0 2024-09-12T02:53:12.427 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.max_background_flushes: -1 2024-09-12T02:53:12.427 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Compression algorithms supported: 2024-09-12T02:53:12.427 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: kZSTD supported: 0 2024-09-12T02:53:12.427 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: kXpressCompression supported: 0 2024-09-12T02:53:12.427 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: kBZip2Compression supported: 0 2024-09-12T02:53:12.427 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-12T02:53:12.427 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: kLZ4Compression supported: 1 2024-09-12T02:53:12.427 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: kZlibCompression supported: 1 2024-09-12T02:53:12.427 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: kLZ4HCCompression supported: 1 2024-09-12T02:53:12.427 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: kSnappyCompression supported: 1 2024-09-12T02:53:12.428 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-12T02:53:12.428 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-12T02:53:12.428 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-12T02:53:12.428 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-12T02:53:12.428 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-12T02:53:12.428 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.merge_operator: 2024-09-12T02:53:12.428 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.compaction_filter: None 2024-09-12T02:53:12.428 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.compaction_filter_factory: None 2024-09-12T02:53:12.428 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.sst_partitioner_factory: None 2024-09-12T02:53:12.428 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-12T02:53:12.428 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-12T02:53:12.428 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555f8c3fb320) 2024-09-12T02:53:12.429 INFO:journalctl@ceph.mon.a.smithi116.stdout: cache_index_and_filter_blocks: 1 2024-09-12T02:53:12.429 INFO:journalctl@ceph.mon.a.smithi116.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-12T02:53:12.429 INFO:journalctl@ceph.mon.a.smithi116.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-12T02:53:12.429 INFO:journalctl@ceph.mon.a.smithi116.stdout: pin_top_level_index_and_filter: 1 2024-09-12T02:53:12.429 INFO:journalctl@ceph.mon.a.smithi116.stdout: index_type: 0 2024-09-12T02:53:12.429 INFO:journalctl@ceph.mon.a.smithi116.stdout: data_block_index_type: 0 2024-09-12T02:53:12.429 INFO:journalctl@ceph.mon.a.smithi116.stdout: index_shortening: 1 2024-09-12T02:53:12.429 INFO:journalctl@ceph.mon.a.smithi116.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-12T02:53:12.429 INFO:journalctl@ceph.mon.a.smithi116.stdout: checksum: 4 2024-09-12T02:53:12.429 INFO:journalctl@ceph.mon.a.smithi116.stdout: no_block_cache: 0 2024-09-12T02:53:12.429 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_cache: 0x555f8c3ef1f0 2024-09-12T02:53:12.429 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_cache_name: BinnedLRUCache 2024-09-12T02:53:12.429 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_cache_options: 2024-09-12T02:53:12.429 INFO:journalctl@ceph.mon.a.smithi116.stdout: capacity : 536870912 2024-09-12T02:53:12.430 INFO:journalctl@ceph.mon.a.smithi116.stdout: num_shard_bits : 4 2024-09-12T02:53:12.430 INFO:journalctl@ceph.mon.a.smithi116.stdout: strict_capacity_limit : 0 2024-09-12T02:53:12.430 INFO:journalctl@ceph.mon.a.smithi116.stdout: high_pri_pool_ratio: 0.000 2024-09-12T02:53:12.430 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_cache_compressed: (nil) 2024-09-12T02:53:12.430 INFO:journalctl@ceph.mon.a.smithi116.stdout: persistent_cache: (nil) 2024-09-12T02:53:12.430 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_size: 4096 2024-09-12T02:53:12.430 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_size_deviation: 10 2024-09-12T02:53:12.430 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_restart_interval: 16 2024-09-12T02:53:12.430 INFO:journalctl@ceph.mon.a.smithi116.stdout: index_block_restart_interval: 1 2024-09-12T02:53:12.430 INFO:journalctl@ceph.mon.a.smithi116.stdout: metadata_block_size: 4096 2024-09-12T02:53:12.430 INFO:journalctl@ceph.mon.a.smithi116.stdout: partition_filters: 0 2024-09-12T02:53:12.430 INFO:journalctl@ceph.mon.a.smithi116.stdout: use_delta_encoding: 1 2024-09-12T02:53:12.430 INFO:journalctl@ceph.mon.a.smithi116.stdout: filter_policy: bloomfilter 2024-09-12T02:53:12.431 INFO:journalctl@ceph.mon.a.smithi116.stdout: whole_key_filtering: 1 2024-09-12T02:53:12.431 INFO:journalctl@ceph.mon.a.smithi116.stdout: verify_compression: 0 2024-09-12T02:53:12.431 INFO:journalctl@ceph.mon.a.smithi116.stdout: read_amp_bytes_per_bit: 0 2024-09-12T02:53:12.431 INFO:journalctl@ceph.mon.a.smithi116.stdout: format_version: 5 2024-09-12T02:53:12.431 INFO:journalctl@ceph.mon.a.smithi116.stdout: enable_index_compression: 1 2024-09-12T02:53:12.431 INFO:journalctl@ceph.mon.a.smithi116.stdout: block_align: 0 2024-09-12T02:53:12.431 INFO:journalctl@ceph.mon.a.smithi116.stdout: max_auto_readahead_size: 262144 2024-09-12T02:53:12.431 INFO:journalctl@ceph.mon.a.smithi116.stdout: prepopulate_block_cache: 0 2024-09-12T02:53:12.431 INFO:journalctl@ceph.mon.a.smithi116.stdout: initial_auto_readahead_size: 8192 2024-09-12T02:53:12.431 INFO:journalctl@ceph.mon.a.smithi116.stdout: num_file_reads_for_auto_readahead: 2 2024-09-12T02:53:12.431 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.write_buffer_size: 33554432 2024-09-12T02:53:12.431 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.max_write_buffer_number: 2 2024-09-12T02:53:12.431 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.compression: NoCompression 2024-09-12T02:53:12.431 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.bottommost_compression: Disabled 2024-09-12T02:53:12.432 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.prefix_extractor: nullptr 2024-09-12T02:53:12.432 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-12T02:53:12.432 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.num_levels: 7 2024-09-12T02:53:12.432 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-12T02:53:12.432 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-12T02:53:12.432 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-12T02:53:12.432 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-12T02:53:12.432 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-12T02:53:12.432 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-12T02:53:12.432 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-12T02:53:12.432 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-12T02:53:12.432 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-12T02:53:12.432 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-12T02:53:12.433 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-12T02:53:12.433 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-12T02:53:12.433 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-12T02:53:12.433 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.compression_opts.level: 32767 2024-09-12T02:53:12.433 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.compression_opts.strategy: 0 2024-09-12T02:53:12.433 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-12T02:53:12.433 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-12T02:53:12.433 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-12T02:53:12.433 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-12T02:53:12.433 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.compression_opts.enabled: false 2024-09-12T02:53:12.433 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-12T02:53:12.433 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-12T02:53:12.433 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-12T02:53:12.433 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-12T02:53:12.434 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.target_file_size_base: 67108864 2024-09-12T02:53:12.434 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-12T02:53:12.434 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-12T02:53:12.434 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-12T02:53:12.434 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-12T02:53:12.434 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-12T02:53:12.434 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-12T02:53:12.434 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-12T02:53:12.434 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-12T02:53:12.434 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-12T02:53:12.434 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-12T02:53:12.434 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-12T02:53:12.434 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-12T02:53:12.434 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-12T02:53:12.435 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-12T02:53:12.435 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.arena_block_size: 1048576 2024-09-12T02:53:12.435 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-12T02:53:12.435 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-12T02:53:12.435 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.disable_auto_compactions: 0 2024-09-12T02:53:12.435 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-12T02:53:12.435 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-12T02:53:12.435 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-12T02:53:12.435 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-12T02:53:12.435 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-12T02:53:12.435 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-12T02:53:12.435 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-12T02:53:12.435 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-12T02:53:12.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-12T02:53:12.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-12T02:53:12.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-12T02:53:12.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.inplace_update_support: 0 2024-09-12T02:53:12.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-12T02:53:12.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-12T02:53:12.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-12T02:53:12.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-12T02:53:12.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.bloom_locality: 0 2024-09-12T02:53:12.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.max_successive_merges: 0 2024-09-12T02:53:12.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-12T02:53:12.436 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.paranoid_file_checks: 0 2024-09-12T02:53:12.437 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.force_consistency_checks: 1 2024-09-12T02:53:12.437 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.report_bg_io_stats: 0 2024-09-12T02:53:12.437 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.ttl: 2592000 2024-09-12T02:53:12.437 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-12T02:53:12.437 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-12T02:53:12.437 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-12T02:53:12.437 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.enable_blob_files: false 2024-09-12T02:53:12.437 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.min_blob_size: 0 2024-09-12T02:53:12.440 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.blob_file_size: 268435456 2024-09-12T02:53:12.440 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-12T02:53:12.440 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-12T02:53:12.440 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-12T02:53:12.441 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-12T02:53:12.441 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-12T02:53:12.441 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.blob_file_starting_level: 0 2024-09-12T02:53:12.441 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-12T02:53:12.441 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-12T02:53:12.441 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-12T02:53:12.441 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 35bce34f-a95a-4dc9-a7a3-c367a06ffaf9 2024-09-12T02:53:12.441 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726109592111193, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-12T02:53:12.441 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-12T02:53:12.441 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726109592112156, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100490, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98384, "index_size": 298, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14320, "raw_average_key_size": 49, "raw_value_size": 90637, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726109592, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "35bce34f-a95a-4dc9-a7a3-c367a06ffaf9", "db_session_id": "OWSRJUI4H4FLQ2NNVFL7", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-12T02:53:12.441 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726109592112251, "job": 1, "event": "recovery_finished"} 2024-09-12T02:53:12.441 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-12T02:53:12.441 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-12T02:53:12.442 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x555f8c41ce00 2024-09-12T02:53:12.442 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: DB pointer 0x555f8c518000 2024-09-12T02:53:12.442 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-12T02:53:12.442 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-12T02:53:12.442 INFO:journalctl@ceph.mon.a.smithi116.stdout: ** DB Stats ** 2024-09-12T02:53:12.442 INFO:journalctl@ceph.mon.a.smithi116.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T02:53:12.442 INFO:journalctl@ceph.mon.a.smithi116.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-12T02:53:12.442 INFO:journalctl@ceph.mon.a.smithi116.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T02:53:12.442 INFO:journalctl@ceph.mon.a.smithi116.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T02:53:12.442 INFO:journalctl@ceph.mon.a.smithi116.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-12T02:53:12.442 INFO:journalctl@ceph.mon.a.smithi116.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T02:53:12.442 INFO:journalctl@ceph.mon.a.smithi116.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T02:53:12.443 INFO:journalctl@ceph.mon.a.smithi116.stdout: 2024-09-12T02:53:12.443 INFO:journalctl@ceph.mon.a.smithi116.stdout: ** Compaction Stats [default] ** 2024-09-12T02:53:12.443 INFO:journalctl@ceph.mon.a.smithi116.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T02:53:12.443 INFO:journalctl@ceph.mon.a.smithi116.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-12T02:53:12.443 INFO:journalctl@ceph.mon.a.smithi116.stdout: L0 2/0 100.00 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 159.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T02:53:12.443 INFO:journalctl@ceph.mon.a.smithi116.stdout: Sum 2/0 100.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 159.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T02:53:12.443 INFO:journalctl@ceph.mon.a.smithi116.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 159.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T02:53:12.443 INFO:journalctl@ceph.mon.a.smithi116.stdout: 2024-09-12T02:53:12.443 INFO:journalctl@ceph.mon.a.smithi116.stdout: ** Compaction Stats [default] ** 2024-09-12T02:53:12.443 INFO:journalctl@ceph.mon.a.smithi116.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T02:53:12.443 INFO:journalctl@ceph.mon.a.smithi116.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-12T02:53:12.443 INFO:journalctl@ceph.mon.a.smithi116.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 159.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T02:53:12.443 INFO:journalctl@ceph.mon.a.smithi116.stdout: 2024-09-12T02:53:12.443 INFO:journalctl@ceph.mon.a.smithi116.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-12T02:53:12.444 INFO:journalctl@ceph.mon.a.smithi116.stdout: 2024-09-12T02:53:12.444 INFO:journalctl@ceph.mon.a.smithi116.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T02:53:12.444 INFO:journalctl@ceph.mon.a.smithi116.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-12T02:53:12.444 INFO:journalctl@ceph.mon.a.smithi116.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-12T02:53:12.444 INFO:journalctl@ceph.mon.a.smithi116.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-12T02:53:12.444 INFO:journalctl@ceph.mon.a.smithi116.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-12T02:53:12.444 INFO:journalctl@ceph.mon.a.smithi116.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-12T02:53:12.444 INFO:journalctl@ceph.mon.a.smithi116.stdout: Cumulative compaction: 0.00 GB write, 31.20 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T02:53:12.444 INFO:journalctl@ceph.mon.a.smithi116.stdout: Interval compaction: 0.00 GB write, 31.20 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T02:53:12.444 INFO:journalctl@ceph.mon.a.smithi116.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-12T02:53:12.444 INFO:journalctl@ceph.mon.a.smithi116.stdout: Block cache BinnedLRUCache@0x555f8c3ef1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-12T02:53:12.444 INFO:journalctl@ceph.mon.a.smithi116.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-12T02:53:12.444 INFO:journalctl@ceph.mon.a.smithi116.stdout: 2024-09-12T02:53:12.445 INFO:journalctl@ceph.mon.a.smithi116.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-12T02:53:12.445 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: starting mon.a rank 0 at public addrs [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] at bind addrs [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T02:53:12.445 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: mon.a@-1(???) e1 preinit fsid eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T02:53:12.445 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: mon.a@-1(???).mds e1 new map 2024-09-12T02:53:12.445 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: mon.a@-1(???).mds e1 print_map 2024-09-12T02:53:12.445 INFO:journalctl@ceph.mon.a.smithi116.stdout: e1 2024-09-12T02:53:12.445 INFO:journalctl@ceph.mon.a.smithi116.stdout: btime 2024-09-12T02:53:07:105500+0000 2024-09-12T02:53:12.445 INFO:journalctl@ceph.mon.a.smithi116.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T02:53:12.445 INFO:journalctl@ceph.mon.a.smithi116.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T02:53:12.445 INFO:journalctl@ceph.mon.a.smithi116.stdout: legacy client fscid: -1 2024-09-12T02:53:12.445 INFO:journalctl@ceph.mon.a.smithi116.stdout: 2024-09-12T02:53:12.445 INFO:journalctl@ceph.mon.a.smithi116.stdout: No filesystems configured 2024-09-12T02:53:12.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T02:53:12.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:53:12.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:53:12.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:53:12.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-12T02:53:12.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 bash[31390]: 3479f0eb634584799d95b5090fc6e44659aea28a4a22f04fc967e4c3ba35010a 2024-09-12T02:53:12.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 systemd[1]: Started Ceph mon.a for eb1ac602-70b1-11ef-bcea-c7b262605968. 2024-09-12T02:53:12.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T02:53:12.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: monmap epoch 1 2024-09-12T02:53:12.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: fsid eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T02:53:12.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: last_changed 2024-09-12T02:53:04.190821+0000 2024-09-12T02:53:12.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: created 2024-09-12T02:53:04.190821+0000 2024-09-12T02:53:12.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: min_mon_release 19 (squid) 2024-09-12T02:53:12.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: election_strategy: 1 2024-09-12T02:53:12.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: 0: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.a 2024-09-12T02:53:12.447 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: fsmap 2024-09-12T02:53:12.447 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: osdmap e1: 0 total, 0 up, 0 in 2024-09-12T02:53:12.447 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:12 smithi116 ceph-mon[31410]: mgrmap e1: no daemons active 2024-09-12T02:53:13.249 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.589+0000 7faa99c53640 1 Processor -- start 2024-09-12T02:53:13.255 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.589+0000 7faa99c53640 1 -- start start 2024-09-12T02:53:13.255 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.590+0000 7faa99c53640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa9407dd70 0x7faa9407c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:13.255 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.590+0000 7faa99c53640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7faa9407e170 con 0x7faa9407dd70 2024-09-12T02:53:13.255 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.590+0000 7faa937fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa9407dd70 0x7faa9407c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:13.256 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.590+0000 7faa937fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa9407dd70 0x7faa9407c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54160/0 (socket says 172.21.15.116:54160) 2024-09-12T02:53:13.256 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.590+0000 7faa937fe640 1 -- 172.21.15.116:0/3755368855 learned_addr learned my addr 172.21.15.116:0/3755368855 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:13.256 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.590+0000 7faa937fe640 1 -- 172.21.15.116:0/3755368855 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9407c820 con 0x7faa9407dd70 2024-09-12T02:53:13.256 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.592+0000 7faa937fe640 1 --2- 172.21.15.116:0/3755368855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa9407dd70 0x7faa9407c2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7faa7c009820 tx=0x7faa7c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad4a5fcb26aece8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:13.256 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.593+0000 7faa927fc640 1 -- 172.21.15.116:0/3755368855 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa7c037070 con 0x7faa9407dd70 2024-09-12T02:53:13.256 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.593+0000 7faa927fc640 1 -- 172.21.15.116:0/3755368855 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faa7c032900 con 0x7faa9407dd70 2024-09-12T02:53:13.256 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.593+0000 7faa927fc640 1 -- 172.21.15.116:0/3755368855 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa7c030850 con 0x7faa9407dd70 2024-09-12T02:53:13.256 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.593+0000 7faa99c53640 1 -- 172.21.15.116:0/3755368855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa9407dd70 msgr2=0x7faa9407c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:13.256 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.593+0000 7faa99c53640 1 --2- 172.21.15.116:0/3755368855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa9407dd70 0x7faa9407c2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7faa7c009820 tx=0x7faa7c02a270 comp rx=0 tx=0).stop 2024-09-12T02:53:13.256 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.593+0000 7faa99c53640 1 -- 172.21.15.116:0/3755368855 shutdown_connections 2024-09-12T02:53:13.256 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.593+0000 7faa99c53640 1 --2- 172.21.15.116:0/3755368855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa9407dd70 0x7faa9407c2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:13.256 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.593+0000 7faa99c53640 1 -- 172.21.15.116:0/3755368855 >> 172.21.15.116:0/3755368855 conn(0x7faa94102eb0 msgr2=0x7faa941052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:13.257 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.593+0000 7faa99c53640 1 -- 172.21.15.116:0/3755368855 shutdown_connections 2024-09-12T02:53:13.257 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.593+0000 7faa99c53640 1 -- 172.21.15.116:0/3755368855 wait complete. 2024-09-12T02:53:13.257 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.594+0000 7faa99c53640 1 Processor -- start 2024-09-12T02:53:13.257 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.594+0000 7faa99c53640 1 -- start start 2024-09-12T02:53:13.257 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.594+0000 7faa99c53640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa9407dd70 0x7faa9419d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:13.257 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.594+0000 7faa99c53640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7faa9410dba0 con 0x7faa9407dd70 2024-09-12T02:53:13.257 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.594+0000 7faa937fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa9407dd70 0x7faa9419d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:13.260 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.594+0000 7faa937fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa9407dd70 0x7faa9419d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54162/0 (socket says 172.21.15.116:54162) 2024-09-12T02:53:13.260 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.594+0000 7faa937fe640 1 -- 172.21.15.116:0/139572505 learned_addr learned my addr 172.21.15.116:0/139572505 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:13.260 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.595+0000 7faa937fe640 1 -- 172.21.15.116:0/139572505 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9419dd00 con 0x7faa9407dd70 2024-09-12T02:53:13.260 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.595+0000 7faa937fe640 1 --2- 172.21.15.116:0/139572505 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa9407dd70 0x7faa9419d7c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7faa7c032ce0 tx=0x7faa7c032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:13.261 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.595+0000 7faa90ff9640 1 -- 172.21.15.116:0/139572505 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa7c042070 con 0x7faa9407dd70 2024-09-12T02:53:13.261 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.595+0000 7faa90ff9640 1 -- 172.21.15.116:0/139572505 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faa7c0308c0 con 0x7faa9407dd70 2024-09-12T02:53:13.261 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.595+0000 7faa99c53640 1 -- 172.21.15.116:0/139572505 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa9419df90 con 0x7faa9407dd70 2024-09-12T02:53:13.261 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.595+0000 7faa90ff9640 1 -- 172.21.15.116:0/139572505 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa7c037040 con 0x7faa9407dd70 2024-09-12T02:53:13.261 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.595+0000 7faa99c53640 1 -- 172.21.15.116:0/139572505 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa941a0c80 con 0x7faa9407dd70 2024-09-12T02:53:13.261 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.595+0000 7faa90ff9640 1 -- 172.21.15.116:0/139572505 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7faa7c045070 con 0x7faa9407dd70 2024-09-12T02:53:13.261 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.595+0000 7faa90ff9640 1 -- 172.21.15.116:0/139572505 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7faa7c047b80 con 0x7faa9407dd70 2024-09-12T02:53:13.261 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.596+0000 7faa99c53640 1 -- 172.21.15.116:0/139572505 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa9407c360 con 0x7faa9407dd70 2024-09-12T02:53:13.261 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.598+0000 7faa90ff9640 1 -- 172.21.15.116:0/139572505 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7faa7c03c7e0 con 0x7faa9407dd70 2024-09-12T02:53:13.261 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.655+0000 7faa99c53640 1 -- 172.21.15.116:0/139572505 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7faa941a0e50 con 0x7faa9407dd70 2024-09-12T02:53:13.261 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.660+0000 7faa90ff9640 1 -- 172.21.15.116:0/139572505 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7faa7c03c9c0 con 0x7faa9407dd70 2024-09-12T02:53:13.261 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.661+0000 7faa99c53640 1 -- 172.21.15.116:0/139572505 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa9407dd70 msgr2=0x7faa9419d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:13.261 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.661+0000 7faa99c53640 1 --2- 172.21.15.116:0/139572505 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa9407dd70 0x7faa9419d7c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7faa7c032ce0 tx=0x7faa7c032d10 comp rx=0 tx=0).stop 2024-09-12T02:53:13.262 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.661+0000 7faa99c53640 1 -- 172.21.15.116:0/139572505 shutdown_connections 2024-09-12T02:53:13.262 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.661+0000 7faa99c53640 1 --2- 172.21.15.116:0/139572505 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa9407dd70 0x7faa9419d7c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:13.262 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.661+0000 7faa99c53640 1 -- 172.21.15.116:0/139572505 >> 172.21.15.116:0/139572505 conn(0x7faa94102eb0 msgr2=0x7faa94103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:13.262 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.661+0000 7faa99c53640 1 -- 172.21.15.116:0/139572505 shutdown_connections 2024-09-12T02:53:13.262 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:12.661+0000 7faa99c53640 1 -- 172.21.15.116:0/139572505 wait complete. 2024-09-12T02:53:13.262 INFO:teuthology.orchestra.run.smithi116.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-12T02:53:13.280 INFO:teuthology.orchestra.run.smithi116.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:53:13.280 INFO:teuthology.orchestra.run.smithi116.stdout:Creating mgr... 2024-09-12T02:53:13.280 INFO:teuthology.orchestra.run.smithi116.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-12T02:53:13.281 INFO:teuthology.orchestra.run.smithi116.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-12T02:53:13.527 INFO:teuthology.orchestra.run.smithi116.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mgr.x 2024-09-12T02:53:13.528 INFO:teuthology.orchestra.run.smithi116.stdout:systemctl: stderr Failed to reset failed state of unit ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-12T02:53:13.675 INFO:teuthology.orchestra.run.smithi116.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-eb1ac602-70b1-11ef-bcea-c7b262605968.target.wants/ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@.service. 2024-09-12T02:53:13.911 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:13 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/139572505' entity='client.admin' 2024-09-12T02:53:13.912 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:13 smithi116 systemd[1]: Starting Ceph mgr.x for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T02:53:14.210 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:13 smithi116 podman[31621]: 2024-09-12 02:53:13.813836301 +0000 UTC m=+0.015351249 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:53:14.210 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:13 smithi116 podman[31621]: 2024-09-12 02:53:13.93253538 +0000 UTC m=+0.134050312 container create 3b0b103b7373583925494cda22ad0e7c824339f5c2a54aadcfba3a8cdd3b52ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:53:14.210 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:14 smithi116 podman[31621]: 2024-09-12 02:53:14.134730725 +0000 UTC m=+0.336245663 container init 3b0b103b7373583925494cda22ad0e7c824339f5c2a54aadcfba3a8cdd3b52ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0) 2024-09-12T02:53:14.210 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:14 smithi116 podman[31621]: 2024-09-12 02:53:14.137623917 +0000 UTC m=+0.339138849 container start 3b0b103b7373583925494cda22ad0e7c824339f5c2a54aadcfba3a8cdd3b52ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:53:14.210 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:14 smithi116 bash[31621]: 3b0b103b7373583925494cda22ad0e7c824339f5c2a54aadcfba3a8cdd3b52ae 2024-09-12T02:53:14.224 INFO:teuthology.orchestra.run.smithi116.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T02:53:14.224 INFO:teuthology.orchestra.run.smithi116.stdout:systemctl: stdout disabled 2024-09-12T02:53:14.232 INFO:teuthology.orchestra.run.smithi116.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T02:53:14.232 INFO:teuthology.orchestra.run.smithi116.stdout:systemctl: stdout inactive 2024-09-12T02:53:14.232 INFO:teuthology.orchestra.run.smithi116.stdout:firewalld.service is not enabled 2024-09-12T02:53:14.232 INFO:teuthology.orchestra.run.smithi116.stdout:Not possible to enable service . firewalld.service is not available 2024-09-12T02:53:14.241 INFO:teuthology.orchestra.run.smithi116.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T02:53:14.241 INFO:teuthology.orchestra.run.smithi116.stdout:systemctl: stdout disabled 2024-09-12T02:53:14.248 INFO:teuthology.orchestra.run.smithi116.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T02:53:14.248 INFO:teuthology.orchestra.run.smithi116.stdout:systemctl: stdout inactive 2024-09-12T02:53:14.248 INFO:teuthology.orchestra.run.smithi116.stdout:firewalld.service is not enabled 2024-09-12T02:53:14.248 INFO:teuthology.orchestra.run.smithi116.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-12T02:53:14.248 INFO:teuthology.orchestra.run.smithi116.stdout:Waiting for mgr to start... 2024-09-12T02:53:14.249 INFO:teuthology.orchestra.run.smithi116.stdout:Waiting for mgr... 2024-09-12T02:53:14.471 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:14 smithi116 systemd[1]: Started Ceph mgr.x for eb1ac602-70b1-11ef-bcea-c7b262605968. 2024-09-12T02:53:14.640 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:14 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:14.260+0000 7f44ed715040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T02:53:14.641 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:14 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:14.319+0000 7f44ed715040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T02:53:14.641 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:14 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:14.470+0000 7f44ed715040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T02:53:14.721 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:14 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:14.539+0000 7f44ed715040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T02:53:14.721 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:14 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:14.694+0000 7f44ed715040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:53:15.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:14 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/532051020' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T02:53:15.052 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:14 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:14.842+0000 7f44ed715040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T02:53:15.366 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:15 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:15.164+0000 7f44ed715040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T02:53:15.366 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:15 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:15.225+0000 7f44ed715040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T02:53:15.370 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout 2024-09-12T02:53:15.370 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:53:15.371 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "fsid": "eb1ac602-70b1-11ef-bcea-c7b262605968", 2024-09-12T02:53:15.371 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T02:53:15.371 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T02:53:15.371 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T02:53:15.371 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T02:53:15.371 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:53:15.371 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T02:53:15.371 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T02:53:15.371 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout 0 2024-09-12T02:53:15.371 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:53:15.371 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T02:53:15.371 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T02:53:15.372 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:53:15.372 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-12T02:53:15.372 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T02:53:15.372 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:53:15.372 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T02:53:15.372 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T02:53:15.372 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:53:15.372 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T02:53:15.372 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:53:15.372 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T02:53:15.372 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T02:53:15.372 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T02:53:15.379 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T02:53:15.380 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T02:53:15.380 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T02:53:15.380 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:53:15.380 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T02:53:15.380 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T02:53:15.380 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T02:53:15.380 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T02:53:15.380 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T02:53:15.380 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T02:53:15.380 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T02:53:15.380 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T02:53:15.380 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T02:53:15.381 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:53:15.381 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T02:53:15.381 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:53:15.381 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T02:53:07:105500+0000", 2024-09-12T02:53:15.381 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T02:53:15.381 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T02:53:15.381 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:53:15.381 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T02:53:15.381 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-12T02:53:15.381 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T02:53:15.381 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T02:53:15.381 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T02:53:15.381 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T02:53:15.381 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T02:53:15.382 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:53:15.382 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T02:53:15.382 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:53:15.382 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T02:53:15.382 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:53:15.382 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T02:53:07.106143+0000", 2024-09-12T02:53:15.382 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T02:53:15.382 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:53:15.382 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T02:53:15.382 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:53:15.382 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.650+0000 7f56f4f65640 1 Processor -- start 2024-09-12T02:53:15.382 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.650+0000 7f56f4f65640 1 -- start start 2024-09-12T02:53:15.382 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.650+0000 7f56f4f65640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56f0106520 0x7f56f0106920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:15.382 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.650+0000 7f56f4f65640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f56f0106ef0 con 0x7f56f0106520 2024-09-12T02:53:15.382 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.650+0000 7f56ee59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56f0106520 0x7f56f0106920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:15.383 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.650+0000 7f56ee59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56f0106520 0x7f56f0106920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54182/0 (socket says 172.21.15.116:54182) 2024-09-12T02:53:15.383 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.650+0000 7f56ee59c640 1 -- 172.21.15.116:0/662081832 learned_addr learned my addr 172.21.15.116:0/662081832 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:15.383 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.651+0000 7f56ee59c640 1 -- 172.21.15.116:0/662081832 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f0107720 con 0x7f56f0106520 2024-09-12T02:53:15.383 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.652+0000 7f56ee59c640 1 --2- 172.21.15.116:0/662081832 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56f0106520 0x7f56f0106920 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f56e0009820 tx=0x7f56e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e180498bd88225 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:15.383 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.652+0000 7f56edd9b640 1 -- 172.21.15.116:0/662081832 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56e0037070 con 0x7f56f0106520 2024-09-12T02:53:15.383 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.652+0000 7f56edd9b640 1 -- 172.21.15.116:0/662081832 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f56e0032900 con 0x7f56f0106520 2024-09-12T02:53:15.383 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.652+0000 7f56edd9b640 1 -- 172.21.15.116:0/662081832 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56e0030850 con 0x7f56f0106520 2024-09-12T02:53:15.383 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.653+0000 7f56f4f65640 1 -- 172.21.15.116:0/662081832 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56f0106520 msgr2=0x7f56f0106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:15.383 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.653+0000 7f56f4f65640 1 --2- 172.21.15.116:0/662081832 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56f0106520 0x7f56f0106920 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f56e0009820 tx=0x7f56e002a270 comp rx=0 tx=0).stop 2024-09-12T02:53:15.383 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.653+0000 7f56f4f65640 1 -- 172.21.15.116:0/662081832 shutdown_connections 2024-09-12T02:53:15.383 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.653+0000 7f56f4f65640 1 --2- 172.21.15.116:0/662081832 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56f0106520 0x7f56f0106920 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:15.383 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.653+0000 7f56f4f65640 1 -- 172.21.15.116:0/662081832 >> 172.21.15.116:0/662081832 conn(0x7f56f0101db0 msgr2=0x7f56f01041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:15.383 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.653+0000 7f56f4f65640 1 -- 172.21.15.116:0/662081832 shutdown_connections 2024-09-12T02:53:15.383 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.653+0000 7f56f4f65640 1 -- 172.21.15.116:0/662081832 wait complete. 2024-09-12T02:53:15.383 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.654+0000 7f56f4f65640 1 Processor -- start 2024-09-12T02:53:15.384 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.654+0000 7f56f4f65640 1 -- start start 2024-09-12T02:53:15.384 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.654+0000 7f56f4f65640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56f0106520 0x7f56f0191750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:15.384 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.654+0000 7f56f4f65640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f56f010aad0 con 0x7f56f0106520 2024-09-12T02:53:15.384 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.654+0000 7f56ee59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56f0106520 0x7f56f0191750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:15.384 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.654+0000 7f56ee59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56f0106520 0x7f56f0191750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54194/0 (socket says 172.21.15.116:54194) 2024-09-12T02:53:15.384 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.654+0000 7f56ee59c640 1 -- 172.21.15.116:0/532051020 learned_addr learned my addr 172.21.15.116:0/532051020 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:15.384 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.654+0000 7f56ee59c640 1 -- 172.21.15.116:0/532051020 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f0191c90 con 0x7f56f0106520 2024-09-12T02:53:15.384 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.654+0000 7f56ee59c640 1 --2- 172.21.15.116:0/532051020 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56f0106520 0x7f56f0191750 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f56e0009940 tx=0x7f56e0032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:15.384 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.655+0000 7f56e77fe640 1 -- 172.21.15.116:0/532051020 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56e0042070 con 0x7f56f0106520 2024-09-12T02:53:15.384 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.655+0000 7f56e77fe640 1 -- 172.21.15.116:0/532051020 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f56e00308c0 con 0x7f56f0106520 2024-09-12T02:53:15.384 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.655+0000 7f56e77fe640 1 -- 172.21.15.116:0/532051020 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56e0037040 con 0x7f56f0106520 2024-09-12T02:53:15.385 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.655+0000 7f56f4f65640 1 -- 172.21.15.116:0/532051020 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f018fe80 con 0x7f56f0106520 2024-09-12T02:53:15.385 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.655+0000 7f56f4f65640 1 -- 172.21.15.116:0/532051020 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f01903c0 con 0x7f56f0106520 2024-09-12T02:53:15.385 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.655+0000 7f56e77fe640 1 -- 172.21.15.116:0/532051020 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f56e0045070 con 0x7f56f0106520 2024-09-12T02:53:15.385 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.655+0000 7f56e77fe640 1 -- 172.21.15.116:0/532051020 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f56e00463b0 con 0x7f56f0106520 2024-09-12T02:53:15.385 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.656+0000 7f56f4f65640 1 -- 172.21.15.116:0/532051020 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56f010ac50 con 0x7f56f0106520 2024-09-12T02:53:15.385 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.658+0000 7f56e77fe640 1 -- 172.21.15.116:0/532051020 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f56e00327b0 con 0x7f56f0106520 2024-09-12T02:53:15.385 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.718+0000 7f56f4f65640 1 -- 172.21.15.116:0/532051020 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f56f0195430 con 0x7f56f0106520 2024-09-12T02:53:15.385 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.718+0000 7f56e77fe640 1 -- 172.21.15.116:0/532051020 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f56e0047440 con 0x7f56f0106520 2024-09-12T02:53:15.385 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.719+0000 7f56f4f65640 1 -- 172.21.15.116:0/532051020 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56f0106520 msgr2=0x7f56f0191750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:15.385 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.719+0000 7f56f4f65640 1 --2- 172.21.15.116:0/532051020 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56f0106520 0x7f56f0191750 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f56e0009940 tx=0x7f56e0032ce0 comp rx=0 tx=0).stop 2024-09-12T02:53:15.385 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.719+0000 7f56f4f65640 1 -- 172.21.15.116:0/532051020 shutdown_connections 2024-09-12T02:53:15.386 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.719+0000 7f56f4f65640 1 --2- 172.21.15.116:0/532051020 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56f0106520 0x7f56f0191750 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:15.386 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.719+0000 7f56f4f65640 1 -- 172.21.15.116:0/532051020 >> 172.21.15.116:0/532051020 conn(0x7f56f0101db0 msgr2=0x7f56f0102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:15.386 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.719+0000 7f56f4f65640 1 -- 172.21.15.116:0/532051020 shutdown_connections 2024-09-12T02:53:15.386 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:14.719+0000 7f56f4f65640 1 -- 172.21.15.116:0/532051020 wait complete. 2024-09-12T02:53:15.386 INFO:teuthology.orchestra.run.smithi116.stdout:mgr not available, waiting (1/15)... 2024-09-12T02:53:15.802 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:15 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:15.412+0000 7f44ed715040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T02:53:16.552 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:16 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:16.292+0000 7f44ed715040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T02:53:16.552 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:16 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:16.358+0000 7f44ed715040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T02:53:17.327 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:17 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:17.068+0000 7f44ed715040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T02:53:17.327 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:17 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:17.183+0000 7f44ed715040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T02:53:17.327 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:17 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:17.326+0000 7f44ed715040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:53:17.615 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:17 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:17.388+0000 7f44ed715040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T02:53:17.615 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:17 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:17.542+0000 7f44ed715040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T02:53:17.615 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:17 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:17.614+0000 7f44ed715040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T02:53:17.911 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:17 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1483384520' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T02:53:17.911 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:17 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:17.676+0000 7f44ed715040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T02:53:17.911 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:17 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:17.844+0000 7f44ed715040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T02:53:17.911 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:17 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:17.910+0000 7f44ed715040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T02:53:18.182 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:17 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:17.977+0000 7f44ed715040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T02:53:18.182 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:18 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:18.061+0000 7f44ed715040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T02:53:18.182 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:18 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:18.121+0000 7f44ed715040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T02:53:18.182 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:18 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:18.181+0000 7f44ed715040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T02:53:18.481 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout 2024-09-12T02:53:18.482 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:53:18.482 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "fsid": "eb1ac602-70b1-11ef-bcea-c7b262605968", 2024-09-12T02:53:18.482 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T02:53:18.482 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T02:53:18.482 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T02:53:18.482 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T02:53:18.482 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:53:18.482 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T02:53:18.482 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T02:53:18.482 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout 0 2024-09-12T02:53:18.482 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:53:18.483 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T02:53:18.483 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T02:53:18.483 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:53:18.483 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-12T02:53:18.483 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T02:53:18.483 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:53:18.483 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T02:53:18.483 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T02:53:18.483 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:53:18.483 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T02:53:18.483 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:53:18.483 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T02:53:18.484 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T02:53:18.484 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T02:53:18.484 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T02:53:18.484 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T02:53:18.484 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T02:53:18.484 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:53:18.484 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T02:53:18.484 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T02:53:18.484 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T02:53:18.484 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T02:53:18.484 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T02:53:18.484 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T02:53:18.485 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T02:53:18.485 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T02:53:18.485 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T02:53:18.485 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:53:18.485 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T02:53:18.485 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:53:18.485 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T02:53:07:105500+0000", 2024-09-12T02:53:18.485 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T02:53:18.485 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T02:53:18.485 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:53:18.485 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T02:53:18.485 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-12T02:53:18.486 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T02:53:18.486 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T02:53:18.486 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T02:53:18.486 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T02:53:18.486 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T02:53:18.486 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:53:18.486 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T02:53:18.486 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:53:18.486 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T02:53:18.486 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:53:18.486 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T02:53:07.106143+0000", 2024-09-12T02:53:18.486 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T02:53:18.486 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:53:18.487 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T02:53:18.487 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:53:18.487 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.728+0000 7ff864bab640 1 Processor -- start 2024-09-12T02:53:18.487 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.728+0000 7ff864bab640 1 -- start start 2024-09-12T02:53:18.487 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.728+0000 7ff864bab640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8601080d0 0x7ff86010a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:18.487 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.728+0000 7ff864bab640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff86007be70 con 0x7ff8601080d0 2024-09-12T02:53:18.487 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.728+0000 7ff85e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8601080d0 0x7ff86010a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:18.487 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.728+0000 7ff85e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8601080d0 0x7ff86010a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54198/0 (socket says 172.21.15.116:54198) 2024-09-12T02:53:18.487 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.728+0000 7ff85e59c640 1 -- 172.21.15.116:0/1005395542 learned_addr learned my addr 172.21.15.116:0/1005395542 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:18.487 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.729+0000 7ff85e59c640 1 -- 172.21.15.116:0/1005395542 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff86010aa00 con 0x7ff8601080d0 2024-09-12T02:53:18.487 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.729+0000 7ff85e59c640 1 --2- 172.21.15.116:0/1005395542 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8601080d0 0x7ff86010a4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff850009820 tx=0x7ff85002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c746fdbaf24ca4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:18.487 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.729+0000 7ff85d59a640 1 -- 172.21.15.116:0/1005395542 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff850037070 con 0x7ff8601080d0 2024-09-12T02:53:18.487 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.729+0000 7ff85d59a640 1 -- 172.21.15.116:0/1005395542 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff850032900 con 0x7ff8601080d0 2024-09-12T02:53:18.487 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.730+0000 7ff85d59a640 1 -- 172.21.15.116:0/1005395542 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff850030850 con 0x7ff8601080d0 2024-09-12T02:53:18.488 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.730+0000 7ff864bab640 1 -- 172.21.15.116:0/1005395542 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8601080d0 msgr2=0x7ff86010a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:18.488 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.730+0000 7ff864bab640 1 --2- 172.21.15.116:0/1005395542 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8601080d0 0x7ff86010a4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff850009820 tx=0x7ff85002a270 comp rx=0 tx=0).stop 2024-09-12T02:53:18.488 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.730+0000 7ff864bab640 1 -- 172.21.15.116:0/1005395542 shutdown_connections 2024-09-12T02:53:18.488 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.730+0000 7ff864bab640 1 --2- 172.21.15.116:0/1005395542 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8601080d0 0x7ff86010a4c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:18.488 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.730+0000 7ff864bab640 1 -- 172.21.15.116:0/1005395542 >> 172.21.15.116:0/1005395542 conn(0x7ff860101db0 msgr2=0x7ff8601041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:18.488 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.730+0000 7ff864bab640 1 -- 172.21.15.116:0/1005395542 shutdown_connections 2024-09-12T02:53:18.488 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.730+0000 7ff864bab640 1 -- 172.21.15.116:0/1005395542 wait complete. 2024-09-12T02:53:18.488 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.731+0000 7ff864bab640 1 Processor -- start 2024-09-12T02:53:18.488 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.731+0000 7ff864bab640 1 -- start start 2024-09-12T02:53:18.488 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.731+0000 7ff864bab640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8601080d0 0x7ff86019b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:18.488 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.731+0000 7ff864bab640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff86010ba40 con 0x7ff8601080d0 2024-09-12T02:53:18.489 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.731+0000 7ff85e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8601080d0 0x7ff86019b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:18.489 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.731+0000 7ff85e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8601080d0 0x7ff86019b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54208/0 (socket says 172.21.15.116:54208) 2024-09-12T02:53:18.489 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.731+0000 7ff85e59c640 1 -- 172.21.15.116:0/1483384520 learned_addr learned my addr 172.21.15.116:0/1483384520 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:18.489 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.732+0000 7ff85e59c640 1 -- 172.21.15.116:0/1483384520 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff86019bb50 con 0x7ff8601080d0 2024-09-12T02:53:18.489 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.732+0000 7ff85e59c640 1 --2- 172.21.15.116:0/1483384520 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8601080d0 0x7ff86019b610 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff850032ce0 tx=0x7ff850032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:18.489 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.732+0000 7ff84f7fe640 1 -- 172.21.15.116:0/1483384520 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff850042070 con 0x7ff8601080d0 2024-09-12T02:53:18.489 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.732+0000 7ff864bab640 1 -- 172.21.15.116:0/1483384520 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff86019bde0 con 0x7ff8601080d0 2024-09-12T02:53:18.489 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.732+0000 7ff84f7fe640 1 -- 172.21.15.116:0/1483384520 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff8500308c0 con 0x7ff8601080d0 2024-09-12T02:53:18.489 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.732+0000 7ff84f7fe640 1 -- 172.21.15.116:0/1483384520 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff850037040 con 0x7ff8601080d0 2024-09-12T02:53:18.489 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.732+0000 7ff864bab640 1 -- 172.21.15.116:0/1483384520 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff86019ead0 con 0x7ff8601080d0 2024-09-12T02:53:18.489 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.732+0000 7ff84f7fe640 1 -- 172.21.15.116:0/1483384520 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff850045070 con 0x7ff8601080d0 2024-09-12T02:53:18.489 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.732+0000 7ff84f7fe640 1 -- 172.21.15.116:0/1483384520 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff850047bc0 con 0x7ff8601080d0 2024-09-12T02:53:18.489 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.733+0000 7ff864bab640 1 -- 172.21.15.116:0/1483384520 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff860108530 con 0x7ff8601080d0 2024-09-12T02:53:18.490 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.735+0000 7ff84f7fe640 1 -- 172.21.15.116:0/1483384520 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7ff85003c7e0 con 0x7ff8601080d0 2024-09-12T02:53:18.490 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.796+0000 7ff864bab640 1 -- 172.21.15.116:0/1483384520 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff86019f040 con 0x7ff8601080d0 2024-09-12T02:53:18.490 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.797+0000 7ff84f7fe640 1 -- 172.21.15.116:0/1483384520 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7ff85003c9c0 con 0x7ff8601080d0 2024-09-12T02:53:18.490 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.798+0000 7ff864bab640 1 -- 172.21.15.116:0/1483384520 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8601080d0 msgr2=0x7ff86019b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:18.490 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.798+0000 7ff864bab640 1 --2- 172.21.15.116:0/1483384520 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8601080d0 0x7ff86019b610 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff850032ce0 tx=0x7ff850032d10 comp rx=0 tx=0).stop 2024-09-12T02:53:18.490 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.798+0000 7ff864bab640 1 -- 172.21.15.116:0/1483384520 shutdown_connections 2024-09-12T02:53:18.490 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.798+0000 7ff864bab640 1 --2- 172.21.15.116:0/1483384520 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8601080d0 0x7ff86019b610 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:18.490 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.798+0000 7ff864bab640 1 -- 172.21.15.116:0/1483384520 >> 172.21.15.116:0/1483384520 conn(0x7ff860101db0 msgr2=0x7ff8601028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:18.490 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.798+0000 7ff864bab640 1 -- 172.21.15.116:0/1483384520 shutdown_connections 2024-09-12T02:53:18.490 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:17.798+0000 7ff864bab640 1 -- 172.21.15.116:0/1483384520 wait complete. 2024-09-12T02:53:18.490 INFO:teuthology.orchestra.run.smithi116.stdout:mgr not available, waiting (2/15)... 2024-09-12T02:53:19.302 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:18 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:18.967+0000 7f44ed715040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T02:53:19.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:19 smithi116 ceph-mon[31410]: Activating manager daemon x 2024-09-12T02:53:19.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:19 smithi116 ceph-mon[31410]: mgrmap e2: x(active, starting, since 0.00324236s) 2024-09-12T02:53:19.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:19 smithi116 ceph-mon[31410]: from='mgr.14100 172.21.15.116:0/232184890' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T02:53:19.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:19 smithi116 ceph-mon[31410]: from='mgr.14100 172.21.15.116:0/232184890' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T02:53:19.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:19 smithi116 ceph-mon[31410]: from='mgr.14100 172.21.15.116:0/232184890' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T02:53:19.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:19 smithi116 ceph-mon[31410]: from='mgr.14100 172.21.15.116:0/232184890' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:53:19.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:19 smithi116 ceph-mon[31410]: from='mgr.14100 172.21.15.116:0/232184890' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T02:53:19.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:19 smithi116 ceph-mon[31410]: Manager daemon x is now available 2024-09-12T02:53:19.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:19 smithi116 ceph-mon[31410]: from='mgr.14100 172.21.15.116:0/232184890' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T02:53:19.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:19 smithi116 ceph-mon[31410]: from='mgr.14100 172.21.15.116:0/232184890' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T02:53:19.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:19 smithi116 ceph-mon[31410]: from='mgr.14100 172.21.15.116:0/232184890' entity='mgr.x' 2024-09-12T02:53:19.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:19 smithi116 ceph-mon[31410]: from='mgr.14100 172.21.15.116:0/232184890' entity='mgr.x' 2024-09-12T02:53:19.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:19 smithi116 ceph-mon[31410]: from='mgr.14100 172.21.15.116:0/232184890' entity='mgr.x' 2024-09-12T02:53:21.174 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:20 smithi116 ceph-mon[31410]: mgrmap e3: x(active, since 1.00697s) 2024-09-12T02:53:22.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:21 smithi116 ceph-mon[31410]: mgrmap e4: x(active, since 2s) 2024-09-12T02:53:22.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:21 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1065783800' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T02:53:23.847 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout 2024-09-12T02:53:23.847 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:53:23.847 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "fsid": "eb1ac602-70b1-11ef-bcea-c7b262605968", 2024-09-12T02:53:23.847 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T02:53:23.847 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T02:53:23.847 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T02:53:23.847 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T02:53:23.847 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:53:23.847 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T02:53:23.847 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T02:53:23.848 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout 0 2024-09-12T02:53:23.848 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:53:23.848 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T02:53:23.848 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T02:53:23.848 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:53:23.848 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-12T02:53:23.848 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T02:53:23.848 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:53:23.848 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T02:53:23.848 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T02:53:23.848 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:53:23.848 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T02:53:23.849 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:53:23.849 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T02:53:23.849 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T02:53:23.849 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T02:53:23.857 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T02:53:23.857 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T02:53:23.857 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T02:53:23.857 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:53:23.858 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T02:53:23.858 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T02:53:23.858 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T02:53:23.858 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T02:53:23.858 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T02:53:23.858 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T02:53:23.858 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T02:53:23.858 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T02:53:23.858 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T02:53:23.858 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:53:23.858 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T02:53:23.858 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:53:23.858 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T02:53:07:105500+0000", 2024-09-12T02:53:23.858 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T02:53:23.859 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T02:53:23.859 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:53:23.859 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T02:53:23.859 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T02:53:23.859 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T02:53:23.859 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T02:53:23.859 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T02:53:23.859 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T02:53:23.859 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T02:53:23.859 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:53:23.859 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T02:53:23.859 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:53:23.859 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T02:53:23.859 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:53:23.859 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T02:53:07.106143+0000", 2024-09-12T02:53:23.860 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T02:53:23.860 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:53:23.860 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T02:53:23.860 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:53:23.860 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.931+0000 7fb4c95d0640 1 Processor -- start 2024-09-12T02:53:23.860 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.931+0000 7fb4c95d0640 1 -- start start 2024-09-12T02:53:23.860 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.931+0000 7fb4c95d0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4c4106520 0x7fb4c4106920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:23.860 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.931+0000 7fb4c95d0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c4106ef0 con 0x7fb4c4106520 2024-09-12T02:53:23.860 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.931+0000 7fb4c2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4c4106520 0x7fb4c4106920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:23.860 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.931+0000 7fb4c2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4c4106520 0x7fb4c4106920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54288/0 (socket says 172.21.15.116:54288) 2024-09-12T02:53:23.860 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.931+0000 7fb4c2ffd640 1 -- 172.21.15.116:0/2961189287 learned_addr learned my addr 172.21.15.116:0/2961189287 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:23.860 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.931+0000 7fb4c2ffd640 1 -- 172.21.15.116:0/2961189287 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c4107720 con 0x7fb4c4106520 2024-09-12T02:53:23.860 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.932+0000 7fb4c2ffd640 1 --2- 172.21.15.116:0/2961189287 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4c4106520 0x7fb4c4106920 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4009820 tx=0x7fb4b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b5d0b4c65afc41c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:23.860 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.932+0000 7fb4c27fc640 1 -- 172.21.15.116:0/2961189287 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4b4037070 con 0x7fb4c4106520 2024-09-12T02:53:23.861 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.932+0000 7fb4c27fc640 1 -- 172.21.15.116:0/2961189287 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb4b4032900 con 0x7fb4c4106520 2024-09-12T02:53:23.861 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.932+0000 7fb4c27fc640 1 -- 172.21.15.116:0/2961189287 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4b4030850 con 0x7fb4c4106520 2024-09-12T02:53:23.861 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.933+0000 7fb4c95d0640 1 -- 172.21.15.116:0/2961189287 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4c4106520 msgr2=0x7fb4c4106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:23.861 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.933+0000 7fb4c95d0640 1 --2- 172.21.15.116:0/2961189287 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4c4106520 0x7fb4c4106920 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4009820 tx=0x7fb4b402a270 comp rx=0 tx=0).stop 2024-09-12T02:53:23.861 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.933+0000 7fb4c95d0640 1 -- 172.21.15.116:0/2961189287 shutdown_connections 2024-09-12T02:53:23.861 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.933+0000 7fb4c95d0640 1 --2- 172.21.15.116:0/2961189287 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4c4106520 0x7fb4c4106920 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:23.861 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.933+0000 7fb4c95d0640 1 -- 172.21.15.116:0/2961189287 >> 172.21.15.116:0/2961189287 conn(0x7fb4c4101cb0 msgr2=0x7fb4c41040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:23.861 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.933+0000 7fb4c95d0640 1 -- 172.21.15.116:0/2961189287 shutdown_connections 2024-09-12T02:53:23.861 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.933+0000 7fb4c95d0640 1 -- 172.21.15.116:0/2961189287 wait complete. 2024-09-12T02:53:23.861 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.934+0000 7fb4c95d0640 1 Processor -- start 2024-09-12T02:53:23.861 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.934+0000 7fb4c95d0640 1 -- start start 2024-09-12T02:53:23.861 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.934+0000 7fb4c95d0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4c4106520 0x7fb4c4191750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:23.861 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.934+0000 7fb4c95d0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c410aad0 con 0x7fb4c4106520 2024-09-12T02:53:23.861 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.934+0000 7fb4c2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4c4106520 0x7fb4c4191750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:23.862 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.934+0000 7fb4c2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4c4106520 0x7fb4c4191750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54304/0 (socket says 172.21.15.116:54304) 2024-09-12T02:53:23.862 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.934+0000 7fb4c2ffd640 1 -- 172.21.15.116:0/1065783800 learned_addr learned my addr 172.21.15.116:0/1065783800 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:23.862 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.934+0000 7fb4c2ffd640 1 -- 172.21.15.116:0/1065783800 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c4191c90 con 0x7fb4c4106520 2024-09-12T02:53:23.862 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.935+0000 7fb4c2ffd640 1 --2- 172.21.15.116:0/1065783800 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4c4106520 0x7fb4c4191750 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4009940 tx=0x7fb4b4032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:23.862 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.935+0000 7fb4c0ff9640 1 -- 172.21.15.116:0/1065783800 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4b4042070 con 0x7fb4c4106520 2024-09-12T02:53:23.862 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.935+0000 7fb4c95d0640 1 -- 172.21.15.116:0/1065783800 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4c418fe80 con 0x7fb4c4106520 2024-09-12T02:53:23.862 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.935+0000 7fb4c0ff9640 1 -- 172.21.15.116:0/1065783800 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb4b40308c0 con 0x7fb4c4106520 2024-09-12T02:53:23.862 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.935+0000 7fb4c0ff9640 1 -- 172.21.15.116:0/1065783800 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4b4037040 con 0x7fb4c4106520 2024-09-12T02:53:23.862 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.935+0000 7fb4c95d0640 1 -- 172.21.15.116:0/1065783800 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4c41903c0 con 0x7fb4c4106520 2024-09-12T02:53:23.862 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.936+0000 7fb4c0ff9640 1 -- 172.21.15.116:0/1065783800 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7fb4b4045070 con 0x7fb4c4106520 2024-09-12T02:53:23.862 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.936+0000 7fb4c95d0640 1 -- 172.21.15.116:0/1065783800 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4c410ac50 con 0x7fb4c4106520 2024-09-12T02:53:23.862 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.936+0000 7fb4c0ff9640 1 --2- 172.21.15.116:0/1065783800 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7fb4b0038e10 0x7fb4b003b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:23.862 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.936+0000 7fb4c0ff9640 1 -- 172.21.15.116:0/1065783800 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb4b406bdb0 con 0x7fb4c4106520 2024-09-12T02:53:23.862 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.936+0000 7fb4ba7fc640 1 --2- 172.21.15.116:0/1065783800 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7fb4b0038e10 0x7fb4b003b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:23.862 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.937+0000 7fb4ba7fc640 1 --2- 172.21.15.116:0/1065783800 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7fb4b0038e10 0x7fb4b003b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac0098c0 tx=0x7fb4ac006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:23.863 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.941+0000 7fb4c0ff9640 1 -- 172.21.15.116:0/1065783800 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb4b404b170 con 0x7fb4c4106520 2024-09-12T02:53:23.863 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:20.976+0000 7fb4c0ff9640 1 -- 172.21.15.116:0/1065783800 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fb4b403c870 con 0x7fb4c4106520 2024-09-12T02:53:23.863 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:21.171+0000 7fb4c95d0640 1 -- 172.21.15.116:0/1065783800 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb4c4195060 con 0x7fb4c4106520 2024-09-12T02:53:23.863 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:21.172+0000 7fb4c0ff9640 1 -- 172.21.15.116:0/1065783800 <== mon.0 v2:172.21.15.116:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7fb4c4195060 con 0x7fb4c4106520 2024-09-12T02:53:23.863 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:21.174+0000 7fb4c95d0640 1 -- 172.21.15.116:0/1065783800 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7fb4b0038e10 msgr2=0x7fb4b003b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:23.863 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:21.174+0000 7fb4c95d0640 1 --2- 172.21.15.116:0/1065783800 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7fb4b0038e10 0x7fb4b003b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac0098c0 tx=0x7fb4ac006db0 comp rx=0 tx=0).stop 2024-09-12T02:53:23.863 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:21.174+0000 7fb4c95d0640 1 -- 172.21.15.116:0/1065783800 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4c4106520 msgr2=0x7fb4c4191750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:23.863 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:21.174+0000 7fb4c95d0640 1 --2- 172.21.15.116:0/1065783800 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4c4106520 0x7fb4c4191750 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4009940 tx=0x7fb4b4032ce0 comp rx=0 tx=0).stop 2024-09-12T02:53:23.863 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:21.174+0000 7fb4c95d0640 1 -- 172.21.15.116:0/1065783800 shutdown_connections 2024-09-12T02:53:23.863 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:21.174+0000 7fb4c95d0640 1 --2- 172.21.15.116:0/1065783800 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7fb4b0038e10 0x7fb4b003b2d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:23.863 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:21.174+0000 7fb4c95d0640 1 --2- 172.21.15.116:0/1065783800 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4c4106520 0x7fb4c4191750 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:23.863 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:21.174+0000 7fb4c95d0640 1 -- 172.21.15.116:0/1065783800 >> 172.21.15.116:0/1065783800 conn(0x7fb4c4101cb0 msgr2=0x7fb4c4102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:23.863 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:21.174+0000 7fb4c95d0640 1 -- 172.21.15.116:0/1065783800 shutdown_connections 2024-09-12T02:53:23.863 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:21.174+0000 7fb4c95d0640 1 -- 172.21.15.116:0/1065783800 wait complete. 2024-09-12T02:53:23.863 INFO:teuthology.orchestra.run.smithi116.stdout:mgr is available 2024-09-12T02:53:27.196 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:26 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/947675535' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-12T02:53:33.395 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout 2024-09-12T02:53:33.402 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout [global] 2024-09-12T02:53:33.403 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout fsid = eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T02:53:33.403 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.116:3300,v1:172.21.15.116:6789] 2024-09-12T02:53:33.403 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-12T02:53:33.403 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-12T02:53:33.403 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-12T02:53:33.403 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-12T02:53:33.403 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout 2024-09-12T02:53:33.403 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-12T02:53:33.403 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-12T02:53:33.403 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout 2024-09-12T02:53:33.403 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout [osd] 2024-09-12T02:53:33.403 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-12T02:53:33.403 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-12T02:53:33.403 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-12T02:53:33.404 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-12T02:53:33.404 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-12T02:53:33.404 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.753+0000 7f3073684640 1 Processor -- start 2024-09-12T02:53:33.404 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.753+0000 7f3073684640 1 -- start start 2024-09-12T02:53:33.404 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.753+0000 7f3073684640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f306c07de50 0x7f306c07c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:33.404 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.753+0000 7f3073684640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f306c07c840 con 0x7f306c07de50 2024-09-12T02:53:33.404 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.753+0000 7f3071420640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f306c07de50 0x7f306c07c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:33.404 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.753+0000 7f3071420640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f306c07de50 0x7f306c07c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45976/0 (socket says 172.21.15.116:45976) 2024-09-12T02:53:33.404 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.753+0000 7f3071420640 1 -- 172.21.15.116:0/3624391743 learned_addr learned my addr 172.21.15.116:0/3624391743 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:33.404 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.754+0000 7f3071420640 1 -- 172.21.15.116:0/3624391743 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f306c07c9c0 con 0x7f306c07de50 2024-09-12T02:53:33.404 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.754+0000 7f3071420640 1 --2- 172.21.15.116:0/3624391743 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f306c07de50 0x7f306c07c300 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3068009820 tx=0x7f306802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e19f7ea3d5f6c49 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:33.404 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.755+0000 7f3063fff640 1 -- 172.21.15.116:0/3624391743 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3068037070 con 0x7f306c07de50 2024-09-12T02:53:33.404 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.755+0000 7f3063fff640 1 -- 172.21.15.116:0/3624391743 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3068032900 con 0x7f306c07de50 2024-09-12T02:53:33.404 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.755+0000 7f3063fff640 1 -- 172.21.15.116:0/3624391743 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30680308e0 con 0x7f306c07de50 2024-09-12T02:53:33.405 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.755+0000 7f3073684640 1 -- 172.21.15.116:0/3624391743 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f306c07de50 msgr2=0x7f306c07c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:33.405 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.755+0000 7f3073684640 1 --2- 172.21.15.116:0/3624391743 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f306c07de50 0x7f306c07c300 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3068009820 tx=0x7f306802a300 comp rx=0 tx=0).stop 2024-09-12T02:53:33.405 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.755+0000 7f3073684640 1 -- 172.21.15.116:0/3624391743 shutdown_connections 2024-09-12T02:53:33.405 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.755+0000 7f3073684640 1 --2- 172.21.15.116:0/3624391743 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f306c07de50 0x7f306c07c300 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:33.405 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.755+0000 7f3073684640 1 -- 172.21.15.116:0/3624391743 >> 172.21.15.116:0/3624391743 conn(0x7f306c102e50 msgr2=0x7f306c105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:33.405 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.755+0000 7f3073684640 1 -- 172.21.15.116:0/3624391743 shutdown_connections 2024-09-12T02:53:33.405 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.755+0000 7f3073684640 1 -- 172.21.15.116:0/3624391743 wait complete. 2024-09-12T02:53:33.405 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.756+0000 7f3073684640 1 Processor -- start 2024-09-12T02:53:33.405 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.756+0000 7f3073684640 1 -- start start 2024-09-12T02:53:33.405 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.756+0000 7f3073684640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f306c07de50 0x7f306c19d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:33.405 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.756+0000 7f3073684640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f306c10dc90 con 0x7f306c07de50 2024-09-12T02:53:33.405 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.756+0000 7f3071420640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f306c07de50 0x7f306c19d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:33.405 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.756+0000 7f3071420640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f306c07de50 0x7f306c19d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45986/0 (socket says 172.21.15.116:45986) 2024-09-12T02:53:33.405 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.756+0000 7f3071420640 1 -- 172.21.15.116:0/947675535 learned_addr learned my addr 172.21.15.116:0/947675535 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:33.406 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.756+0000 7f3071420640 1 -- 172.21.15.116:0/947675535 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f306c19dde0 con 0x7f306c07de50 2024-09-12T02:53:33.406 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.757+0000 7f3071420640 1 --2- 172.21.15.116:0/947675535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f306c07de50 0x7f306c19d8a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3068030610 tx=0x7f3068032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:33.406 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.757+0000 7f30627fc640 1 -- 172.21.15.116:0/947675535 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3068037040 con 0x7f306c07de50 2024-09-12T02:53:33.406 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.757+0000 7f30627fc640 1 -- 172.21.15.116:0/947675535 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3068040070 con 0x7f306c07de50 2024-09-12T02:53:33.406 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.757+0000 7f3073684640 1 -- 172.21.15.116:0/947675535 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f306c19e070 con 0x7f306c07de50 2024-09-12T02:53:33.406 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.757+0000 7f30627fc640 1 -- 172.21.15.116:0/947675535 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f306803a390 con 0x7f306c07de50 2024-09-12T02:53:33.406 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.757+0000 7f3073684640 1 -- 172.21.15.116:0/947675535 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f306c1a0d60 con 0x7f306c07de50 2024-09-12T02:53:33.406 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.758+0000 7f30627fc640 1 -- 172.21.15.116:0/947675535 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f3068032900 con 0x7f306c07de50 2024-09-12T02:53:33.406 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.758+0000 7f30627fc640 1 --2- 172.21.15.116:0/947675535 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7f3058038f70 0x7f305803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:33.406 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.758+0000 7f30627fc640 1 -- 172.21.15.116:0/947675535 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f306806be20 con 0x7f306c07de50 2024-09-12T02:53:33.406 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.758+0000 7f3073684640 1 -- 172.21.15.116:0/947675535 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f306c07c380 con 0x7f306c07de50 2024-09-12T02:53:33.406 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.758+0000 7f3070c1f640 1 --2- 172.21.15.116:0/947675535 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7f3058038f70 0x7f305803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:33.406 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.758+0000 7f3070c1f640 1 --2- 172.21.15.116:0/947675535 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7f3058038f70 0x7f305803b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f305c0098c0 tx=0x7f305c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:33.407 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.763+0000 7f30627fc640 1 -- 172.21.15.116:0/947675535 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3068043360 con 0x7f306c07de50 2024-09-12T02:53:33.407 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.931+0000 7f3073684640 1 -- 172.21.15.116:0/947675535 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f306c1a0f30 con 0x7f306c07de50 2024-09-12T02:53:33.407 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.932+0000 7f30627fc640 1 -- 172.21.15.116:0/947675535 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+431 (secure 0 0 0) 0x7f306804a200 con 0x7f306c07de50 2024-09-12T02:53:33.407 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.934+0000 7f3073684640 1 -- 172.21.15.116:0/947675535 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7f3058038f70 msgr2=0x7f305803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:33.407 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.934+0000 7f3073684640 1 --2- 172.21.15.116:0/947675535 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7f3058038f70 0x7f305803b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f305c0098c0 tx=0x7f305c006db0 comp rx=0 tx=0).stop 2024-09-12T02:53:33.407 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.934+0000 7f3073684640 1 -- 172.21.15.116:0/947675535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f306c07de50 msgr2=0x7f306c19d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:33.407 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.934+0000 7f3073684640 1 --2- 172.21.15.116:0/947675535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f306c07de50 0x7f306c19d8a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3068030610 tx=0x7f3068032670 comp rx=0 tx=0).stop 2024-09-12T02:53:33.407 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.934+0000 7f3073684640 1 -- 172.21.15.116:0/947675535 shutdown_connections 2024-09-12T02:53:33.407 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.934+0000 7f3073684640 1 --2- 172.21.15.116:0/947675535 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7f3058038f70 0x7f305803b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:33.407 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.934+0000 7f3073684640 1 --2- 172.21.15.116:0/947675535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f306c07de50 0x7f306c19d8a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:33.407 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.934+0000 7f3073684640 1 -- 172.21.15.116:0/947675535 >> 172.21.15.116:0/947675535 conn(0x7f306c102e50 msgr2=0x7f306c103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:33.407 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.935+0000 7f3073684640 1 -- 172.21.15.116:0/947675535 shutdown_connections 2024-09-12T02:53:33.408 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:26.935+0000 7f3073684640 1 -- 172.21.15.116:0/947675535 wait complete. 2024-09-12T02:53:33.408 INFO:teuthology.orchestra.run.smithi116.stdout:Enabling cephadm module... 2024-09-12T02:53:34.939 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:34 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/414621197' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-12T02:53:34.939 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:34 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: ignoring --setuser ceph since I am not root 2024-09-12T02:53:34.939 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:34 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: ignoring --setgroup ceph since I am not root 2024-09-12T02:53:34.939 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:34 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:34.730+0000 7ff57c5c4040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T02:53:34.940 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:34 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:34.787+0000 7ff57c5c4040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T02:53:34.940 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:34 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:34.938+0000 7ff57c5c4040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T02:53:35.193 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.318+0000 7ff48c4fc640 1 Processor -- start 2024-09-12T02:53:35.194 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.319+0000 7ff48c4fc640 1 -- start start 2024-09-12T02:53:35.194 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.319+0000 7ff48c4fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4841081c0 0x7ff48410a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:35.194 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.319+0000 7ff48c4fc640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff48407bdf0 con 0x7ff4841081c0 2024-09-12T02:53:35.194 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.319+0000 7ff48a298640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4841081c0 0x7ff48410a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:35.194 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.319+0000 7ff48a298640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4841081c0 0x7ff48410a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52466/0 (socket says 172.21.15.116:52466) 2024-09-12T02:53:35.194 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.319+0000 7ff48a298640 1 -- 172.21.15.116:0/1003987474 learned_addr learned my addr 172.21.15.116:0/1003987474 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:35.194 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.319+0000 7ff48a298640 1 -- 172.21.15.116:0/1003987474 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48410aaf0 con 0x7ff4841081c0 2024-09-12T02:53:35.194 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.320+0000 7ff48a298640 1 --2- 172.21.15.116:0/1003987474 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4841081c0 0x7ff48410a5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff480009820 tx=0x7ff48002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb3e92fb2566cf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:35.194 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.320+0000 7ff489296640 1 -- 172.21.15.116:0/1003987474 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff480037070 con 0x7ff4841081c0 2024-09-12T02:53:35.194 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.320+0000 7ff489296640 1 -- 172.21.15.116:0/1003987474 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff480032900 con 0x7ff4841081c0 2024-09-12T02:53:35.194 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.320+0000 7ff489296640 1 -- 172.21.15.116:0/1003987474 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff480030850 con 0x7ff4841081c0 2024-09-12T02:53:35.195 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.321+0000 7ff48c4fc640 1 -- 172.21.15.116:0/1003987474 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4841081c0 msgr2=0x7ff48410a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:35.195 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.321+0000 7ff48c4fc640 1 --2- 172.21.15.116:0/1003987474 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4841081c0 0x7ff48410a5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff480009820 tx=0x7ff48002a270 comp rx=0 tx=0).stop 2024-09-12T02:53:35.195 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.321+0000 7ff48c4fc640 1 -- 172.21.15.116:0/1003987474 shutdown_connections 2024-09-12T02:53:35.195 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.321+0000 7ff48c4fc640 1 --2- 172.21.15.116:0/1003987474 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4841081c0 0x7ff48410a5b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:35.195 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.321+0000 7ff48c4fc640 1 -- 172.21.15.116:0/1003987474 >> 172.21.15.116:0/1003987474 conn(0x7ff484101de0 msgr2=0x7ff484104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:35.195 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.321+0000 7ff48c4fc640 1 -- 172.21.15.116:0/1003987474 shutdown_connections 2024-09-12T02:53:35.195 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.321+0000 7ff48c4fc640 1 -- 172.21.15.116:0/1003987474 wait complete. 2024-09-12T02:53:35.195 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.322+0000 7ff48c4fc640 1 Processor -- start 2024-09-12T02:53:35.195 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.322+0000 7ff48c4fc640 1 -- start start 2024-09-12T02:53:35.195 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.322+0000 7ff48c4fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4841081c0 0x7ff48419b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:35.195 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.322+0000 7ff48c4fc640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff48410ba60 con 0x7ff4841081c0 2024-09-12T02:53:35.195 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.322+0000 7ff48a298640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4841081c0 0x7ff48419b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:35.195 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.322+0000 7ff48a298640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4841081c0 0x7ff48419b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52476/0 (socket says 172.21.15.116:52476) 2024-09-12T02:53:35.196 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.322+0000 7ff48a298640 1 -- 172.21.15.116:0/414621197 learned_addr learned my addr 172.21.15.116:0/414621197 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:35.196 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.322+0000 7ff48a298640 1 -- 172.21.15.116:0/414621197 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48419bb20 con 0x7ff4841081c0 2024-09-12T02:53:35.196 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.322+0000 7ff48a298640 1 --2- 172.21.15.116:0/414621197 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4841081c0 0x7ff48419b5e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff480032ce0 tx=0x7ff480032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:35.196 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.323+0000 7ff47b7fe640 1 -- 172.21.15.116:0/414621197 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff480042070 con 0x7ff4841081c0 2024-09-12T02:53:35.196 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.323+0000 7ff47b7fe640 1 -- 172.21.15.116:0/414621197 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff4800308c0 con 0x7ff4841081c0 2024-09-12T02:53:35.196 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.323+0000 7ff48c4fc640 1 -- 172.21.15.116:0/414621197 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff48419bdb0 con 0x7ff4841081c0 2024-09-12T02:53:35.196 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.323+0000 7ff47b7fe640 1 -- 172.21.15.116:0/414621197 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff480037040 con 0x7ff4841081c0 2024-09-12T02:53:35.196 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.323+0000 7ff48c4fc640 1 -- 172.21.15.116:0/414621197 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff48419eaa0 con 0x7ff4841081c0 2024-09-12T02:53:35.196 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.324+0000 7ff47b7fe640 1 -- 172.21.15.116:0/414621197 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7ff480045070 con 0x7ff4841081c0 2024-09-12T02:53:35.196 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.324+0000 7ff47b7fe640 1 --2- 172.21.15.116:0/414621197 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7ff470038f70 0x7ff47003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:35.196 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.324+0000 7ff47b7fe640 1 -- 172.21.15.116:0/414621197 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff48006bfa0 con 0x7ff4841081c0 2024-09-12T02:53:35.196 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.324+0000 7ff48c4fc640 1 -- 172.21.15.116:0/414621197 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff484108620 con 0x7ff4841081c0 2024-09-12T02:53:35.197 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.324+0000 7ff489a97640 1 --2- 172.21.15.116:0/414621197 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7ff470038f70 0x7ff47003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:35.197 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.324+0000 7ff489a97640 1 --2- 172.21.15.116:0/414621197 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7ff470038f70 0x7ff47003b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff4740098c0 tx=0x7ff474006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:35.197 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.329+0000 7ff47b7fe640 1 -- 172.21.15.116:0/414621197 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff48004b170 con 0x7ff4841081c0 2024-09-12T02:53:35.197 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.539+0000 7ff48c4fc640 1 -- 172.21.15.116:0/414621197 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7ff48419efd0 con 0x7ff4841081c0 2024-09-12T02:53:35.197 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.590+0000 7ff47b7fe640 1 -- 172.21.15.116:0/414621197 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7ff48006b9c0 con 0x7ff4841081c0 2024-09-12T02:53:35.197 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.592+0000 7ff47b7fe640 1 -- 172.21.15.116:0/414621197 <== mon.0 v2:172.21.15.116:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7ff48003ccc0 con 0x7ff4841081c0 2024-09-12T02:53:35.197 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.593+0000 7ff48c4fc640 1 -- 172.21.15.116:0/414621197 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7ff470038f70 msgr2=0x7ff47003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:35.197 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.593+0000 7ff48c4fc640 1 --2- 172.21.15.116:0/414621197 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7ff470038f70 0x7ff47003b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff4740098c0 tx=0x7ff474006db0 comp rx=0 tx=0).stop 2024-09-12T02:53:35.198 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.593+0000 7ff48c4fc640 1 -- 172.21.15.116:0/414621197 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4841081c0 msgr2=0x7ff48419b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:35.198 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.593+0000 7ff48c4fc640 1 --2- 172.21.15.116:0/414621197 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4841081c0 0x7ff48419b5e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff480032ce0 tx=0x7ff480032d10 comp rx=0 tx=0).stop 2024-09-12T02:53:35.198 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.593+0000 7ff48c4fc640 1 -- 172.21.15.116:0/414621197 shutdown_connections 2024-09-12T02:53:35.198 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.593+0000 7ff48c4fc640 1 --2- 172.21.15.116:0/414621197 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7ff470038f70 0x7ff47003b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:35.198 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.593+0000 7ff48c4fc640 1 --2- 172.21.15.116:0/414621197 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4841081c0 0x7ff48419b5e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:35.198 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.593+0000 7ff48c4fc640 1 -- 172.21.15.116:0/414621197 >> 172.21.15.116:0/414621197 conn(0x7ff484101de0 msgr2=0x7ff4841028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:35.198 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.594+0000 7ff48c4fc640 1 -- 172.21.15.116:0/414621197 shutdown_connections 2024-09-12T02:53:35.198 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:34.594+0000 7ff48c4fc640 1 -- 172.21.15.116:0/414621197 wait complete. 2024-09-12T02:53:35.302 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:35 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:35.005+0000 7ff57c5c4040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T02:53:35.302 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:35 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:35.156+0000 7ff57c5c4040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:53:35.302 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:35 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:35.301+0000 7ff57c5c4040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T02:53:35.839 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:35 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/414621197' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-12T02:53:35.839 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:35 smithi116 ceph-mon[31410]: mgrmap e5: x(active, since 15s) 2024-09-12T02:53:35.840 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:35 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:35.626+0000 7ff57c5c4040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T02:53:35.840 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:35 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:35.690+0000 7ff57c5c4040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T02:53:36.118 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:35 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:35.882+0000 7ff57c5c4040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T02:53:36.410 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:53:36.410 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-12T02:53:36.410 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T02:53:36.411 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-12T02:53:36.411 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-12T02:53:36.411 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:53:36.411 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.633+0000 7f0ee7d62640 1 Processor -- start 2024-09-12T02:53:36.411 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.633+0000 7f0ee7d62640 1 -- start start 2024-09-12T02:53:36.411 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.633+0000 7f0ee7d62640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ee0108090 0x7f0ee010a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:36.411 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.633+0000 7f0ee7d62640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee007beb0 con 0x7f0ee0108090 2024-09-12T02:53:36.411 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.633+0000 7f0ee5afe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ee0108090 0x7f0ee010a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:36.411 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.633+0000 7f0ee5afe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ee0108090 0x7f0ee010a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52506/0 (socket says 172.21.15.116:52506) 2024-09-12T02:53:36.411 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.633+0000 7f0ee5afe640 1 -- 172.21.15.116:0/3917468593 learned_addr learned my addr 172.21.15.116:0/3917468593 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:36.411 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.633+0000 7f0ee5afe640 1 -- 172.21.15.116:0/3917468593 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee010a9c0 con 0x7f0ee0108090 2024-09-12T02:53:36.411 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.634+0000 7f0ee5afe640 1 --2- 172.21.15.116:0/3917468593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ee0108090 0x7f0ee010a480 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0edc009820 tx=0x7f0edc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eeae78c2464357b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:36.411 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.634+0000 7f0ee4afc640 1 -- 172.21.15.116:0/3917468593 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0edc037070 con 0x7f0ee0108090 2024-09-12T02:53:36.412 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.634+0000 7f0ee4afc640 1 -- 172.21.15.116:0/3917468593 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0edc032900 con 0x7f0ee0108090 2024-09-12T02:53:36.412 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.634+0000 7f0ee4afc640 1 -- 172.21.15.116:0/3917468593 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0edc030850 con 0x7f0ee0108090 2024-09-12T02:53:36.412 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.635+0000 7f0ee7d62640 1 -- 172.21.15.116:0/3917468593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ee0108090 msgr2=0x7f0ee010a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:36.412 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.635+0000 7f0ee7d62640 1 --2- 172.21.15.116:0/3917468593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ee0108090 0x7f0ee010a480 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0edc009820 tx=0x7f0edc02a270 comp rx=0 tx=0).stop 2024-09-12T02:53:36.412 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.635+0000 7f0ee7d62640 1 -- 172.21.15.116:0/3917468593 shutdown_connections 2024-09-12T02:53:36.412 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.635+0000 7f0ee7d62640 1 --2- 172.21.15.116:0/3917468593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ee0108090 0x7f0ee010a480 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:36.412 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.635+0000 7f0ee7d62640 1 -- 172.21.15.116:0/3917468593 >> 172.21.15.116:0/3917468593 conn(0x7f0ee0101db0 msgr2=0x7f0ee01041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:36.412 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.635+0000 7f0ee7d62640 1 -- 172.21.15.116:0/3917468593 shutdown_connections 2024-09-12T02:53:36.413 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.635+0000 7f0ee7d62640 1 -- 172.21.15.116:0/3917468593 wait complete. 2024-09-12T02:53:36.413 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.636+0000 7f0ee7d62640 1 Processor -- start 2024-09-12T02:53:36.413 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.636+0000 7f0ee7d62640 1 -- start start 2024-09-12T02:53:36.413 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.636+0000 7f0ee7d62640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ee0108090 0x7f0ee019b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:36.413 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.636+0000 7f0ee7d62640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee010ba60 con 0x7f0ee0108090 2024-09-12T02:53:36.413 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.636+0000 7f0ee5afe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ee0108090 0x7f0ee019b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:36.413 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.636+0000 7f0ee5afe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ee0108090 0x7f0ee019b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52518/0 (socket says 172.21.15.116:52518) 2024-09-12T02:53:36.413 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.636+0000 7f0ee5afe640 1 -- 172.21.15.116:0/243634668 learned_addr learned my addr 172.21.15.116:0/243634668 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:36.413 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.636+0000 7f0ee5afe640 1 -- 172.21.15.116:0/243634668 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee019bb50 con 0x7f0ee0108090 2024-09-12T02:53:36.413 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.636+0000 7f0ee5afe640 1 --2- 172.21.15.116:0/243634668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ee0108090 0x7f0ee019b610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0097f0 tx=0x7f0edc030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:36.413 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.637+0000 7f0ed6ffd640 1 -- 172.21.15.116:0/243634668 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0edc037040 con 0x7f0ee0108090 2024-09-12T02:53:36.413 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.637+0000 7f0ee7d62640 1 -- 172.21.15.116:0/243634668 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ee019bde0 con 0x7f0ee0108090 2024-09-12T02:53:36.414 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.637+0000 7f0ed6ffd640 1 -- 172.21.15.116:0/243634668 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0edc040070 con 0x7f0ee0108090 2024-09-12T02:53:36.414 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.637+0000 7f0ed6ffd640 1 -- 172.21.15.116:0/243634668 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0edc03a390 con 0x7f0ee0108090 2024-09-12T02:53:36.414 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.637+0000 7f0ee7d62640 1 -- 172.21.15.116:0/243634668 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ee019ead0 con 0x7f0ee0108090 2024-09-12T02:53:36.414 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.638+0000 7f0ed6ffd640 1 -- 172.21.15.116:0/243634668 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f0edc03b430 con 0x7f0ee0108090 2024-09-12T02:53:36.414 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.638+0000 7f0ed6ffd640 1 --2- 172.21.15.116:0/243634668 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7f0ecc038f70 0x7f0ecc03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:36.414 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.638+0000 7f0ee7d62640 1 -- 172.21.15.116:0/243634668 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ee01084f0 con 0x7f0ee0108090 2024-09-12T02:53:36.414 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.638+0000 7f0ee52fd640 1 -- 172.21.15.116:0/243634668 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7f0ecc038f70 msgr2=0x7f0ecc03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.116:6800/1791219601 2024-09-12T02:53:36.414 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.638+0000 7f0ed6ffd640 1 -- 172.21.15.116:0/243634668 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0edc06c150 con 0x7f0ee0108090 2024-09-12T02:53:36.414 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.638+0000 7f0ee52fd640 1 --2- 172.21.15.116:0/243634668 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7f0ecc038f70 0x7f0ecc03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:53:36.414 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.643+0000 7f0ed6ffd640 1 -- 172.21.15.116:0/243634668 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0edc06beb0 con 0x7f0ee0108090 2024-09-12T02:53:36.414 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.839+0000 7f0ee52fd640 1 -- 172.21.15.116:0/243634668 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7f0ecc038f70 msgr2=0x7f0ecc03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.116:6800/1791219601 2024-09-12T02:53:36.415 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.839+0000 7f0ee52fd640 1 --2- 172.21.15.116:0/243634668 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7f0ecc038f70 0x7f0ecc03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T02:53:36.415 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.861+0000 7f0ee7d62640 1 -- 172.21.15.116:0/243634668 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f0ee019eca0 con 0x7f0ee0108090 2024-09-12T02:53:36.415 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.861+0000 7f0ed6ffd640 1 -- 172.21.15.116:0/243634668 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f0edc03a670 con 0x7f0ee0108090 2024-09-12T02:53:36.415 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.863+0000 7f0ee7d62640 1 -- 172.21.15.116:0/243634668 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7f0ecc038f70 msgr2=0x7f0ecc03b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:53:36.415 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.863+0000 7f0ee7d62640 1 --2- 172.21.15.116:0/243634668 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7f0ecc038f70 0x7f0ecc03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:36.415 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.863+0000 7f0ee7d62640 1 -- 172.21.15.116:0/243634668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ee0108090 msgr2=0x7f0ee019b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:36.415 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.863+0000 7f0ee7d62640 1 --2- 172.21.15.116:0/243634668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ee0108090 0x7f0ee019b610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0097f0 tx=0x7f0edc030d80 comp rx=0 tx=0).stop 2024-09-12T02:53:36.415 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.863+0000 7f0ee7d62640 1 -- 172.21.15.116:0/243634668 shutdown_connections 2024-09-12T02:53:36.415 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.863+0000 7f0ee7d62640 1 --2- 172.21.15.116:0/243634668 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7f0ecc038f70 0x7f0ecc03b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:36.415 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.863+0000 7f0ee7d62640 1 --2- 172.21.15.116:0/243634668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ee0108090 0x7f0ee019b610 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:36.415 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.863+0000 7f0ee7d62640 1 -- 172.21.15.116:0/243634668 >> 172.21.15.116:0/243634668 conn(0x7f0ee0101db0 msgr2=0x7f0ee01028c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:36.415 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.863+0000 7f0ee7d62640 1 -- 172.21.15.116:0/243634668 shutdown_connections 2024-09-12T02:53:36.415 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:35.863+0000 7f0ee7d62640 1 -- 172.21.15.116:0/243634668 wait complete. 2024-09-12T02:53:36.415 INFO:teuthology.orchestra.run.smithi116.stdout:Waiting for the mgr to restart... 2024-09-12T02:53:36.416 INFO:teuthology.orchestra.run.smithi116.stdout:Waiting for mgr epoch 5... 2024-09-12T02:53:36.754 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:36 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/243634668' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-12T02:53:37.038 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:36 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:36.786+0000 7ff57c5c4040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T02:53:37.038 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:36 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:36.857+0000 7ff57c5c4040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T02:53:37.802 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:37 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:37.565+0000 7ff57c5c4040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T02:53:37.802 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:37 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:37.683+0000 7ff57c5c4040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T02:53:38.098 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:37 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:37.824+0000 7ff57c5c4040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:53:38.099 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:37 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:37.884+0000 7ff57c5c4040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T02:53:38.099 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:38 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:38.032+0000 7ff57c5c4040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T02:53:38.099 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:38 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:38.098+0000 7ff57c5c4040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T02:53:38.388 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:38 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:38.158+0000 7ff57c5c4040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T02:53:38.388 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:38 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:38.323+0000 7ff57c5c4040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T02:53:38.389 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:38 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:38.388+0000 7ff57c5c4040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T02:53:38.655 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:38 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:38.455+0000 7ff57c5c4040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T02:53:38.655 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:38 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:38.536+0000 7ff57c5c4040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T02:53:38.655 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:38 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:38.596+0000 7ff57c5c4040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T02:53:38.655 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:38 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:38.654+0000 7ff57c5c4040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T02:53:39.802 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:39 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:39.431+0000 7ff57c5c4040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T02:53:39.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:39 smithi116 ceph-mon[31410]: Active manager daemon x restarted 2024-09-12T02:53:39.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:39 smithi116 ceph-mon[31410]: Activating manager daemon x 2024-09-12T02:53:39.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:39 smithi116 ceph-mon[31410]: osdmap e2: 0 total, 0 up, 0 in 2024-09-12T02:53:39.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:39 smithi116 ceph-mon[31410]: mgrmap e6: x(active, starting, since 0.00372783s) 2024-09-12T02:53:39.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:39 smithi116 ceph-mon[31410]: from='mgr.14118 172.21.15.116:0/1263688136' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:53:39.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:39 smithi116 ceph-mon[31410]: from='mgr.14118 172.21.15.116:0/1263688136' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T02:53:39.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:39 smithi116 ceph-mon[31410]: from='mgr.14118 172.21.15.116:0/1263688136' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T02:53:39.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:39 smithi116 ceph-mon[31410]: from='mgr.14118 172.21.15.116:0/1263688136' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T02:53:39.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:39 smithi116 ceph-mon[31410]: from='mgr.14118 172.21.15.116:0/1263688136' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T02:53:39.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:39 smithi116 ceph-mon[31410]: Manager daemon x is now available 2024-09-12T02:53:39.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:39 smithi116 ceph-mon[31410]: from='mgr.14118 172.21.15.116:0/1263688136' entity='mgr.x' 2024-09-12T02:53:39.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:39 smithi116 ceph-mon[31410]: from='mgr.14118 172.21.15.116:0/1263688136' entity='mgr.x' 2024-09-12T02:53:39.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:39 smithi116 ceph-mon[31410]: from='mgr.14118 172.21.15.116:0/1263688136' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:53:39.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:39 smithi116 ceph-mon[31410]: from='mgr.14118 172.21.15.116:0/1263688136' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:53:39.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:39 smithi116 ceph-mon[31410]: from='mgr.14118 172.21.15.116:0/1263688136' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T02:53:39.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:39 smithi116 ceph-mon[31410]: from='mgr.14118 172.21.15.116:0/1263688136' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T02:53:40.733 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:40 smithi116 ceph-mon[31410]: Found migration_current of "None". Setting to last migration. 2024-09-12T02:53:40.733 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:40 smithi116 ceph-mon[31410]: from='mgr.14118 172.21.15.116:0/1263688136' entity='mgr.x' 2024-09-12T02:53:40.733 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:40 smithi116 ceph-mon[31410]: from='mgr.14118 172.21.15.116:0/1263688136' entity='mgr.x' 2024-09-12T02:53:40.733 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:40 smithi116 ceph-mon[31410]: mgrmap e7: x(active, since 1.00523s) 2024-09-12T02:53:41.009 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:53:41.009 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-12T02:53:41.009 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-12T02:53:41.010 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:53:41.010 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.832+0000 7f8d00fb4640 1 Processor -- start 2024-09-12T02:53:41.010 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.832+0000 7f8d00fb4640 1 -- start start 2024-09-12T02:53:41.010 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.832+0000 7f8d00fb4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cfc108080 0x7f8cfc10a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:41.010 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.832+0000 7f8d00fb4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc07be60 con 0x7f8cfc108080 2024-09-12T02:53:41.010 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.832+0000 7f8cfa59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cfc108080 0x7f8cfc10a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:41.010 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.832+0000 7f8cfa59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cfc108080 0x7f8cfc10a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52530/0 (socket says 172.21.15.116:52530) 2024-09-12T02:53:41.010 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.832+0000 7f8cfa59c640 1 -- 172.21.15.116:0/259162018 learned_addr learned my addr 172.21.15.116:0/259162018 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:41.010 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.833+0000 7f8cfa59c640 1 -- 172.21.15.116:0/259162018 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cfc10a9b0 con 0x7f8cfc108080 2024-09-12T02:53:41.010 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.833+0000 7f8cfa59c640 1 --2- 172.21.15.116:0/259162018 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cfc108080 0x7f8cfc10a470 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8cec009820 tx=0x7f8cec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f61b2b750de2dc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:41.010 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.833+0000 7f8cf959a640 1 -- 172.21.15.116:0/259162018 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cec037070 con 0x7f8cfc108080 2024-09-12T02:53:41.010 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.833+0000 7f8cf959a640 1 -- 172.21.15.116:0/259162018 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8cec032900 con 0x7f8cfc108080 2024-09-12T02:53:41.010 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.834+0000 7f8cf959a640 1 -- 172.21.15.116:0/259162018 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cec030850 con 0x7f8cfc108080 2024-09-12T02:53:41.011 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.834+0000 7f8d00fb4640 1 -- 172.21.15.116:0/259162018 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cfc108080 msgr2=0x7f8cfc10a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:41.011 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.834+0000 7f8d00fb4640 1 --2- 172.21.15.116:0/259162018 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cfc108080 0x7f8cfc10a470 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8cec009820 tx=0x7f8cec02a270 comp rx=0 tx=0).stop 2024-09-12T02:53:41.011 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.834+0000 7f8d00fb4640 1 -- 172.21.15.116:0/259162018 shutdown_connections 2024-09-12T02:53:41.011 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.834+0000 7f8d00fb4640 1 --2- 172.21.15.116:0/259162018 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cfc108080 0x7f8cfc10a470 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:41.011 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.834+0000 7f8d00fb4640 1 -- 172.21.15.116:0/259162018 >> 172.21.15.116:0/259162018 conn(0x7f8cfc101ca0 msgr2=0x7f8cfc1040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:41.011 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.834+0000 7f8d00fb4640 1 -- 172.21.15.116:0/259162018 shutdown_connections 2024-09-12T02:53:41.011 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.834+0000 7f8d00fb4640 1 -- 172.21.15.116:0/259162018 wait complete. 2024-09-12T02:53:41.011 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.835+0000 7f8d00fb4640 1 Processor -- start 2024-09-12T02:53:41.011 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.835+0000 7f8d00fb4640 1 -- start start 2024-09-12T02:53:41.011 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.835+0000 7f8d00fb4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cfc108080 0x7f8cfc107900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:41.011 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.835+0000 7f8d00fb4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc10b9f0 con 0x7f8cfc108080 2024-09-12T02:53:41.011 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.835+0000 7f8cfa59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cfc108080 0x7f8cfc107900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:41.011 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.835+0000 7f8cfa59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cfc108080 0x7f8cfc107900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52540/0 (socket says 172.21.15.116:52540) 2024-09-12T02:53:41.012 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.835+0000 7f8cfa59c640 1 -- 172.21.15.116:0/1245739191 learned_addr learned my addr 172.21.15.116:0/1245739191 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:41.012 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.836+0000 7f8cfa59c640 1 -- 172.21.15.116:0/1245739191 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cfc107e40 con 0x7f8cfc108080 2024-09-12T02:53:41.012 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.836+0000 7f8cfa59c640 1 --2- 172.21.15.116:0/1245739191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cfc108080 0x7f8cfc107900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0097f0 tx=0x7f8cec030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:41.012 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.836+0000 7f8ceb7fe640 1 -- 172.21.15.116:0/1245739191 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cec037040 con 0x7f8cfc108080 2024-09-12T02:53:41.012 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.836+0000 7f8ceb7fe640 1 -- 172.21.15.116:0/1245739191 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8cec040070 con 0x7f8cfc108080 2024-09-12T02:53:41.012 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.836+0000 7f8d00fb4640 1 -- 172.21.15.116:0/1245739191 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cfc106030 con 0x7f8cfc108080 2024-09-12T02:53:41.012 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.836+0000 7f8ceb7fe640 1 -- 172.21.15.116:0/1245739191 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cec03a390 con 0x7f8cfc108080 2024-09-12T02:53:41.012 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.836+0000 7f8d00fb4640 1 -- 172.21.15.116:0/1245739191 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cfc106510 con 0x7f8cfc108080 2024-09-12T02:53:41.012 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.837+0000 7f8ceb7fe640 1 -- 172.21.15.116:0/1245739191 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f8cec03a5e0 con 0x7f8cfc108080 2024-09-12T02:53:41.012 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.837+0000 7f8ceb7fe640 1 --2- 172.21.15.116:0/1245739191 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7f8ce4038fc0 0x7f8ce403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:41.012 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.837+0000 7f8ceb7fe640 1 -- 172.21.15.116:0/1245739191 --> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8ce403bb90 con 0x7f8ce4038fc0 2024-09-12T02:53:41.012 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.837+0000 7f8ceb7fe640 1 -- 172.21.15.116:0/1245739191 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8cec06c130 con 0x7f8cfc108080 2024-09-12T02:53:41.013 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.837+0000 7f8cf9d9b640 1 -- 172.21.15.116:0/1245739191 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7f8ce4038fc0 msgr2=0x7f8ce403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.116:6800/1791219601 2024-09-12T02:53:41.013 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:36.837+0000 7f8cf9d9b640 1 --2- 172.21.15.116:0/1245739191 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7f8ce4038fc0 0x7f8ce403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:53:41.013 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:37.038+0000 7f8cf9d9b640 1 -- 172.21.15.116:0/1245739191 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7f8ce4038fc0 msgr2=0x7f8ce403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.116:6800/1791219601 2024-09-12T02:53:41.013 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:37.038+0000 7f8cf9d9b640 1 --2- 172.21.15.116:0/1245739191 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7f8ce4038fc0 0x7f8ce403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T02:53:41.013 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:37.439+0000 7f8cf9d9b640 1 -- 172.21.15.116:0/1245739191 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7f8ce4038fc0 msgr2=0x7f8ce403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.116:6800/1791219601 2024-09-12T02:53:41.013 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:37.439+0000 7f8cf9d9b640 1 --2- 172.21.15.116:0/1245739191 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7f8ce4038fc0 0x7f8ce403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-12T02:53:41.013 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:38.240+0000 7f8cf9d9b640 1 -- 172.21.15.116:0/1245739191 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7f8ce4038fc0 msgr2=0x7f8ce403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.116:6800/1791219601 2024-09-12T02:53:41.013 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:38.240+0000 7f8cf9d9b640 1 --2- 172.21.15.116:0/1245739191 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7f8ce4038fc0 0x7f8ce403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-12T02:53:41.013 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:39.435+0000 7f8ceb7fe640 1 -- 172.21.15.116:0/1245739191 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f8cec06af30 con 0x7f8cfc108080 2024-09-12T02:53:41.013 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:39.435+0000 7f8ceb7fe640 1 -- 172.21.15.116:0/1245739191 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7f8ce4038fc0 msgr2=0x7f8ce403b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:53:41.013 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:39.435+0000 7f8ceb7fe640 1 --2- 172.21.15.116:0/1245739191 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7f8ce4038fc0 0x7f8ce403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:41.013 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:40.435+0000 7f8ceb7fe640 1 -- 172.21.15.116:0/1245739191 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f8cec03a8d0 con 0x7f8cfc108080 2024-09-12T02:53:41.013 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:40.435+0000 7f8ceb7fe640 1 --2- 172.21.15.116:0/1245739191 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f8ce403c8f0 0x7f8ce403ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:41.014 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:40.436+0000 7f8ceb7fe640 1 -- 172.21.15.116:0/1245739191 --> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8ce403bb90 con 0x7f8ce403c8f0 2024-09-12T02:53:41.014 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:40.436+0000 7f8cf9d9b640 1 --2- 172.21.15.116:0/1245739191 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f8ce403c8f0 0x7f8ce403ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:41.014 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:40.436+0000 7f8cf9d9b640 1 --2- 172.21.15.116:0/1245739191 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f8ce403c8f0 0x7f8ce403ece0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0003980 tx=0x7f8cf00092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:41.014 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:40.437+0000 7f8ceb7fe640 1 -- 172.21.15.116:0/1245739191 <== mgr.14118 v2:172.21.15.116:6800/2966274423 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f8ce403bb90 con 0x7f8ce403c8f0 2024-09-12T02:53:41.014 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:40.444+0000 7f8d00fb4640 1 -- 172.21.15.116:0/1245739191 --> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f8cfc1084e0 con 0x7f8ce403c8f0 2024-09-12T02:53:41.014 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:40.444+0000 7f8ceb7fe640 1 -- 172.21.15.116:0/1245739191 <== mgr.14118 v2:172.21.15.116:6800/2966274423 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f8cfc1084e0 con 0x7f8ce403c8f0 2024-09-12T02:53:41.014 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:40.444+0000 7f8d00fb4640 1 -- 172.21.15.116:0/1245739191 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f8ce403c8f0 msgr2=0x7f8ce403ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:41.014 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:40.444+0000 7f8d00fb4640 1 --2- 172.21.15.116:0/1245739191 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f8ce403c8f0 0x7f8ce403ece0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0003980 tx=0x7f8cf00092b0 comp rx=0 tx=0).stop 2024-09-12T02:53:41.014 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:40.444+0000 7f8d00fb4640 1 -- 172.21.15.116:0/1245739191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cfc108080 msgr2=0x7f8cfc107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:41.014 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:40.444+0000 7f8d00fb4640 1 --2- 172.21.15.116:0/1245739191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cfc108080 0x7f8cfc107900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0097f0 tx=0x7f8cec030d80 comp rx=0 tx=0).stop 2024-09-12T02:53:41.014 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:40.444+0000 7f8d00fb4640 1 -- 172.21.15.116:0/1245739191 shutdown_connections 2024-09-12T02:53:41.014 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:40.444+0000 7f8d00fb4640 1 --2- 172.21.15.116:0/1245739191 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f8ce403c8f0 0x7f8ce403ece0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:41.014 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:40.444+0000 7f8d00fb4640 1 --2- 172.21.15.116:0/1245739191 >> [v2:172.21.15.116:6800/1791219601,v1:172.21.15.116:6801/1791219601] conn(0x7f8ce4038fc0 0x7f8ce403b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:41.015 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:40.444+0000 7f8d00fb4640 1 --2- 172.21.15.116:0/1245739191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cfc108080 0x7f8cfc107900 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:41.015 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:40.445+0000 7f8d00fb4640 1 -- 172.21.15.116:0/1245739191 >> 172.21.15.116:0/1245739191 conn(0x7f8cfc101ca0 msgr2=0x7f8cfc102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:41.015 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:40.445+0000 7f8d00fb4640 1 -- 172.21.15.116:0/1245739191 shutdown_connections 2024-09-12T02:53:41.015 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:40.445+0000 7f8d00fb4640 1 -- 172.21.15.116:0/1245739191 wait complete. 2024-09-12T02:53:41.015 INFO:teuthology.orchestra.run.smithi116.stdout:mgr epoch 5 is available 2024-09-12T02:53:41.015 INFO:teuthology.orchestra.run.smithi116.stdout:Setting orchestrator backend to cephadm... 2024-09-12T02:53:41.303 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:41 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:41.080+0000 7ff4e9129640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:02:53:41] ENGINE Error in HTTPServer.serve 2024-09-12T02:53:41.303 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:41 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: Traceback (most recent call last): 2024-09-12T02:53:41.303 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:41 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T02:53:41.303 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:41 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: self._connections.run(self.expiration_interval) 2024-09-12T02:53:41.303 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:41 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T02:53:41.303 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:41 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: self._run(expiration_interval) 2024-09-12T02:53:41.303 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:41 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T02:53:41.304 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:41 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T02:53:41.304 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:41 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T02:53:41.304 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:41 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T02:53:41.304 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:41 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T02:53:41.304 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:41 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: s = self.context.wrap_socket( 2024-09-12T02:53:41.304 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:41 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T02:53:41.304 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:41 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: return self.sslsocket_class._create( 2024-09-12T02:53:41.304 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:41 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T02:53:41.304 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:41 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: self.do_handshake() 2024-09-12T02:53:41.304 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:41 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T02:53:41.304 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:41 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: self._sslobj.do_handshake() 2024-09-12T02:53:41.304 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:41 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T02:53:41.304 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:41 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:41.584 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:41 smithi116 ceph-mon[31410]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-12T02:53:41.584 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:41 smithi116 ceph-mon[31410]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-12T02:53:41.584 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:41 smithi116 ceph-mon[31410]: from='mgr.14118 172.21.15.116:0/1263688136' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:53:42.189 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.388+0000 7efd8bbb9640 1 Processor -- start 2024-09-12T02:53:42.189 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.388+0000 7efd8bbb9640 1 -- start start 2024-09-12T02:53:42.189 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.388+0000 7efd8bbb9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd84107740 0x7efd84107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:42.189 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.388+0000 7efd8bbb9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efd84108110 con 0x7efd84107740 2024-09-12T02:53:42.189 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.388+0000 7efd89955640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd84107740 0x7efd84107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:42.189 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.388+0000 7efd89955640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd84107740 0x7efd84107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33106/0 (socket says 172.21.15.116:33106) 2024-09-12T02:53:42.190 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.388+0000 7efd89955640 1 -- 172.21.15.116:0/2271974157 learned_addr learned my addr 172.21.15.116:0/2271974157 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:42.190 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.389+0000 7efd89955640 1 -- 172.21.15.116:0/2271974157 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd841088a0 con 0x7efd84107740 2024-09-12T02:53:42.190 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.389+0000 7efd89955640 1 --2- 172.21.15.116:0/2271974157 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd84107740 0x7efd84107b40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efd80009820 tx=0x7efd8002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6abb8168cdd8c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:42.190 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.390+0000 7efd88953640 1 -- 172.21.15.116:0/2271974157 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd80037070 con 0x7efd84107740 2024-09-12T02:53:42.190 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.390+0000 7efd88953640 1 -- 172.21.15.116:0/2271974157 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efd80032900 con 0x7efd84107740 2024-09-12T02:53:42.190 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.390+0000 7efd88953640 1 -- 172.21.15.116:0/2271974157 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd80030850 con 0x7efd84107740 2024-09-12T02:53:42.190 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.390+0000 7efd8bbb9640 1 -- 172.21.15.116:0/2271974157 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd84107740 msgr2=0x7efd84107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:42.190 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.390+0000 7efd8bbb9640 1 --2- 172.21.15.116:0/2271974157 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd84107740 0x7efd84107b40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efd80009820 tx=0x7efd8002a270 comp rx=0 tx=0).stop 2024-09-12T02:53:42.190 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.390+0000 7efd8bbb9640 1 -- 172.21.15.116:0/2271974157 shutdown_connections 2024-09-12T02:53:42.190 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.390+0000 7efd8bbb9640 1 --2- 172.21.15.116:0/2271974157 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd84107740 0x7efd84107b40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:42.190 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.390+0000 7efd8bbb9640 1 -- 172.21.15.116:0/2271974157 >> 172.21.15.116:0/2271974157 conn(0x7efd84102eb0 msgr2=0x7efd84105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:42.190 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.390+0000 7efd8bbb9640 1 -- 172.21.15.116:0/2271974157 shutdown_connections 2024-09-12T02:53:42.190 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.390+0000 7efd8bbb9640 1 -- 172.21.15.116:0/2271974157 wait complete. 2024-09-12T02:53:42.191 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.391+0000 7efd8bbb9640 1 Processor -- start 2024-09-12T02:53:42.191 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.391+0000 7efd8bbb9640 1 -- start start 2024-09-12T02:53:42.191 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.391+0000 7efd8bbb9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd84107740 0x7efd8419b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:42.191 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.391+0000 7efd8bbb9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efd8410bc50 con 0x7efd84107740 2024-09-12T02:53:42.191 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.391+0000 7efd89955640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd84107740 0x7efd8419b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:42.191 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.391+0000 7efd89955640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd84107740 0x7efd8419b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33108/0 (socket says 172.21.15.116:33108) 2024-09-12T02:53:42.191 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.391+0000 7efd89955640 1 -- 172.21.15.116:0/2373449324 learned_addr learned my addr 172.21.15.116:0/2373449324 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:42.191 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.391+0000 7efd89955640 1 -- 172.21.15.116:0/2373449324 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8419bbc0 con 0x7efd84107740 2024-09-12T02:53:42.191 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.391+0000 7efd89955640 1 --2- 172.21.15.116:0/2373449324 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd84107740 0x7efd8419b680 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efd80009940 tx=0x7efd800326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:42.191 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.392+0000 7efd7affd640 1 -- 172.21.15.116:0/2373449324 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd80042070 con 0x7efd84107740 2024-09-12T02:53:42.191 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.392+0000 7efd7affd640 1 -- 172.21.15.116:0/2373449324 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efd80032be0 con 0x7efd84107740 2024-09-12T02:53:42.191 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.392+0000 7efd7affd640 1 -- 172.21.15.116:0/2373449324 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd80037040 con 0x7efd84107740 2024-09-12T02:53:42.191 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.392+0000 7efd8bbb9640 1 -- 172.21.15.116:0/2373449324 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd8419be50 con 0x7efd84107740 2024-09-12T02:53:42.192 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.392+0000 7efd8bbb9640 1 -- 172.21.15.116:0/2373449324 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd8419eb40 con 0x7efd84107740 2024-09-12T02:53:42.192 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.393+0000 7efd7affd640 1 -- 172.21.15.116:0/2373449324 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7efd80045070 con 0x7efd84107740 2024-09-12T02:53:42.192 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.393+0000 7efd8bbb9640 1 -- 172.21.15.116:0/2373449324 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd84107bc0 con 0x7efd84107740 2024-09-12T02:53:42.192 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.393+0000 7efd7affd640 1 --2- 172.21.15.116:0/2373449324 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7efd70038e60 0x7efd7003b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:42.192 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.393+0000 7efd7affd640 1 -- 172.21.15.116:0/2373449324 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efd8006bd30 con 0x7efd84107740 2024-09-12T02:53:42.192 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.393+0000 7efd89154640 1 --2- 172.21.15.116:0/2373449324 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7efd70038e60 0x7efd7003b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:42.192 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.393+0000 7efd89154640 1 --2- 172.21.15.116:0/2373449324 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7efd70038e60 0x7efd7003b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efd740098c0 tx=0x7efd74006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:42.192 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.398+0000 7efd7affd640 1 -- 172.21.15.116:0/2373449324 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efd80032d90 con 0x7efd84107740 2024-09-12T02:53:42.192 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.490+0000 7efd7affd640 1 -- 172.21.15.116:0/2373449324 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7efd800371e0 con 0x7efd84107740 2024-09-12T02:53:42.192 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.583+0000 7efd8bbb9640 1 -- 172.21.15.116:0/2373449324 --> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7efd841041c0 con 0x7efd70038e60 2024-09-12T02:53:42.192 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.589+0000 7efd7affd640 1 -- 172.21.15.116:0/2373449324 <== mgr.14118 v2:172.21.15.116:6800/2966274423 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efd841041c0 con 0x7efd70038e60 2024-09-12T02:53:42.192 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.591+0000 7efd8bbb9640 1 -- 172.21.15.116:0/2373449324 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7efd70038e60 msgr2=0x7efd7003b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:42.193 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.591+0000 7efd8bbb9640 1 --2- 172.21.15.116:0/2373449324 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7efd70038e60 0x7efd7003b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efd740098c0 tx=0x7efd74006db0 comp rx=0 tx=0).stop 2024-09-12T02:53:42.193 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.591+0000 7efd8bbb9640 1 -- 172.21.15.116:0/2373449324 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd84107740 msgr2=0x7efd8419b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:42.193 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.591+0000 7efd8bbb9640 1 --2- 172.21.15.116:0/2373449324 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd84107740 0x7efd8419b680 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efd80009940 tx=0x7efd800326e0 comp rx=0 tx=0).stop 2024-09-12T02:53:42.193 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.591+0000 7efd8bbb9640 1 -- 172.21.15.116:0/2373449324 shutdown_connections 2024-09-12T02:53:42.193 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.591+0000 7efd8bbb9640 1 --2- 172.21.15.116:0/2373449324 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7efd70038e60 0x7efd7003b320 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:42.193 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.591+0000 7efd8bbb9640 1 --2- 172.21.15.116:0/2373449324 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd84107740 0x7efd8419b680 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:42.193 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.591+0000 7efd8bbb9640 1 -- 172.21.15.116:0/2373449324 >> 172.21.15.116:0/2373449324 conn(0x7efd84102eb0 msgr2=0x7efd84103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:42.193 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.591+0000 7efd8bbb9640 1 -- 172.21.15.116:0/2373449324 shutdown_connections 2024-09-12T02:53:42.193 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:41.591+0000 7efd8bbb9640 1 -- 172.21.15.116:0/2373449324 wait complete. 2024-09-12T02:53:42.736 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:42 smithi116 ceph-mon[31410]: [12/Sep/2024:02:53:40] ENGINE Bus STARTING 2024-09-12T02:53:42.737 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:42 smithi116 ceph-mon[31410]: [12/Sep/2024:02:53:41] ENGINE Serving on https://172.21.15.116:7150 2024-09-12T02:53:42.737 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:42 smithi116 ceph-mon[31410]: [12/Sep/2024:02:53:41] ENGINE Error in HTTPServer.serve 2024-09-12T02:53:42.737 INFO:journalctl@ceph.mon.a.smithi116.stdout: Traceback (most recent call last): 2024-09-12T02:53:42.737 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T02:53:42.737 INFO:journalctl@ceph.mon.a.smithi116.stdout: self._connections.run(self.expiration_interval) 2024-09-12T02:53:42.737 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T02:53:42.737 INFO:journalctl@ceph.mon.a.smithi116.stdout: self._run(expiration_interval) 2024-09-12T02:53:42.737 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T02:53:42.737 INFO:journalctl@ceph.mon.a.smithi116.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T02:53:42.737 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T02:53:42.737 INFO:journalctl@ceph.mon.a.smithi116.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T02:53:42.738 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T02:53:42.738 INFO:journalctl@ceph.mon.a.smithi116.stdout: s = self.context.wrap_socket( 2024-09-12T02:53:42.738 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T02:53:42.738 INFO:journalctl@ceph.mon.a.smithi116.stdout: return self.sslsocket_class._create( 2024-09-12T02:53:42.738 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T02:53:42.738 INFO:journalctl@ceph.mon.a.smithi116.stdout: self.do_handshake() 2024-09-12T02:53:42.738 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T02:53:42.738 INFO:journalctl@ceph.mon.a.smithi116.stdout: self._sslobj.do_handshake() 2024-09-12T02:53:42.738 INFO:journalctl@ceph.mon.a.smithi116.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T02:53:42.738 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:42 smithi116 ceph-mon[31410]: [12/Sep/2024:02:53:41] ENGINE Serving on http://172.21.15.116:8765 2024-09-12T02:53:42.738 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:42 smithi116 ceph-mon[31410]: [12/Sep/2024:02:53:41] ENGINE Bus STARTED 2024-09-12T02:53:42.738 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:42 smithi116 ceph-mon[31410]: mgrmap e8: x(active, since 2s) 2024-09-12T02:53:42.738 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:42 smithi116 ceph-mon[31410]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:53:42.738 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:42 smithi116 ceph-mon[31410]: from='mgr.14118 172.21.15.116:0/1263688136' entity='mgr.x' 2024-09-12T02:53:42.738 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:42 smithi116 ceph-mon[31410]: from='mgr.14118 172.21.15.116:0/1263688136' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:53:43.331 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-12T02:53:43.338 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.559+0000 7ff9f2f9c640 1 Processor -- start 2024-09-12T02:53:43.339 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.559+0000 7ff9f2f9c640 1 -- start start 2024-09-12T02:53:43.339 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.559+0000 7ff9f2f9c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9ec107b20 0x7ff9ec107f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:43.339 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.559+0000 7ff9f2f9c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ec1084f0 con 0x7ff9ec107b20 2024-09-12T02:53:43.339 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.559+0000 7ff9f0d38640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9ec107b20 0x7ff9ec107f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:43.339 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.560+0000 7ff9f0d38640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9ec107b20 0x7ff9ec107f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33124/0 (socket says 172.21.15.116:33124) 2024-09-12T02:53:43.339 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.560+0000 7ff9f0d38640 1 -- 172.21.15.116:0/3061186784 learned_addr learned my addr 172.21.15.116:0/3061186784 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:43.340 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.560+0000 7ff9f0d38640 1 -- 172.21.15.116:0/3061186784 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9ec106780 con 0x7ff9ec107b20 2024-09-12T02:53:43.340 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.560+0000 7ff9f0d38640 1 --2- 172.21.15.116:0/3061186784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9ec107b20 0x7ff9ec107f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc009820 tx=0x7ff9dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9696a31f9d06ab9d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:43.340 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.561+0000 7ff9eb7fe640 1 -- 172.21.15.116:0/3061186784 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9dc037070 con 0x7ff9ec107b20 2024-09-12T02:53:43.340 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.561+0000 7ff9eb7fe640 1 -- 172.21.15.116:0/3061186784 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff9dc032900 con 0x7ff9ec107b20 2024-09-12T02:53:43.340 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.561+0000 7ff9eb7fe640 1 -- 172.21.15.116:0/3061186784 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9dc030850 con 0x7ff9ec107b20 2024-09-12T02:53:43.341 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.561+0000 7ff9f2f9c640 1 -- 172.21.15.116:0/3061186784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9ec107b20 msgr2=0x7ff9ec107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:43.341 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.561+0000 7ff9f2f9c640 1 --2- 172.21.15.116:0/3061186784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9ec107b20 0x7ff9ec107f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc009820 tx=0x7ff9dc02a270 comp rx=0 tx=0).stop 2024-09-12T02:53:43.341 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.561+0000 7ff9f2f9c640 1 -- 172.21.15.116:0/3061186784 shutdown_connections 2024-09-12T02:53:43.341 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.561+0000 7ff9f2f9c640 1 --2- 172.21.15.116:0/3061186784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9ec107b20 0x7ff9ec107f20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.341 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.561+0000 7ff9f2f9c640 1 -- 172.21.15.116:0/3061186784 >> 172.21.15.116:0/3061186784 conn(0x7ff9ec1014a0 msgr2=0x7ff9ec1038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:43.341 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.561+0000 7ff9f2f9c640 1 -- 172.21.15.116:0/3061186784 shutdown_connections 2024-09-12T02:53:43.341 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.561+0000 7ff9f2f9c640 1 -- 172.21.15.116:0/3061186784 wait complete. 2024-09-12T02:53:43.342 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.562+0000 7ff9f2f9c640 1 Processor -- start 2024-09-12T02:53:43.342 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.562+0000 7ff9f2f9c640 1 -- start start 2024-09-12T02:53:43.342 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.562+0000 7ff9f2f9c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9ec107b20 0x7ff9ec1994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:43.342 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.562+0000 7ff9f2f9c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ec105670 con 0x7ff9ec107b20 2024-09-12T02:53:43.342 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.562+0000 7ff9f0d38640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9ec107b20 0x7ff9ec1994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:43.342 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.562+0000 7ff9f0d38640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9ec107b20 0x7ff9ec1994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33136/0 (socket says 172.21.15.116:33136) 2024-09-12T02:53:43.342 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.562+0000 7ff9f0d38640 1 -- 172.21.15.116:0/1481475209 learned_addr learned my addr 172.21.15.116:0/1481475209 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:43.343 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.562+0000 7ff9f0d38640 1 -- 172.21.15.116:0/1481475209 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9ec1999f0 con 0x7ff9ec107b20 2024-09-12T02:53:43.343 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.563+0000 7ff9f0d38640 1 --2- 172.21.15.116:0/1481475209 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9ec107b20 0x7ff9ec1994b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc009940 tx=0x7ff9dc0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:43.343 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.563+0000 7ff9e9ffb640 1 -- 172.21.15.116:0/1481475209 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9dc042070 con 0x7ff9ec107b20 2024-09-12T02:53:43.343 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.563+0000 7ff9e9ffb640 1 -- 172.21.15.116:0/1481475209 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff9dc032be0 con 0x7ff9ec107b20 2024-09-12T02:53:43.343 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.563+0000 7ff9f2f9c640 1 -- 172.21.15.116:0/1481475209 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9ec199c80 con 0x7ff9ec107b20 2024-09-12T02:53:43.343 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.563+0000 7ff9e9ffb640 1 -- 172.21.15.116:0/1481475209 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9dc037040 con 0x7ff9ec107b20 2024-09-12T02:53:43.343 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.563+0000 7ff9f2f9c640 1 -- 172.21.15.116:0/1481475209 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9ec19c970 con 0x7ff9ec107b20 2024-09-12T02:53:43.344 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.564+0000 7ff9e9ffb640 1 -- 172.21.15.116:0/1481475209 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ff9dc045070 con 0x7ff9ec107b20 2024-09-12T02:53:43.344 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.564+0000 7ff9e9ffb640 1 --2- 172.21.15.116:0/1481475209 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7ff9d403b220 0x7ff9d403d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:43.344 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.564+0000 7ff9f2f9c640 1 -- 172.21.15.116:0/1481475209 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9ec107fa0 con 0x7ff9ec107b20 2024-09-12T02:53:43.344 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.564+0000 7ff9e9ffb640 1 -- 172.21.15.116:0/1481475209 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff9dc06be90 con 0x7ff9ec107b20 2024-09-12T02:53:43.344 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.564+0000 7ff9ebfff640 1 --2- 172.21.15.116:0/1481475209 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7ff9d403b220 0x7ff9d403d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:43.344 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.564+0000 7ff9ebfff640 1 --2- 172.21.15.116:0/1481475209 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7ff9d403b220 0x7ff9d403d6e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff9e00098c0 tx=0x7ff9e0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:43.345 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.569+0000 7ff9e9ffb640 1 -- 172.21.15.116:0/1481475209 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff9dc06b860 con 0x7ff9ec107b20 2024-09-12T02:53:43.345 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.736+0000 7ff9f2f9c640 1 -- 172.21.15.116:0/1481475209 --> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7ff9ec062630 con 0x7ff9d403b220 2024-09-12T02:53:43.345 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.736+0000 7ff9e9ffb640 1 -- 172.21.15.116:0/1481475209 <== mgr.14118 v2:172.21.15.116:6800/2966274423 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7ff9ec062630 con 0x7ff9d403b220 2024-09-12T02:53:43.345 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.738+0000 7ff9f2f9c640 1 -- 172.21.15.116:0/1481475209 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7ff9d403b220 msgr2=0x7ff9d403d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:43.345 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.738+0000 7ff9f2f9c640 1 --2- 172.21.15.116:0/1481475209 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7ff9d403b220 0x7ff9d403d6e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff9e00098c0 tx=0x7ff9e0006db0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.345 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.738+0000 7ff9f2f9c640 1 -- 172.21.15.116:0/1481475209 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9ec107b20 msgr2=0x7ff9ec1994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:43.346 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.738+0000 7ff9f2f9c640 1 --2- 172.21.15.116:0/1481475209 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9ec107b20 0x7ff9ec1994b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc009940 tx=0x7ff9dc0326e0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.346 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.738+0000 7ff9f2f9c640 1 -- 172.21.15.116:0/1481475209 shutdown_connections 2024-09-12T02:53:43.346 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.738+0000 7ff9f2f9c640 1 --2- 172.21.15.116:0/1481475209 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7ff9d403b220 0x7ff9d403d6e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.346 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.738+0000 7ff9f2f9c640 1 --2- 172.21.15.116:0/1481475209 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9ec107b20 0x7ff9ec1994b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.346 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.738+0000 7ff9f2f9c640 1 -- 172.21.15.116:0/1481475209 >> 172.21.15.116:0/1481475209 conn(0x7ff9ec1014a0 msgr2=0x7ff9ec18fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:43.346 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.739+0000 7ff9f2f9c640 1 -- 172.21.15.116:0/1481475209 shutdown_connections 2024-09-12T02:53:43.347 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:42.739+0000 7ff9f2f9c640 1 -- 172.21.15.116:0/1481475209 wait complete. 2024-09-12T02:53:43.347 INFO:teuthology.orchestra.run.smithi116.stdout:Generating ssh key... 2024-09-12T02:53:43.904 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:43 smithi116 ceph-mon[31410]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:53:44.303 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:44 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: Generating public/private rsa key pair. 2024-09-12T02:53:44.303 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:44 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: Your identification has been saved in /tmp/tmp_mhds_xp/key 2024-09-12T02:53:44.303 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:44 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: Your public key has been saved in /tmp/tmp_mhds_xp/key.pub 2024-09-12T02:53:44.303 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:44 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: The key fingerprint is: 2024-09-12T02:53:44.303 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:44 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: SHA256:VD63FGgC4ZwYBJ9j6uAxKS8z+5WecrckKUDEdYHK1yM ceph-eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T02:53:44.303 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:44 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: The key's randomart image is: 2024-09-12T02:53:44.303 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:44 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: +---[RSA 3072]----+ 2024-09-12T02:53:44.303 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:44 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: |..o++o+o ... | 2024-09-12T02:53:44.303 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:44 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: |.. o.* ..oo . | 2024-09-12T02:53:44.303 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:44 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: |... B + .oo o | 2024-09-12T02:53:44.303 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:44 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: |.o.E + . + . | 2024-09-12T02:53:44.304 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:44 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: |++o . . S . | 2024-09-12T02:53:44.304 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:44 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: |o=o o | 2024-09-12T02:53:44.304 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:44 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: |+.+ = . | 2024-09-12T02:53:44.304 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:44 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: | =.+.+. | 2024-09-12T02:53:44.304 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:44 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: |...oo... | 2024-09-12T02:53:44.304 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:44 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: +----[SHA256]-----+ 2024-09-12T02:53:44.571 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.725+0000 7fe80e4ea640 1 Processor -- start 2024-09-12T02:53:44.571 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.725+0000 7fe80e4ea640 1 -- start start 2024-09-12T02:53:44.571 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.725+0000 7fe80e4ea640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe808107740 0x7fe808107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:44.571 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.725+0000 7fe80e4ea640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe808108110 con 0x7fe808107740 2024-09-12T02:53:44.571 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.726+0000 7fe807fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe808107740 0x7fe808107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:44.571 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.726+0000 7fe807fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe808107740 0x7fe808107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33150/0 (socket says 172.21.15.116:33150) 2024-09-12T02:53:44.572 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.726+0000 7fe807fff640 1 -- 172.21.15.116:0/3747175590 learned_addr learned my addr 172.21.15.116:0/3747175590 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:44.572 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.726+0000 7fe807fff640 1 -- 172.21.15.116:0/3747175590 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8081088a0 con 0x7fe808107740 2024-09-12T02:53:44.572 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.726+0000 7fe807fff640 1 --2- 172.21.15.116:0/3747175590 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe808107740 0x7fe808107b40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009820 tx=0x7fe7f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e969c4fe98786bf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:44.572 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.727+0000 7fe8077fe640 1 -- 172.21.15.116:0/3747175590 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7f8037070 con 0x7fe808107740 2024-09-12T02:53:44.572 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.727+0000 7fe8077fe640 1 -- 172.21.15.116:0/3747175590 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe7f8032900 con 0x7fe808107740 2024-09-12T02:53:44.572 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.727+0000 7fe8077fe640 1 -- 172.21.15.116:0/3747175590 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7f8030850 con 0x7fe808107740 2024-09-12T02:53:44.572 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.727+0000 7fe80e4ea640 1 -- 172.21.15.116:0/3747175590 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe808107740 msgr2=0x7fe808107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:44.572 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.727+0000 7fe80e4ea640 1 --2- 172.21.15.116:0/3747175590 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe808107740 0x7fe808107b40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009820 tx=0x7fe7f802a270 comp rx=0 tx=0).stop 2024-09-12T02:53:44.572 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.727+0000 7fe80e4ea640 1 -- 172.21.15.116:0/3747175590 shutdown_connections 2024-09-12T02:53:44.573 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.727+0000 7fe80e4ea640 1 --2- 172.21.15.116:0/3747175590 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe808107740 0x7fe808107b40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:44.573 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.727+0000 7fe80e4ea640 1 -- 172.21.15.116:0/3747175590 >> 172.21.15.116:0/3747175590 conn(0x7fe808102eb0 msgr2=0x7fe808105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:44.573 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.727+0000 7fe80e4ea640 1 -- 172.21.15.116:0/3747175590 shutdown_connections 2024-09-12T02:53:44.573 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.727+0000 7fe80e4ea640 1 -- 172.21.15.116:0/3747175590 wait complete. 2024-09-12T02:53:44.573 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.728+0000 7fe80e4ea640 1 Processor -- start 2024-09-12T02:53:44.573 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.728+0000 7fe80e4ea640 1 -- start start 2024-09-12T02:53:44.573 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.728+0000 7fe80e4ea640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe808107740 0x7fe80819b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:44.573 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.728+0000 7fe80e4ea640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe80810bc50 con 0x7fe808107740 2024-09-12T02:53:44.573 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.728+0000 7fe807fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe808107740 0x7fe80819b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:44.573 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.728+0000 7fe807fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe808107740 0x7fe80819b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33154/0 (socket says 172.21.15.116:33154) 2024-09-12T02:53:44.573 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.728+0000 7fe807fff640 1 -- 172.21.15.116:0/1533467555 learned_addr learned my addr 172.21.15.116:0/1533467555 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:44.574 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.729+0000 7fe807fff640 1 -- 172.21.15.116:0/1533467555 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80819bbf0 con 0x7fe808107740 2024-09-12T02:53:44.574 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.729+0000 7fe807fff640 1 --2- 172.21.15.116:0/1533467555 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe808107740 0x7fe80819b6b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009940 tx=0x7fe7f80326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:44.574 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.729+0000 7fe805ffb640 1 -- 172.21.15.116:0/1533467555 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7f8042070 con 0x7fe808107740 2024-09-12T02:53:44.574 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.729+0000 7fe805ffb640 1 -- 172.21.15.116:0/1533467555 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe7f8032be0 con 0x7fe808107740 2024-09-12T02:53:44.574 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.729+0000 7fe80e4ea640 1 -- 172.21.15.116:0/1533467555 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe80819be80 con 0x7fe808107740 2024-09-12T02:53:44.574 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.729+0000 7fe805ffb640 1 -- 172.21.15.116:0/1533467555 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7f8037040 con 0x7fe808107740 2024-09-12T02:53:44.574 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.729+0000 7fe80e4ea640 1 -- 172.21.15.116:0/1533467555 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe80819eb70 con 0x7fe808107740 2024-09-12T02:53:44.574 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.730+0000 7fe805ffb640 1 -- 172.21.15.116:0/1533467555 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fe7f8045070 con 0x7fe808107740 2024-09-12T02:53:44.574 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.730+0000 7fe805ffb640 1 --2- 172.21.15.116:0/1533467555 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fe7f4038f70 0x7fe7f403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:44.574 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.730+0000 7fe80e4ea640 1 -- 172.21.15.116:0/1533467555 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe808107bc0 con 0x7fe808107740 2024-09-12T02:53:44.575 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.730+0000 7fe805ffb640 1 -- 172.21.15.116:0/1533467555 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe7f806bef0 con 0x7fe808107740 2024-09-12T02:53:44.575 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.730+0000 7fe7fffff640 1 --2- 172.21.15.116:0/1533467555 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fe7f4038f70 0x7fe7f403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:44.575 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.730+0000 7fe7fffff640 1 --2- 172.21.15.116:0/1533467555 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fe7f4038f70 0x7fe7f403b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00098c0 tx=0x7fe7f0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:44.575 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.735+0000 7fe805ffb640 1 -- 172.21.15.116:0/1533467555 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe7f8032d90 con 0x7fe808107740 2024-09-12T02:53:44.575 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:43.904+0000 7fe80e4ea640 1 -- 172.21.15.116:0/1533467555 --> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fe8081041a0 con 0x7fe7f4038f70 2024-09-12T02:53:44.575 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.018+0000 7fe805ffb640 1 -- 172.21.15.116:0/1533467555 <== mgr.14118 v2:172.21.15.116:6800/2966274423 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe8081041a0 con 0x7fe7f4038f70 2024-09-12T02:53:44.575 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.020+0000 7fe80e4ea640 1 -- 172.21.15.116:0/1533467555 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fe7f4038f70 msgr2=0x7fe7f403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:44.575 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.020+0000 7fe80e4ea640 1 --2- 172.21.15.116:0/1533467555 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fe7f4038f70 0x7fe7f403b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00098c0 tx=0x7fe7f0006db0 comp rx=0 tx=0).stop 2024-09-12T02:53:44.575 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.020+0000 7fe80e4ea640 1 -- 172.21.15.116:0/1533467555 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe808107740 msgr2=0x7fe80819b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:44.575 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.020+0000 7fe80e4ea640 1 --2- 172.21.15.116:0/1533467555 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe808107740 0x7fe80819b6b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009940 tx=0x7fe7f80326e0 comp rx=0 tx=0).stop 2024-09-12T02:53:44.575 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.020+0000 7fe80e4ea640 1 -- 172.21.15.116:0/1533467555 shutdown_connections 2024-09-12T02:53:44.576 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.020+0000 7fe80e4ea640 1 --2- 172.21.15.116:0/1533467555 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fe7f4038f70 0x7fe7f403b430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:44.576 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.020+0000 7fe80e4ea640 1 --2- 172.21.15.116:0/1533467555 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe808107740 0x7fe80819b6b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:44.576 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.020+0000 7fe80e4ea640 1 -- 172.21.15.116:0/1533467555 >> 172.21.15.116:0/1533467555 conn(0x7fe808102eb0 msgr2=0x7fe808103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:44.576 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.020+0000 7fe80e4ea640 1 -- 172.21.15.116:0/1533467555 shutdown_connections 2024-09-12T02:53:44.576 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.020+0000 7fe80e4ea640 1 -- 172.21.15.116:0/1533467555 wait complete. 2024-09-12T02:53:45.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:45 smithi116 ceph-mon[31410]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:53:45.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:45 smithi116 ceph-mon[31410]: Generating ssh key... 2024-09-12T02:53:45.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:45 smithi116 ceph-mon[31410]: from='mgr.14118 172.21.15.116:0/1263688136' entity='mgr.x' 2024-09-12T02:53:45.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:45 smithi116 ceph-mon[31410]: from='mgr.14118 172.21.15.116:0/1263688136' entity='mgr.x' 2024-09-12T02:53:45.750 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0C7Up4lYIJ4YJgQv/JwnLjQSoSttVjejpz4yaXzwzNqNv3A9GdBpJmjBBzDuQ3mFVEymlLuj+zC5CkkVc8dhrze3rGwMSURlA3bQyORgwvttEVV1qeAqFVJG63j1M+dRYkiMiJmwB8m4A7NaO/ylIkd9IJogpibuRjTrve1inVR9fp66ctXcaKGCdrivMTxmF8JR/NsXRPOEOav75BD62v1ANy3ZyreURasi/UX6PPieD4PjFCfJJi81vZCl2cKMANQy+SVpO32Fxptuj04r7SMXOd+dWH3j7Mc+YuT4LpuftruDpqRSu8gqp5Ir1RdQuiz8EKJRvUr8XkmG9/aSQV9fVgotx0uF3cujVEuQcodLlxkKntVKgAju064r7EEXia50uQqAhHqsQaCpCh1bWkTF1zHxePtemCPcFfJD7veXHD/NmPYQ4J9ZAGy8/f6soq/I6iUoHNoZyAZv3QUpx9SfvRKbd4jRaZOswv/aWTj6Set26RnvB/iGYfcpXpQ8= ceph-eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T02:53:45.750 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.955+0000 7f4a627dc640 1 Processor -- start 2024-09-12T02:53:45.750 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.955+0000 7f4a627dc640 1 -- start start 2024-09-12T02:53:45.750 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.955+0000 7f4a627dc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a5c106520 0x7f4a5c106920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:45.750 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.955+0000 7f4a627dc640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c106ef0 con 0x7f4a5c106520 2024-09-12T02:53:45.751 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.955+0000 7f4a5bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a5c106520 0x7f4a5c106920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:45.751 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.955+0000 7f4a5bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a5c106520 0x7f4a5c106920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33164/0 (socket says 172.21.15.116:33164) 2024-09-12T02:53:45.751 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.955+0000 7f4a5bfff640 1 -- 172.21.15.116:0/905929751 learned_addr learned my addr 172.21.15.116:0/905929751 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:45.751 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.956+0000 7f4a5bfff640 1 -- 172.21.15.116:0/905929751 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5c107720 con 0x7f4a5c106520 2024-09-12T02:53:45.751 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.956+0000 7f4a5bfff640 1 --2- 172.21.15.116:0/905929751 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a5c106520 0x7f4a5c106920 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009820 tx=0x7f4a4c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b35146a66ca864c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:45.751 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.956+0000 7f4a5affd640 1 -- 172.21.15.116:0/905929751 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a4c037070 con 0x7f4a5c106520 2024-09-12T02:53:45.751 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.957+0000 7f4a5affd640 1 -- 172.21.15.116:0/905929751 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4a4c032900 con 0x7f4a5c106520 2024-09-12T02:53:45.751 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.957+0000 7f4a5affd640 1 -- 172.21.15.116:0/905929751 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a4c030850 con 0x7f4a5c106520 2024-09-12T02:53:45.751 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.957+0000 7f4a627dc640 1 -- 172.21.15.116:0/905929751 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a5c106520 msgr2=0x7f4a5c106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:45.752 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.957+0000 7f4a627dc640 1 --2- 172.21.15.116:0/905929751 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a5c106520 0x7f4a5c106920 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009820 tx=0x7f4a4c02a270 comp rx=0 tx=0).stop 2024-09-12T02:53:45.752 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.957+0000 7f4a627dc640 1 -- 172.21.15.116:0/905929751 shutdown_connections 2024-09-12T02:53:45.752 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.957+0000 7f4a627dc640 1 --2- 172.21.15.116:0/905929751 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a5c106520 0x7f4a5c106920 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:45.752 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.957+0000 7f4a627dc640 1 -- 172.21.15.116:0/905929751 >> 172.21.15.116:0/905929751 conn(0x7f4a5c101db0 msgr2=0x7f4a5c1041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:45.752 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.957+0000 7f4a627dc640 1 -- 172.21.15.116:0/905929751 shutdown_connections 2024-09-12T02:53:45.752 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.957+0000 7f4a627dc640 1 -- 172.21.15.116:0/905929751 wait complete. 2024-09-12T02:53:45.752 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.958+0000 7f4a627dc640 1 Processor -- start 2024-09-12T02:53:45.752 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.958+0000 7f4a627dc640 1 -- start start 2024-09-12T02:53:45.752 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.958+0000 7f4a627dc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a5c106520 0x7f4a5c19b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:45.752 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.958+0000 7f4a627dc640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c10aad0 con 0x7f4a5c106520 2024-09-12T02:53:45.752 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.958+0000 7f4a5bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a5c106520 0x7f4a5c19b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:45.752 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.958+0000 7f4a5bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a5c106520 0x7f4a5c19b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33168/0 (socket says 172.21.15.116:33168) 2024-09-12T02:53:45.753 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.958+0000 7f4a5bfff640 1 -- 172.21.15.116:0/217290925 learned_addr learned my addr 172.21.15.116:0/217290925 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:45.753 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.958+0000 7f4a5bfff640 1 -- 172.21.15.116:0/217290925 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5c19bb80 con 0x7f4a5c106520 2024-09-12T02:53:45.753 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.958+0000 7f4a5bfff640 1 --2- 172.21.15.116:0/217290925 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a5c106520 0x7f4a5c19b640 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009940 tx=0x7f4a4c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:45.753 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.959+0000 7f4a597fa640 1 -- 172.21.15.116:0/217290925 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a4c042070 con 0x7f4a5c106520 2024-09-12T02:53:45.753 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.959+0000 7f4a597fa640 1 -- 172.21.15.116:0/217290925 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4a4c0308c0 con 0x7f4a5c106520 2024-09-12T02:53:45.753 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.959+0000 7f4a597fa640 1 -- 172.21.15.116:0/217290925 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a4c037040 con 0x7f4a5c106520 2024-09-12T02:53:45.753 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.959+0000 7f4a627dc640 1 -- 172.21.15.116:0/217290925 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a5c19be10 con 0x7f4a5c106520 2024-09-12T02:53:45.753 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.959+0000 7f4a627dc640 1 -- 172.21.15.116:0/217290925 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a5c19eb00 con 0x7f4a5c106520 2024-09-12T02:53:45.753 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.960+0000 7f4a597fa640 1 -- 172.21.15.116:0/217290925 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f4a4c045070 con 0x7f4a5c106520 2024-09-12T02:53:45.753 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.960+0000 7f4a627dc640 1 -- 172.21.15.116:0/217290925 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a5c10ac50 con 0x7f4a5c106520 2024-09-12T02:53:45.753 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.960+0000 7f4a597fa640 1 --2- 172.21.15.116:0/217290925 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f4a44038f70 0x7f4a4403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:45.753 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.960+0000 7f4a597fa640 1 -- 172.21.15.116:0/217290925 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4a4c06c030 con 0x7f4a5c106520 2024-09-12T02:53:45.753 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.960+0000 7f4a5b7fe640 1 --2- 172.21.15.116:0/217290925 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f4a44038f70 0x7f4a4403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:45.754 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.960+0000 7f4a5b7fe640 1 --2- 172.21.15.116:0/217290925 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f4a44038f70 0x7f4a4403b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4a500098c0 tx=0x7f4a50006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:45.754 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:44.965+0000 7f4a597fa640 1 -- 172.21.15.116:0/217290925 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4a4c03cd90 con 0x7f4a5c106520 2024-09-12T02:53:45.754 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:45.134+0000 7f4a627dc640 1 -- 172.21.15.116:0/217290925 --> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f4a5c102fe0 con 0x7f4a44038f70 2024-09-12T02:53:45.754 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:45.135+0000 7f4a597fa640 1 -- 172.21.15.116:0/217290925 <== mgr.14118 v2:172.21.15.116:6800/2966274423 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f4a5c102fe0 con 0x7f4a44038f70 2024-09-12T02:53:45.754 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:45.137+0000 7f4a627dc640 1 -- 172.21.15.116:0/217290925 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f4a44038f70 msgr2=0x7f4a4403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:45.754 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:45.137+0000 7f4a627dc640 1 --2- 172.21.15.116:0/217290925 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f4a44038f70 0x7f4a4403b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4a500098c0 tx=0x7f4a50006db0 comp rx=0 tx=0).stop 2024-09-12T02:53:45.754 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:45.137+0000 7f4a627dc640 1 -- 172.21.15.116:0/217290925 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a5c106520 msgr2=0x7f4a5c19b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:45.754 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:45.137+0000 7f4a627dc640 1 --2- 172.21.15.116:0/217290925 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a5c106520 0x7f4a5c19b640 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009940 tx=0x7f4a4c032ce0 comp rx=0 tx=0).stop 2024-09-12T02:53:45.754 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:45.137+0000 7f4a627dc640 1 -- 172.21.15.116:0/217290925 shutdown_connections 2024-09-12T02:53:45.754 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:45.137+0000 7f4a627dc640 1 --2- 172.21.15.116:0/217290925 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f4a44038f70 0x7f4a4403b430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:45.754 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:45.137+0000 7f4a627dc640 1 --2- 172.21.15.116:0/217290925 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a5c106520 0x7f4a5c19b640 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:45.754 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:45.138+0000 7f4a627dc640 1 -- 172.21.15.116:0/217290925 >> 172.21.15.116:0/217290925 conn(0x7f4a5c101db0 msgr2=0x7f4a5c1028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:45.754 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:45.138+0000 7f4a627dc640 1 -- 172.21.15.116:0/217290925 shutdown_connections 2024-09-12T02:53:45.755 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:45.138+0000 7f4a627dc640 1 -- 172.21.15.116:0/217290925 wait complete. 2024-09-12T02:53:45.755 INFO:teuthology.orchestra.run.smithi116.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-12T02:53:45.755 INFO:teuthology.orchestra.run.smithi116.stdout:Adding key to root@localhost authorized_keys... 2024-09-12T02:53:45.755 INFO:teuthology.orchestra.run.smithi116.stdout:Adding host smithi116... 2024-09-12T02:53:46.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:46 smithi116 ceph-mon[31410]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:53:47.094 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:47 smithi116 ceph-mon[31410]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi116", "addr": "172.21.15.116", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:53:48.082 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:48 smithi116 ceph-mon[31410]: Deploying cephadm binary to smithi116 2024-09-12T02:53:50.049 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout Added host 'smithi116' with addr '172.21.15.116' 2024-09-12T02:53:50.049 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.147+0000 7f625dccf640 1 Processor -- start 2024-09-12T02:53:50.049 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.148+0000 7f625dccf640 1 -- start start 2024-09-12T02:53:50.049 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.148+0000 7f625dccf640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62581080d0 0x7f625810a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:50.049 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.148+0000 7f625dccf640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f625807be70 con 0x7f62581080d0 2024-09-12T02:53:50.049 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.148+0000 7f62577fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62581080d0 0x7f625810a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:50.049 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.148+0000 7f62577fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62581080d0 0x7f625810a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33180/0 (socket says 172.21.15.116:33180) 2024-09-12T02:53:50.049 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.148+0000 7f62577fe640 1 -- 172.21.15.116:0/1161004930 learned_addr learned my addr 172.21.15.116:0/1161004930 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:50.049 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.148+0000 7f62577fe640 1 -- 172.21.15.116:0/1161004930 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f625810aa00 con 0x7f62581080d0 2024-09-12T02:53:50.050 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.149+0000 7f62577fe640 1 --2- 172.21.15.116:0/1161004930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62581080d0 0x7f625810a4c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6248009820 tx=0x7f624802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f7fa458ac23b80 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:50.050 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.149+0000 7f6256ffd640 1 -- 172.21.15.116:0/1161004930 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6248037070 con 0x7f62581080d0 2024-09-12T02:53:50.050 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.149+0000 7f6256ffd640 1 -- 172.21.15.116:0/1161004930 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6248032900 con 0x7f62581080d0 2024-09-12T02:53:50.050 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.149+0000 7f6256ffd640 1 -- 172.21.15.116:0/1161004930 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6248030850 con 0x7f62581080d0 2024-09-12T02:53:50.050 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.150+0000 7f625dccf640 1 -- 172.21.15.116:0/1161004930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62581080d0 msgr2=0x7f625810a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:50.050 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.150+0000 7f625dccf640 1 --2- 172.21.15.116:0/1161004930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62581080d0 0x7f625810a4c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6248009820 tx=0x7f624802a270 comp rx=0 tx=0).stop 2024-09-12T02:53:50.050 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.150+0000 7f625dccf640 1 -- 172.21.15.116:0/1161004930 shutdown_connections 2024-09-12T02:53:50.050 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.150+0000 7f625dccf640 1 --2- 172.21.15.116:0/1161004930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62581080d0 0x7f625810a4c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:50.050 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.150+0000 7f625dccf640 1 -- 172.21.15.116:0/1161004930 >> 172.21.15.116:0/1161004930 conn(0x7f6258101db0 msgr2=0x7f62581041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:50.050 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.150+0000 7f625dccf640 1 -- 172.21.15.116:0/1161004930 shutdown_connections 2024-09-12T02:53:50.050 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.150+0000 7f625dccf640 1 -- 172.21.15.116:0/1161004930 wait complete. 2024-09-12T02:53:50.050 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.150+0000 7f625dccf640 1 Processor -- start 2024-09-12T02:53:50.050 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.150+0000 7f625dccf640 1 -- start start 2024-09-12T02:53:50.051 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.150+0000 7f625dccf640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62581080d0 0x7f625819b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:50.051 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.151+0000 7f625dccf640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f625810ba40 con 0x7f62581080d0 2024-09-12T02:53:50.051 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.151+0000 7f62577fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62581080d0 0x7f625819b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:50.051 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.151+0000 7f62577fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62581080d0 0x7f625819b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33192/0 (socket says 172.21.15.116:33192) 2024-09-12T02:53:50.051 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.151+0000 7f62577fe640 1 -- 172.21.15.116:0/3536086911 learned_addr learned my addr 172.21.15.116:0/3536086911 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:50.051 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.151+0000 7f62577fe640 1 -- 172.21.15.116:0/3536086911 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f625819bb50 con 0x7f62581080d0 2024-09-12T02:53:50.051 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.151+0000 7f62577fe640 1 --2- 172.21.15.116:0/3536086911 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62581080d0 0x7f625819b610 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f62480097f0 tx=0x7f6248030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:50.051 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.151+0000 7f62557fa640 1 -- 172.21.15.116:0/3536086911 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6248037040 con 0x7f62581080d0 2024-09-12T02:53:50.051 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.151+0000 7f625dccf640 1 -- 172.21.15.116:0/3536086911 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f625819bde0 con 0x7f62581080d0 2024-09-12T02:53:50.051 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.151+0000 7f62557fa640 1 -- 172.21.15.116:0/3536086911 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6248040070 con 0x7f62581080d0 2024-09-12T02:53:50.051 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.151+0000 7f62557fa640 1 -- 172.21.15.116:0/3536086911 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f624803a390 con 0x7f62581080d0 2024-09-12T02:53:50.051 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.151+0000 7f625dccf640 1 -- 172.21.15.116:0/3536086911 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f625819ead0 con 0x7f62581080d0 2024-09-12T02:53:50.051 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.152+0000 7f62557fa640 1 -- 172.21.15.116:0/3536086911 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f6248032900 con 0x7f62581080d0 2024-09-12T02:53:50.052 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.152+0000 7f625dccf640 1 -- 172.21.15.116:0/3536086911 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6258108530 con 0x7f62581080d0 2024-09-12T02:53:50.052 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.152+0000 7f62557fa640 1 --2- 172.21.15.116:0/3536086911 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f6244038f70 0x7f624403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:50.052 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.153+0000 7f62557fa640 1 -- 172.21.15.116:0/3536086911 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f624806c4b0 con 0x7f62581080d0 2024-09-12T02:53:50.052 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.153+0000 7f624ffff640 1 --2- 172.21.15.116:0/3536086911 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f6244038f70 0x7f624403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:50.052 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.153+0000 7f624ffff640 1 --2- 172.21.15.116:0/3536086911 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f6244038f70 0x7f624403b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f62400098c0 tx=0x7f6240006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:50.052 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.157+0000 7f62557fa640 1 -- 172.21.15.116:0/3536086911 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6248040ce0 con 0x7f62581080d0 2024-09-12T02:53:50.052 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:46.327+0000 7f625dccf640 1 -- 172.21.15.116:0/3536086911 --> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi116", "addr": "172.21.15.116", "target": ["mon-mgr", ""]}) -- 0x7f6258103000 con 0x7f6244038f70 2024-09-12T02:53:50.052 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:49.514+0000 7f62557fa640 1 -- 172.21.15.116:0/3536086911 <== mgr.14118 v2:172.21.15.116:6800/2966274423 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f6258103000 con 0x7f6244038f70 2024-09-12T02:53:50.052 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:49.517+0000 7f625dccf640 1 -- 172.21.15.116:0/3536086911 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f6244038f70 msgr2=0x7f624403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:50.052 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:49.517+0000 7f625dccf640 1 --2- 172.21.15.116:0/3536086911 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f6244038f70 0x7f624403b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f62400098c0 tx=0x7f6240006db0 comp rx=0 tx=0).stop 2024-09-12T02:53:50.052 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:49.517+0000 7f625dccf640 1 -- 172.21.15.116:0/3536086911 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62581080d0 msgr2=0x7f625819b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:50.052 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:49.517+0000 7f625dccf640 1 --2- 172.21.15.116:0/3536086911 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62581080d0 0x7f625819b610 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f62480097f0 tx=0x7f6248030d80 comp rx=0 tx=0).stop 2024-09-12T02:53:50.052 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:49.517+0000 7f625dccf640 1 -- 172.21.15.116:0/3536086911 shutdown_connections 2024-09-12T02:53:50.053 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:49.517+0000 7f625dccf640 1 --2- 172.21.15.116:0/3536086911 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f6244038f70 0x7f624403b430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:50.053 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:49.517+0000 7f625dccf640 1 --2- 172.21.15.116:0/3536086911 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62581080d0 0x7f625819b610 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:50.053 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:49.517+0000 7f625dccf640 1 -- 172.21.15.116:0/3536086911 >> 172.21.15.116:0/3536086911 conn(0x7f6258101db0 msgr2=0x7f62581028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:50.053 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:49.517+0000 7f625dccf640 1 -- 172.21.15.116:0/3536086911 shutdown_connections 2024-09-12T02:53:50.053 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:49.517+0000 7f625dccf640 1 -- 172.21.15.116:0/3536086911 wait complete. 2024-09-12T02:53:50.053 INFO:teuthology.orchestra.run.smithi116.stdout:Deploying unmanaged mon service... 2024-09-12T02:53:50.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:50 smithi116 ceph-mon[31410]: from='mgr.14118 172.21.15.116:0/1263688136' entity='mgr.x' 2024-09-12T02:53:50.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:50 smithi116 ceph-mon[31410]: Added host smithi116 2024-09-12T02:53:50.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:50 smithi116 ceph-mon[31410]: from='mgr.14118 172.21.15.116:0/1263688136' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:53:51.552 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-12T02:53:51.552 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.482+0000 7fab825f0640 1 Processor -- start 2024-09-12T02:53:51.552 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.482+0000 7fab825f0640 1 -- start start 2024-09-12T02:53:51.553 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.482+0000 7fab825f0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab7c105ee0 0x7fab7c1062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:51.553 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.483+0000 7fab825f0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c07be30 con 0x7fab7c105ee0 2024-09-12T02:53:51.553 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.483+0000 7fab7bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab7c105ee0 0x7fab7c1062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:51.553 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.483+0000 7fab7bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab7c105ee0 0x7fab7c1062e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33194/0 (socket says 172.21.15.116:33194) 2024-09-12T02:53:51.553 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.483+0000 7fab7bfff640 1 -- 172.21.15.116:0/1456939235 learned_addr learned my addr 172.21.15.116:0/1456939235 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:51.553 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.483+0000 7fab7bfff640 1 -- 172.21.15.116:0/1456939235 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab7c106820 con 0x7fab7c105ee0 2024-09-12T02:53:51.553 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.484+0000 7fab7bfff640 1 --2- 172.21.15.116:0/1456939235 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab7c105ee0 0x7fab7c1062e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fab6c009820 tx=0x7fab6c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e008f10155ece53a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:51.553 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.484+0000 7fab7affd640 1 -- 172.21.15.116:0/1456939235 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab6c037070 con 0x7fab7c105ee0 2024-09-12T02:53:51.553 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.484+0000 7fab7affd640 1 -- 172.21.15.116:0/1456939235 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fab6c032900 con 0x7fab7c105ee0 2024-09-12T02:53:51.553 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.484+0000 7fab7affd640 1 -- 172.21.15.116:0/1456939235 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab6c030850 con 0x7fab7c105ee0 2024-09-12T02:53:51.553 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.484+0000 7fab825f0640 1 -- 172.21.15.116:0/1456939235 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab7c105ee0 msgr2=0x7fab7c1062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:51.554 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.484+0000 7fab825f0640 1 --2- 172.21.15.116:0/1456939235 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab7c105ee0 0x7fab7c1062e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fab6c009820 tx=0x7fab6c02a270 comp rx=0 tx=0).stop 2024-09-12T02:53:51.554 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.484+0000 7fab825f0640 1 -- 172.21.15.116:0/1456939235 shutdown_connections 2024-09-12T02:53:51.554 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.484+0000 7fab825f0640 1 --2- 172.21.15.116:0/1456939235 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab7c105ee0 0x7fab7c1062e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:51.554 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.485+0000 7fab825f0640 1 -- 172.21.15.116:0/1456939235 >> 172.21.15.116:0/1456939235 conn(0x7fab7c101db0 msgr2=0x7fab7c1041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:51.554 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.485+0000 7fab825f0640 1 -- 172.21.15.116:0/1456939235 shutdown_connections 2024-09-12T02:53:51.554 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.485+0000 7fab825f0640 1 -- 172.21.15.116:0/1456939235 wait complete. 2024-09-12T02:53:51.554 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.485+0000 7fab825f0640 1 Processor -- start 2024-09-12T02:53:51.554 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.485+0000 7fab825f0640 1 -- start start 2024-09-12T02:53:51.554 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.485+0000 7fab825f0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab7c105ee0 0x7fab7c19d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:51.554 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.485+0000 7fab825f0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c107920 con 0x7fab7c105ee0 2024-09-12T02:53:51.554 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.485+0000 7fab7bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab7c105ee0 0x7fab7c19d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:51.554 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.486+0000 7fab7bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab7c105ee0 0x7fab7c19d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33204/0 (socket says 172.21.15.116:33204) 2024-09-12T02:53:51.554 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.486+0000 7fab7bfff640 1 -- 172.21.15.116:0/1223098433 learned_addr learned my addr 172.21.15.116:0/1223098433 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:51.555 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.486+0000 7fab7bfff640 1 -- 172.21.15.116:0/1223098433 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab7c19dd10 con 0x7fab7c105ee0 2024-09-12T02:53:51.555 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.486+0000 7fab7bfff640 1 --2- 172.21.15.116:0/1223098433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab7c105ee0 0x7fab7c19d7d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fab6c032ce0 tx=0x7fab6c032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:51.555 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.486+0000 7fab797fa640 1 -- 172.21.15.116:0/1223098433 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab6c042070 con 0x7fab7c105ee0 2024-09-12T02:53:51.555 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.486+0000 7fab797fa640 1 -- 172.21.15.116:0/1223098433 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fab6c0308c0 con 0x7fab7c105ee0 2024-09-12T02:53:51.555 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.486+0000 7fab825f0640 1 -- 172.21.15.116:0/1223098433 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab7c19dfa0 con 0x7fab7c105ee0 2024-09-12T02:53:51.555 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.486+0000 7fab797fa640 1 -- 172.21.15.116:0/1223098433 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab6c037040 con 0x7fab7c105ee0 2024-09-12T02:53:51.555 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.486+0000 7fab825f0640 1 -- 172.21.15.116:0/1223098433 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab7c1a0c90 con 0x7fab7c105ee0 2024-09-12T02:53:51.555 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.487+0000 7fab797fa640 1 -- 172.21.15.116:0/1223098433 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fab6c045070 con 0x7fab7c105ee0 2024-09-12T02:53:51.555 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.487+0000 7fab797fa640 1 --2- 172.21.15.116:0/1223098433 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fab64038fc0 0x7fab6403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:51.555 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.487+0000 7fab825f0640 1 -- 172.21.15.116:0/1223098433 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab7c1062e0 con 0x7fab7c105ee0 2024-09-12T02:53:51.555 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.487+0000 7fab797fa640 1 -- 172.21.15.116:0/1223098433 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fab6c06c0c0 con 0x7fab7c105ee0 2024-09-12T02:53:51.555 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.487+0000 7fab7b7fe640 1 --2- 172.21.15.116:0/1223098433 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fab64038fc0 0x7fab6403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:51.555 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.488+0000 7fab7b7fe640 1 --2- 172.21.15.116:0/1223098433 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fab64038fc0 0x7fab6403b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fab700098c0 tx=0x7fab70006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:51.556 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.492+0000 7fab797fa640 1 -- 172.21.15.116:0/1223098433 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fab6c06ba50 con 0x7fab7c105ee0 2024-09-12T02:53:51.556 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.674+0000 7fab825f0640 1 -- 172.21.15.116:0/1223098433 --> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fab7c02dab0 con 0x7fab64038fc0 2024-09-12T02:53:51.556 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.678+0000 7fab797fa640 1 -- 172.21.15.116:0/1223098433 <== mgr.14118 v2:172.21.15.116:6800/2966274423 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fab7c02dab0 con 0x7fab64038fc0 2024-09-12T02:53:51.556 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.680+0000 7fab825f0640 1 -- 172.21.15.116:0/1223098433 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fab64038fc0 msgr2=0x7fab6403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:51.556 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.680+0000 7fab825f0640 1 --2- 172.21.15.116:0/1223098433 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fab64038fc0 0x7fab6403b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fab700098c0 tx=0x7fab70006db0 comp rx=0 tx=0).stop 2024-09-12T02:53:51.556 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.680+0000 7fab825f0640 1 -- 172.21.15.116:0/1223098433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab7c105ee0 msgr2=0x7fab7c19d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:51.556 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.680+0000 7fab825f0640 1 --2- 172.21.15.116:0/1223098433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab7c105ee0 0x7fab7c19d7d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fab6c032ce0 tx=0x7fab6c032d10 comp rx=0 tx=0).stop 2024-09-12T02:53:51.556 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.680+0000 7fab825f0640 1 -- 172.21.15.116:0/1223098433 shutdown_connections 2024-09-12T02:53:51.556 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.680+0000 7fab825f0640 1 --2- 172.21.15.116:0/1223098433 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fab64038fc0 0x7fab6403b480 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:51.556 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.680+0000 7fab825f0640 1 --2- 172.21.15.116:0/1223098433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab7c105ee0 0x7fab7c19d7d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:51.556 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.680+0000 7fab825f0640 1 -- 172.21.15.116:0/1223098433 >> 172.21.15.116:0/1223098433 conn(0x7fab7c101db0 msgr2=0x7fab7c194490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:51.556 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.680+0000 7fab825f0640 1 -- 172.21.15.116:0/1223098433 shutdown_connections 2024-09-12T02:53:51.556 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:50.680+0000 7fab825f0640 1 -- 172.21.15.116:0/1223098433 wait complete. 2024-09-12T02:53:51.557 INFO:teuthology.orchestra.run.smithi116.stdout:Deploying unmanaged mgr service... 2024-09-12T02:53:51.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:51 smithi116 ceph-mon[31410]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:53:51.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:51 smithi116 ceph-mon[31410]: Saving service mon spec with placement count:5 2024-09-12T02:53:51.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:51 smithi116 ceph-mon[31410]: from='mgr.14118 172.21.15.116:0/1263688136' entity='mgr.x' 2024-09-12T02:53:53.605 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-12T02:53:53.605 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.713+0000 7fe183467640 1 Processor -- start 2024-09-12T02:53:53.605 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.714+0000 7fe183467640 1 -- start start 2024-09-12T02:53:53.605 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.714+0000 7fe183467640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe17c108090 0x7fe17c10a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:53.606 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.714+0000 7fe183467640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe17c07beb0 con 0x7fe17c108090 2024-09-12T02:53:53.606 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.714+0000 7fe181203640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe17c108090 0x7fe17c10a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:53.606 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.714+0000 7fe181203640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe17c108090 0x7fe17c10a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59662/0 (socket says 172.21.15.116:59662) 2024-09-12T02:53:53.606 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.714+0000 7fe181203640 1 -- 172.21.15.116:0/2542007866 learned_addr learned my addr 172.21.15.116:0/2542007866 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:53.606 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.715+0000 7fe181203640 1 -- 172.21.15.116:0/2542007866 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe17c10a9c0 con 0x7fe17c108090 2024-09-12T02:53:53.606 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.716+0000 7fe181203640 1 --2- 172.21.15.116:0/2542007866 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe17c108090 0x7fe17c10a480 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe170009820 tx=0x7fe17002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea18f1336ea6d5a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:53.606 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.716+0000 7fe16ffff640 1 -- 172.21.15.116:0/2542007866 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe170037070 con 0x7fe17c108090 2024-09-12T02:53:53.606 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.716+0000 7fe16ffff640 1 -- 172.21.15.116:0/2542007866 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe170032920 con 0x7fe17c108090 2024-09-12T02:53:53.606 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.716+0000 7fe16ffff640 1 -- 172.21.15.116:0/2542007866 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe170030810 con 0x7fe17c108090 2024-09-12T02:53:53.606 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.717+0000 7fe183467640 1 -- 172.21.15.116:0/2542007866 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe17c108090 msgr2=0x7fe17c10a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:53.606 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.717+0000 7fe183467640 1 --2- 172.21.15.116:0/2542007866 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe17c108090 0x7fe17c10a480 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe170009820 tx=0x7fe17002a270 comp rx=0 tx=0).stop 2024-09-12T02:53:53.606 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.717+0000 7fe183467640 1 -- 172.21.15.116:0/2542007866 shutdown_connections 2024-09-12T02:53:53.607 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.717+0000 7fe183467640 1 --2- 172.21.15.116:0/2542007866 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe17c108090 0x7fe17c10a480 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:53.607 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.717+0000 7fe183467640 1 -- 172.21.15.116:0/2542007866 >> 172.21.15.116:0/2542007866 conn(0x7fe17c101db0 msgr2=0x7fe17c1041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:53.607 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.717+0000 7fe183467640 1 -- 172.21.15.116:0/2542007866 shutdown_connections 2024-09-12T02:53:53.607 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.717+0000 7fe183467640 1 -- 172.21.15.116:0/2542007866 wait complete. 2024-09-12T02:53:53.607 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.718+0000 7fe183467640 1 Processor -- start 2024-09-12T02:53:53.607 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.718+0000 7fe183467640 1 -- start start 2024-09-12T02:53:53.607 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.719+0000 7fe183467640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe17c108090 0x7fe17c19b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:53.607 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.719+0000 7fe183467640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe17c10baa0 con 0x7fe17c108090 2024-09-12T02:53:53.607 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.719+0000 7fe181203640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe17c108090 0x7fe17c19b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:53.607 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.719+0000 7fe181203640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe17c108090 0x7fe17c19b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59672/0 (socket says 172.21.15.116:59672) 2024-09-12T02:53:53.608 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.719+0000 7fe181203640 1 -- 172.21.15.116:0/691493167 learned_addr learned my addr 172.21.15.116:0/691493167 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:53.608 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.719+0000 7fe181203640 1 -- 172.21.15.116:0/691493167 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe17c19bb10 con 0x7fe17c108090 2024-09-12T02:53:53.608 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.719+0000 7fe181203640 1 --2- 172.21.15.116:0/691493167 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe17c108090 0x7fe17c19b5d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe1700097f0 tx=0x7fe1700327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:53.608 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.719+0000 7fe16e7fc640 1 -- 172.21.15.116:0/691493167 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe170037040 con 0x7fe17c108090 2024-09-12T02:53:53.608 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.719+0000 7fe16e7fc640 1 -- 172.21.15.116:0/691493167 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe170030cc0 con 0x7fe17c108090 2024-09-12T02:53:53.608 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.719+0000 7fe16e7fc640 1 -- 172.21.15.116:0/691493167 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe17003a6b0 con 0x7fe17c108090 2024-09-12T02:53:53.608 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.719+0000 7fe183467640 1 -- 172.21.15.116:0/691493167 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe17c19bda0 con 0x7fe17c108090 2024-09-12T02:53:53.608 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.719+0000 7fe183467640 1 -- 172.21.15.116:0/691493167 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe17c19ea90 con 0x7fe17c108090 2024-09-12T02:53:53.608 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.720+0000 7fe16e7fc640 1 -- 172.21.15.116:0/691493167 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fe17003a850 con 0x7fe17c108090 2024-09-12T02:53:53.608 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.720+0000 7fe16e7fc640 1 --2- 172.21.15.116:0/691493167 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fe168038fc0 0x7fe16803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:53.609 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.720+0000 7fe16e7fc640 1 -- 172.21.15.116:0/691493167 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe17006c570 con 0x7fe17c108090 2024-09-12T02:53:53.609 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.720+0000 7fe183467640 1 -- 172.21.15.116:0/691493167 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe17c1084f0 con 0x7fe17c108090 2024-09-12T02:53:53.609 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.721+0000 7fe180a02640 1 --2- 172.21.15.116:0/691493167 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fe168038fc0 0x7fe16803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:53.609 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.721+0000 7fe180a02640 1 --2- 172.21.15.116:0/691493167 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fe168038fc0 0x7fe16803b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe1780098c0 tx=0x7fe178006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:53.609 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.726+0000 7fe16e7fc640 1 -- 172.21.15.116:0/691493167 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe17006bd60 con 0x7fe17c108090 2024-09-12T02:53:53.609 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.921+0000 7fe183467640 1 -- 172.21.15.116:0/691493167 --> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fe17c102fb0 con 0x7fe168038fc0 2024-09-12T02:53:53.609 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.925+0000 7fe16e7fc640 1 -- 172.21.15.116:0/691493167 <== mgr.14118 v2:172.21.15.116:6800/2966274423 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe17c102fb0 con 0x7fe168038fc0 2024-09-12T02:53:53.609 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.927+0000 7fe183467640 1 -- 172.21.15.116:0/691493167 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fe168038fc0 msgr2=0x7fe16803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:53.609 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.927+0000 7fe183467640 1 --2- 172.21.15.116:0/691493167 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fe168038fc0 0x7fe16803b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe1780098c0 tx=0x7fe178006db0 comp rx=0 tx=0).stop 2024-09-12T02:53:53.609 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.927+0000 7fe183467640 1 -- 172.21.15.116:0/691493167 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe17c108090 msgr2=0x7fe17c19b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:53.609 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.927+0000 7fe183467640 1 --2- 172.21.15.116:0/691493167 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe17c108090 0x7fe17c19b5d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe1700097f0 tx=0x7fe1700327b0 comp rx=0 tx=0).stop 2024-09-12T02:53:53.610 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.927+0000 7fe183467640 1 -- 172.21.15.116:0/691493167 shutdown_connections 2024-09-12T02:53:53.610 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.927+0000 7fe183467640 1 --2- 172.21.15.116:0/691493167 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fe168038fc0 0x7fe16803b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:53.610 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.927+0000 7fe183467640 1 --2- 172.21.15.116:0/691493167 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe17c108090 0x7fe17c19b5d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:53.610 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.927+0000 7fe183467640 1 -- 172.21.15.116:0/691493167 >> 172.21.15.116:0/691493167 conn(0x7fe17c101db0 msgr2=0x7fe17c1028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:53.610 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.927+0000 7fe183467640 1 -- 172.21.15.116:0/691493167 shutdown_connections 2024-09-12T02:53:53.610 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:52.927+0000 7fe183467640 1 -- 172.21.15.116:0/691493167 wait complete. 2024-09-12T02:53:53.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:53 smithi116 ceph-mon[31410]: from='mgr.14118 172.21.15.116:0/1263688136' entity='mgr.x' 2024-09-12T02:53:53.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:53 smithi116 ceph-mon[31410]: from='mgr.14118 172.21.15.116:0/1263688136' entity='mgr.x' 2024-09-12T02:53:53.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:53 smithi116 ceph-mon[31410]: from='mgr.14118 172.21.15.116:0/1263688136' entity='mgr.x' 2024-09-12T02:53:54.609 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:54 smithi116 ceph-mon[31410]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:53:54.609 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:54 smithi116 ceph-mon[31410]: Saving service mgr spec with placement count:2 2024-09-12T02:53:54.610 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:54 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1248863715' entity='client.admin' 2024-09-12T02:53:55.481 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.105+0000 7fae04158640 1 Processor -- start 2024-09-12T02:53:55.481 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.105+0000 7fae04158640 1 -- start start 2024-09-12T02:53:55.481 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.105+0000 7fae04158640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadfc107740 0x7fadfc107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:55.481 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.105+0000 7fae04158640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc108110 con 0x7fadfc107740 2024-09-12T02:53:55.482 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.105+0000 7fae01ef4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadfc107740 0x7fadfc107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:55.482 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.105+0000 7fae01ef4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadfc107740 0x7fadfc107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59674/0 (socket says 172.21.15.116:59674) 2024-09-12T02:53:55.482 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.105+0000 7fae01ef4640 1 -- 172.21.15.116:0/4097812552 learned_addr learned my addr 172.21.15.116:0/4097812552 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:55.482 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.106+0000 7fae01ef4640 1 -- 172.21.15.116:0/4097812552 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadfc1088a0 con 0x7fadfc107740 2024-09-12T02:53:55.482 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.106+0000 7fae01ef4640 1 --2- 172.21.15.116:0/4097812552 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadfc107740 0x7fadfc107b40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009820 tx=0x7fadf802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933eccd7b2fbdf5f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:55.482 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.106+0000 7fae00ef2640 1 -- 172.21.15.116:0/4097812552 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadf8037070 con 0x7fadfc107740 2024-09-12T02:53:55.482 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.106+0000 7fae00ef2640 1 -- 172.21.15.116:0/4097812552 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadf8032920 con 0x7fadfc107740 2024-09-12T02:53:55.482 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.106+0000 7fae00ef2640 1 -- 172.21.15.116:0/4097812552 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadf8030810 con 0x7fadfc107740 2024-09-12T02:53:55.483 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.107+0000 7fae04158640 1 -- 172.21.15.116:0/4097812552 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadfc107740 msgr2=0x7fadfc107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:55.483 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.107+0000 7fae04158640 1 --2- 172.21.15.116:0/4097812552 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadfc107740 0x7fadfc107b40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009820 tx=0x7fadf802a270 comp rx=0 tx=0).stop 2024-09-12T02:53:55.483 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.107+0000 7fae04158640 1 -- 172.21.15.116:0/4097812552 shutdown_connections 2024-09-12T02:53:55.483 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.107+0000 7fae04158640 1 --2- 172.21.15.116:0/4097812552 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadfc107740 0x7fadfc107b40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:55.483 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.107+0000 7fae04158640 1 -- 172.21.15.116:0/4097812552 >> 172.21.15.116:0/4097812552 conn(0x7fadfc102eb0 msgr2=0x7fadfc105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:55.483 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.107+0000 7fae04158640 1 -- 172.21.15.116:0/4097812552 shutdown_connections 2024-09-12T02:53:55.483 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.107+0000 7fae04158640 1 -- 172.21.15.116:0/4097812552 wait complete. 2024-09-12T02:53:55.483 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.108+0000 7fae04158640 1 Processor -- start 2024-09-12T02:53:55.483 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.108+0000 7fae04158640 1 -- start start 2024-09-12T02:53:55.483 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.108+0000 7fae04158640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadfc107740 0x7fadfc19b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:55.483 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.108+0000 7fae04158640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc10bc50 con 0x7fadfc107740 2024-09-12T02:53:55.483 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.108+0000 7fae01ef4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadfc107740 0x7fadfc19b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:55.484 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.108+0000 7fae01ef4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadfc107740 0x7fadfc19b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59688/0 (socket says 172.21.15.116:59688) 2024-09-12T02:53:55.484 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.108+0000 7fae01ef4640 1 -- 172.21.15.116:0/1248863715 learned_addr learned my addr 172.21.15.116:0/1248863715 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:55.484 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.108+0000 7fae01ef4640 1 -- 172.21.15.116:0/1248863715 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadfc19bc10 con 0x7fadfc107740 2024-09-12T02:53:55.484 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.108+0000 7fae01ef4640 1 --2- 172.21.15.116:0/1248863715 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadfc107740 0x7fadfc19b6d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fadf8030560 tx=0x7fadf8032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:55.484 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.108+0000 7fadf2ffd640 1 -- 172.21.15.116:0/1248863715 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadf8037040 con 0x7fadfc107740 2024-09-12T02:53:55.484 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.108+0000 7fadf2ffd640 1 -- 172.21.15.116:0/1248863715 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadf8040070 con 0x7fadfc107740 2024-09-12T02:53:55.484 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.108+0000 7fae04158640 1 -- 172.21.15.116:0/1248863715 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadfc19bea0 con 0x7fadfc107740 2024-09-12T02:53:55.484 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.109+0000 7fadf2ffd640 1 -- 172.21.15.116:0/1248863715 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadf803a6d0 con 0x7fadfc107740 2024-09-12T02:53:55.484 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.109+0000 7fae04158640 1 -- 172.21.15.116:0/1248863715 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadfc19eb90 con 0x7fadfc107740 2024-09-12T02:53:55.484 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.109+0000 7fadf2ffd640 1 -- 172.21.15.116:0/1248863715 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fadf803a940 con 0x7fadfc107740 2024-09-12T02:53:55.484 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.109+0000 7fae04158640 1 -- 172.21.15.116:0/1248863715 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadfc107bc0 con 0x7fadfc107740 2024-09-12T02:53:55.484 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.110+0000 7fadf2ffd640 1 --2- 172.21.15.116:0/1248863715 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fade8038f70 0x7fade803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:55.484 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.110+0000 7fadf2ffd640 1 -- 172.21.15.116:0/1248863715 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fadf806c750 con 0x7fadfc107740 2024-09-12T02:53:55.485 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.110+0000 7fae016f3640 1 --2- 172.21.15.116:0/1248863715 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fade8038f70 0x7fade803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:55.485 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.110+0000 7fae016f3640 1 --2- 172.21.15.116:0/1248863715 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fade8038f70 0x7fade803b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fadec0098c0 tx=0x7fadec006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:55.485 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.114+0000 7fadf2ffd640 1 -- 172.21.15.116:0/1248863715 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fadf8032920 con 0x7fadfc107740 2024-09-12T02:53:55.485 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.303+0000 7fae04158640 1 -- 172.21.15.116:0/1248863715 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fadfc19f180 con 0x7fadfc107740 2024-09-12T02:53:55.485 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.305+0000 7fadf2ffd640 1 -- 172.21.15.116:0/1248863715 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fadf806c0a0 con 0x7fadfc107740 2024-09-12T02:53:55.485 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.307+0000 7fae04158640 1 -- 172.21.15.116:0/1248863715 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fade8038f70 msgr2=0x7fade803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:55.485 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.307+0000 7fae04158640 1 --2- 172.21.15.116:0/1248863715 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fade8038f70 0x7fade803b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fadec0098c0 tx=0x7fadec006db0 comp rx=0 tx=0).stop 2024-09-12T02:53:55.485 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.307+0000 7fae04158640 1 -- 172.21.15.116:0/1248863715 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadfc107740 msgr2=0x7fadfc19b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:55.485 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.307+0000 7fae04158640 1 --2- 172.21.15.116:0/1248863715 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadfc107740 0x7fadfc19b6d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fadf8030560 tx=0x7fadf8032670 comp rx=0 tx=0).stop 2024-09-12T02:53:55.485 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.308+0000 7fae04158640 1 -- 172.21.15.116:0/1248863715 shutdown_connections 2024-09-12T02:53:55.485 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.308+0000 7fae04158640 1 --2- 172.21.15.116:0/1248863715 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fade8038f70 0x7fade803b430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:55.485 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.308+0000 7fae04158640 1 --2- 172.21.15.116:0/1248863715 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadfc107740 0x7fadfc19b6d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:55.486 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.308+0000 7fae04158640 1 -- 172.21.15.116:0/1248863715 >> 172.21.15.116:0/1248863715 conn(0x7fadfc102eb0 msgr2=0x7fadfc103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:55.486 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.308+0000 7fae04158640 1 -- 172.21.15.116:0/1248863715 shutdown_connections 2024-09-12T02:53:55.486 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:54.308+0000 7fae04158640 1 -- 172.21.15.116:0/1248863715 wait complete. 2024-09-12T02:53:56.138 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:55 smithi116 ceph-mon[31410]: from='mgr.14118 172.21.15.116:0/1263688136' entity='mgr.x' 2024-09-12T02:53:56.138 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:55 smithi116 ceph-mon[31410]: from='mgr.14118 172.21.15.116:0/1263688136' entity='mgr.x' 2024-09-12T02:53:56.721 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.952+0000 7f05f2705640 1 Processor -- start 2024-09-12T02:53:56.722 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.952+0000 7f05f2705640 1 -- start start 2024-09-12T02:53:56.722 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.952+0000 7f05f2705640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05ec106520 0x7f05ec106920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:56.722 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.952+0000 7f05f2705640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec106ef0 con 0x7f05ec106520 2024-09-12T02:53:56.722 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.953+0000 7f05ebfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05ec106520 0x7f05ec106920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:56.722 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.953+0000 7f05ebfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05ec106520 0x7f05ec106920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59692/0 (socket says 172.21.15.116:59692) 2024-09-12T02:53:56.722 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.953+0000 7f05ebfff640 1 -- 172.21.15.116:0/3164601199 learned_addr learned my addr 172.21.15.116:0/3164601199 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:56.722 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.953+0000 7f05ebfff640 1 -- 172.21.15.116:0/3164601199 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05ec107720 con 0x7f05ec106520 2024-09-12T02:53:56.722 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.954+0000 7f05ebfff640 1 --2- 172.21.15.116:0/3164601199 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05ec106520 0x7f05ec106920 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f05dc009820 tx=0x7f05dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e839b5eba8c9d8ea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:56.722 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.954+0000 7f05eaffd640 1 -- 172.21.15.116:0/3164601199 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05dc037070 con 0x7f05ec106520 2024-09-12T02:53:56.722 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.954+0000 7f05eaffd640 1 -- 172.21.15.116:0/3164601199 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05dc032920 con 0x7f05ec106520 2024-09-12T02:53:56.723 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.954+0000 7f05eaffd640 1 -- 172.21.15.116:0/3164601199 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05dc030810 con 0x7f05ec106520 2024-09-12T02:53:56.723 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.955+0000 7f05f2705640 1 -- 172.21.15.116:0/3164601199 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05ec106520 msgr2=0x7f05ec106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:56.723 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.955+0000 7f05f2705640 1 --2- 172.21.15.116:0/3164601199 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05ec106520 0x7f05ec106920 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f05dc009820 tx=0x7f05dc02a270 comp rx=0 tx=0).stop 2024-09-12T02:53:56.723 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.955+0000 7f05f2705640 1 -- 172.21.15.116:0/3164601199 shutdown_connections 2024-09-12T02:53:56.723 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.955+0000 7f05f2705640 1 --2- 172.21.15.116:0/3164601199 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05ec106520 0x7f05ec106920 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.723 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.955+0000 7f05f2705640 1 -- 172.21.15.116:0/3164601199 >> 172.21.15.116:0/3164601199 conn(0x7f05ec101db0 msgr2=0x7f05ec1041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:56.723 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.955+0000 7f05f2705640 1 -- 172.21.15.116:0/3164601199 shutdown_connections 2024-09-12T02:53:56.723 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.955+0000 7f05f2705640 1 -- 172.21.15.116:0/3164601199 wait complete. 2024-09-12T02:53:56.723 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.955+0000 7f05f2705640 1 Processor -- start 2024-09-12T02:53:56.723 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.955+0000 7f05f2705640 1 -- start start 2024-09-12T02:53:56.723 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.956+0000 7f05f2705640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05ec106520 0x7f05ec19b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:56.723 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.956+0000 7f05f2705640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec10aad0 con 0x7f05ec106520 2024-09-12T02:53:56.723 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.956+0000 7f05ebfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05ec106520 0x7f05ec19b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:56.723 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.956+0000 7f05ebfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05ec106520 0x7f05ec19b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59700/0 (socket says 172.21.15.116:59700) 2024-09-12T02:53:56.724 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.956+0000 7f05ebfff640 1 -- 172.21.15.116:0/1217112505 learned_addr learned my addr 172.21.15.116:0/1217112505 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:56.724 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.956+0000 7f05ebfff640 1 -- 172.21.15.116:0/1217112505 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05ec19bba0 con 0x7f05ec106520 2024-09-12T02:53:56.724 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.956+0000 7f05ebfff640 1 --2- 172.21.15.116:0/1217112505 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05ec106520 0x7f05ec19b660 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f05dc030560 tx=0x7f05dc032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:56.724 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.957+0000 7f05e97fa640 1 -- 172.21.15.116:0/1217112505 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05dc037040 con 0x7f05ec106520 2024-09-12T02:53:56.724 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.957+0000 7f05e97fa640 1 -- 172.21.15.116:0/1217112505 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05dc040070 con 0x7f05ec106520 2024-09-12T02:53:56.724 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.957+0000 7f05f2705640 1 -- 172.21.15.116:0/1217112505 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05ec19be30 con 0x7f05ec106520 2024-09-12T02:53:56.724 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.957+0000 7f05e97fa640 1 -- 172.21.15.116:0/1217112505 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05dc03a740 con 0x7f05ec106520 2024-09-12T02:53:56.724 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.957+0000 7f05f2705640 1 -- 172.21.15.116:0/1217112505 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05ec19eb20 con 0x7f05ec106520 2024-09-12T02:53:56.724 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.958+0000 7f05e97fa640 1 -- 172.21.15.116:0/1217112505 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f05dc03a9b0 con 0x7f05ec106520 2024-09-12T02:53:56.724 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.958+0000 7f05f2705640 1 -- 172.21.15.116:0/1217112505 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05ec10ac50 con 0x7f05ec106520 2024-09-12T02:53:56.724 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.958+0000 7f05e97fa640 1 --2- 172.21.15.116:0/1217112505 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f05d4038f70 0x7f05d403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:56.724 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.958+0000 7f05e97fa640 1 -- 172.21.15.116:0/1217112505 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f05dc06c5c0 con 0x7f05ec106520 2024-09-12T02:53:56.724 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.958+0000 7f05eb7fe640 1 --2- 172.21.15.116:0/1217112505 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f05d4038f70 0x7f05d403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:56.725 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.958+0000 7f05eb7fe640 1 --2- 172.21.15.116:0/1217112505 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f05d4038f70 0x7f05d403b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f05e00098c0 tx=0x7f05e0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:56.725 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:55.963+0000 7f05e97fa640 1 -- 172.21.15.116:0/1217112505 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f05dc041270 con 0x7f05ec106520 2024-09-12T02:53:56.725 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:56.138+0000 7f05f2705640 1 -- 172.21.15.116:0/1217112505 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f05ec02da20 con 0x7f05ec106520 2024-09-12T02:53:56.725 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:56.141+0000 7f05e97fa640 1 -- 172.21.15.116:0/1217112505 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f05dc03ada0 con 0x7f05ec106520 2024-09-12T02:53:56.725 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:56.143+0000 7f05f2705640 1 -- 172.21.15.116:0/1217112505 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f05d4038f70 msgr2=0x7f05d403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:56.725 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:56.143+0000 7f05f2705640 1 --2- 172.21.15.116:0/1217112505 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f05d4038f70 0x7f05d403b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f05e00098c0 tx=0x7f05e0006db0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.725 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:56.143+0000 7f05f2705640 1 -- 172.21.15.116:0/1217112505 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05ec106520 msgr2=0x7f05ec19b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:56.725 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:56.143+0000 7f05f2705640 1 --2- 172.21.15.116:0/1217112505 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05ec106520 0x7f05ec19b660 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f05dc030560 tx=0x7f05dc032670 comp rx=0 tx=0).stop 2024-09-12T02:53:56.725 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:56.144+0000 7f05f2705640 1 -- 172.21.15.116:0/1217112505 shutdown_connections 2024-09-12T02:53:56.725 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:56.144+0000 7f05f2705640 1 --2- 172.21.15.116:0/1217112505 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f05d4038f70 0x7f05d403b430 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.725 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:56.144+0000 7f05f2705640 1 --2- 172.21.15.116:0/1217112505 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05ec106520 0x7f05ec19b660 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:56.725 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:56.144+0000 7f05f2705640 1 -- 172.21.15.116:0/1217112505 >> 172.21.15.116:0/1217112505 conn(0x7f05ec101db0 msgr2=0x7f05ec102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:56.725 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:56.144+0000 7f05f2705640 1 -- 172.21.15.116:0/1217112505 shutdown_connections 2024-09-12T02:53:56.726 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:56.144+0000 7f05f2705640 1 -- 172.21.15.116:0/1217112505 wait complete. 2024-09-12T02:53:56.726 INFO:teuthology.orchestra.run.smithi116.stdout:Enabling the dashboard module... 2024-09-12T02:53:57.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:57 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1217112505' entity='client.admin' 2024-09-12T02:53:58.476 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:58 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2075830089' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-12T02:53:58.477 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:58 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: ignoring --setuser ceph since I am not root 2024-09-12T02:53:58.477 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:58 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: ignoring --setgroup ceph since I am not root 2024-09-12T02:53:58.477 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:58 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:58.290+0000 7fc384f35040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T02:53:58.477 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:58 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:58.347+0000 7fc384f35040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T02:53:58.802 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:58 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:58.498+0000 7fc384f35040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T02:53:58.802 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:58 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:58.566+0000 7fc384f35040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T02:53:58.802 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:58 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:58.715+0000 7fc384f35040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:53:58.809 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.155+0000 7f0b9fa3e640 1 Processor -- start 2024-09-12T02:53:58.809 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.155+0000 7f0b9fa3e640 1 -- start start 2024-09-12T02:53:58.809 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.155+0000 7f0b9fa3e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b98106a40 0x7f0b98106e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:58.809 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.155+0000 7f0b9fa3e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0b98107410 con 0x7f0b98106a40 2024-09-12T02:53:58.809 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.156+0000 7f0b9d7da640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b98106a40 0x7f0b98106e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:58.810 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.156+0000 7f0b9d7da640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b98106a40 0x7f0b98106e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59706/0 (socket says 172.21.15.116:59706) 2024-09-12T02:53:58.810 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.156+0000 7f0b9d7da640 1 -- 172.21.15.116:0/589288333 learned_addr learned my addr 172.21.15.116:0/589288333 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:58.810 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.156+0000 7f0b9d7da640 1 -- 172.21.15.116:0/589288333 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b98107760 con 0x7f0b98106a40 2024-09-12T02:53:58.810 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.157+0000 7f0b9d7da640 1 --2- 172.21.15.116:0/589288333 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b98106a40 0x7f0b98106e40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0b94009820 tx=0x7f0b9402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a9d1063d102d16 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:58.810 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.157+0000 7f0b8bfff640 1 -- 172.21.15.116:0/589288333 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b94037070 con 0x7f0b98106a40 2024-09-12T02:53:58.810 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.157+0000 7f0b8bfff640 1 -- 172.21.15.116:0/589288333 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b94032920 con 0x7f0b98106a40 2024-09-12T02:53:58.810 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.157+0000 7f0b8bfff640 1 -- 172.21.15.116:0/589288333 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b94030810 con 0x7f0b98106a40 2024-09-12T02:53:58.810 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.157+0000 7f0b9fa3e640 1 -- 172.21.15.116:0/589288333 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b98106a40 msgr2=0x7f0b98106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:58.810 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.157+0000 7f0b9fa3e640 1 --2- 172.21.15.116:0/589288333 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b98106a40 0x7f0b98106e40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0b94009820 tx=0x7f0b9402a270 comp rx=0 tx=0).stop 2024-09-12T02:53:58.810 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.158+0000 7f0b9fa3e640 1 -- 172.21.15.116:0/589288333 shutdown_connections 2024-09-12T02:53:58.810 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.158+0000 7f0b9fa3e640 1 --2- 172.21.15.116:0/589288333 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b98106a40 0x7f0b98106e40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:58.810 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.158+0000 7f0b9fa3e640 1 -- 172.21.15.116:0/589288333 >> 172.21.15.116:0/589288333 conn(0x7f0b98101510 msgr2=0x7f0b98103930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:58.810 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.158+0000 7f0b9fa3e640 1 -- 172.21.15.116:0/589288333 shutdown_connections 2024-09-12T02:53:58.811 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.158+0000 7f0b9fa3e640 1 -- 172.21.15.116:0/589288333 wait complete. 2024-09-12T02:53:58.811 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.158+0000 7f0b9fa3e640 1 Processor -- start 2024-09-12T02:53:58.811 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.158+0000 7f0b9fa3e640 1 -- start start 2024-09-12T02:53:58.811 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.158+0000 7f0b9fa3e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b98106a40 0x7f0b98106020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:58.811 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.158+0000 7f0b9fa3e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9810ab10 con 0x7f0b98106a40 2024-09-12T02:53:58.811 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.158+0000 7f0b9d7da640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b98106a40 0x7f0b98106020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:58.811 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.158+0000 7f0b9d7da640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b98106a40 0x7f0b98106020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59722/0 (socket says 172.21.15.116:59722) 2024-09-12T02:53:58.811 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.158+0000 7f0b9d7da640 1 -- 172.21.15.116:0/2075830089 learned_addr learned my addr 172.21.15.116:0/2075830089 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:53:58.811 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.159+0000 7f0b9d7da640 1 -- 172.21.15.116:0/2075830089 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b9819e3c0 con 0x7f0b98106a40 2024-09-12T02:53:58.812 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.159+0000 7f0b9d7da640 1 --2- 172.21.15.116:0/2075830089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b98106a40 0x7f0b98106020 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0b94030560 tx=0x7f0b94032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:58.812 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.159+0000 7f0b8a7fc640 1 -- 172.21.15.116:0/2075830089 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b94037040 con 0x7f0b98106a40 2024-09-12T02:53:58.812 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.159+0000 7f0b9fa3e640 1 -- 172.21.15.116:0/2075830089 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b9819e590 con 0x7f0b98106a40 2024-09-12T02:53:58.812 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.159+0000 7f0b8a7fc640 1 -- 172.21.15.116:0/2075830089 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b94040070 con 0x7f0b98106a40 2024-09-12T02:53:58.812 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.159+0000 7f0b9fa3e640 1 -- 172.21.15.116:0/2075830089 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b9819e930 con 0x7f0b98106a40 2024-09-12T02:53:58.812 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.159+0000 7f0b8a7fc640 1 -- 172.21.15.116:0/2075830089 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b9403a740 con 0x7f0b98106a40 2024-09-12T02:53:58.812 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.160+0000 7f0b8a7fc640 1 -- 172.21.15.116:0/2075830089 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f0b9403a9b0 con 0x7f0b98106a40 2024-09-12T02:53:58.812 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.160+0000 7f0b9fa3e640 1 -- 172.21.15.116:0/2075830089 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b98106e40 con 0x7f0b98106a40 2024-09-12T02:53:58.812 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.160+0000 7f0b8a7fc640 1 --2- 172.21.15.116:0/2075830089 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f0b84038f70 0x7f0b8403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:58.812 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.161+0000 7f0b8a7fc640 1 -- 172.21.15.116:0/2075830089 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0b9406c720 con 0x7f0b98106a40 2024-09-12T02:53:58.813 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.161+0000 7f0b9cfd9640 1 --2- 172.21.15.116:0/2075830089 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f0b84038f70 0x7f0b8403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:58.813 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.161+0000 7f0b9cfd9640 1 --2- 172.21.15.116:0/2075830089 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f0b84038f70 0x7f0b8403b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c0098c0 tx=0x7f0b8c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:58.813 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.165+0000 7f0b8a7fc640 1 -- 172.21.15.116:0/2075830089 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0b9403adf0 con 0x7f0b98106a40 2024-09-12T02:53:58.814 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:57.448+0000 7f0b9fa3e640 1 -- 172.21.15.116:0/2075830089 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f0b9819f170 con 0x7f0b98106a40 2024-09-12T02:53:58.814 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:58.143+0000 7f0b8a7fc640 1 -- 172.21.15.116:0/2075830089 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f0b940328f0 con 0x7f0b98106a40 2024-09-12T02:53:58.814 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:58.145+0000 7f0b8a7fc640 1 -- 172.21.15.116:0/2075830089 <== mon.0 v2:172.21.15.116:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f0b94045980 con 0x7f0b98106a40 2024-09-12T02:53:58.814 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:58.146+0000 7f0b9fa3e640 1 -- 172.21.15.116:0/2075830089 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f0b84038f70 msgr2=0x7f0b8403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:58.814 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:58.146+0000 7f0b9fa3e640 1 --2- 172.21.15.116:0/2075830089 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f0b84038f70 0x7f0b8403b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c0098c0 tx=0x7f0b8c006db0 comp rx=0 tx=0).stop 2024-09-12T02:53:58.814 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:58.146+0000 7f0b9fa3e640 1 -- 172.21.15.116:0/2075830089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b98106a40 msgr2=0x7f0b98106020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:58.814 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:58.146+0000 7f0b9fa3e640 1 --2- 172.21.15.116:0/2075830089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b98106a40 0x7f0b98106020 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0b94030560 tx=0x7f0b94032670 comp rx=0 tx=0).stop 2024-09-12T02:53:58.814 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:58.146+0000 7f0b9fa3e640 1 -- 172.21.15.116:0/2075830089 shutdown_connections 2024-09-12T02:53:58.814 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:58.146+0000 7f0b9fa3e640 1 --2- 172.21.15.116:0/2075830089 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f0b84038f70 0x7f0b8403b430 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:58.814 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:58.146+0000 7f0b9fa3e640 1 --2- 172.21.15.116:0/2075830089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b98106a40 0x7f0b98106020 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:58.814 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:58.146+0000 7f0b9fa3e640 1 -- 172.21.15.116:0/2075830089 >> 172.21.15.116:0/2075830089 conn(0x7f0b98101510 msgr2=0x7f0b98101fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:58.814 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:58.147+0000 7f0b9fa3e640 1 -- 172.21.15.116:0/2075830089 shutdown_connections 2024-09-12T02:53:58.815 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:58.147+0000 7f0b9fa3e640 1 -- 172.21.15.116:0/2075830089 wait complete. 2024-09-12T02:53:59.052 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:58 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:58.862+0000 7fc384f35040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T02:53:59.390 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:59 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2075830089' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-12T02:53:59.390 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:53:59 smithi116 ceph-mon[31410]: mgrmap e9: x(active, since 18s) 2024-09-12T02:53:59.391 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:59 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:59.186+0000 7fc384f35040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T02:53:59.391 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:59 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:59.250+0000 7fc384f35040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T02:53:59.716 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:53:59 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:53:59.446+0000 7fc384f35040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T02:54:00.009 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:54:00.009 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-12T02:54:00.009 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T02:54:00.009 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-12T02:54:00.010 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-12T02:54:00.010 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:54:00.010 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.183+0000 7f741788c640 1 Processor -- start 2024-09-12T02:54:00.010 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.183+0000 7f741788c640 1 -- start start 2024-09-12T02:54:00.010 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.184+0000 7f741788c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74101080d0 0x7f741010a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:00.010 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.184+0000 7f741788c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f741007be50 con 0x7f74101080d0 2024-09-12T02:54:00.010 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.184+0000 7f7415628640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74101080d0 0x7f741010a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:00.010 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.184+0000 7f7415628640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74101080d0 0x7f741010a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59750/0 (socket says 172.21.15.116:59750) 2024-09-12T02:54:00.010 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.184+0000 7f7415628640 1 -- 172.21.15.116:0/2044333871 learned_addr learned my addr 172.21.15.116:0/2044333871 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:54:00.010 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.184+0000 7f7415628640 1 -- 172.21.15.116:0/2044333871 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f741010aa00 con 0x7f74101080d0 2024-09-12T02:54:00.010 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.185+0000 7f7415628640 1 --2- 172.21.15.116:0/2044333871 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74101080d0 0x7f741010a4c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7404009820 tx=0x7f740402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d71024976d7fe40 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:00.010 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.185+0000 7f7403fff640 1 -- 172.21.15.116:0/2044333871 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7404037070 con 0x7f74101080d0 2024-09-12T02:54:00.010 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.185+0000 7f7403fff640 1 -- 172.21.15.116:0/2044333871 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7404032920 con 0x7f74101080d0 2024-09-12T02:54:00.011 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.185+0000 7f7403fff640 1 -- 172.21.15.116:0/2044333871 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7404030810 con 0x7f74101080d0 2024-09-12T02:54:00.011 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.186+0000 7f741788c640 1 -- 172.21.15.116:0/2044333871 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74101080d0 msgr2=0x7f741010a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:00.011 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.186+0000 7f741788c640 1 --2- 172.21.15.116:0/2044333871 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74101080d0 0x7f741010a4c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7404009820 tx=0x7f740402a270 comp rx=0 tx=0).stop 2024-09-12T02:54:00.011 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.186+0000 7f741788c640 1 -- 172.21.15.116:0/2044333871 shutdown_connections 2024-09-12T02:54:00.011 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.186+0000 7f741788c640 1 --2- 172.21.15.116:0/2044333871 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74101080d0 0x7f741010a4c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:00.011 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.186+0000 7f741788c640 1 -- 172.21.15.116:0/2044333871 >> 172.21.15.116:0/2044333871 conn(0x7f7410101db0 msgr2=0x7f74101041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:00.011 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.186+0000 7f741788c640 1 -- 172.21.15.116:0/2044333871 shutdown_connections 2024-09-12T02:54:00.011 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.186+0000 7f741788c640 1 -- 172.21.15.116:0/2044333871 wait complete. 2024-09-12T02:54:00.011 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.186+0000 7f741788c640 1 Processor -- start 2024-09-12T02:54:00.011 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.186+0000 7f741788c640 1 -- start start 2024-09-12T02:54:00.011 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.187+0000 7f741788c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74101080d0 0x7f7410107950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:00.011 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.187+0000 7f741788c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f741010ba80 con 0x7f74101080d0 2024-09-12T02:54:00.011 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.187+0000 7f7415628640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74101080d0 0x7f7410107950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:00.012 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.187+0000 7f7415628640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74101080d0 0x7f7410107950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59754/0 (socket says 172.21.15.116:59754) 2024-09-12T02:54:00.012 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.187+0000 7f7415628640 1 -- 172.21.15.116:0/2751743800 learned_addr learned my addr 172.21.15.116:0/2751743800 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:54:00.012 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.187+0000 7f7415628640 1 -- 172.21.15.116:0/2751743800 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7410107e90 con 0x7f74101080d0 2024-09-12T02:54:00.012 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.187+0000 7f7415628640 1 --2- 172.21.15.116:0/2751743800 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74101080d0 0x7f7410107950 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7404030380 tx=0x7f7404032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:00.012 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.187+0000 7f74027fc640 1 -- 172.21.15.116:0/2751743800 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7404037040 con 0x7f74101080d0 2024-09-12T02:54:00.012 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.187+0000 7f741788c640 1 -- 172.21.15.116:0/2751743800 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7410106080 con 0x7f74101080d0 2024-09-12T02:54:00.012 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.187+0000 7f74027fc640 1 -- 172.21.15.116:0/2751743800 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7404040070 con 0x7f74101080d0 2024-09-12T02:54:00.012 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.187+0000 7f741788c640 1 -- 172.21.15.116:0/2751743800 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74101065c0 con 0x7f74101080d0 2024-09-12T02:54:00.012 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.187+0000 7f74027fc640 1 -- 172.21.15.116:0/2751743800 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f740403a7b0 con 0x7f74101080d0 2024-09-12T02:54:00.012 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.188+0000 7f74027fc640 1 -- 172.21.15.116:0/2751743800 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f740403a9e0 con 0x7f74101080d0 2024-09-12T02:54:00.012 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.188+0000 7f741788c640 1 -- 172.21.15.116:0/2751743800 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7410108530 con 0x7f74101080d0 2024-09-12T02:54:00.012 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.189+0000 7f74027fc640 1 --2- 172.21.15.116:0/2751743800 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f73fc039020 0x7f73fc03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:00.012 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.189+0000 7f74027fc640 1 -- 172.21.15.116:0/2751743800 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f740406c640 con 0x7f74101080d0 2024-09-12T02:54:00.013 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.189+0000 7f7414e27640 1 -- 172.21.15.116:0/2751743800 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f73fc039020 msgr2=0x7f73fc03b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.116:6800/2966274423 2024-09-12T02:54:00.013 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.189+0000 7f7414e27640 1 --2- 172.21.15.116:0/2751743800 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f73fc039020 0x7f73fc03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:54:00.013 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.193+0000 7f74027fc640 1 -- 172.21.15.116:0/2751743800 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7404041270 con 0x7f74101080d0 2024-09-12T02:54:00.013 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.390+0000 7f7414e27640 1 -- 172.21.15.116:0/2751743800 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f73fc039020 msgr2=0x7f73fc03b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.116:6800/2966274423 2024-09-12T02:54:00.013 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.390+0000 7f7414e27640 1 --2- 172.21.15.116:0/2751743800 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f73fc039020 0x7f73fc03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T02:54:00.013 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.412+0000 7f741788c640 1 -- 172.21.15.116:0/2751743800 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f7410106d60 con 0x7f74101080d0 2024-09-12T02:54:00.013 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.413+0000 7f74027fc640 1 -- 172.21.15.116:0/2751743800 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f740403ae20 con 0x7f74101080d0 2024-09-12T02:54:00.013 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.415+0000 7f741788c640 1 -- 172.21.15.116:0/2751743800 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f73fc039020 msgr2=0x7f73fc03b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:54:00.013 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.415+0000 7f741788c640 1 --2- 172.21.15.116:0/2751743800 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f73fc039020 0x7f73fc03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:00.013 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.415+0000 7f741788c640 1 -- 172.21.15.116:0/2751743800 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74101080d0 msgr2=0x7f7410107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:00.013 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.415+0000 7f741788c640 1 --2- 172.21.15.116:0/2751743800 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74101080d0 0x7f7410107950 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7404030380 tx=0x7f7404032670 comp rx=0 tx=0).stop 2024-09-12T02:54:00.013 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.415+0000 7f741788c640 1 -- 172.21.15.116:0/2751743800 shutdown_connections 2024-09-12T02:54:00.013 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.415+0000 7f741788c640 1 --2- 172.21.15.116:0/2751743800 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7f73fc039020 0x7f73fc03b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:00.013 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.415+0000 7f741788c640 1 --2- 172.21.15.116:0/2751743800 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74101080d0 0x7f7410107950 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:00.014 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.415+0000 7f741788c640 1 -- 172.21.15.116:0/2751743800 >> 172.21.15.116:0/2751743800 conn(0x7f7410101db0 msgr2=0x7f74101028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:00.014 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.415+0000 7f741788c640 1 -- 172.21.15.116:0/2751743800 shutdown_connections 2024-09-12T02:54:00.014 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:53:59.415+0000 7f741788c640 1 -- 172.21.15.116:0/2751743800 wait complete. 2024-09-12T02:54:00.014 INFO:teuthology.orchestra.run.smithi116.stdout:Waiting for the mgr to restart... 2024-09-12T02:54:00.014 INFO:teuthology.orchestra.run.smithi116.stdout:Waiting for mgr epoch 9... 2024-09-12T02:54:00.279 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:00 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2751743800' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-12T02:54:00.552 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:00 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:54:00.366+0000 7fc384f35040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T02:54:00.552 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:00 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:54:00.437+0000 7fc384f35040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T02:54:01.302 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:01 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:54:01.149+0000 7fc384f35040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T02:54:01.302 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:01 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:54:01.265+0000 7fc384f35040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T02:54:01.684 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:01 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:54:01.407+0000 7fc384f35040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:54:01.685 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:01 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:54:01.468+0000 7fc384f35040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T02:54:01.685 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:01 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:54:01.616+0000 7fc384f35040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T02:54:01.685 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:01 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:54:01.684+0000 7fc384f35040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T02:54:01.968 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:01 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:54:01.744+0000 7fc384f35040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T02:54:01.968 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:01 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:54:01.901+0000 7fc384f35040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T02:54:01.968 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:01 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:54:01.967+0000 7fc384f35040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T02:54:02.235 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:02 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:54:02.034+0000 7fc384f35040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T02:54:02.236 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:02 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:54:02.117+0000 7fc384f35040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T02:54:02.236 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:02 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:54:02.176+0000 7fc384f35040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T02:54:02.236 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:02 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:54:02.235+0000 7fc384f35040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T02:54:03.302 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:03 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:54:03.024+0000 7fc384f35040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T02:54:03.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:03 smithi116 ceph-mon[31410]: Active manager daemon x restarted 2024-09-12T02:54:03.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:03 smithi116 ceph-mon[31410]: Activating manager daemon x 2024-09-12T02:54:03.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:03 smithi116 ceph-mon[31410]: osdmap e3: 0 total, 0 up, 0 in 2024-09-12T02:54:03.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:03 smithi116 ceph-mon[31410]: mgrmap e10: x(active, starting, since 0.00349451s) 2024-09-12T02:54:03.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:03 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:54:03.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:03 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T02:54:03.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:03 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T02:54:03.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:03 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T02:54:03.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:03 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T02:54:03.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:03 smithi116 ceph-mon[31410]: Manager daemon x is now available 2024-09-12T02:54:03.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:03 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:03.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:03 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T02:54:03.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:03 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T02:54:04.511 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:04 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:54:04.396+0000 7fc2f0249640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:02:54:04] ENGINE Error in HTTPServer.serve 2024-09-12T02:54:04.511 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:04 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: Traceback (most recent call last): 2024-09-12T02:54:04.511 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:04 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T02:54:04.511 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:04 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: self._connections.run(self.expiration_interval) 2024-09-12T02:54:04.511 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:04 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T02:54:04.511 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:04 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: self._run(expiration_interval) 2024-09-12T02:54:04.511 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:04 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T02:54:04.512 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:04 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T02:54:04.512 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:04 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T02:54:04.512 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:04 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T02:54:04.512 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:04 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T02:54:04.512 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:04 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: s = self.context.wrap_socket( 2024-09-12T02:54:04.512 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:04 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T02:54:04.512 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:04 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: return self.sslsocket_class._create( 2024-09-12T02:54:04.512 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:04 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T02:54:04.512 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:04 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: self.do_handshake() 2024-09-12T02:54:04.512 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:04 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T02:54:04.512 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:04 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: self._sslobj.do_handshake() 2024-09-12T02:54:04.512 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:04 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T02:54:04.512 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 02:54:04 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T02:54:04.879 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:54:04.879 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-12T02:54:04.879 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-12T02:54:04.879 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:54:04.879 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.439+0000 7fa390e7f640 1 Processor -- start 2024-09-12T02:54:04.879 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.439+0000 7fa390e7f640 1 -- start start 2024-09-12T02:54:04.880 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.440+0000 7fa390e7f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa38c1080c0 0x7fa38c10a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:04.880 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.440+0000 7fa390e7f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa38c07be20 con 0x7fa38c1080c0 2024-09-12T02:54:04.880 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.440+0000 7fa38a59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa38c1080c0 0x7fa38c10a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:04.880 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.440+0000 7fa38a59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa38c1080c0 0x7fa38c10a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59760/0 (socket says 172.21.15.116:59760) 2024-09-12T02:54:04.880 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.440+0000 7fa38a59c640 1 -- 172.21.15.116:0/716631344 learned_addr learned my addr 172.21.15.116:0/716631344 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:54:04.880 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.440+0000 7fa38a59c640 1 -- 172.21.15.116:0/716631344 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa38c10a9f0 con 0x7fa38c1080c0 2024-09-12T02:54:04.881 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.441+0000 7fa38a59c640 1 --2- 172.21.15.116:0/716631344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa38c1080c0 0x7fa38c10a4b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa37c009820 tx=0x7fa37c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af096be2eaf1195a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:04.881 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.441+0000 7fa38959a640 1 -- 172.21.15.116:0/716631344 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa37c037070 con 0x7fa38c1080c0 2024-09-12T02:54:04.881 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.441+0000 7fa38959a640 1 -- 172.21.15.116:0/716631344 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa37c032920 con 0x7fa38c1080c0 2024-09-12T02:54:04.881 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.441+0000 7fa38959a640 1 -- 172.21.15.116:0/716631344 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa37c030810 con 0x7fa38c1080c0 2024-09-12T02:54:04.881 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.442+0000 7fa390e7f640 1 -- 172.21.15.116:0/716631344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa38c1080c0 msgr2=0x7fa38c10a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:04.881 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.442+0000 7fa390e7f640 1 --2- 172.21.15.116:0/716631344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa38c1080c0 0x7fa38c10a4b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa37c009820 tx=0x7fa37c02a270 comp rx=0 tx=0).stop 2024-09-12T02:54:04.882 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.442+0000 7fa390e7f640 1 -- 172.21.15.116:0/716631344 shutdown_connections 2024-09-12T02:54:04.882 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.442+0000 7fa390e7f640 1 --2- 172.21.15.116:0/716631344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa38c1080c0 0x7fa38c10a4b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:04.882 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.442+0000 7fa390e7f640 1 -- 172.21.15.116:0/716631344 >> 172.21.15.116:0/716631344 conn(0x7fa38c101ca0 msgr2=0x7fa38c1040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:04.882 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.442+0000 7fa390e7f640 1 -- 172.21.15.116:0/716631344 shutdown_connections 2024-09-12T02:54:04.882 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.442+0000 7fa390e7f640 1 -- 172.21.15.116:0/716631344 wait complete. 2024-09-12T02:54:04.882 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.443+0000 7fa390e7f640 1 Processor -- start 2024-09-12T02:54:04.882 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.443+0000 7fa390e7f640 1 -- start start 2024-09-12T02:54:04.882 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.443+0000 7fa390e7f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa38c1080c0 0x7fa38c107940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:04.883 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.443+0000 7fa390e7f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa38c10ba10 con 0x7fa38c1080c0 2024-09-12T02:54:04.883 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.443+0000 7fa38a59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa38c1080c0 0x7fa38c107940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:04.883 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.443+0000 7fa38a59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa38c1080c0 0x7fa38c107940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59776/0 (socket says 172.21.15.116:59776) 2024-09-12T02:54:04.883 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.443+0000 7fa38a59c640 1 -- 172.21.15.116:0/2194999890 learned_addr learned my addr 172.21.15.116:0/2194999890 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:54:04.883 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.443+0000 7fa38a59c640 1 -- 172.21.15.116:0/2194999890 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa38c107e80 con 0x7fa38c1080c0 2024-09-12T02:54:04.883 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.443+0000 7fa38a59c640 1 --2- 172.21.15.116:0/2194999890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa38c1080c0 0x7fa38c107940 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa37c030380 tx=0x7fa37c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:04.883 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.443+0000 7fa3837fe640 1 -- 172.21.15.116:0/2194999890 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa37c037040 con 0x7fa38c1080c0 2024-09-12T02:54:04.883 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.443+0000 7fa3837fe640 1 -- 172.21.15.116:0/2194999890 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa37c040070 con 0x7fa38c1080c0 2024-09-12T02:54:04.883 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.443+0000 7fa390e7f640 1 -- 172.21.15.116:0/2194999890 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa38c106070 con 0x7fa38c1080c0 2024-09-12T02:54:04.884 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.443+0000 7fa3837fe640 1 -- 172.21.15.116:0/2194999890 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa37c03a7b0 con 0x7fa38c1080c0 2024-09-12T02:54:04.884 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.443+0000 7fa390e7f640 1 -- 172.21.15.116:0/2194999890 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa38c1065b0 con 0x7fa38c1080c0 2024-09-12T02:54:04.884 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.444+0000 7fa3837fe640 1 -- 172.21.15.116:0/2194999890 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fa37c03aa30 con 0x7fa38c1080c0 2024-09-12T02:54:04.884 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.444+0000 7fa3837fe640 1 --2- 172.21.15.116:0/2194999890 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fa378038fc0 0x7fa37803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:04.884 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.445+0000 7fa390e7f640 1 -- 172.21.15.116:0/2194999890 --> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa38c108520 con 0x7fa378038fc0 2024-09-12T02:54:04.884 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.445+0000 7fa3837fe640 1 -- 172.21.15.116:0/2194999890 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa37c06c6a0 con 0x7fa38c1080c0 2024-09-12T02:54:04.884 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.445+0000 7fa389d9b640 1 -- 172.21.15.116:0/2194999890 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fa378038fc0 msgr2=0x7fa37803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.116:6800/2966274423 2024-09-12T02:54:04.884 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.445+0000 7fa389d9b640 1 --2- 172.21.15.116:0/2194999890 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fa378038fc0 0x7fa37803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:54:04.884 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.646+0000 7fa389d9b640 1 -- 172.21.15.116:0/2194999890 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fa378038fc0 msgr2=0x7fa37803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.116:6800/2966274423 2024-09-12T02:54:04.884 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:00.646+0000 7fa389d9b640 1 --2- 172.21.15.116:0/2194999890 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fa378038fc0 0x7fa37803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T02:54:04.885 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:01.047+0000 7fa389d9b640 1 -- 172.21.15.116:0/2194999890 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fa378038fc0 msgr2=0x7fa37803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.116:6800/2966274423 2024-09-12T02:54:04.885 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:01.047+0000 7fa389d9b640 1 --2- 172.21.15.116:0/2194999890 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fa378038fc0 0x7fa37803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-12T02:54:04.885 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:01.848+0000 7fa389d9b640 1 -- 172.21.15.116:0/2194999890 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fa378038fc0 msgr2=0x7fa37803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.116:6800/2966274423 2024-09-12T02:54:04.885 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:01.848+0000 7fa389d9b640 1 --2- 172.21.15.116:0/2194999890 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fa378038fc0 0x7fa37803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-12T02:54:04.885 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:03.028+0000 7fa3837fe640 1 -- 172.21.15.116:0/2194999890 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7fa37c06b150 con 0x7fa38c1080c0 2024-09-12T02:54:04.885 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:03.028+0000 7fa3837fe640 1 -- 172.21.15.116:0/2194999890 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fa378038fc0 msgr2=0x7fa37803b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:54:04.885 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:03.028+0000 7fa3837fe640 1 --2- 172.21.15.116:0/2194999890 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fa378038fc0 0x7fa37803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:04.885 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:04.029+0000 7fa3837fe640 1 -- 172.21.15.116:0/2194999890 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fa37c045980 con 0x7fa38c1080c0 2024-09-12T02:54:04.885 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:04.029+0000 7fa3837fe640 1 --2- 172.21.15.116:0/2194999890 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa37803c890 0x7fa37803ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:04.885 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:04.029+0000 7fa3837fe640 1 -- 172.21.15.116:0/2194999890 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa38c108520 con 0x7fa37803c890 2024-09-12T02:54:04.885 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:04.030+0000 7fa389d9b640 1 --2- 172.21.15.116:0/2194999890 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa37803c890 0x7fa37803ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:04.885 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:04.031+0000 7fa389d9b640 1 --2- 172.21.15.116:0/2194999890 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa37803c890 0x7fa37803ec80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa374003980 tx=0x7fa374006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:04.886 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:04.031+0000 7fa3837fe640 1 -- 172.21.15.116:0/2194999890 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fa38c108520 con 0x7fa37803c890 2024-09-12T02:54:04.886 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:04.038+0000 7fa390e7f640 1 -- 172.21.15.116:0/2194999890 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fa38c108520 con 0x7fa37803c890 2024-09-12T02:54:04.886 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:04.038+0000 7fa3837fe640 1 -- 172.21.15.116:0/2194999890 <== mgr.14150 v2:172.21.15.116:6800/2896038008 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fa38c108520 con 0x7fa37803c890 2024-09-12T02:54:04.886 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:04.039+0000 7fa390e7f640 1 -- 172.21.15.116:0/2194999890 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa37803c890 msgr2=0x7fa37803ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:04.886 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:04.039+0000 7fa390e7f640 1 --2- 172.21.15.116:0/2194999890 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa37803c890 0x7fa37803ec80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa374003980 tx=0x7fa374006e10 comp rx=0 tx=0).stop 2024-09-12T02:54:04.886 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:04.039+0000 7fa390e7f640 1 -- 172.21.15.116:0/2194999890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa38c1080c0 msgr2=0x7fa38c107940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:04.886 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:04.039+0000 7fa390e7f640 1 --2- 172.21.15.116:0/2194999890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa38c1080c0 0x7fa38c107940 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa37c030380 tx=0x7fa37c032670 comp rx=0 tx=0).stop 2024-09-12T02:54:04.886 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:04.039+0000 7fa390e7f640 1 -- 172.21.15.116:0/2194999890 shutdown_connections 2024-09-12T02:54:04.886 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:04.039+0000 7fa390e7f640 1 --2- 172.21.15.116:0/2194999890 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa37803c890 0x7fa37803ec80 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:04.886 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:04.039+0000 7fa390e7f640 1 --2- 172.21.15.116:0/2194999890 >> [v2:172.21.15.116:6800/2966274423,v1:172.21.15.116:6801/2966274423] conn(0x7fa378038fc0 0x7fa37803b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:04.886 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:04.039+0000 7fa390e7f640 1 --2- 172.21.15.116:0/2194999890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa38c1080c0 0x7fa38c107940 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:04.886 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:04.039+0000 7fa390e7f640 1 -- 172.21.15.116:0/2194999890 >> 172.21.15.116:0/2194999890 conn(0x7fa38c101ca0 msgr2=0x7fa38c1028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:04.886 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:04.039+0000 7fa390e7f640 1 -- 172.21.15.116:0/2194999890 shutdown_connections 2024-09-12T02:54:04.887 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:04.039+0000 7fa390e7f640 1 -- 172.21.15.116:0/2194999890 wait complete. 2024-09-12T02:54:04.887 INFO:teuthology.orchestra.run.smithi116.stdout:mgr epoch 9 is available 2024-09-12T02:54:04.887 INFO:teuthology.orchestra.run.smithi116.stdout:Generating a dashboard self-signed certificate... 2024-09-12T02:54:05.205 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:05 smithi116 ceph-mon[31410]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-12T02:54:05.205 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:05 smithi116 ceph-mon[31410]: mgrmap e11: x(active, since 1.00627s) 2024-09-12T02:54:05.205 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:05 smithi116 ceph-mon[31410]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-12T02:54:05.206 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:05 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:06.142 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-12T02:54:06.142 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.283+0000 7fb7b5897640 1 Processor -- start 2024-09-12T02:54:06.142 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.283+0000 7fb7b5897640 1 -- start start 2024-09-12T02:54:06.142 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.283+0000 7fb7b5897640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7b0106590 0x7fb7b0106990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:06.142 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.283+0000 7fb7b5897640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b0106f60 con 0x7fb7b0106590 2024-09-12T02:54:06.142 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.284+0000 7fb7aeffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7b0106590 0x7fb7b0106990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:06.142 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.284+0000 7fb7aeffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7b0106590 0x7fb7b0106990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51102/0 (socket says 172.21.15.116:51102) 2024-09-12T02:54:06.142 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.284+0000 7fb7aeffd640 1 -- 172.21.15.116:0/2206315097 learned_addr learned my addr 172.21.15.116:0/2206315097 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:54:06.142 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.284+0000 7fb7aeffd640 1 -- 172.21.15.116:0/2206315097 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b0107790 con 0x7fb7b0106590 2024-09-12T02:54:06.142 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.285+0000 7fb7aeffd640 1 --2- 172.21.15.116:0/2206315097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7b0106590 0x7fb7b0106990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0009820 tx=0x7fb7a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6978357a001e97b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:06.142 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.285+0000 7fb7ae7fc640 1 -- 172.21.15.116:0/2206315097 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7a0037070 con 0x7fb7b0106590 2024-09-12T02:54:06.142 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.285+0000 7fb7ae7fc640 1 -- 172.21.15.116:0/2206315097 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7a0032920 con 0x7fb7b0106590 2024-09-12T02:54:06.143 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.285+0000 7fb7ae7fc640 1 -- 172.21.15.116:0/2206315097 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7a0030810 con 0x7fb7b0106590 2024-09-12T02:54:06.143 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.286+0000 7fb7b5897640 1 -- 172.21.15.116:0/2206315097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7b0106590 msgr2=0x7fb7b0106990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:06.143 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.286+0000 7fb7b5897640 1 --2- 172.21.15.116:0/2206315097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7b0106590 0x7fb7b0106990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0009820 tx=0x7fb7a002a270 comp rx=0 tx=0).stop 2024-09-12T02:54:06.143 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.286+0000 7fb7b5897640 1 -- 172.21.15.116:0/2206315097 shutdown_connections 2024-09-12T02:54:06.143 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.286+0000 7fb7b5897640 1 --2- 172.21.15.116:0/2206315097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7b0106590 0x7fb7b0106990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:06.143 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.286+0000 7fb7b5897640 1 -- 172.21.15.116:0/2206315097 >> 172.21.15.116:0/2206315097 conn(0x7fb7b0101de0 msgr2=0x7fb7b0104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:06.143 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.286+0000 7fb7b5897640 1 -- 172.21.15.116:0/2206315097 shutdown_connections 2024-09-12T02:54:06.143 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.286+0000 7fb7b5897640 1 -- 172.21.15.116:0/2206315097 wait complete. 2024-09-12T02:54:06.143 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.286+0000 7fb7b5897640 1 Processor -- start 2024-09-12T02:54:06.143 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.286+0000 7fb7b5897640 1 -- start start 2024-09-12T02:54:06.143 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.287+0000 7fb7b5897640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7b0106590 0x7fb7b019b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:06.143 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.287+0000 7fb7b5897640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b010ab40 con 0x7fb7b0106590 2024-09-12T02:54:06.143 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.287+0000 7fb7aeffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7b0106590 0x7fb7b019b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:06.144 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.287+0000 7fb7aeffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7b0106590 0x7fb7b019b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51104/0 (socket says 172.21.15.116:51104) 2024-09-12T02:54:06.144 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.287+0000 7fb7aeffd640 1 -- 172.21.15.116:0/999943603 learned_addr learned my addr 172.21.15.116:0/999943603 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:54:06.144 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.287+0000 7fb7aeffd640 1 -- 172.21.15.116:0/999943603 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b019bba0 con 0x7fb7b0106590 2024-09-12T02:54:06.144 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.287+0000 7fb7aeffd640 1 --2- 172.21.15.116:0/999943603 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7b0106590 0x7fb7b019b660 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0030560 tx=0x7fb7a0032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:06.144 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.287+0000 7fb7b4895640 1 -- 172.21.15.116:0/999943603 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7a0037040 con 0x7fb7b0106590 2024-09-12T02:54:06.144 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.287+0000 7fb7b5897640 1 -- 172.21.15.116:0/999943603 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7b019be30 con 0x7fb7b0106590 2024-09-12T02:54:06.144 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.287+0000 7fb7b4895640 1 -- 172.21.15.116:0/999943603 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7a0040070 con 0x7fb7b0106590 2024-09-12T02:54:06.144 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.287+0000 7fb7b4895640 1 -- 172.21.15.116:0/999943603 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7a003a740 con 0x7fb7b0106590 2024-09-12T02:54:06.144 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.287+0000 7fb7b5897640 1 -- 172.21.15.116:0/999943603 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7b019eb20 con 0x7fb7b0106590 2024-09-12T02:54:06.144 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.288+0000 7fb7b4895640 1 -- 172.21.15.116:0/999943603 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fb7a003a9b0 con 0x7fb7b0106590 2024-09-12T02:54:06.144 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.288+0000 7fb7b5897640 1 -- 172.21.15.116:0/999943603 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7b0106a10 con 0x7fb7b0106590 2024-09-12T02:54:06.144 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.288+0000 7fb7b4895640 1 --2- 172.21.15.116:0/999943603 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb79c038eb0 0x7fb79c03b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:06.144 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.288+0000 7fb7b4895640 1 -- 172.21.15.116:0/999943603 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb7a006c660 con 0x7fb7b0106590 2024-09-12T02:54:06.145 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.288+0000 7fb7a7fff640 1 --2- 172.21.15.116:0/999943603 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb79c038eb0 0x7fb79c03b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:06.145 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.289+0000 7fb7a7fff640 1 --2- 172.21.15.116:0/999943603 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb79c038eb0 0x7fb79c03b370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb7980098c0 tx=0x7fb798006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:06.145 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.293+0000 7fb7b4895640 1 -- 172.21.15.116:0/999943603 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb7a003adf0 con 0x7fb7b0106590 2024-09-12T02:54:06.145 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.433+0000 7fb7b4895640 1 -- 172.21.15.116:0/999943603 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fb7a0045980 con 0x7fb7b0106590 2024-09-12T02:54:06.145 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.484+0000 7fb7b5897640 1 -- 172.21.15.116:0/999943603 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fb7b002da10 con 0x7fb79c038eb0 2024-09-12T02:54:06.145 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.568+0000 7fb7b4895640 1 -- 172.21.15.116:0/999943603 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fb7b002da10 con 0x7fb79c038eb0 2024-09-12T02:54:06.145 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.570+0000 7fb7b5897640 1 -- 172.21.15.116:0/999943603 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb79c038eb0 msgr2=0x7fb79c03b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:06.145 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.570+0000 7fb7b5897640 1 --2- 172.21.15.116:0/999943603 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb79c038eb0 0x7fb79c03b370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb7980098c0 tx=0x7fb798006db0 comp rx=0 tx=0).stop 2024-09-12T02:54:06.145 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.570+0000 7fb7b5897640 1 -- 172.21.15.116:0/999943603 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7b0106590 msgr2=0x7fb7b019b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:06.145 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.570+0000 7fb7b5897640 1 --2- 172.21.15.116:0/999943603 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7b0106590 0x7fb7b019b660 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0030560 tx=0x7fb7a0032670 comp rx=0 tx=0).stop 2024-09-12T02:54:06.145 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.570+0000 7fb7b5897640 1 -- 172.21.15.116:0/999943603 shutdown_connections 2024-09-12T02:54:06.145 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.570+0000 7fb7b5897640 1 --2- 172.21.15.116:0/999943603 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb79c038eb0 0x7fb79c03b370 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:06.145 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.570+0000 7fb7b5897640 1 --2- 172.21.15.116:0/999943603 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7b0106590 0x7fb7b019b660 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:06.146 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.570+0000 7fb7b5897640 1 -- 172.21.15.116:0/999943603 >> 172.21.15.116:0/999943603 conn(0x7fb7b0101de0 msgr2=0x7fb7b0102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:06.146 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.570+0000 7fb7b5897640 1 -- 172.21.15.116:0/999943603 shutdown_connections 2024-09-12T02:54:06.146 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:05.571+0000 7fb7b5897640 1 -- 172.21.15.116:0/999943603 wait complete. 2024-09-12T02:54:06.146 INFO:teuthology.orchestra.run.smithi116.stdout:Creating initial admin user... 2024-09-12T02:54:06.259 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:06 smithi116 ceph-mon[31410]: [12/Sep/2024:02:54:04] ENGINE Bus STARTING 2024-09-12T02:54:06.259 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:06 smithi116 ceph-mon[31410]: [12/Sep/2024:02:54:04] ENGINE Serving on https://172.21.15.116:7150 2024-09-12T02:54:06.260 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:06 smithi116 ceph-mon[31410]: [12/Sep/2024:02:54:04] ENGINE Error in HTTPServer.serve 2024-09-12T02:54:06.260 INFO:journalctl@ceph.mon.a.smithi116.stdout: Traceback (most recent call last): 2024-09-12T02:54:06.260 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T02:54:06.260 INFO:journalctl@ceph.mon.a.smithi116.stdout: self._connections.run(self.expiration_interval) 2024-09-12T02:54:06.260 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T02:54:06.260 INFO:journalctl@ceph.mon.a.smithi116.stdout: self._run(expiration_interval) 2024-09-12T02:54:06.260 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T02:54:06.260 INFO:journalctl@ceph.mon.a.smithi116.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T02:54:06.260 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T02:54:06.260 INFO:journalctl@ceph.mon.a.smithi116.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T02:54:06.260 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T02:54:06.260 INFO:journalctl@ceph.mon.a.smithi116.stdout: s = self.context.wrap_socket( 2024-09-12T02:54:06.260 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T02:54:06.261 INFO:journalctl@ceph.mon.a.smithi116.stdout: return self.sslsocket_class._create( 2024-09-12T02:54:06.261 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T02:54:06.261 INFO:journalctl@ceph.mon.a.smithi116.stdout: self.do_handshake() 2024-09-12T02:54:06.261 INFO:journalctl@ceph.mon.a.smithi116.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T02:54:06.261 INFO:journalctl@ceph.mon.a.smithi116.stdout: self._sslobj.do_handshake() 2024-09-12T02:54:06.261 INFO:journalctl@ceph.mon.a.smithi116.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T02:54:06.261 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:06 smithi116 ceph-mon[31410]: [12/Sep/2024:02:54:04] ENGINE Serving on http://172.21.15.116:8765 2024-09-12T02:54:06.261 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:06 smithi116 ceph-mon[31410]: [12/Sep/2024:02:54:04] ENGINE Bus STARTED 2024-09-12T02:54:06.261 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:06 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:06.261 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:06 smithi116 ceph-mon[31410]: mgrmap e12: x(active, since 2s) 2024-09-12T02:54:06.261 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:06 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:06.261 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:06 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:07.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:07 smithi116 ceph-mon[31410]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:54:07.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:07 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:07.634 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$ATxO8ezD5ax5j3Axu0CcBuuP36FalB9i4HHghtWw3NEYrf7FmRx6m", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726109646, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-12T02:54:07.634 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.559+0000 7feb0a937640 1 Processor -- start 2024-09-12T02:54:07.634 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.559+0000 7feb0a937640 1 -- start start 2024-09-12T02:54:07.634 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.559+0000 7feb0a937640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb04107680 0x7feb04107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:07.634 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.559+0000 7feb0a937640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7feb04107fc0 con 0x7feb04107680 2024-09-12T02:54:07.634 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.559+0000 7feb03fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb04107680 0x7feb04107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:07.634 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.559+0000 7feb03fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb04107680 0x7feb04107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51116/0 (socket says 172.21.15.116:51116) 2024-09-12T02:54:07.634 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.559+0000 7feb03fff640 1 -- 172.21.15.116:0/2653246841 learned_addr learned my addr 172.21.15.116:0/2653246841 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:54:07.635 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.559+0000 7feb03fff640 1 -- 172.21.15.116:0/2653246841 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb04108140 con 0x7feb04107680 2024-09-12T02:54:07.635 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.560+0000 7feb03fff640 1 --2- 172.21.15.116:0/2653246841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb04107680 0x7feb04107a80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feaf4009820 tx=0x7feaf402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36315355e7e7d3bf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:07.635 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.560+0000 7feb02ffd640 1 -- 172.21.15.116:0/2653246841 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feaf4037070 con 0x7feb04107680 2024-09-12T02:54:07.635 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.560+0000 7feb02ffd640 1 -- 172.21.15.116:0/2653246841 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaf4032920 con 0x7feb04107680 2024-09-12T02:54:07.635 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.560+0000 7feb02ffd640 1 -- 172.21.15.116:0/2653246841 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feaf4030810 con 0x7feb04107680 2024-09-12T02:54:07.635 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.561+0000 7feb0a937640 1 -- 172.21.15.116:0/2653246841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb04107680 msgr2=0x7feb04107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:07.635 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.561+0000 7feb0a937640 1 --2- 172.21.15.116:0/2653246841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb04107680 0x7feb04107a80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feaf4009820 tx=0x7feaf402a270 comp rx=0 tx=0).stop 2024-09-12T02:54:07.635 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.561+0000 7feb0a937640 1 -- 172.21.15.116:0/2653246841 shutdown_connections 2024-09-12T02:54:07.635 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.561+0000 7feb0a937640 1 --2- 172.21.15.116:0/2653246841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb04107680 0x7feb04107a80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:07.635 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.561+0000 7feb0a937640 1 -- 172.21.15.116:0/2653246841 >> 172.21.15.116:0/2653246841 conn(0x7feb04102eb0 msgr2=0x7feb041052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:07.635 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.561+0000 7feb0a937640 1 -- 172.21.15.116:0/2653246841 shutdown_connections 2024-09-12T02:54:07.635 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.561+0000 7feb0a937640 1 -- 172.21.15.116:0/2653246841 wait complete. 2024-09-12T02:54:07.635 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.561+0000 7feb0a937640 1 Processor -- start 2024-09-12T02:54:07.636 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.562+0000 7feb0a937640 1 -- start start 2024-09-12T02:54:07.636 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.562+0000 7feb0a937640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb04107680 0x7feb0419b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:07.636 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.562+0000 7feb0a937640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7feb0410baf0 con 0x7feb04107680 2024-09-12T02:54:07.636 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.562+0000 7feb03fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb04107680 0x7feb0419b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:07.636 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.562+0000 7feb03fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb04107680 0x7feb0419b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51130/0 (socket says 172.21.15.116:51130) 2024-09-12T02:54:07.636 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.562+0000 7feb03fff640 1 -- 172.21.15.116:0/1862988968 learned_addr learned my addr 172.21.15.116:0/1862988968 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:54:07.636 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.562+0000 7feb03fff640 1 -- 172.21.15.116:0/1862988968 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0419bbe0 con 0x7feb04107680 2024-09-12T02:54:07.636 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.562+0000 7feb03fff640 1 --2- 172.21.15.116:0/1862988968 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb04107680 0x7feb0419b6a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7feaf4030420 tx=0x7feaf4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:07.636 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.563+0000 7feb017fa640 1 -- 172.21.15.116:0/1862988968 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feaf4037040 con 0x7feb04107680 2024-09-12T02:54:07.636 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.563+0000 7feb0a937640 1 -- 172.21.15.116:0/1862988968 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb0419be70 con 0x7feb04107680 2024-09-12T02:54:07.636 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.563+0000 7feb017fa640 1 -- 172.21.15.116:0/1862988968 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaf4040070 con 0x7feb04107680 2024-09-12T02:54:07.636 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.563+0000 7feb017fa640 1 -- 172.21.15.116:0/1862988968 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feaf403a6e0 con 0x7feb04107680 2024-09-12T02:54:07.636 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.563+0000 7feb0a937640 1 -- 172.21.15.116:0/1862988968 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb0419eb60 con 0x7feb04107680 2024-09-12T02:54:07.637 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.563+0000 7feb017fa640 1 -- 172.21.15.116:0/1862988968 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7feaf403a9e0 con 0x7feb04107680 2024-09-12T02:54:07.637 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.564+0000 7feb017fa640 1 --2- 172.21.15.116:0/1862988968 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7feaec038fc0 0x7feaec03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:07.637 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.564+0000 7feb017fa640 1 -- 172.21.15.116:0/1862988968 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7feaf406c700 con 0x7feb04107680 2024-09-12T02:54:07.637 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.564+0000 7feb0a937640 1 -- 172.21.15.116:0/1862988968 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb04107b00 con 0x7feb04107680 2024-09-12T02:54:07.637 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.564+0000 7feb037fe640 1 --2- 172.21.15.116:0/1862988968 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7feaec038fc0 0x7feaec03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:07.637 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.564+0000 7feb037fe640 1 --2- 172.21.15.116:0/1862988968 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7feaec038fc0 0x7feaec03b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7feaf80098c0 tx=0x7feaf8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:07.637 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.568+0000 7feb017fa640 1 -- 172.21.15.116:0/1862988968 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feaf4041270 con 0x7feb04107680 2024-09-12T02:54:07.637 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.755+0000 7feb0a937640 1 -- 172.21.15.116:0/1862988968 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7feb0419ed30 con 0x7feaec038fc0 2024-09-12T02:54:07.637 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.981+0000 7feb017fa640 1 -- 172.21.15.116:0/1862988968 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7feb0419ed30 con 0x7feaec038fc0 2024-09-12T02:54:07.637 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.983+0000 7feb0a937640 1 -- 172.21.15.116:0/1862988968 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7feaec038fc0 msgr2=0x7feaec03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:07.637 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.983+0000 7feb0a937640 1 --2- 172.21.15.116:0/1862988968 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7feaec038fc0 0x7feaec03b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7feaf80098c0 tx=0x7feaf8006db0 comp rx=0 tx=0).stop 2024-09-12T02:54:07.637 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.983+0000 7feb0a937640 1 -- 172.21.15.116:0/1862988968 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb04107680 msgr2=0x7feb0419b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:07.637 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.983+0000 7feb0a937640 1 --2- 172.21.15.116:0/1862988968 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb04107680 0x7feb0419b6a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7feaf4030420 tx=0x7feaf4032670 comp rx=0 tx=0).stop 2024-09-12T02:54:07.638 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.983+0000 7feb0a937640 1 -- 172.21.15.116:0/1862988968 shutdown_connections 2024-09-12T02:54:07.638 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.983+0000 7feb0a937640 1 --2- 172.21.15.116:0/1862988968 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7feaec038fc0 0x7feaec03b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:07.638 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.983+0000 7feb0a937640 1 --2- 172.21.15.116:0/1862988968 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb04107680 0x7feb0419b6a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:07.638 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.983+0000 7feb0a937640 1 -- 172.21.15.116:0/1862988968 >> 172.21.15.116:0/1862988968 conn(0x7feb04102eb0 msgr2=0x7feb04103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:07.638 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.983+0000 7feb0a937640 1 -- 172.21.15.116:0/1862988968 shutdown_connections 2024-09-12T02:54:07.638 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:06.983+0000 7feb0a937640 1 -- 172.21.15.116:0/1862988968 wait complete. 2024-09-12T02:54:07.638 INFO:teuthology.orchestra.run.smithi116.stdout:Fetching dashboard port number... 2024-09-12T02:54:08.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:08 smithi116 ceph-mon[31410]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:54:09.271 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:09 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2480474970' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-12T02:54:09.536 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stdout 8443 2024-09-12T02:54:09.536 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.802+0000 7f6df99c4640 1 Processor -- start 2024-09-12T02:54:09.536 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.802+0000 7f6df99c4640 1 -- start start 2024-09-12T02:54:09.537 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.802+0000 7f6df99c4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6df4107680 0x7f6df4107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:09.537 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.802+0000 7f6df99c4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6df4107fc0 con 0x7f6df4107680 2024-09-12T02:54:09.537 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.802+0000 7f6df2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6df4107680 0x7f6df4107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:09.537 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.802+0000 7f6df2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6df4107680 0x7f6df4107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51142/0 (socket says 172.21.15.116:51142) 2024-09-12T02:54:09.537 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.802+0000 7f6df2ffd640 1 -- 172.21.15.116:0/3389749117 learned_addr learned my addr 172.21.15.116:0/3389749117 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:54:09.537 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.803+0000 7f6df2ffd640 1 -- 172.21.15.116:0/3389749117 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df4108140 con 0x7f6df4107680 2024-09-12T02:54:09.537 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.803+0000 7f6df2ffd640 1 --2- 172.21.15.116:0/3389749117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6df4107680 0x7f6df4107a80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009820 tx=0x7f6de402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d526ef9889f1688 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:09.537 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.803+0000 7f6df1ffb640 1 -- 172.21.15.116:0/3389749117 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6de4037070 con 0x7f6df4107680 2024-09-12T02:54:09.537 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.804+0000 7f6df1ffb640 1 -- 172.21.15.116:0/3389749117 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de4032920 con 0x7f6df4107680 2024-09-12T02:54:09.537 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.804+0000 7f6df1ffb640 1 -- 172.21.15.116:0/3389749117 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6de4030810 con 0x7f6df4107680 2024-09-12T02:54:09.537 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.804+0000 7f6df99c4640 1 -- 172.21.15.116:0/3389749117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6df4107680 msgr2=0x7f6df4107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:09.538 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.804+0000 7f6df99c4640 1 --2- 172.21.15.116:0/3389749117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6df4107680 0x7f6df4107a80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009820 tx=0x7f6de402a270 comp rx=0 tx=0).stop 2024-09-12T02:54:09.538 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.804+0000 7f6df99c4640 1 -- 172.21.15.116:0/3389749117 shutdown_connections 2024-09-12T02:54:09.538 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.804+0000 7f6df99c4640 1 --2- 172.21.15.116:0/3389749117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6df4107680 0x7f6df4107a80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.538 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.804+0000 7f6df99c4640 1 -- 172.21.15.116:0/3389749117 >> 172.21.15.116:0/3389749117 conn(0x7f6df4102eb0 msgr2=0x7f6df41052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:09.538 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.804+0000 7f6df99c4640 1 -- 172.21.15.116:0/3389749117 shutdown_connections 2024-09-12T02:54:09.538 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.804+0000 7f6df99c4640 1 -- 172.21.15.116:0/3389749117 wait complete. 2024-09-12T02:54:09.538 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.805+0000 7f6df99c4640 1 Processor -- start 2024-09-12T02:54:09.538 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.805+0000 7f6df99c4640 1 -- start start 2024-09-12T02:54:09.538 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.805+0000 7f6df99c4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6df4107680 0x7f6df419b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:09.538 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.805+0000 7f6df99c4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6df410baf0 con 0x7f6df4107680 2024-09-12T02:54:09.538 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.805+0000 7f6df2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6df4107680 0x7f6df419b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:09.538 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.805+0000 7f6df2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6df4107680 0x7f6df419b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51148/0 (socket says 172.21.15.116:51148) 2024-09-12T02:54:09.539 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.805+0000 7f6df2ffd640 1 -- 172.21.15.116:0/2480474970 learned_addr learned my addr 172.21.15.116:0/2480474970 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:54:09.539 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.805+0000 7f6df2ffd640 1 -- 172.21.15.116:0/2480474970 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df419bbe0 con 0x7f6df4107680 2024-09-12T02:54:09.539 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.805+0000 7f6df2ffd640 1 --2- 172.21.15.116:0/2480474970 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6df4107680 0x7f6df419b6a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6de4030420 tx=0x7f6de4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:09.539 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.806+0000 7f6df89c2640 1 -- 172.21.15.116:0/2480474970 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6de4037040 con 0x7f6df4107680 2024-09-12T02:54:09.539 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.806+0000 7f6df89c2640 1 -- 172.21.15.116:0/2480474970 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de4040070 con 0x7f6df4107680 2024-09-12T02:54:09.539 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.806+0000 7f6df99c4640 1 -- 172.21.15.116:0/2480474970 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df419be70 con 0x7f6df4107680 2024-09-12T02:54:09.539 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.806+0000 7f6df89c2640 1 -- 172.21.15.116:0/2480474970 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6de403a6e0 con 0x7f6df4107680 2024-09-12T02:54:09.539 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.806+0000 7f6df99c4640 1 -- 172.21.15.116:0/2480474970 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df419eb60 con 0x7f6df4107680 2024-09-12T02:54:09.539 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.806+0000 7f6df89c2640 1 -- 172.21.15.116:0/2480474970 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f6de403a9e0 con 0x7f6df4107680 2024-09-12T02:54:09.539 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.807+0000 7f6df99c4640 1 -- 172.21.15.116:0/2480474970 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6df4107b00 con 0x7f6df4107680 2024-09-12T02:54:09.539 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.807+0000 7f6df89c2640 1 --2- 172.21.15.116:0/2480474970 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6ddc038fc0 0x7f6ddc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:09.539 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.807+0000 7f6df89c2640 1 -- 172.21.15.116:0/2480474970 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6de406c730 con 0x7f6df4107680 2024-09-12T02:54:09.540 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.807+0000 7f6df27fc640 1 --2- 172.21.15.116:0/2480474970 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6ddc038fc0 0x7f6ddc03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:09.540 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.807+0000 7f6df27fc640 1 --2- 172.21.15.116:0/2480474970 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6ddc038fc0 0x7f6ddc03b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6de80098c0 tx=0x7f6de8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:09.540 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.812+0000 7f6df89c2640 1 -- 172.21.15.116:0/2480474970 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6de4041270 con 0x7f6df4107680 2024-09-12T02:54:09.540 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.977+0000 7f6df99c4640 1 -- 172.21.15.116:0/2480474970 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f6df419f150 con 0x7f6df4107680 2024-09-12T02:54:09.540 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.978+0000 7f6df89c2640 1 -- 172.21.15.116:0/2480474970 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f6de4043bf0 con 0x7f6df4107680 2024-09-12T02:54:09.540 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.980+0000 7f6df99c4640 1 -- 172.21.15.116:0/2480474970 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6ddc038fc0 msgr2=0x7f6ddc03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:09.540 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.980+0000 7f6df99c4640 1 --2- 172.21.15.116:0/2480474970 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6ddc038fc0 0x7f6ddc03b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6de80098c0 tx=0x7f6de8006db0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.540 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.980+0000 7f6df99c4640 1 -- 172.21.15.116:0/2480474970 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6df4107680 msgr2=0x7f6df419b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:09.540 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.980+0000 7f6df99c4640 1 --2- 172.21.15.116:0/2480474970 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6df4107680 0x7f6df419b6a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6de4030420 tx=0x7f6de4032670 comp rx=0 tx=0).stop 2024-09-12T02:54:09.540 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.980+0000 7f6df99c4640 1 -- 172.21.15.116:0/2480474970 shutdown_connections 2024-09-12T02:54:09.540 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.980+0000 7f6df99c4640 1 --2- 172.21.15.116:0/2480474970 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6ddc038fc0 0x7f6ddc03b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.540 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.980+0000 7f6df99c4640 1 --2- 172.21.15.116:0/2480474970 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6df4107680 0x7f6df419b6a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:09.541 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.980+0000 7f6df99c4640 1 -- 172.21.15.116:0/2480474970 >> 172.21.15.116:0/2480474970 conn(0x7f6df4102eb0 msgr2=0x7f6df4103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:09.541 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.980+0000 7f6df99c4640 1 -- 172.21.15.116:0/2480474970 shutdown_connections 2024-09-12T02:54:09.541 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:08.980+0000 7f6df99c4640 1 -- 172.21.15.116:0/2480474970 wait complete. 2024-09-12T02:54:09.544 INFO:teuthology.orchestra.run.smithi116.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T02:54:09.544 INFO:teuthology.orchestra.run.smithi116.stdout:systemctl: stdout disabled 2024-09-12T02:54:09.552 INFO:teuthology.orchestra.run.smithi116.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T02:54:09.552 INFO:teuthology.orchestra.run.smithi116.stdout:systemctl: stdout inactive 2024-09-12T02:54:09.552 INFO:teuthology.orchestra.run.smithi116.stdout:firewalld.service is not enabled 2024-09-12T02:54:09.552 INFO:teuthology.orchestra.run.smithi116.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-12T02:54:09.556 INFO:teuthology.orchestra.run.smithi116.stdout:Ceph Dashboard is now available at: 2024-09-12T02:54:09.556 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:54:09.557 INFO:teuthology.orchestra.run.smithi116.stdout: URL: https://smithi116.front.sepia.ceph.com:8443/ 2024-09-12T02:54:09.557 INFO:teuthology.orchestra.run.smithi116.stdout: User: admin 2024-09-12T02:54:09.557 INFO:teuthology.orchestra.run.smithi116.stdout: Password: dr3zavrtd5 2024-09-12T02:54:09.557 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:54:09.557 INFO:teuthology.orchestra.run.smithi116.stdout:Saving cluster configuration to /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/config directory 2024-09-12T02:54:10.255 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:10 smithi116 ceph-mon[31410]: mgrmap e13: x(active, since 6s) 2024-09-12T02:54:10.814 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.993+0000 7fec38e64640 1 Processor -- start 2024-09-12T02:54:10.815 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.993+0000 7fec38e64640 1 -- start start 2024-09-12T02:54:10.815 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.993+0000 7fec38e64640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec34106520 0x7fec34106920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:10.815 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.993+0000 7fec38e64640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fec34106ef0 con 0x7fec34106520 2024-09-12T02:54:10.815 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.994+0000 7fec3259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec34106520 0x7fec34106920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:10.815 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.994+0000 7fec3259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec34106520 0x7fec34106920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51152/0 (socket says 172.21.15.116:51152) 2024-09-12T02:54:10.815 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.994+0000 7fec3259c640 1 -- 172.21.15.116:0/1028908214 learned_addr learned my addr 172.21.15.116:0/1028908214 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:54:10.815 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.994+0000 7fec3259c640 1 -- 172.21.15.116:0/1028908214 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec34107720 con 0x7fec34106520 2024-09-12T02:54:10.815 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.995+0000 7fec3259c640 1 --2- 172.21.15.116:0/1028908214 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec34106520 0x7fec34106920 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fec24009820 tx=0x7fec2402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314317129233d8ad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:10.815 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.995+0000 7fec3159a640 1 -- 172.21.15.116:0/1028908214 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec24037070 con 0x7fec34106520 2024-09-12T02:54:10.820 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.995+0000 7fec3159a640 1 -- 172.21.15.116:0/1028908214 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec24032920 con 0x7fec34106520 2024-09-12T02:54:10.820 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.995+0000 7fec3159a640 1 -- 172.21.15.116:0/1028908214 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec24030810 con 0x7fec34106520 2024-09-12T02:54:10.820 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.995+0000 7fec38e64640 1 -- 172.21.15.116:0/1028908214 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec34106520 msgr2=0x7fec34106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:10.820 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.995+0000 7fec38e64640 1 --2- 172.21.15.116:0/1028908214 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec34106520 0x7fec34106920 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fec24009820 tx=0x7fec2402a270 comp rx=0 tx=0).stop 2024-09-12T02:54:10.820 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.995+0000 7fec38e64640 1 -- 172.21.15.116:0/1028908214 shutdown_connections 2024-09-12T02:54:10.820 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.995+0000 7fec38e64640 1 --2- 172.21.15.116:0/1028908214 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec34106520 0x7fec34106920 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:10.820 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.995+0000 7fec38e64640 1 -- 172.21.15.116:0/1028908214 >> 172.21.15.116:0/1028908214 conn(0x7fec34101db0 msgr2=0x7fec341041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:10.820 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.995+0000 7fec38e64640 1 -- 172.21.15.116:0/1028908214 shutdown_connections 2024-09-12T02:54:10.820 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.995+0000 7fec38e64640 1 -- 172.21.15.116:0/1028908214 wait complete. 2024-09-12T02:54:10.820 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.996+0000 7fec38e64640 1 Processor -- start 2024-09-12T02:54:10.820 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.996+0000 7fec38e64640 1 -- start start 2024-09-12T02:54:10.820 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.996+0000 7fec38e64640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec34106520 0x7fec3407ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:10.821 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.996+0000 7fec38e64640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fec3410aad0 con 0x7fec34106520 2024-09-12T02:54:10.821 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.997+0000 7fec3259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec34106520 0x7fec3407ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:10.821 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.997+0000 7fec3259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec34106520 0x7fec3407ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51168/0 (socket says 172.21.15.116:51168) 2024-09-12T02:54:10.821 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.997+0000 7fec3259c640 1 -- 172.21.15.116:0/508798270 learned_addr learned my addr 172.21.15.116:0/508798270 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:54:10.821 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.997+0000 7fec3259c640 1 -- 172.21.15.116:0/508798270 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3419e390 con 0x7fec34106520 2024-09-12T02:54:10.821 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.997+0000 7fec3259c640 1 --2- 172.21.15.116:0/508798270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec34106520 0x7fec3407ba50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fec24030560 tx=0x7fec24032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:10.821 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.997+0000 7fec237fe640 1 -- 172.21.15.116:0/508798270 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec24037040 con 0x7fec34106520 2024-09-12T02:54:10.821 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.997+0000 7fec237fe640 1 -- 172.21.15.116:0/508798270 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec24040070 con 0x7fec34106520 2024-09-12T02:54:10.821 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.997+0000 7fec38e64640 1 -- 172.21.15.116:0/508798270 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec3419e560 con 0x7fec34106520 2024-09-12T02:54:10.821 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.997+0000 7fec38e64640 1 -- 172.21.15.116:0/508798270 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec3419e900 con 0x7fec34106520 2024-09-12T02:54:10.821 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.997+0000 7fec237fe640 1 -- 172.21.15.116:0/508798270 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec2403a740 con 0x7fec34106520 2024-09-12T02:54:10.821 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.998+0000 7fec237fe640 1 -- 172.21.15.116:0/508798270 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fec2403a8e0 con 0x7fec34106520 2024-09-12T02:54:10.821 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.998+0000 7fec38e64640 1 -- 172.21.15.116:0/508798270 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec341069a0 con 0x7fec34106520 2024-09-12T02:54:10.822 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.998+0000 7fec237fe640 1 --2- 172.21.15.116:0/508798270 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fec1c0390c0 0x7fec1c03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:10.822 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.998+0000 7fec237fe640 1 -- 172.21.15.116:0/508798270 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fec2406c7b0 con 0x7fec34106520 2024-09-12T02:54:10.822 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.998+0000 7fec31d9b640 1 --2- 172.21.15.116:0/508798270 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fec1c0390c0 0x7fec1c03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:10.822 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:09.999+0000 7fec31d9b640 1 --2- 172.21.15.116:0/508798270 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fec1c0390c0 0x7fec1c03b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fec280098c0 tx=0x7fec28006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:10.822 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:10.003+0000 7fec237fe640 1 -- 172.21.15.116:0/508798270 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fec24039970 con 0x7fec34106520 2024-09-12T02:54:10.822 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:10.254+0000 7fec38e64640 1 -- 172.21.15.116:0/508798270 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fec3419ed40 con 0x7fec34106520 2024-09-12T02:54:10.822 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:10.257+0000 7fec237fe640 1 -- 172.21.15.116:0/508798270 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fec24041270 con 0x7fec34106520 2024-09-12T02:54:10.822 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-12T02:54:10.822 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:10.259+0000 7fec38e64640 1 -- 172.21.15.116:0/508798270 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fec1c0390c0 msgr2=0x7fec1c03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:10.822 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:10.259+0000 7fec38e64640 1 --2- 172.21.15.116:0/508798270 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fec1c0390c0 0x7fec1c03b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fec280098c0 tx=0x7fec28006db0 comp rx=0 tx=0).stop 2024-09-12T02:54:10.822 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:10.259+0000 7fec38e64640 1 -- 172.21.15.116:0/508798270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec34106520 msgr2=0x7fec3407ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:10.822 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:10.259+0000 7fec38e64640 1 --2- 172.21.15.116:0/508798270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec34106520 0x7fec3407ba50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fec24030560 tx=0x7fec24032670 comp rx=0 tx=0).stop 2024-09-12T02:54:10.822 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:10.259+0000 7fec38e64640 1 -- 172.21.15.116:0/508798270 shutdown_connections 2024-09-12T02:54:10.822 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:10.259+0000 7fec38e64640 1 --2- 172.21.15.116:0/508798270 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fec1c0390c0 0x7fec1c03b580 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:10.823 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:10.259+0000 7fec38e64640 1 --2- 172.21.15.116:0/508798270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec34106520 0x7fec3407ba50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:10.823 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:10.259+0000 7fec38e64640 1 -- 172.21.15.116:0/508798270 >> 172.21.15.116:0/508798270 conn(0x7fec34101db0 msgr2=0x7fec341028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:10.823 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:10.259+0000 7fec38e64640 1 -- 172.21.15.116:0/508798270 shutdown_connections 2024-09-12T02:54:10.823 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:10.259+0000 7fec38e64640 1 -- 172.21.15.116:0/508798270 wait complete. 2024-09-12T02:54:10.823 INFO:teuthology.orchestra.run.smithi116.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-12T02:54:10.823 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:54:10.823 INFO:teuthology.orchestra.run.smithi116.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:54:10.823 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:54:10.823 INFO:teuthology.orchestra.run.smithi116.stdout:Or, if you are only running a single cluster on this host: 2024-09-12T02:54:10.823 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:54:10.823 INFO:teuthology.orchestra.run.smithi116.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-12T02:54:10.823 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:54:10.823 INFO:teuthology.orchestra.run.smithi116.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-12T02:54:10.823 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:54:10.824 INFO:teuthology.orchestra.run.smithi116.stdout: ceph telemetry on 2024-09-12T02:54:10.824 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:54:10.824 INFO:teuthology.orchestra.run.smithi116.stdout:For more information see: 2024-09-12T02:54:10.824 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:54:10.824 INFO:teuthology.orchestra.run.smithi116.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-12T02:54:10.824 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:54:10.824 INFO:teuthology.orchestra.run.smithi116.stdout:Bootstrap complete. 2024-09-12T02:54:10.861 INFO:tasks.cephadm:Fetching config... 2024-09-12T02:54:10.862 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-12T02:54:10.862 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-12T02:54:10.880 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-12T02:54:10.880 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-12T02:54:10.880 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-12T02:54:10.936 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-12T02:54:10.937 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-12T02:54:10.937 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-12T02:54:11.002 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-12T02:54:11.002 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-12T02:54:11.002 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-12T02:54:11.058 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-12T02:54:11.058 DEBUG:teuthology.orchestra.run.smithi116:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0C7Up4lYIJ4YJgQv/JwnLjQSoSttVjejpz4yaXzwzNqNv3A9GdBpJmjBBzDuQ3mFVEymlLuj+zC5CkkVc8dhrze3rGwMSURlA3bQyORgwvttEVV1qeAqFVJG63j1M+dRYkiMiJmwB8m4A7NaO/ylIkd9IJogpibuRjTrve1inVR9fp66ctXcaKGCdrivMTxmF8JR/NsXRPOEOav75BD62v1ANy3ZyreURasi/UX6PPieD4PjFCfJJi81vZCl2cKMANQy+SVpO32Fxptuj04r7SMXOd+dWH3j7Mc+YuT4LpuftruDpqRSu8gqp5Ir1RdQuiz8EKJRvUr8XkmG9/aSQV9fVgotx0uF3cujVEuQcodLlxkKntVKgAju064r7EEXia50uQqAhHqsQaCpCh1bWkTF1zHxePtemCPcFfJD7veXHD/NmPYQ4J9ZAGy8/f6soq/I6iUoHNoZyAZv3QUpx9SfvRKbd4jRaZOswv/aWTj6Set26RnvB/iGYfcpXpQ8= ceph-eb1ac602-70b1-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T02:54:11.140 INFO:teuthology.orchestra.run.smithi116.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0C7Up4lYIJ4YJgQv/JwnLjQSoSttVjejpz4yaXzwzNqNv3A9GdBpJmjBBzDuQ3mFVEymlLuj+zC5CkkVc8dhrze3rGwMSURlA3bQyORgwvttEVV1qeAqFVJG63j1M+dRYkiMiJmwB8m4A7NaO/ylIkd9IJogpibuRjTrve1inVR9fp66ctXcaKGCdrivMTxmF8JR/NsXRPOEOav75BD62v1ANy3ZyreURasi/UX6PPieD4PjFCfJJi81vZCl2cKMANQy+SVpO32Fxptuj04r7SMXOd+dWH3j7Mc+YuT4LpuftruDpqRSu8gqp5Ir1RdQuiz8EKJRvUr8XkmG9/aSQV9fVgotx0uF3cujVEuQcodLlxkKntVKgAju064r7EEXia50uQqAhHqsQaCpCh1bWkTF1zHxePtemCPcFfJD7veXHD/NmPYQ4J9ZAGy8/f6soq/I6iUoHNoZyAZv3QUpx9SfvRKbd4jRaZOswv/aWTj6Set26RnvB/iGYfcpXpQ8= ceph-eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T02:54:11.153 DEBUG:teuthology.orchestra.run.smithi177:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0C7Up4lYIJ4YJgQv/JwnLjQSoSttVjejpz4yaXzwzNqNv3A9GdBpJmjBBzDuQ3mFVEymlLuj+zC5CkkVc8dhrze3rGwMSURlA3bQyORgwvttEVV1qeAqFVJG63j1M+dRYkiMiJmwB8m4A7NaO/ylIkd9IJogpibuRjTrve1inVR9fp66ctXcaKGCdrivMTxmF8JR/NsXRPOEOav75BD62v1ANy3ZyreURasi/UX6PPieD4PjFCfJJi81vZCl2cKMANQy+SVpO32Fxptuj04r7SMXOd+dWH3j7Mc+YuT4LpuftruDpqRSu8gqp5Ir1RdQuiz8EKJRvUr8XkmG9/aSQV9fVgotx0uF3cujVEuQcodLlxkKntVKgAju064r7EEXia50uQqAhHqsQaCpCh1bWkTF1zHxePtemCPcFfJD7veXHD/NmPYQ4J9ZAGy8/f6soq/I6iUoHNoZyAZv3QUpx9SfvRKbd4jRaZOswv/aWTj6Set26RnvB/iGYfcpXpQ8= ceph-eb1ac602-70b1-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T02:54:11.196 INFO:teuthology.orchestra.run.smithi177.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0C7Up4lYIJ4YJgQv/JwnLjQSoSttVjejpz4yaXzwzNqNv3A9GdBpJmjBBzDuQ3mFVEymlLuj+zC5CkkVc8dhrze3rGwMSURlA3bQyORgwvttEVV1qeAqFVJG63j1M+dRYkiMiJmwB8m4A7NaO/ylIkd9IJogpibuRjTrve1inVR9fp66ctXcaKGCdrivMTxmF8JR/NsXRPOEOav75BD62v1ANy3ZyreURasi/UX6PPieD4PjFCfJJi81vZCl2cKMANQy+SVpO32Fxptuj04r7SMXOd+dWH3j7Mc+YuT4LpuftruDpqRSu8gqp5Ir1RdQuiz8EKJRvUr8XkmG9/aSQV9fVgotx0uF3cujVEuQcodLlxkKntVKgAju064r7EEXia50uQqAhHqsQaCpCh1bWkTF1zHxePtemCPcFfJD7veXHD/NmPYQ4J9ZAGy8/f6soq/I6iUoHNoZyAZv3QUpx9SfvRKbd4jRaZOswv/aWTj6Set26RnvB/iGYfcpXpQ8= ceph-eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T02:54:11.207 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0C7Up4lYIJ4YJgQv/JwnLjQSoSttVjejpz4yaXzwzNqNv3A9GdBpJmjBBzDuQ3mFVEymlLuj+zC5CkkVc8dhrze3rGwMSURlA3bQyORgwvttEVV1qeAqFVJG63j1M+dRYkiMiJmwB8m4A7NaO/ylIkd9IJogpibuRjTrve1inVR9fp66ctXcaKGCdrivMTxmF8JR/NsXRPOEOav75BD62v1ANy3ZyreURasi/UX6PPieD4PjFCfJJi81vZCl2cKMANQy+SVpO32Fxptuj04r7SMXOd+dWH3j7Mc+YuT4LpuftruDpqRSu8gqp5Ir1RdQuiz8EKJRvUr8XkmG9/aSQV9fVgotx0uF3cujVEuQcodLlxkKntVKgAju064r7EEXia50uQqAhHqsQaCpCh1bWkTF1zHxePtemCPcFfJD7veXHD/NmPYQ4J9ZAGy8/f6soq/I6iUoHNoZyAZv3QUpx9SfvRKbd4jRaZOswv/aWTj6Set26RnvB/iGYfcpXpQ8= ceph-eb1ac602-70b1-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T02:54:11.248 INFO:teuthology.orchestra.run.smithi183.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0C7Up4lYIJ4YJgQv/JwnLjQSoSttVjejpz4yaXzwzNqNv3A9GdBpJmjBBzDuQ3mFVEymlLuj+zC5CkkVc8dhrze3rGwMSURlA3bQyORgwvttEVV1qeAqFVJG63j1M+dRYkiMiJmwB8m4A7NaO/ylIkd9IJogpibuRjTrve1inVR9fp66ctXcaKGCdrivMTxmF8JR/NsXRPOEOav75BD62v1ANy3ZyreURasi/UX6PPieD4PjFCfJJi81vZCl2cKMANQy+SVpO32Fxptuj04r7SMXOd+dWH3j7Mc+YuT4LpuftruDpqRSu8gqp5Ir1RdQuiz8EKJRvUr8XkmG9/aSQV9fVgotx0uF3cujVEuQcodLlxkKntVKgAju064r7EEXia50uQqAhHqsQaCpCh1bWkTF1zHxePtemCPcFfJD7veXHD/NmPYQ4J9ZAGy8/f6soq/I6iUoHNoZyAZv3QUpx9SfvRKbd4jRaZOswv/aWTj6Set26RnvB/iGYfcpXpQ8= ceph-eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T02:54:11.261 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-12T02:54:11.512 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:54:11.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:11 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/508798270' entity='client.admin' 2024-09-12T02:54:12.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.024+0000 7fe6bf68e640 1 -- 172.21.15.116:0/380732344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6b8100530 msgr2=0x7fe6b8100930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:12.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.024+0000 7fe6bf68e640 1 --2- 172.21.15.116:0/380732344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6b8100530 0x7fe6b8100930 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009880 tx=0x7fe6b402a5d0 comp rx=0 tx=0).stop 2024-09-12T02:54:12.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.024+0000 7fe6bf68e640 1 -- 172.21.15.116:0/380732344 shutdown_connections 2024-09-12T02:54:12.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.024+0000 7fe6bf68e640 1 --2- 172.21.15.116:0/380732344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6b8100530 0x7fe6b8100930 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:12.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.024+0000 7fe6bf68e640 1 -- 172.21.15.116:0/380732344 >> 172.21.15.116:0/380732344 conn(0x7fe6b80fbca0 msgr2=0x7fe6b80fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:12.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.024+0000 7fe6bf68e640 1 -- 172.21.15.116:0/380732344 shutdown_connections 2024-09-12T02:54:12.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.024+0000 7fe6bf68e640 1 -- 172.21.15.116:0/380732344 wait complete. 2024-09-12T02:54:12.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.025+0000 7fe6bf68e640 1 Processor -- start 2024-09-12T02:54:12.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.025+0000 7fe6bf68e640 1 -- start start 2024-09-12T02:54:12.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.025+0000 7fe6bf68e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6b8100530 0x7fe6b8196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:12.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.025+0000 7fe6bf68e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b8106640 con 0x7fe6b8100530 2024-09-12T02:54:12.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.025+0000 7fe6bd42a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6b8100530 0x7fe6b8196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:12.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.025+0000 7fe6bd42a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6b8100530 0x7fe6b8196e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33392/0 (socket says 172.21.15.116:33392) 2024-09-12T02:54:12.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.025+0000 7fe6bd42a640 1 -- 172.21.15.116:0/3559974547 learned_addr learned my addr 172.21.15.116:0/3559974547 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:54:12.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.025+0000 7fe6bd42a640 1 -- 172.21.15.116:0/3559974547 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b8197380 con 0x7fe6b8100530 2024-09-12T02:54:12.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.025+0000 7fe6bd42a640 1 --2- 172.21.15.116:0/3559974547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6b8100530 0x7fe6b8196e40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4004f60 tx=0x7fe6b40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:12.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.025+0000 7fe6ae7fc640 1 -- 172.21.15.116:0/3559974547 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6b403f070 con 0x7fe6b8100530 2024-09-12T02:54:12.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.026+0000 7fe6ae7fc640 1 -- 172.21.15.116:0/3559974547 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6b4004d40 con 0x7fe6b8100530 2024-09-12T02:54:12.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.026+0000 7fe6bf68e640 1 -- 172.21.15.116:0/3559974547 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6b8197610 con 0x7fe6b8100530 2024-09-12T02:54:12.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.026+0000 7fe6ae7fc640 1 -- 172.21.15.116:0/3559974547 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6b4035030 con 0x7fe6b8100530 2024-09-12T02:54:12.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.026+0000 7fe6bf68e640 1 -- 172.21.15.116:0/3559974547 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6b819a300 con 0x7fe6b8100530 2024-09-12T02:54:12.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.026+0000 7fe6ae7fc640 1 -- 172.21.15.116:0/3559974547 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fe6b4042070 con 0x7fe6b8100530 2024-09-12T02:54:12.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.027+0000 7fe6bf68e640 1 -- 172.21.15.116:0/3559974547 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6b8106dc0 con 0x7fe6b8100530 2024-09-12T02:54:12.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.027+0000 7fe6ae7fc640 1 --2- 172.21.15.116:0/3559974547 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe6a4039060 0x7fe6a403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:12.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.027+0000 7fe6ae7fc640 1 -- 172.21.15.116:0/3559974547 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe6b406cdb0 con 0x7fe6b8100530 2024-09-12T02:54:12.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.027+0000 7fe6bcc29640 1 --2- 172.21.15.116:0/3559974547 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe6a4039060 0x7fe6a403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:12.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.027+0000 7fe6bcc29640 1 --2- 172.21.15.116:0/3559974547 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe6a4039060 0x7fe6a403b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe6a80098c0 tx=0x7fe6a8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:12.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.031+0000 7fe6ae7fc640 1 -- 172.21.15.116:0/3559974547 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe6b403b550 con 0x7fe6b8100530 2024-09-12T02:54:12.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.200+0000 7fe6bf68e640 1 -- 172.21.15.116:0/3559974547 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fe6b802d7c0 con 0x7fe6b8100530 2024-09-12T02:54:12.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.203+0000 7fe6ae7fc640 1 -- 172.21.15.116:0/3559974547 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fe6b4031bf0 con 0x7fe6b8100530 2024-09-12T02:54:12.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.205+0000 7fe6bf68e640 1 -- 172.21.15.116:0/3559974547 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe6a4039060 msgr2=0x7fe6a403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:12.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.205+0000 7fe6bf68e640 1 --2- 172.21.15.116:0/3559974547 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe6a4039060 0x7fe6a403b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe6a80098c0 tx=0x7fe6a8006db0 comp rx=0 tx=0).stop 2024-09-12T02:54:12.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.205+0000 7fe6bf68e640 1 -- 172.21.15.116:0/3559974547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6b8100530 msgr2=0x7fe6b8196e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:12.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.205+0000 7fe6bf68e640 1 --2- 172.21.15.116:0/3559974547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6b8100530 0x7fe6b8196e40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4004f60 tx=0x7fe6b40048f0 comp rx=0 tx=0).stop 2024-09-12T02:54:12.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.205+0000 7fe6bf68e640 1 -- 172.21.15.116:0/3559974547 shutdown_connections 2024-09-12T02:54:12.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.205+0000 7fe6bf68e640 1 --2- 172.21.15.116:0/3559974547 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe6a4039060 0x7fe6a403b520 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:12.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.205+0000 7fe6bf68e640 1 --2- 172.21.15.116:0/3559974547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6b8100530 0x7fe6b8196e40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:12.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.205+0000 7fe6bf68e640 1 -- 172.21.15.116:0/3559974547 >> 172.21.15.116:0/3559974547 conn(0x7fe6b80fbca0 msgr2=0x7fe6b8069ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:12.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.206+0000 7fe6bf68e640 1 -- 172.21.15.116:0/3559974547 shutdown_connections 2024-09-12T02:54:12.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:12.206+0000 7fe6bf68e640 1 -- 172.21.15.116:0/3559974547 wait complete. 2024-09-12T02:54:12.841 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-12T02:54:12.841 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-12T02:54:13.335 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:54:13.467 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:13 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3559974547' entity='client.admin' 2024-09-12T02:54:14.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.250+0000 7f222e6af640 1 -- 172.21.15.116:0/68425543 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22281020c0 msgr2=0x7f22281044b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:14.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.250+0000 7f222e6af640 1 --2- 172.21.15.116:0/68425543 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22281020c0 0x7f22281044b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2218009880 tx=0x7f221802a5d0 comp rx=0 tx=0).stop 2024-09-12T02:54:14.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.250+0000 7f222e6af640 1 -- 172.21.15.116:0/68425543 shutdown_connections 2024-09-12T02:54:14.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.250+0000 7f222e6af640 1 --2- 172.21.15.116:0/68425543 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22281020c0 0x7f22281044b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:14.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.250+0000 7f222e6af640 1 -- 172.21.15.116:0/68425543 >> 172.21.15.116:0/68425543 conn(0x7f22280fbca0 msgr2=0x7f22280fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:14.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.250+0000 7f222e6af640 1 -- 172.21.15.116:0/68425543 shutdown_connections 2024-09-12T02:54:14.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.250+0000 7f222e6af640 1 -- 172.21.15.116:0/68425543 wait complete. 2024-09-12T02:54:14.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.251+0000 7f222e6af640 1 Processor -- start 2024-09-12T02:54:14.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.251+0000 7f222e6af640 1 -- start start 2024-09-12T02:54:14.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.251+0000 7f222e6af640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22281020c0 0x7f2228196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:14.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.251+0000 7f222e6af640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f22281076d0 con 0x7f22281020c0 2024-09-12T02:54:14.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.251+0000 7f2227fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22281020c0 0x7f2228196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:14.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.252+0000 7f2227fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22281020c0 0x7f2228196d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33418/0 (socket says 172.21.15.116:33418) 2024-09-12T02:54:14.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.252+0000 7f2227fff640 1 -- 172.21.15.116:0/4147334599 learned_addr learned my addr 172.21.15.116:0/4147334599 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:54:14.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.252+0000 7f2227fff640 1 -- 172.21.15.116:0/4147334599 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2228197290 con 0x7f22281020c0 2024-09-12T02:54:14.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.252+0000 7f2227fff640 1 --2- 172.21.15.116:0/4147334599 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22281020c0 0x7f2228196d50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2218004f60 tx=0x7f22180048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:14.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.252+0000 7f22257fa640 1 -- 172.21.15.116:0/4147334599 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f221803f070 con 0x7f22281020c0 2024-09-12T02:54:14.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.252+0000 7f22257fa640 1 -- 172.21.15.116:0/4147334599 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2218003ad0 con 0x7f22281020c0 2024-09-12T02:54:14.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.252+0000 7f222e6af640 1 -- 172.21.15.116:0/4147334599 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2228197580 con 0x7f22281020c0 2024-09-12T02:54:14.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.252+0000 7f22257fa640 1 -- 172.21.15.116:0/4147334599 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2218035030 con 0x7f22281020c0 2024-09-12T02:54:14.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.252+0000 7f222e6af640 1 -- 172.21.15.116:0/4147334599 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f222819a210 con 0x7f22281020c0 2024-09-12T02:54:14.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.253+0000 7f222e6af640 1 -- 172.21.15.116:0/4147334599 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2228107e50 con 0x7f22281020c0 2024-09-12T02:54:14.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.253+0000 7f22257fa640 1 -- 172.21.15.116:0/4147334599 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f2218042070 con 0x7f22281020c0 2024-09-12T02:54:14.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.253+0000 7f22257fa640 1 --2- 172.21.15.116:0/4147334599 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2210039060 0x7f221003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:14.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.253+0000 7f22257fa640 1 -- 172.21.15.116:0/4147334599 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f221806cdc0 con 0x7f22281020c0 2024-09-12T02:54:14.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.254+0000 7f22277fe640 1 --2- 172.21.15.116:0/4147334599 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2210039060 0x7f221003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:14.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.254+0000 7f22277fe640 1 --2- 172.21.15.116:0/4147334599 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2210039060 0x7f221003b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f221c0098c0 tx=0x7f221c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:14.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.258+0000 7f22257fa640 1 -- 172.21.15.116:0/4147334599 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2218039350 con 0x7f22281020c0 2024-09-12T02:54:14.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.437+0000 7f222e6af640 1 -- 172.21.15.116:0/4147334599 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f222802d9b0 con 0x7f2210039060 2024-09-12T02:54:14.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.441+0000 7f22257fa640 1 -- 172.21.15.116:0/4147334599 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f222802d9b0 con 0x7f2210039060 2024-09-12T02:54:14.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.443+0000 7f222e6af640 1 -- 172.21.15.116:0/4147334599 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2210039060 msgr2=0x7f221003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:14.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.443+0000 7f222e6af640 1 --2- 172.21.15.116:0/4147334599 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2210039060 0x7f221003b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f221c0098c0 tx=0x7f221c006db0 comp rx=0 tx=0).stop 2024-09-12T02:54:14.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.443+0000 7f222e6af640 1 -- 172.21.15.116:0/4147334599 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22281020c0 msgr2=0x7f2228196d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:14.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.443+0000 7f222e6af640 1 --2- 172.21.15.116:0/4147334599 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22281020c0 0x7f2228196d50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2218004f60 tx=0x7f22180048f0 comp rx=0 tx=0).stop 2024-09-12T02:54:14.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.443+0000 7f222e6af640 1 -- 172.21.15.116:0/4147334599 shutdown_connections 2024-09-12T02:54:14.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.443+0000 7f222e6af640 1 --2- 172.21.15.116:0/4147334599 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2210039060 0x7f221003b520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:14.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.443+0000 7f222e6af640 1 --2- 172.21.15.116:0/4147334599 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22281020c0 0x7f2228196d50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:14.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.443+0000 7f222e6af640 1 -- 172.21.15.116:0/4147334599 >> 172.21.15.116:0/4147334599 conn(0x7f22280fbca0 msgr2=0x7f22280fc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:14.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.444+0000 7f222e6af640 1 -- 172.21.15.116:0/4147334599 shutdown_connections 2024-09-12T02:54:14.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:14.444+0000 7f222e6af640 1 -- 172.21.15.116:0/4147334599 wait complete. 2024-09-12T02:54:14.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:14 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:14.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:14 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:14.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:14 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:54:14.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:14 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:14.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:14 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:14.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:14 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:15.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:15 smithi116 ceph-mon[31410]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:54:15.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:15.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:15.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:15.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:54:15.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:15 smithi116 ceph-mon[31410]: Updating smithi116:/etc/ceph/ceph.conf 2024-09-12T02:54:15.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:15 smithi116 ceph-mon[31410]: Updating smithi116:/var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:54:15.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:15 smithi116 ceph-mon[31410]: Updating smithi116:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:54:15.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:15 smithi116 ceph-mon[31410]: Updating smithi116:/var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:54:15.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:15.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:15.801 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:15.829 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi177 2024-09-12T02:54:15.829 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-12T02:54:15.829 DEBUG:teuthology.orchestra.run.smithi177:> dd of=/etc/ceph/ceph.conf 2024-09-12T02:54:15.849 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-12T02:54:15.849 DEBUG:teuthology.orchestra.run.smithi177:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:54:15.907 INFO:tasks.cephadm:Adding host smithi177 to orchestrator... 2024-09-12T02:54:16.012 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph orch host add smithi177 2024-09-12T02:54:16.256 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:54:16.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:16.717+0000 7efddecac640 1 -- 172.21.15.116:0/2422993202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdd8106590 msgr2=0x7efdd8106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:16.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:16.717+0000 7efddecac640 1 --2- 172.21.15.116:0/2422993202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdd8106590 0x7efdd8106970 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efdc8009c20 tx=0x7efdc802aa90 comp rx=0 tx=0).stop 2024-09-12T02:54:16.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:16.718+0000 7efddecac640 1 -- 172.21.15.116:0/2422993202 shutdown_connections 2024-09-12T02:54:16.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:16.718+0000 7efddecac640 1 --2- 172.21.15.116:0/2422993202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdd8106590 0x7efdd8106970 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:16.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:16.718+0000 7efddecac640 1 -- 172.21.15.116:0/2422993202 >> 172.21.15.116:0/2422993202 conn(0x7efdd80fc1a0 msgr2=0x7efdd80fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:16.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:16.718+0000 7efddecac640 1 -- 172.21.15.116:0/2422993202 shutdown_connections 2024-09-12T02:54:16.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:16.718+0000 7efddecac640 1 -- 172.21.15.116:0/2422993202 wait complete. 2024-09-12T02:54:16.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:16.718+0000 7efddecac640 1 Processor -- start 2024-09-12T02:54:16.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:16.718+0000 7efddecac640 1 -- start start 2024-09-12T02:54:16.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:16.718+0000 7efddecac640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdd8106590 0x7efdd8197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:16.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:16.718+0000 7efddecac640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efdd810abc0 con 0x7efdd8106590 2024-09-12T02:54:16.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:16.718+0000 7efddca48640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdd8106590 0x7efdd8197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:16.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:16.718+0000 7efddca48640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdd8106590 0x7efdd8197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33434/0 (socket says 172.21.15.116:33434) 2024-09-12T02:54:16.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:16.718+0000 7efddca48640 1 -- 172.21.15.116:0/3783708751 learned_addr learned my addr 172.21.15.116:0/3783708751 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:54:16.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:16.719+0000 7efddca48640 1 -- 172.21.15.116:0/3783708751 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd81984a0 con 0x7efdd8106590 2024-09-12T02:54:16.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:16.719+0000 7efddca48640 1 --2- 172.21.15.116:0/3783708751 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdd8106590 0x7efdd8197f60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efdc8005e30 tx=0x7efdc802fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:16.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:16.719+0000 7efdd5ffb640 1 -- 172.21.15.116:0/3783708751 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdc8047070 con 0x7efdd8106590 2024-09-12T02:54:16.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:16.719+0000 7efdd5ffb640 1 -- 172.21.15.116:0/3783708751 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdc8003df0 con 0x7efdd8106590 2024-09-12T02:54:16.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:16.719+0000 7efddecac640 1 -- 172.21.15.116:0/3783708751 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd8198730 con 0x7efdd8106590 2024-09-12T02:54:16.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:16.719+0000 7efdd5ffb640 1 -- 172.21.15.116:0/3783708751 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdc8033050 con 0x7efdd8106590 2024-09-12T02:54:16.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:16.719+0000 7efddecac640 1 -- 172.21.15.116:0/3783708751 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd8198b90 con 0x7efdd8106590 2024-09-12T02:54:16.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:16.720+0000 7efdd5ffb640 1 -- 172.21.15.116:0/3783708751 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7efdc804a070 con 0x7efdd8106590 2024-09-12T02:54:16.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:16.720+0000 7efddecac640 1 -- 172.21.15.116:0/3783708751 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdd8101c00 con 0x7efdd8106590 2024-09-12T02:54:16.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:16.720+0000 7efdd5ffb640 1 --2- 172.21.15.116:0/3783708751 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7efdc4039060 0x7efdc403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:16.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:16.720+0000 7efdd5ffb640 1 -- 172.21.15.116:0/3783708751 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efdc803e030 con 0x7efdd8106590 2024-09-12T02:54:16.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:16.721+0000 7efdd7fff640 1 --2- 172.21.15.116:0/3783708751 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7efdc4039060 0x7efdc403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:16.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:16.721+0000 7efdd7fff640 1 --2- 172.21.15.116:0/3783708751 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7efdc4039060 0x7efdc403b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efdc00098c0 tx=0x7efdc0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:16.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:16.725+0000 7efdd5ffb640 1 -- 172.21.15.116:0/3783708751 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efdc80373b0 con 0x7efdd8106590 2024-09-12T02:54:16.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:16.895+0000 7efddecac640 1 -- 172.21.15.116:0/3783708751 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi177", "target": ["mon-mgr", ""]}) -- 0x7efdd81069f0 con 0x7efdc4039060 2024-09-12T02:54:17.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:17 smithi116 ceph-mon[31410]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi177", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:54:18.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:18 smithi116 ceph-mon[31410]: Deploying cephadm binary to smithi177 2024-09-12T02:54:19.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:19.971+0000 7efdd5ffb640 1 -- 172.21.15.116:0/3783708751 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7efdd81069f0 con 0x7efdc4039060 2024-09-12T02:54:19.972 INFO:teuthology.orchestra.run.smithi116.stdout:Added host 'smithi177' with addr '172.21.15.177' 2024-09-12T02:54:19.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:19.973+0000 7efddecac640 1 -- 172.21.15.116:0/3783708751 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7efdc4039060 msgr2=0x7efdc403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:19.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:19.973+0000 7efddecac640 1 --2- 172.21.15.116:0/3783708751 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7efdc4039060 0x7efdc403b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efdc00098c0 tx=0x7efdc0006db0 comp rx=0 tx=0).stop 2024-09-12T02:54:19.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:19.974+0000 7efddecac640 1 -- 172.21.15.116:0/3783708751 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdd8106590 msgr2=0x7efdd8197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:19.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:19.974+0000 7efddecac640 1 --2- 172.21.15.116:0/3783708751 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdd8106590 0x7efdd8197f60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efdc8005e30 tx=0x7efdc802fc10 comp rx=0 tx=0).stop 2024-09-12T02:54:19.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:19.974+0000 7efddecac640 1 -- 172.21.15.116:0/3783708751 shutdown_connections 2024-09-12T02:54:19.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:19.974+0000 7efddecac640 1 --2- 172.21.15.116:0/3783708751 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7efdc4039060 0x7efdc403b520 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:19.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:19.974+0000 7efddecac640 1 --2- 172.21.15.116:0/3783708751 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdd8106590 0x7efdd8197f60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:19.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:19.974+0000 7efddecac640 1 -- 172.21.15.116:0/3783708751 >> 172.21.15.116:0/3783708751 conn(0x7efdd80fc1a0 msgr2=0x7efdd80fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:19.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:19.974+0000 7efddecac640 1 -- 172.21.15.116:0/3783708751 shutdown_connections 2024-09-12T02:54:19.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:19.974+0000 7efddecac640 1 -- 172.21.15.116:0/3783708751 wait complete. 2024-09-12T02:54:20.595 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-12T02:54:20.836 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:54:21.209 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:20 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:21.209 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:20 smithi116 ceph-mon[31410]: Added host smithi177 2024-09-12T02:54:21.210 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:20 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:21.210 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:20 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:21.210 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:20 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:21.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.288+0000 7f37b4621640 1 -- 172.21.15.116:0/2303468869 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37ac106590 msgr2=0x7f37ac106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:21.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.288+0000 7f37b4621640 1 --2- 172.21.15.116:0/2303468869 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37ac106590 0x7f37ac106970 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009880 tx=0x7f37a802a5d0 comp rx=0 tx=0).stop 2024-09-12T02:54:21.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.288+0000 7f37b4621640 1 -- 172.21.15.116:0/2303468869 shutdown_connections 2024-09-12T02:54:21.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.288+0000 7f37b4621640 1 --2- 172.21.15.116:0/2303468869 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37ac106590 0x7f37ac106970 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:21.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.288+0000 7f37b4621640 1 -- 172.21.15.116:0/2303468869 >> 172.21.15.116:0/2303468869 conn(0x7f37ac0fc1a0 msgr2=0x7f37ac0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:21.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.288+0000 7f37b4621640 1 -- 172.21.15.116:0/2303468869 shutdown_connections 2024-09-12T02:54:21.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.288+0000 7f37b4621640 1 -- 172.21.15.116:0/2303468869 wait complete. 2024-09-12T02:54:21.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.289+0000 7f37b4621640 1 Processor -- start 2024-09-12T02:54:21.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.289+0000 7f37b4621640 1 -- start start 2024-09-12T02:54:21.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.289+0000 7f37b4621640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37ac106590 0x7f37ac197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:21.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.289+0000 7f37b4621640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac10abc0 con 0x7f37ac106590 2024-09-12T02:54:21.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.289+0000 7f37b23bd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37ac106590 0x7f37ac197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:21.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.289+0000 7f37b23bd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37ac106590 0x7f37ac197150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60222/0 (socket says 172.21.15.116:60222) 2024-09-12T02:54:21.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.289+0000 7f37b23bd640 1 -- 172.21.15.116:0/2022122810 learned_addr learned my addr 172.21.15.116:0/2022122810 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:54:21.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.289+0000 7f37b23bd640 1 -- 172.21.15.116:0/2022122810 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ac197690 con 0x7f37ac106590 2024-09-12T02:54:21.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.289+0000 7f37b23bd640 1 --2- 172.21.15.116:0/2022122810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37ac106590 0x7f37ac197150 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009230 tx=0x7f37a8003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:21.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.289+0000 7f37a37fe640 1 -- 172.21.15.116:0/2022122810 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37a803f070 con 0x7f37ac106590 2024-09-12T02:54:21.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.289+0000 7f37a37fe640 1 -- 172.21.15.116:0/2022122810 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a803bc80 con 0x7f37ac106590 2024-09-12T02:54:21.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.289+0000 7f37b4621640 1 -- 172.21.15.116:0/2022122810 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37ac197920 con 0x7f37ac106590 2024-09-12T02:54:21.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.289+0000 7f37a37fe640 1 -- 172.21.15.116:0/2022122810 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37a8035030 con 0x7f37ac106590 2024-09-12T02:54:21.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.290+0000 7f37b4621640 1 -- 172.21.15.116:0/2022122810 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37ac197d80 con 0x7f37ac106590 2024-09-12T02:54:21.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.290+0000 7f37a37fe640 1 -- 172.21.15.116:0/2022122810 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f37a8042070 con 0x7f37ac106590 2024-09-12T02:54:21.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.291+0000 7f37a37fe640 1 --2- 172.21.15.116:0/2022122810 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3798039060 0x7f379803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:21.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.291+0000 7f37a37fe640 1 -- 172.21.15.116:0/2022122810 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f37a806c950 con 0x7f37ac106590 2024-09-12T02:54:21.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.291+0000 7f37b4621640 1 -- 172.21.15.116:0/2022122810 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37ac101c00 con 0x7f37ac106590 2024-09-12T02:54:21.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.291+0000 7f37b1bbc640 1 --2- 172.21.15.116:0/2022122810 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3798039060 0x7f379803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:21.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.291+0000 7f37b1bbc640 1 --2- 172.21.15.116:0/2022122810 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3798039060 0x7f379803b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f379c0098c0 tx=0x7f379c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:21.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.295+0000 7f37a37fe640 1 -- 172.21.15.116:0/2022122810 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f37a80314c0 con 0x7f37ac106590 2024-09-12T02:54:21.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.478+0000 7f37b4621640 1 -- 172.21.15.116:0/2022122810 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f37ac1984b0 con 0x7f3798039060 2024-09-12T02:54:21.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.480+0000 7f37a37fe640 1 -- 172.21.15.116:0/2022122810 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7f37ac1984b0 con 0x7f3798039060 2024-09-12T02:54:21.480 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:54:21.480 INFO:teuthology.orchestra.run.smithi116.stdout:[{"addr": "172.21.15.116", "hostname": "smithi116", "labels": [], "status": ""}, {"addr": "172.21.15.177", "hostname": "smithi177", "labels": [], "status": ""}] 2024-09-12T02:54:21.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.482+0000 7f37b4621640 1 -- 172.21.15.116:0/2022122810 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3798039060 msgr2=0x7f379803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:21.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.482+0000 7f37b4621640 1 --2- 172.21.15.116:0/2022122810 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3798039060 0x7f379803b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f379c0098c0 tx=0x7f379c006db0 comp rx=0 tx=0).stop 2024-09-12T02:54:21.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.482+0000 7f37b4621640 1 -- 172.21.15.116:0/2022122810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37ac106590 msgr2=0x7f37ac197150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:21.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.482+0000 7f37b4621640 1 --2- 172.21.15.116:0/2022122810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37ac106590 0x7f37ac197150 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009230 tx=0x7f37a8003d40 comp rx=0 tx=0).stop 2024-09-12T02:54:21.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.482+0000 7f37b4621640 1 -- 172.21.15.116:0/2022122810 shutdown_connections 2024-09-12T02:54:21.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.482+0000 7f37b4621640 1 --2- 172.21.15.116:0/2022122810 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3798039060 0x7f379803b520 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:21.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.482+0000 7f37b4621640 1 --2- 172.21.15.116:0/2022122810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37ac106590 0x7f37ac197150 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:21.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.482+0000 7f37b4621640 1 -- 172.21.15.116:0/2022122810 >> 172.21.15.116:0/2022122810 conn(0x7f37ac0fc1a0 msgr2=0x7f37ac069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:21.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.482+0000 7f37b4621640 1 -- 172.21.15.116:0/2022122810 shutdown_connections 2024-09-12T02:54:21.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:21.482+0000 7f37b4621640 1 -- 172.21.15.116:0/2022122810 wait complete. 2024-09-12T02:54:22.073 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi183 2024-09-12T02:54:22.074 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-12T02:54:22.074 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.conf 2024-09-12T02:54:22.093 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-12T02:54:22.094 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:54:22.152 INFO:tasks.cephadm:Adding host smithi183 to orchestrator... 2024-09-12T02:54:22.152 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph orch host add smithi183 2024-09-12T02:54:22.391 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:54:22.623 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:22 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:22.624 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:22 smithi116 ceph-mon[31410]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:54:22.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:22.838+0000 7f4394d65640 1 -- 172.21.15.116:0/1432192418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4390106590 msgr2=0x7f4390106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:22.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:22.838+0000 7f4394d65640 1 --2- 172.21.15.116:0/1432192418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4390106590 0x7f4390106970 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4380009880 tx=0x7f438002a5d0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:22.838+0000 7f4394d65640 1 -- 172.21.15.116:0/1432192418 shutdown_connections 2024-09-12T02:54:22.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:22.838+0000 7f4394d65640 1 --2- 172.21.15.116:0/1432192418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4390106590 0x7f4390106970 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:22.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:22.838+0000 7f4394d65640 1 -- 172.21.15.116:0/1432192418 >> 172.21.15.116:0/1432192418 conn(0x7f43900fc1a0 msgr2=0x7f43900fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:22.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:22.838+0000 7f4394d65640 1 -- 172.21.15.116:0/1432192418 shutdown_connections 2024-09-12T02:54:22.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:22.838+0000 7f4394d65640 1 -- 172.21.15.116:0/1432192418 wait complete. 2024-09-12T02:54:22.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:22.839+0000 7f4394d65640 1 Processor -- start 2024-09-12T02:54:22.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:22.839+0000 7f4394d65640 1 -- start start 2024-09-12T02:54:22.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:22.839+0000 7f4394d65640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4390106590 0x7f4390197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:22.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:22.839+0000 7f4394d65640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f439010abc0 con 0x7f4390106590 2024-09-12T02:54:22.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:22.839+0000 7f438e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4390106590 0x7f4390197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:22.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:22.839+0000 7f438e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4390106590 0x7f4390197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60246/0 (socket says 172.21.15.116:60246) 2024-09-12T02:54:22.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:22.839+0000 7f438e59c640 1 -- 172.21.15.116:0/1915814156 learned_addr learned my addr 172.21.15.116:0/1915814156 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:54:22.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:22.839+0000 7f438e59c640 1 -- 172.21.15.116:0/1915814156 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43901984a0 con 0x7f4390106590 2024-09-12T02:54:22.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:22.839+0000 7f438e59c640 1 --2- 172.21.15.116:0/1915814156 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4390106590 0x7f4390197f60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4380009230 tx=0x7f4380003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:22.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:22.839+0000 7f43877fe640 1 -- 172.21.15.116:0/1915814156 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f438003f070 con 0x7f4390106590 2024-09-12T02:54:22.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:22.839+0000 7f4394d65640 1 -- 172.21.15.116:0/1915814156 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4390198730 con 0x7f4390106590 2024-09-12T02:54:22.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:22.839+0000 7f43877fe640 1 -- 172.21.15.116:0/1915814156 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f438003bc80 con 0x7f4390106590 2024-09-12T02:54:22.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:22.839+0000 7f43877fe640 1 -- 172.21.15.116:0/1915814156 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4380035030 con 0x7f4390106590 2024-09-12T02:54:22.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:22.840+0000 7f4394d65640 1 -- 172.21.15.116:0/1915814156 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4390198b90 con 0x7f4390106590 2024-09-12T02:54:22.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:22.841+0000 7f43877fe640 1 -- 172.21.15.116:0/1915814156 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f4380042070 con 0x7f4390106590 2024-09-12T02:54:22.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:22.841+0000 7f4394d65640 1 -- 172.21.15.116:0/1915814156 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4390101c00 con 0x7f4390106590 2024-09-12T02:54:22.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:22.841+0000 7f43877fe640 1 --2- 172.21.15.116:0/1915814156 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f437c039060 0x7f437c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:22.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:22.841+0000 7f43877fe640 1 -- 172.21.15.116:0/1915814156 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f438006cde0 con 0x7f4390106590 2024-09-12T02:54:22.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:22.841+0000 7f438dd9b640 1 --2- 172.21.15.116:0/1915814156 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f437c039060 0x7f437c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:22.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:22.841+0000 7f438dd9b640 1 --2- 172.21.15.116:0/1915814156 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f437c039060 0x7f437c03b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f43780098c0 tx=0x7f4378006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:22.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:22.846+0000 7f43877fe640 1 -- 172.21.15.116:0/1915814156 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f438004c350 con 0x7f4390106590 2024-09-12T02:54:23.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:23.036+0000 7f4394d65640 1 -- 172.21.15.116:0/1915814156 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}) -- 0x7f43901069f0 con 0x7f437c039060 2024-09-12T02:54:24.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:24 smithi116 ceph-mon[31410]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:24.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:24 smithi116 ceph-mon[31410]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:54:25.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:25 smithi116 ceph-mon[31410]: Deploying cephadm binary to smithi183 2024-09-12T02:54:25.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:25 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:25.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:25 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:25.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:25 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:25.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:25 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:25.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:25 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi177", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:54:25.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:25 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:25.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:25 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:54:25.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:25 smithi116 ceph-mon[31410]: Updating smithi177:/etc/ceph/ceph.conf 2024-09-12T02:54:26.176 INFO:teuthology.orchestra.run.smithi116.stdout:Added host 'smithi183' with addr '172.21.15.183' 2024-09-12T02:54:26.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:26.175+0000 7f43877fe640 1 -- 172.21.15.116:0/1915814156 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f43901069f0 con 0x7f437c039060 2024-09-12T02:54:26.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:26.178+0000 7f4394d65640 1 -- 172.21.15.116:0/1915814156 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f437c039060 msgr2=0x7f437c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:26.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:26.178+0000 7f4394d65640 1 --2- 172.21.15.116:0/1915814156 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f437c039060 0x7f437c03b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f43780098c0 tx=0x7f4378006db0 comp rx=0 tx=0).stop 2024-09-12T02:54:26.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:26.178+0000 7f4394d65640 1 -- 172.21.15.116:0/1915814156 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4390106590 msgr2=0x7f4390197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:26.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:26.178+0000 7f4394d65640 1 --2- 172.21.15.116:0/1915814156 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4390106590 0x7f4390197f60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4380009230 tx=0x7f4380003d40 comp rx=0 tx=0).stop 2024-09-12T02:54:26.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:26.178+0000 7f4394d65640 1 -- 172.21.15.116:0/1915814156 shutdown_connections 2024-09-12T02:54:26.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:26.178+0000 7f4394d65640 1 --2- 172.21.15.116:0/1915814156 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f437c039060 0x7f437c03b520 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:26.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:26.178+0000 7f4394d65640 1 --2- 172.21.15.116:0/1915814156 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4390106590 0x7f4390197f60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:26.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:26.178+0000 7f4394d65640 1 -- 172.21.15.116:0/1915814156 >> 172.21.15.116:0/1915814156 conn(0x7f43900fc1a0 msgr2=0x7f43900fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:26.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:26.178+0000 7f4394d65640 1 -- 172.21.15.116:0/1915814156 shutdown_connections 2024-09-12T02:54:26.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:26.178+0000 7f4394d65640 1 -- 172.21.15.116:0/1915814156 wait complete. 2024-09-12T02:54:26.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:26 smithi116 ceph-mon[31410]: Updating smithi177:/var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:54:26.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:26 smithi116 ceph-mon[31410]: Updating smithi177:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:54:26.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:26 smithi116 ceph-mon[31410]: Updating smithi177:/var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:54:26.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:26 smithi116 ceph-mon[31410]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:26.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:26 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:26.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:26 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:26.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:26 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:26.796 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-12T02:54:27.034 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:54:27.464 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:27 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:27.464 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:27 smithi116 ceph-mon[31410]: Added host smithi183 2024-09-12T02:54:27.464 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:27 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:27.464 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:27 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:27.464 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:27 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:27.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.543+0000 7f8a8aaea640 1 -- 172.21.15.116:0/462671858 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a84100990 msgr2=0x7f8a84100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:27.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.543+0000 7f8a8aaea640 1 --2- 172.21.15.116:0/462671858 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a84100990 0x7f8a84100d70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009880 tx=0x7f8a7802a5d0 comp rx=0 tx=0).stop 2024-09-12T02:54:27.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.543+0000 7f8a8aaea640 1 -- 172.21.15.116:0/462671858 shutdown_connections 2024-09-12T02:54:27.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.543+0000 7f8a8aaea640 1 --2- 172.21.15.116:0/462671858 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a84100990 0x7f8a84100d70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:27.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.543+0000 7f8a8aaea640 1 -- 172.21.15.116:0/462671858 >> 172.21.15.116:0/462671858 conn(0x7f8a84076250 msgr2=0x7f8a84076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:27.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.543+0000 7f8a8aaea640 1 -- 172.21.15.116:0/462671858 shutdown_connections 2024-09-12T02:54:27.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.543+0000 7f8a8aaea640 1 -- 172.21.15.116:0/462671858 wait complete. 2024-09-12T02:54:27.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.544+0000 7f8a8aaea640 1 Processor -- start 2024-09-12T02:54:27.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.544+0000 7f8a8aaea640 1 -- start start 2024-09-12T02:54:27.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.544+0000 7f8a8aaea640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a84100990 0x7f8a84196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:27.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.544+0000 7f8a8aaea640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8410ab90 con 0x7f8a84100990 2024-09-12T02:54:27.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.544+0000 7f8a88886640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a84100990 0x7f8a84196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:27.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.544+0000 7f8a88886640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a84100990 0x7f8a84196130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60276/0 (socket says 172.21.15.116:60276) 2024-09-12T02:54:27.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.544+0000 7f8a88886640 1 -- 172.21.15.116:0/3391004125 learned_addr learned my addr 172.21.15.116:0/3391004125 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:54:27.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.545+0000 7f8a88886640 1 -- 172.21.15.116:0/3391004125 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a84196670 con 0x7f8a84100990 2024-09-12T02:54:27.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.545+0000 7f8a88886640 1 --2- 172.21.15.116:0/3391004125 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a84100990 0x7f8a84196130 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009230 tx=0x7f8a78003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:27.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.545+0000 7f8a81ffb640 1 -- 172.21.15.116:0/3391004125 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a7803f070 con 0x7f8a84100990 2024-09-12T02:54:27.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.545+0000 7f8a81ffb640 1 -- 172.21.15.116:0/3391004125 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a7803bc80 con 0x7f8a84100990 2024-09-12T02:54:27.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.545+0000 7f8a8aaea640 1 -- 172.21.15.116:0/3391004125 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a84196960 con 0x7f8a84100990 2024-09-12T02:54:27.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.545+0000 7f8a81ffb640 1 -- 172.21.15.116:0/3391004125 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a78035030 con 0x7f8a84100990 2024-09-12T02:54:27.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.545+0000 7f8a8aaea640 1 -- 172.21.15.116:0/3391004125 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a8419aa00 con 0x7f8a84100990 2024-09-12T02:54:27.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.546+0000 7f8a81ffb640 1 -- 172.21.15.116:0/3391004125 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f8a78042070 con 0x7f8a84100990 2024-09-12T02:54:27.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.547+0000 7f8a81ffb640 1 --2- 172.21.15.116:0/3391004125 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8a740390c0 0x7f8a7403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:27.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.547+0000 7f8a81ffb640 1 -- 172.21.15.116:0/3391004125 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8a7806cd50 con 0x7f8a84100990 2024-09-12T02:54:27.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.547+0000 7f8a8aaea640 1 -- 172.21.15.116:0/3391004125 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a84101c40 con 0x7f8a84100990 2024-09-12T02:54:27.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.547+0000 7f8a83fff640 1 --2- 172.21.15.116:0/3391004125 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8a740390c0 0x7f8a7403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:27.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.547+0000 7f8a83fff640 1 --2- 172.21.15.116:0/3391004125 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8a740390c0 0x7f8a7403b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8a700098c0 tx=0x7f8a70006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:27.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.552+0000 7f8a81ffb640 1 -- 172.21.15.116:0/3391004125 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8a780372d0 con 0x7f8a84100990 2024-09-12T02:54:27.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.722+0000 7f8a8aaea640 1 -- 172.21.15.116:0/3391004125 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8a84100df0 con 0x7f8a740390c0 2024-09-12T02:54:27.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.723+0000 7f8a81ffb640 1 -- 172.21.15.116:0/3391004125 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f8a84100df0 con 0x7f8a740390c0 2024-09-12T02:54:27.724 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:54:27.724 INFO:teuthology.orchestra.run.smithi116.stdout:[{"addr": "172.21.15.116", "hostname": "smithi116", "labels": [], "status": ""}, {"addr": "172.21.15.177", "hostname": "smithi177", "labels": [], "status": ""}, {"addr": "172.21.15.183", "hostname": "smithi183", "labels": [], "status": ""}] 2024-09-12T02:54:27.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.725+0000 7f8a8aaea640 1 -- 172.21.15.116:0/3391004125 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8a740390c0 msgr2=0x7f8a7403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:27.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.725+0000 7f8a8aaea640 1 --2- 172.21.15.116:0/3391004125 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8a740390c0 0x7f8a7403b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8a700098c0 tx=0x7f8a70006db0 comp rx=0 tx=0).stop 2024-09-12T02:54:27.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.725+0000 7f8a8aaea640 1 -- 172.21.15.116:0/3391004125 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a84100990 msgr2=0x7f8a84196130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:27.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.725+0000 7f8a8aaea640 1 --2- 172.21.15.116:0/3391004125 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a84100990 0x7f8a84196130 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009230 tx=0x7f8a78003d40 comp rx=0 tx=0).stop 2024-09-12T02:54:27.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.725+0000 7f8a8aaea640 1 -- 172.21.15.116:0/3391004125 shutdown_connections 2024-09-12T02:54:27.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.725+0000 7f8a8aaea640 1 --2- 172.21.15.116:0/3391004125 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8a740390c0 0x7f8a7403b580 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:27.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.725+0000 7f8a8aaea640 1 --2- 172.21.15.116:0/3391004125 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a84100990 0x7f8a84196130 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:27.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.725+0000 7f8a8aaea640 1 -- 172.21.15.116:0/3391004125 >> 172.21.15.116:0/3391004125 conn(0x7f8a84076250 msgr2=0x7f8a841020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:27.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.725+0000 7f8a8aaea640 1 -- 172.21.15.116:0/3391004125 shutdown_connections 2024-09-12T02:54:27.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:27.725+0000 7f8a8aaea640 1 -- 172.21.15.116:0/3391004125 wait complete. 2024-09-12T02:54:28.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:28 smithi116 ceph-mon[31410]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:28.293 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:28 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:28.320 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-12T02:54:28.320 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-12T02:54:28.577 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:54:29.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:29.063+0000 7f8863455640 1 -- 172.21.15.116:0/2767615310 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f885c107690 msgr2=0x7f885c107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:29.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:29.063+0000 7f8863455640 1 --2- 172.21.15.116:0/2767615310 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f885c107690 0x7f885c107a70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f884c009880 tx=0x7f884c02a5d0 comp rx=0 tx=0).stop 2024-09-12T02:54:29.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:29.063+0000 7f8863455640 1 -- 172.21.15.116:0/2767615310 shutdown_connections 2024-09-12T02:54:29.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:29.063+0000 7f8863455640 1 --2- 172.21.15.116:0/2767615310 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f885c107690 0x7f885c107a70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:29.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:29.063+0000 7f8863455640 1 -- 172.21.15.116:0/2767615310 >> 172.21.15.116:0/2767615310 conn(0x7f885c076250 msgr2=0x7f885c076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:29.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:29.063+0000 7f8863455640 1 -- 172.21.15.116:0/2767615310 shutdown_connections 2024-09-12T02:54:29.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:29.063+0000 7f8863455640 1 -- 172.21.15.116:0/2767615310 wait complete. 2024-09-12T02:54:29.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:29.063+0000 7f8863455640 1 Processor -- start 2024-09-12T02:54:29.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:29.063+0000 7f8863455640 1 -- start start 2024-09-12T02:54:29.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:29.064+0000 7f8863455640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f885c107690 0x7f885c197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:29.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:29.064+0000 7f8863455640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f885c10bce0 con 0x7f885c107690 2024-09-12T02:54:29.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:29.064+0000 7f88611f1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f885c107690 0x7f885c197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:29.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:29.064+0000 7f88611f1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f885c107690 0x7f885c197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60302/0 (socket says 172.21.15.116:60302) 2024-09-12T02:54:29.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:29.064+0000 7f88611f1640 1 -- 172.21.15.116:0/1102187658 learned_addr learned my addr 172.21.15.116:0/1102187658 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:54:29.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:29.064+0000 7f88611f1640 1 -- 172.21.15.116:0/1102187658 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f885c197650 con 0x7f885c107690 2024-09-12T02:54:29.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:29.064+0000 7f88611f1640 1 --2- 172.21.15.116:0/1102187658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f885c107690 0x7f885c197110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f884c009230 tx=0x7f884c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:29.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:29.064+0000 7f88527fc640 1 -- 172.21.15.116:0/1102187658 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f884c03f070 con 0x7f885c107690 2024-09-12T02:54:29.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:29.064+0000 7f88527fc640 1 -- 172.21.15.116:0/1102187658 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f884c03bc80 con 0x7f885c107690 2024-09-12T02:54:29.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:29.064+0000 7f8863455640 1 -- 172.21.15.116:0/1102187658 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f885c1978e0 con 0x7f885c107690 2024-09-12T02:54:29.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:29.064+0000 7f88527fc640 1 -- 172.21.15.116:0/1102187658 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f884c035030 con 0x7f885c107690 2024-09-12T02:54:29.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:29.064+0000 7f8863455640 1 -- 172.21.15.116:0/1102187658 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f885c19b9e0 con 0x7f885c107690 2024-09-12T02:54:29.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:29.065+0000 7f88527fc640 1 -- 172.21.15.116:0/1102187658 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f884c042070 con 0x7f885c107690 2024-09-12T02:54:29.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:29.065+0000 7f88527fc640 1 --2- 172.21.15.116:0/1102187658 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8848039060 0x7f884803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:29.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:29.065+0000 7f88527fc640 1 -- 172.21.15.116:0/1102187658 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f884c06ccf0 con 0x7f885c107690 2024-09-12T02:54:29.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:29.066+0000 7f8863455640 1 -- 172.21.15.116:0/1102187658 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f885c102d30 con 0x7f885c107690 2024-09-12T02:54:29.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:29.066+0000 7f88609f0640 1 --2- 172.21.15.116:0/1102187658 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8848039060 0x7f884803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:29.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:29.066+0000 7f88609f0640 1 --2- 172.21.15.116:0/1102187658 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8848039060 0x7f884803b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f88580098c0 tx=0x7f8858006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:29.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:29.071+0000 7f88527fc640 1 -- 172.21.15.116:0/1102187658 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f884c04c350 con 0x7f885c107690 2024-09-12T02:54:29.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:29.269+0000 7f8863455640 1 -- 172.21.15.116:0/1102187658 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f885c107af0 con 0x7f885c107690 2024-09-12T02:54:29.271 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:29 smithi116 ceph-mon[31410]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:54:30.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:30.178+0000 7f88527fc640 1 -- 172.21.15.116:0/1102187658 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f884c031e20 con 0x7f885c107690 2024-09-12T02:54:30.179 INFO:teuthology.orchestra.run.smithi116.stderr:adjusted tunables profile to default 2024-09-12T02:54:30.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:30.180+0000 7f8863455640 1 -- 172.21.15.116:0/1102187658 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8848039060 msgr2=0x7f884803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:30.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:30.180+0000 7f8863455640 1 --2- 172.21.15.116:0/1102187658 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8848039060 0x7f884803b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f88580098c0 tx=0x7f8858006db0 comp rx=0 tx=0).stop 2024-09-12T02:54:30.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:30.180+0000 7f8863455640 1 -- 172.21.15.116:0/1102187658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f885c107690 msgr2=0x7f885c197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:30.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:30.180+0000 7f8863455640 1 --2- 172.21.15.116:0/1102187658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f885c107690 0x7f885c197110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f884c009230 tx=0x7f884c003d40 comp rx=0 tx=0).stop 2024-09-12T02:54:30.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:30.181+0000 7f8863455640 1 -- 172.21.15.116:0/1102187658 shutdown_connections 2024-09-12T02:54:30.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:30.181+0000 7f8863455640 1 --2- 172.21.15.116:0/1102187658 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8848039060 0x7f884803b520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:30.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:30.181+0000 7f8863455640 1 --2- 172.21.15.116:0/1102187658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f885c107690 0x7f885c197110 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:30.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:30.181+0000 7f8863455640 1 -- 172.21.15.116:0/1102187658 >> 172.21.15.116:0/1102187658 conn(0x7f885c076250 msgr2=0x7f885c0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:30.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:30.181+0000 7f8863455640 1 -- 172.21.15.116:0/1102187658 shutdown_connections 2024-09-12T02:54:30.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:30.181+0000 7f8863455640 1 -- 172.21.15.116:0/1102187658 wait complete. 2024-09-12T02:54:30.481 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:30 smithi116 ceph-mon[31410]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:30.482 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:30 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1102187658' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T02:54:30.841 INFO:tasks.cephadm:Adding mon.a on smithi116 2024-09-12T02:54:30.841 INFO:tasks.cephadm:Adding mon.b on smithi177 2024-09-12T02:54:30.841 INFO:tasks.cephadm:Adding mon.c on smithi183 2024-09-12T02:54:30.841 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi116:172.21.15.116=a;smithi177:172.21.15.177=b;smithi183:172.21.15.183=c' 2024-09-12T02:54:31.093 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:54:31.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:31 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1102187658' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T02:54:31.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:31 smithi116 ceph-mon[31410]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:54:31.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:31 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:31.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:31 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:31.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:31 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:31.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:31 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:31.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:31 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:54:31.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:31 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:31.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:31 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:54:31.630 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.581+0000 7f6a4ae67640 1 -- 172.21.15.183:0/3848860540 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a44100990 msgr2=0x7f6a44100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:31.630 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.581+0000 7f6a4ae67640 1 --2- 172.21.15.183:0/3848860540 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a44100990 0x7f6a44100d70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6a38009880 tx=0x7f6a3802a5d0 comp rx=0 tx=0).stop 2024-09-12T02:54:31.630 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.581+0000 7f6a4ae67640 1 -- 172.21.15.183:0/3848860540 shutdown_connections 2024-09-12T02:54:31.630 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.581+0000 7f6a4ae67640 1 --2- 172.21.15.183:0/3848860540 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a44100990 0x7f6a44100d70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:31.630 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.581+0000 7f6a4ae67640 1 -- 172.21.15.183:0/3848860540 >> 172.21.15.183:0/3848860540 conn(0x7f6a44076250 msgr2=0x7f6a44076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:31.631 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.581+0000 7f6a4ae67640 1 -- 172.21.15.183:0/3848860540 shutdown_connections 2024-09-12T02:54:31.631 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.581+0000 7f6a4ae67640 1 -- 172.21.15.183:0/3848860540 wait complete. 2024-09-12T02:54:31.631 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.582+0000 7f6a4ae67640 1 Processor -- start 2024-09-12T02:54:31.631 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.582+0000 7f6a4ae67640 1 -- start start 2024-09-12T02:54:31.631 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.582+0000 7f6a4ae67640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a44100990 0x7f6a441970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:31.631 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.582+0000 7f6a4ae67640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4410ab90 con 0x7f6a44100990 2024-09-12T02:54:31.631 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.583+0000 7f6a48c03640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a44100990 0x7f6a441970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:31.631 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.583+0000 7f6a48c03640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a44100990 0x7f6a441970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.183:37316/0 (socket says 172.21.15.183:37316) 2024-09-12T02:54:31.631 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.583+0000 7f6a48c03640 1 -- 172.21.15.183:0/2407785117 learned_addr learned my addr 172.21.15.183:0/2407785117 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-12T02:54:31.631 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.583+0000 7f6a48c03640 1 -- 172.21.15.183:0/2407785117 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a44197630 con 0x7f6a44100990 2024-09-12T02:54:31.631 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.583+0000 7f6a48c03640 1 --2- 172.21.15.183:0/2407785117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a44100990 0x7f6a441970f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6a38009230 tx=0x7f6a38003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:31.631 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.583+0000 7f6a41ffb640 1 -- 172.21.15.183:0/2407785117 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a3803f070 con 0x7f6a44100990 2024-09-12T02:54:31.632 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.583+0000 7f6a41ffb640 1 -- 172.21.15.183:0/2407785117 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a3803bc80 con 0x7f6a44100990 2024-09-12T02:54:31.632 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.583+0000 7f6a4ae67640 1 -- 172.21.15.183:0/2407785117 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a44197920 con 0x7f6a44100990 2024-09-12T02:54:31.632 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.583+0000 7f6a41ffb640 1 -- 172.21.15.183:0/2407785117 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a38035030 con 0x7f6a44100990 2024-09-12T02:54:31.632 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.584+0000 7f6a4ae67640 1 -- 172.21.15.183:0/2407785117 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a4419b9c0 con 0x7f6a44100990 2024-09-12T02:54:31.632 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.585+0000 7f6a41ffb640 1 -- 172.21.15.183:0/2407785117 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f6a38042070 con 0x7f6a44100990 2024-09-12T02:54:31.632 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.585+0000 7f6a4ae67640 1 -- 172.21.15.183:0/2407785117 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a44101c40 con 0x7f6a44100990 2024-09-12T02:54:31.632 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.585+0000 7f6a41ffb640 1 --2- 172.21.15.183:0/2407785117 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6a34039060 0x7f6a3403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:31.632 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.585+0000 7f6a41ffb640 1 -- 172.21.15.183:0/2407785117 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6a3806cc50 con 0x7f6a44100990 2024-09-12T02:54:31.632 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.585+0000 7f6a43fff640 1 --2- 172.21.15.183:0/2407785117 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6a34039060 0x7f6a3403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:31.632 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.586+0000 7f6a43fff640 1 --2- 172.21.15.183:0/2407785117 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6a34039060 0x7f6a3403b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6a300098c0 tx=0x7f6a30006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:31.632 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.590+0000 7f6a41ffb640 1 -- 172.21.15.183:0/2407785117 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6a38072020 con 0x7f6a44100990 2024-09-12T02:54:31.773 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.772+0000 7f6a4ae67640 1 -- 172.21.15.183:0/2407785117 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi116:172.21.15.116=a;smithi177:172.21.15.177=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}) -- 0x7f6a44069dd0 con 0x7f6a34039060 2024-09-12T02:54:31.779 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.778+0000 7f6a41ffb640 1 -- 172.21.15.183:0/2407785117 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6a44069dd0 con 0x7f6a34039060 2024-09-12T02:54:31.779 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled mon update... 2024-09-12T02:54:31.781 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.780+0000 7f6a4ae67640 1 -- 172.21.15.183:0/2407785117 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6a34039060 msgr2=0x7f6a3403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:31.781 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.780+0000 7f6a4ae67640 1 --2- 172.21.15.183:0/2407785117 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6a34039060 0x7f6a3403b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6a300098c0 tx=0x7f6a30006db0 comp rx=0 tx=0).stop 2024-09-12T02:54:31.781 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.780+0000 7f6a4ae67640 1 -- 172.21.15.183:0/2407785117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a44100990 msgr2=0x7f6a441970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:31.781 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.780+0000 7f6a4ae67640 1 --2- 172.21.15.183:0/2407785117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a44100990 0x7f6a441970f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6a38009230 tx=0x7f6a38003d40 comp rx=0 tx=0).stop 2024-09-12T02:54:31.781 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.780+0000 7f6a4ae67640 1 -- 172.21.15.183:0/2407785117 shutdown_connections 2024-09-12T02:54:31.781 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.780+0000 7f6a4ae67640 1 --2- 172.21.15.183:0/2407785117 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6a34039060 0x7f6a3403b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:31.781 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.780+0000 7f6a4ae67640 1 --2- 172.21.15.183:0/2407785117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a44100990 0x7f6a441970f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:31.782 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.780+0000 7f6a4ae67640 1 -- 172.21.15.183:0/2407785117 >> 172.21.15.183:0/2407785117 conn(0x7f6a44076250 msgr2=0x7f6a440696c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:31.782 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.780+0000 7f6a4ae67640 1 -- 172.21.15.183:0/2407785117 shutdown_connections 2024-09-12T02:54:31.782 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:31.780+0000 7f6a4ae67640 1 -- 172.21.15.183:0/2407785117 wait complete. 2024-09-12T02:54:32.381 DEBUG:teuthology.orchestra.run.smithi177:mon.b> sudo journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mon.b.service 2024-09-12T02:54:32.384 DEBUG:teuthology.orchestra.run.smithi183:mon.c> sudo journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mon.c.service 2024-09-12T02:54:32.386 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T02:54:32.387 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T02:54:32.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:32 smithi116 ceph-mon[31410]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-12T02:54:32.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:32 smithi116 ceph-mon[31410]: Updating smithi183:/var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:54:32.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:32 smithi116 ceph-mon[31410]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:54:32.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:32 smithi116 ceph-mon[31410]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:32.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:32 smithi116 ceph-mon[31410]: Updating smithi183:/var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:54:32.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:32 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:32.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:32 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:32.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:32 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:32.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:32 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:32.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:32 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:32.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:32 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:32.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:32 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:54:32.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:32 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:32.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:32 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:54:32.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:32 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:33.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:33 smithi116 ceph-mon[31410]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi116:172.21.15.116=a;smithi177:172.21.15.177=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:54:33.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:33 smithi116 ceph-mon[31410]: Saving service mon spec with placement smithi116:172.21.15.116=a;smithi177:172.21.15.177=b;smithi183:172.21.15.183=c;count:3 2024-09-12T02:54:33.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:33 smithi116 ceph-mon[31410]: Deploying daemon mon.c on smithi183 2024-09-12T02:54:34.155 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:54:34.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:34 smithi116 ceph-mon[31410]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:35.126 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.124+0000 7f0afb305640 1 -- 172.21.15.183:0/1760265436 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0af4107660 msgr2=0x7f0af4107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:35.126 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.124+0000 7f0afb305640 1 --2- 172.21.15.183:0/1760265436 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0af4107660 0x7f0af4107a40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0af0009880 tx=0x7f0af002a5d0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.127 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.124+0000 7f0afb305640 1 -- 172.21.15.183:0/1760265436 shutdown_connections 2024-09-12T02:54:35.127 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.124+0000 7f0afb305640 1 --2- 172.21.15.183:0/1760265436 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0af4107660 0x7f0af4107a40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.127 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.124+0000 7f0afb305640 1 -- 172.21.15.183:0/1760265436 >> 172.21.15.183:0/1760265436 conn(0x7f0af40fd2b0 msgr2=0x7f0af40ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:35.127 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.124+0000 7f0afb305640 1 -- 172.21.15.183:0/1760265436 shutdown_connections 2024-09-12T02:54:35.127 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.124+0000 7f0afb305640 1 -- 172.21.15.183:0/1760265436 wait complete. 2024-09-12T02:54:35.127 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.125+0000 7f0afb305640 1 Processor -- start 2024-09-12T02:54:35.127 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.125+0000 7f0afb305640 1 -- start start 2024-09-12T02:54:35.128 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.125+0000 7f0afb305640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0af4107660 0x7f0af41970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:35.128 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.125+0000 7f0afb305640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0af410bc70 con 0x7f0af4107660 2024-09-12T02:54:35.128 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.125+0000 7f0af90a1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0af4107660 0x7f0af41970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:35.128 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.125+0000 7f0af90a1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0af4107660 0x7f0af41970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.183:37336/0 (socket says 172.21.15.183:37336) 2024-09-12T02:54:35.128 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.125+0000 7f0af90a1640 1 -- 172.21.15.183:0/2923814928 learned_addr learned my addr 172.21.15.183:0/2923814928 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-12T02:54:35.128 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.126+0000 7f0af90a1640 1 -- 172.21.15.183:0/2923814928 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af41975e0 con 0x7f0af4107660 2024-09-12T02:54:35.134 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.126+0000 7f0af90a1640 1 --2- 172.21.15.183:0/2923814928 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0af4107660 0x7f0af41970a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0af0009230 tx=0x7f0af0003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:35.134 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.126+0000 7f0aea7fc640 1 -- 172.21.15.183:0/2923814928 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0af003f070 con 0x7f0af4107660 2024-09-12T02:54:35.134 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.126+0000 7f0afb305640 1 -- 172.21.15.183:0/2923814928 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0af41978d0 con 0x7f0af4107660 2024-09-12T02:54:35.134 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.126+0000 7f0aea7fc640 1 -- 172.21.15.183:0/2923814928 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0af003bc80 con 0x7f0af4107660 2024-09-12T02:54:35.134 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.126+0000 7f0aea7fc640 1 -- 172.21.15.183:0/2923814928 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0af0035030 con 0x7f0af4107660 2024-09-12T02:54:35.134 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.126+0000 7f0afb305640 1 -- 172.21.15.183:0/2923814928 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0af419b970 con 0x7f0af4107660 2024-09-12T02:54:35.134 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.127+0000 7f0aea7fc640 1 -- 172.21.15.183:0/2923814928 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f0af0042070 con 0x7f0af4107660 2024-09-12T02:54:35.134 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.127+0000 7f0afb305640 1 -- 172.21.15.183:0/2923814928 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0af4102d10 con 0x7f0af4107660 2024-09-12T02:54:35.135 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.127+0000 7f0aea7fc640 1 --2- 172.21.15.183:0/2923814928 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0ae00390c0 0x7f0ae003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:35.135 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.127+0000 7f0aea7fc640 1 -- 172.21.15.183:0/2923814928 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0af006ccd0 con 0x7f0af4107660 2024-09-12T02:54:35.135 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.127+0000 7f0af88a0640 1 --2- 172.21.15.183:0/2923814928 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0ae00390c0 0x7f0ae003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:35.135 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.128+0000 7f0af88a0640 1 --2- 172.21.15.183:0/2923814928 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0ae00390c0 0x7f0ae003b580 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0ae40098c0 tx=0x7f0ae4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:35.135 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.132+0000 7f0aea7fc640 1 -- 172.21.15.183:0/2923814928 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0af0072020 con 0x7f0af4107660 2024-09-12T02:54:35.383 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.382+0000 7f0afb305640 1 -- 172.21.15.183:0/2923814928 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f0af4198400 con 0x7f0af4107660 2024-09-12T02:54:35.384 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.383+0000 7f0aea7fc640 1 -- 172.21.15.183:0/2923814928 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f0af00314c0 con 0x7f0af4107660 2024-09-12T02:54:35.384 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T02:54:35.384 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":1,"fsid":"eb1ac602-70b1-11ef-bcea-c7b262605968","modified":"2024-09-12T02:53:04.190821Z","created":"2024-09-12T02:53:04.190821Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:3300","nonce":0},{"type":"v1","addr":"172.21.15.116:6789","nonce":0}]},"addr":"172.21.15.116:6789/0","public_addr":"172.21.15.116:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-12T02:54:35.384 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 1 2024-09-12T02:54:35.386 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.385+0000 7f0afb305640 1 -- 172.21.15.183:0/2923814928 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0ae00390c0 msgr2=0x7f0ae003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:35.386 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.385+0000 7f0afb305640 1 --2- 172.21.15.183:0/2923814928 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0ae00390c0 0x7f0ae003b580 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0ae40098c0 tx=0x7f0ae4006db0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.386 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.385+0000 7f0afb305640 1 -- 172.21.15.183:0/2923814928 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0af4107660 msgr2=0x7f0af41970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:35.386 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.385+0000 7f0afb305640 1 --2- 172.21.15.183:0/2923814928 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0af4107660 0x7f0af41970a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0af0009230 tx=0x7f0af0003d40 comp rx=0 tx=0).stop 2024-09-12T02:54:35.387 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.385+0000 7f0afb305640 1 -- 172.21.15.183:0/2923814928 shutdown_connections 2024-09-12T02:54:35.387 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.385+0000 7f0afb305640 1 --2- 172.21.15.183:0/2923814928 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0ae00390c0 0x7f0ae003b580 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.387 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.385+0000 7f0afb305640 1 --2- 172.21.15.183:0/2923814928 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0af4107660 0x7f0af41970a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:35.387 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.385+0000 7f0afb305640 1 -- 172.21.15.183:0/2923814928 >> 172.21.15.183:0/2923814928 conn(0x7f0af40fd2b0 msgr2=0x7f0af40feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:35.387 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.385+0000 7f0afb305640 1 -- 172.21.15.183:0/2923814928 shutdown_connections 2024-09-12T02:54:35.387 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:35.386+0000 7f0afb305640 1 -- 172.21.15.183:0/2923814928 wait complete. 2024-09-12T02:54:36.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 podman[35516]: 2024-09-12 02:54:36.526342273 +0000 UTC m=+0.346457693 container start 05ceb6fdd0c14208962f51656693be07d7af73ae03f3c33494b5f138fd346600 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T02:54:36.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: set uid:gid to 167:167 (ceph:ceph) 2024-09-12T02:54:36.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-12T02:54:36.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: pidfile_write: ignore empty --pid-file 2024-09-12T02:54:36.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:36 smithi116 ceph-mon[31410]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:36.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:36 smithi116 ceph-mon[31410]: from='client.? 172.21.15.183:0/2923814928' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:54:36.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: load: jerasure load: lrc 2024-09-12T02:54:36.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: RocksDB version: 7.9.2 2024-09-12T02:54:36.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Git sha 0 2024-09-12T02:54:36.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-12T02:54:36.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: DB SUMMARY 2024-09-12T02:54:36.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: DB Session ID: W545MSLPITF5VI7T2H4U 2024-09-12T02:54:36.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: CURRENT file: CURRENT 2024-09-12T02:54:36.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: IDENTITY file: IDENTITY 2024-09-12T02:54:36.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-12T02:54:36.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-12T02:54:36.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-12T02:54:36.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.error_if_exists: 0 2024-09-12T02:54:36.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.create_if_missing: 0 2024-09-12T02:54:36.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.paranoid_checks: 1 2024-09-12T02:54:36.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-12T02:54:36.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-12T02:54:36.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-12T02:54:36.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.env: 0x55df4292e940 2024-09-12T02:54:36.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.fs: PosixFileSystem 2024-09-12T02:54:36.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.info_log: 0x55df441c76e0 2024-09-12T02:54:36.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.max_file_opening_threads: 16 2024-09-12T02:54:36.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.statistics: (nil) 2024-09-12T02:54:36.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.use_fsync: 0 2024-09-12T02:54:36.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.max_log_file_size: 0 2024-09-12T02:54:36.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-12T02:54:36.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-12T02:54:36.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.keep_log_file_num: 1000 2024-09-12T02:54:36.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.recycle_log_file_num: 0 2024-09-12T02:54:36.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.allow_fallocate: 1 2024-09-12T02:54:36.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.allow_mmap_reads: 0 2024-09-12T02:54:36.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.allow_mmap_writes: 0 2024-09-12T02:54:36.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.use_direct_reads: 0 2024-09-12T02:54:36.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-12T02:54:36.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.create_missing_column_families: 0 2024-09-12T02:54:36.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.db_log_dir: 2024-09-12T02:54:36.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.wal_dir: 2024-09-12T02:54:36.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-12T02:54:36.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-12T02:54:36.957 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-12T02:54:36.957 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-12T02:54:36.957 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-12T02:54:36.957 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-12T02:54:36.957 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.advise_random_on_open: 1 2024-09-12T02:54:36.957 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.db_write_buffer_size: 0 2024-09-12T02:54:36.957 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.write_buffer_manager: 0x55df441d6dc0 2024-09-12T02:54:36.957 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-12T02:54:36.957 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-12T02:54:36.957 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-12T02:54:36.957 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.rate_limiter: (nil) 2024-09-12T02:54:36.957 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-12T02:54:36.957 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.wal_recovery_mode: 2 2024-09-12T02:54:36.957 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.enable_thread_tracking: 0 2024-09-12T02:54:36.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.enable_pipelined_write: 0 2024-09-12T02:54:36.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.unordered_write: 0 2024-09-12T02:54:36.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-12T02:54:36.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-12T02:54:36.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-12T02:54:36.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-12T02:54:36.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.row_cache: None 2024-09-12T02:54:36.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.wal_filter: None 2024-09-12T02:54:36.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-12T02:54:36.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.allow_ingest_behind: 0 2024-09-12T02:54:36.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.two_write_queues: 0 2024-09-12T02:54:36.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.manual_wal_flush: 0 2024-09-12T02:54:36.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.wal_compression: 0 2024-09-12T02:54:36.958 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.atomic_flush: 0 2024-09-12T02:54:36.959 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-12T02:54:36.959 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-12T02:54:36.959 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-12T02:54:36.959 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.log_readahead_size: 0 2024-09-12T02:54:36.959 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-12T02:54:36.959 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.best_efforts_recovery: 0 2024-09-12T02:54:36.959 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-12T02:54:36.959 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-12T02:54:36.959 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.allow_data_in_errors: 0 2024-09-12T02:54:36.959 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.db_host_id: __hostname__ 2024-09-12T02:54:36.959 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-12T02:54:36.959 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.max_background_jobs: 2 2024-09-12T02:54:36.959 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.max_background_compactions: -1 2024-09-12T02:54:36.959 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.max_subcompactions: 1 2024-09-12T02:54:36.961 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-12T02:54:36.961 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-12T02:54:36.961 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-12T02:54:36.961 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.max_total_wal_size: 0 2024-09-12T02:54:36.961 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-12T02:54:36.961 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-12T02:54:36.961 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-12T02:54:36.961 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-12T02:54:36.961 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.max_open_files: -1 2024-09-12T02:54:36.962 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.bytes_per_sync: 0 2024-09-12T02:54:36.962 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-12T02:54:36.962 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-12T02:54:36.962 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.compaction_readahead_size: 0 2024-09-12T02:54:36.962 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.max_background_flushes: -1 2024-09-12T02:54:36.962 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Compression algorithms supported: 2024-09-12T02:54:36.962 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: kZSTD supported: 0 2024-09-12T02:54:36.962 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: kXpressCompression supported: 0 2024-09-12T02:54:36.962 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: kBZip2Compression supported: 0 2024-09-12T02:54:36.962 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-12T02:54:36.963 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: kLZ4Compression supported: 1 2024-09-12T02:54:36.963 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: kZlibCompression supported: 1 2024-09-12T02:54:36.963 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: kLZ4HCCompression supported: 1 2024-09-12T02:54:36.963 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: kSnappyCompression supported: 1 2024-09-12T02:54:36.963 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-12T02:54:36.963 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-12T02:54:36.963 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-12T02:54:36.963 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-12T02:54:36.963 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-12T02:54:36.963 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.merge_operator: 2024-09-12T02:54:36.963 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.compaction_filter: None 2024-09-12T02:54:36.963 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.compaction_filter_factory: None 2024-09-12T02:54:36.963 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.sst_partitioner_factory: None 2024-09-12T02:54:36.963 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-12T02:54:36.964 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-12T02:54:36.964 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55df441c7320) 2024-09-12T02:54:36.964 INFO:journalctl@ceph.mon.c.smithi183.stdout: cache_index_and_filter_blocks: 1 2024-09-12T02:54:36.964 INFO:journalctl@ceph.mon.c.smithi183.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-12T02:54:36.964 INFO:journalctl@ceph.mon.c.smithi183.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-12T02:54:36.964 INFO:journalctl@ceph.mon.c.smithi183.stdout: pin_top_level_index_and_filter: 1 2024-09-12T02:54:36.964 INFO:journalctl@ceph.mon.c.smithi183.stdout: index_type: 0 2024-09-12T02:54:36.964 INFO:journalctl@ceph.mon.c.smithi183.stdout: data_block_index_type: 0 2024-09-12T02:54:36.964 INFO:journalctl@ceph.mon.c.smithi183.stdout: index_shortening: 1 2024-09-12T02:54:36.964 INFO:journalctl@ceph.mon.c.smithi183.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-12T02:54:36.964 INFO:journalctl@ceph.mon.c.smithi183.stdout: checksum: 4 2024-09-12T02:54:36.964 INFO:journalctl@ceph.mon.c.smithi183.stdout: no_block_cache: 0 2024-09-12T02:54:36.964 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_cache: 0x55df441bb1f0 2024-09-12T02:54:36.965 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_cache_name: BinnedLRUCache 2024-09-12T02:54:36.965 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_cache_options: 2024-09-12T02:54:36.965 INFO:journalctl@ceph.mon.c.smithi183.stdout: capacity : 536870912 2024-09-12T02:54:36.965 INFO:journalctl@ceph.mon.c.smithi183.stdout: num_shard_bits : 4 2024-09-12T02:54:36.965 INFO:journalctl@ceph.mon.c.smithi183.stdout: strict_capacity_limit : 0 2024-09-12T02:54:36.965 INFO:journalctl@ceph.mon.c.smithi183.stdout: high_pri_pool_ratio: 0.000 2024-09-12T02:54:36.965 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_cache_compressed: (nil) 2024-09-12T02:54:36.965 INFO:journalctl@ceph.mon.c.smithi183.stdout: persistent_cache: (nil) 2024-09-12T02:54:36.965 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_size: 4096 2024-09-12T02:54:36.965 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_size_deviation: 10 2024-09-12T02:54:36.965 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_restart_interval: 16 2024-09-12T02:54:36.965 INFO:journalctl@ceph.mon.c.smithi183.stdout: index_block_restart_interval: 1 2024-09-12T02:54:36.965 INFO:journalctl@ceph.mon.c.smithi183.stdout: metadata_block_size: 4096 2024-09-12T02:54:36.965 INFO:journalctl@ceph.mon.c.smithi183.stdout: partition_filters: 0 2024-09-12T02:54:36.966 INFO:journalctl@ceph.mon.c.smithi183.stdout: use_delta_encoding: 1 2024-09-12T02:54:36.966 INFO:journalctl@ceph.mon.c.smithi183.stdout: filter_policy: bloomfilter 2024-09-12T02:54:36.966 INFO:journalctl@ceph.mon.c.smithi183.stdout: whole_key_filtering: 1 2024-09-12T02:54:36.966 INFO:journalctl@ceph.mon.c.smithi183.stdout: verify_compression: 0 2024-09-12T02:54:36.966 INFO:journalctl@ceph.mon.c.smithi183.stdout: read_amp_bytes_per_bit: 0 2024-09-12T02:54:36.966 INFO:journalctl@ceph.mon.c.smithi183.stdout: format_version: 5 2024-09-12T02:54:36.966 INFO:journalctl@ceph.mon.c.smithi183.stdout: enable_index_compression: 1 2024-09-12T02:54:36.966 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_align: 0 2024-09-12T02:54:36.966 INFO:journalctl@ceph.mon.c.smithi183.stdout: max_auto_readahead_size: 262144 2024-09-12T02:54:36.966 INFO:journalctl@ceph.mon.c.smithi183.stdout: prepopulate_block_cache: 0 2024-09-12T02:54:36.966 INFO:journalctl@ceph.mon.c.smithi183.stdout: initial_auto_readahead_size: 8192 2024-09-12T02:54:36.966 INFO:journalctl@ceph.mon.c.smithi183.stdout: num_file_reads_for_auto_readahead: 2 2024-09-12T02:54:36.966 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.write_buffer_size: 33554432 2024-09-12T02:54:36.966 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.max_write_buffer_number: 2 2024-09-12T02:54:36.967 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.compression: NoCompression 2024-09-12T02:54:36.967 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.bottommost_compression: Disabled 2024-09-12T02:54:36.967 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.prefix_extractor: nullptr 2024-09-12T02:54:36.967 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-12T02:54:36.967 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.num_levels: 7 2024-09-12T02:54:36.967 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-12T02:54:36.967 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-12T02:54:36.967 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-12T02:54:36.967 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-12T02:54:36.967 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-12T02:54:36.967 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-12T02:54:36.967 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-12T02:54:36.967 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-12T02:54:36.967 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-12T02:54:36.968 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-12T02:54:36.968 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-12T02:54:36.968 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-12T02:54:36.968 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-12T02:54:36.968 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.compression_opts.level: 32767 2024-09-12T02:54:36.968 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.compression_opts.strategy: 0 2024-09-12T02:54:36.968 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-12T02:54:36.968 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-12T02:54:36.968 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-12T02:54:36.968 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-12T02:54:36.968 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.compression_opts.enabled: false 2024-09-12T02:54:36.968 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-12T02:54:36.968 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-12T02:54:36.969 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-12T02:54:36.969 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-12T02:54:36.969 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.target_file_size_base: 67108864 2024-09-12T02:54:36.969 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-12T02:54:36.969 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-12T02:54:36.969 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-12T02:54:36.969 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-12T02:54:36.969 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-12T02:54:36.969 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-12T02:54:36.969 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-12T02:54:36.969 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-12T02:54:36.969 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-12T02:54:36.969 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-12T02:54:36.969 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-12T02:54:36.970 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-12T02:54:36.970 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-12T02:54:36.970 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-12T02:54:36.970 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.arena_block_size: 1048576 2024-09-12T02:54:36.970 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-12T02:54:36.970 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-12T02:54:36.970 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.disable_auto_compactions: 0 2024-09-12T02:54:36.970 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-12T02:54:36.970 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-12T02:54:36.970 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-12T02:54:36.970 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-12T02:54:36.970 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-12T02:54:36.970 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-12T02:54:36.970 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-12T02:54:36.971 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-12T02:54:36.971 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-12T02:54:36.971 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-12T02:54:36.971 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-12T02:54:36.971 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.inplace_update_support: 0 2024-09-12T02:54:36.971 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-12T02:54:36.971 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-12T02:54:36.971 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-12T02:54:36.971 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-12T02:54:36.971 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.bloom_locality: 0 2024-09-12T02:54:36.971 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.max_successive_merges: 0 2024-09-12T02:54:36.971 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-12T02:54:36.971 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.paranoid_file_checks: 0 2024-09-12T02:54:36.971 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.force_consistency_checks: 1 2024-09-12T02:54:36.972 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.report_bg_io_stats: 0 2024-09-12T02:54:36.972 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.ttl: 2592000 2024-09-12T02:54:36.972 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-12T02:54:36.972 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-12T02:54:36.972 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-12T02:54:36.972 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.enable_blob_files: false 2024-09-12T02:54:36.972 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.min_blob_size: 0 2024-09-12T02:54:36.972 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.blob_file_size: 268435456 2024-09-12T02:54:36.972 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-12T02:54:36.972 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-12T02:54:36.972 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-12T02:54:36.972 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-12T02:54:36.972 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-12T02:54:36.972 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.blob_file_starting_level: 0 2024-09-12T02:54:36.973 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-12T02:54:36.973 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-12T02:54:36.973 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-12T02:54:36.973 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a474c2a7-ebb9-411d-a76e-a097ad48cc6c 2024-09-12T02:54:36.973 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726109676553336, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-12T02:54:36.973 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-12T02:54:36.973 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726109676553737, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726109676, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a474c2a7-ebb9-411d-a76e-a097ad48cc6c", "db_session_id": "W545MSLPITF5VI7T2H4U", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-12T02:54:36.973 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726109676553828, "job": 1, "event": "recovery_finished"} 2024-09-12T02:54:36.973 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-12T02:54:36.973 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-12T02:54:36.973 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55df441e8e00 2024-09-12T02:54:36.973 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: DB pointer 0x55df442e4000 2024-09-12T02:54:36.973 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-12T02:54:36.973 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-12T02:54:36.974 INFO:journalctl@ceph.mon.c.smithi183.stdout: ** DB Stats ** 2024-09-12T02:54:36.974 INFO:journalctl@ceph.mon.c.smithi183.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T02:54:36.974 INFO:journalctl@ceph.mon.c.smithi183.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-12T02:54:36.974 INFO:journalctl@ceph.mon.c.smithi183.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T02:54:36.974 INFO:journalctl@ceph.mon.c.smithi183.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T02:54:36.974 INFO:journalctl@ceph.mon.c.smithi183.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-12T02:54:36.974 INFO:journalctl@ceph.mon.c.smithi183.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T02:54:36.974 INFO:journalctl@ceph.mon.c.smithi183.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T02:54:36.974 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-09-12T02:54:36.974 INFO:journalctl@ceph.mon.c.smithi183.stdout: ** Compaction Stats [default] ** 2024-09-12T02:54:36.974 INFO:journalctl@ceph.mon.c.smithi183.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T02:54:36.974 INFO:journalctl@ceph.mon.c.smithi183.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-12T02:54:36.974 INFO:journalctl@ceph.mon.c.smithi183.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T02:54:36.975 INFO:journalctl@ceph.mon.c.smithi183.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T02:54:36.975 INFO:journalctl@ceph.mon.c.smithi183.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T02:54:36.975 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-09-12T02:54:36.975 INFO:journalctl@ceph.mon.c.smithi183.stdout: ** Compaction Stats [default] ** 2024-09-12T02:54:36.975 INFO:journalctl@ceph.mon.c.smithi183.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T02:54:36.975 INFO:journalctl@ceph.mon.c.smithi183.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-12T02:54:36.975 INFO:journalctl@ceph.mon.c.smithi183.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T02:54:36.975 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-09-12T02:54:36.975 INFO:journalctl@ceph.mon.c.smithi183.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-12T02:54:36.975 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-09-12T02:54:36.975 INFO:journalctl@ceph.mon.c.smithi183.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T02:54:36.975 INFO:journalctl@ceph.mon.c.smithi183.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-12T02:54:36.975 INFO:journalctl@ceph.mon.c.smithi183.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-12T02:54:36.976 INFO:journalctl@ceph.mon.c.smithi183.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-12T02:54:36.976 INFO:journalctl@ceph.mon.c.smithi183.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-12T02:54:36.976 INFO:journalctl@ceph.mon.c.smithi183.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-12T02:54:36.976 INFO:journalctl@ceph.mon.c.smithi183.stdout: Cumulative compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T02:54:36.976 INFO:journalctl@ceph.mon.c.smithi183.stdout: Interval compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T02:54:36.976 INFO:journalctl@ceph.mon.c.smithi183.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-12T02:54:36.976 INFO:journalctl@ceph.mon.c.smithi183.stdout: Block cache BinnedLRUCache@0x55df441bb1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-12T02:54:36.976 INFO:journalctl@ceph.mon.c.smithi183.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-12T02:54:36.976 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-09-12T02:54:36.976 INFO:journalctl@ceph.mon.c.smithi183.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-12T02:54:36.976 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-12T02:54:36.976 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: using public_addr v2:172.21.15.183:0/0 -> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] 2024-09-12T02:54:36.976 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: starting mon.c rank -1 at public addrs [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] at bind addrs [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T02:54:36.976 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: mon.c@-1(???) e0 preinit fsid eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T02:54:36.977 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: mon.c@-1(synchronizing).mds e1 new map 2024-09-12T02:54:36.977 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-12T02:54:36.977 INFO:journalctl@ceph.mon.c.smithi183.stdout: e1 2024-09-12T02:54:36.977 INFO:journalctl@ceph.mon.c.smithi183.stdout: btime 2024-09-12T02:53:07:105500+0000 2024-09-12T02:54:36.977 INFO:journalctl@ceph.mon.c.smithi183.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T02:54:36.977 INFO:journalctl@ceph.mon.c.smithi183.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T02:54:36.977 INFO:journalctl@ceph.mon.c.smithi183.stdout: legacy client fscid: -1 2024-09-12T02:54:36.977 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-09-12T02:54:36.977 INFO:journalctl@ceph.mon.c.smithi183.stdout: No filesystems configured 2024-09-12T02:54:36.977 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-12T02:54:36.977 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-12T02:54:36.977 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-12T02:54:36.977 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-12T02:54:36.978 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-12T02:54:36.978 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-12T02:54:36.978 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T02:54:36.978 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:54:36.978 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:54:36.978 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:54:36.978 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: Updating smithi177:/var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:54:36.978 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: Updating smithi177:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:54:36.978 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: Updating smithi177:/var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:54:36.978 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:36.979 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:36.979 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:36.979 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:36.979 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:36.979 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: Added host smithi183 2024-09-12T02:54:36.979 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:36.979 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:36.979 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:36.979 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:36.979 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:36.979 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:54:36.979 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:36.979 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1102187658' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T02:54:36.980 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1102187658' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T02:54:36.980 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:54:36.980 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:36.980 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:36.980 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:36.980 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:36.980 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:54:36.980 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:36.980 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:54:36.980 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-12T02:54:36.980 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: Updating smithi183:/var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:54:36.980 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:54:36.980 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:36.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: Updating smithi183:/var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:54:36.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:36.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:36.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:36.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:36.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:36.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:36.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:54:36.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:36.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:54:36.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:36.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi116:172.21.15.116=a;smithi177:172.21.15.177=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:54:36.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: Saving service mon spec with placement smithi116:172.21.15.116=a;smithi177:172.21.15.177=b;smithi183:172.21.15.183=c;count:3 2024-09-12T02:54:36.981 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: Deploying daemon mon.c on smithi183 2024-09-12T02:54:36.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:36.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:36.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: from='client.? 172.21.15.183:0/2923814928' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:54:36.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 ceph-mon[35535]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-12T02:54:36.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 bash[35516]: 05ceb6fdd0c14208962f51656693be07d7af73ae03f3c33494b5f138fd346600 2024-09-12T02:54:36.982 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:36 smithi183 systemd[1]: Started Ceph mon.c for eb1ac602-70b1-11ef-bcea-c7b262605968. 2024-09-12T02:54:37.049 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T02:54:37.050 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T02:54:37.298 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:54:41.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:54:41.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:54:41.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:41 smithi183 ceph-mon[35535]: mon.a calling monitor election 2024-09-12T02:54:41.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:41 smithi183 ceph-mon[35535]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:41.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:54:41.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:54:41.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:41 smithi183 ceph-mon[35535]: mon.c calling monitor election 2024-09-12T02:54:41.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:41 smithi183 ceph-mon[35535]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:41.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:54:41.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:54:41.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:41 smithi183 ceph-mon[35535]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:41.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:54:41.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:41 smithi183 ceph-mon[35535]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T02:54:41.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:41 smithi183 ceph-mon[35535]: monmap epoch 2 2024-09-12T02:54:41.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:41 smithi183 ceph-mon[35535]: fsid eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T02:54:41.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:41 smithi183 ceph-mon[35535]: last_changed 2024-09-12T02:54:36.578645+0000 2024-09-12T02:54:41.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:41 smithi183 ceph-mon[35535]: created 2024-09-12T02:53:04.190821+0000 2024-09-12T02:54:41.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:41 smithi183 ceph-mon[35535]: min_mon_release 19 (squid) 2024-09-12T02:54:41.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:41 smithi183 ceph-mon[35535]: election_strategy: 1 2024-09-12T02:54:41.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:41 smithi183 ceph-mon[35535]: 0: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.a 2024-09-12T02:54:41.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:41 smithi183 ceph-mon[35535]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-09-12T02:54:41.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:41 smithi183 ceph-mon[35535]: fsmap 2024-09-12T02:54:41.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:41 smithi183 ceph-mon[35535]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:54:41.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:41 smithi183 ceph-mon[35535]: mgrmap e13: x(active, since 38s) 2024-09-12T02:54:41.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:41 smithi183 ceph-mon[35535]: overall HEALTH_OK 2024-09-12T02:54:41.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:41.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:41.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:41.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:54:41.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:42.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:54:42.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:54:42.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:41 smithi116 ceph-mon[31410]: mon.a calling monitor election 2024-09-12T02:54:42.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:41 smithi116 ceph-mon[31410]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:42.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:54:42.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:54:42.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:41 smithi116 ceph-mon[31410]: mon.c calling monitor election 2024-09-12T02:54:42.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:41 smithi116 ceph-mon[31410]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:42.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:54:42.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:54:42.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:41 smithi116 ceph-mon[31410]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:42.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:54:42.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:41 smithi116 ceph-mon[31410]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T02:54:42.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:41 smithi116 ceph-mon[31410]: monmap epoch 2 2024-09-12T02:54:42.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:41 smithi116 ceph-mon[31410]: fsid eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T02:54:42.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:41 smithi116 ceph-mon[31410]: last_changed 2024-09-12T02:54:36.578645+0000 2024-09-12T02:54:42.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:41 smithi116 ceph-mon[31410]: created 2024-09-12T02:53:04.190821+0000 2024-09-12T02:54:42.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:41 smithi116 ceph-mon[31410]: min_mon_release 19 (squid) 2024-09-12T02:54:42.055 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:41 smithi116 ceph-mon[31410]: election_strategy: 1 2024-09-12T02:54:42.055 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:41 smithi116 ceph-mon[31410]: 0: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.a 2024-09-12T02:54:42.055 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:41 smithi116 ceph-mon[31410]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-09-12T02:54:42.055 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:41 smithi116 ceph-mon[31410]: fsmap 2024-09-12T02:54:42.055 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:41 smithi116 ceph-mon[31410]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:54:42.055 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:41 smithi116 ceph-mon[31410]: mgrmap e13: x(active, since 38s) 2024-09-12T02:54:42.055 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:41 smithi116 ceph-mon[31410]: overall HEALTH_OK 2024-09-12T02:54:42.055 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:42.055 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:42.055 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:42.055 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:54:42.055 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:42.113 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.111+0000 7f54129b2640 1 -- 172.21.15.183:0/3213268636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f53f4003530 msgr2=0x7f53f40059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:42.113 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.111+0000 7f54129b2640 1 --2- 172.21.15.183:0/3213268636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f53f4003530 0x7f53f40059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f540c1016a0 tx=0x7f53fc02c490 comp rx=0 tx=0).stop 2024-09-12T02:54:42.113 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.111+0000 7f54129b2640 1 -- 172.21.15.183:0/3213268636 shutdown_connections 2024-09-12T02:54:42.113 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.111+0000 7f54129b2640 1 --2- 172.21.15.183:0/3213268636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f53f4003530 0x7f53f40059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:42.113 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.111+0000 7f54129b2640 1 -- 172.21.15.183:0/3213268636 >> 172.21.15.183:0/3213268636 conn(0x7f540c0fc220 msgr2=0x7f540c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:42.114 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.111+0000 7f54129b2640 1 -- 172.21.15.183:0/3213268636 shutdown_connections 2024-09-12T02:54:42.114 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.111+0000 7f54129b2640 1 -- 172.21.15.183:0/3213268636 wait complete. 2024-09-12T02:54:42.114 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.112+0000 7f54129b2640 1 Processor -- start 2024-09-12T02:54:42.114 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.113+0000 7f54129b2640 1 -- start start 2024-09-12T02:54:42.115 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.113+0000 7f54129b2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53f4003530 0x7f540c1021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:42.115 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.113+0000 7f54129b2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f540c102730 0x7f540c102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:42.115 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.113+0000 7f54129b2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f540c108670 con 0x7f540c102730 2024-09-12T02:54:42.115 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.113+0000 7f54129b2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1084f0 con 0x7f53f4003530 2024-09-12T02:54:42.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.113+0000 7f540bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53f4003530 0x7f540c1021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:42.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.113+0000 7f540bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53f4003530 0x7f540c1021f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:33532/0 (socket says 172.21.15.183:33532) 2024-09-12T02:54:42.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.113+0000 7f540bfff640 1 -- 172.21.15.183:0/4207204731 learned_addr learned my addr 172.21.15.183:0/4207204731 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-12T02:54:42.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.113+0000 7f540b7fe640 1 --2- 172.21.15.183:0/4207204731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f540c102730 0x7f540c102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:42.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.114+0000 7f540bfff640 1 -- 172.21.15.183:0/4207204731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53f4003530 msgr2=0x7f540c1021f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-12T02:54:42.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.114+0000 7f540bfff640 1 -- 172.21.15.183:0/4207204731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53f4003530 msgr2=0x7f540c1021f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T02:54:42.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.114+0000 7f540bfff640 1 --2- 172.21.15.183:0/4207204731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53f4003530 0x7f540c1021f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T02:54:42.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.114+0000 7f540bfff640 1 --2- 172.21.15.183:0/4207204731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53f4003530 0x7f540c1021f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:54:42.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.114+0000 7f540b7fe640 1 -- 172.21.15.183:0/4207204731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53f4003530 msgr2=0x7f540c1021f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:54:42.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.114+0000 7f540b7fe640 1 --2- 172.21.15.183:0/4207204731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53f4003530 0x7f540c1021f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:42.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.114+0000 7f540b7fe640 1 -- 172.21.15.183:0/4207204731 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540c1040e0 con 0x7f540c102730 2024-09-12T02:54:42.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.114+0000 7f540b7fe640 1 --2- 172.21.15.183:0/4207204731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f540c102730 0x7f540c102b90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f540000b400 tx=0x7f540000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:42.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.114+0000 7f54097fa640 1 -- 172.21.15.183:0/4207204731 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f5400011630 con 0x7f540c102730 2024-09-12T02:54:42.117 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.114+0000 7f54097fa640 1 -- 172.21.15.183:0/4207204731 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5400011cb0 con 0x7f540c102730 2024-09-12T02:54:42.117 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.114+0000 7f54097fa640 1 -- 172.21.15.183:0/4207204731 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f5400010d80 con 0x7f540c102730 2024-09-12T02:54:42.117 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.114+0000 7f54129b2640 1 -- 172.21.15.183:0/4207204731 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540c1a3520 con 0x7f540c102730 2024-09-12T02:54:42.117 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.114+0000 7f54129b2640 1 -- 172.21.15.183:0/4207204731 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f540c1a3ab0 con 0x7f540c102730 2024-09-12T02:54:42.117 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.115+0000 7f54097fa640 1 -- 172.21.15.183:0/4207204731 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f5400018020 con 0x7f540c102730 2024-09-12T02:54:42.118 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.116+0000 7f54097fa640 1 --2- 172.21.15.183:0/4207204731 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f53f8038e50 0x7f53f803b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:42.118 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.116+0000 7f54097fa640 1 -- 172.21.15.183:0/4207204731 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f540004bc90 con 0x7f540c102730 2024-09-12T02:54:42.118 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.116+0000 7f54129b2640 1 -- 172.21.15.183:0/4207204731 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f540c101ca0 con 0x7f540c102730 2024-09-12T02:54:42.118 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.116+0000 7f540bfff640 1 --2- 172.21.15.183:0/4207204731 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f53f8038e50 0x7f53f803b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:42.118 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.117+0000 7f540bfff640 1 --2- 172.21.15.183:0/4207204731 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f53f8038e50 0x7f53f803b310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0129e0 tx=0x7f53fc02c9d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:42.122 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.121+0000 7f54097fa640 1 -- 172.21.15.183:0/4207204731 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f540001f020 con 0x7f540c102730 2024-09-12T02:54:42.353 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.352+0000 7f54129b2640 1 -- 172.21.15.183:0/4207204731 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f540c1a3dc0 con 0x7f540c102730 2024-09-12T02:54:42.354 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.353+0000 7f54097fa640 1 -- 172.21.15.183:0/4207204731 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f5400018310 con 0x7f540c102730 2024-09-12T02:54:42.354 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T02:54:42.354 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":2,"fsid":"eb1ac602-70b1-11ef-bcea-c7b262605968","modified":"2024-09-12T02:54:36.578645Z","created":"2024-09-12T02:53:04.190821Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:3300","nonce":0},{"type":"v1","addr":"172.21.15.116:6789","nonce":0}]},"addr":"172.21.15.116:6789/0","public_addr":"172.21.15.116:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T02:54:42.354 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 2 2024-09-12T02:54:42.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.355+0000 7f54129b2640 1 -- 172.21.15.183:0/4207204731 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f53f8038e50 msgr2=0x7f53f803b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:42.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.355+0000 7f54129b2640 1 --2- 172.21.15.183:0/4207204731 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f53f8038e50 0x7f53f803b310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0129e0 tx=0x7f53fc02c9d0 comp rx=0 tx=0).stop 2024-09-12T02:54:42.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.355+0000 7f54129b2640 1 -- 172.21.15.183:0/4207204731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f540c102730 msgr2=0x7f540c102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:42.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.355+0000 7f54129b2640 1 --2- 172.21.15.183:0/4207204731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f540c102730 0x7f540c102b90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f540000b400 tx=0x7f540000b8c0 comp rx=0 tx=0).stop 2024-09-12T02:54:42.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.355+0000 7f54129b2640 1 -- 172.21.15.183:0/4207204731 shutdown_connections 2024-09-12T02:54:42.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.355+0000 7f54129b2640 1 --2- 172.21.15.183:0/4207204731 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f53f8038e50 0x7f53f803b310 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:42.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.355+0000 7f54129b2640 1 --2- 172.21.15.183:0/4207204731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f540c102730 0x7f540c102b90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:42.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.355+0000 7f54129b2640 1 --2- 172.21.15.183:0/4207204731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53f4003530 0x7f540c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:42.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.355+0000 7f54129b2640 1 -- 172.21.15.183:0/4207204731 >> 172.21.15.183:0/4207204731 conn(0x7f540c0fc220 msgr2=0x7f540c06a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:42.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.355+0000 7f54129b2640 1 -- 172.21.15.183:0/4207204731 shutdown_connections 2024-09-12T02:54:42.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:42.355+0000 7f54129b2640 1 -- 172.21.15.183:0/4207204731 wait complete. 2024-09-12T02:54:42.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:42 smithi183 ceph-mon[35535]: Deploying daemon mon.b on smithi177 2024-09-12T02:54:42.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:42 smithi183 ceph-mon[35535]: from='client.? 172.21.15.183:0/4207204731' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:54:42.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:42 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:54:43.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:42 smithi116 ceph-mon[31410]: Deploying daemon mon.b on smithi177 2024-09-12T02:54:43.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:42 smithi116 ceph-mon[31410]: from='client.? 172.21.15.183:0/4207204731' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:54:43.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:42 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:54:43.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:43 smithi183 ceph-mon[35535]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:43.985 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T02:54:43.985 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T02:54:44.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:43 smithi116 ceph-mon[31410]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:44.240 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:54:44.711 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.643+0000 7f804e82a640 1 -- 172.21.15.183:0/818267149 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8048106660 msgr2=0x7f8048106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:44.711 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.643+0000 7f804e82a640 1 --2- 172.21.15.183:0/818267149 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8048106660 0x7f8048106a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8038009880 tx=0x7f803802a5d0 comp rx=0 tx=0).stop 2024-09-12T02:54:44.711 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.643+0000 7f804e82a640 1 -- 172.21.15.183:0/818267149 shutdown_connections 2024-09-12T02:54:44.711 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.643+0000 7f804e82a640 1 --2- 172.21.15.183:0/818267149 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8048106660 0x7f8048106a40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:44.711 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.643+0000 7f804e82a640 1 -- 172.21.15.183:0/818267149 >> 172.21.15.183:0/818267149 conn(0x7f80480fc250 msgr2=0x7f80480fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:44.711 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.643+0000 7f804e82a640 1 -- 172.21.15.183:0/818267149 shutdown_connections 2024-09-12T02:54:44.712 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.643+0000 7f804e82a640 1 -- 172.21.15.183:0/818267149 wait complete. 2024-09-12T02:54:44.712 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.644+0000 7f804e82a640 1 Processor -- start 2024-09-12T02:54:44.712 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.644+0000 7f804e82a640 1 -- start start 2024-09-12T02:54:44.712 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.644+0000 7f804e82a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8048106660 0x7f80481962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:44.712 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.644+0000 7f804e82a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8048196830 0x7f804819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:44.712 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.644+0000 7f804e82a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f804810ac90 con 0x7f8048106660 2024-09-12T02:54:44.712 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.644+0000 7f804e82a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f804810ab10 con 0x7f8048196830 2024-09-12T02:54:44.712 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.645+0000 7f80477fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8048196830 0x7f804819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:44.712 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.645+0000 7f80477fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8048196830 0x7f804819abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:33548/0 (socket says 172.21.15.183:33548) 2024-09-12T02:54:44.712 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.645+0000 7f8047fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8048106660 0x7f80481962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:44.712 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.645+0000 7f80477fe640 1 -- 172.21.15.183:0/4260026994 learned_addr learned my addr 172.21.15.183:0/4260026994 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-12T02:54:44.712 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.645+0000 7f80477fe640 1 -- 172.21.15.183:0/4260026994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8048196830 msgr2=0x7f804819abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-12T02:54:44.713 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.645+0000 7f80477fe640 1 -- 172.21.15.183:0/4260026994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8048196830 msgr2=0x7f804819abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T02:54:44.713 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.645+0000 7f80477fe640 1 --2- 172.21.15.183:0/4260026994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8048196830 0x7f804819abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T02:54:44.713 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.645+0000 7f80477fe640 1 --2- 172.21.15.183:0/4260026994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8048196830 0x7f804819abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:54:44.713 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.645+0000 7f8047fff640 1 -- 172.21.15.183:0/4260026994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8048196830 msgr2=0x7f804819abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:54:44.713 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.645+0000 7f8047fff640 1 --2- 172.21.15.183:0/4260026994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8048196830 0x7f804819abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:44.713 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.645+0000 7f8047fff640 1 -- 172.21.15.183:0/4260026994 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8048196e50 con 0x7f8048106660 2024-09-12T02:54:44.713 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.645+0000 7f8047fff640 1 --2- 172.21.15.183:0/4260026994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8048106660 0x7f80481962f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8038005ab0 tx=0x7f8038032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:44.713 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.646+0000 7f80457fa640 1 -- 172.21.15.183:0/4260026994 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8038035070 con 0x7f8048106660 2024-09-12T02:54:44.713 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.646+0000 7f804e82a640 1 -- 172.21.15.183:0/4260026994 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f804819b100 con 0x7f8048106660 2024-09-12T02:54:44.713 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.646+0000 7f80457fa640 1 -- 172.21.15.183:0/4260026994 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8038032e40 con 0x7f8048106660 2024-09-12T02:54:44.713 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.646+0000 7f80457fa640 1 -- 172.21.15.183:0/4260026994 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f803803bc10 con 0x7f8048106660 2024-09-12T02:54:44.713 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.646+0000 7f804e82a640 1 -- 172.21.15.183:0/4260026994 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f804819b660 con 0x7f8048106660 2024-09-12T02:54:44.714 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.647+0000 7f80457fa640 1 -- 172.21.15.183:0/4260026994 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f8038042030 con 0x7f8048106660 2024-09-12T02:54:44.714 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.647+0000 7f804e82a640 1 -- 172.21.15.183:0/4260026994 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8048101cd0 con 0x7f8048106660 2024-09-12T02:54:44.714 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.647+0000 7f80457fa640 1 --2- 172.21.15.183:0/4260026994 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8030038e00 0x7f803003b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:44.714 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.647+0000 7f80457fa640 1 -- 172.21.15.183:0/4260026994 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8038040070 con 0x7f8048106660 2024-09-12T02:54:44.714 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.647+0000 7f80477fe640 1 --2- 172.21.15.183:0/4260026994 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8030038e00 0x7f803003b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:44.714 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.647+0000 7f80477fe640 1 --2- 172.21.15.183:0/4260026994 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8030038e00 0x7f803003b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f803c006a90 tx=0x7f803c006a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:44.714 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.652+0000 7f80457fa640 1 -- 172.21.15.183:0/4260026994 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f803803b510 con 0x7f8048106660 2024-09-12T02:54:44.886 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.885+0000 7f804e82a640 1 -- 172.21.15.183:0/4260026994 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f80480008d0 con 0x7f8048106660 2024-09-12T02:54:44.887 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.886+0000 7f80457fa640 1 -- 172.21.15.183:0/4260026994 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f803803bdb0 con 0x7f8048106660 2024-09-12T02:54:44.887 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T02:54:44.887 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":2,"fsid":"eb1ac602-70b1-11ef-bcea-c7b262605968","modified":"2024-09-12T02:54:36.578645Z","created":"2024-09-12T02:53:04.190821Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:3300","nonce":0},{"type":"v1","addr":"172.21.15.116:6789","nonce":0}]},"addr":"172.21.15.116:6789/0","public_addr":"172.21.15.116:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T02:54:44.887 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 2 2024-09-12T02:54:44.889 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.888+0000 7f804e82a640 1 -- 172.21.15.183:0/4260026994 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8030038e00 msgr2=0x7f803003b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:44.889 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.888+0000 7f804e82a640 1 --2- 172.21.15.183:0/4260026994 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8030038e00 0x7f803003b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f803c006a90 tx=0x7f803c006a20 comp rx=0 tx=0).stop 2024-09-12T02:54:44.889 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.888+0000 7f804e82a640 1 -- 172.21.15.183:0/4260026994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8048106660 msgr2=0x7f80481962f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:44.889 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.888+0000 7f804e82a640 1 --2- 172.21.15.183:0/4260026994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8048106660 0x7f80481962f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8038005ab0 tx=0x7f8038032c50 comp rx=0 tx=0).stop 2024-09-12T02:54:44.889 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.888+0000 7f804e82a640 1 -- 172.21.15.183:0/4260026994 shutdown_connections 2024-09-12T02:54:44.890 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.888+0000 7f804e82a640 1 --2- 172.21.15.183:0/4260026994 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8030038e00 0x7f803003b2c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:44.890 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.888+0000 7f804e82a640 1 --2- 172.21.15.183:0/4260026994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8048196830 0x7f804819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:44.890 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.888+0000 7f804e82a640 1 --2- 172.21.15.183:0/4260026994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8048106660 0x7f80481962f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:44.890 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.888+0000 7f804e82a640 1 -- 172.21.15.183:0/4260026994 >> 172.21.15.183:0/4260026994 conn(0x7f80480fc250 msgr2=0x7f80480fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:44.890 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.888+0000 7f804e82a640 1 -- 172.21.15.183:0/4260026994 shutdown_connections 2024-09-12T02:54:44.890 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:44.888+0000 7f804e82a640 1 -- 172.21.15.183:0/4260026994 wait complete. 2024-09-12T02:54:45.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:44 smithi183 ceph-mon[35535]: from='client.? 172.21.15.183:0/4260026994' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:54:45.235 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: load: jerasure load: lrc 2024-09-12T02:54:45.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:44 smithi116 ceph-mon[31410]: from='client.? 172.21.15.183:0/4260026994' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:54:45.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: RocksDB version: 7.9.2 2024-09-12T02:54:45.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Git sha 0 2024-09-12T02:54:45.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-12T02:54:45.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: DB SUMMARY 2024-09-12T02:54:45.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: DB Session ID: THNTX7Q0BXX7H87APJQI 2024-09-12T02:54:45.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: CURRENT file: CURRENT 2024-09-12T02:54:45.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: IDENTITY file: IDENTITY 2024-09-12T02:54:45.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-12T02:54:45.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-12T02:54:45.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-12T02:54:45.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.error_if_exists: 0 2024-09-12T02:54:45.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.create_if_missing: 0 2024-09-12T02:54:45.497 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.paranoid_checks: 1 2024-09-12T02:54:45.497 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-12T02:54:45.497 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-12T02:54:45.497 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-12T02:54:45.497 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.env: 0x5615f7484940 2024-09-12T02:54:45.497 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.fs: PosixFileSystem 2024-09-12T02:54:45.497 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.info_log: 0x5615f97356e0 2024-09-12T02:54:45.497 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.max_file_opening_threads: 16 2024-09-12T02:54:45.497 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.statistics: (nil) 2024-09-12T02:54:45.497 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.use_fsync: 0 2024-09-12T02:54:45.497 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.max_log_file_size: 0 2024-09-12T02:54:45.497 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-12T02:54:45.497 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-12T02:54:45.498 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.keep_log_file_num: 1000 2024-09-12T02:54:45.498 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.recycle_log_file_num: 0 2024-09-12T02:54:45.498 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.allow_fallocate: 1 2024-09-12T02:54:45.498 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.allow_mmap_reads: 0 2024-09-12T02:54:45.498 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.allow_mmap_writes: 0 2024-09-12T02:54:45.498 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.use_direct_reads: 0 2024-09-12T02:54:45.498 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-12T02:54:45.498 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.create_missing_column_families: 0 2024-09-12T02:54:45.498 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.db_log_dir: 2024-09-12T02:54:45.498 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.wal_dir: 2024-09-12T02:54:45.498 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-12T02:54:45.498 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-12T02:54:45.498 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-12T02:54:45.498 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-12T02:54:45.500 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-12T02:54:45.500 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-12T02:54:45.500 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.advise_random_on_open: 1 2024-09-12T02:54:45.500 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.db_write_buffer_size: 0 2024-09-12T02:54:45.500 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.write_buffer_manager: 0x5615f9744dc0 2024-09-12T02:54:45.500 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-12T02:54:45.500 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-12T02:54:45.500 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-12T02:54:45.500 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.rate_limiter: (nil) 2024-09-12T02:54:45.500 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-12T02:54:45.501 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.wal_recovery_mode: 2 2024-09-12T02:54:45.501 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.enable_thread_tracking: 0 2024-09-12T02:54:45.501 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.enable_pipelined_write: 0 2024-09-12T02:54:45.501 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.unordered_write: 0 2024-09-12T02:54:45.501 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-12T02:54:45.501 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-12T02:54:45.501 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-12T02:54:45.501 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-12T02:54:45.501 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.row_cache: None 2024-09-12T02:54:45.501 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.wal_filter: None 2024-09-12T02:54:45.501 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-12T02:54:45.501 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.allow_ingest_behind: 0 2024-09-12T02:54:45.502 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.two_write_queues: 0 2024-09-12T02:54:45.502 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.manual_wal_flush: 0 2024-09-12T02:54:45.502 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.wal_compression: 0 2024-09-12T02:54:45.502 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.atomic_flush: 0 2024-09-12T02:54:45.502 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-12T02:54:45.502 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-12T02:54:45.502 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-12T02:54:45.502 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.log_readahead_size: 0 2024-09-12T02:54:45.502 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-12T02:54:45.502 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.best_efforts_recovery: 0 2024-09-12T02:54:45.502 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-12T02:54:45.502 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-12T02:54:45.502 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.allow_data_in_errors: 0 2024-09-12T02:54:45.502 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.db_host_id: __hostname__ 2024-09-12T02:54:45.502 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-12T02:54:45.503 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.max_background_jobs: 2 2024-09-12T02:54:45.503 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.max_background_compactions: -1 2024-09-12T02:54:45.503 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.max_subcompactions: 1 2024-09-12T02:54:45.503 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-12T02:54:45.503 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-12T02:54:45.503 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-12T02:54:45.503 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.max_total_wal_size: 0 2024-09-12T02:54:45.503 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-12T02:54:45.503 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-12T02:54:45.503 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-12T02:54:45.503 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-12T02:54:45.503 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.max_open_files: -1 2024-09-12T02:54:45.503 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.bytes_per_sync: 0 2024-09-12T02:54:45.503 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-12T02:54:45.504 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-12T02:54:45.504 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.compaction_readahead_size: 0 2024-09-12T02:54:45.504 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.max_background_flushes: -1 2024-09-12T02:54:45.504 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Compression algorithms supported: 2024-09-12T02:54:45.504 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: kZSTD supported: 0 2024-09-12T02:54:45.504 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: kXpressCompression supported: 0 2024-09-12T02:54:45.504 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: kBZip2Compression supported: 0 2024-09-12T02:54:45.504 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-12T02:54:45.504 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: kLZ4Compression supported: 1 2024-09-12T02:54:45.504 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: kZlibCompression supported: 1 2024-09-12T02:54:45.504 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: kLZ4HCCompression supported: 1 2024-09-12T02:54:45.504 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: kSnappyCompression supported: 1 2024-09-12T02:54:45.504 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-12T02:54:45.504 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-12T02:54:45.505 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-12T02:54:45.505 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-12T02:54:45.505 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-12T02:54:45.505 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.merge_operator: 2024-09-12T02:54:45.505 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.compaction_filter: None 2024-09-12T02:54:45.505 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.compaction_filter_factory: None 2024-09-12T02:54:45.505 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.sst_partitioner_factory: None 2024-09-12T02:54:45.505 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-12T02:54:45.505 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-12T02:54:45.505 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5615f9735320) 2024-09-12T02:54:45.505 INFO:journalctl@ceph.mon.b.smithi177.stdout: cache_index_and_filter_blocks: 1 2024-09-12T02:54:45.506 INFO:journalctl@ceph.mon.b.smithi177.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-12T02:54:45.506 INFO:journalctl@ceph.mon.b.smithi177.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-12T02:54:45.506 INFO:journalctl@ceph.mon.b.smithi177.stdout: pin_top_level_index_and_filter: 1 2024-09-12T02:54:45.506 INFO:journalctl@ceph.mon.b.smithi177.stdout: index_type: 0 2024-09-12T02:54:45.506 INFO:journalctl@ceph.mon.b.smithi177.stdout: data_block_index_type: 0 2024-09-12T02:54:45.506 INFO:journalctl@ceph.mon.b.smithi177.stdout: index_shortening: 1 2024-09-12T02:54:45.506 INFO:journalctl@ceph.mon.b.smithi177.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-12T02:54:45.506 INFO:journalctl@ceph.mon.b.smithi177.stdout: checksum: 4 2024-09-12T02:54:45.506 INFO:journalctl@ceph.mon.b.smithi177.stdout: no_block_cache: 0 2024-09-12T02:54:45.506 INFO:journalctl@ceph.mon.b.smithi177.stdout: block_cache: 0x5615f97291f0 2024-09-12T02:54:45.506 INFO:journalctl@ceph.mon.b.smithi177.stdout: block_cache_name: BinnedLRUCache 2024-09-12T02:54:45.506 INFO:journalctl@ceph.mon.b.smithi177.stdout: block_cache_options: 2024-09-12T02:54:45.507 INFO:journalctl@ceph.mon.b.smithi177.stdout: capacity : 536870912 2024-09-12T02:54:45.507 INFO:journalctl@ceph.mon.b.smithi177.stdout: num_shard_bits : 4 2024-09-12T02:54:45.507 INFO:journalctl@ceph.mon.b.smithi177.stdout: strict_capacity_limit : 0 2024-09-12T02:54:45.507 INFO:journalctl@ceph.mon.b.smithi177.stdout: high_pri_pool_ratio: 0.000 2024-09-12T02:54:45.507 INFO:journalctl@ceph.mon.b.smithi177.stdout: block_cache_compressed: (nil) 2024-09-12T02:54:45.507 INFO:journalctl@ceph.mon.b.smithi177.stdout: persistent_cache: (nil) 2024-09-12T02:54:45.507 INFO:journalctl@ceph.mon.b.smithi177.stdout: block_size: 4096 2024-09-12T02:54:45.507 INFO:journalctl@ceph.mon.b.smithi177.stdout: block_size_deviation: 10 2024-09-12T02:54:45.507 INFO:journalctl@ceph.mon.b.smithi177.stdout: block_restart_interval: 16 2024-09-12T02:54:45.507 INFO:journalctl@ceph.mon.b.smithi177.stdout: index_block_restart_interval: 1 2024-09-12T02:54:45.507 INFO:journalctl@ceph.mon.b.smithi177.stdout: metadata_block_size: 4096 2024-09-12T02:54:45.507 INFO:journalctl@ceph.mon.b.smithi177.stdout: partition_filters: 0 2024-09-12T02:54:45.507 INFO:journalctl@ceph.mon.b.smithi177.stdout: use_delta_encoding: 1 2024-09-12T02:54:45.508 INFO:journalctl@ceph.mon.b.smithi177.stdout: filter_policy: bloomfilter 2024-09-12T02:54:45.508 INFO:journalctl@ceph.mon.b.smithi177.stdout: whole_key_filtering: 1 2024-09-12T02:54:45.508 INFO:journalctl@ceph.mon.b.smithi177.stdout: verify_compression: 0 2024-09-12T02:54:45.508 INFO:journalctl@ceph.mon.b.smithi177.stdout: read_amp_bytes_per_bit: 0 2024-09-12T02:54:45.508 INFO:journalctl@ceph.mon.b.smithi177.stdout: format_version: 5 2024-09-12T02:54:45.508 INFO:journalctl@ceph.mon.b.smithi177.stdout: enable_index_compression: 1 2024-09-12T02:54:45.508 INFO:journalctl@ceph.mon.b.smithi177.stdout: block_align: 0 2024-09-12T02:54:45.508 INFO:journalctl@ceph.mon.b.smithi177.stdout: max_auto_readahead_size: 262144 2024-09-12T02:54:45.508 INFO:journalctl@ceph.mon.b.smithi177.stdout: prepopulate_block_cache: 0 2024-09-12T02:54:45.508 INFO:journalctl@ceph.mon.b.smithi177.stdout: initial_auto_readahead_size: 8192 2024-09-12T02:54:45.508 INFO:journalctl@ceph.mon.b.smithi177.stdout: num_file_reads_for_auto_readahead: 2 2024-09-12T02:54:45.509 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.write_buffer_size: 33554432 2024-09-12T02:54:45.509 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.max_write_buffer_number: 2 2024-09-12T02:54:45.509 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.compression: NoCompression 2024-09-12T02:54:45.509 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.bottommost_compression: Disabled 2024-09-12T02:54:45.509 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.prefix_extractor: nullptr 2024-09-12T02:54:45.509 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-12T02:54:45.509 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.num_levels: 7 2024-09-12T02:54:45.509 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-12T02:54:45.509 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-12T02:54:45.509 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-12T02:54:45.509 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-12T02:54:45.509 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-12T02:54:45.509 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-12T02:54:45.510 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-12T02:54:45.510 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-12T02:54:45.510 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-12T02:54:45.510 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-12T02:54:45.510 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-12T02:54:45.510 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-12T02:54:45.510 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-12T02:54:45.510 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.compression_opts.level: 32767 2024-09-12T02:54:45.510 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.compression_opts.strategy: 0 2024-09-12T02:54:45.510 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-12T02:54:45.510 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-12T02:54:45.511 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-12T02:54:45.511 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-12T02:54:45.511 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.compression_opts.enabled: false 2024-09-12T02:54:45.511 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-12T02:54:45.511 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-12T02:54:45.511 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-12T02:54:45.511 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-12T02:54:45.511 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.target_file_size_base: 67108864 2024-09-12T02:54:45.511 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-12T02:54:45.511 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-12T02:54:45.511 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-12T02:54:45.511 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-12T02:54:45.511 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-12T02:54:45.512 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-12T02:54:45.512 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-12T02:54:45.512 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-12T02:54:45.512 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-12T02:54:45.512 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-12T02:54:45.512 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-12T02:54:45.512 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-12T02:54:45.512 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-12T02:54:45.512 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-12T02:54:45.512 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.arena_block_size: 1048576 2024-09-12T02:54:45.512 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-12T02:54:45.513 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-12T02:54:45.513 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.disable_auto_compactions: 0 2024-09-12T02:54:45.513 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-12T02:54:45.513 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-12T02:54:45.513 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-12T02:54:45.513 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-12T02:54:45.513 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-12T02:54:45.513 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-12T02:54:45.513 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-12T02:54:45.513 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-12T02:54:45.514 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-12T02:54:45.514 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-12T02:54:45.514 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-12T02:54:45.514 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.inplace_update_support: 0 2024-09-12T02:54:45.514 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-12T02:54:45.514 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-12T02:54:45.514 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-12T02:54:45.514 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-12T02:54:45.514 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.bloom_locality: 0 2024-09-12T02:54:45.514 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.max_successive_merges: 0 2024-09-12T02:54:45.514 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-12T02:54:45.514 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.paranoid_file_checks: 0 2024-09-12T02:54:45.515 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.force_consistency_checks: 1 2024-09-12T02:54:45.515 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.report_bg_io_stats: 0 2024-09-12T02:54:45.515 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.ttl: 2592000 2024-09-12T02:54:45.515 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-12T02:54:45.515 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-12T02:54:45.515 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-12T02:54:45.515 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.enable_blob_files: false 2024-09-12T02:54:45.515 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.min_blob_size: 0 2024-09-12T02:54:45.515 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.blob_file_size: 268435456 2024-09-12T02:54:45.516 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-12T02:54:45.516 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-12T02:54:45.516 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-12T02:54:45.516 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-12T02:54:45.516 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-12T02:54:45.516 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.blob_file_starting_level: 0 2024-09-12T02:54:45.516 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-12T02:54:45.516 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-12T02:54:45.517 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-12T02:54:45.517 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: cee2b812-ee02-44e4-b33c-99928d84bd07 2024-09-12T02:54:45.517 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726109685237239, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-12T02:54:45.517 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-12T02:54:45.517 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726109685237622, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726109685, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "cee2b812-ee02-44e4-b33c-99928d84bd07", "db_session_id": "THNTX7Q0BXX7H87APJQI", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-12T02:54:45.517 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726109685237716, "job": 1, "event": "recovery_finished"} 2024-09-12T02:54:45.517 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-12T02:54:45.517 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-12T02:54:45.517 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5615f9756e00 2024-09-12T02:54:45.517 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: DB pointer 0x5615f9852000 2024-09-12T02:54:45.518 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-12T02:54:45.518 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-12T02:54:45.518 INFO:journalctl@ceph.mon.b.smithi177.stdout: ** DB Stats ** 2024-09-12T02:54:45.518 INFO:journalctl@ceph.mon.b.smithi177.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T02:54:45.518 INFO:journalctl@ceph.mon.b.smithi177.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-12T02:54:45.518 INFO:journalctl@ceph.mon.b.smithi177.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T02:54:45.518 INFO:journalctl@ceph.mon.b.smithi177.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T02:54:45.518 INFO:journalctl@ceph.mon.b.smithi177.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-12T02:54:45.518 INFO:journalctl@ceph.mon.b.smithi177.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T02:54:45.518 INFO:journalctl@ceph.mon.b.smithi177.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T02:54:45.518 INFO:journalctl@ceph.mon.b.smithi177.stdout: 2024-09-12T02:54:45.518 INFO:journalctl@ceph.mon.b.smithi177.stdout: ** Compaction Stats [default] ** 2024-09-12T02:54:45.519 INFO:journalctl@ceph.mon.b.smithi177.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T02:54:45.519 INFO:journalctl@ceph.mon.b.smithi177.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-12T02:54:45.519 INFO:journalctl@ceph.mon.b.smithi177.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T02:54:45.519 INFO:journalctl@ceph.mon.b.smithi177.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T02:54:45.519 INFO:journalctl@ceph.mon.b.smithi177.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T02:54:45.519 INFO:journalctl@ceph.mon.b.smithi177.stdout: 2024-09-12T02:54:45.519 INFO:journalctl@ceph.mon.b.smithi177.stdout: ** Compaction Stats [default] ** 2024-09-12T02:54:45.519 INFO:journalctl@ceph.mon.b.smithi177.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T02:54:45.519 INFO:journalctl@ceph.mon.b.smithi177.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-12T02:54:45.519 INFO:journalctl@ceph.mon.b.smithi177.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T02:54:45.519 INFO:journalctl@ceph.mon.b.smithi177.stdout: 2024-09-12T02:54:45.519 INFO:journalctl@ceph.mon.b.smithi177.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-12T02:54:45.519 INFO:journalctl@ceph.mon.b.smithi177.stdout: 2024-09-12T02:54:45.520 INFO:journalctl@ceph.mon.b.smithi177.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T02:54:45.520 INFO:journalctl@ceph.mon.b.smithi177.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-12T02:54:45.520 INFO:journalctl@ceph.mon.b.smithi177.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-12T02:54:45.520 INFO:journalctl@ceph.mon.b.smithi177.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-12T02:54:45.520 INFO:journalctl@ceph.mon.b.smithi177.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-12T02:54:45.520 INFO:journalctl@ceph.mon.b.smithi177.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-12T02:54:45.520 INFO:journalctl@ceph.mon.b.smithi177.stdout: Cumulative compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T02:54:45.520 INFO:journalctl@ceph.mon.b.smithi177.stdout: Interval compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T02:54:45.520 INFO:journalctl@ceph.mon.b.smithi177.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-12T02:54:45.520 INFO:journalctl@ceph.mon.b.smithi177.stdout: Block cache BinnedLRUCache@0x5615f97291f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-12T02:54:45.520 INFO:journalctl@ceph.mon.b.smithi177.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-12T02:54:45.520 INFO:journalctl@ceph.mon.b.smithi177.stdout: 2024-09-12T02:54:45.520 INFO:journalctl@ceph.mon.b.smithi177.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-12T02:54:45.520 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-12T02:54:45.521 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: using public_addr v2:172.21.15.177:0/0 -> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] 2024-09-12T02:54:45.521 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: starting mon.b rank -1 at public addrs [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] at bind addrs [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T02:54:45.521 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: mon.b@-1(???) e0 preinit fsid eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T02:54:45.521 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: mon.b@-1(synchronizing).mds e1 new map 2024-09-12T02:54:45.521 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-12T02:54:45.521 INFO:journalctl@ceph.mon.b.smithi177.stdout: e1 2024-09-12T02:54:45.521 INFO:journalctl@ceph.mon.b.smithi177.stdout: btime 2024-09-12T02:53:07:105500+0000 2024-09-12T02:54:45.521 INFO:journalctl@ceph.mon.b.smithi177.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T02:54:45.521 INFO:journalctl@ceph.mon.b.smithi177.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T02:54:45.521 INFO:journalctl@ceph.mon.b.smithi177.stdout: legacy client fscid: -1 2024-09-12T02:54:45.521 INFO:journalctl@ceph.mon.b.smithi177.stdout: 2024-09-12T02:54:45.521 INFO:journalctl@ceph.mon.b.smithi177.stdout: No filesystems configured 2024-09-12T02:54:45.521 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-12T02:54:45.521 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-12T02:54:45.522 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-12T02:54:45.522 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-12T02:54:45.522 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-12T02:54:45.522 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-12T02:54:45.522 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T02:54:45.522 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:54:45.522 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:54:45.522 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:54:45.522 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: Updating smithi177:/var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:54:45.522 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: Updating smithi177:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:54:45.522 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: Updating smithi177:/var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:54:45.522 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:45.522 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:45.522 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:45.523 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:45.523 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:45.523 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: Added host smithi183 2024-09-12T02:54:45.523 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:45.523 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:45.523 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:45.523 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:45.523 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:45.523 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:54:45.523 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:45.523 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1102187658' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T02:54:45.523 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1102187658' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T02:54:45.524 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:54:45.524 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:45.524 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:45.524 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:45.524 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:45.525 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:54:45.525 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:45.525 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:54:45.525 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-12T02:54:45.525 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: Updating smithi183:/var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:54:45.525 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:54:45.525 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:45.525 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: Updating smithi183:/var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:54:45.525 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:45.525 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:45.525 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:45.525 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:45.525 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:45.525 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:45.526 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:54:45.526 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:45.526 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:54:45.526 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:45.526 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi116:172.21.15.116=a;smithi177:172.21.15.177=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:54:45.526 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: Saving service mon spec with placement smithi116:172.21.15.116=a;smithi177:172.21.15.177=b;smithi183:172.21.15.183=c;count:3 2024-09-12T02:54:45.526 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: Deploying daemon mon.c on smithi183 2024-09-12T02:54:45.526 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:45.526 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:45.526 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='client.? 172.21.15.183:0/2923814928' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:54:45.526 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:54:45.526 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:54:45.526 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: mon.a calling monitor election 2024-09-12T02:54:45.526 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:45.527 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:54:45.527 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:54:45.527 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: mon.c calling monitor election 2024-09-12T02:54:45.527 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:45.527 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:54:45.527 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:54:45.527 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:45.527 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:54:45.527 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T02:54:45.527 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: monmap epoch 2 2024-09-12T02:54:45.527 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: fsid eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T02:54:45.527 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: last_changed 2024-09-12T02:54:36.578645+0000 2024-09-12T02:54:45.527 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: created 2024-09-12T02:53:04.190821+0000 2024-09-12T02:54:45.528 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: min_mon_release 19 (squid) 2024-09-12T02:54:45.528 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: election_strategy: 1 2024-09-12T02:54:45.528 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: 0: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.a 2024-09-12T02:54:45.528 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-09-12T02:54:45.528 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: fsmap 2024-09-12T02:54:45.528 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:54:45.528 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: mgrmap e13: x(active, since 38s) 2024-09-12T02:54:45.528 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: overall HEALTH_OK 2024-09-12T02:54:45.528 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:45.528 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:45.528 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:45.528 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:54:45.528 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:45.528 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: Deploying daemon mon.b on smithi177 2024-09-12T02:54:45.529 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='client.? 172.21.15.183:0/4207204731' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:54:45.529 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:54:45.529 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:45.529 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: from='client.? 172.21.15.183:0/4260026994' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:54:45.529 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 ceph-mon[35305]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-12T02:54:45.529 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 bash[35286]: 0b4bfaf4a88e1114e9bae344286a70d7e317ba37741fa287e339718b46bc8590 2024-09-12T02:54:45.529 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:45 smithi177 systemd[1]: Started Ceph mon.b for eb1ac602-70b1-11ef-bcea-c7b262605968. 2024-09-12T02:54:46.455 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T02:54:46.455 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T02:54:46.704 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:54:50.330 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.330+0000 7fe6ab2bc640 1 -- 172.21.15.183:0/1003965854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe690005740 msgr2=0x7fe690007bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:50.330 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.330+0000 7fe6ab2bc640 1 --2- 172.21.15.183:0/1003965854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe690005740 0x7fe690007bd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe6a41016c0 tx=0x7fe6a002c150 comp rx=0 tx=0).stop 2024-09-12T02:54:50.331 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.330+0000 7fe6ab2bc640 1 -- 172.21.15.183:0/1003965854 shutdown_connections 2024-09-12T02:54:50.331 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.330+0000 7fe6ab2bc640 1 --2- 172.21.15.183:0/1003965854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe690005740 0x7fe690007bd0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:50.331 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.330+0000 7fe6ab2bc640 1 -- 172.21.15.183:0/1003965854 >> 172.21.15.183:0/1003965854 conn(0x7fe6a40fc280 msgr2=0x7fe6a40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:50.331 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.330+0000 7fe6ab2bc640 1 -- 172.21.15.183:0/1003965854 shutdown_connections 2024-09-12T02:54:50.331 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.330+0000 7fe6ab2bc640 1 -- 172.21.15.183:0/1003965854 wait complete. 2024-09-12T02:54:50.333 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.331+0000 7fe6ab2bc640 1 Processor -- start 2024-09-12T02:54:50.333 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.331+0000 7fe6ab2bc640 1 -- start start 2024-09-12T02:54:50.333 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.332+0000 7fe6ab2bc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe690005740 0x7fe6a41994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:50.334 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.332+0000 7fe6ab2bc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6a4199a20 0x7fe6a419ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:50.334 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.332+0000 7fe6ab2bc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6a419e2f0 0x7fe6a419e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:50.334 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.332+0000 7fe6ab2bc640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a41075c0 con 0x7fe690005740 2024-09-12T02:54:50.334 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.332+0000 7fe6ab2bc640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a4107440 con 0x7fe6a4199a20 2024-09-12T02:54:50.334 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.332+0000 7fe6ab2bc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a4107740 con 0x7fe6a419e2f0 2024-09-12T02:54:50.334 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.332+0000 7fe6a9859640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6a419e2f0 0x7fe6a419e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:50.334 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.332+0000 7fe6a9058640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe690005740 0x7fe6a41994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:50.334 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.332+0000 7fe6a9859640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6a419e2f0 0x7fe6a419e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:33560/0 (socket says 172.21.15.183:33560) 2024-09-12T02:54:50.334 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.332+0000 7fe6a9859640 1 -- 172.21.15.183:0/2256682672 learned_addr learned my addr 172.21.15.183:0/2256682672 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-12T02:54:50.335 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.332+0000 7fe6a9058640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe690005740 0x7fe6a41994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.183:42578/0 (socket says 172.21.15.183:42578) 2024-09-12T02:54:50.335 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.332+0000 7fe6a9859640 1 -- 172.21.15.183:0/2256682672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6a419e2f0 msgr2=0x7fe6a419e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-12T02:54:50.335 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.332+0000 7fe6a9859640 1 -- 172.21.15.183:0/2256682672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6a419e2f0 msgr2=0x7fe6a419e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T02:54:50.335 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.332+0000 7fe6a9859640 1 --2- 172.21.15.183:0/2256682672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6a419e2f0 0x7fe6a419e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T02:54:50.335 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.332+0000 7fe6a9859640 1 --2- 172.21.15.183:0/2256682672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6a419e2f0 0x7fe6a419e7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:54:50.335 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.332+0000 7fe6a8857640 1 --2- 172.21.15.183:0/2256682672 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6a4199a20 0x7fe6a419ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:50.335 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.332+0000 7fe6a9058640 1 -- 172.21.15.183:0/2256682672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6a419e2f0 msgr2=0x7fe6a419e7a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:54:50.335 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.332+0000 7fe6a9058640 1 --2- 172.21.15.183:0/2256682672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6a419e2f0 0x7fe6a419e7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:50.335 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.332+0000 7fe6a8857640 1 -- 172.21.15.183:0/2256682672 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6a4199a20 msgr2=0x7fe6a419ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-12T02:54:50.335 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.332+0000 7fe6a8857640 1 -- 172.21.15.183:0/2256682672 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6a4199a20 msgr2=0x7fe6a419ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T02:54:50.335 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.332+0000 7fe6a8857640 1 --2- 172.21.15.183:0/2256682672 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6a4199a20 0x7fe6a419ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T02:54:50.336 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.332+0000 7fe6a9058640 1 -- 172.21.15.183:0/2256682672 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6a4199a20 msgr2=0x7fe6a419ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:50.336 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.332+0000 7fe6a8857640 1 --2- 172.21.15.183:0/2256682672 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6a4199a20 0x7fe6a419ddb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:54:50.336 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.332+0000 7fe6a9058640 1 --2- 172.21.15.183:0/2256682672 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6a4199a20 0x7fe6a419ddb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:50.336 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.332+0000 7fe6a9058640 1 -- 172.21.15.183:0/2256682672 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a41a6060 con 0x7fe690005740 2024-09-12T02:54:50.336 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.333+0000 7fe6a9058640 1 --2- 172.21.15.183:0/2256682672 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe690005740 0x7fe6a41994e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe6a41016c0 tx=0x7fe6a0004cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:50.336 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.333+0000 7fe69a7fc640 1 -- 172.21.15.183:0/2256682672 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0042020 con 0x7fe690005740 2024-09-12T02:54:50.336 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.333+0000 7fe69a7fc640 1 -- 172.21.15.183:0/2256682672 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6a00050b0 con 0x7fe690005740 2024-09-12T02:54:50.336 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.333+0000 7fe69a7fc640 1 -- 172.21.15.183:0/2256682672 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0032040 con 0x7fe690005740 2024-09-12T02:54:50.336 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.333+0000 7fe6ab2bc640 1 -- 172.21.15.183:0/2256682672 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a41a6230 con 0x7fe690005740 2024-09-12T02:54:50.336 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.333+0000 7fe6ab2bc640 1 -- 172.21.15.183:0/2256682672 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a41a6790 con 0x7fe690005740 2024-09-12T02:54:50.337 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.334+0000 7fe69a7fc640 1 -- 172.21.15.183:0/2256682672 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fe6a0045020 con 0x7fe690005740 2024-09-12T02:54:50.337 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.334+0000 7fe69a7fc640 1 --2- 172.21.15.183:0/2256682672 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe6880394f0 0x7fe68803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:50.338 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.334+0000 7fe69a7fc640 1 -- 172.21.15.183:0/2256682672 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe6a006de80 con 0x7fe690005740 2024-09-12T02:54:50.338 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.334+0000 7fe6ab2bc640 1 -- 172.21.15.183:0/2256682672 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6a41086a0 con 0x7fe690005740 2024-09-12T02:54:50.338 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.335+0000 7fe6a8857640 1 --2- 172.21.15.183:0/2256682672 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe6880394f0 0x7fe68803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:50.338 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.335+0000 7fe6a8857640 1 --2- 172.21.15.183:0/2256682672 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe6880394f0 0x7fe68803b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe694006fd0 tx=0x7fe694008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:50.339 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.340+0000 7fe69a7fc640 1 -- 172.21.15.183:0/2256682672 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe6a004c020 con 0x7fe690005740 2024-09-12T02:54:50.583 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.583+0000 7fe6ab2bc640 1 -- 172.21.15.183:0/2256682672 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe6a4104fb0 con 0x7fe690005740 2024-09-12T02:54:50.584 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:50 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:54:50.584 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:50 smithi183 ceph-mon[35535]: mon.c calling monitor election 2024-09-12T02:54:50.584 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:50 smithi183 ceph-mon[35535]: mon.a calling monitor election 2024-09-12T02:54:50.584 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:50 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:54:50.584 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:50 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:54:50.584 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:50 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:54:50.584 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:50 smithi183 ceph-mon[35535]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:50.584 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:50 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:54:50.584 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:50 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:54:50.584 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:50 smithi183 ceph-mon[35535]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:50.585 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:50 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:54:50.585 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:50 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:54:50.585 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:50 smithi183 ceph-mon[35535]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T02:54:50.585 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:50 smithi183 ceph-mon[35535]: monmap epoch 3 2024-09-12T02:54:50.585 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:50 smithi183 ceph-mon[35535]: fsid eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T02:54:50.585 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:50 smithi183 ceph-mon[35535]: last_changed 2024-09-12T02:54:45.262980+0000 2024-09-12T02:54:50.585 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:50 smithi183 ceph-mon[35535]: created 2024-09-12T02:53:04.190821+0000 2024-09-12T02:54:50.585 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:50 smithi183 ceph-mon[35535]: min_mon_release 19 (squid) 2024-09-12T02:54:50.585 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:50 smithi183 ceph-mon[35535]: election_strategy: 1 2024-09-12T02:54:50.585 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:50 smithi183 ceph-mon[35535]: 0: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.a 2024-09-12T02:54:50.585 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:50 smithi183 ceph-mon[35535]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-09-12T02:54:50.585 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:50 smithi183 ceph-mon[35535]: 2: [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon.b 2024-09-12T02:54:50.585 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:50 smithi183 ceph-mon[35535]: fsmap 2024-09-12T02:54:50.586 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:50 smithi183 ceph-mon[35535]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:54:50.586 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:50 smithi183 ceph-mon[35535]: mgrmap e13: x(active, since 47s) 2024-09-12T02:54:50.586 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:50 smithi183 ceph-mon[35535]: overall HEALTH_OK 2024-09-12T02:54:50.586 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:50 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:50.586 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:50 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:50.586 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:50 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:50.586 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:50 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:50.586 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.584+0000 7fe69a7fc640 1 -- 172.21.15.183:0/2256682672 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7fe6a004e560 con 0x7fe690005740 2024-09-12T02:54:50.587 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T02:54:50.587 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":3,"fsid":"eb1ac602-70b1-11ef-bcea-c7b262605968","modified":"2024-09-12T02:54:45.262980Z","created":"2024-09-12T02:53:04.190821Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:3300","nonce":0},{"type":"v1","addr":"172.21.15.116:6789","nonce":0}]},"addr":"172.21.15.116:6789/0","public_addr":"172.21.15.116:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:3300","nonce":0},{"type":"v1","addr":"172.21.15.177:6789","nonce":0}]},"addr":"172.21.15.177:6789/0","public_addr":"172.21.15.177:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T02:54:50.587 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 3 2024-09-12T02:54:50.588 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.586+0000 7fe6ab2bc640 1 -- 172.21.15.183:0/2256682672 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe6880394f0 msgr2=0x7fe68803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:50.588 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.586+0000 7fe6ab2bc640 1 --2- 172.21.15.183:0/2256682672 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe6880394f0 0x7fe68803b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe694006fd0 tx=0x7fe694008040 comp rx=0 tx=0).stop 2024-09-12T02:54:50.588 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.586+0000 7fe6ab2bc640 1 -- 172.21.15.183:0/2256682672 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe690005740 msgr2=0x7fe6a41994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:50.588 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.586+0000 7fe6ab2bc640 1 --2- 172.21.15.183:0/2256682672 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe690005740 0x7fe6a41994e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe6a41016c0 tx=0x7fe6a0004cb0 comp rx=0 tx=0).stop 2024-09-12T02:54:50.588 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.586+0000 7fe6ab2bc640 1 -- 172.21.15.183:0/2256682672 shutdown_connections 2024-09-12T02:54:50.588 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.586+0000 7fe6ab2bc640 1 --2- 172.21.15.183:0/2256682672 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe6880394f0 0x7fe68803b9b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:50.588 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.586+0000 7fe6ab2bc640 1 --2- 172.21.15.183:0/2256682672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6a419e2f0 0x7fe6a419e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:50.588 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.586+0000 7fe6ab2bc640 1 --2- 172.21.15.183:0/2256682672 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6a4199a20 0x7fe6a419ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:50.588 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.586+0000 7fe6ab2bc640 1 --2- 172.21.15.183:0/2256682672 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe690005740 0x7fe6a41994e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:50.588 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.586+0000 7fe6ab2bc640 1 -- 172.21.15.183:0/2256682672 >> 172.21.15.183:0/2256682672 conn(0x7fe6a40fc280 msgr2=0x7fe6a40fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:50.588 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.586+0000 7fe6ab2bc640 1 -- 172.21.15.183:0/2256682672 shutdown_connections 2024-09-12T02:54:50.588 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:50.586+0000 7fe6ab2bc640 1 -- 172.21.15.183:0/2256682672 wait complete. 2024-09-12T02:54:50.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:50 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:54:50.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:50 smithi116 ceph-mon[31410]: mon.c calling monitor election 2024-09-12T02:54:50.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:50 smithi116 ceph-mon[31410]: mon.a calling monitor election 2024-09-12T02:54:50.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:50 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:54:50.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:50 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:54:50.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:50 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:54:50.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:50 smithi116 ceph-mon[31410]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:50.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:50 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:54:50.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:50 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:54:50.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:50 smithi116 ceph-mon[31410]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:50.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:50 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:54:50.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:50 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:54:50.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:50 smithi116 ceph-mon[31410]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T02:54:50.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:50 smithi116 ceph-mon[31410]: monmap epoch 3 2024-09-12T02:54:50.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:50 smithi116 ceph-mon[31410]: fsid eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T02:54:50.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:50 smithi116 ceph-mon[31410]: last_changed 2024-09-12T02:54:45.262980+0000 2024-09-12T02:54:50.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:50 smithi116 ceph-mon[31410]: created 2024-09-12T02:53:04.190821+0000 2024-09-12T02:54:50.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:50 smithi116 ceph-mon[31410]: min_mon_release 19 (squid) 2024-09-12T02:54:50.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:50 smithi116 ceph-mon[31410]: election_strategy: 1 2024-09-12T02:54:50.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:50 smithi116 ceph-mon[31410]: 0: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.a 2024-09-12T02:54:50.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:50 smithi116 ceph-mon[31410]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-09-12T02:54:50.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:50 smithi116 ceph-mon[31410]: 2: [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon.b 2024-09-12T02:54:50.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:50 smithi116 ceph-mon[31410]: fsmap 2024-09-12T02:54:50.805 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:50 smithi116 ceph-mon[31410]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:54:50.805 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:50 smithi116 ceph-mon[31410]: mgrmap e13: x(active, since 47s) 2024-09-12T02:54:50.805 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:50 smithi116 ceph-mon[31410]: overall HEALTH_OK 2024-09-12T02:54:50.805 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:50 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:50.805 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:50 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:50.805 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:50 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:50.805 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:50 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:51.141 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-12T02:54:51.141 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-12T02:54:51.396 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:54:51.417 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:51 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:51.417 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:51 smithi183 ceph-mon[35535]: from='client.? 172.21.15.183:0/2256682672' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:54:51.417 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:51 smithi183 ceph-mon[35535]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:51.417 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:51 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:54:51.518 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:51 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:51.519 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:51 smithi116 ceph-mon[31410]: from='client.? 172.21.15.183:0/2256682672' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:54:51.519 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:51 smithi116 ceph-mon[31410]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:51.519 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:51 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:54:51.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.942+0000 7fe8f4c33640 1 -- 172.21.15.116:0/1037755551 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8f01027e0 msgr2=0x7fe8f0102bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:51.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.942+0000 7fe8f4c33640 1 --2- 172.21.15.116:0/1037755551 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8f01027e0 0x7fe8f0102bc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009880 tx=0x7fe8e002a670 comp rx=0 tx=0).stop 2024-09-12T02:54:51.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.942+0000 7fe8f4c33640 1 -- 172.21.15.116:0/1037755551 shutdown_connections 2024-09-12T02:54:51.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.942+0000 7fe8f4c33640 1 --2- 172.21.15.116:0/1037755551 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8f01027e0 0x7fe8f0102bc0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:51.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.942+0000 7fe8f4c33640 1 -- 172.21.15.116:0/1037755551 >> 172.21.15.116:0/1037755551 conn(0x7fe8f00fc1a0 msgr2=0x7fe8f00fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:51.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.942+0000 7fe8f4c33640 1 -- 172.21.15.116:0/1037755551 shutdown_connections 2024-09-12T02:54:51.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.942+0000 7fe8f4c33640 1 -- 172.21.15.116:0/1037755551 wait complete. 2024-09-12T02:54:51.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.943+0000 7fe8f4c33640 1 Processor -- start 2024-09-12T02:54:51.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.943+0000 7fe8f4c33640 1 -- start start 2024-09-12T02:54:51.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.943+0000 7fe8f4c33640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8f01027e0 0x7fe8f0100440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:51.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.943+0000 7fe8f4c33640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8f0100980 0x7fe8f0100de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:51.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.943+0000 7fe8f4c33640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8f01a35e0 0x7fe8f01a59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:51.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.943+0000 7fe8f4c33640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f010bc90 con 0x7fe8f01027e0 2024-09-12T02:54:51.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.943+0000 7fe8f4c33640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f010bb10 con 0x7fe8f0100980 2024-09-12T02:54:51.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.943+0000 7fe8f4c33640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f010be10 con 0x7fe8f01a35e0 2024-09-12T02:54:51.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.943+0000 7fe8ee59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8f01027e0 0x7fe8f0100440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:51.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.943+0000 7fe8ee59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8f01027e0 0x7fe8f0100440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56674/0 (socket says 172.21.15.116:56674) 2024-09-12T02:54:51.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.944+0000 7fe8ee59c640 1 -- 172.21.15.116:0/442861112 learned_addr learned my addr 172.21.15.116:0/442861112 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:54:51.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.944+0000 7fe8eed9d640 1 --2- 172.21.15.116:0/442861112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8f01a35e0 0x7fe8f01a59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:51.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.944+0000 7fe8edd9b640 1 --2- 172.21.15.116:0/442861112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8f0100980 0x7fe8f0100de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:51.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.944+0000 7fe8edd9b640 1 -- 172.21.15.116:0/442861112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8f0100980 msgr2=0x7fe8f0100de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-12T02:54:51.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.944+0000 7fe8edd9b640 1 -- 172.21.15.116:0/442861112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8f0100980 msgr2=0x7fe8f0100de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T02:54:51.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.944+0000 7fe8edd9b640 1 --2- 172.21.15.116:0/442861112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8f0100980 0x7fe8f0100de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T02:54:51.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.944+0000 7fe8edd9b640 1 --2- 172.21.15.116:0/442861112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8f0100980 0x7fe8f0100de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:54:51.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.944+0000 7fe8ee59c640 1 -- 172.21.15.116:0/442861112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8f01a35e0 msgr2=0x7fe8f01a59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:51.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.944+0000 7fe8ee59c640 1 --2- 172.21.15.116:0/442861112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8f01a35e0 0x7fe8f01a59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:51.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.944+0000 7fe8ee59c640 1 -- 172.21.15.116:0/442861112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8f0100980 msgr2=0x7fe8f0100de0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:54:51.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.944+0000 7fe8ee59c640 1 --2- 172.21.15.116:0/442861112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8f0100980 0x7fe8f0100de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:51.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.944+0000 7fe8ee59c640 1 -- 172.21.15.116:0/442861112 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f01a60a0 con 0x7fe8f01027e0 2024-09-12T02:54:51.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.944+0000 7fe8ee59c640 1 --2- 172.21.15.116:0/442861112 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8f01027e0 0x7fe8f0100440 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0031550 tx=0x7fe8e00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:51.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.944+0000 7fe8df7fe640 1 -- 172.21.15.116:0/442861112 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e003f030 con 0x7fe8f01027e0 2024-09-12T02:54:51.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.944+0000 7fe8df7fe640 1 -- 172.21.15.116:0/442861112 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e0003ce0 con 0x7fe8f01027e0 2024-09-12T02:54:51.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.944+0000 7fe8df7fe640 1 -- 172.21.15.116:0/442861112 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0035070 con 0x7fe8f01027e0 2024-09-12T02:54:51.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.944+0000 7fe8f4c33640 1 -- 172.21.15.116:0/442861112 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f01a6330 con 0x7fe8f01027e0 2024-09-12T02:54:51.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.945+0000 7fe8f4c33640 1 -- 172.21.15.116:0/442861112 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f01a67c0 con 0x7fe8f01027e0 2024-09-12T02:54:51.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.946+0000 7fe8df7fe640 1 -- 172.21.15.116:0/442861112 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fe8e0042030 con 0x7fe8f01027e0 2024-09-12T02:54:51.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.946+0000 7fe8f4c33640 1 -- 172.21.15.116:0/442861112 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8f0102c40 con 0x7fe8f01027e0 2024-09-12T02:54:51.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.946+0000 7fe8df7fe640 1 --2- 172.21.15.116:0/442861112 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe8d80394f0 0x7fe8d803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:51.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.946+0000 7fe8df7fe640 1 -- 172.21.15.116:0/442861112 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe8e006cd00 con 0x7fe8f01027e0 2024-09-12T02:54:51.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.946+0000 7fe8edd9b640 1 --2- 172.21.15.116:0/442861112 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe8d80394f0 0x7fe8d803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:51.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.946+0000 7fe8edd9b640 1 --2- 172.21.15.116:0/442861112 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe8d80394f0 0x7fe8d803b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4006fd0 tx=0x7fe8e4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:51.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:51.951+0000 7fe8df7fe640 1 -- 172.21.15.116:0/442861112 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe8e004bd70 con 0x7fe8f01027e0 2024-09-12T02:54:52.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:52.201+0000 7fe8f4c33640 1 -- 172.21.15.116:0/442861112 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fe8f01a6ae0 con 0x7fe8f01027e0 2024-09-12T02:54:52.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:52.202+0000 7fe8df7fe640 1 -- 172.21.15.116:0/442861112 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7fe8e003ac50 con 0x7fe8f01027e0 2024-09-12T02:54:52.203 INFO:teuthology.orchestra.run.smithi116.stdout:# minimal ceph.conf for eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T02:54:52.203 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2024-09-12T02:54:52.203 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T02:54:52.203 INFO:teuthology.orchestra.run.smithi116.stdout: mon_host = [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] 2024-09-12T02:54:52.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:52.204+0000 7fe8f4c33640 1 -- 172.21.15.116:0/442861112 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe8d80394f0 msgr2=0x7fe8d803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:52.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:52.204+0000 7fe8f4c33640 1 --2- 172.21.15.116:0/442861112 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe8d80394f0 0x7fe8d803b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4006fd0 tx=0x7fe8e4008040 comp rx=0 tx=0).stop 2024-09-12T02:54:52.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:52.204+0000 7fe8f4c33640 1 -- 172.21.15.116:0/442861112 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8f01027e0 msgr2=0x7fe8f0100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:52.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:52.204+0000 7fe8f4c33640 1 --2- 172.21.15.116:0/442861112 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8f01027e0 0x7fe8f0100440 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0031550 tx=0x7fe8e00042f0 comp rx=0 tx=0).stop 2024-09-12T02:54:52.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:52.204+0000 7fe8f4c33640 1 -- 172.21.15.116:0/442861112 shutdown_connections 2024-09-12T02:54:52.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:52.204+0000 7fe8f4c33640 1 --2- 172.21.15.116:0/442861112 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe8d80394f0 0x7fe8d803b9b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:52.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:52.204+0000 7fe8f4c33640 1 --2- 172.21.15.116:0/442861112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8f01a35e0 0x7fe8f01a59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:52.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:52.204+0000 7fe8f4c33640 1 --2- 172.21.15.116:0/442861112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8f0100980 0x7fe8f0100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:52.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:52.204+0000 7fe8f4c33640 1 --2- 172.21.15.116:0/442861112 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8f01027e0 0x7fe8f0100440 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:52.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:52.204+0000 7fe8f4c33640 1 -- 172.21.15.116:0/442861112 >> 172.21.15.116:0/442861112 conn(0x7fe8f00fc1a0 msgr2=0x7fe8f01055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:52.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:52.204+0000 7fe8f4c33640 1 -- 172.21.15.116:0/442861112 shutdown_connections 2024-09-12T02:54:52.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:54:52.204+0000 7fe8f4c33640 1 -- 172.21.15.116:0/442861112 wait complete. 2024-09-12T02:54:52.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:54:52.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: mon.c calling monitor election 2024-09-12T02:54:52.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: mon.a calling monitor election 2024-09-12T02:54:52.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:54:52.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:54:52.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:54:52.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:52.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:54:52.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:54:52.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:52.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:54:52.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:54:52.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T02:54:52.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: monmap epoch 3 2024-09-12T02:54:52.747 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: fsid eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T02:54:52.747 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: last_changed 2024-09-12T02:54:45.262980+0000 2024-09-12T02:54:52.747 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: created 2024-09-12T02:53:04.190821+0000 2024-09-12T02:54:52.747 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: min_mon_release 19 (squid) 2024-09-12T02:54:52.747 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: election_strategy: 1 2024-09-12T02:54:52.747 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: 0: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.a 2024-09-12T02:54:52.747 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-09-12T02:54:52.747 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: 2: [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon.b 2024-09-12T02:54:52.747 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: fsmap 2024-09-12T02:54:52.747 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:54:52.747 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: mgrmap e13: x(active, since 47s) 2024-09-12T02:54:52.747 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: overall HEALTH_OK 2024-09-12T02:54:52.747 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:52.747 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:52.748 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:52.748 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:52.748 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:52.748 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: from='client.? 172.21.15.183:0/2256682672' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:54:52.748 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:52.748 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:54:52.878 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-12T02:54:52.878 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-12T02:54:52.878 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T02:54:52.905 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-12T02:54:52.906 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:54:52.971 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-12T02:54:52.971 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T02:54:53.001 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-12T02:54:53.001 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:54:53.069 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-12T02:54:53.069 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T02:54:53.098 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-12T02:54:53.098 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:54:53.111 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: mon.b calling monitor election 2024-09-12T02:54:53.111 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: mon.b calling monitor election 2024-09-12T02:54:53.111 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: mon.c calling monitor election 2024-09-12T02:54:53.111 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: mon.a calling monitor election 2024-09-12T02:54:53.111 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T02:54:53.111 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: monmap epoch 3 2024-09-12T02:54:53.111 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: fsid eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T02:54:53.112 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: last_changed 2024-09-12T02:54:45.262980+0000 2024-09-12T02:54:53.112 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: created 2024-09-12T02:53:04.190821+0000 2024-09-12T02:54:53.112 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: min_mon_release 19 (squid) 2024-09-12T02:54:53.112 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: election_strategy: 1 2024-09-12T02:54:53.112 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: 0: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.a 2024-09-12T02:54:53.112 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-09-12T02:54:53.112 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: 2: [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon.b 2024-09-12T02:54:53.112 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: fsmap 2024-09-12T02:54:53.112 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:54:53.112 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: mgrmap e13: x(active, since 49s) 2024-09-12T02:54:53.112 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: overall HEALTH_OK 2024-09-12T02:54:53.112 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.112 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.113 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.113 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.113 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.113 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.113 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.113 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.113 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.113 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.113 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.113 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:54:53.114 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:54:53.114 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:52 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:53.161 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: mon.b calling monitor election 2024-09-12T02:54:53.161 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: mon.b calling monitor election 2024-09-12T02:54:53.161 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: mon.c calling monitor election 2024-09-12T02:54:53.161 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: mon.a calling monitor election 2024-09-12T02:54:53.161 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T02:54:53.161 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: monmap epoch 3 2024-09-12T02:54:53.162 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: fsid eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T02:54:53.162 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: last_changed 2024-09-12T02:54:45.262980+0000 2024-09-12T02:54:53.162 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: created 2024-09-12T02:53:04.190821+0000 2024-09-12T02:54:53.162 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: min_mon_release 19 (squid) 2024-09-12T02:54:53.162 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: election_strategy: 1 2024-09-12T02:54:53.162 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: 0: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.a 2024-09-12T02:54:53.162 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-09-12T02:54:53.162 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: 2: [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon.b 2024-09-12T02:54:53.162 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: fsmap 2024-09-12T02:54:53.162 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:54:53.163 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: mgrmap e13: x(active, since 49s) 2024-09-12T02:54:53.163 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: overall HEALTH_OK 2024-09-12T02:54:53.163 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.163 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.163 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.163 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.163 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.163 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.163 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.163 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.163 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.164 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.164 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.164 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:54:53.164 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:54:53.164 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:52 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:53.165 INFO:tasks.cephadm:Adding mgr.x on smithi116 2024-09-12T02:54:53.166 INFO:tasks.cephadm:Adding mgr.y on smithi177 2024-09-12T02:54:53.166 INFO:tasks.cephadm:Adding mgr.z on smithi183 2024-09-12T02:54:53.166 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi116=x;smithi177=y;smithi183=z' 2024-09-12T02:54:53.246 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: mon.b calling monitor election 2024-09-12T02:54:53.246 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: mon.b calling monitor election 2024-09-12T02:54:53.246 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: mon.c calling monitor election 2024-09-12T02:54:53.246 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: mon.a calling monitor election 2024-09-12T02:54:53.246 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T02:54:53.247 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: monmap epoch 3 2024-09-12T02:54:53.247 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: fsid eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T02:54:53.247 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: last_changed 2024-09-12T02:54:45.262980+0000 2024-09-12T02:54:53.247 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: created 2024-09-12T02:53:04.190821+0000 2024-09-12T02:54:53.247 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: min_mon_release 19 (squid) 2024-09-12T02:54:53.247 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: election_strategy: 1 2024-09-12T02:54:53.247 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: 0: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.a 2024-09-12T02:54:53.247 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-09-12T02:54:53.247 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: 2: [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon.b 2024-09-12T02:54:53.247 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: fsmap 2024-09-12T02:54:53.247 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:54:53.247 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: mgrmap e13: x(active, since 49s) 2024-09-12T02:54:53.247 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: overall HEALTH_OK 2024-09-12T02:54:53.248 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.248 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.248 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.248 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.248 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.248 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.248 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.248 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.248 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.248 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.248 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:53.249 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:54:53.249 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:54:53.249 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:53.443 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:54:53.914 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.895+0000 7efdf211b640 1 -- 172.21.15.183:0/2811701013 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdec1005f0 msgr2=0x7efdec1009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:53.914 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.895+0000 7efdf211b640 1 --2- 172.21.15.183:0/2811701013 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdec1005f0 0x7efdec1009f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efde4009c20 tx=0x7efde402aa90 comp rx=0 tx=0).stop 2024-09-12T02:54:53.914 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.895+0000 7efdf211b640 1 -- 172.21.15.183:0/2811701013 shutdown_connections 2024-09-12T02:54:53.914 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.895+0000 7efdf211b640 1 --2- 172.21.15.183:0/2811701013 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdec1005f0 0x7efdec1009f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:53.914 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.895+0000 7efdf211b640 1 -- 172.21.15.183:0/2811701013 >> 172.21.15.183:0/2811701013 conn(0x7efdec0fbda0 msgr2=0x7efdec0fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:53.914 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.895+0000 7efdf211b640 1 -- 172.21.15.183:0/2811701013 shutdown_connections 2024-09-12T02:54:53.914 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.895+0000 7efdf211b640 1 -- 172.21.15.183:0/2811701013 wait complete. 2024-09-12T02:54:53.914 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.896+0000 7efdf211b640 1 Processor -- start 2024-09-12T02:54:53.914 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.896+0000 7efdf211b640 1 -- start start 2024-09-12T02:54:53.914 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.896+0000 7efdf211b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdec1005f0 0x7efdec06af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:53.914 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.896+0000 7efdf211b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdec199d10 0x7efdec19c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:53.914 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.896+0000 7efdf211b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdec19c640 0x7efdec19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:53.915 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.896+0000 7efdf211b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efdec106700 con 0x7efdec1005f0 2024-09-12T02:54:53.915 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.896+0000 7efdf211b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efdec106580 con 0x7efdec19c640 2024-09-12T02:54:53.915 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.896+0000 7efdf211b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efdec106880 con 0x7efdec199d10 2024-09-12T02:54:53.915 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.896+0000 7efdeb7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdec1005f0 0x7efdec06af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:53.915 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.896+0000 7efdeaffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdec199d10 0x7efdec19c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:53.915 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.896+0000 7efdeb7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdec1005f0 0x7efdec06af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.183:56466/0 (socket says 172.21.15.183:56466) 2024-09-12T02:54:53.915 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.896+0000 7efdeb7fe640 1 -- 172.21.15.183:0/3338755220 learned_addr learned my addr 172.21.15.183:0/3338755220 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-12T02:54:53.915 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.896+0000 7efdebfff640 1 --2- 172.21.15.183:0/3338755220 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdec19c640 0x7efdec19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:53.915 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.897+0000 7efdeb7fe640 1 -- 172.21.15.183:0/3338755220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdec199d10 msgr2=0x7efdec19c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:53.915 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.897+0000 7efdeb7fe640 1 --2- 172.21.15.183:0/3338755220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdec199d10 0x7efdec19c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:53.915 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.897+0000 7efdeb7fe640 1 -- 172.21.15.183:0/3338755220 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdec19c640 msgr2=0x7efdec19eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:53.915 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.897+0000 7efdeb7fe640 1 --2- 172.21.15.183:0/3338755220 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdec19c640 0x7efdec19eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:53.916 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.897+0000 7efdeb7fe640 1 -- 172.21.15.183:0/3338755220 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdec19f410 con 0x7efdec1005f0 2024-09-12T02:54:53.916 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.897+0000 7efdeb7fe640 1 --2- 172.21.15.183:0/3338755220 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdec1005f0 0x7efdec06af90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efde402b080 tx=0x7efde402e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:53.916 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.897+0000 7efde8ff9640 1 -- 172.21.15.183:0/3338755220 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde4047070 con 0x7efdec1005f0 2024-09-12T02:54:53.916 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.897+0000 7efde8ff9640 1 -- 172.21.15.183:0/3338755220 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efde402fb50 con 0x7efdec1005f0 2024-09-12T02:54:53.916 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.897+0000 7efdf211b640 1 -- 172.21.15.183:0/3338755220 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdec19f640 con 0x7efdec1005f0 2024-09-12T02:54:53.916 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.897+0000 7efde8ff9640 1 -- 172.21.15.183:0/3338755220 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde4033050 con 0x7efdec1005f0 2024-09-12T02:54:53.916 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.897+0000 7efdf211b640 1 -- 172.21.15.183:0/3338755220 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdec19fbf0 con 0x7efdec1005f0 2024-09-12T02:54:53.916 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.898+0000 7efde8ff9640 1 -- 172.21.15.183:0/3338755220 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7efde404a070 con 0x7efdec1005f0 2024-09-12T02:54:53.916 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.898+0000 7efdf211b640 1 -- 172.21.15.183:0/3338755220 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdec18f4c0 con 0x7efdec1005f0 2024-09-12T02:54:53.916 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.898+0000 7efde8ff9640 1 --2- 172.21.15.183:0/3338755220 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7efdd80394f0 0x7efdd803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:53.916 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.898+0000 7efde8ff9640 1 -- 172.21.15.183:0/3338755220 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7efde406d360 con 0x7efdec1005f0 2024-09-12T02:54:53.916 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.898+0000 7efdeaffd640 1 --2- 172.21.15.183:0/3338755220 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7efdd80394f0 0x7efdd803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:53.916 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.899+0000 7efdeaffd640 1 --2- 172.21.15.183:0/3338755220 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7efdd80394f0 0x7efdd803b9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efdec06b510 tx=0x7efdd4009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:53.917 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:53.903+0000 7efde8ff9640 1 -- 172.21.15.183:0/3338755220 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efde4038450 con 0x7efdec1005f0 2024-09-12T02:54:54.080 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:54.080+0000 7efdf211b640 1 -- 172.21.15.183:0/3338755220 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi116=x;smithi177=y;smithi183=z", "target": ["mon-mgr", ""]}) -- 0x7efdec0626c0 con 0x7efdd80394f0 2024-09-12T02:54:54.081 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:54 smithi183 ceph-mon[35535]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T02:54:54.081 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:54 smithi183 ceph-mon[35535]: Reconfiguring daemon mon.a on smithi116 2024-09-12T02:54:54.081 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:54 smithi183 ceph-mon[35535]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:54.081 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:54 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:54.081 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:54 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:54:54.085 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:54.086+0000 7efde8ff9640 1 -- 172.21.15.183:0/3338755220 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7efdec0626c0 con 0x7efdd80394f0 2024-09-12T02:54:54.086 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled mgr update... 2024-09-12T02:54:54.088 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:54.088+0000 7efdf211b640 1 -- 172.21.15.183:0/3338755220 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7efdd80394f0 msgr2=0x7efdd803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:54.088 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:54.088+0000 7efdf211b640 1 --2- 172.21.15.183:0/3338755220 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7efdd80394f0 0x7efdd803b9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efdec06b510 tx=0x7efdd4009190 comp rx=0 tx=0).stop 2024-09-12T02:54:54.089 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:54.088+0000 7efdf211b640 1 -- 172.21.15.183:0/3338755220 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdec1005f0 msgr2=0x7efdec06af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:54.089 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:54.088+0000 7efdf211b640 1 --2- 172.21.15.183:0/3338755220 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdec1005f0 0x7efdec06af90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efde402b080 tx=0x7efde402e3c0 comp rx=0 tx=0).stop 2024-09-12T02:54:54.089 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:54.088+0000 7efdf211b640 1 -- 172.21.15.183:0/3338755220 shutdown_connections 2024-09-12T02:54:54.089 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:54.088+0000 7efdf211b640 1 --2- 172.21.15.183:0/3338755220 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7efdd80394f0 0x7efdd803b9b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:54.089 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:54.088+0000 7efdf211b640 1 --2- 172.21.15.183:0/3338755220 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdec19c640 0x7efdec19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:54.089 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:54.088+0000 7efdf211b640 1 --2- 172.21.15.183:0/3338755220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdec199d10 0x7efdec19c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:54.089 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:54.088+0000 7efdf211b640 1 --2- 172.21.15.183:0/3338755220 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdec1005f0 0x7efdec06af90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:54.089 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:54.088+0000 7efdf211b640 1 -- 172.21.15.183:0/3338755220 >> 172.21.15.183:0/3338755220 conn(0x7efdec0fbda0 msgr2=0x7efdec0fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:54.089 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:54.088+0000 7efdf211b640 1 -- 172.21.15.183:0/3338755220 shutdown_connections 2024-09-12T02:54:54.089 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:54:54.088+0000 7efdf211b640 1 -- 172.21.15.183:0/3338755220 wait complete. 2024-09-12T02:54:54.484 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:54 smithi116 ceph-mon[31410]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T02:54:54.484 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:54 smithi116 ceph-mon[31410]: Reconfiguring daemon mon.a on smithi116 2024-09-12T02:54:54.485 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:54 smithi116 ceph-mon[31410]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:54.485 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:54 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:54.485 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:54 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:54:54.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:54 smithi177 ceph-mon[35305]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T02:54:54.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:54 smithi177 ceph-mon[35305]: Reconfiguring daemon mon.a on smithi116 2024-09-12T02:54:54.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:54 smithi177 ceph-mon[35305]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:54.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:54 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:54.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:54 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:54:54.602 DEBUG:teuthology.orchestra.run.smithi177:mgr.y> sudo journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mgr.y.service 2024-09-12T02:54:54.605 DEBUG:teuthology.orchestra.run.smithi183:mgr.z> sudo journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mgr.z.service 2024-09-12T02:54:54.608 INFO:tasks.cephadm:Deploying OSDs... 2024-09-12T02:54:54.608 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-12T02:54:54.608 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T02:54:54.623 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T02:54:54.624 DEBUG:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_1 2024-09-12T02:54:54.678 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T02:54:54.678 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:54:54.678 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-12T02:54:54.679 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:54:54.679 INFO:teuthology.orchestra.run.smithi116.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:54:54.679 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2024-09-12 02:54:14.374212922 +0000 2024-09-12T02:54:54.679 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2024-09-12 02:48:32.997836949 +0000 2024-09-12T02:54:54.679 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2024-09-12 02:48:32.997836949 +0000 2024-09-12T02:54:54.680 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: 2024-09-12 02:48:32.997836949 +0000 2024-09-12T02:54:54.680 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T02:54:54.742 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2024-09-12T02:54:54.742 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2024-09-12T02:54:54.742 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000208799 s, 2.5 MB/s 2024-09-12T02:54:54.744 DEBUG:teuthology.orchestra.run.smithi116:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T02:54:54.801 DEBUG:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_2 2024-09-12T02:54:54.857 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T02:54:54.858 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:54:54.858 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-12T02:54:54.858 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:54:54.858 INFO:teuthology.orchestra.run.smithi116.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:54:54.858 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2024-09-12 02:54:14.407212354 +0000 2024-09-12T02:54:54.858 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2024-09-12 02:48:33.433829436 +0000 2024-09-12T02:54:54.858 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2024-09-12 02:48:33.433829436 +0000 2024-09-12T02:54:54.859 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: 2024-09-12 02:48:33.433829436 +0000 2024-09-12T02:54:54.859 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T02:54:54.920 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2024-09-12T02:54:54.920 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2024-09-12T02:54:54.920 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.00019205 s, 2.7 MB/s 2024-09-12T02:54:54.921 DEBUG:teuthology.orchestra.run.smithi116:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T02:54:54.978 DEBUG:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_3 2024-09-12T02:54:55.033 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T02:54:55.034 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:54:55.034 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-12T02:54:55.034 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:54:55.034 INFO:teuthology.orchestra.run.smithi116.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:54:55.034 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2024-09-12 02:54:14.440211785 +0000 2024-09-12T02:54:55.034 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2024-09-12 02:48:33.753823922 +0000 2024-09-12T02:54:55.034 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2024-09-12 02:48:33.753823922 +0000 2024-09-12T02:54:55.034 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: 2024-09-12 02:48:33.753823922 +0000 2024-09-12T02:54:55.034 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T02:54:55.098 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2024-09-12T02:54:55.098 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2024-09-12T02:54:55.098 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.00018731 s, 2.7 MB/s 2024-09-12T02:54:55.099 DEBUG:teuthology.orchestra.run.smithi116:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T02:54:55.156 DEBUG:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_4 2024-09-12T02:54:55.212 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T02:54:55.212 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:54:55.212 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-12T02:54:55.212 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:54:55.212 INFO:teuthology.orchestra.run.smithi116.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:54:55.212 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2024-09-12 02:54:14.483211045 +0000 2024-09-12T02:54:55.212 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2024-09-12 02:48:34.089818133 +0000 2024-09-12T02:54:55.213 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2024-09-12 02:48:34.089818133 +0000 2024-09-12T02:54:55.213 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: 2024-09-12 02:48:34.089818133 +0000 2024-09-12T02:54:55.213 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T02:54:55.275 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2024-09-12T02:54:55.275 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2024-09-12T02:54:55.275 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000209335 s, 2.4 MB/s 2024-09-12T02:54:55.277 DEBUG:teuthology.orchestra.run.smithi116:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T02:54:55.333 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-09-12T02:54:55.333 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T02:54:55.353 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T02:54:55.353 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_1 2024-09-12T02:54:55.412 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T02:54:55.412 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:54:55.412 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-12T02:54:55.412 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:54:55.412 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:54:55.412 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-09-12 02:54:44.692149656 +0000 2024-09-12T02:54:55.413 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-09-12 02:48:48.521930810 +0000 2024-09-12T02:54:55.413 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-09-12 02:48:48.521930810 +0000 2024-09-12T02:54:55.413 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-09-12 02:48:48.521930810 +0000 2024-09-12T02:54:55.413 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T02:54:55.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:55 smithi183 ceph-mon[35535]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi116=x;smithi177=y;smithi183=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:54:55.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:55 smithi183 ceph-mon[35535]: Saving service mgr spec with placement smithi116=x;smithi177=y;smithi183=z;count:3 2024-09-12T02:54:55.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:55 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:55.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:55 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:55.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:55 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:55.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:55 smithi183 ceph-mon[35535]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T02:54:55.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:55 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:54:55.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:55 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:54:55.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:55 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:55.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:55 smithi183 ceph-mon[35535]: Reconfiguring daemon mgr.x on smithi116 2024-09-12T02:54:55.479 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:55 smithi177 ceph-mon[35305]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi116=x;smithi177=y;smithi183=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:54:55.480 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:55 smithi177 ceph-mon[35305]: Saving service mgr spec with placement smithi116=x;smithi177=y;smithi183=z;count:3 2024-09-12T02:54:55.480 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:55 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:55.480 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:55 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:55.480 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:55 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:55.480 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:55 smithi177 ceph-mon[35305]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T02:54:55.480 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:55 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:54:55.480 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:55 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:54:55.480 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:55 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:55.480 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:55 smithi177 ceph-mon[35305]: Reconfiguring daemon mgr.x on smithi116 2024-09-12T02:54:55.482 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-09-12T02:54:55.482 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-09-12T02:54:55.482 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000303747 s, 1.7 MB/s 2024-09-12T02:54:55.484 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T02:54:55.494 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:55 smithi116 ceph-mon[31410]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi116=x;smithi177=y;smithi183=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:54:55.495 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:55 smithi116 ceph-mon[31410]: Saving service mgr spec with placement smithi116=x;smithi177=y;smithi183=z;count:3 2024-09-12T02:54:55.495 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:55 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:55.495 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:55 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:55.495 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:55 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:55.495 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:55 smithi116 ceph-mon[31410]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T02:54:55.495 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:55 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:54:55.495 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:55 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:54:55.495 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:55 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:55.495 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:55 smithi116 ceph-mon[31410]: Reconfiguring daemon mgr.x on smithi116 2024-09-12T02:54:55.544 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_2 2024-09-12T02:54:55.601 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T02:54:55.601 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:54:55.601 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-12T02:54:55.601 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:54:55.601 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:54:55.602 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-09-12 02:54:44.692149656 +0000 2024-09-12T02:54:55.602 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-09-12 02:48:48.841921550 +0000 2024-09-12T02:54:55.602 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-09-12 02:48:48.841921550 +0000 2024-09-12T02:54:55.602 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-09-12 02:48:48.841921550 +0000 2024-09-12T02:54:55.602 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T02:54:55.666 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-09-12T02:54:55.666 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-09-12T02:54:55.666 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000208933 s, 2.5 MB/s 2024-09-12T02:54:55.667 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T02:54:55.723 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_3 2024-09-12T02:54:55.778 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T02:54:55.779 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:54:55.779 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-12T02:54:55.779 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:54:55.779 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:54:55.779 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-09-12 02:54:44.693149628 +0000 2024-09-12T02:54:55.779 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-09-12 02:48:49.180911740 +0000 2024-09-12T02:54:55.779 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-09-12 02:48:49.180911740 +0000 2024-09-12T02:54:55.779 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-09-12 02:48:49.180911740 +0000 2024-09-12T02:54:55.779 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T02:54:55.844 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-09-12T02:54:55.845 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-09-12T02:54:55.845 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000190822 s, 2.7 MB/s 2024-09-12T02:54:55.846 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T02:54:55.903 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_4 2024-09-12T02:54:55.958 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T02:54:55.958 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:54:55.959 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-12T02:54:55.959 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:54:55.959 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:54:55.959 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-09-12 02:54:44.693149628 +0000 2024-09-12T02:54:55.959 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-09-12 02:48:49.515902046 +0000 2024-09-12T02:54:55.959 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-09-12 02:48:49.515902046 +0000 2024-09-12T02:54:55.959 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-09-12 02:48:49.515902046 +0000 2024-09-12T02:54:55.959 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T02:54:56.022 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-09-12T02:54:56.022 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-09-12T02:54:56.022 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000216881 s, 2.4 MB/s 2024-09-12T02:54:56.023 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T02:54:56.079 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-12T02:54:56.079 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T02:54:56.099 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T02:54:56.099 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_1 2024-09-12T02:54:56.158 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T02:54:56.158 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:54:56.159 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-12T02:54:56.159 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:54:56.159 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:54:56.159 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-12 02:54:35.468613500 +0000 2024-09-12T02:54:56.159 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-12 02:48:56.163314663 +0000 2024-09-12T02:54:56.159 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-12 02:48:56.163314663 +0000 2024-09-12T02:54:56.159 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-12 02:48:56.163314663 +0000 2024-09-12T02:54:56.159 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T02:54:56.226 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-12T02:54:56.226 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-12T02:54:56.226 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.00023163 s, 2.2 MB/s 2024-09-12T02:54:56.228 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T02:54:56.286 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_2 2024-09-12T02:54:56.344 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T02:54:56.344 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:54:56.344 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-12T02:54:56.344 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:54:56.344 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:54:56.344 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-12 02:54:35.469613473 +0000 2024-09-12T02:54:56.344 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-12 02:48:56.471305989 +0000 2024-09-12T02:54:56.344 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-12 02:48:56.471305989 +0000 2024-09-12T02:54:56.344 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-12 02:48:56.471305989 +0000 2024-09-12T02:54:56.345 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T02:54:56.407 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:56 smithi183 ceph-mon[35535]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:56.408 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:56 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:56.408 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:56 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:56.408 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:56 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:54:56.408 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:56 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:54:56.408 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:56 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:56.410 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-12T02:54:56.410 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-12T02:54:56.410 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.0001926 s, 2.7 MB/s 2024-09-12T02:54:56.412 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T02:54:56.470 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_3 2024-09-12T02:54:56.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:56 smithi177 ceph-mon[35305]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:56.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:56 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:56.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:56 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:56.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:56 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:54:56.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:56 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:54:56.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:56 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:56.527 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T02:54:56.527 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:54:56.527 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-12T02:54:56.527 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:54:56.527 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:54:56.527 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-12 02:54:35.469613473 +0000 2024-09-12T02:54:56.527 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-12 02:48:56.796296834 +0000 2024-09-12T02:54:56.527 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-12 02:48:56.796296834 +0000 2024-09-12T02:54:56.527 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-12 02:48:56.796296834 +0000 2024-09-12T02:54:56.528 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T02:54:56.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:56 smithi116 ceph-mon[31410]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:56.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:56 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:56.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:56 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:56.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:56 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:54:56.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:56 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:54:56.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:56 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:56.592 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-12T02:54:56.592 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-12T02:54:56.593 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000188254 s, 2.7 MB/s 2024-09-12T02:54:56.594 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T02:54:56.651 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_4 2024-09-12T02:54:56.707 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T02:54:56.708 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:54:56.708 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-12T02:54:56.708 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:54:56.708 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:54:56.708 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-12 02:54:35.469613473 +0000 2024-09-12T02:54:56.708 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-12 02:48:57.130287428 +0000 2024-09-12T02:54:56.708 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-12 02:48:57.130287428 +0000 2024-09-12T02:54:56.708 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-12 02:48:57.130287428 +0000 2024-09-12T02:54:56.708 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T02:54:56.772 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-12T02:54:56.772 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-12T02:54:56.772 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000266309 s, 1.9 MB/s 2024-09-12T02:54:56.773 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T02:54:56.830 INFO:tasks.cephadm:Deploying osd.0 on smithi116 with /dev/vg_nvme/lv_4... 2024-09-12T02:54:56.830 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T02:54:57.083 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:54:57.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:57 smithi177 ceph-mon[35305]: Reconfiguring mon.b (monmap changed)... 2024-09-12T02:54:57.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:57 smithi177 ceph-mon[35305]: Reconfiguring daemon mon.b on smithi177 2024-09-12T02:54:57.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:57 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:57.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:57 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:57.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:57 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:54:57.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:57 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:54:57.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:57 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:57.380 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:57 smithi116 ceph-mon[31410]: Reconfiguring mon.b (monmap changed)... 2024-09-12T02:54:57.380 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:57 smithi116 ceph-mon[31410]: Reconfiguring daemon mon.b on smithi177 2024-09-12T02:54:57.381 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:57 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:57.381 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:57 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:57.381 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:57 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:54:57.381 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:57 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:54:57.381 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:57 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:57.448 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:57 smithi183 ceph-mon[35535]: Reconfiguring mon.b (monmap changed)... 2024-09-12T02:54:57.448 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:57 smithi183 ceph-mon[35535]: Reconfiguring daemon mon.b on smithi177 2024-09-12T02:54:57.448 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:57 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:57.448 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:57 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:57.448 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:57 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:54:57.448 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:57 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:54:57.448 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:57 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:58.272 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:58 smithi183 ceph-mon[35535]: Reconfiguring mon.c (monmap changed)... 2024-09-12T02:54:58.273 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:58 smithi183 ceph-mon[35535]: Reconfiguring daemon mon.c on smithi183 2024-09-12T02:54:58.273 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:58 smithi183 ceph-mon[35535]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:58.469 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:58 smithi116 ceph-mon[31410]: Reconfiguring mon.c (monmap changed)... 2024-09-12T02:54:58.469 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:58 smithi116 ceph-mon[31410]: Reconfiguring daemon mon.c on smithi183 2024-09-12T02:54:58.469 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:58 smithi116 ceph-mon[31410]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:58.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:58 smithi177 ceph-mon[35305]: Reconfiguring mon.c (monmap changed)... 2024-09-12T02:54:58.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:58 smithi177 ceph-mon[35305]: Reconfiguring daemon mon.c on smithi183 2024-09-12T02:54:58.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:58 smithi177 ceph-mon[35305]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:59.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:59 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:59.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:59 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:59.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:59 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:59.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:59 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:59.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:59 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:54:59.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:59 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:59.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:59 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:54:59.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:59 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T02:54:59.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:59 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:54:59.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:59 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:59.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:59 smithi116 ceph-mon[31410]: Deploying daemon mgr.z on smithi183 2024-09-12T02:54:59.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:54:59 smithi116 ceph-mon[31410]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:59.598 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:54:59.624 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi116:vg_nvme/lv_4 2024-09-12T02:54:59.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:59 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:59.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:59 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:59.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:59 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:59.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:59 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:59.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:59 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:54:59.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:59 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:59.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:59 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:54:59.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:59 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T02:54:59.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:59 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:54:59.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:59 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:59.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:59 smithi183 ceph-mon[35535]: Deploying daemon mgr.z on smithi183 2024-09-12T02:54:59.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:54:59 smithi183 ceph-mon[35535]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:59.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:59 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:59.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:59 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:59.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:59 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:59.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:59 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:59.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:59 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:54:59.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:59 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:54:59.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:59 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:54:59.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:59 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T02:54:59.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:59 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:54:59.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:59 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:59.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:59 smithi177 ceph-mon[35305]: Deploying daemon mgr.z on smithi183 2024-09-12T02:54:59.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:54:59 smithi177 ceph-mon[35305]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:54:59.862 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:55:00.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.317+0000 7f09b03b4640 1 -- 172.21.15.116:0/1842689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09a8100b00 msgr2=0x7f09a8100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:00.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.317+0000 7f09b03b4640 1 --2- 172.21.15.116:0/1842689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09a8100b00 0x7f09a8100f00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0998009ca0 tx=0x7f099802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:55:00.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.317+0000 7f09b03b4640 1 -- 172.21.15.116:0/1842689 shutdown_connections 2024-09-12T02:55:00.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.317+0000 7f09b03b4640 1 --2- 172.21.15.116:0/1842689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09a81026c0 0x7f09a8108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:00.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.317+0000 7f09b03b4640 1 --2- 172.21.15.116:0/1842689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09a8101d00 0x7f09a8102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:00.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.317+0000 7f09b03b4640 1 --2- 172.21.15.116:0/1842689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09a8100b00 0x7f09a8100f00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:00.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.317+0000 7f09b03b4640 1 -- 172.21.15.116:0/1842689 >> 172.21.15.116:0/1842689 conn(0x7f09a80fc290 msgr2=0x7f09a80fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:00.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.317+0000 7f09b03b4640 1 -- 172.21.15.116:0/1842689 shutdown_connections 2024-09-12T02:55:00.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.317+0000 7f09b03b4640 1 -- 172.21.15.116:0/1842689 wait complete. 2024-09-12T02:55:00.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.317+0000 7f09b03b4640 1 Processor -- start 2024-09-12T02:55:00.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.317+0000 7f09b03b4640 1 -- start start 2024-09-12T02:55:00.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.317+0000 7f09b03b4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09a8100b00 0x7f09a8197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:00.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.317+0000 7f09b03b4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09a8101d00 0x7f09a81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:00.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.317+0000 7f09b03b4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09a81026c0 0x7f09a819f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:00.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.318+0000 7f09b03b4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f09a810c790 con 0x7f09a81026c0 2024-09-12T02:55:00.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.318+0000 7f09b03b4640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f09a810c610 con 0x7f09a8101d00 2024-09-12T02:55:00.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.318+0000 7f09b03b4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f09a810c910 con 0x7f09a8100b00 2024-09-12T02:55:00.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.318+0000 7f09ae951640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09a81026c0 0x7f09a819f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:00.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.318+0000 7f09ae951640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09a81026c0 0x7f09a819f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56694/0 (socket says 172.21.15.116:56694) 2024-09-12T02:55:00.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.318+0000 7f09ae951640 1 -- 172.21.15.116:0/4082596216 learned_addr learned my addr 172.21.15.116:0/4082596216 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:55:00.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.318+0000 7f09ae150640 1 --2- 172.21.15.116:0/4082596216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09a8100b00 0x7f09a8197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:00.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.318+0000 7f09ad94f640 1 --2- 172.21.15.116:0/4082596216 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09a8101d00 0x7f09a81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:00.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.318+0000 7f09ae951640 1 -- 172.21.15.116:0/4082596216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09a8100b00 msgr2=0x7f09a8197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:00.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.318+0000 7f09ae951640 1 --2- 172.21.15.116:0/4082596216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09a8100b00 0x7f09a8197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:00.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.318+0000 7f09ae951640 1 -- 172.21.15.116:0/4082596216 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09a8101d00 msgr2=0x7f09a81983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:00.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.318+0000 7f09ae951640 1 --2- 172.21.15.116:0/4082596216 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09a8101d00 0x7f09a81983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:00.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.318+0000 7f09ae951640 1 -- 172.21.15.116:0/4082596216 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a819fad0 con 0x7f09a81026c0 2024-09-12T02:55:00.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.318+0000 7f09ae951640 1 --2- 172.21.15.116:0/4082596216 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09a81026c0 0x7f09a819f460 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f09a000d750 tx=0x7f09a000db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:00.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.318+0000 7f09a77fe640 1 -- 172.21.15.116:0/4082596216 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0017070 con 0x7f09a81026c0 2024-09-12T02:55:00.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.318+0000 7f09b03b4640 1 -- 172.21.15.116:0/4082596216 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a819fdc0 con 0x7f09a81026c0 2024-09-12T02:55:00.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.318+0000 7f09a77fe640 1 -- 172.21.15.116:0/4082596216 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09a00097c0 con 0x7f09a81026c0 2024-09-12T02:55:00.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.318+0000 7f09a77fe640 1 -- 172.21.15.116:0/4082596216 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0004d00 con 0x7f09a81026c0 2024-09-12T02:55:00.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.318+0000 7f09b03b4640 1 -- 172.21.15.116:0/4082596216 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a81a02d0 con 0x7f09a81026c0 2024-09-12T02:55:00.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.319+0000 7f09a77fe640 1 -- 172.21.15.116:0/4082596216 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f09a0009970 con 0x7f09a81026c0 2024-09-12T02:55:00.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.319+0000 7f09b03b4640 1 -- 172.21.15.116:0/4082596216 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09a8100f00 con 0x7f09a81026c0 2024-09-12T02:55:00.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.319+0000 7f09a77fe640 1 --2- 172.21.15.116:0/4082596216 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f09900392c0 0x7f099003b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:00.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.320+0000 7f09a77fe640 1 -- 172.21.15.116:0/4082596216 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f09a004bc60 con 0x7f09a81026c0 2024-09-12T02:55:00.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.320+0000 7f09ae150640 1 --2- 172.21.15.116:0/4082596216 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f09900392c0 0x7f099003b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:00.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.320+0000 7f09ae150640 1 --2- 172.21.15.116:0/4082596216 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f09900392c0 0x7f099003b780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0998006010 tx=0x7f0998000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:00.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.324+0000 7f09a77fe640 1 -- 172.21.15.116:0/4082596216 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f09a0019310 con 0x7f09a81026c0 2024-09-12T02:55:00.427 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:00 smithi183 systemd[1]: Starting Ceph mgr.z for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T02:55:00.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:00.499+0000 7f09b03b4640 1 -- 172.21.15.116:0/4082596216 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f09a80fe4a0 con 0x7f09900392c0 2024-09-12T02:55:00.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:00 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:55:00.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:00 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:55:00.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:00 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:00.703 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:00 smithi183 podman[37327]: 2024-09-12 02:55:00.328735546 +0000 UTC m=+0.015852413 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:55:00.704 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:00 smithi183 podman[37327]: 2024-09-12 02:55:00.431066807 +0000 UTC m=+0.118183677 container create 74021d43cead5c866c9a20949dc0bdb48bb033987342011401155c05b1ffd400 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-b53b6d3, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-12T02:55:00.704 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:00 smithi183 podman[37327]: 2024-09-12 02:55:00.633726678 +0000 UTC m=+0.320843552 container init 74021d43cead5c866c9a20949dc0bdb48bb033987342011401155c05b1ffd400 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-12T02:55:00.704 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:00 smithi183 podman[37327]: 2024-09-12 02:55:00.636772353 +0000 UTC m=+0.323889222 container start 74021d43cead5c866c9a20949dc0bdb48bb033987342011401155c05b1ffd400 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-12T02:55:00.704 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:00 smithi183 ceph-mgr[37346]: -- 172.21.15.183:0/3622770512 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f84a3f90e0 con 0x55f84a3d7400 2024-09-12T02:55:00.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:00 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:55:00.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:00 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:55:00.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:00 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:00.965 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:00 smithi183 bash[37327]: 74021d43cead5c866c9a20949dc0bdb48bb033987342011401155c05b1ffd400 2024-09-12T02:55:00.965 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:00 smithi183 systemd[1]: Started Ceph mgr.z for eb1ac602-70b1-11ef-bcea-c7b262605968. 2024-09-12T02:55:00.965 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:00 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z[37342]: 2024-09-12T02:55:00.758+0000 7fd26b8c8040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T02:55:00.965 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:00 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z[37342]: 2024-09-12T02:55:00.818+0000 7fd26b8c8040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T02:55:00.965 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:00 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z[37342]: 2024-09-12T02:55:00.964+0000 7fd26b8c8040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T02:55:00.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:00 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:55:00.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:00 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:55:00.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:00 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:01.316 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:01 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z[37342]: 2024-09-12T02:55:01.028+0000 7fd26b8c8040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T02:55:01.317 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:01 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z[37342]: 2024-09-12T02:55:01.176+0000 7fd26b8c8040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:55:01.317 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:01 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z[37342]: 2024-09-12T02:55:01.315+0000 7fd26b8c8040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T02:55:01.876 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:01 smithi116 ceph-mon[31410]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:55:01.876 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:01 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:01.876 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:01 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:01.876 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:01 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:01.876 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:01 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:55:01.876 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:01 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T02:55:01.877 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:01 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:55:01.877 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:01 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:01.877 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:01 smithi116 ceph-mon[31410]: Deploying daemon mgr.y on smithi177 2024-09-12T02:55:01.877 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:01 smithi116 ceph-mon[31410]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:01.953 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:01 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z[37342]: 2024-09-12T02:55:01.632+0000 7fd26b8c8040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T02:55:01.953 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:01 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z[37342]: 2024-09-12T02:55:01.692+0000 7fd26b8c8040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T02:55:01.953 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:01 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z[37342]: 2024-09-12T02:55:01.878+0000 7fd26b8c8040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T02:55:01.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:01 smithi183 ceph-mon[35535]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:55:01.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:01 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:01.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:01 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:01.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:01 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:01.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:01 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:55:01.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:01 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T02:55:01.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:01 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:55:01.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:01 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:01.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:01 smithi183 ceph-mon[35535]: Deploying daemon mgr.y on smithi177 2024-09-12T02:55:01.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:01 smithi183 ceph-mon[35535]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:02.120 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:01 smithi177 ceph-mon[35305]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:55:02.120 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:01 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:02.120 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:01 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:02.121 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:01 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:02.121 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:01 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:55:02.121 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:01 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T02:55:02.121 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:01 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:55:02.121 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:01 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:02.121 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:01 smithi177 ceph-mon[35305]: Deploying daemon mgr.y on smithi177 2024-09-12T02:55:02.121 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:01 smithi177 ceph-mon[35305]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:02.745 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:02 smithi177 systemd[1]: Starting Ceph mgr.y for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T02:55:03.035 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:02 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/522713819' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8cf5a18-dc6f-4bb6-bd74-78adda310118"}]: dispatch 2024-09-12T02:55:03.035 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:02 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/522713819' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8cf5a18-dc6f-4bb6-bd74-78adda310118"}]': finished 2024-09-12T02:55:03.035 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:02 smithi177 ceph-mon[35305]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T02:55:03.035 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:02 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:55:03.035 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:02 smithi177 podman[36657]: 2024-09-12 02:55:02.679393016 +0000 UTC m=+0.015818299 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:55:03.035 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:02 smithi177 podman[36657]: 2024-09-12 02:55:02.789630794 +0000 UTC m=+0.126056119 container create e90c4be67b8a61d1077c97f1607c9f22b3c6d3b6322bbe65600c84dba3c8c1d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3) 2024-09-12T02:55:03.035 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:03 smithi177 podman[36657]: 2024-09-12 02:55:03.00839576 +0000 UTC m=+0.344821043 container init e90c4be67b8a61d1077c97f1607c9f22b3c6d3b6322bbe65600c84dba3c8c1d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-12T02:55:03.036 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:03 smithi177 podman[36657]: 2024-09-12 02:55:03.011329346 +0000 UTC m=+0.347754630 container start e90c4be67b8a61d1077c97f1607c9f22b3c6d3b6322bbe65600c84dba3c8c1d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-12T02:55:03.036 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:03 smithi177 ceph-mgr[36676]: -- 172.21.15.177:0/129724011 <== mon.2 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5623afb630e0 con 0x5623afb41000 2024-09-12T02:55:03.153 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:02 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/522713819' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8cf5a18-dc6f-4bb6-bd74-78adda310118"}]: dispatch 2024-09-12T02:55:03.153 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:02 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/522713819' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8cf5a18-dc6f-4bb6-bd74-78adda310118"}]': finished 2024-09-12T02:55:03.153 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:02 smithi116 ceph-mon[31410]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T02:55:03.153 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:02 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:55:03.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:02 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/522713819' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8cf5a18-dc6f-4bb6-bd74-78adda310118"}]: dispatch 2024-09-12T02:55:03.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:02 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/522713819' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8cf5a18-dc6f-4bb6-bd74-78adda310118"}]': finished 2024-09-12T02:55:03.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:02 smithi183 ceph-mon[35535]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T02:55:03.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:02 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:55:03.203 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:02 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z[37342]: 2024-09-12T02:55:02.761+0000 7fd26b8c8040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T02:55:03.204 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:02 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z[37342]: 2024-09-12T02:55:02.828+0000 7fd26b8c8040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T02:55:03.350 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:03 smithi177 bash[36657]: e90c4be67b8a61d1077c97f1607c9f22b3c6d3b6322bbe65600c84dba3c8c1d7 2024-09-12T02:55:03.350 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:03 smithi177 systemd[1]: Started Ceph mgr.y for eb1ac602-70b1-11ef-bcea-c7b262605968. 2024-09-12T02:55:03.350 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:03 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y[36672]: 2024-09-12T02:55:03.132+0000 7fcd96079040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T02:55:03.350 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:03 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y[36672]: 2024-09-12T02:55:03.193+0000 7fcd96079040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T02:55:03.350 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:03 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y[36672]: 2024-09-12T02:55:03.349+0000 7fcd96079040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T02:55:03.730 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:03 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y[36672]: 2024-09-12T02:55:03.419+0000 7fcd96079040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T02:55:03.730 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:03 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y[36672]: 2024-09-12T02:55:03.579+0000 7fcd96079040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:55:03.730 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:03 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y[36672]: 2024-09-12T02:55:03.729+0000 7fcd96079040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T02:55:03.846 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:03 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z[37342]: 2024-09-12T02:55:03.568+0000 7fd26b8c8040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T02:55:03.846 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:03 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z[37342]: 2024-09-12T02:55:03.703+0000 7fd26b8c8040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T02:55:03.846 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:03 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z[37342]: 2024-09-12T02:55:03.845+0000 7fd26b8c8040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:55:04.133 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:03 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z[37342]: 2024-09-12T02:55:03.907+0000 7fd26b8c8040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T02:55:04.133 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:04 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z[37342]: 2024-09-12T02:55:04.063+0000 7fd26b8c8040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T02:55:04.133 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:04 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z[37342]: 2024-09-12T02:55:04.131+0000 7fd26b8c8040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T02:55:04.308 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:04 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y[36672]: 2024-09-12T02:55:04.056+0000 7fcd96079040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T02:55:04.308 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:04 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y[36672]: 2024-09-12T02:55:04.118+0000 7fcd96079040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T02:55:04.308 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:04 smithi177 ceph-mon[35305]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:04.308 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:04 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:04.308 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:04 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:04.308 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:04 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:04.308 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:04 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:04.308 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:04 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:04.309 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:04 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3053590771' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:55:04.421 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:04 smithi183 ceph-mon[35535]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:04.421 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:04 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:04.421 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:04 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:04.421 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:04 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:04.421 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:04 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:04.421 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:04 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:04.421 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:04 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3053590771' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:55:04.422 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:04 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z[37342]: 2024-09-12T02:55:04.194+0000 7fd26b8c8040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T02:55:04.422 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:04 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z[37342]: 2024-09-12T02:55:04.354+0000 7fd26b8c8040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T02:55:04.422 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:04 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z[37342]: 2024-09-12T02:55:04.420+0000 7fd26b8c8040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T02:55:04.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:04 smithi116 ceph-mon[31410]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:04.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:04 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:04.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:04 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:04.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:04 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:04.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:04 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:04.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:04 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:04.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:04 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3053590771' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:55:04.702 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:04 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z[37342]: 2024-09-12T02:55:04.491+0000 7fd26b8c8040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T02:55:04.703 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:04 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z[37342]: 2024-09-12T02:55:04.579+0000 7fd26b8c8040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T02:55:04.703 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:04 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z[37342]: 2024-09-12T02:55:04.641+0000 7fd26b8c8040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T02:55:04.703 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:04 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z[37342]: 2024-09-12T02:55:04.701+0000 7fd26b8c8040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T02:55:04.745 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:04 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y[36672]: 2024-09-12T02:55:04.307+0000 7fcd96079040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T02:55:05.495 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:05 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y[36672]: 2024-09-12T02:55:05.196+0000 7fcd96079040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T02:55:05.495 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:05 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y[36672]: 2024-09-12T02:55:05.262+0000 7fcd96079040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T02:55:05.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:05 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:05.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:05 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:05.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:05 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:05.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:05 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:05.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:05 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:05.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:05 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:05.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:05 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:05.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:05 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:05.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:05 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:05.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:05 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:05.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:05 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:05.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:05 smithi177 ceph-mon[35305]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:05.703 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 02:55:05 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z[37342]: 2024-09-12T02:55:05.490+0000 7fd26b8c8040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T02:55:05.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:05 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:05.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:05 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:05.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:05 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:05.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:05 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:05.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:05 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:05.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:05 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:05.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:05 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:05.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:05 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:05.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:05 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:05.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:05 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:05.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:05 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:05.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:05 smithi183 ceph-mon[35535]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:05.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:05 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:05.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:05 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:05.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:05 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:05.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:05 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:05.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:05 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:05.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:05 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:05.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:05 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:05.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:05 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:05.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:05 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:05.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:05 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:05.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:05 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:05.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:05 smithi116 ceph-mon[31410]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:06.223 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:05 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y[36672]: 2024-09-12T02:55:05.969+0000 7fcd96079040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T02:55:06.223 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:06 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y[36672]: 2024-09-12T02:55:06.082+0000 7fcd96079040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T02:55:06.223 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:06 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y[36672]: 2024-09-12T02:55:06.222+0000 7fcd96079040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:55:06.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:06.374+0000 7f09a77fe640 1 -- 172.21.15.116:0/4082596216 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mgrmap(e 14) ==== 92061+0+0 (secure 0 0 0) 0x7f09a00811b0 con 0x7f09a81026c0 2024-09-12T02:55:06.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:06 smithi177 ceph-mon[35305]: Standby manager daemon z started 2024-09-12T02:55:06.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:06 smithi177 ceph-mon[35305]: from='mgr.? 172.21.15.183:0/1485220149' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T02:55:06.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:06 smithi177 ceph-mon[35305]: from='mgr.? 172.21.15.183:0/1485220149' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:55:06.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:06 smithi177 ceph-mon[35305]: from='mgr.? 172.21.15.183:0/1485220149' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T02:55:06.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:06 smithi177 ceph-mon[35305]: from='mgr.? 172.21.15.183:0/1485220149' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:55:06.495 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:06 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y[36672]: 2024-09-12T02:55:06.282+0000 7fcd96079040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T02:55:06.495 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:06 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y[36672]: 2024-09-12T02:55:06.430+0000 7fcd96079040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T02:55:06.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:06 smithi183 ceph-mon[35535]: Standby manager daemon z started 2024-09-12T02:55:06.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:06 smithi183 ceph-mon[35535]: from='mgr.? 172.21.15.183:0/1485220149' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T02:55:06.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:06 smithi183 ceph-mon[35535]: from='mgr.? 172.21.15.183:0/1485220149' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:55:06.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:06 smithi183 ceph-mon[35535]: from='mgr.? 172.21.15.183:0/1485220149' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T02:55:06.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:06 smithi183 ceph-mon[35535]: from='mgr.? 172.21.15.183:0/1485220149' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:55:06.775 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:06 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y[36672]: 2024-09-12T02:55:06.497+0000 7fcd96079040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T02:55:06.775 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:06 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y[36672]: 2024-09-12T02:55:06.555+0000 7fcd96079040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T02:55:06.776 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:06 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y[36672]: 2024-09-12T02:55:06.711+0000 7fcd96079040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T02:55:06.776 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:06 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y[36672]: 2024-09-12T02:55:06.775+0000 7fcd96079040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T02:55:06.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:06 smithi116 ceph-mon[31410]: Standby manager daemon z started 2024-09-12T02:55:06.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:06 smithi116 ceph-mon[31410]: from='mgr.? 172.21.15.183:0/1485220149' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T02:55:06.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:06 smithi116 ceph-mon[31410]: from='mgr.? 172.21.15.183:0/1485220149' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:55:06.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:06 smithi116 ceph-mon[31410]: from='mgr.? 172.21.15.183:0/1485220149' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T02:55:06.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:06 smithi116 ceph-mon[31410]: from='mgr.? 172.21.15.183:0/1485220149' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:55:07.036 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:06 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y[36672]: 2024-09-12T02:55:06.840+0000 7fcd96079040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T02:55:07.037 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:06 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y[36672]: 2024-09-12T02:55:06.920+0000 7fcd96079040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T02:55:07.037 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:06 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y[36672]: 2024-09-12T02:55:06.978+0000 7fcd96079040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T02:55:07.037 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:07 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y[36672]: 2024-09-12T02:55:07.036+0000 7fcd96079040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T02:55:07.631 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:07 smithi116 ceph-mon[31410]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-12T02:55:07.632 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:07 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T02:55:07.632 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:07 smithi116 ceph-mon[31410]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:07 smithi183 ceph-mon[35535]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-12T02:55:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:07 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T02:55:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:07 smithi183 ceph-mon[35535]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:07.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:07 smithi177 ceph-mon[35305]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-12T02:55:07.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:07 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T02:55:07.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:07 smithi177 ceph-mon[35305]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:08.244 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 02:55:07 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y[36672]: 2024-09-12T02:55:07.812+0000 7fcd96079040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T02:55:08.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:08.386+0000 7f09a77fe640 1 -- 172.21.15.116:0/4082596216 <== mon.0 v2:172.21.15.116:3300/0 8 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f09a0010460 con 0x7f09a81026c0 2024-09-12T02:55:08.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:08 smithi116 ceph-mon[31410]: Standby manager daemon y started 2024-09-12T02:55:08.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:08 smithi116 ceph-mon[31410]: from='mgr.? 172.21.15.177:0/1276582622' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T02:55:08.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:08 smithi116 ceph-mon[31410]: from='mgr.? 172.21.15.177:0/1276582622' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:55:08.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:08 smithi116 ceph-mon[31410]: from='mgr.? 172.21.15.177:0/1276582622' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T02:55:08.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:08 smithi116 ceph-mon[31410]: from='mgr.? 172.21.15.177:0/1276582622' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:55:08.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:08 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:08 smithi183 ceph-mon[35535]: Standby manager daemon y started 2024-09-12T02:55:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:08 smithi183 ceph-mon[35535]: from='mgr.? 172.21.15.177:0/1276582622' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T02:55:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:08 smithi183 ceph-mon[35535]: from='mgr.? 172.21.15.177:0/1276582622' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:55:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:08 smithi183 ceph-mon[35535]: from='mgr.? 172.21.15.177:0/1276582622' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T02:55:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:08 smithi183 ceph-mon[35535]: from='mgr.? 172.21.15.177:0/1276582622' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:55:08.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:08 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:08.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:08 smithi177 ceph-mon[35305]: Standby manager daemon y started 2024-09-12T02:55:08.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:08 smithi177 ceph-mon[35305]: from='mgr.? 172.21.15.177:0/1276582622' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T02:55:08.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:08 smithi177 ceph-mon[35305]: from='mgr.? 172.21.15.177:0/1276582622' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:55:08.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:08 smithi177 ceph-mon[35305]: from='mgr.? 172.21.15.177:0/1276582622' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T02:55:08.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:08 smithi177 ceph-mon[35305]: from='mgr.? 172.21.15.177:0/1276582622' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:55:08.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:08 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:09.700 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:09 smithi116 ceph-mon[31410]: mgrmap e15: x(active, since 65s), standbys: z, y 2024-09-12T02:55:09.700 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:09 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T02:55:09.700 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:09 smithi116 ceph-mon[31410]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:09.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:09 smithi183 ceph-mon[35535]: mgrmap e15: x(active, since 65s), standbys: z, y 2024-09-12T02:55:09.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:09 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T02:55:09.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:09 smithi183 ceph-mon[35535]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:09.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:09 smithi177 ceph-mon[35305]: mgrmap e15: x(active, since 65s), standbys: z, y 2024-09-12T02:55:09.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:09 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T02:55:09.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:09 smithi177 ceph-mon[35305]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:12.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:12 smithi116 ceph-mon[31410]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:12.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:12 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T02:55:12.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:12 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:12.406 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:12 smithi183 ceph-mon[35535]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:12.406 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:12 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T02:55:12.406 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:12 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:12.448 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:12 smithi177 ceph-mon[35305]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:12.448 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:12 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T02:55:12.448 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:12 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:13.177 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:13 smithi116 ceph-mon[31410]: Deploying daemon osd.0 on smithi116 2024-09-12T02:55:13.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:13 smithi183 ceph-mon[35535]: Deploying daemon osd.0 on smithi116 2024-09-12T02:55:13.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:13 smithi177 ceph-mon[35305]: Deploying daemon osd.0 on smithi116 2024-09-12T02:55:14.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:14 smithi183 ceph-mon[35535]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:14.486 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:14 smithi116 ceph-mon[31410]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:14.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:14 smithi177 ceph-mon[35305]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:16.404 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:16 smithi116 ceph-mon[31410]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:16.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:16 smithi183 ceph-mon[35535]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:16.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:16 smithi177 ceph-mon[35305]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:18.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:18 smithi116 ceph-mon[31410]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:18.649 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:18 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:18.649 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:18 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:18.649 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:18 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:18.650 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:18 smithi177 ceph-mon[35305]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:18.650 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:18 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:18.650 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:18 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:18.650 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:18 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:18.651 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:18 smithi183 ceph-mon[35535]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:18.651 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:18 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:18.651 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:18 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:18.651 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:18 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:20.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:20 smithi183 ceph-mon[35535]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:20.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:20 smithi177 ceph-mon[35305]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:20.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:20 smithi116 ceph-mon[31410]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:21.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:21 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:21.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:21 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:21.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:21 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:21.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:21 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:21.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:21 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:21.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:21 smithi177 ceph-mon[35305]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:21.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:21 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:21.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:21 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:21.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:21 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:21.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:21 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:21.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:21 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:21.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:21 smithi116 ceph-mon[31410]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:21.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:21 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:21.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:21 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:21.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:21 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:21.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:21 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:21.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:21 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:21.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:21 smithi183 ceph-mon[35535]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:21.853 INFO:teuthology.orchestra.run.smithi116.stdout:Created osd(s) 0 on host 'smithi116' 2024-09-12T02:55:21.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:21.853+0000 7f09a77fe640 1 -- 172.21.15.116:0/4082596216 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f09a80fe4a0 con 0x7f09900392c0 2024-09-12T02:55:21.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:21.855+0000 7f09b03b4640 1 -- 172.21.15.116:0/4082596216 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f09900392c0 msgr2=0x7f099003b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:21.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:21.855+0000 7f09b03b4640 1 --2- 172.21.15.116:0/4082596216 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f09900392c0 0x7f099003b780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0998006010 tx=0x7f0998000f40 comp rx=0 tx=0).stop 2024-09-12T02:55:21.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:21.855+0000 7f09b03b4640 1 -- 172.21.15.116:0/4082596216 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09a81026c0 msgr2=0x7f09a819f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:21.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:21.855+0000 7f09b03b4640 1 --2- 172.21.15.116:0/4082596216 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09a81026c0 0x7f09a819f460 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f09a000d750 tx=0x7f09a000db50 comp rx=0 tx=0).stop 2024-09-12T02:55:21.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:21.855+0000 7f09b03b4640 1 -- 172.21.15.116:0/4082596216 shutdown_connections 2024-09-12T02:55:21.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:21.855+0000 7f09b03b4640 1 --2- 172.21.15.116:0/4082596216 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f09900392c0 0x7f099003b780 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:21.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:21.855+0000 7f09b03b4640 1 --2- 172.21.15.116:0/4082596216 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09a81026c0 0x7f09a819f460 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:21.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:21.855+0000 7f09b03b4640 1 --2- 172.21.15.116:0/4082596216 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09a8101d00 0x7f09a81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:21.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:21.855+0000 7f09b03b4640 1 --2- 172.21.15.116:0/4082596216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09a8100b00 0x7f09a8197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:21.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:21.855+0000 7f09b03b4640 1 -- 172.21.15.116:0/4082596216 >> 172.21.15.116:0/4082596216 conn(0x7f09a80fc290 msgr2=0x7f09a80fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:21.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:21.855+0000 7f09b03b4640 1 -- 172.21.15.116:0/4082596216 shutdown_connections 2024-09-12T02:55:21.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:55:21.855+0000 7f09b03b4640 1 -- 172.21.15.116:0/4082596216 wait complete. 2024-09-12T02:55:22.492 DEBUG:teuthology.orchestra.run.smithi116:osd.0> sudo journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.0.service 2024-09-12T02:55:22.495 INFO:tasks.cephadm:Deploying osd.1 on smithi177 with /dev/vg_nvme/lv_4... 2024-09-12T02:55:22.495 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T02:55:22.716 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:22 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:22.717 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:22 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:22.717 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:22 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:22.732 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:55:22.853 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:22 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:22.853 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:22 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:22.853 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:22 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:22.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:22 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:22.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:22 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:22.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:22 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:23.872 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:23 smithi177 ceph-mon[35305]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:23.935 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:23 smithi116 ceph-mon[31410]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:23.936 INFO:journalctl@ceph.osd.0.smithi116.stdout:Sep 12 02:55:23 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-0[45396]: 2024-09-12T02:55:23.731+0000 7f02b6553640 -1 osd.0 0 log_to_monitors true 2024-09-12T02:55:23.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:23 smithi183 ceph-mon[35535]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:24.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:24 smithi177 ceph-mon[35305]: from='osd.0 [v2:172.21.15.116:6802/862366853,v1:172.21.15.116:6803/862366853]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T02:55:24.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:24 smithi183 ceph-mon[35535]: from='osd.0 [v2:172.21.15.116:6802/862366853,v1:172.21.15.116:6803/862366853]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T02:55:25.033 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-12T02:55:25.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:24 smithi116 ceph-mon[31410]: from='osd.0 [v2:172.21.15.116:6802/862366853,v1:172.21.15.116:6803/862366853]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T02:55:25.058 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi177:vg_nvme/lv_4 2024-09-12T02:55:25.309 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:55:25.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:25 smithi116 ceph-mon[31410]: from='osd.0 [v2:172.21.15.116:6802/862366853,v1:172.21.15.116:6803/862366853]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T02:55:25.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:25 smithi116 ceph-mon[31410]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T02:55:25.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:25 smithi116 ceph-mon[31410]: from='osd.0 [v2:172.21.15.116:6802/862366853,v1:172.21.15.116:6803/862366853]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-09-12T02:55:25.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:25 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:55:25.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:25 smithi116 ceph-mon[31410]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:25.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:25 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:25.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:25 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:25.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:25 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:25.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:25 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:25.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:25 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:25.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:25 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:25.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:25 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:25.817 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.768+0000 7fd409b54640 1 -- 172.21.15.177:0/3000540721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd404102e00 msgr2=0x7fd404103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:25.817 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.768+0000 7fd409b54640 1 --2- 172.21.15.177:0/3000540721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd404102e00 0x7fd404103280 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8009900 tx=0x7fd3f802a410 comp rx=0 tx=0).stop 2024-09-12T02:55:25.817 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.768+0000 7fd409b54640 1 -- 172.21.15.177:0/3000540721 shutdown_connections 2024-09-12T02:55:25.817 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.768+0000 7fd409b54640 1 --2- 172.21.15.177:0/3000540721 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4041037c0 0x7fd40410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:25.817 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.768+0000 7fd409b54640 1 --2- 172.21.15.177:0/3000540721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd404102e00 0x7fd404103280 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:25.817 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.768+0000 7fd409b54640 1 --2- 172.21.15.177:0/3000540721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd404101c00 0x7fd404102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:25.817 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.768+0000 7fd409b54640 1 -- 172.21.15.177:0/3000540721 >> 172.21.15.177:0/3000540721 conn(0x7fd4040fd390 msgr2=0x7fd4040ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:25.817 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.768+0000 7fd409b54640 1 -- 172.21.15.177:0/3000540721 shutdown_connections 2024-09-12T02:55:25.817 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.768+0000 7fd409b54640 1 -- 172.21.15.177:0/3000540721 wait complete. 2024-09-12T02:55:25.817 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.768+0000 7fd409b54640 1 Processor -- start 2024-09-12T02:55:25.818 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.769+0000 7fd409b54640 1 -- start start 2024-09-12T02:55:25.818 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.769+0000 7fd409b54640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd404101c00 0x7fd404075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:25.818 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.769+0000 7fd409b54640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd404102e00 0x7fd4040745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:25.818 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.769+0000 7fd409b54640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4041037c0 0x7fd404074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:25.818 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.769+0000 7fd409b54640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd40410d880 con 0x7fd4041037c0 2024-09-12T02:55:25.818 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.769+0000 7fd409b54640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd40410d700 con 0x7fd404101c00 2024-09-12T02:55:25.818 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.769+0000 7fd409b54640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd40410da00 con 0x7fd404102e00 2024-09-12T02:55:25.818 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.769+0000 7fd4037fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd404101c00 0x7fd404075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:25.818 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.769+0000 7fd4037fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd404101c00 0x7fd404075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:39108/0 (socket says 172.21.15.177:39108) 2024-09-12T02:55:25.818 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.769+0000 7fd4037fe640 1 -- 172.21.15.177:0/875637554 learned_addr learned my addr 172.21.15.177:0/875637554 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-12T02:55:25.818 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.769+0000 7fd403fff640 1 --2- 172.21.15.177:0/875637554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4041037c0 0x7fd404074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:25.818 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.769+0000 7fd402ffd640 1 --2- 172.21.15.177:0/875637554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd404102e00 0x7fd4040745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:25.819 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.769+0000 7fd4037fe640 1 -- 172.21.15.177:0/875637554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd404102e00 msgr2=0x7fd4040745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:25.819 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.769+0000 7fd4037fe640 1 --2- 172.21.15.177:0/875637554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd404102e00 0x7fd4040745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:25.819 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.769+0000 7fd4037fe640 1 -- 172.21.15.177:0/875637554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4041037c0 msgr2=0x7fd404074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:25.819 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.769+0000 7fd4037fe640 1 --2- 172.21.15.177:0/875637554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4041037c0 0x7fd404074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:25.819 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.769+0000 7fd4037fe640 1 -- 172.21.15.177:0/875637554 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4040753e0 con 0x7fd404101c00 2024-09-12T02:55:25.819 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.769+0000 7fd4037fe640 1 --2- 172.21.15.177:0/875637554 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd404101c00 0x7fd404075f90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd3f400d950 tx=0x7fd3f400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:25.819 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.769+0000 7fd400ff9640 1 -- 172.21.15.177:0/875637554 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f4017070 con 0x7fd404101c00 2024-09-12T02:55:25.819 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.769+0000 7fd400ff9640 1 -- 172.21.15.177:0/875637554 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3f4009c30 con 0x7fd404101c00 2024-09-12T02:55:25.819 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.769+0000 7fd409b54640 1 -- 172.21.15.177:0/875637554 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4041a20c0 con 0x7fd404101c00 2024-09-12T02:55:25.819 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.770+0000 7fd400ff9640 1 -- 172.21.15.177:0/875637554 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f400f600 con 0x7fd404101c00 2024-09-12T02:55:25.819 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.770+0000 7fd409b54640 1 -- 172.21.15.177:0/875637554 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4041a25d0 con 0x7fd404101c00 2024-09-12T02:55:25.819 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.771+0000 7fd409b54640 1 -- 172.21.15.177:0/875637554 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4041090f0 con 0x7fd404101c00 2024-09-12T02:55:25.819 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.772+0000 7fd400ff9640 1 -- 172.21.15.177:0/875637554 <== mon.2 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7fd3f401a070 con 0x7fd404101c00 2024-09-12T02:55:25.820 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.772+0000 7fd400ff9640 1 --2- 172.21.15.177:0/875637554 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd3ec0a2b70 0x7fd3ec0a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:25.820 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.772+0000 7fd400ff9640 1 -- 172.21.15.177:0/875637554 <== mon.2 v2:172.21.15.177:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7fd3f40ccdb0 con 0x7fd404101c00 2024-09-12T02:55:25.820 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.772+0000 7fd402ffd640 1 --2- 172.21.15.177:0/875637554 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd3ec0a2b70 0x7fd3ec0a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:25.820 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.773+0000 7fd402ffd640 1 --2- 172.21.15.177:0/875637554 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd3ec0a2b70 0x7fd3ec0a5030 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd3f80314e0 tx=0x7fd3f8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:25.820 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.777+0000 7fd400ff9640 1 -- 172.21.15.177:0/875637554 <== mon.2 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd3f4009de0 con 0x7fd404101c00 2024-09-12T02:55:25.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:25 smithi183 ceph-mon[35535]: from='osd.0 [v2:172.21.15.116:6802/862366853,v1:172.21.15.116:6803/862366853]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T02:55:25.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:25 smithi183 ceph-mon[35535]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T02:55:25.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:25 smithi183 ceph-mon[35535]: from='osd.0 [v2:172.21.15.116:6802/862366853,v1:172.21.15.116:6803/862366853]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-09-12T02:55:25.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:25 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:55:25.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:25 smithi183 ceph-mon[35535]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:25.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:25 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:25.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:25 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:25.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:25 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:25.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:25 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:25.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:25 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:25.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:25 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:25.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:25 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:25.989 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:25.988+0000 7fd409b54640 1 -- 172.21.15.177:0/875637554 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd404062630 con 0x7fd3ec0a2b70 2024-09-12T02:55:25.990 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:25 smithi177 ceph-mon[35305]: from='osd.0 [v2:172.21.15.116:6802/862366853,v1:172.21.15.116:6803/862366853]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T02:55:25.990 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:25 smithi177 ceph-mon[35305]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T02:55:25.990 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:25 smithi177 ceph-mon[35305]: from='osd.0 [v2:172.21.15.116:6802/862366853,v1:172.21.15.116:6803/862366853]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-09-12T02:55:25.990 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:25 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:55:25.990 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:25 smithi177 ceph-mon[35305]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:25.990 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:25 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:25.990 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:25 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:25.990 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:25 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:25.990 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:25 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:25.990 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:25 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:25.990 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:25 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:25.990 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:25 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:26.302 INFO:journalctl@ceph.osd.0.smithi116.stdout:Sep 12 02:55:26 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-0[45396]: 2024-09-12T02:55:26.011+0000 7f02b24d6640 -1 osd.0 0 waiting for initial osdmap 2024-09-12T02:55:26.303 INFO:journalctl@ceph.osd.0.smithi116.stdout:Sep 12 02:55:26 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-0[45396]: 2024-09-12T02:55:26.014+0000 7f02aa2f6640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:55:26.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:26 smithi183 ceph-mon[35535]: Detected new or changed devices on smithi116 2024-09-12T02:55:26.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:26 smithi183 ceph-mon[35535]: Adjusting osd_memory_target on smithi116 to 17088M 2024-09-12T02:55:26.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:26 smithi183 ceph-mon[35535]: from='osd.0 [v2:172.21.15.116:6802/862366853,v1:172.21.15.116:6803/862366853]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-09-12T02:55:26.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:26 smithi183 ceph-mon[35535]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T02:55:26.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:26 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:55:26.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:26 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:55:26.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:26 smithi183 ceph-mon[35535]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:55:26.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:26 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:55:26.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:26 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:55:26.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:26 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:26.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:26 smithi183 ceph-mon[35535]: from='osd.0 [v2:172.21.15.116:6802/862366853,v1:172.21.15.116:6803/862366853]' entity='osd.0' 2024-09-12T02:55:26.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:26 smithi177 ceph-mon[35305]: Detected new or changed devices on smithi116 2024-09-12T02:55:26.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:26 smithi177 ceph-mon[35305]: Adjusting osd_memory_target on smithi116 to 17088M 2024-09-12T02:55:26.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:26 smithi177 ceph-mon[35305]: from='osd.0 [v2:172.21.15.116:6802/862366853,v1:172.21.15.116:6803/862366853]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-09-12T02:55:26.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:26 smithi177 ceph-mon[35305]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T02:55:26.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:26 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:55:26.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:26 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:55:26.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:26 smithi177 ceph-mon[35305]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:55:26.996 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:26 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:55:26.996 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:26 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:55:26.996 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:26 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:26.996 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:26 smithi177 ceph-mon[35305]: from='osd.0 [v2:172.21.15.116:6802/862366853,v1:172.21.15.116:6803/862366853]' entity='osd.0' 2024-09-12T02:55:27.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:26 smithi116 ceph-mon[31410]: Detected new or changed devices on smithi116 2024-09-12T02:55:27.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:26 smithi116 ceph-mon[31410]: Adjusting osd_memory_target on smithi116 to 17088M 2024-09-12T02:55:27.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:26 smithi116 ceph-mon[31410]: from='osd.0 [v2:172.21.15.116:6802/862366853,v1:172.21.15.116:6803/862366853]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-09-12T02:55:27.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:26 smithi116 ceph-mon[31410]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T02:55:27.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:26 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:55:27.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:26 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:55:27.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:26 smithi116 ceph-mon[31410]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:55:27.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:26 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:55:27.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:26 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:55:27.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:26 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:27.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:26 smithi116 ceph-mon[31410]: from='osd.0 [v2:172.21.15.116:6802/862366853,v1:172.21.15.116:6803/862366853]' entity='osd.0' 2024-09-12T02:55:27.743 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:27 smithi177 ceph-mon[35305]: purged_snaps scrub starts 2024-09-12T02:55:27.743 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:27 smithi177 ceph-mon[35305]: purged_snaps scrub ok 2024-09-12T02:55:27.743 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:27 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:55:27.743 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:27 smithi177 ceph-mon[35305]: osd.0 [v2:172.21.15.116:6802/862366853,v1:172.21.15.116:6803/862366853] boot 2024-09-12T02:55:27.743 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:27 smithi177 ceph-mon[35305]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T02:55:27.743 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:27 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:55:27.743 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:27 smithi177 ceph-mon[35305]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:27.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:27 smithi183 ceph-mon[35535]: purged_snaps scrub starts 2024-09-12T02:55:27.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:27 smithi183 ceph-mon[35535]: purged_snaps scrub ok 2024-09-12T02:55:27.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:27 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:55:27.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:27 smithi183 ceph-mon[35535]: osd.0 [v2:172.21.15.116:6802/862366853,v1:172.21.15.116:6803/862366853] boot 2024-09-12T02:55:27.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:27 smithi183 ceph-mon[35535]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T02:55:27.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:27 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:55:27.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:27 smithi183 ceph-mon[35535]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:28.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:27 smithi116 ceph-mon[31410]: purged_snaps scrub starts 2024-09-12T02:55:28.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:27 smithi116 ceph-mon[31410]: purged_snaps scrub ok 2024-09-12T02:55:28.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:27 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:55:28.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:27 smithi116 ceph-mon[31410]: osd.0 [v2:172.21.15.116:6802/862366853,v1:172.21.15.116:6803/862366853] boot 2024-09-12T02:55:28.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:27 smithi116 ceph-mon[31410]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T02:55:28.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:27 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:55:28.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:27 smithi116 ceph-mon[31410]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:29.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:29 smithi177 ceph-mon[35305]: from='client.? 172.21.15.177:0/3944884174' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0affd3cb-9a51-42b0-a2cc-f30784a75cf1"}]: dispatch 2024-09-12T02:55:29.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:29 smithi177 ceph-mon[35305]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0affd3cb-9a51-42b0-a2cc-f30784a75cf1"}]: dispatch 2024-09-12T02:55:29.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:29 smithi177 ceph-mon[35305]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0affd3cb-9a51-42b0-a2cc-f30784a75cf1"}]': finished 2024-09-12T02:55:29.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:29 smithi177 ceph-mon[35305]: osdmap e9: 2 total, 1 up, 2 in 2024-09-12T02:55:29.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:29 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:55:29.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:29 smithi177 ceph-mon[35305]: from='client.? 172.21.15.177:0/2359015203' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:55:29.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:29 smithi116 ceph-mon[31410]: from='client.? 172.21.15.177:0/3944884174' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0affd3cb-9a51-42b0-a2cc-f30784a75cf1"}]: dispatch 2024-09-12T02:55:29.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:29 smithi116 ceph-mon[31410]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0affd3cb-9a51-42b0-a2cc-f30784a75cf1"}]: dispatch 2024-09-12T02:55:29.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:29 smithi116 ceph-mon[31410]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0affd3cb-9a51-42b0-a2cc-f30784a75cf1"}]': finished 2024-09-12T02:55:29.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:29 smithi116 ceph-mon[31410]: osdmap e9: 2 total, 1 up, 2 in 2024-09-12T02:55:29.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:29 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:55:29.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:29 smithi116 ceph-mon[31410]: from='client.? 172.21.15.177:0/2359015203' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:55:29.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:29 smithi183 ceph-mon[35535]: from='client.? 172.21.15.177:0/3944884174' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0affd3cb-9a51-42b0-a2cc-f30784a75cf1"}]: dispatch 2024-09-12T02:55:29.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:29 smithi183 ceph-mon[35535]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0affd3cb-9a51-42b0-a2cc-f30784a75cf1"}]: dispatch 2024-09-12T02:55:29.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:29 smithi183 ceph-mon[35535]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0affd3cb-9a51-42b0-a2cc-f30784a75cf1"}]': finished 2024-09-12T02:55:29.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:29 smithi183 ceph-mon[35535]: osdmap e9: 2 total, 1 up, 2 in 2024-09-12T02:55:29.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:29 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:55:29.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:29 smithi183 ceph-mon[35535]: from='client.? 172.21.15.177:0/2359015203' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:55:30.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:30 smithi116 ceph-mon[31410]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:30.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:30 smithi183 ceph-mon[35535]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:30.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:30 smithi177 ceph-mon[35305]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:32.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:32 smithi183 ceph-mon[35535]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:32.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:32 smithi177 ceph-mon[35305]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:32.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:32 smithi116 ceph-mon[31410]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:34.172 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:34 smithi177 ceph-mon[35305]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:34.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:34 smithi183 ceph-mon[35535]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:34.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:34 smithi116 ceph-mon[31410]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:36.409 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:36 smithi177 ceph-mon[35305]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:36.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:36 smithi183 ceph-mon[35535]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:36.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:36 smithi116 ceph-mon[31410]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:37.389 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:37 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T02:55:37.389 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:37 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:37.394 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:37 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T02:55:37.394 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:37 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:37.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:37 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T02:55:37.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:37 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:38.416 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:38 smithi177 ceph-mon[35305]: Deploying daemon osd.1 on smithi177 2024-09-12T02:55:38.417 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:38 smithi177 ceph-mon[35305]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:38.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:38 smithi183 ceph-mon[35535]: Deploying daemon osd.1 on smithi177 2024-09-12T02:55:38.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:38 smithi183 ceph-mon[35535]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:38.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:38 smithi116 ceph-mon[31410]: Deploying daemon osd.1 on smithi177 2024-09-12T02:55:38.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:38 smithi116 ceph-mon[31410]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:40.366 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:40 smithi177 ceph-mon[35305]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:40.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:40 smithi183 ceph-mon[35535]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:40.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:40 smithi116 ceph-mon[31410]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:42.319 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:42 smithi177 ceph-mon[35305]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:42.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:42 smithi183 ceph-mon[35535]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:42.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:42 smithi116 ceph-mon[31410]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:44.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:44 smithi177 ceph-mon[35305]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:44.246 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:44 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:44.246 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:44 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:44.246 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:44 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:44.466 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:44 smithi183 ceph-mon[35535]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:44.466 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:44 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:44.466 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:44 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:44.466 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:44 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:44.557 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:44 smithi116 ceph-mon[31410]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:44.557 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:44 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:44.558 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:44 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:44.558 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:44 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:46.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:46 smithi183 ceph-mon[35535]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:46.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:46 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:46.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:46 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:46.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:46 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:46.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:46 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:46.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:46 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:46.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:46 smithi177 ceph-mon[35305]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:46.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:46 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:46.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:46 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:46.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:46 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:46.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:46 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:46.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:46 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:46.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:46 smithi116 ceph-mon[31410]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:46.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:46 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:46.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:46 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:46.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:46 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:46.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:46 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:46.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:46 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:47.285 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:47.285+0000 7fd400ff9640 1 -- 172.21.15.177:0/875637554 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd404062630 con 0x7fd3ec0a2b70 2024-09-12T02:55:47.286 INFO:teuthology.orchestra.run.smithi177.stdout:Created osd(s) 1 on host 'smithi177' 2024-09-12T02:55:47.287 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:47.287+0000 7fd409b54640 1 -- 172.21.15.177:0/875637554 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd3ec0a2b70 msgr2=0x7fd3ec0a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:47.287 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:47.287+0000 7fd409b54640 1 --2- 172.21.15.177:0/875637554 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd3ec0a2b70 0x7fd3ec0a5030 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd3f80314e0 tx=0x7fd3f8000bc0 comp rx=0 tx=0).stop 2024-09-12T02:55:47.287 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:47.287+0000 7fd409b54640 1 -- 172.21.15.177:0/875637554 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd404101c00 msgr2=0x7fd404075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:47.288 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:47.287+0000 7fd409b54640 1 --2- 172.21.15.177:0/875637554 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd404101c00 0x7fd404075f90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd3f400d950 tx=0x7fd3f400de10 comp rx=0 tx=0).stop 2024-09-12T02:55:47.288 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:47.287+0000 7fd409b54640 1 -- 172.21.15.177:0/875637554 shutdown_connections 2024-09-12T02:55:47.288 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:47.287+0000 7fd409b54640 1 --2- 172.21.15.177:0/875637554 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd3ec0a2b70 0x7fd3ec0a5030 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:47.288 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:47.287+0000 7fd409b54640 1 --2- 172.21.15.177:0/875637554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4041037c0 0x7fd404074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:47.288 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:47.287+0000 7fd409b54640 1 --2- 172.21.15.177:0/875637554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd404102e00 0x7fd4040745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:47.288 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:47.287+0000 7fd409b54640 1 --2- 172.21.15.177:0/875637554 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd404101c00 0x7fd404075f90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:47.288 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:47.287+0000 7fd409b54640 1 -- 172.21.15.177:0/875637554 >> 172.21.15.177:0/875637554 conn(0x7fd4040fd390 msgr2=0x7fd4041062c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:47.288 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:47.287+0000 7fd409b54640 1 -- 172.21.15.177:0/875637554 shutdown_connections 2024-09-12T02:55:47.288 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:55:47.287+0000 7fd409b54640 1 -- 172.21.15.177:0/875637554 wait complete. 2024-09-12T02:55:47.936 DEBUG:teuthology.orchestra.run.smithi177:osd.1> sudo journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.1.service 2024-09-12T02:55:47.938 INFO:tasks.cephadm:Deploying osd.2 on smithi183 with /dev/vg_nvme/lv_4... 2024-09-12T02:55:47.939 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T02:55:48.185 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:48 smithi177 ceph-mon[35305]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:48.186 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:48 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:48.186 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:48 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:48.186 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:48 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:48.199 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:55:48.319 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:48 smithi183 ceph-mon[35535]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:48.319 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:48 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:48.319 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:48 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:48.319 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:48 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:48.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:48 smithi116 ceph-mon[31410]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:48.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:48 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:48.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:48 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:48.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:48 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:49.281 INFO:journalctl@ceph.osd.1.smithi177.stdout:Sep 12 02:55:49 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-1[39769]: 2024-09-12T02:55:49.262+0000 7ff5c13dc640 -1 osd.1 0 log_to_monitors true 2024-09-12T02:55:50.220 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:50 smithi177 ceph-mon[35305]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:50.220 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:50 smithi177 ceph-mon[35305]: from='osd.1 [v2:172.21.15.177:6800/2245800547,v1:172.21.15.177:6801/2245800547]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T02:55:50.220 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:50 smithi177 ceph-mon[35305]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T02:55:50.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:50 smithi183 ceph-mon[35535]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:50.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:50 smithi183 ceph-mon[35535]: from='osd.1 [v2:172.21.15.177:6800/2245800547,v1:172.21.15.177:6801/2245800547]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T02:55:50.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:50 smithi183 ceph-mon[35535]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T02:55:50.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:50 smithi116 ceph-mon[31410]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:50.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:50 smithi116 ceph-mon[31410]: from='osd.1 [v2:172.21.15.177:6800/2245800547,v1:172.21.15.177:6801/2245800547]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T02:55:50.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:50 smithi116 ceph-mon[31410]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T02:55:50.824 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T02:55:50.849 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_4 2024-09-12T02:55:51.089 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:55:51.408 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:51 smithi183 ceph-mon[35535]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T02:55:51.408 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:51 smithi183 ceph-mon[35535]: from='osd.1 [v2:172.21.15.177:6800/2245800547,v1:172.21.15.177:6801/2245800547]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-09-12T02:55:51.409 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:51 smithi183 ceph-mon[35535]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T02:55:51.409 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:51 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:55:51.409 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:51 smithi183 ceph-mon[35535]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-09-12T02:55:51.409 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:51 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:51.409 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:51 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:51.409 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:51 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:51.409 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:51 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:51.409 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:51 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:51.409 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:51 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:51.409 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:51 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:51.409 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:51 smithi183 ceph-mon[35535]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-09-12T02:55:51.492 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.491+0000 7f254239a640 1 -- 172.21.15.183:0/4020789266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f253c100ad0 msgr2=0x7f253c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:51.492 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.491+0000 7f254239a640 1 --2- 172.21.15.183:0/4020789266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f253c100ad0 0x7f253c100ed0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f252c009900 tx=0x7f252c02a410 comp rx=0 tx=0).stop 2024-09-12T02:55:51.492 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.491+0000 7f254239a640 1 -- 172.21.15.183:0/4020789266 shutdown_connections 2024-09-12T02:55:51.492 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.491+0000 7f254239a640 1 --2- 172.21.15.183:0/4020789266 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f253c102690 0x7f253c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:51.493 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.491+0000 7f254239a640 1 --2- 172.21.15.183:0/4020789266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f253c101cd0 0x7f253c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:51.493 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.491+0000 7f254239a640 1 --2- 172.21.15.183:0/4020789266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f253c100ad0 0x7f253c100ed0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:51.493 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.491+0000 7f254239a640 1 -- 172.21.15.183:0/4020789266 >> 172.21.15.183:0/4020789266 conn(0x7f253c0fc280 msgr2=0x7f253c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:51.493 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.491+0000 7f254239a640 1 -- 172.21.15.183:0/4020789266 shutdown_connections 2024-09-12T02:55:51.493 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.491+0000 7f254239a640 1 -- 172.21.15.183:0/4020789266 wait complete. 2024-09-12T02:55:51.493 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.492+0000 7f254239a640 1 Processor -- start 2024-09-12T02:55:51.493 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.492+0000 7f254239a640 1 -- start start 2024-09-12T02:55:51.495 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.492+0000 7f254239a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f253c100ad0 0x7f253c19a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:51.495 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.492+0000 7f254239a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f253c101cd0 0x7f253c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:51.495 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.492+0000 7f254239a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f253c102690 0x7f253c1a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:51.495 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.492+0000 7f254239a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f253c10c7a0 con 0x7f253c100ad0 2024-09-12T02:55:51.495 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.492+0000 7f254239a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f253c10c620 con 0x7f253c102690 2024-09-12T02:55:51.495 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.492+0000 7f254239a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f253c10c920 con 0x7f253c101cd0 2024-09-12T02:55:51.495 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.492+0000 7f253b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f253c101cd0 0x7f253c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:51.495 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.492+0000 7f253b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f253c101cd0 0x7f253c19a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:46146/0 (socket says 172.21.15.183:46146) 2024-09-12T02:55:51.495 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.492+0000 7f253b7fe640 1 -- 172.21.15.183:0/3274963900 learned_addr learned my addr 172.21.15.183:0/3274963900 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-12T02:55:51.495 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.492+0000 7f2540937640 1 --2- 172.21.15.183:0/3274963900 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f253c102690 0x7f253c1a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:51.496 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.492+0000 7f253bfff640 1 --2- 172.21.15.183:0/3274963900 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f253c100ad0 0x7f253c19a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:51.496 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.492+0000 7f2540937640 1 -- 172.21.15.183:0/3274963900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f253c101cd0 msgr2=0x7f253c19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:51.496 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.492+0000 7f2540937640 1 --2- 172.21.15.183:0/3274963900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f253c101cd0 0x7f253c19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:51.496 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.492+0000 7f2540937640 1 -- 172.21.15.183:0/3274963900 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f253c100ad0 msgr2=0x7f253c19a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:51.496 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.492+0000 7f2540937640 1 --2- 172.21.15.183:0/3274963900 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f253c100ad0 0x7f253c19a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:51.496 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.492+0000 7f2540937640 1 -- 172.21.15.183:0/3274963900 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f253c1a1ce0 con 0x7f253c102690 2024-09-12T02:55:51.496 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.492+0000 7f253b7fe640 1 --2- 172.21.15.183:0/3274963900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f253c101cd0 0x7f253c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:55:51.496 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.493+0000 7f2540937640 1 --2- 172.21.15.183:0/3274963900 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f253c102690 0x7f253c1a1670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f253400cab0 tx=0x7f253400cf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:51.496 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.493+0000 7f25397fa640 1 -- 172.21.15.183:0/3274963900 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253400ae20 con 0x7f253c102690 2024-09-12T02:55:51.496 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.493+0000 7f254239a640 1 -- 172.21.15.183:0/3274963900 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f253c1a1fd0 con 0x7f253c102690 2024-09-12T02:55:51.496 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.493+0000 7f25397fa640 1 -- 172.21.15.183:0/3274963900 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f253400f930 con 0x7f253c102690 2024-09-12T02:55:51.496 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.493+0000 7f25397fa640 1 -- 172.21.15.183:0/3274963900 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2534018b30 con 0x7f253c102690 2024-09-12T02:55:51.496 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.493+0000 7f254239a640 1 -- 172.21.15.183:0/3274963900 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f253c1a2430 con 0x7f253c102690 2024-09-12T02:55:51.497 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.494+0000 7f254239a640 1 -- 172.21.15.183:0/3274963900 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f253c1a26f0 con 0x7f253c102690 2024-09-12T02:55:51.497 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:51 smithi177 ceph-mon[35305]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T02:55:51.497 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:51 smithi177 ceph-mon[35305]: from='osd.1 [v2:172.21.15.177:6800/2245800547,v1:172.21.15.177:6801/2245800547]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-09-12T02:55:51.497 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:51 smithi177 ceph-mon[35305]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T02:55:51.497 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:51 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:55:51.498 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:51 smithi177 ceph-mon[35305]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-09-12T02:55:51.498 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:51 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:51.498 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:51 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:51.498 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:51 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:51.498 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:51 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:51.498 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:51 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:51.498 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:51 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:51.498 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:51 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:51.498 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:51 smithi177 ceph-mon[35305]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-09-12T02:55:51.499 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.495+0000 7f25397fa640 1 -- 172.21.15.183:0/3274963900 <== mon.2 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f253400fae0 con 0x7f253c102690 2024-09-12T02:55:51.499 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.496+0000 7f25397fa640 1 --2- 172.21.15.183:0/3274963900 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f25280a2b20 0x7f25280a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:51.499 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.496+0000 7f25397fa640 1 -- 172.21.15.183:0/3274963900 <== mon.2 v2:172.21.15.177:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 2029+0+0 (secure 0 0 0) 0x7f25340ce0a0 con 0x7f253c102690 2024-09-12T02:55:51.499 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.496+0000 7f253bfff640 1 --2- 172.21.15.183:0/3274963900 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f25280a2b20 0x7f25280a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:51.499 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.497+0000 7f253bfff640 1 --2- 172.21.15.183:0/3274963900 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f25280a2b20 0x7f25280a4fe0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f252c02aa00 tx=0x7f252c0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:51.504 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.503+0000 7f25397fa640 1 -- 172.21.15.183:0/3274963900 <== mon.2 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2534014030 con 0x7f253c102690 2024-09-12T02:55:51.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:51 smithi116 ceph-mon[31410]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T02:55:51.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:51 smithi116 ceph-mon[31410]: from='osd.1 [v2:172.21.15.177:6800/2245800547,v1:172.21.15.177:6801/2245800547]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-09-12T02:55:51.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:51 smithi116 ceph-mon[31410]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T02:55:51.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:51 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:55:51.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:51 smithi116 ceph-mon[31410]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-09-12T02:55:51.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:51 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:51.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:51 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:51.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:51 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:51.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:51 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:51.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:51 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:51.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:51 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:51.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:51 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:55:51.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:51 smithi116 ceph-mon[31410]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-09-12T02:55:51.681 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:55:51.680+0000 7f254239a640 1 -- 172.21.15.183:0/3274963900 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f253c1a28c0 con 0x7f25280a2b20 2024-09-12T02:55:51.995 INFO:journalctl@ceph.osd.1.smithi177.stdout:Sep 12 02:55:51 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-1[39769]: 2024-09-12T02:55:51.528+0000 7ff5bd35f640 -1 osd.1 0 waiting for initial osdmap 2024-09-12T02:55:51.995 INFO:journalctl@ceph.osd.1.smithi177.stdout:Sep 12 02:55:51 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-1[39769]: 2024-09-12T02:55:51.531+0000 7ff5b517f640 -1 osd.1 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:55:52.344 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:52 smithi183 ceph-mon[35535]: Detected new or changed devices on smithi177 2024-09-12T02:55:52.344 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:52 smithi183 ceph-mon[35535]: Adjusting osd_memory_target on smithi177 to 17087M 2024-09-12T02:55:52.345 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:52 smithi183 ceph-mon[35535]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:52.345 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:52 smithi183 ceph-mon[35535]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T02:55:52.345 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:52 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:55:52.345 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:52 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:55:52.345 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:52 smithi183 ceph-mon[35535]: from='osd.1 ' entity='osd.1' 2024-09-12T02:55:52.345 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:52 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:55:52.345 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:52 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:55:52.345 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:52 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:52.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:52 smithi177 ceph-mon[35305]: Detected new or changed devices on smithi177 2024-09-12T02:55:52.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:52 smithi177 ceph-mon[35305]: Adjusting osd_memory_target on smithi177 to 17087M 2024-09-12T02:55:52.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:52 smithi177 ceph-mon[35305]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:52.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:52 smithi177 ceph-mon[35305]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T02:55:52.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:55:52.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:55:52.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:52 smithi177 ceph-mon[35305]: from='osd.1 ' entity='osd.1' 2024-09-12T02:55:52.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:55:52.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:55:52.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:52.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:52 smithi116 ceph-mon[31410]: Detected new or changed devices on smithi177 2024-09-12T02:55:52.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:52 smithi116 ceph-mon[31410]: Adjusting osd_memory_target on smithi177 to 17087M 2024-09-12T02:55:52.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:52 smithi116 ceph-mon[31410]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:52.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:52 smithi116 ceph-mon[31410]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T02:55:52.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:52 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:55:52.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:52 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:55:52.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:52 smithi116 ceph-mon[31410]: from='osd.1 ' entity='osd.1' 2024-09-12T02:55:52.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:52 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:55:52.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:52 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:55:52.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:52 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:53.350 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:53 smithi183 ceph-mon[35535]: purged_snaps scrub starts 2024-09-12T02:55:53.350 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:53 smithi183 ceph-mon[35535]: purged_snaps scrub ok 2024-09-12T02:55:53.351 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:53 smithi183 ceph-mon[35535]: from='client.24167 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:55:53.351 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:53 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:55:53.351 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:53 smithi183 ceph-mon[35535]: osd.1 [v2:172.21.15.177:6800/2245800547,v1:172.21.15.177:6801/2245800547] boot 2024-09-12T02:55:53.351 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:53 smithi183 ceph-mon[35535]: osdmap e12: 2 total, 2 up, 2 in 2024-09-12T02:55:53.351 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:53 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:55:53.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:53 smithi177 ceph-mon[35305]: purged_snaps scrub starts 2024-09-12T02:55:53.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:53 smithi177 ceph-mon[35305]: purged_snaps scrub ok 2024-09-12T02:55:53.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:53 smithi177 ceph-mon[35305]: from='client.24167 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:55:53.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:53 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:55:53.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:53 smithi177 ceph-mon[35305]: osd.1 [v2:172.21.15.177:6800/2245800547,v1:172.21.15.177:6801/2245800547] boot 2024-09-12T02:55:53.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:53 smithi177 ceph-mon[35305]: osdmap e12: 2 total, 2 up, 2 in 2024-09-12T02:55:53.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:53 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:55:53.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:53 smithi116 ceph-mon[31410]: purged_snaps scrub starts 2024-09-12T02:55:53.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:53 smithi116 ceph-mon[31410]: purged_snaps scrub ok 2024-09-12T02:55:53.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:53 smithi116 ceph-mon[31410]: from='client.24167 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:55:53.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:53 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:55:53.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:53 smithi116 ceph-mon[31410]: osd.1 [v2:172.21.15.177:6800/2245800547,v1:172.21.15.177:6801/2245800547] boot 2024-09-12T02:55:53.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:53 smithi116 ceph-mon[31410]: osdmap e12: 2 total, 2 up, 2 in 2024-09-12T02:55:53.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:53 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:55:54.369 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:54 smithi183 ceph-mon[35535]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:54.369 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:54 smithi183 ceph-mon[35535]: from='client.? 172.21.15.183:0/1744926187' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ce45ea5-84df-4a35-924f-78f92bd4f2cd"}]: dispatch 2024-09-12T02:55:54.369 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:54 smithi183 ceph-mon[35535]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ce45ea5-84df-4a35-924f-78f92bd4f2cd"}]: dispatch 2024-09-12T02:55:54.369 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:54 smithi183 ceph-mon[35535]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ce45ea5-84df-4a35-924f-78f92bd4f2cd"}]': finished 2024-09-12T02:55:54.369 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:54 smithi183 ceph-mon[35535]: osdmap e13: 3 total, 2 up, 3 in 2024-09-12T02:55:54.370 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:54 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:55:54.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:54 smithi177 ceph-mon[35305]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:54.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:54 smithi177 ceph-mon[35305]: from='client.? 172.21.15.183:0/1744926187' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ce45ea5-84df-4a35-924f-78f92bd4f2cd"}]: dispatch 2024-09-12T02:55:54.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:54 smithi177 ceph-mon[35305]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ce45ea5-84df-4a35-924f-78f92bd4f2cd"}]: dispatch 2024-09-12T02:55:54.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:54 smithi177 ceph-mon[35305]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ce45ea5-84df-4a35-924f-78f92bd4f2cd"}]': finished 2024-09-12T02:55:54.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:54 smithi177 ceph-mon[35305]: osdmap e13: 3 total, 2 up, 3 in 2024-09-12T02:55:54.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:54 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:55:54.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:54 smithi116 ceph-mon[31410]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:55:54.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:54 smithi116 ceph-mon[31410]: from='client.? 172.21.15.183:0/1744926187' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ce45ea5-84df-4a35-924f-78f92bd4f2cd"}]: dispatch 2024-09-12T02:55:54.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:54 smithi116 ceph-mon[31410]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ce45ea5-84df-4a35-924f-78f92bd4f2cd"}]: dispatch 2024-09-12T02:55:54.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:54 smithi116 ceph-mon[31410]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ce45ea5-84df-4a35-924f-78f92bd4f2cd"}]': finished 2024-09-12T02:55:54.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:54 smithi116 ceph-mon[31410]: osdmap e13: 3 total, 2 up, 3 in 2024-09-12T02:55:54.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:54 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:55:55.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:55 smithi183 ceph-mon[35535]: from='client.? 172.21.15.183:0/2411788808' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:55:55.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:55 smithi177 ceph-mon[35305]: from='client.? 172.21.15.183:0/2411788808' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:55:55.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:55 smithi116 ceph-mon[31410]: from='client.? 172.21.15.183:0/2411788808' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:55:56.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:56 smithi183 ceph-mon[35535]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:55:56.453 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:56 smithi116 ceph-mon[31410]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:55:56.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:56 smithi177 ceph-mon[35305]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:55:58.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:55:58 smithi183 ceph-mon[35535]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:55:58.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:55:58 smithi177 ceph-mon[35305]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:55:58.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:55:58 smithi116 ceph-mon[31410]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:00.249 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:00 smithi183 ceph-mon[35535]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:00.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:00 smithi177 ceph-mon[35305]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:00.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:00 smithi116 ceph-mon[31410]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:02.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:02 smithi183 ceph-mon[35535]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:02.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:02 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T02:56:02.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:02 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:02.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:02 smithi177 ceph-mon[35305]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:02.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:02 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T02:56:02.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:02 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:02.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:02 smithi116 ceph-mon[31410]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:02.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:02 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T02:56:02.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:02 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:03.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:03 smithi183 ceph-mon[35535]: Deploying daemon osd.2 on smithi183 2024-09-12T02:56:03.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:03 smithi177 ceph-mon[35305]: Deploying daemon osd.2 on smithi183 2024-09-12T02:56:03.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:03 smithi116 ceph-mon[31410]: Deploying daemon osd.2 on smithi183 2024-09-12T02:56:04.160 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:04 smithi183 ceph-mon[35535]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:04.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:04 smithi177 ceph-mon[35305]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:04.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:04 smithi116 ceph-mon[31410]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:06.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:06 smithi183 ceph-mon[35535]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:06.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:06 smithi177 ceph-mon[35305]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:06.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:06 smithi116 ceph-mon[31410]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:08.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:08 smithi183 ceph-mon[35535]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:08.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:08 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:08.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:08 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:08.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:08 smithi177 ceph-mon[35305]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:08.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:08 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:08.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:08 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:08.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:08 smithi116 ceph-mon[31410]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:08.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:08 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:08.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:08 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:09.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:09 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:09.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:09 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:09.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:09 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:10.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:10 smithi183 ceph-mon[35535]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:10.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:10 smithi177 ceph-mon[35305]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:10.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:10 smithi116 ceph-mon[31410]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:11.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:11 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:11.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:11 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:11.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:11 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:11.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:11 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:11.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:11 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:11.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:11 smithi183 ceph-mon[35535]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:11.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:11 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:11.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:11 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:11.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:11 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:11.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:11 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:11.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:11 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:11.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:11 smithi177 ceph-mon[35305]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:12.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:11 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:12.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:11 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:12.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:11 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:12.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:11 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:12.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:11 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:12.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:11 smithi116 ceph-mon[31410]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:12.412 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:56:12.411+0000 7f25397fa640 1 -- 172.21.15.183:0/3274963900 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f253c1a28c0 con 0x7f25280a2b20 2024-09-12T02:56:12.412 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 2 on host 'smithi183' 2024-09-12T02:56:12.414 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:56:12.413+0000 7f254239a640 1 -- 172.21.15.183:0/3274963900 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f25280a2b20 msgr2=0x7f25280a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:12.414 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:56:12.413+0000 7f254239a640 1 --2- 172.21.15.183:0/3274963900 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f25280a2b20 0x7f25280a4fe0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f252c02aa00 tx=0x7f252c0315b0 comp rx=0 tx=0).stop 2024-09-12T02:56:12.414 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:56:12.413+0000 7f254239a640 1 -- 172.21.15.183:0/3274963900 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f253c102690 msgr2=0x7f253c1a1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:12.415 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:56:12.413+0000 7f254239a640 1 --2- 172.21.15.183:0/3274963900 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f253c102690 0x7f253c1a1670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f253400cab0 tx=0x7f253400cf70 comp rx=0 tx=0).stop 2024-09-12T02:56:12.415 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:56:12.414+0000 7f254239a640 1 -- 172.21.15.183:0/3274963900 shutdown_connections 2024-09-12T02:56:12.415 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:56:12.414+0000 7f254239a640 1 --2- 172.21.15.183:0/3274963900 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f25280a2b20 0x7f25280a4fe0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:12.415 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:56:12.414+0000 7f254239a640 1 --2- 172.21.15.183:0/3274963900 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f253c102690 0x7f253c1a1670 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:12.415 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:56:12.414+0000 7f254239a640 1 --2- 172.21.15.183:0/3274963900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f253c101cd0 0x7f253c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:12.415 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:56:12.414+0000 7f254239a640 1 --2- 172.21.15.183:0/3274963900 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f253c100ad0 0x7f253c19a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:12.415 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:56:12.414+0000 7f254239a640 1 -- 172.21.15.183:0/3274963900 >> 172.21.15.183:0/3274963900 conn(0x7f253c0fc280 msgr2=0x7f253c0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:12.415 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:56:12.414+0000 7f254239a640 1 -- 172.21.15.183:0/3274963900 shutdown_connections 2024-09-12T02:56:12.415 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:56:12.414+0000 7f254239a640 1 -- 172.21.15.183:0/3274963900 wait complete. 2024-09-12T02:56:12.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:12 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:12.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:12 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:12.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:12 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:12.896 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:12 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:12.897 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:12 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:12.897 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:12 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:13.012 DEBUG:teuthology.orchestra.run.smithi183:osd.2> sudo journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.2.service 2024-09-12T02:56:13.014 INFO:tasks.cephadm:Deploying osd.3 on smithi116 with /dev/vg_nvme/lv_3... 2024-09-12T02:56:13.015 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T02:56:13.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:12 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:13.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:12 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:13.043 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:12 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:13.276 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:13.695 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:13 smithi116 ceph-mon[31410]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:13.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:13 smithi183 ceph-mon[35535]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:13.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:13 smithi177 ceph-mon[35305]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:14.255 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 02:56:14 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-2[40517]: 2024-09-12T02:56:14.001+0000 7f566c884640 -1 osd.2 0 log_to_monitors true 2024-09-12T02:56:14.760 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:14 smithi116 ceph-mon[31410]: from='osd.2 [v2:172.21.15.183:6800/2645677759,v1:172.21.15.183:6801/2645677759]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T02:56:14.760 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:14 smithi116 ceph-mon[31410]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T02:56:14.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:14 smithi183 ceph-mon[35535]: from='osd.2 [v2:172.21.15.183:6800/2645677759,v1:172.21.15.183:6801/2645677759]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T02:56:14.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:14 smithi183 ceph-mon[35535]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T02:56:14.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:14 smithi177 ceph-mon[35305]: from='osd.2 [v2:172.21.15.183:6800/2645677759,v1:172.21.15.183:6801/2645677759]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T02:56:14.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:14 smithi177 ceph-mon[35305]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T02:56:15.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:15 smithi116 ceph-mon[31410]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T02:56:15.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:15 smithi116 ceph-mon[31410]: osdmap e14: 3 total, 2 up, 3 in 2024-09-12T02:56:15.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:56:15.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:15 smithi116 ceph-mon[31410]: from='osd.2 [v2:172.21.15.183:6800/2645677759,v1:172.21.15.183:6801/2645677759]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T02:56:15.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:15 smithi116 ceph-mon[31410]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T02:56:15.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:15 smithi116 ceph-mon[31410]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:15.835 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:56:15.840 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:15 smithi183 ceph-mon[35535]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T02:56:15.840 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:15 smithi183 ceph-mon[35535]: osdmap e14: 3 total, 2 up, 3 in 2024-09-12T02:56:15.840 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:15 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:56:15.840 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:15 smithi183 ceph-mon[35535]: from='osd.2 [v2:172.21.15.183:6800/2645677759,v1:172.21.15.183:6801/2645677759]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T02:56:15.840 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:15 smithi183 ceph-mon[35535]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T02:56:15.840 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:15 smithi183 ceph-mon[35535]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:15.861 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi116:vg_nvme/lv_3 2024-09-12T02:56:15.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:15 smithi177 ceph-mon[35305]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T02:56:15.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:15 smithi177 ceph-mon[35305]: osdmap e14: 3 total, 2 up, 3 in 2024-09-12T02:56:15.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:15 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:56:15.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:15 smithi177 ceph-mon[35305]: from='osd.2 [v2:172.21.15.183:6800/2645677759,v1:172.21.15.183:6801/2645677759]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T02:56:15.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:15 smithi177 ceph-mon[35305]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T02:56:15.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:15 smithi177 ceph-mon[35305]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:16.152 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:16.205 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 02:56:16 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-2[40517]: 2024-09-12T02:56:16.029+0000 7f5668807640 -1 osd.2 0 waiting for initial osdmap 2024-09-12T02:56:16.205 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 02:56:16 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-2[40517]: 2024-09-12T02:56:16.032+0000 7f5660627640 -1 osd.2 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:56:16.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.669+0000 7f82b1574640 1 -- 172.21.15.116:0/1774293940 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82ac104f90 msgr2=0x7f82ac107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:16.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.669+0000 7f82b1574640 1 --2- 172.21.15.116:0/1774293940 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82ac104f90 0x7f82ac107380 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f82a0009900 tx=0x7f82a002a410 comp rx=0 tx=0).stop 2024-09-12T02:56:16.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.670+0000 7f82b1574640 1 -- 172.21.15.116:0/1774293940 shutdown_connections 2024-09-12T02:56:16.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.670+0000 7f82b1574640 1 --2- 172.21.15.116:0/1774293940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82ac1078c0 0x7f82ac109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:16.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.670+0000 7f82b1574640 1 --2- 172.21.15.116:0/1774293940 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82ac104f90 0x7f82ac107380 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:16.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.670+0000 7f82b1574640 1 --2- 172.21.15.116:0/1774293940 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82ac102660 0x7f82ac104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:16.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.670+0000 7f82b1574640 1 -- 172.21.15.116:0/1774293940 >> 172.21.15.116:0/1774293940 conn(0x7f82ac0fc280 msgr2=0x7f82ac0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:16.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.670+0000 7f82b1574640 1 -- 172.21.15.116:0/1774293940 shutdown_connections 2024-09-12T02:56:16.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.670+0000 7f82b1574640 1 -- 172.21.15.116:0/1774293940 wait complete. 2024-09-12T02:56:16.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.670+0000 7f82b1574640 1 Processor -- start 2024-09-12T02:56:16.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.670+0000 7f82b1574640 1 -- start start 2024-09-12T02:56:16.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.670+0000 7f82b1574640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82ac102660 0x7f82ac1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:16.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.670+0000 7f82b1574640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82ac104f90 0x7f82ac198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:16.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.670+0000 7f82b1574640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82ac1078c0 0x7f82ac19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:16.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.670+0000 7f82b1574640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac10d930 con 0x7f82ac1078c0 2024-09-12T02:56:16.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.670+0000 7f82b1574640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac10d7b0 con 0x7f82ac102660 2024-09-12T02:56:16.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.670+0000 7f82b1574640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac10dab0 con 0x7f82ac104f90 2024-09-12T02:56:16.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.671+0000 7f82ab7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82ac1078c0 0x7f82ac19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:16.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.671+0000 7f82ab7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82ac1078c0 0x7f82ac19f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56520/0 (socket says 172.21.15.116:56520) 2024-09-12T02:56:16.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.671+0000 7f82ab7fe640 1 -- 172.21.15.116:0/959473433 learned_addr learned my addr 172.21.15.116:0/959473433 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:56:16.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.671+0000 7f82aa7fc640 1 --2- 172.21.15.116:0/959473433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82ac104f90 0x7f82ac198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:16.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.671+0000 7f82aaffd640 1 --2- 172.21.15.116:0/959473433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82ac102660 0x7f82ac1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:16.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.671+0000 7f82ab7fe640 1 -- 172.21.15.116:0/959473433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82ac104f90 msgr2=0x7f82ac198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:16.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.671+0000 7f82ab7fe640 1 --2- 172.21.15.116:0/959473433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82ac104f90 0x7f82ac198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:16.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.671+0000 7f82ab7fe640 1 -- 172.21.15.116:0/959473433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82ac102660 msgr2=0x7f82ac1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:16.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.671+0000 7f82ab7fe640 1 --2- 172.21.15.116:0/959473433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82ac102660 0x7f82ac1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:16.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.671+0000 7f82ab7fe640 1 -- 172.21.15.116:0/959473433 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ac19fcf0 con 0x7f82ac1078c0 2024-09-12T02:56:16.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.671+0000 7f82ab7fe640 1 --2- 172.21.15.116:0/959473433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82ac1078c0 0x7f82ac19f680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f82a400b970 tx=0x7f82a400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:16.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.671+0000 7f8293fff640 1 -- 172.21.15.116:0/959473433 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a4019070 con 0x7f82ac1078c0 2024-09-12T02:56:16.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.671+0000 7f8293fff640 1 -- 172.21.15.116:0/959473433 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82a400cb40 con 0x7f82ac1078c0 2024-09-12T02:56:16.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.671+0000 7f82b1574640 1 -- 172.21.15.116:0/959473433 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82ac19ffe0 con 0x7f82ac1078c0 2024-09-12T02:56:16.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.671+0000 7f8293fff640 1 -- 172.21.15.116:0/959473433 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a4007650 con 0x7f82ac1078c0 2024-09-12T02:56:16.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.671+0000 7f82b1574640 1 -- 172.21.15.116:0/959473433 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82ac1a04c0 con 0x7f82ac1078c0 2024-09-12T02:56:16.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.672+0000 7f82b1574640 1 -- 172.21.15.116:0/959473433 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82ac0fdd70 con 0x7f82ac1078c0 2024-09-12T02:56:16.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.673+0000 7f8293fff640 1 -- 172.21.15.116:0/959473433 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f82a401c070 con 0x7f82ac1078c0 2024-09-12T02:56:16.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.674+0000 7f8293fff640 1 --2- 172.21.15.116:0/959473433 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f82940a2b20 0x7f82940a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:16.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.674+0000 7f82aaffd640 1 --2- 172.21.15.116:0/959473433 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f82940a2b20 0x7f82940a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:16.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.674+0000 7f8293fff640 1 -- 172.21.15.116:0/959473433 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2571+0+0 (secure 0 0 0) 0x7f82a40ccec0 con 0x7f82ac1078c0 2024-09-12T02:56:16.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.674+0000 7f82aaffd640 1 --2- 172.21.15.116:0/959473433 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f82940a2b20 0x7f82940a4fe0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f829c009890 tx=0x7f829c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:16.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.678+0000 7f8293fff640 1 -- 172.21.15.116:0/959473433 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f82a409b570 con 0x7f82ac1078c0 2024-09-12T02:56:16.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:16 smithi116 ceph-mon[31410]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-12T02:56:16.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:16 smithi116 ceph-mon[31410]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T02:56:16.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:16 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:56:16.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:16 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:56:16.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:16 smithi116 ceph-mon[31410]: Detected new or changed devices on smithi183 2024-09-12T02:56:16.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:16 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:16.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:16 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:16.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:16 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:56:16.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:16 smithi116 ceph-mon[31410]: Adjusting osd_memory_target on smithi183 to 17088M 2024-09-12T02:56:16.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:16 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:16.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:16 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:16.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:16 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:16.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:16 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:16.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:16 smithi116 ceph-mon[31410]: from='osd.2 ' entity='osd.2' 2024-09-12T02:56:16.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:16.855+0000 7f82b1574640 1 -- 172.21.15.116:0/959473433 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f82ac062630 con 0x7f82940a2b20 2024-09-12T02:56:16.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:16 smithi183 ceph-mon[35535]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-12T02:56:16.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:16 smithi183 ceph-mon[35535]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T02:56:16.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:16 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:56:16.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:16 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:56:16.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:16 smithi183 ceph-mon[35535]: Detected new or changed devices on smithi183 2024-09-12T02:56:16.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:16 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:16.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:16 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:16.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:16 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:56:16.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:16 smithi183 ceph-mon[35535]: Adjusting osd_memory_target on smithi183 to 17088M 2024-09-12T02:56:16.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:16 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:16.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:16 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:16.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:16 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:16.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:16 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:16.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:16 smithi183 ceph-mon[35535]: from='osd.2 ' entity='osd.2' 2024-09-12T02:56:16.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:16 smithi177 ceph-mon[35305]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-12T02:56:16.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:16 smithi177 ceph-mon[35305]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T02:56:16.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:16 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:56:16.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:16 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:56:16.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:16 smithi177 ceph-mon[35305]: Detected new or changed devices on smithi183 2024-09-12T02:56:16.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:16 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:16.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:16 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:16.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:16 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:56:16.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:16 smithi177 ceph-mon[35305]: Adjusting osd_memory_target on smithi183 to 17088M 2024-09-12T02:56:16.996 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:16 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:16.996 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:16 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:16.996 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:16 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:16.996 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:16 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:16.996 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:16 smithi177 ceph-mon[35305]: from='osd.2 ' entity='osd.2' 2024-09-12T02:56:17.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:17 smithi116 ceph-mon[31410]: purged_snaps scrub starts 2024-09-12T02:56:17.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:17 smithi116 ceph-mon[31410]: purged_snaps scrub ok 2024-09-12T02:56:17.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:17 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:56:17.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:17 smithi116 ceph-mon[31410]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:56:17.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:17 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:56:17.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:17 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:56:17.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:17 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:17.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:17 smithi116 ceph-mon[31410]: osd.2 [v2:172.21.15.183:6800/2645677759,v1:172.21.15.183:6801/2645677759] boot 2024-09-12T02:56:17.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:17 smithi116 ceph-mon[31410]: osdmap e16: 3 total, 3 up, 3 in 2024-09-12T02:56:17.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:17 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:56:17.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:17 smithi116 ceph-mon[31410]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:17.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:17 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:56:17.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:17 smithi183 ceph-mon[35535]: purged_snaps scrub starts 2024-09-12T02:56:17.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:17 smithi183 ceph-mon[35535]: purged_snaps scrub ok 2024-09-12T02:56:17.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:17 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:56:17.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:17 smithi183 ceph-mon[35535]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:56:17.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:17 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:56:17.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:17 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:56:17.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:17 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:17.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:17 smithi183 ceph-mon[35535]: osd.2 [v2:172.21.15.183:6800/2645677759,v1:172.21.15.183:6801/2645677759] boot 2024-09-12T02:56:17.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:17 smithi183 ceph-mon[35535]: osdmap e16: 3 total, 3 up, 3 in 2024-09-12T02:56:17.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:17 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:56:17.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:17 smithi183 ceph-mon[35535]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:17.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:17 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:56:17.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:17 smithi177 ceph-mon[35305]: purged_snaps scrub starts 2024-09-12T02:56:17.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:17 smithi177 ceph-mon[35305]: purged_snaps scrub ok 2024-09-12T02:56:17.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:17 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:56:17.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:17 smithi177 ceph-mon[35305]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:56:17.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:17 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:56:17.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:17 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:56:17.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:17 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:17.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:17 smithi177 ceph-mon[35305]: osd.2 [v2:172.21.15.183:6800/2645677759,v1:172.21.15.183:6801/2645677759] boot 2024-09-12T02:56:17.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:17 smithi177 ceph-mon[35305]: osdmap e16: 3 total, 3 up, 3 in 2024-09-12T02:56:17.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:17 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:56:17.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:17 smithi177 ceph-mon[35305]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:56:17.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:17 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:56:19.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:19 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T02:56:19.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:19 smithi116 ceph-mon[31410]: osdmap e17: 3 total, 3 up, 3 in 2024-09-12T02:56:19.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:19 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:56:19.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:19 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T02:56:19.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:19 smithi183 ceph-mon[35535]: osdmap e17: 3 total, 3 up, 3 in 2024-09-12T02:56:19.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:19 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:56:19.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:19 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T02:56:19.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:19 smithi177 ceph-mon[35305]: osdmap e17: 3 total, 3 up, 3 in 2024-09-12T02:56:19.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:19 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:56:19.661 INFO:journalctl@ceph.osd.0.smithi116.stdout:Sep 12 02:56:19 smithi116 sudo[50231]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T02:56:19.661 INFO:journalctl@ceph.osd.0.smithi116.stdout:Sep 12 02:56:19 smithi116 sudo[50231]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:56:19.661 INFO:journalctl@ceph.osd.0.smithi116.stdout:Sep 12 02:56:19 smithi116 sudo[50231]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:56:19.661 INFO:journalctl@ceph.osd.0.smithi116.stdout:Sep 12 02:56:19 smithi116 sudo[50231]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:56:20.052 INFO:journalctl@ceph.osd.0.smithi116.stdout:Sep 12 02:56:19 smithi116 sudo[50259]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T02:56:20.052 INFO:journalctl@ceph.osd.0.smithi116.stdout:Sep 12 02:56:19 smithi116 sudo[50259]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:56:20.052 INFO:journalctl@ceph.osd.0.smithi116.stdout:Sep 12 02:56:19 smithi116 sudo[50259]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:56:20.052 INFO:journalctl@ceph.osd.0.smithi116.stdout:Sep 12 02:56:19 smithi116 sudo[50259]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:56:20.194 INFO:journalctl@ceph.osd.1.smithi177.stdout:Sep 12 02:56:19 smithi177 sudo[43489]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T02:56:20.194 INFO:journalctl@ceph.osd.1.smithi177.stdout:Sep 12 02:56:19 smithi177 sudo[43489]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:56:20.195 INFO:journalctl@ceph.osd.1.smithi177.stdout:Sep 12 02:56:19 smithi177 sudo[43489]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:56:20.195 INFO:journalctl@ceph.osd.1.smithi177.stdout:Sep 12 02:56:19 smithi177 sudo[43489]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:56:20.195 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:20 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T02:56:20.195 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:20 smithi177 ceph-mon[35305]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T02:56:20.195 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:20 smithi177 ceph-mon[35305]: pgmap v76: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:20.195 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:20 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1957357667' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae5e8178-c998-4648-bc05-4bce822f0cd3"}]: dispatch 2024-09-12T02:56:20.195 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:20 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1957357667' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae5e8178-c998-4648-bc05-4bce822f0cd3"}]': finished 2024-09-12T02:56:20.196 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:20 smithi177 ceph-mon[35305]: osdmap e19: 4 total, 3 up, 4 in 2024-09-12T02:56:20.196 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:20 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:56:20.196 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:20 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:20.196 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:20 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1946216064' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:56:20.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:20 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T02:56:20.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:20 smithi116 ceph-mon[31410]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T02:56:20.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:20 smithi116 ceph-mon[31410]: pgmap v76: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:20.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:20 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1957357667' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae5e8178-c998-4648-bc05-4bce822f0cd3"}]: dispatch 2024-09-12T02:56:20.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:20 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1957357667' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae5e8178-c998-4648-bc05-4bce822f0cd3"}]': finished 2024-09-12T02:56:20.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:20 smithi116 ceph-mon[31410]: osdmap e19: 4 total, 3 up, 4 in 2024-09-12T02:56:20.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:20 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:56:20.304 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:20 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:20.305 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:20 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1946216064' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:56:20.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:20 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T02:56:20.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:20 smithi183 ceph-mon[35535]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T02:56:20.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:20 smithi183 ceph-mon[35535]: pgmap v76: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:20.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:20 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1957357667' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae5e8178-c998-4648-bc05-4bce822f0cd3"}]: dispatch 2024-09-12T02:56:20.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:20 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1957357667' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae5e8178-c998-4648-bc05-4bce822f0cd3"}]': finished 2024-09-12T02:56:20.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:20 smithi183 ceph-mon[35535]: osdmap e19: 4 total, 3 up, 4 in 2024-09-12T02:56:20.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:20 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:56:20.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:20 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:20.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:20 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1946216064' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:56:20.495 INFO:journalctl@ceph.osd.1.smithi177.stdout:Sep 12 02:56:20 smithi177 sudo[43492]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T02:56:20.495 INFO:journalctl@ceph.osd.1.smithi177.stdout:Sep 12 02:56:20 smithi177 sudo[43492]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:56:20.495 INFO:journalctl@ceph.osd.1.smithi177.stdout:Sep 12 02:56:20 smithi177 sudo[43492]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:56:20.495 INFO:journalctl@ceph.osd.1.smithi177.stdout:Sep 12 02:56:20 smithi177 sudo[43492]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:56:20.715 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 02:56:20 smithi183 sudo[44568]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T02:56:20.716 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 02:56:20 smithi183 sudo[44568]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:56:20.716 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 02:56:20 smithi183 sudo[44568]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:56:20.716 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 02:56:20 smithi183 sudo[44568]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:56:21.203 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 02:56:20 smithi183 sudo[44571]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T02:56:21.203 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 02:56:20 smithi183 sudo[44571]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:56:21.203 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 02:56:20 smithi183 sudo[44571]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:56:21.203 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 02:56:20 smithi183 sudo[44571]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:56:21.238 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:20 smithi116 sudo[50496]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T02:56:21.239 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:20 smithi116 sudo[50496]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:56:21.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:20 smithi116 sudo[50496]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:56:21.240 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:20 smithi116 sudo[50496]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:56:21.500 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:21 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:21.500 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:21 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:21.500 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:21 smithi183 ceph-mon[35535]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:56:21.501 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:21 smithi183 sudo[44574]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T02:56:21.501 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:21 smithi183 sudo[44574]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:56:21.501 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:21 smithi183 sudo[44574]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:56:21.501 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:21 smithi183 sudo[44574]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:56:21.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:21 smithi116 sudo[50681]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T02:56:21.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:21 smithi116 sudo[50681]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:56:21.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:21 smithi116 sudo[50681]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:56:21.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:21 smithi116 sudo[50681]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:56:21.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:21 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:21.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:21 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:21.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:21 smithi116 ceph-mon[31410]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:56:21.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:21 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:21.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:21 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:21.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:21 smithi177 ceph-mon[35305]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:56:21.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:21 smithi183 sudo[44577]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T02:56:21.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:21 smithi183 sudo[44577]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:56:21.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:21 smithi183 sudo[44577]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:56:21.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:21 smithi183 sudo[44577]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:56:22.240 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:21 smithi177 sudo[43495]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T02:56:22.241 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:21 smithi177 sudo[43495]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:56:22.241 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:21 smithi177 sudo[43495]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:56:22.241 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:22 smithi177 sudo[43495]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:56:22.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:22.254+0000 7f8293fff640 1 -- 172.21.15.116:0/959473433 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f82a400ccf0 con 0x7f82ac1078c0 2024-09-12T02:56:22.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:22 smithi177 sudo[43498]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T02:56:22.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:22 smithi177 sudo[43498]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:56:22.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:22 smithi177 sudo[43498]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:56:22.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:22 smithi177 sudo[43498]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:56:22.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:22 smithi177 ceph-mon[35305]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:22.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:22 smithi177 ceph-mon[35305]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:56:22.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:22 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:56:22.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:22 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:22.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:22 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:56:22.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:22 smithi177 ceph-mon[35305]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:56:22.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:22 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:56:22.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:22 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:22.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:22 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:56:22.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:22 smithi177 ceph-mon[35305]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:56:22.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:22 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:56:22.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:22 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:22.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:22 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:56:22.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:22 smithi116 ceph-mon[31410]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:22.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:22 smithi116 ceph-mon[31410]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:56:22.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:22 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:56:22.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:22 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:22.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:22 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:56:22.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:22 smithi116 ceph-mon[31410]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:56:22.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:22 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:56:22.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:22 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:22.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:22 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:56:22.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:22 smithi116 ceph-mon[31410]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:56:22.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:22 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:56:22.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:22 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:22.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:22 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:56:22.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:22 smithi183 ceph-mon[35535]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:22.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:22 smithi183 ceph-mon[35535]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:56:22.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:22 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:56:22.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:22 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:22.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:22 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:56:22.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:22 smithi183 ceph-mon[35535]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:56:22.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:22 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:56:22.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:22 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:22.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:22 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:56:22.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:22 smithi183 ceph-mon[35535]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:56:22.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:22 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:56:22.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:22 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:22.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:22 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:56:23.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:23 smithi116 ceph-mon[31410]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:56:23.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:23 smithi116 ceph-mon[31410]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:56:23.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:23 smithi116 ceph-mon[31410]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T02:56:23.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:23 smithi116 ceph-mon[31410]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:23.627 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:23 smithi177 ceph-mon[35305]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:56:23.627 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:23 smithi177 ceph-mon[35305]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:56:23.627 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:23 smithi177 ceph-mon[35305]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T02:56:23.627 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:23 smithi177 ceph-mon[35305]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:23.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:23 smithi183 ceph-mon[35535]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:56:23.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:23 smithi183 ceph-mon[35535]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:56:23.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:23 smithi183 ceph-mon[35535]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T02:56:23.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:23 smithi183 ceph-mon[35535]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:26.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:26 smithi183 ceph-mon[35535]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:26.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:26 smithi177 ceph-mon[35305]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:26.498 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:26 smithi116 ceph-mon[31410]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:28.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:28 smithi116 ceph-mon[31410]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:28.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:28 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T02:56:28.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:28 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:28.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:28 smithi183 ceph-mon[35535]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:28.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:28 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T02:56:28.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:28 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:28.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:28 smithi177 ceph-mon[35305]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:28.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:28 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T02:56:28.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:28 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:29.451 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:29 smithi116 ceph-mon[31410]: Deploying daemon osd.3 on smithi116 2024-09-12T02:56:29.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:29 smithi183 ceph-mon[35535]: Deploying daemon osd.3 on smithi116 2024-09-12T02:56:29.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:29 smithi177 ceph-mon[35305]: Deploying daemon osd.3 on smithi116 2024-09-12T02:56:30.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:30 smithi116 ceph-mon[31410]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:30.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:30 smithi183 ceph-mon[35535]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:30.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:30 smithi177 ceph-mon[35305]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:32.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:32 smithi183 ceph-mon[35535]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:32.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:32 smithi177 ceph-mon[35305]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:32.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:32 smithi116 ceph-mon[31410]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:34.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:34 smithi183 ceph-mon[35535]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:34.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:34 smithi177 ceph-mon[35305]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:34.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:34 smithi116 ceph-mon[31410]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:35.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:35 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:35.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:35 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:35.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:35 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:35.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:35 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:35.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:35 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:35.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:35 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:35.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:35 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:35.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:35 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:35.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:35 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:36.373 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:36 smithi116 ceph-mon[31410]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:36.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:36 smithi183 ceph-mon[35535]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:36.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:36 smithi177 ceph-mon[35305]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:38.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:38 smithi116 ceph-mon[31410]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:38.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:38 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:38.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:38 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:38.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:38 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:38.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:38 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:38.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:38 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:38.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:38.452+0000 7f8293fff640 1 -- 172.21.15.116:0/959473433 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f82ac062630 con 0x7f82940a2b20 2024-09-12T02:56:38.452 INFO:teuthology.orchestra.run.smithi116.stdout:Created osd(s) 3 on host 'smithi116' 2024-09-12T02:56:38.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:38 smithi183 ceph-mon[35535]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:38.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:38 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:38.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:38 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:38.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:38 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:38.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:38 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:38.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:38 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:38.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:38.454+0000 7f82b1574640 1 -- 172.21.15.116:0/959473433 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f82940a2b20 msgr2=0x7f82940a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:38.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:38.454+0000 7f82b1574640 1 --2- 172.21.15.116:0/959473433 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f82940a2b20 0x7f82940a4fe0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f829c009890 tx=0x7f829c008040 comp rx=0 tx=0).stop 2024-09-12T02:56:38.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:38.454+0000 7f82b1574640 1 -- 172.21.15.116:0/959473433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82ac1078c0 msgr2=0x7f82ac19f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:38.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:38.454+0000 7f82b1574640 1 --2- 172.21.15.116:0/959473433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82ac1078c0 0x7f82ac19f680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f82a400b970 tx=0x7f82a400be30 comp rx=0 tx=0).stop 2024-09-12T02:56:38.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:38.454+0000 7f82b1574640 1 -- 172.21.15.116:0/959473433 shutdown_connections 2024-09-12T02:56:38.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:38.454+0000 7f82b1574640 1 --2- 172.21.15.116:0/959473433 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f82940a2b20 0x7f82940a4fe0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:38.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:38.454+0000 7f82b1574640 1 --2- 172.21.15.116:0/959473433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82ac1078c0 0x7f82ac19f680 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:38.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:38.454+0000 7f82b1574640 1 --2- 172.21.15.116:0/959473433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82ac104f90 0x7f82ac198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:38.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:38.454+0000 7f82b1574640 1 --2- 172.21.15.116:0/959473433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82ac102660 0x7f82ac1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:38.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:38.454+0000 7f82b1574640 1 -- 172.21.15.116:0/959473433 >> 172.21.15.116:0/959473433 conn(0x7f82ac0fc280 msgr2=0x7f82ac103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:38.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:38.454+0000 7f82b1574640 1 -- 172.21.15.116:0/959473433 shutdown_connections 2024-09-12T02:56:38.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:56:38.454+0000 7f82b1574640 1 -- 172.21.15.116:0/959473433 wait complete. 2024-09-12T02:56:38.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:38 smithi177 ceph-mon[35305]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:38.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:38 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:38.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:38 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:38.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:38 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:38.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:38 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:38.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:38 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:39.212 DEBUG:teuthology.orchestra.run.smithi116:osd.3> sudo journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.3.service 2024-09-12T02:56:39.214 INFO:tasks.cephadm:Deploying osd.4 on smithi177 with /dev/vg_nvme/lv_3... 2024-09-12T02:56:39.215 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T02:56:39.421 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:39 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:39.421 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:39 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:39.421 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:39 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:39.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:39 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:39.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:39 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:39.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:39 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:39.467 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:56:39.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:39 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:39.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:39 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:39.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:39 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:40.425 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:40 smithi116 ceph-mon[31410]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:40.425 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:40 smithi116 ceph-mon[31410]: from='osd.3 [v2:172.21.15.116:6810/686622148,v1:172.21.15.116:6811/686622148]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T02:56:40.426 INFO:journalctl@ceph.osd.3.smithi116.stdout:Sep 12 02:56:40 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-3[52293]: 2024-09-12T02:56:40.150+0000 7fe486cde640 -1 osd.3 0 log_to_monitors true 2024-09-12T02:56:40.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:40 smithi183 ceph-mon[35535]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:40.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:40 smithi183 ceph-mon[35535]: from='osd.3 [v2:172.21.15.116:6810/686622148,v1:172.21.15.116:6811/686622148]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T02:56:40.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:40 smithi177 ceph-mon[35305]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:40.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:40 smithi177 ceph-mon[35305]: from='osd.3 [v2:172.21.15.116:6810/686622148,v1:172.21.15.116:6811/686622148]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T02:56:41.512 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:41 smithi177 ceph-mon[35305]: from='osd.3 [v2:172.21.15.116:6810/686622148,v1:172.21.15.116:6811/686622148]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T02:56:41.513 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:41 smithi177 ceph-mon[35305]: osdmap e20: 4 total, 3 up, 4 in 2024-09-12T02:56:41.513 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:41 smithi177 ceph-mon[35305]: from='osd.3 [v2:172.21.15.116:6810/686622148,v1:172.21.15.116:6811/686622148]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-09-12T02:56:41.513 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:41 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:56:41.513 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:41 smithi177 ceph-mon[35305]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:41.775 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:41 smithi116 ceph-mon[31410]: from='osd.3 [v2:172.21.15.116:6810/686622148,v1:172.21.15.116:6811/686622148]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T02:56:41.776 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:41 smithi116 ceph-mon[31410]: osdmap e20: 4 total, 3 up, 4 in 2024-09-12T02:56:41.776 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:41 smithi116 ceph-mon[31410]: from='osd.3 [v2:172.21.15.116:6810/686622148,v1:172.21.15.116:6811/686622148]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-09-12T02:56:41.776 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:56:41.776 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:41 smithi116 ceph-mon[31410]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:41.776 INFO:journalctl@ceph.osd.3.smithi116.stdout:Sep 12 02:56:41 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-3[52293]: 2024-09-12T02:56:41.461+0000 7fe482c61640 -1 osd.3 0 waiting for initial osdmap 2024-09-12T02:56:41.776 INFO:journalctl@ceph.osd.3.smithi116.stdout:Sep 12 02:56:41 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-3[52293]: 2024-09-12T02:56:41.466+0000 7fe47ea89640 -1 osd.3 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:56:41.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:41 smithi183 ceph-mon[35535]: from='osd.3 [v2:172.21.15.116:6810/686622148,v1:172.21.15.116:6811/686622148]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T02:56:41.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:41 smithi183 ceph-mon[35535]: osdmap e20: 4 total, 3 up, 4 in 2024-09-12T02:56:41.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:41 smithi183 ceph-mon[35535]: from='osd.3 [v2:172.21.15.116:6810/686622148,v1:172.21.15.116:6811/686622148]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-09-12T02:56:41.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:56:41.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:41 smithi183 ceph-mon[35535]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:56:42.107 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-12T02:56:42.134 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi177:vg_nvme/lv_3 2024-09-12T02:56:42.382 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:56:42.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:42 smithi177 ceph-mon[35305]: from='osd.3 [v2:172.21.15.116:6810/686622148,v1:172.21.15.116:6811/686622148]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-09-12T02:56:42.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:42 smithi177 ceph-mon[35305]: osdmap e21: 4 total, 3 up, 4 in 2024-09-12T02:56:42.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:42 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:56:42.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:42 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:56:42.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:42 smithi177 ceph-mon[35305]: Detected new or changed devices on smithi116 2024-09-12T02:56:42.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:42 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:42.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:42 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:42.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:42 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:56:42.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:42 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:56:42.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:42 smithi177 ceph-mon[35305]: Adjusting osd_memory_target on smithi116 to 8544M 2024-09-12T02:56:42.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:42 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:42.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:42 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:42.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:42 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:42.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:42 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:42.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:42 smithi177 ceph-mon[35305]: osd.3 [v2:172.21.15.116:6810/686622148,v1:172.21.15.116:6811/686622148] boot 2024-09-12T02:56:42.747 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:42 smithi177 ceph-mon[35305]: osdmap e22: 4 total, 4 up, 4 in 2024-09-12T02:56:42.747 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:42 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:56:42.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:42 smithi116 ceph-mon[31410]: from='osd.3 [v2:172.21.15.116:6810/686622148,v1:172.21.15.116:6811/686622148]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-09-12T02:56:42.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:42 smithi116 ceph-mon[31410]: osdmap e21: 4 total, 3 up, 4 in 2024-09-12T02:56:42.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:42 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:56:42.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:42 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:56:42.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:42 smithi116 ceph-mon[31410]: Detected new or changed devices on smithi116 2024-09-12T02:56:42.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:42 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:42.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:42 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:42.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:42 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:56:42.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:42 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:56:42.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:42 smithi116 ceph-mon[31410]: Adjusting osd_memory_target on smithi116 to 8544M 2024-09-12T02:56:42.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:42 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:42.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:42 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:42.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:42 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:42.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:42 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:42.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:42 smithi116 ceph-mon[31410]: osd.3 [v2:172.21.15.116:6810/686622148,v1:172.21.15.116:6811/686622148] boot 2024-09-12T02:56:42.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:42 smithi116 ceph-mon[31410]: osdmap e22: 4 total, 4 up, 4 in 2024-09-12T02:56:42.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:42 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:56:42.854 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:42 smithi183 ceph-mon[35535]: from='osd.3 [v2:172.21.15.116:6810/686622148,v1:172.21.15.116:6811/686622148]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-09-12T02:56:42.854 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:42 smithi183 ceph-mon[35535]: osdmap e21: 4 total, 3 up, 4 in 2024-09-12T02:56:42.854 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:42 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:56:42.854 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:42 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:56:42.854 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:42 smithi183 ceph-mon[35535]: Detected new or changed devices on smithi116 2024-09-12T02:56:42.854 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:42 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:42.854 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:42 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:42.855 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:42 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:56:42.855 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:42 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:56:42.855 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:42 smithi183 ceph-mon[35535]: Adjusting osd_memory_target on smithi116 to 8544M 2024-09-12T02:56:42.855 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:42 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:42.855 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:42 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:42.855 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:42 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:42.855 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:42 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:56:42.855 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:42 smithi183 ceph-mon[35535]: osd.3 [v2:172.21.15.116:6810/686622148,v1:172.21.15.116:6811/686622148] boot 2024-09-12T02:56:42.855 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:42 smithi183 ceph-mon[35535]: osdmap e22: 4 total, 4 up, 4 in 2024-09-12T02:56:42.855 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:42 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:56:42.893 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.891+0000 7f295ad29640 1 -- 172.21.15.177:0/3969521043 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29541037c0 msgr2=0x7f295410a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:42.894 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.891+0000 7f295ad29640 1 --2- 172.21.15.177:0/3969521043 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29541037c0 0x7f295410a050 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f294c009ca0 tx=0x7f294c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:56:42.894 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.891+0000 7f295ad29640 1 -- 172.21.15.177:0/3969521043 shutdown_connections 2024-09-12T02:56:42.894 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.891+0000 7f295ad29640 1 --2- 172.21.15.177:0/3969521043 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29541037c0 0x7f295410a050 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:42.894 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.891+0000 7f295ad29640 1 --2- 172.21.15.177:0/3969521043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2954102e00 0x7f2954103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:42.894 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.891+0000 7f295ad29640 1 --2- 172.21.15.177:0/3969521043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2954101c00 0x7f2954102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:42.894 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.891+0000 7f295ad29640 1 -- 172.21.15.177:0/3969521043 >> 172.21.15.177:0/3969521043 conn(0x7f29540fd390 msgr2=0x7f29540ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:42.894 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.891+0000 7f295ad29640 1 -- 172.21.15.177:0/3969521043 shutdown_connections 2024-09-12T02:56:42.894 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.891+0000 7f295ad29640 1 -- 172.21.15.177:0/3969521043 wait complete. 2024-09-12T02:56:42.894 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.892+0000 7f295ad29640 1 Processor -- start 2024-09-12T02:56:42.894 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.892+0000 7f295ad29640 1 -- start start 2024-09-12T02:56:42.894 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.892+0000 7f295ad29640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2954101c00 0x7f2954197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:42.894 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.892+0000 7f295ad29640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2954102e00 0x7f29541983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:42.895 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.892+0000 7f295ad29640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29541037c0 0x7f295419f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:42.895 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.892+0000 7f295ad29640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f295410d860 con 0x7f2954101c00 2024-09-12T02:56:42.895 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.892+0000 7f295ad29640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f295410d6e0 con 0x7f29541037c0 2024-09-12T02:56:42.895 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.892+0000 7f295ad29640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f295410d9e0 con 0x7f2954102e00 2024-09-12T02:56:42.895 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.892+0000 7f29592c6640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29541037c0 0x7f295419f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:42.895 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.892+0000 7f29592c6640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29541037c0 0x7f295419f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:52424/0 (socket says 172.21.15.177:52424) 2024-09-12T02:56:42.895 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.892+0000 7f29592c6640 1 -- 172.21.15.177:0/214589729 learned_addr learned my addr 172.21.15.177:0/214589729 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-12T02:56:42.895 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.892+0000 7f2953fff640 1 --2- 172.21.15.177:0/214589729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2954102e00 0x7f29541983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:42.895 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.892+0000 7f2958ac5640 1 --2- 172.21.15.177:0/214589729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2954101c00 0x7f2954197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:42.895 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.892+0000 7f29592c6640 1 -- 172.21.15.177:0/214589729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2954102e00 msgr2=0x7f29541983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:42.895 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.892+0000 7f29592c6640 1 --2- 172.21.15.177:0/214589729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2954102e00 0x7f29541983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:42.896 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.892+0000 7f29592c6640 1 -- 172.21.15.177:0/214589729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2954101c00 msgr2=0x7f2954197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:42.896 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.892+0000 7f29592c6640 1 --2- 172.21.15.177:0/214589729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2954101c00 0x7f2954197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:42.896 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.892+0000 7f29592c6640 1 -- 172.21.15.177:0/214589729 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f295419fae0 con 0x7f29541037c0 2024-09-12T02:56:42.896 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.893+0000 7f29592c6640 1 --2- 172.21.15.177:0/214589729 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29541037c0 0x7f295419f470 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f294c005e30 tx=0x7f294c031ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:42.896 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.893+0000 7f2951ffb640 1 -- 172.21.15.177:0/214589729 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c003c30 con 0x7f29541037c0 2024-09-12T02:56:42.896 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.893+0000 7f2951ffb640 1 -- 172.21.15.177:0/214589729 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f294c003dd0 con 0x7f29541037c0 2024-09-12T02:56:42.896 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.893+0000 7f295ad29640 1 -- 172.21.15.177:0/214589729 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f295419fd70 con 0x7f29541037c0 2024-09-12T02:56:42.896 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.893+0000 7f2951ffb640 1 -- 172.21.15.177:0/214589729 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c02fde0 con 0x7f29541037c0 2024-09-12T02:56:42.896 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.893+0000 7f295ad29640 1 -- 172.21.15.177:0/214589729 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29541a0250 con 0x7f29541037c0 2024-09-12T02:56:42.897 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.894+0000 7f295ad29640 1 -- 172.21.15.177:0/214589729 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2954108fd0 con 0x7f29541037c0 2024-09-12T02:56:42.897 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.895+0000 7f2951ffb640 1 -- 172.21.15.177:0/214589729 <== mon.2 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f294c004050 con 0x7f29541037c0 2024-09-12T02:56:42.897 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.896+0000 7f2951ffb640 1 --2- 172.21.15.177:0/214589729 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f29440a2bb0 0x7f29440a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:42.897 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.896+0000 7f2951ffb640 1 -- 172.21.15.177:0/214589729 <== mon.2 v2:172.21.15.177:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 3702+0+0 (secure 0 0 0) 0x7f294c0eefc0 con 0x7f29541037c0 2024-09-12T02:56:42.897 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.896+0000 7f2958ac5640 1 --2- 172.21.15.177:0/214589729 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f29440a2bb0 0x7f29440a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:42.897 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.897+0000 7f2958ac5640 1 --2- 172.21.15.177:0/214589729 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f29440a2bb0 0x7f29440a5070 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2954102c60 tx=0x7f294800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:42.900 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:42.900+0000 7f2951ffb640 1 -- 172.21.15.177:0/214589729 <== mon.2 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f294c0bd120 con 0x7f29541037c0 2024-09-12T02:56:43.072 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:56:43.072+0000 7f295ad29640 1 -- 172.21.15.177:0/214589729 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2954062630 con 0x7f29440a2bb0 2024-09-12T02:56:43.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:43 smithi177 ceph-mon[35305]: purged_snaps scrub starts 2024-09-12T02:56:43.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:43 smithi177 ceph-mon[35305]: purged_snaps scrub ok 2024-09-12T02:56:43.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:43 smithi177 ceph-mon[35305]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:56:43.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:43 smithi177 ceph-mon[35305]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:56:43.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:43 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:56:43.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:43 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:56:43.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:43 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:43.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:43 smithi177 ceph-mon[35305]: osdmap e23: 4 total, 4 up, 4 in 2024-09-12T02:56:43.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:43 smithi116 ceph-mon[31410]: purged_snaps scrub starts 2024-09-12T02:56:43.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:43 smithi116 ceph-mon[31410]: purged_snaps scrub ok 2024-09-12T02:56:43.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:43 smithi116 ceph-mon[31410]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:56:43.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:43 smithi116 ceph-mon[31410]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:56:43.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:43 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:56:43.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:43 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:56:43.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:43 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:43.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:43 smithi116 ceph-mon[31410]: osdmap e23: 4 total, 4 up, 4 in 2024-09-12T02:56:43.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:43 smithi183 ceph-mon[35535]: purged_snaps scrub starts 2024-09-12T02:56:43.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:43 smithi183 ceph-mon[35535]: purged_snaps scrub ok 2024-09-12T02:56:43.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:43 smithi183 ceph-mon[35535]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:56:43.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:43 smithi183 ceph-mon[35535]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:56:43.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:43 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:56:43.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:43 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:56:43.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:43 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:43.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:43 smithi183 ceph-mon[35535]: osdmap e23: 4 total, 4 up, 4 in 2024-09-12T02:56:45.679 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:45 smithi177 ceph-mon[35305]: osdmap e24: 4 total, 4 up, 4 in 2024-09-12T02:56:45.680 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:45 smithi177 ceph-mon[35305]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:56:45.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:45 smithi183 ceph-mon[35535]: osdmap e24: 4 total, 4 up, 4 in 2024-09-12T02:56:45.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:45 smithi183 ceph-mon[35535]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:56:45.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:45 smithi116 ceph-mon[31410]: osdmap e24: 4 total, 4 up, 4 in 2024-09-12T02:56:45.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:45 smithi116 ceph-mon[31410]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:56:46.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:46 smithi183 ceph-mon[35535]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T02:56:46.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:46 smithi183 ceph-mon[35535]: from='client.? 172.21.15.177:0/2090763276' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c81d2e73-0bc2-440d-b5a2-b7428af3ea4d"}]: dispatch 2024-09-12T02:56:46.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:46 smithi183 ceph-mon[35535]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c81d2e73-0bc2-440d-b5a2-b7428af3ea4d"}]: dispatch 2024-09-12T02:56:46.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:46 smithi183 ceph-mon[35535]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c81d2e73-0bc2-440d-b5a2-b7428af3ea4d"}]': finished 2024-09-12T02:56:46.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:46 smithi183 ceph-mon[35535]: osdmap e26: 5 total, 4 up, 5 in 2024-09-12T02:56:46.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:46 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:56:46.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:46 smithi183 ceph-mon[35535]: from='client.? 172.21.15.177:0/940822005' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:56:46.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:46 smithi177 ceph-mon[35305]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T02:56:46.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:46 smithi177 ceph-mon[35305]: from='client.? 172.21.15.177:0/2090763276' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c81d2e73-0bc2-440d-b5a2-b7428af3ea4d"}]: dispatch 2024-09-12T02:56:46.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:46 smithi177 ceph-mon[35305]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c81d2e73-0bc2-440d-b5a2-b7428af3ea4d"}]: dispatch 2024-09-12T02:56:46.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:46 smithi177 ceph-mon[35305]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c81d2e73-0bc2-440d-b5a2-b7428af3ea4d"}]': finished 2024-09-12T02:56:46.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:46 smithi177 ceph-mon[35305]: osdmap e26: 5 total, 4 up, 5 in 2024-09-12T02:56:46.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:46 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:56:46.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:46 smithi177 ceph-mon[35305]: from='client.? 172.21.15.177:0/940822005' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:56:46.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:46 smithi116 ceph-mon[31410]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T02:56:46.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:46 smithi116 ceph-mon[31410]: from='client.? 172.21.15.177:0/2090763276' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c81d2e73-0bc2-440d-b5a2-b7428af3ea4d"}]: dispatch 2024-09-12T02:56:46.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:46 smithi116 ceph-mon[31410]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c81d2e73-0bc2-440d-b5a2-b7428af3ea4d"}]: dispatch 2024-09-12T02:56:46.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:46 smithi116 ceph-mon[31410]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c81d2e73-0bc2-440d-b5a2-b7428af3ea4d"}]': finished 2024-09-12T02:56:46.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:46 smithi116 ceph-mon[31410]: osdmap e26: 5 total, 4 up, 5 in 2024-09-12T02:56:46.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:46 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:56:46.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:46 smithi116 ceph-mon[31410]: from='client.? 172.21.15.177:0/940822005' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:56:47.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:47 smithi183 ceph-mon[35535]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:56:47.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:47 smithi177 ceph-mon[35305]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:56:47.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:47 smithi116 ceph-mon[31410]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:56:50.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:50 smithi177 ceph-mon[35305]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-12T02:56:50.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:50 smithi183 ceph-mon[35535]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-12T02:56:50.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:50 smithi116 ceph-mon[31410]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-12T02:56:52.373 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:52 smithi177 ceph-mon[35305]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-12T02:56:52.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:52 smithi183 ceph-mon[35535]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-12T02:56:52.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:52 smithi116 ceph-mon[31410]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-12T02:56:54.302 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:54 smithi177 ceph-mon[35305]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T02:56:54.302 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:54 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T02:56:54.302 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:54 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:54.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:54 smithi183 ceph-mon[35535]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T02:56:54.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:54 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T02:56:54.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:54 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:54.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:54 smithi116 ceph-mon[31410]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T02:56:54.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:54 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T02:56:54.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:54 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:55.395 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:55 smithi177 ceph-mon[35305]: Deploying daemon osd.4 on smithi177 2024-09-12T02:56:55.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:55 smithi183 ceph-mon[35535]: Deploying daemon osd.4 on smithi177 2024-09-12T02:56:55.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:55 smithi116 ceph-mon[31410]: Deploying daemon osd.4 on smithi177 2024-09-12T02:56:56.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:56 smithi183 ceph-mon[35535]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-12T02:56:56.453 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:56 smithi116 ceph-mon[31410]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-12T02:56:56.459 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:56 smithi177 ceph-mon[35305]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-12T02:56:58.411 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:56:58 smithi177 ceph-mon[35305]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-12T02:56:58.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:56:58 smithi183 ceph-mon[35535]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-12T02:56:58.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:56:58 smithi116 ceph-mon[31410]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-12T02:57:00.407 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:00 smithi177 ceph-mon[35305]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:57:00.407 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:00 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:00.407 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:00 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:00.407 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:00 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:00.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:00 smithi183 ceph-mon[35535]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:57:00.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:00 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:00.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:00 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:00.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:00 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:00.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:00 smithi116 ceph-mon[31410]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:57:00.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:00 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:00.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:00 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:00.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:00 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:02.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:02 smithi183 ceph-mon[35535]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:57:02.453 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:02 smithi177 ceph-mon[35305]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:57:02.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:02 smithi116 ceph-mon[31410]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:57:03.901 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:03 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:03.901 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:03 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:03.902 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:03 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:03.902 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:03 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:03.902 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:03 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:03.902 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:03 smithi177 ceph-mon[35305]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:57:03.940 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:03.939+0000 7f2951ffb640 1 -- 172.21.15.177:0/214589729 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2954062630 con 0x7f29440a2bb0 2024-09-12T02:57:03.941 INFO:teuthology.orchestra.run.smithi177.stdout:Created osd(s) 4 on host 'smithi177' 2024-09-12T02:57:03.943 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:03.942+0000 7f295ad29640 1 -- 172.21.15.177:0/214589729 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f29440a2bb0 msgr2=0x7f29440a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:03.943 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:03.942+0000 7f295ad29640 1 --2- 172.21.15.177:0/214589729 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f29440a2bb0 0x7f29440a5070 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2954102c60 tx=0x7f294800a2d0 comp rx=0 tx=0).stop 2024-09-12T02:57:03.944 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:03.942+0000 7f295ad29640 1 -- 172.21.15.177:0/214589729 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29541037c0 msgr2=0x7f295419f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:03.944 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:03.942+0000 7f295ad29640 1 --2- 172.21.15.177:0/214589729 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29541037c0 0x7f295419f470 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f294c005e30 tx=0x7f294c031ee0 comp rx=0 tx=0).stop 2024-09-12T02:57:03.944 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:03.942+0000 7f295ad29640 1 -- 172.21.15.177:0/214589729 shutdown_connections 2024-09-12T02:57:03.944 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:03.942+0000 7f295ad29640 1 --2- 172.21.15.177:0/214589729 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f29440a2bb0 0x7f29440a5070 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:03.944 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:03.942+0000 7f295ad29640 1 --2- 172.21.15.177:0/214589729 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29541037c0 0x7f295419f470 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:03.944 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:03.942+0000 7f295ad29640 1 --2- 172.21.15.177:0/214589729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2954102e00 0x7f29541983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:03.944 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:03.942+0000 7f295ad29640 1 --2- 172.21.15.177:0/214589729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2954101c00 0x7f2954197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:03.944 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:03.942+0000 7f295ad29640 1 -- 172.21.15.177:0/214589729 >> 172.21.15.177:0/214589729 conn(0x7f29540fd390 msgr2=0x7f29541061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:03.945 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:03.942+0000 7f295ad29640 1 -- 172.21.15.177:0/214589729 shutdown_connections 2024-09-12T02:57:03.945 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:03.942+0000 7f295ad29640 1 -- 172.21.15.177:0/214589729 wait complete. 2024-09-12T02:57:04.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:03 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:04.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:03 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:04.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:03 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:04.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:03 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:04.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:03 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:04.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:03 smithi116 ceph-mon[31410]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:57:04.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:03 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:04.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:03 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:04.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:03 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:04.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:03 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:04.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:03 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:04.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:03 smithi183 ceph-mon[35535]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:57:04.613 DEBUG:teuthology.orchestra.run.smithi177:osd.4> sudo journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.4.service 2024-09-12T02:57:04.615 INFO:tasks.cephadm:Deploying osd.5 on smithi183 with /dev/vg_nvme/lv_3... 2024-09-12T02:57:04.616 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T02:57:04.869 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:57:04.884 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:04 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:04.884 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:04 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:04.885 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:04 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:04.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:04 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:04.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:04 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:04.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:04 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:05.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:04 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:05.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:04 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:05.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:04 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:05.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:05 smithi177 ceph-mon[35305]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:57:05.997 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:05 smithi183 ceph-mon[35535]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:57:06.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:05 smithi116 ceph-mon[31410]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:57:06.296 INFO:journalctl@ceph.osd.4.smithi177.stdout:Sep 12 02:57:06 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-4[46254]: 2024-09-12T02:57:06.023+0000 7fdb60c27640 -1 osd.4 0 log_to_monitors true 2024-09-12T02:57:06.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:06 smithi183 ceph-mon[35535]: from='osd.4 [v2:172.21.15.177:6808/2382938021,v1:172.21.15.177:6809/2382938021]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T02:57:06.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:06 smithi183 ceph-mon[35535]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T02:57:06.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:06 smithi177 ceph-mon[35305]: from='osd.4 [v2:172.21.15.177:6808/2382938021,v1:172.21.15.177:6809/2382938021]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T02:57:06.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:06 smithi177 ceph-mon[35305]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T02:57:07.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:06 smithi116 ceph-mon[31410]: from='osd.4 [v2:172.21.15.177:6808/2382938021,v1:172.21.15.177:6809/2382938021]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T02:57:07.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:06 smithi116 ceph-mon[31410]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T02:57:07.294 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T02:57:07.320 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_3 2024-09-12T02:57:07.580 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:57:07.927 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:07 smithi183 ceph-mon[35535]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T02:57:07.927 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:07 smithi183 ceph-mon[35535]: osdmap e27: 5 total, 4 up, 5 in 2024-09-12T02:57:07.927 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:07 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:57:07.927 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:07 smithi183 ceph-mon[35535]: from='osd.4 [v2:172.21.15.177:6808/2382938021,v1:172.21.15.177:6809/2382938021]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-09-12T02:57:07.927 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:07 smithi183 ceph-mon[35535]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-09-12T02:57:07.927 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:07 smithi183 ceph-mon[35535]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:57:07.928 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:07 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:07.928 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:07 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:07.928 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:07 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:57:07.928 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:07 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:57:07.928 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:07 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:07.928 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:07 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:07.928 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:07 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:07.928 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:07 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:08.027 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.011+0000 7f19f0811640 1 -- 172.21.15.183:0/1277428897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19ec1026b0 msgr2=0x7f19ec108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:08.027 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.011+0000 7f19f0811640 1 --2- 172.21.15.183:0/1277428897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19ec1026b0 0x7f19ec108f40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f19e4009ca0 tx=0x7f19e402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:57:08.027 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.011+0000 7f19f0811640 1 -- 172.21.15.183:0/1277428897 shutdown_connections 2024-09-12T02:57:08.027 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.011+0000 7f19f0811640 1 --2- 172.21.15.183:0/1277428897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19ec1026b0 0x7f19ec108f40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:08.027 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.011+0000 7f19f0811640 1 --2- 172.21.15.183:0/1277428897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19ec101cf0 0x7f19ec102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:08.027 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.011+0000 7f19f0811640 1 --2- 172.21.15.183:0/1277428897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f19ec100af0 0x7f19ec100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:08.027 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.011+0000 7f19f0811640 1 -- 172.21.15.183:0/1277428897 >> 172.21.15.183:0/1277428897 conn(0x7f19ec0fc280 msgr2=0x7f19ec0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:08.027 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.011+0000 7f19f0811640 1 -- 172.21.15.183:0/1277428897 shutdown_connections 2024-09-12T02:57:08.027 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.011+0000 7f19f0811640 1 -- 172.21.15.183:0/1277428897 wait complete. 2024-09-12T02:57:08.027 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.012+0000 7f19f0811640 1 Processor -- start 2024-09-12T02:57:08.028 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.012+0000 7f19f0811640 1 -- start start 2024-09-12T02:57:08.028 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.012+0000 7f19f0811640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19ec100af0 0x7f19ec197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:08.028 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.012+0000 7f19f0811640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f19ec101cf0 0x7f19ec1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:08.028 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.012+0000 7f19f0811640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19ec1026b0 0x7f19ec19f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:08.028 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.012+0000 7f19f0811640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f19ec10c770 con 0x7f19ec101cf0 2024-09-12T02:57:08.028 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.012+0000 7f19f0811640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f19ec10c5f0 con 0x7f19ec100af0 2024-09-12T02:57:08.028 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.012+0000 7f19f0811640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f19ec10c8f0 con 0x7f19ec1026b0 2024-09-12T02:57:08.028 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.013+0000 7f19ead9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19ec1026b0 0x7f19ec19f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:08.028 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.013+0000 7f19ea59c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19ec100af0 0x7f19ec197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:08.028 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.013+0000 7f19ea59c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19ec100af0 0x7f19ec197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.183:40472/0 (socket says 172.21.15.183:40472) 2024-09-12T02:57:08.029 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.013+0000 7f19ead9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19ec1026b0 0x7f19ec19f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:54304/0 (socket says 172.21.15.183:54304) 2024-09-12T02:57:08.029 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.013+0000 7f19ea59c640 1 -- 172.21.15.183:0/3742801353 learned_addr learned my addr 172.21.15.183:0/3742801353 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-12T02:57:08.029 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.013+0000 7f19e9d9b640 1 --2- 172.21.15.183:0/3742801353 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f19ec101cf0 0x7f19ec1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:08.029 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.013+0000 7f19ea59c640 1 -- 172.21.15.183:0/3742801353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19ec1026b0 msgr2=0x7f19ec19f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:08.029 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.013+0000 7f19ea59c640 1 --2- 172.21.15.183:0/3742801353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19ec1026b0 0x7f19ec19f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:08.029 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.013+0000 7f19ea59c640 1 -- 172.21.15.183:0/3742801353 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f19ec101cf0 msgr2=0x7f19ec1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:08.029 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.013+0000 7f19ea59c640 1 --2- 172.21.15.183:0/3742801353 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f19ec101cf0 0x7f19ec1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:08.029 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.013+0000 7f19ea59c640 1 -- 172.21.15.183:0/3742801353 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19ec19fac0 con 0x7f19ec100af0 2024-09-12T02:57:08.029 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.013+0000 7f19ead9d640 1 --2- 172.21.15.183:0/3742801353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19ec1026b0 0x7f19ec19f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:57:08.029 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.013+0000 7f19ea59c640 1 --2- 172.21.15.183:0/3742801353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19ec100af0 0x7f19ec197e90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00d770 tx=0x7f19dc00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:08.029 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.013+0000 7f19db7fe640 1 -- 172.21.15.183:0/3742801353 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc017070 con 0x7f19ec100af0 2024-09-12T02:57:08.029 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.013+0000 7f19f0811640 1 -- 172.21.15.183:0/3742801353 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19ec19fdb0 con 0x7f19ec100af0 2024-09-12T02:57:08.029 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.013+0000 7f19db7fe640 1 -- 172.21.15.183:0/3742801353 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19dc009c30 con 0x7f19ec100af0 2024-09-12T02:57:08.030 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.013+0000 7f19db7fe640 1 -- 172.21.15.183:0/3742801353 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc003dd0 con 0x7f19ec100af0 2024-09-12T02:57:08.030 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.013+0000 7f19f0811640 1 -- 172.21.15.183:0/3742801353 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19ec1a0260 con 0x7f19ec100af0 2024-09-12T02:57:08.030 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.014+0000 7f19f0811640 1 -- 172.21.15.183:0/3742801353 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19ec1a0490 con 0x7f19ec100af0 2024-09-12T02:57:08.030 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.015+0000 7f19db7fe640 1 -- 172.21.15.183:0/3742801353 <== mon.2 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f19dc003f70 con 0x7f19ec100af0 2024-09-12T02:57:08.030 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.016+0000 7f19db7fe640 1 --2- 172.21.15.183:0/3742801353 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f19d40a2bb0 0x7f19d40a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:08.030 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.016+0000 7f19db7fe640 1 -- 172.21.15.183:0/3742801353 <== mon.2 v2:172.21.15.177:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3821+0+0 (secure 0 0 0) 0x7f19dc0cd8c0 con 0x7f19ec100af0 2024-09-12T02:57:08.030 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.016+0000 7f19e9d9b640 1 --2- 172.21.15.183:0/3742801353 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f19d40a2bb0 0x7f19d40a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:08.030 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.017+0000 7f19e9d9b640 1 --2- 172.21.15.183:0/3742801353 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f19d40a2bb0 0x7f19d40a5070 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f19ec1993b0 tx=0x7f19e0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:08.030 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.020+0000 7f19db7fe640 1 -- 172.21.15.183:0/3742801353 <== mon.2 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f19dc09b910 con 0x7f19ec100af0 2024-09-12T02:57:08.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:07 smithi116 ceph-mon[31410]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T02:57:08.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:07 smithi116 ceph-mon[31410]: osdmap e27: 5 total, 4 up, 5 in 2024-09-12T02:57:08.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:07 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:57:08.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:07 smithi116 ceph-mon[31410]: from='osd.4 [v2:172.21.15.177:6808/2382938021,v1:172.21.15.177:6809/2382938021]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-09-12T02:57:08.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:07 smithi116 ceph-mon[31410]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-09-12T02:57:08.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:07 smithi116 ceph-mon[31410]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:57:08.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:07 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:08.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:07 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:08.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:07 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:57:08.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:07 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:57:08.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:07 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:08.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:07 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:08.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:07 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:08.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:07 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:08.113 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:07 smithi177 ceph-mon[35305]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T02:57:08.113 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:07 smithi177 ceph-mon[35305]: osdmap e27: 5 total, 4 up, 5 in 2024-09-12T02:57:08.113 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:07 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:57:08.113 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:07 smithi177 ceph-mon[35305]: from='osd.4 [v2:172.21.15.177:6808/2382938021,v1:172.21.15.177:6809/2382938021]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-09-12T02:57:08.113 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:07 smithi177 ceph-mon[35305]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-09-12T02:57:08.113 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:07 smithi177 ceph-mon[35305]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:57:08.114 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:07 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:08.114 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:07 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:08.114 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:07 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:57:08.114 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:07 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:57:08.114 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:07 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:08.114 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:07 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:08.114 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:07 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:08.114 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:07 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:08.198 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:08.197+0000 7f19f0811640 1 -- 172.21.15.183:0/3742801353 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f19ec06aed0 con 0x7f19d40a2bb0 2024-09-12T02:57:08.494 INFO:journalctl@ceph.osd.4.smithi177.stdout:Sep 12 02:57:08 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-4[46254]: 2024-09-12T02:57:08.112+0000 7fdb5cbaa640 -1 osd.4 0 waiting for initial osdmap 2024-09-12T02:57:08.495 INFO:journalctl@ceph.osd.4.smithi177.stdout:Sep 12 02:57:08 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-4[46254]: 2024-09-12T02:57:08.118+0000 7fdb549ca640 -1 osd.4 28 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:57:08.847 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:08 smithi183 ceph-mon[35535]: Detected new or changed devices on smithi177 2024-09-12T02:57:08.847 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:08 smithi183 ceph-mon[35535]: Adjusting osd_memory_target on smithi177 to 8543M 2024-09-12T02:57:08.847 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:08 smithi183 ceph-mon[35535]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-09-12T02:57:08.847 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:08 smithi183 ceph-mon[35535]: osdmap e28: 5 total, 4 up, 5 in 2024-09-12T02:57:08.848 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:08 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:57:08.848 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:08 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:57:08.848 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:08 smithi183 ceph-mon[35535]: from='osd.4 ' entity='osd.4' 2024-09-12T02:57:08.848 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:08 smithi183 ceph-mon[35535]: from='client.24242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:57:08.848 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:08 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:57:08.848 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:08 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:57:08.848 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:08 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:09.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:08 smithi116 ceph-mon[31410]: Detected new or changed devices on smithi177 2024-09-12T02:57:09.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:08 smithi116 ceph-mon[31410]: Adjusting osd_memory_target on smithi177 to 8543M 2024-09-12T02:57:09.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:08 smithi116 ceph-mon[31410]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-09-12T02:57:09.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:08 smithi116 ceph-mon[31410]: osdmap e28: 5 total, 4 up, 5 in 2024-09-12T02:57:09.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:08 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:57:09.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:08 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:57:09.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:08 smithi116 ceph-mon[31410]: from='osd.4 ' entity='osd.4' 2024-09-12T02:57:09.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:08 smithi116 ceph-mon[31410]: from='client.24242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:57:09.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:08 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:57:09.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:08 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:57:09.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:08 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:09.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:08 smithi177 ceph-mon[35305]: Detected new or changed devices on smithi177 2024-09-12T02:57:09.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:08 smithi177 ceph-mon[35305]: Adjusting osd_memory_target on smithi177 to 8543M 2024-09-12T02:57:09.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:08 smithi177 ceph-mon[35305]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-09-12T02:57:09.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:08 smithi177 ceph-mon[35305]: osdmap e28: 5 total, 4 up, 5 in 2024-09-12T02:57:09.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:08 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:57:09.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:08 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:57:09.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:08 smithi177 ceph-mon[35305]: from='osd.4 ' entity='osd.4' 2024-09-12T02:57:09.246 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:08 smithi177 ceph-mon[35305]: from='client.24242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:57:09.246 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:08 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:57:09.246 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:08 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:57:09.246 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:08 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:09.856 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:09 smithi183 ceph-mon[35535]: purged_snaps scrub starts 2024-09-12T02:57:09.856 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:09 smithi183 ceph-mon[35535]: purged_snaps scrub ok 2024-09-12T02:57:09.856 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:09 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:57:09.856 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:09 smithi183 ceph-mon[35535]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:57:09.856 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:09 smithi183 ceph-mon[35535]: osd.4 [v2:172.21.15.177:6808/2382938021,v1:172.21.15.177:6809/2382938021] boot 2024-09-12T02:57:09.856 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:09 smithi183 ceph-mon[35535]: osdmap e29: 5 total, 5 up, 5 in 2024-09-12T02:57:09.856 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:09 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:57:10.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:09 smithi116 ceph-mon[31410]: purged_snaps scrub starts 2024-09-12T02:57:10.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:09 smithi116 ceph-mon[31410]: purged_snaps scrub ok 2024-09-12T02:57:10.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:09 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:57:10.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:09 smithi116 ceph-mon[31410]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:57:10.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:09 smithi116 ceph-mon[31410]: osd.4 [v2:172.21.15.177:6808/2382938021,v1:172.21.15.177:6809/2382938021] boot 2024-09-12T02:57:10.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:09 smithi116 ceph-mon[31410]: osdmap e29: 5 total, 5 up, 5 in 2024-09-12T02:57:10.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:09 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:57:10.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:09 smithi177 ceph-mon[35305]: purged_snaps scrub starts 2024-09-12T02:57:10.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:09 smithi177 ceph-mon[35305]: purged_snaps scrub ok 2024-09-12T02:57:10.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:09 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:57:10.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:09 smithi177 ceph-mon[35305]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:57:10.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:09 smithi177 ceph-mon[35305]: osd.4 [v2:172.21.15.177:6808/2382938021,v1:172.21.15.177:6809/2382938021] boot 2024-09-12T02:57:10.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:09 smithi177 ceph-mon[35305]: osdmap e29: 5 total, 5 up, 5 in 2024-09-12T02:57:10.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:09 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:57:11.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:10 smithi116 ceph-mon[31410]: from='client.? 172.21.15.183:0/1948364162' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3f168be-9e27-4cf5-b73e-3f47d8990ed3"}]: dispatch 2024-09-12T02:57:11.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:10 smithi116 ceph-mon[31410]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3f168be-9e27-4cf5-b73e-3f47d8990ed3"}]: dispatch 2024-09-12T02:57:11.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:10 smithi116 ceph-mon[31410]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3f168be-9e27-4cf5-b73e-3f47d8990ed3"}]': finished 2024-09-12T02:57:11.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:10 smithi116 ceph-mon[31410]: osdmap e30: 6 total, 5 up, 6 in 2024-09-12T02:57:11.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:10 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:57:11.185 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:10 smithi183 ceph-mon[35535]: from='client.? 172.21.15.183:0/1948364162' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3f168be-9e27-4cf5-b73e-3f47d8990ed3"}]: dispatch 2024-09-12T02:57:11.186 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:10 smithi183 ceph-mon[35535]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3f168be-9e27-4cf5-b73e-3f47d8990ed3"}]: dispatch 2024-09-12T02:57:11.186 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:10 smithi183 ceph-mon[35535]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3f168be-9e27-4cf5-b73e-3f47d8990ed3"}]': finished 2024-09-12T02:57:11.186 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:10 smithi183 ceph-mon[35535]: osdmap e30: 6 total, 5 up, 6 in 2024-09-12T02:57:11.186 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:10 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:57:11.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:10 smithi177 ceph-mon[35305]: from='client.? 172.21.15.183:0/1948364162' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3f168be-9e27-4cf5-b73e-3f47d8990ed3"}]: dispatch 2024-09-12T02:57:11.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:10 smithi177 ceph-mon[35305]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3f168be-9e27-4cf5-b73e-3f47d8990ed3"}]: dispatch 2024-09-12T02:57:11.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:10 smithi177 ceph-mon[35305]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3f168be-9e27-4cf5-b73e-3f47d8990ed3"}]': finished 2024-09-12T02:57:11.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:10 smithi177 ceph-mon[35305]: osdmap e30: 6 total, 5 up, 6 in 2024-09-12T02:57:11.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:10 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:57:12.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:11 smithi116 ceph-mon[31410]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:12.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:11 smithi116 ceph-mon[31410]: from='client.? 172.21.15.183:0/4288967742' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:57:12.145 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:11 smithi177 ceph-mon[35305]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:12.146 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:11 smithi177 ceph-mon[35305]: from='client.? 172.21.15.183:0/4288967742' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:57:12.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:11 smithi183 ceph-mon[35535]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:12.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:11 smithi183 ceph-mon[35535]: from='client.? 172.21.15.183:0/4288967742' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:57:14.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:14 smithi183 ceph-mon[35535]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:14.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:14 smithi177 ceph-mon[35305]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:14.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:14 smithi116 ceph-mon[31410]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:16.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:16 smithi183 ceph-mon[35535]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:16.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:16 smithi177 ceph-mon[35305]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:16.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:16 smithi116 ceph-mon[31410]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:18.273 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:18 smithi183 ceph-mon[35535]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:18.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:18 smithi177 ceph-mon[35305]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:18.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:18 smithi116 ceph-mon[31410]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:20.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:20 smithi183 ceph-mon[35535]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:20.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:20 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T02:57:20.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:20 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:20.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:20 smithi177 ceph-mon[35305]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:20.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:20 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T02:57:20.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:20 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:20.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:20 smithi116 ceph-mon[31410]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:20.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:20 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T02:57:20.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:20 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:21.223 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:21 smithi183 ceph-mon[35535]: Deploying daemon osd.5 on smithi183 2024-09-12T02:57:21.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:21 smithi177 ceph-mon[35305]: Deploying daemon osd.5 on smithi183 2024-09-12T02:57:21.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:21 smithi116 ceph-mon[31410]: Deploying daemon osd.5 on smithi183 2024-09-12T02:57:22.290 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:22 smithi183 ceph-mon[35535]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:22.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:22 smithi177 ceph-mon[35305]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:22.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:22 smithi116 ceph-mon[31410]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:24.271 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:24 smithi183 ceph-mon[35535]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:24.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:24 smithi177 ceph-mon[35305]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:24.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:24 smithi116 ceph-mon[31410]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:26.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:26 smithi183 ceph-mon[35535]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:26.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:26 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:26.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:26 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:26.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:26 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:26.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:26 smithi177 ceph-mon[35305]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:26.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:26 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:26.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:26 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:26.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:26 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:26.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:26 smithi116 ceph-mon[31410]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:26.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:26 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:26.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:26 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:26.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:26 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:28.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:28 smithi183 ceph-mon[35535]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:28.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:28 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:28.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:28 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:28.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:28 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:28.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:28 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:28.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:28 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:28.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:28 smithi177 ceph-mon[35305]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:28.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:28 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:28.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:28 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:28.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:28 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:28.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:28 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:28.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:28 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:28.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:28 smithi116 ceph-mon[31410]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:28.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:28 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:28.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:28 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:28.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:28 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:28.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:28 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:28.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:28 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:29.433 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:29.432+0000 7f19db7fe640 1 -- 172.21.15.183:0/3742801353 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f19ec06aed0 con 0x7f19d40a2bb0 2024-09-12T02:57:29.433 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 5 on host 'smithi183' 2024-09-12T02:57:29.435 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:29.434+0000 7f19f0811640 1 -- 172.21.15.183:0/3742801353 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f19d40a2bb0 msgr2=0x7f19d40a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:29.435 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:29.434+0000 7f19f0811640 1 --2- 172.21.15.183:0/3742801353 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f19d40a2bb0 0x7f19d40a5070 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f19ec1993b0 tx=0x7f19e0005f90 comp rx=0 tx=0).stop 2024-09-12T02:57:29.435 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:29.434+0000 7f19f0811640 1 -- 172.21.15.183:0/3742801353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19ec100af0 msgr2=0x7f19ec197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:29.436 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:29.434+0000 7f19f0811640 1 --2- 172.21.15.183:0/3742801353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19ec100af0 0x7f19ec197e90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00d770 tx=0x7f19dc00dc30 comp rx=0 tx=0).stop 2024-09-12T02:57:29.436 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:29.434+0000 7f19f0811640 1 -- 172.21.15.183:0/3742801353 shutdown_connections 2024-09-12T02:57:29.436 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:29.434+0000 7f19f0811640 1 --2- 172.21.15.183:0/3742801353 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f19d40a2bb0 0x7f19d40a5070 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:29.436 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:29.434+0000 7f19f0811640 1 --2- 172.21.15.183:0/3742801353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19ec1026b0 0x7f19ec19f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:29.436 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:29.434+0000 7f19f0811640 1 --2- 172.21.15.183:0/3742801353 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f19ec101cf0 0x7f19ec1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:29.436 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:29.434+0000 7f19f0811640 1 --2- 172.21.15.183:0/3742801353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19ec100af0 0x7f19ec197e90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:29.436 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:29.435+0000 7f19f0811640 1 -- 172.21.15.183:0/3742801353 >> 172.21.15.183:0/3742801353 conn(0x7f19ec0fc280 msgr2=0x7f19ec06a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:29.436 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:29.435+0000 7f19f0811640 1 -- 172.21.15.183:0/3742801353 shutdown_connections 2024-09-12T02:57:29.436 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:57:29.435+0000 7f19f0811640 1 -- 172.21.15.183:0/3742801353 wait complete. 2024-09-12T02:57:30.127 DEBUG:teuthology.orchestra.run.smithi183:osd.5> sudo journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.5.service 2024-09-12T02:57:30.129 INFO:tasks.cephadm:Deploying osd.6 on smithi116 with /dev/vg_nvme/lv_2... 2024-09-12T02:57:30.129 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T02:57:30.386 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:30.395 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:30 smithi183 ceph-mon[35535]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:30.396 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:30 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:30.396 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:30 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:30.396 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:30 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:30.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:30 smithi177 ceph-mon[35305]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:30.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:30 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:30.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:30 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:30.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:30 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:30.507 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:30 smithi116 ceph-mon[31410]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:30.507 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:30 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:30.507 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:30 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:30.507 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:30 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:32.145 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:57:31 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T02:57:31.821+0000 7fef0f13c640 -1 osd.5 0 log_to_monitors true 2024-09-12T02:57:32.418 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:32 smithi116 ceph-mon[31410]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:32.418 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:32 smithi116 ceph-mon[31410]: from='osd.5 [v2:172.21.15.183:6808/85363432,v1:172.21.15.183:6809/85363432]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T02:57:32.418 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:32 smithi116 ceph-mon[31410]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T02:57:32.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:32 smithi183 ceph-mon[35535]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:32.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:32 smithi183 ceph-mon[35535]: from='osd.5 [v2:172.21.15.183:6808/85363432,v1:172.21.15.183:6809/85363432]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T02:57:32.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:32 smithi183 ceph-mon[35535]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T02:57:32.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:32 smithi177 ceph-mon[35305]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:32.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:32 smithi177 ceph-mon[35305]: from='osd.5 [v2:172.21.15.183:6808/85363432,v1:172.21.15.183:6809/85363432]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T02:57:32.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:32 smithi177 ceph-mon[35305]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T02:57:32.991 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:57:33.017 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi116:vg_nvme/lv_2 2024-09-12T02:57:33.273 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:33.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:33 smithi116 ceph-mon[31410]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T02:57:33.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:33 smithi116 ceph-mon[31410]: osdmap e31: 6 total, 5 up, 6 in 2024-09-12T02:57:33.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:33 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:57:33.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:33 smithi116 ceph-mon[31410]: from='osd.5 [v2:172.21.15.183:6808/85363432,v1:172.21.15.183:6809/85363432]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T02:57:33.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:33 smithi116 ceph-mon[31410]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T02:57:33.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:33 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:33.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:33 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:33.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:33 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:57:33.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:33 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:57:33.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:33 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:33.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:33 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:33.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:33 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:33.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:33 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:33.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:33 smithi183 ceph-mon[35535]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T02:57:33.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:33 smithi183 ceph-mon[35535]: osdmap e31: 6 total, 5 up, 6 in 2024-09-12T02:57:33.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:33 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:57:33.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:33 smithi183 ceph-mon[35535]: from='osd.5 [v2:172.21.15.183:6808/85363432,v1:172.21.15.183:6809/85363432]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T02:57:33.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:33 smithi183 ceph-mon[35535]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T02:57:33.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:33 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:33.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:33 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:33.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:33 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:57:33.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:33 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:57:33.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:33 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:33.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:33 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:33.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:33 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:33.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:33 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:33.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:33 smithi177 ceph-mon[35305]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T02:57:33.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:33 smithi177 ceph-mon[35305]: osdmap e31: 6 total, 5 up, 6 in 2024-09-12T02:57:33.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:33 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:57:33.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:33 smithi177 ceph-mon[35305]: from='osd.5 [v2:172.21.15.183:6808/85363432,v1:172.21.15.183:6809/85363432]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T02:57:33.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:33 smithi177 ceph-mon[35305]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T02:57:33.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:33 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:33.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:33 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:33.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:33 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:57:33.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:33 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:57:33.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:33 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:33.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:33 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:33.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:33 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:33.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:33 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:33.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.764+0000 7f0d765ac640 1 -- 172.21.15.116:0/1723909667 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d70105010 msgr2=0x7f0d70107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:33.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.764+0000 7f0d765ac640 1 --2- 172.21.15.116:0/1723909667 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d70105010 0x7f0d70107400 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0d64009900 tx=0x7f0d6402a410 comp rx=0 tx=0).stop 2024-09-12T02:57:33.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.764+0000 7f0d765ac640 1 -- 172.21.15.116:0/1723909667 shutdown_connections 2024-09-12T02:57:33.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.764+0000 7f0d765ac640 1 --2- 172.21.15.116:0/1723909667 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d70107940 0x7f0d70109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.764+0000 7f0d765ac640 1 --2- 172.21.15.116:0/1723909667 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d70105010 0x7f0d70107400 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.764+0000 7f0d765ac640 1 --2- 172.21.15.116:0/1723909667 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d701026e0 0x7f0d70104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.764+0000 7f0d765ac640 1 -- 172.21.15.116:0/1723909667 >> 172.21.15.116:0/1723909667 conn(0x7f0d700fc280 msgr2=0x7f0d700fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:33.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.764+0000 7f0d765ac640 1 -- 172.21.15.116:0/1723909667 shutdown_connections 2024-09-12T02:57:33.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.764+0000 7f0d765ac640 1 -- 172.21.15.116:0/1723909667 wait complete. 2024-09-12T02:57:33.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.764+0000 7f0d765ac640 1 Processor -- start 2024-09-12T02:57:33.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.764+0000 7f0d765ac640 1 -- start start 2024-09-12T02:57:33.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.765+0000 7f0d765ac640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d701026e0 0x7f0d70197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:33.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.765+0000 7f0d765ac640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d70105010 0x7f0d701984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:33.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.765+0000 7f0d765ac640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d70107940 0x7f0d7019f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:33.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.765+0000 7f0d765ac640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7010d850 con 0x7f0d70105010 2024-09-12T02:57:33.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.765+0000 7f0d765ac640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7010d6d0 con 0x7f0d70107940 2024-09-12T02:57:33.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.765+0000 7f0d765ac640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7010d9d0 con 0x7f0d701026e0 2024-09-12T02:57:33.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.765+0000 7f0d6f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d70105010 0x7f0d701984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:33.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.765+0000 7f0d6f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d70105010 0x7f0d701984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48042/0 (socket says 172.21.15.116:48042) 2024-09-12T02:57:33.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.765+0000 7f0d6f7fe640 1 -- 172.21.15.116:0/2278217234 learned_addr learned my addr 172.21.15.116:0/2278217234 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:57:33.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.765+0000 7f0d6ffff640 1 --2- 172.21.15.116:0/2278217234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d701026e0 0x7f0d70197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:33.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.765+0000 7f0d74b49640 1 --2- 172.21.15.116:0/2278217234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d70107940 0x7f0d7019f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:33.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.765+0000 7f0d6f7fe640 1 -- 172.21.15.116:0/2278217234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d701026e0 msgr2=0x7f0d70197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:33.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.765+0000 7f0d6f7fe640 1 --2- 172.21.15.116:0/2278217234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d701026e0 0x7f0d70197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.765+0000 7f0d6f7fe640 1 -- 172.21.15.116:0/2278217234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d70107940 msgr2=0x7f0d7019f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:33.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.765+0000 7f0d6f7fe640 1 --2- 172.21.15.116:0/2278217234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d70107940 0x7f0d7019f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.765+0000 7f0d6f7fe640 1 -- 172.21.15.116:0/2278217234 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7019fb90 con 0x7f0d70105010 2024-09-12T02:57:33.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.765+0000 7f0d74b49640 1 --2- 172.21.15.116:0/2278217234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d70107940 0x7f0d7019f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:33.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.765+0000 7f0d6f7fe640 1 --2- 172.21.15.116:0/2278217234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d70105010 0x7f0d701984a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0d64031610 tx=0x7f0d64004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:33.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.765+0000 7f0d6d7fa640 1 -- 172.21.15.116:0/2278217234 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6403f030 con 0x7f0d70105010 2024-09-12T02:57:33.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.765+0000 7f0d6d7fa640 1 -- 172.21.15.116:0/2278217234 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d64003ad0 con 0x7f0d70105010 2024-09-12T02:57:33.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.765+0000 7f0d765ac640 1 -- 172.21.15.116:0/2278217234 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d7019fe20 con 0x7f0d70105010 2024-09-12T02:57:33.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.765+0000 7f0d6d7fa640 1 -- 172.21.15.116:0/2278217234 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64035070 con 0x7f0d70105010 2024-09-12T02:57:33.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.766+0000 7f0d765ac640 1 -- 172.21.15.116:0/2278217234 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d701a0280 con 0x7f0d70105010 2024-09-12T02:57:33.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.767+0000 7f0d765ac640 1 -- 172.21.15.116:0/2278217234 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d700fdd70 con 0x7f0d70105010 2024-09-12T02:57:33.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.768+0000 7f0d6d7fa640 1 -- 172.21.15.116:0/2278217234 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f0d64042030 con 0x7f0d70105010 2024-09-12T02:57:33.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.768+0000 7f0d6d7fa640 1 --2- 172.21.15.116:0/2278217234 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0d580a2c30 0x7f0d580a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:33.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.768+0000 7f0d6d7fa640 1 -- 172.21.15.116:0/2278217234 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4253+0+0 (secure 0 0 0) 0x7f0d640ee690 con 0x7f0d70105010 2024-09-12T02:57:33.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.768+0000 7f0d6ffff640 1 --2- 172.21.15.116:0/2278217234 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0d580a2c30 0x7f0d580a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:33.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.769+0000 7f0d6ffff640 1 --2- 172.21.15.116:0/2278217234 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0d580a2c30 0x7f0d580a50f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009bd0 tx=0x7f0d60005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:33.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.773+0000 7f0d6d7fa640 1 -- 172.21.15.116:0/2278217234 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0d640bc5d0 con 0x7f0d70105010 2024-09-12T02:57:33.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:33.949+0000 7f0d765ac640 1 -- 172.21.15.116:0/2278217234 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f0d70062630 con 0x7f0d580a2c30 2024-09-12T02:57:33.953 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:57:33 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T02:57:33.538+0000 7fef0b0bf640 -1 osd.5 0 waiting for initial osdmap 2024-09-12T02:57:33.953 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 02:57:33 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T02:57:33.545+0000 7fef02edf640 -1 osd.5 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:57:34.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:34 smithi116 ceph-mon[31410]: Detected new or changed devices on smithi183 2024-09-12T02:57:34.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:34 smithi116 ceph-mon[31410]: Adjusting osd_memory_target on smithi183 to 8544M 2024-09-12T02:57:34.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:34 smithi116 ceph-mon[31410]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:34.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:34 smithi116 ceph-mon[31410]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-12T02:57:34.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:34 smithi116 ceph-mon[31410]: osdmap e32: 6 total, 5 up, 6 in 2024-09-12T02:57:34.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:34 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:57:34.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:34 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:57:34.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:34 smithi116 ceph-mon[31410]: from='osd.5 ' entity='osd.5' 2024-09-12T02:57:34.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:34 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:57:34.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:34 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:57:34.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:34 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:34.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:34 smithi116 ceph-mon[31410]: osd.5 [v2:172.21.15.183:6808/85363432,v1:172.21.15.183:6809/85363432] boot 2024-09-12T02:57:34.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:34 smithi116 ceph-mon[31410]: osdmap e33: 6 total, 6 up, 6 in 2024-09-12T02:57:34.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:34 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:57:34.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:34 smithi183 ceph-mon[35535]: Detected new or changed devices on smithi183 2024-09-12T02:57:34.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:34 smithi183 ceph-mon[35535]: Adjusting osd_memory_target on smithi183 to 8544M 2024-09-12T02:57:34.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:34 smithi183 ceph-mon[35535]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:34.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:34 smithi183 ceph-mon[35535]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-12T02:57:34.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:34 smithi183 ceph-mon[35535]: osdmap e32: 6 total, 5 up, 6 in 2024-09-12T02:57:34.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:34 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:57:34.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:34 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:57:34.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:34 smithi183 ceph-mon[35535]: from='osd.5 ' entity='osd.5' 2024-09-12T02:57:34.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:34 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:57:34.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:34 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:57:34.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:34 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:34.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:34 smithi183 ceph-mon[35535]: osd.5 [v2:172.21.15.183:6808/85363432,v1:172.21.15.183:6809/85363432] boot 2024-09-12T02:57:34.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:34 smithi183 ceph-mon[35535]: osdmap e33: 6 total, 6 up, 6 in 2024-09-12T02:57:34.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:34 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:57:34.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:34 smithi177 ceph-mon[35305]: Detected new or changed devices on smithi183 2024-09-12T02:57:34.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:34 smithi177 ceph-mon[35305]: Adjusting osd_memory_target on smithi183 to 8544M 2024-09-12T02:57:34.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:34 smithi177 ceph-mon[35305]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:57:34.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:34 smithi177 ceph-mon[35305]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-12T02:57:34.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:34 smithi177 ceph-mon[35305]: osdmap e32: 6 total, 5 up, 6 in 2024-09-12T02:57:34.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:34 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:57:34.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:34 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:57:34.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:34 smithi177 ceph-mon[35305]: from='osd.5 ' entity='osd.5' 2024-09-12T02:57:34.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:34 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:57:34.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:34 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:57:34.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:34 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:34.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:34 smithi177 ceph-mon[35305]: osd.5 [v2:172.21.15.183:6808/85363432,v1:172.21.15.183:6809/85363432] boot 2024-09-12T02:57:34.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:34 smithi177 ceph-mon[35305]: osdmap e33: 6 total, 6 up, 6 in 2024-09-12T02:57:34.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:34 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:57:35.414 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:35 smithi116 ceph-mon[31410]: purged_snaps scrub starts 2024-09-12T02:57:35.414 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:35 smithi116 ceph-mon[31410]: purged_snaps scrub ok 2024-09-12T02:57:35.414 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:35 smithi116 ceph-mon[31410]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:57:35.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:35 smithi183 ceph-mon[35535]: purged_snaps scrub starts 2024-09-12T02:57:35.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:35 smithi183 ceph-mon[35535]: purged_snaps scrub ok 2024-09-12T02:57:35.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:35 smithi183 ceph-mon[35535]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:57:35.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:35 smithi177 ceph-mon[35305]: purged_snaps scrub starts 2024-09-12T02:57:35.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:35 smithi177 ceph-mon[35305]: purged_snaps scrub ok 2024-09-12T02:57:35.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:35 smithi177 ceph-mon[35305]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:57:36.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:36 smithi183 ceph-mon[35535]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 180 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:57:36.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:36 smithi183 ceph-mon[35535]: osdmap e34: 6 total, 6 up, 6 in 2024-09-12T02:57:36.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:36 smithi177 ceph-mon[35305]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 180 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:57:36.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:36 smithi177 ceph-mon[35305]: osdmap e34: 6 total, 6 up, 6 in 2024-09-12T02:57:36.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:36 smithi116 ceph-mon[31410]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 180 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:57:36.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:36 smithi116 ceph-mon[31410]: osdmap e34: 6 total, 6 up, 6 in 2024-09-12T02:57:37.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:37 smithi116 ceph-mon[31410]: osdmap e35: 6 total, 6 up, 6 in 2024-09-12T02:57:37.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:37 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/453633778' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "beb0a7e9-4909-47e2-9ee7-6951d4745e5d"}]: dispatch 2024-09-12T02:57:37.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:37 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/453633778' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "beb0a7e9-4909-47e2-9ee7-6951d4745e5d"}]': finished 2024-09-12T02:57:37.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:37 smithi116 ceph-mon[31410]: osdmap e36: 7 total, 6 up, 7 in 2024-09-12T02:57:37.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:37 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:57:37.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:37 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3153177251' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:57:37.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:37 smithi183 ceph-mon[35535]: osdmap e35: 6 total, 6 up, 6 in 2024-09-12T02:57:37.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:37 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/453633778' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "beb0a7e9-4909-47e2-9ee7-6951d4745e5d"}]: dispatch 2024-09-12T02:57:37.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:37 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/453633778' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "beb0a7e9-4909-47e2-9ee7-6951d4745e5d"}]': finished 2024-09-12T02:57:37.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:37 smithi183 ceph-mon[35535]: osdmap e36: 7 total, 6 up, 7 in 2024-09-12T02:57:37.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:37 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:57:37.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:37 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3153177251' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:57:37.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:37 smithi177 ceph-mon[35305]: osdmap e35: 6 total, 6 up, 6 in 2024-09-12T02:57:37.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:37 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/453633778' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "beb0a7e9-4909-47e2-9ee7-6951d4745e5d"}]: dispatch 2024-09-12T02:57:37.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:37 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/453633778' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "beb0a7e9-4909-47e2-9ee7-6951d4745e5d"}]': finished 2024-09-12T02:57:37.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:37 smithi177 ceph-mon[35305]: osdmap e36: 7 total, 6 up, 7 in 2024-09-12T02:57:37.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:37 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:57:37.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:37 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3153177251' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:57:38.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:38 smithi183 ceph-mon[35535]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 180 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:57:38.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:38 smithi183 ceph-mon[35535]: osdmap e37: 7 total, 6 up, 7 in 2024-09-12T02:57:38.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:38 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:57:38.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:38 smithi177 ceph-mon[35305]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 180 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:57:38.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:38 smithi177 ceph-mon[35305]: osdmap e37: 7 total, 6 up, 7 in 2024-09-12T02:57:38.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:38 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:57:38.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:38 smithi116 ceph-mon[31410]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 180 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:57:38.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:38 smithi116 ceph-mon[31410]: osdmap e37: 7 total, 6 up, 7 in 2024-09-12T02:57:38.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:38 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:57:40.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:40 smithi183 ceph-mon[35535]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T02:57:40.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:40 smithi177 ceph-mon[35305]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T02:57:40.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:40 smithi116 ceph-mon[31410]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T02:57:42.265 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:42 smithi116 ceph-mon[31410]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-12T02:57:42.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:42 smithi183 ceph-mon[35535]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-12T02:57:42.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:42 smithi177 ceph-mon[35305]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-12T02:57:44.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:44 smithi183 ceph-mon[35535]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-12T02:57:44.494 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:44 smithi116 ceph-mon[31410]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-12T02:57:44.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:44 smithi177 ceph-mon[35305]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-12T02:57:46.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:46 smithi116 ceph-mon[31410]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-12T02:57:46.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:46 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T02:57:46.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:46 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:46.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:46 smithi183 ceph-mon[35535]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-12T02:57:46.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:46 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T02:57:46.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:46 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:46.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:46 smithi177 ceph-mon[35305]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-12T02:57:46.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:46 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T02:57:46.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:46 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:47.283 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:47 smithi116 ceph-mon[31410]: Deploying daemon osd.6 on smithi116 2024-09-12T02:57:47.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:47 smithi183 ceph-mon[35535]: Deploying daemon osd.6 on smithi116 2024-09-12T02:57:47.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:47 smithi177 ceph-mon[35305]: Deploying daemon osd.6 on smithi116 2024-09-12T02:57:48.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:48 smithi116 ceph-mon[31410]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T02:57:48.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:48 smithi183 ceph-mon[35535]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T02:57:48.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:48 smithi177 ceph-mon[35305]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T02:57:50.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:50 smithi183 ceph-mon[35535]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-12T02:57:50.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:50 smithi177 ceph-mon[35305]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-12T02:57:50.513 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:50 smithi116 ceph-mon[31410]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-12T02:57:52.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:52 smithi183 ceph-mon[35535]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:57:52.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:52 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:52.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:52 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:52.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:52 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:52.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:52 smithi177 ceph-mon[35305]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:57:52.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:52.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:52.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:52.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:52 smithi116 ceph-mon[31410]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:57:52.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:52 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:52.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:52 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:52.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:52 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:54.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:54 smithi116 ceph-mon[31410]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:57:54.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:54 smithi177 ceph-mon[35305]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:57:54.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:54 smithi183 ceph-mon[35535]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:57:55.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:55.213+0000 7f0d6d7fa640 1 -- 172.21.15.116:0/2278217234 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0d70062630 con 0x7f0d580a2c30 2024-09-12T02:57:55.214 INFO:teuthology.orchestra.run.smithi116.stdout:Created osd(s) 6 on host 'smithi116' 2024-09-12T02:57:55.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:55.216+0000 7f0d765ac640 1 -- 172.21.15.116:0/2278217234 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0d580a2c30 msgr2=0x7f0d580a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:55.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:55.216+0000 7f0d765ac640 1 --2- 172.21.15.116:0/2278217234 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0d580a2c30 0x7f0d580a50f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009bd0 tx=0x7f0d60005f90 comp rx=0 tx=0).stop 2024-09-12T02:57:55.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:55.216+0000 7f0d765ac640 1 -- 172.21.15.116:0/2278217234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d70105010 msgr2=0x7f0d701984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:55.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:55.216+0000 7f0d765ac640 1 --2- 172.21.15.116:0/2278217234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d70105010 0x7f0d701984a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0d64031610 tx=0x7f0d64004270 comp rx=0 tx=0).stop 2024-09-12T02:57:55.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:55.216+0000 7f0d765ac640 1 -- 172.21.15.116:0/2278217234 shutdown_connections 2024-09-12T02:57:55.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:55.216+0000 7f0d765ac640 1 --2- 172.21.15.116:0/2278217234 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0d580a2c30 0x7f0d580a50f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:55.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:55.216+0000 7f0d765ac640 1 --2- 172.21.15.116:0/2278217234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d70107940 0x7f0d7019f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:55.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:55.216+0000 7f0d765ac640 1 --2- 172.21.15.116:0/2278217234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d70105010 0x7f0d701984a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:55.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:55.216+0000 7f0d765ac640 1 --2- 172.21.15.116:0/2278217234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d701026e0 0x7f0d70197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:55.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:55.216+0000 7f0d765ac640 1 -- 172.21.15.116:0/2278217234 >> 172.21.15.116:0/2278217234 conn(0x7f0d700fc280 msgr2=0x7f0d70103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:55.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:55.216+0000 7f0d765ac640 1 -- 172.21.15.116:0/2278217234 shutdown_connections 2024-09-12T02:57:55.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:57:55.216+0000 7f0d765ac640 1 -- 172.21.15.116:0/2278217234 wait complete. 2024-09-12T02:57:55.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:55 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:55.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:55 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:55.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:55 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:55.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:55 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:55.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:55 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:55.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:55 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:55.522 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:55 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:55.522 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:55 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:55.522 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:55 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:55.522 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:55 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:55.522 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:55 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:55.522 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:55 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:55.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:55 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:55.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:55 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:55.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:55 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:55.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:55 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:55.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:55 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:55.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:55 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:55.859 DEBUG:teuthology.orchestra.run.smithi116:osd.6> sudo journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.6.service 2024-09-12T02:57:55.861 INFO:tasks.cephadm:Deploying osd.7 on smithi177 with /dev/vg_nvme/lv_2... 2024-09-12T02:57:55.861 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T02:57:56.115 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:57:56.367 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:56 smithi116 ceph-mon[31410]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:57:56.368 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:56 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:56.368 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:56 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:56.458 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:56 smithi177 ceph-mon[35305]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:57:56.458 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:56 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:56.458 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:56 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:56 smithi183 ceph-mon[35535]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:57:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:56 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:56 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:58.053 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 02:57:57 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T02:57:57.815+0000 7fc3988f3640 -1 osd.6 0 log_to_monitors true 2024-09-12T02:57:58.400 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:58 smithi177 ceph-mon[35305]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:57:58.400 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:58 smithi177 ceph-mon[35305]: from='osd.6 [v2:172.21.15.116:6818/1121291186,v1:172.21.15.116:6819/1121291186]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T02:57:58.411 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:58 smithi116 ceph-mon[31410]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:57:58.411 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:58 smithi116 ceph-mon[31410]: from='osd.6 [v2:172.21.15.116:6818/1121291186,v1:172.21.15.116:6819/1121291186]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T02:57:58.699 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-12T02:57:58.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:58 smithi183 ceph-mon[35535]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:57:58.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:58 smithi183 ceph-mon[35535]: from='osd.6 [v2:172.21.15.116:6818/1121291186,v1:172.21.15.116:6819/1121291186]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T02:57:58.725 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi177:vg_nvme/lv_2 2024-09-12T02:57:58.972 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:57:59.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:59 smithi177 ceph-mon[35305]: from='osd.6 [v2:172.21.15.116:6818/1121291186,v1:172.21.15.116:6819/1121291186]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T02:57:59.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:59 smithi177 ceph-mon[35305]: osdmap e38: 7 total, 6 up, 7 in 2024-09-12T02:57:59.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:59 smithi177 ceph-mon[35305]: from='osd.6 [v2:172.21.15.116:6818/1121291186,v1:172.21.15.116:6819/1121291186]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-09-12T02:57:59.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:59 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:57:59.357 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:59 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:59.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:59 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:59.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:59 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:57:59.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:59 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:57:59.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:59 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:57:59.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:59 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:59.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:59 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:59.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:59 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:59.358 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:57:59 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:59.491 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.452+0000 7f43439d1640 1 -- 172.21.15.177:0/2393326158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f433c102df0 msgr2=0x7f433c103270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:59.491 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.452+0000 7f43439d1640 1 --2- 172.21.15.177:0/2393326158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f433c102df0 0x7f433c103270 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f432c009900 tx=0x7f432c02a410 comp rx=0 tx=0).stop 2024-09-12T02:57:59.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.452+0000 7f43439d1640 1 -- 172.21.15.177:0/2393326158 shutdown_connections 2024-09-12T02:57:59.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.452+0000 7f43439d1640 1 --2- 172.21.15.177:0/2393326158 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f433c1037b0 0x7f433c10a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:59.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.452+0000 7f43439d1640 1 --2- 172.21.15.177:0/2393326158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f433c102df0 0x7f433c103270 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:59.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.452+0000 7f43439d1640 1 --2- 172.21.15.177:0/2393326158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f433c101bf0 0x7f433c101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:59.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.452+0000 7f43439d1640 1 -- 172.21.15.177:0/2393326158 >> 172.21.15.177:0/2393326158 conn(0x7f433c0fd3c0 msgr2=0x7f433c0ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:59.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.452+0000 7f43439d1640 1 -- 172.21.15.177:0/2393326158 shutdown_connections 2024-09-12T02:57:59.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.452+0000 7f43439d1640 1 -- 172.21.15.177:0/2393326158 wait complete. 2024-09-12T02:57:59.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.452+0000 7f43439d1640 1 Processor -- start 2024-09-12T02:57:59.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.452+0000 7f43439d1640 1 -- start start 2024-09-12T02:57:59.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.452+0000 7f43439d1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f433c101bf0 0x7f433c197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:59.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.453+0000 7f43439d1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f433c102df0 0x7f433c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:59.493 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.453+0000 7f43439d1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f433c1037b0 0x7f433c19f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:59.493 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.453+0000 7f43439d1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f433c10d8f0 con 0x7f433c101bf0 2024-09-12T02:57:59.493 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.453+0000 7f43439d1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f433c10d770 con 0x7f433c102df0 2024-09-12T02:57:59.493 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.453+0000 7f43439d1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f433c10da70 con 0x7f433c1037b0 2024-09-12T02:57:59.493 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.453+0000 7f4340f6c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f433c102df0 0x7f433c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:59.493 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.453+0000 7f4340f6c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f433c102df0 0x7f433c198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:40334/0 (socket says 172.21.15.177:40334) 2024-09-12T02:57:59.493 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.453+0000 7f4340f6c640 1 -- 172.21.15.177:0/2528313629 learned_addr learned my addr 172.21.15.177:0/2528313629 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-12T02:57:59.493 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.453+0000 7f4341f6e640 1 --2- 172.21.15.177:0/2528313629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f433c1037b0 0x7f433c19f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:59.493 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.453+0000 7f4340f6c640 1 -- 172.21.15.177:0/2528313629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f433c1037b0 msgr2=0x7f433c19f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:59.493 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.453+0000 7f4340f6c640 1 --2- 172.21.15.177:0/2528313629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f433c1037b0 0x7f433c19f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:59.493 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.453+0000 7f4340f6c640 1 -- 172.21.15.177:0/2528313629 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f433c101bf0 msgr2=0x7f433c197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:59.493 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.453+0000 7f4340f6c640 1 --2- 172.21.15.177:0/2528313629 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f433c101bf0 0x7f433c197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:59.493 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.453+0000 7f4340f6c640 1 -- 172.21.15.177:0/2528313629 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433c19fb70 con 0x7f433c102df0 2024-09-12T02:57:59.494 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.453+0000 7f4341f6e640 1 --2- 172.21.15.177:0/2528313629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f433c1037b0 0x7f433c19f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:59.494 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.453+0000 7f4340f6c640 1 --2- 172.21.15.177:0/2528313629 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f433c102df0 0x7f433c198480 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f432c009230 tx=0x7f432c004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:59.494 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.453+0000 7f43327fc640 1 -- 172.21.15.177:0/2528313629 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c03f030 con 0x7f433c102df0 2024-09-12T02:57:59.494 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.453+0000 7f43327fc640 1 -- 172.21.15.177:0/2528313629 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f432c003ce0 con 0x7f433c102df0 2024-09-12T02:57:59.494 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.453+0000 7f43439d1640 1 -- 172.21.15.177:0/2528313629 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f433c19fe00 con 0x7f433c102df0 2024-09-12T02:57:59.494 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.453+0000 7f43327fc640 1 -- 172.21.15.177:0/2528313629 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c035070 con 0x7f433c102df0 2024-09-12T02:57:59.494 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.453+0000 7f43439d1640 1 -- 172.21.15.177:0/2528313629 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f433c1a02d0 con 0x7f433c102df0 2024-09-12T02:57:59.494 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.454+0000 7f43439d1640 1 -- 172.21.15.177:0/2528313629 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f433c108fa0 con 0x7f433c102df0 2024-09-12T02:57:59.494 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.455+0000 7f43327fc640 1 -- 172.21.15.177:0/2528313629 <== mon.2 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f432c042030 con 0x7f433c102df0 2024-09-12T02:57:59.494 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.456+0000 7f43327fc640 1 --2- 172.21.15.177:0/2528313629 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f43280a2bb0 0x7f43280a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:59.494 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.456+0000 7f43327fc640 1 -- 172.21.15.177:0/2528313629 <== mon.2 v2:172.21.15.177:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4685+0+0 (secure 0 0 0) 0x7f432c0ee8f0 con 0x7f433c102df0 2024-09-12T02:57:59.494 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.456+0000 7f434176d640 1 --2- 172.21.15.177:0/2528313629 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f43280a2bb0 0x7f43280a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:59.494 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.457+0000 7f434176d640 1 --2- 172.21.15.177:0/2528313629 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f43280a2bb0 0x7f43280a5070 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f433c102c50 tx=0x7f4338006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:59.495 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.460+0000 7f43327fc640 1 -- 172.21.15.177:0/2528313629 <== mon.2 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f432c0bc680 con 0x7f433c102df0 2024-09-12T02:57:59.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:59 smithi116 ceph-mon[31410]: from='osd.6 [v2:172.21.15.116:6818/1121291186,v1:172.21.15.116:6819/1121291186]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T02:57:59.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:59 smithi116 ceph-mon[31410]: osdmap e38: 7 total, 6 up, 7 in 2024-09-12T02:57:59.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:59 smithi116 ceph-mon[31410]: from='osd.6 [v2:172.21.15.116:6818/1121291186,v1:172.21.15.116:6819/1121291186]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-09-12T02:57:59.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:59 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:57:59.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:59 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:59.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:59 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:59.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:59 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:57:59.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:59 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:57:59.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:59 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:57:59.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:59 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:59.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:59 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:59.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:59 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:59.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:57:59 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:59.637 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:57:59.636+0000 7f43439d1640 1 -- 172.21.15.177:0/2528313629 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f433c062630 con 0x7f43280a2bb0 2024-09-12T02:57:59.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:59 smithi183 ceph-mon[35535]: from='osd.6 [v2:172.21.15.116:6818/1121291186,v1:172.21.15.116:6819/1121291186]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T02:57:59.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:59 smithi183 ceph-mon[35535]: osdmap e38: 7 total, 6 up, 7 in 2024-09-12T02:57:59.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:59 smithi183 ceph-mon[35535]: from='osd.6 [v2:172.21.15.116:6818/1121291186,v1:172.21.15.116:6819/1121291186]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-09-12T02:57:59.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:59 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:57:59.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:59 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:59.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:59 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:59.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:59 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:57:59.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:59 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:57:59.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:59 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:57:59.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:59 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:57:59.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:59 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:59.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:59 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:59.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:57:59 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:00.052 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 02:57:59 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T02:57:59.629+0000 7fc394876640 -1 osd.6 0 waiting for initial osdmap 2024-09-12T02:58:00.052 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 02:57:59 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T02:57:59.636+0000 7fc38c696640 -1 osd.6 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:58:00.353 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:00 smithi177 ceph-mon[35305]: Detected new or changed devices on smithi116 2024-09-12T02:58:00.353 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:00 smithi177 ceph-mon[35305]: Adjusting osd_memory_target on smithi116 to 5696M 2024-09-12T02:58:00.353 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:00 smithi177 ceph-mon[35305]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:58:00.353 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:00 smithi177 ceph-mon[35305]: from='osd.6 [v2:172.21.15.116:6818/1121291186,v1:172.21.15.116:6819/1121291186]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-09-12T02:58:00.353 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:00 smithi177 ceph-mon[35305]: osdmap e39: 7 total, 6 up, 7 in 2024-09-12T02:58:00.353 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:00 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:58:00.354 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:00 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:58:00.354 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:00 smithi177 ceph-mon[35305]: from='osd.6 [v2:172.21.15.116:6818/1121291186,v1:172.21.15.116:6819/1121291186]' entity='osd.6' 2024-09-12T02:58:00.354 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:00 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:58:00.354 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:00 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:58:00.354 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:00 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:00.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:00 smithi116 ceph-mon[31410]: Detected new or changed devices on smithi116 2024-09-12T02:58:00.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:00 smithi116 ceph-mon[31410]: Adjusting osd_memory_target on smithi116 to 5696M 2024-09-12T02:58:00.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:00 smithi116 ceph-mon[31410]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:58:00.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:00 smithi116 ceph-mon[31410]: from='osd.6 [v2:172.21.15.116:6818/1121291186,v1:172.21.15.116:6819/1121291186]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-09-12T02:58:00.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:00 smithi116 ceph-mon[31410]: osdmap e39: 7 total, 6 up, 7 in 2024-09-12T02:58:00.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:00 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:58:00.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:00 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:58:00.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:00 smithi116 ceph-mon[31410]: from='osd.6 [v2:172.21.15.116:6818/1121291186,v1:172.21.15.116:6819/1121291186]' entity='osd.6' 2024-09-12T02:58:00.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:00 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:58:00.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:00 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:58:00.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:00 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:00.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:00 smithi183 ceph-mon[35535]: Detected new or changed devices on smithi116 2024-09-12T02:58:00.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:00 smithi183 ceph-mon[35535]: Adjusting osd_memory_target on smithi116 to 5696M 2024-09-12T02:58:00.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:00 smithi183 ceph-mon[35535]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:58:00.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:00 smithi183 ceph-mon[35535]: from='osd.6 [v2:172.21.15.116:6818/1121291186,v1:172.21.15.116:6819/1121291186]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-09-12T02:58:00.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:00 smithi183 ceph-mon[35535]: osdmap e39: 7 total, 6 up, 7 in 2024-09-12T02:58:00.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:00 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:58:00.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:00 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:58:00.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:00 smithi183 ceph-mon[35535]: from='osd.6 [v2:172.21.15.116:6818/1121291186,v1:172.21.15.116:6819/1121291186]' entity='osd.6' 2024-09-12T02:58:00.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:00 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:58:00.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:00 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:58:00.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:00 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:01.325 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:01 smithi177 ceph-mon[35305]: purged_snaps scrub starts 2024-09-12T02:58:01.325 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:01 smithi177 ceph-mon[35305]: purged_snaps scrub ok 2024-09-12T02:58:01.325 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:01 smithi177 ceph-mon[35305]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:58:01.325 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:01 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:58:01.325 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:01 smithi177 ceph-mon[35305]: osd.6 [v2:172.21.15.116:6818/1121291186,v1:172.21.15.116:6819/1121291186] boot 2024-09-12T02:58:01.325 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:01 smithi177 ceph-mon[35305]: osdmap e40: 7 total, 7 up, 7 in 2024-09-12T02:58:01.325 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:01 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:58:01.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:01 smithi116 ceph-mon[31410]: purged_snaps scrub starts 2024-09-12T02:58:01.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:01 smithi116 ceph-mon[31410]: purged_snaps scrub ok 2024-09-12T02:58:01.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:01 smithi116 ceph-mon[31410]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:58:01.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:01 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:58:01.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:01 smithi116 ceph-mon[31410]: osd.6 [v2:172.21.15.116:6818/1121291186,v1:172.21.15.116:6819/1121291186] boot 2024-09-12T02:58:01.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:01 smithi116 ceph-mon[31410]: osdmap e40: 7 total, 7 up, 7 in 2024-09-12T02:58:01.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:01 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:58:01.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:01 smithi183 ceph-mon[35535]: purged_snaps scrub starts 2024-09-12T02:58:01.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:01 smithi183 ceph-mon[35535]: purged_snaps scrub ok 2024-09-12T02:58:01.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:01 smithi183 ceph-mon[35535]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:58:01.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:01 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:58:01.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:01 smithi183 ceph-mon[35535]: osd.6 [v2:172.21.15.116:6818/1121291186,v1:172.21.15.116:6819/1121291186] boot 2024-09-12T02:58:01.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:01 smithi183 ceph-mon[35535]: osdmap e40: 7 total, 7 up, 7 in 2024-09-12T02:58:01.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:01 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:58:02.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:02 smithi177 ceph-mon[35305]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:58:02.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:02 smithi177 ceph-mon[35305]: osdmap e41: 7 total, 7 up, 7 in 2024-09-12T02:58:02.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:02 smithi177 ceph-mon[35305]: from='client.? 172.21.15.177:0/1896195103' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "761f4680-f049-473c-bf44-b19acf8de24f"}]: dispatch 2024-09-12T02:58:02.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:02 smithi177 ceph-mon[35305]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "761f4680-f049-473c-bf44-b19acf8de24f"}]: dispatch 2024-09-12T02:58:02.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:02 smithi177 ceph-mon[35305]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "761f4680-f049-473c-bf44-b19acf8de24f"}]': finished 2024-09-12T02:58:02.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:02 smithi177 ceph-mon[35305]: osdmap e42: 8 total, 7 up, 8 in 2024-09-12T02:58:02.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:02 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:58:02.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:02 smithi116 ceph-mon[31410]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:58:02.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:02 smithi116 ceph-mon[31410]: osdmap e41: 7 total, 7 up, 7 in 2024-09-12T02:58:02.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:02 smithi116 ceph-mon[31410]: from='client.? 172.21.15.177:0/1896195103' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "761f4680-f049-473c-bf44-b19acf8de24f"}]: dispatch 2024-09-12T02:58:02.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:02 smithi116 ceph-mon[31410]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "761f4680-f049-473c-bf44-b19acf8de24f"}]: dispatch 2024-09-12T02:58:02.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:02 smithi116 ceph-mon[31410]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "761f4680-f049-473c-bf44-b19acf8de24f"}]': finished 2024-09-12T02:58:02.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:02 smithi116 ceph-mon[31410]: osdmap e42: 8 total, 7 up, 8 in 2024-09-12T02:58:02.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:02 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:58:02.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:02 smithi183 ceph-mon[35535]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:58:02.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:02 smithi183 ceph-mon[35535]: osdmap e41: 7 total, 7 up, 7 in 2024-09-12T02:58:02.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:02 smithi183 ceph-mon[35535]: from='client.? 172.21.15.177:0/1896195103' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "761f4680-f049-473c-bf44-b19acf8de24f"}]: dispatch 2024-09-12T02:58:02.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:02 smithi183 ceph-mon[35535]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "761f4680-f049-473c-bf44-b19acf8de24f"}]: dispatch 2024-09-12T02:58:02.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:02 smithi183 ceph-mon[35535]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "761f4680-f049-473c-bf44-b19acf8de24f"}]': finished 2024-09-12T02:58:02.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:02 smithi183 ceph-mon[35535]: osdmap e42: 8 total, 7 up, 8 in 2024-09-12T02:58:02.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:02 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:58:03.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:03 smithi177 ceph-mon[35305]: from='client.? 172.21.15.177:0/4196143268' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:58:03.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:03 smithi116 ceph-mon[31410]: from='client.? 172.21.15.177:0/4196143268' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:58:03.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:03 smithi183 ceph-mon[35535]: from='client.? 172.21.15.177:0/4196143268' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:58:04.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:04 smithi177 ceph-mon[35305]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:04.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:04 smithi116 ceph-mon[31410]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:04.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:04 smithi183 ceph-mon[35535]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:06.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:06 smithi177 ceph-mon[35305]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:06.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:06 smithi116 ceph-mon[31410]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:06.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:06 smithi183 ceph-mon[35535]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:08.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:08 smithi177 ceph-mon[35305]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:08.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:08 smithi116 ceph-mon[31410]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:08 smithi183 ceph-mon[35535]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:09.441 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:09 smithi177 ceph-mon[35305]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:09.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:09 smithi116 ceph-mon[31410]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:09.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:09 smithi183 ceph-mon[35535]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:10.547 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:10 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T02:58:10.547 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:10 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:10.547 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:10 smithi177 ceph-mon[35305]: Deploying daemon osd.7 on smithi177 2024-09-12T02:58:10.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:10 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T02:58:10.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:10 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:10.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:10 smithi116 ceph-mon[31410]: Deploying daemon osd.7 on smithi177 2024-09-12T02:58:10.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:10 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T02:58:10.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:10 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:10.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:10 smithi183 ceph-mon[35535]: Deploying daemon osd.7 on smithi177 2024-09-12T02:58:11.488 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:11 smithi177 ceph-mon[35305]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:11.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:11 smithi116 ceph-mon[31410]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:11.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:11 smithi183 ceph-mon[35535]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:14.406 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:14 smithi183 ceph-mon[35535]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:14.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:14 smithi177 ceph-mon[35305]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:14.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:14 smithi116 ceph-mon[31410]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:16.451 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:16 smithi177 ceph-mon[35305]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:16.451 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:16 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:16.452 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:16 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:16.452 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:16 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:16.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:16 smithi183 ceph-mon[35535]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:16.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:16 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:16.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:16 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:16.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:16 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:16.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:16 smithi116 ceph-mon[31410]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:16.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:16 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:16.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:16 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:16.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:16 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:18.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:18 smithi183 ceph-mon[35535]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:18.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:18 smithi177 ceph-mon[35305]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:18.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:18 smithi116 ceph-mon[31410]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:19.894 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:58:19.893+0000 7f43327fc640 1 -- 172.21.15.177:0/2528313629 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f433c062630 con 0x7f43280a2bb0 2024-09-12T02:58:19.895 INFO:teuthology.orchestra.run.smithi177.stdout:Created osd(s) 7 on host 'smithi177' 2024-09-12T02:58:19.896 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:58:19.895+0000 7f43439d1640 1 -- 172.21.15.177:0/2528313629 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f43280a2bb0 msgr2=0x7f43280a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:19.896 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:58:19.895+0000 7f43439d1640 1 --2- 172.21.15.177:0/2528313629 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f43280a2bb0 0x7f43280a5070 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f433c102c50 tx=0x7f4338006c90 comp rx=0 tx=0).stop 2024-09-12T02:58:19.896 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:58:19.895+0000 7f43439d1640 1 -- 172.21.15.177:0/2528313629 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f433c102df0 msgr2=0x7f433c198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:19.896 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:58:19.895+0000 7f43439d1640 1 --2- 172.21.15.177:0/2528313629 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f433c102df0 0x7f433c198480 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f432c009230 tx=0x7f432c004360 comp rx=0 tx=0).stop 2024-09-12T02:58:19.896 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:58:19.895+0000 7f43439d1640 1 -- 172.21.15.177:0/2528313629 shutdown_connections 2024-09-12T02:58:19.896 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:58:19.895+0000 7f43439d1640 1 --2- 172.21.15.177:0/2528313629 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f43280a2bb0 0x7f43280a5070 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:19.896 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:58:19.895+0000 7f43439d1640 1 --2- 172.21.15.177:0/2528313629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f433c1037b0 0x7f433c19f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:19.896 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:58:19.895+0000 7f43439d1640 1 --2- 172.21.15.177:0/2528313629 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f433c102df0 0x7f433c198480 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:19.896 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:58:19.895+0000 7f43439d1640 1 --2- 172.21.15.177:0/2528313629 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f433c101bf0 0x7f433c197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:19.896 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:58:19.895+0000 7f43439d1640 1 -- 172.21.15.177:0/2528313629 >> 172.21.15.177:0/2528313629 conn(0x7f433c0fd3c0 msgr2=0x7f433c106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:19.897 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:58:19.896+0000 7f43439d1640 1 -- 172.21.15.177:0/2528313629 shutdown_connections 2024-09-12T02:58:19.897 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:58:19.896+0000 7f43439d1640 1 -- 172.21.15.177:0/2528313629 wait complete. 2024-09-12T02:58:20.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:19 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:20.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:19 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:20.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:19 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:20.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:19 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:58:20.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:19 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:20.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:19 smithi183 ceph-mon[35535]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:20.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:19 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:20.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:19 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:20.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:19 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:20.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:19 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:20.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:19 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:20.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:19 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:20.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:19 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:58:20.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:19 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:20.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:19 smithi177 ceph-mon[35305]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:20.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:19 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:20.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:19 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:20.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:19 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:20.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:19 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:20.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:19 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:20.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:19 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:20.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:19 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:58:20.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:19 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:20.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:19 smithi116 ceph-mon[31410]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:20.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:19 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:20.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:19 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:20.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:19 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:20.594 DEBUG:teuthology.orchestra.run.smithi177:osd.7> sudo journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.7.service 2024-09-12T02:58:20.596 INFO:tasks.cephadm:Deploying osd.8 on smithi183 with /dev/vg_nvme/lv_2... 2024-09-12T02:58:20.596 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T02:58:20.851 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:58:22.421 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:22 smithi183 ceph-mon[35535]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:22.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:22 smithi183 ceph-mon[35535]: from='osd.7 [v2:172.21.15.177:6816/2177665961,v1:172.21.15.177:6817/2177665961]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T02:58:22.422 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:22 smithi183 ceph-mon[35535]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T02:58:22.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:22 smithi177 ceph-mon[35305]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:22.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:22 smithi177 ceph-mon[35305]: from='osd.7 [v2:172.21.15.177:6816/2177665961,v1:172.21.15.177:6817/2177665961]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T02:58:22.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:22 smithi177 ceph-mon[35305]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T02:58:22.496 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 02:58:22 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T02:58:22.048+0000 7f6070295640 -1 osd.7 0 log_to_monitors true 2024-09-12T02:58:22.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:22 smithi116 ceph-mon[31410]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:22.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:22 smithi116 ceph-mon[31410]: from='osd.7 [v2:172.21.15.177:6816/2177665961,v1:172.21.15.177:6817/2177665961]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T02:58:22.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:22 smithi116 ceph-mon[31410]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T02:58:23.433 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 02:58:23 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T02:58:23.136+0000 7f606c218640 -1 osd.7 0 waiting for initial osdmap 2024-09-12T02:58:23.434 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 02:58:23 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T02:58:23.147+0000 7f6068040640 -1 osd.7 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:58:23.434 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:23 smithi177 ceph-mon[35305]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T02:58:23.434 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:23 smithi177 ceph-mon[35305]: from='osd.7 [v2:172.21.15.177:6816/2177665961,v1:172.21.15.177:6817/2177665961]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-09-12T02:58:23.434 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:23 smithi177 ceph-mon[35305]: osdmap e43: 8 total, 7 up, 8 in 2024-09-12T02:58:23.434 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:23 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:58:23.434 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:23 smithi177 ceph-mon[35305]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-09-12T02:58:23.434 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:23 smithi177 ceph-mon[35305]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-09-12T02:58:23.434 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:23 smithi177 ceph-mon[35305]: osdmap e44: 8 total, 7 up, 8 in 2024-09-12T02:58:23.434 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:23 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:58:23.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:23 smithi183 ceph-mon[35535]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T02:58:23.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:23 smithi183 ceph-mon[35535]: from='osd.7 [v2:172.21.15.177:6816/2177665961,v1:172.21.15.177:6817/2177665961]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-09-12T02:58:23.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:23 smithi183 ceph-mon[35535]: osdmap e43: 8 total, 7 up, 8 in 2024-09-12T02:58:23.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:23 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:58:23.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:23 smithi183 ceph-mon[35535]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-09-12T02:58:23.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:23 smithi183 ceph-mon[35535]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-09-12T02:58:23.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:23 smithi183 ceph-mon[35535]: osdmap e44: 8 total, 7 up, 8 in 2024-09-12T02:58:23.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:23 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:58:23.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:23 smithi116 ceph-mon[31410]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T02:58:23.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:23 smithi116 ceph-mon[31410]: from='osd.7 [v2:172.21.15.177:6816/2177665961,v1:172.21.15.177:6817/2177665961]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-09-12T02:58:23.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:23 smithi116 ceph-mon[31410]: osdmap e43: 8 total, 7 up, 8 in 2024-09-12T02:58:23.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:23 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:58:23.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:23 smithi116 ceph-mon[31410]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-09-12T02:58:23.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:23 smithi116 ceph-mon[31410]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-09-12T02:58:23.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:23 smithi116 ceph-mon[31410]: osdmap e44: 8 total, 7 up, 8 in 2024-09-12T02:58:23.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:23 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:58:23.597 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T02:58:23.628 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_2 2024-09-12T02:58:23.877 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:58:24.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.341+0000 7f201d8b4640 1 -- 172.21.15.183:0/926553450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2018101cd0 msgr2=0x7f2018102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:24.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.341+0000 7f201d8b4640 1 --2- 172.21.15.183:0/926553450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2018101cd0 0x7f2018102150 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2000009900 tx=0x7f200002a410 comp rx=0 tx=0).stop 2024-09-12T02:58:24.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.341+0000 7f201d8b4640 1 -- 172.21.15.183:0/926553450 shutdown_connections 2024-09-12T02:58:24.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.341+0000 7f201d8b4640 1 --2- 172.21.15.183:0/926553450 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2018102690 0x7f2018108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:24.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.341+0000 7f201d8b4640 1 --2- 172.21.15.183:0/926553450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2018101cd0 0x7f2018102150 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:24.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.341+0000 7f201d8b4640 1 --2- 172.21.15.183:0/926553450 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2018100ad0 0x7f2018100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:24.357 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.341+0000 7f201d8b4640 1 -- 172.21.15.183:0/926553450 >> 172.21.15.183:0/926553450 conn(0x7f20180fc280 msgr2=0x7f20180fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:24.357 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.341+0000 7f201d8b4640 1 -- 172.21.15.183:0/926553450 shutdown_connections 2024-09-12T02:58:24.357 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.341+0000 7f201d8b4640 1 -- 172.21.15.183:0/926553450 wait complete. 2024-09-12T02:58:24.357 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.342+0000 7f201d8b4640 1 Processor -- start 2024-09-12T02:58:24.357 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.342+0000 7f201d8b4640 1 -- start start 2024-09-12T02:58:24.357 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.342+0000 7f201d8b4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2018100ad0 0x7f2018197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:24.357 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.342+0000 7f201d8b4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2018101cd0 0x7f2018198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:24.357 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.342+0000 7f201d8b4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2018102690 0x7f201819f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:24.357 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.342+0000 7f201d8b4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f201810d880 con 0x7f2018102690 2024-09-12T02:58:24.357 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.342+0000 7f201d8b4640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f201810d700 con 0x7f2018100ad0 2024-09-12T02:58:24.357 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.342+0000 7f201d8b4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f201810da00 con 0x7f2018101cd0 2024-09-12T02:58:24.357 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.342+0000 7f20167fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2018101cd0 0x7f2018198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:24.357 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.342+0000 7f20167fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2018101cd0 0x7f2018198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:32802/0 (socket says 172.21.15.183:32802) 2024-09-12T02:58:24.358 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.342+0000 7f20167fc640 1 -- 172.21.15.183:0/3306470062 learned_addr learned my addr 172.21.15.183:0/3306470062 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-12T02:58:24.358 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.342+0000 7f2016ffd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2018100ad0 0x7f2018197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:24.358 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.343+0000 7f20177fe640 1 --2- 172.21.15.183:0/3306470062 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2018102690 0x7f201819f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:24.358 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.343+0000 7f20167fc640 1 -- 172.21.15.183:0/3306470062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2018100ad0 msgr2=0x7f2018197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:24.358 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.343+0000 7f20167fc640 1 --2- 172.21.15.183:0/3306470062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2018100ad0 0x7f2018197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:24.358 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.343+0000 7f20167fc640 1 -- 172.21.15.183:0/3306470062 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2018102690 msgr2=0x7f201819f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:24.358 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.343+0000 7f20167fc640 1 --2- 172.21.15.183:0/3306470062 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2018102690 0x7f201819f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:24.358 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.343+0000 7f20167fc640 1 -- 172.21.15.183:0/3306470062 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f201819fa60 con 0x7f2018101cd0 2024-09-12T02:58:24.358 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.343+0000 7f20167fc640 1 --2- 172.21.15.183:0/3306470062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2018101cd0 0x7f2018198370 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2000009230 tx=0x7f2000004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:24.358 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.343+0000 7f201c8b2640 1 -- 172.21.15.183:0/3306470062 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200003f030 con 0x7f2018101cd0 2024-09-12T02:58:24.358 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.343+0000 7f201c8b2640 1 -- 172.21.15.183:0/3306470062 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2000003ce0 con 0x7f2018101cd0 2024-09-12T02:58:24.358 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.343+0000 7f201d8b4640 1 -- 172.21.15.183:0/3306470062 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f201819fcf0 con 0x7f2018101cd0 2024-09-12T02:58:24.358 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.343+0000 7f201c8b2640 1 -- 172.21.15.183:0/3306470062 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000035070 con 0x7f2018101cd0 2024-09-12T02:58:24.359 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.343+0000 7f201d8b4640 1 -- 172.21.15.183:0/3306470062 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20181a0180 con 0x7f2018101cd0 2024-09-12T02:58:24.359 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.344+0000 7f201d8b4640 1 -- 172.21.15.183:0/3306470062 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2018107e20 con 0x7f2018101cd0 2024-09-12T02:58:24.359 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.345+0000 7f201c8b2640 1 -- 172.21.15.183:0/3306470062 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f2000042030 con 0x7f2018101cd0 2024-09-12T02:58:24.359 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.346+0000 7f201c8b2640 1 --2- 172.21.15.183:0/3306470062 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f20040a2bb0 0x7f20040a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:24.359 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.346+0000 7f201c8b2640 1 -- 172.21.15.183:0/3306470062 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5430+0+0 (secure 0 0 0) 0x7f20000eeb10 con 0x7f2018101cd0 2024-09-12T02:58:24.359 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.346+0000 7f2016ffd640 1 --2- 172.21.15.183:0/3306470062 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f20040a2bb0 0x7f20040a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:24.359 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.347+0000 7f2016ffd640 1 --2- 172.21.15.183:0/3306470062 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f20040a2bb0 0x7f20040a5070 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2018101b30 tx=0x7f2010007600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:24.359 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.350+0000 7f201c8b2640 1 -- 172.21.15.183:0/3306470062 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f20000bc5b0 con 0x7f2018101cd0 2024-09-12T02:58:24.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:24 smithi183 ceph-mon[35535]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:24.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:24 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:58:24.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:24 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:24.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:24 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:24.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:24 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:58:24.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:24 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:58:24.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:24 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:58:24.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:24 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:24.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:24 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:24.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:24 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:58:24.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:24 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:24.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:24 smithi183 ceph-mon[35535]: osd.7 [v2:172.21.15.177:6816/2177665961,v1:172.21.15.177:6817/2177665961] boot 2024-09-12T02:58:24.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:24 smithi183 ceph-mon[35535]: osdmap e45: 8 total, 8 up, 8 in 2024-09-12T02:58:24.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:24 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:58:24.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:24 smithi177 ceph-mon[35305]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:24.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:24 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:58:24.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:24 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:24.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:24 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:24.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:24 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:58:24.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:24 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:58:24.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:24 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:58:24.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:24 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:24.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:24 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:24.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:24 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:58:24.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:24 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:24.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:24 smithi177 ceph-mon[35305]: osd.7 [v2:172.21.15.177:6816/2177665961,v1:172.21.15.177:6817/2177665961] boot 2024-09-12T02:58:24.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:24 smithi177 ceph-mon[35305]: osdmap e45: 8 total, 8 up, 8 in 2024-09-12T02:58:24.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:24 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:58:24.532 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:24.532+0000 7f201d8b4640 1 -- 172.21.15.183:0/3306470062 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f2018062630 con 0x7f20040a2bb0 2024-09-12T02:58:24.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:24 smithi116 ceph-mon[31410]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:58:24.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:24 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:58:24.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:24 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:24.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:24 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:24.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:24 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:58:24.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:24 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:58:24.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:24 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:58:24.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:24 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:24.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:24 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:24.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:24 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:58:24.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:24 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:24.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:24 smithi116 ceph-mon[31410]: osd.7 [v2:172.21.15.177:6816/2177665961,v1:172.21.15.177:6817/2177665961] boot 2024-09-12T02:58:24.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:24 smithi116 ceph-mon[31410]: osdmap e45: 8 total, 8 up, 8 in 2024-09-12T02:58:24.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:24 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:58:25.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:25 smithi183 ceph-mon[35535]: purged_snaps scrub starts 2024-09-12T02:58:25.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:25 smithi183 ceph-mon[35535]: purged_snaps scrub ok 2024-09-12T02:58:25.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:25 smithi183 ceph-mon[35535]: Detected new or changed devices on smithi177 2024-09-12T02:58:25.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:25 smithi183 ceph-mon[35535]: Adjusting osd_memory_target on smithi177 to 5695M 2024-09-12T02:58:25.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:25 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:58:25.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:25 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:58:25.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:25 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:25.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:25 smithi183 ceph-mon[35535]: osdmap e46: 8 total, 8 up, 8 in 2024-09-12T02:58:25.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:25 smithi177 ceph-mon[35305]: purged_snaps scrub starts 2024-09-12T02:58:25.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:25 smithi177 ceph-mon[35305]: purged_snaps scrub ok 2024-09-12T02:58:25.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:25 smithi177 ceph-mon[35305]: Detected new or changed devices on smithi177 2024-09-12T02:58:25.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:25 smithi177 ceph-mon[35305]: Adjusting osd_memory_target on smithi177 to 5695M 2024-09-12T02:58:25.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:25 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:58:25.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:25 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:58:25.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:25 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:25.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:25 smithi177 ceph-mon[35305]: osdmap e46: 8 total, 8 up, 8 in 2024-09-12T02:58:25.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:25 smithi116 ceph-mon[31410]: purged_snaps scrub starts 2024-09-12T02:58:25.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:25 smithi116 ceph-mon[31410]: purged_snaps scrub ok 2024-09-12T02:58:25.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:25 smithi116 ceph-mon[31410]: Detected new or changed devices on smithi177 2024-09-12T02:58:25.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:25 smithi116 ceph-mon[31410]: Adjusting osd_memory_target on smithi177 to 5695M 2024-09-12T02:58:25.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:25 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:58:25.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:25 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:58:25.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:25 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:25.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:25 smithi116 ceph-mon[31410]: osdmap e46: 8 total, 8 up, 8 in 2024-09-12T02:58:26.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:26 smithi183 ceph-mon[35535]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:58:26.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:26 smithi183 ceph-mon[35535]: pgmap v166: 1 pgs: 1 remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:26.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:26 smithi183 ceph-mon[35535]: osdmap e47: 8 total, 8 up, 8 in 2024-09-12T02:58:26.453 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:26 smithi116 ceph-mon[31410]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:58:26.453 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:26 smithi116 ceph-mon[31410]: pgmap v166: 1 pgs: 1 remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:26.453 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:26 smithi116 ceph-mon[31410]: osdmap e47: 8 total, 8 up, 8 in 2024-09-12T02:58:26.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:26 smithi177 ceph-mon[35305]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:58:26.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:26 smithi177 ceph-mon[35305]: pgmap v166: 1 pgs: 1 remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:26.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:26 smithi177 ceph-mon[35305]: osdmap e47: 8 total, 8 up, 8 in 2024-09-12T02:58:27.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:27 smithi183 ceph-mon[35535]: from='client.? 172.21.15.183:0/1806575987' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2385ea6c-fd5a-4815-9596-77f948629542"}]: dispatch 2024-09-12T02:58:27.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:27 smithi183 ceph-mon[35535]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2385ea6c-fd5a-4815-9596-77f948629542"}]: dispatch 2024-09-12T02:58:27.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:27 smithi183 ceph-mon[35535]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2385ea6c-fd5a-4815-9596-77f948629542"}]': finished 2024-09-12T02:58:27.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:27 smithi183 ceph-mon[35535]: osdmap e48: 9 total, 8 up, 9 in 2024-09-12T02:58:27.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:27 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:58:27.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:27 smithi177 ceph-mon[35305]: from='client.? 172.21.15.183:0/1806575987' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2385ea6c-fd5a-4815-9596-77f948629542"}]: dispatch 2024-09-12T02:58:27.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:27 smithi177 ceph-mon[35305]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2385ea6c-fd5a-4815-9596-77f948629542"}]: dispatch 2024-09-12T02:58:27.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:27 smithi177 ceph-mon[35305]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2385ea6c-fd5a-4815-9596-77f948629542"}]': finished 2024-09-12T02:58:27.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:27 smithi177 ceph-mon[35305]: osdmap e48: 9 total, 8 up, 9 in 2024-09-12T02:58:27.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:27 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:58:27.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:27 smithi116 ceph-mon[31410]: from='client.? 172.21.15.183:0/1806575987' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2385ea6c-fd5a-4815-9596-77f948629542"}]: dispatch 2024-09-12T02:58:27.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:27 smithi116 ceph-mon[31410]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2385ea6c-fd5a-4815-9596-77f948629542"}]: dispatch 2024-09-12T02:58:27.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:27 smithi116 ceph-mon[31410]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2385ea6c-fd5a-4815-9596-77f948629542"}]': finished 2024-09-12T02:58:27.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:27 smithi116 ceph-mon[31410]: osdmap e48: 9 total, 8 up, 9 in 2024-09-12T02:58:27.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:27 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:58:28.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:28 smithi183 ceph-mon[35535]: pgmap v170: 1 pgs: 1 remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:28.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:28 smithi183 ceph-mon[35535]: from='client.? 172.21.15.183:0/64525300' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:58:28.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:28 smithi183 ceph-mon[35535]: osdmap e49: 9 total, 8 up, 9 in 2024-09-12T02:58:28.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:28 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:58:28.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:28 smithi177 ceph-mon[35305]: pgmap v170: 1 pgs: 1 remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:28.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:28 smithi177 ceph-mon[35305]: from='client.? 172.21.15.183:0/64525300' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:58:28.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:28 smithi177 ceph-mon[35305]: osdmap e49: 9 total, 8 up, 9 in 2024-09-12T02:58:28.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:28 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:58:28.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:28 smithi116 ceph-mon[31410]: pgmap v170: 1 pgs: 1 remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:28.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:28 smithi116 ceph-mon[31410]: from='client.? 172.21.15.183:0/64525300' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:58:28.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:28 smithi116 ceph-mon[31410]: osdmap e49: 9 total, 8 up, 9 in 2024-09-12T02:58:28.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:28 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:58:30.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:30 smithi183 ceph-mon[35535]: pgmap v172: 1 pgs: 1 remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:30.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:30 smithi177 ceph-mon[35305]: pgmap v172: 1 pgs: 1 remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:30.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:30 smithi116 ceph-mon[31410]: pgmap v172: 1 pgs: 1 remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:32.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:32 smithi183 ceph-mon[35535]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:32.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:32 smithi177 ceph-mon[35305]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:32.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:32 smithi116 ceph-mon[31410]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:34.340 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:34 smithi183 ceph-mon[35535]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:34.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:34 smithi177 ceph-mon[35305]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:34.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:34 smithi116 ceph-mon[31410]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:35.451 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:35 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T02:58:35.451 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:35 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:35.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:35 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T02:58:35.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:35 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:35.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:35 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T02:58:35.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:35 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:36.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:36 smithi183 ceph-mon[35535]: Deploying daemon osd.8 on smithi183 2024-09-12T02:58:36.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:36 smithi183 ceph-mon[35535]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:36.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:36 smithi177 ceph-mon[35305]: Deploying daemon osd.8 on smithi183 2024-09-12T02:58:36.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:36 smithi177 ceph-mon[35305]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:36.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:36 smithi116 ceph-mon[31410]: Deploying daemon osd.8 on smithi183 2024-09-12T02:58:36.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:36 smithi116 ceph-mon[31410]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:38.427 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:38 smithi183 ceph-mon[35535]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:38.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:38 smithi177 ceph-mon[35305]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:38.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:38 smithi116 ceph-mon[31410]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:40.355 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:40 smithi183 ceph-mon[35535]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:40.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:40 smithi177 ceph-mon[35305]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:40.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:40 smithi116 ceph-mon[31410]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:41.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:41.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:41.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:41.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:41 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:41.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:41 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:41.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:41 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:41.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:41.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:41.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:42.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:42 smithi183 ceph-mon[35535]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:42.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:42 smithi177 ceph-mon[35305]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:42.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:42 smithi116 ceph-mon[31410]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:44.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:44 smithi183 ceph-mon[35535]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:44.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:44 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:44.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:44 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:44.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:44 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:44.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:44 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:58:44.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:44 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:44.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:44 smithi177 ceph-mon[35305]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:44.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:44 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:44.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:44 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:44.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:44 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:44.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:44 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:58:44.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:44 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:44.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:44 smithi116 ceph-mon[31410]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:44.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:44 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:44.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:44 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:44.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:44 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:44.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:44 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:58:44.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:44 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:44.897 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:44.896+0000 7f201c8b2640 1 -- 172.21.15.183:0/3306470062 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2018062630 con 0x7f20040a2bb0 2024-09-12T02:58:44.898 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 8 on host 'smithi183' 2024-09-12T02:58:44.899 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:44.899+0000 7f201d8b4640 1 -- 172.21.15.183:0/3306470062 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f20040a2bb0 msgr2=0x7f20040a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:44.900 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:44.899+0000 7f201d8b4640 1 --2- 172.21.15.183:0/3306470062 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f20040a2bb0 0x7f20040a5070 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2018101b30 tx=0x7f2010007600 comp rx=0 tx=0).stop 2024-09-12T02:58:44.900 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:44.899+0000 7f201d8b4640 1 -- 172.21.15.183:0/3306470062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2018101cd0 msgr2=0x7f2018198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:44.900 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:44.899+0000 7f201d8b4640 1 --2- 172.21.15.183:0/3306470062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2018101cd0 0x7f2018198370 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2000009230 tx=0x7f2000004360 comp rx=0 tx=0).stop 2024-09-12T02:58:44.900 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:44.899+0000 7f201d8b4640 1 -- 172.21.15.183:0/3306470062 shutdown_connections 2024-09-12T02:58:44.900 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:44.899+0000 7f201d8b4640 1 --2- 172.21.15.183:0/3306470062 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f20040a2bb0 0x7f20040a5070 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:44.900 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:44.899+0000 7f201d8b4640 1 --2- 172.21.15.183:0/3306470062 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2018102690 0x7f201819f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:44.900 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:44.899+0000 7f201d8b4640 1 --2- 172.21.15.183:0/3306470062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2018101cd0 0x7f2018198370 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:44.900 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:44.899+0000 7f201d8b4640 1 --2- 172.21.15.183:0/3306470062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2018100ad0 0x7f2018197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:44.900 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:44.899+0000 7f201d8b4640 1 -- 172.21.15.183:0/3306470062 >> 172.21.15.183:0/3306470062 conn(0x7f20180fc280 msgr2=0x7f2018104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:44.900 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:44.899+0000 7f201d8b4640 1 -- 172.21.15.183:0/3306470062 shutdown_connections 2024-09-12T02:58:44.900 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:58:44.899+0000 7f201d8b4640 1 -- 172.21.15.183:0/3306470062 wait complete. 2024-09-12T02:58:45.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:45 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:45.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:45 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:45.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:45 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:45.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:45.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:45.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:45.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:45 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:45.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:45 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:45.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:45 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:45.607 DEBUG:teuthology.orchestra.run.smithi183:osd.8> sudo journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.8.service 2024-09-12T02:58:45.610 INFO:tasks.cephadm:Deploying osd.9 on smithi116 with /dev/vg_nvme/lv_1... 2024-09-12T02:58:45.610 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T02:58:45.867 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:58:46.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:46 smithi116 ceph-mon[31410]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:46.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:46 smithi183 ceph-mon[35535]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:46.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:46 smithi177 ceph-mon[35305]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:46.850 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:58:46 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T02:58:46.797+0000 7f90096eb640 -1 osd.8 0 log_to_monitors true 2024-09-12T02:58:47.256 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:47 smithi116 ceph-mon[31410]: from='osd.8 [v2:172.21.15.183:6816/1305725258,v1:172.21.15.183:6817/1305725258]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T02:58:47.256 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:47 smithi116 ceph-mon[31410]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T02:58:47.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:47 smithi183 ceph-mon[35535]: from='osd.8 [v2:172.21.15.183:6816/1305725258,v1:172.21.15.183:6817/1305725258]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T02:58:47.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:47 smithi183 ceph-mon[35535]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T02:58:47.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:47 smithi177 ceph-mon[35305]: from='osd.8 [v2:172.21.15.183:6816/1305725258,v1:172.21.15.183:6817/1305725258]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T02:58:47.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:47 smithi177 ceph-mon[35305]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T02:58:48.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:48 smithi177 ceph-mon[35305]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:48.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:48 smithi177 ceph-mon[35305]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T02:58:48.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:48 smithi177 ceph-mon[35305]: osdmap e50: 9 total, 8 up, 9 in 2024-09-12T02:58:48.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:48 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:58:48.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:48 smithi177 ceph-mon[35305]: from='osd.8 [v2:172.21.15.183:6816/1305725258,v1:172.21.15.183:6817/1305725258]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T02:58:48.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:48 smithi177 ceph-mon[35305]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T02:58:48.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:48 smithi116 ceph-mon[31410]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:48.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:48 smithi116 ceph-mon[31410]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T02:58:48.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:48 smithi116 ceph-mon[31410]: osdmap e50: 9 total, 8 up, 9 in 2024-09-12T02:58:48.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:48 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:58:48.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:48 smithi116 ceph-mon[31410]: from='osd.8 [v2:172.21.15.183:6816/1305725258,v1:172.21.15.183:6817/1305725258]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T02:58:48.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:48 smithi116 ceph-mon[31410]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T02:58:48.576 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:48 smithi183 ceph-mon[35535]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:48.576 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:48 smithi183 ceph-mon[35535]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T02:58:48.576 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:48 smithi183 ceph-mon[35535]: osdmap e50: 9 total, 8 up, 9 in 2024-09-12T02:58:48.576 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:48 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:58:48.576 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:48 smithi183 ceph-mon[35535]: from='osd.8 [v2:172.21.15.183:6816/1305725258,v1:172.21.15.183:6817/1305725258]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T02:58:48.577 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:48 smithi183 ceph-mon[35535]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T02:58:48.589 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T02:58:48.615 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi116:vg_nvme/lv_1 2024-09-12T02:58:48.892 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:58:48.953 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:58:48 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T02:58:48.608+0000 7f900566e640 -1 osd.8 0 waiting for initial osdmap 2024-09-12T02:58:48.953 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 02:58:48 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T02:58:48.617+0000 7f8ffd48e640 -1 osd.8 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:58:49.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.407+0000 7fd3166c2640 1 -- 172.21.15.116:0/2020151090 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd310100ad0 msgr2=0x7fd310100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:49.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.407+0000 7fd3166c2640 1 --2- 172.21.15.116:0/2020151090 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd310100ad0 0x7fd310100ed0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd300009900 tx=0x7fd30002a410 comp rx=0 tx=0).stop 2024-09-12T02:58:49.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.407+0000 7fd3166c2640 1 -- 172.21.15.116:0/2020151090 shutdown_connections 2024-09-12T02:58:49.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.407+0000 7fd3166c2640 1 --2- 172.21.15.116:0/2020151090 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd310102690 0x7fd310108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:49.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.407+0000 7fd3166c2640 1 --2- 172.21.15.116:0/2020151090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd310101cd0 0x7fd310102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:49.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.407+0000 7fd3166c2640 1 --2- 172.21.15.116:0/2020151090 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd310100ad0 0x7fd310100ed0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:49.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.407+0000 7fd3166c2640 1 -- 172.21.15.116:0/2020151090 >> 172.21.15.116:0/2020151090 conn(0x7fd3100fc280 msgr2=0x7fd3100fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:49.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.407+0000 7fd3166c2640 1 -- 172.21.15.116:0/2020151090 shutdown_connections 2024-09-12T02:58:49.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.407+0000 7fd3166c2640 1 -- 172.21.15.116:0/2020151090 wait complete. 2024-09-12T02:58:49.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.407+0000 7fd3166c2640 1 Processor -- start 2024-09-12T02:58:49.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.408+0000 7fd3166c2640 1 -- start start 2024-09-12T02:58:49.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.408+0000 7fd3166c2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd310100ad0 0x7fd310104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:49.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.408+0000 7fd3166c2640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd310101cd0 0x7fd3101030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:49.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.408+0000 7fd3166c2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd310102690 0x7fd3101035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:49.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.408+0000 7fd3166c2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd31010c7d0 con 0x7fd310100ad0 2024-09-12T02:58:49.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.408+0000 7fd3166c2640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd31010c650 con 0x7fd310101cd0 2024-09-12T02:58:49.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.408+0000 7fd3166c2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd31010c950 con 0x7fd310102690 2024-09-12T02:58:49.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.408+0000 7fd30ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd310100ad0 0x7fd310104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:49.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.408+0000 7fd30ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd310100ad0 0x7fd310104a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48464/0 (socket says 172.21.15.116:48464) 2024-09-12T02:58:49.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.408+0000 7fd30ffff640 1 -- 172.21.15.116:0/1473269022 learned_addr learned my addr 172.21.15.116:0/1473269022 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T02:58:49.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.408+0000 7fd314c5f640 1 --2- 172.21.15.116:0/1473269022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd310102690 0x7fd3101035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:49.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.408+0000 7fd30f7fe640 1 --2- 172.21.15.116:0/1473269022 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd310101cd0 0x7fd3101030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:49.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.408+0000 7fd30ffff640 1 -- 172.21.15.116:0/1473269022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd310102690 msgr2=0x7fd3101035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:49.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.408+0000 7fd30ffff640 1 --2- 172.21.15.116:0/1473269022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd310102690 0x7fd3101035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:49.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.408+0000 7fd30ffff640 1 -- 172.21.15.116:0/1473269022 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd310101cd0 msgr2=0x7fd3101030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:49.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.408+0000 7fd30ffff640 1 --2- 172.21.15.116:0/1473269022 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd310101cd0 0x7fd3101030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:49.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.408+0000 7fd30ffff640 1 -- 172.21.15.116:0/1473269022 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd310103eb0 con 0x7fd310100ad0 2024-09-12T02:58:49.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.409+0000 7fd30ffff640 1 --2- 172.21.15.116:0/1473269022 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd310100ad0 0x7fd310104a60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd300031610 tx=0x7fd3000043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:49.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.409+0000 7fd30d7fa640 1 -- 172.21.15.116:0/1473269022 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30003f030 con 0x7fd310100ad0 2024-09-12T02:58:49.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.409+0000 7fd3166c2640 1 -- 172.21.15.116:0/1473269022 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3101a20d0 con 0x7fd310100ad0 2024-09-12T02:58:49.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.409+0000 7fd30d7fa640 1 -- 172.21.15.116:0/1473269022 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd300003ce0 con 0x7fd310100ad0 2024-09-12T02:58:49.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.409+0000 7fd30d7fa640 1 -- 172.21.15.116:0/1473269022 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd300035070 con 0x7fd310100ad0 2024-09-12T02:58:49.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.409+0000 7fd3166c2640 1 -- 172.21.15.116:0/1473269022 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3101a2650 con 0x7fd310100ad0 2024-09-12T02:58:49.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.410+0000 7fd3166c2640 1 -- 172.21.15.116:0/1473269022 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd310100ed0 con 0x7fd310100ad0 2024-09-12T02:58:49.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.411+0000 7fd30d7fa640 1 -- 172.21.15.116:0/1473269022 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fd300042030 con 0x7fd310100ad0 2024-09-12T02:58:49.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.412+0000 7fd30d7fa640 1 --2- 172.21.15.116:0/1473269022 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd2f80a2b80 0x7fd2f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:49.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.412+0000 7fd30f7fe640 1 --2- 172.21.15.116:0/1473269022 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd2f80a2b80 0x7fd2f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:49.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.412+0000 7fd30d7fa640 1 -- 172.21.15.116:0/1473269022 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5862+0+0 (secure 0 0 0) 0x7fd3000eec70 con 0x7fd310100ad0 2024-09-12T02:58:49.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.412+0000 7fd30f7fe640 1 --2- 172.21.15.116:0/1473269022 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd2f80a2b80 0x7fd2f80a5040 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd304005fd0 tx=0x7fd304005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:49.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.416+0000 7fd30d7fa640 1 -- 172.21.15.116:0/1473269022 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd3000bc560 con 0x7fd310100ad0 2024-09-12T02:58:49.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:49 smithi177 ceph-mon[35305]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-12T02:58:49.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:49 smithi177 ceph-mon[35305]: osdmap e51: 9 total, 8 up, 9 in 2024-09-12T02:58:49.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:49 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:58:49.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:49 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:58:49.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:49 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:49.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:49 smithi177 ceph-mon[35305]: from='osd.8 ' entity='osd.8' 2024-09-12T02:58:49.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:49 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:49.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:49 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:58:49.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:49 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:58:49.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:49 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:58:49.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:49 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:49.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:49 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:49.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:49 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:58:49.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:49 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:49.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:49 smithi116 ceph-mon[31410]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-12T02:58:49.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:49 smithi116 ceph-mon[31410]: osdmap e51: 9 total, 8 up, 9 in 2024-09-12T02:58:49.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:49 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:58:49.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:49 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:58:49.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:49 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:49.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:49 smithi116 ceph-mon[31410]: from='osd.8 ' entity='osd.8' 2024-09-12T02:58:49.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:49 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:49.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:49 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:58:49.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:49 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:58:49.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:49 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:58:49.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:49 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:49.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:49 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:49.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:49 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:58:49.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:49 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:49.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:58:49.598+0000 7fd3166c2640 1 -- 172.21.15.116:0/1473269022 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd3101a2940 con 0x7fd2f80a2b80 2024-09-12T02:58:49.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:49 smithi183 ceph-mon[35535]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-12T02:58:49.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:49 smithi183 ceph-mon[35535]: osdmap e51: 9 total, 8 up, 9 in 2024-09-12T02:58:49.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:49 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:58:49.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:49 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:58:49.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:49 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:49.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:49 smithi183 ceph-mon[35535]: from='osd.8 ' entity='osd.8' 2024-09-12T02:58:49.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:49 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:49.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:49 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:58:49.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:49 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:58:49.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:49 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:58:49.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:49 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:49.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:49 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:49.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:49 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:58:49.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:49 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:58:50.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:50 smithi177 ceph-mon[35305]: purged_snaps scrub starts 2024-09-12T02:58:50.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:50 smithi177 ceph-mon[35305]: purged_snaps scrub ok 2024-09-12T02:58:50.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:50 smithi177 ceph-mon[35305]: Detected new or changed devices on smithi183 2024-09-12T02:58:50.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:50 smithi177 ceph-mon[35305]: Adjusting osd_memory_target on smithi183 to 5696M 2024-09-12T02:58:50.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:50 smithi177 ceph-mon[35305]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:50.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:50 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:58:50.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:50 smithi177 ceph-mon[35305]: osd.8 [v2:172.21.15.183:6816/1305725258,v1:172.21.15.183:6817/1305725258] boot 2024-09-12T02:58:50.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:50 smithi177 ceph-mon[35305]: osdmap e52: 9 total, 9 up, 9 in 2024-09-12T02:58:50.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:50 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:58:50.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:50 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:58:50.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:50 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:58:50.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:50 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:50.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:50 smithi116 ceph-mon[31410]: purged_snaps scrub starts 2024-09-12T02:58:50.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:50 smithi116 ceph-mon[31410]: purged_snaps scrub ok 2024-09-12T02:58:50.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:50 smithi116 ceph-mon[31410]: Detected new or changed devices on smithi183 2024-09-12T02:58:50.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:50 smithi116 ceph-mon[31410]: Adjusting osd_memory_target on smithi183 to 5696M 2024-09-12T02:58:50.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:50 smithi116 ceph-mon[31410]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:50.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:50 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:58:50.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:50 smithi116 ceph-mon[31410]: osd.8 [v2:172.21.15.183:6816/1305725258,v1:172.21.15.183:6817/1305725258] boot 2024-09-12T02:58:50.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:50 smithi116 ceph-mon[31410]: osdmap e52: 9 total, 9 up, 9 in 2024-09-12T02:58:50.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:50 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:58:50.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:50 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:58:50.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:50 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:58:50.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:50 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:50.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:50 smithi183 ceph-mon[35535]: purged_snaps scrub starts 2024-09-12T02:58:50.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:50 smithi183 ceph-mon[35535]: purged_snaps scrub ok 2024-09-12T02:58:50.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:50 smithi183 ceph-mon[35535]: Detected new or changed devices on smithi183 2024-09-12T02:58:50.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:50 smithi183 ceph-mon[35535]: Adjusting osd_memory_target on smithi183 to 5696M 2024-09-12T02:58:50.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:50 smithi183 ceph-mon[35535]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:58:50.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:50 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:58:50.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:50 smithi183 ceph-mon[35535]: osd.8 [v2:172.21.15.183:6816/1305725258,v1:172.21.15.183:6817/1305725258] boot 2024-09-12T02:58:50.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:50 smithi183 ceph-mon[35535]: osdmap e52: 9 total, 9 up, 9 in 2024-09-12T02:58:50.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:50 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:58:50.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:50 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:58:50.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:50 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:58:50.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:50 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:51.484 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:51 smithi116 ceph-mon[31410]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:58:51.485 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:51 smithi116 ceph-mon[31410]: osdmap e53: 9 total, 9 up, 9 in 2024-09-12T02:58:51.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:51 smithi177 ceph-mon[35305]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:58:51.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:51 smithi177 ceph-mon[35305]: osdmap e53: 9 total, 9 up, 9 in 2024-09-12T02:58:51.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:51 smithi183 ceph-mon[35535]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:58:51.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:51 smithi183 ceph-mon[35535]: osdmap e53: 9 total, 9 up, 9 in 2024-09-12T02:58:52.467 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:52 smithi116 ceph-mon[31410]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:58:52.468 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:52 smithi116 ceph-mon[31410]: osdmap e54: 9 total, 9 up, 9 in 2024-09-12T02:58:52.468 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:52 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/477969707' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd0c0dc4-4990-4f12-aa14-f855504cd995"}]: dispatch 2024-09-12T02:58:52.468 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:52 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/477969707' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd0c0dc4-4990-4f12-aa14-f855504cd995"}]': finished 2024-09-12T02:58:52.468 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:52 smithi116 ceph-mon[31410]: osdmap e55: 10 total, 9 up, 10 in 2024-09-12T02:58:52.468 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:52 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:58:52.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:52 smithi177 ceph-mon[35305]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:58:52.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:52 smithi177 ceph-mon[35305]: osdmap e54: 9 total, 9 up, 9 in 2024-09-12T02:58:52.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:52 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/477969707' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd0c0dc4-4990-4f12-aa14-f855504cd995"}]: dispatch 2024-09-12T02:58:52.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:52 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/477969707' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd0c0dc4-4990-4f12-aa14-f855504cd995"}]': finished 2024-09-12T02:58:52.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:52 smithi177 ceph-mon[35305]: osdmap e55: 10 total, 9 up, 10 in 2024-09-12T02:58:52.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:58:52.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:52 smithi183 ceph-mon[35535]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:58:52.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:52 smithi183 ceph-mon[35535]: osdmap e54: 9 total, 9 up, 9 in 2024-09-12T02:58:52.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:52 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/477969707' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd0c0dc4-4990-4f12-aa14-f855504cd995"}]: dispatch 2024-09-12T02:58:52.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:52 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/477969707' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd0c0dc4-4990-4f12-aa14-f855504cd995"}]': finished 2024-09-12T02:58:52.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:52 smithi183 ceph-mon[35535]: osdmap e55: 10 total, 9 up, 10 in 2024-09-12T02:58:52.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:52 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:58:53.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:53 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3447259067' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:58:53.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:53 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3447259067' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:58:53.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:53 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3447259067' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:58:54.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:54 smithi177 ceph-mon[35305]: pgmap v190: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:58:54.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:54 smithi177 ceph-mon[35305]: osdmap e56: 10 total, 9 up, 10 in 2024-09-12T02:58:54.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:54 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:58:54.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:54 smithi116 ceph-mon[31410]: pgmap v190: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:58:54.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:54 smithi116 ceph-mon[31410]: osdmap e56: 10 total, 9 up, 10 in 2024-09-12T02:58:54.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:54 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:58:54.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:54 smithi183 ceph-mon[35535]: pgmap v190: 1 pgs: 1 peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:58:54.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:54 smithi183 ceph-mon[35535]: osdmap e56: 10 total, 9 up, 10 in 2024-09-12T02:58:54.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:54 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:58:56.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:56 smithi177 ceph-mon[35305]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-09-12T02:58:56.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:56 smithi116 ceph-mon[31410]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-09-12T02:58:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:56 smithi183 ceph-mon[35535]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-09-12T02:58:58.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:58:58 smithi177 ceph-mon[35305]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T02:58:58.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:58:58 smithi116 ceph-mon[31410]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T02:58:58.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:58:58 smithi183 ceph-mon[35535]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T02:59:00.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:00 smithi177 ceph-mon[35305]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-12T02:59:00.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:00 smithi116 ceph-mon[31410]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-12T02:59:00.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:00 smithi183 ceph-mon[35535]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-12T02:59:01.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:01 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T02:59:01.560 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:01 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:01.560 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:01 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T02:59:01.561 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:01 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:01.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:01 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T02:59:01.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:01 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:02.471 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:02 smithi116 ceph-mon[31410]: Deploying daemon osd.9 on smithi116 2024-09-12T02:59:02.471 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:02 smithi116 ceph-mon[31410]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-12T02:59:02.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:02 smithi177 ceph-mon[35305]: Deploying daemon osd.9 on smithi116 2024-09-12T02:59:02.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:02 smithi177 ceph-mon[35305]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-12T02:59:02.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:02 smithi183 ceph-mon[35535]: Deploying daemon osd.9 on smithi116 2024-09-12T02:59:02.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:02 smithi183 ceph-mon[35535]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-12T02:59:04.493 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:04 smithi116 ceph-mon[31410]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T02:59:04.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:04 smithi177 ceph-mon[35305]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T02:59:04.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:04 smithi183 ceph-mon[35535]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T02:59:06.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:06 smithi116 ceph-mon[31410]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-12T02:59:06.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:06 smithi177 ceph-mon[35305]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-12T02:59:06.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:06 smithi183 ceph-mon[35535]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-12T02:59:07.527 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:07 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:07.527 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:07 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:07.527 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:07 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:07 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:07 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:07 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:07.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:07 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:07.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:07 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:07.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:07 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:08.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:08 smithi116 ceph-mon[31410]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:59:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:08 smithi183 ceph-mon[35535]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:59:08.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:08 smithi177 ceph-mon[35305]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:59:09.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:09 smithi116 ceph-mon[31410]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:59:09.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:09 smithi183 ceph-mon[35535]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:59:09.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:09 smithi177 ceph-mon[35305]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:59:10.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:59:10.940+0000 7fd30d7fa640 1 -- 172.21.15.116:0/1473269022 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd3101a2940 con 0x7fd2f80a2b80 2024-09-12T02:59:10.941 INFO:teuthology.orchestra.run.smithi116.stdout:Created osd(s) 9 on host 'smithi116' 2024-09-12T02:59:10.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:59:10.942+0000 7fd3166c2640 1 -- 172.21.15.116:0/1473269022 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd2f80a2b80 msgr2=0x7fd2f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:10.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:59:10.942+0000 7fd3166c2640 1 --2- 172.21.15.116:0/1473269022 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd2f80a2b80 0x7fd2f80a5040 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd304005fd0 tx=0x7fd304005850 comp rx=0 tx=0).stop 2024-09-12T02:59:10.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:59:10.942+0000 7fd3166c2640 1 -- 172.21.15.116:0/1473269022 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd310100ad0 msgr2=0x7fd310104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:10.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:59:10.942+0000 7fd3166c2640 1 --2- 172.21.15.116:0/1473269022 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd310100ad0 0x7fd310104a60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd300031610 tx=0x7fd3000043b0 comp rx=0 tx=0).stop 2024-09-12T02:59:10.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:59:10.942+0000 7fd3166c2640 1 -- 172.21.15.116:0/1473269022 shutdown_connections 2024-09-12T02:59:10.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:59:10.942+0000 7fd3166c2640 1 --2- 172.21.15.116:0/1473269022 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd2f80a2b80 0x7fd2f80a5040 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:10.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:59:10.942+0000 7fd3166c2640 1 --2- 172.21.15.116:0/1473269022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd310102690 0x7fd3101035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:10.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:59:10.942+0000 7fd3166c2640 1 --2- 172.21.15.116:0/1473269022 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd310101cd0 0x7fd3101030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:10.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:59:10.942+0000 7fd3166c2640 1 --2- 172.21.15.116:0/1473269022 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd310100ad0 0x7fd310104a60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:10.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:59:10.942+0000 7fd3166c2640 1 -- 172.21.15.116:0/1473269022 >> 172.21.15.116:0/1473269022 conn(0x7fd3100fc280 msgr2=0x7fd31006a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:10.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:59:10.942+0000 7fd3166c2640 1 -- 172.21.15.116:0/1473269022 shutdown_connections 2024-09-12T02:59:10.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T02:59:10.942+0000 7fd3166c2640 1 -- 172.21.15.116:0/1473269022 wait complete. 2024-09-12T02:59:11.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:11 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:11.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:11 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:11.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:11 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:11.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:11 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:11.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:11 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:11.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:11 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:11.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:11 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:11.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:11 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:11.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:11 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:11.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:11 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:11.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:11 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:11.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:11 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:11.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:11 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:11.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:11 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:11.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:11 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:11.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:11 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:11.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:11 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:11.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:11 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:11.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:11 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:11.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:11 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:11.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:11 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:11.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:11 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:11.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:11 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:11.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:11 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:11.642 DEBUG:teuthology.orchestra.run.smithi116:osd.9> sudo journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.9.service 2024-09-12T02:59:11.645 INFO:tasks.cephadm:Deploying osd.10 on smithi177 with /dev/vg_nvme/lv_1... 2024-09-12T02:59:11.645 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T02:59:11.905 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:59:12.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:12 smithi116 ceph-mon[31410]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:59:12.372 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:12 smithi177 ceph-mon[35305]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:59:12.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:12 smithi183 ceph-mon[35535]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:59:13.314 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 02:59:13 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T02:59:13.122+0000 7f6f0211d640 -1 osd.9 0 log_to_monitors true 2024-09-12T02:59:14.264 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:14 smithi116 ceph-mon[31410]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:59:14.264 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:14 smithi116 ceph-mon[31410]: from='osd.9 [v2:172.21.15.116:6826/3831673668,v1:172.21.15.116:6827/3831673668]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T02:59:14.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:14 smithi183 ceph-mon[35535]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:59:14.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:14 smithi183 ceph-mon[35535]: from='osd.9 [v2:172.21.15.116:6826/3831673668,v1:172.21.15.116:6827/3831673668]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T02:59:14.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:14 smithi177 ceph-mon[35305]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:59:14.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:14 smithi177 ceph-mon[35305]: from='osd.9 [v2:172.21.15.116:6826/3831673668,v1:172.21.15.116:6827/3831673668]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T02:59:14.563 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-09-12T02:59:14.589 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi177:vg_nvme/lv_1 2024-09-12T02:59:14.841 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:59:15.148 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:15 smithi177 ceph-mon[35305]: from='osd.9 [v2:172.21.15.116:6826/3831673668,v1:172.21.15.116:6827/3831673668]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T02:59:15.148 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:15 smithi177 ceph-mon[35305]: osdmap e57: 10 total, 9 up, 10 in 2024-09-12T02:59:15.148 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:15 smithi177 ceph-mon[35305]: from='osd.9 [v2:172.21.15.116:6826/3831673668,v1:172.21.15.116:6827/3831673668]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-09-12T02:59:15.148 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:15 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:59:15.148 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:15 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:15.148 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:15 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:15.148 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:15 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:15.149 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:15 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:15.149 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:15 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:15.149 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:15 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:15.149 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:15 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:15.149 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:15 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:15.149 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:15 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:15.149 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:15 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:15.390 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.366+0000 7f676caea640 1 -- 172.21.15.177:0/3736139749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67681026b0 msgr2=0x7f6768108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:15.390 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.366+0000 7f676caea640 1 --2- 172.21.15.177:0/3736139749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67681026b0 0x7f6768108f40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6760009ca0 tx=0x7f676002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.390 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.367+0000 7f676caea640 1 -- 172.21.15.177:0/3736139749 shutdown_connections 2024-09-12T02:59:15.390 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.367+0000 7f676caea640 1 --2- 172.21.15.177:0/3736139749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67681026b0 0x7f6768108f40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.391 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.367+0000 7f676caea640 1 --2- 172.21.15.177:0/3736139749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6768101cf0 0x7f6768102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.391 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.367+0000 7f676caea640 1 --2- 172.21.15.177:0/3736139749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6768100af0 0x7f6768100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.391 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.367+0000 7f676caea640 1 -- 172.21.15.177:0/3736139749 >> 172.21.15.177:0/3736139749 conn(0x7f67680fc280 msgr2=0x7f67680fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:15.391 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.367+0000 7f676caea640 1 -- 172.21.15.177:0/3736139749 shutdown_connections 2024-09-12T02:59:15.391 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.367+0000 7f676caea640 1 -- 172.21.15.177:0/3736139749 wait complete. 2024-09-12T02:59:15.391 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.367+0000 7f676caea640 1 Processor -- start 2024-09-12T02:59:15.391 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.367+0000 7f676caea640 1 -- start start 2024-09-12T02:59:15.392 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.367+0000 7f676caea640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6768100af0 0x7f676806b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:15.392 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.367+0000 7f676caea640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6768101cf0 0x7f676819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:15.392 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.367+0000 7f676caea640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67681026b0 0x7f676819f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:15.392 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.368+0000 7f676caea640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f676810c770 con 0x7f6768100af0 2024-09-12T02:59:15.392 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.368+0000 7f676caea640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f676810c5f0 con 0x7f6768101cf0 2024-09-12T02:59:15.392 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.368+0000 7f676caea640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f676810c8f0 con 0x7f67681026b0 2024-09-12T02:59:15.392 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.368+0000 7f6765d9b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6768101cf0 0x7f676819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:15.392 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.368+0000 7f6765d9b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6768101cf0 0x7f676819cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:35314/0 (socket says 172.21.15.177:35314) 2024-09-12T02:59:15.392 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.368+0000 7f6765d9b640 1 -- 172.21.15.177:0/3215120234 learned_addr learned my addr 172.21.15.177:0/3215120234 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-09-12T02:59:15.393 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.368+0000 7f676659c640 1 --2- 172.21.15.177:0/3215120234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6768100af0 0x7f676806b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:15.393 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.368+0000 7f6766d9d640 1 --2- 172.21.15.177:0/3215120234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67681026b0 0x7f676819f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:15.393 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.368+0000 7f676659c640 1 -- 172.21.15.177:0/3215120234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67681026b0 msgr2=0x7f676819f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:15.393 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.368+0000 7f676659c640 1 --2- 172.21.15.177:0/3215120234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67681026b0 0x7f676819f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.393 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.368+0000 7f676659c640 1 -- 172.21.15.177:0/3215120234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6768101cf0 msgr2=0x7f676819cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:15.393 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.368+0000 7f676659c640 1 --2- 172.21.15.177:0/3215120234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6768101cf0 0x7f676819cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.393 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.368+0000 7f676659c640 1 -- 172.21.15.177:0/3215120234 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676819fac0 con 0x7f6768100af0 2024-09-12T02:59:15.393 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.368+0000 7f6765d9b640 1 --2- 172.21.15.177:0/3215120234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6768101cf0 0x7f676819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:59:15.393 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.368+0000 7f6766d9d640 1 --2- 172.21.15.177:0/3215120234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67681026b0 0x7f676819f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:59:15.393 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.368+0000 7f676659c640 1 --2- 172.21.15.177:0/3215120234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6768100af0 0x7f676806b2f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f675800c940 tx=0x7f675800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:15.393 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.368+0000 7f67577fe640 1 -- 172.21.15.177:0/3215120234 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6758007bb0 con 0x7f6768100af0 2024-09-12T02:59:15.394 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.368+0000 7f67577fe640 1 -- 172.21.15.177:0/3215120234 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6758007d50 con 0x7f6768100af0 2024-09-12T02:59:15.394 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.368+0000 7f67577fe640 1 -- 172.21.15.177:0/3215120234 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6758018790 con 0x7f6768100af0 2024-09-12T02:59:15.394 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.368+0000 7f676caea640 1 -- 172.21.15.177:0/3215120234 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f676819fd50 con 0x7f6768100af0 2024-09-12T02:59:15.394 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.369+0000 7f676caea640 1 -- 172.21.15.177:0/3215120234 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67681a0230 con 0x7f6768100af0 2024-09-12T02:59:15.394 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.370+0000 7f676caea640 1 -- 172.21.15.177:0/3215120234 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6768107ea0 con 0x7f6768100af0 2024-09-12T02:59:15.394 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.371+0000 7f67577fe640 1 -- 172.21.15.177:0/3215120234 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f675800f450 con 0x7f6768100af0 2024-09-12T02:59:15.394 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.371+0000 7f67577fe640 1 --2- 172.21.15.177:0/3215120234 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f67500a2b80 0x7f67500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:15.394 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.371+0000 7f67577fe640 1 -- 172.21.15.177:0/3215120234 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5981+0+0 (secure 0 0 0) 0x7f67580ce6b0 con 0x7f6768100af0 2024-09-12T02:59:15.394 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.371+0000 7f6765d9b640 1 --2- 172.21.15.177:0/3215120234 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f67500a2b80 0x7f67500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:15.394 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.372+0000 7f6765d9b640 1 --2- 172.21.15.177:0/3215120234 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f67500a2b80 0x7f67500a5040 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f676806b870 tx=0x7f675c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:15.394 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.376+0000 7f67577fe640 1 -- 172.21.15.177:0/3215120234 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6758014030 con 0x7f6768100af0 2024-09-12T02:59:15.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:15 smithi183 ceph-mon[35535]: from='osd.9 [v2:172.21.15.116:6826/3831673668,v1:172.21.15.116:6827/3831673668]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T02:59:15.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:15 smithi183 ceph-mon[35535]: osdmap e57: 10 total, 9 up, 10 in 2024-09-12T02:59:15.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:15 smithi183 ceph-mon[35535]: from='osd.9 [v2:172.21.15.116:6826/3831673668,v1:172.21.15.116:6827/3831673668]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-09-12T02:59:15.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:15 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:59:15.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:15 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:15.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:15 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:15.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:15 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:15.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:15 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:15.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:15 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:15.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:15 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:15.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:15 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:15.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:15 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:15.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:15 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:15.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:15 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:15.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 02:59:15 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T02:59:15.104+0000 7f6efe0a0640 -1 osd.9 0 waiting for initial osdmap 2024-09-12T02:59:15.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 02:59:15 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T02:59:15.113+0000 7f6ef9ec8640 -1 osd.9 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:59:15.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:15 smithi116 ceph-mon[31410]: from='osd.9 [v2:172.21.15.116:6826/3831673668,v1:172.21.15.116:6827/3831673668]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T02:59:15.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:15 smithi116 ceph-mon[31410]: osdmap e57: 10 total, 9 up, 10 in 2024-09-12T02:59:15.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:15 smithi116 ceph-mon[31410]: from='osd.9 [v2:172.21.15.116:6826/3831673668,v1:172.21.15.116:6827/3831673668]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-09-12T02:59:15.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:59:15.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:15.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:15.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:15.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:15.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:15.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:15.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:15.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:15.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:15.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:15.557 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:15.557+0000 7f676caea640 1 -- 172.21.15.177:0/3215120234 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6768062630 con 0x7f67500a2b80 2024-09-12T02:59:16.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:16 smithi183 ceph-mon[35535]: Detected new or changed devices on smithi116 2024-09-12T02:59:16.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:16 smithi183 ceph-mon[35535]: Adjusting osd_memory_target on smithi116 to 4272M 2024-09-12T02:59:16.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:16 smithi183 ceph-mon[35535]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:59:16.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:16 smithi183 ceph-mon[35535]: from='osd.9 [v2:172.21.15.116:6826/3831673668,v1:172.21.15.116:6827/3831673668]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-09-12T02:59:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:16 smithi183 ceph-mon[35535]: osdmap e58: 10 total, 9 up, 10 in 2024-09-12T02:59:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:16 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:59:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:16 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:59:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:16 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:59:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:16 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:59:16.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:16 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:16.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:16 smithi177 ceph-mon[35305]: Detected new or changed devices on smithi116 2024-09-12T02:59:16.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:16 smithi177 ceph-mon[35305]: Adjusting osd_memory_target on smithi116 to 4272M 2024-09-12T02:59:16.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:16 smithi177 ceph-mon[35305]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:59:16.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:16 smithi177 ceph-mon[35305]: from='osd.9 [v2:172.21.15.116:6826/3831673668,v1:172.21.15.116:6827/3831673668]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-09-12T02:59:16.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:16 smithi177 ceph-mon[35305]: osdmap e58: 10 total, 9 up, 10 in 2024-09-12T02:59:16.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:16 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:59:16.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:16 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:59:16.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:16 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:59:16.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:16 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:59:16.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:16 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:16.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:16 smithi116 ceph-mon[31410]: Detected new or changed devices on smithi116 2024-09-12T02:59:16.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:16 smithi116 ceph-mon[31410]: Adjusting osd_memory_target on smithi116 to 4272M 2024-09-12T02:59:16.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:16 smithi116 ceph-mon[31410]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:59:16.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:16 smithi116 ceph-mon[31410]: from='osd.9 [v2:172.21.15.116:6826/3831673668,v1:172.21.15.116:6827/3831673668]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-09-12T02:59:16.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:16 smithi116 ceph-mon[31410]: osdmap e58: 10 total, 9 up, 10 in 2024-09-12T02:59:16.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:16 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:59:16.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:16 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:59:16.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:16 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:59:16.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:16 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:59:16.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:16 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:17.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:17 smithi177 ceph-mon[35305]: purged_snaps scrub starts 2024-09-12T02:59:17.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:17 smithi177 ceph-mon[35305]: purged_snaps scrub ok 2024-09-12T02:59:17.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:17 smithi177 ceph-mon[35305]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:59:17.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:17 smithi177 ceph-mon[35305]: osd.9 [v2:172.21.15.116:6826/3831673668,v1:172.21.15.116:6827/3831673668] boot 2024-09-12T02:59:17.246 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:17 smithi177 ceph-mon[35305]: osdmap e59: 10 total, 10 up, 10 in 2024-09-12T02:59:17.246 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:17 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:59:17.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:17 smithi183 ceph-mon[35535]: purged_snaps scrub starts 2024-09-12T02:59:17.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:17 smithi183 ceph-mon[35535]: purged_snaps scrub ok 2024-09-12T02:59:17.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:17 smithi183 ceph-mon[35535]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:59:17.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:17 smithi183 ceph-mon[35535]: osd.9 [v2:172.21.15.116:6826/3831673668,v1:172.21.15.116:6827/3831673668] boot 2024-09-12T02:59:17.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:17 smithi183 ceph-mon[35535]: osdmap e59: 10 total, 10 up, 10 in 2024-09-12T02:59:17.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:17 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:59:17.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:17 smithi116 ceph-mon[31410]: purged_snaps scrub starts 2024-09-12T02:59:17.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:17 smithi116 ceph-mon[31410]: purged_snaps scrub ok 2024-09-12T02:59:17.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:17 smithi116 ceph-mon[31410]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:59:17.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:17 smithi116 ceph-mon[31410]: osd.9 [v2:172.21.15.116:6826/3831673668,v1:172.21.15.116:6827/3831673668] boot 2024-09-12T02:59:17.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:17 smithi116 ceph-mon[31410]: osdmap e59: 10 total, 10 up, 10 in 2024-09-12T02:59:17.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:17 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:59:18.392 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:18 smithi177 ceph-mon[35305]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:59:18.393 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:18 smithi177 ceph-mon[35305]: osdmap e60: 10 total, 10 up, 10 in 2024-09-12T02:59:18.393 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:18 smithi177 ceph-mon[35305]: from='client.? 172.21.15.177:0/2953403357' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "864d1a97-4810-49c8-ad4a-a0c10253833c"}]: dispatch 2024-09-12T02:59:18.393 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:18 smithi177 ceph-mon[35305]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "864d1a97-4810-49c8-ad4a-a0c10253833c"}]: dispatch 2024-09-12T02:59:18.393 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:18 smithi177 ceph-mon[35305]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "864d1a97-4810-49c8-ad4a-a0c10253833c"}]': finished 2024-09-12T02:59:18.393 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:18 smithi177 ceph-mon[35305]: osdmap e61: 11 total, 10 up, 11 in 2024-09-12T02:59:18.393 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:18 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:59:18.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:18 smithi183 ceph-mon[35535]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:59:18.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:18 smithi183 ceph-mon[35535]: osdmap e60: 10 total, 10 up, 10 in 2024-09-12T02:59:18.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:18 smithi183 ceph-mon[35535]: from='client.? 172.21.15.177:0/2953403357' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "864d1a97-4810-49c8-ad4a-a0c10253833c"}]: dispatch 2024-09-12T02:59:18.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:18 smithi183 ceph-mon[35535]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "864d1a97-4810-49c8-ad4a-a0c10253833c"}]: dispatch 2024-09-12T02:59:18.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:18 smithi183 ceph-mon[35535]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "864d1a97-4810-49c8-ad4a-a0c10253833c"}]': finished 2024-09-12T02:59:18.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:18 smithi183 ceph-mon[35535]: osdmap e61: 11 total, 10 up, 11 in 2024-09-12T02:59:18.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:18 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:59:18.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:18 smithi116 ceph-mon[31410]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:59:18.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:18 smithi116 ceph-mon[31410]: osdmap e60: 10 total, 10 up, 10 in 2024-09-12T02:59:18.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:18 smithi116 ceph-mon[31410]: from='client.? 172.21.15.177:0/2953403357' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "864d1a97-4810-49c8-ad4a-a0c10253833c"}]: dispatch 2024-09-12T02:59:18.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:18 smithi116 ceph-mon[31410]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "864d1a97-4810-49c8-ad4a-a0c10253833c"}]: dispatch 2024-09-12T02:59:18.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:18 smithi116 ceph-mon[31410]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "864d1a97-4810-49c8-ad4a-a0c10253833c"}]': finished 2024-09-12T02:59:18.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:18 smithi116 ceph-mon[31410]: osdmap e61: 11 total, 10 up, 11 in 2024-09-12T02:59:18.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:18 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:59:19.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:19 smithi183 ceph-mon[35535]: from='client.? 172.21.15.177:0/4164783210' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:59:19.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:19 smithi177 ceph-mon[35305]: from='client.? 172.21.15.177:0/4164783210' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:59:19.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:19 smithi116 ceph-mon[31410]: from='client.? 172.21.15.177:0/4164783210' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:59:20.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:20 smithi183 ceph-mon[35535]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:20.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:20 smithi177 ceph-mon[35305]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:20.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:20 smithi116 ceph-mon[31410]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:22.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:22 smithi183 ceph-mon[35535]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:22.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:22 smithi177 ceph-mon[35305]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:22.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:22 smithi116 ceph-mon[31410]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:24.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:24 smithi183 ceph-mon[35535]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:24.482 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:24 smithi177 ceph-mon[35305]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:24.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:24 smithi116 ceph-mon[31410]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:26.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:26 smithi183 ceph-mon[35535]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:26.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:26 smithi177 ceph-mon[35305]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:26.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:26 smithi116 ceph-mon[31410]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:27.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:27 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T02:59:27.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:27 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:27.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:27 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T02:59:27.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:27 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:27.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:27 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T02:59:27.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:27 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:28.359 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:28 smithi177 ceph-mon[35305]: Deploying daemon osd.10 on smithi177 2024-09-12T02:59:28.359 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:28 smithi177 ceph-mon[35305]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:28.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:28 smithi183 ceph-mon[35535]: Deploying daemon osd.10 on smithi177 2024-09-12T02:59:28.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:28 smithi183 ceph-mon[35535]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:28.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:28 smithi116 ceph-mon[31410]: Deploying daemon osd.10 on smithi177 2024-09-12T02:59:28.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:28 smithi116 ceph-mon[31410]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:30.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:30 smithi183 ceph-mon[35535]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:30.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:30 smithi177 ceph-mon[35305]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:30.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:30 smithi116 ceph-mon[31410]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:32.361 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:32 smithi177 ceph-mon[35305]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:32.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:32 smithi183 ceph-mon[35535]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:32.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:32 smithi116 ceph-mon[31410]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:33.407 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:33 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:33.407 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:33 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:33.407 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:33 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:33.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:33 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:33.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:33 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:33.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:33 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:33.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:33 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:33.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:33 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:33.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:33 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:34.447 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:34 smithi177 ceph-mon[35305]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:34.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:34 smithi183 ceph-mon[35535]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:34.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:34 smithi116 ceph-mon[31410]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:36.286 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:36 smithi177 ceph-mon[35305]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:36.324 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:36.323+0000 7f67577fe640 1 -- 172.21.15.177:0/3215120234 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f6768062630 con 0x7f67500a2b80 2024-09-12T02:59:36.325 INFO:teuthology.orchestra.run.smithi177.stdout:Created osd(s) 10 on host 'smithi177' 2024-09-12T02:59:36.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:36.325+0000 7f676caea640 1 -- 172.21.15.177:0/3215120234 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f67500a2b80 msgr2=0x7f67500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:36.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:36.325+0000 7f676caea640 1 --2- 172.21.15.177:0/3215120234 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f67500a2b80 0x7f67500a5040 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f676806b870 tx=0x7f675c005f90 comp rx=0 tx=0).stop 2024-09-12T02:59:36.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:36.325+0000 7f676caea640 1 -- 172.21.15.177:0/3215120234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6768100af0 msgr2=0x7f676806b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:36.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:36.325+0000 7f676caea640 1 --2- 172.21.15.177:0/3215120234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6768100af0 0x7f676806b2f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f675800c940 tx=0x7f675800ce00 comp rx=0 tx=0).stop 2024-09-12T02:59:36.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:36.325+0000 7f676caea640 1 -- 172.21.15.177:0/3215120234 shutdown_connections 2024-09-12T02:59:36.328 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:36.325+0000 7f676caea640 1 --2- 172.21.15.177:0/3215120234 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f67500a2b80 0x7f67500a5040 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:36.328 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:36.325+0000 7f676caea640 1 --2- 172.21.15.177:0/3215120234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67681026b0 0x7f676819f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:36.328 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:36.325+0000 7f676caea640 1 --2- 172.21.15.177:0/3215120234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6768101cf0 0x7f676819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:36.328 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:36.325+0000 7f676caea640 1 --2- 172.21.15.177:0/3215120234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6768100af0 0x7f676806b2f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:36.328 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:36.325+0000 7f676caea640 1 -- 172.21.15.177:0/3215120234 >> 172.21.15.177:0/3215120234 conn(0x7f67680fc280 msgr2=0x7f6768105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:36.328 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:36.326+0000 7f676caea640 1 -- 172.21.15.177:0/3215120234 shutdown_connections 2024-09-12T02:59:36.328 INFO:teuthology.orchestra.run.smithi177.stderr:2024-09-12T02:59:36.326+0000 7f676caea640 1 -- 172.21.15.177:0/3215120234 wait complete. 2024-09-12T02:59:36.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:36 smithi183 ceph-mon[35535]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:36.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:36 smithi116 ceph-mon[31410]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:36.951 DEBUG:teuthology.orchestra.run.smithi177:osd.10> sudo journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.10.service 2024-09-12T02:59:36.954 INFO:tasks.cephadm:Deploying osd.11 on smithi183 with /dev/vg_nvme/lv_1... 2024-09-12T02:59:36.954 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T02:59:37.210 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:59:37.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:37 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:37.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:37 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:37.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:37 smithi116 ceph-mon[31410]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:37.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:37 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:37.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:37 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:37.574 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:37 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:37.574 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:37 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:37.574 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:37 smithi183 ceph-mon[35535]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:37.574 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:37 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:37.574 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:37 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:37.609 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:37 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:37.609 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:37 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:37.609 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:37 smithi177 ceph-mon[35305]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:37.609 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:37 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:37.609 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:37 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:39.092 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 02:59:39 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T02:59:39.003+0000 7fe5ad6a7640 -1 osd.10 0 log_to_monitors true 2024-09-12T02:59:39.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:39 smithi183 ceph-mon[35535]: from='osd.10 [v2:172.21.15.177:6824/963663634,v1:172.21.15.177:6825/963663634]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T02:59:39.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:39 smithi183 ceph-mon[35535]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T02:59:39.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:39 smithi177 ceph-mon[35305]: from='osd.10 [v2:172.21.15.177:6824/963663634,v1:172.21.15.177:6825/963663634]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T02:59:39.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:39 smithi177 ceph-mon[35305]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T02:59:39.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:39 smithi116 ceph-mon[31410]: from='osd.10 [v2:172.21.15.177:6824/963663634,v1:172.21.15.177:6825/963663634]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T02:59:39.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:39 smithi116 ceph-mon[31410]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T02:59:39.978 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-12T02:59:40.005 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_1 2024-09-12T02:59:40.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:40 smithi177 ceph-mon[35305]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:40.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:40 smithi177 ceph-mon[35305]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T02:59:40.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:40 smithi177 ceph-mon[35305]: from='osd.10 [v2:172.21.15.177:6824/963663634,v1:172.21.15.177:6825/963663634]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-09-12T02:59:40.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:40 smithi177 ceph-mon[35305]: osdmap e62: 11 total, 10 up, 11 in 2024-09-12T02:59:40.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:40 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:59:40.246 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:40 smithi177 ceph-mon[35305]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-09-12T02:59:40.264 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:59:40.385 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:40 smithi183 ceph-mon[35535]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:40.385 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:40 smithi183 ceph-mon[35535]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T02:59:40.385 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:40 smithi183 ceph-mon[35535]: from='osd.10 [v2:172.21.15.177:6824/963663634,v1:172.21.15.177:6825/963663634]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-09-12T02:59:40.385 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:40 smithi183 ceph-mon[35535]: osdmap e62: 11 total, 10 up, 11 in 2024-09-12T02:59:40.385 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:40 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:59:40.386 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:40 smithi183 ceph-mon[35535]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-09-12T02:59:40.528 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 02:59:40 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T02:59:40.516+0000 7fe5a962a640 -1 osd.10 0 waiting for initial osdmap 2024-09-12T02:59:40.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:40 smithi116 ceph-mon[31410]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:40.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:40 smithi116 ceph-mon[31410]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T02:59:40.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:40 smithi116 ceph-mon[31410]: from='osd.10 [v2:172.21.15.177:6824/963663634,v1:172.21.15.177:6825/963663634]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-09-12T02:59:40.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:40 smithi116 ceph-mon[31410]: osdmap e62: 11 total, 10 up, 11 in 2024-09-12T02:59:40.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:40 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:59:40.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:40 smithi116 ceph-mon[31410]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-09-12T02:59:40.784 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.759+0000 7f8c10b3f640 1 -- 172.21.15.183:0/1152624270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c0c100b10 msgr2=0x7f8c0c100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:40.784 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.759+0000 7f8c10b3f640 1 --2- 172.21.15.183:0/1152624270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c0c100b10 0x7f8c0c100f10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc009900 tx=0x7f8bfc02a410 comp rx=0 tx=0).stop 2024-09-12T02:59:40.784 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.759+0000 7f8c10b3f640 1 -- 172.21.15.183:0/1152624270 shutdown_connections 2024-09-12T02:59:40.784 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.759+0000 7f8c10b3f640 1 --2- 172.21.15.183:0/1152624270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c0c1026d0 0x7f8c0c108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.784 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.759+0000 7f8c10b3f640 1 --2- 172.21.15.183:0/1152624270 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c0c101d10 0x7f8c0c102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.784 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.759+0000 7f8c10b3f640 1 --2- 172.21.15.183:0/1152624270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c0c100b10 0x7f8c0c100f10 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.784 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.759+0000 7f8c10b3f640 1 -- 172.21.15.183:0/1152624270 >> 172.21.15.183:0/1152624270 conn(0x7f8c0c0fc2a0 msgr2=0x7f8c0c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:40.784 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.759+0000 7f8c10b3f640 1 -- 172.21.15.183:0/1152624270 shutdown_connections 2024-09-12T02:59:40.784 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.759+0000 7f8c10b3f640 1 -- 172.21.15.183:0/1152624270 wait complete. 2024-09-12T02:59:40.784 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.760+0000 7f8c10b3f640 1 Processor -- start 2024-09-12T02:59:40.785 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.760+0000 7f8c10b3f640 1 -- start start 2024-09-12T02:59:40.785 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.760+0000 7f8c10b3f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c0c100b10 0x7f8c0c06b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:40.785 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.760+0000 7f8c10b3f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c0c101d10 0x7f8c0c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:40.785 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.760+0000 7f8c10b3f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c0c1026d0 0x7f8c0c19f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:40.785 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.760+0000 7f8c10b3f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c10c790 con 0x7f8c0c101d10 2024-09-12T02:59:40.785 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.760+0000 7f8c10b3f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c10c610 con 0x7f8c0c100b10 2024-09-12T02:59:40.785 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.760+0000 7f8c10b3f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c10c910 con 0x7f8c0c1026d0 2024-09-12T02:59:40.785 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.760+0000 7f8c0ad9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c0c1026d0 0x7f8c0c19f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:40.785 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.760+0000 7f8c0ad9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c0c1026d0 0x7f8c0c19f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:60264/0 (socket says 172.21.15.183:60264) 2024-09-12T02:59:40.785 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.760+0000 7f8c0ad9d640 1 -- 172.21.15.183:0/1366549094 learned_addr learned my addr 172.21.15.183:0/1366549094 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-12T02:59:40.785 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.760+0000 7f8c0a59c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c0c100b10 0x7f8c0c06b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:40.785 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.760+0000 7f8c09d9b640 1 --2- 172.21.15.183:0/1366549094 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c0c101d10 0x7f8c0c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:40.785 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.760+0000 7f8c0ad9d640 1 -- 172.21.15.183:0/1366549094 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c0c100b10 msgr2=0x7f8c0c06b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:40.786 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.760+0000 7f8c0ad9d640 1 --2- 172.21.15.183:0/1366549094 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c0c100b10 0x7f8c0c06b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.786 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.760+0000 7f8c0ad9d640 1 -- 172.21.15.183:0/1366549094 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c0c101d10 msgr2=0x7f8c0c19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:40.786 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.760+0000 7f8c0ad9d640 1 --2- 172.21.15.183:0/1366549094 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c0c101d10 0x7f8c0c19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.786 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.761+0000 7f8c0ad9d640 1 -- 172.21.15.183:0/1366549094 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0c19fbb0 con 0x7f8c0c1026d0 2024-09-12T02:59:40.786 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.761+0000 7f8c09d9b640 1 --2- 172.21.15.183:0/1366549094 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c0c101d10 0x7f8c0c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:59:40.786 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.761+0000 7f8c0ad9d640 1 --2- 172.21.15.183:0/1366549094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c0c1026d0 0x7f8c0c19f2a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8c0400eaa0 tx=0x7f8c0400ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:40.786 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.761+0000 7f8bfb7fe640 1 -- 172.21.15.183:0/1366549094 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0400cdb0 con 0x7f8c0c1026d0 2024-09-12T02:59:40.786 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.761+0000 7f8bfb7fe640 1 -- 172.21.15.183:0/1366549094 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c04005020 con 0x7f8c0c1026d0 2024-09-12T02:59:40.786 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.761+0000 7f8c10b3f640 1 -- 172.21.15.183:0/1366549094 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c0c19fe40 con 0x7f8c0c1026d0 2024-09-12T02:59:40.786 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.761+0000 7f8bfb7fe640 1 -- 172.21.15.183:0/1366549094 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c040116b0 con 0x7f8c0c1026d0 2024-09-12T02:59:40.786 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.761+0000 7f8c10b3f640 1 -- 172.21.15.183:0/1366549094 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c0c1a0300 con 0x7f8c0c1026d0 2024-09-12T02:59:40.786 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.762+0000 7f8c10b3f640 1 -- 172.21.15.183:0/1366549094 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c0c107f00 con 0x7f8c0c1026d0 2024-09-12T02:59:40.787 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.763+0000 7f8bfb7fe640 1 -- 172.21.15.183:0/1366549094 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8c04005560 con 0x7f8c0c1026d0 2024-09-12T02:59:40.787 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.764+0000 7f8bfb7fe640 1 --2- 172.21.15.183:0/1366549094 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8bf40a2b80 0x7f8bf40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:40.787 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.764+0000 7f8bfb7fe640 1 -- 172.21.15.183:0/1366549094 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6414+0+0 (secure 0 0 0) 0x7f8c040ceaa0 con 0x7f8c0c1026d0 2024-09-12T02:59:40.787 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.764+0000 7f8c0a59c640 1 --2- 172.21.15.183:0/1366549094 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8bf40a2b80 0x7f8bf40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:40.787 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.764+0000 7f8c0a59c640 1 --2- 172.21.15.183:0/1366549094 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8bf40a2b80 0x7f8bf40a5040 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc005ab0 tx=0x7f8bfc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:40.787 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.768+0000 7f8bfb7fe640 1 -- 172.21.15.183:0/1366549094 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8c04016030 con 0x7f8c0c1026d0 2024-09-12T02:59:40.944 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T02:59:40.944+0000 7f8c10b3f640 1 -- 172.21.15.183:0/1366549094 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8c0c062630 con 0x7f8bf40a2b80 2024-09-12T02:59:41.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:41 smithi183 ceph-mon[35535]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-09-12T02:59:41.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:41 smithi183 ceph-mon[35535]: osdmap e63: 11 total, 10 up, 11 in 2024-09-12T02:59:41.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:59:41.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:59:41.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:41.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:41.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:41.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:41.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:41.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:41.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:41.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:41.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:41.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:41.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:41.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:41.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:41.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:41.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:41 smithi183 ceph-mon[35535]: from='osd.10 ' entity='osd.10' 2024-09-12T02:59:41.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:59:41.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:59:41.205 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:41.495 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 02:59:40 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T02:59:40.528+0000 7fe5a144a640 -1 osd.10 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:59:41.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:41 smithi177 ceph-mon[35305]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-09-12T02:59:41.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:41 smithi177 ceph-mon[35305]: osdmap e63: 11 total, 10 up, 11 in 2024-09-12T02:59:41.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:41 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:59:41.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:41 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:59:41.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:41 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:41.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:41 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:41.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:41 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:41.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:41 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:41.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:41 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:41.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:41 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:41.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:41 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:41.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:41 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:41.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:41 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:41.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:41 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:41.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:41 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:41.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:41 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:41.497 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:41 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:41.497 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:41 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:41.497 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:41 smithi177 ceph-mon[35305]: from='osd.10 ' entity='osd.10' 2024-09-12T02:59:41.497 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:41 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:59:41.497 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:41 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:59:41.497 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:41 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:41.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:41 smithi116 ceph-mon[31410]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-09-12T02:59:41.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:41 smithi116 ceph-mon[31410]: osdmap e63: 11 total, 10 up, 11 in 2024-09-12T02:59:41.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:59:41.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:59:41.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:41.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:41.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:41.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:41.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:41.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:41.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:41.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:41.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:41.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:41.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:41.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:41.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:41.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:41.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:41 smithi116 ceph-mon[31410]: from='osd.10 ' entity='osd.10' 2024-09-12T02:59:41.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:59:41.554 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:59:41.555 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:42.400 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:42 smithi177 ceph-mon[35305]: purged_snaps scrub starts 2024-09-12T02:59:42.400 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:42 smithi177 ceph-mon[35305]: purged_snaps scrub ok 2024-09-12T02:59:42.400 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:42 smithi177 ceph-mon[35305]: Detected new or changed devices on smithi177 2024-09-12T02:59:42.401 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:42 smithi177 ceph-mon[35305]: Adjusting osd_memory_target on smithi177 to 4271M 2024-09-12T02:59:42.401 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:42 smithi177 ceph-mon[35305]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:59:42.401 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:42 smithi177 ceph-mon[35305]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:42.401 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:42 smithi177 ceph-mon[35305]: osd.10 [v2:172.21.15.177:6824/963663634,v1:172.21.15.177:6825/963663634] boot 2024-09-12T02:59:42.401 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:42 smithi177 ceph-mon[35305]: osdmap e64: 11 total, 11 up, 11 in 2024-09-12T02:59:42.401 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:42 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:59:42.426 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:42 smithi183 ceph-mon[35535]: purged_snaps scrub starts 2024-09-12T02:59:42.426 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:42 smithi183 ceph-mon[35535]: purged_snaps scrub ok 2024-09-12T02:59:42.426 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:42 smithi183 ceph-mon[35535]: Detected new or changed devices on smithi177 2024-09-12T02:59:42.426 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:42 smithi183 ceph-mon[35535]: Adjusting osd_memory_target on smithi177 to 4271M 2024-09-12T02:59:42.426 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:42 smithi183 ceph-mon[35535]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:59:42.426 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:42 smithi183 ceph-mon[35535]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:42.426 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:42 smithi183 ceph-mon[35535]: osd.10 [v2:172.21.15.177:6824/963663634,v1:172.21.15.177:6825/963663634] boot 2024-09-12T02:59:42.426 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:42 smithi183 ceph-mon[35535]: osdmap e64: 11 total, 11 up, 11 in 2024-09-12T02:59:42.426 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:42 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:59:42.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:42 smithi116 ceph-mon[31410]: purged_snaps scrub starts 2024-09-12T02:59:42.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:42 smithi116 ceph-mon[31410]: purged_snaps scrub ok 2024-09-12T02:59:42.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:42 smithi116 ceph-mon[31410]: Detected new or changed devices on smithi177 2024-09-12T02:59:42.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:42 smithi116 ceph-mon[31410]: Adjusting osd_memory_target on smithi177 to 4271M 2024-09-12T02:59:42.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:42 smithi116 ceph-mon[31410]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:59:42.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:42 smithi116 ceph-mon[31410]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:59:42.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:42 smithi116 ceph-mon[31410]: osd.10 [v2:172.21.15.177:6824/963663634,v1:172.21.15.177:6825/963663634] boot 2024-09-12T02:59:42.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:42 smithi116 ceph-mon[31410]: osdmap e64: 11 total, 11 up, 11 in 2024-09-12T02:59:42.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:42 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:59:43.412 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:43 smithi177 ceph-mon[35305]: osdmap e65: 11 total, 11 up, 11 in 2024-09-12T02:59:43.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:43 smithi183 ceph-mon[35535]: osdmap e65: 11 total, 11 up, 11 in 2024-09-12T02:59:43.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:43 smithi116 ceph-mon[31410]: osdmap e65: 11 total, 11 up, 11 in 2024-09-12T02:59:44.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:44 smithi183 ceph-mon[35535]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:59:44.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:44 smithi183 ceph-mon[35535]: osdmap e66: 11 total, 11 up, 11 in 2024-09-12T02:59:44.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:44 smithi183 ceph-mon[35535]: from='client.? 172.21.15.183:0/750955478' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d49c0841-4620-4f2f-a6b5-f2a9aa2e3c95"}]: dispatch 2024-09-12T02:59:44.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:44 smithi183 ceph-mon[35535]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d49c0841-4620-4f2f-a6b5-f2a9aa2e3c95"}]: dispatch 2024-09-12T02:59:44.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:44 smithi183 ceph-mon[35535]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d49c0841-4620-4f2f-a6b5-f2a9aa2e3c95"}]': finished 2024-09-12T02:59:44.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:44 smithi183 ceph-mon[35535]: osdmap e67: 12 total, 11 up, 12 in 2024-09-12T02:59:44.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:44 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:59:44.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:44 smithi183 ceph-mon[35535]: from='client.? 172.21.15.183:0/476106496' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:59:44.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:44 smithi177 ceph-mon[35305]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:59:44.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:44 smithi177 ceph-mon[35305]: osdmap e66: 11 total, 11 up, 11 in 2024-09-12T02:59:44.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:44 smithi177 ceph-mon[35305]: from='client.? 172.21.15.183:0/750955478' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d49c0841-4620-4f2f-a6b5-f2a9aa2e3c95"}]: dispatch 2024-09-12T02:59:44.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:44 smithi177 ceph-mon[35305]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d49c0841-4620-4f2f-a6b5-f2a9aa2e3c95"}]: dispatch 2024-09-12T02:59:44.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:44 smithi177 ceph-mon[35305]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d49c0841-4620-4f2f-a6b5-f2a9aa2e3c95"}]': finished 2024-09-12T02:59:44.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:44 smithi177 ceph-mon[35305]: osdmap e67: 12 total, 11 up, 12 in 2024-09-12T02:59:44.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:44 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:59:44.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:44 smithi177 ceph-mon[35305]: from='client.? 172.21.15.183:0/476106496' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:59:44.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:44 smithi116 ceph-mon[31410]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:59:44.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:44 smithi116 ceph-mon[31410]: osdmap e66: 11 total, 11 up, 11 in 2024-09-12T02:59:44.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:44 smithi116 ceph-mon[31410]: from='client.? 172.21.15.183:0/750955478' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d49c0841-4620-4f2f-a6b5-f2a9aa2e3c95"}]: dispatch 2024-09-12T02:59:44.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:44 smithi116 ceph-mon[31410]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d49c0841-4620-4f2f-a6b5-f2a9aa2e3c95"}]: dispatch 2024-09-12T02:59:44.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:44 smithi116 ceph-mon[31410]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d49c0841-4620-4f2f-a6b5-f2a9aa2e3c95"}]': finished 2024-09-12T02:59:44.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:44 smithi116 ceph-mon[31410]: osdmap e67: 12 total, 11 up, 12 in 2024-09-12T02:59:44.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:44 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:59:44.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:44 smithi116 ceph-mon[31410]: from='client.? 172.21.15.183:0/476106496' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:59:45.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:45 smithi177 ceph-mon[35305]: osdmap e68: 12 total, 11 up, 12 in 2024-09-12T02:59:45.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:45 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:59:45.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:45 smithi177 ceph-mon[35305]: pgmap v229: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T02:59:45.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:45 smithi116 ceph-mon[31410]: osdmap e68: 12 total, 11 up, 12 in 2024-09-12T02:59:45.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:45 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:59:45.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:45 smithi116 ceph-mon[31410]: pgmap v229: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T02:59:45.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:45 smithi183 ceph-mon[35535]: osdmap e68: 12 total, 11 up, 12 in 2024-09-12T02:59:45.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:45 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:59:45.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:45 smithi183 ceph-mon[35535]: pgmap v229: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T02:59:46.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:46 smithi177 ceph-mon[35305]: osdmap e69: 12 total, 11 up, 12 in 2024-09-12T02:59:46.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:46 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:59:46.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:46 smithi116 ceph-mon[31410]: osdmap e69: 12 total, 11 up, 12 in 2024-09-12T02:59:46.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:46 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:59:46.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:46 smithi183 ceph-mon[35535]: osdmap e69: 12 total, 11 up, 12 in 2024-09-12T02:59:46.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:46 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:59:47.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:47 smithi177 ceph-mon[35305]: pgmap v231: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T02:59:47.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:47 smithi116 ceph-mon[31410]: pgmap v231: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T02:59:47.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:47 smithi183 ceph-mon[35535]: pgmap v231: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T02:59:50.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:50 smithi183 ceph-mon[35535]: pgmap v232: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-12T02:59:50.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:50 smithi177 ceph-mon[35305]: pgmap v232: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-12T02:59:50.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:50 smithi116 ceph-mon[31410]: pgmap v232: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-12T02:59:52.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:52 smithi183 ceph-mon[35535]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-12T02:59:52.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:52 smithi177 ceph-mon[35305]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-12T02:59:52.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:52 smithi116 ceph-mon[31410]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-12T02:59:53.417 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:53 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T02:59:53.417 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:53 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:53.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:53 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T02:59:53.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:53 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:53.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:53 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T02:59:53.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:53 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:54.305 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:54 smithi183 ceph-mon[35535]: Deploying daemon osd.11 on smithi183 2024-09-12T02:59:54.305 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:54 smithi183 ceph-mon[35535]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:59:54.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:54 smithi177 ceph-mon[35305]: Deploying daemon osd.11 on smithi183 2024-09-12T02:59:54.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:54 smithi177 ceph-mon[35305]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:59:54.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:54 smithi116 ceph-mon[31410]: Deploying daemon osd.11 on smithi183 2024-09-12T02:59:54.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:54 smithi116 ceph-mon[31410]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:59:56.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:56 smithi183 ceph-mon[35535]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:59:56.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:56 smithi177 ceph-mon[35305]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:59:56.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:56 smithi116 ceph-mon[31410]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:59:58.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:58 smithi183 ceph-mon[35535]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:59:58.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:58 smithi177 ceph-mon[35305]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:59:58.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:58 smithi116 ceph-mon[31410]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:59:59.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:59 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:59.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:59 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:59.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 02:59:59 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:59.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:59 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:59.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:59 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:59.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 02:59:59 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:59.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:59 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:59.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:59 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T02:59:59.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 02:59:59 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:00.337 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:00 smithi183 ceph-mon[35535]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:00:00.337 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:00 smithi183 ceph-mon[35535]: overall HEALTH_OK 2024-09-12T03:00:00.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:00 smithi177 ceph-mon[35305]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:00:00.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:00 smithi177 ceph-mon[35305]: overall HEALTH_OK 2024-09-12T03:00:00.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:00 smithi116 ceph-mon[31410]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:00:00.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:00 smithi116 ceph-mon[31410]: overall HEALTH_OK 2024-09-12T03:00:01.954 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:01.953+0000 7f8bfb7fe640 1 -- 172.21.15.183:0/1366549094 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f8c0c062630 con 0x7f8bf40a2b80 2024-09-12T03:00:01.954 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 11 on host 'smithi183' 2024-09-12T03:00:01.956 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:01.956+0000 7f8c10b3f640 1 -- 172.21.15.183:0/1366549094 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8bf40a2b80 msgr2=0x7f8bf40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:01.956 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:01.956+0000 7f8c10b3f640 1 --2- 172.21.15.183:0/1366549094 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8bf40a2b80 0x7f8bf40a5040 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc005ab0 tx=0x7f8bfc000bc0 comp rx=0 tx=0).stop 2024-09-12T03:00:01.957 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:01.956+0000 7f8c10b3f640 1 -- 172.21.15.183:0/1366549094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c0c1026d0 msgr2=0x7f8c0c19f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:01.957 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:01.956+0000 7f8c10b3f640 1 --2- 172.21.15.183:0/1366549094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c0c1026d0 0x7f8c0c19f2a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8c0400eaa0 tx=0x7f8c0400ef60 comp rx=0 tx=0).stop 2024-09-12T03:00:01.957 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:01.956+0000 7f8c10b3f640 1 -- 172.21.15.183:0/1366549094 shutdown_connections 2024-09-12T03:00:01.957 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:01.956+0000 7f8c10b3f640 1 --2- 172.21.15.183:0/1366549094 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8bf40a2b80 0x7f8bf40a5040 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:01.957 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:01.956+0000 7f8c10b3f640 1 --2- 172.21.15.183:0/1366549094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c0c1026d0 0x7f8c0c19f2a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:01.957 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:01.956+0000 7f8c10b3f640 1 --2- 172.21.15.183:0/1366549094 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c0c101d10 0x7f8c0c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:01.957 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:01.956+0000 7f8c10b3f640 1 --2- 172.21.15.183:0/1366549094 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c0c100b10 0x7f8c0c06b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:01.957 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:01.956+0000 7f8c10b3f640 1 -- 172.21.15.183:0/1366549094 >> 172.21.15.183:0/1366549094 conn(0x7f8c0c0fc2a0 msgr2=0x7f8c0c1050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:01.957 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:01.956+0000 7f8c10b3f640 1 -- 172.21.15.183:0/1366549094 shutdown_connections 2024-09-12T03:00:01.958 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:01.956+0000 7f8c10b3f640 1 -- 172.21.15.183:0/1366549094 wait complete. 2024-09-12T03:00:02.183 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:02 smithi183 ceph-mon[35535]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:00:02.183 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:02 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:02.183 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:02 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:02.183 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:02 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:02.183 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:02 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:02.183 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:02 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:02.184 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:02 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:00:02.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:02 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:02.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:02 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:02.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:02 smithi177 ceph-mon[35305]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:00:02.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:02 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:02.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:02 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:02.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:02 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:02.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:02 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:02.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:02 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:02.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:02 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:00:02.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:02 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:02.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:02 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:02.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:02 smithi116 ceph-mon[31410]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:00:02.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:02 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:02.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:02 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:02.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:02 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:02.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:02 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:02.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:02 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:02.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:02 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:00:02.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:02 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:02.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:02 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:02.601 DEBUG:teuthology.orchestra.run.smithi183:osd.11> sudo journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.11.service 2024-09-12T03:00:02.604 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-12T03:00:02.604 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T03:00:02.868 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:03.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.341+0000 7f9f01c50640 1 -- 172.21.15.116:0/1664952175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9efc104f90 msgr2=0x7f9efc107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:03.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.341+0000 7f9f01c50640 1 --2- 172.21.15.116:0/1664952175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9efc104f90 0x7f9efc107380 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4009900 tx=0x7f9ee402a410 comp rx=0 tx=0).stop 2024-09-12T03:00:03.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.341+0000 7f9f01c50640 1 -- 172.21.15.116:0/1664952175 shutdown_connections 2024-09-12T03:00:03.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.341+0000 7f9f01c50640 1 --2- 172.21.15.116:0/1664952175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9efc1078c0 0x7f9efc109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:03.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.341+0000 7f9f01c50640 1 --2- 172.21.15.116:0/1664952175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9efc104f90 0x7f9efc107380 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:03.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.341+0000 7f9f01c50640 1 --2- 172.21.15.116:0/1664952175 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9efc102660 0x7f9efc104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:03.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.341+0000 7f9f01c50640 1 -- 172.21.15.116:0/1664952175 >> 172.21.15.116:0/1664952175 conn(0x7f9efc0fc280 msgr2=0x7f9efc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:03.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.341+0000 7f9f01c50640 1 -- 172.21.15.116:0/1664952175 shutdown_connections 2024-09-12T03:00:03.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.341+0000 7f9f01c50640 1 -- 172.21.15.116:0/1664952175 wait complete. 2024-09-12T03:00:03.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.342+0000 7f9f01c50640 1 Processor -- start 2024-09-12T03:00:03.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.342+0000 7f9f01c50640 1 -- start start 2024-09-12T03:00:03.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.342+0000 7f9f01c50640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9efc102660 0x7f9efc197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:03.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.342+0000 7f9f01c50640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9efc104f90 0x7f9efc198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:03.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.342+0000 7f9f01c50640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9efc1078c0 0x7f9efc19f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:03.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.342+0000 7f9f01c50640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc10d840 con 0x7f9efc104f90 2024-09-12T03:00:03.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.342+0000 7f9f01c50640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc10d6c0 con 0x7f9efc1078c0 2024-09-12T03:00:03.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.342+0000 7f9f01c50640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc10d9c0 con 0x7f9efc102660 2024-09-12T03:00:03.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.342+0000 7f9ef3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9efc104f90 0x7f9efc198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:03.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.342+0000 7f9ef3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9efc104f90 0x7f9efc198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40522/0 (socket says 172.21.15.116:40522) 2024-09-12T03:00:03.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.342+0000 7f9ef3fff640 1 -- 172.21.15.116:0/2879687721 learned_addr learned my addr 172.21.15.116:0/2879687721 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:00:03.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.343+0000 7f9efbfff640 1 --2- 172.21.15.116:0/2879687721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9efc1078c0 0x7f9efc19f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:03.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.343+0000 7f9efb7fe640 1 --2- 172.21.15.116:0/2879687721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9efc102660 0x7f9efc197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:03.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.343+0000 7f9ef3fff640 1 -- 172.21.15.116:0/2879687721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9efc102660 msgr2=0x7f9efc197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:03.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.343+0000 7f9ef3fff640 1 --2- 172.21.15.116:0/2879687721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9efc102660 0x7f9efc197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:03.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.343+0000 7f9ef3fff640 1 -- 172.21.15.116:0/2879687721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9efc1078c0 msgr2=0x7f9efc19f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:03.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.343+0000 7f9ef3fff640 1 --2- 172.21.15.116:0/2879687721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9efc1078c0 0x7f9efc19f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:03.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.343+0000 7f9ef3fff640 1 -- 172.21.15.116:0/2879687721 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9efc19fb50 con 0x7f9efc104f90 2024-09-12T03:00:03.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.343+0000 7f9ef3fff640 1 --2- 172.21.15.116:0/2879687721 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9efc104f90 0x7f9efc198460 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4031610 tx=0x7f9ee40043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:03.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.343+0000 7f9ef97fa640 1 -- 172.21.15.116:0/2879687721 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee403f030 con 0x7f9efc104f90 2024-09-12T03:00:03.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.343+0000 7f9ef97fa640 1 -- 172.21.15.116:0/2879687721 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ee4003ce0 con 0x7f9efc104f90 2024-09-12T03:00:03.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.343+0000 7f9f01c50640 1 -- 172.21.15.116:0/2879687721 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9efc19fde0 con 0x7f9efc104f90 2024-09-12T03:00:03.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.343+0000 7f9ef97fa640 1 -- 172.21.15.116:0/2879687721 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4035070 con 0x7f9efc104f90 2024-09-12T03:00:03.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.343+0000 7f9f01c50640 1 -- 172.21.15.116:0/2879687721 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9efc1a0260 con 0x7f9efc104f90 2024-09-12T03:00:03.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.344+0000 7f9f01c50640 1 -- 172.21.15.116:0/2879687721 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9efc0fdd70 con 0x7f9efc104f90 2024-09-12T03:00:03.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.345+0000 7f9ef97fa640 1 -- 172.21.15.116:0/2879687721 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f9ee4042030 con 0x7f9efc104f90 2024-09-12T03:00:03.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.346+0000 7f9ef97fa640 1 --2- 172.21.15.116:0/2879687721 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9ee80a2bb0 0x7f9ee80a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:03.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.346+0000 7f9efb7fe640 1 --2- 172.21.15.116:0/2879687721 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9ee80a2bb0 0x7f9ee80a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:03.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.346+0000 7f9ef97fa640 1 -- 172.21.15.116:0/2879687721 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6809+0+0 (secure 0 0 0) 0x7f9ee40ef340 con 0x7f9efc104f90 2024-09-12T03:00:03.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.347+0000 7f9efb7fe640 1 --2- 172.21.15.116:0/2879687721 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9ee80a2bb0 0x7f9ee80a5070 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0058c0 tx=0x7f9eec00a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:03.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.350+0000 7f9ef97fa640 1 -- 172.21.15.116:0/2879687721 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9ee40bc7e0 con 0x7f9efc104f90 2024-09-12T03:00:03.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.522+0000 7f9f01c50640 1 -- 172.21.15.116:0/2879687721 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f9efc02da10 con 0x7f9efc104f90 2024-09-12T03:00:03.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.523+0000 7f9ef97fa640 1 -- 172.21.15.116:0/2879687721 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v69) ==== 74+0+133 (secure 0 0 0) 0x7f9ee40bc0e0 con 0x7f9efc104f90 2024-09-12T03:00:03.524 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:00:03.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.525+0000 7f9f01c50640 1 -- 172.21.15.116:0/2879687721 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9ee80a2bb0 msgr2=0x7f9ee80a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:03.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.525+0000 7f9f01c50640 1 --2- 172.21.15.116:0/2879687721 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9ee80a2bb0 0x7f9ee80a5070 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0058c0 tx=0x7f9eec00a4c0 comp rx=0 tx=0).stop 2024-09-12T03:00:03.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.525+0000 7f9f01c50640 1 -- 172.21.15.116:0/2879687721 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9efc104f90 msgr2=0x7f9efc198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:03.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.525+0000 7f9f01c50640 1 --2- 172.21.15.116:0/2879687721 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9efc104f90 0x7f9efc198460 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4031610 tx=0x7f9ee40043b0 comp rx=0 tx=0).stop 2024-09-12T03:00:03.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.525+0000 7f9f01c50640 1 -- 172.21.15.116:0/2879687721 shutdown_connections 2024-09-12T03:00:03.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.525+0000 7f9f01c50640 1 --2- 172.21.15.116:0/2879687721 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9ee80a2bb0 0x7f9ee80a5070 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:03.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.525+0000 7f9f01c50640 1 --2- 172.21.15.116:0/2879687721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9efc1078c0 0x7f9efc19f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:03.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.525+0000 7f9f01c50640 1 --2- 172.21.15.116:0/2879687721 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9efc104f90 0x7f9efc198460 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:03.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.525+0000 7f9f01c50640 1 --2- 172.21.15.116:0/2879687721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9efc102660 0x7f9efc197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:03.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.525+0000 7f9f01c50640 1 -- 172.21.15.116:0/2879687721 >> 172.21.15.116:0/2879687721 conn(0x7f9efc0fc280 msgr2=0x7f9efc1030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:03.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.525+0000 7f9f01c50640 1 -- 172.21.15.116:0/2879687721 shutdown_connections 2024-09-12T03:00:03.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:03.525+0000 7f9f01c50640 1 -- 172.21.15.116:0/2879687721 wait complete. 2024-09-12T03:00:04.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:04 smithi116 ceph-mon[31410]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:00:04.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:04 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2879687721' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:00:04.357 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":69,"num_osds":12,"num_up_osds":11,"osd_up_since":1726109981,"num_in_osds":12,"osd_in_since":1726109983,"num_remapped_pgs":0} 2024-09-12T03:00:04.366 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:04 smithi183 ceph-mon[35535]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:00:04.366 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:04 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2879687721' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:00:04.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:04 smithi177 ceph-mon[35305]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:00:04.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:04 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2879687721' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:00:04.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:00:04 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:00:04.364+0000 7f86218a3640 -1 osd.11 0 log_to_monitors true 2024-09-12T03:00:05.358 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T03:00:05.382 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:05 smithi183 ceph-mon[35535]: from='osd.11 [v2:172.21.15.183:6824/1818319939,v1:172.21.15.183:6825/1818319939]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:00:05.382 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:05 smithi183 ceph-mon[35535]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:00:05.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:05 smithi177 ceph-mon[35305]: from='osd.11 [v2:172.21.15.183:6824/1818319939,v1:172.21.15.183:6825/1818319939]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:00:05.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:05 smithi177 ceph-mon[35305]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:00:05.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:05 smithi116 ceph-mon[31410]: from='osd.11 [v2:172.21.15.183:6824/1818319939,v1:172.21.15.183:6825/1818319939]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:00:05.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:05 smithi116 ceph-mon[31410]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:00:05.623 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:06.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.131+0000 7f1ef2dee640 1 -- 172.21.15.116:0/3563505519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1eec076220 msgr2=0x7f1eec074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:06.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.131+0000 7f1ef2dee640 1 --2- 172.21.15.116:0/3563505519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1eec076220 0x7f1eec074680 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0009900 tx=0x7f1ee002a410 comp rx=0 tx=0).stop 2024-09-12T03:00:06.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.132+0000 7f1ef2dee640 1 -- 172.21.15.116:0/3563505519 shutdown_connections 2024-09-12T03:00:06.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.132+0000 7f1ef2dee640 1 --2- 172.21.15.116:0/3563505519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1eec10e1e0 0x7f1eec1105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.132+0000 7f1ef2dee640 1 --2- 172.21.15.116:0/3563505519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eec074c50 0x7f1eec0750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.132+0000 7f1ef2dee640 1 --2- 172.21.15.116:0/3563505519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1eec076220 0x7f1eec074680 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.132+0000 7f1ef2dee640 1 -- 172.21.15.116:0/3563505519 >> 172.21.15.116:0/3563505519 conn(0x7f1eec0fd390 msgr2=0x7f1eec0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:06.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.132+0000 7f1ef2dee640 1 -- 172.21.15.116:0/3563505519 shutdown_connections 2024-09-12T03:00:06.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.132+0000 7f1ef2dee640 1 -- 172.21.15.116:0/3563505519 wait complete. 2024-09-12T03:00:06.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.132+0000 7f1ef2dee640 1 Processor -- start 2024-09-12T03:00:06.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.132+0000 7f1ef2dee640 1 -- start start 2024-09-12T03:00:06.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.132+0000 7f1ef2dee640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1eec074c50 0x7f1eec19e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:06.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.132+0000 7f1ef2dee640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eec076220 0x7f1eec19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:06.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.132+0000 7f1ef2dee640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1eec10e1e0 0x7f1eec1a5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:06.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.132+0000 7f1ef2dee640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1eec113f60 con 0x7f1eec10e1e0 2024-09-12T03:00:06.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.132+0000 7f1ef2dee640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1eec113de0 con 0x7f1eec074c50 2024-09-12T03:00:06.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.132+0000 7f1ef2dee640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1eec1140e0 con 0x7f1eec076220 2024-09-12T03:00:06.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.132+0000 7f1ef138b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1eec10e1e0 0x7f1eec1a5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:06.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.132+0000 7f1ef138b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1eec10e1e0 0x7f1eec1a5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40536/0 (socket says 172.21.15.116:40536) 2024-09-12T03:00:06.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.132+0000 7f1ef138b640 1 -- 172.21.15.116:0/3732517599 learned_addr learned my addr 172.21.15.116:0/3732517599 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:00:06.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.133+0000 7f1eebfff640 1 --2- 172.21.15.116:0/3732517599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eec076220 0x7f1eec19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:06.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.133+0000 7f1ef138b640 1 -- 172.21.15.116:0/3732517599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eec076220 msgr2=0x7f1eec19ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:06.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.133+0000 7f1ef138b640 1 --2- 172.21.15.116:0/3732517599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eec076220 0x7f1eec19ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.133+0000 7f1ef138b640 1 -- 172.21.15.116:0/3732517599 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1eec074c50 msgr2=0x7f1eec19e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:06.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.133+0000 7f1ef0b8a640 1 --2- 172.21.15.116:0/3732517599 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1eec074c50 0x7f1eec19e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:06.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.133+0000 7f1ef138b640 1 --2- 172.21.15.116:0/3732517599 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1eec074c50 0x7f1eec19e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.133+0000 7f1ef138b640 1 -- 172.21.15.116:0/3732517599 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eec1a62a0 con 0x7f1eec10e1e0 2024-09-12T03:00:06.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.133+0000 7f1ef138b640 1 --2- 172.21.15.116:0/3732517599 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1eec10e1e0 0x7f1eec1a5c30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1ee400b970 tx=0x7f1ee400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:06.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.133+0000 7f1ee9ffb640 1 -- 172.21.15.116:0/3732517599 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4019070 con 0x7f1eec10e1e0 2024-09-12T03:00:06.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.133+0000 7f1ef2dee640 1 -- 172.21.15.116:0/3732517599 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eec1a6590 con 0x7f1eec10e1e0 2024-09-12T03:00:06.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.133+0000 7f1ee9ffb640 1 -- 172.21.15.116:0/3732517599 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ee400cb40 con 0x7f1eec10e1e0 2024-09-12T03:00:06.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.133+0000 7f1ee9ffb640 1 -- 172.21.15.116:0/3732517599 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4007650 con 0x7f1eec10e1e0 2024-09-12T03:00:06.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.133+0000 7f1ef2dee640 1 -- 172.21.15.116:0/3732517599 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eec1a6b50 con 0x7f1eec10e1e0 2024-09-12T03:00:06.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.134+0000 7f1ef2dee640 1 -- 172.21.15.116:0/3732517599 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eec103b80 con 0x7f1eec10e1e0 2024-09-12T03:00:06.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.135+0000 7f1ee9ffb640 1 -- 172.21.15.116:0/3732517599 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1ee401c070 con 0x7f1eec10e1e0 2024-09-12T03:00:06.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.136+0000 7f1ee9ffb640 1 --2- 172.21.15.116:0/3732517599 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1edc0a2bb0 0x7f1edc0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:06.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.136+0000 7f1ef0b8a640 1 --2- 172.21.15.116:0/3732517599 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1edc0a2bb0 0x7f1edc0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:06.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.136+0000 7f1ee9ffb640 1 -- 172.21.15.116:0/3732517599 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6831+0+0 (secure 0 0 0) 0x7f1ee40ce0c0 con 0x7f1eec10e1e0 2024-09-12T03:00:06.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.136+0000 7f1ef0b8a640 1 --2- 172.21.15.116:0/3732517599 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1edc0a2bb0 0x7f1edc0a5070 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0031610 tx=0x7f1ee0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:06.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.140+0000 7f1ee9ffb640 1 -- 172.21.15.116:0/3732517599 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1ee409b5f0 con 0x7f1eec10e1e0 2024-09-12T03:00:06.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:06 smithi116 ceph-mon[31410]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:00:06.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:06 smithi116 ceph-mon[31410]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T03:00:06.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:06 smithi116 ceph-mon[31410]: osdmap e70: 12 total, 11 up, 12 in 2024-09-12T03:00:06.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:06 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:00:06.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:06 smithi116 ceph-mon[31410]: from='osd.11 [v2:172.21.15.183:6824/1818319939,v1:172.21.15.183:6825/1818319939]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T03:00:06.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:06 smithi116 ceph-mon[31410]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T03:00:06.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:06 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:06.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:06 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:06.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:06 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:06.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:06 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:06.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:06 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:06.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:06 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:06.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:06 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:06.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:06 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:06.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:06 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:06.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:06 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:06.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.310+0000 7f1ef2dee640 1 -- 172.21.15.116:0/3732517599 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f1eec062df0 con 0x7f1eec10e1e0 2024-09-12T03:00:06.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.311+0000 7f1ee9ffb640 1 -- 172.21.15.116:0/3732517599 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7f1ee409aef0 con 0x7f1eec10e1e0 2024-09-12T03:00:06.311 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:00:06.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.313+0000 7f1ef2dee640 1 -- 172.21.15.116:0/3732517599 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1edc0a2bb0 msgr2=0x7f1edc0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:06.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.313+0000 7f1ef2dee640 1 --2- 172.21.15.116:0/3732517599 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1edc0a2bb0 0x7f1edc0a5070 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0031610 tx=0x7f1ee0000bc0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.313+0000 7f1ef2dee640 1 -- 172.21.15.116:0/3732517599 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1eec10e1e0 msgr2=0x7f1eec1a5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:06.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.313+0000 7f1ef2dee640 1 --2- 172.21.15.116:0/3732517599 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1eec10e1e0 0x7f1eec1a5c30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1ee400b970 tx=0x7f1ee400be30 comp rx=0 tx=0).stop 2024-09-12T03:00:06.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.313+0000 7f1ef2dee640 1 -- 172.21.15.116:0/3732517599 shutdown_connections 2024-09-12T03:00:06.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.313+0000 7f1ef2dee640 1 --2- 172.21.15.116:0/3732517599 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1edc0a2bb0 0x7f1edc0a5070 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.313+0000 7f1ef2dee640 1 --2- 172.21.15.116:0/3732517599 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1eec10e1e0 0x7f1eec1a5c30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.313+0000 7f1ef2dee640 1 --2- 172.21.15.116:0/3732517599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eec076220 0x7f1eec19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.313+0000 7f1ef2dee640 1 --2- 172.21.15.116:0/3732517599 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1eec074c50 0x7f1eec19e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:06.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.313+0000 7f1ef2dee640 1 -- 172.21.15.116:0/3732517599 >> 172.21.15.116:0/3732517599 conn(0x7f1eec0fd390 msgr2=0x7f1eec0ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:06.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.313+0000 7f1ef2dee640 1 -- 172.21.15.116:0/3732517599 shutdown_connections 2024-09-12T03:00:06.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:06.313+0000 7f1ef2dee640 1 -- 172.21.15.116:0/3732517599 wait complete. 2024-09-12T03:00:06.453 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:00:06 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:00:06.201+0000 7f861d826640 -1 osd.11 0 waiting for initial osdmap 2024-09-12T03:00:06.453 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:00:06 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:00:06.216+0000 7f861964e640 -1 osd.11 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:00:06.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:06 smithi183 ceph-mon[35535]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:00:06.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:06 smithi183 ceph-mon[35535]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T03:00:06.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:06 smithi183 ceph-mon[35535]: osdmap e70: 12 total, 11 up, 12 in 2024-09-12T03:00:06.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:06 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:00:06.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:06 smithi183 ceph-mon[35535]: from='osd.11 [v2:172.21.15.183:6824/1818319939,v1:172.21.15.183:6825/1818319939]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T03:00:06.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:06 smithi183 ceph-mon[35535]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T03:00:06.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:06 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:06.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:06 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:06.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:06 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:06.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:06 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:06.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:06 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:06.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:06 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:06.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:06 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:06.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:06 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:06.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:06 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:06.455 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:06 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:06.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:06 smithi177 ceph-mon[35305]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:00:06.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:06 smithi177 ceph-mon[35305]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T03:00:06.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:06 smithi177 ceph-mon[35305]: osdmap e70: 12 total, 11 up, 12 in 2024-09-12T03:00:06.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:06 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:00:06.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:06 smithi177 ceph-mon[35305]: from='osd.11 [v2:172.21.15.183:6824/1818319939,v1:172.21.15.183:6825/1818319939]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T03:00:06.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:06 smithi177 ceph-mon[35305]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-12T03:00:06.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:06 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:06.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:06 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:06.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:06 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:06.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:06 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:06.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:06 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:06.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:06 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:06.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:06 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:06.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:06 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:06.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:06 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:06.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:06 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:06.998 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":71,"num_osds":12,"num_up_osds":11,"osd_up_since":1726109981,"num_in_osds":12,"osd_in_since":1726109983,"num_remapped_pgs":0} 2024-09-12T03:00:07.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:07 smithi177 ceph-mon[35305]: Detected new or changed devices on smithi183 2024-09-12T03:00:07.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:07 smithi177 ceph-mon[35305]: Adjusting osd_memory_target on smithi183 to 4272M 2024-09-12T03:00:07.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:07 smithi177 ceph-mon[35305]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-12T03:00:07.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:07 smithi177 ceph-mon[35305]: osdmap e71: 12 total, 11 up, 12 in 2024-09-12T03:00:07.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:07 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:00:07.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:07 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:00:07.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:07 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3732517599' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:00:07.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:07 smithi116 ceph-mon[31410]: Detected new or changed devices on smithi183 2024-09-12T03:00:07.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:07 smithi116 ceph-mon[31410]: Adjusting osd_memory_target on smithi183 to 4272M 2024-09-12T03:00:07.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:07 smithi116 ceph-mon[31410]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-12T03:00:07.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:07 smithi116 ceph-mon[31410]: osdmap e71: 12 total, 11 up, 12 in 2024-09-12T03:00:07.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:07 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:00:07.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:07 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:00:07.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:07 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3732517599' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:00:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:07 smithi183 ceph-mon[35535]: Detected new or changed devices on smithi183 2024-09-12T03:00:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:07 smithi183 ceph-mon[35535]: Adjusting osd_memory_target on smithi183 to 4272M 2024-09-12T03:00:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:07 smithi183 ceph-mon[35535]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-12T03:00:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:07 smithi183 ceph-mon[35535]: osdmap e71: 12 total, 11 up, 12 in 2024-09-12T03:00:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:07 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:00:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:07 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:00:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:07 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3732517599' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:00:08.000 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T03:00:08.258 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:08.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:08 smithi116 ceph-mon[31410]: purged_snaps scrub starts 2024-09-12T03:00:08.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:08 smithi116 ceph-mon[31410]: purged_snaps scrub ok 2024-09-12T03:00:08.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:08 smithi116 ceph-mon[31410]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:00:08.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:08 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:00:08.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:08 smithi116 ceph-mon[31410]: osd.11 [v2:172.21.15.183:6824/1818319939,v1:172.21.15.183:6825/1818319939] boot 2024-09-12T03:00:08.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:08 smithi116 ceph-mon[31410]: osdmap e72: 12 total, 12 up, 12 in 2024-09-12T03:00:08.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:08 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:00:08.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:08 smithi177 ceph-mon[35305]: purged_snaps scrub starts 2024-09-12T03:00:08.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:08 smithi177 ceph-mon[35305]: purged_snaps scrub ok 2024-09-12T03:00:08.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:08 smithi177 ceph-mon[35305]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:00:08.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:08 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:00:08.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:08 smithi177 ceph-mon[35305]: osd.11 [v2:172.21.15.183:6824/1818319939,v1:172.21.15.183:6825/1818319939] boot 2024-09-12T03:00:08.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:08 smithi177 ceph-mon[35305]: osdmap e72: 12 total, 12 up, 12 in 2024-09-12T03:00:08.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:08 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:00:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:08 smithi183 ceph-mon[35535]: purged_snaps scrub starts 2024-09-12T03:00:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:08 smithi183 ceph-mon[35535]: purged_snaps scrub ok 2024-09-12T03:00:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:08 smithi183 ceph-mon[35535]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:00:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:08 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:00:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:08 smithi183 ceph-mon[35535]: osd.11 [v2:172.21.15.183:6824/1818319939,v1:172.21.15.183:6825/1818319939] boot 2024-09-12T03:00:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:08 smithi183 ceph-mon[35535]: osdmap e72: 12 total, 12 up, 12 in 2024-09-12T03:00:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:08 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:00:08.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.727+0000 7f185429f640 1 -- 172.21.15.116:0/2948647234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f184c107900 msgr2=0x7f184c109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:08.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.727+0000 7f185429f640 1 --2- 172.21.15.116:0/2948647234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f184c107900 0x7f184c109d50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1844009ca0 tx=0x7f184402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:00:08.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.728+0000 7f185429f640 1 -- 172.21.15.116:0/2948647234 shutdown_connections 2024-09-12T03:00:08.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.728+0000 7f185429f640 1 --2- 172.21.15.116:0/2948647234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f184c107900 0x7f184c109d50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:08.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.728+0000 7f185429f640 1 --2- 172.21.15.116:0/2948647234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f184c104fd0 0x7f184c1073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:08.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.728+0000 7f185429f640 1 --2- 172.21.15.116:0/2948647234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f184c1026a0 0x7f184c104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:08.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.728+0000 7f185429f640 1 -- 172.21.15.116:0/2948647234 >> 172.21.15.116:0/2948647234 conn(0x7f184c0fc2a0 msgr2=0x7f184c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:08.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.728+0000 7f185429f640 1 -- 172.21.15.116:0/2948647234 shutdown_connections 2024-09-12T03:00:08.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.728+0000 7f185429f640 1 -- 172.21.15.116:0/2948647234 wait complete. 2024-09-12T03:00:08.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.728+0000 7f185429f640 1 Processor -- start 2024-09-12T03:00:08.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.728+0000 7f185429f640 1 -- start start 2024-09-12T03:00:08.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.728+0000 7f185429f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f184c1026a0 0x7f184c102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:08.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.728+0000 7f185429f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f184c104fd0 0x7f184c100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:08.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.728+0000 7f185429f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f184c107900 0x7f184c100b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:08.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.728+0000 7f185429f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f184c10d8a0 con 0x7f184c1026a0 2024-09-12T03:00:08.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.728+0000 7f185429f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f184c10d720 con 0x7f184c107900 2024-09-12T03:00:08.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.728+0000 7f185429f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f184c10da20 con 0x7f184c104fd0 2024-09-12T03:00:08.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.728+0000 7f185203b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f184c1026a0 0x7f184c102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:08.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.729+0000 7f185203b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f184c1026a0 0x7f184c102000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40556/0 (socket says 172.21.15.116:40556) 2024-09-12T03:00:08.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.729+0000 7f185203b640 1 -- 172.21.15.116:0/212770839 learned_addr learned my addr 172.21.15.116:0/212770839 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:00:08.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.729+0000 7f185203b640 1 -- 172.21.15.116:0/212770839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f184c104fd0 msgr2=0x7f184c100650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:08.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.729+0000 7f185203b640 1 --2- 172.21.15.116:0/212770839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f184c104fd0 0x7f184c100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:08.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.729+0000 7f185203b640 1 -- 172.21.15.116:0/212770839 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f184c107900 msgr2=0x7f184c100b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:08.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.729+0000 7f185203b640 1 --2- 172.21.15.116:0/212770839 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f184c107900 0x7f184c100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:08.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.729+0000 7f185203b640 1 -- 172.21.15.116:0/212770839 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184c101450 con 0x7f184c1026a0 2024-09-12T03:00:08.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.729+0000 7f185203b640 1 --2- 172.21.15.116:0/212770839 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f184c1026a0 0x7f184c102000 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f184800d950 tx=0x7f184800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:08.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.729+0000 7f18437fe640 1 -- 172.21.15.116:0/212770839 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848017070 con 0x7f184c1026a0 2024-09-12T03:00:08.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.729+0000 7f18437fe640 1 -- 172.21.15.116:0/212770839 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1848009c30 con 0x7f184c1026a0 2024-09-12T03:00:08.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.729+0000 7f185429f640 1 -- 172.21.15.116:0/212770839 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f184c1a1fd0 con 0x7f184c1026a0 2024-09-12T03:00:08.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.729+0000 7f18437fe640 1 -- 172.21.15.116:0/212770839 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184800f6b0 con 0x7f184c1026a0 2024-09-12T03:00:08.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.729+0000 7f185429f640 1 -- 172.21.15.116:0/212770839 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f184c1a24d0 con 0x7f184c1026a0 2024-09-12T03:00:08.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.730+0000 7f185429f640 1 -- 172.21.15.116:0/212770839 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f184c0fdd90 con 0x7f184c1026a0 2024-09-12T03:00:08.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.731+0000 7f18437fe640 1 -- 172.21.15.116:0/212770839 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f18480109d0 con 0x7f184c1026a0 2024-09-12T03:00:08.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.732+0000 7f18437fe640 1 --2- 172.21.15.116:0/212770839 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f18380a2c00 0x7f18380a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:08.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.732+0000 7f185183a640 1 --2- 172.21.15.116:0/212770839 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f18380a2c00 0x7f18380a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:08.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.732+0000 7f18437fe640 1 -- 172.21.15.116:0/212770839 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f18480ce7a0 con 0x7f184c1026a0 2024-09-12T03:00:08.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.732+0000 7f185183a640 1 --2- 172.21.15.116:0/212770839 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f18380a2c00 0x7f18380a50c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f184c101d80 tx=0x7f183c005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:08.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.735+0000 7f18437fe640 1 -- 172.21.15.116:0/212770839 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1848009de0 con 0x7f184c1026a0 2024-09-12T03:00:08.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.903+0000 7f185429f640 1 -- 172.21.15.116:0/212770839 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f184c02da00 con 0x7f184c1026a0 2024-09-12T03:00:08.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.903+0000 7f18437fe640 1 -- 172.21.15.116:0/212770839 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7f184809bc20 con 0x7f184c1026a0 2024-09-12T03:00:08.904 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:00:08.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.905+0000 7f185429f640 1 -- 172.21.15.116:0/212770839 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f18380a2c00 msgr2=0x7f18380a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:08.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.905+0000 7f185429f640 1 --2- 172.21.15.116:0/212770839 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f18380a2c00 0x7f18380a50c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f184c101d80 tx=0x7f183c005c70 comp rx=0 tx=0).stop 2024-09-12T03:00:08.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.905+0000 7f185429f640 1 -- 172.21.15.116:0/212770839 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f184c1026a0 msgr2=0x7f184c102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:08.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.905+0000 7f185429f640 1 --2- 172.21.15.116:0/212770839 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f184c1026a0 0x7f184c102000 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f184800d950 tx=0x7f184800de10 comp rx=0 tx=0).stop 2024-09-12T03:00:08.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.905+0000 7f185429f640 1 -- 172.21.15.116:0/212770839 shutdown_connections 2024-09-12T03:00:08.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.905+0000 7f185429f640 1 --2- 172.21.15.116:0/212770839 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f18380a2c00 0x7f18380a50c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:08.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.905+0000 7f185429f640 1 --2- 172.21.15.116:0/212770839 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f184c107900 0x7f184c100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:08.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.905+0000 7f185429f640 1 --2- 172.21.15.116:0/212770839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f184c104fd0 0x7f184c100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:08.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.905+0000 7f185429f640 1 --2- 172.21.15.116:0/212770839 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f184c1026a0 0x7f184c102000 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:08.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.905+0000 7f185429f640 1 -- 172.21.15.116:0/212770839 >> 172.21.15.116:0/212770839 conn(0x7f184c0fc2a0 msgr2=0x7f184c102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:08.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.906+0000 7f185429f640 1 -- 172.21.15.116:0/212770839 shutdown_connections 2024-09-12T03:00:08.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:08.906+0000 7f185429f640 1 -- 172.21.15.116:0/212770839 wait complete. 2024-09-12T03:00:09.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:09 smithi177 ceph-mon[35305]: osdmap e73: 12 total, 12 up, 12 in 2024-09-12T03:00:09.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:09 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/212770839' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:00:09.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:09 smithi116 ceph-mon[31410]: osdmap e73: 12 total, 12 up, 12 in 2024-09-12T03:00:09.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:09 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/212770839' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:00:09.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:09 smithi183 ceph-mon[35535]: osdmap e73: 12 total, 12 up, 12 in 2024-09-12T03:00:09.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:09 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/212770839' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:00:09.761 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":73,"num_osds":12,"num_up_osds":12,"osd_up_since":1726110007,"num_in_osds":12,"osd_in_since":1726109983,"num_remapped_pgs":0} 2024-09-12T03:00:09.761 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:00:10.004 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:10.400 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:10 smithi116 ceph-mon[31410]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:10.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:10 smithi177 ceph-mon[35305]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:10.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.479+0000 7fa55c753640 1 -- 172.21.15.116:0/3498983663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa554101de0 msgr2=0x7fa5541021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:10.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.479+0000 7fa55c753640 1 --2- 172.21.15.116:0/3498983663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa554101de0 0x7fa5541021c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa544009900 tx=0x7fa54402a410 comp rx=0 tx=0).stop 2024-09-12T03:00:10.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.480+0000 7fa55c753640 1 -- 172.21.15.116:0/3498983663 shutdown_connections 2024-09-12T03:00:10.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.480+0000 7fa55c753640 1 --2- 172.21.15.116:0/3498983663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa554106f10 0x7fa55410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:10.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.480+0000 7fa55c753640 1 --2- 172.21.15.116:0/3498983663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa554102790 0x7fa5541067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:10.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.480+0000 7fa55c753640 1 --2- 172.21.15.116:0/3498983663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa554101de0 0x7fa5541021c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:10.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.480+0000 7fa55c753640 1 -- 172.21.15.116:0/3498983663 >> 172.21.15.116:0/3498983663 conn(0x7fa5540fd8c0 msgr2=0x7fa5540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:10.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.480+0000 7fa55c753640 1 -- 172.21.15.116:0/3498983663 shutdown_connections 2024-09-12T03:00:10.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.480+0000 7fa55c753640 1 -- 172.21.15.116:0/3498983663 wait complete. 2024-09-12T03:00:10.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.480+0000 7fa55c753640 1 Processor -- start 2024-09-12T03:00:10.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.480+0000 7fa55c753640 1 -- start start 2024-09-12T03:00:10.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.480+0000 7fa55c753640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa554101de0 0x7fa5541985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:10.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.480+0000 7fa55c753640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa554102790 0x7fa554198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:10.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.480+0000 7fa55c753640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa554106f10 0x7fa55419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:10.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.480+0000 7fa55c753640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa554112010 con 0x7fa554106f10 2024-09-12T03:00:10.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.480+0000 7fa55c753640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa554111e90 con 0x7fa554101de0 2024-09-12T03:00:10.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.480+0000 7fa55c753640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa554112190 con 0x7fa554102790 2024-09-12T03:00:10.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.481+0000 7fa55acf0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa554106f10 0x7fa55419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:10.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.481+0000 7fa55acf0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa554106f10 0x7fa55419ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40574/0 (socket says 172.21.15.116:40574) 2024-09-12T03:00:10.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.481+0000 7fa55acf0640 1 -- 172.21.15.116:0/1079412543 learned_addr learned my addr 172.21.15.116:0/1079412543 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:00:10.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.481+0000 7fa55a4ef640 1 --2- 172.21.15.116:0/1079412543 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa554101de0 0x7fa5541985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:10.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.481+0000 7fa559cee640 1 --2- 172.21.15.116:0/1079412543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa554102790 0x7fa554198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:10.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.481+0000 7fa55acf0640 1 -- 172.21.15.116:0/1079412543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa554102790 msgr2=0x7fa554198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:10.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.481+0000 7fa55acf0640 1 --2- 172.21.15.116:0/1079412543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa554102790 0x7fa554198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:10.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.481+0000 7fa55acf0640 1 -- 172.21.15.116:0/1079412543 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa554101de0 msgr2=0x7fa5541985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:10.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.481+0000 7fa55acf0640 1 --2- 172.21.15.116:0/1079412543 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa554101de0 0x7fa5541985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:10.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.481+0000 7fa55acf0640 1 -- 172.21.15.116:0/1079412543 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55419d560 con 0x7fa554106f10 2024-09-12T03:00:10.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.481+0000 7fa55acf0640 1 --2- 172.21.15.116:0/1079412543 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa554106f10 0x7fa55419ce80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00b970 tx=0x7fa54c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:10.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.481+0000 7fa54b7fe640 1 -- 172.21.15.116:0/1079412543 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c019070 con 0x7fa554106f10 2024-09-12T03:00:10.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.481+0000 7fa55c753640 1 -- 172.21.15.116:0/1079412543 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa55419d850 con 0x7fa554106f10 2024-09-12T03:00:10.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.481+0000 7fa54b7fe640 1 -- 172.21.15.116:0/1079412543 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa54c00cb40 con 0x7fa554106f10 2024-09-12T03:00:10.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.481+0000 7fa54b7fe640 1 -- 172.21.15.116:0/1079412543 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c007610 con 0x7fa554106f10 2024-09-12T03:00:10.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.481+0000 7fa55c753640 1 -- 172.21.15.116:0/1079412543 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa554074420 con 0x7fa554106f10 2024-09-12T03:00:10.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.482+0000 7fa55c753640 1 -- 172.21.15.116:0/1079412543 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa554102240 con 0x7fa554106f10 2024-09-12T03:00:10.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.483+0000 7fa54b7fe640 1 -- 172.21.15.116:0/1079412543 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa54c01c070 con 0x7fa554106f10 2024-09-12T03:00:10.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.484+0000 7fa54b7fe640 1 --2- 172.21.15.116:0/1079412543 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa5400a2b70 0x7fa5400a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:10.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.484+0000 7fa55a4ef640 1 --2- 172.21.15.116:0/1079412543 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa5400a2b70 0x7fa5400a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:10.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.484+0000 7fa54b7fe640 1 -- 172.21.15.116:0/1079412543 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fa54c0ce5e0 con 0x7fa554106f10 2024-09-12T03:00:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.484+0000 7fa55a4ef640 1 --2- 172.21.15.116:0/1079412543 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa5400a2b70 0x7fa5400a5030 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa544000c00 tx=0x7fa5440058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.487+0000 7fa54b7fe640 1 -- 172.21.15.116:0/1079412543 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa54c09b390 con 0x7fa554106f10 2024-09-12T03:00:10.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.655+0000 7fa55c753640 1 -- 172.21.15.116:0/1079412543 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa55410a0f0 con 0x7fa554106f10 2024-09-12T03:00:10.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.656+0000 7fa54b7fe640 1 -- 172.21.15.116:0/1079412543 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+18514 (secure 0 0 0) 0x7fa54c09f690 con 0x7fa554106f10 2024-09-12T03:00:10.656 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:00:10.657 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":73,"fsid":"eb1ac602-70b1-11ef-bcea-c7b262605968","created":"2024-09-12T02:53:07.105686+0000","modified":"2024-09-12T03:00:08.209302+0000","last_up_change":"2024-09-12T03:00:07.202639+0000","last_in_change":"2024-09-12T02:59:43.446336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:56:17.347152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8cf5a18-dc6f-4bb6-bd74-78adda310118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6803","nonce":862366853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6805","nonce":862366853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6809","nonce":862366853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6807","nonce":862366853}]},"public_addr":"172.21.15.116:6803/862366853","cluster_addr":"172.21.15.116:6805/862366853","heartbeat_back_addr":"172.21.15.116:6809/862366853","heartbeat_front_addr":"172.21.15.116:6807/862366853","state":["exists","up"]},{"osd":1,"uuid":"0affd3cb-9a51-42b0-a2cc-f30784a75cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6801","nonce":2245800547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6803","nonce":2245800547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6807","nonce":2245800547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6805","nonce":2245800547}]},"public_addr":"172.21.15.177:6801/2245800547","cluster_addr":"172.21.15.177:6803/2245800547","heartbeat_back_addr":"172.21.15.177:6807/2245800547","heartbeat_front_addr":"172.21.15.177:6805/2245800547","state":["exists","up"]},{"osd":2,"uuid":"1ce45ea5-84df-4a35-924f-78f92bd4f2cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6801","nonce":2645677759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6803","nonce":2645677759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6807","nonce":2645677759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6805","nonce":2645677759}]},"public_addr":"172.21.15.183:6801/2645677759","cluster_addr":"172.21.15.183:6803/2645677759","heartbeat_back_addr":"172.21.15.183:6807/2645677759","heartbeat_front_addr":"172.21.15.183:6805/2645677759","state":["exists","up"]},{"osd":3,"uuid":"ae5e8178-c998-4648-bc05-4bce822f0cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6811","nonce":686622148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6813","nonce":686622148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6817","nonce":686622148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6815","nonce":686622148}]},"public_addr":"172.21.15.116:6811/686622148","cluster_addr":"172.21.15.116:6813/686622148","heartbeat_back_addr":"172.21.15.116:6817/686622148","heartbeat_front_addr":"172.21.15.116:6815/686622148","state":["exists","up"]},{"osd":4,"uuid":"c81d2e73-0bc2-440d-b5a2-b7428af3ea4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6809","nonce":2382938021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6811","nonce":2382938021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6815","nonce":2382938021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6813","nonce":2382938021}]},"public_addr":"172.21.15.177:6809/2382938021","cluster_addr":"172.21.15.177:6811/2382938021","heartbeat_back_addr":"172.21.15.177:6815/2382938021","heartbeat_front_addr":"172.21.15.177:6813/2382938021","state":["exists","up"]},{"osd":5,"uuid":"c3f168be-9e27-4cf5-b73e-3f47d8990ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6809","nonce":85363432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6811","nonce":85363432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6815","nonce":85363432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6813","nonce":85363432}]},"public_addr":"172.21.15.183:6809/85363432","cluster_addr":"172.21.15.183:6811/85363432","heartbeat_back_addr":"172.21.15.183:6815/85363432","heartbeat_front_addr":"172.21.15.183:6813/85363432","state":["exists","up"]},{"osd":6,"uuid":"beb0a7e9-4909-47e2-9ee7-6951d4745e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6819","nonce":1121291186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6821","nonce":1121291186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6825","nonce":1121291186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6823","nonce":1121291186}]},"public_addr":"172.21.15.116:6819/1121291186","cluster_addr":"172.21.15.116:6821/1121291186","heartbeat_back_addr":"172.21.15.116:6825/1121291186","heartbeat_front_addr":"172.21.15.116:6823/1121291186","state":["exists","up"]},{"osd":7,"uuid":"761f4680-f049-473c-bf44-b19acf8de24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6817","nonce":2177665961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6819","nonce":2177665961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6823","nonce":2177665961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6821","nonce":2177665961}]},"public_addr":"172.21.15.177:6817/2177665961","cluster_addr":"172.21.15.177:6819/2177665961","heartbeat_back_addr":"172.21.15.177:6823/2177665961","heartbeat_front_addr":"172.21.15.177:6821/2177665961","state":["exists","up"]},{"osd":8,"uuid":"2385ea6c-fd5a-4815-9596-77f948629542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6817","nonce":1305725258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6819","nonce":1305725258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6823","nonce":1305725258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6821","nonce":1305725258}]},"public_addr":"172.21.15.183:6817/1305725258","cluster_addr":"172.21.15.183:6819/1305725258","heartbeat_back_addr":"172.21.15.183:6823/1305725258","heartbeat_front_addr":"172.21.15.183:6821/1305725258","state":["exists","up"]},{"osd":9,"uuid":"fd0c0dc4-4990-4f12-aa14-f855504cd995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6827","nonce":3831673668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6829","nonce":3831673668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6833","nonce":3831673668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6831","nonce":3831673668}]},"public_addr":"172.21.15.116:6827/3831673668","cluster_addr":"172.21.15.116:6829/3831673668","heartbeat_back_addr":"172.21.15.116:6833/3831673668","heartbeat_front_addr":"172.21.15.116:6831/3831673668","state":["exists","up"]},{"osd":10,"uuid":"864d1a97-4810-49c8-ad4a-a0c10253833c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6825","nonce":963663634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6827","nonce":963663634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6831","nonce":963663634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6829","nonce":963663634}]},"public_addr":"172.21.15.177:6825/963663634","cluster_addr":"172.21.15.177:6827/963663634","heartbeat_back_addr":"172.21.15.177:6831/963663634","heartbeat_front_addr":"172.21.15.177:6829/963663634","state":["exists","up"]},{"osd":11,"uuid":"d49c0841-4620-4f2f-a6b5-f2a9aa2e3c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6825","nonce":1818319939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6827","nonce":1818319939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6831","nonce":1818319939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6829","nonce":1818319939}]},"public_addr":"172.21.15.183:6825/1818319939","cluster_addr":"172.21.15.183:6827/1818319939","heartbeat_back_addr":"172.21.15.183:6831/1818319939","heartbeat_front_addr":"172.21.15.183:6829/1818319939","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:24.736944+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:50.310980+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:56:14.989533+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:56:41.187047+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:06.981664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:32.872771+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:58.850089+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:23.043121+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:47.839958+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:14.087708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:40.010548+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:05.364820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.116:6801/2966274423":"2024-09-13T02:54:03.026862+0000","172.21.15.116:6800/2966274423":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/686139455":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/1552047390":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/2180363411":"2024-09-13T02:53:39.433769+0000","172.21.15.116:6801/1791219601":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/3907400460":"2024-09-13T02:54:03.026862+0000","172.21.15.116:6800/1791219601":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/1802566286":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/212337802":"2024-09-13T02:53:39.433769+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:00:10.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.658+0000 7fa55c753640 1 -- 172.21.15.116:0/1079412543 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa5400a2b70 msgr2=0x7fa5400a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:10.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.658+0000 7fa55c753640 1 --2- 172.21.15.116:0/1079412543 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa5400a2b70 0x7fa5400a5030 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa544000c00 tx=0x7fa5440058d0 comp rx=0 tx=0).stop 2024-09-12T03:00:10.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.658+0000 7fa55c753640 1 -- 172.21.15.116:0/1079412543 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa554106f10 msgr2=0x7fa55419ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:10.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.658+0000 7fa55c753640 1 --2- 172.21.15.116:0/1079412543 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa554106f10 0x7fa55419ce80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00b970 tx=0x7fa54c00be30 comp rx=0 tx=0).stop 2024-09-12T03:00:10.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.658+0000 7fa55c753640 1 -- 172.21.15.116:0/1079412543 shutdown_connections 2024-09-12T03:00:10.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.658+0000 7fa55c753640 1 --2- 172.21.15.116:0/1079412543 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa5400a2b70 0x7fa5400a5030 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:10.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.658+0000 7fa55c753640 1 --2- 172.21.15.116:0/1079412543 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa554106f10 0x7fa55419ce80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:10.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.658+0000 7fa55c753640 1 --2- 172.21.15.116:0/1079412543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa554102790 0x7fa554198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:10.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.658+0000 7fa55c753640 1 --2- 172.21.15.116:0/1079412543 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa554101de0 0x7fa5541985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:10.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.658+0000 7fa55c753640 1 -- 172.21.15.116:0/1079412543 >> 172.21.15.116:0/1079412543 conn(0x7fa5540fd8c0 msgr2=0x7fa5540fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:10.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.658+0000 7fa55c753640 1 -- 172.21.15.116:0/1079412543 shutdown_connections 2024-09-12T03:00:10.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:10.658+0000 7fa55c753640 1 -- 172.21.15.116:0/1079412543 wait complete. 2024-09-12T03:00:10.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:10 smithi183 ceph-mon[35535]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:11.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:11 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1079412543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:00:11.493 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T02:56:17.347152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T03:00:11.494 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-12T03:00:11.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:11 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1079412543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:00:11.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:11 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1079412543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:00:11.739 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:12.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.216+0000 7fd4a1a8d640 1 -- 172.21.15.116:0/727566081 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd49c100e20 msgr2=0x7fd49c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:12.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.216+0000 7fd4a1a8d640 1 --2- 172.21.15.116:0/727566081 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd49c100e20 0x7fd49c101200 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd48c009900 tx=0x7fd48c02a480 comp rx=0 tx=0).stop 2024-09-12T03:00:12.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.216+0000 7fd4a1a8d640 1 -- 172.21.15.116:0/727566081 shutdown_connections 2024-09-12T03:00:12.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.216+0000 7fd4a1a8d640 1 --2- 172.21.15.116:0/727566081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd49c102ef0 0x7fd49c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:12.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.216+0000 7fd4a1a8d640 1 --2- 172.21.15.116:0/727566081 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd49c102570 0x7fd49c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:12.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.216+0000 7fd4a1a8d640 1 --2- 172.21.15.116:0/727566081 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd49c100e20 0x7fd49c101200 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:12.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.216+0000 7fd4a1a8d640 1 -- 172.21.15.116:0/727566081 >> 172.21.15.116:0/727566081 conn(0x7fd49c0fc7d0 msgr2=0x7fd49c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:12.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.216+0000 7fd4a1a8d640 1 -- 172.21.15.116:0/727566081 shutdown_connections 2024-09-12T03:00:12.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.216+0000 7fd4a1a8d640 1 -- 172.21.15.116:0/727566081 wait complete. 2024-09-12T03:00:12.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.216+0000 7fd4a1a8d640 1 Processor -- start 2024-09-12T03:00:12.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.217+0000 7fd4a1a8d640 1 -- start start 2024-09-12T03:00:12.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.217+0000 7fd4a1a8d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd49c100e20 0x7fd49c1994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:12.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.217+0000 7fd4a1a8d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd49c102570 0x7fd49c199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:12.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.217+0000 7fd4a1a8d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd49c102ef0 0x7fd49c1a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:12.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.217+0000 7fd4a1a8d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c111020 con 0x7fd49c102ef0 2024-09-12T03:00:12.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.217+0000 7fd4a1a8d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c110ea0 con 0x7fd49c100e20 2024-09-12T03:00:12.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.217+0000 7fd4a1a8d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c1111a0 con 0x7fd49c102570 2024-09-12T03:00:12.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.217+0000 7fd49bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd49c102ef0 0x7fd49c1a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:12.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.217+0000 7fd49bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd49c102ef0 0x7fd49c1a4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49092/0 (socket says 172.21.15.116:49092) 2024-09-12T03:00:12.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.217+0000 7fd49bfff640 1 -- 172.21.15.116:0/540596254 learned_addr learned my addr 172.21.15.116:0/540596254 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:00:12.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.217+0000 7fd49affd640 1 --2- 172.21.15.116:0/540596254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd49c102570 0x7fd49c199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:12.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.217+0000 7fd49b7fe640 1 --2- 172.21.15.116:0/540596254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd49c100e20 0x7fd49c1994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:12.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.217+0000 7fd49bfff640 1 -- 172.21.15.116:0/540596254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd49c102570 msgr2=0x7fd49c199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:12.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.217+0000 7fd49bfff640 1 --2- 172.21.15.116:0/540596254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd49c102570 0x7fd49c199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:12.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.217+0000 7fd49bfff640 1 -- 172.21.15.116:0/540596254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd49c100e20 msgr2=0x7fd49c1994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:12.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.217+0000 7fd49bfff640 1 --2- 172.21.15.116:0/540596254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd49c100e20 0x7fd49c1994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:12.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.217+0000 7fd49bfff640 1 -- 172.21.15.116:0/540596254 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd49c1a4ab0 con 0x7fd49c102ef0 2024-09-12T03:00:12.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.218+0000 7fd49bfff640 1 --2- 172.21.15.116:0/540596254 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd49c102ef0 0x7fd49c1a4570 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd49400b970 tx=0x7fd49400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:12.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.218+0000 7fd498ff9640 1 -- 172.21.15.116:0/540596254 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd494019070 con 0x7fd49c102ef0 2024-09-12T03:00:12.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.218+0000 7fd498ff9640 1 -- 172.21.15.116:0/540596254 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd494014440 con 0x7fd49c102ef0 2024-09-12T03:00:12.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.218+0000 7fd4a1a8d640 1 -- 172.21.15.116:0/540596254 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd49c1a4c80 con 0x7fd49c102ef0 2024-09-12T03:00:12.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.218+0000 7fd498ff9640 1 -- 172.21.15.116:0/540596254 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4940075d0 con 0x7fd49c102ef0 2024-09-12T03:00:12.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.218+0000 7fd4a1a8d640 1 -- 172.21.15.116:0/540596254 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd49c1a5190 con 0x7fd49c102ef0 2024-09-12T03:00:12.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.219+0000 7fd4a1a8d640 1 -- 172.21.15.116:0/540596254 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd49c101280 con 0x7fd49c102ef0 2024-09-12T03:00:12.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.220+0000 7fd498ff9640 1 -- 172.21.15.116:0/540596254 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fd49401c070 con 0x7fd49c102ef0 2024-09-12T03:00:12.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.221+0000 7fd498ff9640 1 --2- 172.21.15.116:0/540596254 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd4880a2b80 0x7fd4880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:12.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.221+0000 7fd49b7fe640 1 --2- 172.21.15.116:0/540596254 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd4880a2b80 0x7fd4880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:12.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.221+0000 7fd498ff9640 1 -- 172.21.15.116:0/540596254 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fd4940ce190 con 0x7fd49c102ef0 2024-09-12T03:00:12.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.221+0000 7fd49b7fe640 1 --2- 172.21.15.116:0/540596254 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd4880a2b80 0x7fd4880a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd48c005ab0 tx=0x7fd48c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:12.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.225+0000 7fd498ff9640 1 -- 172.21.15.116:0/540596254 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd49409b590 con 0x7fd49c102ef0 2024-09-12T03:00:12.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.390+0000 7fd4a1a8d640 1 -- 172.21.15.116:0/540596254 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd49c109090 con 0x7fd49c102ef0 2024-09-12T03:00:12.392 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:12 smithi116 ceph-mon[31410]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:12.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.391+0000 7fd498ff9640 1 -- 172.21.15.116:0/540596254 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7fd49409ae90 con 0x7fd49c102ef0 2024-09-12T03:00:12.393 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2024-09-12T03:00:12.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.393+0000 7fd4a1a8d640 1 -- 172.21.15.116:0/540596254 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd4880a2b80 msgr2=0x7fd4880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:12.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.393+0000 7fd4a1a8d640 1 --2- 172.21.15.116:0/540596254 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd4880a2b80 0x7fd4880a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd48c005ab0 tx=0x7fd48c005a20 comp rx=0 tx=0).stop 2024-09-12T03:00:12.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.393+0000 7fd4a1a8d640 1 -- 172.21.15.116:0/540596254 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd49c102ef0 msgr2=0x7fd49c1a4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:12.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.393+0000 7fd4a1a8d640 1 --2- 172.21.15.116:0/540596254 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd49c102ef0 0x7fd49c1a4570 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd49400b970 tx=0x7fd49400bd70 comp rx=0 tx=0).stop 2024-09-12T03:00:12.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.393+0000 7fd4a1a8d640 1 -- 172.21.15.116:0/540596254 shutdown_connections 2024-09-12T03:00:12.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.393+0000 7fd4a1a8d640 1 --2- 172.21.15.116:0/540596254 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd4880a2b80 0x7fd4880a5040 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:12.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.393+0000 7fd4a1a8d640 1 --2- 172.21.15.116:0/540596254 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd49c102ef0 0x7fd49c1a4570 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:12.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.393+0000 7fd4a1a8d640 1 --2- 172.21.15.116:0/540596254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd49c102570 0x7fd49c199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:12.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.393+0000 7fd4a1a8d640 1 --2- 172.21.15.116:0/540596254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd49c100e20 0x7fd49c1994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:12.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.393+0000 7fd4a1a8d640 1 -- 172.21.15.116:0/540596254 >> 172.21.15.116:0/540596254 conn(0x7fd49c0fc7d0 msgr2=0x7fd49c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:12.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.393+0000 7fd4a1a8d640 1 -- 172.21.15.116:0/540596254 shutdown_connections 2024-09-12T03:00:12.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:12.393+0000 7fd4a1a8d640 1 -- 172.21.15.116:0/540596254 wait complete. 2024-09-12T03:00:12.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:12 smithi177 ceph-mon[35305]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:12.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:12 smithi183 ceph-mon[35535]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:13.209 INFO:tasks.cephadm:Adding mds.a on smithi116 2024-09-12T03:00:13.209 INFO:tasks.cephadm:Adding mds.d on smithi116 2024-09-12T03:00:13.209 INFO:tasks.cephadm:Adding mds.g on smithi116 2024-09-12T03:00:13.209 INFO:tasks.cephadm:Adding mds.j on smithi116 2024-09-12T03:00:13.209 INFO:tasks.cephadm:Adding mds.b on smithi177 2024-09-12T03:00:13.209 INFO:tasks.cephadm:Adding mds.e on smithi177 2024-09-12T03:00:13.210 INFO:tasks.cephadm:Adding mds.h on smithi177 2024-09-12T03:00:13.210 INFO:tasks.cephadm:Adding mds.k on smithi177 2024-09-12T03:00:13.210 INFO:tasks.cephadm:Adding mds.c on smithi183 2024-09-12T03:00:13.210 INFO:tasks.cephadm:Adding mds.f on smithi183 2024-09-12T03:00:13.210 INFO:tasks.cephadm:Adding mds.i on smithi183 2024-09-12T03:00:13.210 INFO:tasks.cephadm:Adding mds.l on smithi183 2024-09-12T03:00:13.210 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi116=a;smithi116=d;smithi116=g;smithi116=j;smithi177=b;smithi177=e;smithi177=h;smithi177=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l' 2024-09-12T03:00:13.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:13 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/540596254' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:00:13.471 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:00:13.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:13 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/540596254' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:00:13.592 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:13 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/540596254' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:00:13.961 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.953+0000 7f1820e34640 1 -- 172.21.15.183:0/1583941635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f181c101cd0 msgr2=0x7f181c102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:13.961 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.953+0000 7f1820e34640 1 --2- 172.21.15.183:0/1583941635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f181c101cd0 0x7f181c102150 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1810009900 tx=0x7f181002a410 comp rx=0 tx=0).stop 2024-09-12T03:00:13.961 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.953+0000 7f1820e34640 1 -- 172.21.15.183:0/1583941635 shutdown_connections 2024-09-12T03:00:13.961 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.953+0000 7f1820e34640 1 --2- 172.21.15.183:0/1583941635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f181c102690 0x7f181c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:13.961 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.953+0000 7f1820e34640 1 --2- 172.21.15.183:0/1583941635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f181c101cd0 0x7f181c102150 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:13.961 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.953+0000 7f1820e34640 1 --2- 172.21.15.183:0/1583941635 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f181c100ad0 0x7f181c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:13.961 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.953+0000 7f1820e34640 1 -- 172.21.15.183:0/1583941635 >> 172.21.15.183:0/1583941635 conn(0x7f181c0fc280 msgr2=0x7f181c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:13.961 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.953+0000 7f1820e34640 1 -- 172.21.15.183:0/1583941635 shutdown_connections 2024-09-12T03:00:13.961 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.953+0000 7f1820e34640 1 -- 172.21.15.183:0/1583941635 wait complete. 2024-09-12T03:00:13.961 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.954+0000 7f1820e34640 1 Processor -- start 2024-09-12T03:00:13.961 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.954+0000 7f1820e34640 1 -- start start 2024-09-12T03:00:13.962 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.954+0000 7f1820e34640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f181c100ad0 0x7f181c197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:13.962 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.954+0000 7f1820e34640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f181c101cd0 0x7f181c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:13.962 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.954+0000 7f1820e34640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f181c102690 0x7f181c19f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:13.962 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.954+0000 7f1820e34640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f181c10c750 con 0x7f181c101cd0 2024-09-12T03:00:13.962 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.954+0000 7f1820e34640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f181c10c5d0 con 0x7f181c102690 2024-09-12T03:00:13.962 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.954+0000 7f1820e34640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f181c10c8d0 con 0x7f181c100ad0 2024-09-12T03:00:13.962 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.954+0000 7f181a59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f181c100ad0 0x7f181c197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:13.962 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.954+0000 7f181a59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f181c100ad0 0x7f181c197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:46062/0 (socket says 172.21.15.183:46062) 2024-09-12T03:00:13.962 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.954+0000 7f181a59c640 1 -- 172.21.15.183:0/2545813418 learned_addr learned my addr 172.21.15.183:0/2545813418 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-12T03:00:13.962 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.955+0000 7f1819d9b640 1 --2- 172.21.15.183:0/2545813418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f181c101cd0 0x7f181c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:13.962 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.955+0000 7f181ad9d640 1 --2- 172.21.15.183:0/2545813418 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f181c102690 0x7f181c19f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:13.962 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.955+0000 7f181a59c640 1 -- 172.21.15.183:0/2545813418 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f181c102690 msgr2=0x7f181c19f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:13.962 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.955+0000 7f181a59c640 1 --2- 172.21.15.183:0/2545813418 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f181c102690 0x7f181c19f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:13.963 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.955+0000 7f181a59c640 1 -- 172.21.15.183:0/2545813418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f181c101cd0 msgr2=0x7f181c198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:13.963 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.955+0000 7f181a59c640 1 --2- 172.21.15.183:0/2545813418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f181c101cd0 0x7f181c198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:13.963 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.955+0000 7f181a59c640 1 -- 172.21.15.183:0/2545813418 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181c19faf0 con 0x7f181c100ad0 2024-09-12T03:00:13.963 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.955+0000 7f181ad9d640 1 --2- 172.21.15.183:0/2545813418 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f181c102690 0x7f181c19f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:00:13.963 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.955+0000 7f181a59c640 1 --2- 172.21.15.183:0/2545813418 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f181c100ad0 0x7f181c197ec0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f180c00c970 tx=0x7f180c00ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:13.963 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.955+0000 7f180b7fe640 1 -- 172.21.15.183:0/2545813418 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c004220 con 0x7f181c100ad0 2024-09-12T03:00:13.963 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.955+0000 7f1820e34640 1 -- 172.21.15.183:0/2545813418 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f181c19fde0 con 0x7f181c100ad0 2024-09-12T03:00:13.963 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.955+0000 7f180b7fe640 1 -- 172.21.15.183:0/2545813418 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f180c0043c0 con 0x7f181c100ad0 2024-09-12T03:00:13.963 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.955+0000 7f180b7fe640 1 -- 172.21.15.183:0/2545813418 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c00f450 con 0x7f181c100ad0 2024-09-12T03:00:13.963 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.955+0000 7f1820e34640 1 -- 172.21.15.183:0/2545813418 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f181c1a0240 con 0x7f181c100ad0 2024-09-12T03:00:13.963 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.956+0000 7f1820e34640 1 -- 172.21.15.183:0/2545813418 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f181c1a04d0 con 0x7f181c100ad0 2024-09-12T03:00:13.963 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.957+0000 7f180b7fe640 1 -- 172.21.15.183:0/2545813418 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f180c0079b0 con 0x7f181c100ad0 2024-09-12T03:00:13.963 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.958+0000 7f180b7fe640 1 --2- 172.21.15.183:0/2545813418 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f18040a2bb0 0x7f18040a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:13.963 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.958+0000 7f180b7fe640 1 -- 172.21.15.183:0/2545813418 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f180c0cecc0 con 0x7f181c100ad0 2024-09-12T03:00:13.964 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.958+0000 7f1819d9b640 1 --2- 172.21.15.183:0/2545813418 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f18040a2bb0 0x7f18040a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:13.964 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.959+0000 7f1819d9b640 1 --2- 172.21.15.183:0/2545813418 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f18040a2bb0 0x7f18040a5070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f18100314e0 tx=0x7f1810000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:13.966 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:13.966+0000 7f180b7fe640 1 -- 172.21.15.183:0/2545813418 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f180c014030 con 0x7f181c100ad0 2024-09-12T03:00:14.150 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:14.149+0000 7f1820e34640 1 -- 172.21.15.183:0/2545813418 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi116=a;smithi116=d;smithi116=g;smithi116=j;smithi177=b;smithi177=e;smithi177=h;smithi177=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}) -- 0x7f181c06a6b0 con 0x7f18040a2bb0 2024-09-12T03:00:14.157 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:14.157+0000 7f180b7fe640 1 -- 172.21.15.183:0/2545813418 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f181c06a6b0 con 0x7f18040a2bb0 2024-09-12T03:00:14.157 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled mds.all update... 2024-09-12T03:00:14.160 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:14.159+0000 7f1820e34640 1 -- 172.21.15.183:0/2545813418 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f18040a2bb0 msgr2=0x7f18040a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:14.160 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:14.159+0000 7f1820e34640 1 --2- 172.21.15.183:0/2545813418 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f18040a2bb0 0x7f18040a5070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f18100314e0 tx=0x7f1810000bc0 comp rx=0 tx=0).stop 2024-09-12T03:00:14.160 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:14.159+0000 7f1820e34640 1 -- 172.21.15.183:0/2545813418 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f181c100ad0 msgr2=0x7f181c197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:14.160 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:14.159+0000 7f1820e34640 1 --2- 172.21.15.183:0/2545813418 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f181c100ad0 0x7f181c197ec0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f180c00c970 tx=0x7f180c00ce30 comp rx=0 tx=0).stop 2024-09-12T03:00:14.160 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:14.159+0000 7f1820e34640 1 -- 172.21.15.183:0/2545813418 shutdown_connections 2024-09-12T03:00:14.160 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:14.159+0000 7f1820e34640 1 --2- 172.21.15.183:0/2545813418 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f18040a2bb0 0x7f18040a5070 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:14.160 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:14.159+0000 7f1820e34640 1 --2- 172.21.15.183:0/2545813418 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f181c102690 0x7f181c19f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:14.160 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:14.159+0000 7f1820e34640 1 --2- 172.21.15.183:0/2545813418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f181c101cd0 0x7f181c198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:14.161 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:14.159+0000 7f1820e34640 1 --2- 172.21.15.183:0/2545813418 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f181c100ad0 0x7f181c197ec0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:14.161 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:14.159+0000 7f1820e34640 1 -- 172.21.15.183:0/2545813418 >> 172.21.15.183:0/2545813418 conn(0x7f181c0fc280 msgr2=0x7f181c069fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:14.161 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:14.159+0000 7f1820e34640 1 -- 172.21.15.183:0/2545813418 shutdown_connections 2024-09-12T03:00:14.161 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-12T03:00:14.159+0000 7f1820e34640 1 -- 172.21.15.183:0/2545813418 wait complete. 2024-09-12T03:00:14.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:14 smithi116 ceph-mon[31410]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:14.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:14 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:14.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:14 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:00:14.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:14 smithi183 ceph-mon[35535]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:14.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:14 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:14.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:14 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:00:14.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:14 smithi177 ceph-mon[35305]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:14.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:14 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:14.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:14 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:00:14.820 DEBUG:teuthology.orchestra.run.smithi116:mds.a> sudo journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.a.service 2024-09-12T03:00:14.823 DEBUG:teuthology.orchestra.run.smithi116:mds.d> sudo journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.d.service 2024-09-12T03:00:14.866 DEBUG:teuthology.orchestra.run.smithi116:mds.g> sudo journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.g.service 2024-09-12T03:00:14.909 DEBUG:teuthology.orchestra.run.smithi116:mds.j> sudo journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.j.service 2024-09-12T03:00:14.952 DEBUG:teuthology.orchestra.run.smithi177:mds.b> sudo journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.b.service 2024-09-12T03:00:14.954 DEBUG:teuthology.orchestra.run.smithi177:mds.e> sudo journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.e.service 2024-09-12T03:00:14.999 DEBUG:teuthology.orchestra.run.smithi177:mds.h> sudo journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.h.service 2024-09-12T03:00:15.043 DEBUG:teuthology.orchestra.run.smithi177:mds.k> sudo journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.k.service 2024-09-12T03:00:15.086 DEBUG:teuthology.orchestra.run.smithi183:mds.c> sudo journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.c.service 2024-09-12T03:00:15.089 DEBUG:teuthology.orchestra.run.smithi183:mds.f> sudo journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.f.service 2024-09-12T03:00:15.131 DEBUG:teuthology.orchestra.run.smithi183:mds.i> sudo journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.i.service 2024-09-12T03:00:15.175 DEBUG:teuthology.orchestra.run.smithi183:mds.l> sudo journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.l.service 2024-09-12T03:00:15.218 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-12T03:00:15.218 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:00:15.428 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:15 smithi183 ceph-mon[35535]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi116=a;smithi116=d;smithi116=g;smithi116=j;smithi177=b;smithi177=e;smithi177=h;smithi177=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:00:15.428 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:15 smithi183 ceph-mon[35535]: Saving service mds.all spec with placement smithi116=a;smithi116=d;smithi116=g;smithi116=j;smithi177=b;smithi177=e;smithi177=h;smithi177=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2024-09-12T03:00:15.428 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:15 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:15.428 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:15 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:15.428 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:15 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:15.428 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:15 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:15.428 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:15 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:15.428 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:15 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:15.459 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:15.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:15 smithi177 ceph-mon[35305]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi116=a;smithi116=d;smithi116=g;smithi116=j;smithi177=b;smithi177=e;smithi177=h;smithi177=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:00:15.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:15 smithi177 ceph-mon[35305]: Saving service mds.all spec with placement smithi116=a;smithi116=d;smithi116=g;smithi116=j;smithi177=b;smithi177=e;smithi177=h;smithi177=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2024-09-12T03:00:15.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:15 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:15.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:15 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:15.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:15 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:15.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:15 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:15.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:15 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:15.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:15 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:15.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:15 smithi116 ceph-mon[31410]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi116=a;smithi116=d;smithi116=g;smithi116=j;smithi177=b;smithi177=e;smithi177=h;smithi177=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:00:15.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:15 smithi116 ceph-mon[31410]: Saving service mds.all spec with placement smithi116=a;smithi116=d;smithi116=g;smithi116=j;smithi177=b;smithi177=e;smithi177=h;smithi177=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2024-09-12T03:00:15.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:15.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:15.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:15.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:15.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:15.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:15.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.921+0000 7fe31a015640 1 -- 172.21.15.116:0/3418795850 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe314102790 msgr2=0x7fe3141067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:15.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.921+0000 7fe31a015640 1 --2- 172.21.15.116:0/3418795850 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe314102790 0x7fe3141067e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe308009900 tx=0x7fe30802a410 comp rx=0 tx=0).stop 2024-09-12T03:00:15.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.921+0000 7fe31a015640 1 -- 172.21.15.116:0/3418795850 shutdown_connections 2024-09-12T03:00:15.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.921+0000 7fe31a015640 1 --2- 172.21.15.116:0/3418795850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe314106f10 0x7fe31410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:15.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.921+0000 7fe31a015640 1 --2- 172.21.15.116:0/3418795850 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe314102790 0x7fe3141067e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:15.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.921+0000 7fe31a015640 1 --2- 172.21.15.116:0/3418795850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe314101de0 0x7fe3141021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:15.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.921+0000 7fe31a015640 1 -- 172.21.15.116:0/3418795850 >> 172.21.15.116:0/3418795850 conn(0x7fe3140fd8c0 msgr2=0x7fe3140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:15.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.921+0000 7fe31a015640 1 -- 172.21.15.116:0/3418795850 shutdown_connections 2024-09-12T03:00:15.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.921+0000 7fe31a015640 1 -- 172.21.15.116:0/3418795850 wait complete. 2024-09-12T03:00:15.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.922+0000 7fe31a015640 1 Processor -- start 2024-09-12T03:00:15.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.922+0000 7fe31a015640 1 -- start start 2024-09-12T03:00:15.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.922+0000 7fe31a015640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe314101de0 0x7fe314198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:15.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.922+0000 7fe31a015640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe314102790 0x7fe314198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:15.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.922+0000 7fe31a015640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe314106f10 0x7fe31419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:15.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.922+0000 7fe31a015640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe314111f30 con 0x7fe314102790 2024-09-12T03:00:15.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.922+0000 7fe31a015640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe314111db0 con 0x7fe314106f10 2024-09-12T03:00:15.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.922+0000 7fe31a015640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe3141120b0 con 0x7fe314101de0 2024-09-12T03:00:15.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.922+0000 7fe312ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe314102790 0x7fe314198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:15.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.923+0000 7fe3137fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe314101de0 0x7fe314198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:15.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.923+0000 7fe312ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe314102790 0x7fe314198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49112/0 (socket says 172.21.15.116:49112) 2024-09-12T03:00:15.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.923+0000 7fe312ffd640 1 -- 172.21.15.116:0/462040549 learned_addr learned my addr 172.21.15.116:0/462040549 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:00:15.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.923+0000 7fe313fff640 1 --2- 172.21.15.116:0/462040549 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe314106f10 0x7fe31419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:15.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.923+0000 7fe312ffd640 1 -- 172.21.15.116:0/462040549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe314101de0 msgr2=0x7fe314198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:15.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.923+0000 7fe312ffd640 1 --2- 172.21.15.116:0/462040549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe314101de0 0x7fe314198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:15.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.923+0000 7fe312ffd640 1 -- 172.21.15.116:0/462040549 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe314106f10 msgr2=0x7fe31419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:15.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.923+0000 7fe312ffd640 1 --2- 172.21.15.116:0/462040549 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe314106f10 0x7fe31419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:15.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.923+0000 7fe312ffd640 1 -- 172.21.15.116:0/462040549 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe31419d4c0 con 0x7fe314102790 2024-09-12T03:00:15.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.923+0000 7fe313fff640 1 --2- 172.21.15.116:0/462040549 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe314106f10 0x7fe31419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:00:15.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.923+0000 7fe3137fe640 1 --2- 172.21.15.116:0/462040549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe314101de0 0x7fe314198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:00:15.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.923+0000 7fe312ffd640 1 --2- 172.21.15.116:0/462040549 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe314102790 0x7fe314198a50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe308006010 tx=0x7fe3080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:15.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.923+0000 7fe310ff9640 1 -- 172.21.15.116:0/462040549 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30803f030 con 0x7fe314102790 2024-09-12T03:00:15.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.923+0000 7fe310ff9640 1 -- 172.21.15.116:0/462040549 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe308003ce0 con 0x7fe314102790 2024-09-12T03:00:15.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.923+0000 7fe31a015640 1 -- 172.21.15.116:0/462040549 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe31419d750 con 0x7fe314102790 2024-09-12T03:00:15.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.923+0000 7fe310ff9640 1 -- 172.21.15.116:0/462040549 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe308035070 con 0x7fe314102790 2024-09-12T03:00:15.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.923+0000 7fe31a015640 1 -- 172.21.15.116:0/462040549 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3141a5090 con 0x7fe314102790 2024-09-12T03:00:15.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.924+0000 7fe31a015640 1 -- 172.21.15.116:0/462040549 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3141032c0 con 0x7fe314102790 2024-09-12T03:00:15.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.925+0000 7fe310ff9640 1 -- 172.21.15.116:0/462040549 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe308042030 con 0x7fe314102790 2024-09-12T03:00:15.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.926+0000 7fe310ff9640 1 --2- 172.21.15.116:0/462040549 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe2fc0a2b80 0x7fe2fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:15.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.926+0000 7fe3137fe640 1 --2- 172.21.15.116:0/462040549 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe2fc0a2b80 0x7fe2fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:15.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.926+0000 7fe310ff9640 1 -- 172.21.15.116:0/462040549 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fe3080ef5a0 con 0x7fe314102790 2024-09-12T03:00:15.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.927+0000 7fe3137fe640 1 --2- 172.21.15.116:0/462040549 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe2fc0a2b80 0x7fe2fc0a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe304005d00 tx=0x7fe30400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:15.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:15.930+0000 7fe310ff9640 1 -- 172.21.15.116:0/462040549 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe3080bc9c0 con 0x7fe314102790 2024-09-12T03:00:16.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:16.130+0000 7fe31a015640 1 -- 172.21.15.116:0/462040549 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe31410a0f0 con 0x7fe314102790 2024-09-12T03:00:16.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:16.131+0000 7fe310ff9640 1 -- 172.21.15.116:0/462040549 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fe3080bc2c0 con 0x7fe314102790 2024-09-12T03:00:16.132 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:00:16.132 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":1,"btime":"2024-09-12T02:53:07:105500+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-12T03:00:16.133 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 1 2024-09-12T03:00:16.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:16.133+0000 7fe31a015640 1 -- 172.21.15.116:0/462040549 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe2fc0a2b80 msgr2=0x7fe2fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:16.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:16.133+0000 7fe31a015640 1 --2- 172.21.15.116:0/462040549 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe2fc0a2b80 0x7fe2fc0a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe304005d00 tx=0x7fe30400b410 comp rx=0 tx=0).stop 2024-09-12T03:00:16.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:16.133+0000 7fe31a015640 1 -- 172.21.15.116:0/462040549 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe314102790 msgr2=0x7fe314198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:16.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:16.133+0000 7fe31a015640 1 --2- 172.21.15.116:0/462040549 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe314102790 0x7fe314198a50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe308006010 tx=0x7fe3080042f0 comp rx=0 tx=0).stop 2024-09-12T03:00:16.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:16.133+0000 7fe31a015640 1 -- 172.21.15.116:0/462040549 shutdown_connections 2024-09-12T03:00:16.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:16.133+0000 7fe31a015640 1 --2- 172.21.15.116:0/462040549 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe2fc0a2b80 0x7fe2fc0a5040 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:16.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:16.133+0000 7fe31a015640 1 --2- 172.21.15.116:0/462040549 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe314106f10 0x7fe31419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:16.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:16.133+0000 7fe31a015640 1 --2- 172.21.15.116:0/462040549 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe314102790 0x7fe314198a50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:16.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:16.133+0000 7fe31a015640 1 --2- 172.21.15.116:0/462040549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe314101de0 0x7fe314198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:16.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:16.133+0000 7fe31a015640 1 -- 172.21.15.116:0/462040549 >> 172.21.15.116:0/462040549 conn(0x7fe3140fd8c0 msgr2=0x7fe3140fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:16.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:16.133+0000 7fe31a015640 1 -- 172.21.15.116:0/462040549 shutdown_connections 2024-09-12T03:00:16.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:16.133+0000 7fe31a015640 1 -- 172.21.15.116:0/462040549 wait complete. 2024-09-12T03:00:16.372 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:16 smithi183 ceph-mon[35535]: Deploying daemon mds.l on smithi183 2024-09-12T03:00:16.372 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:16 smithi183 ceph-mon[35535]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:16.372 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:16 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/462040549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:16.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:16 smithi177 ceph-mon[35305]: Deploying daemon mds.l on smithi183 2024-09-12T03:00:16.497 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:16 smithi177 ceph-mon[35305]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:16.497 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:16 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/462040549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:16.505 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:16 smithi116 ceph-mon[31410]: Deploying daemon mds.l on smithi183 2024-09-12T03:00:16.505 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:16 smithi116 ceph-mon[31410]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:16.505 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:16 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/462040549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:16.703 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 03:00:16 smithi183 systemd[1]: Starting Ceph mds.l for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:00:17.025 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 03:00:16 smithi183 podman[65152]: 2024-09-12 03:00:16.673149833 +0000 UTC m=+0.015989275 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:00:17.025 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 03:00:16 smithi183 podman[65152]: 2024-09-12 03:00:16.779534859 +0000 UTC m=+0.122374286 container create 7d1a29e0bc14897f7cd1e6bb082e0c98a954f1782577e602bb1e8619166a4ebb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-l, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-12T03:00:17.025 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 03:00:16 smithi183 podman[65152]: 2024-09-12 03:00:16.998695392 +0000 UTC m=+0.341534818 container init 7d1a29e0bc14897f7cd1e6bb082e0c98a954f1782577e602bb1e8619166a4ebb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-l, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-12T03:00:17.025 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 03:00:17 smithi183 podman[65152]: 2024-09-12 03:00:17.001408235 +0000 UTC m=+0.344247661 container start 7d1a29e0bc14897f7cd1e6bb082e0c98a954f1782577e602bb1e8619166a4ebb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-l, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-12T03:00:17.453 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 03:00:17 smithi183 ceph-mds[65171]: -- 172.21.15.183:0/243644884 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55569257d860 con 0x55569255a800 2024-09-12T03:00:17.453 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 03:00:17 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-l[65167]: starting mds.l at 2024-09-12T03:00:17.453 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 03:00:17 smithi183 bash[65152]: 7d1a29e0bc14897f7cd1e6bb082e0c98a954f1782577e602bb1e8619166a4ebb 2024-09-12T03:00:17.454 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 03:00:17 smithi183 systemd[1]: Started Ceph mds.l for eb1ac602-70b1-11ef-bcea-c7b262605968. 2024-09-12T03:00:18.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:18 smithi177 ceph-mon[35305]: mds.? [v2:172.21.15.183:6832/1068409597,v1:172.21.15.183:6833/1068409597] up:boot 2024-09-12T03:00:18.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:18 smithi177 ceph-mon[35305]: fsmap 1 up:standby 2024-09-12T03:00:18.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:18 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:00:18.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:18 smithi177 ceph-mon[35305]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:18.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:18 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:18.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:18 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:18.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:18 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:18.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:18 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:18.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:18 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:18.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:18 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:18.246 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:18 smithi177 ceph-mon[35305]: Deploying daemon mds.k on smithi177 2024-09-12T03:00:18.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:18 smithi116 ceph-mon[31410]: mds.? [v2:172.21.15.183:6832/1068409597,v1:172.21.15.183:6833/1068409597] up:boot 2024-09-12T03:00:18.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:18 smithi116 ceph-mon[31410]: fsmap 1 up:standby 2024-09-12T03:00:18.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:18 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:00:18.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:18 smithi116 ceph-mon[31410]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:18.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:18 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:18.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:18 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:18.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:18 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:18.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:18 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:18.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:18 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:18.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:18 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:18.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:18 smithi116 ceph-mon[31410]: Deploying daemon mds.k on smithi177 2024-09-12T03:00:18.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:18 smithi183 ceph-mon[35535]: mds.? [v2:172.21.15.183:6832/1068409597,v1:172.21.15.183:6833/1068409597] up:boot 2024-09-12T03:00:18.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:18 smithi183 ceph-mon[35535]: fsmap 1 up:standby 2024-09-12T03:00:18.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:18 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:00:18.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:18 smithi183 ceph-mon[35535]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:18.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:18 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:18.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:18 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:18.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:18 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:18.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:18 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:18.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:18 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:18.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:18 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:18.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:18 smithi183 ceph-mon[35535]: Deploying daemon mds.k on smithi177 2024-09-12T03:00:18.960 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:00:19.218 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:19.426 INFO:journalctl@ceph.mds.k.smithi177.stdout:Sep 12 03:00:19 smithi177 systemd[1]: Starting Ceph mds.k for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:00:19.729 INFO:journalctl@ceph.mds.k.smithi177.stdout:Sep 12 03:00:19 smithi177 podman[64398]: 2024-09-12 03:00:19.328995367 +0000 UTC m=+0.015845136 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:00:19.729 INFO:journalctl@ceph.mds.k.smithi177.stdout:Sep 12 03:00:19 smithi177 podman[64398]: 2024-09-12 03:00:19.442578031 +0000 UTC m=+0.129427790 container create 2d7de1207a651987336e01a598bf3f177206ec3a2e8010970c7ec41430160ad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-12T03:00:19.729 INFO:journalctl@ceph.mds.k.smithi177.stdout:Sep 12 03:00:19 smithi177 podman[64398]: 2024-09-12 03:00:19.645105047 +0000 UTC m=+0.331954818 container init 2d7de1207a651987336e01a598bf3f177206ec3a2e8010970c7ec41430160ad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-k, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-12T03:00:19.729 INFO:journalctl@ceph.mds.k.smithi177.stdout:Sep 12 03:00:19 smithi177 podman[64398]: 2024-09-12 03:00:19.648002379 +0000 UTC m=+0.334852149 container start 2d7de1207a651987336e01a598bf3f177206ec3a2e8010970c7ec41430160ad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-k, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-12T03:00:19.729 INFO:journalctl@ceph.mds.k.smithi177.stdout:Sep 12 03:00:19 smithi177 ceph-mds[64418]: -- 172.21.15.177:0/2538854552 <== mon.2 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563fe5f59860 con 0x563fe5f36800 2024-09-12T03:00:19.729 INFO:journalctl@ceph.mds.k.smithi177.stdout:Sep 12 03:00:19 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-k[64414]: starting mds.k at 2024-09-12T03:00:19.729 INFO:journalctl@ceph.mds.k.smithi177.stdout:Sep 12 03:00:19 smithi177 bash[64398]: 2d7de1207a651987336e01a598bf3f177206ec3a2e8010970c7ec41430160ad5 2024-09-12T03:00:19.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.740+0000 7f700a26f640 1 -- 172.21.15.116:0/2163839733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7004101de0 msgr2=0x7f70041021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:19.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.740+0000 7f700a26f640 1 --2- 172.21.15.116:0/2163839733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7004101de0 0x7f70041021c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4009900 tx=0x7f6ff402a410 comp rx=0 tx=0).stop 2024-09-12T03:00:19.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.740+0000 7f700a26f640 1 -- 172.21.15.116:0/2163839733 shutdown_connections 2024-09-12T03:00:19.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.740+0000 7f700a26f640 1 --2- 172.21.15.116:0/2163839733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7004106f10 0x7f700410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:19.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.740+0000 7f700a26f640 1 --2- 172.21.15.116:0/2163839733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7004102790 0x7f70041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:19.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.740+0000 7f700a26f640 1 --2- 172.21.15.116:0/2163839733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7004101de0 0x7f70041021c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:19.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.740+0000 7f700a26f640 1 -- 172.21.15.116:0/2163839733 >> 172.21.15.116:0/2163839733 conn(0x7f70040fd8c0 msgr2=0x7f70040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:19.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.740+0000 7f700a26f640 1 -- 172.21.15.116:0/2163839733 shutdown_connections 2024-09-12T03:00:19.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.740+0000 7f700a26f640 1 -- 172.21.15.116:0/2163839733 wait complete. 2024-09-12T03:00:19.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.741+0000 7f700a26f640 1 Processor -- start 2024-09-12T03:00:19.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.741+0000 7f700a26f640 1 -- start start 2024-09-12T03:00:19.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.741+0000 7f700a26f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7004101de0 0x7f70040744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:19.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.741+0000 7f700a26f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7004102790 0x7f7004074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:19.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.741+0000 7f700a26f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7004106f10 0x7f7004074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:19.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.741+0000 7f700a26f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7004111f70 con 0x7f7004106f10 2024-09-12T03:00:19.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.741+0000 7f700a26f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7004111df0 con 0x7f7004102790 2024-09-12T03:00:19.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.741+0000 7f700a26f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f70041120f0 con 0x7f7004101de0 2024-09-12T03:00:19.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.741+0000 7f700880c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7004106f10 0x7f7004074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:19.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.741+0000 7f700880c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7004106f10 0x7f7004074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49128/0 (socket says 172.21.15.116:49128) 2024-09-12T03:00:19.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.741+0000 7f700880c640 1 -- 172.21.15.116:0/2358469064 learned_addr learned my addr 172.21.15.116:0/2358469064 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:00:19.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.741+0000 7f6ffbfff640 1 --2- 172.21.15.116:0/2358469064 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7004102790 0x7f7004074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:19.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.741+0000 7f7003fff640 1 --2- 172.21.15.116:0/2358469064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7004101de0 0x7f70040744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:19.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.741+0000 7f700880c640 1 -- 172.21.15.116:0/2358469064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7004101de0 msgr2=0x7f70040744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:19.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.741+0000 7f700880c640 1 --2- 172.21.15.116:0/2358469064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7004101de0 0x7f70040744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:19.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.741+0000 7f700880c640 1 -- 172.21.15.116:0/2358469064 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7004102790 msgr2=0x7f7004074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:19.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.741+0000 7f700880c640 1 --2- 172.21.15.116:0/2358469064 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7004102790 0x7f7004074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:19.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.741+0000 7f700880c640 1 -- 172.21.15.116:0/2358469064 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70041a6910 con 0x7f7004106f10 2024-09-12T03:00:19.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.742+0000 7f700880c640 1 --2- 172.21.15.116:0/2358469064 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7004106f10 0x7f7004074f70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc00eb10 tx=0x7f6ffc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:19.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.742+0000 7f7001ffb640 1 -- 172.21.15.116:0/2358469064 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc004b20 con 0x7f7004106f10 2024-09-12T03:00:19.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.742+0000 7f7001ffb640 1 -- 172.21.15.116:0/2358469064 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ffc004cc0 con 0x7f7004106f10 2024-09-12T03:00:19.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.742+0000 7f700a26f640 1 -- 172.21.15.116:0/2358469064 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70041a6c00 con 0x7f7004106f10 2024-09-12T03:00:19.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.742+0000 7f7001ffb640 1 -- 172.21.15.116:0/2358469064 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc010760 con 0x7f7004106f10 2024-09-12T03:00:19.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.742+0000 7f700a26f640 1 -- 172.21.15.116:0/2358469064 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70041a70f0 con 0x7f7004106f10 2024-09-12T03:00:19.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.743+0000 7f700a26f640 1 -- 172.21.15.116:0/2358469064 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70041021c0 con 0x7f7004106f10 2024-09-12T03:00:19.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.744+0000 7f7001ffb640 1 -- 172.21.15.116:0/2358469064 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f6ffc005020 con 0x7f7004106f10 2024-09-12T03:00:19.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.744+0000 7f7001ffb640 1 --2- 172.21.15.116:0/2358469064 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6ff00a2b80 0x7f6ff00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:19.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.745+0000 7f7003fff640 1 --2- 172.21.15.116:0/2358469064 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6ff00a2b80 0x7f6ff00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:19.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.745+0000 7f7001ffb640 1 -- 172.21.15.116:0/2358469064 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f6ffc0cf090 con 0x7f7004106f10 2024-09-12T03:00:19.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.745+0000 7f7003fff640 1 --2- 172.21.15.116:0/2358469064 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6ff00a2b80 0x7f6ff00a5040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4009230 tx=0x7f6ff40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:19.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.749+0000 7f7001ffb640 1 -- 172.21.15.116:0/2358469064 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6ffc09c490 con 0x7f7004106f10 2024-09-12T03:00:19.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.960+0000 7f700a26f640 1 -- 172.21.15.116:0/2358469064 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f700410a0f0 con 0x7f7004106f10 2024-09-12T03:00:19.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.961+0000 7f7001ffb640 1 -- 172.21.15.116:0/2358469064 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f6ffc09bd90 con 0x7f7004106f10 2024-09-12T03:00:19.962 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:00:19.962 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":2,"btime":"2024-09-12T03:00:17:029893+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T03:00:19.962 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 2 2024-09-12T03:00:19.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.963+0000 7f700a26f640 1 -- 172.21.15.116:0/2358469064 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6ff00a2b80 msgr2=0x7f6ff00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:19.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.963+0000 7f700a26f640 1 --2- 172.21.15.116:0/2358469064 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6ff00a2b80 0x7f6ff00a5040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4009230 tx=0x7f6ff40058d0 comp rx=0 tx=0).stop 2024-09-12T03:00:19.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.963+0000 7f700a26f640 1 -- 172.21.15.116:0/2358469064 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7004106f10 msgr2=0x7f7004074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:19.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.963+0000 7f700a26f640 1 --2- 172.21.15.116:0/2358469064 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7004106f10 0x7f7004074f70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc00eb10 tx=0x7f6ffc00c2d0 comp rx=0 tx=0).stop 2024-09-12T03:00:19.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.963+0000 7f700a26f640 1 -- 172.21.15.116:0/2358469064 shutdown_connections 2024-09-12T03:00:19.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.963+0000 7f700a26f640 1 --2- 172.21.15.116:0/2358469064 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6ff00a2b80 0x7f6ff00a5040 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:19.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.963+0000 7f700a26f640 1 --2- 172.21.15.116:0/2358469064 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7004106f10 0x7f7004074f70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:19.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.963+0000 7f700a26f640 1 --2- 172.21.15.116:0/2358469064 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7004102790 0x7f7004074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:19.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.963+0000 7f700a26f640 1 --2- 172.21.15.116:0/2358469064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7004101de0 0x7f70040744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:19.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.963+0000 7f700a26f640 1 -- 172.21.15.116:0/2358469064 >> 172.21.15.116:0/2358469064 conn(0x7f70040fd8c0 msgr2=0x7f70040fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:19.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.963+0000 7f700a26f640 1 -- 172.21.15.116:0/2358469064 shutdown_connections 2024-09-12T03:00:19.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:19.964+0000 7f700a26f640 1 -- 172.21.15.116:0/2358469064 wait complete. 2024-09-12T03:00:19.995 INFO:journalctl@ceph.mds.k.smithi177.stdout:Sep 12 03:00:19 smithi177 systemd[1]: Started Ceph mds.k for eb1ac602-70b1-11ef-bcea-c7b262605968. 2024-09-12T03:00:20.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:20 smithi116 ceph-mon[31410]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:20.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:20 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:20.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:20 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:20.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:20 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:20.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:20 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:20.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:20 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:20.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:20 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:20.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:20 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2358469064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:20.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:20 smithi183 ceph-mon[35535]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:20.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:20 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:20.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:20 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:20.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:20 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:20.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:20 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:20.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:20 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:20.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:20 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:20.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:20 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2358469064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:20.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:20 smithi177 ceph-mon[35305]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:20.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:20 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:20.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:20 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:20.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:20 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:20.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:20 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:20.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:20 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:20.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:20 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:20.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:20 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2358469064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:21.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:21 smithi116 ceph-mon[31410]: Deploying daemon mds.j on smithi116 2024-09-12T03:00:21.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:21 smithi116 ceph-mon[31410]: mds.? [v2:172.21.15.177:6832/3132380205,v1:172.21.15.177:6833/3132380205] up:boot 2024-09-12T03:00:21.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:21 smithi116 ceph-mon[31410]: fsmap 2 up:standby 2024-09-12T03:00:21.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:21 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:00:21.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:21 smithi183 ceph-mon[35535]: Deploying daemon mds.j on smithi116 2024-09-12T03:00:21.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:21 smithi183 ceph-mon[35535]: mds.? [v2:172.21.15.177:6832/3132380205,v1:172.21.15.177:6833/3132380205] up:boot 2024-09-12T03:00:21.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:21 smithi183 ceph-mon[35535]: fsmap 2 up:standby 2024-09-12T03:00:21.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:21 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:00:21.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:21 smithi177 ceph-mon[35305]: Deploying daemon mds.j on smithi116 2024-09-12T03:00:21.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:21 smithi177 ceph-mon[35305]: mds.? [v2:172.21.15.177:6832/3132380205,v1:172.21.15.177:6833/3132380205] up:boot 2024-09-12T03:00:21.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:21 smithi177 ceph-mon[35305]: fsmap 2 up:standby 2024-09-12T03:00:21.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:21 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:00:22.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:22 smithi116 ceph-mon[31410]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:22.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:22 smithi183 ceph-mon[35535]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:22.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:22 smithi177 ceph-mon[35305]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:22.696 INFO:journalctl@ceph.mds.j.smithi116.stdout:Sep 12 03:00:22 smithi116 systemd[1]: Starting Ceph mds.j for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:00:22.788 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:00:22.950 INFO:journalctl@ceph.mds.j.smithi116.stdout:Sep 12 03:00:22 smithi116 podman[71373]: 2024-09-12 03:00:22.599265647 +0000 UTC m=+0.015954349 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:00:22.950 INFO:journalctl@ceph.mds.j.smithi116.stdout:Sep 12 03:00:22 smithi116 podman[71373]: 2024-09-12 03:00:22.702885147 +0000 UTC m=+0.119573846 container create 5b19ae306a97636689f4aae1bd9988fc9d4a71f505e5577e3d847e149bc7211b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-j, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-12T03:00:22.950 INFO:journalctl@ceph.mds.j.smithi116.stdout:Sep 12 03:00:22 smithi116 podman[71373]: 2024-09-12 03:00:22.947108335 +0000 UTC m=+0.363797043 container init 5b19ae306a97636689f4aae1bd9988fc9d4a71f505e5577e3d847e149bc7211b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-j, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3) 2024-09-12T03:00:22.950 INFO:journalctl@ceph.mds.j.smithi116.stdout:Sep 12 03:00:22 smithi116 podman[71373]: 2024-09-12 03:00:22.949994543 +0000 UTC m=+0.366683249 container start 5b19ae306a97636689f4aae1bd9988fc9d4a71f505e5577e3d847e149bc7211b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-12T03:00:23.128 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:23.249 INFO:journalctl@ceph.mds.j.smithi116.stdout:Sep 12 03:00:22 smithi116 ceph-mds[71410]: -- 172.21.15.116:0/1519830227 <== mon.0 v2:172.21.15.116:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556c41cf1860 con 0x556c41ccf400 2024-09-12T03:00:23.250 INFO:journalctl@ceph.mds.j.smithi116.stdout:Sep 12 03:00:22 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-j[71406]: starting mds.j at 2024-09-12T03:00:23.250 INFO:journalctl@ceph.mds.j.smithi116.stdout:Sep 12 03:00:23 smithi116 bash[71373]: 5b19ae306a97636689f4aae1bd9988fc9d4a71f505e5577e3d847e149bc7211b 2024-09-12T03:00:23.250 INFO:journalctl@ceph.mds.j.smithi116.stdout:Sep 12 03:00:23 smithi116 systemd[1]: Started Ceph mds.j for eb1ac602-70b1-11ef-bcea-c7b262605968. 2024-09-12T03:00:23.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.664+0000 7f8823bca640 1 -- 172.21.15.116:0/2809833845 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f881c100a80 msgr2=0x7f881c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:23.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.664+0000 7f8823bca640 1 --2- 172.21.15.116:0/2809833845 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f881c100a80 0x7f881c100ee0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8810009930 tx=0x7f881002a420 comp rx=0 tx=0).stop 2024-09-12T03:00:23.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.664+0000 7f8823bca640 1 -- 172.21.15.116:0/2809833845 shutdown_connections 2024-09-12T03:00:23.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.664+0000 7f8823bca640 1 --2- 172.21.15.116:0/2809833845 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f881c101420 0x7f881c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:23.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.664+0000 7f8823bca640 1 --2- 172.21.15.116:0/2809833845 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f881c100a80 0x7f881c100ee0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:23.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.664+0000 7f8823bca640 1 --2- 172.21.15.116:0/2809833845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f881c106a80 0x7f881c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:23.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.664+0000 7f8823bca640 1 -- 172.21.15.116:0/2809833845 >> 172.21.15.116:0/2809833845 conn(0x7f881c0fc7a0 msgr2=0x7f881c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:23.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.664+0000 7f8823bca640 1 -- 172.21.15.116:0/2809833845 shutdown_connections 2024-09-12T03:00:23.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.664+0000 7f8823bca640 1 -- 172.21.15.116:0/2809833845 wait complete. 2024-09-12T03:00:23.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.665+0000 7f8823bca640 1 Processor -- start 2024-09-12T03:00:23.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.665+0000 7f8823bca640 1 -- start start 2024-09-12T03:00:23.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.665+0000 7f8823bca640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f881c100a80 0x7f881c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:23.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.665+0000 7f8823bca640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f881c101420 0x7f881c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:23.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.665+0000 7f8823bca640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f881c106a80 0x7f881c19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:23.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.665+0000 7f8823bca640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f881c110e40 con 0x7f881c100a80 2024-09-12T03:00:23.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.665+0000 7f8823bca640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f881c110cc0 con 0x7f881c101420 2024-09-12T03:00:23.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.665+0000 7f8823bca640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f881c110fc0 con 0x7f881c106a80 2024-09-12T03:00:23.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.665+0000 7f8821966640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f881c100a80 0x7f881c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:23.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.665+0000 7f8821966640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f881c100a80 0x7f881c1983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59896/0 (socket says 172.21.15.116:59896) 2024-09-12T03:00:23.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.665+0000 7f8821966640 1 -- 172.21.15.116:0/3647436958 learned_addr learned my addr 172.21.15.116:0/3647436958 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:00:23.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.665+0000 7f8822167640 1 --2- 172.21.15.116:0/3647436958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f881c106a80 0x7f881c19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:23.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.665+0000 7f8821165640 1 --2- 172.21.15.116:0/3647436958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f881c101420 0x7f881c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:23.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.666+0000 7f8821966640 1 -- 172.21.15.116:0/3647436958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f881c106a80 msgr2=0x7f881c19cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:23.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.666+0000 7f8821966640 1 --2- 172.21.15.116:0/3647436958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f881c106a80 0x7f881c19cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:23.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.666+0000 7f8821966640 1 -- 172.21.15.116:0/3647436958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f881c101420 msgr2=0x7f881c198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:23.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.666+0000 7f8821966640 1 --2- 172.21.15.116:0/3647436958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f881c101420 0x7f881c198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:23.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.666+0000 7f8821966640 1 -- 172.21.15.116:0/3647436958 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881c19d370 con 0x7f881c100a80 2024-09-12T03:00:23.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.666+0000 7f8821165640 1 --2- 172.21.15.116:0/3647436958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f881c101420 0x7f881c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:00:23.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.666+0000 7f8821966640 1 --2- 172.21.15.116:0/3647436958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f881c100a80 0x7f881c1983c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f881800c920 tx=0x7f881800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:23.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.666+0000 7f880effd640 1 -- 172.21.15.116:0/3647436958 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818007c70 con 0x7f881c100a80 2024-09-12T03:00:23.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.666+0000 7f8823bca640 1 -- 172.21.15.116:0/3647436958 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f881c19d660 con 0x7f881c100a80 2024-09-12T03:00:23.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.666+0000 7f880effd640 1 -- 172.21.15.116:0/3647436958 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8818007e10 con 0x7f881c100a80 2024-09-12T03:00:23.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.666+0000 7f880effd640 1 -- 172.21.15.116:0/3647436958 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818018890 con 0x7f881c100a80 2024-09-12T03:00:23.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.666+0000 7f8823bca640 1 -- 172.21.15.116:0/3647436958 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f881c1a4f40 con 0x7f881c100a80 2024-09-12T03:00:23.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.667+0000 7f8823bca640 1 -- 172.21.15.116:0/3647436958 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f881c100ee0 con 0x7f881c100a80 2024-09-12T03:00:23.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.668+0000 7f880effd640 1 -- 172.21.15.116:0/3647436958 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8818018a70 con 0x7f881c100a80 2024-09-12T03:00:23.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.669+0000 7f880effd640 1 --2- 172.21.15.116:0/3647436958 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f88080a2b80 0x7f88080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:23.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.669+0000 7f8821165640 1 --2- 172.21.15.116:0/3647436958 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f88080a2b80 0x7f88080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:23.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.669+0000 7f880effd640 1 -- 172.21.15.116:0/3647436958 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f88180ceca0 con 0x7f881c100a80 2024-09-12T03:00:23.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.670+0000 7f8821165640 1 --2- 172.21.15.116:0/3647436958 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f88080a2b80 0x7f88080a5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8810000c00 tx=0x7f8810005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:23.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.674+0000 7f880effd640 1 -- 172.21.15.116:0/3647436958 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8818014030 con 0x7f881c100a80 2024-09-12T03:00:23.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.892+0000 7f8823bca640 1 -- 172.21.15.116:0/3647436958 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f881c10a080 con 0x7f881c100a80 2024-09-12T03:00:23.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.892+0000 7f880effd640 1 -- 172.21.15.116:0/3647436958 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3094 (secure 0 0 0) 0x7f881809c0a0 con 0x7f881c100a80 2024-09-12T03:00:23.893 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:00:23.893 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":4,"btime":"2024-09-12T03:00:23:167133+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T03:00:23.894 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 4 2024-09-12T03:00:23.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.894+0000 7f8823bca640 1 -- 172.21.15.116:0/3647436958 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f88080a2b80 msgr2=0x7f88080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:23.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.894+0000 7f8823bca640 1 --2- 172.21.15.116:0/3647436958 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f88080a2b80 0x7f88080a5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8810000c00 tx=0x7f8810005a20 comp rx=0 tx=0).stop 2024-09-12T03:00:23.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.894+0000 7f8823bca640 1 -- 172.21.15.116:0/3647436958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f881c100a80 msgr2=0x7f881c1983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:23.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.894+0000 7f8823bca640 1 --2- 172.21.15.116:0/3647436958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f881c100a80 0x7f881c1983c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f881800c920 tx=0x7f881800cde0 comp rx=0 tx=0).stop 2024-09-12T03:00:23.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.895+0000 7f8823bca640 1 -- 172.21.15.116:0/3647436958 shutdown_connections 2024-09-12T03:00:23.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.895+0000 7f8823bca640 1 --2- 172.21.15.116:0/3647436958 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f88080a2b80 0x7f88080a5040 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:23.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.895+0000 7f8823bca640 1 --2- 172.21.15.116:0/3647436958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f881c106a80 0x7f881c19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:23.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.895+0000 7f8823bca640 1 --2- 172.21.15.116:0/3647436958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f881c101420 0x7f881c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:23.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.895+0000 7f8823bca640 1 --2- 172.21.15.116:0/3647436958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f881c100a80 0x7f881c1983c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:23.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.895+0000 7f8823bca640 1 -- 172.21.15.116:0/3647436958 >> 172.21.15.116:0/3647436958 conn(0x7f881c0fc7a0 msgr2=0x7f881c06a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:23.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.895+0000 7f8823bca640 1 -- 172.21.15.116:0/3647436958 shutdown_connections 2024-09-12T03:00:23.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:23.895+0000 7f8823bca640 1 -- 172.21.15.116:0/3647436958 wait complete. 2024-09-12T03:00:24.355 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:24 smithi116 ceph-mon[31410]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:24.356 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:24 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:24.356 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:24 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:24.356 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:24 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:24.356 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:24 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:24.356 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:24 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:24.356 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:24 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:24.356 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:24 smithi116 ceph-mon[31410]: Deploying daemon mds.d on smithi116 2024-09-12T03:00:24.356 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:24 smithi116 ceph-mon[31410]: mds.? [v2:172.21.15.116:6834/3909394246,v1:172.21.15.116:6835/3909394246] up:boot 2024-09-12T03:00:24.356 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:24 smithi116 ceph-mon[31410]: fsmap 3 up:standby 2024-09-12T03:00:24.357 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:24 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T03:00:24.357 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:24 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3647436958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:24.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:24 smithi183 ceph-mon[35535]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:24.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:24 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:24.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:24 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:24.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:24 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:24.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:24 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:24.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:24 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:24.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:24 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:24.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:24 smithi183 ceph-mon[35535]: Deploying daemon mds.d on smithi116 2024-09-12T03:00:24.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:24 smithi183 ceph-mon[35535]: mds.? [v2:172.21.15.116:6834/3909394246,v1:172.21.15.116:6835/3909394246] up:boot 2024-09-12T03:00:24.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:24 smithi183 ceph-mon[35535]: fsmap 3 up:standby 2024-09-12T03:00:24.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:24 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T03:00:24.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:24 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3647436958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:24.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:24 smithi177 ceph-mon[35305]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:24.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:24 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:24.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:24 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:24.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:24 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:24.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:24 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:24.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:24 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:24.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:24 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:24.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:24 smithi177 ceph-mon[35305]: Deploying daemon mds.d on smithi116 2024-09-12T03:00:24.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:24 smithi177 ceph-mon[35305]: mds.? [v2:172.21.15.116:6834/3909394246,v1:172.21.15.116:6835/3909394246] up:boot 2024-09-12T03:00:24.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:24 smithi177 ceph-mon[35305]: fsmap 3 up:standby 2024-09-12T03:00:24.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:24 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T03:00:24.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:24 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3647436958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:26.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:26 smithi116 ceph-mon[31410]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:26.303 INFO:journalctl@ceph.mds.d.smithi116.stdout:Sep 12 03:00:26 smithi116 systemd[1]: Starting Ceph mds.d for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:00:26.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:26 smithi183 ceph-mon[35535]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:26.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:26 smithi177 ceph-mon[35305]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:26.677 INFO:journalctl@ceph.mds.d.smithi116.stdout:Sep 12 03:00:26 smithi116 podman[71787]: 2024-09-12 03:00:26.316419359 +0000 UTC m=+0.016281313 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:00:26.677 INFO:journalctl@ceph.mds.d.smithi116.stdout:Sep 12 03:00:26 smithi116 podman[71787]: 2024-09-12 03:00:26.432702061 +0000 UTC m=+0.132564013 container create f7f6d74490c1f932c47b7764c3eaa66e0982854a787add1eec1b263893224000 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:00:26.677 INFO:journalctl@ceph.mds.d.smithi116.stdout:Sep 12 03:00:26 smithi116 podman[71787]: 2024-09-12 03:00:26.677383638 +0000 UTC m=+0.377245582 container init f7f6d74490c1f932c47b7764c3eaa66e0982854a787add1eec1b263893224000 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-12T03:00:27.052 INFO:journalctl@ceph.mds.d.smithi116.stdout:Sep 12 03:00:26 smithi116 podman[71787]: 2024-09-12 03:00:26.680323669 +0000 UTC m=+0.380185616 container start f7f6d74490c1f932c47b7764c3eaa66e0982854a787add1eec1b263893224000 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-12T03:00:27.052 INFO:journalctl@ceph.mds.d.smithi116.stdout:Sep 12 03:00:26 smithi116 ceph-mds[71806]: -- 172.21.15.116:0/4203968569 <== mon.0 v2:172.21.15.116:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560fb5a7b860 con 0x560fb5a59000 2024-09-12T03:00:27.052 INFO:journalctl@ceph.mds.d.smithi116.stdout:Sep 12 03:00:26 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-d[71802]: starting mds.d at 2024-09-12T03:00:27.053 INFO:journalctl@ceph.mds.d.smithi116.stdout:Sep 12 03:00:26 smithi116 bash[71787]: f7f6d74490c1f932c47b7764c3eaa66e0982854a787add1eec1b263893224000 2024-09-12T03:00:27.053 INFO:journalctl@ceph.mds.d.smithi116.stdout:Sep 12 03:00:26 smithi116 systemd[1]: Started Ceph mds.d for eb1ac602-70b1-11ef-bcea-c7b262605968. 2024-09-12T03:00:27.247 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:00:27.501 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:27.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:27 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:27.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:27 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:27.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:27 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:27.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:27 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:27.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:27 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:27.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:27 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:27.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:27 smithi177 ceph-mon[35305]: Deploying daemon mds.e on smithi177 2024-09-12T03:00:27.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:27 smithi177 ceph-mon[35305]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:27.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:27 smithi177 ceph-mon[35305]: mds.? [v2:172.21.15.116:6836/3839272388,v1:172.21.15.116:6837/3839272388] up:boot 2024-09-12T03:00:27.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:27 smithi177 ceph-mon[35305]: fsmap 4 up:standby 2024-09-12T03:00:27.996 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:27 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T03:00:27.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.971+0000 7fe02b5dd640 1 -- 172.21.15.116:0/2175681250 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0241008c0 msgr2=0x7fe024100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:27.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.971+0000 7fe02b5dd640 1 --2- 172.21.15.116:0/2175681250 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0241008c0 0x7fe024100ca0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe020009900 tx=0x7fe02002a410 comp rx=0 tx=0).stop 2024-09-12T03:00:27.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.971+0000 7fe02b5dd640 1 -- 172.21.15.116:0/2175681250 shutdown_connections 2024-09-12T03:00:27.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.971+0000 7fe02b5dd640 1 --2- 172.21.15.116:0/2175681250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe024105ec0 0x7fe024106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:27.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.971+0000 7fe02b5dd640 1 --2- 172.21.15.116:0/2175681250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0241011e0 0x7fe024105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:27.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.971+0000 7fe02b5dd640 1 --2- 172.21.15.116:0/2175681250 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0241008c0 0x7fe024100ca0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:27.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.971+0000 7fe02b5dd640 1 -- 172.21.15.116:0/2175681250 >> 172.21.15.116:0/2175681250 conn(0x7fe0240fc7d0 msgr2=0x7fe0240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:27.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.972+0000 7fe02b5dd640 1 -- 172.21.15.116:0/2175681250 shutdown_connections 2024-09-12T03:00:27.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.972+0000 7fe02b5dd640 1 -- 172.21.15.116:0/2175681250 wait complete. 2024-09-12T03:00:27.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.972+0000 7fe02b5dd640 1 Processor -- start 2024-09-12T03:00:27.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.972+0000 7fe02b5dd640 1 -- start start 2024-09-12T03:00:27.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.972+0000 7fe02b5dd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0241008c0 0x7fe0241a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:27.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.972+0000 7fe02b5dd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0241011e0 0x7fe0241a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:27.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.972+0000 7fe02b5dd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe024105ec0 0x7fe02419e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:27.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.972+0000 7fe02b5dd640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe024118610 con 0x7fe024105ec0 2024-09-12T03:00:27.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.972+0000 7fe02b5dd640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe024118490 con 0x7fe0241011e0 2024-09-12T03:00:27.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.972+0000 7fe02b5dd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe024118790 con 0x7fe0241008c0 2024-09-12T03:00:27.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.973+0000 7fe029b7a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe024105ec0 0x7fe02419e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:27.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.973+0000 7fe029b7a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe024105ec0 0x7fe02419e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59950/0 (socket says 172.21.15.116:59950) 2024-09-12T03:00:27.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.973+0000 7fe029b7a640 1 -- 172.21.15.116:0/3410814123 learned_addr learned my addr 172.21.15.116:0/3410814123 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:00:27.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.973+0000 7fe028b78640 1 --2- 172.21.15.116:0/3410814123 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0241011e0 0x7fe0241a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:27.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.973+0000 7fe029379640 1 --2- 172.21.15.116:0/3410814123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0241008c0 0x7fe0241a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:27.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.973+0000 7fe029b7a640 1 -- 172.21.15.116:0/3410814123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0241008c0 msgr2=0x7fe0241a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:27.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.973+0000 7fe029b7a640 1 --2- 172.21.15.116:0/3410814123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0241008c0 0x7fe0241a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:27.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.973+0000 7fe029b7a640 1 -- 172.21.15.116:0/3410814123 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0241011e0 msgr2=0x7fe0241a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:27.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.973+0000 7fe029b7a640 1 --2- 172.21.15.116:0/3410814123 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0241011e0 0x7fe0241a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:27.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.973+0000 7fe029b7a640 1 -- 172.21.15.116:0/3410814123 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe02419eff0 con 0x7fe024105ec0 2024-09-12T03:00:27.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.973+0000 7fe029b7a640 1 --2- 172.21.15.116:0/3410814123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe024105ec0 0x7fe02419e8c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe01c009fd0 tx=0x7fe01c00def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:27.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.973+0000 7fe01a7fc640 1 -- 172.21.15.116:0/3410814123 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01c017070 con 0x7fe024105ec0 2024-09-12T03:00:27.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.973+0000 7fe01a7fc640 1 -- 172.21.15.116:0/3410814123 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe01c003e50 con 0x7fe024105ec0 2024-09-12T03:00:27.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.973+0000 7fe01a7fc640 1 -- 172.21.15.116:0/3410814123 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01c010460 con 0x7fe024105ec0 2024-09-12T03:00:27.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.973+0000 7fe02b5dd640 1 -- 172.21.15.116:0/3410814123 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe02419f2e0 con 0x7fe024105ec0 2024-09-12T03:00:28.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.973+0000 7fe02b5dd640 1 -- 172.21.15.116:0/3410814123 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0241ab5d0 con 0x7fe024105ec0 2024-09-12T03:00:28.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.974+0000 7fe02b5dd640 1 -- 172.21.15.116:0/3410814123 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe02406a850 con 0x7fe024105ec0 2024-09-12T03:00:28.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.976+0000 7fe01a7fc640 1 -- 172.21.15.116:0/3410814123 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe01c004000 con 0x7fe024105ec0 2024-09-12T03:00:28.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.976+0000 7fe01a7fc640 1 --2- 172.21.15.116:0/3410814123 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe0100a2b80 0x7fe0100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:28.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.976+0000 7fe01a7fc640 1 -- 172.21.15.116:0/3410814123 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fe01c0ce800 con 0x7fe024105ec0 2024-09-12T03:00:28.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.976+0000 7fe029379640 1 --2- 172.21.15.116:0/3410814123 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe0100a2b80 0x7fe0100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:28.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.977+0000 7fe029379640 1 --2- 172.21.15.116:0/3410814123 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe0100a2b80 0x7fe0100a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe020000c00 tx=0x7fe020005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:28.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:27.980+0000 7fe01a7fc640 1 -- 172.21.15.116:0/3410814123 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe01c09bc00 con 0x7fe024105ec0 2024-09-12T03:00:28.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:28.188+0000 7fe02b5dd640 1 -- 172.21.15.116:0/3410814123 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe024105980 con 0x7fe024105ec0 2024-09-12T03:00:28.188 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:27 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:28.189 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:27 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:28.189 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:27 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:28.189 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:27 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:28.189 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:27 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:28.189 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:27 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:28.189 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:27 smithi116 ceph-mon[31410]: Deploying daemon mds.e on smithi177 2024-09-12T03:00:28.189 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:27 smithi116 ceph-mon[31410]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:28.189 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:27 smithi116 ceph-mon[31410]: mds.? [v2:172.21.15.116:6836/3839272388,v1:172.21.15.116:6837/3839272388] up:boot 2024-09-12T03:00:28.189 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:27 smithi116 ceph-mon[31410]: fsmap 4 up:standby 2024-09-12T03:00:28.189 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:27 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T03:00:28.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:28.188+0000 7fe01a7fc640 1 -- 172.21.15.116:0/3410814123 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3922 (secure 0 0 0) 0x7fe024105980 con 0x7fe024105ec0 2024-09-12T03:00:28.190 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:00:28.190 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":5,"btime":"2024-09-12T03:00:27:107781+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T03:00:28.190 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 5 2024-09-12T03:00:28.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:28.191+0000 7fe02b5dd640 1 -- 172.21.15.116:0/3410814123 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe0100a2b80 msgr2=0x7fe0100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:28.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:28.191+0000 7fe02b5dd640 1 --2- 172.21.15.116:0/3410814123 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe0100a2b80 0x7fe0100a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe020000c00 tx=0x7fe020005a20 comp rx=0 tx=0).stop 2024-09-12T03:00:28.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:28.191+0000 7fe02b5dd640 1 -- 172.21.15.116:0/3410814123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe024105ec0 msgr2=0x7fe02419e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:28.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:28.191+0000 7fe02b5dd640 1 --2- 172.21.15.116:0/3410814123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe024105ec0 0x7fe02419e8c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe01c009fd0 tx=0x7fe01c00def0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:28.191+0000 7fe02b5dd640 1 -- 172.21.15.116:0/3410814123 shutdown_connections 2024-09-12T03:00:28.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:28.191+0000 7fe02b5dd640 1 --2- 172.21.15.116:0/3410814123 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe0100a2b80 0x7fe0100a5040 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:28.191+0000 7fe02b5dd640 1 --2- 172.21.15.116:0/3410814123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe024105ec0 0x7fe02419e8c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:28.191+0000 7fe02b5dd640 1 --2- 172.21.15.116:0/3410814123 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0241011e0 0x7fe0241a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:28.191+0000 7fe02b5dd640 1 --2- 172.21.15.116:0/3410814123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0241008c0 0x7fe0241a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:28.191+0000 7fe02b5dd640 1 -- 172.21.15.116:0/3410814123 >> 172.21.15.116:0/3410814123 conn(0x7fe0240fc7d0 msgr2=0x7fe024103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:28.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:28.191+0000 7fe02b5dd640 1 -- 172.21.15.116:0/3410814123 shutdown_connections 2024-09-12T03:00:28.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:28.191+0000 7fe02b5dd640 1 -- 172.21.15.116:0/3410814123 wait complete. 2024-09-12T03:00:28.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:27 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:28.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:27 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:28.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:27 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:28.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:27 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:28.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:27 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:28.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:27 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:28.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:27 smithi183 ceph-mon[35535]: Deploying daemon mds.e on smithi177 2024-09-12T03:00:28.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:27 smithi183 ceph-mon[35535]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:28.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:27 smithi183 ceph-mon[35535]: mds.? [v2:172.21.15.116:6836/3839272388,v1:172.21.15.116:6837/3839272388] up:boot 2024-09-12T03:00:28.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:27 smithi183 ceph-mon[35535]: fsmap 4 up:standby 2024-09-12T03:00:28.204 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:27 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T03:00:29.165 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:28 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3410814123' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:29.165 INFO:journalctl@ceph.mds.e.smithi177.stdout:Sep 12 03:00:28 smithi177 systemd[1]: Starting Ceph mds.e for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:00:29.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:28 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3410814123' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:29.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:28 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3410814123' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:29.426 INFO:journalctl@ceph.mds.e.smithi177.stdout:Sep 12 03:00:29 smithi177 podman[64711]: 2024-09-12 03:00:29.165286645 +0000 UTC m=+0.109647788 container create 735c3254bcdd08ecc6ffacf4b1af20e25214483c31229bbd04aebb4bd8c5a430 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-12T03:00:29.426 INFO:journalctl@ceph.mds.e.smithi177.stdout:Sep 12 03:00:29 smithi177 podman[64711]: 2024-09-12 03:00:29.071393841 +0000 UTC m=+0.015755002 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:00:29.426 INFO:journalctl@ceph.mds.e.smithi177.stdout:Sep 12 03:00:29 smithi177 podman[64711]: 2024-09-12 03:00:29.426642756 +0000 UTC m=+0.371003898 container init 735c3254bcdd08ecc6ffacf4b1af20e25214483c31229bbd04aebb4bd8c5a430 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-e, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:00:29.745 INFO:journalctl@ceph.mds.e.smithi177.stdout:Sep 12 03:00:29 smithi177 podman[64711]: 2024-09-12 03:00:29.429873672 +0000 UTC m=+0.374234815 container start 735c3254bcdd08ecc6ffacf4b1af20e25214483c31229bbd04aebb4bd8c5a430 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-e, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-12T03:00:29.745 INFO:journalctl@ceph.mds.e.smithi177.stdout:Sep 12 03:00:29 smithi177 ceph-mds[64730]: -- 172.21.15.177:0/2530547765 <== mon.2 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d339509860 con 0x55d3394e7000 2024-09-12T03:00:29.745 INFO:journalctl@ceph.mds.e.smithi177.stdout:Sep 12 03:00:29 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-e[64726]: starting mds.e at 2024-09-12T03:00:29.745 INFO:journalctl@ceph.mds.e.smithi177.stdout:Sep 12 03:00:29 smithi177 bash[64711]: 735c3254bcdd08ecc6ffacf4b1af20e25214483c31229bbd04aebb4bd8c5a430 2024-09-12T03:00:29.745 INFO:journalctl@ceph.mds.e.smithi177.stdout:Sep 12 03:00:29 smithi177 systemd[1]: Started Ceph mds.e for eb1ac602-70b1-11ef-bcea-c7b262605968. 2024-09-12T03:00:30.148 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:29 smithi183 ceph-mon[35535]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:30.148 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:29 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:30.148 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:29 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:30.148 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:29 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:30.148 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:29 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:30.148 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:29 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:30.148 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:29 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:30.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:29 smithi177 ceph-mon[35305]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:30.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:29 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:30.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:29 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:30.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:29 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:30.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:29 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:30.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:29 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:30.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:29 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:30.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:29 smithi116 ceph-mon[31410]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:30.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:29 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:30.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:29 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:30.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:29 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:30.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:29 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:30.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:29 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:30.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:29 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:30.895 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:00:31.123 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:30 smithi183 ceph-mon[35535]: Deploying daemon mds.f on smithi183 2024-09-12T03:00:31.123 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:30 smithi183 ceph-mon[35535]: mds.? [v2:172.21.15.177:6834/2224643300,v1:172.21.15.177:6835/2224643300] up:boot 2024-09-12T03:00:31.123 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:30 smithi183 ceph-mon[35535]: fsmap 5 up:standby 2024-09-12T03:00:31.123 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:30 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:00:31.155 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:31.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:30 smithi177 ceph-mon[35305]: Deploying daemon mds.f on smithi183 2024-09-12T03:00:31.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:30 smithi177 ceph-mon[35305]: mds.? [v2:172.21.15.177:6834/2224643300,v1:172.21.15.177:6835/2224643300] up:boot 2024-09-12T03:00:31.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:30 smithi177 ceph-mon[35305]: fsmap 5 up:standby 2024-09-12T03:00:31.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:30 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:00:31.276 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:30 smithi116 ceph-mon[31410]: Deploying daemon mds.f on smithi183 2024-09-12T03:00:31.276 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:30 smithi116 ceph-mon[31410]: mds.? [v2:172.21.15.177:6834/2224643300,v1:172.21.15.177:6835/2224643300] up:boot 2024-09-12T03:00:31.276 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:30 smithi116 ceph-mon[31410]: fsmap 5 up:standby 2024-09-12T03:00:31.276 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:30 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:00:31.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.641+0000 7fa8282ba640 1 -- 172.21.15.116:0/627543073 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa820106a80 msgr2=0x7fa820106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:31.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.641+0000 7fa8282ba640 1 --2- 172.21.15.116:0/627543073 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa820106a80 0x7fa820106e60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009930 tx=0x7fa81c02a420 comp rx=0 tx=0).stop 2024-09-12T03:00:31.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.642+0000 7fa8282ba640 1 -- 172.21.15.116:0/627543073 shutdown_connections 2024-09-12T03:00:31.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.642+0000 7fa8282ba640 1 --2- 172.21.15.116:0/627543073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa820101420 0x7fa82010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:31.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.642+0000 7fa8282ba640 1 --2- 172.21.15.116:0/627543073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa820100a80 0x7fa820100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:31.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.642+0000 7fa8282ba640 1 --2- 172.21.15.116:0/627543073 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa820106a80 0x7fa820106e60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:31.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.642+0000 7fa8282ba640 1 -- 172.21.15.116:0/627543073 >> 172.21.15.116:0/627543073 conn(0x7fa8200fc7a0 msgr2=0x7fa8200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:31.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.642+0000 7fa8282ba640 1 -- 172.21.15.116:0/627543073 shutdown_connections 2024-09-12T03:00:31.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.642+0000 7fa8282ba640 1 -- 172.21.15.116:0/627543073 wait complete. 2024-09-12T03:00:31.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.642+0000 7fa8282ba640 1 Processor -- start 2024-09-12T03:00:31.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.642+0000 7fa8282ba640 1 -- start start 2024-09-12T03:00:31.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.642+0000 7fa8282ba640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa820100a80 0x7fa8200fce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:31.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.642+0000 7fa8282ba640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa820101420 0x7fa8200fd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:31.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.642+0000 7fa8282ba640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa820106a80 0x7fa8200fd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:31.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.642+0000 7fa8282ba640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa820110ed0 con 0x7fa820100a80 2024-09-12T03:00:31.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.642+0000 7fa8282ba640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa820110d50 con 0x7fa820106a80 2024-09-12T03:00:31.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.642+0000 7fa8282ba640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa820111050 con 0x7fa820101420 2024-09-12T03:00:31.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.643+0000 7fa826056640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa820100a80 0x7fa8200fce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:31.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.643+0000 7fa826056640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa820100a80 0x7fa8200fce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42180/0 (socket says 172.21.15.116:42180) 2024-09-12T03:00:31.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.643+0000 7fa826056640 1 -- 172.21.15.116:0/2383322314 learned_addr learned my addr 172.21.15.116:0/2383322314 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:00:31.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.643+0000 7fa825855640 1 --2- 172.21.15.116:0/2383322314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa820101420 0x7fa8200fd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:31.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.643+0000 7fa826857640 1 --2- 172.21.15.116:0/2383322314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa820106a80 0x7fa8200fd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:31.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.643+0000 7fa826056640 1 -- 172.21.15.116:0/2383322314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa820101420 msgr2=0x7fa8200fd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:31.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.643+0000 7fa826056640 1 --2- 172.21.15.116:0/2383322314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa820101420 0x7fa8200fd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:31.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.643+0000 7fa826056640 1 -- 172.21.15.116:0/2383322314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa820106a80 msgr2=0x7fa8200fd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:31.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.643+0000 7fa826056640 1 --2- 172.21.15.116:0/2383322314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa820106a80 0x7fa8200fd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:31.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.643+0000 7fa826056640 1 -- 172.21.15.116:0/2383322314 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8201a68d0 con 0x7fa820100a80 2024-09-12T03:00:31.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.643+0000 7fa826056640 1 --2- 172.21.15.116:0/2383322314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa820100a80 0x7fa8200fce30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa81c02a940 tx=0x7fa81c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:31.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.643+0000 7fa8177fe640 1 -- 172.21.15.116:0/2383322314 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c03f070 con 0x7fa820100a80 2024-09-12T03:00:31.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.643+0000 7fa8177fe640 1 -- 172.21.15.116:0/2383322314 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa81c031c90 con 0x7fa820100a80 2024-09-12T03:00:31.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.643+0000 7fa8282ba640 1 -- 172.21.15.116:0/2383322314 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8201a6b60 con 0x7fa820100a80 2024-09-12T03:00:31.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.643+0000 7fa8177fe640 1 -- 172.21.15.116:0/2383322314 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c035030 con 0x7fa820100a80 2024-09-12T03:00:31.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.643+0000 7fa8282ba640 1 -- 172.21.15.116:0/2383322314 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8201a7090 con 0x7fa820100a80 2024-09-12T03:00:31.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.644+0000 7fa8282ba640 1 -- 172.21.15.116:0/2383322314 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa820100ee0 con 0x7fa820100a80 2024-09-12T03:00:31.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.645+0000 7fa8177fe640 1 -- 172.21.15.116:0/2383322314 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa81c042070 con 0x7fa820100a80 2024-09-12T03:00:31.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.646+0000 7fa8177fe640 1 --2- 172.21.15.116:0/2383322314 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa80c0a2c00 0x7fa80c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:31.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.646+0000 7fa825855640 1 --2- 172.21.15.116:0/2383322314 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa80c0a2c00 0x7fa80c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:31.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.646+0000 7fa8177fe640 1 -- 172.21.15.116:0/2383322314 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fa81c0ef210 con 0x7fa820100a80 2024-09-12T03:00:31.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.647+0000 7fa825855640 1 --2- 172.21.15.116:0/2383322314 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa80c0a2c00 0x7fa80c0a50c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa8200fe8c0 tx=0x7fa81000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:31.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.650+0000 7fa8177fe640 1 -- 172.21.15.116:0/2383322314 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa81c0bbfc0 con 0x7fa820100a80 2024-09-12T03:00:31.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.851+0000 7fa8282ba640 1 -- 172.21.15.116:0/2383322314 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa82010a000 con 0x7fa820100a80 2024-09-12T03:00:31.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.852+0000 7fa8177fe640 1 -- 172.21.15.116:0/2383322314 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4750 (secure 0 0 0) 0x7fa81c0c02c0 con 0x7fa820100a80 2024-09-12T03:00:31.852 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:00:31.852 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":6,"btime":"2024-09-12T03:00:29:856761+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-12T03:00:31.853 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 6 2024-09-12T03:00:31.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.854+0000 7fa8282ba640 1 -- 172.21.15.116:0/2383322314 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa80c0a2c00 msgr2=0x7fa80c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:31.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.854+0000 7fa8282ba640 1 --2- 172.21.15.116:0/2383322314 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa80c0a2c00 0x7fa80c0a50c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa8200fe8c0 tx=0x7fa81000b390 comp rx=0 tx=0).stop 2024-09-12T03:00:31.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.854+0000 7fa8282ba640 1 -- 172.21.15.116:0/2383322314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa820100a80 msgr2=0x7fa8200fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:31.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.854+0000 7fa8282ba640 1 --2- 172.21.15.116:0/2383322314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa820100a80 0x7fa8200fce30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa81c02a940 tx=0x7fa81c003b90 comp rx=0 tx=0).stop 2024-09-12T03:00:31.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.854+0000 7fa8282ba640 1 -- 172.21.15.116:0/2383322314 shutdown_connections 2024-09-12T03:00:31.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.854+0000 7fa8282ba640 1 --2- 172.21.15.116:0/2383322314 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa80c0a2c00 0x7fa80c0a50c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:31.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.854+0000 7fa8282ba640 1 --2- 172.21.15.116:0/2383322314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa820106a80 0x7fa8200fd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:31.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.854+0000 7fa8282ba640 1 --2- 172.21.15.116:0/2383322314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa820101420 0x7fa8200fd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:31.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.854+0000 7fa8282ba640 1 --2- 172.21.15.116:0/2383322314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa820100a80 0x7fa8200fce30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:31.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.854+0000 7fa8282ba640 1 -- 172.21.15.116:0/2383322314 >> 172.21.15.116:0/2383322314 conn(0x7fa8200fc7a0 msgr2=0x7fa82006aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:31.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.854+0000 7fa8282ba640 1 -- 172.21.15.116:0/2383322314 shutdown_connections 2024-09-12T03:00:31.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:31.854+0000 7fa8282ba640 1 -- 172.21.15.116:0/2383322314 wait complete. 2024-09-12T03:00:32.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:31 smithi183 ceph-mon[35535]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:32.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:31 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2383322314' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:32.203 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 03:00:32 smithi183 systemd[1]: Started Ceph mds.f for eb1ac602-70b1-11ef-bcea-c7b262605968. 2024-09-12T03:00:32.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:31 smithi177 ceph-mon[35305]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:32.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:31 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2383322314' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:32.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:31 smithi116 ceph-mon[31410]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:32.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:31 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2383322314' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:33.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:33 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:33.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:33 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:33.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:33 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:33.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:33 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:33.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:33 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:33.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:33 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:33.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:33 smithi183 ceph-mon[35535]: Deploying daemon mds.c on smithi183 2024-09-12T03:00:33.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:33 smithi183 ceph-mon[35535]: mds.? [v2:172.21.15.183:6834/4028826399,v1:172.21.15.183:6835/4028826399] up:boot 2024-09-12T03:00:33.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:33 smithi183 ceph-mon[35535]: fsmap 6 up:standby 2024-09-12T03:00:33.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:33 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:00:33.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:33 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:33.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:33 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:33.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:33 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:33.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:33 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:33.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:33 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:33.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:33 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:33.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:33 smithi177 ceph-mon[35305]: Deploying daemon mds.c on smithi183 2024-09-12T03:00:33.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:33 smithi177 ceph-mon[35305]: mds.? [v2:172.21.15.183:6834/4028826399,v1:172.21.15.183:6835/4028826399] up:boot 2024-09-12T03:00:33.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:33 smithi177 ceph-mon[35305]: fsmap 6 up:standby 2024-09-12T03:00:33.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:33 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:00:33.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:33 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:33.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:33 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:33.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:33 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:33.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:33 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:33.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:33 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:33.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:33 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:33.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:33 smithi116 ceph-mon[31410]: Deploying daemon mds.c on smithi183 2024-09-12T03:00:33.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:33 smithi116 ceph-mon[31410]: mds.? [v2:172.21.15.183:6834/4028826399,v1:172.21.15.183:6835/4028826399] up:boot 2024-09-12T03:00:33.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:33 smithi116 ceph-mon[31410]: fsmap 6 up:standby 2024-09-12T03:00:33.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:33 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:00:34.103 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 03:00:34 smithi183 systemd[1]: Starting Ceph mds.c for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:00:34.370 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:34 smithi183 ceph-mon[35535]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:34.370 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 03:00:34 smithi183 podman[65755]: 2024-09-12 03:00:34.137992993 +0000 UTC m=+0.015795106 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:00:34.371 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 03:00:34 smithi183 podman[65755]: 2024-09-12 03:00:34.255434273 +0000 UTC m=+0.133236424 container create 7fbbfbd37f266079448dfa64480626c73ce7935d437e3a9e10357d79f69ef43a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-12T03:00:34.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:34 smithi177 ceph-mon[35305]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:34.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:34 smithi116 ceph-mon[31410]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:34.613 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:00:34.882 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:34.953 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 03:00:34 smithi183 podman[65755]: 2024-09-12 03:00:34.483108062 +0000 UTC m=+0.360910170 container init 7fbbfbd37f266079448dfa64480626c73ce7935d437e3a9e10357d79f69ef43a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-c, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-12T03:00:34.953 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 03:00:34 smithi183 podman[65755]: 2024-09-12 03:00:34.485804694 +0000 UTC m=+0.363606806 container start 7fbbfbd37f266079448dfa64480626c73ce7935d437e3a9e10357d79f69ef43a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-c, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-12T03:00:34.953 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 03:00:34 smithi183 ceph-mds[65775]: -- 172.21.15.183:0/3591816662 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559622ce9860 con 0x559622cc7400 2024-09-12T03:00:34.953 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 03:00:34 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-c[65771]: starting mds.c at 2024-09-12T03:00:34.953 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 03:00:34 smithi183 bash[65755]: 7fbbfbd37f266079448dfa64480626c73ce7935d437e3a9e10357d79f69ef43a 2024-09-12T03:00:34.953 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 03:00:34 smithi183 systemd[1]: Started Ceph mds.c for eb1ac602-70b1-11ef-bcea-c7b262605968. 2024-09-12T03:00:35.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.384+0000 7fb939873640 1 -- 172.21.15.116:0/2955297836 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb934101420 msgr2=0x7fb93410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:35.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.384+0000 7fb939873640 1 --2- 172.21.15.116:0/2955297836 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb934101420 0x7fb93410d950 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb924009930 tx=0x7fb92402a420 comp rx=0 tx=0).stop 2024-09-12T03:00:35.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.384+0000 7fb939873640 1 -- 172.21.15.116:0/2955297836 shutdown_connections 2024-09-12T03:00:35.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.384+0000 7fb939873640 1 --2- 172.21.15.116:0/2955297836 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb934101420 0x7fb93410d950 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:35.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.384+0000 7fb939873640 1 --2- 172.21.15.116:0/2955297836 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb934100a80 0x7fb934100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:35.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.384+0000 7fb939873640 1 --2- 172.21.15.116:0/2955297836 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb934106a80 0x7fb934106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:35.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.384+0000 7fb939873640 1 -- 172.21.15.116:0/2955297836 >> 172.21.15.116:0/2955297836 conn(0x7fb9340fc7a0 msgr2=0x7fb9340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:35.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.384+0000 7fb939873640 1 -- 172.21.15.116:0/2955297836 shutdown_connections 2024-09-12T03:00:35.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.384+0000 7fb939873640 1 -- 172.21.15.116:0/2955297836 wait complete. 2024-09-12T03:00:35.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.384+0000 7fb939873640 1 Processor -- start 2024-09-12T03:00:35.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.385+0000 7fb939873640 1 -- start start 2024-09-12T03:00:35.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.385+0000 7fb939873640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb934100a80 0x7fb9341992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:35.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.385+0000 7fb939873640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb934101420 0x7fb9341997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:35.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.385+0000 7fb939873640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb934106a80 0x7fb9341a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:35.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.385+0000 7fb939873640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb934110e80 con 0x7fb934100a80 2024-09-12T03:00:35.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.385+0000 7fb939873640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb934110d00 con 0x7fb934101420 2024-09-12T03:00:35.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.385+0000 7fb939873640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb934111000 con 0x7fb934106a80 2024-09-12T03:00:35.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.385+0000 7fb932ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb934100a80 0x7fb9341992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:35.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.385+0000 7fb932ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb934100a80 0x7fb9341992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42196/0 (socket says 172.21.15.116:42196) 2024-09-12T03:00:35.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.385+0000 7fb932ffd640 1 -- 172.21.15.116:0/470265925 learned_addr learned my addr 172.21.15.116:0/470265925 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:00:35.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.385+0000 7fb9327fc640 1 --2- 172.21.15.116:0/470265925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb934101420 0x7fb9341997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:35.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.385+0000 7fb9337fe640 1 --2- 172.21.15.116:0/470265925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb934106a80 0x7fb9341a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:35.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.385+0000 7fb932ffd640 1 -- 172.21.15.116:0/470265925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb934106a80 msgr2=0x7fb9341a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:35.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.385+0000 7fb932ffd640 1 --2- 172.21.15.116:0/470265925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb934106a80 0x7fb9341a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:35.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.385+0000 7fb932ffd640 1 -- 172.21.15.116:0/470265925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb934101420 msgr2=0x7fb9341997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:35.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.385+0000 7fb932ffd640 1 --2- 172.21.15.116:0/470265925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb934101420 0x7fb9341997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:35.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.385+0000 7fb932ffd640 1 -- 172.21.15.116:0/470265925 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9341a4870 con 0x7fb934100a80 2024-09-12T03:00:35.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.385+0000 7fb9327fc640 1 --2- 172.21.15.116:0/470265925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb934101420 0x7fb9341997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:00:35.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.385+0000 7fb932ffd640 1 --2- 172.21.15.116:0/470265925 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb934100a80 0x7fb9341992b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00e9f0 tx=0x7fb92c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:35.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.386+0000 7fb938871640 1 -- 172.21.15.116:0/470265925 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c00cd80 con 0x7fb934100a80 2024-09-12T03:00:35.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.386+0000 7fb938871640 1 -- 172.21.15.116:0/470265925 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb92c010910 con 0x7fb934100a80 2024-09-12T03:00:35.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.386+0000 7fb939873640 1 -- 172.21.15.116:0/470265925 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9341a4a40 con 0x7fb934100a80 2024-09-12T03:00:35.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.386+0000 7fb938871640 1 -- 172.21.15.116:0/470265925 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c018990 con 0x7fb934100a80 2024-09-12T03:00:35.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.386+0000 7fb939873640 1 -- 172.21.15.116:0/470265925 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9341a4f50 con 0x7fb934100a80 2024-09-12T03:00:35.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.387+0000 7fb939873640 1 -- 172.21.15.116:0/470265925 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb934100ee0 con 0x7fb934100a80 2024-09-12T03:00:35.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.388+0000 7fb938871640 1 -- 172.21.15.116:0/470265925 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fb92c010ac0 con 0x7fb934100a80 2024-09-12T03:00:35.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.389+0000 7fb938871640 1 --2- 172.21.15.116:0/470265925 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb9200a2b80 0x7fb9200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:35.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.389+0000 7fb9327fc640 1 --2- 172.21.15.116:0/470265925 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb9200a2b80 0x7fb9200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:35.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.389+0000 7fb938871640 1 -- 172.21.15.116:0/470265925 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fb92c0cee30 con 0x7fb934100a80 2024-09-12T03:00:35.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.389+0000 7fb9327fc640 1 --2- 172.21.15.116:0/470265925 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb9200a2b80 0x7fb9200a5040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb91c009fd0 tx=0x7fb91c009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:35.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.393+0000 7fb938871640 1 -- 172.21.15.116:0/470265925 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb92c09bbe0 con 0x7fb934100a80 2024-09-12T03:00:35.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.593+0000 7fb939873640 1 -- 172.21.15.116:0/470265925 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb934109010 con 0x7fb934100a80 2024-09-12T03:00:35.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.594+0000 7fb938871640 1 -- 172.21.15.116:0/470265925 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6406 (secure 0 0 0) 0x7fb92c09fee0 con 0x7fb934100a80 2024-09-12T03:00:35.594 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:00:35.594 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":8,"btime":"2024-09-12T03:00:35:104021+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-12T03:00:35.595 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 8 2024-09-12T03:00:35.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.597+0000 7fb939873640 1 -- 172.21.15.116:0/470265925 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb9200a2b80 msgr2=0x7fb9200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:35.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.597+0000 7fb939873640 1 --2- 172.21.15.116:0/470265925 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb9200a2b80 0x7fb9200a5040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb91c009fd0 tx=0x7fb91c009680 comp rx=0 tx=0).stop 2024-09-12T03:00:35.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.597+0000 7fb939873640 1 -- 172.21.15.116:0/470265925 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb934100a80 msgr2=0x7fb9341992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:35.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.597+0000 7fb939873640 1 --2- 172.21.15.116:0/470265925 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb934100a80 0x7fb9341992b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00e9f0 tx=0x7fb92c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:00:35.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.597+0000 7fb939873640 1 -- 172.21.15.116:0/470265925 shutdown_connections 2024-09-12T03:00:35.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.597+0000 7fb939873640 1 --2- 172.21.15.116:0/470265925 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb9200a2b80 0x7fb9200a5040 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:35.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.597+0000 7fb939873640 1 --2- 172.21.15.116:0/470265925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb934106a80 0x7fb9341a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:35.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.597+0000 7fb939873640 1 --2- 172.21.15.116:0/470265925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb934101420 0x7fb9341997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:35.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.597+0000 7fb939873640 1 --2- 172.21.15.116:0/470265925 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb934100a80 0x7fb9341992b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:35.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.597+0000 7fb939873640 1 -- 172.21.15.116:0/470265925 >> 172.21.15.116:0/470265925 conn(0x7fb9340fc7a0 msgr2=0x7fb9340fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:35.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.597+0000 7fb939873640 1 -- 172.21.15.116:0/470265925 shutdown_connections 2024-09-12T03:00:35.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:35.597+0000 7fb939873640 1 -- 172.21.15.116:0/470265925 wait complete. 2024-09-12T03:00:35.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:35 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:35.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:35 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:35.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:35 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:35.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:35 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:35.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:35 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:35.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:35 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:35.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:35 smithi183 ceph-mon[35535]: Deploying daemon mds.h on smithi177 2024-09-12T03:00:35.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:35 smithi183 ceph-mon[35535]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:35.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:35 smithi183 ceph-mon[35535]: mds.? [v2:172.21.15.183:6836/1288373243,v1:172.21.15.183:6837/1288373243] up:boot 2024-09-12T03:00:35.956 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:35 smithi183 ceph-mon[35535]: fsmap 7 up:standby 2024-09-12T03:00:35.957 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:35 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:00:35.957 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:35 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/470265925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:35.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:35 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:35.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:35 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:35.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:35 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:35.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:35 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:35.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:35 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:35.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:35 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:35.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:35 smithi177 ceph-mon[35305]: Deploying daemon mds.h on smithi177 2024-09-12T03:00:35.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:35 smithi177 ceph-mon[35305]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:35.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:35 smithi177 ceph-mon[35305]: mds.? [v2:172.21.15.183:6836/1288373243,v1:172.21.15.183:6837/1288373243] up:boot 2024-09-12T03:00:35.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:35 smithi177 ceph-mon[35305]: fsmap 7 up:standby 2024-09-12T03:00:35.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:35 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:00:35.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:35 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/470265925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:36.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:35 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:36.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:35 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:36.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:35 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:36.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:35 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:36.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:35 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:36.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:35 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:36.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:35 smithi116 ceph-mon[31410]: Deploying daemon mds.h on smithi177 2024-09-12T03:00:36.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:35 smithi116 ceph-mon[31410]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:36.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:35 smithi116 ceph-mon[31410]: mds.? [v2:172.21.15.183:6836/1288373243,v1:172.21.15.183:6837/1288373243] up:boot 2024-09-12T03:00:36.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:35 smithi116 ceph-mon[31410]: fsmap 7 up:standby 2024-09-12T03:00:36.040 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:35 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:00:36.040 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:35 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/470265925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:36.745 INFO:journalctl@ceph.mds.h.smithi177.stdout:Sep 12 03:00:36 smithi177 systemd[1]: Starting Ceph mds.h for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:00:37.149 INFO:journalctl@ceph.mds.h.smithi177.stdout:Sep 12 03:00:36 smithi177 podman[65013]: 2024-09-12 03:00:36.763783144 +0000 UTC m=+0.015955606 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:00:37.149 INFO:journalctl@ceph.mds.h.smithi177.stdout:Sep 12 03:00:36 smithi177 podman[65013]: 2024-09-12 03:00:36.887965449 +0000 UTC m=+0.140137901 container create 0aabcdea56ce5275b17ac77b009dcb5884a1fea5be9c05803f960a0411750cd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-h, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-09-12T03:00:37.149 INFO:journalctl@ceph.mds.h.smithi177.stdout:Sep 12 03:00:37 smithi177 podman[65013]: 2024-09-12 03:00:37.149724326 +0000 UTC m=+0.401896778 container init 0aabcdea56ce5275b17ac77b009dcb5884a1fea5be9c05803f960a0411750cd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD) 2024-09-12T03:00:37.495 INFO:journalctl@ceph.mds.h.smithi177.stdout:Sep 12 03:00:37 smithi177 podman[65013]: 2024-09-12 03:00:37.152824364 +0000 UTC m=+0.404996817 container start 0aabcdea56ce5275b17ac77b009dcb5884a1fea5be9c05803f960a0411750cd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-h, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-12T03:00:37.495 INFO:journalctl@ceph.mds.h.smithi177.stdout:Sep 12 03:00:37 smithi177 ceph-mds[65032]: -- 172.21.15.177:0/3885443102 <== mon.2 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b0aa39b860 con 0x55b0aa379000 2024-09-12T03:00:37.495 INFO:journalctl@ceph.mds.h.smithi177.stdout:Sep 12 03:00:37 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-h[65028]: starting mds.h at 2024-09-12T03:00:37.495 INFO:journalctl@ceph.mds.h.smithi177.stdout:Sep 12 03:00:37 smithi177 bash[65013]: 0aabcdea56ce5275b17ac77b009dcb5884a1fea5be9c05803f960a0411750cd4 2024-09-12T03:00:37.495 INFO:journalctl@ceph.mds.h.smithi177.stdout:Sep 12 03:00:37 smithi177 systemd[1]: Started Ceph mds.h for eb1ac602-70b1-11ef-bcea-c7b262605968. 2024-09-12T03:00:38.311 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:00:38.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:38 smithi183 ceph-mon[35535]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:38.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:38 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:38.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:38 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:38.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:38 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:38.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:38 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:38.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:38 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:38.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:38 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:38.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:38 smithi177 ceph-mon[35305]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:38.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:38 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:38.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:38 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:38.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:38 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:38.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:38 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:38.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:38 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:38.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:38 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:38.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:38 smithi116 ceph-mon[31410]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:38.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:38 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:38.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:38 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:38.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:38 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:38.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:38 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:38.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:38 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:38.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:38 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:38.570 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:39.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.065+0000 7f48fdf2d640 1 -- 172.21.15.116:0/1890628564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48f8101de0 msgr2=0x7f48f81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:39.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.065+0000 7f48fdf2d640 1 --2- 172.21.15.116:0/1890628564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48f8101de0 0x7f48f81021c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009900 tx=0x7f48e802a410 comp rx=0 tx=0).stop 2024-09-12T03:00:39.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.065+0000 7f48fdf2d640 1 -- 172.21.15.116:0/1890628564 shutdown_connections 2024-09-12T03:00:39.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.065+0000 7f48fdf2d640 1 --2- 172.21.15.116:0/1890628564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48f8106f10 0x7f48f810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.065+0000 7f48fdf2d640 1 --2- 172.21.15.116:0/1890628564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48f8102790 0x7f48f81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.065+0000 7f48fdf2d640 1 --2- 172.21.15.116:0/1890628564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48f8101de0 0x7f48f81021c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.065+0000 7f48fdf2d640 1 -- 172.21.15.116:0/1890628564 >> 172.21.15.116:0/1890628564 conn(0x7f48f80fd8c0 msgr2=0x7f48f80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:39.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.065+0000 7f48fdf2d640 1 -- 172.21.15.116:0/1890628564 shutdown_connections 2024-09-12T03:00:39.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.065+0000 7f48fdf2d640 1 -- 172.21.15.116:0/1890628564 wait complete. 2024-09-12T03:00:39.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.066+0000 7f48fdf2d640 1 Processor -- start 2024-09-12T03:00:39.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.066+0000 7f48fdf2d640 1 -- start start 2024-09-12T03:00:39.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.066+0000 7f48fdf2d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48f8101de0 0x7f48f8198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:39.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.066+0000 7f48fdf2d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48f8102790 0x7f48f8198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:39.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.066+0000 7f48fdf2d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48f8106f10 0x7f48f819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:39.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.066+0000 7f48fdf2d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f48f8111fb0 con 0x7f48f8101de0 2024-09-12T03:00:39.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.066+0000 7f48fdf2d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f48f8111e30 con 0x7f48f8102790 2024-09-12T03:00:39.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.066+0000 7f48fdf2d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f48f8112130 con 0x7f48f8106f10 2024-09-12T03:00:39.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.066+0000 7f48f77fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48f8101de0 0x7f48f8198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:39.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.066+0000 7f48f77fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48f8101de0 0x7f48f8198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42206/0 (socket says 172.21.15.116:42206) 2024-09-12T03:00:39.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.066+0000 7f48f77fe640 1 -- 172.21.15.116:0/1543441735 learned_addr learned my addr 172.21.15.116:0/1543441735 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:00:39.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.067+0000 7f48f7fff640 1 --2- 172.21.15.116:0/1543441735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48f8106f10 0x7f48f819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:39.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.067+0000 7f48f6ffd640 1 --2- 172.21.15.116:0/1543441735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48f8102790 0x7f48f8198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:39.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.067+0000 7f48f77fe640 1 -- 172.21.15.116:0/1543441735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48f8106f10 msgr2=0x7f48f819cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:39.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.067+0000 7f48f77fe640 1 --2- 172.21.15.116:0/1543441735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48f8106f10 0x7f48f819cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.067+0000 7f48f77fe640 1 -- 172.21.15.116:0/1543441735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48f8102790 msgr2=0x7f48f8198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:39.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.067+0000 7f48f77fe640 1 --2- 172.21.15.116:0/1543441735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48f8102790 0x7f48f8198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.067+0000 7f48f77fe640 1 -- 172.21.15.116:0/1543441735 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f819d4c0 con 0x7f48f8101de0 2024-09-12T03:00:39.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.067+0000 7f48f6ffd640 1 --2- 172.21.15.116:0/1543441735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48f8102790 0x7f48f8198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:00:39.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.067+0000 7f48f77fe640 1 --2- 172.21.15.116:0/1543441735 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48f8101de0 0x7f48f8198510 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f48e8006010 tx=0x7f48e80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:39.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.067+0000 7f48f4ff9640 1 -- 172.21.15.116:0/1543441735 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e803f030 con 0x7f48f8101de0 2024-09-12T03:00:39.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.067+0000 7f48f4ff9640 1 -- 172.21.15.116:0/1543441735 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48e8003ce0 con 0x7f48f8101de0 2024-09-12T03:00:39.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.067+0000 7f48fdf2d640 1 -- 172.21.15.116:0/1543441735 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f819d750 con 0x7f48f8101de0 2024-09-12T03:00:39.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.067+0000 7f48f4ff9640 1 -- 172.21.15.116:0/1543441735 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8035070 con 0x7f48f8101de0 2024-09-12T03:00:39.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.067+0000 7f48fdf2d640 1 -- 172.21.15.116:0/1543441735 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f81a5090 con 0x7f48f8101de0 2024-09-12T03:00:39.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.068+0000 7f48fdf2d640 1 -- 172.21.15.116:0/1543441735 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48f81032c0 con 0x7f48f8101de0 2024-09-12T03:00:39.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.069+0000 7f48f4ff9640 1 -- 172.21.15.116:0/1543441735 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f48e8042030 con 0x7f48f8101de0 2024-09-12T03:00:39.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.070+0000 7f48f4ff9640 1 --2- 172.21.15.116:0/1543441735 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f48e40a2b80 0x7f48e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:39.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.071+0000 7f48f6ffd640 1 --2- 172.21.15.116:0/1543441735 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f48e40a2b80 0x7f48e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:39.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.071+0000 7f48f4ff9640 1 -- 172.21.15.116:0/1543441735 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f48e80ef260 con 0x7f48f8101de0 2024-09-12T03:00:39.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.071+0000 7f48f6ffd640 1 --2- 172.21.15.116:0/1543441735 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f48e40a2b80 0x7f48e40a5040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f48e0006fd0 tx=0x7f48e0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:39.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.075+0000 7f48f4ff9640 1 -- 172.21.15.116:0/1543441735 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f48e80bc720 con 0x7f48f8101de0 2024-09-12T03:00:39.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.275+0000 7f48fdf2d640 1 -- 172.21.15.116:0/1543441735 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48f810a0f0 con 0x7f48f8101de0 2024-09-12T03:00:39.276 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:39 smithi116 ceph-mon[31410]: Deploying daemon mds.i on smithi183 2024-09-12T03:00:39.276 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:39 smithi116 ceph-mon[31410]: mds.? [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] up:boot 2024-09-12T03:00:39.276 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:39 smithi116 ceph-mon[31410]: fsmap 8 up:standby 2024-09-12T03:00:39.276 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:39 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:00:39.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.276+0000 7f48f4ff9640 1 -- 172.21.15.116:0/1543441735 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7234 (secure 0 0 0) 0x7f48e80bc020 con 0x7f48f8101de0 2024-09-12T03:00:39.277 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:00:39.277 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":9,"btime":"2024-09-12T03:00:38:154826+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-12T03:00:39.277 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 9 2024-09-12T03:00:39.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.278+0000 7f48fdf2d640 1 -- 172.21.15.116:0/1543441735 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f48e40a2b80 msgr2=0x7f48e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:39.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.278+0000 7f48fdf2d640 1 --2- 172.21.15.116:0/1543441735 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f48e40a2b80 0x7f48e40a5040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f48e0006fd0 tx=0x7f48e0008040 comp rx=0 tx=0).stop 2024-09-12T03:00:39.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.278+0000 7f48fdf2d640 1 -- 172.21.15.116:0/1543441735 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48f8101de0 msgr2=0x7f48f8198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:39.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.278+0000 7f48fdf2d640 1 --2- 172.21.15.116:0/1543441735 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48f8101de0 0x7f48f8198510 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f48e8006010 tx=0x7f48e80042f0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.278+0000 7f48fdf2d640 1 -- 172.21.15.116:0/1543441735 shutdown_connections 2024-09-12T03:00:39.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.278+0000 7f48fdf2d640 1 --2- 172.21.15.116:0/1543441735 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f48e40a2b80 0x7f48e40a5040 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.278+0000 7f48fdf2d640 1 --2- 172.21.15.116:0/1543441735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48f8106f10 0x7f48f819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.278+0000 7f48fdf2d640 1 --2- 172.21.15.116:0/1543441735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48f8102790 0x7f48f8198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.278+0000 7f48fdf2d640 1 --2- 172.21.15.116:0/1543441735 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48f8101de0 0x7f48f8198510 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.278+0000 7f48fdf2d640 1 -- 172.21.15.116:0/1543441735 >> 172.21.15.116:0/1543441735 conn(0x7f48f80fd8c0 msgr2=0x7f48f80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:39.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.278+0000 7f48fdf2d640 1 -- 172.21.15.116:0/1543441735 shutdown_connections 2024-09-12T03:00:39.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:39.278+0000 7f48fdf2d640 1 -- 172.21.15.116:0/1543441735 wait complete. 2024-09-12T03:00:39.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:39 smithi183 ceph-mon[35535]: Deploying daemon mds.i on smithi183 2024-09-12T03:00:39.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:39 smithi183 ceph-mon[35535]: mds.? [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] up:boot 2024-09-12T03:00:39.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:39 smithi183 ceph-mon[35535]: fsmap 8 up:standby 2024-09-12T03:00:39.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:39 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:00:39.454 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 03:00:39 smithi183 systemd[1]: Starting Ceph mds.i for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:00:39.514 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:39 smithi177 ceph-mon[35305]: Deploying daemon mds.i on smithi183 2024-09-12T03:00:39.514 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:39 smithi177 ceph-mon[35305]: mds.? [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] up:boot 2024-09-12T03:00:39.514 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:39 smithi177 ceph-mon[35305]: fsmap 8 up:standby 2024-09-12T03:00:39.514 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:39 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:00:39.805 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 03:00:39 smithi183 podman[66059]: 2024-09-12 03:00:39.433593477 +0000 UTC m=+0.016090313 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:00:39.805 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 03:00:39 smithi183 podman[66059]: 2024-09-12 03:00:39.560343794 +0000 UTC m=+0.142840621 container create a6a2f21bf45e4df9f25767643bbb3b53a9182e7c1d3c1d1513ecdfcdd3a0e627 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-i, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-12T03:00:40.170 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 03:00:39 smithi183 podman[66059]: 2024-09-12 03:00:39.804766501 +0000 UTC m=+0.387263335 container init a6a2f21bf45e4df9f25767643bbb3b53a9182e7c1d3c1d1513ecdfcdd3a0e627 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-i, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-12T03:00:40.170 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 03:00:39 smithi183 podman[66059]: 2024-09-12 03:00:39.807917209 +0000 UTC m=+0.390414036 container start a6a2f21bf45e4df9f25767643bbb3b53a9182e7c1d3c1d1513ecdfcdd3a0e627 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-i, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-12T03:00:40.170 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 03:00:39 smithi183 ceph-mds[66078]: -- 172.21.15.183:0/376231913 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f782f8b860 con 0x55f782f69400 2024-09-12T03:00:40.170 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 03:00:39 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-i[66074]: starting mds.i at 2024-09-12T03:00:40.170 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 03:00:39 smithi183 bash[66059]: a6a2f21bf45e4df9f25767643bbb3b53a9182e7c1d3c1d1513ecdfcdd3a0e627 2024-09-12T03:00:40.170 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 03:00:39 smithi183 systemd[1]: Started Ceph mds.i for eb1ac602-70b1-11ef-bcea-c7b262605968. 2024-09-12T03:00:40.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:40 smithi116 ceph-mon[31410]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:40.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:40 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1543441735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:40.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:40 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:40.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:40 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:40.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:40 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:40.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:40 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:40.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:40 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:40.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:40 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:40.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:40 smithi183 ceph-mon[35535]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:40.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:40 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1543441735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:40.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:40 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:40.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:40 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:40.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:40 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:40.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:40 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:40.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:40 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:40.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:40 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:40.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:40 smithi177 ceph-mon[35305]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:40.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:40 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1543441735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:40.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:40 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:40.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:40 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:40.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:40 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:40.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:40 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:40.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:40 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:40.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:40 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:41.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:41 smithi183 ceph-mon[35535]: Deploying daemon mds.g on smithi116 2024-09-12T03:00:41.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:41 smithi183 ceph-mon[35535]: mds.? [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] up:boot 2024-09-12T03:00:41.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:41 smithi183 ceph-mon[35535]: fsmap 9 up:standby 2024-09-12T03:00:41.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:41 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:00:41.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:41 smithi177 ceph-mon[35305]: Deploying daemon mds.g on smithi116 2024-09-12T03:00:41.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:41 smithi177 ceph-mon[35305]: mds.? [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] up:boot 2024-09-12T03:00:41.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:41 smithi177 ceph-mon[35305]: fsmap 9 up:standby 2024-09-12T03:00:41.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:41 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:00:41.500 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:41 smithi116 ceph-mon[31410]: Deploying daemon mds.g on smithi116 2024-09-12T03:00:41.500 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:41 smithi116 ceph-mon[31410]: mds.? [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] up:boot 2024-09-12T03:00:41.500 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:41 smithi116 ceph-mon[31410]: fsmap 9 up:standby 2024-09-12T03:00:41.500 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:41 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:00:41.991 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:00:42.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:42 smithi183 ceph-mon[35535]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:42.454 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:42 smithi116 ceph-mon[31410]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:42.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:42 smithi177 ceph-mon[35305]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:42.627 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:42.750 INFO:journalctl@ceph.mds.g.smithi116.stdout:Sep 12 03:00:42 smithi116 systemd[1]: Started Ceph mds.g for eb1ac602-70b1-11ef-bcea-c7b262605968. 2024-09-12T03:00:43.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.113+0000 7f9eed8d6640 1 -- 172.21.15.116:0/683676434 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee8102de0 msgr2=0x7f9ee81031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:43.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.113+0000 7f9eed8d6640 1 --2- 172.21.15.116:0/683676434 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee8102de0 0x7f9ee81031c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009900 tx=0x7f9ed802a480 comp rx=0 tx=0).stop 2024-09-12T03:00:43.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.113+0000 7f9eed8d6640 1 -- 172.21.15.116:0/683676434 shutdown_connections 2024-09-12T03:00:43.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.113+0000 7f9eed8d6640 1 --2- 172.21.15.116:0/683676434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ee810c180 0x7f9ee810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:43.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.113+0000 7f9eed8d6640 1 --2- 172.21.15.116:0/683676434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ee8103700 0x7f9ee810bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:43.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.113+0000 7f9eed8d6640 1 --2- 172.21.15.116:0/683676434 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee8102de0 0x7f9ee81031c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:43.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.113+0000 7f9eed8d6640 1 -- 172.21.15.116:0/683676434 >> 172.21.15.116:0/683676434 conn(0x7f9ee80fc7a0 msgr2=0x7f9ee80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:43.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.113+0000 7f9eed8d6640 1 -- 172.21.15.116:0/683676434 shutdown_connections 2024-09-12T03:00:43.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.113+0000 7f9eed8d6640 1 -- 172.21.15.116:0/683676434 wait complete. 2024-09-12T03:00:43.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.114+0000 7f9eed8d6640 1 Processor -- start 2024-09-12T03:00:43.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.114+0000 7f9eed8d6640 1 -- start start 2024-09-12T03:00:43.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.114+0000 7f9eed8d6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ee8102de0 0x7f9ee8198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:43.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.114+0000 7f9eed8d6640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ee8103700 0x7f9ee8198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:43.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.114+0000 7f9eed8d6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee810c180 0x7f9ee819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:43.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.114+0000 7f9eed8d6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee8111f70 con 0x7f9ee810c180 2024-09-12T03:00:43.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.114+0000 7f9eed8d6640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee8111df0 con 0x7f9ee8103700 2024-09-12T03:00:43.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.114+0000 7f9eed8d6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee81120f0 con 0x7f9ee8102de0 2024-09-12T03:00:43.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.114+0000 7f9ee77fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee810c180 0x7f9ee819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:43.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.114+0000 7f9ee77fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee810c180 0x7f9ee819cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36988/0 (socket says 172.21.15.116:36988) 2024-09-12T03:00:43.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.114+0000 7f9ee77fe640 1 -- 172.21.15.116:0/74033821 learned_addr learned my addr 172.21.15.116:0/74033821 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:00:43.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.114+0000 7f9ee6ffd640 1 --2- 172.21.15.116:0/74033821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ee8102de0 0x7f9ee8198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:43.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.115+0000 7f9edffff640 1 --2- 172.21.15.116:0/74033821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ee8103700 0x7f9ee8198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:43.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.115+0000 7f9ee77fe640 1 -- 172.21.15.116:0/74033821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ee8102de0 msgr2=0x7f9ee8198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:43.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.115+0000 7f9ee77fe640 1 --2- 172.21.15.116:0/74033821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ee8102de0 0x7f9ee8198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:43.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.115+0000 7f9ee77fe640 1 -- 172.21.15.116:0/74033821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ee8103700 msgr2=0x7f9ee8198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:43.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.115+0000 7f9ee77fe640 1 --2- 172.21.15.116:0/74033821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ee8103700 0x7f9ee8198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:43.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.115+0000 7f9ee77fe640 1 -- 172.21.15.116:0/74033821 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee819d400 con 0x7f9ee810c180 2024-09-12T03:00:43.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.115+0000 7f9edffff640 1 --2- 172.21.15.116:0/74033821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ee8103700 0x7f9ee8198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:00:43.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.115+0000 7f9ee77fe640 1 --2- 172.21.15.116:0/74033821 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee810c180 0x7f9ee819cd20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9ee000eaa0 tx=0x7f9ee000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:43.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.115+0000 7f9eec8d4640 1 -- 172.21.15.116:0/74033821 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee000cdb0 con 0x7f9ee810c180 2024-09-12T03:00:43.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.115+0000 7f9eec8d4640 1 -- 172.21.15.116:0/74033821 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ee0005020 con 0x7f9ee810c180 2024-09-12T03:00:43.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.115+0000 7f9eed8d6640 1 -- 172.21.15.116:0/74033821 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee819d6f0 con 0x7f9ee810c180 2024-09-12T03:00:43.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.115+0000 7f9eec8d4640 1 -- 172.21.15.116:0/74033821 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee00116b0 con 0x7f9ee810c180 2024-09-12T03:00:43.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.115+0000 7f9eed8d6640 1 -- 172.21.15.116:0/74033821 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee81a4fd0 con 0x7f9ee810c180 2024-09-12T03:00:43.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.116+0000 7f9eed8d6640 1 -- 172.21.15.116:0/74033821 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ee81031c0 con 0x7f9ee810c180 2024-09-12T03:00:43.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.117+0000 7f9eec8d4640 1 -- 172.21.15.116:0/74033821 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f9ee0005560 con 0x7f9ee810c180 2024-09-12T03:00:43.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.118+0000 7f9eec8d4640 1 --2- 172.21.15.116:0/74033821 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9ed40a2b80 0x7f9ed40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:43.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.118+0000 7f9ee6ffd640 1 --2- 172.21.15.116:0/74033821 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9ed40a2b80 0x7f9ed40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:43.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.118+0000 7f9eec8d4640 1 -- 172.21.15.116:0/74033821 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f9ee00cf200 con 0x7f9ee810c180 2024-09-12T03:00:43.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.119+0000 7f9ee6ffd640 1 --2- 172.21.15.116:0/74033821 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9ed40a2b80 0x7f9ed40a5040 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009230 tx=0x7f9ed80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:43.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.122+0000 7f9eec8d4640 1 -- 172.21.15.116:0/74033821 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9ee0016030 con 0x7f9ee810c180 2024-09-12T03:00:43.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.333+0000 7f9eed8d6640 1 -- 172.21.15.116:0/74033821 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ee81997b0 con 0x7f9ee810c180 2024-09-12T03:00:43.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.334+0000 7f9eec8d4640 1 -- 172.21.15.116:0/74033821 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8893 (secure 0 0 0) 0x7f9ee009c560 con 0x7f9ee810c180 2024-09-12T03:00:43.335 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:00:43.335 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":11,"btime":"2024-09-12T03:00:43:184839+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T03:00:43.335 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 11 2024-09-12T03:00:43.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.336+0000 7f9eed8d6640 1 -- 172.21.15.116:0/74033821 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9ed40a2b80 msgr2=0x7f9ed40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:43.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.336+0000 7f9eed8d6640 1 --2- 172.21.15.116:0/74033821 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9ed40a2b80 0x7f9ed40a5040 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009230 tx=0x7f9ed80058d0 comp rx=0 tx=0).stop 2024-09-12T03:00:43.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.336+0000 7f9eed8d6640 1 -- 172.21.15.116:0/74033821 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee810c180 msgr2=0x7f9ee819cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:43.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.336+0000 7f9eed8d6640 1 --2- 172.21.15.116:0/74033821 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee810c180 0x7f9ee819cd20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9ee000eaa0 tx=0x7f9ee000ef60 comp rx=0 tx=0).stop 2024-09-12T03:00:43.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.336+0000 7f9eed8d6640 1 -- 172.21.15.116:0/74033821 shutdown_connections 2024-09-12T03:00:43.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.336+0000 7f9eed8d6640 1 --2- 172.21.15.116:0/74033821 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9ed40a2b80 0x7f9ed40a5040 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:43.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.336+0000 7f9eed8d6640 1 --2- 172.21.15.116:0/74033821 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ee810c180 0x7f9ee819cd20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:43.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.336+0000 7f9eed8d6640 1 --2- 172.21.15.116:0/74033821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ee8103700 0x7f9ee8198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:43.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.336+0000 7f9eed8d6640 1 --2- 172.21.15.116:0/74033821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ee8102de0 0x7f9ee8198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:43.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.336+0000 7f9eed8d6640 1 -- 172.21.15.116:0/74033821 >> 172.21.15.116:0/74033821 conn(0x7f9ee80fc7a0 msgr2=0x7f9ee80fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:43.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.336+0000 7f9eed8d6640 1 -- 172.21.15.116:0/74033821 shutdown_connections 2024-09-12T03:00:43.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:43.336+0000 7f9eed8d6640 1 -- 172.21.15.116:0/74033821 wait complete. 2024-09-12T03:00:43.668 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:43 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:43.668 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:43 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:43.668 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:43 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:43.668 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:43 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:43.668 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:43 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:43.669 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:43 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:43.669 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:43 smithi177 ceph-mon[35305]: Deploying daemon mds.a on smithi116 2024-09-12T03:00:43.669 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:43 smithi177 ceph-mon[35305]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:43.669 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:43 smithi177 ceph-mon[35305]: mds.? [v2:172.21.15.116:6838/1740752718,v1:172.21.15.116:6839/1740752718] up:boot 2024-09-12T03:00:43.669 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:43 smithi177 ceph-mon[35305]: fsmap 10 up:standby 2024-09-12T03:00:43.669 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:43 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:00:43.669 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:43 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/74033821' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:43.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:43 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:43.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:43 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:43.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:43 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:43.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:43 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:43.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:43 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:43.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:43 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:43.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:43 smithi183 ceph-mon[35535]: Deploying daemon mds.a on smithi116 2024-09-12T03:00:43.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:43 smithi183 ceph-mon[35535]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:43.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:43 smithi183 ceph-mon[35535]: mds.? [v2:172.21.15.116:6838/1740752718,v1:172.21.15.116:6839/1740752718] up:boot 2024-09-12T03:00:43.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:43 smithi183 ceph-mon[35535]: fsmap 10 up:standby 2024-09-12T03:00:43.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:43 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:00:43.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:43 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/74033821' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:44.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:43 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:44.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:43 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:44.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:43 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:44.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:43 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:44.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:43 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:44.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:43 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:44.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:43 smithi116 ceph-mon[31410]: Deploying daemon mds.a on smithi116 2024-09-12T03:00:44.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:43 smithi116 ceph-mon[31410]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:44.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:43 smithi116 ceph-mon[31410]: mds.? [v2:172.21.15.116:6838/1740752718,v1:172.21.15.116:6839/1740752718] up:boot 2024-09-12T03:00:44.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:43 smithi116 ceph-mon[31410]: fsmap 10 up:standby 2024-09-12T03:00:44.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:43 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:00:44.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:43 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/74033821' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:45.751 INFO:journalctl@ceph.mds.a.smithi116.stdout:Sep 12 03:00:45 smithi116 systemd[1]: Starting Ceph mds.a for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:00:46.034 INFO:journalctl@ceph.mds.a.smithi116.stdout:Sep 12 03:00:45 smithi116 podman[72972]: 2024-09-12 03:00:45.654154093 +0000 UTC m=+0.015907552 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:00:46.034 INFO:journalctl@ceph.mds.a.smithi116.stdout:Sep 12 03:00:45 smithi116 podman[72972]: 2024-09-12 03:00:45.789771753 +0000 UTC m=+0.151525198 container create d519324b9a1bf92d6198c7de34b57305b1bec4a4bb05a77133dc404713c3c26a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:00:46.034 INFO:journalctl@ceph.mds.a.smithi116.stdout:Sep 12 03:00:46 smithi116 podman[72972]: 2024-09-12 03:00:46.034371363 +0000 UTC m=+0.396124818 container init d519324b9a1bf92d6198c7de34b57305b1bec4a4bb05a77133dc404713c3c26a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-12T03:00:46.179 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:00:46.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:46 smithi116 ceph-mon[31410]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:46.302 INFO:journalctl@ceph.mds.a.smithi116.stdout:Sep 12 03:00:46 smithi116 podman[72972]: 2024-09-12 03:00:46.037385242 +0000 UTC m=+0.399138706 container start d519324b9a1bf92d6198c7de34b57305b1bec4a4bb05a77133dc404713c3c26a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-12T03:00:46.303 INFO:journalctl@ceph.mds.a.smithi116.stdout:Sep 12 03:00:46 smithi116 ceph-mds[72991]: -- 172.21.15.116:0/1484761831 <== mon.0 v2:172.21.15.116:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5580e0d27860 con 0x5580e0d05000 2024-09-12T03:00:46.303 INFO:journalctl@ceph.mds.a.smithi116.stdout:Sep 12 03:00:46 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-a[72987]: starting mds.a at 2024-09-12T03:00:46.303 INFO:journalctl@ceph.mds.a.smithi116.stdout:Sep 12 03:00:46 smithi116 bash[72972]: d519324b9a1bf92d6198c7de34b57305b1bec4a4bb05a77133dc404713c3c26a 2024-09-12T03:00:46.303 INFO:journalctl@ceph.mds.a.smithi116.stdout:Sep 12 03:00:46 smithi116 systemd[1]: Started Ceph mds.a for eb1ac602-70b1-11ef-bcea-c7b262605968. 2024-09-12T03:00:46.433 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:46.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:46 smithi183 ceph-mon[35535]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:46.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:46 smithi177 ceph-mon[35305]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:46.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.908+0000 7fdce10fc640 1 -- 172.21.15.116:0/3815760965 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdcdc1082e0 msgr2=0x7fdcdc10ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:46.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.908+0000 7fdce10fc640 1 --2- 172.21.15.116:0/3815760965 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdcdc1082e0 0x7fdcdc10ec50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdccc009cd0 tx=0x7fdccc02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:00:46.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.909+0000 7fdce10fc640 1 -- 172.21.15.116:0/3815760965 shutdown_connections 2024-09-12T03:00:46.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.909+0000 7fdce10fc640 1 --2- 172.21.15.116:0/3815760965 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdcdc1082e0 0x7fdcdc10ec50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:46.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.909+0000 7fdce10fc640 1 --2- 172.21.15.116:0/3815760965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcdc103760 0x7fdcdc107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:46.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.909+0000 7fdce10fc640 1 --2- 172.21.15.116:0/3815760965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcdc102db0 0x7fdcdc103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:46.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.909+0000 7fdce10fc640 1 -- 172.21.15.116:0/3815760965 >> 172.21.15.116:0/3815760965 conn(0x7fdcdc0fc7d0 msgr2=0x7fdcdc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:46.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.909+0000 7fdce10fc640 1 -- 172.21.15.116:0/3815760965 shutdown_connections 2024-09-12T03:00:46.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.909+0000 7fdce10fc640 1 -- 172.21.15.116:0/3815760965 wait complete. 2024-09-12T03:00:46.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.909+0000 7fdce10fc640 1 Processor -- start 2024-09-12T03:00:46.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.909+0000 7fdce10fc640 1 -- start start 2024-09-12T03:00:46.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.910+0000 7fdce10fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcdc102db0 0x7fdcdc198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:46.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.910+0000 7fdce10fc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcdc103760 0x7fdcdc198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:46.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.910+0000 7fdce10fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdcdc1082e0 0x7fdcdc19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:46.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.910+0000 7fdce10fc640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc1120b0 con 0x7fdcdc1082e0 2024-09-12T03:00:46.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.910+0000 7fdce10fc640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc111f30 con 0x7fdcdc103760 2024-09-12T03:00:46.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.910+0000 7fdce10fc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc112230 con 0x7fdcdc102db0 2024-09-12T03:00:46.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.910+0000 7fdcdb59e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdcdc1082e0 0x7fdcdc19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:46.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.910+0000 7fdcdb59e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdcdc1082e0 0x7fdcdc19ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37014/0 (socket says 172.21.15.116:37014) 2024-09-12T03:00:46.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.910+0000 7fdcdb59e640 1 -- 172.21.15.116:0/3620882987 learned_addr learned my addr 172.21.15.116:0/3620882987 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:00:46.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.910+0000 7fdcdad9d640 1 --2- 172.21.15.116:0/3620882987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcdc102db0 0x7fdcdc198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:46.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.910+0000 7fdcda59c640 1 --2- 172.21.15.116:0/3620882987 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcdc103760 0x7fdcdc198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:46.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.910+0000 7fdcdb59e640 1 -- 172.21.15.116:0/3620882987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcdc102db0 msgr2=0x7fdcdc198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:46.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.910+0000 7fdcdb59e640 1 --2- 172.21.15.116:0/3620882987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcdc102db0 0x7fdcdc198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:46.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.910+0000 7fdcdb59e640 1 -- 172.21.15.116:0/3620882987 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcdc103760 msgr2=0x7fdcdc198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:46.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.910+0000 7fdcdb59e640 1 --2- 172.21.15.116:0/3620882987 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcdc103760 0x7fdcdc198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:46.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.910+0000 7fdcdb59e640 1 -- 172.21.15.116:0/3620882987 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcdc19d500 con 0x7fdcdc1082e0 2024-09-12T03:00:46.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.910+0000 7fdcdb59e640 1 --2- 172.21.15.116:0/3620882987 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdcdc1082e0 0x7fdcdc19ce20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdccc005e30 tx=0x7fdccc003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:46.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.910+0000 7fdcd3fff640 1 -- 172.21.15.116:0/3620882987 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc004040 con 0x7fdcdc1082e0 2024-09-12T03:00:46.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.911+0000 7fdcd3fff640 1 -- 172.21.15.116:0/3620882987 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdccc0046c0 con 0x7fdcdc1082e0 2024-09-12T03:00:46.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.911+0000 7fdcd3fff640 1 -- 172.21.15.116:0/3620882987 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc03a400 con 0x7fdcdc1082e0 2024-09-12T03:00:46.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.911+0000 7fdce10fc640 1 -- 172.21.15.116:0/3620882987 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcdc19d790 con 0x7fdcdc1082e0 2024-09-12T03:00:46.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.911+0000 7fdce10fc640 1 -- 172.21.15.116:0/3620882987 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcdc1a50d0 con 0x7fdcdc1082e0 2024-09-12T03:00:46.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.912+0000 7fdce10fc640 1 -- 172.21.15.116:0/3620882987 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcdc103190 con 0x7fdcdc1082e0 2024-09-12T03:00:46.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.913+0000 7fdcd3fff640 1 -- 172.21.15.116:0/3620882987 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fdccc030630 con 0x7fdcdc1082e0 2024-09-12T03:00:46.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.914+0000 7fdcd3fff640 1 --2- 172.21.15.116:0/3620882987 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fdcc80a2b80 0x7fdcc80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:46.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.914+0000 7fdcdad9d640 1 --2- 172.21.15.116:0/3620882987 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fdcc80a2b80 0x7fdcc80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:46.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.914+0000 7fdcd3fff640 1 -- 172.21.15.116:0/3620882987 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fdccc02e030 con 0x7fdcdc1082e0 2024-09-12T03:00:46.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.914+0000 7fdcdad9d640 1 --2- 172.21.15.116:0/3620882987 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fdcc80a2b80 0x7fdcc80a5040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4005fd0 tx=0x7fdcd400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:46.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:46.917+0000 7fdcd3fff640 1 -- 172.21.15.116:0/3620882987 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdccc0bd180 con 0x7fdcdc1082e0 2024-09-12T03:00:47.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:47.117+0000 7fdce10fc640 1 -- 172.21.15.116:0/3620882987 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdcdc10a2c0 con 0x7fdcdc1082e0 2024-09-12T03:00:47.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:47.118+0000 7fdcd3fff640 1 -- 172.21.15.116:0/3620882987 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9722 (secure 0 0 0) 0x7fdccc0bca80 con 0x7fdcdc1082e0 2024-09-12T03:00:47.118 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:00:47.119 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":12,"btime":"2024-09-12T03:00:46:156672+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T03:00:47.119 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 12 2024-09-12T03:00:47.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:47.120+0000 7fdce10fc640 1 -- 172.21.15.116:0/3620882987 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fdcc80a2b80 msgr2=0x7fdcc80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:47.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:47.120+0000 7fdce10fc640 1 --2- 172.21.15.116:0/3620882987 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fdcc80a2b80 0x7fdcc80a5040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4005fd0 tx=0x7fdcd400b390 comp rx=0 tx=0).stop 2024-09-12T03:00:47.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:47.120+0000 7fdce10fc640 1 -- 172.21.15.116:0/3620882987 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdcdc1082e0 msgr2=0x7fdcdc19ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:47.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:47.120+0000 7fdce10fc640 1 --2- 172.21.15.116:0/3620882987 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdcdc1082e0 0x7fdcdc19ce20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdccc005e30 tx=0x7fdccc003970 comp rx=0 tx=0).stop 2024-09-12T03:00:47.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:47.120+0000 7fdce10fc640 1 -- 172.21.15.116:0/3620882987 shutdown_connections 2024-09-12T03:00:47.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:47.120+0000 7fdce10fc640 1 --2- 172.21.15.116:0/3620882987 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fdcc80a2b80 0x7fdcc80a5040 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:47.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:47.120+0000 7fdce10fc640 1 --2- 172.21.15.116:0/3620882987 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdcdc1082e0 0x7fdcdc19ce20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:47.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:47.120+0000 7fdce10fc640 1 --2- 172.21.15.116:0/3620882987 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcdc103760 0x7fdcdc198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:47.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:47.120+0000 7fdce10fc640 1 --2- 172.21.15.116:0/3620882987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcdc102db0 0x7fdcdc198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:47.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:47.120+0000 7fdce10fc640 1 -- 172.21.15.116:0/3620882987 >> 172.21.15.116:0/3620882987 conn(0x7fdcdc0fc7d0 msgr2=0x7fdcdc106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:47.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:47.120+0000 7fdce10fc640 1 -- 172.21.15.116:0/3620882987 shutdown_connections 2024-09-12T03:00:47.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:47.120+0000 7fdce10fc640 1 -- 172.21.15.116:0/3620882987 wait complete. 2024-09-12T03:00:47.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:47 smithi177 ceph-mon[35305]: mds.? [v2:172.21.15.116:6840/2495398640,v1:172.21.15.116:6841/2495398640] up:boot 2024-09-12T03:00:47.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:47 smithi177 ceph-mon[35305]: fsmap 11 up:standby 2024-09-12T03:00:47.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:47 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:00:47.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:47 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:47.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:47 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:47.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:47 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:47.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:47 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:47.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:47 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:47.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:47 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:47.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:47 smithi177 ceph-mon[35305]: Deploying daemon mds.b on smithi177 2024-09-12T03:00:47.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:47 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3620882987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:47.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:47 smithi183 ceph-mon[35535]: mds.? [v2:172.21.15.116:6840/2495398640,v1:172.21.15.116:6841/2495398640] up:boot 2024-09-12T03:00:47.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:47 smithi183 ceph-mon[35535]: fsmap 11 up:standby 2024-09-12T03:00:47.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:47 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:00:47.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:47 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:47.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:47 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:47.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:47 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:47.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:47 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:47.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:47 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:47.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:47 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:47.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:47 smithi183 ceph-mon[35535]: Deploying daemon mds.b on smithi177 2024-09-12T03:00:47.454 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:47 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3620882987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:47.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:47 smithi116 ceph-mon[31410]: mds.? [v2:172.21.15.116:6840/2495398640,v1:172.21.15.116:6841/2495398640] up:boot 2024-09-12T03:00:47.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:47 smithi116 ceph-mon[31410]: fsmap 11 up:standby 2024-09-12T03:00:47.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:47 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:00:47.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:47 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:47.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:47 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:47.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:47 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:47.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:47 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:00:47.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:47 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:00:47.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:47 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:47.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:47 smithi116 ceph-mon[31410]: Deploying daemon mds.b on smithi177 2024-09-12T03:00:47.510 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:47 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3620882987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:48.433 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:48 smithi177 ceph-mon[35305]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:48.433 INFO:journalctl@ceph.mds.b.smithi177.stdout:Sep 12 03:00:48 smithi177 systemd[1]: Starting Ceph mds.b for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:00:48.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:48 smithi183 ceph-mon[35535]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:48.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:48 smithi116 ceph-mon[31410]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:48.697 INFO:journalctl@ceph.mds.b.smithi177.stdout:Sep 12 03:00:48 smithi177 podman[65325]: 2024-09-12 03:00:48.336141415 +0000 UTC m=+0.016235976 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:00:48.697 INFO:journalctl@ceph.mds.b.smithi177.stdout:Sep 12 03:00:48 smithi177 podman[65325]: 2024-09-12 03:00:48.443946786 +0000 UTC m=+0.124041334 container create 14a73268da50e4c4e77ed332f29012e459761332e3d5a1472e38dc9acac2eb24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-b, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-12T03:00:48.698 INFO:journalctl@ceph.mds.b.smithi177.stdout:Sep 12 03:00:48 smithi177 podman[65325]: 2024-09-12 03:00:48.67164901 +0000 UTC m=+0.351743559 container init 14a73268da50e4c4e77ed332f29012e459761332e3d5a1472e38dc9acac2eb24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-b, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:00:48.698 INFO:journalctl@ceph.mds.b.smithi177.stdout:Sep 12 03:00:48 smithi177 podman[65325]: 2024-09-12 03:00:48.674604731 +0000 UTC m=+0.354699289 container start 14a73268da50e4c4e77ed332f29012e459761332e3d5a1472e38dc9acac2eb24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-b, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:00:48.997 INFO:journalctl@ceph.mds.b.smithi177.stdout:Sep 12 03:00:48 smithi177 ceph-mds[65343]: -- 172.21.15.177:0/3395582728 <== mon.2 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e8563ad860 con 0x55e85638a800 2024-09-12T03:00:48.997 INFO:journalctl@ceph.mds.b.smithi177.stdout:Sep 12 03:00:48 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-b[65339]: starting mds.b at 2024-09-12T03:00:48.997 INFO:journalctl@ceph.mds.b.smithi177.stdout:Sep 12 03:00:48 smithi177 bash[65325]: 14a73268da50e4c4e77ed332f29012e459761332e3d5a1472e38dc9acac2eb24 2024-09-12T03:00:48.997 INFO:journalctl@ceph.mds.b.smithi177.stdout:Sep 12 03:00:48 smithi177 systemd[1]: Started Ceph mds.b for eb1ac602-70b1-11ef-bcea-c7b262605968. 2024-09-12T03:00:49.863 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:00:49.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:49 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:49.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:49 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:49.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:49 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:49.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:49 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:49.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:49 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:49.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:49 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:00:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:49 smithi183 ceph-mon[35535]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:49 smithi183 ceph-mon[35535]: mds.? [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] up:boot 2024-09-12T03:00:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:49 smithi183 ceph-mon[35535]: fsmap 12 up:standby 2024-09-12T03:00:49.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:49 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:00:50.121 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:50.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:49 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:50.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:49 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:50.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:49 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:50.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:49 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:50.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:49 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:50.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:49 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:00:50.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:49 smithi177 ceph-mon[35305]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:50.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:49 smithi177 ceph-mon[35305]: mds.? [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] up:boot 2024-09-12T03:00:50.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:49 smithi177 ceph-mon[35305]: fsmap 12 up:standby 2024-09-12T03:00:50.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:49 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:00:50.247 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:49 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:50.247 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:49 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:50.247 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:49 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:50.247 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:49 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:50.247 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:49 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:50.247 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:49 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:00:50.247 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:49 smithi116 ceph-mon[31410]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:50.247 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:49 smithi116 ceph-mon[31410]: mds.? [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] up:boot 2024-09-12T03:00:50.247 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:49 smithi116 ceph-mon[31410]: fsmap 12 up:standby 2024-09-12T03:00:50.247 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:49 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:00:50.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.603+0000 7f48b3b4e640 1 -- 172.21.15.116:0/1722328545 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48ac100a90 msgr2=0x7f48ac100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:50.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.603+0000 7f48b3b4e640 1 --2- 172.21.15.116:0/1722328545 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48ac100a90 0x7f48ac100ef0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f489c009930 tx=0x7f489c02a420 comp rx=0 tx=0).stop 2024-09-12T03:00:50.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.603+0000 7f48b3b4e640 1 -- 172.21.15.116:0/1722328545 shutdown_connections 2024-09-12T03:00:50.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.603+0000 7f48b3b4e640 1 --2- 172.21.15.116:0/1722328545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48ac101430 0x7f48ac10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:50.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.603+0000 7f48b3b4e640 1 --2- 172.21.15.116:0/1722328545 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48ac100a90 0x7f48ac100ef0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:50.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.603+0000 7f48b3b4e640 1 --2- 172.21.15.116:0/1722328545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48ac106a90 0x7f48ac106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:50.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.603+0000 7f48b3b4e640 1 -- 172.21.15.116:0/1722328545 >> 172.21.15.116:0/1722328545 conn(0x7f48ac0fc7d0 msgr2=0x7f48ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:50.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.603+0000 7f48b3b4e640 1 -- 172.21.15.116:0/1722328545 shutdown_connections 2024-09-12T03:00:50.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.603+0000 7f48b3b4e640 1 -- 172.21.15.116:0/1722328545 wait complete. 2024-09-12T03:00:50.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.604+0000 7f48b3b4e640 1 Processor -- start 2024-09-12T03:00:50.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.604+0000 7f48b3b4e640 1 -- start start 2024-09-12T03:00:50.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.604+0000 7f48b3b4e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48ac100a90 0x7f48ac1a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:50.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.604+0000 7f48b3b4e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48ac101430 0x7f48ac1a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:50.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.604+0000 7f48b3b4e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48ac106a90 0x7f48ac19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:50.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.604+0000 7f48b3b4e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac110e90 con 0x7f48ac100a90 2024-09-12T03:00:50.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.604+0000 7f48b3b4e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac110d10 con 0x7f48ac106a90 2024-09-12T03:00:50.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.604+0000 7f48b3b4e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac111010 con 0x7f48ac101430 2024-09-12T03:00:50.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.604+0000 7f48b18ea640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48ac100a90 0x7f48ac1a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:50.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.604+0000 7f48b18ea640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48ac100a90 0x7f48ac1a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37022/0 (socket says 172.21.15.116:37022) 2024-09-12T03:00:50.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.604+0000 7f48b18ea640 1 -- 172.21.15.116:0/3466842980 learned_addr learned my addr 172.21.15.116:0/3466842980 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:00:50.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.604+0000 7f48b10e9640 1 --2- 172.21.15.116:0/3466842980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48ac101430 0x7f48ac1a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:50.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.604+0000 7f48b20eb640 1 --2- 172.21.15.116:0/3466842980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48ac106a90 0x7f48ac19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:50.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.604+0000 7f48b18ea640 1 -- 172.21.15.116:0/3466842980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48ac101430 msgr2=0x7f48ac1a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:50.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.604+0000 7f48b18ea640 1 --2- 172.21.15.116:0/3466842980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48ac101430 0x7f48ac1a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:50.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.604+0000 7f48b18ea640 1 -- 172.21.15.116:0/3466842980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48ac106a90 msgr2=0x7f48ac19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:50.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.604+0000 7f48b18ea640 1 --2- 172.21.15.116:0/3466842980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48ac106a90 0x7f48ac19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:50.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.604+0000 7f48b18ea640 1 -- 172.21.15.116:0/3466842980 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48ac19acf0 con 0x7f48ac100a90 2024-09-12T03:00:50.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.605+0000 7f48b18ea640 1 --2- 172.21.15.116:0/3466842980 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48ac100a90 0x7f48ac1a0400 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f48a800d950 tx=0x7f48a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:50.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.605+0000 7f48a2ffd640 1 -- 172.21.15.116:0/3466842980 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a8017070 con 0x7f48ac100a90 2024-09-12T03:00:50.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.605+0000 7f48a2ffd640 1 -- 172.21.15.116:0/3466842980 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48a8009c30 con 0x7f48ac100a90 2024-09-12T03:00:50.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.605+0000 7f48b3b4e640 1 -- 172.21.15.116:0/3466842980 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48ac19afe0 con 0x7f48ac100a90 2024-09-12T03:00:50.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.605+0000 7f48a2ffd640 1 -- 172.21.15.116:0/3466842980 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a800f6b0 con 0x7f48ac100a90 2024-09-12T03:00:50.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.605+0000 7f48b3b4e640 1 -- 172.21.15.116:0/3466842980 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48ac1a72d0 con 0x7f48ac100a90 2024-09-12T03:00:50.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.606+0000 7f48b3b4e640 1 -- 172.21.15.116:0/3466842980 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48ac100ef0 con 0x7f48ac100a90 2024-09-12T03:00:50.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.608+0000 7f48a2ffd640 1 -- 172.21.15.116:0/3466842980 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f48a80109d0 con 0x7f48ac100a90 2024-09-12T03:00:50.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.608+0000 7f48a2ffd640 1 --2- 172.21.15.116:0/3466842980 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f48980a2b80 0x7f48980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:50.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.608+0000 7f48b10e9640 1 --2- 172.21.15.116:0/3466842980 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f48980a2b80 0x7f48980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:50.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.608+0000 7f48a2ffd640 1 -- 172.21.15.116:0/3466842980 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f48a80ce6a0 con 0x7f48ac100a90 2024-09-12T03:00:50.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.609+0000 7f48b10e9640 1 --2- 172.21.15.116:0/3466842980 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f48980a2b80 0x7f48980a5040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f489c009900 tx=0x7f489c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:50.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.612+0000 7f48a2ffd640 1 -- 172.21.15.116:0/3466842980 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f48a8009de0 con 0x7f48ac100a90 2024-09-12T03:00:50.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.833+0000 7f48b3b4e640 1 -- 172.21.15.116:0/3466842980 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48ac1a74a0 con 0x7f48ac100a90 2024-09-12T03:00:50.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.834+0000 7f48a2ffd640 1 -- 172.21.15.116:0/3466842980 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10551 (secure 0 0 0) 0x7f48a809b450 con 0x7f48ac100a90 2024-09-12T03:00:50.835 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:00:50.835 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":13,"btime":"2024-09-12T03:00:49:201818+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24494,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-12T03:00:50.835 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 13 2024-09-12T03:00:50.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.836+0000 7f48b3b4e640 1 -- 172.21.15.116:0/3466842980 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f48980a2b80 msgr2=0x7f48980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:50.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.836+0000 7f48b3b4e640 1 --2- 172.21.15.116:0/3466842980 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f48980a2b80 0x7f48980a5040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f489c009900 tx=0x7f489c0058d0 comp rx=0 tx=0).stop 2024-09-12T03:00:50.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.836+0000 7f48b3b4e640 1 -- 172.21.15.116:0/3466842980 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48ac100a90 msgr2=0x7f48ac1a0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:50.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.836+0000 7f48b3b4e640 1 --2- 172.21.15.116:0/3466842980 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48ac100a90 0x7f48ac1a0400 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f48a800d950 tx=0x7f48a800de10 comp rx=0 tx=0).stop 2024-09-12T03:00:50.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.836+0000 7f48b3b4e640 1 -- 172.21.15.116:0/3466842980 shutdown_connections 2024-09-12T03:00:50.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.836+0000 7f48b3b4e640 1 --2- 172.21.15.116:0/3466842980 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f48980a2b80 0x7f48980a5040 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:50.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.836+0000 7f48b3b4e640 1 --2- 172.21.15.116:0/3466842980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48ac106a90 0x7f48ac19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:50.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.836+0000 7f48b3b4e640 1 --2- 172.21.15.116:0/3466842980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48ac101430 0x7f48ac1a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:50.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.836+0000 7f48b3b4e640 1 --2- 172.21.15.116:0/3466842980 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48ac100a90 0x7f48ac1a0400 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:50.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.836+0000 7f48b3b4e640 1 -- 172.21.15.116:0/3466842980 >> 172.21.15.116:0/3466842980 conn(0x7f48ac0fc7d0 msgr2=0x7f48ac10c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:50.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.836+0000 7f48b3b4e640 1 -- 172.21.15.116:0/3466842980 shutdown_connections 2024-09-12T03:00:50.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:50.836+0000 7f48b3b4e640 1 -- 172.21.15.116:0/3466842980 wait complete. 2024-09-12T03:00:51.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:51 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:51.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:51 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:51.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:51 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:51.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:51 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:51.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:51 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3466842980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:51.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:51 smithi116 ceph-mon[31410]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:51.601 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-12T03:00:51.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:51 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:51.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:51 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:51.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:51 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:51.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:51 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:51.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:51 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3466842980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:51.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:51 smithi183 ceph-mon[35535]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:51.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:51 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:51.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:51 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:51.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:51 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:51.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:51 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:51.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:51 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3466842980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:51.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:51 smithi177 ceph-mon[35305]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:51.863 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:52.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.369+0000 7fa3666b2640 1 -- 172.21.15.116:0/1224808101 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa360106f10 msgr2=0x7fa36010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:52.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.369+0000 7fa3666b2640 1 --2- 172.21.15.116:0/1224808101 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa360106f10 0x7fa36010ea90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa358009ca0 tx=0x7fa35802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:00:52.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.370+0000 7fa3666b2640 1 -- 172.21.15.116:0/1224808101 shutdown_connections 2024-09-12T03:00:52.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.370+0000 7fa3666b2640 1 --2- 172.21.15.116:0/1224808101 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa360106f10 0x7fa36010ea90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:52.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.370+0000 7fa3666b2640 1 --2- 172.21.15.116:0/1224808101 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa360102790 0x7fa3601067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:52.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.370+0000 7fa3666b2640 1 --2- 172.21.15.116:0/1224808101 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa360101de0 0x7fa3601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:52.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.370+0000 7fa3666b2640 1 -- 172.21.15.116:0/1224808101 >> 172.21.15.116:0/1224808101 conn(0x7fa3600fd8c0 msgr2=0x7fa3600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:52.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.370+0000 7fa3666b2640 1 -- 172.21.15.116:0/1224808101 shutdown_connections 2024-09-12T03:00:52.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.370+0000 7fa3666b2640 1 -- 172.21.15.116:0/1224808101 wait complete. 2024-09-12T03:00:52.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.370+0000 7fa3666b2640 1 Processor -- start 2024-09-12T03:00:52.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.370+0000 7fa3666b2640 1 -- start start 2024-09-12T03:00:52.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.370+0000 7fa3666b2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa360101de0 0x7fa3600744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:52.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.370+0000 7fa3666b2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa360102790 0x7fa360074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:52.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.370+0000 7fa3666b2640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa360106f10 0x7fa360074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:52.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.370+0000 7fa3666b2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa360111f70 con 0x7fa360101de0 2024-09-12T03:00:52.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.370+0000 7fa3666b2640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa360111df0 con 0x7fa360106f10 2024-09-12T03:00:52.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.370+0000 7fa3666b2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa3601120f0 con 0x7fa360102790 2024-09-12T03:00:52.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.371+0000 7fa35ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa360101de0 0x7fa3600744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:52.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.371+0000 7fa35ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa360101de0 0x7fa3600744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48170/0 (socket says 172.21.15.116:48170) 2024-09-12T03:00:52.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.371+0000 7fa35ffff640 1 -- 172.21.15.116:0/1012609967 learned_addr learned my addr 172.21.15.116:0/1012609967 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:00:52.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.371+0000 7fa364c4f640 1 --2- 172.21.15.116:0/1012609967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa360106f10 0x7fa360074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:52.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.371+0000 7fa357fff640 1 --2- 172.21.15.116:0/1012609967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa360102790 0x7fa360074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:52.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.371+0000 7fa35ffff640 1 -- 172.21.15.116:0/1012609967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa360102790 msgr2=0x7fa360074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:52.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.371+0000 7fa35ffff640 1 --2- 172.21.15.116:0/1012609967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa360102790 0x7fa360074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:52.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.371+0000 7fa35ffff640 1 -- 172.21.15.116:0/1012609967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa360106f10 msgr2=0x7fa360074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:52.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.371+0000 7fa35ffff640 1 --2- 172.21.15.116:0/1012609967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa360106f10 0x7fa360074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:52.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.371+0000 7fa35ffff640 1 -- 172.21.15.116:0/1012609967 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3601a6920 con 0x7fa360101de0 2024-09-12T03:00:52.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.371+0000 7fa364c4f640 1 --2- 172.21.15.116:0/1012609967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa360106f10 0x7fa360074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:00:52.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.371+0000 7fa35ffff640 1 --2- 172.21.15.116:0/1012609967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa360101de0 0x7fa3600744f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa35000d950 tx=0x7fa35000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:52.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.371+0000 7fa35dffb640 1 -- 172.21.15.116:0/1012609967 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa350017070 con 0x7fa360101de0 2024-09-12T03:00:52.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.371+0000 7fa35dffb640 1 -- 172.21.15.116:0/1012609967 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa350009c30 con 0x7fa360101de0 2024-09-12T03:00:52.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.371+0000 7fa3666b2640 1 -- 172.21.15.116:0/1012609967 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3601a6c10 con 0x7fa360101de0 2024-09-12T03:00:52.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.371+0000 7fa35dffb640 1 -- 172.21.15.116:0/1012609967 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35000f6b0 con 0x7fa360101de0 2024-09-12T03:00:52.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.371+0000 7fa3666b2640 1 -- 172.21.15.116:0/1012609967 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3601a7150 con 0x7fa360101de0 2024-09-12T03:00:52.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.372+0000 7fa3666b2640 1 -- 172.21.15.116:0/1012609967 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3601021c0 con 0x7fa360101de0 2024-09-12T03:00:52.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.373+0000 7fa35dffb640 1 -- 172.21.15.116:0/1012609967 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa3500109d0 con 0x7fa360101de0 2024-09-12T03:00:52.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.374+0000 7fa35dffb640 1 --2- 172.21.15.116:0/1012609967 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa34c0a2b80 0x7fa34c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:52.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.374+0000 7fa357fff640 1 --2- 172.21.15.116:0/1012609967 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa34c0a2b80 0x7fa34c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:52.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.374+0000 7fa35dffb640 1 -- 172.21.15.116:0/1012609967 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fa3500ce8a0 con 0x7fa360101de0 2024-09-12T03:00:52.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.374+0000 7fa357fff640 1 --2- 172.21.15.116:0/1012609967 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa34c0a2b80 0x7fa34c0a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa348005d70 tx=0x7fa348005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:52.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.378+0000 7fa35dffb640 1 -- 172.21.15.116:0/1012609967 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa350009de0 con 0x7fa360101de0 2024-09-12T03:00:52.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.563+0000 7fa3666b2640 1 -- 172.21.15.116:0/1012609967 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa36010a0f0 con 0x7fa360101de0 2024-09-12T03:00:52.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.563+0000 7fa35dffb640 1 -- 172.21.15.116:0/1012609967 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v73) ==== 84+0+65 (secure 0 0 0) 0x7fa35009bca0 con 0x7fa360101de0 2024-09-12T03:00:52.563 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:00:52.563 INFO:teuthology.orchestra.run.smithi116.stdout:[ 2024-09-12T03:00:52.564 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:00:52.564 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 1, 2024-09-12T03:00:52.564 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": ".mgr" 2024-09-12T03:00:52.564 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:00:52.564 INFO:teuthology.orchestra.run.smithi116.stdout:] 2024-09-12T03:00:52.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.565+0000 7fa3666b2640 1 -- 172.21.15.116:0/1012609967 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa34c0a2b80 msgr2=0x7fa34c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:52.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.565+0000 7fa3666b2640 1 --2- 172.21.15.116:0/1012609967 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa34c0a2b80 0x7fa34c0a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa348005d70 tx=0x7fa348005d00 comp rx=0 tx=0).stop 2024-09-12T03:00:52.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.565+0000 7fa3666b2640 1 -- 172.21.15.116:0/1012609967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa360101de0 msgr2=0x7fa3600744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:52.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.565+0000 7fa3666b2640 1 --2- 172.21.15.116:0/1012609967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa360101de0 0x7fa3600744f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa35000d950 tx=0x7fa35000de10 comp rx=0 tx=0).stop 2024-09-12T03:00:52.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.565+0000 7fa3666b2640 1 -- 172.21.15.116:0/1012609967 shutdown_connections 2024-09-12T03:00:52.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.565+0000 7fa3666b2640 1 --2- 172.21.15.116:0/1012609967 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa34c0a2b80 0x7fa34c0a5040 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:52.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.565+0000 7fa3666b2640 1 --2- 172.21.15.116:0/1012609967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa360106f10 0x7fa360074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:52.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.565+0000 7fa3666b2640 1 --2- 172.21.15.116:0/1012609967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa360102790 0x7fa360074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:52.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.565+0000 7fa3666b2640 1 --2- 172.21.15.116:0/1012609967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa360101de0 0x7fa3600744f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:52.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.565+0000 7fa3666b2640 1 -- 172.21.15.116:0/1012609967 >> 172.21.15.116:0/1012609967 conn(0x7fa3600fd8c0 msgr2=0x7fa3600fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:52.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.565+0000 7fa3666b2640 1 -- 172.21.15.116:0/1012609967 shutdown_connections 2024-09-12T03:00:52.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:52.565+0000 7fa3666b2640 1 -- 172.21.15.116:0/1012609967 wait complete. 2024-09-12T03:00:52.893 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:52 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:52.893 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:52 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:52.893 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:52 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:52.893 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:52 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:52.894 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:52 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:52.894 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:52 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:00:52.894 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:52 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1012609967' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T03:00:53.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:52 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:53.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:52 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:53.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:52 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:53.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:52 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:53.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:52 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:53.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:52 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:00:53.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:52 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1012609967' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T03:00:53.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:53.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:53.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:53.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:53.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:53.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:52 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:00:53.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:52 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1012609967' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T03:00:53.974 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T03:00:53.974 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T03:00:54.247 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:54.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:54 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:54.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:54 smithi116 ceph-mon[31410]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:54.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:54 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:54.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:54 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:54.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:54 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:54.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:54 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:54.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:54 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:54.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:54 smithi183 ceph-mon[35535]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:54.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:54 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:54.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:54 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:54.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:54 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:54.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:54 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:54.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:54 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:54.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:54 smithi177 ceph-mon[35305]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:54.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:54 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:54.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:54 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:54.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:54 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:54.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:54 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:54.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.778+0000 7fa8b07ca640 1 -- 172.21.15.116:0/188708478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8a8106a80 msgr2=0x7fa8a8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:54.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.779+0000 7fa8b07ca640 1 --2- 172.21.15.116:0/188708478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8a8106a80 0x7fa8a8106e60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4009930 tx=0x7fa8a402a420 comp rx=0 tx=0).stop 2024-09-12T03:00:54.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.779+0000 7fa8b07ca640 1 -- 172.21.15.116:0/188708478 shutdown_connections 2024-09-12T03:00:54.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.779+0000 7fa8b07ca640 1 --2- 172.21.15.116:0/188708478 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8a8101420 0x7fa8a810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:54.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.779+0000 7fa8b07ca640 1 --2- 172.21.15.116:0/188708478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8a8100a80 0x7fa8a8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:54.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.779+0000 7fa8b07ca640 1 --2- 172.21.15.116:0/188708478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8a8106a80 0x7fa8a8106e60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:54.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.779+0000 7fa8b07ca640 1 -- 172.21.15.116:0/188708478 >> 172.21.15.116:0/188708478 conn(0x7fa8a80fc7a0 msgr2=0x7fa8a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:54.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.779+0000 7fa8b07ca640 1 -- 172.21.15.116:0/188708478 shutdown_connections 2024-09-12T03:00:54.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.779+0000 7fa8b07ca640 1 -- 172.21.15.116:0/188708478 wait complete. 2024-09-12T03:00:54.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.779+0000 7fa8b07ca640 1 Processor -- start 2024-09-12T03:00:54.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.779+0000 7fa8b07ca640 1 -- start start 2024-09-12T03:00:54.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.779+0000 7fa8b07ca640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8a8100a80 0x7fa8a8102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:54.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.779+0000 7fa8b07ca640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8a8101420 0x7fa8a8102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:54.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.779+0000 7fa8b07ca640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8a8106a80 0x7fa8a81030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:54.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.779+0000 7fa8b07ca640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a8110ea0 con 0x7fa8a8106a80 2024-09-12T03:00:54.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.779+0000 7fa8b07ca640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a8110d20 con 0x7fa8a8101420 2024-09-12T03:00:54.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.779+0000 7fa8b07ca640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a8111020 con 0x7fa8a8100a80 2024-09-12T03:00:54.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.780+0000 7fa8aed67640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8a8106a80 0x7fa8a81030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:54.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.780+0000 7fa8aed67640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8a8106a80 0x7fa8a81030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48186/0 (socket says 172.21.15.116:48186) 2024-09-12T03:00:54.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.780+0000 7fa8aed67640 1 -- 172.21.15.116:0/3740164534 learned_addr learned my addr 172.21.15.116:0/3740164534 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:00:54.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.780+0000 7fa8add65640 1 --2- 172.21.15.116:0/3740164534 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8a8101420 0x7fa8a8102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:54.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.780+0000 7fa8ae566640 1 --2- 172.21.15.116:0/3740164534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8a8100a80 0x7fa8a8102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:54.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.780+0000 7fa8aed67640 1 -- 172.21.15.116:0/3740164534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8a8100a80 msgr2=0x7fa8a8102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:54.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.780+0000 7fa8aed67640 1 --2- 172.21.15.116:0/3740164534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8a8100a80 0x7fa8a8102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:54.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.780+0000 7fa8aed67640 1 -- 172.21.15.116:0/3740164534 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8a8101420 msgr2=0x7fa8a8102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:54.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.780+0000 7fa8aed67640 1 --2- 172.21.15.116:0/3740164534 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8a8101420 0x7fa8a8102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:54.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.780+0000 7fa8aed67640 1 -- 172.21.15.116:0/3740164534 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a81a6990 con 0x7fa8a8106a80 2024-09-12T03:00:54.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.780+0000 7fa8aed67640 1 --2- 172.21.15.116:0/3740164534 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8a8106a80 0x7fa8a81030d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa8a000eaa0 tx=0x7fa8a000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:54.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.780+0000 7fa89f7fe640 1 -- 172.21.15.116:0/3740164534 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0004d20 con 0x7fa8a8106a80 2024-09-12T03:00:54.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.780+0000 7fa8b07ca640 1 -- 172.21.15.116:0/3740164534 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8a81a6c80 con 0x7fa8a8106a80 2024-09-12T03:00:54.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.780+0000 7fa89f7fe640 1 -- 172.21.15.116:0/3740164534 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8a0005020 con 0x7fa8a8106a80 2024-09-12T03:00:54.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.780+0000 7fa89f7fe640 1 -- 172.21.15.116:0/3740164534 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a00116d0 con 0x7fa8a8106a80 2024-09-12T03:00:54.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.780+0000 7fa8b07ca640 1 -- 172.21.15.116:0/3740164534 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8a81a7170 con 0x7fa8a8106a80 2024-09-12T03:00:54.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.781+0000 7fa8b07ca640 1 -- 172.21.15.116:0/3740164534 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8a8100ee0 con 0x7fa8a8106a80 2024-09-12T03:00:54.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.782+0000 7fa89f7fe640 1 -- 172.21.15.116:0/3740164534 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa8a00051d0 con 0x7fa8a8106a80 2024-09-12T03:00:54.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.783+0000 7fa89f7fe640 1 --2- 172.21.15.116:0/3740164534 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa8940a2b80 0x7fa8940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:54.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.783+0000 7fa8ae566640 1 --2- 172.21.15.116:0/3740164534 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa8940a2b80 0x7fa8940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:54.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.783+0000 7fa89f7fe640 1 -- 172.21.15.116:0/3740164534 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fa8a00cee10 con 0x7fa8a8106a80 2024-09-12T03:00:54.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.783+0000 7fa8ae566640 1 --2- 172.21.15.116:0/3740164534 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa8940a2b80 0x7fa8940a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4000c00 tx=0x7fa8a4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:54.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.787+0000 7fa89f7fe640 1 -- 172.21.15.116:0/3740164534 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa8a0016030 con 0x7fa8a8106a80 2024-09-12T03:00:54.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:54.955+0000 7fa8b07ca640 1 -- 172.21.15.116:0/3740164534 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa8a81a7400 con 0x7fa8a8106a80 2024-09-12T03:00:55.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:55.298+0000 7fa89f7fe640 1 -- 172.21.15.116:0/3740164534 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v74) ==== 138+0+0 (secure 0 0 0) 0x7fa8a009c210 con 0x7fa8a8106a80 2024-09-12T03:00:55.407 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' created 2024-09-12T03:00:55.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:55.300+0000 7fa8b07ca640 1 -- 172.21.15.116:0/3740164534 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa8940a2b80 msgr2=0x7fa8940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:55.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:55.300+0000 7fa8b07ca640 1 --2- 172.21.15.116:0/3740164534 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa8940a2b80 0x7fa8940a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4000c00 tx=0x7fa8a4005a20 comp rx=0 tx=0).stop 2024-09-12T03:00:55.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:55.300+0000 7fa8b07ca640 1 -- 172.21.15.116:0/3740164534 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8a8106a80 msgr2=0x7fa8a81030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:55.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:55.300+0000 7fa8b07ca640 1 --2- 172.21.15.116:0/3740164534 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8a8106a80 0x7fa8a81030d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa8a000eaa0 tx=0x7fa8a000eea0 comp rx=0 tx=0).stop 2024-09-12T03:00:55.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:55.300+0000 7fa8b07ca640 1 -- 172.21.15.116:0/3740164534 shutdown_connections 2024-09-12T03:00:55.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:55.300+0000 7fa8b07ca640 1 --2- 172.21.15.116:0/3740164534 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa8940a2b80 0x7fa8940a5040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:55.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:55.300+0000 7fa8b07ca640 1 --2- 172.21.15.116:0/3740164534 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8a8106a80 0x7fa8a81030d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:55.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:55.300+0000 7fa8b07ca640 1 --2- 172.21.15.116:0/3740164534 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8a8101420 0x7fa8a8102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:55.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:55.300+0000 7fa8b07ca640 1 --2- 172.21.15.116:0/3740164534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8a8100a80 0x7fa8a8102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:55.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:55.300+0000 7fa8b07ca640 1 -- 172.21.15.116:0/3740164534 >> 172.21.15.116:0/3740164534 conn(0x7fa8a80fc7a0 msgr2=0x7fa8a81048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:55.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:55.301+0000 7fa8b07ca640 1 -- 172.21.15.116:0/3740164534 shutdown_connections 2024-09-12T03:00:55.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:55.301+0000 7fa8b07ca640 1 -- 172.21.15.116:0/3740164534 wait complete. 2024-09-12T03:00:55.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:55 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:55.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:55 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:55.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:55 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:55.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:55 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:55.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:55 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:55.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:55 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3740164534' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T03:00:55.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:55 smithi116 ceph-mon[31410]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:55.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:55 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:55.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:55 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:55.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:55 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:55.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:55 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:55.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:55 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:55.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:55 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3740164534' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T03:00:55.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:55 smithi183 ceph-mon[35535]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:55.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:55 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:55.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:55 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:55.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:55 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:55.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:55 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:55.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:55 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:00:55.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:55 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3740164534' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T03:00:55.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:55 smithi177 ceph-mon[35305]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:56.020 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T03:00:56.266 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:56.558 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:56 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3740164534' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T03:00:56.558 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:56 smithi116 ceph-mon[31410]: osdmap e74: 12 total, 12 up, 12 in 2024-09-12T03:00:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:56 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3740164534' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T03:00:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:56 smithi183 ceph-mon[35535]: osdmap e74: 12 total, 12 up, 12 in 2024-09-12T03:00:56.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:56 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3740164534' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T03:00:56.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:56 smithi177 ceph-mon[35305]: osdmap e74: 12 total, 12 up, 12 in 2024-09-12T03:00:56.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.755+0000 7f1d3144c640 1 -- 172.21.15.116:0/3357254048 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d2c100e20 msgr2=0x7f1d2c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:56.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.755+0000 7f1d3144c640 1 --2- 172.21.15.116:0/3357254048 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d2c100e20 0x7f1d2c101200 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c009900 tx=0x7f1d1c02a480 comp rx=0 tx=0).stop 2024-09-12T03:00:56.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.755+0000 7f1d3144c640 1 -- 172.21.15.116:0/3357254048 shutdown_connections 2024-09-12T03:00:56.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.755+0000 7f1d3144c640 1 --2- 172.21.15.116:0/3357254048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d2c102ef0 0x7f1d2c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:56.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.755+0000 7f1d3144c640 1 --2- 172.21.15.116:0/3357254048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d2c102570 0x7f1d2c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:56.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.755+0000 7f1d3144c640 1 --2- 172.21.15.116:0/3357254048 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d2c100e20 0x7f1d2c101200 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:56.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.755+0000 7f1d3144c640 1 -- 172.21.15.116:0/3357254048 >> 172.21.15.116:0/3357254048 conn(0x7f1d2c0fc7d0 msgr2=0x7f1d2c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:56.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.755+0000 7f1d3144c640 1 -- 172.21.15.116:0/3357254048 shutdown_connections 2024-09-12T03:00:56.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.755+0000 7f1d3144c640 1 -- 172.21.15.116:0/3357254048 wait complete. 2024-09-12T03:00:56.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.756+0000 7f1d3144c640 1 Processor -- start 2024-09-12T03:00:56.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.756+0000 7f1d3144c640 1 -- start start 2024-09-12T03:00:56.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.756+0000 7f1d3144c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d2c100e20 0x7f1d2c1974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:56.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.756+0000 7f1d3144c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d2c102570 0x7f1d2c197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:56.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.756+0000 7f1d3144c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d2c102ef0 0x7f1d2c19bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:56.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.756+0000 7f1d3144c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c110fa0 con 0x7f1d2c102570 2024-09-12T03:00:56.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.756+0000 7f1d3144c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c110e20 con 0x7f1d2c102ef0 2024-09-12T03:00:56.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.756+0000 7f1d3144c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c111120 con 0x7f1d2c100e20 2024-09-12T03:00:56.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.756+0000 7f1d23fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d2c102570 0x7f1d2c197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:56.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.756+0000 7f1d23fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d2c102570 0x7f1d2c197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48196/0 (socket says 172.21.15.116:48196) 2024-09-12T03:00:56.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.756+0000 7f1d23fff640 1 -- 172.21.15.116:0/249343881 learned_addr learned my addr 172.21.15.116:0/249343881 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:00:56.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.756+0000 7f1d2affd640 1 --2- 172.21.15.116:0/249343881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d2c100e20 0x7f1d2c1974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:56.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.757+0000 7f1d2b7fe640 1 --2- 172.21.15.116:0/249343881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d2c102ef0 0x7f1d2c19bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:56.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.757+0000 7f1d23fff640 1 -- 172.21.15.116:0/249343881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d2c100e20 msgr2=0x7f1d2c1974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:56.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.757+0000 7f1d23fff640 1 --2- 172.21.15.116:0/249343881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d2c100e20 0x7f1d2c1974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:56.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.757+0000 7f1d23fff640 1 -- 172.21.15.116:0/249343881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d2c102ef0 msgr2=0x7f1d2c19bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:56.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.757+0000 7f1d23fff640 1 --2- 172.21.15.116:0/249343881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d2c102ef0 0x7f1d2c19bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:56.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.757+0000 7f1d23fff640 1 -- 172.21.15.116:0/249343881 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d2c19c470 con 0x7f1d2c102570 2024-09-12T03:00:56.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.757+0000 7f1d23fff640 1 --2- 172.21.15.116:0/249343881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d2c102570 0x7f1d2c197a00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1d14009970 tx=0x7f1d14009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:56.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.757+0000 7f1d28ff9640 1 -- 172.21.15.116:0/249343881 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1400d020 con 0x7f1d2c102570 2024-09-12T03:00:56.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.757+0000 7f1d3144c640 1 -- 172.21.15.116:0/249343881 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d2c19c760 con 0x7f1d2c102570 2024-09-12T03:00:56.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.757+0000 7f1d28ff9640 1 -- 172.21.15.116:0/249343881 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d14007d40 con 0x7f1d2c102570 2024-09-12T03:00:56.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.757+0000 7f1d28ff9640 1 -- 172.21.15.116:0/249343881 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1400ccc0 con 0x7f1d2c102570 2024-09-12T03:00:56.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.757+0000 7f1d3144c640 1 -- 172.21.15.116:0/249343881 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d2c069f70 con 0x7f1d2c102570 2024-09-12T03:00:56.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.758+0000 7f1d3144c640 1 -- 172.21.15.116:0/249343881 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d2c101200 con 0x7f1d2c102570 2024-09-12T03:00:56.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.759+0000 7f1d28ff9640 1 -- 172.21.15.116:0/249343881 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1d1400c460 con 0x7f1d2c102570 2024-09-12T03:00:56.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.760+0000 7f1d28ff9640 1 --2- 172.21.15.116:0/249343881 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1d180a2b80 0x7f1d180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:56.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.760+0000 7f1d2affd640 1 --2- 172.21.15.116:0/249343881 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1d180a2b80 0x7f1d180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:56.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.760+0000 7f1d28ff9640 1 -- 172.21.15.116:0/249343881 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7497+0+0 (secure 0 0 0) 0x7f1d140d01a0 con 0x7f1d2c102570 2024-09-12T03:00:56.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.760+0000 7f1d2affd640 1 --2- 172.21.15.116:0/249343881 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1d180a2b80 0x7f1d180a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c009230 tx=0x7f1d1c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:56.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.764+0000 7f1d28ff9640 1 -- 172.21.15.116:0/249343881 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d1409d430 con 0x7f1d2c102570 2024-09-12T03:00:56.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:56.930+0000 7f1d3144c640 1 -- 172.21.15.116:0/249343881 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1d2c06a990 con 0x7f1d2c102570 2024-09-12T03:00:57.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:57.316+0000 7f1d28ff9640 1 -- 172.21.15.116:0/249343881 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v76) ==== 170+0+0 (secure 0 0 0) 0x7f1d1409cd30 con 0x7f1d2c102570 2024-09-12T03:00:57.316 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' created 2024-09-12T03:00:57.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:57.318+0000 7f1d3144c640 1 -- 172.21.15.116:0/249343881 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1d180a2b80 msgr2=0x7f1d180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:57.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:57.318+0000 7f1d3144c640 1 --2- 172.21.15.116:0/249343881 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1d180a2b80 0x7f1d180a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c009230 tx=0x7f1d1c0058d0 comp rx=0 tx=0).stop 2024-09-12T03:00:57.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:57.318+0000 7f1d3144c640 1 -- 172.21.15.116:0/249343881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d2c102570 msgr2=0x7f1d2c197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:57.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:57.318+0000 7f1d3144c640 1 --2- 172.21.15.116:0/249343881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d2c102570 0x7f1d2c197a00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1d14009970 tx=0x7f1d14009e30 comp rx=0 tx=0).stop 2024-09-12T03:00:57.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:57.318+0000 7f1d3144c640 1 -- 172.21.15.116:0/249343881 shutdown_connections 2024-09-12T03:00:57.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:57.318+0000 7f1d3144c640 1 --2- 172.21.15.116:0/249343881 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1d180a2b80 0x7f1d180a5040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:57.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:57.318+0000 7f1d3144c640 1 --2- 172.21.15.116:0/249343881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d2c102ef0 0x7f1d2c19bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:57.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:57.318+0000 7f1d3144c640 1 --2- 172.21.15.116:0/249343881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d2c102570 0x7f1d2c197a00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:57.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:57.318+0000 7f1d3144c640 1 --2- 172.21.15.116:0/249343881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d2c100e20 0x7f1d2c1974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:57.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:57.318+0000 7f1d3144c640 1 -- 172.21.15.116:0/249343881 >> 172.21.15.116:0/249343881 conn(0x7f1d2c0fc7d0 msgr2=0x7f1d2c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:57.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:57.318+0000 7f1d3144c640 1 -- 172.21.15.116:0/249343881 shutdown_connections 2024-09-12T03:00:57.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:57.318+0000 7f1d3144c640 1 -- 172.21.15.116:0/249343881 wait complete. 2024-09-12T03:00:57.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:57 smithi183 ceph-mon[35535]: osdmap e75: 12 total, 12 up, 12 in 2024-09-12T03:00:57.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:57 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/249343881' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T03:00:57.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:57 smithi183 ceph-mon[35535]: pgmap v272: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:57.599 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:57 smithi116 ceph-mon[31410]: osdmap e75: 12 total, 12 up, 12 in 2024-09-12T03:00:57.599 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:57 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/249343881' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T03:00:57.599 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:57 smithi116 ceph-mon[31410]: pgmap v272: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:57.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:57 smithi177 ceph-mon[35305]: osdmap e75: 12 total, 12 up, 12 in 2024-09-12T03:00:57.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:57 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/249343881' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T03:00:57.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:57 smithi177 ceph-mon[35305]: pgmap v272: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:58.054 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T03:00:58.310 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:58.600 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:58 smithi116 ceph-mon[31410]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T03:00:58.601 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:58 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/249343881' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T03:00:58.601 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:58 smithi116 ceph-mon[31410]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T03:00:58.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:58 smithi183 ceph-mon[35535]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T03:00:58.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:58 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/249343881' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T03:00:58.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:58 smithi183 ceph-mon[35535]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T03:00:58.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:58 smithi177 ceph-mon[35305]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T03:00:58.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:58 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/249343881' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T03:00:58.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:58 smithi177 ceph-mon[35305]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T03:00:59.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.985+0000 7f746b326640 1 -- 172.21.15.116:0/895783871 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f746410c180 msgr2=0x7f746410e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:59.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.985+0000 7f746b326640 1 --2- 172.21.15.116:0/895783871 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f746410c180 0x7f746410e600 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f745c009ca0 tx=0x7f745c02a950 comp rx=0 tx=0).stop 2024-09-12T03:00:59.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.985+0000 7f746b326640 1 -- 172.21.15.116:0/895783871 shutdown_connections 2024-09-12T03:00:59.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.985+0000 7f746b326640 1 --2- 172.21.15.116:0/895783871 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f746410c180 0x7f746410e600 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:59.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.985+0000 7f746b326640 1 --2- 172.21.15.116:0/895783871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74641036c0 0x7f746410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:59.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.985+0000 7f746b326640 1 --2- 172.21.15.116:0/895783871 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7464102da0 0x7f7464103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:59.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.985+0000 7f746b326640 1 -- 172.21.15.116:0/895783871 >> 172.21.15.116:0/895783871 conn(0x7f74640fc7a0 msgr2=0x7f74640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:59.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.985+0000 7f746b326640 1 -- 172.21.15.116:0/895783871 shutdown_connections 2024-09-12T03:00:59.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.985+0000 7f746b326640 1 -- 172.21.15.116:0/895783871 wait complete. 2024-09-12T03:00:59.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.986+0000 7f746b326640 1 Processor -- start 2024-09-12T03:00:59.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.986+0000 7f746b326640 1 -- start start 2024-09-12T03:00:59.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.986+0000 7f746b326640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7464102da0 0x7f7464198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:59.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.986+0000 7f746b326640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74641036c0 0x7f74641988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:59.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.986+0000 7f746b326640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f746410c180 0x7f746419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:59.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.986+0000 7f746b326640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7464111fa0 con 0x7f7464102da0 2024-09-12T03:00:59.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.986+0000 7f746b326640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7464111e20 con 0x7f746410c180 2024-09-12T03:00:59.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.986+0000 7f746b326640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7464112120 con 0x7f74641036c0 2024-09-12T03:00:59.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.986+0000 7f74690c2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7464102da0 0x7f7464198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:59.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.986+0000 7f74690c2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7464102da0 0x7f7464198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48228/0 (socket says 172.21.15.116:48228) 2024-09-12T03:00:59.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.986+0000 7f74690c2640 1 -- 172.21.15.116:0/4049283516 learned_addr learned my addr 172.21.15.116:0/4049283516 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:00:59.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.987+0000 7f74698c3640 1 --2- 172.21.15.116:0/4049283516 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f746410c180 0x7f746419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:59.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.987+0000 7f74690c2640 1 -- 172.21.15.116:0/4049283516 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74641036c0 msgr2=0x7f74641988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:59.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.987+0000 7f74690c2640 1 --2- 172.21.15.116:0/4049283516 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74641036c0 0x7f74641988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:59.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.987+0000 7f74690c2640 1 -- 172.21.15.116:0/4049283516 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f746410c180 msgr2=0x7f746419cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:59.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.987+0000 7f74690c2640 1 --2- 172.21.15.116:0/4049283516 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f746410c180 0x7f746419cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:59.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.987+0000 7f74690c2640 1 -- 172.21.15.116:0/4049283516 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746419d340 con 0x7f7464102da0 2024-09-12T03:00:59.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.987+0000 7f74690c2640 1 --2- 172.21.15.116:0/4049283516 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7464102da0 0x7f7464198390 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f746000c920 tx=0x7f746000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:59.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.987+0000 7f745a7fc640 1 -- 172.21.15.116:0/4049283516 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460007c70 con 0x7f7464102da0 2024-09-12T03:00:59.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.987+0000 7f745a7fc640 1 -- 172.21.15.116:0/4049283516 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7460007e10 con 0x7f7464102da0 2024-09-12T03:00:59.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.987+0000 7f746b326640 1 -- 172.21.15.116:0/4049283516 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f746419d630 con 0x7f7464102da0 2024-09-12T03:00:59.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.987+0000 7f745a7fc640 1 -- 172.21.15.116:0/4049283516 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460018890 con 0x7f7464102da0 2024-09-12T03:00:59.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.987+0000 7f746b326640 1 -- 172.21.15.116:0/4049283516 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74641a4f10 con 0x7f7464102da0 2024-09-12T03:00:59.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.988+0000 7f746b326640 1 -- 172.21.15.116:0/4049283516 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7464103180 con 0x7f7464102da0 2024-09-12T03:00:59.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.990+0000 7f745a7fc640 1 -- 172.21.15.116:0/4049283516 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f7460018a50 con 0x7f7464102da0 2024-09-12T03:00:59.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.990+0000 7f745a7fc640 1 --2- 172.21.15.116:0/4049283516 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f74500a2c00 0x7f74500a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:59.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.990+0000 7f74688c1640 1 --2- 172.21.15.116:0/4049283516 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f74500a2c00 0x7f74500a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:59.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.990+0000 7f745a7fc640 1 -- 172.21.15.116:0/4049283516 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7879+0+0 (secure 0 0 0) 0x7f74600cf280 con 0x7f7464102da0 2024-09-12T03:00:59.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.991+0000 7f74688c1640 1 --2- 172.21.15.116:0/4049283516 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f74500a2c00 0x7f74500a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f74640fed30 tx=0x7f7454006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:59.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:58.994+0000 7f745a7fc640 1 -- 172.21.15.116:0/4049283516 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7460014030 con 0x7f7464102da0 2024-09-12T03:00:59.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:59.193+0000 7f746b326640 1 -- 172.21.15.116:0/4049283516 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f74641996f0 con 0x7f7464102da0 2024-09-12T03:00:59.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:59.334+0000 7f745a7fc640 1 -- 172.21.15.116:0/4049283516 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f746009c4a0 con 0x7f7464102da0 2024-09-12T03:00:59.335 INFO:teuthology.orchestra.run.smithi116.stderr:new fs with metadata pool 2 and data pool 3 2024-09-12T03:00:59.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:59.336+0000 7f746b326640 1 -- 172.21.15.116:0/4049283516 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f74500a2c00 msgr2=0x7f74500a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:59.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:59.336+0000 7f746b326640 1 --2- 172.21.15.116:0/4049283516 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f74500a2c00 0x7f74500a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f74640fed30 tx=0x7f7454006c90 comp rx=0 tx=0).stop 2024-09-12T03:00:59.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:59.336+0000 7f746b326640 1 -- 172.21.15.116:0/4049283516 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7464102da0 msgr2=0x7f7464198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:59.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:59.336+0000 7f746b326640 1 --2- 172.21.15.116:0/4049283516 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7464102da0 0x7f7464198390 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f746000c920 tx=0x7f746000cde0 comp rx=0 tx=0).stop 2024-09-12T03:00:59.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:59.336+0000 7f746b326640 1 -- 172.21.15.116:0/4049283516 shutdown_connections 2024-09-12T03:00:59.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:59.336+0000 7f746b326640 1 --2- 172.21.15.116:0/4049283516 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f74500a2c00 0x7f74500a50c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:59.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:59.336+0000 7f746b326640 1 --2- 172.21.15.116:0/4049283516 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f746410c180 0x7f746419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:59.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:59.336+0000 7f746b326640 1 --2- 172.21.15.116:0/4049283516 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74641036c0 0x7f74641988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:59.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:59.336+0000 7f746b326640 1 --2- 172.21.15.116:0/4049283516 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7464102da0 0x7f7464198390 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:59.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:59.336+0000 7f746b326640 1 -- 172.21.15.116:0/4049283516 >> 172.21.15.116:0/4049283516 conn(0x7f74640fc7a0 msgr2=0x7f74640fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:59.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:59.336+0000 7f746b326640 1 -- 172.21.15.116:0/4049283516 shutdown_connections 2024-09-12T03:00:59.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:00:59.336+0000 7f746b326640 1 -- 172.21.15.116:0/4049283516 wait complete. 2024-09-12T03:00:59.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:59 smithi116 ceph-mon[31410]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T03:00:59.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:59 smithi116 ceph-mon[31410]: pgmap v275: 97 pgs: 39 creating+peering, 48 unknown, 10 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:59.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:59 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/4049283516' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T03:00:59.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:00:59 smithi116 ceph-mon[31410]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-12T03:00:59.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:59 smithi183 ceph-mon[35535]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T03:00:59.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:59 smithi183 ceph-mon[35535]: pgmap v275: 97 pgs: 39 creating+peering, 48 unknown, 10 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:59.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:59 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/4049283516' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T03:00:59.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:00:59 smithi183 ceph-mon[35535]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-12T03:00:59.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:59 smithi177 ceph-mon[35305]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T03:00:59.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:59 smithi177 ceph-mon[35305]: pgmap v275: 97 pgs: 39 creating+peering, 48 unknown, 10 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:00:59.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:59 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/4049283516' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T03:00:59.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:00:59 smithi177 ceph-mon[35305]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-12T03:01:00.050 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:01:00.297 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:00.686 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:00 smithi116 ceph-mon[31410]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T03:01:00.686 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:00 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/4049283516' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T03:01:00.687 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:00 smithi116 ceph-mon[31410]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-12T03:01:00.687 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:00 smithi116 ceph-mon[31410]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-12T03:01:00.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:00 smithi183 ceph-mon[35535]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T03:01:00.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:00 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/4049283516' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T03:01:00.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:00 smithi183 ceph-mon[35535]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-12T03:01:00.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:00 smithi183 ceph-mon[35535]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-12T03:01:00.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:00 smithi177 ceph-mon[35305]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T03:01:00.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:00 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/4049283516' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T03:01:00.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:00 smithi177 ceph-mon[35305]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-12T03:01:00.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:00 smithi177 ceph-mon[35305]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-12T03:01:00.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.765+0000 7fb8bf1f0640 1 -- 172.21.15.116:0/1484118884 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8b8102ef0 msgr2=0x7fb8b810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:00.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.765+0000 7fb8bf1f0640 1 --2- 172.21.15.116:0/1484118884 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8b8102ef0 0x7fb8b810da40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0009900 tx=0x7fb8a002a480 comp rx=0 tx=0).stop 2024-09-12T03:01:00.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.765+0000 7fb8bf1f0640 1 -- 172.21.15.116:0/1484118884 shutdown_connections 2024-09-12T03:01:00.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.765+0000 7fb8bf1f0640 1 --2- 172.21.15.116:0/1484118884 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8b8102ef0 0x7fb8b810da40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.765+0000 7fb8bf1f0640 1 --2- 172.21.15.116:0/1484118884 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8b8102570 0x7fb8b81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.765+0000 7fb8bf1f0640 1 --2- 172.21.15.116:0/1484118884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8b8100e20 0x7fb8b8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.765+0000 7fb8bf1f0640 1 -- 172.21.15.116:0/1484118884 >> 172.21.15.116:0/1484118884 conn(0x7fb8b80fc7d0 msgr2=0x7fb8b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:00.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.765+0000 7fb8bf1f0640 1 -- 172.21.15.116:0/1484118884 shutdown_connections 2024-09-12T03:01:00.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.765+0000 7fb8bf1f0640 1 -- 172.21.15.116:0/1484118884 wait complete. 2024-09-12T03:01:00.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.765+0000 7fb8bf1f0640 1 Processor -- start 2024-09-12T03:01:00.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.765+0000 7fb8bf1f0640 1 -- start start 2024-09-12T03:01:00.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.765+0000 7fb8bf1f0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8b8100e20 0x7fb8b8198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:00.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.765+0000 7fb8bf1f0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8b8102570 0x7fb8b81989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:00.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.765+0000 7fb8bf1f0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8b8102ef0 0x7fb8b819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:00.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.765+0000 7fb8bf1f0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b8110fa0 con 0x7fb8b8102ef0 2024-09-12T03:01:00.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.765+0000 7fb8bf1f0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b8110e20 con 0x7fb8b8100e20 2024-09-12T03:01:00.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.765+0000 7fb8bf1f0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b8111120 con 0x7fb8b8102570 2024-09-12T03:01:00.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.766+0000 7fb8bd78d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8b8102ef0 0x7fb8b819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:00.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.766+0000 7fb8bd78d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8b8102ef0 0x7fb8b819cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48254/0 (socket says 172.21.15.116:48254) 2024-09-12T03:01:00.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.766+0000 7fb8bd78d640 1 -- 172.21.15.116:0/2596081394 learned_addr learned my addr 172.21.15.116:0/2596081394 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:00.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.766+0000 7fb8bcf8c640 1 --2- 172.21.15.116:0/2596081394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8b8100e20 0x7fb8b8198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:00.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.766+0000 7fb8b7fff640 1 --2- 172.21.15.116:0/2596081394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8b8102570 0x7fb8b81989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:00.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.766+0000 7fb8bd78d640 1 -- 172.21.15.116:0/2596081394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8b8102570 msgr2=0x7fb8b81989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:00.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.766+0000 7fb8bd78d640 1 --2- 172.21.15.116:0/2596081394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8b8102570 0x7fb8b81989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.766+0000 7fb8bd78d640 1 -- 172.21.15.116:0/2596081394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8b8100e20 msgr2=0x7fb8b8198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:00.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.766+0000 7fb8bd78d640 1 --2- 172.21.15.116:0/2596081394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8b8100e20 0x7fb8b8198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.766+0000 7fb8bd78d640 1 -- 172.21.15.116:0/2596081394 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b819d440 con 0x7fb8b8102ef0 2024-09-12T03:01:00.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.766+0000 7fb8b7fff640 1 --2- 172.21.15.116:0/2596081394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8b8102570 0x7fb8b81989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:01:00.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.766+0000 7fb8bd78d640 1 --2- 172.21.15.116:0/2596081394 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8b8102ef0 0x7fb8b819cd60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0005ab0 tx=0x7fb8a0003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:00.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.766+0000 7fb8b5ffb640 1 -- 172.21.15.116:0/2596081394 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a003f070 con 0x7fb8b8102ef0 2024-09-12T03:01:00.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.766+0000 7fb8b5ffb640 1 -- 172.21.15.116:0/2596081394 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8a0031e30 con 0x7fb8b8102ef0 2024-09-12T03:01:00.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.766+0000 7fb8b5ffb640 1 -- 172.21.15.116:0/2596081394 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0035030 con 0x7fb8b8102ef0 2024-09-12T03:01:00.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.766+0000 7fb8bf1f0640 1 -- 172.21.15.116:0/2596081394 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8b819d6d0 con 0x7fb8b8102ef0 2024-09-12T03:01:00.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.767+0000 7fb8bf1f0640 1 -- 172.21.15.116:0/2596081394 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8b81a5010 con 0x7fb8b8102ef0 2024-09-12T03:01:00.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.767+0000 7fb8bf1f0640 1 -- 172.21.15.116:0/2596081394 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8b8101200 con 0x7fb8b8102ef0 2024-09-12T03:01:00.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.769+0000 7fb8b5ffb640 1 -- 172.21.15.116:0/2596081394 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fb8a0042070 con 0x7fb8b8102ef0 2024-09-12T03:01:00.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.769+0000 7fb8b5ffb640 1 --2- 172.21.15.116:0/2596081394 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb8a40a2c00 0x7fb8a40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:00.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.770+0000 7fb8bcf8c640 1 --2- 172.21.15.116:0/2596081394 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb8a40a2c00 0x7fb8a40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:00.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.770+0000 7fb8b5ffb640 1 -- 172.21.15.116:0/2596081394 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fb8a00efa60 con 0x7fb8b8102ef0 2024-09-12T03:01:00.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.770+0000 7fb8bcf8c640 1 --2- 172.21.15.116:0/2596081394 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb8a40a2c00 0x7fb8a40a50c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb8b000abe0 tx=0x7fb8b000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:00.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.774+0000 7fb8b5ffb640 1 -- 172.21.15.116:0/2596081394 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb8a0003e20 con 0x7fb8b8102ef0 2024-09-12T03:01:00.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.941+0000 7fb8bf1f0640 1 -- 172.21.15.116:0/2596081394 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb8b810a0a0 con 0x7fb8b8102ef0 2024-09-12T03:01:00.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.943+0000 7fb8b5ffb640 1 -- 172.21.15.116:0/2596081394 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22325 (secure 0 0 0) 0x7fb8a00bcb10 con 0x7fb8b8102ef0 2024-09-12T03:01:00.944 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:01:00.944 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":78,"fsid":"eb1ac602-70b1-11ef-bcea-c7b262605968","created":"2024-09-12T02:53:07.105686+0000","modified":"2024-09-12T03:00:59.195124+0000","last_up_change":"2024-09-12T03:00:07.202639+0000","last_in_change":"2024-09-12T02:59:43.446336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:56:17.347152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:00:55.191040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:00:57.171142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8cf5a18-dc6f-4bb6-bd74-78adda310118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6803","nonce":862366853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6805","nonce":862366853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6809","nonce":862366853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6807","nonce":862366853}]},"public_addr":"172.21.15.116:6803/862366853","cluster_addr":"172.21.15.116:6805/862366853","heartbeat_back_addr":"172.21.15.116:6809/862366853","heartbeat_front_addr":"172.21.15.116:6807/862366853","state":["exists","up"]},{"osd":1,"uuid":"0affd3cb-9a51-42b0-a2cc-f30784a75cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6801","nonce":2245800547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6803","nonce":2245800547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6807","nonce":2245800547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6805","nonce":2245800547}]},"public_addr":"172.21.15.177:6801/2245800547","cluster_addr":"172.21.15.177:6803/2245800547","heartbeat_back_addr":"172.21.15.177:6807/2245800547","heartbeat_front_addr":"172.21.15.177:6805/2245800547","state":["exists","up"]},{"osd":2,"uuid":"1ce45ea5-84df-4a35-924f-78f92bd4f2cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6801","nonce":2645677759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6803","nonce":2645677759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6807","nonce":2645677759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6805","nonce":2645677759}]},"public_addr":"172.21.15.183:6801/2645677759","cluster_addr":"172.21.15.183:6803/2645677759","heartbeat_back_addr":"172.21.15.183:6807/2645677759","heartbeat_front_addr":"172.21.15.183:6805/2645677759","state":["exists","up"]},{"osd":3,"uuid":"ae5e8178-c998-4648-bc05-4bce822f0cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6811","nonce":686622148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6813","nonce":686622148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6817","nonce":686622148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6815","nonce":686622148}]},"public_addr":"172.21.15.116:6811/686622148","cluster_addr":"172.21.15.116:6813/686622148","heartbeat_back_addr":"172.21.15.116:6817/686622148","heartbeat_front_addr":"172.21.15.116:6815/686622148","state":["exists","up"]},{"osd":4,"uuid":"c81d2e73-0bc2-440d-b5a2-b7428af3ea4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6809","nonce":2382938021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6811","nonce":2382938021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6815","nonce":2382938021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6813","nonce":2382938021}]},"public_addr":"172.21.15.177:6809/2382938021","cluster_addr":"172.21.15.177:6811/2382938021","heartbeat_back_addr":"172.21.15.177:6815/2382938021","heartbeat_front_addr":"172.21.15.177:6813/2382938021","state":["exists","up"]},{"osd":5,"uuid":"c3f168be-9e27-4cf5-b73e-3f47d8990ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6809","nonce":85363432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6811","nonce":85363432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6815","nonce":85363432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6813","nonce":85363432}]},"public_addr":"172.21.15.183:6809/85363432","cluster_addr":"172.21.15.183:6811/85363432","heartbeat_back_addr":"172.21.15.183:6815/85363432","heartbeat_front_addr":"172.21.15.183:6813/85363432","state":["exists","up"]},{"osd":6,"uuid":"beb0a7e9-4909-47e2-9ee7-6951d4745e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6819","nonce":1121291186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6821","nonce":1121291186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6825","nonce":1121291186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6823","nonce":1121291186}]},"public_addr":"172.21.15.116:6819/1121291186","cluster_addr":"172.21.15.116:6821/1121291186","heartbeat_back_addr":"172.21.15.116:6825/1121291186","heartbeat_front_addr":"172.21.15.116:6823/1121291186","state":["exists","up"]},{"osd":7,"uuid":"761f4680-f049-473c-bf44-b19acf8de24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6817","nonce":2177665961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6819","nonce":2177665961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6823","nonce":2177665961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6821","nonce":2177665961}]},"public_addr":"172.21.15.177:6817/2177665961","cluster_addr":"172.21.15.177:6819/2177665961","heartbeat_back_addr":"172.21.15.177:6823/2177665961","heartbeat_front_addr":"172.21.15.177:6821/2177665961","state":["exists","up"]},{"osd":8,"uuid":"2385ea6c-fd5a-4815-9596-77f948629542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6817","nonce":1305725258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6819","nonce":1305725258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6823","nonce":1305725258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6821","nonce":1305725258}]},"public_addr":"172.21.15.183:6817/1305725258","cluster_addr":"172.21.15.183:6819/1305725258","heartbeat_back_addr":"172.21.15.183:6823/1305725258","heartbeat_front_addr":"172.21.15.183:6821/1305725258","state":["exists","up"]},{"osd":9,"uuid":"fd0c0dc4-4990-4f12-aa14-f855504cd995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6827","nonce":3831673668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6829","nonce":3831673668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6833","nonce":3831673668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6831","nonce":3831673668}]},"public_addr":"172.21.15.116:6827/3831673668","cluster_addr":"172.21.15.116:6829/3831673668","heartbeat_back_addr":"172.21.15.116:6833/3831673668","heartbeat_front_addr":"172.21.15.116:6831/3831673668","state":["exists","up"]},{"osd":10,"uuid":"864d1a97-4810-49c8-ad4a-a0c10253833c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6825","nonce":963663634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6827","nonce":963663634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6831","nonce":963663634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6829","nonce":963663634}]},"public_addr":"172.21.15.177:6825/963663634","cluster_addr":"172.21.15.177:6827/963663634","heartbeat_back_addr":"172.21.15.177:6831/963663634","heartbeat_front_addr":"172.21.15.177:6829/963663634","state":["exists","up"]},{"osd":11,"uuid":"d49c0841-4620-4f2f-a6b5-f2a9aa2e3c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6825","nonce":1818319939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6827","nonce":1818319939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6831","nonce":1818319939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6829","nonce":1818319939}]},"public_addr":"172.21.15.183:6825/1818319939","cluster_addr":"172.21.15.183:6827/1818319939","heartbeat_back_addr":"172.21.15.183:6831/1818319939","heartbeat_front_addr":"172.21.15.183:6829/1818319939","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:24.736944+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:50.310980+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:56:14.989533+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:56:41.187047+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:06.981664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:32.872771+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:58.850089+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:23.043121+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:47.839958+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:14.087708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:40.010548+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:05.364820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.116:6801/2966274423":"2024-09-13T02:54:03.026862+0000","172.21.15.116:6800/2966274423":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/686139455":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/1552047390":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/2180363411":"2024-09-13T02:53:39.433769+0000","172.21.15.116:6801/1791219601":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/3907400460":"2024-09-13T02:54:03.026862+0000","172.21.15.116:6800/1791219601":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/1802566286":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/212337802":"2024-09-13T02:53:39.433769+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:01:00.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.946+0000 7fb8bf1f0640 1 -- 172.21.15.116:0/2596081394 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb8a40a2c00 msgr2=0x7fb8a40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:00.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.946+0000 7fb8bf1f0640 1 --2- 172.21.15.116:0/2596081394 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb8a40a2c00 0x7fb8a40a50c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb8b000abe0 tx=0x7fb8b000a2d0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.946+0000 7fb8bf1f0640 1 -- 172.21.15.116:0/2596081394 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8b8102ef0 msgr2=0x7fb8b819cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:00.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.946+0000 7fb8bf1f0640 1 --2- 172.21.15.116:0/2596081394 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8b8102ef0 0x7fb8b819cd60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0005ab0 tx=0x7fb8a0003c30 comp rx=0 tx=0).stop 2024-09-12T03:01:00.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.946+0000 7fb8bf1f0640 1 -- 172.21.15.116:0/2596081394 shutdown_connections 2024-09-12T03:01:00.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.946+0000 7fb8bf1f0640 1 --2- 172.21.15.116:0/2596081394 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb8a40a2c00 0x7fb8a40a50c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.946+0000 7fb8bf1f0640 1 --2- 172.21.15.116:0/2596081394 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8b8102ef0 0x7fb8b819cd60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.946+0000 7fb8bf1f0640 1 --2- 172.21.15.116:0/2596081394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8b8102570 0x7fb8b81989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.946+0000 7fb8bf1f0640 1 --2- 172.21.15.116:0/2596081394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8b8100e20 0x7fb8b8198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:00.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.946+0000 7fb8bf1f0640 1 -- 172.21.15.116:0/2596081394 >> 172.21.15.116:0/2596081394 conn(0x7fb8b80fc7d0 msgr2=0x7fb8b806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:00.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.946+0000 7fb8bf1f0640 1 -- 172.21.15.116:0/2596081394 shutdown_connections 2024-09-12T03:01:00.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:00.946+0000 7fb8bf1f0640 1 -- 172.21.15.116:0/2596081394 wait complete. 2024-09-12T03:01:01.641 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:01 smithi116 ceph-mon[31410]: mds.? [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] up:active 2024-09-12T03:01:01.641 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:01 smithi116 ceph-mon[31410]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:01:01.641 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:01 smithi116 ceph-mon[31410]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:01:01.641 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:01 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2596081394' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:01:01.641 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:01 smithi116 ceph-mon[31410]: pgmap v277: 97 pgs: 39 creating+peering, 12 unknown, 46 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 213 B/s wr, 4 op/s 2024-09-12T03:01:01.668 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:01:01.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:01 smithi183 ceph-mon[35535]: mds.? [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] up:active 2024-09-12T03:01:01.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:01 smithi183 ceph-mon[35535]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:01:01.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:01 smithi183 ceph-mon[35535]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:01:01.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:01 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2596081394' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:01:01.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:01 smithi183 ceph-mon[35535]: pgmap v277: 97 pgs: 39 creating+peering, 12 unknown, 46 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 213 B/s wr, 4 op/s 2024-09-12T03:01:01.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:01 smithi177 ceph-mon[35305]: mds.? [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] up:active 2024-09-12T03:01:01.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:01 smithi177 ceph-mon[35305]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:01:01.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:01 smithi177 ceph-mon[35305]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:01:01.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:01 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2596081394' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:01:01.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:01 smithi177 ceph-mon[35305]: pgmap v277: 97 pgs: 39 creating+peering, 12 unknown, 46 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 213 B/s wr, 4 op/s 2024-09-12T03:01:01.917 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:02.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.407+0000 7fa9de50a640 1 -- 172.21.15.116:0/1992099736 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9d8100a80 msgr2=0x7fa9d8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:02.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.407+0000 7fa9de50a640 1 --2- 172.21.15.116:0/1992099736 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9d8100a80 0x7fa9d8100ee0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009930 tx=0x7fa9cc02a420 comp rx=0 tx=0).stop 2024-09-12T03:01:02.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.407+0000 7fa9de50a640 1 -- 172.21.15.116:0/1992099736 shutdown_connections 2024-09-12T03:01:02.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.407+0000 7fa9de50a640 1 --2- 172.21.15.116:0/1992099736 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9d8101420 0x7fa9d810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:02.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.407+0000 7fa9de50a640 1 --2- 172.21.15.116:0/1992099736 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9d8100a80 0x7fa9d8100ee0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:02.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.407+0000 7fa9de50a640 1 --2- 172.21.15.116:0/1992099736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9d8106a80 0x7fa9d8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:02.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.407+0000 7fa9de50a640 1 -- 172.21.15.116:0/1992099736 >> 172.21.15.116:0/1992099736 conn(0x7fa9d80fc7a0 msgr2=0x7fa9d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:02.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.407+0000 7fa9de50a640 1 -- 172.21.15.116:0/1992099736 shutdown_connections 2024-09-12T03:01:02.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.407+0000 7fa9de50a640 1 -- 172.21.15.116:0/1992099736 wait complete. 2024-09-12T03:01:02.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.407+0000 7fa9de50a640 1 Processor -- start 2024-09-12T03:01:02.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.407+0000 7fa9de50a640 1 -- start start 2024-09-12T03:01:02.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.408+0000 7fa9de50a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9d8100a80 0x7fa9d8197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:02.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.408+0000 7fa9de50a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9d8101420 0x7fa9d8197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:02.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.408+0000 7fa9de50a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9d8106a80 0x7fa9d819bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:02.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.408+0000 7fa9de50a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d8110ec0 con 0x7fa9d8106a80 2024-09-12T03:01:02.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.408+0000 7fa9de50a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d8110d40 con 0x7fa9d8100a80 2024-09-12T03:01:02.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.408+0000 7fa9de50a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d8111040 con 0x7fa9d8101420 2024-09-12T03:01:02.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.408+0000 7fa9dcaa7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9d8106a80 0x7fa9d819bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:02.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.408+0000 7fa9dcaa7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9d8106a80 0x7fa9d819bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57496/0 (socket says 172.21.15.116:57496) 2024-09-12T03:01:02.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.408+0000 7fa9dcaa7640 1 -- 172.21.15.116:0/2443389751 learned_addr learned my addr 172.21.15.116:0/2443389751 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:02.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.408+0000 7fa9d77fe640 1 --2- 172.21.15.116:0/2443389751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9d8101420 0x7fa9d8197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:02.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.408+0000 7fa9d7fff640 1 --2- 172.21.15.116:0/2443389751 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9d8100a80 0x7fa9d8197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:02.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.408+0000 7fa9dcaa7640 1 -- 172.21.15.116:0/2443389751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9d8101420 msgr2=0x7fa9d8197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:02.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.408+0000 7fa9dcaa7640 1 --2- 172.21.15.116:0/2443389751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9d8101420 0x7fa9d8197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:02.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.408+0000 7fa9dcaa7640 1 -- 172.21.15.116:0/2443389751 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9d8100a80 msgr2=0x7fa9d8197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:02.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.408+0000 7fa9dcaa7640 1 --2- 172.21.15.116:0/2443389751 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9d8100a80 0x7fa9d8197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:02.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.408+0000 7fa9dcaa7640 1 -- 172.21.15.116:0/2443389751 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d819c3b0 con 0x7fa9d8106a80 2024-09-12T03:01:02.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.408+0000 7fa9d7fff640 1 --2- 172.21.15.116:0/2443389751 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9d8100a80 0x7fa9d8197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:01:02.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.409+0000 7fa9dcaa7640 1 --2- 172.21.15.116:0/2443389751 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9d8106a80 0x7fa9d819bcd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000b970 tx=0x7fa9d000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:02.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.409+0000 7fa9d57fa640 1 -- 172.21.15.116:0/2443389751 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0019070 con 0x7fa9d8106a80 2024-09-12T03:01:02.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.409+0000 7fa9d57fa640 1 -- 172.21.15.116:0/2443389751 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9d000cb40 con 0x7fa9d8106a80 2024-09-12T03:01:02.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.409+0000 7fa9de50a640 1 -- 172.21.15.116:0/2443389751 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d819c6a0 con 0x7fa9d8106a80 2024-09-12T03:01:02.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.409+0000 7fa9d57fa640 1 -- 172.21.15.116:0/2443389751 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0007650 con 0x7fa9d8106a80 2024-09-12T03:01:02.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.409+0000 7fa9de50a640 1 -- 172.21.15.116:0/2443389751 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d81a3f80 con 0x7fa9d8106a80 2024-09-12T03:01:02.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.410+0000 7fa9de50a640 1 -- 172.21.15.116:0/2443389751 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9d8100ee0 con 0x7fa9d8106a80 2024-09-12T03:01:02.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.411+0000 7fa9d57fa640 1 -- 172.21.15.116:0/2443389751 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa9d001c070 con 0x7fa9d8106a80 2024-09-12T03:01:02.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.412+0000 7fa9d57fa640 1 --2- 172.21.15.116:0/2443389751 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa9c00a2b80 0x7fa9c00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:02.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.412+0000 7fa9d7fff640 1 --2- 172.21.15.116:0/2443389751 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa9c00a2b80 0x7fa9c00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:02.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.412+0000 7fa9d57fa640 1 -- 172.21.15.116:0/2443389751 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fa9d00cea00 con 0x7fa9d8106a80 2024-09-12T03:01:02.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.412+0000 7fa9d7fff640 1 --2- 172.21.15.116:0/2443389751 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa9c00a2b80 0x7fa9c00a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa9c80058c0 tx=0x7fa9c8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:02.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.415+0000 7fa9d57fa640 1 -- 172.21.15.116:0/2443389751 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa9d009bab0 con 0x7fa9d8106a80 2024-09-12T03:01:02.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.582+0000 7fa9de50a640 1 -- 172.21.15.116:0/2443389751 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9d8109070 con 0x7fa9d8106a80 2024-09-12T03:01:02.582 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:02 smithi116 ceph-mon[31410]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T03:01:02.583 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:02 smithi116 ceph-mon[31410]: Cluster is now healthy 2024-09-12T03:01:02.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.583+0000 7fa9d57fa640 1 -- 172.21.15.116:0/2443389751 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22325 (secure 0 0 0) 0x7fa9d009b3b0 con 0x7fa9d8106a80 2024-09-12T03:01:02.584 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:01:02.584 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":78,"fsid":"eb1ac602-70b1-11ef-bcea-c7b262605968","created":"2024-09-12T02:53:07.105686+0000","modified":"2024-09-12T03:00:59.195124+0000","last_up_change":"2024-09-12T03:00:07.202639+0000","last_in_change":"2024-09-12T02:59:43.446336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:56:17.347152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:00:55.191040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:00:57.171142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8cf5a18-dc6f-4bb6-bd74-78adda310118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6803","nonce":862366853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6805","nonce":862366853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6809","nonce":862366853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6807","nonce":862366853}]},"public_addr":"172.21.15.116:6803/862366853","cluster_addr":"172.21.15.116:6805/862366853","heartbeat_back_addr":"172.21.15.116:6809/862366853","heartbeat_front_addr":"172.21.15.116:6807/862366853","state":["exists","up"]},{"osd":1,"uuid":"0affd3cb-9a51-42b0-a2cc-f30784a75cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6801","nonce":2245800547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6803","nonce":2245800547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6807","nonce":2245800547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6805","nonce":2245800547}]},"public_addr":"172.21.15.177:6801/2245800547","cluster_addr":"172.21.15.177:6803/2245800547","heartbeat_back_addr":"172.21.15.177:6807/2245800547","heartbeat_front_addr":"172.21.15.177:6805/2245800547","state":["exists","up"]},{"osd":2,"uuid":"1ce45ea5-84df-4a35-924f-78f92bd4f2cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6801","nonce":2645677759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6803","nonce":2645677759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6807","nonce":2645677759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6805","nonce":2645677759}]},"public_addr":"172.21.15.183:6801/2645677759","cluster_addr":"172.21.15.183:6803/2645677759","heartbeat_back_addr":"172.21.15.183:6807/2645677759","heartbeat_front_addr":"172.21.15.183:6805/2645677759","state":["exists","up"]},{"osd":3,"uuid":"ae5e8178-c998-4648-bc05-4bce822f0cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6811","nonce":686622148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6813","nonce":686622148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6817","nonce":686622148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6815","nonce":686622148}]},"public_addr":"172.21.15.116:6811/686622148","cluster_addr":"172.21.15.116:6813/686622148","heartbeat_back_addr":"172.21.15.116:6817/686622148","heartbeat_front_addr":"172.21.15.116:6815/686622148","state":["exists","up"]},{"osd":4,"uuid":"c81d2e73-0bc2-440d-b5a2-b7428af3ea4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6809","nonce":2382938021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6811","nonce":2382938021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6815","nonce":2382938021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6813","nonce":2382938021}]},"public_addr":"172.21.15.177:6809/2382938021","cluster_addr":"172.21.15.177:6811/2382938021","heartbeat_back_addr":"172.21.15.177:6815/2382938021","heartbeat_front_addr":"172.21.15.177:6813/2382938021","state":["exists","up"]},{"osd":5,"uuid":"c3f168be-9e27-4cf5-b73e-3f47d8990ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6809","nonce":85363432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6811","nonce":85363432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6815","nonce":85363432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6813","nonce":85363432}]},"public_addr":"172.21.15.183:6809/85363432","cluster_addr":"172.21.15.183:6811/85363432","heartbeat_back_addr":"172.21.15.183:6815/85363432","heartbeat_front_addr":"172.21.15.183:6813/85363432","state":["exists","up"]},{"osd":6,"uuid":"beb0a7e9-4909-47e2-9ee7-6951d4745e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6819","nonce":1121291186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6821","nonce":1121291186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6825","nonce":1121291186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6823","nonce":1121291186}]},"public_addr":"172.21.15.116:6819/1121291186","cluster_addr":"172.21.15.116:6821/1121291186","heartbeat_back_addr":"172.21.15.116:6825/1121291186","heartbeat_front_addr":"172.21.15.116:6823/1121291186","state":["exists","up"]},{"osd":7,"uuid":"761f4680-f049-473c-bf44-b19acf8de24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6817","nonce":2177665961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6819","nonce":2177665961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6823","nonce":2177665961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6821","nonce":2177665961}]},"public_addr":"172.21.15.177:6817/2177665961","cluster_addr":"172.21.15.177:6819/2177665961","heartbeat_back_addr":"172.21.15.177:6823/2177665961","heartbeat_front_addr":"172.21.15.177:6821/2177665961","state":["exists","up"]},{"osd":8,"uuid":"2385ea6c-fd5a-4815-9596-77f948629542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6817","nonce":1305725258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6819","nonce":1305725258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6823","nonce":1305725258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6821","nonce":1305725258}]},"public_addr":"172.21.15.183:6817/1305725258","cluster_addr":"172.21.15.183:6819/1305725258","heartbeat_back_addr":"172.21.15.183:6823/1305725258","heartbeat_front_addr":"172.21.15.183:6821/1305725258","state":["exists","up"]},{"osd":9,"uuid":"fd0c0dc4-4990-4f12-aa14-f855504cd995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6827","nonce":3831673668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6829","nonce":3831673668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6833","nonce":3831673668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6831","nonce":3831673668}]},"public_addr":"172.21.15.116:6827/3831673668","cluster_addr":"172.21.15.116:6829/3831673668","heartbeat_back_addr":"172.21.15.116:6833/3831673668","heartbeat_front_addr":"172.21.15.116:6831/3831673668","state":["exists","up"]},{"osd":10,"uuid":"864d1a97-4810-49c8-ad4a-a0c10253833c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6825","nonce":963663634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6827","nonce":963663634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6831","nonce":963663634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6829","nonce":963663634}]},"public_addr":"172.21.15.177:6825/963663634","cluster_addr":"172.21.15.177:6827/963663634","heartbeat_back_addr":"172.21.15.177:6831/963663634","heartbeat_front_addr":"172.21.15.177:6829/963663634","state":["exists","up"]},{"osd":11,"uuid":"d49c0841-4620-4f2f-a6b5-f2a9aa2e3c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6825","nonce":1818319939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6827","nonce":1818319939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6831","nonce":1818319939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6829","nonce":1818319939}]},"public_addr":"172.21.15.183:6825/1818319939","cluster_addr":"172.21.15.183:6827/1818319939","heartbeat_back_addr":"172.21.15.183:6831/1818319939","heartbeat_front_addr":"172.21.15.183:6829/1818319939","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:24.736944+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:50.310980+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:56:14.989533+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:56:41.187047+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:06.981664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:32.872771+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:58.850089+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:23.043121+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:47.839958+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:14.087708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:40.010548+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:05.364820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.116:6801/2966274423":"2024-09-13T02:54:03.026862+0000","172.21.15.116:6800/2966274423":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/686139455":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/1552047390":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/2180363411":"2024-09-13T02:53:39.433769+0000","172.21.15.116:6801/1791219601":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/3907400460":"2024-09-13T02:54:03.026862+0000","172.21.15.116:6800/1791219601":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/1802566286":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/212337802":"2024-09-13T02:53:39.433769+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:01:02.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.585+0000 7fa9de50a640 1 -- 172.21.15.116:0/2443389751 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa9c00a2b80 msgr2=0x7fa9c00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:02.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.585+0000 7fa9de50a640 1 --2- 172.21.15.116:0/2443389751 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa9c00a2b80 0x7fa9c00a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa9c80058c0 tx=0x7fa9c8005850 comp rx=0 tx=0).stop 2024-09-12T03:01:02.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.585+0000 7fa9de50a640 1 -- 172.21.15.116:0/2443389751 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9d8106a80 msgr2=0x7fa9d819bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:02.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.585+0000 7fa9de50a640 1 --2- 172.21.15.116:0/2443389751 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9d8106a80 0x7fa9d819bcd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000b970 tx=0x7fa9d000be30 comp rx=0 tx=0).stop 2024-09-12T03:01:02.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.586+0000 7fa9de50a640 1 -- 172.21.15.116:0/2443389751 shutdown_connections 2024-09-12T03:01:02.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.586+0000 7fa9de50a640 1 --2- 172.21.15.116:0/2443389751 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa9c00a2b80 0x7fa9c00a5040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:02.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.586+0000 7fa9de50a640 1 --2- 172.21.15.116:0/2443389751 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9d8106a80 0x7fa9d819bcd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:02.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.586+0000 7fa9de50a640 1 --2- 172.21.15.116:0/2443389751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9d8101420 0x7fa9d8197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:02.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.586+0000 7fa9de50a640 1 --2- 172.21.15.116:0/2443389751 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9d8100a80 0x7fa9d8197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:02.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.586+0000 7fa9de50a640 1 -- 172.21.15.116:0/2443389751 >> 172.21.15.116:0/2443389751 conn(0x7fa9d80fc7a0 msgr2=0x7fa9d80fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:02.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.586+0000 7fa9de50a640 1 -- 172.21.15.116:0/2443389751 shutdown_connections 2024-09-12T03:01:02.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:02.586+0000 7fa9de50a640 1 -- 172.21.15.116:0/2443389751 wait complete. 2024-09-12T03:01:02.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:02 smithi183 ceph-mon[35535]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T03:01:02.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:02 smithi183 ceph-mon[35535]: Cluster is now healthy 2024-09-12T03:01:02.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:02 smithi177 ceph-mon[35305]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T03:01:02.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:02 smithi177 ceph-mon[35305]: Cluster is now healthy 2024-09-12T03:01:03.293 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-12T03:01:03.541 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:03.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:03 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2443389751' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:01:03.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:03 smithi116 ceph-mon[31410]: pgmap v278: 97 pgs: 13 creating+peering, 84 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-12T03:01:03.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:03 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2443389751' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:01:03.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:03 smithi183 ceph-mon[35535]: pgmap v278: 97 pgs: 13 creating+peering, 84 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-12T03:01:03.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:03 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2443389751' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:01:03.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:03 smithi177 ceph-mon[35305]: pgmap v278: 97 pgs: 13 creating+peering, 84 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-09-12T03:01:04.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.042+0000 7f8385d24640 1 -- 172.21.15.116:0/4184291933 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8380102d30 msgr2=0x7f8380103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:04.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.042+0000 7f8385d24640 1 --2- 172.21.15.116:0/4184291933 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8380102d30 0x7f8380103110 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8370009930 tx=0x7f837002a420 comp rx=0 tx=0).stop 2024-09-12T03:01:04.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.042+0000 7f8385d24640 1 -- 172.21.15.116:0/4184291933 shutdown_connections 2024-09-12T03:01:04.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.042+0000 7f8385d24640 1 --2- 172.21.15.116:0/4184291933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f83801082a0 0x7f838010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:04.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.042+0000 7f8385d24640 1 --2- 172.21.15.116:0/4184291933 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83801036e0 0x7f8380107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:04.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.042+0000 7f8385d24640 1 --2- 172.21.15.116:0/4184291933 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8380102d30 0x7f8380103110 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:04.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.042+0000 7f8385d24640 1 -- 172.21.15.116:0/4184291933 >> 172.21.15.116:0/4184291933 conn(0x7f83800fc7d0 msgr2=0x7f83800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:04.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.042+0000 7f8385d24640 1 -- 172.21.15.116:0/4184291933 shutdown_connections 2024-09-12T03:01:04.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.042+0000 7f8385d24640 1 -- 172.21.15.116:0/4184291933 wait complete. 2024-09-12T03:01:04.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.043+0000 7f8385d24640 1 Processor -- start 2024-09-12T03:01:04.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.043+0000 7f8385d24640 1 -- start start 2024-09-12T03:01:04.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.043+0000 7f8385d24640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8380102d30 0x7f83801985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:04.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.043+0000 7f8385d24640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f83801036e0 0x7f8380198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:04.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.043+0000 7f8385d24640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83801082a0 0x7f838019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:04.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.043+0000 7f8385d24640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f83801120a0 con 0x7f83801036e0 2024-09-12T03:01:04.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.043+0000 7f8385d24640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8380111f20 con 0x7f83801082a0 2024-09-12T03:01:04.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.043+0000 7f8385d24640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8380112220 con 0x7f8380102d30 2024-09-12T03:01:04.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.043+0000 7f8376ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f83801036e0 0x7f8380198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:04.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.043+0000 7f8376ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f83801036e0 0x7f8380198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57512/0 (socket says 172.21.15.116:57512) 2024-09-12T03:01:04.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.043+0000 7f8376ffd640 1 -- 172.21.15.116:0/365334429 learned_addr learned my addr 172.21.15.116:0/365334429 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:04.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.043+0000 7f837f7fe640 1 --2- 172.21.15.116:0/365334429 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8380102d30 0x7f83801985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:04.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.043+0000 7f837ffff640 1 --2- 172.21.15.116:0/365334429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83801082a0 0x7f838019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:04.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.044+0000 7f8376ffd640 1 -- 172.21.15.116:0/365334429 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8380102d30 msgr2=0x7f83801985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:04.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.044+0000 7f8376ffd640 1 --2- 172.21.15.116:0/365334429 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8380102d30 0x7f83801985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:04.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.044+0000 7f8376ffd640 1 -- 172.21.15.116:0/365334429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83801082a0 msgr2=0x7f838019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:04.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.044+0000 7f8376ffd640 1 --2- 172.21.15.116:0/365334429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83801082a0 0x7f838019ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:04.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.044+0000 7f8376ffd640 1 -- 172.21.15.116:0/365334429 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f838019d570 con 0x7f83801036e0 2024-09-12T03:01:04.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.044+0000 7f837ffff640 1 --2- 172.21.15.116:0/365334429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83801082a0 0x7f838019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:01:04.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.044+0000 7f837f7fe640 1 --2- 172.21.15.116:0/365334429 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8380102d30 0x7f83801985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:01:04.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.044+0000 7f8376ffd640 1 --2- 172.21.15.116:0/365334429 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f83801036e0 0x7f8380198b00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f836800d740 tx=0x7f836800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:04.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.044+0000 7f837d7fa640 1 -- 172.21.15.116:0/365334429 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368017070 con 0x7f83801036e0 2024-09-12T03:01:04.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.044+0000 7f8385d24640 1 -- 172.21.15.116:0/365334429 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f838019d860 con 0x7f83801036e0 2024-09-12T03:01:04.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.044+0000 7f837d7fa640 1 -- 172.21.15.116:0/365334429 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f836800de50 con 0x7f83801036e0 2024-09-12T03:01:04.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.044+0000 7f837d7fa640 1 -- 172.21.15.116:0/365334429 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836800f6b0 con 0x7f83801036e0 2024-09-12T03:01:04.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.044+0000 7f8385d24640 1 -- 172.21.15.116:0/365334429 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83801a5140 con 0x7f83801036e0 2024-09-12T03:01:04.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.045+0000 7f8385d24640 1 -- 172.21.15.116:0/365334429 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f838006a850 con 0x7f83801036e0 2024-09-12T03:01:04.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.046+0000 7f837d7fa640 1 -- 172.21.15.116:0/365334429 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f836801a070 con 0x7f83801036e0 2024-09-12T03:01:04.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.047+0000 7f837d7fa640 1 --2- 172.21.15.116:0/365334429 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f836c0a2b80 0x7f836c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:04.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.047+0000 7f837f7fe640 1 --2- 172.21.15.116:0/365334429 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f836c0a2b80 0x7f836c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:04.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.047+0000 7f837d7fa640 1 -- 172.21.15.116:0/365334429 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f83680ce9b0 con 0x7f83801036e0 2024-09-12T03:01:04.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.047+0000 7f837f7fe640 1 --2- 172.21.15.116:0/365334429 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f836c0a2b80 0x7f836c0a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8370000c00 tx=0x7f8370005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:04.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.051+0000 7f837d7fa640 1 -- 172.21.15.116:0/365334429 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f836809ba60 con 0x7f83801036e0 2024-09-12T03:01:04.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.252+0000 7f8385d24640 1 -- 172.21.15.116:0/365334429 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f838010a2b0 con 0x7f83801036e0 2024-09-12T03:01:04.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.357+0000 7f837d7fa640 1 -- 172.21.15.116:0/365334429 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f836809b360 con 0x7f83801036e0 2024-09-12T03:01:04.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.359+0000 7f8385d24640 1 -- 172.21.15.116:0/365334429 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f836c0a2b80 msgr2=0x7f836c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:04.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.359+0000 7f8385d24640 1 --2- 172.21.15.116:0/365334429 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f836c0a2b80 0x7f836c0a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8370000c00 tx=0x7f8370005a20 comp rx=0 tx=0).stop 2024-09-12T03:01:04.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.359+0000 7f8385d24640 1 -- 172.21.15.116:0/365334429 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f83801036e0 msgr2=0x7f8380198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:04.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.359+0000 7f8385d24640 1 --2- 172.21.15.116:0/365334429 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f83801036e0 0x7f8380198b00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f836800d740 tx=0x7f836800dc00 comp rx=0 tx=0).stop 2024-09-12T03:01:04.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.360+0000 7f8385d24640 1 -- 172.21.15.116:0/365334429 shutdown_connections 2024-09-12T03:01:04.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.360+0000 7f8385d24640 1 --2- 172.21.15.116:0/365334429 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f836c0a2b80 0x7f836c0a5040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:04.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.360+0000 7f8385d24640 1 --2- 172.21.15.116:0/365334429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83801082a0 0x7f838019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:04.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.360+0000 7f8385d24640 1 --2- 172.21.15.116:0/365334429 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f83801036e0 0x7f8380198b00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:04.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.360+0000 7f8385d24640 1 --2- 172.21.15.116:0/365334429 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8380102d30 0x7f83801985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:04.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.360+0000 7f8385d24640 1 -- 172.21.15.116:0/365334429 >> 172.21.15.116:0/365334429 conn(0x7f83800fc7d0 msgr2=0x7f8380105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:04.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.360+0000 7f8385d24640 1 -- 172.21.15.116:0/365334429 shutdown_connections 2024-09-12T03:01:04.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:04.360+0000 7f8385d24640 1 -- 172.21.15.116:0/365334429 wait complete. 2024-09-12T03:01:04.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:04 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/365334429' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T03:01:04.495 INFO:journalctl@ceph.mds.b.smithi177.stdout:Sep 12 03:01:04 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-b[65339]: 2024-09-12T03:01:04.210+0000 7f60a7b52640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T03:01:04.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:04 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/365334429' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T03:01:04.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:04 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/365334429' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T03:01:05.081 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-09-12T03:01:05.081 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-09-12T03:01:05.331 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:05.616 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:05 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/365334429' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T03:01:05.616 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:05 smithi116 ceph-mon[31410]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:01:05.616 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:05 smithi116 ceph-mon[31410]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-12T03:01:05.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:05 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/365334429' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T03:01:05.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:05 smithi183 ceph-mon[35535]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:01:05.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:05 smithi183 ceph-mon[35535]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-12T03:01:05.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:05 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/365334429' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T03:01:05.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:05 smithi177 ceph-mon[35305]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:01:05.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:05 smithi177 ceph-mon[35305]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-12T03:01:05.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.825+0000 7f183fef0640 1 -- 172.21.15.116:0/298215957 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1838102d40 msgr2=0x7f1838103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:05.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.825+0000 7f183fef0640 1 --2- 172.21.15.116:0/298215957 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1838102d40 0x7f1838103120 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1834009900 tx=0x7f183402a480 comp rx=0 tx=0).stop 2024-09-12T03:01:05.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.825+0000 7f183fef0640 1 -- 172.21.15.116:0/298215957 shutdown_connections 2024-09-12T03:01:05.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.825+0000 7f183fef0640 1 --2- 172.21.15.116:0/298215957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f183810c170 0x7f183810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:05.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.825+0000 7f183fef0640 1 --2- 172.21.15.116:0/298215957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1838103660 0x7f183810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:05.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.825+0000 7f183fef0640 1 --2- 172.21.15.116:0/298215957 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1838102d40 0x7f1838103120 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:05.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.825+0000 7f183fef0640 1 -- 172.21.15.116:0/298215957 >> 172.21.15.116:0/298215957 conn(0x7f18380fc7a0 msgr2=0x7f18380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:05.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.826+0000 7f183fef0640 1 -- 172.21.15.116:0/298215957 shutdown_connections 2024-09-12T03:01:05.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.826+0000 7f183fef0640 1 -- 172.21.15.116:0/298215957 wait complete. 2024-09-12T03:01:05.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.826+0000 7f183fef0640 1 Processor -- start 2024-09-12T03:01:05.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.826+0000 7f183fef0640 1 -- start start 2024-09-12T03:01:05.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.826+0000 7f183fef0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1838102d40 0x7f1838198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:05.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.826+0000 7f183fef0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1838103660 0x7f1838198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:05.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.826+0000 7f183fef0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f183810c170 0x7f183819cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:05.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.826+0000 7f183fef0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1838112030 con 0x7f1838102d40 2024-09-12T03:01:05.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.826+0000 7f183fef0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1838111eb0 con 0x7f1838103660 2024-09-12T03:01:05.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.826+0000 7f183fef0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f18381121b0 con 0x7f183810c170 2024-09-12T03:01:05.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.826+0000 7f183dc8c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1838102d40 0x7f1838198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:05.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.826+0000 7f183dc8c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1838102d40 0x7f1838198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57520/0 (socket says 172.21.15.116:57520) 2024-09-12T03:01:05.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.826+0000 7f183dc8c640 1 -- 172.21.15.116:0/3782737763 learned_addr learned my addr 172.21.15.116:0/3782737763 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:05.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.827+0000 7f183d48b640 1 --2- 172.21.15.116:0/3782737763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1838103660 0x7f1838198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:05.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.827+0000 7f183dc8c640 1 -- 172.21.15.116:0/3782737763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f183810c170 msgr2=0x7f183819cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:05.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.827+0000 7f183dc8c640 1 --2- 172.21.15.116:0/3782737763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f183810c170 0x7f183819cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:05.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.827+0000 7f183dc8c640 1 -- 172.21.15.116:0/3782737763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1838103660 msgr2=0x7f1838198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:05.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.827+0000 7f183dc8c640 1 --2- 172.21.15.116:0/3782737763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1838103660 0x7f1838198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:05.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.827+0000 7f183dc8c640 1 -- 172.21.15.116:0/3782737763 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183819d5c0 con 0x7f1838102d40 2024-09-12T03:01:05.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.827+0000 7f183d48b640 1 --2- 172.21.15.116:0/3782737763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1838103660 0x7f1838198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:01:05.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.827+0000 7f183dc8c640 1 --2- 172.21.15.116:0/3782737763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1838102d40 0x7f1838198610 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1834005ab0 tx=0x7f18340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:05.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.827+0000 7f182effd640 1 -- 172.21.15.116:0/3782737763 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183403f030 con 0x7f1838102d40 2024-09-12T03:01:05.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.827+0000 7f182effd640 1 -- 172.21.15.116:0/3782737763 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1834003ce0 con 0x7f1838102d40 2024-09-12T03:01:05.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.827+0000 7f183fef0640 1 -- 172.21.15.116:0/3782737763 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f183819d850 con 0x7f1838102d40 2024-09-12T03:01:05.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.827+0000 7f182effd640 1 -- 172.21.15.116:0/3782737763 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834035070 con 0x7f1838102d40 2024-09-12T03:01:05.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.827+0000 7f183fef0640 1 -- 172.21.15.116:0/3782737763 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18381a5190 con 0x7f1838102d40 2024-09-12T03:01:05.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.828+0000 7f183fef0640 1 -- 172.21.15.116:0/3782737763 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f183806b070 con 0x7f1838102d40 2024-09-12T03:01:05.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.829+0000 7f182effd640 1 -- 172.21.15.116:0/3782737763 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1834042030 con 0x7f1838102d40 2024-09-12T03:01:05.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.830+0000 7f182effd640 1 --2- 172.21.15.116:0/3782737763 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f18240a2b80 0x7f18240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:05.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.830+0000 7f183d48b640 1 --2- 172.21.15.116:0/3782737763 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f18240a2b80 0x7f18240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:05.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.830+0000 7f182effd640 1 -- 172.21.15.116:0/3782737763 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f18340efae0 con 0x7f1838102d40 2024-09-12T03:01:05.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.830+0000 7f183d48b640 1 --2- 172.21.15.116:0/3782737763 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f18240a2b80 0x7f18240a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f18380fedc0 tx=0x7f1828008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:05.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:05.833+0000 7f182effd640 1 -- 172.21.15.116:0/3782737763 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f18340bcb90 con 0x7f1838102d40 2024-09-12T03:01:06.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:06.034+0000 7f183fef0640 1 -- 172.21.15.116:0/3782737763 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f1838107210 con 0x7f1838102d40 2024-09-12T03:01:06.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:06.373+0000 7f182effd640 1 -- 172.21.15.116:0/3782737763 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f18340bc490 con 0x7f1838102d40 2024-09-12T03:01:06.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:06.376+0000 7f183fef0640 1 -- 172.21.15.116:0/3782737763 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f18240a2b80 msgr2=0x7f18240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:06.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:06.376+0000 7f183fef0640 1 --2- 172.21.15.116:0/3782737763 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f18240a2b80 0x7f18240a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f18380fedc0 tx=0x7f1828008040 comp rx=0 tx=0).stop 2024-09-12T03:01:06.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:06.376+0000 7f183fef0640 1 -- 172.21.15.116:0/3782737763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1838102d40 msgr2=0x7f1838198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:06.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:06.376+0000 7f183fef0640 1 --2- 172.21.15.116:0/3782737763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1838102d40 0x7f1838198610 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1834005ab0 tx=0x7f18340042f0 comp rx=0 tx=0).stop 2024-09-12T03:01:06.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:06.376+0000 7f183fef0640 1 -- 172.21.15.116:0/3782737763 shutdown_connections 2024-09-12T03:01:06.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:06.376+0000 7f183fef0640 1 --2- 172.21.15.116:0/3782737763 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f18240a2b80 0x7f18240a5040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:06.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:06.376+0000 7f183fef0640 1 --2- 172.21.15.116:0/3782737763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f183810c170 0x7f183819cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:06.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:06.376+0000 7f183fef0640 1 --2- 172.21.15.116:0/3782737763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1838103660 0x7f1838198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:06.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:06.376+0000 7f183fef0640 1 --2- 172.21.15.116:0/3782737763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1838102d40 0x7f1838198610 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:06.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:06.376+0000 7f183fef0640 1 -- 172.21.15.116:0/3782737763 >> 172.21.15.116:0/3782737763 conn(0x7f18380fc7a0 msgr2=0x7f18380fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:06.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:06.376+0000 7f183fef0640 1 -- 172.21.15.116:0/3782737763 shutdown_connections 2024-09-12T03:01:06.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:06.376+0000 7f183fef0640 1 -- 172.21.15.116:0/3782737763 wait complete. 2024-09-12T03:01:06.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:06 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3782737763' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-12T03:01:06.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:06 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3782737763' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-12T03:01:06.764 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:06 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3782737763' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-12T03:01:07.102 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-12T03:01:07.348 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:07.621 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:07 smithi116 ceph-mon[31410]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T03:01:07.622 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:07 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3782737763' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-12T03:01:07.622 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:07 smithi116 ceph-mon[31410]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T03:01:07.622 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:07 smithi116 ceph-mon[31410]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:01:07.622 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:07 smithi116 ceph-mon[31410]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-12T03:01:07.622 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:07 smithi116 ceph-mon[31410]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-12T03:01:07.622 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:07 smithi116 ceph-mon[31410]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-12T03:01:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:07 smithi183 ceph-mon[35535]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T03:01:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:07 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3782737763' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-12T03:01:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:07 smithi183 ceph-mon[35535]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T03:01:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:07 smithi183 ceph-mon[35535]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:01:07.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:07 smithi183 ceph-mon[35535]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-12T03:01:07.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:07 smithi183 ceph-mon[35535]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-12T03:01:07.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:07 smithi183 ceph-mon[35535]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-12T03:01:07.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:07 smithi177 ceph-mon[35305]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T03:01:07.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:07 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3782737763' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-12T03:01:07.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:07 smithi177 ceph-mon[35305]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T03:01:07.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:07 smithi177 ceph-mon[35305]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:01:07.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:07 smithi177 ceph-mon[35305]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-12T03:01:07.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:07 smithi177 ceph-mon[35305]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-12T03:01:07.746 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:07 smithi177 ceph-mon[35305]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-12T03:01:07.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.848+0000 7f2a0b22f640 1 -- 172.21.15.116:0/540817754 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a04100a80 msgr2=0x7f2a04100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:07.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.848+0000 7f2a0b22f640 1 --2- 172.21.15.116:0/540817754 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a04100a80 0x7f2a04100ee0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f29f0009930 tx=0x7f29f002a420 comp rx=0 tx=0).stop 2024-09-12T03:01:07.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.848+0000 7f2a0b22f640 1 -- 172.21.15.116:0/540817754 shutdown_connections 2024-09-12T03:01:07.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.848+0000 7f2a0b22f640 1 --2- 172.21.15.116:0/540817754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a04101420 0x7f2a0410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:07.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.848+0000 7f2a0b22f640 1 --2- 172.21.15.116:0/540817754 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a04100a80 0x7f2a04100ee0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:07.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.848+0000 7f2a0b22f640 1 --2- 172.21.15.116:0/540817754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a04106a80 0x7f2a04106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:07.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.848+0000 7f2a0b22f640 1 -- 172.21.15.116:0/540817754 >> 172.21.15.116:0/540817754 conn(0x7f2a040fc7a0 msgr2=0x7f2a040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:07.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.848+0000 7f2a0b22f640 1 -- 172.21.15.116:0/540817754 shutdown_connections 2024-09-12T03:01:07.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.848+0000 7f2a0b22f640 1 -- 172.21.15.116:0/540817754 wait complete. 2024-09-12T03:01:07.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.849+0000 7f2a0b22f640 1 Processor -- start 2024-09-12T03:01:07.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.849+0000 7f2a0b22f640 1 -- start start 2024-09-12T03:01:07.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.849+0000 7f2a0b22f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a04100a80 0x7f2a04199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:07.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.849+0000 7f2a0b22f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a04101420 0x7f2a041999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:07.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.849+0000 7f2a0b22f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a04106a80 0x7f2a041a44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:07.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.849+0000 7f2a0b22f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2a04110f70 con 0x7f2a04106a80 2024-09-12T03:01:07.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.849+0000 7f2a0b22f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2a04110df0 con 0x7f2a04101420 2024-09-12T03:01:07.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.849+0000 7f2a0b22f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a041110f0 con 0x7f2a04100a80 2024-09-12T03:01:07.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.849+0000 7f2a097cc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a04106a80 0x7f2a041a44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:07.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.849+0000 7f2a097cc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a04106a80 0x7f2a041a44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57538/0 (socket says 172.21.15.116:57538) 2024-09-12T03:01:07.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.849+0000 7f2a097cc640 1 -- 172.21.15.116:0/4232103932 learned_addr learned my addr 172.21.15.116:0/4232103932 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:07.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.849+0000 7f2a097cc640 1 -- 172.21.15.116:0/4232103932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a04100a80 msgr2=0x7f2a04199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:07.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.849+0000 7f2a08fcb640 1 --2- 172.21.15.116:0/4232103932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a04100a80 0x7f2a04199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:07.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.849+0000 7f2a03fff640 1 --2- 172.21.15.116:0/4232103932 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a04101420 0x7f2a041999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:07.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.849+0000 7f2a097cc640 1 --2- 172.21.15.116:0/4232103932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a04100a80 0x7f2a04199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:07.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.850+0000 7f2a097cc640 1 -- 172.21.15.116:0/4232103932 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a04101420 msgr2=0x7f2a041999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:07.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.850+0000 7f2a097cc640 1 --2- 172.21.15.116:0/4232103932 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a04101420 0x7f2a041999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:07.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.850+0000 7f2a097cc640 1 -- 172.21.15.116:0/4232103932 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a041a4a20 con 0x7f2a04106a80 2024-09-12T03:01:07.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.850+0000 7f2a08fcb640 1 --2- 172.21.15.116:0/4232103932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a04100a80 0x7f2a04199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:01:07.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.850+0000 7f2a097cc640 1 --2- 172.21.15.116:0/4232103932 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a04106a80 0x7f2a041a44e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f29fc00b970 tx=0x7f29fc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:07.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.850+0000 7f2a01ffb640 1 -- 172.21.15.116:0/4232103932 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc019070 con 0x7f2a04106a80 2024-09-12T03:01:07.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.850+0000 7f2a0b22f640 1 -- 172.21.15.116:0/4232103932 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a041a4bf0 con 0x7f2a04106a80 2024-09-12T03:01:07.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.850+0000 7f2a01ffb640 1 -- 172.21.15.116:0/4232103932 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29fc00cb40 con 0x7f2a04106a80 2024-09-12T03:01:07.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.850+0000 7f2a01ffb640 1 -- 172.21.15.116:0/4232103932 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc007650 con 0x7f2a04106a80 2024-09-12T03:01:07.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.850+0000 7f2a0b22f640 1 -- 172.21.15.116:0/4232103932 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a041a5100 con 0x7f2a04106a80 2024-09-12T03:01:07.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.851+0000 7f2a0b22f640 1 -- 172.21.15.116:0/4232103932 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a04100f60 con 0x7f2a04106a80 2024-09-12T03:01:07.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.852+0000 7f2a01ffb640 1 -- 172.21.15.116:0/4232103932 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f29fc01c070 con 0x7f2a04106a80 2024-09-12T03:01:07.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.853+0000 7f2a01ffb640 1 --2- 172.21.15.116:0/4232103932 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f29f40a2b80 0x7f29f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:07.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.853+0000 7f2a08fcb640 1 --2- 172.21.15.116:0/4232103932 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f29f40a2b80 0x7f29f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:07.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.853+0000 7f2a01ffb640 1 -- 172.21.15.116:0/4232103932 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f29fc0ce580 con 0x7f2a04106a80 2024-09-12T03:01:07.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.853+0000 7f2a08fcb640 1 --2- 172.21.15.116:0/4232103932 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f29f40a2b80 0x7f29f40a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f29f8006fd0 tx=0x7f29f8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:07.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:07.857+0000 7f2a01ffb640 1 -- 172.21.15.116:0/4232103932 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f29fc09b630 con 0x7f2a04106a80 2024-09-12T03:01:08.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:08.059+0000 7f2a0b22f640 1 -- 172.21.15.116:0/4232103932 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f2a0402d7c0 con 0x7f2a04106a80 2024-09-12T03:01:08.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:08.388+0000 7f2a01ffb640 1 -- 172.21.15.116:0/4232103932 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f29fc09af30 con 0x7f2a04106a80 2024-09-12T03:01:08.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:08.390+0000 7f2a0b22f640 1 -- 172.21.15.116:0/4232103932 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f29f40a2b80 msgr2=0x7f29f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:08.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:08.390+0000 7f2a0b22f640 1 --2- 172.21.15.116:0/4232103932 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f29f40a2b80 0x7f29f40a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f29f8006fd0 tx=0x7f29f8008040 comp rx=0 tx=0).stop 2024-09-12T03:01:08.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:08.390+0000 7f2a0b22f640 1 -- 172.21.15.116:0/4232103932 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a04106a80 msgr2=0x7f2a041a44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:08.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:08.390+0000 7f2a0b22f640 1 --2- 172.21.15.116:0/4232103932 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a04106a80 0x7f2a041a44e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f29fc00b970 tx=0x7f29fc00be30 comp rx=0 tx=0).stop 2024-09-12T03:01:08.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:08.390+0000 7f2a0b22f640 1 -- 172.21.15.116:0/4232103932 shutdown_connections 2024-09-12T03:01:08.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:08.390+0000 7f2a0b22f640 1 --2- 172.21.15.116:0/4232103932 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f29f40a2b80 0x7f29f40a5040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:08.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:08.390+0000 7f2a0b22f640 1 --2- 172.21.15.116:0/4232103932 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a04106a80 0x7f2a041a44e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:08.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:08.390+0000 7f2a0b22f640 1 --2- 172.21.15.116:0/4232103932 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a04101420 0x7f2a041999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:08.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:08.390+0000 7f2a0b22f640 1 --2- 172.21.15.116:0/4232103932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a04100a80 0x7f2a04199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:08.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:08.390+0000 7f2a0b22f640 1 -- 172.21.15.116:0/4232103932 >> 172.21.15.116:0/4232103932 conn(0x7f2a040fc7a0 msgr2=0x7f2a040fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:08.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:08.390+0000 7f2a0b22f640 1 -- 172.21.15.116:0/4232103932 shutdown_connections 2024-09-12T03:01:08.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:08.390+0000 7f2a0b22f640 1 -- 172.21.15.116:0/4232103932 wait complete. 2024-09-12T03:01:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:08 smithi183 ceph-mon[35535]: mds.? [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] up:active 2024-09-12T03:01:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:08 smithi183 ceph-mon[35535]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T03:01:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:08 smithi183 ceph-mon[35535]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T03:01:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:08 smithi183 ceph-mon[35535]: Cluster is now healthy 2024-09-12T03:01:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:08 smithi183 ceph-mon[35535]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-12T03:01:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:08 smithi183 ceph-mon[35535]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-12T03:01:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:08 smithi183 ceph-mon[35535]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-12T03:01:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:08 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/4232103932' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T03:01:08.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:08 smithi177 ceph-mon[35305]: mds.? [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] up:active 2024-09-12T03:01:08.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:08 smithi177 ceph-mon[35305]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T03:01:08.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:08 smithi177 ceph-mon[35305]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T03:01:08.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:08 smithi177 ceph-mon[35305]: Cluster is now healthy 2024-09-12T03:01:08.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:08 smithi177 ceph-mon[35305]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-12T03:01:08.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:08 smithi177 ceph-mon[35305]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-12T03:01:08.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:08 smithi177 ceph-mon[35305]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-12T03:01:08.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:08 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/4232103932' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T03:01:08.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:08 smithi116 ceph-mon[31410]: mds.? [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] up:active 2024-09-12T03:01:08.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:08 smithi116 ceph-mon[31410]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T03:01:08.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:08 smithi116 ceph-mon[31410]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T03:01:08.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:08 smithi116 ceph-mon[31410]: Cluster is now healthy 2024-09-12T03:01:08.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:08 smithi116 ceph-mon[31410]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-12T03:01:08.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:08 smithi116 ceph-mon[31410]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-12T03:01:08.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:08 smithi116 ceph-mon[31410]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-12T03:01:08.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:08 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/4232103932' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T03:01:09.250 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-12T03:01:09.495 INFO:journalctl@ceph.mds.b.smithi177.stdout:Sep 12 03:01:09 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-b[65339]: 2024-09-12T03:01:09.211+0000 7f60a7b52640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-12T03:01:09.495 INFO:journalctl@ceph.mds.b.smithi177.stdout:Sep 12 03:01:09 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-b[65339]: 2024-09-12T03:01:09.211+0000 7f60a7b52640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-12T03:01:09.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:09 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/4232103932' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T03:01:09.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:09 smithi177 ceph-mon[35305]: mds.? [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] up:active 2024-09-12T03:01:09.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:09 smithi177 ceph-mon[35305]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-12T03:01:09.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:09 smithi177 ceph-mon[35305]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:01:09.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:09 smithi177 ceph-mon[35305]: pgmap v281: 97 pgs: 97 active+clean; 580 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 6 op/s 2024-09-12T03:01:09.507 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:09.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:09 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/4232103932' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T03:01:09.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:09 smithi116 ceph-mon[31410]: mds.? [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] up:active 2024-09-12T03:01:09.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:09 smithi116 ceph-mon[31410]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-12T03:01:09.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:09 smithi116 ceph-mon[31410]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:01:09.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:09 smithi116 ceph-mon[31410]: pgmap v281: 97 pgs: 97 active+clean; 580 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 6 op/s 2024-09-12T03:01:09.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:09 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/4232103932' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T03:01:09.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:09 smithi183 ceph-mon[35535]: mds.? [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] up:active 2024-09-12T03:01:09.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:09 smithi183 ceph-mon[35535]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-12T03:01:09.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:09 smithi183 ceph-mon[35535]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:01:09.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:09 smithi183 ceph-mon[35535]: pgmap v281: 97 pgs: 97 active+clean; 580 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 6 op/s 2024-09-12T03:01:10.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.007+0000 7efedd675640 1 -- 172.21.15.116:0/3167146838 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efed81037a0 msgr2=0x7efed810bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:10.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.007+0000 7efedd675640 1 --2- 172.21.15.116:0/3167146838 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efed81037a0 0x7efed810bc00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efecc009900 tx=0x7efecc02a410 comp rx=0 tx=0).stop 2024-09-12T03:01:10.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.007+0000 7efedd675640 1 -- 172.21.15.116:0/3167146838 shutdown_connections 2024-09-12T03:01:10.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.007+0000 7efedd675640 1 --2- 172.21.15.116:0/3167146838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efed810c2f0 0x7efed810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.007+0000 7efedd675640 1 --2- 172.21.15.116:0/3167146838 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efed81037a0 0x7efed810bc00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.007+0000 7efedd675640 1 --2- 172.21.15.116:0/3167146838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efed8102df0 0x7efed81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.007+0000 7efedd675640 1 -- 172.21.15.116:0/3167146838 >> 172.21.15.116:0/3167146838 conn(0x7efed80fc7d0 msgr2=0x7efed80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:10.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.007+0000 7efedd675640 1 -- 172.21.15.116:0/3167146838 shutdown_connections 2024-09-12T03:01:10.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.007+0000 7efedd675640 1 -- 172.21.15.116:0/3167146838 wait complete. 2024-09-12T03:01:10.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.008+0000 7efedd675640 1 Processor -- start 2024-09-12T03:01:10.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.008+0000 7efedd675640 1 -- start start 2024-09-12T03:01:10.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.009+0000 7efedd675640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efed8102df0 0x7efed81987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:10.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.009+0000 7efedd675640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efed81037a0 0x7efed8198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:10.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.009+0000 7efedd675640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efed810c2f0 0x7efed819c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:10.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.009+0000 7efedd675640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efed81121b0 con 0x7efed8102df0 2024-09-12T03:01:10.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.009+0000 7efedd675640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efed8112030 con 0x7efed810c2f0 2024-09-12T03:01:10.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.009+0000 7efedd675640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efed8112330 con 0x7efed81037a0 2024-09-12T03:01:10.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.009+0000 7efed6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efed8102df0 0x7efed81987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:10.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.009+0000 7efed6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efed8102df0 0x7efed81987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57546/0 (socket says 172.21.15.116:57546) 2024-09-12T03:01:10.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.009+0000 7efed6ffd640 1 -- 172.21.15.116:0/2026256951 learned_addr learned my addr 172.21.15.116:0/2026256951 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:10.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.009+0000 7efed77fe640 1 --2- 172.21.15.116:0/2026256951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efed810c2f0 0x7efed819c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:10.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.009+0000 7efed67fc640 1 --2- 172.21.15.116:0/2026256951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efed81037a0 0x7efed8198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:10.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.009+0000 7efed6ffd640 1 -- 172.21.15.116:0/2026256951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efed81037a0 msgr2=0x7efed8198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:10.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.009+0000 7efed6ffd640 1 --2- 172.21.15.116:0/2026256951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efed81037a0 0x7efed8198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.009+0000 7efed6ffd640 1 -- 172.21.15.116:0/2026256951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efed810c2f0 msgr2=0x7efed819c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:10.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.009+0000 7efed6ffd640 1 --2- 172.21.15.116:0/2026256951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efed810c2f0 0x7efed819c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.009+0000 7efed6ffd640 1 -- 172.21.15.116:0/2026256951 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed819c740 con 0x7efed8102df0 2024-09-12T03:01:10.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.009+0000 7efed67fc640 1 --2- 172.21.15.116:0/2026256951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efed81037a0 0x7efed8198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:01:10.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.009+0000 7efed6ffd640 1 --2- 172.21.15.116:0/2026256951 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efed8102df0 0x7efed81987f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efec800ba50 tx=0x7efec800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:10.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.009+0000 7efebffff640 1 -- 172.21.15.116:0/2026256951 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec8019070 con 0x7efed8102df0 2024-09-12T03:01:10.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.009+0000 7efebffff640 1 -- 172.21.15.116:0/2026256951 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efec800cbe0 con 0x7efed8102df0 2024-09-12T03:01:10.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.009+0000 7efedd675640 1 -- 172.21.15.116:0/2026256951 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed819ca30 con 0x7efed8102df0 2024-09-12T03:01:10.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.009+0000 7efebffff640 1 -- 172.21.15.116:0/2026256951 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec80076e0 con 0x7efed8102df0 2024-09-12T03:01:10.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.009+0000 7efedd675640 1 -- 172.21.15.116:0/2026256951 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed81a53a0 con 0x7efed8102df0 2024-09-12T03:01:10.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.010+0000 7efedd675640 1 -- 172.21.15.116:0/2026256951 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efed806a790 con 0x7efed8102df0 2024-09-12T03:01:10.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.012+0000 7efebffff640 1 -- 172.21.15.116:0/2026256951 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7efec801c070 con 0x7efed8102df0 2024-09-12T03:01:10.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.012+0000 7efebffff640 1 --2- 172.21.15.116:0/2026256951 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7efec00a2b80 0x7efec00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:10.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.012+0000 7efed67fc640 1 --2- 172.21.15.116:0/2026256951 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7efec00a2b80 0x7efec00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:10.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.012+0000 7efebffff640 1 -- 172.21.15.116:0/2026256951 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7efec80cea30 con 0x7efed8102df0 2024-09-12T03:01:10.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.013+0000 7efed67fc640 1 --2- 172.21.15.116:0/2026256951 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7efec00a2b80 0x7efec00a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efed810d7a0 tx=0x7efecc032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:10.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.015+0000 7efebffff640 1 -- 172.21.15.116:0/2026256951 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efec809bae0 con 0x7efed8102df0 2024-09-12T03:01:10.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.216+0000 7efedd675640 1 -- 172.21.15.116:0/2026256951 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7efed8199b80 con 0x7efed8102df0 2024-09-12T03:01:10.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.400+0000 7efebffff640 1 -- 172.21.15.116:0/2026256951 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7efed8199b80 con 0x7efed8102df0 2024-09-12T03:01:10.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.402+0000 7efedd675640 1 -- 172.21.15.116:0/2026256951 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7efec00a2b80 msgr2=0x7efec00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:10.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.402+0000 7efedd675640 1 --2- 172.21.15.116:0/2026256951 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7efec00a2b80 0x7efec00a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efed810d7a0 tx=0x7efecc032d50 comp rx=0 tx=0).stop 2024-09-12T03:01:10.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.402+0000 7efedd675640 1 -- 172.21.15.116:0/2026256951 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efed8102df0 msgr2=0x7efed81987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:10.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.402+0000 7efedd675640 1 --2- 172.21.15.116:0/2026256951 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efed8102df0 0x7efed81987f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efec800ba50 tx=0x7efec800bf10 comp rx=0 tx=0).stop 2024-09-12T03:01:10.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.403+0000 7efedd675640 1 -- 172.21.15.116:0/2026256951 shutdown_connections 2024-09-12T03:01:10.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.403+0000 7efedd675640 1 --2- 172.21.15.116:0/2026256951 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7efec00a2b80 0x7efec00a5040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.403+0000 7efedd675640 1 --2- 172.21.15.116:0/2026256951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efed810c2f0 0x7efed819c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.403+0000 7efedd675640 1 --2- 172.21.15.116:0/2026256951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efed81037a0 0x7efed8198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.403+0000 7efedd675640 1 --2- 172.21.15.116:0/2026256951 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efed8102df0 0x7efed81987f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.403+0000 7efedd675640 1 -- 172.21.15.116:0/2026256951 >> 172.21.15.116:0/2026256951 conn(0x7efed80fc7d0 msgr2=0x7efed80fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:10.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.403+0000 7efedd675640 1 -- 172.21.15.116:0/2026256951 shutdown_connections 2024-09-12T03:01:10.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:10.403+0000 7efedd675640 1 -- 172.21.15.116:0/2026256951 wait complete. 2024-09-12T03:01:10.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:10 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2026256951' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T03:01:10.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:10 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2026256951' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T03:01:10.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:10 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2026256951' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T03:01:11.161 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-09-12T03:01:11.161 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-12T03:01:11.410 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:11.679 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:11 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2026256951' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T03:01:11.680 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:11 smithi116 ceph-mon[31410]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:01:11.680 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:11 smithi116 ceph-mon[31410]: pgmap v282: 97 pgs: 97 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 3.1 KiB/s wr, 9 op/s 2024-09-12T03:01:11.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:11 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2026256951' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T03:01:11.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:11 smithi183 ceph-mon[35535]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:01:11.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:11 smithi183 ceph-mon[35535]: pgmap v282: 97 pgs: 97 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 3.1 KiB/s wr, 9 op/s 2024-09-12T03:01:11.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:11 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2026256951' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T03:01:11.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:11 smithi177 ceph-mon[35305]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:01:11.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:11 smithi177 ceph-mon[35305]: pgmap v282: 97 pgs: 97 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 3.1 KiB/s wr, 9 op/s 2024-09-12T03:01:11.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.890+0000 7f1f599c4640 1 -- 172.21.15.116:0/433932413 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f54103740 msgr2=0x7f1f54107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:11.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.890+0000 7f1f599c4640 1 --2- 172.21.15.116:0/433932413 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f54103740 0x7f1f54107b80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1f48009930 tx=0x7f1f4802a420 comp rx=0 tx=0).stop 2024-09-12T03:01:11.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.890+0000 7f1f599c4640 1 -- 172.21.15.116:0/433932413 shutdown_connections 2024-09-12T03:01:11.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.890+0000 7f1f599c4640 1 --2- 172.21.15.116:0/433932413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f54108290 0x7f1f5410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.890+0000 7f1f599c4640 1 --2- 172.21.15.116:0/433932413 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f54103740 0x7f1f54107b80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.890+0000 7f1f599c4640 1 --2- 172.21.15.116:0/433932413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f54102d90 0x7f1f54103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.890+0000 7f1f599c4640 1 -- 172.21.15.116:0/433932413 >> 172.21.15.116:0/433932413 conn(0x7f1f540fc7d0 msgr2=0x7f1f540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:11.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.890+0000 7f1f599c4640 1 -- 172.21.15.116:0/433932413 shutdown_connections 2024-09-12T03:01:11.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.890+0000 7f1f599c4640 1 -- 172.21.15.116:0/433932413 wait complete. 2024-09-12T03:01:11.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.891+0000 7f1f599c4640 1 Processor -- start 2024-09-12T03:01:11.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.891+0000 7f1f599c4640 1 -- start start 2024-09-12T03:01:11.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.891+0000 7f1f599c4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f54102d90 0x7f1f5419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:11.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.891+0000 7f1f599c4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f54103740 0x7f1f5419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:11.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.892+0000 7f1f599c4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f54108290 0x7f1f541985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:11.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.892+0000 7f1f599c4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1f54112130 con 0x7f1f54108290 2024-09-12T03:01:11.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.892+0000 7f1f599c4640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1f54111fb0 con 0x7f1f54103740 2024-09-12T03:01:11.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.892+0000 7f1f599c4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1f541122b0 con 0x7f1f54102d90 2024-09-12T03:01:11.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.892+0000 7f1f537fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f54108290 0x7f1f541985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:11.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.892+0000 7f1f537fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f54108290 0x7f1f541985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54912/0 (socket says 172.21.15.116:54912) 2024-09-12T03:01:11.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.892+0000 7f1f537fe640 1 -- 172.21.15.116:0/4252292305 learned_addr learned my addr 172.21.15.116:0/4252292305 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:11.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.892+0000 7f1f52ffd640 1 --2- 172.21.15.116:0/4252292305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f54102d90 0x7f1f5419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:11.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.892+0000 7f1f527fc640 1 --2- 172.21.15.116:0/4252292305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f54103740 0x7f1f5419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:11.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.892+0000 7f1f537fe640 1 -- 172.21.15.116:0/4252292305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f54102d90 msgr2=0x7f1f5419e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:11.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.892+0000 7f1f537fe640 1 --2- 172.21.15.116:0/4252292305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f54102d90 0x7f1f5419e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.892+0000 7f1f537fe640 1 -- 172.21.15.116:0/4252292305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f54103740 msgr2=0x7f1f5419e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:11.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.892+0000 7f1f537fe640 1 --2- 172.21.15.116:0/4252292305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f54103740 0x7f1f5419e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.892+0000 7f1f537fe640 1 -- 172.21.15.116:0/4252292305 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f54198ce0 con 0x7f1f54108290 2024-09-12T03:01:11.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.892+0000 7f1f527fc640 1 --2- 172.21.15.116:0/4252292305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f54103740 0x7f1f5419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:01:11.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.892+0000 7f1f537fe640 1 --2- 172.21.15.116:0/4252292305 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f54108290 0x7f1f541985b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c009fd0 tx=0x7f1f4c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:11.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.892+0000 7f1f589c2640 1 -- 172.21.15.116:0/4252292305 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c011830 con 0x7f1f54108290 2024-09-12T03:01:11.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.892+0000 7f1f589c2640 1 -- 172.21.15.116:0/4252292305 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f4c016070 con 0x7f1f54108290 2024-09-12T03:01:11.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.892+0000 7f1f599c4640 1 -- 172.21.15.116:0/4252292305 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f54198fd0 con 0x7f1f54108290 2024-09-12T03:01:11.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.892+0000 7f1f589c2640 1 -- 172.21.15.116:0/4252292305 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c00f2e0 con 0x7f1f54108290 2024-09-12T03:01:11.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.892+0000 7f1f599c4640 1 -- 172.21.15.116:0/4252292305 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f541a52c0 con 0x7f1f54108290 2024-09-12T03:01:11.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.894+0000 7f1f599c4640 1 -- 172.21.15.116:0/4252292305 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f5406a7f0 con 0x7f1f54108290 2024-09-12T03:01:11.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.894+0000 7f1f589c2640 1 -- 172.21.15.116:0/4252292305 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1f4c0119d0 con 0x7f1f54108290 2024-09-12T03:01:11.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.895+0000 7f1f589c2640 1 --2- 172.21.15.116:0/4252292305 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1f3c0a2b80 0x7f1f3c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:11.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.895+0000 7f1f52ffd640 1 --2- 172.21.15.116:0/4252292305 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1f3c0a2b80 0x7f1f3c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:11.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.896+0000 7f1f589c2640 1 -- 172.21.15.116:0/4252292305 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f1f4c0ce580 con 0x7f1f54108290 2024-09-12T03:01:11.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.896+0000 7f1f52ffd640 1 --2- 172.21.15.116:0/4252292305 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1f3c0a2b80 0x7f1f3c0a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400a400 tx=0x7f1f4400d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:11.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:11.900+0000 7f1f589c2640 1 -- 172.21.15.116:0/4252292305 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1f4c09b630 con 0x7f1f54108290 2024-09-12T03:01:12.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:12.088+0000 7f1f599c4640 1 -- 172.21.15.116:0/4252292305 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f1f54062630 con 0x7f1f3c0a2b80 2024-09-12T03:01:12.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:12.109+0000 7f1f589c2640 1 -- 172.21.15.116:0/4252292305 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1f54062630 con 0x7f1f3c0a2b80 2024-09-12T03:01:12.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:12.111+0000 7f1f599c4640 1 -- 172.21.15.116:0/4252292305 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1f3c0a2b80 msgr2=0x7f1f3c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:12.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:12.111+0000 7f1f599c4640 1 --2- 172.21.15.116:0/4252292305 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1f3c0a2b80 0x7f1f3c0a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400a400 tx=0x7f1f4400d040 comp rx=0 tx=0).stop 2024-09-12T03:01:12.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:12.111+0000 7f1f599c4640 1 -- 172.21.15.116:0/4252292305 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f54108290 msgr2=0x7f1f541985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:12.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:12.111+0000 7f1f599c4640 1 --2- 172.21.15.116:0/4252292305 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f54108290 0x7f1f541985b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c009fd0 tx=0x7f1f4c00b8e0 comp rx=0 tx=0).stop 2024-09-12T03:01:12.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:12.111+0000 7f1f599c4640 1 -- 172.21.15.116:0/4252292305 shutdown_connections 2024-09-12T03:01:12.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:12.111+0000 7f1f599c4640 1 --2- 172.21.15.116:0/4252292305 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1f3c0a2b80 0x7f1f3c0a5040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:12.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:12.111+0000 7f1f599c4640 1 --2- 172.21.15.116:0/4252292305 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f54108290 0x7f1f541985b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:12.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:12.111+0000 7f1f599c4640 1 --2- 172.21.15.116:0/4252292305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f54103740 0x7f1f5419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:12.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:12.111+0000 7f1f599c4640 1 --2- 172.21.15.116:0/4252292305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f54102d90 0x7f1f5419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:12.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:12.111+0000 7f1f599c4640 1 -- 172.21.15.116:0/4252292305 >> 172.21.15.116:0/4252292305 conn(0x7f1f540fc7d0 msgr2=0x7f1f54105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:12.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:12.112+0000 7f1f599c4640 1 -- 172.21.15.116:0/4252292305 shutdown_connections 2024-09-12T03:01:12.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:12.112+0000 7f1f599c4640 1 -- 172.21.15.116:0/4252292305 wait complete. 2024-09-12T03:01:12.410 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 03:01:12 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T03:01:12.093+0000 7fc2f0aca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:01:12.410 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 03:01:12 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T03:01:12.093+0000 7fc2f0aca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:01:12.410 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 03:01:12 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T03:01:12.093+0000 7fc2f0aca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:01:12.410 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 03:01:12 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T03:01:12.093+0000 7fc2f0aca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:01:12.410 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 03:01:12 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T03:01:12.093+0000 7fc2f0aca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:01:12.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:12 smithi183 ceph-mon[35535]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:01:12.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:12 smithi177 ceph-mon[35305]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:01:12.780 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:12 smithi116 ceph-mon[31410]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:01:12.807 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-12T03:01:13.052 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:13.434 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:13 smithi116 ceph-mon[31410]: pgmap v283: 97 pgs: 97 active+clean; 582 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-09-12T03:01:13.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.518+0000 7fe5cc590640 1 -- 172.21.15.116:0/1755052731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5c4101de0 msgr2=0x7fe5c41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:13.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.518+0000 7fe5cc590640 1 --2- 172.21.15.116:0/1755052731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5c4101de0 0x7fe5c41021c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009900 tx=0x7fe5c002a410 comp rx=0 tx=0).stop 2024-09-12T03:01:13.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.518+0000 7fe5cc590640 1 -- 172.21.15.116:0/1755052731 shutdown_connections 2024-09-12T03:01:13.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.518+0000 7fe5cc590640 1 --2- 172.21.15.116:0/1755052731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5c4106f10 0x7fe5c410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:13.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.518+0000 7fe5cc590640 1 --2- 172.21.15.116:0/1755052731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5c4102790 0x7fe5c41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:13.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.518+0000 7fe5cc590640 1 --2- 172.21.15.116:0/1755052731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5c4101de0 0x7fe5c41021c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:13.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.518+0000 7fe5cc590640 1 -- 172.21.15.116:0/1755052731 >> 172.21.15.116:0/1755052731 conn(0x7fe5c40fd8c0 msgr2=0x7fe5c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:13.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.518+0000 7fe5cc590640 1 -- 172.21.15.116:0/1755052731 shutdown_connections 2024-09-12T03:01:13.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.518+0000 7fe5cc590640 1 -- 172.21.15.116:0/1755052731 wait complete. 2024-09-12T03:01:13.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.518+0000 7fe5cc590640 1 Processor -- start 2024-09-12T03:01:13.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.518+0000 7fe5cc590640 1 -- start start 2024-09-12T03:01:13.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.518+0000 7fe5cc590640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5c4101de0 0x7fe5c4198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:13.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.519+0000 7fe5cc590640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5c4102790 0x7fe5c4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:13.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.519+0000 7fe5cc590640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5c4106f10 0x7fe5c419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:13.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.519+0000 7fe5cc590640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c4111fb0 con 0x7fe5c4106f10 2024-09-12T03:01:13.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.519+0000 7fe5cc590640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c4111e30 con 0x7fe5c4102790 2024-09-12T03:01:13.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.519+0000 7fe5cc590640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c4112130 con 0x7fe5c4101de0 2024-09-12T03:01:13.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.519+0000 7fe5cab2d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5c4106f10 0x7fe5c419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:13.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.519+0000 7fe5cab2d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5c4106f10 0x7fe5c419cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54938/0 (socket says 172.21.15.116:54938) 2024-09-12T03:01:13.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.519+0000 7fe5cab2d640 1 -- 172.21.15.116:0/3169857844 learned_addr learned my addr 172.21.15.116:0/3169857844 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:13.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.519+0000 7fe5c9b2b640 1 --2- 172.21.15.116:0/3169857844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5c4102790 0x7fe5c4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:13.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.519+0000 7fe5cab2d640 1 -- 172.21.15.116:0/3169857844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5c4101de0 msgr2=0x7fe5c4198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:13.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.519+0000 7fe5cab2d640 1 --2- 172.21.15.116:0/3169857844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5c4101de0 0x7fe5c4198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:13.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.519+0000 7fe5cab2d640 1 -- 172.21.15.116:0/3169857844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5c4102790 msgr2=0x7fe5c4198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:13.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.519+0000 7fe5cab2d640 1 --2- 172.21.15.116:0/3169857844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5c4102790 0x7fe5c4198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:13.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.519+0000 7fe5cab2d640 1 -- 172.21.15.116:0/3169857844 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c419d4c0 con 0x7fe5c4106f10 2024-09-12T03:01:13.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.519+0000 7fe5c9b2b640 1 --2- 172.21.15.116:0/3169857844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5c4102790 0x7fe5c4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:01:13.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.519+0000 7fe5cab2d640 1 --2- 172.21.15.116:0/3169857844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5c4106f10 0x7fe5c419cde0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc00eaa0 tx=0x7fe5bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:13.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.519+0000 7fe5bb7fe640 1 -- 172.21.15.116:0/3169857844 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc00cdb0 con 0x7fe5c4106f10 2024-09-12T03:01:13.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.519+0000 7fe5cc590640 1 -- 172.21.15.116:0/3169857844 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c419d7b0 con 0x7fe5c4106f10 2024-09-12T03:01:13.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.519+0000 7fe5bb7fe640 1 -- 172.21.15.116:0/3169857844 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5bc005020 con 0x7fe5c4106f10 2024-09-12T03:01:13.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.519+0000 7fe5bb7fe640 1 -- 172.21.15.116:0/3169857844 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc0116d0 con 0x7fe5c4106f10 2024-09-12T03:01:13.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.519+0000 7fe5cc590640 1 -- 172.21.15.116:0/3169857844 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c41a5090 con 0x7fe5c4106f10 2024-09-12T03:01:13.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.520+0000 7fe5cc590640 1 -- 172.21.15.116:0/3169857844 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5c41021c0 con 0x7fe5c4106f10 2024-09-12T03:01:13.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.521+0000 7fe5bb7fe640 1 -- 172.21.15.116:0/3169857844 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe5bc0054d0 con 0x7fe5c4106f10 2024-09-12T03:01:13.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.522+0000 7fe5bb7fe640 1 --2- 172.21.15.116:0/3169857844 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe5b00a2c10 0x7fe5b00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:13.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.522+0000 7fe5ca32c640 1 --2- 172.21.15.116:0/3169857844 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe5b00a2c10 0x7fe5b00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:13.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.522+0000 7fe5bb7fe640 1 -- 172.21.15.116:0/3169857844 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fe5bc0cf2f0 con 0x7fe5c4106f10 2024-09-12T03:01:13.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.523+0000 7fe5ca32c640 1 --2- 172.21.15.116:0/3169857844 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe5b00a2c10 0x7fe5b00a50d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe5c002a930 tx=0x7fe5c0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:13.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.526+0000 7fe5bb7fe640 1 -- 172.21.15.116:0/3169857844 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe5bc016030 con 0x7fe5c4106f10 2024-09-12T03:01:13.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:13 smithi183 ceph-mon[35535]: pgmap v283: 97 pgs: 97 active+clean; 582 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-09-12T03:01:13.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.719+0000 7fe5cc590640 1 -- 172.21.15.116:0/3169857844 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe5c4062630 con 0x7fe5b00a2c10 2024-09-12T03:01:13.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.742+0000 7fe5bb7fe640 1 -- 172.21.15.116:0/3169857844 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe5c4062630 con 0x7fe5b00a2c10 2024-09-12T03:01:13.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:13 smithi177 ceph-mon[35305]: pgmap v283: 97 pgs: 97 active+clean; 582 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-09-12T03:01:13.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.744+0000 7fe5cc590640 1 -- 172.21.15.116:0/3169857844 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe5b00a2c10 msgr2=0x7fe5b00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:13.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.744+0000 7fe5cc590640 1 --2- 172.21.15.116:0/3169857844 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe5b00a2c10 0x7fe5b00a50d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe5c002a930 tx=0x7fe5c0000bc0 comp rx=0 tx=0).stop 2024-09-12T03:01:13.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.744+0000 7fe5cc590640 1 -- 172.21.15.116:0/3169857844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5c4106f10 msgr2=0x7fe5c419cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:13.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.744+0000 7fe5cc590640 1 --2- 172.21.15.116:0/3169857844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5c4106f10 0x7fe5c419cde0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc00eaa0 tx=0x7fe5bc00ef60 comp rx=0 tx=0).stop 2024-09-12T03:01:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.744+0000 7fe5cc590640 1 -- 172.21.15.116:0/3169857844 shutdown_connections 2024-09-12T03:01:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.744+0000 7fe5cc590640 1 --2- 172.21.15.116:0/3169857844 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe5b00a2c10 0x7fe5b00a50d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.744+0000 7fe5cc590640 1 --2- 172.21.15.116:0/3169857844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5c4106f10 0x7fe5c419cde0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.745+0000 7fe5cc590640 1 --2- 172.21.15.116:0/3169857844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5c4102790 0x7fe5c4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.745+0000 7fe5cc590640 1 --2- 172.21.15.116:0/3169857844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5c4101de0 0x7fe5c4198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.745+0000 7fe5cc590640 1 -- 172.21.15.116:0/3169857844 >> 172.21.15.116:0/3169857844 conn(0x7fe5c40fd8c0 msgr2=0x7fe5c40febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.745+0000 7fe5cc590640 1 -- 172.21.15.116:0/3169857844 shutdown_connections 2024-09-12T03:01:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:13.745+0000 7fe5cc590640 1 -- 172.21.15.116:0/3169857844 wait complete. 2024-09-12T03:01:14.494 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-12T03:01:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:14 smithi183 ceph-mon[35535]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T03:01:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:14 smithi183 ceph-mon[35535]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:01:14.741 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:14.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:14 smithi177 ceph-mon[35305]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T03:01:14.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:14 smithi177 ceph-mon[35305]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:01:14.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:14 smithi116 ceph-mon[31410]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T03:01:14.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:14 smithi116 ceph-mon[31410]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:01:15.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.247+0000 7f910ae47640 1 -- 172.21.15.116:0/1327193477 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9104100a80 msgr2=0x7f9104100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:15.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.247+0000 7f910ae47640 1 --2- 172.21.15.116:0/1327193477 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9104100a80 0x7f9104100ee0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f90f0009930 tx=0x7f90f002a420 comp rx=0 tx=0).stop 2024-09-12T03:01:15.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.247+0000 7f910ae47640 1 -- 172.21.15.116:0/1327193477 shutdown_connections 2024-09-12T03:01:15.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.247+0000 7f910ae47640 1 --2- 172.21.15.116:0/1327193477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9104101420 0x7f910410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.247+0000 7f910ae47640 1 --2- 172.21.15.116:0/1327193477 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9104100a80 0x7f9104100ee0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.247+0000 7f910ae47640 1 --2- 172.21.15.116:0/1327193477 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9104106a80 0x7f9104106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.247+0000 7f910ae47640 1 -- 172.21.15.116:0/1327193477 >> 172.21.15.116:0/1327193477 conn(0x7f91040fc7a0 msgr2=0x7f91040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:15.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.247+0000 7f910ae47640 1 -- 172.21.15.116:0/1327193477 shutdown_connections 2024-09-12T03:01:15.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.247+0000 7f910ae47640 1 -- 172.21.15.116:0/1327193477 wait complete. 2024-09-12T03:01:15.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.247+0000 7f910ae47640 1 Processor -- start 2024-09-12T03:01:15.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.247+0000 7f910ae47640 1 -- start start 2024-09-12T03:01:15.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.248+0000 7f910ae47640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9104100a80 0x7f910419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:15.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.248+0000 7f910ae47640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9104101420 0x7f910419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:15.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.248+0000 7f910ae47640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9104106a80 0x7f910419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:15.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.248+0000 7f910ae47640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9104114130 con 0x7f9104100a80 2024-09-12T03:01:15.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.248+0000 7f910ae47640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9104113fb0 con 0x7f9104106a80 2024-09-12T03:01:15.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.248+0000 7f910ae47640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f91041142b0 con 0x7f9104101420 2024-09-12T03:01:15.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.248+0000 7f9108be3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9104100a80 0x7f910419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:15.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.248+0000 7f9108be3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9104100a80 0x7f910419a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54966/0 (socket says 172.21.15.116:54966) 2024-09-12T03:01:15.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.248+0000 7f9108be3640 1 -- 172.21.15.116:0/2685092532 learned_addr learned my addr 172.21.15.116:0/2685092532 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:15.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.248+0000 7f91093e4640 1 --2- 172.21.15.116:0/2685092532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9104106a80 0x7f910419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:15.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.248+0000 7f9103fff640 1 --2- 172.21.15.116:0/2685092532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9104101420 0x7f910419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:15.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.248+0000 7f9108be3640 1 -- 172.21.15.116:0/2685092532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9104101420 msgr2=0x7f910419ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:15.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.248+0000 7f9108be3640 1 --2- 172.21.15.116:0/2685092532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9104101420 0x7f910419ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.248+0000 7f9108be3640 1 -- 172.21.15.116:0/2685092532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9104106a80 msgr2=0x7f910419eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:15.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.248+0000 7f9108be3640 1 --2- 172.21.15.116:0/2685092532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9104106a80 0x7f910419eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.248+0000 7f9108be3640 1 -- 172.21.15.116:0/2685092532 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f910419f5c0 con 0x7f9104100a80 2024-09-12T03:01:15.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.248+0000 7f9108be3640 1 --2- 172.21.15.116:0/2685092532 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9104100a80 0x7f910419a610 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f90f800e9f0 tx=0x7f90f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:15.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.248+0000 7f9101ffb640 1 -- 172.21.15.116:0/2685092532 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f800cd80 con 0x7f9104100a80 2024-09-12T03:01:15.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.248+0000 7f910ae47640 1 -- 172.21.15.116:0/2685092532 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f910419f8b0 con 0x7f9104100a80 2024-09-12T03:01:15.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.248+0000 7f9101ffb640 1 -- 172.21.15.116:0/2685092532 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90f8010910 con 0x7f9104100a80 2024-09-12T03:01:15.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.248+0000 7f9101ffb640 1 -- 172.21.15.116:0/2685092532 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f8018990 con 0x7f9104100a80 2024-09-12T03:01:15.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.249+0000 7f910ae47640 1 -- 172.21.15.116:0/2685092532 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91041a7190 con 0x7f9104100a80 2024-09-12T03:01:15.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.250+0000 7f910ae47640 1 -- 172.21.15.116:0/2685092532 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9104100ee0 con 0x7f9104100a80 2024-09-12T03:01:15.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.251+0000 7f9101ffb640 1 -- 172.21.15.116:0/2685092532 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f90f8010ac0 con 0x7f9104100a80 2024-09-12T03:01:15.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.252+0000 7f9101ffb640 1 --2- 172.21.15.116:0/2685092532 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f90f40a2c00 0x7f90f40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:15.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.252+0000 7f9103fff640 1 --2- 172.21.15.116:0/2685092532 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f90f40a2c00 0x7f90f40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:15.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.252+0000 7f9101ffb640 1 -- 172.21.15.116:0/2685092532 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f90f80cf360 con 0x7f9104100a80 2024-09-12T03:01:15.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.252+0000 7f9103fff640 1 --2- 172.21.15.116:0/2685092532 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f90f40a2c00 0x7f90f40a50c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f90f00096c0 tx=0x7f90f0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:15.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.256+0000 7f9101ffb640 1 -- 172.21.15.116:0/2685092532 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f90f809c310 con 0x7f9104100a80 2024-09-12T03:01:15.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.451+0000 7f910ae47640 1 -- 172.21.15.116:0/2685092532 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f91040fd5c0 con 0x7f90f40a2c00 2024-09-12T03:01:15.456 INFO:teuthology.orchestra.run.smithi116.stdout:/volumes/qa/sv_0/97eaecf8-0139-4d29-88cc-7478aa625f6f 2024-09-12T03:01:15.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.456+0000 7f9101ffb640 1 -- 172.21.15.116:0/2685092532 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f91040fd5c0 con 0x7f90f40a2c00 2024-09-12T03:01:15.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.458+0000 7f910ae47640 1 -- 172.21.15.116:0/2685092532 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f90f40a2c00 msgr2=0x7f90f40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:15.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.458+0000 7f910ae47640 1 --2- 172.21.15.116:0/2685092532 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f90f40a2c00 0x7f90f40a50c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f90f00096c0 tx=0x7f90f0000bc0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.458+0000 7f910ae47640 1 -- 172.21.15.116:0/2685092532 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9104100a80 msgr2=0x7f910419a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:15.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.458+0000 7f910ae47640 1 --2- 172.21.15.116:0/2685092532 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9104100a80 0x7f910419a610 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f90f800e9f0 tx=0x7f90f800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.458+0000 7f910ae47640 1 -- 172.21.15.116:0/2685092532 shutdown_connections 2024-09-12T03:01:15.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.458+0000 7f910ae47640 1 --2- 172.21.15.116:0/2685092532 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f90f40a2c00 0x7f90f40a50c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.458+0000 7f910ae47640 1 --2- 172.21.15.116:0/2685092532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9104106a80 0x7f910419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.458+0000 7f910ae47640 1 --2- 172.21.15.116:0/2685092532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9104101420 0x7f910419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.458+0000 7f910ae47640 1 --2- 172.21.15.116:0/2685092532 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9104100a80 0x7f910419a610 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.458+0000 7f910ae47640 1 -- 172.21.15.116:0/2685092532 >> 172.21.15.116:0/2685092532 conn(0x7f91040fc7a0 msgr2=0x7f91040fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:15.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.458+0000 7f910ae47640 1 -- 172.21.15.116:0/2685092532 shutdown_connections 2024-09-12T03:01:15.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:15.458+0000 7f910ae47640 1 -- 172.21.15.116:0/2685092532 wait complete. 2024-09-12T03:01:15.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:15 smithi183 ceph-mon[35535]: pgmap v284: 97 pgs: 97 active+clean; 582 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-12T03:01:15.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:15 smithi177 ceph-mon[35305]: pgmap v284: 97 pgs: 97 active+clean; 582 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-12T03:01:15.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:15 smithi116 ceph-mon[31410]: pgmap v284: 97 pgs: 97 active+clean; 582 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-12T03:01:16.305 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-12T03:01:16.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:16 smithi116 ceph-mon[31410]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:01:16.556 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:16.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:16 smithi183 ceph-mon[35535]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:01:16.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:16 smithi177 ceph-mon[35305]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:01:17.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.046+0000 7f4f1022c640 1 -- 172.21.15.116:0/2112905938 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f08106050 msgr2=0x7f4f0810dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:17.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.046+0000 7f4f1022c640 1 --2- 172.21.15.116:0/2112905938 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f08106050 0x7f4f0810dbd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4f00009ca0 tx=0x7f4f0002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:01:17.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.046+0000 7f4f1022c640 1 -- 172.21.15.116:0/2112905938 shutdown_connections 2024-09-12T03:01:17.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.046+0000 7f4f1022c640 1 --2- 172.21.15.116:0/2112905938 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f08106050 0x7f4f0810dbd0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:17.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.046+0000 7f4f1022c640 1 --2- 172.21.15.116:0/2112905938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f08101df0 0x7f4f08105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:17.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.046+0000 7f4f1022c640 1 --2- 172.21.15.116:0/2112905938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f08101440 0x7f4f08101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:17.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.046+0000 7f4f1022c640 1 -- 172.21.15.116:0/2112905938 >> 172.21.15.116:0/2112905938 conn(0x7f4f08077210 msgr2=0x7f4f080fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:17.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.046+0000 7f4f1022c640 1 -- 172.21.15.116:0/2112905938 shutdown_connections 2024-09-12T03:01:17.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.046+0000 7f4f1022c640 1 -- 172.21.15.116:0/2112905938 wait complete. 2024-09-12T03:01:17.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.047+0000 7f4f1022c640 1 Processor -- start 2024-09-12T03:01:17.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.047+0000 7f4f1022c640 1 -- start start 2024-09-12T03:01:17.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.047+0000 7f4f1022c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f08101440 0x7f4f080fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:17.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.047+0000 7f4f1022c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f08101df0 0x7f4f080fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:17.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.047+0000 7f4f1022c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f08106050 0x7f4f080fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:17.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.047+0000 7f4f1022c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08111050 con 0x7f4f08101440 2024-09-12T03:01:17.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.047+0000 7f4f1022c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08110ed0 con 0x7f4f08101df0 2024-09-12T03:01:17.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.047+0000 7f4f1022c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4f081111d0 con 0x7f4f08106050 2024-09-12T03:01:17.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.047+0000 7f4f0dfc8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f08101440 0x7f4f080fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:17.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.047+0000 7f4f0dfc8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f08101440 0x7f4f080fc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54992/0 (socket says 172.21.15.116:54992) 2024-09-12T03:01:17.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.047+0000 7f4f0dfc8640 1 -- 172.21.15.116:0/3685530910 learned_addr learned my addr 172.21.15.116:0/3685530910 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:17.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.047+0000 7f4f0e7c9640 1 --2- 172.21.15.116:0/3685530910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f08106050 0x7f4f080fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:17.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.047+0000 7f4f0d7c7640 1 --2- 172.21.15.116:0/3685530910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f08101df0 0x7f4f080fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:17.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.047+0000 7f4f0dfc8640 1 -- 172.21.15.116:0/3685530910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f08106050 msgr2=0x7f4f080fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:17.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.047+0000 7f4f0dfc8640 1 --2- 172.21.15.116:0/3685530910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f08106050 0x7f4f080fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:17.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.048+0000 7f4f0dfc8640 1 -- 172.21.15.116:0/3685530910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f08101df0 msgr2=0x7f4f080fcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:17.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.048+0000 7f4f0dfc8640 1 --2- 172.21.15.116:0/3685530910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f08101df0 0x7f4f080fcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:17.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.048+0000 7f4f0dfc8640 1 -- 172.21.15.116:0/3685530910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f081a6b60 con 0x7f4f08101440 2024-09-12T03:01:17.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.048+0000 7f4f0e7c9640 1 --2- 172.21.15.116:0/3685530910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f08106050 0x7f4f080fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:01:17.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.048+0000 7f4f0d7c7640 1 --2- 172.21.15.116:0/3685530910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f08101df0 0x7f4f080fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:01:17.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.048+0000 7f4f0dfc8640 1 --2- 172.21.15.116:0/3685530910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f08101440 0x7f4f080fc690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400c890 tx=0x7f4f0400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:17.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.048+0000 7f4efeffd640 1 -- 172.21.15.116:0/3685530910 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f04004220 con 0x7f4f08101440 2024-09-12T03:01:17.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.048+0000 7f4efeffd640 1 -- 172.21.15.116:0/3685530910 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f040043c0 con 0x7f4f08101440 2024-09-12T03:01:17.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.048+0000 7f4f1022c640 1 -- 172.21.15.116:0/3685530910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f081a6e50 con 0x7f4f08101440 2024-09-12T03:01:17.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.048+0000 7f4efeffd640 1 -- 172.21.15.116:0/3685530910 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f040049e0 con 0x7f4f08101440 2024-09-12T03:01:17.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.048+0000 7f4f1022c640 1 -- 172.21.15.116:0/3685530910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f081a7300 con 0x7f4f08101440 2024-09-12T03:01:17.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.049+0000 7f4f1022c640 1 -- 172.21.15.116:0/3685530910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f08101820 con 0x7f4f08101440 2024-09-12T03:01:17.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.050+0000 7f4efeffd640 1 -- 172.21.15.116:0/3685530910 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f4f040079b0 con 0x7f4f08101440 2024-09-12T03:01:17.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.051+0000 7f4efeffd640 1 --2- 172.21.15.116:0/3685530910 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4ef40a2c10 0x7f4ef40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:17.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.051+0000 7f4f0d7c7640 1 --2- 172.21.15.116:0/3685530910 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4ef40a2c10 0x7f4ef40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:17.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.051+0000 7f4efeffd640 1 -- 172.21.15.116:0/3685530910 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f4f040cf130 con 0x7f4f08101440 2024-09-12T03:01:17.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.051+0000 7f4f0d7c7640 1 --2- 172.21.15.116:0/3685530910 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4ef40a2c10 0x7f4ef40a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8005fd0 tx=0x7f4ef80094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:17.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.055+0000 7f4efeffd640 1 -- 172.21.15.116:0/3685530910 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4f04014030 con 0x7f4f08101440 2024-09-12T03:01:17.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.247+0000 7f4f1022c640 1 -- 172.21.15.116:0/3685530910 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4f08062630 con 0x7f4ef40a2c10 2024-09-12T03:01:17.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.270+0000 7f4efeffd640 1 -- 172.21.15.116:0/3685530910 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4f08062630 con 0x7f4ef40a2c10 2024-09-12T03:01:17.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.272+0000 7f4f1022c640 1 -- 172.21.15.116:0/3685530910 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4ef40a2c10 msgr2=0x7f4ef40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:17.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.272+0000 7f4f1022c640 1 --2- 172.21.15.116:0/3685530910 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4ef40a2c10 0x7f4ef40a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8005fd0 tx=0x7f4ef80094d0 comp rx=0 tx=0).stop 2024-09-12T03:01:17.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.272+0000 7f4f1022c640 1 -- 172.21.15.116:0/3685530910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f08101440 msgr2=0x7f4f080fc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:17.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.272+0000 7f4f1022c640 1 --2- 172.21.15.116:0/3685530910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f08101440 0x7f4f080fc690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400c890 tx=0x7f4f0400cd50 comp rx=0 tx=0).stop 2024-09-12T03:01:17.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.273+0000 7f4f1022c640 1 -- 172.21.15.116:0/3685530910 shutdown_connections 2024-09-12T03:01:17.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.273+0000 7f4f1022c640 1 --2- 172.21.15.116:0/3685530910 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4ef40a2c10 0x7f4ef40a50d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:17.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.273+0000 7f4f1022c640 1 --2- 172.21.15.116:0/3685530910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f08106050 0x7f4f080fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:17.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.273+0000 7f4f1022c640 1 --2- 172.21.15.116:0/3685530910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f08101df0 0x7f4f080fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:17.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.273+0000 7f4f1022c640 1 --2- 172.21.15.116:0/3685530910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f08101440 0x7f4f080fc690 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:17.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.273+0000 7f4f1022c640 1 -- 172.21.15.116:0/3685530910 >> 172.21.15.116:0/3685530910 conn(0x7f4f08077210 msgr2=0x7f4f081003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:17.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.273+0000 7f4f1022c640 1 -- 172.21.15.116:0/3685530910 shutdown_connections 2024-09-12T03:01:17.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:17.273+0000 7f4f1022c640 1 -- 172.21.15.116:0/3685530910 wait complete. 2024-09-12T03:01:17.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:17 smithi116 ceph-mon[31410]: pgmap v285: 97 pgs: 97 active+clean; 602 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.9 KiB/s wr, 12 op/s 2024-09-12T03:01:17.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:17 smithi116 ceph-mon[31410]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:01:17.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:17 smithi183 ceph-mon[35535]: pgmap v285: 97 pgs: 97 active+clean; 602 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.9 KiB/s wr, 12 op/s 2024-09-12T03:01:17.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:17 smithi183 ceph-mon[35535]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:01:17.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:17 smithi177 ceph-mon[35305]: pgmap v285: 97 pgs: 97 active+clean; 602 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.9 KiB/s wr, 12 op/s 2024-09-12T03:01:17.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:17 smithi177 ceph-mon[35305]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:01:18.078 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-12T03:01:18.323 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:18.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.812+0000 7f5726256640 1 -- 172.21.15.116:0/3560557897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5720102ef0 msgr2=0x7f572010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:18.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.812+0000 7f5726256640 1 --2- 172.21.15.116:0/3560557897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5720102ef0 0x7f572010da40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5718009ca0 tx=0x7f571802a950 comp rx=0 tx=0).stop 2024-09-12T03:01:18.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.812+0000 7f5726256640 1 -- 172.21.15.116:0/3560557897 shutdown_connections 2024-09-12T03:01:18.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.812+0000 7f5726256640 1 --2- 172.21.15.116:0/3560557897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5720102ef0 0x7f572010da40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.812+0000 7f5726256640 1 --2- 172.21.15.116:0/3560557897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5720102570 0x7f57201029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.812+0000 7f5726256640 1 --2- 172.21.15.116:0/3560557897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5720100e20 0x7f5720101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.812+0000 7f5726256640 1 -- 172.21.15.116:0/3560557897 >> 172.21.15.116:0/3560557897 conn(0x7f57200fc7d0 msgr2=0x7f57200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:18.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.812+0000 7f5726256640 1 -- 172.21.15.116:0/3560557897 shutdown_connections 2024-09-12T03:01:18.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.812+0000 7f5726256640 1 -- 172.21.15.116:0/3560557897 wait complete. 2024-09-12T03:01:18.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.813+0000 7f5726256640 1 Processor -- start 2024-09-12T03:01:18.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.813+0000 7f5726256640 1 -- start start 2024-09-12T03:01:18.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.813+0000 7f5726256640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5720100e20 0x7f5720199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:18.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.813+0000 7f5726256640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5720102570 0x7f5720199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:18.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.813+0000 7f5726256640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5720102ef0 0x7f57201a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:18.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.813+0000 7f5726256640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5720110f80 con 0x7f5720102ef0 2024-09-12T03:01:18.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.813+0000 7f5726256640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5720110e00 con 0x7f5720100e20 2024-09-12T03:01:18.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.813+0000 7f5726256640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5720111100 con 0x7f5720102570 2024-09-12T03:01:18.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.813+0000 7f571ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5720102ef0 0x7f57201a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:18.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.813+0000 7f571ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5720102ef0 0x7f57201a42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55006/0 (socket says 172.21.15.116:55006) 2024-09-12T03:01:18.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.813+0000 7f571ffff640 1 -- 172.21.15.116:0/3484668249 learned_addr learned my addr 172.21.15.116:0/3484668249 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:18.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.813+0000 7f5717fff640 1 --2- 172.21.15.116:0/3484668249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5720102570 0x7f5720199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:18.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.813+0000 7f571f7fe640 1 --2- 172.21.15.116:0/3484668249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5720100e20 0x7f5720199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:18.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.813+0000 7f571ffff640 1 -- 172.21.15.116:0/3484668249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5720102570 msgr2=0x7f5720199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:18.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.813+0000 7f571ffff640 1 --2- 172.21.15.116:0/3484668249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5720102570 0x7f5720199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.813+0000 7f571ffff640 1 -- 172.21.15.116:0/3484668249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5720100e20 msgr2=0x7f5720199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:18.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.813+0000 7f571ffff640 1 --2- 172.21.15.116:0/3484668249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5720100e20 0x7f5720199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.813+0000 7f571ffff640 1 -- 172.21.15.116:0/3484668249 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57201a47f0 con 0x7f5720102ef0 2024-09-12T03:01:18.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.813+0000 7f571ffff640 1 --2- 172.21.15.116:0/3484668249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5720102ef0 0x7f57201a42b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f571802ae70 tx=0x7f571802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:18.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.814+0000 7f571d7fa640 1 -- 172.21.15.116:0/3484668249 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5718034040 con 0x7f5720102ef0 2024-09-12T03:01:18.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.814+0000 7f571d7fa640 1 -- 172.21.15.116:0/3484668249 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5718030920 con 0x7f5720102ef0 2024-09-12T03:01:18.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.814+0000 7f5726256640 1 -- 172.21.15.116:0/3484668249 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57201a49c0 con 0x7f5720102ef0 2024-09-12T03:01:18.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.814+0000 7f571d7fa640 1 -- 172.21.15.116:0/3484668249 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57180047f0 con 0x7f5720102ef0 2024-09-12T03:01:18.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.814+0000 7f5726256640 1 -- 172.21.15.116:0/3484668249 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57201a4e10 con 0x7f5720102ef0 2024-09-12T03:01:18.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.815+0000 7f5726256640 1 -- 172.21.15.116:0/3484668249 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5720101200 con 0x7f5720102ef0 2024-09-12T03:01:18.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.816+0000 7f571d7fa640 1 -- 172.21.15.116:0/3484668249 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f5718037020 con 0x7f5720102ef0 2024-09-12T03:01:18.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.816+0000 7f571d7fa640 1 --2- 172.21.15.116:0/3484668249 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f570c0a2c10 0x7f570c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:18.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.817+0000 7f571f7fe640 1 --2- 172.21.15.116:0/3484668249 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f570c0a2c10 0x7f570c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:18.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.817+0000 7f571d7fa640 1 -- 172.21.15.116:0/3484668249 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f57180efdc0 con 0x7f5720102ef0 2024-09-12T03:01:18.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.817+0000 7f571f7fe640 1 --2- 172.21.15.116:0/3484668249 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f570c0a2c10 0x7f570c0a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5710005fd0 tx=0x7f571000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:18.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:18.821+0000 7f571d7fa640 1 -- 172.21.15.116:0/3484668249 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f57180bce80 con 0x7f5720102ef0 2024-09-12T03:01:19.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:19.012+0000 7f5726256640 1 -- 172.21.15.116:0/3484668249 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f5720106ed0 con 0x7f570c0a2c10 2024-09-12T03:01:19.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:19.017+0000 7f571d7fa640 1 -- 172.21.15.116:0/3484668249 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f5720106ed0 con 0x7f570c0a2c10 2024-09-12T03:01:19.017 INFO:teuthology.orchestra.run.smithi116.stdout:/volumes/qa/sv_1/8f4fbfe1-cd89-4b49-9931-ffd07687f37b 2024-09-12T03:01:19.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:19.019+0000 7f5726256640 1 -- 172.21.15.116:0/3484668249 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f570c0a2c10 msgr2=0x7f570c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:19.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:19.019+0000 7f5726256640 1 --2- 172.21.15.116:0/3484668249 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f570c0a2c10 0x7f570c0a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5710005fd0 tx=0x7f571000b390 comp rx=0 tx=0).stop 2024-09-12T03:01:19.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:19.019+0000 7f5726256640 1 -- 172.21.15.116:0/3484668249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5720102ef0 msgr2=0x7f57201a42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:19.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:19.019+0000 7f5726256640 1 --2- 172.21.15.116:0/3484668249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5720102ef0 0x7f57201a42b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f571802ae70 tx=0x7f571802f5d0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:19.019+0000 7f5726256640 1 -- 172.21.15.116:0/3484668249 shutdown_connections 2024-09-12T03:01:19.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:19.019+0000 7f5726256640 1 --2- 172.21.15.116:0/3484668249 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f570c0a2c10 0x7f570c0a50d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:19.019+0000 7f5726256640 1 --2- 172.21.15.116:0/3484668249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5720102ef0 0x7f57201a42b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:19.019+0000 7f5726256640 1 --2- 172.21.15.116:0/3484668249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5720102570 0x7f5720199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:19.019+0000 7f5726256640 1 --2- 172.21.15.116:0/3484668249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5720100e20 0x7f5720199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:19.019+0000 7f5726256640 1 -- 172.21.15.116:0/3484668249 >> 172.21.15.116:0/3484668249 conn(0x7f57200fc7d0 msgr2=0x7f57201067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:19.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:19.019+0000 7f5726256640 1 -- 172.21.15.116:0/3484668249 shutdown_connections 2024-09-12T03:01:19.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:19.019+0000 7f5726256640 1 -- 172.21.15.116:0/3484668249 wait complete. 2024-09-12T03:01:19.762 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:01:20.008 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:20.428 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:20 smithi116 ceph-mon[31410]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:01:20.429 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:20 smithi116 ceph-mon[31410]: pgmap v286: 97 pgs: 97 active+clean; 602 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 4.1 KiB/s wr, 13 op/s 2024-09-12T03:01:20.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:20 smithi183 ceph-mon[35535]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:01:20.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:20 smithi183 ceph-mon[35535]: pgmap v286: 97 pgs: 97 active+clean; 602 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 4.1 KiB/s wr, 13 op/s 2024-09-12T03:01:20.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:20 smithi177 ceph-mon[35305]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:01:20.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:20 smithi177 ceph-mon[35305]: pgmap v286: 97 pgs: 97 active+clean; 602 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 4.1 KiB/s wr, 13 op/s 2024-09-12T03:01:20.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.508+0000 7f081c83b640 1 -- 172.21.15.116:0/2328395870 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0818106f10 msgr2=0x7f081810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:20.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.508+0000 7f081c83b640 1 --2- 172.21.15.116:0/2328395870 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0818106f10 0x7f081810ea90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0810009ca0 tx=0x7f081002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:01:20.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.508+0000 7f081c83b640 1 -- 172.21.15.116:0/2328395870 shutdown_connections 2024-09-12T03:01:20.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.508+0000 7f081c83b640 1 --2- 172.21.15.116:0/2328395870 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0818106f10 0x7f081810ea90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:20.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.508+0000 7f081c83b640 1 --2- 172.21.15.116:0/2328395870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0818102790 0x7f08181067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:20.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.508+0000 7f081c83b640 1 --2- 172.21.15.116:0/2328395870 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0818101de0 0x7f08181021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:20.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.508+0000 7f081c83b640 1 -- 172.21.15.116:0/2328395870 >> 172.21.15.116:0/2328395870 conn(0x7f08180fd8c0 msgr2=0x7f08180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:20.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.508+0000 7f081c83b640 1 -- 172.21.15.116:0/2328395870 shutdown_connections 2024-09-12T03:01:20.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.508+0000 7f081c83b640 1 -- 172.21.15.116:0/2328395870 wait complete. 2024-09-12T03:01:20.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.509+0000 7f081c83b640 1 Processor -- start 2024-09-12T03:01:20.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.509+0000 7f081c83b640 1 -- start start 2024-09-12T03:01:20.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.509+0000 7f081c83b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0818101de0 0x7f0818198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:20.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.509+0000 7f081c83b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0818102790 0x7f0818198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:20.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.509+0000 7f081c83b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0818106f10 0x7f081819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:20.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.509+0000 7f081c83b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0818111f70 con 0x7f0818102790 2024-09-12T03:01:20.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.509+0000 7f081c83b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0818111df0 con 0x7f0818101de0 2024-09-12T03:01:20.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.509+0000 7f081c83b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f08181120f0 con 0x7f0818106f10 2024-09-12T03:01:20.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.509+0000 7f080ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0818102790 0x7f0818198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:20.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.509+0000 7f080ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0818102790 0x7f0818198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55018/0 (socket says 172.21.15.116:55018) 2024-09-12T03:01:20.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.509+0000 7f080ffff640 1 -- 172.21.15.116:0/461397325 learned_addr learned my addr 172.21.15.116:0/461397325 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:20.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.510+0000 7f081659c640 1 --2- 172.21.15.116:0/461397325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0818101de0 0x7f0818198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:20.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.510+0000 7f0816d9d640 1 --2- 172.21.15.116:0/461397325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0818106f10 0x7f081819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:20.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.510+0000 7f080ffff640 1 -- 172.21.15.116:0/461397325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0818106f10 msgr2=0x7f081819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:20.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.510+0000 7f080ffff640 1 --2- 172.21.15.116:0/461397325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0818106f10 0x7f081819cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:20.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.510+0000 7f080ffff640 1 -- 172.21.15.116:0/461397325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0818101de0 msgr2=0x7f0818198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:20.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.510+0000 7f080ffff640 1 --2- 172.21.15.116:0/461397325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0818101de0 0x7f0818198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:20.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.510+0000 7f080ffff640 1 -- 172.21.15.116:0/461397325 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f081819d2e0 con 0x7f0818102790 2024-09-12T03:01:20.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.510+0000 7f080ffff640 1 --2- 172.21.15.116:0/461397325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0818102790 0x7f0818198870 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f080000c890 tx=0x7f080000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:20.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.510+0000 7f080f7fe640 1 -- 172.21.15.116:0/461397325 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0800004220 con 0x7f0818102790 2024-09-12T03:01:20.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.510+0000 7f081c83b640 1 -- 172.21.15.116:0/461397325 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f081819d5d0 con 0x7f0818102790 2024-09-12T03:01:20.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.510+0000 7f080f7fe640 1 -- 172.21.15.116:0/461397325 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08000043c0 con 0x7f0818102790 2024-09-12T03:01:20.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.510+0000 7f080f7fe640 1 -- 172.21.15.116:0/461397325 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0800004a00 con 0x7f0818102790 2024-09-12T03:01:20.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.510+0000 7f081c83b640 1 -- 172.21.15.116:0/461397325 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08181a4eb0 con 0x7f0818102790 2024-09-12T03:01:20.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.511+0000 7f081c83b640 1 -- 172.21.15.116:0/461397325 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08181021c0 con 0x7f0818102790 2024-09-12T03:01:20.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.512+0000 7f080f7fe640 1 -- 172.21.15.116:0/461397325 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f08000079b0 con 0x7f0818102790 2024-09-12T03:01:20.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.513+0000 7f080f7fe640 1 --2- 172.21.15.116:0/461397325 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f08040a2c10 0x7f08040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:20.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.513+0000 7f081659c640 1 --2- 172.21.15.116:0/461397325 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f08040a2c10 0x7f08040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:20.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.513+0000 7f080f7fe640 1 -- 172.21.15.116:0/461397325 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f08000cec50 con 0x7f0818102790 2024-09-12T03:01:20.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.513+0000 7f081659c640 1 --2- 172.21.15.116:0/461397325 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f08040a2c10 0x7f08040a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f080800abe0 tx=0x7f080800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:20.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.517+0000 7f080f7fe640 1 -- 172.21.15.116:0/461397325 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0800014030 con 0x7f0818102790 2024-09-12T03:01:20.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.718+0000 7f081c83b640 1 -- 172.21.15.116:0/461397325 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f081802d7c0 con 0x7f0818102790 2024-09-12T03:01:20.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.719+0000 7f080f7fe640 1 -- 172.21.15.116:0/461397325 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11938 (secure 0 0 0) 0x7f080009bb80 con 0x7f0818102790 2024-09-12T03:01:20.720 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:01:20.720 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":24,"btime":"2024-09-12T03:01:10:397269+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:01:10.218314+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24467,24475,24494]},"id":1}]} 2024-09-12T03:01:20.720 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 24 2024-09-12T03:01:20.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.721+0000 7f081c83b640 1 -- 172.21.15.116:0/461397325 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f08040a2c10 msgr2=0x7f08040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:20.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.721+0000 7f081c83b640 1 --2- 172.21.15.116:0/461397325 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f08040a2c10 0x7f08040a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f080800abe0 tx=0x7f080800a2d0 comp rx=0 tx=0).stop 2024-09-12T03:01:20.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.721+0000 7f081c83b640 1 -- 172.21.15.116:0/461397325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0818102790 msgr2=0x7f0818198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:20.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.721+0000 7f081c83b640 1 --2- 172.21.15.116:0/461397325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0818102790 0x7f0818198870 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f080000c890 tx=0x7f080000cc90 comp rx=0 tx=0).stop 2024-09-12T03:01:20.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.721+0000 7f081c83b640 1 -- 172.21.15.116:0/461397325 shutdown_connections 2024-09-12T03:01:20.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.721+0000 7f081c83b640 1 --2- 172.21.15.116:0/461397325 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f08040a2c10 0x7f08040a50d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:20.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.721+0000 7f081c83b640 1 --2- 172.21.15.116:0/461397325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0818106f10 0x7f081819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:20.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.721+0000 7f081c83b640 1 --2- 172.21.15.116:0/461397325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0818102790 0x7f0818198870 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:20.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.721+0000 7f081c83b640 1 --2- 172.21.15.116:0/461397325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0818101de0 0x7f0818198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:20.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.721+0000 7f081c83b640 1 -- 172.21.15.116:0/461397325 >> 172.21.15.116:0/461397325 conn(0x7f08180fd8c0 msgr2=0x7f08180febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:20.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.721+0000 7f081c83b640 1 -- 172.21.15.116:0/461397325 shutdown_connections 2024-09-12T03:01:20.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:20.722+0000 7f081c83b640 1 -- 172.21.15.116:0/461397325 wait complete. 2024-09-12T03:01:21.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:21 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/461397325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:01:21.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:21 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/461397325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:01:21.499 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:21 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/461397325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:01:21.528 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:01:21.775 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:22.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.277+0000 7f64085ae640 1 -- 172.21.15.116:0/3832127227 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6400074e30 msgr2=0x7f640010df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:22.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.277+0000 7f64085ae640 1 --2- 172.21.15.116:0/3832127227 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6400074e30 0x7f640010df10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f63f4009900 tx=0x7f63f402a480 comp rx=0 tx=0).stop 2024-09-12T03:01:22.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.277+0000 7f64085ae640 1 -- 172.21.15.116:0/3832127227 shutdown_connections 2024-09-12T03:01:22.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.277+0000 7f64085ae640 1 --2- 172.21.15.116:0/3832127227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f640010e450 0x7f6400110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:22.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.277+0000 7f64085ae640 1 --2- 172.21.15.116:0/3832127227 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6400074e30 0x7f640010df10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:22.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.277+0000 7f64085ae640 1 --2- 172.21.15.116:0/3832127227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6400074510 0x7f64000748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:22.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.277+0000 7f64085ae640 1 -- 172.21.15.116:0/3832127227 >> 172.21.15.116:0/3832127227 conn(0x7f64000fd8f0 msgr2=0x7f64000ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:22.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.277+0000 7f64085ae640 1 -- 172.21.15.116:0/3832127227 shutdown_connections 2024-09-12T03:01:22.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.277+0000 7f64085ae640 1 -- 172.21.15.116:0/3832127227 wait complete. 2024-09-12T03:01:22.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.277+0000 7f64085ae640 1 Processor -- start 2024-09-12T03:01:22.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.277+0000 7f64085ae640 1 -- start start 2024-09-12T03:01:22.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.278+0000 7f64085ae640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6400074510 0x7f64001a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:22.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.278+0000 7f64085ae640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6400074e30 0x7f64001a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:22.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.278+0000 7f64085ae640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f640010e450 0x7f640019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:22.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.278+0000 7f64085ae640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6400114150 con 0x7f6400074e30 2024-09-12T03:01:22.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.278+0000 7f64085ae640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6400113fd0 con 0x7f640010e450 2024-09-12T03:01:22.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.278+0000 7f64085ae640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f64001142d0 con 0x7f6400074510 2024-09-12T03:01:22.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.278+0000 7f6405b49640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6400074e30 0x7f64001a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:22.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.278+0000 7f6405b49640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6400074e30 0x7f64001a0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34074/0 (socket says 172.21.15.116:34074) 2024-09-12T03:01:22.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.278+0000 7f6405b49640 1 -- 172.21.15.116:0/1633068931 learned_addr learned my addr 172.21.15.116:0/1633068931 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:22.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.278+0000 7f6406b4b640 1 --2- 172.21.15.116:0/1633068931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f640010e450 0x7f640019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:22.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.278+0000 7f640634a640 1 --2- 172.21.15.116:0/1633068931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6400074510 0x7f64001a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:22.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.278+0000 7f6405b49640 1 -- 172.21.15.116:0/1633068931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6400074510 msgr2=0x7f64001a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:22.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.278+0000 7f6405b49640 1 --2- 172.21.15.116:0/1633068931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6400074510 0x7f64001a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:22.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.278+0000 7f6405b49640 1 -- 172.21.15.116:0/1633068931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f640010e450 msgr2=0x7f640019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:22.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.278+0000 7f6405b49640 1 --2- 172.21.15.116:0/1633068931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f640010e450 0x7f640019a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:22.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.278+0000 7f6405b49640 1 -- 172.21.15.116:0/1633068931 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640019acc0 con 0x7f6400074e30 2024-09-12T03:01:22.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.278+0000 7f6405b49640 1 --2- 172.21.15.116:0/1633068931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6400074e30 0x7f64001a0910 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f63f4006010 tx=0x7f63f40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:22.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.278+0000 7f63ff7fe640 1 -- 172.21.15.116:0/1633068931 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f403f030 con 0x7f6400074e30 2024-09-12T03:01:22.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.278+0000 7f64085ae640 1 -- 172.21.15.116:0/1633068931 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f640019af50 con 0x7f6400074e30 2024-09-12T03:01:22.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.278+0000 7f63ff7fe640 1 -- 172.21.15.116:0/1633068931 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63f4003ce0 con 0x7f6400074e30 2024-09-12T03:01:22.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.278+0000 7f64085ae640 1 -- 172.21.15.116:0/1633068931 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64001a72a0 con 0x7f6400074e30 2024-09-12T03:01:22.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.279+0000 7f63ff7fe640 1 -- 172.21.15.116:0/1633068931 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f4035070 con 0x7f6400074e30 2024-09-12T03:01:22.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.279+0000 7f64085ae640 1 -- 172.21.15.116:0/1633068931 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64000748f0 con 0x7f6400074e30 2024-09-12T03:01:22.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.281+0000 7f63ff7fe640 1 -- 172.21.15.116:0/1633068931 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f63f4042030 con 0x7f6400074e30 2024-09-12T03:01:22.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.281+0000 7f63ff7fe640 1 --2- 172.21.15.116:0/1633068931 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f63e80a2c10 0x7f63e80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:22.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.281+0000 7f640634a640 1 --2- 172.21.15.116:0/1633068931 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f63e80a2c10 0x7f63e80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:22.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.281+0000 7f63ff7fe640 1 -- 172.21.15.116:0/1633068931 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f63f40ef920 con 0x7f6400074e30 2024-09-12T03:01:22.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.282+0000 7f640634a640 1 --2- 172.21.15.116:0/1633068931 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f63e80a2c10 0x7f63e80a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f63f0005c40 tx=0x7f63f0008230 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:22.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.284+0000 7f63ff7fe640 1 -- 172.21.15.116:0/1633068931 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f63f40bc7d0 con 0x7f6400074e30 2024-09-12T03:01:22.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:22 smithi183 ceph-mon[35535]: pgmap v287: 97 pgs: 97 active+clean; 602 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 4.0 KiB/s wr, 12 op/s 2024-09-12T03:01:22.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.459+0000 7f64085ae640 1 -- 172.21.15.116:0/1633068931 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64001095d0 con 0x7f6400074e30 2024-09-12T03:01:22.459 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:22 smithi116 ceph-mon[31410]: pgmap v287: 97 pgs: 97 active+clean; 602 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 4.0 KiB/s wr, 12 op/s 2024-09-12T03:01:22.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.460+0000 7f63ff7fe640 1 -- 172.21.15.116:0/1633068931 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22325 (secure 0 0 0) 0x7f63f40bc0d0 con 0x7f6400074e30 2024-09-12T03:01:22.461 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:01:22.461 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":78,"fsid":"eb1ac602-70b1-11ef-bcea-c7b262605968","created":"2024-09-12T02:53:07.105686+0000","modified":"2024-09-12T03:00:59.195124+0000","last_up_change":"2024-09-12T03:00:07.202639+0000","last_in_change":"2024-09-12T02:59:43.446336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:56:17.347152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:00:55.191040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:00:57.171142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8cf5a18-dc6f-4bb6-bd74-78adda310118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6803","nonce":862366853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6805","nonce":862366853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6809","nonce":862366853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6807","nonce":862366853}]},"public_addr":"172.21.15.116:6803/862366853","cluster_addr":"172.21.15.116:6805/862366853","heartbeat_back_addr":"172.21.15.116:6809/862366853","heartbeat_front_addr":"172.21.15.116:6807/862366853","state":["exists","up"]},{"osd":1,"uuid":"0affd3cb-9a51-42b0-a2cc-f30784a75cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6801","nonce":2245800547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6803","nonce":2245800547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6807","nonce":2245800547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6805","nonce":2245800547}]},"public_addr":"172.21.15.177:6801/2245800547","cluster_addr":"172.21.15.177:6803/2245800547","heartbeat_back_addr":"172.21.15.177:6807/2245800547","heartbeat_front_addr":"172.21.15.177:6805/2245800547","state":["exists","up"]},{"osd":2,"uuid":"1ce45ea5-84df-4a35-924f-78f92bd4f2cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6801","nonce":2645677759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6803","nonce":2645677759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6807","nonce":2645677759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6805","nonce":2645677759}]},"public_addr":"172.21.15.183:6801/2645677759","cluster_addr":"172.21.15.183:6803/2645677759","heartbeat_back_addr":"172.21.15.183:6807/2645677759","heartbeat_front_addr":"172.21.15.183:6805/2645677759","state":["exists","up"]},{"osd":3,"uuid":"ae5e8178-c998-4648-bc05-4bce822f0cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6811","nonce":686622148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6813","nonce":686622148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6817","nonce":686622148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6815","nonce":686622148}]},"public_addr":"172.21.15.116:6811/686622148","cluster_addr":"172.21.15.116:6813/686622148","heartbeat_back_addr":"172.21.15.116:6817/686622148","heartbeat_front_addr":"172.21.15.116:6815/686622148","state":["exists","up"]},{"osd":4,"uuid":"c81d2e73-0bc2-440d-b5a2-b7428af3ea4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6809","nonce":2382938021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6811","nonce":2382938021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6815","nonce":2382938021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6813","nonce":2382938021}]},"public_addr":"172.21.15.177:6809/2382938021","cluster_addr":"172.21.15.177:6811/2382938021","heartbeat_back_addr":"172.21.15.177:6815/2382938021","heartbeat_front_addr":"172.21.15.177:6813/2382938021","state":["exists","up"]},{"osd":5,"uuid":"c3f168be-9e27-4cf5-b73e-3f47d8990ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6809","nonce":85363432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6811","nonce":85363432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6815","nonce":85363432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6813","nonce":85363432}]},"public_addr":"172.21.15.183:6809/85363432","cluster_addr":"172.21.15.183:6811/85363432","heartbeat_back_addr":"172.21.15.183:6815/85363432","heartbeat_front_addr":"172.21.15.183:6813/85363432","state":["exists","up"]},{"osd":6,"uuid":"beb0a7e9-4909-47e2-9ee7-6951d4745e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6819","nonce":1121291186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6821","nonce":1121291186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6825","nonce":1121291186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6823","nonce":1121291186}]},"public_addr":"172.21.15.116:6819/1121291186","cluster_addr":"172.21.15.116:6821/1121291186","heartbeat_back_addr":"172.21.15.116:6825/1121291186","heartbeat_front_addr":"172.21.15.116:6823/1121291186","state":["exists","up"]},{"osd":7,"uuid":"761f4680-f049-473c-bf44-b19acf8de24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6817","nonce":2177665961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6819","nonce":2177665961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6823","nonce":2177665961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6821","nonce":2177665961}]},"public_addr":"172.21.15.177:6817/2177665961","cluster_addr":"172.21.15.177:6819/2177665961","heartbeat_back_addr":"172.21.15.177:6823/2177665961","heartbeat_front_addr":"172.21.15.177:6821/2177665961","state":["exists","up"]},{"osd":8,"uuid":"2385ea6c-fd5a-4815-9596-77f948629542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6817","nonce":1305725258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6819","nonce":1305725258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6823","nonce":1305725258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6821","nonce":1305725258}]},"public_addr":"172.21.15.183:6817/1305725258","cluster_addr":"172.21.15.183:6819/1305725258","heartbeat_back_addr":"172.21.15.183:6823/1305725258","heartbeat_front_addr":"172.21.15.183:6821/1305725258","state":["exists","up"]},{"osd":9,"uuid":"fd0c0dc4-4990-4f12-aa14-f855504cd995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6827","nonce":3831673668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6829","nonce":3831673668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6833","nonce":3831673668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6831","nonce":3831673668}]},"public_addr":"172.21.15.116:6827/3831673668","cluster_addr":"172.21.15.116:6829/3831673668","heartbeat_back_addr":"172.21.15.116:6833/3831673668","heartbeat_front_addr":"172.21.15.116:6831/3831673668","state":["exists","up"]},{"osd":10,"uuid":"864d1a97-4810-49c8-ad4a-a0c10253833c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6825","nonce":963663634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6827","nonce":963663634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6831","nonce":963663634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6829","nonce":963663634}]},"public_addr":"172.21.15.177:6825/963663634","cluster_addr":"172.21.15.177:6827/963663634","heartbeat_back_addr":"172.21.15.177:6831/963663634","heartbeat_front_addr":"172.21.15.177:6829/963663634","state":["exists","up"]},{"osd":11,"uuid":"d49c0841-4620-4f2f-a6b5-f2a9aa2e3c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6825","nonce":1818319939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6827","nonce":1818319939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6831","nonce":1818319939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6829","nonce":1818319939}]},"public_addr":"172.21.15.183:6825/1818319939","cluster_addr":"172.21.15.183:6827/1818319939","heartbeat_back_addr":"172.21.15.183:6831/1818319939","heartbeat_front_addr":"172.21.15.183:6829/1818319939","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:24.736944+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:50.310980+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:56:14.989533+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:56:41.187047+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:06.981664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:32.872771+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:58.850089+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:23.043121+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:47.839958+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:14.087708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:40.010548+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:05.364820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.116:6801/2966274423":"2024-09-13T02:54:03.026862+0000","172.21.15.116:6800/2966274423":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/686139455":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/1552047390":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/2180363411":"2024-09-13T02:53:39.433769+0000","172.21.15.116:6801/1791219601":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/3907400460":"2024-09-13T02:54:03.026862+0000","172.21.15.116:6800/1791219601":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/1802566286":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/212337802":"2024-09-13T02:53:39.433769+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:01:22.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.463+0000 7f64085ae640 1 -- 172.21.15.116:0/1633068931 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f63e80a2c10 msgr2=0x7f63e80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:22.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.463+0000 7f64085ae640 1 --2- 172.21.15.116:0/1633068931 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f63e80a2c10 0x7f63e80a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f63f0005c40 tx=0x7f63f0008230 comp rx=0 tx=0).stop 2024-09-12T03:01:22.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.463+0000 7f64085ae640 1 -- 172.21.15.116:0/1633068931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6400074e30 msgr2=0x7f64001a0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:22.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.463+0000 7f64085ae640 1 --2- 172.21.15.116:0/1633068931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6400074e30 0x7f64001a0910 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f63f4006010 tx=0x7f63f40042f0 comp rx=0 tx=0).stop 2024-09-12T03:01:22.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.463+0000 7f64085ae640 1 -- 172.21.15.116:0/1633068931 shutdown_connections 2024-09-12T03:01:22.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.463+0000 7f64085ae640 1 --2- 172.21.15.116:0/1633068931 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f63e80a2c10 0x7f63e80a50d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:22.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.463+0000 7f64085ae640 1 --2- 172.21.15.116:0/1633068931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f640010e450 0x7f640019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:22.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.463+0000 7f64085ae640 1 --2- 172.21.15.116:0/1633068931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6400074e30 0x7f64001a0910 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:22.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.463+0000 7f64085ae640 1 --2- 172.21.15.116:0/1633068931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6400074510 0x7f64001a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:22.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.463+0000 7f64085ae640 1 -- 172.21.15.116:0/1633068931 >> 172.21.15.116:0/1633068931 conn(0x7f64000fd8f0 msgr2=0x7f64000ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:22.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.463+0000 7f64085ae640 1 -- 172.21.15.116:0/1633068931 shutdown_connections 2024-09-12T03:01:22.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:22.463+0000 7f64085ae640 1 -- 172.21.15.116:0/1633068931 wait complete. 2024-09-12T03:01:22.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:22 smithi177 ceph-mon[35305]: pgmap v287: 97 pgs: 97 active+clean; 602 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 4.0 KiB/s wr, 12 op/s 2024-09-12T03:01:23.204 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T03:01:23.205 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:01:23.451 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:23.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:23 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1633068931' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:01:23.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:23 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1633068931' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:01:23.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:23 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1633068931' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:01:23.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.933+0000 7fc7f11dc640 1 -- 172.21.15.116:0/3554707917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc7ec101420 msgr2=0x7fc7ec10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:23.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.933+0000 7fc7f11dc640 1 --2- 172.21.15.116:0/3554707917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc7ec101420 0x7fc7ec10d950 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4009cd0 tx=0x7fc7e402a8e0 comp rx=0 tx=0).stop 2024-09-12T03:01:23.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.933+0000 7fc7f11dc640 1 -- 172.21.15.116:0/3554707917 shutdown_connections 2024-09-12T03:01:23.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.933+0000 7fc7f11dc640 1 --2- 172.21.15.116:0/3554707917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc7ec101420 0x7fc7ec10d950 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:23.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.933+0000 7fc7f11dc640 1 --2- 172.21.15.116:0/3554707917 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7ec100a80 0x7fc7ec100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:23.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.933+0000 7fc7f11dc640 1 --2- 172.21.15.116:0/3554707917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7ec106a80 0x7fc7ec106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:23.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.933+0000 7fc7f11dc640 1 -- 172.21.15.116:0/3554707917 >> 172.21.15.116:0/3554707917 conn(0x7fc7ec0fc7a0 msgr2=0x7fc7ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:23.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.933+0000 7fc7f11dc640 1 -- 172.21.15.116:0/3554707917 shutdown_connections 2024-09-12T03:01:23.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.933+0000 7fc7f11dc640 1 -- 172.21.15.116:0/3554707917 wait complete. 2024-09-12T03:01:23.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.933+0000 7fc7f11dc640 1 Processor -- start 2024-09-12T03:01:23.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.934+0000 7fc7f11dc640 1 -- start start 2024-09-12T03:01:23.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.934+0000 7fc7f11dc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7ec100a80 0x7fc7ec1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:23.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.934+0000 7fc7f11dc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7ec101420 0x7fc7ec198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:23.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.934+0000 7fc7f11dc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc7ec106a80 0x7fc7ec19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:23.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.934+0000 7fc7f11dc640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ec112020 con 0x7fc7ec106a80 2024-09-12T03:01:23.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.934+0000 7fc7f11dc640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ec111ea0 con 0x7fc7ec100a80 2024-09-12T03:01:23.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.934+0000 7fc7f11dc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ec1121a0 con 0x7fc7ec101420 2024-09-12T03:01:23.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.934+0000 7fc7eb59e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc7ec106a80 0x7fc7ec19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:23.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.934+0000 7fc7eb59e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc7ec106a80 0x7fc7ec19ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34096/0 (socket says 172.21.15.116:34096) 2024-09-12T03:01:23.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.934+0000 7fc7eb59e640 1 -- 172.21.15.116:0/1622956393 learned_addr learned my addr 172.21.15.116:0/1622956393 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:23.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.934+0000 7fc7ea59c640 1 --2- 172.21.15.116:0/1622956393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7ec101420 0x7fc7ec198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:23.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.934+0000 7fc7ead9d640 1 --2- 172.21.15.116:0/1622956393 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7ec100a80 0x7fc7ec1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:23.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.934+0000 7fc7eb59e640 1 -- 172.21.15.116:0/1622956393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7ec101420 msgr2=0x7fc7ec198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:23.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.934+0000 7fc7eb59e640 1 --2- 172.21.15.116:0/1622956393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7ec101420 0x7fc7ec198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:23.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.934+0000 7fc7eb59e640 1 -- 172.21.15.116:0/1622956393 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7ec100a80 msgr2=0x7fc7ec1985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:23.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.934+0000 7fc7eb59e640 1 --2- 172.21.15.116:0/1622956393 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7ec100a80 0x7fc7ec1985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:23.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.934+0000 7fc7eb59e640 1 -- 172.21.15.116:0/1622956393 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ec19d550 con 0x7fc7ec106a80 2024-09-12T03:01:23.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.934+0000 7fc7ead9d640 1 --2- 172.21.15.116:0/1622956393 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7ec100a80 0x7fc7ec1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:01:23.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.934+0000 7fc7eb59e640 1 --2- 172.21.15.116:0/1622956393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc7ec106a80 0x7fc7ec19ce70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4006010 tx=0x7fc7e4003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:23.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.935+0000 7fc7e3fff640 1 -- 172.21.15.116:0/1622956393 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4004450 con 0x7fc7ec106a80 2024-09-12T03:01:23.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.935+0000 7fc7f11dc640 1 -- 172.21.15.116:0/1622956393 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7ec19d7e0 con 0x7fc7ec106a80 2024-09-12T03:01:23.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.935+0000 7fc7e3fff640 1 -- 172.21.15.116:0/1622956393 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7e4003e10 con 0x7fc7ec106a80 2024-09-12T03:01:23.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.935+0000 7fc7e3fff640 1 -- 172.21.15.116:0/1622956393 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e40466e0 con 0x7fc7ec106a80 2024-09-12T03:01:23.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.935+0000 7fc7f11dc640 1 -- 172.21.15.116:0/1622956393 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7ec1a5120 con 0x7fc7ec106a80 2024-09-12T03:01:23.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.936+0000 7fc7f11dc640 1 -- 172.21.15.116:0/1622956393 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7ec100f60 con 0x7fc7ec106a80 2024-09-12T03:01:23.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.937+0000 7fc7e3fff640 1 -- 172.21.15.116:0/1622956393 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc7e40305a0 con 0x7fc7ec106a80 2024-09-12T03:01:23.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.938+0000 7fc7e3fff640 1 --2- 172.21.15.116:0/1622956393 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc7d80a2c10 0x7fc7d80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:23.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.938+0000 7fc7ead9d640 1 --2- 172.21.15.116:0/1622956393 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc7d80a2c10 0x7fc7d80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:23.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.938+0000 7fc7e3fff640 1 -- 172.21.15.116:0/1622956393 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fc7e402e030 con 0x7fc7ec106a80 2024-09-12T03:01:23.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.938+0000 7fc7ead9d640 1 --2- 172.21.15.116:0/1622956393 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc7d80a2c10 0x7fc7d80a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc006fd0 tx=0x7fc7dc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:23.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:23.942+0000 7fc7e3fff640 1 -- 172.21.15.116:0/1622956393 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc7e404b060 con 0x7fc7ec106a80 2024-09-12T03:01:24.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:24.105+0000 7fc7f11dc640 1 -- 172.21.15.116:0/1622956393 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc7ec062630 con 0x7fc7d80a2c10 2024-09-12T03:01:24.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:24.112+0000 7fc7e3fff640 1 -- 172.21.15.116:0/1622956393 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258838 (secure 0 0 0) 0x7fc7ec062630 con 0x7fc7d80a2c10 2024-09-12T03:01:24.115 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:01:24.116 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-09-12T03:01:24.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:24.115+0000 7fc7f11dc640 1 -- 172.21.15.116:0/1622956393 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc7d80a2c10 msgr2=0x7fc7d80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:24.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:24.115+0000 7fc7f11dc640 1 --2- 172.21.15.116:0/1622956393 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc7d80a2c10 0x7fc7d80a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc006fd0 tx=0x7fc7dc008040 comp rx=0 tx=0).stop 2024-09-12T03:01:24.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:24.115+0000 7fc7f11dc640 1 -- 172.21.15.116:0/1622956393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc7ec106a80 msgr2=0x7fc7ec19ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:24.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:24.115+0000 7fc7f11dc640 1 --2- 172.21.15.116:0/1622956393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc7ec106a80 0x7fc7ec19ce70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4006010 tx=0x7fc7e4003c20 comp rx=0 tx=0).stop 2024-09-12T03:01:24.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:24.115+0000 7fc7f11dc640 1 -- 172.21.15.116:0/1622956393 shutdown_connections 2024-09-12T03:01:24.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:24.115+0000 7fc7f11dc640 1 --2- 172.21.15.116:0/1622956393 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc7d80a2c10 0x7fc7d80a50d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:24.115+0000 7fc7f11dc640 1 --2- 172.21.15.116:0/1622956393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc7ec106a80 0x7fc7ec19ce70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:24.115+0000 7fc7f11dc640 1 --2- 172.21.15.116:0/1622956393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7ec101420 0x7fc7ec198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:24.115+0000 7fc7f11dc640 1 --2- 172.21.15.116:0/1622956393 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7ec100a80 0x7fc7ec1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:24.115+0000 7fc7f11dc640 1 -- 172.21.15.116:0/1622956393 >> 172.21.15.116:0/1622956393 conn(0x7fc7ec0fc7a0 msgr2=0x7fc7ec0fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:24.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:24.115+0000 7fc7f11dc640 1 -- 172.21.15.116:0/1622956393 shutdown_connections 2024-09-12T03:01:24.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:24.115+0000 7fc7f11dc640 1 -- 172.21.15.116:0/1622956393 wait complete. 2024-09-12T03:01:24.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:24 smithi183 ceph-mon[35535]: pgmap v288: 97 pgs: 97 active+clean; 626 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-12T03:01:24.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:24 smithi177 ceph-mon[35305]: pgmap v288: 97 pgs: 97 active+clean; 626 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-12T03:01:24.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:24 smithi116 ceph-mon[31410]: pgmap v288: 97 pgs: 97 active+clean; 626 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-09-12T03:01:24.935 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-09-12T03:01:23.112538+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641014,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":225,"num_read_kb":214,"num_write":333,"num_write_kb":3031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":267,"ondisk_log_size":267,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368036,"kb_used_data":10468,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573788,"statfs":{"total":1151940427776,"available":1151563558912,"internally_reserved":0,"allocated":10719232,"data_stored":6888089,"data_compressed":107490,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":45604,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":81,"num_read_kb":90,"num_write":54,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003119"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.348943+0000","last_change":"2024-09-12T03:00:58.348563+0000","last_active":"2024-09-12T03:00:59.348943+0000","last_peered":"2024-09-12T03:00:59.348943+0000","last_clean":"2024-09-12T03:00:59.348943+0000","last_became_active":"2024-09-12T03:00:58.347940+0000","last_became_peered":"2024-09-12T03:00:58.347940+0000","last_unstale":"2024-09-12T03:00:59.348943+0000","last_undegraded":"2024-09-12T03:00:59.348943+0000","last_fullsized":"2024-09-12T03:00:59.348943+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:43.489238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342719+0000","last_change":"2024-09-12T03:00:58.351969+0000","last_active":"2024-09-12T03:00:59.342719+0000","last_peered":"2024-09-12T03:00:59.342719+0000","last_clean":"2024-09-12T03:00:59.342719+0000","last_became_active":"2024-09-12T03:00:58.351679+0000","last_became_peered":"2024-09-12T03:00:58.351679+0000","last_unstale":"2024-09-12T03:00:59.342719+0000","last_undegraded":"2024-09-12T03:00:59.342719+0000","last_fullsized":"2024-09-12T03:00:59.342719+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:27:01.174890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342541+0000","last_change":"2024-09-12T03:00:58.346045+0000","last_active":"2024-09-12T03:00:59.342541+0000","last_peered":"2024-09-12T03:00:59.342541+0000","last_clean":"2024-09-12T03:00:59.342541+0000","last_became_active":"2024-09-12T03:00:58.345719+0000","last_became_peered":"2024-09-12T03:00:58.345719+0000","last_unstale":"2024-09-12T03:00:59.342541+0000","last_undegraded":"2024-09-12T03:00:59.342541+0000","last_fullsized":"2024-09-12T03:00:59.342541+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:12.312650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343947+0000","last_change":"2024-09-12T03:00:58.348165+0000","last_active":"2024-09-12T03:00:59.343947+0000","last_peered":"2024-09-12T03:00:59.343947+0000","last_clean":"2024-09-12T03:00:59.343947+0000","last_became_active":"2024-09-12T03:00:58.347734+0000","last_became_peered":"2024-09-12T03:00:58.347734+0000","last_unstale":"2024-09-12T03:00:59.343947+0000","last_undegraded":"2024-09-12T03:00:59.343947+0000","last_fullsized":"2024-09-12T03:00:59.343947+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:05.945900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343877+0000","last_change":"2024-09-12T03:00:58.339316+0000","last_active":"2024-09-12T03:00:59.343877+0000","last_peered":"2024-09-12T03:00:59.343877+0000","last_clean":"2024-09-12T03:00:59.343877+0000","last_became_active":"2024-09-12T03:00:58.338964+0000","last_became_peered":"2024-09-12T03:00:58.338964+0000","last_unstale":"2024-09-12T03:00:59.343877+0000","last_undegraded":"2024-09-12T03:00:59.343877+0000","last_fullsized":"2024-09-12T03:00:59.343877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:50.721276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342586+0000","last_change":"2024-09-12T03:00:58.346838+0000","last_active":"2024-09-12T03:00:59.342586+0000","last_peered":"2024-09-12T03:00:59.342586+0000","last_clean":"2024-09-12T03:00:59.342586+0000","last_became_active":"2024-09-12T03:00:58.346486+0000","last_became_peered":"2024-09-12T03:00:58.346486+0000","last_unstale":"2024-09-12T03:00:59.342586+0000","last_undegraded":"2024-09-12T03:00:59.342586+0000","last_fullsized":"2024-09-12T03:00:59.342586+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:28.687727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343160+0000","last_change":"2024-09-12T03:00:58.346712+0000","last_active":"2024-09-12T03:00:59.343160+0000","last_peered":"2024-09-12T03:00:59.343160+0000","last_clean":"2024-09-12T03:00:59.343160+0000","last_became_active":"2024-09-12T03:00:58.346397+0000","last_became_peered":"2024-09-12T03:00:58.346397+0000","last_unstale":"2024-09-12T03:00:59.343160+0000","last_undegraded":"2024-09-12T03:00:59.343160+0000","last_fullsized":"2024-09-12T03:00:59.343160+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:48:00.624733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343665+0000","last_change":"2024-09-12T03:00:58.356166+0000","last_active":"2024-09-12T03:00:59.343665+0000","last_peered":"2024-09-12T03:00:59.343665+0000","last_clean":"2024-09-12T03:00:59.343665+0000","last_became_active":"2024-09-12T03:00:58.354944+0000","last_became_peered":"2024-09-12T03:00:58.354944+0000","last_unstale":"2024-09-12T03:00:59.343665+0000","last_undegraded":"2024-09-12T03:00:59.343665+0000","last_fullsized":"2024-09-12T03:00:59.343665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:15.840149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.329171+0000","last_change":"2024-09-12T03:00:58.340182+0000","last_active":"2024-09-12T03:00:59.329171+0000","last_peered":"2024-09-12T03:00:59.329171+0000","last_clean":"2024-09-12T03:00:59.329171+0000","last_became_active":"2024-09-12T03:00:58.339576+0000","last_became_peered":"2024-09-12T03:00:58.339576+0000","last_unstale":"2024-09-12T03:00:59.329171+0000","last_undegraded":"2024-09-12T03:00:59.329171+0000","last_fullsized":"2024-09-12T03:00:59.329171+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:16.534844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344683+0000","last_change":"2024-09-12T03:00:58.351218+0000","last_active":"2024-09-12T03:00:59.344683+0000","last_peered":"2024-09-12T03:00:59.344683+0000","last_clean":"2024-09-12T03:00:59.344683+0000","last_became_active":"2024-09-12T03:00:58.350871+0000","last_became_peered":"2024-09-12T03:00:58.350871+0000","last_unstale":"2024-09-12T03:00:59.344683+0000","last_undegraded":"2024-09-12T03:00:59.344683+0000","last_fullsized":"2024-09-12T03:00:59.344683+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:38.367683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343826+0000","last_change":"2024-09-12T03:00:58.351894+0000","last_active":"2024-09-12T03:00:59.343826+0000","last_peered":"2024-09-12T03:00:59.343826+0000","last_clean":"2024-09-12T03:00:59.343826+0000","last_became_active":"2024-09-12T03:00:58.351445+0000","last_became_peered":"2024-09-12T03:00:58.351445+0000","last_unstale":"2024-09-12T03:00:59.343826+0000","last_undegraded":"2024-09-12T03:00:59.343826+0000","last_fullsized":"2024-09-12T03:00:59.343826+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:42.093871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344004+0000","last_change":"2024-09-12T03:00:58.353460+0000","last_active":"2024-09-12T03:00:59.344004+0000","last_peered":"2024-09-12T03:00:59.344004+0000","last_clean":"2024-09-12T03:00:59.344004+0000","last_became_active":"2024-09-12T03:00:58.353111+0000","last_became_peered":"2024-09-12T03:00:58.353111+0000","last_unstale":"2024-09-12T03:00:59.344004+0000","last_undegraded":"2024-09-12T03:00:59.344004+0000","last_fullsized":"2024-09-12T03:00:59.344004+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:15.916765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.330518+0000","last_change":"2024-09-12T03:00:58.341080+0000","last_active":"2024-09-12T03:00:59.330518+0000","last_peered":"2024-09-12T03:00:59.330518+0000","last_clean":"2024-09-12T03:00:59.330518+0000","last_became_active":"2024-09-12T03:00:58.340782+0000","last_became_peered":"2024-09-12T03:00:58.340782+0000","last_unstale":"2024-09-12T03:00:59.330518+0000","last_undegraded":"2024-09-12T03:00:59.330518+0000","last_fullsized":"2024-09-12T03:00:59.330518+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:36.569700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342701+0000","last_change":"2024-09-12T03:00:58.346791+0000","last_active":"2024-09-12T03:00:59.342701+0000","last_peered":"2024-09-12T03:00:59.342701+0000","last_clean":"2024-09-12T03:00:59.342701+0000","last_became_active":"2024-09-12T03:00:58.346500+0000","last_became_peered":"2024-09-12T03:00:58.346500+0000","last_unstale":"2024-09-12T03:00:59.342701+0000","last_undegraded":"2024-09-12T03:00:59.342701+0000","last_fullsized":"2024-09-12T03:00:59.342701+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:32.017589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342415+0000","last_change":"2024-09-12T03:00:58.349522+0000","last_active":"2024-09-12T03:00:59.342415+0000","last_peered":"2024-09-12T03:00:59.342415+0000","last_clean":"2024-09-12T03:00:59.342415+0000","last_became_active":"2024-09-12T03:00:58.349246+0000","last_became_peered":"2024-09-12T03:00:58.349246+0000","last_unstale":"2024-09-12T03:00:59.342415+0000","last_undegraded":"2024-09-12T03:00:59.342415+0000","last_fullsized":"2024-09-12T03:00:59.342415+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:39.253919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328399+0000","last_change":"2024-09-12T03:00:58.349909+0000","last_active":"2024-09-12T03:00:59.328399+0000","last_peered":"2024-09-12T03:00:59.328399+0000","last_clean":"2024-09-12T03:00:59.328399+0000","last_became_active":"2024-09-12T03:00:58.349590+0000","last_became_peered":"2024-09-12T03:00:58.349590+0000","last_unstale":"2024-09-12T03:00:59.328399+0000","last_undegraded":"2024-09-12T03:00:59.328399+0000","last_fullsized":"2024-09-12T03:00:59.328399+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:04.540709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.349104+0000","last_change":"2024-09-12T03:00:58.348284+0000","last_active":"2024-09-12T03:00:59.349104+0000","last_peered":"2024-09-12T03:00:59.349104+0000","last_clean":"2024-09-12T03:00:59.349104+0000","last_became_active":"2024-09-12T03:00:58.347465+0000","last_became_peered":"2024-09-12T03:00:58.347465+0000","last_unstale":"2024-09-12T03:00:59.349104+0000","last_undegraded":"2024-09-12T03:00:59.349104+0000","last_fullsized":"2024-09-12T03:00:59.349104+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:13.191899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343137+0000","last_change":"2024-09-12T03:00:58.337740+0000","last_active":"2024-09-12T03:00:59.343137+0000","last_peered":"2024-09-12T03:00:59.343137+0000","last_clean":"2024-09-12T03:00:59.343137+0000","last_became_active":"2024-09-12T03:00:58.337393+0000","last_became_peered":"2024-09-12T03:00:58.337393+0000","last_unstale":"2024-09-12T03:00:59.343137+0000","last_undegraded":"2024-09-12T03:00:59.343137+0000","last_fullsized":"2024-09-12T03:00:59.343137+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:28:56.339167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344750+0000","last_change":"2024-09-12T03:00:58.345779+0000","last_active":"2024-09-12T03:00:59.344750+0000","last_peered":"2024-09-12T03:00:59.344750+0000","last_clean":"2024-09-12T03:00:59.344750+0000","last_became_active":"2024-09-12T03:00:58.345210+0000","last_became_peered":"2024-09-12T03:00:58.345210+0000","last_unstale":"2024-09-12T03:00:59.344750+0000","last_undegraded":"2024-09-12T03:00:59.344750+0000","last_fullsized":"2024-09-12T03:00:59.344750+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:06:58.271185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.348332+0000","last_change":"2024-09-12T03:00:58.343877+0000","last_active":"2024-09-12T03:00:59.348332+0000","last_peered":"2024-09-12T03:00:59.348332+0000","last_clean":"2024-09-12T03:00:59.348332+0000","last_became_active":"2024-09-12T03:00:58.343545+0000","last_became_peered":"2024-09-12T03:00:58.343545+0000","last_unstale":"2024-09-12T03:00:59.348332+0000","last_undegraded":"2024-09-12T03:00:59.348332+0000","last_fullsized":"2024-09-12T03:00:59.348332+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:52:31.851975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.330596+0000","last_change":"2024-09-12T03:00:58.349233+0000","last_active":"2024-09-12T03:00:59.330596+0000","last_peered":"2024-09-12T03:00:59.330596+0000","last_clean":"2024-09-12T03:00:59.330596+0000","last_became_active":"2024-09-12T03:00:58.348958+0000","last_became_peered":"2024-09-12T03:00:58.348958+0000","last_unstale":"2024-09-12T03:00:59.330596+0000","last_undegraded":"2024-09-12T03:00:59.330596+0000","last_fullsized":"2024-09-12T03:00:59.330596+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:32:10.271817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328487+0000","last_change":"2024-09-12T03:00:58.358024+0000","last_active":"2024-09-12T03:00:59.328487+0000","last_peered":"2024-09-12T03:00:59.328487+0000","last_clean":"2024-09-12T03:00:59.328487+0000","last_became_active":"2024-09-12T03:00:58.357434+0000","last_became_peered":"2024-09-12T03:00:58.357434+0000","last_unstale":"2024-09-12T03:00:59.328487+0000","last_undegraded":"2024-09-12T03:00:59.328487+0000","last_fullsized":"2024-09-12T03:00:59.328487+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:01:20.119733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.345193+0000","last_change":"2024-09-12T03:00:58.355063+0000","last_active":"2024-09-12T03:00:59.345193+0000","last_peered":"2024-09-12T03:00:59.345193+0000","last_clean":"2024-09-12T03:00:59.345193+0000","last_became_active":"2024-09-12T03:00:58.354760+0000","last_became_peered":"2024-09-12T03:00:58.354760+0000","last_unstale":"2024-09-12T03:00:59.345193+0000","last_undegraded":"2024-09-12T03:00:59.345193+0000","last_fullsized":"2024-09-12T03:00:59.345193+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:13.464413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328351+0000","last_change":"2024-09-12T03:00:58.341916+0000","last_active":"2024-09-12T03:00:59.328351+0000","last_peered":"2024-09-12T03:00:59.328351+0000","last_clean":"2024-09-12T03:00:59.328351+0000","last_became_active":"2024-09-12T03:00:58.340815+0000","last_became_peered":"2024-09-12T03:00:58.340815+0000","last_unstale":"2024-09-12T03:00:59.328351+0000","last_undegraded":"2024-09-12T03:00:59.328351+0000","last_fullsized":"2024-09-12T03:00:59.328351+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:13.777947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343925+0000","last_change":"2024-09-12T03:00:58.357765+0000","last_active":"2024-09-12T03:00:59.343925+0000","last_peered":"2024-09-12T03:00:59.343925+0000","last_clean":"2024-09-12T03:00:59.343925+0000","last_became_active":"2024-09-12T03:00:58.357480+0000","last_became_peered":"2024-09-12T03:00:58.357480+0000","last_unstale":"2024-09-12T03:00:59.343925+0000","last_undegraded":"2024-09-12T03:00:59.343925+0000","last_fullsized":"2024-09-12T03:00:59.343925+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:26.884381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344699+0000","last_change":"2024-09-12T03:00:58.349504+0000","last_active":"2024-09-12T03:00:59.344699+0000","last_peered":"2024-09-12T03:00:59.344699+0000","last_clean":"2024-09-12T03:00:59.344699+0000","last_became_active":"2024-09-12T03:00:58.349136+0000","last_became_peered":"2024-09-12T03:00:58.349136+0000","last_unstale":"2024-09-12T03:00:59.344699+0000","last_undegraded":"2024-09-12T03:00:59.344699+0000","last_fullsized":"2024-09-12T03:00:59.344699+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:09:55.229516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.348719+0000","last_change":"2024-09-12T03:00:58.341902+0000","last_active":"2024-09-12T03:00:59.348719+0000","last_peered":"2024-09-12T03:00:59.348719+0000","last_clean":"2024-09-12T03:00:59.348719+0000","last_became_active":"2024-09-12T03:00:58.341592+0000","last_became_peered":"2024-09-12T03:00:58.341592+0000","last_unstale":"2024-09-12T03:00:59.348719+0000","last_undegraded":"2024-09-12T03:00:59.348719+0000","last_fullsized":"2024-09-12T03:00:59.348719+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:13.852511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.330540+0000","last_change":"2024-09-12T03:00:58.348012+0000","last_active":"2024-09-12T03:00:59.330540+0000","last_peered":"2024-09-12T03:00:59.330540+0000","last_clean":"2024-09-12T03:00:59.330540+0000","last_became_active":"2024-09-12T03:00:58.347338+0000","last_became_peered":"2024-09-12T03:00:58.347338+0000","last_unstale":"2024-09-12T03:00:59.330540+0000","last_undegraded":"2024-09-12T03:00:59.330540+0000","last_fullsized":"2024-09-12T03:00:59.330540+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:41.478032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342928+0000","last_change":"2024-09-12T03:00:58.345833+0000","last_active":"2024-09-12T03:00:59.342928+0000","last_peered":"2024-09-12T03:00:59.342928+0000","last_clean":"2024-09-12T03:00:59.342928+0000","last_became_active":"2024-09-12T03:00:58.345482+0000","last_became_peered":"2024-09-12T03:00:58.345482+0000","last_unstale":"2024-09-12T03:00:59.342928+0000","last_undegraded":"2024-09-12T03:00:59.342928+0000","last_fullsized":"2024-09-12T03:00:59.342928+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:32.037412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343059+0000","last_change":"2024-09-12T03:00:58.352290+0000","last_active":"2024-09-12T03:00:59.343059+0000","last_peered":"2024-09-12T03:00:59.343059+0000","last_clean":"2024-09-12T03:00:59.343059+0000","last_became_active":"2024-09-12T03:00:58.351918+0000","last_became_peered":"2024-09-12T03:00:58.351918+0000","last_unstale":"2024-09-12T03:00:59.343059+0000","last_undegraded":"2024-09-12T03:00:59.343059+0000","last_fullsized":"2024-09-12T03:00:59.343059+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:25.941080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343279+0000","last_change":"2024-09-12T03:00:58.348021+0000","last_active":"2024-09-12T03:00:59.343279+0000","last_peered":"2024-09-12T03:00:59.343279+0000","last_clean":"2024-09-12T03:00:59.343279+0000","last_became_active":"2024-09-12T03:00:58.347634+0000","last_became_peered":"2024-09-12T03:00:58.347634+0000","last_unstale":"2024-09-12T03:00:59.343279+0000","last_undegraded":"2024-09-12T03:00:59.343279+0000","last_fullsized":"2024-09-12T03:00:59.343279+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:47.933067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.347866+0000","last_change":"2024-09-12T03:00:58.340891+0000","last_active":"2024-09-12T03:00:59.347866+0000","last_peered":"2024-09-12T03:00:59.347866+0000","last_clean":"2024-09-12T03:00:59.347866+0000","last_became_active":"2024-09-12T03:00:58.340504+0000","last_became_peered":"2024-09-12T03:00:58.340504+0000","last_unstale":"2024-09-12T03:00:59.347866+0000","last_undegraded":"2024-09-12T03:00:59.347866+0000","last_fullsized":"2024-09-12T03:00:59.347866+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:32.117635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343541+0000","last_change":"2024-09-12T03:00:58.339343+0000","last_active":"2024-09-12T03:00:59.343541+0000","last_peered":"2024-09-12T03:00:59.343541+0000","last_clean":"2024-09-12T03:00:59.343541+0000","last_became_active":"2024-09-12T03:00:58.339001+0000","last_became_peered":"2024-09-12T03:00:58.339001+0000","last_unstale":"2024-09-12T03:00:59.343541+0000","last_undegraded":"2024-09-12T03:00:59.343541+0000","last_fullsized":"2024-09-12T03:00:59.343541+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:02.043454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:17.414797+0000","last_change":"2024-09-12T03:00:56.315550+0000","last_active":"2024-09-12T03:01:17.414797+0000","last_peered":"2024-09-12T03:01:17.414797+0000","last_clean":"2024-09-12T03:01:17.414797+0000","last_became_active":"2024-09-12T03:00:56.315225+0000","last_became_peered":"2024-09-12T03:00:56.315225+0000","last_unstale":"2024-09-12T03:01:17.414797+0000","last_undegraded":"2024-09-12T03:01:17.414797+0000","last_fullsized":"2024-09-12T03:01:17.414797+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:46.070895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328646+0000","last_change":"2024-09-12T03:00:58.338379+0000","last_active":"2024-09-12T03:00:59.328646+0000","last_peered":"2024-09-12T03:00:59.328646+0000","last_clean":"2024-09-12T03:00:59.328646+0000","last_became_active":"2024-09-12T03:00:58.337795+0000","last_became_peered":"2024-09-12T03:00:58.337795+0000","last_unstale":"2024-09-12T03:00:59.328646+0000","last_undegraded":"2024-09-12T03:00:59.328646+0000","last_fullsized":"2024-09-12T03:00:59.328646+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:43.299329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.397617+0000","last_change":"2024-09-12T03:00:56.316385+0000","last_active":"2024-09-12T03:01:07.397617+0000","last_peered":"2024-09-12T03:01:07.397617+0000","last_clean":"2024-09-12T03:01:07.397617+0000","last_became_active":"2024-09-12T03:00:56.316064+0000","last_became_peered":"2024-09-12T03:00:56.316064+0000","last_unstale":"2024-09-12T03:01:07.397617+0000","last_undegraded":"2024-09-12T03:01:07.397617+0000","last_fullsized":"2024-09-12T03:01:07.397617+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:12:43.323654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.397886+0000","last_change":"2024-09-12T03:00:56.313065+0000","last_active":"2024-09-12T03:01:07.397886+0000","last_peered":"2024-09-12T03:01:07.397886+0000","last_clean":"2024-09-12T03:01:07.397886+0000","last_became_active":"2024-09-12T03:00:56.312716+0000","last_became_peered":"2024-09-12T03:00:56.312716+0000","last_unstale":"2024-09-12T03:01:07.397886+0000","last_undegraded":"2024-09-12T03:01:07.397886+0000","last_fullsized":"2024-09-12T03:01:07.397886+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:51.089089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.346221+0000","last_change":"2024-09-12T03:00:58.346542+0000","last_active":"2024-09-12T03:00:59.346221+0000","last_peered":"2024-09-12T03:00:59.346221+0000","last_clean":"2024-09-12T03:00:59.346221+0000","last_became_active":"2024-09-12T03:00:58.346260+0000","last_became_peered":"2024-09-12T03:00:58.346260+0000","last_unstale":"2024-09-12T03:00:59.346221+0000","last_undegraded":"2024-09-12T03:00:59.346221+0000","last_fullsized":"2024-09-12T03:00:59.346221+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:37.034969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328738+0000","last_change":"2024-09-12T03:00:56.317308+0000","last_active":"2024-09-12T03:00:59.328738+0000","last_peered":"2024-09-12T03:00:59.328738+0000","last_clean":"2024-09-12T03:00:59.328738+0000","last_became_active":"2024-09-12T03:00:56.316935+0000","last_became_peered":"2024-09-12T03:00:56.316935+0000","last_unstale":"2024-09-12T03:00:59.328738+0000","last_undegraded":"2024-09-12T03:00:59.328738+0000","last_fullsized":"2024-09-12T03:00:59.328738+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:06.324489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343312+0000","last_change":"2024-09-12T03:00:58.341565+0000","last_active":"2024-09-12T03:00:59.343312+0000","last_peered":"2024-09-12T03:00:59.343312+0000","last_clean":"2024-09-12T03:00:59.343312+0000","last_became_active":"2024-09-12T03:00:58.341285+0000","last_became_peered":"2024-09-12T03:00:58.341285+0000","last_unstale":"2024-09-12T03:00:59.343312+0000","last_undegraded":"2024-09-12T03:00:59.343312+0000","last_fullsized":"2024-09-12T03:00:59.343312+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:59.156700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:19.417025+0000","last_change":"2024-09-12T03:00:56.314201+0000","last_active":"2024-09-12T03:01:19.417025+0000","last_peered":"2024-09-12T03:01:19.417025+0000","last_clean":"2024-09-12T03:01:19.417025+0000","last_became_active":"2024-09-12T03:00:56.313880+0000","last_became_peered":"2024-09-12T03:00:56.313880+0000","last_unstale":"2024-09-12T03:01:19.417025+0000","last_undegraded":"2024-09-12T03:01:19.417025+0000","last_fullsized":"2024-09-12T03:01:19.417025+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:57:12.952630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.329144+0000","last_change":"2024-09-12T03:00:58.345274+0000","last_active":"2024-09-12T03:00:59.329144+0000","last_peered":"2024-09-12T03:00:59.329144+0000","last_clean":"2024-09-12T03:00:59.329144+0000","last_became_active":"2024-09-12T03:00:58.344937+0000","last_became_peered":"2024-09-12T03:00:58.344937+0000","last_unstale":"2024-09-12T03:00:59.329144+0000","last_undegraded":"2024-09-12T03:00:59.329144+0000","last_fullsized":"2024-09-12T03:00:59.329144+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:17.288909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:06.390277+0000","last_change":"2024-09-12T03:00:56.314709+0000","last_active":"2024-09-12T03:01:06.390277+0000","last_peered":"2024-09-12T03:01:06.390277+0000","last_clean":"2024-09-12T03:01:06.390277+0000","last_became_active":"2024-09-12T03:00:56.314349+0000","last_became_peered":"2024-09-12T03:00:56.314349+0000","last_unstale":"2024-09-12T03:01:06.390277+0000","last_undegraded":"2024-09-12T03:01:06.390277+0000","last_fullsized":"2024-09-12T03:01:06.390277+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:18:35.371661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343140+0000","last_change":"2024-09-12T03:00:58.346055+0000","last_active":"2024-09-12T03:00:59.343140+0000","last_peered":"2024-09-12T03:00:59.343140+0000","last_clean":"2024-09-12T03:00:59.343140+0000","last_became_active":"2024-09-12T03:00:58.345701+0000","last_became_peered":"2024-09-12T03:00:58.345701+0000","last_unstale":"2024-09-12T03:00:59.343140+0000","last_undegraded":"2024-09-12T03:00:59.343140+0000","last_fullsized":"2024-09-12T03:00:59.343140+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:33.977245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:06.391491+0000","last_change":"2024-09-12T03:00:56.435962+0000","last_active":"2024-09-12T03:01:06.391491+0000","last_peered":"2024-09-12T03:01:06.391491+0000","last_clean":"2024-09-12T03:01:06.391491+0000","last_became_active":"2024-09-12T03:00:56.435515+0000","last_became_peered":"2024-09-12T03:00:56.435515+0000","last_unstale":"2024-09-12T03:01:06.391491+0000","last_undegraded":"2024-09-12T03:01:06.391491+0000","last_fullsized":"2024-09-12T03:01:06.391491+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:32.953436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328662+0000","last_change":"2024-09-12T03:00:58.344239+0000","last_active":"2024-09-12T03:00:59.328662+0000","last_peered":"2024-09-12T03:00:59.328662+0000","last_clean":"2024-09-12T03:00:59.328662+0000","last_became_active":"2024-09-12T03:00:58.343876+0000","last_became_peered":"2024-09-12T03:00:58.343876+0000","last_unstale":"2024-09-12T03:00:59.328662+0000","last_undegraded":"2024-09-12T03:00:59.328662+0000","last_fullsized":"2024-09-12T03:00:59.328662+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:49.401295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'22","reported_seq":72,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:20.419578+0000","last_change":"2024-09-12T03:00:56.317708+0000","last_active":"2024-09-12T03:01:20.419578+0000","last_peered":"2024-09-12T03:01:20.419578+0000","last_clean":"2024-09-12T03:01:20.419578+0000","last_became_active":"2024-09-12T03:00:56.317317+0000","last_became_peered":"2024-09-12T03:00:56.317317+0000","last_unstale":"2024-09-12T03:01:20.419578+0000","last_undegraded":"2024-09-12T03:01:20.419578+0000","last_fullsized":"2024-09-12T03:01:20.419578+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:47.788186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46802,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":61,"num_write":23,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.346470+0000","last_change":"2024-09-12T03:00:58.343602+0000","last_active":"2024-09-12T03:00:59.346470+0000","last_peered":"2024-09-12T03:00:59.346470+0000","last_clean":"2024-09-12T03:00:59.346470+0000","last_became_active":"2024-09-12T03:00:58.343301+0000","last_became_peered":"2024-09-12T03:00:58.343301+0000","last_unstale":"2024-09-12T03:00:59.346470+0000","last_undegraded":"2024-09-12T03:00:59.346470+0000","last_fullsized":"2024-09-12T03:00:59.346470+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:04.056118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.398339+0000","last_change":"2024-09-12T03:00:56.317874+0000","last_active":"2024-09-12T03:01:07.398339+0000","last_peered":"2024-09-12T03:01:07.398339+0000","last_clean":"2024-09-12T03:01:07.398339+0000","last_became_active":"2024-09-12T03:00:56.317566+0000","last_became_peered":"2024-09-12T03:00:56.317566+0000","last_unstale":"2024-09-12T03:01:07.398339+0000","last_undegraded":"2024-09-12T03:01:07.398339+0000","last_fullsized":"2024-09-12T03:01:07.398339+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:54.203730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.330344+0000","last_change":"2024-09-12T03:00:58.351539+0000","last_active":"2024-09-12T03:00:59.330344+0000","last_peered":"2024-09-12T03:00:59.330344+0000","last_clean":"2024-09-12T03:00:59.330344+0000","last_became_active":"2024-09-12T03:00:58.351249+0000","last_became_peered":"2024-09-12T03:00:58.351249+0000","last_unstale":"2024-09-12T03:00:59.330344+0000","last_undegraded":"2024-09-12T03:00:59.330344+0000","last_fullsized":"2024-09-12T03:00:59.330344+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:40.498867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:06.390652+0000","last_change":"2024-09-12T03:00:56.314489+0000","last_active":"2024-09-12T03:01:06.390652+0000","last_peered":"2024-09-12T03:01:06.390652+0000","last_clean":"2024-09-12T03:01:06.390652+0000","last_became_active":"2024-09-12T03:00:56.314142+0000","last_became_peered":"2024-09-12T03:00:56.314142+0000","last_unstale":"2024-09-12T03:01:06.390652+0000","last_undegraded":"2024-09-12T03:01:06.390652+0000","last_fullsized":"2024-09-12T03:01:06.390652+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:21.541713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343350+0000","last_change":"2024-09-12T03:00:58.348752+0000","last_active":"2024-09-12T03:00:59.343350+0000","last_peered":"2024-09-12T03:00:59.343350+0000","last_clean":"2024-09-12T03:00:59.343350+0000","last_became_active":"2024-09-12T03:00:58.348391+0000","last_became_peered":"2024-09-12T03:00:58.348391+0000","last_unstale":"2024-09-12T03:00:59.343350+0000","last_undegraded":"2024-09-12T03:00:59.343350+0000","last_fullsized":"2024-09-12T03:00:59.343350+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:33:15.626306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'3","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:18.415644+0000","last_change":"2024-09-12T03:00:56.311702+0000","last_active":"2024-09-12T03:01:18.415644+0000","last_peered":"2024-09-12T03:01:18.415644+0000","last_clean":"2024-09-12T03:01:18.415644+0000","last_became_active":"2024-09-12T03:00:56.311353+0000","last_became_peered":"2024-09-12T03:00:56.311353+0000","last_unstale":"2024-09-12T03:01:18.415644+0000","last_undegraded":"2024-09-12T03:01:18.415644+0000","last_fullsized":"2024-09-12T03:01:18.415644+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:01.004858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.345441+0000","last_change":"2024-09-12T03:00:58.351973+0000","last_active":"2024-09-12T03:00:59.345441+0000","last_peered":"2024-09-12T03:00:59.345441+0000","last_clean":"2024-09-12T03:00:59.345441+0000","last_became_active":"2024-09-12T03:00:58.351599+0000","last_became_peered":"2024-09-12T03:00:58.351599+0000","last_unstale":"2024-09-12T03:00:59.345441+0000","last_undegraded":"2024-09-12T03:00:59.345441+0000","last_fullsized":"2024-09-12T03:00:59.345441+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:26.398828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.399761+0000","last_change":"2024-09-12T03:00:56.436185+0000","last_active":"2024-09-12T03:01:07.399761+0000","last_peered":"2024-09-12T03:01:07.399761+0000","last_clean":"2024-09-12T03:01:07.399761+0000","last_became_active":"2024-09-12T03:00:56.435717+0000","last_became_peered":"2024-09-12T03:00:56.435717+0000","last_unstale":"2024-09-12T03:01:07.399761+0000","last_undegraded":"2024-09-12T03:01:07.399761+0000","last_fullsized":"2024-09-12T03:01:07.399761+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:27.303182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328571+0000","last_change":"2024-09-12T03:00:58.346851+0000","last_active":"2024-09-12T03:00:59.328571+0000","last_peered":"2024-09-12T03:00:59.328571+0000","last_clean":"2024-09-12T03:00:59.328571+0000","last_became_active":"2024-09-12T03:00:58.346490+0000","last_became_peered":"2024-09-12T03:00:58.346490+0000","last_unstale":"2024-09-12T03:00:59.328571+0000","last_undegraded":"2024-09-12T03:00:59.328571+0000","last_fullsized":"2024-09-12T03:00:59.328571+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:34:27.775668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:08.394235+0000","last_change":"2024-09-12T03:00:56.311333+0000","last_active":"2024-09-12T03:01:08.394235+0000","last_peered":"2024-09-12T03:01:08.394235+0000","last_clean":"2024-09-12T03:01:08.394235+0000","last_became_active":"2024-09-12T03:00:56.310871+0000","last_became_peered":"2024-09-12T03:00:56.310871+0000","last_unstale":"2024-09-12T03:01:08.394235+0000","last_undegraded":"2024-09-12T03:01:08.394235+0000","last_fullsized":"2024-09-12T03:01:08.394235+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:59.344572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.348769+0000","last_change":"2024-09-12T03:00:58.349401+0000","last_active":"2024-09-12T03:00:59.348769+0000","last_peered":"2024-09-12T03:00:59.348769+0000","last_clean":"2024-09-12T03:00:59.348769+0000","last_became_active":"2024-09-12T03:00:58.348996+0000","last_became_peered":"2024-09-12T03:00:58.348996+0000","last_unstale":"2024-09-12T03:00:59.348769+0000","last_undegraded":"2024-09-12T03:00:59.348769+0000","last_fullsized":"2024-09-12T03:00:59.348769+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:20:26.201684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344392+0000","last_change":"2024-09-12T03:00:56.314182+0000","last_active":"2024-09-12T03:00:59.344392+0000","last_peered":"2024-09-12T03:00:59.344392+0000","last_clean":"2024-09-12T03:00:59.344392+0000","last_became_active":"2024-09-12T03:00:56.313749+0000","last_became_peered":"2024-09-12T03:00:56.313749+0000","last_unstale":"2024-09-12T03:00:59.344392+0000","last_undegraded":"2024-09-12T03:00:59.344392+0000","last_fullsized":"2024-09-12T03:00:59.344392+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:31:45.767500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343307+0000","last_change":"2024-09-12T03:00:58.344356+0000","last_active":"2024-09-12T03:00:59.343307+0000","last_peered":"2024-09-12T03:00:59.343307+0000","last_clean":"2024-09-12T03:00:59.343307+0000","last_became_active":"2024-09-12T03:00:58.344035+0000","last_became_peered":"2024-09-12T03:00:58.344035+0000","last_unstale":"2024-09-12T03:00:59.343307+0000","last_undegraded":"2024-09-12T03:00:59.343307+0000","last_fullsized":"2024-09-12T03:00:59.343307+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:45:18.681018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.347000+0000","last_change":"2024-09-12T03:00:56.315653+0000","last_active":"2024-09-12T03:00:59.347000+0000","last_peered":"2024-09-12T03:00:59.347000+0000","last_clean":"2024-09-12T03:00:59.347000+0000","last_became_active":"2024-09-12T03:00:56.315284+0000","last_became_peered":"2024-09-12T03:00:56.315284+0000","last_unstale":"2024-09-12T03:00:59.347000+0000","last_undegraded":"2024-09-12T03:00:59.347000+0000","last_fullsized":"2024-09-12T03:00:59.347000+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:42.934020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343195+0000","last_change":"2024-09-12T03:00:58.357792+0000","last_active":"2024-09-12T03:00:59.343195+0000","last_peered":"2024-09-12T03:00:59.343195+0000","last_clean":"2024-09-12T03:00:59.343195+0000","last_became_active":"2024-09-12T03:00:58.357451+0000","last_became_peered":"2024-09-12T03:00:58.357451+0000","last_unstale":"2024-09-12T03:00:59.343195+0000","last_undegraded":"2024-09-12T03:00:59.343195+0000","last_fullsized":"2024-09-12T03:00:59.343195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:20.582607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.398109+0000","last_change":"2024-09-12T03:00:56.746780+0000","last_active":"2024-09-12T03:01:07.398109+0000","last_peered":"2024-09-12T03:01:07.398109+0000","last_clean":"2024-09-12T03:01:07.398109+0000","last_became_active":"2024-09-12T03:00:56.746414+0000","last_became_peered":"2024-09-12T03:00:56.746414+0000","last_unstale":"2024-09-12T03:01:07.398109+0000","last_undegraded":"2024-09-12T03:01:07.398109+0000","last_fullsized":"2024-09-12T03:01:07.398109+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:15:35.214196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343974+0000","last_change":"2024-09-12T03:00:58.351713+0000","last_active":"2024-09-12T03:00:59.343974+0000","last_peered":"2024-09-12T03:00:59.343974+0000","last_clean":"2024-09-12T03:00:59.343974+0000","last_became_active":"2024-09-12T03:00:58.351371+0000","last_became_peered":"2024-09-12T03:00:58.351371+0000","last_unstale":"2024-09-12T03:00:59.343974+0000","last_undegraded":"2024-09-12T03:00:59.343974+0000","last_fullsized":"2024-09-12T03:00:59.343974+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:13.346956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'137","reported_seq":331,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.345546+0000","last_change":"2024-09-12T02:59:45.461482+0000","last_active":"2024-09-12T03:00:59.345546+0000","last_peered":"2024-09-12T03:00:59.345546+0000","last_clean":"2024-09-12T03:00:59.345546+0000","last_became_active":"2024-09-12T02:59:45.461058+0000","last_became_peered":"2024-09-12T02:59:45.461058+0000","last_unstale":"2024-09-12T03:00:59.345546+0000","last_undegraded":"2024-09-12T03:00:59.345546+0000","last_fullsized":"2024-09-12T03:00:59.345546+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:18.034438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:18.034438+0000","last_clean_scrub_stamp":"2024-09-12T02:56:18.034438+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:09.251858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:08.398498+0000","last_change":"2024-09-12T03:00:56.315245+0000","last_active":"2024-09-12T03:01:08.398498+0000","last_peered":"2024-09-12T03:01:08.398498+0000","last_clean":"2024-09-12T03:01:08.398498+0000","last_became_active":"2024-09-12T03:00:56.314887+0000","last_became_peered":"2024-09-12T03:00:56.314887+0000","last_unstale":"2024-09-12T03:01:08.398498+0000","last_undegraded":"2024-09-12T03:01:08.398498+0000","last_fullsized":"2024-09-12T03:01:08.398498+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:48.750657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344703+0000","last_change":"2024-09-12T03:00:58.357874+0000","last_active":"2024-09-12T03:00:59.344703+0000","last_peered":"2024-09-12T03:00:59.344703+0000","last_clean":"2024-09-12T03:00:59.344703+0000","last_became_active":"2024-09-12T03:00:58.357575+0000","last_became_peered":"2024-09-12T03:00:58.357575+0000","last_unstale":"2024-09-12T03:00:59.344703+0000","last_undegraded":"2024-09-12T03:00:59.344703+0000","last_fullsized":"2024-09-12T03:00:59.344703+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:46.125620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.398754+0000","last_change":"2024-09-12T03:00:56.312263+0000","last_active":"2024-09-12T03:01:07.398754+0000","last_peered":"2024-09-12T03:01:07.398754+0000","last_clean":"2024-09-12T03:01:07.398754+0000","last_became_active":"2024-09-12T03:00:56.311894+0000","last_became_peered":"2024-09-12T03:00:56.311894+0000","last_unstale":"2024-09-12T03:01:07.398754+0000","last_undegraded":"2024-09-12T03:01:07.398754+0000","last_fullsized":"2024-09-12T03:01:07.398754+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:52.434203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.330287+0000","last_change":"2024-09-12T03:00:58.350762+0000","last_active":"2024-09-12T03:00:59.330287+0000","last_peered":"2024-09-12T03:00:59.330287+0000","last_clean":"2024-09-12T03:00:59.330287+0000","last_became_active":"2024-09-12T03:00:58.348038+0000","last_became_peered":"2024-09-12T03:00:58.348038+0000","last_unstale":"2024-09-12T03:00:59.330287+0000","last_undegraded":"2024-09-12T03:00:59.330287+0000","last_fullsized":"2024-09-12T03:00:59.330287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:08.916636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:19.416442+0000","last_change":"2024-09-12T03:00:56.435641+0000","last_active":"2024-09-12T03:01:19.416442+0000","last_peered":"2024-09-12T03:01:19.416442+0000","last_clean":"2024-09-12T03:01:19.416442+0000","last_became_active":"2024-09-12T03:00:56.435319+0000","last_became_peered":"2024-09-12T03:00:56.435319+0000","last_unstale":"2024-09-12T03:01:19.416442+0000","last_undegraded":"2024-09-12T03:01:19.416442+0000","last_fullsized":"2024-09-12T03:01:19.416442+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:56.894978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328360+0000","last_change":"2024-09-12T03:00:58.344708+0000","last_active":"2024-09-12T03:00:59.328360+0000","last_peered":"2024-09-12T03:00:59.328360+0000","last_clean":"2024-09-12T03:00:59.328360+0000","last_became_active":"2024-09-12T03:00:58.344389+0000","last_became_peered":"2024-09-12T03:00:58.344389+0000","last_unstale":"2024-09-12T03:00:59.328360+0000","last_undegraded":"2024-09-12T03:00:59.328360+0000","last_fullsized":"2024-09-12T03:00:59.328360+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:08.160383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343624+0000","last_change":"2024-09-12T03:00:56.317068+0000","last_active":"2024-09-12T03:00:59.343624+0000","last_peered":"2024-09-12T03:00:59.343624+0000","last_clean":"2024-09-12T03:00:59.343624+0000","last_became_active":"2024-09-12T03:00:56.316698+0000","last_became_peered":"2024-09-12T03:00:56.316698+0000","last_unstale":"2024-09-12T03:00:59.343624+0000","last_undegraded":"2024-09-12T03:00:59.343624+0000","last_fullsized":"2024-09-12T03:00:59.343624+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:41:19.199418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343531+0000","last_change":"2024-09-12T03:00:58.347814+0000","last_active":"2024-09-12T03:00:59.343531+0000","last_peered":"2024-09-12T03:00:59.343531+0000","last_clean":"2024-09-12T03:00:59.343531+0000","last_became_active":"2024-09-12T03:00:58.347112+0000","last_became_peered":"2024-09-12T03:00:58.347112+0000","last_unstale":"2024-09-12T03:00:59.343531+0000","last_undegraded":"2024-09-12T03:00:59.343531+0000","last_fullsized":"2024-09-12T03:00:59.343531+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:59.325113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342760+0000","last_change":"2024-09-12T03:00:56.435771+0000","last_active":"2024-09-12T03:00:59.342760+0000","last_peered":"2024-09-12T03:00:59.342760+0000","last_clean":"2024-09-12T03:00:59.342760+0000","last_became_active":"2024-09-12T03:00:56.435448+0000","last_became_peered":"2024-09-12T03:00:56.435448+0000","last_unstale":"2024-09-12T03:00:59.342760+0000","last_undegraded":"2024-09-12T03:00:59.342760+0000","last_fullsized":"2024-09-12T03:00:59.342760+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:33.650338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:01.118993+0000","last_change":"2024-09-12T03:00:58.341071+0000","last_active":"2024-09-12T03:01:01.118993+0000","last_peered":"2024-09-12T03:01:01.118993+0000","last_clean":"2024-09-12T03:01:01.118993+0000","last_became_active":"2024-09-12T03:00:58.340753+0000","last_became_peered":"2024-09-12T03:00:58.340753+0000","last_unstale":"2024-09-12T03:01:01.118993+0000","last_undegraded":"2024-09-12T03:01:01.118993+0000","last_fullsized":"2024-09-12T03:01:01.118993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:21.217915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:06.388962+0000","last_change":"2024-09-12T03:00:56.436058+0000","last_active":"2024-09-12T03:01:06.388962+0000","last_peered":"2024-09-12T03:01:06.388962+0000","last_clean":"2024-09-12T03:01:06.388962+0000","last_became_active":"2024-09-12T03:00:56.435518+0000","last_became_peered":"2024-09-12T03:00:56.435518+0000","last_unstale":"2024-09-12T03:01:06.388962+0000","last_undegraded":"2024-09-12T03:01:06.388962+0000","last_fullsized":"2024-09-12T03:01:06.388962+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:32.235097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.330626+0000","last_change":"2024-09-12T03:00:58.348236+0000","last_active":"2024-09-12T03:00:59.330626+0000","last_peered":"2024-09-12T03:00:59.330626+0000","last_clean":"2024-09-12T03:00:59.330626+0000","last_became_active":"2024-09-12T03:00:58.347727+0000","last_became_peered":"2024-09-12T03:00:58.347727+0000","last_unstale":"2024-09-12T03:00:59.330626+0000","last_undegraded":"2024-09-12T03:00:59.330626+0000","last_fullsized":"2024-09-12T03:00:59.330626+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:19.233163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'8","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:19.344447+0000","last_change":"2024-09-12T03:00:56.310537+0000","last_active":"2024-09-12T03:01:19.344447+0000","last_peered":"2024-09-12T03:01:19.344447+0000","last_clean":"2024-09-12T03:01:19.344447+0000","last_became_active":"2024-09-12T03:00:56.310135+0000","last_became_peered":"2024-09-12T03:00:56.310135+0000","last_unstale":"2024-09-12T03:01:19.344447+0000","last_undegraded":"2024-09-12T03:01:19.344447+0000","last_fullsized":"2024-09-12T03:01:19.344447+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:10:55.584339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344075+0000","last_change":"2024-09-12T03:00:58.358417+0000","last_active":"2024-09-12T03:00:59.344075+0000","last_peered":"2024-09-12T03:00:59.344075+0000","last_clean":"2024-09-12T03:00:59.344075+0000","last_became_active":"2024-09-12T03:00:58.358119+0000","last_became_peered":"2024-09-12T03:00:58.358119+0000","last_unstale":"2024-09-12T03:00:59.344075+0000","last_undegraded":"2024-09-12T03:00:59.344075+0000","last_fullsized":"2024-09-12T03:00:59.344075+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:10.897594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:06.388934+0000","last_change":"2024-09-12T03:00:56.747044+0000","last_active":"2024-09-12T03:01:06.388934+0000","last_peered":"2024-09-12T03:01:06.388934+0000","last_clean":"2024-09-12T03:01:06.388934+0000","last_became_active":"2024-09-12T03:00:56.746699+0000","last_became_peered":"2024-09-12T03:00:56.746699+0000","last_unstale":"2024-09-12T03:01:06.388934+0000","last_undegraded":"2024-09-12T03:01:06.388934+0000","last_fullsized":"2024-09-12T03:01:06.388934+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:11.420109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342687+0000","last_change":"2024-09-12T03:00:58.346854+0000","last_active":"2024-09-12T03:00:59.342687+0000","last_peered":"2024-09-12T03:00:59.342687+0000","last_clean":"2024-09-12T03:00:59.342687+0000","last_became_active":"2024-09-12T03:00:58.346535+0000","last_became_peered":"2024-09-12T03:00:58.346535+0000","last_unstale":"2024-09-12T03:00:59.342687+0000","last_undegraded":"2024-09-12T03:00:59.342687+0000","last_fullsized":"2024-09-12T03:00:59.342687+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:46.637914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:06.389702+0000","last_change":"2024-09-12T03:00:56.435957+0000","last_active":"2024-09-12T03:01:06.389702+0000","last_peered":"2024-09-12T03:01:06.389702+0000","last_clean":"2024-09-12T03:01:06.389702+0000","last_became_active":"2024-09-12T03:00:56.435603+0000","last_became_peered":"2024-09-12T03:00:56.435603+0000","last_unstale":"2024-09-12T03:01:06.389702+0000","last_undegraded":"2024-09-12T03:01:06.389702+0000","last_fullsized":"2024-09-12T03:01:06.389702+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:34.553189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343477+0000","last_change":"2024-09-12T03:00:58.337784+0000","last_active":"2024-09-12T03:00:59.343477+0000","last_peered":"2024-09-12T03:00:59.343477+0000","last_clean":"2024-09-12T03:00:59.343477+0000","last_became_active":"2024-09-12T03:00:58.337415+0000","last_became_peered":"2024-09-12T03:00:58.337415+0000","last_unstale":"2024-09-12T03:00:59.343477+0000","last_undegraded":"2024-09-12T03:00:59.343477+0000","last_fullsized":"2024-09-12T03:00:59.343477+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:38:15.557067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:06.390162+0000","last_change":"2024-09-12T03:00:56.315356+0000","last_active":"2024-09-12T03:01:06.390162+0000","last_peered":"2024-09-12T03:01:06.390162+0000","last_clean":"2024-09-12T03:01:06.390162+0000","last_became_active":"2024-09-12T03:00:56.314873+0000","last_became_peered":"2024-09-12T03:00:56.314873+0000","last_unstale":"2024-09-12T03:01:06.390162+0000","last_undegraded":"2024-09-12T03:01:06.390162+0000","last_fullsized":"2024-09-12T03:01:06.390162+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:55.710156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.348074+0000","last_change":"2024-09-12T03:00:58.346706+0000","last_active":"2024-09-12T03:00:59.348074+0000","last_peered":"2024-09-12T03:00:59.348074+0000","last_clean":"2024-09-12T03:00:59.348074+0000","last_became_active":"2024-09-12T03:00:58.346363+0000","last_became_peered":"2024-09-12T03:00:58.346363+0000","last_unstale":"2024-09-12T03:00:59.348074+0000","last_undegraded":"2024-09-12T03:00:59.348074+0000","last_fullsized":"2024-09-12T03:00:59.348074+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:13.468639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343312+0000","last_change":"2024-09-12T03:00:56.314659+0000","last_active":"2024-09-12T03:00:59.343312+0000","last_peered":"2024-09-12T03:00:59.343312+0000","last_clean":"2024-09-12T03:00:59.343312+0000","last_became_active":"2024-09-12T03:00:56.314316+0000","last_became_peered":"2024-09-12T03:00:56.314316+0000","last_unstale":"2024-09-12T03:00:59.343312+0000","last_undegraded":"2024-09-12T03:00:59.343312+0000","last_fullsized":"2024-09-12T03:00:59.343312+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:26:23.883066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.345830+0000","last_change":"2024-09-12T03:00:58.356682+0000","last_active":"2024-09-12T03:00:59.345830+0000","last_peered":"2024-09-12T03:00:59.345830+0000","last_clean":"2024-09-12T03:00:59.345830+0000","last_became_active":"2024-09-12T03:00:58.356383+0000","last_became_peered":"2024-09-12T03:00:58.356383+0000","last_unstale":"2024-09-12T03:00:59.345830+0000","last_undegraded":"2024-09-12T03:00:59.345830+0000","last_fullsized":"2024-09-12T03:00:59.345830+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:14:16.428393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:06.390865+0000","last_change":"2024-09-12T03:00:56.314943+0000","last_active":"2024-09-12T03:01:06.390865+0000","last_peered":"2024-09-12T03:01:06.390865+0000","last_clean":"2024-09-12T03:01:06.390865+0000","last_became_active":"2024-09-12T03:00:56.314602+0000","last_became_peered":"2024-09-12T03:00:56.314602+0000","last_unstale":"2024-09-12T03:01:06.390865+0000","last_undegraded":"2024-09-12T03:01:06.390865+0000","last_fullsized":"2024-09-12T03:01:06.390865+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:31.207880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343048+0000","last_change":"2024-09-12T03:00:58.351774+0000","last_active":"2024-09-12T03:00:59.343048+0000","last_peered":"2024-09-12T03:00:59.343048+0000","last_clean":"2024-09-12T03:00:59.343048+0000","last_became_active":"2024-09-12T03:00:58.351453+0000","last_became_peered":"2024-09-12T03:00:58.351453+0000","last_unstale":"2024-09-12T03:00:59.343048+0000","last_undegraded":"2024-09-12T03:00:59.343048+0000","last_fullsized":"2024-09-12T03:00:59.343048+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:18:24.260110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328442+0000","last_change":"2024-09-12T03:00:58.346886+0000","last_active":"2024-09-12T03:00:59.328442+0000","last_peered":"2024-09-12T03:00:59.328442+0000","last_clean":"2024-09-12T03:00:59.328442+0000","last_became_active":"2024-09-12T03:00:58.346499+0000","last_became_peered":"2024-09-12T03:00:58.346499+0000","last_unstale":"2024-09-12T03:00:59.328442+0000","last_undegraded":"2024-09-12T03:00:59.328442+0000","last_fullsized":"2024-09-12T03:00:59.328442+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:02:24.045610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.350709+0000","last_change":"2024-09-12T03:00:56.317443+0000","last_active":"2024-09-12T03:00:59.350709+0000","last_peered":"2024-09-12T03:00:59.350709+0000","last_clean":"2024-09-12T03:00:59.350709+0000","last_became_active":"2024-09-12T03:00:56.317045+0000","last_became_peered":"2024-09-12T03:00:56.317045+0000","last_unstale":"2024-09-12T03:00:59.350709+0000","last_undegraded":"2024-09-12T03:00:59.350709+0000","last_fullsized":"2024-09-12T03:00:59.350709+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:07.687606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328499+0000","last_change":"2024-09-12T03:00:58.337257+0000","last_active":"2024-09-12T03:00:59.328499+0000","last_peered":"2024-09-12T03:00:59.328499+0000","last_clean":"2024-09-12T03:00:59.328499+0000","last_became_active":"2024-09-12T03:00:58.336847+0000","last_became_peered":"2024-09-12T03:00:58.336847+0000","last_unstale":"2024-09-12T03:00:59.328499+0000","last_undegraded":"2024-09-12T03:00:59.328499+0000","last_fullsized":"2024-09-12T03:00:59.328499+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:22.846206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.398760+0000","last_change":"2024-09-12T03:00:56.316255+0000","last_active":"2024-09-12T03:01:07.398760+0000","last_peered":"2024-09-12T03:01:07.398760+0000","last_clean":"2024-09-12T03:01:07.398760+0000","last_became_active":"2024-09-12T03:00:56.315906+0000","last_became_peered":"2024-09-12T03:00:56.315906+0000","last_unstale":"2024-09-12T03:01:07.398760+0000","last_undegraded":"2024-09-12T03:01:07.398760+0000","last_fullsized":"2024-09-12T03:01:07.398760+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:14.798601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342990+0000","last_change":"2024-09-12T03:00:58.347206+0000","last_active":"2024-09-12T03:00:59.342990+0000","last_peered":"2024-09-12T03:00:59.342990+0000","last_clean":"2024-09-12T03:00:59.342990+0000","last_became_active":"2024-09-12T03:00:58.346923+0000","last_became_peered":"2024-09-12T03:00:58.346923+0000","last_unstale":"2024-09-12T03:00:59.342990+0000","last_undegraded":"2024-09-12T03:00:59.342990+0000","last_fullsized":"2024-09-12T03:00:59.342990+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:41.376211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.347646+0000","last_change":"2024-09-12T03:00:56.435431+0000","last_active":"2024-09-12T03:00:59.347646+0000","last_peered":"2024-09-12T03:00:59.347646+0000","last_clean":"2024-09-12T03:00:59.347646+0000","last_became_active":"2024-09-12T03:00:56.435088+0000","last_became_peered":"2024-09-12T03:00:56.435088+0000","last_unstale":"2024-09-12T03:00:59.347646+0000","last_undegraded":"2024-09-12T03:00:59.347646+0000","last_fullsized":"2024-09-12T03:00:59.347646+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:24.738340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343937+0000","last_change":"2024-09-12T03:00:58.351737+0000","last_active":"2024-09-12T03:00:59.343937+0000","last_peered":"2024-09-12T03:00:59.343937+0000","last_clean":"2024-09-12T03:00:59.343937+0000","last_became_active":"2024-09-12T03:00:58.351407+0000","last_became_peered":"2024-09-12T03:00:58.351407+0000","last_unstale":"2024-09-12T03:00:59.343937+0000","last_undegraded":"2024-09-12T03:00:59.343937+0000","last_fullsized":"2024-09-12T03:00:59.343937+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:34.206703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.399005+0000","last_change":"2024-09-12T03:00:56.310910+0000","last_active":"2024-09-12T03:01:07.399005+0000","last_peered":"2024-09-12T03:01:07.399005+0000","last_clean":"2024-09-12T03:01:07.399005+0000","last_became_active":"2024-09-12T03:00:56.310657+0000","last_became_peered":"2024-09-12T03:00:56.310657+0000","last_unstale":"2024-09-12T03:01:07.399005+0000","last_undegraded":"2024-09-12T03:01:07.399005+0000","last_fullsized":"2024-09-12T03:01:07.399005+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:06.117215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":50412,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":99,"num_read_kb":105,"num_write":150,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":372736,"data_stored":145499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245899,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645329,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":414814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906966,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":1053097,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070492,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":415050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299425,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":868352,"data_stored":414970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528357,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":1006509,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":40,"seq":171798691883,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":856064,"data_stored":414439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":33,"seq":141733920815,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":1006193,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051637,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":860160,"data_stored":414876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280570,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":414750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476799,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":462384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607620,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":897024,"data_stored":455297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":876544,"data_stored":415710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":41065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":48152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":48380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:01:24.940 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:01:25.187 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:25.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:25 smithi177 ceph-mon[35305]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:01:25.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:25 smithi183 ceph-mon[35535]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:01:25.453 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:25 smithi116 ceph-mon[31410]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:01:25.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.673+0000 7fae98133640 1 -- 172.21.15.116:0/3556342203 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae90112b90 msgr2=0x7fae90114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:25.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.673+0000 7fae98133640 1 --2- 172.21.15.116:0/3556342203 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae90112b90 0x7fae90114f80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fae88009ca0 tx=0x7fae8802a950 comp rx=0 tx=0).stop 2024-09-12T03:01:25.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.673+0000 7fae98133640 1 -- 172.21.15.116:0/3556342203 shutdown_connections 2024-09-12T03:01:25.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.673+0000 7fae98133640 1 --2- 172.21.15.116:0/3556342203 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae90112b90 0x7fae90114f80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:25.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.673+0000 7fae98133640 1 --2- 172.21.15.116:0/3556342203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae90074eb0 0x7fae90112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:25.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.673+0000 7fae98133640 1 --2- 172.21.15.116:0/3556342203 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae90074590 0x7fae90074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:25.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.673+0000 7fae98133640 1 -- 172.21.15.116:0/3556342203 >> 172.21.15.116:0/3556342203 conn(0x7fae900fd8c0 msgr2=0x7fae900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:25.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.674+0000 7fae98133640 1 -- 172.21.15.116:0/3556342203 shutdown_connections 2024-09-12T03:01:25.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.674+0000 7fae98133640 1 -- 172.21.15.116:0/3556342203 wait complete. 2024-09-12T03:01:25.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.674+0000 7fae98133640 1 Processor -- start 2024-09-12T03:01:25.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.674+0000 7fae98133640 1 -- start start 2024-09-12T03:01:25.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.674+0000 7fae98133640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae90074590 0x7fae901a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:25.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.674+0000 7fae98133640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae90074eb0 0x7fae901a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:25.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.674+0000 7fae98133640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae90112b90 0x7fae9019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:25.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.674+0000 7fae98133640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fae901185b0 con 0x7fae90112b90 2024-09-12T03:01:25.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.674+0000 7fae98133640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fae90118430 con 0x7fae90074eb0 2024-09-12T03:01:25.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.674+0000 7fae98133640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fae90118730 con 0x7fae90074590 2024-09-12T03:01:25.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.674+0000 7fae966d0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae90112b90 0x7fae9019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:25.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.674+0000 7fae966d0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae90112b90 0x7fae9019e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34102/0 (socket says 172.21.15.116:34102) 2024-09-12T03:01:25.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.674+0000 7fae966d0640 1 -- 172.21.15.116:0/4008908079 learned_addr learned my addr 172.21.15.116:0/4008908079 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:25.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.675+0000 7fae95ecf640 1 --2- 172.21.15.116:0/4008908079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae90074590 0x7fae901a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:25.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.675+0000 7fae956ce640 1 --2- 172.21.15.116:0/4008908079 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae90074eb0 0x7fae901a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:25.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.675+0000 7fae966d0640 1 -- 172.21.15.116:0/4008908079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae90074590 msgr2=0x7fae901a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:25.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.675+0000 7fae966d0640 1 --2- 172.21.15.116:0/4008908079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae90074590 0x7fae901a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:25.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.675+0000 7fae966d0640 1 -- 172.21.15.116:0/4008908079 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae90074eb0 msgr2=0x7fae901a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:25.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.675+0000 7fae966d0640 1 --2- 172.21.15.116:0/4008908079 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae90074eb0 0x7fae901a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:25.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.675+0000 7fae966d0640 1 -- 172.21.15.116:0/4008908079 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9019f000 con 0x7fae90112b90 2024-09-12T03:01:25.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.675+0000 7fae956ce640 1 --2- 172.21.15.116:0/4008908079 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae90074eb0 0x7fae901a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:01:25.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.675+0000 7fae966d0640 1 --2- 172.21.15.116:0/4008908079 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae90112b90 0x7fae9019e8d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fae8802ae70 tx=0x7fae8802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:25.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.675+0000 7fae86ffd640 1 -- 172.21.15.116:0/4008908079 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae88034040 con 0x7fae90112b90 2024-09-12T03:01:25.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.675+0000 7fae86ffd640 1 -- 172.21.15.116:0/4008908079 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae88030920 con 0x7fae90112b90 2024-09-12T03:01:25.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.675+0000 7fae86ffd640 1 -- 172.21.15.116:0/4008908079 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae88045480 con 0x7fae90112b90 2024-09-12T03:01:25.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.675+0000 7fae98133640 1 -- 172.21.15.116:0/4008908079 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae9019f290 con 0x7fae90112b90 2024-09-12T03:01:25.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.675+0000 7fae98133640 1 -- 172.21.15.116:0/4008908079 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae901ab5e0 con 0x7fae90112b90 2024-09-12T03:01:25.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.676+0000 7fae98133640 1 -- 172.21.15.116:0/4008908079 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae90074970 con 0x7fae90112b90 2024-09-12T03:01:25.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.677+0000 7fae86ffd640 1 -- 172.21.15.116:0/4008908079 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fae88037020 con 0x7fae90112b90 2024-09-12T03:01:25.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.678+0000 7fae86ffd640 1 --2- 172.21.15.116:0/4008908079 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fae7c0a2c10 0x7fae7c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:25.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.678+0000 7fae95ecf640 1 --2- 172.21.15.116:0/4008908079 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fae7c0a2c10 0x7fae7c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:25.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.678+0000 7fae86ffd640 1 -- 172.21.15.116:0/4008908079 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fae880efec0 con 0x7fae90112b90 2024-09-12T03:01:25.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.678+0000 7fae95ecf640 1 --2- 172.21.15.116:0/4008908079 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fae7c0a2c10 0x7fae7c0a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fae8c005fd0 tx=0x7fae8c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:25.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.682+0000 7fae86ffd640 1 -- 172.21.15.116:0/4008908079 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fae880bcf80 con 0x7fae90112b90 2024-09-12T03:01:25.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.846+0000 7fae98133640 1 -- 172.21.15.116:0/4008908079 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fae900ff7d0 con 0x7fae7c0a2c10 2024-09-12T03:01:25.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.852+0000 7fae86ffd640 1 -- 172.21.15.116:0/4008908079 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258836 (secure 0 0 0) 0x7fae900ff7d0 con 0x7fae7c0a2c10 2024-09-12T03:01:25.856 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:01:25.857 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-09-12T03:01:25.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.855+0000 7fae98133640 1 -- 172.21.15.116:0/4008908079 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fae7c0a2c10 msgr2=0x7fae7c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:25.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.855+0000 7fae98133640 1 --2- 172.21.15.116:0/4008908079 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fae7c0a2c10 0x7fae7c0a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fae8c005fd0 tx=0x7fae8c005850 comp rx=0 tx=0).stop 2024-09-12T03:01:25.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.855+0000 7fae98133640 1 -- 172.21.15.116:0/4008908079 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae90112b90 msgr2=0x7fae9019e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:25.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.855+0000 7fae98133640 1 --2- 172.21.15.116:0/4008908079 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae90112b90 0x7fae9019e8d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fae8802ae70 tx=0x7fae8802f5b0 comp rx=0 tx=0).stop 2024-09-12T03:01:25.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.855+0000 7fae98133640 1 -- 172.21.15.116:0/4008908079 shutdown_connections 2024-09-12T03:01:25.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.855+0000 7fae98133640 1 --2- 172.21.15.116:0/4008908079 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fae7c0a2c10 0x7fae7c0a50d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:25.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.855+0000 7fae98133640 1 --2- 172.21.15.116:0/4008908079 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae90112b90 0x7fae9019e8d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:25.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.855+0000 7fae98133640 1 --2- 172.21.15.116:0/4008908079 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae90074eb0 0x7fae901a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:25.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.855+0000 7fae98133640 1 --2- 172.21.15.116:0/4008908079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae90074590 0x7fae901a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:25.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.855+0000 7fae98133640 1 -- 172.21.15.116:0/4008908079 >> 172.21.15.116:0/4008908079 conn(0x7fae900fd8c0 msgr2=0x7fae900ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:25.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.855+0000 7fae98133640 1 -- 172.21.15.116:0/4008908079 shutdown_connections 2024-09-12T03:01:25.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:25.855+0000 7fae98133640 1 -- 172.21.15.116:0/4008908079 wait complete. 2024-09-12T03:01:26.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:26 smithi183 ceph-mon[35535]: pgmap v289: 97 pgs: 97 active+clean; 626 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-09-12T03:01:26.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:26 smithi177 ceph-mon[35305]: pgmap v289: 97 pgs: 97 active+clean; 626 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-09-12T03:01:26.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:26 smithi116 ceph-mon[31410]: pgmap v289: 97 pgs: 97 active+clean; 626 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-09-12T03:01:26.678 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-09-12T03:01:25.112991+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641014,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":235,"num_read_kb":214,"num_write":333,"num_write_kb":3031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":267,"ondisk_log_size":267,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368044,"kb_used_data":10476,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573780,"statfs":{"total":1151940427776,"available":1151563550720,"internally_reserved":0,"allocated":10727424,"data_stored":6893826,"data_compressed":107490,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":45424,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":76,"num_write":27,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003043"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.348943+0000","last_change":"2024-09-12T03:00:58.348563+0000","last_active":"2024-09-12T03:00:59.348943+0000","last_peered":"2024-09-12T03:00:59.348943+0000","last_clean":"2024-09-12T03:00:59.348943+0000","last_became_active":"2024-09-12T03:00:58.347940+0000","last_became_peered":"2024-09-12T03:00:58.347940+0000","last_unstale":"2024-09-12T03:00:59.348943+0000","last_undegraded":"2024-09-12T03:00:59.348943+0000","last_fullsized":"2024-09-12T03:00:59.348943+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:43.489238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342719+0000","last_change":"2024-09-12T03:00:58.351969+0000","last_active":"2024-09-12T03:00:59.342719+0000","last_peered":"2024-09-12T03:00:59.342719+0000","last_clean":"2024-09-12T03:00:59.342719+0000","last_became_active":"2024-09-12T03:00:58.351679+0000","last_became_peered":"2024-09-12T03:00:58.351679+0000","last_unstale":"2024-09-12T03:00:59.342719+0000","last_undegraded":"2024-09-12T03:00:59.342719+0000","last_fullsized":"2024-09-12T03:00:59.342719+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:27:01.174890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342541+0000","last_change":"2024-09-12T03:00:58.346045+0000","last_active":"2024-09-12T03:00:59.342541+0000","last_peered":"2024-09-12T03:00:59.342541+0000","last_clean":"2024-09-12T03:00:59.342541+0000","last_became_active":"2024-09-12T03:00:58.345719+0000","last_became_peered":"2024-09-12T03:00:58.345719+0000","last_unstale":"2024-09-12T03:00:59.342541+0000","last_undegraded":"2024-09-12T03:00:59.342541+0000","last_fullsized":"2024-09-12T03:00:59.342541+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:12.312650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343947+0000","last_change":"2024-09-12T03:00:58.348165+0000","last_active":"2024-09-12T03:00:59.343947+0000","last_peered":"2024-09-12T03:00:59.343947+0000","last_clean":"2024-09-12T03:00:59.343947+0000","last_became_active":"2024-09-12T03:00:58.347734+0000","last_became_peered":"2024-09-12T03:00:58.347734+0000","last_unstale":"2024-09-12T03:00:59.343947+0000","last_undegraded":"2024-09-12T03:00:59.343947+0000","last_fullsized":"2024-09-12T03:00:59.343947+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:05.945900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343877+0000","last_change":"2024-09-12T03:00:58.339316+0000","last_active":"2024-09-12T03:00:59.343877+0000","last_peered":"2024-09-12T03:00:59.343877+0000","last_clean":"2024-09-12T03:00:59.343877+0000","last_became_active":"2024-09-12T03:00:58.338964+0000","last_became_peered":"2024-09-12T03:00:58.338964+0000","last_unstale":"2024-09-12T03:00:59.343877+0000","last_undegraded":"2024-09-12T03:00:59.343877+0000","last_fullsized":"2024-09-12T03:00:59.343877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:50.721276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342586+0000","last_change":"2024-09-12T03:00:58.346838+0000","last_active":"2024-09-12T03:00:59.342586+0000","last_peered":"2024-09-12T03:00:59.342586+0000","last_clean":"2024-09-12T03:00:59.342586+0000","last_became_active":"2024-09-12T03:00:58.346486+0000","last_became_peered":"2024-09-12T03:00:58.346486+0000","last_unstale":"2024-09-12T03:00:59.342586+0000","last_undegraded":"2024-09-12T03:00:59.342586+0000","last_fullsized":"2024-09-12T03:00:59.342586+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:28.687727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343160+0000","last_change":"2024-09-12T03:00:58.346712+0000","last_active":"2024-09-12T03:00:59.343160+0000","last_peered":"2024-09-12T03:00:59.343160+0000","last_clean":"2024-09-12T03:00:59.343160+0000","last_became_active":"2024-09-12T03:00:58.346397+0000","last_became_peered":"2024-09-12T03:00:58.346397+0000","last_unstale":"2024-09-12T03:00:59.343160+0000","last_undegraded":"2024-09-12T03:00:59.343160+0000","last_fullsized":"2024-09-12T03:00:59.343160+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:48:00.624733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343665+0000","last_change":"2024-09-12T03:00:58.356166+0000","last_active":"2024-09-12T03:00:59.343665+0000","last_peered":"2024-09-12T03:00:59.343665+0000","last_clean":"2024-09-12T03:00:59.343665+0000","last_became_active":"2024-09-12T03:00:58.354944+0000","last_became_peered":"2024-09-12T03:00:58.354944+0000","last_unstale":"2024-09-12T03:00:59.343665+0000","last_undegraded":"2024-09-12T03:00:59.343665+0000","last_fullsized":"2024-09-12T03:00:59.343665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:15.840149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.329171+0000","last_change":"2024-09-12T03:00:58.340182+0000","last_active":"2024-09-12T03:00:59.329171+0000","last_peered":"2024-09-12T03:00:59.329171+0000","last_clean":"2024-09-12T03:00:59.329171+0000","last_became_active":"2024-09-12T03:00:58.339576+0000","last_became_peered":"2024-09-12T03:00:58.339576+0000","last_unstale":"2024-09-12T03:00:59.329171+0000","last_undegraded":"2024-09-12T03:00:59.329171+0000","last_fullsized":"2024-09-12T03:00:59.329171+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:16.534844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344683+0000","last_change":"2024-09-12T03:00:58.351218+0000","last_active":"2024-09-12T03:00:59.344683+0000","last_peered":"2024-09-12T03:00:59.344683+0000","last_clean":"2024-09-12T03:00:59.344683+0000","last_became_active":"2024-09-12T03:00:58.350871+0000","last_became_peered":"2024-09-12T03:00:58.350871+0000","last_unstale":"2024-09-12T03:00:59.344683+0000","last_undegraded":"2024-09-12T03:00:59.344683+0000","last_fullsized":"2024-09-12T03:00:59.344683+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:38.367683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343826+0000","last_change":"2024-09-12T03:00:58.351894+0000","last_active":"2024-09-12T03:00:59.343826+0000","last_peered":"2024-09-12T03:00:59.343826+0000","last_clean":"2024-09-12T03:00:59.343826+0000","last_became_active":"2024-09-12T03:00:58.351445+0000","last_became_peered":"2024-09-12T03:00:58.351445+0000","last_unstale":"2024-09-12T03:00:59.343826+0000","last_undegraded":"2024-09-12T03:00:59.343826+0000","last_fullsized":"2024-09-12T03:00:59.343826+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:42.093871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344004+0000","last_change":"2024-09-12T03:00:58.353460+0000","last_active":"2024-09-12T03:00:59.344004+0000","last_peered":"2024-09-12T03:00:59.344004+0000","last_clean":"2024-09-12T03:00:59.344004+0000","last_became_active":"2024-09-12T03:00:58.353111+0000","last_became_peered":"2024-09-12T03:00:58.353111+0000","last_unstale":"2024-09-12T03:00:59.344004+0000","last_undegraded":"2024-09-12T03:00:59.344004+0000","last_fullsized":"2024-09-12T03:00:59.344004+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:15.916765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.330518+0000","last_change":"2024-09-12T03:00:58.341080+0000","last_active":"2024-09-12T03:00:59.330518+0000","last_peered":"2024-09-12T03:00:59.330518+0000","last_clean":"2024-09-12T03:00:59.330518+0000","last_became_active":"2024-09-12T03:00:58.340782+0000","last_became_peered":"2024-09-12T03:00:58.340782+0000","last_unstale":"2024-09-12T03:00:59.330518+0000","last_undegraded":"2024-09-12T03:00:59.330518+0000","last_fullsized":"2024-09-12T03:00:59.330518+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:36.569700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342701+0000","last_change":"2024-09-12T03:00:58.346791+0000","last_active":"2024-09-12T03:00:59.342701+0000","last_peered":"2024-09-12T03:00:59.342701+0000","last_clean":"2024-09-12T03:00:59.342701+0000","last_became_active":"2024-09-12T03:00:58.346500+0000","last_became_peered":"2024-09-12T03:00:58.346500+0000","last_unstale":"2024-09-12T03:00:59.342701+0000","last_undegraded":"2024-09-12T03:00:59.342701+0000","last_fullsized":"2024-09-12T03:00:59.342701+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:32.017589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342415+0000","last_change":"2024-09-12T03:00:58.349522+0000","last_active":"2024-09-12T03:00:59.342415+0000","last_peered":"2024-09-12T03:00:59.342415+0000","last_clean":"2024-09-12T03:00:59.342415+0000","last_became_active":"2024-09-12T03:00:58.349246+0000","last_became_peered":"2024-09-12T03:00:58.349246+0000","last_unstale":"2024-09-12T03:00:59.342415+0000","last_undegraded":"2024-09-12T03:00:59.342415+0000","last_fullsized":"2024-09-12T03:00:59.342415+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:39.253919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328399+0000","last_change":"2024-09-12T03:00:58.349909+0000","last_active":"2024-09-12T03:00:59.328399+0000","last_peered":"2024-09-12T03:00:59.328399+0000","last_clean":"2024-09-12T03:00:59.328399+0000","last_became_active":"2024-09-12T03:00:58.349590+0000","last_became_peered":"2024-09-12T03:00:58.349590+0000","last_unstale":"2024-09-12T03:00:59.328399+0000","last_undegraded":"2024-09-12T03:00:59.328399+0000","last_fullsized":"2024-09-12T03:00:59.328399+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:04.540709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.349104+0000","last_change":"2024-09-12T03:00:58.348284+0000","last_active":"2024-09-12T03:00:59.349104+0000","last_peered":"2024-09-12T03:00:59.349104+0000","last_clean":"2024-09-12T03:00:59.349104+0000","last_became_active":"2024-09-12T03:00:58.347465+0000","last_became_peered":"2024-09-12T03:00:58.347465+0000","last_unstale":"2024-09-12T03:00:59.349104+0000","last_undegraded":"2024-09-12T03:00:59.349104+0000","last_fullsized":"2024-09-12T03:00:59.349104+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:13.191899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343137+0000","last_change":"2024-09-12T03:00:58.337740+0000","last_active":"2024-09-12T03:00:59.343137+0000","last_peered":"2024-09-12T03:00:59.343137+0000","last_clean":"2024-09-12T03:00:59.343137+0000","last_became_active":"2024-09-12T03:00:58.337393+0000","last_became_peered":"2024-09-12T03:00:58.337393+0000","last_unstale":"2024-09-12T03:00:59.343137+0000","last_undegraded":"2024-09-12T03:00:59.343137+0000","last_fullsized":"2024-09-12T03:00:59.343137+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:28:56.339167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344750+0000","last_change":"2024-09-12T03:00:58.345779+0000","last_active":"2024-09-12T03:00:59.344750+0000","last_peered":"2024-09-12T03:00:59.344750+0000","last_clean":"2024-09-12T03:00:59.344750+0000","last_became_active":"2024-09-12T03:00:58.345210+0000","last_became_peered":"2024-09-12T03:00:58.345210+0000","last_unstale":"2024-09-12T03:00:59.344750+0000","last_undegraded":"2024-09-12T03:00:59.344750+0000","last_fullsized":"2024-09-12T03:00:59.344750+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:06:58.271185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.348332+0000","last_change":"2024-09-12T03:00:58.343877+0000","last_active":"2024-09-12T03:00:59.348332+0000","last_peered":"2024-09-12T03:00:59.348332+0000","last_clean":"2024-09-12T03:00:59.348332+0000","last_became_active":"2024-09-12T03:00:58.343545+0000","last_became_peered":"2024-09-12T03:00:58.343545+0000","last_unstale":"2024-09-12T03:00:59.348332+0000","last_undegraded":"2024-09-12T03:00:59.348332+0000","last_fullsized":"2024-09-12T03:00:59.348332+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:52:31.851975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.330596+0000","last_change":"2024-09-12T03:00:58.349233+0000","last_active":"2024-09-12T03:00:59.330596+0000","last_peered":"2024-09-12T03:00:59.330596+0000","last_clean":"2024-09-12T03:00:59.330596+0000","last_became_active":"2024-09-12T03:00:58.348958+0000","last_became_peered":"2024-09-12T03:00:58.348958+0000","last_unstale":"2024-09-12T03:00:59.330596+0000","last_undegraded":"2024-09-12T03:00:59.330596+0000","last_fullsized":"2024-09-12T03:00:59.330596+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:32:10.271817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328487+0000","last_change":"2024-09-12T03:00:58.358024+0000","last_active":"2024-09-12T03:00:59.328487+0000","last_peered":"2024-09-12T03:00:59.328487+0000","last_clean":"2024-09-12T03:00:59.328487+0000","last_became_active":"2024-09-12T03:00:58.357434+0000","last_became_peered":"2024-09-12T03:00:58.357434+0000","last_unstale":"2024-09-12T03:00:59.328487+0000","last_undegraded":"2024-09-12T03:00:59.328487+0000","last_fullsized":"2024-09-12T03:00:59.328487+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:01:20.119733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.345193+0000","last_change":"2024-09-12T03:00:58.355063+0000","last_active":"2024-09-12T03:00:59.345193+0000","last_peered":"2024-09-12T03:00:59.345193+0000","last_clean":"2024-09-12T03:00:59.345193+0000","last_became_active":"2024-09-12T03:00:58.354760+0000","last_became_peered":"2024-09-12T03:00:58.354760+0000","last_unstale":"2024-09-12T03:00:59.345193+0000","last_undegraded":"2024-09-12T03:00:59.345193+0000","last_fullsized":"2024-09-12T03:00:59.345193+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:13.464413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328351+0000","last_change":"2024-09-12T03:00:58.341916+0000","last_active":"2024-09-12T03:00:59.328351+0000","last_peered":"2024-09-12T03:00:59.328351+0000","last_clean":"2024-09-12T03:00:59.328351+0000","last_became_active":"2024-09-12T03:00:58.340815+0000","last_became_peered":"2024-09-12T03:00:58.340815+0000","last_unstale":"2024-09-12T03:00:59.328351+0000","last_undegraded":"2024-09-12T03:00:59.328351+0000","last_fullsized":"2024-09-12T03:00:59.328351+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:13.777947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343925+0000","last_change":"2024-09-12T03:00:58.357765+0000","last_active":"2024-09-12T03:00:59.343925+0000","last_peered":"2024-09-12T03:00:59.343925+0000","last_clean":"2024-09-12T03:00:59.343925+0000","last_became_active":"2024-09-12T03:00:58.357480+0000","last_became_peered":"2024-09-12T03:00:58.357480+0000","last_unstale":"2024-09-12T03:00:59.343925+0000","last_undegraded":"2024-09-12T03:00:59.343925+0000","last_fullsized":"2024-09-12T03:00:59.343925+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:26.884381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344699+0000","last_change":"2024-09-12T03:00:58.349504+0000","last_active":"2024-09-12T03:00:59.344699+0000","last_peered":"2024-09-12T03:00:59.344699+0000","last_clean":"2024-09-12T03:00:59.344699+0000","last_became_active":"2024-09-12T03:00:58.349136+0000","last_became_peered":"2024-09-12T03:00:58.349136+0000","last_unstale":"2024-09-12T03:00:59.344699+0000","last_undegraded":"2024-09-12T03:00:59.344699+0000","last_fullsized":"2024-09-12T03:00:59.344699+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:09:55.229516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.348719+0000","last_change":"2024-09-12T03:00:58.341902+0000","last_active":"2024-09-12T03:00:59.348719+0000","last_peered":"2024-09-12T03:00:59.348719+0000","last_clean":"2024-09-12T03:00:59.348719+0000","last_became_active":"2024-09-12T03:00:58.341592+0000","last_became_peered":"2024-09-12T03:00:58.341592+0000","last_unstale":"2024-09-12T03:00:59.348719+0000","last_undegraded":"2024-09-12T03:00:59.348719+0000","last_fullsized":"2024-09-12T03:00:59.348719+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:13.852511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.330540+0000","last_change":"2024-09-12T03:00:58.348012+0000","last_active":"2024-09-12T03:00:59.330540+0000","last_peered":"2024-09-12T03:00:59.330540+0000","last_clean":"2024-09-12T03:00:59.330540+0000","last_became_active":"2024-09-12T03:00:58.347338+0000","last_became_peered":"2024-09-12T03:00:58.347338+0000","last_unstale":"2024-09-12T03:00:59.330540+0000","last_undegraded":"2024-09-12T03:00:59.330540+0000","last_fullsized":"2024-09-12T03:00:59.330540+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:41.478032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342928+0000","last_change":"2024-09-12T03:00:58.345833+0000","last_active":"2024-09-12T03:00:59.342928+0000","last_peered":"2024-09-12T03:00:59.342928+0000","last_clean":"2024-09-12T03:00:59.342928+0000","last_became_active":"2024-09-12T03:00:58.345482+0000","last_became_peered":"2024-09-12T03:00:58.345482+0000","last_unstale":"2024-09-12T03:00:59.342928+0000","last_undegraded":"2024-09-12T03:00:59.342928+0000","last_fullsized":"2024-09-12T03:00:59.342928+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:32.037412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343059+0000","last_change":"2024-09-12T03:00:58.352290+0000","last_active":"2024-09-12T03:00:59.343059+0000","last_peered":"2024-09-12T03:00:59.343059+0000","last_clean":"2024-09-12T03:00:59.343059+0000","last_became_active":"2024-09-12T03:00:58.351918+0000","last_became_peered":"2024-09-12T03:00:58.351918+0000","last_unstale":"2024-09-12T03:00:59.343059+0000","last_undegraded":"2024-09-12T03:00:59.343059+0000","last_fullsized":"2024-09-12T03:00:59.343059+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:25.941080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343279+0000","last_change":"2024-09-12T03:00:58.348021+0000","last_active":"2024-09-12T03:00:59.343279+0000","last_peered":"2024-09-12T03:00:59.343279+0000","last_clean":"2024-09-12T03:00:59.343279+0000","last_became_active":"2024-09-12T03:00:58.347634+0000","last_became_peered":"2024-09-12T03:00:58.347634+0000","last_unstale":"2024-09-12T03:00:59.343279+0000","last_undegraded":"2024-09-12T03:00:59.343279+0000","last_fullsized":"2024-09-12T03:00:59.343279+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:47.933067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.347866+0000","last_change":"2024-09-12T03:00:58.340891+0000","last_active":"2024-09-12T03:00:59.347866+0000","last_peered":"2024-09-12T03:00:59.347866+0000","last_clean":"2024-09-12T03:00:59.347866+0000","last_became_active":"2024-09-12T03:00:58.340504+0000","last_became_peered":"2024-09-12T03:00:58.340504+0000","last_unstale":"2024-09-12T03:00:59.347866+0000","last_undegraded":"2024-09-12T03:00:59.347866+0000","last_fullsized":"2024-09-12T03:00:59.347866+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:32.117635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343541+0000","last_change":"2024-09-12T03:00:58.339343+0000","last_active":"2024-09-12T03:00:59.343541+0000","last_peered":"2024-09-12T03:00:59.343541+0000","last_clean":"2024-09-12T03:00:59.343541+0000","last_became_active":"2024-09-12T03:00:58.339001+0000","last_became_peered":"2024-09-12T03:00:58.339001+0000","last_unstale":"2024-09-12T03:00:59.343541+0000","last_undegraded":"2024-09-12T03:00:59.343541+0000","last_fullsized":"2024-09-12T03:00:59.343541+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:02.043454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:22.421941+0000","last_change":"2024-09-12T03:00:56.315550+0000","last_active":"2024-09-12T03:01:22.421941+0000","last_peered":"2024-09-12T03:01:22.421941+0000","last_clean":"2024-09-12T03:01:22.421941+0000","last_became_active":"2024-09-12T03:00:56.315225+0000","last_became_peered":"2024-09-12T03:00:56.315225+0000","last_unstale":"2024-09-12T03:01:22.421941+0000","last_undegraded":"2024-09-12T03:01:22.421941+0000","last_fullsized":"2024-09-12T03:01:22.421941+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:46.070895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328646+0000","last_change":"2024-09-12T03:00:58.338379+0000","last_active":"2024-09-12T03:00:59.328646+0000","last_peered":"2024-09-12T03:00:59.328646+0000","last_clean":"2024-09-12T03:00:59.328646+0000","last_became_active":"2024-09-12T03:00:58.337795+0000","last_became_peered":"2024-09-12T03:00:58.337795+0000","last_unstale":"2024-09-12T03:00:59.328646+0000","last_undegraded":"2024-09-12T03:00:59.328646+0000","last_fullsized":"2024-09-12T03:00:59.328646+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:43.299329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.397617+0000","last_change":"2024-09-12T03:00:56.316385+0000","last_active":"2024-09-12T03:01:07.397617+0000","last_peered":"2024-09-12T03:01:07.397617+0000","last_clean":"2024-09-12T03:01:07.397617+0000","last_became_active":"2024-09-12T03:00:56.316064+0000","last_became_peered":"2024-09-12T03:00:56.316064+0000","last_unstale":"2024-09-12T03:01:07.397617+0000","last_undegraded":"2024-09-12T03:01:07.397617+0000","last_fullsized":"2024-09-12T03:01:07.397617+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:12:43.323654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.397886+0000","last_change":"2024-09-12T03:00:56.313065+0000","last_active":"2024-09-12T03:01:07.397886+0000","last_peered":"2024-09-12T03:01:07.397886+0000","last_clean":"2024-09-12T03:01:07.397886+0000","last_became_active":"2024-09-12T03:00:56.312716+0000","last_became_peered":"2024-09-12T03:00:56.312716+0000","last_unstale":"2024-09-12T03:01:07.397886+0000","last_undegraded":"2024-09-12T03:01:07.397886+0000","last_fullsized":"2024-09-12T03:01:07.397886+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:51.089089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.346221+0000","last_change":"2024-09-12T03:00:58.346542+0000","last_active":"2024-09-12T03:00:59.346221+0000","last_peered":"2024-09-12T03:00:59.346221+0000","last_clean":"2024-09-12T03:00:59.346221+0000","last_became_active":"2024-09-12T03:00:58.346260+0000","last_became_peered":"2024-09-12T03:00:58.346260+0000","last_unstale":"2024-09-12T03:00:59.346221+0000","last_undegraded":"2024-09-12T03:00:59.346221+0000","last_fullsized":"2024-09-12T03:00:59.346221+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:37.034969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328738+0000","last_change":"2024-09-12T03:00:56.317308+0000","last_active":"2024-09-12T03:00:59.328738+0000","last_peered":"2024-09-12T03:00:59.328738+0000","last_clean":"2024-09-12T03:00:59.328738+0000","last_became_active":"2024-09-12T03:00:56.316935+0000","last_became_peered":"2024-09-12T03:00:56.316935+0000","last_unstale":"2024-09-12T03:00:59.328738+0000","last_undegraded":"2024-09-12T03:00:59.328738+0000","last_fullsized":"2024-09-12T03:00:59.328738+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:06.324489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343312+0000","last_change":"2024-09-12T03:00:58.341565+0000","last_active":"2024-09-12T03:00:59.343312+0000","last_peered":"2024-09-12T03:00:59.343312+0000","last_clean":"2024-09-12T03:00:59.343312+0000","last_became_active":"2024-09-12T03:00:58.341285+0000","last_became_peered":"2024-09-12T03:00:58.341285+0000","last_unstale":"2024-09-12T03:00:59.343312+0000","last_undegraded":"2024-09-12T03:00:59.343312+0000","last_fullsized":"2024-09-12T03:00:59.343312+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:59.156700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:19.417025+0000","last_change":"2024-09-12T03:00:56.314201+0000","last_active":"2024-09-12T03:01:19.417025+0000","last_peered":"2024-09-12T03:01:19.417025+0000","last_clean":"2024-09-12T03:01:19.417025+0000","last_became_active":"2024-09-12T03:00:56.313880+0000","last_became_peered":"2024-09-12T03:00:56.313880+0000","last_unstale":"2024-09-12T03:01:19.417025+0000","last_undegraded":"2024-09-12T03:01:19.417025+0000","last_fullsized":"2024-09-12T03:01:19.417025+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:57:12.952630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.329144+0000","last_change":"2024-09-12T03:00:58.345274+0000","last_active":"2024-09-12T03:00:59.329144+0000","last_peered":"2024-09-12T03:00:59.329144+0000","last_clean":"2024-09-12T03:00:59.329144+0000","last_became_active":"2024-09-12T03:00:58.344937+0000","last_became_peered":"2024-09-12T03:00:58.344937+0000","last_unstale":"2024-09-12T03:00:59.329144+0000","last_undegraded":"2024-09-12T03:00:59.329144+0000","last_fullsized":"2024-09-12T03:00:59.329144+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:17.288909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:06.390277+0000","last_change":"2024-09-12T03:00:56.314709+0000","last_active":"2024-09-12T03:01:06.390277+0000","last_peered":"2024-09-12T03:01:06.390277+0000","last_clean":"2024-09-12T03:01:06.390277+0000","last_became_active":"2024-09-12T03:00:56.314349+0000","last_became_peered":"2024-09-12T03:00:56.314349+0000","last_unstale":"2024-09-12T03:01:06.390277+0000","last_undegraded":"2024-09-12T03:01:06.390277+0000","last_fullsized":"2024-09-12T03:01:06.390277+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:18:35.371661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343140+0000","last_change":"2024-09-12T03:00:58.346055+0000","last_active":"2024-09-12T03:00:59.343140+0000","last_peered":"2024-09-12T03:00:59.343140+0000","last_clean":"2024-09-12T03:00:59.343140+0000","last_became_active":"2024-09-12T03:00:58.345701+0000","last_became_peered":"2024-09-12T03:00:58.345701+0000","last_unstale":"2024-09-12T03:00:59.343140+0000","last_undegraded":"2024-09-12T03:00:59.343140+0000","last_fullsized":"2024-09-12T03:00:59.343140+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:33.977245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:06.391491+0000","last_change":"2024-09-12T03:00:56.435962+0000","last_active":"2024-09-12T03:01:06.391491+0000","last_peered":"2024-09-12T03:01:06.391491+0000","last_clean":"2024-09-12T03:01:06.391491+0000","last_became_active":"2024-09-12T03:00:56.435515+0000","last_became_peered":"2024-09-12T03:00:56.435515+0000","last_unstale":"2024-09-12T03:01:06.391491+0000","last_undegraded":"2024-09-12T03:01:06.391491+0000","last_fullsized":"2024-09-12T03:01:06.391491+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:32.953436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328662+0000","last_change":"2024-09-12T03:00:58.344239+0000","last_active":"2024-09-12T03:00:59.328662+0000","last_peered":"2024-09-12T03:00:59.328662+0000","last_clean":"2024-09-12T03:00:59.328662+0000","last_became_active":"2024-09-12T03:00:58.343876+0000","last_became_peered":"2024-09-12T03:00:58.343876+0000","last_unstale":"2024-09-12T03:00:59.328662+0000","last_undegraded":"2024-09-12T03:00:59.328662+0000","last_fullsized":"2024-09-12T03:00:59.328662+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:49.401295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'22","reported_seq":72,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:20.419578+0000","last_change":"2024-09-12T03:00:56.317708+0000","last_active":"2024-09-12T03:01:20.419578+0000","last_peered":"2024-09-12T03:01:20.419578+0000","last_clean":"2024-09-12T03:01:20.419578+0000","last_became_active":"2024-09-12T03:00:56.317317+0000","last_became_peered":"2024-09-12T03:00:56.317317+0000","last_unstale":"2024-09-12T03:01:20.419578+0000","last_undegraded":"2024-09-12T03:01:20.419578+0000","last_fullsized":"2024-09-12T03:01:20.419578+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:47.788186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46802,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":61,"num_write":23,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.346470+0000","last_change":"2024-09-12T03:00:58.343602+0000","last_active":"2024-09-12T03:00:59.346470+0000","last_peered":"2024-09-12T03:00:59.346470+0000","last_clean":"2024-09-12T03:00:59.346470+0000","last_became_active":"2024-09-12T03:00:58.343301+0000","last_became_peered":"2024-09-12T03:00:58.343301+0000","last_unstale":"2024-09-12T03:00:59.346470+0000","last_undegraded":"2024-09-12T03:00:59.346470+0000","last_fullsized":"2024-09-12T03:00:59.346470+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:04.056118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.398339+0000","last_change":"2024-09-12T03:00:56.317874+0000","last_active":"2024-09-12T03:01:07.398339+0000","last_peered":"2024-09-12T03:01:07.398339+0000","last_clean":"2024-09-12T03:01:07.398339+0000","last_became_active":"2024-09-12T03:00:56.317566+0000","last_became_peered":"2024-09-12T03:00:56.317566+0000","last_unstale":"2024-09-12T03:01:07.398339+0000","last_undegraded":"2024-09-12T03:01:07.398339+0000","last_fullsized":"2024-09-12T03:01:07.398339+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:54.203730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.330344+0000","last_change":"2024-09-12T03:00:58.351539+0000","last_active":"2024-09-12T03:00:59.330344+0000","last_peered":"2024-09-12T03:00:59.330344+0000","last_clean":"2024-09-12T03:00:59.330344+0000","last_became_active":"2024-09-12T03:00:58.351249+0000","last_became_peered":"2024-09-12T03:00:58.351249+0000","last_unstale":"2024-09-12T03:00:59.330344+0000","last_undegraded":"2024-09-12T03:00:59.330344+0000","last_fullsized":"2024-09-12T03:00:59.330344+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:40.498867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:06.390652+0000","last_change":"2024-09-12T03:00:56.314489+0000","last_active":"2024-09-12T03:01:06.390652+0000","last_peered":"2024-09-12T03:01:06.390652+0000","last_clean":"2024-09-12T03:01:06.390652+0000","last_became_active":"2024-09-12T03:00:56.314142+0000","last_became_peered":"2024-09-12T03:00:56.314142+0000","last_unstale":"2024-09-12T03:01:06.390652+0000","last_undegraded":"2024-09-12T03:01:06.390652+0000","last_fullsized":"2024-09-12T03:01:06.390652+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:21.541713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343350+0000","last_change":"2024-09-12T03:00:58.348752+0000","last_active":"2024-09-12T03:00:59.343350+0000","last_peered":"2024-09-12T03:00:59.343350+0000","last_clean":"2024-09-12T03:00:59.343350+0000","last_became_active":"2024-09-12T03:00:58.348391+0000","last_became_peered":"2024-09-12T03:00:58.348391+0000","last_unstale":"2024-09-12T03:00:59.343350+0000","last_undegraded":"2024-09-12T03:00:59.343350+0000","last_fullsized":"2024-09-12T03:00:59.343350+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:33:15.626306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'3","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:23.422960+0000","last_change":"2024-09-12T03:00:56.311702+0000","last_active":"2024-09-12T03:01:23.422960+0000","last_peered":"2024-09-12T03:01:23.422960+0000","last_clean":"2024-09-12T03:01:23.422960+0000","last_became_active":"2024-09-12T03:00:56.311353+0000","last_became_peered":"2024-09-12T03:00:56.311353+0000","last_unstale":"2024-09-12T03:01:23.422960+0000","last_undegraded":"2024-09-12T03:01:23.422960+0000","last_fullsized":"2024-09-12T03:01:23.422960+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:01.004858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.345441+0000","last_change":"2024-09-12T03:00:58.351973+0000","last_active":"2024-09-12T03:00:59.345441+0000","last_peered":"2024-09-12T03:00:59.345441+0000","last_clean":"2024-09-12T03:00:59.345441+0000","last_became_active":"2024-09-12T03:00:58.351599+0000","last_became_peered":"2024-09-12T03:00:58.351599+0000","last_unstale":"2024-09-12T03:00:59.345441+0000","last_undegraded":"2024-09-12T03:00:59.345441+0000","last_fullsized":"2024-09-12T03:00:59.345441+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:26.398828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.399761+0000","last_change":"2024-09-12T03:00:56.436185+0000","last_active":"2024-09-12T03:01:07.399761+0000","last_peered":"2024-09-12T03:01:07.399761+0000","last_clean":"2024-09-12T03:01:07.399761+0000","last_became_active":"2024-09-12T03:00:56.435717+0000","last_became_peered":"2024-09-12T03:00:56.435717+0000","last_unstale":"2024-09-12T03:01:07.399761+0000","last_undegraded":"2024-09-12T03:01:07.399761+0000","last_fullsized":"2024-09-12T03:01:07.399761+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:27.303182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328571+0000","last_change":"2024-09-12T03:00:58.346851+0000","last_active":"2024-09-12T03:00:59.328571+0000","last_peered":"2024-09-12T03:00:59.328571+0000","last_clean":"2024-09-12T03:00:59.328571+0000","last_became_active":"2024-09-12T03:00:58.346490+0000","last_became_peered":"2024-09-12T03:00:58.346490+0000","last_unstale":"2024-09-12T03:00:59.328571+0000","last_undegraded":"2024-09-12T03:00:59.328571+0000","last_fullsized":"2024-09-12T03:00:59.328571+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:34:27.775668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:08.394235+0000","last_change":"2024-09-12T03:00:56.311333+0000","last_active":"2024-09-12T03:01:08.394235+0000","last_peered":"2024-09-12T03:01:08.394235+0000","last_clean":"2024-09-12T03:01:08.394235+0000","last_became_active":"2024-09-12T03:00:56.310871+0000","last_became_peered":"2024-09-12T03:00:56.310871+0000","last_unstale":"2024-09-12T03:01:08.394235+0000","last_undegraded":"2024-09-12T03:01:08.394235+0000","last_fullsized":"2024-09-12T03:01:08.394235+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:59.344572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.348769+0000","last_change":"2024-09-12T03:00:58.349401+0000","last_active":"2024-09-12T03:00:59.348769+0000","last_peered":"2024-09-12T03:00:59.348769+0000","last_clean":"2024-09-12T03:00:59.348769+0000","last_became_active":"2024-09-12T03:00:58.348996+0000","last_became_peered":"2024-09-12T03:00:58.348996+0000","last_unstale":"2024-09-12T03:00:59.348769+0000","last_undegraded":"2024-09-12T03:00:59.348769+0000","last_fullsized":"2024-09-12T03:00:59.348769+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:20:26.201684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344392+0000","last_change":"2024-09-12T03:00:56.314182+0000","last_active":"2024-09-12T03:00:59.344392+0000","last_peered":"2024-09-12T03:00:59.344392+0000","last_clean":"2024-09-12T03:00:59.344392+0000","last_became_active":"2024-09-12T03:00:56.313749+0000","last_became_peered":"2024-09-12T03:00:56.313749+0000","last_unstale":"2024-09-12T03:00:59.344392+0000","last_undegraded":"2024-09-12T03:00:59.344392+0000","last_fullsized":"2024-09-12T03:00:59.344392+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:31:45.767500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343307+0000","last_change":"2024-09-12T03:00:58.344356+0000","last_active":"2024-09-12T03:00:59.343307+0000","last_peered":"2024-09-12T03:00:59.343307+0000","last_clean":"2024-09-12T03:00:59.343307+0000","last_became_active":"2024-09-12T03:00:58.344035+0000","last_became_peered":"2024-09-12T03:00:58.344035+0000","last_unstale":"2024-09-12T03:00:59.343307+0000","last_undegraded":"2024-09-12T03:00:59.343307+0000","last_fullsized":"2024-09-12T03:00:59.343307+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:45:18.681018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.347000+0000","last_change":"2024-09-12T03:00:56.315653+0000","last_active":"2024-09-12T03:00:59.347000+0000","last_peered":"2024-09-12T03:00:59.347000+0000","last_clean":"2024-09-12T03:00:59.347000+0000","last_became_active":"2024-09-12T03:00:56.315284+0000","last_became_peered":"2024-09-12T03:00:56.315284+0000","last_unstale":"2024-09-12T03:00:59.347000+0000","last_undegraded":"2024-09-12T03:00:59.347000+0000","last_fullsized":"2024-09-12T03:00:59.347000+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:42.934020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343195+0000","last_change":"2024-09-12T03:00:58.357792+0000","last_active":"2024-09-12T03:00:59.343195+0000","last_peered":"2024-09-12T03:00:59.343195+0000","last_clean":"2024-09-12T03:00:59.343195+0000","last_became_active":"2024-09-12T03:00:58.357451+0000","last_became_peered":"2024-09-12T03:00:58.357451+0000","last_unstale":"2024-09-12T03:00:59.343195+0000","last_undegraded":"2024-09-12T03:00:59.343195+0000","last_fullsized":"2024-09-12T03:00:59.343195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:20.582607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.398109+0000","last_change":"2024-09-12T03:00:56.746780+0000","last_active":"2024-09-12T03:01:07.398109+0000","last_peered":"2024-09-12T03:01:07.398109+0000","last_clean":"2024-09-12T03:01:07.398109+0000","last_became_active":"2024-09-12T03:00:56.746414+0000","last_became_peered":"2024-09-12T03:00:56.746414+0000","last_unstale":"2024-09-12T03:01:07.398109+0000","last_undegraded":"2024-09-12T03:01:07.398109+0000","last_fullsized":"2024-09-12T03:01:07.398109+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:15:35.214196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343974+0000","last_change":"2024-09-12T03:00:58.351713+0000","last_active":"2024-09-12T03:00:59.343974+0000","last_peered":"2024-09-12T03:00:59.343974+0000","last_clean":"2024-09-12T03:00:59.343974+0000","last_became_active":"2024-09-12T03:00:58.351371+0000","last_became_peered":"2024-09-12T03:00:58.351371+0000","last_unstale":"2024-09-12T03:00:59.343974+0000","last_undegraded":"2024-09-12T03:00:59.343974+0000","last_fullsized":"2024-09-12T03:00:59.343974+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:13.346956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'137","reported_seq":331,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.345546+0000","last_change":"2024-09-12T02:59:45.461482+0000","last_active":"2024-09-12T03:00:59.345546+0000","last_peered":"2024-09-12T03:00:59.345546+0000","last_clean":"2024-09-12T03:00:59.345546+0000","last_became_active":"2024-09-12T02:59:45.461058+0000","last_became_peered":"2024-09-12T02:59:45.461058+0000","last_unstale":"2024-09-12T03:00:59.345546+0000","last_undegraded":"2024-09-12T03:00:59.345546+0000","last_fullsized":"2024-09-12T03:00:59.345546+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:18.034438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:18.034438+0000","last_clean_scrub_stamp":"2024-09-12T02:56:18.034438+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:09.251858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:08.398498+0000","last_change":"2024-09-12T03:00:56.315245+0000","last_active":"2024-09-12T03:01:08.398498+0000","last_peered":"2024-09-12T03:01:08.398498+0000","last_clean":"2024-09-12T03:01:08.398498+0000","last_became_active":"2024-09-12T03:00:56.314887+0000","last_became_peered":"2024-09-12T03:00:56.314887+0000","last_unstale":"2024-09-12T03:01:08.398498+0000","last_undegraded":"2024-09-12T03:01:08.398498+0000","last_fullsized":"2024-09-12T03:01:08.398498+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:48.750657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344703+0000","last_change":"2024-09-12T03:00:58.357874+0000","last_active":"2024-09-12T03:00:59.344703+0000","last_peered":"2024-09-12T03:00:59.344703+0000","last_clean":"2024-09-12T03:00:59.344703+0000","last_became_active":"2024-09-12T03:00:58.357575+0000","last_became_peered":"2024-09-12T03:00:58.357575+0000","last_unstale":"2024-09-12T03:00:59.344703+0000","last_undegraded":"2024-09-12T03:00:59.344703+0000","last_fullsized":"2024-09-12T03:00:59.344703+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:46.125620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.398754+0000","last_change":"2024-09-12T03:00:56.312263+0000","last_active":"2024-09-12T03:01:07.398754+0000","last_peered":"2024-09-12T03:01:07.398754+0000","last_clean":"2024-09-12T03:01:07.398754+0000","last_became_active":"2024-09-12T03:00:56.311894+0000","last_became_peered":"2024-09-12T03:00:56.311894+0000","last_unstale":"2024-09-12T03:01:07.398754+0000","last_undegraded":"2024-09-12T03:01:07.398754+0000","last_fullsized":"2024-09-12T03:01:07.398754+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:52.434203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.330287+0000","last_change":"2024-09-12T03:00:58.350762+0000","last_active":"2024-09-12T03:00:59.330287+0000","last_peered":"2024-09-12T03:00:59.330287+0000","last_clean":"2024-09-12T03:00:59.330287+0000","last_became_active":"2024-09-12T03:00:58.348038+0000","last_became_peered":"2024-09-12T03:00:58.348038+0000","last_unstale":"2024-09-12T03:00:59.330287+0000","last_undegraded":"2024-09-12T03:00:59.330287+0000","last_fullsized":"2024-09-12T03:00:59.330287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:08.916636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:19.416442+0000","last_change":"2024-09-12T03:00:56.435641+0000","last_active":"2024-09-12T03:01:19.416442+0000","last_peered":"2024-09-12T03:01:19.416442+0000","last_clean":"2024-09-12T03:01:19.416442+0000","last_became_active":"2024-09-12T03:00:56.435319+0000","last_became_peered":"2024-09-12T03:00:56.435319+0000","last_unstale":"2024-09-12T03:01:19.416442+0000","last_undegraded":"2024-09-12T03:01:19.416442+0000","last_fullsized":"2024-09-12T03:01:19.416442+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:56.894978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328360+0000","last_change":"2024-09-12T03:00:58.344708+0000","last_active":"2024-09-12T03:00:59.328360+0000","last_peered":"2024-09-12T03:00:59.328360+0000","last_clean":"2024-09-12T03:00:59.328360+0000","last_became_active":"2024-09-12T03:00:58.344389+0000","last_became_peered":"2024-09-12T03:00:58.344389+0000","last_unstale":"2024-09-12T03:00:59.328360+0000","last_undegraded":"2024-09-12T03:00:59.328360+0000","last_fullsized":"2024-09-12T03:00:59.328360+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:08.160383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343624+0000","last_change":"2024-09-12T03:00:56.317068+0000","last_active":"2024-09-12T03:00:59.343624+0000","last_peered":"2024-09-12T03:00:59.343624+0000","last_clean":"2024-09-12T03:00:59.343624+0000","last_became_active":"2024-09-12T03:00:56.316698+0000","last_became_peered":"2024-09-12T03:00:56.316698+0000","last_unstale":"2024-09-12T03:00:59.343624+0000","last_undegraded":"2024-09-12T03:00:59.343624+0000","last_fullsized":"2024-09-12T03:00:59.343624+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:41:19.199418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343531+0000","last_change":"2024-09-12T03:00:58.347814+0000","last_active":"2024-09-12T03:00:59.343531+0000","last_peered":"2024-09-12T03:00:59.343531+0000","last_clean":"2024-09-12T03:00:59.343531+0000","last_became_active":"2024-09-12T03:00:58.347112+0000","last_became_peered":"2024-09-12T03:00:58.347112+0000","last_unstale":"2024-09-12T03:00:59.343531+0000","last_undegraded":"2024-09-12T03:00:59.343531+0000","last_fullsized":"2024-09-12T03:00:59.343531+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:59.325113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342760+0000","last_change":"2024-09-12T03:00:56.435771+0000","last_active":"2024-09-12T03:00:59.342760+0000","last_peered":"2024-09-12T03:00:59.342760+0000","last_clean":"2024-09-12T03:00:59.342760+0000","last_became_active":"2024-09-12T03:00:56.435448+0000","last_became_peered":"2024-09-12T03:00:56.435448+0000","last_unstale":"2024-09-12T03:00:59.342760+0000","last_undegraded":"2024-09-12T03:00:59.342760+0000","last_fullsized":"2024-09-12T03:00:59.342760+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:33.650338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:01.118993+0000","last_change":"2024-09-12T03:00:58.341071+0000","last_active":"2024-09-12T03:01:01.118993+0000","last_peered":"2024-09-12T03:01:01.118993+0000","last_clean":"2024-09-12T03:01:01.118993+0000","last_became_active":"2024-09-12T03:00:58.340753+0000","last_became_peered":"2024-09-12T03:00:58.340753+0000","last_unstale":"2024-09-12T03:01:01.118993+0000","last_undegraded":"2024-09-12T03:01:01.118993+0000","last_fullsized":"2024-09-12T03:01:01.118993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:21.217915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:06.388962+0000","last_change":"2024-09-12T03:00:56.436058+0000","last_active":"2024-09-12T03:01:06.388962+0000","last_peered":"2024-09-12T03:01:06.388962+0000","last_clean":"2024-09-12T03:01:06.388962+0000","last_became_active":"2024-09-12T03:00:56.435518+0000","last_became_peered":"2024-09-12T03:00:56.435518+0000","last_unstale":"2024-09-12T03:01:06.388962+0000","last_undegraded":"2024-09-12T03:01:06.388962+0000","last_fullsized":"2024-09-12T03:01:06.388962+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:32.235097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.330626+0000","last_change":"2024-09-12T03:00:58.348236+0000","last_active":"2024-09-12T03:00:59.330626+0000","last_peered":"2024-09-12T03:00:59.330626+0000","last_clean":"2024-09-12T03:00:59.330626+0000","last_became_active":"2024-09-12T03:00:58.347727+0000","last_became_peered":"2024-09-12T03:00:58.347727+0000","last_unstale":"2024-09-12T03:00:59.330626+0000","last_undegraded":"2024-09-12T03:00:59.330626+0000","last_fullsized":"2024-09-12T03:00:59.330626+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:19.233163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'8","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:19.344447+0000","last_change":"2024-09-12T03:00:56.310537+0000","last_active":"2024-09-12T03:01:19.344447+0000","last_peered":"2024-09-12T03:01:19.344447+0000","last_clean":"2024-09-12T03:01:19.344447+0000","last_became_active":"2024-09-12T03:00:56.310135+0000","last_became_peered":"2024-09-12T03:00:56.310135+0000","last_unstale":"2024-09-12T03:01:19.344447+0000","last_undegraded":"2024-09-12T03:01:19.344447+0000","last_fullsized":"2024-09-12T03:01:19.344447+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:10:55.584339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344075+0000","last_change":"2024-09-12T03:00:58.358417+0000","last_active":"2024-09-12T03:00:59.344075+0000","last_peered":"2024-09-12T03:00:59.344075+0000","last_clean":"2024-09-12T03:00:59.344075+0000","last_became_active":"2024-09-12T03:00:58.358119+0000","last_became_peered":"2024-09-12T03:00:58.358119+0000","last_unstale":"2024-09-12T03:00:59.344075+0000","last_undegraded":"2024-09-12T03:00:59.344075+0000","last_fullsized":"2024-09-12T03:00:59.344075+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:10.897594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:06.388934+0000","last_change":"2024-09-12T03:00:56.747044+0000","last_active":"2024-09-12T03:01:06.388934+0000","last_peered":"2024-09-12T03:01:06.388934+0000","last_clean":"2024-09-12T03:01:06.388934+0000","last_became_active":"2024-09-12T03:00:56.746699+0000","last_became_peered":"2024-09-12T03:00:56.746699+0000","last_unstale":"2024-09-12T03:01:06.388934+0000","last_undegraded":"2024-09-12T03:01:06.388934+0000","last_fullsized":"2024-09-12T03:01:06.388934+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:11.420109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342687+0000","last_change":"2024-09-12T03:00:58.346854+0000","last_active":"2024-09-12T03:00:59.342687+0000","last_peered":"2024-09-12T03:00:59.342687+0000","last_clean":"2024-09-12T03:00:59.342687+0000","last_became_active":"2024-09-12T03:00:58.346535+0000","last_became_peered":"2024-09-12T03:00:58.346535+0000","last_unstale":"2024-09-12T03:00:59.342687+0000","last_undegraded":"2024-09-12T03:00:59.342687+0000","last_fullsized":"2024-09-12T03:00:59.342687+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:46.637914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:06.389702+0000","last_change":"2024-09-12T03:00:56.435957+0000","last_active":"2024-09-12T03:01:06.389702+0000","last_peered":"2024-09-12T03:01:06.389702+0000","last_clean":"2024-09-12T03:01:06.389702+0000","last_became_active":"2024-09-12T03:00:56.435603+0000","last_became_peered":"2024-09-12T03:00:56.435603+0000","last_unstale":"2024-09-12T03:01:06.389702+0000","last_undegraded":"2024-09-12T03:01:06.389702+0000","last_fullsized":"2024-09-12T03:01:06.389702+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:34.553189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343477+0000","last_change":"2024-09-12T03:00:58.337784+0000","last_active":"2024-09-12T03:00:59.343477+0000","last_peered":"2024-09-12T03:00:59.343477+0000","last_clean":"2024-09-12T03:00:59.343477+0000","last_became_active":"2024-09-12T03:00:58.337415+0000","last_became_peered":"2024-09-12T03:00:58.337415+0000","last_unstale":"2024-09-12T03:00:59.343477+0000","last_undegraded":"2024-09-12T03:00:59.343477+0000","last_fullsized":"2024-09-12T03:00:59.343477+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:38:15.557067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:06.390162+0000","last_change":"2024-09-12T03:00:56.315356+0000","last_active":"2024-09-12T03:01:06.390162+0000","last_peered":"2024-09-12T03:01:06.390162+0000","last_clean":"2024-09-12T03:01:06.390162+0000","last_became_active":"2024-09-12T03:00:56.314873+0000","last_became_peered":"2024-09-12T03:00:56.314873+0000","last_unstale":"2024-09-12T03:01:06.390162+0000","last_undegraded":"2024-09-12T03:01:06.390162+0000","last_fullsized":"2024-09-12T03:01:06.390162+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:55.710156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.348074+0000","last_change":"2024-09-12T03:00:58.346706+0000","last_active":"2024-09-12T03:00:59.348074+0000","last_peered":"2024-09-12T03:00:59.348074+0000","last_clean":"2024-09-12T03:00:59.348074+0000","last_became_active":"2024-09-12T03:00:58.346363+0000","last_became_peered":"2024-09-12T03:00:58.346363+0000","last_unstale":"2024-09-12T03:00:59.348074+0000","last_undegraded":"2024-09-12T03:00:59.348074+0000","last_fullsized":"2024-09-12T03:00:59.348074+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:13.468639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343312+0000","last_change":"2024-09-12T03:00:56.314659+0000","last_active":"2024-09-12T03:00:59.343312+0000","last_peered":"2024-09-12T03:00:59.343312+0000","last_clean":"2024-09-12T03:00:59.343312+0000","last_became_active":"2024-09-12T03:00:56.314316+0000","last_became_peered":"2024-09-12T03:00:56.314316+0000","last_unstale":"2024-09-12T03:00:59.343312+0000","last_undegraded":"2024-09-12T03:00:59.343312+0000","last_fullsized":"2024-09-12T03:00:59.343312+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:26:23.883066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.345830+0000","last_change":"2024-09-12T03:00:58.356682+0000","last_active":"2024-09-12T03:00:59.345830+0000","last_peered":"2024-09-12T03:00:59.345830+0000","last_clean":"2024-09-12T03:00:59.345830+0000","last_became_active":"2024-09-12T03:00:58.356383+0000","last_became_peered":"2024-09-12T03:00:58.356383+0000","last_unstale":"2024-09-12T03:00:59.345830+0000","last_undegraded":"2024-09-12T03:00:59.345830+0000","last_fullsized":"2024-09-12T03:00:59.345830+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:14:16.428393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:06.390865+0000","last_change":"2024-09-12T03:00:56.314943+0000","last_active":"2024-09-12T03:01:06.390865+0000","last_peered":"2024-09-12T03:01:06.390865+0000","last_clean":"2024-09-12T03:01:06.390865+0000","last_became_active":"2024-09-12T03:00:56.314602+0000","last_became_peered":"2024-09-12T03:00:56.314602+0000","last_unstale":"2024-09-12T03:01:06.390865+0000","last_undegraded":"2024-09-12T03:01:06.390865+0000","last_fullsized":"2024-09-12T03:01:06.390865+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:31.207880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343048+0000","last_change":"2024-09-12T03:00:58.351774+0000","last_active":"2024-09-12T03:00:59.343048+0000","last_peered":"2024-09-12T03:00:59.343048+0000","last_clean":"2024-09-12T03:00:59.343048+0000","last_became_active":"2024-09-12T03:00:58.351453+0000","last_became_peered":"2024-09-12T03:00:58.351453+0000","last_unstale":"2024-09-12T03:00:59.343048+0000","last_undegraded":"2024-09-12T03:00:59.343048+0000","last_fullsized":"2024-09-12T03:00:59.343048+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:18:24.260110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328442+0000","last_change":"2024-09-12T03:00:58.346886+0000","last_active":"2024-09-12T03:00:59.328442+0000","last_peered":"2024-09-12T03:00:59.328442+0000","last_clean":"2024-09-12T03:00:59.328442+0000","last_became_active":"2024-09-12T03:00:58.346499+0000","last_became_peered":"2024-09-12T03:00:58.346499+0000","last_unstale":"2024-09-12T03:00:59.328442+0000","last_undegraded":"2024-09-12T03:00:59.328442+0000","last_fullsized":"2024-09-12T03:00:59.328442+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:02:24.045610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.350709+0000","last_change":"2024-09-12T03:00:56.317443+0000","last_active":"2024-09-12T03:00:59.350709+0000","last_peered":"2024-09-12T03:00:59.350709+0000","last_clean":"2024-09-12T03:00:59.350709+0000","last_became_active":"2024-09-12T03:00:56.317045+0000","last_became_peered":"2024-09-12T03:00:56.317045+0000","last_unstale":"2024-09-12T03:00:59.350709+0000","last_undegraded":"2024-09-12T03:00:59.350709+0000","last_fullsized":"2024-09-12T03:00:59.350709+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:07.687606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328499+0000","last_change":"2024-09-12T03:00:58.337257+0000","last_active":"2024-09-12T03:00:59.328499+0000","last_peered":"2024-09-12T03:00:59.328499+0000","last_clean":"2024-09-12T03:00:59.328499+0000","last_became_active":"2024-09-12T03:00:58.336847+0000","last_became_peered":"2024-09-12T03:00:58.336847+0000","last_unstale":"2024-09-12T03:00:59.328499+0000","last_undegraded":"2024-09-12T03:00:59.328499+0000","last_fullsized":"2024-09-12T03:00:59.328499+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:22.846206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.398760+0000","last_change":"2024-09-12T03:00:56.316255+0000","last_active":"2024-09-12T03:01:07.398760+0000","last_peered":"2024-09-12T03:01:07.398760+0000","last_clean":"2024-09-12T03:01:07.398760+0000","last_became_active":"2024-09-12T03:00:56.315906+0000","last_became_peered":"2024-09-12T03:00:56.315906+0000","last_unstale":"2024-09-12T03:01:07.398760+0000","last_undegraded":"2024-09-12T03:01:07.398760+0000","last_fullsized":"2024-09-12T03:01:07.398760+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:14.798601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342990+0000","last_change":"2024-09-12T03:00:58.347206+0000","last_active":"2024-09-12T03:00:59.342990+0000","last_peered":"2024-09-12T03:00:59.342990+0000","last_clean":"2024-09-12T03:00:59.342990+0000","last_became_active":"2024-09-12T03:00:58.346923+0000","last_became_peered":"2024-09-12T03:00:58.346923+0000","last_unstale":"2024-09-12T03:00:59.342990+0000","last_undegraded":"2024-09-12T03:00:59.342990+0000","last_fullsized":"2024-09-12T03:00:59.342990+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:41.376211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.347646+0000","last_change":"2024-09-12T03:00:56.435431+0000","last_active":"2024-09-12T03:00:59.347646+0000","last_peered":"2024-09-12T03:00:59.347646+0000","last_clean":"2024-09-12T03:00:59.347646+0000","last_became_active":"2024-09-12T03:00:56.435088+0000","last_became_peered":"2024-09-12T03:00:56.435088+0000","last_unstale":"2024-09-12T03:00:59.347646+0000","last_undegraded":"2024-09-12T03:00:59.347646+0000","last_fullsized":"2024-09-12T03:00:59.347646+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:24.738340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343937+0000","last_change":"2024-09-12T03:00:58.351737+0000","last_active":"2024-09-12T03:00:59.343937+0000","last_peered":"2024-09-12T03:00:59.343937+0000","last_clean":"2024-09-12T03:00:59.343937+0000","last_became_active":"2024-09-12T03:00:58.351407+0000","last_became_peered":"2024-09-12T03:00:58.351407+0000","last_unstale":"2024-09-12T03:00:59.343937+0000","last_undegraded":"2024-09-12T03:00:59.343937+0000","last_fullsized":"2024-09-12T03:00:59.343937+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:34.206703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.399005+0000","last_change":"2024-09-12T03:00:56.310910+0000","last_active":"2024-09-12T03:01:07.399005+0000","last_peered":"2024-09-12T03:01:07.399005+0000","last_clean":"2024-09-12T03:01:07.399005+0000","last_became_active":"2024-09-12T03:00:56.310657+0000","last_became_peered":"2024-09-12T03:00:56.310657+0000","last_unstale":"2024-09-12T03:01:07.399005+0000","last_undegraded":"2024-09-12T03:01:07.399005+0000","last_fullsized":"2024-09-12T03:01:07.399005+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:06.117215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":50412,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":109,"num_read_kb":105,"num_write":150,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":151236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245899,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645330,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":414814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906966,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":1053097,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070492,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":415050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299425,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":868352,"data_stored":414970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528358,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":1006509,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":40,"seq":171798691883,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":856064,"data_stored":414439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":33,"seq":141733920816,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":1006193,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051638,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":860160,"data_stored":414876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280570,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":414750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476799,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":462384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607621,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":461034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":876544,"data_stored":415710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":46802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":48152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":48380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:01:26.683 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T03:01:26.683 INFO:tasks.cephadm:Setting up client nodes... 2024-09-12T03:01:26.684 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-12T03:01:26.931 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:27.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:27 smithi116 ceph-mon[31410]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:01:27.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:27 smithi183 ceph-mon[35535]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:01:27.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.445+0000 7f8fd5ce4640 1 -- 172.21.15.116:0/1675689530 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8fd0102690 msgr2=0x7f8fd0108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:27.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.445+0000 7f8fd5ce4640 1 --2- 172.21.15.116:0/1675689530 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8fd0102690 0x7f8fd0108f20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009ca0 tx=0x7f8fc802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:01:27.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.445+0000 7f8fd5ce4640 1 -- 172.21.15.116:0/1675689530 shutdown_connections 2024-09-12T03:01:27.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.445+0000 7f8fd5ce4640 1 --2- 172.21.15.116:0/1675689530 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8fd0102690 0x7f8fd0108f20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:27.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.445+0000 7f8fd5ce4640 1 --2- 172.21.15.116:0/1675689530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fd0101cd0 0x7f8fd0102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:27.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.446+0000 7f8fd5ce4640 1 --2- 172.21.15.116:0/1675689530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fd0100ad0 0x7f8fd0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:27.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.446+0000 7f8fd5ce4640 1 -- 172.21.15.116:0/1675689530 >> 172.21.15.116:0/1675689530 conn(0x7f8fd00fc280 msgr2=0x7f8fd00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:27.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.446+0000 7f8fd5ce4640 1 -- 172.21.15.116:0/1675689530 shutdown_connections 2024-09-12T03:01:27.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.446+0000 7f8fd5ce4640 1 -- 172.21.15.116:0/1675689530 wait complete. 2024-09-12T03:01:27.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.446+0000 7f8fd5ce4640 1 Processor -- start 2024-09-12T03:01:27.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.446+0000 7f8fd5ce4640 1 -- start start 2024-09-12T03:01:27.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.446+0000 7f8fd5ce4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fd0100ad0 0x7f8fd0197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:27.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.446+0000 7f8fd5ce4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8fd0101cd0 0x7f8fd01983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:27.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.447+0000 7f8fd5ce4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fd0102690 0x7f8fd019f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:27.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.447+0000 7f8fd5ce4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd010c750 con 0x7f8fd0101cd0 2024-09-12T03:01:27.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.447+0000 7f8fd5ce4640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd010c5d0 con 0x7f8fd0100ad0 2024-09-12T03:01:27.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.447+0000 7f8fd5ce4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd010c8d0 con 0x7f8fd0102690 2024-09-12T03:01:27.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.447+0000 7f8fceffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8fd0101cd0 0x7f8fd01983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:27.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.447+0000 7f8fceffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8fd0101cd0 0x7f8fd01983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34126/0 (socket says 172.21.15.116:34126) 2024-09-12T03:01:27.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.447+0000 7f8fceffd640 1 -- 172.21.15.116:0/3657772731 learned_addr learned my addr 172.21.15.116:0/3657772731 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:27.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.447+0000 7f8fcf7fe640 1 --2- 172.21.15.116:0/3657772731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fd0100ad0 0x7f8fd0197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:27.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.447+0000 7f8fcffff640 1 --2- 172.21.15.116:0/3657772731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fd0102690 0x7f8fd019f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:27.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.447+0000 7f8fceffd640 1 -- 172.21.15.116:0/3657772731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fd0102690 msgr2=0x7f8fd019f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:27.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.447+0000 7f8fceffd640 1 --2- 172.21.15.116:0/3657772731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fd0102690 0x7f8fd019f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:27.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.447+0000 7f8fceffd640 1 -- 172.21.15.116:0/3657772731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fd0100ad0 msgr2=0x7f8fd0197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:27.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.447+0000 7f8fceffd640 1 --2- 172.21.15.116:0/3657772731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fd0100ad0 0x7f8fd0197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:27.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.447+0000 7f8fceffd640 1 -- 172.21.15.116:0/3657772731 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd019fab0 con 0x7f8fd0101cd0 2024-09-12T03:01:27.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.447+0000 7f8fceffd640 1 --2- 172.21.15.116:0/3657772731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8fd0101cd0 0x7f8fd01983c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400d950 tx=0x7f8fc400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:27.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.447+0000 7f8fccff9640 1 -- 172.21.15.116:0/3657772731 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4017030 con 0x7f8fd0101cd0 2024-09-12T03:01:27.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.447+0000 7f8fd5ce4640 1 -- 172.21.15.116:0/3657772731 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd019fda0 con 0x7f8fd0101cd0 2024-09-12T03:01:27.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.447+0000 7f8fccff9640 1 -- 172.21.15.116:0/3657772731 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fc4004020 con 0x7f8fd0101cd0 2024-09-12T03:01:27.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.447+0000 7f8fccff9640 1 -- 172.21.15.116:0/3657772731 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc40047a0 con 0x7f8fd0101cd0 2024-09-12T03:01:27.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.447+0000 7f8fd5ce4640 1 -- 172.21.15.116:0/3657772731 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd01a0260 con 0x7f8fd0101cd0 2024-09-12T03:01:27.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.449+0000 7f8fd5ce4640 1 -- 172.21.15.116:0/3657772731 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fd01a0520 con 0x7f8fd0101cd0 2024-09-12T03:01:27.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.450+0000 7f8fccff9640 1 -- 172.21.15.116:0/3657772731 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f8fc40041d0 con 0x7f8fd0101cd0 2024-09-12T03:01:27.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.451+0000 7f8fccff9640 1 --2- 172.21.15.116:0/3657772731 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8fb80a2c40 0x7f8fb80a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:27.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.451+0000 7f8fcf7fe640 1 --2- 172.21.15.116:0/3657772731 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8fb80a2c40 0x7f8fb80a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:27.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.451+0000 7f8fccff9640 1 -- 172.21.15.116:0/3657772731 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f8fc40ce910 con 0x7f8fd0101cd0 2024-09-12T03:01:27.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.452+0000 7f8fcf7fe640 1 --2- 172.21.15.116:0/3657772731 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8fb80a2c40 0x7f8fb80a5100 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0005fd0 tx=0x7f8fc000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:27.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.455+0000 7f8fccff9640 1 -- 172.21.15.116:0/3657772731 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8fc409b8c0 con 0x7f8fd0101cd0 2024-09-12T03:01:27.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:27 smithi177 ceph-mon[35305]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:01:27.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.691+0000 7f8fd5ce4640 1 -- 172.21.15.116:0/3657772731 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f8fd01a08b0 con 0x7f8fd0101cd0 2024-09-12T03:01:27.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.694+0000 7f8fccff9640 1 -- 172.21.15.116:0/3657772731 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f8fc409b1c0 con 0x7f8fd0101cd0 2024-09-12T03:01:27.694 INFO:teuthology.orchestra.run.smithi116.stdout:[client.0] 2024-09-12T03:01:27.694 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQCHWeJmRQRQKRAAcrUwExzP3NXXhDTk1nPmmA== 2024-09-12T03:01:27.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.696+0000 7f8fd5ce4640 1 -- 172.21.15.116:0/3657772731 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8fb80a2c40 msgr2=0x7f8fb80a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:27.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.696+0000 7f8fd5ce4640 1 --2- 172.21.15.116:0/3657772731 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8fb80a2c40 0x7f8fb80a5100 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0005fd0 tx=0x7f8fc000b390 comp rx=0 tx=0).stop 2024-09-12T03:01:27.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.696+0000 7f8fd5ce4640 1 -- 172.21.15.116:0/3657772731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8fd0101cd0 msgr2=0x7f8fd01983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:27.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.696+0000 7f8fd5ce4640 1 --2- 172.21.15.116:0/3657772731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8fd0101cd0 0x7f8fd01983c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400d950 tx=0x7f8fc400de10 comp rx=0 tx=0).stop 2024-09-12T03:01:27.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.696+0000 7f8fd5ce4640 1 -- 172.21.15.116:0/3657772731 shutdown_connections 2024-09-12T03:01:27.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.696+0000 7f8fd5ce4640 1 --2- 172.21.15.116:0/3657772731 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8fb80a2c40 0x7f8fb80a5100 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:27.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.696+0000 7f8fd5ce4640 1 --2- 172.21.15.116:0/3657772731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fd0102690 0x7f8fd019f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:27.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.696+0000 7f8fd5ce4640 1 --2- 172.21.15.116:0/3657772731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8fd0101cd0 0x7f8fd01983c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:27.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.696+0000 7f8fd5ce4640 1 --2- 172.21.15.116:0/3657772731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fd0100ad0 0x7f8fd0197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:27.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.696+0000 7f8fd5ce4640 1 -- 172.21.15.116:0/3657772731 >> 172.21.15.116:0/3657772731 conn(0x7f8fd00fc280 msgr2=0x7f8fd0069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:27.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.696+0000 7f8fd5ce4640 1 -- 172.21.15.116:0/3657772731 shutdown_connections 2024-09-12T03:01:27.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:27.696+0000 7f8fd5ce4640 1 -- 172.21.15.116:0/3657772731 wait complete. 2024-09-12T03:01:28.417 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:28 smithi116 ceph-mon[31410]: pgmap v290: 97 pgs: 97 active+clean; 636 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.6 KiB/s wr, 9 op/s 2024-09-12T03:01:28.417 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:28 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3657772731' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T03:01:28.417 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:28 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3657772731' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T03:01:28.446 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-12T03:01:28.446 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-12T03:01:28.446 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-12T03:01:28.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:28 smithi183 ceph-mon[35535]: pgmap v290: 97 pgs: 97 active+clean; 636 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.6 KiB/s wr, 9 op/s 2024-09-12T03:01:28.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:28 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3657772731' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T03:01:28.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:28 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3657772731' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T03:01:28.484 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-12T03:01:28.484 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-12T03:01:28.485 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-12T03:01:28.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:28 smithi177 ceph-mon[35305]: pgmap v290: 97 pgs: 97 active+clean; 636 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.6 KiB/s wr, 9 op/s 2024-09-12T03:01:28.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:28 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3657772731' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T03:01:28.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:28 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3657772731' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T03:01:28.768 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:29.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.267+0000 7f1cf70d0640 1 -- 172.21.15.116:0/1135436844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cf0100e20 msgr2=0x7f1cf0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:29.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.267+0000 7f1cf70d0640 1 --2- 172.21.15.116:0/1135436844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cf0100e20 0x7f1cf0101200 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009ca0 tx=0x7f1ce802a950 comp rx=0 tx=0).stop 2024-09-12T03:01:29.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.267+0000 7f1cf70d0640 1 -- 172.21.15.116:0/1135436844 shutdown_connections 2024-09-12T03:01:29.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.267+0000 7f1cf70d0640 1 --2- 172.21.15.116:0/1135436844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cf0102ef0 0x7f1cf010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:29.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.267+0000 7f1cf70d0640 1 --2- 172.21.15.116:0/1135436844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cf0102570 0x7f1cf01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:29.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.267+0000 7f1cf70d0640 1 --2- 172.21.15.116:0/1135436844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cf0100e20 0x7f1cf0101200 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:29.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.267+0000 7f1cf70d0640 1 -- 172.21.15.116:0/1135436844 >> 172.21.15.116:0/1135436844 conn(0x7f1cf00fc7d0 msgr2=0x7f1cf00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:29.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.268+0000 7f1cf70d0640 1 -- 172.21.15.116:0/1135436844 shutdown_connections 2024-09-12T03:01:29.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.268+0000 7f1cf70d0640 1 -- 172.21.15.116:0/1135436844 wait complete. 2024-09-12T03:01:29.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.268+0000 7f1cf70d0640 1 Processor -- start 2024-09-12T03:01:29.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.268+0000 7f1cf70d0640 1 -- start start 2024-09-12T03:01:29.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.268+0000 7f1cf70d0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cf0100e20 0x7f1cf0196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:29.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.268+0000 7f1cf70d0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cf0102570 0x7f1cf0196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:29.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.268+0000 7f1cf70d0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cf0102ef0 0x7f1cf019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:29.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.268+0000 7f1cf70d0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf010fec0 con 0x7f1cf0102ef0 2024-09-12T03:01:29.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.268+0000 7f1cf70d0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf010fd40 con 0x7f1cf0100e20 2024-09-12T03:01:29.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.268+0000 7f1cf70d0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf0110040 con 0x7f1cf0102570 2024-09-12T03:01:29.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.268+0000 7f1cf566d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cf0102ef0 0x7f1cf019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:29.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.268+0000 7f1cf566d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cf0102ef0 0x7f1cf019ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34140/0 (socket says 172.21.15.116:34140) 2024-09-12T03:01:29.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.268+0000 7f1cf566d640 1 -- 172.21.15.116:0/1641229787 learned_addr learned my addr 172.21.15.116:0/1641229787 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:29.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.269+0000 7f1cf4e6c640 1 --2- 172.21.15.116:0/1641229787 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cf0100e20 0x7f1cf0196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:29.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.269+0000 7f1ceffff640 1 --2- 172.21.15.116:0/1641229787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cf0102570 0x7f1cf0196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:29.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.269+0000 7f1cf566d640 1 -- 172.21.15.116:0/1641229787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cf0102570 msgr2=0x7f1cf0196870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:29.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.269+0000 7f1cf566d640 1 --2- 172.21.15.116:0/1641229787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cf0102570 0x7f1cf0196870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:29.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.269+0000 7f1cf566d640 1 -- 172.21.15.116:0/1641229787 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cf0100e20 msgr2=0x7f1cf0196330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:29.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.269+0000 7f1cf566d640 1 --2- 172.21.15.116:0/1641229787 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cf0100e20 0x7f1cf0196330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:29.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.269+0000 7f1cf566d640 1 -- 172.21.15.116:0/1641229787 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf019b2e0 con 0x7f1cf0102ef0 2024-09-12T03:01:29.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.269+0000 7f1cf566d640 1 --2- 172.21.15.116:0/1641229787 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cf0102ef0 0x7f1cf019ac00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000d750 tx=0x7f1ce000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:29.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.269+0000 7f1cedffb640 1 -- 172.21.15.116:0/1641229787 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0017070 con 0x7f1cf0102ef0 2024-09-12T03:01:29.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.269+0000 7f1cedffb640 1 -- 172.21.15.116:0/1641229787 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ce0005500 con 0x7f1cf0102ef0 2024-09-12T03:01:29.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.269+0000 7f1cf70d0640 1 -- 172.21.15.116:0/1641229787 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf019b5d0 con 0x7f1cf0102ef0 2024-09-12T03:01:29.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.269+0000 7f1cedffb640 1 -- 172.21.15.116:0/1641229787 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0004d90 con 0x7f1cf0102ef0 2024-09-12T03:01:29.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.269+0000 7f1cf70d0640 1 -- 172.21.15.116:0/1641229787 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf01a2eb0 con 0x7f1cf0102ef0 2024-09-12T03:01:29.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.270+0000 7f1cf70d0640 1 -- 172.21.15.116:0/1641229787 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cf0101200 con 0x7f1cf0102ef0 2024-09-12T03:01:29.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.271+0000 7f1cedffb640 1 -- 172.21.15.116:0/1641229787 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f1ce0004f30 con 0x7f1cf0102ef0 2024-09-12T03:01:29.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.272+0000 7f1cedffb640 1 --2- 172.21.15.116:0/1641229787 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1cd80a4ea0 0x7f1cd80a7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:29.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.272+0000 7f1cf4e6c640 1 --2- 172.21.15.116:0/1641229787 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1cd80a4ea0 0x7f1cd80a7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:29.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.272+0000 7f1cedffb640 1 -- 172.21.15.116:0/1641229787 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f1ce00ce940 con 0x7f1cf0102ef0 2024-09-12T03:01:29.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.273+0000 7f1cf4e6c640 1 --2- 172.21.15.116:0/1641229787 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1cd80a4ea0 0x7f1cd80a7360 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8004ad0 tx=0x7f1ce8004a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:29.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.276+0000 7f1cedffb640 1 -- 172.21.15.116:0/1641229787 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1ce009b8f0 con 0x7f1cf0102ef0 2024-09-12T03:01:29.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.489+0000 7f1cf70d0640 1 -- 172.21.15.116:0/1641229787 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f1cf01a30e0 con 0x7f1cf0102ef0 2024-09-12T03:01:29.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.493+0000 7f1cedffb640 1 -- 172.21.15.116:0/1641229787 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263210 (secure 0 0 0) 0x7f1ce009b1f0 con 0x7f1cf0102ef0 2024-09-12T03:01:29.498 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:01:29.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.496+0000 7f1cf70d0640 1 -- 172.21.15.116:0/1641229787 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1cd80a4ea0 msgr2=0x7f1cd80a7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:29.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.496+0000 7f1cf70d0640 1 --2- 172.21.15.116:0/1641229787 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1cd80a4ea0 0x7f1cd80a7360 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8004ad0 tx=0x7f1ce8004a60 comp rx=0 tx=0).stop 2024-09-12T03:01:29.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.496+0000 7f1cf70d0640 1 -- 172.21.15.116:0/1641229787 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cf0102ef0 msgr2=0x7f1cf019ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:29.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.496+0000 7f1cf70d0640 1 --2- 172.21.15.116:0/1641229787 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cf0102ef0 0x7f1cf019ac00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000d750 tx=0x7f1ce000dc10 comp rx=0 tx=0).stop 2024-09-12T03:01:29.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.496+0000 7f1cf70d0640 1 -- 172.21.15.116:0/1641229787 shutdown_connections 2024-09-12T03:01:29.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.496+0000 7f1cf70d0640 1 --2- 172.21.15.116:0/1641229787 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1cd80a4ea0 0x7f1cd80a7360 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:29.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.496+0000 7f1cf70d0640 1 --2- 172.21.15.116:0/1641229787 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cf0102ef0 0x7f1cf019ac00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:29.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.496+0000 7f1cf70d0640 1 --2- 172.21.15.116:0/1641229787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cf0102570 0x7f1cf0196870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:29.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.496+0000 7f1cf70d0640 1 --2- 172.21.15.116:0/1641229787 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cf0100e20 0x7f1cf0196330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:29.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.496+0000 7f1cf70d0640 1 -- 172.21.15.116:0/1641229787 >> 172.21.15.116:0/1641229787 conn(0x7f1cf00fc7d0 msgr2=0x7f1cf00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:29.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.496+0000 7f1cf70d0640 1 -- 172.21.15.116:0/1641229787 shutdown_connections 2024-09-12T03:01:29.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:29.496+0000 7f1cf70d0640 1 -- 172.21.15.116:0/1641229787 wait complete. 2024-09-12T03:01:30.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:30 smithi183 ceph-mon[35535]: pgmap v291: 97 pgs: 97 active+clean; 636 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.7 KiB/s wr, 7 op/s 2024-09-12T03:01:30.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:30 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1641229787' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T03:01:30.460 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:30 smithi116 ceph-mon[31410]: pgmap v291: 97 pgs: 97 active+clean; 636 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.7 KiB/s wr, 7 op/s 2024-09-12T03:01:30.460 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:30 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1641229787' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T03:01:30.461 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":2896038008},{"type":"v1","addr":"172.21.15.116:6801","nonce":2896038008}]},"active_addr":"172.21.15.116:6801/2896038008","active_change":"2024-09-12T02:54:03.026986+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24115,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24122,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.116:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.116:0","nonce":3637480333}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.116:0","nonce":2736300694}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.116:0","nonce":3973599330}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.116:0","nonce":1731424106}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.116:0","nonce":1615031364}]}]} 2024-09-12T03:01:30.466 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-12T03:01:30.466 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-12T03:01:30.467 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:01:30.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:30 smithi177 ceph-mon[35305]: pgmap v291: 97 pgs: 97 active+clean; 636 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.7 KiB/s wr, 7 op/s 2024-09-12T03:01:30.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:30 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1641229787' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T03:01:30.711 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:31.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.187+0000 7f3e22c0d640 1 -- 172.21.15.116:0/3430420482 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e1c101420 msgr2=0x7f3e1c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:31.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.187+0000 7f3e22c0d640 1 --2- 172.21.15.116:0/3430420482 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e1c101420 0x7f3e1c10d950 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3e14009cd0 tx=0x7f3e1402a8e0 comp rx=0 tx=0).stop 2024-09-12T03:01:31.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.187+0000 7f3e22c0d640 1 -- 172.21.15.116:0/3430420482 shutdown_connections 2024-09-12T03:01:31.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.187+0000 7f3e22c0d640 1 --2- 172.21.15.116:0/3430420482 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e1c101420 0x7f3e1c10d950 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:31.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.187+0000 7f3e22c0d640 1 --2- 172.21.15.116:0/3430420482 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e1c100a80 0x7f3e1c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:31.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.187+0000 7f3e22c0d640 1 --2- 172.21.15.116:0/3430420482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e1c106a80 0x7f3e1c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:31.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.187+0000 7f3e22c0d640 1 -- 172.21.15.116:0/3430420482 >> 172.21.15.116:0/3430420482 conn(0x7f3e1c0fc7a0 msgr2=0x7f3e1c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:31.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.187+0000 7f3e22c0d640 1 -- 172.21.15.116:0/3430420482 shutdown_connections 2024-09-12T03:01:31.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.187+0000 7f3e22c0d640 1 -- 172.21.15.116:0/3430420482 wait complete. 2024-09-12T03:01:31.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.188+0000 7f3e22c0d640 1 Processor -- start 2024-09-12T03:01:31.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.188+0000 7f3e22c0d640 1 -- start start 2024-09-12T03:01:31.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.188+0000 7f3e22c0d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e1c100a80 0x7f3e1c0fcd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:31.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.188+0000 7f3e22c0d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e1c101420 0x7f3e1c0fd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:31.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.188+0000 7f3e22c0d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e1c106a80 0x7f3e1c0fd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:31.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.188+0000 7f3e22c0d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c110ec0 con 0x7f3e1c106a80 2024-09-12T03:01:31.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.188+0000 7f3e22c0d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c110d40 con 0x7f3e1c101420 2024-09-12T03:01:31.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.188+0000 7f3e22c0d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c111040 con 0x7f3e1c100a80 2024-09-12T03:01:31.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.188+0000 7f3e211aa640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e1c106a80 0x7f3e1c0fd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:31.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.188+0000 7f3e211aa640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e1c106a80 0x7f3e1c0fd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39464/0 (socket says 172.21.15.116:39464) 2024-09-12T03:01:31.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.188+0000 7f3e211aa640 1 -- 172.21.15.116:0/3314919966 learned_addr learned my addr 172.21.15.116:0/3314919966 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:31.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.188+0000 7f3e209a9640 1 --2- 172.21.15.116:0/3314919966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e1c100a80 0x7f3e1c0fcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:31.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.188+0000 7f3e1bfff640 1 --2- 172.21.15.116:0/3314919966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e1c101420 0x7f3e1c0fd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:31.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.188+0000 7f3e211aa640 1 -- 172.21.15.116:0/3314919966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e1c100a80 msgr2=0x7f3e1c0fcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:31.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.188+0000 7f3e211aa640 1 --2- 172.21.15.116:0/3314919966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e1c100a80 0x7f3e1c0fcd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:31.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.188+0000 7f3e211aa640 1 -- 172.21.15.116:0/3314919966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e1c101420 msgr2=0x7f3e1c0fd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:31.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.188+0000 7f3e211aa640 1 --2- 172.21.15.116:0/3314919966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e1c101420 0x7f3e1c0fd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:31.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.188+0000 7f3e211aa640 1 -- 172.21.15.116:0/3314919966 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e1c1a6950 con 0x7f3e1c106a80 2024-09-12T03:01:31.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.189+0000 7f3e211aa640 1 --2- 172.21.15.116:0/3314919966 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e1c106a80 0x7f3e1c0fd800 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3e14005cc0 tx=0x7f3e14003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:31.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.189+0000 7f3e19ffb640 1 -- 172.21.15.116:0/3314919966 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e14004040 con 0x7f3e1c106a80 2024-09-12T03:01:31.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.189+0000 7f3e19ffb640 1 -- 172.21.15.116:0/3314919966 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e140046c0 con 0x7f3e1c106a80 2024-09-12T03:01:31.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.189+0000 7f3e22c0d640 1 -- 172.21.15.116:0/3314919966 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e1c1a6be0 con 0x7f3e1c106a80 2024-09-12T03:01:31.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.189+0000 7f3e19ffb640 1 -- 172.21.15.116:0/3314919966 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e14046480 con 0x7f3e1c106a80 2024-09-12T03:01:31.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.189+0000 7f3e22c0d640 1 -- 172.21.15.116:0/3314919966 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e1c1a7170 con 0x7f3e1c106a80 2024-09-12T03:01:31.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.190+0000 7f3e22c0d640 1 -- 172.21.15.116:0/3314919966 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e1c100ee0 con 0x7f3e1c106a80 2024-09-12T03:01:31.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.191+0000 7f3e19ffb640 1 -- 172.21.15.116:0/3314919966 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f3e140305a0 con 0x7f3e1c106a80 2024-09-12T03:01:31.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.192+0000 7f3e19ffb640 1 --2- 172.21.15.116:0/3314919966 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3e0c0a2c90 0x7f3e0c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:31.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.192+0000 7f3e209a9640 1 --2- 172.21.15.116:0/3314919966 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3e0c0a2c90 0x7f3e0c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:31.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.192+0000 7f3e19ffb640 1 -- 172.21.15.116:0/3314919966 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f3e1402e030 con 0x7f3e1c106a80 2024-09-12T03:01:31.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.192+0000 7f3e209a9640 1 --2- 172.21.15.116:0/3314919966 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3e0c0a2c90 0x7f3e0c0a5150 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3e10005fd0 tx=0x7f3e1000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:31.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.196+0000 7f3e19ffb640 1 -- 172.21.15.116:0/3314919966 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3e1404b060 con 0x7f3e1c106a80 2024-09-12T03:01:31.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.363+0000 7f3e22c0d640 1 -- 172.21.15.116:0/3314919966 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e1c109a90 con 0x7f3e1c106a80 2024-09-12T03:01:31.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.364+0000 7f3e19ffb640 1 -- 172.21.15.116:0/3314919966 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22325 (secure 0 0 0) 0x7f3e140bd770 con 0x7f3e1c106a80 2024-09-12T03:01:31.365 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:01:31.366 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":78,"fsid":"eb1ac602-70b1-11ef-bcea-c7b262605968","created":"2024-09-12T02:53:07.105686+0000","modified":"2024-09-12T03:00:59.195124+0000","last_up_change":"2024-09-12T03:00:07.202639+0000","last_in_change":"2024-09-12T02:59:43.446336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:56:17.347152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:00:55.191040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:00:57.171142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8cf5a18-dc6f-4bb6-bd74-78adda310118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6803","nonce":862366853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6805","nonce":862366853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6809","nonce":862366853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6807","nonce":862366853}]},"public_addr":"172.21.15.116:6803/862366853","cluster_addr":"172.21.15.116:6805/862366853","heartbeat_back_addr":"172.21.15.116:6809/862366853","heartbeat_front_addr":"172.21.15.116:6807/862366853","state":["exists","up"]},{"osd":1,"uuid":"0affd3cb-9a51-42b0-a2cc-f30784a75cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6801","nonce":2245800547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6803","nonce":2245800547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6807","nonce":2245800547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6805","nonce":2245800547}]},"public_addr":"172.21.15.177:6801/2245800547","cluster_addr":"172.21.15.177:6803/2245800547","heartbeat_back_addr":"172.21.15.177:6807/2245800547","heartbeat_front_addr":"172.21.15.177:6805/2245800547","state":["exists","up"]},{"osd":2,"uuid":"1ce45ea5-84df-4a35-924f-78f92bd4f2cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6801","nonce":2645677759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6803","nonce":2645677759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6807","nonce":2645677759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6805","nonce":2645677759}]},"public_addr":"172.21.15.183:6801/2645677759","cluster_addr":"172.21.15.183:6803/2645677759","heartbeat_back_addr":"172.21.15.183:6807/2645677759","heartbeat_front_addr":"172.21.15.183:6805/2645677759","state":["exists","up"]},{"osd":3,"uuid":"ae5e8178-c998-4648-bc05-4bce822f0cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6811","nonce":686622148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6813","nonce":686622148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6817","nonce":686622148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6815","nonce":686622148}]},"public_addr":"172.21.15.116:6811/686622148","cluster_addr":"172.21.15.116:6813/686622148","heartbeat_back_addr":"172.21.15.116:6817/686622148","heartbeat_front_addr":"172.21.15.116:6815/686622148","state":["exists","up"]},{"osd":4,"uuid":"c81d2e73-0bc2-440d-b5a2-b7428af3ea4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6809","nonce":2382938021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6811","nonce":2382938021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6815","nonce":2382938021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6813","nonce":2382938021}]},"public_addr":"172.21.15.177:6809/2382938021","cluster_addr":"172.21.15.177:6811/2382938021","heartbeat_back_addr":"172.21.15.177:6815/2382938021","heartbeat_front_addr":"172.21.15.177:6813/2382938021","state":["exists","up"]},{"osd":5,"uuid":"c3f168be-9e27-4cf5-b73e-3f47d8990ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6809","nonce":85363432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6811","nonce":85363432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6815","nonce":85363432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6813","nonce":85363432}]},"public_addr":"172.21.15.183:6809/85363432","cluster_addr":"172.21.15.183:6811/85363432","heartbeat_back_addr":"172.21.15.183:6815/85363432","heartbeat_front_addr":"172.21.15.183:6813/85363432","state":["exists","up"]},{"osd":6,"uuid":"beb0a7e9-4909-47e2-9ee7-6951d4745e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6819","nonce":1121291186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6821","nonce":1121291186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6825","nonce":1121291186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6823","nonce":1121291186}]},"public_addr":"172.21.15.116:6819/1121291186","cluster_addr":"172.21.15.116:6821/1121291186","heartbeat_back_addr":"172.21.15.116:6825/1121291186","heartbeat_front_addr":"172.21.15.116:6823/1121291186","state":["exists","up"]},{"osd":7,"uuid":"761f4680-f049-473c-bf44-b19acf8de24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6817","nonce":2177665961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6819","nonce":2177665961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6823","nonce":2177665961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6821","nonce":2177665961}]},"public_addr":"172.21.15.177:6817/2177665961","cluster_addr":"172.21.15.177:6819/2177665961","heartbeat_back_addr":"172.21.15.177:6823/2177665961","heartbeat_front_addr":"172.21.15.177:6821/2177665961","state":["exists","up"]},{"osd":8,"uuid":"2385ea6c-fd5a-4815-9596-77f948629542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6817","nonce":1305725258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6819","nonce":1305725258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6823","nonce":1305725258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6821","nonce":1305725258}]},"public_addr":"172.21.15.183:6817/1305725258","cluster_addr":"172.21.15.183:6819/1305725258","heartbeat_back_addr":"172.21.15.183:6823/1305725258","heartbeat_front_addr":"172.21.15.183:6821/1305725258","state":["exists","up"]},{"osd":9,"uuid":"fd0c0dc4-4990-4f12-aa14-f855504cd995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6827","nonce":3831673668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6829","nonce":3831673668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6833","nonce":3831673668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6831","nonce":3831673668}]},"public_addr":"172.21.15.116:6827/3831673668","cluster_addr":"172.21.15.116:6829/3831673668","heartbeat_back_addr":"172.21.15.116:6833/3831673668","heartbeat_front_addr":"172.21.15.116:6831/3831673668","state":["exists","up"]},{"osd":10,"uuid":"864d1a97-4810-49c8-ad4a-a0c10253833c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6825","nonce":963663634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6827","nonce":963663634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6831","nonce":963663634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6829","nonce":963663634}]},"public_addr":"172.21.15.177:6825/963663634","cluster_addr":"172.21.15.177:6827/963663634","heartbeat_back_addr":"172.21.15.177:6831/963663634","heartbeat_front_addr":"172.21.15.177:6829/963663634","state":["exists","up"]},{"osd":11,"uuid":"d49c0841-4620-4f2f-a6b5-f2a9aa2e3c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6825","nonce":1818319939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6827","nonce":1818319939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6831","nonce":1818319939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6829","nonce":1818319939}]},"public_addr":"172.21.15.183:6825/1818319939","cluster_addr":"172.21.15.183:6827/1818319939","heartbeat_back_addr":"172.21.15.183:6831/1818319939","heartbeat_front_addr":"172.21.15.183:6829/1818319939","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:24.736944+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:50.310980+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:56:14.989533+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:56:41.187047+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:06.981664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:32.872771+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:58.850089+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:23.043121+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:47.839958+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:14.087708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:40.010548+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:05.364820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.116:6801/2966274423":"2024-09-13T02:54:03.026862+0000","172.21.15.116:6800/2966274423":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/686139455":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/1552047390":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/2180363411":"2024-09-13T02:53:39.433769+0000","172.21.15.116:6801/1791219601":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/3907400460":"2024-09-13T02:54:03.026862+0000","172.21.15.116:6800/1791219601":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/1802566286":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/212337802":"2024-09-13T02:53:39.433769+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:01:31.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.367+0000 7f3e22c0d640 1 -- 172.21.15.116:0/3314919966 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3e0c0a2c90 msgr2=0x7f3e0c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:31.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.367+0000 7f3e22c0d640 1 --2- 172.21.15.116:0/3314919966 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3e0c0a2c90 0x7f3e0c0a5150 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3e10005fd0 tx=0x7f3e1000b390 comp rx=0 tx=0).stop 2024-09-12T03:01:31.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.367+0000 7f3e22c0d640 1 -- 172.21.15.116:0/3314919966 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e1c106a80 msgr2=0x7f3e1c0fd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:31.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.367+0000 7f3e22c0d640 1 --2- 172.21.15.116:0/3314919966 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e1c106a80 0x7f3e1c0fd800 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3e14005cc0 tx=0x7f3e14003970 comp rx=0 tx=0).stop 2024-09-12T03:01:31.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.367+0000 7f3e22c0d640 1 -- 172.21.15.116:0/3314919966 shutdown_connections 2024-09-12T03:01:31.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.367+0000 7f3e22c0d640 1 --2- 172.21.15.116:0/3314919966 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3e0c0a2c90 0x7f3e0c0a5150 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:31.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.367+0000 7f3e22c0d640 1 --2- 172.21.15.116:0/3314919966 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e1c106a80 0x7f3e1c0fd800 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:31.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.367+0000 7f3e22c0d640 1 --2- 172.21.15.116:0/3314919966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e1c101420 0x7f3e1c0fd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:31.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.367+0000 7f3e22c0d640 1 --2- 172.21.15.116:0/3314919966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e1c100a80 0x7f3e1c0fcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:31.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.367+0000 7f3e22c0d640 1 -- 172.21.15.116:0/3314919966 >> 172.21.15.116:0/3314919966 conn(0x7f3e1c0fc7a0 msgr2=0x7f3e1c06a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:31.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.367+0000 7f3e22c0d640 1 -- 172.21.15.116:0/3314919966 shutdown_connections 2024-09-12T03:01:31.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:31.367+0000 7f3e22c0d640 1 -- 172.21.15.116:0/3314919966 wait complete. 2024-09-12T03:01:32.083 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-12T03:01:32.083 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:01:32.330 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:32.451 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:32 smithi116 ceph-mon[31410]: pgmap v292: 97 pgs: 97 active+clean; 636 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-09-12T03:01:32.451 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:32 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3314919966' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:01:32.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:32 smithi183 ceph-mon[35535]: pgmap v292: 97 pgs: 97 active+clean; 636 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-09-12T03:01:32.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:32 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3314919966' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:01:32.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:32 smithi177 ceph-mon[35305]: pgmap v292: 97 pgs: 97 active+clean; 636 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-09-12T03:01:32.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:32 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3314919966' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:01:32.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.832+0000 7f7697dff640 1 -- 172.21.15.116:0/1794001505 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7690101420 msgr2=0x7f769010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:32.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.832+0000 7f7697dff640 1 --2- 172.21.15.116:0/1794001505 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7690101420 0x7f769010d950 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7680009900 tx=0x7f768002a410 comp rx=0 tx=0).stop 2024-09-12T03:01:32.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.832+0000 7f7697dff640 1 -- 172.21.15.116:0/1794001505 shutdown_connections 2024-09-12T03:01:32.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.832+0000 7f7697dff640 1 --2- 172.21.15.116:0/1794001505 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7690101420 0x7f769010d950 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:32.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.832+0000 7f7697dff640 1 --2- 172.21.15.116:0/1794001505 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7690100a80 0x7f7690100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:32.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.832+0000 7f7697dff640 1 --2- 172.21.15.116:0/1794001505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7690106a80 0x7f7690106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:32.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.832+0000 7f7697dff640 1 -- 172.21.15.116:0/1794001505 >> 172.21.15.116:0/1794001505 conn(0x7f76900fc7a0 msgr2=0x7f76900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:32.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.832+0000 7f7697dff640 1 -- 172.21.15.116:0/1794001505 shutdown_connections 2024-09-12T03:01:32.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.832+0000 7f7697dff640 1 -- 172.21.15.116:0/1794001505 wait complete. 2024-09-12T03:01:32.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.833+0000 7f7697dff640 1 Processor -- start 2024-09-12T03:01:32.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.833+0000 7f7697dff640 1 -- start start 2024-09-12T03:01:32.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.833+0000 7f7697dff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7690100a80 0x7f7690198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:32.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.833+0000 7f7697dff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7690101420 0x7f7690198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:32.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.833+0000 7f7697dff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7690106a80 0x7f769019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:32.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.833+0000 7f7697dff640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7690112020 con 0x7f7690100a80 2024-09-12T03:01:32.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.833+0000 7f7697dff640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7690111ea0 con 0x7f7690106a80 2024-09-12T03:01:32.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.833+0000 7f7697dff640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f76901121a0 con 0x7f7690101420 2024-09-12T03:01:32.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.833+0000 7f7695b9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7690100a80 0x7f7690198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:32.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.833+0000 7f7695b9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7690100a80 0x7f7690198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39488/0 (socket says 172.21.15.116:39488) 2024-09-12T03:01:32.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.833+0000 7f7695b9b640 1 -- 172.21.15.116:0/3490573619 learned_addr learned my addr 172.21.15.116:0/3490573619 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:32.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.833+0000 7f769539a640 1 --2- 172.21.15.116:0/3490573619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7690101420 0x7f7690198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:32.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.833+0000 7f769639c640 1 --2- 172.21.15.116:0/3490573619 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7690106a80 0x7f769019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:32.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.833+0000 7f7695b9b640 1 -- 172.21.15.116:0/3490573619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7690101420 msgr2=0x7f7690198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:32.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.833+0000 7f7695b9b640 1 --2- 172.21.15.116:0/3490573619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7690101420 0x7f7690198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:32.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.833+0000 7f7695b9b640 1 -- 172.21.15.116:0/3490573619 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7690106a80 msgr2=0x7f769019ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:32.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.833+0000 7f7695b9b640 1 --2- 172.21.15.116:0/3490573619 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7690106a80 0x7f769019ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:32.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.833+0000 7f7695b9b640 1 -- 172.21.15.116:0/3490573619 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f769019d4f0 con 0x7f7690100a80 2024-09-12T03:01:32.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.833+0000 7f7695b9b640 1 --2- 172.21.15.116:0/3490573619 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7690100a80 0x7f7690198540 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f768800b4a0 tx=0x7f768800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:32.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.833+0000 7f768effd640 1 -- 172.21.15.116:0/3490573619 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7688014070 con 0x7f7690100a80 2024-09-12T03:01:32.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.833+0000 7f768effd640 1 -- 172.21.15.116:0/3490573619 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7688010af0 con 0x7f7690100a80 2024-09-12T03:01:32.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.833+0000 7f7697dff640 1 -- 172.21.15.116:0/3490573619 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f769019d7e0 con 0x7f7690100a80 2024-09-12T03:01:32.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.833+0000 7f768effd640 1 -- 172.21.15.116:0/3490573619 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7688017d10 con 0x7f7690100a80 2024-09-12T03:01:32.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.834+0000 7f7697dff640 1 -- 172.21.15.116:0/3490573619 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76901a50c0 con 0x7f7690100a80 2024-09-12T03:01:32.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.834+0000 7f7697dff640 1 -- 172.21.15.116:0/3490573619 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7690100f60 con 0x7f7690100a80 2024-09-12T03:01:32.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.836+0000 7f768effd640 1 -- 172.21.15.116:0/3490573619 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f7688017070 con 0x7f7690100a80 2024-09-12T03:01:32.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.836+0000 7f768effd640 1 --2- 172.21.15.116:0/3490573619 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f76780a2c10 0x7f76780a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:32.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.836+0000 7f769539a640 1 --2- 172.21.15.116:0/3490573619 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f76780a2c10 0x7f76780a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:32.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.837+0000 7f768effd640 1 -- 172.21.15.116:0/3490573619 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f76880ce400 con 0x7f7690100a80 2024-09-12T03:01:32.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.837+0000 7f769539a640 1 --2- 172.21.15.116:0/3490573619 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f76780a2c10 0x7f76780a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f76900fed80 tx=0x7f76840094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:32.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:32.841+0000 7f768effd640 1 -- 172.21.15.116:0/3490573619 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f768809b4c0 con 0x7f7690100a80 2024-09-12T03:01:33.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:33.007+0000 7f7697dff640 1 -- 172.21.15.116:0/3490573619 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7690109070 con 0x7f7690100a80 2024-09-12T03:01:33.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:33.009+0000 7f768effd640 1 -- 172.21.15.116:0/3490573619 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22325 (secure 0 0 0) 0x7f768809adc0 con 0x7f7690100a80 2024-09-12T03:01:33.009 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:01:33.009 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":78,"fsid":"eb1ac602-70b1-11ef-bcea-c7b262605968","created":"2024-09-12T02:53:07.105686+0000","modified":"2024-09-12T03:00:59.195124+0000","last_up_change":"2024-09-12T03:00:07.202639+0000","last_in_change":"2024-09-12T02:59:43.446336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:56:17.347152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:00:55.191040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:00:57.171142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8cf5a18-dc6f-4bb6-bd74-78adda310118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6803","nonce":862366853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6805","nonce":862366853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6809","nonce":862366853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6807","nonce":862366853}]},"public_addr":"172.21.15.116:6803/862366853","cluster_addr":"172.21.15.116:6805/862366853","heartbeat_back_addr":"172.21.15.116:6809/862366853","heartbeat_front_addr":"172.21.15.116:6807/862366853","state":["exists","up"]},{"osd":1,"uuid":"0affd3cb-9a51-42b0-a2cc-f30784a75cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6801","nonce":2245800547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6803","nonce":2245800547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6807","nonce":2245800547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6805","nonce":2245800547}]},"public_addr":"172.21.15.177:6801/2245800547","cluster_addr":"172.21.15.177:6803/2245800547","heartbeat_back_addr":"172.21.15.177:6807/2245800547","heartbeat_front_addr":"172.21.15.177:6805/2245800547","state":["exists","up"]},{"osd":2,"uuid":"1ce45ea5-84df-4a35-924f-78f92bd4f2cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6801","nonce":2645677759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6803","nonce":2645677759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6807","nonce":2645677759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6805","nonce":2645677759}]},"public_addr":"172.21.15.183:6801/2645677759","cluster_addr":"172.21.15.183:6803/2645677759","heartbeat_back_addr":"172.21.15.183:6807/2645677759","heartbeat_front_addr":"172.21.15.183:6805/2645677759","state":["exists","up"]},{"osd":3,"uuid":"ae5e8178-c998-4648-bc05-4bce822f0cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6811","nonce":686622148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6813","nonce":686622148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6817","nonce":686622148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6815","nonce":686622148}]},"public_addr":"172.21.15.116:6811/686622148","cluster_addr":"172.21.15.116:6813/686622148","heartbeat_back_addr":"172.21.15.116:6817/686622148","heartbeat_front_addr":"172.21.15.116:6815/686622148","state":["exists","up"]},{"osd":4,"uuid":"c81d2e73-0bc2-440d-b5a2-b7428af3ea4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6809","nonce":2382938021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6811","nonce":2382938021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6815","nonce":2382938021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6813","nonce":2382938021}]},"public_addr":"172.21.15.177:6809/2382938021","cluster_addr":"172.21.15.177:6811/2382938021","heartbeat_back_addr":"172.21.15.177:6815/2382938021","heartbeat_front_addr":"172.21.15.177:6813/2382938021","state":["exists","up"]},{"osd":5,"uuid":"c3f168be-9e27-4cf5-b73e-3f47d8990ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6809","nonce":85363432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6811","nonce":85363432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6815","nonce":85363432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6813","nonce":85363432}]},"public_addr":"172.21.15.183:6809/85363432","cluster_addr":"172.21.15.183:6811/85363432","heartbeat_back_addr":"172.21.15.183:6815/85363432","heartbeat_front_addr":"172.21.15.183:6813/85363432","state":["exists","up"]},{"osd":6,"uuid":"beb0a7e9-4909-47e2-9ee7-6951d4745e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6819","nonce":1121291186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6821","nonce":1121291186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6825","nonce":1121291186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6823","nonce":1121291186}]},"public_addr":"172.21.15.116:6819/1121291186","cluster_addr":"172.21.15.116:6821/1121291186","heartbeat_back_addr":"172.21.15.116:6825/1121291186","heartbeat_front_addr":"172.21.15.116:6823/1121291186","state":["exists","up"]},{"osd":7,"uuid":"761f4680-f049-473c-bf44-b19acf8de24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6817","nonce":2177665961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6819","nonce":2177665961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6823","nonce":2177665961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6821","nonce":2177665961}]},"public_addr":"172.21.15.177:6817/2177665961","cluster_addr":"172.21.15.177:6819/2177665961","heartbeat_back_addr":"172.21.15.177:6823/2177665961","heartbeat_front_addr":"172.21.15.177:6821/2177665961","state":["exists","up"]},{"osd":8,"uuid":"2385ea6c-fd5a-4815-9596-77f948629542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6817","nonce":1305725258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6819","nonce":1305725258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6823","nonce":1305725258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6821","nonce":1305725258}]},"public_addr":"172.21.15.183:6817/1305725258","cluster_addr":"172.21.15.183:6819/1305725258","heartbeat_back_addr":"172.21.15.183:6823/1305725258","heartbeat_front_addr":"172.21.15.183:6821/1305725258","state":["exists","up"]},{"osd":9,"uuid":"fd0c0dc4-4990-4f12-aa14-f855504cd995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6827","nonce":3831673668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6829","nonce":3831673668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6833","nonce":3831673668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6831","nonce":3831673668}]},"public_addr":"172.21.15.116:6827/3831673668","cluster_addr":"172.21.15.116:6829/3831673668","heartbeat_back_addr":"172.21.15.116:6833/3831673668","heartbeat_front_addr":"172.21.15.116:6831/3831673668","state":["exists","up"]},{"osd":10,"uuid":"864d1a97-4810-49c8-ad4a-a0c10253833c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6825","nonce":963663634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6827","nonce":963663634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6831","nonce":963663634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6829","nonce":963663634}]},"public_addr":"172.21.15.177:6825/963663634","cluster_addr":"172.21.15.177:6827/963663634","heartbeat_back_addr":"172.21.15.177:6831/963663634","heartbeat_front_addr":"172.21.15.177:6829/963663634","state":["exists","up"]},{"osd":11,"uuid":"d49c0841-4620-4f2f-a6b5-f2a9aa2e3c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6825","nonce":1818319939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6827","nonce":1818319939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6831","nonce":1818319939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6829","nonce":1818319939}]},"public_addr":"172.21.15.183:6825/1818319939","cluster_addr":"172.21.15.183:6827/1818319939","heartbeat_back_addr":"172.21.15.183:6831/1818319939","heartbeat_front_addr":"172.21.15.183:6829/1818319939","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:24.736944+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:50.310980+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:56:14.989533+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:56:41.187047+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:06.981664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:32.872771+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:58.850089+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:23.043121+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:47.839958+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:14.087708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:40.010548+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:05.364820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.116:6801/2966274423":"2024-09-13T02:54:03.026862+0000","172.21.15.116:6800/2966274423":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/686139455":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/1552047390":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/2180363411":"2024-09-13T02:53:39.433769+0000","172.21.15.116:6801/1791219601":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/3907400460":"2024-09-13T02:54:03.026862+0000","172.21.15.116:6800/1791219601":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/1802566286":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/212337802":"2024-09-13T02:53:39.433769+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:01:33.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:33.012+0000 7f7697dff640 1 -- 172.21.15.116:0/3490573619 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f76780a2c10 msgr2=0x7f76780a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:33.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:33.012+0000 7f7697dff640 1 --2- 172.21.15.116:0/3490573619 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f76780a2c10 0x7f76780a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f76900fed80 tx=0x7f76840094f0 comp rx=0 tx=0).stop 2024-09-12T03:01:33.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:33.012+0000 7f7697dff640 1 -- 172.21.15.116:0/3490573619 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7690100a80 msgr2=0x7f7690198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:33.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:33.012+0000 7f7697dff640 1 --2- 172.21.15.116:0/3490573619 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7690100a80 0x7f7690198540 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f768800b4a0 tx=0x7f768800b960 comp rx=0 tx=0).stop 2024-09-12T03:01:33.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:33.012+0000 7f7697dff640 1 -- 172.21.15.116:0/3490573619 shutdown_connections 2024-09-12T03:01:33.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:33.012+0000 7f7697dff640 1 --2- 172.21.15.116:0/3490573619 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f76780a2c10 0x7f76780a50d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:33.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:33.012+0000 7f7697dff640 1 --2- 172.21.15.116:0/3490573619 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7690106a80 0x7f769019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:33.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:33.012+0000 7f7697dff640 1 --2- 172.21.15.116:0/3490573619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7690101420 0x7f7690198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:33.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:33.012+0000 7f7697dff640 1 --2- 172.21.15.116:0/3490573619 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7690100a80 0x7f7690198540 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:33.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:33.012+0000 7f7697dff640 1 -- 172.21.15.116:0/3490573619 >> 172.21.15.116:0/3490573619 conn(0x7f76900fc7a0 msgr2=0x7f76900fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:33.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:33.012+0000 7f7697dff640 1 -- 172.21.15.116:0/3490573619 shutdown_connections 2024-09-12T03:01:33.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:33.012+0000 7f7697dff640 1 -- 172.21.15.116:0/3490573619 wait complete. 2024-09-12T03:01:33.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:33 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3490573619' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:01:33.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:33 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3490573619' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:01:33.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:33 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3490573619' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:01:33.884 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-12T03:01:33.884 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-12T03:01:33.884 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-12T03:01:33.885 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-12T03:01:33.885 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-12T03:01:33.886 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-12T03:01:33.886 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-12T03:01:33.886 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-12T03:01:33.887 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-12T03:01:33.887 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-12T03:01:33.888 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-12T03:01:33.888 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-12T03:01:34.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:34 smithi177 ceph-mon[35305]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-09-12T03:01:34.528 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:34.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:34 smithi116 ceph-mon[31410]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-09-12T03:01:34.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:34 smithi183 ceph-mon[35535]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-09-12T03:01:34.957 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:34.958 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:34.964 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:34.964 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:34.964 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:34.971 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:34.971 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:34.984 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:34.984 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:34.987 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:35.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.201+0000 7fe0345f5640 1 -- 172.21.15.116:0/2836116234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe02c101420 msgr2=0x7fe02c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:35.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.201+0000 7fe0345f5640 1 --2- 172.21.15.116:0/2836116234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe02c101420 0x7fe02c10d950 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe024009930 tx=0x7fe02402a420 comp rx=0 tx=0).stop 2024-09-12T03:01:35.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.201+0000 7fe0345f5640 1 -- 172.21.15.116:0/2836116234 shutdown_connections 2024-09-12T03:01:35.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.201+0000 7fe0345f5640 1 --2- 172.21.15.116:0/2836116234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe02c101420 0x7fe02c10d950 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:35.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.201+0000 7fe0345f5640 1 --2- 172.21.15.116:0/2836116234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe02c100a80 0x7fe02c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:35.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.201+0000 7fe0345f5640 1 --2- 172.21.15.116:0/2836116234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe02c106a80 0x7fe02c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:35.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.201+0000 7fe0345f5640 1 -- 172.21.15.116:0/2836116234 >> 172.21.15.116:0/2836116234 conn(0x7fe02c0fc7a0 msgr2=0x7fe02c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:35.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.201+0000 7fe0345f5640 1 -- 172.21.15.116:0/2836116234 shutdown_connections 2024-09-12T03:01:35.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.201+0000 7fe0345f5640 1 -- 172.21.15.116:0/2836116234 wait complete. 2024-09-12T03:01:35.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.202+0000 7fe0345f5640 1 Processor -- start 2024-09-12T03:01:35.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.202+0000 7fe0345f5640 1 -- start start 2024-09-12T03:01:35.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.202+0000 7fe0345f5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe02c100a80 0x7fe02c1973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:35.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.202+0000 7fe0345f5640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe02c101420 0x7fe02c1978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:35.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.202+0000 7fe0345f5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe02c106a80 0x7fe02c19bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:35.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.202+0000 7fe0345f5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe02c110ec0 con 0x7fe02c100a80 2024-09-12T03:01:35.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.202+0000 7fe0345f5640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe02c110d40 con 0x7fe02c101420 2024-09-12T03:01:35.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.202+0000 7fe0345f5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe02c111040 con 0x7fe02c106a80 2024-09-12T03:01:35.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.202+0000 7fe032391640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe02c100a80 0x7fe02c1973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:35.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.202+0000 7fe032391640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe02c100a80 0x7fe02c1973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39500/0 (socket says 172.21.15.116:39500) 2024-09-12T03:01:35.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.202+0000 7fe032391640 1 -- 172.21.15.116:0/3188792021 learned_addr learned my addr 172.21.15.116:0/3188792021 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:35.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.202+0000 7fe032391640 1 -- 172.21.15.116:0/3188792021 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe02c106a80 msgr2=0x7fe02c19bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:35.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.202+0000 7fe032391640 1 --2- 172.21.15.116:0/3188792021 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe02c106a80 0x7fe02c19bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:35.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.202+0000 7fe032391640 1 -- 172.21.15.116:0/3188792021 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe02c101420 msgr2=0x7fe02c1978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:35.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.202+0000 7fe031b90640 1 --2- 172.21.15.116:0/3188792021 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe02c101420 0x7fe02c1978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:35.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.202+0000 7fe032391640 1 --2- 172.21.15.116:0/3188792021 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe02c101420 0x7fe02c1978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:35.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.202+0000 7fe032391640 1 -- 172.21.15.116:0/3188792021 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe02c19c350 con 0x7fe02c100a80 2024-09-12T03:01:35.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.203+0000 7fe032391640 1 --2- 172.21.15.116:0/3188792021 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe02c100a80 0x7fe02c1973a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe02800cce0 tx=0x7fe02800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:35.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.203+0000 7fe0237fe640 1 -- 172.21.15.116:0/3188792021 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe028004b20 con 0x7fe02c100a80 2024-09-12T03:01:35.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.203+0000 7fe0237fe640 1 -- 172.21.15.116:0/3188792021 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe028004cc0 con 0x7fe02c100a80 2024-09-12T03:01:35.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.203+0000 7fe0237fe640 1 -- 172.21.15.116:0/3188792021 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02800f8c0 con 0x7fe02c100a80 2024-09-12T03:01:35.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.203+0000 7fe0345f5640 1 -- 172.21.15.116:0/3188792021 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe02c19c640 con 0x7fe02c100a80 2024-09-12T03:01:35.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.203+0000 7fe0345f5640 1 -- 172.21.15.116:0/3188792021 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe02c1a3f20 con 0x7fe02c100a80 2024-09-12T03:01:35.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.204+0000 7fe0345f5640 1 -- 172.21.15.116:0/3188792021 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe02c062940 con 0x7fe02c100a80 2024-09-12T03:01:35.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.205+0000 7fe0237fe640 1 -- 172.21.15.116:0/3188792021 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe028004e70 con 0x7fe02c100a80 2024-09-12T03:01:35.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.205+0000 7fe0237fe640 1 --2- 172.21.15.116:0/3188792021 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe0180a2c90 0x7fe0180a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:35.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.206+0000 7fe031b90640 1 --2- 172.21.15.116:0/3188792021 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe0180a2c90 0x7fe0180a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:35.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.206+0000 7fe0237fe640 1 -- 172.21.15.116:0/3188792021 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fe0280cf500 con 0x7fe02c100a80 2024-09-12T03:01:35.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.206+0000 7fe0237fe640 1 --2- 172.21.15.116:0/3188792021 >> [v2:172.21.15.116:6810/686622148,v1:172.21.15.116:6811/686622148] conn(0x7fe0180ac4e0 0x7fe0180ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:35.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.206+0000 7fe0237fe640 1 -- 172.21.15.116:0/3188792021 --> [v2:172.21.15.116:6810/686622148,v1:172.21.15.116:6811/686622148] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe0180aefb0 con 0x7fe0180ac4e0 2024-09-12T03:01:35.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.206+0000 7fe0237fe640 1 -- 172.21.15.116:0/3188792021 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7fe0280cf9e0 con 0x7fe02c100a80 2024-09-12T03:01:35.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.206+0000 7fe032b92640 1 --2- 172.21.15.116:0/3188792021 >> [v2:172.21.15.116:6810/686622148,v1:172.21.15.116:6811/686622148] conn(0x7fe0180ac4e0 0x7fe0180ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:35.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.206+0000 7fe031b90640 1 --2- 172.21.15.116:0/3188792021 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe0180a2c90 0x7fe0180a5150 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe02c0fedc0 tx=0x7fe01c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:35.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.206+0000 7fe032b92640 1 --2- 172.21.15.116:0/3188792021 >> [v2:172.21.15.116:6810/686622148,v1:172.21.15.116:6811/686622148] conn(0x7fe0180ac4e0 0x7fe0180ae900 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:35.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.207+0000 7fe0237fe640 1 -- 172.21.15.116:0/3188792021 <== osd.3 v2:172.21.15.116:6810/686622148 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe0180aefb0 con 0x7fe0180ac4e0 2024-09-12T03:01:35.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.222+0000 7fe0345f5640 1 -- 172.21.15.116:0/3188792021 --> [v2:172.21.15.116:6810/686622148,v1:172.21.15.116:6811/686622148] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe02c062dc0 con 0x7fe0180ac4e0 2024-09-12T03:01:35.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.222+0000 7fe0237fe640 1 -- 172.21.15.116:0/3188792021 <== osd.3 v2:172.21.15.116:6810/686622148 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe02c062dc0 con 0x7fe0180ac4e0 2024-09-12T03:01:35.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.222+0000 7fe0345f5640 1 -- 172.21.15.116:0/3188792021 >> [v2:172.21.15.116:6810/686622148,v1:172.21.15.116:6811/686622148] conn(0x7fe0180ac4e0 msgr2=0x7fe0180ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:35.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.222+0000 7fe0345f5640 1 --2- 172.21.15.116:0/3188792021 >> [v2:172.21.15.116:6810/686622148,v1:172.21.15.116:6811/686622148] conn(0x7fe0180ac4e0 0x7fe0180ae900 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:35.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.223+0000 7fe0345f5640 1 -- 172.21.15.116:0/3188792021 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe0180a2c90 msgr2=0x7fe0180a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:35.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.223+0000 7fe0345f5640 1 --2- 172.21.15.116:0/3188792021 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe0180a2c90 0x7fe0180a5150 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe02c0fedc0 tx=0x7fe01c008040 comp rx=0 tx=0).stop 2024-09-12T03:01:35.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.223+0000 7fe0345f5640 1 -- 172.21.15.116:0/3188792021 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe02c100a80 msgr2=0x7fe02c1973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:35.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.223+0000 7fe0345f5640 1 --2- 172.21.15.116:0/3188792021 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe02c100a80 0x7fe02c1973a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe02800cce0 tx=0x7fe02800a620 comp rx=0 tx=0).stop 2024-09-12T03:01:35.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.223+0000 7fe032b92640 1 -- 172.21.15.116:0/3188792021 reap_dead start 2024-09-12T03:01:35.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.223+0000 7fe0345f5640 1 -- 172.21.15.116:0/3188792021 shutdown_connections 2024-09-12T03:01:35.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.223+0000 7fe0345f5640 1 -- 172.21.15.116:0/3188792021 >> 172.21.15.116:0/3188792021 conn(0x7fe02c0fc7a0 msgr2=0x7fe02c0fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:35.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.223+0000 7fe0345f5640 1 -- 172.21.15.116:0/3188792021 shutdown_connections 2024-09-12T03:01:35.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:35.223+0000 7fe0345f5640 1 -- 172.21.15.116:0/3188792021 wait complete. 2024-09-12T03:01:35.226 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:36.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:36 smithi116 ceph-mon[31410]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-12T03:01:36.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:36 smithi177 ceph-mon[35305]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-12T03:01:36.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:36 smithi183 ceph-mon[35535]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-12T03:01:36.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.734+0000 7f7e2ff89640 1 -- 172.21.15.116:0/1497757609 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e28101280 msgr2=0x7f7e2810df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:36.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.734+0000 7f7e2ff89640 1 --2- 172.21.15.116:0/1497757609 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e28101280 0x7f7e2810df40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009900 tx=0x7f7e1802a410 comp rx=0 tx=0).stop 2024-09-12T03:01:36.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.734+0000 7f7e2ff89640 1 -- 172.21.15.116:0/1497757609 shutdown_connections 2024-09-12T03:01:36.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.734+0000 7f7e2ff89640 1 --2- 172.21.15.116:0/1497757609 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e2810e480 0x7f7e28110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:36.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.734+0000 7f7e2ff89640 1 --2- 172.21.15.116:0/1497757609 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e28101280 0x7f7e2810df40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:36.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.734+0000 7f7e2ff89640 1 --2- 172.21.15.116:0/1497757609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e281008d0 0x7f7e28100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:36.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.734+0000 7f7e2ff89640 1 -- 172.21.15.116:0/1497757609 >> 172.21.15.116:0/1497757609 conn(0x7f7e280fc7a0 msgr2=0x7f7e280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:36.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.734+0000 7f7e2ff89640 1 -- 172.21.15.116:0/1497757609 shutdown_connections 2024-09-12T03:01:36.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.734+0000 7f7e2ff89640 1 -- 172.21.15.116:0/1497757609 wait complete. 2024-09-12T03:01:36.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.735+0000 7f7e2ff89640 1 Processor -- start 2024-09-12T03:01:36.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.735+0000 7f7e2ff89640 1 -- start start 2024-09-12T03:01:36.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.735+0000 7f7e2ff89640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e281008d0 0x7f7e2819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:36.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.735+0000 7f7e2ff89640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e28101280 0x7f7e2819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:36.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.735+0000 7f7e2ff89640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e2810e480 0x7f7e2819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:36.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.735+0000 7f7e2ff89640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7e281141b0 con 0x7f7e281008d0 2024-09-12T03:01:36.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.735+0000 7f7e2ff89640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7e28114030 con 0x7f7e2810e480 2024-09-12T03:01:36.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.735+0000 7f7e2ff89640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7e28114330 con 0x7f7e28101280 2024-09-12T03:01:36.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.735+0000 7f7e2dd25640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e281008d0 0x7f7e2819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:36.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.735+0000 7f7e2dd25640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e281008d0 0x7f7e2819a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39522/0 (socket says 172.21.15.116:39522) 2024-09-12T03:01:36.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.735+0000 7f7e2dd25640 1 -- 172.21.15.116:0/2067816902 learned_addr learned my addr 172.21.15.116:0/2067816902 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:36.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.735+0000 7f7e2e526640 1 --2- 172.21.15.116:0/2067816902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e2810e480 0x7f7e2819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:36.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.735+0000 7f7e2d524640 1 --2- 172.21.15.116:0/2067816902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e28101280 0x7f7e2819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:36.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.735+0000 7f7e2dd25640 1 -- 172.21.15.116:0/2067816902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e28101280 msgr2=0x7f7e2819abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:36.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.735+0000 7f7e2dd25640 1 --2- 172.21.15.116:0/2067816902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e28101280 0x7f7e2819abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:36.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.735+0000 7f7e2dd25640 1 -- 172.21.15.116:0/2067816902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e2810e480 msgr2=0x7f7e2819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:36.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.735+0000 7f7e2dd25640 1 --2- 172.21.15.116:0/2067816902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e2810e480 0x7f7e2819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:36.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.735+0000 7f7e2dd25640 1 -- 172.21.15.116:0/2067816902 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2819f660 con 0x7f7e281008d0 2024-09-12T03:01:36.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.735+0000 7f7e2e526640 1 --2- 172.21.15.116:0/2067816902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e2810e480 0x7f7e2819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:01:36.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.736+0000 7f7e2dd25640 1 --2- 172.21.15.116:0/2067816902 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e281008d0 0x7f7e2819a6b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400e9f0 tx=0x7f7e2400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:36.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.736+0000 7f7e1effd640 1 -- 172.21.15.116:0/2067816902 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2400cd80 con 0x7f7e281008d0 2024-09-12T03:01:36.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.736+0000 7f7e2ff89640 1 -- 172.21.15.116:0/2067816902 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e2819f950 con 0x7f7e281008d0 2024-09-12T03:01:36.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.736+0000 7f7e1effd640 1 -- 172.21.15.116:0/2067816902 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e24005500 con 0x7f7e281008d0 2024-09-12T03:01:36.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.736+0000 7f7e1effd640 1 -- 172.21.15.116:0/2067816902 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e24010690 con 0x7f7e281008d0 2024-09-12T03:01:36.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.736+0000 7f7e2ff89640 1 -- 172.21.15.116:0/2067816902 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e281a7230 con 0x7f7e281008d0 2024-09-12T03:01:36.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.737+0000 7f7e2ff89640 1 -- 172.21.15.116:0/2067816902 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7e28062980 con 0x7f7e281008d0 2024-09-12T03:01:36.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.738+0000 7f7e1effd640 1 -- 172.21.15.116:0/2067816902 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f7e240046c0 con 0x7f7e281008d0 2024-09-12T03:01:36.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.739+0000 7f7e1effd640 1 --2- 172.21.15.116:0/2067816902 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7e140a2c10 0x7f7e140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:36.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.739+0000 7f7e2d524640 1 --2- 172.21.15.116:0/2067816902 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7e140a2c10 0x7f7e140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:36.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.739+0000 7f7e1effd640 1 -- 172.21.15.116:0/2067816902 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f7e240cf9f0 con 0x7f7e281008d0 2024-09-12T03:01:36.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.739+0000 7f7e1effd640 1 --2- 172.21.15.116:0/2067816902 >> [v2:172.21.15.177:6816/2177665961,v1:172.21.15.177:6817/2177665961] conn(0x7f7e140ac460 0x7f7e140ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:36.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.739+0000 7f7e1effd640 1 -- 172.21.15.116:0/2067816902 --> [v2:172.21.15.177:6816/2177665961,v1:172.21.15.177:6817/2177665961] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7e24004470 con 0x7f7e140ac460 2024-09-12T03:01:36.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.739+0000 7f7e1effd640 1 -- 172.21.15.116:0/2067816902 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f7e240cfed0 con 0x7f7e281008d0 2024-09-12T03:01:36.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.739+0000 7f7e2d524640 1 --2- 172.21.15.116:0/2067816902 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7e140a2c10 0x7f7e140a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7e2810ed80 tx=0x7f7e18005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:36.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.739+0000 7f7e2e526640 1 --2- 172.21.15.116:0/2067816902 >> [v2:172.21.15.177:6816/2177665961,v1:172.21.15.177:6817/2177665961] conn(0x7f7e140ac460 0x7f7e140ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:36.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.740+0000 7f7e2e526640 1 --2- 172.21.15.116:0/2067816902 >> [v2:172.21.15.177:6816/2177665961,v1:172.21.15.177:6817/2177665961] conn(0x7f7e140ac460 0x7f7e140ae880 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:36.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.740+0000 7f7e1effd640 1 -- 172.21.15.116:0/2067816902 <== osd.7 v2:172.21.15.177:6816/2177665961 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7e24004470 con 0x7f7e140ac460 2024-09-12T03:01:36.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.755+0000 7f7e2ff89640 1 -- 172.21.15.116:0/2067816902 --> [v2:172.21.15.177:6816/2177665961,v1:172.21.15.177:6817/2177665961] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7e28062e00 con 0x7f7e140ac460 2024-09-12T03:01:36.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.755+0000 7f7e1effd640 1 -- 172.21.15.116:0/2067816902 <== osd.7 v2:172.21.15.177:6816/2177665961 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7e28062e00 con 0x7f7e140ac460 2024-09-12T03:01:36.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.756+0000 7f7e2ff89640 1 -- 172.21.15.116:0/2067816902 >> [v2:172.21.15.177:6816/2177665961,v1:172.21.15.177:6817/2177665961] conn(0x7f7e140ac460 msgr2=0x7f7e140ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:36.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.756+0000 7f7e2ff89640 1 --2- 172.21.15.116:0/2067816902 >> [v2:172.21.15.177:6816/2177665961,v1:172.21.15.177:6817/2177665961] conn(0x7f7e140ac460 0x7f7e140ae880 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:36.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.756+0000 7f7e2ff89640 1 -- 172.21.15.116:0/2067816902 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7e140a2c10 msgr2=0x7f7e140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:36.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.756+0000 7f7e2ff89640 1 --2- 172.21.15.116:0/2067816902 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7e140a2c10 0x7f7e140a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7e2810ed80 tx=0x7f7e18005a20 comp rx=0 tx=0).stop 2024-09-12T03:01:36.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.756+0000 7f7e2ff89640 1 -- 172.21.15.116:0/2067816902 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e281008d0 msgr2=0x7f7e2819a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:36.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.756+0000 7f7e2ff89640 1 --2- 172.21.15.116:0/2067816902 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e281008d0 0x7f7e2819a6b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400e9f0 tx=0x7f7e2400edf0 comp rx=0 tx=0).stop 2024-09-12T03:01:36.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.756+0000 7f7e2e526640 1 -- 172.21.15.116:0/2067816902 reap_dead start 2024-09-12T03:01:36.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.756+0000 7f7e2ff89640 1 -- 172.21.15.116:0/2067816902 shutdown_connections 2024-09-12T03:01:36.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.756+0000 7f7e2ff89640 1 -- 172.21.15.116:0/2067816902 >> 172.21.15.116:0/2067816902 conn(0x7f7e280fc7a0 msgr2=0x7f7e280feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:36.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.756+0000 7f7e2ff89640 1 -- 172.21.15.116:0/2067816902 shutdown_connections 2024-09-12T03:01:36.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.756+0000 7f7e2ff89640 1 -- 172.21.15.116:0/2067816902 wait complete. 2024-09-12T03:01:37.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.616+0000 7fc33cb02640 1 -- 172.21.15.116:0/2105564532 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc338101420 msgr2=0x7fc33810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:37.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.616+0000 7fc33cb02640 1 --2- 172.21.15.116:0/2105564532 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc338101420 0x7fc33810d950 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc330009cd0 tx=0x7fc33002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:01:37.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.616+0000 7fc33cb02640 1 -- 172.21.15.116:0/2105564532 shutdown_connections 2024-09-12T03:01:37.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.616+0000 7fc33cb02640 1 --2- 172.21.15.116:0/2105564532 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc338101420 0x7fc33810d950 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:37.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.616+0000 7fc33cb02640 1 --2- 172.21.15.116:0/2105564532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc338100a80 0x7fc338100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:37.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.616+0000 7fc33cb02640 1 --2- 172.21.15.116:0/2105564532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc338106a80 0x7fc338106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:37.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.616+0000 7fc33cb02640 1 -- 172.21.15.116:0/2105564532 >> 172.21.15.116:0/2105564532 conn(0x7fc3380fc7a0 msgr2=0x7fc3380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:37.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.616+0000 7fc33cb02640 1 -- 172.21.15.116:0/2105564532 shutdown_connections 2024-09-12T03:01:37.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.616+0000 7fc33cb02640 1 -- 172.21.15.116:0/2105564532 wait complete. 2024-09-12T03:01:37.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.617+0000 7fc33cb02640 1 Processor -- start 2024-09-12T03:01:37.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.617+0000 7fc33cb02640 1 -- start start 2024-09-12T03:01:37.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.617+0000 7fc33cb02640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc338100a80 0x7fc3381992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:37.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.617+0000 7fc33cb02640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc338101420 0x7fc338199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:37.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.617+0000 7fc33cb02640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc338106a80 0x7fc3381a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:37.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.617+0000 7fc33cb02640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc338110ea0 con 0x7fc338106a80 2024-09-12T03:01:37.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.617+0000 7fc33cb02640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc338110d20 con 0x7fc338100a80 2024-09-12T03:01:37.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.617+0000 7fc33cb02640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc338111020 con 0x7fc338101420 2024-09-12T03:01:37.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.618+0000 7fc336d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc338106a80 0x7fc3381a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:37.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.618+0000 7fc336d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc338106a80 0x7fc3381a4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39512/0 (socket says 172.21.15.116:39512) 2024-09-12T03:01:37.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.618+0000 7fc336d9d640 1 -- 172.21.15.116:0/1806361254 learned_addr learned my addr 172.21.15.116:0/1806361254 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:37.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.618+0000 7fc335d9b640 1 --2- 172.21.15.116:0/1806361254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc338101420 0x7fc338199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:37.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.618+0000 7fc33659c640 1 --2- 172.21.15.116:0/1806361254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc338100a80 0x7fc3381992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:37.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.618+0000 7fc336d9d640 1 -- 172.21.15.116:0/1806361254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc338101420 msgr2=0x7fc338199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:37.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.618+0000 7fc336d9d640 1 --2- 172.21.15.116:0/1806361254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc338101420 0x7fc338199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:37.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.618+0000 7fc336d9d640 1 -- 172.21.15.116:0/1806361254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc338100a80 msgr2=0x7fc3381992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:37.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.618+0000 7fc336d9d640 1 --2- 172.21.15.116:0/1806361254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc338100a80 0x7fc3381992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:37.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.618+0000 7fc336d9d640 1 -- 172.21.15.116:0/1806361254 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3381a48b0 con 0x7fc338106a80 2024-09-12T03:01:37.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.618+0000 7fc33659c640 1 --2- 172.21.15.116:0/1806361254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc338100a80 0x7fc3381992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:01:37.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.618+0000 7fc336d9d640 1 --2- 172.21.15.116:0/1806361254 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc338106a80 0x7fc3381a4370 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc330006010 tx=0x7fc330003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:37.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.618+0000 7fc3277fe640 1 -- 172.21.15.116:0/1806361254 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc330004450 con 0x7fc338106a80 2024-09-12T03:01:37.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.618+0000 7fc33cb02640 1 -- 172.21.15.116:0/1806361254 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3381a4a80 con 0x7fc338106a80 2024-09-12T03:01:37.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.618+0000 7fc3277fe640 1 -- 172.21.15.116:0/1806361254 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc330003e10 con 0x7fc338106a80 2024-09-12T03:01:37.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.618+0000 7fc3277fe640 1 -- 172.21.15.116:0/1806361254 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3300466e0 con 0x7fc338106a80 2024-09-12T03:01:37.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.618+0000 7fc33cb02640 1 -- 172.21.15.116:0/1806361254 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3381a4f30 con 0x7fc338106a80 2024-09-12T03:01:37.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.619+0000 7fc33cb02640 1 -- 172.21.15.116:0/1806361254 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc338062900 con 0x7fc338106a80 2024-09-12T03:01:37.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.620+0000 7fc3277fe640 1 -- 172.21.15.116:0/1806361254 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc3300305a0 con 0x7fc338106a80 2024-09-12T03:01:37.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.621+0000 7fc3277fe640 1 --2- 172.21.15.116:0/1806361254 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc3200a2c10 0x7fc3200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:37.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.621+0000 7fc33659c640 1 --2- 172.21.15.116:0/1806361254 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc3200a2c10 0x7fc3200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:37.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.621+0000 7fc3277fe640 1 -- 172.21.15.116:0/1806361254 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fc33002e030 con 0x7fc338106a80 2024-09-12T03:01:37.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.621+0000 7fc3277fe640 1 --2- 172.21.15.116:0/1806361254 >> [v2:172.21.15.177:6800/2245800547,v1:172.21.15.177:6801/2245800547] conn(0x7fc3200ac460 0x7fc3200ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:37.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.621+0000 7fc3277fe640 1 -- 172.21.15.116:0/1806361254 --> [v2:172.21.15.177:6800/2245800547,v1:172.21.15.177:6801/2245800547] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc3200aef30 con 0x7fc3200ac460 2024-09-12T03:01:37.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.621+0000 7fc3277fe640 1 -- 172.21.15.116:0/1806361254 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7fc3300f0fc0 con 0x7fc338106a80 2024-09-12T03:01:37.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.621+0000 7fc33659c640 1 --2- 172.21.15.116:0/1806361254 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc3200a2c10 0x7fc3200a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc3280058c0 tx=0x7fc328005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:37.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.622+0000 7fc335d9b640 1 --2- 172.21.15.116:0/1806361254 >> [v2:172.21.15.177:6800/2245800547,v1:172.21.15.177:6801/2245800547] conn(0x7fc3200ac460 0x7fc3200ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:37.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.622+0000 7fc335d9b640 1 --2- 172.21.15.116:0/1806361254 >> [v2:172.21.15.177:6800/2245800547,v1:172.21.15.177:6801/2245800547] conn(0x7fc3200ac460 0x7fc3200ae880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:37.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.623+0000 7fc3277fe640 1 -- 172.21.15.116:0/1806361254 <== osd.1 v2:172.21.15.177:6800/2245800547 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc3200aef30 con 0x7fc3200ac460 2024-09-12T03:01:37.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.636+0000 7fc33cb02640 1 -- 172.21.15.116:0/1806361254 --> [v2:172.21.15.177:6800/2245800547,v1:172.21.15.177:6801/2245800547] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc338062d80 con 0x7fc3200ac460 2024-09-12T03:01:37.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.637+0000 7fc3277fe640 1 -- 172.21.15.116:0/1806361254 <== osd.1 v2:172.21.15.177:6800/2245800547 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc338062d80 con 0x7fc3200ac460 2024-09-12T03:01:37.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.637+0000 7fc33cb02640 1 -- 172.21.15.116:0/1806361254 >> [v2:172.21.15.177:6800/2245800547,v1:172.21.15.177:6801/2245800547] conn(0x7fc3200ac460 msgr2=0x7fc3200ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:37.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.637+0000 7fc33cb02640 1 --2- 172.21.15.116:0/1806361254 >> [v2:172.21.15.177:6800/2245800547,v1:172.21.15.177:6801/2245800547] conn(0x7fc3200ac460 0x7fc3200ae880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:37.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.637+0000 7fc33cb02640 1 -- 172.21.15.116:0/1806361254 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc3200a2c10 msgr2=0x7fc3200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:37.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.637+0000 7fc33cb02640 1 --2- 172.21.15.116:0/1806361254 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc3200a2c10 0x7fc3200a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc3280058c0 tx=0x7fc328005d30 comp rx=0 tx=0).stop 2024-09-12T03:01:37.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.637+0000 7fc33cb02640 1 -- 172.21.15.116:0/1806361254 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc338106a80 msgr2=0x7fc3381a4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:37.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.637+0000 7fc33cb02640 1 --2- 172.21.15.116:0/1806361254 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc338106a80 0x7fc3381a4370 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc330006010 tx=0x7fc330003c20 comp rx=0 tx=0).stop 2024-09-12T03:01:37.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.637+0000 7fc336d9d640 1 -- 172.21.15.116:0/1806361254 reap_dead start 2024-09-12T03:01:37.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.637+0000 7fc33cb02640 1 -- 172.21.15.116:0/1806361254 shutdown_connections 2024-09-12T03:01:37.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.637+0000 7fc33cb02640 1 -- 172.21.15.116:0/1806361254 >> 172.21.15.116:0/1806361254 conn(0x7fc3380fc7a0 msgr2=0x7fc3380fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:37.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.637+0000 7fc33cb02640 1 -- 172.21.15.116:0/1806361254 shutdown_connections 2024-09-12T03:01:37.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:36.637+0000 7fc33cb02640 1 -- 172.21.15.116:0/1806361254 wait complete. 2024-09-12T03:01:38.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.085+0000 7f5e14641640 1 -- 172.21.15.116:0/4191232432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e0c102df0 msgr2=0x7f5e0c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:38.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.085+0000 7f5e14641640 1 --2- 172.21.15.116:0/4191232432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e0c102df0 0x7f5e0c1031d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5e00009900 tx=0x7f5e0002a410 comp rx=0 tx=0).stop 2024-09-12T03:01:38.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.085+0000 7f5e14641640 1 -- 172.21.15.116:0/4191232432 shutdown_connections 2024-09-12T03:01:38.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.085+0000 7f5e14641640 1 --2- 172.21.15.116:0/4191232432 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e0c10c2f0 0x7f5e0c10e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:38.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.085+0000 7f5e14641640 1 --2- 172.21.15.116:0/4191232432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e0c1037a0 0x7f5e0c10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:38.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.085+0000 7f5e14641640 1 --2- 172.21.15.116:0/4191232432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e0c102df0 0x7f5e0c1031d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:38.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.085+0000 7f5e14641640 1 -- 172.21.15.116:0/4191232432 >> 172.21.15.116:0/4191232432 conn(0x7f5e0c0fc7d0 msgr2=0x7f5e0c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:38.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.085+0000 7f5e14641640 1 -- 172.21.15.116:0/4191232432 shutdown_connections 2024-09-12T03:01:38.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.085+0000 7f5e14641640 1 -- 172.21.15.116:0/4191232432 wait complete. 2024-09-12T03:01:38.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.086+0000 7f5e14641640 1 Processor -- start 2024-09-12T03:01:38.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.086+0000 7f5e14641640 1 -- start start 2024-09-12T03:01:38.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.086+0000 7f5e14641640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e0c102df0 0x7f5e0c198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:38.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.086+0000 7f5e14641640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e0c1037a0 0x7f5e0c198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:38.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.086+0000 7f5e14641640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e0c10c2f0 0x7f5e0c19cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:38.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.086+0000 7f5e14641640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c112110 con 0x7f5e0c1037a0 2024-09-12T03:01:38.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.086+0000 7f5e14641640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c111f90 con 0x7f5e0c10c2f0 2024-09-12T03:01:38.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.086+0000 7f5e14641640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c112290 con 0x7f5e0c102df0 2024-09-12T03:01:38.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.086+0000 7f5e11bdc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e0c1037a0 0x7f5e0c198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:38.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.086+0000 7f5e11bdc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e0c1037a0 0x7f5e0c198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39534/0 (socket says 172.21.15.116:39534) 2024-09-12T03:01:38.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.086+0000 7f5e11bdc640 1 -- 172.21.15.116:0/1903519709 learned_addr learned my addr 172.21.15.116:0/1903519709 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:38.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.086+0000 7f5e123dd640 1 --2- 172.21.15.116:0/1903519709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e0c102df0 0x7f5e0c198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:38.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.086+0000 7f5e12bde640 1 --2- 172.21.15.116:0/1903519709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e0c10c2f0 0x7f5e0c19cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:38.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.086+0000 7f5e11bdc640 1 -- 172.21.15.116:0/1903519709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e0c102df0 msgr2=0x7f5e0c198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:38.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.086+0000 7f5e11bdc640 1 --2- 172.21.15.116:0/1903519709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e0c102df0 0x7f5e0c198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:38.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.086+0000 7f5e11bdc640 1 -- 172.21.15.116:0/1903519709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e0c10c2f0 msgr2=0x7f5e0c19cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:38.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.086+0000 7f5e11bdc640 1 --2- 172.21.15.116:0/1903519709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e0c10c2f0 0x7f5e0c19cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:38.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.086+0000 7f5e11bdc640 1 -- 172.21.15.116:0/1903519709 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0c19d5e0 con 0x7f5e0c1037a0 2024-09-12T03:01:38.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.087+0000 7f5e11bdc640 1 --2- 172.21.15.116:0/1903519709 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e0c1037a0 0x7f5e0c198b70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5e0800c970 tx=0x7f5e0800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:38.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.087+0000 7f5dff7fe640 1 -- 172.21.15.116:0/1903519709 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e08004220 con 0x7f5e0c1037a0 2024-09-12T03:01:38.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.087+0000 7f5e14641640 1 -- 172.21.15.116:0/1903519709 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e0c19d8d0 con 0x7f5e0c1037a0 2024-09-12T03:01:38.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.087+0000 7f5dff7fe640 1 -- 172.21.15.116:0/1903519709 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e080043c0 con 0x7f5e0c1037a0 2024-09-12T03:01:38.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.087+0000 7f5dff7fe640 1 -- 172.21.15.116:0/1903519709 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e08004a00 con 0x7f5e0c1037a0 2024-09-12T03:01:38.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.087+0000 7f5e14641640 1 -- 172.21.15.116:0/1903519709 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e0c1a51b0 con 0x7f5e0c1037a0 2024-09-12T03:01:38.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.088+0000 7f5e14641640 1 -- 172.21.15.116:0/1903519709 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5e0c062900 con 0x7f5e0c1037a0 2024-09-12T03:01:38.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.089+0000 7f5dff7fe640 1 -- 172.21.15.116:0/1903519709 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f5e080079b0 con 0x7f5e0c1037a0 2024-09-12T03:01:38.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.089+0000 7f5dff7fe640 1 --2- 172.21.15.116:0/1903519709 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5df80a2c10 0x7f5df80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:38.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.089+0000 7f5e123dd640 1 --2- 172.21.15.116:0/1903519709 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5df80a2c10 0x7f5df80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:38.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.089+0000 7f5dff7fe640 1 -- 172.21.15.116:0/1903519709 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f5e080cf160 con 0x7f5e0c1037a0 2024-09-12T03:01:38.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.090+0000 7f5dff7fe640 1 --2- 172.21.15.116:0/1903519709 >> [v2:172.21.15.183:6808/85363432,v1:172.21.15.183:6809/85363432] conn(0x7f5df80ac460 0x7f5df80ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:38.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.090+0000 7f5dff7fe640 1 -- 172.21.15.116:0/1903519709 --> [v2:172.21.15.183:6808/85363432,v1:172.21.15.183:6809/85363432] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5e08023ef0 con 0x7f5df80ac460 2024-09-12T03:01:38.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.090+0000 7f5dff7fe640 1 -- 172.21.15.116:0/1903519709 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f5e080cf640 con 0x7f5e0c1037a0 2024-09-12T03:01:38.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.090+0000 7f5e123dd640 1 --2- 172.21.15.116:0/1903519709 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5df80a2c10 0x7f5df80a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5e00009230 tx=0x7f5e000058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:38.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.090+0000 7f5e12bde640 1 --2- 172.21.15.116:0/1903519709 >> [v2:172.21.15.183:6808/85363432,v1:172.21.15.183:6809/85363432] conn(0x7f5df80ac460 0x7f5df80ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:38.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.090+0000 7f5e12bde640 1 --2- 172.21.15.116:0/1903519709 >> [v2:172.21.15.183:6808/85363432,v1:172.21.15.183:6809/85363432] conn(0x7f5df80ac460 0x7f5df80ae880 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:38.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.091+0000 7f5dff7fe640 1 -- 172.21.15.116:0/1903519709 <== osd.5 v2:172.21.15.183:6808/85363432 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5e08023ef0 con 0x7f5df80ac460 2024-09-12T03:01:38.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.104+0000 7f5e14641640 1 -- 172.21.15.116:0/1903519709 --> [v2:172.21.15.183:6808/85363432,v1:172.21.15.183:6809/85363432] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5e0c062d80 con 0x7f5df80ac460 2024-09-12T03:01:38.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.105+0000 7f5dff7fe640 1 -- 172.21.15.116:0/1903519709 <== osd.5 v2:172.21.15.183:6808/85363432 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5e0c062d80 con 0x7f5df80ac460 2024-09-12T03:01:38.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.105+0000 7f5e14641640 1 -- 172.21.15.116:0/1903519709 >> [v2:172.21.15.183:6808/85363432,v1:172.21.15.183:6809/85363432] conn(0x7f5df80ac460 msgr2=0x7f5df80ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:38.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.105+0000 7f5e14641640 1 --2- 172.21.15.116:0/1903519709 >> [v2:172.21.15.183:6808/85363432,v1:172.21.15.183:6809/85363432] conn(0x7f5df80ac460 0x7f5df80ae880 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:38.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.105+0000 7f5e14641640 1 -- 172.21.15.116:0/1903519709 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5df80a2c10 msgr2=0x7f5df80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:38.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.105+0000 7f5e14641640 1 --2- 172.21.15.116:0/1903519709 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5df80a2c10 0x7f5df80a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5e00009230 tx=0x7f5e000058d0 comp rx=0 tx=0).stop 2024-09-12T03:01:38.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.105+0000 7f5e14641640 1 -- 172.21.15.116:0/1903519709 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e0c1037a0 msgr2=0x7f5e0c198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:38.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.105+0000 7f5e14641640 1 --2- 172.21.15.116:0/1903519709 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e0c1037a0 0x7f5e0c198b70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5e0800c970 tx=0x7f5e0800ce30 comp rx=0 tx=0).stop 2024-09-12T03:01:38.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.105+0000 7f5e12bde640 1 -- 172.21.15.116:0/1903519709 reap_dead start 2024-09-12T03:01:38.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.105+0000 7f5e14641640 1 -- 172.21.15.116:0/1903519709 shutdown_connections 2024-09-12T03:01:38.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.105+0000 7f5e14641640 1 -- 172.21.15.116:0/1903519709 >> 172.21.15.116:0/1903519709 conn(0x7f5e0c0fc7d0 msgr2=0x7f5e0c0fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:38.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.106+0000 7f5e14641640 1 -- 172.21.15.116:0/1903519709 shutdown_connections 2024-09-12T03:01:38.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.106+0000 7f5e14641640 1 -- 172.21.15.116:0/1903519709 wait complete. 2024-09-12T03:01:38.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:38 smithi116 ceph-mon[31410]: pgmap v295: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-12T03:01:38.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:38 smithi177 ceph-mon[35305]: pgmap v295: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-12T03:01:38.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:38 smithi183 ceph-mon[35535]: pgmap v295: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-12T03:01:38.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.726+0000 7f4e7b8f0640 1 -- 172.21.15.116:0/2452854456 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e7410c170 msgr2=0x7f4e7410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:38.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.726+0000 7f4e7b8f0640 1 --2- 172.21.15.116:0/2452854456 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e7410c170 0x7f4e7410e610 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c009ca0 tx=0x7f4e6c02a950 comp rx=0 tx=0).stop 2024-09-12T03:01:38.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.726+0000 7f4e7b8f0640 1 -- 172.21.15.116:0/2452854456 shutdown_connections 2024-09-12T03:01:38.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.726+0000 7f4e7b8f0640 1 --2- 172.21.15.116:0/2452854456 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e7410c170 0x7f4e7410e610 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:38.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.726+0000 7f4e7b8f0640 1 --2- 172.21.15.116:0/2452854456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e74103660 0x7f4e7410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:38.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.726+0000 7f4e7b8f0640 1 --2- 172.21.15.116:0/2452854456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e74102d40 0x7f4e74103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:38.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.726+0000 7f4e7b8f0640 1 -- 172.21.15.116:0/2452854456 >> 172.21.15.116:0/2452854456 conn(0x7f4e740fc7a0 msgr2=0x7f4e740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:38.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.726+0000 7f4e7b8f0640 1 -- 172.21.15.116:0/2452854456 shutdown_connections 2024-09-12T03:01:38.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.726+0000 7f4e7b8f0640 1 -- 172.21.15.116:0/2452854456 wait complete. 2024-09-12T03:01:38.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.727+0000 7f4e7b8f0640 1 Processor -- start 2024-09-12T03:01:38.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.727+0000 7f4e7b8f0640 1 -- start start 2024-09-12T03:01:38.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.727+0000 7f4e7b8f0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e74102d40 0x7f4e74198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:38.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.727+0000 7f4e7b8f0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e74103660 0x7f4e74198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:38.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.727+0000 7f4e7b8f0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e7410c170 0x7f4e7419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:38.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.727+0000 7f4e7b8f0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4e74112030 con 0x7f4e74102d40 2024-09-12T03:01:38.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.727+0000 7f4e7b8f0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4e74111eb0 con 0x7f4e74103660 2024-09-12T03:01:38.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.727+0000 7f4e7b8f0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4e741121b0 con 0x7f4e7410c170 2024-09-12T03:01:38.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.727+0000 7f4e7968c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e74102d40 0x7f4e74198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:38.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.727+0000 7f4e7968c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e74102d40 0x7f4e74198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39566/0 (socket says 172.21.15.116:39566) 2024-09-12T03:01:38.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.727+0000 7f4e7968c640 1 -- 172.21.15.116:0/4176702031 learned_addr learned my addr 172.21.15.116:0/4176702031 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:38.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.728+0000 7f4e78e8b640 1 --2- 172.21.15.116:0/4176702031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e74103660 0x7f4e74198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:38.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.728+0000 7f4e79e8d640 1 --2- 172.21.15.116:0/4176702031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e7410c170 0x7f4e7419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:38.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.728+0000 7f4e7968c640 1 -- 172.21.15.116:0/4176702031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e7410c170 msgr2=0x7f4e7419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:38.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.728+0000 7f4e7968c640 1 --2- 172.21.15.116:0/4176702031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e7410c170 0x7f4e7419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:38.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.728+0000 7f4e7968c640 1 -- 172.21.15.116:0/4176702031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e74103660 msgr2=0x7f4e74198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:38.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.728+0000 7f4e7968c640 1 --2- 172.21.15.116:0/4176702031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e74103660 0x7f4e74198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:38.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.728+0000 7f4e7968c640 1 -- 172.21.15.116:0/4176702031 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7419d520 con 0x7f4e74102d40 2024-09-12T03:01:38.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.728+0000 7f4e78e8b640 1 --2- 172.21.15.116:0/4176702031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e74103660 0x7f4e74198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:01:38.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.728+0000 7f4e79e8d640 1 --2- 172.21.15.116:0/4176702031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e7410c170 0x7f4e7419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:01:38.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.728+0000 7f4e7968c640 1 --2- 172.21.15.116:0/4176702031 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e74102d40 0x7f4e74198570 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000b4a0 tx=0x7f4e7000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:38.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.728+0000 7f4e6a7fc640 1 -- 172.21.15.116:0/4176702031 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70014070 con 0x7f4e74102d40 2024-09-12T03:01:38.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.728+0000 7f4e6a7fc640 1 -- 172.21.15.116:0/4176702031 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e70010af0 con 0x7f4e74102d40 2024-09-12T03:01:38.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.728+0000 7f4e7b8f0640 1 -- 172.21.15.116:0/4176702031 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e7419d810 con 0x7f4e74102d40 2024-09-12T03:01:38.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.728+0000 7f4e6a7fc640 1 -- 172.21.15.116:0/4176702031 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70017d10 con 0x7f4e74102d40 2024-09-12T03:01:38.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.728+0000 7f4e7b8f0640 1 -- 172.21.15.116:0/4176702031 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e741a50f0 con 0x7f4e74102d40 2024-09-12T03:01:38.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.729+0000 7f4e7b8f0640 1 -- 172.21.15.116:0/4176702031 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4e74062980 con 0x7f4e74102d40 2024-09-12T03:01:38.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.730+0000 7f4e6a7fc640 1 -- 172.21.15.116:0/4176702031 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f4e70017070 con 0x7f4e74102d40 2024-09-12T03:01:38.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.731+0000 7f4e6a7fc640 1 --2- 172.21.15.116:0/4176702031 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4e600a2c10 0x7f4e600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:38.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.731+0000 7f4e78e8b640 1 --2- 172.21.15.116:0/4176702031 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4e600a2c10 0x7f4e600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:38.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.731+0000 7f4e6a7fc640 1 -- 172.21.15.116:0/4176702031 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f4e700ce760 con 0x7f4e74102d40 2024-09-12T03:01:38.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.731+0000 7f4e6a7fc640 1 --2- 172.21.15.116:0/4176702031 >> [v2:172.21.15.116:6802/862366853,v1:172.21.15.116:6803/862366853] conn(0x7f4e600ac460 0x7f4e600ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:38.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.731+0000 7f4e6a7fc640 1 -- 172.21.15.116:0/4176702031 --> [v2:172.21.15.116:6802/862366853,v1:172.21.15.116:6803/862366853] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4e600aef30 con 0x7f4e600ac460 2024-09-12T03:01:38.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.731+0000 7f4e6a7fc640 1 -- 172.21.15.116:0/4176702031 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f4e700cec40 con 0x7f4e74102d40 2024-09-12T03:01:38.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.731+0000 7f4e78e8b640 1 --2- 172.21.15.116:0/4176702031 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4e600a2c10 0x7f4e600a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4e740feda0 tx=0x7f4e64008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:38.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.731+0000 7f4e79e8d640 1 --2- 172.21.15.116:0/4176702031 >> [v2:172.21.15.116:6802/862366853,v1:172.21.15.116:6803/862366853] conn(0x7f4e600ac460 0x7f4e600ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:38.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.732+0000 7f4e79e8d640 1 --2- 172.21.15.116:0/4176702031 >> [v2:172.21.15.116:6802/862366853,v1:172.21.15.116:6803/862366853] conn(0x7f4e600ac460 0x7f4e600ae880 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:38.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.732+0000 7f4e6a7fc640 1 -- 172.21.15.116:0/4176702031 <== osd.0 v2:172.21.15.116:6802/862366853 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4e600aef30 con 0x7f4e600ac460 2024-09-12T03:01:38.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.746+0000 7f4e7b8f0640 1 -- 172.21.15.116:0/4176702031 --> [v2:172.21.15.116:6802/862366853,v1:172.21.15.116:6803/862366853] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4e74062e00 con 0x7f4e600ac460 2024-09-12T03:01:38.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.746+0000 7f4e6a7fc640 1 -- 172.21.15.116:0/4176702031 <== osd.0 v2:172.21.15.116:6802/862366853 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4e74062e00 con 0x7f4e600ac460 2024-09-12T03:01:38.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.746+0000 7f4e7b8f0640 1 -- 172.21.15.116:0/4176702031 >> [v2:172.21.15.116:6802/862366853,v1:172.21.15.116:6803/862366853] conn(0x7f4e600ac460 msgr2=0x7f4e600ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:38.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.746+0000 7f4e7b8f0640 1 --2- 172.21.15.116:0/4176702031 >> [v2:172.21.15.116:6802/862366853,v1:172.21.15.116:6803/862366853] conn(0x7f4e600ac460 0x7f4e600ae880 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:38.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.746+0000 7f4e7b8f0640 1 -- 172.21.15.116:0/4176702031 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4e600a2c10 msgr2=0x7f4e600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:38.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.746+0000 7f4e7b8f0640 1 --2- 172.21.15.116:0/4176702031 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4e600a2c10 0x7f4e600a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4e740feda0 tx=0x7f4e64008040 comp rx=0 tx=0).stop 2024-09-12T03:01:38.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.746+0000 7f4e7b8f0640 1 -- 172.21.15.116:0/4176702031 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e74102d40 msgr2=0x7f4e74198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:38.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.746+0000 7f4e7b8f0640 1 --2- 172.21.15.116:0/4176702031 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e74102d40 0x7f4e74198570 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000b4a0 tx=0x7f4e7000b960 comp rx=0 tx=0).stop 2024-09-12T03:01:38.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.746+0000 7f4e79e8d640 1 -- 172.21.15.116:0/4176702031 reap_dead start 2024-09-12T03:01:38.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.747+0000 7f4e7b8f0640 1 -- 172.21.15.116:0/4176702031 shutdown_connections 2024-09-12T03:01:38.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.747+0000 7f4e7b8f0640 1 -- 172.21.15.116:0/4176702031 >> 172.21.15.116:0/4176702031 conn(0x7f4e740fc7a0 msgr2=0x7f4e740fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:38.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.747+0000 7f4e7b8f0640 1 -- 172.21.15.116:0/4176702031 shutdown_connections 2024-09-12T03:01:38.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.747+0000 7f4e7b8f0640 1 -- 172.21.15.116:0/4176702031 wait complete. 2024-09-12T03:01:39.806 INFO:teuthology.orchestra.run.smithi116.stdout:34359738445 2024-09-12T03:01:39.806 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-12T03:01:39.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.624+0000 7f747123e640 1 -- 172.21.15.116:0/3469546880 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f746c101420 msgr2=0x7f746c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:39.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.624+0000 7f747123e640 1 --2- 172.21.15.116:0/3469546880 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f746c101420 0x7f746c10d950 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7464009cd0 tx=0x7f746402a8e0 comp rx=0 tx=0).stop 2024-09-12T03:01:39.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.624+0000 7f747123e640 1 -- 172.21.15.116:0/3469546880 shutdown_connections 2024-09-12T03:01:39.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.624+0000 7f747123e640 1 --2- 172.21.15.116:0/3469546880 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f746c101420 0x7f746c10d950 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:39.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.624+0000 7f747123e640 1 --2- 172.21.15.116:0/3469546880 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f746c100a80 0x7f746c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:39.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.624+0000 7f747123e640 1 --2- 172.21.15.116:0/3469546880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f746c106a80 0x7f746c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:39.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.624+0000 7f747123e640 1 -- 172.21.15.116:0/3469546880 >> 172.21.15.116:0/3469546880 conn(0x7f746c0fc7a0 msgr2=0x7f746c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:39.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.624+0000 7f747123e640 1 -- 172.21.15.116:0/3469546880 shutdown_connections 2024-09-12T03:01:39.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.624+0000 7f747123e640 1 -- 172.21.15.116:0/3469546880 wait complete. 2024-09-12T03:01:39.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.625+0000 7f747123e640 1 Processor -- start 2024-09-12T03:01:39.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.625+0000 7f747123e640 1 -- start start 2024-09-12T03:01:39.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.625+0000 7f747123e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f746c100a80 0x7f746c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:39.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.625+0000 7f747123e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f746c101420 0x7f746c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:39.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.625+0000 7f747123e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f746c106a80 0x7f746c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:39.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.625+0000 7f747123e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f746c111fa0 con 0x7f746c101420 2024-09-12T03:01:39.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.625+0000 7f747123e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f746c111e20 con 0x7f746c100a80 2024-09-12T03:01:39.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.625+0000 7f747123e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f746c112120 con 0x7f746c106a80 2024-09-12T03:01:39.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.625+0000 7f746a59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f746c101420 0x7f746c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:39.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.625+0000 7f746a59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f746c101420 0x7f746c1988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39552/0 (socket says 172.21.15.116:39552) 2024-09-12T03:01:39.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.625+0000 7f746a59c640 1 -- 172.21.15.116:0/2240871562 learned_addr learned my addr 172.21.15.116:0/2240871562 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:39.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.625+0000 7f746ad9d640 1 --2- 172.21.15.116:0/2240871562 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f746c100a80 0x7f746c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:39.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.625+0000 7f746b59e640 1 --2- 172.21.15.116:0/2240871562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f746c106a80 0x7f746c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:39.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.626+0000 7f746a59c640 1 -- 172.21.15.116:0/2240871562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f746c106a80 msgr2=0x7f746c19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:39.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.626+0000 7f746a59c640 1 --2- 172.21.15.116:0/2240871562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f746c106a80 0x7f746c19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:39.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.626+0000 7f746a59c640 1 -- 172.21.15.116:0/2240871562 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f746c100a80 msgr2=0x7f746c1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:39.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.626+0000 7f746a59c640 1 --2- 172.21.15.116:0/2240871562 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f746c100a80 0x7f746c1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:39.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.626+0000 7f746a59c640 1 -- 172.21.15.116:0/2240871562 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746c19d350 con 0x7f746c101420 2024-09-12T03:01:39.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.626+0000 7f746a59c640 1 --2- 172.21.15.116:0/2240871562 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f746c101420 0x7f746c1988e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f746000c920 tx=0x7f746000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:39.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.626+0000 7f7453fff640 1 -- 172.21.15.116:0/2240871562 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460007bb0 con 0x7f746c101420 2024-09-12T03:01:39.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.626+0000 7f747123e640 1 -- 172.21.15.116:0/2240871562 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f746c19d640 con 0x7f746c101420 2024-09-12T03:01:39.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.626+0000 7f7453fff640 1 -- 172.21.15.116:0/2240871562 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7460007d50 con 0x7f746c101420 2024-09-12T03:01:39.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.626+0000 7f7453fff640 1 -- 172.21.15.116:0/2240871562 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460018780 con 0x7f746c101420 2024-09-12T03:01:39.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.626+0000 7f747123e640 1 -- 172.21.15.116:0/2240871562 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f746c1a4f20 con 0x7f746c101420 2024-09-12T03:01:39.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.627+0000 7f747123e640 1 -- 172.21.15.116:0/2240871562 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f746c0fd750 con 0x7f746c101420 2024-09-12T03:01:39.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.628+0000 7f7453fff640 1 -- 172.21.15.116:0/2240871562 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f746000f450 con 0x7f746c101420 2024-09-12T03:01:39.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.629+0000 7f7453fff640 1 --2- 172.21.15.116:0/2240871562 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f74540a2c10 0x7f74540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:39.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.629+0000 7f746ad9d640 1 --2- 172.21.15.116:0/2240871562 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f74540a2c10 0x7f74540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:39.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.629+0000 7f7453fff640 1 -- 172.21.15.116:0/2240871562 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f74600cf510 con 0x7f746c101420 2024-09-12T03:01:39.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.629+0000 7f7453fff640 1 --2- 172.21.15.116:0/2240871562 >> [v2:172.21.15.183:6824/1818319939,v1:172.21.15.183:6825/1818319939] conn(0x7f74540ac460 0x7f74540ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:39.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.629+0000 7f7453fff640 1 -- 172.21.15.116:0/2240871562 --> [v2:172.21.15.183:6824/1818319939,v1:172.21.15.183:6825/1818319939] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7460015e70 con 0x7f74540ac460 2024-09-12T03:01:39.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.629+0000 7f7453fff640 1 -- 172.21.15.116:0/2240871562 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f74600cf9f0 con 0x7f746c101420 2024-09-12T03:01:39.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.629+0000 7f746ad9d640 1 --2- 172.21.15.116:0/2240871562 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f74540a2c10 0x7f74540a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f745c009990 tx=0x7f745c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:39.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.629+0000 7f746b59e640 1 --2- 172.21.15.116:0/2240871562 >> [v2:172.21.15.183:6824/1818319939,v1:172.21.15.183:6825/1818319939] conn(0x7f74540ac460 0x7f74540ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:39.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.630+0000 7f746b59e640 1 --2- 172.21.15.116:0/2240871562 >> [v2:172.21.15.183:6824/1818319939,v1:172.21.15.183:6825/1818319939] conn(0x7f74540ac460 0x7f74540ae880 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:39.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.630+0000 7f7453fff640 1 -- 172.21.15.116:0/2240871562 <== osd.11 v2:172.21.15.183:6824/1818319939 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7460015e70 con 0x7f74540ac460 2024-09-12T03:01:39.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.644+0000 7f747123e640 1 -- 172.21.15.116:0/2240871562 --> [v2:172.21.15.183:6824/1818319939,v1:172.21.15.183:6825/1818319939] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f746c1021c0 con 0x7f74540ac460 2024-09-12T03:01:39.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.644+0000 7f7453fff640 1 -- 172.21.15.116:0/2240871562 <== osd.11 v2:172.21.15.183:6824/1818319939 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f746c1021c0 con 0x7f74540ac460 2024-09-12T03:01:39.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.644+0000 7f747123e640 1 -- 172.21.15.116:0/2240871562 >> [v2:172.21.15.183:6824/1818319939,v1:172.21.15.183:6825/1818319939] conn(0x7f74540ac460 msgr2=0x7f74540ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:39.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.644+0000 7f747123e640 1 --2- 172.21.15.116:0/2240871562 >> [v2:172.21.15.183:6824/1818319939,v1:172.21.15.183:6825/1818319939] conn(0x7f74540ac460 0x7f74540ae880 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:39.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.644+0000 7f747123e640 1 -- 172.21.15.116:0/2240871562 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f74540a2c10 msgr2=0x7f74540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:39.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.644+0000 7f747123e640 1 --2- 172.21.15.116:0/2240871562 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f74540a2c10 0x7f74540a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f745c009990 tx=0x7f745c009110 comp rx=0 tx=0).stop 2024-09-12T03:01:39.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.645+0000 7f747123e640 1 -- 172.21.15.116:0/2240871562 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f746c101420 msgr2=0x7f746c1988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:39.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.645+0000 7f747123e640 1 --2- 172.21.15.116:0/2240871562 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f746c101420 0x7f746c1988e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f746000c920 tx=0x7f746000cd20 comp rx=0 tx=0).stop 2024-09-12T03:01:39.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.645+0000 7f746b59e640 1 -- 172.21.15.116:0/2240871562 reap_dead start 2024-09-12T03:01:39.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.645+0000 7f747123e640 1 -- 172.21.15.116:0/2240871562 shutdown_connections 2024-09-12T03:01:39.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.645+0000 7f747123e640 1 -- 172.21.15.116:0/2240871562 >> 172.21.15.116:0/2240871562 conn(0x7f746c0fc7a0 msgr2=0x7f746c0fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:39.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.645+0000 7f747123e640 1 -- 172.21.15.116:0/2240871562 shutdown_connections 2024-09-12T03:01:39.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:38.645+0000 7f747123e640 1 -- 172.21.15.116:0/2240871562 wait complete. 2024-09-12T03:01:40.472 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:40 smithi116 ceph-mon[31410]: pgmap v296: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T03:01:40.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:40 smithi177 ceph-mon[35305]: pgmap v296: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T03:01:40.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:40 smithi183 ceph-mon[35535]: pgmap v296: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T03:01:40.933 INFO:teuthology.orchestra.run.smithi116.stdout:51539607624 2024-09-12T03:01:40.934 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-12T03:01:41.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.311+0000 7fc82312c640 1 -- 172.21.15.116:0/510543583 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc81c106a60 msgr2=0x7fc81c106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:41.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.311+0000 7fc82312c640 1 --2- 172.21.15.116:0/510543583 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc81c106a60 0x7fc81c106e40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc810009930 tx=0x7fc81002a420 comp rx=0 tx=0).stop 2024-09-12T03:01:41.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.311+0000 7fc82312c640 1 -- 172.21.15.116:0/510543583 shutdown_connections 2024-09-12T03:01:41.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.311+0000 7fc82312c640 1 --2- 172.21.15.116:0/510543583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc81c101400 0x7fc81c10d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:41.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.311+0000 7fc82312c640 1 --2- 172.21.15.116:0/510543583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc81c100a60 0x7fc81c100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:41.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.311+0000 7fc82312c640 1 --2- 172.21.15.116:0/510543583 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc81c106a60 0x7fc81c106e40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:41.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.311+0000 7fc82312c640 1 -- 172.21.15.116:0/510543583 >> 172.21.15.116:0/510543583 conn(0x7fc81c0fc780 msgr2=0x7fc81c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:41.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.311+0000 7fc82312c640 1 -- 172.21.15.116:0/510543583 shutdown_connections 2024-09-12T03:01:41.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.311+0000 7fc82312c640 1 -- 172.21.15.116:0/510543583 wait complete. 2024-09-12T03:01:41.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.311+0000 7fc82312c640 1 Processor -- start 2024-09-12T03:01:41.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.311+0000 7fc82312c640 1 -- start start 2024-09-12T03:01:41.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.312+0000 7fc82312c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc81c100a60 0x7fc81c197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:41.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.312+0000 7fc82312c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc81c101400 0x7fc81c197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:41.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.312+0000 7fc82312c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc81c106a60 0x7fc81c19bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:41.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.312+0000 7fc82312c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c110e60 con 0x7fc81c106a60 2024-09-12T03:01:41.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.312+0000 7fc82312c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c110ce0 con 0x7fc81c100a60 2024-09-12T03:01:41.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.312+0000 7fc82312c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c110fe0 con 0x7fc81c101400 2024-09-12T03:01:41.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.312+0000 7fc8216c9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc81c106a60 0x7fc81c19bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:41.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.312+0000 7fc8216c9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc81c106a60 0x7fc81c19bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60196/0 (socket says 172.21.15.116:60196) 2024-09-12T03:01:41.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.312+0000 7fc8216c9640 1 -- 172.21.15.116:0/34598210 learned_addr learned my addr 172.21.15.116:0/34598210 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:41.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.312+0000 7fc820ec8640 1 --2- 172.21.15.116:0/34598210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc81c100a60 0x7fc81c197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:41.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.312+0000 7fc81bfff640 1 --2- 172.21.15.116:0/34598210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc81c101400 0x7fc81c197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:41.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.312+0000 7fc8216c9640 1 -- 172.21.15.116:0/34598210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc81c101400 msgr2=0x7fc81c197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:41.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.312+0000 7fc8216c9640 1 --2- 172.21.15.116:0/34598210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc81c101400 0x7fc81c197850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:41.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.312+0000 7fc8216c9640 1 -- 172.21.15.116:0/34598210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc81c100a60 msgr2=0x7fc81c197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:41.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.312+0000 7fc8216c9640 1 --2- 172.21.15.116:0/34598210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc81c100a60 0x7fc81c197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:41.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.312+0000 7fc8216c9640 1 -- 172.21.15.116:0/34598210 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc81c19c2c0 con 0x7fc81c106a60 2024-09-12T03:01:41.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.312+0000 7fc820ec8640 1 --2- 172.21.15.116:0/34598210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc81c100a60 0x7fc81c197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:01:41.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.312+0000 7fc8216c9640 1 --2- 172.21.15.116:0/34598210 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc81c106a60 0x7fc81c19bbe0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc81400cad0 tx=0x7fc81400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:41.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.312+0000 7fc819ffb640 1 -- 172.21.15.116:0/34598210 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81400ae20 con 0x7fc81c106a60 2024-09-12T03:01:41.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.312+0000 7fc82312c640 1 -- 172.21.15.116:0/34598210 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc81c19c5b0 con 0x7fc81c106a60 2024-09-12T03:01:41.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.312+0000 7fc819ffb640 1 -- 172.21.15.116:0/34598210 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc814005020 con 0x7fc81c106a60 2024-09-12T03:01:41.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.312+0000 7fc819ffb640 1 -- 172.21.15.116:0/34598210 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81400f6f0 con 0x7fc81c106a60 2024-09-12T03:01:41.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.313+0000 7fc82312c640 1 -- 172.21.15.116:0/34598210 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc81c1a3e90 con 0x7fc81c106a60 2024-09-12T03:01:41.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.314+0000 7fc82312c640 1 -- 172.21.15.116:0/34598210 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc81c062940 con 0x7fc81c106a60 2024-09-12T03:01:41.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.315+0000 7fc819ffb640 1 -- 172.21.15.116:0/34598210 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc8140051d0 con 0x7fc81c106a60 2024-09-12T03:01:41.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.315+0000 7fc819ffb640 1 --2- 172.21.15.116:0/34598210 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc80c0a2c10 0x7fc80c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:41.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.315+0000 7fc820ec8640 1 --2- 172.21.15.116:0/34598210 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc80c0a2c10 0x7fc80c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:41.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.315+0000 7fc819ffb640 1 -- 172.21.15.116:0/34598210 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fc8140cf630 con 0x7fc81c106a60 2024-09-12T03:01:41.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.315+0000 7fc819ffb640 1 --2- 172.21.15.116:0/34598210 >> [v2:172.21.15.177:6808/2382938021,v1:172.21.15.177:6809/2382938021] conn(0x7fc80c0ac460 0x7fc80c0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:41.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.315+0000 7fc819ffb640 1 -- 172.21.15.116:0/34598210 --> [v2:172.21.15.177:6808/2382938021,v1:172.21.15.177:6809/2382938021] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc814012ec0 con 0x7fc80c0ac460 2024-09-12T03:01:41.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.315+0000 7fc819ffb640 1 -- 172.21.15.116:0/34598210 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7fc8140cf9e0 con 0x7fc81c106a60 2024-09-12T03:01:41.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.316+0000 7fc820ec8640 1 --2- 172.21.15.116:0/34598210 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc80c0a2c10 0x7fc80c0a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc810000c00 tx=0x7fc810005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:41.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.316+0000 7fc81bfff640 1 --2- 172.21.15.116:0/34598210 >> [v2:172.21.15.177:6808/2382938021,v1:172.21.15.177:6809/2382938021] conn(0x7fc80c0ac460 0x7fc80c0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:41.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.316+0000 7fc81bfff640 1 --2- 172.21.15.116:0/34598210 >> [v2:172.21.15.177:6808/2382938021,v1:172.21.15.177:6809/2382938021] conn(0x7fc80c0ac460 0x7fc80c0ae880 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:41.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.317+0000 7fc819ffb640 1 -- 172.21.15.116:0/34598210 <== osd.4 v2:172.21.15.177:6808/2382938021 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc814012ec0 con 0x7fc80c0ac460 2024-09-12T03:01:41.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.330+0000 7fc82312c640 1 -- 172.21.15.116:0/34598210 --> [v2:172.21.15.177:6808/2382938021,v1:172.21.15.177:6809/2382938021] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc81c062dc0 con 0x7fc80c0ac460 2024-09-12T03:01:41.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.331+0000 7fc819ffb640 1 -- 172.21.15.116:0/34598210 <== osd.4 v2:172.21.15.177:6808/2382938021 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc81c062dc0 con 0x7fc80c0ac460 2024-09-12T03:01:41.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.331+0000 7fc82312c640 1 -- 172.21.15.116:0/34598210 >> [v2:172.21.15.177:6808/2382938021,v1:172.21.15.177:6809/2382938021] conn(0x7fc80c0ac460 msgr2=0x7fc80c0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:41.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.331+0000 7fc82312c640 1 --2- 172.21.15.116:0/34598210 >> [v2:172.21.15.177:6808/2382938021,v1:172.21.15.177:6809/2382938021] conn(0x7fc80c0ac460 0x7fc80c0ae880 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:41.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.331+0000 7fc82312c640 1 -- 172.21.15.116:0/34598210 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc80c0a2c10 msgr2=0x7fc80c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:41.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.331+0000 7fc82312c640 1 --2- 172.21.15.116:0/34598210 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc80c0a2c10 0x7fc80c0a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc810000c00 tx=0x7fc810005a20 comp rx=0 tx=0).stop 2024-09-12T03:01:41.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.331+0000 7fc82312c640 1 -- 172.21.15.116:0/34598210 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc81c106a60 msgr2=0x7fc81c19bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:41.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.331+0000 7fc82312c640 1 --2- 172.21.15.116:0/34598210 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc81c106a60 0x7fc81c19bbe0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc81400cad0 tx=0x7fc81400cf90 comp rx=0 tx=0).stop 2024-09-12T03:01:41.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.331+0000 7fc8216c9640 1 -- 172.21.15.116:0/34598210 reap_dead start 2024-09-12T03:01:41.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.331+0000 7fc82312c640 1 -- 172.21.15.116:0/34598210 shutdown_connections 2024-09-12T03:01:41.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.331+0000 7fc82312c640 1 -- 172.21.15.116:0/34598210 >> 172.21.15.116:0/34598210 conn(0x7fc81c0fc780 msgr2=0x7fc81c0fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:41.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.332+0000 7fc82312c640 1 -- 172.21.15.116:0/34598210 shutdown_connections 2024-09-12T03:01:41.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:41.332+0000 7fc82312c640 1 -- 172.21.15.116:0/34598210 wait complete. 2024-09-12T03:01:42.165 INFO:teuthology.orchestra.run.smithi116.stdout:141733920819 2024-09-12T03:01:42.166 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-12T03:01:42.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:42 smithi116 ceph-mon[31410]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T03:01:42.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:42 smithi177 ceph-mon[35305]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T03:01:42.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:42 smithi183 ceph-mon[35535]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T03:01:43.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.757+0000 7ff93dfb7640 1 -- 172.21.15.116:0/422274440 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff938101de0 msgr2=0x7ff9381021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:43.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.757+0000 7ff93dfb7640 1 --2- 172.21.15.116:0/422274440 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff938101de0 0x7ff9381021c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff928009900 tx=0x7ff92802a410 comp rx=0 tx=0).stop 2024-09-12T03:01:43.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.757+0000 7ff93dfb7640 1 -- 172.21.15.116:0/422274440 shutdown_connections 2024-09-12T03:01:43.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.757+0000 7ff93dfb7640 1 --2- 172.21.15.116:0/422274440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff938106f10 0x7ff93810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:43.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.757+0000 7ff93dfb7640 1 --2- 172.21.15.116:0/422274440 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff938102790 0x7ff9381067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:43.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.757+0000 7ff93dfb7640 1 --2- 172.21.15.116:0/422274440 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff938101de0 0x7ff9381021c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:43.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.757+0000 7ff93dfb7640 1 -- 172.21.15.116:0/422274440 >> 172.21.15.116:0/422274440 conn(0x7ff9380fd8c0 msgr2=0x7ff9380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:43.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.757+0000 7ff93dfb7640 1 -- 172.21.15.116:0/422274440 shutdown_connections 2024-09-12T03:01:43.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.757+0000 7ff93dfb7640 1 -- 172.21.15.116:0/422274440 wait complete. 2024-09-12T03:01:43.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.757+0000 7ff93dfb7640 1 Processor -- start 2024-09-12T03:01:43.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.758+0000 7ff93dfb7640 1 -- start start 2024-09-12T03:01:43.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.758+0000 7ff93dfb7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff938101de0 0x7ff9381985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:43.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.758+0000 7ff93dfb7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff938102790 0x7ff938198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:43.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.758+0000 7ff93dfb7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff938106f10 0x7ff93819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:43.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.758+0000 7ff93dfb7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff938112010 con 0x7ff938106f10 2024-09-12T03:01:43.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.758+0000 7ff93dfb7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff938111e90 con 0x7ff938102790 2024-09-12T03:01:43.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.758+0000 7ff93dfb7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff938112190 con 0x7ff938101de0 2024-09-12T03:01:43.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.758+0000 7ff937fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff938106f10 0x7ff93819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:43.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.758+0000 7ff937fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff938106f10 0x7ff93819ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60230/0 (socket says 172.21.15.116:60230) 2024-09-12T03:01:43.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.758+0000 7ff937fff640 1 -- 172.21.15.116:0/1578129923 learned_addr learned my addr 172.21.15.116:0/1578129923 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:43.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.758+0000 7ff9377fe640 1 --2- 172.21.15.116:0/1578129923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff938101de0 0x7ff9381985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:43.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.758+0000 7ff936ffd640 1 --2- 172.21.15.116:0/1578129923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff938102790 0x7ff938198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:43.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.758+0000 7ff937fff640 1 -- 172.21.15.116:0/1578129923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff938101de0 msgr2=0x7ff9381985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:43.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.758+0000 7ff937fff640 1 --2- 172.21.15.116:0/1578129923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff938101de0 0x7ff9381985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:43.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.758+0000 7ff937fff640 1 -- 172.21.15.116:0/1578129923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff938102790 msgr2=0x7ff938198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:43.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.758+0000 7ff937fff640 1 --2- 172.21.15.116:0/1578129923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff938102790 0x7ff938198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:43.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.758+0000 7ff937fff640 1 -- 172.21.15.116:0/1578129923 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff93819d560 con 0x7ff938106f10 2024-09-12T03:01:43.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.758+0000 7ff936ffd640 1 --2- 172.21.15.116:0/1578129923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff938102790 0x7ff938198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:01:43.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.758+0000 7ff937fff640 1 --2- 172.21.15.116:0/1578129923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff938106f10 0x7ff93819ce80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff93000b970 tx=0x7ff93000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:43.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.759+0000 7ff934ff9640 1 -- 172.21.15.116:0/1578129923 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff930019070 con 0x7ff938106f10 2024-09-12T03:01:43.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.759+0000 7ff934ff9640 1 -- 172.21.15.116:0/1578129923 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff93000cb40 con 0x7ff938106f10 2024-09-12T03:01:43.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.759+0000 7ff93dfb7640 1 -- 172.21.15.116:0/1578129923 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff93819d850 con 0x7ff938106f10 2024-09-12T03:01:43.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.759+0000 7ff934ff9640 1 -- 172.21.15.116:0/1578129923 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff930007650 con 0x7ff938106f10 2024-09-12T03:01:43.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.759+0000 7ff93dfb7640 1 -- 172.21.15.116:0/1578129923 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9381a5130 con 0x7ff938106f10 2024-09-12T03:01:43.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.759+0000 7ff93dfb7640 1 -- 172.21.15.116:0/1578129923 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff938062980 con 0x7ff938106f10 2024-09-12T03:01:43.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.761+0000 7ff934ff9640 1 -- 172.21.15.116:0/1578129923 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff93001c070 con 0x7ff938106f10 2024-09-12T03:01:43.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.761+0000 7ff934ff9640 1 --2- 172.21.15.116:0/1578129923 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff9200a2c10 0x7ff9200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:43.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.761+0000 7ff9377fe640 1 --2- 172.21.15.116:0/1578129923 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff9200a2c10 0x7ff9200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:43.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.761+0000 7ff934ff9640 1 -- 172.21.15.116:0/1578129923 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7ff9300ced00 con 0x7ff938106f10 2024-09-12T03:01:43.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.762+0000 7ff934ff9640 1 --2- 172.21.15.116:0/1578129923 >> [v2:172.21.15.183:6800/2645677759,v1:172.21.15.183:6801/2645677759] conn(0x7ff9200ac460 0x7ff9200ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:43.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.762+0000 7ff934ff9640 1 -- 172.21.15.116:0/1578129923 --> [v2:172.21.15.183:6800/2645677759,v1:172.21.15.183:6801/2645677759] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff9300047d0 con 0x7ff9200ac460 2024-09-12T03:01:43.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.762+0000 7ff934ff9640 1 -- 172.21.15.116:0/1578129923 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7ff9300cf1e0 con 0x7ff938106f10 2024-09-12T03:01:43.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.762+0000 7ff9377fe640 1 --2- 172.21.15.116:0/1578129923 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff9200a2c10 0x7ff9200a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff928000c00 tx=0x7ff9280058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:43.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.762+0000 7ff936ffd640 1 --2- 172.21.15.116:0/1578129923 >> [v2:172.21.15.183:6800/2645677759,v1:172.21.15.183:6801/2645677759] conn(0x7ff9200ac460 0x7ff9200ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:43.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.762+0000 7ff936ffd640 1 --2- 172.21.15.116:0/1578129923 >> [v2:172.21.15.183:6800/2645677759,v1:172.21.15.183:6801/2645677759] conn(0x7ff9200ac460 0x7ff9200ae880 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:43.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.763+0000 7ff934ff9640 1 -- 172.21.15.116:0/1578129923 <== osd.2 v2:172.21.15.183:6800/2645677759 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff9300047d0 con 0x7ff9200ac460 2024-09-12T03:01:43.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.777+0000 7ff93dfb7640 1 -- 172.21.15.116:0/1578129923 --> [v2:172.21.15.183:6800/2645677759,v1:172.21.15.183:6801/2645677759] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff938062e00 con 0x7ff9200ac460 2024-09-12T03:01:43.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.777+0000 7ff934ff9640 1 -- 172.21.15.116:0/1578129923 <== osd.2 v2:172.21.15.183:6800/2645677759 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff938062e00 con 0x7ff9200ac460 2024-09-12T03:01:43.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.778+0000 7ff93dfb7640 1 -- 172.21.15.116:0/1578129923 >> [v2:172.21.15.183:6800/2645677759,v1:172.21.15.183:6801/2645677759] conn(0x7ff9200ac460 msgr2=0x7ff9200ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:43.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.778+0000 7ff93dfb7640 1 --2- 172.21.15.116:0/1578129923 >> [v2:172.21.15.183:6800/2645677759,v1:172.21.15.183:6801/2645677759] conn(0x7ff9200ac460 0x7ff9200ae880 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:43.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.778+0000 7ff93dfb7640 1 -- 172.21.15.116:0/1578129923 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff9200a2c10 msgr2=0x7ff9200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:43.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.778+0000 7ff93dfb7640 1 --2- 172.21.15.116:0/1578129923 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff9200a2c10 0x7ff9200a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff928000c00 tx=0x7ff9280058d0 comp rx=0 tx=0).stop 2024-09-12T03:01:43.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.778+0000 7ff93dfb7640 1 -- 172.21.15.116:0/1578129923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff938106f10 msgr2=0x7ff93819ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:43.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.778+0000 7ff93dfb7640 1 --2- 172.21.15.116:0/1578129923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff938106f10 0x7ff93819ce80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff93000b970 tx=0x7ff93000be30 comp rx=0 tx=0).stop 2024-09-12T03:01:43.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.778+0000 7ff937fff640 1 -- 172.21.15.116:0/1578129923 reap_dead start 2024-09-12T03:01:43.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.778+0000 7ff93dfb7640 1 -- 172.21.15.116:0/1578129923 shutdown_connections 2024-09-12T03:01:43.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.778+0000 7ff93dfb7640 1 -- 172.21.15.116:0/1578129923 >> 172.21.15.116:0/1578129923 conn(0x7ff9380fd8c0 msgr2=0x7ff9380fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:43.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.778+0000 7ff93dfb7640 1 -- 172.21.15.116:0/1578129923 shutdown_connections 2024-09-12T03:01:43.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.778+0000 7ff93dfb7640 1 -- 172.21.15.116:0/1578129923 wait complete. 2024-09-12T03:01:43.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.104+0000 7fda257b0640 1 -- 172.21.15.116:0/1930231772 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda20102790 msgr2=0x7fda201067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:43.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.104+0000 7fda257b0640 1 --2- 172.21.15.116:0/1930231772 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda20102790 0x7fda201067e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fda08009900 tx=0x7fda0802a410 comp rx=0 tx=0).stop 2024-09-12T03:01:43.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.104+0000 7fda257b0640 1 -- 172.21.15.116:0/1930231772 shutdown_connections 2024-09-12T03:01:43.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.104+0000 7fda257b0640 1 --2- 172.21.15.116:0/1930231772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda20106f10 0x7fda2010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:43.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.104+0000 7fda257b0640 1 --2- 172.21.15.116:0/1930231772 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda20102790 0x7fda201067e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:43.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.104+0000 7fda257b0640 1 --2- 172.21.15.116:0/1930231772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda20101de0 0x7fda201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:43.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.104+0000 7fda257b0640 1 -- 172.21.15.116:0/1930231772 >> 172.21.15.116:0/1930231772 conn(0x7fda200fd8c0 msgr2=0x7fda200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:43.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.104+0000 7fda257b0640 1 -- 172.21.15.116:0/1930231772 shutdown_connections 2024-09-12T03:01:43.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.104+0000 7fda257b0640 1 -- 172.21.15.116:0/1930231772 wait complete. 2024-09-12T03:01:43.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.104+0000 7fda257b0640 1 Processor -- start 2024-09-12T03:01:43.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.104+0000 7fda257b0640 1 -- start start 2024-09-12T03:01:43.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.105+0000 7fda257b0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda20101de0 0x7fda20198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:43.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.105+0000 7fda257b0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda20102790 0x7fda20198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:43.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.105+0000 7fda257b0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda20106f10 0x7fda2019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:43.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.105+0000 7fda257b0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fda20111f70 con 0x7fda20102790 2024-09-12T03:01:43.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.105+0000 7fda257b0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fda20111df0 con 0x7fda20101de0 2024-09-12T03:01:43.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.105+0000 7fda257b0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fda201120f0 con 0x7fda20106f10 2024-09-12T03:01:43.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.105+0000 7fda167fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda20102790 0x7fda20198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:43.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.105+0000 7fda167fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda20102790 0x7fda20198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60208/0 (socket says 172.21.15.116:60208) 2024-09-12T03:01:43.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.105+0000 7fda167fc640 1 -- 172.21.15.116:0/3582220940 learned_addr learned my addr 172.21.15.116:0/3582220940 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:43.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.105+0000 7fda1effd640 1 --2- 172.21.15.116:0/3582220940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda20101de0 0x7fda20198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:43.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.105+0000 7fda1f7fe640 1 --2- 172.21.15.116:0/3582220940 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda20106f10 0x7fda2019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:43.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.105+0000 7fda167fc640 1 -- 172.21.15.116:0/3582220940 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda20106f10 msgr2=0x7fda2019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:43.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.105+0000 7fda167fc640 1 --2- 172.21.15.116:0/3582220940 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda20106f10 0x7fda2019cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:43.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.105+0000 7fda167fc640 1 -- 172.21.15.116:0/3582220940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda20101de0 msgr2=0x7fda20198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:43.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.105+0000 7fda167fc640 1 --2- 172.21.15.116:0/3582220940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda20101de0 0x7fda20198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:43.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.105+0000 7fda167fc640 1 -- 172.21.15.116:0/3582220940 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2019d400 con 0x7fda20102790 2024-09-12T03:01:43.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.105+0000 7fda167fc640 1 --2- 172.21.15.116:0/3582220940 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda20102790 0x7fda20198990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fda08006010 tx=0x7fda080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:43.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.105+0000 7fda1cff9640 1 -- 172.21.15.116:0/3582220940 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0803f030 con 0x7fda20102790 2024-09-12T03:01:43.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.105+0000 7fda1cff9640 1 -- 172.21.15.116:0/3582220940 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda08003ce0 con 0x7fda20102790 2024-09-12T03:01:43.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.105+0000 7fda257b0640 1 -- 172.21.15.116:0/3582220940 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda2019d690 con 0x7fda20102790 2024-09-12T03:01:43.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.105+0000 7fda1cff9640 1 -- 172.21.15.116:0/3582220940 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda08035070 con 0x7fda20102790 2024-09-12T03:01:43.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.105+0000 7fda257b0640 1 -- 172.21.15.116:0/3582220940 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda201a4fd0 con 0x7fda20102790 2024-09-12T03:01:43.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.106+0000 7fda257b0640 1 -- 172.21.15.116:0/3582220940 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fda20062980 con 0x7fda20102790 2024-09-12T03:01:43.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.108+0000 7fda1cff9640 1 -- 172.21.15.116:0/3582220940 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fda08042030 con 0x7fda20102790 2024-09-12T03:01:43.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.108+0000 7fda1cff9640 1 --2- 172.21.15.116:0/3582220940 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fda0c0a2c10 0x7fda0c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:43.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.108+0000 7fda1effd640 1 --2- 172.21.15.116:0/3582220940 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fda0c0a2c10 0x7fda0c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:43.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.108+0000 7fda1cff9640 1 -- 172.21.15.116:0/3582220940 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fda080efd30 con 0x7fda20102790 2024-09-12T03:01:43.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.108+0000 7fda1cff9640 1 --2- 172.21.15.116:0/3582220940 >> [v2:172.21.15.177:6824/963663634,v1:172.21.15.177:6825/963663634] conn(0x7fda0c0ac460 0x7fda0c0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:43.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.108+0000 7fda1cff9640 1 -- 172.21.15.116:0/3582220940 --> [v2:172.21.15.177:6824/963663634,v1:172.21.15.177:6825/963663634] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fda08040e70 con 0x7fda0c0ac460 2024-09-12T03:01:43.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.109+0000 7fda1cff9640 1 -- 172.21.15.116:0/3582220940 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7fda080f0210 con 0x7fda20102790 2024-09-12T03:01:43.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.109+0000 7fda1effd640 1 --2- 172.21.15.116:0/3582220940 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fda0c0a2c10 0x7fda0c0a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fda10005fd0 tx=0x7fda10005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:43.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.109+0000 7fda1f7fe640 1 --2- 172.21.15.116:0/3582220940 >> [v2:172.21.15.177:6824/963663634,v1:172.21.15.177:6825/963663634] conn(0x7fda0c0ac460 0x7fda0c0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:43.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.109+0000 7fda1f7fe640 1 --2- 172.21.15.116:0/3582220940 >> [v2:172.21.15.177:6824/963663634,v1:172.21.15.177:6825/963663634] conn(0x7fda0c0ac460 0x7fda0c0ae880 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:43.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.110+0000 7fda1cff9640 1 -- 172.21.15.116:0/3582220940 <== osd.10 v2:172.21.15.177:6824/963663634 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fda08040e70 con 0x7fda0c0ac460 2024-09-12T03:01:43.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.124+0000 7fda257b0640 1 -- 172.21.15.116:0/3582220940 --> [v2:172.21.15.177:6824/963663634,v1:172.21.15.177:6825/963663634] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fda20062e00 con 0x7fda0c0ac460 2024-09-12T03:01:43.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.124+0000 7fda1cff9640 1 -- 172.21.15.116:0/3582220940 <== osd.10 v2:172.21.15.177:6824/963663634 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fda20062e00 con 0x7fda0c0ac460 2024-09-12T03:01:43.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.124+0000 7fda257b0640 1 -- 172.21.15.116:0/3582220940 >> [v2:172.21.15.177:6824/963663634,v1:172.21.15.177:6825/963663634] conn(0x7fda0c0ac460 msgr2=0x7fda0c0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:43.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.124+0000 7fda257b0640 1 --2- 172.21.15.116:0/3582220940 >> [v2:172.21.15.177:6824/963663634,v1:172.21.15.177:6825/963663634] conn(0x7fda0c0ac460 0x7fda0c0ae880 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:43.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.124+0000 7fda257b0640 1 -- 172.21.15.116:0/3582220940 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fda0c0a2c10 msgr2=0x7fda0c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:43.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.124+0000 7fda257b0640 1 --2- 172.21.15.116:0/3582220940 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fda0c0a2c10 0x7fda0c0a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fda10005fd0 tx=0x7fda10005850 comp rx=0 tx=0).stop 2024-09-12T03:01:43.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.124+0000 7fda257b0640 1 -- 172.21.15.116:0/3582220940 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda20102790 msgr2=0x7fda20198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:43.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.124+0000 7fda257b0640 1 --2- 172.21.15.116:0/3582220940 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda20102790 0x7fda20198990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fda08006010 tx=0x7fda080042f0 comp rx=0 tx=0).stop 2024-09-12T03:01:43.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.124+0000 7fda1f7fe640 1 -- 172.21.15.116:0/3582220940 reap_dead start 2024-09-12T03:01:43.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.125+0000 7fda257b0640 1 -- 172.21.15.116:0/3582220940 shutdown_connections 2024-09-12T03:01:43.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.125+0000 7fda257b0640 1 -- 172.21.15.116:0/3582220940 >> 172.21.15.116:0/3582220940 conn(0x7fda200fd8c0 msgr2=0x7fda200feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:43.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.125+0000 7fda257b0640 1 -- 172.21.15.116:0/3582220940 shutdown_connections 2024-09-12T03:01:43.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:42.125+0000 7fda257b0640 1 -- 172.21.15.116:0/3582220940 wait complete. 2024-09-12T03:01:43.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.943+0000 7f0f282d5640 1 -- 172.21.15.116:0/3364468774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f20106a90 msgr2=0x7f0f20106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:43.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.943+0000 7f0f282d5640 1 --2- 172.21.15.116:0/3364468774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f20106a90 0x7f0f20106e70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0f10009930 tx=0x7f0f1002a420 comp rx=0 tx=0).stop 2024-09-12T03:01:43.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.943+0000 7f0f282d5640 1 -- 172.21.15.116:0/3364468774 shutdown_connections 2024-09-12T03:01:43.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.943+0000 7f0f282d5640 1 --2- 172.21.15.116:0/3364468774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f20101430 0x7f0f2010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:43.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.943+0000 7f0f282d5640 1 --2- 172.21.15.116:0/3364468774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f20100a90 0x7f0f20100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:43.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.943+0000 7f0f282d5640 1 --2- 172.21.15.116:0/3364468774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f20106a90 0x7f0f20106e70 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:43.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.943+0000 7f0f282d5640 1 -- 172.21.15.116:0/3364468774 >> 172.21.15.116:0/3364468774 conn(0x7f0f200fc7d0 msgr2=0x7f0f200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:43.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.943+0000 7f0f282d5640 1 -- 172.21.15.116:0/3364468774 shutdown_connections 2024-09-12T03:01:43.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.943+0000 7f0f282d5640 1 -- 172.21.15.116:0/3364468774 wait complete. 2024-09-12T03:01:43.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.944+0000 7f0f282d5640 1 Processor -- start 2024-09-12T03:01:43.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.944+0000 7f0f282d5640 1 -- start start 2024-09-12T03:01:43.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.944+0000 7f0f282d5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f20100a90 0x7f0f20199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:43.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.944+0000 7f0f282d5640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f20101430 0x7f0f20199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:43.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.944+0000 7f0f282d5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f20106a90 0x7f0f201a44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:43.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.944+0000 7f0f282d5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0f20110f30 con 0x7f0f20100a90 2024-09-12T03:01:43.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.944+0000 7f0f282d5640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0f20110db0 con 0x7f0f20101430 2024-09-12T03:01:43.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.944+0000 7f0f282d5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0f201110b0 con 0x7f0f20106a90 2024-09-12T03:01:43.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.944+0000 7f0f26071640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f20100a90 0x7f0f20199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:43.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.944+0000 7f0f26071640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f20100a90 0x7f0f20199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60250/0 (socket says 172.21.15.116:60250) 2024-09-12T03:01:43.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.944+0000 7f0f26071640 1 -- 172.21.15.116:0/3182974044 learned_addr learned my addr 172.21.15.116:0/3182974044 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:43.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.944+0000 7f0f26872640 1 --2- 172.21.15.116:0/3182974044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f20106a90 0x7f0f201a44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:43.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.944+0000 7f0f25870640 1 --2- 172.21.15.116:0/3182974044 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f20101430 0x7f0f20199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:43.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.944+0000 7f0f26071640 1 -- 172.21.15.116:0/3182974044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f20106a90 msgr2=0x7f0f201a44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:43.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.944+0000 7f0f26071640 1 --2- 172.21.15.116:0/3182974044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f20106a90 0x7f0f201a44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:43.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.944+0000 7f0f26071640 1 -- 172.21.15.116:0/3182974044 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f20101430 msgr2=0x7f0f20199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:43.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.944+0000 7f0f26071640 1 --2- 172.21.15.116:0/3182974044 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f20101430 0x7f0f20199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:43.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.944+0000 7f0f26071640 1 -- 172.21.15.116:0/3182974044 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f201a4a00 con 0x7f0f20100a90 2024-09-12T03:01:43.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.944+0000 7f0f25870640 1 --2- 172.21.15.116:0/3182974044 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f20101430 0x7f0f20199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:01:43.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.945+0000 7f0f26071640 1 --2- 172.21.15.116:0/3182974044 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f20100a90 0x7f0f20199440 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0f10009a50 tx=0x7f0f10004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:43.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.945+0000 7f0f177fe640 1 -- 172.21.15.116:0/3182974044 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1003f030 con 0x7f0f20100a90 2024-09-12T03:01:43.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.945+0000 7f0f177fe640 1 -- 172.21.15.116:0/3182974044 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f10003ce0 con 0x7f0f20100a90 2024-09-12T03:01:43.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.945+0000 7f0f282d5640 1 -- 172.21.15.116:0/3182974044 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f201a4bd0 con 0x7f0f20100a90 2024-09-12T03:01:43.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.945+0000 7f0f177fe640 1 -- 172.21.15.116:0/3182974044 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f10035070 con 0x7f0f20100a90 2024-09-12T03:01:43.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.945+0000 7f0f282d5640 1 -- 172.21.15.116:0/3182974044 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f201a5080 con 0x7f0f20100a90 2024-09-12T03:01:43.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.946+0000 7f0f282d5640 1 -- 172.21.15.116:0/3182974044 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0f20062880 con 0x7f0f20100a90 2024-09-12T03:01:43.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.947+0000 7f0f177fe640 1 -- 172.21.15.116:0/3182974044 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f0f10042030 con 0x7f0f20100a90 2024-09-12T03:01:43.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.948+0000 7f0f177fe640 1 --2- 172.21.15.116:0/3182974044 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0f0c0a2c10 0x7f0f0c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:43.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.948+0000 7f0f25870640 1 --2- 172.21.15.116:0/3182974044 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0f0c0a2c10 0x7f0f0c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:43.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.948+0000 7f0f177fe640 1 -- 172.21.15.116:0/3182974044 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f0f100ef9f0 con 0x7f0f20100a90 2024-09-12T03:01:43.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.948+0000 7f0f177fe640 1 --2- 172.21.15.116:0/3182974044 >> [v2:172.21.15.183:6816/1305725258,v1:172.21.15.183:6817/1305725258] conn(0x7f0f0c0ac330 0x7f0f0c0ae750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:43.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.948+0000 7f0f177fe640 1 -- 172.21.15.116:0/3182974044 --> [v2:172.21.15.183:6816/1305725258,v1:172.21.15.183:6817/1305725258] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0f0c0aee00 con 0x7f0f0c0ac330 2024-09-12T03:01:43.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.948+0000 7f0f177fe640 1 -- 172.21.15.116:0/3182974044 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f0f100efed0 con 0x7f0f20100a90 2024-09-12T03:01:43.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.948+0000 7f0f25870640 1 --2- 172.21.15.116:0/3182974044 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0f0c0a2c10 0x7f0f0c0a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c006fd0 tx=0x7f0f1c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:43.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.948+0000 7f0f26872640 1 --2- 172.21.15.116:0/3182974044 >> [v2:172.21.15.183:6816/1305725258,v1:172.21.15.183:6817/1305725258] conn(0x7f0f0c0ac330 0x7f0f0c0ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:43.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.949+0000 7f0f26872640 1 --2- 172.21.15.116:0/3182974044 >> [v2:172.21.15.183:6816/1305725258,v1:172.21.15.183:6817/1305725258] conn(0x7f0f0c0ac330 0x7f0f0c0ae750 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:43.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.949+0000 7f0f177fe640 1 -- 172.21.15.116:0/3182974044 <== osd.8 v2:172.21.15.183:6816/1305725258 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0f0c0aee00 con 0x7f0f0c0ac330 2024-09-12T03:01:43.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.963+0000 7f0f282d5640 1 -- 172.21.15.116:0/3182974044 --> [v2:172.21.15.183:6816/1305725258,v1:172.21.15.183:6817/1305725258] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0f20062d00 con 0x7f0f0c0ac330 2024-09-12T03:01:43.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.964+0000 7f0f177fe640 1 -- 172.21.15.116:0/3182974044 <== osd.8 v2:172.21.15.183:6816/1305725258 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0f20062d00 con 0x7f0f0c0ac330 2024-09-12T03:01:43.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.964+0000 7f0f282d5640 1 -- 172.21.15.116:0/3182974044 >> [v2:172.21.15.183:6816/1305725258,v1:172.21.15.183:6817/1305725258] conn(0x7f0f0c0ac330 msgr2=0x7f0f0c0ae750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:43.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.964+0000 7f0f282d5640 1 --2- 172.21.15.116:0/3182974044 >> [v2:172.21.15.183:6816/1305725258,v1:172.21.15.183:6817/1305725258] conn(0x7f0f0c0ac330 0x7f0f0c0ae750 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:43.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.964+0000 7f0f282d5640 1 -- 172.21.15.116:0/3182974044 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0f0c0a2c10 msgr2=0x7f0f0c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:43.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.964+0000 7f0f282d5640 1 --2- 172.21.15.116:0/3182974044 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0f0c0a2c10 0x7f0f0c0a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c006fd0 tx=0x7f0f1c008040 comp rx=0 tx=0).stop 2024-09-12T03:01:43.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.964+0000 7f0f282d5640 1 -- 172.21.15.116:0/3182974044 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f20100a90 msgr2=0x7f0f20199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:43.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.964+0000 7f0f282d5640 1 --2- 172.21.15.116:0/3182974044 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f20100a90 0x7f0f20199440 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0f10009a50 tx=0x7f0f10004270 comp rx=0 tx=0).stop 2024-09-12T03:01:43.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.964+0000 7f0f26872640 1 -- 172.21.15.116:0/3182974044 reap_dead start 2024-09-12T03:01:43.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.964+0000 7f0f282d5640 1 -- 172.21.15.116:0/3182974044 shutdown_connections 2024-09-12T03:01:43.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.964+0000 7f0f282d5640 1 -- 172.21.15.116:0/3182974044 >> 172.21.15.116:0/3182974044 conn(0x7f0f200fc7d0 msgr2=0x7f0f200fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:43.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.964+0000 7f0f282d5640 1 -- 172.21.15.116:0/3182974044 shutdown_connections 2024-09-12T03:01:43.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:43.964+0000 7f0f282d5640 1 -- 172.21.15.116:0/3182974044 wait complete. 2024-09-12T03:01:44.470 INFO:teuthology.orchestra.run.smithi116.stdout:193273528361 2024-09-12T03:01:44.471 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-12T03:01:44.492 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:44 smithi116 ceph-mon[31410]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T03:01:44.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:44 smithi177 ceph-mon[35305]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T03:01:44.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.419+0000 7f1e4f084640 1 -- 172.21.15.116:0/4244221872 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e48106f00 msgr2=0x7f1e4810ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:44.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.419+0000 7f1e4f084640 1 --2- 172.21.15.116:0/4244221872 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e48106f00 0x7f1e4810ea80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009900 tx=0x7f1e4002a410 comp rx=0 tx=0).stop 2024-09-12T03:01:44.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.419+0000 7f1e4f084640 1 -- 172.21.15.116:0/4244221872 shutdown_connections 2024-09-12T03:01:44.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.419+0000 7f1e4f084640 1 --2- 172.21.15.116:0/4244221872 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e48106f00 0x7f1e4810ea80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:44.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.419+0000 7f1e4f084640 1 --2- 172.21.15.116:0/4244221872 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e48102780 0x7f1e481067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:44.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.419+0000 7f1e4f084640 1 --2- 172.21.15.116:0/4244221872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e48101dd0 0x7f1e481021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:44.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.419+0000 7f1e4f084640 1 -- 172.21.15.116:0/4244221872 >> 172.21.15.116:0/4244221872 conn(0x7f1e480fd8b0 msgr2=0x7f1e480ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:44.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.419+0000 7f1e4f084640 1 -- 172.21.15.116:0/4244221872 shutdown_connections 2024-09-12T03:01:44.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.420+0000 7f1e4f084640 1 -- 172.21.15.116:0/4244221872 wait complete. 2024-09-12T03:01:44.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.420+0000 7f1e4f084640 1 Processor -- start 2024-09-12T03:01:44.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.420+0000 7f1e4f084640 1 -- start start 2024-09-12T03:01:44.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.420+0000 7f1e4f084640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e48101dd0 0x7f1e4819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:44.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.420+0000 7f1e4f084640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e48102780 0x7f1e4819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:44.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.420+0000 7f1e4f084640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e48106f00 0x7f1e4819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:44.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.420+0000 7f1e4f084640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1e48111f60 con 0x7f1e48101dd0 2024-09-12T03:01:44.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.420+0000 7f1e4f084640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1e48111de0 con 0x7f1e48102780 2024-09-12T03:01:44.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.420+0000 7f1e4f084640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1e481120e0 con 0x7f1e48106f00 2024-09-12T03:01:44.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.420+0000 7f1e4ce20640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e48101dd0 0x7f1e4819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:44.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.420+0000 7f1e4ce20640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e48101dd0 0x7f1e4819a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60282/0 (socket says 172.21.15.116:60282) 2024-09-12T03:01:44.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.420+0000 7f1e4ce20640 1 -- 172.21.15.116:0/3850333195 learned_addr learned my addr 172.21.15.116:0/3850333195 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:44.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.420+0000 7f1e3ffff640 1 --2- 172.21.15.116:0/3850333195 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e48102780 0x7f1e4819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:44.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.421+0000 7f1e4d621640 1 --2- 172.21.15.116:0/3850333195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e48106f00 0x7f1e4819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:44.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.421+0000 7f1e4ce20640 1 -- 172.21.15.116:0/3850333195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e48106f00 msgr2=0x7f1e4819ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:44.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.421+0000 7f1e4ce20640 1 --2- 172.21.15.116:0/3850333195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e48106f00 0x7f1e4819ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:44.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.421+0000 7f1e4ce20640 1 -- 172.21.15.116:0/3850333195 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e48102780 msgr2=0x7f1e4819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:44.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.421+0000 7f1e4ce20640 1 --2- 172.21.15.116:0/3850333195 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e48102780 0x7f1e4819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:44.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.421+0000 7f1e4ce20640 1 -- 172.21.15.116:0/3850333195 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4819f570 con 0x7f1e48101dd0 2024-09-12T03:01:44.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.421+0000 7f1e4d621640 1 --2- 172.21.15.116:0/3850333195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e48106f00 0x7f1e4819ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:01:44.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.421+0000 7f1e3ffff640 1 --2- 172.21.15.116:0/3850333195 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e48102780 0x7f1e4819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:44.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.421+0000 7f1e4ce20640 1 --2- 172.21.15.116:0/3850333195 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e48101dd0 0x7f1e4819a5c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400ea80 tx=0x7f1e4400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:44.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.421+0000 7f1e3dffb640 1 -- 172.21.15.116:0/3850333195 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4400ce40 con 0x7f1e48101dd0 2024-09-12T03:01:44.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.421+0000 7f1e3dffb640 1 -- 172.21.15.116:0/3850333195 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e44005020 con 0x7f1e48101dd0 2024-09-12T03:01:44.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.421+0000 7f1e4f084640 1 -- 172.21.15.116:0/3850333195 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e4819f860 con 0x7f1e48101dd0 2024-09-12T03:01:44.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.421+0000 7f1e3dffb640 1 -- 172.21.15.116:0/3850333195 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e44011760 con 0x7f1e48101dd0 2024-09-12T03:01:44.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.421+0000 7f1e4f084640 1 -- 172.21.15.116:0/3850333195 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e481a7140 con 0x7f1e48101dd0 2024-09-12T03:01:44.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.422+0000 7f1e4f084640 1 -- 172.21.15.116:0/3850333195 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1e48075780 con 0x7f1e48101dd0 2024-09-12T03:01:44.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.423+0000 7f1e3dffb640 1 -- 172.21.15.116:0/3850333195 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f1e440051d0 con 0x7f1e48101dd0 2024-09-12T03:01:44.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.424+0000 7f1e3dffb640 1 --2- 172.21.15.116:0/3850333195 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1e300a2c90 0x7f1e300a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:44.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.424+0000 7f1e3dffb640 1 -- 172.21.15.116:0/3850333195 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f1e440cf7a0 con 0x7f1e48101dd0 2024-09-12T03:01:44.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.424+0000 7f1e3ffff640 1 --2- 172.21.15.116:0/3850333195 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1e300a2c90 0x7f1e300a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:44.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.424+0000 7f1e3dffb640 1 --2- 172.21.15.116:0/3850333195 >> [v2:172.21.15.116:6826/3831673668,v1:172.21.15.116:6827/3831673668] conn(0x7f1e300ac4e0 0x7f1e300ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:44.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.424+0000 7f1e3dffb640 1 -- 172.21.15.116:0/3850333195 --> [v2:172.21.15.116:6826/3831673668,v1:172.21.15.116:6827/3831673668] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1e44026ea0 con 0x7f1e300ac4e0 2024-09-12T03:01:44.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.424+0000 7f1e3dffb640 1 -- 172.21.15.116:0/3850333195 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f1e440cfc80 con 0x7f1e48101dd0 2024-09-12T03:01:44.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.424+0000 7f1e4d621640 1 --2- 172.21.15.116:0/3850333195 >> [v2:172.21.15.116:6826/3831673668,v1:172.21.15.116:6827/3831673668] conn(0x7f1e300ac4e0 0x7f1e300ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:44.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.424+0000 7f1e3ffff640 1 --2- 172.21.15.116:0/3850333195 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1e300a2c90 0x7f1e300a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1e480ff6c0 tx=0x7f1e38005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:44.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.424+0000 7f1e4d621640 1 --2- 172.21.15.116:0/3850333195 >> [v2:172.21.15.116:6826/3831673668,v1:172.21.15.116:6827/3831673668] conn(0x7f1e300ac4e0 0x7f1e300ae900 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:44.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.425+0000 7f1e3dffb640 1 -- 172.21.15.116:0/3850333195 <== osd.9 v2:172.21.15.116:6826/3831673668 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1e44026ea0 con 0x7f1e300ac4e0 2024-09-12T03:01:44.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.439+0000 7f1e4f084640 1 -- 172.21.15.116:0/3850333195 --> [v2:172.21.15.116:6826/3831673668,v1:172.21.15.116:6827/3831673668] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1e481032b0 con 0x7f1e300ac4e0 2024-09-12T03:01:44.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.439+0000 7f1e3dffb640 1 -- 172.21.15.116:0/3850333195 <== osd.9 v2:172.21.15.116:6826/3831673668 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1e481032b0 con 0x7f1e300ac4e0 2024-09-12T03:01:44.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.440+0000 7f1e4f084640 1 -- 172.21.15.116:0/3850333195 >> [v2:172.21.15.116:6826/3831673668,v1:172.21.15.116:6827/3831673668] conn(0x7f1e300ac4e0 msgr2=0x7f1e300ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:44.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.440+0000 7f1e4f084640 1 --2- 172.21.15.116:0/3850333195 >> [v2:172.21.15.116:6826/3831673668,v1:172.21.15.116:6827/3831673668] conn(0x7f1e300ac4e0 0x7f1e300ae900 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:44.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.440+0000 7f1e4f084640 1 -- 172.21.15.116:0/3850333195 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1e300a2c90 msgr2=0x7f1e300a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:44.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.440+0000 7f1e4f084640 1 --2- 172.21.15.116:0/3850333195 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1e300a2c90 0x7f1e300a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1e480ff6c0 tx=0x7f1e38005f90 comp rx=0 tx=0).stop 2024-09-12T03:01:44.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.440+0000 7f1e4f084640 1 -- 172.21.15.116:0/3850333195 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e48101dd0 msgr2=0x7f1e4819a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:44.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.440+0000 7f1e4f084640 1 --2- 172.21.15.116:0/3850333195 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e48101dd0 0x7f1e4819a5c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400ea80 tx=0x7f1e4400ef40 comp rx=0 tx=0).stop 2024-09-12T03:01:44.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.440+0000 7f1e4d621640 1 -- 172.21.15.116:0/3850333195 reap_dead start 2024-09-12T03:01:44.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.440+0000 7f1e4f084640 1 -- 172.21.15.116:0/3850333195 shutdown_connections 2024-09-12T03:01:44.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.440+0000 7f1e4f084640 1 -- 172.21.15.116:0/3850333195 >> 172.21.15.116:0/3850333195 conn(0x7f1e480fd8b0 msgr2=0x7f1e480ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:44.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.440+0000 7f1e4f084640 1 -- 172.21.15.116:0/3850333195 shutdown_connections 2024-09-12T03:01:44.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:44.440+0000 7f1e4f084640 1 -- 172.21.15.116:0/3850333195 wait complete. 2024-09-12T03:01:44.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:44 smithi183 ceph-mon[35535]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T03:01:45.249 INFO:teuthology.orchestra.run.smithi116.stdout:253403070497 2024-09-12T03:01:45.249 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-12T03:01:45.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.432+0000 7f2f923c6640 1 -- 172.21.15.116:0/3106322234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f8c103740 msgr2=0x7f2f8c107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:45.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.432+0000 7f2f923c6640 1 --2- 172.21.15.116:0/3106322234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f8c103740 0x7f2f8c107b80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2f80009930 tx=0x7f2f8002a420 comp rx=0 tx=0).stop 2024-09-12T03:01:45.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.432+0000 7f2f923c6640 1 -- 172.21.15.116:0/3106322234 shutdown_connections 2024-09-12T03:01:45.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.432+0000 7f2f923c6640 1 --2- 172.21.15.116:0/3106322234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f8c108290 0x7f2f8c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:45.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.432+0000 7f2f923c6640 1 --2- 172.21.15.116:0/3106322234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f8c103740 0x7f2f8c107b80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:45.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.432+0000 7f2f923c6640 1 --2- 172.21.15.116:0/3106322234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f8c102d90 0x7f2f8c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:45.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.432+0000 7f2f923c6640 1 -- 172.21.15.116:0/3106322234 >> 172.21.15.116:0/3106322234 conn(0x7f2f8c0fc7d0 msgr2=0x7f2f8c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:45.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.432+0000 7f2f923c6640 1 -- 172.21.15.116:0/3106322234 shutdown_connections 2024-09-12T03:01:45.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.432+0000 7f2f923c6640 1 -- 172.21.15.116:0/3106322234 wait complete. 2024-09-12T03:01:45.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.432+0000 7f2f923c6640 1 Processor -- start 2024-09-12T03:01:45.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.432+0000 7f2f923c6640 1 -- start start 2024-09-12T03:01:45.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.433+0000 7f2f923c6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f8c102d90 0x7f2f8c1009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:45.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.433+0000 7f2f923c6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f8c103740 0x7f2f8c100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:45.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.433+0000 7f2f923c6640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f8c108290 0x7f2f8c101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:45.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.433+0000 7f2f923c6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c1120b0 con 0x7f2f8c103740 2024-09-12T03:01:45.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.433+0000 7f2f923c6640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c111f30 con 0x7f2f8c108290 2024-09-12T03:01:45.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.433+0000 7f2f923c6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c112230 con 0x7f2f8c102d90 2024-09-12T03:01:45.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.433+0000 7f2f8b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f8c103740 0x7f2f8c100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:45.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.433+0000 7f2f8b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f8c103740 0x7f2f8c100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60296/0 (socket says 172.21.15.116:60296) 2024-09-12T03:01:45.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.433+0000 7f2f8b7fe640 1 -- 172.21.15.116:0/4047342875 learned_addr learned my addr 172.21.15.116:0/4047342875 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:45.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.433+0000 7f2f8bfff640 1 --2- 172.21.15.116:0/4047342875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f8c102d90 0x7f2f8c1009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:45.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.433+0000 7f2f90963640 1 --2- 172.21.15.116:0/4047342875 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f8c108290 0x7f2f8c101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:45.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.433+0000 7f2f8b7fe640 1 -- 172.21.15.116:0/4047342875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f8c102d90 msgr2=0x7f2f8c1009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:45.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.433+0000 7f2f8b7fe640 1 --2- 172.21.15.116:0/4047342875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f8c102d90 0x7f2f8c1009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:45.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.433+0000 7f2f8b7fe640 1 -- 172.21.15.116:0/4047342875 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f8c108290 msgr2=0x7f2f8c101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:45.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.433+0000 7f2f8b7fe640 1 --2- 172.21.15.116:0/4047342875 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f8c108290 0x7f2f8c101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:45.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.433+0000 7f2f8b7fe640 1 -- 172.21.15.116:0/4047342875 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f8c1a6b30 con 0x7f2f8c103740 2024-09-12T03:01:45.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.433+0000 7f2f90963640 1 --2- 172.21.15.116:0/4047342875 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f8c108290 0x7f2f8c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:01:45.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.433+0000 7f2f8b7fe640 1 --2- 172.21.15.116:0/4047342875 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f8c103740 0x7f2f8c100f30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2f80005940 tx=0x7f2f80004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:45.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.434+0000 7f2f897fa640 1 -- 172.21.15.116:0/4047342875 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8003f030 con 0x7f2f8c103740 2024-09-12T03:01:45.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.434+0000 7f2f923c6640 1 -- 172.21.15.116:0/4047342875 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f8c1a6dc0 con 0x7f2f8c103740 2024-09-12T03:01:45.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.434+0000 7f2f897fa640 1 -- 172.21.15.116:0/4047342875 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f80003ad0 con 0x7f2f8c103740 2024-09-12T03:01:45.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.434+0000 7f2f897fa640 1 -- 172.21.15.116:0/4047342875 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f80035070 con 0x7f2f8c103740 2024-09-12T03:01:45.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.434+0000 7f2f923c6640 1 -- 172.21.15.116:0/4047342875 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f8c1a7260 con 0x7f2f8c103740 2024-09-12T03:01:45.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.434+0000 7f2f923c6640 1 -- 172.21.15.116:0/4047342875 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2f8c062900 con 0x7f2f8c103740 2024-09-12T03:01:45.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.436+0000 7f2f897fa640 1 -- 172.21.15.116:0/4047342875 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2f80042030 con 0x7f2f8c103740 2024-09-12T03:01:45.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.436+0000 7f2f897fa640 1 --2- 172.21.15.116:0/4047342875 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2f740a2c90 0x7f2f740a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.437+0000 7f2f8bfff640 1 --2- 172.21.15.116:0/4047342875 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2f740a2c90 0x7f2f740a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.437+0000 7f2f897fa640 1 -- 172.21.15.116:0/4047342875 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f2f800efa60 con 0x7f2f8c103740 2024-09-12T03:01:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.437+0000 7f2f897fa640 1 --2- 172.21.15.116:0/4047342875 >> [v2:172.21.15.116:6818/1121291186,v1:172.21.15.116:6819/1121291186] conn(0x7f2f740ac4e0 0x7f2f740ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.437+0000 7f2f897fa640 1 -- 172.21.15.116:0/4047342875 --> [v2:172.21.15.116:6818/1121291186,v1:172.21.15.116:6819/1121291186] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2f80001d30 con 0x7f2f740ac4e0 2024-09-12T03:01:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.437+0000 7f2f897fa640 1 -- 172.21.15.116:0/4047342875 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f2f800eff40 con 0x7f2f8c103740 2024-09-12T03:01:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.437+0000 7f2f90963640 1 --2- 172.21.15.116:0/4047342875 >> [v2:172.21.15.116:6818/1121291186,v1:172.21.15.116:6819/1121291186] conn(0x7f2f740ac4e0 0x7f2f740ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.437+0000 7f2f8bfff640 1 --2- 172.21.15.116:0/4047342875 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2f740a2c90 0x7f2f740a5150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c009990 tx=0x7f2f7c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.437+0000 7f2f90963640 1 --2- 172.21.15.116:0/4047342875 >> [v2:172.21.15.116:6818/1121291186,v1:172.21.15.116:6819/1121291186] conn(0x7f2f740ac4e0 0x7f2f740ae900 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.438+0000 7f2f897fa640 1 -- 172.21.15.116:0/4047342875 <== osd.6 v2:172.21.15.116:6818/1121291186 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2f80001d30 con 0x7f2f740ac4e0 2024-09-12T03:01:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.451+0000 7f2f923c6640 1 -- 172.21.15.116:0/4047342875 --> [v2:172.21.15.116:6818/1121291186,v1:172.21.15.116:6819/1121291186] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2f8c062d30 con 0x7f2f740ac4e0 2024-09-12T03:01:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.451+0000 7f2f897fa640 1 -- 172.21.15.116:0/4047342875 <== osd.6 v2:172.21.15.116:6818/1121291186 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2f8c062d30 con 0x7f2f740ac4e0 2024-09-12T03:01:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.452+0000 7f2f923c6640 1 -- 172.21.15.116:0/4047342875 >> [v2:172.21.15.116:6818/1121291186,v1:172.21.15.116:6819/1121291186] conn(0x7f2f740ac4e0 msgr2=0x7f2f740ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.452+0000 7f2f923c6640 1 --2- 172.21.15.116:0/4047342875 >> [v2:172.21.15.116:6818/1121291186,v1:172.21.15.116:6819/1121291186] conn(0x7f2f740ac4e0 0x7f2f740ae900 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.452+0000 7f2f923c6640 1 -- 172.21.15.116:0/4047342875 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2f740a2c90 msgr2=0x7f2f740a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.452+0000 7f2f923c6640 1 --2- 172.21.15.116:0/4047342875 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2f740a2c90 0x7f2f740a5150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c009990 tx=0x7f2f7c009110 comp rx=0 tx=0).stop 2024-09-12T03:01:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.452+0000 7f2f923c6640 1 -- 172.21.15.116:0/4047342875 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f8c103740 msgr2=0x7f2f8c100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.452+0000 7f2f923c6640 1 --2- 172.21.15.116:0/4047342875 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f8c103740 0x7f2f8c100f30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2f80005940 tx=0x7f2f80004270 comp rx=0 tx=0).stop 2024-09-12T03:01:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.452+0000 7f2f90963640 1 -- 172.21.15.116:0/4047342875 reap_dead start 2024-09-12T03:01:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.452+0000 7f2f923c6640 1 -- 172.21.15.116:0/4047342875 shutdown_connections 2024-09-12T03:01:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.452+0000 7f2f923c6640 1 -- 172.21.15.116:0/4047342875 >> 172.21.15.116:0/4047342875 conn(0x7f2f8c0fc7d0 msgr2=0x7f2f8c105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.452+0000 7f2f923c6640 1 -- 172.21.15.116:0/4047342875 shutdown_connections 2024-09-12T03:01:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:45.452+0000 7f2f923c6640 1 -- 172.21.15.116:0/4047342875 wait complete. 2024-09-12T03:01:46.157 INFO:teuthology.orchestra.run.smithi116.stdout:223338299430 2024-09-12T03:01:46.157 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-12T03:01:46.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:46 smithi177 ceph-mon[35305]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T03:01:46.505 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:46 smithi183 ceph-mon[35535]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T03:01:46.548 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:46 smithi116 ceph-mon[31410]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T03:01:46.574 INFO:teuthology.orchestra.run.smithi116.stdout:68719476804 2024-09-12T03:01:46.575 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-12T03:01:47.033 INFO:teuthology.orchestra.run.smithi116.stdout:94489280573 2024-09-12T03:01:47.033 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-12T03:01:47.526 INFO:teuthology.orchestra.run.smithi116.stdout:171798691888 2024-09-12T03:01:47.526 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-12T03:01:47.998 INFO:teuthology.orchestra.run.smithi116.stdout:124554051642 2024-09-12T03:01:47.999 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-12T03:01:48.488 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:48 smithi116 ceph-mon[31410]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T03:01:48.489 INFO:teuthology.orchestra.run.smithi116.stdout:274877906971 2024-09-12T03:01:48.489 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-12T03:01:48.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:48 smithi177 ceph-mon[35305]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T03:01:48.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:48 smithi183 ceph-mon[35535]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T03:01:48.955 INFO:teuthology.orchestra.run.smithi116.stdout:309237645334 2024-09-12T03:01:48.956 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-12T03:01:49.055 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:49.055 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:49.057 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:49.059 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:49.062 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:49.065 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:49.065 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:49.069 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:49.069 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:49.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.825+0000 7f14753e1640 1 -- 172.21.15.116:0/557423348 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1470106f20 msgr2=0x7f147010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:49.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.825+0000 7f14753e1640 1 --2- 172.21.15.116:0/557423348 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1470106f20 0x7f147010eaa0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1468009ca0 tx=0x7f146802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:01:49.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.825+0000 7f14753e1640 1 -- 172.21.15.116:0/557423348 shutdown_connections 2024-09-12T03:01:49.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.825+0000 7f14753e1640 1 --2- 172.21.15.116:0/557423348 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1470106f20 0x7f147010eaa0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:49.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.825+0000 7f14753e1640 1 --2- 172.21.15.116:0/557423348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14701027a0 0x7f14701067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:49.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.825+0000 7f14753e1640 1 --2- 172.21.15.116:0/557423348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1470101df0 0x7f14701021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:49.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.825+0000 7f14753e1640 1 -- 172.21.15.116:0/557423348 >> 172.21.15.116:0/557423348 conn(0x7f14700fd8b0 msgr2=0x7f14700ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:49.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.825+0000 7f14753e1640 1 -- 172.21.15.116:0/557423348 shutdown_connections 2024-09-12T03:01:49.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.825+0000 7f14753e1640 1 -- 172.21.15.116:0/557423348 wait complete. 2024-09-12T03:01:49.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.826+0000 7f14753e1640 1 Processor -- start 2024-09-12T03:01:49.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.826+0000 7f14753e1640 1 -- start start 2024-09-12T03:01:49.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.826+0000 7f14753e1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1470101df0 0x7f14700744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:49.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.826+0000 7f14753e1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14701027a0 0x7f1470074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:49.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.826+0000 7f14753e1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1470106f20 0x7f1470074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:49.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.826+0000 7f14753e1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1470111f80 con 0x7f1470101df0 2024-09-12T03:01:49.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.826+0000 7f14753e1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1470111e00 con 0x7f1470106f20 2024-09-12T03:01:49.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.826+0000 7f14753e1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1470112100 con 0x7f14701027a0 2024-09-12T03:01:49.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.826+0000 7f146effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1470101df0 0x7f14700744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:49.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.826+0000 7f146effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1470101df0 0x7f14700744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60314/0 (socket says 172.21.15.116:60314) 2024-09-12T03:01:49.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.826+0000 7f146effd640 1 -- 172.21.15.116:0/1426200226 learned_addr learned my addr 172.21.15.116:0/1426200226 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:49.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.826+0000 7f146f7fe640 1 --2- 172.21.15.116:0/1426200226 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1470106f20 0x7f1470074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:49.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.826+0000 7f146e7fc640 1 --2- 172.21.15.116:0/1426200226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14701027a0 0x7f1470074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:49.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.827+0000 7f146effd640 1 -- 172.21.15.116:0/1426200226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14701027a0 msgr2=0x7f1470074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:49.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.827+0000 7f146effd640 1 --2- 172.21.15.116:0/1426200226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14701027a0 0x7f1470074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:49.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.827+0000 7f146effd640 1 -- 172.21.15.116:0/1426200226 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1470106f20 msgr2=0x7f1470074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:49.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.827+0000 7f146effd640 1 --2- 172.21.15.116:0/1426200226 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1470106f20 0x7f1470074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:49.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.827+0000 7f146effd640 1 -- 172.21.15.116:0/1426200226 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14701a6840 con 0x7f1470101df0 2024-09-12T03:01:49.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.827+0000 7f146f7fe640 1 --2- 172.21.15.116:0/1426200226 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1470106f20 0x7f1470074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:01:49.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.827+0000 7f146effd640 1 --2- 172.21.15.116:0/1426200226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1470101df0 0x7f14700744f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f146000d950 tx=0x7f146000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:49.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.827+0000 7f1457fff640 1 -- 172.21.15.116:0/1426200226 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1460017070 con 0x7f1470101df0 2024-09-12T03:01:49.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.827+0000 7f14753e1640 1 -- 172.21.15.116:0/1426200226 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14701a6b30 con 0x7f1470101df0 2024-09-12T03:01:49.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.827+0000 7f1457fff640 1 -- 172.21.15.116:0/1426200226 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1460009c30 con 0x7f1470101df0 2024-09-12T03:01:49.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.827+0000 7f1457fff640 1 -- 172.21.15.116:0/1426200226 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146000f6b0 con 0x7f1470101df0 2024-09-12T03:01:49.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.827+0000 7f14753e1640 1 -- 172.21.15.116:0/1426200226 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14701a70c0 con 0x7f1470101df0 2024-09-12T03:01:49.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.828+0000 7f14753e1640 1 -- 172.21.15.116:0/1426200226 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14701021d0 con 0x7f1470101df0 2024-09-12T03:01:49.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.829+0000 7f1457fff640 1 -- 172.21.15.116:0/1426200226 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f14600109d0 con 0x7f1470101df0 2024-09-12T03:01:49.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.830+0000 7f1457fff640 1 --2- 172.21.15.116:0/1426200226 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f14580a2c10 0x7f14580a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:49.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.830+0000 7f146e7fc640 1 --2- 172.21.15.116:0/1426200226 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f14580a2c10 0x7f14580a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:49.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.830+0000 7f1457fff640 1 -- 172.21.15.116:0/1426200226 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f14600cece0 con 0x7f1470101df0 2024-09-12T03:01:49.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.830+0000 7f146e7fc640 1 --2- 172.21.15.116:0/1426200226 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f14580a2c10 0x7f14580a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1464005fd0 tx=0x7f146400b510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:49.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:49.834+0000 7f1457fff640 1 -- 172.21.15.116:0/1426200226 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1460009de0 con 0x7f1470101df0 2024-09-12T03:01:50.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.014+0000 7f14753e1640 1 -- 172.21.15.116:0/1426200226 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f147010a160 con 0x7f1470101df0 2024-09-12T03:01:50.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.014+0000 7f1457fff640 1 -- 172.21.15.116:0/1426200226 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f146009bc90 con 0x7f1470101df0 2024-09-12T03:01:50.015 INFO:teuthology.orchestra.run.smithi116.stdout:253403070498 2024-09-12T03:01:50.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.016+0000 7f14753e1640 1 -- 172.21.15.116:0/1426200226 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f14580a2c10 msgr2=0x7f14580a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:50.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.016+0000 7f14753e1640 1 --2- 172.21.15.116:0/1426200226 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f14580a2c10 0x7f14580a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1464005fd0 tx=0x7f146400b510 comp rx=0 tx=0).stop 2024-09-12T03:01:50.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.016+0000 7f14753e1640 1 -- 172.21.15.116:0/1426200226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1470101df0 msgr2=0x7f14700744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:50.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.016+0000 7f14753e1640 1 --2- 172.21.15.116:0/1426200226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1470101df0 0x7f14700744f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f146000d950 tx=0x7f146000de10 comp rx=0 tx=0).stop 2024-09-12T03:01:50.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.017+0000 7f14753e1640 1 -- 172.21.15.116:0/1426200226 shutdown_connections 2024-09-12T03:01:50.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.017+0000 7f14753e1640 1 --2- 172.21.15.116:0/1426200226 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f14580a2c10 0x7f14580a50d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:50.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.017+0000 7f14753e1640 1 --2- 172.21.15.116:0/1426200226 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1470106f20 0x7f1470074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:50.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.017+0000 7f14753e1640 1 --2- 172.21.15.116:0/1426200226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14701027a0 0x7f1470074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:50.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.017+0000 7f14753e1640 1 --2- 172.21.15.116:0/1426200226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1470101df0 0x7f14700744f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:50.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.017+0000 7f14753e1640 1 -- 172.21.15.116:0/1426200226 >> 172.21.15.116:0/1426200226 conn(0x7f14700fd8b0 msgr2=0x7f14700feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:50.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.017+0000 7f14753e1640 1 -- 172.21.15.116:0/1426200226 shutdown_connections 2024-09-12T03:01:50.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.017+0000 7f14753e1640 1 -- 172.21.15.116:0/1426200226 wait complete. 2024-09-12T03:01:50.264 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:50 smithi116 ceph-mon[31410]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:01:50.264 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:50 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1426200226' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T03:01:50.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:50 smithi177 ceph-mon[35305]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:01:50.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:50 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1426200226' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T03:01:50.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.524+0000 7f646bd5a640 1 -- 172.21.15.116:0/1910364398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6464102ef0 msgr2=0x7f646410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:50.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.524+0000 7f646bd5a640 1 --2- 172.21.15.116:0/1910364398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6464102ef0 0x7f646410da40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f645c009ca0 tx=0x7f645c02a950 comp rx=0 tx=0).stop 2024-09-12T03:01:50.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.524+0000 7f646bd5a640 1 -- 172.21.15.116:0/1910364398 shutdown_connections 2024-09-12T03:01:50.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.524+0000 7f646bd5a640 1 --2- 172.21.15.116:0/1910364398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6464102ef0 0x7f646410da40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:50.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.524+0000 7f646bd5a640 1 --2- 172.21.15.116:0/1910364398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6464102570 0x7f64641029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:50.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.524+0000 7f646bd5a640 1 --2- 172.21.15.116:0/1910364398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6464100e20 0x7f6464101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:50.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.524+0000 7f646bd5a640 1 -- 172.21.15.116:0/1910364398 >> 172.21.15.116:0/1910364398 conn(0x7f64640fc7d0 msgr2=0x7f64640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:50.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.524+0000 7f646bd5a640 1 -- 172.21.15.116:0/1910364398 shutdown_connections 2024-09-12T03:01:50.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.524+0000 7f646bd5a640 1 -- 172.21.15.116:0/1910364398 wait complete. 2024-09-12T03:01:50.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.524+0000 7f646bd5a640 1 Processor -- start 2024-09-12T03:01:50.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.524+0000 7f646bd5a640 1 -- start start 2024-09-12T03:01:50.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.525+0000 7f646bd5a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6464100e20 0x7f64641985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:50.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.525+0000 7f646bd5a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6464102570 0x7f6464198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:50.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.525+0000 7f646bd5a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6464102ef0 0x7f646419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:50.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.525+0000 7f646bd5a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6464112120 con 0x7f6464102ef0 2024-09-12T03:01:50.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.525+0000 7f646bd5a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6464111fa0 con 0x7f6464102570 2024-09-12T03:01:50.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.525+0000 7f646bd5a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f64641122a0 con 0x7f6464100e20 2024-09-12T03:01:50.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.525+0000 7f646a2f7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6464102ef0 0x7f646419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:50.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.525+0000 7f646a2f7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6464102ef0 0x7f646419ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60336/0 (socket says 172.21.15.116:60336) 2024-09-12T03:01:50.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.525+0000 7f646a2f7640 1 -- 172.21.15.116:0/4107238840 learned_addr learned my addr 172.21.15.116:0/4107238840 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:50.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.525+0000 7f6469af6640 1 --2- 172.21.15.116:0/4107238840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6464100e20 0x7f64641985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:50.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.525+0000 7f64692f5640 1 --2- 172.21.15.116:0/4107238840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6464102570 0x7f6464198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:50.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.525+0000 7f646a2f7640 1 -- 172.21.15.116:0/4107238840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6464100e20 msgr2=0x7f64641985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:50.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.525+0000 7f646a2f7640 1 --2- 172.21.15.116:0/4107238840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6464100e20 0x7f64641985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:50.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.525+0000 7f646a2f7640 1 -- 172.21.15.116:0/4107238840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6464102570 msgr2=0x7f6464198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:50.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.525+0000 7f646a2f7640 1 --2- 172.21.15.116:0/4107238840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6464102570 0x7f6464198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:50.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.525+0000 7f646a2f7640 1 -- 172.21.15.116:0/4107238840 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f646419d590 con 0x7f6464102ef0 2024-09-12T03:01:50.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.525+0000 7f64692f5640 1 --2- 172.21.15.116:0/4107238840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6464102570 0x7f6464198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:01:50.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.525+0000 7f646a2f7640 1 --2- 172.21.15.116:0/4107238840 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6464102ef0 0x7f646419ceb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f645c006010 tx=0x7f645c02f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:50.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.525+0000 7f645affd640 1 -- 172.21.15.116:0/4107238840 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645c034040 con 0x7f6464102ef0 2024-09-12T03:01:50.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.526+0000 7f646bd5a640 1 -- 172.21.15.116:0/4107238840 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f646419d820 con 0x7f6464102ef0 2024-09-12T03:01:50.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.526+0000 7f645affd640 1 -- 172.21.15.116:0/4107238840 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f645c003e00 con 0x7f6464102ef0 2024-09-12T03:01:50.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.526+0000 7f645affd640 1 -- 172.21.15.116:0/4107238840 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645c045480 con 0x7f6464102ef0 2024-09-12T03:01:50.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.526+0000 7f646bd5a640 1 -- 172.21.15.116:0/4107238840 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64641a5160 con 0x7f6464102ef0 2024-09-12T03:01:50.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.526+0000 7f646bd5a640 1 -- 172.21.15.116:0/4107238840 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6464101280 con 0x7f6464102ef0 2024-09-12T03:01:50.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.528+0000 7f645affd640 1 -- 172.21.15.116:0/4107238840 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f645c037020 con 0x7f6464102ef0 2024-09-12T03:01:50.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.528+0000 7f645affd640 1 --2- 172.21.15.116:0/4107238840 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f64500a2c10 0x7f64500a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:50.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.528+0000 7f6469af6640 1 --2- 172.21.15.116:0/4107238840 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f64500a2c10 0x7f64500a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:50.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.528+0000 7f645affd640 1 -- 172.21.15.116:0/4107238840 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f645c0f0130 con 0x7f6464102ef0 2024-09-12T03:01:50.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.529+0000 7f6469af6640 1 --2- 172.21.15.116:0/4107238840 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f64500a2c10 0x7f64500a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6460009890 tx=0x7f6460008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:50.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.532+0000 7f645affd640 1 -- 172.21.15.116:0/4107238840 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f645c0bd1f0 con 0x7f6464102ef0 2024-09-12T03:01:50.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:50 smithi183 ceph-mon[35535]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:01:50.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:50 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1426200226' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T03:01:50.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.703+0000 7f646bd5a640 1 -- 172.21.15.116:0/4107238840 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f6464108b00 con 0x7f6464102ef0 2024-09-12T03:01:50.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.703+0000 7f645affd640 1 -- 172.21.15.116:0/4107238840 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f645c0bcaf0 con 0x7f6464102ef0 2024-09-12T03:01:50.704 INFO:teuthology.orchestra.run.smithi116.stdout:51539607627 2024-09-12T03:01:50.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.705+0000 7f646bd5a640 1 -- 172.21.15.116:0/4107238840 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f64500a2c10 msgr2=0x7f64500a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:50.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.705+0000 7f646bd5a640 1 --2- 172.21.15.116:0/4107238840 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f64500a2c10 0x7f64500a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6460009890 tx=0x7f6460008040 comp rx=0 tx=0).stop 2024-09-12T03:01:50.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.705+0000 7f646bd5a640 1 -- 172.21.15.116:0/4107238840 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6464102ef0 msgr2=0x7f646419ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:50.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.705+0000 7f646bd5a640 1 --2- 172.21.15.116:0/4107238840 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6464102ef0 0x7f646419ceb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f645c006010 tx=0x7f645c02f6f0 comp rx=0 tx=0).stop 2024-09-12T03:01:50.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.705+0000 7f646bd5a640 1 -- 172.21.15.116:0/4107238840 shutdown_connections 2024-09-12T03:01:50.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.705+0000 7f646bd5a640 1 --2- 172.21.15.116:0/4107238840 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f64500a2c10 0x7f64500a50d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:50.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.705+0000 7f646bd5a640 1 --2- 172.21.15.116:0/4107238840 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6464102ef0 0x7f646419ceb0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:50.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.705+0000 7f646bd5a640 1 --2- 172.21.15.116:0/4107238840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6464102570 0x7f6464198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:50.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.705+0000 7f646bd5a640 1 --2- 172.21.15.116:0/4107238840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6464100e20 0x7f64641985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:50.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.705+0000 7f646bd5a640 1 -- 172.21.15.116:0/4107238840 >> 172.21.15.116:0/4107238840 conn(0x7f64640fc7d0 msgr2=0x7f64640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:50.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.705+0000 7f646bd5a640 1 -- 172.21.15.116:0/4107238840 shutdown_connections 2024-09-12T03:01:50.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:50.705+0000 7f646bd5a640 1 -- 172.21.15.116:0/4107238840 wait complete. 2024-09-12T03:01:51.210 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:51.211 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070497 got 253403070498 for osd.9 2024-09-12T03:01:51.211 DEBUG:teuthology.parallel:result is None 2024-09-12T03:01:51.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:51 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/4107238840' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T03:01:51.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:51 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/4107238840' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T03:01:51.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:51 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/4107238840' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T03:01:52.235 INFO:tasks.cephadm.ceph_manager.ceph:need seq 51539607624 got 51539607627 for osd.1 2024-09-12T03:01:52.235 DEBUG:teuthology.parallel:result is None 2024-09-12T03:01:52.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:52 smithi177 ceph-mon[35305]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:01:52.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:52 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/4086721705' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T03:01:52.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:52 smithi116 ceph-mon[31410]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:01:52.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:52 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/4086721705' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T03:01:52.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:52 smithi183 ceph-mon[35535]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:01:52.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:52 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/4086721705' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T03:01:53.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.398+0000 7f0516cdc640 1 -- 172.21.15.116:0/2462994264 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0510100a80 msgr2=0x7f0510100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:53.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.398+0000 7f0516cdc640 1 --2- 172.21.15.116:0/2462994264 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0510100a80 0x7f0510100ee0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009930 tx=0x7f04fc02a420 comp rx=0 tx=0).stop 2024-09-12T03:01:53.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.398+0000 7f0516cdc640 1 -- 172.21.15.116:0/2462994264 shutdown_connections 2024-09-12T03:01:53.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.398+0000 7f0516cdc640 1 --2- 172.21.15.116:0/2462994264 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0510101420 0x7f051010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:53.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.398+0000 7f0516cdc640 1 --2- 172.21.15.116:0/2462994264 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0510100a80 0x7f0510100ee0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:53.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.398+0000 7f0516cdc640 1 --2- 172.21.15.116:0/2462994264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0510106a80 0x7f0510106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:53.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.398+0000 7f0516cdc640 1 -- 172.21.15.116:0/2462994264 >> 172.21.15.116:0/2462994264 conn(0x7f05100fc7a0 msgr2=0x7f05100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:53.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.398+0000 7f0516cdc640 1 -- 172.21.15.116:0/2462994264 shutdown_connections 2024-09-12T03:01:53.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.398+0000 7f0516cdc640 1 -- 172.21.15.116:0/2462994264 wait complete. 2024-09-12T03:01:53.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.398+0000 7f0516cdc640 1 Processor -- start 2024-09-12T03:01:53.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.399+0000 7f0516cdc640 1 -- start start 2024-09-12T03:01:53.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.399+0000 7f0516cdc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0510100a80 0x7f05101962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:53.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.399+0000 7f0516cdc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0510101420 0x7f05101967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:53.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.399+0000 7f0516cdc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0510106a80 0x7f051019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:53.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.399+0000 7f0516cdc640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f051010fdb0 con 0x7f0510101420 2024-09-12T03:01:53.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.399+0000 7f0516cdc640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f051010fc30 con 0x7f0510100a80 2024-09-12T03:01:53.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.399+0000 7f0516cdc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f051010ff30 con 0x7f0510106a80 2024-09-12T03:01:53.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.399+0000 7f050ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0510101420 0x7f05101967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:53.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.399+0000 7f050ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0510101420 0x7f05101967f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34168/0 (socket says 172.21.15.116:34168) 2024-09-12T03:01:53.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.399+0000 7f050ffff640 1 -- 172.21.15.116:0/4086721705 learned_addr learned my addr 172.21.15.116:0/4086721705 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:53.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.399+0000 7f0515279640 1 --2- 172.21.15.116:0/4086721705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0510106a80 0x7f051019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:53.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.399+0000 7f0514a78640 1 --2- 172.21.15.116:0/4086721705 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0510100a80 0x7f05101962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:53.362 INFO:teuthology.orchestra.run.smithi116.stdout:141733920822 2024-09-12T03:01:53.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.399+0000 7f050ffff640 1 -- 172.21.15.116:0/4086721705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0510106a80 msgr2=0x7f051019ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:53.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.399+0000 7f050ffff640 1 --2- 172.21.15.116:0/4086721705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0510106a80 0x7f051019ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:53.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.399+0000 7f050ffff640 1 -- 172.21.15.116:0/4086721705 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0510100a80 msgr2=0x7f05101962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:53.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.399+0000 7f050ffff640 1 --2- 172.21.15.116:0/4086721705 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0510100a80 0x7f05101962b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:53.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.399+0000 7f050ffff640 1 -- 172.21.15.116:0/4086721705 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f051019b260 con 0x7f0510101420 2024-09-12T03:01:53.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.399+0000 7f050ffff640 1 --2- 172.21.15.116:0/4086721705 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0510101420 0x7f05101967f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009a50 tx=0x7f04fc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:53.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.399+0000 7f050dffb640 1 -- 172.21.15.116:0/4086721705 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc03f030 con 0x7f0510101420 2024-09-12T03:01:53.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.399+0000 7f0516cdc640 1 -- 172.21.15.116:0/4086721705 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f051019b4f0 con 0x7f0510101420 2024-09-12T03:01:53.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.399+0000 7f050dffb640 1 -- 172.21.15.116:0/4086721705 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04fc003ce0 con 0x7f0510101420 2024-09-12T03:01:53.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.399+0000 7f050dffb640 1 -- 172.21.15.116:0/4086721705 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc035070 con 0x7f0510101420 2024-09-12T03:01:53.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.400+0000 7f0516cdc640 1 -- 172.21.15.116:0/4086721705 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05101a2e30 con 0x7f0510101420 2024-09-12T03:01:53.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.401+0000 7f0516cdc640 1 -- 172.21.15.116:0/4086721705 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05101021c0 con 0x7f0510101420 2024-09-12T03:01:53.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.402+0000 7f050dffb640 1 -- 172.21.15.116:0/4086721705 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f04fc042030 con 0x7f0510101420 2024-09-12T03:01:53.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.402+0000 7f050dffb640 1 --2- 172.21.15.116:0/4086721705 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f05000a4e20 0x7f05000a72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:53.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.402+0000 7f0514a78640 1 --2- 172.21.15.116:0/4086721705 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f05000a4e20 0x7f05000a72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:53.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.402+0000 7f050dffb640 1 -- 172.21.15.116:0/4086721705 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f04fc0ef670 con 0x7f0510101420 2024-09-12T03:01:53.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.403+0000 7f0514a78640 1 --2- 172.21.15.116:0/4086721705 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f05000a4e20 0x7f05000a72e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0504005fd0 tx=0x7f0504005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:53.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.406+0000 7f050dffb640 1 -- 172.21.15.116:0/4086721705 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f04fc0bc620 con 0x7f0510101420 2024-09-12T03:01:53.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.576+0000 7f0516cdc640 1 -- 172.21.15.116:0/4086721705 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f0510109040 con 0x7f0510101420 2024-09-12T03:01:53.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.576+0000 7f050dffb640 1 -- 172.21.15.116:0/4086721705 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f04fc0bbf20 con 0x7f0510101420 2024-09-12T03:01:53.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.578+0000 7f0516cdc640 1 -- 172.21.15.116:0/4086721705 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f05000a4e20 msgr2=0x7f05000a72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:53.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.578+0000 7f0516cdc640 1 --2- 172.21.15.116:0/4086721705 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f05000a4e20 0x7f05000a72e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0504005fd0 tx=0x7f0504005e20 comp rx=0 tx=0).stop 2024-09-12T03:01:53.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.578+0000 7f0516cdc640 1 -- 172.21.15.116:0/4086721705 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0510101420 msgr2=0x7f05101967f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:53.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.578+0000 7f0516cdc640 1 --2- 172.21.15.116:0/4086721705 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0510101420 0x7f05101967f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009a50 tx=0x7f04fc004270 comp rx=0 tx=0).stop 2024-09-12T03:01:53.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.579+0000 7f0516cdc640 1 -- 172.21.15.116:0/4086721705 shutdown_connections 2024-09-12T03:01:53.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.579+0000 7f0516cdc640 1 --2- 172.21.15.116:0/4086721705 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f05000a4e20 0x7f05000a72e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:53.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.579+0000 7f0516cdc640 1 --2- 172.21.15.116:0/4086721705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0510106a80 0x7f051019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:53.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.579+0000 7f0516cdc640 1 --2- 172.21.15.116:0/4086721705 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0510101420 0x7f05101967f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:53.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.579+0000 7f0516cdc640 1 --2- 172.21.15.116:0/4086721705 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0510100a80 0x7f05101962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:53.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.579+0000 7f0516cdc640 1 -- 172.21.15.116:0/4086721705 >> 172.21.15.116:0/4086721705 conn(0x7f05100fc7a0 msgr2=0x7f05100fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:53.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.579+0000 7f0516cdc640 1 -- 172.21.15.116:0/4086721705 shutdown_connections 2024-09-12T03:01:53.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:51.579+0000 7f0516cdc640 1 -- 172.21.15.116:0/4086721705 wait complete. 2024-09-12T03:01:53.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:53 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1096169489' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T03:01:53.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:53 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1096169489' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T03:01:53.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:53 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1096169489' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T03:01:53.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.803+0000 7f2ba4e5c640 1 -- 172.21.15.116:0/4002630411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ba0100e20 msgr2=0x7f2ba0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:53.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.803+0000 7f2ba4e5c640 1 --2- 172.21.15.116:0/4002630411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ba0100e20 0x7f2ba0101200 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2b90009ca0 tx=0x7f2b9002a950 comp rx=0 tx=0).stop 2024-09-12T03:01:53.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.803+0000 7f2ba4e5c640 1 -- 172.21.15.116:0/4002630411 shutdown_connections 2024-09-12T03:01:53.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.803+0000 7f2ba4e5c640 1 --2- 172.21.15.116:0/4002630411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ba0102ef0 0x7f2ba010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:53.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.803+0000 7f2ba4e5c640 1 --2- 172.21.15.116:0/4002630411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ba0102570 0x7f2ba01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:53.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.803+0000 7f2ba4e5c640 1 --2- 172.21.15.116:0/4002630411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ba0100e20 0x7f2ba0101200 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:53.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.803+0000 7f2ba4e5c640 1 -- 172.21.15.116:0/4002630411 >> 172.21.15.116:0/4002630411 conn(0x7f2ba00fc7d0 msgr2=0x7f2ba00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:53.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.803+0000 7f2ba4e5c640 1 -- 172.21.15.116:0/4002630411 shutdown_connections 2024-09-12T03:01:53.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.803+0000 7f2ba4e5c640 1 -- 172.21.15.116:0/4002630411 wait complete. 2024-09-12T03:01:53.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.804+0000 7f2ba4e5c640 1 Processor -- start 2024-09-12T03:01:53.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.804+0000 7f2ba4e5c640 1 -- start start 2024-09-12T03:01:53.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.804+0000 7f2ba4e5c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ba0100e20 0x7f2ba01985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:53.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.804+0000 7f2ba4e5c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ba0102570 0x7f2ba0198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:53.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.804+0000 7f2ba4e5c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ba0102ef0 0x7f2ba019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:53.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.804+0000 7f2ba4e5c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba0110fa0 con 0x7f2ba0102570 2024-09-12T03:01:53.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.804+0000 7f2ba4e5c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba0110e20 con 0x7f2ba0102ef0 2024-09-12T03:01:53.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.804+0000 7f2ba4e5c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba0111120 con 0x7f2ba0100e20 2024-09-12T03:01:53.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.805+0000 7f2b9dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ba0102570 0x7f2ba0198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:53.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.805+0000 7f2b9dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ba0102570 0x7f2ba0198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34210/0 (socket says 172.21.15.116:34210) 2024-09-12T03:01:53.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.805+0000 7f2b9dd9b640 1 -- 172.21.15.116:0/2810781178 learned_addr learned my addr 172.21.15.116:0/2810781178 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:53.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.805+0000 7f2b9e59c640 1 --2- 172.21.15.116:0/2810781178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ba0100e20 0x7f2ba01985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:53.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.805+0000 7f2b9ed9d640 1 --2- 172.21.15.116:0/2810781178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ba0102ef0 0x7f2ba019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:53.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.805+0000 7f2b9dd9b640 1 -- 172.21.15.116:0/2810781178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ba0100e20 msgr2=0x7f2ba01985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:53.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.805+0000 7f2b9dd9b640 1 --2- 172.21.15.116:0/2810781178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ba0100e20 0x7f2ba01985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:53.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.805+0000 7f2b9dd9b640 1 -- 172.21.15.116:0/2810781178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ba0102ef0 msgr2=0x7f2ba019cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:53.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.805+0000 7f2b9dd9b640 1 --2- 172.21.15.116:0/2810781178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ba0102ef0 0x7f2ba019cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:53.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.805+0000 7f2b9dd9b640 1 -- 172.21.15.116:0/2810781178 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba019d5a0 con 0x7f2ba0102570 2024-09-12T03:01:53.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.805+0000 7f2b9ed9d640 1 --2- 172.21.15.116:0/2810781178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ba0102ef0 0x7f2ba019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:01:53.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.805+0000 7f2b9dd9b640 1 --2- 172.21.15.116:0/2810781178 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ba0102570 0x7f2ba0198b30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400c890 tx=0x7f2b9400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:53.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.805+0000 7f2b8f7fe640 1 -- 172.21.15.116:0/2810781178 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b94004220 con 0x7f2ba0102570 2024-09-12T03:01:53.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.805+0000 7f2b8f7fe640 1 -- 172.21.15.116:0/2810781178 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b940043c0 con 0x7f2ba0102570 2024-09-12T03:01:53.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.805+0000 7f2ba4e5c640 1 -- 172.21.15.116:0/2810781178 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ba019d890 con 0x7f2ba0102570 2024-09-12T03:01:53.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.805+0000 7f2b8f7fe640 1 -- 172.21.15.116:0/2810781178 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b940049e0 con 0x7f2ba0102570 2024-09-12T03:01:53.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.805+0000 7f2ba4e5c640 1 -- 172.21.15.116:0/2810781178 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ba01a5170 con 0x7f2ba0102570 2024-09-12T03:01:53.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.806+0000 7f2ba4e5c640 1 -- 172.21.15.116:0/2810781178 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ba0101200 con 0x7f2ba0102570 2024-09-12T03:01:53.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.807+0000 7f2b8f7fe640 1 -- 172.21.15.116:0/2810781178 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2b940079b0 con 0x7f2ba0102570 2024-09-12T03:01:53.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.808+0000 7f2b8f7fe640 1 --2- 172.21.15.116:0/2810781178 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2b880a2c10 0x7f2b880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:53.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.808+0000 7f2b9e59c640 1 --2- 172.21.15.116:0/2810781178 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2b880a2c10 0x7f2b880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:53.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.808+0000 7f2b8f7fe640 1 -- 172.21.15.116:0/2810781178 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f2b940cf340 con 0x7f2ba0102570 2024-09-12T03:01:53.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.809+0000 7f2b9e59c640 1 --2- 172.21.15.116:0/2810781178 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2b880a2c10 0x7f2b880a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2b900093d0 tx=0x7f2b90009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:53.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.811+0000 7f2b8f7fe640 1 -- 172.21.15.116:0/2810781178 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2b94014030 con 0x7f2ba0102570 2024-09-12T03:01:53.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.986+0000 7f12a64a9640 1 -- 172.21.15.116:0/996780622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12a0101de0 msgr2=0x7f12a01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:53.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.986+0000 7f12a64a9640 1 --2- 172.21.15.116:0/996780622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12a0101de0 0x7f12a01021c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1290009900 tx=0x7f129002a410 comp rx=0 tx=0).stop 2024-09-12T03:01:53.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.987+0000 7f12a64a9640 1 -- 172.21.15.116:0/996780622 shutdown_connections 2024-09-12T03:01:53.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.987+0000 7f12a64a9640 1 --2- 172.21.15.116:0/996780622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12a0106f10 0x7f12a010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:53.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.987+0000 7f12a64a9640 1 --2- 172.21.15.116:0/996780622 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12a0102790 0x7f12a01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:53.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.987+0000 7f12a64a9640 1 --2- 172.21.15.116:0/996780622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12a0101de0 0x7f12a01021c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:53.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.987+0000 7f12a64a9640 1 -- 172.21.15.116:0/996780622 >> 172.21.15.116:0/996780622 conn(0x7f12a00fd8c0 msgr2=0x7f12a00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:53.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.987+0000 7f12a64a9640 1 -- 172.21.15.116:0/996780622 shutdown_connections 2024-09-12T03:01:53.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.987+0000 7f12a64a9640 1 -- 172.21.15.116:0/996780622 wait complete. 2024-09-12T03:01:53.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.987+0000 7f12a64a9640 1 Processor -- start 2024-09-12T03:01:53.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.987+0000 7f12a64a9640 1 -- start start 2024-09-12T03:01:53.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.987+0000 7f12a64a9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12a0101de0 0x7f12a00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:53.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.987+0000 7f12a64a9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12a0102790 0x7f12a0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:53.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.987+0000 7f12a64a9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12a0106f10 0x7f12a0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:53.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.987+0000 7f12a64a9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f12a0111f90 con 0x7f12a0106f10 2024-09-12T03:01:53.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.987+0000 7f12a64a9640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f12a0111e10 con 0x7f12a0101de0 2024-09-12T03:01:53.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.987+0000 7f12a64a9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f12a0112110 con 0x7f12a0102790 2024-09-12T03:01:53.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.988+0000 7f12a4a46640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12a0106f10 0x7f12a0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:53.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.988+0000 7f12a4a46640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12a0106f10 0x7f12a0074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34190/0 (socket says 172.21.15.116:34190) 2024-09-12T03:01:53.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.988+0000 7f12a4a46640 1 -- 172.21.15.116:0/1096169489 learned_addr learned my addr 172.21.15.116:0/1096169489 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:53.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.988+0000 7f1297fff640 1 --2- 172.21.15.116:0/1096169489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12a0102790 0x7f12a0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:53.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.988+0000 7f129ffff640 1 --2- 172.21.15.116:0/1096169489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12a0101de0 0x7f12a00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:53.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.988+0000 7f12a4a46640 1 -- 172.21.15.116:0/1096169489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12a0102790 msgr2=0x7f12a0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:53.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.988+0000 7f12a4a46640 1 --2- 172.21.15.116:0/1096169489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12a0102790 0x7f12a0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:53.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.988+0000 7f12a4a46640 1 -- 172.21.15.116:0/1096169489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12a0101de0 msgr2=0x7f12a00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:53.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.988+0000 7f12a4a46640 1 --2- 172.21.15.116:0/1096169489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12a0101de0 0x7f12a00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:53.948 INFO:teuthology.orchestra.run.smithi116.stdout:171798691889 2024-09-12T03:01:53.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.988+0000 7f12a4a46640 1 -- 172.21.15.116:0/1096169489 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a01a6890 con 0x7f12a0106f10 2024-09-12T03:01:53.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.988+0000 7f1297fff640 1 --2- 172.21.15.116:0/1096169489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12a0102790 0x7f12a0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:01:53.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.988+0000 7f129ffff640 1 --2- 172.21.15.116:0/1096169489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12a0101de0 0x7f12a00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:01:53.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.988+0000 7f12a4a46640 1 --2- 172.21.15.116:0/1096169489 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12a0106f10 0x7f12a0074f70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f129800b970 tx=0x7f129800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:53.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.988+0000 7f129dffb640 1 -- 172.21.15.116:0/1096169489 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1298019070 con 0x7f12a0106f10 2024-09-12T03:01:53.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.988+0000 7f129dffb640 1 -- 172.21.15.116:0/1096169489 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f129800cb40 con 0x7f12a0106f10 2024-09-12T03:01:53.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.988+0000 7f12a64a9640 1 -- 172.21.15.116:0/1096169489 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12a01a6b80 con 0x7f12a0106f10 2024-09-12T03:01:53.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.988+0000 7f129dffb640 1 -- 172.21.15.116:0/1096169489 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1298007650 con 0x7f12a0106f10 2024-09-12T03:01:53.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.988+0000 7f12a64a9640 1 -- 172.21.15.116:0/1096169489 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12a01a70c0 con 0x7f12a0106f10 2024-09-12T03:01:53.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.989+0000 7f12a64a9640 1 -- 172.21.15.116:0/1096169489 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12a01021c0 con 0x7f12a0106f10 2024-09-12T03:01:53.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.990+0000 7f129dffb640 1 -- 172.21.15.116:0/1096169489 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f129801c070 con 0x7f12a0106f10 2024-09-12T03:01:53.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.991+0000 7f129dffb640 1 --2- 172.21.15.116:0/1096169489 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f128c0a2c10 0x7f128c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:53.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.991+0000 7f129ffff640 1 --2- 172.21.15.116:0/1096169489 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f128c0a2c10 0x7f128c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:53.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.991+0000 7f129dffb640 1 -- 172.21.15.116:0/1096169489 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f12980ce550 con 0x7f12a0106f10 2024-09-12T03:01:53.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.991+0000 7f129ffff640 1 --2- 172.21.15.116:0/1096169489 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f128c0a2c10 0x7f128c0a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1290000c00 tx=0x7f1290032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:53.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:52.995+0000 7f129dffb640 1 -- 172.21.15.116:0/1096169489 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f129809b610 con 0x7f12a0106f10 2024-09-12T03:01:53.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.168+0000 7f12a64a9640 1 -- 172.21.15.116:0/1096169489 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f12a010a0f0 con 0x7f12a0106f10 2024-09-12T03:01:53.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.168+0000 7f129dffb640 1 -- 172.21.15.116:0/1096169489 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f129809af10 con 0x7f12a0106f10 2024-09-12T03:01:53.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.170+0000 7f12a64a9640 1 -- 172.21.15.116:0/1096169489 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f128c0a2c10 msgr2=0x7f128c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:53.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.170+0000 7f12a64a9640 1 --2- 172.21.15.116:0/1096169489 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f128c0a2c10 0x7f128c0a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1290000c00 tx=0x7f1290032d50 comp rx=0 tx=0).stop 2024-09-12T03:01:53.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.170+0000 7f12a64a9640 1 -- 172.21.15.116:0/1096169489 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12a0106f10 msgr2=0x7f12a0074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:53.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.170+0000 7f12a64a9640 1 --2- 172.21.15.116:0/1096169489 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12a0106f10 0x7f12a0074f70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f129800b970 tx=0x7f129800be30 comp rx=0 tx=0).stop 2024-09-12T03:01:53.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.170+0000 7f12a64a9640 1 -- 172.21.15.116:0/1096169489 shutdown_connections 2024-09-12T03:01:53.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.170+0000 7f12a64a9640 1 --2- 172.21.15.116:0/1096169489 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f128c0a2c10 0x7f128c0a50d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:53.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.170+0000 7f12a64a9640 1 --2- 172.21.15.116:0/1096169489 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12a0106f10 0x7f12a0074f70 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:53.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.170+0000 7f12a64a9640 1 --2- 172.21.15.116:0/1096169489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12a0102790 0x7f12a0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:53.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.170+0000 7f12a64a9640 1 --2- 172.21.15.116:0/1096169489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12a0101de0 0x7f12a00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:53.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.170+0000 7f12a64a9640 1 -- 172.21.15.116:0/1096169489 >> 172.21.15.116:0/1096169489 conn(0x7f12a00fd8c0 msgr2=0x7f12a00fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:53.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.170+0000 7f12a64a9640 1 -- 172.21.15.116:0/1096169489 shutdown_connections 2024-09-12T03:01:53.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.170+0000 7f12a64a9640 1 -- 172.21.15.116:0/1096169489 wait complete. 2024-09-12T03:01:53.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.980+0000 7f2ba4e5c640 1 -- 172.21.15.116:0/2810781178 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f2ba0199950 con 0x7f2ba0102570 2024-09-12T03:01:53.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.981+0000 7f2b8f7fe640 1 -- 172.21.15.116:0/2810781178 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2ba0199950 con 0x7f2ba0102570 2024-09-12T03:01:53.981 INFO:teuthology.orchestra.run.smithi116.stdout:94489280577 2024-09-12T03:01:53.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.983+0000 7f2ba4e5c640 1 -- 172.21.15.116:0/2810781178 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2b880a2c10 msgr2=0x7f2b880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:53.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.983+0000 7f2ba4e5c640 1 --2- 172.21.15.116:0/2810781178 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2b880a2c10 0x7f2b880a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2b900093d0 tx=0x7f2b90009360 comp rx=0 tx=0).stop 2024-09-12T03:01:53.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.983+0000 7f2ba4e5c640 1 -- 172.21.15.116:0/2810781178 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ba0102570 msgr2=0x7f2ba0198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:53.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.983+0000 7f2ba4e5c640 1 --2- 172.21.15.116:0/2810781178 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ba0102570 0x7f2ba0198b30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400c890 tx=0x7f2b9400cd50 comp rx=0 tx=0).stop 2024-09-12T03:01:53.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.983+0000 7f2ba4e5c640 1 -- 172.21.15.116:0/2810781178 shutdown_connections 2024-09-12T03:01:53.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.983+0000 7f2ba4e5c640 1 --2- 172.21.15.116:0/2810781178 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2b880a2c10 0x7f2b880a50d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:53.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.983+0000 7f2ba4e5c640 1 --2- 172.21.15.116:0/2810781178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ba0102ef0 0x7f2ba019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:53.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.983+0000 7f2ba4e5c640 1 --2- 172.21.15.116:0/2810781178 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ba0102570 0x7f2ba0198b30 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:53.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.983+0000 7f2ba4e5c640 1 --2- 172.21.15.116:0/2810781178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ba0100e20 0x7f2ba01985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:53.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.983+0000 7f2ba4e5c640 1 -- 172.21.15.116:0/2810781178 >> 172.21.15.116:0/2810781178 conn(0x7f2ba00fc7d0 msgr2=0x7f2ba006a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:53.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.983+0000 7f2ba4e5c640 1 -- 172.21.15.116:0/2810781178 shutdown_connections 2024-09-12T03:01:53.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:53.983+0000 7f2ba4e5c640 1 -- 172.21.15.116:0/2810781178 wait complete. 2024-09-12T03:01:54.497 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:54 smithi116 ceph-mon[31410]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:01:54.497 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:54 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2810781178' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T03:01:54.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.606+0000 7f8442e46640 1 -- 172.21.15.116:0/2757628966 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f843c101df0 msgr2=0x7f843c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:54.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.606+0000 7f8442e46640 1 --2- 172.21.15.116:0/2757628966 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f843c101df0 0x7f843c1021d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8430009900 tx=0x7f843002a410 comp rx=0 tx=0).stop 2024-09-12T03:01:54.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.606+0000 7f8442e46640 1 -- 172.21.15.116:0/2757628966 shutdown_connections 2024-09-12T03:01:54.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.606+0000 7f8442e46640 1 --2- 172.21.15.116:0/2757628966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f843c106f20 0x7f843c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:54.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.606+0000 7f8442e46640 1 --2- 172.21.15.116:0/2757628966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f843c1027a0 0x7f843c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:54.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.606+0000 7f8442e46640 1 --2- 172.21.15.116:0/2757628966 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f843c101df0 0x7f843c1021d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:54.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.606+0000 7f8442e46640 1 -- 172.21.15.116:0/2757628966 >> 172.21.15.116:0/2757628966 conn(0x7f843c0fd8f0 msgr2=0x7f843c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:54.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.606+0000 7f8442e46640 1 -- 172.21.15.116:0/2757628966 shutdown_connections 2024-09-12T03:01:54.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.606+0000 7f8442e46640 1 -- 172.21.15.116:0/2757628966 wait complete. 2024-09-12T03:01:54.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.607+0000 7f8442e46640 1 Processor -- start 2024-09-12T03:01:54.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.607+0000 7f8442e46640 1 -- start start 2024-09-12T03:01:54.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.607+0000 7f8442e46640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f843c101df0 0x7f843c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:54.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.607+0000 7f8442e46640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f843c1027a0 0x7f843c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:54.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.607+0000 7f8442e46640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f843c106f20 0x7f843c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:54.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.607+0000 7f8442e46640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f843c111f80 con 0x7f843c101df0 2024-09-12T03:01:54.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.607+0000 7f8440be2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f843c101df0 0x7f843c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:54.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.607+0000 7f8442e46640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f843c111e00 con 0x7f843c106f20 2024-09-12T03:01:54.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.607+0000 7f8442e46640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f843c112100 con 0x7f843c1027a0 2024-09-12T03:01:54.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.607+0000 7f843bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f843c1027a0 0x7f843c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:54.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.607+0000 7f843bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f843c1027a0 0x7f843c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.116:59598/0 (socket says 172.21.15.116:59598) 2024-09-12T03:01:54.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.607+0000 7f843bfff640 1 -- 172.21.15.116:0/79731312 learned_addr learned my addr 172.21.15.116:0/79731312 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:54.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.608+0000 7f84413e3640 1 --2- 172.21.15.116:0/79731312 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f843c106f20 0x7f843c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:54.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.608+0000 7f843bfff640 1 -- 172.21.15.116:0/79731312 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f843c106f20 msgr2=0x7f843c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:54.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.608+0000 7f843bfff640 1 --2- 172.21.15.116:0/79731312 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f843c106f20 0x7f843c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:54.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.608+0000 7f843bfff640 1 -- 172.21.15.116:0/79731312 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f843c101df0 msgr2=0x7f843c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:54.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.608+0000 7f843bfff640 1 --2- 172.21.15.116:0/79731312 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f843c101df0 0x7f843c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:54.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.608+0000 7f843bfff640 1 -- 172.21.15.116:0/79731312 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843c1a6960 con 0x7f843c1027a0 2024-09-12T03:01:54.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.608+0000 7f8440be2640 1 --2- 172.21.15.116:0/79731312 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f843c101df0 0x7f843c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:54.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.608+0000 7f843bfff640 1 --2- 172.21.15.116:0/79731312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f843c1027a0 0x7f843c074a30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f842800e9f0 tx=0x7f842800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:54.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.608+0000 7f8439ffb640 1 -- 172.21.15.116:0/79731312 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842800cd80 con 0x7f843c1027a0 2024-09-12T03:01:54.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.608+0000 7f8439ffb640 1 -- 172.21.15.116:0/79731312 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8428005500 con 0x7f843c1027a0 2024-09-12T03:01:54.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.608+0000 7f8442e46640 1 -- 172.21.15.116:0/79731312 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f843c1a6c50 con 0x7f843c1027a0 2024-09-12T03:01:54.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.609+0000 7f8439ffb640 1 -- 172.21.15.116:0/79731312 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8428010690 con 0x7f843c1027a0 2024-09-12T03:01:54.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.609+0000 7f8442e46640 1 -- 172.21.15.116:0/79731312 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f843c1a7190 con 0x7f843c1027a0 2024-09-12T03:01:54.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.610+0000 7f8442e46640 1 -- 172.21.15.116:0/79731312 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f843c1021d0 con 0x7f843c1027a0 2024-09-12T03:01:54.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.611+0000 7f8439ffb640 1 -- 172.21.15.116:0/79731312 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f8428004f60 con 0x7f843c1027a0 2024-09-12T03:01:54.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.612+0000 7f8439ffb640 1 --2- 172.21.15.116:0/79731312 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f842c0a2c90 0x7f842c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:54.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.612+0000 7f8439ffb640 1 -- 172.21.15.116:0/79731312 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f84280cf430 con 0x7f843c1027a0 2024-09-12T03:01:54.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.612+0000 7f8440be2640 1 --2- 172.21.15.116:0/79731312 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f842c0a2c90 0x7f842c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:54.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.613+0000 7f8440be2640 1 --2- 172.21.15.116:0/79731312 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f842c0a2c90 0x7f842c0a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8430009230 tx=0x7f84300058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:54.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.617+0000 7f8439ffb640 1 -- 172.21.15.116:0/79731312 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f842809c3e0 con 0x7f843c1027a0 2024-09-12T03:01:54.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:54 smithi183 ceph-mon[35535]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:01:54.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:54 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2810781178' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T03:01:54.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:54 smithi177 ceph-mon[35305]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:01:54.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:54 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2810781178' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T03:01:54.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.808+0000 7f8442e46640 1 -- 172.21.15.116:0/79731312 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f843c10a100 con 0x7f843c1027a0 2024-09-12T03:01:54.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.809+0000 7f8439ffb640 1 -- 172.21.15.116:0/79731312 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f842809bce0 con 0x7f843c1027a0 2024-09-12T03:01:54.809 INFO:teuthology.orchestra.run.smithi116.stdout:68719476806 2024-09-12T03:01:54.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.811+0000 7f8442e46640 1 -- 172.21.15.116:0/79731312 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f842c0a2c90 msgr2=0x7f842c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:54.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.811+0000 7f8442e46640 1 --2- 172.21.15.116:0/79731312 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f842c0a2c90 0x7f842c0a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8430009230 tx=0x7f84300058d0 comp rx=0 tx=0).stop 2024-09-12T03:01:54.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.811+0000 7f8442e46640 1 -- 172.21.15.116:0/79731312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f843c1027a0 msgr2=0x7f843c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:54.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.811+0000 7f8442e46640 1 --2- 172.21.15.116:0/79731312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f843c1027a0 0x7f843c074a30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f842800e9f0 tx=0x7f842800edf0 comp rx=0 tx=0).stop 2024-09-12T03:01:54.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.811+0000 7f8442e46640 1 -- 172.21.15.116:0/79731312 shutdown_connections 2024-09-12T03:01:54.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.811+0000 7f8442e46640 1 --2- 172.21.15.116:0/79731312 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f842c0a2c90 0x7f842c0a5150 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:54.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.811+0000 7f8442e46640 1 --2- 172.21.15.116:0/79731312 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f843c106f20 0x7f843c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:54.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.811+0000 7f8442e46640 1 --2- 172.21.15.116:0/79731312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f843c1027a0 0x7f843c074a30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:54.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.811+0000 7f8442e46640 1 --2- 172.21.15.116:0/79731312 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f843c101df0 0x7f843c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:54.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.811+0000 7f8442e46640 1 -- 172.21.15.116:0/79731312 >> 172.21.15.116:0/79731312 conn(0x7f843c0fd8f0 msgr2=0x7f843c0fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:54.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.811+0000 7f8442e46640 1 -- 172.21.15.116:0/79731312 shutdown_connections 2024-09-12T03:01:54.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.811+0000 7f8442e46640 1 -- 172.21.15.116:0/79731312 wait complete. 2024-09-12T03:01:55.174 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920819 got 141733920822 for osd.5 2024-09-12T03:01:55.174 DEBUG:teuthology.parallel:result is None 2024-09-12T03:01:55.305 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:55 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:01:55.305 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:55 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1384725868' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T03:01:55.305 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:55 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1879918626' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T03:01:55.305 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:55 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/79731312' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T03:01:55.465 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:55 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:01:55.465 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:55 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1384725868' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T03:01:55.465 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:55 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1879918626' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T03:01:55.465 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:55 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/79731312' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T03:01:55.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:55 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:01:55.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:55 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1384725868' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T03:01:55.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:55 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1879918626' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T03:01:55.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:55 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/79731312' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T03:01:55.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.356+0000 7f1cdf3b5640 1 -- 172.21.15.116:0/1240614216 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cd8102570 msgr2=0x7f1cd81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:55.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.356+0000 7f1cdf3b5640 1 --2- 172.21.15.116:0/1240614216 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cd8102570 0x7f1cd81029b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8009900 tx=0x7f1cc802a480 comp rx=0 tx=0).stop 2024-09-12T03:01:55.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.356+0000 7f1cdf3b5640 1 -- 172.21.15.116:0/1240614216 shutdown_connections 2024-09-12T03:01:55.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.356+0000 7f1cdf3b5640 1 --2- 172.21.15.116:0/1240614216 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cd8102ef0 0x7f1cd810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.356+0000 7f1cdf3b5640 1 --2- 172.21.15.116:0/1240614216 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cd8102570 0x7f1cd81029b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.356+0000 7f1cdf3b5640 1 --2- 172.21.15.116:0/1240614216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cd8100e20 0x7f1cd8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.356+0000 7f1cdf3b5640 1 -- 172.21.15.116:0/1240614216 >> 172.21.15.116:0/1240614216 conn(0x7f1cd80fc7d0 msgr2=0x7f1cd80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:55.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.356+0000 7f1cdf3b5640 1 -- 172.21.15.116:0/1240614216 shutdown_connections 2024-09-12T03:01:55.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.356+0000 7f1cdf3b5640 1 -- 172.21.15.116:0/1240614216 wait complete. 2024-09-12T03:01:55.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.357+0000 7f1cdf3b5640 1 Processor -- start 2024-09-12T03:01:55.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.357+0000 7f1cdf3b5640 1 -- start start 2024-09-12T03:01:55.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.357+0000 7f1cdf3b5640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cd8100e20 0x7f1cd8197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:55.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.357+0000 7f1cdf3b5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cd8102570 0x7f1cd8197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:55.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.357+0000 7f1cdf3b5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cd8102ef0 0x7f1cd8197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:55.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.357+0000 7f1cdf3b5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd8110fa0 con 0x7f1cd8102ef0 2024-09-12T03:01:55.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.357+0000 7f1cdf3b5640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd8110e20 con 0x7f1cd8100e20 2024-09-12T03:01:55.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.357+0000 7f1cdf3b5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd8111120 con 0x7f1cd8102570 2024-09-12T03:01:55.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.357+0000 7f1cdd952640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cd8102ef0 0x7f1cd8197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:55.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.357+0000 7f1cdd952640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cd8102ef0 0x7f1cd8197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34232/0 (socket says 172.21.15.116:34232) 2024-09-12T03:01:55.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.357+0000 7f1cdd952640 1 -- 172.21.15.116:0/1384725868 learned_addr learned my addr 172.21.15.116:0/1384725868 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:55.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.357+0000 7f1cdc950640 1 --2- 172.21.15.116:0/1384725868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cd8102570 0x7f1cd8197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:55.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.357+0000 7f1cdd151640 1 --2- 172.21.15.116:0/1384725868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cd8100e20 0x7f1cd8197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:55.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.358+0000 7f1cdd952640 1 -- 172.21.15.116:0/1384725868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cd8102570 msgr2=0x7f1cd8197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:55.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.358+0000 7f1cdd952640 1 --2- 172.21.15.116:0/1384725868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cd8102570 0x7f1cd8197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.358+0000 7f1cdd952640 1 -- 172.21.15.116:0/1384725868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cd8100e20 msgr2=0x7f1cd8197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:55.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.358+0000 7f1cdd952640 1 --2- 172.21.15.116:0/1384725868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cd8100e20 0x7f1cd8197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.358+0000 7f1cdd952640 1 -- 172.21.15.116:0/1384725868 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd81a6a90 con 0x7f1cd8102ef0 2024-09-12T03:01:55.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.358+0000 7f1cdd151640 1 --2- 172.21.15.116:0/1384725868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cd8100e20 0x7f1cd8197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:01:55.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.358+0000 7f1cdd952640 1 --2- 172.21.15.116:0/1384725868 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cd8102ef0 0x7f1cd8197ca0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1cd000eaa0 tx=0x7f1cd000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:55.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.358+0000 7f1cce7fc640 1 -- 172.21.15.116:0/1384725868 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd000cdb0 con 0x7f1cd8102ef0 2024-09-12T03:01:55.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.358+0000 7f1cce7fc640 1 -- 172.21.15.116:0/1384725868 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cd0005020 con 0x7f1cd8102ef0 2024-09-12T03:01:55.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.358+0000 7f1cdf3b5640 1 -- 172.21.15.116:0/1384725868 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd81a6d80 con 0x7f1cd8102ef0 2024-09-12T03:01:55.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.358+0000 7f1cce7fc640 1 -- 172.21.15.116:0/1384725868 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd00116b0 con 0x7f1cd8102ef0 2024-09-12T03:01:55.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.358+0000 7f1cdf3b5640 1 -- 172.21.15.116:0/1384725868 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd81a7240 con 0x7f1cd8102ef0 2024-09-12T03:01:55.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.359+0000 7f1cdf3b5640 1 -- 172.21.15.116:0/1384725868 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cd8101200 con 0x7f1cd8102ef0 2024-09-12T03:01:55.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.360+0000 7f1cce7fc640 1 -- 172.21.15.116:0/1384725868 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f1cd0005560 con 0x7f1cd8102ef0 2024-09-12T03:01:55.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.361+0000 7f1cce7fc640 1 --2- 172.21.15.116:0/1384725868 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1cc40a2c10 0x7f1cc40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:55.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.361+0000 7f1cce7fc640 1 -- 172.21.15.116:0/1384725868 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f1cd00cf290 con 0x7f1cd8102ef0 2024-09-12T03:01:55.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.361+0000 7f1cdd151640 1 --2- 172.21.15.116:0/1384725868 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1cc40a2c10 0x7f1cc40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:55.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.361+0000 7f1cdd151640 1 --2- 172.21.15.116:0/1384725868 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1cc40a2c10 0x7f1cc40a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4009930 tx=0x7f1cd4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:55.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.365+0000 7f1cce7fc640 1 -- 172.21.15.116:0/1384725868 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1cd0016030 con 0x7f1cd8102ef0 2024-09-12T03:01:55.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.577+0000 7f1cdf3b5640 1 -- 172.21.15.116:0/1384725868 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f1cd81090c0 con 0x7f1cd8102ef0 2024-09-12T03:01:55.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.578+0000 7f1cce7fc640 1 -- 172.21.15.116:0/1384725868 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1cd009c240 con 0x7f1cd8102ef0 2024-09-12T03:01:55.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.580+0000 7f1cdf3b5640 1 -- 172.21.15.116:0/1384725868 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1cc40a2c10 msgr2=0x7f1cc40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:55.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.580+0000 7f1cdf3b5640 1 --2- 172.21.15.116:0/1384725868 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1cc40a2c10 0x7f1cc40a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4009930 tx=0x7f1cd4008040 comp rx=0 tx=0).stop 2024-09-12T03:01:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.580+0000 7f1cdf3b5640 1 -- 172.21.15.116:0/1384725868 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cd8102ef0 msgr2=0x7f1cd8197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.580+0000 7f1cdf3b5640 1 --2- 172.21.15.116:0/1384725868 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cd8102ef0 0x7f1cd8197ca0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1cd000eaa0 tx=0x7f1cd000ef60 comp rx=0 tx=0).stop 2024-09-12T03:01:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.580+0000 7f1cdf3b5640 1 -- 172.21.15.116:0/1384725868 shutdown_connections 2024-09-12T03:01:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.580+0000 7f1cdf3b5640 1 --2- 172.21.15.116:0/1384725868 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1cc40a2c10 0x7f1cc40a50d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.580+0000 7f1cdf3b5640 1 --2- 172.21.15.116:0/1384725868 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cd8102ef0 0x7f1cd8197ca0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.580+0000 7f1cdf3b5640 1 --2- 172.21.15.116:0/1384725868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cd8102570 0x7f1cd8197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.580+0000 7f1cdf3b5640 1 --2- 172.21.15.116:0/1384725868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cd8100e20 0x7f1cd8197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.580+0000 7f1cdf3b5640 1 -- 172.21.15.116:0/1384725868 >> 172.21.15.116:0/1384725868 conn(0x7f1cd80fc7d0 msgr2=0x7f1cd80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.580+0000 7f1cdf3b5640 1 -- 172.21.15.116:0/1384725868 shutdown_connections 2024-09-12T03:01:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.580+0000 7f1cdf3b5640 1 -- 172.21.15.116:0/1384725868 wait complete. 2024-09-12T03:01:55.824 INFO:teuthology.orchestra.run.smithi116.stdout:34359738448 2024-09-12T03:01:56.521 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:56 smithi116 ceph-mon[31410]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-09-12T03:01:56.521 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:56 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:01:56.521 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:56 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:01:56.521 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:56 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:01:56.521 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:56 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:01:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:56 smithi183 ceph-mon[35535]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-09-12T03:01:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:56 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:01:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:56 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:01:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:56 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:01:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:56 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:01:56.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:56 smithi177 ceph-mon[35305]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-09-12T03:01:56.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:56 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:01:56.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:56 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:01:56.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:56 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:01:56.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:56 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:01:56.955 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691888 got 171798691889 for osd.6 2024-09-12T03:01:56.956 DEBUG:teuthology.parallel:result is None 2024-09-12T03:01:57.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.118+0000 7f509caf6640 1 -- 172.21.15.116:0/3091906670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5098102570 msgr2=0x7f50981029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:57.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.118+0000 7f509caf6640 1 --2- 172.21.15.116:0/3091906670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5098102570 0x7f50981029b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f508c009900 tx=0x7f508c02a480 comp rx=0 tx=0).stop 2024-09-12T03:01:57.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.118+0000 7f509caf6640 1 -- 172.21.15.116:0/3091906670 shutdown_connections 2024-09-12T03:01:57.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.118+0000 7f509caf6640 1 --2- 172.21.15.116:0/3091906670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5098102ef0 0x7f509810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.118+0000 7f509caf6640 1 --2- 172.21.15.116:0/3091906670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5098102570 0x7f50981029b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.118+0000 7f509caf6640 1 --2- 172.21.15.116:0/3091906670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5098100e20 0x7f5098101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.118+0000 7f509caf6640 1 -- 172.21.15.116:0/3091906670 >> 172.21.15.116:0/3091906670 conn(0x7f50980fc7d0 msgr2=0x7f50980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:57.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.118+0000 7f509caf6640 1 -- 172.21.15.116:0/3091906670 shutdown_connections 2024-09-12T03:01:57.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.119+0000 7f509caf6640 1 -- 172.21.15.116:0/3091906670 wait complete. 2024-09-12T03:01:57.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.119+0000 7f509caf6640 1 Processor -- start 2024-09-12T03:01:57.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.119+0000 7f509caf6640 1 -- start start 2024-09-12T03:01:57.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.119+0000 7f509caf6640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5098100e20 0x7f50981985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:57.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.119+0000 7f509caf6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5098102570 0x7f5098198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:57.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.119+0000 7f509caf6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5098102ef0 0x7f509819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:57.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.119+0000 7f509caf6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5098110fc0 con 0x7f5098102570 2024-09-12T03:01:57.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.119+0000 7f509caf6640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5098110e40 con 0x7f5098100e20 2024-09-12T03:01:57.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.119+0000 7f509caf6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5098111140 con 0x7f5098102ef0 2024-09-12T03:01:57.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.120+0000 7f5095d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5098102570 0x7f5098198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:57.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.120+0000 7f5095d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5098102570 0x7f5098198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34302/0 (socket says 172.21.15.116:34302) 2024-09-12T03:01:57.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.120+0000 7f5095d9b640 1 -- 172.21.15.116:0/2258551388 learned_addr learned my addr 172.21.15.116:0/2258551388 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:57.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.120+0000 7f509659c640 1 --2- 172.21.15.116:0/2258551388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5098100e20 0x7f50981985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:57.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.120+0000 7f5095d9b640 1 -- 172.21.15.116:0/2258551388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5098102ef0 msgr2=0x7f509819ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:57.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.120+0000 7f5095d9b640 1 --2- 172.21.15.116:0/2258551388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5098102ef0 0x7f509819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.120+0000 7f5095d9b640 1 -- 172.21.15.116:0/2258551388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5098100e20 msgr2=0x7f50981985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:57.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.120+0000 7f5095d9b640 1 --2- 172.21.15.116:0/2258551388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5098100e20 0x7f50981985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.120+0000 7f5095d9b640 1 -- 172.21.15.116:0/2258551388 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f509819d590 con 0x7f5098102570 2024-09-12T03:01:57.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.120+0000 7f509659c640 1 --2- 172.21.15.116:0/2258551388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5098100e20 0x7f50981985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:01:57.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.120+0000 7f5095d9b640 1 --2- 172.21.15.116:0/2258551388 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5098102570 0x7f5098198b20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f508c006010 tx=0x7f508c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:57.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.121+0000 7f50877fe640 1 -- 172.21.15.116:0/2258551388 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c03f070 con 0x7f5098102570 2024-09-12T03:01:57.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.121+0000 7f509caf6640 1 -- 172.21.15.116:0/2258551388 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f509819d820 con 0x7f5098102570 2024-09-12T03:01:57.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.121+0000 7f50877fe640 1 -- 172.21.15.116:0/2258551388 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f508c031c30 con 0x7f5098102570 2024-09-12T03:01:57.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.121+0000 7f50877fe640 1 -- 172.21.15.116:0/2258551388 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c035030 con 0x7f5098102570 2024-09-12T03:01:57.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.121+0000 7f509caf6640 1 -- 172.21.15.116:0/2258551388 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50981a5160 con 0x7f5098102570 2024-09-12T03:01:57.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.122+0000 7f509caf6640 1 -- 172.21.15.116:0/2258551388 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5098101200 con 0x7f5098102570 2024-09-12T03:01:57.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.124+0000 7f50877fe640 1 -- 172.21.15.116:0/2258551388 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f508c042070 con 0x7f5098102570 2024-09-12T03:01:57.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.124+0000 7f50877fe640 1 --2- 172.21.15.116:0/2258551388 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f50800a2c10 0x7f50800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:57.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.124+0000 7f509659c640 1 --2- 172.21.15.116:0/2258551388 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f50800a2c10 0x7f50800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:57.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.125+0000 7f50877fe640 1 -- 172.21.15.116:0/2258551388 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f508c0efd50 con 0x7f5098102570 2024-09-12T03:01:57.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.125+0000 7f509659c640 1 --2- 172.21.15.116:0/2258551388 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f50800a2c10 0x7f50800a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5088009930 tx=0x7f5088008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:57.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.127+0000 7f50877fe640 1 -- 172.21.15.116:0/2258551388 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f508c031de0 con 0x7f5098102570 2024-09-12T03:01:57.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.303+0000 7f509caf6640 1 -- 172.21.15.116:0/2258551388 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f509810a0a0 con 0x7f5098102570 2024-09-12T03:01:57.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.304+0000 7f50877fe640 1 -- 172.21.15.116:0/2258551388 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f508c0bcd00 con 0x7f5098102570 2024-09-12T03:01:57.304 INFO:teuthology.orchestra.run.smithi116.stdout:223338299433 2024-09-12T03:01:57.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.306+0000 7f509caf6640 1 -- 172.21.15.116:0/2258551388 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f50800a2c10 msgr2=0x7f50800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:57.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.306+0000 7f509caf6640 1 --2- 172.21.15.116:0/2258551388 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f50800a2c10 0x7f50800a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5088009930 tx=0x7f5088008040 comp rx=0 tx=0).stop 2024-09-12T03:01:57.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.306+0000 7f509caf6640 1 -- 172.21.15.116:0/2258551388 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5098102570 msgr2=0x7f5098198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:57.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.306+0000 7f509caf6640 1 --2- 172.21.15.116:0/2258551388 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5098102570 0x7f5098198b20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f508c006010 tx=0x7f508c003c10 comp rx=0 tx=0).stop 2024-09-12T03:01:57.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.306+0000 7f509caf6640 1 -- 172.21.15.116:0/2258551388 shutdown_connections 2024-09-12T03:01:57.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.306+0000 7f509caf6640 1 --2- 172.21.15.116:0/2258551388 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f50800a2c10 0x7f50800a50d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.306+0000 7f509caf6640 1 --2- 172.21.15.116:0/2258551388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5098102ef0 0x7f509819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.306+0000 7f509caf6640 1 --2- 172.21.15.116:0/2258551388 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5098102570 0x7f5098198b20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.306+0000 7f509caf6640 1 --2- 172.21.15.116:0/2258551388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5098100e20 0x7f50981985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.306+0000 7f509caf6640 1 -- 172.21.15.116:0/2258551388 >> 172.21.15.116:0/2258551388 conn(0x7f50980fc7d0 msgr2=0x7f509806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:57.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.306+0000 7f509caf6640 1 -- 172.21.15.116:0/2258551388 shutdown_connections 2024-09-12T03:01:57.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:57.306+0000 7f509caf6640 1 -- 172.21.15.116:0/2258551388 wait complete. 2024-09-12T03:01:57.539 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:57 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1899483713' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T03:01:57.676 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738445 got 34359738448 for osd.0 2024-09-12T03:01:57.676 DEBUG:teuthology.parallel:result is None 2024-09-12T03:01:57.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:57 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1899483713' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T03:01:57.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:57 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1899483713' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T03:01:57.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.556+0000 7fe756d32640 1 -- 172.21.15.116:0/2137611923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7501009d0 msgr2=0x7fe750100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:57.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.556+0000 7fe756d32640 1 --2- 172.21.15.116:0/2137611923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7501009d0 0x7fe750100db0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe744009900 tx=0x7fe74402a410 comp rx=0 tx=0).stop 2024-09-12T03:01:57.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.556+0000 7fe756d32640 1 -- 172.21.15.116:0/2137611923 shutdown_connections 2024-09-12T03:01:57.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.556+0000 7fe756d32640 1 --2- 172.21.15.116:0/2137611923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe75010e4f0 0x7fe750110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.556+0000 7fe756d32640 1 --2- 172.21.15.116:0/2137611923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe750101380 0x7fe75010dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.556+0000 7fe756d32640 1 --2- 172.21.15.116:0/2137611923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7501009d0 0x7fe750100db0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.556+0000 7fe756d32640 1 -- 172.21.15.116:0/2137611923 >> 172.21.15.116:0/2137611923 conn(0x7fe7500fc7d0 msgr2=0x7fe7500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:57.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.556+0000 7fe756d32640 1 -- 172.21.15.116:0/2137611923 shutdown_connections 2024-09-12T03:01:57.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.556+0000 7fe756d32640 1 -- 172.21.15.116:0/2137611923 wait complete. 2024-09-12T03:01:57.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.557+0000 7fe756d32640 1 Processor -- start 2024-09-12T03:01:57.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.557+0000 7fe756d32640 1 -- start start 2024-09-12T03:01:57.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.557+0000 7fe756d32640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7501009d0 0x7fe7501a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:57.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.557+0000 7fe756d32640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe750101380 0x7fe7501a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:57.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.557+0000 7fe756d32640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe75010e4f0 0x7fe75019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:57.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.557+0000 7fe756d32640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe750114330 con 0x7fe750101380 2024-09-12T03:01:57.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.557+0000 7fe756d32640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe7501141b0 con 0x7fe7501009d0 2024-09-12T03:01:57.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.557+0000 7fe756d32640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe7501144b0 con 0x7fe75010e4f0 2024-09-12T03:01:57.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.557+0000 7fe74ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe750101380 0x7fe7501a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:57.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.557+0000 7fe74ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe750101380 0x7fe7501a0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34278/0 (socket says 172.21.15.116:34278) 2024-09-12T03:01:57.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.557+0000 7fe74ffff640 1 -- 172.21.15.116:0/1899483713 learned_addr learned my addr 172.21.15.116:0/1899483713 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:57.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.557+0000 7fe7552cf640 1 --2- 172.21.15.116:0/1899483713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe75010e4f0 0x7fe75019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:57.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.557+0000 7fe754ace640 1 --2- 172.21.15.116:0/1899483713 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7501009d0 0x7fe7501a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:57.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.557+0000 7fe74ffff640 1 -- 172.21.15.116:0/1899483713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe75010e4f0 msgr2=0x7fe75019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:57.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.557+0000 7fe74ffff640 1 --2- 172.21.15.116:0/1899483713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe75010e4f0 0x7fe75019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.557+0000 7fe74ffff640 1 -- 172.21.15.116:0/1899483713 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7501009d0 msgr2=0x7fe7501a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:57.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.557+0000 7fe74ffff640 1 --2- 172.21.15.116:0/1899483713 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7501009d0 0x7fe7501a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.557+0000 7fe74ffff640 1 -- 172.21.15.116:0/1899483713 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe75019aeb0 con 0x7fe750101380 2024-09-12T03:01:57.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.558+0000 7fe754ace640 1 --2- 172.21.15.116:0/1899483713 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7501009d0 0x7fe7501a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:01:57.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.558+0000 7fe74ffff640 1 --2- 172.21.15.116:0/1899483713 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe750101380 0x7fe7501a0ab0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe73c00b4a0 tx=0x7fe73c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:57.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.558+0000 7fe74dffb640 1 -- 172.21.15.116:0/1899483713 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73c014070 con 0x7fe750101380 2024-09-12T03:01:57.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.558+0000 7fe756d32640 1 -- 172.21.15.116:0/1899483713 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe75019b1a0 con 0x7fe750101380 2024-09-12T03:01:57.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.558+0000 7fe74dffb640 1 -- 172.21.15.116:0/1899483713 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe73c010af0 con 0x7fe750101380 2024-09-12T03:01:57.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.558+0000 7fe74dffb640 1 -- 172.21.15.116:0/1899483713 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73c017d10 con 0x7fe750101380 2024-09-12T03:01:57.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.558+0000 7fe756d32640 1 -- 172.21.15.116:0/1899483713 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7501a74f0 con 0x7fe750101380 2024-09-12T03:01:57.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.559+0000 7fe756d32640 1 -- 172.21.15.116:0/1899483713 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe75006a7d0 con 0x7fe750101380 2024-09-12T03:01:57.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.560+0000 7fe74dffb640 1 -- 172.21.15.116:0/1899483713 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe73c017070 con 0x7fe750101380 2024-09-12T03:01:57.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.561+0000 7fe74dffb640 1 --2- 172.21.15.116:0/1899483713 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe7400a2c10 0x7fe7400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:57.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.561+0000 7fe754ace640 1 --2- 172.21.15.116:0/1899483713 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe7400a2c10 0x7fe7400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:57.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.561+0000 7fe74dffb640 1 -- 172.21.15.116:0/1899483713 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fe73c0ce2e0 con 0x7fe750101380 2024-09-12T03:01:57.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.561+0000 7fe754ace640 1 --2- 172.21.15.116:0/1899483713 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe7400a2c10 0x7fe7400a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe744000c00 tx=0x7fe744005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:57.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.565+0000 7fe74dffb640 1 -- 172.21.15.116:0/1899483713 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe73c09b3a0 con 0x7fe750101380 2024-09-12T03:01:57.820 INFO:teuthology.orchestra.run.smithi116.stdout:124554051645 2024-09-12T03:01:57.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.743+0000 7fe756d32640 1 -- 172.21.15.116:0/1899483713 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fe75019bb70 con 0x7fe750101380 2024-09-12T03:01:57.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.744+0000 7fe74dffb640 1 -- 172.21.15.116:0/1899483713 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe73c09aca0 con 0x7fe750101380 2024-09-12T03:01:57.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.746+0000 7fe756d32640 1 -- 172.21.15.116:0/1899483713 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe7400a2c10 msgr2=0x7fe7400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:57.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.746+0000 7fe756d32640 1 --2- 172.21.15.116:0/1899483713 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe7400a2c10 0x7fe7400a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe744000c00 tx=0x7fe744005a40 comp rx=0 tx=0).stop 2024-09-12T03:01:57.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.746+0000 7fe756d32640 1 -- 172.21.15.116:0/1899483713 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe750101380 msgr2=0x7fe7501a0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:57.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.746+0000 7fe756d32640 1 --2- 172.21.15.116:0/1899483713 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe750101380 0x7fe7501a0ab0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe73c00b4a0 tx=0x7fe73c00b960 comp rx=0 tx=0).stop 2024-09-12T03:01:57.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.746+0000 7fe756d32640 1 -- 172.21.15.116:0/1899483713 shutdown_connections 2024-09-12T03:01:57.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.746+0000 7fe756d32640 1 --2- 172.21.15.116:0/1899483713 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe7400a2c10 0x7fe7400a50d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.746+0000 7fe756d32640 1 --2- 172.21.15.116:0/1899483713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe75010e4f0 0x7fe75019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.746+0000 7fe756d32640 1 --2- 172.21.15.116:0/1899483713 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe750101380 0x7fe7501a0ab0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.746+0000 7fe756d32640 1 --2- 172.21.15.116:0/1899483713 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7501009d0 0x7fe7501a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.746+0000 7fe756d32640 1 -- 172.21.15.116:0/1899483713 >> 172.21.15.116:0/1899483713 conn(0x7fe7500fc7d0 msgr2=0x7fe7500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:57.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.746+0000 7fe756d32640 1 -- 172.21.15.116:0/1899483713 shutdown_connections 2024-09-12T03:01:57.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:56.746+0000 7fe756d32640 1 -- 172.21.15.116:0/1899483713 wait complete. 2024-09-12T03:01:58.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.466+0000 7f9618231640 1 -- 172.21.15.116:0/1764098046 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9610101df0 msgr2=0x7f96101021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:58.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.466+0000 7f9618231640 1 --2- 172.21.15.116:0/1764098046 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9610101df0 0x7f96101021d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f960c009900 tx=0x7f960c02a410 comp rx=0 tx=0).stop 2024-09-12T03:01:58.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.466+0000 7f9618231640 1 -- 172.21.15.116:0/1764098046 shutdown_connections 2024-09-12T03:01:58.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.466+0000 7f9618231640 1 --2- 172.21.15.116:0/1764098046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9610106f20 0x7f961010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:58.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.466+0000 7f9618231640 1 --2- 172.21.15.116:0/1764098046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96101027a0 0x7f96101067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:58.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.466+0000 7f9618231640 1 --2- 172.21.15.116:0/1764098046 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9610101df0 0x7f96101021d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:58.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.466+0000 7f9618231640 1 -- 172.21.15.116:0/1764098046 >> 172.21.15.116:0/1764098046 conn(0x7f96100fd8f0 msgr2=0x7f96100ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:58.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.466+0000 7f9618231640 1 -- 172.21.15.116:0/1764098046 shutdown_connections 2024-09-12T03:01:58.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.466+0000 7f9618231640 1 -- 172.21.15.116:0/1764098046 wait complete. 2024-09-12T03:01:58.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.467+0000 7f9618231640 1 Processor -- start 2024-09-12T03:01:58.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.467+0000 7f9618231640 1 -- start start 2024-09-12T03:01:58.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.467+0000 7f9618231640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9610101df0 0x7f9610198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:58.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.467+0000 7f9618231640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96101027a0 0x7f9610198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:58.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.467+0000 7f9618231640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9610106f20 0x7f961019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:58.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.467+0000 7f9618231640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9610111fc0 con 0x7f9610106f20 2024-09-12T03:01:58.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.467+0000 7f9618231640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9610111e40 con 0x7f96101027a0 2024-09-12T03:01:58.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.467+0000 7f9618231640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9610112140 con 0x7f9610101df0 2024-09-12T03:01:58.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.467+0000 7f96167ce640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9610106f20 0x7f961019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:58.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.467+0000 7f96167ce640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9610106f20 0x7f961019ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34252/0 (socket says 172.21.15.116:34252) 2024-09-12T03:01:58.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.467+0000 7f9615fcd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9610101df0 0x7f9610198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:58.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.467+0000 7f96167ce640 1 -- 172.21.15.116:0/1879918626 learned_addr learned my addr 172.21.15.116:0/1879918626 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:01:58.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.468+0000 7f96157cc640 1 --2- 172.21.15.116:0/1879918626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96101027a0 0x7f9610198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:58.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.468+0000 7f96167ce640 1 -- 172.21.15.116:0/1879918626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9610101df0 msgr2=0x7f9610198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:58.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.468+0000 7f96167ce640 1 --2- 172.21.15.116:0/1879918626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9610101df0 0x7f9610198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:58.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.468+0000 7f96167ce640 1 -- 172.21.15.116:0/1879918626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96101027a0 msgr2=0x7f9610198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:58.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.468+0000 7f96167ce640 1 --2- 172.21.15.116:0/1879918626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96101027a0 0x7f9610198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:58.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.468+0000 7f96167ce640 1 -- 172.21.15.116:0/1879918626 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f961019d4f0 con 0x7f9610106f20 2024-09-12T03:01:58.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.468+0000 7f96167ce640 1 --2- 172.21.15.116:0/1879918626 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9610106f20 0x7f961019ce10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f960800eaa0 tx=0x7f960800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:58.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.468+0000 7f9606ffd640 1 -- 172.21.15.116:0/1879918626 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960800cdb0 con 0x7f9610106f20 2024-09-12T03:01:58.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.468+0000 7f9618231640 1 -- 172.21.15.116:0/1879918626 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f961019d7e0 con 0x7f9610106f20 2024-09-12T03:01:58.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.468+0000 7f9606ffd640 1 -- 172.21.15.116:0/1879918626 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9608005020 con 0x7f9610106f20 2024-09-12T03:01:58.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.468+0000 7f9606ffd640 1 -- 172.21.15.116:0/1879918626 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96080116d0 con 0x7f9610106f20 2024-09-12T03:01:58.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.468+0000 7f9618231640 1 -- 172.21.15.116:0/1879918626 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96101a50c0 con 0x7f9610106f20 2024-09-12T03:01:58.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.469+0000 7f9618231640 1 -- 172.21.15.116:0/1879918626 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96101021d0 con 0x7f9610106f20 2024-09-12T03:01:58.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.470+0000 7f9606ffd640 1 -- 172.21.15.116:0/1879918626 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f96080054d0 con 0x7f9610106f20 2024-09-12T03:01:58.146 INFO:teuthology.orchestra.run.smithi116.stdout:193273528364 2024-09-12T03:01:58.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.471+0000 7f9606ffd640 1 --2- 172.21.15.116:0/1879918626 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f95fc0a2c10 0x7f95fc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:58.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.471+0000 7f9615fcd640 1 --2- 172.21.15.116:0/1879918626 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f95fc0a2c10 0x7f95fc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:58.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.471+0000 7f9606ffd640 1 -- 172.21.15.116:0/1879918626 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f96080cf5f0 con 0x7f9610106f20 2024-09-12T03:01:58.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.471+0000 7f9615fcd640 1 --2- 172.21.15.116:0/1879918626 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f95fc0a2c10 0x7f95fc0a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f960c009230 tx=0x7f960c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:58.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.474+0000 7f9606ffd640 1 -- 172.21.15.116:0/1879918626 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9608016030 con 0x7f9610106f20 2024-09-12T03:01:58.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.656+0000 7f9618231640 1 -- 172.21.15.116:0/1879918626 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f961010a100 con 0x7f9610106f20 2024-09-12T03:01:58.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.656+0000 7f9606ffd640 1 -- 172.21.15.116:0/1879918626 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f960809c5a0 con 0x7f9610106f20 2024-09-12T03:01:58.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.658+0000 7f9618231640 1 -- 172.21.15.116:0/1879918626 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f95fc0a2c10 msgr2=0x7f95fc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:58.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.658+0000 7f9618231640 1 --2- 172.21.15.116:0/1879918626 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f95fc0a2c10 0x7f95fc0a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f960c009230 tx=0x7f960c0058d0 comp rx=0 tx=0).stop 2024-09-12T03:01:58.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.658+0000 7f9618231640 1 -- 172.21.15.116:0/1879918626 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9610106f20 msgr2=0x7f961019ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:58.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.658+0000 7f9618231640 1 --2- 172.21.15.116:0/1879918626 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9610106f20 0x7f961019ce10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f960800eaa0 tx=0x7f960800ef60 comp rx=0 tx=0).stop 2024-09-12T03:01:58.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.658+0000 7f9618231640 1 -- 172.21.15.116:0/1879918626 shutdown_connections 2024-09-12T03:01:58.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.658+0000 7f9618231640 1 --2- 172.21.15.116:0/1879918626 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f95fc0a2c10 0x7f95fc0a50d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:58.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.658+0000 7f9618231640 1 --2- 172.21.15.116:0/1879918626 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9610106f20 0x7f961019ce10 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:58.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.658+0000 7f9618231640 1 --2- 172.21.15.116:0/1879918626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f96101027a0 0x7f9610198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:58.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.658+0000 7f9618231640 1 --2- 172.21.15.116:0/1879918626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9610101df0 0x7f9610198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:58.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.658+0000 7f9618231640 1 -- 172.21.15.116:0/1879918626 >> 172.21.15.116:0/1879918626 conn(0x7f96100fd8f0 msgr2=0x7f96100fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:58.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.658+0000 7f9618231640 1 -- 172.21.15.116:0/1879918626 shutdown_connections 2024-09-12T03:01:58.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:01:54.658+0000 7f9618231640 1 -- 172.21.15.116:0/1879918626 wait complete. 2024-09-12T03:01:58.471 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:58 smithi116 ceph-mon[31410]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-09-12T03:01:58.472 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:01:58 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2258551388' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T03:01:58.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:58 smithi183 ceph-mon[35535]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-09-12T03:01:58.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:01:58 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2258551388' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T03:01:58.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:58 smithi177 ceph-mon[35305]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-09-12T03:01:58.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:01:58 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2258551388' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T03:01:58.814 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528361 got 193273528364 for osd.7 2024-09-12T03:01:58.814 DEBUG:teuthology.parallel:result is None 2024-09-12T03:01:59.260 INFO:tasks.cephadm.ceph_manager.ceph:need seq 68719476804 got 68719476806 for osd.2 2024-09-12T03:01:59.260 DEBUG:teuthology.parallel:result is None 2024-09-12T03:02:00.030 INFO:tasks.cephadm.ceph_manager.ceph:need seq 124554051642 got 124554051645 for osd.4 2024-09-12T03:02:00.030 DEBUG:teuthology.parallel:result is None 2024-09-12T03:02:00.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:00 smithi116 ceph-mon[31410]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:02:00.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:00 smithi183 ceph-mon[35535]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:02:00.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:00 smithi177 ceph-mon[35305]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:02:00.805 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299430 got 223338299433 for osd.8 2024-09-12T03:02:00.805 DEBUG:teuthology.parallel:result is None 2024-09-12T03:02:00.904 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:00.904 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:00.938 INFO:tasks.cephadm.ceph_manager.ceph:need seq 94489280573 got 94489280577 for osd.3 2024-09-12T03:02:00.938 DEBUG:teuthology.parallel:result is None 2024-09-12T03:02:01.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.497+0000 7f6ece425640 1 -- 172.21.15.116:0/1104750471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ec8100e20 msgr2=0x7f6ec8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:01.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.497+0000 7f6ece425640 1 --2- 172.21.15.116:0/1104750471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ec8100e20 0x7f6ec8101200 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8009900 tx=0x7f6eb802a480 comp rx=0 tx=0).stop 2024-09-12T03:02:01.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.497+0000 7f6ece425640 1 -- 172.21.15.116:0/1104750471 shutdown_connections 2024-09-12T03:02:01.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.497+0000 7f6ece425640 1 --2- 172.21.15.116:0/1104750471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ec8102ef0 0x7f6ec810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:01.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.497+0000 7f6ece425640 1 --2- 172.21.15.116:0/1104750471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ec8102570 0x7f6ec81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:01.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.497+0000 7f6ece425640 1 --2- 172.21.15.116:0/1104750471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ec8100e20 0x7f6ec8101200 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:01.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.497+0000 7f6ece425640 1 -- 172.21.15.116:0/1104750471 >> 172.21.15.116:0/1104750471 conn(0x7f6ec80fc7d0 msgr2=0x7f6ec80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:01.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.497+0000 7f6ece425640 1 -- 172.21.15.116:0/1104750471 shutdown_connections 2024-09-12T03:02:01.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.497+0000 7f6ece425640 1 -- 172.21.15.116:0/1104750471 wait complete. 2024-09-12T03:02:01.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.497+0000 7f6ece425640 1 Processor -- start 2024-09-12T03:02:01.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.497+0000 7f6ece425640 1 -- start start 2024-09-12T03:02:01.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.498+0000 7f6ece425640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ec8100e20 0x7f6ec8198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:01.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.498+0000 7f6ece425640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ec8102570 0x7f6ec8198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:01.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.498+0000 7f6ece425640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ec8102ef0 0x7f6ec819cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:01.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.498+0000 7f6ece425640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec81120f0 con 0x7f6ec8102ef0 2024-09-12T03:02:01.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.498+0000 7f6ece425640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec8111f70 con 0x7f6ec8100e20 2024-09-12T03:02:01.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.498+0000 7f6ece425640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec8112270 con 0x7f6ec8102570 2024-09-12T03:02:01.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.498+0000 7f6ecc9c2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ec8102ef0 0x7f6ec819cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:01.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.498+0000 7f6ecc9c2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ec8102ef0 0x7f6ec819cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60104/0 (socket says 172.21.15.116:60104) 2024-09-12T03:02:01.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.498+0000 7f6ec7fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ec8100e20 0x7f6ec8198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:01.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.498+0000 7f6ecc9c2640 1 -- 172.21.15.116:0/2209796091 learned_addr learned my addr 172.21.15.116:0/2209796091 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:01.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.498+0000 7f6ec77fe640 1 --2- 172.21.15.116:0/2209796091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ec8102570 0x7f6ec8198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:01.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.498+0000 7f6ecc9c2640 1 -- 172.21.15.116:0/2209796091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ec8102570 msgr2=0x7f6ec8198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:01.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.498+0000 7f6ecc9c2640 1 --2- 172.21.15.116:0/2209796091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ec8102570 0x7f6ec8198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:01.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.498+0000 7f6ecc9c2640 1 -- 172.21.15.116:0/2209796091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ec8100e20 msgr2=0x7f6ec8198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:01.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.498+0000 7f6ecc9c2640 1 --2- 172.21.15.116:0/2209796091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ec8100e20 0x7f6ec8198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:01.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.498+0000 7f6ecc9c2640 1 -- 172.21.15.116:0/2209796091 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ec819d620 con 0x7f6ec8102ef0 2024-09-12T03:02:01.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.498+0000 7f6ecc9c2640 1 --2- 172.21.15.116:0/2209796091 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ec8102ef0 0x7f6ec819cf40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6ec000b970 tx=0x7f6ec000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:01.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.499+0000 7f6ec57fa640 1 -- 172.21.15.116:0/2209796091 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0019070 con 0x7f6ec8102ef0 2024-09-12T03:02:01.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.499+0000 7f6ec57fa640 1 -- 172.21.15.116:0/2209796091 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ec000cb40 con 0x7f6ec8102ef0 2024-09-12T03:02:01.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.499+0000 7f6ec57fa640 1 -- 172.21.15.116:0/2209796091 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0007650 con 0x7f6ec8102ef0 2024-09-12T03:02:01.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.499+0000 7f6ece425640 1 -- 172.21.15.116:0/2209796091 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ec819d910 con 0x7f6ec8102ef0 2024-09-12T03:02:01.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.499+0000 7f6ece425640 1 -- 172.21.15.116:0/2209796091 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ec81a51f0 con 0x7f6ec8102ef0 2024-09-12T03:02:01.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.500+0000 7f6ece425640 1 -- 172.21.15.116:0/2209796091 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ec8101200 con 0x7f6ec8102ef0 2024-09-12T03:02:01.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.501+0000 7f6ec57fa640 1 -- 172.21.15.116:0/2209796091 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f6ec001c070 con 0x7f6ec8102ef0 2024-09-12T03:02:01.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.501+0000 7f6ec57fa640 1 --2- 172.21.15.116:0/2209796091 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6eb00a2c10 0x7f6eb00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:01.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.501+0000 7f6ec7fff640 1 --2- 172.21.15.116:0/2209796091 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6eb00a2c10 0x7f6eb00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:01.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.502+0000 7f6ec57fa640 1 -- 172.21.15.116:0/2209796091 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f6ec00ce9b0 con 0x7f6ec8102ef0 2024-09-12T03:02:01.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.502+0000 7f6ec7fff640 1 --2- 172.21.15.116:0/2209796091 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6eb00a2c10 0x7f6eb00a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8000c00 tx=0x7f6eb8005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:01.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.505+0000 7f6ec57fa640 1 -- 172.21.15.116:0/2209796091 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6ec009b420 con 0x7f6ec8102ef0 2024-09-12T03:02:01.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.680+0000 7f6ece425640 1 -- 172.21.15.116:0/2209796091 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f6ec8109090 con 0x7f6ec8102ef0 2024-09-12T03:02:01.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.680+0000 7f6ec57fa640 1 -- 172.21.15.116:0/2209796091 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f6ec009f720 con 0x7f6ec8102ef0 2024-09-12T03:02:01.681 INFO:teuthology.orchestra.run.smithi116.stdout:274877906974 2024-09-12T03:02:01.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.682+0000 7f6ece425640 1 -- 172.21.15.116:0/2209796091 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6eb00a2c10 msgr2=0x7f6eb00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:01.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.682+0000 7f6ece425640 1 --2- 172.21.15.116:0/2209796091 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6eb00a2c10 0x7f6eb00a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8000c00 tx=0x7f6eb8005e50 comp rx=0 tx=0).stop 2024-09-12T03:02:01.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.682+0000 7f6ece425640 1 -- 172.21.15.116:0/2209796091 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ec8102ef0 msgr2=0x7f6ec819cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:01.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.682+0000 7f6ece425640 1 --2- 172.21.15.116:0/2209796091 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ec8102ef0 0x7f6ec819cf40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6ec000b970 tx=0x7f6ec000be30 comp rx=0 tx=0).stop 2024-09-12T03:02:01.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.682+0000 7f6ece425640 1 -- 172.21.15.116:0/2209796091 shutdown_connections 2024-09-12T03:02:01.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.682+0000 7f6ece425640 1 --2- 172.21.15.116:0/2209796091 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6eb00a2c10 0x7f6eb00a50d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:01.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.682+0000 7f6ece425640 1 --2- 172.21.15.116:0/2209796091 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ec8102ef0 0x7f6ec819cf40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:01.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.682+0000 7f6ece425640 1 --2- 172.21.15.116:0/2209796091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ec8102570 0x7f6ec8198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:01.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.682+0000 7f6ece425640 1 --2- 172.21.15.116:0/2209796091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ec8100e20 0x7f6ec8198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:01.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.682+0000 7f6ece425640 1 -- 172.21.15.116:0/2209796091 >> 172.21.15.116:0/2209796091 conn(0x7f6ec80fc7d0 msgr2=0x7f6ec80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:01.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.682+0000 7f6ece425640 1 -- 172.21.15.116:0/2209796091 shutdown_connections 2024-09-12T03:02:01.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.682+0000 7f6ece425640 1 -- 172.21.15.116:0/2209796091 wait complete. 2024-09-12T03:02:01.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.883+0000 7fb8147f2640 1 -- 172.21.15.116:0/3581801081 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb80c101420 msgr2=0x7fb80c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:01.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.883+0000 7fb8147f2640 1 --2- 172.21.15.116:0/3581801081 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb80c101420 0x7fb80c10d950 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009ca0 tx=0x7fb7fc02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:02:01.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.883+0000 7fb8147f2640 1 -- 172.21.15.116:0/3581801081 shutdown_connections 2024-09-12T03:02:01.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.883+0000 7fb8147f2640 1 --2- 172.21.15.116:0/3581801081 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb80c101420 0x7fb80c10d950 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:01.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.883+0000 7fb8147f2640 1 --2- 172.21.15.116:0/3581801081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb80c100a80 0x7fb80c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:01.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.883+0000 7fb8147f2640 1 --2- 172.21.15.116:0/3581801081 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb80c106a80 0x7fb80c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:01.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.883+0000 7fb8147f2640 1 -- 172.21.15.116:0/3581801081 >> 172.21.15.116:0/3581801081 conn(0x7fb80c0fc7a0 msgr2=0x7fb80c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:01.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.883+0000 7fb8147f2640 1 -- 172.21.15.116:0/3581801081 shutdown_connections 2024-09-12T03:02:01.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.883+0000 7fb8147f2640 1 -- 172.21.15.116:0/3581801081 wait complete. 2024-09-12T03:02:01.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.883+0000 7fb8147f2640 1 Processor -- start 2024-09-12T03:02:01.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.884+0000 7fb8147f2640 1 -- start start 2024-09-12T03:02:01.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.884+0000 7fb8147f2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb80c100a80 0x7fb80c10e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:01.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.884+0000 7fb8147f2640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb80c101420 0x7fb80c10e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:01.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.884+0000 7fb8147f2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb80c106a80 0x7fb80c10eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:01.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.884+0000 7fb8147f2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c111fc0 con 0x7fb80c100a80 2024-09-12T03:02:01.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.884+0000 7fb8147f2640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c111e40 con 0x7fb80c101420 2024-09-12T03:02:01.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.884+0000 7fb8147f2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c112140 con 0x7fb80c106a80 2024-09-12T03:02:01.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.884+0000 7fb81258e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb80c100a80 0x7fb80c10e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:01.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.884+0000 7fb81258e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb80c100a80 0x7fb80c10e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60120/0 (socket says 172.21.15.116:60120) 2024-09-12T03:02:01.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.884+0000 7fb81258e640 1 -- 172.21.15.116:0/1247059525 learned_addr learned my addr 172.21.15.116:0/1247059525 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:01.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.884+0000 7fb812d8f640 1 --2- 172.21.15.116:0/1247059525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb80c106a80 0x7fb80c10eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:01.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.884+0000 7fb81258e640 1 -- 172.21.15.116:0/1247059525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb80c106a80 msgr2=0x7fb80c10eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:01.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.884+0000 7fb81258e640 1 --2- 172.21.15.116:0/1247059525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb80c106a80 0x7fb80c10eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:01.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.884+0000 7fb81258e640 1 -- 172.21.15.116:0/1247059525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb80c101420 msgr2=0x7fb80c10e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:01.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.884+0000 7fb81258e640 1 --2- 172.21.15.116:0/1247059525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb80c101420 0x7fb80c10e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:01.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.884+0000 7fb81258e640 1 -- 172.21.15.116:0/1247059525 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb80c1a6920 con 0x7fb80c100a80 2024-09-12T03:02:01.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.884+0000 7fb812d8f640 1 --2- 172.21.15.116:0/1247059525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb80c106a80 0x7fb80c10eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:01.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.884+0000 7fb81258e640 1 --2- 172.21.15.116:0/1247059525 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb80c100a80 0x7fb80c10e0e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb80400b4a0 tx=0x7fb80400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:01.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.884+0000 7fb80b7fe640 1 -- 172.21.15.116:0/1247059525 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb804014070 con 0x7fb80c100a80 2024-09-12T03:02:01.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.884+0000 7fb80b7fe640 1 -- 172.21.15.116:0/1247059525 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb804010af0 con 0x7fb80c100a80 2024-09-12T03:02:01.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.884+0000 7fb8147f2640 1 -- 172.21.15.116:0/1247059525 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb80c1a6c10 con 0x7fb80c100a80 2024-09-12T03:02:01.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.884+0000 7fb80b7fe640 1 -- 172.21.15.116:0/1247059525 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb804017d10 con 0x7fb80c100a80 2024-09-12T03:02:01.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.885+0000 7fb8147f2640 1 -- 172.21.15.116:0/1247059525 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb80c1a7150 con 0x7fb80c100a80 2024-09-12T03:02:01.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.885+0000 7fb8147f2640 1 -- 172.21.15.116:0/1247059525 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb80c100ee0 con 0x7fb80c100a80 2024-09-12T03:02:01.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.887+0000 7fb80b7fe640 1 -- 172.21.15.116:0/1247059525 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fb804017070 con 0x7fb80c100a80 2024-09-12T03:02:01.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.887+0000 7fb80b7fe640 1 --2- 172.21.15.116:0/1247059525 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb7f40a2c10 0x7fb7f40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:01.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.887+0000 7fb811d8d640 1 --2- 172.21.15.116:0/1247059525 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb7f40a2c10 0x7fb7f40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:01.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.887+0000 7fb80b7fe640 1 -- 172.21.15.116:0/1247059525 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fb8040ce330 con 0x7fb80c100a80 2024-09-12T03:02:01.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.888+0000 7fb811d8d640 1 --2- 172.21.15.116:0/1247059525 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb7f40a2c10 0x7fb7f40a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb80c10fb70 tx=0x7fb800005bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:01.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:01.891+0000 7fb80b7fe640 1 -- 172.21.15.116:0/1247059525 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb80409b3f0 con 0x7fb80c100a80 2024-09-12T03:02:02.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:02.062+0000 7fb8147f2640 1 -- 172.21.15.116:0/1247059525 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fb80c10f610 con 0x7fb80c100a80 2024-09-12T03:02:02.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:02.063+0000 7fb80b7fe640 1 -- 172.21.15.116:0/1247059525 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fb80409acf0 con 0x7fb80c100a80 2024-09-12T03:02:02.064 INFO:teuthology.orchestra.run.smithi116.stdout:309237645338 2024-09-12T03:02:02.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:02.065+0000 7fb8147f2640 1 -- 172.21.15.116:0/1247059525 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb7f40a2c10 msgr2=0x7fb7f40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:02.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:02.065+0000 7fb8147f2640 1 --2- 172.21.15.116:0/1247059525 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb7f40a2c10 0x7fb7f40a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb80c10fb70 tx=0x7fb800005bf0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:02.065+0000 7fb8147f2640 1 -- 172.21.15.116:0/1247059525 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb80c100a80 msgr2=0x7fb80c10e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:02.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:02.065+0000 7fb8147f2640 1 --2- 172.21.15.116:0/1247059525 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb80c100a80 0x7fb80c10e0e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb80400b4a0 tx=0x7fb80400b960 comp rx=0 tx=0).stop 2024-09-12T03:02:02.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:02.065+0000 7fb8147f2640 1 -- 172.21.15.116:0/1247059525 shutdown_connections 2024-09-12T03:02:02.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:02.065+0000 7fb8147f2640 1 --2- 172.21.15.116:0/1247059525 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb7f40a2c10 0x7fb7f40a50d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:02.065+0000 7fb8147f2640 1 --2- 172.21.15.116:0/1247059525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb80c106a80 0x7fb80c10eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:02.065+0000 7fb8147f2640 1 --2- 172.21.15.116:0/1247059525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb80c101420 0x7fb80c10e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:02.065+0000 7fb8147f2640 1 --2- 172.21.15.116:0/1247059525 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb80c100a80 0x7fb80c10e0e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:02.065+0000 7fb8147f2640 1 -- 172.21.15.116:0/1247059525 >> 172.21.15.116:0/1247059525 conn(0x7fb80c0fc7a0 msgr2=0x7fb80c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:02.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:02.065+0000 7fb8147f2640 1 -- 172.21.15.116:0/1247059525 shutdown_connections 2024-09-12T03:02:02.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:02.065+0000 7fb8147f2640 1 -- 172.21.15.116:0/1247059525 wait complete. 2024-09-12T03:02:02.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:02 smithi116 ceph-mon[31410]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:02:02.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:02 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2209796091' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T03:02:02.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:02 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1247059525' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T03:02:02.585 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906971 got 274877906974 for osd.10 2024-09-12T03:02:02.585 DEBUG:teuthology.parallel:result is None 2024-09-12T03:02:02.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:02 smithi183 ceph-mon[35535]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:02:02.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:02 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2209796091' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T03:02:02.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:02 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1247059525' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T03:02:02.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:02 smithi177 ceph-mon[35305]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:02:02.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:02 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2209796091' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T03:02:02.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:02 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1247059525' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T03:02:03.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:03 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-12T03:02:03.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:03 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-12T03:02:03.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:03 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-09-12T03:02:03.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:03 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-09-12T03:02:03.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:03 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-09-12T03:02:03.404 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:03 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-09-12T03:02:03.404 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:03 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-12T03:02:03.430 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645334 got 309237645338 for osd.11 2024-09-12T03:02:03.430 DEBUG:teuthology.parallel:result is None 2024-09-12T03:02:03.430 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T03:02:03.430 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:02:03.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:03 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-12T03:02:03.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:03 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-12T03:02:03.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:03 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-09-12T03:02:03.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:03 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-09-12T03:02:03.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:03 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-09-12T03:02:03.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:03 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-09-12T03:02:03.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:03 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-12T03:02:03.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:03 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-12T03:02:03.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:03 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-12T03:02:03.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:03 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-09-12T03:02:03.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:03 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-09-12T03:02:03.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:03 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-09-12T03:02:03.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:03 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-09-12T03:02:03.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:03 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-12T03:02:04.250 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:04.529 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:04 smithi116 ceph-mon[31410]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:02:04.529 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:04 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-12T03:02:04.529 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:04 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-12T03:02:04.529 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:04 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-12T03:02:04.529 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:04 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-09-12T03:02:04.529 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:04 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-09-12T03:02:04.529 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:04 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-12T03:02:04.530 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:04 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-12T03:02:04.530 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:04 smithi116 ceph-mon[31410]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T03:02:04.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:04 smithi183 ceph-mon[35535]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:02:04.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:04 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-12T03:02:04.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:04 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-12T03:02:04.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:04 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-12T03:02:04.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:04 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-09-12T03:02:04.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:04 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-09-12T03:02:04.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:04 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-12T03:02:04.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:04 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-12T03:02:04.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:04 smithi183 ceph-mon[35535]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T03:02:04.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.739+0000 7f0d49efe640 1 -- 172.21.15.116:0/1523314800 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d44103660 msgr2=0x7f0d4410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:04.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.739+0000 7f0d49efe640 1 --2- 172.21.15.116:0/1523314800 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d44103660 0x7f0d4410bb50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009900 tx=0x7f0d3802a480 comp rx=0 tx=0).stop 2024-09-12T03:02:04.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.739+0000 7f0d49efe640 1 -- 172.21.15.116:0/1523314800 shutdown_connections 2024-09-12T03:02:04.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.739+0000 7f0d49efe640 1 --2- 172.21.15.116:0/1523314800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d4410c170 0x7f0d4410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.739+0000 7f0d49efe640 1 --2- 172.21.15.116:0/1523314800 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d44103660 0x7f0d4410bb50 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.740+0000 7f0d49efe640 1 --2- 172.21.15.116:0/1523314800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d44102d40 0x7f0d44103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.740+0000 7f0d49efe640 1 -- 172.21.15.116:0/1523314800 >> 172.21.15.116:0/1523314800 conn(0x7f0d440fc7a0 msgr2=0x7f0d440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:04.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.740+0000 7f0d49efe640 1 -- 172.21.15.116:0/1523314800 shutdown_connections 2024-09-12T03:02:04.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.740+0000 7f0d49efe640 1 -- 172.21.15.116:0/1523314800 wait complete. 2024-09-12T03:02:04.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.740+0000 7f0d49efe640 1 Processor -- start 2024-09-12T03:02:04.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.740+0000 7f0d49efe640 1 -- start start 2024-09-12T03:02:04.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.741+0000 7f0d49efe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d44102d40 0x7f0d441984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:04.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.741+0000 7f0d49efe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d44103660 0x7f0d441989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:04.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.741+0000 7f0d49efe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d4410c170 0x7f0d4419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:04.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.741+0000 7f0d49efe640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0d44111fb0 con 0x7f0d44103660 2024-09-12T03:02:04.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.741+0000 7f0d49efe640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0d44111e30 con 0x7f0d44102d40 2024-09-12T03:02:04.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.741+0000 7f0d49efe640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0d44112130 con 0x7f0d4410c170 2024-09-12T03:02:04.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.741+0000 7f0d42ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d44103660 0x7f0d441989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:04.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.741+0000 7f0d42ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d44103660 0x7f0d441989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60136/0 (socket says 172.21.15.116:60136) 2024-09-12T03:02:04.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.741+0000 7f0d42ffd640 1 -- 172.21.15.116:0/1604711368 learned_addr learned my addr 172.21.15.116:0/1604711368 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:04.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.741+0000 7f0d437fe640 1 --2- 172.21.15.116:0/1604711368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d44102d40 0x7f0d441984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:04.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.741+0000 7f0d42ffd640 1 -- 172.21.15.116:0/1604711368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d4410c170 msgr2=0x7f0d4419cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:04.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.741+0000 7f0d43fff640 1 --2- 172.21.15.116:0/1604711368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d4410c170 0x7f0d4419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:04.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.741+0000 7f0d42ffd640 1 --2- 172.21.15.116:0/1604711368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d4410c170 0x7f0d4419cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.741+0000 7f0d42ffd640 1 -- 172.21.15.116:0/1604711368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d44102d40 msgr2=0x7f0d441984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:04.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.741+0000 7f0d42ffd640 1 --2- 172.21.15.116:0/1604711368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d44102d40 0x7f0d441984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.741+0000 7f0d42ffd640 1 -- 172.21.15.116:0/1604711368 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4419d450 con 0x7f0d44103660 2024-09-12T03:02:04.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.741+0000 7f0d437fe640 1 --2- 172.21.15.116:0/1604711368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d44102d40 0x7f0d441984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:04.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.741+0000 7f0d43fff640 1 --2- 172.21.15.116:0/1604711368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d4410c170 0x7f0d4419cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:04.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.741+0000 7f0d42ffd640 1 --2- 172.21.15.116:0/1604711368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d44103660 0x7f0d441989e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0d38006010 tx=0x7f0d380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:04.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.741+0000 7f0d40ff9640 1 -- 172.21.15.116:0/1604711368 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3803f030 con 0x7f0d44103660 2024-09-12T03:02:04.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.741+0000 7f0d40ff9640 1 -- 172.21.15.116:0/1604711368 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d38003ce0 con 0x7f0d44103660 2024-09-12T03:02:04.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.741+0000 7f0d49efe640 1 -- 172.21.15.116:0/1604711368 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4419d6e0 con 0x7f0d44103660 2024-09-12T03:02:04.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.741+0000 7f0d40ff9640 1 -- 172.21.15.116:0/1604711368 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38035070 con 0x7f0d44103660 2024-09-12T03:02:04.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.741+0000 7f0d49efe640 1 -- 172.21.15.116:0/1604711368 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d441a5020 con 0x7f0d44103660 2024-09-12T03:02:04.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.742+0000 7f0d49efe640 1 -- 172.21.15.116:0/1604711368 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d44103120 con 0x7f0d44103660 2024-09-12T03:02:04.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.744+0000 7f0d40ff9640 1 -- 172.21.15.116:0/1604711368 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f0d38042030 con 0x7f0d44103660 2024-09-12T03:02:04.747 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:04 smithi177 ceph-mon[35305]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:02:04.748 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:04 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-12T03:02:04.748 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:04 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-12T03:02:04.748 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:04 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-12T03:02:04.748 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:04 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-09-12T03:02:04.748 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:04 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-09-12T03:02:04.748 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:04 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-12T03:02:04.748 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:04 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-12T03:02:04.748 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:04 smithi177 ceph-mon[35305]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T03:02:04.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.744+0000 7f0d40ff9640 1 --2- 172.21.15.116:0/1604711368 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0d2c0a2c10 0x7f0d2c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:04.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.745+0000 7f0d437fe640 1 --2- 172.21.15.116:0/1604711368 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0d2c0a2c10 0x7f0d2c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:04.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.745+0000 7f0d40ff9640 1 -- 172.21.15.116:0/1604711368 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f0d380efa70 con 0x7f0d44103660 2024-09-12T03:02:04.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.745+0000 7f0d437fe640 1 --2- 172.21.15.116:0/1604711368 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0d2c0a2c10 0x7f0d2c0a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0d34005fd0 tx=0x7f0d340093e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:04.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.748+0000 7f0d40ff9640 1 -- 172.21.15.116:0/1604711368 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0d380bc960 con 0x7f0d44103660 2024-09-12T03:02:04.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.922+0000 7f0d49efe640 1 -- 172.21.15.116:0/1604711368 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0d44062630 con 0x7f0d2c0a2c10 2024-09-12T03:02:04.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.929+0000 7f0d40ff9640 1 -- 172.21.15.116:0/1604711368 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258764 (secure 0 0 0) 0x7f0d44062630 con 0x7f0d2c0a2c10 2024-09-12T03:02:04.932 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:02:04.934 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-09-12T03:02:04.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.932+0000 7f0d49efe640 1 -- 172.21.15.116:0/1604711368 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0d2c0a2c10 msgr2=0x7f0d2c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:04.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.932+0000 7f0d49efe640 1 --2- 172.21.15.116:0/1604711368 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0d2c0a2c10 0x7f0d2c0a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0d34005fd0 tx=0x7f0d340093e0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.932+0000 7f0d49efe640 1 -- 172.21.15.116:0/1604711368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d44103660 msgr2=0x7f0d441989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:04.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.932+0000 7f0d49efe640 1 --2- 172.21.15.116:0/1604711368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d44103660 0x7f0d441989e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0d38006010 tx=0x7f0d380042f0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.932+0000 7f0d49efe640 1 -- 172.21.15.116:0/1604711368 shutdown_connections 2024-09-12T03:02:04.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.932+0000 7f0d49efe640 1 --2- 172.21.15.116:0/1604711368 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0d2c0a2c10 0x7f0d2c0a50d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.932+0000 7f0d49efe640 1 --2- 172.21.15.116:0/1604711368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d4410c170 0x7f0d4419cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.932+0000 7f0d49efe640 1 --2- 172.21.15.116:0/1604711368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d44103660 0x7f0d441989e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.932+0000 7f0d49efe640 1 --2- 172.21.15.116:0/1604711368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d44102d40 0x7f0d441984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.932+0000 7f0d49efe640 1 -- 172.21.15.116:0/1604711368 >> 172.21.15.116:0/1604711368 conn(0x7f0d440fc7a0 msgr2=0x7f0d440fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:04.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.933+0000 7f0d49efe640 1 -- 172.21.15.116:0/1604711368 shutdown_connections 2024-09-12T03:02:04.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:04.933+0000 7f0d49efe640 1 -- 172.21.15.116:0/1604711368 wait complete. 2024-09-12T03:02:05.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:05 smithi116 ceph-mon[31410]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T03:02:05.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:05 smithi116 ceph-mon[31410]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:02:05.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:05 smithi116 ceph-mon[31410]: pgmap v311: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2024-09-12T03:02:05.687 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-12T03:02:04.281230+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655964,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":473,"num_read_kb":350,"num_write":344,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368080,"kb_used_data":10512,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573744,"statfs":{"total":1151940427776,"available":1151563513856,"internally_reserved":0,"allocated":10764288,"data_stored":6938676,"data_compressed":107490,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.160727"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.348943+0000","last_change":"2024-09-12T03:00:58.348563+0000","last_active":"2024-09-12T03:00:59.348943+0000","last_peered":"2024-09-12T03:00:59.348943+0000","last_clean":"2024-09-12T03:00:59.348943+0000","last_became_active":"2024-09-12T03:00:58.347940+0000","last_became_peered":"2024-09-12T03:00:58.347940+0000","last_unstale":"2024-09-12T03:00:59.348943+0000","last_undegraded":"2024-09-12T03:00:59.348943+0000","last_fullsized":"2024-09-12T03:00:59.348943+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:43.489238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342719+0000","last_change":"2024-09-12T03:00:58.351969+0000","last_active":"2024-09-12T03:00:59.342719+0000","last_peered":"2024-09-12T03:00:59.342719+0000","last_clean":"2024-09-12T03:00:59.342719+0000","last_became_active":"2024-09-12T03:00:58.351679+0000","last_became_peered":"2024-09-12T03:00:58.351679+0000","last_unstale":"2024-09-12T03:00:59.342719+0000","last_undegraded":"2024-09-12T03:00:59.342719+0000","last_fullsized":"2024-09-12T03:00:59.342719+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:27:01.174890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342541+0000","last_change":"2024-09-12T03:00:58.346045+0000","last_active":"2024-09-12T03:00:59.342541+0000","last_peered":"2024-09-12T03:00:59.342541+0000","last_clean":"2024-09-12T03:00:59.342541+0000","last_became_active":"2024-09-12T03:00:58.345719+0000","last_became_peered":"2024-09-12T03:00:58.345719+0000","last_unstale":"2024-09-12T03:00:59.342541+0000","last_undegraded":"2024-09-12T03:00:59.342541+0000","last_fullsized":"2024-09-12T03:00:59.342541+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:12.312650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343947+0000","last_change":"2024-09-12T03:00:58.348165+0000","last_active":"2024-09-12T03:00:59.343947+0000","last_peered":"2024-09-12T03:00:59.343947+0000","last_clean":"2024-09-12T03:00:59.343947+0000","last_became_active":"2024-09-12T03:00:58.347734+0000","last_became_peered":"2024-09-12T03:00:58.347734+0000","last_unstale":"2024-09-12T03:00:59.343947+0000","last_undegraded":"2024-09-12T03:00:59.343947+0000","last_fullsized":"2024-09-12T03:00:59.343947+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:05.945900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343877+0000","last_change":"2024-09-12T03:00:58.339316+0000","last_active":"2024-09-12T03:00:59.343877+0000","last_peered":"2024-09-12T03:00:59.343877+0000","last_clean":"2024-09-12T03:00:59.343877+0000","last_became_active":"2024-09-12T03:00:58.338964+0000","last_became_peered":"2024-09-12T03:00:58.338964+0000","last_unstale":"2024-09-12T03:00:59.343877+0000","last_undegraded":"2024-09-12T03:00:59.343877+0000","last_fullsized":"2024-09-12T03:00:59.343877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:50.721276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342586+0000","last_change":"2024-09-12T03:00:58.346838+0000","last_active":"2024-09-12T03:00:59.342586+0000","last_peered":"2024-09-12T03:00:59.342586+0000","last_clean":"2024-09-12T03:00:59.342586+0000","last_became_active":"2024-09-12T03:00:58.346486+0000","last_became_peered":"2024-09-12T03:00:58.346486+0000","last_unstale":"2024-09-12T03:00:59.342586+0000","last_undegraded":"2024-09-12T03:00:59.342586+0000","last_fullsized":"2024-09-12T03:00:59.342586+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:28.687727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.283454+0000","last_change":"2024-09-12T03:00:58.346712+0000","last_active":"2024-09-12T03:02:03.283454+0000","last_peered":"2024-09-12T03:02:03.283454+0000","last_clean":"2024-09-12T03:02:03.283454+0000","last_became_active":"2024-09-12T03:00:58.346397+0000","last_became_peered":"2024-09-12T03:00:58.346397+0000","last_unstale":"2024-09-12T03:02:03.283454+0000","last_undegraded":"2024-09-12T03:02:03.283454+0000","last_fullsized":"2024-09-12T03:02:03.283454+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:48:00.624733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343665+0000","last_change":"2024-09-12T03:00:58.356166+0000","last_active":"2024-09-12T03:00:59.343665+0000","last_peered":"2024-09-12T03:00:59.343665+0000","last_clean":"2024-09-12T03:00:59.343665+0000","last_became_active":"2024-09-12T03:00:58.354944+0000","last_became_peered":"2024-09-12T03:00:58.354944+0000","last_unstale":"2024-09-12T03:00:59.343665+0000","last_undegraded":"2024-09-12T03:00:59.343665+0000","last_fullsized":"2024-09-12T03:00:59.343665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:15.840149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.329171+0000","last_change":"2024-09-12T03:00:58.340182+0000","last_active":"2024-09-12T03:00:59.329171+0000","last_peered":"2024-09-12T03:00:59.329171+0000","last_clean":"2024-09-12T03:00:59.329171+0000","last_became_active":"2024-09-12T03:00:58.339576+0000","last_became_peered":"2024-09-12T03:00:58.339576+0000","last_unstale":"2024-09-12T03:00:59.329171+0000","last_undegraded":"2024-09-12T03:00:59.329171+0000","last_fullsized":"2024-09-12T03:00:59.329171+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:16.534844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344683+0000","last_change":"2024-09-12T03:00:58.351218+0000","last_active":"2024-09-12T03:00:59.344683+0000","last_peered":"2024-09-12T03:00:59.344683+0000","last_clean":"2024-09-12T03:00:59.344683+0000","last_became_active":"2024-09-12T03:00:58.350871+0000","last_became_peered":"2024-09-12T03:00:58.350871+0000","last_unstale":"2024-09-12T03:00:59.344683+0000","last_undegraded":"2024-09-12T03:00:59.344683+0000","last_fullsized":"2024-09-12T03:00:59.344683+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:38.367683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.283798+0000","last_change":"2024-09-12T03:00:58.351894+0000","last_active":"2024-09-12T03:02:03.283798+0000","last_peered":"2024-09-12T03:02:03.283798+0000","last_clean":"2024-09-12T03:02:03.283798+0000","last_became_active":"2024-09-12T03:00:58.351445+0000","last_became_peered":"2024-09-12T03:00:58.351445+0000","last_unstale":"2024-09-12T03:02:03.283798+0000","last_undegraded":"2024-09-12T03:02:03.283798+0000","last_fullsized":"2024-09-12T03:02:03.283798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:42.093871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344004+0000","last_change":"2024-09-12T03:00:58.353460+0000","last_active":"2024-09-12T03:00:59.344004+0000","last_peered":"2024-09-12T03:00:59.344004+0000","last_clean":"2024-09-12T03:00:59.344004+0000","last_became_active":"2024-09-12T03:00:58.353111+0000","last_became_peered":"2024-09-12T03:00:58.353111+0000","last_unstale":"2024-09-12T03:00:59.344004+0000","last_undegraded":"2024-09-12T03:00:59.344004+0000","last_fullsized":"2024-09-12T03:00:59.344004+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:15.916765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.330518+0000","last_change":"2024-09-12T03:00:58.341080+0000","last_active":"2024-09-12T03:00:59.330518+0000","last_peered":"2024-09-12T03:00:59.330518+0000","last_clean":"2024-09-12T03:00:59.330518+0000","last_became_active":"2024-09-12T03:00:58.340782+0000","last_became_peered":"2024-09-12T03:00:58.340782+0000","last_unstale":"2024-09-12T03:00:59.330518+0000","last_undegraded":"2024-09-12T03:00:59.330518+0000","last_fullsized":"2024-09-12T03:00:59.330518+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:36.569700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342701+0000","last_change":"2024-09-12T03:00:58.346791+0000","last_active":"2024-09-12T03:00:59.342701+0000","last_peered":"2024-09-12T03:00:59.342701+0000","last_clean":"2024-09-12T03:00:59.342701+0000","last_became_active":"2024-09-12T03:00:58.346500+0000","last_became_peered":"2024-09-12T03:00:58.346500+0000","last_unstale":"2024-09-12T03:00:59.342701+0000","last_undegraded":"2024-09-12T03:00:59.342701+0000","last_fullsized":"2024-09-12T03:00:59.342701+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:32.017589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342415+0000","last_change":"2024-09-12T03:00:58.349522+0000","last_active":"2024-09-12T03:00:59.342415+0000","last_peered":"2024-09-12T03:00:59.342415+0000","last_clean":"2024-09-12T03:00:59.342415+0000","last_became_active":"2024-09-12T03:00:58.349246+0000","last_became_peered":"2024-09-12T03:00:58.349246+0000","last_unstale":"2024-09-12T03:00:59.342415+0000","last_undegraded":"2024-09-12T03:00:59.342415+0000","last_fullsized":"2024-09-12T03:00:59.342415+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:39.253919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328399+0000","last_change":"2024-09-12T03:00:58.349909+0000","last_active":"2024-09-12T03:00:59.328399+0000","last_peered":"2024-09-12T03:00:59.328399+0000","last_clean":"2024-09-12T03:00:59.328399+0000","last_became_active":"2024-09-12T03:00:58.349590+0000","last_became_peered":"2024-09-12T03:00:58.349590+0000","last_unstale":"2024-09-12T03:00:59.328399+0000","last_undegraded":"2024-09-12T03:00:59.328399+0000","last_fullsized":"2024-09-12T03:00:59.328399+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:04.540709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.349104+0000","last_change":"2024-09-12T03:00:58.348284+0000","last_active":"2024-09-12T03:00:59.349104+0000","last_peered":"2024-09-12T03:00:59.349104+0000","last_clean":"2024-09-12T03:00:59.349104+0000","last_became_active":"2024-09-12T03:00:58.347465+0000","last_became_peered":"2024-09-12T03:00:58.347465+0000","last_unstale":"2024-09-12T03:00:59.349104+0000","last_undegraded":"2024-09-12T03:00:59.349104+0000","last_fullsized":"2024-09-12T03:00:59.349104+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:13.191899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343137+0000","last_change":"2024-09-12T03:00:58.337740+0000","last_active":"2024-09-12T03:00:59.343137+0000","last_peered":"2024-09-12T03:00:59.343137+0000","last_clean":"2024-09-12T03:00:59.343137+0000","last_became_active":"2024-09-12T03:00:58.337393+0000","last_became_peered":"2024-09-12T03:00:58.337393+0000","last_unstale":"2024-09-12T03:00:59.343137+0000","last_undegraded":"2024-09-12T03:00:59.343137+0000","last_fullsized":"2024-09-12T03:00:59.343137+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:28:56.339167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344750+0000","last_change":"2024-09-12T03:00:58.345779+0000","last_active":"2024-09-12T03:00:59.344750+0000","last_peered":"2024-09-12T03:00:59.344750+0000","last_clean":"2024-09-12T03:00:59.344750+0000","last_became_active":"2024-09-12T03:00:58.345210+0000","last_became_peered":"2024-09-12T03:00:58.345210+0000","last_unstale":"2024-09-12T03:00:59.344750+0000","last_undegraded":"2024-09-12T03:00:59.344750+0000","last_fullsized":"2024-09-12T03:00:59.344750+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:06:58.271185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.348332+0000","last_change":"2024-09-12T03:00:58.343877+0000","last_active":"2024-09-12T03:00:59.348332+0000","last_peered":"2024-09-12T03:00:59.348332+0000","last_clean":"2024-09-12T03:00:59.348332+0000","last_became_active":"2024-09-12T03:00:58.343545+0000","last_became_peered":"2024-09-12T03:00:58.343545+0000","last_unstale":"2024-09-12T03:00:59.348332+0000","last_undegraded":"2024-09-12T03:00:59.348332+0000","last_fullsized":"2024-09-12T03:00:59.348332+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:52:31.851975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.330596+0000","last_change":"2024-09-12T03:00:58.349233+0000","last_active":"2024-09-12T03:00:59.330596+0000","last_peered":"2024-09-12T03:00:59.330596+0000","last_clean":"2024-09-12T03:00:59.330596+0000","last_became_active":"2024-09-12T03:00:58.348958+0000","last_became_peered":"2024-09-12T03:00:58.348958+0000","last_unstale":"2024-09-12T03:00:59.330596+0000","last_undegraded":"2024-09-12T03:00:59.330596+0000","last_fullsized":"2024-09-12T03:00:59.330596+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:32:10.271817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328487+0000","last_change":"2024-09-12T03:00:58.358024+0000","last_active":"2024-09-12T03:00:59.328487+0000","last_peered":"2024-09-12T03:00:59.328487+0000","last_clean":"2024-09-12T03:00:59.328487+0000","last_became_active":"2024-09-12T03:00:58.357434+0000","last_became_peered":"2024-09-12T03:00:58.357434+0000","last_unstale":"2024-09-12T03:00:59.328487+0000","last_undegraded":"2024-09-12T03:00:59.328487+0000","last_fullsized":"2024-09-12T03:00:59.328487+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:01:20.119733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.345193+0000","last_change":"2024-09-12T03:00:58.355063+0000","last_active":"2024-09-12T03:00:59.345193+0000","last_peered":"2024-09-12T03:00:59.345193+0000","last_clean":"2024-09-12T03:00:59.345193+0000","last_became_active":"2024-09-12T03:00:58.354760+0000","last_became_peered":"2024-09-12T03:00:58.354760+0000","last_unstale":"2024-09-12T03:00:59.345193+0000","last_undegraded":"2024-09-12T03:00:59.345193+0000","last_fullsized":"2024-09-12T03:00:59.345193+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:13.464413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.280040+0000","last_change":"2024-09-12T03:00:58.341916+0000","last_active":"2024-09-12T03:02:03.280040+0000","last_peered":"2024-09-12T03:02:03.280040+0000","last_clean":"2024-09-12T03:02:03.280040+0000","last_became_active":"2024-09-12T03:00:58.340815+0000","last_became_peered":"2024-09-12T03:00:58.340815+0000","last_unstale":"2024-09-12T03:02:03.280040+0000","last_undegraded":"2024-09-12T03:02:03.280040+0000","last_fullsized":"2024-09-12T03:02:03.280040+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:13.777947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343925+0000","last_change":"2024-09-12T03:00:58.357765+0000","last_active":"2024-09-12T03:00:59.343925+0000","last_peered":"2024-09-12T03:00:59.343925+0000","last_clean":"2024-09-12T03:00:59.343925+0000","last_became_active":"2024-09-12T03:00:58.357480+0000","last_became_peered":"2024-09-12T03:00:58.357480+0000","last_unstale":"2024-09-12T03:00:59.343925+0000","last_undegraded":"2024-09-12T03:00:59.343925+0000","last_fullsized":"2024-09-12T03:00:59.343925+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:26.884381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344699+0000","last_change":"2024-09-12T03:00:58.349504+0000","last_active":"2024-09-12T03:00:59.344699+0000","last_peered":"2024-09-12T03:00:59.344699+0000","last_clean":"2024-09-12T03:00:59.344699+0000","last_became_active":"2024-09-12T03:00:58.349136+0000","last_became_peered":"2024-09-12T03:00:58.349136+0000","last_unstale":"2024-09-12T03:00:59.344699+0000","last_undegraded":"2024-09-12T03:00:59.344699+0000","last_fullsized":"2024-09-12T03:00:59.344699+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:09:55.229516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.348719+0000","last_change":"2024-09-12T03:00:58.341902+0000","last_active":"2024-09-12T03:00:59.348719+0000","last_peered":"2024-09-12T03:00:59.348719+0000","last_clean":"2024-09-12T03:00:59.348719+0000","last_became_active":"2024-09-12T03:00:58.341592+0000","last_became_peered":"2024-09-12T03:00:58.341592+0000","last_unstale":"2024-09-12T03:00:59.348719+0000","last_undegraded":"2024-09-12T03:00:59.348719+0000","last_fullsized":"2024-09-12T03:00:59.348719+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:13.852511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.330540+0000","last_change":"2024-09-12T03:00:58.348012+0000","last_active":"2024-09-12T03:00:59.330540+0000","last_peered":"2024-09-12T03:00:59.330540+0000","last_clean":"2024-09-12T03:00:59.330540+0000","last_became_active":"2024-09-12T03:00:58.347338+0000","last_became_peered":"2024-09-12T03:00:58.347338+0000","last_unstale":"2024-09-12T03:00:59.330540+0000","last_undegraded":"2024-09-12T03:00:59.330540+0000","last_fullsized":"2024-09-12T03:00:59.330540+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:41.478032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342928+0000","last_change":"2024-09-12T03:00:58.345833+0000","last_active":"2024-09-12T03:00:59.342928+0000","last_peered":"2024-09-12T03:00:59.342928+0000","last_clean":"2024-09-12T03:00:59.342928+0000","last_became_active":"2024-09-12T03:00:58.345482+0000","last_became_peered":"2024-09-12T03:00:58.345482+0000","last_unstale":"2024-09-12T03:00:59.342928+0000","last_undegraded":"2024-09-12T03:00:59.342928+0000","last_fullsized":"2024-09-12T03:00:59.342928+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:32.037412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343059+0000","last_change":"2024-09-12T03:00:58.352290+0000","last_active":"2024-09-12T03:00:59.343059+0000","last_peered":"2024-09-12T03:00:59.343059+0000","last_clean":"2024-09-12T03:00:59.343059+0000","last_became_active":"2024-09-12T03:00:58.351918+0000","last_became_peered":"2024-09-12T03:00:58.351918+0000","last_unstale":"2024-09-12T03:00:59.343059+0000","last_undegraded":"2024-09-12T03:00:59.343059+0000","last_fullsized":"2024-09-12T03:00:59.343059+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:25.941080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343279+0000","last_change":"2024-09-12T03:00:58.348021+0000","last_active":"2024-09-12T03:00:59.343279+0000","last_peered":"2024-09-12T03:00:59.343279+0000","last_clean":"2024-09-12T03:00:59.343279+0000","last_became_active":"2024-09-12T03:00:58.347634+0000","last_became_peered":"2024-09-12T03:00:58.347634+0000","last_unstale":"2024-09-12T03:00:59.343279+0000","last_undegraded":"2024-09-12T03:00:59.343279+0000","last_fullsized":"2024-09-12T03:00:59.343279+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:47.933067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.347866+0000","last_change":"2024-09-12T03:00:58.340891+0000","last_active":"2024-09-12T03:00:59.347866+0000","last_peered":"2024-09-12T03:00:59.347866+0000","last_clean":"2024-09-12T03:00:59.347866+0000","last_became_active":"2024-09-12T03:00:58.340504+0000","last_became_peered":"2024-09-12T03:00:58.340504+0000","last_unstale":"2024-09-12T03:00:59.347866+0000","last_undegraded":"2024-09-12T03:00:59.347866+0000","last_fullsized":"2024-09-12T03:00:59.347866+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:32.117635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343541+0000","last_change":"2024-09-12T03:00:58.339343+0000","last_active":"2024-09-12T03:00:59.343541+0000","last_peered":"2024-09-12T03:00:59.343541+0000","last_clean":"2024-09-12T03:00:59.343541+0000","last_became_active":"2024-09-12T03:00:58.339001+0000","last_became_peered":"2024-09-12T03:00:58.339001+0000","last_unstale":"2024-09-12T03:00:59.343541+0000","last_undegraded":"2024-09-12T03:00:59.343541+0000","last_fullsized":"2024-09-12T03:00:59.343541+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:02.043454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":83,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.281673+0000","last_change":"2024-09-12T03:00:56.315550+0000","last_active":"2024-09-12T03:02:03.281673+0000","last_peered":"2024-09-12T03:02:03.281673+0000","last_clean":"2024-09-12T03:02:03.281673+0000","last_became_active":"2024-09-12T03:00:56.315225+0000","last_became_peered":"2024-09-12T03:00:56.315225+0000","last_unstale":"2024-09-12T03:02:03.281673+0000","last_undegraded":"2024-09-12T03:02:03.281673+0000","last_fullsized":"2024-09-12T03:02:03.281673+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:46.070895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328646+0000","last_change":"2024-09-12T03:00:58.338379+0000","last_active":"2024-09-12T03:00:59.328646+0000","last_peered":"2024-09-12T03:00:59.328646+0000","last_clean":"2024-09-12T03:00:59.328646+0000","last_became_active":"2024-09-12T03:00:58.337795+0000","last_became_peered":"2024-09-12T03:00:58.337795+0000","last_unstale":"2024-09-12T03:00:59.328646+0000","last_undegraded":"2024-09-12T03:00:59.328646+0000","last_fullsized":"2024-09-12T03:00:59.328646+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:43.299329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.397617+0000","last_change":"2024-09-12T03:00:56.316385+0000","last_active":"2024-09-12T03:01:07.397617+0000","last_peered":"2024-09-12T03:01:07.397617+0000","last_clean":"2024-09-12T03:01:07.397617+0000","last_became_active":"2024-09-12T03:00:56.316064+0000","last_became_peered":"2024-09-12T03:00:56.316064+0000","last_unstale":"2024-09-12T03:01:07.397617+0000","last_undegraded":"2024-09-12T03:01:07.397617+0000","last_fullsized":"2024-09-12T03:01:07.397617+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:12:43.323654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.397886+0000","last_change":"2024-09-12T03:00:56.313065+0000","last_active":"2024-09-12T03:01:07.397886+0000","last_peered":"2024-09-12T03:01:07.397886+0000","last_clean":"2024-09-12T03:01:07.397886+0000","last_became_active":"2024-09-12T03:00:56.312716+0000","last_became_peered":"2024-09-12T03:00:56.312716+0000","last_unstale":"2024-09-12T03:01:07.397886+0000","last_undegraded":"2024-09-12T03:01:07.397886+0000","last_fullsized":"2024-09-12T03:01:07.397886+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:51.089089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.346221+0000","last_change":"2024-09-12T03:00:58.346542+0000","last_active":"2024-09-12T03:00:59.346221+0000","last_peered":"2024-09-12T03:00:59.346221+0000","last_clean":"2024-09-12T03:00:59.346221+0000","last_became_active":"2024-09-12T03:00:58.346260+0000","last_became_peered":"2024-09-12T03:00:58.346260+0000","last_unstale":"2024-09-12T03:00:59.346221+0000","last_undegraded":"2024-09-12T03:00:59.346221+0000","last_fullsized":"2024-09-12T03:00:59.346221+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:37.034969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.280332+0000","last_change":"2024-09-12T03:00:56.317308+0000","last_active":"2024-09-12T03:02:03.280332+0000","last_peered":"2024-09-12T03:02:03.280332+0000","last_clean":"2024-09-12T03:02:03.280332+0000","last_became_active":"2024-09-12T03:00:56.316935+0000","last_became_peered":"2024-09-12T03:00:56.316935+0000","last_unstale":"2024-09-12T03:02:03.280332+0000","last_undegraded":"2024-09-12T03:02:03.280332+0000","last_fullsized":"2024-09-12T03:02:03.280332+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:06.324489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343312+0000","last_change":"2024-09-12T03:00:58.341565+0000","last_active":"2024-09-12T03:00:59.343312+0000","last_peered":"2024-09-12T03:00:59.343312+0000","last_clean":"2024-09-12T03:00:59.343312+0000","last_became_active":"2024-09-12T03:00:58.341285+0000","last_became_peered":"2024-09-12T03:00:58.341285+0000","last_unstale":"2024-09-12T03:00:59.343312+0000","last_undegraded":"2024-09-12T03:00:59.343312+0000","last_fullsized":"2024-09-12T03:00:59.343312+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:59.156700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":74,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:59.476351+0000","last_change":"2024-09-12T03:00:56.314201+0000","last_active":"2024-09-12T03:01:59.476351+0000","last_peered":"2024-09-12T03:01:59.476351+0000","last_clean":"2024-09-12T03:01:59.476351+0000","last_became_active":"2024-09-12T03:00:56.313880+0000","last_became_peered":"2024-09-12T03:00:56.313880+0000","last_unstale":"2024-09-12T03:01:59.476351+0000","last_undegraded":"2024-09-12T03:01:59.476351+0000","last_fullsized":"2024-09-12T03:01:59.476351+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:57:12.952630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.329144+0000","last_change":"2024-09-12T03:00:58.345274+0000","last_active":"2024-09-12T03:00:59.329144+0000","last_peered":"2024-09-12T03:00:59.329144+0000","last_clean":"2024-09-12T03:00:59.329144+0000","last_became_active":"2024-09-12T03:00:58.344937+0000","last_became_peered":"2024-09-12T03:00:58.344937+0000","last_unstale":"2024-09-12T03:00:59.329144+0000","last_undegraded":"2024-09-12T03:00:59.329144+0000","last_fullsized":"2024-09-12T03:00:59.329144+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:17.288909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.283992+0000","last_change":"2024-09-12T03:00:56.314709+0000","last_active":"2024-09-12T03:02:03.283992+0000","last_peered":"2024-09-12T03:02:03.283992+0000","last_clean":"2024-09-12T03:02:03.283992+0000","last_became_active":"2024-09-12T03:00:56.314349+0000","last_became_peered":"2024-09-12T03:00:56.314349+0000","last_unstale":"2024-09-12T03:02:03.283992+0000","last_undegraded":"2024-09-12T03:02:03.283992+0000","last_fullsized":"2024-09-12T03:02:03.283992+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:18:35.371661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343140+0000","last_change":"2024-09-12T03:00:58.346055+0000","last_active":"2024-09-12T03:00:59.343140+0000","last_peered":"2024-09-12T03:00:59.343140+0000","last_clean":"2024-09-12T03:00:59.343140+0000","last_became_active":"2024-09-12T03:00:58.345701+0000","last_became_peered":"2024-09-12T03:00:58.345701+0000","last_unstale":"2024-09-12T03:00:59.343140+0000","last_undegraded":"2024-09-12T03:00:59.343140+0000","last_fullsized":"2024-09-12T03:00:59.343140+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:33.977245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.282131+0000","last_change":"2024-09-12T03:00:56.435962+0000","last_active":"2024-09-12T03:02:03.282131+0000","last_peered":"2024-09-12T03:02:03.282131+0000","last_clean":"2024-09-12T03:02:03.282131+0000","last_became_active":"2024-09-12T03:00:56.435515+0000","last_became_peered":"2024-09-12T03:00:56.435515+0000","last_unstale":"2024-09-12T03:02:03.282131+0000","last_undegraded":"2024-09-12T03:02:03.282131+0000","last_fullsized":"2024-09-12T03:02:03.282131+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:32.953436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328662+0000","last_change":"2024-09-12T03:00:58.344239+0000","last_active":"2024-09-12T03:00:59.328662+0000","last_peered":"2024-09-12T03:00:59.328662+0000","last_clean":"2024-09-12T03:00:59.328662+0000","last_became_active":"2024-09-12T03:00:58.343876+0000","last_became_peered":"2024-09-12T03:00:58.343876+0000","last_unstale":"2024-09-12T03:00:59.328662+0000","last_undegraded":"2024-09-12T03:00:59.328662+0000","last_fullsized":"2024-09-12T03:00:59.328662+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:49.401295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'29","reported_seq":162,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:02:00.477921+0000","last_change":"2024-09-12T03:00:56.317708+0000","last_active":"2024-09-12T03:02:00.477921+0000","last_peered":"2024-09-12T03:02:00.477921+0000","last_clean":"2024-09-12T03:02:00.477921+0000","last_became_active":"2024-09-12T03:00:56.317317+0000","last_became_peered":"2024-09-12T03:00:56.317317+0000","last_unstale":"2024-09-12T03:02:00.477921+0000","last_undegraded":"2024-09-12T03:02:00.477921+0000","last_fullsized":"2024-09-12T03:02:00.477921+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:47.788186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61752,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":115,"num_read_kb":117,"num_write":30,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.346470+0000","last_change":"2024-09-12T03:00:58.343602+0000","last_active":"2024-09-12T03:00:59.346470+0000","last_peered":"2024-09-12T03:00:59.346470+0000","last_clean":"2024-09-12T03:00:59.346470+0000","last_became_active":"2024-09-12T03:00:58.343301+0000","last_became_peered":"2024-09-12T03:00:58.343301+0000","last_unstale":"2024-09-12T03:00:59.346470+0000","last_undegraded":"2024-09-12T03:00:59.346470+0000","last_fullsized":"2024-09-12T03:00:59.346470+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:04.056118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.398339+0000","last_change":"2024-09-12T03:00:56.317874+0000","last_active":"2024-09-12T03:01:07.398339+0000","last_peered":"2024-09-12T03:01:07.398339+0000","last_clean":"2024-09-12T03:01:07.398339+0000","last_became_active":"2024-09-12T03:00:56.317566+0000","last_became_peered":"2024-09-12T03:00:56.317566+0000","last_unstale":"2024-09-12T03:01:07.398339+0000","last_undegraded":"2024-09-12T03:01:07.398339+0000","last_fullsized":"2024-09-12T03:01:07.398339+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:54.203730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.330344+0000","last_change":"2024-09-12T03:00:58.351539+0000","last_active":"2024-09-12T03:00:59.330344+0000","last_peered":"2024-09-12T03:00:59.330344+0000","last_clean":"2024-09-12T03:00:59.330344+0000","last_became_active":"2024-09-12T03:00:58.351249+0000","last_became_peered":"2024-09-12T03:00:58.351249+0000","last_unstale":"2024-09-12T03:00:59.330344+0000","last_undegraded":"2024-09-12T03:00:59.330344+0000","last_fullsized":"2024-09-12T03:00:59.330344+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:40.498867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.280655+0000","last_change":"2024-09-12T03:00:56.314489+0000","last_active":"2024-09-12T03:02:03.280655+0000","last_peered":"2024-09-12T03:02:03.280655+0000","last_clean":"2024-09-12T03:02:03.280655+0000","last_became_active":"2024-09-12T03:00:56.314142+0000","last_became_peered":"2024-09-12T03:00:56.314142+0000","last_unstale":"2024-09-12T03:02:03.280655+0000","last_undegraded":"2024-09-12T03:02:03.280655+0000","last_fullsized":"2024-09-12T03:02:03.280655+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:21.541713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.283805+0000","last_change":"2024-09-12T03:00:58.348752+0000","last_active":"2024-09-12T03:02:03.283805+0000","last_peered":"2024-09-12T03:02:03.283805+0000","last_clean":"2024-09-12T03:02:03.283805+0000","last_became_active":"2024-09-12T03:00:58.348391+0000","last_became_peered":"2024-09-12T03:00:58.348391+0000","last_unstale":"2024-09-12T03:02:03.283805+0000","last_undegraded":"2024-09-12T03:02:03.283805+0000","last_fullsized":"2024-09-12T03:02:03.283805+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:33:15.626306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'3","reported_seq":75,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:58.474714+0000","last_change":"2024-09-12T03:00:56.311702+0000","last_active":"2024-09-12T03:01:58.474714+0000","last_peered":"2024-09-12T03:01:58.474714+0000","last_clean":"2024-09-12T03:01:58.474714+0000","last_became_active":"2024-09-12T03:00:56.311353+0000","last_became_peered":"2024-09-12T03:00:56.311353+0000","last_unstale":"2024-09-12T03:01:58.474714+0000","last_undegraded":"2024-09-12T03:01:58.474714+0000","last_fullsized":"2024-09-12T03:01:58.474714+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:01.004858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.283381+0000","last_change":"2024-09-12T03:00:58.351973+0000","last_active":"2024-09-12T03:02:03.283381+0000","last_peered":"2024-09-12T03:02:03.283381+0000","last_clean":"2024-09-12T03:02:03.283381+0000","last_became_active":"2024-09-12T03:00:58.351599+0000","last_became_peered":"2024-09-12T03:00:58.351599+0000","last_unstale":"2024-09-12T03:02:03.283381+0000","last_undegraded":"2024-09-12T03:02:03.283381+0000","last_fullsized":"2024-09-12T03:02:03.283381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:26.398828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.399761+0000","last_change":"2024-09-12T03:00:56.436185+0000","last_active":"2024-09-12T03:01:07.399761+0000","last_peered":"2024-09-12T03:01:07.399761+0000","last_clean":"2024-09-12T03:01:07.399761+0000","last_became_active":"2024-09-12T03:00:56.435717+0000","last_became_peered":"2024-09-12T03:00:56.435717+0000","last_unstale":"2024-09-12T03:01:07.399761+0000","last_undegraded":"2024-09-12T03:01:07.399761+0000","last_fullsized":"2024-09-12T03:01:07.399761+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:27.303182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"peering","last_fresh":"2024-09-12T03:00:59.328638+0000","last_change":"2024-09-12T03:02:03.280677+0000","last_active":"2024-09-12T03:02:03.280677+0000","last_peered":"2024-09-12T03:00:59.328638+0000","last_clean":"2024-09-12T03:00:59.328638+0000","last_became_active":"2024-09-12T03:00:58.346490+0000","last_became_peered":"2024-09-12T03:00:58.346490+0000","last_unstale":"2024-09-12T03:00:59.328638+0000","last_undegraded":"2024-09-12T03:00:59.328638+0000","last_fullsized":"2024-09-12T03:00:59.328638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4,8],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:08.394235+0000","last_change":"2024-09-12T03:00:56.311333+0000","last_active":"2024-09-12T03:01:08.394235+0000","last_peered":"2024-09-12T03:01:08.394235+0000","last_clean":"2024-09-12T03:01:08.394235+0000","last_became_active":"2024-09-12T03:00:56.310871+0000","last_became_peered":"2024-09-12T03:00:56.310871+0000","last_unstale":"2024-09-12T03:01:08.394235+0000","last_undegraded":"2024-09-12T03:01:08.394235+0000","last_fullsized":"2024-09-12T03:01:08.394235+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:59.344572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.348769+0000","last_change":"2024-09-12T03:00:58.349401+0000","last_active":"2024-09-12T03:00:59.348769+0000","last_peered":"2024-09-12T03:00:59.348769+0000","last_clean":"2024-09-12T03:00:59.348769+0000","last_became_active":"2024-09-12T03:00:58.348996+0000","last_became_peered":"2024-09-12T03:00:58.348996+0000","last_unstale":"2024-09-12T03:00:59.348769+0000","last_undegraded":"2024-09-12T03:00:59.348769+0000","last_fullsized":"2024-09-12T03:00:59.348769+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:20:26.201684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344392+0000","last_change":"2024-09-12T03:00:56.314182+0000","last_active":"2024-09-12T03:00:59.344392+0000","last_peered":"2024-09-12T03:00:59.344392+0000","last_clean":"2024-09-12T03:00:59.344392+0000","last_became_active":"2024-09-12T03:00:56.313749+0000","last_became_peered":"2024-09-12T03:00:56.313749+0000","last_unstale":"2024-09-12T03:00:59.344392+0000","last_undegraded":"2024-09-12T03:00:59.344392+0000","last_fullsized":"2024-09-12T03:00:59.344392+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:31:45.767500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.284069+0000","last_change":"2024-09-12T03:00:58.344356+0000","last_active":"2024-09-12T03:02:03.284069+0000","last_peered":"2024-09-12T03:02:03.284069+0000","last_clean":"2024-09-12T03:02:03.284069+0000","last_became_active":"2024-09-12T03:00:58.344035+0000","last_became_peered":"2024-09-12T03:00:58.344035+0000","last_unstale":"2024-09-12T03:02:03.284069+0000","last_undegraded":"2024-09-12T03:02:03.284069+0000","last_fullsized":"2024-09-12T03:02:03.284069+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:45:18.681018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.282252+0000","last_change":"2024-09-12T03:00:56.315653+0000","last_active":"2024-09-12T03:02:03.282252+0000","last_peered":"2024-09-12T03:02:03.282252+0000","last_clean":"2024-09-12T03:02:03.282252+0000","last_became_active":"2024-09-12T03:00:56.315284+0000","last_became_peered":"2024-09-12T03:00:56.315284+0000","last_unstale":"2024-09-12T03:02:03.282252+0000","last_undegraded":"2024-09-12T03:02:03.282252+0000","last_fullsized":"2024-09-12T03:02:03.282252+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:42.934020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":15,"reported_epoch":78,"state":"peering","last_fresh":"2024-09-12T03:01:17.267451+0000","last_change":"2024-09-12T03:02:03.283802+0000","last_active":"2024-09-12T03:02:03.283802+0000","last_peered":"2024-09-12T03:01:17.267451+0000","last_clean":"2024-09-12T03:01:17.267451+0000","last_became_active":"2024-09-12T03:00:58.357451+0000","last_became_peered":"2024-09-12T03:00:58.357451+0000","last_unstale":"2024-09-12T03:01:17.267451+0000","last_undegraded":"2024-09-12T03:01:17.267451+0000","last_fullsized":"2024-09-12T03:01:17.267451+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5,9],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.398109+0000","last_change":"2024-09-12T03:00:56.746780+0000","last_active":"2024-09-12T03:01:07.398109+0000","last_peered":"2024-09-12T03:01:07.398109+0000","last_clean":"2024-09-12T03:01:07.398109+0000","last_became_active":"2024-09-12T03:00:56.746414+0000","last_became_peered":"2024-09-12T03:00:56.746414+0000","last_unstale":"2024-09-12T03:01:07.398109+0000","last_undegraded":"2024-09-12T03:01:07.398109+0000","last_fullsized":"2024-09-12T03:01:07.398109+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:15:35.214196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.283877+0000","last_change":"2024-09-12T03:00:58.351713+0000","last_active":"2024-09-12T03:02:03.283877+0000","last_peered":"2024-09-12T03:02:03.283877+0000","last_clean":"2024-09-12T03:02:03.283877+0000","last_became_active":"2024-09-12T03:00:58.351371+0000","last_became_peered":"2024-09-12T03:00:58.351371+0000","last_unstale":"2024-09-12T03:02:03.283877+0000","last_undegraded":"2024-09-12T03:02:03.283877+0000","last_fullsized":"2024-09-12T03:02:03.283877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:13.346956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'137","reported_seq":333,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.283543+0000","last_change":"2024-09-12T02:59:45.461482+0000","last_active":"2024-09-12T03:02:03.283543+0000","last_peered":"2024-09-12T03:02:03.283543+0000","last_clean":"2024-09-12T03:02:03.283543+0000","last_became_active":"2024-09-12T02:59:45.461058+0000","last_became_peered":"2024-09-12T02:59:45.461058+0000","last_unstale":"2024-09-12T03:02:03.283543+0000","last_undegraded":"2024-09-12T03:02:03.283543+0000","last_fullsized":"2024-09-12T03:02:03.283543+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:18.034438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:18.034438+0000","last_clean_scrub_stamp":"2024-09-12T02:56:18.034438+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:09.251858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:08.398498+0000","last_change":"2024-09-12T03:00:56.315245+0000","last_active":"2024-09-12T03:01:08.398498+0000","last_peered":"2024-09-12T03:01:08.398498+0000","last_clean":"2024-09-12T03:01:08.398498+0000","last_became_active":"2024-09-12T03:00:56.314887+0000","last_became_peered":"2024-09-12T03:00:56.314887+0000","last_unstale":"2024-09-12T03:01:08.398498+0000","last_undegraded":"2024-09-12T03:01:08.398498+0000","last_fullsized":"2024-09-12T03:01:08.398498+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:48.750657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344703+0000","last_change":"2024-09-12T03:00:58.357874+0000","last_active":"2024-09-12T03:00:59.344703+0000","last_peered":"2024-09-12T03:00:59.344703+0000","last_clean":"2024-09-12T03:00:59.344703+0000","last_became_active":"2024-09-12T03:00:58.357575+0000","last_became_peered":"2024-09-12T03:00:58.357575+0000","last_unstale":"2024-09-12T03:00:59.344703+0000","last_undegraded":"2024-09-12T03:00:59.344703+0000","last_fullsized":"2024-09-12T03:00:59.344703+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:46.125620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.283498+0000","last_change":"2024-09-12T03:00:56.312263+0000","last_active":"2024-09-12T03:02:03.283498+0000","last_peered":"2024-09-12T03:02:03.283498+0000","last_clean":"2024-09-12T03:02:03.283498+0000","last_became_active":"2024-09-12T03:00:56.311894+0000","last_became_peered":"2024-09-12T03:00:56.311894+0000","last_unstale":"2024-09-12T03:02:03.283498+0000","last_undegraded":"2024-09-12T03:02:03.283498+0000","last_fullsized":"2024-09-12T03:02:03.283498+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:52.434203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.330287+0000","last_change":"2024-09-12T03:00:58.350762+0000","last_active":"2024-09-12T03:00:59.330287+0000","last_peered":"2024-09-12T03:00:59.330287+0000","last_clean":"2024-09-12T03:00:59.330287+0000","last_became_active":"2024-09-12T03:00:58.348038+0000","last_became_peered":"2024-09-12T03:00:58.348038+0000","last_unstale":"2024-09-12T03:00:59.330287+0000","last_undegraded":"2024-09-12T03:00:59.330287+0000","last_fullsized":"2024-09-12T03:00:59.330287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:08.916636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":75,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:59.475763+0000","last_change":"2024-09-12T03:00:56.435641+0000","last_active":"2024-09-12T03:01:59.475763+0000","last_peered":"2024-09-12T03:01:59.475763+0000","last_clean":"2024-09-12T03:01:59.475763+0000","last_became_active":"2024-09-12T03:00:56.435319+0000","last_became_peered":"2024-09-12T03:00:56.435319+0000","last_unstale":"2024-09-12T03:01:59.475763+0000","last_undegraded":"2024-09-12T03:01:59.475763+0000","last_fullsized":"2024-09-12T03:01:59.475763+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:56.894978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.280141+0000","last_change":"2024-09-12T03:00:58.344708+0000","last_active":"2024-09-12T03:02:03.280141+0000","last_peered":"2024-09-12T03:02:03.280141+0000","last_clean":"2024-09-12T03:02:03.280141+0000","last_became_active":"2024-09-12T03:00:58.344389+0000","last_became_peered":"2024-09-12T03:00:58.344389+0000","last_unstale":"2024-09-12T03:02:03.280141+0000","last_undegraded":"2024-09-12T03:02:03.280141+0000","last_fullsized":"2024-09-12T03:02:03.280141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:08.160383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.283775+0000","last_change":"2024-09-12T03:00:56.317068+0000","last_active":"2024-09-12T03:02:03.283775+0000","last_peered":"2024-09-12T03:02:03.283775+0000","last_clean":"2024-09-12T03:02:03.283775+0000","last_became_active":"2024-09-12T03:00:56.316698+0000","last_became_peered":"2024-09-12T03:00:56.316698+0000","last_unstale":"2024-09-12T03:02:03.283775+0000","last_undegraded":"2024-09-12T03:02:03.283775+0000","last_fullsized":"2024-09-12T03:02:03.283775+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:41:19.199418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343531+0000","last_change":"2024-09-12T03:00:58.347814+0000","last_active":"2024-09-12T03:00:59.343531+0000","last_peered":"2024-09-12T03:00:59.343531+0000","last_clean":"2024-09-12T03:00:59.343531+0000","last_became_active":"2024-09-12T03:00:58.347112+0000","last_became_peered":"2024-09-12T03:00:58.347112+0000","last_unstale":"2024-09-12T03:00:59.343531+0000","last_undegraded":"2024-09-12T03:00:59.343531+0000","last_fullsized":"2024-09-12T03:00:59.343531+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:59.325113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342760+0000","last_change":"2024-09-12T03:00:56.435771+0000","last_active":"2024-09-12T03:00:59.342760+0000","last_peered":"2024-09-12T03:00:59.342760+0000","last_clean":"2024-09-12T03:00:59.342760+0000","last_became_active":"2024-09-12T03:00:56.435448+0000","last_became_peered":"2024-09-12T03:00:56.435448+0000","last_unstale":"2024-09-12T03:00:59.342760+0000","last_undegraded":"2024-09-12T03:00:59.342760+0000","last_fullsized":"2024-09-12T03:00:59.342760+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:33.650338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:01.118993+0000","last_change":"2024-09-12T03:00:58.341071+0000","last_active":"2024-09-12T03:01:01.118993+0000","last_peered":"2024-09-12T03:01:01.118993+0000","last_clean":"2024-09-12T03:01:01.118993+0000","last_became_active":"2024-09-12T03:00:58.340753+0000","last_became_peered":"2024-09-12T03:00:58.340753+0000","last_unstale":"2024-09-12T03:01:01.118993+0000","last_undegraded":"2024-09-12T03:01:01.118993+0000","last_fullsized":"2024-09-12T03:01:01.118993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:21.217915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:06.388962+0000","last_change":"2024-09-12T03:00:56.436058+0000","last_active":"2024-09-12T03:01:06.388962+0000","last_peered":"2024-09-12T03:01:06.388962+0000","last_clean":"2024-09-12T03:01:06.388962+0000","last_became_active":"2024-09-12T03:00:56.435518+0000","last_became_peered":"2024-09-12T03:00:56.435518+0000","last_unstale":"2024-09-12T03:01:06.388962+0000","last_undegraded":"2024-09-12T03:01:06.388962+0000","last_fullsized":"2024-09-12T03:01:06.388962+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:32.235097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.330626+0000","last_change":"2024-09-12T03:00:58.348236+0000","last_active":"2024-09-12T03:00:59.330626+0000","last_peered":"2024-09-12T03:00:59.330626+0000","last_clean":"2024-09-12T03:00:59.330626+0000","last_became_active":"2024-09-12T03:00:58.347727+0000","last_became_peered":"2024-09-12T03:00:58.347727+0000","last_unstale":"2024-09-12T03:00:59.330626+0000","last_undegraded":"2024-09-12T03:00:59.330626+0000","last_fullsized":"2024-09-12T03:00:59.330626+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:19.233163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'10","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:23.344906+0000","last_change":"2024-09-12T03:00:56.310537+0000","last_active":"2024-09-12T03:01:23.344906+0000","last_peered":"2024-09-12T03:01:23.344906+0000","last_clean":"2024-09-12T03:01:23.344906+0000","last_became_active":"2024-09-12T03:00:56.310135+0000","last_became_peered":"2024-09-12T03:00:56.310135+0000","last_unstale":"2024-09-12T03:01:23.344906+0000","last_undegraded":"2024-09-12T03:01:23.344906+0000","last_fullsized":"2024-09-12T03:01:23.344906+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:10:55.584339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344075+0000","last_change":"2024-09-12T03:00:58.358417+0000","last_active":"2024-09-12T03:00:59.344075+0000","last_peered":"2024-09-12T03:00:59.344075+0000","last_clean":"2024-09-12T03:00:59.344075+0000","last_became_active":"2024-09-12T03:00:58.358119+0000","last_became_peered":"2024-09-12T03:00:58.358119+0000","last_unstale":"2024-09-12T03:00:59.344075+0000","last_undegraded":"2024-09-12T03:00:59.344075+0000","last_fullsized":"2024-09-12T03:00:59.344075+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:10.897594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:06.388934+0000","last_change":"2024-09-12T03:00:56.747044+0000","last_active":"2024-09-12T03:01:06.388934+0000","last_peered":"2024-09-12T03:01:06.388934+0000","last_clean":"2024-09-12T03:01:06.388934+0000","last_became_active":"2024-09-12T03:00:56.746699+0000","last_became_peered":"2024-09-12T03:00:56.746699+0000","last_unstale":"2024-09-12T03:01:06.388934+0000","last_undegraded":"2024-09-12T03:01:06.388934+0000","last_fullsized":"2024-09-12T03:01:06.388934+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:11.420109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342687+0000","last_change":"2024-09-12T03:00:58.346854+0000","last_active":"2024-09-12T03:00:59.342687+0000","last_peered":"2024-09-12T03:00:59.342687+0000","last_clean":"2024-09-12T03:00:59.342687+0000","last_became_active":"2024-09-12T03:00:58.346535+0000","last_became_peered":"2024-09-12T03:00:58.346535+0000","last_unstale":"2024-09-12T03:00:59.342687+0000","last_undegraded":"2024-09-12T03:00:59.342687+0000","last_fullsized":"2024-09-12T03:00:59.342687+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:46.637914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:06.389702+0000","last_change":"2024-09-12T03:00:56.435957+0000","last_active":"2024-09-12T03:01:06.389702+0000","last_peered":"2024-09-12T03:01:06.389702+0000","last_clean":"2024-09-12T03:01:06.389702+0000","last_became_active":"2024-09-12T03:00:56.435603+0000","last_became_peered":"2024-09-12T03:00:56.435603+0000","last_unstale":"2024-09-12T03:01:06.389702+0000","last_undegraded":"2024-09-12T03:01:06.389702+0000","last_fullsized":"2024-09-12T03:01:06.389702+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:34.553189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343477+0000","last_change":"2024-09-12T03:00:58.337784+0000","last_active":"2024-09-12T03:00:59.343477+0000","last_peered":"2024-09-12T03:00:59.343477+0000","last_clean":"2024-09-12T03:00:59.343477+0000","last_became_active":"2024-09-12T03:00:58.337415+0000","last_became_peered":"2024-09-12T03:00:58.337415+0000","last_unstale":"2024-09-12T03:00:59.343477+0000","last_undegraded":"2024-09-12T03:00:59.343477+0000","last_fullsized":"2024-09-12T03:00:59.343477+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:38:15.557067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:06.390162+0000","last_change":"2024-09-12T03:00:56.315356+0000","last_active":"2024-09-12T03:01:06.390162+0000","last_peered":"2024-09-12T03:01:06.390162+0000","last_clean":"2024-09-12T03:01:06.390162+0000","last_became_active":"2024-09-12T03:00:56.314873+0000","last_became_peered":"2024-09-12T03:00:56.314873+0000","last_unstale":"2024-09-12T03:01:06.390162+0000","last_undegraded":"2024-09-12T03:01:06.390162+0000","last_fullsized":"2024-09-12T03:01:06.390162+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:55.710156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.348074+0000","last_change":"2024-09-12T03:00:58.346706+0000","last_active":"2024-09-12T03:00:59.348074+0000","last_peered":"2024-09-12T03:00:59.348074+0000","last_clean":"2024-09-12T03:00:59.348074+0000","last_became_active":"2024-09-12T03:00:58.346363+0000","last_became_peered":"2024-09-12T03:00:58.346363+0000","last_unstale":"2024-09-12T03:00:59.348074+0000","last_undegraded":"2024-09-12T03:00:59.348074+0000","last_fullsized":"2024-09-12T03:00:59.348074+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:13.468639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343312+0000","last_change":"2024-09-12T03:00:56.314659+0000","last_active":"2024-09-12T03:00:59.343312+0000","last_peered":"2024-09-12T03:00:59.343312+0000","last_clean":"2024-09-12T03:00:59.343312+0000","last_became_active":"2024-09-12T03:00:56.314316+0000","last_became_peered":"2024-09-12T03:00:56.314316+0000","last_unstale":"2024-09-12T03:00:59.343312+0000","last_undegraded":"2024-09-12T03:00:59.343312+0000","last_fullsized":"2024-09-12T03:00:59.343312+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:26:23.883066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.345830+0000","last_change":"2024-09-12T03:00:58.356682+0000","last_active":"2024-09-12T03:00:59.345830+0000","last_peered":"2024-09-12T03:00:59.345830+0000","last_clean":"2024-09-12T03:00:59.345830+0000","last_became_active":"2024-09-12T03:00:58.356383+0000","last_became_peered":"2024-09-12T03:00:58.356383+0000","last_unstale":"2024-09-12T03:00:59.345830+0000","last_undegraded":"2024-09-12T03:00:59.345830+0000","last_fullsized":"2024-09-12T03:00:59.345830+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:14:16.428393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:06.390865+0000","last_change":"2024-09-12T03:00:56.314943+0000","last_active":"2024-09-12T03:01:06.390865+0000","last_peered":"2024-09-12T03:01:06.390865+0000","last_clean":"2024-09-12T03:01:06.390865+0000","last_became_active":"2024-09-12T03:00:56.314602+0000","last_became_peered":"2024-09-12T03:00:56.314602+0000","last_unstale":"2024-09-12T03:01:06.390865+0000","last_undegraded":"2024-09-12T03:01:06.390865+0000","last_fullsized":"2024-09-12T03:01:06.390865+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:31.207880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.283651+0000","last_change":"2024-09-12T03:00:58.351774+0000","last_active":"2024-09-12T03:02:03.283651+0000","last_peered":"2024-09-12T03:02:03.283651+0000","last_clean":"2024-09-12T03:02:03.283651+0000","last_became_active":"2024-09-12T03:00:58.351453+0000","last_became_peered":"2024-09-12T03:00:58.351453+0000","last_unstale":"2024-09-12T03:02:03.283651+0000","last_undegraded":"2024-09-12T03:02:03.283651+0000","last_fullsized":"2024-09-12T03:02:03.283651+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:18:24.260110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.281997+0000","last_change":"2024-09-12T03:00:58.346886+0000","last_active":"2024-09-12T03:02:03.281997+0000","last_peered":"2024-09-12T03:02:03.281997+0000","last_clean":"2024-09-12T03:02:03.281997+0000","last_became_active":"2024-09-12T03:00:58.346499+0000","last_became_peered":"2024-09-12T03:00:58.346499+0000","last_unstale":"2024-09-12T03:02:03.281997+0000","last_undegraded":"2024-09-12T03:02:03.281997+0000","last_fullsized":"2024-09-12T03:02:03.281997+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:02:24.045610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.350709+0000","last_change":"2024-09-12T03:00:56.317443+0000","last_active":"2024-09-12T03:00:59.350709+0000","last_peered":"2024-09-12T03:00:59.350709+0000","last_clean":"2024-09-12T03:00:59.350709+0000","last_became_active":"2024-09-12T03:00:56.317045+0000","last_became_peered":"2024-09-12T03:00:56.317045+0000","last_unstale":"2024-09-12T03:00:59.350709+0000","last_undegraded":"2024-09-12T03:00:59.350709+0000","last_fullsized":"2024-09-12T03:00:59.350709+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:07.687606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328499+0000","last_change":"2024-09-12T03:00:58.337257+0000","last_active":"2024-09-12T03:00:59.328499+0000","last_peered":"2024-09-12T03:00:59.328499+0000","last_clean":"2024-09-12T03:00:59.328499+0000","last_became_active":"2024-09-12T03:00:58.336847+0000","last_became_peered":"2024-09-12T03:00:58.336847+0000","last_unstale":"2024-09-12T03:00:59.328499+0000","last_undegraded":"2024-09-12T03:00:59.328499+0000","last_fullsized":"2024-09-12T03:00:59.328499+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:22.846206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.398760+0000","last_change":"2024-09-12T03:00:56.316255+0000","last_active":"2024-09-12T03:01:07.398760+0000","last_peered":"2024-09-12T03:01:07.398760+0000","last_clean":"2024-09-12T03:01:07.398760+0000","last_became_active":"2024-09-12T03:00:56.315906+0000","last_became_peered":"2024-09-12T03:00:56.315906+0000","last_unstale":"2024-09-12T03:01:07.398760+0000","last_undegraded":"2024-09-12T03:01:07.398760+0000","last_fullsized":"2024-09-12T03:01:07.398760+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:14.798601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342990+0000","last_change":"2024-09-12T03:00:58.347206+0000","last_active":"2024-09-12T03:00:59.342990+0000","last_peered":"2024-09-12T03:00:59.342990+0000","last_clean":"2024-09-12T03:00:59.342990+0000","last_became_active":"2024-09-12T03:00:58.346923+0000","last_became_peered":"2024-09-12T03:00:58.346923+0000","last_unstale":"2024-09-12T03:00:59.342990+0000","last_undegraded":"2024-09-12T03:00:59.342990+0000","last_fullsized":"2024-09-12T03:00:59.342990+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:41.376211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.347646+0000","last_change":"2024-09-12T03:00:56.435431+0000","last_active":"2024-09-12T03:00:59.347646+0000","last_peered":"2024-09-12T03:00:59.347646+0000","last_clean":"2024-09-12T03:00:59.347646+0000","last_became_active":"2024-09-12T03:00:56.435088+0000","last_became_peered":"2024-09-12T03:00:56.435088+0000","last_unstale":"2024-09-12T03:00:59.347646+0000","last_undegraded":"2024-09-12T03:00:59.347646+0000","last_fullsized":"2024-09-12T03:00:59.347646+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:24.738340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343937+0000","last_change":"2024-09-12T03:00:58.351737+0000","last_active":"2024-09-12T03:00:59.343937+0000","last_peered":"2024-09-12T03:00:59.343937+0000","last_clean":"2024-09-12T03:00:59.343937+0000","last_became_active":"2024-09-12T03:00:58.351407+0000","last_became_peered":"2024-09-12T03:00:58.351407+0000","last_unstale":"2024-09-12T03:00:59.343937+0000","last_undegraded":"2024-09-12T03:00:59.343937+0000","last_fullsized":"2024-09-12T03:00:59.343937+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:34.206703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.399005+0000","last_change":"2024-09-12T03:00:56.310910+0000","last_active":"2024-09-12T03:01:07.399005+0000","last_peered":"2024-09-12T03:01:07.399005+0000","last_clean":"2024-09-12T03:01:07.399005+0000","last_became_active":"2024-09-12T03:00:56.310657+0000","last_became_peered":"2024-09-12T03:00:56.310657+0000","last_unstale":"2024-09-12T03:01:07.399005+0000","last_undegraded":"2024-09-12T03:01:07.399005+0000","last_fullsized":"2024-09-12T03:01:07.399005+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:06.117215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65362,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":347,"num_read_kb":241,"num_write":161,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":196086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245899,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645339,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":414814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906975,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":1068047,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070501,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":415050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299434,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":868352,"data_stored":414970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528367,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":1006509,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":40,"seq":171798691892,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":856064,"data_stored":414439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":33,"seq":141733920825,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":1006193,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051646,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":860160,"data_stored":414876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280579,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":414750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476808,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":942080,"data_stored":477334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607630,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":475984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":876544,"data_stored":415710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:02:05.692 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:02:05.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:05 smithi183 ceph-mon[35535]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T03:02:05.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:05 smithi183 ceph-mon[35535]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:02:05.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:05 smithi183 ceph-mon[35535]: pgmap v311: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2024-09-12T03:02:05.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:05 smithi177 ceph-mon[35305]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T03:02:05.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:05 smithi177 ceph-mon[35305]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:02:05.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:05 smithi177 ceph-mon[35305]: pgmap v311: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2024-09-12T03:02:05.958 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:06.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:06 smithi116 ceph-mon[31410]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-12T03:02:06.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:06 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:02:06.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:06 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:02:06.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:06 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:06.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:06 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:02:06.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:06 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:02:06.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.482+0000 7f9c15252640 1 -- 172.21.15.116:0/829507738 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c10102ef0 msgr2=0x7f9c1010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:06.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.482+0000 7f9c15252640 1 --2- 172.21.15.116:0/829507738 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c10102ef0 0x7f9c1010da40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9c08009ca0 tx=0x7f9c0802a950 comp rx=0 tx=0).stop 2024-09-12T03:02:06.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.482+0000 7f9c15252640 1 -- 172.21.15.116:0/829507738 shutdown_connections 2024-09-12T03:02:06.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.482+0000 7f9c15252640 1 --2- 172.21.15.116:0/829507738 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c10102ef0 0x7f9c1010da40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.482+0000 7f9c15252640 1 --2- 172.21.15.116:0/829507738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c10102570 0x7f9c101029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.482+0000 7f9c15252640 1 --2- 172.21.15.116:0/829507738 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c10100e20 0x7f9c10101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.482+0000 7f9c15252640 1 -- 172.21.15.116:0/829507738 >> 172.21.15.116:0/829507738 conn(0x7f9c100fc7d0 msgr2=0x7f9c100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:06.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.482+0000 7f9c15252640 1 -- 172.21.15.116:0/829507738 shutdown_connections 2024-09-12T03:02:06.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.482+0000 7f9c15252640 1 -- 172.21.15.116:0/829507738 wait complete. 2024-09-12T03:02:06.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.482+0000 7f9c15252640 1 Processor -- start 2024-09-12T03:02:06.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.483+0000 7f9c15252640 1 -- start start 2024-09-12T03:02:06.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.483+0000 7f9c15252640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c10100e20 0x7f9c101984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:06.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.483+0000 7f9c15252640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c10102570 0x7f9c10198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:06.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.483+0000 7f9c15252640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c10102ef0 0x7f9c1019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:06.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.483+0000 7f9c15252640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9c10110fa0 con 0x7f9c10102ef0 2024-09-12T03:02:06.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.483+0000 7f9c15252640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9c10110e20 con 0x7f9c10102570 2024-09-12T03:02:06.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.483+0000 7f9c15252640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9c10111120 con 0x7f9c10100e20 2024-09-12T03:02:06.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.483+0000 7f9c0f59e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c10102ef0 0x7f9c1019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:06.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.483+0000 7f9c0f59e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c10102ef0 0x7f9c1019cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60154/0 (socket says 172.21.15.116:60154) 2024-09-12T03:02:06.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.483+0000 7f9c0f59e640 1 -- 172.21.15.116:0/2412810762 learned_addr learned my addr 172.21.15.116:0/2412810762 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:06.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.483+0000 7f9c0ed9d640 1 --2- 172.21.15.116:0/2412810762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c10100e20 0x7f9c101984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:06.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.483+0000 7f9c0e59c640 1 --2- 172.21.15.116:0/2412810762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c10102570 0x7f9c10198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:06.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.483+0000 7f9c0f59e640 1 -- 172.21.15.116:0/2412810762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c10100e20 msgr2=0x7f9c101984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:06.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.483+0000 7f9c0f59e640 1 --2- 172.21.15.116:0/2412810762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c10100e20 0x7f9c101984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.483+0000 7f9c0f59e640 1 -- 172.21.15.116:0/2412810762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c10102570 msgr2=0x7f9c10198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:06.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.483+0000 7f9c0f59e640 1 --2- 172.21.15.116:0/2412810762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c10102570 0x7f9c10198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.483+0000 7f9c0f59e640 1 -- 172.21.15.116:0/2412810762 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1019d480 con 0x7f9c10102ef0 2024-09-12T03:02:06.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.483+0000 7f9c0e59c640 1 --2- 172.21.15.116:0/2412810762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c10102570 0x7f9c10198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:06.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.484+0000 7f9c0f59e640 1 --2- 172.21.15.116:0/2412810762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c10102ef0 0x7f9c1019cda0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9c08006010 tx=0x7f9c0802f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:06.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.484+0000 7f9bf7fff640 1 -- 172.21.15.116:0/2412810762 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08034040 con 0x7f9c10102ef0 2024-09-12T03:02:06.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.484+0000 7f9c15252640 1 -- 172.21.15.116:0/2412810762 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c1019d710 con 0x7f9c10102ef0 2024-09-12T03:02:06.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.484+0000 7f9bf7fff640 1 -- 172.21.15.116:0/2412810762 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c08003e00 con 0x7f9c10102ef0 2024-09-12T03:02:06.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.484+0000 7f9bf7fff640 1 -- 172.21.15.116:0/2412810762 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08045480 con 0x7f9c10102ef0 2024-09-12T03:02:06.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.484+0000 7f9c15252640 1 -- 172.21.15.116:0/2412810762 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c101a5050 con 0x7f9c10102ef0 2024-09-12T03:02:06.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.485+0000 7f9c15252640 1 -- 172.21.15.116:0/2412810762 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c10101200 con 0x7f9c10102ef0 2024-09-12T03:02:06.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.486+0000 7f9bf7fff640 1 -- 172.21.15.116:0/2412810762 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f9c08037020 con 0x7f9c10102ef0 2024-09-12T03:02:06.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.486+0000 7f9bf7fff640 1 --2- 172.21.15.116:0/2412810762 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9bf80a2c90 0x7f9bf80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:06.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.487+0000 7f9bf7fff640 1 -- 172.21.15.116:0/2412810762 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f9c080f0120 con 0x7f9c10102ef0 2024-09-12T03:02:06.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.487+0000 7f9c0ed9d640 1 --2- 172.21.15.116:0/2412810762 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9bf80a2c90 0x7f9bf80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:06.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.487+0000 7f9c0ed9d640 1 --2- 172.21.15.116:0/2412810762 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9bf80a2c90 0x7f9bf80a5150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000abe0 tx=0x7f9c0000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:06.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.490+0000 7f9bf7fff640 1 -- 172.21.15.116:0/2412810762 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9c080bd060 con 0x7f9c10102ef0 2024-09-12T03:02:06.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.654+0000 7f9c15252640 1 -- 172.21.15.116:0/2412810762 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9c10062630 con 0x7f9bf80a2c90 2024-09-12T03:02:06.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.661+0000 7f9bf7fff640 1 -- 172.21.15.116:0/2412810762 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258764 (secure 0 0 0) 0x7f9c10062630 con 0x7f9bf80a2c90 2024-09-12T03:02:06.664 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:02:06.665 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-09-12T03:02:06.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.663+0000 7f9c15252640 1 -- 172.21.15.116:0/2412810762 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9bf80a2c90 msgr2=0x7f9bf80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:06.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.663+0000 7f9c15252640 1 --2- 172.21.15.116:0/2412810762 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9bf80a2c90 0x7f9bf80a5150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000abe0 tx=0x7f9c0000a2d0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.663+0000 7f9c15252640 1 -- 172.21.15.116:0/2412810762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c10102ef0 msgr2=0x7f9c1019cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:06.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.663+0000 7f9c15252640 1 --2- 172.21.15.116:0/2412810762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c10102ef0 0x7f9c1019cda0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9c08006010 tx=0x7f9c0802f6f0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.663+0000 7f9c15252640 1 -- 172.21.15.116:0/2412810762 shutdown_connections 2024-09-12T03:02:06.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.663+0000 7f9c15252640 1 --2- 172.21.15.116:0/2412810762 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9bf80a2c90 0x7f9bf80a5150 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.663+0000 7f9c15252640 1 --2- 172.21.15.116:0/2412810762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c10102ef0 0x7f9c1019cda0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.663+0000 7f9c15252640 1 --2- 172.21.15.116:0/2412810762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c10102570 0x7f9c10198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.663+0000 7f9c15252640 1 --2- 172.21.15.116:0/2412810762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c10100e20 0x7f9c101984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.663+0000 7f9c15252640 1 -- 172.21.15.116:0/2412810762 >> 172.21.15.116:0/2412810762 conn(0x7f9c100fc7d0 msgr2=0x7f9c1006a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:06.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.663+0000 7f9c15252640 1 -- 172.21.15.116:0/2412810762 shutdown_connections 2024-09-12T03:02:06.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:06.663+0000 7f9c15252640 1 -- 172.21.15.116:0/2412810762 wait complete. 2024-09-12T03:02:06.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:06 smithi183 ceph-mon[35535]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-12T03:02:06.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:06 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:02:06.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:06 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:02:06.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:06 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:06.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:06 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:02:06.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:06 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:02:06.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:06 smithi177 ceph-mon[35305]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-12T03:02:06.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:06 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:02:06.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:06 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:02:06.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:06 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:06.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:06 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:02:06.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:06 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:02:07.444 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:07 smithi116 ceph-mon[31410]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:02:07.444 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:07 smithi116 ceph-mon[31410]: pgmap v312: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:02:07.474 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-09-12T03:02:05.123067+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655964,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":478,"num_read_kb":350,"num_write":344,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368096,"kb_used_data":10524,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573728,"statfs":{"total":1151940427776,"available":1151563497472,"internally_reserved":0,"allocated":10776576,"data_stored":6947261,"data_compressed":107490,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002159"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.348943+0000","last_change":"2024-09-12T03:00:58.348563+0000","last_active":"2024-09-12T03:00:59.348943+0000","last_peered":"2024-09-12T03:00:59.348943+0000","last_clean":"2024-09-12T03:00:59.348943+0000","last_became_active":"2024-09-12T03:00:58.347940+0000","last_became_peered":"2024-09-12T03:00:58.347940+0000","last_unstale":"2024-09-12T03:00:59.348943+0000","last_undegraded":"2024-09-12T03:00:59.348943+0000","last_fullsized":"2024-09-12T03:00:59.348943+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:43.489238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342719+0000","last_change":"2024-09-12T03:00:58.351969+0000","last_active":"2024-09-12T03:00:59.342719+0000","last_peered":"2024-09-12T03:00:59.342719+0000","last_clean":"2024-09-12T03:00:59.342719+0000","last_became_active":"2024-09-12T03:00:58.351679+0000","last_became_peered":"2024-09-12T03:00:58.351679+0000","last_unstale":"2024-09-12T03:00:59.342719+0000","last_undegraded":"2024-09-12T03:00:59.342719+0000","last_fullsized":"2024-09-12T03:00:59.342719+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:27:01.174890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342541+0000","last_change":"2024-09-12T03:00:58.346045+0000","last_active":"2024-09-12T03:00:59.342541+0000","last_peered":"2024-09-12T03:00:59.342541+0000","last_clean":"2024-09-12T03:00:59.342541+0000","last_became_active":"2024-09-12T03:00:58.345719+0000","last_became_peered":"2024-09-12T03:00:58.345719+0000","last_unstale":"2024-09-12T03:00:59.342541+0000","last_undegraded":"2024-09-12T03:00:59.342541+0000","last_fullsized":"2024-09-12T03:00:59.342541+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:12.312650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289352+0000","last_change":"2024-09-12T03:00:58.348165+0000","last_active":"2024-09-12T03:02:04.289352+0000","last_peered":"2024-09-12T03:02:04.289352+0000","last_clean":"2024-09-12T03:02:04.289352+0000","last_became_active":"2024-09-12T03:00:58.347734+0000","last_became_peered":"2024-09-12T03:00:58.347734+0000","last_unstale":"2024-09-12T03:02:04.289352+0000","last_undegraded":"2024-09-12T03:02:04.289352+0000","last_fullsized":"2024-09-12T03:02:04.289352+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:05.945900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343877+0000","last_change":"2024-09-12T03:00:58.339316+0000","last_active":"2024-09-12T03:00:59.343877+0000","last_peered":"2024-09-12T03:00:59.343877+0000","last_clean":"2024-09-12T03:00:59.343877+0000","last_became_active":"2024-09-12T03:00:58.338964+0000","last_became_peered":"2024-09-12T03:00:58.338964+0000","last_unstale":"2024-09-12T03:00:59.343877+0000","last_undegraded":"2024-09-12T03:00:59.343877+0000","last_fullsized":"2024-09-12T03:00:59.343877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:50.721276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342586+0000","last_change":"2024-09-12T03:00:58.346838+0000","last_active":"2024-09-12T03:00:59.342586+0000","last_peered":"2024-09-12T03:00:59.342586+0000","last_clean":"2024-09-12T03:00:59.342586+0000","last_became_active":"2024-09-12T03:00:58.346486+0000","last_became_peered":"2024-09-12T03:00:58.346486+0000","last_unstale":"2024-09-12T03:00:59.342586+0000","last_undegraded":"2024-09-12T03:00:59.342586+0000","last_fullsized":"2024-09-12T03:00:59.342586+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:28.687727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.283454+0000","last_change":"2024-09-12T03:00:58.346712+0000","last_active":"2024-09-12T03:02:03.283454+0000","last_peered":"2024-09-12T03:02:03.283454+0000","last_clean":"2024-09-12T03:02:03.283454+0000","last_became_active":"2024-09-12T03:00:58.346397+0000","last_became_peered":"2024-09-12T03:00:58.346397+0000","last_unstale":"2024-09-12T03:02:03.283454+0000","last_undegraded":"2024-09-12T03:02:03.283454+0000","last_fullsized":"2024-09-12T03:02:03.283454+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:48:00.624733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343665+0000","last_change":"2024-09-12T03:00:58.356166+0000","last_active":"2024-09-12T03:00:59.343665+0000","last_peered":"2024-09-12T03:00:59.343665+0000","last_clean":"2024-09-12T03:00:59.343665+0000","last_became_active":"2024-09-12T03:00:58.354944+0000","last_became_peered":"2024-09-12T03:00:58.354944+0000","last_unstale":"2024-09-12T03:00:59.343665+0000","last_undegraded":"2024-09-12T03:00:59.343665+0000","last_fullsized":"2024-09-12T03:00:59.343665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:15.840149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.329171+0000","last_change":"2024-09-12T03:00:58.340182+0000","last_active":"2024-09-12T03:00:59.329171+0000","last_peered":"2024-09-12T03:00:59.329171+0000","last_clean":"2024-09-12T03:00:59.329171+0000","last_became_active":"2024-09-12T03:00:58.339576+0000","last_became_peered":"2024-09-12T03:00:58.339576+0000","last_unstale":"2024-09-12T03:00:59.329171+0000","last_undegraded":"2024-09-12T03:00:59.329171+0000","last_fullsized":"2024-09-12T03:00:59.329171+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:16.534844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344683+0000","last_change":"2024-09-12T03:00:58.351218+0000","last_active":"2024-09-12T03:00:59.344683+0000","last_peered":"2024-09-12T03:00:59.344683+0000","last_clean":"2024-09-12T03:00:59.344683+0000","last_became_active":"2024-09-12T03:00:58.350871+0000","last_became_peered":"2024-09-12T03:00:58.350871+0000","last_unstale":"2024-09-12T03:00:59.344683+0000","last_undegraded":"2024-09-12T03:00:59.344683+0000","last_fullsized":"2024-09-12T03:00:59.344683+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:38.367683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.283798+0000","last_change":"2024-09-12T03:00:58.351894+0000","last_active":"2024-09-12T03:02:03.283798+0000","last_peered":"2024-09-12T03:02:03.283798+0000","last_clean":"2024-09-12T03:02:03.283798+0000","last_became_active":"2024-09-12T03:00:58.351445+0000","last_became_peered":"2024-09-12T03:00:58.351445+0000","last_unstale":"2024-09-12T03:02:03.283798+0000","last_undegraded":"2024-09-12T03:02:03.283798+0000","last_fullsized":"2024-09-12T03:02:03.283798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:42.093871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289589+0000","last_change":"2024-09-12T03:00:58.353460+0000","last_active":"2024-09-12T03:02:04.289589+0000","last_peered":"2024-09-12T03:02:04.289589+0000","last_clean":"2024-09-12T03:02:04.289589+0000","last_became_active":"2024-09-12T03:00:58.353111+0000","last_became_peered":"2024-09-12T03:00:58.353111+0000","last_unstale":"2024-09-12T03:02:04.289589+0000","last_undegraded":"2024-09-12T03:02:04.289589+0000","last_fullsized":"2024-09-12T03:02:04.289589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:15.916765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.330518+0000","last_change":"2024-09-12T03:00:58.341080+0000","last_active":"2024-09-12T03:00:59.330518+0000","last_peered":"2024-09-12T03:00:59.330518+0000","last_clean":"2024-09-12T03:00:59.330518+0000","last_became_active":"2024-09-12T03:00:58.340782+0000","last_became_peered":"2024-09-12T03:00:58.340782+0000","last_unstale":"2024-09-12T03:00:59.330518+0000","last_undegraded":"2024-09-12T03:00:59.330518+0000","last_fullsized":"2024-09-12T03:00:59.330518+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:36.569700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342701+0000","last_change":"2024-09-12T03:00:58.346791+0000","last_active":"2024-09-12T03:00:59.342701+0000","last_peered":"2024-09-12T03:00:59.342701+0000","last_clean":"2024-09-12T03:00:59.342701+0000","last_became_active":"2024-09-12T03:00:58.346500+0000","last_became_peered":"2024-09-12T03:00:58.346500+0000","last_unstale":"2024-09-12T03:00:59.342701+0000","last_undegraded":"2024-09-12T03:00:59.342701+0000","last_fullsized":"2024-09-12T03:00:59.342701+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:32.017589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342415+0000","last_change":"2024-09-12T03:00:58.349522+0000","last_active":"2024-09-12T03:00:59.342415+0000","last_peered":"2024-09-12T03:00:59.342415+0000","last_clean":"2024-09-12T03:00:59.342415+0000","last_became_active":"2024-09-12T03:00:58.349246+0000","last_became_peered":"2024-09-12T03:00:58.349246+0000","last_unstale":"2024-09-12T03:00:59.342415+0000","last_undegraded":"2024-09-12T03:00:59.342415+0000","last_fullsized":"2024-09-12T03:00:59.342415+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:39.253919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328399+0000","last_change":"2024-09-12T03:00:58.349909+0000","last_active":"2024-09-12T03:00:59.328399+0000","last_peered":"2024-09-12T03:00:59.328399+0000","last_clean":"2024-09-12T03:00:59.328399+0000","last_became_active":"2024-09-12T03:00:58.349590+0000","last_became_peered":"2024-09-12T03:00:58.349590+0000","last_unstale":"2024-09-12T03:00:59.328399+0000","last_undegraded":"2024-09-12T03:00:59.328399+0000","last_fullsized":"2024-09-12T03:00:59.328399+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:04.540709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.349104+0000","last_change":"2024-09-12T03:00:58.348284+0000","last_active":"2024-09-12T03:00:59.349104+0000","last_peered":"2024-09-12T03:00:59.349104+0000","last_clean":"2024-09-12T03:00:59.349104+0000","last_became_active":"2024-09-12T03:00:58.347465+0000","last_became_peered":"2024-09-12T03:00:58.347465+0000","last_unstale":"2024-09-12T03:00:59.349104+0000","last_undegraded":"2024-09-12T03:00:59.349104+0000","last_fullsized":"2024-09-12T03:00:59.349104+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:13.191899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343137+0000","last_change":"2024-09-12T03:00:58.337740+0000","last_active":"2024-09-12T03:00:59.343137+0000","last_peered":"2024-09-12T03:00:59.343137+0000","last_clean":"2024-09-12T03:00:59.343137+0000","last_became_active":"2024-09-12T03:00:58.337393+0000","last_became_peered":"2024-09-12T03:00:58.337393+0000","last_unstale":"2024-09-12T03:00:59.343137+0000","last_undegraded":"2024-09-12T03:00:59.343137+0000","last_fullsized":"2024-09-12T03:00:59.343137+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:28:56.339167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344750+0000","last_change":"2024-09-12T03:00:58.345779+0000","last_active":"2024-09-12T03:00:59.344750+0000","last_peered":"2024-09-12T03:00:59.344750+0000","last_clean":"2024-09-12T03:00:59.344750+0000","last_became_active":"2024-09-12T03:00:58.345210+0000","last_became_peered":"2024-09-12T03:00:58.345210+0000","last_unstale":"2024-09-12T03:00:59.344750+0000","last_undegraded":"2024-09-12T03:00:59.344750+0000","last_fullsized":"2024-09-12T03:00:59.344750+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:06:58.271185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.348332+0000","last_change":"2024-09-12T03:00:58.343877+0000","last_active":"2024-09-12T03:00:59.348332+0000","last_peered":"2024-09-12T03:00:59.348332+0000","last_clean":"2024-09-12T03:00:59.348332+0000","last_became_active":"2024-09-12T03:00:58.343545+0000","last_became_peered":"2024-09-12T03:00:58.343545+0000","last_unstale":"2024-09-12T03:00:59.348332+0000","last_undegraded":"2024-09-12T03:00:59.348332+0000","last_fullsized":"2024-09-12T03:00:59.348332+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:52:31.851975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.330596+0000","last_change":"2024-09-12T03:00:58.349233+0000","last_active":"2024-09-12T03:00:59.330596+0000","last_peered":"2024-09-12T03:00:59.330596+0000","last_clean":"2024-09-12T03:00:59.330596+0000","last_became_active":"2024-09-12T03:00:58.348958+0000","last_became_peered":"2024-09-12T03:00:58.348958+0000","last_unstale":"2024-09-12T03:00:59.330596+0000","last_undegraded":"2024-09-12T03:00:59.330596+0000","last_fullsized":"2024-09-12T03:00:59.330596+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:32:10.271817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328487+0000","last_change":"2024-09-12T03:00:58.358024+0000","last_active":"2024-09-12T03:00:59.328487+0000","last_peered":"2024-09-12T03:00:59.328487+0000","last_clean":"2024-09-12T03:00:59.328487+0000","last_became_active":"2024-09-12T03:00:58.357434+0000","last_became_peered":"2024-09-12T03:00:58.357434+0000","last_unstale":"2024-09-12T03:00:59.328487+0000","last_undegraded":"2024-09-12T03:00:59.328487+0000","last_fullsized":"2024-09-12T03:00:59.328487+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:01:20.119733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.345193+0000","last_change":"2024-09-12T03:00:58.355063+0000","last_active":"2024-09-12T03:00:59.345193+0000","last_peered":"2024-09-12T03:00:59.345193+0000","last_clean":"2024-09-12T03:00:59.345193+0000","last_became_active":"2024-09-12T03:00:58.354760+0000","last_became_peered":"2024-09-12T03:00:58.354760+0000","last_unstale":"2024-09-12T03:00:59.345193+0000","last_undegraded":"2024-09-12T03:00:59.345193+0000","last_fullsized":"2024-09-12T03:00:59.345193+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:13.464413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.280040+0000","last_change":"2024-09-12T03:00:58.341916+0000","last_active":"2024-09-12T03:02:03.280040+0000","last_peered":"2024-09-12T03:02:03.280040+0000","last_clean":"2024-09-12T03:02:03.280040+0000","last_became_active":"2024-09-12T03:00:58.340815+0000","last_became_peered":"2024-09-12T03:00:58.340815+0000","last_unstale":"2024-09-12T03:02:03.280040+0000","last_undegraded":"2024-09-12T03:02:03.280040+0000","last_fullsized":"2024-09-12T03:02:03.280040+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:13.777947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343925+0000","last_change":"2024-09-12T03:00:58.357765+0000","last_active":"2024-09-12T03:00:59.343925+0000","last_peered":"2024-09-12T03:00:59.343925+0000","last_clean":"2024-09-12T03:00:59.343925+0000","last_became_active":"2024-09-12T03:00:58.357480+0000","last_became_peered":"2024-09-12T03:00:58.357480+0000","last_unstale":"2024-09-12T03:00:59.343925+0000","last_undegraded":"2024-09-12T03:00:59.343925+0000","last_fullsized":"2024-09-12T03:00:59.343925+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:26.884381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344699+0000","last_change":"2024-09-12T03:00:58.349504+0000","last_active":"2024-09-12T03:00:59.344699+0000","last_peered":"2024-09-12T03:00:59.344699+0000","last_clean":"2024-09-12T03:00:59.344699+0000","last_became_active":"2024-09-12T03:00:58.349136+0000","last_became_peered":"2024-09-12T03:00:58.349136+0000","last_unstale":"2024-09-12T03:00:59.344699+0000","last_undegraded":"2024-09-12T03:00:59.344699+0000","last_fullsized":"2024-09-12T03:00:59.344699+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:09:55.229516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.348719+0000","last_change":"2024-09-12T03:00:58.341902+0000","last_active":"2024-09-12T03:00:59.348719+0000","last_peered":"2024-09-12T03:00:59.348719+0000","last_clean":"2024-09-12T03:00:59.348719+0000","last_became_active":"2024-09-12T03:00:58.341592+0000","last_became_peered":"2024-09-12T03:00:58.341592+0000","last_unstale":"2024-09-12T03:00:59.348719+0000","last_undegraded":"2024-09-12T03:00:59.348719+0000","last_fullsized":"2024-09-12T03:00:59.348719+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:13.852511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.330540+0000","last_change":"2024-09-12T03:00:58.348012+0000","last_active":"2024-09-12T03:00:59.330540+0000","last_peered":"2024-09-12T03:00:59.330540+0000","last_clean":"2024-09-12T03:00:59.330540+0000","last_became_active":"2024-09-12T03:00:58.347338+0000","last_became_peered":"2024-09-12T03:00:58.347338+0000","last_unstale":"2024-09-12T03:00:59.330540+0000","last_undegraded":"2024-09-12T03:00:59.330540+0000","last_fullsized":"2024-09-12T03:00:59.330540+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:41.478032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342928+0000","last_change":"2024-09-12T03:00:58.345833+0000","last_active":"2024-09-12T03:00:59.342928+0000","last_peered":"2024-09-12T03:00:59.342928+0000","last_clean":"2024-09-12T03:00:59.342928+0000","last_became_active":"2024-09-12T03:00:58.345482+0000","last_became_peered":"2024-09-12T03:00:58.345482+0000","last_unstale":"2024-09-12T03:00:59.342928+0000","last_undegraded":"2024-09-12T03:00:59.342928+0000","last_fullsized":"2024-09-12T03:00:59.342928+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:32.037412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343059+0000","last_change":"2024-09-12T03:00:58.352290+0000","last_active":"2024-09-12T03:00:59.343059+0000","last_peered":"2024-09-12T03:00:59.343059+0000","last_clean":"2024-09-12T03:00:59.343059+0000","last_became_active":"2024-09-12T03:00:58.351918+0000","last_became_peered":"2024-09-12T03:00:58.351918+0000","last_unstale":"2024-09-12T03:00:59.343059+0000","last_undegraded":"2024-09-12T03:00:59.343059+0000","last_fullsized":"2024-09-12T03:00:59.343059+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:25.941080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343279+0000","last_change":"2024-09-12T03:00:58.348021+0000","last_active":"2024-09-12T03:00:59.343279+0000","last_peered":"2024-09-12T03:00:59.343279+0000","last_clean":"2024-09-12T03:00:59.343279+0000","last_became_active":"2024-09-12T03:00:58.347634+0000","last_became_peered":"2024-09-12T03:00:58.347634+0000","last_unstale":"2024-09-12T03:00:59.343279+0000","last_undegraded":"2024-09-12T03:00:59.343279+0000","last_fullsized":"2024-09-12T03:00:59.343279+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:47.933067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.347866+0000","last_change":"2024-09-12T03:00:58.340891+0000","last_active":"2024-09-12T03:00:59.347866+0000","last_peered":"2024-09-12T03:00:59.347866+0000","last_clean":"2024-09-12T03:00:59.347866+0000","last_became_active":"2024-09-12T03:00:58.340504+0000","last_became_peered":"2024-09-12T03:00:58.340504+0000","last_unstale":"2024-09-12T03:00:59.347866+0000","last_undegraded":"2024-09-12T03:00:59.347866+0000","last_fullsized":"2024-09-12T03:00:59.347866+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:32.117635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343541+0000","last_change":"2024-09-12T03:00:58.339343+0000","last_active":"2024-09-12T03:00:59.343541+0000","last_peered":"2024-09-12T03:00:59.343541+0000","last_clean":"2024-09-12T03:00:59.343541+0000","last_became_active":"2024-09-12T03:00:58.339001+0000","last_became_peered":"2024-09-12T03:00:58.339001+0000","last_unstale":"2024-09-12T03:00:59.343541+0000","last_undegraded":"2024-09-12T03:00:59.343541+0000","last_fullsized":"2024-09-12T03:00:59.343541+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:02.043454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":83,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.281673+0000","last_change":"2024-09-12T03:00:56.315550+0000","last_active":"2024-09-12T03:02:03.281673+0000","last_peered":"2024-09-12T03:02:03.281673+0000","last_clean":"2024-09-12T03:02:03.281673+0000","last_became_active":"2024-09-12T03:00:56.315225+0000","last_became_peered":"2024-09-12T03:00:56.315225+0000","last_unstale":"2024-09-12T03:02:03.281673+0000","last_undegraded":"2024-09-12T03:02:03.281673+0000","last_fullsized":"2024-09-12T03:02:03.281673+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:46.070895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328646+0000","last_change":"2024-09-12T03:00:58.338379+0000","last_active":"2024-09-12T03:00:59.328646+0000","last_peered":"2024-09-12T03:00:59.328646+0000","last_clean":"2024-09-12T03:00:59.328646+0000","last_became_active":"2024-09-12T03:00:58.337795+0000","last_became_peered":"2024-09-12T03:00:58.337795+0000","last_unstale":"2024-09-12T03:00:59.328646+0000","last_undegraded":"2024-09-12T03:00:59.328646+0000","last_fullsized":"2024-09-12T03:00:59.328646+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:43.299329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.397617+0000","last_change":"2024-09-12T03:00:56.316385+0000","last_active":"2024-09-12T03:01:07.397617+0000","last_peered":"2024-09-12T03:01:07.397617+0000","last_clean":"2024-09-12T03:01:07.397617+0000","last_became_active":"2024-09-12T03:00:56.316064+0000","last_became_peered":"2024-09-12T03:00:56.316064+0000","last_unstale":"2024-09-12T03:01:07.397617+0000","last_undegraded":"2024-09-12T03:01:07.397617+0000","last_fullsized":"2024-09-12T03:01:07.397617+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:12:43.323654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.397886+0000","last_change":"2024-09-12T03:00:56.313065+0000","last_active":"2024-09-12T03:01:07.397886+0000","last_peered":"2024-09-12T03:01:07.397886+0000","last_clean":"2024-09-12T03:01:07.397886+0000","last_became_active":"2024-09-12T03:00:56.312716+0000","last_became_peered":"2024-09-12T03:00:56.312716+0000","last_unstale":"2024-09-12T03:01:07.397886+0000","last_undegraded":"2024-09-12T03:01:07.397886+0000","last_fullsized":"2024-09-12T03:01:07.397886+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:51.089089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.346221+0000","last_change":"2024-09-12T03:00:58.346542+0000","last_active":"2024-09-12T03:00:59.346221+0000","last_peered":"2024-09-12T03:00:59.346221+0000","last_clean":"2024-09-12T03:00:59.346221+0000","last_became_active":"2024-09-12T03:00:58.346260+0000","last_became_peered":"2024-09-12T03:00:58.346260+0000","last_unstale":"2024-09-12T03:00:59.346221+0000","last_undegraded":"2024-09-12T03:00:59.346221+0000","last_fullsized":"2024-09-12T03:00:59.346221+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:37.034969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.280332+0000","last_change":"2024-09-12T03:00:56.317308+0000","last_active":"2024-09-12T03:02:03.280332+0000","last_peered":"2024-09-12T03:02:03.280332+0000","last_clean":"2024-09-12T03:02:03.280332+0000","last_became_active":"2024-09-12T03:00:56.316935+0000","last_became_peered":"2024-09-12T03:00:56.316935+0000","last_unstale":"2024-09-12T03:02:03.280332+0000","last_undegraded":"2024-09-12T03:02:03.280332+0000","last_fullsized":"2024-09-12T03:02:03.280332+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:06.324489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343312+0000","last_change":"2024-09-12T03:00:58.341565+0000","last_active":"2024-09-12T03:00:59.343312+0000","last_peered":"2024-09-12T03:00:59.343312+0000","last_clean":"2024-09-12T03:00:59.343312+0000","last_became_active":"2024-09-12T03:00:58.341285+0000","last_became_peered":"2024-09-12T03:00:58.341285+0000","last_unstale":"2024-09-12T03:00:59.343312+0000","last_undegraded":"2024-09-12T03:00:59.343312+0000","last_fullsized":"2024-09-12T03:00:59.343312+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:59.156700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":74,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:59.476351+0000","last_change":"2024-09-12T03:00:56.314201+0000","last_active":"2024-09-12T03:01:59.476351+0000","last_peered":"2024-09-12T03:01:59.476351+0000","last_clean":"2024-09-12T03:01:59.476351+0000","last_became_active":"2024-09-12T03:00:56.313880+0000","last_became_peered":"2024-09-12T03:00:56.313880+0000","last_unstale":"2024-09-12T03:01:59.476351+0000","last_undegraded":"2024-09-12T03:01:59.476351+0000","last_fullsized":"2024-09-12T03:01:59.476351+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:57:12.952630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.329144+0000","last_change":"2024-09-12T03:00:58.345274+0000","last_active":"2024-09-12T03:00:59.329144+0000","last_peered":"2024-09-12T03:00:59.329144+0000","last_clean":"2024-09-12T03:00:59.329144+0000","last_became_active":"2024-09-12T03:00:58.344937+0000","last_became_peered":"2024-09-12T03:00:58.344937+0000","last_unstale":"2024-09-12T03:00:59.329144+0000","last_undegraded":"2024-09-12T03:00:59.329144+0000","last_fullsized":"2024-09-12T03:00:59.329144+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:17.288909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.283992+0000","last_change":"2024-09-12T03:00:56.314709+0000","last_active":"2024-09-12T03:02:03.283992+0000","last_peered":"2024-09-12T03:02:03.283992+0000","last_clean":"2024-09-12T03:02:03.283992+0000","last_became_active":"2024-09-12T03:00:56.314349+0000","last_became_peered":"2024-09-12T03:00:56.314349+0000","last_unstale":"2024-09-12T03:02:03.283992+0000","last_undegraded":"2024-09-12T03:02:03.283992+0000","last_fullsized":"2024-09-12T03:02:03.283992+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:18:35.371661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343140+0000","last_change":"2024-09-12T03:00:58.346055+0000","last_active":"2024-09-12T03:00:59.343140+0000","last_peered":"2024-09-12T03:00:59.343140+0000","last_clean":"2024-09-12T03:00:59.343140+0000","last_became_active":"2024-09-12T03:00:58.345701+0000","last_became_peered":"2024-09-12T03:00:58.345701+0000","last_unstale":"2024-09-12T03:00:59.343140+0000","last_undegraded":"2024-09-12T03:00:59.343140+0000","last_fullsized":"2024-09-12T03:00:59.343140+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:33.977245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.282131+0000","last_change":"2024-09-12T03:00:56.435962+0000","last_active":"2024-09-12T03:02:03.282131+0000","last_peered":"2024-09-12T03:02:03.282131+0000","last_clean":"2024-09-12T03:02:03.282131+0000","last_became_active":"2024-09-12T03:00:56.435515+0000","last_became_peered":"2024-09-12T03:00:56.435515+0000","last_unstale":"2024-09-12T03:02:03.282131+0000","last_undegraded":"2024-09-12T03:02:03.282131+0000","last_fullsized":"2024-09-12T03:02:03.282131+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:32.953436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328662+0000","last_change":"2024-09-12T03:00:58.344239+0000","last_active":"2024-09-12T03:00:59.328662+0000","last_peered":"2024-09-12T03:00:59.328662+0000","last_clean":"2024-09-12T03:00:59.328662+0000","last_became_active":"2024-09-12T03:00:58.343876+0000","last_became_peered":"2024-09-12T03:00:58.343876+0000","last_unstale":"2024-09-12T03:00:59.328662+0000","last_undegraded":"2024-09-12T03:00:59.328662+0000","last_fullsized":"2024-09-12T03:00:59.328662+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:49.401295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'29","reported_seq":162,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:02:00.477921+0000","last_change":"2024-09-12T03:00:56.317708+0000","last_active":"2024-09-12T03:02:00.477921+0000","last_peered":"2024-09-12T03:02:00.477921+0000","last_clean":"2024-09-12T03:02:00.477921+0000","last_became_active":"2024-09-12T03:00:56.317317+0000","last_became_peered":"2024-09-12T03:00:56.317317+0000","last_unstale":"2024-09-12T03:02:00.477921+0000","last_undegraded":"2024-09-12T03:02:00.477921+0000","last_fullsized":"2024-09-12T03:02:00.477921+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:47.788186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61752,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":115,"num_read_kb":117,"num_write":30,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.346470+0000","last_change":"2024-09-12T03:00:58.343602+0000","last_active":"2024-09-12T03:00:59.346470+0000","last_peered":"2024-09-12T03:00:59.346470+0000","last_clean":"2024-09-12T03:00:59.346470+0000","last_became_active":"2024-09-12T03:00:58.343301+0000","last_became_peered":"2024-09-12T03:00:58.343301+0000","last_unstale":"2024-09-12T03:00:59.346470+0000","last_undegraded":"2024-09-12T03:00:59.346470+0000","last_fullsized":"2024-09-12T03:00:59.346470+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:04.056118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.398339+0000","last_change":"2024-09-12T03:00:56.317874+0000","last_active":"2024-09-12T03:01:07.398339+0000","last_peered":"2024-09-12T03:01:07.398339+0000","last_clean":"2024-09-12T03:01:07.398339+0000","last_became_active":"2024-09-12T03:00:56.317566+0000","last_became_peered":"2024-09-12T03:00:56.317566+0000","last_unstale":"2024-09-12T03:01:07.398339+0000","last_undegraded":"2024-09-12T03:01:07.398339+0000","last_fullsized":"2024-09-12T03:01:07.398339+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:54.203730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.330344+0000","last_change":"2024-09-12T03:00:58.351539+0000","last_active":"2024-09-12T03:00:59.330344+0000","last_peered":"2024-09-12T03:00:59.330344+0000","last_clean":"2024-09-12T03:00:59.330344+0000","last_became_active":"2024-09-12T03:00:58.351249+0000","last_became_peered":"2024-09-12T03:00:58.351249+0000","last_unstale":"2024-09-12T03:00:59.330344+0000","last_undegraded":"2024-09-12T03:00:59.330344+0000","last_fullsized":"2024-09-12T03:00:59.330344+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:40.498867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.280655+0000","last_change":"2024-09-12T03:00:56.314489+0000","last_active":"2024-09-12T03:02:03.280655+0000","last_peered":"2024-09-12T03:02:03.280655+0000","last_clean":"2024-09-12T03:02:03.280655+0000","last_became_active":"2024-09-12T03:00:56.314142+0000","last_became_peered":"2024-09-12T03:00:56.314142+0000","last_unstale":"2024-09-12T03:02:03.280655+0000","last_undegraded":"2024-09-12T03:02:03.280655+0000","last_fullsized":"2024-09-12T03:02:03.280655+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:21.541713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.283805+0000","last_change":"2024-09-12T03:00:58.348752+0000","last_active":"2024-09-12T03:02:03.283805+0000","last_peered":"2024-09-12T03:02:03.283805+0000","last_clean":"2024-09-12T03:02:03.283805+0000","last_became_active":"2024-09-12T03:00:58.348391+0000","last_became_peered":"2024-09-12T03:00:58.348391+0000","last_unstale":"2024-09-12T03:02:03.283805+0000","last_undegraded":"2024-09-12T03:02:03.283805+0000","last_fullsized":"2024-09-12T03:02:03.283805+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:33:15.626306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'3","reported_seq":84,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289366+0000","last_change":"2024-09-12T03:00:56.311702+0000","last_active":"2024-09-12T03:02:04.289366+0000","last_peered":"2024-09-12T03:02:04.289366+0000","last_clean":"2024-09-12T03:02:04.289366+0000","last_became_active":"2024-09-12T03:00:56.311353+0000","last_became_peered":"2024-09-12T03:00:56.311353+0000","last_unstale":"2024-09-12T03:02:04.289366+0000","last_undegraded":"2024-09-12T03:02:04.289366+0000","last_fullsized":"2024-09-12T03:02:04.289366+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:01.004858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":60,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.283381+0000","last_change":"2024-09-12T03:00:58.351973+0000","last_active":"2024-09-12T03:02:03.283381+0000","last_peered":"2024-09-12T03:02:03.283381+0000","last_clean":"2024-09-12T03:02:03.283381+0000","last_became_active":"2024-09-12T03:00:58.351599+0000","last_became_peered":"2024-09-12T03:00:58.351599+0000","last_unstale":"2024-09-12T03:02:03.283381+0000","last_undegraded":"2024-09-12T03:02:03.283381+0000","last_fullsized":"2024-09-12T03:02:03.283381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:26.398828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.399761+0000","last_change":"2024-09-12T03:00:56.436185+0000","last_active":"2024-09-12T03:01:07.399761+0000","last_peered":"2024-09-12T03:01:07.399761+0000","last_clean":"2024-09-12T03:01:07.399761+0000","last_became_active":"2024-09-12T03:00:56.435717+0000","last_became_peered":"2024-09-12T03:00:56.435717+0000","last_unstale":"2024-09-12T03:01:07.399761+0000","last_undegraded":"2024-09-12T03:01:07.399761+0000","last_fullsized":"2024-09-12T03:01:07.399761+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:27.303182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"peering","last_fresh":"2024-09-12T03:00:59.328638+0000","last_change":"2024-09-12T03:02:03.280677+0000","last_active":"2024-09-12T03:02:03.280677+0000","last_peered":"2024-09-12T03:00:59.328638+0000","last_clean":"2024-09-12T03:00:59.328638+0000","last_became_active":"2024-09-12T03:00:58.346490+0000","last_became_peered":"2024-09-12T03:00:58.346490+0000","last_unstale":"2024-09-12T03:00:59.328638+0000","last_undegraded":"2024-09-12T03:00:59.328638+0000","last_fullsized":"2024-09-12T03:00:59.328638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4,8],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:08.394235+0000","last_change":"2024-09-12T03:00:56.311333+0000","last_active":"2024-09-12T03:01:08.394235+0000","last_peered":"2024-09-12T03:01:08.394235+0000","last_clean":"2024-09-12T03:01:08.394235+0000","last_became_active":"2024-09-12T03:00:56.310871+0000","last_became_peered":"2024-09-12T03:00:56.310871+0000","last_unstale":"2024-09-12T03:01:08.394235+0000","last_undegraded":"2024-09-12T03:01:08.394235+0000","last_fullsized":"2024-09-12T03:01:08.394235+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:59.344572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.348769+0000","last_change":"2024-09-12T03:00:58.349401+0000","last_active":"2024-09-12T03:00:59.348769+0000","last_peered":"2024-09-12T03:00:59.348769+0000","last_clean":"2024-09-12T03:00:59.348769+0000","last_became_active":"2024-09-12T03:00:58.348996+0000","last_became_peered":"2024-09-12T03:00:58.348996+0000","last_unstale":"2024-09-12T03:00:59.348769+0000","last_undegraded":"2024-09-12T03:00:59.348769+0000","last_fullsized":"2024-09-12T03:00:59.348769+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:20:26.201684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344392+0000","last_change":"2024-09-12T03:00:56.314182+0000","last_active":"2024-09-12T03:00:59.344392+0000","last_peered":"2024-09-12T03:00:59.344392+0000","last_clean":"2024-09-12T03:00:59.344392+0000","last_became_active":"2024-09-12T03:00:56.313749+0000","last_became_peered":"2024-09-12T03:00:56.313749+0000","last_unstale":"2024-09-12T03:00:59.344392+0000","last_undegraded":"2024-09-12T03:00:59.344392+0000","last_fullsized":"2024-09-12T03:00:59.344392+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:31:45.767500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.284069+0000","last_change":"2024-09-12T03:00:58.344356+0000","last_active":"2024-09-12T03:02:03.284069+0000","last_peered":"2024-09-12T03:02:03.284069+0000","last_clean":"2024-09-12T03:02:03.284069+0000","last_became_active":"2024-09-12T03:00:58.344035+0000","last_became_peered":"2024-09-12T03:00:58.344035+0000","last_unstale":"2024-09-12T03:02:03.284069+0000","last_undegraded":"2024-09-12T03:02:03.284069+0000","last_fullsized":"2024-09-12T03:02:03.284069+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:45:18.681018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.282252+0000","last_change":"2024-09-12T03:00:56.315653+0000","last_active":"2024-09-12T03:02:03.282252+0000","last_peered":"2024-09-12T03:02:03.282252+0000","last_clean":"2024-09-12T03:02:03.282252+0000","last_became_active":"2024-09-12T03:00:56.315284+0000","last_became_peered":"2024-09-12T03:00:56.315284+0000","last_unstale":"2024-09-12T03:02:03.282252+0000","last_undegraded":"2024-09-12T03:02:03.282252+0000","last_fullsized":"2024-09-12T03:02:03.282252+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:42.934020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":15,"reported_epoch":78,"state":"peering","last_fresh":"2024-09-12T03:01:17.267451+0000","last_change":"2024-09-12T03:02:03.283802+0000","last_active":"2024-09-12T03:02:03.283802+0000","last_peered":"2024-09-12T03:01:17.267451+0000","last_clean":"2024-09-12T03:01:17.267451+0000","last_became_active":"2024-09-12T03:00:58.357451+0000","last_became_peered":"2024-09-12T03:00:58.357451+0000","last_unstale":"2024-09-12T03:01:17.267451+0000","last_undegraded":"2024-09-12T03:01:17.267451+0000","last_fullsized":"2024-09-12T03:01:17.267451+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5,9],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.398109+0000","last_change":"2024-09-12T03:00:56.746780+0000","last_active":"2024-09-12T03:01:07.398109+0000","last_peered":"2024-09-12T03:01:07.398109+0000","last_clean":"2024-09-12T03:01:07.398109+0000","last_became_active":"2024-09-12T03:00:56.746414+0000","last_became_peered":"2024-09-12T03:00:56.746414+0000","last_unstale":"2024-09-12T03:01:07.398109+0000","last_undegraded":"2024-09-12T03:01:07.398109+0000","last_fullsized":"2024-09-12T03:01:07.398109+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:15:35.214196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.283877+0000","last_change":"2024-09-12T03:00:58.351713+0000","last_active":"2024-09-12T03:02:03.283877+0000","last_peered":"2024-09-12T03:02:03.283877+0000","last_clean":"2024-09-12T03:02:03.283877+0000","last_became_active":"2024-09-12T03:00:58.351371+0000","last_became_peered":"2024-09-12T03:00:58.351371+0000","last_unstale":"2024-09-12T03:02:03.283877+0000","last_undegraded":"2024-09-12T03:02:03.283877+0000","last_fullsized":"2024-09-12T03:02:03.283877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:13.346956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'137","reported_seq":333,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.283543+0000","last_change":"2024-09-12T02:59:45.461482+0000","last_active":"2024-09-12T03:02:03.283543+0000","last_peered":"2024-09-12T03:02:03.283543+0000","last_clean":"2024-09-12T03:02:03.283543+0000","last_became_active":"2024-09-12T02:59:45.461058+0000","last_became_peered":"2024-09-12T02:59:45.461058+0000","last_unstale":"2024-09-12T03:02:03.283543+0000","last_undegraded":"2024-09-12T03:02:03.283543+0000","last_fullsized":"2024-09-12T03:02:03.283543+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:18.034438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:18.034438+0000","last_clean_scrub_stamp":"2024-09-12T02:56:18.034438+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:09.251858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289291+0000","last_change":"2024-09-12T03:00:56.315245+0000","last_active":"2024-09-12T03:02:04.289291+0000","last_peered":"2024-09-12T03:02:04.289291+0000","last_clean":"2024-09-12T03:02:04.289291+0000","last_became_active":"2024-09-12T03:00:56.314887+0000","last_became_peered":"2024-09-12T03:00:56.314887+0000","last_unstale":"2024-09-12T03:02:04.289291+0000","last_undegraded":"2024-09-12T03:02:04.289291+0000","last_fullsized":"2024-09-12T03:02:04.289291+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:48.750657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344703+0000","last_change":"2024-09-12T03:00:58.357874+0000","last_active":"2024-09-12T03:00:59.344703+0000","last_peered":"2024-09-12T03:00:59.344703+0000","last_clean":"2024-09-12T03:00:59.344703+0000","last_became_active":"2024-09-12T03:00:58.357575+0000","last_became_peered":"2024-09-12T03:00:58.357575+0000","last_unstale":"2024-09-12T03:00:59.344703+0000","last_undegraded":"2024-09-12T03:00:59.344703+0000","last_fullsized":"2024-09-12T03:00:59.344703+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:46.125620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.283498+0000","last_change":"2024-09-12T03:00:56.312263+0000","last_active":"2024-09-12T03:02:03.283498+0000","last_peered":"2024-09-12T03:02:03.283498+0000","last_clean":"2024-09-12T03:02:03.283498+0000","last_became_active":"2024-09-12T03:00:56.311894+0000","last_became_peered":"2024-09-12T03:00:56.311894+0000","last_unstale":"2024-09-12T03:02:03.283498+0000","last_undegraded":"2024-09-12T03:02:03.283498+0000","last_fullsized":"2024-09-12T03:02:03.283498+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:52.434203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.330287+0000","last_change":"2024-09-12T03:00:58.350762+0000","last_active":"2024-09-12T03:00:59.330287+0000","last_peered":"2024-09-12T03:00:59.330287+0000","last_clean":"2024-09-12T03:00:59.330287+0000","last_became_active":"2024-09-12T03:00:58.348038+0000","last_became_peered":"2024-09-12T03:00:58.348038+0000","last_unstale":"2024-09-12T03:00:59.330287+0000","last_undegraded":"2024-09-12T03:00:59.330287+0000","last_fullsized":"2024-09-12T03:00:59.330287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:08.916636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":75,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:59.475763+0000","last_change":"2024-09-12T03:00:56.435641+0000","last_active":"2024-09-12T03:01:59.475763+0000","last_peered":"2024-09-12T03:01:59.475763+0000","last_clean":"2024-09-12T03:01:59.475763+0000","last_became_active":"2024-09-12T03:00:56.435319+0000","last_became_peered":"2024-09-12T03:00:56.435319+0000","last_unstale":"2024-09-12T03:01:59.475763+0000","last_undegraded":"2024-09-12T03:01:59.475763+0000","last_fullsized":"2024-09-12T03:01:59.475763+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:56.894978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.280141+0000","last_change":"2024-09-12T03:00:58.344708+0000","last_active":"2024-09-12T03:02:03.280141+0000","last_peered":"2024-09-12T03:02:03.280141+0000","last_clean":"2024-09-12T03:02:03.280141+0000","last_became_active":"2024-09-12T03:00:58.344389+0000","last_became_peered":"2024-09-12T03:00:58.344389+0000","last_unstale":"2024-09-12T03:02:03.280141+0000","last_undegraded":"2024-09-12T03:02:03.280141+0000","last_fullsized":"2024-09-12T03:02:03.280141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:08.160383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.283775+0000","last_change":"2024-09-12T03:00:56.317068+0000","last_active":"2024-09-12T03:02:03.283775+0000","last_peered":"2024-09-12T03:02:03.283775+0000","last_clean":"2024-09-12T03:02:03.283775+0000","last_became_active":"2024-09-12T03:00:56.316698+0000","last_became_peered":"2024-09-12T03:00:56.316698+0000","last_unstale":"2024-09-12T03:02:03.283775+0000","last_undegraded":"2024-09-12T03:02:03.283775+0000","last_fullsized":"2024-09-12T03:02:03.283775+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:41:19.199418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343531+0000","last_change":"2024-09-12T03:00:58.347814+0000","last_active":"2024-09-12T03:00:59.343531+0000","last_peered":"2024-09-12T03:00:59.343531+0000","last_clean":"2024-09-12T03:00:59.343531+0000","last_became_active":"2024-09-12T03:00:58.347112+0000","last_became_peered":"2024-09-12T03:00:58.347112+0000","last_unstale":"2024-09-12T03:00:59.343531+0000","last_undegraded":"2024-09-12T03:00:59.343531+0000","last_fullsized":"2024-09-12T03:00:59.343531+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:59.325113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342760+0000","last_change":"2024-09-12T03:00:56.435771+0000","last_active":"2024-09-12T03:00:59.342760+0000","last_peered":"2024-09-12T03:00:59.342760+0000","last_clean":"2024-09-12T03:00:59.342760+0000","last_became_active":"2024-09-12T03:00:56.435448+0000","last_became_peered":"2024-09-12T03:00:56.435448+0000","last_unstale":"2024-09-12T03:00:59.342760+0000","last_undegraded":"2024-09-12T03:00:59.342760+0000","last_fullsized":"2024-09-12T03:00:59.342760+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:33.650338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:01.118993+0000","last_change":"2024-09-12T03:00:58.341071+0000","last_active":"2024-09-12T03:01:01.118993+0000","last_peered":"2024-09-12T03:01:01.118993+0000","last_clean":"2024-09-12T03:01:01.118993+0000","last_became_active":"2024-09-12T03:00:58.340753+0000","last_became_peered":"2024-09-12T03:00:58.340753+0000","last_unstale":"2024-09-12T03:01:01.118993+0000","last_undegraded":"2024-09-12T03:01:01.118993+0000","last_fullsized":"2024-09-12T03:01:01.118993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:21.217915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:06.388962+0000","last_change":"2024-09-12T03:00:56.436058+0000","last_active":"2024-09-12T03:01:06.388962+0000","last_peered":"2024-09-12T03:01:06.388962+0000","last_clean":"2024-09-12T03:01:06.388962+0000","last_became_active":"2024-09-12T03:00:56.435518+0000","last_became_peered":"2024-09-12T03:00:56.435518+0000","last_unstale":"2024-09-12T03:01:06.388962+0000","last_undegraded":"2024-09-12T03:01:06.388962+0000","last_fullsized":"2024-09-12T03:01:06.388962+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:32.235097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.330626+0000","last_change":"2024-09-12T03:00:58.348236+0000","last_active":"2024-09-12T03:00:59.330626+0000","last_peered":"2024-09-12T03:00:59.330626+0000","last_clean":"2024-09-12T03:00:59.330626+0000","last_became_active":"2024-09-12T03:00:58.347727+0000","last_became_peered":"2024-09-12T03:00:58.347727+0000","last_unstale":"2024-09-12T03:00:59.330626+0000","last_undegraded":"2024-09-12T03:00:59.330626+0000","last_fullsized":"2024-09-12T03:00:59.330626+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:19.233163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'10","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:23.344906+0000","last_change":"2024-09-12T03:00:56.310537+0000","last_active":"2024-09-12T03:01:23.344906+0000","last_peered":"2024-09-12T03:01:23.344906+0000","last_clean":"2024-09-12T03:01:23.344906+0000","last_became_active":"2024-09-12T03:00:56.310135+0000","last_became_peered":"2024-09-12T03:00:56.310135+0000","last_unstale":"2024-09-12T03:01:23.344906+0000","last_undegraded":"2024-09-12T03:01:23.344906+0000","last_fullsized":"2024-09-12T03:01:23.344906+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:10:55.584339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344075+0000","last_change":"2024-09-12T03:00:58.358417+0000","last_active":"2024-09-12T03:00:59.344075+0000","last_peered":"2024-09-12T03:00:59.344075+0000","last_clean":"2024-09-12T03:00:59.344075+0000","last_became_active":"2024-09-12T03:00:58.358119+0000","last_became_peered":"2024-09-12T03:00:58.358119+0000","last_unstale":"2024-09-12T03:00:59.344075+0000","last_undegraded":"2024-09-12T03:00:59.344075+0000","last_fullsized":"2024-09-12T03:00:59.344075+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:10.897594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289526+0000","last_change":"2024-09-12T03:00:56.747044+0000","last_active":"2024-09-12T03:02:04.289526+0000","last_peered":"2024-09-12T03:02:04.289526+0000","last_clean":"2024-09-12T03:02:04.289526+0000","last_became_active":"2024-09-12T03:00:56.746699+0000","last_became_peered":"2024-09-12T03:00:56.746699+0000","last_unstale":"2024-09-12T03:02:04.289526+0000","last_undegraded":"2024-09-12T03:02:04.289526+0000","last_fullsized":"2024-09-12T03:02:04.289526+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:11.420109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342687+0000","last_change":"2024-09-12T03:00:58.346854+0000","last_active":"2024-09-12T03:00:59.342687+0000","last_peered":"2024-09-12T03:00:59.342687+0000","last_clean":"2024-09-12T03:00:59.342687+0000","last_became_active":"2024-09-12T03:00:58.346535+0000","last_became_peered":"2024-09-12T03:00:58.346535+0000","last_unstale":"2024-09-12T03:00:59.342687+0000","last_undegraded":"2024-09-12T03:00:59.342687+0000","last_fullsized":"2024-09-12T03:00:59.342687+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:46.637914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:06.389702+0000","last_change":"2024-09-12T03:00:56.435957+0000","last_active":"2024-09-12T03:01:06.389702+0000","last_peered":"2024-09-12T03:01:06.389702+0000","last_clean":"2024-09-12T03:01:06.389702+0000","last_became_active":"2024-09-12T03:00:56.435603+0000","last_became_peered":"2024-09-12T03:00:56.435603+0000","last_unstale":"2024-09-12T03:01:06.389702+0000","last_undegraded":"2024-09-12T03:01:06.389702+0000","last_fullsized":"2024-09-12T03:01:06.389702+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:34.553189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343477+0000","last_change":"2024-09-12T03:00:58.337784+0000","last_active":"2024-09-12T03:00:59.343477+0000","last_peered":"2024-09-12T03:00:59.343477+0000","last_clean":"2024-09-12T03:00:59.343477+0000","last_became_active":"2024-09-12T03:00:58.337415+0000","last_became_peered":"2024-09-12T03:00:58.337415+0000","last_unstale":"2024-09-12T03:00:59.343477+0000","last_undegraded":"2024-09-12T03:00:59.343477+0000","last_fullsized":"2024-09-12T03:00:59.343477+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:38:15.557067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:06.390162+0000","last_change":"2024-09-12T03:00:56.315356+0000","last_active":"2024-09-12T03:01:06.390162+0000","last_peered":"2024-09-12T03:01:06.390162+0000","last_clean":"2024-09-12T03:01:06.390162+0000","last_became_active":"2024-09-12T03:00:56.314873+0000","last_became_peered":"2024-09-12T03:00:56.314873+0000","last_unstale":"2024-09-12T03:01:06.390162+0000","last_undegraded":"2024-09-12T03:01:06.390162+0000","last_fullsized":"2024-09-12T03:01:06.390162+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:55.710156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.348074+0000","last_change":"2024-09-12T03:00:58.346706+0000","last_active":"2024-09-12T03:00:59.348074+0000","last_peered":"2024-09-12T03:00:59.348074+0000","last_clean":"2024-09-12T03:00:59.348074+0000","last_became_active":"2024-09-12T03:00:58.346363+0000","last_became_peered":"2024-09-12T03:00:58.346363+0000","last_unstale":"2024-09-12T03:00:59.348074+0000","last_undegraded":"2024-09-12T03:00:59.348074+0000","last_fullsized":"2024-09-12T03:00:59.348074+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:13.468639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343312+0000","last_change":"2024-09-12T03:00:56.314659+0000","last_active":"2024-09-12T03:00:59.343312+0000","last_peered":"2024-09-12T03:00:59.343312+0000","last_clean":"2024-09-12T03:00:59.343312+0000","last_became_active":"2024-09-12T03:00:56.314316+0000","last_became_peered":"2024-09-12T03:00:56.314316+0000","last_unstale":"2024-09-12T03:00:59.343312+0000","last_undegraded":"2024-09-12T03:00:59.343312+0000","last_fullsized":"2024-09-12T03:00:59.343312+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:26:23.883066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.345830+0000","last_change":"2024-09-12T03:00:58.356682+0000","last_active":"2024-09-12T03:00:59.345830+0000","last_peered":"2024-09-12T03:00:59.345830+0000","last_clean":"2024-09-12T03:00:59.345830+0000","last_became_active":"2024-09-12T03:00:58.356383+0000","last_became_peered":"2024-09-12T03:00:58.356383+0000","last_unstale":"2024-09-12T03:00:59.345830+0000","last_undegraded":"2024-09-12T03:00:59.345830+0000","last_fullsized":"2024-09-12T03:00:59.345830+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:14:16.428393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:06.390865+0000","last_change":"2024-09-12T03:00:56.314943+0000","last_active":"2024-09-12T03:01:06.390865+0000","last_peered":"2024-09-12T03:01:06.390865+0000","last_clean":"2024-09-12T03:01:06.390865+0000","last_became_active":"2024-09-12T03:00:56.314602+0000","last_became_peered":"2024-09-12T03:00:56.314602+0000","last_unstale":"2024-09-12T03:01:06.390865+0000","last_undegraded":"2024-09-12T03:01:06.390865+0000","last_fullsized":"2024-09-12T03:01:06.390865+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:31.207880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.283651+0000","last_change":"2024-09-12T03:00:58.351774+0000","last_active":"2024-09-12T03:02:03.283651+0000","last_peered":"2024-09-12T03:02:03.283651+0000","last_clean":"2024-09-12T03:02:03.283651+0000","last_became_active":"2024-09-12T03:00:58.351453+0000","last_became_peered":"2024-09-12T03:00:58.351453+0000","last_unstale":"2024-09-12T03:02:03.283651+0000","last_undegraded":"2024-09-12T03:02:03.283651+0000","last_fullsized":"2024-09-12T03:02:03.283651+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:18:24.260110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.281997+0000","last_change":"2024-09-12T03:00:58.346886+0000","last_active":"2024-09-12T03:02:03.281997+0000","last_peered":"2024-09-12T03:02:03.281997+0000","last_clean":"2024-09-12T03:02:03.281997+0000","last_became_active":"2024-09-12T03:00:58.346499+0000","last_became_peered":"2024-09-12T03:00:58.346499+0000","last_unstale":"2024-09-12T03:02:03.281997+0000","last_undegraded":"2024-09-12T03:02:03.281997+0000","last_fullsized":"2024-09-12T03:02:03.281997+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:02:24.045610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.350709+0000","last_change":"2024-09-12T03:00:56.317443+0000","last_active":"2024-09-12T03:00:59.350709+0000","last_peered":"2024-09-12T03:00:59.350709+0000","last_clean":"2024-09-12T03:00:59.350709+0000","last_became_active":"2024-09-12T03:00:56.317045+0000","last_became_peered":"2024-09-12T03:00:56.317045+0000","last_unstale":"2024-09-12T03:00:59.350709+0000","last_undegraded":"2024-09-12T03:00:59.350709+0000","last_fullsized":"2024-09-12T03:00:59.350709+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:07.687606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328499+0000","last_change":"2024-09-12T03:00:58.337257+0000","last_active":"2024-09-12T03:00:59.328499+0000","last_peered":"2024-09-12T03:00:59.328499+0000","last_clean":"2024-09-12T03:00:59.328499+0000","last_became_active":"2024-09-12T03:00:58.336847+0000","last_became_peered":"2024-09-12T03:00:58.336847+0000","last_unstale":"2024-09-12T03:00:59.328499+0000","last_undegraded":"2024-09-12T03:00:59.328499+0000","last_fullsized":"2024-09-12T03:00:59.328499+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:22.846206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.398760+0000","last_change":"2024-09-12T03:00:56.316255+0000","last_active":"2024-09-12T03:01:07.398760+0000","last_peered":"2024-09-12T03:01:07.398760+0000","last_clean":"2024-09-12T03:01:07.398760+0000","last_became_active":"2024-09-12T03:00:56.315906+0000","last_became_peered":"2024-09-12T03:00:56.315906+0000","last_unstale":"2024-09-12T03:01:07.398760+0000","last_undegraded":"2024-09-12T03:01:07.398760+0000","last_fullsized":"2024-09-12T03:01:07.398760+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:14.798601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342990+0000","last_change":"2024-09-12T03:00:58.347206+0000","last_active":"2024-09-12T03:00:59.342990+0000","last_peered":"2024-09-12T03:00:59.342990+0000","last_clean":"2024-09-12T03:00:59.342990+0000","last_became_active":"2024-09-12T03:00:58.346923+0000","last_became_peered":"2024-09-12T03:00:58.346923+0000","last_unstale":"2024-09-12T03:00:59.342990+0000","last_undegraded":"2024-09-12T03:00:59.342990+0000","last_fullsized":"2024-09-12T03:00:59.342990+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:41.376211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.347646+0000","last_change":"2024-09-12T03:00:56.435431+0000","last_active":"2024-09-12T03:00:59.347646+0000","last_peered":"2024-09-12T03:00:59.347646+0000","last_clean":"2024-09-12T03:00:59.347646+0000","last_became_active":"2024-09-12T03:00:56.435088+0000","last_became_peered":"2024-09-12T03:00:56.435088+0000","last_unstale":"2024-09-12T03:00:59.347646+0000","last_undegraded":"2024-09-12T03:00:59.347646+0000","last_fullsized":"2024-09-12T03:00:59.347646+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:24.738340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343937+0000","last_change":"2024-09-12T03:00:58.351737+0000","last_active":"2024-09-12T03:00:59.343937+0000","last_peered":"2024-09-12T03:00:59.343937+0000","last_clean":"2024-09-12T03:00:59.343937+0000","last_became_active":"2024-09-12T03:00:58.351407+0000","last_became_peered":"2024-09-12T03:00:58.351407+0000","last_unstale":"2024-09-12T03:00:59.343937+0000","last_undegraded":"2024-09-12T03:00:59.343937+0000","last_fullsized":"2024-09-12T03:00:59.343937+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:34.206703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.399005+0000","last_change":"2024-09-12T03:00:56.310910+0000","last_active":"2024-09-12T03:01:07.399005+0000","last_peered":"2024-09-12T03:01:07.399005+0000","last_clean":"2024-09-12T03:01:07.399005+0000","last_became_active":"2024-09-12T03:00:56.310657+0000","last_became_peered":"2024-09-12T03:00:56.310657+0000","last_unstale":"2024-09-12T03:01:07.399005+0000","last_undegraded":"2024-09-12T03:01:07.399005+0000","last_fullsized":"2024-09-12T03:01:07.399005+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:06.117215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65362,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":352,"num_read_kb":241,"num_write":161,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":196086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245899,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645339,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":414814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906975,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":1068047,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070501,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":415050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299434,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":868352,"data_stored":414970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528367,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":1006509,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":40,"seq":171798691892,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":856064,"data_stored":414439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":33,"seq":141733920825,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":1006193,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051647,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":872448,"data_stored":423461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280579,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":414750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476808,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":942080,"data_stored":477334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607630,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":475984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":876544,"data_stored":415710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:02:07.479 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:02:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:07 smithi183 ceph-mon[35535]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:02:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:07 smithi183 ceph-mon[35535]: pgmap v312: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:02:07.729 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:07.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:07 smithi177 ceph-mon[35305]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:02:07.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:07 smithi177 ceph-mon[35305]: pgmap v312: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:02:08.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.242+0000 7f8ffa32b640 1 -- 172.21.15.116:0/757961985 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ff4102570 msgr2=0x7f8ff41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:08.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.242+0000 7f8ffa32b640 1 --2- 172.21.15.116:0/757961985 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ff4102570 0x7f8ff41029b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc009900 tx=0x7f8fdc02a480 comp rx=0 tx=0).stop 2024-09-12T03:02:08.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.242+0000 7f8ffa32b640 1 -- 172.21.15.116:0/757961985 shutdown_connections 2024-09-12T03:02:08.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.242+0000 7f8ffa32b640 1 --2- 172.21.15.116:0/757961985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ff4102ef0 0x7f8ff410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:08.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.242+0000 7f8ffa32b640 1 --2- 172.21.15.116:0/757961985 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ff4102570 0x7f8ff41029b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:08.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.242+0000 7f8ffa32b640 1 --2- 172.21.15.116:0/757961985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ff4100e20 0x7f8ff4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:08.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.242+0000 7f8ffa32b640 1 -- 172.21.15.116:0/757961985 >> 172.21.15.116:0/757961985 conn(0x7f8ff40fc7d0 msgr2=0x7f8ff40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:08.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.242+0000 7f8ffa32b640 1 -- 172.21.15.116:0/757961985 shutdown_connections 2024-09-12T03:02:08.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.242+0000 7f8ffa32b640 1 -- 172.21.15.116:0/757961985 wait complete. 2024-09-12T03:02:08.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.243+0000 7f8ffa32b640 1 Processor -- start 2024-09-12T03:02:08.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.243+0000 7f8ffa32b640 1 -- start start 2024-09-12T03:02:08.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.243+0000 7f8ffa32b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ff4100e20 0x7f8ff40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:08.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.243+0000 7f8ffa32b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ff4102570 0x7f8ff40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:08.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.243+0000 7f8ffa32b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ff4102ef0 0x7f8ff40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:08.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.243+0000 7f8ffa32b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff4110fa0 con 0x7f8ff4102570 2024-09-12T03:02:08.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.243+0000 7f8ffa32b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff4110e20 con 0x7f8ff4102ef0 2024-09-12T03:02:08.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.243+0000 7f8ffa32b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff4111120 con 0x7f8ff4100e20 2024-09-12T03:02:08.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.243+0000 7f8febfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ff4102570 0x7f8ff40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:08.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.243+0000 7f8febfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ff4102570 0x7f8ff40fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60174/0 (socket says 172.21.15.116:60174) 2024-09-12T03:02:08.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.243+0000 7f8febfff640 1 -- 172.21.15.116:0/2718915191 learned_addr learned my addr 172.21.15.116:0/2718915191 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:08.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.243+0000 7f8ff3fff640 1 --2- 172.21.15.116:0/2718915191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ff4100e20 0x7f8ff40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:08.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.243+0000 7f8ff88c8640 1 --2- 172.21.15.116:0/2718915191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ff4102ef0 0x7f8ff40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:08.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.243+0000 7f8febfff640 1 -- 172.21.15.116:0/2718915191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ff4100e20 msgr2=0x7f8ff40fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:08.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.243+0000 7f8febfff640 1 --2- 172.21.15.116:0/2718915191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ff4100e20 0x7f8ff40fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:08.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.243+0000 7f8febfff640 1 -- 172.21.15.116:0/2718915191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ff4102ef0 msgr2=0x7f8ff40fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:08.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.243+0000 7f8febfff640 1 --2- 172.21.15.116:0/2718915191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ff4102ef0 0x7f8ff40fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:08.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.243+0000 7f8febfff640 1 -- 172.21.15.116:0/2718915191 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff41a6960 con 0x7f8ff4102570 2024-09-12T03:02:08.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.243+0000 7f8ff88c8640 1 --2- 172.21.15.116:0/2718915191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ff4102ef0 0x7f8ff40fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:08.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.244+0000 7f8febfff640 1 --2- 172.21.15.116:0/2718915191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ff4102570 0x7f8ff40fd1c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc006010 tx=0x7f8fdc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:08.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.244+0000 7f8ff1ffb640 1 -- 172.21.15.116:0/2718915191 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc03f070 con 0x7f8ff4102570 2024-09-12T03:02:08.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.244+0000 7f8ff1ffb640 1 -- 172.21.15.116:0/2718915191 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fdc003e30 con 0x7f8ff4102570 2024-09-12T03:02:08.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.244+0000 7f8ffa32b640 1 -- 172.21.15.116:0/2718915191 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff41a6bf0 con 0x7f8ff4102570 2024-09-12T03:02:08.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.244+0000 7f8ff1ffb640 1 -- 172.21.15.116:0/2718915191 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc035030 con 0x7f8ff4102570 2024-09-12T03:02:08.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.244+0000 7f8ffa32b640 1 -- 172.21.15.116:0/2718915191 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff41a70d0 con 0x7f8ff4102570 2024-09-12T03:02:08.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.245+0000 7f8ffa32b640 1 -- 172.21.15.116:0/2718915191 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ff4101200 con 0x7f8ff4102570 2024-09-12T03:02:08.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.246+0000 7f8ff1ffb640 1 -- 172.21.15.116:0/2718915191 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f8fdc042070 con 0x7f8ff4102570 2024-09-12T03:02:08.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.246+0000 7f8ff1ffb640 1 --2- 172.21.15.116:0/2718915191 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8fe00a2c10 0x7f8fe00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:08.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.247+0000 7f8ff3fff640 1 --2- 172.21.15.116:0/2718915191 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8fe00a2c10 0x7f8fe00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:08.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.247+0000 7f8ff1ffb640 1 -- 172.21.15.116:0/2718915191 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f8fdc0f0000 con 0x7f8ff4102570 2024-09-12T03:02:08.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.247+0000 7f8ff3fff640 1 --2- 172.21.15.116:0/2718915191 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8fe00a2c10 0x7f8fe00a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4005e10 tx=0x7f8fe4005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:08.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.250+0000 7f8ff1ffb640 1 -- 172.21.15.116:0/2718915191 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8fdc0bce30 con 0x7f8ff4102570 2024-09-12T03:02:08.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.415+0000 7f8ffa32b640 1 -- 172.21.15.116:0/2718915191 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8ff4062630 con 0x7f8fe00a2c10 2024-09-12T03:02:08.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.421+0000 7f8ff1ffb640 1 -- 172.21.15.116:0/2718915191 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258770 (secure 0 0 0) 0x7f8ff4062630 con 0x7f8fe00a2c10 2024-09-12T03:02:08.425 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:02:08.426 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-09-12T03:02:08.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.425+0000 7f8ffa32b640 1 -- 172.21.15.116:0/2718915191 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8fe00a2c10 msgr2=0x7f8fe00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:08.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.425+0000 7f8ffa32b640 1 --2- 172.21.15.116:0/2718915191 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8fe00a2c10 0x7f8fe00a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4005e10 tx=0x7f8fe4005da0 comp rx=0 tx=0).stop 2024-09-12T03:02:08.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.425+0000 7f8ffa32b640 1 -- 172.21.15.116:0/2718915191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ff4102570 msgr2=0x7f8ff40fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:08.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.425+0000 7f8ffa32b640 1 --2- 172.21.15.116:0/2718915191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ff4102570 0x7f8ff40fd1c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc006010 tx=0x7f8fdc003c10 comp rx=0 tx=0).stop 2024-09-12T03:02:08.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.425+0000 7f8ffa32b640 1 -- 172.21.15.116:0/2718915191 shutdown_connections 2024-09-12T03:02:08.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.425+0000 7f8ffa32b640 1 --2- 172.21.15.116:0/2718915191 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8fe00a2c10 0x7f8fe00a50d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:08.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.425+0000 7f8ffa32b640 1 --2- 172.21.15.116:0/2718915191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ff4102ef0 0x7f8ff40fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:08.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.425+0000 7f8ffa32b640 1 --2- 172.21.15.116:0/2718915191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ff4102570 0x7f8ff40fd1c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:08.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.425+0000 7f8ffa32b640 1 --2- 172.21.15.116:0/2718915191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ff4100e20 0x7f8ff40fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:08.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.425+0000 7f8ffa32b640 1 -- 172.21.15.116:0/2718915191 >> 172.21.15.116:0/2718915191 conn(0x7f8ff40fc7d0 msgr2=0x7f8ff406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:08.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.426+0000 7f8ffa32b640 1 -- 172.21.15.116:0/2718915191 shutdown_connections 2024-09-12T03:02:08.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:08.426+0000 7f8ffa32b640 1 -- 172.21.15.116:0/2718915191 wait complete. 2024-09-12T03:02:09.190 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2024-09-12T03:02:07.123445+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655964,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":498,"num_read_kb":365,"num_write":344,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":292,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368212,"kb_used_data":10620,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573612,"statfs":{"total":1151940427776,"available":1151563378688,"internally_reserved":0,"allocated":10874880,"data_stored":7015352,"data_compressed":107490,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001986"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.348943+0000","last_change":"2024-09-12T03:00:58.348563+0000","last_active":"2024-09-12T03:00:59.348943+0000","last_peered":"2024-09-12T03:00:59.348943+0000","last_clean":"2024-09-12T03:00:59.348943+0000","last_became_active":"2024-09-12T03:00:58.347940+0000","last_became_peered":"2024-09-12T03:00:58.347940+0000","last_unstale":"2024-09-12T03:00:59.348943+0000","last_undegraded":"2024-09-12T03:00:59.348943+0000","last_fullsized":"2024-09-12T03:00:59.348943+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:43.489238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342719+0000","last_change":"2024-09-12T03:00:58.351969+0000","last_active":"2024-09-12T03:00:59.342719+0000","last_peered":"2024-09-12T03:00:59.342719+0000","last_clean":"2024-09-12T03:00:59.342719+0000","last_became_active":"2024-09-12T03:00:58.351679+0000","last_became_peered":"2024-09-12T03:00:58.351679+0000","last_unstale":"2024-09-12T03:00:59.342719+0000","last_undegraded":"2024-09-12T03:00:59.342719+0000","last_fullsized":"2024-09-12T03:00:59.342719+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:27:01.174890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342541+0000","last_change":"2024-09-12T03:00:58.346045+0000","last_active":"2024-09-12T03:00:59.342541+0000","last_peered":"2024-09-12T03:00:59.342541+0000","last_clean":"2024-09-12T03:00:59.342541+0000","last_became_active":"2024-09-12T03:00:58.345719+0000","last_became_peered":"2024-09-12T03:00:58.345719+0000","last_unstale":"2024-09-12T03:00:59.342541+0000","last_undegraded":"2024-09-12T03:00:59.342541+0000","last_fullsized":"2024-09-12T03:00:59.342541+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:12.312650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289352+0000","last_change":"2024-09-12T03:00:58.348165+0000","last_active":"2024-09-12T03:02:04.289352+0000","last_peered":"2024-09-12T03:02:04.289352+0000","last_clean":"2024-09-12T03:02:04.289352+0000","last_became_active":"2024-09-12T03:00:58.347734+0000","last_became_peered":"2024-09-12T03:00:58.347734+0000","last_unstale":"2024-09-12T03:02:04.289352+0000","last_undegraded":"2024-09-12T03:02:04.289352+0000","last_fullsized":"2024-09-12T03:02:04.289352+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:05.945900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.291897+0000","last_change":"2024-09-12T03:00:58.339316+0000","last_active":"2024-09-12T03:02:04.291897+0000","last_peered":"2024-09-12T03:02:04.291897+0000","last_clean":"2024-09-12T03:02:04.291897+0000","last_became_active":"2024-09-12T03:00:58.338964+0000","last_became_peered":"2024-09-12T03:00:58.338964+0000","last_unstale":"2024-09-12T03:02:04.291897+0000","last_undegraded":"2024-09-12T03:02:04.291897+0000","last_fullsized":"2024-09-12T03:02:04.291897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:50.721276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282483+0000","last_change":"2024-09-12T03:00:58.346838+0000","last_active":"2024-09-12T03:02:04.282483+0000","last_peered":"2024-09-12T03:02:04.282483+0000","last_clean":"2024-09-12T03:02:04.282483+0000","last_became_active":"2024-09-12T03:00:58.346486+0000","last_became_peered":"2024-09-12T03:00:58.346486+0000","last_unstale":"2024-09-12T03:02:04.282483+0000","last_undegraded":"2024-09-12T03:02:04.282483+0000","last_fullsized":"2024-09-12T03:02:04.282483+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:28.687727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.283454+0000","last_change":"2024-09-12T03:00:58.346712+0000","last_active":"2024-09-12T03:02:03.283454+0000","last_peered":"2024-09-12T03:02:03.283454+0000","last_clean":"2024-09-12T03:02:03.283454+0000","last_became_active":"2024-09-12T03:00:58.346397+0000","last_became_peered":"2024-09-12T03:00:58.346397+0000","last_unstale":"2024-09-12T03:02:03.283454+0000","last_undegraded":"2024-09-12T03:02:03.283454+0000","last_fullsized":"2024-09-12T03:02:03.283454+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:48:00.624733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343665+0000","last_change":"2024-09-12T03:00:58.356166+0000","last_active":"2024-09-12T03:00:59.343665+0000","last_peered":"2024-09-12T03:00:59.343665+0000","last_clean":"2024-09-12T03:00:59.343665+0000","last_became_active":"2024-09-12T03:00:58.354944+0000","last_became_peered":"2024-09-12T03:00:58.354944+0000","last_unstale":"2024-09-12T03:00:59.343665+0000","last_undegraded":"2024-09-12T03:00:59.343665+0000","last_fullsized":"2024-09-12T03:00:59.343665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:15.840149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.329171+0000","last_change":"2024-09-12T03:00:58.340182+0000","last_active":"2024-09-12T03:00:59.329171+0000","last_peered":"2024-09-12T03:00:59.329171+0000","last_clean":"2024-09-12T03:00:59.329171+0000","last_became_active":"2024-09-12T03:00:58.339576+0000","last_became_peered":"2024-09-12T03:00:58.339576+0000","last_unstale":"2024-09-12T03:00:59.329171+0000","last_undegraded":"2024-09-12T03:00:59.329171+0000","last_fullsized":"2024-09-12T03:00:59.329171+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:16.534844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282812+0000","last_change":"2024-09-12T03:00:58.351218+0000","last_active":"2024-09-12T03:02:04.282812+0000","last_peered":"2024-09-12T03:02:04.282812+0000","last_clean":"2024-09-12T03:02:04.282812+0000","last_became_active":"2024-09-12T03:00:58.350871+0000","last_became_peered":"2024-09-12T03:00:58.350871+0000","last_unstale":"2024-09-12T03:02:04.282812+0000","last_undegraded":"2024-09-12T03:02:04.282812+0000","last_fullsized":"2024-09-12T03:02:04.282812+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:38.367683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.283798+0000","last_change":"2024-09-12T03:00:58.351894+0000","last_active":"2024-09-12T03:02:03.283798+0000","last_peered":"2024-09-12T03:02:03.283798+0000","last_clean":"2024-09-12T03:02:03.283798+0000","last_became_active":"2024-09-12T03:00:58.351445+0000","last_became_peered":"2024-09-12T03:00:58.351445+0000","last_unstale":"2024-09-12T03:02:03.283798+0000","last_undegraded":"2024-09-12T03:02:03.283798+0000","last_fullsized":"2024-09-12T03:02:03.283798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:42.093871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289589+0000","last_change":"2024-09-12T03:00:58.353460+0000","last_active":"2024-09-12T03:02:04.289589+0000","last_peered":"2024-09-12T03:02:04.289589+0000","last_clean":"2024-09-12T03:02:04.289589+0000","last_became_active":"2024-09-12T03:00:58.353111+0000","last_became_peered":"2024-09-12T03:00:58.353111+0000","last_unstale":"2024-09-12T03:02:04.289589+0000","last_undegraded":"2024-09-12T03:02:04.289589+0000","last_fullsized":"2024-09-12T03:02:04.289589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:15.916765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332731+0000","last_change":"2024-09-12T03:00:58.341080+0000","last_active":"2024-09-12T03:02:04.332731+0000","last_peered":"2024-09-12T03:02:04.332731+0000","last_clean":"2024-09-12T03:02:04.332731+0000","last_became_active":"2024-09-12T03:00:58.340782+0000","last_became_peered":"2024-09-12T03:00:58.340782+0000","last_unstale":"2024-09-12T03:02:04.332731+0000","last_undegraded":"2024-09-12T03:02:04.332731+0000","last_fullsized":"2024-09-12T03:02:04.332731+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:36.569700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342701+0000","last_change":"2024-09-12T03:00:58.346791+0000","last_active":"2024-09-12T03:00:59.342701+0000","last_peered":"2024-09-12T03:00:59.342701+0000","last_clean":"2024-09-12T03:00:59.342701+0000","last_became_active":"2024-09-12T03:00:58.346500+0000","last_became_peered":"2024-09-12T03:00:58.346500+0000","last_unstale":"2024-09-12T03:00:59.342701+0000","last_undegraded":"2024-09-12T03:00:59.342701+0000","last_fullsized":"2024-09-12T03:00:59.342701+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:32.017589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342415+0000","last_change":"2024-09-12T03:00:58.349522+0000","last_active":"2024-09-12T03:00:59.342415+0000","last_peered":"2024-09-12T03:00:59.342415+0000","last_clean":"2024-09-12T03:00:59.342415+0000","last_became_active":"2024-09-12T03:00:58.349246+0000","last_became_peered":"2024-09-12T03:00:58.349246+0000","last_unstale":"2024-09-12T03:00:59.342415+0000","last_undegraded":"2024-09-12T03:00:59.342415+0000","last_fullsized":"2024-09-12T03:00:59.342415+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:39.253919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328399+0000","last_change":"2024-09-12T03:00:58.349909+0000","last_active":"2024-09-12T03:00:59.328399+0000","last_peered":"2024-09-12T03:00:59.328399+0000","last_clean":"2024-09-12T03:00:59.328399+0000","last_became_active":"2024-09-12T03:00:58.349590+0000","last_became_peered":"2024-09-12T03:00:58.349590+0000","last_unstale":"2024-09-12T03:00:59.328399+0000","last_undegraded":"2024-09-12T03:00:59.328399+0000","last_fullsized":"2024-09-12T03:00:59.328399+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:04.540709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.349104+0000","last_change":"2024-09-12T03:00:58.348284+0000","last_active":"2024-09-12T03:00:59.349104+0000","last_peered":"2024-09-12T03:00:59.349104+0000","last_clean":"2024-09-12T03:00:59.349104+0000","last_became_active":"2024-09-12T03:00:58.347465+0000","last_became_peered":"2024-09-12T03:00:58.347465+0000","last_unstale":"2024-09-12T03:00:59.349104+0000","last_undegraded":"2024-09-12T03:00:59.349104+0000","last_fullsized":"2024-09-12T03:00:59.349104+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:13.191899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284483+0000","last_change":"2024-09-12T03:00:58.337740+0000","last_active":"2024-09-12T03:02:04.284483+0000","last_peered":"2024-09-12T03:02:04.284483+0000","last_clean":"2024-09-12T03:02:04.284483+0000","last_became_active":"2024-09-12T03:00:58.337393+0000","last_became_peered":"2024-09-12T03:00:58.337393+0000","last_unstale":"2024-09-12T03:02:04.284483+0000","last_undegraded":"2024-09-12T03:02:04.284483+0000","last_fullsized":"2024-09-12T03:02:04.284483+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:28:56.339167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344750+0000","last_change":"2024-09-12T03:00:58.345779+0000","last_active":"2024-09-12T03:00:59.344750+0000","last_peered":"2024-09-12T03:00:59.344750+0000","last_clean":"2024-09-12T03:00:59.344750+0000","last_became_active":"2024-09-12T03:00:58.345210+0000","last_became_peered":"2024-09-12T03:00:58.345210+0000","last_unstale":"2024-09-12T03:00:59.344750+0000","last_undegraded":"2024-09-12T03:00:59.344750+0000","last_fullsized":"2024-09-12T03:00:59.344750+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:06:58.271185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.348332+0000","last_change":"2024-09-12T03:00:58.343877+0000","last_active":"2024-09-12T03:00:59.348332+0000","last_peered":"2024-09-12T03:00:59.348332+0000","last_clean":"2024-09-12T03:00:59.348332+0000","last_became_active":"2024-09-12T03:00:58.343545+0000","last_became_peered":"2024-09-12T03:00:58.343545+0000","last_unstale":"2024-09-12T03:00:59.348332+0000","last_undegraded":"2024-09-12T03:00:59.348332+0000","last_fullsized":"2024-09-12T03:00:59.348332+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:52:31.851975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332823+0000","last_change":"2024-09-12T03:00:58.349233+0000","last_active":"2024-09-12T03:02:04.332823+0000","last_peered":"2024-09-12T03:02:04.332823+0000","last_clean":"2024-09-12T03:02:04.332823+0000","last_became_active":"2024-09-12T03:00:58.348958+0000","last_became_peered":"2024-09-12T03:00:58.348958+0000","last_unstale":"2024-09-12T03:02:04.332823+0000","last_undegraded":"2024-09-12T03:02:04.332823+0000","last_fullsized":"2024-09-12T03:02:04.332823+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:32:10.271817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328487+0000","last_change":"2024-09-12T03:00:58.358024+0000","last_active":"2024-09-12T03:00:59.328487+0000","last_peered":"2024-09-12T03:00:59.328487+0000","last_clean":"2024-09-12T03:00:59.328487+0000","last_became_active":"2024-09-12T03:00:58.357434+0000","last_became_peered":"2024-09-12T03:00:58.357434+0000","last_unstale":"2024-09-12T03:00:59.328487+0000","last_undegraded":"2024-09-12T03:00:59.328487+0000","last_fullsized":"2024-09-12T03:00:59.328487+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:01:20.119733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.345193+0000","last_change":"2024-09-12T03:00:58.355063+0000","last_active":"2024-09-12T03:00:59.345193+0000","last_peered":"2024-09-12T03:00:59.345193+0000","last_clean":"2024-09-12T03:00:59.345193+0000","last_became_active":"2024-09-12T03:00:58.354760+0000","last_became_peered":"2024-09-12T03:00:58.354760+0000","last_unstale":"2024-09-12T03:00:59.345193+0000","last_undegraded":"2024-09-12T03:00:59.345193+0000","last_fullsized":"2024-09-12T03:00:59.345193+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:13.464413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.280040+0000","last_change":"2024-09-12T03:00:58.341916+0000","last_active":"2024-09-12T03:02:03.280040+0000","last_peered":"2024-09-12T03:02:03.280040+0000","last_clean":"2024-09-12T03:02:03.280040+0000","last_became_active":"2024-09-12T03:00:58.340815+0000","last_became_peered":"2024-09-12T03:00:58.340815+0000","last_unstale":"2024-09-12T03:02:03.280040+0000","last_undegraded":"2024-09-12T03:02:03.280040+0000","last_fullsized":"2024-09-12T03:02:03.280040+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:13.777947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343925+0000","last_change":"2024-09-12T03:00:58.357765+0000","last_active":"2024-09-12T03:00:59.343925+0000","last_peered":"2024-09-12T03:00:59.343925+0000","last_clean":"2024-09-12T03:00:59.343925+0000","last_became_active":"2024-09-12T03:00:58.357480+0000","last_became_peered":"2024-09-12T03:00:58.357480+0000","last_unstale":"2024-09-12T03:00:59.343925+0000","last_undegraded":"2024-09-12T03:00:59.343925+0000","last_fullsized":"2024-09-12T03:00:59.343925+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:26.884381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344699+0000","last_change":"2024-09-12T03:00:58.349504+0000","last_active":"2024-09-12T03:00:59.344699+0000","last_peered":"2024-09-12T03:00:59.344699+0000","last_clean":"2024-09-12T03:00:59.344699+0000","last_became_active":"2024-09-12T03:00:58.349136+0000","last_became_peered":"2024-09-12T03:00:58.349136+0000","last_unstale":"2024-09-12T03:00:59.344699+0000","last_undegraded":"2024-09-12T03:00:59.344699+0000","last_fullsized":"2024-09-12T03:00:59.344699+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:09:55.229516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.348719+0000","last_change":"2024-09-12T03:00:58.341902+0000","last_active":"2024-09-12T03:00:59.348719+0000","last_peered":"2024-09-12T03:00:59.348719+0000","last_clean":"2024-09-12T03:00:59.348719+0000","last_became_active":"2024-09-12T03:00:58.341592+0000","last_became_peered":"2024-09-12T03:00:58.341592+0000","last_unstale":"2024-09-12T03:00:59.348719+0000","last_undegraded":"2024-09-12T03:00:59.348719+0000","last_fullsized":"2024-09-12T03:00:59.348719+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:13.852511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332298+0000","last_change":"2024-09-12T03:00:58.348012+0000","last_active":"2024-09-12T03:02:04.332298+0000","last_peered":"2024-09-12T03:02:04.332298+0000","last_clean":"2024-09-12T03:02:04.332298+0000","last_became_active":"2024-09-12T03:00:58.347338+0000","last_became_peered":"2024-09-12T03:00:58.347338+0000","last_unstale":"2024-09-12T03:02:04.332298+0000","last_undegraded":"2024-09-12T03:02:04.332298+0000","last_fullsized":"2024-09-12T03:02:04.332298+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:41.478032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342928+0000","last_change":"2024-09-12T03:00:58.345833+0000","last_active":"2024-09-12T03:00:59.342928+0000","last_peered":"2024-09-12T03:00:59.342928+0000","last_clean":"2024-09-12T03:00:59.342928+0000","last_became_active":"2024-09-12T03:00:58.345482+0000","last_became_peered":"2024-09-12T03:00:58.345482+0000","last_unstale":"2024-09-12T03:00:59.342928+0000","last_undegraded":"2024-09-12T03:00:59.342928+0000","last_fullsized":"2024-09-12T03:00:59.342928+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:32.037412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343059+0000","last_change":"2024-09-12T03:00:58.352290+0000","last_active":"2024-09-12T03:00:59.343059+0000","last_peered":"2024-09-12T03:00:59.343059+0000","last_clean":"2024-09-12T03:00:59.343059+0000","last_became_active":"2024-09-12T03:00:58.351918+0000","last_became_peered":"2024-09-12T03:00:58.351918+0000","last_unstale":"2024-09-12T03:00:59.343059+0000","last_undegraded":"2024-09-12T03:00:59.343059+0000","last_fullsized":"2024-09-12T03:00:59.343059+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:25.941080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343279+0000","last_change":"2024-09-12T03:00:58.348021+0000","last_active":"2024-09-12T03:00:59.343279+0000","last_peered":"2024-09-12T03:00:59.343279+0000","last_clean":"2024-09-12T03:00:59.343279+0000","last_became_active":"2024-09-12T03:00:58.347634+0000","last_became_peered":"2024-09-12T03:00:58.347634+0000","last_unstale":"2024-09-12T03:00:59.343279+0000","last_undegraded":"2024-09-12T03:00:59.343279+0000","last_fullsized":"2024-09-12T03:00:59.343279+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:47.933067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.347866+0000","last_change":"2024-09-12T03:00:58.340891+0000","last_active":"2024-09-12T03:00:59.347866+0000","last_peered":"2024-09-12T03:00:59.347866+0000","last_clean":"2024-09-12T03:00:59.347866+0000","last_became_active":"2024-09-12T03:00:58.340504+0000","last_became_peered":"2024-09-12T03:00:58.340504+0000","last_unstale":"2024-09-12T03:00:59.347866+0000","last_undegraded":"2024-09-12T03:00:59.347866+0000","last_fullsized":"2024-09-12T03:00:59.347866+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:32.117635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.291866+0000","last_change":"2024-09-12T03:00:58.339343+0000","last_active":"2024-09-12T03:02:04.291866+0000","last_peered":"2024-09-12T03:02:04.291866+0000","last_clean":"2024-09-12T03:02:04.291866+0000","last_became_active":"2024-09-12T03:00:58.339001+0000","last_became_peered":"2024-09-12T03:00:58.339001+0000","last_unstale":"2024-09-12T03:02:04.291866+0000","last_undegraded":"2024-09-12T03:02:04.291866+0000","last_fullsized":"2024-09-12T03:02:04.291866+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:02.043454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":83,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.281673+0000","last_change":"2024-09-12T03:00:56.315550+0000","last_active":"2024-09-12T03:02:03.281673+0000","last_peered":"2024-09-12T03:02:03.281673+0000","last_clean":"2024-09-12T03:02:03.281673+0000","last_became_active":"2024-09-12T03:00:56.315225+0000","last_became_peered":"2024-09-12T03:00:56.315225+0000","last_unstale":"2024-09-12T03:02:03.281673+0000","last_undegraded":"2024-09-12T03:02:03.281673+0000","last_fullsized":"2024-09-12T03:02:03.281673+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:46.070895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.290858+0000","last_change":"2024-09-12T03:02:04.290498+0000","last_active":"2024-09-12T03:02:04.290858+0000","last_peered":"2024-09-12T03:02:04.290858+0000","last_clean":"2024-09-12T03:02:04.290858+0000","last_became_active":"2024-09-12T03:02:04.290162+0000","last_became_peered":"2024-09-12T03:02:04.290162+0000","last_unstale":"2024-09-12T03:02:04.290858+0000","last_undegraded":"2024-09-12T03:02:04.290858+0000","last_fullsized":"2024-09-12T03:02:04.290858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:29.777155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282566+0000","last_change":"2024-09-12T03:00:56.316385+0000","last_active":"2024-09-12T03:02:04.282566+0000","last_peered":"2024-09-12T03:02:04.282566+0000","last_clean":"2024-09-12T03:02:04.282566+0000","last_became_active":"2024-09-12T03:00:56.316064+0000","last_became_peered":"2024-09-12T03:00:56.316064+0000","last_unstale":"2024-09-12T03:02:04.282566+0000","last_undegraded":"2024-09-12T03:02:04.282566+0000","last_fullsized":"2024-09-12T03:02:04.282566+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:12:43.323654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332424+0000","last_change":"2024-09-12T03:00:56.313065+0000","last_active":"2024-09-12T03:02:04.332424+0000","last_peered":"2024-09-12T03:02:04.332424+0000","last_clean":"2024-09-12T03:02:04.332424+0000","last_became_active":"2024-09-12T03:00:56.312716+0000","last_became_peered":"2024-09-12T03:00:56.312716+0000","last_unstale":"2024-09-12T03:02:04.332424+0000","last_undegraded":"2024-09-12T03:02:04.332424+0000","last_fullsized":"2024-09-12T03:02:04.332424+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:51.089089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.346221+0000","last_change":"2024-09-12T03:00:58.346542+0000","last_active":"2024-09-12T03:00:59.346221+0000","last_peered":"2024-09-12T03:00:59.346221+0000","last_clean":"2024-09-12T03:00:59.346221+0000","last_became_active":"2024-09-12T03:00:58.346260+0000","last_became_peered":"2024-09-12T03:00:58.346260+0000","last_unstale":"2024-09-12T03:00:59.346221+0000","last_undegraded":"2024-09-12T03:00:59.346221+0000","last_fullsized":"2024-09-12T03:00:59.346221+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:37.034969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.280332+0000","last_change":"2024-09-12T03:00:56.317308+0000","last_active":"2024-09-12T03:02:03.280332+0000","last_peered":"2024-09-12T03:02:03.280332+0000","last_clean":"2024-09-12T03:02:03.280332+0000","last_became_active":"2024-09-12T03:00:56.316935+0000","last_became_peered":"2024-09-12T03:00:56.316935+0000","last_unstale":"2024-09-12T03:02:03.280332+0000","last_undegraded":"2024-09-12T03:02:03.280332+0000","last_fullsized":"2024-09-12T03:02:03.280332+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:06.324489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.283990+0000","last_change":"2024-09-12T03:00:58.341565+0000","last_active":"2024-09-12T03:02:04.283990+0000","last_peered":"2024-09-12T03:02:04.283990+0000","last_clean":"2024-09-12T03:02:04.283990+0000","last_became_active":"2024-09-12T03:00:58.341285+0000","last_became_peered":"2024-09-12T03:00:58.341285+0000","last_unstale":"2024-09-12T03:02:04.283990+0000","last_undegraded":"2024-09-12T03:02:04.283990+0000","last_fullsized":"2024-09-12T03:02:04.283990+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:59.156700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":83,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.483794+0000","last_change":"2024-09-12T03:00:56.314201+0000","last_active":"2024-09-12T03:02:04.483794+0000","last_peered":"2024-09-12T03:02:04.483794+0000","last_clean":"2024-09-12T03:02:04.483794+0000","last_became_active":"2024-09-12T03:00:56.313880+0000","last_became_peered":"2024-09-12T03:00:56.313880+0000","last_unstale":"2024-09-12T03:02:04.483794+0000","last_undegraded":"2024-09-12T03:02:04.483794+0000","last_fullsized":"2024-09-12T03:02:04.483794+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:57:12.952630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.329144+0000","last_change":"2024-09-12T03:00:58.345274+0000","last_active":"2024-09-12T03:00:59.329144+0000","last_peered":"2024-09-12T03:00:59.329144+0000","last_clean":"2024-09-12T03:00:59.329144+0000","last_became_active":"2024-09-12T03:00:58.344937+0000","last_became_peered":"2024-09-12T03:00:58.344937+0000","last_unstale":"2024-09-12T03:00:59.329144+0000","last_undegraded":"2024-09-12T03:00:59.329144+0000","last_fullsized":"2024-09-12T03:00:59.329144+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:17.288909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.283992+0000","last_change":"2024-09-12T03:00:56.314709+0000","last_active":"2024-09-12T03:02:03.283992+0000","last_peered":"2024-09-12T03:02:03.283992+0000","last_clean":"2024-09-12T03:02:03.283992+0000","last_became_active":"2024-09-12T03:00:56.314349+0000","last_became_peered":"2024-09-12T03:00:56.314349+0000","last_unstale":"2024-09-12T03:02:03.283992+0000","last_undegraded":"2024-09-12T03:02:03.283992+0000","last_fullsized":"2024-09-12T03:02:03.283992+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:18:35.371661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343140+0000","last_change":"2024-09-12T03:00:58.346055+0000","last_active":"2024-09-12T03:00:59.343140+0000","last_peered":"2024-09-12T03:00:59.343140+0000","last_clean":"2024-09-12T03:00:59.343140+0000","last_became_active":"2024-09-12T03:00:58.345701+0000","last_became_peered":"2024-09-12T03:00:58.345701+0000","last_unstale":"2024-09-12T03:00:59.343140+0000","last_undegraded":"2024-09-12T03:00:59.343140+0000","last_fullsized":"2024-09-12T03:00:59.343140+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:33.977245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.282131+0000","last_change":"2024-09-12T03:00:56.435962+0000","last_active":"2024-09-12T03:02:03.282131+0000","last_peered":"2024-09-12T03:02:03.282131+0000","last_clean":"2024-09-12T03:02:03.282131+0000","last_became_active":"2024-09-12T03:00:56.435515+0000","last_became_peered":"2024-09-12T03:00:56.435515+0000","last_unstale":"2024-09-12T03:02:03.282131+0000","last_undegraded":"2024-09-12T03:02:03.282131+0000","last_fullsized":"2024-09-12T03:02:03.282131+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:32.953436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328662+0000","last_change":"2024-09-12T03:00:58.344239+0000","last_active":"2024-09-12T03:00:59.328662+0000","last_peered":"2024-09-12T03:00:59.328662+0000","last_clean":"2024-09-12T03:00:59.328662+0000","last_became_active":"2024-09-12T03:00:58.343876+0000","last_became_peered":"2024-09-12T03:00:58.343876+0000","last_unstale":"2024-09-12T03:00:59.328662+0000","last_undegraded":"2024-09-12T03:00:59.328662+0000","last_fullsized":"2024-09-12T03:00:59.328662+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:49.401295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'29","reported_seq":176,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:05.485172+0000","last_change":"2024-09-12T03:00:56.317708+0000","last_active":"2024-09-12T03:02:05.485172+0000","last_peered":"2024-09-12T03:02:05.485172+0000","last_clean":"2024-09-12T03:02:05.485172+0000","last_became_active":"2024-09-12T03:00:56.317317+0000","last_became_peered":"2024-09-12T03:00:56.317317+0000","last_unstale":"2024-09-12T03:02:05.485172+0000","last_undegraded":"2024-09-12T03:02:05.485172+0000","last_fullsized":"2024-09-12T03:02:05.485172+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:47.788186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61752,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":125,"num_read_kb":122,"num_write":30,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292083+0000","last_change":"2024-09-12T03:00:58.343602+0000","last_active":"2024-09-12T03:02:04.292083+0000","last_peered":"2024-09-12T03:02:04.292083+0000","last_clean":"2024-09-12T03:02:04.292083+0000","last_became_active":"2024-09-12T03:00:58.343301+0000","last_became_peered":"2024-09-12T03:00:58.343301+0000","last_unstale":"2024-09-12T03:02:04.292083+0000","last_undegraded":"2024-09-12T03:02:04.292083+0000","last_fullsized":"2024-09-12T03:02:04.292083+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:04.056118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.398339+0000","last_change":"2024-09-12T03:00:56.317874+0000","last_active":"2024-09-12T03:01:07.398339+0000","last_peered":"2024-09-12T03:01:07.398339+0000","last_clean":"2024-09-12T03:01:07.398339+0000","last_became_active":"2024-09-12T03:00:56.317566+0000","last_became_peered":"2024-09-12T03:00:56.317566+0000","last_unstale":"2024-09-12T03:01:07.398339+0000","last_undegraded":"2024-09-12T03:01:07.398339+0000","last_fullsized":"2024-09-12T03:01:07.398339+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:54.203730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332601+0000","last_change":"2024-09-12T03:00:58.351539+0000","last_active":"2024-09-12T03:02:04.332601+0000","last_peered":"2024-09-12T03:02:04.332601+0000","last_clean":"2024-09-12T03:02:04.332601+0000","last_became_active":"2024-09-12T03:00:58.351249+0000","last_became_peered":"2024-09-12T03:00:58.351249+0000","last_unstale":"2024-09-12T03:02:04.332601+0000","last_undegraded":"2024-09-12T03:02:04.332601+0000","last_fullsized":"2024-09-12T03:02:04.332601+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:40.498867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.280655+0000","last_change":"2024-09-12T03:00:56.314489+0000","last_active":"2024-09-12T03:02:03.280655+0000","last_peered":"2024-09-12T03:02:03.280655+0000","last_clean":"2024-09-12T03:02:03.280655+0000","last_became_active":"2024-09-12T03:00:56.314142+0000","last_became_peered":"2024-09-12T03:00:56.314142+0000","last_unstale":"2024-09-12T03:02:03.280655+0000","last_undegraded":"2024-09-12T03:02:03.280655+0000","last_fullsized":"2024-09-12T03:02:03.280655+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:21.541713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.283805+0000","last_change":"2024-09-12T03:00:58.348752+0000","last_active":"2024-09-12T03:02:03.283805+0000","last_peered":"2024-09-12T03:02:03.283805+0000","last_clean":"2024-09-12T03:02:03.283805+0000","last_became_active":"2024-09-12T03:00:58.348391+0000","last_became_peered":"2024-09-12T03:00:58.348391+0000","last_unstale":"2024-09-12T03:02:03.283805+0000","last_undegraded":"2024-09-12T03:02:03.283805+0000","last_fullsized":"2024-09-12T03:02:03.283805+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:33:15.626306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'3","reported_seq":84,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289366+0000","last_change":"2024-09-12T03:00:56.311702+0000","last_active":"2024-09-12T03:02:04.289366+0000","last_peered":"2024-09-12T03:02:04.289366+0000","last_clean":"2024-09-12T03:02:04.289366+0000","last_became_active":"2024-09-12T03:00:56.311353+0000","last_became_peered":"2024-09-12T03:00:56.311353+0000","last_unstale":"2024-09-12T03:02:04.289366+0000","last_undegraded":"2024-09-12T03:02:04.289366+0000","last_fullsized":"2024-09-12T03:02:04.289366+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:01.004858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":60,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.283381+0000","last_change":"2024-09-12T03:00:58.351973+0000","last_active":"2024-09-12T03:02:03.283381+0000","last_peered":"2024-09-12T03:02:03.283381+0000","last_clean":"2024-09-12T03:02:03.283381+0000","last_became_active":"2024-09-12T03:00:58.351599+0000","last_became_peered":"2024-09-12T03:00:58.351599+0000","last_unstale":"2024-09-12T03:02:03.283381+0000","last_undegraded":"2024-09-12T03:02:03.283381+0000","last_fullsized":"2024-09-12T03:02:03.283381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:26.398828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.291749+0000","last_change":"2024-09-12T03:00:56.436185+0000","last_active":"2024-09-12T03:02:04.291749+0000","last_peered":"2024-09-12T03:02:04.291749+0000","last_clean":"2024-09-12T03:02:04.291749+0000","last_became_active":"2024-09-12T03:00:56.435717+0000","last_became_peered":"2024-09-12T03:00:56.435717+0000","last_unstale":"2024-09-12T03:02:04.291749+0000","last_undegraded":"2024-09-12T03:02:04.291749+0000","last_fullsized":"2024-09-12T03:02:04.291749+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:27.303182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":78,"state":"peering","last_fresh":"2024-09-12T03:00:59.328638+0000","last_change":"2024-09-12T03:02:03.280677+0000","last_active":"2024-09-12T03:02:03.280677+0000","last_peered":"2024-09-12T03:00:59.328638+0000","last_clean":"2024-09-12T03:00:59.328638+0000","last_became_active":"2024-09-12T03:00:58.346490+0000","last_became_peered":"2024-09-12T03:00:58.346490+0000","last_unstale":"2024-09-12T03:00:59.328638+0000","last_undegraded":"2024-09-12T03:00:59.328638+0000","last_fullsized":"2024-09-12T03:00:59.328638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4,8],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:08.394235+0000","last_change":"2024-09-12T03:00:56.311333+0000","last_active":"2024-09-12T03:01:08.394235+0000","last_peered":"2024-09-12T03:01:08.394235+0000","last_clean":"2024-09-12T03:01:08.394235+0000","last_became_active":"2024-09-12T03:00:56.310871+0000","last_became_peered":"2024-09-12T03:00:56.310871+0000","last_unstale":"2024-09-12T03:01:08.394235+0000","last_undegraded":"2024-09-12T03:01:08.394235+0000","last_fullsized":"2024-09-12T03:01:08.394235+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:59.344572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.348769+0000","last_change":"2024-09-12T03:00:58.349401+0000","last_active":"2024-09-12T03:00:59.348769+0000","last_peered":"2024-09-12T03:00:59.348769+0000","last_clean":"2024-09-12T03:00:59.348769+0000","last_became_active":"2024-09-12T03:00:58.348996+0000","last_became_peered":"2024-09-12T03:00:58.348996+0000","last_unstale":"2024-09-12T03:00:59.348769+0000","last_undegraded":"2024-09-12T03:00:59.348769+0000","last_fullsized":"2024-09-12T03:00:59.348769+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:20:26.201684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332966+0000","last_change":"2024-09-12T03:00:56.314182+0000","last_active":"2024-09-12T03:02:04.332966+0000","last_peered":"2024-09-12T03:02:04.332966+0000","last_clean":"2024-09-12T03:02:04.332966+0000","last_became_active":"2024-09-12T03:00:56.313749+0000","last_became_peered":"2024-09-12T03:00:56.313749+0000","last_unstale":"2024-09-12T03:02:04.332966+0000","last_undegraded":"2024-09-12T03:02:04.332966+0000","last_fullsized":"2024-09-12T03:02:04.332966+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:31:45.767500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.284069+0000","last_change":"2024-09-12T03:00:58.344356+0000","last_active":"2024-09-12T03:02:03.284069+0000","last_peered":"2024-09-12T03:02:03.284069+0000","last_clean":"2024-09-12T03:02:03.284069+0000","last_became_active":"2024-09-12T03:00:58.344035+0000","last_became_peered":"2024-09-12T03:00:58.344035+0000","last_unstale":"2024-09-12T03:02:03.284069+0000","last_undegraded":"2024-09-12T03:02:03.284069+0000","last_fullsized":"2024-09-12T03:02:03.284069+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:45:18.681018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.282252+0000","last_change":"2024-09-12T03:00:56.315653+0000","last_active":"2024-09-12T03:02:03.282252+0000","last_peered":"2024-09-12T03:02:03.282252+0000","last_clean":"2024-09-12T03:02:03.282252+0000","last_became_active":"2024-09-12T03:00:56.315284+0000","last_became_peered":"2024-09-12T03:00:56.315284+0000","last_unstale":"2024-09-12T03:02:03.282252+0000","last_undegraded":"2024-09-12T03:02:03.282252+0000","last_fullsized":"2024-09-12T03:02:03.282252+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:42.934020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":15,"reported_epoch":78,"state":"peering","last_fresh":"2024-09-12T03:01:17.267451+0000","last_change":"2024-09-12T03:02:03.283802+0000","last_active":"2024-09-12T03:02:03.283802+0000","last_peered":"2024-09-12T03:01:17.267451+0000","last_clean":"2024-09-12T03:01:17.267451+0000","last_became_active":"2024-09-12T03:00:58.357451+0000","last_became_peered":"2024-09-12T03:00:58.357451+0000","last_unstale":"2024-09-12T03:01:17.267451+0000","last_undegraded":"2024-09-12T03:01:17.267451+0000","last_fullsized":"2024-09-12T03:01:17.267451+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5,9],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:07.398109+0000","last_change":"2024-09-12T03:00:56.746780+0000","last_active":"2024-09-12T03:01:07.398109+0000","last_peered":"2024-09-12T03:01:07.398109+0000","last_clean":"2024-09-12T03:01:07.398109+0000","last_became_active":"2024-09-12T03:00:56.746414+0000","last_became_peered":"2024-09-12T03:00:56.746414+0000","last_unstale":"2024-09-12T03:01:07.398109+0000","last_undegraded":"2024-09-12T03:01:07.398109+0000","last_fullsized":"2024-09-12T03:01:07.398109+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:15:35.214196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.283877+0000","last_change":"2024-09-12T03:00:58.351713+0000","last_active":"2024-09-12T03:02:03.283877+0000","last_peered":"2024-09-12T03:02:03.283877+0000","last_clean":"2024-09-12T03:02:03.283877+0000","last_became_active":"2024-09-12T03:00:58.351371+0000","last_became_peered":"2024-09-12T03:00:58.351371+0000","last_unstale":"2024-09-12T03:02:03.283877+0000","last_undegraded":"2024-09-12T03:02:03.283877+0000","last_fullsized":"2024-09-12T03:02:03.283877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:13.346956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'137","reported_seq":333,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.283543+0000","last_change":"2024-09-12T02:59:45.461482+0000","last_active":"2024-09-12T03:02:03.283543+0000","last_peered":"2024-09-12T03:02:03.283543+0000","last_clean":"2024-09-12T03:02:03.283543+0000","last_became_active":"2024-09-12T02:59:45.461058+0000","last_became_peered":"2024-09-12T02:59:45.461058+0000","last_unstale":"2024-09-12T03:02:03.283543+0000","last_undegraded":"2024-09-12T03:02:03.283543+0000","last_fullsized":"2024-09-12T03:02:03.283543+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:18.034438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:18.034438+0000","last_clean_scrub_stamp":"2024-09-12T02:56:18.034438+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:09.251858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289291+0000","last_change":"2024-09-12T03:00:56.315245+0000","last_active":"2024-09-12T03:02:04.289291+0000","last_peered":"2024-09-12T03:02:04.289291+0000","last_clean":"2024-09-12T03:02:04.289291+0000","last_became_active":"2024-09-12T03:00:56.314887+0000","last_became_peered":"2024-09-12T03:00:56.314887+0000","last_unstale":"2024-09-12T03:02:04.289291+0000","last_undegraded":"2024-09-12T03:02:04.289291+0000","last_fullsized":"2024-09-12T03:02:04.289291+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:48.750657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344703+0000","last_change":"2024-09-12T03:00:58.357874+0000","last_active":"2024-09-12T03:00:59.344703+0000","last_peered":"2024-09-12T03:00:59.344703+0000","last_clean":"2024-09-12T03:00:59.344703+0000","last_became_active":"2024-09-12T03:00:58.357575+0000","last_became_peered":"2024-09-12T03:00:58.357575+0000","last_unstale":"2024-09-12T03:00:59.344703+0000","last_undegraded":"2024-09-12T03:00:59.344703+0000","last_fullsized":"2024-09-12T03:00:59.344703+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:46.125620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.283498+0000","last_change":"2024-09-12T03:00:56.312263+0000","last_active":"2024-09-12T03:02:03.283498+0000","last_peered":"2024-09-12T03:02:03.283498+0000","last_clean":"2024-09-12T03:02:03.283498+0000","last_became_active":"2024-09-12T03:00:56.311894+0000","last_became_peered":"2024-09-12T03:00:56.311894+0000","last_unstale":"2024-09-12T03:02:03.283498+0000","last_undegraded":"2024-09-12T03:02:03.283498+0000","last_fullsized":"2024-09-12T03:02:03.283498+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:52.434203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332322+0000","last_change":"2024-09-12T03:00:58.350762+0000","last_active":"2024-09-12T03:02:04.332322+0000","last_peered":"2024-09-12T03:02:04.332322+0000","last_clean":"2024-09-12T03:02:04.332322+0000","last_became_active":"2024-09-12T03:00:58.348038+0000","last_became_peered":"2024-09-12T03:00:58.348038+0000","last_unstale":"2024-09-12T03:02:04.332322+0000","last_undegraded":"2024-09-12T03:02:04.332322+0000","last_fullsized":"2024-09-12T03:02:04.332322+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:08.916636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":84,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.483168+0000","last_change":"2024-09-12T03:00:56.435641+0000","last_active":"2024-09-12T03:02:04.483168+0000","last_peered":"2024-09-12T03:02:04.483168+0000","last_clean":"2024-09-12T03:02:04.483168+0000","last_became_active":"2024-09-12T03:00:56.435319+0000","last_became_peered":"2024-09-12T03:00:56.435319+0000","last_unstale":"2024-09-12T03:02:04.483168+0000","last_undegraded":"2024-09-12T03:02:04.483168+0000","last_fullsized":"2024-09-12T03:02:04.483168+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:56.894978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.280141+0000","last_change":"2024-09-12T03:00:58.344708+0000","last_active":"2024-09-12T03:02:03.280141+0000","last_peered":"2024-09-12T03:02:03.280141+0000","last_clean":"2024-09-12T03:02:03.280141+0000","last_became_active":"2024-09-12T03:00:58.344389+0000","last_became_peered":"2024-09-12T03:00:58.344389+0000","last_unstale":"2024-09-12T03:02:03.280141+0000","last_undegraded":"2024-09-12T03:02:03.280141+0000","last_fullsized":"2024-09-12T03:02:03.280141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:08.160383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.283775+0000","last_change":"2024-09-12T03:00:56.317068+0000","last_active":"2024-09-12T03:02:03.283775+0000","last_peered":"2024-09-12T03:02:03.283775+0000","last_clean":"2024-09-12T03:02:03.283775+0000","last_became_active":"2024-09-12T03:00:56.316698+0000","last_became_peered":"2024-09-12T03:00:56.316698+0000","last_unstale":"2024-09-12T03:02:03.283775+0000","last_undegraded":"2024-09-12T03:02:03.283775+0000","last_fullsized":"2024-09-12T03:02:03.283775+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:41:19.199418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282614+0000","last_change":"2024-09-12T03:00:58.347814+0000","last_active":"2024-09-12T03:02:04.282614+0000","last_peered":"2024-09-12T03:02:04.282614+0000","last_clean":"2024-09-12T03:02:04.282614+0000","last_became_active":"2024-09-12T03:00:58.347112+0000","last_became_peered":"2024-09-12T03:00:58.347112+0000","last_unstale":"2024-09-12T03:02:04.282614+0000","last_undegraded":"2024-09-12T03:02:04.282614+0000","last_fullsized":"2024-09-12T03:02:04.282614+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:59.325113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284265+0000","last_change":"2024-09-12T03:00:56.435771+0000","last_active":"2024-09-12T03:02:04.284265+0000","last_peered":"2024-09-12T03:02:04.284265+0000","last_clean":"2024-09-12T03:02:04.284265+0000","last_became_active":"2024-09-12T03:00:56.435448+0000","last_became_peered":"2024-09-12T03:00:56.435448+0000","last_unstale":"2024-09-12T03:02:04.284265+0000","last_undegraded":"2024-09-12T03:02:04.284265+0000","last_fullsized":"2024-09-12T03:02:04.284265+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:33.650338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282832+0000","last_change":"2024-09-12T03:00:58.341071+0000","last_active":"2024-09-12T03:02:04.282832+0000","last_peered":"2024-09-12T03:02:04.282832+0000","last_clean":"2024-09-12T03:02:04.282832+0000","last_became_active":"2024-09-12T03:00:58.340753+0000","last_became_peered":"2024-09-12T03:00:58.340753+0000","last_unstale":"2024-09-12T03:02:04.282832+0000","last_undegraded":"2024-09-12T03:02:04.282832+0000","last_fullsized":"2024-09-12T03:02:04.282832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:21.217915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292254+0000","last_change":"2024-09-12T03:00:56.436058+0000","last_active":"2024-09-12T03:02:04.292254+0000","last_peered":"2024-09-12T03:02:04.292254+0000","last_clean":"2024-09-12T03:02:04.292254+0000","last_became_active":"2024-09-12T03:00:56.435518+0000","last_became_peered":"2024-09-12T03:00:56.435518+0000","last_unstale":"2024-09-12T03:02:04.292254+0000","last_undegraded":"2024-09-12T03:02:04.292254+0000","last_fullsized":"2024-09-12T03:02:04.292254+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:32.235097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332392+0000","last_change":"2024-09-12T03:00:58.348236+0000","last_active":"2024-09-12T03:02:04.332392+0000","last_peered":"2024-09-12T03:02:04.332392+0000","last_clean":"2024-09-12T03:02:04.332392+0000","last_became_active":"2024-09-12T03:00:58.347727+0000","last_became_peered":"2024-09-12T03:00:58.347727+0000","last_unstale":"2024-09-12T03:02:04.332392+0000","last_undegraded":"2024-09-12T03:02:04.332392+0000","last_fullsized":"2024-09-12T03:02:04.332392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:19.233163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284575+0000","last_change":"2024-09-12T03:00:56.310537+0000","last_active":"2024-09-12T03:02:04.284575+0000","last_peered":"2024-09-12T03:02:04.284575+0000","last_clean":"2024-09-12T03:02:04.284575+0000","last_became_active":"2024-09-12T03:00:56.310135+0000","last_became_peered":"2024-09-12T03:00:56.310135+0000","last_unstale":"2024-09-12T03:02:04.284575+0000","last_undegraded":"2024-09-12T03:02:04.284575+0000","last_fullsized":"2024-09-12T03:02:04.284575+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:10:55.584339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.344075+0000","last_change":"2024-09-12T03:00:58.358417+0000","last_active":"2024-09-12T03:00:59.344075+0000","last_peered":"2024-09-12T03:00:59.344075+0000","last_clean":"2024-09-12T03:00:59.344075+0000","last_became_active":"2024-09-12T03:00:58.358119+0000","last_became_peered":"2024-09-12T03:00:58.358119+0000","last_unstale":"2024-09-12T03:00:59.344075+0000","last_undegraded":"2024-09-12T03:00:59.344075+0000","last_fullsized":"2024-09-12T03:00:59.344075+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:10.897594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289526+0000","last_change":"2024-09-12T03:00:56.747044+0000","last_active":"2024-09-12T03:02:04.289526+0000","last_peered":"2024-09-12T03:02:04.289526+0000","last_clean":"2024-09-12T03:02:04.289526+0000","last_became_active":"2024-09-12T03:00:56.746699+0000","last_became_peered":"2024-09-12T03:00:56.746699+0000","last_unstale":"2024-09-12T03:02:04.289526+0000","last_undegraded":"2024-09-12T03:02:04.289526+0000","last_fullsized":"2024-09-12T03:02:04.289526+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:11.420109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342687+0000","last_change":"2024-09-12T03:00:58.346854+0000","last_active":"2024-09-12T03:00:59.342687+0000","last_peered":"2024-09-12T03:00:59.342687+0000","last_clean":"2024-09-12T03:00:59.342687+0000","last_became_active":"2024-09-12T03:00:58.346535+0000","last_became_peered":"2024-09-12T03:00:58.346535+0000","last_unstale":"2024-09-12T03:00:59.342687+0000","last_undegraded":"2024-09-12T03:00:59.342687+0000","last_fullsized":"2024-09-12T03:00:59.342687+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:46.637914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:06.389702+0000","last_change":"2024-09-12T03:00:56.435957+0000","last_active":"2024-09-12T03:01:06.389702+0000","last_peered":"2024-09-12T03:01:06.389702+0000","last_clean":"2024-09-12T03:01:06.389702+0000","last_became_active":"2024-09-12T03:00:56.435603+0000","last_became_peered":"2024-09-12T03:00:56.435603+0000","last_unstale":"2024-09-12T03:01:06.389702+0000","last_undegraded":"2024-09-12T03:01:06.389702+0000","last_fullsized":"2024-09-12T03:01:06.389702+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:34.553189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.291897+0000","last_change":"2024-09-12T03:00:58.337784+0000","last_active":"2024-09-12T03:02:04.291897+0000","last_peered":"2024-09-12T03:02:04.291897+0000","last_clean":"2024-09-12T03:02:04.291897+0000","last_became_active":"2024-09-12T03:00:58.337415+0000","last_became_peered":"2024-09-12T03:00:58.337415+0000","last_unstale":"2024-09-12T03:02:04.291897+0000","last_undegraded":"2024-09-12T03:02:04.291897+0000","last_fullsized":"2024-09-12T03:02:04.291897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:38:15.557067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:01:06.390162+0000","last_change":"2024-09-12T03:00:56.315356+0000","last_active":"2024-09-12T03:01:06.390162+0000","last_peered":"2024-09-12T03:01:06.390162+0000","last_clean":"2024-09-12T03:01:06.390162+0000","last_became_active":"2024-09-12T03:00:56.314873+0000","last_became_peered":"2024-09-12T03:00:56.314873+0000","last_unstale":"2024-09-12T03:01:06.390162+0000","last_undegraded":"2024-09-12T03:01:06.390162+0000","last_fullsized":"2024-09-12T03:01:06.390162+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:55.710156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.348074+0000","last_change":"2024-09-12T03:00:58.346706+0000","last_active":"2024-09-12T03:00:59.348074+0000","last_peered":"2024-09-12T03:00:59.348074+0000","last_clean":"2024-09-12T03:00:59.348074+0000","last_became_active":"2024-09-12T03:00:58.346363+0000","last_became_peered":"2024-09-12T03:00:58.346363+0000","last_unstale":"2024-09-12T03:00:59.348074+0000","last_undegraded":"2024-09-12T03:00:59.348074+0000","last_fullsized":"2024-09-12T03:00:59.348074+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:13.468639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343312+0000","last_change":"2024-09-12T03:00:56.314659+0000","last_active":"2024-09-12T03:00:59.343312+0000","last_peered":"2024-09-12T03:00:59.343312+0000","last_clean":"2024-09-12T03:00:59.343312+0000","last_became_active":"2024-09-12T03:00:56.314316+0000","last_became_peered":"2024-09-12T03:00:56.314316+0000","last_unstale":"2024-09-12T03:00:59.343312+0000","last_undegraded":"2024-09-12T03:00:59.343312+0000","last_fullsized":"2024-09-12T03:00:59.343312+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:26:23.883066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.345830+0000","last_change":"2024-09-12T03:00:58.356682+0000","last_active":"2024-09-12T03:00:59.345830+0000","last_peered":"2024-09-12T03:00:59.345830+0000","last_clean":"2024-09-12T03:00:59.345830+0000","last_became_active":"2024-09-12T03:00:58.356383+0000","last_became_peered":"2024-09-12T03:00:58.356383+0000","last_unstale":"2024-09-12T03:00:59.345830+0000","last_undegraded":"2024-09-12T03:00:59.345830+0000","last_fullsized":"2024-09-12T03:00:59.345830+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:14:16.428393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292053+0000","last_change":"2024-09-12T03:00:56.314943+0000","last_active":"2024-09-12T03:02:04.292053+0000","last_peered":"2024-09-12T03:02:04.292053+0000","last_clean":"2024-09-12T03:02:04.292053+0000","last_became_active":"2024-09-12T03:00:56.314602+0000","last_became_peered":"2024-09-12T03:00:56.314602+0000","last_unstale":"2024-09-12T03:02:04.292053+0000","last_undegraded":"2024-09-12T03:02:04.292053+0000","last_fullsized":"2024-09-12T03:02:04.292053+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:31.207880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.283651+0000","last_change":"2024-09-12T03:00:58.351774+0000","last_active":"2024-09-12T03:02:03.283651+0000","last_peered":"2024-09-12T03:02:03.283651+0000","last_clean":"2024-09-12T03:02:03.283651+0000","last_became_active":"2024-09-12T03:00:58.351453+0000","last_became_peered":"2024-09-12T03:00:58.351453+0000","last_unstale":"2024-09-12T03:02:03.283651+0000","last_undegraded":"2024-09-12T03:02:03.283651+0000","last_fullsized":"2024-09-12T03:02:03.283651+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:18:24.260110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T03:02:03.281997+0000","last_change":"2024-09-12T03:00:58.346886+0000","last_active":"2024-09-12T03:02:03.281997+0000","last_peered":"2024-09-12T03:02:03.281997+0000","last_clean":"2024-09-12T03:02:03.281997+0000","last_became_active":"2024-09-12T03:00:58.346499+0000","last_became_peered":"2024-09-12T03:00:58.346499+0000","last_unstale":"2024-09-12T03:02:03.281997+0000","last_undegraded":"2024-09-12T03:02:03.281997+0000","last_fullsized":"2024-09-12T03:02:03.281997+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:02:24.045610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292497+0000","last_change":"2024-09-12T03:00:56.317443+0000","last_active":"2024-09-12T03:02:04.292497+0000","last_peered":"2024-09-12T03:02:04.292497+0000","last_clean":"2024-09-12T03:02:04.292497+0000","last_became_active":"2024-09-12T03:00:56.317045+0000","last_became_peered":"2024-09-12T03:00:56.317045+0000","last_unstale":"2024-09-12T03:02:04.292497+0000","last_undegraded":"2024-09-12T03:02:04.292497+0000","last_fullsized":"2024-09-12T03:02:04.292497+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:07.687606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.328499+0000","last_change":"2024-09-12T03:00:58.337257+0000","last_active":"2024-09-12T03:00:59.328499+0000","last_peered":"2024-09-12T03:00:59.328499+0000","last_clean":"2024-09-12T03:00:59.328499+0000","last_became_active":"2024-09-12T03:00:58.336847+0000","last_became_peered":"2024-09-12T03:00:58.336847+0000","last_unstale":"2024-09-12T03:00:59.328499+0000","last_undegraded":"2024-09-12T03:00:59.328499+0000","last_fullsized":"2024-09-12T03:00:59.328499+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:22.846206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292165+0000","last_change":"2024-09-12T03:00:56.316255+0000","last_active":"2024-09-12T03:02:04.292165+0000","last_peered":"2024-09-12T03:02:04.292165+0000","last_clean":"2024-09-12T03:02:04.292165+0000","last_became_active":"2024-09-12T03:00:56.315906+0000","last_became_peered":"2024-09-12T03:00:56.315906+0000","last_unstale":"2024-09-12T03:02:04.292165+0000","last_undegraded":"2024-09-12T03:02:04.292165+0000","last_fullsized":"2024-09-12T03:02:04.292165+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:14.798601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.342990+0000","last_change":"2024-09-12T03:00:58.347206+0000","last_active":"2024-09-12T03:00:59.342990+0000","last_peered":"2024-09-12T03:00:59.342990+0000","last_clean":"2024-09-12T03:00:59.342990+0000","last_became_active":"2024-09-12T03:00:58.346923+0000","last_became_peered":"2024-09-12T03:00:58.346923+0000","last_unstale":"2024-09-12T03:00:59.342990+0000","last_undegraded":"2024-09-12T03:00:59.342990+0000","last_fullsized":"2024-09-12T03:00:59.342990+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:41.376211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.347646+0000","last_change":"2024-09-12T03:00:56.435431+0000","last_active":"2024-09-12T03:00:59.347646+0000","last_peered":"2024-09-12T03:00:59.347646+0000","last_clean":"2024-09-12T03:00:59.347646+0000","last_became_active":"2024-09-12T03:00:56.435088+0000","last_became_peered":"2024-09-12T03:00:56.435088+0000","last_unstale":"2024-09-12T03:00:59.347646+0000","last_undegraded":"2024-09-12T03:00:59.347646+0000","last_fullsized":"2024-09-12T03:00:59.347646+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:24.738340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T03:00:59.343937+0000","last_change":"2024-09-12T03:00:58.351737+0000","last_active":"2024-09-12T03:00:59.343937+0000","last_peered":"2024-09-12T03:00:59.343937+0000","last_clean":"2024-09-12T03:00:59.343937+0000","last_became_active":"2024-09-12T03:00:58.351407+0000","last_became_peered":"2024-09-12T03:00:58.351407+0000","last_unstale":"2024-09-12T03:00:59.343937+0000","last_undegraded":"2024-09-12T03:00:59.343937+0000","last_fullsized":"2024-09-12T03:00:59.343937+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:34.206703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284979+0000","last_change":"2024-09-12T03:00:56.310910+0000","last_active":"2024-09-12T03:02:04.284979+0000","last_peered":"2024-09-12T03:02:04.284979+0000","last_clean":"2024-09-12T03:02:04.284979+0000","last_became_active":"2024-09-12T03:00:56.310657+0000","last_became_peered":"2024-09-12T03:00:56.310657+0000","last_unstale":"2024-09-12T03:02:04.284979+0000","last_undegraded":"2024-09-12T03:02:04.284979+0000","last_fullsized":"2024-09-12T03:02:04.284979+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:06.117215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65362,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":372,"num_read_kb":256,"num_write":161,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":196064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245899,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645339,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":414814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906976,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1028096,"data_stored":1085046,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070502,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":897024,"data_stored":432166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299435,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":431947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528367,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":1006509,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":40,"seq":171798691893,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":880640,"data_stored":431438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":33,"seq":141733920825,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":1006193,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051647,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":872448,"data_stored":423461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280579,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":414750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476808,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":942080,"data_stored":477334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607630,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":475984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":876544,"data_stored":415710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:02:09.194 INFO:tasks.cephadm.ceph_manager.ceph:PG 3.11 is not active+clean 2024-09-12T03:02:09.195 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 14, 'reported_epoch': 78, 'state': 'peering', 'last_fresh': '2024-09-12T03:00:59.328638+0000', 'last_change': '2024-09-12T03:02:03.280677+0000', 'last_active': '2024-09-12T03:02:03.280677+0000', 'last_peered': '2024-09-12T03:00:59.328638+0000', 'last_clean': '2024-09-12T03:00:59.328638+0000', 'last_became_active': '2024-09-12T03:00:58.346490+0000', 'last_became_peered': '2024-09-12T03:00:58.346490+0000', 'last_unstale': '2024-09-12T03:00:59.328638+0000', 'last_undegraded': '2024-09-12T03:00:59.328638+0000', 'last_fullsized': '2024-09-12T03:00:59.328638+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 77, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:00:57.310000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:00:57.310000+0000', 'last_clean_scrub_stamp': '2024-09-12T03:00:57.310000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 8, 1], 'acting': [5, 8, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1, 4, 8], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T03:02:09.195 INFO:tasks.cephadm.ceph_manager.ceph:PG 3.c is not active+clean 2024-09-12T03:02:09.195 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '3.c', 'version': "78'1", 'reported_seq': 15, 'reported_epoch': 78, 'state': 'peering', 'last_fresh': '2024-09-12T03:01:17.267451+0000', 'last_change': '2024-09-12T03:02:03.283802+0000', 'last_active': '2024-09-12T03:02:03.283802+0000', 'last_peered': '2024-09-12T03:01:17.267451+0000', 'last_clean': '2024-09-12T03:01:17.267451+0000', 'last_became_active': '2024-09-12T03:00:58.357451+0000', 'last_became_peered': '2024-09-12T03:00:58.357451+0000', 'last_unstale': '2024-09-12T03:01:17.267451+0000', 'last_undegraded': '2024-09-12T03:01:17.267451+0000', 'last_fullsized': '2024-09-12T03:01:17.267451+0000', 'mapping_epoch': 79, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 76, 'last_epoch_clean': 77, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:00:57.310000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:00:57.310000+0000', 'last_clean_scrub_stamp': '2024-09-12T03:00:57.310000+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 117, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 3, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1, 'num_write_kb': 1, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 9, 5], 'acting': [7, 9, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3, 5, 9], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T03:02:09.195 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph status --format=json 2024-09-12T03:02:09.441 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:09.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.934+0000 7facd00a5640 1 -- 172.21.15.116:0/3756381900 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facc8102d30 msgr2=0x7facc8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:09.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.934+0000 7facd00a5640 1 --2- 172.21.15.116:0/3756381900 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facc8102d30 0x7facc8103110 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7facc0009930 tx=0x7facc002a420 comp rx=0 tx=0).stop 2024-09-12T03:02:09.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.934+0000 7facd00a5640 1 -- 172.21.15.116:0/3756381900 shutdown_connections 2024-09-12T03:02:09.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.934+0000 7facd00a5640 1 --2- 172.21.15.116:0/3756381900 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7facc81082a0 0x7facc810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.934+0000 7facd00a5640 1 --2- 172.21.15.116:0/3756381900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facc81036e0 0x7facc8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.934+0000 7facd00a5640 1 --2- 172.21.15.116:0/3756381900 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facc8102d30 0x7facc8103110 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.934+0000 7facd00a5640 1 -- 172.21.15.116:0/3756381900 >> 172.21.15.116:0/3756381900 conn(0x7facc80fc7d0 msgr2=0x7facc80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:09.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.934+0000 7facd00a5640 1 -- 172.21.15.116:0/3756381900 shutdown_connections 2024-09-12T03:02:09.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.934+0000 7facd00a5640 1 -- 172.21.15.116:0/3756381900 wait complete. 2024-09-12T03:02:09.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.935+0000 7facd00a5640 1 Processor -- start 2024-09-12T03:02:09.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.935+0000 7facd00a5640 1 -- start start 2024-09-12T03:02:09.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.935+0000 7facd00a5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facc8102d30 0x7facc81985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.935+0000 7facd00a5640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7facc81036e0 0x7facc8198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.935+0000 7facd00a5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facc81082a0 0x7facc819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.935+0000 7facd00a5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7facc81120a0 con 0x7facc81082a0 2024-09-12T03:02:09.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.935+0000 7facd00a5640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7facc8111f20 con 0x7facc81036e0 2024-09-12T03:02:09.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.935+0000 7facd00a5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7facc8112220 con 0x7facc8102d30 2024-09-12T03:02:09.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.935+0000 7facce642640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facc81082a0 0x7facc819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.935+0000 7facce642640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facc81082a0 0x7facc819ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60188/0 (socket says 172.21.15.116:60188) 2024-09-12T03:02:09.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.935+0000 7facce642640 1 -- 172.21.15.116:0/3800675012 learned_addr learned my addr 172.21.15.116:0/3800675012 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:09.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.936+0000 7faccde41640 1 --2- 172.21.15.116:0/3800675012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facc8102d30 0x7facc81985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.936+0000 7facce642640 1 -- 172.21.15.116:0/3800675012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facc8102d30 msgr2=0x7facc81985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:09.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.936+0000 7facce642640 1 --2- 172.21.15.116:0/3800675012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facc8102d30 0x7facc81985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.936+0000 7facce642640 1 -- 172.21.15.116:0/3800675012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7facc81036e0 msgr2=0x7facc8198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:09.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.936+0000 7facce642640 1 --2- 172.21.15.116:0/3800675012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7facc81036e0 0x7facc8198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.936+0000 7facce642640 1 -- 172.21.15.116:0/3800675012 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc819d590 con 0x7facc81082a0 2024-09-12T03:02:09.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.936+0000 7facce642640 1 --2- 172.21.15.116:0/3800675012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facc81082a0 0x7facc819ceb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7facb800cad0 tx=0x7facb800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:09.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.936+0000 7facc6ffd640 1 -- 172.21.15.116:0/3800675012 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb800ae20 con 0x7facc81082a0 2024-09-12T03:02:09.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.936+0000 7facc6ffd640 1 -- 172.21.15.116:0/3800675012 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facb8005020 con 0x7facc81082a0 2024-09-12T03:02:09.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.936+0000 7facd00a5640 1 -- 172.21.15.116:0/3800675012 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc819d880 con 0x7facc81082a0 2024-09-12T03:02:09.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.936+0000 7facc6ffd640 1 -- 172.21.15.116:0/3800675012 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb800f6f0 con 0x7facc81082a0 2024-09-12T03:02:09.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.936+0000 7facd00a5640 1 -- 172.21.15.116:0/3800675012 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc81a5160 con 0x7facc81082a0 2024-09-12T03:02:09.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.937+0000 7facd00a5640 1 -- 172.21.15.116:0/3800675012 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facc806a850 con 0x7facc81082a0 2024-09-12T03:02:09.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.938+0000 7facc6ffd640 1 -- 172.21.15.116:0/3800675012 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7facb80051d0 con 0x7facc81082a0 2024-09-12T03:02:09.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.939+0000 7facc6ffd640 1 --2- 172.21.15.116:0/3800675012 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7facb00a2c90 0x7facb00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.939+0000 7faccde41640 1 --2- 172.21.15.116:0/3800675012 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7facb00a2c90 0x7facb00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.939+0000 7facc6ffd640 1 -- 172.21.15.116:0/3800675012 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7facb80cf540 con 0x7facc81082a0 2024-09-12T03:02:09.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.939+0000 7faccde41640 1 --2- 172.21.15.116:0/3800675012 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7facb00a2c90 0x7facb00a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7facc0000c00 tx=0x7facc0031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:09.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:09.942+0000 7facc6ffd640 1 -- 172.21.15.116:0/3800675012 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7facb8014030 con 0x7facc81082a0 2024-09-12T03:02:10.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:10.171+0000 7facd00a5640 1 -- 172.21.15.116:0/3800675012 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7facc8103110 con 0x7facc81082a0 2024-09-12T03:02:10.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:10.172+0000 7facc6ffd640 1 -- 172.21.15.116:0/3800675012 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2564 (secure 0 0 0) 0x7facc8103110 con 0x7facc81082a0 2024-09-12T03:02:10.172 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:02:10.172 INFO:teuthology.orchestra.run.smithi116.stdout:{"fsid":"eb1ac602-70b1-11ef-bcea-c7b262605968","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 1 pg peering","count":1},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":437,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1726110007,"num_in_osds":12,"osd_in_since":1726109983,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":95},{"state_name":"peering","count":2}],"num_pgs":97,"num_pools":3,"num_objects":64,"data_bytes":655964,"bytes_used":377204736,"bytes_avail":1151563223040,"bytes_total":1151940427776,"inactive_pgs_ratio":0.020618556067347527,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":2,"recovering_keys_per_sec":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"read_bytes_sec":3711,"read_op_per_sec":6,"write_op_per_sec":0},"fsmap":{"epoch":24,"btime":"2024-09-12T03:01:10:397269+0000","id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"e","status":"up:standby-replay","gid":24443},{"filesystem_id":1,"rank":1,"name":"f","status":"up:standby-replay","gid":24448},{"filesystem_id":1,"rank":0,"name":"c","status":"up:standby-replay","gid":24454},{"filesystem_id":1,"rank":2,"name":"h","status":"up:active","gid":24467},{"filesystem_id":1,"rank":1,"name":"i","status":"up:active","gid":24475},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24494}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.116:8443/"}},"servicemap":{"epoch":25,"modified":"2024-09-12T03:02:05.123122+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-12T03:02:10.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:10.174+0000 7facd00a5640 1 -- 172.21.15.116:0/3800675012 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7facb00a2c90 msgr2=0x7facb00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:10.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:10.174+0000 7facd00a5640 1 --2- 172.21.15.116:0/3800675012 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7facb00a2c90 0x7facb00a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7facc0000c00 tx=0x7facc0031490 comp rx=0 tx=0).stop 2024-09-12T03:02:10.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:10.174+0000 7facd00a5640 1 -- 172.21.15.116:0/3800675012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facc81082a0 msgr2=0x7facc819ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:10.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:10.174+0000 7facd00a5640 1 --2- 172.21.15.116:0/3800675012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facc81082a0 0x7facc819ceb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7facb800cad0 tx=0x7facb800cf90 comp rx=0 tx=0).stop 2024-09-12T03:02:10.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:10.175+0000 7facd00a5640 1 -- 172.21.15.116:0/3800675012 shutdown_connections 2024-09-12T03:02:10.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:10.175+0000 7facd00a5640 1 --2- 172.21.15.116:0/3800675012 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7facb00a2c90 0x7facb00a5150 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:10.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:10.175+0000 7facd00a5640 1 --2- 172.21.15.116:0/3800675012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facc81082a0 0x7facc819ceb0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:10.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:10.175+0000 7facd00a5640 1 --2- 172.21.15.116:0/3800675012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7facc81036e0 0x7facc8198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:10.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:10.175+0000 7facd00a5640 1 --2- 172.21.15.116:0/3800675012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facc8102d30 0x7facc81985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:10.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:10.175+0000 7facd00a5640 1 -- 172.21.15.116:0/3800675012 >> 172.21.15.116:0/3800675012 conn(0x7facc80fc7d0 msgr2=0x7facc8105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:10.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:10.175+0000 7facd00a5640 1 -- 172.21.15.116:0/3800675012 shutdown_connections 2024-09-12T03:02:10.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:10.175+0000 7facd00a5640 1 -- 172.21.15.116:0/3800675012 wait complete. 2024-09-12T03:02:10.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:10 smithi183 ceph-mon[35535]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:02:10.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:10 smithi183 ceph-mon[35535]: pgmap v313: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 2 B/s, 0 objects/s recovering 2024-09-12T03:02:10.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:10 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3800675012' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-12T03:02:10.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:10 smithi177 ceph-mon[35305]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:02:10.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:10 smithi177 ceph-mon[35305]: pgmap v313: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 2 B/s, 0 objects/s recovering 2024-09-12T03:02:10.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:10 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3800675012' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-12T03:02:10.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:10 smithi116 ceph-mon[31410]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:02:10.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:10 smithi116 ceph-mon[31410]: pgmap v313: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 2 B/s, 0 objects/s recovering 2024-09-12T03:02:10.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:10 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3800675012' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-12T03:02:10.941 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-09-12T03:02:10.941 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:02:11.190 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:11.492 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:11 smithi116 ceph-mon[31410]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-12T03:02:11.492 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:11 smithi116 ceph-mon[31410]: Cluster is now healthy 2024-09-12T03:02:11.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:11 smithi177 ceph-mon[35305]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-12T03:02:11.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:11 smithi177 ceph-mon[35305]: Cluster is now healthy 2024-09-12T03:02:11.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:11 smithi183 ceph-mon[35535]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-12T03:02:11.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:11 smithi183 ceph-mon[35535]: Cluster is now healthy 2024-09-12T03:02:11.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.694+0000 7fe083632640 1 -- 172.21.15.116:0/3781576564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe07c108160 msgr2=0x7fe07c10eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:11.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.694+0000 7fe083632640 1 --2- 172.21.15.116:0/3781576564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe07c108160 0x7fe07c10eb00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe074009cd0 tx=0x7fe07402a910 comp rx=0 tx=0).stop 2024-09-12T03:02:11.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.694+0000 7fe083632640 1 -- 172.21.15.116:0/3781576564 shutdown_connections 2024-09-12T03:02:11.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.694+0000 7fe083632640 1 --2- 172.21.15.116:0/3781576564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe07c108160 0x7fe07c10eb00 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:11.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.694+0000 7fe083632640 1 --2- 172.21.15.116:0/3781576564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe07c101290 0x7fe07c107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:11.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.694+0000 7fe083632640 1 --2- 172.21.15.116:0/3781576564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe07c1008e0 0x7fe07c100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:11.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.694+0000 7fe083632640 1 -- 172.21.15.116:0/3781576564 >> 172.21.15.116:0/3781576564 conn(0x7fe07c077210 msgr2=0x7fe07c077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:11.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.694+0000 7fe083632640 1 -- 172.21.15.116:0/3781576564 shutdown_connections 2024-09-12T03:02:11.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.694+0000 7fe083632640 1 -- 172.21.15.116:0/3781576564 wait complete. 2024-09-12T03:02:11.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.694+0000 7fe083632640 1 Processor -- start 2024-09-12T03:02:11.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.695+0000 7fe083632640 1 -- start start 2024-09-12T03:02:11.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.695+0000 7fe083632640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe07c1008e0 0x7fe07c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:11.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.695+0000 7fe083632640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe07c101290 0x7fe07c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:11.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.695+0000 7fe083632640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe07c108160 0x7fe07c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:11.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.695+0000 7fe083632640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe07c112010 con 0x7fe07c101290 2024-09-12T03:02:11.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.695+0000 7fe083632640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe07c111e90 con 0x7fe07c1008e0 2024-09-12T03:02:11.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.695+0000 7fe083632640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe07c112190 con 0x7fe07c108160 2024-09-12T03:02:11.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.695+0000 7fe080bcd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe07c101290 0x7fe07c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:11.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.695+0000 7fe080bcd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe07c101290 0x7fe07c1989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43320/0 (socket says 172.21.15.116:43320) 2024-09-12T03:02:11.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.695+0000 7fe080bcd640 1 -- 172.21.15.116:0/51508257 learned_addr learned my addr 172.21.15.116:0/51508257 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:11.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.695+0000 7fe0813ce640 1 --2- 172.21.15.116:0/51508257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe07c1008e0 0x7fe07c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:11.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.695+0000 7fe081bcf640 1 --2- 172.21.15.116:0/51508257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe07c108160 0x7fe07c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:11.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.695+0000 7fe080bcd640 1 -- 172.21.15.116:0/51508257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe07c108160 msgr2=0x7fe07c19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:11.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.695+0000 7fe080bcd640 1 --2- 172.21.15.116:0/51508257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe07c108160 0x7fe07c19cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:11.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.695+0000 7fe080bcd640 1 -- 172.21.15.116:0/51508257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe07c1008e0 msgr2=0x7fe07c198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:11.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.695+0000 7fe080bcd640 1 --2- 172.21.15.116:0/51508257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe07c1008e0 0x7fe07c198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:11.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.695+0000 7fe080bcd640 1 -- 172.21.15.116:0/51508257 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe07c19d430 con 0x7fe07c101290 2024-09-12T03:02:11.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.696+0000 7fe080bcd640 1 --2- 172.21.15.116:0/51508257 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe07c101290 0x7fe07c1989c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00b430 tx=0x7fe06c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:11.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.696+0000 7fe0727fc640 1 -- 172.21.15.116:0/51508257 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06c015020 con 0x7fe07c101290 2024-09-12T03:02:11.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.696+0000 7fe0727fc640 1 -- 172.21.15.116:0/51508257 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe06c011d10 con 0x7fe07c101290 2024-09-12T03:02:11.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.696+0000 7fe083632640 1 -- 172.21.15.116:0/51508257 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe07c19d720 con 0x7fe07c101290 2024-09-12T03:02:11.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.696+0000 7fe0727fc640 1 -- 172.21.15.116:0/51508257 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06c00f2e0 con 0x7fe07c101290 2024-09-12T03:02:11.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.696+0000 7fe083632640 1 -- 172.21.15.116:0/51508257 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe07c1a5000 con 0x7fe07c101290 2024-09-12T03:02:11.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.697+0000 7fe083632640 1 -- 172.21.15.116:0/51508257 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe07c100cc0 con 0x7fe07c101290 2024-09-12T03:02:11.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.698+0000 7fe0727fc640 1 -- 172.21.15.116:0/51508257 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe06c018020 con 0x7fe07c101290 2024-09-12T03:02:11.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.698+0000 7fe0727fc640 1 --2- 172.21.15.116:0/51508257 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe0680a2c10 0x7fe0680a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:11.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.698+0000 7fe0813ce640 1 --2- 172.21.15.116:0/51508257 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe0680a2c10 0x7fe0680a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:11.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.699+0000 7fe0727fc640 1 -- 172.21.15.116:0/51508257 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe06c0ce900 con 0x7fe07c101290 2024-09-12T03:02:11.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.699+0000 7fe0813ce640 1 --2- 172.21.15.116:0/51508257 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe0680a2c10 0x7fe0680a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe07800bbe0 tx=0x7fe07800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:11.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.702+0000 7fe0727fc640 1 -- 172.21.15.116:0/51508257 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe06c01f020 con 0x7fe07c101290 2024-09-12T03:02:11.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.864+0000 7fe083632640 1 -- 172.21.15.116:0/51508257 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe07c062630 con 0x7fe0680a2c10 2024-09-12T03:02:11.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.873+0000 7fe0727fc640 1 -- 172.21.15.116:0/51508257 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258862 (secure 0 0 0) 0x7fe07c062630 con 0x7fe0680a2c10 2024-09-12T03:02:11.876 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:02:11.877 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-09-12T03:02:11.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.876+0000 7fe083632640 1 -- 172.21.15.116:0/51508257 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe0680a2c10 msgr2=0x7fe0680a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:11.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.876+0000 7fe083632640 1 --2- 172.21.15.116:0/51508257 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe0680a2c10 0x7fe0680a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe07800bbe0 tx=0x7fe07800b390 comp rx=0 tx=0).stop 2024-09-12T03:02:11.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.876+0000 7fe083632640 1 -- 172.21.15.116:0/51508257 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe07c101290 msgr2=0x7fe07c1989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:11.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.876+0000 7fe083632640 1 --2- 172.21.15.116:0/51508257 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe07c101290 0x7fe07c1989c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00b430 tx=0x7fe06c00b8f0 comp rx=0 tx=0).stop 2024-09-12T03:02:11.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.876+0000 7fe083632640 1 -- 172.21.15.116:0/51508257 shutdown_connections 2024-09-12T03:02:11.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.876+0000 7fe083632640 1 --2- 172.21.15.116:0/51508257 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe0680a2c10 0x7fe0680a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:11.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.876+0000 7fe083632640 1 --2- 172.21.15.116:0/51508257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe07c108160 0x7fe07c19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:11.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.876+0000 7fe083632640 1 --2- 172.21.15.116:0/51508257 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe07c101290 0x7fe07c1989c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:11.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.876+0000 7fe083632640 1 --2- 172.21.15.116:0/51508257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe07c1008e0 0x7fe07c198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:11.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.876+0000 7fe083632640 1 -- 172.21.15.116:0/51508257 >> 172.21.15.116:0/51508257 conn(0x7fe07c077210 msgr2=0x7fe07c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:11.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.876+0000 7fe083632640 1 -- 172.21.15.116:0/51508257 shutdown_connections 2024-09-12T03:02:11.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:11.876+0000 7fe083632640 1 -- 172.21.15.116:0/51508257 wait complete. 2024-09-12T03:02:12.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:12 smithi177 ceph-mon[35305]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 17 B/s, 0 objects/s recovering 2024-09-12T03:02:12.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:12 smithi116 ceph-mon[31410]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 17 B/s, 0 objects/s recovering 2024-09-12T03:02:12.648 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-12T03:02:11.124371+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655964,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":507,"num_read_kb":364,"num_write":344,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180875,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":288,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368444,"kb_used_data":10800,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573380,"statfs":{"total":1151940427776,"available":1151563141120,"internally_reserved":0,"allocated":11059200,"data_stored":7142664,"data_compressed":107490,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":14,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":139,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001765"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.287303+0000","last_change":"2024-09-12T03:00:58.348563+0000","last_active":"2024-09-12T03:02:04.287303+0000","last_peered":"2024-09-12T03:02:04.287303+0000","last_clean":"2024-09-12T03:02:04.287303+0000","last_became_active":"2024-09-12T03:00:58.347940+0000","last_became_peered":"2024-09-12T03:00:58.347940+0000","last_unstale":"2024-09-12T03:02:04.287303+0000","last_undegraded":"2024-09-12T03:02:04.287303+0000","last_fullsized":"2024-09-12T03:02:04.287303+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:43.489238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332599+0000","last_change":"2024-09-12T03:00:58.351969+0000","last_active":"2024-09-12T03:02:04.332599+0000","last_peered":"2024-09-12T03:02:04.332599+0000","last_clean":"2024-09-12T03:02:04.332599+0000","last_became_active":"2024-09-12T03:00:58.351679+0000","last_became_peered":"2024-09-12T03:00:58.351679+0000","last_unstale":"2024-09-12T03:02:04.332599+0000","last_undegraded":"2024-09-12T03:02:04.332599+0000","last_fullsized":"2024-09-12T03:02:04.332599+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:27:01.174890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332354+0000","last_change":"2024-09-12T03:00:58.346045+0000","last_active":"2024-09-12T03:02:04.332354+0000","last_peered":"2024-09-12T03:02:04.332354+0000","last_clean":"2024-09-12T03:02:04.332354+0000","last_became_active":"2024-09-12T03:00:58.345719+0000","last_became_peered":"2024-09-12T03:00:58.345719+0000","last_unstale":"2024-09-12T03:02:04.332354+0000","last_undegraded":"2024-09-12T03:02:04.332354+0000","last_fullsized":"2024-09-12T03:02:04.332354+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:12.312650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289352+0000","last_change":"2024-09-12T03:00:58.348165+0000","last_active":"2024-09-12T03:02:04.289352+0000","last_peered":"2024-09-12T03:02:04.289352+0000","last_clean":"2024-09-12T03:02:04.289352+0000","last_became_active":"2024-09-12T03:00:58.347734+0000","last_became_peered":"2024-09-12T03:00:58.347734+0000","last_unstale":"2024-09-12T03:02:04.289352+0000","last_undegraded":"2024-09-12T03:02:04.289352+0000","last_fullsized":"2024-09-12T03:02:04.289352+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:05.945900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.291897+0000","last_change":"2024-09-12T03:00:58.339316+0000","last_active":"2024-09-12T03:02:04.291897+0000","last_peered":"2024-09-12T03:02:04.291897+0000","last_clean":"2024-09-12T03:02:04.291897+0000","last_became_active":"2024-09-12T03:00:58.338964+0000","last_became_peered":"2024-09-12T03:00:58.338964+0000","last_unstale":"2024-09-12T03:02:04.291897+0000","last_undegraded":"2024-09-12T03:02:04.291897+0000","last_fullsized":"2024-09-12T03:02:04.291897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:50.721276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282483+0000","last_change":"2024-09-12T03:00:58.346838+0000","last_active":"2024-09-12T03:02:04.282483+0000","last_peered":"2024-09-12T03:02:04.282483+0000","last_clean":"2024-09-12T03:02:04.282483+0000","last_became_active":"2024-09-12T03:00:58.346486+0000","last_became_peered":"2024-09-12T03:00:58.346486+0000","last_unstale":"2024-09-12T03:02:04.282483+0000","last_undegraded":"2024-09-12T03:02:04.282483+0000","last_fullsized":"2024-09-12T03:02:04.282483+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:28.687727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284091+0000","last_change":"2024-09-12T03:00:58.346712+0000","last_active":"2024-09-12T03:02:04.284091+0000","last_peered":"2024-09-12T03:02:04.284091+0000","last_clean":"2024-09-12T03:02:04.284091+0000","last_became_active":"2024-09-12T03:00:58.346397+0000","last_became_peered":"2024-09-12T03:00:58.346397+0000","last_unstale":"2024-09-12T03:02:04.284091+0000","last_undegraded":"2024-09-12T03:02:04.284091+0000","last_fullsized":"2024-09-12T03:02:04.284091+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:48:00.624733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282190+0000","last_change":"2024-09-12T03:00:58.356166+0000","last_active":"2024-09-12T03:02:04.282190+0000","last_peered":"2024-09-12T03:02:04.282190+0000","last_clean":"2024-09-12T03:02:04.282190+0000","last_became_active":"2024-09-12T03:00:58.354944+0000","last_became_peered":"2024-09-12T03:00:58.354944+0000","last_unstale":"2024-09-12T03:02:04.282190+0000","last_undegraded":"2024-09-12T03:02:04.282190+0000","last_fullsized":"2024-09-12T03:02:04.282190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:15.840149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332814+0000","last_change":"2024-09-12T03:00:58.340182+0000","last_active":"2024-09-12T03:02:04.332814+0000","last_peered":"2024-09-12T03:02:04.332814+0000","last_clean":"2024-09-12T03:02:04.332814+0000","last_became_active":"2024-09-12T03:00:58.339576+0000","last_became_peered":"2024-09-12T03:00:58.339576+0000","last_unstale":"2024-09-12T03:02:04.332814+0000","last_undegraded":"2024-09-12T03:02:04.332814+0000","last_fullsized":"2024-09-12T03:02:04.332814+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:16.534844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282812+0000","last_change":"2024-09-12T03:00:58.351218+0000","last_active":"2024-09-12T03:02:04.282812+0000","last_peered":"2024-09-12T03:02:04.282812+0000","last_clean":"2024-09-12T03:02:04.282812+0000","last_became_active":"2024-09-12T03:00:58.350871+0000","last_became_peered":"2024-09-12T03:00:58.350871+0000","last_unstale":"2024-09-12T03:02:04.282812+0000","last_undegraded":"2024-09-12T03:02:04.282812+0000","last_fullsized":"2024-09-12T03:02:04.282812+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:38.367683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284465+0000","last_change":"2024-09-12T03:00:58.351894+0000","last_active":"2024-09-12T03:02:04.284465+0000","last_peered":"2024-09-12T03:02:04.284465+0000","last_clean":"2024-09-12T03:02:04.284465+0000","last_became_active":"2024-09-12T03:00:58.351445+0000","last_became_peered":"2024-09-12T03:00:58.351445+0000","last_unstale":"2024-09-12T03:02:04.284465+0000","last_undegraded":"2024-09-12T03:02:04.284465+0000","last_fullsized":"2024-09-12T03:02:04.284465+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:42.093871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289589+0000","last_change":"2024-09-12T03:00:58.353460+0000","last_active":"2024-09-12T03:02:04.289589+0000","last_peered":"2024-09-12T03:02:04.289589+0000","last_clean":"2024-09-12T03:02:04.289589+0000","last_became_active":"2024-09-12T03:00:58.353111+0000","last_became_peered":"2024-09-12T03:00:58.353111+0000","last_unstale":"2024-09-12T03:02:04.289589+0000","last_undegraded":"2024-09-12T03:02:04.289589+0000","last_fullsized":"2024-09-12T03:02:04.289589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:15.916765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332731+0000","last_change":"2024-09-12T03:00:58.341080+0000","last_active":"2024-09-12T03:02:04.332731+0000","last_peered":"2024-09-12T03:02:04.332731+0000","last_clean":"2024-09-12T03:02:04.332731+0000","last_became_active":"2024-09-12T03:00:58.340782+0000","last_became_peered":"2024-09-12T03:00:58.340782+0000","last_unstale":"2024-09-12T03:02:04.332731+0000","last_undegraded":"2024-09-12T03:02:04.332731+0000","last_fullsized":"2024-09-12T03:02:04.332731+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:36.569700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332843+0000","last_change":"2024-09-12T03:00:58.346791+0000","last_active":"2024-09-12T03:02:04.332843+0000","last_peered":"2024-09-12T03:02:04.332843+0000","last_clean":"2024-09-12T03:02:04.332843+0000","last_became_active":"2024-09-12T03:00:58.346500+0000","last_became_peered":"2024-09-12T03:00:58.346500+0000","last_unstale":"2024-09-12T03:02:04.332843+0000","last_undegraded":"2024-09-12T03:02:04.332843+0000","last_fullsized":"2024-09-12T03:02:04.332843+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:32.017589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332195+0000","last_change":"2024-09-12T03:00:58.349522+0000","last_active":"2024-09-12T03:02:04.332195+0000","last_peered":"2024-09-12T03:02:04.332195+0000","last_clean":"2024-09-12T03:02:04.332195+0000","last_became_active":"2024-09-12T03:00:58.349246+0000","last_became_peered":"2024-09-12T03:00:58.349246+0000","last_unstale":"2024-09-12T03:02:04.332195+0000","last_undegraded":"2024-09-12T03:02:04.332195+0000","last_fullsized":"2024-09-12T03:02:04.332195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:39.253919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332262+0000","last_change":"2024-09-12T03:00:58.349909+0000","last_active":"2024-09-12T03:02:04.332262+0000","last_peered":"2024-09-12T03:02:04.332262+0000","last_clean":"2024-09-12T03:02:04.332262+0000","last_became_active":"2024-09-12T03:00:58.349590+0000","last_became_peered":"2024-09-12T03:00:58.349590+0000","last_unstale":"2024-09-12T03:02:04.332262+0000","last_undegraded":"2024-09-12T03:02:04.332262+0000","last_fullsized":"2024-09-12T03:02:04.332262+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:04.540709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.287399+0000","last_change":"2024-09-12T03:00:58.348284+0000","last_active":"2024-09-12T03:02:04.287399+0000","last_peered":"2024-09-12T03:02:04.287399+0000","last_clean":"2024-09-12T03:02:04.287399+0000","last_became_active":"2024-09-12T03:00:58.347465+0000","last_became_peered":"2024-09-12T03:00:58.347465+0000","last_unstale":"2024-09-12T03:02:04.287399+0000","last_undegraded":"2024-09-12T03:02:04.287399+0000","last_fullsized":"2024-09-12T03:02:04.287399+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:13.191899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284483+0000","last_change":"2024-09-12T03:00:58.337740+0000","last_active":"2024-09-12T03:02:04.284483+0000","last_peered":"2024-09-12T03:02:04.284483+0000","last_clean":"2024-09-12T03:02:04.284483+0000","last_became_active":"2024-09-12T03:00:58.337393+0000","last_became_peered":"2024-09-12T03:00:58.337393+0000","last_unstale":"2024-09-12T03:02:04.284483+0000","last_undegraded":"2024-09-12T03:02:04.284483+0000","last_fullsized":"2024-09-12T03:02:04.284483+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:28:56.339167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.290193+0000","last_change":"2024-09-12T03:00:58.345779+0000","last_active":"2024-09-12T03:02:04.290193+0000","last_peered":"2024-09-12T03:02:04.290193+0000","last_clean":"2024-09-12T03:02:04.290193+0000","last_became_active":"2024-09-12T03:00:58.345210+0000","last_became_peered":"2024-09-12T03:00:58.345210+0000","last_unstale":"2024-09-12T03:02:04.290193+0000","last_undegraded":"2024-09-12T03:02:04.290193+0000","last_fullsized":"2024-09-12T03:02:04.290193+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:06:58.271185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.294381+0000","last_change":"2024-09-12T03:02:04.294381+0000","last_active":"2024-09-12T03:02:04.294381+0000","last_peered":"2024-09-12T03:02:04.294381+0000","last_clean":"2024-09-12T03:02:04.294381+0000","last_became_active":"2024-09-12T03:02:04.293795+0000","last_became_peered":"2024-09-12T03:02:04.293795+0000","last_unstale":"2024-09-12T03:02:04.294381+0000","last_undegraded":"2024-09-12T03:02:04.294381+0000","last_fullsized":"2024-09-12T03:02:04.294381+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:06:58.271007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332823+0000","last_change":"2024-09-12T03:00:58.349233+0000","last_active":"2024-09-12T03:02:04.332823+0000","last_peered":"2024-09-12T03:02:04.332823+0000","last_clean":"2024-09-12T03:02:04.332823+0000","last_became_active":"2024-09-12T03:00:58.348958+0000","last_became_peered":"2024-09-12T03:00:58.348958+0000","last_unstale":"2024-09-12T03:02:04.332823+0000","last_undegraded":"2024-09-12T03:02:04.332823+0000","last_fullsized":"2024-09-12T03:02:04.332823+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:32:10.271817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332400+0000","last_change":"2024-09-12T03:00:58.358024+0000","last_active":"2024-09-12T03:02:04.332400+0000","last_peered":"2024-09-12T03:02:04.332400+0000","last_clean":"2024-09-12T03:02:04.332400+0000","last_became_active":"2024-09-12T03:00:58.357434+0000","last_became_peered":"2024-09-12T03:00:58.357434+0000","last_unstale":"2024-09-12T03:02:04.332400+0000","last_undegraded":"2024-09-12T03:02:04.332400+0000","last_fullsized":"2024-09-12T03:02:04.332400+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:01:20.119733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282240+0000","last_change":"2024-09-12T03:00:58.355063+0000","last_active":"2024-09-12T03:02:04.282240+0000","last_peered":"2024-09-12T03:02:04.282240+0000","last_clean":"2024-09-12T03:02:04.282240+0000","last_became_active":"2024-09-12T03:00:58.354760+0000","last_became_peered":"2024-09-12T03:00:58.354760+0000","last_unstale":"2024-09-12T03:02:04.282240+0000","last_undegraded":"2024-09-12T03:02:04.282240+0000","last_fullsized":"2024-09-12T03:02:04.282240+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:13.464413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.288000+0000","last_change":"2024-09-12T03:00:58.341916+0000","last_active":"2024-09-12T03:02:04.288000+0000","last_peered":"2024-09-12T03:02:04.288000+0000","last_clean":"2024-09-12T03:02:04.288000+0000","last_became_active":"2024-09-12T03:00:58.340815+0000","last_became_peered":"2024-09-12T03:00:58.340815+0000","last_unstale":"2024-09-12T03:02:04.288000+0000","last_undegraded":"2024-09-12T03:02:04.288000+0000","last_fullsized":"2024-09-12T03:02:04.288000+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:13.777947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.283555+0000","last_change":"2024-09-12T03:00:58.357765+0000","last_active":"2024-09-12T03:02:04.283555+0000","last_peered":"2024-09-12T03:02:04.283555+0000","last_clean":"2024-09-12T03:02:04.283555+0000","last_became_active":"2024-09-12T03:00:58.357480+0000","last_became_peered":"2024-09-12T03:00:58.357480+0000","last_unstale":"2024-09-12T03:02:04.283555+0000","last_undegraded":"2024-09-12T03:02:04.283555+0000","last_fullsized":"2024-09-12T03:02:04.283555+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:26.884381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.290553+0000","last_change":"2024-09-12T03:00:58.349504+0000","last_active":"2024-09-12T03:02:04.290553+0000","last_peered":"2024-09-12T03:02:04.290553+0000","last_clean":"2024-09-12T03:02:04.290553+0000","last_became_active":"2024-09-12T03:00:58.349136+0000","last_became_peered":"2024-09-12T03:00:58.349136+0000","last_unstale":"2024-09-12T03:02:04.290553+0000","last_undegraded":"2024-09-12T03:02:04.290553+0000","last_fullsized":"2024-09-12T03:02:04.290553+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:09:55.229516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.286399+0000","last_change":"2024-09-12T03:00:58.341902+0000","last_active":"2024-09-12T03:02:04.286399+0000","last_peered":"2024-09-12T03:02:04.286399+0000","last_clean":"2024-09-12T03:02:04.286399+0000","last_became_active":"2024-09-12T03:00:58.341592+0000","last_became_peered":"2024-09-12T03:00:58.341592+0000","last_unstale":"2024-09-12T03:02:04.286399+0000","last_undegraded":"2024-09-12T03:02:04.286399+0000","last_fullsized":"2024-09-12T03:02:04.286399+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:13.852511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332298+0000","last_change":"2024-09-12T03:00:58.348012+0000","last_active":"2024-09-12T03:02:04.332298+0000","last_peered":"2024-09-12T03:02:04.332298+0000","last_clean":"2024-09-12T03:02:04.332298+0000","last_became_active":"2024-09-12T03:00:58.347338+0000","last_became_peered":"2024-09-12T03:00:58.347338+0000","last_unstale":"2024-09-12T03:02:04.332298+0000","last_undegraded":"2024-09-12T03:02:04.332298+0000","last_fullsized":"2024-09-12T03:02:04.332298+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:41.478032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332497+0000","last_change":"2024-09-12T03:00:58.345833+0000","last_active":"2024-09-12T03:02:04.332497+0000","last_peered":"2024-09-12T03:02:04.332497+0000","last_clean":"2024-09-12T03:02:04.332497+0000","last_became_active":"2024-09-12T03:00:58.345482+0000","last_became_peered":"2024-09-12T03:00:58.345482+0000","last_unstale":"2024-09-12T03:02:04.332497+0000","last_undegraded":"2024-09-12T03:02:04.332497+0000","last_fullsized":"2024-09-12T03:02:04.332497+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:32.037412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289995+0000","last_change":"2024-09-12T03:00:58.352290+0000","last_active":"2024-09-12T03:02:04.289995+0000","last_peered":"2024-09-12T03:02:04.289995+0000","last_clean":"2024-09-12T03:02:04.289995+0000","last_became_active":"2024-09-12T03:00:58.351918+0000","last_became_peered":"2024-09-12T03:00:58.351918+0000","last_unstale":"2024-09-12T03:02:04.289995+0000","last_undegraded":"2024-09-12T03:02:04.289995+0000","last_fullsized":"2024-09-12T03:02:04.289995+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:25.941080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.290160+0000","last_change":"2024-09-12T03:00:58.348021+0000","last_active":"2024-09-12T03:02:04.290160+0000","last_peered":"2024-09-12T03:02:04.290160+0000","last_clean":"2024-09-12T03:02:04.290160+0000","last_became_active":"2024-09-12T03:00:58.347634+0000","last_became_peered":"2024-09-12T03:00:58.347634+0000","last_unstale":"2024-09-12T03:02:04.290160+0000","last_undegraded":"2024-09-12T03:02:04.290160+0000","last_fullsized":"2024-09-12T03:02:04.290160+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:47.933067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.285993+0000","last_change":"2024-09-12T03:00:58.340891+0000","last_active":"2024-09-12T03:02:04.285993+0000","last_peered":"2024-09-12T03:02:04.285993+0000","last_clean":"2024-09-12T03:02:04.285993+0000","last_became_active":"2024-09-12T03:00:58.340504+0000","last_became_peered":"2024-09-12T03:00:58.340504+0000","last_unstale":"2024-09-12T03:02:04.285993+0000","last_undegraded":"2024-09-12T03:02:04.285993+0000","last_fullsized":"2024-09-12T03:02:04.285993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:32.117635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.291866+0000","last_change":"2024-09-12T03:00:58.339343+0000","last_active":"2024-09-12T03:02:04.291866+0000","last_peered":"2024-09-12T03:02:04.291866+0000","last_clean":"2024-09-12T03:02:04.291866+0000","last_became_active":"2024-09-12T03:00:58.339001+0000","last_became_peered":"2024-09-12T03:00:58.339001+0000","last_unstale":"2024-09-12T03:02:04.291866+0000","last_undegraded":"2024-09-12T03:02:04.291866+0000","last_fullsized":"2024-09-12T03:02:04.291866+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:02.043454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":90,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:07.488587+0000","last_change":"2024-09-12T03:00:56.315550+0000","last_active":"2024-09-12T03:02:07.488587+0000","last_peered":"2024-09-12T03:02:07.488587+0000","last_clean":"2024-09-12T03:02:07.488587+0000","last_became_active":"2024-09-12T03:00:56.315225+0000","last_became_peered":"2024-09-12T03:00:56.315225+0000","last_unstale":"2024-09-12T03:02:07.488587+0000","last_undegraded":"2024-09-12T03:02:07.488587+0000","last_fullsized":"2024-09-12T03:02:07.488587+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:46.070895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":63,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.290858+0000","last_change":"2024-09-12T03:02:04.290498+0000","last_active":"2024-09-12T03:02:04.290858+0000","last_peered":"2024-09-12T03:02:04.290858+0000","last_clean":"2024-09-12T03:02:04.290858+0000","last_became_active":"2024-09-12T03:02:04.290162+0000","last_became_peered":"2024-09-12T03:02:04.290162+0000","last_unstale":"2024-09-12T03:02:04.290858+0000","last_undegraded":"2024-09-12T03:02:04.290858+0000","last_fullsized":"2024-09-12T03:02:04.290858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:29.777155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282566+0000","last_change":"2024-09-12T03:00:56.316385+0000","last_active":"2024-09-12T03:02:04.282566+0000","last_peered":"2024-09-12T03:02:04.282566+0000","last_clean":"2024-09-12T03:02:04.282566+0000","last_became_active":"2024-09-12T03:00:56.316064+0000","last_became_peered":"2024-09-12T03:00:56.316064+0000","last_unstale":"2024-09-12T03:02:04.282566+0000","last_undegraded":"2024-09-12T03:02:04.282566+0000","last_fullsized":"2024-09-12T03:02:04.282566+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:12:43.323654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332424+0000","last_change":"2024-09-12T03:00:56.313065+0000","last_active":"2024-09-12T03:02:04.332424+0000","last_peered":"2024-09-12T03:02:04.332424+0000","last_clean":"2024-09-12T03:02:04.332424+0000","last_became_active":"2024-09-12T03:00:56.312716+0000","last_became_peered":"2024-09-12T03:00:56.312716+0000","last_unstale":"2024-09-12T03:02:04.332424+0000","last_undegraded":"2024-09-12T03:02:04.332424+0000","last_fullsized":"2024-09-12T03:02:04.332424+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:51.089089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282199+0000","last_change":"2024-09-12T03:00:58.346542+0000","last_active":"2024-09-12T03:02:04.282199+0000","last_peered":"2024-09-12T03:02:04.282199+0000","last_clean":"2024-09-12T03:02:04.282199+0000","last_became_active":"2024-09-12T03:00:58.346260+0000","last_became_peered":"2024-09-12T03:00:58.346260+0000","last_unstale":"2024-09-12T03:02:04.282199+0000","last_undegraded":"2024-09-12T03:02:04.282199+0000","last_fullsized":"2024-09-12T03:02:04.282199+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:37.034969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289182+0000","last_change":"2024-09-12T03:00:56.317308+0000","last_active":"2024-09-12T03:02:04.289182+0000","last_peered":"2024-09-12T03:02:04.289182+0000","last_clean":"2024-09-12T03:02:04.289182+0000","last_became_active":"2024-09-12T03:00:56.316935+0000","last_became_peered":"2024-09-12T03:00:56.316935+0000","last_unstale":"2024-09-12T03:02:04.289182+0000","last_undegraded":"2024-09-12T03:02:04.289182+0000","last_fullsized":"2024-09-12T03:02:04.289182+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:06.324489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.283990+0000","last_change":"2024-09-12T03:00:58.341565+0000","last_active":"2024-09-12T03:02:04.283990+0000","last_peered":"2024-09-12T03:02:04.283990+0000","last_clean":"2024-09-12T03:02:04.283990+0000","last_became_active":"2024-09-12T03:00:58.341285+0000","last_became_peered":"2024-09-12T03:00:58.341285+0000","last_unstale":"2024-09-12T03:02:04.283990+0000","last_undegraded":"2024-09-12T03:02:04.283990+0000","last_fullsized":"2024-09-12T03:02:04.283990+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:59.156700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":83,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.483794+0000","last_change":"2024-09-12T03:00:56.314201+0000","last_active":"2024-09-12T03:02:04.483794+0000","last_peered":"2024-09-12T03:02:04.483794+0000","last_clean":"2024-09-12T03:02:04.483794+0000","last_became_active":"2024-09-12T03:00:56.313880+0000","last_became_peered":"2024-09-12T03:00:56.313880+0000","last_unstale":"2024-09-12T03:02:04.483794+0000","last_undegraded":"2024-09-12T03:02:04.483794+0000","last_fullsized":"2024-09-12T03:02:04.483794+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:57:12.952630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332657+0000","last_change":"2024-09-12T03:00:58.345274+0000","last_active":"2024-09-12T03:02:04.332657+0000","last_peered":"2024-09-12T03:02:04.332657+0000","last_clean":"2024-09-12T03:02:04.332657+0000","last_became_active":"2024-09-12T03:00:58.344937+0000","last_became_peered":"2024-09-12T03:00:58.344937+0000","last_unstale":"2024-09-12T03:02:04.332657+0000","last_undegraded":"2024-09-12T03:02:04.332657+0000","last_fullsized":"2024-09-12T03:02:04.332657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:17.288909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284614+0000","last_change":"2024-09-12T03:00:56.314709+0000","last_active":"2024-09-12T03:02:04.284614+0000","last_peered":"2024-09-12T03:02:04.284614+0000","last_clean":"2024-09-12T03:02:04.284614+0000","last_became_active":"2024-09-12T03:00:56.314349+0000","last_became_peered":"2024-09-12T03:00:56.314349+0000","last_unstale":"2024-09-12T03:02:04.284614+0000","last_undegraded":"2024-09-12T03:02:04.284614+0000","last_fullsized":"2024-09-12T03:02:04.284614+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:18:35.371661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332796+0000","last_change":"2024-09-12T03:00:58.346055+0000","last_active":"2024-09-12T03:02:04.332796+0000","last_peered":"2024-09-12T03:02:04.332796+0000","last_clean":"2024-09-12T03:02:04.332796+0000","last_became_active":"2024-09-12T03:00:58.345701+0000","last_became_peered":"2024-09-12T03:00:58.345701+0000","last_unstale":"2024-09-12T03:02:04.332796+0000","last_undegraded":"2024-09-12T03:02:04.332796+0000","last_fullsized":"2024-09-12T03:02:04.332796+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:33.977245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.287450+0000","last_change":"2024-09-12T03:00:56.435962+0000","last_active":"2024-09-12T03:02:04.287450+0000","last_peered":"2024-09-12T03:02:04.287450+0000","last_clean":"2024-09-12T03:02:04.287450+0000","last_became_active":"2024-09-12T03:00:56.435515+0000","last_became_peered":"2024-09-12T03:00:56.435515+0000","last_unstale":"2024-09-12T03:02:04.287450+0000","last_undegraded":"2024-09-12T03:02:04.287450+0000","last_fullsized":"2024-09-12T03:02:04.287450+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:32.953436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332423+0000","last_change":"2024-09-12T03:00:58.344239+0000","last_active":"2024-09-12T03:02:04.332423+0000","last_peered":"2024-09-12T03:02:04.332423+0000","last_clean":"2024-09-12T03:02:04.332423+0000","last_became_active":"2024-09-12T03:00:58.343876+0000","last_became_peered":"2024-09-12T03:00:58.343876+0000","last_unstale":"2024-09-12T03:02:04.332423+0000","last_undegraded":"2024-09-12T03:02:04.332423+0000","last_fullsized":"2024-09-12T03:02:04.332423+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:49.401295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'29","reported_seq":176,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:05.485172+0000","last_change":"2024-09-12T03:00:56.317708+0000","last_active":"2024-09-12T03:02:05.485172+0000","last_peered":"2024-09-12T03:02:05.485172+0000","last_clean":"2024-09-12T03:02:05.485172+0000","last_became_active":"2024-09-12T03:00:56.317317+0000","last_became_peered":"2024-09-12T03:00:56.317317+0000","last_unstale":"2024-09-12T03:02:05.485172+0000","last_undegraded":"2024-09-12T03:02:05.485172+0000","last_fullsized":"2024-09-12T03:02:05.485172+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:47.788186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61752,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":125,"num_read_kb":122,"num_write":30,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292083+0000","last_change":"2024-09-12T03:00:58.343602+0000","last_active":"2024-09-12T03:02:04.292083+0000","last_peered":"2024-09-12T03:02:04.292083+0000","last_clean":"2024-09-12T03:02:04.292083+0000","last_became_active":"2024-09-12T03:00:58.343301+0000","last_became_peered":"2024-09-12T03:00:58.343301+0000","last_unstale":"2024-09-12T03:02:04.292083+0000","last_undegraded":"2024-09-12T03:02:04.292083+0000","last_fullsized":"2024-09-12T03:02:04.292083+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:04.056118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.290204+0000","last_change":"2024-09-12T03:00:56.317874+0000","last_active":"2024-09-12T03:02:04.290204+0000","last_peered":"2024-09-12T03:02:04.290204+0000","last_clean":"2024-09-12T03:02:04.290204+0000","last_became_active":"2024-09-12T03:00:56.317566+0000","last_became_peered":"2024-09-12T03:00:56.317566+0000","last_unstale":"2024-09-12T03:02:04.290204+0000","last_undegraded":"2024-09-12T03:02:04.290204+0000","last_fullsized":"2024-09-12T03:02:04.290204+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:54.203730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332601+0000","last_change":"2024-09-12T03:00:58.351539+0000","last_active":"2024-09-12T03:02:04.332601+0000","last_peered":"2024-09-12T03:02:04.332601+0000","last_clean":"2024-09-12T03:02:04.332601+0000","last_became_active":"2024-09-12T03:00:58.351249+0000","last_became_peered":"2024-09-12T03:00:58.351249+0000","last_unstale":"2024-09-12T03:02:04.332601+0000","last_undegraded":"2024-09-12T03:02:04.332601+0000","last_fullsized":"2024-09-12T03:02:04.332601+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:40.498867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.287068+0000","last_change":"2024-09-12T03:00:56.314489+0000","last_active":"2024-09-12T03:02:04.287068+0000","last_peered":"2024-09-12T03:02:04.287068+0000","last_clean":"2024-09-12T03:02:04.287068+0000","last_became_active":"2024-09-12T03:00:56.314142+0000","last_became_peered":"2024-09-12T03:00:56.314142+0000","last_unstale":"2024-09-12T03:02:04.287068+0000","last_undegraded":"2024-09-12T03:02:04.287068+0000","last_fullsized":"2024-09-12T03:02:04.287068+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:21.541713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284409+0000","last_change":"2024-09-12T03:00:58.348752+0000","last_active":"2024-09-12T03:02:04.284409+0000","last_peered":"2024-09-12T03:02:04.284409+0000","last_clean":"2024-09-12T03:02:04.284409+0000","last_became_active":"2024-09-12T03:00:58.348391+0000","last_became_peered":"2024-09-12T03:00:58.348391+0000","last_unstale":"2024-09-12T03:02:04.284409+0000","last_undegraded":"2024-09-12T03:02:04.284409+0000","last_fullsized":"2024-09-12T03:02:04.284409+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:33:15.626306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'3","reported_seq":89,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:08.489087+0000","last_change":"2024-09-12T03:00:56.311702+0000","last_active":"2024-09-12T03:02:08.489087+0000","last_peered":"2024-09-12T03:02:08.489087+0000","last_clean":"2024-09-12T03:02:08.489087+0000","last_became_active":"2024-09-12T03:00:56.311353+0000","last_became_peered":"2024-09-12T03:00:56.311353+0000","last_unstale":"2024-09-12T03:02:08.489087+0000","last_undegraded":"2024-09-12T03:02:08.489087+0000","last_fullsized":"2024-09-12T03:02:08.489087+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:01.004858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284051+0000","last_change":"2024-09-12T03:00:58.351973+0000","last_active":"2024-09-12T03:02:04.284051+0000","last_peered":"2024-09-12T03:02:04.284051+0000","last_clean":"2024-09-12T03:02:04.284051+0000","last_became_active":"2024-09-12T03:00:58.351599+0000","last_became_peered":"2024-09-12T03:00:58.351599+0000","last_unstale":"2024-09-12T03:02:04.284051+0000","last_undegraded":"2024-09-12T03:02:04.284051+0000","last_fullsized":"2024-09-12T03:02:04.284051+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:26.398828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.291749+0000","last_change":"2024-09-12T03:00:56.436185+0000","last_active":"2024-09-12T03:02:04.291749+0000","last_peered":"2024-09-12T03:02:04.291749+0000","last_clean":"2024-09-12T03:02:04.291749+0000","last_became_active":"2024-09-12T03:00:56.435717+0000","last_became_peered":"2024-09-12T03:00:56.435717+0000","last_unstale":"2024-09-12T03:02:04.291749+0000","last_undegraded":"2024-09-12T03:02:04.291749+0000","last_fullsized":"2024-09-12T03:02:04.291749+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:27.303182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.333661+0000","last_change":"2024-09-12T03:02:04.333661+0000","last_active":"2024-09-12T03:02:04.333661+0000","last_peered":"2024-09-12T03:02:04.333661+0000","last_clean":"2024-09-12T03:02:04.333661+0000","last_became_active":"2024-09-12T03:02:04.333217+0000","last_became_peered":"2024-09-12T03:02:04.333217+0000","last_unstale":"2024-09-12T03:02:04.333661+0000","last_undegraded":"2024-09-12T03:02:04.333661+0000","last_fullsized":"2024-09-12T03:02:04.333661+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:23:47.212734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.297341+0000","last_change":"2024-09-12T03:02:04.296884+0000","last_active":"2024-09-12T03:02:04.297341+0000","last_peered":"2024-09-12T03:02:04.297341+0000","last_clean":"2024-09-12T03:02:04.297341+0000","last_became_active":"2024-09-12T03:02:04.290894+0000","last_became_peered":"2024-09-12T03:02:04.290894+0000","last_unstale":"2024-09-12T03:02:04.297341+0000","last_undegraded":"2024-09-12T03:02:04.297341+0000","last_fullsized":"2024-09-12T03:02:04.297341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:49:19.749568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.334096+0000","last_change":"2024-09-12T03:02:04.334096+0000","last_active":"2024-09-12T03:02:04.334096+0000","last_peered":"2024-09-12T03:02:04.334096+0000","last_clean":"2024-09-12T03:02:04.334096+0000","last_became_active":"2024-09-12T03:02:04.333641+0000","last_became_peered":"2024-09-12T03:02:04.333641+0000","last_unstale":"2024-09-12T03:02:04.334096+0000","last_undegraded":"2024-09-12T03:02:04.334096+0000","last_fullsized":"2024-09-12T03:02:04.334096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:24.085293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332966+0000","last_change":"2024-09-12T03:00:56.314182+0000","last_active":"2024-09-12T03:02:04.332966+0000","last_peered":"2024-09-12T03:02:04.332966+0000","last_clean":"2024-09-12T03:02:04.332966+0000","last_became_active":"2024-09-12T03:00:56.313749+0000","last_became_peered":"2024-09-12T03:00:56.313749+0000","last_unstale":"2024-09-12T03:02:04.332966+0000","last_undegraded":"2024-09-12T03:02:04.332966+0000","last_fullsized":"2024-09-12T03:02:04.332966+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:31:45.767500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284764+0000","last_change":"2024-09-12T03:00:58.344356+0000","last_active":"2024-09-12T03:02:04.284764+0000","last_peered":"2024-09-12T03:02:04.284764+0000","last_clean":"2024-09-12T03:02:04.284764+0000","last_became_active":"2024-09-12T03:00:58.344035+0000","last_became_peered":"2024-09-12T03:00:58.344035+0000","last_unstale":"2024-09-12T03:02:04.284764+0000","last_undegraded":"2024-09-12T03:02:04.284764+0000","last_fullsized":"2024-09-12T03:02:04.284764+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:45:18.681018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.287585+0000","last_change":"2024-09-12T03:00:56.315653+0000","last_active":"2024-09-12T03:02:04.287585+0000","last_peered":"2024-09-12T03:02:04.287585+0000","last_clean":"2024-09-12T03:02:04.287585+0000","last_became_active":"2024-09-12T03:00:56.315284+0000","last_became_peered":"2024-09-12T03:00:56.315284+0000","last_unstale":"2024-09-12T03:02:04.287585+0000","last_undegraded":"2024-09-12T03:02:04.287585+0000","last_fullsized":"2024-09-12T03:02:04.287585+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:42.934020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.291128+0000","last_change":"2024-09-12T03:02:04.291128+0000","last_active":"2024-09-12T03:02:04.291128+0000","last_peered":"2024-09-12T03:02:04.291128+0000","last_clean":"2024-09-12T03:02:04.291128+0000","last_became_active":"2024-09-12T03:02:04.287836+0000","last_became_peered":"2024-09-12T03:02:04.287836+0000","last_unstale":"2024-09-12T03:02:04.291128+0000","last_undegraded":"2024-09-12T03:02:04.291128+0000","last_fullsized":"2024-09-12T03:02:04.291128+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:23.956579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.290577+0000","last_change":"2024-09-12T03:00:56.746780+0000","last_active":"2024-09-12T03:02:04.290577+0000","last_peered":"2024-09-12T03:02:04.290577+0000","last_clean":"2024-09-12T03:02:04.290577+0000","last_became_active":"2024-09-12T03:00:56.746414+0000","last_became_peered":"2024-09-12T03:00:56.746414+0000","last_unstale":"2024-09-12T03:02:04.290577+0000","last_undegraded":"2024-09-12T03:02:04.290577+0000","last_fullsized":"2024-09-12T03:02:04.290577+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:15:35.214196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284593+0000","last_change":"2024-09-12T03:00:58.351713+0000","last_active":"2024-09-12T03:02:04.284593+0000","last_peered":"2024-09-12T03:02:04.284593+0000","last_clean":"2024-09-12T03:02:04.284593+0000","last_became_active":"2024-09-12T03:00:58.351371+0000","last_became_peered":"2024-09-12T03:00:58.351371+0000","last_unstale":"2024-09-12T03:02:04.284593+0000","last_undegraded":"2024-09-12T03:02:04.284593+0000","last_fullsized":"2024-09-12T03:02:04.284593+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:13.346956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'137","reported_seq":335,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284267+0000","last_change":"2024-09-12T02:59:45.461482+0000","last_active":"2024-09-12T03:02:04.284267+0000","last_peered":"2024-09-12T03:02:04.284267+0000","last_clean":"2024-09-12T03:02:04.284267+0000","last_became_active":"2024-09-12T02:59:45.461058+0000","last_became_peered":"2024-09-12T02:59:45.461058+0000","last_unstale":"2024-09-12T03:02:04.284267+0000","last_undegraded":"2024-09-12T03:02:04.284267+0000","last_fullsized":"2024-09-12T03:02:04.284267+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:18.034438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:18.034438+0000","last_clean_scrub_stamp":"2024-09-12T02:56:18.034438+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:09.251858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289291+0000","last_change":"2024-09-12T03:00:56.315245+0000","last_active":"2024-09-12T03:02:04.289291+0000","last_peered":"2024-09-12T03:02:04.289291+0000","last_clean":"2024-09-12T03:02:04.289291+0000","last_became_active":"2024-09-12T03:00:56.314887+0000","last_became_peered":"2024-09-12T03:00:56.314887+0000","last_unstale":"2024-09-12T03:02:04.289291+0000","last_undegraded":"2024-09-12T03:02:04.289291+0000","last_fullsized":"2024-09-12T03:02:04.289291+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:48.750657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.283183+0000","last_change":"2024-09-12T03:00:58.357874+0000","last_active":"2024-09-12T03:02:04.283183+0000","last_peered":"2024-09-12T03:02:04.283183+0000","last_clean":"2024-09-12T03:02:04.283183+0000","last_became_active":"2024-09-12T03:00:58.357575+0000","last_became_peered":"2024-09-12T03:00:58.357575+0000","last_unstale":"2024-09-12T03:02:04.283183+0000","last_undegraded":"2024-09-12T03:02:04.283183+0000","last_fullsized":"2024-09-12T03:02:04.283183+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:46.125620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284212+0000","last_change":"2024-09-12T03:00:56.312263+0000","last_active":"2024-09-12T03:02:04.284212+0000","last_peered":"2024-09-12T03:02:04.284212+0000","last_clean":"2024-09-12T03:02:04.284212+0000","last_became_active":"2024-09-12T03:00:56.311894+0000","last_became_peered":"2024-09-12T03:00:56.311894+0000","last_unstale":"2024-09-12T03:02:04.284212+0000","last_undegraded":"2024-09-12T03:02:04.284212+0000","last_fullsized":"2024-09-12T03:02:04.284212+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:52.434203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332322+0000","last_change":"2024-09-12T03:00:58.350762+0000","last_active":"2024-09-12T03:02:04.332322+0000","last_peered":"2024-09-12T03:02:04.332322+0000","last_clean":"2024-09-12T03:02:04.332322+0000","last_became_active":"2024-09-12T03:00:58.348038+0000","last_became_peered":"2024-09-12T03:00:58.348038+0000","last_unstale":"2024-09-12T03:02:04.332322+0000","last_undegraded":"2024-09-12T03:02:04.332322+0000","last_fullsized":"2024-09-12T03:02:04.332322+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:08.916636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":84,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.483168+0000","last_change":"2024-09-12T03:00:56.435641+0000","last_active":"2024-09-12T03:02:04.483168+0000","last_peered":"2024-09-12T03:02:04.483168+0000","last_clean":"2024-09-12T03:02:04.483168+0000","last_became_active":"2024-09-12T03:00:56.435319+0000","last_became_peered":"2024-09-12T03:00:56.435319+0000","last_unstale":"2024-09-12T03:02:04.483168+0000","last_undegraded":"2024-09-12T03:02:04.483168+0000","last_fullsized":"2024-09-12T03:02:04.483168+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:56.894978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.288114+0000","last_change":"2024-09-12T03:00:58.344708+0000","last_active":"2024-09-12T03:02:04.288114+0000","last_peered":"2024-09-12T03:02:04.288114+0000","last_clean":"2024-09-12T03:02:04.288114+0000","last_became_active":"2024-09-12T03:00:58.344389+0000","last_became_peered":"2024-09-12T03:00:58.344389+0000","last_unstale":"2024-09-12T03:02:04.288114+0000","last_undegraded":"2024-09-12T03:02:04.288114+0000","last_fullsized":"2024-09-12T03:02:04.288114+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:08.160383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284317+0000","last_change":"2024-09-12T03:00:56.317068+0000","last_active":"2024-09-12T03:02:04.284317+0000","last_peered":"2024-09-12T03:02:04.284317+0000","last_clean":"2024-09-12T03:02:04.284317+0000","last_became_active":"2024-09-12T03:00:56.316698+0000","last_became_peered":"2024-09-12T03:00:56.316698+0000","last_unstale":"2024-09-12T03:02:04.284317+0000","last_undegraded":"2024-09-12T03:02:04.284317+0000","last_fullsized":"2024-09-12T03:02:04.284317+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:41:19.199418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282614+0000","last_change":"2024-09-12T03:00:58.347814+0000","last_active":"2024-09-12T03:02:04.282614+0000","last_peered":"2024-09-12T03:02:04.282614+0000","last_clean":"2024-09-12T03:02:04.282614+0000","last_became_active":"2024-09-12T03:00:58.347112+0000","last_became_peered":"2024-09-12T03:00:58.347112+0000","last_unstale":"2024-09-12T03:02:04.282614+0000","last_undegraded":"2024-09-12T03:02:04.282614+0000","last_fullsized":"2024-09-12T03:02:04.282614+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:59.325113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284265+0000","last_change":"2024-09-12T03:00:56.435771+0000","last_active":"2024-09-12T03:02:04.284265+0000","last_peered":"2024-09-12T03:02:04.284265+0000","last_clean":"2024-09-12T03:02:04.284265+0000","last_became_active":"2024-09-12T03:00:56.435448+0000","last_became_peered":"2024-09-12T03:00:56.435448+0000","last_unstale":"2024-09-12T03:02:04.284265+0000","last_undegraded":"2024-09-12T03:02:04.284265+0000","last_fullsized":"2024-09-12T03:02:04.284265+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:33.650338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282832+0000","last_change":"2024-09-12T03:00:58.341071+0000","last_active":"2024-09-12T03:02:04.282832+0000","last_peered":"2024-09-12T03:02:04.282832+0000","last_clean":"2024-09-12T03:02:04.282832+0000","last_became_active":"2024-09-12T03:00:58.340753+0000","last_became_peered":"2024-09-12T03:00:58.340753+0000","last_unstale":"2024-09-12T03:02:04.282832+0000","last_undegraded":"2024-09-12T03:02:04.282832+0000","last_fullsized":"2024-09-12T03:02:04.282832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:21.217915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292254+0000","last_change":"2024-09-12T03:00:56.436058+0000","last_active":"2024-09-12T03:02:04.292254+0000","last_peered":"2024-09-12T03:02:04.292254+0000","last_clean":"2024-09-12T03:02:04.292254+0000","last_became_active":"2024-09-12T03:00:56.435518+0000","last_became_peered":"2024-09-12T03:00:56.435518+0000","last_unstale":"2024-09-12T03:02:04.292254+0000","last_undegraded":"2024-09-12T03:02:04.292254+0000","last_fullsized":"2024-09-12T03:02:04.292254+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:32.235097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332392+0000","last_change":"2024-09-12T03:00:58.348236+0000","last_active":"2024-09-12T03:02:04.332392+0000","last_peered":"2024-09-12T03:02:04.332392+0000","last_clean":"2024-09-12T03:02:04.332392+0000","last_became_active":"2024-09-12T03:00:58.347727+0000","last_became_peered":"2024-09-12T03:00:58.347727+0000","last_unstale":"2024-09-12T03:02:04.332392+0000","last_undegraded":"2024-09-12T03:02:04.332392+0000","last_fullsized":"2024-09-12T03:02:04.332392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:19.233163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284575+0000","last_change":"2024-09-12T03:00:56.310537+0000","last_active":"2024-09-12T03:02:04.284575+0000","last_peered":"2024-09-12T03:02:04.284575+0000","last_clean":"2024-09-12T03:02:04.284575+0000","last_became_active":"2024-09-12T03:00:56.310135+0000","last_became_peered":"2024-09-12T03:00:56.310135+0000","last_unstale":"2024-09-12T03:02:04.284575+0000","last_undegraded":"2024-09-12T03:02:04.284575+0000","last_fullsized":"2024-09-12T03:02:04.284575+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:10:55.584339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.283645+0000","last_change":"2024-09-12T03:00:58.358417+0000","last_active":"2024-09-12T03:02:04.283645+0000","last_peered":"2024-09-12T03:02:04.283645+0000","last_clean":"2024-09-12T03:02:04.283645+0000","last_became_active":"2024-09-12T03:00:58.358119+0000","last_became_peered":"2024-09-12T03:00:58.358119+0000","last_unstale":"2024-09-12T03:02:04.283645+0000","last_undegraded":"2024-09-12T03:02:04.283645+0000","last_fullsized":"2024-09-12T03:02:04.283645+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:10.897594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289526+0000","last_change":"2024-09-12T03:00:56.747044+0000","last_active":"2024-09-12T03:02:04.289526+0000","last_peered":"2024-09-12T03:02:04.289526+0000","last_clean":"2024-09-12T03:02:04.289526+0000","last_became_active":"2024-09-12T03:00:56.746699+0000","last_became_peered":"2024-09-12T03:00:56.746699+0000","last_unstale":"2024-09-12T03:02:04.289526+0000","last_undegraded":"2024-09-12T03:02:04.289526+0000","last_fullsized":"2024-09-12T03:02:04.289526+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:11.420109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332683+0000","last_change":"2024-09-12T03:00:58.346854+0000","last_active":"2024-09-12T03:02:04.332683+0000","last_peered":"2024-09-12T03:02:04.332683+0000","last_clean":"2024-09-12T03:02:04.332683+0000","last_became_active":"2024-09-12T03:00:58.346535+0000","last_became_peered":"2024-09-12T03:00:58.346535+0000","last_unstale":"2024-09-12T03:02:04.332683+0000","last_undegraded":"2024-09-12T03:02:04.332683+0000","last_fullsized":"2024-09-12T03:02:04.332683+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:46.637914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332365+0000","last_change":"2024-09-12T03:00:56.435957+0000","last_active":"2024-09-12T03:02:04.332365+0000","last_peered":"2024-09-12T03:02:04.332365+0000","last_clean":"2024-09-12T03:02:04.332365+0000","last_became_active":"2024-09-12T03:00:56.435603+0000","last_became_peered":"2024-09-12T03:00:56.435603+0000","last_unstale":"2024-09-12T03:02:04.332365+0000","last_undegraded":"2024-09-12T03:02:04.332365+0000","last_fullsized":"2024-09-12T03:02:04.332365+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:34.553189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.291897+0000","last_change":"2024-09-12T03:00:58.337784+0000","last_active":"2024-09-12T03:02:04.291897+0000","last_peered":"2024-09-12T03:02:04.291897+0000","last_clean":"2024-09-12T03:02:04.291897+0000","last_became_active":"2024-09-12T03:00:58.337415+0000","last_became_peered":"2024-09-12T03:00:58.337415+0000","last_unstale":"2024-09-12T03:02:04.291897+0000","last_undegraded":"2024-09-12T03:02:04.291897+0000","last_fullsized":"2024-09-12T03:02:04.291897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:38:15.557067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282354+0000","last_change":"2024-09-12T03:00:56.315356+0000","last_active":"2024-09-12T03:02:04.282354+0000","last_peered":"2024-09-12T03:02:04.282354+0000","last_clean":"2024-09-12T03:02:04.282354+0000","last_became_active":"2024-09-12T03:00:56.314873+0000","last_became_peered":"2024-09-12T03:00:56.314873+0000","last_unstale":"2024-09-12T03:02:04.282354+0000","last_undegraded":"2024-09-12T03:02:04.282354+0000","last_fullsized":"2024-09-12T03:02:04.282354+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:55.710156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.286199+0000","last_change":"2024-09-12T03:00:58.346706+0000","last_active":"2024-09-12T03:02:04.286199+0000","last_peered":"2024-09-12T03:02:04.286199+0000","last_clean":"2024-09-12T03:02:04.286199+0000","last_became_active":"2024-09-12T03:00:58.346363+0000","last_became_peered":"2024-09-12T03:00:58.346363+0000","last_unstale":"2024-09-12T03:02:04.286199+0000","last_undegraded":"2024-09-12T03:02:04.286199+0000","last_fullsized":"2024-09-12T03:02:04.286199+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:13.468639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.299582+0000","last_change":"2024-09-12T03:02:04.299147+0000","last_active":"2024-09-12T03:02:04.299582+0000","last_peered":"2024-09-12T03:02:04.299582+0000","last_clean":"2024-09-12T03:02:04.299582+0000","last_became_active":"2024-09-12T03:02:04.293211+0000","last_became_peered":"2024-09-12T03:02:04.293211+0000","last_unstale":"2024-09-12T03:02:04.299582+0000","last_undegraded":"2024-09-12T03:02:04.299582+0000","last_fullsized":"2024-09-12T03:02:04.299582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:01:39.222476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282418+0000","last_change":"2024-09-12T03:00:58.356682+0000","last_active":"2024-09-12T03:02:04.282418+0000","last_peered":"2024-09-12T03:02:04.282418+0000","last_clean":"2024-09-12T03:02:04.282418+0000","last_became_active":"2024-09-12T03:00:58.356383+0000","last_became_peered":"2024-09-12T03:00:58.356383+0000","last_unstale":"2024-09-12T03:02:04.282418+0000","last_undegraded":"2024-09-12T03:02:04.282418+0000","last_fullsized":"2024-09-12T03:02:04.282418+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:14:16.428393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292053+0000","last_change":"2024-09-12T03:00:56.314943+0000","last_active":"2024-09-12T03:02:04.292053+0000","last_peered":"2024-09-12T03:02:04.292053+0000","last_clean":"2024-09-12T03:02:04.292053+0000","last_became_active":"2024-09-12T03:00:56.314602+0000","last_became_peered":"2024-09-12T03:00:56.314602+0000","last_unstale":"2024-09-12T03:02:04.292053+0000","last_undegraded":"2024-09-12T03:02:04.292053+0000","last_fullsized":"2024-09-12T03:02:04.292053+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:31.207880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284215+0000","last_change":"2024-09-12T03:00:58.351774+0000","last_active":"2024-09-12T03:02:04.284215+0000","last_peered":"2024-09-12T03:02:04.284215+0000","last_clean":"2024-09-12T03:02:04.284215+0000","last_became_active":"2024-09-12T03:00:58.351453+0000","last_became_peered":"2024-09-12T03:00:58.351453+0000","last_unstale":"2024-09-12T03:02:04.284215+0000","last_undegraded":"2024-09-12T03:02:04.284215+0000","last_fullsized":"2024-09-12T03:02:04.284215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:18:24.260110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.287322+0000","last_change":"2024-09-12T03:00:58.346886+0000","last_active":"2024-09-12T03:02:04.287322+0000","last_peered":"2024-09-12T03:02:04.287322+0000","last_clean":"2024-09-12T03:02:04.287322+0000","last_became_active":"2024-09-12T03:00:58.346499+0000","last_became_peered":"2024-09-12T03:00:58.346499+0000","last_unstale":"2024-09-12T03:02:04.287322+0000","last_undegraded":"2024-09-12T03:02:04.287322+0000","last_fullsized":"2024-09-12T03:02:04.287322+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:02:24.045610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292497+0000","last_change":"2024-09-12T03:00:56.317443+0000","last_active":"2024-09-12T03:02:04.292497+0000","last_peered":"2024-09-12T03:02:04.292497+0000","last_clean":"2024-09-12T03:02:04.292497+0000","last_became_active":"2024-09-12T03:00:56.317045+0000","last_became_peered":"2024-09-12T03:00:56.317045+0000","last_unstale":"2024-09-12T03:02:04.292497+0000","last_undegraded":"2024-09-12T03:02:04.292497+0000","last_fullsized":"2024-09-12T03:02:04.292497+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:07.687606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332419+0000","last_change":"2024-09-12T03:00:58.337257+0000","last_active":"2024-09-12T03:02:04.332419+0000","last_peered":"2024-09-12T03:02:04.332419+0000","last_clean":"2024-09-12T03:02:04.332419+0000","last_became_active":"2024-09-12T03:00:58.336847+0000","last_became_peered":"2024-09-12T03:00:58.336847+0000","last_unstale":"2024-09-12T03:02:04.332419+0000","last_undegraded":"2024-09-12T03:02:04.332419+0000","last_fullsized":"2024-09-12T03:02:04.332419+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:22.846206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292165+0000","last_change":"2024-09-12T03:00:56.316255+0000","last_active":"2024-09-12T03:02:04.292165+0000","last_peered":"2024-09-12T03:02:04.292165+0000","last_clean":"2024-09-12T03:02:04.292165+0000","last_became_active":"2024-09-12T03:00:56.315906+0000","last_became_peered":"2024-09-12T03:00:56.315906+0000","last_unstale":"2024-09-12T03:02:04.292165+0000","last_undegraded":"2024-09-12T03:02:04.292165+0000","last_fullsized":"2024-09-12T03:02:04.292165+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:14.798601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332714+0000","last_change":"2024-09-12T03:00:58.347206+0000","last_active":"2024-09-12T03:02:04.332714+0000","last_peered":"2024-09-12T03:02:04.332714+0000","last_clean":"2024-09-12T03:02:04.332714+0000","last_became_active":"2024-09-12T03:00:58.346923+0000","last_became_peered":"2024-09-12T03:00:58.346923+0000","last_unstale":"2024-09-12T03:02:04.332714+0000","last_undegraded":"2024-09-12T03:02:04.332714+0000","last_fullsized":"2024-09-12T03:02:04.332714+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:41.376211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.283694+0000","last_change":"2024-09-12T03:00:56.435431+0000","last_active":"2024-09-12T03:02:04.283694+0000","last_peered":"2024-09-12T03:02:04.283694+0000","last_clean":"2024-09-12T03:02:04.283694+0000","last_became_active":"2024-09-12T03:00:56.435088+0000","last_became_peered":"2024-09-12T03:00:56.435088+0000","last_unstale":"2024-09-12T03:02:04.283694+0000","last_undegraded":"2024-09-12T03:02:04.283694+0000","last_fullsized":"2024-09-12T03:02:04.283694+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:24.738340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.283572+0000","last_change":"2024-09-12T03:00:58.351737+0000","last_active":"2024-09-12T03:02:04.283572+0000","last_peered":"2024-09-12T03:02:04.283572+0000","last_clean":"2024-09-12T03:02:04.283572+0000","last_became_active":"2024-09-12T03:00:58.351407+0000","last_became_peered":"2024-09-12T03:00:58.351407+0000","last_unstale":"2024-09-12T03:02:04.283572+0000","last_undegraded":"2024-09-12T03:02:04.283572+0000","last_fullsized":"2024-09-12T03:02:04.283572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:34.206703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284979+0000","last_change":"2024-09-12T03:00:56.310910+0000","last_active":"2024-09-12T03:02:04.284979+0000","last_peered":"2024-09-12T03:02:04.284979+0000","last_clean":"2024-09-12T03:02:04.284979+0000","last_became_active":"2024-09-12T03:00:56.310657+0000","last_became_peered":"2024-09-12T03:00:56.310657+0000","last_unstale":"2024-09-12T03:02:04.284979+0000","last_undegraded":"2024-09-12T03:02:04.284979+0000","last_fullsized":"2024-09-12T03:02:04.284979+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:06.117215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65362,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":381,"num_read_kb":255,"num_write":161,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":196086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245899,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645340,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":880640,"data_stored":431835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906976,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1028096,"data_stored":1085046,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070502,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":897024,"data_stored":432166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299436,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":431947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528368,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":942080,"data_stored":1023508,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":40,"seq":171798691893,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":880640,"data_stored":431438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":33,"seq":141733920826,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":954368,"data_stored":1023192,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051648,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":884736,"data_stored":431875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280580,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":892928,"data_stored":431632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476809,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":966656,"data_stored":494333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607631,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":942080,"data_stored":492983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":901120,"data_stored":432709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:02:12.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:12 smithi183 ceph-mon[35535]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 17 B/s, 0 objects/s recovering 2024-09-12T03:02:13.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:13 smithi177 ceph-mon[35305]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:02:13.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:13 smithi116 ceph-mon[31410]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:02:13.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:13 smithi183 ceph-mon[35535]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:02:14.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:14 smithi177 ceph-mon[35305]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 14 B/s, 0 objects/s recovering 2024-09-12T03:02:14.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:14 smithi116 ceph-mon[31410]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 14 B/s, 0 objects/s recovering 2024-09-12T03:02:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:14 smithi183 ceph-mon[35535]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 14 B/s, 0 objects/s recovering 2024-09-12T03:02:15.652 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:02:15.910 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:16.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:16 smithi116 ceph-mon[31410]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 12 B/s, 0 objects/s recovering 2024-09-12T03:02:16.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.392+0000 7f782781a640 1 -- 172.21.15.116:0/2467366833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7820101420 msgr2=0x7f782010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:16.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.392+0000 7f782781a640 1 --2- 172.21.15.116:0/2467366833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7820101420 0x7f782010d950 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7810009cd0 tx=0x7f781002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:02:16.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.392+0000 7f782781a640 1 -- 172.21.15.116:0/2467366833 shutdown_connections 2024-09-12T03:02:16.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.392+0000 7f782781a640 1 --2- 172.21.15.116:0/2467366833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7820101420 0x7f782010d950 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:16.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.392+0000 7f782781a640 1 --2- 172.21.15.116:0/2467366833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7820100a80 0x7f7820100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:16.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.392+0000 7f782781a640 1 --2- 172.21.15.116:0/2467366833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7820106a80 0x7f7820106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:16.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.392+0000 7f782781a640 1 -- 172.21.15.116:0/2467366833 >> 172.21.15.116:0/2467366833 conn(0x7f78200fc7a0 msgr2=0x7f78200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:16.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.392+0000 7f782781a640 1 -- 172.21.15.116:0/2467366833 shutdown_connections 2024-09-12T03:02:16.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.392+0000 7f782781a640 1 -- 172.21.15.116:0/2467366833 wait complete. 2024-09-12T03:02:16.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.392+0000 7f782781a640 1 Processor -- start 2024-09-12T03:02:16.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.392+0000 7f782781a640 1 -- start start 2024-09-12T03:02:16.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.392+0000 7f782781a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7820100a80 0x7f78201974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:16.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.393+0000 7f782781a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7820101420 0x7f78201979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:16.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.393+0000 7f782781a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7820106a80 0x7f782019bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:16.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.393+0000 7f782781a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7820110f20 con 0x7f7820106a80 2024-09-12T03:02:16.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.393+0000 7f782781a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7820110da0 con 0x7f7820100a80 2024-09-12T03:02:16.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.393+0000 7f782781a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f78201110a0 con 0x7f7820101420 2024-09-12T03:02:16.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.393+0000 7f7825db7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7820106a80 0x7f782019bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:16.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.393+0000 7f7825db7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7820106a80 0x7f782019bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43338/0 (socket says 172.21.15.116:43338) 2024-09-12T03:02:16.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.393+0000 7f7825db7640 1 -- 172.21.15.116:0/3865218146 learned_addr learned my addr 172.21.15.116:0/3865218146 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:16.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.393+0000 7f78255b6640 1 --2- 172.21.15.116:0/3865218146 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7820100a80 0x7f78201974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:16.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.393+0000 7f7824db5640 1 --2- 172.21.15.116:0/3865218146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7820101420 0x7f78201979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:16.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.393+0000 7f7825db7640 1 -- 172.21.15.116:0/3865218146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7820101420 msgr2=0x7f78201979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:16.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.393+0000 7f7825db7640 1 --2- 172.21.15.116:0/3865218146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7820101420 0x7f78201979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:16.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.393+0000 7f7825db7640 1 -- 172.21.15.116:0/3865218146 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7820100a80 msgr2=0x7f78201974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:16.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.393+0000 7f7825db7640 1 --2- 172.21.15.116:0/3865218146 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7820100a80 0x7f78201974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:16.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.393+0000 7f7825db7640 1 -- 172.21.15.116:0/3865218146 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f782019c450 con 0x7f7820106a80 2024-09-12T03:02:16.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.393+0000 7f7824db5640 1 --2- 172.21.15.116:0/3865218146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7820101420 0x7f78201979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:16.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.393+0000 7f7825db7640 1 --2- 172.21.15.116:0/3865218146 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7820106a80 0x7f782019bd70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7810006010 tx=0x7f7810003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:16.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.393+0000 7f781e7fc640 1 -- 172.21.15.116:0/3865218146 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7810004450 con 0x7f7820106a80 2024-09-12T03:02:16.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.393+0000 7f781e7fc640 1 -- 172.21.15.116:0/3865218146 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7810003e10 con 0x7f7820106a80 2024-09-12T03:02:16.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.393+0000 7f782781a640 1 -- 172.21.15.116:0/3865218146 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f782019c6e0 con 0x7f7820106a80 2024-09-12T03:02:16.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.393+0000 7f781e7fc640 1 -- 172.21.15.116:0/3865218146 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7810046480 con 0x7f7820106a80 2024-09-12T03:02:16.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.393+0000 7f782781a640 1 -- 172.21.15.116:0/3865218146 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78201a4020 con 0x7f7820106a80 2024-09-12T03:02:16.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.394+0000 7f782781a640 1 -- 172.21.15.116:0/3865218146 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7820100f60 con 0x7f7820106a80 2024-09-12T03:02:16.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.395+0000 7f781e7fc640 1 -- 172.21.15.116:0/3865218146 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f78100045f0 con 0x7f7820106a80 2024-09-12T03:02:16.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.396+0000 7f781e7fc640 1 --2- 172.21.15.116:0/3865218146 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f78080a2c10 0x7f78080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:16.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.396+0000 7f78255b6640 1 --2- 172.21.15.116:0/3865218146 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f78080a2c10 0x7f78080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:16.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.396+0000 7f781e7fc640 1 -- 172.21.15.116:0/3865218146 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f781002e030 con 0x7f7820106a80 2024-09-12T03:02:16.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.396+0000 7f78255b6640 1 --2- 172.21.15.116:0/3865218146 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f78080a2c10 0x7f78080a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7818009990 tx=0x7f7818009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:16.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.400+0000 7f781e7fc640 1 -- 172.21.15.116:0/3865218146 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f781004b060 con 0x7f7820106a80 2024-09-12T03:02:16.505 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:16 smithi183 ceph-mon[35535]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 12 B/s, 0 objects/s recovering 2024-09-12T03:02:16.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.591+0000 7f782781a640 1 -- 172.21.15.116:0/3865218146 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7820062630 con 0x7f78080a2c10 2024-09-12T03:02:16.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.598+0000 7f781e7fc640 1 -- 172.21.15.116:0/3865218146 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258863 (secure 0 0 0) 0x7f7820062630 con 0x7f78080a2c10 2024-09-12T03:02:16.601 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:02:16.602 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-09-12T03:02:16.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.602+0000 7f782781a640 1 -- 172.21.15.116:0/3865218146 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f78080a2c10 msgr2=0x7f78080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:16.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.602+0000 7f782781a640 1 --2- 172.21.15.116:0/3865218146 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f78080a2c10 0x7f78080a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7818009990 tx=0x7f7818009110 comp rx=0 tx=0).stop 2024-09-12T03:02:16.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.602+0000 7f782781a640 1 -- 172.21.15.116:0/3865218146 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7820106a80 msgr2=0x7f782019bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:16.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.602+0000 7f782781a640 1 --2- 172.21.15.116:0/3865218146 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7820106a80 0x7f782019bd70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7810006010 tx=0x7f7810003c20 comp rx=0 tx=0).stop 2024-09-12T03:02:16.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.602+0000 7f782781a640 1 -- 172.21.15.116:0/3865218146 shutdown_connections 2024-09-12T03:02:16.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.602+0000 7f782781a640 1 --2- 172.21.15.116:0/3865218146 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f78080a2c10 0x7f78080a50d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:16.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.602+0000 7f782781a640 1 --2- 172.21.15.116:0/3865218146 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7820106a80 0x7f782019bd70 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:16.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.602+0000 7f782781a640 1 --2- 172.21.15.116:0/3865218146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7820101420 0x7f78201979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:16.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.602+0000 7f782781a640 1 --2- 172.21.15.116:0/3865218146 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7820100a80 0x7f78201974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:16.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.602+0000 7f782781a640 1 -- 172.21.15.116:0/3865218146 >> 172.21.15.116:0/3865218146 conn(0x7f78200fc7a0 msgr2=0x7f78200fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:16.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.602+0000 7f782781a640 1 -- 172.21.15.116:0/3865218146 shutdown_connections 2024-09-12T03:02:16.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:16.602+0000 7f782781a640 1 -- 172.21.15.116:0/3865218146 wait complete. 2024-09-12T03:02:16.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:16 smithi177 ceph-mon[35305]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 12 B/s, 0 objects/s recovering 2024-09-12T03:02:17.358 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-12T03:02:15.125385+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655964,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":537,"num_read_kb":379,"num_write":344,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180875,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368444,"kb_used_data":10800,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573380,"statfs":{"total":1151940427776,"available":1151563141120,"internally_reserved":0,"allocated":11059200,"data_stored":7142664,"data_compressed":107490,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":64,"num_read_kb":29,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":139,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.844155"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.287303+0000","last_change":"2024-09-12T03:00:58.348563+0000","last_active":"2024-09-12T03:02:04.287303+0000","last_peered":"2024-09-12T03:02:04.287303+0000","last_clean":"2024-09-12T03:02:04.287303+0000","last_became_active":"2024-09-12T03:00:58.347940+0000","last_became_peered":"2024-09-12T03:00:58.347940+0000","last_unstale":"2024-09-12T03:02:04.287303+0000","last_undegraded":"2024-09-12T03:02:04.287303+0000","last_fullsized":"2024-09-12T03:02:04.287303+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:43.489238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332599+0000","last_change":"2024-09-12T03:00:58.351969+0000","last_active":"2024-09-12T03:02:04.332599+0000","last_peered":"2024-09-12T03:02:04.332599+0000","last_clean":"2024-09-12T03:02:04.332599+0000","last_became_active":"2024-09-12T03:00:58.351679+0000","last_became_peered":"2024-09-12T03:00:58.351679+0000","last_unstale":"2024-09-12T03:02:04.332599+0000","last_undegraded":"2024-09-12T03:02:04.332599+0000","last_fullsized":"2024-09-12T03:02:04.332599+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:27:01.174890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332354+0000","last_change":"2024-09-12T03:00:58.346045+0000","last_active":"2024-09-12T03:02:04.332354+0000","last_peered":"2024-09-12T03:02:04.332354+0000","last_clean":"2024-09-12T03:02:04.332354+0000","last_became_active":"2024-09-12T03:00:58.345719+0000","last_became_peered":"2024-09-12T03:00:58.345719+0000","last_unstale":"2024-09-12T03:02:04.332354+0000","last_undegraded":"2024-09-12T03:02:04.332354+0000","last_fullsized":"2024-09-12T03:02:04.332354+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:12.312650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289352+0000","last_change":"2024-09-12T03:00:58.348165+0000","last_active":"2024-09-12T03:02:04.289352+0000","last_peered":"2024-09-12T03:02:04.289352+0000","last_clean":"2024-09-12T03:02:04.289352+0000","last_became_active":"2024-09-12T03:00:58.347734+0000","last_became_peered":"2024-09-12T03:00:58.347734+0000","last_unstale":"2024-09-12T03:02:04.289352+0000","last_undegraded":"2024-09-12T03:02:04.289352+0000","last_fullsized":"2024-09-12T03:02:04.289352+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:05.945900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.291897+0000","last_change":"2024-09-12T03:00:58.339316+0000","last_active":"2024-09-12T03:02:04.291897+0000","last_peered":"2024-09-12T03:02:04.291897+0000","last_clean":"2024-09-12T03:02:04.291897+0000","last_became_active":"2024-09-12T03:00:58.338964+0000","last_became_peered":"2024-09-12T03:00:58.338964+0000","last_unstale":"2024-09-12T03:02:04.291897+0000","last_undegraded":"2024-09-12T03:02:04.291897+0000","last_fullsized":"2024-09-12T03:02:04.291897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:50.721276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282483+0000","last_change":"2024-09-12T03:00:58.346838+0000","last_active":"2024-09-12T03:02:04.282483+0000","last_peered":"2024-09-12T03:02:04.282483+0000","last_clean":"2024-09-12T03:02:04.282483+0000","last_became_active":"2024-09-12T03:00:58.346486+0000","last_became_peered":"2024-09-12T03:00:58.346486+0000","last_unstale":"2024-09-12T03:02:04.282483+0000","last_undegraded":"2024-09-12T03:02:04.282483+0000","last_fullsized":"2024-09-12T03:02:04.282483+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:28.687727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284091+0000","last_change":"2024-09-12T03:00:58.346712+0000","last_active":"2024-09-12T03:02:04.284091+0000","last_peered":"2024-09-12T03:02:04.284091+0000","last_clean":"2024-09-12T03:02:04.284091+0000","last_became_active":"2024-09-12T03:00:58.346397+0000","last_became_peered":"2024-09-12T03:00:58.346397+0000","last_unstale":"2024-09-12T03:02:04.284091+0000","last_undegraded":"2024-09-12T03:02:04.284091+0000","last_fullsized":"2024-09-12T03:02:04.284091+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:48:00.624733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282190+0000","last_change":"2024-09-12T03:00:58.356166+0000","last_active":"2024-09-12T03:02:04.282190+0000","last_peered":"2024-09-12T03:02:04.282190+0000","last_clean":"2024-09-12T03:02:04.282190+0000","last_became_active":"2024-09-12T03:00:58.354944+0000","last_became_peered":"2024-09-12T03:00:58.354944+0000","last_unstale":"2024-09-12T03:02:04.282190+0000","last_undegraded":"2024-09-12T03:02:04.282190+0000","last_fullsized":"2024-09-12T03:02:04.282190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:15.840149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332814+0000","last_change":"2024-09-12T03:00:58.340182+0000","last_active":"2024-09-12T03:02:04.332814+0000","last_peered":"2024-09-12T03:02:04.332814+0000","last_clean":"2024-09-12T03:02:04.332814+0000","last_became_active":"2024-09-12T03:00:58.339576+0000","last_became_peered":"2024-09-12T03:00:58.339576+0000","last_unstale":"2024-09-12T03:02:04.332814+0000","last_undegraded":"2024-09-12T03:02:04.332814+0000","last_fullsized":"2024-09-12T03:02:04.332814+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:16.534844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282812+0000","last_change":"2024-09-12T03:00:58.351218+0000","last_active":"2024-09-12T03:02:04.282812+0000","last_peered":"2024-09-12T03:02:04.282812+0000","last_clean":"2024-09-12T03:02:04.282812+0000","last_became_active":"2024-09-12T03:00:58.350871+0000","last_became_peered":"2024-09-12T03:00:58.350871+0000","last_unstale":"2024-09-12T03:02:04.282812+0000","last_undegraded":"2024-09-12T03:02:04.282812+0000","last_fullsized":"2024-09-12T03:02:04.282812+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:38.367683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284465+0000","last_change":"2024-09-12T03:00:58.351894+0000","last_active":"2024-09-12T03:02:04.284465+0000","last_peered":"2024-09-12T03:02:04.284465+0000","last_clean":"2024-09-12T03:02:04.284465+0000","last_became_active":"2024-09-12T03:00:58.351445+0000","last_became_peered":"2024-09-12T03:00:58.351445+0000","last_unstale":"2024-09-12T03:02:04.284465+0000","last_undegraded":"2024-09-12T03:02:04.284465+0000","last_fullsized":"2024-09-12T03:02:04.284465+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:42.093871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289589+0000","last_change":"2024-09-12T03:00:58.353460+0000","last_active":"2024-09-12T03:02:04.289589+0000","last_peered":"2024-09-12T03:02:04.289589+0000","last_clean":"2024-09-12T03:02:04.289589+0000","last_became_active":"2024-09-12T03:00:58.353111+0000","last_became_peered":"2024-09-12T03:00:58.353111+0000","last_unstale":"2024-09-12T03:02:04.289589+0000","last_undegraded":"2024-09-12T03:02:04.289589+0000","last_fullsized":"2024-09-12T03:02:04.289589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:15.916765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332731+0000","last_change":"2024-09-12T03:00:58.341080+0000","last_active":"2024-09-12T03:02:04.332731+0000","last_peered":"2024-09-12T03:02:04.332731+0000","last_clean":"2024-09-12T03:02:04.332731+0000","last_became_active":"2024-09-12T03:00:58.340782+0000","last_became_peered":"2024-09-12T03:00:58.340782+0000","last_unstale":"2024-09-12T03:02:04.332731+0000","last_undegraded":"2024-09-12T03:02:04.332731+0000","last_fullsized":"2024-09-12T03:02:04.332731+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:36.569700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332843+0000","last_change":"2024-09-12T03:00:58.346791+0000","last_active":"2024-09-12T03:02:04.332843+0000","last_peered":"2024-09-12T03:02:04.332843+0000","last_clean":"2024-09-12T03:02:04.332843+0000","last_became_active":"2024-09-12T03:00:58.346500+0000","last_became_peered":"2024-09-12T03:00:58.346500+0000","last_unstale":"2024-09-12T03:02:04.332843+0000","last_undegraded":"2024-09-12T03:02:04.332843+0000","last_fullsized":"2024-09-12T03:02:04.332843+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:32.017589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332195+0000","last_change":"2024-09-12T03:00:58.349522+0000","last_active":"2024-09-12T03:02:04.332195+0000","last_peered":"2024-09-12T03:02:04.332195+0000","last_clean":"2024-09-12T03:02:04.332195+0000","last_became_active":"2024-09-12T03:00:58.349246+0000","last_became_peered":"2024-09-12T03:00:58.349246+0000","last_unstale":"2024-09-12T03:02:04.332195+0000","last_undegraded":"2024-09-12T03:02:04.332195+0000","last_fullsized":"2024-09-12T03:02:04.332195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:39.253919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332262+0000","last_change":"2024-09-12T03:00:58.349909+0000","last_active":"2024-09-12T03:02:04.332262+0000","last_peered":"2024-09-12T03:02:04.332262+0000","last_clean":"2024-09-12T03:02:04.332262+0000","last_became_active":"2024-09-12T03:00:58.349590+0000","last_became_peered":"2024-09-12T03:00:58.349590+0000","last_unstale":"2024-09-12T03:02:04.332262+0000","last_undegraded":"2024-09-12T03:02:04.332262+0000","last_fullsized":"2024-09-12T03:02:04.332262+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:04.540709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.287399+0000","last_change":"2024-09-12T03:00:58.348284+0000","last_active":"2024-09-12T03:02:04.287399+0000","last_peered":"2024-09-12T03:02:04.287399+0000","last_clean":"2024-09-12T03:02:04.287399+0000","last_became_active":"2024-09-12T03:00:58.347465+0000","last_became_peered":"2024-09-12T03:00:58.347465+0000","last_unstale":"2024-09-12T03:02:04.287399+0000","last_undegraded":"2024-09-12T03:02:04.287399+0000","last_fullsized":"2024-09-12T03:02:04.287399+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:13.191899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284483+0000","last_change":"2024-09-12T03:00:58.337740+0000","last_active":"2024-09-12T03:02:04.284483+0000","last_peered":"2024-09-12T03:02:04.284483+0000","last_clean":"2024-09-12T03:02:04.284483+0000","last_became_active":"2024-09-12T03:00:58.337393+0000","last_became_peered":"2024-09-12T03:00:58.337393+0000","last_unstale":"2024-09-12T03:02:04.284483+0000","last_undegraded":"2024-09-12T03:02:04.284483+0000","last_fullsized":"2024-09-12T03:02:04.284483+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:28:56.339167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.290193+0000","last_change":"2024-09-12T03:00:58.345779+0000","last_active":"2024-09-12T03:02:04.290193+0000","last_peered":"2024-09-12T03:02:04.290193+0000","last_clean":"2024-09-12T03:02:04.290193+0000","last_became_active":"2024-09-12T03:00:58.345210+0000","last_became_peered":"2024-09-12T03:00:58.345210+0000","last_unstale":"2024-09-12T03:02:04.290193+0000","last_undegraded":"2024-09-12T03:02:04.290193+0000","last_fullsized":"2024-09-12T03:02:04.290193+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:06:58.271185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.294381+0000","last_change":"2024-09-12T03:02:04.294381+0000","last_active":"2024-09-12T03:02:04.294381+0000","last_peered":"2024-09-12T03:02:04.294381+0000","last_clean":"2024-09-12T03:02:04.294381+0000","last_became_active":"2024-09-12T03:02:04.293795+0000","last_became_peered":"2024-09-12T03:02:04.293795+0000","last_unstale":"2024-09-12T03:02:04.294381+0000","last_undegraded":"2024-09-12T03:02:04.294381+0000","last_fullsized":"2024-09-12T03:02:04.294381+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:06:58.271007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332823+0000","last_change":"2024-09-12T03:00:58.349233+0000","last_active":"2024-09-12T03:02:04.332823+0000","last_peered":"2024-09-12T03:02:04.332823+0000","last_clean":"2024-09-12T03:02:04.332823+0000","last_became_active":"2024-09-12T03:00:58.348958+0000","last_became_peered":"2024-09-12T03:00:58.348958+0000","last_unstale":"2024-09-12T03:02:04.332823+0000","last_undegraded":"2024-09-12T03:02:04.332823+0000","last_fullsized":"2024-09-12T03:02:04.332823+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:32:10.271817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332400+0000","last_change":"2024-09-12T03:00:58.358024+0000","last_active":"2024-09-12T03:02:04.332400+0000","last_peered":"2024-09-12T03:02:04.332400+0000","last_clean":"2024-09-12T03:02:04.332400+0000","last_became_active":"2024-09-12T03:00:58.357434+0000","last_became_peered":"2024-09-12T03:00:58.357434+0000","last_unstale":"2024-09-12T03:02:04.332400+0000","last_undegraded":"2024-09-12T03:02:04.332400+0000","last_fullsized":"2024-09-12T03:02:04.332400+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:01:20.119733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282240+0000","last_change":"2024-09-12T03:00:58.355063+0000","last_active":"2024-09-12T03:02:04.282240+0000","last_peered":"2024-09-12T03:02:04.282240+0000","last_clean":"2024-09-12T03:02:04.282240+0000","last_became_active":"2024-09-12T03:00:58.354760+0000","last_became_peered":"2024-09-12T03:00:58.354760+0000","last_unstale":"2024-09-12T03:02:04.282240+0000","last_undegraded":"2024-09-12T03:02:04.282240+0000","last_fullsized":"2024-09-12T03:02:04.282240+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:13.464413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.288000+0000","last_change":"2024-09-12T03:00:58.341916+0000","last_active":"2024-09-12T03:02:04.288000+0000","last_peered":"2024-09-12T03:02:04.288000+0000","last_clean":"2024-09-12T03:02:04.288000+0000","last_became_active":"2024-09-12T03:00:58.340815+0000","last_became_peered":"2024-09-12T03:00:58.340815+0000","last_unstale":"2024-09-12T03:02:04.288000+0000","last_undegraded":"2024-09-12T03:02:04.288000+0000","last_fullsized":"2024-09-12T03:02:04.288000+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:13.777947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.283555+0000","last_change":"2024-09-12T03:00:58.357765+0000","last_active":"2024-09-12T03:02:04.283555+0000","last_peered":"2024-09-12T03:02:04.283555+0000","last_clean":"2024-09-12T03:02:04.283555+0000","last_became_active":"2024-09-12T03:00:58.357480+0000","last_became_peered":"2024-09-12T03:00:58.357480+0000","last_unstale":"2024-09-12T03:02:04.283555+0000","last_undegraded":"2024-09-12T03:02:04.283555+0000","last_fullsized":"2024-09-12T03:02:04.283555+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:26.884381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.290553+0000","last_change":"2024-09-12T03:00:58.349504+0000","last_active":"2024-09-12T03:02:04.290553+0000","last_peered":"2024-09-12T03:02:04.290553+0000","last_clean":"2024-09-12T03:02:04.290553+0000","last_became_active":"2024-09-12T03:00:58.349136+0000","last_became_peered":"2024-09-12T03:00:58.349136+0000","last_unstale":"2024-09-12T03:02:04.290553+0000","last_undegraded":"2024-09-12T03:02:04.290553+0000","last_fullsized":"2024-09-12T03:02:04.290553+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:09:55.229516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.286399+0000","last_change":"2024-09-12T03:00:58.341902+0000","last_active":"2024-09-12T03:02:04.286399+0000","last_peered":"2024-09-12T03:02:04.286399+0000","last_clean":"2024-09-12T03:02:04.286399+0000","last_became_active":"2024-09-12T03:00:58.341592+0000","last_became_peered":"2024-09-12T03:00:58.341592+0000","last_unstale":"2024-09-12T03:02:04.286399+0000","last_undegraded":"2024-09-12T03:02:04.286399+0000","last_fullsized":"2024-09-12T03:02:04.286399+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:13.852511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332298+0000","last_change":"2024-09-12T03:00:58.348012+0000","last_active":"2024-09-12T03:02:04.332298+0000","last_peered":"2024-09-12T03:02:04.332298+0000","last_clean":"2024-09-12T03:02:04.332298+0000","last_became_active":"2024-09-12T03:00:58.347338+0000","last_became_peered":"2024-09-12T03:00:58.347338+0000","last_unstale":"2024-09-12T03:02:04.332298+0000","last_undegraded":"2024-09-12T03:02:04.332298+0000","last_fullsized":"2024-09-12T03:02:04.332298+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:41.478032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332497+0000","last_change":"2024-09-12T03:00:58.345833+0000","last_active":"2024-09-12T03:02:04.332497+0000","last_peered":"2024-09-12T03:02:04.332497+0000","last_clean":"2024-09-12T03:02:04.332497+0000","last_became_active":"2024-09-12T03:00:58.345482+0000","last_became_peered":"2024-09-12T03:00:58.345482+0000","last_unstale":"2024-09-12T03:02:04.332497+0000","last_undegraded":"2024-09-12T03:02:04.332497+0000","last_fullsized":"2024-09-12T03:02:04.332497+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:32.037412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289995+0000","last_change":"2024-09-12T03:00:58.352290+0000","last_active":"2024-09-12T03:02:04.289995+0000","last_peered":"2024-09-12T03:02:04.289995+0000","last_clean":"2024-09-12T03:02:04.289995+0000","last_became_active":"2024-09-12T03:00:58.351918+0000","last_became_peered":"2024-09-12T03:00:58.351918+0000","last_unstale":"2024-09-12T03:02:04.289995+0000","last_undegraded":"2024-09-12T03:02:04.289995+0000","last_fullsized":"2024-09-12T03:02:04.289995+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:25.941080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.290160+0000","last_change":"2024-09-12T03:00:58.348021+0000","last_active":"2024-09-12T03:02:04.290160+0000","last_peered":"2024-09-12T03:02:04.290160+0000","last_clean":"2024-09-12T03:02:04.290160+0000","last_became_active":"2024-09-12T03:00:58.347634+0000","last_became_peered":"2024-09-12T03:00:58.347634+0000","last_unstale":"2024-09-12T03:02:04.290160+0000","last_undegraded":"2024-09-12T03:02:04.290160+0000","last_fullsized":"2024-09-12T03:02:04.290160+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:47.933067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.285993+0000","last_change":"2024-09-12T03:00:58.340891+0000","last_active":"2024-09-12T03:02:04.285993+0000","last_peered":"2024-09-12T03:02:04.285993+0000","last_clean":"2024-09-12T03:02:04.285993+0000","last_became_active":"2024-09-12T03:00:58.340504+0000","last_became_peered":"2024-09-12T03:00:58.340504+0000","last_unstale":"2024-09-12T03:02:04.285993+0000","last_undegraded":"2024-09-12T03:02:04.285993+0000","last_fullsized":"2024-09-12T03:02:04.285993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:32.117635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.291866+0000","last_change":"2024-09-12T03:00:58.339343+0000","last_active":"2024-09-12T03:02:04.291866+0000","last_peered":"2024-09-12T03:02:04.291866+0000","last_clean":"2024-09-12T03:02:04.291866+0000","last_became_active":"2024-09-12T03:00:58.339001+0000","last_became_peered":"2024-09-12T03:00:58.339001+0000","last_unstale":"2024-09-12T03:02:04.291866+0000","last_undegraded":"2024-09-12T03:02:04.291866+0000","last_fullsized":"2024-09-12T03:02:04.291866+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:02.043454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":95,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:12.496113+0000","last_change":"2024-09-12T03:00:56.315550+0000","last_active":"2024-09-12T03:02:12.496113+0000","last_peered":"2024-09-12T03:02:12.496113+0000","last_clean":"2024-09-12T03:02:12.496113+0000","last_became_active":"2024-09-12T03:00:56.315225+0000","last_became_peered":"2024-09-12T03:00:56.315225+0000","last_unstale":"2024-09-12T03:02:12.496113+0000","last_undegraded":"2024-09-12T03:02:12.496113+0000","last_fullsized":"2024-09-12T03:02:12.496113+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:46.070895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.290858+0000","last_change":"2024-09-12T03:02:04.290498+0000","last_active":"2024-09-12T03:02:04.290858+0000","last_peered":"2024-09-12T03:02:04.290858+0000","last_clean":"2024-09-12T03:02:04.290858+0000","last_became_active":"2024-09-12T03:02:04.290162+0000","last_became_peered":"2024-09-12T03:02:04.290162+0000","last_unstale":"2024-09-12T03:02:04.290858+0000","last_undegraded":"2024-09-12T03:02:04.290858+0000","last_fullsized":"2024-09-12T03:02:04.290858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:29.777155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282566+0000","last_change":"2024-09-12T03:00:56.316385+0000","last_active":"2024-09-12T03:02:04.282566+0000","last_peered":"2024-09-12T03:02:04.282566+0000","last_clean":"2024-09-12T03:02:04.282566+0000","last_became_active":"2024-09-12T03:00:56.316064+0000","last_became_peered":"2024-09-12T03:00:56.316064+0000","last_unstale":"2024-09-12T03:02:04.282566+0000","last_undegraded":"2024-09-12T03:02:04.282566+0000","last_fullsized":"2024-09-12T03:02:04.282566+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:12:43.323654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332424+0000","last_change":"2024-09-12T03:00:56.313065+0000","last_active":"2024-09-12T03:02:04.332424+0000","last_peered":"2024-09-12T03:02:04.332424+0000","last_clean":"2024-09-12T03:02:04.332424+0000","last_became_active":"2024-09-12T03:00:56.312716+0000","last_became_peered":"2024-09-12T03:00:56.312716+0000","last_unstale":"2024-09-12T03:02:04.332424+0000","last_undegraded":"2024-09-12T03:02:04.332424+0000","last_fullsized":"2024-09-12T03:02:04.332424+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:51.089089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282199+0000","last_change":"2024-09-12T03:00:58.346542+0000","last_active":"2024-09-12T03:02:04.282199+0000","last_peered":"2024-09-12T03:02:04.282199+0000","last_clean":"2024-09-12T03:02:04.282199+0000","last_became_active":"2024-09-12T03:00:58.346260+0000","last_became_peered":"2024-09-12T03:00:58.346260+0000","last_unstale":"2024-09-12T03:02:04.282199+0000","last_undegraded":"2024-09-12T03:02:04.282199+0000","last_fullsized":"2024-09-12T03:02:04.282199+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:37.034969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289182+0000","last_change":"2024-09-12T03:00:56.317308+0000","last_active":"2024-09-12T03:02:04.289182+0000","last_peered":"2024-09-12T03:02:04.289182+0000","last_clean":"2024-09-12T03:02:04.289182+0000","last_became_active":"2024-09-12T03:00:56.316935+0000","last_became_peered":"2024-09-12T03:00:56.316935+0000","last_unstale":"2024-09-12T03:02:04.289182+0000","last_undegraded":"2024-09-12T03:02:04.289182+0000","last_fullsized":"2024-09-12T03:02:04.289182+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:06.324489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.283990+0000","last_change":"2024-09-12T03:00:58.341565+0000","last_active":"2024-09-12T03:02:04.283990+0000","last_peered":"2024-09-12T03:02:04.283990+0000","last_clean":"2024-09-12T03:02:04.283990+0000","last_became_active":"2024-09-12T03:00:58.341285+0000","last_became_peered":"2024-09-12T03:00:58.341285+0000","last_unstale":"2024-09-12T03:02:04.283990+0000","last_undegraded":"2024-09-12T03:02:04.283990+0000","last_fullsized":"2024-09-12T03:02:04.283990+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:59.156700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":88,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:09.490675+0000","last_change":"2024-09-12T03:00:56.314201+0000","last_active":"2024-09-12T03:02:09.490675+0000","last_peered":"2024-09-12T03:02:09.490675+0000","last_clean":"2024-09-12T03:02:09.490675+0000","last_became_active":"2024-09-12T03:00:56.313880+0000","last_became_peered":"2024-09-12T03:00:56.313880+0000","last_unstale":"2024-09-12T03:02:09.490675+0000","last_undegraded":"2024-09-12T03:02:09.490675+0000","last_fullsized":"2024-09-12T03:02:09.490675+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:57:12.952630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":63,"num_read_kb":63,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332657+0000","last_change":"2024-09-12T03:00:58.345274+0000","last_active":"2024-09-12T03:02:04.332657+0000","last_peered":"2024-09-12T03:02:04.332657+0000","last_clean":"2024-09-12T03:02:04.332657+0000","last_became_active":"2024-09-12T03:00:58.344937+0000","last_became_peered":"2024-09-12T03:00:58.344937+0000","last_unstale":"2024-09-12T03:02:04.332657+0000","last_undegraded":"2024-09-12T03:02:04.332657+0000","last_fullsized":"2024-09-12T03:02:04.332657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:17.288909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284614+0000","last_change":"2024-09-12T03:00:56.314709+0000","last_active":"2024-09-12T03:02:04.284614+0000","last_peered":"2024-09-12T03:02:04.284614+0000","last_clean":"2024-09-12T03:02:04.284614+0000","last_became_active":"2024-09-12T03:00:56.314349+0000","last_became_peered":"2024-09-12T03:00:56.314349+0000","last_unstale":"2024-09-12T03:02:04.284614+0000","last_undegraded":"2024-09-12T03:02:04.284614+0000","last_fullsized":"2024-09-12T03:02:04.284614+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:18:35.371661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332796+0000","last_change":"2024-09-12T03:00:58.346055+0000","last_active":"2024-09-12T03:02:04.332796+0000","last_peered":"2024-09-12T03:02:04.332796+0000","last_clean":"2024-09-12T03:02:04.332796+0000","last_became_active":"2024-09-12T03:00:58.345701+0000","last_became_peered":"2024-09-12T03:00:58.345701+0000","last_unstale":"2024-09-12T03:02:04.332796+0000","last_undegraded":"2024-09-12T03:02:04.332796+0000","last_fullsized":"2024-09-12T03:02:04.332796+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:33.977245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.287450+0000","last_change":"2024-09-12T03:00:56.435962+0000","last_active":"2024-09-12T03:02:04.287450+0000","last_peered":"2024-09-12T03:02:04.287450+0000","last_clean":"2024-09-12T03:02:04.287450+0000","last_became_active":"2024-09-12T03:00:56.435515+0000","last_became_peered":"2024-09-12T03:00:56.435515+0000","last_unstale":"2024-09-12T03:02:04.287450+0000","last_undegraded":"2024-09-12T03:02:04.287450+0000","last_fullsized":"2024-09-12T03:02:04.287450+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:32.953436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332423+0000","last_change":"2024-09-12T03:00:58.344239+0000","last_active":"2024-09-12T03:02:04.332423+0000","last_peered":"2024-09-12T03:02:04.332423+0000","last_clean":"2024-09-12T03:02:04.332423+0000","last_became_active":"2024-09-12T03:00:58.343876+0000","last_became_peered":"2024-09-12T03:00:58.343876+0000","last_unstale":"2024-09-12T03:02:04.332423+0000","last_undegraded":"2024-09-12T03:02:04.332423+0000","last_fullsized":"2024-09-12T03:02:04.332423+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:49.401295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'29","reported_seq":186,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:10.492320+0000","last_change":"2024-09-12T03:00:56.317708+0000","last_active":"2024-09-12T03:02:10.492320+0000","last_peered":"2024-09-12T03:02:10.492320+0000","last_clean":"2024-09-12T03:02:10.492320+0000","last_became_active":"2024-09-12T03:00:56.317317+0000","last_became_peered":"2024-09-12T03:00:56.317317+0000","last_unstale":"2024-09-12T03:02:10.492320+0000","last_undegraded":"2024-09-12T03:02:10.492320+0000","last_fullsized":"2024-09-12T03:02:10.492320+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:47.788186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61752,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":135,"num_read_kb":127,"num_write":30,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292083+0000","last_change":"2024-09-12T03:00:58.343602+0000","last_active":"2024-09-12T03:02:04.292083+0000","last_peered":"2024-09-12T03:02:04.292083+0000","last_clean":"2024-09-12T03:02:04.292083+0000","last_became_active":"2024-09-12T03:00:58.343301+0000","last_became_peered":"2024-09-12T03:00:58.343301+0000","last_unstale":"2024-09-12T03:02:04.292083+0000","last_undegraded":"2024-09-12T03:02:04.292083+0000","last_fullsized":"2024-09-12T03:02:04.292083+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:04.056118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.290204+0000","last_change":"2024-09-12T03:00:56.317874+0000","last_active":"2024-09-12T03:02:04.290204+0000","last_peered":"2024-09-12T03:02:04.290204+0000","last_clean":"2024-09-12T03:02:04.290204+0000","last_became_active":"2024-09-12T03:00:56.317566+0000","last_became_peered":"2024-09-12T03:00:56.317566+0000","last_unstale":"2024-09-12T03:02:04.290204+0000","last_undegraded":"2024-09-12T03:02:04.290204+0000","last_fullsized":"2024-09-12T03:02:04.290204+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:54.203730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332601+0000","last_change":"2024-09-12T03:00:58.351539+0000","last_active":"2024-09-12T03:02:04.332601+0000","last_peered":"2024-09-12T03:02:04.332601+0000","last_clean":"2024-09-12T03:02:04.332601+0000","last_became_active":"2024-09-12T03:00:58.351249+0000","last_became_peered":"2024-09-12T03:00:58.351249+0000","last_unstale":"2024-09-12T03:02:04.332601+0000","last_undegraded":"2024-09-12T03:02:04.332601+0000","last_fullsized":"2024-09-12T03:02:04.332601+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:40.498867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.287068+0000","last_change":"2024-09-12T03:00:56.314489+0000","last_active":"2024-09-12T03:02:04.287068+0000","last_peered":"2024-09-12T03:02:04.287068+0000","last_clean":"2024-09-12T03:02:04.287068+0000","last_became_active":"2024-09-12T03:00:56.314142+0000","last_became_peered":"2024-09-12T03:00:56.314142+0000","last_unstale":"2024-09-12T03:02:04.287068+0000","last_undegraded":"2024-09-12T03:02:04.287068+0000","last_fullsized":"2024-09-12T03:02:04.287068+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:21.541713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284409+0000","last_change":"2024-09-12T03:00:58.348752+0000","last_active":"2024-09-12T03:02:04.284409+0000","last_peered":"2024-09-12T03:02:04.284409+0000","last_clean":"2024-09-12T03:02:04.284409+0000","last_became_active":"2024-09-12T03:00:58.348391+0000","last_became_peered":"2024-09-12T03:00:58.348391+0000","last_unstale":"2024-09-12T03:02:04.284409+0000","last_undegraded":"2024-09-12T03:02:04.284409+0000","last_fullsized":"2024-09-12T03:02:04.284409+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:33:15.626306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'3","reported_seq":94,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:13.496601+0000","last_change":"2024-09-12T03:00:56.311702+0000","last_active":"2024-09-12T03:02:13.496601+0000","last_peered":"2024-09-12T03:02:13.496601+0000","last_clean":"2024-09-12T03:02:13.496601+0000","last_became_active":"2024-09-12T03:00:56.311353+0000","last_became_peered":"2024-09-12T03:00:56.311353+0000","last_unstale":"2024-09-12T03:02:13.496601+0000","last_undegraded":"2024-09-12T03:02:13.496601+0000","last_fullsized":"2024-09-12T03:02:13.496601+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:01.004858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284051+0000","last_change":"2024-09-12T03:00:58.351973+0000","last_active":"2024-09-12T03:02:04.284051+0000","last_peered":"2024-09-12T03:02:04.284051+0000","last_clean":"2024-09-12T03:02:04.284051+0000","last_became_active":"2024-09-12T03:00:58.351599+0000","last_became_peered":"2024-09-12T03:00:58.351599+0000","last_unstale":"2024-09-12T03:02:04.284051+0000","last_undegraded":"2024-09-12T03:02:04.284051+0000","last_fullsized":"2024-09-12T03:02:04.284051+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:26.398828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.291749+0000","last_change":"2024-09-12T03:00:56.436185+0000","last_active":"2024-09-12T03:02:04.291749+0000","last_peered":"2024-09-12T03:02:04.291749+0000","last_clean":"2024-09-12T03:02:04.291749+0000","last_became_active":"2024-09-12T03:00:56.435717+0000","last_became_peered":"2024-09-12T03:00:56.435717+0000","last_unstale":"2024-09-12T03:02:04.291749+0000","last_undegraded":"2024-09-12T03:02:04.291749+0000","last_fullsized":"2024-09-12T03:02:04.291749+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:27.303182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.333661+0000","last_change":"2024-09-12T03:02:04.333661+0000","last_active":"2024-09-12T03:02:04.333661+0000","last_peered":"2024-09-12T03:02:04.333661+0000","last_clean":"2024-09-12T03:02:04.333661+0000","last_became_active":"2024-09-12T03:02:04.333217+0000","last_became_peered":"2024-09-12T03:02:04.333217+0000","last_unstale":"2024-09-12T03:02:04.333661+0000","last_undegraded":"2024-09-12T03:02:04.333661+0000","last_fullsized":"2024-09-12T03:02:04.333661+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:23:47.212734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.297341+0000","last_change":"2024-09-12T03:02:04.296884+0000","last_active":"2024-09-12T03:02:04.297341+0000","last_peered":"2024-09-12T03:02:04.297341+0000","last_clean":"2024-09-12T03:02:04.297341+0000","last_became_active":"2024-09-12T03:02:04.290894+0000","last_became_peered":"2024-09-12T03:02:04.290894+0000","last_unstale":"2024-09-12T03:02:04.297341+0000","last_undegraded":"2024-09-12T03:02:04.297341+0000","last_fullsized":"2024-09-12T03:02:04.297341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:49:19.749568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.334096+0000","last_change":"2024-09-12T03:02:04.334096+0000","last_active":"2024-09-12T03:02:04.334096+0000","last_peered":"2024-09-12T03:02:04.334096+0000","last_clean":"2024-09-12T03:02:04.334096+0000","last_became_active":"2024-09-12T03:02:04.333641+0000","last_became_peered":"2024-09-12T03:02:04.333641+0000","last_unstale":"2024-09-12T03:02:04.334096+0000","last_undegraded":"2024-09-12T03:02:04.334096+0000","last_fullsized":"2024-09-12T03:02:04.334096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:24.085293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332966+0000","last_change":"2024-09-12T03:00:56.314182+0000","last_active":"2024-09-12T03:02:04.332966+0000","last_peered":"2024-09-12T03:02:04.332966+0000","last_clean":"2024-09-12T03:02:04.332966+0000","last_became_active":"2024-09-12T03:00:56.313749+0000","last_became_peered":"2024-09-12T03:00:56.313749+0000","last_unstale":"2024-09-12T03:02:04.332966+0000","last_undegraded":"2024-09-12T03:02:04.332966+0000","last_fullsized":"2024-09-12T03:02:04.332966+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:31:45.767500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284764+0000","last_change":"2024-09-12T03:00:58.344356+0000","last_active":"2024-09-12T03:02:04.284764+0000","last_peered":"2024-09-12T03:02:04.284764+0000","last_clean":"2024-09-12T03:02:04.284764+0000","last_became_active":"2024-09-12T03:00:58.344035+0000","last_became_peered":"2024-09-12T03:00:58.344035+0000","last_unstale":"2024-09-12T03:02:04.284764+0000","last_undegraded":"2024-09-12T03:02:04.284764+0000","last_fullsized":"2024-09-12T03:02:04.284764+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:45:18.681018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.287585+0000","last_change":"2024-09-12T03:00:56.315653+0000","last_active":"2024-09-12T03:02:04.287585+0000","last_peered":"2024-09-12T03:02:04.287585+0000","last_clean":"2024-09-12T03:02:04.287585+0000","last_became_active":"2024-09-12T03:00:56.315284+0000","last_became_peered":"2024-09-12T03:00:56.315284+0000","last_unstale":"2024-09-12T03:02:04.287585+0000","last_undegraded":"2024-09-12T03:02:04.287585+0000","last_fullsized":"2024-09-12T03:02:04.287585+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:42.934020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.291128+0000","last_change":"2024-09-12T03:02:04.291128+0000","last_active":"2024-09-12T03:02:04.291128+0000","last_peered":"2024-09-12T03:02:04.291128+0000","last_clean":"2024-09-12T03:02:04.291128+0000","last_became_active":"2024-09-12T03:02:04.287836+0000","last_became_peered":"2024-09-12T03:02:04.287836+0000","last_unstale":"2024-09-12T03:02:04.291128+0000","last_undegraded":"2024-09-12T03:02:04.291128+0000","last_fullsized":"2024-09-12T03:02:04.291128+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:23.956579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.290577+0000","last_change":"2024-09-12T03:00:56.746780+0000","last_active":"2024-09-12T03:02:04.290577+0000","last_peered":"2024-09-12T03:02:04.290577+0000","last_clean":"2024-09-12T03:02:04.290577+0000","last_became_active":"2024-09-12T03:00:56.746414+0000","last_became_peered":"2024-09-12T03:00:56.746414+0000","last_unstale":"2024-09-12T03:02:04.290577+0000","last_undegraded":"2024-09-12T03:02:04.290577+0000","last_fullsized":"2024-09-12T03:02:04.290577+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:15:35.214196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284593+0000","last_change":"2024-09-12T03:00:58.351713+0000","last_active":"2024-09-12T03:02:04.284593+0000","last_peered":"2024-09-12T03:02:04.284593+0000","last_clean":"2024-09-12T03:02:04.284593+0000","last_became_active":"2024-09-12T03:00:58.351371+0000","last_became_peered":"2024-09-12T03:00:58.351371+0000","last_unstale":"2024-09-12T03:02:04.284593+0000","last_undegraded":"2024-09-12T03:02:04.284593+0000","last_fullsized":"2024-09-12T03:02:04.284593+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:13.346956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'137","reported_seq":335,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284267+0000","last_change":"2024-09-12T02:59:45.461482+0000","last_active":"2024-09-12T03:02:04.284267+0000","last_peered":"2024-09-12T03:02:04.284267+0000","last_clean":"2024-09-12T03:02:04.284267+0000","last_became_active":"2024-09-12T02:59:45.461058+0000","last_became_peered":"2024-09-12T02:59:45.461058+0000","last_unstale":"2024-09-12T03:02:04.284267+0000","last_undegraded":"2024-09-12T03:02:04.284267+0000","last_fullsized":"2024-09-12T03:02:04.284267+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:18.034438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:18.034438+0000","last_clean_scrub_stamp":"2024-09-12T02:56:18.034438+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:09.251858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289291+0000","last_change":"2024-09-12T03:00:56.315245+0000","last_active":"2024-09-12T03:02:04.289291+0000","last_peered":"2024-09-12T03:02:04.289291+0000","last_clean":"2024-09-12T03:02:04.289291+0000","last_became_active":"2024-09-12T03:00:56.314887+0000","last_became_peered":"2024-09-12T03:00:56.314887+0000","last_unstale":"2024-09-12T03:02:04.289291+0000","last_undegraded":"2024-09-12T03:02:04.289291+0000","last_fullsized":"2024-09-12T03:02:04.289291+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:48.750657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.283183+0000","last_change":"2024-09-12T03:00:58.357874+0000","last_active":"2024-09-12T03:02:04.283183+0000","last_peered":"2024-09-12T03:02:04.283183+0000","last_clean":"2024-09-12T03:02:04.283183+0000","last_became_active":"2024-09-12T03:00:58.357575+0000","last_became_peered":"2024-09-12T03:00:58.357575+0000","last_unstale":"2024-09-12T03:02:04.283183+0000","last_undegraded":"2024-09-12T03:02:04.283183+0000","last_fullsized":"2024-09-12T03:02:04.283183+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:46.125620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284212+0000","last_change":"2024-09-12T03:00:56.312263+0000","last_active":"2024-09-12T03:02:04.284212+0000","last_peered":"2024-09-12T03:02:04.284212+0000","last_clean":"2024-09-12T03:02:04.284212+0000","last_became_active":"2024-09-12T03:00:56.311894+0000","last_became_peered":"2024-09-12T03:00:56.311894+0000","last_unstale":"2024-09-12T03:02:04.284212+0000","last_undegraded":"2024-09-12T03:02:04.284212+0000","last_fullsized":"2024-09-12T03:02:04.284212+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:52.434203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332322+0000","last_change":"2024-09-12T03:00:58.350762+0000","last_active":"2024-09-12T03:02:04.332322+0000","last_peered":"2024-09-12T03:02:04.332322+0000","last_clean":"2024-09-12T03:02:04.332322+0000","last_became_active":"2024-09-12T03:00:58.348038+0000","last_became_peered":"2024-09-12T03:00:58.348038+0000","last_unstale":"2024-09-12T03:02:04.332322+0000","last_undegraded":"2024-09-12T03:02:04.332322+0000","last_fullsized":"2024-09-12T03:02:04.332322+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:08.916636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":89,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:09.489783+0000","last_change":"2024-09-12T03:00:56.435641+0000","last_active":"2024-09-12T03:02:09.489783+0000","last_peered":"2024-09-12T03:02:09.489783+0000","last_clean":"2024-09-12T03:02:09.489783+0000","last_became_active":"2024-09-12T03:00:56.435319+0000","last_became_peered":"2024-09-12T03:00:56.435319+0000","last_unstale":"2024-09-12T03:02:09.489783+0000","last_undegraded":"2024-09-12T03:02:09.489783+0000","last_fullsized":"2024-09-12T03:02:09.489783+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:56.894978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":63,"num_read_kb":63,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.288114+0000","last_change":"2024-09-12T03:00:58.344708+0000","last_active":"2024-09-12T03:02:04.288114+0000","last_peered":"2024-09-12T03:02:04.288114+0000","last_clean":"2024-09-12T03:02:04.288114+0000","last_became_active":"2024-09-12T03:00:58.344389+0000","last_became_peered":"2024-09-12T03:00:58.344389+0000","last_unstale":"2024-09-12T03:02:04.288114+0000","last_undegraded":"2024-09-12T03:02:04.288114+0000","last_fullsized":"2024-09-12T03:02:04.288114+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:08.160383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284317+0000","last_change":"2024-09-12T03:00:56.317068+0000","last_active":"2024-09-12T03:02:04.284317+0000","last_peered":"2024-09-12T03:02:04.284317+0000","last_clean":"2024-09-12T03:02:04.284317+0000","last_became_active":"2024-09-12T03:00:56.316698+0000","last_became_peered":"2024-09-12T03:00:56.316698+0000","last_unstale":"2024-09-12T03:02:04.284317+0000","last_undegraded":"2024-09-12T03:02:04.284317+0000","last_fullsized":"2024-09-12T03:02:04.284317+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:41:19.199418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282614+0000","last_change":"2024-09-12T03:00:58.347814+0000","last_active":"2024-09-12T03:02:04.282614+0000","last_peered":"2024-09-12T03:02:04.282614+0000","last_clean":"2024-09-12T03:02:04.282614+0000","last_became_active":"2024-09-12T03:00:58.347112+0000","last_became_peered":"2024-09-12T03:00:58.347112+0000","last_unstale":"2024-09-12T03:02:04.282614+0000","last_undegraded":"2024-09-12T03:02:04.282614+0000","last_fullsized":"2024-09-12T03:02:04.282614+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:59.325113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284265+0000","last_change":"2024-09-12T03:00:56.435771+0000","last_active":"2024-09-12T03:02:04.284265+0000","last_peered":"2024-09-12T03:02:04.284265+0000","last_clean":"2024-09-12T03:02:04.284265+0000","last_became_active":"2024-09-12T03:00:56.435448+0000","last_became_peered":"2024-09-12T03:00:56.435448+0000","last_unstale":"2024-09-12T03:02:04.284265+0000","last_undegraded":"2024-09-12T03:02:04.284265+0000","last_fullsized":"2024-09-12T03:02:04.284265+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:33.650338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282832+0000","last_change":"2024-09-12T03:00:58.341071+0000","last_active":"2024-09-12T03:02:04.282832+0000","last_peered":"2024-09-12T03:02:04.282832+0000","last_clean":"2024-09-12T03:02:04.282832+0000","last_became_active":"2024-09-12T03:00:58.340753+0000","last_became_peered":"2024-09-12T03:00:58.340753+0000","last_unstale":"2024-09-12T03:02:04.282832+0000","last_undegraded":"2024-09-12T03:02:04.282832+0000","last_fullsized":"2024-09-12T03:02:04.282832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:21.217915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292254+0000","last_change":"2024-09-12T03:00:56.436058+0000","last_active":"2024-09-12T03:02:04.292254+0000","last_peered":"2024-09-12T03:02:04.292254+0000","last_clean":"2024-09-12T03:02:04.292254+0000","last_became_active":"2024-09-12T03:00:56.435518+0000","last_became_peered":"2024-09-12T03:00:56.435518+0000","last_unstale":"2024-09-12T03:02:04.292254+0000","last_undegraded":"2024-09-12T03:02:04.292254+0000","last_fullsized":"2024-09-12T03:02:04.292254+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:32.235097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332392+0000","last_change":"2024-09-12T03:00:58.348236+0000","last_active":"2024-09-12T03:02:04.332392+0000","last_peered":"2024-09-12T03:02:04.332392+0000","last_clean":"2024-09-12T03:02:04.332392+0000","last_became_active":"2024-09-12T03:00:58.347727+0000","last_became_peered":"2024-09-12T03:00:58.347727+0000","last_unstale":"2024-09-12T03:02:04.332392+0000","last_undegraded":"2024-09-12T03:02:04.332392+0000","last_fullsized":"2024-09-12T03:02:04.332392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:19.233163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284575+0000","last_change":"2024-09-12T03:00:56.310537+0000","last_active":"2024-09-12T03:02:04.284575+0000","last_peered":"2024-09-12T03:02:04.284575+0000","last_clean":"2024-09-12T03:02:04.284575+0000","last_became_active":"2024-09-12T03:00:56.310135+0000","last_became_peered":"2024-09-12T03:00:56.310135+0000","last_unstale":"2024-09-12T03:02:04.284575+0000","last_undegraded":"2024-09-12T03:02:04.284575+0000","last_fullsized":"2024-09-12T03:02:04.284575+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:10:55.584339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.283645+0000","last_change":"2024-09-12T03:00:58.358417+0000","last_active":"2024-09-12T03:02:04.283645+0000","last_peered":"2024-09-12T03:02:04.283645+0000","last_clean":"2024-09-12T03:02:04.283645+0000","last_became_active":"2024-09-12T03:00:58.358119+0000","last_became_peered":"2024-09-12T03:00:58.358119+0000","last_unstale":"2024-09-12T03:02:04.283645+0000","last_undegraded":"2024-09-12T03:02:04.283645+0000","last_fullsized":"2024-09-12T03:02:04.283645+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:10.897594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289526+0000","last_change":"2024-09-12T03:00:56.747044+0000","last_active":"2024-09-12T03:02:04.289526+0000","last_peered":"2024-09-12T03:02:04.289526+0000","last_clean":"2024-09-12T03:02:04.289526+0000","last_became_active":"2024-09-12T03:00:56.746699+0000","last_became_peered":"2024-09-12T03:00:56.746699+0000","last_unstale":"2024-09-12T03:02:04.289526+0000","last_undegraded":"2024-09-12T03:02:04.289526+0000","last_fullsized":"2024-09-12T03:02:04.289526+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:11.420109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332683+0000","last_change":"2024-09-12T03:00:58.346854+0000","last_active":"2024-09-12T03:02:04.332683+0000","last_peered":"2024-09-12T03:02:04.332683+0000","last_clean":"2024-09-12T03:02:04.332683+0000","last_became_active":"2024-09-12T03:00:58.346535+0000","last_became_peered":"2024-09-12T03:00:58.346535+0000","last_unstale":"2024-09-12T03:02:04.332683+0000","last_undegraded":"2024-09-12T03:02:04.332683+0000","last_fullsized":"2024-09-12T03:02:04.332683+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:46.637914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332365+0000","last_change":"2024-09-12T03:00:56.435957+0000","last_active":"2024-09-12T03:02:04.332365+0000","last_peered":"2024-09-12T03:02:04.332365+0000","last_clean":"2024-09-12T03:02:04.332365+0000","last_became_active":"2024-09-12T03:00:56.435603+0000","last_became_peered":"2024-09-12T03:00:56.435603+0000","last_unstale":"2024-09-12T03:02:04.332365+0000","last_undegraded":"2024-09-12T03:02:04.332365+0000","last_fullsized":"2024-09-12T03:02:04.332365+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:34.553189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.291897+0000","last_change":"2024-09-12T03:00:58.337784+0000","last_active":"2024-09-12T03:02:04.291897+0000","last_peered":"2024-09-12T03:02:04.291897+0000","last_clean":"2024-09-12T03:02:04.291897+0000","last_became_active":"2024-09-12T03:00:58.337415+0000","last_became_peered":"2024-09-12T03:00:58.337415+0000","last_unstale":"2024-09-12T03:02:04.291897+0000","last_undegraded":"2024-09-12T03:02:04.291897+0000","last_fullsized":"2024-09-12T03:02:04.291897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:38:15.557067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282354+0000","last_change":"2024-09-12T03:00:56.315356+0000","last_active":"2024-09-12T03:02:04.282354+0000","last_peered":"2024-09-12T03:02:04.282354+0000","last_clean":"2024-09-12T03:02:04.282354+0000","last_became_active":"2024-09-12T03:00:56.314873+0000","last_became_peered":"2024-09-12T03:00:56.314873+0000","last_unstale":"2024-09-12T03:02:04.282354+0000","last_undegraded":"2024-09-12T03:02:04.282354+0000","last_fullsized":"2024-09-12T03:02:04.282354+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:55.710156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.286199+0000","last_change":"2024-09-12T03:00:58.346706+0000","last_active":"2024-09-12T03:02:04.286199+0000","last_peered":"2024-09-12T03:02:04.286199+0000","last_clean":"2024-09-12T03:02:04.286199+0000","last_became_active":"2024-09-12T03:00:58.346363+0000","last_became_peered":"2024-09-12T03:00:58.346363+0000","last_unstale":"2024-09-12T03:02:04.286199+0000","last_undegraded":"2024-09-12T03:02:04.286199+0000","last_fullsized":"2024-09-12T03:02:04.286199+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:13.468639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.299582+0000","last_change":"2024-09-12T03:02:04.299147+0000","last_active":"2024-09-12T03:02:04.299582+0000","last_peered":"2024-09-12T03:02:04.299582+0000","last_clean":"2024-09-12T03:02:04.299582+0000","last_became_active":"2024-09-12T03:02:04.293211+0000","last_became_peered":"2024-09-12T03:02:04.293211+0000","last_unstale":"2024-09-12T03:02:04.299582+0000","last_undegraded":"2024-09-12T03:02:04.299582+0000","last_fullsized":"2024-09-12T03:02:04.299582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:01:39.222476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282418+0000","last_change":"2024-09-12T03:00:58.356682+0000","last_active":"2024-09-12T03:02:04.282418+0000","last_peered":"2024-09-12T03:02:04.282418+0000","last_clean":"2024-09-12T03:02:04.282418+0000","last_became_active":"2024-09-12T03:00:58.356383+0000","last_became_peered":"2024-09-12T03:00:58.356383+0000","last_unstale":"2024-09-12T03:02:04.282418+0000","last_undegraded":"2024-09-12T03:02:04.282418+0000","last_fullsized":"2024-09-12T03:02:04.282418+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:14:16.428393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292053+0000","last_change":"2024-09-12T03:00:56.314943+0000","last_active":"2024-09-12T03:02:04.292053+0000","last_peered":"2024-09-12T03:02:04.292053+0000","last_clean":"2024-09-12T03:02:04.292053+0000","last_became_active":"2024-09-12T03:00:56.314602+0000","last_became_peered":"2024-09-12T03:00:56.314602+0000","last_unstale":"2024-09-12T03:02:04.292053+0000","last_undegraded":"2024-09-12T03:02:04.292053+0000","last_fullsized":"2024-09-12T03:02:04.292053+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:31.207880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284215+0000","last_change":"2024-09-12T03:00:58.351774+0000","last_active":"2024-09-12T03:02:04.284215+0000","last_peered":"2024-09-12T03:02:04.284215+0000","last_clean":"2024-09-12T03:02:04.284215+0000","last_became_active":"2024-09-12T03:00:58.351453+0000","last_became_peered":"2024-09-12T03:00:58.351453+0000","last_unstale":"2024-09-12T03:02:04.284215+0000","last_undegraded":"2024-09-12T03:02:04.284215+0000","last_fullsized":"2024-09-12T03:02:04.284215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:18:24.260110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.287322+0000","last_change":"2024-09-12T03:00:58.346886+0000","last_active":"2024-09-12T03:02:04.287322+0000","last_peered":"2024-09-12T03:02:04.287322+0000","last_clean":"2024-09-12T03:02:04.287322+0000","last_became_active":"2024-09-12T03:00:58.346499+0000","last_became_peered":"2024-09-12T03:00:58.346499+0000","last_unstale":"2024-09-12T03:02:04.287322+0000","last_undegraded":"2024-09-12T03:02:04.287322+0000","last_fullsized":"2024-09-12T03:02:04.287322+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:02:24.045610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292497+0000","last_change":"2024-09-12T03:00:56.317443+0000","last_active":"2024-09-12T03:02:04.292497+0000","last_peered":"2024-09-12T03:02:04.292497+0000","last_clean":"2024-09-12T03:02:04.292497+0000","last_became_active":"2024-09-12T03:00:56.317045+0000","last_became_peered":"2024-09-12T03:00:56.317045+0000","last_unstale":"2024-09-12T03:02:04.292497+0000","last_undegraded":"2024-09-12T03:02:04.292497+0000","last_fullsized":"2024-09-12T03:02:04.292497+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:07.687606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332419+0000","last_change":"2024-09-12T03:00:58.337257+0000","last_active":"2024-09-12T03:02:04.332419+0000","last_peered":"2024-09-12T03:02:04.332419+0000","last_clean":"2024-09-12T03:02:04.332419+0000","last_became_active":"2024-09-12T03:00:58.336847+0000","last_became_peered":"2024-09-12T03:00:58.336847+0000","last_unstale":"2024-09-12T03:02:04.332419+0000","last_undegraded":"2024-09-12T03:02:04.332419+0000","last_fullsized":"2024-09-12T03:02:04.332419+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:22.846206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292165+0000","last_change":"2024-09-12T03:00:56.316255+0000","last_active":"2024-09-12T03:02:04.292165+0000","last_peered":"2024-09-12T03:02:04.292165+0000","last_clean":"2024-09-12T03:02:04.292165+0000","last_became_active":"2024-09-12T03:00:56.315906+0000","last_became_peered":"2024-09-12T03:00:56.315906+0000","last_unstale":"2024-09-12T03:02:04.292165+0000","last_undegraded":"2024-09-12T03:02:04.292165+0000","last_fullsized":"2024-09-12T03:02:04.292165+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:14.798601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332714+0000","last_change":"2024-09-12T03:00:58.347206+0000","last_active":"2024-09-12T03:02:04.332714+0000","last_peered":"2024-09-12T03:02:04.332714+0000","last_clean":"2024-09-12T03:02:04.332714+0000","last_became_active":"2024-09-12T03:00:58.346923+0000","last_became_peered":"2024-09-12T03:00:58.346923+0000","last_unstale":"2024-09-12T03:02:04.332714+0000","last_undegraded":"2024-09-12T03:02:04.332714+0000","last_fullsized":"2024-09-12T03:02:04.332714+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:41.376211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.283694+0000","last_change":"2024-09-12T03:00:56.435431+0000","last_active":"2024-09-12T03:02:04.283694+0000","last_peered":"2024-09-12T03:02:04.283694+0000","last_clean":"2024-09-12T03:02:04.283694+0000","last_became_active":"2024-09-12T03:00:56.435088+0000","last_became_peered":"2024-09-12T03:00:56.435088+0000","last_unstale":"2024-09-12T03:02:04.283694+0000","last_undegraded":"2024-09-12T03:02:04.283694+0000","last_fullsized":"2024-09-12T03:02:04.283694+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:24.738340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.283572+0000","last_change":"2024-09-12T03:00:58.351737+0000","last_active":"2024-09-12T03:02:04.283572+0000","last_peered":"2024-09-12T03:02:04.283572+0000","last_clean":"2024-09-12T03:02:04.283572+0000","last_became_active":"2024-09-12T03:00:58.351407+0000","last_became_peered":"2024-09-12T03:00:58.351407+0000","last_unstale":"2024-09-12T03:02:04.283572+0000","last_undegraded":"2024-09-12T03:02:04.283572+0000","last_fullsized":"2024-09-12T03:02:04.283572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:34.206703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284979+0000","last_change":"2024-09-12T03:00:56.310910+0000","last_active":"2024-09-12T03:02:04.284979+0000","last_peered":"2024-09-12T03:02:04.284979+0000","last_clean":"2024-09-12T03:02:04.284979+0000","last_became_active":"2024-09-12T03:00:56.310657+0000","last_became_peered":"2024-09-12T03:00:56.310657+0000","last_unstale":"2024-09-12T03:02:04.284979+0000","last_undegraded":"2024-09-12T03:02:04.284979+0000","last_fullsized":"2024-09-12T03:02:04.284979+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:06.117215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65362,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":411,"num_read_kb":270,"num_write":161,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":196086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245899,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645341,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":880640,"data_stored":431835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906977,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1028096,"data_stored":1085046,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070503,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":897024,"data_stored":432166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299436,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":431947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528369,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":942080,"data_stored":1023508,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":40,"seq":171798691894,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":880640,"data_stored":431438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":33,"seq":141733920827,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":954368,"data_stored":1023192,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051649,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":884736,"data_stored":431875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280581,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":892928,"data_stored":431632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476810,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":966656,"data_stored":494333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607632,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":942080,"data_stored":492983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":901120,"data_stored":432709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:02:17.362 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T03:02:17.362 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-12T03:02:17.362 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-12T03:02:17.362 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-12T03:02:17.609 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:18.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.123+0000 7f384e588640 1 -- 172.21.15.116:0/3493093061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38481055b0 msgr2=0x7f384810d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:18.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.123+0000 7f384e588640 1 --2- 172.21.15.116:0/3493093061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38481055b0 0x7f384810d130 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3840009ca0 tx=0x7f384002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.123+0000 7f384e588640 1 -- 172.21.15.116:0/3493093061 shutdown_connections 2024-09-12T03:02:18.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.123+0000 7f384e588640 1 --2- 172.21.15.116:0/3493093061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38481055b0 0x7f384810d130 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.123+0000 7f384e588640 1 --2- 172.21.15.116:0/3493093061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3848100e30 0x7f3848104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.123+0000 7f384e588640 1 --2- 172.21.15.116:0/3493093061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3848100480 0x7f3848100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.123+0000 7f384e588640 1 -- 172.21.15.116:0/3493093061 >> 172.21.15.116:0/3493093061 conn(0x7f3848077210 msgr2=0x7f38480fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:18.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.123+0000 7f384e588640 1 -- 172.21.15.116:0/3493093061 shutdown_connections 2024-09-12T03:02:18.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.123+0000 7f384e588640 1 -- 172.21.15.116:0/3493093061 wait complete. 2024-09-12T03:02:18.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.123+0000 7f384e588640 1 Processor -- start 2024-09-12T03:02:18.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.123+0000 7f384e588640 1 -- start start 2024-09-12T03:02:18.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.124+0000 7f384e588640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3848100480 0x7f3848197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:18.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.124+0000 7f384e588640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3848100e30 0x7f3848197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:18.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.124+0000 7f384e588640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38481055b0 0x7f384819bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:18.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.124+0000 7f384e588640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f38481105c0 con 0x7f3848100e30 2024-09-12T03:02:18.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.124+0000 7f384e588640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3848110440 con 0x7f3848100480 2024-09-12T03:02:18.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.124+0000 7f384e588640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3848110740 con 0x7f38481055b0 2024-09-12T03:02:18.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.124+0000 7f38477fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3848100e30 0x7f3848197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:18.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.124+0000 7f38477fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3848100e30 0x7f3848197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43364/0 (socket says 172.21.15.116:43364) 2024-09-12T03:02:18.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.124+0000 7f38477fe640 1 -- 172.21.15.116:0/1030921060 learned_addr learned my addr 172.21.15.116:0/1030921060 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:18.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.124+0000 7f3847fff640 1 --2- 172.21.15.116:0/1030921060 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3848100480 0x7f3848197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:18.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.124+0000 7f38477fe640 1 -- 172.21.15.116:0/1030921060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38481055b0 msgr2=0x7f384819bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:18.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.124+0000 7f384cb25640 1 --2- 172.21.15.116:0/1030921060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38481055b0 0x7f384819bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:18.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.124+0000 7f38477fe640 1 --2- 172.21.15.116:0/1030921060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38481055b0 0x7f384819bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.124+0000 7f38477fe640 1 -- 172.21.15.116:0/1030921060 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3848100480 msgr2=0x7f3848197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:18.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.124+0000 7f38477fe640 1 --2- 172.21.15.116:0/1030921060 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3848100480 0x7f3848197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.124+0000 7f38477fe640 1 -- 172.21.15.116:0/1030921060 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f384819c2d0 con 0x7f3848100e30 2024-09-12T03:02:18.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.124+0000 7f3847fff640 1 --2- 172.21.15.116:0/1030921060 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3848100480 0x7f3848197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:18.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.124+0000 7f384cb25640 1 --2- 172.21.15.116:0/1030921060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38481055b0 0x7f384819bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:18.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.124+0000 7f38477fe640 1 --2- 172.21.15.116:0/1030921060 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3848100e30 0x7f3848197860 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3830007b50 tx=0x7f383000e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:18.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.124+0000 7f38457fa640 1 -- 172.21.15.116:0/1030921060 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3830007c40 con 0x7f3848100e30 2024-09-12T03:02:18.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.124+0000 7f38457fa640 1 -- 172.21.15.116:0/1030921060 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3830007de0 con 0x7f3848100e30 2024-09-12T03:02:18.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.124+0000 7f384e588640 1 -- 172.21.15.116:0/1030921060 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f384819c5c0 con 0x7f3848100e30 2024-09-12T03:02:18.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.124+0000 7f38457fa640 1 -- 172.21.15.116:0/1030921060 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383000ae60 con 0x7f3848100e30 2024-09-12T03:02:18.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.124+0000 7f384e588640 1 -- 172.21.15.116:0/1030921060 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3848103410 con 0x7f3848100e30 2024-09-12T03:02:18.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.126+0000 7f384e588640 1 -- 172.21.15.116:0/1030921060 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3848100860 con 0x7f3848100e30 2024-09-12T03:02:18.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.127+0000 7f38457fa640 1 -- 172.21.15.116:0/1030921060 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f383000a440 con 0x7f3848100e30 2024-09-12T03:02:18.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.127+0000 7f38457fa640 1 --2- 172.21.15.116:0/1030921060 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f38340a2c10 0x7f38340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:18.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.127+0000 7f3847fff640 1 --2- 172.21.15.116:0/1030921060 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f38340a2c10 0x7f38340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:18.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.127+0000 7f38457fa640 1 -- 172.21.15.116:0/1030921060 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f38300d0710 con 0x7f3848100e30 2024-09-12T03:02:18.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.128+0000 7f3847fff640 1 --2- 172.21.15.116:0/1030921060 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f38340a2c10 0x7f38340a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f38380058c0 tx=0x7f3838005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:18.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.131+0000 7f38457fa640 1 -- 172.21.15.116:0/1030921060 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3830023060 con 0x7f3848100e30 2024-09-12T03:02:18.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:18 smithi116 ceph-mon[31410]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:02:18.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:18 smithi116 ceph-mon[31410]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 11 B/s, 0 objects/s recovering 2024-09-12T03:02:18.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.339+0000 7f384e588640 1 -- 172.21.15.116:0/1030921060 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3848103670 con 0x7f3848100e30 2024-09-12T03:02:18.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.339+0000 7f38457fa640 1 -- 172.21.15.116:0/1030921060 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f383009d5d0 con 0x7f3848100e30 2024-09-12T03:02:18.340 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:02:18.340 INFO:teuthology.orchestra.run.smithi116.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-12T03:02:18.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.341+0000 7f384e588640 1 -- 172.21.15.116:0/1030921060 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f38340a2c10 msgr2=0x7f38340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:18.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.341+0000 7f384e588640 1 --2- 172.21.15.116:0/1030921060 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f38340a2c10 0x7f38340a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f38380058c0 tx=0x7f3838005ec0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.341+0000 7f384e588640 1 -- 172.21.15.116:0/1030921060 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3848100e30 msgr2=0x7f3848197860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:18.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.341+0000 7f384e588640 1 --2- 172.21.15.116:0/1030921060 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3848100e30 0x7f3848197860 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3830007b50 tx=0x7f383000e420 comp rx=0 tx=0).stop 2024-09-12T03:02:18.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.342+0000 7f384e588640 1 -- 172.21.15.116:0/1030921060 shutdown_connections 2024-09-12T03:02:18.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.342+0000 7f384e588640 1 --2- 172.21.15.116:0/1030921060 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f38340a2c10 0x7f38340a50d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.342+0000 7f384e588640 1 --2- 172.21.15.116:0/1030921060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38481055b0 0x7f384819bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.342+0000 7f384e588640 1 --2- 172.21.15.116:0/1030921060 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3848100e30 0x7f3848197860 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.342+0000 7f384e588640 1 --2- 172.21.15.116:0/1030921060 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3848100480 0x7f3848197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.342+0000 7f384e588640 1 -- 172.21.15.116:0/1030921060 >> 172.21.15.116:0/1030921060 conn(0x7f3848077210 msgr2=0x7f38480fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:18.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.342+0000 7f384e588640 1 -- 172.21.15.116:0/1030921060 shutdown_connections 2024-09-12T03:02:18.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:18.342+0000 7f384e588640 1 -- 172.21.15.116:0/1030921060 wait complete. 2024-09-12T03:02:18.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:18 smithi183 ceph-mon[35535]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:02:18.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:18 smithi183 ceph-mon[35535]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 11 B/s, 0 objects/s recovering 2024-09-12T03:02:18.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:18 smithi177 ceph-mon[35305]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:02:18.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:18 smithi177 ceph-mon[35305]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 11 B/s, 0 objects/s recovering 2024-09-12T03:02:19.048 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-12T03:02:19.048 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:02:19.294 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:19.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:19 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1030921060' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T03:02:19.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:19 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1030921060' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T03:02:19.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:19 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1030921060' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T03:02:19.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.802+0000 7f42eaddb640 1 -- 172.21.15.116:0/2021601658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42e4102da0 msgr2=0x7f42e4103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:19.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.802+0000 7f42eaddb640 1 --2- 172.21.15.116:0/2021601658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42e4102da0 0x7f42e4103180 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f42d4009ca0 tx=0x7f42d402a950 comp rx=0 tx=0).stop 2024-09-12T03:02:19.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.802+0000 7f42eaddb640 1 -- 172.21.15.116:0/2021601658 shutdown_connections 2024-09-12T03:02:19.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.802+0000 7f42eaddb640 1 --2- 172.21.15.116:0/2021601658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42e410c180 0x7f42e410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:19.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.802+0000 7f42eaddb640 1 --2- 172.21.15.116:0/2021601658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42e41036c0 0x7f42e410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:19.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.803+0000 7f42eaddb640 1 --2- 172.21.15.116:0/2021601658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42e4102da0 0x7f42e4103180 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:19.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.803+0000 7f42eaddb640 1 -- 172.21.15.116:0/2021601658 >> 172.21.15.116:0/2021601658 conn(0x7f42e40fc7a0 msgr2=0x7f42e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:19.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.803+0000 7f42eaddb640 1 -- 172.21.15.116:0/2021601658 shutdown_connections 2024-09-12T03:02:19.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.803+0000 7f42eaddb640 1 -- 172.21.15.116:0/2021601658 wait complete. 2024-09-12T03:02:19.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.803+0000 7f42eaddb640 1 Processor -- start 2024-09-12T03:02:19.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.803+0000 7f42eaddb640 1 -- start start 2024-09-12T03:02:19.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.803+0000 7f42eaddb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42e4102da0 0x7f42e4100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:19.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.803+0000 7f42eaddb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42e41036c0 0x7f42e4100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:19.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.803+0000 7f42eaddb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42e410c180 0x7f42e4101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:19.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.803+0000 7f42eaddb640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f42e4111fa0 con 0x7f42e41036c0 2024-09-12T03:02:19.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.803+0000 7f42eaddb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f42e4111e20 con 0x7f42e410c180 2024-09-12T03:02:19.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.803+0000 7f42eaddb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f42e4112120 con 0x7f42e4102da0 2024-09-12T03:02:19.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.803+0000 7f42e3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42e41036c0 0x7f42e4100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:19.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.803+0000 7f42e3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42e41036c0 0x7f42e4100f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43378/0 (socket says 172.21.15.116:43378) 2024-09-12T03:02:19.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.803+0000 7f42e3fff640 1 -- 172.21.15.116:0/3932997825 learned_addr learned my addr 172.21.15.116:0/3932997825 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:19.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.804+0000 7f42e8b77640 1 --2- 172.21.15.116:0/3932997825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42e4102da0 0x7f42e4100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:19.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.804+0000 7f42e9378640 1 --2- 172.21.15.116:0/3932997825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42e410c180 0x7f42e4101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:19.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.804+0000 7f42e3fff640 1 -- 172.21.15.116:0/3932997825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42e4102da0 msgr2=0x7f42e4100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:19.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.804+0000 7f42e3fff640 1 --2- 172.21.15.116:0/3932997825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42e4102da0 0x7f42e4100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:19.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.804+0000 7f42e3fff640 1 -- 172.21.15.116:0/3932997825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42e410c180 msgr2=0x7f42e4101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:19.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.804+0000 7f42e3fff640 1 --2- 172.21.15.116:0/3932997825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42e410c180 0x7f42e4101480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:19.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.804+0000 7f42e3fff640 1 -- 172.21.15.116:0/3932997825 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e41a6920 con 0x7f42e41036c0 2024-09-12T03:02:19.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.804+0000 7f42e8b77640 1 --2- 172.21.15.116:0/3932997825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42e4102da0 0x7f42e4100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:19.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.804+0000 7f42e3fff640 1 --2- 172.21.15.116:0/3932997825 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42e41036c0 0x7f42e4100f40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f42d800c890 tx=0x7f42d800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:19.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.804+0000 7f42e1ffb640 1 -- 172.21.15.116:0/3932997825 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d8004220 con 0x7f42e41036c0 2024-09-12T03:02:19.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.804+0000 7f42e1ffb640 1 -- 172.21.15.116:0/3932997825 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42d80043c0 con 0x7f42e41036c0 2024-09-12T03:02:19.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.804+0000 7f42eaddb640 1 -- 172.21.15.116:0/3932997825 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e41a6c10 con 0x7f42e41036c0 2024-09-12T03:02:19.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.804+0000 7f42e1ffb640 1 -- 172.21.15.116:0/3932997825 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d80049e0 con 0x7f42e41036c0 2024-09-12T03:02:19.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.804+0000 7f42eaddb640 1 -- 172.21.15.116:0/3932997825 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e41a7150 con 0x7f42e41036c0 2024-09-12T03:02:19.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.805+0000 7f42eaddb640 1 -- 172.21.15.116:0/3932997825 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42e4103180 con 0x7f42e41036c0 2024-09-12T03:02:19.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.806+0000 7f42e1ffb640 1 -- 172.21.15.116:0/3932997825 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f42d80079b0 con 0x7f42e41036c0 2024-09-12T03:02:19.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.807+0000 7f42e1ffb640 1 --2- 172.21.15.116:0/3932997825 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f42cc0a2c10 0x7f42cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:19.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.807+0000 7f42e8b77640 1 --2- 172.21.15.116:0/3932997825 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f42cc0a2c10 0x7f42cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:19.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.807+0000 7f42e1ffb640 1 -- 172.21.15.116:0/3932997825 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f42d80cf2b0 con 0x7f42e41036c0 2024-09-12T03:02:19.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.807+0000 7f42e8b77640 1 --2- 172.21.15.116:0/3932997825 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f42cc0a2c10 0x7f42cc0a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f42d4006010 tx=0x7f42d4005f80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:19.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:19.811+0000 7f42e1ffb640 1 -- 172.21.15.116:0/3932997825 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f42d8014030 con 0x7f42e41036c0 2024-09-12T03:02:20.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:20.013+0000 7f42eaddb640 1 -- 172.21.15.116:0/3932997825 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42e4101f30 con 0x7f42e41036c0 2024-09-12T03:02:20.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:20.014+0000 7f42e1ffb640 1 -- 172.21.15.116:0/3932997825 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11938 (secure 0 0 0) 0x7f42d809c0e0 con 0x7f42e41036c0 2024-09-12T03:02:20.014 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:02:20.015 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":24,"btime":"2024-09-12T03:01:10:397269+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:01:10.218314+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24467,24475,24494]},"id":1}]} 2024-09-12T03:02:20.015 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 24 2024-09-12T03:02:20.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:20.017+0000 7f42eaddb640 1 -- 172.21.15.116:0/3932997825 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f42cc0a2c10 msgr2=0x7f42cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:20.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:20.017+0000 7f42eaddb640 1 --2- 172.21.15.116:0/3932997825 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f42cc0a2c10 0x7f42cc0a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f42d4006010 tx=0x7f42d4005f80 comp rx=0 tx=0).stop 2024-09-12T03:02:20.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:20.017+0000 7f42eaddb640 1 -- 172.21.15.116:0/3932997825 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42e41036c0 msgr2=0x7f42e4100f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:20.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:20.017+0000 7f42eaddb640 1 --2- 172.21.15.116:0/3932997825 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42e41036c0 0x7f42e4100f40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f42d800c890 tx=0x7f42d800cd50 comp rx=0 tx=0).stop 2024-09-12T03:02:20.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:20.017+0000 7f42eaddb640 1 -- 172.21.15.116:0/3932997825 shutdown_connections 2024-09-12T03:02:20.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:20.017+0000 7f42eaddb640 1 --2- 172.21.15.116:0/3932997825 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f42cc0a2c10 0x7f42cc0a50d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:20.017+0000 7f42eaddb640 1 --2- 172.21.15.116:0/3932997825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42e410c180 0x7f42e4101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:20.017+0000 7f42eaddb640 1 --2- 172.21.15.116:0/3932997825 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42e41036c0 0x7f42e4100f40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:20.017+0000 7f42eaddb640 1 --2- 172.21.15.116:0/3932997825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42e4102da0 0x7f42e4100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:20.017+0000 7f42eaddb640 1 -- 172.21.15.116:0/3932997825 >> 172.21.15.116:0/3932997825 conn(0x7f42e40fc7a0 msgr2=0x7f42e40fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:20.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:20.017+0000 7f42eaddb640 1 -- 172.21.15.116:0/3932997825 shutdown_connections 2024-09-12T03:02:20.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:20.017+0000 7f42eaddb640 1 -- 172.21.15.116:0/3932997825 wait complete. 2024-09-12T03:02:20.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:20 smithi116 ceph-mon[31410]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 11 B/s, 0 objects/s recovering 2024-09-12T03:02:20.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:20 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3932997825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:20.675 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:02:20.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:20 smithi183 ceph-mon[35535]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 11 B/s, 0 objects/s recovering 2024-09-12T03:02:20.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:20 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3932997825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:20.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:20 smithi177 ceph-mon[35305]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 11 B/s, 0 objects/s recovering 2024-09-12T03:02:20.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:20 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3932997825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:20.921 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:21.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.427+0000 7f7d5e3d0640 1 -- 172.21.15.116:0/3326392719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d58100e20 msgr2=0x7f7d58101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:21.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.427+0000 7f7d5e3d0640 1 --2- 172.21.15.116:0/3326392719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d58100e20 0x7f7d58101200 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009900 tx=0x7f7d4802a480 comp rx=0 tx=0).stop 2024-09-12T03:02:21.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.427+0000 7f7d5e3d0640 1 -- 172.21.15.116:0/3326392719 shutdown_connections 2024-09-12T03:02:21.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.427+0000 7f7d5e3d0640 1 --2- 172.21.15.116:0/3326392719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d58102ef0 0x7f7d5810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:21.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.427+0000 7f7d5e3d0640 1 --2- 172.21.15.116:0/3326392719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d58102570 0x7f7d581029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:21.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.427+0000 7f7d5e3d0640 1 --2- 172.21.15.116:0/3326392719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d58100e20 0x7f7d58101200 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:21.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.427+0000 7f7d5e3d0640 1 -- 172.21.15.116:0/3326392719 >> 172.21.15.116:0/3326392719 conn(0x7f7d580fc7d0 msgr2=0x7f7d580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:21.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.427+0000 7f7d5e3d0640 1 -- 172.21.15.116:0/3326392719 shutdown_connections 2024-09-12T03:02:21.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.427+0000 7f7d5e3d0640 1 -- 172.21.15.116:0/3326392719 wait complete. 2024-09-12T03:02:21.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.427+0000 7f7d5e3d0640 1 Processor -- start 2024-09-12T03:02:21.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.427+0000 7f7d5e3d0640 1 -- start start 2024-09-12T03:02:21.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.428+0000 7f7d5e3d0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d58100e20 0x7f7d581993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:21.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.428+0000 7f7d5e3d0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d58102570 0x7f7d58199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:21.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.428+0000 7f7d5e3d0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d58102ef0 0x7f7d581a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:21.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.428+0000 7f7d5e3d0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7d58110f50 con 0x7f7d58102570 2024-09-12T03:02:21.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.428+0000 7f7d5e3d0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7d58110dd0 con 0x7f7d58102ef0 2024-09-12T03:02:21.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.428+0000 7f7d5e3d0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d581110d0 con 0x7f7d58100e20 2024-09-12T03:02:21.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.428+0000 7f7d577fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d58102570 0x7f7d58199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:21.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.428+0000 7f7d577fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d58102570 0x7f7d58199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32876/0 (socket says 172.21.15.116:32876) 2024-09-12T03:02:21.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.428+0000 7f7d577fe640 1 -- 172.21.15.116:0/2453210272 learned_addr learned my addr 172.21.15.116:0/2453210272 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:21.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.428+0000 7f7d57fff640 1 --2- 172.21.15.116:0/2453210272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d58100e20 0x7f7d581993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:21.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.428+0000 7f7d5c96d640 1 --2- 172.21.15.116:0/2453210272 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d58102ef0 0x7f7d581a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:21.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.428+0000 7f7d577fe640 1 -- 172.21.15.116:0/2453210272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d58100e20 msgr2=0x7f7d581993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:21.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.428+0000 7f7d577fe640 1 --2- 172.21.15.116:0/2453210272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d58100e20 0x7f7d581993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:21.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.428+0000 7f7d577fe640 1 -- 172.21.15.116:0/2453210272 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d58102ef0 msgr2=0x7f7d581a4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:21.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.428+0000 7f7d577fe640 1 --2- 172.21.15.116:0/2453210272 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d58102ef0 0x7f7d581a4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:21.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.428+0000 7f7d577fe640 1 -- 172.21.15.116:0/2453210272 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d581a4980 con 0x7f7d58102570 2024-09-12T03:02:21.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.428+0000 7f7d577fe640 1 --2- 172.21.15.116:0/2453210272 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d58102570 0x7f7d58199900 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00d950 tx=0x7f7d4c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:21.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.428+0000 7f7d557fa640 1 -- 172.21.15.116:0/2453210272 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c017070 con 0x7f7d58102570 2024-09-12T03:02:21.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.428+0000 7f7d5e3d0640 1 -- 172.21.15.116:0/2453210272 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d581a4b50 con 0x7f7d58102570 2024-09-12T03:02:21.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.428+0000 7f7d557fa640 1 -- 172.21.15.116:0/2453210272 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d4c009c30 con 0x7f7d58102570 2024-09-12T03:02:21.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.428+0000 7f7d557fa640 1 -- 172.21.15.116:0/2453210272 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c00f6b0 con 0x7f7d58102570 2024-09-12T03:02:21.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.428+0000 7f7d5e3d0640 1 -- 172.21.15.116:0/2453210272 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d581a5040 con 0x7f7d58102570 2024-09-12T03:02:21.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.429+0000 7f7d5e3d0640 1 -- 172.21.15.116:0/2453210272 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d58101200 con 0x7f7d58102570 2024-09-12T03:02:21.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.430+0000 7f7d557fa640 1 -- 172.21.15.116:0/2453210272 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f7d4c0109d0 con 0x7f7d58102570 2024-09-12T03:02:21.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.431+0000 7f7d557fa640 1 --2- 172.21.15.116:0/2453210272 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7d400a2c10 0x7f7d400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:21.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.431+0000 7f7d57fff640 1 --2- 172.21.15.116:0/2453210272 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7d400a2c10 0x7f7d400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:21.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.431+0000 7f7d557fa640 1 -- 172.21.15.116:0/2453210272 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f7d4c0ceec0 con 0x7f7d58102570 2024-09-12T03:02:21.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.431+0000 7f7d57fff640 1 --2- 172.21.15.116:0/2453210272 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7d400a2c10 0x7f7d400a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7d48005ab0 tx=0x7f7d48005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:21.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.435+0000 7f7d557fa640 1 -- 172.21.15.116:0/2453210272 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7d4c009de0 con 0x7f7d58102570 2024-09-12T03:02:21.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.635+0000 7f7d5e3d0640 1 -- 172.21.15.116:0/2453210272 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d58109090 con 0x7f7d58102570 2024-09-12T03:02:21.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.636+0000 7f7d557fa640 1 -- 172.21.15.116:0/2453210272 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11938 (secure 0 0 0) 0x7f7d4c09bc90 con 0x7f7d58102570 2024-09-12T03:02:21.636 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:02:21.636 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":24,"btime":"2024-09-12T03:01:10:397269+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:01:10.218314+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24467,24475,24494]},"id":1}]} 2024-09-12T03:02:21.637 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 24 2024-09-12T03:02:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.638+0000 7f7d5e3d0640 1 -- 172.21.15.116:0/2453210272 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7d400a2c10 msgr2=0x7f7d400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.638+0000 7f7d5e3d0640 1 --2- 172.21.15.116:0/2453210272 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7d400a2c10 0x7f7d400a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7d48005ab0 tx=0x7f7d48005a20 comp rx=0 tx=0).stop 2024-09-12T03:02:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.638+0000 7f7d5e3d0640 1 -- 172.21.15.116:0/2453210272 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d58102570 msgr2=0x7f7d58199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.638+0000 7f7d5e3d0640 1 --2- 172.21.15.116:0/2453210272 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d58102570 0x7f7d58199900 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00d950 tx=0x7f7d4c00de10 comp rx=0 tx=0).stop 2024-09-12T03:02:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.638+0000 7f7d5e3d0640 1 -- 172.21.15.116:0/2453210272 shutdown_connections 2024-09-12T03:02:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.638+0000 7f7d5e3d0640 1 --2- 172.21.15.116:0/2453210272 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7d400a2c10 0x7f7d400a50d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.638+0000 7f7d5e3d0640 1 --2- 172.21.15.116:0/2453210272 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d58102ef0 0x7f7d581a4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.638+0000 7f7d5e3d0640 1 --2- 172.21.15.116:0/2453210272 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d58102570 0x7f7d58199900 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.638+0000 7f7d5e3d0640 1 --2- 172.21.15.116:0/2453210272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d58100e20 0x7f7d581993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.638+0000 7f7d5e3d0640 1 -- 172.21.15.116:0/2453210272 >> 172.21.15.116:0/2453210272 conn(0x7f7d580fc7d0 msgr2=0x7f7d580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.638+0000 7f7d5e3d0640 1 -- 172.21.15.116:0/2453210272 shutdown_connections 2024-09-12T03:02:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:21.638+0000 7f7d5e3d0640 1 -- 172.21.15.116:0/2453210272 wait complete. 2024-09-12T03:02:22.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:22 smithi116 ceph-mon[31410]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-09-12T03:02:22.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:22 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2453210272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:22.392 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:02:22.641 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:22.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:22 smithi183 ceph-mon[35535]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-09-12T03:02:22.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:22 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2453210272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:22.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:22 smithi177 ceph-mon[35305]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-09-12T03:02:22.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:22 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2453210272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:23.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.165+0000 7efc57eeb640 1 -- 172.21.15.116:0/592845618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc50103660 msgr2=0x7efc5010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:23.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.165+0000 7efc57eeb640 1 --2- 172.21.15.116:0/592845618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc50103660 0x7efc5010bb50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efc44009900 tx=0x7efc4402a480 comp rx=0 tx=0).stop 2024-09-12T03:02:23.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.165+0000 7efc57eeb640 1 -- 172.21.15.116:0/592845618 shutdown_connections 2024-09-12T03:02:23.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.165+0000 7efc57eeb640 1 --2- 172.21.15.116:0/592845618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc5010c170 0x7efc5010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.165+0000 7efc57eeb640 1 --2- 172.21.15.116:0/592845618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc50103660 0x7efc5010bb50 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.165+0000 7efc57eeb640 1 --2- 172.21.15.116:0/592845618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc50102d40 0x7efc50103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.165+0000 7efc57eeb640 1 -- 172.21.15.116:0/592845618 >> 172.21.15.116:0/592845618 conn(0x7efc500fc7a0 msgr2=0x7efc500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:23.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.165+0000 7efc57eeb640 1 -- 172.21.15.116:0/592845618 shutdown_connections 2024-09-12T03:02:23.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.165+0000 7efc57eeb640 1 -- 172.21.15.116:0/592845618 wait complete. 2024-09-12T03:02:23.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.166+0000 7efc57eeb640 1 Processor -- start 2024-09-12T03:02:23.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.166+0000 7efc57eeb640 1 -- start start 2024-09-12T03:02:23.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.166+0000 7efc57eeb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc50102d40 0x7efc501009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:23.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.166+0000 7efc57eeb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc50103660 0x7efc50100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:23.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.166+0000 7efc57eeb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc5010c170 0x7efc50101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:23.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.166+0000 7efc57eeb640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efc50111fb0 con 0x7efc5010c170 2024-09-12T03:02:23.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.166+0000 7efc57eeb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efc50111e30 con 0x7efc50103660 2024-09-12T03:02:23.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.166+0000 7efc57eeb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efc50112130 con 0x7efc50102d40 2024-09-12T03:02:23.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.166+0000 7efc56488640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc5010c170 0x7efc50101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:23.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.166+0000 7efc56488640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc5010c170 0x7efc50101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32888/0 (socket says 172.21.15.116:32888) 2024-09-12T03:02:23.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.166+0000 7efc56488640 1 -- 172.21.15.116:0/2959091605 learned_addr learned my addr 172.21.15.116:0/2959091605 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:23.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.166+0000 7efc55c87640 1 --2- 172.21.15.116:0/2959091605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc50102d40 0x7efc501009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:23.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.167+0000 7efc56488640 1 -- 172.21.15.116:0/2959091605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc50102d40 msgr2=0x7efc501009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:23.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.167+0000 7efc56488640 1 --2- 172.21.15.116:0/2959091605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc50102d40 0x7efc501009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.167+0000 7efc55486640 1 --2- 172.21.15.116:0/2959091605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc50103660 0x7efc50100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:23.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.167+0000 7efc56488640 1 -- 172.21.15.116:0/2959091605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc50103660 msgr2=0x7efc50100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:23.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.167+0000 7efc56488640 1 --2- 172.21.15.116:0/2959091605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc50103660 0x7efc50100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.167+0000 7efc56488640 1 -- 172.21.15.116:0/2959091605 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc501a6950 con 0x7efc5010c170 2024-09-12T03:02:23.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.167+0000 7efc56488640 1 --2- 172.21.15.116:0/2959091605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc5010c170 0x7efc50101420 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efc4800b970 tx=0x7efc4800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:23.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.167+0000 7efc42ffd640 1 -- 172.21.15.116:0/2959091605 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48019070 con 0x7efc5010c170 2024-09-12T03:02:23.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.167+0000 7efc57eeb640 1 -- 172.21.15.116:0/2959091605 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc501a6c40 con 0x7efc5010c170 2024-09-12T03:02:23.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.167+0000 7efc42ffd640 1 -- 172.21.15.116:0/2959091605 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc4800cb40 con 0x7efc5010c170 2024-09-12T03:02:23.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.167+0000 7efc42ffd640 1 -- 172.21.15.116:0/2959091605 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48007650 con 0x7efc5010c170 2024-09-12T03:02:23.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.167+0000 7efc57eeb640 1 -- 172.21.15.116:0/2959091605 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc501a7180 con 0x7efc5010c170 2024-09-12T03:02:23.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.168+0000 7efc57eeb640 1 -- 172.21.15.116:0/2959091605 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc50103120 con 0x7efc5010c170 2024-09-12T03:02:23.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.169+0000 7efc42ffd640 1 -- 172.21.15.116:0/2959091605 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7efc4801c070 con 0x7efc5010c170 2024-09-12T03:02:23.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.170+0000 7efc42ffd640 1 --2- 172.21.15.116:0/2959091605 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7efc3c0a2c10 0x7efc3c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:23.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.170+0000 7efc55c87640 1 --2- 172.21.15.116:0/2959091605 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7efc3c0a2c10 0x7efc3c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:23.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.170+0000 7efc42ffd640 1 -- 172.21.15.116:0/2959091605 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7efc480ceaa0 con 0x7efc5010c170 2024-09-12T03:02:23.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.171+0000 7efc55c87640 1 --2- 172.21.15.116:0/2959091605 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7efc3c0a2c10 0x7efc3c0a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00bbe0 tx=0x7efc4c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:23.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.174+0000 7efc42ffd640 1 -- 172.21.15.116:0/2959091605 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efc4809b390 con 0x7efc5010c170 2024-09-12T03:02:23.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.345+0000 7efc57eeb640 1 -- 172.21.15.116:0/2959091605 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc50101ed0 con 0x7efc5010c170 2024-09-12T03:02:23.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.347+0000 7efc42ffd640 1 -- 172.21.15.116:0/2959091605 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22710 (secure 0 0 0) 0x7efc50101ed0 con 0x7efc5010c170 2024-09-12T03:02:23.347 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:02:23.348 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":80,"fsid":"eb1ac602-70b1-11ef-bcea-c7b262605968","created":"2024-09-12T02:53:07.105686+0000","modified":"2024-09-12T03:02:04.272898+0000","last_up_change":"2024-09-12T03:00:07.202639+0000","last_in_change":"2024-09-12T02:59:43.446336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:56:17.347152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:00:55.191040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:00:57.171142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8cf5a18-dc6f-4bb6-bd74-78adda310118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6803","nonce":862366853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6805","nonce":862366853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6809","nonce":862366853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6807","nonce":862366853}]},"public_addr":"172.21.15.116:6803/862366853","cluster_addr":"172.21.15.116:6805/862366853","heartbeat_back_addr":"172.21.15.116:6809/862366853","heartbeat_front_addr":"172.21.15.116:6807/862366853","state":["exists","up"]},{"osd":1,"uuid":"0affd3cb-9a51-42b0-a2cc-f30784a75cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6801","nonce":2245800547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6803","nonce":2245800547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6807","nonce":2245800547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6805","nonce":2245800547}]},"public_addr":"172.21.15.177:6801/2245800547","cluster_addr":"172.21.15.177:6803/2245800547","heartbeat_back_addr":"172.21.15.177:6807/2245800547","heartbeat_front_addr":"172.21.15.177:6805/2245800547","state":["exists","up"]},{"osd":2,"uuid":"1ce45ea5-84df-4a35-924f-78f92bd4f2cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6801","nonce":2645677759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6803","nonce":2645677759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6807","nonce":2645677759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6805","nonce":2645677759}]},"public_addr":"172.21.15.183:6801/2645677759","cluster_addr":"172.21.15.183:6803/2645677759","heartbeat_back_addr":"172.21.15.183:6807/2645677759","heartbeat_front_addr":"172.21.15.183:6805/2645677759","state":["exists","up"]},{"osd":3,"uuid":"ae5e8178-c998-4648-bc05-4bce822f0cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6811","nonce":686622148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6813","nonce":686622148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6817","nonce":686622148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6815","nonce":686622148}]},"public_addr":"172.21.15.116:6811/686622148","cluster_addr":"172.21.15.116:6813/686622148","heartbeat_back_addr":"172.21.15.116:6817/686622148","heartbeat_front_addr":"172.21.15.116:6815/686622148","state":["exists","up"]},{"osd":4,"uuid":"c81d2e73-0bc2-440d-b5a2-b7428af3ea4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6809","nonce":2382938021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6811","nonce":2382938021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6815","nonce":2382938021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6813","nonce":2382938021}]},"public_addr":"172.21.15.177:6809/2382938021","cluster_addr":"172.21.15.177:6811/2382938021","heartbeat_back_addr":"172.21.15.177:6815/2382938021","heartbeat_front_addr":"172.21.15.177:6813/2382938021","state":["exists","up"]},{"osd":5,"uuid":"c3f168be-9e27-4cf5-b73e-3f47d8990ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6809","nonce":85363432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6811","nonce":85363432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6815","nonce":85363432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6813","nonce":85363432}]},"public_addr":"172.21.15.183:6809/85363432","cluster_addr":"172.21.15.183:6811/85363432","heartbeat_back_addr":"172.21.15.183:6815/85363432","heartbeat_front_addr":"172.21.15.183:6813/85363432","state":["exists","up"]},{"osd":6,"uuid":"beb0a7e9-4909-47e2-9ee7-6951d4745e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6819","nonce":1121291186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6821","nonce":1121291186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6825","nonce":1121291186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6823","nonce":1121291186}]},"public_addr":"172.21.15.116:6819/1121291186","cluster_addr":"172.21.15.116:6821/1121291186","heartbeat_back_addr":"172.21.15.116:6825/1121291186","heartbeat_front_addr":"172.21.15.116:6823/1121291186","state":["exists","up"]},{"osd":7,"uuid":"761f4680-f049-473c-bf44-b19acf8de24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6817","nonce":2177665961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6819","nonce":2177665961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6823","nonce":2177665961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6821","nonce":2177665961}]},"public_addr":"172.21.15.177:6817/2177665961","cluster_addr":"172.21.15.177:6819/2177665961","heartbeat_back_addr":"172.21.15.177:6823/2177665961","heartbeat_front_addr":"172.21.15.177:6821/2177665961","state":["exists","up"]},{"osd":8,"uuid":"2385ea6c-fd5a-4815-9596-77f948629542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6817","nonce":1305725258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6819","nonce":1305725258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6823","nonce":1305725258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6821","nonce":1305725258}]},"public_addr":"172.21.15.183:6817/1305725258","cluster_addr":"172.21.15.183:6819/1305725258","heartbeat_back_addr":"172.21.15.183:6823/1305725258","heartbeat_front_addr":"172.21.15.183:6821/1305725258","state":["exists","up"]},{"osd":9,"uuid":"fd0c0dc4-4990-4f12-aa14-f855504cd995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6827","nonce":3831673668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6829","nonce":3831673668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6833","nonce":3831673668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6831","nonce":3831673668}]},"public_addr":"172.21.15.116:6827/3831673668","cluster_addr":"172.21.15.116:6829/3831673668","heartbeat_back_addr":"172.21.15.116:6833/3831673668","heartbeat_front_addr":"172.21.15.116:6831/3831673668","state":["exists","up"]},{"osd":10,"uuid":"864d1a97-4810-49c8-ad4a-a0c10253833c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6825","nonce":963663634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6827","nonce":963663634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6831","nonce":963663634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6829","nonce":963663634}]},"public_addr":"172.21.15.177:6825/963663634","cluster_addr":"172.21.15.177:6827/963663634","heartbeat_back_addr":"172.21.15.177:6831/963663634","heartbeat_front_addr":"172.21.15.177:6829/963663634","state":["exists","up"]},{"osd":11,"uuid":"d49c0841-4620-4f2f-a6b5-f2a9aa2e3c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6825","nonce":1818319939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6827","nonce":1818319939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6831","nonce":1818319939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6829","nonce":1818319939}]},"public_addr":"172.21.15.183:6825/1818319939","cluster_addr":"172.21.15.183:6827/1818319939","heartbeat_back_addr":"172.21.15.183:6831/1818319939","heartbeat_front_addr":"172.21.15.183:6829/1818319939","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:24.736944+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:50.310980+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:56:14.989533+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:56:41.187047+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:06.981664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:32.872771+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:58.850089+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:23.043121+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:47.839958+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:14.087708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:40.010548+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:05.364820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.116:6801/2966274423":"2024-09-13T02:54:03.026862+0000","172.21.15.116:6800/2966274423":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/686139455":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/1552047390":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/2180363411":"2024-09-13T02:53:39.433769+0000","172.21.15.116:6801/1791219601":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/3907400460":"2024-09-13T02:54:03.026862+0000","172.21.15.116:6800/1791219601":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/1802566286":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/212337802":"2024-09-13T02:53:39.433769+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:02:23.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.349+0000 7efc57eeb640 1 -- 172.21.15.116:0/2959091605 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7efc3c0a2c10 msgr2=0x7efc3c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:23.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.349+0000 7efc57eeb640 1 --2- 172.21.15.116:0/2959091605 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7efc3c0a2c10 0x7efc3c0a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00bbe0 tx=0x7efc4c00b390 comp rx=0 tx=0).stop 2024-09-12T03:02:23.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.349+0000 7efc57eeb640 1 -- 172.21.15.116:0/2959091605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc5010c170 msgr2=0x7efc50101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:23.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.349+0000 7efc57eeb640 1 --2- 172.21.15.116:0/2959091605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc5010c170 0x7efc50101420 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efc4800b970 tx=0x7efc4800be30 comp rx=0 tx=0).stop 2024-09-12T03:02:23.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.349+0000 7efc57eeb640 1 -- 172.21.15.116:0/2959091605 shutdown_connections 2024-09-12T03:02:23.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.349+0000 7efc57eeb640 1 --2- 172.21.15.116:0/2959091605 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7efc3c0a2c10 0x7efc3c0a50d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.349+0000 7efc57eeb640 1 --2- 172.21.15.116:0/2959091605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc5010c170 0x7efc50101420 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.349+0000 7efc57eeb640 1 --2- 172.21.15.116:0/2959091605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc50103660 0x7efc50100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.349+0000 7efc57eeb640 1 --2- 172.21.15.116:0/2959091605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc50102d40 0x7efc501009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:23.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.349+0000 7efc57eeb640 1 -- 172.21.15.116:0/2959091605 >> 172.21.15.116:0/2959091605 conn(0x7efc500fc7a0 msgr2=0x7efc500fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:23.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.349+0000 7efc57eeb640 1 -- 172.21.15.116:0/2959091605 shutdown_connections 2024-09-12T03:02:23.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:23.349+0000 7efc57eeb640 1 -- 172.21.15.116:0/2959091605 wait complete. 2024-09-12T03:02:24.067 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:02:24.315 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:24.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:24 smithi116 ceph-mon[31410]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:02:24.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:24 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2959091605' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:02:24.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:24 smithi183 ceph-mon[35535]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:02:24.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:24 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2959091605' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:02:24.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:24 smithi177 ceph-mon[35305]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:02:24.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:24 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2959091605' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:02:24.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.796+0000 7f643e296640 1 -- 172.21.15.116:0/1363929222 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6438100a60 msgr2=0x7f6438100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:24.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.796+0000 7f643e296640 1 --2- 172.21.15.116:0/1363929222 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6438100a60 0x7f6438100ec0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6420009930 tx=0x7f642002a420 comp rx=0 tx=0).stop 2024-09-12T03:02:24.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.796+0000 7f643e296640 1 -- 172.21.15.116:0/1363929222 shutdown_connections 2024-09-12T03:02:24.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.796+0000 7f643e296640 1 --2- 172.21.15.116:0/1363929222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6438101400 0x7f643810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.796+0000 7f643e296640 1 --2- 172.21.15.116:0/1363929222 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6438100a60 0x7f6438100ec0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.796+0000 7f643e296640 1 --2- 172.21.15.116:0/1363929222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6438106a60 0x7f6438106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.796+0000 7f643e296640 1 -- 172.21.15.116:0/1363929222 >> 172.21.15.116:0/1363929222 conn(0x7f64380fc780 msgr2=0x7f64380feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:24.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.796+0000 7f643e296640 1 -- 172.21.15.116:0/1363929222 shutdown_connections 2024-09-12T03:02:24.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.796+0000 7f643e296640 1 -- 172.21.15.116:0/1363929222 wait complete. 2024-09-12T03:02:24.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.797+0000 7f643e296640 1 Processor -- start 2024-09-12T03:02:24.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.797+0000 7f643e296640 1 -- start start 2024-09-12T03:02:24.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.798+0000 7f643e296640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6438100a60 0x7f64381973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:24.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.798+0000 7f643e296640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6438101400 0x7f6438197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:24.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.798+0000 7f643e296640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6438106a60 0x7f643819bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:24.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.798+0000 7f643e296640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6438110ea0 con 0x7f6438101400 2024-09-12T03:02:24.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.798+0000 7f643e296640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6438110d20 con 0x7f6438100a60 2024-09-12T03:02:24.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.798+0000 7f643e296640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6438111020 con 0x7f6438106a60 2024-09-12T03:02:24.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.798+0000 7f64377fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6438101400 0x7f6438197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:24.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.798+0000 7f64377fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6438101400 0x7f6438197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32902/0 (socket says 172.21.15.116:32902) 2024-09-12T03:02:24.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.798+0000 7f64377fe640 1 -- 172.21.15.116:0/1421755495 learned_addr learned my addr 172.21.15.116:0/1421755495 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:24.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.798+0000 7f6437fff640 1 --2- 172.21.15.116:0/1421755495 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6438100a60 0x7f64381973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:24.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.798+0000 7f643c833640 1 --2- 172.21.15.116:0/1421755495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6438106a60 0x7f643819bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:24.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.798+0000 7f64377fe640 1 -- 172.21.15.116:0/1421755495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6438106a60 msgr2=0x7f643819bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:24.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.798+0000 7f64377fe640 1 --2- 172.21.15.116:0/1421755495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6438106a60 0x7f643819bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.798+0000 7f64377fe640 1 -- 172.21.15.116:0/1421755495 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6438100a60 msgr2=0x7f64381973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:24.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.798+0000 7f64377fe640 1 --2- 172.21.15.116:0/1421755495 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6438100a60 0x7f64381973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.798+0000 7f64377fe640 1 -- 172.21.15.116:0/1421755495 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643819c390 con 0x7f6438101400 2024-09-12T03:02:24.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.798+0000 7f643c833640 1 --2- 172.21.15.116:0/1421755495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6438106a60 0x7f643819bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:24.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.798+0000 7f64377fe640 1 --2- 172.21.15.116:0/1421755495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6438101400 0x7f6438197920 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6420009230 tx=0x7f6420004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:24.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.798+0000 7f64357fa640 1 -- 172.21.15.116:0/1421755495 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642003f030 con 0x7f6438101400 2024-09-12T03:02:24.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.798+0000 7f64357fa640 1 -- 172.21.15.116:0/1421755495 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6420003ad0 con 0x7f6438101400 2024-09-12T03:02:24.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.798+0000 7f643e296640 1 -- 172.21.15.116:0/1421755495 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f643819c620 con 0x7f6438101400 2024-09-12T03:02:24.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.798+0000 7f64357fa640 1 -- 172.21.15.116:0/1421755495 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6420035070 con 0x7f6438101400 2024-09-12T03:02:24.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.798+0000 7f643e296640 1 -- 172.21.15.116:0/1421755495 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f643819ca70 con 0x7f6438101400 2024-09-12T03:02:24.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.800+0000 7f643e296640 1 -- 172.21.15.116:0/1421755495 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6438100ec0 con 0x7f6438101400 2024-09-12T03:02:24.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.801+0000 7f64357fa640 1 -- 172.21.15.116:0/1421755495 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f6420042030 con 0x7f6438101400 2024-09-12T03:02:24.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.801+0000 7f64357fa640 1 --2- 172.21.15.116:0/1421755495 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f64240a2c10 0x7f64240a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:24.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.802+0000 7f6437fff640 1 --2- 172.21.15.116:0/1421755495 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f64240a2c10 0x7f64240a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:24.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.802+0000 7f64357fa640 1 -- 172.21.15.116:0/1421755495 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f64200ef750 con 0x7f6438101400 2024-09-12T03:02:24.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.802+0000 7f6437fff640 1 --2- 172.21.15.116:0/1421755495 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f64240a2c10 0x7f64240a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6428005fd0 tx=0x7f6428005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:24.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:24.806+0000 7f64357fa640 1 -- 172.21.15.116:0/1421755495 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f64200bc580 con 0x7f6438101400 2024-09-12T03:02:25.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:25.006+0000 7f643e296640 1 -- 172.21.15.116:0/1421755495 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6438108c40 con 0x7f6438101400 2024-09-12T03:02:25.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:25.007+0000 7f64357fa640 1 -- 172.21.15.116:0/1421755495 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11938 (secure 0 0 0) 0x7f64200bbe80 con 0x7f6438101400 2024-09-12T03:02:25.007 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:02:25.007 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":24,"btime":"2024-09-12T03:01:10:397269+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:01:10.218314+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24467,24475,24494]},"id":1}]} 2024-09-12T03:02:25.007 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 24 2024-09-12T03:02:25.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:25.009+0000 7f643e296640 1 -- 172.21.15.116:0/1421755495 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f64240a2c10 msgr2=0x7f64240a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:25.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:25.009+0000 7f643e296640 1 --2- 172.21.15.116:0/1421755495 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f64240a2c10 0x7f64240a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6428005fd0 tx=0x7f6428005850 comp rx=0 tx=0).stop 2024-09-12T03:02:25.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:25.009+0000 7f643e296640 1 -- 172.21.15.116:0/1421755495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6438101400 msgr2=0x7f6438197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:25.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:25.009+0000 7f643e296640 1 --2- 172.21.15.116:0/1421755495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6438101400 0x7f6438197920 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6420009230 tx=0x7f6420004270 comp rx=0 tx=0).stop 2024-09-12T03:02:25.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:25.009+0000 7f643e296640 1 -- 172.21.15.116:0/1421755495 shutdown_connections 2024-09-12T03:02:25.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:25.009+0000 7f643e296640 1 --2- 172.21.15.116:0/1421755495 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f64240a2c10 0x7f64240a50d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:25.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:25.009+0000 7f643e296640 1 --2- 172.21.15.116:0/1421755495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6438106a60 0x7f643819bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:25.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:25.009+0000 7f643e296640 1 --2- 172.21.15.116:0/1421755495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6438101400 0x7f6438197920 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:25.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:25.009+0000 7f643e296640 1 --2- 172.21.15.116:0/1421755495 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6438100a60 0x7f64381973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:25.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:25.009+0000 7f643e296640 1 -- 172.21.15.116:0/1421755495 >> 172.21.15.116:0/1421755495 conn(0x7f64380fc780 msgr2=0x7f64380fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:25.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:25.009+0000 7f643e296640 1 -- 172.21.15.116:0/1421755495 shutdown_connections 2024-09-12T03:02:25.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:25.009+0000 7f643e296640 1 -- 172.21.15.116:0/1421755495 wait complete. 2024-09-12T03:02:25.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:25 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1421755495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:25.697 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T03:00:59.195360+0000', 'modified': '2024-09-12T03:01:10.218314+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24494, 'mds_1': 24475, 'mds_2': 24467}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24443': {'gid': 24443, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.177:6835/2224643300', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6834', 'nonce': 2224643300}, {'type': 'v1', 'addr': '172.21.15.177:6835', 'nonce': 2224643300}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/4028826399', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 4028826399}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 4028826399}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6837/1288373243', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 1288373243}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 1288373243}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.177:6837/1862929607', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6836', 'nonce': 1862929607}, {'type': 'v1', 'addr': '172.21.15.177:6837', 'nonce': 1862929607}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.183:6839/1983839027', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 1983839027}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 1983839027}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.177:6839/2069130361', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6838', 'nonce': 2069130361}, {'type': 'v1', 'addr': '172.21.15.177:6839', 'nonce': 2069130361}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24494, 'qdb_cluster': [24467, 24475, 24494]} 2024-09-12T03:02:25.697 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-12T03:02:25.697 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-12T03:02:25.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:25 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1421755495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:25.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:25 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1421755495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:25.943 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:26.364 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:26 smithi116 ceph-mon[31410]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:02:26.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.444+0000 7fa9f2cdf640 1 -- 172.21.15.116:0/1552825839 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9ec101df0 msgr2=0x7fa9ec1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:26.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.444+0000 7fa9f2cdf640 1 --2- 172.21.15.116:0/1552825839 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9ec101df0 0x7fa9ec1021d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0009900 tx=0x7fa9e002a410 comp rx=0 tx=0).stop 2024-09-12T03:02:26.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.444+0000 7fa9f2cdf640 1 -- 172.21.15.116:0/1552825839 shutdown_connections 2024-09-12T03:02:26.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.444+0000 7fa9f2cdf640 1 --2- 172.21.15.116:0/1552825839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9ec106f20 0x7fa9ec10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.444+0000 7fa9f2cdf640 1 --2- 172.21.15.116:0/1552825839 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9ec1027a0 0x7fa9ec1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.444+0000 7fa9f2cdf640 1 --2- 172.21.15.116:0/1552825839 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9ec101df0 0x7fa9ec1021d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.444+0000 7fa9f2cdf640 1 -- 172.21.15.116:0/1552825839 >> 172.21.15.116:0/1552825839 conn(0x7fa9ec0fd8f0 msgr2=0x7fa9ec0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:26.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.444+0000 7fa9f2cdf640 1 -- 172.21.15.116:0/1552825839 shutdown_connections 2024-09-12T03:02:26.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.444+0000 7fa9f2cdf640 1 -- 172.21.15.116:0/1552825839 wait complete. 2024-09-12T03:02:26.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.445+0000 7fa9f2cdf640 1 Processor -- start 2024-09-12T03:02:26.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.445+0000 7fa9f2cdf640 1 -- start start 2024-09-12T03:02:26.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.445+0000 7fa9f2cdf640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9ec101df0 0x7fa9ec198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:26.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.445+0000 7fa9f2cdf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9ec1027a0 0x7fa9ec198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:26.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.445+0000 7fa9f2cdf640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9ec106f20 0x7fa9ec19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:26.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.445+0000 7fa9f2cdf640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec111fa0 con 0x7fa9ec101df0 2024-09-12T03:02:26.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.445+0000 7fa9f2cdf640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec111e20 con 0x7fa9ec106f20 2024-09-12T03:02:26.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.445+0000 7fa9f2cdf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec112120 con 0x7fa9ec1027a0 2024-09-12T03:02:26.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.445+0000 7fa9f0a7b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9ec101df0 0x7fa9ec198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:26.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.445+0000 7fa9f0a7b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9ec101df0 0x7fa9ec198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32910/0 (socket says 172.21.15.116:32910) 2024-09-12T03:02:26.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.445+0000 7fa9f0a7b640 1 -- 172.21.15.116:0/1743527556 learned_addr learned my addr 172.21.15.116:0/1743527556 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:26.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.445+0000 7fa9ebfff640 1 --2- 172.21.15.116:0/1743527556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9ec1027a0 0x7fa9ec198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:26.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.445+0000 7fa9f127c640 1 --2- 172.21.15.116:0/1743527556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9ec106f20 0x7fa9ec19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:26.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.445+0000 7fa9f0a7b640 1 -- 172.21.15.116:0/1743527556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9ec1027a0 msgr2=0x7fa9ec198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:26.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.445+0000 7fa9f0a7b640 1 --2- 172.21.15.116:0/1743527556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9ec1027a0 0x7fa9ec198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.445+0000 7fa9f0a7b640 1 -- 172.21.15.116:0/1743527556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9ec106f20 msgr2=0x7fa9ec19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:26.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.445+0000 7fa9f0a7b640 1 --2- 172.21.15.116:0/1743527556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9ec106f20 0x7fa9ec19cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.445+0000 7fa9f0a7b640 1 -- 172.21.15.116:0/1743527556 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ec19d3f0 con 0x7fa9ec101df0 2024-09-12T03:02:26.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.445+0000 7fa9ebfff640 1 --2- 172.21.15.116:0/1743527556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9ec1027a0 0x7fa9ec198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:26.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.445+0000 7fa9f0a7b640 1 --2- 172.21.15.116:0/1743527556 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9ec101df0 0x7fa9ec198440 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0005ab0 tx=0x7fa9e00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:26.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.446+0000 7fa9e9ffb640 1 -- 172.21.15.116:0/1743527556 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e003f030 con 0x7fa9ec101df0 2024-09-12T03:02:26.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.446+0000 7fa9f2cdf640 1 -- 172.21.15.116:0/1743527556 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9ec19d680 con 0x7fa9ec101df0 2024-09-12T03:02:26.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.446+0000 7fa9e9ffb640 1 -- 172.21.15.116:0/1743527556 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e0003ce0 con 0x7fa9ec101df0 2024-09-12T03:02:26.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.446+0000 7fa9e9ffb640 1 -- 172.21.15.116:0/1743527556 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0035070 con 0x7fa9ec101df0 2024-09-12T03:02:26.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.446+0000 7fa9f2cdf640 1 -- 172.21.15.116:0/1743527556 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9ec1a4fc0 con 0x7fa9ec101df0 2024-09-12T03:02:26.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.448+0000 7fa9e9ffb640 1 -- 172.21.15.116:0/1743527556 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa9e0042030 con 0x7fa9ec101df0 2024-09-12T03:02:26.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.448+0000 7fa9f50b8740 1 Processor -- start 2024-09-12T03:02:26.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.448+0000 7fa9f50b8740 1 -- start start 2024-09-12T03:02:26.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.449+0000 7fa9e9ffb640 1 --2- 172.21.15.116:0/1743527556 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa9dc0a2c10 0x7fa9dc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:26.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.449+0000 7fa9f50b8740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56244356b9d0 0x56244356bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:26.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.449+0000 7fa9f50b8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56244356c2f0 0x562443576670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:26.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.449+0000 7fa9f50b8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562443576bb0 0x562443578fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:26.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.449+0000 7fa9f50b8740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5624434d2090 con 0x56244356b9d0 2024-09-12T03:02:26.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.449+0000 7fa9f50b8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562443479a70 con 0x56244356c2f0 2024-09-12T03:02:26.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.449+0000 7fa9f50b8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5624434ce050 con 0x562443576bb0 2024-09-12T03:02:26.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.449+0000 7fa9f127c640 1 --2- 172.21.15.116:0/1743527556 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa9dc0a2c10 0x7fa9dc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:26.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.449+0000 7fa9e9ffb640 1 -- 172.21.15.116:0/1743527556 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa9e00efba0 con 0x7fa9ec101df0 2024-09-12T03:02:26.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.449+0000 7fa9f0a7b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56244356b9d0 0x56244356bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:26.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.449+0000 7fa9f0a7b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56244356b9d0 0x56244356bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32926/0 (socket says 172.21.15.116:32926) 2024-09-12T03:02:26.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.449+0000 7fa9f0a7b640 1 -- 172.21.15.116:0/2921602918 learned_addr learned my addr 172.21.15.116:0/2921602918 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:26.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.449+0000 7fa9ebfff640 1 --2- 172.21.15.116:0/2921602918 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56244356c2f0 0x562443576670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:26.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.449+0000 7fa9f127c640 1 --2- 172.21.15.116:0/2921602918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562443576bb0 0x562443578fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:26.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.449+0000 7fa9f0a7b640 1 -- 172.21.15.116:0/2921602918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562443576bb0 msgr2=0x562443578fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:26.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.449+0000 7fa9f0a7b640 1 --2- 172.21.15.116:0/2921602918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562443576bb0 0x562443578fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.449+0000 7fa9f0a7b640 1 -- 172.21.15.116:0/2921602918 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56244356c2f0 msgr2=0x562443576670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:26.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.449+0000 7fa9f0a7b640 1 --2- 172.21.15.116:0/2921602918 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56244356c2f0 0x562443576670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.449+0000 7fa9f0a7b640 1 -- 172.21.15.116:0/2921602918 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562443486130 con 0x56244356b9d0 2024-09-12T03:02:26.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.449+0000 7fa9f127c640 1 --2- 172.21.15.116:0/1743527556 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa9dc0a2c10 0x7fa9dc0a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4014400 tx=0x7fa9e4014800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:26.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.449+0000 7fa9f0a7b640 1 --2- 172.21.15.116:0/2921602918 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56244356b9d0 0x56244356bdb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0005ec0 tx=0x7fa9e002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88609f1bdd5b67e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:26.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.449+0000 7fa9d6ffd640 1 -- 172.21.15.116:0/2921602918 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0049040 con 0x56244356b9d0 2024-09-12T03:02:26.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.449+0000 7fa9d6ffd640 1 -- 172.21.15.116:0/2921602918 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e00bc970 con 0x56244356b9d0 2024-09-12T03:02:26.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.450+0000 7fa9d6ffd640 1 -- 172.21.15.116:0/2921602918 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e00b88e0 con 0x56244356b9d0 2024-09-12T03:02:26.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.450+0000 7fa9f50b8740 1 -- 172.21.15.116:0/2921602918 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56244356b9d0 msgr2=0x56244356bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:26.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.450+0000 7fa9f50b8740 1 --2- 172.21.15.116:0/2921602918 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56244356b9d0 0x56244356bdb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0005ec0 tx=0x7fa9e002a410 comp rx=0 tx=0).stop 2024-09-12T03:02:26.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.450+0000 7fa9f50b8740 1 -- 172.21.15.116:0/2921602918 shutdown_connections 2024-09-12T03:02:26.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.450+0000 7fa9f50b8740 1 --2- 172.21.15.116:0/2921602918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562443576bb0 0x562443578fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.450+0000 7fa9f50b8740 1 --2- 172.21.15.116:0/2921602918 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56244356c2f0 0x562443576670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.450+0000 7fa9f50b8740 1 --2- 172.21.15.116:0/2921602918 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56244356b9d0 0x56244356bdb0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.450+0000 7fa9f50b8740 1 -- 172.21.15.116:0/2921602918 >> 172.21.15.116:0/2921602918 conn(0x562443567f60 msgr2=0x562443568340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:26.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.450+0000 7fa9f50b8740 1 -- 172.21.15.116:0/2921602918 shutdown_connections 2024-09-12T03:02:26.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.450+0000 7fa9f50b8740 1 -- 172.21.15.116:0/2921602918 wait complete. 2024-09-12T03:02:26.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.451+0000 7fa9f50b8740 1 -- start start 2024-09-12T03:02:26.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.451+0000 7fa9f50b8740 1 Processor -- start 2024-09-12T03:02:26.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.451+0000 7fa9f50b8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56244356b9d0 0x5624435764c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:26.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.451+0000 7fa9f50b8740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56244356c2f0 0x562443573810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:26.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.451+0000 7fa9f50b8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562443576bb0 0x562443573d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:26.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.451+0000 7fa9f50b8740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562443579540 con 0x56244356c2f0 2024-09-12T03:02:26.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.451+0000 7fa9f50b8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56244348cf20 con 0x56244356b9d0 2024-09-12T03:02:26.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.451+0000 7fa9f50b8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5624435796c0 con 0x562443576bb0 2024-09-12T03:02:26.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.451+0000 7fa9f50b8740 10 client.0 _ll_register_callbacks cb 0x562443456030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:26.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.451+0000 7fa9ebfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56244356c2f0 0x562443573810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:26.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.451+0000 7fa9ebfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56244356c2f0 0x562443573810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32938/0 (socket says 172.21.15.116:32938) 2024-09-12T03:02:26.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.451+0000 7fa9ebfff640 1 -- 172.21.15.116:0/1342791504 learned_addr learned my addr 172.21.15.116:0/1342791504 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:26.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.451+0000 7fa9f0a7b640 1 --2- 172.21.15.116:0/1342791504 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56244356b9d0 0x5624435764c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:26.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.451+0000 7fa9ebfff640 1 -- 172.21.15.116:0/1342791504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562443576bb0 msgr2=0x562443573d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:26.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.451+0000 7fa9ebfff640 1 --2- 172.21.15.116:0/1342791504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562443576bb0 0x562443573d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.451+0000 7fa9ebfff640 1 -- 172.21.15.116:0/1342791504 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56244356b9d0 msgr2=0x5624435764c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:26.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.451+0000 7fa9ebfff640 1 --2- 172.21.15.116:0/1342791504 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56244356b9d0 0x5624435764c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.451+0000 7fa9ebfff640 1 -- 172.21.15.116:0/1342791504 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624432432b0 con 0x56244356c2f0 2024-09-12T03:02:26.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.452+0000 7fa9ebfff640 1 --2- 172.21.15.116:0/1342791504 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56244356c2f0 0x562443573810 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8011800 tx=0x7fa9d8011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fc9b9bb6a5c0b8d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:26.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.452+0000 7fa9cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:02:26.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.452+0000 7fa9cbfff640 1 -- 172.21.15.116:0/1342791504 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d800f9a0 con 0x56244356c2f0 2024-09-12T03:02:26.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.452+0000 7fa9f50b8740 1 -- 172.21.15.116:0/1342791504 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9e00bc970 con 0x56244356c2f0 2024-09-12T03:02:26.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.452+0000 7fa9cbfff640 1 -- 172.21.15.116:0/1342791504 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa9d800fb40 con 0x56244356c2f0 2024-09-12T03:02:26.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.452+0000 7fa9cbfff640 1 -- 172.21.15.116:0/1342791504 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa9d800fd80 con 0x56244356c2f0 2024-09-12T03:02:26.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.452+0000 7fa9f50b8740 10 client.14997 fetch_fsmap learned FSMap version 24 2024-09-12T03:02:26.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.452+0000 7fa9f50b8740 1 -- 172.21.15.116:0/1342791504 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562443584e90 con 0x56244356c2f0 2024-09-12T03:02:26.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.452+0000 7fa9cbfff640 1 -- 172.21.15.116:0/1342791504 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e00bc970 con 0x56244356c2f0 2024-09-12T03:02:26.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.452+0000 7fa9cbfff640 1 -- 172.21.15.116:0/1342791504 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fa9d8018030 con 0x56244356c2f0 2024-09-12T03:02:26.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.452+0000 7fa9f50b8740 10 client.14997 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:02:26.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.452+0000 7fa9f50b8740 10 client.14997 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T03:02:26.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.452+0000 7fa9f50b8740 20 client.14997 populate_metadata read hostname 'smithi116' 2024-09-12T03:02:26.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.452+0000 7fa9f50b8740 1 --2- 172.21.15.116:0/1342791504 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x562443587540 0x5624435889b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:26.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.452+0000 7fa9f50b8740 4 client.14997 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:02:26.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.452+0000 7fa9f50b8740 1 -- 172.21.15.116:0/1342791504 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624434df600 con 0x562443587540 2024-09-12T03:02:26.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.453+0000 7fa9f0a7b640 1 --2- 172.21.15.116:0/1342791504 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x562443587540 0x5624435889b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:26.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.453+0000 7fa9f0a7b640 1 --2- 172.21.15.116:0/1342791504 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x562443587540 0x5624435889b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4b04dee92d7afde6 server_cookie=4a07efa875efd2be in_seq=0 out_seq=0 2024-09-12T03:02:26.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.453+0000 7fa9cbfff640 10 client.14997 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:02:26.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.454+0000 7fa9cbfff640 1 -- 172.21.15.116:0/1342791504 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5624434df600 con 0x562443587540 2024-09-12T03:02:26.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.454+0000 7fa9cbfff640 10 client.14997 handle_command_reply: tid=0 2024-09-12T03:02:26.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.454+0000 7fa9f50b8740 1 client.14997 shutdown 2024-09-12T03:02:26.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.454+0000 7fa9f50b8740 1 -- 172.21.15.116:0/1342791504 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56244356c2f0 msgr2=0x562443573810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:26.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.454+0000 7fa9f50b8740 1 --2- 172.21.15.116:0/1342791504 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56244356c2f0 0x562443573810 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8011800 tx=0x7fa9d8011cc0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.454+0000 7fa9f50b8740 1 -- 172.21.15.116:0/1342791504 shutdown_connections 2024-09-12T03:02:26.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.454+0000 7fa9f50b8740 1 --2- 172.21.15.116:0/1342791504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562443576bb0 0x562443573d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.454+0000 7fa9f50b8740 1 --2- 172.21.15.116:0/1342791504 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x562443587540 0x5624435889b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.454+0000 7fa9f50b8740 1 --2- 172.21.15.116:0/1342791504 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56244356c2f0 0x562443573810 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.454+0000 7fa9cbfff640 0 client.14997 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:02:26.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.454+0000 7fa9f50b8740 1 --2- 172.21.15.116:0/1342791504 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56244356b9d0 0x5624435764c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.454+0000 7fa9f50b8740 1 -- 172.21.15.116:0/1342791504 >> 172.21.15.116:0/1342791504 conn(0x562443567f60 msgr2=0x56244356dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:26.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.454+0000 7fa9f50b8740 1 -- 172.21.15.116:0/1342791504 shutdown_connections 2024-09-12T03:02:26.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.454+0000 7fa9f50b8740 1 -- 172.21.15.116:0/1342791504 wait complete. 2024-09-12T03:02:26.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.454+0000 7fa9f50b8740 20 client.14997 trim_cache size 0 max 16384 2024-09-12T03:02:26.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.472+0000 7fa9f50b8740 1 Processor -- start 2024-09-12T03:02:26.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.472+0000 7fa9f50b8740 1 -- start start 2024-09-12T03:02:26.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.472+0000 7fa9f50b8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56244356b9d0 0x562443582590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:26.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.472+0000 7fa9f50b8740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56244356c2f0 0x562443590c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:26.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.472+0000 7fa9f50b8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562443576bb0 0x5624435911d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:26.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.472+0000 7fa9f50b8740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562443586cf0 con 0x56244356c2f0 2024-09-12T03:02:26.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.472+0000 7fa9f50b8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5624435873c0 con 0x562443576bb0 2024-09-12T03:02:26.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.472+0000 7fa9f50b8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562443587a90 con 0x56244356b9d0 2024-09-12T03:02:26.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.472+0000 7fa9ebfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56244356c2f0 0x562443590c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:26.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.472+0000 7fa9ebfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56244356c2f0 0x562443590c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32940/0 (socket says 172.21.15.116:32940) 2024-09-12T03:02:26.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.472+0000 7fa9ebfff640 1 -- 172.21.15.116:0/1094062620 learned_addr learned my addr 172.21.15.116:0/1094062620 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:26.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.472+0000 7fa9ebfff640 1 -- 172.21.15.116:0/1094062620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56244356b9d0 msgr2=0x562443582590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:26.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.472+0000 7fa9ebfff640 1 --2- 172.21.15.116:0/1094062620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56244356b9d0 0x562443582590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.472+0000 7fa9ebfff640 1 -- 172.21.15.116:0/1094062620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562443576bb0 msgr2=0x5624435911d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:26.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.472+0000 7fa9ebfff640 1 --2- 172.21.15.116:0/1094062620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562443576bb0 0x5624435911d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.472+0000 7fa9ebfff640 1 -- 172.21.15.116:0/1094062620 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562443582ad0 con 0x56244356c2f0 2024-09-12T03:02:26.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.472+0000 7fa9ebfff640 1 --2- 172.21.15.116:0/1094062620 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56244356c2f0 0x562443590c90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8004d00 tx=0x7fa9d80087c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c72ebcd4fe26f15f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:26.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.472+0000 7fa9d67fc640 1 -- 172.21.15.116:0/1094062620 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8004320 con 0x56244356c2f0 2024-09-12T03:02:26.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.472+0000 7fa9d67fc640 1 -- 172.21.15.116:0/1094062620 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9d800f9a0 con 0x56244356c2f0 2024-09-12T03:02:26.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.472+0000 7fa9d67fc640 1 -- 172.21.15.116:0/1094062620 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8005550 con 0x56244356c2f0 2024-09-12T03:02:26.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.473+0000 7fa9f50b8740 1 -- 172.21.15.116:0/1094062620 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56244356c2f0 msgr2=0x562443590c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:26.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.473+0000 7fa9f50b8740 1 --2- 172.21.15.116:0/1094062620 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56244356c2f0 0x562443590c90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8004d00 tx=0x7fa9d80087c0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.473+0000 7fa9f50b8740 1 -- 172.21.15.116:0/1094062620 shutdown_connections 2024-09-12T03:02:26.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.473+0000 7fa9f50b8740 1 --2- 172.21.15.116:0/1094062620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562443576bb0 0x5624435911d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.473+0000 7fa9f50b8740 1 --2- 172.21.15.116:0/1094062620 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56244356c2f0 0x562443590c90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.473+0000 7fa9f50b8740 1 --2- 172.21.15.116:0/1094062620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56244356b9d0 0x562443582590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.473+0000 7fa9f50b8740 1 -- 172.21.15.116:0/1094062620 >> 172.21.15.116:0/1094062620 conn(0x562443567f60 msgr2=0x562443568340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:26.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.473+0000 7fa9f50b8740 1 -- 172.21.15.116:0/1094062620 shutdown_connections 2024-09-12T03:02:26.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.473+0000 7fa9f50b8740 1 -- 172.21.15.116:0/1094062620 wait complete. 2024-09-12T03:02:26.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.473+0000 7fa9f50b8740 1 -- start start 2024-09-12T03:02:26.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.474+0000 7fa9f50b8740 1 Processor -- start 2024-09-12T03:02:26.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.474+0000 7fa9f50b8740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56244356b9d0 0x56244358c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:26.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.474+0000 7fa9f50b8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56244356c2f0 0x56244358cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:26.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.474+0000 7fa9f50b8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562443576bb0 0x56244358d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:26.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.474+0000 7fa9f50b8740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562443588830 con 0x56244356b9d0 2024-09-12T03:02:26.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.474+0000 7fa9f50b8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562443588160 con 0x56244356c2f0 2024-09-12T03:02:26.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.474+0000 7fa9f50b8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562443588f00 con 0x562443576bb0 2024-09-12T03:02:26.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.474+0000 7fa9f50b8740 10 client.0 _ll_register_callbacks cb 0x7fa9d8011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:26.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.474+0000 7fa9f0a7b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56244356b9d0 0x56244358c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:26.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.474+0000 7fa9f0a7b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56244356b9d0 0x56244358c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32944/0 (socket says 172.21.15.116:32944) 2024-09-12T03:02:26.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.474+0000 7fa9f0a7b640 1 -- 172.21.15.116:0/1473591057 learned_addr learned my addr 172.21.15.116:0/1473591057 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:26.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.474+0000 7fa9ebfff640 1 --2- 172.21.15.116:0/1473591057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56244356c2f0 0x56244358cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:26.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.474+0000 7fa9f0a7b640 1 -- 172.21.15.116:0/1473591057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562443576bb0 msgr2=0x56244358d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:26.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.474+0000 7fa9f127c640 1 --2- 172.21.15.116:0/1473591057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562443576bb0 0x56244358d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:26.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.474+0000 7fa9f0a7b640 1 --2- 172.21.15.116:0/1473591057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562443576bb0 0x56244358d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.474+0000 7fa9f0a7b640 1 -- 172.21.15.116:0/1473591057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56244356c2f0 msgr2=0x56244358cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:26.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.474+0000 7fa9f0a7b640 1 --2- 172.21.15.116:0/1473591057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56244356c2f0 0x56244358cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.474+0000 7fa9f0a7b640 1 -- 172.21.15.116:0/1473591057 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624435f0c50 con 0x56244356b9d0 2024-09-12T03:02:26.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.475+0000 7fa9f0a7b640 1 --2- 172.21.15.116:0/1473591057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56244356b9d0 0x56244358c9a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0000c00 tx=0x7fa9e00ed4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f915acbc17c6e11 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:26.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.475+0000 7fa9cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:02:26.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.475+0000 7fa9cbfff640 1 -- 172.21.15.116:0/1473591057 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0049040 con 0x56244356b9d0 2024-09-12T03:02:26.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.475+0000 7fa9cbfff640 1 -- 172.21.15.116:0/1473591057 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa9e00b88e0 con 0x56244356b9d0 2024-09-12T03:02:26.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.475+0000 7fa9f50b8740 1 -- 172.21.15.116:0/1473591057 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9d800f9a0 con 0x56244356b9d0 2024-09-12T03:02:26.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.475+0000 7fa9cbfff640 1 -- 172.21.15.116:0/1473591057 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa9e00c0620 con 0x56244356b9d0 2024-09-12T03:02:26.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.475+0000 7fa9f50b8740 10 client.15003 fetch_fsmap learned FSMap version 24 2024-09-12T03:02:26.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.475+0000 7fa9f50b8740 1 -- 172.21.15.116:0/1473591057 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5624435f1f00 con 0x56244356b9d0 2024-09-12T03:02:26.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.475+0000 7fa9cbfff640 1 -- 172.21.15.116:0/1473591057 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9d800f9a0 con 0x56244356b9d0 2024-09-12T03:02:26.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.475+0000 7fa9cbfff640 1 -- 172.21.15.116:0/1473591057 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fa9e00dc110 con 0x56244356b9d0 2024-09-12T03:02:26.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.475+0000 7fa9f50b8740 10 client.15003 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:02:26.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.475+0000 7fa9f50b8740 10 client.15003 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T03:02:26.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.475+0000 7fa9f50b8740 20 client.15003 populate_metadata read hostname 'smithi116' 2024-09-12T03:02:26.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.475+0000 7fa9f50b8740 1 --2- 172.21.15.116:0/1473591057 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5624435f69c0 0x5624435f56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:26.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.475+0000 7fa9f50b8740 4 client.15003 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:02:26.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.475+0000 7fa9f50b8740 1 -- 172.21.15.116:0/1473591057 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "status"}) -- 0x5624434d9b10 con 0x5624435f69c0 2024-09-12T03:02:26.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.476+0000 7fa9ebfff640 1 --2- 172.21.15.116:0/1473591057 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5624435f69c0 0x5624435f56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:26.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.476+0000 7fa9ebfff640 1 --2- 172.21.15.116:0/1473591057 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5624435f69c0 0x5624435f56d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1a0e1242bb69ecbf server_cookie=1092649733caab4c in_seq=0 out_seq=0 2024-09-12T03:02:26.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.476+0000 7fa9cbfff640 10 client.15003 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:02:26.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.476+0000 7fa9cbfff640 1 -- 172.21.15.116:0/1473591057 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5624434d9b10 con 0x5624435f69c0 2024-09-12T03:02:26.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.476+0000 7fa9cbfff640 10 client.15003 handle_command_reply: tid=0 2024-09-12T03:02:26.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.476+0000 7fa9f50b8740 1 client.15003 shutdown 2024-09-12T03:02:26.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.476+0000 7fa9f50b8740 1 -- 172.21.15.116:0/1473591057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56244356b9d0 msgr2=0x56244358c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:26.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.476+0000 7fa9f50b8740 1 --2- 172.21.15.116:0/1473591057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56244356b9d0 0x56244358c9a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0000c00 tx=0x7fa9e00ed4e0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.476+0000 7fa9f50b8740 1 -- 172.21.15.116:0/1473591057 shutdown_connections 2024-09-12T03:02:26.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.476+0000 7fa9f50b8740 1 --2- 172.21.15.116:0/1473591057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562443576bb0 0x56244358d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.476+0000 7fa9f50b8740 1 --2- 172.21.15.116:0/1473591057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56244356c2f0 0x56244358cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.476+0000 7fa9f50b8740 1 --2- 172.21.15.116:0/1473591057 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5624435f69c0 0x5624435f56d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.476+0000 7fa9f50b8740 1 --2- 172.21.15.116:0/1473591057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56244356b9d0 0x56244358c9a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.476+0000 7fa9f50b8740 1 -- 172.21.15.116:0/1473591057 >> 172.21.15.116:0/1473591057 conn(0x562443567f60 msgr2=0x5624435667e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:26.514 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:02:26.514 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_fsid": "eb1ac602-70b1-11ef-bcea-c7b262605968", 2024-09-12T03:02:26.515 INFO:teuthology.orchestra.run.smithi116.stdout: "whoami": 2, 2024-09-12T03:02:26.515 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 24467, 2024-09-12T03:02:26.515 INFO:teuthology.orchestra.run.smithi116.stdout: "want_state": "up:active", 2024-09-12T03:02:26.515 INFO:teuthology.orchestra.run.smithi116.stdout: "state": "up:active", 2024-09-12T03:02:26.515 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs", 2024-09-12T03:02:26.515 INFO:teuthology.orchestra.run.smithi116.stdout: "rank_uptime": 79.087070800000006, 2024-09-12T03:02:26.515 INFO:teuthology.orchestra.run.smithi116.stdout: "mdsmap_epoch": 24, 2024-09-12T03:02:26.515 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch": 80, 2024-09-12T03:02:26.515 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch_barrier": 78, 2024-09-12T03:02:26.515 INFO:teuthology.orchestra.run.smithi116.stdout: "uptime": 109.298973696 2024-09-12T03:02:26.515 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:02:26.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.476+0000 7fa9cbfff640 0 client.15003 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:02:26.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.477+0000 7fa9f50b8740 1 -- 172.21.15.116:0/1473591057 shutdown_connections 2024-09-12T03:02:26.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.477+0000 7fa9f50b8740 1 -- 172.21.15.116:0/1473591057 wait complete. 2024-09-12T03:02:26.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.477+0000 7fa9f50b8740 20 client.15003 trim_cache size 0 max 16384 2024-09-12T03:02:26.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.477+0000 7fa9f2cdf640 1 -- 172.21.15.116:0/1743527556 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa9dc0a2c10 msgr2=0x7fa9dc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:26.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.477+0000 7fa9f2cdf640 1 --2- 172.21.15.116:0/1743527556 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa9dc0a2c10 0x7fa9dc0a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4014400 tx=0x7fa9e4014800 comp rx=0 tx=0).stop 2024-09-12T03:02:26.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.477+0000 7fa9f2cdf640 1 -- 172.21.15.116:0/1743527556 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9ec101df0 msgr2=0x7fa9ec198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:26.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.477+0000 7fa9f2cdf640 1 --2- 172.21.15.116:0/1743527556 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9ec101df0 0x7fa9ec198440 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0005ab0 tx=0x7fa9e00042f0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.477+0000 7fa9f2cdf640 1 -- 172.21.15.116:0/1743527556 shutdown_connections 2024-09-12T03:02:26.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.477+0000 7fa9f2cdf640 1 --2- 172.21.15.116:0/1743527556 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa9dc0a2c10 0x7fa9dc0a50d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.477+0000 7fa9f2cdf640 1 --2- 172.21.15.116:0/1743527556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9ec106f20 0x7fa9ec19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.477+0000 7fa9f2cdf640 1 --2- 172.21.15.116:0/1743527556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9ec1027a0 0x7fa9ec198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.477+0000 7fa9f2cdf640 1 --2- 172.21.15.116:0/1743527556 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9ec101df0 0x7fa9ec198440 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:26.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.477+0000 7fa9f2cdf640 1 -- 172.21.15.116:0/1743527556 >> 172.21.15.116:0/1743527556 conn(0x7fa9ec0fd8f0 msgr2=0x7fa9ec0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:26.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.477+0000 7fa9f2cdf640 1 -- 172.21.15.116:0/1743527556 shutdown_connections 2024-09-12T03:02:26.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:26.477+0000 7fa9f2cdf640 1 -- 172.21.15.116:0/1743527556 wait complete. 2024-09-12T03:02:26.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:26 smithi183 ceph-mon[35535]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:02:26.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:26 smithi177 ceph-mon[35305]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:02:27.265 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T03:02:27.532 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:28.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.031+0000 7f07c4371640 1 -- 172.21.15.116:0/317420880 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07bc106f20 msgr2=0x7f07bc10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:28.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.031+0000 7f07c4371640 1 --2- 172.21.15.116:0/317420880 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07bc106f20 0x7f07bc10eaa0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f07b8009ca0 tx=0x7f07b802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.031+0000 7f07c4371640 1 -- 172.21.15.116:0/317420880 shutdown_connections 2024-09-12T03:02:28.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.031+0000 7f07c4371640 1 --2- 172.21.15.116:0/317420880 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07bc106f20 0x7f07bc10eaa0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.031+0000 7f07c4371640 1 --2- 172.21.15.116:0/317420880 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07bc1027a0 0x7f07bc1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.031+0000 7f07c4371640 1 --2- 172.21.15.116:0/317420880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07bc101df0 0x7f07bc1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.031+0000 7f07c4371640 1 -- 172.21.15.116:0/317420880 >> 172.21.15.116:0/317420880 conn(0x7f07bc0fd8f0 msgr2=0x7f07bc0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:28.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.031+0000 7f07c4371640 1 -- 172.21.15.116:0/317420880 shutdown_connections 2024-09-12T03:02:28.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.031+0000 7f07c4371640 1 -- 172.21.15.116:0/317420880 wait complete. 2024-09-12T03:02:28.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.032+0000 7f07c4371640 1 Processor -- start 2024-09-12T03:02:28.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.032+0000 7f07c4371640 1 -- start start 2024-09-12T03:02:28.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.032+0000 7f07c4371640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07bc101df0 0x7f07bc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:28.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.032+0000 7f07c4371640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07bc1027a0 0x7f07bc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:28.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.032+0000 7f07c4371640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07bc106f20 0x7f07bc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:28.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.032+0000 7f07c4371640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc111f80 con 0x7f07bc106f20 2024-09-12T03:02:28.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.032+0000 7f07c4371640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc111e00 con 0x7f07bc101df0 2024-09-12T03:02:28.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.032+0000 7f07c4371640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc112100 con 0x7f07bc1027a0 2024-09-12T03:02:28.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.032+0000 7f07c290e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07bc106f20 0x7f07bc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:28.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.032+0000 7f07c290e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07bc106f20 0x7f07bc074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32948/0 (socket says 172.21.15.116:32948) 2024-09-12T03:02:28.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.032+0000 7f07c290e640 1 -- 172.21.15.116:0/2668981537 learned_addr learned my addr 172.21.15.116:0/2668981537 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:28.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.032+0000 7f07c190c640 1 --2- 172.21.15.116:0/2668981537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07bc1027a0 0x7f07bc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:28.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.032+0000 7f07c210d640 1 --2- 172.21.15.116:0/2668981537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07bc101df0 0x7f07bc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:28.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.033+0000 7f07c290e640 1 -- 172.21.15.116:0/2668981537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07bc1027a0 msgr2=0x7f07bc074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:28.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.033+0000 7f07c290e640 1 --2- 172.21.15.116:0/2668981537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07bc1027a0 0x7f07bc074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.033+0000 7f07c290e640 1 -- 172.21.15.116:0/2668981537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07bc101df0 msgr2=0x7f07bc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:28.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.033+0000 7f07c290e640 1 --2- 172.21.15.116:0/2668981537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07bc101df0 0x7f07bc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.033+0000 7f07c290e640 1 -- 172.21.15.116:0/2668981537 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07bc1aadd0 con 0x7f07bc106f20 2024-09-12T03:02:28.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.033+0000 7f07c290e640 1 --2- 172.21.15.116:0/2668981537 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07bc106f20 0x7f07bc074f70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f07b8031a20 tx=0x7f07b8031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:28.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.033+0000 7f07b37fe640 1 -- 172.21.15.116:0/2668981537 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8003c30 con 0x7f07bc106f20 2024-09-12T03:02:28.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.033+0000 7f07c4371640 1 -- 172.21.15.116:0/2668981537 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07bc1aafa0 con 0x7f07bc106f20 2024-09-12T03:02:28.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.033+0000 7f07b37fe640 1 -- 172.21.15.116:0/2668981537 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07b8003dd0 con 0x7f07bc106f20 2024-09-12T03:02:28.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.033+0000 7f07b37fe640 1 -- 172.21.15.116:0/2668981537 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b802fcf0 con 0x7f07bc106f20 2024-09-12T03:02:28.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.033+0000 7f07c4371640 1 -- 172.21.15.116:0/2668981537 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07bc1ab500 con 0x7f07bc106f20 2024-09-12T03:02:28.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.035+0000 7f07b37fe640 1 -- 172.21.15.116:0/2668981537 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f07b8004050 con 0x7f07bc106f20 2024-09-12T03:02:28.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.036+0000 7f07b37fe640 1 --2- 172.21.15.116:0/2668981537 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f07a80a2c10 0x7f07a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:28.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.036+0000 7f07c210d640 1 --2- 172.21.15.116:0/2668981537 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f07a80a2c10 0x7f07a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:28.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.036+0000 7f07b37fe640 1 -- 172.21.15.116:0/2668981537 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f07b80f0330 con 0x7f07bc106f20 2024-09-12T03:02:28.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.036+0000 7f07c210d640 1 --2- 172.21.15.116:0/2668981537 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f07a80a2c10 0x7f07a80a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f07b40058c0 tx=0x7f07b400a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:28.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.036+0000 7f07c674a740 1 Processor -- start 2024-09-12T03:02:28.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.037+0000 7f07c674a740 1 -- start start 2024-09-12T03:02:28.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.037+0000 7f07c674a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1e02ea9d0 0x55b1e02eadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:28.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.037+0000 7f07c674a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1e02eb2f0 0x55b1e02f5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:28.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.037+0000 7f07c674a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b1e02f5bb0 0x55b1e02f7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:28.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.037+0000 7f07c674a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b1e0251090 con 0x55b1e02f5bb0 2024-09-12T03:02:28.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.037+0000 7f07c674a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b1e01f8a70 con 0x55b1e02ea9d0 2024-09-12T03:02:28.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.037+0000 7f07c674a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b1e024d050 con 0x55b1e02eb2f0 2024-09-12T03:02:28.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.037+0000 7f07c290e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b1e02f5bb0 0x55b1e02f7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:28.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.037+0000 7f07c290e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b1e02f5bb0 0x55b1e02f7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32950/0 (socket says 172.21.15.116:32950) 2024-09-12T03:02:28.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.037+0000 7f07c290e640 1 -- 172.21.15.116:0/3457554160 learned_addr learned my addr 172.21.15.116:0/3457554160 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:28.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.037+0000 7f07c190c640 1 --2- 172.21.15.116:0/3457554160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1e02eb2f0 0x55b1e02f5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:28.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.037+0000 7f07c210d640 1 --2- 172.21.15.116:0/3457554160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1e02ea9d0 0x55b1e02eadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:28.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.037+0000 7f07c290e640 1 -- 172.21.15.116:0/3457554160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1e02eb2f0 msgr2=0x55b1e02f5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:28.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.037+0000 7f07c290e640 1 --2- 172.21.15.116:0/3457554160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1e02eb2f0 0x55b1e02f5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.037+0000 7f07c290e640 1 -- 172.21.15.116:0/3457554160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1e02ea9d0 msgr2=0x55b1e02eadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:28.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.037+0000 7f07c290e640 1 --2- 172.21.15.116:0/3457554160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1e02ea9d0 0x55b1e02eadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.037+0000 7f07c290e640 1 -- 172.21.15.116:0/3457554160 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1e0205130 con 0x55b1e02f5bb0 2024-09-12T03:02:28.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.037+0000 7f07c210d640 1 --2- 172.21.15.116:0/3457554160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1e02ea9d0 0x55b1e02eadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:28.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.038+0000 7f07c290e640 1 --2- 172.21.15.116:0/3457554160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b1e02f5bb0 0x55b1e02f7fa0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f07b8005e30 tx=0x7f07b80b8fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b90e7fe9e9aeaa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:28.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.038+0000 7f07b0dea640 1 -- 172.21.15.116:0/3457554160 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b802fe90 con 0x55b1e02f5bb0 2024-09-12T03:02:28.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.038+0000 7f07b0dea640 1 -- 172.21.15.116:0/3457554160 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07b8036040 con 0x55b1e02f5bb0 2024-09-12T03:02:28.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.038+0000 7f07b0dea640 1 -- 172.21.15.116:0/3457554160 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b80b9270 con 0x55b1e02f5bb0 2024-09-12T03:02:28.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.038+0000 7f07c674a740 1 -- 172.21.15.116:0/3457554160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b1e02f5bb0 msgr2=0x55b1e02f7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:28.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.038+0000 7f07c674a740 1 --2- 172.21.15.116:0/3457554160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b1e02f5bb0 0x55b1e02f7fa0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f07b8005e30 tx=0x7f07b80b8fd0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.039+0000 7f07c674a740 1 -- 172.21.15.116:0/3457554160 shutdown_connections 2024-09-12T03:02:28.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.039+0000 7f07c674a740 1 --2- 172.21.15.116:0/3457554160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b1e02f5bb0 0x55b1e02f7fa0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.039+0000 7f07c674a740 1 --2- 172.21.15.116:0/3457554160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1e02eb2f0 0x55b1e02f5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.039+0000 7f07c674a740 1 --2- 172.21.15.116:0/3457554160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1e02ea9d0 0x55b1e02eadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.039+0000 7f07c674a740 1 -- 172.21.15.116:0/3457554160 >> 172.21.15.116:0/3457554160 conn(0x55b1e02e6f60 msgr2=0x55b1e02e7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:28.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.039+0000 7f07c674a740 1 -- 172.21.15.116:0/3457554160 shutdown_connections 2024-09-12T03:02:28.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.039+0000 7f07c674a740 1 -- 172.21.15.116:0/3457554160 wait complete. 2024-09-12T03:02:28.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.039+0000 7f07c674a740 1 -- start start 2024-09-12T03:02:28.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.039+0000 7f07c674a740 1 Processor -- start 2024-09-12T03:02:28.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.039+0000 7f07c674a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1e02ea9d0 0x55b1e02f54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:28.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.039+0000 7f07c674a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b1e02eb2f0 0x55b1e02f27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:28.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.039+0000 7f07c674a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1e02f5bb0 0x55b1e02f2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:28.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.039+0000 7f07c674a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b1e0251090 con 0x55b1e02eb2f0 2024-09-12T03:02:28.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.039+0000 7f07c674a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b1e020bf20 con 0x55b1e02f5bb0 2024-09-12T03:02:28.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.039+0000 7f07c674a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b1e02f8540 con 0x55b1e02ea9d0 2024-09-12T03:02:28.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.039+0000 7f07c674a740 10 client.0 _ll_register_callbacks cb 0x55b1e01d5030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:28.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.040+0000 7f07c190c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b1e02eb2f0 0x55b1e02f27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:28.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.040+0000 7f07c190c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b1e02eb2f0 0x55b1e02f27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32964/0 (socket says 172.21.15.116:32964) 2024-09-12T03:02:28.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.040+0000 7f07c190c640 1 -- 172.21.15.116:0/4260376299 learned_addr learned my addr 172.21.15.116:0/4260376299 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:28.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.040+0000 7f07c210d640 1 --2- 172.21.15.116:0/4260376299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1e02ea9d0 0x55b1e02f54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:28.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.040+0000 7f07c290e640 1 --2- 172.21.15.116:0/4260376299 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1e02f5bb0 0x55b1e02f2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:28.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.040+0000 7f07c190c640 1 -- 172.21.15.116:0/4260376299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1e02ea9d0 msgr2=0x55b1e02f54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:28.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.040+0000 7f07c190c640 1 --2- 172.21.15.116:0/4260376299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1e02ea9d0 0x55b1e02f54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.040+0000 7f07c190c640 1 -- 172.21.15.116:0/4260376299 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1e02f5bb0 msgr2=0x55b1e02f2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:28.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.040+0000 7f07c190c640 1 --2- 172.21.15.116:0/4260376299 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1e02f5bb0 0x55b1e02f2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.040+0000 7f07c190c640 1 -- 172.21.15.116:0/4260376299 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1dffc22b0 con 0x55b1e02eb2f0 2024-09-12T03:02:28.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.040+0000 7f07c210d640 1 --2- 172.21.15.116:0/4260376299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1e02ea9d0 0x55b1e02f54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:28.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.040+0000 7f07c190c640 1 --2- 172.21.15.116:0/4260376299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b1e02eb2f0 0x55b1e02f27f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f07ac00b9f0 tx=0x7f07ac00beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a57a3d2b62c19bb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:28.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.040+0000 7f07a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:02:28.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.040+0000 7f07a5ffb640 1 -- 172.21.15.116:0/4260376299 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac003e20 con 0x55b1e02eb2f0 2024-09-12T03:02:28.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.040+0000 7f07a5ffb640 1 -- 172.21.15.116:0/4260376299 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f07ac005020 con 0x55b1e02eb2f0 2024-09-12T03:02:28.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.040+0000 7f07c674a740 1 -- 172.21.15.116:0/4260376299 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07b8036040 con 0x55b1e02eb2f0 2024-09-12T03:02:28.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.040+0000 7f07a5ffb640 1 -- 172.21.15.116:0/4260376299 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f07ac005220 con 0x55b1e02eb2f0 2024-09-12T03:02:28.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.040+0000 7f07c674a740 10 client.15015 fetch_fsmap learned FSMap version 24 2024-09-12T03:02:28.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.040+0000 7f07c674a740 1 -- 172.21.15.116:0/4260376299 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b1e0303d00 con 0x55b1e02eb2f0 2024-09-12T03:02:28.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.040+0000 7f07a5ffb640 1 -- 172.21.15.116:0/4260376299 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07b8036040 con 0x55b1e02eb2f0 2024-09-12T03:02:28.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.041+0000 7f07a5ffb640 1 -- 172.21.15.116:0/4260376299 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f07ac01b630 con 0x55b1e02eb2f0 2024-09-12T03:02:28.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.041+0000 7f07c674a740 10 client.15015 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:02:28.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.041+0000 7f07c674a740 10 client.15015 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T03:02:28.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.041+0000 7f07c674a740 20 client.15015 populate_metadata read hostname 'smithi116' 2024-09-12T03:02:28.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.041+0000 7f07c674a740 1 --2- 172.21.15.116:0/4260376299 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55b1e0306320 0x55b1e0307790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:28.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.041+0000 7f07c674a740 4 client.15015 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:02:28.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.041+0000 7f07c674a740 1 -- 172.21.15.116:0/4260376299 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1e025e600 con 0x55b1e0306320 2024-09-12T03:02:28.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.041+0000 7f07c210d640 1 --2- 172.21.15.116:0/4260376299 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55b1e0306320 0x55b1e0307790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:28.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.041+0000 7f07a5ffb640 10 client.15015 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:02:28.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.041+0000 7f07c210d640 1 --2- 172.21.15.116:0/4260376299 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55b1e0306320 0x55b1e0307790 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a7478f95f92bfc23 server_cookie=7aa4d733cdc79714 in_seq=0 out_seq=0 2024-09-12T03:02:28.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.042+0000 7f07a5ffb640 1 -- 172.21.15.116:0/4260376299 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b1e025e600 con 0x55b1e0306320 2024-09-12T03:02:28.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.042+0000 7f07a5ffb640 10 client.15015 handle_command_reply: tid=0 2024-09-12T03:02:28.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.042+0000 7f07c674a740 1 client.15015 shutdown 2024-09-12T03:02:28.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.042+0000 7f07c674a740 1 -- 172.21.15.116:0/4260376299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b1e02eb2f0 msgr2=0x55b1e02f27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:28.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.042+0000 7f07c674a740 1 --2- 172.21.15.116:0/4260376299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b1e02eb2f0 0x55b1e02f27f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f07ac00b9f0 tx=0x7f07ac00beb0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.043+0000 7f07c674a740 1 -- 172.21.15.116:0/4260376299 shutdown_connections 2024-09-12T03:02:28.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.043+0000 7f07c674a740 1 --2- 172.21.15.116:0/4260376299 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55b1e0306320 0x55b1e0307790 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.043+0000 7f07c674a740 1 --2- 172.21.15.116:0/4260376299 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1e02f5bb0 0x55b1e02f2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.043+0000 7f07c674a740 1 --2- 172.21.15.116:0/4260376299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b1e02eb2f0 0x55b1e02f27f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.043+0000 7f07c674a740 1 --2- 172.21.15.116:0/4260376299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1e02ea9d0 0x55b1e02f54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.043+0000 7f07c674a740 1 -- 172.21.15.116:0/4260376299 >> 172.21.15.116:0/4260376299 conn(0x55b1e02e6f60 msgr2=0x55b1e02ecc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:28.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.043+0000 7f07a5ffb640 0 client.15015 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:02:28.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.043+0000 7f07c674a740 1 -- 172.21.15.116:0/4260376299 shutdown_connections 2024-09-12T03:02:28.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.043+0000 7f07c674a740 1 -- 172.21.15.116:0/4260376299 wait complete. 2024-09-12T03:02:28.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.043+0000 7f07c674a740 20 client.15015 trim_cache size 0 max 16384 2024-09-12T03:02:28.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.060+0000 7f07c674a740 1 Processor -- start 2024-09-12T03:02:28.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.060+0000 7f07c674a740 1 -- start start 2024-09-12T03:02:28.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.060+0000 7f07c674a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1e02ea9d0 0x55b1e0301590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:28.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.060+0000 7f07c674a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b1e02eb2f0 0x55b1e02f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:28.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.060+0000 7f07c674a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1e02f5bb0 0x55b1e02f4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:28.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.060+0000 7f07c674a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b1e0305ad0 con 0x55b1e02eb2f0 2024-09-12T03:02:28.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.060+0000 7f07c674a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b1e03061a0 con 0x55b1e02f5bb0 2024-09-12T03:02:28.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.060+0000 7f07c674a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b1e0306870 con 0x55b1e02ea9d0 2024-09-12T03:02:28.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.061+0000 7f07c190c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b1e02eb2f0 0x55b1e02f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:28.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.061+0000 7f07c190c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b1e02eb2f0 0x55b1e02f3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32970/0 (socket says 172.21.15.116:32970) 2024-09-12T03:02:28.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.061+0000 7f07c190c640 1 -- 172.21.15.116:0/119307607 learned_addr learned my addr 172.21.15.116:0/119307607 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:28.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.061+0000 7f07c210d640 1 --2- 172.21.15.116:0/119307607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1e02ea9d0 0x55b1e0301590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:28.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.061+0000 7f07c190c640 1 -- 172.21.15.116:0/119307607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1e02ea9d0 msgr2=0x55b1e0301590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:28.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.061+0000 7f07c190c640 1 --2- 172.21.15.116:0/119307607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1e02ea9d0 0x55b1e0301590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.061+0000 7f07c190c640 1 -- 172.21.15.116:0/119307607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1e02f5bb0 msgr2=0x55b1e02f4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:28.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.061+0000 7f07c290e640 1 --2- 172.21.15.116:0/119307607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1e02f5bb0 0x55b1e02f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:28.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.061+0000 7f07c190c640 1 --2- 172.21.15.116:0/119307607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1e02f5bb0 0x55b1e02f4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.061+0000 7f07c190c640 1 -- 172.21.15.116:0/119307607 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1e02ea800 con 0x55b1e02eb2f0 2024-09-12T03:02:28.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.061+0000 7f07c190c640 1 --2- 172.21.15.116:0/119307607 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b1e02eb2f0 0x55b1e02f3f40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f07ac01baa0 tx=0x7f07ac0051b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c686d69a650e2a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:28.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.061+0000 7f07a7fff640 1 -- 172.21.15.116:0/119307607 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac003e20 con 0x55b1e02eb2f0 2024-09-12T03:02:28.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.061+0000 7f07a7fff640 1 -- 172.21.15.116:0/119307607 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07ac0053c0 con 0x55b1e02eb2f0 2024-09-12T03:02:28.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.061+0000 7f07a7fff640 1 -- 172.21.15.116:0/119307607 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac004de0 con 0x55b1e02eb2f0 2024-09-12T03:02:28.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.062+0000 7f07c674a740 1 -- 172.21.15.116:0/119307607 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b1e02eb2f0 msgr2=0x55b1e02f3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:28.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.062+0000 7f07c674a740 1 --2- 172.21.15.116:0/119307607 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b1e02eb2f0 0x55b1e02f3f40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f07ac01baa0 tx=0x7f07ac0051b0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.062+0000 7f07c674a740 1 -- 172.21.15.116:0/119307607 shutdown_connections 2024-09-12T03:02:28.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.062+0000 7f07c674a740 1 --2- 172.21.15.116:0/119307607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1e02f5bb0 0x55b1e02f4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.062+0000 7f07c674a740 1 --2- 172.21.15.116:0/119307607 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b1e02eb2f0 0x55b1e02f3f40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.062+0000 7f07c674a740 1 --2- 172.21.15.116:0/119307607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1e02ea9d0 0x55b1e0301590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.062+0000 7f07c674a740 1 -- 172.21.15.116:0/119307607 >> 172.21.15.116:0/119307607 conn(0x55b1e02e6f60 msgr2=0x55b1e02e7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:28.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.062+0000 7f07c674a740 1 -- 172.21.15.116:0/119307607 shutdown_connections 2024-09-12T03:02:28.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.062+0000 7f07c674a740 1 -- 172.21.15.116:0/119307607 wait complete. 2024-09-12T03:02:28.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.062+0000 7f07c674a740 1 -- start start 2024-09-12T03:02:28.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.063+0000 7f07c674a740 1 Processor -- start 2024-09-12T03:02:28.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.063+0000 7f07c674a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b1e02ea9d0 0x55b1e030fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:28.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.063+0000 7f07c674a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1e02eb2f0 0x55b1e0310300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:28.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.063+0000 7f07c674a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1e02f5bb0 0x55b1e02ef4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:28.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.063+0000 7f07c674a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b1e0307610 con 0x55b1e02ea9d0 2024-09-12T03:02:28.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.063+0000 7f07c674a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b1e0306f40 con 0x55b1e02f5bb0 2024-09-12T03:02:28.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.063+0000 7f07c674a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b1e0307ce0 con 0x55b1e02eb2f0 2024-09-12T03:02:28.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.063+0000 7f07c210d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b1e02ea9d0 0x55b1e030fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:28.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.063+0000 7f07c210d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b1e02ea9d0 0x55b1e030fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32972/0 (socket says 172.21.15.116:32972) 2024-09-12T03:02:28.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.063+0000 7f07c210d640 1 -- 172.21.15.116:0/553347914 learned_addr learned my addr 172.21.15.116:0/553347914 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:28.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.063+0000 7f07c674a740 10 client.0 _ll_register_callbacks cb 0x7f07ac004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:28.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.063+0000 7f07c190c640 1 --2- 172.21.15.116:0/553347914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1e02eb2f0 0x55b1e0310300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:28.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.063+0000 7f07c210d640 1 -- 172.21.15.116:0/553347914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1e02eb2f0 msgr2=0x55b1e0310300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:28.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.063+0000 7f07c210d640 1 --2- 172.21.15.116:0/553347914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1e02eb2f0 0x55b1e0310300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.063+0000 7f07c210d640 1 -- 172.21.15.116:0/553347914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1e02f5bb0 msgr2=0x55b1e02ef4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:28.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.063+0000 7f07c290e640 1 --2- 172.21.15.116:0/553347914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1e02f5bb0 0x55b1e02ef4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:28.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.063+0000 7f07c210d640 1 --2- 172.21.15.116:0/553347914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1e02f5bb0 0x55b1e02ef4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.063+0000 7f07c210d640 1 -- 172.21.15.116:0/553347914 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1e02efd10 con 0x55b1e02ea9d0 2024-09-12T03:02:28.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.063+0000 7f07c290e640 1 --2- 172.21.15.116:0/553347914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1e02f5bb0 0x55b1e02ef4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:28.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.063+0000 7f07c210d640 1 --2- 172.21.15.116:0/553347914 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b1e02ea9d0 0x55b1e030fdc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f07b4010910 tx=0x7f07b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e7bf639feff6ae server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:28.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.063+0000 7f07a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:02:28.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.064+0000 7f07a5ffb640 1 -- 172.21.15.116:0/553347914 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b4016070 con 0x55b1e02ea9d0 2024-09-12T03:02:28.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.064+0000 7f07c674a740 1 -- 172.21.15.116:0/553347914 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07ac0053c0 con 0x55b1e02ea9d0 2024-09-12T03:02:28.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.064+0000 7f07a5ffb640 1 -- 172.21.15.116:0/553347914 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f07b4028790 con 0x55b1e02ea9d0 2024-09-12T03:02:28.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.064+0000 7f07a5ffb640 1 -- 172.21.15.116:0/553347914 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f07b4028990 con 0x55b1e02ea9d0 2024-09-12T03:02:28.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.064+0000 7f07c674a740 10 client.15021 fetch_fsmap learned FSMap version 24 2024-09-12T03:02:28.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.064+0000 7f07c674a740 1 -- 172.21.15.116:0/553347914 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b1e030e5d0 con 0x55b1e02ea9d0 2024-09-12T03:02:28.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.064+0000 7f07a5ffb640 1 -- 172.21.15.116:0/553347914 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07ac0053c0 con 0x55b1e02ea9d0 2024-09-12T03:02:28.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.064+0000 7f07a5ffb640 1 -- 172.21.15.116:0/553347914 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f07b402f070 con 0x55b1e02ea9d0 2024-09-12T03:02:28.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.064+0000 7f07c674a740 10 client.15021 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:02:28.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.064+0000 7f07c674a740 10 client.15021 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T03:02:28.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.064+0000 7f07c674a740 20 client.15021 populate_metadata read hostname 'smithi116' 2024-09-12T03:02:28.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.064+0000 7f07c674a740 1 --2- 172.21.15.116:0/553347914 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55b1e03753b0 0x55b1e03740c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:28.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.064+0000 7f07c674a740 4 client.15021 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:02:28.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.064+0000 7f07c674a740 1 -- 172.21.15.116:0/553347914 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "status"}) -- 0x55b1e0258b10 con 0x55b1e03753b0 2024-09-12T03:02:28.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.064+0000 7f07c190c640 1 --2- 172.21.15.116:0/553347914 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55b1e03753b0 0x55b1e03740c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:28.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.065+0000 7f07c190c640 1 --2- 172.21.15.116:0/553347914 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55b1e03753b0 0x55b1e03740c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f755eec8781cdf7 server_cookie=daf429de3c0466e6 in_seq=0 out_seq=0 2024-09-12T03:02:28.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.065+0000 7f07a5ffb640 10 client.15021 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:02:28.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.065+0000 7f07a5ffb640 1 -- 172.21.15.116:0/553347914 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b1e0258b10 con 0x55b1e03753b0 2024-09-12T03:02:28.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.065+0000 7f07a5ffb640 10 client.15021 handle_command_reply: tid=0 2024-09-12T03:02:28.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.065+0000 7f07c674a740 1 client.15021 shutdown 2024-09-12T03:02:28.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.065+0000 7f07c674a740 1 -- 172.21.15.116:0/553347914 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b1e02ea9d0 msgr2=0x55b1e030fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:28.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.065+0000 7f07c674a740 1 --2- 172.21.15.116:0/553347914 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b1e02ea9d0 0x55b1e030fdc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f07b4010910 tx=0x7f07b4004730 comp rx=0 tx=0).stop 2024-09-12T03:02:28.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.065+0000 7f07c674a740 1 -- 172.21.15.116:0/553347914 shutdown_connections 2024-09-12T03:02:28.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.065+0000 7f07c674a740 1 --2- 172.21.15.116:0/553347914 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55b1e03753b0 0x55b1e03740c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.065+0000 7f07c674a740 1 --2- 172.21.15.116:0/553347914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b1e02f5bb0 0x55b1e02ef4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.065+0000 7f07c674a740 1 --2- 172.21.15.116:0/553347914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b1e02eb2f0 0x55b1e0310300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.065+0000 7f07c674a740 1 --2- 172.21.15.116:0/553347914 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b1e02ea9d0 0x55b1e030fdc0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.065+0000 7f07c674a740 1 -- 172.21.15.116:0/553347914 >> 172.21.15.116:0/553347914 conn(0x55b1e02e6f60 msgr2=0x55b1e02e57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:28.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.065+0000 7f07a5ffb640 0 client.15021 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:02:28.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.065+0000 7f07c674a740 1 -- 172.21.15.116:0/553347914 shutdown_connections 2024-09-12T03:02:28.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.065+0000 7f07c674a740 1 -- 172.21.15.116:0/553347914 wait complete. 2024-09-12T03:02:28.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.066+0000 7f07c674a740 20 client.15021 trim_cache size 0 max 16384 2024-09-12T03:02:28.073 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:02:28.073 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_fsid": "eb1ac602-70b1-11ef-bcea-c7b262605968", 2024-09-12T03:02:28.073 INFO:teuthology.orchestra.run.smithi116.stdout: "whoami": 1, 2024-09-12T03:02:28.073 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 24475, 2024-09-12T03:02:28.073 INFO:teuthology.orchestra.run.smithi116.stdout: "want_state": "up:active", 2024-09-12T03:02:28.073 INFO:teuthology.orchestra.run.smithi116.stdout: "state": "up:active", 2024-09-12T03:02:28.073 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs", 2024-09-12T03:02:28.073 INFO:teuthology.orchestra.run.smithi116.stdout: "rank_uptime": 81.684335211999993, 2024-09-12T03:02:28.073 INFO:teuthology.orchestra.run.smithi116.stdout: "mdsmap_epoch": 24, 2024-09-12T03:02:28.073 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch": 80, 2024-09-12T03:02:28.074 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch_barrier": 78, 2024-09-12T03:02:28.074 INFO:teuthology.orchestra.run.smithi116.stdout: "uptime": 108.232808276 2024-09-12T03:02:28.074 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:02:28.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.066+0000 7f07c4371640 1 -- 172.21.15.116:0/2668981537 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f07a80a2c10 msgr2=0x7f07a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:28.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.066+0000 7f07c4371640 1 --2- 172.21.15.116:0/2668981537 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f07a80a2c10 0x7f07a80a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f07b40058c0 tx=0x7f07b400a250 comp rx=0 tx=0).stop 2024-09-12T03:02:28.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.066+0000 7f07c4371640 1 -- 172.21.15.116:0/2668981537 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07bc106f20 msgr2=0x7f07bc074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:28.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.066+0000 7f07c4371640 1 --2- 172.21.15.116:0/2668981537 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07bc106f20 0x7f07bc074f70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f07b8031a20 tx=0x7f07b8031a50 comp rx=0 tx=0).stop 2024-09-12T03:02:28.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.066+0000 7f07c4371640 1 -- 172.21.15.116:0/2668981537 shutdown_connections 2024-09-12T03:02:28.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.066+0000 7f07c4371640 1 --2- 172.21.15.116:0/2668981537 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f07a80a2c10 0x7f07a80a50d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.066+0000 7f07c4371640 1 --2- 172.21.15.116:0/2668981537 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07bc106f20 0x7f07bc074f70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.066+0000 7f07c4371640 1 --2- 172.21.15.116:0/2668981537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07bc1027a0 0x7f07bc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.066+0000 7f07c4371640 1 --2- 172.21.15.116:0/2668981537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07bc101df0 0x7f07bc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.066+0000 7f07c4371640 1 -- 172.21.15.116:0/2668981537 >> 172.21.15.116:0/2668981537 conn(0x7f07bc0fd8f0 msgr2=0x7f07bc0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:28.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.066+0000 7f07c4371640 1 -- 172.21.15.116:0/2668981537 shutdown_connections 2024-09-12T03:02:28.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:28.066+0000 7f07c4371640 1 -- 172.21.15.116:0/2668981537 wait complete. 2024-09-12T03:02:28.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:28 smithi116 ceph-mon[31410]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:02:28.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:28 smithi183 ceph-mon[35535]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:02:28.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:28 smithi177 ceph-mon[35305]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:02:28.780 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T03:02:29.023 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:29.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.497+0000 7f646c25f640 1 -- 172.21.15.116:0/3999632272 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6464102790 msgr2=0x7f64641067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:29.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.497+0000 7f646c25f640 1 --2- 172.21.15.116:0/3999632272 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6464102790 0x7f64641067e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6454009900 tx=0x7f645402a410 comp rx=0 tx=0).stop 2024-09-12T03:02:29.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.497+0000 7f646c25f640 1 -- 172.21.15.116:0/3999632272 shutdown_connections 2024-09-12T03:02:29.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.497+0000 7f646c25f640 1 --2- 172.21.15.116:0/3999632272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6464106f10 0x7f646410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.497+0000 7f646c25f640 1 --2- 172.21.15.116:0/3999632272 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6464102790 0x7f64641067e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.497+0000 7f646c25f640 1 --2- 172.21.15.116:0/3999632272 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6464101de0 0x7f64641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.497+0000 7f646c25f640 1 -- 172.21.15.116:0/3999632272 >> 172.21.15.116:0/3999632272 conn(0x7f64640fd880 msgr2=0x7f64640ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:29.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.497+0000 7f646c25f640 1 -- 172.21.15.116:0/3999632272 shutdown_connections 2024-09-12T03:02:29.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.497+0000 7f646c25f640 1 -- 172.21.15.116:0/3999632272 wait complete. 2024-09-12T03:02:29.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.498+0000 7f646c25f640 1 Processor -- start 2024-09-12T03:02:29.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.498+0000 7f646c25f640 1 -- start start 2024-09-12T03:02:29.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.498+0000 7f646c25f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6464101de0 0x7f64640744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:29.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.498+0000 7f646c25f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6464102790 0x7f6464074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:29.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.498+0000 7f646c25f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6464106f10 0x7f6464074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:29.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.498+0000 7f646c25f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6464111fc0 con 0x7f6464102790 2024-09-12T03:02:29.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.498+0000 7f646c25f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6464111e40 con 0x7f6464101de0 2024-09-12T03:02:29.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.498+0000 7f646c25f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6464112140 con 0x7f6464106f10 2024-09-12T03:02:29.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.498+0000 7f64697fa640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6464102790 0x7f6464074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:29.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.498+0000 7f64697fa640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6464102790 0x7f6464074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32992/0 (socket says 172.21.15.116:32992) 2024-09-12T03:02:29.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.498+0000 7f64697fa640 1 -- 172.21.15.116:0/1945622054 learned_addr learned my addr 172.21.15.116:0/1945622054 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:29.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.498+0000 7f6469ffb640 1 --2- 172.21.15.116:0/1945622054 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6464101de0 0x7f64640744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:29.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.498+0000 7f646a7fc640 1 --2- 172.21.15.116:0/1945622054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6464106f10 0x7f6464074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:29.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.498+0000 7f64697fa640 1 -- 172.21.15.116:0/1945622054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6464106f10 msgr2=0x7f6464074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:29.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.498+0000 7f64697fa640 1 --2- 172.21.15.116:0/1945622054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6464106f10 0x7f6464074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.498+0000 7f64697fa640 1 -- 172.21.15.116:0/1945622054 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6464101de0 msgr2=0x7f64640744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:29.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.498+0000 7f64697fa640 1 --2- 172.21.15.116:0/1945622054 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6464101de0 0x7f64640744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.498+0000 7f64697fa640 1 -- 172.21.15.116:0/1945622054 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64641a68e0 con 0x7f6464102790 2024-09-12T03:02:29.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.499+0000 7f64697fa640 1 --2- 172.21.15.116:0/1945622054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6464102790 0x7f6464074a30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6454004770 tx=0x7f6454003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:29.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.499+0000 7f645affd640 1 -- 172.21.15.116:0/1945622054 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645403f070 con 0x7f6464102790 2024-09-12T03:02:29.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.499+0000 7f646c25f640 1 -- 172.21.15.116:0/1945622054 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64641a6b70 con 0x7f6464102790 2024-09-12T03:02:29.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.499+0000 7f645affd640 1 -- 172.21.15.116:0/1945622054 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6454031ca0 con 0x7f6464102790 2024-09-12T03:02:29.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.499+0000 7f645affd640 1 -- 172.21.15.116:0/1945622054 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6454035030 con 0x7f6464102790 2024-09-12T03:02:29.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.499+0000 7f646c25f640 1 -- 172.21.15.116:0/1945622054 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64641a7050 con 0x7f6464102790 2024-09-12T03:02:29.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.501+0000 7f645affd640 1 -- 172.21.15.116:0/1945622054 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f6454042070 con 0x7f6464102790 2024-09-12T03:02:29.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.501+0000 7f646e638740 1 Processor -- start 2024-09-12T03:02:29.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.502+0000 7f646e638740 1 -- start start 2024-09-12T03:02:29.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.502+0000 7f646e638740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6631ca9d0 0x55b6631cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:29.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.502+0000 7f646e638740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6631cb2f0 0x55b6631d5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:29.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.502+0000 7f646e638740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6631d5bb0 0x55b6631d7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:29.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.502+0000 7f646e638740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b663131090 con 0x55b6631ca9d0 2024-09-12T03:02:29.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.502+0000 7f646e638740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b6630d8a70 con 0x55b6631d5bb0 2024-09-12T03:02:29.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.502+0000 7f646e638740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b66312d050 con 0x55b6631cb2f0 2024-09-12T03:02:29.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.502+0000 7f645affd640 1 --2- 172.21.15.116:0/1945622054 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f64500a2c10 0x7f64500a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:29.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.502+0000 7f646a7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6631ca9d0 0x55b6631cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:29.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.502+0000 7f646a7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6631ca9d0 0x55b6631cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33004/0 (socket says 172.21.15.116:33004) 2024-09-12T03:02:29.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.502+0000 7f646a7fc640 1 -- 172.21.15.116:0/344514512 learned_addr learned my addr 172.21.15.116:0/344514512 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:29.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.502+0000 7f6469ffb640 1 --2- 172.21.15.116:0/344514512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6631cb2f0 0x55b6631d5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:29.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.502+0000 7f645affd640 1 -- 172.21.15.116:0/1945622054 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f64540ef920 con 0x7f6464102790 2024-09-12T03:02:29.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.502+0000 7f646a7fc640 1 --2- 172.21.15.116:0/1945622054 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f64500a2c10 0x7f64500a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:29.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.502+0000 7f64697fa640 1 --2- 172.21.15.116:0/344514512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6631d5bb0 0x55b6631d7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:29.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.502+0000 7f646a7fc640 1 -- 172.21.15.116:0/344514512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6631cb2f0 msgr2=0x55b6631d5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:29.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.502+0000 7f646a7fc640 1 --2- 172.21.15.116:0/344514512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6631cb2f0 0x55b6631d5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.502+0000 7f646a7fc640 1 -- 172.21.15.116:0/344514512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6631d5bb0 msgr2=0x55b6631d7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:29.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.502+0000 7f646a7fc640 1 --2- 172.21.15.116:0/344514512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6631d5bb0 0x55b6631d7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.502+0000 7f646a7fc640 1 -- 172.21.15.116:0/344514512 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6630e5130 con 0x55b6631ca9d0 2024-09-12T03:02:29.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.502+0000 7f64697fa640 1 --2- 172.21.15.116:0/344514512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6631d5bb0 0x55b6631d7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:29.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.502+0000 7f646a7fc640 1 --2- 172.21.15.116:0/344514512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6631ca9d0 0x55b6631cadb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f645c013940 tx=0x7f645c013e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec986ea364b16a96 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:29.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.503+0000 7f644ffff640 1 -- 172.21.15.116:0/344514512 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645c021030 con 0x55b6631ca9d0 2024-09-12T03:02:29.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.503+0000 7f644ffff640 1 -- 172.21.15.116:0/344514512 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f645c0043a0 con 0x55b6631ca9d0 2024-09-12T03:02:29.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.503+0000 7f644ffff640 1 -- 172.21.15.116:0/344514512 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645c01dad0 con 0x55b6631ca9d0 2024-09-12T03:02:29.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.503+0000 7f646e638740 1 -- 172.21.15.116:0/344514512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6631ca9d0 msgr2=0x55b6631cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:29.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.503+0000 7f646e638740 1 --2- 172.21.15.116:0/344514512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6631ca9d0 0x55b6631cadb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f645c013940 tx=0x7f645c013e00 comp rx=0 tx=0).stop 2024-09-12T03:02:29.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.503+0000 7f646a7fc640 1 --2- 172.21.15.116:0/1945622054 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f64500a2c10 0x7f64500a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f645c021000 tx=0x7f645c020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:29.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.503+0000 7f646e638740 1 -- 172.21.15.116:0/344514512 shutdown_connections 2024-09-12T03:02:29.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.503+0000 7f646e638740 1 --2- 172.21.15.116:0/344514512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6631d5bb0 0x55b6631d7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.503+0000 7f646e638740 1 --2- 172.21.15.116:0/344514512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6631cb2f0 0x55b6631d5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.503+0000 7f646e638740 1 --2- 172.21.15.116:0/344514512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6631ca9d0 0x55b6631cadb0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.503+0000 7f646e638740 1 -- 172.21.15.116:0/344514512 >> 172.21.15.116:0/344514512 conn(0x55b6631c6f60 msgr2=0x55b6631c7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:29.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.504+0000 7f646e638740 1 -- 172.21.15.116:0/344514512 shutdown_connections 2024-09-12T03:02:29.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.504+0000 7f646e638740 1 -- 172.21.15.116:0/344514512 wait complete. 2024-09-12T03:02:29.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.504+0000 7f646e638740 1 -- start start 2024-09-12T03:02:29.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.504+0000 7f646e638740 1 Processor -- start 2024-09-12T03:02:29.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.504+0000 7f646e638740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6631ca9d0 0x55b6631d5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:29.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.504+0000 7f646e638740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6631cb2f0 0x55b6631d27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:29.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.504+0000 7f646e638740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6631d5bb0 0x55b6631d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:29.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.504+0000 7f646e638740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b6631d8780 con 0x55b6631cb2f0 2024-09-12T03:02:29.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.504+0000 7f646e638740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b6630ebf20 con 0x55b6631d5bb0 2024-09-12T03:02:29.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.504+0000 7f646e638740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b6631d8900 con 0x55b6631ca9d0 2024-09-12T03:02:29.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.505+0000 7f646e638740 10 client.0 _ll_register_callbacks cb 0x55b6630b5030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:29.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.505+0000 7f64697fa640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6631cb2f0 0x55b6631d27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:29.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.505+0000 7f64697fa640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6631cb2f0 0x55b6631d27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33010/0 (socket says 172.21.15.116:33010) 2024-09-12T03:02:29.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.505+0000 7f64697fa640 1 -- 172.21.15.116:0/2758596122 learned_addr learned my addr 172.21.15.116:0/2758596122 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:29.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.505+0000 7f646a7fc640 1 --2- 172.21.15.116:0/2758596122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6631d5bb0 0x55b6631d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:29.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.505+0000 7f6469ffb640 1 --2- 172.21.15.116:0/2758596122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6631ca9d0 0x55b6631d5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:29.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.505+0000 7f64697fa640 1 -- 172.21.15.116:0/2758596122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6631ca9d0 msgr2=0x55b6631d5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:29.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.505+0000 7f64697fa640 1 --2- 172.21.15.116:0/2758596122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6631ca9d0 0x55b6631d5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.505+0000 7f64697fa640 1 -- 172.21.15.116:0/2758596122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6631d5bb0 msgr2=0x55b6631d2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:29.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.505+0000 7f64697fa640 1 --2- 172.21.15.116:0/2758596122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6631d5bb0 0x55b6631d2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.505+0000 7f64697fa640 1 -- 172.21.15.116:0/2758596122 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b662ea22b0 con 0x55b6631cb2f0 2024-09-12T03:02:29.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.505+0000 7f64697fa640 1 --2- 172.21.15.116:0/2758596122 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6631cb2f0 0x55b6631d27b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f64540bc030 tx=0x7f64540bba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1276ceae2247fd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:29.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.505+0000 7f644d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:02:29.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.505+0000 7f644d7fa640 1 -- 172.21.15.116:0/2758596122 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6454049040 con 0x55b6631cb2f0 2024-09-12T03:02:29.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.505+0000 7f644d7fa640 1 -- 172.21.15.116:0/2758596122 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f64540ed4f0 con 0x55b6631cb2f0 2024-09-12T03:02:29.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.505+0000 7f646e638740 1 -- 172.21.15.116:0/2758596122 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f645c0043a0 con 0x55b6631cb2f0 2024-09-12T03:02:29.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.505+0000 7f644d7fa640 1 -- 172.21.15.116:0/2758596122 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f64540c4c60 con 0x55b6631cb2f0 2024-09-12T03:02:29.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.505+0000 7f646e638740 10 client.15033 fetch_fsmap learned FSMap version 24 2024-09-12T03:02:29.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.505+0000 7f646e638740 1 -- 172.21.15.116:0/2758596122 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b6631e4010 con 0x55b6631cb2f0 2024-09-12T03:02:29.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.505+0000 7f644d7fa640 1 -- 172.21.15.116:0/2758596122 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f645c0043a0 con 0x55b6631cb2f0 2024-09-12T03:02:29.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.506+0000 7f644d7fa640 1 -- 172.21.15.116:0/2758596122 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f64540c47d0 con 0x55b6631cb2f0 2024-09-12T03:02:29.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.506+0000 7f646e638740 10 client.15033 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:02:29.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.506+0000 7f646e638740 10 client.15033 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:02:29.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.506+0000 7f646e638740 20 client.15033 populate_metadata read hostname 'smithi116' 2024-09-12T03:02:29.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.506+0000 7f646e638740 1 --2- 172.21.15.116:0/2758596122 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b6631e6670 0x55b6631e7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:29.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.506+0000 7f646e638740 4 client.15033 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:02:29.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.506+0000 7f646e638740 1 -- 172.21.15.116:0/2758596122 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b66313e600 con 0x55b6631e6670 2024-09-12T03:02:29.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.506+0000 7f6469ffb640 1 --2- 172.21.15.116:0/2758596122 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b6631e6670 0x55b6631e7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:29.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.507+0000 7f6469ffb640 1 --2- 172.21.15.116:0/2758596122 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b6631e6670 0x55b6631e7ae0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1e2e2520568255c server_cookie=b6383c977345d265 in_seq=0 out_seq=0 2024-09-12T03:02:29.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.507+0000 7f644d7fa640 10 client.15033 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:02:29.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.507+0000 7f644d7fa640 1 -- 172.21.15.116:0/2758596122 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b66313e600 con 0x55b6631e6670 2024-09-12T03:02:29.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.507+0000 7f644d7fa640 10 client.15033 handle_command_reply: tid=0 2024-09-12T03:02:29.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.507+0000 7f646e638740 1 client.15033 shutdown 2024-09-12T03:02:29.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.507+0000 7f646e638740 1 -- 172.21.15.116:0/2758596122 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6631cb2f0 msgr2=0x55b6631d27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:29.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.507+0000 7f646e638740 1 --2- 172.21.15.116:0/2758596122 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6631cb2f0 0x55b6631d27b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f64540bc030 tx=0x7f64540bba00 comp rx=0 tx=0).stop 2024-09-12T03:02:29.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.508+0000 7f646e638740 1 -- 172.21.15.116:0/2758596122 shutdown_connections 2024-09-12T03:02:29.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.508+0000 7f646e638740 1 --2- 172.21.15.116:0/2758596122 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b6631e6670 0x55b6631e7ae0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.508+0000 7f646e638740 1 --2- 172.21.15.116:0/2758596122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6631d5bb0 0x55b6631d2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.508+0000 7f646e638740 1 --2- 172.21.15.116:0/2758596122 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6631cb2f0 0x55b6631d27b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.508+0000 7f646e638740 1 --2- 172.21.15.116:0/2758596122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6631ca9d0 0x55b6631d5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.508+0000 7f644d7fa640 0 client.15033 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:02:29.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.508+0000 7f646e638740 1 -- 172.21.15.116:0/2758596122 >> 172.21.15.116:0/2758596122 conn(0x55b6631c6f60 msgr2=0x55b6631ccc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:29.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.508+0000 7f646e638740 1 -- 172.21.15.116:0/2758596122 shutdown_connections 2024-09-12T03:02:29.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.508+0000 7f646e638740 1 -- 172.21.15.116:0/2758596122 wait complete. 2024-09-12T03:02:29.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.508+0000 7f646e638740 20 client.15033 trim_cache size 0 max 16384 2024-09-12T03:02:29.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.526+0000 7f646e638740 1 Processor -- start 2024-09-12T03:02:29.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.526+0000 7f646e638740 1 -- start start 2024-09-12T03:02:29.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.526+0000 7f646e638740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6631ca9d0 0x55b6631e6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:29.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.526+0000 7f646e638740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6631cb2f0 0x55b6631d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:29.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.526+0000 7f646e638740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6631d5bb0 0x55b6631d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:29.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.526+0000 7f646e638740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b6631e5e20 con 0x55b6631d5bb0 2024-09-12T03:02:29.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.526+0000 7f646e638740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b6631e64f0 con 0x55b6631ca9d0 2024-09-12T03:02:29.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.526+0000 7f646e638740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b6631e6bc0 con 0x55b6631cb2f0 2024-09-12T03:02:29.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.526+0000 7f646a7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6631d5bb0 0x55b6631d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:29.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.526+0000 7f646a7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6631d5bb0 0x55b6631d1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33022/0 (socket says 172.21.15.116:33022) 2024-09-12T03:02:29.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.526+0000 7f646a7fc640 1 -- 172.21.15.116:0/77625653 learned_addr learned my addr 172.21.15.116:0/77625653 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:29.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.526+0000 7f646a7fc640 1 -- 172.21.15.116:0/77625653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6631cb2f0 msgr2=0x55b6631d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:29.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.526+0000 7f646a7fc640 1 --2- 172.21.15.116:0/77625653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6631cb2f0 0x55b6631d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.526+0000 7f646a7fc640 1 -- 172.21.15.116:0/77625653 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6631ca9d0 msgr2=0x55b6631e6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:29.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.526+0000 7f646a7fc640 1 --2- 172.21.15.116:0/77625653 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6631ca9d0 0x55b6631e6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.526+0000 7f646a7fc640 1 -- 172.21.15.116:0/77625653 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6631ca800 con 0x55b6631d5bb0 2024-09-12T03:02:29.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.527+0000 7f646a7fc640 1 --2- 172.21.15.116:0/77625653 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6631d5bb0 0x55b6631d1640 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f645c012010 tx=0x7f645c01d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2782af114b880eb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:29.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.527+0000 7f644f7fe640 1 -- 172.21.15.116:0/77625653 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645c01d7e0 con 0x55b6631d5bb0 2024-09-12T03:02:29.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.527+0000 7f644f7fe640 1 -- 172.21.15.116:0/77625653 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f645c033070 con 0x55b6631d5bb0 2024-09-12T03:02:29.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.527+0000 7f644f7fe640 1 -- 172.21.15.116:0/77625653 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645c003ec0 con 0x55b6631d5bb0 2024-09-12T03:02:29.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.527+0000 7f646e638740 1 -- 172.21.15.116:0/77625653 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6631d5bb0 msgr2=0x55b6631d1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:29.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.527+0000 7f646e638740 1 --2- 172.21.15.116:0/77625653 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6631d5bb0 0x55b6631d1640 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f645c012010 tx=0x7f645c01d6b0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.527+0000 7f646e638740 1 -- 172.21.15.116:0/77625653 shutdown_connections 2024-09-12T03:02:29.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.527+0000 7f646e638740 1 --2- 172.21.15.116:0/77625653 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6631d5bb0 0x55b6631d1640 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.527+0000 7f646e638740 1 --2- 172.21.15.116:0/77625653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6631cb2f0 0x55b6631d1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.527+0000 7f646e638740 1 --2- 172.21.15.116:0/77625653 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6631ca9d0 0x55b6631e6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.527+0000 7f646e638740 1 -- 172.21.15.116:0/77625653 >> 172.21.15.116:0/77625653 conn(0x55b6631c6f60 msgr2=0x55b6631e5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:29.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.527+0000 7f646e638740 1 -- 172.21.15.116:0/77625653 shutdown_connections 2024-09-12T03:02:29.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.527+0000 7f646e638740 1 -- 172.21.15.116:0/77625653 wait complete. 2024-09-12T03:02:29.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.528+0000 7f646e638740 1 -- start start 2024-09-12T03:02:29.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.528+0000 7f646e638740 1 Processor -- start 2024-09-12T03:02:29.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.528+0000 7f646e638740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6631ca9d0 0x55b6631f0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:29.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.528+0000 7f646e638740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6631cb2f0 0x55b6631eb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:29.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.528+0000 7f646e638740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6631d5bb0 0x55b6631ebe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:29.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.528+0000 7f646e638740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b6631e7960 con 0x55b6631d5bb0 2024-09-12T03:02:29.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.528+0000 7f646e638740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b6631e7290 con 0x55b6631cb2f0 2024-09-12T03:02:29.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.528+0000 7f646e638740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b6631e8030 con 0x55b6631ca9d0 2024-09-12T03:02:29.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.528+0000 7f646e638740 10 client.0 _ll_register_callbacks cb 0x7f64540b8380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:29.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.528+0000 7f646a7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6631d5bb0 0x55b6631ebe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:29.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.528+0000 7f646a7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6631d5bb0 0x55b6631ebe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33030/0 (socket says 172.21.15.116:33030) 2024-09-12T03:02:29.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.528+0000 7f646a7fc640 1 -- 172.21.15.116:0/2077025435 learned_addr learned my addr 172.21.15.116:0/2077025435 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:29.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.529+0000 7f6469ffb640 1 --2- 172.21.15.116:0/2077025435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6631ca9d0 0x55b6631f0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:29.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.529+0000 7f64697fa640 1 --2- 172.21.15.116:0/2077025435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6631cb2f0 0x55b6631eb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:29.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.529+0000 7f646a7fc640 1 -- 172.21.15.116:0/2077025435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6631ca9d0 msgr2=0x55b6631f0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:29.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.529+0000 7f646a7fc640 1 --2- 172.21.15.116:0/2077025435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6631ca9d0 0x55b6631f0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.529+0000 7f646a7fc640 1 -- 172.21.15.116:0/2077025435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6631cb2f0 msgr2=0x55b6631eb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:29.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.529+0000 7f646a7fc640 1 --2- 172.21.15.116:0/2077025435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6631cb2f0 0x55b6631eb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.529+0000 7f646a7fc640 1 -- 172.21.15.116:0/2077025435 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6631c7bf0 con 0x55b6631d5bb0 2024-09-12T03:02:29.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.529+0000 7f64697fa640 1 --2- 172.21.15.116:0/2077025435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6631cb2f0 0x55b6631eb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:29.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.529+0000 7f646a7fc640 1 --2- 172.21.15.116:0/2077025435 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6631d5bb0 0x55b6631ebe10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f645c01d650 tx=0x7f645c013490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=810e51daa208c895 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:29.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.529+0000 7f644d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:02:29.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.529+0000 7f644d7fa640 1 -- 172.21.15.116:0/2077025435 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645c01d9b0 con 0x55b6631d5bb0 2024-09-12T03:02:29.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.529+0000 7f644d7fa640 1 -- 172.21.15.116:0/2077025435 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f645c004a00 con 0x55b6631d5bb0 2024-09-12T03:02:29.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.529+0000 7f646e638740 1 -- 172.21.15.116:0/2077025435 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f645c033070 con 0x55b6631d5bb0 2024-09-12T03:02:29.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.529+0000 7f644d7fa640 1 -- 172.21.15.116:0/2077025435 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f645c003ec0 con 0x55b6631d5bb0 2024-09-12T03:02:29.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.529+0000 7f646e638740 10 client.15039 fetch_fsmap learned FSMap version 24 2024-09-12T03:02:29.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.529+0000 7f646e638740 1 -- 172.21.15.116:0/2077025435 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b663250bd0 con 0x55b6631d5bb0 2024-09-12T03:02:29.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.530+0000 7f644d7fa640 1 -- 172.21.15.116:0/2077025435 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f645c033070 con 0x55b6631d5bb0 2024-09-12T03:02:29.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.530+0000 7f644d7fa640 1 -- 172.21.15.116:0/2077025435 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f645c0417c0 con 0x55b6631d5bb0 2024-09-12T03:02:29.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.530+0000 7f646e638740 10 client.15039 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:02:29.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.530+0000 7f646e638740 10 client.15039 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:02:29.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.530+0000 7f646e638740 20 client.15039 populate_metadata read hostname 'smithi116' 2024-09-12T03:02:29.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.530+0000 7f646e638740 1 --2- 172.21.15.116:0/2077025435 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b663255090 0x55b663253da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:29.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.530+0000 7f646e638740 4 client.15039 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:02:29.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.530+0000 7f646e638740 1 -- 172.21.15.116:0/2077025435 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "status"}) -- 0x55b663138b10 con 0x55b663255090 2024-09-12T03:02:29.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.530+0000 7f6469ffb640 1 --2- 172.21.15.116:0/2077025435 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b663255090 0x55b663253da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:29.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.530+0000 7f644d7fa640 10 client.15039 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:02:29.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.530+0000 7f6469ffb640 1 --2- 172.21.15.116:0/2077025435 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b663255090 0x55b663253da0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4af5375fa6393397 server_cookie=2b3c01bb343be0b3 in_seq=0 out_seq=0 2024-09-12T03:02:29.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.531+0000 7f644d7fa640 1 -- 172.21.15.116:0/2077025435 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b663138b10 con 0x55b663255090 2024-09-12T03:02:29.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.531+0000 7f644d7fa640 10 client.15039 handle_command_reply: tid=0 2024-09-12T03:02:29.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.531+0000 7f646e638740 1 client.15039 shutdown 2024-09-12T03:02:29.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.531+0000 7f646e638740 1 -- 172.21.15.116:0/2077025435 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6631d5bb0 msgr2=0x55b6631ebe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:29.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.531+0000 7f646e638740 1 --2- 172.21.15.116:0/2077025435 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6631d5bb0 0x55b6631ebe10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f645c01d650 tx=0x7f645c013490 comp rx=0 tx=0).stop 2024-09-12T03:02:29.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.531+0000 7f646e638740 1 -- 172.21.15.116:0/2077025435 shutdown_connections 2024-09-12T03:02:29.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.531+0000 7f646e638740 1 --2- 172.21.15.116:0/2077025435 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b663255090 0x55b663253da0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.531+0000 7f646e638740 1 --2- 172.21.15.116:0/2077025435 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6631d5bb0 0x55b6631ebe10 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.531+0000 7f646e638740 1 --2- 172.21.15.116:0/2077025435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6631cb2f0 0x55b6631eb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.531+0000 7f644d7fa640 0 client.15039 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:02:29.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.531+0000 7f646e638740 1 --2- 172.21.15.116:0/2077025435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6631ca9d0 0x55b6631f0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.531+0000 7f646e638740 1 -- 172.21.15.116:0/2077025435 >> 172.21.15.116:0/2077025435 conn(0x55b6631c6f60 msgr2=0x55b6631d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:29.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.531+0000 7f646e638740 1 -- 172.21.15.116:0/2077025435 shutdown_connections 2024-09-12T03:02:29.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.531+0000 7f646e638740 1 -- 172.21.15.116:0/2077025435 wait complete. 2024-09-12T03:02:29.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.531+0000 7f646e638740 20 client.15039 trim_cache size 0 max 16384 2024-09-12T03:02:29.545 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:02:29.545 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_fsid": "eb1ac602-70b1-11ef-bcea-c7b262605968", 2024-09-12T03:02:29.546 INFO:teuthology.orchestra.run.smithi116.stdout: "whoami": 0, 2024-09-12T03:02:29.546 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 24494, 2024-09-12T03:02:29.546 INFO:teuthology.orchestra.run.smithi116.stdout: "want_state": "up:active", 2024-09-12T03:02:29.546 INFO:teuthology.orchestra.run.smithi116.stdout: "state": "up:active", 2024-09-12T03:02:29.546 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs", 2024-09-12T03:02:29.546 INFO:teuthology.orchestra.run.smithi116.stdout: "rank_uptime": 90.195306923000004, 2024-09-12T03:02:29.546 INFO:teuthology.orchestra.run.smithi116.stdout: "mdsmap_epoch": 24, 2024-09-12T03:02:29.546 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch": 80, 2024-09-12T03:02:29.546 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch_barrier": 78, 2024-09-12T03:02:29.547 INFO:teuthology.orchestra.run.smithi116.stdout: "uptime": 100.831263226 2024-09-12T03:02:29.547 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:02:29.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.532+0000 7f646c25f640 1 -- 172.21.15.116:0/1945622054 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f64500a2c10 msgr2=0x7f64500a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:29.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.532+0000 7f646c25f640 1 --2- 172.21.15.116:0/1945622054 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f64500a2c10 0x7f64500a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f645c021000 tx=0x7f645c020000 comp rx=0 tx=0).stop 2024-09-12T03:02:29.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.532+0000 7f646c25f640 1 -- 172.21.15.116:0/1945622054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6464102790 msgr2=0x7f6464074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:29.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.532+0000 7f646c25f640 1 --2- 172.21.15.116:0/1945622054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6464102790 0x7f6464074a30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6454004770 tx=0x7f6454003c10 comp rx=0 tx=0).stop 2024-09-12T03:02:29.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.532+0000 7f646c25f640 1 -- 172.21.15.116:0/1945622054 shutdown_connections 2024-09-12T03:02:29.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.532+0000 7f646c25f640 1 --2- 172.21.15.116:0/1945622054 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f64500a2c10 0x7f64500a50d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.532+0000 7f646c25f640 1 --2- 172.21.15.116:0/1945622054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6464106f10 0x7f6464074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.532+0000 7f646c25f640 1 --2- 172.21.15.116:0/1945622054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6464102790 0x7f6464074a30 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.532+0000 7f646c25f640 1 --2- 172.21.15.116:0/1945622054 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6464101de0 0x7f64640744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.532+0000 7f646c25f640 1 -- 172.21.15.116:0/1945622054 >> 172.21.15.116:0/1945622054 conn(0x7f64640fd880 msgr2=0x7f64640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:29.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.532+0000 7f646c25f640 1 -- 172.21.15.116:0/1945622054 shutdown_connections 2024-09-12T03:02:29.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:29.532+0000 7f646c25f640 1 -- 172.21.15.116:0/1945622054 wait complete. 2024-09-12T03:02:30.418 INFO:tasks.cephadm:Setup complete, yielding 2024-09-12T03:02:30.418 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-12T03:02:30.432 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi116.front.sepia.ceph.com 2024-09-12T03:02:30.432 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-12T03:02:30.456 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:30 smithi116 ceph-mon[31410]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:02:30.680 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:30.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:30 smithi183 ceph-mon[35535]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:02:30.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:30 smithi177 ceph-mon[35305]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:02:31.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.177+0000 7f9707856640 1 -- 172.21.15.116:0/672000290 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9700101bf0 msgr2=0x7f9700101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:31.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.177+0000 7f9707856640 1 --2- 172.21.15.116:0/672000290 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9700101bf0 0x7f9700101ff0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f96fc009900 tx=0x7f96fc02a410 comp rx=0 tx=0).stop 2024-09-12T03:02:31.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.177+0000 7f9707856640 1 -- 172.21.15.116:0/672000290 shutdown_connections 2024-09-12T03:02:31.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.177+0000 7f9707856640 1 --2- 172.21.15.116:0/672000290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97001037b0 0x7f970010a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:31.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.177+0000 7f9707856640 1 --2- 172.21.15.116:0/672000290 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9700102df0 0x7f9700103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:31.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.177+0000 7f9707856640 1 --2- 172.21.15.116:0/672000290 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9700101bf0 0x7f9700101ff0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:31.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.177+0000 7f9707856640 1 -- 172.21.15.116:0/672000290 >> 172.21.15.116:0/672000290 conn(0x7f97000fd3c0 msgr2=0x7f97000ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:31.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.177+0000 7f9707856640 1 -- 172.21.15.116:0/672000290 shutdown_connections 2024-09-12T03:02:31.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.177+0000 7f9707856640 1 -- 172.21.15.116:0/672000290 wait complete. 2024-09-12T03:02:31.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.178+0000 7f9707856640 1 Processor -- start 2024-09-12T03:02:31.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.178+0000 7f9707856640 1 -- start start 2024-09-12T03:02:31.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.178+0000 7f9707856640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9700101bf0 0x7f9700197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:31.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.178+0000 7f9707856640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9700102df0 0x7f97001983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:31.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.178+0000 7f9707856640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97001037b0 0x7f970019f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:31.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.178+0000 7f9707856640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f970010d880 con 0x7f9700101bf0 2024-09-12T03:02:31.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.178+0000 7f9707856640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f970010d700 con 0x7f9700102df0 2024-09-12T03:02:31.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.178+0000 7f9707856640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f970010da00 con 0x7f97001037b0 2024-09-12T03:02:31.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.178+0000 7f97055f2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9700101bf0 0x7f9700197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:31.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.178+0000 7f97055f2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9700101bf0 0x7f9700197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47032/0 (socket says 172.21.15.116:47032) 2024-09-12T03:02:31.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.178+0000 7f97055f2640 1 -- 172.21.15.116:0/4247598299 learned_addr learned my addr 172.21.15.116:0/4247598299 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:31.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.178+0000 7f9704df1640 1 --2- 172.21.15.116:0/4247598299 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9700102df0 0x7f97001983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:31.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.178+0000 7f9705df3640 1 --2- 172.21.15.116:0/4247598299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97001037b0 0x7f970019f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:31.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.178+0000 7f97055f2640 1 -- 172.21.15.116:0/4247598299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97001037b0 msgr2=0x7f970019f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:31.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.178+0000 7f97055f2640 1 --2- 172.21.15.116:0/4247598299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97001037b0 0x7f970019f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:31.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.178+0000 7f97055f2640 1 -- 172.21.15.116:0/4247598299 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9700102df0 msgr2=0x7f97001983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:31.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.178+0000 7f97055f2640 1 --2- 172.21.15.116:0/4247598299 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9700102df0 0x7f97001983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:31.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.178+0000 7f97055f2640 1 -- 172.21.15.116:0/4247598299 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970019fae0 con 0x7f9700101bf0 2024-09-12T03:02:31.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.178+0000 7f9704df1640 1 --2- 172.21.15.116:0/4247598299 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9700102df0 0x7f97001983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:31.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.178+0000 7f97055f2640 1 --2- 172.21.15.116:0/4247598299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9700101bf0 0x7f9700197eb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f96fc031610 tx=0x7f96fc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:31.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.179+0000 7f96f67fc640 1 -- 172.21.15.116:0/4247598299 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc03f030 con 0x7f9700101bf0 2024-09-12T03:02:31.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.179+0000 7f96f67fc640 1 -- 172.21.15.116:0/4247598299 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96fc003ad0 con 0x7f9700101bf0 2024-09-12T03:02:31.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.179+0000 7f9707856640 1 -- 172.21.15.116:0/4247598299 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f970019fd70 con 0x7f9700101bf0 2024-09-12T03:02:31.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.179+0000 7f96f67fc640 1 -- 172.21.15.116:0/4247598299 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc035070 con 0x7f9700101bf0 2024-09-12T03:02:31.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.179+0000 7f9707856640 1 -- 172.21.15.116:0/4247598299 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97001a01a0 con 0x7f9700101bf0 2024-09-12T03:02:31.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.180+0000 7f9707856640 1 -- 172.21.15.116:0/4247598299 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9700108fa0 con 0x7f9700101bf0 2024-09-12T03:02:31.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.181+0000 7f96f67fc640 1 -- 172.21.15.116:0/4247598299 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f96fc042030 con 0x7f9700101bf0 2024-09-12T03:02:31.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.181+0000 7f96f67fc640 1 --2- 172.21.15.116:0/4247598299 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f96ec0a2c40 0x7f96ec0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:31.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.182+0000 7f9704df1640 1 --2- 172.21.15.116:0/4247598299 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f96ec0a2c40 0x7f96ec0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:31.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.182+0000 7f96f67fc640 1 -- 172.21.15.116:0/4247598299 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f96fc0ef8b0 con 0x7f9700101bf0 2024-09-12T03:02:31.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.182+0000 7f9704df1640 1 --2- 172.21.15.116:0/4247598299 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f96ec0a2c40 0x7f96ec0a5100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f97001993d0 tx=0x7f96f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:31.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.186+0000 7f96f67fc640 1 -- 172.21.15.116:0/4247598299 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f96fc0bc6e0 con 0x7f9700101bf0 2024-09-12T03:02:31.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.371+0000 7f9707856640 1 -- 172.21.15.116:0/4247598299 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f9700062630 con 0x7f96ec0a2c40 2024-09-12T03:02:31.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.372+0000 7f96f67fc640 1 -- 172.21.15.116:0/4247598299 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f9700062630 con 0x7f96ec0a2c40 2024-09-12T03:02:31.373 INFO:teuthology.orchestra.run.smithi116.stdout:Backend: cephadm 2024-09-12T03:02:31.373 INFO:teuthology.orchestra.run.smithi116.stdout:Available: Yes 2024-09-12T03:02:31.373 INFO:teuthology.orchestra.run.smithi116.stdout:Paused: No 2024-09-12T03:02:31.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.374+0000 7f9707856640 1 -- 172.21.15.116:0/4247598299 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f96ec0a2c40 msgr2=0x7f96ec0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:31.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.374+0000 7f9707856640 1 --2- 172.21.15.116:0/4247598299 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f96ec0a2c40 0x7f96ec0a5100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f97001993d0 tx=0x7f96f0005f90 comp rx=0 tx=0).stop 2024-09-12T03:02:31.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.374+0000 7f9707856640 1 -- 172.21.15.116:0/4247598299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9700101bf0 msgr2=0x7f9700197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:31.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.374+0000 7f9707856640 1 --2- 172.21.15.116:0/4247598299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9700101bf0 0x7f9700197eb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f96fc031610 tx=0x7f96fc004270 comp rx=0 tx=0).stop 2024-09-12T03:02:31.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.375+0000 7f9707856640 1 -- 172.21.15.116:0/4247598299 shutdown_connections 2024-09-12T03:02:31.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.375+0000 7f9707856640 1 --2- 172.21.15.116:0/4247598299 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f96ec0a2c40 0x7f96ec0a5100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:31.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.375+0000 7f9707856640 1 --2- 172.21.15.116:0/4247598299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97001037b0 0x7f970019f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:31.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.375+0000 7f9707856640 1 --2- 172.21.15.116:0/4247598299 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9700102df0 0x7f97001983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:31.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.375+0000 7f9707856640 1 --2- 172.21.15.116:0/4247598299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9700101bf0 0x7f9700197eb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:31.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.375+0000 7f9707856640 1 -- 172.21.15.116:0/4247598299 >> 172.21.15.116:0/4247598299 conn(0x7f97000fd3c0 msgr2=0x7f9700106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:31.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.375+0000 7f9707856640 1 -- 172.21.15.116:0/4247598299 shutdown_connections 2024-09-12T03:02:31.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:31.375+0000 7f9707856640 1 -- 172.21.15.116:0/4247598299 wait complete. 2024-09-12T03:02:32.058 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-12T03:02:32.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:32 smithi116 ceph-mon[31410]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-12T03:02:32.346 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:32.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:32 smithi183 ceph-mon[35535]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-12T03:02:32.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:32 smithi177 ceph-mon[35305]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-12T03:02:32.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.830+0000 7fb26f171640 1 -- 172.21.15.116:0/2442443701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2681026d0 msgr2=0x7fb268108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:32.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.830+0000 7fb26f171640 1 --2- 172.21.15.116:0/2442443701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2681026d0 0x7fb268108f60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb260009ca0 tx=0x7fb26002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:02:32.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.830+0000 7fb26f171640 1 -- 172.21.15.116:0/2442443701 shutdown_connections 2024-09-12T03:02:32.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.830+0000 7fb26f171640 1 --2- 172.21.15.116:0/2442443701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2681026d0 0x7fb268108f60 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:32.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.830+0000 7fb26f171640 1 --2- 172.21.15.116:0/2442443701 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb268101d10 0x7fb268102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:32.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.830+0000 7fb26f171640 1 --2- 172.21.15.116:0/2442443701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb268100b10 0x7fb268100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:32.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.830+0000 7fb26f171640 1 -- 172.21.15.116:0/2442443701 >> 172.21.15.116:0/2442443701 conn(0x7fb2680fc2a0 msgr2=0x7fb2680fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:32.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.830+0000 7fb26f171640 1 -- 172.21.15.116:0/2442443701 shutdown_connections 2024-09-12T03:02:32.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.830+0000 7fb26f171640 1 -- 172.21.15.116:0/2442443701 wait complete. 2024-09-12T03:02:32.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.830+0000 7fb26f171640 1 Processor -- start 2024-09-12T03:02:32.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.831+0000 7fb26f171640 1 -- start start 2024-09-12T03:02:32.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.831+0000 7fb26f171640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb268100b10 0x7fb26806b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:32.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.831+0000 7fb26f171640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb268101d10 0x7fb26819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:32.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.831+0000 7fb26f171640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2681026d0 0x7fb26819f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:32.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.831+0000 7fb26f171640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb26810c890 con 0x7fb268101d10 2024-09-12T03:02:32.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.831+0000 7fb26f171640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb26810c710 con 0x7fb2681026d0 2024-09-12T03:02:32.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.831+0000 7fb26f171640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb26810ca10 con 0x7fb268100b10 2024-09-12T03:02:32.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.831+0000 7fb267fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb268101d10 0x7fb26819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:32.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.831+0000 7fb267fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb268101d10 0x7fb26819cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47058/0 (socket says 172.21.15.116:47058) 2024-09-12T03:02:32.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.831+0000 7fb267fff640 1 -- 172.21.15.116:0/2418332362 learned_addr learned my addr 172.21.15.116:0/2418332362 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:32.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.831+0000 7fb26cf0d640 1 --2- 172.21.15.116:0/2418332362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb268100b10 0x7fb26806b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:32.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.831+0000 7fb26d70e640 1 --2- 172.21.15.116:0/2418332362 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2681026d0 0x7fb26819f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:32.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.831+0000 7fb267fff640 1 -- 172.21.15.116:0/2418332362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb268100b10 msgr2=0x7fb26806b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:32.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.831+0000 7fb267fff640 1 --2- 172.21.15.116:0/2418332362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb268100b10 0x7fb26806b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:32.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.831+0000 7fb267fff640 1 -- 172.21.15.116:0/2418332362 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2681026d0 msgr2=0x7fb26819f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:32.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.831+0000 7fb267fff640 1 --2- 172.21.15.116:0/2418332362 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2681026d0 0x7fb26819f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:32.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.831+0000 7fb267fff640 1 -- 172.21.15.116:0/2418332362 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb26819fce0 con 0x7fb268101d10 2024-09-12T03:02:32.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.831+0000 7fb26cf0d640 1 --2- 172.21.15.116:0/2418332362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb268100b10 0x7fb26806b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:32.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.831+0000 7fb26d70e640 1 --2- 172.21.15.116:0/2418332362 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2681026d0 0x7fb26819f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:32.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.831+0000 7fb267fff640 1 --2- 172.21.15.116:0/2418332362 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb268101d10 0x7fb26819cdb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb25400b4a0 tx=0x7fb25400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:32.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.831+0000 7fb265ffb640 1 -- 172.21.15.116:0/2418332362 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb254014070 con 0x7fb268101d10 2024-09-12T03:02:32.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.831+0000 7fb265ffb640 1 -- 172.21.15.116:0/2418332362 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb254010af0 con 0x7fb268101d10 2024-09-12T03:02:32.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.831+0000 7fb26f171640 1 -- 172.21.15.116:0/2418332362 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb26819ff70 con 0x7fb268101d10 2024-09-12T03:02:32.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.831+0000 7fb265ffb640 1 -- 172.21.15.116:0/2418332362 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb254017d10 con 0x7fb268101d10 2024-09-12T03:02:32.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.832+0000 7fb26f171640 1 -- 172.21.15.116:0/2418332362 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2681a04a0 con 0x7fb268101d10 2024-09-12T03:02:32.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.833+0000 7fb26f171640 1 -- 172.21.15.116:0/2418332362 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb268107f30 con 0x7fb268101d10 2024-09-12T03:02:32.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.834+0000 7fb265ffb640 1 -- 172.21.15.116:0/2418332362 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fb254017070 con 0x7fb268101d10 2024-09-12T03:02:32.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.834+0000 7fb265ffb640 1 --2- 172.21.15.116:0/2418332362 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb2580a2c10 0x7fb2580a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:32.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.834+0000 7fb265ffb640 1 -- 172.21.15.116:0/2418332362 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fb2540ce4a0 con 0x7fb268101d10 2024-09-12T03:02:32.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.834+0000 7fb26cf0d640 1 --2- 172.21.15.116:0/2418332362 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb2580a2c10 0x7fb2580a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:32.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.835+0000 7fb26cf0d640 1 --2- 172.21.15.116:0/2418332362 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb2580a2c10 0x7fb2580a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb25c006fd0 tx=0x7fb25c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:32.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:32.839+0000 7fb265ffb640 1 -- 172.21.15.116:0/2418332362 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb25409b3e0 con 0x7fb268101d10 2024-09-12T03:02:33.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:33.012+0000 7fb26f171640 1 -- 172.21.15.116:0/2418332362 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fb268062630 con 0x7fb2580a2c10 2024-09-12T03:02:33.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:33.024+0000 7fb265ffb640 1 -- 172.21.15.116:0/2418332362 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7fb268062630 con 0x7fb2580a2c10 2024-09-12T03:02:33.025 INFO:teuthology.orchestra.run.smithi116.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-12T03:02:33.025 INFO:teuthology.orchestra.run.smithi116.stdout:mds.a smithi116 running (107s) 27s ago 106s 15.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 d519324b9a1b 2024-09-12T03:02:33.025 INFO:teuthology.orchestra.run.smithi116.stdout:mds.b smithi177 running (104s) 37s ago 104s 26.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 14a73268da50 2024-09-12T03:02:33.025 INFO:teuthology.orchestra.run.smithi116.stdout:mds.c smithi183 running (118s) 37s ago 118s 17.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 7fbbfbd37f26 2024-09-12T03:02:33.025 INFO:teuthology.orchestra.run.smithi116.stdout:mds.d smithi116 running (2m) 27s ago 2m 15.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 f7f6d74490c1 2024-09-12T03:02:33.025 INFO:teuthology.orchestra.run.smithi116.stdout:mds.e smithi177 running (2m) 37s ago 2m 16.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 735c3254bcdd 2024-09-12T03:02:33.025 INFO:teuthology.orchestra.run.smithi116.stdout:mds.f smithi183 running (2m) 37s ago 2m 18.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 29b50457e05d 2024-09-12T03:02:33.025 INFO:teuthology.orchestra.run.smithi116.stdout:mds.g smithi116 running (110s) 27s ago 110s 16.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 199e293b8297 2024-09-12T03:02:33.025 INFO:teuthology.orchestra.run.smithi116.stdout:mds.h smithi177 running (116s) 37s ago 115s 17.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 0aabcdea56ce 2024-09-12T03:02:33.025 INFO:teuthology.orchestra.run.smithi116.stdout:mds.i smithi183 running (113s) 37s ago 113s 17.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 a6a2f21bf45e 2024-09-12T03:02:33.025 INFO:teuthology.orchestra.run.smithi116.stdout:mds.j smithi116 running (2m) 27s ago 2m 13.6M - 19.1.1-298-gb53b6d3e 54467abc55f8 5b19ae306a97 2024-09-12T03:02:33.025 INFO:teuthology.orchestra.run.smithi116.stdout:mds.k smithi177 running (2m) 37s ago 2m 14.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 2d7de1207a65 2024-09-12T03:02:33.025 INFO:teuthology.orchestra.run.smithi116.stdout:mds.l smithi183 running (2m) 37s ago 2m 14.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 7d1a29e0bc14 2024-09-12T03:02:33.026 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.x smithi116 *:9283,8765 running (9m) 27s ago 9m 508M - 19.1.1-298-gb53b6d3e 54467abc55f8 3b0b103b7373 2024-09-12T03:02:33.026 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.y smithi177 *:8443,8765 running (7m) 37s ago 7m 442M - 19.1.1-298-gb53b6d3e 54467abc55f8 e90c4be67b8a 2024-09-12T03:02:33.026 INFO:teuthology.orchestra.run.smithi116.stdout:mgr.z smithi183 *:8443,8765 running (7m) 37s ago 7m 444M - 19.1.1-298-gb53b6d3e 54467abc55f8 74021d43cead 2024-09-12T03:02:33.026 INFO:teuthology.orchestra.run.smithi116.stdout:mon.a smithi116 running (9m) 27s ago 9m 53.1M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 3479f0eb6345 2024-09-12T03:02:33.026 INFO:teuthology.orchestra.run.smithi116.stdout:mon.b smithi177 running (7m) 37s ago 7m 38.2M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 0b4bfaf4a88e 2024-09-12T03:02:33.026 INFO:teuthology.orchestra.run.smithi116.stdout:mon.c smithi183 running (7m) 37s ago 7m 40.5M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 05ceb6fdd0c1 2024-09-12T03:02:33.026 INFO:teuthology.orchestra.run.smithi116.stdout:osd.0 smithi116 running (7m) 27s ago 7m 58.0M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 69274d5ee1fd 2024-09-12T03:02:33.026 INFO:teuthology.orchestra.run.smithi116.stdout:osd.1 smithi177 running (6m) 37s ago 6m 57.1M 4271M 19.1.1-298-gb53b6d3e 54467abc55f8 9f6b17aca9fd 2024-09-12T03:02:33.026 INFO:teuthology.orchestra.run.smithi116.stdout:osd.2 smithi183 running (6m) 37s ago 6m 57.9M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 58496194ce45 2024-09-12T03:02:33.026 INFO:teuthology.orchestra.run.smithi116.stdout:osd.3 smithi116 running (5m) 27s ago 5m 42.9M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 9c2d49b0330e 2024-09-12T03:02:33.026 INFO:teuthology.orchestra.run.smithi116.stdout:osd.4 smithi177 running (5m) 37s ago 5m 51.0M 4271M 19.1.1-298-gb53b6d3e 54467abc55f8 2fa119bca937 2024-09-12T03:02:33.026 INFO:teuthology.orchestra.run.smithi116.stdout:osd.5 smithi183 running (5m) 37s ago 5m 53.6M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 2d9d2fc285e1 2024-09-12T03:02:33.026 INFO:teuthology.orchestra.run.smithi116.stdout:osd.6 smithi116 running (4m) 27s ago 4m 51.0M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 d7a5e87090af 2024-09-12T03:02:33.026 INFO:teuthology.orchestra.run.smithi116.stdout:osd.7 smithi177 running (4m) 37s ago 4m 44.0M 4271M 19.1.1-298-gb53b6d3e 54467abc55f8 ce0e2a06101e 2024-09-12T03:02:33.027 INFO:teuthology.orchestra.run.smithi116.stdout:osd.8 smithi183 running (3m) 37s ago 3m 54.9M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 faf5be3c6bc1 2024-09-12T03:02:33.027 INFO:teuthology.orchestra.run.smithi116.stdout:osd.9 smithi116 running (3m) 27s ago 3m 40.4M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 d20ea0fc4a88 2024-09-12T03:02:33.027 INFO:teuthology.orchestra.run.smithi116.stdout:osd.10 smithi177 running (3m) 37s ago 3m 54.6M 4271M 19.1.1-298-gb53b6d3e 54467abc55f8 12214f9b0b40 2024-09-12T03:02:33.027 INFO:teuthology.orchestra.run.smithi116.stdout:osd.11 smithi183 running (2m) 37s ago 2m 39.6M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 a2c40c7e535b 2024-09-12T03:02:33.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:33.026+0000 7fb26f171640 1 -- 172.21.15.116:0/2418332362 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb2580a2c10 msgr2=0x7fb2580a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:33.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:33.026+0000 7fb26f171640 1 --2- 172.21.15.116:0/2418332362 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb2580a2c10 0x7fb2580a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb25c006fd0 tx=0x7fb25c008040 comp rx=0 tx=0).stop 2024-09-12T03:02:33.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:33.026+0000 7fb26f171640 1 -- 172.21.15.116:0/2418332362 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb268101d10 msgr2=0x7fb26819cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:33.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:33.026+0000 7fb26f171640 1 --2- 172.21.15.116:0/2418332362 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb268101d10 0x7fb26819cdb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb25400b4a0 tx=0x7fb25400b960 comp rx=0 tx=0).stop 2024-09-12T03:02:33.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:33.026+0000 7fb26f171640 1 -- 172.21.15.116:0/2418332362 shutdown_connections 2024-09-12T03:02:33.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:33.026+0000 7fb26f171640 1 --2- 172.21.15.116:0/2418332362 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb2580a2c10 0x7fb2580a50d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:33.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:33.026+0000 7fb26f171640 1 --2- 172.21.15.116:0/2418332362 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2681026d0 0x7fb26819f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:33.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:33.026+0000 7fb26f171640 1 --2- 172.21.15.116:0/2418332362 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb268101d10 0x7fb26819cdb0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:33.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:33.026+0000 7fb26f171640 1 --2- 172.21.15.116:0/2418332362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb268100b10 0x7fb26806b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:33.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:33.026+0000 7fb26f171640 1 -- 172.21.15.116:0/2418332362 >> 172.21.15.116:0/2418332362 conn(0x7fb2680fc2a0 msgr2=0x7fb268105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:33.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:33.026+0000 7fb26f171640 1 -- 172.21.15.116:0/2418332362 shutdown_connections 2024-09-12T03:02:33.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:33.026+0000 7fb26f171640 1 -- 172.21.15.116:0/2418332362 wait complete. 2024-09-12T03:02:33.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:33 smithi116 ceph-mon[31410]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:02:33.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:33 smithi116 ceph-mon[31410]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:02:33.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:33 smithi116 ceph-mon[31410]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:02:33.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:33 smithi183 ceph-mon[35535]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:02:33.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:33 smithi183 ceph-mon[35535]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:02:33.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:33 smithi183 ceph-mon[35535]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:02:33.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:33 smithi177 ceph-mon[35305]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:02:33.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:33 smithi177 ceph-mon[35305]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:02:33.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:33 smithi177 ceph-mon[35305]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:02:33.861 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-12T03:02:34.106 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:34.501 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:34 smithi116 ceph-mon[31410]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-12T03:02:34.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.579+0000 7fc9a07f6640 1 -- 172.21.15.116:0/1257239807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc998102680 msgr2=0x7fc998104a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:34.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.579+0000 7fc9a07f6640 1 --2- 172.21.15.116:0/1257239807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc998102680 0x7fc998104a70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc994009900 tx=0x7fc99402a410 comp rx=0 tx=0).stop 2024-09-12T03:02:34.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.579+0000 7fc9a07f6640 1 -- 172.21.15.116:0/1257239807 shutdown_connections 2024-09-12T03:02:34.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.579+0000 7fc9a07f6640 1 --2- 172.21.15.116:0/1257239807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9981078e0 0x7fc998109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:34.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.579+0000 7fc9a07f6640 1 --2- 172.21.15.116:0/1257239807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc998104fb0 0x7fc9981073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:34.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.579+0000 7fc9a07f6640 1 --2- 172.21.15.116:0/1257239807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc998102680 0x7fc998104a70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:34.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.579+0000 7fc9a07f6640 1 -- 172.21.15.116:0/1257239807 >> 172.21.15.116:0/1257239807 conn(0x7fc9980fc2a0 msgr2=0x7fc9980fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:34.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.579+0000 7fc9a07f6640 1 -- 172.21.15.116:0/1257239807 shutdown_connections 2024-09-12T03:02:34.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.580+0000 7fc9a07f6640 1 -- 172.21.15.116:0/1257239807 wait complete. 2024-09-12T03:02:34.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.580+0000 7fc9a07f6640 1 Processor -- start 2024-09-12T03:02:34.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.580+0000 7fc9a07f6640 1 -- start start 2024-09-12T03:02:34.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.580+0000 7fc9a07f6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc998102680 0x7fc9981980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:34.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.580+0000 7fc9a07f6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc998104fb0 0x7fc9981985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:34.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.580+0000 7fc9a07f6640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9981078e0 0x7fc99819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:34.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.580+0000 7fc9a07f6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc99810d970 con 0x7fc998104fb0 2024-09-12T03:02:34.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.580+0000 7fc9a07f6640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc99810d7f0 con 0x7fc9981078e0 2024-09-12T03:02:34.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.580+0000 7fc9a07f6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc99810daf0 con 0x7fc998102680 2024-09-12T03:02:34.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.580+0000 7fc99dd91640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc998104fb0 0x7fc9981985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:34.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.580+0000 7fc99dd91640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc998104fb0 0x7fc9981985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47070/0 (socket says 172.21.15.116:47070) 2024-09-12T03:02:34.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.580+0000 7fc99dd91640 1 -- 172.21.15.116:0/2529393061 learned_addr learned my addr 172.21.15.116:0/2529393061 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:34.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.580+0000 7fc99e592640 1 --2- 172.21.15.116:0/2529393061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc998102680 0x7fc9981980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:34.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.581+0000 7fc99dd91640 1 -- 172.21.15.116:0/2529393061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc998102680 msgr2=0x7fc9981980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:34.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.581+0000 7fc99dd91640 1 --2- 172.21.15.116:0/2529393061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc998102680 0x7fc9981980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:34.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.581+0000 7fc99dd91640 1 -- 172.21.15.116:0/2529393061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9981078e0 msgr2=0x7fc99819f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:34.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.581+0000 7fc99ed93640 1 --2- 172.21.15.116:0/2529393061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9981078e0 0x7fc99819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:34.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.581+0000 7fc99dd91640 1 --2- 172.21.15.116:0/2529393061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9981078e0 0x7fc99819f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:34.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.581+0000 7fc99dd91640 1 -- 172.21.15.116:0/2529393061 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99819fce0 con 0x7fc998104fb0 2024-09-12T03:02:34.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.581+0000 7fc99ed93640 1 --2- 172.21.15.116:0/2529393061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9981078e0 0x7fc99819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:34.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.581+0000 7fc99dd91640 1 --2- 172.21.15.116:0/2529393061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc998104fb0 0x7fc9981985f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc98800b450 tx=0x7fc98800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:34.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.581+0000 7fc98f7fe640 1 -- 172.21.15.116:0/2529393061 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988015020 con 0x7fc998104fb0 2024-09-12T03:02:34.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.581+0000 7fc98f7fe640 1 -- 172.21.15.116:0/2529393061 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc988011d70 con 0x7fc998104fb0 2024-09-12T03:02:34.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.581+0000 7fc9a07f6640 1 -- 172.21.15.116:0/2529393061 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99819ffd0 con 0x7fc998104fb0 2024-09-12T03:02:34.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.581+0000 7fc98f7fe640 1 -- 172.21.15.116:0/2529393061 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98800f340 con 0x7fc998104fb0 2024-09-12T03:02:34.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.581+0000 7fc9a07f6640 1 -- 172.21.15.116:0/2529393061 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9981a0500 con 0x7fc998104fb0 2024-09-12T03:02:34.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.582+0000 7fc9a07f6640 1 -- 172.21.15.116:0/2529393061 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9980fdd90 con 0x7fc998104fb0 2024-09-12T03:02:34.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.583+0000 7fc98f7fe640 1 -- 172.21.15.116:0/2529393061 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc988018020 con 0x7fc998104fb0 2024-09-12T03:02:34.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.584+0000 7fc98f7fe640 1 --2- 172.21.15.116:0/2529393061 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc9840a2bb0 0x7fc9840a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:34.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.584+0000 7fc99e592640 1 --2- 172.21.15.116:0/2529393061 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc9840a2bb0 0x7fc9840a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:34.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.584+0000 7fc98f7fe640 1 -- 172.21.15.116:0/2529393061 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fc9880ceb00 con 0x7fc998104fb0 2024-09-12T03:02:34.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.584+0000 7fc99e592640 1 --2- 172.21.15.116:0/2529393061 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc9840a2bb0 0x7fc9840a5070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc994000c00 tx=0x7fc994031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:34.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.588+0000 7fc98f7fe640 1 -- 172.21.15.116:0/2529393061 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc98801f020 con 0x7fc998104fb0 2024-09-12T03:02:34.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:34 smithi183 ceph-mon[35535]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-12T03:02:34.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:34 smithi177 ceph-mon[35305]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-12T03:02:34.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.762+0000 7fc9a07f6640 1 -- 172.21.15.116:0/2529393061 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fc998062630 con 0x7fc9840a2bb0 2024-09-12T03:02:34.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.768+0000 7fc98f7fe640 1 -- 172.21.15.116:0/2529393061 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fc998062630 con 0x7fc9840a2bb0 2024-09-12T03:02:34.768 INFO:teuthology.orchestra.run.smithi116.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-12T03:02:34.768 INFO:teuthology.orchestra.run.smithi116.stdout:mds.all 12/12 38s ago 2m smithi116=a;smithi116=d;smithi116=g;smithi116=j;smithi177=b;smithi177=e;smithi177=h;smithi177=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2024-09-12T03:02:34.768 INFO:teuthology.orchestra.run.smithi116.stdout:mgr 3/3 38s ago 7m smithi116=x;smithi177=y;smithi183=z;count:3 2024-09-12T03:02:34.768 INFO:teuthology.orchestra.run.smithi116.stdout:mon 3/3 38s ago 8m smithi116:172.21.15.116=a;smithi177:172.21.15.177=b;smithi183:172.21.15.183=c;count:3 2024-09-12T03:02:34.768 INFO:teuthology.orchestra.run.smithi116.stdout:osd 12 38s ago - 2024-09-12T03:02:34.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.770+0000 7fc9a07f6640 1 -- 172.21.15.116:0/2529393061 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc9840a2bb0 msgr2=0x7fc9840a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:34.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.770+0000 7fc9a07f6640 1 --2- 172.21.15.116:0/2529393061 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc9840a2bb0 0x7fc9840a5070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc994000c00 tx=0x7fc994031400 comp rx=0 tx=0).stop 2024-09-12T03:02:34.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.770+0000 7fc9a07f6640 1 -- 172.21.15.116:0/2529393061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc998104fb0 msgr2=0x7fc9981985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:34.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.770+0000 7fc9a07f6640 1 --2- 172.21.15.116:0/2529393061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc998104fb0 0x7fc9981985f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc98800b450 tx=0x7fc98800b910 comp rx=0 tx=0).stop 2024-09-12T03:02:34.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.770+0000 7fc9a07f6640 1 -- 172.21.15.116:0/2529393061 shutdown_connections 2024-09-12T03:02:34.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.770+0000 7fc9a07f6640 1 --2- 172.21.15.116:0/2529393061 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc9840a2bb0 0x7fc9840a5070 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:34.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.770+0000 7fc9a07f6640 1 --2- 172.21.15.116:0/2529393061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9981078e0 0x7fc99819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:34.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.770+0000 7fc9a07f6640 1 --2- 172.21.15.116:0/2529393061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc998104fb0 0x7fc9981985f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:34.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.770+0000 7fc9a07f6640 1 --2- 172.21.15.116:0/2529393061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc998102680 0x7fc9981980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:34.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.770+0000 7fc9a07f6640 1 -- 172.21.15.116:0/2529393061 >> 172.21.15.116:0/2529393061 conn(0x7fc9980fc2a0 msgr2=0x7fc998102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:34.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.770+0000 7fc9a07f6640 1 -- 172.21.15.116:0/2529393061 shutdown_connections 2024-09-12T03:02:34.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:34.770+0000 7fc9a07f6640 1 -- 172.21.15.116:0/2529393061 wait complete. 2024-09-12T03:02:35.490 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:35 smithi116 ceph-mon[31410]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:02:35.491 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:35 smithi116 ceph-mon[31410]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:02:35.518 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-12T03:02:35.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:35 smithi183 ceph-mon[35535]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:02:35.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:35 smithi183 ceph-mon[35535]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:02:35.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:35 smithi177 ceph-mon[35305]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:02:35.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:35 smithi177 ceph-mon[35305]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:02:35.763 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:36.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.277+0000 7f5d74077640 1 -- 172.21.15.116:0/1235659717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d6c101760 msgr2=0x7f5d6c109000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:36.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.277+0000 7f5d74077640 1 --2- 172.21.15.116:0/1235659717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d6c101760 0x7f5d6c109000 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5d64009cd0 tx=0x7f5d6402a8e0 comp rx=0 tx=0).stop 2024-09-12T03:02:36.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.277+0000 7f5d74077640 1 -- 172.21.15.116:0/1235659717 shutdown_connections 2024-09-12T03:02:36.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.277+0000 7f5d74077640 1 --2- 172.21.15.116:0/1235659717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d6c101760 0x7f5d6c109000 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:36.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.277+0000 7f5d74077640 1 --2- 172.21.15.116:0/1235659717 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d6c100da0 0x7f5d6c101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:36.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.277+0000 7f5d74077640 1 --2- 172.21.15.116:0/1235659717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d6c1020f0 0x7f5d6c1024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:36.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.277+0000 7f5d74077640 1 -- 172.21.15.116:0/1235659717 >> 172.21.15.116:0/1235659717 conn(0x7f5d6c0fba40 msgr2=0x7f5d6c0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:36.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.277+0000 7f5d74077640 1 -- 172.21.15.116:0/1235659717 shutdown_connections 2024-09-12T03:02:36.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.277+0000 7f5d74077640 1 -- 172.21.15.116:0/1235659717 wait complete. 2024-09-12T03:02:36.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.277+0000 7f5d74077640 1 Processor -- start 2024-09-12T03:02:36.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.277+0000 7f5d74077640 1 -- start start 2024-09-12T03:02:36.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.278+0000 7f5d74077640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d6c100da0 0x7f5d6c1007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:36.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.278+0000 7f5d74077640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d6c101760 0x7f5d6c19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:36.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.278+0000 7f5d74077640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d6c1020f0 0x7f5d6c19f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:36.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.278+0000 7f5d74077640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c10c790 con 0x7f5d6c1020f0 2024-09-12T03:02:36.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.278+0000 7f5d74077640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c10c610 con 0x7f5d6c101760 2024-09-12T03:02:36.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.278+0000 7f5d74077640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6c10c910 con 0x7f5d6c100da0 2024-09-12T03:02:36.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.278+0000 7f5d72614640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d6c1020f0 0x7f5d6c19f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:36.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.278+0000 7f5d72614640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d6c1020f0 0x7f5d6c19f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47076/0 (socket says 172.21.15.116:47076) 2024-09-12T03:02:36.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.278+0000 7f5d72614640 1 -- 172.21.15.116:0/591739193 learned_addr learned my addr 172.21.15.116:0/591739193 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:36.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.278+0000 7f5d71e13640 1 --2- 172.21.15.116:0/591739193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d6c100da0 0x7f5d6c1007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:36.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.278+0000 7f5d71612640 1 --2- 172.21.15.116:0/591739193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d6c101760 0x7f5d6c19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:36.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.278+0000 7f5d72614640 1 -- 172.21.15.116:0/591739193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d6c100da0 msgr2=0x7f5d6c1007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:36.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.278+0000 7f5d72614640 1 --2- 172.21.15.116:0/591739193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d6c100da0 0x7f5d6c1007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:36.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.278+0000 7f5d72614640 1 -- 172.21.15.116:0/591739193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d6c101760 msgr2=0x7f5d6c19cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:36.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.278+0000 7f5d72614640 1 --2- 172.21.15.116:0/591739193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d6c101760 0x7f5d6c19cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:36.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.278+0000 7f5d72614640 1 -- 172.21.15.116:0/591739193 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6c19fb10 con 0x7f5d6c1020f0 2024-09-12T03:02:36.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.278+0000 7f5d71612640 1 --2- 172.21.15.116:0/591739193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d6c101760 0x7f5d6c19cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:36.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.278+0000 7f5d72614640 1 --2- 172.21.15.116:0/591739193 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d6c1020f0 0x7f5d6c19f200 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5d64009df0 tx=0x7f5d64003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:36.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.278+0000 7f5d5effd640 1 -- 172.21.15.116:0/591739193 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d64004450 con 0x7f5d6c1020f0 2024-09-12T03:02:36.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.278+0000 7f5d5effd640 1 -- 172.21.15.116:0/591739193 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d64003e40 con 0x7f5d6c1020f0 2024-09-12T03:02:36.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.278+0000 7f5d74077640 1 -- 172.21.15.116:0/591739193 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d6c19fd40 con 0x7f5d6c1020f0 2024-09-12T03:02:36.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.278+0000 7f5d5effd640 1 -- 172.21.15.116:0/591739193 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d640466e0 con 0x7f5d6c1020f0 2024-09-12T03:02:36.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.278+0000 7f5d74077640 1 -- 172.21.15.116:0/591739193 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d6c1a02d0 con 0x7f5d6c1020f0 2024-09-12T03:02:36.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.279+0000 7f5d74077640 1 -- 172.21.15.116:0/591739193 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d6c107ee0 con 0x7f5d6c1020f0 2024-09-12T03:02:36.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.281+0000 7f5d5effd640 1 -- 172.21.15.116:0/591739193 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5d640305a0 con 0x7f5d6c1020f0 2024-09-12T03:02:36.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.281+0000 7f5d5effd640 1 --2- 172.21.15.116:0/591739193 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5d580a2c00 0x7f5d580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:36.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.282+0000 7f5d71e13640 1 --2- 172.21.15.116:0/591739193 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5d580a2c00 0x7f5d580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:36.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.282+0000 7f5d5effd640 1 -- 172.21.15.116:0/591739193 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f5d6402e030 con 0x7f5d6c1020f0 2024-09-12T03:02:36.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.282+0000 7f5d71e13640 1 --2- 172.21.15.116:0/591739193 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5d580a2c00 0x7f5d580a50c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c06a550 tx=0x7f5d60005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:36.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.285+0000 7f5d5effd640 1 -- 172.21.15.116:0/591739193 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5d6404b060 con 0x7f5d6c1020f0 2024-09-12T03:02:36.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.457+0000 7f5d74077640 1 -- 172.21.15.116:0/591739193 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f5d6c062630 con 0x7f5d580a2c00 2024-09-12T03:02:36.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.459+0000 7f5d5effd640 1 -- 172.21.15.116:0/591739193 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f5d6c062630 con 0x7f5d580a2c00 2024-09-12T03:02:36.459 INFO:teuthology.orchestra.run.smithi116.stdout:HOST ADDR LABELS STATUS 2024-09-12T03:02:36.459 INFO:teuthology.orchestra.run.smithi116.stdout:smithi116 172.21.15.116 2024-09-12T03:02:36.459 INFO:teuthology.orchestra.run.smithi116.stdout:smithi177 172.21.15.177 2024-09-12T03:02:36.459 INFO:teuthology.orchestra.run.smithi116.stdout:smithi183 172.21.15.183 2024-09-12T03:02:36.459 INFO:teuthology.orchestra.run.smithi116.stdout:3 hosts in cluster 2024-09-12T03:02:36.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.461+0000 7f5d74077640 1 -- 172.21.15.116:0/591739193 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5d580a2c00 msgr2=0x7f5d580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:36.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.461+0000 7f5d74077640 1 --2- 172.21.15.116:0/591739193 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5d580a2c00 0x7f5d580a50c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c06a550 tx=0x7f5d60005f90 comp rx=0 tx=0).stop 2024-09-12T03:02:36.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.461+0000 7f5d74077640 1 -- 172.21.15.116:0/591739193 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d6c1020f0 msgr2=0x7f5d6c19f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:36.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.461+0000 7f5d74077640 1 --2- 172.21.15.116:0/591739193 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d6c1020f0 0x7f5d6c19f200 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5d64009df0 tx=0x7f5d64003c20 comp rx=0 tx=0).stop 2024-09-12T03:02:36.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.461+0000 7f5d74077640 1 -- 172.21.15.116:0/591739193 shutdown_connections 2024-09-12T03:02:36.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.461+0000 7f5d74077640 1 --2- 172.21.15.116:0/591739193 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5d580a2c00 0x7f5d580a50c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:36.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.461+0000 7f5d74077640 1 --2- 172.21.15.116:0/591739193 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d6c1020f0 0x7f5d6c19f200 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:36.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.461+0000 7f5d74077640 1 --2- 172.21.15.116:0/591739193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d6c101760 0x7f5d6c19cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:36.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.461+0000 7f5d74077640 1 --2- 172.21.15.116:0/591739193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d6c100da0 0x7f5d6c1007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:36.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.461+0000 7f5d74077640 1 -- 172.21.15.116:0/591739193 >> 172.21.15.116:0/591739193 conn(0x7f5d6c0fba40 msgr2=0x7f5d6c1050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:36.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.461+0000 7f5d74077640 1 -- 172.21.15.116:0/591739193 shutdown_connections 2024-09-12T03:02:36.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:36.461+0000 7f5d74077640 1 -- 172.21.15.116:0/591739193 wait complete. 2024-09-12T03:02:37.195 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-12T03:02:37.443 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:37.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.941+0000 7f2589963640 1 -- 172.21.15.116:0/3287038768 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25841037c0 msgr2=0x7f258410a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:37.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.941+0000 7f2589963640 1 --2- 172.21.15.116:0/3287038768 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25841037c0 0x7f258410a050 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f257c009ca0 tx=0x7f257c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.941+0000 7f2589963640 1 -- 172.21.15.116:0/3287038768 shutdown_connections 2024-09-12T03:02:37.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.941+0000 7f2589963640 1 --2- 172.21.15.116:0/3287038768 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25841037c0 0x7f258410a050 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.941+0000 7f2589963640 1 --2- 172.21.15.116:0/3287038768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2584102e00 0x7f2584103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.941+0000 7f2589963640 1 --2- 172.21.15.116:0/3287038768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2584101c00 0x7f2584102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.941+0000 7f2589963640 1 -- 172.21.15.116:0/3287038768 >> 172.21.15.116:0/3287038768 conn(0x7f25840fd390 msgr2=0x7f25840ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:37.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.941+0000 7f2589963640 1 -- 172.21.15.116:0/3287038768 shutdown_connections 2024-09-12T03:02:37.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.941+0000 7f2589963640 1 -- 172.21.15.116:0/3287038768 wait complete. 2024-09-12T03:02:37.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.941+0000 7f2589963640 1 Processor -- start 2024-09-12T03:02:37.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.941+0000 7f2589963640 1 -- start start 2024-09-12T03:02:37.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.942+0000 7f2589963640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2584101c00 0x7f2584197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:37.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.942+0000 7f2589963640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2584102e00 0x7f2584198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:37.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.942+0000 7f2589963640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25841037c0 0x7f258419f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:37.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.942+0000 7f2589963640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f258410d900 con 0x7f2584102e00 2024-09-12T03:02:37.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.942+0000 7f2589963640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f258410d780 con 0x7f2584101c00 2024-09-12T03:02:37.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.942+0000 7f2589963640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f258410da80 con 0x7f25841037c0 2024-09-12T03:02:37.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.942+0000 7f257a7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2584102e00 0x7f2584198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:37.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.942+0000 7f257a7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2584102e00 0x7f2584198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47094/0 (socket says 172.21.15.116:47094) 2024-09-12T03:02:37.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.942+0000 7f257a7fc640 1 -- 172.21.15.116:0/3926830927 learned_addr learned my addr 172.21.15.116:0/3926830927 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:37.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.942+0000 7f2582ffd640 1 --2- 172.21.15.116:0/3926830927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2584101c00 0x7f2584197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:37.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.942+0000 7f25837fe640 1 --2- 172.21.15.116:0/3926830927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25841037c0 0x7f258419f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:37.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.942+0000 7f257a7fc640 1 -- 172.21.15.116:0/3926830927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25841037c0 msgr2=0x7f258419f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:37.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.942+0000 7f257a7fc640 1 --2- 172.21.15.116:0/3926830927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25841037c0 0x7f258419f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.942+0000 7f257a7fc640 1 -- 172.21.15.116:0/3926830927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2584101c00 msgr2=0x7f2584197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:37.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.942+0000 7f257a7fc640 1 --2- 172.21.15.116:0/3926830927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2584101c00 0x7f2584197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.942+0000 7f257a7fc640 1 -- 172.21.15.116:0/3926830927 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f258419fb80 con 0x7f2584102e00 2024-09-12T03:02:37.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.942+0000 7f2582ffd640 1 --2- 172.21.15.116:0/3926830927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2584101c00 0x7f2584197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:37.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.942+0000 7f257a7fc640 1 --2- 172.21.15.116:0/3926830927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2584102e00 0x7f2584198490 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f256c00d950 tx=0x7f256c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:37.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.942+0000 7f2588961640 1 -- 172.21.15.116:0/3926830927 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c017070 con 0x7f2584102e00 2024-09-12T03:02:37.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.943+0000 7f2589963640 1 -- 172.21.15.116:0/3926830927 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f258419fe70 con 0x7f2584102e00 2024-09-12T03:02:37.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.943+0000 7f2588961640 1 -- 172.21.15.116:0/3926830927 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f256c009c30 con 0x7f2584102e00 2024-09-12T03:02:37.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.943+0000 7f2588961640 1 -- 172.21.15.116:0/3926830927 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c00f6b0 con 0x7f2584102e00 2024-09-12T03:02:37.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.943+0000 7f2589963640 1 -- 172.21.15.116:0/3926830927 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25841a03f0 con 0x7f2584102e00 2024-09-12T03:02:37.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.944+0000 7f2589963640 1 -- 172.21.15.116:0/3926830927 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2584108f60 con 0x7f2584102e00 2024-09-12T03:02:37.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.945+0000 7f2588961640 1 -- 172.21.15.116:0/3926830927 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f256c0109d0 con 0x7f2584102e00 2024-09-12T03:02:37.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.946+0000 7f2588961640 1 --2- 172.21.15.116:0/3926830927 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f25700a2bb0 0x7f25700a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:37.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.946+0000 7f2582ffd640 1 --2- 172.21.15.116:0/3926830927 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f25700a2bb0 0x7f25700a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:37.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.946+0000 7f2588961640 1 -- 172.21.15.116:0/3926830927 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f256c0cec10 con 0x7f2584102e00 2024-09-12T03:02:37.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.947+0000 7f2582ffd640 1 --2- 172.21.15.116:0/3926830927 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f25700a2bb0 0x7f25700a5070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2584102c60 tx=0x7f2574008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:37.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:37.949+0000 7f2588961640 1 -- 172.21.15.116:0/3926830927 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f256c09bba0 con 0x7f2584102e00 2024-09-12T03:02:38.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:38.123+0000 7f2589963640 1 -- 172.21.15.116:0/3926830927 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f2584062630 con 0x7f25700a2bb0 2024-09-12T03:02:38.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:38.124+0000 7f2588961640 1 -- 172.21.15.116:0/3926830927 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f2584062630 con 0x7f25700a2bb0 2024-09-12T03:02:38.125 INFO:teuthology.orchestra.run.smithi116.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-12T03:02:38.125 INFO:teuthology.orchestra.run.smithi116.stdout:smithi116 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400VJ400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-12T03:02:38.125 INFO:teuthology.orchestra.run.smithi116.stdout:smithi177 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233007S400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-12T03:02:38.125 INFO:teuthology.orchestra.run.smithi116.stdout:smithi183 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500QJ400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-12T03:02:38.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:38.127+0000 7f2589963640 1 -- 172.21.15.116:0/3926830927 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f25700a2bb0 msgr2=0x7f25700a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:38.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:38.127+0000 7f2589963640 1 --2- 172.21.15.116:0/3926830927 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f25700a2bb0 0x7f25700a5070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2584102c60 tx=0x7f2574008040 comp rx=0 tx=0).stop 2024-09-12T03:02:38.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:38.127+0000 7f2589963640 1 -- 172.21.15.116:0/3926830927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2584102e00 msgr2=0x7f2584198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:38.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:38.127+0000 7f2589963640 1 --2- 172.21.15.116:0/3926830927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2584102e00 0x7f2584198490 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f256c00d950 tx=0x7f256c00de10 comp rx=0 tx=0).stop 2024-09-12T03:02:38.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:38.127+0000 7f2589963640 1 -- 172.21.15.116:0/3926830927 shutdown_connections 2024-09-12T03:02:38.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:38.127+0000 7f2589963640 1 --2- 172.21.15.116:0/3926830927 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f25700a2bb0 0x7f25700a5070 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:38.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:38.127+0000 7f2589963640 1 --2- 172.21.15.116:0/3926830927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25841037c0 0x7f258419f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:38.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:38.127+0000 7f2589963640 1 --2- 172.21.15.116:0/3926830927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2584102e00 0x7f2584198490 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:38.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:38.127+0000 7f2589963640 1 --2- 172.21.15.116:0/3926830927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2584101c00 0x7f2584197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:38.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:38.127+0000 7f2589963640 1 -- 172.21.15.116:0/3926830927 >> 172.21.15.116:0/3926830927 conn(0x7f25840fd390 msgr2=0x7f2584106130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:38.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:38.127+0000 7f2589963640 1 -- 172.21.15.116:0/3926830927 shutdown_connections 2024-09-12T03:02:38.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:38.128+0000 7f2589963640 1 -- 172.21.15.116:0/3926830927 wait complete. 2024-09-12T03:02:38.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:38 smithi183 ceph-mon[35535]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:02:38.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:38 smithi183 ceph-mon[35535]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T03:02:38.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:38 smithi177 ceph-mon[35305]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:02:38.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:38 smithi177 ceph-mon[35305]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T03:02:38.497 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:38 smithi116 ceph-mon[31410]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:02:38.498 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:38 smithi116 ceph-mon[31410]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T03:02:38.842 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-12T03:02:38.851 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi116.front.sepia.ceph.com 2024-09-12T03:02:38.852 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-12T03:02:39.098 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:39.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:39 smithi183 ceph-mon[35535]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:02:39.493 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:39 smithi116 ceph-mon[31410]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:02:39.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:39 smithi177 ceph-mon[35305]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:02:39.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.571+0000 7f68c39ce640 1 -- 172.21.15.116:0/493886579 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68bc1026c0 msgr2=0x7f68bc104ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:39.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.571+0000 7f68c39ce640 1 --2- 172.21.15.116:0/493886579 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68bc1026c0 0x7f68bc104ab0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009ca0 tx=0x7f68ac02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:02:39.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.572+0000 7f68c39ce640 1 -- 172.21.15.116:0/493886579 shutdown_connections 2024-09-12T03:02:39.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.572+0000 7f68c39ce640 1 --2- 172.21.15.116:0/493886579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68bc107920 0x7f68bc109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:39.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.572+0000 7f68c39ce640 1 --2- 172.21.15.116:0/493886579 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68bc104ff0 0x7f68bc1073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:39.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.572+0000 7f68c39ce640 1 --2- 172.21.15.116:0/493886579 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68bc1026c0 0x7f68bc104ab0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:39.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.572+0000 7f68c39ce640 1 -- 172.21.15.116:0/493886579 >> 172.21.15.116:0/493886579 conn(0x7f68bc0fc280 msgr2=0x7f68bc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:39.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.572+0000 7f68c39ce640 1 -- 172.21.15.116:0/493886579 shutdown_connections 2024-09-12T03:02:39.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.572+0000 7f68c39ce640 1 -- 172.21.15.116:0/493886579 wait complete. 2024-09-12T03:02:39.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.572+0000 7f68c39ce640 1 Processor -- start 2024-09-12T03:02:39.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.572+0000 7f68c39ce640 1 -- start start 2024-09-12T03:02:39.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.572+0000 7f68c39ce640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68bc1026c0 0x7f68bc197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:39.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.572+0000 7f68c39ce640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68bc104ff0 0x7f68bc198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:39.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.572+0000 7f68c39ce640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68bc107920 0x7f68bc19f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:39.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.572+0000 7f68c39ce640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc10d8e0 con 0x7f68bc107920 2024-09-12T03:02:39.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.572+0000 7f68c39ce640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc10d760 con 0x7f68bc104ff0 2024-09-12T03:02:39.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.572+0000 7f68c39ce640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc10da60 con 0x7f68bc1026c0 2024-09-12T03:02:39.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.572+0000 7f68c1f6b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68bc107920 0x7f68bc19f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:39.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.573+0000 7f68c1f6b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68bc107920 0x7f68bc19f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47114/0 (socket says 172.21.15.116:47114) 2024-09-12T03:02:39.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.573+0000 7f68c1f6b640 1 -- 172.21.15.116:0/685085725 learned_addr learned my addr 172.21.15.116:0/685085725 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:39.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.573+0000 7f68c1f6b640 1 -- 172.21.15.116:0/685085725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68bc1026c0 msgr2=0x7f68bc197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:39.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.573+0000 7f68c0f69640 1 --2- 172.21.15.116:0/685085725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68bc104ff0 0x7f68bc198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:39.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.573+0000 7f68c1f6b640 1 --2- 172.21.15.116:0/685085725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68bc1026c0 0x7f68bc197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:39.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.573+0000 7f68c1f6b640 1 -- 172.21.15.116:0/685085725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68bc104ff0 msgr2=0x7f68bc198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:39.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.573+0000 7f68c1f6b640 1 --2- 172.21.15.116:0/685085725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68bc104ff0 0x7f68bc198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:39.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.573+0000 7f68c1f6b640 1 -- 172.21.15.116:0/685085725 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68bc19fb00 con 0x7f68bc107920 2024-09-12T03:02:39.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.573+0000 7f68c1f6b640 1 --2- 172.21.15.116:0/685085725 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68bc107920 0x7f68bc19f490 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f68b4009fd0 tx=0x7f68b400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:39.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.573+0000 7f68ba7fc640 1 -- 172.21.15.116:0/685085725 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b4011830 con 0x7f68bc107920 2024-09-12T03:02:39.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.573+0000 7f68ba7fc640 1 -- 172.21.15.116:0/685085725 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68b4016070 con 0x7f68bc107920 2024-09-12T03:02:39.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.573+0000 7f68c39ce640 1 -- 172.21.15.116:0/685085725 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68bc19fdf0 con 0x7f68bc107920 2024-09-12T03:02:39.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.573+0000 7f68ba7fc640 1 -- 172.21.15.116:0/685085725 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b400f2e0 con 0x7f68bc107920 2024-09-12T03:02:39.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.573+0000 7f68c39ce640 1 -- 172.21.15.116:0/685085725 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68bc1a0320 con 0x7f68bc107920 2024-09-12T03:02:39.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.574+0000 7f68c39ce640 1 -- 172.21.15.116:0/685085725 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68bc0fdd70 con 0x7f68bc107920 2024-09-12T03:02:39.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.575+0000 7f68ba7fc640 1 -- 172.21.15.116:0/685085725 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f68b40119d0 con 0x7f68bc107920 2024-09-12T03:02:39.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.576+0000 7f68ba7fc640 1 --2- 172.21.15.116:0/685085725 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f68a40a2bb0 0x7f68a40a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:39.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.576+0000 7f68c176a640 1 --2- 172.21.15.116:0/685085725 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f68a40a2bb0 0x7f68a40a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:39.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.576+0000 7f68ba7fc640 1 -- 172.21.15.116:0/685085725 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f68b40ce680 con 0x7f68bc107920 2024-09-12T03:02:39.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.576+0000 7f68c176a640 1 --2- 172.21.15.116:0/685085725 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f68a40a2bb0 0x7f68a40a5070 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f68ac005e30 tx=0x7f68ac004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:39.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.580+0000 7f68ba7fc640 1 -- 172.21.15.116:0/685085725 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f68b409b610 con 0x7f68bc107920 2024-09-12T03:02:39.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.781+0000 7f68c39ce640 1 -- 172.21.15.116:0/685085725 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f68bc062df0 con 0x7f68bc107920 2024-09-12T03:02:39.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.782+0000 7f68ba7fc640 1 -- 172.21.15.116:0/685085725 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3104 (secure 0 0 0) 0x7f68b409af10 con 0x7f68bc107920 2024-09-12T03:02:39.782 INFO:teuthology.orchestra.run.smithi116.stdout:e24 2024-09-12T03:02:39.783 INFO:teuthology.orchestra.run.smithi116.stdout:btime 2024-09-12T03:01:10:397269+0000 2024-09-12T03:02:39.783 INFO:teuthology.orchestra.run.smithi116.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T03:02:39.783 INFO:teuthology.orchestra.run.smithi116.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T03:02:39.783 INFO:teuthology.orchestra.run.smithi116.stdout:legacy client fscid: 1 2024-09-12T03:02:39.783 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:02:39.783 INFO:teuthology.orchestra.run.smithi116.stdout:Filesystem 'cephfs' (1) 2024-09-12T03:02:39.783 INFO:teuthology.orchestra.run.smithi116.stdout:fs_name cephfs 2024-09-12T03:02:39.783 INFO:teuthology.orchestra.run.smithi116.stdout:epoch 24 2024-09-12T03:02:39.783 INFO:teuthology.orchestra.run.smithi116.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-12T03:02:39.783 INFO:teuthology.orchestra.run.smithi116.stdout:created 2024-09-12T03:00:59.195360+0000 2024-09-12T03:02:39.783 INFO:teuthology.orchestra.run.smithi116.stdout:modified 2024-09-12T03:01:10.218314+0000 2024-09-12T03:02:39.783 INFO:teuthology.orchestra.run.smithi116.stdout:tableserver 0 2024-09-12T03:02:39.783 INFO:teuthology.orchestra.run.smithi116.stdout:root 0 2024-09-12T03:02:39.784 INFO:teuthology.orchestra.run.smithi116.stdout:session_timeout 300 2024-09-12T03:02:39.784 INFO:teuthology.orchestra.run.smithi116.stdout:session_autoclose 300 2024-09-12T03:02:39.784 INFO:teuthology.orchestra.run.smithi116.stdout:max_file_size 1099511627776 2024-09-12T03:02:39.784 INFO:teuthology.orchestra.run.smithi116.stdout:max_xattr_size 65536 2024-09-12T03:02:39.784 INFO:teuthology.orchestra.run.smithi116.stdout:required_client_features {} 2024-09-12T03:02:39.784 INFO:teuthology.orchestra.run.smithi116.stdout:last_failure 0 2024-09-12T03:02:39.784 INFO:teuthology.orchestra.run.smithi116.stdout:last_failure_osd_epoch 0 2024-09-12T03:02:39.784 INFO:teuthology.orchestra.run.smithi116.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T03:02:39.784 INFO:teuthology.orchestra.run.smithi116.stdout:max_mds 3 2024-09-12T03:02:39.784 INFO:teuthology.orchestra.run.smithi116.stdout:in 0,1,2 2024-09-12T03:02:39.784 INFO:teuthology.orchestra.run.smithi116.stdout:up {0=24494,1=24475,2=24467} 2024-09-12T03:02:39.784 INFO:teuthology.orchestra.run.smithi116.stdout:failed 2024-09-12T03:02:39.784 INFO:teuthology.orchestra.run.smithi116.stdout:damaged 2024-09-12T03:02:39.784 INFO:teuthology.orchestra.run.smithi116.stdout:stopped 2024-09-12T03:02:39.785 INFO:teuthology.orchestra.run.smithi116.stdout:data_pools [3] 2024-09-12T03:02:39.785 INFO:teuthology.orchestra.run.smithi116.stdout:metadata_pool 2 2024-09-12T03:02:39.785 INFO:teuthology.orchestra.run.smithi116.stdout:inline_data disabled 2024-09-12T03:02:39.785 INFO:teuthology.orchestra.run.smithi116.stdout:balancer 2024-09-12T03:02:39.785 INFO:teuthology.orchestra.run.smithi116.stdout:bal_rank_mask -1 2024-09-12T03:02:39.785 INFO:teuthology.orchestra.run.smithi116.stdout:standby_count_wanted 0 2024-09-12T03:02:39.785 INFO:teuthology.orchestra.run.smithi116.stdout:qdb_cluster leader: 24494 members: 24467,24475,24494 2024-09-12T03:02:39.785 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b{0:24494} state up:active seq 4 addr [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:02:39.785 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c{0:24454} state up:standby-replay seq 1 addr [v2:172.21.15.183:6836/1288373243,v1:172.21.15.183:6837/1288373243] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:02:39.785 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.i{1:24475} state up:active seq 8 addr [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:02:39.785 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.f{1:24448} state up:standby-replay seq 1 addr [v2:172.21.15.183:6834/4028826399,v1:172.21.15.183:6835/4028826399] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:02:39.785 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.h{2:24467} state up:active seq 9 addr [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:02:39.785 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.e{2:24443} state up:standby-replay seq 1 addr [v2:172.21.15.177:6834/2224643300,v1:172.21.15.177:6835/2224643300] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:02:39.786 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:02:39.786 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:02:39.786 INFO:teuthology.orchestra.run.smithi116.stdout:Standby daemons: 2024-09-12T03:02:39.786 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:02:39.786 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.j{-1:14547} state up:standby seq 1 addr [v2:172.21.15.116:6834/3909394246,v1:172.21.15.116:6835/3909394246] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:02:39.786 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d{-1:14559} state up:standby seq 1 addr [v2:172.21.15.116:6836/3839272388,v1:172.21.15.116:6837/3839272388] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:02:39.786 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.g{-1:14604} state up:standby seq 1 addr [v2:172.21.15.116:6838/1740752718,v1:172.21.15.116:6839/1740752718] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:02:39.786 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a{-1:14616} state up:standby seq 1 addr [v2:172.21.15.116:6840/2495398640,v1:172.21.15.116:6841/2495398640] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:02:39.786 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.k{-1:24410} state up:standby seq 1 addr [v2:172.21.15.177:6832/3132380205,v1:172.21.15.177:6833/3132380205] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:02:39.786 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.l{-1:24415} state up:standby seq 1 addr [v2:172.21.15.183:6832/1068409597,v1:172.21.15.183:6833/1068409597] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:02:39.787 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 24 2024-09-12T03:02:39.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.784+0000 7f68c39ce640 1 -- 172.21.15.116:0/685085725 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f68a40a2bb0 msgr2=0x7f68a40a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:39.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.784+0000 7f68c39ce640 1 --2- 172.21.15.116:0/685085725 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f68a40a2bb0 0x7f68a40a5070 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f68ac005e30 tx=0x7f68ac004a40 comp rx=0 tx=0).stop 2024-09-12T03:02:39.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.784+0000 7f68c39ce640 1 -- 172.21.15.116:0/685085725 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68bc107920 msgr2=0x7f68bc19f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:39.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.784+0000 7f68c39ce640 1 --2- 172.21.15.116:0/685085725 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68bc107920 0x7f68bc19f490 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f68b4009fd0 tx=0x7f68b400b8e0 comp rx=0 tx=0).stop 2024-09-12T03:02:39.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.784+0000 7f68c39ce640 1 -- 172.21.15.116:0/685085725 shutdown_connections 2024-09-12T03:02:39.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.784+0000 7f68c39ce640 1 --2- 172.21.15.116:0/685085725 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f68a40a2bb0 0x7f68a40a5070 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:39.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.784+0000 7f68c39ce640 1 --2- 172.21.15.116:0/685085725 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68bc107920 0x7f68bc19f490 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:39.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.784+0000 7f68c39ce640 1 --2- 172.21.15.116:0/685085725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68bc104ff0 0x7f68bc198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:39.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.784+0000 7f68c39ce640 1 --2- 172.21.15.116:0/685085725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68bc1026c0 0x7f68bc197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:39.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.784+0000 7f68c39ce640 1 -- 172.21.15.116:0/685085725 >> 172.21.15.116:0/685085725 conn(0x7f68bc0fc280 msgr2=0x7f68bc102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:39.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.784+0000 7f68c39ce640 1 -- 172.21.15.116:0/685085725 shutdown_connections 2024-09-12T03:02:39.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:39.784+0000 7f68c39ce640 1 -- 172.21.15.116:0/685085725 wait complete. 2024-09-12T03:02:40.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:40 smithi183 ceph-mon[35535]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T03:02:40.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:40 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/685085725' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T03:02:40.481 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:40 smithi116 ceph-mon[31410]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T03:02:40.482 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:40 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/685085725' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T03:02:40.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:40 smithi177 ceph-mon[35305]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T03:02:40.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:40 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/685085725' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T03:02:40.511 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-12T03:02:40.756 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:41.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.382+0000 7f250a0b7640 1 -- 172.21.15.116:0/3022820005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2504100b10 msgr2=0x7f2504100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:41.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.382+0000 7f250a0b7640 1 --2- 172.21.15.116:0/3022820005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2504100b10 0x7f2504100f10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f24f4009900 tx=0x7f24f402a410 comp rx=0 tx=0).stop 2024-09-12T03:02:41.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.382+0000 7f250a0b7640 1 -- 172.21.15.116:0/3022820005 shutdown_connections 2024-09-12T03:02:41.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.382+0000 7f250a0b7640 1 --2- 172.21.15.116:0/3022820005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f25041026d0 0x7f2504108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:41.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.382+0000 7f250a0b7640 1 --2- 172.21.15.116:0/3022820005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2504101d10 0x7f2504102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:41.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.382+0000 7f250a0b7640 1 --2- 172.21.15.116:0/3022820005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2504100b10 0x7f2504100f10 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:41.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.382+0000 7f250a0b7640 1 -- 172.21.15.116:0/3022820005 >> 172.21.15.116:0/3022820005 conn(0x7f25040fc2a0 msgr2=0x7f25040fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:41.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.382+0000 7f250a0b7640 1 -- 172.21.15.116:0/3022820005 shutdown_connections 2024-09-12T03:02:41.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.382+0000 7f250a0b7640 1 -- 172.21.15.116:0/3022820005 wait complete. 2024-09-12T03:02:41.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.382+0000 7f250a0b7640 1 Processor -- start 2024-09-12T03:02:41.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.383+0000 7f250a0b7640 1 -- start start 2024-09-12T03:02:41.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.383+0000 7f250a0b7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2504100b10 0x7f250406b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:41.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.383+0000 7f250a0b7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2504101d10 0x7f250419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:41.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.383+0000 7f250a0b7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25041026d0 0x7f250419f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:41.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.383+0000 7f250a0b7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f250410c7f0 con 0x7f2504100b10 2024-09-12T03:02:41.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.383+0000 7f250a0b7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f250410c670 con 0x7f2504101d10 2024-09-12T03:02:41.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.383+0000 7f250a0b7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f250410c970 con 0x7f25041026d0 2024-09-12T03:02:41.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.383+0000 7f25037fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2504100b10 0x7f250406b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:41.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.383+0000 7f25037fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2504100b10 0x7f250406b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33466/0 (socket says 172.21.15.116:33466) 2024-09-12T03:02:41.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.383+0000 7f25037fe640 1 -- 172.21.15.116:0/1082242446 learned_addr learned my addr 172.21.15.116:0/1082242446 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:41.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.383+0000 7f25037fe640 1 -- 172.21.15.116:0/1082242446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25041026d0 msgr2=0x7f250419f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:41.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.383+0000 7f2503fff640 1 --2- 172.21.15.116:0/1082242446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25041026d0 0x7f250419f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:41.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.383+0000 7f25037fe640 1 --2- 172.21.15.116:0/1082242446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25041026d0 0x7f250419f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:41.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.383+0000 7f25037fe640 1 -- 172.21.15.116:0/1082242446 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2504101d10 msgr2=0x7f250419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:41.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.383+0000 7f25037fe640 1 --2- 172.21.15.116:0/1082242446 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2504101d10 0x7f250419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:41.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.383+0000 7f25037fe640 1 -- 172.21.15.116:0/1082242446 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250419fc00 con 0x7f2504100b10 2024-09-12T03:02:41.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.383+0000 7f2503fff640 1 --2- 172.21.15.116:0/1082242446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25041026d0 0x7f250419f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:41.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.383+0000 7f25037fe640 1 --2- 172.21.15.116:0/1082242446 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2504100b10 0x7f250406b2f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f24f4031610 tx=0x7f24f40043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:41.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.383+0000 7f2500ff9640 1 -- 172.21.15.116:0/1082242446 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f403f030 con 0x7f2504100b10 2024-09-12T03:02:41.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.383+0000 7f2500ff9640 1 -- 172.21.15.116:0/1082242446 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24f4003ce0 con 0x7f2504100b10 2024-09-12T03:02:41.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.383+0000 7f250a0b7640 1 -- 172.21.15.116:0/1082242446 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f250419fe30 con 0x7f2504100b10 2024-09-12T03:02:41.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.383+0000 7f2500ff9640 1 -- 172.21.15.116:0/1082242446 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f4035070 con 0x7f2504100b10 2024-09-12T03:02:41.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.384+0000 7f250a0b7640 1 -- 172.21.15.116:0/1082242446 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25041a0300 con 0x7f2504100b10 2024-09-12T03:02:41.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.385+0000 7f250a0b7640 1 -- 172.21.15.116:0/1082242446 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2504107f80 con 0x7f2504100b10 2024-09-12T03:02:41.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.386+0000 7f2500ff9640 1 -- 172.21.15.116:0/1082242446 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f24f4042030 con 0x7f2504100b10 2024-09-12T03:02:41.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.386+0000 7f2500ff9640 1 --2- 172.21.15.116:0/1082242446 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f24ec0a2b80 0x7f24ec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:41.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.386+0000 7f2502ffd640 1 --2- 172.21.15.116:0/1082242446 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f24ec0a2b80 0x7f24ec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:41.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.386+0000 7f2500ff9640 1 -- 172.21.15.116:0/1082242446 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f24f40ef660 con 0x7f2504100b10 2024-09-12T03:02:41.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.387+0000 7f2502ffd640 1 --2- 172.21.15.116:0/1082242446 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f24ec0a2b80 0x7f24ec0a5040 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f24f8005fd0 tx=0x7f24f8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:41.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.390+0000 7f2500ff9640 1 -- 172.21.15.116:0/1082242446 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f24f40bc5f0 con 0x7f2504100b10 2024-09-12T03:02:41.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.558+0000 7f250a0b7640 1 -- 172.21.15.116:0/1082242446 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f250402d7c0 con 0x7f2504100b10 2024-09-12T03:02:41.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.558+0000 7f2500ff9640 1 -- 172.21.15.116:0/1082242446 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v80) ==== 56+0+5274 (secure 0 0 0) 0x7f24f40bbef0 con 0x7f2504100b10 2024-09-12T03:02:41.559 INFO:teuthology.orchestra.run.smithi116.stdout:epoch 80 2024-09-12T03:02:41.559 INFO:teuthology.orchestra.run.smithi116.stdout:fsid eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T03:02:41.559 INFO:teuthology.orchestra.run.smithi116.stdout:created 2024-09-12T02:53:07.105686+0000 2024-09-12T03:02:41.559 INFO:teuthology.orchestra.run.smithi116.stdout:modified 2024-09-12T03:02:04.272898+0000 2024-09-12T03:02:41.559 INFO:teuthology.orchestra.run.smithi116.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-12T03:02:41.559 INFO:teuthology.orchestra.run.smithi116.stdout:crush_version 26 2024-09-12T03:02:41.559 INFO:teuthology.orchestra.run.smithi116.stdout:full_ratio 0.95 2024-09-12T03:02:41.559 INFO:teuthology.orchestra.run.smithi116.stdout:backfillfull_ratio 0.9 2024-09-12T03:02:41.559 INFO:teuthology.orchestra.run.smithi116.stdout:nearfull_ratio 0.85 2024-09-12T03:02:41.560 INFO:teuthology.orchestra.run.smithi116.stdout:require_min_compat_client luminous 2024-09-12T03:02:41.560 INFO:teuthology.orchestra.run.smithi116.stdout:min_compat_client luminous 2024-09-12T03:02:41.560 INFO:teuthology.orchestra.run.smithi116.stdout:require_osd_release squid 2024-09-12T03:02:41.560 INFO:teuthology.orchestra.run.smithi116.stdout:stretch_mode_enabled false 2024-09-12T03:02:41.560 INFO:teuthology.orchestra.run.smithi116.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 19 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-12T03:02:41.560 INFO:teuthology.orchestra.run.smithi116.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-12T03:02:41.560 INFO:teuthology.orchestra.run.smithi116.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-12T03:02:41.560 INFO:teuthology.orchestra.run.smithi116.stdout:max_osd 12 2024-09-12T03:02:41.560 INFO:teuthology.orchestra.run.smithi116.stdout:osd.0 up in weight 1 up_from 8 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.116:6802/862366853,v1:172.21.15.116:6803/862366853] [v2:172.21.15.116:6804/862366853,v1:172.21.15.116:6805/862366853] exists,up c8cf5a18-dc6f-4bb6-bd74-78adda310118 2024-09-12T03:02:41.560 INFO:teuthology.orchestra.run.smithi116.stdout:osd.1 up in weight 1 up_from 12 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.177:6800/2245800547,v1:172.21.15.177:6801/2245800547] [v2:172.21.15.177:6802/2245800547,v1:172.21.15.177:6803/2245800547] exists,up 0affd3cb-9a51-42b0-a2cc-f30784a75cf1 2024-09-12T03:02:41.560 INFO:teuthology.orchestra.run.smithi116.stdout:osd.2 up in weight 1 up_from 16 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6800/2645677759,v1:172.21.15.183:6801/2645677759] [v2:172.21.15.183:6802/2645677759,v1:172.21.15.183:6803/2645677759] exists,up 1ce45ea5-84df-4a35-924f-78f92bd4f2cd 2024-09-12T03:02:41.561 INFO:teuthology.orchestra.run.smithi116.stdout:osd.3 up in weight 1 up_from 22 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.116:6810/686622148,v1:172.21.15.116:6811/686622148] [v2:172.21.15.116:6812/686622148,v1:172.21.15.116:6813/686622148] exists,up ae5e8178-c998-4648-bc05-4bce822f0cd3 2024-09-12T03:02:41.561 INFO:teuthology.orchestra.run.smithi116.stdout:osd.4 up in weight 1 up_from 29 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.177:6808/2382938021,v1:172.21.15.177:6809/2382938021] [v2:172.21.15.177:6810/2382938021,v1:172.21.15.177:6811/2382938021] exists,up c81d2e73-0bc2-440d-b5a2-b7428af3ea4d 2024-09-12T03:02:41.561 INFO:teuthology.orchestra.run.smithi116.stdout:osd.5 up in weight 1 up_from 33 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6808/85363432,v1:172.21.15.183:6809/85363432] [v2:172.21.15.183:6810/85363432,v1:172.21.15.183:6811/85363432] exists,up c3f168be-9e27-4cf5-b73e-3f47d8990ed3 2024-09-12T03:02:41.561 INFO:teuthology.orchestra.run.smithi116.stdout:osd.6 up in weight 1 up_from 40 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.116:6818/1121291186,v1:172.21.15.116:6819/1121291186] [v2:172.21.15.116:6820/1121291186,v1:172.21.15.116:6821/1121291186] exists,up beb0a7e9-4909-47e2-9ee7-6951d4745e5d 2024-09-12T03:02:41.561 INFO:teuthology.orchestra.run.smithi116.stdout:osd.7 up in weight 1 up_from 45 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.177:6816/2177665961,v1:172.21.15.177:6817/2177665961] [v2:172.21.15.177:6818/2177665961,v1:172.21.15.177:6819/2177665961] exists,up 761f4680-f049-473c-bf44-b19acf8de24f 2024-09-12T03:02:41.561 INFO:teuthology.orchestra.run.smithi116.stdout:osd.8 up in weight 1 up_from 52 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6816/1305725258,v1:172.21.15.183:6817/1305725258] [v2:172.21.15.183:6818/1305725258,v1:172.21.15.183:6819/1305725258] exists,up 2385ea6c-fd5a-4815-9596-77f948629542 2024-09-12T03:02:41.561 INFO:teuthology.orchestra.run.smithi116.stdout:osd.9 up in weight 1 up_from 59 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.116:6826/3831673668,v1:172.21.15.116:6827/3831673668] [v2:172.21.15.116:6828/3831673668,v1:172.21.15.116:6829/3831673668] exists,up fd0c0dc4-4990-4f12-aa14-f855504cd995 2024-09-12T03:02:41.561 INFO:teuthology.orchestra.run.smithi116.stdout:osd.10 up in weight 1 up_from 64 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.177:6824/963663634,v1:172.21.15.177:6825/963663634] [v2:172.21.15.177:6826/963663634,v1:172.21.15.177:6827/963663634] exists,up 864d1a97-4810-49c8-ad4a-a0c10253833c 2024-09-12T03:02:41.561 INFO:teuthology.orchestra.run.smithi116.stdout:osd.11 up in weight 1 up_from 72 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6824/1818319939,v1:172.21.15.183:6825/1818319939] [v2:172.21.15.183:6826/1818319939,v1:172.21.15.183:6827/1818319939] exists,up d49c0841-4620-4f2f-a6b5-f2a9aa2e3c95 2024-09-12T03:02:41.561 INFO:teuthology.orchestra.run.smithi116.stdout:pg_upmap_items 2.a [2,11] 2024-09-12T03:02:41.562 INFO:teuthology.orchestra.run.smithi116.stdout:pg_upmap_items 2.f [8,11] 2024-09-12T03:02:41.562 INFO:teuthology.orchestra.run.smithi116.stdout:pg_upmap_items 3.c [3,9] 2024-09-12T03:02:41.562 INFO:teuthology.orchestra.run.smithi116.stdout:pg_upmap_items 3.e [4,6] 2024-09-12T03:02:41.562 INFO:teuthology.orchestra.run.smithi116.stdout:pg_upmap_items 3.11 [4,8] 2024-09-12T03:02:41.562 INFO:teuthology.orchestra.run.smithi116.stdout:pg_upmap_items 3.1b [5,9] 2024-09-12T03:02:41.562 INFO:teuthology.orchestra.run.smithi116.stdout:pg_upmap_items 3.2c [4,9] 2024-09-12T03:02:41.562 INFO:teuthology.orchestra.run.smithi116.stdout:blocklist 172.21.15.116:6801/2966274423 expires 2024-09-13T02:54:03.026862+0000 2024-09-12T03:02:41.562 INFO:teuthology.orchestra.run.smithi116.stdout:blocklist 172.21.15.116:6800/2966274423 expires 2024-09-13T02:54:03.026862+0000 2024-09-12T03:02:41.563 INFO:teuthology.orchestra.run.smithi116.stdout:blocklist 172.21.15.116:0/686139455 expires 2024-09-13T02:54:03.026862+0000 2024-09-12T03:02:41.563 INFO:teuthology.orchestra.run.smithi116.stdout:blocklist 172.21.15.116:0/1552047390 expires 2024-09-13T02:54:03.026862+0000 2024-09-12T03:02:41.563 INFO:teuthology.orchestra.run.smithi116.stdout:blocklist 172.21.15.116:0/2180363411 expires 2024-09-13T02:53:39.433769+0000 2024-09-12T03:02:41.563 INFO:teuthology.orchestra.run.smithi116.stdout:blocklist 172.21.15.116:6801/1791219601 expires 2024-09-13T02:53:39.433769+0000 2024-09-12T03:02:41.563 INFO:teuthology.orchestra.run.smithi116.stdout:blocklist 172.21.15.116:0/3907400460 expires 2024-09-13T02:54:03.026862+0000 2024-09-12T03:02:41.563 INFO:teuthology.orchestra.run.smithi116.stdout:blocklist 172.21.15.116:6800/1791219601 expires 2024-09-13T02:53:39.433769+0000 2024-09-12T03:02:41.563 INFO:teuthology.orchestra.run.smithi116.stdout:blocklist 172.21.15.116:0/1802566286 expires 2024-09-13T02:53:39.433769+0000 2024-09-12T03:02:41.563 INFO:teuthology.orchestra.run.smithi116.stdout:blocklist 172.21.15.116:0/212337802 expires 2024-09-13T02:53:39.433769+0000 2024-09-12T03:02:41.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.561+0000 7f250a0b7640 1 -- 172.21.15.116:0/1082242446 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f24ec0a2b80 msgr2=0x7f24ec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:41.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.561+0000 7f250a0b7640 1 --2- 172.21.15.116:0/1082242446 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f24ec0a2b80 0x7f24ec0a5040 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f24f8005fd0 tx=0x7f24f8005850 comp rx=0 tx=0).stop 2024-09-12T03:02:41.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.561+0000 7f250a0b7640 1 -- 172.21.15.116:0/1082242446 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2504100b10 msgr2=0x7f250406b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:41.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.561+0000 7f250a0b7640 1 --2- 172.21.15.116:0/1082242446 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2504100b10 0x7f250406b2f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f24f4031610 tx=0x7f24f40043b0 comp rx=0 tx=0).stop 2024-09-12T03:02:41.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.561+0000 7f250a0b7640 1 -- 172.21.15.116:0/1082242446 shutdown_connections 2024-09-12T03:02:41.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.561+0000 7f250a0b7640 1 --2- 172.21.15.116:0/1082242446 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f24ec0a2b80 0x7f24ec0a5040 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:41.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.561+0000 7f250a0b7640 1 --2- 172.21.15.116:0/1082242446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25041026d0 0x7f250419f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:41.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.561+0000 7f250a0b7640 1 --2- 172.21.15.116:0/1082242446 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2504101d10 0x7f250419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:41.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.561+0000 7f250a0b7640 1 --2- 172.21.15.116:0/1082242446 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2504100b10 0x7f250406b2f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:41.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.561+0000 7f250a0b7640 1 -- 172.21.15.116:0/1082242446 >> 172.21.15.116:0/1082242446 conn(0x7f25040fc2a0 msgr2=0x7f2504105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:41.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.561+0000 7f250a0b7640 1 -- 172.21.15.116:0/1082242446 shutdown_connections 2024-09-12T03:02:41.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:41.561+0000 7f250a0b7640 1 -- 172.21.15.116:0/1082242446 wait complete. 2024-09-12T03:02:42.353 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-12T03:02:42.365 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:02:42.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:42 smithi183 ceph-mon[35535]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T03:02:42.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:42 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1082242446' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T03:02:42.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:42 smithi177 ceph-mon[35305]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T03:02:42.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:42 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1082242446' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T03:02:42.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:42 smithi116 ceph-mon[31410]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T03:02:42.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:42 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1082242446' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T03:02:42.613 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:43.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.089+0000 7f2e4e758640 1 -- 172.21.15.116:0/3244654120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e48102d40 msgr2=0x7f2e48103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:43.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.089+0000 7f2e4e758640 1 --2- 172.21.15.116:0/3244654120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e48102d40 0x7f2e48103120 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009900 tx=0x7f2e3802a480 comp rx=0 tx=0).stop 2024-09-12T03:02:43.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.089+0000 7f2e4e758640 1 -- 172.21.15.116:0/3244654120 shutdown_connections 2024-09-12T03:02:43.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.089+0000 7f2e4e758640 1 --2- 172.21.15.116:0/3244654120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e4810c170 0x7f2e4810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:43.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.089+0000 7f2e4e758640 1 --2- 172.21.15.116:0/3244654120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e48103660 0x7f2e4810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:43.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.089+0000 7f2e4e758640 1 --2- 172.21.15.116:0/3244654120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e48102d40 0x7f2e48103120 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:43.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.089+0000 7f2e4e758640 1 -- 172.21.15.116:0/3244654120 >> 172.21.15.116:0/3244654120 conn(0x7f2e480fc7a0 msgr2=0x7f2e480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:43.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.089+0000 7f2e4e758640 1 -- 172.21.15.116:0/3244654120 shutdown_connections 2024-09-12T03:02:43.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.089+0000 7f2e4e758640 1 -- 172.21.15.116:0/3244654120 wait complete. 2024-09-12T03:02:43.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.090+0000 7f2e4e758640 1 Processor -- start 2024-09-12T03:02:43.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.090+0000 7f2e4e758640 1 -- start start 2024-09-12T03:02:43.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.090+0000 7f2e4e758640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e48102d40 0x7f2e48198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:43.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.090+0000 7f2e4e758640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e48103660 0x7f2e481989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:43.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.090+0000 7f2e4e758640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e4810c170 0x7f2e4819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:43.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.090+0000 7f2e4e758640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2e48111fb0 con 0x7f2e48102d40 2024-09-12T03:02:43.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.090+0000 7f2e4e758640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2e48111e30 con 0x7f2e48103660 2024-09-12T03:02:43.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.090+0000 7f2e4e758640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2e48112130 con 0x7f2e4810c170 2024-09-12T03:02:43.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.090+0000 7f2e47fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e48102d40 0x7f2e48198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:43.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.090+0000 7f2e47fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e48102d40 0x7f2e48198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33488/0 (socket says 172.21.15.116:33488) 2024-09-12T03:02:43.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.090+0000 7f2e47fff640 1 -- 172.21.15.116:0/2168901342 learned_addr learned my addr 172.21.15.116:0/2168901342 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:43.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.090+0000 7f2e477fe640 1 --2- 172.21.15.116:0/2168901342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e48103660 0x7f2e481989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:43.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.090+0000 7f2e4ccf5640 1 --2- 172.21.15.116:0/2168901342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e4810c170 0x7f2e4819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:43.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.090+0000 7f2e47fff640 1 -- 172.21.15.116:0/2168901342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e4810c170 msgr2=0x7f2e4819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:43.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.090+0000 7f2e47fff640 1 --2- 172.21.15.116:0/2168901342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e4810c170 0x7f2e4819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:43.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.090+0000 7f2e47fff640 1 -- 172.21.15.116:0/2168901342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e48103660 msgr2=0x7f2e481989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:43.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.090+0000 7f2e47fff640 1 --2- 172.21.15.116:0/2168901342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e48103660 0x7f2e481989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:43.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.090+0000 7f2e47fff640 1 -- 172.21.15.116:0/2168901342 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4819d420 con 0x7f2e48102d40 2024-09-12T03:02:43.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.090+0000 7f2e4ccf5640 1 --2- 172.21.15.116:0/2168901342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e4810c170 0x7f2e4819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:43.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.091+0000 7f2e47fff640 1 --2- 172.21.15.116:0/2168901342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e48102d40 0x7f2e48198470 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2e38005ab0 tx=0x7f2e380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:43.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.091+0000 7f2e457fa640 1 -- 172.21.15.116:0/2168901342 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3803f030 con 0x7f2e48102d40 2024-09-12T03:02:43.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.091+0000 7f2e457fa640 1 -- 172.21.15.116:0/2168901342 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e38003ce0 con 0x7f2e48102d40 2024-09-12T03:02:43.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.091+0000 7f2e4e758640 1 -- 172.21.15.116:0/2168901342 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e4819d6b0 con 0x7f2e48102d40 2024-09-12T03:02:43.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.091+0000 7f2e457fa640 1 -- 172.21.15.116:0/2168901342 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38035070 con 0x7f2e48102d40 2024-09-12T03:02:43.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.091+0000 7f2e4e758640 1 -- 172.21.15.116:0/2168901342 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e481a4ff0 con 0x7f2e48102d40 2024-09-12T03:02:43.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.092+0000 7f2e4e758640 1 -- 172.21.15.116:0/2168901342 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e4806b070 con 0x7f2e48102d40 2024-09-12T03:02:43.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.093+0000 7f2e457fa640 1 -- 172.21.15.116:0/2168901342 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2e38042030 con 0x7f2e48102d40 2024-09-12T03:02:43.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.094+0000 7f2e457fa640 1 --2- 172.21.15.116:0/2168901342 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2e300a2b80 0x7f2e300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:43.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.094+0000 7f2e477fe640 1 --2- 172.21.15.116:0/2168901342 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2e300a2b80 0x7f2e300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:43.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.094+0000 7f2e457fa640 1 -- 172.21.15.116:0/2168901342 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f2e380ef920 con 0x7f2e48102d40 2024-09-12T03:02:43.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.094+0000 7f2e477fe640 1 --2- 172.21.15.116:0/2168901342 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2e300a2b80 0x7f2e300a5040 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2e4810d500 tx=0x7f2e3c00a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:43.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.098+0000 7f2e457fa640 1 -- 172.21.15.116:0/2168901342 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2e380bc8b0 con 0x7f2e48102d40 2024-09-12T03:02:43.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.300+0000 7f2e4e758640 1 -- 172.21.15.116:0/2168901342 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e48107210 con 0x7f2e48102d40 2024-09-12T03:02:43.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.301+0000 7f2e457fa640 1 -- 172.21.15.116:0/2168901342 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11938 (secure 0 0 0) 0x7f2e380bc1b0 con 0x7f2e48102d40 2024-09-12T03:02:43.332 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:02:43.332 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":24,"btime":"2024-09-12T03:01:10:397269+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:01:10.218314+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24467,24475,24494]},"id":1}]} 2024-09-12T03:02:43.333 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 24 2024-09-12T03:02:43.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.303+0000 7f2e4e758640 1 -- 172.21.15.116:0/2168901342 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2e300a2b80 msgr2=0x7f2e300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:43.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.303+0000 7f2e4e758640 1 --2- 172.21.15.116:0/2168901342 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2e300a2b80 0x7f2e300a5040 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2e4810d500 tx=0x7f2e3c00a250 comp rx=0 tx=0).stop 2024-09-12T03:02:43.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.303+0000 7f2e4e758640 1 -- 172.21.15.116:0/2168901342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e48102d40 msgr2=0x7f2e48198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:43.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.303+0000 7f2e4e758640 1 --2- 172.21.15.116:0/2168901342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e48102d40 0x7f2e48198470 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2e38005ab0 tx=0x7f2e380042f0 comp rx=0 tx=0).stop 2024-09-12T03:02:43.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.303+0000 7f2e4e758640 1 -- 172.21.15.116:0/2168901342 shutdown_connections 2024-09-12T03:02:43.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.303+0000 7f2e4e758640 1 --2- 172.21.15.116:0/2168901342 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2e300a2b80 0x7f2e300a5040 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:43.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.303+0000 7f2e4e758640 1 --2- 172.21.15.116:0/2168901342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e4810c170 0x7f2e4819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:43.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.303+0000 7f2e4e758640 1 --2- 172.21.15.116:0/2168901342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e48103660 0x7f2e481989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:43.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.303+0000 7f2e4e758640 1 --2- 172.21.15.116:0/2168901342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e48102d40 0x7f2e48198470 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:43.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.303+0000 7f2e4e758640 1 -- 172.21.15.116:0/2168901342 >> 172.21.15.116:0/2168901342 conn(0x7f2e480fc7a0 msgr2=0x7f2e480fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:43.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.303+0000 7f2e4e758640 1 -- 172.21.15.116:0/2168901342 shutdown_connections 2024-09-12T03:02:43.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:43.303+0000 7f2e4e758640 1 -- 172.21.15.116:0/2168901342 wait complete. 2024-09-12T03:02:44.139 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:02:44.389 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:44.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:44 smithi183 ceph-mon[35535]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T03:02:44.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:44 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2168901342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:44.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:44 smithi177 ceph-mon[35305]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T03:02:44.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:44 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2168901342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:44.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:44 smithi116 ceph-mon[31410]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T03:02:44.509 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:44 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2168901342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:44.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.863+0000 7f6f98861640 1 -- 172.21.15.116:0/79349260 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f94106f20 msgr2=0x7f6f9410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:44.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.863+0000 7f6f98861640 1 --2- 172.21.15.116:0/79349260 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f94106f20 0x7f6f9410eaa0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6f84009900 tx=0x7f6f8402a410 comp rx=0 tx=0).stop 2024-09-12T03:02:44.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.863+0000 7f6f98861640 1 -- 172.21.15.116:0/79349260 shutdown_connections 2024-09-12T03:02:44.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.863+0000 7f6f98861640 1 --2- 172.21.15.116:0/79349260 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f94106f20 0x7f6f9410eaa0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.863+0000 7f6f98861640 1 --2- 172.21.15.116:0/79349260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f941027a0 0x7f6f941067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.863+0000 7f6f98861640 1 --2- 172.21.15.116:0/79349260 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f94101df0 0x7f6f941021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.863+0000 7f6f98861640 1 -- 172.21.15.116:0/79349260 >> 172.21.15.116:0/79349260 conn(0x7f6f940fd8f0 msgr2=0x7f6f940ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:44.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.863+0000 7f6f98861640 1 -- 172.21.15.116:0/79349260 shutdown_connections 2024-09-12T03:02:44.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.863+0000 7f6f98861640 1 -- 172.21.15.116:0/79349260 wait complete. 2024-09-12T03:02:44.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.864+0000 7f6f98861640 1 Processor -- start 2024-09-12T03:02:44.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.864+0000 7f6f98861640 1 -- start start 2024-09-12T03:02:44.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.864+0000 7f6f98861640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f94101df0 0x7f6f9419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:44.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.864+0000 7f6f98861640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f941027a0 0x7f6f9419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:44.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.864+0000 7f6f98861640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f94106f20 0x7f6f9419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:44.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.864+0000 7f6f98861640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6f94111f80 con 0x7f6f941027a0 2024-09-12T03:02:44.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.864+0000 7f6f98861640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6f94111e00 con 0x7f6f94106f20 2024-09-12T03:02:44.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.864+0000 7f6f98861640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6f94112100 con 0x7f6f94101df0 2024-09-12T03:02:44.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.864+0000 7f6f91d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f941027a0 0x7f6f9419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:44.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.864+0000 7f6f91d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f941027a0 0x7f6f9419aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33496/0 (socket says 172.21.15.116:33496) 2024-09-12T03:02:44.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.864+0000 7f6f91d9b640 1 -- 172.21.15.116:0/3510288649 learned_addr learned my addr 172.21.15.116:0/3510288649 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:44.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.864+0000 7f6f9259c640 1 --2- 172.21.15.116:0/3510288649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f94101df0 0x7f6f9419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:44.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.864+0000 7f6f92d9d640 1 --2- 172.21.15.116:0/3510288649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f94106f20 0x7f6f9419ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:44.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.864+0000 7f6f91d9b640 1 -- 172.21.15.116:0/3510288649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f94101df0 msgr2=0x7f6f9419a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:44.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.864+0000 7f6f91d9b640 1 --2- 172.21.15.116:0/3510288649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f94101df0 0x7f6f9419a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.864+0000 7f6f91d9b640 1 -- 172.21.15.116:0/3510288649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f94106f20 msgr2=0x7f6f9419ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:44.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.864+0000 7f6f91d9b640 1 --2- 172.21.15.116:0/3510288649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f94106f20 0x7f6f9419ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.865+0000 7f6f91d9b640 1 -- 172.21.15.116:0/3510288649 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f9419f540 con 0x7f6f941027a0 2024-09-12T03:02:44.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.865+0000 7f6f91d9b640 1 --2- 172.21.15.116:0/3510288649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f941027a0 0x7f6f9419aad0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c00c920 tx=0x7f6f7c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:44.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.865+0000 7f6f8b7fe640 1 -- 172.21.15.116:0/3510288649 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c007c70 con 0x7f6f941027a0 2024-09-12T03:02:44.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.865+0000 7f6f8b7fe640 1 -- 172.21.15.116:0/3510288649 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f7c007e10 con 0x7f6f941027a0 2024-09-12T03:02:44.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.865+0000 7f6f98861640 1 -- 172.21.15.116:0/3510288649 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f9419f830 con 0x7f6f941027a0 2024-09-12T03:02:44.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.865+0000 7f6f8b7fe640 1 -- 172.21.15.116:0/3510288649 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c018890 con 0x7f6f941027a0 2024-09-12T03:02:44.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.865+0000 7f6f98861640 1 -- 172.21.15.116:0/3510288649 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f941a7110 con 0x7f6f941027a0 2024-09-12T03:02:44.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.866+0000 7f6f98861640 1 -- 172.21.15.116:0/3510288649 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f941021d0 con 0x7f6f941027a0 2024-09-12T03:02:44.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.867+0000 7f6f8b7fe640 1 -- 172.21.15.116:0/3510288649 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6f7c018a70 con 0x7f6f941027a0 2024-09-12T03:02:44.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.868+0000 7f6f8b7fe640 1 --2- 172.21.15.116:0/3510288649 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6f800a2b80 0x7f6f800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:44.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.868+0000 7f6f9259c640 1 --2- 172.21.15.116:0/3510288649 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6f800a2b80 0x7f6f800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:44.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.868+0000 7f6f8b7fe640 1 -- 172.21.15.116:0/3510288649 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f6f7c0cf190 con 0x7f6f941027a0 2024-09-12T03:02:44.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.868+0000 7f6f9259c640 1 --2- 172.21.15.116:0/3510288649 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6f800a2b80 0x7f6f800a5040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c005c40 tx=0x7f6f8c00a780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:44.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:44.872+0000 7f6f8b7fe640 1 -- 172.21.15.116:0/3510288649 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6f7c014030 con 0x7f6f941027a0 2024-09-12T03:02:45.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:45.074+0000 7f6f98861640 1 -- 172.21.15.116:0/3510288649 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f941a7440 con 0x7f6f941027a0 2024-09-12T03:02:45.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:45.074+0000 7f6f8b7fe640 1 -- 172.21.15.116:0/3510288649 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11938 (secure 0 0 0) 0x7f6f7c09c120 con 0x7f6f941027a0 2024-09-12T03:02:45.075 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:02:45.075 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":24,"btime":"2024-09-12T03:01:10:397269+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:01:10.218314+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24467,24475,24494]},"id":1}]} 2024-09-12T03:02:45.075 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 24 2024-09-12T03:02:45.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:45.076+0000 7f6f98861640 1 -- 172.21.15.116:0/3510288649 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6f800a2b80 msgr2=0x7f6f800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:45.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:45.076+0000 7f6f98861640 1 --2- 172.21.15.116:0/3510288649 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6f800a2b80 0x7f6f800a5040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c005c40 tx=0x7f6f8c00a780 comp rx=0 tx=0).stop 2024-09-12T03:02:45.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:45.077+0000 7f6f98861640 1 -- 172.21.15.116:0/3510288649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f941027a0 msgr2=0x7f6f9419aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:45.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:45.077+0000 7f6f98861640 1 --2- 172.21.15.116:0/3510288649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f941027a0 0x7f6f9419aad0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c00c920 tx=0x7f6f7c00cde0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:45.077+0000 7f6f98861640 1 -- 172.21.15.116:0/3510288649 shutdown_connections 2024-09-12T03:02:45.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:45.077+0000 7f6f98861640 1 --2- 172.21.15.116:0/3510288649 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6f800a2b80 0x7f6f800a5040 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:45.077+0000 7f6f98861640 1 --2- 172.21.15.116:0/3510288649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f94106f20 0x7f6f9419ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:45.077+0000 7f6f98861640 1 --2- 172.21.15.116:0/3510288649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f941027a0 0x7f6f9419aad0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:45.077+0000 7f6f98861640 1 --2- 172.21.15.116:0/3510288649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f94101df0 0x7f6f9419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:45.077+0000 7f6f98861640 1 -- 172.21.15.116:0/3510288649 >> 172.21.15.116:0/3510288649 conn(0x7f6f940fd8f0 msgr2=0x7f6f940fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:45.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:45.077+0000 7f6f98861640 1 -- 172.21.15.116:0/3510288649 shutdown_connections 2024-09-12T03:02:45.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:45.077+0000 7f6f98861640 1 -- 172.21.15.116:0/3510288649 wait complete. 2024-09-12T03:02:45.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:45 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3510288649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:45.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:45 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3510288649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:45.546 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:45 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3510288649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:45.912 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:02:46.160 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:46.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:46 smithi183 ceph-mon[35535]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T03:02:46.464 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:46 smithi116 ceph-mon[31410]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T03:02:46.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:46 smithi177 ceph-mon[35305]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T03:02:46.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.664+0000 7f916d0c7640 1 -- 172.21.15.116:0/149175822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9168100350 msgr2=0x7f9168100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:46.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.664+0000 7f916d0c7640 1 --2- 172.21.15.116:0/149175822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9168100350 0x7f9168100730 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9158009930 tx=0x7f915802a420 comp rx=0 tx=0).stop 2024-09-12T03:02:46.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.665+0000 7f916d0c7640 1 -- 172.21.15.116:0/149175822 shutdown_connections 2024-09-12T03:02:46.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.665+0000 7f916d0c7640 1 --2- 172.21.15.116:0/149175822 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9168106fa0 0x7f916810ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:46.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.665+0000 7f916d0c7640 1 --2- 172.21.15.116:0/149175822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9168100d00 0x7f9168106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:46.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.665+0000 7f916d0c7640 1 --2- 172.21.15.116:0/149175822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9168100350 0x7f9168100730 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:46.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.665+0000 7f916d0c7640 1 -- 172.21.15.116:0/149175822 >> 172.21.15.116:0/149175822 conn(0x7f91680fbee0 msgr2=0x7f91680fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:46.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.665+0000 7f916d0c7640 1 -- 172.21.15.116:0/149175822 shutdown_connections 2024-09-12T03:02:46.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.665+0000 7f916d0c7640 1 -- 172.21.15.116:0/149175822 wait complete. 2024-09-12T03:02:46.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.665+0000 7f916d0c7640 1 Processor -- start 2024-09-12T03:02:46.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.665+0000 7f916d0c7640 1 -- start start 2024-09-12T03:02:46.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.666+0000 7f916d0c7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9168100350 0x7f91681984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:46.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.666+0000 7f916d0c7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9168100d00 0x7f91681989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:46.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.666+0000 7f916d0c7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9168106fa0 0x7f916819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:46.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.666+0000 7f916d0c7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9168111fc0 con 0x7f9168100d00 2024-09-12T03:02:46.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.666+0000 7f916d0c7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9168111e40 con 0x7f9168106fa0 2024-09-12T03:02:46.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.666+0000 7f916d0c7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9168112140 con 0x7f9168100350 2024-09-12T03:02:46.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.666+0000 7f916659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9168100d00 0x7f91681989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:46.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.666+0000 7f916659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9168100d00 0x7f91681989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33522/0 (socket says 172.21.15.116:33522) 2024-09-12T03:02:46.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.666+0000 7f916659c640 1 -- 172.21.15.116:0/2239529649 learned_addr learned my addr 172.21.15.116:0/2239529649 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:46.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.666+0000 7f9166d9d640 1 --2- 172.21.15.116:0/2239529649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9168100350 0x7f91681984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:46.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.666+0000 7f916759e640 1 --2- 172.21.15.116:0/2239529649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9168106fa0 0x7f916819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:46.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.666+0000 7f916659c640 1 -- 172.21.15.116:0/2239529649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9168100350 msgr2=0x7f91681984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:46.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.666+0000 7f916659c640 1 --2- 172.21.15.116:0/2239529649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9168100350 0x7f91681984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:46.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.666+0000 7f916659c640 1 -- 172.21.15.116:0/2239529649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9168106fa0 msgr2=0x7f916819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:46.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.666+0000 7f916659c640 1 --2- 172.21.15.116:0/2239529649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9168106fa0 0x7f916819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:46.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.666+0000 7f916659c640 1 -- 172.21.15.116:0/2239529649 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f916819d460 con 0x7f9168100d00 2024-09-12T03:02:46.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.666+0000 7f916759e640 1 --2- 172.21.15.116:0/2239529649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9168106fa0 0x7f916819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:46.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.666+0000 7f916659c640 1 --2- 172.21.15.116:0/2239529649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9168100d00 0x7f91681989f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f915c005fd0 tx=0x7f915c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:46.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.666+0000 7f914ffff640 1 -- 172.21.15.116:0/2239529649 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c017070 con 0x7f9168100d00 2024-09-12T03:02:46.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.667+0000 7f916d0c7640 1 -- 172.21.15.116:0/2239529649 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f916819d750 con 0x7f9168100d00 2024-09-12T03:02:46.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.667+0000 7f914ffff640 1 -- 172.21.15.116:0/2239529649 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f915c009c30 con 0x7f9168100d00 2024-09-12T03:02:46.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.667+0000 7f914ffff640 1 -- 172.21.15.116:0/2239529649 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c010460 con 0x7f9168100d00 2024-09-12T03:02:46.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.667+0000 7f916d0c7640 1 -- 172.21.15.116:0/2239529649 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91681a5030 con 0x7f9168100d00 2024-09-12T03:02:46.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.668+0000 7f916d0c7640 1 -- 172.21.15.116:0/2239529649 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9168100730 con 0x7f9168100d00 2024-09-12T03:02:46.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.669+0000 7f914ffff640 1 -- 172.21.15.116:0/2239529649 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f915c003e70 con 0x7f9168100d00 2024-09-12T03:02:46.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.669+0000 7f914ffff640 1 --2- 172.21.15.116:0/2239529649 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f91500a2b80 0x7f91500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:46.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.669+0000 7f914ffff640 1 -- 172.21.15.116:0/2239529649 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f915c0cf4b0 con 0x7f9168100d00 2024-09-12T03:02:46.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.669+0000 7f9166d9d640 1 --2- 172.21.15.116:0/2239529649 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f91500a2b80 0x7f91500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:46.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.670+0000 7f9166d9d640 1 --2- 172.21.15.116:0/2239529649 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f91500a2b80 0x7f91500a5040 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9158009900 tx=0x7f91580058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:46.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.672+0000 7f914ffff640 1 -- 172.21.15.116:0/2239529649 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f915c020030 con 0x7f9168100d00 2024-09-12T03:02:46.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.839+0000 7f916d0c7640 1 -- 172.21.15.116:0/2239529649 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9168199810 con 0x7f9168100d00 2024-09-12T03:02:46.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.840+0000 7f914ffff640 1 -- 172.21.15.116:0/2239529649 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22710 (secure 0 0 0) 0x7f9168199810 con 0x7f9168100d00 2024-09-12T03:02:46.841 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:02:46.841 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":80,"fsid":"eb1ac602-70b1-11ef-bcea-c7b262605968","created":"2024-09-12T02:53:07.105686+0000","modified":"2024-09-12T03:02:04.272898+0000","last_up_change":"2024-09-12T03:00:07.202639+0000","last_in_change":"2024-09-12T02:59:43.446336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:56:17.347152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:00:55.191040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:00:57.171142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8cf5a18-dc6f-4bb6-bd74-78adda310118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6803","nonce":862366853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6805","nonce":862366853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6809","nonce":862366853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6807","nonce":862366853}]},"public_addr":"172.21.15.116:6803/862366853","cluster_addr":"172.21.15.116:6805/862366853","heartbeat_back_addr":"172.21.15.116:6809/862366853","heartbeat_front_addr":"172.21.15.116:6807/862366853","state":["exists","up"]},{"osd":1,"uuid":"0affd3cb-9a51-42b0-a2cc-f30784a75cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6801","nonce":2245800547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6803","nonce":2245800547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6807","nonce":2245800547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6805","nonce":2245800547}]},"public_addr":"172.21.15.177:6801/2245800547","cluster_addr":"172.21.15.177:6803/2245800547","heartbeat_back_addr":"172.21.15.177:6807/2245800547","heartbeat_front_addr":"172.21.15.177:6805/2245800547","state":["exists","up"]},{"osd":2,"uuid":"1ce45ea5-84df-4a35-924f-78f92bd4f2cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6801","nonce":2645677759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6803","nonce":2645677759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6807","nonce":2645677759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6805","nonce":2645677759}]},"public_addr":"172.21.15.183:6801/2645677759","cluster_addr":"172.21.15.183:6803/2645677759","heartbeat_back_addr":"172.21.15.183:6807/2645677759","heartbeat_front_addr":"172.21.15.183:6805/2645677759","state":["exists","up"]},{"osd":3,"uuid":"ae5e8178-c998-4648-bc05-4bce822f0cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6811","nonce":686622148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6813","nonce":686622148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6817","nonce":686622148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6815","nonce":686622148}]},"public_addr":"172.21.15.116:6811/686622148","cluster_addr":"172.21.15.116:6813/686622148","heartbeat_back_addr":"172.21.15.116:6817/686622148","heartbeat_front_addr":"172.21.15.116:6815/686622148","state":["exists","up"]},{"osd":4,"uuid":"c81d2e73-0bc2-440d-b5a2-b7428af3ea4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6809","nonce":2382938021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6811","nonce":2382938021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6815","nonce":2382938021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6813","nonce":2382938021}]},"public_addr":"172.21.15.177:6809/2382938021","cluster_addr":"172.21.15.177:6811/2382938021","heartbeat_back_addr":"172.21.15.177:6815/2382938021","heartbeat_front_addr":"172.21.15.177:6813/2382938021","state":["exists","up"]},{"osd":5,"uuid":"c3f168be-9e27-4cf5-b73e-3f47d8990ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6809","nonce":85363432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6811","nonce":85363432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6815","nonce":85363432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6813","nonce":85363432}]},"public_addr":"172.21.15.183:6809/85363432","cluster_addr":"172.21.15.183:6811/85363432","heartbeat_back_addr":"172.21.15.183:6815/85363432","heartbeat_front_addr":"172.21.15.183:6813/85363432","state":["exists","up"]},{"osd":6,"uuid":"beb0a7e9-4909-47e2-9ee7-6951d4745e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6819","nonce":1121291186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6821","nonce":1121291186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6825","nonce":1121291186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6823","nonce":1121291186}]},"public_addr":"172.21.15.116:6819/1121291186","cluster_addr":"172.21.15.116:6821/1121291186","heartbeat_back_addr":"172.21.15.116:6825/1121291186","heartbeat_front_addr":"172.21.15.116:6823/1121291186","state":["exists","up"]},{"osd":7,"uuid":"761f4680-f049-473c-bf44-b19acf8de24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6817","nonce":2177665961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6819","nonce":2177665961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6823","nonce":2177665961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6821","nonce":2177665961}]},"public_addr":"172.21.15.177:6817/2177665961","cluster_addr":"172.21.15.177:6819/2177665961","heartbeat_back_addr":"172.21.15.177:6823/2177665961","heartbeat_front_addr":"172.21.15.177:6821/2177665961","state":["exists","up"]},{"osd":8,"uuid":"2385ea6c-fd5a-4815-9596-77f948629542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6817","nonce":1305725258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6819","nonce":1305725258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6823","nonce":1305725258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6821","nonce":1305725258}]},"public_addr":"172.21.15.183:6817/1305725258","cluster_addr":"172.21.15.183:6819/1305725258","heartbeat_back_addr":"172.21.15.183:6823/1305725258","heartbeat_front_addr":"172.21.15.183:6821/1305725258","state":["exists","up"]},{"osd":9,"uuid":"fd0c0dc4-4990-4f12-aa14-f855504cd995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6827","nonce":3831673668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6829","nonce":3831673668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6833","nonce":3831673668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6831","nonce":3831673668}]},"public_addr":"172.21.15.116:6827/3831673668","cluster_addr":"172.21.15.116:6829/3831673668","heartbeat_back_addr":"172.21.15.116:6833/3831673668","heartbeat_front_addr":"172.21.15.116:6831/3831673668","state":["exists","up"]},{"osd":10,"uuid":"864d1a97-4810-49c8-ad4a-a0c10253833c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6825","nonce":963663634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6827","nonce":963663634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6831","nonce":963663634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6829","nonce":963663634}]},"public_addr":"172.21.15.177:6825/963663634","cluster_addr":"172.21.15.177:6827/963663634","heartbeat_back_addr":"172.21.15.177:6831/963663634","heartbeat_front_addr":"172.21.15.177:6829/963663634","state":["exists","up"]},{"osd":11,"uuid":"d49c0841-4620-4f2f-a6b5-f2a9aa2e3c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6825","nonce":1818319939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6827","nonce":1818319939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6831","nonce":1818319939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6829","nonce":1818319939}]},"public_addr":"172.21.15.183:6825/1818319939","cluster_addr":"172.21.15.183:6827/1818319939","heartbeat_back_addr":"172.21.15.183:6831/1818319939","heartbeat_front_addr":"172.21.15.183:6829/1818319939","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:24.736944+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:50.310980+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:56:14.989533+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:56:41.187047+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:06.981664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:32.872771+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:58.850089+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:23.043121+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:47.839958+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:14.087708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:40.010548+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:05.364820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.116:6801/2966274423":"2024-09-13T02:54:03.026862+0000","172.21.15.116:6800/2966274423":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/686139455":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/1552047390":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/2180363411":"2024-09-13T02:53:39.433769+0000","172.21.15.116:6801/1791219601":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/3907400460":"2024-09-13T02:54:03.026862+0000","172.21.15.116:6800/1791219601":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/1802566286":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/212337802":"2024-09-13T02:53:39.433769+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:02:46.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.843+0000 7f916d0c7640 1 -- 172.21.15.116:0/2239529649 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f91500a2b80 msgr2=0x7f91500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:46.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.843+0000 7f916d0c7640 1 --2- 172.21.15.116:0/2239529649 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f91500a2b80 0x7f91500a5040 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9158009900 tx=0x7f91580058d0 comp rx=0 tx=0).stop 2024-09-12T03:02:46.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.843+0000 7f916d0c7640 1 -- 172.21.15.116:0/2239529649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9168100d00 msgr2=0x7f91681989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:46.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.843+0000 7f916d0c7640 1 --2- 172.21.15.116:0/2239529649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9168100d00 0x7f91681989f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f915c005fd0 tx=0x7f915c00dde0 comp rx=0 tx=0).stop 2024-09-12T03:02:46.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.843+0000 7f916d0c7640 1 -- 172.21.15.116:0/2239529649 shutdown_connections 2024-09-12T03:02:46.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.843+0000 7f916d0c7640 1 --2- 172.21.15.116:0/2239529649 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f91500a2b80 0x7f91500a5040 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:46.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.843+0000 7f916d0c7640 1 --2- 172.21.15.116:0/2239529649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9168106fa0 0x7f916819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:46.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.843+0000 7f916d0c7640 1 --2- 172.21.15.116:0/2239529649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9168100d00 0x7f91681989f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:46.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.843+0000 7f916d0c7640 1 --2- 172.21.15.116:0/2239529649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9168100350 0x7f91681984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:46.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.843+0000 7f916d0c7640 1 -- 172.21.15.116:0/2239529649 >> 172.21.15.116:0/2239529649 conn(0x7f91680fbee0 msgr2=0x7f9168105aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:46.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.843+0000 7f916d0c7640 1 -- 172.21.15.116:0/2239529649 shutdown_connections 2024-09-12T03:02:46.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:46.843+0000 7f916d0c7640 1 -- 172.21.15.116:0/2239529649 wait complete. 2024-09-12T03:02:47.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:47 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2239529649' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:02:47.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:47 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2239529649' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:02:47.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:47 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2239529649' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:02:47.704 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T03:00:59.195360+0000', 'modified': '2024-09-12T03:01:10.218314+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24494, 'mds_1': 24475, 'mds_2': 24467}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24443': {'gid': 24443, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.177:6835/2224643300', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6834', 'nonce': 2224643300}, {'type': 'v1', 'addr': '172.21.15.177:6835', 'nonce': 2224643300}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/4028826399', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 4028826399}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 4028826399}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6837/1288373243', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 1288373243}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 1288373243}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.177:6837/1862929607', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6836', 'nonce': 1862929607}, {'type': 'v1', 'addr': '172.21.15.177:6837', 'nonce': 1862929607}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.183:6839/1983839027', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 1983839027}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 1983839027}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.177:6839/2069130361', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6838', 'nonce': 2069130361}, {'type': 'v1', 'addr': '172.21.15.177:6839', 'nonce': 2069130361}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24494, 'qdb_cluster': [24467, 24475, 24494]} 2024-09-12T03:02:47.704 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-12T03:02:47.705 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-12T03:02:47.949 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:48.361 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:48 smithi116 ceph-mon[31410]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T03:02:48.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.442+0000 7f081e633640 1 -- 172.21.15.116:0/1460874805 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0818105a90 msgr2=0x7f0818105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:48.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.442+0000 7f081e633640 1 --2- 172.21.15.116:0/1460874805 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0818105a90 0x7f0818105e70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0808009900 tx=0x7f080802a410 comp rx=0 tx=0).stop 2024-09-12T03:02:48.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.442+0000 7f081e633640 1 -- 172.21.15.116:0/1460874805 shutdown_connections 2024-09-12T03:02:48.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.442+0000 7f081e633640 1 --2- 172.21.15.116:0/1460874805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0818101bc0 0x7f081810d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.442+0000 7f081e633640 1 --2- 172.21.15.116:0/1460874805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0818101240 0x7f0818101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.442+0000 7f081e633640 1 --2- 172.21.15.116:0/1460874805 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0818105a90 0x7f0818105e70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.442+0000 7f081e633640 1 -- 172.21.15.116:0/1460874805 >> 172.21.15.116:0/1460874805 conn(0x7f0818077210 msgr2=0x7f08180fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:48.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.442+0000 7f081e633640 1 -- 172.21.15.116:0/1460874805 shutdown_connections 2024-09-12T03:02:48.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.442+0000 7f081e633640 1 -- 172.21.15.116:0/1460874805 wait complete. 2024-09-12T03:02:48.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.442+0000 7f081e633640 1 Processor -- start 2024-09-12T03:02:48.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.443+0000 7f081e633640 1 -- start start 2024-09-12T03:02:48.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.443+0000 7f081e633640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0818101240 0x7f08180fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:48.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.443+0000 7f081e633640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0818101bc0 0x7f08180fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:48.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.443+0000 7f081e633640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0818105a90 0x7f08180fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:48.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.443+0000 7f081e633640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0818110f70 con 0x7f0818101bc0 2024-09-12T03:02:48.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.443+0000 7f081e633640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0818110df0 con 0x7f0818101240 2024-09-12T03:02:48.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.443+0000 7f081e633640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f08181110f0 con 0x7f0818105a90 2024-09-12T03:02:48.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.443+0000 7f08177fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0818101bc0 0x7f08180fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:48.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.443+0000 7f08177fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0818101bc0 0x7f08180fcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33530/0 (socket says 172.21.15.116:33530) 2024-09-12T03:02:48.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.443+0000 7f08177fe640 1 -- 172.21.15.116:0/2973301726 learned_addr learned my addr 172.21.15.116:0/2973301726 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:48.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.443+0000 7f081cbd0640 1 --2- 172.21.15.116:0/2973301726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0818105a90 0x7f08180fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:48.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.443+0000 7f0817fff640 1 --2- 172.21.15.116:0/2973301726 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0818101240 0x7f08180fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:48.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.443+0000 7f08177fe640 1 -- 172.21.15.116:0/2973301726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0818105a90 msgr2=0x7f08180fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:48.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.443+0000 7f08177fe640 1 --2- 172.21.15.116:0/2973301726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0818105a90 0x7f08180fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.443+0000 7f08177fe640 1 -- 172.21.15.116:0/2973301726 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0818101240 msgr2=0x7f08180fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:48.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.443+0000 7f08177fe640 1 --2- 172.21.15.116:0/2973301726 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0818101240 0x7f08180fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.443+0000 7f08177fe640 1 -- 172.21.15.116:0/2973301726 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08181a6960 con 0x7f0818101bc0 2024-09-12T03:02:48.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.443+0000 7f0817fff640 1 --2- 172.21.15.116:0/2973301726 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0818101240 0x7f08180fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:48.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.444+0000 7f08177fe640 1 --2- 172.21.15.116:0/2973301726 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0818101bc0 0x7f08180fcbd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f080c00e9f0 tx=0x7f080c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:48.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.444+0000 7f08157fa640 1 -- 172.21.15.116:0/2973301726 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c00cd80 con 0x7f0818101bc0 2024-09-12T03:02:48.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.444+0000 7f081e633640 1 -- 172.21.15.116:0/2973301726 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08181a6c50 con 0x7f0818101bc0 2024-09-12T03:02:48.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.444+0000 7f08157fa640 1 -- 172.21.15.116:0/2973301726 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f080c010910 con 0x7f0818101bc0 2024-09-12T03:02:48.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.444+0000 7f08157fa640 1 -- 172.21.15.116:0/2973301726 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c018990 con 0x7f0818101bc0 2024-09-12T03:02:48.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.444+0000 7f081e633640 1 -- 172.21.15.116:0/2973301726 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08181a7190 con 0x7f0818101bc0 2024-09-12T03:02:48.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.446+0000 7f08157fa640 1 -- 172.21.15.116:0/2973301726 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f080c010ac0 con 0x7f0818101bc0 2024-09-12T03:02:48.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.447+0000 7f0820a0c740 1 Processor -- start 2024-09-12T03:02:48.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.447+0000 7f08157fa640 1 --2- 172.21.15.116:0/2973301726 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f08000a2b80 0x7f08000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:48.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.447+0000 7f0820a0c740 1 -- start start 2024-09-12T03:02:48.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.447+0000 7f081cbd0640 1 --2- 172.21.15.116:0/2973301726 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f08000a2b80 0x7f08000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:48.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.447+0000 7f08157fa640 1 -- 172.21.15.116:0/2973301726 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f080c0cf410 con 0x7f0818101bc0 2024-09-12T03:02:48.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.447+0000 7f0820a0c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611db4ad9d0 0x5611db4addb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:48.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.447+0000 7f0820a0c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5611db4ae2f0 0x5611db4b8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:48.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.447+0000 7f0820a0c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611db4b8bb0 0x5611db4bafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:48.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.447+0000 7f0820a0c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5611db414090 con 0x5611db4b8bb0 2024-09-12T03:02:48.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.447+0000 7f0820a0c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5611db3bba70 con 0x5611db4ae2f0 2024-09-12T03:02:48.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.447+0000 7f0820a0c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5611db410050 con 0x5611db4ad9d0 2024-09-12T03:02:48.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.447+0000 7f081cbd0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611db4b8bb0 0x5611db4bafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:48.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.447+0000 7f081cbd0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611db4b8bb0 0x5611db4bafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33536/0 (socket says 172.21.15.116:33536) 2024-09-12T03:02:48.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.447+0000 7f081cbd0640 1 -- 172.21.15.116:0/3744667401 learned_addr learned my addr 172.21.15.116:0/3744667401 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:48.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.447+0000 7f0817fff640 1 --2- 172.21.15.116:0/3744667401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611db4ad9d0 0x5611db4addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:48.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.447+0000 7f081cbd0640 1 --2- 172.21.15.116:0/2973301726 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f08000a2b80 0x7f08000a5040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0810005fd0 tx=0x7f08100107a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:48.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.447+0000 7f08177fe640 1 --2- 172.21.15.116:0/3744667401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5611db4ae2f0 0x5611db4b8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:48.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.447+0000 7f081cbd0640 1 -- 172.21.15.116:0/3744667401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611db4ad9d0 msgr2=0x5611db4addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:48.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.447+0000 7f081cbd0640 1 --2- 172.21.15.116:0/3744667401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611db4ad9d0 0x5611db4addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.447+0000 7f081cbd0640 1 -- 172.21.15.116:0/3744667401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5611db4ae2f0 msgr2=0x5611db4b8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:48.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.447+0000 7f081cbd0640 1 --2- 172.21.15.116:0/3744667401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5611db4ae2f0 0x5611db4b8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.447+0000 7f081cbd0640 1 -- 172.21.15.116:0/3744667401 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611db3c8130 con 0x5611db4b8bb0 2024-09-12T03:02:48.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.448+0000 7f081cbd0640 1 --2- 172.21.15.116:0/3744667401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611db4b8bb0 0x5611db4bafa0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0810013820 tx=0x7f081001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa428f0a66738b60 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:48.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.448+0000 7f07fe7fc640 1 -- 172.21.15.116:0/3744667401 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08100130d0 con 0x5611db4b8bb0 2024-09-12T03:02:48.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.448+0000 7f07fe7fc640 1 -- 172.21.15.116:0/3744667401 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0810013270 con 0x5611db4b8bb0 2024-09-12T03:02:48.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.448+0000 7f07fe7fc640 1 -- 172.21.15.116:0/3744667401 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810011d40 con 0x5611db4b8bb0 2024-09-12T03:02:48.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.449+0000 7f0820a0c740 1 -- 172.21.15.116:0/3744667401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611db4b8bb0 msgr2=0x5611db4bafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:48.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.449+0000 7f0820a0c740 1 --2- 172.21.15.116:0/3744667401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611db4b8bb0 0x5611db4bafa0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0810013820 tx=0x7f081001b000 comp rx=0 tx=0).stop 2024-09-12T03:02:48.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.449+0000 7f0820a0c740 1 -- 172.21.15.116:0/3744667401 shutdown_connections 2024-09-12T03:02:48.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.449+0000 7f0820a0c740 1 --2- 172.21.15.116:0/3744667401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611db4b8bb0 0x5611db4bafa0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.449+0000 7f0820a0c740 1 --2- 172.21.15.116:0/3744667401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5611db4ae2f0 0x5611db4b8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.449+0000 7f0820a0c740 1 --2- 172.21.15.116:0/3744667401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611db4ad9d0 0x5611db4addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.449+0000 7f0820a0c740 1 -- 172.21.15.116:0/3744667401 >> 172.21.15.116:0/3744667401 conn(0x5611db4a9f60 msgr2=0x5611db4aa340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:48.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.449+0000 7f0820a0c740 1 -- 172.21.15.116:0/3744667401 shutdown_connections 2024-09-12T03:02:48.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.449+0000 7f0820a0c740 1 -- 172.21.15.116:0/3744667401 wait complete. 2024-09-12T03:02:48.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.449+0000 7f0820a0c740 1 -- start start 2024-09-12T03:02:48.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.449+0000 7f0820a0c740 1 Processor -- start 2024-09-12T03:02:48.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.449+0000 7f0820a0c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5611db4ad9d0 0x5611db4b8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:48.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.449+0000 7f0820a0c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611db4ae2f0 0x5611db4b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:48.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.449+0000 7f0820a0c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611db4b8bb0 0x5611db4b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:48.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.449+0000 7f0820a0c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5611db4bb780 con 0x5611db4ae2f0 2024-09-12T03:02:48.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.449+0000 7f0820a0c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5611db3cef20 con 0x5611db4ad9d0 2024-09-12T03:02:48.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.449+0000 7f0820a0c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5611db4bb900 con 0x5611db4b8bb0 2024-09-12T03:02:48.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.449+0000 7f0820a0c740 10 client.0 _ll_register_callbacks cb 0x5611db398030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:48.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.450+0000 7f08177fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611db4ae2f0 0x5611db4b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:48.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.450+0000 7f08177fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611db4ae2f0 0x5611db4b57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33538/0 (socket says 172.21.15.116:33538) 2024-09-12T03:02:48.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.450+0000 7f08177fe640 1 -- 172.21.15.116:0/3247447874 learned_addr learned my addr 172.21.15.116:0/3247447874 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:48.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.450+0000 7f081cbd0640 1 --2- 172.21.15.116:0/3247447874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611db4b8bb0 0x5611db4b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:48.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.450+0000 7f0817fff640 1 --2- 172.21.15.116:0/3247447874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5611db4ad9d0 0x5611db4b8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:48.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.450+0000 7f08177fe640 1 -- 172.21.15.116:0/3247447874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611db4b8bb0 msgr2=0x5611db4b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:48.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.450+0000 7f08177fe640 1 --2- 172.21.15.116:0/3247447874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611db4b8bb0 0x5611db4b5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.450+0000 7f08177fe640 1 -- 172.21.15.116:0/3247447874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5611db4ad9d0 msgr2=0x5611db4b8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:48.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.450+0000 7f08177fe640 1 --2- 172.21.15.116:0/3247447874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5611db4ad9d0 0x5611db4b8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.450+0000 7f08177fe640 1 -- 172.21.15.116:0/3247447874 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611db1852b0 con 0x5611db4ae2f0 2024-09-12T03:02:48.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.450+0000 7f08177fe640 1 --2- 172.21.15.116:0/3247447874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611db4ae2f0 0x5611db4b57b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f080c09b940 tx=0x7f080c09c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafd45a587dc8eec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:48.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.450+0000 7f07f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:02:48.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.450+0000 7f07f77fe640 1 -- 172.21.15.116:0/3247447874 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c018b30 con 0x5611db4ae2f0 2024-09-12T03:02:48.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.450+0000 7f07f77fe640 1 -- 172.21.15.116:0/3247447874 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f080c018cd0 con 0x5611db4ae2f0 2024-09-12T03:02:48.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.450+0000 7f0820a0c740 1 -- 172.21.15.116:0/3247447874 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0810013270 con 0x5611db4ae2f0 2024-09-12T03:02:48.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.450+0000 7f07f77fe640 1 -- 172.21.15.116:0/3247447874 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f080c09b090 con 0x5611db4ae2f0 2024-09-12T03:02:48.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.450+0000 7f0820a0c740 10 client.15111 fetch_fsmap learned FSMap version 24 2024-09-12T03:02:48.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.450+0000 7f0820a0c740 1 -- 172.21.15.116:0/3247447874 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5611db4c7100 con 0x5611db4ae2f0 2024-09-12T03:02:48.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.450+0000 7f07f77fe640 1 -- 172.21.15.116:0/3247447874 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0810013270 con 0x5611db4ae2f0 2024-09-12T03:02:48.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.451+0000 7f07f77fe640 1 -- 172.21.15.116:0/3247447874 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f080c09ab00 con 0x5611db4ae2f0 2024-09-12T03:02:48.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.451+0000 7f0820a0c740 10 client.15111 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:02:48.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.451+0000 7f0820a0c740 10 client.15111 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T03:02:48.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.451+0000 7f0820a0c740 20 client.15111 populate_metadata read hostname 'smithi116' 2024-09-12T03:02:48.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.451+0000 7f0820a0c740 1 --2- 172.21.15.116:0/3247447874 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5611db4c9720 0x5611db4cab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:48.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.451+0000 7f0820a0c740 4 client.15111 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:02:48.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.451+0000 7f0820a0c740 1 -- 172.21.15.116:0/3247447874 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611db421600 con 0x5611db4c9720 2024-09-12T03:02:48.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.451+0000 7f0817fff640 1 --2- 172.21.15.116:0/3247447874 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5611db4c9720 0x5611db4cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:48.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.451+0000 7f07f77fe640 10 client.15111 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:02:48.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.451+0000 7f0817fff640 1 --2- 172.21.15.116:0/3247447874 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5611db4c9720 0x5611db4cab90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=25ffc9185fd81336 server_cookie=6c7f21866ae93bea in_seq=0 out_seq=0 2024-09-12T03:02:48.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.452+0000 7f07f77fe640 1 -- 172.21.15.116:0/3247447874 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5611db421600 con 0x5611db4c9720 2024-09-12T03:02:48.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.452+0000 7f07f77fe640 10 client.15111 handle_command_reply: tid=0 2024-09-12T03:02:48.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.452+0000 7f0820a0c740 1 client.15111 shutdown 2024-09-12T03:02:48.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.452+0000 7f0820a0c740 1 -- 172.21.15.116:0/3247447874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611db4ae2f0 msgr2=0x5611db4b57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:48.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.452+0000 7f0820a0c740 1 --2- 172.21.15.116:0/3247447874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611db4ae2f0 0x5611db4b57b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f080c09b940 tx=0x7f080c09c480 comp rx=0 tx=0).stop 2024-09-12T03:02:48.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.452+0000 7f0820a0c740 1 -- 172.21.15.116:0/3247447874 shutdown_connections 2024-09-12T03:02:48.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.452+0000 7f0820a0c740 1 --2- 172.21.15.116:0/3247447874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611db4b8bb0 0x5611db4b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.452+0000 7f0820a0c740 1 --2- 172.21.15.116:0/3247447874 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5611db4c9720 0x5611db4cab90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.452+0000 7f0820a0c740 1 --2- 172.21.15.116:0/3247447874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611db4ae2f0 0x5611db4b57b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.452+0000 7f07f77fe640 0 client.15111 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:02:48.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.452+0000 7f0820a0c740 1 --2- 172.21.15.116:0/3247447874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5611db4ad9d0 0x5611db4b8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.452+0000 7f0820a0c740 1 -- 172.21.15.116:0/3247447874 >> 172.21.15.116:0/3247447874 conn(0x5611db4a9f60 msgr2=0x5611db4afc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:48.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.453+0000 7f0820a0c740 1 -- 172.21.15.116:0/3247447874 shutdown_connections 2024-09-12T03:02:48.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.453+0000 7f0820a0c740 1 -- 172.21.15.116:0/3247447874 wait complete. 2024-09-12T03:02:48.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.453+0000 7f0820a0c740 20 client.15111 trim_cache size 0 max 16384 2024-09-12T03:02:48.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.470+0000 7f0820a0c740 1 Processor -- start 2024-09-12T03:02:48.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.470+0000 7f0820a0c740 1 -- start start 2024-09-12T03:02:48.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.470+0000 7f0820a0c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5611db4ad9d0 0x5611db4cab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:48.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.470+0000 7f0820a0c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611db4ae2f0 0x5611db4b5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:48.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.470+0000 7f0820a0c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611db4b8bb0 0x5611db4b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:48.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.470+0000 7f0820a0c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5611db4c8ed0 con 0x5611db4b8bb0 2024-09-12T03:02:48.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.470+0000 7f0820a0c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5611db4c95a0 con 0x5611db4ad9d0 2024-09-12T03:02:48.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.470+0000 7f0820a0c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5611db4c9c70 con 0x5611db4ae2f0 2024-09-12T03:02:48.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.470+0000 7f081cbd0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611db4b8bb0 0x5611db4b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:48.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.470+0000 7f081cbd0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611db4b8bb0 0x5611db4b5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33552/0 (socket says 172.21.15.116:33552) 2024-09-12T03:02:48.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.470+0000 7f081cbd0640 1 -- 172.21.15.116:0/1886628172 learned_addr learned my addr 172.21.15.116:0/1886628172 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:48.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.471+0000 7f0817fff640 1 --2- 172.21.15.116:0/1886628172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5611db4ad9d0 0x5611db4cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:48.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.471+0000 7f081cbd0640 1 -- 172.21.15.116:0/1886628172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611db4ae2f0 msgr2=0x5611db4b5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:48.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.471+0000 7f081cbd0640 1 --2- 172.21.15.116:0/1886628172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611db4ae2f0 0x5611db4b5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.471+0000 7f081cbd0640 1 -- 172.21.15.116:0/1886628172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5611db4ad9d0 msgr2=0x5611db4cab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:48.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.471+0000 7f081cbd0640 1 --2- 172.21.15.116:0/1886628172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5611db4ad9d0 0x5611db4cab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.471+0000 7f081cbd0640 1 -- 172.21.15.116:0/1886628172 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611db4b62f0 con 0x5611db4b8bb0 2024-09-12T03:02:48.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.471+0000 7f081cbd0640 1 --2- 172.21.15.116:0/1886628172 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611db4b8bb0 0x5611db4b5cb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x5611db4ab2e0 tx=0x7f0810009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cddf900c69596cc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:48.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.471+0000 7f07fdffb640 1 -- 172.21.15.116:0/1886628172 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810011440 con 0x5611db4b8bb0 2024-09-12T03:02:48.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.471+0000 7f07fdffb640 1 -- 172.21.15.116:0/1886628172 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0810003e50 con 0x5611db4b8bb0 2024-09-12T03:02:48.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.471+0000 7f07fdffb640 1 -- 172.21.15.116:0/1886628172 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081000e8d0 con 0x5611db4b8bb0 2024-09-12T03:02:48.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.472+0000 7f0820a0c740 1 -- 172.21.15.116:0/1886628172 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611db4b8bb0 msgr2=0x5611db4b5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:48.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.472+0000 7f0820a0c740 1 --2- 172.21.15.116:0/1886628172 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611db4b8bb0 0x5611db4b5cb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x5611db4ab2e0 tx=0x7f0810009750 comp rx=0 tx=0).stop 2024-09-12T03:02:48.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.472+0000 7f0820a0c740 1 -- 172.21.15.116:0/1886628172 shutdown_connections 2024-09-12T03:02:48.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.472+0000 7f0820a0c740 1 --2- 172.21.15.116:0/1886628172 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611db4b8bb0 0x5611db4b5cb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.472+0000 7f0820a0c740 1 --2- 172.21.15.116:0/1886628172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611db4ae2f0 0x5611db4b5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.472+0000 7f0820a0c740 1 --2- 172.21.15.116:0/1886628172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5611db4ad9d0 0x5611db4cab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.472+0000 7f0820a0c740 1 -- 172.21.15.116:0/1886628172 >> 172.21.15.116:0/1886628172 conn(0x5611db4a9f60 msgr2=0x5611db4a4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:48.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.472+0000 7f0820a0c740 1 -- 172.21.15.116:0/1886628172 shutdown_connections 2024-09-12T03:02:48.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.472+0000 7f0820a0c740 1 -- 172.21.15.116:0/1886628172 wait complete. 2024-09-12T03:02:48.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.472+0000 7f0820a0c740 1 -- start start 2024-09-12T03:02:48.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.472+0000 7f0820a0c740 1 Processor -- start 2024-09-12T03:02:48.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.473+0000 7f0820a0c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5611db4ad9d0 0x5611db4d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:48.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.473+0000 7f0820a0c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611db4ae2f0 0x5611db4d1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:48.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.473+0000 7f0820a0c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611db4b8bb0 0x5611db4cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:48.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.473+0000 7f0820a0c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5611db4caa10 con 0x5611db4b8bb0 2024-09-12T03:02:48.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.473+0000 7f0820a0c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5611db4ca340 con 0x5611db4ad9d0 2024-09-12T03:02:48.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.473+0000 7f0820a0c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5611db4cb0e0 con 0x5611db4ae2f0 2024-09-12T03:02:48.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.473+0000 7f0820a0c740 10 client.0 _ll_register_callbacks cb 0x7f080c097ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:48.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.473+0000 7f081cbd0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611db4b8bb0 0x5611db4cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:48.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.473+0000 7f081cbd0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611db4b8bb0 0x5611db4cc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33568/0 (socket says 172.21.15.116:33568) 2024-09-12T03:02:48.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.473+0000 7f081cbd0640 1 -- 172.21.15.116:0/3171939242 learned_addr learned my addr 172.21.15.116:0/3171939242 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:48.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.473+0000 7f0817fff640 1 --2- 172.21.15.116:0/3171939242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5611db4ad9d0 0x5611db4d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:48.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.473+0000 7f08177fe640 1 --2- 172.21.15.116:0/3171939242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611db4ae2f0 0x5611db4d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:48.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.473+0000 7f081cbd0640 1 -- 172.21.15.116:0/3171939242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611db4ae2f0 msgr2=0x5611db4d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:48.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.473+0000 7f081cbd0640 1 --2- 172.21.15.116:0/3171939242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611db4ae2f0 0x5611db4d1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.473+0000 7f081cbd0640 1 -- 172.21.15.116:0/3171939242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5611db4ad9d0 msgr2=0x5611db4d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:48.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.473+0000 7f081cbd0640 1 --2- 172.21.15.116:0/3171939242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5611db4ad9d0 0x5611db4d0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.473+0000 7f081cbd0640 1 -- 172.21.15.116:0/3171939242 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611db4cc970 con 0x5611db4b8bb0 2024-09-12T03:02:48.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.473+0000 7f08177fe640 1 --2- 172.21.15.116:0/3171939242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611db4ae2f0 0x5611db4d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:48.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.473+0000 7f081cbd0640 1 --2- 172.21.15.116:0/3171939242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611db4b8bb0 0x5611db4cc330 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0810011010 tx=0x7f0810013380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed4628c37d2d3cd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:48.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.473+0000 7f07f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:02:48.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.473+0000 7f07f77fe640 1 -- 172.21.15.116:0/3171939242 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810036060 con 0x5611db4b8bb0 2024-09-12T03:02:48.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.473+0000 7f07f77fe640 1 -- 172.21.15.116:0/3171939242 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f0810011640 con 0x5611db4b8bb0 2024-09-12T03:02:48.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.473+0000 7f0820a0c740 1 -- 172.21.15.116:0/3171939242 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0810003e50 con 0x5611db4b8bb0 2024-09-12T03:02:48.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.474+0000 7f07f77fe640 1 -- 172.21.15.116:0/3171939242 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0810003970 con 0x5611db4b8bb0 2024-09-12T03:02:48.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.474+0000 7f0820a0c740 10 client.15117 fetch_fsmap learned FSMap version 24 2024-09-12T03:02:48.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.474+0000 7f0820a0c740 1 -- 172.21.15.116:0/3171939242 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5611db4d0300 con 0x5611db4b8bb0 2024-09-12T03:02:48.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.474+0000 7f07f77fe640 1 -- 172.21.15.116:0/3171939242 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0810003e50 con 0x5611db4b8bb0 2024-09-12T03:02:48.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.474+0000 7f07f77fe640 1 -- 172.21.15.116:0/3171939242 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f0810005480 con 0x5611db4b8bb0 2024-09-12T03:02:48.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.474+0000 7f0820a0c740 10 client.15117 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:02:48.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.474+0000 7f0820a0c740 10 client.15117 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T03:02:48.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.474+0000 7f0820a0c740 20 client.15117 populate_metadata read hostname 'smithi116' 2024-09-12T03:02:48.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.474+0000 7f0820a0c740 1 --2- 172.21.15.116:0/3171939242 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5611db537a30 0x5611db536740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:48.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.474+0000 7f0820a0c740 4 client.15117 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:02:48.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.474+0000 7f0820a0c740 1 -- 172.21.15.116:0/3171939242 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "status"}) -- 0x5611db41bb10 con 0x5611db537a30 2024-09-12T03:02:48.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.474+0000 7f0817fff640 1 --2- 172.21.15.116:0/3171939242 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5611db537a30 0x5611db536740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:48.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.475+0000 7f07f77fe640 10 client.15117 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:02:48.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.475+0000 7f0817fff640 1 --2- 172.21.15.116:0/3171939242 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5611db537a30 0x5611db536740 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bbfc62707d56d409 server_cookie=4d35828ab345f154 in_seq=0 out_seq=0 2024-09-12T03:02:48.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.475+0000 7f07f77fe640 1 -- 172.21.15.116:0/3171939242 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5611db41bb10 con 0x5611db537a30 2024-09-12T03:02:48.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.475+0000 7f07f77fe640 10 client.15117 handle_command_reply: tid=0 2024-09-12T03:02:48.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.475+0000 7f0820a0c740 1 client.15117 shutdown 2024-09-12T03:02:48.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.475+0000 7f0820a0c740 1 -- 172.21.15.116:0/3171939242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611db4b8bb0 msgr2=0x5611db4cc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:48.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.475+0000 7f0820a0c740 1 --2- 172.21.15.116:0/3171939242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611db4b8bb0 0x5611db4cc330 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0810011010 tx=0x7f0810013380 comp rx=0 tx=0).stop 2024-09-12T03:02:48.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.475+0000 7f0820a0c740 1 -- 172.21.15.116:0/3171939242 shutdown_connections 2024-09-12T03:02:48.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.475+0000 7f0820a0c740 1 --2- 172.21.15.116:0/3171939242 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5611db537a30 0x5611db536740 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.475+0000 7f0820a0c740 1 --2- 172.21.15.116:0/3171939242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611db4b8bb0 0x5611db4cc330 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.475+0000 7f0820a0c740 1 --2- 172.21.15.116:0/3171939242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611db4ae2f0 0x5611db4d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.475+0000 7f0820a0c740 1 --2- 172.21.15.116:0/3171939242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5611db4ad9d0 0x5611db4d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.475+0000 7f0820a0c740 1 -- 172.21.15.116:0/3171939242 >> 172.21.15.116:0/3171939242 conn(0x5611db4a9f60 msgr2=0x5611db4b5770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:48.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.475+0000 7f07f77fe640 0 client.15117 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:02:48.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.475+0000 7f0820a0c740 1 -- 172.21.15.116:0/3171939242 shutdown_connections 2024-09-12T03:02:48.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.475+0000 7f0820a0c740 1 -- 172.21.15.116:0/3171939242 wait complete. 2024-09-12T03:02:48.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.475+0000 7f0820a0c740 20 client.15117 trim_cache size 0 max 16384 2024-09-12T03:02:48.502 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:02:48.502 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_fsid": "eb1ac602-70b1-11ef-bcea-c7b262605968", 2024-09-12T03:02:48.502 INFO:teuthology.orchestra.run.smithi116.stdout: "whoami": 2, 2024-09-12T03:02:48.503 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 24467, 2024-09-12T03:02:48.503 INFO:teuthology.orchestra.run.smithi116.stdout: "want_state": "up:active", 2024-09-12T03:02:48.503 INFO:teuthology.orchestra.run.smithi116.stdout: "state": "up:active", 2024-09-12T03:02:48.503 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs", 2024-09-12T03:02:48.503 INFO:teuthology.orchestra.run.smithi116.stdout: "rank_uptime": 101.08566008699999, 2024-09-12T03:02:48.503 INFO:teuthology.orchestra.run.smithi116.stdout: "mdsmap_epoch": 24, 2024-09-12T03:02:48.503 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch": 80, 2024-09-12T03:02:48.504 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch_barrier": 78, 2024-09-12T03:02:48.504 INFO:teuthology.orchestra.run.smithi116.stdout: "uptime": 131.297563591 2024-09-12T03:02:48.504 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:02:48.504 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:48 smithi177 ceph-mon[35305]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T03:02:48.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.476+0000 7f081e633640 1 -- 172.21.15.116:0/2973301726 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f08000a2b80 msgr2=0x7f08000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:48.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.476+0000 7f081e633640 1 --2- 172.21.15.116:0/2973301726 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f08000a2b80 0x7f08000a5040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0810005fd0 tx=0x7f08100107a0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.476+0000 7f081e633640 1 -- 172.21.15.116:0/2973301726 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0818101bc0 msgr2=0x7f08180fcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:48.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.476+0000 7f081e633640 1 --2- 172.21.15.116:0/2973301726 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0818101bc0 0x7f08180fcbd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f080c00e9f0 tx=0x7f080c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.476+0000 7f081e633640 1 -- 172.21.15.116:0/2973301726 shutdown_connections 2024-09-12T03:02:48.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.476+0000 7f081e633640 1 --2- 172.21.15.116:0/2973301726 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f08000a2b80 0x7f08000a5040 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.476+0000 7f081e633640 1 --2- 172.21.15.116:0/2973301726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0818105a90 0x7f08180fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.476+0000 7f081e633640 1 --2- 172.21.15.116:0/2973301726 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0818101bc0 0x7f08180fcbd0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.476+0000 7f081e633640 1 --2- 172.21.15.116:0/2973301726 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0818101240 0x7f08180fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:48.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.476+0000 7f081e633640 1 -- 172.21.15.116:0/2973301726 >> 172.21.15.116:0/2973301726 conn(0x7f0818077210 msgr2=0x7f0818100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:48.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.476+0000 7f081e633640 1 -- 172.21.15.116:0/2973301726 shutdown_connections 2024-09-12T03:02:48.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:48.476+0000 7f081e633640 1 -- 172.21.15.116:0/2973301726 wait complete. 2024-09-12T03:02:48.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:48 smithi183 ceph-mon[35535]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T03:02:49.259 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T03:02:49.504 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:50.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.017+0000 7f4587385640 1 -- 172.21.15.116:0/3628053475 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45801036e0 msgr2=0x7f4580107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:50.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.017+0000 7f4587385640 1 --2- 172.21.15.116:0/3628053475 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45801036e0 0x7f4580107b90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4574009930 tx=0x7f457402a420 comp rx=0 tx=0).stop 2024-09-12T03:02:50.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.017+0000 7f4587385640 1 -- 172.21.15.116:0/3628053475 shutdown_connections 2024-09-12T03:02:50.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.017+0000 7f4587385640 1 --2- 172.21.15.116:0/3628053475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45801082a0 0x7f458010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.017+0000 7f4587385640 1 --2- 172.21.15.116:0/3628053475 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45801036e0 0x7f4580107b90 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.017+0000 7f4587385640 1 --2- 172.21.15.116:0/3628053475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4580102d30 0x7f4580103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.017+0000 7f4587385640 1 -- 172.21.15.116:0/3628053475 >> 172.21.15.116:0/3628053475 conn(0x7f45800fc7d0 msgr2=0x7f45800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:50.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.017+0000 7f4587385640 1 -- 172.21.15.116:0/3628053475 shutdown_connections 2024-09-12T03:02:50.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.017+0000 7f4587385640 1 -- 172.21.15.116:0/3628053475 wait complete. 2024-09-12T03:02:50.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.017+0000 7f4587385640 1 Processor -- start 2024-09-12T03:02:50.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.017+0000 7f4587385640 1 -- start start 2024-09-12T03:02:50.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.017+0000 7f4587385640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4580102d30 0x7f4580198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.018+0000 7f4587385640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45801036e0 0x7f4580198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.018+0000 7f4587385640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45801082a0 0x7f458019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.018+0000 7f4587385640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f45801120c0 con 0x7f4580102d30 2024-09-12T03:02:50.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.018+0000 7f4587385640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4580111f40 con 0x7f45801082a0 2024-09-12T03:02:50.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.018+0000 7f4587385640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4580112240 con 0x7f45801036e0 2024-09-12T03:02:50.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.018+0000 7f4585121640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4580102d30 0x7f4580198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.018+0000 7f4585121640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4580102d30 0x7f4580198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33574/0 (socket says 172.21.15.116:33574) 2024-09-12T03:02:50.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.018+0000 7f4585121640 1 -- 172.21.15.116:0/319608432 learned_addr learned my addr 172.21.15.116:0/319608432 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:50.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.018+0000 7f4584920640 1 --2- 172.21.15.116:0/319608432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45801036e0 0x7f4580198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.018+0000 7f4585922640 1 --2- 172.21.15.116:0/319608432 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45801082a0 0x7f458019cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.018+0000 7f4585121640 1 -- 172.21.15.116:0/319608432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45801036e0 msgr2=0x7f4580198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:50.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.018+0000 7f4585121640 1 --2- 172.21.15.116:0/319608432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45801036e0 0x7f4580198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.018+0000 7f4585121640 1 -- 172.21.15.116:0/319608432 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45801082a0 msgr2=0x7f458019cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:50.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.018+0000 7f4585121640 1 --2- 172.21.15.116:0/319608432 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45801082a0 0x7f458019cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.018+0000 7f4585121640 1 -- 172.21.15.116:0/319608432 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458019d5d0 con 0x7f4580102d30 2024-09-12T03:02:50.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.018+0000 7f4585121640 1 --2- 172.21.15.116:0/319608432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4580102d30 0x7f4580198620 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f457000cce0 tx=0x7f457000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:50.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.018+0000 7f457e7fc640 1 -- 172.21.15.116:0/319608432 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4570004b20 con 0x7f4580102d30 2024-09-12T03:02:50.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.018+0000 7f457e7fc640 1 -- 172.21.15.116:0/319608432 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4570004cc0 con 0x7f4580102d30 2024-09-12T03:02:50.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.018+0000 7f457e7fc640 1 -- 172.21.15.116:0/319608432 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457000f7d0 con 0x7f4580102d30 2024-09-12T03:02:50.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.018+0000 7f4587385640 1 -- 172.21.15.116:0/319608432 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f458019d8c0 con 0x7f4580102d30 2024-09-12T03:02:50.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.018+0000 7f4587385640 1 -- 172.21.15.116:0/319608432 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45801a51a0 con 0x7f4580102d30 2024-09-12T03:02:50.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.020+0000 7f457e7fc640 1 -- 172.21.15.116:0/319608432 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f457000f970 con 0x7f4580102d30 2024-09-12T03:02:50.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.021+0000 7f458975e740 1 Processor -- start 2024-09-12T03:02:50.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.021+0000 7f458975e740 1 -- start start 2024-09-12T03:02:50.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.021+0000 7f457e7fc640 1 --2- 172.21.15.116:0/319608432 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f45680a2c00 0x7f45680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.021+0000 7f457e7fc640 1 -- 172.21.15.116:0/319608432 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f45700cf100 con 0x7f4580102d30 2024-09-12T03:02:50.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.021+0000 7f4585922640 1 --2- 172.21.15.116:0/319608432 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f45680a2c00 0x7f45680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.021+0000 7f458975e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bb0ebf9d0 0x561bb0ebfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.021+0000 7f458975e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bb0ec02f0 0x561bb0eca670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.021+0000 7f458975e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561bb0ecabb0 0x561bb0eccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.021+0000 7f458975e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561bb0e26090 con 0x561bb0ec02f0 2024-09-12T03:02:50.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.021+0000 7f458975e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561bb0dcda70 con 0x561bb0ecabb0 2024-09-12T03:02:50.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.021+0000 7f458975e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561bb0e22050 con 0x561bb0ebf9d0 2024-09-12T03:02:50.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.021+0000 7f4584920640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bb0ec02f0 0x561bb0eca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.021+0000 7f4584920640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bb0ec02f0 0x561bb0eca670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33588/0 (socket says 172.21.15.116:33588) 2024-09-12T03:02:50.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.021+0000 7f4584920640 1 -- 172.21.15.116:0/1381586772 learned_addr learned my addr 172.21.15.116:0/1381586772 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:50.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.022+0000 7f4585922640 1 --2- 172.21.15.116:0/319608432 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f45680a2c00 0x7f45680a50c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f457800a800 tx=0x7f457800ac00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:50.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.022+0000 7f4584920640 1 -- 172.21.15.116:0/1381586772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bb0ebf9d0 msgr2=0x561bb0ebfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:50.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.022+0000 7f4584920640 1 --2- 172.21.15.116:0/1381586772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bb0ebf9d0 0x561bb0ebfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.022+0000 7f4585922640 1 --2- 172.21.15.116:0/1381586772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561bb0ecabb0 0x561bb0eccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.022+0000 7f4584920640 1 -- 172.21.15.116:0/1381586772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561bb0ecabb0 msgr2=0x561bb0eccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:50.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.022+0000 7f4584920640 1 --2- 172.21.15.116:0/1381586772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561bb0ecabb0 0x561bb0eccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.022+0000 7f4584920640 1 -- 172.21.15.116:0/1381586772 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bb0dda130 con 0x561bb0ec02f0 2024-09-12T03:02:50.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.022+0000 7f4584920640 1 --2- 172.21.15.116:0/1381586772 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bb0ec02f0 0x561bb0eca670 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f45800697d0 tx=0x7f4574031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3e1de8c744aa17 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:50.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.022+0000 7f45677fe640 1 -- 172.21.15.116:0/1381586772 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457403f070 con 0x561bb0ec02f0 2024-09-12T03:02:50.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.022+0000 7f45677fe640 1 -- 172.21.15.116:0/1381586772 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4574031a20 con 0x561bb0ec02f0 2024-09-12T03:02:50.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.022+0000 7f45677fe640 1 -- 172.21.15.116:0/1381586772 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4574035070 con 0x561bb0ec02f0 2024-09-12T03:02:50.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.023+0000 7f458975e740 1 -- 172.21.15.116:0/1381586772 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bb0ec02f0 msgr2=0x561bb0eca670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:50.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.023+0000 7f458975e740 1 --2- 172.21.15.116:0/1381586772 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bb0ec02f0 0x561bb0eca670 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f45800697d0 tx=0x7f4574031630 comp rx=0 tx=0).stop 2024-09-12T03:02:50.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.023+0000 7f458975e740 1 -- 172.21.15.116:0/1381586772 shutdown_connections 2024-09-12T03:02:50.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.023+0000 7f458975e740 1 --2- 172.21.15.116:0/1381586772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561bb0ecabb0 0x561bb0eccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.023+0000 7f458975e740 1 --2- 172.21.15.116:0/1381586772 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bb0ec02f0 0x561bb0eca670 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.023+0000 7f458975e740 1 --2- 172.21.15.116:0/1381586772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bb0ebf9d0 0x561bb0ebfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.023+0000 7f458975e740 1 -- 172.21.15.116:0/1381586772 >> 172.21.15.116:0/1381586772 conn(0x561bb0ebbf60 msgr2=0x561bb0ebc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:50.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.023+0000 7f458975e740 1 -- 172.21.15.116:0/1381586772 shutdown_connections 2024-09-12T03:02:50.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.023+0000 7f458975e740 1 -- 172.21.15.116:0/1381586772 wait complete. 2024-09-12T03:02:50.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.023+0000 7f458975e740 1 -- start start 2024-09-12T03:02:50.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.023+0000 7f458975e740 1 Processor -- start 2024-09-12T03:02:50.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.024+0000 7f458975e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561bb0ebf9d0 0x561bb0eca530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.024+0000 7f458975e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bb0ec02f0 0x561bb0ec7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.024+0000 7f458975e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bb0ecabb0 0x561bb0ec7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.024+0000 7f458975e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561bb0ecd780 con 0x561bb0ec02f0 2024-09-12T03:02:50.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.024+0000 7f458975e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561bb0de0f20 con 0x561bb0ebf9d0 2024-09-12T03:02:50.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.024+0000 7f458975e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561bb0ecd900 con 0x561bb0ecabb0 2024-09-12T03:02:50.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.024+0000 7f458975e740 10 client.0 _ll_register_callbacks cb 0x561bb0daa030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:50.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.024+0000 7f4584920640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bb0ec02f0 0x561bb0ec7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.024+0000 7f4584920640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bb0ec02f0 0x561bb0ec7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33592/0 (socket says 172.21.15.116:33592) 2024-09-12T03:02:50.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.024+0000 7f4584920640 1 -- 172.21.15.116:0/3859298964 learned_addr learned my addr 172.21.15.116:0/3859298964 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:50.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.024+0000 7f4585121640 1 --2- 172.21.15.116:0/3859298964 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561bb0ebf9d0 0x561bb0eca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.024+0000 7f4584920640 1 -- 172.21.15.116:0/3859298964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bb0ecabb0 msgr2=0x561bb0ec7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:50.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.024+0000 7f4585922640 1 --2- 172.21.15.116:0/3859298964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bb0ecabb0 0x561bb0ec7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.024+0000 7f4584920640 1 --2- 172.21.15.116:0/3859298964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bb0ecabb0 0x561bb0ec7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.024+0000 7f4584920640 1 -- 172.21.15.116:0/3859298964 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561bb0ebf9d0 msgr2=0x561bb0eca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:50.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.024+0000 7f4584920640 1 --2- 172.21.15.116:0/3859298964 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561bb0ebf9d0 0x561bb0eca530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.024+0000 7f4584920640 1 -- 172.21.15.116:0/3859298964 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bb0b972b0 con 0x561bb0ec02f0 2024-09-12T03:02:50.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.024+0000 7f4585121640 1 --2- 172.21.15.116:0/3859298964 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561bb0ebf9d0 0x561bb0eca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:50.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.024+0000 7f4584920640 1 --2- 172.21.15.116:0/3859298964 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bb0ec02f0 0x561bb0ec7880 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4574004030 tx=0x7f4574003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670c183cf9e7ea94 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:50.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.024+0000 7f4564ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:02:50.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.024+0000 7f4564ff9640 1 -- 172.21.15.116:0/3859298964 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4574035070 con 0x561bb0ec02f0 2024-09-12T03:02:50.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.024+0000 7f4564ff9640 1 -- 172.21.15.116:0/3859298964 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f457403b430 con 0x561bb0ec02f0 2024-09-12T03:02:50.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.024+0000 7f458975e740 1 -- 172.21.15.116:0/3859298964 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4574031a20 con 0x561bb0ec02f0 2024-09-12T03:02:50.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.025+0000 7f4564ff9640 1 -- 172.21.15.116:0/3859298964 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f457403f040 con 0x561bb0ec02f0 2024-09-12T03:02:50.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.025+0000 7f458975e740 10 client.15129 fetch_fsmap learned FSMap version 24 2024-09-12T03:02:50.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.025+0000 7f458975e740 1 -- 172.21.15.116:0/3859298964 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561bb0ed9170 con 0x561bb0ec02f0 2024-09-12T03:02:50.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.025+0000 7f4564ff9640 1 -- 172.21.15.116:0/3859298964 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4574031a20 con 0x561bb0ec02f0 2024-09-12T03:02:50.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.025+0000 7f4564ff9640 1 -- 172.21.15.116:0/3859298964 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f4574050790 con 0x561bb0ec02f0 2024-09-12T03:02:50.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.025+0000 7f458975e740 10 client.15129 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:02:50.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.025+0000 7f458975e740 10 client.15129 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T03:02:50.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.025+0000 7f458975e740 20 client.15129 populate_metadata read hostname 'smithi116' 2024-09-12T03:02:50.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.025+0000 7f458975e740 1 --2- 172.21.15.116:0/3859298964 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x561bb0edb270 0x561bb0edc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.025+0000 7f458975e740 4 client.15129 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:02:50.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.025+0000 7f458975e740 1 -- 172.21.15.116:0/3859298964 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bb0e33600 con 0x561bb0edb270 2024-09-12T03:02:50.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.025+0000 7f4585121640 1 --2- 172.21.15.116:0/3859298964 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x561bb0edb270 0x561bb0edc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.026+0000 7f4585121640 1 --2- 172.21.15.116:0/3859298964 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x561bb0edb270 0x561bb0edc6e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec28c3989fe357f5 server_cookie=c2e2c0225de0cb9 in_seq=0 out_seq=0 2024-09-12T03:02:50.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.026+0000 7f4564ff9640 10 client.15129 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:02:50.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.026+0000 7f4564ff9640 1 -- 172.21.15.116:0/3859298964 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561bb0e33600 con 0x561bb0edb270 2024-09-12T03:02:50.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.026+0000 7f4564ff9640 10 client.15129 handle_command_reply: tid=0 2024-09-12T03:02:50.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.027+0000 7f458975e740 1 client.15129 shutdown 2024-09-12T03:02:50.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.027+0000 7f458975e740 1 -- 172.21.15.116:0/3859298964 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bb0ec02f0 msgr2=0x561bb0ec7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:50.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.027+0000 7f458975e740 1 --2- 172.21.15.116:0/3859298964 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bb0ec02f0 0x561bb0ec7880 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4574004030 tx=0x7f4574003b60 comp rx=0 tx=0).stop 2024-09-12T03:02:50.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.027+0000 7f458975e740 1 -- 172.21.15.116:0/3859298964 shutdown_connections 2024-09-12T03:02:50.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.027+0000 7f458975e740 1 --2- 172.21.15.116:0/3859298964 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x561bb0edb270 0x561bb0edc6e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.027+0000 7f458975e740 1 --2- 172.21.15.116:0/3859298964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bb0ecabb0 0x561bb0ec7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.027+0000 7f458975e740 1 --2- 172.21.15.116:0/3859298964 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bb0ec02f0 0x561bb0ec7880 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.027+0000 7f4564ff9640 0 client.15129 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:02:50.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.027+0000 7f458975e740 1 --2- 172.21.15.116:0/3859298964 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561bb0ebf9d0 0x561bb0eca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.027+0000 7f458975e740 1 -- 172.21.15.116:0/3859298964 >> 172.21.15.116:0/3859298964 conn(0x561bb0ebbf60 msgr2=0x561bb0ec1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:50.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.027+0000 7f458975e740 1 -- 172.21.15.116:0/3859298964 shutdown_connections 2024-09-12T03:02:50.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.027+0000 7f458975e740 1 -- 172.21.15.116:0/3859298964 wait complete. 2024-09-12T03:02:50.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.027+0000 7f458975e740 20 client.15129 trim_cache size 0 max 16384 2024-09-12T03:02:50.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.045+0000 7f458975e740 1 Processor -- start 2024-09-12T03:02:50.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.045+0000 7f458975e740 1 -- start start 2024-09-12T03:02:50.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.045+0000 7f458975e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bb0ebf9d0 0x561bb0edaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.045+0000 7f458975e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561bb0ec02f0 0x561bb0ec97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.045+0000 7f458975e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bb0ecabb0 0x561bb0ee4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.045+0000 7f458975e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561bb0edaa20 con 0x561bb0ebf9d0 2024-09-12T03:02:50.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.045+0000 7f458975e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561bb0edb0f0 con 0x561bb0ec02f0 2024-09-12T03:02:50.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.045+0000 7f458975e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561bb0edb7c0 con 0x561bb0ecabb0 2024-09-12T03:02:50.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.045+0000 7f4585121640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bb0ebf9d0 0x561bb0edaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.045+0000 7f4585121640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bb0ebf9d0 0x561bb0edaba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33602/0 (socket says 172.21.15.116:33602) 2024-09-12T03:02:50.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.045+0000 7f4585121640 1 -- 172.21.15.116:0/1779231854 learned_addr learned my addr 172.21.15.116:0/1779231854 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:50.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.045+0000 7f4584920640 1 --2- 172.21.15.116:0/1779231854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561bb0ec02f0 0x561bb0ec97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.045+0000 7f4585121640 1 -- 172.21.15.116:0/1779231854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bb0ecabb0 msgr2=0x561bb0ee4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:50.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.045+0000 7f4585121640 1 --2- 172.21.15.116:0/1779231854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bb0ecabb0 0x561bb0ee4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.045+0000 7f4585121640 1 -- 172.21.15.116:0/1779231854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561bb0ec02f0 msgr2=0x561bb0ec97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:50.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.045+0000 7f4585121640 1 --2- 172.21.15.116:0/1779231854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561bb0ec02f0 0x561bb0ec97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.045+0000 7f4585121640 1 -- 172.21.15.116:0/1779231854 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bb0ebf800 con 0x561bb0ebf9d0 2024-09-12T03:02:50.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.045+0000 7f4584920640 1 --2- 172.21.15.116:0/1779231854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561bb0ec02f0 0x561bb0ec97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:50.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.045+0000 7f4585121640 1 --2- 172.21.15.116:0/1779231854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bb0ebf9d0 0x561bb0edaba0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f45700058f0 tx=0x7f4570005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=effa9b899c0c1a8a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:50.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.045+0000 7f4566ffd640 1 -- 172.21.15.116:0/1779231854 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45700b3070 con 0x561bb0ebf9d0 2024-09-12T03:02:50.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.046+0000 7f4566ffd640 1 -- 172.21.15.116:0/1779231854 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45700a4600 con 0x561bb0ebf9d0 2024-09-12T03:02:50.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.046+0000 7f4566ffd640 1 -- 172.21.15.116:0/1779231854 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457001d040 con 0x561bb0ebf9d0 2024-09-12T03:02:50.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.046+0000 7f458975e740 1 -- 172.21.15.116:0/1779231854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bb0ebf9d0 msgr2=0x561bb0edaba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:50.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.046+0000 7f458975e740 1 --2- 172.21.15.116:0/1779231854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bb0ebf9d0 0x561bb0edaba0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f45700058f0 tx=0x7f4570005840 comp rx=0 tx=0).stop 2024-09-12T03:02:50.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.046+0000 7f458975e740 1 -- 172.21.15.116:0/1779231854 shutdown_connections 2024-09-12T03:02:50.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.046+0000 7f458975e740 1 --2- 172.21.15.116:0/1779231854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bb0ecabb0 0x561bb0ee4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.046+0000 7f458975e740 1 --2- 172.21.15.116:0/1779231854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561bb0ec02f0 0x561bb0ec97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.046+0000 7f458975e740 1 --2- 172.21.15.116:0/1779231854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bb0ebf9d0 0x561bb0edaba0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.046+0000 7f458975e740 1 -- 172.21.15.116:0/1779231854 >> 172.21.15.116:0/1779231854 conn(0x561bb0ebbf60 msgr2=0x561bb0eb6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:50.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.046+0000 7f458975e740 1 -- 172.21.15.116:0/1779231854 shutdown_connections 2024-09-12T03:02:50.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.046+0000 7f458975e740 1 -- 172.21.15.116:0/1779231854 wait complete. 2024-09-12T03:02:50.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.046+0000 7f458975e740 1 -- start start 2024-09-12T03:02:50.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.047+0000 7f458975e740 1 Processor -- start 2024-09-12T03:02:50.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.047+0000 7f458975e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bb0ebf9d0 0x561bb0f3db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.047+0000 7f458975e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561bb0ec02f0 0x561bb0f3e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.047+0000 7f458975e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bb0ecabb0 0x561bb0f489c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.047+0000 7f458975e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561bb0edc560 con 0x561bb0ebf9d0 2024-09-12T03:02:50.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.047+0000 7f458975e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561bb0edbe90 con 0x561bb0ec02f0 2024-09-12T03:02:50.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.047+0000 7f458975e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561bb0edcc30 con 0x561bb0ecabb0 2024-09-12T03:02:50.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.047+0000 7f458975e740 10 client.0 _ll_register_callbacks cb 0x7f4574005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:50.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.047+0000 7f4585121640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bb0ebf9d0 0x561bb0f3db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.047+0000 7f4585121640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bb0ebf9d0 0x561bb0f3db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33616/0 (socket says 172.21.15.116:33616) 2024-09-12T03:02:50.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.047+0000 7f4585121640 1 -- 172.21.15.116:0/2230257906 learned_addr learned my addr 172.21.15.116:0/2230257906 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:50.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.047+0000 7f4584920640 1 --2- 172.21.15.116:0/2230257906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561bb0ec02f0 0x561bb0f3e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.047+0000 7f4585121640 1 -- 172.21.15.116:0/2230257906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bb0ecabb0 msgr2=0x561bb0f489c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:50.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.047+0000 7f4585121640 1 --2- 172.21.15.116:0/2230257906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bb0ecabb0 0x561bb0f489c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.047+0000 7f4585121640 1 -- 172.21.15.116:0/2230257906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561bb0ec02f0 msgr2=0x561bb0f3e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:50.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.047+0000 7f4585121640 1 --2- 172.21.15.116:0/2230257906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561bb0ec02f0 0x561bb0f3e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.047+0000 7f4585121640 1 -- 172.21.15.116:0/2230257906 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bb0f49000 con 0x561bb0ebf9d0 2024-09-12T03:02:50.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.047+0000 7f4584920640 1 --2- 172.21.15.116:0/2230257906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561bb0ec02f0 0x561bb0f3e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:50.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.048+0000 7f4585121640 1 --2- 172.21.15.116:0/2230257906 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bb0ebf9d0 0x561bb0f3db20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f457000ab20 tx=0x7f45700975c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2226f9abd1079467 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:50.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.048+0000 7f4564ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:02:50.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.048+0000 7f4564ff9640 1 -- 172.21.15.116:0/2230257906 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457001d070 con 0x561bb0ebf9d0 2024-09-12T03:02:50.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.048+0000 7f4564ff9640 1 -- 172.21.15.116:0/2230257906 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f45700ba2c0 con 0x561bb0ebf9d0 2024-09-12T03:02:50.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.048+0000 7f458975e740 1 -- 172.21.15.116:0/2230257906 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45700a4600 con 0x561bb0ebf9d0 2024-09-12T03:02:50.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.048+0000 7f4564ff9640 1 -- 172.21.15.116:0/2230257906 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f45700b3040 con 0x561bb0ebf9d0 2024-09-12T03:02:50.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.048+0000 7f458975e740 10 client.15135 fetch_fsmap learned FSMap version 24 2024-09-12T03:02:50.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.048+0000 7f458975e740 1 -- 172.21.15.116:0/2230257906 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561bb0f4a210 con 0x561bb0ebf9d0 2024-09-12T03:02:50.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.048+0000 7f4564ff9640 1 -- 172.21.15.116:0/2230257906 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45700a4600 con 0x561bb0ebf9d0 2024-09-12T03:02:50.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.048+0000 7f4564ff9640 1 -- 172.21.15.116:0/2230257906 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f457009ade0 con 0x561bb0ebf9d0 2024-09-12T03:02:50.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.048+0000 7f458975e740 10 client.15135 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:02:50.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.048+0000 7f458975e740 10 client.15135 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T03:02:50.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.048+0000 7f458975e740 20 client.15135 populate_metadata read hostname 'smithi116' 2024-09-12T03:02:50.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.048+0000 7f458975e740 1 --2- 172.21.15.116:0/2230257906 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x561bb0edb270 0x561bb0f4d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:50.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.048+0000 7f458975e740 4 client.15135 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:02:50.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.048+0000 7f458975e740 1 -- 172.21.15.116:0/2230257906 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "status"}) -- 0x561bb0e2db10 con 0x561bb0edb270 2024-09-12T03:02:50.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.049+0000 7f4584920640 1 --2- 172.21.15.116:0/2230257906 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x561bb0edb270 0x561bb0f4d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:50.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.049+0000 7f4584920640 1 --2- 172.21.15.116:0/2230257906 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x561bb0edb270 0x561bb0f4d5a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2ccca8742d5cbec8 server_cookie=87b297919d319b4b in_seq=0 out_seq=0 2024-09-12T03:02:50.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.049+0000 7f4564ff9640 10 client.15135 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:02:50.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.049+0000 7f4564ff9640 1 -- 172.21.15.116:0/2230257906 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561bb0e2db10 con 0x561bb0edb270 2024-09-12T03:02:50.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.049+0000 7f4564ff9640 10 client.15135 handle_command_reply: tid=0 2024-09-12T03:02:50.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.049+0000 7f458975e740 1 client.15135 shutdown 2024-09-12T03:02:50.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.049+0000 7f458975e740 1 -- 172.21.15.116:0/2230257906 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bb0ebf9d0 msgr2=0x561bb0f3db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:50.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.049+0000 7f458975e740 1 --2- 172.21.15.116:0/2230257906 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bb0ebf9d0 0x561bb0f3db20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f457000ab20 tx=0x7f45700975c0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.049+0000 7f458975e740 1 -- 172.21.15.116:0/2230257906 shutdown_connections 2024-09-12T03:02:50.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.049+0000 7f458975e740 1 --2- 172.21.15.116:0/2230257906 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x561bb0edb270 0x561bb0f4d5a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.049+0000 7f458975e740 1 --2- 172.21.15.116:0/2230257906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561bb0ecabb0 0x561bb0f489c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.049+0000 7f458975e740 1 --2- 172.21.15.116:0/2230257906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561bb0ec02f0 0x561bb0f3e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.049+0000 7f458975e740 1 --2- 172.21.15.116:0/2230257906 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bb0ebf9d0 0x561bb0f3db20 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.049+0000 7f458975e740 1 -- 172.21.15.116:0/2230257906 >> 172.21.15.116:0/2230257906 conn(0x561bb0ebbf60 msgr2=0x561bb0ec7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:50.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.049+0000 7f4564ff9640 0 client.15135 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:02:50.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.050+0000 7f458975e740 1 -- 172.21.15.116:0/2230257906 shutdown_connections 2024-09-12T03:02:50.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.050+0000 7f458975e740 1 -- 172.21.15.116:0/2230257906 wait complete. 2024-09-12T03:02:50.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.050+0000 7f458975e740 20 client.15135 trim_cache size 0 max 16384 2024-09-12T03:02:50.066 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:02:50.066 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_fsid": "eb1ac602-70b1-11ef-bcea-c7b262605968", 2024-09-12T03:02:50.066 INFO:teuthology.orchestra.run.smithi116.stdout: "whoami": 1, 2024-09-12T03:02:50.066 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 24475, 2024-09-12T03:02:50.066 INFO:teuthology.orchestra.run.smithi116.stdout: "want_state": "up:active", 2024-09-12T03:02:50.067 INFO:teuthology.orchestra.run.smithi116.stdout: "state": "up:active", 2024-09-12T03:02:50.067 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs", 2024-09-12T03:02:50.067 INFO:teuthology.orchestra.run.smithi116.stdout: "rank_uptime": 103.668490969, 2024-09-12T03:02:50.067 INFO:teuthology.orchestra.run.smithi116.stdout: "mdsmap_epoch": 24, 2024-09-12T03:02:50.067 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch": 80, 2024-09-12T03:02:50.067 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch_barrier": 78, 2024-09-12T03:02:50.067 INFO:teuthology.orchestra.run.smithi116.stdout: "uptime": 130.21696322599999 2024-09-12T03:02:50.067 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:02:50.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.050+0000 7f4587385640 1 -- 172.21.15.116:0/319608432 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f45680a2c00 msgr2=0x7f45680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:50.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.050+0000 7f4587385640 1 --2- 172.21.15.116:0/319608432 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f45680a2c00 0x7f45680a50c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f457800a800 tx=0x7f457800ac00 comp rx=0 tx=0).stop 2024-09-12T03:02:50.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.050+0000 7f4587385640 1 -- 172.21.15.116:0/319608432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4580102d30 msgr2=0x7f4580198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:50.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.050+0000 7f4587385640 1 --2- 172.21.15.116:0/319608432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4580102d30 0x7f4580198620 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f457000cce0 tx=0x7f457000a620 comp rx=0 tx=0).stop 2024-09-12T03:02:50.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.050+0000 7f4587385640 1 -- 172.21.15.116:0/319608432 shutdown_connections 2024-09-12T03:02:50.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.050+0000 7f4587385640 1 --2- 172.21.15.116:0/319608432 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f45680a2c00 0x7f45680a50c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.050+0000 7f4587385640 1 --2- 172.21.15.116:0/319608432 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45801082a0 0x7f458019cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.050+0000 7f4587385640 1 --2- 172.21.15.116:0/319608432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45801036e0 0x7f4580198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.050+0000 7f4587385640 1 --2- 172.21.15.116:0/319608432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4580102d30 0x7f4580198620 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:50.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.050+0000 7f4587385640 1 -- 172.21.15.116:0/319608432 >> 172.21.15.116:0/319608432 conn(0x7f45800fc7d0 msgr2=0x7f4580105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:50.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.050+0000 7f4587385640 1 -- 172.21.15.116:0/319608432 shutdown_connections 2024-09-12T03:02:50.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:50.050+0000 7f4587385640 1 -- 172.21.15.116:0/319608432 wait complete. 2024-09-12T03:02:50.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:50 smithi177 ceph-mon[35305]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:02:50.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:50 smithi116 ceph-mon[31410]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:02:50.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:50 smithi183 ceph-mon[35535]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:02:50.853 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T03:02:51.099 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:51.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.591+0000 7f2ee6313640 1 -- 172.21.15.116:0/3360249605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ee0101de0 msgr2=0x7f2ee01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:51.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.591+0000 7f2ee6313640 1 --2- 172.21.15.116:0/3360249605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ee0101de0 0x7f2ee01021c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0009900 tx=0x7f2ed002a410 comp rx=0 tx=0).stop 2024-09-12T03:02:51.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.591+0000 7f2ee6313640 1 -- 172.21.15.116:0/3360249605 shutdown_connections 2024-09-12T03:02:51.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.591+0000 7f2ee6313640 1 --2- 172.21.15.116:0/3360249605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ee0106f10 0x7f2ee010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.591+0000 7f2ee6313640 1 --2- 172.21.15.116:0/3360249605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ee0102790 0x7f2ee01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.591+0000 7f2ee6313640 1 --2- 172.21.15.116:0/3360249605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ee0101de0 0x7f2ee01021c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.591+0000 7f2ee6313640 1 -- 172.21.15.116:0/3360249605 >> 172.21.15.116:0/3360249605 conn(0x7f2ee00fd8c0 msgr2=0x7f2ee00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:51.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.591+0000 7f2ee6313640 1 -- 172.21.15.116:0/3360249605 shutdown_connections 2024-09-12T03:02:51.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.591+0000 7f2ee6313640 1 -- 172.21.15.116:0/3360249605 wait complete. 2024-09-12T03:02:51.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.592+0000 7f2ee6313640 1 Processor -- start 2024-09-12T03:02:51.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.592+0000 7f2ee6313640 1 -- start start 2024-09-12T03:02:51.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.592+0000 7f2ee6313640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ee0101de0 0x7f2ee0198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.592+0000 7f2ee6313640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ee0102790 0x7f2ee0198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.592+0000 7f2ee6313640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ee0106f10 0x7f2ee019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.592+0000 7f2ee6313640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee0111f70 con 0x7f2ee0106f10 2024-09-12T03:02:51.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.592+0000 7f2ee6313640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee0111df0 con 0x7f2ee0102790 2024-09-12T03:02:51.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.592+0000 7f2ee6313640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee01120f0 con 0x7f2ee0101de0 2024-09-12T03:02:51.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.592+0000 7f2ee48b0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ee0106f10 0x7f2ee019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.592+0000 7f2ee48b0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ee0106f10 0x7f2ee019cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58206/0 (socket says 172.21.15.116:58206) 2024-09-12T03:02:51.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.592+0000 7f2ee48b0640 1 -- 172.21.15.116:0/1878145011 learned_addr learned my addr 172.21.15.116:0/1878145011 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:51.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.592+0000 7f2edffff640 1 --2- 172.21.15.116:0/1878145011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ee0101de0 0x7f2ee0198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.592+0000 7f2edf7fe640 1 --2- 172.21.15.116:0/1878145011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ee0102790 0x7f2ee0198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.593+0000 7f2ee48b0640 1 -- 172.21.15.116:0/1878145011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ee0101de0 msgr2=0x7f2ee0198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:51.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.593+0000 7f2ee48b0640 1 --2- 172.21.15.116:0/1878145011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ee0101de0 0x7f2ee0198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.593+0000 7f2ee48b0640 1 -- 172.21.15.116:0/1878145011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ee0102790 msgr2=0x7f2ee0198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:51.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.593+0000 7f2ee48b0640 1 --2- 172.21.15.116:0/1878145011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ee0102790 0x7f2ee0198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.593+0000 7f2ee48b0640 1 -- 172.21.15.116:0/1878145011 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee019d3e0 con 0x7f2ee0106f10 2024-09-12T03:02:51.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.593+0000 7f2edf7fe640 1 --2- 172.21.15.116:0/1878145011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ee0102790 0x7f2ee0198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:51.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.593+0000 7f2ee48b0640 1 --2- 172.21.15.116:0/1878145011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ee0106f10 0x7f2ee019cd00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800eaa0 tx=0x7f2ed800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:51.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.593+0000 7f2edd7fa640 1 -- 172.21.15.116:0/1878145011 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed800cdb0 con 0x7f2ee0106f10 2024-09-12T03:02:51.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.593+0000 7f2ee6313640 1 -- 172.21.15.116:0/1878145011 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee019d6d0 con 0x7f2ee0106f10 2024-09-12T03:02:51.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.593+0000 7f2edd7fa640 1 -- 172.21.15.116:0/1878145011 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed8005020 con 0x7f2ee0106f10 2024-09-12T03:02:51.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.593+0000 7f2edd7fa640 1 -- 172.21.15.116:0/1878145011 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed80116d0 con 0x7f2ee0106f10 2024-09-12T03:02:51.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.593+0000 7f2ee6313640 1 -- 172.21.15.116:0/1878145011 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee01a4fb0 con 0x7f2ee0106f10 2024-09-12T03:02:51.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.595+0000 7f2edd7fa640 1 -- 172.21.15.116:0/1878145011 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2ed80054d0 con 0x7f2ee0106f10 2024-09-12T03:02:51.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.596+0000 7f2ee86ec740 1 Processor -- start 2024-09-12T03:02:51.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.596+0000 7f2ee86ec740 1 -- start start 2024-09-12T03:02:51.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.596+0000 7f2edd7fa640 1 --2- 172.21.15.116:0/1878145011 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2ecc0a2b80 0x7f2ecc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.596+0000 7f2ee86ec740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1fae419d0 0x55d1fae41db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.596+0000 7f2ee86ec740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1fae422f0 0x55d1fae4c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.596+0000 7f2edf7fe640 1 --2- 172.21.15.116:0/1878145011 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2ecc0a2b80 0x7f2ecc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.596+0000 7f2ee86ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1fae4cbb0 0x55d1fae4efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.596+0000 7f2ee86ec740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d1fada8090 con 0x55d1fae419d0 2024-09-12T03:02:51.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.596+0000 7f2ee86ec740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d1fad4fa70 con 0x55d1fae422f0 2024-09-12T03:02:51.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.596+0000 7f2ee86ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d1fada4050 con 0x55d1fae4cbb0 2024-09-12T03:02:51.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.596+0000 7f2edd7fa640 1 -- 172.21.15.116:0/1878145011 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f2ed80cf210 con 0x7f2ee0106f10 2024-09-12T03:02:51.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.596+0000 7f2edffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1fae419d0 0x55d1fae41db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.596+0000 7f2edffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1fae419d0 0x55d1fae41db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58210/0 (socket says 172.21.15.116:58210) 2024-09-12T03:02:51.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.596+0000 7f2edffff640 1 -- 172.21.15.116:0/1346607122 learned_addr learned my addr 172.21.15.116:0/1346607122 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:51.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.596+0000 7f2ee48b0640 1 --2- 172.21.15.116:0/1346607122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1fae4cbb0 0x55d1fae4efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.596+0000 7f2edf7fe640 1 --2- 172.21.15.116:0/1346607122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1fae422f0 0x55d1fae4c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.596+0000 7f2edffff640 1 -- 172.21.15.116:0/1346607122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1fae4cbb0 msgr2=0x55d1fae4efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:51.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.596+0000 7f2edffff640 1 --2- 172.21.15.116:0/1346607122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1fae4cbb0 0x55d1fae4efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.596+0000 7f2edffff640 1 -- 172.21.15.116:0/1346607122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1fae422f0 msgr2=0x55d1fae4c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:51.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.596+0000 7f2edffff640 1 --2- 172.21.15.116:0/1346607122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1fae422f0 0x55d1fae4c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.596+0000 7f2edffff640 1 -- 172.21.15.116:0/1346607122 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1fad5c130 con 0x55d1fae419d0 2024-09-12T03:02:51.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.596+0000 7f2edf7fe640 1 --2- 172.21.15.116:0/1346607122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1fae422f0 0x55d1fae4c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:51.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.596+0000 7f2edf7fe640 1 --2- 172.21.15.116:0/1878145011 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2ecc0a2b80 0x7f2ecc0a5040 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8005fd0 tx=0x7f2ec800e350 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:51.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.597+0000 7f2edffff640 1 --2- 172.21.15.116:0/1346607122 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1fae419d0 0x55d1fae41db0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0009900 tx=0x7f2ed0031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b5bf83298d2ff6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:51.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.597+0000 7f2ed67fc640 1 -- 172.21.15.116:0/1346607122 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0040070 con 0x55d1fae419d0 2024-09-12T03:02:51.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.597+0000 7f2ed67fc640 1 -- 172.21.15.116:0/1346607122 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed0005020 con 0x55d1fae419d0 2024-09-12T03:02:51.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.597+0000 7f2ed67fc640 1 -- 172.21.15.116:0/1346607122 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0035040 con 0x55d1fae419d0 2024-09-12T03:02:51.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.597+0000 7f2ee86ec740 1 -- 172.21.15.116:0/1346607122 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1fae419d0 msgr2=0x55d1fae41db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:51.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.597+0000 7f2ee86ec740 1 --2- 172.21.15.116:0/1346607122 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1fae419d0 0x55d1fae41db0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0009900 tx=0x7f2ed0031630 comp rx=0 tx=0).stop 2024-09-12T03:02:51.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.597+0000 7f2ee86ec740 1 -- 172.21.15.116:0/1346607122 shutdown_connections 2024-09-12T03:02:51.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.597+0000 7f2ee86ec740 1 --2- 172.21.15.116:0/1346607122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1fae4cbb0 0x55d1fae4efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.597+0000 7f2ee86ec740 1 --2- 172.21.15.116:0/1346607122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1fae422f0 0x55d1fae4c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.597+0000 7f2ee86ec740 1 --2- 172.21.15.116:0/1346607122 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1fae419d0 0x55d1fae41db0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.597+0000 7f2ee86ec740 1 -- 172.21.15.116:0/1346607122 >> 172.21.15.116:0/1346607122 conn(0x55d1fae3df60 msgr2=0x55d1fae3e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:51.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.598+0000 7f2ee86ec740 1 -- 172.21.15.116:0/1346607122 shutdown_connections 2024-09-12T03:02:51.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.598+0000 7f2ee86ec740 1 -- 172.21.15.116:0/1346607122 wait complete. 2024-09-12T03:02:51.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.598+0000 7f2ee86ec740 1 -- start start 2024-09-12T03:02:51.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.598+0000 7f2ee86ec740 1 Processor -- start 2024-09-12T03:02:51.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.598+0000 7f2ee86ec740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1fae419d0 0x55d1fae4c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.598+0000 7f2ee86ec740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1fae422f0 0x55d1fae496a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.598+0000 7f2ee86ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1fae4cbb0 0x55d1fae49be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.598+0000 7f2ee86ec740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d1fae53b90 con 0x55d1fae422f0 2024-09-12T03:02:51.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.598+0000 7f2ee86ec740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d1fad62f20 con 0x55d1fae419d0 2024-09-12T03:02:51.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.598+0000 7f2ee86ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d1fae53d10 con 0x55d1fae4cbb0 2024-09-12T03:02:51.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.598+0000 7f2ee86ec740 10 client.0 _ll_register_callbacks cb 0x55d1fad2c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:51.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.598+0000 7f2edf7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1fae422f0 0x55d1fae496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.598+0000 7f2edf7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1fae422f0 0x55d1fae496a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58216/0 (socket says 172.21.15.116:58216) 2024-09-12T03:02:51.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.598+0000 7f2edf7fe640 1 -- 172.21.15.116:0/1097170455 learned_addr learned my addr 172.21.15.116:0/1097170455 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:51.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.599+0000 7f2ee48b0640 1 --2- 172.21.15.116:0/1097170455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1fae4cbb0 0x55d1fae49be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.599+0000 7f2edffff640 1 --2- 172.21.15.116:0/1097170455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1fae419d0 0x55d1fae4c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.599+0000 7f2edf7fe640 1 -- 172.21.15.116:0/1097170455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1fae4cbb0 msgr2=0x55d1fae49be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:51.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.599+0000 7f2edf7fe640 1 --2- 172.21.15.116:0/1097170455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1fae4cbb0 0x55d1fae49be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.599+0000 7f2edf7fe640 1 -- 172.21.15.116:0/1097170455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1fae419d0 msgr2=0x55d1fae4c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:51.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.599+0000 7f2edf7fe640 1 --2- 172.21.15.116:0/1097170455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1fae419d0 0x55d1fae4c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.599+0000 7f2edf7fe640 1 -- 172.21.15.116:0/1097170455 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1fab192b0 con 0x55d1fae422f0 2024-09-12T03:02:51.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.599+0000 7f2edffff640 1 --2- 172.21.15.116:0/1097170455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1fae419d0 0x55d1fae4c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:51.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.599+0000 7f2edf7fe640 1 --2- 172.21.15.116:0/1097170455 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1fae422f0 0x55d1fae496a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8010a70 tx=0x7f2ec8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f701c34f4d8aed0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:51.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.599+0000 7f2ebf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:02:51.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.599+0000 7f2ebf7fe640 1 -- 172.21.15.116:0/1097170455 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8023020 con 0x55d1fae422f0 2024-09-12T03:02:51.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.599+0000 7f2ebf7fe640 1 -- 172.21.15.116:0/1097170455 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f2ec801f450 con 0x55d1fae422f0 2024-09-12T03:02:51.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.599+0000 7f2ee86ec740 1 -- 172.21.15.116:0/1097170455 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ed0005020 con 0x55d1fae422f0 2024-09-12T03:02:51.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.599+0000 7f2ebf7fe640 1 -- 172.21.15.116:0/1097170455 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2ec801f650 con 0x55d1fae422f0 2024-09-12T03:02:51.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.599+0000 7f2ee86ec740 10 client.15147 fetch_fsmap learned FSMap version 24 2024-09-12T03:02:51.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.599+0000 7f2ee86ec740 1 -- 172.21.15.116:0/1097170455 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d1fae5f370 con 0x55d1fae422f0 2024-09-12T03:02:51.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.599+0000 7f2ebf7fe640 1 -- 172.21.15.116:0/1097170455 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed0005020 con 0x55d1fae422f0 2024-09-12T03:02:51.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.600+0000 7f2ebf7fe640 1 -- 172.21.15.116:0/1097170455 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f2ec800f8f0 con 0x55d1fae422f0 2024-09-12T03:02:51.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.600+0000 7f2ee86ec740 10 client.15147 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:02:51.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.600+0000 7f2ee86ec740 10 client.15147 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:02:51.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.600+0000 7f2ee86ec740 20 client.15147 populate_metadata read hostname 'smithi116' 2024-09-12T03:02:51.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.600+0000 7f2ee86ec740 1 --2- 172.21.15.116:0/1097170455 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d1fae619c0 0x55d1fae62e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.600+0000 7f2ee86ec740 4 client.15147 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:02:51.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.600+0000 7f2ee86ec740 1 -- 172.21.15.116:0/1097170455 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1fadb5600 con 0x55d1fae619c0 2024-09-12T03:02:51.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.600+0000 7f2edffff640 1 --2- 172.21.15.116:0/1097170455 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d1fae619c0 0x55d1fae62e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.601+0000 7f2ebf7fe640 10 client.15147 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:02:51.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.601+0000 7f2edffff640 1 --2- 172.21.15.116:0/1097170455 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d1fae619c0 0x55d1fae62e30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eab727ec926352c2 server_cookie=c1c5e86e4928d10c in_seq=0 out_seq=0 2024-09-12T03:02:51.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.601+0000 7f2ebf7fe640 1 -- 172.21.15.116:0/1097170455 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1fadb5600 con 0x55d1fae619c0 2024-09-12T03:02:51.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.601+0000 7f2ebf7fe640 10 client.15147 handle_command_reply: tid=0 2024-09-12T03:02:51.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.601+0000 7f2ee86ec740 1 client.15147 shutdown 2024-09-12T03:02:51.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.601+0000 7f2ee86ec740 1 -- 172.21.15.116:0/1097170455 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1fae422f0 msgr2=0x55d1fae496a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:51.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.601+0000 7f2ee86ec740 1 --2- 172.21.15.116:0/1097170455 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1fae422f0 0x55d1fae496a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8010a70 tx=0x7f2ec8004730 comp rx=0 tx=0).stop 2024-09-12T03:02:51.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.602+0000 7f2ee86ec740 1 -- 172.21.15.116:0/1097170455 shutdown_connections 2024-09-12T03:02:51.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.602+0000 7f2ee86ec740 1 --2- 172.21.15.116:0/1097170455 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d1fae619c0 0x55d1fae62e30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.602+0000 7f2ee86ec740 1 --2- 172.21.15.116:0/1097170455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1fae4cbb0 0x55d1fae49be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.602+0000 7f2ee86ec740 1 --2- 172.21.15.116:0/1097170455 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1fae422f0 0x55d1fae496a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.602+0000 7f2ee86ec740 1 --2- 172.21.15.116:0/1097170455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1fae419d0 0x55d1fae4c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.602+0000 7f2ebf7fe640 0 client.15147 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:02:51.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.602+0000 7f2ee86ec740 1 -- 172.21.15.116:0/1097170455 >> 172.21.15.116:0/1097170455 conn(0x55d1fae3df60 msgr2=0x55d1fae43c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:51.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.602+0000 7f2ee86ec740 1 -- 172.21.15.116:0/1097170455 shutdown_connections 2024-09-12T03:02:51.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.602+0000 7f2ee86ec740 1 -- 172.21.15.116:0/1097170455 wait complete. 2024-09-12T03:02:51.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.602+0000 7f2ee86ec740 20 client.15147 trim_cache size 0 max 16384 2024-09-12T03:02:51.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.619+0000 7f2ee86ec740 1 Processor -- start 2024-09-12T03:02:51.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.619+0000 7f2ee86ec740 1 -- start start 2024-09-12T03:02:51.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.619+0000 7f2ee86ec740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1fae419d0 0x55d1fae62e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.619+0000 7f2ee86ec740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1fae422f0 0x55d1fae49660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.619+0000 7f2ee86ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1fae4cbb0 0x55d1fae49ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.619+0000 7f2ee86ec740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d1fae61170 con 0x55d1fae422f0 2024-09-12T03:02:51.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.619+0000 7f2ee86ec740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d1fae61840 con 0x55d1fae419d0 2024-09-12T03:02:51.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.619+0000 7f2ee86ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d1fae61f10 con 0x55d1fae4cbb0 2024-09-12T03:02:51.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.619+0000 7f2edf7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1fae422f0 0x55d1fae49660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.619+0000 7f2edf7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1fae422f0 0x55d1fae49660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58224/0 (socket says 172.21.15.116:58224) 2024-09-12T03:02:51.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.619+0000 7f2edf7fe640 1 -- 172.21.15.116:0/3526029781 learned_addr learned my addr 172.21.15.116:0/3526029781 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:51.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.620+0000 7f2ee48b0640 1 --2- 172.21.15.116:0/3526029781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1fae4cbb0 0x55d1fae49ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.620+0000 7f2edf7fe640 1 -- 172.21.15.116:0/3526029781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1fae4cbb0 msgr2=0x55d1fae49ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:51.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.620+0000 7f2edf7fe640 1 --2- 172.21.15.116:0/3526029781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1fae4cbb0 0x55d1fae49ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.620+0000 7f2edffff640 1 --2- 172.21.15.116:0/3526029781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1fae419d0 0x55d1fae62e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.620+0000 7f2edf7fe640 1 -- 172.21.15.116:0/3526029781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1fae419d0 msgr2=0x55d1fae62e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:51.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.620+0000 7f2edf7fe640 1 --2- 172.21.15.116:0/3526029781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1fae419d0 0x55d1fae62e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.620+0000 7f2edf7fe640 1 -- 172.21.15.116:0/3526029781 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1fae4c6c0 con 0x55d1fae422f0 2024-09-12T03:02:51.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.620+0000 7f2ee48b0640 1 --2- 172.21.15.116:0/3526029781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1fae4cbb0 0x55d1fae49ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:51.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.620+0000 7f2edffff640 1 --2- 172.21.15.116:0/3526029781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1fae419d0 0x55d1fae62e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:51.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.620+0000 7f2edf7fe640 1 --2- 172.21.15.116:0/3526029781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1fae422f0 0x55d1fae49660 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800fd10 tx=0x7f2ec800ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49cc8382fc6b5f2a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:51.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.620+0000 7f2ed5ffb640 1 -- 172.21.15.116:0/3526029781 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8023070 con 0x55d1fae422f0 2024-09-12T03:02:51.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.620+0000 7f2ed5ffb640 1 -- 172.21.15.116:0/3526029781 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ec8005500 con 0x55d1fae422f0 2024-09-12T03:02:51.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.620+0000 7f2ed5ffb640 1 -- 172.21.15.116:0/3526029781 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8004a00 con 0x55d1fae422f0 2024-09-12T03:02:51.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.621+0000 7f2ee86ec740 1 -- 172.21.15.116:0/3526029781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1fae422f0 msgr2=0x55d1fae49660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:51.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.621+0000 7f2ee86ec740 1 --2- 172.21.15.116:0/3526029781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1fae422f0 0x55d1fae49660 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800fd10 tx=0x7f2ec800ff80 comp rx=0 tx=0).stop 2024-09-12T03:02:51.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.621+0000 7f2ee86ec740 1 -- 172.21.15.116:0/3526029781 shutdown_connections 2024-09-12T03:02:51.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.621+0000 7f2ee86ec740 1 --2- 172.21.15.116:0/3526029781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1fae4cbb0 0x55d1fae49ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.621+0000 7f2ee86ec740 1 --2- 172.21.15.116:0/3526029781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1fae422f0 0x55d1fae49660 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.621+0000 7f2ee86ec740 1 --2- 172.21.15.116:0/3526029781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1fae419d0 0x55d1fae62e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.621+0000 7f2ee86ec740 1 -- 172.21.15.116:0/3526029781 >> 172.21.15.116:0/3526029781 conn(0x55d1fae3df60 msgr2=0x55d1fae38ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:51.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.621+0000 7f2ee86ec740 1 -- 172.21.15.116:0/3526029781 shutdown_connections 2024-09-12T03:02:51.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.621+0000 7f2ee86ec740 1 -- 172.21.15.116:0/3526029781 wait complete. 2024-09-12T03:02:51.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.621+0000 7f2ee86ec740 1 -- start start 2024-09-12T03:02:51.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.621+0000 7f2ee86ec740 1 Processor -- start 2024-09-12T03:02:51.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.621+0000 7f2ee86ec740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1fae419d0 0x55d1fae6b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.621+0000 7f2ee86ec740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1fae422f0 0x55d1fae6b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.621+0000 7f2ee86ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1fae4cbb0 0x55d1fae64610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.621+0000 7f2ee86ec740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d1fae62cb0 con 0x55d1fae422f0 2024-09-12T03:02:51.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.621+0000 7f2ee86ec740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d1fae625e0 con 0x55d1fae419d0 2024-09-12T03:02:51.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.621+0000 7f2ee86ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d1fae63380 con 0x55d1fae4cbb0 2024-09-12T03:02:51.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.621+0000 7f2ee86ec740 10 client.0 _ll_register_callbacks cb 0x7f2ec8016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:51.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.621+0000 7f2edf7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1fae422f0 0x55d1fae6b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.621+0000 7f2edf7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1fae422f0 0x55d1fae6b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58238/0 (socket says 172.21.15.116:58238) 2024-09-12T03:02:51.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.622+0000 7f2edf7fe640 1 -- 172.21.15.116:0/2001734472 learned_addr learned my addr 172.21.15.116:0/2001734472 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:51.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.622+0000 7f2ee48b0640 1 --2- 172.21.15.116:0/2001734472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1fae4cbb0 0x55d1fae64610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.622+0000 7f2edffff640 1 --2- 172.21.15.116:0/2001734472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1fae419d0 0x55d1fae6b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.622+0000 7f2edf7fe640 1 -- 172.21.15.116:0/2001734472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1fae4cbb0 msgr2=0x55d1fae64610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:51.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.622+0000 7f2edf7fe640 1 --2- 172.21.15.116:0/2001734472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1fae4cbb0 0x55d1fae64610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.622+0000 7f2edf7fe640 1 -- 172.21.15.116:0/2001734472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1fae419d0 msgr2=0x55d1fae6b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:51.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.622+0000 7f2edf7fe640 1 --2- 172.21.15.116:0/2001734472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1fae419d0 0x55d1fae6b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.622+0000 7f2edf7fe640 1 -- 172.21.15.116:0/2001734472 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1fae64e40 con 0x55d1fae422f0 2024-09-12T03:02:51.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.622+0000 7f2edf7fe640 1 --2- 172.21.15.116:0/2001734472 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1fae422f0 0x55d1fae6b820 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8012010 tx=0x7f2ec8012890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410cb264ccf647df server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:51.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.622+0000 7f2ebf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:02:51.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.622+0000 7f2ebf7fe640 1 -- 172.21.15.116:0/2001734472 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8023040 con 0x55d1fae422f0 2024-09-12T03:02:51.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.622+0000 7f2ebf7fe640 1 -- 172.21.15.116:0/2001734472 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f2ec8005230 con 0x55d1fae422f0 2024-09-12T03:02:51.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.622+0000 7f2ee86ec740 1 -- 172.21.15.116:0/2001734472 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ec8005500 con 0x55d1fae422f0 2024-09-12T03:02:51.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.622+0000 7f2ebf7fe640 1 -- 172.21.15.116:0/2001734472 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2ec800f3c0 con 0x55d1fae422f0 2024-09-12T03:02:51.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.622+0000 7f2ee86ec740 10 client.15153 fetch_fsmap learned FSMap version 24 2024-09-12T03:02:51.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.622+0000 7f2ee86ec740 1 -- 172.21.15.116:0/2001734472 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d1fae69810 con 0x55d1fae422f0 2024-09-12T03:02:51.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.622+0000 7f2ebf7fe640 1 -- 172.21.15.116:0/2001734472 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ec8005500 con 0x55d1fae422f0 2024-09-12T03:02:51.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.623+0000 7f2ebf7fe640 1 -- 172.21.15.116:0/2001734472 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f2ec8031070 con 0x55d1fae422f0 2024-09-12T03:02:51.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.623+0000 7f2ee86ec740 10 client.15153 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T03:02:51.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.623+0000 7f2ee86ec740 10 client.15153 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:02:51.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.623+0000 7f2ee86ec740 20 client.15153 populate_metadata read hostname 'smithi116' 2024-09-12T03:02:51.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.623+0000 7f2ee86ec740 1 --2- 172.21.15.116:0/2001734472 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d1faed02f0 0x55d1faecf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.623+0000 7f2ee86ec740 4 client.15153 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:02:51.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.623+0000 7f2ee86ec740 1 -- 172.21.15.116:0/2001734472 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "status"}) -- 0x55d1fadafb10 con 0x55d1faed02f0 2024-09-12T03:02:51.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.623+0000 7f2edffff640 1 --2- 172.21.15.116:0/2001734472 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d1faed02f0 0x55d1faecf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.623+0000 7f2ebf7fe640 10 client.15153 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:02:51.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.623+0000 7f2edffff640 1 --2- 172.21.15.116:0/2001734472 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d1faed02f0 0x55d1faecf000 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=436fa0e59d9b3e9c server_cookie=958eca6a4cdd673d in_seq=0 out_seq=0 2024-09-12T03:02:51.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.624+0000 7f2ebf7fe640 1 -- 172.21.15.116:0/2001734472 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55d1fadafb10 con 0x55d1faed02f0 2024-09-12T03:02:51.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.624+0000 7f2ebf7fe640 10 client.15153 handle_command_reply: tid=0 2024-09-12T03:02:51.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.624+0000 7f2ee86ec740 1 client.15153 shutdown 2024-09-12T03:02:51.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.624+0000 7f2ee86ec740 1 -- 172.21.15.116:0/2001734472 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1fae422f0 msgr2=0x55d1fae6b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:51.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.624+0000 7f2ee86ec740 1 --2- 172.21.15.116:0/2001734472 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1fae422f0 0x55d1fae6b820 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8012010 tx=0x7f2ec8012890 comp rx=0 tx=0).stop 2024-09-12T03:02:51.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.624+0000 7f2ee86ec740 1 -- 172.21.15.116:0/2001734472 shutdown_connections 2024-09-12T03:02:51.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.624+0000 7f2ee86ec740 1 --2- 172.21.15.116:0/2001734472 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d1faed02f0 0x55d1faecf000 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.624+0000 7f2ee86ec740 1 --2- 172.21.15.116:0/2001734472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1fae4cbb0 0x55d1fae64610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.624+0000 7f2ebf7fe640 0 client.15153 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:02:51.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.624+0000 7f2ee86ec740 1 --2- 172.21.15.116:0/2001734472 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1fae422f0 0x55d1fae6b820 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.624+0000 7f2ee86ec740 1 --2- 172.21.15.116:0/2001734472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1fae419d0 0x55d1fae6b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.624+0000 7f2ee86ec740 1 -- 172.21.15.116:0/2001734472 >> 172.21.15.116:0/2001734472 conn(0x55d1fae3df60 msgr2=0x55d1fae49660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:51.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.624+0000 7f2ee86ec740 1 -- 172.21.15.116:0/2001734472 shutdown_connections 2024-09-12T03:02:51.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.624+0000 7f2ee86ec740 1 -- 172.21.15.116:0/2001734472 wait complete. 2024-09-12T03:02:51.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.624+0000 7f2ee86ec740 20 client.15153 trim_cache size 0 max 16384 2024-09-12T03:02:51.645 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:02:51.645 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_fsid": "eb1ac602-70b1-11ef-bcea-c7b262605968", 2024-09-12T03:02:51.646 INFO:teuthology.orchestra.run.smithi116.stdout: "whoami": 0, 2024-09-12T03:02:51.646 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 24494, 2024-09-12T03:02:51.646 INFO:teuthology.orchestra.run.smithi116.stdout: "want_state": "up:active", 2024-09-12T03:02:51.646 INFO:teuthology.orchestra.run.smithi116.stdout: "state": "up:active", 2024-09-12T03:02:51.646 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs", 2024-09-12T03:02:51.646 INFO:teuthology.orchestra.run.smithi116.stdout: "rank_uptime": 112.28821195, 2024-09-12T03:02:51.646 INFO:teuthology.orchestra.run.smithi116.stdout: "mdsmap_epoch": 24, 2024-09-12T03:02:51.646 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch": 80, 2024-09-12T03:02:51.646 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch_barrier": 78, 2024-09-12T03:02:51.646 INFO:teuthology.orchestra.run.smithi116.stdout: "uptime": 122.924168834 2024-09-12T03:02:51.646 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:02:51.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.624+0000 7f2ee6313640 1 -- 172.21.15.116:0/1878145011 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2ecc0a2b80 msgr2=0x7f2ecc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:51.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.624+0000 7f2ee6313640 1 --2- 172.21.15.116:0/1878145011 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2ecc0a2b80 0x7f2ecc0a5040 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8005fd0 tx=0x7f2ec800e350 comp rx=0 tx=0).stop 2024-09-12T03:02:51.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.624+0000 7f2ee6313640 1 -- 172.21.15.116:0/1878145011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ee0106f10 msgr2=0x7f2ee019cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:51.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.624+0000 7f2ee6313640 1 --2- 172.21.15.116:0/1878145011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ee0106f10 0x7f2ee019cd00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800eaa0 tx=0x7f2ed800ef60 comp rx=0 tx=0).stop 2024-09-12T03:02:51.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.624+0000 7f2ee6313640 1 -- 172.21.15.116:0/1878145011 shutdown_connections 2024-09-12T03:02:51.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.624+0000 7f2ee6313640 1 --2- 172.21.15.116:0/1878145011 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2ecc0a2b80 0x7f2ecc0a5040 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.624+0000 7f2ee6313640 1 --2- 172.21.15.116:0/1878145011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ee0106f10 0x7f2ee019cd00 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.624+0000 7f2ee6313640 1 --2- 172.21.15.116:0/1878145011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ee0102790 0x7f2ee0198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.624+0000 7f2ee6313640 1 --2- 172.21.15.116:0/1878145011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ee0101de0 0x7f2ee0198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.624+0000 7f2ee6313640 1 -- 172.21.15.116:0/1878145011 >> 172.21.15.116:0/1878145011 conn(0x7f2ee00fd8c0 msgr2=0x7f2ee00febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:51.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.625+0000 7f2ee6313640 1 -- 172.21.15.116:0/1878145011 shutdown_connections 2024-09-12T03:02:51.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:51.625+0000 7f2ee6313640 1 -- 172.21.15.116:0/1878145011 wait complete. 2024-09-12T03:02:52.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:52 smithi116 ceph-mon[31410]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-12T03:02:52.436 INFO:teuthology.run_tasks:Running task sequential... 2024-09-12T03:02:52.448 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-12T03:02:52.448 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-12T03:02:52.449 INFO:teuthology.task.print:Enabling mgr modules 2024-09-12T03:02:52.449 INFO:teuthology.run_tasks:Running task kclient... 2024-09-12T03:02:52.508 INFO:tasks.kclient:Mounting kernel clients... 2024-09-12T03:02:52.509 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-12T03:02:52.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:02:52.509 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-09-12T03:02:52.510 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:52 smithi177 ceph-mon[35305]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-12T03:02:52.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:02:52.527 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link delete ceph-brx 2024-09-12T03:02:52.594 INFO:teuthology.orchestra.run.smithi116.stderr:Cannot find device "ceph-brx" 2024-09-12T03:02:52.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:02:52.596 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-12T03:02:52.596 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T03:02:52.596 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T03:02:52.596 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-12T03:02:52.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:52 smithi183 ceph-mon[35535]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-12T03:02:52.883 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:53.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.382+0000 7f58f03d6640 1 -- 172.21.15.116:0/1842299011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58e8102790 msgr2=0x7f58e81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:53.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.382+0000 7f58f03d6640 1 --2- 172.21.15.116:0/1842299011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58e8102790 0x7f58e81067e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009ca0 tx=0x7f58e402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:02:53.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.382+0000 7f58f03d6640 1 -- 172.21.15.116:0/1842299011 shutdown_connections 2024-09-12T03:02:53.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.382+0000 7f58f03d6640 1 --2- 172.21.15.116:0/1842299011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58e8106f10 0x7f58e810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:53.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.382+0000 7f58f03d6640 1 --2- 172.21.15.116:0/1842299011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58e8102790 0x7f58e81067e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:53.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.382+0000 7f58f03d6640 1 --2- 172.21.15.116:0/1842299011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58e8101de0 0x7f58e81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:53.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.382+0000 7f58f03d6640 1 -- 172.21.15.116:0/1842299011 >> 172.21.15.116:0/1842299011 conn(0x7f58e80fd8c0 msgr2=0x7f58e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:53.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.382+0000 7f58f03d6640 1 -- 172.21.15.116:0/1842299011 shutdown_connections 2024-09-12T03:02:53.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.382+0000 7f58f03d6640 1 -- 172.21.15.116:0/1842299011 wait complete. 2024-09-12T03:02:53.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.383+0000 7f58f03d6640 1 Processor -- start 2024-09-12T03:02:53.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.383+0000 7f58f03d6640 1 -- start start 2024-09-12T03:02:53.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.383+0000 7f58f03d6640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58e8101de0 0x7f58e80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:53.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.383+0000 7f58f03d6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58e8102790 0x7f58e8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:53.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.383+0000 7f58f03d6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58e8106f10 0x7f58e8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:53.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.383+0000 7f58f03d6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f58e8112010 con 0x7f58e8106f10 2024-09-12T03:02:53.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.383+0000 7f58f03d6640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f58e8111e90 con 0x7f58e8101de0 2024-09-12T03:02:53.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.383+0000 7f58f03d6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f58e8112190 con 0x7f58e8102790 2024-09-12T03:02:53.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.383+0000 7f58ee973640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58e8106f10 0x7f58e8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:53.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.383+0000 7f58ee973640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58e8106f10 0x7f58e8074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58248/0 (socket says 172.21.15.116:58248) 2024-09-12T03:02:53.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.383+0000 7f58ee973640 1 -- 172.21.15.116:0/2593669674 learned_addr learned my addr 172.21.15.116:0/2593669674 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:53.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.383+0000 7f58ed971640 1 --2- 172.21.15.116:0/2593669674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58e8102790 0x7f58e8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:53.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.383+0000 7f58ee172640 1 --2- 172.21.15.116:0/2593669674 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58e8101de0 0x7f58e80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:53.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.383+0000 7f58ee973640 1 -- 172.21.15.116:0/2593669674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58e8102790 msgr2=0x7f58e8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:53.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.383+0000 7f58ee973640 1 --2- 172.21.15.116:0/2593669674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58e8102790 0x7f58e8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:53.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.383+0000 7f58ee973640 1 -- 172.21.15.116:0/2593669674 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58e8101de0 msgr2=0x7f58e80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:53.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.383+0000 7f58ee973640 1 --2- 172.21.15.116:0/2593669674 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58e8101de0 0x7f58e80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:53.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.383+0000 7f58ee973640 1 -- 172.21.15.116:0/2593669674 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e81a6a70 con 0x7f58e8106f10 2024-09-12T03:02:53.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.383+0000 7f58ed971640 1 --2- 172.21.15.116:0/2593669674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58e8102790 0x7f58e8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:53.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.383+0000 7f58ee172640 1 --2- 172.21.15.116:0/2593669674 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58e8101de0 0x7f58e80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:53.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.383+0000 7f58ee973640 1 --2- 172.21.15.116:0/2593669674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58e8106f10 0x7f58e8074f70 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009fd0 tx=0x7f58e000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:53.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.384+0000 7f58db7fe640 1 -- 172.21.15.116:0/2593669674 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e0011830 con 0x7f58e8106f10 2024-09-12T03:02:53.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.384+0000 7f58db7fe640 1 -- 172.21.15.116:0/2593669674 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58e0016070 con 0x7f58e8106f10 2024-09-12T03:02:53.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.384+0000 7f58f03d6640 1 -- 172.21.15.116:0/2593669674 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e81a6d60 con 0x7f58e8106f10 2024-09-12T03:02:53.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.384+0000 7f58db7fe640 1 -- 172.21.15.116:0/2593669674 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e000f2e0 con 0x7f58e8106f10 2024-09-12T03:02:53.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.384+0000 7f58f03d6640 1 -- 172.21.15.116:0/2593669674 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e81a7250 con 0x7f58e8106f10 2024-09-12T03:02:53.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.385+0000 7f58f03d6640 1 -- 172.21.15.116:0/2593669674 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58e8102240 con 0x7f58e8106f10 2024-09-12T03:02:53.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.386+0000 7f58db7fe640 1 -- 172.21.15.116:0/2593669674 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f58e00119d0 con 0x7f58e8106f10 2024-09-12T03:02:53.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.386+0000 7f58db7fe640 1 --2- 172.21.15.116:0/2593669674 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f58d40a2b80 0x7f58d40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:53.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.387+0000 7f58ee172640 1 --2- 172.21.15.116:0/2593669674 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f58d40a2b80 0x7f58d40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:53.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.387+0000 7f58db7fe640 1 -- 172.21.15.116:0/2593669674 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f58e00ce770 con 0x7f58e8106f10 2024-09-12T03:02:53.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.387+0000 7f58ee172640 1 --2- 172.21.15.116:0/2593669674 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f58d40a2b80 0x7f58d40a5040 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f58dc006fd0 tx=0x7f58dc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:53.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.391+0000 7f58db7fe640 1 -- 172.21.15.116:0/2593669674 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f58e009b700 con 0x7f58e8106f10 2024-09-12T03:02:53.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.592+0000 7f58f03d6640 1 -- 172.21.15.116:0/2593669674 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f58e802d7c0 con 0x7f58e8106f10 2024-09-12T03:02:53.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.592+0000 7f58db7fe640 1 -- 172.21.15.116:0/2593669674 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f58e00bb0c0 con 0x7f58e8106f10 2024-09-12T03:02:53.593 INFO:teuthology.orchestra.run.smithi116.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T03:02:53.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.594+0000 7f58f03d6640 1 -- 172.21.15.116:0/2593669674 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f58d40a2b80 msgr2=0x7f58d40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:53.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.594+0000 7f58f03d6640 1 --2- 172.21.15.116:0/2593669674 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f58d40a2b80 0x7f58d40a5040 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f58dc006fd0 tx=0x7f58dc008040 comp rx=0 tx=0).stop 2024-09-12T03:02:53.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.594+0000 7f58f03d6640 1 -- 172.21.15.116:0/2593669674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58e8106f10 msgr2=0x7f58e8074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:53.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.594+0000 7f58f03d6640 1 --2- 172.21.15.116:0/2593669674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58e8106f10 0x7f58e8074f70 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009fd0 tx=0x7f58e000b8e0 comp rx=0 tx=0).stop 2024-09-12T03:02:53.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.595+0000 7f58f03d6640 1 -- 172.21.15.116:0/2593669674 shutdown_connections 2024-09-12T03:02:53.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.595+0000 7f58f03d6640 1 --2- 172.21.15.116:0/2593669674 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f58d40a2b80 0x7f58d40a5040 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:53.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.595+0000 7f58f03d6640 1 --2- 172.21.15.116:0/2593669674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58e8106f10 0x7f58e8074f70 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:53.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.595+0000 7f58f03d6640 1 --2- 172.21.15.116:0/2593669674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58e8102790 0x7f58e8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:53.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.595+0000 7f58f03d6640 1 --2- 172.21.15.116:0/2593669674 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58e8101de0 0x7f58e80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:53.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.595+0000 7f58f03d6640 1 -- 172.21.15.116:0/2593669674 >> 172.21.15.116:0/2593669674 conn(0x7f58e80fd8c0 msgr2=0x7f58e80fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:53.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.595+0000 7f58f03d6640 1 -- 172.21.15.116:0/2593669674 shutdown_connections 2024-09-12T03:02:53.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:53.595+0000 7f58f03d6640 1 -- 172.21.15.116:0/2593669674 wait complete. 2024-09-12T03:02:54.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:54 smithi116 ceph-mon[31410]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:02:54.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:54 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2593669674' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T03:02:54.363 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T03:02:54.363 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:02:54.363 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2024-09-12T03:02:54.363 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T03:02:54.363 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T03:02:54.363 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T03:02:54.363 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T03:02:54.363 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:02:54.363 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:02:54.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:02:54.364 DEBUG:teuthology.orchestra.run.smithi116:> ip addr 2024-09-12T03:02:54.380 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:02:54.381 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:02:54.381 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:02:54.381 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:02:54.381 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-09-12T03:02:54.381 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:02:54.381 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:02:54.381 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:02:54.381 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f0 2024-09-12T03:02:54.381 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:02:54.381 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:02:54.381 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f1 2024-09-12T03:02:54.381 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:02:54.381 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:02:54.382 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:02:54.382 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:02:54.382 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-12T03:02:54.382 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft 42117sec preferred_lft 42117sec 2024-09-12T03:02:54.382 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link noprefixroute 2024-09-12T03:02:54.382 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:02:54.382 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-12T03:02:54.383 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:02:54.383 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-09-12T03:02:54.383 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link add name ceph-brx type bridge 2024-09-12T03:02:54.383 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip addr flush dev ceph-brx 2024-09-12T03:02:54.383 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set ceph-brx up 2024-09-12T03:02:54.383 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-12T03:02:54.383 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-09-12T03:02:54.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:02:54.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:54 smithi177 ceph-mon[35305]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:02:54.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:54 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2593669674' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T03:02:54.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:02:54.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:02:54.592 DEBUG:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:02:54.616 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-09-12T03:02:54.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:02:54.617 DEBUG:teuthology.orchestra.run.smithi116:> route 2024-09-12T03:02:54.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:54 smithi183 ceph-mon[35535]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:02:54.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:54 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2593669674' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T03:02:54.704 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2024-09-12T03:02:54.704 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:02:54.704 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-12T03:02:54.704 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-12T03:02:54.704 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:02:54.705 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:02:54.705 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-09-12T03:02:54.705 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-12T03:02:54.705 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-12T03:02:54.705 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-12T03:02:54.705 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-09-12T03:02:54.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:02:54.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:02:54.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:02:54.957 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-09-12T03:02:54.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:02:54.973 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list-id 2024-09-12T03:02:55.029 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:02:55.029 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-09-12T03:02:55.029 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:02:55.029 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-12T03:02:55.029 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-09-12T03:02:55.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:02:55.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:02:55.147 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-12T03:02:55.147 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:02:55.147 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-09-12T03:02:55.148 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-12T03:02:55.148 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-12T03:02:55.148 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-12T03:02:55.148 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-12T03:02:55.148 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-12T03:02:55.148 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-09-12T03:02:55.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:02:55.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:02:55.270 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:02:55.270 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-09-12T03:02:55.270 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2024-09-12T03:02:55.270 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set dev brx.0 master ceph-brx 2024-09-12T03:02:55.270 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-09-12T03:02:55.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:02:55.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:02:55.346 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:02:55.346 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T03:02:55.366 INFO:teuthology.orchestra.run.smithi116.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:02:55.366 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:02:55.366 DEBUG:teuthology.orchestra.run.smithi116:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:02:55.423 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-12T03:02:55.423 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-12T03:02:55.423 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/8f4fbfe1-cd89-4b49-9931-ffd07687f37b 2024-09-12T03:02:55.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:02:55.423 DEBUG:teuthology.orchestra.run.smithi116:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/8f4fbfe1-cd89-4b49-9931-ffd07687f37b /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-12T03:02:55.514 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-12T03:02:55.514 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-12T03:02:55.514 INFO:teuthology.orchestra.run.smithi116.stdout:invalid new device string format 2024-09-12T03:02:55.644 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-12T03:02:55.644 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-12T03:02:55.644 INFO:teuthology.orchestra.run.smithi116.stdout:invalid new device string format 2024-09-12T03:02:55.644 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: resolved to: "172.21.15.116:3300,172.21.15.177:3300,172.21.15.183:3300" 2024-09-12T03:02:55.644 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.116:3300,172.21.15.177:3300,172.21.15.183:3300:/volumes/qa/sv_1/8f4fbfe1-cd89-4b49-9931-ffd07687f37b 2024-09-12T03:02:55.644 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=eb1ac602-70b1-11ef-bcea-c7b262605968" will pass to kernel 2024-09-12T03:02:55.645 INFO:teuthology.orchestra.run.smithi116.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-12T03:02:55.645 INFO:teuthology.orchestra.run.smithi116.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:02:55.645 INFO:teuthology.orchestra.run.smithi116.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:02:55.645 INFO:teuthology.orchestra.run.smithi116.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:02:55.645 INFO:teuthology.orchestra.run.smithi116.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:02:55.647 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:02:55.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:02:55.647 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:02:55.677 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:02:55.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:02:55.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:02:55.773 INFO:teuthology.orchestra.run.smithi116.stdout:client15162 2024-09-12T03:02:55.774 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-12T03:02:55.774 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:02:55.791 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:02:55.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:02:55.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:02:55.886 INFO:teuthology.orchestra.run.smithi116.stdout:client15162 2024-09-12T03:02:55.887 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:02:55.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:02:55.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:02:55.942 INFO:teuthology.orchestra.run.smithi116.stdout:client15162 2024-09-12T03:02:55.943 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968.client15162/status') 2024-09-12T03:02:55.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:02:56.000 INFO:teuthology.orchestra.run.smithi116.stdout:instance: client.15162 (3)192.168.0.1:0/829247883 2024-09-12T03:02:56.000 INFO:teuthology.orchestra.run.smithi116.stdout:blocklisted: false 2024-09-12T03:02:56.001 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2024-09-12T03:02:56.001 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2024-09-12T03:02:56.001 INFO:teuthology.orchestra.run.smithi116.stderr:69 bytes copied, 2.338e-05 s, 3.0 MB/s 2024-09-12T03:02:56.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:02:56.008 INFO:teuthology.run_tasks:Running task exec... 2024-09-12T03:02:56.017 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T03:02:56.018 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi116.front.sepia.ceph.com 2024-09-12T03:02:56.018 DEBUG:teuthology.orchestra.run.smithi116:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-12T03:02:56.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.118+0000 7f81a5d23640 1 -- 172.21.15.116:0/1448076600 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81a0101c40 msgr2=0x7f81a01020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:56.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.118+0000 7f81a5d23640 1 --2- 172.21.15.116:0/1448076600 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81a0101c40 0x7f81a01020c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8194009900 tx=0x7f819402a410 comp rx=0 tx=0).stop 2024-09-12T03:02:56.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.118+0000 7f81a5d23640 1 -- 172.21.15.116:0/1448076600 shutdown_connections 2024-09-12T03:02:56.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.118+0000 7f81a5d23640 1 --2- 172.21.15.116:0/1448076600 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81a0102600 0x7f81a0108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.118+0000 7f81a5d23640 1 --2- 172.21.15.116:0/1448076600 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81a0101c40 0x7f81a01020c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.118+0000 7f81a5d23640 1 --2- 172.21.15.116:0/1448076600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81a0100a50 0x7f81a0100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.118+0000 7f81a5d23640 1 -- 172.21.15.116:0/1448076600 >> 172.21.15.116:0/1448076600 conn(0x7f81a00fc1b0 msgr2=0x7f81a00fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:56.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.119+0000 7f81a5d23640 1 -- 172.21.15.116:0/1448076600 shutdown_connections 2024-09-12T03:02:56.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.119+0000 7f81a5d23640 1 -- 172.21.15.116:0/1448076600 wait complete. 2024-09-12T03:02:56.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.119+0000 7f81a5d23640 1 Processor -- start 2024-09-12T03:02:56.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.119+0000 7f81a5d23640 1 -- start start 2024-09-12T03:02:56.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.119+0000 7f81a5d23640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81a0100a50 0x7f81a0196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:56.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.119+0000 7f81a5d23640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81a0101c40 0x7f81a0197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:56.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.119+0000 7f81a5d23640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81a0102600 0x7f81a019e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:56.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.119+0000 7f81a5d23640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f81a010c780 con 0x7f81a0102600 2024-09-12T03:02:56.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.119+0000 7f81a5d23640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f81a010c600 con 0x7f81a0101c40 2024-09-12T03:02:56.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.119+0000 7f81a5d23640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f81a010c900 con 0x7f81a0100a50 2024-09-12T03:02:56.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.119+0000 7f819ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81a0102600 0x7f81a019e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:56.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.120+0000 7f819ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81a0102600 0x7f81a019e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58264/0 (socket says 172.21.15.116:58264) 2024-09-12T03:02:56.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.120+0000 7f819ffff640 1 -- 172.21.15.116:0/703545163 learned_addr learned my addr 172.21.15.116:0/703545163 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:56.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.120+0000 7f819f7fe640 1 --2- 172.21.15.116:0/703545163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81a0100a50 0x7f81a0196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:56.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.120+0000 7f819ffff640 1 -- 172.21.15.116:0/703545163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81a0100a50 msgr2=0x7f81a0196d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:56.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.120+0000 7f819ffff640 1 --2- 172.21.15.116:0/703545163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81a0100a50 0x7f81a0196d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.120+0000 7f819ffff640 1 -- 172.21.15.116:0/703545163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81a0101c40 msgr2=0x7f81a0197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:56.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.120+0000 7f819effd640 1 --2- 172.21.15.116:0/703545163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81a0101c40 0x7f81a0197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:56.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.120+0000 7f819ffff640 1 --2- 172.21.15.116:0/703545163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81a0101c40 0x7f81a0197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.120+0000 7f819ffff640 1 -- 172.21.15.116:0/703545163 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a019e920 con 0x7f81a0102600 2024-09-12T03:02:56.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.120+0000 7f819effd640 1 --2- 172.21.15.116:0/703545163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81a0101c40 0x7f81a0197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:56.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.120+0000 7f819ffff640 1 --2- 172.21.15.116:0/703545163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81a0102600 0x7f81a019e280 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f819800b970 tx=0x7f819800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:56.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.120+0000 7f819cff9640 1 -- 172.21.15.116:0/703545163 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198019070 con 0x7f81a0102600 2024-09-12T03:02:56.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.120+0000 7f819cff9640 1 -- 172.21.15.116:0/703545163 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f819800cb40 con 0x7f81a0102600 2024-09-12T03:02:56.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.120+0000 7f81a5d23640 1 -- 172.21.15.116:0/703545163 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a019ec10 con 0x7f81a0102600 2024-09-12T03:02:56.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.120+0000 7f819cff9640 1 -- 172.21.15.116:0/703545163 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198007650 con 0x7f81a0102600 2024-09-12T03:02:56.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.120+0000 7f81a5d23640 1 -- 172.21.15.116:0/703545163 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a019f160 con 0x7f81a0102600 2024-09-12T03:02:56.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.121+0000 7f81a5d23640 1 -- 172.21.15.116:0/703545163 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81a0107d90 con 0x7f81a0102600 2024-09-12T03:02:56.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.122+0000 7f819cff9640 1 -- 172.21.15.116:0/703545163 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f819801c070 con 0x7f81a0102600 2024-09-12T03:02:56.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.123+0000 7f819cff9640 1 --2- 172.21.15.116:0/703545163 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f81880a2cb0 0x7f81880a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:56.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.123+0000 7f819cff9640 1 -- 172.21.15.116:0/703545163 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f81980ce6d0 con 0x7f81a0102600 2024-09-12T03:02:56.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.123+0000 7f819f7fe640 1 --2- 172.21.15.116:0/703545163 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f81880a2cb0 0x7f81880a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:56.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.123+0000 7f819f7fe640 1 --2- 172.21.15.116:0/703545163 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f81880a2cb0 0x7f81880a5170 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f81a01981e0 tx=0x7f8190005c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:56.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.127+0000 7f819cff9640 1 -- 172.21.15.116:0/703545163 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f819809b660 con 0x7f81a0102600 2024-09-12T03:02:56.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:56 smithi116 ceph-mon[31410]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:02:56.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:56.332+0000 7f81a5d23640 1 -- 172.21.15.116:0/703545163 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f81a0062e90 con 0x7f81a0102600 2024-09-12T03:02:56.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:56 smithi177 ceph-mon[35305]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:02:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:56 smithi183 ceph-mon[35535]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:02:57.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.228+0000 7f819cff9640 1 -- 172.21.15.116:0/703545163 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f819809af60 con 0x7f81a0102600 2024-09-12T03:02:57.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.231+0000 7f81a5d23640 1 -- 172.21.15.116:0/703545163 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f81880a2cb0 msgr2=0x7f81880a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:57.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.231+0000 7f81a5d23640 1 --2- 172.21.15.116:0/703545163 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f81880a2cb0 0x7f81880a5170 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f81a01981e0 tx=0x7f8190005c80 comp rx=0 tx=0).stop 2024-09-12T03:02:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.231+0000 7f81a5d23640 1 -- 172.21.15.116:0/703545163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81a0102600 msgr2=0x7f81a019e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.231+0000 7f81a5d23640 1 --2- 172.21.15.116:0/703545163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81a0102600 0x7f81a019e280 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f819800b970 tx=0x7f819800be30 comp rx=0 tx=0).stop 2024-09-12T03:02:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.231+0000 7f81a5d23640 1 -- 172.21.15.116:0/703545163 shutdown_connections 2024-09-12T03:02:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.231+0000 7f81a5d23640 1 --2- 172.21.15.116:0/703545163 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f81880a2cb0 0x7f81880a5170 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.231+0000 7f81a5d23640 1 --2- 172.21.15.116:0/703545163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81a0102600 0x7f81a019e280 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.231+0000 7f81a5d23640 1 --2- 172.21.15.116:0/703545163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81a0101c40 0x7f81a0197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.231+0000 7f81a5d23640 1 --2- 172.21.15.116:0/703545163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81a0100a50 0x7f81a0196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.231+0000 7f81a5d23640 1 -- 172.21.15.116:0/703545163 >> 172.21.15.116:0/703545163 conn(0x7f81a00fc1b0 msgr2=0x7f81a0104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.231+0000 7f81a5d23640 1 -- 172.21.15.116:0/703545163 shutdown_connections 2024-09-12T03:02:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.231+0000 7f81a5d23640 1 -- 172.21.15.116:0/703545163 wait complete. 2024-09-12T03:02:57.246 DEBUG:teuthology.orchestra.run.smithi116:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-12T03:02:57.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.354+0000 7ff10d15b640 1 -- 172.21.15.116:0/3159480625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff108101c30 msgr2=0x7ff1081020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:57.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.354+0000 7ff10d15b640 1 --2- 172.21.15.116:0/3159480625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff108101c30 0x7ff1081020b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc009900 tx=0x7ff0fc02a410 comp rx=0 tx=0).stop 2024-09-12T03:02:57.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.355+0000 7ff10d15b640 1 -- 172.21.15.116:0/3159480625 shutdown_connections 2024-09-12T03:02:57.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.355+0000 7ff10d15b640 1 --2- 172.21.15.116:0/3159480625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1081025f0 0x7ff108108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:57.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.355+0000 7ff10d15b640 1 --2- 172.21.15.116:0/3159480625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff108101c30 0x7ff1081020b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:57.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.355+0000 7ff10d15b640 1 --2- 172.21.15.116:0/3159480625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff108100a40 0x7ff108100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:57.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.355+0000 7ff10d15b640 1 -- 172.21.15.116:0/3159480625 >> 172.21.15.116:0/3159480625 conn(0x7ff1080fc1e0 msgr2=0x7ff1080fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:57.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.355+0000 7ff10d15b640 1 -- 172.21.15.116:0/3159480625 shutdown_connections 2024-09-12T03:02:57.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.355+0000 7ff10d15b640 1 -- 172.21.15.116:0/3159480625 wait complete. 2024-09-12T03:02:57.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.355+0000 7ff10d15b640 1 Processor -- start 2024-09-12T03:02:57.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.355+0000 7ff10d15b640 1 -- start start 2024-09-12T03:02:57.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.356+0000 7ff10d15b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff108100a40 0x7ff108197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:57.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.356+0000 7ff10d15b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff108101c30 0x7ff108198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:57.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.356+0000 7ff10d15b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1081025f0 0x7ff10819f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:57.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.356+0000 7ff10d15b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff10810c790 con 0x7ff1081025f0 2024-09-12T03:02:57.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.356+0000 7ff10d15b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff10810c610 con 0x7ff108101c30 2024-09-12T03:02:57.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.356+0000 7ff10d15b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff10810c910 con 0x7ff108100a40 2024-09-12T03:02:57.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.356+0000 7ff10759e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1081025f0 0x7ff10819f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:57.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.356+0000 7ff10759e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1081025f0 0x7ff10819f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58284/0 (socket says 172.21.15.116:58284) 2024-09-12T03:02:57.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.356+0000 7ff10759e640 1 -- 172.21.15.116:0/666646386 learned_addr learned my addr 172.21.15.116:0/666646386 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:57.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.356+0000 7ff10659c640 1 --2- 172.21.15.116:0/666646386 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff108101c30 0x7ff108198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:57.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.356+0000 7ff106d9d640 1 --2- 172.21.15.116:0/666646386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff108100a40 0x7ff108197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:57.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.356+0000 7ff10759e640 1 -- 172.21.15.116:0/666646386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff108100a40 msgr2=0x7ff108197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:57.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.356+0000 7ff10759e640 1 --2- 172.21.15.116:0/666646386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff108100a40 0x7ff108197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:57.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.356+0000 7ff10759e640 1 -- 172.21.15.116:0/666646386 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff108101c30 msgr2=0x7ff108198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:57.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.356+0000 7ff10759e640 1 --2- 172.21.15.116:0/666646386 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff108101c30 0x7ff108198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:57.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.356+0000 7ff10759e640 1 -- 172.21.15.116:0/666646386 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff10819fa40 con 0x7ff1081025f0 2024-09-12T03:02:57.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.356+0000 7ff10659c640 1 --2- 172.21.15.116:0/666646386 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff108101c30 0x7ff108198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:57.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.356+0000 7ff10759e640 1 --2- 172.21.15.116:0/666646386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1081025f0 0x7ff10819f3d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff10000cce0 tx=0x7ff10000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:57.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.357+0000 7ff0effff640 1 -- 172.21.15.116:0/666646386 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10000f450 con 0x7ff1081025f0 2024-09-12T03:02:57.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.357+0000 7ff0effff640 1 -- 172.21.15.116:0/666646386 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff10000fad0 con 0x7ff1081025f0 2024-09-12T03:02:57.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.357+0000 7ff10d15b640 1 -- 172.21.15.116:0/666646386 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff10819fd30 con 0x7ff1081025f0 2024-09-12T03:02:57.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.357+0000 7ff0effff640 1 -- 172.21.15.116:0/666646386 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff100018b60 con 0x7ff1081025f0 2024-09-12T03:02:57.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.357+0000 7ff10d15b640 1 -- 172.21.15.116:0/666646386 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1081a0220 con 0x7ff1081025f0 2024-09-12T03:02:57.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.358+0000 7ff10d15b640 1 -- 172.21.15.116:0/666646386 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1081a04b0 con 0x7ff1081025f0 2024-09-12T03:02:57.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.360+0000 7ff0effff640 1 -- 172.21.15.116:0/666646386 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff10000fc80 con 0x7ff1081025f0 2024-09-12T03:02:57.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.360+0000 7ff0effff640 1 --2- 172.21.15.116:0/666646386 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff0f00a2c30 0x7ff0f00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:57.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.361+0000 7ff0effff640 1 -- 172.21.15.116:0/666646386 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff1000cf930 con 0x7ff1081025f0 2024-09-12T03:02:57.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.361+0000 7ff106d9d640 1 --2- 172.21.15.116:0/666646386 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff0f00a2c30 0x7ff0f00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:57.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.361+0000 7ff106d9d640 1 --2- 172.21.15.116:0/666646386 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff0f00a2c30 0x7ff0f00a50f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff108101a90 tx=0x7ff0f8009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:57.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.363+0000 7ff0effff640 1 -- 172.21.15.116:0/666646386 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff100014030 con 0x7ff1081025f0 2024-09-12T03:02:57.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:57 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/703545163' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-12T03:02:57.515 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:57 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/703545163' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-12T03:02:57.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.559+0000 7ff10d15b640 1 -- 172.21.15.116:0/666646386 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7ff1080fe380 con 0x7ff0f00a2c30 2024-09-12T03:02:57.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.577+0000 7ff0effff640 1 -- 172.21.15.116:0/666646386 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7ff1080fe380 con 0x7ff0f00a2c30 2024-09-12T03:02:57.577 INFO:teuthology.orchestra.run.smithi116.stdout:{} 2024-09-12T03:02:57.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.579+0000 7ff10d15b640 1 -- 172.21.15.116:0/666646386 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff0f00a2c30 msgr2=0x7ff0f00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:57.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.579+0000 7ff10d15b640 1 --2- 172.21.15.116:0/666646386 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff0f00a2c30 0x7ff0f00a50f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff108101a90 tx=0x7ff0f8009110 comp rx=0 tx=0).stop 2024-09-12T03:02:57.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.579+0000 7ff10d15b640 1 -- 172.21.15.116:0/666646386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1081025f0 msgr2=0x7ff10819f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:57.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.579+0000 7ff10d15b640 1 --2- 172.21.15.116:0/666646386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1081025f0 0x7ff10819f3d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff10000cce0 tx=0x7ff10000a620 comp rx=0 tx=0).stop 2024-09-12T03:02:57.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.579+0000 7ff10d15b640 1 -- 172.21.15.116:0/666646386 shutdown_connections 2024-09-12T03:02:57.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.579+0000 7ff10d15b640 1 --2- 172.21.15.116:0/666646386 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff0f00a2c30 0x7ff0f00a50f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:57.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.579+0000 7ff10d15b640 1 --2- 172.21.15.116:0/666646386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1081025f0 0x7ff10819f3d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:57.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.579+0000 7ff10d15b640 1 --2- 172.21.15.116:0/666646386 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff108101c30 0x7ff108198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:57.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.579+0000 7ff10d15b640 1 --2- 172.21.15.116:0/666646386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff108100a40 0x7ff108197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:57.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.579+0000 7ff10d15b640 1 -- 172.21.15.116:0/666646386 >> 172.21.15.116:0/666646386 conn(0x7ff1080fc1e0 msgr2=0x7ff1080fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:57.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.579+0000 7ff10d15b640 1 -- 172.21.15.116:0/666646386 shutdown_connections 2024-09-12T03:02:57.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.579+0000 7ff10d15b640 1 -- 172.21.15.116:0/666646386 wait complete. 2024-09-12T03:02:57.590 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-12T03:02:57.601 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-12T03:02:57.611 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:02:57.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:57 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/703545163' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-12T03:02:57.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.713+0000 7ff9c992a640 1 -- 172.21.15.116:0/420778428 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9c4102650 msgr2=0x7ff9c4104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:57.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.713+0000 7ff9c992a640 1 --2- 172.21.15.116:0/420778428 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9c4102650 0x7ff9c4104a40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4009900 tx=0x7ff9b402a410 comp rx=0 tx=0).stop 2024-09-12T03:02:57.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.713+0000 7ff9c992a640 1 -- 172.21.15.116:0/420778428 shutdown_connections 2024-09-12T03:02:57.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.713+0000 7ff9c992a640 1 --2- 172.21.15.116:0/420778428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9c41078b0 0x7ff9c4109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:57.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.713+0000 7ff9c992a640 1 --2- 172.21.15.116:0/420778428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9c4104f80 0x7ff9c4107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:57.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.713+0000 7ff9c992a640 1 --2- 172.21.15.116:0/420778428 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9c4102650 0x7ff9c4104a40 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:57.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.713+0000 7ff9c992a640 1 -- 172.21.15.116:0/420778428 >> 172.21.15.116:0/420778428 conn(0x7ff9c40fc190 msgr2=0x7ff9c40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:57.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.713+0000 7ff9c992a640 1 -- 172.21.15.116:0/420778428 shutdown_connections 2024-09-12T03:02:57.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.713+0000 7ff9c992a640 1 -- 172.21.15.116:0/420778428 wait complete. 2024-09-12T03:02:57.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.714+0000 7ff9c992a640 1 Processor -- start 2024-09-12T03:02:57.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.714+0000 7ff9c992a640 1 -- start start 2024-09-12T03:02:57.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.714+0000 7ff9c992a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9c4102650 0x7ff9c4197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:57.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.714+0000 7ff9c992a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9c4104f80 0x7ff9c4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:57.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.714+0000 7ff9c992a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9c41078b0 0x7ff9c419f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:57.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.714+0000 7ff9c992a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c410d7f0 con 0x7ff9c41078b0 2024-09-12T03:02:57.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.714+0000 7ff9c992a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c410d670 con 0x7ff9c4104f80 2024-09-12T03:02:57.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.714+0000 7ff9c992a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c410d970 con 0x7ff9c4102650 2024-09-12T03:02:57.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.715+0000 7ff9c37fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9c41078b0 0x7ff9c419f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:57.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.715+0000 7ff9c37fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9c41078b0 0x7ff9c419f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58312/0 (socket says 172.21.15.116:58312) 2024-09-12T03:02:57.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.715+0000 7ff9c37fe640 1 -- 172.21.15.116:0/1945736740 learned_addr learned my addr 172.21.15.116:0/1945736740 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:57.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.715+0000 7ff9c27fc640 1 --2- 172.21.15.116:0/1945736740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9c4104f80 0x7ff9c4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:57.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.715+0000 7ff9c2ffd640 1 --2- 172.21.15.116:0/1945736740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9c4102650 0x7ff9c4197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:57.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.715+0000 7ff9c37fe640 1 -- 172.21.15.116:0/1945736740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9c4102650 msgr2=0x7ff9c4197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:57.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.715+0000 7ff9c37fe640 1 --2- 172.21.15.116:0/1945736740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9c4102650 0x7ff9c4197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:57.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.715+0000 7ff9c37fe640 1 -- 172.21.15.116:0/1945736740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9c4104f80 msgr2=0x7ff9c4198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:57.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.715+0000 7ff9c37fe640 1 --2- 172.21.15.116:0/1945736740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9c4104f80 0x7ff9c4198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:57.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.715+0000 7ff9c37fe640 1 -- 172.21.15.116:0/1945736740 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c419faf0 con 0x7ff9c41078b0 2024-09-12T03:02:57.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.715+0000 7ff9c37fe640 1 --2- 172.21.15.116:0/1945736740 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9c41078b0 0x7ff9c419f480 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00eb40 tx=0x7ff9bc00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:57.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.715+0000 7ff9c8928640 1 -- 172.21.15.116:0/1945736740 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc00ce40 con 0x7ff9c41078b0 2024-09-12T03:02:57.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.715+0000 7ff9c8928640 1 -- 172.21.15.116:0/1945736740 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9bc005020 con 0x7ff9c41078b0 2024-09-12T03:02:57.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.715+0000 7ff9c992a640 1 -- 172.21.15.116:0/1945736740 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c419fde0 con 0x7ff9c41078b0 2024-09-12T03:02:57.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.715+0000 7ff9c8928640 1 -- 172.21.15.116:0/1945736740 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc010690 con 0x7ff9c41078b0 2024-09-12T03:02:57.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.715+0000 7ff9c992a640 1 -- 172.21.15.116:0/1945736740 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c41a02f0 con 0x7ff9c41078b0 2024-09-12T03:02:57.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.716+0000 7ff9c992a640 1 -- 172.21.15.116:0/1945736740 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9c40fdc80 con 0x7ff9c41078b0 2024-09-12T03:02:57.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.718+0000 7ff9c8928640 1 -- 172.21.15.116:0/1945736740 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff9bc0051d0 con 0x7ff9c41078b0 2024-09-12T03:02:57.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.718+0000 7ff9c8928640 1 --2- 172.21.15.116:0/1945736740 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff9ac0a2c10 0x7ff9ac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:57.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.719+0000 7ff9c8928640 1 -- 172.21.15.116:0/1945736740 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff9bc0cf6a0 con 0x7ff9c41078b0 2024-09-12T03:02:57.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.719+0000 7ff9c2ffd640 1 --2- 172.21.15.116:0/1945736740 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff9ac0a2c10 0x7ff9ac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:57.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.719+0000 7ff9c2ffd640 1 --2- 172.21.15.116:0/1945736740 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff9ac0a2c10 0x7ff9ac0a50d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4000c00 tx=0x7ff9b40315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:57.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.723+0000 7ff9c8928640 1 -- 172.21.15.116:0/1945736740 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff9bc09c630 con 0x7ff9c41078b0 2024-09-12T03:02:57.802 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 03:02:57 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T03:02:57.563+0000 7fc2f0aca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:02:57.803 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 03:02:57 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T03:02:57.563+0000 7fc2f0aca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:02:57.803 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 03:02:57 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T03:02:57.563+0000 7fc2f0aca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:02:57.803 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 03:02:57 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T03:02:57.563+0000 7fc2f0aca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:02:57.803 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 03:02:57 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T03:02:57.563+0000 7fc2f0aca640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:02:57.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.898+0000 7ff9c992a640 1 -- 172.21.15.116:0/1945736740 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9c4062e90 con 0x7ff9c41078b0 2024-09-12T03:02:57.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.900+0000 7ff9c8928640 1 -- 172.21.15.116:0/1945736740 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22710 (secure 0 0 0) 0x7ff9bc09bf30 con 0x7ff9c41078b0 2024-09-12T03:02:57.900 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:02:57.900 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":80,"fsid":"eb1ac602-70b1-11ef-bcea-c7b262605968","created":"2024-09-12T02:53:07.105686+0000","modified":"2024-09-12T03:02:04.272898+0000","last_up_change":"2024-09-12T03:00:07.202639+0000","last_in_change":"2024-09-12T02:59:43.446336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:56:17.347152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:00:55.191040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:00:57.171142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8cf5a18-dc6f-4bb6-bd74-78adda310118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6803","nonce":862366853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6805","nonce":862366853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6809","nonce":862366853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6807","nonce":862366853}]},"public_addr":"172.21.15.116:6803/862366853","cluster_addr":"172.21.15.116:6805/862366853","heartbeat_back_addr":"172.21.15.116:6809/862366853","heartbeat_front_addr":"172.21.15.116:6807/862366853","state":["exists","up"]},{"osd":1,"uuid":"0affd3cb-9a51-42b0-a2cc-f30784a75cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6801","nonce":2245800547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6803","nonce":2245800547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6807","nonce":2245800547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6805","nonce":2245800547}]},"public_addr":"172.21.15.177:6801/2245800547","cluster_addr":"172.21.15.177:6803/2245800547","heartbeat_back_addr":"172.21.15.177:6807/2245800547","heartbeat_front_addr":"172.21.15.177:6805/2245800547","state":["exists","up"]},{"osd":2,"uuid":"1ce45ea5-84df-4a35-924f-78f92bd4f2cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6801","nonce":2645677759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6803","nonce":2645677759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6807","nonce":2645677759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6805","nonce":2645677759}]},"public_addr":"172.21.15.183:6801/2645677759","cluster_addr":"172.21.15.183:6803/2645677759","heartbeat_back_addr":"172.21.15.183:6807/2645677759","heartbeat_front_addr":"172.21.15.183:6805/2645677759","state":["exists","up"]},{"osd":3,"uuid":"ae5e8178-c998-4648-bc05-4bce822f0cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6811","nonce":686622148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6813","nonce":686622148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6817","nonce":686622148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6815","nonce":686622148}]},"public_addr":"172.21.15.116:6811/686622148","cluster_addr":"172.21.15.116:6813/686622148","heartbeat_back_addr":"172.21.15.116:6817/686622148","heartbeat_front_addr":"172.21.15.116:6815/686622148","state":["exists","up"]},{"osd":4,"uuid":"c81d2e73-0bc2-440d-b5a2-b7428af3ea4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6809","nonce":2382938021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6811","nonce":2382938021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6815","nonce":2382938021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6813","nonce":2382938021}]},"public_addr":"172.21.15.177:6809/2382938021","cluster_addr":"172.21.15.177:6811/2382938021","heartbeat_back_addr":"172.21.15.177:6815/2382938021","heartbeat_front_addr":"172.21.15.177:6813/2382938021","state":["exists","up"]},{"osd":5,"uuid":"c3f168be-9e27-4cf5-b73e-3f47d8990ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6809","nonce":85363432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6811","nonce":85363432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6815","nonce":85363432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6813","nonce":85363432}]},"public_addr":"172.21.15.183:6809/85363432","cluster_addr":"172.21.15.183:6811/85363432","heartbeat_back_addr":"172.21.15.183:6815/85363432","heartbeat_front_addr":"172.21.15.183:6813/85363432","state":["exists","up"]},{"osd":6,"uuid":"beb0a7e9-4909-47e2-9ee7-6951d4745e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6819","nonce":1121291186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6821","nonce":1121291186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6825","nonce":1121291186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6823","nonce":1121291186}]},"public_addr":"172.21.15.116:6819/1121291186","cluster_addr":"172.21.15.116:6821/1121291186","heartbeat_back_addr":"172.21.15.116:6825/1121291186","heartbeat_front_addr":"172.21.15.116:6823/1121291186","state":["exists","up"]},{"osd":7,"uuid":"761f4680-f049-473c-bf44-b19acf8de24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6817","nonce":2177665961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6819","nonce":2177665961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6823","nonce":2177665961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6821","nonce":2177665961}]},"public_addr":"172.21.15.177:6817/2177665961","cluster_addr":"172.21.15.177:6819/2177665961","heartbeat_back_addr":"172.21.15.177:6823/2177665961","heartbeat_front_addr":"172.21.15.177:6821/2177665961","state":["exists","up"]},{"osd":8,"uuid":"2385ea6c-fd5a-4815-9596-77f948629542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6817","nonce":1305725258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6819","nonce":1305725258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6823","nonce":1305725258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6821","nonce":1305725258}]},"public_addr":"172.21.15.183:6817/1305725258","cluster_addr":"172.21.15.183:6819/1305725258","heartbeat_back_addr":"172.21.15.183:6823/1305725258","heartbeat_front_addr":"172.21.15.183:6821/1305725258","state":["exists","up"]},{"osd":9,"uuid":"fd0c0dc4-4990-4f12-aa14-f855504cd995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6827","nonce":3831673668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6829","nonce":3831673668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6833","nonce":3831673668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6831","nonce":3831673668}]},"public_addr":"172.21.15.116:6827/3831673668","cluster_addr":"172.21.15.116:6829/3831673668","heartbeat_back_addr":"172.21.15.116:6833/3831673668","heartbeat_front_addr":"172.21.15.116:6831/3831673668","state":["exists","up"]},{"osd":10,"uuid":"864d1a97-4810-49c8-ad4a-a0c10253833c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6825","nonce":963663634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6827","nonce":963663634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6831","nonce":963663634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6829","nonce":963663634}]},"public_addr":"172.21.15.177:6825/963663634","cluster_addr":"172.21.15.177:6827/963663634","heartbeat_back_addr":"172.21.15.177:6831/963663634","heartbeat_front_addr":"172.21.15.177:6829/963663634","state":["exists","up"]},{"osd":11,"uuid":"d49c0841-4620-4f2f-a6b5-f2a9aa2e3c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6825","nonce":1818319939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6827","nonce":1818319939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6831","nonce":1818319939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6829","nonce":1818319939}]},"public_addr":"172.21.15.183:6825/1818319939","cluster_addr":"172.21.15.183:6827/1818319939","heartbeat_back_addr":"172.21.15.183:6831/1818319939","heartbeat_front_addr":"172.21.15.183:6829/1818319939","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:24.736944+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:50.310980+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:56:14.989533+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:56:41.187047+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:06.981664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:32.872771+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:58.850089+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:23.043121+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:47.839958+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:14.087708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:40.010548+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:05.364820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.116:6801/2966274423":"2024-09-13T02:54:03.026862+0000","172.21.15.116:6800/2966274423":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/686139455":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/1552047390":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/2180363411":"2024-09-13T02:53:39.433769+0000","172.21.15.116:6801/1791219601":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/3907400460":"2024-09-13T02:54:03.026862+0000","172.21.15.116:6800/1791219601":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/1802566286":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/212337802":"2024-09-13T02:53:39.433769+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:02:57.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.902+0000 7ff9c992a640 1 -- 172.21.15.116:0/1945736740 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff9ac0a2c10 msgr2=0x7ff9ac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:57.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.902+0000 7ff9c992a640 1 --2- 172.21.15.116:0/1945736740 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff9ac0a2c10 0x7ff9ac0a50d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4000c00 tx=0x7ff9b40315b0 comp rx=0 tx=0).stop 2024-09-12T03:02:57.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.902+0000 7ff9c992a640 1 -- 172.21.15.116:0/1945736740 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9c41078b0 msgr2=0x7ff9c419f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:57.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.902+0000 7ff9c992a640 1 --2- 172.21.15.116:0/1945736740 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9c41078b0 0x7ff9c419f480 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00eb40 tx=0x7ff9bc00ef40 comp rx=0 tx=0).stop 2024-09-12T03:02:57.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.902+0000 7ff9c992a640 1 -- 172.21.15.116:0/1945736740 shutdown_connections 2024-09-12T03:02:57.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.902+0000 7ff9c992a640 1 --2- 172.21.15.116:0/1945736740 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff9ac0a2c10 0x7ff9ac0a50d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:57.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.902+0000 7ff9c992a640 1 --2- 172.21.15.116:0/1945736740 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9c41078b0 0x7ff9c419f480 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:57.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.902+0000 7ff9c992a640 1 --2- 172.21.15.116:0/1945736740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9c4104f80 0x7ff9c4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:57.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.902+0000 7ff9c992a640 1 --2- 172.21.15.116:0/1945736740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9c4102650 0x7ff9c4197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:57.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.902+0000 7ff9c992a640 1 -- 172.21.15.116:0/1945736740 >> 172.21.15.116:0/1945736740 conn(0x7ff9c40fc190 msgr2=0x7ff9c4103010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:57.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.902+0000 7ff9c992a640 1 -- 172.21.15.116:0/1945736740 shutdown_connections 2024-09-12T03:02:57.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:57.902+0000 7ff9c992a640 1 -- 172.21.15.116:0/1945736740 wait complete. 2024-09-12T03:02:57.914 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T02:56:17.347152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-12T03:00:55.191040+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-12T03:00:57.171142+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T03:02:57.914 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T03:02:58.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.020+0000 7ff40e80e640 1 -- 172.21.15.116:0/47253888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4081077e0 msgr2=0x7ff408109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:58.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.020+0000 7ff40e80e640 1 --2- 172.21.15.116:0/47253888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4081077e0 0x7ff408109c50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff400009ca0 tx=0x7ff40002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.020+0000 7ff40e80e640 1 -- 172.21.15.116:0/47253888 shutdown_connections 2024-09-12T03:02:58.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.020+0000 7ff40e80e640 1 --2- 172.21.15.116:0/47253888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4081077e0 0x7ff408109c50 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.020+0000 7ff40e80e640 1 --2- 172.21.15.116:0/47253888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff408104eb0 0x7ff4081072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.020+0000 7ff40e80e640 1 --2- 172.21.15.116:0/47253888 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff408102580 0x7ff408104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.020+0000 7ff40e80e640 1 -- 172.21.15.116:0/47253888 >> 172.21.15.116:0/47253888 conn(0x7ff4080fc140 msgr2=0x7ff4080fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:58.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.020+0000 7ff40e80e640 1 -- 172.21.15.116:0/47253888 shutdown_connections 2024-09-12T03:02:58.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.020+0000 7ff40e80e640 1 -- 172.21.15.116:0/47253888 wait complete. 2024-09-12T03:02:58.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.020+0000 7ff40e80e640 1 Processor -- start 2024-09-12T03:02:58.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.021+0000 7ff40e80e640 1 -- start start 2024-09-12T03:02:58.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.021+0000 7ff40e80e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff408102580 0x7ff408197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:58.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.021+0000 7ff40e80e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff408104eb0 0x7ff4081983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:58.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.021+0000 7ff40e80e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff4081077e0 0x7ff40819f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:58.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.021+0000 7ff40e80e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff40810d7c0 con 0x7ff408104eb0 2024-09-12T03:02:58.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.021+0000 7ff40e80e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff40810d640 con 0x7ff4081077e0 2024-09-12T03:02:58.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.021+0000 7ff40e80e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff40810d940 con 0x7ff408102580 2024-09-12T03:02:58.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.021+0000 7ff3fffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff408104eb0 0x7ff4081983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:58.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.021+0000 7ff3fffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff408104eb0 0x7ff4081983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58340/0 (socket says 172.21.15.116:58340) 2024-09-12T03:02:58.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.021+0000 7ff3fffff640 1 -- 172.21.15.116:0/1098658426 learned_addr learned my addr 172.21.15.116:0/1098658426 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:58.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.021+0000 7ff40cdab640 1 --2- 172.21.15.116:0/1098658426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff4081077e0 0x7ff40819f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:58.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.021+0000 7ff407fff640 1 --2- 172.21.15.116:0/1098658426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff408102580 0x7ff408197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:58.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.021+0000 7ff3fffff640 1 -- 172.21.15.116:0/1098658426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff408102580 msgr2=0x7ff408197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:58.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.021+0000 7ff3fffff640 1 --2- 172.21.15.116:0/1098658426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff408102580 0x7ff408197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.021+0000 7ff3fffff640 1 -- 172.21.15.116:0/1098658426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff4081077e0 msgr2=0x7ff40819f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:58.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.021+0000 7ff3fffff640 1 --2- 172.21.15.116:0/1098658426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff4081077e0 0x7ff40819f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.021+0000 7ff3fffff640 1 -- 172.21.15.116:0/1098658426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff40819fac0 con 0x7ff408104eb0 2024-09-12T03:02:58.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.021+0000 7ff3fffff640 1 --2- 172.21.15.116:0/1098658426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff408104eb0 0x7ff4081983e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000d740 tx=0x7ff3f000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:58.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.021+0000 7ff405ffb640 1 -- 172.21.15.116:0/1098658426 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0017070 con 0x7ff408104eb0 2024-09-12T03:02:58.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.022+0000 7ff405ffb640 1 -- 172.21.15.116:0/1098658426 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3f000de50 con 0x7ff408104eb0 2024-09-12T03:02:58.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.022+0000 7ff40e80e640 1 -- 172.21.15.116:0/1098658426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff40819fdb0 con 0x7ff408104eb0 2024-09-12T03:02:58.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.022+0000 7ff405ffb640 1 -- 172.21.15.116:0/1098658426 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0004e00 con 0x7ff408104eb0 2024-09-12T03:02:58.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.022+0000 7ff40e80e640 1 -- 172.21.15.116:0/1098658426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4081a0220 con 0x7ff408104eb0 2024-09-12T03:02:58.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.022+0000 7ff40e80e640 1 -- 172.21.15.116:0/1098658426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4080fdc30 con 0x7ff408104eb0 2024-09-12T03:02:58.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.024+0000 7ff405ffb640 1 -- 172.21.15.116:0/1098658426 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff3f0004fa0 con 0x7ff408104eb0 2024-09-12T03:02:58.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.025+0000 7ff405ffb640 1 --2- 172.21.15.116:0/1098658426 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff3f40a2b90 0x7ff3f40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:58.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.025+0000 7ff407fff640 1 --2- 172.21.15.116:0/1098658426 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff3f40a2b90 0x7ff3f40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:58.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.025+0000 7ff405ffb640 1 -- 172.21.15.116:0/1098658426 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff3f00cec70 con 0x7ff408104eb0 2024-09-12T03:02:58.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.025+0000 7ff407fff640 1 --2- 172.21.15.116:0/1098658426 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff3f40a2b90 0x7ff3f40a5050 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8009bd0 tx=0x7ff3f8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:58.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.028+0000 7ff405ffb640 1 -- 172.21.15.116:0/1098658426 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff3f009b5b0 con 0x7ff408104eb0 2024-09-12T03:02:58.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.201+0000 7ff40e80e640 1 -- 172.21.15.116:0/1098658426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff40802da10 con 0x7ff408104eb0 2024-09-12T03:02:58.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.201+0000 7ff405ffb640 1 -- 172.21.15.116:0/1098658426 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) ==== 93+0+10 (secure 0 0 0) 0x7ff3f009f8b0 con 0x7ff408104eb0 2024-09-12T03:02:58.202 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2024-09-12T03:02:58.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.203+0000 7ff40e80e640 1 -- 172.21.15.116:0/1098658426 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff3f40a2b90 msgr2=0x7ff3f40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:58.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.203+0000 7ff40e80e640 1 --2- 172.21.15.116:0/1098658426 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff3f40a2b90 0x7ff3f40a5050 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8009bd0 tx=0x7ff3f8005f90 comp rx=0 tx=0).stop 2024-09-12T03:02:58.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.203+0000 7ff40e80e640 1 -- 172.21.15.116:0/1098658426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff408104eb0 msgr2=0x7ff4081983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:58.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.203+0000 7ff40e80e640 1 --2- 172.21.15.116:0/1098658426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff408104eb0 0x7ff4081983e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000d740 tx=0x7ff3f000dc00 comp rx=0 tx=0).stop 2024-09-12T03:02:58.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.203+0000 7ff40e80e640 1 -- 172.21.15.116:0/1098658426 shutdown_connections 2024-09-12T03:02:58.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.203+0000 7ff40e80e640 1 --2- 172.21.15.116:0/1098658426 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff3f40a2b90 0x7ff3f40a5050 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.203+0000 7ff40e80e640 1 --2- 172.21.15.116:0/1098658426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff4081077e0 0x7ff40819f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.203+0000 7ff40e80e640 1 --2- 172.21.15.116:0/1098658426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff408104eb0 0x7ff4081983e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.203+0000 7ff40e80e640 1 --2- 172.21.15.116:0/1098658426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff408102580 0x7ff408197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.203+0000 7ff40e80e640 1 -- 172.21.15.116:0/1098658426 >> 172.21.15.116:0/1098658426 conn(0x7ff4080fc140 msgr2=0x7ff408102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:58.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.204+0000 7ff40e80e640 1 -- 172.21.15.116:0/1098658426 shutdown_connections 2024-09-12T03:02:58.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.204+0000 7ff40e80e640 1 -- 172.21.15.116:0/1098658426 wait complete. 2024-09-12T03:02:58.214 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-12T03:02:58.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.318+0000 7f86ffeac640 1 -- 172.21.15.116:0/947215902 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86f8104eb0 msgr2=0x7f86f81072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:58.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.318+0000 7f86ffeac640 1 --2- 172.21.15.116:0/947215902 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86f8104eb0 0x7f86f81072a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009900 tx=0x7f86f402a410 comp rx=0 tx=0).stop 2024-09-12T03:02:58.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.318+0000 7f86ffeac640 1 -- 172.21.15.116:0/947215902 shutdown_connections 2024-09-12T03:02:58.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.318+0000 7f86ffeac640 1 --2- 172.21.15.116:0/947215902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86f81077e0 0x7f86f8109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.318+0000 7f86ffeac640 1 --2- 172.21.15.116:0/947215902 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86f8104eb0 0x7f86f81072a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.318+0000 7f86ffeac640 1 --2- 172.21.15.116:0/947215902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f86f8102580 0x7f86f8104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.318+0000 7f86ffeac640 1 -- 172.21.15.116:0/947215902 >> 172.21.15.116:0/947215902 conn(0x7f86f80fc140 msgr2=0x7f86f80fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:58.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.318+0000 7f86ffeac640 1 -- 172.21.15.116:0/947215902 shutdown_connections 2024-09-12T03:02:58.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.318+0000 7f86ffeac640 1 -- 172.21.15.116:0/947215902 wait complete. 2024-09-12T03:02:58.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.319+0000 7f86ffeac640 1 Processor -- start 2024-09-12T03:02:58.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.319+0000 7f86ffeac640 1 -- start start 2024-09-12T03:02:58.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.319+0000 7f86ffeac640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86f8102580 0x7f86f8101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:58.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.319+0000 7f86ffeac640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86f8104eb0 0x7f86f8100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:58.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.319+0000 7f86ffeac640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f86f81077e0 0x7f86f8100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:58.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.319+0000 7f86ffeac640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f86f810d790 con 0x7f86f8104eb0 2024-09-12T03:02:58.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.319+0000 7f86ffeac640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f86f810d610 con 0x7f86f81077e0 2024-09-12T03:02:58.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.319+0000 7f86ffeac640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f86f810d910 con 0x7f86f8102580 2024-09-12T03:02:58.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.319+0000 7f86fd447640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86f8104eb0 0x7f86f8100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:58.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.319+0000 7f86fd447640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86f8104eb0 0x7f86f8100530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58362/0 (socket says 172.21.15.116:58362) 2024-09-12T03:02:58.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.319+0000 7f86fd447640 1 -- 172.21.15.116:0/39773808 learned_addr learned my addr 172.21.15.116:0/39773808 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:58.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.320+0000 7f86fe449640 1 --2- 172.21.15.116:0/39773808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f86f81077e0 0x7f86f8100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:58.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.320+0000 7f86fdc48640 1 --2- 172.21.15.116:0/39773808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86f8102580 0x7f86f8101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:58.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.320+0000 7f86fd447640 1 -- 172.21.15.116:0/39773808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86f8102580 msgr2=0x7f86f8101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:58.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.320+0000 7f86fd447640 1 --2- 172.21.15.116:0/39773808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86f8102580 0x7f86f8101ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.320+0000 7f86fd447640 1 -- 172.21.15.116:0/39773808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f86f81077e0 msgr2=0x7f86f8100a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:58.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.320+0000 7f86fd447640 1 --2- 172.21.15.116:0/39773808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f86f81077e0 0x7f86f8100a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.320+0000 7f86fd447640 1 -- 172.21.15.116:0/39773808 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f8101300 con 0x7f86f8104eb0 2024-09-12T03:02:58.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.320+0000 7f86fe449640 1 --2- 172.21.15.116:0/39773808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f86f81077e0 0x7f86f8100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:58.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.320+0000 7f86fdc48640 1 --2- 172.21.15.116:0/39773808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86f8102580 0x7f86f8101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:58.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.320+0000 7f86fd447640 1 --2- 172.21.15.116:0/39773808 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86f8104eb0 0x7f86f8100530 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f86f4031610 tx=0x7f86f40043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:58.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.320+0000 7f86eaffd640 1 -- 172.21.15.116:0/39773808 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f403f030 con 0x7f86f8104eb0 2024-09-12T03:02:58.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.320+0000 7f86eaffd640 1 -- 172.21.15.116:0/39773808 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86f4003ce0 con 0x7f86f8104eb0 2024-09-12T03:02:58.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.320+0000 7f86ffeac640 1 -- 172.21.15.116:0/39773808 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f81a1f30 con 0x7f86f8104eb0 2024-09-12T03:02:58.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.320+0000 7f86eaffd640 1 -- 172.21.15.116:0/39773808 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f4035070 con 0x7f86f8104eb0 2024-09-12T03:02:58.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.320+0000 7f86ffeac640 1 -- 172.21.15.116:0/39773808 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f81a2390 con 0x7f86f8104eb0 2024-09-12T03:02:58.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.321+0000 7f86ffeac640 1 -- 172.21.15.116:0/39773808 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86f80fdc30 con 0x7f86f8104eb0 2024-09-12T03:02:58.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.322+0000 7f86eaffd640 1 -- 172.21.15.116:0/39773808 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f86f4042030 con 0x7f86f8104eb0 2024-09-12T03:02:58.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.323+0000 7f86eaffd640 1 --2- 172.21.15.116:0/39773808 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f86e40a2b90 0x7f86e40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:58.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.323+0000 7f86fdc48640 1 --2- 172.21.15.116:0/39773808 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f86e40a2b90 0x7f86e40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:58.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.323+0000 7f86eaffd640 1 -- 172.21.15.116:0/39773808 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f86f40ef5d0 con 0x7f86f8104eb0 2024-09-12T03:02:58.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.323+0000 7f86fdc48640 1 --2- 172.21.15.116:0/39773808 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f86e40a2b90 0x7f86e40a5050 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00baa0 tx=0x7f86ec005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:58.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.327+0000 7f86eaffd640 1 -- 172.21.15.116:0/39773808 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f86f40bc560 con 0x7f86f8104eb0 2024-09-12T03:02:58.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:58 smithi177 ceph-mon[35305]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T03:02:58.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:58 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/703545163' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-12T03:02:58.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:58 smithi177 ceph-mon[35305]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:02:58.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:58 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1945736740' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:02:58.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:58 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1098658426' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:02:58.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.504+0000 7f86ffeac640 1 -- 172.21.15.116:0/39773808 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f86f8062f30 con 0x7f86f8104eb0 2024-09-12T03:02:58.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.504+0000 7f86eaffd640 1 -- 172.21.15.116:0/39773808 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) ==== 104+0+11 (secure 0 0 0) 0x7f86f40bbe60 con 0x7f86f8104eb0 2024-09-12T03:02:58.505 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-09-12T03:02:58.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.506+0000 7f86ffeac640 1 -- 172.21.15.116:0/39773808 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f86e40a2b90 msgr2=0x7f86e40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:58.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.506+0000 7f86ffeac640 1 --2- 172.21.15.116:0/39773808 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f86e40a2b90 0x7f86e40a5050 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00baa0 tx=0x7f86ec005d90 comp rx=0 tx=0).stop 2024-09-12T03:02:58.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.506+0000 7f86ffeac640 1 -- 172.21.15.116:0/39773808 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86f8104eb0 msgr2=0x7f86f8100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:58.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.506+0000 7f86ffeac640 1 --2- 172.21.15.116:0/39773808 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86f8104eb0 0x7f86f8100530 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f86f4031610 tx=0x7f86f40043b0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.507+0000 7f86ffeac640 1 -- 172.21.15.116:0/39773808 shutdown_connections 2024-09-12T03:02:58.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.507+0000 7f86ffeac640 1 --2- 172.21.15.116:0/39773808 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f86e40a2b90 0x7f86e40a5050 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.507+0000 7f86ffeac640 1 --2- 172.21.15.116:0/39773808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f86f81077e0 0x7f86f8100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.507+0000 7f86ffeac640 1 --2- 172.21.15.116:0/39773808 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86f8104eb0 0x7f86f8100530 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.507+0000 7f86ffeac640 1 --2- 172.21.15.116:0/39773808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86f8102580 0x7f86f8101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.507+0000 7f86ffeac640 1 -- 172.21.15.116:0/39773808 >> 172.21.15.116:0/39773808 conn(0x7f86f80fc140 msgr2=0x7f86f8102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:58.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.507+0000 7f86ffeac640 1 -- 172.21.15.116:0/39773808 shutdown_connections 2024-09-12T03:02:58.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.507+0000 7f86ffeac640 1 -- 172.21.15.116:0/39773808 wait complete. 2024-09-12T03:02:58.513 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:58 smithi116 ceph-mon[31410]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T03:02:58.513 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:58 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/703545163' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-12T03:02:58.513 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:58 smithi116 ceph-mon[31410]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:02:58.513 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:58 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1945736740' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:02:58.513 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:58 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1098658426' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:02:58.518 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-12T03:02:58.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.620+0000 7f690fd31640 1 -- 172.21.15.116:0/205518035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6908100a00 msgr2=0x7f6908100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:58.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.620+0000 7f690fd31640 1 --2- 172.21.15.116:0/205518035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6908100a00 0x7f6908100e00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6904009900 tx=0x7f690402a410 comp rx=0 tx=0).stop 2024-09-12T03:02:58.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.620+0000 7f690fd31640 1 -- 172.21.15.116:0/205518035 shutdown_connections 2024-09-12T03:02:58.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.620+0000 7f690fd31640 1 --2- 172.21.15.116:0/205518035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69081025b0 0x7f6908108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.620+0000 7f690fd31640 1 --2- 172.21.15.116:0/205518035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6908101bf0 0x7f6908102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.620+0000 7f690fd31640 1 --2- 172.21.15.116:0/205518035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6908100a00 0x7f6908100e00 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.620+0000 7f690fd31640 1 -- 172.21.15.116:0/205518035 >> 172.21.15.116:0/205518035 conn(0x7f69080fc140 msgr2=0x7f69080fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:58.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.620+0000 7f690fd31640 1 -- 172.21.15.116:0/205518035 shutdown_connections 2024-09-12T03:02:58.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.620+0000 7f690fd31640 1 -- 172.21.15.116:0/205518035 wait complete. 2024-09-12T03:02:58.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.620+0000 7f690fd31640 1 Processor -- start 2024-09-12T03:02:58.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.621+0000 7f690fd31640 1 -- start start 2024-09-12T03:02:58.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.621+0000 7f690fd31640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6908100a00 0x7f6908069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:58.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.621+0000 7f690fd31640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6908101bf0 0x7f690819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:58.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.621+0000 7f690fd31640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69081025b0 0x7f690819f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:58.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.621+0000 7f690fd31640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f690810c680 con 0x7f6908100a00 2024-09-12T03:02:58.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.621+0000 7f690fd31640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f690810c500 con 0x7f6908101bf0 2024-09-12T03:02:58.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.621+0000 7f690fd31640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f690810c800 con 0x7f69081025b0 2024-09-12T03:02:58.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.621+0000 7f690dacd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6908100a00 0x7f6908069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:58.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.621+0000 7f690dacd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6908100a00 0x7f6908069ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58374/0 (socket says 172.21.15.116:58374) 2024-09-12T03:02:58.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.621+0000 7f690dacd640 1 -- 172.21.15.116:0/3166659217 learned_addr learned my addr 172.21.15.116:0/3166659217 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:58.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.621+0000 7f690d2cc640 1 --2- 172.21.15.116:0/3166659217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6908101bf0 0x7f690819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:58.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.621+0000 7f690e2ce640 1 --2- 172.21.15.116:0/3166659217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69081025b0 0x7f690819f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:58.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.621+0000 7f690dacd640 1 -- 172.21.15.116:0/3166659217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69081025b0 msgr2=0x7f690819f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:58.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.621+0000 7f690dacd640 1 --2- 172.21.15.116:0/3166659217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69081025b0 0x7f690819f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.621+0000 7f690dacd640 1 -- 172.21.15.116:0/3166659217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6908101bf0 msgr2=0x7f690819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:58.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.621+0000 7f690dacd640 1 --2- 172.21.15.116:0/3166659217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6908101bf0 0x7f690819cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.621+0000 7f690dacd640 1 -- 172.21.15.116:0/3166659217 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f690819fb50 con 0x7f6908100a00 2024-09-12T03:02:58.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.621+0000 7f690dacd640 1 --2- 172.21.15.116:0/3166659217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6908100a00 0x7f6908069ef0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6904031610 tx=0x7f69040043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:58.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.621+0000 7f68faffd640 1 -- 172.21.15.116:0/3166659217 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690403f030 con 0x7f6908100a00 2024-09-12T03:02:58.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.621+0000 7f690fd31640 1 -- 172.21.15.116:0/3166659217 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f690819fd80 con 0x7f6908100a00 2024-09-12T03:02:58.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.621+0000 7f68faffd640 1 -- 172.21.15.116:0/3166659217 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6904003ce0 con 0x7f6908100a00 2024-09-12T03:02:58.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.621+0000 7f68faffd640 1 -- 172.21.15.116:0/3166659217 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6904035070 con 0x7f6908100a00 2024-09-12T03:02:58.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.622+0000 7f690fd31640 1 -- 172.21.15.116:0/3166659217 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69081a01c0 con 0x7f6908100a00 2024-09-12T03:02:58.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.623+0000 7f690fd31640 1 -- 172.21.15.116:0/3166659217 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6908107ee0 con 0x7f6908100a00 2024-09-12T03:02:58.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.624+0000 7f68faffd640 1 -- 172.21.15.116:0/3166659217 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6904042030 con 0x7f6908100a00 2024-09-12T03:02:58.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.624+0000 7f68faffd640 1 --2- 172.21.15.116:0/3166659217 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f68f40a2b90 0x7f68f40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:58.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.624+0000 7f690d2cc640 1 --2- 172.21.15.116:0/3166659217 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f68f40a2b90 0x7f68f40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:58.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.624+0000 7f68faffd640 1 -- 172.21.15.116:0/3166659217 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f69040ef330 con 0x7f6908100a00 2024-09-12T03:02:58.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.625+0000 7f690d2cc640 1 --2- 172.21.15.116:0/3166659217 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f68f40a2b90 0x7f68f40a5050 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f68fc005fd0 tx=0x7f68fc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:58.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.628+0000 7f68faffd640 1 -- 172.21.15.116:0/3166659217 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f69040bc2c0 con 0x7f6908100a00 2024-09-12T03:02:58.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:58 smithi183 ceph-mon[35535]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T03:02:58.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:58 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/703545163' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-12T03:02:58.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:58 smithi183 ceph-mon[35535]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:02:58.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:58 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1945736740' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:02:58.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:58 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1098658426' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:02:58.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.793+0000 7f690fd31640 1 -- 172.21.15.116:0/3166659217 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6908062ef0 con 0x7f6908100a00 2024-09-12T03:02:58.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.794+0000 7f68faffd640 1 -- 172.21.15.116:0/3166659217 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) ==== 100+0+11 (secure 0 0 0) 0x7f69040bbbc0 con 0x7f6908100a00 2024-09-12T03:02:58.794 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 64 2024-09-12T03:02:58.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.796+0000 7f690fd31640 1 -- 172.21.15.116:0/3166659217 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f68f40a2b90 msgr2=0x7f68f40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:58.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.796+0000 7f690fd31640 1 --2- 172.21.15.116:0/3166659217 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f68f40a2b90 0x7f68f40a5050 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f68fc005fd0 tx=0x7f68fc005850 comp rx=0 tx=0).stop 2024-09-12T03:02:58.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.796+0000 7f690fd31640 1 -- 172.21.15.116:0/3166659217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6908100a00 msgr2=0x7f6908069ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:58.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.796+0000 7f690fd31640 1 --2- 172.21.15.116:0/3166659217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6908100a00 0x7f6908069ef0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6904031610 tx=0x7f69040043b0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.796+0000 7f690fd31640 1 -- 172.21.15.116:0/3166659217 shutdown_connections 2024-09-12T03:02:58.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.796+0000 7f690fd31640 1 --2- 172.21.15.116:0/3166659217 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f68f40a2b90 0x7f68f40a5050 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.796+0000 7f690fd31640 1 --2- 172.21.15.116:0/3166659217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69081025b0 0x7f690819f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.796+0000 7f690fd31640 1 --2- 172.21.15.116:0/3166659217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6908101bf0 0x7f690819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.796+0000 7f690fd31640 1 --2- 172.21.15.116:0/3166659217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6908100a00 0x7f6908069ef0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.796+0000 7f690fd31640 1 -- 172.21.15.116:0/3166659217 >> 172.21.15.116:0/3166659217 conn(0x7f69080fc140 msgr2=0x7f69081050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:58.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.796+0000 7f690fd31640 1 -- 172.21.15.116:0/3166659217 shutdown_connections 2024-09-12T03:02:58.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:58.796+0000 7f690fd31640 1 -- 172.21.15.116:0/3166659217 wait complete. 2024-09-12T03:02:58.807 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-12T03:02:58.807 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:02:59.052 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:59.457 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:59 smithi116 ceph-mon[31410]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:02:59.457 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:59 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/39773808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:02:59.457 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:02:59 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3166659217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:02:59.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:59 smithi177 ceph-mon[35305]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:02:59.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:59 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/39773808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:02:59.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:02:59 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3166659217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:02:59.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.538+0000 7f42ef5cf640 1 -- 172.21.15.116:0/4248891296 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42e8100a80 msgr2=0x7f42e8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:59.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.538+0000 7f42ef5cf640 1 --2- 172.21.15.116:0/4248891296 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42e8100a80 0x7f42e8100ee0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f42d8009930 tx=0x7f42d802a420 comp rx=0 tx=0).stop 2024-09-12T03:02:59.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.538+0000 7f42ef5cf640 1 -- 172.21.15.116:0/4248891296 shutdown_connections 2024-09-12T03:02:59.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.538+0000 7f42ef5cf640 1 --2- 172.21.15.116:0/4248891296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42e8101420 0x7f42e810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:59.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.538+0000 7f42ef5cf640 1 --2- 172.21.15.116:0/4248891296 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42e8100a80 0x7f42e8100ee0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:59.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.538+0000 7f42ef5cf640 1 --2- 172.21.15.116:0/4248891296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42e8106a80 0x7f42e8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:59.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.538+0000 7f42ef5cf640 1 -- 172.21.15.116:0/4248891296 >> 172.21.15.116:0/4248891296 conn(0x7f42e80fc7a0 msgr2=0x7f42e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:59.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.538+0000 7f42ef5cf640 1 -- 172.21.15.116:0/4248891296 shutdown_connections 2024-09-12T03:02:59.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.538+0000 7f42ef5cf640 1 -- 172.21.15.116:0/4248891296 wait complete. 2024-09-12T03:02:59.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.539+0000 7f42ef5cf640 1 Processor -- start 2024-09-12T03:02:59.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.539+0000 7f42ef5cf640 1 -- start start 2024-09-12T03:02:59.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.539+0000 7f42ef5cf640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42e8100a80 0x7f42e8199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:59.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.539+0000 7f42ef5cf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42e8101420 0x7f42e8199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:59.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.539+0000 7f42ef5cf640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42e8106a80 0x7f42e81a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:59.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.539+0000 7f42ef5cf640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f42e8110e80 con 0x7f42e8106a80 2024-09-12T03:02:59.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.539+0000 7f42ef5cf640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f42e8110d00 con 0x7f42e8100a80 2024-09-12T03:02:59.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.539+0000 7f42ef5cf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f42e8111000 con 0x7f42e8101420 2024-09-12T03:02:59.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.539+0000 7f42edb6c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42e8106a80 0x7f42e81a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:59.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.539+0000 7f42edb6c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42e8106a80 0x7f42e81a4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58390/0 (socket says 172.21.15.116:58390) 2024-09-12T03:02:59.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.539+0000 7f42edb6c640 1 -- 172.21.15.116:0/2902694513 learned_addr learned my addr 172.21.15.116:0/2902694513 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:02:59.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.539+0000 7f42ecb6a640 1 --2- 172.21.15.116:0/2902694513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42e8101420 0x7f42e8199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:59.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.540+0000 7f42ed36b640 1 --2- 172.21.15.116:0/2902694513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42e8100a80 0x7f42e8199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:59.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.540+0000 7f42edb6c640 1 -- 172.21.15.116:0/2902694513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42e8101420 msgr2=0x7f42e8199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:59.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.540+0000 7f42edb6c640 1 --2- 172.21.15.116:0/2902694513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42e8101420 0x7f42e8199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:59.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.540+0000 7f42edb6c640 1 -- 172.21.15.116:0/2902694513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42e8100a80 msgr2=0x7f42e8199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:59.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.540+0000 7f42edb6c640 1 --2- 172.21.15.116:0/2902694513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42e8100a80 0x7f42e8199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:59.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.540+0000 7f42edb6c640 1 -- 172.21.15.116:0/2902694513 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e81a48d0 con 0x7f42e8106a80 2024-09-12T03:02:59.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.540+0000 7f42ed36b640 1 --2- 172.21.15.116:0/2902694513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42e8100a80 0x7f42e8199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:59.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.540+0000 7f42edb6c640 1 --2- 172.21.15.116:0/2902694513 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42e8106a80 0x7f42e81a4390 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f42e000eaa0 tx=0x7f42e000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:59.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.540+0000 7f42de7fc640 1 -- 172.21.15.116:0/2902694513 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e000cdb0 con 0x7f42e8106a80 2024-09-12T03:02:59.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.540+0000 7f42de7fc640 1 -- 172.21.15.116:0/2902694513 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42e0005020 con 0x7f42e8106a80 2024-09-12T03:02:59.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.540+0000 7f42ef5cf640 1 -- 172.21.15.116:0/2902694513 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e81a4aa0 con 0x7f42e8106a80 2024-09-12T03:02:59.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.540+0000 7f42de7fc640 1 -- 172.21.15.116:0/2902694513 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e00116b0 con 0x7f42e8106a80 2024-09-12T03:02:59.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.540+0000 7f42ef5cf640 1 -- 172.21.15.116:0/2902694513 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e81a4f60 con 0x7f42e8106a80 2024-09-12T03:02:59.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.541+0000 7f42ef5cf640 1 -- 172.21.15.116:0/2902694513 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42e8100ee0 con 0x7f42e8106a80 2024-09-12T03:02:59.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.543+0000 7f42de7fc640 1 -- 172.21.15.116:0/2902694513 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f42e0005560 con 0x7f42e8106a80 2024-09-12T03:02:59.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.543+0000 7f42de7fc640 1 --2- 172.21.15.116:0/2902694513 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f42d40a2c90 0x7f42d40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:59.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.543+0000 7f42de7fc640 1 -- 172.21.15.116:0/2902694513 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f42e00cf710 con 0x7f42e8106a80 2024-09-12T03:02:59.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.543+0000 7f42ed36b640 1 --2- 172.21.15.116:0/2902694513 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f42d40a2c90 0x7f42d40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:59.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.544+0000 7f42ed36b640 1 --2- 172.21.15.116:0/2902694513 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f42d40a2c90 0x7f42d40a5150 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009ba0 tx=0x7f42e4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:59.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.546+0000 7f42de7fc640 1 -- 172.21.15.116:0/2902694513 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f42e0016030 con 0x7f42e8106a80 2024-09-12T03:02:59.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:59 smithi183 ceph-mon[35535]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:02:59.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:59 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/39773808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:02:59.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:02:59 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3166659217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:02:59.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.746+0000 7f42ef5cf640 1 -- 172.21.15.116:0/2902694513 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42e806ad80 con 0x7f42e8106a80 2024-09-12T03:02:59.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.747+0000 7f42de7fc640 1 -- 172.21.15.116:0/2902694513 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f42e806ad80 con 0x7f42e8106a80 2024-09-12T03:02:59.748 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:02:59.748 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:02:59.748 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:02:59.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.749+0000 7f42ef5cf640 1 -- 172.21.15.116:0/2902694513 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f42d40a2c90 msgr2=0x7f42d40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:59.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.749+0000 7f42ef5cf640 1 --2- 172.21.15.116:0/2902694513 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f42d40a2c90 0x7f42d40a5150 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009ba0 tx=0x7f42e4005f90 comp rx=0 tx=0).stop 2024-09-12T03:02:59.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.749+0000 7f42ef5cf640 1 -- 172.21.15.116:0/2902694513 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42e8106a80 msgr2=0x7f42e81a4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:59.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.749+0000 7f42ef5cf640 1 --2- 172.21.15.116:0/2902694513 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42e8106a80 0x7f42e81a4390 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f42e000eaa0 tx=0x7f42e000ef60 comp rx=0 tx=0).stop 2024-09-12T03:02:59.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.749+0000 7f42ef5cf640 1 -- 172.21.15.116:0/2902694513 shutdown_connections 2024-09-12T03:02:59.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.749+0000 7f42ef5cf640 1 --2- 172.21.15.116:0/2902694513 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f42d40a2c90 0x7f42d40a5150 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:59.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.749+0000 7f42ef5cf640 1 --2- 172.21.15.116:0/2902694513 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42e8106a80 0x7f42e81a4390 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:59.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.749+0000 7f42ef5cf640 1 --2- 172.21.15.116:0/2902694513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42e8101420 0x7f42e8199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:59.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.749+0000 7f42ef5cf640 1 --2- 172.21.15.116:0/2902694513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42e8100a80 0x7f42e8199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:59.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.749+0000 7f42ef5cf640 1 -- 172.21.15.116:0/2902694513 >> 172.21.15.116:0/2902694513 conn(0x7f42e80fc7a0 msgr2=0x7f42e80fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:59.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.749+0000 7f42ef5cf640 1 -- 172.21.15.116:0/2902694513 shutdown_connections 2024-09-12T03:02:59.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:02:59.749+0000 7f42ef5cf640 1 -- 172.21.15.116:0/2902694513 wait complete. 2024-09-12T03:03:00.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:00 smithi177 ceph-mon[35305]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T03:03:00.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:00 smithi177 ceph-mon[35305]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-12T03:03:00.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:00 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2902694513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:00.537 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:00 smithi116 ceph-mon[31410]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T03:03:00.537 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:00 smithi116 ceph-mon[31410]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-12T03:03:00.537 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:00 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2902694513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:00.566 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-12T03:03:00.567 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-12T03:03:00.567 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:03:00.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.669+0000 7f9140053640 1 -- 172.21.15.116:0/2642775331 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9138104ee0 msgr2=0x7f91381072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:00.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.669+0000 7f9140053640 1 --2- 172.21.15.116:0/2642775331 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9138104ee0 0x7f91381072d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9128009900 tx=0x7f912802a410 comp rx=0 tx=0).stop 2024-09-12T03:03:00.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.669+0000 7f9140053640 1 -- 172.21.15.116:0/2642775331 shutdown_connections 2024-09-12T03:03:00.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.669+0000 7f9140053640 1 --2- 172.21.15.116:0/2642775331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9138107810 0x7f9138109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.669+0000 7f9140053640 1 --2- 172.21.15.116:0/2642775331 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9138104ee0 0x7f91381072d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.669+0000 7f9140053640 1 --2- 172.21.15.116:0/2642775331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91381025b0 0x7f91381049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.669+0000 7f9140053640 1 -- 172.21.15.116:0/2642775331 >> 172.21.15.116:0/2642775331 conn(0x7f91380fc1b0 msgr2=0x7f91380fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:00.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.670+0000 7f9140053640 1 -- 172.21.15.116:0/2642775331 shutdown_connections 2024-09-12T03:03:00.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.670+0000 7f9140053640 1 -- 172.21.15.116:0/2642775331 wait complete. 2024-09-12T03:03:00.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.670+0000 7f9140053640 1 Processor -- start 2024-09-12T03:03:00.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.670+0000 7f9140053640 1 -- start start 2024-09-12T03:03:00.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.671+0000 7f9140053640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91381025b0 0x7f9138197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:00.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.671+0000 7f9140053640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9138104ee0 0x7f9138198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:00.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.671+0000 7f9140053640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9138107810 0x7f913819f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:00.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.671+0000 7f9140053640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f913810d7e0 con 0x7f9138104ee0 2024-09-12T03:03:00.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.671+0000 7f9140053640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f913810d660 con 0x7f9138107810 2024-09-12T03:03:00.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.671+0000 7f9140053640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f913810d960 con 0x7f91381025b0 2024-09-12T03:03:00.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.671+0000 7f913d5ee640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9138104ee0 0x7f9138198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:00.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.671+0000 7f913d5ee640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9138104ee0 0x7f9138198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58406/0 (socket says 172.21.15.116:58406) 2024-09-12T03:03:00.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.671+0000 7f913d5ee640 1 -- 172.21.15.116:0/1868801951 learned_addr learned my addr 172.21.15.116:0/1868801951 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:00.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.671+0000 7f913ddef640 1 --2- 172.21.15.116:0/1868801951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91381025b0 0x7f9138197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:00.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.671+0000 7f913e5f0640 1 --2- 172.21.15.116:0/1868801951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9138107810 0x7f913819f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:00.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.671+0000 7f913d5ee640 1 -- 172.21.15.116:0/1868801951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91381025b0 msgr2=0x7f9138197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:00.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.671+0000 7f913d5ee640 1 --2- 172.21.15.116:0/1868801951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91381025b0 0x7f9138197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.671+0000 7f913d5ee640 1 -- 172.21.15.116:0/1868801951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9138107810 msgr2=0x7f913819f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:00.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.671+0000 7f913d5ee640 1 --2- 172.21.15.116:0/1868801951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9138107810 0x7f913819f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.671+0000 7f913d5ee640 1 -- 172.21.15.116:0/1868801951 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f913819fb20 con 0x7f9138104ee0 2024-09-12T03:03:00.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.671+0000 7f913d5ee640 1 --2- 172.21.15.116:0/1868801951 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9138104ee0 0x7f9138198440 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9128031610 tx=0x7f9128004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:00.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.671+0000 7f912effd640 1 -- 172.21.15.116:0/1868801951 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912803f030 con 0x7f9138104ee0 2024-09-12T03:03:00.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.671+0000 7f912effd640 1 -- 172.21.15.116:0/1868801951 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9128003ad0 con 0x7f9138104ee0 2024-09-12T03:03:00.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.671+0000 7f9140053640 1 -- 172.21.15.116:0/1868801951 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f913819fdb0 con 0x7f9138104ee0 2024-09-12T03:03:00.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.671+0000 7f912effd640 1 -- 172.21.15.116:0/1868801951 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9128035070 con 0x7f9138104ee0 2024-09-12T03:03:00.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.672+0000 7f9140053640 1 -- 172.21.15.116:0/1868801951 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91381a0240 con 0x7f9138104ee0 2024-09-12T03:03:00.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.672+0000 7f9140053640 1 -- 172.21.15.116:0/1868801951 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91380fdca0 con 0x7f9138104ee0 2024-09-12T03:03:00.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.674+0000 7f912effd640 1 -- 172.21.15.116:0/1868801951 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f9128042030 con 0x7f9138104ee0 2024-09-12T03:03:00.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.675+0000 7f912effd640 1 --2- 172.21.15.116:0/1868801951 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f91240a2c20 0x7f91240a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:00.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.675+0000 7f913ddef640 1 --2- 172.21.15.116:0/1868801951 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f91240a2c20 0x7f91240a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:00.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.675+0000 7f912effd640 1 -- 172.21.15.116:0/1868801951 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f91280efa60 con 0x7f9138104ee0 2024-09-12T03:03:00.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.675+0000 7f913ddef640 1 --2- 172.21.15.116:0/1868801951 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f91240a2c20 0x7f91240a50e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9134009bd0 tx=0x7f9134005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:00.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.678+0000 7f912effd640 1 -- 172.21.15.116:0/1868801951 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f91280bc240 con 0x7f9138104ee0 2024-09-12T03:03:00.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:00 smithi183 ceph-mon[35535]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T03:03:00.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:00 smithi183 ceph-mon[35535]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-12T03:03:00.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:00 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2902694513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:00.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.842+0000 7f9140053640 1 -- 172.21.15.116:0/1868801951 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9138062630 con 0x7f91240a2c20 2024-09-12T03:03:00.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.849+0000 7f912effd640 1 -- 172.21.15.116:0/1868801951 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258977 (secure 0 0 0) 0x7f9138062630 con 0x7f91240a2c20 2024-09-12T03:03:00.851 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:03:00.853 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-09-12T03:03:00.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.851+0000 7f9140053640 1 -- 172.21.15.116:0/1868801951 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f91240a2c20 msgr2=0x7f91240a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:00.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.851+0000 7f9140053640 1 --2- 172.21.15.116:0/1868801951 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f91240a2c20 0x7f91240a50e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9134009bd0 tx=0x7f9134005f90 comp rx=0 tx=0).stop 2024-09-12T03:03:00.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.851+0000 7f9140053640 1 -- 172.21.15.116:0/1868801951 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9138104ee0 msgr2=0x7f9138198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:00.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.851+0000 7f9140053640 1 --2- 172.21.15.116:0/1868801951 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9138104ee0 0x7f9138198440 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9128031610 tx=0x7f9128004270 comp rx=0 tx=0).stop 2024-09-12T03:03:00.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.851+0000 7f9140053640 1 -- 172.21.15.116:0/1868801951 shutdown_connections 2024-09-12T03:03:00.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.851+0000 7f9140053640 1 --2- 172.21.15.116:0/1868801951 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f91240a2c20 0x7f91240a50e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.851+0000 7f9140053640 1 --2- 172.21.15.116:0/1868801951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9138107810 0x7f913819f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.851+0000 7f9140053640 1 --2- 172.21.15.116:0/1868801951 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9138104ee0 0x7f9138198440 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.851+0000 7f9140053640 1 --2- 172.21.15.116:0/1868801951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91381025b0 0x7f9138197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.852+0000 7f9140053640 1 -- 172.21.15.116:0/1868801951 >> 172.21.15.116:0/1868801951 conn(0x7f91380fc1b0 msgr2=0x7f91381030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:00.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.852+0000 7f9140053640 1 -- 172.21.15.116:0/1868801951 shutdown_connections 2024-09-12T03:03:00.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:00.852+0000 7f9140053640 1 -- 172.21.15.116:0/1868801951 wait complete. 2024-09-12T03:03:00.863 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-09-12T03:02:59.135937+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656521,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":798,"num_read_kb":515,"num_write":353,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180875,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":783,"ondisk_log_size":783,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368384,"kb_used_data":10816,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573440,"statfs":{"total":1151940427776,"available":1151563202560,"internally_reserved":0,"allocated":11075584,"data_stored":7150101,"data_compressed":107490,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002931"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.133420+0000","last_change":"2024-09-12T03:00:58.348563+0000","last_active":"2024-09-12T03:02:33.133420+0000","last_peered":"2024-09-12T03:02:33.133420+0000","last_clean":"2024-09-12T03:02:33.133420+0000","last_became_active":"2024-09-12T03:00:58.347940+0000","last_became_peered":"2024-09-12T03:00:58.347940+0000","last_unstale":"2024-09-12T03:02:33.133420+0000","last_undegraded":"2024-09-12T03:02:33.133420+0000","last_fullsized":"2024-09-12T03:02:33.133420+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:43.489238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.119217+0000","last_change":"2024-09-12T03:00:58.351969+0000","last_active":"2024-09-12T03:02:33.119217+0000","last_peered":"2024-09-12T03:02:33.119217+0000","last_clean":"2024-09-12T03:02:33.119217+0000","last_became_active":"2024-09-12T03:00:58.351679+0000","last_became_peered":"2024-09-12T03:00:58.351679+0000","last_unstale":"2024-09-12T03:02:33.119217+0000","last_undegraded":"2024-09-12T03:02:33.119217+0000","last_fullsized":"2024-09-12T03:02:33.119217+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:27:01.174890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120115+0000","last_change":"2024-09-12T03:00:58.346045+0000","last_active":"2024-09-12T03:02:33.120115+0000","last_peered":"2024-09-12T03:02:33.120115+0000","last_clean":"2024-09-12T03:02:33.120115+0000","last_became_active":"2024-09-12T03:00:58.345719+0000","last_became_peered":"2024-09-12T03:00:58.345719+0000","last_unstale":"2024-09-12T03:02:33.120115+0000","last_undegraded":"2024-09-12T03:02:33.120115+0000","last_fullsized":"2024-09-12T03:02:33.120115+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:12.312650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.110625+0000","last_change":"2024-09-12T03:00:58.348165+0000","last_active":"2024-09-12T03:02:33.110625+0000","last_peered":"2024-09-12T03:02:33.110625+0000","last_clean":"2024-09-12T03:02:33.110625+0000","last_became_active":"2024-09-12T03:00:58.347734+0000","last_became_peered":"2024-09-12T03:00:58.347734+0000","last_unstale":"2024-09-12T03:02:33.110625+0000","last_undegraded":"2024-09-12T03:02:33.110625+0000","last_fullsized":"2024-09-12T03:02:33.110625+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:05.945900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.108519+0000","last_change":"2024-09-12T03:00:58.339316+0000","last_active":"2024-09-12T03:02:33.108519+0000","last_peered":"2024-09-12T03:02:33.108519+0000","last_clean":"2024-09-12T03:02:33.108519+0000","last_became_active":"2024-09-12T03:00:58.338964+0000","last_became_peered":"2024-09-12T03:00:58.338964+0000","last_unstale":"2024-09-12T03:02:33.108519+0000","last_undegraded":"2024-09-12T03:02:33.108519+0000","last_fullsized":"2024-09-12T03:02:33.108519+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:50.721276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.110921+0000","last_change":"2024-09-12T03:00:58.346838+0000","last_active":"2024-09-12T03:02:33.110921+0000","last_peered":"2024-09-12T03:02:33.110921+0000","last_clean":"2024-09-12T03:02:33.110921+0000","last_became_active":"2024-09-12T03:00:58.346486+0000","last_became_peered":"2024-09-12T03:00:58.346486+0000","last_unstale":"2024-09-12T03:02:33.110921+0000","last_undegraded":"2024-09-12T03:02:33.110921+0000","last_fullsized":"2024-09-12T03:02:33.110921+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:28.687727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.122919+0000","last_change":"2024-09-12T03:00:58.346712+0000","last_active":"2024-09-12T03:02:33.122919+0000","last_peered":"2024-09-12T03:02:33.122919+0000","last_clean":"2024-09-12T03:02:33.122919+0000","last_became_active":"2024-09-12T03:00:58.346397+0000","last_became_peered":"2024-09-12T03:00:58.346397+0000","last_unstale":"2024-09-12T03:02:33.122919+0000","last_undegraded":"2024-09-12T03:02:33.122919+0000","last_fullsized":"2024-09-12T03:02:33.122919+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:48:00.624733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.118885+0000","last_change":"2024-09-12T03:00:58.356166+0000","last_active":"2024-09-12T03:02:33.118885+0000","last_peered":"2024-09-12T03:02:33.118885+0000","last_clean":"2024-09-12T03:02:33.118885+0000","last_became_active":"2024-09-12T03:00:58.354944+0000","last_became_peered":"2024-09-12T03:00:58.354944+0000","last_unstale":"2024-09-12T03:02:33.118885+0000","last_undegraded":"2024-09-12T03:02:33.118885+0000","last_fullsized":"2024-09-12T03:02:33.118885+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:15.840149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.133064+0000","last_change":"2024-09-12T03:00:58.340182+0000","last_active":"2024-09-12T03:02:33.133064+0000","last_peered":"2024-09-12T03:02:33.133064+0000","last_clean":"2024-09-12T03:02:33.133064+0000","last_became_active":"2024-09-12T03:00:58.339576+0000","last_became_peered":"2024-09-12T03:00:58.339576+0000","last_unstale":"2024-09-12T03:02:33.133064+0000","last_undegraded":"2024-09-12T03:02:33.133064+0000","last_fullsized":"2024-09-12T03:02:33.133064+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:16.534844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.117254+0000","last_change":"2024-09-12T03:00:58.351218+0000","last_active":"2024-09-12T03:02:33.117254+0000","last_peered":"2024-09-12T03:02:33.117254+0000","last_clean":"2024-09-12T03:02:33.117254+0000","last_became_active":"2024-09-12T03:00:58.350871+0000","last_became_peered":"2024-09-12T03:00:58.350871+0000","last_unstale":"2024-09-12T03:02:33.117254+0000","last_undegraded":"2024-09-12T03:02:33.117254+0000","last_fullsized":"2024-09-12T03:02:33.117254+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:38.367683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.130892+0000","last_change":"2024-09-12T03:00:58.351894+0000","last_active":"2024-09-12T03:02:33.130892+0000","last_peered":"2024-09-12T03:02:33.130892+0000","last_clean":"2024-09-12T03:02:33.130892+0000","last_became_active":"2024-09-12T03:00:58.351445+0000","last_became_peered":"2024-09-12T03:00:58.351445+0000","last_unstale":"2024-09-12T03:02:33.130892+0000","last_undegraded":"2024-09-12T03:02:33.130892+0000","last_fullsized":"2024-09-12T03:02:33.130892+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:42.093871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.107323+0000","last_change":"2024-09-12T03:00:58.353460+0000","last_active":"2024-09-12T03:02:33.107323+0000","last_peered":"2024-09-12T03:02:33.107323+0000","last_clean":"2024-09-12T03:02:33.107323+0000","last_became_active":"2024-09-12T03:00:58.353111+0000","last_became_peered":"2024-09-12T03:00:58.353111+0000","last_unstale":"2024-09-12T03:02:33.107323+0000","last_undegraded":"2024-09-12T03:02:33.107323+0000","last_fullsized":"2024-09-12T03:02:33.107323+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:15.916765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.111860+0000","last_change":"2024-09-12T03:00:58.341080+0000","last_active":"2024-09-12T03:02:33.111860+0000","last_peered":"2024-09-12T03:02:33.111860+0000","last_clean":"2024-09-12T03:02:33.111860+0000","last_became_active":"2024-09-12T03:00:58.340782+0000","last_became_peered":"2024-09-12T03:00:58.340782+0000","last_unstale":"2024-09-12T03:02:33.111860+0000","last_undegraded":"2024-09-12T03:02:33.111860+0000","last_fullsized":"2024-09-12T03:02:33.111860+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:36.569700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.106506+0000","last_change":"2024-09-12T03:00:58.346791+0000","last_active":"2024-09-12T03:02:33.106506+0000","last_peered":"2024-09-12T03:02:33.106506+0000","last_clean":"2024-09-12T03:02:33.106506+0000","last_became_active":"2024-09-12T03:00:58.346500+0000","last_became_peered":"2024-09-12T03:00:58.346500+0000","last_unstale":"2024-09-12T03:02:33.106506+0000","last_undegraded":"2024-09-12T03:02:33.106506+0000","last_fullsized":"2024-09-12T03:02:33.106506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:32.017589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.112821+0000","last_change":"2024-09-12T03:00:58.349522+0000","last_active":"2024-09-12T03:02:33.112821+0000","last_peered":"2024-09-12T03:02:33.112821+0000","last_clean":"2024-09-12T03:02:33.112821+0000","last_became_active":"2024-09-12T03:00:58.349246+0000","last_became_peered":"2024-09-12T03:00:58.349246+0000","last_unstale":"2024-09-12T03:02:33.112821+0000","last_undegraded":"2024-09-12T03:02:33.112821+0000","last_fullsized":"2024-09-12T03:02:33.112821+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:39.253919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.134146+0000","last_change":"2024-09-12T03:00:58.349909+0000","last_active":"2024-09-12T03:02:33.134146+0000","last_peered":"2024-09-12T03:02:33.134146+0000","last_clean":"2024-09-12T03:02:33.134146+0000","last_became_active":"2024-09-12T03:00:58.349590+0000","last_became_peered":"2024-09-12T03:00:58.349590+0000","last_unstale":"2024-09-12T03:02:33.134146+0000","last_undegraded":"2024-09-12T03:02:33.134146+0000","last_fullsized":"2024-09-12T03:02:33.134146+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:04.540709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.133565+0000","last_change":"2024-09-12T03:00:58.348284+0000","last_active":"2024-09-12T03:02:33.133565+0000","last_peered":"2024-09-12T03:02:33.133565+0000","last_clean":"2024-09-12T03:02:33.133565+0000","last_became_active":"2024-09-12T03:00:58.347465+0000","last_became_peered":"2024-09-12T03:00:58.347465+0000","last_unstale":"2024-09-12T03:02:33.133565+0000","last_undegraded":"2024-09-12T03:02:33.133565+0000","last_fullsized":"2024-09-12T03:02:33.133565+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:13.191899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.122657+0000","last_change":"2024-09-12T03:00:58.337740+0000","last_active":"2024-09-12T03:02:33.122657+0000","last_peered":"2024-09-12T03:02:33.122657+0000","last_clean":"2024-09-12T03:02:33.122657+0000","last_became_active":"2024-09-12T03:00:58.337393+0000","last_became_peered":"2024-09-12T03:00:58.337393+0000","last_unstale":"2024-09-12T03:02:33.122657+0000","last_undegraded":"2024-09-12T03:02:33.122657+0000","last_fullsized":"2024-09-12T03:02:33.122657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:28:56.339167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.111666+0000","last_change":"2024-09-12T03:00:58.345779+0000","last_active":"2024-09-12T03:02:33.111666+0000","last_peered":"2024-09-12T03:02:33.111666+0000","last_clean":"2024-09-12T03:02:33.111666+0000","last_became_active":"2024-09-12T03:00:58.345210+0000","last_became_peered":"2024-09-12T03:00:58.345210+0000","last_unstale":"2024-09-12T03:02:33.111666+0000","last_undegraded":"2024-09-12T03:02:33.111666+0000","last_fullsized":"2024-09-12T03:02:33.111666+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:06:58.271185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120653+0000","last_change":"2024-09-12T03:02:04.294381+0000","last_active":"2024-09-12T03:02:33.120653+0000","last_peered":"2024-09-12T03:02:33.120653+0000","last_clean":"2024-09-12T03:02:33.120653+0000","last_became_active":"2024-09-12T03:02:04.293795+0000","last_became_peered":"2024-09-12T03:02:04.293795+0000","last_unstale":"2024-09-12T03:02:33.120653+0000","last_undegraded":"2024-09-12T03:02:33.120653+0000","last_fullsized":"2024-09-12T03:02:33.120653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:06:58.271007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.124292+0000","last_change":"2024-09-12T03:00:58.349233+0000","last_active":"2024-09-12T03:02:33.124292+0000","last_peered":"2024-09-12T03:02:33.124292+0000","last_clean":"2024-09-12T03:02:33.124292+0000","last_became_active":"2024-09-12T03:00:58.348958+0000","last_became_peered":"2024-09-12T03:00:58.348958+0000","last_unstale":"2024-09-12T03:02:33.124292+0000","last_undegraded":"2024-09-12T03:02:33.124292+0000","last_fullsized":"2024-09-12T03:02:33.124292+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:32:10.271817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.134703+0000","last_change":"2024-09-12T03:00:58.358024+0000","last_active":"2024-09-12T03:02:33.134703+0000","last_peered":"2024-09-12T03:02:33.134703+0000","last_clean":"2024-09-12T03:02:33.134703+0000","last_became_active":"2024-09-12T03:00:58.357434+0000","last_became_peered":"2024-09-12T03:00:58.357434+0000","last_unstale":"2024-09-12T03:02:33.134703+0000","last_undegraded":"2024-09-12T03:02:33.134703+0000","last_fullsized":"2024-09-12T03:02:33.134703+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:01:20.119733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120095+0000","last_change":"2024-09-12T03:00:58.355063+0000","last_active":"2024-09-12T03:02:33.120095+0000","last_peered":"2024-09-12T03:02:33.120095+0000","last_clean":"2024-09-12T03:02:33.120095+0000","last_became_active":"2024-09-12T03:00:58.354760+0000","last_became_peered":"2024-09-12T03:00:58.354760+0000","last_unstale":"2024-09-12T03:02:33.120095+0000","last_undegraded":"2024-09-12T03:02:33.120095+0000","last_fullsized":"2024-09-12T03:02:33.120095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:13.464413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.112255+0000","last_change":"2024-09-12T03:00:58.341916+0000","last_active":"2024-09-12T03:02:33.112255+0000","last_peered":"2024-09-12T03:02:33.112255+0000","last_clean":"2024-09-12T03:02:33.112255+0000","last_became_active":"2024-09-12T03:00:58.340815+0000","last_became_peered":"2024-09-12T03:00:58.340815+0000","last_unstale":"2024-09-12T03:02:33.112255+0000","last_undegraded":"2024-09-12T03:02:33.112255+0000","last_fullsized":"2024-09-12T03:02:33.112255+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:13.777947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.117061+0000","last_change":"2024-09-12T03:00:58.357765+0000","last_active":"2024-09-12T03:02:33.117061+0000","last_peered":"2024-09-12T03:02:33.117061+0000","last_clean":"2024-09-12T03:02:33.117061+0000","last_became_active":"2024-09-12T03:00:58.357480+0000","last_became_peered":"2024-09-12T03:00:58.357480+0000","last_unstale":"2024-09-12T03:02:33.117061+0000","last_undegraded":"2024-09-12T03:02:33.117061+0000","last_fullsized":"2024-09-12T03:02:33.117061+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:26.884381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.108215+0000","last_change":"2024-09-12T03:00:58.349504+0000","last_active":"2024-09-12T03:02:33.108215+0000","last_peered":"2024-09-12T03:02:33.108215+0000","last_clean":"2024-09-12T03:02:33.108215+0000","last_became_active":"2024-09-12T03:00:58.349136+0000","last_became_peered":"2024-09-12T03:00:58.349136+0000","last_unstale":"2024-09-12T03:02:33.108215+0000","last_undegraded":"2024-09-12T03:02:33.108215+0000","last_fullsized":"2024-09-12T03:02:33.108215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:09:55.229516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.122920+0000","last_change":"2024-09-12T03:00:58.341902+0000","last_active":"2024-09-12T03:02:33.122920+0000","last_peered":"2024-09-12T03:02:33.122920+0000","last_clean":"2024-09-12T03:02:33.122920+0000","last_became_active":"2024-09-12T03:00:58.341592+0000","last_became_peered":"2024-09-12T03:00:58.341592+0000","last_unstale":"2024-09-12T03:02:33.122920+0000","last_undegraded":"2024-09-12T03:02:33.122920+0000","last_fullsized":"2024-09-12T03:02:33.122920+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:13.852511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.114521+0000","last_change":"2024-09-12T03:00:58.348012+0000","last_active":"2024-09-12T03:02:33.114521+0000","last_peered":"2024-09-12T03:02:33.114521+0000","last_clean":"2024-09-12T03:02:33.114521+0000","last_became_active":"2024-09-12T03:00:58.347338+0000","last_became_peered":"2024-09-12T03:00:58.347338+0000","last_unstale":"2024-09-12T03:02:33.114521+0000","last_undegraded":"2024-09-12T03:02:33.114521+0000","last_fullsized":"2024-09-12T03:02:33.114521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:41.478032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.112751+0000","last_change":"2024-09-12T03:00:58.345833+0000","last_active":"2024-09-12T03:02:33.112751+0000","last_peered":"2024-09-12T03:02:33.112751+0000","last_clean":"2024-09-12T03:02:33.112751+0000","last_became_active":"2024-09-12T03:00:58.345482+0000","last_became_peered":"2024-09-12T03:00:58.345482+0000","last_unstale":"2024-09-12T03:02:33.112751+0000","last_undegraded":"2024-09-12T03:02:33.112751+0000","last_fullsized":"2024-09-12T03:02:33.112751+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:32.037412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.112502+0000","last_change":"2024-09-12T03:00:58.352290+0000","last_active":"2024-09-12T03:02:33.112502+0000","last_peered":"2024-09-12T03:02:33.112502+0000","last_clean":"2024-09-12T03:02:33.112502+0000","last_became_active":"2024-09-12T03:00:58.351918+0000","last_became_peered":"2024-09-12T03:00:58.351918+0000","last_unstale":"2024-09-12T03:02:33.112502+0000","last_undegraded":"2024-09-12T03:02:33.112502+0000","last_fullsized":"2024-09-12T03:02:33.112502+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:25.941080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.114976+0000","last_change":"2024-09-12T03:00:58.348021+0000","last_active":"2024-09-12T03:02:33.114976+0000","last_peered":"2024-09-12T03:02:33.114976+0000","last_clean":"2024-09-12T03:02:33.114976+0000","last_became_active":"2024-09-12T03:00:58.347634+0000","last_became_peered":"2024-09-12T03:00:58.347634+0000","last_unstale":"2024-09-12T03:02:33.114976+0000","last_undegraded":"2024-09-12T03:02:33.114976+0000","last_fullsized":"2024-09-12T03:02:33.114976+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:47.933067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.123449+0000","last_change":"2024-09-12T03:00:58.340891+0000","last_active":"2024-09-12T03:02:33.123449+0000","last_peered":"2024-09-12T03:02:33.123449+0000","last_clean":"2024-09-12T03:02:33.123449+0000","last_became_active":"2024-09-12T03:00:58.340504+0000","last_became_peered":"2024-09-12T03:00:58.340504+0000","last_unstale":"2024-09-12T03:02:33.123449+0000","last_undegraded":"2024-09-12T03:02:33.123449+0000","last_fullsized":"2024-09-12T03:02:33.123449+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:32.117635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.108754+0000","last_change":"2024-09-12T03:00:58.339343+0000","last_active":"2024-09-12T03:02:33.108754+0000","last_peered":"2024-09-12T03:02:33.108754+0000","last_clean":"2024-09-12T03:02:33.108754+0000","last_became_active":"2024-09-12T03:00:58.339001+0000","last_became_peered":"2024-09-12T03:00:58.339001+0000","last_unstale":"2024-09-12T03:02:33.108754+0000","last_undegraded":"2024-09-12T03:02:33.108754+0000","last_fullsized":"2024-09-12T03:02:33.108754+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:02.043454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":135,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:52.555932+0000","last_change":"2024-09-12T03:00:56.315550+0000","last_active":"2024-09-12T03:02:52.555932+0000","last_peered":"2024-09-12T03:02:52.555932+0000","last_clean":"2024-09-12T03:02:52.555932+0000","last_became_active":"2024-09-12T03:00:56.315225+0000","last_became_peered":"2024-09-12T03:00:56.315225+0000","last_unstale":"2024-09-12T03:02:52.555932+0000","last_undegraded":"2024-09-12T03:02:52.555932+0000","last_fullsized":"2024-09-12T03:02:52.555932+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:46.070895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.116508+0000","last_change":"2024-09-12T03:02:04.290498+0000","last_active":"2024-09-12T03:02:33.116508+0000","last_peered":"2024-09-12T03:02:33.116508+0000","last_clean":"2024-09-12T03:02:33.116508+0000","last_became_active":"2024-09-12T03:02:04.290162+0000","last_became_peered":"2024-09-12T03:02:04.290162+0000","last_unstale":"2024-09-12T03:02:33.116508+0000","last_undegraded":"2024-09-12T03:02:33.116508+0000","last_fullsized":"2024-09-12T03:02:33.116508+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:29.777155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282566+0000","last_change":"2024-09-12T03:00:56.316385+0000","last_active":"2024-09-12T03:02:04.282566+0000","last_peered":"2024-09-12T03:02:04.282566+0000","last_clean":"2024-09-12T03:02:04.282566+0000","last_became_active":"2024-09-12T03:00:56.316064+0000","last_became_peered":"2024-09-12T03:00:56.316064+0000","last_unstale":"2024-09-12T03:02:04.282566+0000","last_undegraded":"2024-09-12T03:02:04.282566+0000","last_fullsized":"2024-09-12T03:02:04.282566+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:12:43.323654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332424+0000","last_change":"2024-09-12T03:00:56.313065+0000","last_active":"2024-09-12T03:02:04.332424+0000","last_peered":"2024-09-12T03:02:04.332424+0000","last_clean":"2024-09-12T03:02:04.332424+0000","last_became_active":"2024-09-12T03:00:56.312716+0000","last_became_peered":"2024-09-12T03:00:56.312716+0000","last_unstale":"2024-09-12T03:02:04.332424+0000","last_undegraded":"2024-09-12T03:02:04.332424+0000","last_fullsized":"2024-09-12T03:02:04.332424+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:51.089089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120152+0000","last_change":"2024-09-12T03:00:58.346542+0000","last_active":"2024-09-12T03:02:33.120152+0000","last_peered":"2024-09-12T03:02:33.120152+0000","last_clean":"2024-09-12T03:02:33.120152+0000","last_became_active":"2024-09-12T03:00:58.346260+0000","last_became_peered":"2024-09-12T03:00:58.346260+0000","last_unstale":"2024-09-12T03:02:33.120152+0000","last_undegraded":"2024-09-12T03:02:33.120152+0000","last_fullsized":"2024-09-12T03:02:33.120152+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:37.034969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289182+0000","last_change":"2024-09-12T03:00:56.317308+0000","last_active":"2024-09-12T03:02:04.289182+0000","last_peered":"2024-09-12T03:02:04.289182+0000","last_clean":"2024-09-12T03:02:04.289182+0000","last_became_active":"2024-09-12T03:00:56.316935+0000","last_became_peered":"2024-09-12T03:00:56.316935+0000","last_unstale":"2024-09-12T03:02:04.289182+0000","last_undegraded":"2024-09-12T03:02:04.289182+0000","last_fullsized":"2024-09-12T03:02:04.289182+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:06.324489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.118033+0000","last_change":"2024-09-12T03:00:58.341565+0000","last_active":"2024-09-12T03:02:33.118033+0000","last_peered":"2024-09-12T03:02:33.118033+0000","last_clean":"2024-09-12T03:02:33.118033+0000","last_became_active":"2024-09-12T03:00:58.341285+0000","last_became_peered":"2024-09-12T03:00:58.341285+0000","last_unstale":"2024-09-12T03:02:33.118033+0000","last_undegraded":"2024-09-12T03:02:33.118033+0000","last_fullsized":"2024-09-12T03:02:33.118033+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:59.156700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":133,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:54.558057+0000","last_change":"2024-09-12T03:00:56.314201+0000","last_active":"2024-09-12T03:02:54.558057+0000","last_peered":"2024-09-12T03:02:54.558057+0000","last_clean":"2024-09-12T03:02:54.558057+0000","last_became_active":"2024-09-12T03:00:56.313880+0000","last_became_peered":"2024-09-12T03:00:56.313880+0000","last_unstale":"2024-09-12T03:02:54.558057+0000","last_undegraded":"2024-09-12T03:02:54.558057+0000","last_fullsized":"2024-09-12T03:02:54.558057+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:57:12.952630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.132547+0000","last_change":"2024-09-12T03:00:58.345274+0000","last_active":"2024-09-12T03:02:33.132547+0000","last_peered":"2024-09-12T03:02:33.132547+0000","last_clean":"2024-09-12T03:02:33.132547+0000","last_became_active":"2024-09-12T03:00:58.344937+0000","last_became_peered":"2024-09-12T03:00:58.344937+0000","last_unstale":"2024-09-12T03:02:33.132547+0000","last_undegraded":"2024-09-12T03:02:33.132547+0000","last_fullsized":"2024-09-12T03:02:33.132547+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:17.288909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284614+0000","last_change":"2024-09-12T03:00:56.314709+0000","last_active":"2024-09-12T03:02:04.284614+0000","last_peered":"2024-09-12T03:02:04.284614+0000","last_clean":"2024-09-12T03:02:04.284614+0000","last_became_active":"2024-09-12T03:00:56.314349+0000","last_became_peered":"2024-09-12T03:00:56.314349+0000","last_unstale":"2024-09-12T03:02:04.284614+0000","last_undegraded":"2024-09-12T03:02:04.284614+0000","last_fullsized":"2024-09-12T03:02:04.284614+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:18:35.371661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.113713+0000","last_change":"2024-09-12T03:00:58.346055+0000","last_active":"2024-09-12T03:02:33.113713+0000","last_peered":"2024-09-12T03:02:33.113713+0000","last_clean":"2024-09-12T03:02:33.113713+0000","last_became_active":"2024-09-12T03:00:58.345701+0000","last_became_peered":"2024-09-12T03:00:58.345701+0000","last_unstale":"2024-09-12T03:02:33.113713+0000","last_undegraded":"2024-09-12T03:02:33.113713+0000","last_fullsized":"2024-09-12T03:02:33.113713+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:33.977245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.287450+0000","last_change":"2024-09-12T03:00:56.435962+0000","last_active":"2024-09-12T03:02:04.287450+0000","last_peered":"2024-09-12T03:02:04.287450+0000","last_clean":"2024-09-12T03:02:04.287450+0000","last_became_active":"2024-09-12T03:00:56.435515+0000","last_became_peered":"2024-09-12T03:00:56.435515+0000","last_unstale":"2024-09-12T03:02:04.287450+0000","last_undegraded":"2024-09-12T03:02:04.287450+0000","last_fullsized":"2024-09-12T03:02:04.287450+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:32.953436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.135045+0000","last_change":"2024-09-12T03:00:58.344239+0000","last_active":"2024-09-12T03:02:33.135045+0000","last_peered":"2024-09-12T03:02:33.135045+0000","last_clean":"2024-09-12T03:02:33.135045+0000","last_became_active":"2024-09-12T03:00:58.343876+0000","last_became_peered":"2024-09-12T03:00:58.343876+0000","last_unstale":"2024-09-12T03:02:33.135045+0000","last_undegraded":"2024-09-12T03:02:33.135045+0000","last_fullsized":"2024-09-12T03:02:33.135045+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:49.401295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'34","reported_seq":282,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:55.643246+0000","last_change":"2024-09-12T03:00:56.317708+0000","last_active":"2024-09-12T03:02:55.643246+0000","last_peered":"2024-09-12T03:02:55.643246+0000","last_clean":"2024-09-12T03:02:55.643246+0000","last_became_active":"2024-09-12T03:00:56.317317+0000","last_became_peered":"2024-09-12T03:00:56.317317+0000","last_unstale":"2024-09-12T03:02:55.643246+0000","last_undegraded":"2024-09-12T03:02:55.643246+0000","last_fullsized":"2024-09-12T03:02:55.643246+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:47.788186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62309,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":226,"num_read_kb":173,"num_write":35,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.123764+0000","last_change":"2024-09-12T03:00:58.343602+0000","last_active":"2024-09-12T03:02:33.123764+0000","last_peered":"2024-09-12T03:02:33.123764+0000","last_clean":"2024-09-12T03:02:33.123764+0000","last_became_active":"2024-09-12T03:00:58.343301+0000","last_became_peered":"2024-09-12T03:00:58.343301+0000","last_unstale":"2024-09-12T03:02:33.123764+0000","last_undegraded":"2024-09-12T03:02:33.123764+0000","last_fullsized":"2024-09-12T03:02:33.123764+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:04.056118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.290204+0000","last_change":"2024-09-12T03:00:56.317874+0000","last_active":"2024-09-12T03:02:04.290204+0000","last_peered":"2024-09-12T03:02:04.290204+0000","last_clean":"2024-09-12T03:02:04.290204+0000","last_became_active":"2024-09-12T03:00:56.317566+0000","last_became_peered":"2024-09-12T03:00:56.317566+0000","last_unstale":"2024-09-12T03:02:04.290204+0000","last_undegraded":"2024-09-12T03:02:04.290204+0000","last_fullsized":"2024-09-12T03:02:04.290204+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:54.203730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.121582+0000","last_change":"2024-09-12T03:00:58.351539+0000","last_active":"2024-09-12T03:02:33.121582+0000","last_peered":"2024-09-12T03:02:33.121582+0000","last_clean":"2024-09-12T03:02:33.121582+0000","last_became_active":"2024-09-12T03:00:58.351249+0000","last_became_peered":"2024-09-12T03:00:58.351249+0000","last_unstale":"2024-09-12T03:02:33.121582+0000","last_undegraded":"2024-09-12T03:02:33.121582+0000","last_fullsized":"2024-09-12T03:02:33.121582+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:40.498867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.287068+0000","last_change":"2024-09-12T03:00:56.314489+0000","last_active":"2024-09-12T03:02:04.287068+0000","last_peered":"2024-09-12T03:02:04.287068+0000","last_clean":"2024-09-12T03:02:04.287068+0000","last_became_active":"2024-09-12T03:00:56.314142+0000","last_became_peered":"2024-09-12T03:00:56.314142+0000","last_unstale":"2024-09-12T03:02:04.287068+0000","last_undegraded":"2024-09-12T03:02:04.287068+0000","last_fullsized":"2024-09-12T03:02:04.287068+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:21.541713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.123797+0000","last_change":"2024-09-12T03:00:58.348752+0000","last_active":"2024-09-12T03:02:33.123797+0000","last_peered":"2024-09-12T03:02:33.123797+0000","last_clean":"2024-09-12T03:02:33.123797+0000","last_became_active":"2024-09-12T03:00:58.348391+0000","last_became_peered":"2024-09-12T03:00:58.348391+0000","last_unstale":"2024-09-12T03:02:33.123797+0000","last_undegraded":"2024-09-12T03:02:33.123797+0000","last_fullsized":"2024-09-12T03:02:33.123797+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:33:15.626306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'3","reported_seq":134,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:53.556390+0000","last_change":"2024-09-12T03:00:56.311702+0000","last_active":"2024-09-12T03:02:53.556390+0000","last_peered":"2024-09-12T03:02:53.556390+0000","last_clean":"2024-09-12T03:02:53.556390+0000","last_became_active":"2024-09-12T03:00:56.311353+0000","last_became_peered":"2024-09-12T03:00:56.311353+0000","last_unstale":"2024-09-12T03:02:53.556390+0000","last_undegraded":"2024-09-12T03:02:53.556390+0000","last_fullsized":"2024-09-12T03:02:53.556390+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:01.004858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.123076+0000","last_change":"2024-09-12T03:00:58.351973+0000","last_active":"2024-09-12T03:02:33.123076+0000","last_peered":"2024-09-12T03:02:33.123076+0000","last_clean":"2024-09-12T03:02:33.123076+0000","last_became_active":"2024-09-12T03:00:58.351599+0000","last_became_peered":"2024-09-12T03:00:58.351599+0000","last_unstale":"2024-09-12T03:02:33.123076+0000","last_undegraded":"2024-09-12T03:02:33.123076+0000","last_fullsized":"2024-09-12T03:02:33.123076+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:26.398828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.291749+0000","last_change":"2024-09-12T03:00:56.436185+0000","last_active":"2024-09-12T03:02:04.291749+0000","last_peered":"2024-09-12T03:02:04.291749+0000","last_clean":"2024-09-12T03:02:04.291749+0000","last_became_active":"2024-09-12T03:00:56.435717+0000","last_became_peered":"2024-09-12T03:00:56.435717+0000","last_unstale":"2024-09-12T03:02:04.291749+0000","last_undegraded":"2024-09-12T03:02:04.291749+0000","last_fullsized":"2024-09-12T03:02:04.291749+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:27.303182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.119766+0000","last_change":"2024-09-12T03:02:04.333661+0000","last_active":"2024-09-12T03:02:33.119766+0000","last_peered":"2024-09-12T03:02:33.119766+0000","last_clean":"2024-09-12T03:02:33.119766+0000","last_became_active":"2024-09-12T03:02:04.333217+0000","last_became_peered":"2024-09-12T03:02:04.333217+0000","last_unstale":"2024-09-12T03:02:33.119766+0000","last_undegraded":"2024-09-12T03:02:33.119766+0000","last_fullsized":"2024-09-12T03:02:33.119766+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:23:47.212734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.297341+0000","last_change":"2024-09-12T03:02:04.296884+0000","last_active":"2024-09-12T03:02:04.297341+0000","last_peered":"2024-09-12T03:02:04.297341+0000","last_clean":"2024-09-12T03:02:04.297341+0000","last_became_active":"2024-09-12T03:02:04.290894+0000","last_became_peered":"2024-09-12T03:02:04.290894+0000","last_unstale":"2024-09-12T03:02:04.297341+0000","last_undegraded":"2024-09-12T03:02:04.297341+0000","last_fullsized":"2024-09-12T03:02:04.297341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:49:19.749568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.118228+0000","last_change":"2024-09-12T03:02:04.334096+0000","last_active":"2024-09-12T03:02:33.118228+0000","last_peered":"2024-09-12T03:02:33.118228+0000","last_clean":"2024-09-12T03:02:33.118228+0000","last_became_active":"2024-09-12T03:02:04.333641+0000","last_became_peered":"2024-09-12T03:02:04.333641+0000","last_unstale":"2024-09-12T03:02:33.118228+0000","last_undegraded":"2024-09-12T03:02:33.118228+0000","last_fullsized":"2024-09-12T03:02:33.118228+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:24.085293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332966+0000","last_change":"2024-09-12T03:00:56.314182+0000","last_active":"2024-09-12T03:02:04.332966+0000","last_peered":"2024-09-12T03:02:04.332966+0000","last_clean":"2024-09-12T03:02:04.332966+0000","last_became_active":"2024-09-12T03:00:56.313749+0000","last_became_peered":"2024-09-12T03:00:56.313749+0000","last_unstale":"2024-09-12T03:02:04.332966+0000","last_undegraded":"2024-09-12T03:02:04.332966+0000","last_fullsized":"2024-09-12T03:02:04.332966+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:31:45.767500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.125121+0000","last_change":"2024-09-12T03:00:58.344356+0000","last_active":"2024-09-12T03:02:33.125121+0000","last_peered":"2024-09-12T03:02:33.125121+0000","last_clean":"2024-09-12T03:02:33.125121+0000","last_became_active":"2024-09-12T03:00:58.344035+0000","last_became_peered":"2024-09-12T03:00:58.344035+0000","last_unstale":"2024-09-12T03:02:33.125121+0000","last_undegraded":"2024-09-12T03:02:33.125121+0000","last_fullsized":"2024-09-12T03:02:33.125121+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:45:18.681018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.287585+0000","last_change":"2024-09-12T03:00:56.315653+0000","last_active":"2024-09-12T03:02:04.287585+0000","last_peered":"2024-09-12T03:02:04.287585+0000","last_clean":"2024-09-12T03:02:04.287585+0000","last_became_active":"2024-09-12T03:00:56.315284+0000","last_became_peered":"2024-09-12T03:00:56.315284+0000","last_unstale":"2024-09-12T03:02:04.287585+0000","last_undegraded":"2024-09-12T03:02:04.287585+0000","last_fullsized":"2024-09-12T03:02:04.287585+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:42.934020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.122059+0000","last_change":"2024-09-12T03:02:04.291128+0000","last_active":"2024-09-12T03:02:33.122059+0000","last_peered":"2024-09-12T03:02:33.122059+0000","last_clean":"2024-09-12T03:02:33.122059+0000","last_became_active":"2024-09-12T03:02:04.287836+0000","last_became_peered":"2024-09-12T03:02:04.287836+0000","last_unstale":"2024-09-12T03:02:33.122059+0000","last_undegraded":"2024-09-12T03:02:33.122059+0000","last_fullsized":"2024-09-12T03:02:33.122059+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:23.956579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.290577+0000","last_change":"2024-09-12T03:00:56.746780+0000","last_active":"2024-09-12T03:02:04.290577+0000","last_peered":"2024-09-12T03:02:04.290577+0000","last_clean":"2024-09-12T03:02:04.290577+0000","last_became_active":"2024-09-12T03:00:56.746414+0000","last_became_peered":"2024-09-12T03:00:56.746414+0000","last_unstale":"2024-09-12T03:02:04.290577+0000","last_undegraded":"2024-09-12T03:02:04.290577+0000","last_fullsized":"2024-09-12T03:02:04.290577+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:15:35.214196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.131100+0000","last_change":"2024-09-12T03:00:58.351713+0000","last_active":"2024-09-12T03:02:33.131100+0000","last_peered":"2024-09-12T03:02:33.131100+0000","last_clean":"2024-09-12T03:02:33.131100+0000","last_became_active":"2024-09-12T03:00:58.351371+0000","last_became_peered":"2024-09-12T03:00:58.351371+0000","last_unstale":"2024-09-12T03:02:33.131100+0000","last_undegraded":"2024-09-12T03:02:33.131100+0000","last_fullsized":"2024-09-12T03:02:33.131100+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:13.346956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'137","reported_seq":336,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284318+0000","last_change":"2024-09-12T02:59:45.461482+0000","last_active":"2024-09-12T03:02:04.284318+0000","last_peered":"2024-09-12T03:02:04.284318+0000","last_clean":"2024-09-12T03:02:04.284318+0000","last_became_active":"2024-09-12T02:59:45.461058+0000","last_became_peered":"2024-09-12T02:59:45.461058+0000","last_unstale":"2024-09-12T03:02:04.284318+0000","last_undegraded":"2024-09-12T03:02:04.284318+0000","last_fullsized":"2024-09-12T03:02:04.284318+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:18.034438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:18.034438+0000","last_clean_scrub_stamp":"2024-09-12T02:56:18.034438+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:09.251858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289291+0000","last_change":"2024-09-12T03:00:56.315245+0000","last_active":"2024-09-12T03:02:04.289291+0000","last_peered":"2024-09-12T03:02:04.289291+0000","last_clean":"2024-09-12T03:02:04.289291+0000","last_became_active":"2024-09-12T03:00:56.314887+0000","last_became_peered":"2024-09-12T03:00:56.314887+0000","last_unstale":"2024-09-12T03:02:04.289291+0000","last_undegraded":"2024-09-12T03:02:04.289291+0000","last_fullsized":"2024-09-12T03:02:04.289291+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:48.750657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.112960+0000","last_change":"2024-09-12T03:00:58.357874+0000","last_active":"2024-09-12T03:02:33.112960+0000","last_peered":"2024-09-12T03:02:33.112960+0000","last_clean":"2024-09-12T03:02:33.112960+0000","last_became_active":"2024-09-12T03:00:58.357575+0000","last_became_peered":"2024-09-12T03:00:58.357575+0000","last_unstale":"2024-09-12T03:02:33.112960+0000","last_undegraded":"2024-09-12T03:02:33.112960+0000","last_fullsized":"2024-09-12T03:02:33.112960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:46.125620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284212+0000","last_change":"2024-09-12T03:00:56.312263+0000","last_active":"2024-09-12T03:02:04.284212+0000","last_peered":"2024-09-12T03:02:04.284212+0000","last_clean":"2024-09-12T03:02:04.284212+0000","last_became_active":"2024-09-12T03:00:56.311894+0000","last_became_peered":"2024-09-12T03:00:56.311894+0000","last_unstale":"2024-09-12T03:02:04.284212+0000","last_undegraded":"2024-09-12T03:02:04.284212+0000","last_fullsized":"2024-09-12T03:02:04.284212+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:52.434203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.110253+0000","last_change":"2024-09-12T03:00:58.350762+0000","last_active":"2024-09-12T03:02:33.110253+0000","last_peered":"2024-09-12T03:02:33.110253+0000","last_clean":"2024-09-12T03:02:33.110253+0000","last_became_active":"2024-09-12T03:00:58.348038+0000","last_became_peered":"2024-09-12T03:00:58.348038+0000","last_unstale":"2024-09-12T03:02:33.110253+0000","last_undegraded":"2024-09-12T03:02:33.110253+0000","last_fullsized":"2024-09-12T03:02:33.110253+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:08.916636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":134,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:54.557217+0000","last_change":"2024-09-12T03:00:56.435641+0000","last_active":"2024-09-12T03:02:54.557217+0000","last_peered":"2024-09-12T03:02:54.557217+0000","last_clean":"2024-09-12T03:02:54.557217+0000","last_became_active":"2024-09-12T03:00:56.435319+0000","last_became_peered":"2024-09-12T03:00:56.435319+0000","last_unstale":"2024-09-12T03:02:54.557217+0000","last_undegraded":"2024-09-12T03:02:54.557217+0000","last_fullsized":"2024-09-12T03:02:54.557217+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:56.894978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.112715+0000","last_change":"2024-09-12T03:00:58.344708+0000","last_active":"2024-09-12T03:02:33.112715+0000","last_peered":"2024-09-12T03:02:33.112715+0000","last_clean":"2024-09-12T03:02:33.112715+0000","last_became_active":"2024-09-12T03:00:58.344389+0000","last_became_peered":"2024-09-12T03:00:58.344389+0000","last_unstale":"2024-09-12T03:02:33.112715+0000","last_undegraded":"2024-09-12T03:02:33.112715+0000","last_fullsized":"2024-09-12T03:02:33.112715+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:08.160383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284317+0000","last_change":"2024-09-12T03:00:56.317068+0000","last_active":"2024-09-12T03:02:04.284317+0000","last_peered":"2024-09-12T03:02:04.284317+0000","last_clean":"2024-09-12T03:02:04.284317+0000","last_became_active":"2024-09-12T03:00:56.316698+0000","last_became_peered":"2024-09-12T03:00:56.316698+0000","last_unstale":"2024-09-12T03:02:04.284317+0000","last_undegraded":"2024-09-12T03:02:04.284317+0000","last_fullsized":"2024-09-12T03:02:04.284317+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:41:19.199418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.109469+0000","last_change":"2024-09-12T03:00:58.347814+0000","last_active":"2024-09-12T03:02:33.109469+0000","last_peered":"2024-09-12T03:02:33.109469+0000","last_clean":"2024-09-12T03:02:33.109469+0000","last_became_active":"2024-09-12T03:00:58.347112+0000","last_became_peered":"2024-09-12T03:00:58.347112+0000","last_unstale":"2024-09-12T03:02:33.109469+0000","last_undegraded":"2024-09-12T03:02:33.109469+0000","last_fullsized":"2024-09-12T03:02:33.109469+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:59.325113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284265+0000","last_change":"2024-09-12T03:00:56.435771+0000","last_active":"2024-09-12T03:02:04.284265+0000","last_peered":"2024-09-12T03:02:04.284265+0000","last_clean":"2024-09-12T03:02:04.284265+0000","last_became_active":"2024-09-12T03:00:56.435448+0000","last_became_peered":"2024-09-12T03:00:56.435448+0000","last_unstale":"2024-09-12T03:02:04.284265+0000","last_undegraded":"2024-09-12T03:02:04.284265+0000","last_fullsized":"2024-09-12T03:02:04.284265+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:33.650338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.117580+0000","last_change":"2024-09-12T03:00:58.341071+0000","last_active":"2024-09-12T03:02:33.117580+0000","last_peered":"2024-09-12T03:02:33.117580+0000","last_clean":"2024-09-12T03:02:33.117580+0000","last_became_active":"2024-09-12T03:00:58.340753+0000","last_became_peered":"2024-09-12T03:00:58.340753+0000","last_unstale":"2024-09-12T03:02:33.117580+0000","last_undegraded":"2024-09-12T03:02:33.117580+0000","last_fullsized":"2024-09-12T03:02:33.117580+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:21.217915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292254+0000","last_change":"2024-09-12T03:00:56.436058+0000","last_active":"2024-09-12T03:02:04.292254+0000","last_peered":"2024-09-12T03:02:04.292254+0000","last_clean":"2024-09-12T03:02:04.292254+0000","last_became_active":"2024-09-12T03:00:56.435518+0000","last_became_peered":"2024-09-12T03:00:56.435518+0000","last_unstale":"2024-09-12T03:02:04.292254+0000","last_undegraded":"2024-09-12T03:02:04.292254+0000","last_fullsized":"2024-09-12T03:02:04.292254+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:32.235097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.110650+0000","last_change":"2024-09-12T03:00:58.348236+0000","last_active":"2024-09-12T03:02:33.110650+0000","last_peered":"2024-09-12T03:02:33.110650+0000","last_clean":"2024-09-12T03:02:33.110650+0000","last_became_active":"2024-09-12T03:00:58.347727+0000","last_became_peered":"2024-09-12T03:00:58.347727+0000","last_unstale":"2024-09-12T03:02:33.110650+0000","last_undegraded":"2024-09-12T03:02:33.110650+0000","last_fullsized":"2024-09-12T03:02:33.110650+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:19.233163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'12","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.354650+0000","last_change":"2024-09-12T03:00:56.310537+0000","last_active":"2024-09-12T03:02:33.354650+0000","last_peered":"2024-09-12T03:02:33.354650+0000","last_clean":"2024-09-12T03:02:33.354650+0000","last_became_active":"2024-09-12T03:00:56.310135+0000","last_became_peered":"2024-09-12T03:00:56.310135+0000","last_unstale":"2024-09-12T03:02:33.354650+0000","last_undegraded":"2024-09-12T03:02:33.354650+0000","last_fullsized":"2024-09-12T03:02:33.354650+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:10:55.584339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.121993+0000","last_change":"2024-09-12T03:00:58.358417+0000","last_active":"2024-09-12T03:02:33.121993+0000","last_peered":"2024-09-12T03:02:33.121993+0000","last_clean":"2024-09-12T03:02:33.121993+0000","last_became_active":"2024-09-12T03:00:58.358119+0000","last_became_peered":"2024-09-12T03:00:58.358119+0000","last_unstale":"2024-09-12T03:02:33.121993+0000","last_undegraded":"2024-09-12T03:02:33.121993+0000","last_fullsized":"2024-09-12T03:02:33.121993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:10.897594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289526+0000","last_change":"2024-09-12T03:00:56.747044+0000","last_active":"2024-09-12T03:02:04.289526+0000","last_peered":"2024-09-12T03:02:04.289526+0000","last_clean":"2024-09-12T03:02:04.289526+0000","last_became_active":"2024-09-12T03:00:56.746699+0000","last_became_peered":"2024-09-12T03:00:56.746699+0000","last_unstale":"2024-09-12T03:02:04.289526+0000","last_undegraded":"2024-09-12T03:02:04.289526+0000","last_fullsized":"2024-09-12T03:02:04.289526+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:11.420109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.113058+0000","last_change":"2024-09-12T03:00:58.346854+0000","last_active":"2024-09-12T03:02:33.113058+0000","last_peered":"2024-09-12T03:02:33.113058+0000","last_clean":"2024-09-12T03:02:33.113058+0000","last_became_active":"2024-09-12T03:00:58.346535+0000","last_became_peered":"2024-09-12T03:00:58.346535+0000","last_unstale":"2024-09-12T03:02:33.113058+0000","last_undegraded":"2024-09-12T03:02:33.113058+0000","last_fullsized":"2024-09-12T03:02:33.113058+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:46.637914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332365+0000","last_change":"2024-09-12T03:00:56.435957+0000","last_active":"2024-09-12T03:02:04.332365+0000","last_peered":"2024-09-12T03:02:04.332365+0000","last_clean":"2024-09-12T03:02:04.332365+0000","last_became_active":"2024-09-12T03:00:56.435603+0000","last_became_peered":"2024-09-12T03:00:56.435603+0000","last_unstale":"2024-09-12T03:02:04.332365+0000","last_undegraded":"2024-09-12T03:02:04.332365+0000","last_fullsized":"2024-09-12T03:02:04.332365+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:34.553189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.089792+0000","last_change":"2024-09-12T03:00:58.337784+0000","last_active":"2024-09-12T03:02:33.089792+0000","last_peered":"2024-09-12T03:02:33.089792+0000","last_clean":"2024-09-12T03:02:33.089792+0000","last_became_active":"2024-09-12T03:00:58.337415+0000","last_became_peered":"2024-09-12T03:00:58.337415+0000","last_unstale":"2024-09-12T03:02:33.089792+0000","last_undegraded":"2024-09-12T03:02:33.089792+0000","last_fullsized":"2024-09-12T03:02:33.089792+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:38:15.557067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282354+0000","last_change":"2024-09-12T03:00:56.315356+0000","last_active":"2024-09-12T03:02:04.282354+0000","last_peered":"2024-09-12T03:02:04.282354+0000","last_clean":"2024-09-12T03:02:04.282354+0000","last_became_active":"2024-09-12T03:00:56.314873+0000","last_became_peered":"2024-09-12T03:00:56.314873+0000","last_unstale":"2024-09-12T03:02:04.282354+0000","last_undegraded":"2024-09-12T03:02:04.282354+0000","last_fullsized":"2024-09-12T03:02:04.282354+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:55.710156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.118974+0000","last_change":"2024-09-12T03:00:58.346706+0000","last_active":"2024-09-12T03:02:33.118974+0000","last_peered":"2024-09-12T03:02:33.118974+0000","last_clean":"2024-09-12T03:02:33.118974+0000","last_became_active":"2024-09-12T03:00:58.346363+0000","last_became_peered":"2024-09-12T03:00:58.346363+0000","last_unstale":"2024-09-12T03:02:33.118974+0000","last_undegraded":"2024-09-12T03:02:33.118974+0000","last_fullsized":"2024-09-12T03:02:33.118974+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:13.468639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.299582+0000","last_change":"2024-09-12T03:02:04.299147+0000","last_active":"2024-09-12T03:02:04.299582+0000","last_peered":"2024-09-12T03:02:04.299582+0000","last_clean":"2024-09-12T03:02:04.299582+0000","last_became_active":"2024-09-12T03:02:04.293211+0000","last_became_peered":"2024-09-12T03:02:04.293211+0000","last_unstale":"2024-09-12T03:02:04.299582+0000","last_undegraded":"2024-09-12T03:02:04.299582+0000","last_fullsized":"2024-09-12T03:02:04.299582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:01:39.222476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.118849+0000","last_change":"2024-09-12T03:00:58.356682+0000","last_active":"2024-09-12T03:02:33.118849+0000","last_peered":"2024-09-12T03:02:33.118849+0000","last_clean":"2024-09-12T03:02:33.118849+0000","last_became_active":"2024-09-12T03:00:58.356383+0000","last_became_peered":"2024-09-12T03:00:58.356383+0000","last_unstale":"2024-09-12T03:02:33.118849+0000","last_undegraded":"2024-09-12T03:02:33.118849+0000","last_fullsized":"2024-09-12T03:02:33.118849+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:14:16.428393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292053+0000","last_change":"2024-09-12T03:00:56.314943+0000","last_active":"2024-09-12T03:02:04.292053+0000","last_peered":"2024-09-12T03:02:04.292053+0000","last_clean":"2024-09-12T03:02:04.292053+0000","last_became_active":"2024-09-12T03:00:56.314602+0000","last_became_peered":"2024-09-12T03:00:56.314602+0000","last_unstale":"2024-09-12T03:02:04.292053+0000","last_undegraded":"2024-09-12T03:02:04.292053+0000","last_fullsized":"2024-09-12T03:02:04.292053+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:31.207880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.131538+0000","last_change":"2024-09-12T03:00:58.351774+0000","last_active":"2024-09-12T03:02:33.131538+0000","last_peered":"2024-09-12T03:02:33.131538+0000","last_clean":"2024-09-12T03:02:33.131538+0000","last_became_active":"2024-09-12T03:00:58.351453+0000","last_became_peered":"2024-09-12T03:00:58.351453+0000","last_unstale":"2024-09-12T03:02:33.131538+0000","last_undegraded":"2024-09-12T03:02:33.131538+0000","last_fullsized":"2024-09-12T03:02:33.131538+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:18:24.260110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.130930+0000","last_change":"2024-09-12T03:00:58.346886+0000","last_active":"2024-09-12T03:02:33.130930+0000","last_peered":"2024-09-12T03:02:33.130930+0000","last_clean":"2024-09-12T03:02:33.130930+0000","last_became_active":"2024-09-12T03:00:58.346499+0000","last_became_peered":"2024-09-12T03:00:58.346499+0000","last_unstale":"2024-09-12T03:02:33.130930+0000","last_undegraded":"2024-09-12T03:02:33.130930+0000","last_fullsized":"2024-09-12T03:02:33.130930+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:02:24.045610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292497+0000","last_change":"2024-09-12T03:00:56.317443+0000","last_active":"2024-09-12T03:02:04.292497+0000","last_peered":"2024-09-12T03:02:04.292497+0000","last_clean":"2024-09-12T03:02:04.292497+0000","last_became_active":"2024-09-12T03:00:56.317045+0000","last_became_peered":"2024-09-12T03:00:56.317045+0000","last_unstale":"2024-09-12T03:02:04.292497+0000","last_undegraded":"2024-09-12T03:02:04.292497+0000","last_fullsized":"2024-09-12T03:02:04.292497+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:07.687606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.132782+0000","last_change":"2024-09-12T03:00:58.337257+0000","last_active":"2024-09-12T03:02:33.132782+0000","last_peered":"2024-09-12T03:02:33.132782+0000","last_clean":"2024-09-12T03:02:33.132782+0000","last_became_active":"2024-09-12T03:00:58.336847+0000","last_became_peered":"2024-09-12T03:00:58.336847+0000","last_unstale":"2024-09-12T03:02:33.132782+0000","last_undegraded":"2024-09-12T03:02:33.132782+0000","last_fullsized":"2024-09-12T03:02:33.132782+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:22.846206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292165+0000","last_change":"2024-09-12T03:00:56.316255+0000","last_active":"2024-09-12T03:02:04.292165+0000","last_peered":"2024-09-12T03:02:04.292165+0000","last_clean":"2024-09-12T03:02:04.292165+0000","last_became_active":"2024-09-12T03:00:56.315906+0000","last_became_peered":"2024-09-12T03:00:56.315906+0000","last_unstale":"2024-09-12T03:02:04.292165+0000","last_undegraded":"2024-09-12T03:02:04.292165+0000","last_fullsized":"2024-09-12T03:02:04.292165+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:14.798601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120273+0000","last_change":"2024-09-12T03:00:58.347206+0000","last_active":"2024-09-12T03:02:33.120273+0000","last_peered":"2024-09-12T03:02:33.120273+0000","last_clean":"2024-09-12T03:02:33.120273+0000","last_became_active":"2024-09-12T03:00:58.346923+0000","last_became_peered":"2024-09-12T03:00:58.346923+0000","last_unstale":"2024-09-12T03:02:33.120273+0000","last_undegraded":"2024-09-12T03:02:33.120273+0000","last_fullsized":"2024-09-12T03:02:33.120273+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:41.376211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.283743+0000","last_change":"2024-09-12T03:00:56.435431+0000","last_active":"2024-09-12T03:02:04.283743+0000","last_peered":"2024-09-12T03:02:04.283743+0000","last_clean":"2024-09-12T03:02:04.283743+0000","last_became_active":"2024-09-12T03:00:56.435088+0000","last_became_peered":"2024-09-12T03:00:56.435088+0000","last_unstale":"2024-09-12T03:02:04.283743+0000","last_undegraded":"2024-09-12T03:02:04.283743+0000","last_fullsized":"2024-09-12T03:02:04.283743+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:24.738340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.114461+0000","last_change":"2024-09-12T03:00:58.351737+0000","last_active":"2024-09-12T03:02:33.114461+0000","last_peered":"2024-09-12T03:02:33.114461+0000","last_clean":"2024-09-12T03:02:33.114461+0000","last_became_active":"2024-09-12T03:00:58.351407+0000","last_became_peered":"2024-09-12T03:00:58.351407+0000","last_unstale":"2024-09-12T03:02:33.114461+0000","last_undegraded":"2024-09-12T03:02:33.114461+0000","last_fullsized":"2024-09-12T03:02:33.114461+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:34.206703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284979+0000","last_change":"2024-09-12T03:00:56.310910+0000","last_active":"2024-09-12T03:02:04.284979+0000","last_peered":"2024-09-12T03:02:04.284979+0000","last_clean":"2024-09-12T03:02:04.284979+0000","last_became_active":"2024-09-12T03:00:56.310657+0000","last_became_peered":"2024-09-12T03:00:56.310657+0000","last_unstale":"2024-09-12T03:02:04.284979+0000","last_undegraded":"2024-09-12T03:02:04.284979+0000","last_fullsized":"2024-09-12T03:02:04.284979+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:06.117215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65919,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":672,"num_read_kb":406,"num_write":170,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":438272,"data_stored":209289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245899,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645350,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":431835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906986,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":1085603,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070512,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":432166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299445,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":431947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528377,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":1023508,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":40,"seq":171798691903,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":880640,"data_stored":431438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":33,"seq":141733920835,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":1023192,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051657,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":431875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280590,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":431632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476819,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":494890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607641,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":499306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":432709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":63887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:03:00.869 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:03:01.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.002+0000 7fc0f4ce4640 1 -- 172.21.15.116:0/3516416749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0f0102d30 msgr2=0x7fc0f01031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:01.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.002+0000 7fc0f4ce4640 1 --2- 172.21.15.116:0/3516416749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0f0102d30 0x7fc0f01031b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8009900 tx=0x7fc0d802a410 comp rx=0 tx=0).stop 2024-09-12T03:03:01.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.002+0000 7fc0f4ce4640 1 -- 172.21.15.116:0/3516416749 shutdown_connections 2024-09-12T03:03:01.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.002+0000 7fc0f4ce4640 1 --2- 172.21.15.116:0/3516416749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0f01036f0 0x7fc0f0109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:01.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.002+0000 7fc0f4ce4640 1 --2- 172.21.15.116:0/3516416749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0f0102d30 0x7fc0f01031b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:01.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.002+0000 7fc0f4ce4640 1 --2- 172.21.15.116:0/3516416749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0f0101b40 0x7fc0f0101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:01.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.002+0000 7fc0f4ce4640 1 -- 172.21.15.116:0/3516416749 >> 172.21.15.116:0/3516416749 conn(0x7fc0f00fd2e0 msgr2=0x7fc0f00ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:01.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.002+0000 7fc0f4ce4640 1 -- 172.21.15.116:0/3516416749 shutdown_connections 2024-09-12T03:03:01.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.002+0000 7fc0f4ce4640 1 -- 172.21.15.116:0/3516416749 wait complete. 2024-09-12T03:03:01.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.003+0000 7fc0f4ce4640 1 Processor -- start 2024-09-12T03:03:01.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.003+0000 7fc0f4ce4640 1 -- start start 2024-09-12T03:03:01.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.003+0000 7fc0f4ce4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0f0101b40 0x7fc0f0075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:01.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.003+0000 7fc0f4ce4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0f0102d30 0x7fc0f00745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:01.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.003+0000 7fc0f4ce4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0f01036f0 0x7fc0f0074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:01.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.003+0000 7fc0f4ce4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f010d7a0 con 0x7fc0f01036f0 2024-09-12T03:03:01.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.003+0000 7fc0f4ce4640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f010d620 con 0x7fc0f0101b40 2024-09-12T03:03:01.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.003+0000 7fc0f4ce4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f010d920 con 0x7fc0f0102d30 2024-09-12T03:03:01.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.004+0000 7fc0eed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0f01036f0 0x7fc0f0074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:01.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.004+0000 7fc0eed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0f01036f0 0x7fc0f0074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39120/0 (socket says 172.21.15.116:39120) 2024-09-12T03:03:01.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.004+0000 7fc0eed9d640 1 -- 172.21.15.116:0/1240676459 learned_addr learned my addr 172.21.15.116:0/1240676459 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:01.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.004+0000 7fc0e5d9b640 1 --2- 172.21.15.116:0/1240676459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0f0102d30 0x7fc0f00745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:01.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.004+0000 7fc0ee59c640 1 --2- 172.21.15.116:0/1240676459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0f0101b40 0x7fc0f0075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:01.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.004+0000 7fc0eed9d640 1 -- 172.21.15.116:0/1240676459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0f0102d30 msgr2=0x7fc0f00745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:01.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.004+0000 7fc0eed9d640 1 --2- 172.21.15.116:0/1240676459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0f0102d30 0x7fc0f00745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:01.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.004+0000 7fc0eed9d640 1 -- 172.21.15.116:0/1240676459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0f0101b40 msgr2=0x7fc0f0075fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:01.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.004+0000 7fc0eed9d640 1 --2- 172.21.15.116:0/1240676459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0f0101b40 0x7fc0f0075fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:01.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.004+0000 7fc0eed9d640 1 -- 172.21.15.116:0/1240676459 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f00753b0 con 0x7fc0f01036f0 2024-09-12T03:03:01.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.004+0000 7fc0eed9d640 1 --2- 172.21.15.116:0/1240676459 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0f01036f0 0x7fc0f0074b20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc0e800cad0 tx=0x7fc0e800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:01.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.004+0000 7fc0e7fff640 1 -- 172.21.15.116:0/1240676459 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e800ae20 con 0x7fc0f01036f0 2024-09-12T03:03:01.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.004+0000 7fc0e7fff640 1 -- 172.21.15.116:0/1240676459 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0e8005020 con 0x7fc0f01036f0 2024-09-12T03:03:01.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.004+0000 7fc0f4ce4640 1 -- 172.21.15.116:0/1240676459 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0f01a1f70 con 0x7fc0f01036f0 2024-09-12T03:03:01.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.004+0000 7fc0e7fff640 1 -- 172.21.15.116:0/1240676459 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e800f6f0 con 0x7fc0f01036f0 2024-09-12T03:03:01.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.004+0000 7fc0f4ce4640 1 -- 172.21.15.116:0/1240676459 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0f01a2380 con 0x7fc0f01036f0 2024-09-12T03:03:01.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.005+0000 7fc0f4ce4640 1 -- 172.21.15.116:0/1240676459 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0f0108fa0 con 0x7fc0f01036f0 2024-09-12T03:03:01.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.006+0000 7fc0e7fff640 1 -- 172.21.15.116:0/1240676459 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fc0e80051d0 con 0x7fc0f01036f0 2024-09-12T03:03:01.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.007+0000 7fc0e7fff640 1 --2- 172.21.15.116:0/1240676459 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc0dc0a2c20 0x7fc0dc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:01.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.007+0000 7fc0ee59c640 1 --2- 172.21.15.116:0/1240676459 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc0dc0a2c20 0x7fc0dc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:01.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.007+0000 7fc0e7fff640 1 -- 172.21.15.116:0/1240676459 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fc0e80cf280 con 0x7fc0f01036f0 2024-09-12T03:03:01.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.007+0000 7fc0ee59c640 1 --2- 172.21.15.116:0/1240676459 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc0dc0a2c20 0x7fc0dc0a50e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0005fd0 tx=0x7fc0e0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:01.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.011+0000 7fc0e7fff640 1 -- 172.21.15.116:0/1240676459 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc0e8014030 con 0x7fc0f01036f0 2024-09-12T03:03:01.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.174+0000 7fc0f4ce4640 1 -- 172.21.15.116:0/1240676459 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc0f0062630 con 0x7fc0dc0a2c20 2024-09-12T03:03:01.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.180+0000 7fc0e7fff640 1 -- 172.21.15.116:0/1240676459 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258980 (secure 0 0 0) 0x7fc0f0062630 con 0x7fc0dc0a2c20 2024-09-12T03:03:01.184 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:03:01.185 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-09-12T03:03:01.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.183+0000 7fc0f4ce4640 1 -- 172.21.15.116:0/1240676459 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc0dc0a2c20 msgr2=0x7fc0dc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:01.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.183+0000 7fc0f4ce4640 1 --2- 172.21.15.116:0/1240676459 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc0dc0a2c20 0x7fc0dc0a50e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0005fd0 tx=0x7fc0e0005850 comp rx=0 tx=0).stop 2024-09-12T03:03:01.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.183+0000 7fc0f4ce4640 1 -- 172.21.15.116:0/1240676459 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0f01036f0 msgr2=0x7fc0f0074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:01.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.183+0000 7fc0f4ce4640 1 --2- 172.21.15.116:0/1240676459 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0f01036f0 0x7fc0f0074b20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc0e800cad0 tx=0x7fc0e800cf90 comp rx=0 tx=0).stop 2024-09-12T03:03:01.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.183+0000 7fc0f4ce4640 1 -- 172.21.15.116:0/1240676459 shutdown_connections 2024-09-12T03:03:01.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.183+0000 7fc0f4ce4640 1 --2- 172.21.15.116:0/1240676459 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc0dc0a2c20 0x7fc0dc0a50e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:01.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.183+0000 7fc0f4ce4640 1 --2- 172.21.15.116:0/1240676459 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0f01036f0 0x7fc0f0074b20 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:01.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.183+0000 7fc0f4ce4640 1 --2- 172.21.15.116:0/1240676459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0f0102d30 0x7fc0f00745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:01.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.183+0000 7fc0f4ce4640 1 --2- 172.21.15.116:0/1240676459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0f0101b40 0x7fc0f0075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:01.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.183+0000 7fc0f4ce4640 1 -- 172.21.15.116:0/1240676459 >> 172.21.15.116:0/1240676459 conn(0x7fc0f00fd2e0 msgr2=0x7fc0f0106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:01.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.184+0000 7fc0f4ce4640 1 -- 172.21.15.116:0/1240676459 shutdown_connections 2024-09-12T03:03:01.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.184+0000 7fc0f4ce4640 1 -- 172.21.15.116:0/1240676459 wait complete. 2024-09-12T03:03:01.194 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-09-12T03:03:01.136507+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665200,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":827,"num_read_kb":536,"num_write":356,"num_write_kb":3066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180875,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":786,"ondisk_log_size":786,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368392,"kb_used_data":10824,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573432,"statfs":{"total":1151940427776,"available":1151563194368,"internally_reserved":0,"allocated":11083776,"data_stored":7159566,"data_compressed":107490,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8987,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":51,"num_write":7,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003020"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.133420+0000","last_change":"2024-09-12T03:00:58.348563+0000","last_active":"2024-09-12T03:02:33.133420+0000","last_peered":"2024-09-12T03:02:33.133420+0000","last_clean":"2024-09-12T03:02:33.133420+0000","last_became_active":"2024-09-12T03:00:58.347940+0000","last_became_peered":"2024-09-12T03:00:58.347940+0000","last_unstale":"2024-09-12T03:02:33.133420+0000","last_undegraded":"2024-09-12T03:02:33.133420+0000","last_fullsized":"2024-09-12T03:02:33.133420+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:43.489238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.119217+0000","last_change":"2024-09-12T03:00:58.351969+0000","last_active":"2024-09-12T03:02:33.119217+0000","last_peered":"2024-09-12T03:02:33.119217+0000","last_clean":"2024-09-12T03:02:33.119217+0000","last_became_active":"2024-09-12T03:00:58.351679+0000","last_became_peered":"2024-09-12T03:00:58.351679+0000","last_unstale":"2024-09-12T03:02:33.119217+0000","last_undegraded":"2024-09-12T03:02:33.119217+0000","last_fullsized":"2024-09-12T03:02:33.119217+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:27:01.174890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120115+0000","last_change":"2024-09-12T03:00:58.346045+0000","last_active":"2024-09-12T03:02:33.120115+0000","last_peered":"2024-09-12T03:02:33.120115+0000","last_clean":"2024-09-12T03:02:33.120115+0000","last_became_active":"2024-09-12T03:00:58.345719+0000","last_became_peered":"2024-09-12T03:00:58.345719+0000","last_unstale":"2024-09-12T03:02:33.120115+0000","last_undegraded":"2024-09-12T03:02:33.120115+0000","last_fullsized":"2024-09-12T03:02:33.120115+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:12.312650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.111069+0000","last_change":"2024-09-12T03:00:58.348165+0000","last_active":"2024-09-12T03:02:33.111069+0000","last_peered":"2024-09-12T03:02:33.111069+0000","last_clean":"2024-09-12T03:02:33.111069+0000","last_became_active":"2024-09-12T03:00:58.347734+0000","last_became_peered":"2024-09-12T03:00:58.347734+0000","last_unstale":"2024-09-12T03:02:33.111069+0000","last_undegraded":"2024-09-12T03:02:33.111069+0000","last_fullsized":"2024-09-12T03:02:33.111069+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:05.945900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.108519+0000","last_change":"2024-09-12T03:00:58.339316+0000","last_active":"2024-09-12T03:02:33.108519+0000","last_peered":"2024-09-12T03:02:33.108519+0000","last_clean":"2024-09-12T03:02:33.108519+0000","last_became_active":"2024-09-12T03:00:58.338964+0000","last_became_peered":"2024-09-12T03:00:58.338964+0000","last_unstale":"2024-09-12T03:02:33.108519+0000","last_undegraded":"2024-09-12T03:02:33.108519+0000","last_fullsized":"2024-09-12T03:02:33.108519+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:50.721276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.110921+0000","last_change":"2024-09-12T03:00:58.346838+0000","last_active":"2024-09-12T03:02:33.110921+0000","last_peered":"2024-09-12T03:02:33.110921+0000","last_clean":"2024-09-12T03:02:33.110921+0000","last_became_active":"2024-09-12T03:00:58.346486+0000","last_became_peered":"2024-09-12T03:00:58.346486+0000","last_unstale":"2024-09-12T03:02:33.110921+0000","last_undegraded":"2024-09-12T03:02:33.110921+0000","last_fullsized":"2024-09-12T03:02:33.110921+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:28.687727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.122919+0000","last_change":"2024-09-12T03:00:58.346712+0000","last_active":"2024-09-12T03:02:33.122919+0000","last_peered":"2024-09-12T03:02:33.122919+0000","last_clean":"2024-09-12T03:02:33.122919+0000","last_became_active":"2024-09-12T03:00:58.346397+0000","last_became_peered":"2024-09-12T03:00:58.346397+0000","last_unstale":"2024-09-12T03:02:33.122919+0000","last_undegraded":"2024-09-12T03:02:33.122919+0000","last_fullsized":"2024-09-12T03:02:33.122919+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:48:00.624733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.118885+0000","last_change":"2024-09-12T03:00:58.356166+0000","last_active":"2024-09-12T03:02:33.118885+0000","last_peered":"2024-09-12T03:02:33.118885+0000","last_clean":"2024-09-12T03:02:33.118885+0000","last_became_active":"2024-09-12T03:00:58.354944+0000","last_became_peered":"2024-09-12T03:00:58.354944+0000","last_unstale":"2024-09-12T03:02:33.118885+0000","last_undegraded":"2024-09-12T03:02:33.118885+0000","last_fullsized":"2024-09-12T03:02:33.118885+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:15.840149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.133064+0000","last_change":"2024-09-12T03:00:58.340182+0000","last_active":"2024-09-12T03:02:33.133064+0000","last_peered":"2024-09-12T03:02:33.133064+0000","last_clean":"2024-09-12T03:02:33.133064+0000","last_became_active":"2024-09-12T03:00:58.339576+0000","last_became_peered":"2024-09-12T03:00:58.339576+0000","last_unstale":"2024-09-12T03:02:33.133064+0000","last_undegraded":"2024-09-12T03:02:33.133064+0000","last_fullsized":"2024-09-12T03:02:33.133064+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:16.534844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.117254+0000","last_change":"2024-09-12T03:00:58.351218+0000","last_active":"2024-09-12T03:02:33.117254+0000","last_peered":"2024-09-12T03:02:33.117254+0000","last_clean":"2024-09-12T03:02:33.117254+0000","last_became_active":"2024-09-12T03:00:58.350871+0000","last_became_peered":"2024-09-12T03:00:58.350871+0000","last_unstale":"2024-09-12T03:02:33.117254+0000","last_undegraded":"2024-09-12T03:02:33.117254+0000","last_fullsized":"2024-09-12T03:02:33.117254+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:38.367683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.130892+0000","last_change":"2024-09-12T03:00:58.351894+0000","last_active":"2024-09-12T03:02:33.130892+0000","last_peered":"2024-09-12T03:02:33.130892+0000","last_clean":"2024-09-12T03:02:33.130892+0000","last_became_active":"2024-09-12T03:00:58.351445+0000","last_became_peered":"2024-09-12T03:00:58.351445+0000","last_unstale":"2024-09-12T03:02:33.130892+0000","last_undegraded":"2024-09-12T03:02:33.130892+0000","last_fullsized":"2024-09-12T03:02:33.130892+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:42.093871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.107323+0000","last_change":"2024-09-12T03:00:58.353460+0000","last_active":"2024-09-12T03:02:33.107323+0000","last_peered":"2024-09-12T03:02:33.107323+0000","last_clean":"2024-09-12T03:02:33.107323+0000","last_became_active":"2024-09-12T03:00:58.353111+0000","last_became_peered":"2024-09-12T03:00:58.353111+0000","last_unstale":"2024-09-12T03:02:33.107323+0000","last_undegraded":"2024-09-12T03:02:33.107323+0000","last_fullsized":"2024-09-12T03:02:33.107323+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:15.916765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.111860+0000","last_change":"2024-09-12T03:00:58.341080+0000","last_active":"2024-09-12T03:02:33.111860+0000","last_peered":"2024-09-12T03:02:33.111860+0000","last_clean":"2024-09-12T03:02:33.111860+0000","last_became_active":"2024-09-12T03:00:58.340782+0000","last_became_peered":"2024-09-12T03:00:58.340782+0000","last_unstale":"2024-09-12T03:02:33.111860+0000","last_undegraded":"2024-09-12T03:02:33.111860+0000","last_fullsized":"2024-09-12T03:02:33.111860+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:36.569700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.106506+0000","last_change":"2024-09-12T03:00:58.346791+0000","last_active":"2024-09-12T03:02:33.106506+0000","last_peered":"2024-09-12T03:02:33.106506+0000","last_clean":"2024-09-12T03:02:33.106506+0000","last_became_active":"2024-09-12T03:00:58.346500+0000","last_became_peered":"2024-09-12T03:00:58.346500+0000","last_unstale":"2024-09-12T03:02:33.106506+0000","last_undegraded":"2024-09-12T03:02:33.106506+0000","last_fullsized":"2024-09-12T03:02:33.106506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:32.017589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.112821+0000","last_change":"2024-09-12T03:00:58.349522+0000","last_active":"2024-09-12T03:02:33.112821+0000","last_peered":"2024-09-12T03:02:33.112821+0000","last_clean":"2024-09-12T03:02:33.112821+0000","last_became_active":"2024-09-12T03:00:58.349246+0000","last_became_peered":"2024-09-12T03:00:58.349246+0000","last_unstale":"2024-09-12T03:02:33.112821+0000","last_undegraded":"2024-09-12T03:02:33.112821+0000","last_fullsized":"2024-09-12T03:02:33.112821+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:39.253919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.134146+0000","last_change":"2024-09-12T03:00:58.349909+0000","last_active":"2024-09-12T03:02:33.134146+0000","last_peered":"2024-09-12T03:02:33.134146+0000","last_clean":"2024-09-12T03:02:33.134146+0000","last_became_active":"2024-09-12T03:00:58.349590+0000","last_became_peered":"2024-09-12T03:00:58.349590+0000","last_unstale":"2024-09-12T03:02:33.134146+0000","last_undegraded":"2024-09-12T03:02:33.134146+0000","last_fullsized":"2024-09-12T03:02:33.134146+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:04.540709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.133565+0000","last_change":"2024-09-12T03:00:58.348284+0000","last_active":"2024-09-12T03:02:33.133565+0000","last_peered":"2024-09-12T03:02:33.133565+0000","last_clean":"2024-09-12T03:02:33.133565+0000","last_became_active":"2024-09-12T03:00:58.347465+0000","last_became_peered":"2024-09-12T03:00:58.347465+0000","last_unstale":"2024-09-12T03:02:33.133565+0000","last_undegraded":"2024-09-12T03:02:33.133565+0000","last_fullsized":"2024-09-12T03:02:33.133565+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:13.191899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.122657+0000","last_change":"2024-09-12T03:00:58.337740+0000","last_active":"2024-09-12T03:02:33.122657+0000","last_peered":"2024-09-12T03:02:33.122657+0000","last_clean":"2024-09-12T03:02:33.122657+0000","last_became_active":"2024-09-12T03:00:58.337393+0000","last_became_peered":"2024-09-12T03:00:58.337393+0000","last_unstale":"2024-09-12T03:02:33.122657+0000","last_undegraded":"2024-09-12T03:02:33.122657+0000","last_fullsized":"2024-09-12T03:02:33.122657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:28:56.339167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.111666+0000","last_change":"2024-09-12T03:00:58.345779+0000","last_active":"2024-09-12T03:02:33.111666+0000","last_peered":"2024-09-12T03:02:33.111666+0000","last_clean":"2024-09-12T03:02:33.111666+0000","last_became_active":"2024-09-12T03:00:58.345210+0000","last_became_peered":"2024-09-12T03:00:58.345210+0000","last_unstale":"2024-09-12T03:02:33.111666+0000","last_undegraded":"2024-09-12T03:02:33.111666+0000","last_fullsized":"2024-09-12T03:02:33.111666+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:06:58.271185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120653+0000","last_change":"2024-09-12T03:02:04.294381+0000","last_active":"2024-09-12T03:02:33.120653+0000","last_peered":"2024-09-12T03:02:33.120653+0000","last_clean":"2024-09-12T03:02:33.120653+0000","last_became_active":"2024-09-12T03:02:04.293795+0000","last_became_peered":"2024-09-12T03:02:04.293795+0000","last_unstale":"2024-09-12T03:02:33.120653+0000","last_undegraded":"2024-09-12T03:02:33.120653+0000","last_fullsized":"2024-09-12T03:02:33.120653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:06:58.271007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.124292+0000","last_change":"2024-09-12T03:00:58.349233+0000","last_active":"2024-09-12T03:02:33.124292+0000","last_peered":"2024-09-12T03:02:33.124292+0000","last_clean":"2024-09-12T03:02:33.124292+0000","last_became_active":"2024-09-12T03:00:58.348958+0000","last_became_peered":"2024-09-12T03:00:58.348958+0000","last_unstale":"2024-09-12T03:02:33.124292+0000","last_undegraded":"2024-09-12T03:02:33.124292+0000","last_fullsized":"2024-09-12T03:02:33.124292+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:32:10.271817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.134703+0000","last_change":"2024-09-12T03:00:58.358024+0000","last_active":"2024-09-12T03:02:33.134703+0000","last_peered":"2024-09-12T03:02:33.134703+0000","last_clean":"2024-09-12T03:02:33.134703+0000","last_became_active":"2024-09-12T03:00:58.357434+0000","last_became_peered":"2024-09-12T03:00:58.357434+0000","last_unstale":"2024-09-12T03:02:33.134703+0000","last_undegraded":"2024-09-12T03:02:33.134703+0000","last_fullsized":"2024-09-12T03:02:33.134703+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:01:20.119733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120095+0000","last_change":"2024-09-12T03:00:58.355063+0000","last_active":"2024-09-12T03:02:33.120095+0000","last_peered":"2024-09-12T03:02:33.120095+0000","last_clean":"2024-09-12T03:02:33.120095+0000","last_became_active":"2024-09-12T03:00:58.354760+0000","last_became_peered":"2024-09-12T03:00:58.354760+0000","last_unstale":"2024-09-12T03:02:33.120095+0000","last_undegraded":"2024-09-12T03:02:33.120095+0000","last_fullsized":"2024-09-12T03:02:33.120095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:13.464413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.112255+0000","last_change":"2024-09-12T03:00:58.341916+0000","last_active":"2024-09-12T03:02:33.112255+0000","last_peered":"2024-09-12T03:02:33.112255+0000","last_clean":"2024-09-12T03:02:33.112255+0000","last_became_active":"2024-09-12T03:00:58.340815+0000","last_became_peered":"2024-09-12T03:00:58.340815+0000","last_unstale":"2024-09-12T03:02:33.112255+0000","last_undegraded":"2024-09-12T03:02:33.112255+0000","last_fullsized":"2024-09-12T03:02:33.112255+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:13.777947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.117061+0000","last_change":"2024-09-12T03:00:58.357765+0000","last_active":"2024-09-12T03:02:33.117061+0000","last_peered":"2024-09-12T03:02:33.117061+0000","last_clean":"2024-09-12T03:02:33.117061+0000","last_became_active":"2024-09-12T03:00:58.357480+0000","last_became_peered":"2024-09-12T03:00:58.357480+0000","last_unstale":"2024-09-12T03:02:33.117061+0000","last_undegraded":"2024-09-12T03:02:33.117061+0000","last_fullsized":"2024-09-12T03:02:33.117061+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:26.884381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.108215+0000","last_change":"2024-09-12T03:00:58.349504+0000","last_active":"2024-09-12T03:02:33.108215+0000","last_peered":"2024-09-12T03:02:33.108215+0000","last_clean":"2024-09-12T03:02:33.108215+0000","last_became_active":"2024-09-12T03:00:58.349136+0000","last_became_peered":"2024-09-12T03:00:58.349136+0000","last_unstale":"2024-09-12T03:02:33.108215+0000","last_undegraded":"2024-09-12T03:02:33.108215+0000","last_fullsized":"2024-09-12T03:02:33.108215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:09:55.229516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.122920+0000","last_change":"2024-09-12T03:00:58.341902+0000","last_active":"2024-09-12T03:02:33.122920+0000","last_peered":"2024-09-12T03:02:33.122920+0000","last_clean":"2024-09-12T03:02:33.122920+0000","last_became_active":"2024-09-12T03:00:58.341592+0000","last_became_peered":"2024-09-12T03:00:58.341592+0000","last_unstale":"2024-09-12T03:02:33.122920+0000","last_undegraded":"2024-09-12T03:02:33.122920+0000","last_fullsized":"2024-09-12T03:02:33.122920+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:13.852511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.114521+0000","last_change":"2024-09-12T03:00:58.348012+0000","last_active":"2024-09-12T03:02:33.114521+0000","last_peered":"2024-09-12T03:02:33.114521+0000","last_clean":"2024-09-12T03:02:33.114521+0000","last_became_active":"2024-09-12T03:00:58.347338+0000","last_became_peered":"2024-09-12T03:00:58.347338+0000","last_unstale":"2024-09-12T03:02:33.114521+0000","last_undegraded":"2024-09-12T03:02:33.114521+0000","last_fullsized":"2024-09-12T03:02:33.114521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:41.478032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.112751+0000","last_change":"2024-09-12T03:00:58.345833+0000","last_active":"2024-09-12T03:02:33.112751+0000","last_peered":"2024-09-12T03:02:33.112751+0000","last_clean":"2024-09-12T03:02:33.112751+0000","last_became_active":"2024-09-12T03:00:58.345482+0000","last_became_peered":"2024-09-12T03:00:58.345482+0000","last_unstale":"2024-09-12T03:02:33.112751+0000","last_undegraded":"2024-09-12T03:02:33.112751+0000","last_fullsized":"2024-09-12T03:02:33.112751+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:32.037412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.112502+0000","last_change":"2024-09-12T03:00:58.352290+0000","last_active":"2024-09-12T03:02:33.112502+0000","last_peered":"2024-09-12T03:02:33.112502+0000","last_clean":"2024-09-12T03:02:33.112502+0000","last_became_active":"2024-09-12T03:00:58.351918+0000","last_became_peered":"2024-09-12T03:00:58.351918+0000","last_unstale":"2024-09-12T03:02:33.112502+0000","last_undegraded":"2024-09-12T03:02:33.112502+0000","last_fullsized":"2024-09-12T03:02:33.112502+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:25.941080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.114976+0000","last_change":"2024-09-12T03:00:58.348021+0000","last_active":"2024-09-12T03:02:33.114976+0000","last_peered":"2024-09-12T03:02:33.114976+0000","last_clean":"2024-09-12T03:02:33.114976+0000","last_became_active":"2024-09-12T03:00:58.347634+0000","last_became_peered":"2024-09-12T03:00:58.347634+0000","last_unstale":"2024-09-12T03:02:33.114976+0000","last_undegraded":"2024-09-12T03:02:33.114976+0000","last_fullsized":"2024-09-12T03:02:33.114976+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:47.933067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.123449+0000","last_change":"2024-09-12T03:00:58.340891+0000","last_active":"2024-09-12T03:02:33.123449+0000","last_peered":"2024-09-12T03:02:33.123449+0000","last_clean":"2024-09-12T03:02:33.123449+0000","last_became_active":"2024-09-12T03:00:58.340504+0000","last_became_peered":"2024-09-12T03:00:58.340504+0000","last_unstale":"2024-09-12T03:02:33.123449+0000","last_undegraded":"2024-09-12T03:02:33.123449+0000","last_fullsized":"2024-09-12T03:02:33.123449+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:32.117635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.108754+0000","last_change":"2024-09-12T03:00:58.339343+0000","last_active":"2024-09-12T03:02:33.108754+0000","last_peered":"2024-09-12T03:02:33.108754+0000","last_clean":"2024-09-12T03:02:33.108754+0000","last_became_active":"2024-09-12T03:00:58.339001+0000","last_became_peered":"2024-09-12T03:00:58.339001+0000","last_unstale":"2024-09-12T03:02:33.108754+0000","last_undegraded":"2024-09-12T03:02:33.108754+0000","last_fullsized":"2024-09-12T03:02:33.108754+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:02.043454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":140,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:57.563136+0000","last_change":"2024-09-12T03:00:56.315550+0000","last_active":"2024-09-12T03:02:57.563136+0000","last_peered":"2024-09-12T03:02:57.563136+0000","last_clean":"2024-09-12T03:02:57.563136+0000","last_became_active":"2024-09-12T03:00:56.315225+0000","last_became_peered":"2024-09-12T03:00:56.315225+0000","last_unstale":"2024-09-12T03:02:57.563136+0000","last_undegraded":"2024-09-12T03:02:57.563136+0000","last_fullsized":"2024-09-12T03:02:57.563136+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:46.070895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.116508+0000","last_change":"2024-09-12T03:02:04.290498+0000","last_active":"2024-09-12T03:02:33.116508+0000","last_peered":"2024-09-12T03:02:33.116508+0000","last_clean":"2024-09-12T03:02:33.116508+0000","last_became_active":"2024-09-12T03:02:04.290162+0000","last_became_peered":"2024-09-12T03:02:04.290162+0000","last_unstale":"2024-09-12T03:02:33.116508+0000","last_undegraded":"2024-09-12T03:02:33.116508+0000","last_fullsized":"2024-09-12T03:02:33.116508+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:29.777155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282566+0000","last_change":"2024-09-12T03:00:56.316385+0000","last_active":"2024-09-12T03:02:04.282566+0000","last_peered":"2024-09-12T03:02:04.282566+0000","last_clean":"2024-09-12T03:02:04.282566+0000","last_became_active":"2024-09-12T03:00:56.316064+0000","last_became_peered":"2024-09-12T03:00:56.316064+0000","last_unstale":"2024-09-12T03:02:04.282566+0000","last_undegraded":"2024-09-12T03:02:04.282566+0000","last_fullsized":"2024-09-12T03:02:04.282566+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:12:43.323654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332424+0000","last_change":"2024-09-12T03:00:56.313065+0000","last_active":"2024-09-12T03:02:04.332424+0000","last_peered":"2024-09-12T03:02:04.332424+0000","last_clean":"2024-09-12T03:02:04.332424+0000","last_became_active":"2024-09-12T03:00:56.312716+0000","last_became_peered":"2024-09-12T03:00:56.312716+0000","last_unstale":"2024-09-12T03:02:04.332424+0000","last_undegraded":"2024-09-12T03:02:04.332424+0000","last_fullsized":"2024-09-12T03:02:04.332424+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:51.089089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120152+0000","last_change":"2024-09-12T03:00:58.346542+0000","last_active":"2024-09-12T03:02:33.120152+0000","last_peered":"2024-09-12T03:02:33.120152+0000","last_clean":"2024-09-12T03:02:33.120152+0000","last_became_active":"2024-09-12T03:00:58.346260+0000","last_became_peered":"2024-09-12T03:00:58.346260+0000","last_unstale":"2024-09-12T03:02:33.120152+0000","last_undegraded":"2024-09-12T03:02:33.120152+0000","last_fullsized":"2024-09-12T03:02:33.120152+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:37.034969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289182+0000","last_change":"2024-09-12T03:00:56.317308+0000","last_active":"2024-09-12T03:02:04.289182+0000","last_peered":"2024-09-12T03:02:04.289182+0000","last_clean":"2024-09-12T03:02:04.289182+0000","last_became_active":"2024-09-12T03:00:56.316935+0000","last_became_peered":"2024-09-12T03:00:56.316935+0000","last_unstale":"2024-09-12T03:02:04.289182+0000","last_undegraded":"2024-09-12T03:02:04.289182+0000","last_fullsized":"2024-09-12T03:02:04.289182+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:06.324489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.118033+0000","last_change":"2024-09-12T03:00:58.341565+0000","last_active":"2024-09-12T03:02:33.118033+0000","last_peered":"2024-09-12T03:02:33.118033+0000","last_clean":"2024-09-12T03:02:33.118033+0000","last_became_active":"2024-09-12T03:00:58.341285+0000","last_became_peered":"2024-09-12T03:00:58.341285+0000","last_unstale":"2024-09-12T03:02:33.118033+0000","last_undegraded":"2024-09-12T03:02:33.118033+0000","last_fullsized":"2024-09-12T03:02:33.118033+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:59.156700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":138,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:59.565408+0000","last_change":"2024-09-12T03:00:56.314201+0000","last_active":"2024-09-12T03:02:59.565408+0000","last_peered":"2024-09-12T03:02:59.565408+0000","last_clean":"2024-09-12T03:02:59.565408+0000","last_became_active":"2024-09-12T03:00:56.313880+0000","last_became_peered":"2024-09-12T03:00:56.313880+0000","last_unstale":"2024-09-12T03:02:59.565408+0000","last_undegraded":"2024-09-12T03:02:59.565408+0000","last_fullsized":"2024-09-12T03:02:59.565408+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:57:12.952630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.132547+0000","last_change":"2024-09-12T03:00:58.345274+0000","last_active":"2024-09-12T03:02:33.132547+0000","last_peered":"2024-09-12T03:02:33.132547+0000","last_clean":"2024-09-12T03:02:33.132547+0000","last_became_active":"2024-09-12T03:00:58.344937+0000","last_became_peered":"2024-09-12T03:00:58.344937+0000","last_unstale":"2024-09-12T03:02:33.132547+0000","last_undegraded":"2024-09-12T03:02:33.132547+0000","last_fullsized":"2024-09-12T03:02:33.132547+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:17.288909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284614+0000","last_change":"2024-09-12T03:00:56.314709+0000","last_active":"2024-09-12T03:02:04.284614+0000","last_peered":"2024-09-12T03:02:04.284614+0000","last_clean":"2024-09-12T03:02:04.284614+0000","last_became_active":"2024-09-12T03:00:56.314349+0000","last_became_peered":"2024-09-12T03:00:56.314349+0000","last_unstale":"2024-09-12T03:02:04.284614+0000","last_undegraded":"2024-09-12T03:02:04.284614+0000","last_fullsized":"2024-09-12T03:02:04.284614+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:18:35.371661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.113713+0000","last_change":"2024-09-12T03:00:58.346055+0000","last_active":"2024-09-12T03:02:33.113713+0000","last_peered":"2024-09-12T03:02:33.113713+0000","last_clean":"2024-09-12T03:02:33.113713+0000","last_became_active":"2024-09-12T03:00:58.345701+0000","last_became_peered":"2024-09-12T03:00:58.345701+0000","last_unstale":"2024-09-12T03:02:33.113713+0000","last_undegraded":"2024-09-12T03:02:33.113713+0000","last_fullsized":"2024-09-12T03:02:33.113713+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:33.977245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.287450+0000","last_change":"2024-09-12T03:00:56.435962+0000","last_active":"2024-09-12T03:02:04.287450+0000","last_peered":"2024-09-12T03:02:04.287450+0000","last_clean":"2024-09-12T03:02:04.287450+0000","last_became_active":"2024-09-12T03:00:56.435515+0000","last_became_peered":"2024-09-12T03:00:56.435515+0000","last_unstale":"2024-09-12T03:02:04.287450+0000","last_undegraded":"2024-09-12T03:02:04.287450+0000","last_fullsized":"2024-09-12T03:02:04.287450+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:32.953436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.135045+0000","last_change":"2024-09-12T03:00:58.344239+0000","last_active":"2024-09-12T03:02:33.135045+0000","last_peered":"2024-09-12T03:02:33.135045+0000","last_clean":"2024-09-12T03:02:33.135045+0000","last_became_active":"2024-09-12T03:00:58.343876+0000","last_became_peered":"2024-09-12T03:00:58.343876+0000","last_unstale":"2024-09-12T03:02:33.135045+0000","last_undegraded":"2024-09-12T03:02:33.135045+0000","last_fullsized":"2024-09-12T03:02:33.135045+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:49.401295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'36","reported_seq":297,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:03:00.567788+0000","last_change":"2024-09-12T03:00:56.317708+0000","last_active":"2024-09-12T03:03:00.567788+0000","last_peered":"2024-09-12T03:03:00.567788+0000","last_clean":"2024-09-12T03:03:00.567788+0000","last_became_active":"2024-09-12T03:00:56.317317+0000","last_became_peered":"2024-09-12T03:00:56.317317+0000","last_unstale":"2024-09-12T03:03:00.567788+0000","last_undegraded":"2024-09-12T03:03:00.567788+0000","last_fullsized":"2024-09-12T03:03:00.567788+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:47.788186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70595,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":239,"num_read_kb":188,"num_write":37,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.123764+0000","last_change":"2024-09-12T03:00:58.343602+0000","last_active":"2024-09-12T03:02:33.123764+0000","last_peered":"2024-09-12T03:02:33.123764+0000","last_clean":"2024-09-12T03:02:33.123764+0000","last_became_active":"2024-09-12T03:00:58.343301+0000","last_became_peered":"2024-09-12T03:00:58.343301+0000","last_unstale":"2024-09-12T03:02:33.123764+0000","last_undegraded":"2024-09-12T03:02:33.123764+0000","last_fullsized":"2024-09-12T03:02:33.123764+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:04.056118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.290204+0000","last_change":"2024-09-12T03:00:56.317874+0000","last_active":"2024-09-12T03:02:04.290204+0000","last_peered":"2024-09-12T03:02:04.290204+0000","last_clean":"2024-09-12T03:02:04.290204+0000","last_became_active":"2024-09-12T03:00:56.317566+0000","last_became_peered":"2024-09-12T03:00:56.317566+0000","last_unstale":"2024-09-12T03:02:04.290204+0000","last_undegraded":"2024-09-12T03:02:04.290204+0000","last_fullsized":"2024-09-12T03:02:04.290204+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:54.203730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.121582+0000","last_change":"2024-09-12T03:00:58.351539+0000","last_active":"2024-09-12T03:02:33.121582+0000","last_peered":"2024-09-12T03:02:33.121582+0000","last_clean":"2024-09-12T03:02:33.121582+0000","last_became_active":"2024-09-12T03:00:58.351249+0000","last_became_peered":"2024-09-12T03:00:58.351249+0000","last_unstale":"2024-09-12T03:02:33.121582+0000","last_undegraded":"2024-09-12T03:02:33.121582+0000","last_fullsized":"2024-09-12T03:02:33.121582+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:40.498867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.287068+0000","last_change":"2024-09-12T03:00:56.314489+0000","last_active":"2024-09-12T03:02:04.287068+0000","last_peered":"2024-09-12T03:02:04.287068+0000","last_clean":"2024-09-12T03:02:04.287068+0000","last_became_active":"2024-09-12T03:00:56.314142+0000","last_became_peered":"2024-09-12T03:00:56.314142+0000","last_unstale":"2024-09-12T03:02:04.287068+0000","last_undegraded":"2024-09-12T03:02:04.287068+0000","last_fullsized":"2024-09-12T03:02:04.287068+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:21.541713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.123797+0000","last_change":"2024-09-12T03:00:58.348752+0000","last_active":"2024-09-12T03:02:33.123797+0000","last_peered":"2024-09-12T03:02:33.123797+0000","last_clean":"2024-09-12T03:02:33.123797+0000","last_became_active":"2024-09-12T03:00:58.348391+0000","last_became_peered":"2024-09-12T03:00:58.348391+0000","last_unstale":"2024-09-12T03:02:33.123797+0000","last_undegraded":"2024-09-12T03:02:33.123797+0000","last_fullsized":"2024-09-12T03:02:33.123797+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:33:15.626306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'4","reported_seq":141,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:58.565339+0000","last_change":"2024-09-12T03:00:56.311702+0000","last_active":"2024-09-12T03:02:58.565339+0000","last_peered":"2024-09-12T03:02:58.565339+0000","last_clean":"2024-09-12T03:02:58.565339+0000","last_became_active":"2024-09-12T03:00:56.311353+0000","last_became_peered":"2024-09-12T03:00:56.311353+0000","last_unstale":"2024-09-12T03:02:58.565339+0000","last_undegraded":"2024-09-12T03:02:58.565339+0000","last_fullsized":"2024-09-12T03:02:58.565339+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:01.004858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.123076+0000","last_change":"2024-09-12T03:00:58.351973+0000","last_active":"2024-09-12T03:02:33.123076+0000","last_peered":"2024-09-12T03:02:33.123076+0000","last_clean":"2024-09-12T03:02:33.123076+0000","last_became_active":"2024-09-12T03:00:58.351599+0000","last_became_peered":"2024-09-12T03:00:58.351599+0000","last_unstale":"2024-09-12T03:02:33.123076+0000","last_undegraded":"2024-09-12T03:02:33.123076+0000","last_fullsized":"2024-09-12T03:02:33.123076+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:26.398828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.291749+0000","last_change":"2024-09-12T03:00:56.436185+0000","last_active":"2024-09-12T03:02:04.291749+0000","last_peered":"2024-09-12T03:02:04.291749+0000","last_clean":"2024-09-12T03:02:04.291749+0000","last_became_active":"2024-09-12T03:00:56.435717+0000","last_became_peered":"2024-09-12T03:00:56.435717+0000","last_unstale":"2024-09-12T03:02:04.291749+0000","last_undegraded":"2024-09-12T03:02:04.291749+0000","last_fullsized":"2024-09-12T03:02:04.291749+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:27.303182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.119766+0000","last_change":"2024-09-12T03:02:04.333661+0000","last_active":"2024-09-12T03:02:33.119766+0000","last_peered":"2024-09-12T03:02:33.119766+0000","last_clean":"2024-09-12T03:02:33.119766+0000","last_became_active":"2024-09-12T03:02:04.333217+0000","last_became_peered":"2024-09-12T03:02:04.333217+0000","last_unstale":"2024-09-12T03:02:33.119766+0000","last_undegraded":"2024-09-12T03:02:33.119766+0000","last_fullsized":"2024-09-12T03:02:33.119766+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:23:47.212734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.297341+0000","last_change":"2024-09-12T03:02:04.296884+0000","last_active":"2024-09-12T03:02:04.297341+0000","last_peered":"2024-09-12T03:02:04.297341+0000","last_clean":"2024-09-12T03:02:04.297341+0000","last_became_active":"2024-09-12T03:02:04.290894+0000","last_became_peered":"2024-09-12T03:02:04.290894+0000","last_unstale":"2024-09-12T03:02:04.297341+0000","last_undegraded":"2024-09-12T03:02:04.297341+0000","last_fullsized":"2024-09-12T03:02:04.297341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:49:19.749568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.118228+0000","last_change":"2024-09-12T03:02:04.334096+0000","last_active":"2024-09-12T03:02:33.118228+0000","last_peered":"2024-09-12T03:02:33.118228+0000","last_clean":"2024-09-12T03:02:33.118228+0000","last_became_active":"2024-09-12T03:02:04.333641+0000","last_became_peered":"2024-09-12T03:02:04.333641+0000","last_unstale":"2024-09-12T03:02:33.118228+0000","last_undegraded":"2024-09-12T03:02:33.118228+0000","last_fullsized":"2024-09-12T03:02:33.118228+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:24.085293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332966+0000","last_change":"2024-09-12T03:00:56.314182+0000","last_active":"2024-09-12T03:02:04.332966+0000","last_peered":"2024-09-12T03:02:04.332966+0000","last_clean":"2024-09-12T03:02:04.332966+0000","last_became_active":"2024-09-12T03:00:56.313749+0000","last_became_peered":"2024-09-12T03:00:56.313749+0000","last_unstale":"2024-09-12T03:02:04.332966+0000","last_undegraded":"2024-09-12T03:02:04.332966+0000","last_fullsized":"2024-09-12T03:02:04.332966+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:31:45.767500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.125121+0000","last_change":"2024-09-12T03:00:58.344356+0000","last_active":"2024-09-12T03:02:33.125121+0000","last_peered":"2024-09-12T03:02:33.125121+0000","last_clean":"2024-09-12T03:02:33.125121+0000","last_became_active":"2024-09-12T03:00:58.344035+0000","last_became_peered":"2024-09-12T03:00:58.344035+0000","last_unstale":"2024-09-12T03:02:33.125121+0000","last_undegraded":"2024-09-12T03:02:33.125121+0000","last_fullsized":"2024-09-12T03:02:33.125121+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:45:18.681018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.287585+0000","last_change":"2024-09-12T03:00:56.315653+0000","last_active":"2024-09-12T03:02:04.287585+0000","last_peered":"2024-09-12T03:02:04.287585+0000","last_clean":"2024-09-12T03:02:04.287585+0000","last_became_active":"2024-09-12T03:00:56.315284+0000","last_became_peered":"2024-09-12T03:00:56.315284+0000","last_unstale":"2024-09-12T03:02:04.287585+0000","last_undegraded":"2024-09-12T03:02:04.287585+0000","last_fullsized":"2024-09-12T03:02:04.287585+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:42.934020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.122059+0000","last_change":"2024-09-12T03:02:04.291128+0000","last_active":"2024-09-12T03:02:33.122059+0000","last_peered":"2024-09-12T03:02:33.122059+0000","last_clean":"2024-09-12T03:02:33.122059+0000","last_became_active":"2024-09-12T03:02:04.287836+0000","last_became_peered":"2024-09-12T03:02:04.287836+0000","last_unstale":"2024-09-12T03:02:33.122059+0000","last_undegraded":"2024-09-12T03:02:33.122059+0000","last_fullsized":"2024-09-12T03:02:33.122059+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:23.956579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.290577+0000","last_change":"2024-09-12T03:00:56.746780+0000","last_active":"2024-09-12T03:02:04.290577+0000","last_peered":"2024-09-12T03:02:04.290577+0000","last_clean":"2024-09-12T03:02:04.290577+0000","last_became_active":"2024-09-12T03:00:56.746414+0000","last_became_peered":"2024-09-12T03:00:56.746414+0000","last_unstale":"2024-09-12T03:02:04.290577+0000","last_undegraded":"2024-09-12T03:02:04.290577+0000","last_fullsized":"2024-09-12T03:02:04.290577+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:15:35.214196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.131100+0000","last_change":"2024-09-12T03:00:58.351713+0000","last_active":"2024-09-12T03:02:33.131100+0000","last_peered":"2024-09-12T03:02:33.131100+0000","last_clean":"2024-09-12T03:02:33.131100+0000","last_became_active":"2024-09-12T03:00:58.351371+0000","last_became_peered":"2024-09-12T03:00:58.351371+0000","last_unstale":"2024-09-12T03:02:33.131100+0000","last_undegraded":"2024-09-12T03:02:33.131100+0000","last_fullsized":"2024-09-12T03:02:33.131100+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:13.346956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'137","reported_seq":336,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284318+0000","last_change":"2024-09-12T02:59:45.461482+0000","last_active":"2024-09-12T03:02:04.284318+0000","last_peered":"2024-09-12T03:02:04.284318+0000","last_clean":"2024-09-12T03:02:04.284318+0000","last_became_active":"2024-09-12T02:59:45.461058+0000","last_became_peered":"2024-09-12T02:59:45.461058+0000","last_unstale":"2024-09-12T03:02:04.284318+0000","last_undegraded":"2024-09-12T03:02:04.284318+0000","last_fullsized":"2024-09-12T03:02:04.284318+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:18.034438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:18.034438+0000","last_clean_scrub_stamp":"2024-09-12T02:56:18.034438+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:09.251858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289291+0000","last_change":"2024-09-12T03:00:56.315245+0000","last_active":"2024-09-12T03:02:04.289291+0000","last_peered":"2024-09-12T03:02:04.289291+0000","last_clean":"2024-09-12T03:02:04.289291+0000","last_became_active":"2024-09-12T03:00:56.314887+0000","last_became_peered":"2024-09-12T03:00:56.314887+0000","last_unstale":"2024-09-12T03:02:04.289291+0000","last_undegraded":"2024-09-12T03:02:04.289291+0000","last_fullsized":"2024-09-12T03:02:04.289291+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:48.750657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.112960+0000","last_change":"2024-09-12T03:00:58.357874+0000","last_active":"2024-09-12T03:02:33.112960+0000","last_peered":"2024-09-12T03:02:33.112960+0000","last_clean":"2024-09-12T03:02:33.112960+0000","last_became_active":"2024-09-12T03:00:58.357575+0000","last_became_peered":"2024-09-12T03:00:58.357575+0000","last_unstale":"2024-09-12T03:02:33.112960+0000","last_undegraded":"2024-09-12T03:02:33.112960+0000","last_fullsized":"2024-09-12T03:02:33.112960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:46.125620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284282+0000","last_change":"2024-09-12T03:00:56.312263+0000","last_active":"2024-09-12T03:02:04.284282+0000","last_peered":"2024-09-12T03:02:04.284282+0000","last_clean":"2024-09-12T03:02:04.284282+0000","last_became_active":"2024-09-12T03:00:56.311894+0000","last_became_peered":"2024-09-12T03:00:56.311894+0000","last_unstale":"2024-09-12T03:02:04.284282+0000","last_undegraded":"2024-09-12T03:02:04.284282+0000","last_fullsized":"2024-09-12T03:02:04.284282+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:52.434203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.110253+0000","last_change":"2024-09-12T03:00:58.350762+0000","last_active":"2024-09-12T03:02:33.110253+0000","last_peered":"2024-09-12T03:02:33.110253+0000","last_clean":"2024-09-12T03:02:33.110253+0000","last_became_active":"2024-09-12T03:00:58.348038+0000","last_became_peered":"2024-09-12T03:00:58.348038+0000","last_unstale":"2024-09-12T03:02:33.110253+0000","last_undegraded":"2024-09-12T03:02:33.110253+0000","last_fullsized":"2024-09-12T03:02:33.110253+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:08.916636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":134,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:54.557217+0000","last_change":"2024-09-12T03:00:56.435641+0000","last_active":"2024-09-12T03:02:54.557217+0000","last_peered":"2024-09-12T03:02:54.557217+0000","last_clean":"2024-09-12T03:02:54.557217+0000","last_became_active":"2024-09-12T03:00:56.435319+0000","last_became_peered":"2024-09-12T03:00:56.435319+0000","last_unstale":"2024-09-12T03:02:54.557217+0000","last_undegraded":"2024-09-12T03:02:54.557217+0000","last_fullsized":"2024-09-12T03:02:54.557217+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:56.894978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.112818+0000","last_change":"2024-09-12T03:00:58.344708+0000","last_active":"2024-09-12T03:02:33.112818+0000","last_peered":"2024-09-12T03:02:33.112818+0000","last_clean":"2024-09-12T03:02:33.112818+0000","last_became_active":"2024-09-12T03:00:58.344389+0000","last_became_peered":"2024-09-12T03:00:58.344389+0000","last_unstale":"2024-09-12T03:02:33.112818+0000","last_undegraded":"2024-09-12T03:02:33.112818+0000","last_fullsized":"2024-09-12T03:02:33.112818+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:08.160383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284317+0000","last_change":"2024-09-12T03:00:56.317068+0000","last_active":"2024-09-12T03:02:04.284317+0000","last_peered":"2024-09-12T03:02:04.284317+0000","last_clean":"2024-09-12T03:02:04.284317+0000","last_became_active":"2024-09-12T03:00:56.316698+0000","last_became_peered":"2024-09-12T03:00:56.316698+0000","last_unstale":"2024-09-12T03:02:04.284317+0000","last_undegraded":"2024-09-12T03:02:04.284317+0000","last_fullsized":"2024-09-12T03:02:04.284317+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:41:19.199418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.109469+0000","last_change":"2024-09-12T03:00:58.347814+0000","last_active":"2024-09-12T03:02:33.109469+0000","last_peered":"2024-09-12T03:02:33.109469+0000","last_clean":"2024-09-12T03:02:33.109469+0000","last_became_active":"2024-09-12T03:00:58.347112+0000","last_became_peered":"2024-09-12T03:00:58.347112+0000","last_unstale":"2024-09-12T03:02:33.109469+0000","last_undegraded":"2024-09-12T03:02:33.109469+0000","last_fullsized":"2024-09-12T03:02:33.109469+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:59.325113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284265+0000","last_change":"2024-09-12T03:00:56.435771+0000","last_active":"2024-09-12T03:02:04.284265+0000","last_peered":"2024-09-12T03:02:04.284265+0000","last_clean":"2024-09-12T03:02:04.284265+0000","last_became_active":"2024-09-12T03:00:56.435448+0000","last_became_peered":"2024-09-12T03:00:56.435448+0000","last_unstale":"2024-09-12T03:02:04.284265+0000","last_undegraded":"2024-09-12T03:02:04.284265+0000","last_fullsized":"2024-09-12T03:02:04.284265+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:33.650338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.117580+0000","last_change":"2024-09-12T03:00:58.341071+0000","last_active":"2024-09-12T03:02:33.117580+0000","last_peered":"2024-09-12T03:02:33.117580+0000","last_clean":"2024-09-12T03:02:33.117580+0000","last_became_active":"2024-09-12T03:00:58.340753+0000","last_became_peered":"2024-09-12T03:00:58.340753+0000","last_unstale":"2024-09-12T03:02:33.117580+0000","last_undegraded":"2024-09-12T03:02:33.117580+0000","last_fullsized":"2024-09-12T03:02:33.117580+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:21.217915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292254+0000","last_change":"2024-09-12T03:00:56.436058+0000","last_active":"2024-09-12T03:02:04.292254+0000","last_peered":"2024-09-12T03:02:04.292254+0000","last_clean":"2024-09-12T03:02:04.292254+0000","last_became_active":"2024-09-12T03:00:56.435518+0000","last_became_peered":"2024-09-12T03:00:56.435518+0000","last_unstale":"2024-09-12T03:02:04.292254+0000","last_undegraded":"2024-09-12T03:02:04.292254+0000","last_fullsized":"2024-09-12T03:02:04.292254+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:32.235097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.110650+0000","last_change":"2024-09-12T03:00:58.348236+0000","last_active":"2024-09-12T03:02:33.110650+0000","last_peered":"2024-09-12T03:02:33.110650+0000","last_clean":"2024-09-12T03:02:33.110650+0000","last_became_active":"2024-09-12T03:00:58.347727+0000","last_became_peered":"2024-09-12T03:00:58.347727+0000","last_unstale":"2024-09-12T03:02:33.110650+0000","last_undegraded":"2024-09-12T03:02:33.110650+0000","last_fullsized":"2024-09-12T03:02:33.110650+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:19.233163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'12","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.354650+0000","last_change":"2024-09-12T03:00:56.310537+0000","last_active":"2024-09-12T03:02:33.354650+0000","last_peered":"2024-09-12T03:02:33.354650+0000","last_clean":"2024-09-12T03:02:33.354650+0000","last_became_active":"2024-09-12T03:00:56.310135+0000","last_became_peered":"2024-09-12T03:00:56.310135+0000","last_unstale":"2024-09-12T03:02:33.354650+0000","last_undegraded":"2024-09-12T03:02:33.354650+0000","last_fullsized":"2024-09-12T03:02:33.354650+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:10:55.584339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.121993+0000","last_change":"2024-09-12T03:00:58.358417+0000","last_active":"2024-09-12T03:02:33.121993+0000","last_peered":"2024-09-12T03:02:33.121993+0000","last_clean":"2024-09-12T03:02:33.121993+0000","last_became_active":"2024-09-12T03:00:58.358119+0000","last_became_peered":"2024-09-12T03:00:58.358119+0000","last_unstale":"2024-09-12T03:02:33.121993+0000","last_undegraded":"2024-09-12T03:02:33.121993+0000","last_fullsized":"2024-09-12T03:02:33.121993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:10.897594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289526+0000","last_change":"2024-09-12T03:00:56.747044+0000","last_active":"2024-09-12T03:02:04.289526+0000","last_peered":"2024-09-12T03:02:04.289526+0000","last_clean":"2024-09-12T03:02:04.289526+0000","last_became_active":"2024-09-12T03:00:56.746699+0000","last_became_peered":"2024-09-12T03:00:56.746699+0000","last_unstale":"2024-09-12T03:02:04.289526+0000","last_undegraded":"2024-09-12T03:02:04.289526+0000","last_fullsized":"2024-09-12T03:02:04.289526+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:11.420109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.113058+0000","last_change":"2024-09-12T03:00:58.346854+0000","last_active":"2024-09-12T03:02:33.113058+0000","last_peered":"2024-09-12T03:02:33.113058+0000","last_clean":"2024-09-12T03:02:33.113058+0000","last_became_active":"2024-09-12T03:00:58.346535+0000","last_became_peered":"2024-09-12T03:00:58.346535+0000","last_unstale":"2024-09-12T03:02:33.113058+0000","last_undegraded":"2024-09-12T03:02:33.113058+0000","last_fullsized":"2024-09-12T03:02:33.113058+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:46.637914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332365+0000","last_change":"2024-09-12T03:00:56.435957+0000","last_active":"2024-09-12T03:02:04.332365+0000","last_peered":"2024-09-12T03:02:04.332365+0000","last_clean":"2024-09-12T03:02:04.332365+0000","last_became_active":"2024-09-12T03:00:56.435603+0000","last_became_peered":"2024-09-12T03:00:56.435603+0000","last_unstale":"2024-09-12T03:02:04.332365+0000","last_undegraded":"2024-09-12T03:02:04.332365+0000","last_fullsized":"2024-09-12T03:02:04.332365+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:34.553189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.089792+0000","last_change":"2024-09-12T03:00:58.337784+0000","last_active":"2024-09-12T03:02:33.089792+0000","last_peered":"2024-09-12T03:02:33.089792+0000","last_clean":"2024-09-12T03:02:33.089792+0000","last_became_active":"2024-09-12T03:00:58.337415+0000","last_became_peered":"2024-09-12T03:00:58.337415+0000","last_unstale":"2024-09-12T03:02:33.089792+0000","last_undegraded":"2024-09-12T03:02:33.089792+0000","last_fullsized":"2024-09-12T03:02:33.089792+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:38:15.557067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282354+0000","last_change":"2024-09-12T03:00:56.315356+0000","last_active":"2024-09-12T03:02:04.282354+0000","last_peered":"2024-09-12T03:02:04.282354+0000","last_clean":"2024-09-12T03:02:04.282354+0000","last_became_active":"2024-09-12T03:00:56.314873+0000","last_became_peered":"2024-09-12T03:00:56.314873+0000","last_unstale":"2024-09-12T03:02:04.282354+0000","last_undegraded":"2024-09-12T03:02:04.282354+0000","last_fullsized":"2024-09-12T03:02:04.282354+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:55.710156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.118974+0000","last_change":"2024-09-12T03:00:58.346706+0000","last_active":"2024-09-12T03:02:33.118974+0000","last_peered":"2024-09-12T03:02:33.118974+0000","last_clean":"2024-09-12T03:02:33.118974+0000","last_became_active":"2024-09-12T03:00:58.346363+0000","last_became_peered":"2024-09-12T03:00:58.346363+0000","last_unstale":"2024-09-12T03:02:33.118974+0000","last_undegraded":"2024-09-12T03:02:33.118974+0000","last_fullsized":"2024-09-12T03:02:33.118974+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:13.468639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.299582+0000","last_change":"2024-09-12T03:02:04.299147+0000","last_active":"2024-09-12T03:02:04.299582+0000","last_peered":"2024-09-12T03:02:04.299582+0000","last_clean":"2024-09-12T03:02:04.299582+0000","last_became_active":"2024-09-12T03:02:04.293211+0000","last_became_peered":"2024-09-12T03:02:04.293211+0000","last_unstale":"2024-09-12T03:02:04.299582+0000","last_undegraded":"2024-09-12T03:02:04.299582+0000","last_fullsized":"2024-09-12T03:02:04.299582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:01:39.222476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.118849+0000","last_change":"2024-09-12T03:00:58.356682+0000","last_active":"2024-09-12T03:02:33.118849+0000","last_peered":"2024-09-12T03:02:33.118849+0000","last_clean":"2024-09-12T03:02:33.118849+0000","last_became_active":"2024-09-12T03:00:58.356383+0000","last_became_peered":"2024-09-12T03:00:58.356383+0000","last_unstale":"2024-09-12T03:02:33.118849+0000","last_undegraded":"2024-09-12T03:02:33.118849+0000","last_fullsized":"2024-09-12T03:02:33.118849+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:14:16.428393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292053+0000","last_change":"2024-09-12T03:00:56.314943+0000","last_active":"2024-09-12T03:02:04.292053+0000","last_peered":"2024-09-12T03:02:04.292053+0000","last_clean":"2024-09-12T03:02:04.292053+0000","last_became_active":"2024-09-12T03:00:56.314602+0000","last_became_peered":"2024-09-12T03:00:56.314602+0000","last_unstale":"2024-09-12T03:02:04.292053+0000","last_undegraded":"2024-09-12T03:02:04.292053+0000","last_fullsized":"2024-09-12T03:02:04.292053+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:31.207880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.131538+0000","last_change":"2024-09-12T03:00:58.351774+0000","last_active":"2024-09-12T03:02:33.131538+0000","last_peered":"2024-09-12T03:02:33.131538+0000","last_clean":"2024-09-12T03:02:33.131538+0000","last_became_active":"2024-09-12T03:00:58.351453+0000","last_became_peered":"2024-09-12T03:00:58.351453+0000","last_unstale":"2024-09-12T03:02:33.131538+0000","last_undegraded":"2024-09-12T03:02:33.131538+0000","last_fullsized":"2024-09-12T03:02:33.131538+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:18:24.260110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.130930+0000","last_change":"2024-09-12T03:00:58.346886+0000","last_active":"2024-09-12T03:02:33.130930+0000","last_peered":"2024-09-12T03:02:33.130930+0000","last_clean":"2024-09-12T03:02:33.130930+0000","last_became_active":"2024-09-12T03:00:58.346499+0000","last_became_peered":"2024-09-12T03:00:58.346499+0000","last_unstale":"2024-09-12T03:02:33.130930+0000","last_undegraded":"2024-09-12T03:02:33.130930+0000","last_fullsized":"2024-09-12T03:02:33.130930+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:02:24.045610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292497+0000","last_change":"2024-09-12T03:00:56.317443+0000","last_active":"2024-09-12T03:02:04.292497+0000","last_peered":"2024-09-12T03:02:04.292497+0000","last_clean":"2024-09-12T03:02:04.292497+0000","last_became_active":"2024-09-12T03:00:56.317045+0000","last_became_peered":"2024-09-12T03:00:56.317045+0000","last_unstale":"2024-09-12T03:02:04.292497+0000","last_undegraded":"2024-09-12T03:02:04.292497+0000","last_fullsized":"2024-09-12T03:02:04.292497+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:07.687606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.132782+0000","last_change":"2024-09-12T03:00:58.337257+0000","last_active":"2024-09-12T03:02:33.132782+0000","last_peered":"2024-09-12T03:02:33.132782+0000","last_clean":"2024-09-12T03:02:33.132782+0000","last_became_active":"2024-09-12T03:00:58.336847+0000","last_became_peered":"2024-09-12T03:00:58.336847+0000","last_unstale":"2024-09-12T03:02:33.132782+0000","last_undegraded":"2024-09-12T03:02:33.132782+0000","last_fullsized":"2024-09-12T03:02:33.132782+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:22.846206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292165+0000","last_change":"2024-09-12T03:00:56.316255+0000","last_active":"2024-09-12T03:02:04.292165+0000","last_peered":"2024-09-12T03:02:04.292165+0000","last_clean":"2024-09-12T03:02:04.292165+0000","last_became_active":"2024-09-12T03:00:56.315906+0000","last_became_peered":"2024-09-12T03:00:56.315906+0000","last_unstale":"2024-09-12T03:02:04.292165+0000","last_undegraded":"2024-09-12T03:02:04.292165+0000","last_fullsized":"2024-09-12T03:02:04.292165+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:14.798601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120273+0000","last_change":"2024-09-12T03:00:58.347206+0000","last_active":"2024-09-12T03:02:33.120273+0000","last_peered":"2024-09-12T03:02:33.120273+0000","last_clean":"2024-09-12T03:02:33.120273+0000","last_became_active":"2024-09-12T03:00:58.346923+0000","last_became_peered":"2024-09-12T03:00:58.346923+0000","last_unstale":"2024-09-12T03:02:33.120273+0000","last_undegraded":"2024-09-12T03:02:33.120273+0000","last_fullsized":"2024-09-12T03:02:33.120273+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:41.376211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.283743+0000","last_change":"2024-09-12T03:00:56.435431+0000","last_active":"2024-09-12T03:02:04.283743+0000","last_peered":"2024-09-12T03:02:04.283743+0000","last_clean":"2024-09-12T03:02:04.283743+0000","last_became_active":"2024-09-12T03:00:56.435088+0000","last_became_peered":"2024-09-12T03:00:56.435088+0000","last_unstale":"2024-09-12T03:02:04.283743+0000","last_undegraded":"2024-09-12T03:02:04.283743+0000","last_fullsized":"2024-09-12T03:02:04.283743+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:24.738340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.114461+0000","last_change":"2024-09-12T03:00:58.351737+0000","last_active":"2024-09-12T03:02:33.114461+0000","last_peered":"2024-09-12T03:02:33.114461+0000","last_clean":"2024-09-12T03:02:33.114461+0000","last_became_active":"2024-09-12T03:00:58.351407+0000","last_became_peered":"2024-09-12T03:00:58.351407+0000","last_unstale":"2024-09-12T03:02:33.114461+0000","last_undegraded":"2024-09-12T03:02:33.114461+0000","last_fullsized":"2024-09-12T03:02:33.114461+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:34.206703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284979+0000","last_change":"2024-09-12T03:00:56.310910+0000","last_active":"2024-09-12T03:02:04.284979+0000","last_peered":"2024-09-12T03:02:04.284979+0000","last_clean":"2024-09-12T03:02:04.284979+0000","last_became_active":"2024-09-12T03:00:56.310657+0000","last_became_peered":"2024-09-12T03:00:56.310657+0000","last_unstale":"2024-09-12T03:02:04.284979+0000","last_undegraded":"2024-09-12T03:02:04.284979+0000","last_fullsized":"2024-09-12T03:02:04.284979+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:06.117215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74598,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":701,"num_read_kb":427,"num_write":173,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":446464,"data_stored":218754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245899,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645350,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":431835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906987,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":1094282,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070512,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":432166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299446,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":431947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528378,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":1023901,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":40,"seq":171798691903,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":880640,"data_stored":431438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":33,"seq":141733920836,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":1023192,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051658,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":432268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280590,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":431632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476819,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":494890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607641,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":499306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":432709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":72566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:03:01.199 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-12T03:03:01.199 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:03:01.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.333+0000 7fd7ec968640 1 -- 172.21.15.116:0/1868686968 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7e8102040 msgr2=0x7fd7e8102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:01.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.333+0000 7fd7ec968640 1 --2- 172.21.15.116:0/1868686968 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7e8102040 0x7fd7e8102440 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0009900 tx=0x7fd7d002a410 comp rx=0 tx=0).stop 2024-09-12T03:03:01.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.334+0000 7fd7ec968640 1 -- 172.21.15.116:0/1868686968 shutdown_connections 2024-09-12T03:03:01.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.334+0000 7fd7ec968640 1 --2- 172.21.15.116:0/1868686968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7e8101650 0x7fd7e8108ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:01.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.334+0000 7fd7ec968640 1 --2- 172.21.15.116:0/1868686968 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7e8100c90 0x7fd7e8101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:01.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.334+0000 7fd7ec968640 1 --2- 172.21.15.116:0/1868686968 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7e8102040 0x7fd7e8102440 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:01.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.334+0000 7fd7ec968640 1 -- 172.21.15.116:0/1868686968 >> 172.21.15.116:0/1868686968 conn(0x7fd7e80fb950 msgr2=0x7fd7e80fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:01.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.334+0000 7fd7ec968640 1 -- 172.21.15.116:0/1868686968 shutdown_connections 2024-09-12T03:03:01.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.334+0000 7fd7ec968640 1 -- 172.21.15.116:0/1868686968 wait complete. 2024-09-12T03:03:01.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.334+0000 7fd7ec968640 1 Processor -- start 2024-09-12T03:03:01.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.334+0000 7fd7ec968640 1 -- start start 2024-09-12T03:03:01.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.335+0000 7fd7ec968640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7e8100c90 0x7fd7e81006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:01.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.335+0000 7fd7ec968640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7e8101650 0x7fd7e819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:01.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.335+0000 7fd7ec968640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7e8102040 0x7fd7e819f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:01.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.335+0000 7fd7ec968640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e810c6c0 con 0x7fd7e8101650 2024-09-12T03:03:01.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.335+0000 7fd7ec968640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e810c540 con 0x7fd7e8102040 2024-09-12T03:03:01.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.335+0000 7fd7ec968640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e810c840 con 0x7fd7e8100c90 2024-09-12T03:03:01.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.335+0000 7fd7e5d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7e8101650 0x7fd7e819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:01.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.335+0000 7fd7e5d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7e8101650 0x7fd7e819cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39126/0 (socket says 172.21.15.116:39126) 2024-09-12T03:03:01.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.335+0000 7fd7e5d9b640 1 -- 172.21.15.116:0/4254694527 learned_addr learned my addr 172.21.15.116:0/4254694527 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:01.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.335+0000 7fd7e6d9d640 1 --2- 172.21.15.116:0/4254694527 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7e8102040 0x7fd7e819f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:01.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.335+0000 7fd7e5d9b640 1 -- 172.21.15.116:0/4254694527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7e8100c90 msgr2=0x7fd7e81006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:01.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.335+0000 7fd7e5d9b640 1 --2- 172.21.15.116:0/4254694527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7e8100c90 0x7fd7e81006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:01.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.335+0000 7fd7e5d9b640 1 -- 172.21.15.116:0/4254694527 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7e8102040 msgr2=0x7fd7e819f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:01.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.335+0000 7fd7e5d9b640 1 --2- 172.21.15.116:0/4254694527 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7e8102040 0x7fd7e819f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:01.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.335+0000 7fd7e5d9b640 1 -- 172.21.15.116:0/4254694527 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7e819fb20 con 0x7fd7e8101650 2024-09-12T03:03:01.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.335+0000 7fd7e5d9b640 1 --2- 172.21.15.116:0/4254694527 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7e8101650 0x7fd7e819cbf0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd7d800e9c0 tx=0x7fd7d800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:01.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.335+0000 7fd7df7fe640 1 -- 172.21.15.116:0/4254694527 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d800cd80 con 0x7fd7e8101650 2024-09-12T03:03:01.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.335+0000 7fd7ec968640 1 -- 172.21.15.116:0/4254694527 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7e819fdb0 con 0x7fd7e8101650 2024-09-12T03:03:01.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.335+0000 7fd7df7fe640 1 -- 172.21.15.116:0/4254694527 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7d80035f0 con 0x7fd7e8101650 2024-09-12T03:03:01.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.335+0000 7fd7df7fe640 1 -- 172.21.15.116:0/4254694527 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8010690 con 0x7fd7e8101650 2024-09-12T03:03:01.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.335+0000 7fd7ec968640 1 -- 172.21.15.116:0/4254694527 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7e81a0270 con 0x7fd7e8101650 2024-09-12T03:03:01.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.336+0000 7fd7ec968640 1 -- 172.21.15.116:0/4254694527 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7e8107f10 con 0x7fd7e8101650 2024-09-12T03:03:01.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.337+0000 7fd7df7fe640 1 -- 172.21.15.116:0/4254694527 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fd7d80037a0 con 0x7fd7e8101650 2024-09-12T03:03:01.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.338+0000 7fd7df7fe640 1 --2- 172.21.15.116:0/4254694527 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd7d40a2c20 0x7fd7d40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:01.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.338+0000 7fd7e659c640 1 --2- 172.21.15.116:0/4254694527 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd7d40a2c20 0x7fd7d40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:01.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.339+0000 7fd7df7fe640 1 -- 172.21.15.116:0/4254694527 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd7d80cf6d0 con 0x7fd7e8101650 2024-09-12T03:03:01.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.339+0000 7fd7e659c640 1 --2- 172.21.15.116:0/4254694527 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd7d40a2c20 0x7fd7d40a50e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00096c0 tx=0x7fd7d0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:01.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.343+0000 7fd7df7fe640 1 -- 172.21.15.116:0/4254694527 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd7d809c500 con 0x7fd7e8101650 2024-09-12T03:03:01.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.506+0000 7fd7ec968640 1 -- 172.21.15.116:0/4254694527 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd7e8062630 con 0x7fd7d40a2c20 2024-09-12T03:03:01.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.512+0000 7fd7df7fe640 1 -- 172.21.15.116:0/4254694527 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258980 (secure 0 0 0) 0x7fd7e8062630 con 0x7fd7d40a2c20 2024-09-12T03:03:01.515 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:03:01.516 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-09-12T03:03:01.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.515+0000 7fd7ec968640 1 -- 172.21.15.116:0/4254694527 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd7d40a2c20 msgr2=0x7fd7d40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:01.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.515+0000 7fd7ec968640 1 --2- 172.21.15.116:0/4254694527 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd7d40a2c20 0x7fd7d40a50e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00096c0 tx=0x7fd7d0031400 comp rx=0 tx=0).stop 2024-09-12T03:03:01.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.515+0000 7fd7ec968640 1 -- 172.21.15.116:0/4254694527 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7e8101650 msgr2=0x7fd7e819cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:01.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.515+0000 7fd7ec968640 1 --2- 172.21.15.116:0/4254694527 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7e8101650 0x7fd7e819cbf0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd7d800e9c0 tx=0x7fd7d800ee80 comp rx=0 tx=0).stop 2024-09-12T03:03:01.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.515+0000 7fd7ec968640 1 -- 172.21.15.116:0/4254694527 shutdown_connections 2024-09-12T03:03:01.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.515+0000 7fd7ec968640 1 --2- 172.21.15.116:0/4254694527 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd7d40a2c20 0x7fd7d40a50e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:01.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.515+0000 7fd7ec968640 1 --2- 172.21.15.116:0/4254694527 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7e8102040 0x7fd7e819f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:01.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.515+0000 7fd7ec968640 1 --2- 172.21.15.116:0/4254694527 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7e8101650 0x7fd7e819cbf0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:01.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.515+0000 7fd7ec968640 1 --2- 172.21.15.116:0/4254694527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7e8100c90 0x7fd7e81006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:01.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.515+0000 7fd7ec968640 1 -- 172.21.15.116:0/4254694527 >> 172.21.15.116:0/4254694527 conn(0x7fd7e80fb950 msgr2=0x7fd7e81050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:01.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.515+0000 7fd7ec968640 1 -- 172.21.15.116:0/4254694527 shutdown_connections 2024-09-12T03:03:01.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:01.515+0000 7fd7ec968640 1 -- 172.21.15.116:0/4254694527 wait complete. 2024-09-12T03:03:01.526 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-09-12T03:03:01.136507+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665200,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":827,"num_read_kb":536,"num_write":356,"num_write_kb":3066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180875,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":786,"ondisk_log_size":786,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368392,"kb_used_data":10824,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573432,"statfs":{"total":1151940427776,"available":1151563194368,"internally_reserved":0,"allocated":11083776,"data_stored":7159566,"data_compressed":107490,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8987,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":51,"num_write":7,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003020"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.133420+0000","last_change":"2024-09-12T03:00:58.348563+0000","last_active":"2024-09-12T03:02:33.133420+0000","last_peered":"2024-09-12T03:02:33.133420+0000","last_clean":"2024-09-12T03:02:33.133420+0000","last_became_active":"2024-09-12T03:00:58.347940+0000","last_became_peered":"2024-09-12T03:00:58.347940+0000","last_unstale":"2024-09-12T03:02:33.133420+0000","last_undegraded":"2024-09-12T03:02:33.133420+0000","last_fullsized":"2024-09-12T03:02:33.133420+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:43.489238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.119217+0000","last_change":"2024-09-12T03:00:58.351969+0000","last_active":"2024-09-12T03:02:33.119217+0000","last_peered":"2024-09-12T03:02:33.119217+0000","last_clean":"2024-09-12T03:02:33.119217+0000","last_became_active":"2024-09-12T03:00:58.351679+0000","last_became_peered":"2024-09-12T03:00:58.351679+0000","last_unstale":"2024-09-12T03:02:33.119217+0000","last_undegraded":"2024-09-12T03:02:33.119217+0000","last_fullsized":"2024-09-12T03:02:33.119217+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:27:01.174890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120115+0000","last_change":"2024-09-12T03:00:58.346045+0000","last_active":"2024-09-12T03:02:33.120115+0000","last_peered":"2024-09-12T03:02:33.120115+0000","last_clean":"2024-09-12T03:02:33.120115+0000","last_became_active":"2024-09-12T03:00:58.345719+0000","last_became_peered":"2024-09-12T03:00:58.345719+0000","last_unstale":"2024-09-12T03:02:33.120115+0000","last_undegraded":"2024-09-12T03:02:33.120115+0000","last_fullsized":"2024-09-12T03:02:33.120115+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:12.312650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.111069+0000","last_change":"2024-09-12T03:00:58.348165+0000","last_active":"2024-09-12T03:02:33.111069+0000","last_peered":"2024-09-12T03:02:33.111069+0000","last_clean":"2024-09-12T03:02:33.111069+0000","last_became_active":"2024-09-12T03:00:58.347734+0000","last_became_peered":"2024-09-12T03:00:58.347734+0000","last_unstale":"2024-09-12T03:02:33.111069+0000","last_undegraded":"2024-09-12T03:02:33.111069+0000","last_fullsized":"2024-09-12T03:02:33.111069+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:05.945900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.108519+0000","last_change":"2024-09-12T03:00:58.339316+0000","last_active":"2024-09-12T03:02:33.108519+0000","last_peered":"2024-09-12T03:02:33.108519+0000","last_clean":"2024-09-12T03:02:33.108519+0000","last_became_active":"2024-09-12T03:00:58.338964+0000","last_became_peered":"2024-09-12T03:00:58.338964+0000","last_unstale":"2024-09-12T03:02:33.108519+0000","last_undegraded":"2024-09-12T03:02:33.108519+0000","last_fullsized":"2024-09-12T03:02:33.108519+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:50.721276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.110921+0000","last_change":"2024-09-12T03:00:58.346838+0000","last_active":"2024-09-12T03:02:33.110921+0000","last_peered":"2024-09-12T03:02:33.110921+0000","last_clean":"2024-09-12T03:02:33.110921+0000","last_became_active":"2024-09-12T03:00:58.346486+0000","last_became_peered":"2024-09-12T03:00:58.346486+0000","last_unstale":"2024-09-12T03:02:33.110921+0000","last_undegraded":"2024-09-12T03:02:33.110921+0000","last_fullsized":"2024-09-12T03:02:33.110921+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:28.687727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.122919+0000","last_change":"2024-09-12T03:00:58.346712+0000","last_active":"2024-09-12T03:02:33.122919+0000","last_peered":"2024-09-12T03:02:33.122919+0000","last_clean":"2024-09-12T03:02:33.122919+0000","last_became_active":"2024-09-12T03:00:58.346397+0000","last_became_peered":"2024-09-12T03:00:58.346397+0000","last_unstale":"2024-09-12T03:02:33.122919+0000","last_undegraded":"2024-09-12T03:02:33.122919+0000","last_fullsized":"2024-09-12T03:02:33.122919+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:48:00.624733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.118885+0000","last_change":"2024-09-12T03:00:58.356166+0000","last_active":"2024-09-12T03:02:33.118885+0000","last_peered":"2024-09-12T03:02:33.118885+0000","last_clean":"2024-09-12T03:02:33.118885+0000","last_became_active":"2024-09-12T03:00:58.354944+0000","last_became_peered":"2024-09-12T03:00:58.354944+0000","last_unstale":"2024-09-12T03:02:33.118885+0000","last_undegraded":"2024-09-12T03:02:33.118885+0000","last_fullsized":"2024-09-12T03:02:33.118885+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:15.840149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.133064+0000","last_change":"2024-09-12T03:00:58.340182+0000","last_active":"2024-09-12T03:02:33.133064+0000","last_peered":"2024-09-12T03:02:33.133064+0000","last_clean":"2024-09-12T03:02:33.133064+0000","last_became_active":"2024-09-12T03:00:58.339576+0000","last_became_peered":"2024-09-12T03:00:58.339576+0000","last_unstale":"2024-09-12T03:02:33.133064+0000","last_undegraded":"2024-09-12T03:02:33.133064+0000","last_fullsized":"2024-09-12T03:02:33.133064+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:16.534844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.117254+0000","last_change":"2024-09-12T03:00:58.351218+0000","last_active":"2024-09-12T03:02:33.117254+0000","last_peered":"2024-09-12T03:02:33.117254+0000","last_clean":"2024-09-12T03:02:33.117254+0000","last_became_active":"2024-09-12T03:00:58.350871+0000","last_became_peered":"2024-09-12T03:00:58.350871+0000","last_unstale":"2024-09-12T03:02:33.117254+0000","last_undegraded":"2024-09-12T03:02:33.117254+0000","last_fullsized":"2024-09-12T03:02:33.117254+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:38.367683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.130892+0000","last_change":"2024-09-12T03:00:58.351894+0000","last_active":"2024-09-12T03:02:33.130892+0000","last_peered":"2024-09-12T03:02:33.130892+0000","last_clean":"2024-09-12T03:02:33.130892+0000","last_became_active":"2024-09-12T03:00:58.351445+0000","last_became_peered":"2024-09-12T03:00:58.351445+0000","last_unstale":"2024-09-12T03:02:33.130892+0000","last_undegraded":"2024-09-12T03:02:33.130892+0000","last_fullsized":"2024-09-12T03:02:33.130892+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:42.093871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.107323+0000","last_change":"2024-09-12T03:00:58.353460+0000","last_active":"2024-09-12T03:02:33.107323+0000","last_peered":"2024-09-12T03:02:33.107323+0000","last_clean":"2024-09-12T03:02:33.107323+0000","last_became_active":"2024-09-12T03:00:58.353111+0000","last_became_peered":"2024-09-12T03:00:58.353111+0000","last_unstale":"2024-09-12T03:02:33.107323+0000","last_undegraded":"2024-09-12T03:02:33.107323+0000","last_fullsized":"2024-09-12T03:02:33.107323+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:15.916765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.111860+0000","last_change":"2024-09-12T03:00:58.341080+0000","last_active":"2024-09-12T03:02:33.111860+0000","last_peered":"2024-09-12T03:02:33.111860+0000","last_clean":"2024-09-12T03:02:33.111860+0000","last_became_active":"2024-09-12T03:00:58.340782+0000","last_became_peered":"2024-09-12T03:00:58.340782+0000","last_unstale":"2024-09-12T03:02:33.111860+0000","last_undegraded":"2024-09-12T03:02:33.111860+0000","last_fullsized":"2024-09-12T03:02:33.111860+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:36.569700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.106506+0000","last_change":"2024-09-12T03:00:58.346791+0000","last_active":"2024-09-12T03:02:33.106506+0000","last_peered":"2024-09-12T03:02:33.106506+0000","last_clean":"2024-09-12T03:02:33.106506+0000","last_became_active":"2024-09-12T03:00:58.346500+0000","last_became_peered":"2024-09-12T03:00:58.346500+0000","last_unstale":"2024-09-12T03:02:33.106506+0000","last_undegraded":"2024-09-12T03:02:33.106506+0000","last_fullsized":"2024-09-12T03:02:33.106506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:32.017589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.112821+0000","last_change":"2024-09-12T03:00:58.349522+0000","last_active":"2024-09-12T03:02:33.112821+0000","last_peered":"2024-09-12T03:02:33.112821+0000","last_clean":"2024-09-12T03:02:33.112821+0000","last_became_active":"2024-09-12T03:00:58.349246+0000","last_became_peered":"2024-09-12T03:00:58.349246+0000","last_unstale":"2024-09-12T03:02:33.112821+0000","last_undegraded":"2024-09-12T03:02:33.112821+0000","last_fullsized":"2024-09-12T03:02:33.112821+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:39.253919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.134146+0000","last_change":"2024-09-12T03:00:58.349909+0000","last_active":"2024-09-12T03:02:33.134146+0000","last_peered":"2024-09-12T03:02:33.134146+0000","last_clean":"2024-09-12T03:02:33.134146+0000","last_became_active":"2024-09-12T03:00:58.349590+0000","last_became_peered":"2024-09-12T03:00:58.349590+0000","last_unstale":"2024-09-12T03:02:33.134146+0000","last_undegraded":"2024-09-12T03:02:33.134146+0000","last_fullsized":"2024-09-12T03:02:33.134146+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:04.540709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.133565+0000","last_change":"2024-09-12T03:00:58.348284+0000","last_active":"2024-09-12T03:02:33.133565+0000","last_peered":"2024-09-12T03:02:33.133565+0000","last_clean":"2024-09-12T03:02:33.133565+0000","last_became_active":"2024-09-12T03:00:58.347465+0000","last_became_peered":"2024-09-12T03:00:58.347465+0000","last_unstale":"2024-09-12T03:02:33.133565+0000","last_undegraded":"2024-09-12T03:02:33.133565+0000","last_fullsized":"2024-09-12T03:02:33.133565+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:13.191899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.122657+0000","last_change":"2024-09-12T03:00:58.337740+0000","last_active":"2024-09-12T03:02:33.122657+0000","last_peered":"2024-09-12T03:02:33.122657+0000","last_clean":"2024-09-12T03:02:33.122657+0000","last_became_active":"2024-09-12T03:00:58.337393+0000","last_became_peered":"2024-09-12T03:00:58.337393+0000","last_unstale":"2024-09-12T03:02:33.122657+0000","last_undegraded":"2024-09-12T03:02:33.122657+0000","last_fullsized":"2024-09-12T03:02:33.122657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:28:56.339167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.111666+0000","last_change":"2024-09-12T03:00:58.345779+0000","last_active":"2024-09-12T03:02:33.111666+0000","last_peered":"2024-09-12T03:02:33.111666+0000","last_clean":"2024-09-12T03:02:33.111666+0000","last_became_active":"2024-09-12T03:00:58.345210+0000","last_became_peered":"2024-09-12T03:00:58.345210+0000","last_unstale":"2024-09-12T03:02:33.111666+0000","last_undegraded":"2024-09-12T03:02:33.111666+0000","last_fullsized":"2024-09-12T03:02:33.111666+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:06:58.271185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120653+0000","last_change":"2024-09-12T03:02:04.294381+0000","last_active":"2024-09-12T03:02:33.120653+0000","last_peered":"2024-09-12T03:02:33.120653+0000","last_clean":"2024-09-12T03:02:33.120653+0000","last_became_active":"2024-09-12T03:02:04.293795+0000","last_became_peered":"2024-09-12T03:02:04.293795+0000","last_unstale":"2024-09-12T03:02:33.120653+0000","last_undegraded":"2024-09-12T03:02:33.120653+0000","last_fullsized":"2024-09-12T03:02:33.120653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:06:58.271007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.124292+0000","last_change":"2024-09-12T03:00:58.349233+0000","last_active":"2024-09-12T03:02:33.124292+0000","last_peered":"2024-09-12T03:02:33.124292+0000","last_clean":"2024-09-12T03:02:33.124292+0000","last_became_active":"2024-09-12T03:00:58.348958+0000","last_became_peered":"2024-09-12T03:00:58.348958+0000","last_unstale":"2024-09-12T03:02:33.124292+0000","last_undegraded":"2024-09-12T03:02:33.124292+0000","last_fullsized":"2024-09-12T03:02:33.124292+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:32:10.271817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.134703+0000","last_change":"2024-09-12T03:00:58.358024+0000","last_active":"2024-09-12T03:02:33.134703+0000","last_peered":"2024-09-12T03:02:33.134703+0000","last_clean":"2024-09-12T03:02:33.134703+0000","last_became_active":"2024-09-12T03:00:58.357434+0000","last_became_peered":"2024-09-12T03:00:58.357434+0000","last_unstale":"2024-09-12T03:02:33.134703+0000","last_undegraded":"2024-09-12T03:02:33.134703+0000","last_fullsized":"2024-09-12T03:02:33.134703+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:01:20.119733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120095+0000","last_change":"2024-09-12T03:00:58.355063+0000","last_active":"2024-09-12T03:02:33.120095+0000","last_peered":"2024-09-12T03:02:33.120095+0000","last_clean":"2024-09-12T03:02:33.120095+0000","last_became_active":"2024-09-12T03:00:58.354760+0000","last_became_peered":"2024-09-12T03:00:58.354760+0000","last_unstale":"2024-09-12T03:02:33.120095+0000","last_undegraded":"2024-09-12T03:02:33.120095+0000","last_fullsized":"2024-09-12T03:02:33.120095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:13.464413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.112255+0000","last_change":"2024-09-12T03:00:58.341916+0000","last_active":"2024-09-12T03:02:33.112255+0000","last_peered":"2024-09-12T03:02:33.112255+0000","last_clean":"2024-09-12T03:02:33.112255+0000","last_became_active":"2024-09-12T03:00:58.340815+0000","last_became_peered":"2024-09-12T03:00:58.340815+0000","last_unstale":"2024-09-12T03:02:33.112255+0000","last_undegraded":"2024-09-12T03:02:33.112255+0000","last_fullsized":"2024-09-12T03:02:33.112255+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:13.777947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.117061+0000","last_change":"2024-09-12T03:00:58.357765+0000","last_active":"2024-09-12T03:02:33.117061+0000","last_peered":"2024-09-12T03:02:33.117061+0000","last_clean":"2024-09-12T03:02:33.117061+0000","last_became_active":"2024-09-12T03:00:58.357480+0000","last_became_peered":"2024-09-12T03:00:58.357480+0000","last_unstale":"2024-09-12T03:02:33.117061+0000","last_undegraded":"2024-09-12T03:02:33.117061+0000","last_fullsized":"2024-09-12T03:02:33.117061+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:26.884381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.108215+0000","last_change":"2024-09-12T03:00:58.349504+0000","last_active":"2024-09-12T03:02:33.108215+0000","last_peered":"2024-09-12T03:02:33.108215+0000","last_clean":"2024-09-12T03:02:33.108215+0000","last_became_active":"2024-09-12T03:00:58.349136+0000","last_became_peered":"2024-09-12T03:00:58.349136+0000","last_unstale":"2024-09-12T03:02:33.108215+0000","last_undegraded":"2024-09-12T03:02:33.108215+0000","last_fullsized":"2024-09-12T03:02:33.108215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:09:55.229516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.122920+0000","last_change":"2024-09-12T03:00:58.341902+0000","last_active":"2024-09-12T03:02:33.122920+0000","last_peered":"2024-09-12T03:02:33.122920+0000","last_clean":"2024-09-12T03:02:33.122920+0000","last_became_active":"2024-09-12T03:00:58.341592+0000","last_became_peered":"2024-09-12T03:00:58.341592+0000","last_unstale":"2024-09-12T03:02:33.122920+0000","last_undegraded":"2024-09-12T03:02:33.122920+0000","last_fullsized":"2024-09-12T03:02:33.122920+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:13.852511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.114521+0000","last_change":"2024-09-12T03:00:58.348012+0000","last_active":"2024-09-12T03:02:33.114521+0000","last_peered":"2024-09-12T03:02:33.114521+0000","last_clean":"2024-09-12T03:02:33.114521+0000","last_became_active":"2024-09-12T03:00:58.347338+0000","last_became_peered":"2024-09-12T03:00:58.347338+0000","last_unstale":"2024-09-12T03:02:33.114521+0000","last_undegraded":"2024-09-12T03:02:33.114521+0000","last_fullsized":"2024-09-12T03:02:33.114521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:41.478032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.112751+0000","last_change":"2024-09-12T03:00:58.345833+0000","last_active":"2024-09-12T03:02:33.112751+0000","last_peered":"2024-09-12T03:02:33.112751+0000","last_clean":"2024-09-12T03:02:33.112751+0000","last_became_active":"2024-09-12T03:00:58.345482+0000","last_became_peered":"2024-09-12T03:00:58.345482+0000","last_unstale":"2024-09-12T03:02:33.112751+0000","last_undegraded":"2024-09-12T03:02:33.112751+0000","last_fullsized":"2024-09-12T03:02:33.112751+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:32.037412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.112502+0000","last_change":"2024-09-12T03:00:58.352290+0000","last_active":"2024-09-12T03:02:33.112502+0000","last_peered":"2024-09-12T03:02:33.112502+0000","last_clean":"2024-09-12T03:02:33.112502+0000","last_became_active":"2024-09-12T03:00:58.351918+0000","last_became_peered":"2024-09-12T03:00:58.351918+0000","last_unstale":"2024-09-12T03:02:33.112502+0000","last_undegraded":"2024-09-12T03:02:33.112502+0000","last_fullsized":"2024-09-12T03:02:33.112502+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:25.941080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.114976+0000","last_change":"2024-09-12T03:00:58.348021+0000","last_active":"2024-09-12T03:02:33.114976+0000","last_peered":"2024-09-12T03:02:33.114976+0000","last_clean":"2024-09-12T03:02:33.114976+0000","last_became_active":"2024-09-12T03:00:58.347634+0000","last_became_peered":"2024-09-12T03:00:58.347634+0000","last_unstale":"2024-09-12T03:02:33.114976+0000","last_undegraded":"2024-09-12T03:02:33.114976+0000","last_fullsized":"2024-09-12T03:02:33.114976+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:47.933067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.123449+0000","last_change":"2024-09-12T03:00:58.340891+0000","last_active":"2024-09-12T03:02:33.123449+0000","last_peered":"2024-09-12T03:02:33.123449+0000","last_clean":"2024-09-12T03:02:33.123449+0000","last_became_active":"2024-09-12T03:00:58.340504+0000","last_became_peered":"2024-09-12T03:00:58.340504+0000","last_unstale":"2024-09-12T03:02:33.123449+0000","last_undegraded":"2024-09-12T03:02:33.123449+0000","last_fullsized":"2024-09-12T03:02:33.123449+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:32.117635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.108754+0000","last_change":"2024-09-12T03:00:58.339343+0000","last_active":"2024-09-12T03:02:33.108754+0000","last_peered":"2024-09-12T03:02:33.108754+0000","last_clean":"2024-09-12T03:02:33.108754+0000","last_became_active":"2024-09-12T03:00:58.339001+0000","last_became_peered":"2024-09-12T03:00:58.339001+0000","last_unstale":"2024-09-12T03:02:33.108754+0000","last_undegraded":"2024-09-12T03:02:33.108754+0000","last_fullsized":"2024-09-12T03:02:33.108754+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:02.043454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":140,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:57.563136+0000","last_change":"2024-09-12T03:00:56.315550+0000","last_active":"2024-09-12T03:02:57.563136+0000","last_peered":"2024-09-12T03:02:57.563136+0000","last_clean":"2024-09-12T03:02:57.563136+0000","last_became_active":"2024-09-12T03:00:56.315225+0000","last_became_peered":"2024-09-12T03:00:56.315225+0000","last_unstale":"2024-09-12T03:02:57.563136+0000","last_undegraded":"2024-09-12T03:02:57.563136+0000","last_fullsized":"2024-09-12T03:02:57.563136+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:46.070895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.116508+0000","last_change":"2024-09-12T03:02:04.290498+0000","last_active":"2024-09-12T03:02:33.116508+0000","last_peered":"2024-09-12T03:02:33.116508+0000","last_clean":"2024-09-12T03:02:33.116508+0000","last_became_active":"2024-09-12T03:02:04.290162+0000","last_became_peered":"2024-09-12T03:02:04.290162+0000","last_unstale":"2024-09-12T03:02:33.116508+0000","last_undegraded":"2024-09-12T03:02:33.116508+0000","last_fullsized":"2024-09-12T03:02:33.116508+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:29.777155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282566+0000","last_change":"2024-09-12T03:00:56.316385+0000","last_active":"2024-09-12T03:02:04.282566+0000","last_peered":"2024-09-12T03:02:04.282566+0000","last_clean":"2024-09-12T03:02:04.282566+0000","last_became_active":"2024-09-12T03:00:56.316064+0000","last_became_peered":"2024-09-12T03:00:56.316064+0000","last_unstale":"2024-09-12T03:02:04.282566+0000","last_undegraded":"2024-09-12T03:02:04.282566+0000","last_fullsized":"2024-09-12T03:02:04.282566+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:12:43.323654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332424+0000","last_change":"2024-09-12T03:00:56.313065+0000","last_active":"2024-09-12T03:02:04.332424+0000","last_peered":"2024-09-12T03:02:04.332424+0000","last_clean":"2024-09-12T03:02:04.332424+0000","last_became_active":"2024-09-12T03:00:56.312716+0000","last_became_peered":"2024-09-12T03:00:56.312716+0000","last_unstale":"2024-09-12T03:02:04.332424+0000","last_undegraded":"2024-09-12T03:02:04.332424+0000","last_fullsized":"2024-09-12T03:02:04.332424+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:51.089089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120152+0000","last_change":"2024-09-12T03:00:58.346542+0000","last_active":"2024-09-12T03:02:33.120152+0000","last_peered":"2024-09-12T03:02:33.120152+0000","last_clean":"2024-09-12T03:02:33.120152+0000","last_became_active":"2024-09-12T03:00:58.346260+0000","last_became_peered":"2024-09-12T03:00:58.346260+0000","last_unstale":"2024-09-12T03:02:33.120152+0000","last_undegraded":"2024-09-12T03:02:33.120152+0000","last_fullsized":"2024-09-12T03:02:33.120152+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:37.034969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289182+0000","last_change":"2024-09-12T03:00:56.317308+0000","last_active":"2024-09-12T03:02:04.289182+0000","last_peered":"2024-09-12T03:02:04.289182+0000","last_clean":"2024-09-12T03:02:04.289182+0000","last_became_active":"2024-09-12T03:00:56.316935+0000","last_became_peered":"2024-09-12T03:00:56.316935+0000","last_unstale":"2024-09-12T03:02:04.289182+0000","last_undegraded":"2024-09-12T03:02:04.289182+0000","last_fullsized":"2024-09-12T03:02:04.289182+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:06.324489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.118033+0000","last_change":"2024-09-12T03:00:58.341565+0000","last_active":"2024-09-12T03:02:33.118033+0000","last_peered":"2024-09-12T03:02:33.118033+0000","last_clean":"2024-09-12T03:02:33.118033+0000","last_became_active":"2024-09-12T03:00:58.341285+0000","last_became_peered":"2024-09-12T03:00:58.341285+0000","last_unstale":"2024-09-12T03:02:33.118033+0000","last_undegraded":"2024-09-12T03:02:33.118033+0000","last_fullsized":"2024-09-12T03:02:33.118033+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:59.156700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":138,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:59.565408+0000","last_change":"2024-09-12T03:00:56.314201+0000","last_active":"2024-09-12T03:02:59.565408+0000","last_peered":"2024-09-12T03:02:59.565408+0000","last_clean":"2024-09-12T03:02:59.565408+0000","last_became_active":"2024-09-12T03:00:56.313880+0000","last_became_peered":"2024-09-12T03:00:56.313880+0000","last_unstale":"2024-09-12T03:02:59.565408+0000","last_undegraded":"2024-09-12T03:02:59.565408+0000","last_fullsized":"2024-09-12T03:02:59.565408+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:57:12.952630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.132547+0000","last_change":"2024-09-12T03:00:58.345274+0000","last_active":"2024-09-12T03:02:33.132547+0000","last_peered":"2024-09-12T03:02:33.132547+0000","last_clean":"2024-09-12T03:02:33.132547+0000","last_became_active":"2024-09-12T03:00:58.344937+0000","last_became_peered":"2024-09-12T03:00:58.344937+0000","last_unstale":"2024-09-12T03:02:33.132547+0000","last_undegraded":"2024-09-12T03:02:33.132547+0000","last_fullsized":"2024-09-12T03:02:33.132547+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:17.288909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284614+0000","last_change":"2024-09-12T03:00:56.314709+0000","last_active":"2024-09-12T03:02:04.284614+0000","last_peered":"2024-09-12T03:02:04.284614+0000","last_clean":"2024-09-12T03:02:04.284614+0000","last_became_active":"2024-09-12T03:00:56.314349+0000","last_became_peered":"2024-09-12T03:00:56.314349+0000","last_unstale":"2024-09-12T03:02:04.284614+0000","last_undegraded":"2024-09-12T03:02:04.284614+0000","last_fullsized":"2024-09-12T03:02:04.284614+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:18:35.371661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.113713+0000","last_change":"2024-09-12T03:00:58.346055+0000","last_active":"2024-09-12T03:02:33.113713+0000","last_peered":"2024-09-12T03:02:33.113713+0000","last_clean":"2024-09-12T03:02:33.113713+0000","last_became_active":"2024-09-12T03:00:58.345701+0000","last_became_peered":"2024-09-12T03:00:58.345701+0000","last_unstale":"2024-09-12T03:02:33.113713+0000","last_undegraded":"2024-09-12T03:02:33.113713+0000","last_fullsized":"2024-09-12T03:02:33.113713+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:33.977245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.287450+0000","last_change":"2024-09-12T03:00:56.435962+0000","last_active":"2024-09-12T03:02:04.287450+0000","last_peered":"2024-09-12T03:02:04.287450+0000","last_clean":"2024-09-12T03:02:04.287450+0000","last_became_active":"2024-09-12T03:00:56.435515+0000","last_became_peered":"2024-09-12T03:00:56.435515+0000","last_unstale":"2024-09-12T03:02:04.287450+0000","last_undegraded":"2024-09-12T03:02:04.287450+0000","last_fullsized":"2024-09-12T03:02:04.287450+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:32.953436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.135045+0000","last_change":"2024-09-12T03:00:58.344239+0000","last_active":"2024-09-12T03:02:33.135045+0000","last_peered":"2024-09-12T03:02:33.135045+0000","last_clean":"2024-09-12T03:02:33.135045+0000","last_became_active":"2024-09-12T03:00:58.343876+0000","last_became_peered":"2024-09-12T03:00:58.343876+0000","last_unstale":"2024-09-12T03:02:33.135045+0000","last_undegraded":"2024-09-12T03:02:33.135045+0000","last_fullsized":"2024-09-12T03:02:33.135045+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:49.401295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'36","reported_seq":297,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:03:00.567788+0000","last_change":"2024-09-12T03:00:56.317708+0000","last_active":"2024-09-12T03:03:00.567788+0000","last_peered":"2024-09-12T03:03:00.567788+0000","last_clean":"2024-09-12T03:03:00.567788+0000","last_became_active":"2024-09-12T03:00:56.317317+0000","last_became_peered":"2024-09-12T03:00:56.317317+0000","last_unstale":"2024-09-12T03:03:00.567788+0000","last_undegraded":"2024-09-12T03:03:00.567788+0000","last_fullsized":"2024-09-12T03:03:00.567788+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:47.788186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70595,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":239,"num_read_kb":188,"num_write":37,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.123764+0000","last_change":"2024-09-12T03:00:58.343602+0000","last_active":"2024-09-12T03:02:33.123764+0000","last_peered":"2024-09-12T03:02:33.123764+0000","last_clean":"2024-09-12T03:02:33.123764+0000","last_became_active":"2024-09-12T03:00:58.343301+0000","last_became_peered":"2024-09-12T03:00:58.343301+0000","last_unstale":"2024-09-12T03:02:33.123764+0000","last_undegraded":"2024-09-12T03:02:33.123764+0000","last_fullsized":"2024-09-12T03:02:33.123764+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:04.056118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.290204+0000","last_change":"2024-09-12T03:00:56.317874+0000","last_active":"2024-09-12T03:02:04.290204+0000","last_peered":"2024-09-12T03:02:04.290204+0000","last_clean":"2024-09-12T03:02:04.290204+0000","last_became_active":"2024-09-12T03:00:56.317566+0000","last_became_peered":"2024-09-12T03:00:56.317566+0000","last_unstale":"2024-09-12T03:02:04.290204+0000","last_undegraded":"2024-09-12T03:02:04.290204+0000","last_fullsized":"2024-09-12T03:02:04.290204+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:54.203730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.121582+0000","last_change":"2024-09-12T03:00:58.351539+0000","last_active":"2024-09-12T03:02:33.121582+0000","last_peered":"2024-09-12T03:02:33.121582+0000","last_clean":"2024-09-12T03:02:33.121582+0000","last_became_active":"2024-09-12T03:00:58.351249+0000","last_became_peered":"2024-09-12T03:00:58.351249+0000","last_unstale":"2024-09-12T03:02:33.121582+0000","last_undegraded":"2024-09-12T03:02:33.121582+0000","last_fullsized":"2024-09-12T03:02:33.121582+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:40.498867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.287068+0000","last_change":"2024-09-12T03:00:56.314489+0000","last_active":"2024-09-12T03:02:04.287068+0000","last_peered":"2024-09-12T03:02:04.287068+0000","last_clean":"2024-09-12T03:02:04.287068+0000","last_became_active":"2024-09-12T03:00:56.314142+0000","last_became_peered":"2024-09-12T03:00:56.314142+0000","last_unstale":"2024-09-12T03:02:04.287068+0000","last_undegraded":"2024-09-12T03:02:04.287068+0000","last_fullsized":"2024-09-12T03:02:04.287068+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:21.541713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.123797+0000","last_change":"2024-09-12T03:00:58.348752+0000","last_active":"2024-09-12T03:02:33.123797+0000","last_peered":"2024-09-12T03:02:33.123797+0000","last_clean":"2024-09-12T03:02:33.123797+0000","last_became_active":"2024-09-12T03:00:58.348391+0000","last_became_peered":"2024-09-12T03:00:58.348391+0000","last_unstale":"2024-09-12T03:02:33.123797+0000","last_undegraded":"2024-09-12T03:02:33.123797+0000","last_fullsized":"2024-09-12T03:02:33.123797+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:33:15.626306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'4","reported_seq":141,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:58.565339+0000","last_change":"2024-09-12T03:00:56.311702+0000","last_active":"2024-09-12T03:02:58.565339+0000","last_peered":"2024-09-12T03:02:58.565339+0000","last_clean":"2024-09-12T03:02:58.565339+0000","last_became_active":"2024-09-12T03:00:56.311353+0000","last_became_peered":"2024-09-12T03:00:56.311353+0000","last_unstale":"2024-09-12T03:02:58.565339+0000","last_undegraded":"2024-09-12T03:02:58.565339+0000","last_fullsized":"2024-09-12T03:02:58.565339+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:01.004858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.123076+0000","last_change":"2024-09-12T03:00:58.351973+0000","last_active":"2024-09-12T03:02:33.123076+0000","last_peered":"2024-09-12T03:02:33.123076+0000","last_clean":"2024-09-12T03:02:33.123076+0000","last_became_active":"2024-09-12T03:00:58.351599+0000","last_became_peered":"2024-09-12T03:00:58.351599+0000","last_unstale":"2024-09-12T03:02:33.123076+0000","last_undegraded":"2024-09-12T03:02:33.123076+0000","last_fullsized":"2024-09-12T03:02:33.123076+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:26.398828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.291749+0000","last_change":"2024-09-12T03:00:56.436185+0000","last_active":"2024-09-12T03:02:04.291749+0000","last_peered":"2024-09-12T03:02:04.291749+0000","last_clean":"2024-09-12T03:02:04.291749+0000","last_became_active":"2024-09-12T03:00:56.435717+0000","last_became_peered":"2024-09-12T03:00:56.435717+0000","last_unstale":"2024-09-12T03:02:04.291749+0000","last_undegraded":"2024-09-12T03:02:04.291749+0000","last_fullsized":"2024-09-12T03:02:04.291749+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:27.303182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.119766+0000","last_change":"2024-09-12T03:02:04.333661+0000","last_active":"2024-09-12T03:02:33.119766+0000","last_peered":"2024-09-12T03:02:33.119766+0000","last_clean":"2024-09-12T03:02:33.119766+0000","last_became_active":"2024-09-12T03:02:04.333217+0000","last_became_peered":"2024-09-12T03:02:04.333217+0000","last_unstale":"2024-09-12T03:02:33.119766+0000","last_undegraded":"2024-09-12T03:02:33.119766+0000","last_fullsized":"2024-09-12T03:02:33.119766+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:23:47.212734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.297341+0000","last_change":"2024-09-12T03:02:04.296884+0000","last_active":"2024-09-12T03:02:04.297341+0000","last_peered":"2024-09-12T03:02:04.297341+0000","last_clean":"2024-09-12T03:02:04.297341+0000","last_became_active":"2024-09-12T03:02:04.290894+0000","last_became_peered":"2024-09-12T03:02:04.290894+0000","last_unstale":"2024-09-12T03:02:04.297341+0000","last_undegraded":"2024-09-12T03:02:04.297341+0000","last_fullsized":"2024-09-12T03:02:04.297341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:49:19.749568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.118228+0000","last_change":"2024-09-12T03:02:04.334096+0000","last_active":"2024-09-12T03:02:33.118228+0000","last_peered":"2024-09-12T03:02:33.118228+0000","last_clean":"2024-09-12T03:02:33.118228+0000","last_became_active":"2024-09-12T03:02:04.333641+0000","last_became_peered":"2024-09-12T03:02:04.333641+0000","last_unstale":"2024-09-12T03:02:33.118228+0000","last_undegraded":"2024-09-12T03:02:33.118228+0000","last_fullsized":"2024-09-12T03:02:33.118228+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:24.085293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332966+0000","last_change":"2024-09-12T03:00:56.314182+0000","last_active":"2024-09-12T03:02:04.332966+0000","last_peered":"2024-09-12T03:02:04.332966+0000","last_clean":"2024-09-12T03:02:04.332966+0000","last_became_active":"2024-09-12T03:00:56.313749+0000","last_became_peered":"2024-09-12T03:00:56.313749+0000","last_unstale":"2024-09-12T03:02:04.332966+0000","last_undegraded":"2024-09-12T03:02:04.332966+0000","last_fullsized":"2024-09-12T03:02:04.332966+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:31:45.767500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.125121+0000","last_change":"2024-09-12T03:00:58.344356+0000","last_active":"2024-09-12T03:02:33.125121+0000","last_peered":"2024-09-12T03:02:33.125121+0000","last_clean":"2024-09-12T03:02:33.125121+0000","last_became_active":"2024-09-12T03:00:58.344035+0000","last_became_peered":"2024-09-12T03:00:58.344035+0000","last_unstale":"2024-09-12T03:02:33.125121+0000","last_undegraded":"2024-09-12T03:02:33.125121+0000","last_fullsized":"2024-09-12T03:02:33.125121+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:45:18.681018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.287585+0000","last_change":"2024-09-12T03:00:56.315653+0000","last_active":"2024-09-12T03:02:04.287585+0000","last_peered":"2024-09-12T03:02:04.287585+0000","last_clean":"2024-09-12T03:02:04.287585+0000","last_became_active":"2024-09-12T03:00:56.315284+0000","last_became_peered":"2024-09-12T03:00:56.315284+0000","last_unstale":"2024-09-12T03:02:04.287585+0000","last_undegraded":"2024-09-12T03:02:04.287585+0000","last_fullsized":"2024-09-12T03:02:04.287585+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:42.934020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.122059+0000","last_change":"2024-09-12T03:02:04.291128+0000","last_active":"2024-09-12T03:02:33.122059+0000","last_peered":"2024-09-12T03:02:33.122059+0000","last_clean":"2024-09-12T03:02:33.122059+0000","last_became_active":"2024-09-12T03:02:04.287836+0000","last_became_peered":"2024-09-12T03:02:04.287836+0000","last_unstale":"2024-09-12T03:02:33.122059+0000","last_undegraded":"2024-09-12T03:02:33.122059+0000","last_fullsized":"2024-09-12T03:02:33.122059+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:23.956579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.290577+0000","last_change":"2024-09-12T03:00:56.746780+0000","last_active":"2024-09-12T03:02:04.290577+0000","last_peered":"2024-09-12T03:02:04.290577+0000","last_clean":"2024-09-12T03:02:04.290577+0000","last_became_active":"2024-09-12T03:00:56.746414+0000","last_became_peered":"2024-09-12T03:00:56.746414+0000","last_unstale":"2024-09-12T03:02:04.290577+0000","last_undegraded":"2024-09-12T03:02:04.290577+0000","last_fullsized":"2024-09-12T03:02:04.290577+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:15:35.214196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.131100+0000","last_change":"2024-09-12T03:00:58.351713+0000","last_active":"2024-09-12T03:02:33.131100+0000","last_peered":"2024-09-12T03:02:33.131100+0000","last_clean":"2024-09-12T03:02:33.131100+0000","last_became_active":"2024-09-12T03:00:58.351371+0000","last_became_peered":"2024-09-12T03:00:58.351371+0000","last_unstale":"2024-09-12T03:02:33.131100+0000","last_undegraded":"2024-09-12T03:02:33.131100+0000","last_fullsized":"2024-09-12T03:02:33.131100+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:13.346956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'137","reported_seq":336,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284318+0000","last_change":"2024-09-12T02:59:45.461482+0000","last_active":"2024-09-12T03:02:04.284318+0000","last_peered":"2024-09-12T03:02:04.284318+0000","last_clean":"2024-09-12T03:02:04.284318+0000","last_became_active":"2024-09-12T02:59:45.461058+0000","last_became_peered":"2024-09-12T02:59:45.461058+0000","last_unstale":"2024-09-12T03:02:04.284318+0000","last_undegraded":"2024-09-12T03:02:04.284318+0000","last_fullsized":"2024-09-12T03:02:04.284318+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:18.034438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:18.034438+0000","last_clean_scrub_stamp":"2024-09-12T02:56:18.034438+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:09.251858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289291+0000","last_change":"2024-09-12T03:00:56.315245+0000","last_active":"2024-09-12T03:02:04.289291+0000","last_peered":"2024-09-12T03:02:04.289291+0000","last_clean":"2024-09-12T03:02:04.289291+0000","last_became_active":"2024-09-12T03:00:56.314887+0000","last_became_peered":"2024-09-12T03:00:56.314887+0000","last_unstale":"2024-09-12T03:02:04.289291+0000","last_undegraded":"2024-09-12T03:02:04.289291+0000","last_fullsized":"2024-09-12T03:02:04.289291+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:48.750657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.112960+0000","last_change":"2024-09-12T03:00:58.357874+0000","last_active":"2024-09-12T03:02:33.112960+0000","last_peered":"2024-09-12T03:02:33.112960+0000","last_clean":"2024-09-12T03:02:33.112960+0000","last_became_active":"2024-09-12T03:00:58.357575+0000","last_became_peered":"2024-09-12T03:00:58.357575+0000","last_unstale":"2024-09-12T03:02:33.112960+0000","last_undegraded":"2024-09-12T03:02:33.112960+0000","last_fullsized":"2024-09-12T03:02:33.112960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:46.125620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284282+0000","last_change":"2024-09-12T03:00:56.312263+0000","last_active":"2024-09-12T03:02:04.284282+0000","last_peered":"2024-09-12T03:02:04.284282+0000","last_clean":"2024-09-12T03:02:04.284282+0000","last_became_active":"2024-09-12T03:00:56.311894+0000","last_became_peered":"2024-09-12T03:00:56.311894+0000","last_unstale":"2024-09-12T03:02:04.284282+0000","last_undegraded":"2024-09-12T03:02:04.284282+0000","last_fullsized":"2024-09-12T03:02:04.284282+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:52.434203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.110253+0000","last_change":"2024-09-12T03:00:58.350762+0000","last_active":"2024-09-12T03:02:33.110253+0000","last_peered":"2024-09-12T03:02:33.110253+0000","last_clean":"2024-09-12T03:02:33.110253+0000","last_became_active":"2024-09-12T03:00:58.348038+0000","last_became_peered":"2024-09-12T03:00:58.348038+0000","last_unstale":"2024-09-12T03:02:33.110253+0000","last_undegraded":"2024-09-12T03:02:33.110253+0000","last_fullsized":"2024-09-12T03:02:33.110253+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:08.916636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":134,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:54.557217+0000","last_change":"2024-09-12T03:00:56.435641+0000","last_active":"2024-09-12T03:02:54.557217+0000","last_peered":"2024-09-12T03:02:54.557217+0000","last_clean":"2024-09-12T03:02:54.557217+0000","last_became_active":"2024-09-12T03:00:56.435319+0000","last_became_peered":"2024-09-12T03:00:56.435319+0000","last_unstale":"2024-09-12T03:02:54.557217+0000","last_undegraded":"2024-09-12T03:02:54.557217+0000","last_fullsized":"2024-09-12T03:02:54.557217+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:56.894978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.112818+0000","last_change":"2024-09-12T03:00:58.344708+0000","last_active":"2024-09-12T03:02:33.112818+0000","last_peered":"2024-09-12T03:02:33.112818+0000","last_clean":"2024-09-12T03:02:33.112818+0000","last_became_active":"2024-09-12T03:00:58.344389+0000","last_became_peered":"2024-09-12T03:00:58.344389+0000","last_unstale":"2024-09-12T03:02:33.112818+0000","last_undegraded":"2024-09-12T03:02:33.112818+0000","last_fullsized":"2024-09-12T03:02:33.112818+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:08.160383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284317+0000","last_change":"2024-09-12T03:00:56.317068+0000","last_active":"2024-09-12T03:02:04.284317+0000","last_peered":"2024-09-12T03:02:04.284317+0000","last_clean":"2024-09-12T03:02:04.284317+0000","last_became_active":"2024-09-12T03:00:56.316698+0000","last_became_peered":"2024-09-12T03:00:56.316698+0000","last_unstale":"2024-09-12T03:02:04.284317+0000","last_undegraded":"2024-09-12T03:02:04.284317+0000","last_fullsized":"2024-09-12T03:02:04.284317+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:41:19.199418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.109469+0000","last_change":"2024-09-12T03:00:58.347814+0000","last_active":"2024-09-12T03:02:33.109469+0000","last_peered":"2024-09-12T03:02:33.109469+0000","last_clean":"2024-09-12T03:02:33.109469+0000","last_became_active":"2024-09-12T03:00:58.347112+0000","last_became_peered":"2024-09-12T03:00:58.347112+0000","last_unstale":"2024-09-12T03:02:33.109469+0000","last_undegraded":"2024-09-12T03:02:33.109469+0000","last_fullsized":"2024-09-12T03:02:33.109469+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:59.325113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284265+0000","last_change":"2024-09-12T03:00:56.435771+0000","last_active":"2024-09-12T03:02:04.284265+0000","last_peered":"2024-09-12T03:02:04.284265+0000","last_clean":"2024-09-12T03:02:04.284265+0000","last_became_active":"2024-09-12T03:00:56.435448+0000","last_became_peered":"2024-09-12T03:00:56.435448+0000","last_unstale":"2024-09-12T03:02:04.284265+0000","last_undegraded":"2024-09-12T03:02:04.284265+0000","last_fullsized":"2024-09-12T03:02:04.284265+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:33.650338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.117580+0000","last_change":"2024-09-12T03:00:58.341071+0000","last_active":"2024-09-12T03:02:33.117580+0000","last_peered":"2024-09-12T03:02:33.117580+0000","last_clean":"2024-09-12T03:02:33.117580+0000","last_became_active":"2024-09-12T03:00:58.340753+0000","last_became_peered":"2024-09-12T03:00:58.340753+0000","last_unstale":"2024-09-12T03:02:33.117580+0000","last_undegraded":"2024-09-12T03:02:33.117580+0000","last_fullsized":"2024-09-12T03:02:33.117580+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:21.217915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292254+0000","last_change":"2024-09-12T03:00:56.436058+0000","last_active":"2024-09-12T03:02:04.292254+0000","last_peered":"2024-09-12T03:02:04.292254+0000","last_clean":"2024-09-12T03:02:04.292254+0000","last_became_active":"2024-09-12T03:00:56.435518+0000","last_became_peered":"2024-09-12T03:00:56.435518+0000","last_unstale":"2024-09-12T03:02:04.292254+0000","last_undegraded":"2024-09-12T03:02:04.292254+0000","last_fullsized":"2024-09-12T03:02:04.292254+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:32.235097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.110650+0000","last_change":"2024-09-12T03:00:58.348236+0000","last_active":"2024-09-12T03:02:33.110650+0000","last_peered":"2024-09-12T03:02:33.110650+0000","last_clean":"2024-09-12T03:02:33.110650+0000","last_became_active":"2024-09-12T03:00:58.347727+0000","last_became_peered":"2024-09-12T03:00:58.347727+0000","last_unstale":"2024-09-12T03:02:33.110650+0000","last_undegraded":"2024-09-12T03:02:33.110650+0000","last_fullsized":"2024-09-12T03:02:33.110650+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:19.233163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'12","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.354650+0000","last_change":"2024-09-12T03:00:56.310537+0000","last_active":"2024-09-12T03:02:33.354650+0000","last_peered":"2024-09-12T03:02:33.354650+0000","last_clean":"2024-09-12T03:02:33.354650+0000","last_became_active":"2024-09-12T03:00:56.310135+0000","last_became_peered":"2024-09-12T03:00:56.310135+0000","last_unstale":"2024-09-12T03:02:33.354650+0000","last_undegraded":"2024-09-12T03:02:33.354650+0000","last_fullsized":"2024-09-12T03:02:33.354650+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:10:55.584339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.121993+0000","last_change":"2024-09-12T03:00:58.358417+0000","last_active":"2024-09-12T03:02:33.121993+0000","last_peered":"2024-09-12T03:02:33.121993+0000","last_clean":"2024-09-12T03:02:33.121993+0000","last_became_active":"2024-09-12T03:00:58.358119+0000","last_became_peered":"2024-09-12T03:00:58.358119+0000","last_unstale":"2024-09-12T03:02:33.121993+0000","last_undegraded":"2024-09-12T03:02:33.121993+0000","last_fullsized":"2024-09-12T03:02:33.121993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:10.897594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289526+0000","last_change":"2024-09-12T03:00:56.747044+0000","last_active":"2024-09-12T03:02:04.289526+0000","last_peered":"2024-09-12T03:02:04.289526+0000","last_clean":"2024-09-12T03:02:04.289526+0000","last_became_active":"2024-09-12T03:00:56.746699+0000","last_became_peered":"2024-09-12T03:00:56.746699+0000","last_unstale":"2024-09-12T03:02:04.289526+0000","last_undegraded":"2024-09-12T03:02:04.289526+0000","last_fullsized":"2024-09-12T03:02:04.289526+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:11.420109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.113058+0000","last_change":"2024-09-12T03:00:58.346854+0000","last_active":"2024-09-12T03:02:33.113058+0000","last_peered":"2024-09-12T03:02:33.113058+0000","last_clean":"2024-09-12T03:02:33.113058+0000","last_became_active":"2024-09-12T03:00:58.346535+0000","last_became_peered":"2024-09-12T03:00:58.346535+0000","last_unstale":"2024-09-12T03:02:33.113058+0000","last_undegraded":"2024-09-12T03:02:33.113058+0000","last_fullsized":"2024-09-12T03:02:33.113058+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:46.637914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332365+0000","last_change":"2024-09-12T03:00:56.435957+0000","last_active":"2024-09-12T03:02:04.332365+0000","last_peered":"2024-09-12T03:02:04.332365+0000","last_clean":"2024-09-12T03:02:04.332365+0000","last_became_active":"2024-09-12T03:00:56.435603+0000","last_became_peered":"2024-09-12T03:00:56.435603+0000","last_unstale":"2024-09-12T03:02:04.332365+0000","last_undegraded":"2024-09-12T03:02:04.332365+0000","last_fullsized":"2024-09-12T03:02:04.332365+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:34.553189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.089792+0000","last_change":"2024-09-12T03:00:58.337784+0000","last_active":"2024-09-12T03:02:33.089792+0000","last_peered":"2024-09-12T03:02:33.089792+0000","last_clean":"2024-09-12T03:02:33.089792+0000","last_became_active":"2024-09-12T03:00:58.337415+0000","last_became_peered":"2024-09-12T03:00:58.337415+0000","last_unstale":"2024-09-12T03:02:33.089792+0000","last_undegraded":"2024-09-12T03:02:33.089792+0000","last_fullsized":"2024-09-12T03:02:33.089792+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:38:15.557067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282354+0000","last_change":"2024-09-12T03:00:56.315356+0000","last_active":"2024-09-12T03:02:04.282354+0000","last_peered":"2024-09-12T03:02:04.282354+0000","last_clean":"2024-09-12T03:02:04.282354+0000","last_became_active":"2024-09-12T03:00:56.314873+0000","last_became_peered":"2024-09-12T03:00:56.314873+0000","last_unstale":"2024-09-12T03:02:04.282354+0000","last_undegraded":"2024-09-12T03:02:04.282354+0000","last_fullsized":"2024-09-12T03:02:04.282354+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:55.710156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.118974+0000","last_change":"2024-09-12T03:00:58.346706+0000","last_active":"2024-09-12T03:02:33.118974+0000","last_peered":"2024-09-12T03:02:33.118974+0000","last_clean":"2024-09-12T03:02:33.118974+0000","last_became_active":"2024-09-12T03:00:58.346363+0000","last_became_peered":"2024-09-12T03:00:58.346363+0000","last_unstale":"2024-09-12T03:02:33.118974+0000","last_undegraded":"2024-09-12T03:02:33.118974+0000","last_fullsized":"2024-09-12T03:02:33.118974+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:13.468639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.299582+0000","last_change":"2024-09-12T03:02:04.299147+0000","last_active":"2024-09-12T03:02:04.299582+0000","last_peered":"2024-09-12T03:02:04.299582+0000","last_clean":"2024-09-12T03:02:04.299582+0000","last_became_active":"2024-09-12T03:02:04.293211+0000","last_became_peered":"2024-09-12T03:02:04.293211+0000","last_unstale":"2024-09-12T03:02:04.299582+0000","last_undegraded":"2024-09-12T03:02:04.299582+0000","last_fullsized":"2024-09-12T03:02:04.299582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:01:39.222476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.118849+0000","last_change":"2024-09-12T03:00:58.356682+0000","last_active":"2024-09-12T03:02:33.118849+0000","last_peered":"2024-09-12T03:02:33.118849+0000","last_clean":"2024-09-12T03:02:33.118849+0000","last_became_active":"2024-09-12T03:00:58.356383+0000","last_became_peered":"2024-09-12T03:00:58.356383+0000","last_unstale":"2024-09-12T03:02:33.118849+0000","last_undegraded":"2024-09-12T03:02:33.118849+0000","last_fullsized":"2024-09-12T03:02:33.118849+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:14:16.428393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292053+0000","last_change":"2024-09-12T03:00:56.314943+0000","last_active":"2024-09-12T03:02:04.292053+0000","last_peered":"2024-09-12T03:02:04.292053+0000","last_clean":"2024-09-12T03:02:04.292053+0000","last_became_active":"2024-09-12T03:00:56.314602+0000","last_became_peered":"2024-09-12T03:00:56.314602+0000","last_unstale":"2024-09-12T03:02:04.292053+0000","last_undegraded":"2024-09-12T03:02:04.292053+0000","last_fullsized":"2024-09-12T03:02:04.292053+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:31.207880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.131538+0000","last_change":"2024-09-12T03:00:58.351774+0000","last_active":"2024-09-12T03:02:33.131538+0000","last_peered":"2024-09-12T03:02:33.131538+0000","last_clean":"2024-09-12T03:02:33.131538+0000","last_became_active":"2024-09-12T03:00:58.351453+0000","last_became_peered":"2024-09-12T03:00:58.351453+0000","last_unstale":"2024-09-12T03:02:33.131538+0000","last_undegraded":"2024-09-12T03:02:33.131538+0000","last_fullsized":"2024-09-12T03:02:33.131538+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:18:24.260110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.130930+0000","last_change":"2024-09-12T03:00:58.346886+0000","last_active":"2024-09-12T03:02:33.130930+0000","last_peered":"2024-09-12T03:02:33.130930+0000","last_clean":"2024-09-12T03:02:33.130930+0000","last_became_active":"2024-09-12T03:00:58.346499+0000","last_became_peered":"2024-09-12T03:00:58.346499+0000","last_unstale":"2024-09-12T03:02:33.130930+0000","last_undegraded":"2024-09-12T03:02:33.130930+0000","last_fullsized":"2024-09-12T03:02:33.130930+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:02:24.045610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292497+0000","last_change":"2024-09-12T03:00:56.317443+0000","last_active":"2024-09-12T03:02:04.292497+0000","last_peered":"2024-09-12T03:02:04.292497+0000","last_clean":"2024-09-12T03:02:04.292497+0000","last_became_active":"2024-09-12T03:00:56.317045+0000","last_became_peered":"2024-09-12T03:00:56.317045+0000","last_unstale":"2024-09-12T03:02:04.292497+0000","last_undegraded":"2024-09-12T03:02:04.292497+0000","last_fullsized":"2024-09-12T03:02:04.292497+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:07.687606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.132782+0000","last_change":"2024-09-12T03:00:58.337257+0000","last_active":"2024-09-12T03:02:33.132782+0000","last_peered":"2024-09-12T03:02:33.132782+0000","last_clean":"2024-09-12T03:02:33.132782+0000","last_became_active":"2024-09-12T03:00:58.336847+0000","last_became_peered":"2024-09-12T03:00:58.336847+0000","last_unstale":"2024-09-12T03:02:33.132782+0000","last_undegraded":"2024-09-12T03:02:33.132782+0000","last_fullsized":"2024-09-12T03:02:33.132782+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:22.846206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292165+0000","last_change":"2024-09-12T03:00:56.316255+0000","last_active":"2024-09-12T03:02:04.292165+0000","last_peered":"2024-09-12T03:02:04.292165+0000","last_clean":"2024-09-12T03:02:04.292165+0000","last_became_active":"2024-09-12T03:00:56.315906+0000","last_became_peered":"2024-09-12T03:00:56.315906+0000","last_unstale":"2024-09-12T03:02:04.292165+0000","last_undegraded":"2024-09-12T03:02:04.292165+0000","last_fullsized":"2024-09-12T03:02:04.292165+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:14.798601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120273+0000","last_change":"2024-09-12T03:00:58.347206+0000","last_active":"2024-09-12T03:02:33.120273+0000","last_peered":"2024-09-12T03:02:33.120273+0000","last_clean":"2024-09-12T03:02:33.120273+0000","last_became_active":"2024-09-12T03:00:58.346923+0000","last_became_peered":"2024-09-12T03:00:58.346923+0000","last_unstale":"2024-09-12T03:02:33.120273+0000","last_undegraded":"2024-09-12T03:02:33.120273+0000","last_fullsized":"2024-09-12T03:02:33.120273+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:41.376211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.283743+0000","last_change":"2024-09-12T03:00:56.435431+0000","last_active":"2024-09-12T03:02:04.283743+0000","last_peered":"2024-09-12T03:02:04.283743+0000","last_clean":"2024-09-12T03:02:04.283743+0000","last_became_active":"2024-09-12T03:00:56.435088+0000","last_became_peered":"2024-09-12T03:00:56.435088+0000","last_unstale":"2024-09-12T03:02:04.283743+0000","last_undegraded":"2024-09-12T03:02:04.283743+0000","last_fullsized":"2024-09-12T03:02:04.283743+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:24.738340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.114461+0000","last_change":"2024-09-12T03:00:58.351737+0000","last_active":"2024-09-12T03:02:33.114461+0000","last_peered":"2024-09-12T03:02:33.114461+0000","last_clean":"2024-09-12T03:02:33.114461+0000","last_became_active":"2024-09-12T03:00:58.351407+0000","last_became_peered":"2024-09-12T03:00:58.351407+0000","last_unstale":"2024-09-12T03:02:33.114461+0000","last_undegraded":"2024-09-12T03:02:33.114461+0000","last_fullsized":"2024-09-12T03:02:33.114461+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:34.206703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284979+0000","last_change":"2024-09-12T03:00:56.310910+0000","last_active":"2024-09-12T03:02:04.284979+0000","last_peered":"2024-09-12T03:02:04.284979+0000","last_clean":"2024-09-12T03:02:04.284979+0000","last_became_active":"2024-09-12T03:00:56.310657+0000","last_became_peered":"2024-09-12T03:00:56.310657+0000","last_unstale":"2024-09-12T03:02:04.284979+0000","last_undegraded":"2024-09-12T03:02:04.284979+0000","last_fullsized":"2024-09-12T03:02:04.284979+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:06.117215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74598,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":701,"num_read_kb":427,"num_write":173,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":446464,"data_stored":218754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245899,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645350,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":431835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906987,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":1094282,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070512,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":432166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299446,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":431947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528378,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":1023901,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":40,"seq":171798691903,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":880640,"data_stored":431438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":33,"seq":141733920836,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":1023192,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051658,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":432268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280590,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":431632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476819,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":494890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607641,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":499306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":432709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":72566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:03:01.531 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:03:01.809 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:02.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.269+0000 7f33cdd6c640 1 -- 172.21.15.116:0/1874835269 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33c8100e20 msgr2=0x7f33c8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:02.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.269+0000 7f33cdd6c640 1 --2- 172.21.15.116:0/1874835269 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33c8100e20 0x7f33c8101200 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f33b8009ca0 tx=0x7f33b802a950 comp rx=0 tx=0).stop 2024-09-12T03:03:02.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.269+0000 7f33cdd6c640 1 -- 172.21.15.116:0/1874835269 shutdown_connections 2024-09-12T03:03:02.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.269+0000 7f33cdd6c640 1 --2- 172.21.15.116:0/1874835269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33c8102ef0 0x7f33c810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:02.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.269+0000 7f33cdd6c640 1 --2- 172.21.15.116:0/1874835269 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33c8102570 0x7f33c81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:02.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.269+0000 7f33cdd6c640 1 --2- 172.21.15.116:0/1874835269 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33c8100e20 0x7f33c8101200 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:02.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.269+0000 7f33cdd6c640 1 -- 172.21.15.116:0/1874835269 >> 172.21.15.116:0/1874835269 conn(0x7f33c80fc7d0 msgr2=0x7f33c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:02.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.270+0000 7f33cdd6c640 1 -- 172.21.15.116:0/1874835269 shutdown_connections 2024-09-12T03:03:02.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.270+0000 7f33cdd6c640 1 -- 172.21.15.116:0/1874835269 wait complete. 2024-09-12T03:03:02.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.270+0000 7f33cdd6c640 1 Processor -- start 2024-09-12T03:03:02.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.270+0000 7f33cdd6c640 1 -- start start 2024-09-12T03:03:02.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.270+0000 7f33cdd6c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33c8100e20 0x7f33c81984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:02.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.270+0000 7f33cdd6c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33c8102570 0x7f33c8198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:02.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.270+0000 7f33cdd6c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33c8102ef0 0x7f33c819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:02.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.270+0000 7f33cdd6c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f33c8112060 con 0x7f33c8102570 2024-09-12T03:03:02.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.270+0000 7f33cdd6c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f33c8111ee0 con 0x7f33c8100e20 2024-09-12T03:03:02.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.270+0000 7f33cdd6c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f33c81121e0 con 0x7f33c8102ef0 2024-09-12T03:03:02.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.271+0000 7f33c6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33c8102570 0x7f33c8198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:02.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.271+0000 7f33c6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33c8102570 0x7f33c8198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39150/0 (socket says 172.21.15.116:39150) 2024-09-12T03:03:02.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.271+0000 7f33c6ffd640 1 -- 172.21.15.116:0/1723348490 learned_addr learned my addr 172.21.15.116:0/1723348490 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:02.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.271+0000 7f33c7fff640 1 --2- 172.21.15.116:0/1723348490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33c8102ef0 0x7f33c819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:02.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.271+0000 7f33c77fe640 1 --2- 172.21.15.116:0/1723348490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33c8100e20 0x7f33c81984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:02.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.271+0000 7f33c6ffd640 1 -- 172.21.15.116:0/1723348490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33c8102ef0 msgr2=0x7f33c819cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:02.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.271+0000 7f33c6ffd640 1 --2- 172.21.15.116:0/1723348490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33c8102ef0 0x7f33c819cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:02.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.271+0000 7f33c6ffd640 1 -- 172.21.15.116:0/1723348490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33c8100e20 msgr2=0x7f33c81984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:02.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.271+0000 7f33c6ffd640 1 --2- 172.21.15.116:0/1723348490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33c8100e20 0x7f33c81984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:02.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.271+0000 7f33c6ffd640 1 -- 172.21.15.116:0/1723348490 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c819d470 con 0x7f33c8102570 2024-09-12T03:03:02.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.271+0000 7f33c77fe640 1 --2- 172.21.15.116:0/1723348490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33c8100e20 0x7f33c81984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:03:02.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.271+0000 7f33c6ffd640 1 --2- 172.21.15.116:0/1723348490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33c8102570 0x7f33c8198a00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00d950 tx=0x7f33bc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:02.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.271+0000 7f33c4ff9640 1 -- 172.21.15.116:0/1723348490 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc017070 con 0x7f33c8102570 2024-09-12T03:03:02.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.271+0000 7f33c4ff9640 1 -- 172.21.15.116:0/1723348490 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33bc009c30 con 0x7f33c8102570 2024-09-12T03:03:02.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.271+0000 7f33cdd6c640 1 -- 172.21.15.116:0/1723348490 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c819d760 con 0x7f33c8102570 2024-09-12T03:03:02.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.271+0000 7f33c4ff9640 1 -- 172.21.15.116:0/1723348490 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc00f6b0 con 0x7f33c8102570 2024-09-12T03:03:02.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.271+0000 7f33cdd6c640 1 -- 172.21.15.116:0/1723348490 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c81a5040 con 0x7f33c8102570 2024-09-12T03:03:02.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.272+0000 7f33cdd6c640 1 -- 172.21.15.116:0/1723348490 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c8101200 con 0x7f33c8102570 2024-09-12T03:03:02.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.274+0000 7f33c4ff9640 1 -- 172.21.15.116:0/1723348490 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f33bc0109d0 con 0x7f33c8102570 2024-09-12T03:03:02.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.274+0000 7f33c4ff9640 1 --2- 172.21.15.116:0/1723348490 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f33b00a2c10 0x7f33b00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:02.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.274+0000 7f33c77fe640 1 --2- 172.21.15.116:0/1723348490 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f33b00a2c10 0x7f33b00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:02.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.274+0000 7f33c4ff9640 1 -- 172.21.15.116:0/1723348490 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f33bc0cee30 con 0x7f33c8102570 2024-09-12T03:03:02.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.275+0000 7f33c77fe640 1 --2- 172.21.15.116:0/1723348490 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f33b00a2c10 0x7f33b00a50d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f33b8006010 tx=0x7f33b8005f80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:02.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.278+0000 7f33c4ff9640 1 -- 172.21.15.116:0/1723348490 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f33bc009de0 con 0x7f33c8102570 2024-09-12T03:03:02.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.480+0000 7f33cdd6c640 1 -- 172.21.15.116:0/1723348490 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33c81a5240 con 0x7f33c8102570 2024-09-12T03:03:02.481 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:02 smithi116 ceph-mon[31410]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:03:02.482 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:02 smithi116 ceph-mon[31410]: pgmap v339: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-12T03:03:02.482 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:02 smithi116 ceph-mon[31410]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:03:02.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.480+0000 7f33c4ff9640 1 -- 172.21.15.116:0/1723348490 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f33bc09bc60 con 0x7f33c8102570 2024-09-12T03:03:02.482 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:03:02.483 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:03:02.483 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:03:02.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.483+0000 7f33cdd6c640 1 -- 172.21.15.116:0/1723348490 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f33b00a2c10 msgr2=0x7f33b00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:02.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.483+0000 7f33cdd6c640 1 --2- 172.21.15.116:0/1723348490 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f33b00a2c10 0x7f33b00a50d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f33b8006010 tx=0x7f33b8005f80 comp rx=0 tx=0).stop 2024-09-12T03:03:02.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.483+0000 7f33cdd6c640 1 -- 172.21.15.116:0/1723348490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33c8102570 msgr2=0x7f33c8198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:02.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.483+0000 7f33cdd6c640 1 --2- 172.21.15.116:0/1723348490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33c8102570 0x7f33c8198a00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00d950 tx=0x7f33bc00de10 comp rx=0 tx=0).stop 2024-09-12T03:03:02.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.483+0000 7f33cdd6c640 1 -- 172.21.15.116:0/1723348490 shutdown_connections 2024-09-12T03:03:02.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.484+0000 7f33cdd6c640 1 --2- 172.21.15.116:0/1723348490 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f33b00a2c10 0x7f33b00a50d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:02.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.484+0000 7f33cdd6c640 1 --2- 172.21.15.116:0/1723348490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33c8102ef0 0x7f33c819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:02.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.484+0000 7f33cdd6c640 1 --2- 172.21.15.116:0/1723348490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33c8102570 0x7f33c8198a00 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:02.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.484+0000 7f33cdd6c640 1 --2- 172.21.15.116:0/1723348490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f33c8100e20 0x7f33c81984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:02.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.484+0000 7f33cdd6c640 1 -- 172.21.15.116:0/1723348490 >> 172.21.15.116:0/1723348490 conn(0x7f33c80fc7d0 msgr2=0x7f33c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:02.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.484+0000 7f33cdd6c640 1 -- 172.21.15.116:0/1723348490 shutdown_connections 2024-09-12T03:03:02.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:02.484+0000 7f33cdd6c640 1 -- 172.21.15.116:0/1723348490 wait complete. 2024-09-12T03:03:02.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:02 smithi177 ceph-mon[35305]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:03:02.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:02 smithi177 ceph-mon[35305]: pgmap v339: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-12T03:03:02.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:02 smithi177 ceph-mon[35305]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:03:02.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:02 smithi183 ceph-mon[35535]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:03:02.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:02 smithi183 ceph-mon[35535]: pgmap v339: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-12T03:03:02.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:02 smithi183 ceph-mon[35535]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:03:03.199 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:03:03.446 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:03.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:03 smithi177 ceph-mon[35305]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:03:03.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:03 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1723348490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:03.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:03 smithi116 ceph-mon[31410]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:03:03.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:03 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1723348490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:03.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:03 smithi183 ceph-mon[35535]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:03:03.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:03 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1723348490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:03.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.943+0000 7f6a67301640 1 -- 172.21.15.116:0/3912684312 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a60100a70 msgr2=0x7f6a6010ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:03.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.943+0000 7f6a67301640 1 --2- 172.21.15.116:0/3912684312 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a60100a70 0x7f6a6010ea60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6a58009900 tx=0x7f6a5802a410 comp rx=0 tx=0).stop 2024-09-12T03:03:03.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.943+0000 7f6a67301640 1 -- 172.21.15.116:0/3912684312 shutdown_connections 2024-09-12T03:03:03.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.943+0000 7f6a67301640 1 --2- 172.21.15.116:0/3912684312 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a60100a70 0x7f6a6010ea60 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:03.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.943+0000 7f6a67301640 1 --2- 172.21.15.116:0/3912684312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a601000d0 0x7f6a60100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:03.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.943+0000 7f6a67301640 1 --2- 172.21.15.116:0/3912684312 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a60106ba0 0x7f6a60106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:03.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.943+0000 7f6a67301640 1 -- 172.21.15.116:0/3912684312 >> 172.21.15.116:0/3912684312 conn(0x7f6a60077210 msgr2=0x7f6a600fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:03.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.943+0000 7f6a67301640 1 -- 172.21.15.116:0/3912684312 shutdown_connections 2024-09-12T03:03:03.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.943+0000 7f6a67301640 1 -- 172.21.15.116:0/3912684312 wait complete. 2024-09-12T03:03:03.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.944+0000 7f6a67301640 1 Processor -- start 2024-09-12T03:03:03.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.944+0000 7f6a67301640 1 -- start start 2024-09-12T03:03:03.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.944+0000 7f6a67301640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a601000d0 0x7f6a601012b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:03.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.944+0000 7f6a67301640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a60100a70 0x7f6a601017f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:03.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.944+0000 7f6a67301640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a60106ba0 0x7f6a60101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:03.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.944+0000 7f6a67301640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6a60112090 con 0x7f6a60106ba0 2024-09-12T03:03:03.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.944+0000 7f6a67301640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6a60111f10 con 0x7f6a60100a70 2024-09-12T03:03:03.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.944+0000 7f6a67301640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6a60112210 con 0x7f6a601000d0 2024-09-12T03:03:03.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.944+0000 7f6a6589e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a60106ba0 0x7f6a60101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:03.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.944+0000 7f6a6589e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a60106ba0 0x7f6a60101d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39180/0 (socket says 172.21.15.116:39180) 2024-09-12T03:03:03.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.944+0000 7f6a6589e640 1 -- 172.21.15.116:0/528062344 learned_addr learned my addr 172.21.15.116:0/528062344 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:03.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.944+0000 7f6a6509d640 1 --2- 172.21.15.116:0/528062344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a601000d0 0x7f6a601012b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:03.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.944+0000 7f6a6489c640 1 --2- 172.21.15.116:0/528062344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a60100a70 0x7f6a601017f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:03.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.945+0000 7f6a6589e640 1 -- 172.21.15.116:0/528062344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a601000d0 msgr2=0x7f6a601012b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:03.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.945+0000 7f6a6589e640 1 --2- 172.21.15.116:0/528062344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a601000d0 0x7f6a601012b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:03.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.945+0000 7f6a6589e640 1 -- 172.21.15.116:0/528062344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a60100a70 msgr2=0x7f6a601017f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:03.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.945+0000 7f6a6589e640 1 --2- 172.21.15.116:0/528062344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a60100a70 0x7f6a601017f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:03.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.945+0000 7f6a6589e640 1 -- 172.21.15.116:0/528062344 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a601a6880 con 0x7f6a60106ba0 2024-09-12T03:03:03.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.945+0000 7f6a6589e640 1 --2- 172.21.15.116:0/528062344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a60106ba0 0x7f6a60101d30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6a58004770 tx=0x7f6a58003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:03.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.945+0000 7f6a567fc640 1 -- 172.21.15.116:0/528062344 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5803f070 con 0x7f6a60106ba0 2024-09-12T03:03:03.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.945+0000 7f6a67301640 1 -- 172.21.15.116:0/528062344 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a601a6b10 con 0x7f6a60106ba0 2024-09-12T03:03:03.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.945+0000 7f6a567fc640 1 -- 172.21.15.116:0/528062344 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a58031c30 con 0x7f6a60106ba0 2024-09-12T03:03:03.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.945+0000 7f6a567fc640 1 -- 172.21.15.116:0/528062344 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a58035030 con 0x7f6a60106ba0 2024-09-12T03:03:03.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.945+0000 7f6a67301640 1 -- 172.21.15.116:0/528062344 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a601a7040 con 0x7f6a60106ba0 2024-09-12T03:03:03.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.946+0000 7f6a67301640 1 -- 172.21.15.116:0/528062344 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a60100530 con 0x7f6a60106ba0 2024-09-12T03:03:03.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.947+0000 7f6a567fc640 1 -- 172.21.15.116:0/528062344 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f6a58042070 con 0x7f6a60106ba0 2024-09-12T03:03:03.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.948+0000 7f6a567fc640 1 --2- 172.21.15.116:0/528062344 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6a4c0a2c10 0x7f6a4c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:03.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.948+0000 7f6a6509d640 1 --2- 172.21.15.116:0/528062344 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6a4c0a2c10 0x7f6a4c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:03.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.948+0000 7f6a567fc640 1 -- 172.21.15.116:0/528062344 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f6a580f00b0 con 0x7f6a60106ba0 2024-09-12T03:03:03.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.949+0000 7f6a6509d640 1 --2- 172.21.15.116:0/528062344 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6a4c0a2c10 0x7f6a4c0a50d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c007e40 tx=0x7f6a5c0075b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:03.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:03.952+0000 7f6a567fc640 1 -- 172.21.15.116:0/528062344 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6a580bc890 con 0x7f6a60106ba0 2024-09-12T03:03:04.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:04.121+0000 7f6a67301640 1 -- 172.21.15.116:0/528062344 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6a601a7270 con 0x7f6a60106ba0 2024-09-12T03:03:04.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:04.122+0000 7f6a567fc640 1 -- 172.21.15.116:0/528062344 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22710 (secure 0 0 0) 0x7f6a580c0b90 con 0x7f6a60106ba0 2024-09-12T03:03:04.124 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:03:04.124 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":80,"fsid":"eb1ac602-70b1-11ef-bcea-c7b262605968","created":"2024-09-12T02:53:07.105686+0000","modified":"2024-09-12T03:02:04.272898+0000","last_up_change":"2024-09-12T03:00:07.202639+0000","last_in_change":"2024-09-12T02:59:43.446336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:56:17.347152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:00:55.191040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:00:57.171142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8cf5a18-dc6f-4bb6-bd74-78adda310118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6803","nonce":862366853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6805","nonce":862366853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6809","nonce":862366853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6807","nonce":862366853}]},"public_addr":"172.21.15.116:6803/862366853","cluster_addr":"172.21.15.116:6805/862366853","heartbeat_back_addr":"172.21.15.116:6809/862366853","heartbeat_front_addr":"172.21.15.116:6807/862366853","state":["exists","up"]},{"osd":1,"uuid":"0affd3cb-9a51-42b0-a2cc-f30784a75cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6801","nonce":2245800547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6803","nonce":2245800547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6807","nonce":2245800547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6805","nonce":2245800547}]},"public_addr":"172.21.15.177:6801/2245800547","cluster_addr":"172.21.15.177:6803/2245800547","heartbeat_back_addr":"172.21.15.177:6807/2245800547","heartbeat_front_addr":"172.21.15.177:6805/2245800547","state":["exists","up"]},{"osd":2,"uuid":"1ce45ea5-84df-4a35-924f-78f92bd4f2cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6801","nonce":2645677759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6803","nonce":2645677759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6807","nonce":2645677759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6805","nonce":2645677759}]},"public_addr":"172.21.15.183:6801/2645677759","cluster_addr":"172.21.15.183:6803/2645677759","heartbeat_back_addr":"172.21.15.183:6807/2645677759","heartbeat_front_addr":"172.21.15.183:6805/2645677759","state":["exists","up"]},{"osd":3,"uuid":"ae5e8178-c998-4648-bc05-4bce822f0cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6811","nonce":686622148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6813","nonce":686622148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6817","nonce":686622148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6815","nonce":686622148}]},"public_addr":"172.21.15.116:6811/686622148","cluster_addr":"172.21.15.116:6813/686622148","heartbeat_back_addr":"172.21.15.116:6817/686622148","heartbeat_front_addr":"172.21.15.116:6815/686622148","state":["exists","up"]},{"osd":4,"uuid":"c81d2e73-0bc2-440d-b5a2-b7428af3ea4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6809","nonce":2382938021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6811","nonce":2382938021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6815","nonce":2382938021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6813","nonce":2382938021}]},"public_addr":"172.21.15.177:6809/2382938021","cluster_addr":"172.21.15.177:6811/2382938021","heartbeat_back_addr":"172.21.15.177:6815/2382938021","heartbeat_front_addr":"172.21.15.177:6813/2382938021","state":["exists","up"]},{"osd":5,"uuid":"c3f168be-9e27-4cf5-b73e-3f47d8990ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6809","nonce":85363432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6811","nonce":85363432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6815","nonce":85363432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6813","nonce":85363432}]},"public_addr":"172.21.15.183:6809/85363432","cluster_addr":"172.21.15.183:6811/85363432","heartbeat_back_addr":"172.21.15.183:6815/85363432","heartbeat_front_addr":"172.21.15.183:6813/85363432","state":["exists","up"]},{"osd":6,"uuid":"beb0a7e9-4909-47e2-9ee7-6951d4745e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6819","nonce":1121291186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6821","nonce":1121291186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6825","nonce":1121291186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6823","nonce":1121291186}]},"public_addr":"172.21.15.116:6819/1121291186","cluster_addr":"172.21.15.116:6821/1121291186","heartbeat_back_addr":"172.21.15.116:6825/1121291186","heartbeat_front_addr":"172.21.15.116:6823/1121291186","state":["exists","up"]},{"osd":7,"uuid":"761f4680-f049-473c-bf44-b19acf8de24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6817","nonce":2177665961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6819","nonce":2177665961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6823","nonce":2177665961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6821","nonce":2177665961}]},"public_addr":"172.21.15.177:6817/2177665961","cluster_addr":"172.21.15.177:6819/2177665961","heartbeat_back_addr":"172.21.15.177:6823/2177665961","heartbeat_front_addr":"172.21.15.177:6821/2177665961","state":["exists","up"]},{"osd":8,"uuid":"2385ea6c-fd5a-4815-9596-77f948629542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6817","nonce":1305725258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6819","nonce":1305725258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6823","nonce":1305725258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6821","nonce":1305725258}]},"public_addr":"172.21.15.183:6817/1305725258","cluster_addr":"172.21.15.183:6819/1305725258","heartbeat_back_addr":"172.21.15.183:6823/1305725258","heartbeat_front_addr":"172.21.15.183:6821/1305725258","state":["exists","up"]},{"osd":9,"uuid":"fd0c0dc4-4990-4f12-aa14-f855504cd995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6827","nonce":3831673668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6829","nonce":3831673668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6833","nonce":3831673668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6831","nonce":3831673668}]},"public_addr":"172.21.15.116:6827/3831673668","cluster_addr":"172.21.15.116:6829/3831673668","heartbeat_back_addr":"172.21.15.116:6833/3831673668","heartbeat_front_addr":"172.21.15.116:6831/3831673668","state":["exists","up"]},{"osd":10,"uuid":"864d1a97-4810-49c8-ad4a-a0c10253833c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6825","nonce":963663634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6827","nonce":963663634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6831","nonce":963663634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6829","nonce":963663634}]},"public_addr":"172.21.15.177:6825/963663634","cluster_addr":"172.21.15.177:6827/963663634","heartbeat_back_addr":"172.21.15.177:6831/963663634","heartbeat_front_addr":"172.21.15.177:6829/963663634","state":["exists","up"]},{"osd":11,"uuid":"d49c0841-4620-4f2f-a6b5-f2a9aa2e3c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6825","nonce":1818319939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6827","nonce":1818319939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6831","nonce":1818319939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6829","nonce":1818319939}]},"public_addr":"172.21.15.183:6825/1818319939","cluster_addr":"172.21.15.183:6827/1818319939","heartbeat_back_addr":"172.21.15.183:6831/1818319939","heartbeat_front_addr":"172.21.15.183:6829/1818319939","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:24.736944+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:50.310980+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:56:14.989533+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:56:41.187047+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:06.981664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:32.872771+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:58.850089+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:23.043121+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:47.839958+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:14.087708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:40.010548+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:05.364820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.116:6801/2966274423":"2024-09-13T02:54:03.026862+0000","172.21.15.116:6800/2966274423":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/686139455":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/1552047390":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/2180363411":"2024-09-13T02:53:39.433769+0000","172.21.15.116:6801/1791219601":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/3907400460":"2024-09-13T02:54:03.026862+0000","172.21.15.116:6800/1791219601":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/1802566286":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/212337802":"2024-09-13T02:53:39.433769+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:03:04.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:04.125+0000 7f6a67301640 1 -- 172.21.15.116:0/528062344 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6a4c0a2c10 msgr2=0x7f6a4c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:04.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:04.125+0000 7f6a67301640 1 --2- 172.21.15.116:0/528062344 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6a4c0a2c10 0x7f6a4c0a50d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c007e40 tx=0x7f6a5c0075b0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:04.125+0000 7f6a67301640 1 -- 172.21.15.116:0/528062344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a60106ba0 msgr2=0x7f6a60101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:04.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:04.125+0000 7f6a67301640 1 --2- 172.21.15.116:0/528062344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a60106ba0 0x7f6a60101d30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6a58004770 tx=0x7f6a58003c10 comp rx=0 tx=0).stop 2024-09-12T03:03:04.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:04.125+0000 7f6a67301640 1 -- 172.21.15.116:0/528062344 shutdown_connections 2024-09-12T03:03:04.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:04.125+0000 7f6a67301640 1 --2- 172.21.15.116:0/528062344 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6a4c0a2c10 0x7f6a4c0a50d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:04.125+0000 7f6a67301640 1 --2- 172.21.15.116:0/528062344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a60106ba0 0x7f6a60101d30 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:04.125+0000 7f6a67301640 1 --2- 172.21.15.116:0/528062344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a60100a70 0x7f6a601017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:04.125+0000 7f6a67301640 1 --2- 172.21.15.116:0/528062344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a601000d0 0x7f6a601012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:04.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:04.125+0000 7f6a67301640 1 -- 172.21.15.116:0/528062344 >> 172.21.15.116:0/528062344 conn(0x7f6a60077210 msgr2=0x7f6a60104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:04.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:04.125+0000 7f6a67301640 1 -- 172.21.15.116:0/528062344 shutdown_connections 2024-09-12T03:03:04.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:04.125+0000 7f6a67301640 1 -- 172.21.15.116:0/528062344 wait complete. 2024-09-12T03:03:04.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:04 smithi177 ceph-mon[35305]: pgmap v340: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-12T03:03:04.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:04 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/528062344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:03:04.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:04 smithi116 ceph-mon[31410]: pgmap v340: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-12T03:03:04.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:04 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/528062344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:03:04.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:04 smithi183 ceph-mon[35535]: pgmap v340: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-12T03:03:04.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:04 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/528062344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:03:04.833 DEBUG:tasks.fwd_scrub:Yielding 2024-09-12T03:03:04.833 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-12T03:03:04.848 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T03:03:04.848 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:03:04.848 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-12T03:03:04.849 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:03:04.849 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:03:05.107 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:05.107 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:05.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.669+0000 7fb443ef7640 1 -- 172.21.15.116:0/38014575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb43c100b30 msgr2=0x7fb43c100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:05.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.669+0000 7fb443ef7640 1 --2- 172.21.15.116:0/38014575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb43c100b30 0x7fb43c100f10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb438009900 tx=0x7fb43802a470 comp rx=0 tx=0).stop 2024-09-12T03:03:05.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.669+0000 7fb443ef7640 1 -- 172.21.15.116:0/38014575 shutdown_connections 2024-09-12T03:03:05.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.669+0000 7fb443ef7640 1 --2- 172.21.15.116:0/38014575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb43c105f40 0x7fb43c1063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:05.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.669+0000 7fb443ef7640 1 --2- 172.21.15.116:0/38014575 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb43c101450 0x7fb43c105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:05.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.669+0000 7fb443ef7640 1 --2- 172.21.15.116:0/38014575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb43c100b30 0x7fb43c100f10 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:05.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.669+0000 7fb443ef7640 1 -- 172.21.15.116:0/38014575 >> 172.21.15.116:0/38014575 conn(0x7fb43c0fc7a0 msgr2=0x7fb43c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:05.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.669+0000 7fb443ef7640 1 -- 172.21.15.116:0/38014575 shutdown_connections 2024-09-12T03:03:05.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.669+0000 7fb443ef7640 1 -- 172.21.15.116:0/38014575 wait complete. 2024-09-12T03:03:05.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.670+0000 7fb443ef7640 1 Processor -- start 2024-09-12T03:03:05.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.670+0000 7fb443ef7640 1 -- start start 2024-09-12T03:03:05.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.670+0000 7fb443ef7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb43c100b30 0x7fb43c1a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:05.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.670+0000 7fb443ef7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb43c101450 0x7fb43c1a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:05.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.670+0000 7fb443ef7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb43c105f40 0x7fb43c19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:05.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.670+0000 7fb443ef7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c118550 con 0x7fb43c105f40 2024-09-12T03:03:05.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.670+0000 7fb443ef7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c1183d0 con 0x7fb43c100b30 2024-09-12T03:03:05.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.670+0000 7fb443ef7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c1186d0 con 0x7fb43c101450 2024-09-12T03:03:05.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.670+0000 7fb442494640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb43c105f40 0x7fb43c19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:05.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.670+0000 7fb442494640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb43c105f40 0x7fb43c19e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39198/0 (socket says 172.21.15.116:39198) 2024-09-12T03:03:05.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.670+0000 7fb442494640 1 -- 172.21.15.116:0/3607282067 learned_addr learned my addr 172.21.15.116:0/3607282067 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:05.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.670+0000 7fb441492640 1 --2- 172.21.15.116:0/3607282067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb43c101450 0x7fb43c1a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:05.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.670+0000 7fb441c93640 1 --2- 172.21.15.116:0/3607282067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb43c100b30 0x7fb43c1a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:05.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.670+0000 7fb442494640 1 -- 172.21.15.116:0/3607282067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb43c101450 msgr2=0x7fb43c1a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:05.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.670+0000 7fb442494640 1 --2- 172.21.15.116:0/3607282067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb43c101450 0x7fb43c1a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:05.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.670+0000 7fb442494640 1 -- 172.21.15.116:0/3607282067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb43c100b30 msgr2=0x7fb43c1a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:05.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.670+0000 7fb442494640 1 --2- 172.21.15.116:0/3607282067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb43c100b30 0x7fb43c1a4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:05.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.670+0000 7fb442494640 1 -- 172.21.15.116:0/3607282067 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb43c19efa0 con 0x7fb43c105f40 2024-09-12T03:03:05.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.670+0000 7fb441492640 1 --2- 172.21.15.116:0/3607282067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb43c101450 0x7fb43c1a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:03:05.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.670+0000 7fb441c93640 1 --2- 172.21.15.116:0/3607282067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb43c100b30 0x7fb43c1a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:03:05.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.670+0000 7fb442494640 1 --2- 172.21.15.116:0/3607282067 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb43c105f40 0x7fb43c19e800 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb43400cce0 tx=0x7fb43400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:05.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.671+0000 7fb432ffd640 1 -- 172.21.15.116:0/3607282067 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43400f450 con 0x7fb43c105f40 2024-09-12T03:03:05.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.671+0000 7fb443ef7640 1 -- 172.21.15.116:0/3607282067 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb43c19f290 con 0x7fb43c105f40 2024-09-12T03:03:05.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.671+0000 7fb432ffd640 1 -- 172.21.15.116:0/3607282067 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb43400fad0 con 0x7fb43c105f40 2024-09-12T03:03:05.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.671+0000 7fb432ffd640 1 -- 172.21.15.116:0/3607282067 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb434018c10 con 0x7fb43c105f40 2024-09-12T03:03:05.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.671+0000 7fb443ef7640 1 -- 172.21.15.116:0/3607282067 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb43c1ab4e0 con 0x7fb43c105f40 2024-09-12T03:03:05.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.672+0000 7fb443ef7640 1 -- 172.21.15.116:0/3607282067 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb43c06b070 con 0x7fb43c105f40 2024-09-12T03:03:05.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.673+0000 7fb432ffd640 1 -- 172.21.15.116:0/3607282067 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fb43400f5f0 con 0x7fb43c105f40 2024-09-12T03:03:05.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.673+0000 7fb432ffd640 1 --2- 172.21.15.116:0/3607282067 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb4280a2c10 0x7fb4280a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:05.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.673+0000 7fb441c93640 1 --2- 172.21.15.116:0/3607282067 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb4280a2c10 0x7fb4280a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:05.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.673+0000 7fb432ffd640 1 -- 172.21.15.116:0/3607282067 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fb4340cf890 con 0x7fb43c105f40 2024-09-12T03:03:05.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.674+0000 7fb441c93640 1 --2- 172.21.15.116:0/3607282067 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb4280a2c10 0x7fb4280a50d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb438000c00 tx=0x7fb438032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:05.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.677+0000 7fb432ffd640 1 -- 172.21.15.116:0/3607282067 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb434014030 con 0x7fb43c105f40 2024-09-12T03:03:05.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.851+0000 7fb443ef7640 1 -- 172.21.15.116:0/3607282067 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb43c105010 con 0x7fb4280a2c10 2024-09-12T03:03:05.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.860+0000 7fb432ffd640 1 -- 172.21.15.116:0/3607282067 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258981 (secure 0 0 0) 0x7fb43c105010 con 0x7fb4280a2c10 2024-09-12T03:03:05.863 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:03:05.871 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-09-12T03:03:05.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.863+0000 7fb443ef7640 1 -- 172.21.15.116:0/3607282067 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb4280a2c10 msgr2=0x7fb4280a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:05.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.863+0000 7fb443ef7640 1 --2- 172.21.15.116:0/3607282067 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb4280a2c10 0x7fb4280a50d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb438000c00 tx=0x7fb438032c70 comp rx=0 tx=0).stop 2024-09-12T03:03:05.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.863+0000 7fb443ef7640 1 -- 172.21.15.116:0/3607282067 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb43c105f40 msgr2=0x7fb43c19e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:05.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.863+0000 7fb443ef7640 1 --2- 172.21.15.116:0/3607282067 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb43c105f40 0x7fb43c19e800 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb43400cce0 tx=0x7fb43400a620 comp rx=0 tx=0).stop 2024-09-12T03:03:05.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.863+0000 7fb443ef7640 1 -- 172.21.15.116:0/3607282067 shutdown_connections 2024-09-12T03:03:05.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.863+0000 7fb443ef7640 1 --2- 172.21.15.116:0/3607282067 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb4280a2c10 0x7fb4280a50d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:05.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.863+0000 7fb443ef7640 1 --2- 172.21.15.116:0/3607282067 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb43c105f40 0x7fb43c19e800 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:05.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.863+0000 7fb443ef7640 1 --2- 172.21.15.116:0/3607282067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb43c101450 0x7fb43c1a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:05.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.863+0000 7fb443ef7640 1 --2- 172.21.15.116:0/3607282067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb43c100b30 0x7fb43c1a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:05.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.863+0000 7fb443ef7640 1 -- 172.21.15.116:0/3607282067 >> 172.21.15.116:0/3607282067 conn(0x7fb43c0fc7a0 msgr2=0x7fb43c1038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:05.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.863+0000 7fb443ef7640 1 -- 172.21.15.116:0/3607282067 shutdown_connections 2024-09-12T03:03:05.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:05.863+0000 7fb443ef7640 1 -- 172.21.15.116:0/3607282067 wait complete. 2024-09-12T03:03:06.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.034+0000 7f77599e7640 1 -- 172.21.15.116:0/43812323 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7754105a90 msgr2=0x7f7754105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:06.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.034+0000 7f77599e7640 1 --2- 172.21.15.116:0/43812323 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7754105a90 0x7f7754105e70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7744009900 tx=0x7f774402a410 comp rx=0 tx=0).stop 2024-09-12T03:03:06.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.034+0000 7f77599e7640 1 -- 172.21.15.116:0/43812323 shutdown_connections 2024-09-12T03:03:06.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.034+0000 7f77599e7640 1 --2- 172.21.15.116:0/43812323 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7754101bc0 0x7f775410d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.034+0000 7f77599e7640 1 --2- 172.21.15.116:0/43812323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7754101240 0x7f7754101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.034+0000 7f77599e7640 1 --2- 172.21.15.116:0/43812323 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7754105a90 0x7f7754105e70 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.034+0000 7f77599e7640 1 -- 172.21.15.116:0/43812323 >> 172.21.15.116:0/43812323 conn(0x7f7754077210 msgr2=0x7f77540fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:06.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.034+0000 7f77599e7640 1 -- 172.21.15.116:0/43812323 shutdown_connections 2024-09-12T03:03:06.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.034+0000 7f77599e7640 1 -- 172.21.15.116:0/43812323 wait complete. 2024-09-12T03:03:06.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.035+0000 7f77599e7640 1 Processor -- start 2024-09-12T03:03:06.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.035+0000 7f77599e7640 1 -- start start 2024-09-12T03:03:06.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.035+0000 7f77599e7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7754101240 0x7f7754199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:06.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.035+0000 7f77599e7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7754101bc0 0x7f7754199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:06.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.035+0000 7f77599e7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7754105a90 0x7f77541a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:06.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.035+0000 7f77599e7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7754110f70 con 0x7f7754101bc0 2024-09-12T03:03:06.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.035+0000 7f77599e7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7754110df0 con 0x7f7754105a90 2024-09-12T03:03:06.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.035+0000 7f77599e7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f77541110f0 con 0x7f7754101240 2024-09-12T03:03:06.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.035+0000 7f774bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7754101bc0 0x7f7754199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:06.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.035+0000 7f774bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7754101bc0 0x7f7754199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39224/0 (socket says 172.21.15.116:39224) 2024-09-12T03:03:06.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.035+0000 7f774bfff640 1 -- 172.21.15.116:0/3561604327 learned_addr learned my addr 172.21.15.116:0/3561604327 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:06.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.035+0000 7f77537fe640 1 --2- 172.21.15.116:0/3561604327 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7754105a90 0x7f77541a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:06.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.035+0000 7f7752ffd640 1 --2- 172.21.15.116:0/3561604327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7754101240 0x7f7754199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:06.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.036+0000 7f774bfff640 1 -- 172.21.15.116:0/3561604327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7754101240 msgr2=0x7f7754199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:06.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.036+0000 7f774bfff640 1 --2- 172.21.15.116:0/3561604327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7754101240 0x7f7754199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.036+0000 7f774bfff640 1 -- 172.21.15.116:0/3561604327 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7754105a90 msgr2=0x7f77541a43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:06.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.036+0000 7f774bfff640 1 --2- 172.21.15.116:0/3561604327 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7754105a90 0x7f77541a43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.036+0000 7f774bfff640 1 -- 172.21.15.116:0/3561604327 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77541a48e0 con 0x7f7754101bc0 2024-09-12T03:03:06.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.036+0000 7f77537fe640 1 --2- 172.21.15.116:0/3561604327 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7754105a90 0x7f77541a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:03:06.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.036+0000 7f774bfff640 1 --2- 172.21.15.116:0/3561604327 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7754101bc0 0x7f7754199860 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f773c00d950 tx=0x7f773c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:06.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.036+0000 7f77589e5640 1 -- 172.21.15.116:0/3561604327 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c017070 con 0x7f7754101bc0 2024-09-12T03:03:06.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.036+0000 7f77599e7640 1 -- 172.21.15.116:0/3561604327 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77541a4ab0 con 0x7f7754101bc0 2024-09-12T03:03:06.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.036+0000 7f77589e5640 1 -- 172.21.15.116:0/3561604327 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f773c009c30 con 0x7f7754101bc0 2024-09-12T03:03:06.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.036+0000 7f77589e5640 1 -- 172.21.15.116:0/3561604327 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c00f6b0 con 0x7f7754101bc0 2024-09-12T03:03:06.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.036+0000 7f77599e7640 1 -- 172.21.15.116:0/3561604327 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77541a4fc0 con 0x7f7754101bc0 2024-09-12T03:03:06.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.039+0000 7f77589e5640 1 -- 172.21.15.116:0/3561604327 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f773c0109d0 con 0x7f7754101bc0 2024-09-12T03:03:06.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.039+0000 7f775bdc0740 1 Processor -- start 2024-09-12T03:03:06.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.039+0000 7f775bdc0740 1 -- start start 2024-09-12T03:03:06.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.039+0000 7f775bdc0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc7a214b60 0x55fc7a214f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:06.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.039+0000 7f775bdc0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc7a215480 0x55fc7a21f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:06.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.039+0000 7f775bdc0740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc7a21fd40 0x55fc7a222130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:06.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.039+0000 7f775bdc0740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55fc7a17a280 con 0x55fc7a21fd40 2024-09-12T03:03:06.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.039+0000 7f775bdc0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55fc7a121b10 con 0x55fc7a214b60 2024-09-12T03:03:06.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.039+0000 7f775bdc0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fc7a176240 con 0x55fc7a215480 2024-09-12T03:03:06.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.039+0000 7f774bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc7a21fd40 0x55fc7a222130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:06.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.039+0000 7f774bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc7a21fd40 0x55fc7a222130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39232/0 (socket says 172.21.15.116:39232) 2024-09-12T03:03:06.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.039+0000 7f774bfff640 1 -- 172.21.15.116:0/2471524455 learned_addr learned my addr 172.21.15.116:0/2471524455 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:06.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.040+0000 7f7752ffd640 1 --2- 172.21.15.116:0/2471524455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc7a215480 0x55fc7a21f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:06.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.040+0000 7f77537fe640 1 --2- 172.21.15.116:0/2471524455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc7a214b60 0x55fc7a214f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:06.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.040+0000 7f774bfff640 1 -- 172.21.15.116:0/2471524455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc7a215480 msgr2=0x55fc7a21f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:06.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.040+0000 7f774bfff640 1 --2- 172.21.15.116:0/2471524455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc7a215480 0x55fc7a21f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.040+0000 7f774bfff640 1 -- 172.21.15.116:0/2471524455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc7a214b60 msgr2=0x55fc7a214f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:06.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.040+0000 7f774bfff640 1 --2- 172.21.15.116:0/2471524455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc7a214b60 0x55fc7a214f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.040+0000 7f774bfff640 1 -- 172.21.15.116:0/2471524455 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc7a12e1f0 con 0x55fc7a21fd40 2024-09-12T03:03:06.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.040+0000 7f774bfff640 1 --2- 172.21.15.116:0/2471524455 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc7a21fd40 0x55fc7a222130 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f773c0cc800 tx=0x7f773c0d75f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7000c584c54de294 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:06.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.040+0000 7f77589e5640 1 --2- 172.21.15.116:0/3561604327 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f77400a2c10 0x7f77400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:06.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.040+0000 7f7749ffb640 1 -- 172.21.15.116:0/2471524455 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c0e2070 con 0x55fc7a21fd40 2024-09-12T03:03:06.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.040+0000 7f77537fe640 1 --2- 172.21.15.116:0/3561604327 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f77400a2c10 0x7f77400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:06.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.040+0000 7f7749ffb640 1 -- 172.21.15.116:0/2471524455 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f773c0d7e20 con 0x55fc7a21fd40 2024-09-12T03:03:06.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.040+0000 7f7749ffb640 1 -- 172.21.15.116:0/2471524455 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c081ab0 con 0x55fc7a21fd40 2024-09-12T03:03:06.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.040+0000 7f77589e5640 1 -- 172.21.15.116:0/3561604327 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f773c0ceee0 con 0x7f7754101bc0 2024-09-12T03:03:06.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.041+0000 7f775bdc0740 1 -- 172.21.15.116:0/2471524455 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc7a21fd40 msgr2=0x55fc7a222130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:06.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.041+0000 7f77537fe640 1 --2- 172.21.15.116:0/3561604327 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f77400a2c10 0x7f77400a50d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f774c007f10 tx=0x7f774c0076a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:06.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.041+0000 7f775bdc0740 1 --2- 172.21.15.116:0/2471524455 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc7a21fd40 0x55fc7a222130 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f773c0cc800 tx=0x7f773c0d75f0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.041+0000 7f775bdc0740 1 -- 172.21.15.116:0/2471524455 shutdown_connections 2024-09-12T03:03:06.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.041+0000 7f775bdc0740 1 --2- 172.21.15.116:0/2471524455 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc7a21fd40 0x55fc7a222130 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.041+0000 7f775bdc0740 1 --2- 172.21.15.116:0/2471524455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc7a215480 0x55fc7a21f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.041+0000 7f775bdc0740 1 --2- 172.21.15.116:0/2471524455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc7a214b60 0x55fc7a214f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.041+0000 7f775bdc0740 1 -- 172.21.15.116:0/2471524455 >> 172.21.15.116:0/2471524455 conn(0x55fc7a2110f0 msgr2=0x55fc7a2114d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:06.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.041+0000 7f775bdc0740 1 -- 172.21.15.116:0/2471524455 shutdown_connections 2024-09-12T03:03:06.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.041+0000 7f775bdc0740 1 -- 172.21.15.116:0/2471524455 wait complete. 2024-09-12T03:03:06.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.041+0000 7f775bdc0740 1 -- start start 2024-09-12T03:03:06.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.042+0000 7f775bdc0740 1 Processor -- start 2024-09-12T03:03:06.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.042+0000 7f775bdc0740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc7a214b60 0x55fc7a21cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:06.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.042+0000 7f775bdc0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc7a215480 0x55fc7a21d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:06.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.042+0000 7f775bdc0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc7a21fd40 0x55fc7a21d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:06.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.042+0000 7f775bdc0740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55fc7a222910 con 0x55fc7a214b60 2024-09-12T03:03:06.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.042+0000 7f775bdc0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55fc7a134f10 con 0x55fc7a21fd40 2024-09-12T03:03:06.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.042+0000 7f775bdc0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fc7a222a90 con 0x55fc7a215480 2024-09-12T03:03:06.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.042+0000 7f775bdc0740 10 client.0 _ll_register_callbacks cb 0x55fc7a0fda40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:06.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.042+0000 7f7752ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc7a214b60 0x55fc7a21cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:06.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.042+0000 7f7752ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc7a214b60 0x55fc7a21cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39236/0 (socket says 172.21.15.116:39236) 2024-09-12T03:03:06.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.042+0000 7f7752ffd640 1 -- 172.21.15.116:0/3483150320 learned_addr learned my addr 172.21.15.116:0/3483150320 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:06.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.042+0000 7f774bfff640 1 --2- 172.21.15.116:0/3483150320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc7a215480 0x55fc7a21d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:06.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.042+0000 7f77537fe640 1 --2- 172.21.15.116:0/3483150320 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc7a21fd40 0x55fc7a21d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:06.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.042+0000 7f7752ffd640 1 -- 172.21.15.116:0/3483150320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc7a215480 msgr2=0x55fc7a21d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:06.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.042+0000 7f7752ffd640 1 --2- 172.21.15.116:0/3483150320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc7a215480 0x55fc7a21d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.042+0000 7f7752ffd640 1 -- 172.21.15.116:0/3483150320 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc7a21fd40 msgr2=0x55fc7a21d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:06.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.042+0000 7f7752ffd640 1 --2- 172.21.15.116:0/3483150320 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc7a21fd40 0x55fc7a21d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.042+0000 7f7752ffd640 1 -- 172.21.15.116:0/3483150320 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc79eeb6a0 con 0x55fc7a214b60 2024-09-12T03:03:06.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.042+0000 7f7752ffd640 1 --2- 172.21.15.116:0/3483150320 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc7a214b60 0x55fc7a21cba0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f774402a930 tx=0x7f7744005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a99184507b4a30 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:06.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.042+0000 7f7732ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:03:06.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.043+0000 7f7732ffd640 1 -- 172.21.15.116:0/3483150320 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7744035040 con 0x55fc7a214b60 2024-09-12T03:03:06.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.043+0000 7f7732ffd640 1 -- 172.21.15.116:0/3483150320 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f77440048c0 con 0x55fc7a214b60 2024-09-12T03:03:06.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.043+0000 7f775bdc0740 1 -- 172.21.15.116:0/3483150320 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f773c0d7e20 con 0x55fc7a214b60 2024-09-12T03:03:06.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.043+0000 7f7732ffd640 1 -- 172.21.15.116:0/3483150320 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7744004ac0 con 0x55fc7a214b60 2024-09-12T03:03:06.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.043+0000 7f775bdc0740 10 client.15258 fetch_fsmap learned FSMap version 25 2024-09-12T03:03:06.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.043+0000 7f775bdc0740 1 -- 172.21.15.116:0/3483150320 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fc7a22e4a0 con 0x55fc7a214b60 2024-09-12T03:03:06.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.043+0000 7f7732ffd640 1 -- 172.21.15.116:0/3483150320 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f773c0d7e20 con 0x55fc7a214b60 2024-09-12T03:03:06.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.043+0000 7f7732ffd640 1 -- 172.21.15.116:0/3483150320 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7744031540 con 0x55fc7a214b60 2024-09-12T03:03:06.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.043+0000 7f775bdc0740 10 client.15258 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:03:06.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.043+0000 7f775bdc0740 10 client.15258 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:03:06.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.043+0000 7f775bdc0740 20 client.15258 populate_metadata read hostname 'smithi116' 2024-09-12T03:03:06.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.043+0000 7f775bdc0740 1 --2- 172.21.15.116:0/3483150320 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55fc7a22f960 0x55fc7a231d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:06.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.043+0000 7f775bdc0740 4 client.15258 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:03:06.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.043+0000 7f775bdc0740 1 -- 172.21.15.116:0/3483150320 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc7a1877f0 con 0x55fc7a22f960 2024-09-12T03:03:06.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.044+0000 7f774bfff640 1 --2- 172.21.15.116:0/3483150320 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55fc7a22f960 0x55fc7a231d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:06.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.044+0000 7f7732ffd640 10 client.15258 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:06.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.044+0000 7f774bfff640 1 --2- 172.21.15.116:0/3483150320 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55fc7a22f960 0x55fc7a231d50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=234fc425866e8adc server_cookie=1743108a0ad9bf67 in_seq=0 out_seq=0 2024-09-12T03:03:06.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.045+0000 7f7732ffd640 1 -- 172.21.15.116:0/3483150320 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fc7a1877f0 con 0x55fc7a22f960 2024-09-12T03:03:06.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.045+0000 7f7732ffd640 10 client.15258 handle_command_reply: tid=0 2024-09-12T03:03:06.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.045+0000 7f775bdc0740 1 client.15258 shutdown 2024-09-12T03:03:06.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.045+0000 7f775bdc0740 1 -- 172.21.15.116:0/3483150320 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc7a214b60 msgr2=0x55fc7a21cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:06.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.045+0000 7f775bdc0740 1 --2- 172.21.15.116:0/3483150320 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc7a214b60 0x55fc7a21cba0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f774402a930 tx=0x7f7744005250 comp rx=0 tx=0).stop 2024-09-12T03:03:06.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.045+0000 7f775bdc0740 1 -- 172.21.15.116:0/3483150320 shutdown_connections 2024-09-12T03:03:06.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.045+0000 7f775bdc0740 1 --2- 172.21.15.116:0/3483150320 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55fc7a22f960 0x55fc7a231d50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.045+0000 7f775bdc0740 1 --2- 172.21.15.116:0/3483150320 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc7a21fd40 0x55fc7a21d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.045+0000 7f775bdc0740 1 --2- 172.21.15.116:0/3483150320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc7a215480 0x55fc7a21d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.045+0000 7f775bdc0740 1 --2- 172.21.15.116:0/3483150320 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc7a214b60 0x55fc7a21cba0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.045+0000 7f775bdc0740 1 -- 172.21.15.116:0/3483150320 >> 172.21.15.116:0/3483150320 conn(0x55fc7a2110f0 msgr2=0x55fc7a216dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:06.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.045+0000 7f7732ffd640 0 client.15258 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:06.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.045+0000 7f775bdc0740 1 -- 172.21.15.116:0/3483150320 shutdown_connections 2024-09-12T03:03:06.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.045+0000 7f775bdc0740 1 -- 172.21.15.116:0/3483150320 wait complete. 2024-09-12T03:03:06.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.045+0000 7f775bdc0740 20 client.15258 trim_cache size 0 max 16384 2024-09-12T03:03:06.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.062+0000 7f775bdc0740 1 Processor -- start 2024-09-12T03:03:06.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.062+0000 7f775bdc0740 1 -- start start 2024-09-12T03:03:06.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.062+0000 7f775bdc0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc7a214b60 0x55fc7a21c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:06.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.063+0000 7f775bdc0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc7a215480 0x55fc7a21ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:06.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.063+0000 7f775bdc0740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc7a21fd40 0x55fc7a237620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:06.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.063+0000 7f775bdc0740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f77541afd80 con 0x55fc7a21fd40 2024-09-12T03:03:06.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.063+0000 7f775bdc0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f77541ad650 con 0x55fc7a215480 2024-09-12T03:03:06.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.063+0000 7f775bdc0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f77541ab4a0 con 0x55fc7a214b60 2024-09-12T03:03:06.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.063+0000 7f77537fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc7a21fd40 0x55fc7a237620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:06.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.063+0000 7f77537fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc7a21fd40 0x55fc7a237620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39246/0 (socket says 172.21.15.116:39246) 2024-09-12T03:03:06.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.063+0000 7f77537fe640 1 -- 172.21.15.116:0/469995216 learned_addr learned my addr 172.21.15.116:0/469995216 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:06.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.063+0000 7f7752ffd640 1 --2- 172.21.15.116:0/469995216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc7a214b60 0x55fc7a21c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:06.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.063+0000 7f77537fe640 1 -- 172.21.15.116:0/469995216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc7a214b60 msgr2=0x55fc7a21c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:06.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.063+0000 7f77537fe640 1 --2- 172.21.15.116:0/469995216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc7a214b60 0x55fc7a21c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.063+0000 7f77537fe640 1 -- 172.21.15.116:0/469995216 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc7a215480 msgr2=0x55fc7a21ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:06.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.063+0000 7f774bfff640 1 --2- 172.21.15.116:0/469995216 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc7a215480 0x55fc7a21ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:06.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.063+0000 7f77537fe640 1 --2- 172.21.15.116:0/469995216 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc7a215480 0x55fc7a21ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.063+0000 7f77537fe640 1 -- 172.21.15.116:0/469995216 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc7a2152b0 con 0x55fc7a21fd40 2024-09-12T03:03:06.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.063+0000 7f7752ffd640 1 --2- 172.21.15.116:0/469995216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc7a214b60 0x55fc7a21c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:03:06.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.063+0000 7f774bfff640 1 --2- 172.21.15.116:0/469995216 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc7a215480 0x55fc7a21ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:03:06.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.063+0000 7f77537fe640 1 --2- 172.21.15.116:0/469995216 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc7a21fd40 0x55fc7a237620 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f774c01d730 tx=0x7f774c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91038794b9688eb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:06.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.063+0000 7f77497fa640 1 -- 172.21.15.116:0/469995216 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c02b070 con 0x55fc7a21fd40 2024-09-12T03:03:06.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.063+0000 7f77497fa640 1 -- 172.21.15.116:0/469995216 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f774c01cbb0 con 0x55fc7a21fd40 2024-09-12T03:03:06.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.063+0000 7f77497fa640 1 -- 172.21.15.116:0/469995216 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c026cb0 con 0x55fc7a21fd40 2024-09-12T03:03:06.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.064+0000 7f775bdc0740 1 -- 172.21.15.116:0/469995216 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc7a21fd40 msgr2=0x55fc7a237620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:06.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.064+0000 7f775bdc0740 1 --2- 172.21.15.116:0/469995216 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc7a21fd40 0x55fc7a237620 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f774c01d730 tx=0x7f774c014000 comp rx=0 tx=0).stop 2024-09-12T03:03:06.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.064+0000 7f775bdc0740 1 -- 172.21.15.116:0/469995216 shutdown_connections 2024-09-12T03:03:06.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.064+0000 7f775bdc0740 1 --2- 172.21.15.116:0/469995216 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc7a21fd40 0x55fc7a237620 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.064+0000 7f775bdc0740 1 --2- 172.21.15.116:0/469995216 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc7a215480 0x55fc7a21ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.064+0000 7f775bdc0740 1 --2- 172.21.15.116:0/469995216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc7a214b60 0x55fc7a21c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.064+0000 7f775bdc0740 1 -- 172.21.15.116:0/469995216 >> 172.21.15.116:0/469995216 conn(0x55fc7a2110f0 msgr2=0x55fc7a21e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:06.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.064+0000 7f775bdc0740 1 -- 172.21.15.116:0/469995216 shutdown_connections 2024-09-12T03:03:06.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.064+0000 7f775bdc0740 1 -- 172.21.15.116:0/469995216 wait complete. 2024-09-12T03:03:06.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.064+0000 7f775bdc0740 1 -- start start 2024-09-12T03:03:06.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.065+0000 7f775bdc0740 1 Processor -- start 2024-09-12T03:03:06.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.065+0000 7f775bdc0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc7a214b60 0x55fc7a2284c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:06.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.065+0000 7f775bdc0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc7a215480 0x55fc7a219930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:06.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.065+0000 7f775bdc0740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc7a21fd40 0x55fc7a219e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:06.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.065+0000 7f775bdc0740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f77541abb10 con 0x55fc7a21fd40 2024-09-12T03:03:06.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.065+0000 7f775bdc0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f77541acf80 con 0x55fc7a215480 2024-09-12T03:03:06.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.065+0000 7f775bdc0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f77541ac1e0 con 0x55fc7a214b60 2024-09-12T03:03:06.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.065+0000 7f775bdc0740 10 client.0 _ll_register_callbacks cb 0x7f7744004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:06.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.065+0000 7f77537fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc7a21fd40 0x55fc7a219e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:06.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.065+0000 7f77537fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc7a21fd40 0x55fc7a219e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39250/0 (socket says 172.21.15.116:39250) 2024-09-12T03:03:06.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.065+0000 7f77537fe640 1 -- 172.21.15.116:0/3994746762 learned_addr learned my addr 172.21.15.116:0/3994746762 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:06.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.065+0000 7f7752ffd640 1 --2- 172.21.15.116:0/3994746762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc7a214b60 0x55fc7a2284c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:06.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.065+0000 7f774bfff640 1 --2- 172.21.15.116:0/3994746762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc7a215480 0x55fc7a219930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:06.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.065+0000 7f77537fe640 1 -- 172.21.15.116:0/3994746762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc7a214b60 msgr2=0x55fc7a2284c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:06.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.065+0000 7f77537fe640 1 --2- 172.21.15.116:0/3994746762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc7a214b60 0x55fc7a2284c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.065+0000 7f77537fe640 1 -- 172.21.15.116:0/3994746762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc7a215480 msgr2=0x55fc7a219930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:06.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.065+0000 7f77537fe640 1 --2- 172.21.15.116:0/3994746762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc7a215480 0x55fc7a219930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.065+0000 7f77537fe640 1 -- 172.21.15.116:0/3994746762 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc7a21a670 con 0x55fc7a21fd40 2024-09-12T03:03:06.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.066+0000 7f77537fe640 1 --2- 172.21.15.116:0/3994746762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc7a21fd40 0x55fc7a219e70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f774c01c900 tx=0x7f774c0072a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eaebb89887b9e15 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:06.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.066+0000 7f7732ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:03:06.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.066+0000 7f7732ffd640 1 -- 172.21.15.116:0/3994746762 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c036060 con 0x55fc7a21fd40 2024-09-12T03:03:06.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.066+0000 7f7732ffd640 1 -- 172.21.15.116:0/3994746762 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f774c004c50 con 0x55fc7a21fd40 2024-09-12T03:03:06.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.066+0000 7f775bdc0740 1 -- 172.21.15.116:0/3994746762 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f774c01cbb0 con 0x55fc7a21fd40 2024-09-12T03:03:06.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.066+0000 7f7732ffd640 1 -- 172.21.15.116:0/3994746762 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f774c02b070 con 0x55fc7a21fd40 2024-09-12T03:03:06.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.066+0000 7f775bdc0740 10 client.15264 fetch_fsmap learned FSMap version 25 2024-09-12T03:03:06.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.066+0000 7f775bdc0740 1 -- 172.21.15.116:0/3994746762 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fc7a29a5a0 con 0x55fc7a21fd40 2024-09-12T03:03:06.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.066+0000 7f7732ffd640 1 -- 172.21.15.116:0/3994746762 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f774c01cbb0 con 0x55fc7a21fd40 2024-09-12T03:03:06.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.066+0000 7f7732ffd640 1 -- 172.21.15.116:0/3994746762 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f774c005450 con 0x55fc7a21fd40 2024-09-12T03:03:06.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.066+0000 7f775bdc0740 10 client.15264 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:03:06.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.066+0000 7f775bdc0740 10 client.15264 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:03:06.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.066+0000 7f775bdc0740 20 client.15264 populate_metadata read hostname 'smithi116' 2024-09-12T03:03:06.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.066+0000 7f775bdc0740 1 --2- 172.21.15.116:0/3994746762 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55fc7a29aca0 0x55fc7a29d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:06.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.066+0000 7f775bdc0740 4 client.15264 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:03:06.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.067+0000 7f775bdc0740 1 -- 172.21.15.116:0/3994746762 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fc7a181d00 con 0x55fc7a29aca0 2024-09-12T03:03:06.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.067+0000 7f7752ffd640 1 --2- 172.21.15.116:0/3994746762 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55fc7a29aca0 0x55fc7a29d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:06.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.067+0000 7f7732ffd640 10 client.15264 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:06.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.067+0000 7f7752ffd640 1 --2- 172.21.15.116:0/3994746762 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55fc7a29aca0 0x55fc7a29d0c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0a244e098298cc4 server_cookie=5b8b4116b890fa26 in_seq=0 out_seq=0 2024-09-12T03:03:06.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.068+0000 7f7732ffd640 1 -- 172.21.15.116:0/3994746762 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fc7a181d00 con 0x55fc7a29aca0 2024-09-12T03:03:06.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.068+0000 7f7732ffd640 10 client.15264 handle_command_reply: tid=0 2024-09-12T03:03:06.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.068+0000 7f775bdc0740 1 client.15264 shutdown 2024-09-12T03:03:06.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.068+0000 7f775bdc0740 1 -- 172.21.15.116:0/3994746762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc7a21fd40 msgr2=0x55fc7a219e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:06.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.068+0000 7f775bdc0740 1 --2- 172.21.15.116:0/3994746762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc7a21fd40 0x55fc7a219e70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f774c01c900 tx=0x7f774c0072a0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.068+0000 7f775bdc0740 1 -- 172.21.15.116:0/3994746762 shutdown_connections 2024-09-12T03:03:06.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.068+0000 7f775bdc0740 1 --2- 172.21.15.116:0/3994746762 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55fc7a29aca0 0x55fc7a29d0c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.068+0000 7f775bdc0740 1 --2- 172.21.15.116:0/3994746762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc7a21fd40 0x55fc7a219e70 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.068+0000 7f775bdc0740 1 --2- 172.21.15.116:0/3994746762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc7a215480 0x55fc7a219930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.068+0000 7f775bdc0740 1 --2- 172.21.15.116:0/3994746762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc7a214b60 0x55fc7a2284c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.068+0000 7f7732ffd640 0 client.15264 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:06.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.068+0000 7f775bdc0740 1 -- 172.21.15.116:0/3994746762 >> 172.21.15.116:0/3994746762 conn(0x55fc7a2110f0 msgr2=0x55fc7a210270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:06.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.068+0000 7f775bdc0740 1 -- 172.21.15.116:0/3994746762 shutdown_connections 2024-09-12T03:03:06.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.068+0000 7f775bdc0740 1 -- 172.21.15.116:0/3994746762 wait complete. 2024-09-12T03:03:06.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.069+0000 7f775bdc0740 20 client.15264 trim_cache size 0 max 16384 2024-09-12T03:03:06.075 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:03:06.075 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:03:06.075 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "b6271d4f-9b9d-48d5-bebe-81c6ec2676bd", 2024-09-12T03:03:06.076 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:03:06.076 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:03:06.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.069+0000 7f77599e7640 1 -- 172.21.15.116:0/3561604327 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f77400a2c10 msgr2=0x7f77400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:06.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.069+0000 7f77599e7640 1 --2- 172.21.15.116:0/3561604327 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f77400a2c10 0x7f77400a50d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f774c007f10 tx=0x7f774c0076a0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.069+0000 7f77599e7640 1 -- 172.21.15.116:0/3561604327 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7754101bc0 msgr2=0x7f7754199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:06.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.069+0000 7f77599e7640 1 --2- 172.21.15.116:0/3561604327 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7754101bc0 0x7f7754199860 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f773c00d950 tx=0x7f773c00de10 comp rx=0 tx=0).stop 2024-09-12T03:03:06.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.069+0000 7f77599e7640 1 -- 172.21.15.116:0/3561604327 shutdown_connections 2024-09-12T03:03:06.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.069+0000 7f77599e7640 1 --2- 172.21.15.116:0/3561604327 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f77400a2c10 0x7f77400a50d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.069+0000 7f77599e7640 1 --2- 172.21.15.116:0/3561604327 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7754105a90 0x7f77541a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.069+0000 7f77599e7640 1 --2- 172.21.15.116:0/3561604327 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7754101bc0 0x7f7754199860 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.069+0000 7f77599e7640 1 --2- 172.21.15.116:0/3561604327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7754101240 0x7f7754199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.069+0000 7f77599e7640 1 -- 172.21.15.116:0/3561604327 >> 172.21.15.116:0/3561604327 conn(0x7f7754077210 msgr2=0x7f7754103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:06.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.069+0000 7f77599e7640 1 -- 172.21.15.116:0/3561604327 shutdown_connections 2024-09-12T03:03:06.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:06.069+0000 7f77599e7640 1 -- 172.21.15.116:0/3561604327 wait complete. 2024-09-12T03:03:06.409 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:06 smithi116 ceph-mon[31410]: pgmap v341: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-12T03:03:06.409 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:06 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:03:06.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:06 smithi183 ceph-mon[35535]: pgmap v341: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-12T03:03:06.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:06 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:03:06.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:06 smithi177 ceph-mon[35305]: pgmap v341: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-12T03:03:06.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:06 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:03:07.109 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:03:07.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:07 smithi116 ceph-mon[31410]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:03:07.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:07 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:03:07.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:07 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:07.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:07 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:03:07.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:07 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:07.584 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-09-12T03:03:05.137466+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665200,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":842,"num_read_kb":541,"num_write":361,"num_write_kb":3069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180875,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":789,"ondisk_log_size":789,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368404,"kb_used_data":10836,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573420,"statfs":{"total":1151940427776,"available":1151563182080,"internally_reserved":0,"allocated":11096064,"data_stored":7170372,"data_compressed":107490,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8987,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":74,"num_read_kb":41,"num_write":12,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002965"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.133534+0000","last_change":"2024-09-12T03:00:58.348563+0000","last_active":"2024-09-12T03:02:33.133534+0000","last_peered":"2024-09-12T03:02:33.133534+0000","last_clean":"2024-09-12T03:02:33.133534+0000","last_became_active":"2024-09-12T03:00:58.347940+0000","last_became_peered":"2024-09-12T03:00:58.347940+0000","last_unstale":"2024-09-12T03:02:33.133534+0000","last_undegraded":"2024-09-12T03:02:33.133534+0000","last_fullsized":"2024-09-12T03:02:33.133534+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:43.489238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.119339+0000","last_change":"2024-09-12T03:00:58.351969+0000","last_active":"2024-09-12T03:02:33.119339+0000","last_peered":"2024-09-12T03:02:33.119339+0000","last_clean":"2024-09-12T03:02:33.119339+0000","last_became_active":"2024-09-12T03:00:58.351679+0000","last_became_peered":"2024-09-12T03:00:58.351679+0000","last_unstale":"2024-09-12T03:02:33.119339+0000","last_undegraded":"2024-09-12T03:02:33.119339+0000","last_fullsized":"2024-09-12T03:02:33.119339+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:27:01.174890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120214+0000","last_change":"2024-09-12T03:00:58.346045+0000","last_active":"2024-09-12T03:02:33.120214+0000","last_peered":"2024-09-12T03:02:33.120214+0000","last_clean":"2024-09-12T03:02:33.120214+0000","last_became_active":"2024-09-12T03:00:58.345719+0000","last_became_peered":"2024-09-12T03:00:58.345719+0000","last_unstale":"2024-09-12T03:02:33.120214+0000","last_undegraded":"2024-09-12T03:02:33.120214+0000","last_fullsized":"2024-09-12T03:02:33.120214+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:12.312650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.111069+0000","last_change":"2024-09-12T03:00:58.348165+0000","last_active":"2024-09-12T03:02:33.111069+0000","last_peered":"2024-09-12T03:02:33.111069+0000","last_clean":"2024-09-12T03:02:33.111069+0000","last_became_active":"2024-09-12T03:00:58.347734+0000","last_became_peered":"2024-09-12T03:00:58.347734+0000","last_unstale":"2024-09-12T03:02:33.111069+0000","last_undegraded":"2024-09-12T03:02:33.111069+0000","last_fullsized":"2024-09-12T03:02:33.111069+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:05.945900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.108519+0000","last_change":"2024-09-12T03:00:58.339316+0000","last_active":"2024-09-12T03:02:33.108519+0000","last_peered":"2024-09-12T03:02:33.108519+0000","last_clean":"2024-09-12T03:02:33.108519+0000","last_became_active":"2024-09-12T03:00:58.338964+0000","last_became_peered":"2024-09-12T03:00:58.338964+0000","last_unstale":"2024-09-12T03:02:33.108519+0000","last_undegraded":"2024-09-12T03:02:33.108519+0000","last_fullsized":"2024-09-12T03:02:33.108519+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:50.721276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.110921+0000","last_change":"2024-09-12T03:00:58.346838+0000","last_active":"2024-09-12T03:02:33.110921+0000","last_peered":"2024-09-12T03:02:33.110921+0000","last_clean":"2024-09-12T03:02:33.110921+0000","last_became_active":"2024-09-12T03:00:58.346486+0000","last_became_peered":"2024-09-12T03:00:58.346486+0000","last_unstale":"2024-09-12T03:02:33.110921+0000","last_undegraded":"2024-09-12T03:02:33.110921+0000","last_fullsized":"2024-09-12T03:02:33.110921+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:28.687727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.122919+0000","last_change":"2024-09-12T03:00:58.346712+0000","last_active":"2024-09-12T03:02:33.122919+0000","last_peered":"2024-09-12T03:02:33.122919+0000","last_clean":"2024-09-12T03:02:33.122919+0000","last_became_active":"2024-09-12T03:00:58.346397+0000","last_became_peered":"2024-09-12T03:00:58.346397+0000","last_unstale":"2024-09-12T03:02:33.122919+0000","last_undegraded":"2024-09-12T03:02:33.122919+0000","last_fullsized":"2024-09-12T03:02:33.122919+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:48:00.624733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.118885+0000","last_change":"2024-09-12T03:00:58.356166+0000","last_active":"2024-09-12T03:02:33.118885+0000","last_peered":"2024-09-12T03:02:33.118885+0000","last_clean":"2024-09-12T03:02:33.118885+0000","last_became_active":"2024-09-12T03:00:58.354944+0000","last_became_peered":"2024-09-12T03:00:58.354944+0000","last_unstale":"2024-09-12T03:02:33.118885+0000","last_undegraded":"2024-09-12T03:02:33.118885+0000","last_fullsized":"2024-09-12T03:02:33.118885+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:15.840149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.133064+0000","last_change":"2024-09-12T03:00:58.340182+0000","last_active":"2024-09-12T03:02:33.133064+0000","last_peered":"2024-09-12T03:02:33.133064+0000","last_clean":"2024-09-12T03:02:33.133064+0000","last_became_active":"2024-09-12T03:00:58.339576+0000","last_became_peered":"2024-09-12T03:00:58.339576+0000","last_unstale":"2024-09-12T03:02:33.133064+0000","last_undegraded":"2024-09-12T03:02:33.133064+0000","last_fullsized":"2024-09-12T03:02:33.133064+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:16.534844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.117254+0000","last_change":"2024-09-12T03:00:58.351218+0000","last_active":"2024-09-12T03:02:33.117254+0000","last_peered":"2024-09-12T03:02:33.117254+0000","last_clean":"2024-09-12T03:02:33.117254+0000","last_became_active":"2024-09-12T03:00:58.350871+0000","last_became_peered":"2024-09-12T03:00:58.350871+0000","last_unstale":"2024-09-12T03:02:33.117254+0000","last_undegraded":"2024-09-12T03:02:33.117254+0000","last_fullsized":"2024-09-12T03:02:33.117254+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:38.367683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.130892+0000","last_change":"2024-09-12T03:00:58.351894+0000","last_active":"2024-09-12T03:02:33.130892+0000","last_peered":"2024-09-12T03:02:33.130892+0000","last_clean":"2024-09-12T03:02:33.130892+0000","last_became_active":"2024-09-12T03:00:58.351445+0000","last_became_peered":"2024-09-12T03:00:58.351445+0000","last_unstale":"2024-09-12T03:02:33.130892+0000","last_undegraded":"2024-09-12T03:02:33.130892+0000","last_fullsized":"2024-09-12T03:02:33.130892+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:42.093871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.107323+0000","last_change":"2024-09-12T03:00:58.353460+0000","last_active":"2024-09-12T03:02:33.107323+0000","last_peered":"2024-09-12T03:02:33.107323+0000","last_clean":"2024-09-12T03:02:33.107323+0000","last_became_active":"2024-09-12T03:00:58.353111+0000","last_became_peered":"2024-09-12T03:00:58.353111+0000","last_unstale":"2024-09-12T03:02:33.107323+0000","last_undegraded":"2024-09-12T03:02:33.107323+0000","last_fullsized":"2024-09-12T03:02:33.107323+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:15.916765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.111860+0000","last_change":"2024-09-12T03:00:58.341080+0000","last_active":"2024-09-12T03:02:33.111860+0000","last_peered":"2024-09-12T03:02:33.111860+0000","last_clean":"2024-09-12T03:02:33.111860+0000","last_became_active":"2024-09-12T03:00:58.340782+0000","last_became_peered":"2024-09-12T03:00:58.340782+0000","last_unstale":"2024-09-12T03:02:33.111860+0000","last_undegraded":"2024-09-12T03:02:33.111860+0000","last_fullsized":"2024-09-12T03:02:33.111860+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:36.569700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.106506+0000","last_change":"2024-09-12T03:00:58.346791+0000","last_active":"2024-09-12T03:02:33.106506+0000","last_peered":"2024-09-12T03:02:33.106506+0000","last_clean":"2024-09-12T03:02:33.106506+0000","last_became_active":"2024-09-12T03:00:58.346500+0000","last_became_peered":"2024-09-12T03:00:58.346500+0000","last_unstale":"2024-09-12T03:02:33.106506+0000","last_undegraded":"2024-09-12T03:02:33.106506+0000","last_fullsized":"2024-09-12T03:02:33.106506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:32.017589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.112821+0000","last_change":"2024-09-12T03:00:58.349522+0000","last_active":"2024-09-12T03:02:33.112821+0000","last_peered":"2024-09-12T03:02:33.112821+0000","last_clean":"2024-09-12T03:02:33.112821+0000","last_became_active":"2024-09-12T03:00:58.349246+0000","last_became_peered":"2024-09-12T03:00:58.349246+0000","last_unstale":"2024-09-12T03:02:33.112821+0000","last_undegraded":"2024-09-12T03:02:33.112821+0000","last_fullsized":"2024-09-12T03:02:33.112821+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:39.253919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.134146+0000","last_change":"2024-09-12T03:00:58.349909+0000","last_active":"2024-09-12T03:02:33.134146+0000","last_peered":"2024-09-12T03:02:33.134146+0000","last_clean":"2024-09-12T03:02:33.134146+0000","last_became_active":"2024-09-12T03:00:58.349590+0000","last_became_peered":"2024-09-12T03:00:58.349590+0000","last_unstale":"2024-09-12T03:02:33.134146+0000","last_undegraded":"2024-09-12T03:02:33.134146+0000","last_fullsized":"2024-09-12T03:02:33.134146+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:04.540709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.133565+0000","last_change":"2024-09-12T03:00:58.348284+0000","last_active":"2024-09-12T03:02:33.133565+0000","last_peered":"2024-09-12T03:02:33.133565+0000","last_clean":"2024-09-12T03:02:33.133565+0000","last_became_active":"2024-09-12T03:00:58.347465+0000","last_became_peered":"2024-09-12T03:00:58.347465+0000","last_unstale":"2024-09-12T03:02:33.133565+0000","last_undegraded":"2024-09-12T03:02:33.133565+0000","last_fullsized":"2024-09-12T03:02:33.133565+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:13.191899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.122657+0000","last_change":"2024-09-12T03:00:58.337740+0000","last_active":"2024-09-12T03:02:33.122657+0000","last_peered":"2024-09-12T03:02:33.122657+0000","last_clean":"2024-09-12T03:02:33.122657+0000","last_became_active":"2024-09-12T03:00:58.337393+0000","last_became_peered":"2024-09-12T03:00:58.337393+0000","last_unstale":"2024-09-12T03:02:33.122657+0000","last_undegraded":"2024-09-12T03:02:33.122657+0000","last_fullsized":"2024-09-12T03:02:33.122657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:28:56.339167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.111666+0000","last_change":"2024-09-12T03:00:58.345779+0000","last_active":"2024-09-12T03:02:33.111666+0000","last_peered":"2024-09-12T03:02:33.111666+0000","last_clean":"2024-09-12T03:02:33.111666+0000","last_became_active":"2024-09-12T03:00:58.345210+0000","last_became_peered":"2024-09-12T03:00:58.345210+0000","last_unstale":"2024-09-12T03:02:33.111666+0000","last_undegraded":"2024-09-12T03:02:33.111666+0000","last_fullsized":"2024-09-12T03:02:33.111666+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:06:58.271185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120653+0000","last_change":"2024-09-12T03:02:04.294381+0000","last_active":"2024-09-12T03:02:33.120653+0000","last_peered":"2024-09-12T03:02:33.120653+0000","last_clean":"2024-09-12T03:02:33.120653+0000","last_became_active":"2024-09-12T03:02:04.293795+0000","last_became_peered":"2024-09-12T03:02:04.293795+0000","last_unstale":"2024-09-12T03:02:33.120653+0000","last_undegraded":"2024-09-12T03:02:33.120653+0000","last_fullsized":"2024-09-12T03:02:33.120653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:06:58.271007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.124292+0000","last_change":"2024-09-12T03:00:58.349233+0000","last_active":"2024-09-12T03:02:33.124292+0000","last_peered":"2024-09-12T03:02:33.124292+0000","last_clean":"2024-09-12T03:02:33.124292+0000","last_became_active":"2024-09-12T03:00:58.348958+0000","last_became_peered":"2024-09-12T03:00:58.348958+0000","last_unstale":"2024-09-12T03:02:33.124292+0000","last_undegraded":"2024-09-12T03:02:33.124292+0000","last_fullsized":"2024-09-12T03:02:33.124292+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:32:10.271817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.134703+0000","last_change":"2024-09-12T03:00:58.358024+0000","last_active":"2024-09-12T03:02:33.134703+0000","last_peered":"2024-09-12T03:02:33.134703+0000","last_clean":"2024-09-12T03:02:33.134703+0000","last_became_active":"2024-09-12T03:00:58.357434+0000","last_became_peered":"2024-09-12T03:00:58.357434+0000","last_unstale":"2024-09-12T03:02:33.134703+0000","last_undegraded":"2024-09-12T03:02:33.134703+0000","last_fullsized":"2024-09-12T03:02:33.134703+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:01:20.119733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120095+0000","last_change":"2024-09-12T03:00:58.355063+0000","last_active":"2024-09-12T03:02:33.120095+0000","last_peered":"2024-09-12T03:02:33.120095+0000","last_clean":"2024-09-12T03:02:33.120095+0000","last_became_active":"2024-09-12T03:00:58.354760+0000","last_became_peered":"2024-09-12T03:00:58.354760+0000","last_unstale":"2024-09-12T03:02:33.120095+0000","last_undegraded":"2024-09-12T03:02:33.120095+0000","last_fullsized":"2024-09-12T03:02:33.120095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:13.464413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.112255+0000","last_change":"2024-09-12T03:00:58.341916+0000","last_active":"2024-09-12T03:02:33.112255+0000","last_peered":"2024-09-12T03:02:33.112255+0000","last_clean":"2024-09-12T03:02:33.112255+0000","last_became_active":"2024-09-12T03:00:58.340815+0000","last_became_peered":"2024-09-12T03:00:58.340815+0000","last_unstale":"2024-09-12T03:02:33.112255+0000","last_undegraded":"2024-09-12T03:02:33.112255+0000","last_fullsized":"2024-09-12T03:02:33.112255+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:13.777947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.117061+0000","last_change":"2024-09-12T03:00:58.357765+0000","last_active":"2024-09-12T03:02:33.117061+0000","last_peered":"2024-09-12T03:02:33.117061+0000","last_clean":"2024-09-12T03:02:33.117061+0000","last_became_active":"2024-09-12T03:00:58.357480+0000","last_became_peered":"2024-09-12T03:00:58.357480+0000","last_unstale":"2024-09-12T03:02:33.117061+0000","last_undegraded":"2024-09-12T03:02:33.117061+0000","last_fullsized":"2024-09-12T03:02:33.117061+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:26.884381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.108215+0000","last_change":"2024-09-12T03:00:58.349504+0000","last_active":"2024-09-12T03:02:33.108215+0000","last_peered":"2024-09-12T03:02:33.108215+0000","last_clean":"2024-09-12T03:02:33.108215+0000","last_became_active":"2024-09-12T03:00:58.349136+0000","last_became_peered":"2024-09-12T03:00:58.349136+0000","last_unstale":"2024-09-12T03:02:33.108215+0000","last_undegraded":"2024-09-12T03:02:33.108215+0000","last_fullsized":"2024-09-12T03:02:33.108215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:09:55.229516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.122920+0000","last_change":"2024-09-12T03:00:58.341902+0000","last_active":"2024-09-12T03:02:33.122920+0000","last_peered":"2024-09-12T03:02:33.122920+0000","last_clean":"2024-09-12T03:02:33.122920+0000","last_became_active":"2024-09-12T03:00:58.341592+0000","last_became_peered":"2024-09-12T03:00:58.341592+0000","last_unstale":"2024-09-12T03:02:33.122920+0000","last_undegraded":"2024-09-12T03:02:33.122920+0000","last_fullsized":"2024-09-12T03:02:33.122920+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:13.852511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.114521+0000","last_change":"2024-09-12T03:00:58.348012+0000","last_active":"2024-09-12T03:02:33.114521+0000","last_peered":"2024-09-12T03:02:33.114521+0000","last_clean":"2024-09-12T03:02:33.114521+0000","last_became_active":"2024-09-12T03:00:58.347338+0000","last_became_peered":"2024-09-12T03:00:58.347338+0000","last_unstale":"2024-09-12T03:02:33.114521+0000","last_undegraded":"2024-09-12T03:02:33.114521+0000","last_fullsized":"2024-09-12T03:02:33.114521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:41.478032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.113228+0000","last_change":"2024-09-12T03:00:58.345833+0000","last_active":"2024-09-12T03:02:33.113228+0000","last_peered":"2024-09-12T03:02:33.113228+0000","last_clean":"2024-09-12T03:02:33.113228+0000","last_became_active":"2024-09-12T03:00:58.345482+0000","last_became_peered":"2024-09-12T03:00:58.345482+0000","last_unstale":"2024-09-12T03:02:33.113228+0000","last_undegraded":"2024-09-12T03:02:33.113228+0000","last_fullsized":"2024-09-12T03:02:33.113228+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:32.037412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.112949+0000","last_change":"2024-09-12T03:00:58.352290+0000","last_active":"2024-09-12T03:02:33.112949+0000","last_peered":"2024-09-12T03:02:33.112949+0000","last_clean":"2024-09-12T03:02:33.112949+0000","last_became_active":"2024-09-12T03:00:58.351918+0000","last_became_peered":"2024-09-12T03:00:58.351918+0000","last_unstale":"2024-09-12T03:02:33.112949+0000","last_undegraded":"2024-09-12T03:02:33.112949+0000","last_fullsized":"2024-09-12T03:02:33.112949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:25.941080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.116096+0000","last_change":"2024-09-12T03:00:58.348021+0000","last_active":"2024-09-12T03:02:33.116096+0000","last_peered":"2024-09-12T03:02:33.116096+0000","last_clean":"2024-09-12T03:02:33.116096+0000","last_became_active":"2024-09-12T03:00:58.347634+0000","last_became_peered":"2024-09-12T03:00:58.347634+0000","last_unstale":"2024-09-12T03:02:33.116096+0000","last_undegraded":"2024-09-12T03:02:33.116096+0000","last_fullsized":"2024-09-12T03:02:33.116096+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:47.933067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.123517+0000","last_change":"2024-09-12T03:00:58.340891+0000","last_active":"2024-09-12T03:02:33.123517+0000","last_peered":"2024-09-12T03:02:33.123517+0000","last_clean":"2024-09-12T03:02:33.123517+0000","last_became_active":"2024-09-12T03:00:58.340504+0000","last_became_peered":"2024-09-12T03:00:58.340504+0000","last_unstale":"2024-09-12T03:02:33.123517+0000","last_undegraded":"2024-09-12T03:02:33.123517+0000","last_fullsized":"2024-09-12T03:02:33.123517+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:32.117635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.108754+0000","last_change":"2024-09-12T03:00:58.339343+0000","last_active":"2024-09-12T03:02:33.108754+0000","last_peered":"2024-09-12T03:02:33.108754+0000","last_clean":"2024-09-12T03:02:33.108754+0000","last_became_active":"2024-09-12T03:00:58.339001+0000","last_became_peered":"2024-09-12T03:00:58.339001+0000","last_unstale":"2024-09-12T03:02:33.108754+0000","last_undegraded":"2024-09-12T03:02:33.108754+0000","last_fullsized":"2024-09-12T03:02:33.108754+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:02.043454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":145,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:03:02.571564+0000","last_change":"2024-09-12T03:00:56.315550+0000","last_active":"2024-09-12T03:03:02.571564+0000","last_peered":"2024-09-12T03:03:02.571564+0000","last_clean":"2024-09-12T03:03:02.571564+0000","last_became_active":"2024-09-12T03:00:56.315225+0000","last_became_peered":"2024-09-12T03:00:56.315225+0000","last_unstale":"2024-09-12T03:03:02.571564+0000","last_undegraded":"2024-09-12T03:03:02.571564+0000","last_fullsized":"2024-09-12T03:03:02.571564+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:46.070895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.116508+0000","last_change":"2024-09-12T03:02:04.290498+0000","last_active":"2024-09-12T03:02:33.116508+0000","last_peered":"2024-09-12T03:02:33.116508+0000","last_clean":"2024-09-12T03:02:33.116508+0000","last_became_active":"2024-09-12T03:02:04.290162+0000","last_became_peered":"2024-09-12T03:02:04.290162+0000","last_unstale":"2024-09-12T03:02:33.116508+0000","last_undegraded":"2024-09-12T03:02:33.116508+0000","last_fullsized":"2024-09-12T03:02:33.116508+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:29.777155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282566+0000","last_change":"2024-09-12T03:00:56.316385+0000","last_active":"2024-09-12T03:02:04.282566+0000","last_peered":"2024-09-12T03:02:04.282566+0000","last_clean":"2024-09-12T03:02:04.282566+0000","last_became_active":"2024-09-12T03:00:56.316064+0000","last_became_peered":"2024-09-12T03:00:56.316064+0000","last_unstale":"2024-09-12T03:02:04.282566+0000","last_undegraded":"2024-09-12T03:02:04.282566+0000","last_fullsized":"2024-09-12T03:02:04.282566+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:12:43.323654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332424+0000","last_change":"2024-09-12T03:00:56.313065+0000","last_active":"2024-09-12T03:02:04.332424+0000","last_peered":"2024-09-12T03:02:04.332424+0000","last_clean":"2024-09-12T03:02:04.332424+0000","last_became_active":"2024-09-12T03:00:56.312716+0000","last_became_peered":"2024-09-12T03:00:56.312716+0000","last_unstale":"2024-09-12T03:02:04.332424+0000","last_undegraded":"2024-09-12T03:02:04.332424+0000","last_fullsized":"2024-09-12T03:02:04.332424+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:51.089089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120152+0000","last_change":"2024-09-12T03:00:58.346542+0000","last_active":"2024-09-12T03:02:33.120152+0000","last_peered":"2024-09-12T03:02:33.120152+0000","last_clean":"2024-09-12T03:02:33.120152+0000","last_became_active":"2024-09-12T03:00:58.346260+0000","last_became_peered":"2024-09-12T03:00:58.346260+0000","last_unstale":"2024-09-12T03:02:33.120152+0000","last_undegraded":"2024-09-12T03:02:33.120152+0000","last_fullsized":"2024-09-12T03:02:33.120152+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:37.034969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289182+0000","last_change":"2024-09-12T03:00:56.317308+0000","last_active":"2024-09-12T03:02:04.289182+0000","last_peered":"2024-09-12T03:02:04.289182+0000","last_clean":"2024-09-12T03:02:04.289182+0000","last_became_active":"2024-09-12T03:00:56.316935+0000","last_became_peered":"2024-09-12T03:00:56.316935+0000","last_unstale":"2024-09-12T03:02:04.289182+0000","last_undegraded":"2024-09-12T03:02:04.289182+0000","last_fullsized":"2024-09-12T03:02:04.289182+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:06.324489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.118033+0000","last_change":"2024-09-12T03:00:58.341565+0000","last_active":"2024-09-12T03:02:33.118033+0000","last_peered":"2024-09-12T03:02:33.118033+0000","last_clean":"2024-09-12T03:02:33.118033+0000","last_became_active":"2024-09-12T03:00:58.341285+0000","last_became_peered":"2024-09-12T03:00:58.341285+0000","last_unstale":"2024-09-12T03:02:33.118033+0000","last_undegraded":"2024-09-12T03:02:33.118033+0000","last_fullsized":"2024-09-12T03:02:33.118033+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:59.156700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":138,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:59.565408+0000","last_change":"2024-09-12T03:00:56.314201+0000","last_active":"2024-09-12T03:02:59.565408+0000","last_peered":"2024-09-12T03:02:59.565408+0000","last_clean":"2024-09-12T03:02:59.565408+0000","last_became_active":"2024-09-12T03:00:56.313880+0000","last_became_peered":"2024-09-12T03:00:56.313880+0000","last_unstale":"2024-09-12T03:02:59.565408+0000","last_undegraded":"2024-09-12T03:02:59.565408+0000","last_fullsized":"2024-09-12T03:02:59.565408+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:57:12.952630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.132547+0000","last_change":"2024-09-12T03:00:58.345274+0000","last_active":"2024-09-12T03:02:33.132547+0000","last_peered":"2024-09-12T03:02:33.132547+0000","last_clean":"2024-09-12T03:02:33.132547+0000","last_became_active":"2024-09-12T03:00:58.344937+0000","last_became_peered":"2024-09-12T03:00:58.344937+0000","last_unstale":"2024-09-12T03:02:33.132547+0000","last_undegraded":"2024-09-12T03:02:33.132547+0000","last_fullsized":"2024-09-12T03:02:33.132547+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:17.288909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284614+0000","last_change":"2024-09-12T03:00:56.314709+0000","last_active":"2024-09-12T03:02:04.284614+0000","last_peered":"2024-09-12T03:02:04.284614+0000","last_clean":"2024-09-12T03:02:04.284614+0000","last_became_active":"2024-09-12T03:00:56.314349+0000","last_became_peered":"2024-09-12T03:00:56.314349+0000","last_unstale":"2024-09-12T03:02:04.284614+0000","last_undegraded":"2024-09-12T03:02:04.284614+0000","last_fullsized":"2024-09-12T03:02:04.284614+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:18:35.371661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.113713+0000","last_change":"2024-09-12T03:00:58.346055+0000","last_active":"2024-09-12T03:02:33.113713+0000","last_peered":"2024-09-12T03:02:33.113713+0000","last_clean":"2024-09-12T03:02:33.113713+0000","last_became_active":"2024-09-12T03:00:58.345701+0000","last_became_peered":"2024-09-12T03:00:58.345701+0000","last_unstale":"2024-09-12T03:02:33.113713+0000","last_undegraded":"2024-09-12T03:02:33.113713+0000","last_fullsized":"2024-09-12T03:02:33.113713+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:33.977245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.287450+0000","last_change":"2024-09-12T03:00:56.435962+0000","last_active":"2024-09-12T03:02:04.287450+0000","last_peered":"2024-09-12T03:02:04.287450+0000","last_clean":"2024-09-12T03:02:04.287450+0000","last_became_active":"2024-09-12T03:00:56.435515+0000","last_became_peered":"2024-09-12T03:00:56.435515+0000","last_unstale":"2024-09-12T03:02:04.287450+0000","last_undegraded":"2024-09-12T03:02:04.287450+0000","last_fullsized":"2024-09-12T03:02:04.287450+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:32.953436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.135045+0000","last_change":"2024-09-12T03:00:58.344239+0000","last_active":"2024-09-12T03:02:33.135045+0000","last_peered":"2024-09-12T03:02:33.135045+0000","last_clean":"2024-09-12T03:02:33.135045+0000","last_became_active":"2024-09-12T03:00:58.343876+0000","last_became_peered":"2024-09-12T03:00:58.343876+0000","last_unstale":"2024-09-12T03:02:33.135045+0000","last_undegraded":"2024-09-12T03:02:33.135045+0000","last_fullsized":"2024-09-12T03:02:33.135045+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:49.401295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'36","reported_seq":297,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:03:00.567788+0000","last_change":"2024-09-12T03:00:56.317708+0000","last_active":"2024-09-12T03:03:00.567788+0000","last_peered":"2024-09-12T03:03:00.567788+0000","last_clean":"2024-09-12T03:03:00.567788+0000","last_became_active":"2024-09-12T03:00:56.317317+0000","last_became_peered":"2024-09-12T03:00:56.317317+0000","last_unstale":"2024-09-12T03:03:00.567788+0000","last_undegraded":"2024-09-12T03:03:00.567788+0000","last_fullsized":"2024-09-12T03:03:00.567788+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:47.788186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70595,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":239,"num_read_kb":188,"num_write":37,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.123764+0000","last_change":"2024-09-12T03:00:58.343602+0000","last_active":"2024-09-12T03:02:33.123764+0000","last_peered":"2024-09-12T03:02:33.123764+0000","last_clean":"2024-09-12T03:02:33.123764+0000","last_became_active":"2024-09-12T03:00:58.343301+0000","last_became_peered":"2024-09-12T03:00:58.343301+0000","last_unstale":"2024-09-12T03:02:33.123764+0000","last_undegraded":"2024-09-12T03:02:33.123764+0000","last_fullsized":"2024-09-12T03:02:33.123764+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:04.056118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.290204+0000","last_change":"2024-09-12T03:00:56.317874+0000","last_active":"2024-09-12T03:02:04.290204+0000","last_peered":"2024-09-12T03:02:04.290204+0000","last_clean":"2024-09-12T03:02:04.290204+0000","last_became_active":"2024-09-12T03:00:56.317566+0000","last_became_peered":"2024-09-12T03:00:56.317566+0000","last_unstale":"2024-09-12T03:02:04.290204+0000","last_undegraded":"2024-09-12T03:02:04.290204+0000","last_fullsized":"2024-09-12T03:02:04.290204+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:54.203730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.121582+0000","last_change":"2024-09-12T03:00:58.351539+0000","last_active":"2024-09-12T03:02:33.121582+0000","last_peered":"2024-09-12T03:02:33.121582+0000","last_clean":"2024-09-12T03:02:33.121582+0000","last_became_active":"2024-09-12T03:00:58.351249+0000","last_became_peered":"2024-09-12T03:00:58.351249+0000","last_unstale":"2024-09-12T03:02:33.121582+0000","last_undegraded":"2024-09-12T03:02:33.121582+0000","last_fullsized":"2024-09-12T03:02:33.121582+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:40.498867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.287068+0000","last_change":"2024-09-12T03:00:56.314489+0000","last_active":"2024-09-12T03:02:04.287068+0000","last_peered":"2024-09-12T03:02:04.287068+0000","last_clean":"2024-09-12T03:02:04.287068+0000","last_became_active":"2024-09-12T03:00:56.314142+0000","last_became_peered":"2024-09-12T03:00:56.314142+0000","last_unstale":"2024-09-12T03:02:04.287068+0000","last_undegraded":"2024-09-12T03:02:04.287068+0000","last_fullsized":"2024-09-12T03:02:04.287068+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:21.541713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.123797+0000","last_change":"2024-09-12T03:00:58.348752+0000","last_active":"2024-09-12T03:02:33.123797+0000","last_peered":"2024-09-12T03:02:33.123797+0000","last_clean":"2024-09-12T03:02:33.123797+0000","last_became_active":"2024-09-12T03:00:58.348391+0000","last_became_peered":"2024-09-12T03:00:58.348391+0000","last_unstale":"2024-09-12T03:02:33.123797+0000","last_undegraded":"2024-09-12T03:02:33.123797+0000","last_fullsized":"2024-09-12T03:02:33.123797+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:33:15.626306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'4","reported_seq":146,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:03:03.572627+0000","last_change":"2024-09-12T03:00:56.311702+0000","last_active":"2024-09-12T03:03:03.572627+0000","last_peered":"2024-09-12T03:03:03.572627+0000","last_clean":"2024-09-12T03:03:03.572627+0000","last_became_active":"2024-09-12T03:00:56.311353+0000","last_became_peered":"2024-09-12T03:00:56.311353+0000","last_unstale":"2024-09-12T03:03:03.572627+0000","last_undegraded":"2024-09-12T03:03:03.572627+0000","last_fullsized":"2024-09-12T03:03:03.572627+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:01.004858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.123076+0000","last_change":"2024-09-12T03:00:58.351973+0000","last_active":"2024-09-12T03:02:33.123076+0000","last_peered":"2024-09-12T03:02:33.123076+0000","last_clean":"2024-09-12T03:02:33.123076+0000","last_became_active":"2024-09-12T03:00:58.351599+0000","last_became_peered":"2024-09-12T03:00:58.351599+0000","last_unstale":"2024-09-12T03:02:33.123076+0000","last_undegraded":"2024-09-12T03:02:33.123076+0000","last_fullsized":"2024-09-12T03:02:33.123076+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:26.398828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.291749+0000","last_change":"2024-09-12T03:00:56.436185+0000","last_active":"2024-09-12T03:02:04.291749+0000","last_peered":"2024-09-12T03:02:04.291749+0000","last_clean":"2024-09-12T03:02:04.291749+0000","last_became_active":"2024-09-12T03:00:56.435717+0000","last_became_peered":"2024-09-12T03:00:56.435717+0000","last_unstale":"2024-09-12T03:02:04.291749+0000","last_undegraded":"2024-09-12T03:02:04.291749+0000","last_fullsized":"2024-09-12T03:02:04.291749+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:27.303182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.119766+0000","last_change":"2024-09-12T03:02:04.333661+0000","last_active":"2024-09-12T03:02:33.119766+0000","last_peered":"2024-09-12T03:02:33.119766+0000","last_clean":"2024-09-12T03:02:33.119766+0000","last_became_active":"2024-09-12T03:02:04.333217+0000","last_became_peered":"2024-09-12T03:02:04.333217+0000","last_unstale":"2024-09-12T03:02:33.119766+0000","last_undegraded":"2024-09-12T03:02:33.119766+0000","last_fullsized":"2024-09-12T03:02:33.119766+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:23:47.212734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'3","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:08.214796+0000","last_change":"2024-09-12T03:02:04.296884+0000","last_active":"2024-09-12T03:02:08.214796+0000","last_peered":"2024-09-12T03:02:08.214796+0000","last_clean":"2024-09-12T03:02:08.214796+0000","last_became_active":"2024-09-12T03:02:04.290894+0000","last_became_peered":"2024-09-12T03:02:04.290894+0000","last_unstale":"2024-09-12T03:02:08.214796+0000","last_undegraded":"2024-09-12T03:02:08.214796+0000","last_fullsized":"2024-09-12T03:02:08.214796+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:49:19.749568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.118228+0000","last_change":"2024-09-12T03:02:04.334096+0000","last_active":"2024-09-12T03:02:33.118228+0000","last_peered":"2024-09-12T03:02:33.118228+0000","last_clean":"2024-09-12T03:02:33.118228+0000","last_became_active":"2024-09-12T03:02:04.333641+0000","last_became_peered":"2024-09-12T03:02:04.333641+0000","last_unstale":"2024-09-12T03:02:33.118228+0000","last_undegraded":"2024-09-12T03:02:33.118228+0000","last_fullsized":"2024-09-12T03:02:33.118228+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:24.085293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332966+0000","last_change":"2024-09-12T03:00:56.314182+0000","last_active":"2024-09-12T03:02:04.332966+0000","last_peered":"2024-09-12T03:02:04.332966+0000","last_clean":"2024-09-12T03:02:04.332966+0000","last_became_active":"2024-09-12T03:00:56.313749+0000","last_became_peered":"2024-09-12T03:00:56.313749+0000","last_unstale":"2024-09-12T03:02:04.332966+0000","last_undegraded":"2024-09-12T03:02:04.332966+0000","last_fullsized":"2024-09-12T03:02:04.332966+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:31:45.767500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.125121+0000","last_change":"2024-09-12T03:00:58.344356+0000","last_active":"2024-09-12T03:02:33.125121+0000","last_peered":"2024-09-12T03:02:33.125121+0000","last_clean":"2024-09-12T03:02:33.125121+0000","last_became_active":"2024-09-12T03:00:58.344035+0000","last_became_peered":"2024-09-12T03:00:58.344035+0000","last_unstale":"2024-09-12T03:02:33.125121+0000","last_undegraded":"2024-09-12T03:02:33.125121+0000","last_fullsized":"2024-09-12T03:02:33.125121+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:45:18.681018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.287585+0000","last_change":"2024-09-12T03:00:56.315653+0000","last_active":"2024-09-12T03:02:04.287585+0000","last_peered":"2024-09-12T03:02:04.287585+0000","last_clean":"2024-09-12T03:02:04.287585+0000","last_became_active":"2024-09-12T03:00:56.315284+0000","last_became_peered":"2024-09-12T03:00:56.315284+0000","last_unstale":"2024-09-12T03:02:04.287585+0000","last_undegraded":"2024-09-12T03:02:04.287585+0000","last_fullsized":"2024-09-12T03:02:04.287585+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:42.934020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.122059+0000","last_change":"2024-09-12T03:02:04.291128+0000","last_active":"2024-09-12T03:02:33.122059+0000","last_peered":"2024-09-12T03:02:33.122059+0000","last_clean":"2024-09-12T03:02:33.122059+0000","last_became_active":"2024-09-12T03:02:04.287836+0000","last_became_peered":"2024-09-12T03:02:04.287836+0000","last_unstale":"2024-09-12T03:02:33.122059+0000","last_undegraded":"2024-09-12T03:02:33.122059+0000","last_fullsized":"2024-09-12T03:02:33.122059+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:23.956579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.290577+0000","last_change":"2024-09-12T03:00:56.746780+0000","last_active":"2024-09-12T03:02:04.290577+0000","last_peered":"2024-09-12T03:02:04.290577+0000","last_clean":"2024-09-12T03:02:04.290577+0000","last_became_active":"2024-09-12T03:00:56.746414+0000","last_became_peered":"2024-09-12T03:00:56.746414+0000","last_unstale":"2024-09-12T03:02:04.290577+0000","last_undegraded":"2024-09-12T03:02:04.290577+0000","last_fullsized":"2024-09-12T03:02:04.290577+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:15:35.214196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.131100+0000","last_change":"2024-09-12T03:00:58.351713+0000","last_active":"2024-09-12T03:02:33.131100+0000","last_peered":"2024-09-12T03:02:33.131100+0000","last_clean":"2024-09-12T03:02:33.131100+0000","last_became_active":"2024-09-12T03:00:58.351371+0000","last_became_peered":"2024-09-12T03:00:58.351371+0000","last_unstale":"2024-09-12T03:02:33.131100+0000","last_undegraded":"2024-09-12T03:02:33.131100+0000","last_fullsized":"2024-09-12T03:02:33.131100+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:13.346956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'137","reported_seq":336,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284318+0000","last_change":"2024-09-12T02:59:45.461482+0000","last_active":"2024-09-12T03:02:04.284318+0000","last_peered":"2024-09-12T03:02:04.284318+0000","last_clean":"2024-09-12T03:02:04.284318+0000","last_became_active":"2024-09-12T02:59:45.461058+0000","last_became_peered":"2024-09-12T02:59:45.461058+0000","last_unstale":"2024-09-12T03:02:04.284318+0000","last_undegraded":"2024-09-12T03:02:04.284318+0000","last_fullsized":"2024-09-12T03:02:04.284318+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:18.034438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:18.034438+0000","last_clean_scrub_stamp":"2024-09-12T02:56:18.034438+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:09.251858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289360+0000","last_change":"2024-09-12T03:00:56.315245+0000","last_active":"2024-09-12T03:02:04.289360+0000","last_peered":"2024-09-12T03:02:04.289360+0000","last_clean":"2024-09-12T03:02:04.289360+0000","last_became_active":"2024-09-12T03:00:56.314887+0000","last_became_peered":"2024-09-12T03:00:56.314887+0000","last_unstale":"2024-09-12T03:02:04.289360+0000","last_undegraded":"2024-09-12T03:02:04.289360+0000","last_fullsized":"2024-09-12T03:02:04.289360+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:48.750657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.113189+0000","last_change":"2024-09-12T03:00:58.357874+0000","last_active":"2024-09-12T03:02:33.113189+0000","last_peered":"2024-09-12T03:02:33.113189+0000","last_clean":"2024-09-12T03:02:33.113189+0000","last_became_active":"2024-09-12T03:00:58.357575+0000","last_became_peered":"2024-09-12T03:00:58.357575+0000","last_unstale":"2024-09-12T03:02:33.113189+0000","last_undegraded":"2024-09-12T03:02:33.113189+0000","last_fullsized":"2024-09-12T03:02:33.113189+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:46.125620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284282+0000","last_change":"2024-09-12T03:00:56.312263+0000","last_active":"2024-09-12T03:02:04.284282+0000","last_peered":"2024-09-12T03:02:04.284282+0000","last_clean":"2024-09-12T03:02:04.284282+0000","last_became_active":"2024-09-12T03:00:56.311894+0000","last_became_peered":"2024-09-12T03:00:56.311894+0000","last_unstale":"2024-09-12T03:02:04.284282+0000","last_undegraded":"2024-09-12T03:02:04.284282+0000","last_fullsized":"2024-09-12T03:02:04.284282+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:52.434203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.110253+0000","last_change":"2024-09-12T03:00:58.350762+0000","last_active":"2024-09-12T03:02:33.110253+0000","last_peered":"2024-09-12T03:02:33.110253+0000","last_clean":"2024-09-12T03:02:33.110253+0000","last_became_active":"2024-09-12T03:00:58.348038+0000","last_became_peered":"2024-09-12T03:00:58.348038+0000","last_unstale":"2024-09-12T03:02:33.110253+0000","last_undegraded":"2024-09-12T03:02:33.110253+0000","last_fullsized":"2024-09-12T03:02:33.110253+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:08.916636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":141,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:03:00.567092+0000","last_change":"2024-09-12T03:00:56.435641+0000","last_active":"2024-09-12T03:03:00.567092+0000","last_peered":"2024-09-12T03:03:00.567092+0000","last_clean":"2024-09-12T03:03:00.567092+0000","last_became_active":"2024-09-12T03:00:56.435319+0000","last_became_peered":"2024-09-12T03:00:56.435319+0000","last_unstale":"2024-09-12T03:03:00.567092+0000","last_undegraded":"2024-09-12T03:03:00.567092+0000","last_fullsized":"2024-09-12T03:03:00.567092+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:56.894978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.112818+0000","last_change":"2024-09-12T03:00:58.344708+0000","last_active":"2024-09-12T03:02:33.112818+0000","last_peered":"2024-09-12T03:02:33.112818+0000","last_clean":"2024-09-12T03:02:33.112818+0000","last_became_active":"2024-09-12T03:00:58.344389+0000","last_became_peered":"2024-09-12T03:00:58.344389+0000","last_unstale":"2024-09-12T03:02:33.112818+0000","last_undegraded":"2024-09-12T03:02:33.112818+0000","last_fullsized":"2024-09-12T03:02:33.112818+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:08.160383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284373+0000","last_change":"2024-09-12T03:00:56.317068+0000","last_active":"2024-09-12T03:02:04.284373+0000","last_peered":"2024-09-12T03:02:04.284373+0000","last_clean":"2024-09-12T03:02:04.284373+0000","last_became_active":"2024-09-12T03:00:56.316698+0000","last_became_peered":"2024-09-12T03:00:56.316698+0000","last_unstale":"2024-09-12T03:02:04.284373+0000","last_undegraded":"2024-09-12T03:02:04.284373+0000","last_fullsized":"2024-09-12T03:02:04.284373+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:41:19.199418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.109469+0000","last_change":"2024-09-12T03:00:58.347814+0000","last_active":"2024-09-12T03:02:33.109469+0000","last_peered":"2024-09-12T03:02:33.109469+0000","last_clean":"2024-09-12T03:02:33.109469+0000","last_became_active":"2024-09-12T03:00:58.347112+0000","last_became_peered":"2024-09-12T03:00:58.347112+0000","last_unstale":"2024-09-12T03:02:33.109469+0000","last_undegraded":"2024-09-12T03:02:33.109469+0000","last_fullsized":"2024-09-12T03:02:33.109469+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:59.325113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284305+0000","last_change":"2024-09-12T03:00:56.435771+0000","last_active":"2024-09-12T03:02:04.284305+0000","last_peered":"2024-09-12T03:02:04.284305+0000","last_clean":"2024-09-12T03:02:04.284305+0000","last_became_active":"2024-09-12T03:00:56.435448+0000","last_became_peered":"2024-09-12T03:00:56.435448+0000","last_unstale":"2024-09-12T03:02:04.284305+0000","last_undegraded":"2024-09-12T03:02:04.284305+0000","last_fullsized":"2024-09-12T03:02:04.284305+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:33.650338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.117580+0000","last_change":"2024-09-12T03:00:58.341071+0000","last_active":"2024-09-12T03:02:33.117580+0000","last_peered":"2024-09-12T03:02:33.117580+0000","last_clean":"2024-09-12T03:02:33.117580+0000","last_became_active":"2024-09-12T03:00:58.340753+0000","last_became_peered":"2024-09-12T03:00:58.340753+0000","last_unstale":"2024-09-12T03:02:33.117580+0000","last_undegraded":"2024-09-12T03:02:33.117580+0000","last_fullsized":"2024-09-12T03:02:33.117580+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:21.217915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292254+0000","last_change":"2024-09-12T03:00:56.436058+0000","last_active":"2024-09-12T03:02:04.292254+0000","last_peered":"2024-09-12T03:02:04.292254+0000","last_clean":"2024-09-12T03:02:04.292254+0000","last_became_active":"2024-09-12T03:00:56.435518+0000","last_became_peered":"2024-09-12T03:00:56.435518+0000","last_unstale":"2024-09-12T03:02:04.292254+0000","last_undegraded":"2024-09-12T03:02:04.292254+0000","last_fullsized":"2024-09-12T03:02:04.292254+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:32.235097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.110650+0000","last_change":"2024-09-12T03:00:58.348236+0000","last_active":"2024-09-12T03:02:33.110650+0000","last_peered":"2024-09-12T03:02:33.110650+0000","last_clean":"2024-09-12T03:02:33.110650+0000","last_became_active":"2024-09-12T03:00:58.347727+0000","last_became_peered":"2024-09-12T03:00:58.347727+0000","last_unstale":"2024-09-12T03:02:33.110650+0000","last_undegraded":"2024-09-12T03:02:33.110650+0000","last_fullsized":"2024-09-12T03:02:33.110650+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:19.233163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'13","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:56.357641+0000","last_change":"2024-09-12T03:00:56.310537+0000","last_active":"2024-09-12T03:02:56.357641+0000","last_peered":"2024-09-12T03:02:56.357641+0000","last_clean":"2024-09-12T03:02:56.357641+0000","last_became_active":"2024-09-12T03:00:56.310135+0000","last_became_peered":"2024-09-12T03:00:56.310135+0000","last_unstale":"2024-09-12T03:02:56.357641+0000","last_undegraded":"2024-09-12T03:02:56.357641+0000","last_fullsized":"2024-09-12T03:02:56.357641+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:10:55.584339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.121993+0000","last_change":"2024-09-12T03:00:58.358417+0000","last_active":"2024-09-12T03:02:33.121993+0000","last_peered":"2024-09-12T03:02:33.121993+0000","last_clean":"2024-09-12T03:02:33.121993+0000","last_became_active":"2024-09-12T03:00:58.358119+0000","last_became_peered":"2024-09-12T03:00:58.358119+0000","last_unstale":"2024-09-12T03:02:33.121993+0000","last_undegraded":"2024-09-12T03:02:33.121993+0000","last_fullsized":"2024-09-12T03:02:33.121993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:10.897594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289526+0000","last_change":"2024-09-12T03:00:56.747044+0000","last_active":"2024-09-12T03:02:04.289526+0000","last_peered":"2024-09-12T03:02:04.289526+0000","last_clean":"2024-09-12T03:02:04.289526+0000","last_became_active":"2024-09-12T03:00:56.746699+0000","last_became_peered":"2024-09-12T03:00:56.746699+0000","last_unstale":"2024-09-12T03:02:04.289526+0000","last_undegraded":"2024-09-12T03:02:04.289526+0000","last_fullsized":"2024-09-12T03:02:04.289526+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:11.420109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.113058+0000","last_change":"2024-09-12T03:00:58.346854+0000","last_active":"2024-09-12T03:02:33.113058+0000","last_peered":"2024-09-12T03:02:33.113058+0000","last_clean":"2024-09-12T03:02:33.113058+0000","last_became_active":"2024-09-12T03:00:58.346535+0000","last_became_peered":"2024-09-12T03:00:58.346535+0000","last_unstale":"2024-09-12T03:02:33.113058+0000","last_undegraded":"2024-09-12T03:02:33.113058+0000","last_fullsized":"2024-09-12T03:02:33.113058+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:46.637914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332365+0000","last_change":"2024-09-12T03:00:56.435957+0000","last_active":"2024-09-12T03:02:04.332365+0000","last_peered":"2024-09-12T03:02:04.332365+0000","last_clean":"2024-09-12T03:02:04.332365+0000","last_became_active":"2024-09-12T03:00:56.435603+0000","last_became_peered":"2024-09-12T03:00:56.435603+0000","last_unstale":"2024-09-12T03:02:04.332365+0000","last_undegraded":"2024-09-12T03:02:04.332365+0000","last_fullsized":"2024-09-12T03:02:04.332365+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:34.553189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.089792+0000","last_change":"2024-09-12T03:00:58.337784+0000","last_active":"2024-09-12T03:02:33.089792+0000","last_peered":"2024-09-12T03:02:33.089792+0000","last_clean":"2024-09-12T03:02:33.089792+0000","last_became_active":"2024-09-12T03:00:58.337415+0000","last_became_peered":"2024-09-12T03:00:58.337415+0000","last_unstale":"2024-09-12T03:02:33.089792+0000","last_undegraded":"2024-09-12T03:02:33.089792+0000","last_fullsized":"2024-09-12T03:02:33.089792+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:38:15.557067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282354+0000","last_change":"2024-09-12T03:00:56.315356+0000","last_active":"2024-09-12T03:02:04.282354+0000","last_peered":"2024-09-12T03:02:04.282354+0000","last_clean":"2024-09-12T03:02:04.282354+0000","last_became_active":"2024-09-12T03:00:56.314873+0000","last_became_peered":"2024-09-12T03:00:56.314873+0000","last_unstale":"2024-09-12T03:02:04.282354+0000","last_undegraded":"2024-09-12T03:02:04.282354+0000","last_fullsized":"2024-09-12T03:02:04.282354+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:55.710156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.118974+0000","last_change":"2024-09-12T03:00:58.346706+0000","last_active":"2024-09-12T03:02:33.118974+0000","last_peered":"2024-09-12T03:02:33.118974+0000","last_clean":"2024-09-12T03:02:33.118974+0000","last_became_active":"2024-09-12T03:00:58.346363+0000","last_became_peered":"2024-09-12T03:00:58.346363+0000","last_unstale":"2024-09-12T03:02:33.118974+0000","last_undegraded":"2024-09-12T03:02:33.118974+0000","last_fullsized":"2024-09-12T03:02:33.118974+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:13.468639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.299582+0000","last_change":"2024-09-12T03:02:04.299147+0000","last_active":"2024-09-12T03:02:04.299582+0000","last_peered":"2024-09-12T03:02:04.299582+0000","last_clean":"2024-09-12T03:02:04.299582+0000","last_became_active":"2024-09-12T03:02:04.293211+0000","last_became_peered":"2024-09-12T03:02:04.293211+0000","last_unstale":"2024-09-12T03:02:04.299582+0000","last_undegraded":"2024-09-12T03:02:04.299582+0000","last_fullsized":"2024-09-12T03:02:04.299582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:01:39.222476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.118849+0000","last_change":"2024-09-12T03:00:58.356682+0000","last_active":"2024-09-12T03:02:33.118849+0000","last_peered":"2024-09-12T03:02:33.118849+0000","last_clean":"2024-09-12T03:02:33.118849+0000","last_became_active":"2024-09-12T03:00:58.356383+0000","last_became_peered":"2024-09-12T03:00:58.356383+0000","last_unstale":"2024-09-12T03:02:33.118849+0000","last_undegraded":"2024-09-12T03:02:33.118849+0000","last_fullsized":"2024-09-12T03:02:33.118849+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:14:16.428393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292053+0000","last_change":"2024-09-12T03:00:56.314943+0000","last_active":"2024-09-12T03:02:04.292053+0000","last_peered":"2024-09-12T03:02:04.292053+0000","last_clean":"2024-09-12T03:02:04.292053+0000","last_became_active":"2024-09-12T03:00:56.314602+0000","last_became_peered":"2024-09-12T03:00:56.314602+0000","last_unstale":"2024-09-12T03:02:04.292053+0000","last_undegraded":"2024-09-12T03:02:04.292053+0000","last_fullsized":"2024-09-12T03:02:04.292053+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:31.207880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.131538+0000","last_change":"2024-09-12T03:00:58.351774+0000","last_active":"2024-09-12T03:02:33.131538+0000","last_peered":"2024-09-12T03:02:33.131538+0000","last_clean":"2024-09-12T03:02:33.131538+0000","last_became_active":"2024-09-12T03:00:58.351453+0000","last_became_peered":"2024-09-12T03:00:58.351453+0000","last_unstale":"2024-09-12T03:02:33.131538+0000","last_undegraded":"2024-09-12T03:02:33.131538+0000","last_fullsized":"2024-09-12T03:02:33.131538+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:18:24.260110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.130930+0000","last_change":"2024-09-12T03:00:58.346886+0000","last_active":"2024-09-12T03:02:33.130930+0000","last_peered":"2024-09-12T03:02:33.130930+0000","last_clean":"2024-09-12T03:02:33.130930+0000","last_became_active":"2024-09-12T03:00:58.346499+0000","last_became_peered":"2024-09-12T03:00:58.346499+0000","last_unstale":"2024-09-12T03:02:33.130930+0000","last_undegraded":"2024-09-12T03:02:33.130930+0000","last_fullsized":"2024-09-12T03:02:33.130930+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:02:24.045610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292497+0000","last_change":"2024-09-12T03:00:56.317443+0000","last_active":"2024-09-12T03:02:04.292497+0000","last_peered":"2024-09-12T03:02:04.292497+0000","last_clean":"2024-09-12T03:02:04.292497+0000","last_became_active":"2024-09-12T03:00:56.317045+0000","last_became_peered":"2024-09-12T03:00:56.317045+0000","last_unstale":"2024-09-12T03:02:04.292497+0000","last_undegraded":"2024-09-12T03:02:04.292497+0000","last_fullsized":"2024-09-12T03:02:04.292497+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:07.687606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.132782+0000","last_change":"2024-09-12T03:00:58.337257+0000","last_active":"2024-09-12T03:02:33.132782+0000","last_peered":"2024-09-12T03:02:33.132782+0000","last_clean":"2024-09-12T03:02:33.132782+0000","last_became_active":"2024-09-12T03:00:58.336847+0000","last_became_peered":"2024-09-12T03:00:58.336847+0000","last_unstale":"2024-09-12T03:02:33.132782+0000","last_undegraded":"2024-09-12T03:02:33.132782+0000","last_fullsized":"2024-09-12T03:02:33.132782+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:22.846206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292165+0000","last_change":"2024-09-12T03:00:56.316255+0000","last_active":"2024-09-12T03:02:04.292165+0000","last_peered":"2024-09-12T03:02:04.292165+0000","last_clean":"2024-09-12T03:02:04.292165+0000","last_became_active":"2024-09-12T03:00:56.315906+0000","last_became_peered":"2024-09-12T03:00:56.315906+0000","last_unstale":"2024-09-12T03:02:04.292165+0000","last_undegraded":"2024-09-12T03:02:04.292165+0000","last_fullsized":"2024-09-12T03:02:04.292165+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:14.798601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120352+0000","last_change":"2024-09-12T03:00:58.347206+0000","last_active":"2024-09-12T03:02:33.120352+0000","last_peered":"2024-09-12T03:02:33.120352+0000","last_clean":"2024-09-12T03:02:33.120352+0000","last_became_active":"2024-09-12T03:00:58.346923+0000","last_became_peered":"2024-09-12T03:00:58.346923+0000","last_unstale":"2024-09-12T03:02:33.120352+0000","last_undegraded":"2024-09-12T03:02:33.120352+0000","last_fullsized":"2024-09-12T03:02:33.120352+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:41.376211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.283743+0000","last_change":"2024-09-12T03:00:56.435431+0000","last_active":"2024-09-12T03:02:04.283743+0000","last_peered":"2024-09-12T03:02:04.283743+0000","last_clean":"2024-09-12T03:02:04.283743+0000","last_became_active":"2024-09-12T03:00:56.435088+0000","last_became_peered":"2024-09-12T03:00:56.435088+0000","last_unstale":"2024-09-12T03:02:04.283743+0000","last_undegraded":"2024-09-12T03:02:04.283743+0000","last_fullsized":"2024-09-12T03:02:04.283743+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:24.738340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.114461+0000","last_change":"2024-09-12T03:00:58.351737+0000","last_active":"2024-09-12T03:02:33.114461+0000","last_peered":"2024-09-12T03:02:33.114461+0000","last_clean":"2024-09-12T03:02:33.114461+0000","last_became_active":"2024-09-12T03:00:58.351407+0000","last_became_peered":"2024-09-12T03:00:58.351407+0000","last_unstale":"2024-09-12T03:02:33.114461+0000","last_undegraded":"2024-09-12T03:02:33.114461+0000","last_fullsized":"2024-09-12T03:02:33.114461+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:34.206703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.285034+0000","last_change":"2024-09-12T03:00:56.310910+0000","last_active":"2024-09-12T03:02:04.285034+0000","last_peered":"2024-09-12T03:02:04.285034+0000","last_clean":"2024-09-12T03:02:04.285034+0000","last_became_active":"2024-09-12T03:00:56.310657+0000","last_became_peered":"2024-09-12T03:00:56.310657+0000","last_unstale":"2024-09-12T03:02:04.285034+0000","last_undegraded":"2024-09-12T03:02:04.285034+0000","last_fullsized":"2024-09-12T03:02:04.285034+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:06.117215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74598,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":716,"num_read_kb":432,"num_write":178,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":223794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245899,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645351,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":431835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906987,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":1094282,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070513,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":432166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299446,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":431947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528379,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":1023901,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":40,"seq":171798691904,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":880640,"data_stored":431438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":33,"seq":141733920837,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":1023192,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051659,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":432268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280591,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":431632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476820,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":503176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607642,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":501826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":432709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":70595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":71945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":72566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:03:07.588 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:03:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:07 smithi183 ceph-mon[35535]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:03:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:07 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:03:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:07 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:07 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:03:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:07 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:07.710 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:07.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:07 smithi177 ceph-mon[35305]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:03:07.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:07 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:03:07.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:07 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:07.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:07 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:03:07.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:07 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:08.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:08 smithi116 ceph-mon[31410]: pgmap v342: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-09-12T03:03:08.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:08 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:03:08.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:08 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:03:08.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:08 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:03:08.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:08 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:03:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:08 smithi183 ceph-mon[35535]: pgmap v342: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-09-12T03:03:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:08 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:03:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:08 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:03:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:08 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:03:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:08 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:03:08.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:08 smithi177 ceph-mon[35305]: pgmap v342: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-09-12T03:03:08.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:08 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:03:08.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:08 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:03:08.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:08 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:03:08.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:08 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:03:08.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.968+0000 7fa2c4dfd640 1 -- 172.21.15.116:0/2620435897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2c0100e20 msgr2=0x7fa2c0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:08.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.968+0000 7fa2c4dfd640 1 --2- 172.21.15.116:0/2620435897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2c0100e20 0x7fa2c0101200 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0009ca0 tx=0x7fa2b002a950 comp rx=0 tx=0).stop 2024-09-12T03:03:08.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.968+0000 7fa2c4dfd640 1 -- 172.21.15.116:0/2620435897 shutdown_connections 2024-09-12T03:03:08.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.968+0000 7fa2c4dfd640 1 --2- 172.21.15.116:0/2620435897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2c0102ef0 0x7fa2c010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:08.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.968+0000 7fa2c4dfd640 1 --2- 172.21.15.116:0/2620435897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2c0102570 0x7fa2c01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:08.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.968+0000 7fa2c4dfd640 1 --2- 172.21.15.116:0/2620435897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2c0100e20 0x7fa2c0101200 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:08.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.968+0000 7fa2c4dfd640 1 -- 172.21.15.116:0/2620435897 >> 172.21.15.116:0/2620435897 conn(0x7fa2c00fc7d0 msgr2=0x7fa2c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:08.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.968+0000 7fa2c4dfd640 1 -- 172.21.15.116:0/2620435897 shutdown_connections 2024-09-12T03:03:08.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.968+0000 7fa2c4dfd640 1 -- 172.21.15.116:0/2620435897 wait complete. 2024-09-12T03:03:08.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.969+0000 7fa2c4dfd640 1 Processor -- start 2024-09-12T03:03:08.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.969+0000 7fa2c4dfd640 1 -- start start 2024-09-12T03:03:08.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.969+0000 7fa2c4dfd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2c0100e20 0x7fa2c0198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:08.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.969+0000 7fa2c4dfd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2c0102570 0x7fa2c0198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:08.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.969+0000 7fa2c4dfd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2c0102ef0 0x7fa2c019cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:08.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.969+0000 7fa2c4dfd640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c0110fc0 con 0x7fa2c0100e20 2024-09-12T03:03:08.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.969+0000 7fa2c4dfd640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c0110e40 con 0x7fa2c0102ef0 2024-09-12T03:03:08.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.969+0000 7fa2c4dfd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c0111140 con 0x7fa2c0102570 2024-09-12T03:03:08.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.969+0000 7fa2be59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2c0100e20 0x7fa2c0198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:08.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.969+0000 7fa2be59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2c0100e20 0x7fa2c0198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39270/0 (socket says 172.21.15.116:39270) 2024-09-12T03:03:08.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.969+0000 7fa2be59c640 1 -- 172.21.15.116:0/133289802 learned_addr learned my addr 172.21.15.116:0/133289802 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:08.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.969+0000 7fa2bdd9b640 1 --2- 172.21.15.116:0/133289802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2c0102570 0x7fa2c0198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:08.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.969+0000 7fa2be59c640 1 -- 172.21.15.116:0/133289802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2c0102570 msgr2=0x7fa2c0198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:08.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.969+0000 7fa2be59c640 1 --2- 172.21.15.116:0/133289802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2c0102570 0x7fa2c0198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:08.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.969+0000 7fa2be59c640 1 -- 172.21.15.116:0/133289802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2c0102ef0 msgr2=0x7fa2c019cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:08.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.969+0000 7fa2bed9d640 1 --2- 172.21.15.116:0/133289802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2c0102ef0 0x7fa2c019cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:08.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.969+0000 7fa2be59c640 1 --2- 172.21.15.116:0/133289802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2c0102ef0 0x7fa2c019cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:08.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.969+0000 7fa2be59c640 1 -- 172.21.15.116:0/133289802 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c019d5c0 con 0x7fa2c0100e20 2024-09-12T03:03:08.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.969+0000 7fa2bed9d640 1 --2- 172.21.15.116:0/133289802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2c0102ef0 0x7fa2c019cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:03:08.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.970+0000 7fa2be59c640 1 --2- 172.21.15.116:0/133289802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2c0100e20 0x7fa2c0198610 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0005cc0 tx=0x7fa2b002f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:08.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.970+0000 7fa2b77fe640 1 -- 172.21.15.116:0/133289802 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0034040 con 0x7fa2c0100e20 2024-09-12T03:03:08.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.970+0000 7fa2b77fe640 1 -- 172.21.15.116:0/133289802 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2b0030920 con 0x7fa2c0100e20 2024-09-12T03:03:08.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.970+0000 7fa2c4dfd640 1 -- 172.21.15.116:0/133289802 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c019d850 con 0x7fa2c0100e20 2024-09-12T03:03:08.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.970+0000 7fa2b77fe640 1 -- 172.21.15.116:0/133289802 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0004830 con 0x7fa2c0100e20 2024-09-12T03:03:08.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.970+0000 7fa2c4dfd640 1 -- 172.21.15.116:0/133289802 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c01a5190 con 0x7fa2c0100e20 2024-09-12T03:03:08.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.972+0000 7fa2b77fe640 1 -- 172.21.15.116:0/133289802 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fa2b0037020 con 0x7fa2c0100e20 2024-09-12T03:03:08.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.972+0000 7fa2c71d6740 1 Processor -- start 2024-09-12T03:03:08.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.973+0000 7fa2c71d6740 1 -- start start 2024-09-12T03:03:08.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.973+0000 7fa2c71d6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6873f54a0 0x55a6873f5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:08.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.973+0000 7fa2c71d6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6873f5dc0 0x55a687400140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:08.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.973+0000 7fa2c71d6740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a687400680 0x55a687402a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:08.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.973+0000 7fa2c71d6740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a68735b2e0 con 0x55a687400680 2024-09-12T03:03:08.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.973+0000 7fa2c71d6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a687302ce0 con 0x55a6873f5dc0 2024-09-12T03:03:08.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.973+0000 7fa2c71d6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a6873572a0 con 0x55a6873f54a0 2024-09-12T03:03:08.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.973+0000 7fa2bdd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a687400680 0x55a687402a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:08.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.973+0000 7fa2bdd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a687400680 0x55a687402a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39278/0 (socket says 172.21.15.116:39278) 2024-09-12T03:03:08.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.973+0000 7fa2bdd9b640 1 -- 172.21.15.116:0/3835804065 learned_addr learned my addr 172.21.15.116:0/3835804065 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:08.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.973+0000 7fa2b77fe640 1 --2- 172.21.15.116:0/133289802 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa2ac0a2c90 0x7fa2ac0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:08.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.973+0000 7fa2bed9d640 1 --2- 172.21.15.116:0/3835804065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6873f54a0 0x55a6873f5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:08.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.973+0000 7fa2be59c640 1 --2- 172.21.15.116:0/3835804065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6873f5dc0 0x55a687400140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:08.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.973+0000 7fa2bed9d640 1 --2- 172.21.15.116:0/133289802 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa2ac0a2c90 0x7fa2ac0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:08.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.973+0000 7fa2bdd9b640 1 -- 172.21.15.116:0/3835804065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6873f54a0 msgr2=0x55a6873f5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:08.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.973+0000 7fa2bdd9b640 1 --2- 172.21.15.116:0/3835804065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6873f54a0 0x55a6873f5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:08.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.973+0000 7fa2bdd9b640 1 -- 172.21.15.116:0/3835804065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6873f5dc0 msgr2=0x55a687400140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:08.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.973+0000 7fa2bdd9b640 1 --2- 172.21.15.116:0/3835804065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6873f5dc0 0x55a687400140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:08.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.973+0000 7fa2bdd9b640 1 -- 172.21.15.116:0/3835804065 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6872e7440 con 0x55a687400680 2024-09-12T03:03:08.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.973+0000 7fa2bed9d640 1 --2- 172.21.15.116:0/3835804065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6873f54a0 0x55a6873f5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:03:08.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.973+0000 7fa2b77fe640 1 -- 172.21.15.116:0/133289802 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa2b00f00f0 con 0x7fa2c0100e20 2024-09-12T03:03:08.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.974+0000 7fa2bdd9b640 1 --2- 172.21.15.116:0/3835804065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a687400680 0x55a687402a70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800bc90 tx=0x7fa2a800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c4d87dd9af0a154 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:08.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.974+0000 7fa2b4ff9640 1 -- 172.21.15.116:0/3835804065 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8015070 con 0x55a687400680 2024-09-12T03:03:08.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.974+0000 7fa2bed9d640 1 --2- 172.21.15.116:0/133289802 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa2ac0a2c90 0x7fa2ac0a5150 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800ca50 tx=0x7fa2b80046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:08.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.974+0000 7fa2b4ff9640 1 -- 172.21.15.116:0/3835804065 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2a801a940 con 0x55a687400680 2024-09-12T03:03:08.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.974+0000 7fa2b4ff9640 1 -- 172.21.15.116:0/3835804065 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a80118c0 con 0x55a687400680 2024-09-12T03:03:08.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.975+0000 7fa2c71d6740 1 -- 172.21.15.116:0/3835804065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a687400680 msgr2=0x55a687402a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:08.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.975+0000 7fa2c71d6740 1 --2- 172.21.15.116:0/3835804065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a687400680 0x55a687402a70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800bc90 tx=0x7fa2a800f730 comp rx=0 tx=0).stop 2024-09-12T03:03:08.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.975+0000 7fa2c71d6740 1 -- 172.21.15.116:0/3835804065 shutdown_connections 2024-09-12T03:03:08.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.975+0000 7fa2c71d6740 1 --2- 172.21.15.116:0/3835804065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a687400680 0x55a687402a70 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:08.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.975+0000 7fa2c71d6740 1 --2- 172.21.15.116:0/3835804065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6873f5dc0 0x55a687400140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:08.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.975+0000 7fa2c71d6740 1 --2- 172.21.15.116:0/3835804065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6873f54a0 0x55a6873f5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:08.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.975+0000 7fa2c71d6740 1 -- 172.21.15.116:0/3835804065 >> 172.21.15.116:0/3835804065 conn(0x55a6873f1a30 msgr2=0x55a6873f1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:08.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.975+0000 7fa2c71d6740 1 -- 172.21.15.116:0/3835804065 shutdown_connections 2024-09-12T03:03:08.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.975+0000 7fa2c71d6740 1 -- 172.21.15.116:0/3835804065 wait complete. 2024-09-12T03:03:08.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.975+0000 7fa2c71d6740 1 -- start start 2024-09-12T03:03:08.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.975+0000 7fa2c71d6740 1 Processor -- start 2024-09-12T03:03:08.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.976+0000 7fa2c71d6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6873f54a0 0x55a6873fff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:08.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.976+0000 7fa2c71d6740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a6873f5dc0 0x55a6873fd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:08.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.976+0000 7fa2c71d6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a687400680 0x55a6873fd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:08.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.976+0000 7fa2c71d6740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a687403250 con 0x55a6873f5dc0 2024-09-12T03:03:08.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.976+0000 7fa2c71d6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a687316050 con 0x55a6873f54a0 2024-09-12T03:03:08.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.976+0000 7fa2c71d6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a6874033d0 con 0x55a687400680 2024-09-12T03:03:08.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.976+0000 7fa2c71d6740 10 client.0 _ll_register_callbacks cb 0x55a687356160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:08.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.976+0000 7fa2bdd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a6873f5dc0 0x55a6873fd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:08.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.976+0000 7fa2bdd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a6873f5dc0 0x55a6873fd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39284/0 (socket says 172.21.15.116:39284) 2024-09-12T03:03:08.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.976+0000 7fa2bdd9b640 1 -- 172.21.15.116:0/1789605137 learned_addr learned my addr 172.21.15.116:0/1789605137 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:08.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.976+0000 7fa2be59c640 1 --2- 172.21.15.116:0/1789605137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6873f54a0 0x55a6873fff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:08.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.976+0000 7fa2bdd9b640 1 -- 172.21.15.116:0/1789605137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a687400680 msgr2=0x55a6873fd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:08.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.976+0000 7fa2bed9d640 1 --2- 172.21.15.116:0/1789605137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a687400680 0x55a6873fd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:08.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.976+0000 7fa2bdd9b640 1 --2- 172.21.15.116:0/1789605137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a687400680 0x55a6873fd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:08.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.976+0000 7fa2bdd9b640 1 -- 172.21.15.116:0/1789605137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6873f54a0 msgr2=0x55a6873fff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:08.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.976+0000 7fa2bdd9b640 1 --2- 172.21.15.116:0/1789605137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6873f54a0 0x55a6873fff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:08.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.976+0000 7fa2bdd9b640 1 -- 172.21.15.116:0/1789605137 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6870cc3c0 con 0x55a6873f5dc0 2024-09-12T03:03:08.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.976+0000 7fa2bed9d640 1 --2- 172.21.15.116:0/1789605137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a687400680 0x55a6873fd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:03:08.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.976+0000 7fa2bdd9b640 1 --2- 172.21.15.116:0/1789605137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a6873f5dc0 0x55a6873fd220 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8011520 tx=0x7fa2a801a460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b43c9f03223b2b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:08.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.976+0000 7fa2a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:03:08.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.976+0000 7fa2a5ffb640 1 -- 172.21.15.116:0/1789605137 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8028060 con 0x55a6873f5dc0 2024-09-12T03:03:08.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.976+0000 7fa2a5ffb640 1 -- 172.21.15.116:0/1789605137 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa2a801aaf0 con 0x55a6873f5dc0 2024-09-12T03:03:08.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.976+0000 7fa2c71d6740 1 -- 172.21.15.116:0/1789605137 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2a801a940 con 0x55a6873f5dc0 2024-09-12T03:03:08.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.976+0000 7fa2a5ffb640 1 -- 172.21.15.116:0/1789605137 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa2a8015070 con 0x55a6873f5dc0 2024-09-12T03:03:08.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.976+0000 7fa2c71d6740 10 client.15276 fetch_fsmap learned FSMap version 25 2024-09-12T03:03:08.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.976+0000 7fa2c71d6740 1 -- 172.21.15.116:0/1789605137 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a68740ec20 con 0x55a6873f5dc0 2024-09-12T03:03:08.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.977+0000 7fa2a5ffb640 1 -- 172.21.15.116:0/1789605137 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2a801a940 con 0x55a6873f5dc0 2024-09-12T03:03:08.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.977+0000 7fa2a5ffb640 1 -- 172.21.15.116:0/1789605137 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa2a8004b00 con 0x55a6873f5dc0 2024-09-12T03:03:08.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.977+0000 7fa2c71d6740 10 client.15276 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:03:08.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.977+0000 7fa2c71d6740 10 client.15276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:03:08.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.977+0000 7fa2c71d6740 20 client.15276 populate_metadata read hostname 'smithi116' 2024-09-12T03:03:08.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.977+0000 7fa2c71d6740 1 --2- 172.21.15.116:0/1789605137 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a6874100e0 0x55a6874124d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:08.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.977+0000 7fa2c71d6740 4 client.15276 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:03:08.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.977+0000 7fa2c71d6740 1 -- 172.21.15.116:0/1789605137 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a687368850 con 0x55a6874100e0 2024-09-12T03:03:08.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.977+0000 7fa2be59c640 1 --2- 172.21.15.116:0/1789605137 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a6874100e0 0x55a6874124d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:08.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.978+0000 7fa2be59c640 1 --2- 172.21.15.116:0/1789605137 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a6874100e0 0x55a6874124d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f04bfdc458ea2e4 server_cookie=16d1c7b8f1d7bf89 in_seq=0 out_seq=0 2024-09-12T03:03:08.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.978+0000 7fa2a5ffb640 10 client.15276 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:08.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.978+0000 7fa2a5ffb640 1 -- 172.21.15.116:0/1789605137 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a687368850 con 0x55a6874100e0 2024-09-12T03:03:08.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.978+0000 7fa2a5ffb640 10 client.15276 handle_command_reply: tid=0 2024-09-12T03:03:08.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.978+0000 7fa2c71d6740 1 client.15276 shutdown 2024-09-12T03:03:08.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.978+0000 7fa2c71d6740 1 -- 172.21.15.116:0/1789605137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a6873f5dc0 msgr2=0x55a6873fd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:08.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.978+0000 7fa2c71d6740 1 --2- 172.21.15.116:0/1789605137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a6873f5dc0 0x55a6873fd220 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8011520 tx=0x7fa2a801a460 comp rx=0 tx=0).stop 2024-09-12T03:03:08.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.978+0000 7fa2c71d6740 1 -- 172.21.15.116:0/1789605137 shutdown_connections 2024-09-12T03:03:08.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.978+0000 7fa2c71d6740 1 --2- 172.21.15.116:0/1789605137 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a6874100e0 0x55a6874124d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:08.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.978+0000 7fa2c71d6740 1 --2- 172.21.15.116:0/1789605137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a687400680 0x55a6873fd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:08.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.978+0000 7fa2c71d6740 1 --2- 172.21.15.116:0/1789605137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a6873f5dc0 0x55a6873fd220 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:08.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.978+0000 7fa2c71d6740 1 --2- 172.21.15.116:0/1789605137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6873f54a0 0x55a6873fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:08.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.978+0000 7fa2a5ffb640 0 client.15276 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:09.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.978+0000 7fa2c71d6740 1 -- 172.21.15.116:0/1789605137 >> 172.21.15.116:0/1789605137 conn(0x55a6873f1a30 msgr2=0x55a6873f7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:09.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.978+0000 7fa2c71d6740 1 -- 172.21.15.116:0/1789605137 shutdown_connections 2024-09-12T03:03:09.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.978+0000 7fa2c71d6740 1 -- 172.21.15.116:0/1789605137 wait complete. 2024-09-12T03:03:09.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.979+0000 7fa2c71d6740 20 client.15276 trim_cache size 0 max 16384 2024-09-12T03:03:09.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.996+0000 7fa2c71d6740 1 Processor -- start 2024-09-12T03:03:09.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.996+0000 7fa2c71d6740 1 -- start start 2024-09-12T03:03:09.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.996+0000 7fa2c71d6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6873f54a0 0x55a68740c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:09.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.996+0000 7fa2c71d6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6873f5dc0 0x55a6873f8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:09.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.996+0000 7fa2c71d6740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a687400680 0x55a6873ff190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:09.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.996+0000 7fa2c71d6740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c01afc90 con 0x55a687400680 2024-09-12T03:03:09.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.996+0000 7fa2c71d6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c01ad560 con 0x55a6873f5dc0 2024-09-12T03:03:09.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.996+0000 7fa2c71d6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c01ab3d0 con 0x55a6873f54a0 2024-09-12T03:03:09.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.996+0000 7fa2bed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a687400680 0x55a6873ff190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:09.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.996+0000 7fa2bed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a687400680 0x55a6873ff190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39286/0 (socket says 172.21.15.116:39286) 2024-09-12T03:03:09.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.996+0000 7fa2bed9d640 1 -- 172.21.15.116:0/80910768 learned_addr learned my addr 172.21.15.116:0/80910768 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:09.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.996+0000 7fa2bdd9b640 1 --2- 172.21.15.116:0/80910768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6873f5dc0 0x55a6873f8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:09.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.996+0000 7fa2be59c640 1 --2- 172.21.15.116:0/80910768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6873f54a0 0x55a68740c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:09.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.996+0000 7fa2bed9d640 1 -- 172.21.15.116:0/80910768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6873f54a0 msgr2=0x55a68740c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:09.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.996+0000 7fa2bed9d640 1 --2- 172.21.15.116:0/80910768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6873f54a0 0x55a68740c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:09.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.996+0000 7fa2bed9d640 1 -- 172.21.15.116:0/80910768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6873f5dc0 msgr2=0x55a6873f8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:09.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.996+0000 7fa2bed9d640 1 --2- 172.21.15.116:0/80910768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6873f5dc0 0x55a6873f8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:09.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.996+0000 7fa2bed9d640 1 -- 172.21.15.116:0/80910768 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6874002e0 con 0x55a687400680 2024-09-12T03:03:09.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.996+0000 7fa2bdd9b640 1 --2- 172.21.15.116:0/80910768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6873f5dc0 0x55a6873f8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:03:09.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.996+0000 7fa2be59c640 1 --2- 172.21.15.116:0/80910768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6873f54a0 0x55a68740c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:03:09.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.997+0000 7fa2bed9d640 1 --2- 172.21.15.116:0/80910768 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a687400680 0x55a6873ff190 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800faf0 tx=0x7fa2b8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a17f2494be908c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:09.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.997+0000 7fa2a7fff640 1 -- 172.21.15.116:0/80910768 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8026070 con 0x55a687400680 2024-09-12T03:03:09.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.997+0000 7fa2a7fff640 1 -- 172.21.15.116:0/80910768 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2b8013b80 con 0x55a687400680 2024-09-12T03:03:09.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.997+0000 7fa2a7fff640 1 -- 172.21.15.116:0/80910768 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8019070 con 0x55a687400680 2024-09-12T03:03:09.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.997+0000 7fa2c71d6740 1 -- 172.21.15.116:0/80910768 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a687400680 msgr2=0x55a6873ff190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:09.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.997+0000 7fa2c71d6740 1 --2- 172.21.15.116:0/80910768 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a687400680 0x55a6873ff190 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800faf0 tx=0x7fa2b8012000 comp rx=0 tx=0).stop 2024-09-12T03:03:09.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.997+0000 7fa2c71d6740 1 -- 172.21.15.116:0/80910768 shutdown_connections 2024-09-12T03:03:09.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.997+0000 7fa2c71d6740 1 --2- 172.21.15.116:0/80910768 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a687400680 0x55a6873ff190 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:09.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.997+0000 7fa2c71d6740 1 --2- 172.21.15.116:0/80910768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6873f5dc0 0x55a6873f8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:09.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.997+0000 7fa2c71d6740 1 --2- 172.21.15.116:0/80910768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6873f54a0 0x55a68740c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:09.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.997+0000 7fa2c71d6740 1 -- 172.21.15.116:0/80910768 >> 172.21.15.116:0/80910768 conn(0x55a6873f1a30 msgr2=0x55a6873fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:09.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.997+0000 7fa2c71d6740 1 -- 172.21.15.116:0/80910768 shutdown_connections 2024-09-12T03:03:09.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.998+0000 7fa2c71d6740 1 -- 172.21.15.116:0/80910768 wait complete. 2024-09-12T03:03:09.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.998+0000 7fa2c71d6740 1 -- start start 2024-09-12T03:03:09.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.998+0000 7fa2c71d6740 1 Processor -- start 2024-09-12T03:03:09.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.998+0000 7fa2c71d6740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a6873f54a0 0x55a687404770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:09.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.998+0000 7fa2c71d6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6873f5dc0 0x55a687404cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:09.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.998+0000 7fa2c71d6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a687400680 0x55a687479aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:09.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.998+0000 7fa2c71d6740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c01aba20 con 0x55a6873f54a0 2024-09-12T03:03:09.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.998+0000 7fa2c71d6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c01ace90 con 0x55a6873f5dc0 2024-09-12T03:03:09.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.998+0000 7fa2c71d6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c01ac0f0 con 0x55a687400680 2024-09-12T03:03:09.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.998+0000 7fa2c71d6740 10 client.0 _ll_register_callbacks cb 0x7fa2a800bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:09.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.998+0000 7fa2be59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a6873f54a0 0x55a687404770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:09.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.998+0000 7fa2be59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a6873f54a0 0x55a687404770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39296/0 (socket says 172.21.15.116:39296) 2024-09-12T03:03:09.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.998+0000 7fa2be59c640 1 -- 172.21.15.116:0/1742660884 learned_addr learned my addr 172.21.15.116:0/1742660884 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:09.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.999+0000 7fa2bed9d640 1 --2- 172.21.15.116:0/1742660884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a687400680 0x55a687479aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:09.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.999+0000 7fa2bdd9b640 1 --2- 172.21.15.116:0/1742660884 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6873f5dc0 0x55a687404cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:09.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.999+0000 7fa2be59c640 1 -- 172.21.15.116:0/1742660884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a687400680 msgr2=0x55a687479aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:09.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.999+0000 7fa2be59c640 1 --2- 172.21.15.116:0/1742660884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a687400680 0x55a687479aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:09.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.999+0000 7fa2be59c640 1 -- 172.21.15.116:0/1742660884 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6873f5dc0 msgr2=0x55a687404cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:09.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.999+0000 7fa2be59c640 1 --2- 172.21.15.116:0/1742660884 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6873f5dc0 0x55a687404cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:09.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.999+0000 7fa2be59c640 1 -- 172.21.15.116:0/1742660884 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a687479fe0 con 0x55a6873f54a0 2024-09-12T03:03:09.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.999+0000 7fa2bdd9b640 1 --2- 172.21.15.116:0/1742660884 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6873f5dc0 0x55a687404cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:03:09.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.999+0000 7fa2be59c640 1 --2- 172.21.15.116:0/1742660884 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a6873f54a0 0x55a687404770 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00049d0 tx=0x7fa2b0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9da413da885bdf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:09.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.999+0000 7fa2a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:03:09.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.999+0000 7fa2a5ffb640 1 -- 172.21.15.116:0/1742660884 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b00c0ce0 con 0x55a6873f54a0 2024-09-12T03:03:09.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.999+0000 7fa2a5ffb640 1 -- 172.21.15.116:0/1742660884 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa2b0004040 con 0x55a6873f54a0 2024-09-12T03:03:09.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.999+0000 7fa2c71d6740 1 -- 172.21.15.116:0/1742660884 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2b8013b80 con 0x55a6873f54a0 2024-09-12T03:03:09.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.999+0000 7fa2a5ffb640 1 -- 172.21.15.116:0/1742660884 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa2b00c55a0 con 0x55a6873f54a0 2024-09-12T03:03:09.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.999+0000 7fa2c71d6740 10 client.15282 fetch_fsmap learned FSMap version 25 2024-09-12T03:03:09.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.999+0000 7fa2c71d6740 1 -- 172.21.15.116:0/1742660884 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a68747b160 con 0x55a6873f54a0 2024-09-12T03:03:09.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.999+0000 7fa2a5ffb640 1 -- 172.21.15.116:0/1742660884 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2b8013b80 con 0x55a6873f54a0 2024-09-12T03:03:09.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:08.999+0000 7fa2a5ffb640 1 -- 172.21.15.116:0/1742660884 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa2b004d030 con 0x55a6873f54a0 2024-09-12T03:03:09.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.000+0000 7fa2c71d6740 10 client.15282 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:03:09.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.000+0000 7fa2c71d6740 10 client.15282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:03:09.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.000+0000 7fa2c71d6740 20 client.15282 populate_metadata read hostname 'smithi116' 2024-09-12T03:03:09.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.000+0000 7fa2c71d6740 1 --2- 172.21.15.116:0/1742660884 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a68747b8d0 0x55a68747dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:09.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.000+0000 7fa2c71d6740 4 client.15282 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:03:09.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.000+0000 7fa2c71d6740 1 -- 172.21.15.116:0/1742660884 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a687362d60 con 0x55a68747b8d0 2024-09-12T03:03:09.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.000+0000 7fa2bdd9b640 1 --2- 172.21.15.116:0/1742660884 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a68747b8d0 0x55a68747dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:09.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.000+0000 7fa2a5ffb640 10 client.15282 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:09.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.000+0000 7fa2bdd9b640 1 --2- 172.21.15.116:0/1742660884 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a68747b8d0 0x55a68747dce0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfe029144bd4c325 server_cookie=70db82c1735e8c71 in_seq=0 out_seq=0 2024-09-12T03:03:09.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.001+0000 7fa2a5ffb640 1 -- 172.21.15.116:0/1742660884 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a687362d60 con 0x55a68747b8d0 2024-09-12T03:03:09.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.001+0000 7fa2a5ffb640 10 client.15282 handle_command_reply: tid=0 2024-09-12T03:03:09.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.001+0000 7fa2c71d6740 1 client.15282 shutdown 2024-09-12T03:03:09.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.001+0000 7fa2c71d6740 1 -- 172.21.15.116:0/1742660884 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a6873f54a0 msgr2=0x55a687404770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:09.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.001+0000 7fa2c71d6740 1 --2- 172.21.15.116:0/1742660884 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a6873f54a0 0x55a687404770 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00049d0 tx=0x7fa2b0003f10 comp rx=0 tx=0).stop 2024-09-12T03:03:09.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.001+0000 7fa2c71d6740 1 -- 172.21.15.116:0/1742660884 shutdown_connections 2024-09-12T03:03:09.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.001+0000 7fa2c71d6740 1 --2- 172.21.15.116:0/1742660884 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a68747b8d0 0x55a68747dce0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:09.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.001+0000 7fa2c71d6740 1 --2- 172.21.15.116:0/1742660884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a687400680 0x55a687479aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:09.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.001+0000 7fa2c71d6740 1 --2- 172.21.15.116:0/1742660884 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a6873f5dc0 0x55a687404cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:09.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.001+0000 7fa2c71d6740 1 --2- 172.21.15.116:0/1742660884 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a6873f54a0 0x55a687404770 secure :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00049d0 tx=0x7fa2b0003f10 comp rx=0 tx=0).stop 2024-09-12T03:03:09.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.001+0000 7fa2c71d6740 1 -- 172.21.15.116:0/1742660884 >> 172.21.15.116:0/1742660884 conn(0x55a6873f1a30 msgr2=0x55a6873f0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:09.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.001+0000 7fa2a5ffb640 0 client.15282 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:09.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.001+0000 7fa2c71d6740 1 -- 172.21.15.116:0/1742660884 shutdown_connections 2024-09-12T03:03:09.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.001+0000 7fa2c71d6740 1 -- 172.21.15.116:0/1742660884 wait complete. 2024-09-12T03:03:09.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.001+0000 7fa2c71d6740 20 client.15282 trim_cache size 0 max 16384 2024-09-12T03:03:09.011 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:03:09.011 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:03:09.011 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": { 2024-09-12T03:03:09.011 INFO:teuthology.orchestra.run.smithi116.stdout: "b6271d4f-9b9d-48d5-bebe-81c6ec2676bd": { 2024-09-12T03:03:09.011 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/", 2024-09-12T03:03:09.012 INFO:teuthology.orchestra.run.smithi116.stdout: "tag": "b6271d4f-9b9d-48d5-bebe-81c6ec2676bd", 2024-09-12T03:03:09.012 INFO:teuthology.orchestra.run.smithi116.stdout: "options": "recursive,force" 2024-09-12T03:03:09.012 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:03:09.012 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:03:09.012 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:03:09.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.002+0000 7fa2c4dfd640 1 -- 172.21.15.116:0/133289802 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa2ac0a2c90 msgr2=0x7fa2ac0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:09.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.002+0000 7fa2c4dfd640 1 --2- 172.21.15.116:0/133289802 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa2ac0a2c90 0x7fa2ac0a5150 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800ca50 tx=0x7fa2b80046c0 comp rx=0 tx=0).stop 2024-09-12T03:03:09.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.002+0000 7fa2c4dfd640 1 -- 172.21.15.116:0/133289802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2c0100e20 msgr2=0x7fa2c0198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:09.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.002+0000 7fa2c4dfd640 1 --2- 172.21.15.116:0/133289802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2c0100e20 0x7fa2c0198610 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0005cc0 tx=0x7fa2b002f650 comp rx=0 tx=0).stop 2024-09-12T03:03:09.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.002+0000 7fa2c4dfd640 1 -- 172.21.15.116:0/133289802 shutdown_connections 2024-09-12T03:03:09.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.002+0000 7fa2c4dfd640 1 --2- 172.21.15.116:0/133289802 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa2ac0a2c90 0x7fa2ac0a5150 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:09.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.002+0000 7fa2c4dfd640 1 --2- 172.21.15.116:0/133289802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2c0102ef0 0x7fa2c019cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:09.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.002+0000 7fa2c4dfd640 1 --2- 172.21.15.116:0/133289802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2c0102570 0x7fa2c0198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:09.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.002+0000 7fa2c4dfd640 1 --2- 172.21.15.116:0/133289802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2c0100e20 0x7fa2c0198610 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:09.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.002+0000 7fa2c4dfd640 1 -- 172.21.15.116:0/133289802 >> 172.21.15.116:0/133289802 conn(0x7fa2c00fc7d0 msgr2=0x7fa2c006a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:09.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.002+0000 7fa2c4dfd640 1 -- 172.21.15.116:0/133289802 shutdown_connections 2024-09-12T03:03:09.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:09.002+0000 7fa2c4dfd640 1 -- 172.21.15.116:0/133289802 wait complete. 2024-09-12T03:03:09.014 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:09.736 INFO:tasks.cephfs.filesystem:scrub status for tag:b6271d4f-9b9d-48d5-bebe-81c6ec2676bd - {'path': '/', 'tag': 'b6271d4f-9b9d-48d5-bebe-81c6ec2676bd', 'options': 'recursive,force'} 2024-09-12T03:03:10.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.200+0000 7fe6d0c53640 1 -- 172.21.15.116:0/1698047075 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6cc106f10 msgr2=0x7fe6cc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:10.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.200+0000 7fe6d0c53640 1 --2- 172.21.15.116:0/1698047075 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6cc106f10 0x7fe6cc10ea90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4009ca0 tx=0x7fe6c402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:03:10.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.200+0000 7fe6d0c53640 1 -- 172.21.15.116:0/1698047075 shutdown_connections 2024-09-12T03:03:10.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.200+0000 7fe6d0c53640 1 --2- 172.21.15.116:0/1698047075 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6cc106f10 0x7fe6cc10ea90 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:10.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.200+0000 7fe6d0c53640 1 --2- 172.21.15.116:0/1698047075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6cc102790 0x7fe6cc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:10.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.200+0000 7fe6d0c53640 1 --2- 172.21.15.116:0/1698047075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6cc101de0 0x7fe6cc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:10.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.200+0000 7fe6d0c53640 1 -- 172.21.15.116:0/1698047075 >> 172.21.15.116:0/1698047075 conn(0x7fe6cc0fd8c0 msgr2=0x7fe6cc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:10.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.200+0000 7fe6d0c53640 1 -- 172.21.15.116:0/1698047075 shutdown_connections 2024-09-12T03:03:10.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.200+0000 7fe6d0c53640 1 -- 172.21.15.116:0/1698047075 wait complete. 2024-09-12T03:03:10.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.201+0000 7fe6d0c53640 1 Processor -- start 2024-09-12T03:03:10.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.201+0000 7fe6d0c53640 1 -- start start 2024-09-12T03:03:10.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.201+0000 7fe6d0c53640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6cc101de0 0x7fe6cc1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:10.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.201+0000 7fe6d0c53640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6cc102790 0x7fe6cc198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:10.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.201+0000 7fe6d0c53640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6cc106f10 0x7fe6cc19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:10.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.201+0000 7fe6d0c53640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc112010 con 0x7fe6cc101de0 2024-09-12T03:03:10.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.201+0000 7fe6d0c53640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc111e90 con 0x7fe6cc102790 2024-09-12T03:03:10.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.201+0000 7fe6d0c53640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc112190 con 0x7fe6cc106f10 2024-09-12T03:03:10.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.201+0000 7fe6ca59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6cc101de0 0x7fe6cc1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:10.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.201+0000 7fe6ca59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6cc101de0 0x7fe6cc1985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39304/0 (socket says 172.21.15.116:39304) 2024-09-12T03:03:10.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.201+0000 7fe6ca59c640 1 -- 172.21.15.116:0/3643201944 learned_addr learned my addr 172.21.15.116:0/3643201944 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:10.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.201+0000 7fe6cad9d640 1 --2- 172.21.15.116:0/3643201944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6cc106f10 0x7fe6cc19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:10.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.201+0000 7fe6c9d9b640 1 --2- 172.21.15.116:0/3643201944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6cc102790 0x7fe6cc198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:10.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.201+0000 7fe6ca59c640 1 -- 172.21.15.116:0/3643201944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6cc106f10 msgr2=0x7fe6cc19cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:10.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.201+0000 7fe6ca59c640 1 --2- 172.21.15.116:0/3643201944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6cc106f10 0x7fe6cc19cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:10.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.201+0000 7fe6ca59c640 1 -- 172.21.15.116:0/3643201944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6cc102790 msgr2=0x7fe6cc198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:10.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.201+0000 7fe6ca59c640 1 --2- 172.21.15.116:0/3643201944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6cc102790 0x7fe6cc198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:10.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.201+0000 7fe6ca59c640 1 -- 172.21.15.116:0/3643201944 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6cc19d5a0 con 0x7fe6cc101de0 2024-09-12T03:03:10.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.201+0000 7fe6cad9d640 1 --2- 172.21.15.116:0/3643201944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6cc106f10 0x7fe6cc19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:03:10.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.202+0000 7fe6ca59c640 1 --2- 172.21.15.116:0/3643201944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6cc101de0 0x7fe6cc1985f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe6b400b4a0 tx=0x7fe6b400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:10.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.202+0000 7fe6c37fe640 1 -- 172.21.15.116:0/3643201944 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4014070 con 0x7fe6cc101de0 2024-09-12T03:03:10.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.202+0000 7fe6c37fe640 1 -- 172.21.15.116:0/3643201944 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6b4010af0 con 0x7fe6cc101de0 2024-09-12T03:03:10.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.202+0000 7fe6d0c53640 1 -- 172.21.15.116:0/3643201944 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6cc19d890 con 0x7fe6cc101de0 2024-09-12T03:03:10.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.202+0000 7fe6c37fe640 1 -- 172.21.15.116:0/3643201944 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4017d10 con 0x7fe6cc101de0 2024-09-12T03:03:10.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.202+0000 7fe6d0c53640 1 -- 172.21.15.116:0/3643201944 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6cc1a5170 con 0x7fe6cc101de0 2024-09-12T03:03:10.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.203+0000 7fe6d0c53640 1 -- 172.21.15.116:0/3643201944 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6cc102240 con 0x7fe6cc101de0 2024-09-12T03:03:10.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.204+0000 7fe6c37fe640 1 -- 172.21.15.116:0/3643201944 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fe6b4017070 con 0x7fe6cc101de0 2024-09-12T03:03:10.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.205+0000 7fe6c37fe640 1 --2- 172.21.15.116:0/3643201944 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe6b80a2c90 0x7fe6b80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:10.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.205+0000 7fe6c37fe640 1 -- 172.21.15.116:0/3643201944 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe6b40ce4a0 con 0x7fe6cc101de0 2024-09-12T03:03:10.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.205+0000 7fe6c9d9b640 1 --2- 172.21.15.116:0/3643201944 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe6b80a2c90 0x7fe6b80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:10.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.205+0000 7fe6c9d9b640 1 --2- 172.21.15.116:0/3643201944 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe6b80a2c90 0x7fe6b80a5150 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc0ffe70 tx=0x7fe6bc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:10.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.209+0000 7fe6c37fe640 1 -- 172.21.15.116:0/3643201944 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe6b409b3e0 con 0x7fe6cc101de0 2024-09-12T03:03:10.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.385+0000 7fe6d0c53640 1 -- 172.21.15.116:0/3643201944 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe6cc062630 con 0x7fe6b80a2c90 2024-09-12T03:03:10.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.392+0000 7fe6c37fe640 1 -- 172.21.15.116:0/3643201944 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258980 (secure 0 0 0) 0x7fe6cc062630 con 0x7fe6b80a2c90 2024-09-12T03:03:10.395 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:03:10.396 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-09-12T03:03:10.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.395+0000 7fe6d0c53640 1 -- 172.21.15.116:0/3643201944 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe6b80a2c90 msgr2=0x7fe6b80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:10.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.395+0000 7fe6d0c53640 1 --2- 172.21.15.116:0/3643201944 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe6b80a2c90 0x7fe6b80a5150 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc0ffe70 tx=0x7fe6bc008040 comp rx=0 tx=0).stop 2024-09-12T03:03:10.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.395+0000 7fe6d0c53640 1 -- 172.21.15.116:0/3643201944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6cc101de0 msgr2=0x7fe6cc1985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:10.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.395+0000 7fe6d0c53640 1 --2- 172.21.15.116:0/3643201944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6cc101de0 0x7fe6cc1985f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe6b400b4a0 tx=0x7fe6b400b960 comp rx=0 tx=0).stop 2024-09-12T03:03:10.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.395+0000 7fe6d0c53640 1 -- 172.21.15.116:0/3643201944 shutdown_connections 2024-09-12T03:03:10.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.395+0000 7fe6d0c53640 1 --2- 172.21.15.116:0/3643201944 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe6b80a2c90 0x7fe6b80a5150 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:10.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.395+0000 7fe6d0c53640 1 --2- 172.21.15.116:0/3643201944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6cc106f10 0x7fe6cc19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:10.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.395+0000 7fe6d0c53640 1 --2- 172.21.15.116:0/3643201944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6cc102790 0x7fe6cc198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:10.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.395+0000 7fe6d0c53640 1 --2- 172.21.15.116:0/3643201944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6cc101de0 0x7fe6cc1985f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:10.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.395+0000 7fe6d0c53640 1 -- 172.21.15.116:0/3643201944 >> 172.21.15.116:0/3643201944 conn(0x7fe6cc0fd8c0 msgr2=0x7fe6cc0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:10.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.395+0000 7fe6d0c53640 1 -- 172.21.15.116:0/3643201944 shutdown_connections 2024-09-12T03:03:10.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:10.395+0000 7fe6d0c53640 1 -- 172.21.15.116:0/3643201944 wait complete. 2024-09-12T03:03:10.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:10 smithi116 ceph-mon[31410]: pgmap v343: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:03:10.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:10 smithi183 ceph-mon[35535]: pgmap v343: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:03:10.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:10 smithi177 ceph-mon[35305]: pgmap v343: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:03:11.105 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-09-12T03:03:09.138296+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665200,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":864,"num_read_kb":558,"num_write":361,"num_write_kb":3069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180875,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":789,"ondisk_log_size":789,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368404,"kb_used_data":10836,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573420,"statfs":{"total":1151940427776,"available":1151563182080,"internally_reserved":0,"allocated":11096064,"data_stored":7170372,"data_compressed":107490,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8679,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":43,"num_write":8,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002869"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.133534+0000","last_change":"2024-09-12T03:00:58.348563+0000","last_active":"2024-09-12T03:02:33.133534+0000","last_peered":"2024-09-12T03:02:33.133534+0000","last_clean":"2024-09-12T03:02:33.133534+0000","last_became_active":"2024-09-12T03:00:58.347940+0000","last_became_peered":"2024-09-12T03:00:58.347940+0000","last_unstale":"2024-09-12T03:02:33.133534+0000","last_undegraded":"2024-09-12T03:02:33.133534+0000","last_fullsized":"2024-09-12T03:02:33.133534+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:43.489238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.119339+0000","last_change":"2024-09-12T03:00:58.351969+0000","last_active":"2024-09-12T03:02:33.119339+0000","last_peered":"2024-09-12T03:02:33.119339+0000","last_clean":"2024-09-12T03:02:33.119339+0000","last_became_active":"2024-09-12T03:00:58.351679+0000","last_became_peered":"2024-09-12T03:00:58.351679+0000","last_unstale":"2024-09-12T03:02:33.119339+0000","last_undegraded":"2024-09-12T03:02:33.119339+0000","last_fullsized":"2024-09-12T03:02:33.119339+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:27:01.174890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120214+0000","last_change":"2024-09-12T03:00:58.346045+0000","last_active":"2024-09-12T03:02:33.120214+0000","last_peered":"2024-09-12T03:02:33.120214+0000","last_clean":"2024-09-12T03:02:33.120214+0000","last_became_active":"2024-09-12T03:00:58.345719+0000","last_became_peered":"2024-09-12T03:00:58.345719+0000","last_unstale":"2024-09-12T03:02:33.120214+0000","last_undegraded":"2024-09-12T03:02:33.120214+0000","last_fullsized":"2024-09-12T03:02:33.120214+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:12.312650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.111069+0000","last_change":"2024-09-12T03:00:58.348165+0000","last_active":"2024-09-12T03:02:33.111069+0000","last_peered":"2024-09-12T03:02:33.111069+0000","last_clean":"2024-09-12T03:02:33.111069+0000","last_became_active":"2024-09-12T03:00:58.347734+0000","last_became_peered":"2024-09-12T03:00:58.347734+0000","last_unstale":"2024-09-12T03:02:33.111069+0000","last_undegraded":"2024-09-12T03:02:33.111069+0000","last_fullsized":"2024-09-12T03:02:33.111069+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:05.945900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.108519+0000","last_change":"2024-09-12T03:00:58.339316+0000","last_active":"2024-09-12T03:02:33.108519+0000","last_peered":"2024-09-12T03:02:33.108519+0000","last_clean":"2024-09-12T03:02:33.108519+0000","last_became_active":"2024-09-12T03:00:58.338964+0000","last_became_peered":"2024-09-12T03:00:58.338964+0000","last_unstale":"2024-09-12T03:02:33.108519+0000","last_undegraded":"2024-09-12T03:02:33.108519+0000","last_fullsized":"2024-09-12T03:02:33.108519+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:50.721276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.110921+0000","last_change":"2024-09-12T03:00:58.346838+0000","last_active":"2024-09-12T03:02:33.110921+0000","last_peered":"2024-09-12T03:02:33.110921+0000","last_clean":"2024-09-12T03:02:33.110921+0000","last_became_active":"2024-09-12T03:00:58.346486+0000","last_became_peered":"2024-09-12T03:00:58.346486+0000","last_unstale":"2024-09-12T03:02:33.110921+0000","last_undegraded":"2024-09-12T03:02:33.110921+0000","last_fullsized":"2024-09-12T03:02:33.110921+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:28.687727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.122919+0000","last_change":"2024-09-12T03:00:58.346712+0000","last_active":"2024-09-12T03:02:33.122919+0000","last_peered":"2024-09-12T03:02:33.122919+0000","last_clean":"2024-09-12T03:02:33.122919+0000","last_became_active":"2024-09-12T03:00:58.346397+0000","last_became_peered":"2024-09-12T03:00:58.346397+0000","last_unstale":"2024-09-12T03:02:33.122919+0000","last_undegraded":"2024-09-12T03:02:33.122919+0000","last_fullsized":"2024-09-12T03:02:33.122919+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:48:00.624733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.118885+0000","last_change":"2024-09-12T03:00:58.356166+0000","last_active":"2024-09-12T03:02:33.118885+0000","last_peered":"2024-09-12T03:02:33.118885+0000","last_clean":"2024-09-12T03:02:33.118885+0000","last_became_active":"2024-09-12T03:00:58.354944+0000","last_became_peered":"2024-09-12T03:00:58.354944+0000","last_unstale":"2024-09-12T03:02:33.118885+0000","last_undegraded":"2024-09-12T03:02:33.118885+0000","last_fullsized":"2024-09-12T03:02:33.118885+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:15.840149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.133064+0000","last_change":"2024-09-12T03:00:58.340182+0000","last_active":"2024-09-12T03:02:33.133064+0000","last_peered":"2024-09-12T03:02:33.133064+0000","last_clean":"2024-09-12T03:02:33.133064+0000","last_became_active":"2024-09-12T03:00:58.339576+0000","last_became_peered":"2024-09-12T03:00:58.339576+0000","last_unstale":"2024-09-12T03:02:33.133064+0000","last_undegraded":"2024-09-12T03:02:33.133064+0000","last_fullsized":"2024-09-12T03:02:33.133064+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:16.534844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.117254+0000","last_change":"2024-09-12T03:00:58.351218+0000","last_active":"2024-09-12T03:02:33.117254+0000","last_peered":"2024-09-12T03:02:33.117254+0000","last_clean":"2024-09-12T03:02:33.117254+0000","last_became_active":"2024-09-12T03:00:58.350871+0000","last_became_peered":"2024-09-12T03:00:58.350871+0000","last_unstale":"2024-09-12T03:02:33.117254+0000","last_undegraded":"2024-09-12T03:02:33.117254+0000","last_fullsized":"2024-09-12T03:02:33.117254+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:38.367683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.130892+0000","last_change":"2024-09-12T03:00:58.351894+0000","last_active":"2024-09-12T03:02:33.130892+0000","last_peered":"2024-09-12T03:02:33.130892+0000","last_clean":"2024-09-12T03:02:33.130892+0000","last_became_active":"2024-09-12T03:00:58.351445+0000","last_became_peered":"2024-09-12T03:00:58.351445+0000","last_unstale":"2024-09-12T03:02:33.130892+0000","last_undegraded":"2024-09-12T03:02:33.130892+0000","last_fullsized":"2024-09-12T03:02:33.130892+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:42.093871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.107323+0000","last_change":"2024-09-12T03:00:58.353460+0000","last_active":"2024-09-12T03:02:33.107323+0000","last_peered":"2024-09-12T03:02:33.107323+0000","last_clean":"2024-09-12T03:02:33.107323+0000","last_became_active":"2024-09-12T03:00:58.353111+0000","last_became_peered":"2024-09-12T03:00:58.353111+0000","last_unstale":"2024-09-12T03:02:33.107323+0000","last_undegraded":"2024-09-12T03:02:33.107323+0000","last_fullsized":"2024-09-12T03:02:33.107323+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:15.916765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.111860+0000","last_change":"2024-09-12T03:00:58.341080+0000","last_active":"2024-09-12T03:02:33.111860+0000","last_peered":"2024-09-12T03:02:33.111860+0000","last_clean":"2024-09-12T03:02:33.111860+0000","last_became_active":"2024-09-12T03:00:58.340782+0000","last_became_peered":"2024-09-12T03:00:58.340782+0000","last_unstale":"2024-09-12T03:02:33.111860+0000","last_undegraded":"2024-09-12T03:02:33.111860+0000","last_fullsized":"2024-09-12T03:02:33.111860+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:36.569700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.106506+0000","last_change":"2024-09-12T03:00:58.346791+0000","last_active":"2024-09-12T03:02:33.106506+0000","last_peered":"2024-09-12T03:02:33.106506+0000","last_clean":"2024-09-12T03:02:33.106506+0000","last_became_active":"2024-09-12T03:00:58.346500+0000","last_became_peered":"2024-09-12T03:00:58.346500+0000","last_unstale":"2024-09-12T03:02:33.106506+0000","last_undegraded":"2024-09-12T03:02:33.106506+0000","last_fullsized":"2024-09-12T03:02:33.106506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:32.017589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.112821+0000","last_change":"2024-09-12T03:00:58.349522+0000","last_active":"2024-09-12T03:02:33.112821+0000","last_peered":"2024-09-12T03:02:33.112821+0000","last_clean":"2024-09-12T03:02:33.112821+0000","last_became_active":"2024-09-12T03:00:58.349246+0000","last_became_peered":"2024-09-12T03:00:58.349246+0000","last_unstale":"2024-09-12T03:02:33.112821+0000","last_undegraded":"2024-09-12T03:02:33.112821+0000","last_fullsized":"2024-09-12T03:02:33.112821+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:39.253919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.134146+0000","last_change":"2024-09-12T03:00:58.349909+0000","last_active":"2024-09-12T03:02:33.134146+0000","last_peered":"2024-09-12T03:02:33.134146+0000","last_clean":"2024-09-12T03:02:33.134146+0000","last_became_active":"2024-09-12T03:00:58.349590+0000","last_became_peered":"2024-09-12T03:00:58.349590+0000","last_unstale":"2024-09-12T03:02:33.134146+0000","last_undegraded":"2024-09-12T03:02:33.134146+0000","last_fullsized":"2024-09-12T03:02:33.134146+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:04.540709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.133565+0000","last_change":"2024-09-12T03:00:58.348284+0000","last_active":"2024-09-12T03:02:33.133565+0000","last_peered":"2024-09-12T03:02:33.133565+0000","last_clean":"2024-09-12T03:02:33.133565+0000","last_became_active":"2024-09-12T03:00:58.347465+0000","last_became_peered":"2024-09-12T03:00:58.347465+0000","last_unstale":"2024-09-12T03:02:33.133565+0000","last_undegraded":"2024-09-12T03:02:33.133565+0000","last_fullsized":"2024-09-12T03:02:33.133565+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:13.191899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.122657+0000","last_change":"2024-09-12T03:00:58.337740+0000","last_active":"2024-09-12T03:02:33.122657+0000","last_peered":"2024-09-12T03:02:33.122657+0000","last_clean":"2024-09-12T03:02:33.122657+0000","last_became_active":"2024-09-12T03:00:58.337393+0000","last_became_peered":"2024-09-12T03:00:58.337393+0000","last_unstale":"2024-09-12T03:02:33.122657+0000","last_undegraded":"2024-09-12T03:02:33.122657+0000","last_fullsized":"2024-09-12T03:02:33.122657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:28:56.339167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.111666+0000","last_change":"2024-09-12T03:00:58.345779+0000","last_active":"2024-09-12T03:02:33.111666+0000","last_peered":"2024-09-12T03:02:33.111666+0000","last_clean":"2024-09-12T03:02:33.111666+0000","last_became_active":"2024-09-12T03:00:58.345210+0000","last_became_peered":"2024-09-12T03:00:58.345210+0000","last_unstale":"2024-09-12T03:02:33.111666+0000","last_undegraded":"2024-09-12T03:02:33.111666+0000","last_fullsized":"2024-09-12T03:02:33.111666+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:06:58.271185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120653+0000","last_change":"2024-09-12T03:02:04.294381+0000","last_active":"2024-09-12T03:02:33.120653+0000","last_peered":"2024-09-12T03:02:33.120653+0000","last_clean":"2024-09-12T03:02:33.120653+0000","last_became_active":"2024-09-12T03:02:04.293795+0000","last_became_peered":"2024-09-12T03:02:04.293795+0000","last_unstale":"2024-09-12T03:02:33.120653+0000","last_undegraded":"2024-09-12T03:02:33.120653+0000","last_fullsized":"2024-09-12T03:02:33.120653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:06:58.271007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.124292+0000","last_change":"2024-09-12T03:00:58.349233+0000","last_active":"2024-09-12T03:02:33.124292+0000","last_peered":"2024-09-12T03:02:33.124292+0000","last_clean":"2024-09-12T03:02:33.124292+0000","last_became_active":"2024-09-12T03:00:58.348958+0000","last_became_peered":"2024-09-12T03:00:58.348958+0000","last_unstale":"2024-09-12T03:02:33.124292+0000","last_undegraded":"2024-09-12T03:02:33.124292+0000","last_fullsized":"2024-09-12T03:02:33.124292+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:32:10.271817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.134703+0000","last_change":"2024-09-12T03:00:58.358024+0000","last_active":"2024-09-12T03:02:33.134703+0000","last_peered":"2024-09-12T03:02:33.134703+0000","last_clean":"2024-09-12T03:02:33.134703+0000","last_became_active":"2024-09-12T03:00:58.357434+0000","last_became_peered":"2024-09-12T03:00:58.357434+0000","last_unstale":"2024-09-12T03:02:33.134703+0000","last_undegraded":"2024-09-12T03:02:33.134703+0000","last_fullsized":"2024-09-12T03:02:33.134703+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:01:20.119733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120095+0000","last_change":"2024-09-12T03:00:58.355063+0000","last_active":"2024-09-12T03:02:33.120095+0000","last_peered":"2024-09-12T03:02:33.120095+0000","last_clean":"2024-09-12T03:02:33.120095+0000","last_became_active":"2024-09-12T03:00:58.354760+0000","last_became_peered":"2024-09-12T03:00:58.354760+0000","last_unstale":"2024-09-12T03:02:33.120095+0000","last_undegraded":"2024-09-12T03:02:33.120095+0000","last_fullsized":"2024-09-12T03:02:33.120095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:13.464413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.112255+0000","last_change":"2024-09-12T03:00:58.341916+0000","last_active":"2024-09-12T03:02:33.112255+0000","last_peered":"2024-09-12T03:02:33.112255+0000","last_clean":"2024-09-12T03:02:33.112255+0000","last_became_active":"2024-09-12T03:00:58.340815+0000","last_became_peered":"2024-09-12T03:00:58.340815+0000","last_unstale":"2024-09-12T03:02:33.112255+0000","last_undegraded":"2024-09-12T03:02:33.112255+0000","last_fullsized":"2024-09-12T03:02:33.112255+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:13.777947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.117197+0000","last_change":"2024-09-12T03:00:58.357765+0000","last_active":"2024-09-12T03:02:33.117197+0000","last_peered":"2024-09-12T03:02:33.117197+0000","last_clean":"2024-09-12T03:02:33.117197+0000","last_became_active":"2024-09-12T03:00:58.357480+0000","last_became_peered":"2024-09-12T03:00:58.357480+0000","last_unstale":"2024-09-12T03:02:33.117197+0000","last_undegraded":"2024-09-12T03:02:33.117197+0000","last_fullsized":"2024-09-12T03:02:33.117197+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:26.884381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.108940+0000","last_change":"2024-09-12T03:00:58.349504+0000","last_active":"2024-09-12T03:02:33.108940+0000","last_peered":"2024-09-12T03:02:33.108940+0000","last_clean":"2024-09-12T03:02:33.108940+0000","last_became_active":"2024-09-12T03:00:58.349136+0000","last_became_peered":"2024-09-12T03:00:58.349136+0000","last_unstale":"2024-09-12T03:02:33.108940+0000","last_undegraded":"2024-09-12T03:02:33.108940+0000","last_fullsized":"2024-09-12T03:02:33.108940+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:09:55.229516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.123028+0000","last_change":"2024-09-12T03:00:58.341902+0000","last_active":"2024-09-12T03:02:33.123028+0000","last_peered":"2024-09-12T03:02:33.123028+0000","last_clean":"2024-09-12T03:02:33.123028+0000","last_became_active":"2024-09-12T03:00:58.341592+0000","last_became_peered":"2024-09-12T03:00:58.341592+0000","last_unstale":"2024-09-12T03:02:33.123028+0000","last_undegraded":"2024-09-12T03:02:33.123028+0000","last_fullsized":"2024-09-12T03:02:33.123028+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:13.852511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.114521+0000","last_change":"2024-09-12T03:00:58.348012+0000","last_active":"2024-09-12T03:02:33.114521+0000","last_peered":"2024-09-12T03:02:33.114521+0000","last_clean":"2024-09-12T03:02:33.114521+0000","last_became_active":"2024-09-12T03:00:58.347338+0000","last_became_peered":"2024-09-12T03:00:58.347338+0000","last_unstale":"2024-09-12T03:02:33.114521+0000","last_undegraded":"2024-09-12T03:02:33.114521+0000","last_fullsized":"2024-09-12T03:02:33.114521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:41.478032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.113228+0000","last_change":"2024-09-12T03:00:58.345833+0000","last_active":"2024-09-12T03:02:33.113228+0000","last_peered":"2024-09-12T03:02:33.113228+0000","last_clean":"2024-09-12T03:02:33.113228+0000","last_became_active":"2024-09-12T03:00:58.345482+0000","last_became_peered":"2024-09-12T03:00:58.345482+0000","last_unstale":"2024-09-12T03:02:33.113228+0000","last_undegraded":"2024-09-12T03:02:33.113228+0000","last_fullsized":"2024-09-12T03:02:33.113228+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:32.037412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.112949+0000","last_change":"2024-09-12T03:00:58.352290+0000","last_active":"2024-09-12T03:02:33.112949+0000","last_peered":"2024-09-12T03:02:33.112949+0000","last_clean":"2024-09-12T03:02:33.112949+0000","last_became_active":"2024-09-12T03:00:58.351918+0000","last_became_peered":"2024-09-12T03:00:58.351918+0000","last_unstale":"2024-09-12T03:02:33.112949+0000","last_undegraded":"2024-09-12T03:02:33.112949+0000","last_fullsized":"2024-09-12T03:02:33.112949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:25.941080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.116096+0000","last_change":"2024-09-12T03:00:58.348021+0000","last_active":"2024-09-12T03:02:33.116096+0000","last_peered":"2024-09-12T03:02:33.116096+0000","last_clean":"2024-09-12T03:02:33.116096+0000","last_became_active":"2024-09-12T03:00:58.347634+0000","last_became_peered":"2024-09-12T03:00:58.347634+0000","last_unstale":"2024-09-12T03:02:33.116096+0000","last_undegraded":"2024-09-12T03:02:33.116096+0000","last_fullsized":"2024-09-12T03:02:33.116096+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:47.933067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.123517+0000","last_change":"2024-09-12T03:00:58.340891+0000","last_active":"2024-09-12T03:02:33.123517+0000","last_peered":"2024-09-12T03:02:33.123517+0000","last_clean":"2024-09-12T03:02:33.123517+0000","last_became_active":"2024-09-12T03:00:58.340504+0000","last_became_peered":"2024-09-12T03:00:58.340504+0000","last_unstale":"2024-09-12T03:02:33.123517+0000","last_undegraded":"2024-09-12T03:02:33.123517+0000","last_fullsized":"2024-09-12T03:02:33.123517+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:32.117635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.108754+0000","last_change":"2024-09-12T03:00:58.339343+0000","last_active":"2024-09-12T03:02:33.108754+0000","last_peered":"2024-09-12T03:02:33.108754+0000","last_clean":"2024-09-12T03:02:33.108754+0000","last_became_active":"2024-09-12T03:00:58.339001+0000","last_became_peered":"2024-09-12T03:00:58.339001+0000","last_unstale":"2024-09-12T03:02:33.108754+0000","last_undegraded":"2024-09-12T03:02:33.108754+0000","last_fullsized":"2024-09-12T03:02:33.108754+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:02.043454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":145,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:03:02.571564+0000","last_change":"2024-09-12T03:00:56.315550+0000","last_active":"2024-09-12T03:03:02.571564+0000","last_peered":"2024-09-12T03:03:02.571564+0000","last_clean":"2024-09-12T03:03:02.571564+0000","last_became_active":"2024-09-12T03:00:56.315225+0000","last_became_peered":"2024-09-12T03:00:56.315225+0000","last_unstale":"2024-09-12T03:03:02.571564+0000","last_undegraded":"2024-09-12T03:03:02.571564+0000","last_fullsized":"2024-09-12T03:03:02.571564+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:46.070895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.116508+0000","last_change":"2024-09-12T03:02:04.290498+0000","last_active":"2024-09-12T03:02:33.116508+0000","last_peered":"2024-09-12T03:02:33.116508+0000","last_clean":"2024-09-12T03:02:33.116508+0000","last_became_active":"2024-09-12T03:02:04.290162+0000","last_became_peered":"2024-09-12T03:02:04.290162+0000","last_unstale":"2024-09-12T03:02:33.116508+0000","last_undegraded":"2024-09-12T03:02:33.116508+0000","last_fullsized":"2024-09-12T03:02:33.116508+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:29.777155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282566+0000","last_change":"2024-09-12T03:00:56.316385+0000","last_active":"2024-09-12T03:02:04.282566+0000","last_peered":"2024-09-12T03:02:04.282566+0000","last_clean":"2024-09-12T03:02:04.282566+0000","last_became_active":"2024-09-12T03:00:56.316064+0000","last_became_peered":"2024-09-12T03:00:56.316064+0000","last_unstale":"2024-09-12T03:02:04.282566+0000","last_undegraded":"2024-09-12T03:02:04.282566+0000","last_fullsized":"2024-09-12T03:02:04.282566+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:12:43.323654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332424+0000","last_change":"2024-09-12T03:00:56.313065+0000","last_active":"2024-09-12T03:02:04.332424+0000","last_peered":"2024-09-12T03:02:04.332424+0000","last_clean":"2024-09-12T03:02:04.332424+0000","last_became_active":"2024-09-12T03:00:56.312716+0000","last_became_peered":"2024-09-12T03:00:56.312716+0000","last_unstale":"2024-09-12T03:02:04.332424+0000","last_undegraded":"2024-09-12T03:02:04.332424+0000","last_fullsized":"2024-09-12T03:02:04.332424+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:51.089089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120152+0000","last_change":"2024-09-12T03:00:58.346542+0000","last_active":"2024-09-12T03:02:33.120152+0000","last_peered":"2024-09-12T03:02:33.120152+0000","last_clean":"2024-09-12T03:02:33.120152+0000","last_became_active":"2024-09-12T03:00:58.346260+0000","last_became_peered":"2024-09-12T03:00:58.346260+0000","last_unstale":"2024-09-12T03:02:33.120152+0000","last_undegraded":"2024-09-12T03:02:33.120152+0000","last_fullsized":"2024-09-12T03:02:33.120152+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:37.034969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289182+0000","last_change":"2024-09-12T03:00:56.317308+0000","last_active":"2024-09-12T03:02:04.289182+0000","last_peered":"2024-09-12T03:02:04.289182+0000","last_clean":"2024-09-12T03:02:04.289182+0000","last_became_active":"2024-09-12T03:00:56.316935+0000","last_became_peered":"2024-09-12T03:00:56.316935+0000","last_unstale":"2024-09-12T03:02:04.289182+0000","last_undegraded":"2024-09-12T03:02:04.289182+0000","last_fullsized":"2024-09-12T03:02:04.289182+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:06.324489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.118033+0000","last_change":"2024-09-12T03:00:58.341565+0000","last_active":"2024-09-12T03:02:33.118033+0000","last_peered":"2024-09-12T03:02:33.118033+0000","last_clean":"2024-09-12T03:02:33.118033+0000","last_became_active":"2024-09-12T03:00:58.341285+0000","last_became_peered":"2024-09-12T03:00:58.341285+0000","last_unstale":"2024-09-12T03:02:33.118033+0000","last_undegraded":"2024-09-12T03:02:33.118033+0000","last_fullsized":"2024-09-12T03:02:33.118033+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:59.156700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":143,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:03:04.574157+0000","last_change":"2024-09-12T03:00:56.314201+0000","last_active":"2024-09-12T03:03:04.574157+0000","last_peered":"2024-09-12T03:03:04.574157+0000","last_clean":"2024-09-12T03:03:04.574157+0000","last_became_active":"2024-09-12T03:00:56.313880+0000","last_became_peered":"2024-09-12T03:00:56.313880+0000","last_unstale":"2024-09-12T03:03:04.574157+0000","last_undegraded":"2024-09-12T03:03:04.574157+0000","last_fullsized":"2024-09-12T03:03:04.574157+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:57:12.952630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.132547+0000","last_change":"2024-09-12T03:00:58.345274+0000","last_active":"2024-09-12T03:02:33.132547+0000","last_peered":"2024-09-12T03:02:33.132547+0000","last_clean":"2024-09-12T03:02:33.132547+0000","last_became_active":"2024-09-12T03:00:58.344937+0000","last_became_peered":"2024-09-12T03:00:58.344937+0000","last_unstale":"2024-09-12T03:02:33.132547+0000","last_undegraded":"2024-09-12T03:02:33.132547+0000","last_fullsized":"2024-09-12T03:02:33.132547+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:17.288909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284614+0000","last_change":"2024-09-12T03:00:56.314709+0000","last_active":"2024-09-12T03:02:04.284614+0000","last_peered":"2024-09-12T03:02:04.284614+0000","last_clean":"2024-09-12T03:02:04.284614+0000","last_became_active":"2024-09-12T03:00:56.314349+0000","last_became_peered":"2024-09-12T03:00:56.314349+0000","last_unstale":"2024-09-12T03:02:04.284614+0000","last_undegraded":"2024-09-12T03:02:04.284614+0000","last_fullsized":"2024-09-12T03:02:04.284614+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:18:35.371661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.113713+0000","last_change":"2024-09-12T03:00:58.346055+0000","last_active":"2024-09-12T03:02:33.113713+0000","last_peered":"2024-09-12T03:02:33.113713+0000","last_clean":"2024-09-12T03:02:33.113713+0000","last_became_active":"2024-09-12T03:00:58.345701+0000","last_became_peered":"2024-09-12T03:00:58.345701+0000","last_unstale":"2024-09-12T03:02:33.113713+0000","last_undegraded":"2024-09-12T03:02:33.113713+0000","last_fullsized":"2024-09-12T03:02:33.113713+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:33.977245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.287450+0000","last_change":"2024-09-12T03:00:56.435962+0000","last_active":"2024-09-12T03:02:04.287450+0000","last_peered":"2024-09-12T03:02:04.287450+0000","last_clean":"2024-09-12T03:02:04.287450+0000","last_became_active":"2024-09-12T03:00:56.435515+0000","last_became_peered":"2024-09-12T03:00:56.435515+0000","last_unstale":"2024-09-12T03:02:04.287450+0000","last_undegraded":"2024-09-12T03:02:04.287450+0000","last_fullsized":"2024-09-12T03:02:04.287450+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:32.953436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.135045+0000","last_change":"2024-09-12T03:00:58.344239+0000","last_active":"2024-09-12T03:02:33.135045+0000","last_peered":"2024-09-12T03:02:33.135045+0000","last_clean":"2024-09-12T03:02:33.135045+0000","last_became_active":"2024-09-12T03:00:58.343876+0000","last_became_peered":"2024-09-12T03:00:58.343876+0000","last_unstale":"2024-09-12T03:02:33.135045+0000","last_undegraded":"2024-09-12T03:02:33.135045+0000","last_fullsized":"2024-09-12T03:02:33.135045+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:49.401295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'36","reported_seq":308,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:03:05.576075+0000","last_change":"2024-09-12T03:00:56.317708+0000","last_active":"2024-09-12T03:03:05.576075+0000","last_peered":"2024-09-12T03:03:05.576075+0000","last_clean":"2024-09-12T03:03:05.576075+0000","last_became_active":"2024-09-12T03:00:56.317317+0000","last_became_peered":"2024-09-12T03:00:56.317317+0000","last_unstale":"2024-09-12T03:03:05.576075+0000","last_undegraded":"2024-09-12T03:03:05.576075+0000","last_fullsized":"2024-09-12T03:03:05.576075+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:47.788186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70595,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":250,"num_read_kb":194,"num_write":37,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.123764+0000","last_change":"2024-09-12T03:00:58.343602+0000","last_active":"2024-09-12T03:02:33.123764+0000","last_peered":"2024-09-12T03:02:33.123764+0000","last_clean":"2024-09-12T03:02:33.123764+0000","last_became_active":"2024-09-12T03:00:58.343301+0000","last_became_peered":"2024-09-12T03:00:58.343301+0000","last_unstale":"2024-09-12T03:02:33.123764+0000","last_undegraded":"2024-09-12T03:02:33.123764+0000","last_fullsized":"2024-09-12T03:02:33.123764+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:04.056118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.290204+0000","last_change":"2024-09-12T03:00:56.317874+0000","last_active":"2024-09-12T03:02:04.290204+0000","last_peered":"2024-09-12T03:02:04.290204+0000","last_clean":"2024-09-12T03:02:04.290204+0000","last_became_active":"2024-09-12T03:00:56.317566+0000","last_became_peered":"2024-09-12T03:00:56.317566+0000","last_unstale":"2024-09-12T03:02:04.290204+0000","last_undegraded":"2024-09-12T03:02:04.290204+0000","last_fullsized":"2024-09-12T03:02:04.290204+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:54.203730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.121582+0000","last_change":"2024-09-12T03:00:58.351539+0000","last_active":"2024-09-12T03:02:33.121582+0000","last_peered":"2024-09-12T03:02:33.121582+0000","last_clean":"2024-09-12T03:02:33.121582+0000","last_became_active":"2024-09-12T03:00:58.351249+0000","last_became_peered":"2024-09-12T03:00:58.351249+0000","last_unstale":"2024-09-12T03:02:33.121582+0000","last_undegraded":"2024-09-12T03:02:33.121582+0000","last_fullsized":"2024-09-12T03:02:33.121582+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:40.498867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.287068+0000","last_change":"2024-09-12T03:00:56.314489+0000","last_active":"2024-09-12T03:02:04.287068+0000","last_peered":"2024-09-12T03:02:04.287068+0000","last_clean":"2024-09-12T03:02:04.287068+0000","last_became_active":"2024-09-12T03:00:56.314142+0000","last_became_peered":"2024-09-12T03:00:56.314142+0000","last_unstale":"2024-09-12T03:02:04.287068+0000","last_undegraded":"2024-09-12T03:02:04.287068+0000","last_fullsized":"2024-09-12T03:02:04.287068+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:21.541713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.123797+0000","last_change":"2024-09-12T03:00:58.348752+0000","last_active":"2024-09-12T03:02:33.123797+0000","last_peered":"2024-09-12T03:02:33.123797+0000","last_clean":"2024-09-12T03:02:33.123797+0000","last_became_active":"2024-09-12T03:00:58.348391+0000","last_became_peered":"2024-09-12T03:00:58.348391+0000","last_unstale":"2024-09-12T03:02:33.123797+0000","last_undegraded":"2024-09-12T03:02:33.123797+0000","last_fullsized":"2024-09-12T03:02:33.123797+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:33:15.626306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'4","reported_seq":146,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:03:03.572627+0000","last_change":"2024-09-12T03:00:56.311702+0000","last_active":"2024-09-12T03:03:03.572627+0000","last_peered":"2024-09-12T03:03:03.572627+0000","last_clean":"2024-09-12T03:03:03.572627+0000","last_became_active":"2024-09-12T03:00:56.311353+0000","last_became_peered":"2024-09-12T03:00:56.311353+0000","last_unstale":"2024-09-12T03:03:03.572627+0000","last_undegraded":"2024-09-12T03:03:03.572627+0000","last_fullsized":"2024-09-12T03:03:03.572627+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:01.004858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.123076+0000","last_change":"2024-09-12T03:00:58.351973+0000","last_active":"2024-09-12T03:02:33.123076+0000","last_peered":"2024-09-12T03:02:33.123076+0000","last_clean":"2024-09-12T03:02:33.123076+0000","last_became_active":"2024-09-12T03:00:58.351599+0000","last_became_peered":"2024-09-12T03:00:58.351599+0000","last_unstale":"2024-09-12T03:02:33.123076+0000","last_undegraded":"2024-09-12T03:02:33.123076+0000","last_fullsized":"2024-09-12T03:02:33.123076+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:26.398828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.291749+0000","last_change":"2024-09-12T03:00:56.436185+0000","last_active":"2024-09-12T03:02:04.291749+0000","last_peered":"2024-09-12T03:02:04.291749+0000","last_clean":"2024-09-12T03:02:04.291749+0000","last_became_active":"2024-09-12T03:00:56.435717+0000","last_became_peered":"2024-09-12T03:00:56.435717+0000","last_unstale":"2024-09-12T03:02:04.291749+0000","last_undegraded":"2024-09-12T03:02:04.291749+0000","last_fullsized":"2024-09-12T03:02:04.291749+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:27.303182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.119766+0000","last_change":"2024-09-12T03:02:04.333661+0000","last_active":"2024-09-12T03:02:33.119766+0000","last_peered":"2024-09-12T03:02:33.119766+0000","last_clean":"2024-09-12T03:02:33.119766+0000","last_became_active":"2024-09-12T03:02:04.333217+0000","last_became_peered":"2024-09-12T03:02:04.333217+0000","last_unstale":"2024-09-12T03:02:33.119766+0000","last_undegraded":"2024-09-12T03:02:33.119766+0000","last_fullsized":"2024-09-12T03:02:33.119766+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:23:47.212734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'3","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:08.214796+0000","last_change":"2024-09-12T03:02:04.296884+0000","last_active":"2024-09-12T03:02:08.214796+0000","last_peered":"2024-09-12T03:02:08.214796+0000","last_clean":"2024-09-12T03:02:08.214796+0000","last_became_active":"2024-09-12T03:02:04.290894+0000","last_became_peered":"2024-09-12T03:02:04.290894+0000","last_unstale":"2024-09-12T03:02:08.214796+0000","last_undegraded":"2024-09-12T03:02:08.214796+0000","last_fullsized":"2024-09-12T03:02:08.214796+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:49:19.749568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.118228+0000","last_change":"2024-09-12T03:02:04.334096+0000","last_active":"2024-09-12T03:02:33.118228+0000","last_peered":"2024-09-12T03:02:33.118228+0000","last_clean":"2024-09-12T03:02:33.118228+0000","last_became_active":"2024-09-12T03:02:04.333641+0000","last_became_peered":"2024-09-12T03:02:04.333641+0000","last_unstale":"2024-09-12T03:02:33.118228+0000","last_undegraded":"2024-09-12T03:02:33.118228+0000","last_fullsized":"2024-09-12T03:02:33.118228+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:24.085293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332966+0000","last_change":"2024-09-12T03:00:56.314182+0000","last_active":"2024-09-12T03:02:04.332966+0000","last_peered":"2024-09-12T03:02:04.332966+0000","last_clean":"2024-09-12T03:02:04.332966+0000","last_became_active":"2024-09-12T03:00:56.313749+0000","last_became_peered":"2024-09-12T03:00:56.313749+0000","last_unstale":"2024-09-12T03:02:04.332966+0000","last_undegraded":"2024-09-12T03:02:04.332966+0000","last_fullsized":"2024-09-12T03:02:04.332966+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:31:45.767500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.125121+0000","last_change":"2024-09-12T03:00:58.344356+0000","last_active":"2024-09-12T03:02:33.125121+0000","last_peered":"2024-09-12T03:02:33.125121+0000","last_clean":"2024-09-12T03:02:33.125121+0000","last_became_active":"2024-09-12T03:00:58.344035+0000","last_became_peered":"2024-09-12T03:00:58.344035+0000","last_unstale":"2024-09-12T03:02:33.125121+0000","last_undegraded":"2024-09-12T03:02:33.125121+0000","last_fullsized":"2024-09-12T03:02:33.125121+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:45:18.681018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.287585+0000","last_change":"2024-09-12T03:00:56.315653+0000","last_active":"2024-09-12T03:02:04.287585+0000","last_peered":"2024-09-12T03:02:04.287585+0000","last_clean":"2024-09-12T03:02:04.287585+0000","last_became_active":"2024-09-12T03:00:56.315284+0000","last_became_peered":"2024-09-12T03:00:56.315284+0000","last_unstale":"2024-09-12T03:02:04.287585+0000","last_undegraded":"2024-09-12T03:02:04.287585+0000","last_fullsized":"2024-09-12T03:02:04.287585+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:42.934020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.122059+0000","last_change":"2024-09-12T03:02:04.291128+0000","last_active":"2024-09-12T03:02:33.122059+0000","last_peered":"2024-09-12T03:02:33.122059+0000","last_clean":"2024-09-12T03:02:33.122059+0000","last_became_active":"2024-09-12T03:02:04.287836+0000","last_became_peered":"2024-09-12T03:02:04.287836+0000","last_unstale":"2024-09-12T03:02:33.122059+0000","last_undegraded":"2024-09-12T03:02:33.122059+0000","last_fullsized":"2024-09-12T03:02:33.122059+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:23.956579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.290577+0000","last_change":"2024-09-12T03:00:56.746780+0000","last_active":"2024-09-12T03:02:04.290577+0000","last_peered":"2024-09-12T03:02:04.290577+0000","last_clean":"2024-09-12T03:02:04.290577+0000","last_became_active":"2024-09-12T03:00:56.746414+0000","last_became_peered":"2024-09-12T03:00:56.746414+0000","last_unstale":"2024-09-12T03:02:04.290577+0000","last_undegraded":"2024-09-12T03:02:04.290577+0000","last_fullsized":"2024-09-12T03:02:04.290577+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:15:35.214196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.131100+0000","last_change":"2024-09-12T03:00:58.351713+0000","last_active":"2024-09-12T03:02:33.131100+0000","last_peered":"2024-09-12T03:02:33.131100+0000","last_clean":"2024-09-12T03:02:33.131100+0000","last_became_active":"2024-09-12T03:00:58.351371+0000","last_became_peered":"2024-09-12T03:00:58.351371+0000","last_unstale":"2024-09-12T03:02:33.131100+0000","last_undegraded":"2024-09-12T03:02:33.131100+0000","last_fullsized":"2024-09-12T03:02:33.131100+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:13.346956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'137","reported_seq":336,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284318+0000","last_change":"2024-09-12T02:59:45.461482+0000","last_active":"2024-09-12T03:02:04.284318+0000","last_peered":"2024-09-12T03:02:04.284318+0000","last_clean":"2024-09-12T03:02:04.284318+0000","last_became_active":"2024-09-12T02:59:45.461058+0000","last_became_peered":"2024-09-12T02:59:45.461058+0000","last_unstale":"2024-09-12T03:02:04.284318+0000","last_undegraded":"2024-09-12T03:02:04.284318+0000","last_fullsized":"2024-09-12T03:02:04.284318+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:18.034438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:18.034438+0000","last_clean_scrub_stamp":"2024-09-12T02:56:18.034438+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:09.251858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289360+0000","last_change":"2024-09-12T03:00:56.315245+0000","last_active":"2024-09-12T03:02:04.289360+0000","last_peered":"2024-09-12T03:02:04.289360+0000","last_clean":"2024-09-12T03:02:04.289360+0000","last_became_active":"2024-09-12T03:00:56.314887+0000","last_became_peered":"2024-09-12T03:00:56.314887+0000","last_unstale":"2024-09-12T03:02:04.289360+0000","last_undegraded":"2024-09-12T03:02:04.289360+0000","last_fullsized":"2024-09-12T03:02:04.289360+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:48.750657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.113189+0000","last_change":"2024-09-12T03:00:58.357874+0000","last_active":"2024-09-12T03:02:33.113189+0000","last_peered":"2024-09-12T03:02:33.113189+0000","last_clean":"2024-09-12T03:02:33.113189+0000","last_became_active":"2024-09-12T03:00:58.357575+0000","last_became_peered":"2024-09-12T03:00:58.357575+0000","last_unstale":"2024-09-12T03:02:33.113189+0000","last_undegraded":"2024-09-12T03:02:33.113189+0000","last_fullsized":"2024-09-12T03:02:33.113189+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:46.125620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284282+0000","last_change":"2024-09-12T03:00:56.312263+0000","last_active":"2024-09-12T03:02:04.284282+0000","last_peered":"2024-09-12T03:02:04.284282+0000","last_clean":"2024-09-12T03:02:04.284282+0000","last_became_active":"2024-09-12T03:00:56.311894+0000","last_became_peered":"2024-09-12T03:00:56.311894+0000","last_unstale":"2024-09-12T03:02:04.284282+0000","last_undegraded":"2024-09-12T03:02:04.284282+0000","last_fullsized":"2024-09-12T03:02:04.284282+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:52.434203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.114489+0000","last_change":"2024-09-12T03:00:58.350762+0000","last_active":"2024-09-12T03:02:33.114489+0000","last_peered":"2024-09-12T03:02:33.114489+0000","last_clean":"2024-09-12T03:02:33.114489+0000","last_became_active":"2024-09-12T03:00:58.348038+0000","last_became_peered":"2024-09-12T03:00:58.348038+0000","last_unstale":"2024-09-12T03:02:33.114489+0000","last_undegraded":"2024-09-12T03:02:33.114489+0000","last_fullsized":"2024-09-12T03:02:33.114489+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:08.916636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":146,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:03:04.573615+0000","last_change":"2024-09-12T03:00:56.435641+0000","last_active":"2024-09-12T03:03:04.573615+0000","last_peered":"2024-09-12T03:03:04.573615+0000","last_clean":"2024-09-12T03:03:04.573615+0000","last_became_active":"2024-09-12T03:00:56.435319+0000","last_became_peered":"2024-09-12T03:00:56.435319+0000","last_unstale":"2024-09-12T03:03:04.573615+0000","last_undegraded":"2024-09-12T03:03:04.573615+0000","last_fullsized":"2024-09-12T03:03:04.573615+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:56.894978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.112818+0000","last_change":"2024-09-12T03:00:58.344708+0000","last_active":"2024-09-12T03:02:33.112818+0000","last_peered":"2024-09-12T03:02:33.112818+0000","last_clean":"2024-09-12T03:02:33.112818+0000","last_became_active":"2024-09-12T03:00:58.344389+0000","last_became_peered":"2024-09-12T03:00:58.344389+0000","last_unstale":"2024-09-12T03:02:33.112818+0000","last_undegraded":"2024-09-12T03:02:33.112818+0000","last_fullsized":"2024-09-12T03:02:33.112818+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:08.160383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284373+0000","last_change":"2024-09-12T03:00:56.317068+0000","last_active":"2024-09-12T03:02:04.284373+0000","last_peered":"2024-09-12T03:02:04.284373+0000","last_clean":"2024-09-12T03:02:04.284373+0000","last_became_active":"2024-09-12T03:00:56.316698+0000","last_became_peered":"2024-09-12T03:00:56.316698+0000","last_unstale":"2024-09-12T03:02:04.284373+0000","last_undegraded":"2024-09-12T03:02:04.284373+0000","last_fullsized":"2024-09-12T03:02:04.284373+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:41:19.199418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.110097+0000","last_change":"2024-09-12T03:00:58.347814+0000","last_active":"2024-09-12T03:02:33.110097+0000","last_peered":"2024-09-12T03:02:33.110097+0000","last_clean":"2024-09-12T03:02:33.110097+0000","last_became_active":"2024-09-12T03:00:58.347112+0000","last_became_peered":"2024-09-12T03:00:58.347112+0000","last_unstale":"2024-09-12T03:02:33.110097+0000","last_undegraded":"2024-09-12T03:02:33.110097+0000","last_fullsized":"2024-09-12T03:02:33.110097+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:59.325113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284305+0000","last_change":"2024-09-12T03:00:56.435771+0000","last_active":"2024-09-12T03:02:04.284305+0000","last_peered":"2024-09-12T03:02:04.284305+0000","last_clean":"2024-09-12T03:02:04.284305+0000","last_became_active":"2024-09-12T03:00:56.435448+0000","last_became_peered":"2024-09-12T03:00:56.435448+0000","last_unstale":"2024-09-12T03:02:04.284305+0000","last_undegraded":"2024-09-12T03:02:04.284305+0000","last_fullsized":"2024-09-12T03:02:04.284305+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:33.650338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.118219+0000","last_change":"2024-09-12T03:00:58.341071+0000","last_active":"2024-09-12T03:02:33.118219+0000","last_peered":"2024-09-12T03:02:33.118219+0000","last_clean":"2024-09-12T03:02:33.118219+0000","last_became_active":"2024-09-12T03:00:58.340753+0000","last_became_peered":"2024-09-12T03:00:58.340753+0000","last_unstale":"2024-09-12T03:02:33.118219+0000","last_undegraded":"2024-09-12T03:02:33.118219+0000","last_fullsized":"2024-09-12T03:02:33.118219+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:21.217915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292316+0000","last_change":"2024-09-12T03:00:56.436058+0000","last_active":"2024-09-12T03:02:04.292316+0000","last_peered":"2024-09-12T03:02:04.292316+0000","last_clean":"2024-09-12T03:02:04.292316+0000","last_became_active":"2024-09-12T03:00:56.435518+0000","last_became_peered":"2024-09-12T03:00:56.435518+0000","last_unstale":"2024-09-12T03:02:04.292316+0000","last_undegraded":"2024-09-12T03:02:04.292316+0000","last_fullsized":"2024-09-12T03:02:04.292316+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:32.235097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.110699+0000","last_change":"2024-09-12T03:00:58.348236+0000","last_active":"2024-09-12T03:02:33.110699+0000","last_peered":"2024-09-12T03:02:33.110699+0000","last_clean":"2024-09-12T03:02:33.110699+0000","last_became_active":"2024-09-12T03:00:58.347727+0000","last_became_peered":"2024-09-12T03:00:58.347727+0000","last_unstale":"2024-09-12T03:02:33.110699+0000","last_undegraded":"2024-09-12T03:02:33.110699+0000","last_fullsized":"2024-09-12T03:02:33.110699+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:19.233163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'13","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:58.357683+0000","last_change":"2024-09-12T03:00:56.310537+0000","last_active":"2024-09-12T03:02:58.357683+0000","last_peered":"2024-09-12T03:02:58.357683+0000","last_clean":"2024-09-12T03:02:58.357683+0000","last_became_active":"2024-09-12T03:00:56.310135+0000","last_became_peered":"2024-09-12T03:00:56.310135+0000","last_unstale":"2024-09-12T03:02:58.357683+0000","last_undegraded":"2024-09-12T03:02:58.357683+0000","last_fullsized":"2024-09-12T03:02:58.357683+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:10:55.584339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.123219+0000","last_change":"2024-09-12T03:00:58.358417+0000","last_active":"2024-09-12T03:02:33.123219+0000","last_peered":"2024-09-12T03:02:33.123219+0000","last_clean":"2024-09-12T03:02:33.123219+0000","last_became_active":"2024-09-12T03:00:58.358119+0000","last_became_peered":"2024-09-12T03:00:58.358119+0000","last_unstale":"2024-09-12T03:02:33.123219+0000","last_undegraded":"2024-09-12T03:02:33.123219+0000","last_fullsized":"2024-09-12T03:02:33.123219+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:10.897594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289526+0000","last_change":"2024-09-12T03:00:56.747044+0000","last_active":"2024-09-12T03:02:04.289526+0000","last_peered":"2024-09-12T03:02:04.289526+0000","last_clean":"2024-09-12T03:02:04.289526+0000","last_became_active":"2024-09-12T03:00:56.746699+0000","last_became_peered":"2024-09-12T03:00:56.746699+0000","last_unstale":"2024-09-12T03:02:04.289526+0000","last_undegraded":"2024-09-12T03:02:04.289526+0000","last_fullsized":"2024-09-12T03:02:04.289526+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:11.420109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.113393+0000","last_change":"2024-09-12T03:00:58.346854+0000","last_active":"2024-09-12T03:02:33.113393+0000","last_peered":"2024-09-12T03:02:33.113393+0000","last_clean":"2024-09-12T03:02:33.113393+0000","last_became_active":"2024-09-12T03:00:58.346535+0000","last_became_peered":"2024-09-12T03:00:58.346535+0000","last_unstale":"2024-09-12T03:02:33.113393+0000","last_undegraded":"2024-09-12T03:02:33.113393+0000","last_fullsized":"2024-09-12T03:02:33.113393+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:46.637914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332423+0000","last_change":"2024-09-12T03:00:56.435957+0000","last_active":"2024-09-12T03:02:04.332423+0000","last_peered":"2024-09-12T03:02:04.332423+0000","last_clean":"2024-09-12T03:02:04.332423+0000","last_became_active":"2024-09-12T03:00:56.435603+0000","last_became_peered":"2024-09-12T03:00:56.435603+0000","last_unstale":"2024-09-12T03:02:04.332423+0000","last_undegraded":"2024-09-12T03:02:04.332423+0000","last_fullsized":"2024-09-12T03:02:04.332423+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:34.553189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.089792+0000","last_change":"2024-09-12T03:00:58.337784+0000","last_active":"2024-09-12T03:02:33.089792+0000","last_peered":"2024-09-12T03:02:33.089792+0000","last_clean":"2024-09-12T03:02:33.089792+0000","last_became_active":"2024-09-12T03:00:58.337415+0000","last_became_peered":"2024-09-12T03:00:58.337415+0000","last_unstale":"2024-09-12T03:02:33.089792+0000","last_undegraded":"2024-09-12T03:02:33.089792+0000","last_fullsized":"2024-09-12T03:02:33.089792+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:38:15.557067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282398+0000","last_change":"2024-09-12T03:00:56.315356+0000","last_active":"2024-09-12T03:02:04.282398+0000","last_peered":"2024-09-12T03:02:04.282398+0000","last_clean":"2024-09-12T03:02:04.282398+0000","last_became_active":"2024-09-12T03:00:56.314873+0000","last_became_peered":"2024-09-12T03:00:56.314873+0000","last_unstale":"2024-09-12T03:02:04.282398+0000","last_undegraded":"2024-09-12T03:02:04.282398+0000","last_fullsized":"2024-09-12T03:02:04.282398+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:55.710156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120404+0000","last_change":"2024-09-12T03:00:58.346706+0000","last_active":"2024-09-12T03:02:33.120404+0000","last_peered":"2024-09-12T03:02:33.120404+0000","last_clean":"2024-09-12T03:02:33.120404+0000","last_became_active":"2024-09-12T03:00:58.346363+0000","last_became_peered":"2024-09-12T03:00:58.346363+0000","last_unstale":"2024-09-12T03:02:33.120404+0000","last_undegraded":"2024-09-12T03:02:33.120404+0000","last_fullsized":"2024-09-12T03:02:33.120404+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:13.468639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.299582+0000","last_change":"2024-09-12T03:02:04.299147+0000","last_active":"2024-09-12T03:02:04.299582+0000","last_peered":"2024-09-12T03:02:04.299582+0000","last_clean":"2024-09-12T03:02:04.299582+0000","last_became_active":"2024-09-12T03:02:04.293211+0000","last_became_peered":"2024-09-12T03:02:04.293211+0000","last_unstale":"2024-09-12T03:02:04.299582+0000","last_undegraded":"2024-09-12T03:02:04.299582+0000","last_fullsized":"2024-09-12T03:02:04.299582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:01:39.222476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.118849+0000","last_change":"2024-09-12T03:00:58.356682+0000","last_active":"2024-09-12T03:02:33.118849+0000","last_peered":"2024-09-12T03:02:33.118849+0000","last_clean":"2024-09-12T03:02:33.118849+0000","last_became_active":"2024-09-12T03:00:58.356383+0000","last_became_peered":"2024-09-12T03:00:58.356383+0000","last_unstale":"2024-09-12T03:02:33.118849+0000","last_undegraded":"2024-09-12T03:02:33.118849+0000","last_fullsized":"2024-09-12T03:02:33.118849+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:14:16.428393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292053+0000","last_change":"2024-09-12T03:00:56.314943+0000","last_active":"2024-09-12T03:02:04.292053+0000","last_peered":"2024-09-12T03:02:04.292053+0000","last_clean":"2024-09-12T03:02:04.292053+0000","last_became_active":"2024-09-12T03:00:56.314602+0000","last_became_peered":"2024-09-12T03:00:56.314602+0000","last_unstale":"2024-09-12T03:02:04.292053+0000","last_undegraded":"2024-09-12T03:02:04.292053+0000","last_fullsized":"2024-09-12T03:02:04.292053+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:31.207880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.131538+0000","last_change":"2024-09-12T03:00:58.351774+0000","last_active":"2024-09-12T03:02:33.131538+0000","last_peered":"2024-09-12T03:02:33.131538+0000","last_clean":"2024-09-12T03:02:33.131538+0000","last_became_active":"2024-09-12T03:00:58.351453+0000","last_became_peered":"2024-09-12T03:00:58.351453+0000","last_unstale":"2024-09-12T03:02:33.131538+0000","last_undegraded":"2024-09-12T03:02:33.131538+0000","last_fullsized":"2024-09-12T03:02:33.131538+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:18:24.260110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.130930+0000","last_change":"2024-09-12T03:00:58.346886+0000","last_active":"2024-09-12T03:02:33.130930+0000","last_peered":"2024-09-12T03:02:33.130930+0000","last_clean":"2024-09-12T03:02:33.130930+0000","last_became_active":"2024-09-12T03:00:58.346499+0000","last_became_peered":"2024-09-12T03:00:58.346499+0000","last_unstale":"2024-09-12T03:02:33.130930+0000","last_undegraded":"2024-09-12T03:02:33.130930+0000","last_fullsized":"2024-09-12T03:02:33.130930+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:02:24.045610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292497+0000","last_change":"2024-09-12T03:00:56.317443+0000","last_active":"2024-09-12T03:02:04.292497+0000","last_peered":"2024-09-12T03:02:04.292497+0000","last_clean":"2024-09-12T03:02:04.292497+0000","last_became_active":"2024-09-12T03:00:56.317045+0000","last_became_peered":"2024-09-12T03:00:56.317045+0000","last_unstale":"2024-09-12T03:02:04.292497+0000","last_undegraded":"2024-09-12T03:02:04.292497+0000","last_fullsized":"2024-09-12T03:02:04.292497+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:07.687606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.132782+0000","last_change":"2024-09-12T03:00:58.337257+0000","last_active":"2024-09-12T03:02:33.132782+0000","last_peered":"2024-09-12T03:02:33.132782+0000","last_clean":"2024-09-12T03:02:33.132782+0000","last_became_active":"2024-09-12T03:00:58.336847+0000","last_became_peered":"2024-09-12T03:00:58.336847+0000","last_unstale":"2024-09-12T03:02:33.132782+0000","last_undegraded":"2024-09-12T03:02:33.132782+0000","last_fullsized":"2024-09-12T03:02:33.132782+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:22.846206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292165+0000","last_change":"2024-09-12T03:00:56.316255+0000","last_active":"2024-09-12T03:02:04.292165+0000","last_peered":"2024-09-12T03:02:04.292165+0000","last_clean":"2024-09-12T03:02:04.292165+0000","last_became_active":"2024-09-12T03:00:56.315906+0000","last_became_peered":"2024-09-12T03:00:56.315906+0000","last_unstale":"2024-09-12T03:02:04.292165+0000","last_undegraded":"2024-09-12T03:02:04.292165+0000","last_fullsized":"2024-09-12T03:02:04.292165+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:14.798601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120352+0000","last_change":"2024-09-12T03:00:58.347206+0000","last_active":"2024-09-12T03:02:33.120352+0000","last_peered":"2024-09-12T03:02:33.120352+0000","last_clean":"2024-09-12T03:02:33.120352+0000","last_became_active":"2024-09-12T03:00:58.346923+0000","last_became_peered":"2024-09-12T03:00:58.346923+0000","last_unstale":"2024-09-12T03:02:33.120352+0000","last_undegraded":"2024-09-12T03:02:33.120352+0000","last_fullsized":"2024-09-12T03:02:33.120352+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:41.376211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.283743+0000","last_change":"2024-09-12T03:00:56.435431+0000","last_active":"2024-09-12T03:02:04.283743+0000","last_peered":"2024-09-12T03:02:04.283743+0000","last_clean":"2024-09-12T03:02:04.283743+0000","last_became_active":"2024-09-12T03:00:56.435088+0000","last_became_peered":"2024-09-12T03:00:56.435088+0000","last_unstale":"2024-09-12T03:02:04.283743+0000","last_undegraded":"2024-09-12T03:02:04.283743+0000","last_fullsized":"2024-09-12T03:02:04.283743+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:24.738340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.114461+0000","last_change":"2024-09-12T03:00:58.351737+0000","last_active":"2024-09-12T03:02:33.114461+0000","last_peered":"2024-09-12T03:02:33.114461+0000","last_clean":"2024-09-12T03:02:33.114461+0000","last_became_active":"2024-09-12T03:00:58.351407+0000","last_became_peered":"2024-09-12T03:00:58.351407+0000","last_unstale":"2024-09-12T03:02:33.114461+0000","last_undegraded":"2024-09-12T03:02:33.114461+0000","last_fullsized":"2024-09-12T03:02:33.114461+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:34.206703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:03:02.121967+0000","last_change":"2024-09-12T03:00:56.310910+0000","last_active":"2024-09-12T03:03:02.121967+0000","last_peered":"2024-09-12T03:03:02.121967+0000","last_clean":"2024-09-12T03:03:02.121967+0000","last_became_active":"2024-09-12T03:00:56.310657+0000","last_became_peered":"2024-09-12T03:00:56.310657+0000","last_unstale":"2024-09-12T03:03:02.121967+0000","last_undegraded":"2024-09-12T03:03:02.121967+0000","last_fullsized":"2024-09-12T03:03:02.121967+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:06.117215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74598,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":738,"num_read_kb":449,"num_write":178,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":223794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245899,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645352,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":431835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906988,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":1094282,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070514,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":432166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299447,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":431947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528379,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":1023901,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":40,"seq":171798691905,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":880640,"data_stored":431438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":33,"seq":141733920837,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":1023192,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051659,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":432268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280592,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":431632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476821,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":503176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607643,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":501826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":432709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":70595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":71945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":72566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:03:11.109 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T03:03:11.109 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:03:11.378 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:12.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.879+0000 7fe78e6a7640 1 -- 172.21.15.116:0/1718760915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe788100a90 msgr2=0x7fe788100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:12.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.879+0000 7fe78e6a7640 1 --2- 172.21.15.116:0/1718760915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe788100a90 0x7fe788100ef0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe778009930 tx=0x7fe77802a420 comp rx=0 tx=0).stop 2024-09-12T03:03:12.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.880+0000 7fe78e6a7640 1 -- 172.21.15.116:0/1718760915 shutdown_connections 2024-09-12T03:03:12.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.880+0000 7fe78e6a7640 1 --2- 172.21.15.116:0/1718760915 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe788101430 0x7fe78810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:12.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.880+0000 7fe78e6a7640 1 --2- 172.21.15.116:0/1718760915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe788100a90 0x7fe788100ef0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:12.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.880+0000 7fe78e6a7640 1 --2- 172.21.15.116:0/1718760915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe788106a90 0x7fe788106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:12.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.880+0000 7fe78e6a7640 1 -- 172.21.15.116:0/1718760915 >> 172.21.15.116:0/1718760915 conn(0x7fe7880fc7d0 msgr2=0x7fe7880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:12.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.880+0000 7fe78e6a7640 1 -- 172.21.15.116:0/1718760915 shutdown_connections 2024-09-12T03:03:12.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.880+0000 7fe78e6a7640 1 -- 172.21.15.116:0/1718760915 wait complete. 2024-09-12T03:03:12.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.880+0000 7fe78e6a7640 1 Processor -- start 2024-09-12T03:03:12.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.880+0000 7fe78e6a7640 1 -- start start 2024-09-12T03:03:12.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.881+0000 7fe78e6a7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe788100a90 0x7fe788199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:12.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.881+0000 7fe78e6a7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe788101430 0x7fe7881998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:12.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.881+0000 7fe78e6a7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe788106a90 0x7fe7881a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:12.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.881+0000 7fe78e6a7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe788110eb0 con 0x7fe788100a90 2024-09-12T03:03:12.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.881+0000 7fe78e6a7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe788110d30 con 0x7fe788106a90 2024-09-12T03:03:12.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.881+0000 7fe78e6a7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe788111030 con 0x7fe788101430 2024-09-12T03:03:12.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.881+0000 7fe787fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe788100a90 0x7fe788199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:12.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.881+0000 7fe787fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe788100a90 0x7fe788199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43524/0 (socket says 172.21.15.116:43524) 2024-09-12T03:03:12.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.881+0000 7fe787fff640 1 -- 172.21.15.116:0/2527112611 learned_addr learned my addr 172.21.15.116:0/2527112611 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:12.090 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:03:12.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.881+0000 7fe78cc44640 1 --2- 172.21.15.116:0/2527112611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe788106a90 0x7fe7881a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:12.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.881+0000 7fe7877fe640 1 --2- 172.21.15.116:0/2527112611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe788101430 0x7fe7881998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:12.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.881+0000 7fe787fff640 1 -- 172.21.15.116:0/2527112611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe788101430 msgr2=0x7fe7881998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:12.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.881+0000 7fe787fff640 1 --2- 172.21.15.116:0/2527112611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe788101430 0x7fe7881998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:12.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.881+0000 7fe787fff640 1 -- 172.21.15.116:0/2527112611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe788106a90 msgr2=0x7fe7881a4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:12.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.881+0000 7fe787fff640 1 --2- 172.21.15.116:0/2527112611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe788106a90 0x7fe7881a4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:12.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.881+0000 7fe787fff640 1 -- 172.21.15.116:0/2527112611 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7881a4940 con 0x7fe788100a90 2024-09-12T03:03:12.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.881+0000 7fe787fff640 1 --2- 172.21.15.116:0/2527112611 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe788100a90 0x7fe788199380 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe77000b4a0 tx=0x7fe77000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:12.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.881+0000 7fe7857fa640 1 -- 172.21.15.116:0/2527112611 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe770014070 con 0x7fe788100a90 2024-09-12T03:03:12.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.881+0000 7fe7857fa640 1 -- 172.21.15.116:0/2527112611 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe770010af0 con 0x7fe788100a90 2024-09-12T03:03:12.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.881+0000 7fe78e6a7640 1 -- 172.21.15.116:0/2527112611 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7881a4b10 con 0x7fe788100a90 2024-09-12T03:03:12.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.881+0000 7fe7857fa640 1 -- 172.21.15.116:0/2527112611 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe770017d10 con 0x7fe788100a90 2024-09-12T03:03:12.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.882+0000 7fe78e6a7640 1 -- 172.21.15.116:0/2527112611 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7881a4fd0 con 0x7fe788100a90 2024-09-12T03:03:12.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.883+0000 7fe78e6a7640 1 -- 172.21.15.116:0/2527112611 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe788100ef0 con 0x7fe788100a90 2024-09-12T03:03:12.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.884+0000 7fe7857fa640 1 -- 172.21.15.116:0/2527112611 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fe770017070 con 0x7fe788100a90 2024-09-12T03:03:12.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.884+0000 7fe7857fa640 1 --2- 172.21.15.116:0/2527112611 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe7740a2c10 0x7fe7740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:12.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.884+0000 7fe7857fa640 1 -- 172.21.15.116:0/2527112611 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe7700ce3b0 con 0x7fe788100a90 2024-09-12T03:03:12.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.884+0000 7fe7877fe640 1 --2- 172.21.15.116:0/2527112611 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe7740a2c10 0x7fe7740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:12.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.885+0000 7fe7877fe640 1 --2- 172.21.15.116:0/2527112611 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe7740a2c10 0x7fe7740a50d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe778009900 tx=0x7fe7780058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:12.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:11.889+0000 7fe7857fa640 1 -- 172.21.15.116:0/2527112611 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe77009aca0 con 0x7fe788100a90 2024-09-12T03:03:12.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:12.053+0000 7fe78e6a7640 1 -- 172.21.15.116:0/2527112611 --> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe788062630 con 0x7fe7740a2c10 2024-09-12T03:03:12.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:12.059+0000 7fe7857fa640 1 -- 172.21.15.116:0/2527112611 <== mgr.14150 v2:172.21.15.116:6800/2896038008 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258980 (secure 0 0 0) 0x7fe788062630 con 0x7fe7740a2c10 2024-09-12T03:03:12.094 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-09-12T03:03:12.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:12.090+0000 7fe78e6a7640 1 -- 172.21.15.116:0/2527112611 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe7740a2c10 msgr2=0x7fe7740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:12.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:12.090+0000 7fe78e6a7640 1 --2- 172.21.15.116:0/2527112611 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe7740a2c10 0x7fe7740a50d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe778009900 tx=0x7fe7780058d0 comp rx=0 tx=0).stop 2024-09-12T03:03:12.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:12.090+0000 7fe78e6a7640 1 -- 172.21.15.116:0/2527112611 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe788100a90 msgr2=0x7fe788199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:12.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:12.090+0000 7fe78e6a7640 1 --2- 172.21.15.116:0/2527112611 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe788100a90 0x7fe788199380 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe77000b4a0 tx=0x7fe77000b960 comp rx=0 tx=0).stop 2024-09-12T03:03:12.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:12.090+0000 7fe78e6a7640 1 -- 172.21.15.116:0/2527112611 shutdown_connections 2024-09-12T03:03:12.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:12.090+0000 7fe78e6a7640 1 --2- 172.21.15.116:0/2527112611 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe7740a2c10 0x7fe7740a50d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:12.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:12.090+0000 7fe78e6a7640 1 --2- 172.21.15.116:0/2527112611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe788106a90 0x7fe7881a4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:12.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:12.090+0000 7fe78e6a7640 1 --2- 172.21.15.116:0/2527112611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe788101430 0x7fe7881998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:12.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:12.090+0000 7fe78e6a7640 1 --2- 172.21.15.116:0/2527112611 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe788100a90 0x7fe788199380 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:12.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:12.090+0000 7fe78e6a7640 1 -- 172.21.15.116:0/2527112611 >> 172.21.15.116:0/2527112611 conn(0x7fe7880fc7d0 msgr2=0x7fe7880fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:12.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:12.090+0000 7fe78e6a7640 1 -- 172.21.15.116:0/2527112611 shutdown_connections 2024-09-12T03:03:12.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:12.090+0000 7fe78e6a7640 1 -- 172.21.15.116:0/2527112611 wait complete. 2024-09-12T03:03:12.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:12 smithi183 ceph-mon[35535]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:03:12.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:12 smithi183 ceph-mon[35535]: pgmap v344: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-12T03:03:12.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:12 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:03:12.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:12 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:03:12.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:12 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:03:12.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:12 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:03:12.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:12 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:03:12.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:12 smithi177 ceph-mon[35305]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:03:12.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:12 smithi177 ceph-mon[35305]: pgmap v344: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-12T03:03:12.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:12 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:03:12.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:12 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:03:12.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:12 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:03:12.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:12 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:03:12.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:12 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:03:12.529 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:12 smithi116 ceph-mon[31410]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:03:12.529 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:12 smithi116 ceph-mon[31410]: pgmap v344: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-12T03:03:12.529 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:12 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:03:12.529 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:12 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:03:12.529 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:12 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:03:12.530 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:12 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:03:12.530 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:12 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:03:12.816 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-09-12T03:03:11.138777+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665200,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":889,"num_read_kb":568,"num_write":361,"num_write_kb":3069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180875,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":789,"ondisk_log_size":789,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368404,"kb_used_data":10836,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573420,"statfs":{"total":1151940427776,"available":1151563182080,"internally_reserved":0,"allocated":11096064,"data_stored":7170372,"data_compressed":107490,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8679,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":91,"num_read_kb":53,"num_write":8,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002839"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.133534+0000","last_change":"2024-09-12T03:00:58.348563+0000","last_active":"2024-09-12T03:02:33.133534+0000","last_peered":"2024-09-12T03:02:33.133534+0000","last_clean":"2024-09-12T03:02:33.133534+0000","last_became_active":"2024-09-12T03:00:58.347940+0000","last_became_peered":"2024-09-12T03:00:58.347940+0000","last_unstale":"2024-09-12T03:02:33.133534+0000","last_undegraded":"2024-09-12T03:02:33.133534+0000","last_fullsized":"2024-09-12T03:02:33.133534+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:43.489238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.119339+0000","last_change":"2024-09-12T03:00:58.351969+0000","last_active":"2024-09-12T03:02:33.119339+0000","last_peered":"2024-09-12T03:02:33.119339+0000","last_clean":"2024-09-12T03:02:33.119339+0000","last_became_active":"2024-09-12T03:00:58.351679+0000","last_became_peered":"2024-09-12T03:00:58.351679+0000","last_unstale":"2024-09-12T03:02:33.119339+0000","last_undegraded":"2024-09-12T03:02:33.119339+0000","last_fullsized":"2024-09-12T03:02:33.119339+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:27:01.174890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120214+0000","last_change":"2024-09-12T03:00:58.346045+0000","last_active":"2024-09-12T03:02:33.120214+0000","last_peered":"2024-09-12T03:02:33.120214+0000","last_clean":"2024-09-12T03:02:33.120214+0000","last_became_active":"2024-09-12T03:00:58.345719+0000","last_became_peered":"2024-09-12T03:00:58.345719+0000","last_unstale":"2024-09-12T03:02:33.120214+0000","last_undegraded":"2024-09-12T03:02:33.120214+0000","last_fullsized":"2024-09-12T03:02:33.120214+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:12.312650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.111069+0000","last_change":"2024-09-12T03:00:58.348165+0000","last_active":"2024-09-12T03:02:33.111069+0000","last_peered":"2024-09-12T03:02:33.111069+0000","last_clean":"2024-09-12T03:02:33.111069+0000","last_became_active":"2024-09-12T03:00:58.347734+0000","last_became_peered":"2024-09-12T03:00:58.347734+0000","last_unstale":"2024-09-12T03:02:33.111069+0000","last_undegraded":"2024-09-12T03:02:33.111069+0000","last_fullsized":"2024-09-12T03:02:33.111069+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:05.945900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.108519+0000","last_change":"2024-09-12T03:00:58.339316+0000","last_active":"2024-09-12T03:02:33.108519+0000","last_peered":"2024-09-12T03:02:33.108519+0000","last_clean":"2024-09-12T03:02:33.108519+0000","last_became_active":"2024-09-12T03:00:58.338964+0000","last_became_peered":"2024-09-12T03:00:58.338964+0000","last_unstale":"2024-09-12T03:02:33.108519+0000","last_undegraded":"2024-09-12T03:02:33.108519+0000","last_fullsized":"2024-09-12T03:02:33.108519+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:01:50.721276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.110921+0000","last_change":"2024-09-12T03:00:58.346838+0000","last_active":"2024-09-12T03:02:33.110921+0000","last_peered":"2024-09-12T03:02:33.110921+0000","last_clean":"2024-09-12T03:02:33.110921+0000","last_became_active":"2024-09-12T03:00:58.346486+0000","last_became_peered":"2024-09-12T03:00:58.346486+0000","last_unstale":"2024-09-12T03:02:33.110921+0000","last_undegraded":"2024-09-12T03:02:33.110921+0000","last_fullsized":"2024-09-12T03:02:33.110921+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:28.687727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.122919+0000","last_change":"2024-09-12T03:00:58.346712+0000","last_active":"2024-09-12T03:02:33.122919+0000","last_peered":"2024-09-12T03:02:33.122919+0000","last_clean":"2024-09-12T03:02:33.122919+0000","last_became_active":"2024-09-12T03:00:58.346397+0000","last_became_peered":"2024-09-12T03:00:58.346397+0000","last_unstale":"2024-09-12T03:02:33.122919+0000","last_undegraded":"2024-09-12T03:02:33.122919+0000","last_fullsized":"2024-09-12T03:02:33.122919+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:48:00.624733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.118885+0000","last_change":"2024-09-12T03:00:58.356166+0000","last_active":"2024-09-12T03:02:33.118885+0000","last_peered":"2024-09-12T03:02:33.118885+0000","last_clean":"2024-09-12T03:02:33.118885+0000","last_became_active":"2024-09-12T03:00:58.354944+0000","last_became_peered":"2024-09-12T03:00:58.354944+0000","last_unstale":"2024-09-12T03:02:33.118885+0000","last_undegraded":"2024-09-12T03:02:33.118885+0000","last_fullsized":"2024-09-12T03:02:33.118885+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:15.840149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.133064+0000","last_change":"2024-09-12T03:00:58.340182+0000","last_active":"2024-09-12T03:02:33.133064+0000","last_peered":"2024-09-12T03:02:33.133064+0000","last_clean":"2024-09-12T03:02:33.133064+0000","last_became_active":"2024-09-12T03:00:58.339576+0000","last_became_peered":"2024-09-12T03:00:58.339576+0000","last_unstale":"2024-09-12T03:02:33.133064+0000","last_undegraded":"2024-09-12T03:02:33.133064+0000","last_fullsized":"2024-09-12T03:02:33.133064+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:16.534844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.117254+0000","last_change":"2024-09-12T03:00:58.351218+0000","last_active":"2024-09-12T03:02:33.117254+0000","last_peered":"2024-09-12T03:02:33.117254+0000","last_clean":"2024-09-12T03:02:33.117254+0000","last_became_active":"2024-09-12T03:00:58.350871+0000","last_became_peered":"2024-09-12T03:00:58.350871+0000","last_unstale":"2024-09-12T03:02:33.117254+0000","last_undegraded":"2024-09-12T03:02:33.117254+0000","last_fullsized":"2024-09-12T03:02:33.117254+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:18:38.367683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.130892+0000","last_change":"2024-09-12T03:00:58.351894+0000","last_active":"2024-09-12T03:02:33.130892+0000","last_peered":"2024-09-12T03:02:33.130892+0000","last_clean":"2024-09-12T03:02:33.130892+0000","last_became_active":"2024-09-12T03:00:58.351445+0000","last_became_peered":"2024-09-12T03:00:58.351445+0000","last_unstale":"2024-09-12T03:02:33.130892+0000","last_undegraded":"2024-09-12T03:02:33.130892+0000","last_fullsized":"2024-09-12T03:02:33.130892+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:22:42.093871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.107323+0000","last_change":"2024-09-12T03:00:58.353460+0000","last_active":"2024-09-12T03:02:33.107323+0000","last_peered":"2024-09-12T03:02:33.107323+0000","last_clean":"2024-09-12T03:02:33.107323+0000","last_became_active":"2024-09-12T03:00:58.353111+0000","last_became_peered":"2024-09-12T03:00:58.353111+0000","last_unstale":"2024-09-12T03:02:33.107323+0000","last_undegraded":"2024-09-12T03:02:33.107323+0000","last_fullsized":"2024-09-12T03:02:33.107323+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:15.916765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.111860+0000","last_change":"2024-09-12T03:00:58.341080+0000","last_active":"2024-09-12T03:02:33.111860+0000","last_peered":"2024-09-12T03:02:33.111860+0000","last_clean":"2024-09-12T03:02:33.111860+0000","last_became_active":"2024-09-12T03:00:58.340782+0000","last_became_peered":"2024-09-12T03:00:58.340782+0000","last_unstale":"2024-09-12T03:02:33.111860+0000","last_undegraded":"2024-09-12T03:02:33.111860+0000","last_fullsized":"2024-09-12T03:02:33.111860+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:52:36.569700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.106506+0000","last_change":"2024-09-12T03:00:58.346791+0000","last_active":"2024-09-12T03:02:33.106506+0000","last_peered":"2024-09-12T03:02:33.106506+0000","last_clean":"2024-09-12T03:02:33.106506+0000","last_became_active":"2024-09-12T03:00:58.346500+0000","last_became_peered":"2024-09-12T03:00:58.346500+0000","last_unstale":"2024-09-12T03:02:33.106506+0000","last_undegraded":"2024-09-12T03:02:33.106506+0000","last_fullsized":"2024-09-12T03:02:33.106506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:32.017589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.112821+0000","last_change":"2024-09-12T03:00:58.349522+0000","last_active":"2024-09-12T03:02:33.112821+0000","last_peered":"2024-09-12T03:02:33.112821+0000","last_clean":"2024-09-12T03:02:33.112821+0000","last_became_active":"2024-09-12T03:00:58.349246+0000","last_became_peered":"2024-09-12T03:00:58.349246+0000","last_unstale":"2024-09-12T03:02:33.112821+0000","last_undegraded":"2024-09-12T03:02:33.112821+0000","last_fullsized":"2024-09-12T03:02:33.112821+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:39.253919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.134146+0000","last_change":"2024-09-12T03:00:58.349909+0000","last_active":"2024-09-12T03:02:33.134146+0000","last_peered":"2024-09-12T03:02:33.134146+0000","last_clean":"2024-09-12T03:02:33.134146+0000","last_became_active":"2024-09-12T03:00:58.349590+0000","last_became_peered":"2024-09-12T03:00:58.349590+0000","last_unstale":"2024-09-12T03:02:33.134146+0000","last_undegraded":"2024-09-12T03:02:33.134146+0000","last_fullsized":"2024-09-12T03:02:33.134146+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:04.540709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.133565+0000","last_change":"2024-09-12T03:00:58.348284+0000","last_active":"2024-09-12T03:02:33.133565+0000","last_peered":"2024-09-12T03:02:33.133565+0000","last_clean":"2024-09-12T03:02:33.133565+0000","last_became_active":"2024-09-12T03:00:58.347465+0000","last_became_peered":"2024-09-12T03:00:58.347465+0000","last_unstale":"2024-09-12T03:02:33.133565+0000","last_undegraded":"2024-09-12T03:02:33.133565+0000","last_fullsized":"2024-09-12T03:02:33.133565+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:13.191899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.122657+0000","last_change":"2024-09-12T03:00:58.337740+0000","last_active":"2024-09-12T03:02:33.122657+0000","last_peered":"2024-09-12T03:02:33.122657+0000","last_clean":"2024-09-12T03:02:33.122657+0000","last_became_active":"2024-09-12T03:00:58.337393+0000","last_became_peered":"2024-09-12T03:00:58.337393+0000","last_unstale":"2024-09-12T03:02:33.122657+0000","last_undegraded":"2024-09-12T03:02:33.122657+0000","last_fullsized":"2024-09-12T03:02:33.122657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:28:56.339167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.111666+0000","last_change":"2024-09-12T03:00:58.345779+0000","last_active":"2024-09-12T03:02:33.111666+0000","last_peered":"2024-09-12T03:02:33.111666+0000","last_clean":"2024-09-12T03:02:33.111666+0000","last_became_active":"2024-09-12T03:00:58.345210+0000","last_became_peered":"2024-09-12T03:00:58.345210+0000","last_unstale":"2024-09-12T03:02:33.111666+0000","last_undegraded":"2024-09-12T03:02:33.111666+0000","last_fullsized":"2024-09-12T03:02:33.111666+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:06:58.271185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120653+0000","last_change":"2024-09-12T03:02:04.294381+0000","last_active":"2024-09-12T03:02:33.120653+0000","last_peered":"2024-09-12T03:02:33.120653+0000","last_clean":"2024-09-12T03:02:33.120653+0000","last_became_active":"2024-09-12T03:02:04.293795+0000","last_became_peered":"2024-09-12T03:02:04.293795+0000","last_unstale":"2024-09-12T03:02:33.120653+0000","last_undegraded":"2024-09-12T03:02:33.120653+0000","last_fullsized":"2024-09-12T03:02:33.120653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:06:58.271007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.124292+0000","last_change":"2024-09-12T03:00:58.349233+0000","last_active":"2024-09-12T03:02:33.124292+0000","last_peered":"2024-09-12T03:02:33.124292+0000","last_clean":"2024-09-12T03:02:33.124292+0000","last_became_active":"2024-09-12T03:00:58.348958+0000","last_became_peered":"2024-09-12T03:00:58.348958+0000","last_unstale":"2024-09-12T03:02:33.124292+0000","last_undegraded":"2024-09-12T03:02:33.124292+0000","last_fullsized":"2024-09-12T03:02:33.124292+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:32:10.271817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.134703+0000","last_change":"2024-09-12T03:00:58.358024+0000","last_active":"2024-09-12T03:02:33.134703+0000","last_peered":"2024-09-12T03:02:33.134703+0000","last_clean":"2024-09-12T03:02:33.134703+0000","last_became_active":"2024-09-12T03:00:58.357434+0000","last_became_peered":"2024-09-12T03:00:58.357434+0000","last_unstale":"2024-09-12T03:02:33.134703+0000","last_undegraded":"2024-09-12T03:02:33.134703+0000","last_fullsized":"2024-09-12T03:02:33.134703+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:01:20.119733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120095+0000","last_change":"2024-09-12T03:00:58.355063+0000","last_active":"2024-09-12T03:02:33.120095+0000","last_peered":"2024-09-12T03:02:33.120095+0000","last_clean":"2024-09-12T03:02:33.120095+0000","last_became_active":"2024-09-12T03:00:58.354760+0000","last_became_peered":"2024-09-12T03:00:58.354760+0000","last_unstale":"2024-09-12T03:02:33.120095+0000","last_undegraded":"2024-09-12T03:02:33.120095+0000","last_fullsized":"2024-09-12T03:02:33.120095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:13.464413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.112372+0000","last_change":"2024-09-12T03:00:58.341916+0000","last_active":"2024-09-12T03:02:33.112372+0000","last_peered":"2024-09-12T03:02:33.112372+0000","last_clean":"2024-09-12T03:02:33.112372+0000","last_became_active":"2024-09-12T03:00:58.340815+0000","last_became_peered":"2024-09-12T03:00:58.340815+0000","last_unstale":"2024-09-12T03:02:33.112372+0000","last_undegraded":"2024-09-12T03:02:33.112372+0000","last_fullsized":"2024-09-12T03:02:33.112372+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:13.777947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.117197+0000","last_change":"2024-09-12T03:00:58.357765+0000","last_active":"2024-09-12T03:02:33.117197+0000","last_peered":"2024-09-12T03:02:33.117197+0000","last_clean":"2024-09-12T03:02:33.117197+0000","last_became_active":"2024-09-12T03:00:58.357480+0000","last_became_peered":"2024-09-12T03:00:58.357480+0000","last_unstale":"2024-09-12T03:02:33.117197+0000","last_undegraded":"2024-09-12T03:02:33.117197+0000","last_fullsized":"2024-09-12T03:02:33.117197+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:26.884381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.108940+0000","last_change":"2024-09-12T03:00:58.349504+0000","last_active":"2024-09-12T03:02:33.108940+0000","last_peered":"2024-09-12T03:02:33.108940+0000","last_clean":"2024-09-12T03:02:33.108940+0000","last_became_active":"2024-09-12T03:00:58.349136+0000","last_became_peered":"2024-09-12T03:00:58.349136+0000","last_unstale":"2024-09-12T03:02:33.108940+0000","last_undegraded":"2024-09-12T03:02:33.108940+0000","last_fullsized":"2024-09-12T03:02:33.108940+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:09:55.229516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.123028+0000","last_change":"2024-09-12T03:00:58.341902+0000","last_active":"2024-09-12T03:02:33.123028+0000","last_peered":"2024-09-12T03:02:33.123028+0000","last_clean":"2024-09-12T03:02:33.123028+0000","last_became_active":"2024-09-12T03:00:58.341592+0000","last_became_peered":"2024-09-12T03:00:58.341592+0000","last_unstale":"2024-09-12T03:02:33.123028+0000","last_undegraded":"2024-09-12T03:02:33.123028+0000","last_fullsized":"2024-09-12T03:02:33.123028+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:13.852511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.114521+0000","last_change":"2024-09-12T03:00:58.348012+0000","last_active":"2024-09-12T03:02:33.114521+0000","last_peered":"2024-09-12T03:02:33.114521+0000","last_clean":"2024-09-12T03:02:33.114521+0000","last_became_active":"2024-09-12T03:00:58.347338+0000","last_became_peered":"2024-09-12T03:00:58.347338+0000","last_unstale":"2024-09-12T03:02:33.114521+0000","last_undegraded":"2024-09-12T03:02:33.114521+0000","last_fullsized":"2024-09-12T03:02:33.114521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:41.478032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.113228+0000","last_change":"2024-09-12T03:00:58.345833+0000","last_active":"2024-09-12T03:02:33.113228+0000","last_peered":"2024-09-12T03:02:33.113228+0000","last_clean":"2024-09-12T03:02:33.113228+0000","last_became_active":"2024-09-12T03:00:58.345482+0000","last_became_peered":"2024-09-12T03:00:58.345482+0000","last_unstale":"2024-09-12T03:02:33.113228+0000","last_undegraded":"2024-09-12T03:02:33.113228+0000","last_fullsized":"2024-09-12T03:02:33.113228+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:32.037412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.112949+0000","last_change":"2024-09-12T03:00:58.352290+0000","last_active":"2024-09-12T03:02:33.112949+0000","last_peered":"2024-09-12T03:02:33.112949+0000","last_clean":"2024-09-12T03:02:33.112949+0000","last_became_active":"2024-09-12T03:00:58.351918+0000","last_became_peered":"2024-09-12T03:00:58.351918+0000","last_unstale":"2024-09-12T03:02:33.112949+0000","last_undegraded":"2024-09-12T03:02:33.112949+0000","last_fullsized":"2024-09-12T03:02:33.112949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:25.941080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.116096+0000","last_change":"2024-09-12T03:00:58.348021+0000","last_active":"2024-09-12T03:02:33.116096+0000","last_peered":"2024-09-12T03:02:33.116096+0000","last_clean":"2024-09-12T03:02:33.116096+0000","last_became_active":"2024-09-12T03:00:58.347634+0000","last_became_peered":"2024-09-12T03:00:58.347634+0000","last_unstale":"2024-09-12T03:02:33.116096+0000","last_undegraded":"2024-09-12T03:02:33.116096+0000","last_fullsized":"2024-09-12T03:02:33.116096+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:47.933067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.123517+0000","last_change":"2024-09-12T03:00:58.340891+0000","last_active":"2024-09-12T03:02:33.123517+0000","last_peered":"2024-09-12T03:02:33.123517+0000","last_clean":"2024-09-12T03:02:33.123517+0000","last_became_active":"2024-09-12T03:00:58.340504+0000","last_became_peered":"2024-09-12T03:00:58.340504+0000","last_unstale":"2024-09-12T03:02:33.123517+0000","last_undegraded":"2024-09-12T03:02:33.123517+0000","last_fullsized":"2024-09-12T03:02:33.123517+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:27:32.117635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.108754+0000","last_change":"2024-09-12T03:00:58.339343+0000","last_active":"2024-09-12T03:02:33.108754+0000","last_peered":"2024-09-12T03:02:33.108754+0000","last_clean":"2024-09-12T03:02:33.108754+0000","last_became_active":"2024-09-12T03:00:58.339001+0000","last_became_peered":"2024-09-12T03:00:58.339001+0000","last_unstale":"2024-09-12T03:02:33.108754+0000","last_undegraded":"2024-09-12T03:02:33.108754+0000","last_fullsized":"2024-09-12T03:02:33.108754+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:02.043454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":150,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:03:07.578429+0000","last_change":"2024-09-12T03:00:56.315550+0000","last_active":"2024-09-12T03:03:07.578429+0000","last_peered":"2024-09-12T03:03:07.578429+0000","last_clean":"2024-09-12T03:03:07.578429+0000","last_became_active":"2024-09-12T03:00:56.315225+0000","last_became_peered":"2024-09-12T03:00:56.315225+0000","last_unstale":"2024-09-12T03:03:07.578429+0000","last_undegraded":"2024-09-12T03:03:07.578429+0000","last_fullsized":"2024-09-12T03:03:07.578429+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:46.070895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.116508+0000","last_change":"2024-09-12T03:02:04.290498+0000","last_active":"2024-09-12T03:02:33.116508+0000","last_peered":"2024-09-12T03:02:33.116508+0000","last_clean":"2024-09-12T03:02:33.116508+0000","last_became_active":"2024-09-12T03:02:04.290162+0000","last_became_peered":"2024-09-12T03:02:04.290162+0000","last_unstale":"2024-09-12T03:02:33.116508+0000","last_undegraded":"2024-09-12T03:02:33.116508+0000","last_fullsized":"2024-09-12T03:02:33.116508+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:29.777155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282566+0000","last_change":"2024-09-12T03:00:56.316385+0000","last_active":"2024-09-12T03:02:04.282566+0000","last_peered":"2024-09-12T03:02:04.282566+0000","last_clean":"2024-09-12T03:02:04.282566+0000","last_became_active":"2024-09-12T03:00:56.316064+0000","last_became_peered":"2024-09-12T03:00:56.316064+0000","last_unstale":"2024-09-12T03:02:04.282566+0000","last_undegraded":"2024-09-12T03:02:04.282566+0000","last_fullsized":"2024-09-12T03:02:04.282566+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:12:43.323654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332424+0000","last_change":"2024-09-12T03:00:56.313065+0000","last_active":"2024-09-12T03:02:04.332424+0000","last_peered":"2024-09-12T03:02:04.332424+0000","last_clean":"2024-09-12T03:02:04.332424+0000","last_became_active":"2024-09-12T03:00:56.312716+0000","last_became_peered":"2024-09-12T03:00:56.312716+0000","last_unstale":"2024-09-12T03:02:04.332424+0000","last_undegraded":"2024-09-12T03:02:04.332424+0000","last_fullsized":"2024-09-12T03:02:04.332424+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:32:51.089089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120152+0000","last_change":"2024-09-12T03:00:58.346542+0000","last_active":"2024-09-12T03:02:33.120152+0000","last_peered":"2024-09-12T03:02:33.120152+0000","last_clean":"2024-09-12T03:02:33.120152+0000","last_became_active":"2024-09-12T03:00:58.346260+0000","last_became_peered":"2024-09-12T03:00:58.346260+0000","last_unstale":"2024-09-12T03:02:33.120152+0000","last_undegraded":"2024-09-12T03:02:33.120152+0000","last_fullsized":"2024-09-12T03:02:33.120152+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:37.034969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289182+0000","last_change":"2024-09-12T03:00:56.317308+0000","last_active":"2024-09-12T03:02:04.289182+0000","last_peered":"2024-09-12T03:02:04.289182+0000","last_clean":"2024-09-12T03:02:04.289182+0000","last_became_active":"2024-09-12T03:00:56.316935+0000","last_became_peered":"2024-09-12T03:00:56.316935+0000","last_unstale":"2024-09-12T03:02:04.289182+0000","last_undegraded":"2024-09-12T03:02:04.289182+0000","last_fullsized":"2024-09-12T03:02:04.289182+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:15:06.324489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.118033+0000","last_change":"2024-09-12T03:00:58.341565+0000","last_active":"2024-09-12T03:02:33.118033+0000","last_peered":"2024-09-12T03:02:33.118033+0000","last_clean":"2024-09-12T03:02:33.118033+0000","last_became_active":"2024-09-12T03:00:58.341285+0000","last_became_peered":"2024-09-12T03:00:58.341285+0000","last_unstale":"2024-09-12T03:02:33.118033+0000","last_undegraded":"2024-09-12T03:02:33.118033+0000","last_fullsized":"2024-09-12T03:02:33.118033+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:28:59.156700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":148,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:03:09.580304+0000","last_change":"2024-09-12T03:00:56.314201+0000","last_active":"2024-09-12T03:03:09.580304+0000","last_peered":"2024-09-12T03:03:09.580304+0000","last_clean":"2024-09-12T03:03:09.580304+0000","last_became_active":"2024-09-12T03:00:56.313880+0000","last_became_peered":"2024-09-12T03:00:56.313880+0000","last_unstale":"2024-09-12T03:03:09.580304+0000","last_undegraded":"2024-09-12T03:03:09.580304+0000","last_fullsized":"2024-09-12T03:03:09.580304+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:57:12.952630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":123,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.132547+0000","last_change":"2024-09-12T03:00:58.345274+0000","last_active":"2024-09-12T03:02:33.132547+0000","last_peered":"2024-09-12T03:02:33.132547+0000","last_clean":"2024-09-12T03:02:33.132547+0000","last_became_active":"2024-09-12T03:00:58.344937+0000","last_became_peered":"2024-09-12T03:00:58.344937+0000","last_unstale":"2024-09-12T03:02:33.132547+0000","last_undegraded":"2024-09-12T03:02:33.132547+0000","last_fullsized":"2024-09-12T03:02:33.132547+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:17.288909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284614+0000","last_change":"2024-09-12T03:00:56.314709+0000","last_active":"2024-09-12T03:02:04.284614+0000","last_peered":"2024-09-12T03:02:04.284614+0000","last_clean":"2024-09-12T03:02:04.284614+0000","last_became_active":"2024-09-12T03:00:56.314349+0000","last_became_peered":"2024-09-12T03:00:56.314349+0000","last_unstale":"2024-09-12T03:02:04.284614+0000","last_undegraded":"2024-09-12T03:02:04.284614+0000","last_fullsized":"2024-09-12T03:02:04.284614+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:18:35.371661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.113713+0000","last_change":"2024-09-12T03:00:58.346055+0000","last_active":"2024-09-12T03:02:33.113713+0000","last_peered":"2024-09-12T03:02:33.113713+0000","last_clean":"2024-09-12T03:02:33.113713+0000","last_became_active":"2024-09-12T03:00:58.345701+0000","last_became_peered":"2024-09-12T03:00:58.345701+0000","last_unstale":"2024-09-12T03:02:33.113713+0000","last_undegraded":"2024-09-12T03:02:33.113713+0000","last_fullsized":"2024-09-12T03:02:33.113713+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:57:33.977245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.287450+0000","last_change":"2024-09-12T03:00:56.435962+0000","last_active":"2024-09-12T03:02:04.287450+0000","last_peered":"2024-09-12T03:02:04.287450+0000","last_clean":"2024-09-12T03:02:04.287450+0000","last_became_active":"2024-09-12T03:00:56.435515+0000","last_became_peered":"2024-09-12T03:00:56.435515+0000","last_unstale":"2024-09-12T03:02:04.287450+0000","last_undegraded":"2024-09-12T03:02:04.287450+0000","last_fullsized":"2024-09-12T03:02:04.287450+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:32.953436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.135045+0000","last_change":"2024-09-12T03:00:58.344239+0000","last_active":"2024-09-12T03:02:33.135045+0000","last_peered":"2024-09-12T03:02:33.135045+0000","last_clean":"2024-09-12T03:02:33.135045+0000","last_became_active":"2024-09-12T03:00:58.343876+0000","last_became_peered":"2024-09-12T03:00:58.343876+0000","last_unstale":"2024-09-12T03:02:33.135045+0000","last_undegraded":"2024-09-12T03:02:33.135045+0000","last_fullsized":"2024-09-12T03:02:33.135045+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:49.401295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'36","reported_seq":318,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:03:10.581673+0000","last_change":"2024-09-12T03:00:56.317708+0000","last_active":"2024-09-12T03:03:10.581673+0000","last_peered":"2024-09-12T03:03:10.581673+0000","last_clean":"2024-09-12T03:03:10.581673+0000","last_became_active":"2024-09-12T03:00:56.317317+0000","last_became_peered":"2024-09-12T03:00:56.317317+0000","last_unstale":"2024-09-12T03:03:10.581673+0000","last_undegraded":"2024-09-12T03:03:10.581673+0000","last_fullsized":"2024-09-12T03:03:10.581673+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:47.788186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70595,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":260,"num_read_kb":199,"num_write":37,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.123764+0000","last_change":"2024-09-12T03:00:58.343602+0000","last_active":"2024-09-12T03:02:33.123764+0000","last_peered":"2024-09-12T03:02:33.123764+0000","last_clean":"2024-09-12T03:02:33.123764+0000","last_became_active":"2024-09-12T03:00:58.343301+0000","last_became_peered":"2024-09-12T03:00:58.343301+0000","last_unstale":"2024-09-12T03:02:33.123764+0000","last_undegraded":"2024-09-12T03:02:33.123764+0000","last_fullsized":"2024-09-12T03:02:33.123764+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:24:04.056118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.290204+0000","last_change":"2024-09-12T03:00:56.317874+0000","last_active":"2024-09-12T03:02:04.290204+0000","last_peered":"2024-09-12T03:02:04.290204+0000","last_clean":"2024-09-12T03:02:04.290204+0000","last_became_active":"2024-09-12T03:00:56.317566+0000","last_became_peered":"2024-09-12T03:00:56.317566+0000","last_unstale":"2024-09-12T03:02:04.290204+0000","last_undegraded":"2024-09-12T03:02:04.290204+0000","last_fullsized":"2024-09-12T03:02:04.290204+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:54.203730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.121582+0000","last_change":"2024-09-12T03:00:58.351539+0000","last_active":"2024-09-12T03:02:33.121582+0000","last_peered":"2024-09-12T03:02:33.121582+0000","last_clean":"2024-09-12T03:02:33.121582+0000","last_became_active":"2024-09-12T03:00:58.351249+0000","last_became_peered":"2024-09-12T03:00:58.351249+0000","last_unstale":"2024-09-12T03:02:33.121582+0000","last_undegraded":"2024-09-12T03:02:33.121582+0000","last_fullsized":"2024-09-12T03:02:33.121582+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:40.498867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.287068+0000","last_change":"2024-09-12T03:00:56.314489+0000","last_active":"2024-09-12T03:02:04.287068+0000","last_peered":"2024-09-12T03:02:04.287068+0000","last_clean":"2024-09-12T03:02:04.287068+0000","last_became_active":"2024-09-12T03:00:56.314142+0000","last_became_peered":"2024-09-12T03:00:56.314142+0000","last_unstale":"2024-09-12T03:02:04.287068+0000","last_undegraded":"2024-09-12T03:02:04.287068+0000","last_fullsized":"2024-09-12T03:02:04.287068+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:21.541713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.123797+0000","last_change":"2024-09-12T03:00:58.348752+0000","last_active":"2024-09-12T03:02:33.123797+0000","last_peered":"2024-09-12T03:02:33.123797+0000","last_clean":"2024-09-12T03:02:33.123797+0000","last_became_active":"2024-09-12T03:00:58.348391+0000","last_became_peered":"2024-09-12T03:00:58.348391+0000","last_unstale":"2024-09-12T03:02:33.123797+0000","last_undegraded":"2024-09-12T03:02:33.123797+0000","last_fullsized":"2024-09-12T03:02:33.123797+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:33:15.626306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'4","reported_seq":151,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:03:08.579245+0000","last_change":"2024-09-12T03:00:56.311702+0000","last_active":"2024-09-12T03:03:08.579245+0000","last_peered":"2024-09-12T03:03:08.579245+0000","last_clean":"2024-09-12T03:03:08.579245+0000","last_became_active":"2024-09-12T03:00:56.311353+0000","last_became_peered":"2024-09-12T03:00:56.311353+0000","last_unstale":"2024-09-12T03:03:08.579245+0000","last_undegraded":"2024-09-12T03:03:08.579245+0000","last_fullsized":"2024-09-12T03:03:08.579245+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:01.004858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.123076+0000","last_change":"2024-09-12T03:00:58.351973+0000","last_active":"2024-09-12T03:02:33.123076+0000","last_peered":"2024-09-12T03:02:33.123076+0000","last_clean":"2024-09-12T03:02:33.123076+0000","last_became_active":"2024-09-12T03:00:58.351599+0000","last_became_peered":"2024-09-12T03:00:58.351599+0000","last_unstale":"2024-09-12T03:02:33.123076+0000","last_undegraded":"2024-09-12T03:02:33.123076+0000","last_fullsized":"2024-09-12T03:02:33.123076+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:26.398828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.291749+0000","last_change":"2024-09-12T03:00:56.436185+0000","last_active":"2024-09-12T03:02:04.291749+0000","last_peered":"2024-09-12T03:02:04.291749+0000","last_clean":"2024-09-12T03:02:04.291749+0000","last_became_active":"2024-09-12T03:00:56.435717+0000","last_became_peered":"2024-09-12T03:00:56.435717+0000","last_unstale":"2024-09-12T03:02:04.291749+0000","last_undegraded":"2024-09-12T03:02:04.291749+0000","last_fullsized":"2024-09-12T03:02:04.291749+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:27.303182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.119766+0000","last_change":"2024-09-12T03:02:04.333661+0000","last_active":"2024-09-12T03:02:33.119766+0000","last_peered":"2024-09-12T03:02:33.119766+0000","last_clean":"2024-09-12T03:02:33.119766+0000","last_became_active":"2024-09-12T03:02:04.333217+0000","last_became_peered":"2024-09-12T03:02:04.333217+0000","last_unstale":"2024-09-12T03:02:33.119766+0000","last_undegraded":"2024-09-12T03:02:33.119766+0000","last_fullsized":"2024-09-12T03:02:33.119766+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:23:47.212734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'3","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:08.214796+0000","last_change":"2024-09-12T03:02:04.296884+0000","last_active":"2024-09-12T03:02:08.214796+0000","last_peered":"2024-09-12T03:02:08.214796+0000","last_clean":"2024-09-12T03:02:08.214796+0000","last_became_active":"2024-09-12T03:02:04.290894+0000","last_became_peered":"2024-09-12T03:02:04.290894+0000","last_unstale":"2024-09-12T03:02:08.214796+0000","last_undegraded":"2024-09-12T03:02:08.214796+0000","last_fullsized":"2024-09-12T03:02:08.214796+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:49:19.749568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.118228+0000","last_change":"2024-09-12T03:02:04.334096+0000","last_active":"2024-09-12T03:02:33.118228+0000","last_peered":"2024-09-12T03:02:33.118228+0000","last_clean":"2024-09-12T03:02:33.118228+0000","last_became_active":"2024-09-12T03:02:04.333641+0000","last_became_peered":"2024-09-12T03:02:04.333641+0000","last_unstale":"2024-09-12T03:02:33.118228+0000","last_undegraded":"2024-09-12T03:02:33.118228+0000","last_fullsized":"2024-09-12T03:02:33.118228+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:25:24.085293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332966+0000","last_change":"2024-09-12T03:00:56.314182+0000","last_active":"2024-09-12T03:02:04.332966+0000","last_peered":"2024-09-12T03:02:04.332966+0000","last_clean":"2024-09-12T03:02:04.332966+0000","last_became_active":"2024-09-12T03:00:56.313749+0000","last_became_peered":"2024-09-12T03:00:56.313749+0000","last_unstale":"2024-09-12T03:02:04.332966+0000","last_undegraded":"2024-09-12T03:02:04.332966+0000","last_fullsized":"2024-09-12T03:02:04.332966+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:31:45.767500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.125121+0000","last_change":"2024-09-12T03:00:58.344356+0000","last_active":"2024-09-12T03:02:33.125121+0000","last_peered":"2024-09-12T03:02:33.125121+0000","last_clean":"2024-09-12T03:02:33.125121+0000","last_became_active":"2024-09-12T03:00:58.344035+0000","last_became_peered":"2024-09-12T03:00:58.344035+0000","last_unstale":"2024-09-12T03:02:33.125121+0000","last_undegraded":"2024-09-12T03:02:33.125121+0000","last_fullsized":"2024-09-12T03:02:33.125121+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:45:18.681018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.287585+0000","last_change":"2024-09-12T03:00:56.315653+0000","last_active":"2024-09-12T03:02:04.287585+0000","last_peered":"2024-09-12T03:02:04.287585+0000","last_clean":"2024-09-12T03:02:04.287585+0000","last_became_active":"2024-09-12T03:00:56.315284+0000","last_became_peered":"2024-09-12T03:00:56.315284+0000","last_unstale":"2024-09-12T03:02:04.287585+0000","last_undegraded":"2024-09-12T03:02:04.287585+0000","last_fullsized":"2024-09-12T03:02:04.287585+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:42.934020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.122059+0000","last_change":"2024-09-12T03:02:04.291128+0000","last_active":"2024-09-12T03:02:33.122059+0000","last_peered":"2024-09-12T03:02:33.122059+0000","last_clean":"2024-09-12T03:02:33.122059+0000","last_became_active":"2024-09-12T03:02:04.287836+0000","last_became_peered":"2024-09-12T03:02:04.287836+0000","last_unstale":"2024-09-12T03:02:33.122059+0000","last_undegraded":"2024-09-12T03:02:33.122059+0000","last_fullsized":"2024-09-12T03:02:33.122059+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:23.956579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.290577+0000","last_change":"2024-09-12T03:00:56.746780+0000","last_active":"2024-09-12T03:02:04.290577+0000","last_peered":"2024-09-12T03:02:04.290577+0000","last_clean":"2024-09-12T03:02:04.290577+0000","last_became_active":"2024-09-12T03:00:56.746414+0000","last_became_peered":"2024-09-12T03:00:56.746414+0000","last_unstale":"2024-09-12T03:02:04.290577+0000","last_undegraded":"2024-09-12T03:02:04.290577+0000","last_fullsized":"2024-09-12T03:02:04.290577+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:15:35.214196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.131100+0000","last_change":"2024-09-12T03:00:58.351713+0000","last_active":"2024-09-12T03:02:33.131100+0000","last_peered":"2024-09-12T03:02:33.131100+0000","last_clean":"2024-09-12T03:02:33.131100+0000","last_became_active":"2024-09-12T03:00:58.351371+0000","last_became_peered":"2024-09-12T03:00:58.351371+0000","last_unstale":"2024-09-12T03:02:33.131100+0000","last_undegraded":"2024-09-12T03:02:33.131100+0000","last_fullsized":"2024-09-12T03:02:33.131100+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:13.346956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'137","reported_seq":336,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284318+0000","last_change":"2024-09-12T02:59:45.461482+0000","last_active":"2024-09-12T03:02:04.284318+0000","last_peered":"2024-09-12T03:02:04.284318+0000","last_clean":"2024-09-12T03:02:04.284318+0000","last_became_active":"2024-09-12T02:59:45.461058+0000","last_became_peered":"2024-09-12T02:59:45.461058+0000","last_unstale":"2024-09-12T03:02:04.284318+0000","last_undegraded":"2024-09-12T03:02:04.284318+0000","last_fullsized":"2024-09-12T03:02:04.284318+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:18.034438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:18.034438+0000","last_clean_scrub_stamp":"2024-09-12T02:56:18.034438+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:05:09.251858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289360+0000","last_change":"2024-09-12T03:00:56.315245+0000","last_active":"2024-09-12T03:02:04.289360+0000","last_peered":"2024-09-12T03:02:04.289360+0000","last_clean":"2024-09-12T03:02:04.289360+0000","last_became_active":"2024-09-12T03:00:56.314887+0000","last_became_peered":"2024-09-12T03:00:56.314887+0000","last_unstale":"2024-09-12T03:02:04.289360+0000","last_undegraded":"2024-09-12T03:02:04.289360+0000","last_fullsized":"2024-09-12T03:02:04.289360+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:48.750657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.113189+0000","last_change":"2024-09-12T03:00:58.357874+0000","last_active":"2024-09-12T03:02:33.113189+0000","last_peered":"2024-09-12T03:02:33.113189+0000","last_clean":"2024-09-12T03:02:33.113189+0000","last_became_active":"2024-09-12T03:00:58.357575+0000","last_became_peered":"2024-09-12T03:00:58.357575+0000","last_unstale":"2024-09-12T03:02:33.113189+0000","last_undegraded":"2024-09-12T03:02:33.113189+0000","last_fullsized":"2024-09-12T03:02:33.113189+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:46.125620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284282+0000","last_change":"2024-09-12T03:00:56.312263+0000","last_active":"2024-09-12T03:02:04.284282+0000","last_peered":"2024-09-12T03:02:04.284282+0000","last_clean":"2024-09-12T03:02:04.284282+0000","last_became_active":"2024-09-12T03:00:56.311894+0000","last_became_peered":"2024-09-12T03:00:56.311894+0000","last_unstale":"2024-09-12T03:02:04.284282+0000","last_undegraded":"2024-09-12T03:02:04.284282+0000","last_fullsized":"2024-09-12T03:02:04.284282+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:52.434203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.114489+0000","last_change":"2024-09-12T03:00:58.350762+0000","last_active":"2024-09-12T03:02:33.114489+0000","last_peered":"2024-09-12T03:02:33.114489+0000","last_clean":"2024-09-12T03:02:33.114489+0000","last_became_active":"2024-09-12T03:00:58.348038+0000","last_became_peered":"2024-09-12T03:00:58.348038+0000","last_unstale":"2024-09-12T03:02:33.114489+0000","last_undegraded":"2024-09-12T03:02:33.114489+0000","last_fullsized":"2024-09-12T03:02:33.114489+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:08.916636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":146,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:03:04.573615+0000","last_change":"2024-09-12T03:00:56.435641+0000","last_active":"2024-09-12T03:03:04.573615+0000","last_peered":"2024-09-12T03:03:04.573615+0000","last_clean":"2024-09-12T03:03:04.573615+0000","last_became_active":"2024-09-12T03:00:56.435319+0000","last_became_peered":"2024-09-12T03:00:56.435319+0000","last_unstale":"2024-09-12T03:03:04.573615+0000","last_undegraded":"2024-09-12T03:03:04.573615+0000","last_fullsized":"2024-09-12T03:03:04.573615+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:56.894978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.112818+0000","last_change":"2024-09-12T03:00:58.344708+0000","last_active":"2024-09-12T03:02:33.112818+0000","last_peered":"2024-09-12T03:02:33.112818+0000","last_clean":"2024-09-12T03:02:33.112818+0000","last_became_active":"2024-09-12T03:00:58.344389+0000","last_became_peered":"2024-09-12T03:00:58.344389+0000","last_unstale":"2024-09-12T03:02:33.112818+0000","last_undegraded":"2024-09-12T03:02:33.112818+0000","last_fullsized":"2024-09-12T03:02:33.112818+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:08.160383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284373+0000","last_change":"2024-09-12T03:00:56.317068+0000","last_active":"2024-09-12T03:02:04.284373+0000","last_peered":"2024-09-12T03:02:04.284373+0000","last_clean":"2024-09-12T03:02:04.284373+0000","last_became_active":"2024-09-12T03:00:56.316698+0000","last_became_peered":"2024-09-12T03:00:56.316698+0000","last_unstale":"2024-09-12T03:02:04.284373+0000","last_undegraded":"2024-09-12T03:02:04.284373+0000","last_fullsized":"2024-09-12T03:02:04.284373+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:41:19.199418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.110097+0000","last_change":"2024-09-12T03:00:58.347814+0000","last_active":"2024-09-12T03:02:33.110097+0000","last_peered":"2024-09-12T03:02:33.110097+0000","last_clean":"2024-09-12T03:02:33.110097+0000","last_became_active":"2024-09-12T03:00:58.347112+0000","last_became_peered":"2024-09-12T03:00:58.347112+0000","last_unstale":"2024-09-12T03:02:33.110097+0000","last_undegraded":"2024-09-12T03:02:33.110097+0000","last_fullsized":"2024-09-12T03:02:33.110097+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:59.325113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.284305+0000","last_change":"2024-09-12T03:00:56.435771+0000","last_active":"2024-09-12T03:02:04.284305+0000","last_peered":"2024-09-12T03:02:04.284305+0000","last_clean":"2024-09-12T03:02:04.284305+0000","last_became_active":"2024-09-12T03:00:56.435448+0000","last_became_peered":"2024-09-12T03:00:56.435448+0000","last_unstale":"2024-09-12T03:02:04.284305+0000","last_undegraded":"2024-09-12T03:02:04.284305+0000","last_fullsized":"2024-09-12T03:02:04.284305+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:59:33.650338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.118219+0000","last_change":"2024-09-12T03:00:58.341071+0000","last_active":"2024-09-12T03:02:33.118219+0000","last_peered":"2024-09-12T03:02:33.118219+0000","last_clean":"2024-09-12T03:02:33.118219+0000","last_became_active":"2024-09-12T03:00:58.340753+0000","last_became_peered":"2024-09-12T03:00:58.340753+0000","last_unstale":"2024-09-12T03:02:33.118219+0000","last_undegraded":"2024-09-12T03:02:33.118219+0000","last_fullsized":"2024-09-12T03:02:33.118219+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:21.217915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292316+0000","last_change":"2024-09-12T03:00:56.436058+0000","last_active":"2024-09-12T03:02:04.292316+0000","last_peered":"2024-09-12T03:02:04.292316+0000","last_clean":"2024-09-12T03:02:04.292316+0000","last_became_active":"2024-09-12T03:00:56.435518+0000","last_became_peered":"2024-09-12T03:00:56.435518+0000","last_unstale":"2024-09-12T03:02:04.292316+0000","last_undegraded":"2024-09-12T03:02:04.292316+0000","last_fullsized":"2024-09-12T03:02:04.292316+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:32.235097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.110699+0000","last_change":"2024-09-12T03:00:58.348236+0000","last_active":"2024-09-12T03:02:33.110699+0000","last_peered":"2024-09-12T03:02:33.110699+0000","last_clean":"2024-09-12T03:02:33.110699+0000","last_became_active":"2024-09-12T03:00:58.347727+0000","last_became_peered":"2024-09-12T03:00:58.347727+0000","last_unstale":"2024-09-12T03:02:33.110699+0000","last_undegraded":"2024-09-12T03:02:33.110699+0000","last_fullsized":"2024-09-12T03:02:33.110699+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:09:19.233163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'13","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:58.357683+0000","last_change":"2024-09-12T03:00:56.310537+0000","last_active":"2024-09-12T03:02:58.357683+0000","last_peered":"2024-09-12T03:02:58.357683+0000","last_clean":"2024-09-12T03:02:58.357683+0000","last_became_active":"2024-09-12T03:00:56.310135+0000","last_became_peered":"2024-09-12T03:00:56.310135+0000","last_unstale":"2024-09-12T03:02:58.357683+0000","last_undegraded":"2024-09-12T03:02:58.357683+0000","last_fullsized":"2024-09-12T03:02:58.357683+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:10:55.584339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.123219+0000","last_change":"2024-09-12T03:00:58.358417+0000","last_active":"2024-09-12T03:02:33.123219+0000","last_peered":"2024-09-12T03:02:33.123219+0000","last_clean":"2024-09-12T03:02:33.123219+0000","last_became_active":"2024-09-12T03:00:58.358119+0000","last_became_peered":"2024-09-12T03:00:58.358119+0000","last_unstale":"2024-09-12T03:02:33.123219+0000","last_undegraded":"2024-09-12T03:02:33.123219+0000","last_fullsized":"2024-09-12T03:02:33.123219+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:10.897594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.289583+0000","last_change":"2024-09-12T03:00:56.747044+0000","last_active":"2024-09-12T03:02:04.289583+0000","last_peered":"2024-09-12T03:02:04.289583+0000","last_clean":"2024-09-12T03:02:04.289583+0000","last_became_active":"2024-09-12T03:00:56.746699+0000","last_became_peered":"2024-09-12T03:00:56.746699+0000","last_unstale":"2024-09-12T03:02:04.289583+0000","last_undegraded":"2024-09-12T03:02:04.289583+0000","last_fullsized":"2024-09-12T03:02:04.289583+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:18:11.420109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.113393+0000","last_change":"2024-09-12T03:00:58.346854+0000","last_active":"2024-09-12T03:02:33.113393+0000","last_peered":"2024-09-12T03:02:33.113393+0000","last_clean":"2024-09-12T03:02:33.113393+0000","last_became_active":"2024-09-12T03:00:58.346535+0000","last_became_peered":"2024-09-12T03:00:58.346535+0000","last_unstale":"2024-09-12T03:02:33.113393+0000","last_undegraded":"2024-09-12T03:02:33.113393+0000","last_fullsized":"2024-09-12T03:02:33.113393+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:46.637914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.332423+0000","last_change":"2024-09-12T03:00:56.435957+0000","last_active":"2024-09-12T03:02:04.332423+0000","last_peered":"2024-09-12T03:02:04.332423+0000","last_clean":"2024-09-12T03:02:04.332423+0000","last_became_active":"2024-09-12T03:00:56.435603+0000","last_became_peered":"2024-09-12T03:00:56.435603+0000","last_unstale":"2024-09-12T03:02:04.332423+0000","last_undegraded":"2024-09-12T03:02:04.332423+0000","last_fullsized":"2024-09-12T03:02:04.332423+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:34.553189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.122738+0000","last_change":"2024-09-12T03:00:58.337784+0000","last_active":"2024-09-12T03:02:33.122738+0000","last_peered":"2024-09-12T03:02:33.122738+0000","last_clean":"2024-09-12T03:02:33.122738+0000","last_became_active":"2024-09-12T03:00:58.337415+0000","last_became_peered":"2024-09-12T03:00:58.337415+0000","last_unstale":"2024-09-12T03:02:33.122738+0000","last_undegraded":"2024-09-12T03:02:33.122738+0000","last_fullsized":"2024-09-12T03:02:33.122738+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:38:15.557067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.282398+0000","last_change":"2024-09-12T03:00:56.315356+0000","last_active":"2024-09-12T03:02:04.282398+0000","last_peered":"2024-09-12T03:02:04.282398+0000","last_clean":"2024-09-12T03:02:04.282398+0000","last_became_active":"2024-09-12T03:00:56.314873+0000","last_became_peered":"2024-09-12T03:00:56.314873+0000","last_unstale":"2024-09-12T03:02:04.282398+0000","last_undegraded":"2024-09-12T03:02:04.282398+0000","last_fullsized":"2024-09-12T03:02:04.282398+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:55.710156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120404+0000","last_change":"2024-09-12T03:00:58.346706+0000","last_active":"2024-09-12T03:02:33.120404+0000","last_peered":"2024-09-12T03:02:33.120404+0000","last_clean":"2024-09-12T03:02:33.120404+0000","last_became_active":"2024-09-12T03:00:58.346363+0000","last_became_peered":"2024-09-12T03:00:58.346363+0000","last_unstale":"2024-09-12T03:02:33.120404+0000","last_undegraded":"2024-09-12T03:02:33.120404+0000","last_fullsized":"2024-09-12T03:02:33.120404+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:39:13.468639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.299582+0000","last_change":"2024-09-12T03:02:04.299147+0000","last_active":"2024-09-12T03:02:04.299582+0000","last_peered":"2024-09-12T03:02:04.299582+0000","last_clean":"2024-09-12T03:02:04.299582+0000","last_became_active":"2024-09-12T03:02:04.293211+0000","last_became_peered":"2024-09-12T03:02:04.293211+0000","last_unstale":"2024-09-12T03:02:04.299582+0000","last_undegraded":"2024-09-12T03:02:04.299582+0000","last_fullsized":"2024-09-12T03:02:04.299582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:01:39.222476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.118849+0000","last_change":"2024-09-12T03:00:58.356682+0000","last_active":"2024-09-12T03:02:33.118849+0000","last_peered":"2024-09-12T03:02:33.118849+0000","last_clean":"2024-09-12T03:02:33.118849+0000","last_became_active":"2024-09-12T03:00:58.356383+0000","last_became_peered":"2024-09-12T03:00:58.356383+0000","last_unstale":"2024-09-12T03:02:33.118849+0000","last_undegraded":"2024-09-12T03:02:33.118849+0000","last_fullsized":"2024-09-12T03:02:33.118849+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:14:16.428393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292053+0000","last_change":"2024-09-12T03:00:56.314943+0000","last_active":"2024-09-12T03:02:04.292053+0000","last_peered":"2024-09-12T03:02:04.292053+0000","last_clean":"2024-09-12T03:02:04.292053+0000","last_became_active":"2024-09-12T03:00:56.314602+0000","last_became_peered":"2024-09-12T03:00:56.314602+0000","last_unstale":"2024-09-12T03:02:04.292053+0000","last_undegraded":"2024-09-12T03:02:04.292053+0000","last_fullsized":"2024-09-12T03:02:04.292053+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:41:31.207880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.131847+0000","last_change":"2024-09-12T03:00:58.351774+0000","last_active":"2024-09-12T03:02:33.131847+0000","last_peered":"2024-09-12T03:02:33.131847+0000","last_clean":"2024-09-12T03:02:33.131847+0000","last_became_active":"2024-09-12T03:00:58.351453+0000","last_became_peered":"2024-09-12T03:00:58.351453+0000","last_unstale":"2024-09-12T03:02:33.131847+0000","last_undegraded":"2024-09-12T03:02:33.131847+0000","last_fullsized":"2024-09-12T03:02:33.131847+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:18:24.260110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.130930+0000","last_change":"2024-09-12T03:00:58.346886+0000","last_active":"2024-09-12T03:02:33.130930+0000","last_peered":"2024-09-12T03:02:33.130930+0000","last_clean":"2024-09-12T03:02:33.130930+0000","last_became_active":"2024-09-12T03:00:58.346499+0000","last_became_peered":"2024-09-12T03:00:58.346499+0000","last_unstale":"2024-09-12T03:02:33.130930+0000","last_undegraded":"2024-09-12T03:02:33.130930+0000","last_fullsized":"2024-09-12T03:02:33.130930+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:02:24.045610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292497+0000","last_change":"2024-09-12T03:00:56.317443+0000","last_active":"2024-09-12T03:02:04.292497+0000","last_peered":"2024-09-12T03:02:04.292497+0000","last_clean":"2024-09-12T03:02:04.292497+0000","last_became_active":"2024-09-12T03:00:56.317045+0000","last_became_peered":"2024-09-12T03:00:56.317045+0000","last_unstale":"2024-09-12T03:02:04.292497+0000","last_undegraded":"2024-09-12T03:02:04.292497+0000","last_fullsized":"2024-09-12T03:02:04.292497+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:07.687606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.132782+0000","last_change":"2024-09-12T03:00:58.337257+0000","last_active":"2024-09-12T03:02:33.132782+0000","last_peered":"2024-09-12T03:02:33.132782+0000","last_clean":"2024-09-12T03:02:33.132782+0000","last_became_active":"2024-09-12T03:00:58.336847+0000","last_became_peered":"2024-09-12T03:00:58.336847+0000","last_unstale":"2024-09-12T03:02:33.132782+0000","last_undegraded":"2024-09-12T03:02:33.132782+0000","last_fullsized":"2024-09-12T03:02:33.132782+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:22.846206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.292165+0000","last_change":"2024-09-12T03:00:56.316255+0000","last_active":"2024-09-12T03:02:04.292165+0000","last_peered":"2024-09-12T03:02:04.292165+0000","last_clean":"2024-09-12T03:02:04.292165+0000","last_became_active":"2024-09-12T03:00:56.315906+0000","last_became_peered":"2024-09-12T03:00:56.315906+0000","last_unstale":"2024-09-12T03:02:04.292165+0000","last_undegraded":"2024-09-12T03:02:04.292165+0000","last_fullsized":"2024-09-12T03:02:04.292165+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:14.798601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.120352+0000","last_change":"2024-09-12T03:00:58.347206+0000","last_active":"2024-09-12T03:02:33.120352+0000","last_peered":"2024-09-12T03:02:33.120352+0000","last_clean":"2024-09-12T03:02:33.120352+0000","last_became_active":"2024-09-12T03:00:58.346923+0000","last_became_peered":"2024-09-12T03:00:58.346923+0000","last_unstale":"2024-09-12T03:02:33.120352+0000","last_undegraded":"2024-09-12T03:02:33.120352+0000","last_fullsized":"2024-09-12T03:02:33.120352+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:41.376211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:04.283743+0000","last_change":"2024-09-12T03:00:56.435431+0000","last_active":"2024-09-12T03:02:04.283743+0000","last_peered":"2024-09-12T03:02:04.283743+0000","last_clean":"2024-09-12T03:02:04.283743+0000","last_became_active":"2024-09-12T03:00:56.435088+0000","last_became_peered":"2024-09-12T03:00:56.435088+0000","last_unstale":"2024-09-12T03:02:04.283743+0000","last_undegraded":"2024-09-12T03:02:04.283743+0000","last_fullsized":"2024-09-12T03:02:04.283743+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:24.738340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:02:33.114461+0000","last_change":"2024-09-12T03:00:58.351737+0000","last_active":"2024-09-12T03:02:33.114461+0000","last_peered":"2024-09-12T03:02:33.114461+0000","last_clean":"2024-09-12T03:02:33.114461+0000","last_became_active":"2024-09-12T03:00:58.351407+0000","last_became_peered":"2024-09-12T03:00:58.351407+0000","last_unstale":"2024-09-12T03:02:33.114461+0000","last_undegraded":"2024-09-12T03:02:33.114461+0000","last_fullsized":"2024-09-12T03:02:33.114461+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:57.310000+0000","last_clean_scrub_stamp":"2024-09-12T03:00:57.310000+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:34.206703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:03:02.121967+0000","last_change":"2024-09-12T03:00:56.310910+0000","last_active":"2024-09-12T03:03:02.121967+0000","last_peered":"2024-09-12T03:03:02.121967+0000","last_clean":"2024-09-12T03:03:02.121967+0000","last_became_active":"2024-09-12T03:00:56.310657+0000","last_became_peered":"2024-09-12T03:00:56.310657+0000","last_unstale":"2024-09-12T03:03:02.121967+0000","last_undegraded":"2024-09-12T03:03:02.121967+0000","last_fullsized":"2024-09-12T03:03:02.121967+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:00:55.295051+0000","last_clean_scrub_stamp":"2024-09-12T03:00:55.295051+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:06.117215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74598,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":763,"num_read_kb":459,"num_write":178,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":223794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":245899,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645352,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":431835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906989,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":1094282,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070514,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":432166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299448,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":431947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528380,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":1023901,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":40,"seq":171798691905,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":880640,"data_stored":431438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":33,"seq":141733920838,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":1023192,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051660,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":432268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280592,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":431632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476821,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":503176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607643,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":501826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":432709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34193,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35830,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":70595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":71945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":72566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:03:12.824 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:03:13.075 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:13.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:13 smithi183 ceph-mon[35535]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:03:13.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:13 smithi177 ceph-mon[35305]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:03:13.496 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:13 smithi116 ceph-mon[31410]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:03:13.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.575+0000 7fc2c8ed6640 1 -- 172.21.15.116:0/416218611 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2c4102d40 msgr2=0x7fc2c4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:13.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.575+0000 7fc2c8ed6640 1 --2- 172.21.15.116:0/416218611 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2c4102d40 0x7fc2c4103120 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4009900 tx=0x7fc2b402a480 comp rx=0 tx=0).stop 2024-09-12T03:03:13.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.575+0000 7fc2c8ed6640 1 -- 172.21.15.116:0/416218611 shutdown_connections 2024-09-12T03:03:13.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.575+0000 7fc2c8ed6640 1 --2- 172.21.15.116:0/416218611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2c410c170 0x7fc2c410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:13.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.575+0000 7fc2c8ed6640 1 --2- 172.21.15.116:0/416218611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2c4103660 0x7fc2c410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:13.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.575+0000 7fc2c8ed6640 1 --2- 172.21.15.116:0/416218611 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2c4102d40 0x7fc2c4103120 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:13.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.576+0000 7fc2c8ed6640 1 -- 172.21.15.116:0/416218611 >> 172.21.15.116:0/416218611 conn(0x7fc2c40fc7a0 msgr2=0x7fc2c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:13.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.576+0000 7fc2c8ed6640 1 -- 172.21.15.116:0/416218611 shutdown_connections 2024-09-12T03:03:13.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.576+0000 7fc2c8ed6640 1 -- 172.21.15.116:0/416218611 wait complete. 2024-09-12T03:03:13.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.576+0000 7fc2c8ed6640 1 Processor -- start 2024-09-12T03:03:13.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.576+0000 7fc2c8ed6640 1 -- start start 2024-09-12T03:03:13.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.577+0000 7fc2c8ed6640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2c4102d40 0x7fc2c4198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:13.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.577+0000 7fc2c8ed6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2c4103660 0x7fc2c41989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:13.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.577+0000 7fc2c8ed6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2c410c170 0x7fc2c419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:13.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.577+0000 7fc2c8ed6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c4111fb0 con 0x7fc2c410c170 2024-09-12T03:03:13.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.577+0000 7fc2c8ed6640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c4111e30 con 0x7fc2c4102d40 2024-09-12T03:03:13.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.577+0000 7fc2c8ed6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c4112130 con 0x7fc2c4103660 2024-09-12T03:03:13.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.577+0000 7fc2c2d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2c410c170 0x7fc2c419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:13.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.577+0000 7fc2c2d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2c410c170 0x7fc2c419cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43536/0 (socket says 172.21.15.116:43536) 2024-09-12T03:03:13.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.577+0000 7fc2c2d9d640 1 -- 172.21.15.116:0/4011019071 learned_addr learned my addr 172.21.15.116:0/4011019071 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:13.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.577+0000 7fc2c259c640 1 --2- 172.21.15.116:0/4011019071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2c4102d40 0x7fc2c4198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:13.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.577+0000 7fc2c1d9b640 1 --2- 172.21.15.116:0/4011019071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2c4103660 0x7fc2c41989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:13.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.577+0000 7fc2c2d9d640 1 -- 172.21.15.116:0/4011019071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2c4103660 msgr2=0x7fc2c41989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:13.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.577+0000 7fc2c2d9d640 1 --2- 172.21.15.116:0/4011019071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2c4103660 0x7fc2c41989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:13.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.577+0000 7fc2c2d9d640 1 -- 172.21.15.116:0/4011019071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2c4102d40 msgr2=0x7fc2c4198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:13.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.577+0000 7fc2c2d9d640 1 --2- 172.21.15.116:0/4011019071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2c4102d40 0x7fc2c4198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:13.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.577+0000 7fc2c2d9d640 1 -- 172.21.15.116:0/4011019071 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c419d420 con 0x7fc2c410c170 2024-09-12T03:03:13.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.577+0000 7fc2c2d9d640 1 --2- 172.21.15.116:0/4011019071 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2c410c170 0x7fc2c419cd40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00b970 tx=0x7fc2bc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:13.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.578+0000 7fc2b37fe640 1 -- 172.21.15.116:0/4011019071 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc019070 con 0x7fc2c410c170 2024-09-12T03:03:13.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.578+0000 7fc2c8ed6640 1 -- 172.21.15.116:0/4011019071 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2c419d710 con 0x7fc2c410c170 2024-09-12T03:03:13.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.578+0000 7fc2b37fe640 1 -- 172.21.15.116:0/4011019071 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2bc00cb40 con 0x7fc2c410c170 2024-09-12T03:03:13.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.578+0000 7fc2b37fe640 1 -- 172.21.15.116:0/4011019071 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc007610 con 0x7fc2c410c170 2024-09-12T03:03:13.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.578+0000 7fc2c8ed6640 1 -- 172.21.15.116:0/4011019071 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2c41a4ff0 con 0x7fc2c410c170 2024-09-12T03:03:13.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.579+0000 7fc2c8ed6640 1 -- 172.21.15.116:0/4011019071 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2c4103120 con 0x7fc2c410c170 2024-09-12T03:03:13.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.580+0000 7fc2b37fe640 1 -- 172.21.15.116:0/4011019071 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fc2bc01c070 con 0x7fc2c410c170 2024-09-12T03:03:13.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.580+0000 7fc2b37fe640 1 --2- 172.21.15.116:0/4011019071 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc2ac0a2c10 0x7fc2ac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:13.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.580+0000 7fc2c259c640 1 --2- 172.21.15.116:0/4011019071 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc2ac0a2c10 0x7fc2ac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:13.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.580+0000 7fc2b37fe640 1 -- 172.21.15.116:0/4011019071 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fc2bc0ce600 con 0x7fc2c410c170 2024-09-12T03:03:13.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.581+0000 7fc2c259c640 1 --2- 172.21.15.116:0/4011019071 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc2ac0a2c10 0x7fc2ac0a50d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4009230 tx=0x7fc2b40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:13.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.584+0000 7fc2b37fe640 1 -- 172.21.15.116:0/4011019071 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc2bc09aef0 con 0x7fc2c410c170 2024-09-12T03:03:13.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.788+0000 7fc2c8ed6640 1 -- 172.21.15.116:0/4011019071 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2c402d7c0 con 0x7fc2c410c170 2024-09-12T03:03:13.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.789+0000 7fc2b37fe640 1 -- 172.21.15.116:0/4011019071 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7fc2bc09f1f0 con 0x7fc2c410c170 2024-09-12T03:03:13.789 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:03:13.789 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:03:13.790 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:03:13.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.791+0000 7fc2c8ed6640 1 -- 172.21.15.116:0/4011019071 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc2ac0a2c10 msgr2=0x7fc2ac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:13.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.791+0000 7fc2c8ed6640 1 --2- 172.21.15.116:0/4011019071 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc2ac0a2c10 0x7fc2ac0a50d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4009230 tx=0x7fc2b40058d0 comp rx=0 tx=0).stop 2024-09-12T03:03:13.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.791+0000 7fc2c8ed6640 1 -- 172.21.15.116:0/4011019071 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2c410c170 msgr2=0x7fc2c419cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:13.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.791+0000 7fc2c8ed6640 1 --2- 172.21.15.116:0/4011019071 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2c410c170 0x7fc2c419cd40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00b970 tx=0x7fc2bc00be30 comp rx=0 tx=0).stop 2024-09-12T03:03:13.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.791+0000 7fc2c8ed6640 1 -- 172.21.15.116:0/4011019071 shutdown_connections 2024-09-12T03:03:13.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.791+0000 7fc2c8ed6640 1 --2- 172.21.15.116:0/4011019071 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc2ac0a2c10 0x7fc2ac0a50d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:13.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.791+0000 7fc2c8ed6640 1 --2- 172.21.15.116:0/4011019071 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2c410c170 0x7fc2c419cd40 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:13.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.791+0000 7fc2c8ed6640 1 --2- 172.21.15.116:0/4011019071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2c4103660 0x7fc2c41989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:13.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.791+0000 7fc2c8ed6640 1 --2- 172.21.15.116:0/4011019071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2c4102d40 0x7fc2c4198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:13.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.791+0000 7fc2c8ed6640 1 -- 172.21.15.116:0/4011019071 >> 172.21.15.116:0/4011019071 conn(0x7fc2c40fc7a0 msgr2=0x7fc2c40fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:13.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.791+0000 7fc2c8ed6640 1 -- 172.21.15.116:0/4011019071 shutdown_connections 2024-09-12T03:03:13.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:13.791+0000 7fc2c8ed6640 1 -- 172.21.15.116:0/4011019071 wait complete. 2024-09-12T03:03:14.409 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:14 smithi116 ceph-mon[31410]: pgmap v345: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T03:03:14.409 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:14 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/4011019071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:14.435 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:03:14.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:14 smithi183 ceph-mon[35535]: pgmap v345: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T03:03:14.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:14 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/4011019071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:14.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:14 smithi177 ceph-mon[35305]: pgmap v345: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T03:03:14.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:14 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/4011019071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:14.680 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:15.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.209+0000 7f4d6ebfa640 1 -- 172.21.15.116:0/2335278850 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d68100a80 msgr2=0x7f4d68100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:15.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.209+0000 7f4d6ebfa640 1 --2- 172.21.15.116:0/2335278850 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d68100a80 0x7f4d68100ee0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4d54009930 tx=0x7f4d5402a420 comp rx=0 tx=0).stop 2024-09-12T03:03:15.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.209+0000 7f4d6ebfa640 1 -- 172.21.15.116:0/2335278850 shutdown_connections 2024-09-12T03:03:15.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.209+0000 7f4d6ebfa640 1 --2- 172.21.15.116:0/2335278850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d68101420 0x7f4d6810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:15.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.209+0000 7f4d6ebfa640 1 --2- 172.21.15.116:0/2335278850 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d68100a80 0x7f4d68100ee0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:15.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.209+0000 7f4d6ebfa640 1 --2- 172.21.15.116:0/2335278850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d68106a80 0x7f4d68106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:15.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.209+0000 7f4d6ebfa640 1 -- 172.21.15.116:0/2335278850 >> 172.21.15.116:0/2335278850 conn(0x7f4d680fc7a0 msgr2=0x7f4d680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:15.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.209+0000 7f4d6ebfa640 1 -- 172.21.15.116:0/2335278850 shutdown_connections 2024-09-12T03:03:15.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.209+0000 7f4d6ebfa640 1 -- 172.21.15.116:0/2335278850 wait complete. 2024-09-12T03:03:15.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.209+0000 7f4d6ebfa640 1 Processor -- start 2024-09-12T03:03:15.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.210+0000 7f4d6ebfa640 1 -- start start 2024-09-12T03:03:15.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.210+0000 7f4d6ebfa640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d68100a80 0x7f4d68197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:15.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.210+0000 7f4d6ebfa640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d68101420 0x7f4d68197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:15.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.210+0000 7f4d6ebfa640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d68106a80 0x7f4d6819baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:15.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.210+0000 7f4d6ebfa640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4d68110e80 con 0x7f4d68100a80 2024-09-12T03:03:15.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.210+0000 7f4d6ebfa640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4d68110d00 con 0x7f4d68101420 2024-09-12T03:03:15.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.210+0000 7f4d6ebfa640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4d68111000 con 0x7f4d68106a80 2024-09-12T03:03:15.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.210+0000 7f4d6c996640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d68100a80 0x7f4d68197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:15.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.210+0000 7f4d6c996640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d68100a80 0x7f4d68197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43554/0 (socket says 172.21.15.116:43554) 2024-09-12T03:03:15.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.210+0000 7f4d6c996640 1 -- 172.21.15.116:0/3345431909 learned_addr learned my addr 172.21.15.116:0/3345431909 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:15.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.210+0000 7f4d6d197640 1 --2- 172.21.15.116:0/3345431909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d68106a80 0x7f4d6819baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:15.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.210+0000 7f4d6c996640 1 -- 172.21.15.116:0/3345431909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d68106a80 msgr2=0x7f4d6819baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:15.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.210+0000 7f4d6c996640 1 --2- 172.21.15.116:0/3345431909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d68106a80 0x7f4d6819baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:15.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.210+0000 7f4d6c996640 1 -- 172.21.15.116:0/3345431909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d68101420 msgr2=0x7f4d68197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:15.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.210+0000 7f4d6c996640 1 --2- 172.21.15.116:0/3345431909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d68101420 0x7f4d68197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:15.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.210+0000 7f4d6c996640 1 -- 172.21.15.116:0/3345431909 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6819c1d0 con 0x7f4d68100a80 2024-09-12T03:03:15.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.210+0000 7f4d6c996640 1 --2- 172.21.15.116:0/3345431909 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d68100a80 0x7f4d68197220 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c00c920 tx=0x7f4d5c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:15.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.210+0000 7f4d65ffb640 1 -- 172.21.15.116:0/3345431909 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c007c70 con 0x7f4d68100a80 2024-09-12T03:03:15.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.211+0000 7f4d6ebfa640 1 -- 172.21.15.116:0/3345431909 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d6819c4c0 con 0x7f4d68100a80 2024-09-12T03:03:15.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.211+0000 7f4d65ffb640 1 -- 172.21.15.116:0/3345431909 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d5c007e10 con 0x7f4d68100a80 2024-09-12T03:03:15.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.211+0000 7f4d65ffb640 1 -- 172.21.15.116:0/3345431909 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c018890 con 0x7f4d68100a80 2024-09-12T03:03:15.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.211+0000 7f4d6ebfa640 1 -- 172.21.15.116:0/3345431909 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d681a3da0 con 0x7f4d68100a80 2024-09-12T03:03:15.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.212+0000 7f4d6ebfa640 1 -- 172.21.15.116:0/3345431909 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d68100ee0 con 0x7f4d68100a80 2024-09-12T03:03:15.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.213+0000 7f4d65ffb640 1 -- 172.21.15.116:0/3345431909 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f4d5c018a70 con 0x7f4d68100a80 2024-09-12T03:03:15.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.213+0000 7f4d65ffb640 1 --2- 172.21.15.116:0/3345431909 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4d580a2c10 0x7f4d580a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:15.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.213+0000 7f4d67fff640 1 --2- 172.21.15.116:0/3345431909 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4d580a2c10 0x7f4d580a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:15.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.213+0000 7f4d65ffb640 1 -- 172.21.15.116:0/3345431909 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f4d5c0cf320 con 0x7f4d68100a80 2024-09-12T03:03:15.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.214+0000 7f4d67fff640 1 --2- 172.21.15.116:0/3345431909 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4d580a2c10 0x7f4d580a50d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4d5402a940 tx=0x7f4d54000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:15.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.217+0000 7f4d65ffb640 1 -- 172.21.15.116:0/3345431909 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4d5c014030 con 0x7f4d68100a80 2024-09-12T03:03:15.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.416+0000 7f4d6ebfa640 1 -- 172.21.15.116:0/3345431909 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d681a4030 con 0x7f4d68100a80 2024-09-12T03:03:15.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.417+0000 7f4d65ffb640 1 -- 172.21.15.116:0/3345431909 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f4d5c09c150 con 0x7f4d68100a80 2024-09-12T03:03:15.419 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:03:15.419 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:03:15.420 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:03:15.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.419+0000 7f4d6ebfa640 1 -- 172.21.15.116:0/3345431909 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4d580a2c10 msgr2=0x7f4d580a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:15.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.419+0000 7f4d6ebfa640 1 --2- 172.21.15.116:0/3345431909 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4d580a2c10 0x7f4d580a50d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4d5402a940 tx=0x7f4d54000bc0 comp rx=0 tx=0).stop 2024-09-12T03:03:15.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.419+0000 7f4d6ebfa640 1 -- 172.21.15.116:0/3345431909 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d68100a80 msgr2=0x7f4d68197220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:15.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.419+0000 7f4d6ebfa640 1 --2- 172.21.15.116:0/3345431909 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d68100a80 0x7f4d68197220 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c00c920 tx=0x7f4d5c00cde0 comp rx=0 tx=0).stop 2024-09-12T03:03:15.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.419+0000 7f4d6ebfa640 1 -- 172.21.15.116:0/3345431909 shutdown_connections 2024-09-12T03:03:15.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.419+0000 7f4d6ebfa640 1 --2- 172.21.15.116:0/3345431909 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4d580a2c10 0x7f4d580a50d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:15.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.419+0000 7f4d6ebfa640 1 --2- 172.21.15.116:0/3345431909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d68106a80 0x7f4d6819baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:15.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.419+0000 7f4d6ebfa640 1 --2- 172.21.15.116:0/3345431909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d68101420 0x7f4d68197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:15.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.419+0000 7f4d6ebfa640 1 --2- 172.21.15.116:0/3345431909 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d68100a80 0x7f4d68197220 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:15.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.419+0000 7f4d6ebfa640 1 -- 172.21.15.116:0/3345431909 >> 172.21.15.116:0/3345431909 conn(0x7f4d680fc7a0 msgr2=0x7f4d680fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:15.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.420+0000 7f4d6ebfa640 1 -- 172.21.15.116:0/3345431909 shutdown_connections 2024-09-12T03:03:15.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:15.420+0000 7f4d6ebfa640 1 -- 172.21.15.116:0/3345431909 wait complete. 2024-09-12T03:03:16.150 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:03:16.400 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:16.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:16 smithi183 ceph-mon[35535]: pgmap v346: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-12T03:03:16.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:16 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3345431909' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:16.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:16 smithi177 ceph-mon[35305]: pgmap v346: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-12T03:03:16.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:16 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3345431909' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:16.521 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:16 smithi116 ceph-mon[31410]: pgmap v346: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-12T03:03:16.521 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:16 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3345431909' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:16.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.915+0000 7f367e249640 1 -- 172.21.15.116:0/774355985 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f367810c170 msgr2=0x7f367810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:16.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.915+0000 7f367e249640 1 --2- 172.21.15.116:0/774355985 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f367810c170 0x7f367810e610 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3670009900 tx=0x7f367002a480 comp rx=0 tx=0).stop 2024-09-12T03:03:16.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.915+0000 7f367e249640 1 -- 172.21.15.116:0/774355985 shutdown_connections 2024-09-12T03:03:16.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.915+0000 7f367e249640 1 --2- 172.21.15.116:0/774355985 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f367810c170 0x7f367810e610 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:16.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.915+0000 7f367e249640 1 --2- 172.21.15.116:0/774355985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3678103660 0x7f367810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:16.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.915+0000 7f367e249640 1 --2- 172.21.15.116:0/774355985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3678102d40 0x7f3678103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:16.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.915+0000 7f367e249640 1 -- 172.21.15.116:0/774355985 >> 172.21.15.116:0/774355985 conn(0x7f36780fc7a0 msgr2=0x7f36780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:16.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.915+0000 7f367e249640 1 -- 172.21.15.116:0/774355985 shutdown_connections 2024-09-12T03:03:16.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.915+0000 7f367e249640 1 -- 172.21.15.116:0/774355985 wait complete. 2024-09-12T03:03:16.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.916+0000 7f367e249640 1 Processor -- start 2024-09-12T03:03:16.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.916+0000 7f367e249640 1 -- start start 2024-09-12T03:03:16.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.916+0000 7f367e249640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3678102d40 0x7f3678198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:16.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.916+0000 7f367e249640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3678103660 0x7f3678198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:16.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.916+0000 7f367e249640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f367810c170 0x7f367819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:16.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.916+0000 7f367e249640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3678111f70 con 0x7f3678102d40 2024-09-12T03:03:16.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.916+0000 7f367e249640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3678111df0 con 0x7f367810c170 2024-09-12T03:03:16.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.916+0000 7f367e249640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f36781120f0 con 0x7f3678103660 2024-09-12T03:03:16.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.916+0000 7f36777fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3678102d40 0x7f3678198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:16.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.916+0000 7f36777fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3678102d40 0x7f3678198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43568/0 (socket says 172.21.15.116:43568) 2024-09-12T03:03:16.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.916+0000 7f36777fe640 1 -- 172.21.15.116:0/2208400614 learned_addr learned my addr 172.21.15.116:0/2208400614 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:16.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.917+0000 7f3677fff640 1 --2- 172.21.15.116:0/2208400614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f367810c170 0x7f367819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:16.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.917+0000 7f36777fe640 1 -- 172.21.15.116:0/2208400614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3678103660 msgr2=0x7f3678198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:16.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.917+0000 7f3676ffd640 1 --2- 172.21.15.116:0/2208400614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3678103660 0x7f3678198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:16.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.917+0000 7f36777fe640 1 --2- 172.21.15.116:0/2208400614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3678103660 0x7f3678198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:16.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.917+0000 7f36777fe640 1 -- 172.21.15.116:0/2208400614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f367810c170 msgr2=0x7f367819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:16.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.917+0000 7f36777fe640 1 --2- 172.21.15.116:0/2208400614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f367810c170 0x7f367819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:16.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.917+0000 7f36777fe640 1 -- 172.21.15.116:0/2208400614 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f367819d3c0 con 0x7f3678102d40 2024-09-12T03:03:16.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.917+0000 7f3677fff640 1 --2- 172.21.15.116:0/2208400614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f367810c170 0x7f367819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:03:16.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.917+0000 7f36777fe640 1 --2- 172.21.15.116:0/2208400614 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3678102d40 0x7f3678198410 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f366800ea80 tx=0x7f366800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:16.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.917+0000 7f3674ff9640 1 -- 172.21.15.116:0/2208400614 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366800ce40 con 0x7f3678102d40 2024-09-12T03:03:16.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.917+0000 7f367e249640 1 -- 172.21.15.116:0/2208400614 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f367819d6b0 con 0x7f3678102d40 2024-09-12T03:03:16.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.917+0000 7f3674ff9640 1 -- 172.21.15.116:0/2208400614 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3668005020 con 0x7f3678102d40 2024-09-12T03:03:16.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.917+0000 7f3674ff9640 1 -- 172.21.15.116:0/2208400614 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3668011760 con 0x7f3678102d40 2024-09-12T03:03:16.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.917+0000 7f367e249640 1 -- 172.21.15.116:0/2208400614 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36781a4f90 con 0x7f3678102d40 2024-09-12T03:03:16.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.918+0000 7f367e249640 1 -- 172.21.15.116:0/2208400614 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3678103120 con 0x7f3678102d40 2024-09-12T03:03:16.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.920+0000 7f3674ff9640 1 -- 172.21.15.116:0/2208400614 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f36680051d0 con 0x7f3678102d40 2024-09-12T03:03:16.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.921+0000 7f3674ff9640 1 --2- 172.21.15.116:0/2208400614 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f36640a2c10 0x7f36640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:16.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.921+0000 7f3676ffd640 1 --2- 172.21.15.116:0/2208400614 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f36640a2c10 0x7f36640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:16.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.921+0000 7f3674ff9640 1 -- 172.21.15.116:0/2208400614 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f36680cf890 con 0x7f3678102d40 2024-09-12T03:03:16.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.921+0000 7f3676ffd640 1 --2- 172.21.15.116:0/2208400614 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f36640a2c10 0x7f36640a50d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f367810d490 tx=0x7f3660005c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:16.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:16.924+0000 7f3674ff9640 1 -- 172.21.15.116:0/2208400614 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3668016030 con 0x7f3678102d40 2024-09-12T03:03:17.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:17.093+0000 7f367e249640 1 -- 172.21.15.116:0/2208400614 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3678199770 con 0x7f3678102d40 2024-09-12T03:03:17.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:17.094+0000 7f3674ff9640 1 -- 172.21.15.116:0/2208400614 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22710 (secure 0 0 0) 0x7f366809c6c0 con 0x7f3678102d40 2024-09-12T03:03:17.095 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:03:17.095 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":80,"fsid":"eb1ac602-70b1-11ef-bcea-c7b262605968","created":"2024-09-12T02:53:07.105686+0000","modified":"2024-09-12T03:02:04.272898+0000","last_up_change":"2024-09-12T03:00:07.202639+0000","last_in_change":"2024-09-12T02:59:43.446336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:56:17.347152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:00:55.191040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:00:57.171142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8cf5a18-dc6f-4bb6-bd74-78adda310118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6803","nonce":862366853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6805","nonce":862366853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6809","nonce":862366853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6807","nonce":862366853}]},"public_addr":"172.21.15.116:6803/862366853","cluster_addr":"172.21.15.116:6805/862366853","heartbeat_back_addr":"172.21.15.116:6809/862366853","heartbeat_front_addr":"172.21.15.116:6807/862366853","state":["exists","up"]},{"osd":1,"uuid":"0affd3cb-9a51-42b0-a2cc-f30784a75cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6801","nonce":2245800547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6803","nonce":2245800547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6807","nonce":2245800547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6805","nonce":2245800547}]},"public_addr":"172.21.15.177:6801/2245800547","cluster_addr":"172.21.15.177:6803/2245800547","heartbeat_back_addr":"172.21.15.177:6807/2245800547","heartbeat_front_addr":"172.21.15.177:6805/2245800547","state":["exists","up"]},{"osd":2,"uuid":"1ce45ea5-84df-4a35-924f-78f92bd4f2cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6801","nonce":2645677759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6803","nonce":2645677759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6807","nonce":2645677759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6805","nonce":2645677759}]},"public_addr":"172.21.15.183:6801/2645677759","cluster_addr":"172.21.15.183:6803/2645677759","heartbeat_back_addr":"172.21.15.183:6807/2645677759","heartbeat_front_addr":"172.21.15.183:6805/2645677759","state":["exists","up"]},{"osd":3,"uuid":"ae5e8178-c998-4648-bc05-4bce822f0cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6811","nonce":686622148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6813","nonce":686622148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6817","nonce":686622148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6815","nonce":686622148}]},"public_addr":"172.21.15.116:6811/686622148","cluster_addr":"172.21.15.116:6813/686622148","heartbeat_back_addr":"172.21.15.116:6817/686622148","heartbeat_front_addr":"172.21.15.116:6815/686622148","state":["exists","up"]},{"osd":4,"uuid":"c81d2e73-0bc2-440d-b5a2-b7428af3ea4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6809","nonce":2382938021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6811","nonce":2382938021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6815","nonce":2382938021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6813","nonce":2382938021}]},"public_addr":"172.21.15.177:6809/2382938021","cluster_addr":"172.21.15.177:6811/2382938021","heartbeat_back_addr":"172.21.15.177:6815/2382938021","heartbeat_front_addr":"172.21.15.177:6813/2382938021","state":["exists","up"]},{"osd":5,"uuid":"c3f168be-9e27-4cf5-b73e-3f47d8990ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6809","nonce":85363432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6811","nonce":85363432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6815","nonce":85363432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6813","nonce":85363432}]},"public_addr":"172.21.15.183:6809/85363432","cluster_addr":"172.21.15.183:6811/85363432","heartbeat_back_addr":"172.21.15.183:6815/85363432","heartbeat_front_addr":"172.21.15.183:6813/85363432","state":["exists","up"]},{"osd":6,"uuid":"beb0a7e9-4909-47e2-9ee7-6951d4745e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6819","nonce":1121291186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6821","nonce":1121291186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6825","nonce":1121291186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6823","nonce":1121291186}]},"public_addr":"172.21.15.116:6819/1121291186","cluster_addr":"172.21.15.116:6821/1121291186","heartbeat_back_addr":"172.21.15.116:6825/1121291186","heartbeat_front_addr":"172.21.15.116:6823/1121291186","state":["exists","up"]},{"osd":7,"uuid":"761f4680-f049-473c-bf44-b19acf8de24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6817","nonce":2177665961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6819","nonce":2177665961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6823","nonce":2177665961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6821","nonce":2177665961}]},"public_addr":"172.21.15.177:6817/2177665961","cluster_addr":"172.21.15.177:6819/2177665961","heartbeat_back_addr":"172.21.15.177:6823/2177665961","heartbeat_front_addr":"172.21.15.177:6821/2177665961","state":["exists","up"]},{"osd":8,"uuid":"2385ea6c-fd5a-4815-9596-77f948629542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6817","nonce":1305725258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6819","nonce":1305725258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6823","nonce":1305725258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6821","nonce":1305725258}]},"public_addr":"172.21.15.183:6817/1305725258","cluster_addr":"172.21.15.183:6819/1305725258","heartbeat_back_addr":"172.21.15.183:6823/1305725258","heartbeat_front_addr":"172.21.15.183:6821/1305725258","state":["exists","up"]},{"osd":9,"uuid":"fd0c0dc4-4990-4f12-aa14-f855504cd995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6827","nonce":3831673668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6829","nonce":3831673668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6833","nonce":3831673668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6831","nonce":3831673668}]},"public_addr":"172.21.15.116:6827/3831673668","cluster_addr":"172.21.15.116:6829/3831673668","heartbeat_back_addr":"172.21.15.116:6833/3831673668","heartbeat_front_addr":"172.21.15.116:6831/3831673668","state":["exists","up"]},{"osd":10,"uuid":"864d1a97-4810-49c8-ad4a-a0c10253833c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6825","nonce":963663634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6827","nonce":963663634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6831","nonce":963663634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6829","nonce":963663634}]},"public_addr":"172.21.15.177:6825/963663634","cluster_addr":"172.21.15.177:6827/963663634","heartbeat_back_addr":"172.21.15.177:6831/963663634","heartbeat_front_addr":"172.21.15.177:6829/963663634","state":["exists","up"]},{"osd":11,"uuid":"d49c0841-4620-4f2f-a6b5-f2a9aa2e3c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6825","nonce":1818319939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6827","nonce":1818319939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6831","nonce":1818319939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6829","nonce":1818319939}]},"public_addr":"172.21.15.183:6825/1818319939","cluster_addr":"172.21.15.183:6827/1818319939","heartbeat_back_addr":"172.21.15.183:6831/1818319939","heartbeat_front_addr":"172.21.15.183:6829/1818319939","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:24.736944+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:50.310980+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:56:14.989533+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:56:41.187047+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:06.981664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:32.872771+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:58.850089+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:23.043121+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:47.839958+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:14.087708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:40.010548+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:05.364820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.116:6801/2966274423":"2024-09-13T02:54:03.026862+0000","172.21.15.116:6800/2966274423":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/686139455":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/1552047390":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/2180363411":"2024-09-13T02:53:39.433769+0000","172.21.15.116:6801/1791219601":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/3907400460":"2024-09-13T02:54:03.026862+0000","172.21.15.116:6800/1791219601":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/1802566286":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/212337802":"2024-09-13T02:53:39.433769+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:03:17.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:17.096+0000 7f367e249640 1 -- 172.21.15.116:0/2208400614 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f36640a2c10 msgr2=0x7f36640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:17.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:17.096+0000 7f367e249640 1 --2- 172.21.15.116:0/2208400614 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f36640a2c10 0x7f36640a50d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f367810d490 tx=0x7f3660005c80 comp rx=0 tx=0).stop 2024-09-12T03:03:17.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:17.096+0000 7f367e249640 1 -- 172.21.15.116:0/2208400614 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3678102d40 msgr2=0x7f3678198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:17.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:17.096+0000 7f367e249640 1 --2- 172.21.15.116:0/2208400614 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3678102d40 0x7f3678198410 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f366800ea80 tx=0x7f366800ef40 comp rx=0 tx=0).stop 2024-09-12T03:03:17.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:17.096+0000 7f367e249640 1 -- 172.21.15.116:0/2208400614 shutdown_connections 2024-09-12T03:03:17.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:17.096+0000 7f367e249640 1 --2- 172.21.15.116:0/2208400614 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f36640a2c10 0x7f36640a50d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:17.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:17.096+0000 7f367e249640 1 --2- 172.21.15.116:0/2208400614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f367810c170 0x7f367819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:17.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:17.096+0000 7f367e249640 1 --2- 172.21.15.116:0/2208400614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3678103660 0x7f3678198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:17.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:17.096+0000 7f367e249640 1 --2- 172.21.15.116:0/2208400614 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3678102d40 0x7f3678198410 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:17.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:17.096+0000 7f367e249640 1 -- 172.21.15.116:0/2208400614 >> 172.21.15.116:0/2208400614 conn(0x7f36780fc7a0 msgr2=0x7f36780fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:17.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:17.096+0000 7f367e249640 1 -- 172.21.15.116:0/2208400614 shutdown_connections 2024-09-12T03:03:17.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:17.096+0000 7f367e249640 1 -- 172.21.15.116:0/2208400614 wait complete. 2024-09-12T03:03:17.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:17 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2208400614' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:03:17.465 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:17 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2208400614' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:03:17.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:17 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2208400614' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:03:17.812 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-12T03:03:18.056 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:18.445 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:18 smithi116 ceph-mon[31410]: pgmap v347: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-09-12T03:03:18.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:18 smithi183 ceph-mon[35535]: pgmap v347: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-09-12T03:03:18.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:18 smithi177 ceph-mon[35305]: pgmap v347: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-09-12T03:03:19.272 INFO:teuthology.orchestra.run.smithi116.stdout:{"run_dir":"/var/run/ceph"} 2024-09-12T03:03:19.272 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-12T03:03:19.273 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 481 2024-09-12T03:03:19.273 DEBUG:tasks.quiescer:Yielding 2024-09-12T03:03:19.273 INFO:teuthology.run_tasks:Running task workunit... 2024-09-12T03:03:19.368 INFO:tasks.workunit:Pulling workunits from ref 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T03:03:19.368 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-12T03:03:19.368 INFO:tasks.workunit:timeout=3h 2024-09-12T03:03:19.369 INFO:tasks.workunit:cleanup=True 2024-09-12T03:03:19.369 DEBUG:teuthology.orchestra.run.smithi116:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-12T03:03:19.369 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:03:19.386 INFO:teuthology.orchestra.run.smithi116.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-12T03:03:19.387 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-12T03:03:19.387 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-12T03:03:19.387 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:03:19.387 INFO:teuthology.orchestra.run.smithi116.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-12T03:03:19.387 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2024-09-12 03:01:17.252083011 +0000 2024-09-12T03:03:19.387 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2024-09-12 03:01:17.252083011 +0000 2024-09-12T03:03:19.387 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2024-09-12 03:02:55.674111078 +0000 2024-09-12T03:03:19.387 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: 2024-09-12 03:01:17.252083011 +0000 2024-09-12T03:03:19.387 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-12T03:03:19.387 DEBUG:teuthology.orchestra.run.smithi116:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-12T03:03:19.448 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:19 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:03:19.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:19 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:03:19.457 DEBUG:teuthology.orchestra.run.smithi116:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T03:03:19.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:19 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:03:19.513 INFO:tasks.workunit.client.0.smithi116.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-12T03:03:19.623 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:20.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.152+0000 7f312d9b1640 1 -- 172.21.15.116:0/2384865061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3128105a90 msgr2=0x7f3128105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:20.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.152+0000 7f312d9b1640 1 --2- 172.21.15.116:0/2384865061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3128105a90 0x7f3128105e70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3118009900 tx=0x7f311802a410 comp rx=0 tx=0).stop 2024-09-12T03:03:20.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.152+0000 7f312d9b1640 1 -- 172.21.15.116:0/2384865061 shutdown_connections 2024-09-12T03:03:20.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.152+0000 7f312d9b1640 1 --2- 172.21.15.116:0/2384865061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3128101bc0 0x7f312810d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:20.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.152+0000 7f312d9b1640 1 --2- 172.21.15.116:0/2384865061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3128101240 0x7f3128101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:20.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.152+0000 7f312d9b1640 1 --2- 172.21.15.116:0/2384865061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3128105a90 0x7f3128105e70 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:20.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.152+0000 7f312d9b1640 1 -- 172.21.15.116:0/2384865061 >> 172.21.15.116:0/2384865061 conn(0x7f3128077210 msgr2=0x7f31280fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:20.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.152+0000 7f312d9b1640 1 -- 172.21.15.116:0/2384865061 shutdown_connections 2024-09-12T03:03:20.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.152+0000 7f312d9b1640 1 -- 172.21.15.116:0/2384865061 wait complete. 2024-09-12T03:03:20.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.152+0000 7f312d9b1640 1 Processor -- start 2024-09-12T03:03:20.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.153+0000 7f312d9b1640 1 -- start start 2024-09-12T03:03:20.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.153+0000 7f312d9b1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3128101240 0x7f312819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:20.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.153+0000 7f312d9b1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3128101bc0 0x7f312819ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:20.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.153+0000 7f312d9b1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3128105a90 0x7f312819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:20.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.153+0000 7f312d9b1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3128110f70 con 0x7f3128101240 2024-09-12T03:03:20.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.153+0000 7f312d9b1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3128110df0 con 0x7f3128105a90 2024-09-12T03:03:20.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.153+0000 7f312d9b1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f31281110f0 con 0x7f3128101bc0 2024-09-12T03:03:20.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.153+0000 7f3126ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3128101240 0x7f312819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:20.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.153+0000 7f3126ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3128101240 0x7f312819a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43580/0 (socket says 172.21.15.116:43580) 2024-09-12T03:03:20.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.153+0000 7f3126ffd640 1 -- 172.21.15.116:0/2786085009 learned_addr learned my addr 172.21.15.116:0/2786085009 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:20.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.153+0000 7f31267fc640 1 --2- 172.21.15.116:0/2786085009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3128101bc0 0x7f312819ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:20.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.153+0000 7f3126ffd640 1 -- 172.21.15.116:0/2786085009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3128101bc0 msgr2=0x7f312819ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:20.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.153+0000 7f3126ffd640 1 --2- 172.21.15.116:0/2786085009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3128101bc0 0x7f312819ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:20.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.153+0000 7f3126ffd640 1 -- 172.21.15.116:0/2786085009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3128105a90 msgr2=0x7f312819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:20.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.153+0000 7f3126ffd640 1 --2- 172.21.15.116:0/2786085009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3128105a90 0x7f312819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:20.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.153+0000 7f3126ffd640 1 -- 172.21.15.116:0/2786085009 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312819f6e0 con 0x7f3128101240 2024-09-12T03:03:20.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.154+0000 7f3126ffd640 1 --2- 172.21.15.116:0/2786085009 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3128101240 0x7f312819a730 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3118006010 tx=0x7f3118003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:20.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.154+0000 7f312c9af640 1 -- 172.21.15.116:0/2786085009 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311803f070 con 0x7f3128101240 2024-09-12T03:03:20.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.154+0000 7f312d9b1640 1 -- 172.21.15.116:0/2786085009 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f312819f970 con 0x7f3128101240 2024-09-12T03:03:20.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.154+0000 7f312c9af640 1 -- 172.21.15.116:0/2786085009 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3118031c30 con 0x7f3128101240 2024-09-12T03:03:20.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.154+0000 7f312c9af640 1 -- 172.21.15.116:0/2786085009 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118035030 con 0x7f3128101240 2024-09-12T03:03:20.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.154+0000 7f312d9b1640 1 -- 172.21.15.116:0/2786085009 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31281a72b0 con 0x7f3128101240 2024-09-12T03:03:20.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.155+0000 7f312d9b1640 1 -- 172.21.15.116:0/2786085009 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3128101680 con 0x7f3128101240 2024-09-12T03:03:20.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.157+0000 7f312c9af640 1 -- 172.21.15.116:0/2786085009 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f3118042070 con 0x7f3128101240 2024-09-12T03:03:20.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.158+0000 7f312c9af640 1 --2- 172.21.15.116:0/2786085009 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f31100a2c10 0x7f31100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:20.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.158+0000 7f31267fc640 1 --2- 172.21.15.116:0/2786085009 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f31100a2c10 0x7f31100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:20.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.158+0000 7f312c9af640 1 -- 172.21.15.116:0/2786085009 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f31180efec0 con 0x7f3128101240 2024-09-12T03:03:20.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.159+0000 7f31267fc640 1 --2- 172.21.15.116:0/2786085009 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f31100a2c10 0x7f31100a50d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3128105050 tx=0x7f311c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:20.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.161+0000 7f312c9af640 1 -- 172.21.15.116:0/2786085009 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3118031de0 con 0x7f3128101240 2024-09-12T03:03:20.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.367+0000 7f312d9b1640 1 -- 172.21.15.116:0/2786085009 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31280fdcd0 con 0x7f3128101240 2024-09-12T03:03:20.367 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:20 smithi116 ceph-mon[31410]: pgmap v348: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:03:20.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.368+0000 7f312c9af640 1 -- 172.21.15.116:0/2786085009 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f31180bcd50 con 0x7f3128101240 2024-09-12T03:03:20.368 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:03:20.368 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:03:20.369 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:03:20.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.371+0000 7f312d9b1640 1 -- 172.21.15.116:0/2786085009 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f31100a2c10 msgr2=0x7f31100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:20.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.371+0000 7f312d9b1640 1 --2- 172.21.15.116:0/2786085009 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f31100a2c10 0x7f31100a50d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3128105050 tx=0x7f311c00b390 comp rx=0 tx=0).stop 2024-09-12T03:03:20.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.371+0000 7f312d9b1640 1 -- 172.21.15.116:0/2786085009 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3128101240 msgr2=0x7f312819a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:20.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.371+0000 7f312d9b1640 1 --2- 172.21.15.116:0/2786085009 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3128101240 0x7f312819a730 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3118006010 tx=0x7f3118003c10 comp rx=0 tx=0).stop 2024-09-12T03:03:20.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.371+0000 7f312d9b1640 1 -- 172.21.15.116:0/2786085009 shutdown_connections 2024-09-12T03:03:20.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.371+0000 7f312d9b1640 1 --2- 172.21.15.116:0/2786085009 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f31100a2c10 0x7f31100a50d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:20.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.371+0000 7f312d9b1640 1 --2- 172.21.15.116:0/2786085009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3128105a90 0x7f312819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:20.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.371+0000 7f312d9b1640 1 --2- 172.21.15.116:0/2786085009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3128101bc0 0x7f312819ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:20.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.371+0000 7f312d9b1640 1 --2- 172.21.15.116:0/2786085009 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3128101240 0x7f312819a730 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:20.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.371+0000 7f312d9b1640 1 -- 172.21.15.116:0/2786085009 >> 172.21.15.116:0/2786085009 conn(0x7f3128077210 msgr2=0x7f3128104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:20.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.371+0000 7f312d9b1640 1 -- 172.21.15.116:0/2786085009 shutdown_connections 2024-09-12T03:03:20.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:20.371+0000 7f312d9b1640 1 -- 172.21.15.116:0/2786085009 wait complete. 2024-09-12T03:03:20.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:20 smithi183 ceph-mon[35535]: pgmap v348: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:03:20.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:20 smithi177 ceph-mon[35305]: pgmap v348: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:03:21.094 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T03:00:59.195360+0000', 'modified': '2024-09-12T03:02:56.334134+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24494, 'mds_1': 24475, 'mds_2': 24467}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24443': {'gid': 24443, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.177:6835/2224643300', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6834', 'nonce': 2224643300}, {'type': 'v1', 'addr': '172.21.15.177:6835', 'nonce': 2224643300}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/4028826399', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 4028826399}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 4028826399}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6837/1288373243', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 1288373243}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 1288373243}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.177:6837/1862929607', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6836', 'nonce': 1862929607}, {'type': 'v1', 'addr': '172.21.15.177:6837', 'nonce': 1862929607}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.183:6839/1983839027', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 1983839027}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 1983839027}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.177:6839/2069130361', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6838', 'nonce': 2069130361}, {'type': 'v1', 'addr': '172.21.15.177:6839', 'nonce': 2069130361}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24494, 'qdb_cluster': [24494, 24475, 24467]} 2024-09-12T03:03:21.094 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-12T03:03:21.095 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-12T03:03:21.384 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:21.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:21 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2786085009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:21.505 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:21 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2786085009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:21.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:21 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2786085009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:21.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.903+0000 7fea667b2640 1 -- 172.21.15.116:0/644776254 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea601055b0 msgr2=0x7fea6010d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:21.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.903+0000 7fea667b2640 1 --2- 172.21.15.116:0/644776254 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea601055b0 0x7fea6010d130 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fea58009ca0 tx=0x7fea5802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.903+0000 7fea667b2640 1 -- 172.21.15.116:0/644776254 shutdown_connections 2024-09-12T03:03:21.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.903+0000 7fea667b2640 1 --2- 172.21.15.116:0/644776254 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea601055b0 0x7fea6010d130 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.903+0000 7fea667b2640 1 --2- 172.21.15.116:0/644776254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea60100e30 0x7fea60104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.903+0000 7fea667b2640 1 --2- 172.21.15.116:0/644776254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea60100480 0x7fea60100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.903+0000 7fea667b2640 1 -- 172.21.15.116:0/644776254 >> 172.21.15.116:0/644776254 conn(0x7fea60077210 msgr2=0x7fea600fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:21.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.903+0000 7fea667b2640 1 -- 172.21.15.116:0/644776254 shutdown_connections 2024-09-12T03:03:21.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.903+0000 7fea667b2640 1 -- 172.21.15.116:0/644776254 wait complete. 2024-09-12T03:03:21.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.904+0000 7fea667b2640 1 Processor -- start 2024-09-12T03:03:21.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.904+0000 7fea667b2640 1 -- start start 2024-09-12T03:03:21.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.904+0000 7fea667b2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea60100480 0x7fea60197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:21.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.904+0000 7fea667b2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea60100e30 0x7fea60197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:21.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.904+0000 7fea667b2640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea601055b0 0x7fea6019bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:21.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.904+0000 7fea667b2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fea60110630 con 0x7fea60100e30 2024-09-12T03:03:21.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.904+0000 7fea667b2640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fea601104b0 con 0x7fea601055b0 2024-09-12T03:03:21.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.904+0000 7fea667b2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fea601107b0 con 0x7fea60100480 2024-09-12T03:03:21.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.905+0000 7fea5f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea60100e30 0x7fea60197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:21.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.905+0000 7fea5f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea60100e30 0x7fea60197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44138/0 (socket says 172.21.15.116:44138) 2024-09-12T03:03:21.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.905+0000 7fea5f7fe640 1 -- 172.21.15.116:0/1366155956 learned_addr learned my addr 172.21.15.116:0/1366155956 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:21.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.905+0000 7fea64d4f640 1 --2- 172.21.15.116:0/1366155956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea601055b0 0x7fea6019bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:21.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.905+0000 7fea5ffff640 1 --2- 172.21.15.116:0/1366155956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea60100480 0x7fea60197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:21.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.905+0000 7fea5f7fe640 1 -- 172.21.15.116:0/1366155956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea60100480 msgr2=0x7fea60197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:21.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.905+0000 7fea5f7fe640 1 --2- 172.21.15.116:0/1366155956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea60100480 0x7fea60197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.905+0000 7fea5f7fe640 1 -- 172.21.15.116:0/1366155956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea601055b0 msgr2=0x7fea6019bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:21.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.905+0000 7fea5f7fe640 1 --2- 172.21.15.116:0/1366155956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea601055b0 0x7fea6019bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.905+0000 7fea5f7fe640 1 -- 172.21.15.116:0/1366155956 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6019c2e0 con 0x7fea60100e30 2024-09-12T03:03:21.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.905+0000 7fea5f7fe640 1 --2- 172.21.15.116:0/1366155956 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea60100e30 0x7fea60197870 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fea4800b4a0 tx=0x7fea4800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:21.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.905+0000 7fea5d7fa640 1 -- 172.21.15.116:0/1366155956 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48014070 con 0x7fea60100e30 2024-09-12T03:03:21.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.905+0000 7fea5d7fa640 1 -- 172.21.15.116:0/1366155956 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea48010af0 con 0x7fea60100e30 2024-09-12T03:03:21.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.905+0000 7fea667b2640 1 -- 172.21.15.116:0/1366155956 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea6019c5d0 con 0x7fea60100e30 2024-09-12T03:03:21.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.905+0000 7fea5d7fa640 1 -- 172.21.15.116:0/1366155956 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48017d10 con 0x7fea60100e30 2024-09-12T03:03:21.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.905+0000 7fea667b2640 1 -- 172.21.15.116:0/1366155956 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea601a3eb0 con 0x7fea60100e30 2024-09-12T03:03:21.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.908+0000 7fea68b8b740 1 Processor -- start 2024-09-12T03:03:21.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.908+0000 7fea68b8b740 1 -- start start 2024-09-12T03:03:21.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.908+0000 7fea5d7fa640 1 -- 172.21.15.116:0/1366155956 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fea48017070 con 0x7fea60100e30 2024-09-12T03:03:21.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.908+0000 7fea68b8b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6f38be9d0 0x55b6f38bedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:21.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.908+0000 7fea68b8b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6f38bf2f0 0x55b6f38c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:21.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.908+0000 7fea68b8b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6f38c9bb0 0x55b6f38cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:21.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.908+0000 7fea68b8b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b6f3825090 con 0x55b6f38c9bb0 2024-09-12T03:03:21.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.908+0000 7fea68b8b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b6f37cca70 con 0x55b6f38bf2f0 2024-09-12T03:03:21.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.908+0000 7fea68b8b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b6f3821050 con 0x55b6f38be9d0 2024-09-12T03:03:21.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.909+0000 7fea5f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6f38c9bb0 0x55b6f38cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:21.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.909+0000 7fea5f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6f38c9bb0 0x55b6f38cbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44150/0 (socket says 172.21.15.116:44150) 2024-09-12T03:03:21.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.909+0000 7fea5f7fe640 1 -- 172.21.15.116:0/1621853816 learned_addr learned my addr 172.21.15.116:0/1621853816 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:21.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.909+0000 7fea5ffff640 1 --2- 172.21.15.116:0/1621853816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6f38bf2f0 0x55b6f38c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:21.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.909+0000 7fea64d4f640 1 --2- 172.21.15.116:0/1621853816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6f38be9d0 0x55b6f38bedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:21.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.909+0000 7fea5f7fe640 1 -- 172.21.15.116:0/1621853816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6f38be9d0 msgr2=0x55b6f38bedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:21.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.909+0000 7fea5f7fe640 1 --2- 172.21.15.116:0/1621853816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6f38be9d0 0x55b6f38bedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.909+0000 7fea5f7fe640 1 -- 172.21.15.116:0/1621853816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6f38bf2f0 msgr2=0x55b6f38c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:21.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.909+0000 7fea5f7fe640 1 --2- 172.21.15.116:0/1621853816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6f38bf2f0 0x55b6f38c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.909+0000 7fea5f7fe640 1 -- 172.21.15.116:0/1621853816 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6f37d9130 con 0x55b6f38c9bb0 2024-09-12T03:03:21.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.909+0000 7fea5d7fa640 1 --2- 172.21.15.116:0/1366155956 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fea4c0a2c10 0x7fea4c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:21.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.909+0000 7fea5f7fe640 1 --2- 172.21.15.116:0/1621853816 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6f38c9bb0 0x55b6f38cbfa0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fea480cbf00 tx=0x7fea480d4b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dfc922c89d0bf6d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:21.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.909+0000 7fea64d4f640 1 --2- 172.21.15.116:0/1366155956 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fea4c0a2c10 0x7fea4c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:21.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.909+0000 7fea5d7fa640 1 -- 172.21.15.116:0/1366155956 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fea480ce4d0 con 0x7fea60100e30 2024-09-12T03:03:21.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.909+0000 7fea567fc640 1 -- 172.21.15.116:0/1621853816 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48096fb0 con 0x55b6f38c9bb0 2024-09-12T03:03:21.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.909+0000 7fea567fc640 1 -- 172.21.15.116:0/1621853816 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea4809b410 con 0x55b6f38c9bb0 2024-09-12T03:03:21.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.910+0000 7fea567fc640 1 -- 172.21.15.116:0/1621853816 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4809a670 con 0x55b6f38c9bb0 2024-09-12T03:03:21.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.910+0000 7fea68b8b740 1 -- 172.21.15.116:0/1621853816 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6f38c9bb0 msgr2=0x55b6f38cbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:21.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.910+0000 7fea64d4f640 1 --2- 172.21.15.116:0/1366155956 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fea4c0a2c10 0x7fea4c0a50d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fea58009430 tx=0x7fea58009380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:21.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.910+0000 7fea68b8b740 1 --2- 172.21.15.116:0/1621853816 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6f38c9bb0 0x55b6f38cbfa0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fea480cbf00 tx=0x7fea480d4b40 comp rx=0 tx=0).stop 2024-09-12T03:03:21.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.910+0000 7fea68b8b740 1 -- 172.21.15.116:0/1621853816 shutdown_connections 2024-09-12T03:03:21.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.910+0000 7fea68b8b740 1 --2- 172.21.15.116:0/1621853816 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6f38c9bb0 0x55b6f38cbfa0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.910+0000 7fea68b8b740 1 --2- 172.21.15.116:0/1621853816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6f38bf2f0 0x55b6f38c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.910+0000 7fea68b8b740 1 --2- 172.21.15.116:0/1621853816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6f38be9d0 0x55b6f38bedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.910+0000 7fea68b8b740 1 -- 172.21.15.116:0/1621853816 >> 172.21.15.116:0/1621853816 conn(0x55b6f38baf60 msgr2=0x55b6f38bb340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:21.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.910+0000 7fea68b8b740 1 -- 172.21.15.116:0/1621853816 shutdown_connections 2024-09-12T03:03:21.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.910+0000 7fea68b8b740 1 -- 172.21.15.116:0/1621853816 wait complete. 2024-09-12T03:03:21.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.911+0000 7fea68b8b740 1 -- start start 2024-09-12T03:03:21.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.911+0000 7fea68b8b740 1 Processor -- start 2024-09-12T03:03:21.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.911+0000 7fea68b8b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6f38be9d0 0x55b6f38c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:21.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.911+0000 7fea68b8b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6f38bf2f0 0x55b6f38c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:21.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.911+0000 7fea68b8b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6f38c9bb0 0x55b6f38d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:21.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.911+0000 7fea68b8b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b6f38cc780 con 0x55b6f38be9d0 2024-09-12T03:03:21.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.911+0000 7fea68b8b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b6f37dff20 con 0x55b6f38bf2f0 2024-09-12T03:03:21.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.911+0000 7fea68b8b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b6f38cc900 con 0x55b6f38c9bb0 2024-09-12T03:03:21.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.911+0000 7fea68b8b740 10 client.0 _ll_register_callbacks cb 0x55b6f37a9030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:21.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.911+0000 7fea5ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6f38be9d0 0x55b6f38c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:21.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.911+0000 7fea5ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6f38be9d0 0x55b6f38c8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44164/0 (socket says 172.21.15.116:44164) 2024-09-12T03:03:21.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.911+0000 7fea5ffff640 1 -- 172.21.15.116:0/1842206623 learned_addr learned my addr 172.21.15.116:0/1842206623 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:21.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.911+0000 7fea5f7fe640 1 --2- 172.21.15.116:0/1842206623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6f38bf2f0 0x55b6f38c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:21.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.911+0000 7fea64d4f640 1 --2- 172.21.15.116:0/1842206623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6f38c9bb0 0x55b6f38d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:21.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.912+0000 7fea5ffff640 1 -- 172.21.15.116:0/1842206623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6f38c9bb0 msgr2=0x55b6f38d01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:21.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.912+0000 7fea5ffff640 1 --2- 172.21.15.116:0/1842206623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6f38c9bb0 0x55b6f38d01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.912+0000 7fea5ffff640 1 -- 172.21.15.116:0/1842206623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6f38bf2f0 msgr2=0x55b6f38c90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:21.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.912+0000 7fea5ffff640 1 --2- 172.21.15.116:0/1842206623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6f38bf2f0 0x55b6f38c90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.912+0000 7fea5ffff640 1 -- 172.21.15.116:0/1842206623 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6f35962b0 con 0x55b6f38be9d0 2024-09-12T03:03:21.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.912+0000 7fea64d4f640 1 --2- 172.21.15.116:0/1842206623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6f38c9bb0 0x55b6f38d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:03:21.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.912+0000 7fea5ffff640 1 --2- 172.21.15.116:0/1842206623 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6f38be9d0 0x55b6f38c8bb0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fea50010740 tx=0x7fea50010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1212439c95e85b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:21.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.912+0000 7fea3f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:03:21.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.912+0000 7fea3f7fe640 1 -- 172.21.15.116:0/1842206623 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5001d060 con 0x55b6f38be9d0 2024-09-12T03:03:21.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.912+0000 7fea3f7fe640 1 -- 172.21.15.116:0/1842206623 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fea5000eda0 con 0x55b6f38be9d0 2024-09-12T03:03:21.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.912+0000 7fea68b8b740 1 -- 172.21.15.116:0/1842206623 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea4809b410 con 0x55b6f38be9d0 2024-09-12T03:03:21.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.912+0000 7fea3f7fe640 1 -- 172.21.15.116:0/1842206623 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fea50018430 con 0x55b6f38be9d0 2024-09-12T03:03:21.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.912+0000 7fea68b8b740 10 client.15330 fetch_fsmap learned FSMap version 25 2024-09-12T03:03:21.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.912+0000 7fea68b8b740 1 -- 172.21.15.116:0/1842206623 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b6f38da300 con 0x55b6f38be9d0 2024-09-12T03:03:21.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.912+0000 7fea3f7fe640 1 -- 172.21.15.116:0/1842206623 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea4809b410 con 0x55b6f38be9d0 2024-09-12T03:03:21.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.913+0000 7fea3f7fe640 1 -- 172.21.15.116:0/1842206623 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fea50022510 con 0x55b6f38be9d0 2024-09-12T03:03:21.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.913+0000 7fea68b8b740 10 client.15330 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:03:21.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.913+0000 7fea68b8b740 10 client.15330 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T03:03:21.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.913+0000 7fea68b8b740 20 client.15330 populate_metadata read hostname 'smithi116' 2024-09-12T03:03:21.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.913+0000 7fea68b8b740 1 --2- 172.21.15.116:0/1842206623 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55b6f38dc8c0 0x55b6f38ddd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:21.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.913+0000 7fea68b8b740 4 client.15330 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:03:21.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.913+0000 7fea68b8b740 1 -- 172.21.15.116:0/1842206623 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6f3832600 con 0x55b6f38dc8c0 2024-09-12T03:03:21.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.913+0000 7fea5f7fe640 1 --2- 172.21.15.116:0/1842206623 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55b6f38dc8c0 0x55b6f38ddd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:21.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.914+0000 7fea3f7fe640 10 client.15330 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:03:21.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.914+0000 7fea5f7fe640 1 --2- 172.21.15.116:0/1842206623 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55b6f38dc8c0 0x55b6f38ddd30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9a72324914c04e11 server_cookie=e12c79c5d92efcb2 in_seq=0 out_seq=0 2024-09-12T03:03:21.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.914+0000 7fea3f7fe640 1 -- 172.21.15.116:0/1842206623 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b6f3832600 con 0x55b6f38dc8c0 2024-09-12T03:03:21.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.915+0000 7fea3f7fe640 10 client.15330 handle_command_reply: tid=0 2024-09-12T03:03:21.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.915+0000 7fea68b8b740 1 client.15330 shutdown 2024-09-12T03:03:21.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.915+0000 7fea68b8b740 1 -- 172.21.15.116:0/1842206623 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6f38be9d0 msgr2=0x55b6f38c8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:21.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.915+0000 7fea68b8b740 1 --2- 172.21.15.116:0/1842206623 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6f38be9d0 0x55b6f38c8bb0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fea50010740 tx=0x7fea50010c00 comp rx=0 tx=0).stop 2024-09-12T03:03:21.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.915+0000 7fea68b8b740 1 -- 172.21.15.116:0/1842206623 shutdown_connections 2024-09-12T03:03:21.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.915+0000 7fea68b8b740 1 --2- 172.21.15.116:0/1842206623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6f38c9bb0 0x55b6f38d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.915+0000 7fea68b8b740 1 --2- 172.21.15.116:0/1842206623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6f38bf2f0 0x55b6f38c90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.915+0000 7fea68b8b740 1 --2- 172.21.15.116:0/1842206623 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55b6f38dc8c0 0x55b6f38ddd30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.915+0000 7fea68b8b740 1 --2- 172.21.15.116:0/1842206623 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6f38be9d0 0x55b6f38c8bb0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.915+0000 7fea68b8b740 1 -- 172.21.15.116:0/1842206623 >> 172.21.15.116:0/1842206623 conn(0x55b6f38baf60 msgr2=0x55b6f38c0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:21.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.915+0000 7fea3f7fe640 0 client.15330 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:03:21.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.915+0000 7fea68b8b740 1 -- 172.21.15.116:0/1842206623 shutdown_connections 2024-09-12T03:03:21.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.915+0000 7fea68b8b740 1 -- 172.21.15.116:0/1842206623 wait complete. 2024-09-12T03:03:21.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.915+0000 7fea68b8b740 20 client.15330 trim_cache size 0 max 16384 2024-09-12T03:03:21.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.932+0000 7fea68b8b740 1 Processor -- start 2024-09-12T03:03:21.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.932+0000 7fea68b8b740 1 -- start start 2024-09-12T03:03:21.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.933+0000 7fea68b8b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6f38be9d0 0x55b6f38d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:21.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.933+0000 7fea68b8b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6f38bf2f0 0x55b6f38c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:21.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.933+0000 7fea68b8b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6f38c9bb0 0x55b6f38c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:21.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.933+0000 7fea68b8b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b6f38dc070 con 0x55b6f38bf2f0 2024-09-12T03:03:21.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.933+0000 7fea68b8b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b6f38dc740 con 0x55b6f38be9d0 2024-09-12T03:03:21.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.933+0000 7fea68b8b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b6f38dce10 con 0x55b6f38c9bb0 2024-09-12T03:03:21.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.933+0000 7fea5f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6f38bf2f0 0x55b6f38c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:21.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.933+0000 7fea5f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6f38bf2f0 0x55b6f38c8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44170/0 (socket says 172.21.15.116:44170) 2024-09-12T03:03:21.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.933+0000 7fea5f7fe640 1 -- 172.21.15.116:0/3457621690 learned_addr learned my addr 172.21.15.116:0/3457621690 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:21.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.933+0000 7fea5ffff640 1 --2- 172.21.15.116:0/3457621690 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6f38be9d0 0x55b6f38d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:21.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.933+0000 7fea64d4f640 1 --2- 172.21.15.116:0/3457621690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6f38c9bb0 0x55b6f38c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:21.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.933+0000 7fea5f7fe640 1 -- 172.21.15.116:0/3457621690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6f38c9bb0 msgr2=0x55b6f38c90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:21.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.933+0000 7fea5f7fe640 1 --2- 172.21.15.116:0/3457621690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6f38c9bb0 0x55b6f38c90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.933+0000 7fea5f7fe640 1 -- 172.21.15.116:0/3457621690 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6f38be9d0 msgr2=0x55b6f38d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:21.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.933+0000 7fea5f7fe640 1 --2- 172.21.15.116:0/3457621690 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6f38be9d0 0x55b6f38d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.933+0000 7fea5f7fe640 1 -- 172.21.15.116:0/3457621690 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6f38cc890 con 0x55b6f38bf2f0 2024-09-12T03:03:21.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.933+0000 7fea5ffff640 1 --2- 172.21.15.116:0/3457621690 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6f38be9d0 0x55b6f38d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:03:21.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.933+0000 7fea64d4f640 1 --2- 172.21.15.116:0/3457621690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6f38c9bb0 0x55b6f38c90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:03:21.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.933+0000 7fea5f7fe640 1 --2- 172.21.15.116:0/3457621690 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6f38bf2f0 0x55b6f38c8b90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fea4809aca0 tx=0x7fea480ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a8b1189564190b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:21.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.933+0000 7fea55ffb640 1 -- 172.21.15.116:0/3457621690 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4801e040 con 0x55b6f38bf2f0 2024-09-12T03:03:21.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.933+0000 7fea55ffb640 1 -- 172.21.15.116:0/3457621690 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea4809b410 con 0x55b6f38bf2f0 2024-09-12T03:03:21.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.933+0000 7fea55ffb640 1 -- 172.21.15.116:0/3457621690 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4809a850 con 0x55b6f38bf2f0 2024-09-12T03:03:21.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.934+0000 7fea68b8b740 1 -- 172.21.15.116:0/3457621690 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6f38bf2f0 msgr2=0x55b6f38c8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:21.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.934+0000 7fea68b8b740 1 --2- 172.21.15.116:0/3457621690 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6f38bf2f0 0x55b6f38c8b90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fea4809aca0 tx=0x7fea480ce040 comp rx=0 tx=0).stop 2024-09-12T03:03:21.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.934+0000 7fea68b8b740 1 -- 172.21.15.116:0/3457621690 shutdown_connections 2024-09-12T03:03:21.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.934+0000 7fea68b8b740 1 --2- 172.21.15.116:0/3457621690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6f38c9bb0 0x55b6f38c90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.934+0000 7fea68b8b740 1 --2- 172.21.15.116:0/3457621690 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6f38bf2f0 0x55b6f38c8b90 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.934+0000 7fea68b8b740 1 --2- 172.21.15.116:0/3457621690 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6f38be9d0 0x55b6f38d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.934+0000 7fea68b8b740 1 -- 172.21.15.116:0/3457621690 >> 172.21.15.116:0/3457621690 conn(0x55b6f38baf60 msgr2=0x55b6f38c7ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:21.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.934+0000 7fea68b8b740 1 -- 172.21.15.116:0/3457621690 shutdown_connections 2024-09-12T03:03:21.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.934+0000 7fea68b8b740 1 -- 172.21.15.116:0/3457621690 wait complete. 2024-09-12T03:03:21.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.934+0000 7fea68b8b740 1 -- start start 2024-09-12T03:03:21.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.935+0000 7fea68b8b740 1 Processor -- start 2024-09-12T03:03:21.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.935+0000 7fea68b8b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6f38be9d0 0x55b6f38d7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:21.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.935+0000 7fea68b8b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6f38bf2f0 0x55b6f38d0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:21.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.935+0000 7fea68b8b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6f38c9bb0 0x55b6f38d13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:21.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.935+0000 7fea68b8b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b6f38ddbb0 con 0x55b6f38bf2f0 2024-09-12T03:03:21.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.935+0000 7fea68b8b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b6f38dd4e0 con 0x55b6f38c9bb0 2024-09-12T03:03:21.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.935+0000 7fea68b8b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b6f38de280 con 0x55b6f38be9d0 2024-09-12T03:03:21.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.935+0000 7fea68b8b740 10 client.0 _ll_register_callbacks cb 0x7fea5000b620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:21.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.935+0000 7fea5f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6f38bf2f0 0x55b6f38d0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:21.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.935+0000 7fea5f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6f38bf2f0 0x55b6f38d0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44184/0 (socket says 172.21.15.116:44184) 2024-09-12T03:03:21.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.935+0000 7fea5f7fe640 1 -- 172.21.15.116:0/1743215654 learned_addr learned my addr 172.21.15.116:0/1743215654 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:21.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.935+0000 7fea5ffff640 1 --2- 172.21.15.116:0/1743215654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6f38be9d0 0x55b6f38d7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:21.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.935+0000 7fea64d4f640 1 --2- 172.21.15.116:0/1743215654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6f38c9bb0 0x55b6f38d13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:21.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.935+0000 7fea5f7fe640 1 -- 172.21.15.116:0/1743215654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6f38be9d0 msgr2=0x55b6f38d7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:21.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.935+0000 7fea5f7fe640 1 --2- 172.21.15.116:0/1743215654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6f38be9d0 0x55b6f38d7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.935+0000 7fea5f7fe640 1 -- 172.21.15.116:0/1743215654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6f38c9bb0 msgr2=0x55b6f38d13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:21.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.935+0000 7fea5f7fe640 1 --2- 172.21.15.116:0/1743215654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6f38c9bb0 0x55b6f38d13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.935+0000 7fea5f7fe640 1 -- 172.21.15.116:0/1743215654 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6f38d19b0 con 0x55b6f38bf2f0 2024-09-12T03:03:21.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.935+0000 7fea5f7fe640 1 --2- 172.21.15.116:0/1743215654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6f38bf2f0 0x55b6f38d0e80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fea480cbf00 tx=0x7fea480047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f974c43e5c308795 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:21.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.935+0000 7fea3f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:03:21.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.936+0000 7fea3f7fe640 1 -- 172.21.15.116:0/1743215654 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48077070 con 0x55b6f38bf2f0 2024-09-12T03:03:21.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.936+0000 7fea3f7fe640 1 -- 172.21.15.116:0/1743215654 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fea480cc100 con 0x55b6f38bf2f0 2024-09-12T03:03:21.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.936+0000 7fea68b8b740 1 -- 172.21.15.116:0/1743215654 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea4809b410 con 0x55b6f38bf2f0 2024-09-12T03:03:21.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.936+0000 7fea3f7fe640 1 -- 172.21.15.116:0/1743215654 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fea4801e050 con 0x55b6f38bf2f0 2024-09-12T03:03:21.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.936+0000 7fea68b8b740 10 client.15336 fetch_fsmap learned FSMap version 25 2024-09-12T03:03:21.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.936+0000 7fea68b8b740 1 -- 172.21.15.116:0/1743215654 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b6f3945d50 con 0x55b6f38bf2f0 2024-09-12T03:03:21.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.936+0000 7fea3f7fe640 1 -- 172.21.15.116:0/1743215654 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea4809b410 con 0x55b6f38bf2f0 2024-09-12T03:03:21.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.936+0000 7fea3f7fe640 1 -- 172.21.15.116:0/1743215654 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fea48097380 con 0x55b6f38bf2f0 2024-09-12T03:03:21.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.936+0000 7fea68b8b740 10 client.15336 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:03:21.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.936+0000 7fea68b8b740 10 client.15336 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T03:03:21.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.936+0000 7fea68b8b740 20 client.15336 populate_metadata read hostname 'smithi116' 2024-09-12T03:03:21.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.936+0000 7fea68b8b740 1 --2- 172.21.15.116:0/1743215654 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55b6f394a2c0 0x55b6f3948fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:21.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.936+0000 7fea68b8b740 4 client.15336 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:03:21.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.936+0000 7fea68b8b740 1 -- 172.21.15.116:0/1743215654 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "status"}) -- 0x55b6f382cb10 con 0x55b6f394a2c0 2024-09-12T03:03:21.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.937+0000 7fea5ffff640 1 --2- 172.21.15.116:0/1743215654 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55b6f394a2c0 0x55b6f3948fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:21.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.937+0000 7fea5ffff640 1 --2- 172.21.15.116:0/1743215654 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55b6f394a2c0 0x55b6f3948fd0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f09ca638b65abec6 server_cookie=dac91a0d2679e150 in_seq=0 out_seq=0 2024-09-12T03:03:21.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.937+0000 7fea3f7fe640 10 client.15336 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:03:21.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.937+0000 7fea3f7fe640 1 -- 172.21.15.116:0/1743215654 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b6f382cb10 con 0x55b6f394a2c0 2024-09-12T03:03:21.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.937+0000 7fea3f7fe640 10 client.15336 handle_command_reply: tid=0 2024-09-12T03:03:21.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.937+0000 7fea68b8b740 1 client.15336 shutdown 2024-09-12T03:03:21.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.937+0000 7fea68b8b740 1 -- 172.21.15.116:0/1743215654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6f38bf2f0 msgr2=0x55b6f38d0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:21.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.937+0000 7fea68b8b740 1 --2- 172.21.15.116:0/1743215654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6f38bf2f0 0x55b6f38d0e80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fea480cbf00 tx=0x7fea480047d0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.937+0000 7fea68b8b740 1 -- 172.21.15.116:0/1743215654 shutdown_connections 2024-09-12T03:03:21.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.937+0000 7fea68b8b740 1 --2- 172.21.15.116:0/1743215654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6f38c9bb0 0x55b6f38d13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.937+0000 7fea68b8b740 1 --2- 172.21.15.116:0/1743215654 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55b6f394a2c0 0x55b6f3948fd0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.937+0000 7fea68b8b740 1 --2- 172.21.15.116:0/1743215654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6f38bf2f0 0x55b6f38d0e80 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.937+0000 7fea68b8b740 1 --2- 172.21.15.116:0/1743215654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6f38be9d0 0x55b6f38d7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.937+0000 7fea68b8b740 1 -- 172.21.15.116:0/1743215654 >> 172.21.15.116:0/1743215654 conn(0x55b6f38baf60 msgr2=0x55b6f38c7ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:21.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.937+0000 7fea3f7fe640 0 client.15336 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:03:21.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.937+0000 7fea68b8b740 1 -- 172.21.15.116:0/1743215654 shutdown_connections 2024-09-12T03:03:21.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.938+0000 7fea68b8b740 1 -- 172.21.15.116:0/1743215654 wait complete. 2024-09-12T03:03:21.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.938+0000 7fea68b8b740 20 client.15336 trim_cache size 0 max 16384 2024-09-12T03:03:21.948 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:03:21.948 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_fsid": "eb1ac602-70b1-11ef-bcea-c7b262605968", 2024-09-12T03:03:21.948 INFO:teuthology.orchestra.run.smithi116.stdout: "whoami": 2, 2024-09-12T03:03:21.948 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 24467, 2024-09-12T03:03:21.948 INFO:teuthology.orchestra.run.smithi116.stdout: "want_state": "up:active", 2024-09-12T03:03:21.948 INFO:teuthology.orchestra.run.smithi116.stdout: "state": "up:active", 2024-09-12T03:03:21.948 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs", 2024-09-12T03:03:21.948 INFO:teuthology.orchestra.run.smithi116.stdout: "rank_uptime": 134.54806230200001, 2024-09-12T03:03:21.948 INFO:teuthology.orchestra.run.smithi116.stdout: "mdsmap_epoch": 25, 2024-09-12T03:03:21.949 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch": 80, 2024-09-12T03:03:21.949 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch_barrier": 80, 2024-09-12T03:03:21.949 INFO:teuthology.orchestra.run.smithi116.stdout: "uptime": 164.75996549300001 2024-09-12T03:03:21.949 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:03:21.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.938+0000 7fea667b2640 1 -- 172.21.15.116:0/1366155956 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fea4c0a2c10 msgr2=0x7fea4c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:21.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.938+0000 7fea667b2640 1 --2- 172.21.15.116:0/1366155956 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fea4c0a2c10 0x7fea4c0a50d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fea58009430 tx=0x7fea58009380 comp rx=0 tx=0).stop 2024-09-12T03:03:21.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.938+0000 7fea667b2640 1 -- 172.21.15.116:0/1366155956 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea60100e30 msgr2=0x7fea60197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:21.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.938+0000 7fea667b2640 1 --2- 172.21.15.116:0/1366155956 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea60100e30 0x7fea60197870 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fea4800b4a0 tx=0x7fea4800b960 comp rx=0 tx=0).stop 2024-09-12T03:03:21.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.938+0000 7fea667b2640 1 -- 172.21.15.116:0/1366155956 shutdown_connections 2024-09-12T03:03:21.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.938+0000 7fea667b2640 1 --2- 172.21.15.116:0/1366155956 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fea4c0a2c10 0x7fea4c0a50d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.938+0000 7fea667b2640 1 --2- 172.21.15.116:0/1366155956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea601055b0 0x7fea6019bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.938+0000 7fea667b2640 1 --2- 172.21.15.116:0/1366155956 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea60100e30 0x7fea60197870 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.938+0000 7fea667b2640 1 --2- 172.21.15.116:0/1366155956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea60100480 0x7fea60197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.938+0000 7fea667b2640 1 -- 172.21.15.116:0/1366155956 >> 172.21.15.116:0/1366155956 conn(0x7fea60077210 msgr2=0x7fea600fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:21.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.938+0000 7fea667b2640 1 -- 172.21.15.116:0/1366155956 shutdown_connections 2024-09-12T03:03:21.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:21.938+0000 7fea667b2640 1 -- 172.21.15.116:0/1366155956 wait complete. 2024-09-12T03:03:22.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:22 smithi177 ceph-mon[35305]: pgmap v349: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T03:03:22.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:22 smithi116 ceph-mon[31410]: pgmap v349: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T03:03:22.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:22 smithi183 ceph-mon[35535]: pgmap v349: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T03:03:22.839 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T03:03:23.090 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:23.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.570+0000 7f1330106640 1 -- 172.21.15.116:0/217105519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1328100a80 msgr2=0x7f1328100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:23.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.570+0000 7f1330106640 1 --2- 172.21.15.116:0/217105519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1328100a80 0x7f1328100ee0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f131c009930 tx=0x7f131c02a420 comp rx=0 tx=0).stop 2024-09-12T03:03:23.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.570+0000 7f1330106640 1 -- 172.21.15.116:0/217105519 shutdown_connections 2024-09-12T03:03:23.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.570+0000 7f1330106640 1 --2- 172.21.15.116:0/217105519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1328101420 0x7f132810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.570+0000 7f1330106640 1 --2- 172.21.15.116:0/217105519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1328100a80 0x7f1328100ee0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.570+0000 7f1330106640 1 --2- 172.21.15.116:0/217105519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1328106a80 0x7f1328106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.570+0000 7f1330106640 1 -- 172.21.15.116:0/217105519 >> 172.21.15.116:0/217105519 conn(0x7f13280fc7a0 msgr2=0x7f13280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:23.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.570+0000 7f1330106640 1 -- 172.21.15.116:0/217105519 shutdown_connections 2024-09-12T03:03:23.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.570+0000 7f1330106640 1 -- 172.21.15.116:0/217105519 wait complete. 2024-09-12T03:03:23.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.571+0000 7f1330106640 1 Processor -- start 2024-09-12T03:03:23.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.571+0000 7f1330106640 1 -- start start 2024-09-12T03:03:23.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.571+0000 7f1330106640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1328100a80 0x7f13280fcca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:23.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.571+0000 7f1330106640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1328101420 0x7f13280fd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:23.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.571+0000 7f1330106640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1328106a80 0x7f13280fd720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:23.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.571+0000 7f1330106640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1328110e80 con 0x7f1328100a80 2024-09-12T03:03:23.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.571+0000 7f1330106640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1328110d00 con 0x7f1328101420 2024-09-12T03:03:23.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.571+0000 7f1330106640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1328111000 con 0x7f1328106a80 2024-09-12T03:03:23.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.571+0000 7f132dea2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1328100a80 0x7f13280fcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:23.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.571+0000 7f132dea2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1328100a80 0x7f13280fcca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44196/0 (socket says 172.21.15.116:44196) 2024-09-12T03:03:23.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.571+0000 7f132dea2640 1 -- 172.21.15.116:0/216215048 learned_addr learned my addr 172.21.15.116:0/216215048 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:23.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.571+0000 7f132d6a1640 1 --2- 172.21.15.116:0/216215048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1328101420 0x7f13280fd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:23.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.571+0000 7f132e6a3640 1 --2- 172.21.15.116:0/216215048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1328106a80 0x7f13280fd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:23.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.571+0000 7f132dea2640 1 -- 172.21.15.116:0/216215048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1328106a80 msgr2=0x7f13280fd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:23.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.571+0000 7f132dea2640 1 --2- 172.21.15.116:0/216215048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1328106a80 0x7f13280fd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.571+0000 7f132dea2640 1 -- 172.21.15.116:0/216215048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1328101420 msgr2=0x7f13280fd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:23.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.571+0000 7f132dea2640 1 --2- 172.21.15.116:0/216215048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1328101420 0x7f13280fd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.571+0000 7f132dea2640 1 -- 172.21.15.116:0/216215048 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13281a6810 con 0x7f1328100a80 2024-09-12T03:03:23.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.572+0000 7f132dea2640 1 --2- 172.21.15.116:0/216215048 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1328100a80 0x7f13280fcca0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f132000c920 tx=0x7f132000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:23.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.572+0000 7f1326ffd640 1 -- 172.21.15.116:0/216215048 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320007c70 con 0x7f1328100a80 2024-09-12T03:03:23.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.572+0000 7f1326ffd640 1 -- 172.21.15.116:0/216215048 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1320007e10 con 0x7f1328100a80 2024-09-12T03:03:23.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.572+0000 7f1330106640 1 -- 172.21.15.116:0/216215048 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13281a6b00 con 0x7f1328100a80 2024-09-12T03:03:23.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.572+0000 7f1326ffd640 1 -- 172.21.15.116:0/216215048 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320018890 con 0x7f1328100a80 2024-09-12T03:03:23.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.572+0000 7f1330106640 1 -- 172.21.15.116:0/216215048 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13281a7060 con 0x7f1328100a80 2024-09-12T03:03:23.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.574+0000 7f1326ffd640 1 -- 172.21.15.116:0/216215048 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f1320018a70 con 0x7f1328100a80 2024-09-12T03:03:23.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.574+0000 7f13324df740 1 Processor -- start 2024-09-12T03:03:23.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.574+0000 7f13324df740 1 -- start start 2024-09-12T03:03:23.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.575+0000 7f13324df740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561cf6bc79d0 0x561cf6bc7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:23.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.575+0000 7f13324df740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561cf6bc82f0 0x561cf6bd2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:23.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.575+0000 7f13324df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cf6bd2bb0 0x561cf6bd4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:23.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.575+0000 7f13324df740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561cf6b2e090 con 0x561cf6bc79d0 2024-09-12T03:03:23.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.575+0000 7f13324df740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561cf6ad5a70 con 0x561cf6bc82f0 2024-09-12T03:03:23.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.575+0000 7f13324df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561cf6b2a050 con 0x561cf6bd2bb0 2024-09-12T03:03:23.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.575+0000 7f132e6a3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561cf6bc79d0 0x561cf6bc7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:23.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.575+0000 7f132e6a3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561cf6bc79d0 0x561cf6bc7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44212/0 (socket says 172.21.15.116:44212) 2024-09-12T03:03:23.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.575+0000 7f132e6a3640 1 -- 172.21.15.116:0/274080627 learned_addr learned my addr 172.21.15.116:0/274080627 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:23.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.575+0000 7f1326ffd640 1 --2- 172.21.15.116:0/216215048 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f13100a2c90 0x7f13100a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:23.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.575+0000 7f132dea2640 1 --2- 172.21.15.116:0/274080627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561cf6bc82f0 0x561cf6bd2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:23.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.575+0000 7f132e6a3640 1 --2- 172.21.15.116:0/216215048 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f13100a2c90 0x7f13100a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:23.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.575+0000 7f132d6a1640 1 --2- 172.21.15.116:0/274080627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cf6bd2bb0 0x561cf6bd4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:23.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.575+0000 7f132e6a3640 1 -- 172.21.15.116:0/274080627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cf6bd2bb0 msgr2=0x561cf6bd4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:23.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.575+0000 7f132e6a3640 1 --2- 172.21.15.116:0/274080627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cf6bd2bb0 0x561cf6bd4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.575+0000 7f132e6a3640 1 -- 172.21.15.116:0/274080627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561cf6bc82f0 msgr2=0x561cf6bd2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:23.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.575+0000 7f1326ffd640 1 -- 172.21.15.116:0/216215048 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f13200cf2a0 con 0x7f1328100a80 2024-09-12T03:03:23.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.575+0000 7f132e6a3640 1 --2- 172.21.15.116:0/274080627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561cf6bc82f0 0x561cf6bd2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.575+0000 7f132e6a3640 1 -- 172.21.15.116:0/274080627 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cf6ae2130 con 0x561cf6bc79d0 2024-09-12T03:03:23.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.575+0000 7f132dea2640 1 --2- 172.21.15.116:0/274080627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561cf6bc82f0 0x561cf6bd2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:03:23.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.575+0000 7f132e6a3640 1 --2- 172.21.15.116:0/274080627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561cf6bc79d0 0x561cf6bc7db0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1318012910 tx=0x7f1318012dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e0ec3b0be72435 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:23.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.576+0000 7f130ffff640 1 -- 172.21.15.116:0/274080627 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1318005020 con 0x561cf6bc79d0 2024-09-12T03:03:23.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.576+0000 7f130ffff640 1 -- 172.21.15.116:0/274080627 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13180056a0 con 0x561cf6bc79d0 2024-09-12T03:03:23.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.576+0000 7f130ffff640 1 -- 172.21.15.116:0/274080627 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131801c7f0 con 0x561cf6bc79d0 2024-09-12T03:03:23.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.576+0000 7f13324df740 1 -- 172.21.15.116:0/274080627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561cf6bc79d0 msgr2=0x561cf6bc7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:23.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.576+0000 7f132e6a3640 1 --2- 172.21.15.116:0/216215048 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f13100a2c90 0x7f13100a5150 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f131801f000 tx=0x7f131801e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:23.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.576+0000 7f13324df740 1 --2- 172.21.15.116:0/274080627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561cf6bc79d0 0x561cf6bc7db0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1318012910 tx=0x7f1318012dd0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.576+0000 7f13324df740 1 -- 172.21.15.116:0/274080627 shutdown_connections 2024-09-12T03:03:23.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.576+0000 7f13324df740 1 --2- 172.21.15.116:0/274080627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cf6bd2bb0 0x561cf6bd4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.576+0000 7f13324df740 1 --2- 172.21.15.116:0/274080627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561cf6bc82f0 0x561cf6bd2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.576+0000 7f13324df740 1 --2- 172.21.15.116:0/274080627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561cf6bc79d0 0x561cf6bc7db0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.576+0000 7f13324df740 1 -- 172.21.15.116:0/274080627 >> 172.21.15.116:0/274080627 conn(0x561cf6bc3f60 msgr2=0x561cf6bc4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:23.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.577+0000 7f13324df740 1 -- 172.21.15.116:0/274080627 shutdown_connections 2024-09-12T03:03:23.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.577+0000 7f13324df740 1 -- 172.21.15.116:0/274080627 wait complete. 2024-09-12T03:03:23.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.577+0000 7f13324df740 1 -- start start 2024-09-12T03:03:23.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.577+0000 7f13324df740 1 Processor -- start 2024-09-12T03:03:23.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.577+0000 7f13324df740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561cf6bc79d0 0x561cf6bd2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:23.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.577+0000 7f13324df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cf6bc82f0 0x561cf6bcf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:23.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.577+0000 7f13324df740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561cf6bd2bb0 0x561cf6bcfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:23.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.577+0000 7f13324df740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561cf6bd5780 con 0x561cf6bc79d0 2024-09-12T03:03:23.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.577+0000 7f13324df740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561cf6ae8f20 con 0x561cf6bd2bb0 2024-09-12T03:03:23.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.577+0000 7f13324df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561cf6bd5900 con 0x561cf6bc82f0 2024-09-12T03:03:23.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.577+0000 7f13324df740 10 client.0 _ll_register_callbacks cb 0x561cf6ab2030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:23.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.578+0000 7f132dea2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561cf6bc79d0 0x561cf6bd2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:23.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.578+0000 7f132dea2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561cf6bc79d0 0x561cf6bd2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44220/0 (socket says 172.21.15.116:44220) 2024-09-12T03:03:23.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.578+0000 7f132dea2640 1 -- 172.21.15.116:0/3540948285 learned_addr learned my addr 172.21.15.116:0/3540948285 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:23.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.578+0000 7f132e6a3640 1 --2- 172.21.15.116:0/3540948285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561cf6bd2bb0 0x561cf6bcfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:23.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.578+0000 7f132d6a1640 1 --2- 172.21.15.116:0/3540948285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cf6bc82f0 0x561cf6bcf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:23.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.578+0000 7f132dea2640 1 -- 172.21.15.116:0/3540948285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cf6bc82f0 msgr2=0x561cf6bcf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:23.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.578+0000 7f132dea2640 1 --2- 172.21.15.116:0/3540948285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cf6bc82f0 0x561cf6bcf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.578+0000 7f132dea2640 1 -- 172.21.15.116:0/3540948285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561cf6bd2bb0 msgr2=0x561cf6bcfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:23.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.578+0000 7f132dea2640 1 --2- 172.21.15.116:0/3540948285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561cf6bd2bb0 0x561cf6bcfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.578+0000 7f132dea2640 1 -- 172.21.15.116:0/3540948285 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cf689f2b0 con 0x561cf6bc79d0 2024-09-12T03:03:23.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.578+0000 7f132dea2640 1 --2- 172.21.15.116:0/3540948285 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561cf6bc79d0 0x561cf6bd2410 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f132000c8f0 tx=0x7f1320097d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3777b3d449a0751f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:23.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.578+0000 7f130d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:03:23.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.578+0000 7f130d7fa640 1 -- 172.21.15.116:0/3540948285 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132009b9d0 con 0x561cf6bc79d0 2024-09-12T03:03:23.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.578+0000 7f130d7fa640 1 -- 172.21.15.116:0/3540948285 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f13200cced0 con 0x561cf6bc79d0 2024-09-12T03:03:23.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.578+0000 7f13324df740 1 -- 172.21.15.116:0/3540948285 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13180056a0 con 0x561cf6bc79d0 2024-09-12T03:03:23.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.578+0000 7f130d7fa640 1 -- 172.21.15.116:0/3540948285 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f132009fd80 con 0x561cf6bc79d0 2024-09-12T03:03:23.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.578+0000 7f13324df740 10 client.15348 fetch_fsmap learned FSMap version 25 2024-09-12T03:03:23.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.578+0000 7f13324df740 1 -- 172.21.15.116:0/3540948285 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561cf6be1120 con 0x561cf6bc79d0 2024-09-12T03:03:23.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.578+0000 7f130d7fa640 1 -- 172.21.15.116:0/3540948285 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13180056a0 con 0x561cf6bc79d0 2024-09-12T03:03:23.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.579+0000 7f130d7fa640 1 -- 172.21.15.116:0/3540948285 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f132009ae20 con 0x561cf6bc79d0 2024-09-12T03:03:23.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.579+0000 7f13324df740 10 client.15348 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:03:23.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.579+0000 7f13324df740 10 client.15348 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T03:03:23.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.579+0000 7f13324df740 20 client.15348 populate_metadata read hostname 'smithi116' 2024-09-12T03:03:23.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.579+0000 7f13324df740 1 --2- 172.21.15.116:0/3540948285 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x561cf6be3630 0x561cf6be4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:23.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.579+0000 7f13324df740 4 client.15348 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:03:23.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.579+0000 7f13324df740 1 -- 172.21.15.116:0/3540948285 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cf6b3b600 con 0x561cf6be3630 2024-09-12T03:03:23.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.579+0000 7f132d6a1640 1 --2- 172.21.15.116:0/3540948285 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x561cf6be3630 0x561cf6be4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:23.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.579+0000 7f130d7fa640 10 client.15348 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:03:23.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.579+0000 7f132d6a1640 1 --2- 172.21.15.116:0/3540948285 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x561cf6be3630 0x561cf6be4aa0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=17533672d2eaefd2 server_cookie=6646438d04e25ee9 in_seq=0 out_seq=0 2024-09-12T03:03:23.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.580+0000 7f130d7fa640 1 -- 172.21.15.116:0/3540948285 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561cf6b3b600 con 0x561cf6be3630 2024-09-12T03:03:23.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.580+0000 7f130d7fa640 10 client.15348 handle_command_reply: tid=0 2024-09-12T03:03:23.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.580+0000 7f13324df740 1 client.15348 shutdown 2024-09-12T03:03:23.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.580+0000 7f13324df740 1 -- 172.21.15.116:0/3540948285 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561cf6bc79d0 msgr2=0x561cf6bd2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:23.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.580+0000 7f13324df740 1 --2- 172.21.15.116:0/3540948285 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561cf6bc79d0 0x561cf6bd2410 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f132000c8f0 tx=0x7f1320097d30 comp rx=0 tx=0).stop 2024-09-12T03:03:23.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.581+0000 7f13324df740 1 -- 172.21.15.116:0/3540948285 shutdown_connections 2024-09-12T03:03:23.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.581+0000 7f13324df740 1 --2- 172.21.15.116:0/3540948285 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x561cf6be3630 0x561cf6be4aa0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.581+0000 7f13324df740 1 --2- 172.21.15.116:0/3540948285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561cf6bd2bb0 0x561cf6bcfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.581+0000 7f13324df740 1 --2- 172.21.15.116:0/3540948285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cf6bc82f0 0x561cf6bcf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.581+0000 7f13324df740 1 --2- 172.21.15.116:0/3540948285 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561cf6bc79d0 0x561cf6bd2410 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.581+0000 7f130d7fa640 0 client.15348 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:03:23.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.581+0000 7f13324df740 1 -- 172.21.15.116:0/3540948285 >> 172.21.15.116:0/3540948285 conn(0x561cf6bc3f60 msgr2=0x561cf6bc9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:23.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.581+0000 7f13324df740 1 -- 172.21.15.116:0/3540948285 shutdown_connections 2024-09-12T03:03:23.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.581+0000 7f13324df740 1 -- 172.21.15.116:0/3540948285 wait complete. 2024-09-12T03:03:23.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.581+0000 7f13324df740 20 client.15348 trim_cache size 0 max 16384 2024-09-12T03:03:23.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.598+0000 7f13324df740 1 Processor -- start 2024-09-12T03:03:23.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.598+0000 7f13324df740 1 -- start start 2024-09-12T03:03:23.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.598+0000 7f13324df740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561cf6bc79d0 0x561cf6be4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:23.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.598+0000 7f13324df740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561cf6bc82f0 0x561cf6bcf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:23.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.598+0000 7f13324df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cf6bd2bb0 0x561cf6bcfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:23.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.598+0000 7f13324df740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561cf6be2de0 con 0x561cf6bc82f0 2024-09-12T03:03:23.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.598+0000 7f13324df740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561cf6be34b0 con 0x561cf6bc79d0 2024-09-12T03:03:23.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.598+0000 7f13324df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561cf6be3b80 con 0x561cf6bd2bb0 2024-09-12T03:03:23.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.598+0000 7f132d6a1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561cf6bc82f0 0x561cf6bcf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:23.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.598+0000 7f132d6a1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561cf6bc82f0 0x561cf6bcf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44226/0 (socket says 172.21.15.116:44226) 2024-09-12T03:03:23.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.598+0000 7f132d6a1640 1 -- 172.21.15.116:0/493370225 learned_addr learned my addr 172.21.15.116:0/493370225 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:23.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.598+0000 7f132dea2640 1 --2- 172.21.15.116:0/493370225 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561cf6bc79d0 0x561cf6be4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:23.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.599+0000 7f132e6a3640 1 --2- 172.21.15.116:0/493370225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cf6bd2bb0 0x561cf6bcfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:23.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.599+0000 7f132d6a1640 1 -- 172.21.15.116:0/493370225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cf6bd2bb0 msgr2=0x561cf6bcfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:23.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.599+0000 7f132d6a1640 1 --2- 172.21.15.116:0/493370225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cf6bd2bb0 0x561cf6bcfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.599+0000 7f132d6a1640 1 -- 172.21.15.116:0/493370225 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561cf6bc79d0 msgr2=0x561cf6be4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:23.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.599+0000 7f132d6a1640 1 --2- 172.21.15.116:0/493370225 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561cf6bc79d0 0x561cf6be4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.599+0000 7f132d6a1640 1 -- 172.21.15.116:0/493370225 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cf6bd5890 con 0x561cf6bc82f0 2024-09-12T03:03:23.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.599+0000 7f132e6a3640 1 --2- 172.21.15.116:0/493370225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cf6bd2bb0 0x561cf6bcfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:03:23.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.599+0000 7f132d6a1640 1 --2- 172.21.15.116:0/493370225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561cf6bc82f0 0x561cf6bcf720 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f131c02a940 tx=0x7f131c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a13b5b4dfd6e56 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:23.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.599+0000 7f130f7fe640 1 -- 172.21.15.116:0/493370225 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c048070 con 0x561cf6bc82f0 2024-09-12T03:03:23.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.599+0000 7f130f7fe640 1 -- 172.21.15.116:0/493370225 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f131c004240 con 0x561cf6bc82f0 2024-09-12T03:03:23.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.599+0000 7f130f7fe640 1 -- 172.21.15.116:0/493370225 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c035040 con 0x561cf6bc82f0 2024-09-12T03:03:23.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.600+0000 7f13324df740 1 -- 172.21.15.116:0/493370225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561cf6bc82f0 msgr2=0x561cf6bcf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:23.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.600+0000 7f13324df740 1 --2- 172.21.15.116:0/493370225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561cf6bc82f0 0x561cf6bcf720 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f131c02a940 tx=0x7f131c005d50 comp rx=0 tx=0).stop 2024-09-12T03:03:23.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.600+0000 7f13324df740 1 -- 172.21.15.116:0/493370225 shutdown_connections 2024-09-12T03:03:23.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.600+0000 7f13324df740 1 --2- 172.21.15.116:0/493370225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cf6bd2bb0 0x561cf6bcfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.600+0000 7f13324df740 1 --2- 172.21.15.116:0/493370225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561cf6bc82f0 0x561cf6bcf720 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.600+0000 7f13324df740 1 --2- 172.21.15.116:0/493370225 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561cf6bc79d0 0x561cf6be4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.600+0000 7f13324df740 1 -- 172.21.15.116:0/493370225 >> 172.21.15.116:0/493370225 conn(0x561cf6bc3f60 msgr2=0x561cf6be25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:23.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.600+0000 7f13324df740 1 -- 172.21.15.116:0/493370225 shutdown_connections 2024-09-12T03:03:23.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.600+0000 7f13324df740 1 -- 172.21.15.116:0/493370225 wait complete. 2024-09-12T03:03:23.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.600+0000 7f13324df740 1 -- start start 2024-09-12T03:03:23.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.600+0000 7f13324df740 1 Processor -- start 2024-09-12T03:03:23.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.601+0000 7f13324df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cf6bc79d0 0x561cf6c45fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:23.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.601+0000 7f13324df740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561cf6bc82f0 0x561cf6c46500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:23.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.601+0000 7f13324df740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561cf6bd2bb0 0x561cf6be82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:23.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.601+0000 7f13324df740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561cf6be4920 con 0x561cf6bd2bb0 2024-09-12T03:03:23.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.601+0000 7f13324df740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561cf6be4250 con 0x561cf6bc82f0 2024-09-12T03:03:23.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.601+0000 7f13324df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561cf6be4ff0 con 0x561cf6bc79d0 2024-09-12T03:03:23.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.601+0000 7f13324df740 10 client.0 _ll_register_callbacks cb 0x7f132000fe30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:23.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.601+0000 7f132e6a3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561cf6bd2bb0 0x561cf6be82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:23.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.601+0000 7f132dea2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cf6bc79d0 0x561cf6c45fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:23.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.601+0000 7f132e6a3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561cf6bd2bb0 0x561cf6be82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44240/0 (socket says 172.21.15.116:44240) 2024-09-12T03:03:23.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.601+0000 7f132e6a3640 1 -- 172.21.15.116:0/4012994662 learned_addr learned my addr 172.21.15.116:0/4012994662 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:23.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.601+0000 7f132d6a1640 1 --2- 172.21.15.116:0/4012994662 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561cf6bc82f0 0x561cf6c46500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:23.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.601+0000 7f132dea2640 1 -- 172.21.15.116:0/4012994662 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561cf6bc82f0 msgr2=0x561cf6c46500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:23.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.601+0000 7f132dea2640 1 --2- 172.21.15.116:0/4012994662 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561cf6bc82f0 0x561cf6c46500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.601+0000 7f132dea2640 1 -- 172.21.15.116:0/4012994662 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561cf6bd2bb0 msgr2=0x561cf6be82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:23.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.601+0000 7f132dea2640 1 --2- 172.21.15.116:0/4012994662 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561cf6bd2bb0 0x561cf6be82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.601+0000 7f132dea2640 1 -- 172.21.15.116:0/4012994662 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cf6be89c0 con 0x561cf6bc79d0 2024-09-12T03:03:23.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.601+0000 7f132d6a1640 1 --2- 172.21.15.116:0/4012994662 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561cf6bc82f0 0x561cf6c46500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:23.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.601+0000 7f132e6a3640 1 --2- 172.21.15.116:0/4012994662 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561cf6bd2bb0 0x561cf6be82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:23.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.601+0000 7f132dea2640 1 --2- 172.21.15.116:0/4012994662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cf6bc79d0 0x561cf6c45fc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f132009b6d0 tx=0x7f1320000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74eec9a07e352a15 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:23.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.601+0000 7f130d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T03:03:23.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.602+0000 7f130d7fa640 1 -- 172.21.15.116:0/4012994662 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132009b510 con 0x561cf6bc79d0 2024-09-12T03:03:23.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.602+0000 7f130d7fa640 1 -- 172.21.15.116:0/4012994662 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1320097830 con 0x561cf6bc79d0 2024-09-12T03:03:23.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.602+0000 7f13324df740 1 -- 172.21.15.116:0/4012994662 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f131c004240 con 0x561cf6bc79d0 2024-09-12T03:03:23.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.602+0000 7f130d7fa640 1 -- 172.21.15.116:0/4012994662 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f132009b7a0 con 0x561cf6bc79d0 2024-09-12T03:03:23.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.602+0000 7f13324df740 10 client.24880 fetch_fsmap learned FSMap version 25 2024-09-12T03:03:23.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.602+0000 7f13324df740 1 -- 172.21.15.116:0/4012994662 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561cf6c4d790 con 0x561cf6bc79d0 2024-09-12T03:03:23.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.602+0000 7f130d7fa640 1 -- 172.21.15.116:0/4012994662 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f131c004240 con 0x561cf6bc79d0 2024-09-12T03:03:23.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.602+0000 7f130d7fa640 1 -- 172.21.15.116:0/4012994662 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f13200ba1a0 con 0x561cf6bc79d0 2024-09-12T03:03:23.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.602+0000 7f13324df740 10 client.24880 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:03:23.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.602+0000 7f13324df740 10 client.24880 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T03:03:23.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.602+0000 7f13324df740 20 client.24880 populate_metadata read hostname 'smithi116' 2024-09-12T03:03:23.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.602+0000 7f13324df740 1 --2- 172.21.15.116:0/4012994662 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x561cf6c523b0 0x561cf6c510c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:23.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.602+0000 7f13324df740 4 client.24880 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:03:23.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.602+0000 7f13324df740 1 -- 172.21.15.116:0/4012994662 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "status"}) -- 0x561cf6b35b10 con 0x561cf6c523b0 2024-09-12T03:03:23.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.602+0000 7f132d6a1640 1 --2- 172.21.15.116:0/4012994662 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x561cf6c523b0 0x561cf6c510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:23.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.603+0000 7f130d7fa640 10 client.24880 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:03:23.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.603+0000 7f132d6a1640 1 --2- 172.21.15.116:0/4012994662 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x561cf6c523b0 0x561cf6c510c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fa1f5e595a5ce512 server_cookie=cfcfa3d952e10f06 in_seq=0 out_seq=0 2024-09-12T03:03:23.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.603+0000 7f130d7fa640 1 -- 172.21.15.116:0/4012994662 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x561cf6b35b10 con 0x561cf6c523b0 2024-09-12T03:03:23.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.603+0000 7f130d7fa640 10 client.24880 handle_command_reply: tid=0 2024-09-12T03:03:23.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.603+0000 7f13324df740 1 client.24880 shutdown 2024-09-12T03:03:23.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.603+0000 7f13324df740 1 -- 172.21.15.116:0/4012994662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cf6bc79d0 msgr2=0x561cf6c45fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:23.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.603+0000 7f13324df740 1 --2- 172.21.15.116:0/4012994662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cf6bc79d0 0x561cf6c45fc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f132009b6d0 tx=0x7f1320000bc0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.603+0000 7f13324df740 1 -- 172.21.15.116:0/4012994662 shutdown_connections 2024-09-12T03:03:23.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.603+0000 7f13324df740 1 --2- 172.21.15.116:0/4012994662 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x561cf6c523b0 0x561cf6c510c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.603+0000 7f13324df740 1 --2- 172.21.15.116:0/4012994662 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561cf6bd2bb0 0x561cf6be82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.603+0000 7f130d7fa640 0 client.24880 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:03:23.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.603+0000 7f13324df740 1 --2- 172.21.15.116:0/4012994662 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561cf6bc82f0 0x561cf6c46500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.603+0000 7f13324df740 1 --2- 172.21.15.116:0/4012994662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cf6bc79d0 0x561cf6c45fc0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.603+0000 7f13324df740 1 -- 172.21.15.116:0/4012994662 >> 172.21.15.116:0/4012994662 conn(0x561cf6bc3f60 msgr2=0x561cf6bc4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:23.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.603+0000 7f13324df740 1 -- 172.21.15.116:0/4012994662 shutdown_connections 2024-09-12T03:03:23.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.603+0000 7f13324df740 1 -- 172.21.15.116:0/4012994662 wait complete. 2024-09-12T03:03:23.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.603+0000 7f13324df740 20 client.24880 trim_cache size 0 max 16384 2024-09-12T03:03:23.625 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:03:23.626 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_fsid": "eb1ac602-70b1-11ef-bcea-c7b262605968", 2024-09-12T03:03:23.626 INFO:teuthology.orchestra.run.smithi116.stdout: "whoami": 1, 2024-09-12T03:03:23.626 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 24475, 2024-09-12T03:03:23.626 INFO:teuthology.orchestra.run.smithi116.stdout: "want_state": "up:active", 2024-09-12T03:03:23.626 INFO:teuthology.orchestra.run.smithi116.stdout: "state": "up:active", 2024-09-12T03:03:23.626 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs", 2024-09-12T03:03:23.626 INFO:teuthology.orchestra.run.smithi116.stdout: "rank_uptime": 137.22233020900001, 2024-09-12T03:03:23.626 INFO:teuthology.orchestra.run.smithi116.stdout: "mdsmap_epoch": 25, 2024-09-12T03:03:23.626 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch": 80, 2024-09-12T03:03:23.626 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch_barrier": 80, 2024-09-12T03:03:23.626 INFO:teuthology.orchestra.run.smithi116.stdout: "uptime": 163.77080253099999 2024-09-12T03:03:23.627 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:03:23.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.604+0000 7f1330106640 1 -- 172.21.15.116:0/216215048 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f13100a2c90 msgr2=0x7f13100a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:23.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.604+0000 7f1330106640 1 --2- 172.21.15.116:0/216215048 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f13100a2c90 0x7f13100a5150 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f131801f000 tx=0x7f131801e000 comp rx=0 tx=0).stop 2024-09-12T03:03:23.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.604+0000 7f1330106640 1 -- 172.21.15.116:0/216215048 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1328100a80 msgr2=0x7f13280fcca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:23.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.604+0000 7f1330106640 1 --2- 172.21.15.116:0/216215048 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1328100a80 0x7f13280fcca0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f132000c920 tx=0x7f132000cde0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.604+0000 7f1330106640 1 -- 172.21.15.116:0/216215048 shutdown_connections 2024-09-12T03:03:23.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.604+0000 7f1330106640 1 --2- 172.21.15.116:0/216215048 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f13100a2c90 0x7f13100a5150 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.604+0000 7f1330106640 1 --2- 172.21.15.116:0/216215048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1328106a80 0x7f13280fd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.604+0000 7f1330106640 1 --2- 172.21.15.116:0/216215048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1328101420 0x7f13280fd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.604+0000 7f1330106640 1 --2- 172.21.15.116:0/216215048 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1328100a80 0x7f13280fcca0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:23.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.604+0000 7f1330106640 1 -- 172.21.15.116:0/216215048 >> 172.21.15.116:0/216215048 conn(0x7f13280fc7a0 msgr2=0x7f132806a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:23.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.604+0000 7f1330106640 1 -- 172.21.15.116:0/216215048 shutdown_connections 2024-09-12T03:03:23.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:23.604+0000 7f1330106640 1 -- 172.21.15.116:0/216215048 wait complete. 2024-09-12T03:03:24.407 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T03:03:24.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:24 smithi177 ceph-mon[35305]: pgmap v350: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-12T03:03:24.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:24 smithi116 ceph-mon[31410]: pgmap v350: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-12T03:03:24.654 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:24.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:24 smithi183 ceph-mon[35535]: pgmap v350: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-12T03:03:25.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.149+0000 7f44fd609640 1 -- 172.21.15.116:0/1591086351 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44f8101e10 msgr2=0x7f44f81021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:25.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.149+0000 7f44fd609640 1 --2- 172.21.15.116:0/1591086351 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44f8101e10 0x7f44f81021f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f44e8009ca0 tx=0x7f44e802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.149+0000 7f44fd609640 1 -- 172.21.15.116:0/1591086351 shutdown_connections 2024-09-12T03:03:25.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.149+0000 7f44fd609640 1 --2- 172.21.15.116:0/1591086351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44f8106f40 0x7f44f810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.149+0000 7f44fd609640 1 --2- 172.21.15.116:0/1591086351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44f81027c0 0x7f44f8106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.149+0000 7f44fd609640 1 --2- 172.21.15.116:0/1591086351 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44f8101e10 0x7f44f81021f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.149+0000 7f44fd609640 1 -- 172.21.15.116:0/1591086351 >> 172.21.15.116:0/1591086351 conn(0x7f44f80fd8d0 msgr2=0x7f44f80ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:25.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.149+0000 7f44fd609640 1 -- 172.21.15.116:0/1591086351 shutdown_connections 2024-09-12T03:03:25.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.149+0000 7f44fd609640 1 -- 172.21.15.116:0/1591086351 wait complete. 2024-09-12T03:03:25.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.150+0000 7f44fd609640 1 Processor -- start 2024-09-12T03:03:25.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.150+0000 7f44fd609640 1 -- start start 2024-09-12T03:03:25.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.150+0000 7f44fd609640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44f8101e10 0x7f44f81984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:25.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.150+0000 7f44fd609640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44f81027c0 0x7f44f81989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:25.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.150+0000 7f44fd609640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44f8106f40 0x7f44f819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:25.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.150+0000 7f44fd609640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f44f8111fc0 con 0x7f44f8106f40 2024-09-12T03:03:25.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.150+0000 7f44fd609640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f44f8111e40 con 0x7f44f81027c0 2024-09-12T03:03:25.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.151+0000 7f44fd609640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f44f8112140 con 0x7f44f8101e10 2024-09-12T03:03:25.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.151+0000 7f44f77fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44f8106f40 0x7f44f819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:25.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.151+0000 7f44f77fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44f8106f40 0x7f44f819cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44254/0 (socket says 172.21.15.116:44254) 2024-09-12T03:03:25.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.151+0000 7f44f77fe640 1 -- 172.21.15.116:0/235038046 learned_addr learned my addr 172.21.15.116:0/235038046 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:25.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.151+0000 7f44f67fc640 1 --2- 172.21.15.116:0/235038046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44f81027c0 0x7f44f81989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:25.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.151+0000 7f44f6ffd640 1 --2- 172.21.15.116:0/235038046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44f8101e10 0x7f44f81984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:25.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.151+0000 7f44f77fe640 1 -- 172.21.15.116:0/235038046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44f8101e10 msgr2=0x7f44f81984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:25.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.151+0000 7f44f77fe640 1 --2- 172.21.15.116:0/235038046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44f8101e10 0x7f44f81984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.151+0000 7f44f77fe640 1 -- 172.21.15.116:0/235038046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44f81027c0 msgr2=0x7f44f81989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:25.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.151+0000 7f44f77fe640 1 --2- 172.21.15.116:0/235038046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44f81027c0 0x7f44f81989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.151+0000 7f44f77fe640 1 -- 172.21.15.116:0/235038046 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f819d450 con 0x7f44f8106f40 2024-09-12T03:03:25.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.151+0000 7f44f77fe640 1 --2- 172.21.15.116:0/235038046 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44f8106f40 0x7f44f819cd70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f44f000c970 tx=0x7f44f000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:25.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.151+0000 7f44effff640 1 -- 172.21.15.116:0/235038046 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f000ae60 con 0x7f44f8106f40 2024-09-12T03:03:25.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.151+0000 7f44effff640 1 -- 172.21.15.116:0/235038046 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44f0005500 con 0x7f44f8106f40 2024-09-12T03:03:25.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.151+0000 7f44fd609640 1 -- 172.21.15.116:0/235038046 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44f819d740 con 0x7f44f8106f40 2024-09-12T03:03:25.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.151+0000 7f44effff640 1 -- 172.21.15.116:0/235038046 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f000f450 con 0x7f44f8106f40 2024-09-12T03:03:25.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.151+0000 7f44fd609640 1 -- 172.21.15.116:0/235038046 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44f81a5020 con 0x7f44f8106f40 2024-09-12T03:03:25.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.153+0000 7f44effff640 1 -- 172.21.15.116:0/235038046 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f44f0004ee0 con 0x7f44f8106f40 2024-09-12T03:03:25.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.154+0000 7f44ff9e2740 1 Processor -- start 2024-09-12T03:03:25.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.154+0000 7f44ff9e2740 1 -- start start 2024-09-12T03:03:25.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.154+0000 7f44ff9e2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7edad1890 0x55a7edad1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:25.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.154+0000 7f44ff9e2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7edad21b0 0x55a7edae0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:25.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.154+0000 7f44ff9e2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7edae0fc0 0x55a7edae33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:25.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.154+0000 7f44ff9e2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a7eda38090 con 0x55a7edad21b0 2024-09-12T03:03:25.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.154+0000 7f44ff9e2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a7ed9dfa70 con 0x55a7edae0fc0 2024-09-12T03:03:25.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.154+0000 7f44ff9e2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a7eda34050 con 0x55a7edad1890 2024-09-12T03:03:25.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.154+0000 7f44effff640 1 --2- 172.21.15.116:0/235038046 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f44e40a2c10 0x7f44e40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:25.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.154+0000 7f44f6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7edad21b0 0x55a7edae0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:25.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.154+0000 7f44f6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7edad21b0 0x55a7edae0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44266/0 (socket says 172.21.15.116:44266) 2024-09-12T03:03:25.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.154+0000 7f44f6ffd640 1 -- 172.21.15.116:0/2944788160 learned_addr learned my addr 172.21.15.116:0/2944788160 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:25.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.154+0000 7f44f77fe640 1 --2- 172.21.15.116:0/235038046 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f44e40a2c10 0x7f44e40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:25.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.155+0000 7f44effff640 1 -- 172.21.15.116:0/235038046 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f44f00ceea0 con 0x7f44f8106f40 2024-09-12T03:03:25.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.155+0000 7f44f67fc640 1 --2- 172.21.15.116:0/2944788160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7edae0fc0 0x55a7edae33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:25.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.155+0000 7f44f6ffd640 1 -- 172.21.15.116:0/2944788160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7edad1890 msgr2=0x55a7edad1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:25.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.155+0000 7f44f67fc640 1 --2- 172.21.15.116:0/2944788160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7edad1890 0x55a7edad1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:25.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.155+0000 7f44f6ffd640 1 --2- 172.21.15.116:0/2944788160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7edad1890 0x55a7edad1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.155+0000 7f44f6ffd640 1 -- 172.21.15.116:0/2944788160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7edae0fc0 msgr2=0x55a7edae33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:25.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.155+0000 7f44f6ffd640 1 --2- 172.21.15.116:0/2944788160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7edae0fc0 0x55a7edae33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.155+0000 7f44f6ffd640 1 -- 172.21.15.116:0/2944788160 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7ed9ec130 con 0x55a7edad21b0 2024-09-12T03:03:25.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.155+0000 7f44f67fc640 1 --2- 172.21.15.116:0/2944788160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7edad1890 0x55a7edad1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:03:25.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.155+0000 7f44f77fe640 1 --2- 172.21.15.116:0/235038046 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f44e40a2c10 0x7f44e40a50d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f44f009b2d0 tx=0x7f44f0004940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:25.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.155+0000 7f44f6ffd640 1 --2- 172.21.15.116:0/2944788160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7edad21b0 0x55a7edae0a80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f44e8005cc0 tx=0x7f44e8009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d051ff9c313246 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:25.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.155+0000 7f44ed7fa640 1 -- 172.21.15.116:0/2944788160 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e8003970 con 0x55a7edad21b0 2024-09-12T03:03:25.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.155+0000 7f44ed7fa640 1 -- 172.21.15.116:0/2944788160 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44e8036050 con 0x55a7edad21b0 2024-09-12T03:03:25.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.155+0000 7f44ed7fa640 1 -- 172.21.15.116:0/2944788160 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e80043e0 con 0x55a7edad21b0 2024-09-12T03:03:25.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.156+0000 7f44ff9e2740 1 -- 172.21.15.116:0/2944788160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7edad21b0 msgr2=0x55a7edae0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:25.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.156+0000 7f44ff9e2740 1 --2- 172.21.15.116:0/2944788160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7edad21b0 0x55a7edae0a80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f44e8005cc0 tx=0x7f44e8009930 comp rx=0 tx=0).stop 2024-09-12T03:03:25.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.156+0000 7f44ff9e2740 1 -- 172.21.15.116:0/2944788160 shutdown_connections 2024-09-12T03:03:25.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.156+0000 7f44ff9e2740 1 --2- 172.21.15.116:0/2944788160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7edae0fc0 0x55a7edae33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.156+0000 7f44ff9e2740 1 --2- 172.21.15.116:0/2944788160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7edad21b0 0x55a7edae0a80 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.156+0000 7f44ff9e2740 1 --2- 172.21.15.116:0/2944788160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7edad1890 0x55a7edad1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.156+0000 7f44ff9e2740 1 -- 172.21.15.116:0/2944788160 >> 172.21.15.116:0/2944788160 conn(0x55a7edacdf60 msgr2=0x55a7edace340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:25.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.156+0000 7f44ff9e2740 1 -- 172.21.15.116:0/2944788160 shutdown_connections 2024-09-12T03:03:25.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.156+0000 7f44ff9e2740 1 -- 172.21.15.116:0/2944788160 wait complete. 2024-09-12T03:03:25.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.156+0000 7f44ff9e2740 1 -- start start 2024-09-12T03:03:25.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.156+0000 7f44ff9e2740 1 Processor -- start 2024-09-12T03:03:25.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.157+0000 7f44ff9e2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7edad1890 0x55a7edae0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:25.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.157+0000 7f44ff9e2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7edad21b0 0x55a7edaddb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:25.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.157+0000 7f44ff9e2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7edae0fc0 0x55a7edade0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:25.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.157+0000 7f44ff9e2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a7edae39b0 con 0x55a7edad21b0 2024-09-12T03:03:25.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.157+0000 7f44ff9e2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a7ed9f2f20 con 0x55a7edae0fc0 2024-09-12T03:03:25.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.157+0000 7f44ff9e2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a7edae3b30 con 0x55a7edad1890 2024-09-12T03:03:25.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.157+0000 7f44ff9e2740 10 client.0 _ll_register_callbacks cb 0x55a7ed9bc030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:25.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.157+0000 7f44f6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7edad21b0 0x55a7edaddb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:25.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.157+0000 7f44f6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7edad21b0 0x55a7edaddb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44270/0 (socket says 172.21.15.116:44270) 2024-09-12T03:03:25.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.157+0000 7f44f6ffd640 1 -- 172.21.15.116:0/944017904 learned_addr learned my addr 172.21.15.116:0/944017904 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:25.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.157+0000 7f44f67fc640 1 --2- 172.21.15.116:0/944017904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7edad1890 0x55a7edae0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:25.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.157+0000 7f44f67fc640 1 --2- 172.21.15.116:0/944017904 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7edae0fc0 0x55a7edade0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:25.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.157+0000 7f44f6ffd640 1 -- 172.21.15.116:0/944017904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7edad1890 msgr2=0x55a7edae0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:25.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.157+0000 7f44f6ffd640 1 --2- 172.21.15.116:0/944017904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7edad1890 0x55a7edae0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.157+0000 7f44f6ffd640 1 -- 172.21.15.116:0/944017904 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7edae0fc0 msgr2=0x55a7edade0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:25.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.157+0000 7f44f6ffd640 1 --2- 172.21.15.116:0/944017904 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7edae0fc0 0x55a7edade0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.157+0000 7f44f6ffd640 1 -- 172.21.15.116:0/944017904 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7ed7a92b0 con 0x55a7edad21b0 2024-09-12T03:03:25.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.157+0000 7f44f67fc640 1 --2- 172.21.15.116:0/944017904 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7edae0fc0 0x55a7edade0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:03:25.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.157+0000 7f44f6ffd640 1 --2- 172.21.15.116:0/944017904 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7edad21b0 0x55a7edaddb70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f44e802a900 tx=0x7f44e8031930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab7851d122a8567d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:25.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.157+0000 7f44de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:03:25.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.158+0000 7f44de7fc640 1 -- 172.21.15.116:0/944017904 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e8003c40 con 0x55a7edad21b0 2024-09-12T03:03:25.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.158+0000 7f44de7fc640 1 -- 172.21.15.116:0/944017904 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f44e8003de0 con 0x55a7edad21b0 2024-09-12T03:03:25.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.158+0000 7f44ff9e2740 1 -- 172.21.15.116:0/944017904 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44e8036050 con 0x55a7edad21b0 2024-09-12T03:03:25.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.158+0000 7f44de7fc640 1 -- 172.21.15.116:0/944017904 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f44e8004120 con 0x55a7edad21b0 2024-09-12T03:03:25.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.158+0000 7f44ff9e2740 10 client.15366 fetch_fsmap learned FSMap version 25 2024-09-12T03:03:25.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.158+0000 7f44ff9e2740 1 -- 172.21.15.116:0/944017904 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a7edae39b0 con 0x55a7edad21b0 2024-09-12T03:03:25.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.158+0000 7f44de7fc640 1 -- 172.21.15.116:0/944017904 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44e8036050 con 0x55a7edad21b0 2024-09-12T03:03:25.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.158+0000 7f44de7fc640 1 -- 172.21.15.116:0/944017904 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f44e8056030 con 0x55a7edad21b0 2024-09-12T03:03:25.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.158+0000 7f44ff9e2740 10 client.15366 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:03:25.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.158+0000 7f44ff9e2740 10 client.15366 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:03:25.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.158+0000 7f44ff9e2740 20 client.15366 populate_metadata read hostname 'smithi116' 2024-09-12T03:03:25.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.158+0000 7f44ff9e2740 1 --2- 172.21.15.116:0/944017904 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a7edaf1600 0x55a7edaf2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:25.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.158+0000 7f44ff9e2740 4 client.15366 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:03:25.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.158+0000 7f44ff9e2740 1 -- 172.21.15.116:0/944017904 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7eda45600 con 0x55a7edaf1600 2024-09-12T03:03:25.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.159+0000 7f44f67fc640 1 --2- 172.21.15.116:0/944017904 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a7edaf1600 0x55a7edaf2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:25.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.159+0000 7f44f67fc640 1 --2- 172.21.15.116:0/944017904 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a7edaf1600 0x55a7edaf2a70 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81680f7aa8027931 server_cookie=75501c6f67b7074f in_seq=0 out_seq=0 2024-09-12T03:03:25.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.159+0000 7f44de7fc640 10 client.15366 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:25.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.160+0000 7f44de7fc640 1 -- 172.21.15.116:0/944017904 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a7eda45600 con 0x55a7edaf1600 2024-09-12T03:03:25.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.160+0000 7f44de7fc640 10 client.15366 handle_command_reply: tid=0 2024-09-12T03:03:25.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.160+0000 7f44ff9e2740 1 client.15366 shutdown 2024-09-12T03:03:25.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.160+0000 7f44ff9e2740 1 -- 172.21.15.116:0/944017904 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7edad21b0 msgr2=0x55a7edaddb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:25.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.160+0000 7f44ff9e2740 1 --2- 172.21.15.116:0/944017904 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7edad21b0 0x55a7edaddb70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f44e802a900 tx=0x7f44e8031930 comp rx=0 tx=0).stop 2024-09-12T03:03:25.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.160+0000 7f44ff9e2740 1 -- 172.21.15.116:0/944017904 shutdown_connections 2024-09-12T03:03:25.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.160+0000 7f44ff9e2740 1 --2- 172.21.15.116:0/944017904 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a7edaf1600 0x55a7edaf2a70 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.160+0000 7f44ff9e2740 1 --2- 172.21.15.116:0/944017904 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7edae0fc0 0x55a7edade0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.160+0000 7f44ff9e2740 1 --2- 172.21.15.116:0/944017904 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7edad21b0 0x55a7edaddb70 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.160+0000 7f44ff9e2740 1 --2- 172.21.15.116:0/944017904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7edad1890 0x55a7edae0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.160+0000 7f44ff9e2740 1 -- 172.21.15.116:0/944017904 >> 172.21.15.116:0/944017904 conn(0x55a7edacdf60 msgr2=0x55a7edad3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:25.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.160+0000 7f44de7fc640 0 client.15366 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:25.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.160+0000 7f44ff9e2740 1 -- 172.21.15.116:0/944017904 shutdown_connections 2024-09-12T03:03:25.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.160+0000 7f44ff9e2740 1 -- 172.21.15.116:0/944017904 wait complete. 2024-09-12T03:03:25.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.160+0000 7f44ff9e2740 20 client.15366 trim_cache size 0 max 16384 2024-09-12T03:03:25.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.177+0000 7f44ff9e2740 1 Processor -- start 2024-09-12T03:03:25.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.177+0000 7f44ff9e2740 1 -- start start 2024-09-12T03:03:25.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.178+0000 7f44ff9e2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7edad1890 0x55a7edaf1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:25.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.178+0000 7f44ff9e2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7edad21b0 0x55a7edad8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:25.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.178+0000 7f44ff9e2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7edae0fc0 0x55a7edad8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:25.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.178+0000 7f44ff9e2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a7edaf0db0 con 0x55a7edae0fc0 2024-09-12T03:03:25.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.178+0000 7f44ff9e2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a7edaf1480 con 0x55a7edad1890 2024-09-12T03:03:25.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.178+0000 7f44ff9e2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a7edaf1b50 con 0x55a7edad21b0 2024-09-12T03:03:25.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.178+0000 7f44f67fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7edae0fc0 0x55a7edad8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:25.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.178+0000 7f44f67fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7edae0fc0 0x55a7edad8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44282/0 (socket says 172.21.15.116:44282) 2024-09-12T03:03:25.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.178+0000 7f44f67fc640 1 -- 172.21.15.116:0/3622656227 learned_addr learned my addr 172.21.15.116:0/3622656227 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:25.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.178+0000 7f44f67fc640 1 --2- 172.21.15.116:0/3622656227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7edad1890 0x55a7edaf1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:25.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.178+0000 7f44f6ffd640 1 --2- 172.21.15.116:0/3622656227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7edad21b0 0x55a7edad8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:25.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.178+0000 7f44f67fc640 1 -- 172.21.15.116:0/3622656227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7edad21b0 msgr2=0x55a7edad8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:25.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.178+0000 7f44f67fc640 1 --2- 172.21.15.116:0/3622656227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7edad21b0 0x55a7edad8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.178+0000 7f44f67fc640 1 -- 172.21.15.116:0/3622656227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7edad1890 msgr2=0x55a7edaf1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:25.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.178+0000 7f44f67fc640 1 --2- 172.21.15.116:0/3622656227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7edad1890 0x55a7edaf1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.178+0000 7f44f67fc640 1 -- 172.21.15.116:0/3622656227 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7edacdd70 con 0x55a7edae0fc0 2024-09-12T03:03:25.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.178+0000 7f44f67fc640 1 --2- 172.21.15.116:0/3622656227 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7edae0fc0 0x55a7edad8f60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f44e00165f0 tx=0x7f44e0009470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c717e080c68b80 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:25.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.178+0000 7f44ecff9640 1 -- 172.21.15.116:0/3622656227 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0024020 con 0x55a7edae0fc0 2024-09-12T03:03:25.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.179+0000 7f44ecff9640 1 -- 172.21.15.116:0/3622656227 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44e0015b00 con 0x55a7edae0fc0 2024-09-12T03:03:25.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.179+0000 7f44ecff9640 1 -- 172.21.15.116:0/3622656227 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e001ea30 con 0x55a7edae0fc0 2024-09-12T03:03:25.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.179+0000 7f44ff9e2740 1 -- 172.21.15.116:0/3622656227 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7edae0fc0 msgr2=0x55a7edad8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:25.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.179+0000 7f44ff9e2740 1 --2- 172.21.15.116:0/3622656227 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7edae0fc0 0x55a7edad8f60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f44e00165f0 tx=0x7f44e0009470 comp rx=0 tx=0).stop 2024-09-12T03:03:25.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.179+0000 7f44ff9e2740 1 -- 172.21.15.116:0/3622656227 shutdown_connections 2024-09-12T03:03:25.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.179+0000 7f44ff9e2740 1 --2- 172.21.15.116:0/3622656227 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7edae0fc0 0x55a7edad8f60 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.179+0000 7f44ff9e2740 1 --2- 172.21.15.116:0/3622656227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7edad21b0 0x55a7edad8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.179+0000 7f44ff9e2740 1 --2- 172.21.15.116:0/3622656227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7edad1890 0x55a7edaf1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.179+0000 7f44ff9e2740 1 -- 172.21.15.116:0/3622656227 >> 172.21.15.116:0/3622656227 conn(0x55a7edacdf60 msgr2=0x55a7edac8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:25.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.179+0000 7f44ff9e2740 1 -- 172.21.15.116:0/3622656227 shutdown_connections 2024-09-12T03:03:25.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.179+0000 7f44ff9e2740 1 -- 172.21.15.116:0/3622656227 wait complete. 2024-09-12T03:03:25.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.179+0000 7f44ff9e2740 1 -- start start 2024-09-12T03:03:25.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.180+0000 7f44ff9e2740 1 Processor -- start 2024-09-12T03:03:25.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.180+0000 7f44ff9e2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7edad1890 0x55a7edafaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:25.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.180+0000 7f44ff9e2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7edad21b0 0x55a7edafb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:25.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.180+0000 7f44ff9e2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7edae0fc0 0x55a7edaf4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:25.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.180+0000 7f44ff9e2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a7edaf28f0 con 0x55a7edad21b0 2024-09-12T03:03:25.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.180+0000 7f44ff9e2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a7edaf2220 con 0x55a7edae0fc0 2024-09-12T03:03:25.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.180+0000 7f44ff9e2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a7edaf2fc0 con 0x55a7edad1890 2024-09-12T03:03:25.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.180+0000 7f44ff9e2740 10 client.0 _ll_register_callbacks cb 0x7f44e8009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:25.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.180+0000 7f44f6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7edad21b0 0x55a7edafb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:25.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.180+0000 7f44f6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7edad21b0 0x55a7edafb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44284/0 (socket says 172.21.15.116:44284) 2024-09-12T03:03:25.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.180+0000 7f44f6ffd640 1 -- 172.21.15.116:0/628181678 learned_addr learned my addr 172.21.15.116:0/628181678 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:25.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.180+0000 7f44f67fc640 1 --2- 172.21.15.116:0/628181678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7edad1890 0x55a7edafaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:25.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.180+0000 7f44f6ffd640 1 -- 172.21.15.116:0/628181678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7edad1890 msgr2=0x55a7edafaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:25.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.180+0000 7f44f6ffd640 1 --2- 172.21.15.116:0/628181678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7edad1890 0x55a7edafaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.180+0000 7f44f6ffd640 1 -- 172.21.15.116:0/628181678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7edae0fc0 msgr2=0x55a7edaf4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:25.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.180+0000 7f44f67fc640 1 --2- 172.21.15.116:0/628181678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7edae0fc0 0x55a7edaf4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:25.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.180+0000 7f44f6ffd640 1 --2- 172.21.15.116:0/628181678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7edae0fc0 0x55a7edaf4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.180+0000 7f44f6ffd640 1 -- 172.21.15.116:0/628181678 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7edaf4ab0 con 0x55a7edad21b0 2024-09-12T03:03:25.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.180+0000 7f44f6ffd640 1 --2- 172.21.15.116:0/628181678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7edad21b0 0x55a7edafb3f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f44e802fde0 tx=0x7f44e8031c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123257f68590e512 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:25.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.180+0000 7f44de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:03:25.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.181+0000 7f44de7fc640 1 -- 172.21.15.116:0/628181678 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e802f310 con 0x55a7edad21b0 2024-09-12T03:03:25.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.181+0000 7f44de7fc640 1 -- 172.21.15.116:0/628181678 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f44e80305c0 con 0x55a7edad21b0 2024-09-12T03:03:25.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.181+0000 7f44ff9e2740 1 -- 172.21.15.116:0/628181678 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44e0015b00 con 0x55a7edad21b0 2024-09-12T03:03:25.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.181+0000 7f44de7fc640 1 -- 172.21.15.116:0/628181678 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f44e8009540 con 0x55a7edad21b0 2024-09-12T03:03:25.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.181+0000 7f44ff9e2740 10 client.15372 fetch_fsmap learned FSMap version 25 2024-09-12T03:03:25.208 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:03:25.208 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_fsid": "eb1ac602-70b1-11ef-bcea-c7b262605968", 2024-09-12T03:03:25.208 INFO:teuthology.orchestra.run.smithi116.stdout: "whoami": 0, 2024-09-12T03:03:25.208 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 24494, 2024-09-12T03:03:25.208 INFO:teuthology.orchestra.run.smithi116.stdout: "want_state": "up:active", 2024-09-12T03:03:25.209 INFO:teuthology.orchestra.run.smithi116.stdout: "state": "up:active", 2024-09-12T03:03:25.209 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs", 2024-09-12T03:03:25.209 INFO:teuthology.orchestra.run.smithi116.stdout: "rank_uptime": 145.84677750700001, 2024-09-12T03:03:25.209 INFO:teuthology.orchestra.run.smithi116.stdout: "mdsmap_epoch": 25, 2024-09-12T03:03:25.209 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch": 80, 2024-09-12T03:03:25.209 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch_barrier": 80, 2024-09-12T03:03:25.209 INFO:teuthology.orchestra.run.smithi116.stdout: "uptime": 156.482734295 2024-09-12T03:03:25.209 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:03:25.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.181+0000 7f44ff9e2740 1 -- 172.21.15.116:0/628181678 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a7edaf9420 con 0x55a7edad21b0 2024-09-12T03:03:25.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.181+0000 7f44de7fc640 1 -- 172.21.15.116:0/628181678 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44e0015b00 con 0x55a7edad21b0 2024-09-12T03:03:25.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.181+0000 7f44de7fc640 1 -- 172.21.15.116:0/628181678 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f44e8056040 con 0x55a7edad21b0 2024-09-12T03:03:25.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.181+0000 7f44ff9e2740 10 client.15372 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:03:25.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.181+0000 7f44ff9e2740 10 client.15372 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:03:25.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.181+0000 7f44ff9e2740 20 client.15372 populate_metadata read hostname 'smithi116' 2024-09-12T03:03:25.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.181+0000 7f44ff9e2740 1 --2- 172.21.15.116:0/628181678 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a7edb60280 0x55a7edb5ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:25.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.181+0000 7f44ff9e2740 4 client.15372 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:03:25.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.181+0000 7f44ff9e2740 1 -- 172.21.15.116:0/628181678 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "status"}) -- 0x55a7eda3fb10 con 0x55a7edb60280 2024-09-12T03:03:25.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.181+0000 7f44f67fc640 1 --2- 172.21.15.116:0/628181678 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a7edb60280 0x55a7edb5ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:25.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.182+0000 7f44de7fc640 10 client.15372 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:25.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.182+0000 7f44f67fc640 1 --2- 172.21.15.116:0/628181678 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a7edb60280 0x55a7edb5ef90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b6deea02ee379af server_cookie=722720e80527f866 in_seq=0 out_seq=0 2024-09-12T03:03:25.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.182+0000 7f44de7fc640 1 -- 172.21.15.116:0/628181678 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a7eda3fb10 con 0x55a7edb60280 2024-09-12T03:03:25.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.182+0000 7f44de7fc640 10 client.15372 handle_command_reply: tid=0 2024-09-12T03:03:25.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.182+0000 7f44ff9e2740 1 client.15372 shutdown 2024-09-12T03:03:25.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.182+0000 7f44ff9e2740 1 -- 172.21.15.116:0/628181678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7edad21b0 msgr2=0x55a7edafb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:25.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.182+0000 7f44ff9e2740 1 --2- 172.21.15.116:0/628181678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7edad21b0 0x55a7edafb3f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f44e802fde0 tx=0x7f44e8031c40 comp rx=0 tx=0).stop 2024-09-12T03:03:25.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.182+0000 7f44ff9e2740 1 -- 172.21.15.116:0/628181678 shutdown_connections 2024-09-12T03:03:25.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.182+0000 7f44ff9e2740 1 --2- 172.21.15.116:0/628181678 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a7edb60280 0x55a7edb5ef90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.182+0000 7f44ff9e2740 1 --2- 172.21.15.116:0/628181678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7edae0fc0 0x55a7edaf4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.182+0000 7f44de7fc640 0 client.15372 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:25.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.182+0000 7f44ff9e2740 1 --2- 172.21.15.116:0/628181678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7edad21b0 0x55a7edafb3f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.182+0000 7f44ff9e2740 1 --2- 172.21.15.116:0/628181678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7edad1890 0x55a7edafaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.182+0000 7f44ff9e2740 1 -- 172.21.15.116:0/628181678 >> 172.21.15.116:0/628181678 conn(0x55a7edacdf60 msgr2=0x55a7edad8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:25.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.182+0000 7f44ff9e2740 1 -- 172.21.15.116:0/628181678 shutdown_connections 2024-09-12T03:03:25.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.182+0000 7f44ff9e2740 1 -- 172.21.15.116:0/628181678 wait complete. 2024-09-12T03:03:25.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.182+0000 7f44ff9e2740 20 client.15372 trim_cache size 0 max 16384 2024-09-12T03:03:25.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.183+0000 7f44fd609640 1 -- 172.21.15.116:0/235038046 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f44e40a2c10 msgr2=0x7f44e40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:25.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.183+0000 7f44fd609640 1 --2- 172.21.15.116:0/235038046 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f44e40a2c10 0x7f44e40a50d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f44f009b2d0 tx=0x7f44f0004940 comp rx=0 tx=0).stop 2024-09-12T03:03:25.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.183+0000 7f44fd609640 1 -- 172.21.15.116:0/235038046 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44f8106f40 msgr2=0x7f44f819cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:25.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.183+0000 7f44fd609640 1 --2- 172.21.15.116:0/235038046 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44f8106f40 0x7f44f819cd70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f44f000c970 tx=0x7f44f000cd70 comp rx=0 tx=0).stop 2024-09-12T03:03:25.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.183+0000 7f44fd609640 1 -- 172.21.15.116:0/235038046 shutdown_connections 2024-09-12T03:03:25.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.183+0000 7f44fd609640 1 --2- 172.21.15.116:0/235038046 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f44e40a2c10 0x7f44e40a50d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.183+0000 7f44fd609640 1 --2- 172.21.15.116:0/235038046 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44f8106f40 0x7f44f819cd70 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.183+0000 7f44fd609640 1 --2- 172.21.15.116:0/235038046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44f81027c0 0x7f44f81989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.183+0000 7f44fd609640 1 --2- 172.21.15.116:0/235038046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44f8101e10 0x7f44f81984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:25.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.183+0000 7f44fd609640 1 -- 172.21.15.116:0/235038046 >> 172.21.15.116:0/235038046 conn(0x7f44f80fd8d0 msgr2=0x7f44f80fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:25.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.183+0000 7f44fd609640 1 -- 172.21.15.116:0/235038046 shutdown_connections 2024-09-12T03:03:25.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:25.183+0000 7f44fd609640 1 -- 172.21.15.116:0/235038046 wait complete. 2024-09-12T03:03:25.944 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-12T03:03:26.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:26 smithi177 ceph-mon[35305]: pgmap v351: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-12T03:03:26.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:26 smithi116 ceph-mon[31410]: pgmap v351: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-12T03:03:26.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:26 smithi183 ceph-mon[35535]: pgmap v351: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-12T03:03:28.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:28 smithi177 ceph-mon[35305]: pgmap v352: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-12T03:03:28.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:28 smithi116 ceph-mon[31410]: pgmap v352: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-12T03:03:28.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:28 smithi183 ceph-mon[35535]: pgmap v352: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-12T03:03:30.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:30 smithi177 ceph-mon[35305]: pgmap v353: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-12T03:03:30.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:30 smithi116 ceph-mon[31410]: pgmap v353: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-12T03:03:30.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:30 smithi183 ceph-mon[35535]: pgmap v353: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-12T03:03:32.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:32 smithi177 ceph-mon[35305]: pgmap v354: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-12T03:03:32.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:32 smithi116 ceph-mon[31410]: pgmap v354: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-12T03:03:32.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:32 smithi183 ceph-mon[35535]: pgmap v354: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-12T03:03:34.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:34 smithi177 ceph-mon[35305]: pgmap v355: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:03:34.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:34 smithi116 ceph-mon[31410]: pgmap v355: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:03:34.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:34 smithi183 ceph-mon[35535]: pgmap v355: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:03:36.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:36 smithi177 ceph-mon[35305]: pgmap v356: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:03:36.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:36 smithi116 ceph-mon[31410]: pgmap v356: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:03:36.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:36 smithi183 ceph-mon[35535]: pgmap v356: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:03:38.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:38 smithi177 ceph-mon[35305]: pgmap v357: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:03:38.511 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:38 smithi116 ceph-mon[31410]: pgmap v357: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:03:38.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:38 smithi183 ceph-mon[35535]: pgmap v357: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:03:39.738 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:03:40.007 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:40.479 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:40 smithi116 ceph-mon[31410]: pgmap v358: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:03:40.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:40 smithi177 ceph-mon[35305]: pgmap v358: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:03:40.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.563+0000 7f245e20a640 1 -- 172.21.15.116:0/2198886272 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f245810c150 msgr2=0x7f245810e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:40.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.563+0000 7f245e20a640 1 --2- 172.21.15.116:0/2198886272 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f245810c150 0x7f245810e5f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2450009ca0 tx=0x7f245002a950 comp rx=0 tx=0).stop 2024-09-12T03:03:40.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.563+0000 7f245e20a640 1 -- 172.21.15.116:0/2198886272 shutdown_connections 2024-09-12T03:03:40.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.563+0000 7f245e20a640 1 --2- 172.21.15.116:0/2198886272 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f245810c150 0x7f245810e5f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.563+0000 7f245e20a640 1 --2- 172.21.15.116:0/2198886272 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2458103690 0x7f245810bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.563+0000 7f245e20a640 1 --2- 172.21.15.116:0/2198886272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2458102d70 0x7f2458103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.563+0000 7f245e20a640 1 -- 172.21.15.116:0/2198886272 >> 172.21.15.116:0/2198886272 conn(0x7f24580fc790 msgr2=0x7f24580febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:40.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.563+0000 7f245e20a640 1 -- 172.21.15.116:0/2198886272 shutdown_connections 2024-09-12T03:03:40.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.563+0000 7f245e20a640 1 -- 172.21.15.116:0/2198886272 wait complete. 2024-09-12T03:03:40.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.563+0000 7f245e20a640 1 Processor -- start 2024-09-12T03:03:40.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.564+0000 7f245e20a640 1 -- start start 2024-09-12T03:03:40.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.564+0000 7f245e20a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2458102d70 0x7f2458198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:40.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.564+0000 7f245e20a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2458103690 0x7f2458198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:40.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.564+0000 7f245e20a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f245810c150 0x7f245819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:40.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.564+0000 7f245e20a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2458111f50 con 0x7f245810c150 2024-09-12T03:03:40.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.564+0000 7f245e20a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2458111dd0 con 0x7f2458103690 2024-09-12T03:03:40.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.564+0000 7f245e20a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f24581120d0 con 0x7f2458102d70 2024-09-12T03:03:40.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.564+0000 7f2457fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f245810c150 0x7f245819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:40.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.564+0000 7f2457fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f245810c150 0x7f245819cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58714/0 (socket says 172.21.15.116:58714) 2024-09-12T03:03:40.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.564+0000 7f2457fff640 1 -- 172.21.15.116:0/3154099962 learned_addr learned my addr 172.21.15.116:0/3154099962 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:40.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.564+0000 7f24577fe640 1 --2- 172.21.15.116:0/3154099962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2458102d70 0x7f2458198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:40.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.564+0000 7f2456ffd640 1 --2- 172.21.15.116:0/3154099962 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2458103690 0x7f2458198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:40.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.564+0000 7f2457fff640 1 -- 172.21.15.116:0/3154099962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2458102d70 msgr2=0x7f2458198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:40.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.564+0000 7f2457fff640 1 --2- 172.21.15.116:0/3154099962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2458102d70 0x7f2458198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.564+0000 7f2457fff640 1 -- 172.21.15.116:0/3154099962 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2458103690 msgr2=0x7f2458198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:40.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.564+0000 7f2457fff640 1 --2- 172.21.15.116:0/3154099962 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2458103690 0x7f2458198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.564+0000 7f2457fff640 1 -- 172.21.15.116:0/3154099962 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f245819d3e0 con 0x7f245810c150 2024-09-12T03:03:40.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.564+0000 7f2456ffd640 1 --2- 172.21.15.116:0/3154099962 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2458103690 0x7f2458198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:40.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.565+0000 7f2457fff640 1 --2- 172.21.15.116:0/3154099962 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f245810c150 0x7f245819cd00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f245002ae70 tx=0x7f245002f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:40.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.565+0000 7f2454ff9640 1 -- 172.21.15.116:0/3154099962 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2450034040 con 0x7f245810c150 2024-09-12T03:03:40.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.565+0000 7f245e20a640 1 -- 172.21.15.116:0/3154099962 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f245819d670 con 0x7f245810c150 2024-09-12T03:03:40.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.565+0000 7f2454ff9640 1 -- 172.21.15.116:0/3154099962 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2450030920 con 0x7f245810c150 2024-09-12T03:03:40.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.565+0000 7f2454ff9640 1 -- 172.21.15.116:0/3154099962 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2450045480 con 0x7f245810c150 2024-09-12T03:03:40.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.565+0000 7f245e20a640 1 -- 172.21.15.116:0/3154099962 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24581a4fb0 con 0x7f245810c150 2024-09-12T03:03:40.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.568+0000 7f24605e3740 1 Processor -- start 2024-09-12T03:03:40.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.568+0000 7f2454ff9640 1 -- 172.21.15.116:0/3154099962 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f2450037020 con 0x7f245810c150 2024-09-12T03:03:40.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.568+0000 7f24605e3740 1 -- start start 2024-09-12T03:03:40.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.568+0000 7f24605e3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3c165a3a0 0x55b3c165a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:40.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.568+0000 7f24605e3740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3c165acc0 0x55b3c1669550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:40.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.568+0000 7f24605e3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3c1669a90 0x55b3c166be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:40.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.568+0000 7f24605e3740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b3c15c02e0 con 0x55b3c165acc0 2024-09-12T03:03:40.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.568+0000 7f24605e3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b3c1567ce0 con 0x55b3c1669a90 2024-09-12T03:03:40.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.568+0000 7f24605e3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b3c15bc2a0 con 0x55b3c165a3a0 2024-09-12T03:03:40.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.568+0000 7f24577fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3c165acc0 0x55b3c1669550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:40.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.568+0000 7f24577fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3c165acc0 0x55b3c1669550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58726/0 (socket says 172.21.15.116:58726) 2024-09-12T03:03:40.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.568+0000 7f24577fe640 1 -- 172.21.15.116:0/3862290482 learned_addr learned my addr 172.21.15.116:0/3862290482 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:40.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.569+0000 7f2456ffd640 1 --2- 172.21.15.116:0/3862290482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3c165a3a0 0x55b3c165a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:40.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.569+0000 7f2456ffd640 1 --2- 172.21.15.116:0/3862290482 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3c1669a90 0x55b3c166be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:40.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.569+0000 7f2454ff9640 1 --2- 172.21.15.116:0/3154099962 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f24400a2c10 0x7f24400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:40.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.569+0000 7f24577fe640 1 -- 172.21.15.116:0/3862290482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3c165a3a0 msgr2=0x55b3c165a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:40.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.569+0000 7f24577fe640 1 --2- 172.21.15.116:0/3862290482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3c165a3a0 0x55b3c165a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.569+0000 7f24577fe640 1 -- 172.21.15.116:0/3862290482 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3c1669a90 msgr2=0x55b3c166be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:40.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.569+0000 7f24577fe640 1 --2- 172.21.15.116:0/3862290482 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3c1669a90 0x55b3c166be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.569+0000 7f24577fe640 1 -- 172.21.15.116:0/3862290482 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3c154c440 con 0x55b3c165acc0 2024-09-12T03:03:40.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.569+0000 7f2454ff9640 1 -- 172.21.15.116:0/3154099962 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f24500eff40 con 0x7f245810c150 2024-09-12T03:03:40.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.569+0000 7f2457fff640 1 --2- 172.21.15.116:0/3154099962 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f24400a2c10 0x7f24400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:40.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.569+0000 7f24577fe640 1 --2- 172.21.15.116:0/3862290482 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3c165acc0 0x55b3c1669550 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f244800b760 tx=0x7f244800bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee692be53be0a5e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:40.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.569+0000 7f243dffb640 1 -- 172.21.15.116:0/3862290482 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2448003d50 con 0x55b3c165acc0 2024-09-12T03:03:40.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.569+0000 7f243dffb640 1 -- 172.21.15.116:0/3862290482 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2448015070 con 0x55b3c165acc0 2024-09-12T03:03:40.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.569+0000 7f243dffb640 1 -- 172.21.15.116:0/3862290482 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2448004e00 con 0x55b3c165acc0 2024-09-12T03:03:40.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.569+0000 7f2457fff640 1 --2- 172.21.15.116:0/3154099962 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f24400a2c10 0x7f24400a50d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f245008f810 tx=0x7f245002a950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:40.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.570+0000 7f24605e3740 1 -- 172.21.15.116:0/3862290482 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3c165acc0 msgr2=0x55b3c1669550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:40.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.570+0000 7f24605e3740 1 --2- 172.21.15.116:0/3862290482 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3c165acc0 0x55b3c1669550 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f244800b760 tx=0x7f244800bc20 comp rx=0 tx=0).stop 2024-09-12T03:03:40.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.570+0000 7f24605e3740 1 -- 172.21.15.116:0/3862290482 shutdown_connections 2024-09-12T03:03:40.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.570+0000 7f24605e3740 1 --2- 172.21.15.116:0/3862290482 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3c1669a90 0x55b3c166be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.570+0000 7f24605e3740 1 --2- 172.21.15.116:0/3862290482 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3c165acc0 0x55b3c1669550 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.570+0000 7f24605e3740 1 --2- 172.21.15.116:0/3862290482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3c165a3a0 0x55b3c165a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.570+0000 7f24605e3740 1 -- 172.21.15.116:0/3862290482 >> 172.21.15.116:0/3862290482 conn(0x55b3c1656a30 msgr2=0x55b3c1656e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:40.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.570+0000 7f24605e3740 1 -- 172.21.15.116:0/3862290482 shutdown_connections 2024-09-12T03:03:40.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.570+0000 7f24605e3740 1 -- 172.21.15.116:0/3862290482 wait complete. 2024-09-12T03:03:40.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.571+0000 7f24605e3740 1 -- start start 2024-09-12T03:03:40.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.571+0000 7f24605e3740 1 Processor -- start 2024-09-12T03:03:40.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.571+0000 7f24605e3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3c165a3a0 0x55b3c1669450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:40.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.571+0000 7f24605e3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3c165acc0 0x55b3c1666700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:40.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.571+0000 7f24605e3740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3c1669a90 0x55b3c1666cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:40.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.571+0000 7f24605e3740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b3c166c6c0 con 0x55b3c1669a90 2024-09-12T03:03:40.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.571+0000 7f24605e3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b3c157b050 con 0x55b3c165a3a0 2024-09-12T03:03:40.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.571+0000 7f24605e3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b3c166c840 con 0x55b3c165acc0 2024-09-12T03:03:40.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.571+0000 7f24605e3740 10 client.0 _ll_register_callbacks cb 0x55b3c15bb160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:40.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.571+0000 7f2456ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3c1669a90 0x55b3c1666cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:40.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.571+0000 7f2456ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3c1669a90 0x55b3c1666cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58732/0 (socket says 172.21.15.116:58732) 2024-09-12T03:03:40.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.571+0000 7f2456ffd640 1 -- 172.21.15.116:0/2185231278 learned_addr learned my addr 172.21.15.116:0/2185231278 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:40.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.572+0000 7f2456ffd640 1 --2- 172.21.15.116:0/2185231278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3c165a3a0 0x55b3c1669450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:40.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.572+0000 7f24577fe640 1 --2- 172.21.15.116:0/2185231278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3c165acc0 0x55b3c1666700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:40.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.572+0000 7f2456ffd640 1 -- 172.21.15.116:0/2185231278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3c165acc0 msgr2=0x55b3c1666700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:40.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.572+0000 7f2456ffd640 1 --2- 172.21.15.116:0/2185231278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3c165acc0 0x55b3c1666700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.572+0000 7f2456ffd640 1 -- 172.21.15.116:0/2185231278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3c165a3a0 msgr2=0x55b3c1669450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:40.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.572+0000 7f2456ffd640 1 --2- 172.21.15.116:0/2185231278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3c165a3a0 0x55b3c1669450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.572+0000 7f2456ffd640 1 -- 172.21.15.116:0/2185231278 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3c13313c0 con 0x55b3c1669a90 2024-09-12T03:03:40.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.572+0000 7f24577fe640 1 --2- 172.21.15.116:0/2185231278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3c165acc0 0x55b3c1666700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:03:40.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.572+0000 7f2456ffd640 1 --2- 172.21.15.116:0/2185231278 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3c1669a90 0x55b3c1666cf0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f244c012c10 tx=0x7f244c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66e379702521526 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:40.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.572+0000 7f2436ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:03:40.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.572+0000 7f2436ffd640 1 -- 172.21.15.116:0/2185231278 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c010da0 con 0x55b3c1669a90 2024-09-12T03:03:40.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.572+0000 7f2436ffd640 1 -- 172.21.15.116:0/2185231278 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f244c004020 con 0x55b3c1669a90 2024-09-12T03:03:40.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.572+0000 7f24605e3740 1 -- 172.21.15.116:0/2185231278 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2448015070 con 0x55b3c1669a90 2024-09-12T03:03:40.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.572+0000 7f2436ffd640 1 -- 172.21.15.116:0/2185231278 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f244c028070 con 0x55b3c1669a90 2024-09-12T03:03:40.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.572+0000 7f24605e3740 10 client.15384 fetch_fsmap learned FSMap version 25 2024-09-12T03:03:40.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.572+0000 7f24605e3740 1 -- 172.21.15.116:0/2185231278 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b3c1678180 con 0x55b3c1669a90 2024-09-12T03:03:40.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.573+0000 7f2436ffd640 1 -- 172.21.15.116:0/2185231278 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2448015070 con 0x55b3c1669a90 2024-09-12T03:03:40.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.573+0000 7f2436ffd640 1 -- 172.21.15.116:0/2185231278 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f244c01e070 con 0x55b3c1669a90 2024-09-12T03:03:40.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.573+0000 7f24605e3740 10 client.15384 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:03:40.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.573+0000 7f24605e3740 10 client.15384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:03:40.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.573+0000 7f24605e3740 20 client.15384 populate_metadata read hostname 'smithi116' 2024-09-12T03:03:40.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.573+0000 7f24605e3740 1 --2- 172.21.15.116:0/2185231278 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b3c1679640 0x55b3c167ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:40.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.573+0000 7f24605e3740 4 client.15384 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:03:40.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.573+0000 7f24605e3740 1 -- 172.21.15.116:0/2185231278 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3c15cd850 con 0x55b3c1679640 2024-09-12T03:03:40.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.573+0000 7f24577fe640 1 --2- 172.21.15.116:0/2185231278 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b3c1679640 0x55b3c167ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:40.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.574+0000 7f24577fe640 1 --2- 172.21.15.116:0/2185231278 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b3c1679640 0x55b3c167ba50 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96e4505a7320f6a server_cookie=5b2b1b8b698fbcf0 in_seq=0 out_seq=0 2024-09-12T03:03:40.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.574+0000 7f2436ffd640 10 client.15384 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:40.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.574+0000 7f2436ffd640 1 -- 172.21.15.116:0/2185231278 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b3c15cd850 con 0x55b3c1679640 2024-09-12T03:03:40.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.574+0000 7f2436ffd640 10 client.15384 handle_command_reply: tid=0 2024-09-12T03:03:40.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.574+0000 7f24605e3740 1 client.15384 shutdown 2024-09-12T03:03:40.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.575+0000 7f24605e3740 1 -- 172.21.15.116:0/2185231278 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3c1669a90 msgr2=0x55b3c1666cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:40.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.575+0000 7f24605e3740 1 --2- 172.21.15.116:0/2185231278 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3c1669a90 0x55b3c1666cf0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f244c012c10 tx=0x7f244c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.575+0000 7f24605e3740 1 -- 172.21.15.116:0/2185231278 shutdown_connections 2024-09-12T03:03:40.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.575+0000 7f24605e3740 1 --2- 172.21.15.116:0/2185231278 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b3c1679640 0x55b3c167ba50 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.575+0000 7f24605e3740 1 --2- 172.21.15.116:0/2185231278 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3c1669a90 0x55b3c1666cf0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.575+0000 7f24605e3740 1 --2- 172.21.15.116:0/2185231278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3c165acc0 0x55b3c1666700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.575+0000 7f2436ffd640 0 client.15384 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:40.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.575+0000 7f24605e3740 1 --2- 172.21.15.116:0/2185231278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3c165a3a0 0x55b3c1669450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.575+0000 7f24605e3740 1 -- 172.21.15.116:0/2185231278 >> 172.21.15.116:0/2185231278 conn(0x55b3c1656a30 msgr2=0x55b3c165c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:40.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.575+0000 7f24605e3740 1 -- 172.21.15.116:0/2185231278 shutdown_connections 2024-09-12T03:03:40.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.575+0000 7f24605e3740 1 -- 172.21.15.116:0/2185231278 wait complete. 2024-09-12T03:03:40.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.575+0000 7f24605e3740 20 client.15384 trim_cache size 0 max 16384 2024-09-12T03:03:40.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.606+0000 7f24605e3740 1 Processor -- start 2024-09-12T03:03:40.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.606+0000 7f24605e3740 1 -- start start 2024-09-12T03:03:40.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.606+0000 7f24605e3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3c165a3a0 0x55b3c1675960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:40.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.606+0000 7f24605e3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3c165acc0 0x55b3c1681320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:40.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.606+0000 7f24605e3740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3c1669a90 0x55b3c1681860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:40.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.606+0000 7f24605e3740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f24581adb40 con 0x55b3c1669a90 2024-09-12T03:03:40.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.606+0000 7f24605e3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f24581ab410 con 0x55b3c165acc0 2024-09-12T03:03:40.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.606+0000 7f24605e3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f24581a9280 con 0x55b3c165a3a0 2024-09-12T03:03:40.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.606+0000 7f2456ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3c1669a90 0x55b3c1681860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:40.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.606+0000 7f2456ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3c1669a90 0x55b3c1681860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58736/0 (socket says 172.21.15.116:58736) 2024-09-12T03:03:40.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.607+0000 7f2456ffd640 1 -- 172.21.15.116:0/204473510 learned_addr learned my addr 172.21.15.116:0/204473510 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:40.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.607+0000 7f24577fe640 1 --2- 172.21.15.116:0/204473510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3c165acc0 0x55b3c1681320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:40.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.607+0000 7f2456ffd640 1 --2- 172.21.15.116:0/204473510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3c165a3a0 0x55b3c1675960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:40.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.607+0000 7f2456ffd640 1 -- 172.21.15.116:0/204473510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3c165a3a0 msgr2=0x55b3c1675960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:40.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.607+0000 7f2456ffd640 1 --2- 172.21.15.116:0/204473510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3c165a3a0 0x55b3c1675960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.607+0000 7f2456ffd640 1 -- 172.21.15.116:0/204473510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3c165acc0 msgr2=0x55b3c1681320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:40.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.607+0000 7f2456ffd640 1 --2- 172.21.15.116:0/204473510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3c165acc0 0x55b3c1681320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.607+0000 7f2456ffd640 1 -- 172.21.15.116:0/204473510 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3c16564d0 con 0x55b3c1669a90 2024-09-12T03:03:40.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.607+0000 7f2456ffd640 1 --2- 172.21.15.116:0/204473510 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3c1669a90 0x55b3c1681860 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f244c0126f0 tx=0x7f244c00b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afbc8e4c49550911 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:40.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.607+0000 7f243d7fa640 1 -- 172.21.15.116:0/204473510 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c010da0 con 0x55b3c1669a90 2024-09-12T03:03:40.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.607+0000 7f243d7fa640 1 -- 172.21.15.116:0/204473510 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f244c022640 con 0x55b3c1669a90 2024-09-12T03:03:40.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.607+0000 7f243d7fa640 1 -- 172.21.15.116:0/204473510 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c004730 con 0x55b3c1669a90 2024-09-12T03:03:40.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.608+0000 7f24605e3740 1 -- 172.21.15.116:0/204473510 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3c1669a90 msgr2=0x55b3c1681860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:40.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.608+0000 7f24605e3740 1 --2- 172.21.15.116:0/204473510 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3c1669a90 0x55b3c1681860 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f244c0126f0 tx=0x7f244c00b5b0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.608+0000 7f24605e3740 1 -- 172.21.15.116:0/204473510 shutdown_connections 2024-09-12T03:03:40.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.608+0000 7f24605e3740 1 --2- 172.21.15.116:0/204473510 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3c1669a90 0x55b3c1681860 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.608+0000 7f24605e3740 1 --2- 172.21.15.116:0/204473510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3c165acc0 0x55b3c1681320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.608+0000 7f24605e3740 1 --2- 172.21.15.116:0/204473510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3c165a3a0 0x55b3c1675960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.608+0000 7f24605e3740 1 -- 172.21.15.116:0/204473510 >> 172.21.15.116:0/204473510 conn(0x55b3c1656a30 msgr2=0x55b3c1661d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:40.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.608+0000 7f24605e3740 1 -- 172.21.15.116:0/204473510 shutdown_connections 2024-09-12T03:03:40.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.608+0000 7f24605e3740 1 -- 172.21.15.116:0/204473510 wait complete. 2024-09-12T03:03:40.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.609+0000 7f24605e3740 1 -- start start 2024-09-12T03:03:40.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.609+0000 7f24605e3740 1 Processor -- start 2024-09-12T03:03:40.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.609+0000 7f24605e3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3c165a3a0 0x55b3c166db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:40.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.609+0000 7f24605e3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3c165acc0 0x55b3c166e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:40.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.609+0000 7f24605e3740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3c1669a90 0x55b3c166e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:40.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.609+0000 7f24605e3740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f24581a98d0 con 0x55b3c1669a90 2024-09-12T03:03:40.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.609+0000 7f24605e3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f24581aad40 con 0x55b3c165a3a0 2024-09-12T03:03:40.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.609+0000 7f24605e3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f24581a9fa0 con 0x55b3c165acc0 2024-09-12T03:03:40.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.609+0000 7f24605e3740 10 client.0 _ll_register_callbacks cb 0x7f244c012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:40.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.609+0000 7f2456ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3c1669a90 0x55b3c166e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:40.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.609+0000 7f2456ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3c1669a90 0x55b3c166e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58750/0 (socket says 172.21.15.116:58750) 2024-09-12T03:03:40.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.609+0000 7f2456ffd640 1 -- 172.21.15.116:0/1392200829 learned_addr learned my addr 172.21.15.116:0/1392200829 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:40.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.609+0000 7f2456ffd640 1 --2- 172.21.15.116:0/1392200829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3c165a3a0 0x55b3c166db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:40.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.610+0000 7f24577fe640 1 --2- 172.21.15.116:0/1392200829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3c165acc0 0x55b3c166e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:40.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.610+0000 7f2456ffd640 1 -- 172.21.15.116:0/1392200829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3c165acc0 msgr2=0x55b3c166e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:40.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.610+0000 7f2456ffd640 1 --2- 172.21.15.116:0/1392200829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3c165acc0 0x55b3c166e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.610+0000 7f2456ffd640 1 -- 172.21.15.116:0/1392200829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3c165a3a0 msgr2=0x55b3c166db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:40.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.610+0000 7f2456ffd640 1 --2- 172.21.15.116:0/1392200829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3c165a3a0 0x55b3c166db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.610+0000 7f2456ffd640 1 -- 172.21.15.116:0/1392200829 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3c16e34f0 con 0x55b3c1669a90 2024-09-12T03:03:40.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.610+0000 7f2456ffd640 1 --2- 172.21.15.116:0/1392200829 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3c1669a90 0x55b3c166e5d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f244c022ce0 tx=0x7f244c022540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3344802edf65d279 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:40.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.610+0000 7f2436ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:03:40.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.610+0000 7f2436ffd640 1 -- 172.21.15.116:0/1392200829 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c003f80 con 0x55b3c1669a90 2024-09-12T03:03:40.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.610+0000 7f2436ffd640 1 -- 172.21.15.116:0/1392200829 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f244c004120 con 0x55b3c1669a90 2024-09-12T03:03:40.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.610+0000 7f24605e3740 1 -- 172.21.15.116:0/1392200829 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f244c022640 con 0x55b3c1669a90 2024-09-12T03:03:40.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.610+0000 7f2436ffd640 1 -- 172.21.15.116:0/1392200829 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f244c004320 con 0x55b3c1669a90 2024-09-12T03:03:40.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.610+0000 7f24605e3740 10 client.15390 fetch_fsmap learned FSMap version 25 2024-09-12T03:03:40.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.610+0000 7f24605e3740 1 -- 172.21.15.116:0/1392200829 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b3c16e47e0 con 0x55b3c1669a90 2024-09-12T03:03:40.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.611+0000 7f2436ffd640 1 -- 172.21.15.116:0/1392200829 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f244c022640 con 0x55b3c1669a90 2024-09-12T03:03:40.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.611+0000 7f2436ffd640 1 -- 172.21.15.116:0/1392200829 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f244c03a040 con 0x55b3c1669a90 2024-09-12T03:03:40.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.611+0000 7f24605e3740 10 client.15390 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:03:40.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.611+0000 7f24605e3740 10 client.15390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:03:40.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.611+0000 7f24605e3740 20 client.15390 populate_metadata read hostname 'smithi116' 2024-09-12T03:03:40.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.611+0000 7f24605e3740 1 --2- 172.21.15.116:0/1392200829 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b3c16e4ea0 0x55b3c16e72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:40.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.611+0000 7f24605e3740 4 client.15390 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:03:40.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.611+0000 7f24605e3740 1 -- 172.21.15.116:0/1392200829 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b3c15c7d60 con 0x55b3c16e4ea0 2024-09-12T03:03:40.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.611+0000 7f24577fe640 1 --2- 172.21.15.116:0/1392200829 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b3c16e4ea0 0x55b3c16e72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:40.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.612+0000 7f2436ffd640 10 client.15390 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:40.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.612+0000 7f24577fe640 1 --2- 172.21.15.116:0/1392200829 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b3c16e4ea0 0x55b3c16e72c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd21a90c853fa604 server_cookie=5171459c815f53e in_seq=0 out_seq=0 2024-09-12T03:03:40.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.612+0000 7f2436ffd640 1 -- 172.21.15.116:0/1392200829 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b3c15c7d60 con 0x55b3c16e4ea0 2024-09-12T03:03:40.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.612+0000 7f2436ffd640 10 client.15390 handle_command_reply: tid=0 2024-09-12T03:03:40.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.612+0000 7f24605e3740 1 client.15390 shutdown 2024-09-12T03:03:40.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.612+0000 7f24605e3740 1 -- 172.21.15.116:0/1392200829 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3c1669a90 msgr2=0x55b3c166e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:40.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.612+0000 7f24605e3740 1 --2- 172.21.15.116:0/1392200829 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3c1669a90 0x55b3c166e5d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f244c022ce0 tx=0x7f244c022540 comp rx=0 tx=0).stop 2024-09-12T03:03:40.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.612+0000 7f24605e3740 1 -- 172.21.15.116:0/1392200829 shutdown_connections 2024-09-12T03:03:40.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.612+0000 7f24605e3740 1 --2- 172.21.15.116:0/1392200829 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b3c16e4ea0 0x55b3c16e72c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.612+0000 7f24605e3740 1 --2- 172.21.15.116:0/1392200829 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3c1669a90 0x55b3c166e5d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.612+0000 7f24605e3740 1 --2- 172.21.15.116:0/1392200829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3c165acc0 0x55b3c166e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.612+0000 7f24605e3740 1 --2- 172.21.15.116:0/1392200829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3c165a3a0 0x55b3c166db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.612+0000 7f24605e3740 1 -- 172.21.15.116:0/1392200829 >> 172.21.15.116:0/1392200829 conn(0x55b3c1656a30 msgr2=0x55b3c165bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:40.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.612+0000 7f2436ffd640 0 client.15390 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:40.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.612+0000 7f24605e3740 1 -- 172.21.15.116:0/1392200829 shutdown_connections 2024-09-12T03:03:40.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.612+0000 7f24605e3740 1 -- 172.21.15.116:0/1392200829 wait complete. 2024-09-12T03:03:40.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.612+0000 7f24605e3740 20 client.15390 trim_cache size 0 max 16384 2024-09-12T03:03:40.618 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:03:40.619 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:03:40.619 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:03:40.619 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:03:40.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.613+0000 7f245e20a640 1 -- 172.21.15.116:0/3154099962 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f24400a2c10 msgr2=0x7f24400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:40.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.613+0000 7f245e20a640 1 --2- 172.21.15.116:0/3154099962 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f24400a2c10 0x7f24400a50d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f245008f810 tx=0x7f245002a950 comp rx=0 tx=0).stop 2024-09-12T03:03:40.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.613+0000 7f245e20a640 1 -- 172.21.15.116:0/3154099962 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f245810c150 msgr2=0x7f245819cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:40.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.613+0000 7f245e20a640 1 --2- 172.21.15.116:0/3154099962 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f245810c150 0x7f245819cd00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f245002ae70 tx=0x7f245002f5b0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.613+0000 7f245e20a640 1 -- 172.21.15.116:0/3154099962 shutdown_connections 2024-09-12T03:03:40.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.613+0000 7f245e20a640 1 --2- 172.21.15.116:0/3154099962 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f24400a2c10 0x7f24400a50d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.613+0000 7f245e20a640 1 --2- 172.21.15.116:0/3154099962 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f245810c150 0x7f245819cd00 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.613+0000 7f245e20a640 1 --2- 172.21.15.116:0/3154099962 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2458103690 0x7f2458198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.613+0000 7f245e20a640 1 --2- 172.21.15.116:0/3154099962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2458102d70 0x7f2458198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:40.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.613+0000 7f245e20a640 1 -- 172.21.15.116:0/3154099962 >> 172.21.15.116:0/3154099962 conn(0x7f24580fc790 msgr2=0x7f24580fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:40.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.613+0000 7f245e20a640 1 -- 172.21.15.116:0/3154099962 shutdown_connections 2024-09-12T03:03:40.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:40.613+0000 7f245e20a640 1 -- 172.21.15.116:0/3154099962 wait complete. 2024-09-12T03:03:40.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:40 smithi183 ceph-mon[35535]: pgmap v358: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:03:40.945 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-09-12T03:03:40.945 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:03:41.372 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:03:41.372 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:03:41.453 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:41.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.945+0000 7f438944d640 1 -- 172.21.15.116:0/32412116 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f438410e4d0 msgr2=0x7f43841108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:41.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.945+0000 7f438944d640 1 --2- 172.21.15.116:0/32412116 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f438410e4d0 0x7f43841108e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f437c009ca0 tx=0x7f437c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.945+0000 7f438944d640 1 -- 172.21.15.116:0/32412116 shutdown_connections 2024-09-12T03:03:41.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.945+0000 7f438944d640 1 --2- 172.21.15.116:0/32412116 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f438410e4d0 0x7f43841108e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.945+0000 7f438944d640 1 --2- 172.21.15.116:0/32412116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4384101360 0x7f438410df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.945+0000 7f438944d640 1 --2- 172.21.15.116:0/32412116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43841009b0 0x7f4384100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.945+0000 7f438944d640 1 -- 172.21.15.116:0/32412116 >> 172.21.15.116:0/32412116 conn(0x7f43840fc7d0 msgr2=0x7f43840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:41.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.945+0000 7f438944d640 1 -- 172.21.15.116:0/32412116 shutdown_connections 2024-09-12T03:03:41.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.946+0000 7f438944d640 1 -- 172.21.15.116:0/32412116 wait complete. 2024-09-12T03:03:41.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.946+0000 7f438944d640 1 Processor -- start 2024-09-12T03:03:41.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.946+0000 7f438944d640 1 -- start start 2024-09-12T03:03:41.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.947+0000 7f438944d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43841009b0 0x7f438419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:41.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.947+0000 7f438944d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4384101360 0x7f438419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:41.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.947+0000 7f438944d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f438410e4d0 0x7f438419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:41.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.947+0000 7f438944d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4384114200 con 0x7f438410e4d0 2024-09-12T03:03:41.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.947+0000 7f438944d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4384114080 con 0x7f43841009b0 2024-09-12T03:03:41.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.947+0000 7f438944d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4384114380 con 0x7f4384101360 2024-09-12T03:03:41.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.947+0000 7f43837fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f438410e4d0 0x7f438419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:41.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.947+0000 7f43837fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f438410e4d0 0x7f438419efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52248/0 (socket says 172.21.15.116:52248) 2024-09-12T03:03:41.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.947+0000 7f43837fe640 1 -- 172.21.15.116:0/3382665097 learned_addr learned my addr 172.21.15.116:0/3382665097 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:41.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.947+0000 7f43827fc640 1 --2- 172.21.15.116:0/3382665097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4384101360 0x7f438419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:41.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.947+0000 7f4382ffd640 1 --2- 172.21.15.116:0/3382665097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43841009b0 0x7f438419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.947+0000 7f43837fe640 1 -- 172.21.15.116:0/3382665097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4384101360 msgr2=0x7f438419ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.947+0000 7f43837fe640 1 --2- 172.21.15.116:0/3382665097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4384101360 0x7f438419ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.947+0000 7f43837fe640 1 -- 172.21.15.116:0/3382665097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43841009b0 msgr2=0x7f438419a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.947+0000 7f43837fe640 1 --2- 172.21.15.116:0/3382665097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43841009b0 0x7f438419a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.947+0000 7f43837fe640 1 -- 172.21.15.116:0/3382665097 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f438419f690 con 0x7f438410e4d0 2024-09-12T03:03:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.947+0000 7f43837fe640 1 --2- 172.21.15.116:0/3382665097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f438410e4d0 0x7f438419efb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f437c031a20 tx=0x7f437c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.948+0000 7f436bfff640 1 -- 172.21.15.116:0/3382665097 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437c003c30 con 0x7f438410e4d0 2024-09-12T03:03:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.948+0000 7f438944d640 1 -- 172.21.15.116:0/3382665097 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f438419f920 con 0x7f438410e4d0 2024-09-12T03:03:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.948+0000 7f436bfff640 1 -- 172.21.15.116:0/3382665097 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f437c003dd0 con 0x7f438410e4d0 2024-09-12T03:03:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.948+0000 7f436bfff640 1 -- 172.21.15.116:0/3382665097 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437c02fc40 con 0x7f438410e4d0 2024-09-12T03:03:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.948+0000 7f438944d640 1 -- 172.21.15.116:0/3382665097 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43841a7260 con 0x7f438410e4d0 2024-09-12T03:03:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.949+0000 7f438944d640 1 -- 172.21.15.116:0/3382665097 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4384100d90 con 0x7f438410e4d0 2024-09-12T03:03:41.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.950+0000 7f436bfff640 1 -- 172.21.15.116:0/3382665097 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f437c004050 con 0x7f438410e4d0 2024-09-12T03:03:41.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.951+0000 7f436bfff640 1 --2- 172.21.15.116:0/3382665097 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f436c0a2c10 0x7f436c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:41.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.951+0000 7f4382ffd640 1 --2- 172.21.15.116:0/3382665097 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f436c0a2c10 0x7f436c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:41.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.951+0000 7f436bfff640 1 -- 172.21.15.116:0/3382665097 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f437c0f03c0 con 0x7f438410e4d0 2024-09-12T03:03:41.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.951+0000 7f4382ffd640 1 --2- 172.21.15.116:0/3382665097 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f436c0a2c10 0x7f436c0a50d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4374009bd0 tx=0x7f4374005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:41.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:41.958+0000 7f436bfff640 1 -- 172.21.15.116:0/3382665097 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f437c0bd420 con 0x7f438410e4d0 2024-09-12T03:03:42.054 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:42.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:42.206+0000 7f438944d640 1 -- 172.21.15.116:0/3382665097 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4384109660 con 0x7f438410e4d0 2024-09-12T03:03:42.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:42.206+0000 7f436bfff640 1 -- 172.21.15.116:0/3382665097 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f437c0bcd20 con 0x7f438410e4d0 2024-09-12T03:03:42.207 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:03:42.207 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:03:42.207 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:03:42.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:42.208+0000 7f438944d640 1 -- 172.21.15.116:0/3382665097 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f436c0a2c10 msgr2=0x7f436c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:42.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:42.208+0000 7f438944d640 1 --2- 172.21.15.116:0/3382665097 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f436c0a2c10 0x7f436c0a50d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4374009bd0 tx=0x7f4374005f90 comp rx=0 tx=0).stop 2024-09-12T03:03:42.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:42.208+0000 7f438944d640 1 -- 172.21.15.116:0/3382665097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f438410e4d0 msgr2=0x7f438419efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:42.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:42.208+0000 7f438944d640 1 --2- 172.21.15.116:0/3382665097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f438410e4d0 0x7f438419efb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f437c031a20 tx=0x7f437c031a50 comp rx=0 tx=0).stop 2024-09-12T03:03:42.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:42.209+0000 7f438944d640 1 -- 172.21.15.116:0/3382665097 shutdown_connections 2024-09-12T03:03:42.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:42.209+0000 7f438944d640 1 --2- 172.21.15.116:0/3382665097 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f436c0a2c10 0x7f436c0a50d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:42.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:42.209+0000 7f438944d640 1 --2- 172.21.15.116:0/3382665097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f438410e4d0 0x7f438419efb0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:42.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:42.209+0000 7f438944d640 1 --2- 172.21.15.116:0/3382665097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4384101360 0x7f438419ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:42.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:42.209+0000 7f438944d640 1 --2- 172.21.15.116:0/3382665097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43841009b0 0x7f438419a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:42.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:42.209+0000 7f438944d640 1 -- 172.21.15.116:0/3382665097 >> 172.21.15.116:0/3382665097 conn(0x7f43840fc7d0 msgr2=0x7f43840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:42.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:42.209+0000 7f438944d640 1 -- 172.21.15.116:0/3382665097 shutdown_connections 2024-09-12T03:03:42.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:42.209+0000 7f438944d640 1 -- 172.21.15.116:0/3382665097 wait complete. 2024-09-12T03:03:42.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:42 smithi177 ceph-mon[35305]: pgmap v359: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-12T03:03:42.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:42 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3382665097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:42.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:42 smithi116 ceph-mon[31410]: pgmap v359: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-12T03:03:42.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:42 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3382665097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:42.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:42 smithi183 ceph-mon[35535]: pgmap v359: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-12T03:03:42.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:42 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3382665097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:43.015 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0' 2024-09-12T03:03:43.015 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0 2024-09-12T03:03:43.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.278+0000 7f6e31db6640 1 -- 172.21.15.116:0/799675824 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e2c102570 msgr2=0x7f6e2c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:43.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.278+0000 7f6e31db6640 1 --2- 172.21.15.116:0/799675824 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e2c102570 0x7f6e2c1029b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6e20009900 tx=0x7f6e2002a480 comp rx=0 tx=0).stop 2024-09-12T03:03:43.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.278+0000 7f6e31db6640 1 -- 172.21.15.116:0/799675824 shutdown_connections 2024-09-12T03:03:43.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.278+0000 7f6e31db6640 1 --2- 172.21.15.116:0/799675824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e2c102ef0 0x7f6e2c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.278+0000 7f6e31db6640 1 --2- 172.21.15.116:0/799675824 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e2c102570 0x7f6e2c1029b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.278+0000 7f6e31db6640 1 --2- 172.21.15.116:0/799675824 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e2c100e20 0x7f6e2c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.278+0000 7f6e31db6640 1 -- 172.21.15.116:0/799675824 >> 172.21.15.116:0/799675824 conn(0x7f6e2c0fc7d0 msgr2=0x7f6e2c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:43.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.278+0000 7f6e31db6640 1 -- 172.21.15.116:0/799675824 shutdown_connections 2024-09-12T03:03:43.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.278+0000 7f6e31db6640 1 -- 172.21.15.116:0/799675824 wait complete. 2024-09-12T03:03:43.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.278+0000 7f6e31db6640 1 Processor -- start 2024-09-12T03:03:43.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.278+0000 7f6e31db6640 1 -- start start 2024-09-12T03:03:43.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.279+0000 7f6e31db6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e2c100e20 0x7f6e2c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:43.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.279+0000 7f6e31db6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e2c102570 0x7f6e2c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:43.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.279+0000 7f6e31db6640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e2c102ef0 0x7f6e2c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:43.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.279+0000 7f6e31db6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c110f50 con 0x7f6e2c102570 2024-09-12T03:03:43.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.279+0000 7f6e31db6640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c110dd0 con 0x7f6e2c102ef0 2024-09-12T03:03:43.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.279+0000 7f6e31db6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c1110d0 con 0x7f6e2c100e20 2024-09-12T03:03:43.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.279+0000 7f6e2affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e2c102570 0x7f6e2c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:43.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.279+0000 7f6e2affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e2c102570 0x7f6e2c198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52266/0 (socket says 172.21.15.116:52266) 2024-09-12T03:03:43.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.279+0000 7f6e2affd640 1 -- 172.21.15.116:0/191223688 learned_addr learned my addr 172.21.15.116:0/191223688 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:43.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.279+0000 7f6e2b7fe640 1 --2- 172.21.15.116:0/191223688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e2c100e20 0x7f6e2c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:43.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.279+0000 7f6e2bfff640 1 --2- 172.21.15.116:0/191223688 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e2c102ef0 0x7f6e2c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:43.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.279+0000 7f6e2affd640 1 -- 172.21.15.116:0/191223688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e2c100e20 msgr2=0x7f6e2c1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:43.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.279+0000 7f6e2affd640 1 --2- 172.21.15.116:0/191223688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e2c100e20 0x7f6e2c1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.279+0000 7f6e2affd640 1 -- 172.21.15.116:0/191223688 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e2c102ef0 msgr2=0x7f6e2c19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:43.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.279+0000 7f6e2affd640 1 --2- 172.21.15.116:0/191223688 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e2c102ef0 0x7f6e2c19ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.279+0000 7f6e2affd640 1 -- 172.21.15.116:0/191223688 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e2c19d560 con 0x7f6e2c102570 2024-09-12T03:03:43.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.279+0000 7f6e2affd640 1 --2- 172.21.15.116:0/191223688 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e2c102570 0x7f6e2c198af0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6e20006010 tx=0x7f6e20003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:43.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.280+0000 7f6e28ff9640 1 -- 172.21.15.116:0/191223688 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2003f070 con 0x7f6e2c102570 2024-09-12T03:03:43.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.280+0000 7f6e28ff9640 1 -- 172.21.15.116:0/191223688 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e20031c30 con 0x7f6e2c102570 2024-09-12T03:03:43.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.280+0000 7f6e31db6640 1 -- 172.21.15.116:0/191223688 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e2c19d7f0 con 0x7f6e2c102570 2024-09-12T03:03:43.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.280+0000 7f6e31db6640 1 -- 172.21.15.116:0/191223688 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e2c0fcbb0 con 0x7f6e2c102570 2024-09-12T03:03:43.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.280+0000 7f6e28ff9640 1 -- 172.21.15.116:0/191223688 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e20049060 con 0x7f6e2c102570 2024-09-12T03:03:43.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.281+0000 7f6e31db6640 1 -- 172.21.15.116:0/191223688 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e2c1021b0 con 0x7f6e2c102570 2024-09-12T03:03:43.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.283+0000 7f6e28ff9640 1 -- 172.21.15.116:0/191223688 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f6e20042070 con 0x7f6e2c102570 2024-09-12T03:03:43.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.283+0000 7f6e28ff9640 1 --2- 172.21.15.116:0/191223688 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6e140a2c10 0x7f6e140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:43.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.283+0000 7f6e2b7fe640 1 --2- 172.21.15.116:0/191223688 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6e140a2c10 0x7f6e140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:43.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.283+0000 7f6e28ff9640 1 -- 172.21.15.116:0/191223688 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f6e200f0310 con 0x7f6e2c102570 2024-09-12T03:03:43.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.284+0000 7f6e2b7fe640 1 --2- 172.21.15.116:0/191223688 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6e140a2c10 0x7f6e140a50d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c0095f0 tx=0x7f6e1c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:43.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.287+0000 7f6e28ff9640 1 -- 172.21.15.116:0/191223688 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6e200bcaf0 con 0x7f6e2c102570 2024-09-12T03:03:43.393 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:43.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.528+0000 7f6e31db6640 1 -- 172.21.15.116:0/191223688 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e2c101200 con 0x7f6e2c102570 2024-09-12T03:03:43.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.529+0000 7f6e28ff9640 1 -- 172.21.15.116:0/191223688 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f6e2c101200 con 0x7f6e2c102570 2024-09-12T03:03:43.530 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:03:43.530 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:03:43.530 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:03:43.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.531+0000 7f6e31db6640 1 -- 172.21.15.116:0/191223688 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6e140a2c10 msgr2=0x7f6e140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:43.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.531+0000 7f6e31db6640 1 --2- 172.21.15.116:0/191223688 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6e140a2c10 0x7f6e140a50d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c0095f0 tx=0x7f6e1c005f90 comp rx=0 tx=0).stop 2024-09-12T03:03:43.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.531+0000 7f6e31db6640 1 -- 172.21.15.116:0/191223688 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e2c102570 msgr2=0x7f6e2c198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:43.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.531+0000 7f6e31db6640 1 --2- 172.21.15.116:0/191223688 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e2c102570 0x7f6e2c198af0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6e20006010 tx=0x7f6e20003c10 comp rx=0 tx=0).stop 2024-09-12T03:03:43.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.531+0000 7f6e31db6640 1 -- 172.21.15.116:0/191223688 shutdown_connections 2024-09-12T03:03:43.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.531+0000 7f6e31db6640 1 --2- 172.21.15.116:0/191223688 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6e140a2c10 0x7f6e140a50d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.531+0000 7f6e31db6640 1 --2- 172.21.15.116:0/191223688 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e2c102ef0 0x7f6e2c19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.531+0000 7f6e31db6640 1 --2- 172.21.15.116:0/191223688 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e2c102570 0x7f6e2c198af0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.531+0000 7f6e31db6640 1 --2- 172.21.15.116:0/191223688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e2c100e20 0x7f6e2c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.531+0000 7f6e31db6640 1 -- 172.21.15.116:0/191223688 >> 172.21.15.116:0/191223688 conn(0x7f6e2c0fc7d0 msgr2=0x7f6e2c06a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:43.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.531+0000 7f6e31db6640 1 -- 172.21.15.116:0/191223688 shutdown_connections 2024-09-12T03:03:43.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:43.531+0000 7f6e31db6640 1 -- 172.21.15.116:0/191223688 wait complete. 2024-09-12T03:03:44.380 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:44 smithi116 ceph-mon[31410]: pgmap v360: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:03:44.381 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:44 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/191223688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:44.407 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:03:44.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:44 smithi177 ceph-mon[35305]: pgmap v360: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:03:44.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:44 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/191223688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:44.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.585+0000 7f63215c1640 1 -- 172.21.15.116:0/2121293284 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f631c101dd0 msgr2=0x7f631c1021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:44.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.585+0000 7f63215c1640 1 --2- 172.21.15.116:0/2121293284 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f631c101dd0 0x7f631c1021b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f630c009900 tx=0x7f630c02a410 comp rx=0 tx=0).stop 2024-09-12T03:03:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.585+0000 7f63215c1640 1 -- 172.21.15.116:0/2121293284 shutdown_connections 2024-09-12T03:03:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.585+0000 7f63215c1640 1 --2- 172.21.15.116:0/2121293284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f631c106f00 0x7f631c10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.585+0000 7f63215c1640 1 --2- 172.21.15.116:0/2121293284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f631c102780 0x7f631c1067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.585+0000 7f63215c1640 1 --2- 172.21.15.116:0/2121293284 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f631c101dd0 0x7f631c1021b0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.585+0000 7f63215c1640 1 -- 172.21.15.116:0/2121293284 >> 172.21.15.116:0/2121293284 conn(0x7f631c0fd8b0 msgr2=0x7f631c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.585+0000 7f63215c1640 1 -- 172.21.15.116:0/2121293284 shutdown_connections 2024-09-12T03:03:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.585+0000 7f63215c1640 1 -- 172.21.15.116:0/2121293284 wait complete. 2024-09-12T03:03:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.586+0000 7f63215c1640 1 Processor -- start 2024-09-12T03:03:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.586+0000 7f63215c1640 1 -- start start 2024-09-12T03:03:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.587+0000 7f63215c1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f631c101dd0 0x7f631c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.587+0000 7f63215c1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f631c102780 0x7f631c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.587+0000 7f63215c1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f631c106f00 0x7f631c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.587+0000 7f63215c1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f631c111f60 con 0x7f631c102780 2024-09-12T03:03:44.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.587+0000 7f63215c1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f631c111de0 con 0x7f631c106f00 2024-09-12T03:03:44.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.587+0000 7f63215c1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f631c1120e0 con 0x7f631c101dd0 2024-09-12T03:03:44.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.587+0000 7f631a7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f631c102780 0x7f631c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:44.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.587+0000 7f631a7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f631c102780 0x7f631c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52278/0 (socket says 172.21.15.116:52278) 2024-09-12T03:03:44.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.587+0000 7f631a7fc640 1 -- 172.21.15.116:0/2276685196 learned_addr learned my addr 172.21.15.116:0/2276685196 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:44.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.587+0000 7f631affd640 1 --2- 172.21.15.116:0/2276685196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f631c101dd0 0x7f631c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:44.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.587+0000 7f631b7fe640 1 --2- 172.21.15.116:0/2276685196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f631c106f00 0x7f631c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:44.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.587+0000 7f631a7fc640 1 -- 172.21.15.116:0/2276685196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f631c101dd0 msgr2=0x7f631c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:44.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.587+0000 7f631a7fc640 1 --2- 172.21.15.116:0/2276685196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f631c101dd0 0x7f631c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.587+0000 7f631a7fc640 1 -- 172.21.15.116:0/2276685196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f631c106f00 msgr2=0x7f631c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:44.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.587+0000 7f631a7fc640 1 --2- 172.21.15.116:0/2276685196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f631c106f00 0x7f631c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.587+0000 7f631a7fc640 1 -- 172.21.15.116:0/2276685196 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f631c1a67c0 con 0x7f631c102780 2024-09-12T03:03:44.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.587+0000 7f631a7fc640 1 --2- 172.21.15.116:0/2276685196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f631c102780 0x7f631c074a30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f631000c920 tx=0x7f631000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:44.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.588+0000 7f6303fff640 1 -- 172.21.15.116:0/2276685196 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6310007c70 con 0x7f631c102780 2024-09-12T03:03:44.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.588+0000 7f63215c1640 1 -- 172.21.15.116:0/2276685196 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f631c1a6ab0 con 0x7f631c102780 2024-09-12T03:03:44.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.588+0000 7f6303fff640 1 -- 172.21.15.116:0/2276685196 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6310007e10 con 0x7f631c102780 2024-09-12T03:03:44.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.588+0000 7f6303fff640 1 -- 172.21.15.116:0/2276685196 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6310018890 con 0x7f631c102780 2024-09-12T03:03:44.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.588+0000 7f63215c1640 1 -- 172.21.15.116:0/2276685196 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f631c1a7040 con 0x7f631c102780 2024-09-12T03:03:44.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.591+0000 7f6303fff640 1 -- 172.21.15.116:0/2276685196 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f6310018a50 con 0x7f631c102780 2024-09-12T03:03:44.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.591+0000 7f632399a740 1 Processor -- start 2024-09-12T03:03:44.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.591+0000 7f632399a740 1 -- start start 2024-09-12T03:03:44.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.592+0000 7f632399a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f3de92760 0x556f3de92b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:44.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.592+0000 7f632399a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556f3de93080 0x556f3de9d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:44.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.592+0000 7f632399a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f3de9d940 0x556f3de9fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:44.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.592+0000 7f632399a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556f3ddf82a0 con 0x556f3de9d940 2024-09-12T03:03:44.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.592+0000 7f632399a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556f3dd9fb10 con 0x556f3de93080 2024-09-12T03:03:44.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.592+0000 7f632399a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556f3ddf4260 con 0x556f3de92760 2024-09-12T03:03:44.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.592+0000 7f631a7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f3de9d940 0x556f3de9fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:44.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.592+0000 7f631b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f3de92760 0x556f3de92b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:44.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.592+0000 7f631b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f3de92760 0x556f3de92b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.116:40322/0 (socket says 172.21.15.116:40322) 2024-09-12T03:03:44.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.592+0000 7f631b7fe640 1 -- 172.21.15.116:0/4032627677 learned_addr learned my addr 172.21.15.116:0/4032627677 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:44.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.592+0000 7f631a7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f3de9d940 0x556f3de9fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52288/0 (socket says 172.21.15.116:52288) 2024-09-12T03:03:44.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.592+0000 7f631affd640 1 --2- 172.21.15.116:0/4032627677 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556f3de93080 0x556f3de9d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:44.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.592+0000 7f6303fff640 1 --2- 172.21.15.116:0/2276685196 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f63040a2c90 0x7f63040a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:44.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.592+0000 7f631b7fe640 1 -- 172.21.15.116:0/4032627677 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556f3de93080 msgr2=0x556f3de9d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:44.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.592+0000 7f631b7fe640 1 --2- 172.21.15.116:0/4032627677 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556f3de93080 0x556f3de9d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.592+0000 7f631b7fe640 1 -- 172.21.15.116:0/4032627677 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f3de9d940 msgr2=0x556f3de9fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:44.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.593+0000 7f6303fff640 1 -- 172.21.15.116:0/2276685196 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f63100cf280 con 0x7f631c102780 2024-09-12T03:03:44.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.593+0000 7f631b7fe640 1 --2- 172.21.15.116:0/4032627677 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f3de9d940 0x556f3de9fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.593+0000 7f631b7fe640 1 -- 172.21.15.116:0/4032627677 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f3db69b90 con 0x556f3de92760 2024-09-12T03:03:44.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.593+0000 7f631a7fc640 1 --2- 172.21.15.116:0/4032627677 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f3de9d940 0x556f3de9fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:44.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.593+0000 7f631b7fe640 1 --2- 172.21.15.116:0/2276685196 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f63040a2c90 0x7f63040a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:44.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.593+0000 7f631b7fe640 1 --2- 172.21.15.116:0/4032627677 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f3de92760 0x556f3de92b40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f63140072e0 tx=0x7f631400f850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d8f260d0bdaaad9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:44.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.593+0000 7f63017fa640 1 -- 172.21.15.116:0/4032627677 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6314019870 con 0x556f3de92760 2024-09-12T03:03:44.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.593+0000 7f63017fa640 1 -- 172.21.15.116:0/4032627677 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f631401a410 con 0x556f3de92760 2024-09-12T03:03:44.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.593+0000 7f631b7fe640 1 --2- 172.21.15.116:0/2276685196 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f63040a2c90 0x7f63040a5150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6314020000 tx=0x7f631401d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:44.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.593+0000 7f63017fa640 1 -- 172.21.15.116:0/4032627677 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631400d580 con 0x556f3de92760 2024-09-12T03:03:44.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.594+0000 7f632399a740 1 -- 172.21.15.116:0/4032627677 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f3de92760 msgr2=0x556f3de92b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:44.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.594+0000 7f632399a740 1 --2- 172.21.15.116:0/4032627677 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f3de92760 0x556f3de92b40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f63140072e0 tx=0x7f631400f850 comp rx=0 tx=0).stop 2024-09-12T03:03:44.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.594+0000 7f632399a740 1 -- 172.21.15.116:0/4032627677 shutdown_connections 2024-09-12T03:03:44.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.594+0000 7f632399a740 1 --2- 172.21.15.116:0/4032627677 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f3de9d940 0x556f3de9fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.594+0000 7f632399a740 1 --2- 172.21.15.116:0/4032627677 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556f3de93080 0x556f3de9d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.594+0000 7f632399a740 1 --2- 172.21.15.116:0/4032627677 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f3de92760 0x556f3de92b40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.594+0000 7f632399a740 1 -- 172.21.15.116:0/4032627677 >> 172.21.15.116:0/4032627677 conn(0x556f3de8ecf0 msgr2=0x556f3de8f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:44.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.594+0000 7f632399a740 1 -- 172.21.15.116:0/4032627677 shutdown_connections 2024-09-12T03:03:44.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.594+0000 7f632399a740 1 -- 172.21.15.116:0/4032627677 wait complete. 2024-09-12T03:03:44.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.595+0000 7f632399a740 1 -- start start 2024-09-12T03:03:44.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.595+0000 7f632399a740 1 Processor -- start 2024-09-12T03:03:44.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.595+0000 7f632399a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556f3de92760 0x556f3de9a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:44.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.595+0000 7f632399a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f3de93080 0x556f3de9ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:44.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.595+0000 7f632399a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f3de9d940 0x556f3de9b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:44.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.595+0000 7f632399a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556f3dea0510 con 0x556f3de93080 2024-09-12T03:03:44.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.595+0000 7f632399a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556f3dd806e0 con 0x556f3de92760 2024-09-12T03:03:44.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.595+0000 7f632399a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556f3dea0690 con 0x556f3de9d940 2024-09-12T03:03:44.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.595+0000 7f632399a740 10 client.0 _ll_register_callbacks cb 0x556f3dda4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:44.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.595+0000 7f631a7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f3de93080 0x556f3de9ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:44.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.595+0000 7f631a7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f3de93080 0x556f3de9ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52304/0 (socket says 172.21.15.116:52304) 2024-09-12T03:03:44.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.595+0000 7f631a7fc640 1 -- 172.21.15.116:0/3995468155 learned_addr learned my addr 172.21.15.116:0/3995468155 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:44.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.596+0000 7f631affd640 1 --2- 172.21.15.116:0/3995468155 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556f3de92760 0x556f3de9a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:44.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.596+0000 7f631affd640 1 -- 172.21.15.116:0/3995468155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f3de9d940 msgr2=0x556f3de9b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:44.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.596+0000 7f631affd640 1 --2- 172.21.15.116:0/3995468155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f3de9d940 0x556f3de9b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.596+0000 7f631affd640 1 -- 172.21.15.116:0/3995468155 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f3de93080 msgr2=0x556f3de9ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:44.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.596+0000 7f631affd640 1 --2- 172.21.15.116:0/3995468155 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f3de93080 0x556f3de9ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.596+0000 7f631affd640 1 -- 172.21.15.116:0/3995468155 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f3dea9a00 con 0x556f3de92760 2024-09-12T03:03:44.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.596+0000 7f631a7fc640 1 --2- 172.21.15.116:0/3995468155 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f3de93080 0x556f3de9ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:44.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.596+0000 7f631affd640 1 --2- 172.21.15.116:0/3995468155 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556f3de92760 0x556f3de9a810 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f630c005940 tx=0x7f630c0319d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc377b64cc322f0b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:44.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.596+0000 7f62feff5640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-09-12T03:03:44.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.596+0000 7f62feff5640 1 -- 172.21.15.116:0/3995468155 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c035040 con 0x556f3de92760 2024-09-12T03:03:44.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.597+0000 7f62feff5640 1 -- 172.21.15.116:0/3995468155 <== mon.2 v2:172.21.15.177:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f630c0042c0 con 0x556f3de92760 2024-09-12T03:03:44.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.597+0000 7f632399a740 1 -- 172.21.15.116:0/3995468155 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f631401a410 con 0x556f3de92760 2024-09-12T03:03:44.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.597+0000 7f62feff5640 1 -- 172.21.15.116:0/3995468155 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f630c0044c0 con 0x556f3de92760 2024-09-12T03:03:44.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.597+0000 7f632399a740 10 client.24893 fetch_fsmap learned FSMap version 25 2024-09-12T03:03:44.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.597+0000 7f632399a740 1 -- 172.21.15.116:0/3995468155 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556f3deac370 con 0x556f3de92760 2024-09-12T03:03:44.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.597+0000 7f62feff5640 1 -- 172.21.15.116:0/3995468155 <== mon.2 v2:172.21.15.177:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f631401a410 con 0x556f3de92760 2024-09-12T03:03:44.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.597+0000 7f62feff5640 1 -- 172.21.15.116:0/3995468155 <== mon.2 v2:172.21.15.177:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f630c045850 con 0x556f3de92760 2024-09-12T03:03:44.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.598+0000 7f632399a740 10 client.24893 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:03:44.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.598+0000 7f632399a740 10 client.24893 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:03:44.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.598+0000 7f632399a740 20 client.24893 populate_metadata read hostname 'smithi116' 2024-09-12T03:03:44.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.598+0000 7f632399a740 1 --2- 172.21.15.116:0/3995468155 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556f3deadc10 0x556f3deb00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:44.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.598+0000 7f632399a740 4 client.24893 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:03:44.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.598+0000 7f632399a740 1 -- 172.21.15.116:0/3995468155 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f3de059d0 con 0x556f3deadc10 2024-09-12T03:03:44.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.598+0000 7f631a7fc640 1 --2- 172.21.15.116:0/3995468155 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556f3deadc10 0x556f3deb00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:44.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.598+0000 7f62feff5640 10 client.24893 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:44.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.598+0000 7f631a7fc640 1 --2- 172.21.15.116:0/3995468155 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556f3deadc10 0x556f3deb00a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea63b2af4544f625 server_cookie=e26b55655fbd1894 in_seq=0 out_seq=0 2024-09-12T03:03:44.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.599+0000 7f62feff5640 1 -- 172.21.15.116:0/3995468155 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556f3de059d0 con 0x556f3deadc10 2024-09-12T03:03:44.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.599+0000 7f62feff5640 10 client.24893 handle_command_reply: tid=0 2024-09-12T03:03:44.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.599+0000 7f632399a740 1 client.24893 shutdown 2024-09-12T03:03:44.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.600+0000 7f632399a740 1 -- 172.21.15.116:0/3995468155 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556f3de92760 msgr2=0x556f3de9a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:44.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.600+0000 7f632399a740 1 --2- 172.21.15.116:0/3995468155 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556f3de92760 0x556f3de9a810 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f630c005940 tx=0x7f630c0319d0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.600+0000 7f632399a740 1 -- 172.21.15.116:0/3995468155 shutdown_connections 2024-09-12T03:03:44.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.600+0000 7f632399a740 1 --2- 172.21.15.116:0/3995468155 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556f3deadc10 0x556f3deb00a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.600+0000 7f632399a740 1 --2- 172.21.15.116:0/3995468155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f3de9d940 0x556f3de9b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.600+0000 7f632399a740 1 --2- 172.21.15.116:0/3995468155 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f3de93080 0x556f3de9ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.600+0000 7f632399a740 1 --2- 172.21.15.116:0/3995468155 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556f3de92760 0x556f3de9a810 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.600+0000 7f632399a740 1 -- 172.21.15.116:0/3995468155 >> 172.21.15.116:0/3995468155 conn(0x556f3de8ecf0 msgr2=0x556f3de949d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:44.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.600+0000 7f62feff5640 0 client.24893 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:44.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.600+0000 7f632399a740 1 -- 172.21.15.116:0/3995468155 shutdown_connections 2024-09-12T03:03:44.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.600+0000 7f632399a740 1 -- 172.21.15.116:0/3995468155 wait complete. 2024-09-12T03:03:44.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.600+0000 7f632399a740 20 client.24893 trim_cache size 0 max 16384 2024-09-12T03:03:44.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.634+0000 7f632399a740 1 Processor -- start 2024-09-12T03:03:44.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.634+0000 7f632399a740 1 -- start start 2024-09-12T03:03:44.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.634+0000 7f632399a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556f3de92760 0x556f3de95d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:44.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.634+0000 7f632399a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f3de93080 0x556f3de948c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:44.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.634+0000 7f632399a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f3de9d940 0x556f3deb5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:44.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.634+0000 7f632399a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f631c1b1ec0 con 0x556f3de93080 2024-09-12T03:03:44.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.634+0000 7f632399a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f631c1af790 con 0x556f3de92760 2024-09-12T03:03:44.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.634+0000 7f632399a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f631c1ad5e0 con 0x556f3de9d940 2024-09-12T03:03:44.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.634+0000 7f631affd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556f3de92760 0x556f3de95d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:44.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.635+0000 7f631b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f3de9d940 0x556f3deb5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:44.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.635+0000 7f631affd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556f3de92760 0x556f3de95d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.116:54786/0 (socket says 172.21.15.116:54786) 2024-09-12T03:03:44.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.635+0000 7f631affd640 1 -- 172.21.15.116:0/3245934618 learned_addr learned my addr 172.21.15.116:0/3245934618 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:44.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.635+0000 7f631affd640 1 -- 172.21.15.116:0/3245934618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f3de9d940 msgr2=0x556f3deb5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:44.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.635+0000 7f631affd640 1 --2- 172.21.15.116:0/3245934618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f3de9d940 0x556f3deb5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.635+0000 7f631affd640 1 -- 172.21.15.116:0/3245934618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f3de93080 msgr2=0x556f3de948c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:44.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.635+0000 7f631affd640 1 --2- 172.21.15.116:0/3245934618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f3de93080 0x556f3de948c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.635+0000 7f631affd640 1 -- 172.21.15.116:0/3245934618 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f3de997c0 con 0x556f3de92760 2024-09-12T03:03:44.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.635+0000 7f631affd640 1 --2- 172.21.15.116:0/3245934618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556f3de92760 0x556f3de95d50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f630c045c20 tx=0x7f630c02a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae0480a69b27f58 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:44.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.635+0000 7f6300ff9640 1 -- 172.21.15.116:0/3245934618 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c035030 con 0x556f3de92760 2024-09-12T03:03:44.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.636+0000 7f6300ff9640 1 -- 172.21.15.116:0/3245934618 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f630c031a40 con 0x556f3de92760 2024-09-12T03:03:44.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.636+0000 7f6300ff9640 1 -- 172.21.15.116:0/3245934618 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c004a00 con 0x556f3de92760 2024-09-12T03:03:44.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.636+0000 7f632399a740 1 -- 172.21.15.116:0/3245934618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556f3de92760 msgr2=0x556f3de95d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:44.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.636+0000 7f632399a740 1 --2- 172.21.15.116:0/3245934618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556f3de92760 0x556f3de95d50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f630c045c20 tx=0x7f630c02a410 comp rx=0 tx=0).stop 2024-09-12T03:03:44.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.637+0000 7f632399a740 1 -- 172.21.15.116:0/3245934618 shutdown_connections 2024-09-12T03:03:44.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.637+0000 7f632399a740 1 --2- 172.21.15.116:0/3245934618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f3de9d940 0x556f3deb5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.637+0000 7f632399a740 1 --2- 172.21.15.116:0/3245934618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f3de93080 0x556f3de948c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.637+0000 7f632399a740 1 --2- 172.21.15.116:0/3245934618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556f3de92760 0x556f3de95d50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.637+0000 7f632399a740 1 -- 172.21.15.116:0/3245934618 >> 172.21.15.116:0/3245934618 conn(0x556f3de8ecf0 msgr2=0x556f3de941b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:44.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.637+0000 7f632399a740 1 -- 172.21.15.116:0/3245934618 shutdown_connections 2024-09-12T03:03:44.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.637+0000 7f632399a740 1 -- 172.21.15.116:0/3245934618 wait complete. 2024-09-12T03:03:44.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.637+0000 7f632399a740 1 -- start start 2024-09-12T03:03:44.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.637+0000 7f632399a740 1 Processor -- start 2024-09-12T03:03:44.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.638+0000 7f632399a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556f3de92760 0x556f3dea5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:44.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.638+0000 7f632399a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f3de93080 0x556f3dea1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:44.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.638+0000 7f632399a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f3de9d940 0x556f3dea1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:44.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.638+0000 7f632399a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f631c1adc50 con 0x556f3de93080 2024-09-12T03:03:44.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.638+0000 7f632399a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f631c1af0c0 con 0x556f3de92760 2024-09-12T03:03:44.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.638+0000 7f632399a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f631c1ae320 con 0x556f3de9d940 2024-09-12T03:03:44.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.638+0000 7f632399a740 10 client.0 _ll_register_callbacks cb 0x7f630c031660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:44.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.638+0000 7f631a7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f3de93080 0x556f3dea1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:44.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.638+0000 7f631a7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f3de93080 0x556f3dea1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52328/0 (socket says 172.21.15.116:52328) 2024-09-12T03:03:44.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.638+0000 7f631a7fc640 1 -- 172.21.15.116:0/3552116414 learned_addr learned my addr 172.21.15.116:0/3552116414 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:44.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.638+0000 7f631affd640 1 --2- 172.21.15.116:0/3552116414 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556f3de92760 0x556f3dea5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:44.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.638+0000 7f631b7fe640 1 --2- 172.21.15.116:0/3552116414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f3de9d940 0x556f3dea1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:44.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.638+0000 7f631affd640 1 -- 172.21.15.116:0/3552116414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f3de9d940 msgr2=0x556f3dea1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:44.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.638+0000 7f631affd640 1 --2- 172.21.15.116:0/3552116414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f3de9d940 0x556f3dea1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.638+0000 7f631affd640 1 -- 172.21.15.116:0/3552116414 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f3de93080 msgr2=0x556f3dea1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:44.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.638+0000 7f631affd640 1 --2- 172.21.15.116:0/3552116414 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f3de93080 0x556f3dea1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.638+0000 7f631affd640 1 -- 172.21.15.116:0/3552116414 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f3dea1c10 con 0x556f3de92760 2024-09-12T03:03:44.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.639+0000 7f631affd640 1 --2- 172.21.15.116:0/3552116414 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556f3de92760 0x556f3dea5de0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f630c045ca0 tx=0x7f630c005990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94cc49fe8ced033d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:44.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.639+0000 7f62feff5640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-09-12T03:03:44.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.639+0000 7f62feff5640 1 -- 172.21.15.116:0/3552116414 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c045680 con 0x556f3de92760 2024-09-12T03:03:44.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.639+0000 7f62feff5640 1 -- 172.21.15.116:0/3552116414 <== mon.2 v2:172.21.15.177:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f630c045820 con 0x556f3de92760 2024-09-12T03:03:44.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.639+0000 7f632399a740 1 -- 172.21.15.116:0/3552116414 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f630c031a40 con 0x556f3de92760 2024-09-12T03:03:44.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.640+0000 7f62feff5640 1 -- 172.21.15.116:0/3552116414 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f630c045a20 con 0x556f3de92760 2024-09-12T03:03:44.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.640+0000 7f632399a740 10 client.24899 fetch_fsmap learned FSMap version 25 2024-09-12T03:03:44.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.640+0000 7f632399a740 1 -- 172.21.15.116:0/3552116414 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556f3df186e0 con 0x556f3de92760 2024-09-12T03:03:44.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.640+0000 7f62feff5640 1 -- 172.21.15.116:0/3552116414 <== mon.2 v2:172.21.15.177:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f630c031a40 con 0x556f3de92760 2024-09-12T03:03:44.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.640+0000 7f62feff5640 1 -- 172.21.15.116:0/3552116414 <== mon.2 v2:172.21.15.177:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f630c03cb60 con 0x556f3de92760 2024-09-12T03:03:44.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.641+0000 7f632399a740 10 client.24899 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:03:44.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.641+0000 7f632399a740 10 client.24899 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:03:44.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.641+0000 7f632399a740 20 client.24899 populate_metadata read hostname 'smithi116' 2024-09-12T03:03:44.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.641+0000 7f632399a740 1 --2- 172.21.15.116:0/3552116414 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556f3df18da0 0x556f3df1b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:44.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.641+0000 7f632399a740 4 client.24899 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}] 2024-09-12T03:03:44.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.641+0000 7f632399a740 1 -- 172.21.15.116:0/3552116414 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}) -- 0x556f3ddffd20 con 0x556f3df18da0 2024-09-12T03:03:44.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.641+0000 7f631a7fc640 1 --2- 172.21.15.116:0/3552116414 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556f3df18da0 0x556f3df1b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:44.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.641+0000 7f631a7fc640 1 --2- 172.21.15.116:0/3552116414 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556f3df18da0 0x556f3df1b190 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e59eae61131dc8b5 server_cookie=b495dc86632d58ce in_seq=0 out_seq=0 2024-09-12T03:03:44.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.641+0000 7f62feff5640 10 client.24899 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:44.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.651+0000 7f62feff5640 1 -- 172.21.15.116:0/3552116414 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x556f3ddffd20 con 0x556f3df18da0 2024-09-12T03:03:44.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.651+0000 7f62feff5640 10 client.24899 handle_command_reply: tid=0 2024-09-12T03:03:44.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.651+0000 7f632399a740 1 client.24899 shutdown 2024-09-12T03:03:44.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.651+0000 7f632399a740 1 -- 172.21.15.116:0/3552116414 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556f3de92760 msgr2=0x556f3dea5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:44.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.651+0000 7f632399a740 1 --2- 172.21.15.116:0/3552116414 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556f3de92760 0x556f3dea5de0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f630c045ca0 tx=0x7f630c005990 comp rx=0 tx=0).stop 2024-09-12T03:03:44.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.651+0000 7f632399a740 1 -- 172.21.15.116:0/3552116414 shutdown_connections 2024-09-12T03:03:44.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.651+0000 7f632399a740 1 --2- 172.21.15.116:0/3552116414 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556f3df18da0 0x556f3df1b190 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.651+0000 7f632399a740 1 --2- 172.21.15.116:0/3552116414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f3de9d940 0x556f3dea1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.651+0000 7f632399a740 1 --2- 172.21.15.116:0/3552116414 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f3de93080 0x556f3dea1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.651+0000 7f62feff5640 0 client.24899 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:44.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.652+0000 7f632399a740 1 --2- 172.21.15.116:0/3552116414 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556f3de92760 0x556f3dea5de0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.652+0000 7f632399a740 1 -- 172.21.15.116:0/3552116414 >> 172.21.15.116:0/3552116414 conn(0x556f3de8ecf0 msgr2=0x556f3de8de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:44.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.652+0000 7f632399a740 1 -- 172.21.15.116:0/3552116414 shutdown_connections 2024-09-12T03:03:44.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.652+0000 7f632399a740 1 -- 172.21.15.116:0/3552116414 wait complete. 2024-09-12T03:03:44.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.652+0000 7f632399a740 20 client.24899 trim_cache size 0 max 16384 2024-09-12T03:03:44.653 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:03:44.654 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 25, 2024-09-12T03:03:44.654 INFO:teuthology.orchestra.run.smithi116.stdout: "leader": 24494, 2024-09-12T03:03:44.654 INFO:teuthology.orchestra.run.smithi116.stdout: "set_version": 2, 2024-09-12T03:03:44.654 INFO:teuthology.orchestra.run.smithi116.stdout: "sets": { 2024-09-12T03:03:44.654 INFO:teuthology.orchestra.run.smithi116.stdout: "fc4784bc": { 2024-09-12T03:03:44.654 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 2, 2024-09-12T03:03:44.654 INFO:teuthology.orchestra.run.smithi116.stdout: "age_ref": 0.0, 2024-09-12T03:03:44.654 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:03:44.654 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "QUIESCED", 2024-09-12T03:03:44.654 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:03:44.654 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:03:44.655 INFO:teuthology.orchestra.run.smithi116.stdout: "timeout": 90.0, 2024-09-12T03:03:44.655 INFO:teuthology.orchestra.run.smithi116.stdout: "expiration": 144.5, 2024-09-12T03:03:44.655 INFO:teuthology.orchestra.run.smithi116.stdout: "members": { 2024-09-12T03:03:44.655 INFO:teuthology.orchestra.run.smithi116.stdout: "file:/": { 2024-09-12T03:03:44.655 INFO:teuthology.orchestra.run.smithi116.stdout: "excluded": false, 2024-09-12T03:03:44.655 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:03:44.655 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "QUIESCED", 2024-09-12T03:03:44.655 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:03:44.655 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:03:44.655 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:03:44.655 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:03:44.655 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:03:44.655 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:03:44.656 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:03:44.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.653+0000 7f63215c1640 1 -- 172.21.15.116:0/2276685196 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f63040a2c90 msgr2=0x7f63040a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:44.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.653+0000 7f63215c1640 1 --2- 172.21.15.116:0/2276685196 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f63040a2c90 0x7f63040a5150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6314020000 tx=0x7f631401d000 comp rx=0 tx=0).stop 2024-09-12T03:03:44.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.653+0000 7f63215c1640 1 -- 172.21.15.116:0/2276685196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f631c102780 msgr2=0x7f631c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:44.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.653+0000 7f63215c1640 1 --2- 172.21.15.116:0/2276685196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f631c102780 0x7f631c074a30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f631000c920 tx=0x7f631000cde0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.653+0000 7f63215c1640 1 -- 172.21.15.116:0/2276685196 shutdown_connections 2024-09-12T03:03:44.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.653+0000 7f63215c1640 1 --2- 172.21.15.116:0/2276685196 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f63040a2c90 0x7f63040a5150 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.653+0000 7f63215c1640 1 --2- 172.21.15.116:0/2276685196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f631c106f00 0x7f631c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.653+0000 7f63215c1640 1 --2- 172.21.15.116:0/2276685196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f631c102780 0x7f631c074a30 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.653+0000 7f63215c1640 1 --2- 172.21.15.116:0/2276685196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f631c101dd0 0x7f631c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:44.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.653+0000 7f63215c1640 1 -- 172.21.15.116:0/2276685196 >> 172.21.15.116:0/2276685196 conn(0x7f631c0fd8b0 msgr2=0x7f631c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:44.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.653+0000 7f63215c1640 1 -- 172.21.15.116:0/2276685196 shutdown_connections 2024-09-12T03:03:44.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:44.653+0000 7f63215c1640 1 -- 172.21.15.116:0/2276685196 wait complete. 2024-09-12T03:03:44.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:44 smithi183 ceph-mon[35535]: pgmap v360: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:03:44.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:44 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/191223688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:44.908 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:46.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:46 smithi177 ceph-mon[35305]: pgmap v361: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:03:46.505 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:46 smithi183 ceph-mon[35535]: pgmap v361: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:03:46.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:46 smithi116 ceph-mon[31410]: pgmap v361: 97 pgs: 97 active+clean; 655 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:03:47.133 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'fc4784bc', quiesce took 6.2 seconds. Will release after: 18.3 2024-09-12T03:03:48.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:48 smithi177 ceph-mon[35305]: pgmap v362: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T03:03:48.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:48 smithi116 ceph-mon[31410]: pgmap v362: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T03:03:48.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:48 smithi183 ceph-mon[35535]: pgmap v362: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T03:03:50.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.076+0000 7f768f76b640 1 -- 172.21.15.116:0/400592613 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7688102570 msgr2=0x7f76881029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:50.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.076+0000 7f768f76b640 1 --2- 172.21.15.116:0/400592613 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7688102570 0x7f76881029b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7678009900 tx=0x7f767802a480 comp rx=0 tx=0).stop 2024-09-12T03:03:50.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.076+0000 7f768f76b640 1 -- 172.21.15.116:0/400592613 shutdown_connections 2024-09-12T03:03:50.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.076+0000 7f768f76b640 1 --2- 172.21.15.116:0/400592613 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7688102ef0 0x7f768810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.076+0000 7f768f76b640 1 --2- 172.21.15.116:0/400592613 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7688102570 0x7f76881029b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.076+0000 7f768f76b640 1 --2- 172.21.15.116:0/400592613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7688100e20 0x7f7688101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.076+0000 7f768f76b640 1 -- 172.21.15.116:0/400592613 >> 172.21.15.116:0/400592613 conn(0x7f76880fc7d0 msgr2=0x7f76880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:50.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.076+0000 7f768f76b640 1 -- 172.21.15.116:0/400592613 shutdown_connections 2024-09-12T03:03:50.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.076+0000 7f768f76b640 1 -- 172.21.15.116:0/400592613 wait complete. 2024-09-12T03:03:50.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.077+0000 7f768f76b640 1 Processor -- start 2024-09-12T03:03:50.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.077+0000 7f768f76b640 1 -- start start 2024-09-12T03:03:50.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.077+0000 7f768f76b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7688100e20 0x7f7688199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:50.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.077+0000 7f768f76b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7688102570 0x7f76881998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:50.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.077+0000 7f768f76b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7688102ef0 0x7f76881a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:50.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.077+0000 7f768f76b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7688110f80 con 0x7f7688100e20 2024-09-12T03:03:50.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.077+0000 7f768f76b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7688110e00 con 0x7f7688102570 2024-09-12T03:03:50.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.077+0000 7f768f76b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7688111100 con 0x7f7688102ef0 2024-09-12T03:03:50.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.077+0000 7f768d507640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7688100e20 0x7f7688199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:50.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.077+0000 7f768d507640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7688100e20 0x7f7688199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52336/0 (socket says 172.21.15.116:52336) 2024-09-12T03:03:50.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.077+0000 7f768d507640 1 -- 172.21.15.116:0/4039810434 learned_addr learned my addr 172.21.15.116:0/4039810434 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:50.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.078+0000 7f768cd06640 1 --2- 172.21.15.116:0/4039810434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7688102570 0x7f76881998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:50.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.078+0000 7f768dd08640 1 --2- 172.21.15.116:0/4039810434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7688102ef0 0x7f76881a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:50.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.078+0000 7f768d507640 1 -- 172.21.15.116:0/4039810434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7688102ef0 msgr2=0x7f76881a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:50.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.078+0000 7f768d507640 1 --2- 172.21.15.116:0/4039810434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7688102ef0 0x7f76881a4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.078+0000 7f768d507640 1 -- 172.21.15.116:0/4039810434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7688102570 msgr2=0x7f76881998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:50.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.078+0000 7f768d507640 1 --2- 172.21.15.116:0/4039810434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7688102570 0x7f76881998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.078+0000 7f768d507640 1 -- 172.21.15.116:0/4039810434 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7688197180 con 0x7f7688100e20 2024-09-12T03:03:50.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.078+0000 7f768d507640 1 --2- 172.21.15.116:0/4039810434 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7688100e20 0x7f7688199390 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f768400bea0 tx=0x7f768400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:50.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.078+0000 7f767e7fc640 1 -- 172.21.15.116:0/4039810434 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768401c070 con 0x7f7688100e20 2024-09-12T03:03:50.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.078+0000 7f767e7fc640 1 -- 172.21.15.116:0/4039810434 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7684009d00 con 0x7f7688100e20 2024-09-12T03:03:50.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.078+0000 7f768f76b640 1 -- 172.21.15.116:0/4039810434 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7688197470 con 0x7f7688100e20 2024-09-12T03:03:50.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.078+0000 7f768f76b640 1 -- 172.21.15.116:0/4039810434 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76881979b0 con 0x7f7688100e20 2024-09-12T03:03:50.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.079+0000 7f767e7fc640 1 -- 172.21.15.116:0/4039810434 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768401fd70 con 0x7f7688100e20 2024-09-12T03:03:50.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.081+0000 7f7691b44740 1 Processor -- start 2024-09-12T03:03:50.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.081+0000 7f7691b44740 1 -- start start 2024-09-12T03:03:50.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.081+0000 7f7691b44740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d7bddbdb40 0x55d7bddbc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:50.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.081+0000 7f7691b44740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7bddbc5d0 0x55d7bddbca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:50.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.082+0000 7f7691b44740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d7bddc7370 0x55d7bddc9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:50.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.082+0000 7f7691b44740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d7bdd222f0 con 0x55d7bddbdb40 2024-09-12T03:03:50.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.082+0000 7f7691b44740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d7bdcc9cf0 con 0x55d7bddc7370 2024-09-12T03:03:50.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.082+0000 7f7691b44740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d7bdd1e2b0 con 0x55d7bddbc5d0 2024-09-12T03:03:50.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.082+0000 7f768dd08640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d7bddbdb40 0x55d7bddbc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:50.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.082+0000 7f768dd08640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d7bddbdb40 0x55d7bddbc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52350/0 (socket says 172.21.15.116:52350) 2024-09-12T03:03:50.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.082+0000 7f768dd08640 1 -- 172.21.15.116:0/3099225204 learned_addr learned my addr 172.21.15.116:0/3099225204 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:50.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.082+0000 7f767e7fc640 1 -- 172.21.15.116:0/4039810434 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f7684009820 con 0x7f7688100e20 2024-09-12T03:03:50.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.082+0000 7f768cd06640 1 --2- 172.21.15.116:0/3099225204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d7bddc7370 0x55d7bddc9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:50.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.082+0000 7f768dd08640 1 -- 172.21.15.116:0/3099225204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7bddbc5d0 msgr2=0x55d7bddbca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:50.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.082+0000 7f768dd08640 1 --2- 172.21.15.116:0/3099225204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7bddbc5d0 0x55d7bddbca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.082+0000 7f768dd08640 1 -- 172.21.15.116:0/3099225204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d7bddc7370 msgr2=0x55d7bddc9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:50.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.082+0000 7f768dd08640 1 --2- 172.21.15.116:0/3099225204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d7bddc7370 0x55d7bddc9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.082+0000 7f768dd08640 1 -- 172.21.15.116:0/3099225204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7bdcae3a0 con 0x55d7bddbdb40 2024-09-12T03:03:50.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.082+0000 7f768cd06640 1 --2- 172.21.15.116:0/3099225204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d7bddc7370 0x55d7bddc9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:03:50.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.082+0000 7f768dd08640 1 --2- 172.21.15.116:0/3099225204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d7bddbdb40 0x55d7bddbc090 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7680009f20 tx=0x7f768000f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4075c7486706bb87 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:50.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.082+0000 7f76737fe640 1 -- 172.21.15.116:0/3099225204 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7680015070 con 0x55d7bddbdb40 2024-09-12T03:03:50.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.083+0000 7f76737fe640 1 -- 172.21.15.116:0/3099225204 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f768001ab40 con 0x55d7bddbdb40 2024-09-12T03:03:50.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.083+0000 7f76737fe640 1 -- 172.21.15.116:0/3099225204 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7680011c50 con 0x55d7bddbdb40 2024-09-12T03:03:50.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.083+0000 7f767e7fc640 1 --2- 172.21.15.116:0/4039810434 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f76740a2c90 0x7f76740a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:50.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.083+0000 7f767e7fc640 1 -- 172.21.15.116:0/4039810434 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f76840d75d0 con 0x7f7688100e20 2024-09-12T03:03:50.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.083+0000 7f768d507640 1 --2- 172.21.15.116:0/4039810434 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f76740a2c90 0x7f76740a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:50.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.084+0000 7f768d507640 1 --2- 172.21.15.116:0/4039810434 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f76740a2c90 0x7f76740a5150 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f76840ac4f0 tx=0x7f7684025000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:50.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.084+0000 7f7691b44740 1 -- 172.21.15.116:0/3099225204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d7bddbdb40 msgr2=0x55d7bddbc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:50.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.084+0000 7f7691b44740 1 --2- 172.21.15.116:0/3099225204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d7bddbdb40 0x55d7bddbc090 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7680009f20 tx=0x7f768000f7f0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.084+0000 7f7691b44740 1 -- 172.21.15.116:0/3099225204 shutdown_connections 2024-09-12T03:03:50.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.084+0000 7f7691b44740 1 --2- 172.21.15.116:0/3099225204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d7bddc7370 0x55d7bddc9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.084+0000 7f7691b44740 1 --2- 172.21.15.116:0/3099225204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7bddbc5d0 0x55d7bddbca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.084+0000 7f7691b44740 1 --2- 172.21.15.116:0/3099225204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d7bddbdb40 0x55d7bddbc090 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.084+0000 7f7691b44740 1 -- 172.21.15.116:0/3099225204 >> 172.21.15.116:0/3099225204 conn(0x55d7bdd85310 msgr2=0x55d7bddb8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:50.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.084+0000 7f7691b44740 1 -- 172.21.15.116:0/3099225204 shutdown_connections 2024-09-12T03:03:50.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.084+0000 7f7691b44740 1 -- 172.21.15.116:0/3099225204 wait complete. 2024-09-12T03:03:50.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.084+0000 7f7691b44740 1 -- start start 2024-09-12T03:03:50.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.085+0000 7f7691b44740 1 Processor -- start 2024-09-12T03:03:50.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.085+0000 7f7691b44740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d7bddbc5d0 0x55d7bddc6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:50.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.085+0000 7f7691b44740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7bddbdb40 0x55d7bddc4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:50.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.085+0000 7f768dd08640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d7bddbc5d0 0x55d7bddc6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:50.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.085+0000 7f768dd08640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d7bddbc5d0 0x55d7bddc6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52358/0 (socket says 172.21.15.116:52358) 2024-09-12T03:03:50.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.085+0000 7f7691b44740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d7bddc7370 0x55d7bddc4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:50.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.085+0000 7f768dd08640 1 -- 172.21.15.116:0/3771246967 learned_addr learned my addr 172.21.15.116:0/3771246967 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:50.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.085+0000 7f7691b44740 1 -- 172.21.15.116:0/3771246967 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d7bddca070 con 0x55d7bddbc5d0 2024-09-12T03:03:50.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.085+0000 7f7691b44740 1 -- 172.21.15.116:0/3771246967 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d7bdcdd060 con 0x55d7bddc7370 2024-09-12T03:03:50.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.085+0000 7f7691b44740 1 -- 172.21.15.116:0/3771246967 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d7bddca1f0 con 0x55d7bddbdb40 2024-09-12T03:03:50.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.085+0000 7f768cd06640 1 --2- 172.21.15.116:0/3771246967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7bddbdb40 0x55d7bddc4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:50.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.086+0000 7f768dd08640 1 -- 172.21.15.116:0/3771246967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7bddbdb40 msgr2=0x55d7bddc4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:50.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.086+0000 7f768dd08640 1 --2- 172.21.15.116:0/3771246967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7bddbdb40 0x55d7bddc4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.086+0000 7f768dd08640 1 -- 172.21.15.116:0/3771246967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d7bddc7370 msgr2=0x55d7bddc4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:50.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.086+0000 7f768dd08640 1 --2- 172.21.15.116:0/3771246967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d7bddc7370 0x55d7bddc4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.085+0000 7f7691b44740 10 client.0 _ll_register_callbacks cb 0x55d7bda308c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:50.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.086+0000 7f768dd08640 1 -- 172.21.15.116:0/3771246967 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7bda933c0 con 0x55d7bddbc5d0 2024-09-12T03:03:50.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.086+0000 7f7691b44740 1 -- 172.21.15.116:0/3771246967 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f768001ab40 con 0x55d7bddbc5d0 2024-09-12T03:03:50.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.086+0000 7f768dd08640 1 --2- 172.21.15.116:0/3771246967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d7bddbc5d0 0x55d7bddc6d70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7680011590 tx=0x7f7680009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b991ed1550835441 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:50.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.086+0000 7f7670ff9640 10 client.15429 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:03:50.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.086+0000 7f7670ff9640 1 -- 172.21.15.116:0/3771246967 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7680028060 con 0x55d7bddbc5d0 2024-09-12T03:03:50.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.086+0000 7f7670ff9640 1 -- 172.21.15.116:0/3771246967 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f768001a460 con 0x55d7bddbc5d0 2024-09-12T03:03:50.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.086+0000 7f7670ff9640 1 -- 172.21.15.116:0/3771246967 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7680015070 con 0x55d7bddbc5d0 2024-09-12T03:03:50.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.086+0000 7f7691b44740 10 client.15429 fetch_fsmap learned FSMap version 25 2024-09-12T03:03:50.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.086+0000 7f7691b44740 1 -- 172.21.15.116:0/3771246967 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55d7bddd5a70 con 0x55d7bddbc5d0 2024-09-12T03:03:50.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.087+0000 7f7670ff9640 1 -- 172.21.15.116:0/3771246967 <== mon.0 v2:172.21.15.116:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f768001a6a0 con 0x55d7bddbc5d0 2024-09-12T03:03:50.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.087+0000 7f7691b44740 10 client.15429 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:03:50.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.087+0000 7f7691b44740 10 client.15429 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:03:50.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.087+0000 7f7691b44740 20 client.15429 populate_metadata read hostname 'smithi116' 2024-09-12T03:03:50.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.087+0000 7f7691b44740 1 --2- 172.21.15.116:0/3771246967 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55d7bddd6dc0 0x55d7bddd91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:50.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.087+0000 7f7691b44740 4 client.15429 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:03:50.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.087+0000 7f7691b44740 1 -- 172.21.15.116:0/3771246967 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7bdd2f860 con 0x55d7bddd6dc0 2024-09-12T03:03:50.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.087+0000 7f768cd06640 1 --2- 172.21.15.116:0/3771246967 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55d7bddd6dc0 0x55d7bddd91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:50.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.088+0000 7f768cd06640 1 --2- 172.21.15.116:0/3771246967 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55d7bddd6dc0 0x55d7bddd91b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c14f1a8d5b772a88 server_cookie=4a85ce2f83313b45 in_seq=0 out_seq=0 2024-09-12T03:03:50.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.088+0000 7f7670ff9640 10 client.15429 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:03:50.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.088+0000 7f7670ff9640 1 -- 172.21.15.116:0/3771246967 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d7bdd2f860 con 0x55d7bddd6dc0 2024-09-12T03:03:50.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.088+0000 7f7670ff9640 10 client.15429 handle_command_reply: tid=0 2024-09-12T03:03:50.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.089+0000 7f7691b44740 1 client.15429 shutdown 2024-09-12T03:03:50.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.089+0000 7f7691b44740 1 -- 172.21.15.116:0/3771246967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d7bddbc5d0 msgr2=0x55d7bddc6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:50.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.089+0000 7f7691b44740 1 --2- 172.21.15.116:0/3771246967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d7bddbc5d0 0x55d7bddc6d70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7680011590 tx=0x7f7680009c90 comp rx=0 tx=0).stop 2024-09-12T03:03:50.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.089+0000 7f7691b44740 1 -- 172.21.15.116:0/3771246967 shutdown_connections 2024-09-12T03:03:50.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.089+0000 7f7691b44740 1 --2- 172.21.15.116:0/3771246967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d7bddc7370 0x55d7bddc4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.089+0000 7f7691b44740 1 --2- 172.21.15.116:0/3771246967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7bddbdb40 0x55d7bddc4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.089+0000 7f7691b44740 1 --2- 172.21.15.116:0/3771246967 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55d7bddd6dc0 0x55d7bddd91b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.089+0000 7f7691b44740 1 --2- 172.21.15.116:0/3771246967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d7bddbc5d0 0x55d7bddc6d70 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.089+0000 7f7691b44740 1 -- 172.21.15.116:0/3771246967 >> 172.21.15.116:0/3771246967 conn(0x55d7bdd85310 msgr2=0x55d7bddbe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:50.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.089+0000 7f7670ff9640 0 client.15429 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:03:50.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.089+0000 7f7691b44740 1 -- 172.21.15.116:0/3771246967 shutdown_connections 2024-09-12T03:03:50.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.089+0000 7f7691b44740 1 -- 172.21.15.116:0/3771246967 wait complete. 2024-09-12T03:03:50.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.089+0000 7f7691b44740 20 client.15429 trim_cache size 0 max 16384 2024-09-12T03:03:50.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.111+0000 7f7691b44740 1 Processor -- start 2024-09-12T03:03:50.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.111+0000 7f7691b44740 1 -- start start 2024-09-12T03:03:50.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.111+0000 7f7691b44740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d7bddbc5d0 0x55d7bddb86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:50.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.111+0000 7f7691b44740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d7bddbdb40 0x55d7bdddbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:50.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.111+0000 7f7691b44740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7bddc7370 0x55d7bdddc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:50.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.111+0000 7f7691b44740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f76881adb40 con 0x55d7bddbdb40 2024-09-12T03:03:50.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.111+0000 7f7691b44740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f76881ab410 con 0x55d7bddbc5d0 2024-09-12T03:03:50.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.111+0000 7f7691b44740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f76881a9280 con 0x55d7bddc7370 2024-09-12T03:03:50.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.111+0000 7f768cd06640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d7bddbdb40 0x55d7bdddbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:50.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.111+0000 7f768cd06640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d7bddbdb40 0x55d7bdddbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52360/0 (socket says 172.21.15.116:52360) 2024-09-12T03:03:50.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.111+0000 7f768cd06640 1 -- 172.21.15.116:0/618130812 learned_addr learned my addr 172.21.15.116:0/618130812 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:50.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.112+0000 7f768cd06640 1 -- 172.21.15.116:0/618130812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7bddc7370 msgr2=0x55d7bdddc330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:03:50.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.112+0000 7f768cd06640 1 --2- 172.21.15.116:0/618130812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7bddc7370 0x55d7bdddc330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.112+0000 7f768cd06640 1 -- 172.21.15.116:0/618130812 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d7bddbc5d0 msgr2=0x55d7bddb86e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:03:50.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.112+0000 7f768cd06640 1 --2- 172.21.15.116:0/618130812 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d7bddbc5d0 0x55d7bddb86e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.112+0000 7f768cd06640 1 -- 172.21.15.116:0/618130812 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7bddb84f0 con 0x55d7bddbdb40 2024-09-12T03:03:50.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.112+0000 7f768cd06640 1 --2- 172.21.15.116:0/618130812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d7bddbdb40 0x55d7bdddbdf0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f767802a9a0 tx=0x7f7678005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96c84722127abfd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:50.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.112+0000 7f7672ffd640 1 -- 172.21.15.116:0/618130812 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7678035040 con 0x55d7bddbdb40 2024-09-12T03:03:50.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.112+0000 7f7672ffd640 1 -- 172.21.15.116:0/618130812 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7678048070 con 0x55d7bddbdb40 2024-09-12T03:03:50.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.112+0000 7f7672ffd640 1 -- 172.21.15.116:0/618130812 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7678004e60 con 0x55d7bddbdb40 2024-09-12T03:03:50.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.113+0000 7f7691b44740 1 -- 172.21.15.116:0/618130812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d7bddbdb40 msgr2=0x55d7bdddbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:50.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.113+0000 7f7691b44740 1 --2- 172.21.15.116:0/618130812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d7bddbdb40 0x55d7bdddbdf0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f767802a9a0 tx=0x7f7678005a20 comp rx=0 tx=0).stop 2024-09-12T03:03:50.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.113+0000 7f7691b44740 1 -- 172.21.15.116:0/618130812 shutdown_connections 2024-09-12T03:03:50.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.113+0000 7f7691b44740 1 --2- 172.21.15.116:0/618130812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7bddc7370 0x55d7bdddc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.113+0000 7f7691b44740 1 --2- 172.21.15.116:0/618130812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d7bddbdb40 0x55d7bdddbdf0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.113+0000 7f7691b44740 1 --2- 172.21.15.116:0/618130812 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d7bddbc5d0 0x55d7bddb86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.113+0000 7f7691b44740 1 -- 172.21.15.116:0/618130812 >> 172.21.15.116:0/618130812 conn(0x55d7bdd85310 msgr2=0x55d7bddc3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:50.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.113+0000 7f7691b44740 1 -- 172.21.15.116:0/618130812 shutdown_connections 2024-09-12T03:03:50.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.113+0000 7f7691b44740 1 -- 172.21.15.116:0/618130812 wait complete. 2024-09-12T03:03:50.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.113+0000 7f7691b44740 1 -- start start 2024-09-12T03:03:50.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.114+0000 7f7691b44740 1 Processor -- start 2024-09-12T03:03:50.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.114+0000 7f7691b44740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d7bddbc5d0 0x55d7bddd9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:50.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.114+0000 7f7691b44740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d7bddbdb40 0x55d7bddda3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:50.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.114+0000 7f7691b44740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7bddc7370 0x55d7bddda910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:50.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.114+0000 7f7691b44740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f76881a98d0 con 0x55d7bddbc5d0 2024-09-12T03:03:50.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.114+0000 7f7691b44740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f76881aad40 con 0x55d7bddbdb40 2024-09-12T03:03:50.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.114+0000 7f7691b44740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f76881a9fa0 con 0x55d7bddc7370 2024-09-12T03:03:50.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.114+0000 7f7691b44740 10 client.0 _ll_register_callbacks cb 0x7f7680009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:50.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.114+0000 7f768dd08640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d7bddbc5d0 0x55d7bddd9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:50.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.114+0000 7f768dd08640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d7bddbc5d0 0x55d7bddd9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52374/0 (socket says 172.21.15.116:52374) 2024-09-12T03:03:50.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.114+0000 7f768dd08640 1 -- 172.21.15.116:0/3873812020 learned_addr learned my addr 172.21.15.116:0/3873812020 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:50.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.114+0000 7f768cd06640 1 --2- 172.21.15.116:0/3873812020 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d7bddbdb40 0x55d7bddda3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:50.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.115+0000 7f768dd08640 1 --2- 172.21.15.116:0/3873812020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7bddc7370 0x55d7bddda910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:50.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.115+0000 7f768dd08640 1 -- 172.21.15.116:0/3873812020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7bddc7370 msgr2=0x55d7bddda910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:50.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.115+0000 7f768dd08640 1 --2- 172.21.15.116:0/3873812020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7bddc7370 0x55d7bddda910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.115+0000 7f768dd08640 1 -- 172.21.15.116:0/3873812020 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d7bddbdb40 msgr2=0x55d7bddda3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:50.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.115+0000 7f768dd08640 1 --2- 172.21.15.116:0/3873812020 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d7bddbdb40 0x55d7bddda3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.115+0000 7f768dd08640 1 -- 172.21.15.116:0/3873812020 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7bde409d0 con 0x55d7bddbc5d0 2024-09-12T03:03:50.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.115+0000 7f768cd06640 1 --2- 172.21.15.116:0/3873812020 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d7bddbdb40 0x55d7bddda3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:03:50.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.115+0000 7f768dd08640 1 --2- 172.21.15.116:0/3873812020 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d7bddbc5d0 0x55d7bddd9e70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7680009740 tx=0x7f7680015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f23cecb932df5a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:50.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.115+0000 7f7670ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:03:50.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.115+0000 7f7670ff9640 1 -- 172.21.15.116:0/3873812020 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768001a680 con 0x55d7bddbc5d0 2024-09-12T03:03:50.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.115+0000 7f7670ff9640 1 -- 172.21.15.116:0/3873812020 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f7680005160 con 0x55d7bddbc5d0 2024-09-12T03:03:50.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.115+0000 7f7691b44740 1 -- 172.21.15.116:0/3873812020 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7678048070 con 0x55d7bddbc5d0 2024-09-12T03:03:50.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.116+0000 7f7670ff9640 1 -- 172.21.15.116:0/3873812020 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f768003f050 con 0x55d7bddbc5d0 2024-09-12T03:03:50.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.116+0000 7f7691b44740 10 client.15435 fetch_fsmap learned FSMap version 25 2024-09-12T03:03:50.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.116+0000 7f7691b44740 1 -- 172.21.15.116:0/3873812020 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d7bde41cd0 con 0x55d7bddbc5d0 2024-09-12T03:03:50.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.116+0000 7f7670ff9640 1 -- 172.21.15.116:0/3873812020 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7678048070 con 0x55d7bddbc5d0 2024-09-12T03:03:50.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.116+0000 7f7670ff9640 1 -- 172.21.15.116:0/3873812020 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f76800118c0 con 0x55d7bddbc5d0 2024-09-12T03:03:50.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.116+0000 7f7691b44740 10 client.15435 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:03:50.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.116+0000 7f7691b44740 10 client.15435 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:03:50.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.116+0000 7f7691b44740 20 client.15435 populate_metadata read hostname 'smithi116' 2024-09-12T03:03:50.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.116+0000 7f7691b44740 1 --2- 172.21.15.116:0/3873812020 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55d7bde423f0 0x55d7bde44810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:50.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.116+0000 7f7691b44740 4 client.15435 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:03:50.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.116+0000 7f7691b44740 1 -- 172.21.15.116:0/3873812020 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d7bdd29d70 con 0x55d7bde423f0 2024-09-12T03:03:50.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.116+0000 7f768cd06640 1 --2- 172.21.15.116:0/3873812020 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55d7bde423f0 0x55d7bde44810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:50.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.117+0000 7f7670ff9640 10 client.15435 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:03:50.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.117+0000 7f768cd06640 1 --2- 172.21.15.116:0/3873812020 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55d7bde423f0 0x55d7bde44810 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=82ccdd51a47affc5 server_cookie=83859f39651c83ed in_seq=0 out_seq=0 2024-09-12T03:03:50.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.117+0000 7f7670ff9640 1 -- 172.21.15.116:0/3873812020 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d7bdd29d70 con 0x55d7bde423f0 2024-09-12T03:03:50.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.117+0000 7f7670ff9640 10 client.15435 handle_command_reply: tid=0 2024-09-12T03:03:50.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.117+0000 7f7691b44740 1 client.15435 shutdown 2024-09-12T03:03:50.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.117+0000 7f7691b44740 1 -- 172.21.15.116:0/3873812020 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d7bddbc5d0 msgr2=0x55d7bddd9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:50.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.117+0000 7f7691b44740 1 --2- 172.21.15.116:0/3873812020 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d7bddbc5d0 0x55d7bddd9e70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7680009740 tx=0x7f7680015000 comp rx=0 tx=0).stop 2024-09-12T03:03:50.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.117+0000 7f7691b44740 1 -- 172.21.15.116:0/3873812020 shutdown_connections 2024-09-12T03:03:50.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.117+0000 7f7691b44740 1 --2- 172.21.15.116:0/3873812020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7bddc7370 0x55d7bddda910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.117+0000 7f7691b44740 1 --2- 172.21.15.116:0/3873812020 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d7bddbdb40 0x55d7bddda3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.117+0000 7f7691b44740 1 --2- 172.21.15.116:0/3873812020 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55d7bde423f0 0x55d7bde44810 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.117+0000 7f7691b44740 1 --2- 172.21.15.116:0/3873812020 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d7bddbc5d0 0x55d7bddd9e70 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.117+0000 7f7670ff9640 0 client.15435 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:03:50.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.118+0000 7f7691b44740 1 -- 172.21.15.116:0/3873812020 >> 172.21.15.116:0/3873812020 conn(0x55d7bdd85310 msgr2=0x55d7bddb74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:50.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.118+0000 7f7691b44740 1 -- 172.21.15.116:0/3873812020 shutdown_connections 2024-09-12T03:03:50.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.118+0000 7f7691b44740 1 -- 172.21.15.116:0/3873812020 wait complete. 2024-09-12T03:03:50.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.118+0000 7f7691b44740 20 client.15435 trim_cache size 0 max 16384 2024-09-12T03:03:50.123 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:03:50.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.118+0000 7f768f76b640 1 -- 172.21.15.116:0/4039810434 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f76740a2c90 msgr2=0x7f76740a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:50.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.118+0000 7f768f76b640 1 --2- 172.21.15.116:0/4039810434 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f76740a2c90 0x7f76740a5150 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f76840ac4f0 tx=0x7f7684025000 comp rx=0 tx=0).stop 2024-09-12T03:03:50.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.118+0000 7f768f76b640 1 -- 172.21.15.116:0/4039810434 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7688100e20 msgr2=0x7f7688199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:50.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.118+0000 7f768f76b640 1 --2- 172.21.15.116:0/4039810434 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7688100e20 0x7f7688199390 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f768400bea0 tx=0x7f768400bf90 comp rx=0 tx=0).stop 2024-09-12T03:03:50.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.119+0000 7f768f76b640 1 -- 172.21.15.116:0/4039810434 shutdown_connections 2024-09-12T03:03:50.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.119+0000 7f768f76b640 1 --2- 172.21.15.116:0/4039810434 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f76740a2c90 0x7f76740a5150 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.119+0000 7f768f76b640 1 --2- 172.21.15.116:0/4039810434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7688102ef0 0x7f76881a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.119+0000 7f768f76b640 1 --2- 172.21.15.116:0/4039810434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7688102570 0x7f76881998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.119+0000 7f768f76b640 1 --2- 172.21.15.116:0/4039810434 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7688100e20 0x7f7688199390 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:50.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.119+0000 7f768f76b640 1 -- 172.21.15.116:0/4039810434 >> 172.21.15.116:0/4039810434 conn(0x7f76880fc7d0 msgr2=0x7f76880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:50.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.119+0000 7f768f76b640 1 -- 172.21.15.116:0/4039810434 shutdown_connections 2024-09-12T03:03:50.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:50.119+0000 7f768f76b640 1 -- 172.21.15.116:0/4039810434 wait complete. 2024-09-12T03:03:50.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:50 smithi116 ceph-mon[31410]: pgmap v363: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-12T03:03:50.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:50 smithi177 ceph-mon[35305]: pgmap v363: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-12T03:03:50.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:50 smithi183 ceph-mon[35535]: pgmap v363: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-12T03:03:51.103 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:03:51.426 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:51.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.935+0000 7f7af0f85640 1 -- 172.21.15.116:0/4038056150 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7aec100920 msgr2=0x7f7aec100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:51.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.935+0000 7f7af0f85640 1 --2- 172.21.15.116:0/4038056150 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7aec100920 0x7f7aec100d00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009900 tx=0x7f7adc02a410 comp rx=0 tx=0).stop 2024-09-12T03:03:51.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.935+0000 7f7af0f85640 1 -- 172.21.15.116:0/4038056150 shutdown_connections 2024-09-12T03:03:51.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.935+0000 7f7af0f85640 1 --2- 172.21.15.116:0/4038056150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7aec10e4d0 0x7f7aec1108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.935+0000 7f7af0f85640 1 --2- 172.21.15.116:0/4038056150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7aec1012d0 0x7f7aec10df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.935+0000 7f7af0f85640 1 --2- 172.21.15.116:0/4038056150 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7aec100920 0x7f7aec100d00 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.935+0000 7f7af0f85640 1 -- 172.21.15.116:0/4038056150 >> 172.21.15.116:0/4038056150 conn(0x7f7aec0fc790 msgr2=0x7f7aec0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:51.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.935+0000 7f7af0f85640 1 -- 172.21.15.116:0/4038056150 shutdown_connections 2024-09-12T03:03:51.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.935+0000 7f7af0f85640 1 -- 172.21.15.116:0/4038056150 wait complete. 2024-09-12T03:03:51.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.936+0000 7f7af0f85640 1 Processor -- start 2024-09-12T03:03:51.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.936+0000 7f7af0f85640 1 -- start start 2024-09-12T03:03:51.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.937+0000 7f7af0f85640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7aec100920 0x7f7aec19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:51.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.937+0000 7f7af0f85640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7aec1012d0 0x7f7aec19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:51.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.937+0000 7f7af0f85640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7aec10e4d0 0x7f7aec19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:51.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.937+0000 7f7aea59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7aec100920 0x7f7aec19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:51.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.937+0000 7f7aea59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7aec100920 0x7f7aec19a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39328/0 (socket says 172.21.15.116:39328) 2024-09-12T03:03:51.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.937+0000 7f7aea59c640 1 -- 172.21.15.116:0/2614653359 learned_addr learned my addr 172.21.15.116:0/2614653359 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:51.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.937+0000 7f7af0f85640 1 -- 172.21.15.116:0/2614653359 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec114160 con 0x7f7aec100920 2024-09-12T03:03:51.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.937+0000 7f7af0f85640 1 -- 172.21.15.116:0/2614653359 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec113fe0 con 0x7f7aec1012d0 2024-09-12T03:03:51.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.937+0000 7f7ae9d9b640 1 --2- 172.21.15.116:0/2614653359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7aec1012d0 0x7f7aec19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:51.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.937+0000 7f7af0f85640 1 -- 172.21.15.116:0/2614653359 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec1142e0 con 0x7f7aec10e4d0 2024-09-12T03:03:51.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.937+0000 7f7aead9d640 1 --2- 172.21.15.116:0/2614653359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7aec10e4d0 0x7f7aec19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:51.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.937+0000 7f7aea59c640 1 -- 172.21.15.116:0/2614653359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7aec10e4d0 msgr2=0x7f7aec19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:51.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.937+0000 7f7aea59c640 1 --2- 172.21.15.116:0/2614653359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7aec10e4d0 0x7f7aec19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.937+0000 7f7aea59c640 1 -- 172.21.15.116:0/2614653359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7aec1012d0 msgr2=0x7f7aec19abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:51.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.937+0000 7f7aea59c640 1 --2- 172.21.15.116:0/2614653359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7aec1012d0 0x7f7aec19abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.937+0000 7f7aea59c640 1 -- 172.21.15.116:0/2614653359 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aec19f630 con 0x7f7aec100920 2024-09-12T03:03:51.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.937+0000 7f7ae9d9b640 1 --2- 172.21.15.116:0/2614653359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7aec1012d0 0x7f7aec19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:51.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.938+0000 7f7aea59c640 1 --2- 172.21.15.116:0/2614653359 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7aec100920 0x7f7aec19a680 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7adc006010 tx=0x7f7adc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:51.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.938+0000 7f7adb7fe640 1 -- 172.21.15.116:0/2614653359 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc03f030 con 0x7f7aec100920 2024-09-12T03:03:51.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.938+0000 7f7adb7fe640 1 -- 172.21.15.116:0/2614653359 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7adc003ce0 con 0x7f7aec100920 2024-09-12T03:03:51.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.938+0000 7f7af0f85640 1 -- 172.21.15.116:0/2614653359 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aec19f8c0 con 0x7f7aec100920 2024-09-12T03:03:51.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.938+0000 7f7adb7fe640 1 -- 172.21.15.116:0/2614653359 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc035070 con 0x7f7aec100920 2024-09-12T03:03:51.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.938+0000 7f7af0f85640 1 -- 172.21.15.116:0/2614653359 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aec1a7200 con 0x7f7aec100920 2024-09-12T03:03:51.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.940+0000 7f7adb7fe640 1 -- 172.21.15.116:0/2614653359 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f7adc042030 con 0x7f7aec100920 2024-09-12T03:03:51.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.941+0000 7f7adb7fe640 1 --2- 172.21.15.116:0/2614653359 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7ad40a2c90 0x7f7ad40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:51.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.942+0000 7f7af335e740 1 Processor -- start 2024-09-12T03:03:51.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.942+0000 7f7ae9d9b640 1 --2- 172.21.15.116:0/2614653359 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7ad40a2c90 0x7f7ad40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:51.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.942+0000 7f7adb7fe640 1 -- 172.21.15.116:0/2614653359 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f7adc0eeec0 con 0x7f7aec100920 2024-09-12T03:03:51.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.942+0000 7f7af335e740 1 -- start start 2024-09-12T03:03:51.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.942+0000 7f7af335e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb8f7ab40 0x556fb8f79090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:51.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.942+0000 7f7af335e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fb8f795d0 0x556fb8f79a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:51.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.942+0000 7f7af335e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fb8f84370 0x556fb8f86760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:51.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.942+0000 7f7af335e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556fb8edf2f0 con 0x556fb8f795d0 2024-09-12T03:03:51.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.942+0000 7f7af335e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556fb8e86cf0 con 0x556fb8f84370 2024-09-12T03:03:51.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.942+0000 7f7af335e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556fb8edb2b0 con 0x556fb8f7ab40 2024-09-12T03:03:51.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.943+0000 7f7ae9d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fb8f795d0 0x556fb8f79a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:51.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.943+0000 7f7ae9d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fb8f795d0 0x556fb8f79a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39342/0 (socket says 172.21.15.116:39342) 2024-09-12T03:03:51.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.943+0000 7f7ae9d9b640 1 -- 172.21.15.116:0/4230210446 learned_addr learned my addr 172.21.15.116:0/4230210446 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:51.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.943+0000 7f7aea59c640 1 --2- 172.21.15.116:0/4230210446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb8f7ab40 0x556fb8f79090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:51.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.943+0000 7f7ae9d9b640 1 --2- 172.21.15.116:0/2614653359 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7ad40a2c90 0x7f7ad40a5150 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7aec10e8f0 tx=0x7f7ae000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:51.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.943+0000 7f7aead9d640 1 --2- 172.21.15.116:0/4230210446 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fb8f84370 0x556fb8f86760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:51.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.943+0000 7f7ae9d9b640 1 -- 172.21.15.116:0/4230210446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb8f7ab40 msgr2=0x556fb8f79090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:51.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.943+0000 7f7ae9d9b640 1 --2- 172.21.15.116:0/4230210446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb8f7ab40 0x556fb8f79090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.943+0000 7f7ae9d9b640 1 -- 172.21.15.116:0/4230210446 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fb8f84370 msgr2=0x556fb8f86760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:51.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.943+0000 7f7ae9d9b640 1 --2- 172.21.15.116:0/4230210446 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fb8f84370 0x556fb8f86760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.943+0000 7f7ae9d9b640 1 -- 172.21.15.116:0/4230210446 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fb8e6b3a0 con 0x556fb8f795d0 2024-09-12T03:03:51.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.943+0000 7f7aea59c640 1 --2- 172.21.15.116:0/4230210446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb8f7ab40 0x556fb8f79090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:03:51.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.943+0000 7f7ae9d9b640 1 --2- 172.21.15.116:0/4230210446 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fb8f795d0 0x556fb8f79a10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7ae000b620 tx=0x7f7ae0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f586aef4e5e6aa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:51.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.944+0000 7f7ad8ff9640 1 -- 172.21.15.116:0/4230210446 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae001c7b0 con 0x556fb8f795d0 2024-09-12T03:03:51.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.944+0000 7f7ad8ff9640 1 -- 172.21.15.116:0/4230210446 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ae001ce30 con 0x556fb8f795d0 2024-09-12T03:03:51.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.944+0000 7f7ad8ff9640 1 -- 172.21.15.116:0/4230210446 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae001e680 con 0x556fb8f795d0 2024-09-12T03:03:51.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.945+0000 7f7af335e740 1 -- 172.21.15.116:0/4230210446 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fb8f795d0 msgr2=0x556fb8f79a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:51.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.945+0000 7f7af335e740 1 --2- 172.21.15.116:0/4230210446 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fb8f795d0 0x556fb8f79a10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7ae000b620 tx=0x7f7ae0013000 comp rx=0 tx=0).stop 2024-09-12T03:03:51.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.945+0000 7f7af335e740 1 -- 172.21.15.116:0/4230210446 shutdown_connections 2024-09-12T03:03:51.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.945+0000 7f7af335e740 1 --2- 172.21.15.116:0/4230210446 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fb8f84370 0x556fb8f86760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.945+0000 7f7af335e740 1 --2- 172.21.15.116:0/4230210446 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fb8f795d0 0x556fb8f79a10 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.945+0000 7f7af335e740 1 --2- 172.21.15.116:0/4230210446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb8f7ab40 0x556fb8f79090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.945+0000 7f7af335e740 1 -- 172.21.15.116:0/4230210446 >> 172.21.15.116:0/4230210446 conn(0x556fb8f42310 msgr2=0x556fb8f75c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:51.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.945+0000 7f7af335e740 1 -- 172.21.15.116:0/4230210446 shutdown_connections 2024-09-12T03:03:51.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.945+0000 7f7af335e740 1 -- 172.21.15.116:0/4230210446 wait complete. 2024-09-12T03:03:51.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.945+0000 7f7af335e740 1 -- start start 2024-09-12T03:03:51.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.945+0000 7f7af335e740 1 Processor -- start 2024-09-12T03:03:51.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.946+0000 7f7af335e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fb8f795d0 0x556fb8f83e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:51.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.946+0000 7f7af335e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fb8f7ab40 0x556fb8f810d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:51.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.946+0000 7f7af335e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb8f84370 0x556fb8f81860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:51.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.946+0000 7f7af335e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556fb8f87070 con 0x556fb8f7ab40 2024-09-12T03:03:51.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.946+0000 7f7af335e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556fb8e9a060 con 0x556fb8f795d0 2024-09-12T03:03:51.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.946+0000 7f7af335e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556fb8f871f0 con 0x556fb8f84370 2024-09-12T03:03:51.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.946+0000 7f7ae9d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fb8f7ab40 0x556fb8f810d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:51.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.946+0000 7f7af335e740 10 client.0 _ll_register_callbacks cb 0x556fb8bed8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:51.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.946+0000 7f7ae9d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fb8f7ab40 0x556fb8f810d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39354/0 (socket says 172.21.15.116:39354) 2024-09-12T03:03:51.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.946+0000 7f7ae9d9b640 1 -- 172.21.15.116:0/4156156841 learned_addr learned my addr 172.21.15.116:0/4156156841 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:51.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.946+0000 7f7aea59c640 1 --2- 172.21.15.116:0/4156156841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fb8f795d0 0x556fb8f83e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:51.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.946+0000 7f7aead9d640 1 --2- 172.21.15.116:0/4156156841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb8f84370 0x556fb8f81860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:51.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.946+0000 7f7aea59c640 1 -- 172.21.15.116:0/4156156841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb8f84370 msgr2=0x556fb8f81860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:51.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.946+0000 7f7aea59c640 1 --2- 172.21.15.116:0/4156156841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb8f84370 0x556fb8f81860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.946+0000 7f7aea59c640 1 -- 172.21.15.116:0/4156156841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fb8f7ab40 msgr2=0x556fb8f810d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:51.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.946+0000 7f7aea59c640 1 --2- 172.21.15.116:0/4156156841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fb8f7ab40 0x556fb8f810d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.946+0000 7f7aea59c640 1 -- 172.21.15.116:0/4156156841 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fb8c503c0 con 0x556fb8f795d0 2024-09-12T03:03:51.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.947+0000 7f7aea59c640 1 --2- 172.21.15.116:0/4156156841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fb8f795d0 0x556fb8f83e20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0ec8a0 tx=0x7f7adc0bb0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa2dd44424e0e368 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:51.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.947+0000 7f7ad1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-09-12T03:03:51.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.947+0000 7f7ad1ffb640 1 -- 172.21.15.116:0/4156156841 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc049040 con 0x556fb8f795d0 2024-09-12T03:03:51.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.947+0000 7f7ad1ffb640 1 -- 172.21.15.116:0/4156156841 <== mon.2 v2:172.21.15.177:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f7adc0b7dd0 con 0x556fb8f795d0 2024-09-12T03:03:51.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.947+0000 7f7af335e740 1 -- 172.21.15.116:0/4156156841 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ae001ce30 con 0x556fb8f795d0 2024-09-12T03:03:51.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.947+0000 7f7ad1ffb640 1 -- 172.21.15.116:0/4156156841 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7adc03b580 con 0x556fb8f795d0 2024-09-12T03:03:51.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.947+0000 7f7af335e740 10 client.24923 fetch_fsmap learned FSMap version 25 2024-09-12T03:03:51.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.947+0000 7f7af335e740 1 -- 172.21.15.116:0/4156156841 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556fb8f92be0 con 0x556fb8f795d0 2024-09-12T03:03:51.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.947+0000 7f7ad1ffb640 1 -- 172.21.15.116:0/4156156841 <== mon.2 v2:172.21.15.177:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ae001ce30 con 0x556fb8f795d0 2024-09-12T03:03:51.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.948+0000 7f7ad1ffb640 1 -- 172.21.15.116:0/4156156841 <== mon.2 v2:172.21.15.177:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7adc0bb9c0 con 0x556fb8f795d0 2024-09-12T03:03:51.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.948+0000 7f7af335e740 10 client.24923 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:03:51.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.948+0000 7f7af335e740 10 client.24923 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:03:51.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.948+0000 7f7af335e740 20 client.24923 populate_metadata read hostname 'smithi116' 2024-09-12T03:03:51.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.948+0000 7f7af335e740 1 --2- 172.21.15.116:0/4156156841 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x556fb8f94220 0x556fb8f96680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:51.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.948+0000 7f7af335e740 4 client.24923 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:03:51.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.948+0000 7f7af335e740 1 -- 172.21.15.116:0/4156156841 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fb8eec860 con 0x556fb8f94220 2024-09-12T03:03:51.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.948+0000 7f7ae9d9b640 1 --2- 172.21.15.116:0/4156156841 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x556fb8f94220 0x556fb8f96680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:51.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.949+0000 7f7ae9d9b640 1 --2- 172.21.15.116:0/4156156841 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x556fb8f94220 0x556fb8f96680 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=361ce8aeef6d8a64 server_cookie=da2ed3f788045c55 in_seq=0 out_seq=0 2024-09-12T03:03:51.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.949+0000 7f7ad1ffb640 10 client.24923 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:03:51.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.949+0000 7f7ad1ffb640 1 -- 172.21.15.116:0/4156156841 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556fb8eec860 con 0x556fb8f94220 2024-09-12T03:03:51.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.949+0000 7f7ad1ffb640 10 client.24923 handle_command_reply: tid=0 2024-09-12T03:03:51.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.949+0000 7f7af335e740 1 client.24923 shutdown 2024-09-12T03:03:51.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.950+0000 7f7af335e740 1 -- 172.21.15.116:0/4156156841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fb8f795d0 msgr2=0x556fb8f83e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:51.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.950+0000 7f7af335e740 1 --2- 172.21.15.116:0/4156156841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fb8f795d0 0x556fb8f83e20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0ec8a0 tx=0x7f7adc0bb0c0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.950+0000 7f7af335e740 1 -- 172.21.15.116:0/4156156841 shutdown_connections 2024-09-12T03:03:51.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.950+0000 7f7af335e740 1 --2- 172.21.15.116:0/4156156841 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x556fb8f94220 0x556fb8f96680 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.950+0000 7f7af335e740 1 --2- 172.21.15.116:0/4156156841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb8f84370 0x556fb8f81860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.950+0000 7f7af335e740 1 --2- 172.21.15.116:0/4156156841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fb8f7ab40 0x556fb8f810d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.950+0000 7f7af335e740 1 --2- 172.21.15.116:0/4156156841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fb8f795d0 0x556fb8f83e20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.950+0000 7f7af335e740 1 -- 172.21.15.116:0/4156156841 >> 172.21.15.116:0/4156156841 conn(0x556fb8f42310 msgr2=0x556fb8f7b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:51.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.950+0000 7f7ad1ffb640 0 client.24923 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:03:51.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.950+0000 7f7af335e740 1 -- 172.21.15.116:0/4156156841 shutdown_connections 2024-09-12T03:03:51.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.950+0000 7f7af335e740 1 -- 172.21.15.116:0/4156156841 wait complete. 2024-09-12T03:03:51.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.950+0000 7f7af335e740 20 client.24923 trim_cache size 0 max 16384 2024-09-12T03:03:51.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.974+0000 7f7af335e740 1 Processor -- start 2024-09-12T03:03:51.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.974+0000 7f7af335e740 1 -- start start 2024-09-12T03:03:51.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.974+0000 7f7af335e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fb8f795d0 0x556fb8f80990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:51.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.974+0000 7f7af335e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb8f7ab40 0x556fb8f74c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:51.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.974+0000 7f7af335e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fb8f84370 0x556fb8f75150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:51.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.974+0000 7f7af335e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec1b1fe0 con 0x556fb8f795d0 2024-09-12T03:03:51.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.974+0000 7f7af335e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec1af8b0 con 0x556fb8f84370 2024-09-12T03:03:51.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.974+0000 7f7af335e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec1ad700 con 0x556fb8f7ab40 2024-09-12T03:03:51.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.975+0000 7f7ae9d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb8f7ab40 0x556fb8f74c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:51.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.975+0000 7f7ae9d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb8f7ab40 0x556fb8f74c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.116:33052/0 (socket says 172.21.15.116:33052) 2024-09-12T03:03:51.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.975+0000 7f7ae9d9b640 1 -- 172.21.15.116:0/1342688265 learned_addr learned my addr 172.21.15.116:0/1342688265 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:51.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.975+0000 7f7aea59c640 1 --2- 172.21.15.116:0/1342688265 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fb8f795d0 0x556fb8f80990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:51.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.975+0000 7f7aead9d640 1 --2- 172.21.15.116:0/1342688265 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fb8f84370 0x556fb8f75150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:51.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.975+0000 7f7ae9d9b640 1 -- 172.21.15.116:0/1342688265 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fb8f84370 msgr2=0x556fb8f75150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:51.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.975+0000 7f7ae9d9b640 1 --2- 172.21.15.116:0/1342688265 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fb8f84370 0x556fb8f75150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.975+0000 7f7ae9d9b640 1 -- 172.21.15.116:0/1342688265 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fb8f795d0 msgr2=0x556fb8f80990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:51.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.975+0000 7f7ae9d9b640 1 --2- 172.21.15.116:0/1342688265 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fb8f795d0 0x556fb8f80990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.975+0000 7f7ae9d9b640 1 -- 172.21.15.116:0/1342688265 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fb8f80080 con 0x556fb8f7ab40 2024-09-12T03:03:51.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.975+0000 7f7aea59c640 1 --2- 172.21.15.116:0/1342688265 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fb8f795d0 0x556fb8f80990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:03:51.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.975+0000 7f7ae9d9b640 1 --2- 172.21.15.116:0/1342688265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb8f7ab40 0x556fb8f74c10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0027c40 tx=0x7f7ae0015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24b4e23e2c0fc8df server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:51.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.975+0000 7f7ad3fff640 1 -- 172.21.15.116:0/1342688265 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0027920 con 0x556fb8f7ab40 2024-09-12T03:03:51.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.975+0000 7f7ad3fff640 1 -- 172.21.15.116:0/1342688265 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ae0005520 con 0x556fb8f7ab40 2024-09-12T03:03:51.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.975+0000 7f7ad3fff640 1 -- 172.21.15.116:0/1342688265 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae001c5b0 con 0x556fb8f7ab40 2024-09-12T03:03:51.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.976+0000 7f7af335e740 1 -- 172.21.15.116:0/1342688265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb8f7ab40 msgr2=0x556fb8f74c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:51.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.976+0000 7f7af335e740 1 --2- 172.21.15.116:0/1342688265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb8f7ab40 0x556fb8f74c10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0027c40 tx=0x7f7ae0015000 comp rx=0 tx=0).stop 2024-09-12T03:03:51.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.976+0000 7f7af335e740 1 -- 172.21.15.116:0/1342688265 shutdown_connections 2024-09-12T03:03:51.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.976+0000 7f7af335e740 1 --2- 172.21.15.116:0/1342688265 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fb8f84370 0x556fb8f75150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.976+0000 7f7af335e740 1 --2- 172.21.15.116:0/1342688265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb8f7ab40 0x556fb8f74c10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.976+0000 7f7af335e740 1 --2- 172.21.15.116:0/1342688265 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fb8f795d0 0x556fb8f80990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.976+0000 7f7af335e740 1 -- 172.21.15.116:0/1342688265 >> 172.21.15.116:0/1342688265 conn(0x556fb8f42310 msgr2=0x556fb8f74500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:51.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.977+0000 7f7af335e740 1 -- 172.21.15.116:0/1342688265 shutdown_connections 2024-09-12T03:03:51.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.977+0000 7f7af335e740 1 -- 172.21.15.116:0/1342688265 wait complete. 2024-09-12T03:03:51.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.977+0000 7f7af335e740 1 -- start start 2024-09-12T03:03:51.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.977+0000 7f7af335e740 1 Processor -- start 2024-09-12T03:03:51.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.977+0000 7f7af335e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fb8f795d0 0x556fb8f7e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:51.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.977+0000 7f7af335e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb8f7ab40 0x556fb8f7e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:51.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.978+0000 7f7af335e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fb8f84370 0x556fb8f7ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:51.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.978+0000 7f7aea59c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fb8f795d0 0x556fb8f7e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:51.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.978+0000 7f7aea59c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fb8f795d0 0x556fb8f7e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.116:44844/0 (socket says 172.21.15.116:44844) 2024-09-12T03:03:51.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.978+0000 7f7aea59c640 1 -- 172.21.15.116:0/3755380512 learned_addr learned my addr 172.21.15.116:0/3755380512 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:51.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.978+0000 7f7ae9d9b640 1 --2- 172.21.15.116:0/3755380512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb8f7ab40 0x556fb8f7e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:51.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.978+0000 7f7af335e740 1 -- 172.21.15.116:0/3755380512 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec1add70 con 0x556fb8f84370 2024-09-12T03:03:51.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.978+0000 7f7af335e740 1 -- 172.21.15.116:0/3755380512 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec1af1e0 con 0x556fb8f795d0 2024-09-12T03:03:51.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.978+0000 7f7af335e740 1 -- 172.21.15.116:0/3755380512 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec1ae440 con 0x556fb8f7ab40 2024-09-12T03:03:51.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.978+0000 7f7aead9d640 1 --2- 172.21.15.116:0/3755380512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fb8f84370 0x556fb8f7ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:51.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.978+0000 7f7af335e740 10 client.0 _ll_register_callbacks cb 0x7f7adc003e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:51.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.978+0000 7f7ae9d9b640 1 -- 172.21.15.116:0/3755380512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fb8f795d0 msgr2=0x556fb8f7e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:51.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.978+0000 7f7ae9d9b640 1 --2- 172.21.15.116:0/3755380512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fb8f795d0 0x556fb8f7e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.978+0000 7f7ae9d9b640 1 -- 172.21.15.116:0/3755380512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fb8f84370 msgr2=0x556fb8f7ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:51.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.978+0000 7f7ae9d9b640 1 --2- 172.21.15.116:0/3755380512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fb8f84370 0x556fb8f7ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.978+0000 7f7ae9d9b640 1 -- 172.21.15.116:0/3755380512 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fb9006930 con 0x556fb8f7ab40 2024-09-12T03:03:51.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.978+0000 7f7aead9d640 1 --2- 172.21.15.116:0/3755380512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fb8f84370 0x556fb8f7ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:51.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.978+0000 7f7ae9d9b640 1 --2- 172.21.15.116:0/3755380512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb8f7ab40 0x556fb8f7e7e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x556fb8f96c80 tx=0x7f7ae0005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f7defeaac2e13e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:51.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.978+0000 7f7ad1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T03:03:51.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.979+0000 7f7ad1ffb640 1 -- 172.21.15.116:0/3755380512 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0027720 con 0x556fb8f7ab40 2024-09-12T03:03:51.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.979+0000 7f7ad1ffb640 1 -- 172.21.15.116:0/3755380512 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f7ae00278c0 con 0x556fb8f7ab40 2024-09-12T03:03:51.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.979+0000 7f7af335e740 1 -- 172.21.15.116:0/3755380512 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ae0005520 con 0x556fb8f7ab40 2024-09-12T03:03:51.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.979+0000 7f7ad1ffb640 1 -- 172.21.15.116:0/3755380512 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7ae0004c40 con 0x556fb8f7ab40 2024-09-12T03:03:51.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.979+0000 7f7af335e740 10 client.24958 fetch_fsmap learned FSMap version 25 2024-09-12T03:03:51.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.979+0000 7f7af335e740 1 -- 172.21.15.116:0/3755380512 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556fb9007b90 con 0x556fb8f7ab40 2024-09-12T03:03:51.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.979+0000 7f7ad1ffb640 1 -- 172.21.15.116:0/3755380512 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ae00052c0 con 0x556fb8f7ab40 2024-09-12T03:03:51.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.980+0000 7f7ad1ffb640 1 -- 172.21.15.116:0/3755380512 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7ae001cbc0 con 0x556fb8f7ab40 2024-09-12T03:03:51.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.980+0000 7f7af335e740 10 client.24958 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:03:51.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.980+0000 7f7af335e740 10 client.24958 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:03:51.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.980+0000 7f7af335e740 20 client.24958 populate_metadata read hostname 'smithi116' 2024-09-12T03:03:51.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.980+0000 7f7af335e740 1 --2- 172.21.15.116:0/3755380512 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x556fb9007fb0 0x556fb900a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:51.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.980+0000 7f7af335e740 4 client.24958 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:03:51.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.980+0000 7f7af335e740 1 -- 172.21.15.116:0/3755380512 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f7ae0005520 con 0x556fb9007fb0 2024-09-12T03:03:51.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.980+0000 7f7aea59c640 1 --2- 172.21.15.116:0/3755380512 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x556fb9007fb0 0x556fb900a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:51.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.981+0000 7f7ad1ffb640 10 client.24958 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:03:51.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.981+0000 7f7aea59c640 1 --2- 172.21.15.116:0/3755380512 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x556fb9007fb0 0x556fb900a3d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=39864bb4598ffdfb server_cookie=867b0b96d681b27d in_seq=0 out_seq=0 2024-09-12T03:03:51.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.981+0000 7f7ad1ffb640 1 -- 172.21.15.116:0/3755380512 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f7ae0005520 con 0x556fb9007fb0 2024-09-12T03:03:51.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.981+0000 7f7ad1ffb640 10 client.24958 handle_command_reply: tid=0 2024-09-12T03:03:51.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.981+0000 7f7af335e740 1 client.24958 shutdown 2024-09-12T03:03:51.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.981+0000 7f7af335e740 1 -- 172.21.15.116:0/3755380512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb8f7ab40 msgr2=0x556fb8f7e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:51.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.981+0000 7f7af335e740 1 --2- 172.21.15.116:0/3755380512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb8f7ab40 0x556fb8f7e7e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x556fb8f96c80 tx=0x7f7ae0005850 comp rx=0 tx=0).stop 2024-09-12T03:03:51.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.982+0000 7f7af335e740 1 -- 172.21.15.116:0/3755380512 shutdown_connections 2024-09-12T03:03:51.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.982+0000 7f7af335e740 1 --2- 172.21.15.116:0/3755380512 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x556fb9007fb0 0x556fb900a3d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.982+0000 7f7af335e740 1 --2- 172.21.15.116:0/3755380512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fb8f84370 0x556fb8f7ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.982+0000 7f7ad1ffb640 0 client.24958 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:03:51.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.982+0000 7f7af335e740 1 --2- 172.21.15.116:0/3755380512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fb8f7ab40 0x556fb8f7e7e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.982+0000 7f7af335e740 1 --2- 172.21.15.116:0/3755380512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fb8f795d0 0x556fb8f7e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.982+0000 7f7af335e740 1 -- 172.21.15.116:0/3755380512 >> 172.21.15.116:0/3755380512 conn(0x556fb8f42310 msgr2=0x556fb8f7b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:51.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.982+0000 7f7af335e740 1 -- 172.21.15.116:0/3755380512 shutdown_connections 2024-09-12T03:03:51.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.982+0000 7f7af335e740 1 -- 172.21.15.116:0/3755380512 wait complete. 2024-09-12T03:03:51.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.982+0000 7f7af335e740 20 client.24958 trim_cache size 0 max 16384 2024-09-12T03:03:51.987 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:03:51.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.983+0000 7f7af0f85640 1 -- 172.21.15.116:0/2614653359 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7ad40a2c90 msgr2=0x7f7ad40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:51.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.983+0000 7f7af0f85640 1 --2- 172.21.15.116:0/2614653359 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7ad40a2c90 0x7f7ad40a5150 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7aec10e8f0 tx=0x7f7ae000b390 comp rx=0 tx=0).stop 2024-09-12T03:03:51.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.983+0000 7f7af0f85640 1 -- 172.21.15.116:0/2614653359 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7aec100920 msgr2=0x7f7aec19a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:51.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.983+0000 7f7af0f85640 1 --2- 172.21.15.116:0/2614653359 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7aec100920 0x7f7aec19a680 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7adc006010 tx=0x7f7adc0042f0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.983+0000 7f7af0f85640 1 -- 172.21.15.116:0/2614653359 shutdown_connections 2024-09-12T03:03:51.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.983+0000 7f7af0f85640 1 --2- 172.21.15.116:0/2614653359 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7ad40a2c90 0x7f7ad40a5150 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.983+0000 7f7af0f85640 1 --2- 172.21.15.116:0/2614653359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7aec10e4d0 0x7f7aec19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.983+0000 7f7af0f85640 1 --2- 172.21.15.116:0/2614653359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7aec1012d0 0x7f7aec19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.983+0000 7f7af0f85640 1 --2- 172.21.15.116:0/2614653359 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7aec100920 0x7f7aec19a680 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.983+0000 7f7af0f85640 1 -- 172.21.15.116:0/2614653359 >> 172.21.15.116:0/2614653359 conn(0x7f7aec0fc790 msgr2=0x7f7aec0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:51.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.984+0000 7f7af0f85640 1 -- 172.21.15.116:0/2614653359 shutdown_connections 2024-09-12T03:03:51.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:51.984+0000 7f7af0f85640 1 -- 172.21.15.116:0/2614653359 wait complete. 2024-09-12T03:03:52.250 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:52 smithi116 ceph-mon[31410]: pgmap v364: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T03:03:52.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:52 smithi183 ceph-mon[35535]: pgmap v364: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T03:03:52.724 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:03:52.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:52 smithi177 ceph-mon[35305]: pgmap v364: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T03:03:53.012 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:53.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.555+0000 7f020af8f640 1 -- 172.21.15.116:0/2412353075 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0204100e20 msgr2=0x7f0204101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:53.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.555+0000 7f020af8f640 1 --2- 172.21.15.116:0/2412353075 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0204100e20 0x7f0204101200 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f01f8009900 tx=0x7f01f802a480 comp rx=0 tx=0).stop 2024-09-12T03:03:53.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.555+0000 7f020af8f640 1 -- 172.21.15.116:0/2412353075 shutdown_connections 2024-09-12T03:03:53.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.555+0000 7f020af8f640 1 --2- 172.21.15.116:0/2412353075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0204102ef0 0x7f020410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.555+0000 7f020af8f640 1 --2- 172.21.15.116:0/2412353075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0204102570 0x7f02041029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.555+0000 7f020af8f640 1 --2- 172.21.15.116:0/2412353075 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0204100e20 0x7f0204101200 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.555+0000 7f020af8f640 1 -- 172.21.15.116:0/2412353075 >> 172.21.15.116:0/2412353075 conn(0x7f02040fc7d0 msgr2=0x7f02040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:53.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.555+0000 7f020af8f640 1 -- 172.21.15.116:0/2412353075 shutdown_connections 2024-09-12T03:03:53.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.555+0000 7f020af8f640 1 -- 172.21.15.116:0/2412353075 wait complete. 2024-09-12T03:03:53.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.556+0000 7f020af8f640 1 Processor -- start 2024-09-12T03:03:53.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.556+0000 7f020af8f640 1 -- start start 2024-09-12T03:03:53.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.556+0000 7f020af8f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0204100e20 0x7f02041987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:53.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.556+0000 7f020af8f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0204102570 0x7f0204198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:53.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.556+0000 7f020af8f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0204102ef0 0x7f020419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:53.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.556+0000 7f020af8f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0204110f50 con 0x7f0204100e20 2024-09-12T03:03:53.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.556+0000 7f020af8f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0204110dd0 con 0x7f0204102570 2024-09-12T03:03:53.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.556+0000 7f020af8f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f02041110d0 con 0x7f0204102ef0 2024-09-12T03:03:53.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.556+0000 7f0208d2b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0204100e20 0x7f02041987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:53.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.556+0000 7f0208d2b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0204100e20 0x7f02041987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39392/0 (socket says 172.21.15.116:39392) 2024-09-12T03:03:53.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.556+0000 7f0208d2b640 1 -- 172.21.15.116:0/2088091804 learned_addr learned my addr 172.21.15.116:0/2088091804 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:53.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.557+0000 7f0203fff640 1 --2- 172.21.15.116:0/2088091804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0204102570 0x7f0204198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:53.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.557+0000 7f020952c640 1 --2- 172.21.15.116:0/2088091804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0204102ef0 0x7f020419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:53.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.557+0000 7f0208d2b640 1 -- 172.21.15.116:0/2088091804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0204102ef0 msgr2=0x7f020419ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:53.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.557+0000 7f0208d2b640 1 --2- 172.21.15.116:0/2088091804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0204102ef0 0x7f020419ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.557+0000 7f0208d2b640 1 -- 172.21.15.116:0/2088091804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0204102570 msgr2=0x7f0204198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:53.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.557+0000 7f0208d2b640 1 --2- 172.21.15.116:0/2088091804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0204102570 0x7f0204198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.557+0000 7f0208d2b640 1 -- 172.21.15.116:0/2088091804 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f020419d550 con 0x7f0204100e20 2024-09-12T03:03:53.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.557+0000 7f0208d2b640 1 --2- 172.21.15.116:0/2088091804 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0204100e20 0x7f02041987e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f01f8006010 tx=0x7f01f8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:53.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.557+0000 7f0201ffb640 1 -- 172.21.15.116:0/2088091804 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f803f070 con 0x7f0204100e20 2024-09-12T03:03:53.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.558+0000 7f0201ffb640 1 -- 172.21.15.116:0/2088091804 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01f8031c30 con 0x7f0204100e20 2024-09-12T03:03:53.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.558+0000 7f020af8f640 1 -- 172.21.15.116:0/2088091804 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f020419d7e0 con 0x7f0204100e20 2024-09-12T03:03:53.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.558+0000 7f0201ffb640 1 -- 172.21.15.116:0/2088091804 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8035030 con 0x7f0204100e20 2024-09-12T03:03:53.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.558+0000 7f020af8f640 1 -- 172.21.15.116:0/2088091804 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02041116f0 con 0x7f0204100e20 2024-09-12T03:03:53.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.561+0000 7f020d368740 1 Processor -- start 2024-09-12T03:03:53.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.561+0000 7f020d368740 1 -- start start 2024-09-12T03:03:53.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.561+0000 7f0201ffb640 1 -- 172.21.15.116:0/2088091804 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f01f8042070 con 0x7f0204100e20 2024-09-12T03:03:53.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.561+0000 7f020d368740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55badfd1ab40 0x55badfd19090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:53.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.561+0000 7f020d368740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55badfd195d0 0x55badfd19a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:53.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.561+0000 7f020d368740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55badfd24370 0x55badfd26760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:53.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.561+0000 7f020d368740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55badfc7f2f0 con 0x55badfd1ab40 2024-09-12T03:03:53.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.561+0000 7f020d368740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55badfc26cf0 con 0x55badfd24370 2024-09-12T03:03:53.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.561+0000 7f020d368740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55badfc7b2b0 con 0x55badfd195d0 2024-09-12T03:03:53.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.561+0000 7f020952c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55badfd1ab40 0x55badfd19090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:53.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.561+0000 7f020952c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55badfd1ab40 0x55badfd19090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39404/0 (socket says 172.21.15.116:39404) 2024-09-12T03:03:53.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.561+0000 7f020952c640 1 -- 172.21.15.116:0/3639789491 learned_addr learned my addr 172.21.15.116:0/3639789491 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:53.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.562+0000 7f0208d2b640 1 --2- 172.21.15.116:0/3639789491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55badfd195d0 0x55badfd19a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:53.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.562+0000 7f020952c640 1 -- 172.21.15.116:0/3639789491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55badfd195d0 msgr2=0x55badfd19a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:53.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.562+0000 7f020952c640 1 --2- 172.21.15.116:0/3639789491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55badfd195d0 0x55badfd19a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.562+0000 7f020952c640 1 -- 172.21.15.116:0/3639789491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55badfd24370 msgr2=0x55badfd26760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:03:53.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.562+0000 7f020952c640 1 --2- 172.21.15.116:0/3639789491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55badfd24370 0x55badfd26760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.562+0000 7f020952c640 1 -- 172.21.15.116:0/3639789491 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55badfc0b3a0 con 0x55badfd1ab40 2024-09-12T03:03:53.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.562+0000 7f020952c640 1 --2- 172.21.15.116:0/3639789491 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55badfd1ab40 0x55badfd19090 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f01fc009f90 tx=0x7f01fc0096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3082599eb9b7159 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:53.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.562+0000 7f0201ffb640 1 --2- 172.21.15.116:0/2088091804 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f01f40a2c10 0x7f01f40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:53.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.563+0000 7f01eeffd640 1 -- 172.21.15.116:0/3639789491 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc021070 con 0x55badfd1ab40 2024-09-12T03:03:53.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.563+0000 7f01eeffd640 1 -- 172.21.15.116:0/3639789491 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01fc01c430 con 0x55badfd1ab40 2024-09-12T03:03:53.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.563+0000 7f01eeffd640 1 -- 172.21.15.116:0/3639789491 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc01b5d0 con 0x55badfd1ab40 2024-09-12T03:03:53.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.563+0000 7f0201ffb640 1 -- 172.21.15.116:0/2088091804 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f01f80ef960 con 0x7f0204100e20 2024-09-12T03:03:53.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.563+0000 7f0208d2b640 1 --2- 172.21.15.116:0/2088091804 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f01f40a2c10 0x7f01f40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:53.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.564+0000 7f020d368740 1 -- 172.21.15.116:0/3639789491 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55badfd1ab40 msgr2=0x55badfd19090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:53.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.564+0000 7f020d368740 1 --2- 172.21.15.116:0/3639789491 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55badfd1ab40 0x55badfd19090 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f01fc009f90 tx=0x7f01fc0096a0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.564+0000 7f0208d2b640 1 --2- 172.21.15.116:0/2088091804 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f01f40a2c10 0x7f01f40a50d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55badfd16ac0 tx=0x7f01f80f5510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:53.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.564+0000 7f020d368740 1 -- 172.21.15.116:0/3639789491 shutdown_connections 2024-09-12T03:03:53.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.564+0000 7f020d368740 1 --2- 172.21.15.116:0/3639789491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55badfd24370 0x55badfd26760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.564+0000 7f020d368740 1 --2- 172.21.15.116:0/3639789491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55badfd195d0 0x55badfd19a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.564+0000 7f020d368740 1 --2- 172.21.15.116:0/3639789491 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55badfd1ab40 0x55badfd19090 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.564+0000 7f020d368740 1 -- 172.21.15.116:0/3639789491 >> 172.21.15.116:0/3639789491 conn(0x55badfce2310 msgr2=0x55badfd15c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:53.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.564+0000 7f020d368740 1 -- 172.21.15.116:0/3639789491 shutdown_connections 2024-09-12T03:03:53.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.564+0000 7f020d368740 1 -- 172.21.15.116:0/3639789491 wait complete. 2024-09-12T03:03:53.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.565+0000 7f020d368740 1 -- start start 2024-09-12T03:03:53.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.565+0000 7f020d368740 1 Processor -- start 2024-09-12T03:03:53.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.565+0000 7f020d368740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55badfd195d0 0x55badfd23dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:53.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.565+0000 7f020d368740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55badfd1ab40 0x55badfd21080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:53.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.565+0000 7f020d368740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55badfd24370 0x55badfd21810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:53.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.565+0000 7f020d368740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55badfd27070 con 0x55badfd24370 2024-09-12T03:03:53.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.565+0000 7f020d368740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55badfc3a060 con 0x55badfd1ab40 2024-09-12T03:03:53.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.565+0000 7f020d368740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55badfd271f0 con 0x55badfd195d0 2024-09-12T03:03:53.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.565+0000 7f020952c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55badfd24370 0x55badfd21810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:53.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.565+0000 7f020952c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55badfd24370 0x55badfd21810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39406/0 (socket says 172.21.15.116:39406) 2024-09-12T03:03:53.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.565+0000 7f020952c640 1 -- 172.21.15.116:0/4250870531 learned_addr learned my addr 172.21.15.116:0/4250870531 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:53.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.565+0000 7f020952c640 1 --2- 172.21.15.116:0/4250870531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55badfd195d0 0x55badfd23dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:53.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.565+0000 7f0203fff640 1 --2- 172.21.15.116:0/4250870531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55badfd1ab40 0x55badfd21080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:53.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.566+0000 7f020952c640 1 -- 172.21.15.116:0/4250870531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55badfd195d0 msgr2=0x55badfd23dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:53.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.566+0000 7f020952c640 1 --2- 172.21.15.116:0/4250870531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55badfd195d0 0x55badfd23dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.566+0000 7f020952c640 1 -- 172.21.15.116:0/4250870531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55badfd1ab40 msgr2=0x55badfd21080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:53.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.566+0000 7f020952c640 1 --2- 172.21.15.116:0/4250870531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55badfd1ab40 0x55badfd21080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.566+0000 7f020952c640 1 -- 172.21.15.116:0/4250870531 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55badf9f03c0 con 0x55badfd24370 2024-09-12T03:03:53.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.566+0000 7f020d368740 10 client.0 _ll_register_callbacks cb 0x55badf98d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:53.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.566+0000 7f020d368740 1 -- 172.21.15.116:0/4250870531 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01fc01c430 con 0x55badfd24370 2024-09-12T03:03:53.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.566+0000 7f020952c640 1 --2- 172.21.15.116:0/4250870531 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55badfd24370 0x55badfd21810 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f01fc01f040 tx=0x7f01fc01ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca1772e66abab26 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:53.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.566+0000 7f01e3fff640 10 client.15465 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:03:53.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.566+0000 7f01e3fff640 1 -- 172.21.15.116:0/4250870531 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc021040 con 0x55badfd24370 2024-09-12T03:03:53.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.566+0000 7f01e3fff640 1 -- 172.21.15.116:0/4250870531 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f01fc003b70 con 0x55badfd24370 2024-09-12T03:03:53.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.566+0000 7f01e3fff640 1 -- 172.21.15.116:0/4250870531 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f01fc003d70 con 0x55badfd24370 2024-09-12T03:03:53.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.566+0000 7f020d368740 10 client.15465 fetch_fsmap learned FSMap version 25 2024-09-12T03:03:53.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.566+0000 7f020d368740 1 -- 172.21.15.116:0/4250870531 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55badfd32a40 con 0x55badfd24370 2024-09-12T03:03:53.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.567+0000 7f01e3fff640 1 -- 172.21.15.116:0/4250870531 <== mon.0 v2:172.21.15.116:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f01fc027050 con 0x55badfd24370 2024-09-12T03:03:53.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.567+0000 7f020d368740 10 client.15465 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:03:53.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.567+0000 7f020d368740 10 client.15465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:03:53.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.567+0000 7f020d368740 20 client.15465 populate_metadata read hostname 'smithi116' 2024-09-12T03:03:53.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.567+0000 7f020d368740 1 --2- 172.21.15.116:0/4250870531 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55badfd33df0 0x55badfd361e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:53.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.567+0000 7f020d368740 4 client.15465 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:03:53.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.567+0000 7f020d368740 1 -- 172.21.15.116:0/4250870531 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55badfc8c860 con 0x55badfd33df0 2024-09-12T03:03:53.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.567+0000 7f0203fff640 1 --2- 172.21.15.116:0/4250870531 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55badfd33df0 0x55badfd361e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:53.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.568+0000 7f01e3fff640 10 client.15465 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:53.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.568+0000 7f0203fff640 1 --2- 172.21.15.116:0/4250870531 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55badfd33df0 0x55badfd361e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d77a27a1cbabb53 server_cookie=62aa06e906b0e7cd in_seq=0 out_seq=0 2024-09-12T03:03:53.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.568+0000 7f01e3fff640 1 -- 172.21.15.116:0/4250870531 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55badfc8c860 con 0x55badfd33df0 2024-09-12T03:03:53.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.568+0000 7f01e3fff640 10 client.15465 handle_command_reply: tid=0 2024-09-12T03:03:53.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.568+0000 7f020d368740 1 client.15465 shutdown 2024-09-12T03:03:53.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.569+0000 7f020d368740 1 -- 172.21.15.116:0/4250870531 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55badfd24370 msgr2=0x55badfd21810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:53.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.569+0000 7f020d368740 1 --2- 172.21.15.116:0/4250870531 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55badfd24370 0x55badfd21810 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f01fc01f040 tx=0x7f01fc01ceb0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.569+0000 7f020d368740 1 -- 172.21.15.116:0/4250870531 shutdown_connections 2024-09-12T03:03:53.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.569+0000 7f020d368740 1 --2- 172.21.15.116:0/4250870531 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55badfd33df0 0x55badfd361e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.569+0000 7f01e3fff640 0 client.15465 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:53.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.569+0000 7f020d368740 1 --2- 172.21.15.116:0/4250870531 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55badfd24370 0x55badfd21810 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.569+0000 7f020d368740 1 --2- 172.21.15.116:0/4250870531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55badfd1ab40 0x55badfd21080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.569+0000 7f020d368740 1 --2- 172.21.15.116:0/4250870531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55badfd195d0 0x55badfd23dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.569+0000 7f020d368740 1 -- 172.21.15.116:0/4250870531 >> 172.21.15.116:0/4250870531 conn(0x55badfce2310 msgr2=0x55badfd1b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:53.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.569+0000 7f020d368740 1 -- 172.21.15.116:0/4250870531 shutdown_connections 2024-09-12T03:03:53.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.569+0000 7f020d368740 1 -- 172.21.15.116:0/4250870531 wait complete. 2024-09-12T03:03:53.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.569+0000 7f020d368740 20 client.15465 trim_cache size 0 max 16384 2024-09-12T03:03:53.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.582+0000 7f020d368740 1 Processor -- start 2024-09-12T03:03:53.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.582+0000 7f020d368740 1 -- start start 2024-09-12T03:03:53.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.582+0000 7f020d368740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55badfd195d0 0x55badfd10c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:53.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.582+0000 7f020d368740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55badfd1ab40 0x55badfd3b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:53.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.582+0000 7f020d368740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55badfd24370 0x55badfd3bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:53.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.582+0000 7f020d368740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f02041adb50 con 0x55badfd1ab40 2024-09-12T03:03:53.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.582+0000 7f020d368740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f02041ab420 con 0x55badfd24370 2024-09-12T03:03:53.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.582+0000 7f020d368740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f02041a9270 con 0x55badfd195d0 2024-09-12T03:03:53.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.582+0000 7f0203fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55badfd1ab40 0x55badfd3b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:53.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.582+0000 7f0203fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55badfd1ab40 0x55badfd3b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39420/0 (socket says 172.21.15.116:39420) 2024-09-12T03:03:53.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.582+0000 7f0203fff640 1 -- 172.21.15.116:0/1828683576 learned_addr learned my addr 172.21.15.116:0/1828683576 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:53.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.583+0000 7f020952c640 1 --2- 172.21.15.116:0/1828683576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55badfd195d0 0x55badfd10c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:53.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.583+0000 7f020952c640 1 --2- 172.21.15.116:0/1828683576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55badfd24370 0x55badfd3bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:53.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.583+0000 7f0203fff640 1 -- 172.21.15.116:0/1828683576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55badfd195d0 msgr2=0x55badfd10c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:53.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.583+0000 7f0203fff640 1 --2- 172.21.15.116:0/1828683576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55badfd195d0 0x55badfd10c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.583+0000 7f0203fff640 1 -- 172.21.15.116:0/1828683576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55badfd24370 msgr2=0x55badfd3bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:53.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.583+0000 7f0203fff640 1 --2- 172.21.15.116:0/1828683576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55badfd24370 0x55badfd3bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.583+0000 7f0203fff640 1 -- 172.21.15.116:0/1828683576 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55badfd1c120 con 0x55badfd1ab40 2024-09-12T03:03:53.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.583+0000 7f020952c640 1 --2- 172.21.15.116:0/1828683576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55badfd24370 0x55badfd3bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:03:53.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.583+0000 7f0203fff640 1 --2- 172.21.15.116:0/1828683576 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55badfd1ab40 0x55badfd3b780 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f01f0005d00 tx=0x7f01f000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b2da6ccb7a4903 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:53.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.583+0000 7f01ee7fc640 1 -- 172.21.15.116:0/1828683576 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f0004220 con 0x55badfd1ab40 2024-09-12T03:03:53.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.583+0000 7f01ee7fc640 1 -- 172.21.15.116:0/1828683576 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01f001e030 con 0x55badfd1ab40 2024-09-12T03:03:53.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.583+0000 7f01ee7fc640 1 -- 172.21.15.116:0/1828683576 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f0004d70 con 0x55badfd1ab40 2024-09-12T03:03:53.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.584+0000 7f020d368740 1 -- 172.21.15.116:0/1828683576 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55badfd1ab40 msgr2=0x55badfd3b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:53.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.584+0000 7f020d368740 1 --2- 172.21.15.116:0/1828683576 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55badfd1ab40 0x55badfd3b780 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f01f0005d00 tx=0x7f01f000b410 comp rx=0 tx=0).stop 2024-09-12T03:03:53.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.584+0000 7f020d368740 1 -- 172.21.15.116:0/1828683576 shutdown_connections 2024-09-12T03:03:53.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.584+0000 7f020d368740 1 --2- 172.21.15.116:0/1828683576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55badfd24370 0x55badfd3bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.584+0000 7f020d368740 1 --2- 172.21.15.116:0/1828683576 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55badfd1ab40 0x55badfd3b780 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.584+0000 7f020d368740 1 --2- 172.21.15.116:0/1828683576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55badfd195d0 0x55badfd10c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.584+0000 7f020d368740 1 -- 172.21.15.116:0/1828683576 >> 172.21.15.116:0/1828683576 conn(0x55badfce2310 msgr2=0x55badfd1c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:53.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.584+0000 7f020d368740 1 -- 172.21.15.116:0/1828683576 shutdown_connections 2024-09-12T03:03:53.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.584+0000 7f020d368740 1 -- 172.21.15.116:0/1828683576 wait complete. 2024-09-12T03:03:53.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.585+0000 7f020d368740 1 -- start start 2024-09-12T03:03:53.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.585+0000 7f020d368740 1 Processor -- start 2024-09-12T03:03:53.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.585+0000 7f020d368740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55badfd195d0 0x55badfd2ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:53.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.585+0000 7f020d368740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55badfd24370 0x55badfd28ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:53.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.585+0000 7f020d368740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55badfd33df0 0x55badfd29220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:53.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.585+0000 7f020d368740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f02041a98e0 con 0x55badfd195d0 2024-09-12T03:03:53.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.585+0000 7f020d368740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f02041aad50 con 0x55badfd33df0 2024-09-12T03:03:53.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.585+0000 7f020d368740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f02041a9fb0 con 0x55badfd24370 2024-09-12T03:03:53.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.585+0000 7f020d368740 10 client.0 _ll_register_callbacks cb 0x7f01fc01f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:53.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.586+0000 7f020952c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55badfd195d0 0x55badfd2ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:53.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.586+0000 7f020952c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55badfd195d0 0x55badfd2ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39432/0 (socket says 172.21.15.116:39432) 2024-09-12T03:03:53.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.586+0000 7f020952c640 1 -- 172.21.15.116:0/2940827602 learned_addr learned my addr 172.21.15.116:0/2940827602 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:53.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.586+0000 7f020952c640 1 --2- 172.21.15.116:0/2940827602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55badfd33df0 0x55badfd29220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:53.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.586+0000 7f020952c640 1 -- 172.21.15.116:0/2940827602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55badfd24370 msgr2=0x55badfd28ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:03:53.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.586+0000 7f020952c640 1 --2- 172.21.15.116:0/2940827602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55badfd24370 0x55badfd28ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.586+0000 7f020952c640 1 -- 172.21.15.116:0/2940827602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55badfd33df0 msgr2=0x55badfd29220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:53.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.586+0000 7f020952c640 1 --2- 172.21.15.116:0/2940827602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55badfd33df0 0x55badfd29220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.586+0000 7f020952c640 1 -- 172.21.15.116:0/2940827602 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55badfd29860 con 0x55badfd195d0 2024-09-12T03:03:53.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.586+0000 7f020952c640 1 --2- 172.21.15.116:0/2940827602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55badfd195d0 0x55badfd2ca40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f01fc013010 tx=0x7f01fc01cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe00708225b723e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:53.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.586+0000 7f01e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:03:53.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.586+0000 7f01e3fff640 1 -- 172.21.15.116:0/2940827602 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc021040 con 0x55badfd195d0 2024-09-12T03:03:53.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.586+0000 7f01e3fff640 1 -- 172.21.15.116:0/2940827602 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f01fc00bc40 con 0x55badfd195d0 2024-09-12T03:03:53.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.586+0000 7f020d368740 1 -- 172.21.15.116:0/2940827602 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01f001e030 con 0x55badfd195d0 2024-09-12T03:03:53.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.587+0000 7f01e3fff640 1 -- 172.21.15.116:0/2940827602 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f01fc00b630 con 0x55badfd195d0 2024-09-12T03:03:53.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.587+0000 7f020d368740 10 client.15471 fetch_fsmap learned FSMap version 25 2024-09-12T03:03:53.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.587+0000 7f020d368740 1 -- 172.21.15.116:0/2940827602 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55badfd9f570 con 0x55badfd195d0 2024-09-12T03:03:53.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.587+0000 7f01e3fff640 1 -- 172.21.15.116:0/2940827602 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01fc045030 con 0x55badfd195d0 2024-09-12T03:03:53.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.587+0000 7f01e3fff640 1 -- 172.21.15.116:0/2940827602 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f01fc027050 con 0x55badfd195d0 2024-09-12T03:03:53.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.587+0000 7f020d368740 10 client.15471 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:03:53.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.587+0000 7f020d368740 10 client.15471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:03:53.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.587+0000 7f020d368740 20 client.15471 populate_metadata read hostname 'smithi116' 2024-09-12T03:03:53.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.587+0000 7f020d368740 1 --2- 172.21.15.116:0/2940827602 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55badfd9fb20 0x55badfda1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:53.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.587+0000 7f020d368740 4 client.15471 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:03:53.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.587+0000 7f020d368740 1 -- 172.21.15.116:0/2940827602 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f01f001e030 con 0x55badfd9fb20 2024-09-12T03:03:53.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.588+0000 7f0203fff640 1 --2- 172.21.15.116:0/2940827602 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55badfd9fb20 0x55badfda1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:53.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.588+0000 7f01e3fff640 10 client.15471 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:53.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.588+0000 7f0203fff640 1 --2- 172.21.15.116:0/2940827602 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55badfd9fb20 0x55badfda1f40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1577d95808dd331c server_cookie=7aee024ad3fad105 in_seq=0 out_seq=0 2024-09-12T03:03:53.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.589+0000 7f01e3fff640 1 -- 172.21.15.116:0/2940827602 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f01f001e030 con 0x55badfd9fb20 2024-09-12T03:03:53.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.589+0000 7f01e3fff640 10 client.15471 handle_command_reply: tid=0 2024-09-12T03:03:53.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.589+0000 7f020d368740 1 client.15471 shutdown 2024-09-12T03:03:53.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.589+0000 7f020d368740 1 -- 172.21.15.116:0/2940827602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55badfd195d0 msgr2=0x55badfd2ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:53.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.589+0000 7f020d368740 1 --2- 172.21.15.116:0/2940827602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55badfd195d0 0x55badfd2ca40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f01fc013010 tx=0x7f01fc01cd10 comp rx=0 tx=0).stop 2024-09-12T03:03:53.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.589+0000 7f020d368740 1 -- 172.21.15.116:0/2940827602 shutdown_connections 2024-09-12T03:03:53.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.589+0000 7f020d368740 1 --2- 172.21.15.116:0/2940827602 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55badfd9fb20 0x55badfda1f40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.589+0000 7f01e3fff640 0 client.15471 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:53.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.589+0000 7f020d368740 1 --2- 172.21.15.116:0/2940827602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55badfd33df0 0x55badfd29220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.589+0000 7f020d368740 1 --2- 172.21.15.116:0/2940827602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55badfd24370 0x55badfd28ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.589+0000 7f020d368740 1 --2- 172.21.15.116:0/2940827602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55badfd195d0 0x55badfd2ca40 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.589+0000 7f020d368740 1 -- 172.21.15.116:0/2940827602 >> 172.21.15.116:0/2940827602 conn(0x55badfce2310 msgr2=0x55badfd228e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:53.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.589+0000 7f020d368740 1 -- 172.21.15.116:0/2940827602 shutdown_connections 2024-09-12T03:03:53.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.589+0000 7f020d368740 1 -- 172.21.15.116:0/2940827602 wait complete. 2024-09-12T03:03:53.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.589+0000 7f020d368740 20 client.15471 trim_cache size 0 max 16384 2024-09-12T03:03:53.594 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:03:53.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.590+0000 7f020af8f640 1 -- 172.21.15.116:0/2088091804 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f01f40a2c10 msgr2=0x7f01f40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:53.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.590+0000 7f020af8f640 1 --2- 172.21.15.116:0/2088091804 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f01f40a2c10 0x7f01f40a50d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55badfd16ac0 tx=0x7f01f80f5510 comp rx=0 tx=0).stop 2024-09-12T03:03:53.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.590+0000 7f020af8f640 1 -- 172.21.15.116:0/2088091804 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0204100e20 msgr2=0x7f02041987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:53.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.590+0000 7f020af8f640 1 --2- 172.21.15.116:0/2088091804 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0204100e20 0x7f02041987e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f01f8006010 tx=0x7f01f8003c10 comp rx=0 tx=0).stop 2024-09-12T03:03:53.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.590+0000 7f020af8f640 1 -- 172.21.15.116:0/2088091804 shutdown_connections 2024-09-12T03:03:53.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.590+0000 7f020af8f640 1 --2- 172.21.15.116:0/2088091804 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f01f40a2c10 0x7f01f40a50d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.590+0000 7f020af8f640 1 --2- 172.21.15.116:0/2088091804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0204102ef0 0x7f020419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.590+0000 7f020af8f640 1 --2- 172.21.15.116:0/2088091804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0204102570 0x7f0204198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.590+0000 7f020af8f640 1 --2- 172.21.15.116:0/2088091804 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0204100e20 0x7f02041987e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.590+0000 7f020af8f640 1 -- 172.21.15.116:0/2088091804 >> 172.21.15.116:0/2088091804 conn(0x7f02040fc7d0 msgr2=0x7f02041038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:53.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.590+0000 7f020af8f640 1 -- 172.21.15.116:0/2088091804 shutdown_connections 2024-09-12T03:03:53.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:53.590+0000 7f020af8f640 1 -- 172.21.15.116:0/2088091804 wait complete. 2024-09-12T03:03:54.381 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:54 smithi116 ceph-mon[31410]: pgmap v365: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-12T03:03:54.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:54 smithi183 ceph-mon[35535]: pgmap v365: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-12T03:03:54.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:54 smithi177 ceph-mon[35305]: pgmap v365: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-12T03:03:55.410 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:03:55.411 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:03:55.792 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:56.483 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:56 smithi116 ceph-mon[31410]: pgmap v366: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-12T03:03:56.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.482+0000 7f4f33366640 1 -- 172.21.15.116:0/930897500 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f2c102ef0 msgr2=0x7f4f2c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:56.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.483+0000 7f4f33366640 1 --2- 172.21.15.116:0/930897500 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f2c102ef0 0x7f4f2c10da40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4f24009ca0 tx=0x7f4f2402a950 comp rx=0 tx=0).stop 2024-09-12T03:03:56.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.483+0000 7f4f33366640 1 -- 172.21.15.116:0/930897500 shutdown_connections 2024-09-12T03:03:56.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.483+0000 7f4f33366640 1 --2- 172.21.15.116:0/930897500 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f2c102ef0 0x7f4f2c10da40 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.483+0000 7f4f33366640 1 --2- 172.21.15.116:0/930897500 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f2c102570 0x7f4f2c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.483+0000 7f4f33366640 1 --2- 172.21.15.116:0/930897500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f2c100e20 0x7f4f2c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.483+0000 7f4f33366640 1 -- 172.21.15.116:0/930897500 >> 172.21.15.116:0/930897500 conn(0x7f4f2c0fc7d0 msgr2=0x7f4f2c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:56.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.483+0000 7f4f33366640 1 -- 172.21.15.116:0/930897500 shutdown_connections 2024-09-12T03:03:56.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.483+0000 7f4f33366640 1 -- 172.21.15.116:0/930897500 wait complete. 2024-09-12T03:03:56.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.485+0000 7f4f33366640 1 Processor -- start 2024-09-12T03:03:56.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.485+0000 7f4f33366640 1 -- start start 2024-09-12T03:03:56.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.485+0000 7f4f33366640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f2c100e20 0x7f4f2c19ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:56.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.485+0000 7f4f33366640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f2c102570 0x7f4f2c19d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:56.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.485+0000 7f4f33366640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f2c102ef0 0x7f4f2c198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:56.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.485+0000 7f4f33366640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c110f80 con 0x7f4f2c102ef0 2024-09-12T03:03:56.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.486+0000 7f4f33366640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c110e00 con 0x7f4f2c100e20 2024-09-12T03:03:56.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.486+0000 7f4f33366640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c111100 con 0x7f4f2c102570 2024-09-12T03:03:56.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.486+0000 7f4f31903640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f2c102ef0 0x7f4f2c198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:56.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.486+0000 7f4f31903640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f2c102ef0 0x7f4f2c198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39454/0 (socket says 172.21.15.116:39454) 2024-09-12T03:03:56.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.486+0000 7f4f31903640 1 -- 172.21.15.116:0/1899275369 learned_addr learned my addr 172.21.15.116:0/1899275369 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:56.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.486+0000 7f4f31102640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f2c100e20 0x7f4f2c19ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:56.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.486+0000 7f4f30901640 1 --2- 172.21.15.116:0/1899275369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f2c102570 0x7f4f2c19d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:56.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.486+0000 7f4f31903640 1 -- 172.21.15.116:0/1899275369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f2c102570 msgr2=0x7f4f2c19d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:56.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.486+0000 7f4f31903640 1 --2- 172.21.15.116:0/1899275369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f2c102570 0x7f4f2c19d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.486+0000 7f4f31903640 1 -- 172.21.15.116:0/1899275369 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f2c100e20 msgr2=0x7f4f2c19ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:56.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.486+0000 7f4f31903640 1 --2- 172.21.15.116:0/1899275369 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f2c100e20 0x7f4f2c19ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.486+0000 7f4f31903640 1 -- 172.21.15.116:0/1899275369 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f2c198d50 con 0x7f4f2c102ef0 2024-09-12T03:03:56.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.486+0000 7f4f30901640 1 --2- 172.21.15.116:0/1899275369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f2c102570 0x7f4f2c19d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:03:56.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.486+0000 7f4f31903640 1 --2- 172.21.15.116:0/1899275369 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f2c102ef0 0x7f4f2c198620 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4f24004af0 tx=0x7f4f2402f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:56.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.487+0000 7f4f227fc640 1 -- 172.21.15.116:0/1899275369 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f24034040 con 0x7f4f2c102ef0 2024-09-12T03:03:56.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.487+0000 7f4f33366640 1 -- 172.21.15.116:0/1899275369 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f2c198fe0 con 0x7f4f2c102ef0 2024-09-12T03:03:56.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.487+0000 7f4f227fc640 1 -- 172.21.15.116:0/1899275369 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f24030920 con 0x7f4f2c102ef0 2024-09-12T03:03:56.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.487+0000 7f4f227fc640 1 -- 172.21.15.116:0/1899275369 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f24045480 con 0x7f4f2c102ef0 2024-09-12T03:03:56.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.487+0000 7f4f33366640 1 -- 172.21.15.116:0/1899275369 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f2c111700 con 0x7f4f2c102ef0 2024-09-12T03:03:56.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.490+0000 7f4f3573f740 1 Processor -- start 2024-09-12T03:03:56.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.490+0000 7f4f227fc640 1 -- 172.21.15.116:0/1899275369 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f4f24037020 con 0x7f4f2c102ef0 2024-09-12T03:03:56.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.490+0000 7f4f3573f740 1 -- start start 2024-09-12T03:03:56.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.491+0000 7f4f3573f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56531a41db80 0x56531a41df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:56.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.491+0000 7f4f3573f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56531a41e4a0 0x56531a428820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:56.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.491+0000 7f4f3573f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56531a428d60 0x56531a42b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:56.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.491+0000 7f4f3573f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56531a383280 con 0x56531a41e4a0 2024-09-12T03:03:56.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.491+0000 7f4f3573f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56531a32ab10 con 0x56531a428d60 2024-09-12T03:03:56.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.491+0000 7f4f3573f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56531a37f240 con 0x56531a41db80 2024-09-12T03:03:56.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.491+0000 7f4f31102640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56531a41e4a0 0x56531a428820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:56.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.491+0000 7f4f31102640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56531a41e4a0 0x56531a428820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39466/0 (socket says 172.21.15.116:39466) 2024-09-12T03:03:56.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.491+0000 7f4f31102640 1 -- 172.21.15.116:0/3125507233 learned_addr learned my addr 172.21.15.116:0/3125507233 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:56.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.491+0000 7f4f31102640 1 -- 172.21.15.116:0/3125507233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56531a41db80 msgr2=0x56531a41df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:56.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.491+0000 7f4f31102640 1 --2- 172.21.15.116:0/3125507233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56531a41db80 0x56531a41df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.491+0000 7f4f31102640 1 -- 172.21.15.116:0/3125507233 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56531a428d60 msgr2=0x56531a42b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:56.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.491+0000 7f4f31102640 1 --2- 172.21.15.116:0/3125507233 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56531a428d60 0x56531a42b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.491+0000 7f4f31102640 1 -- 172.21.15.116:0/3125507233 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56531a3371f0 con 0x56531a41e4a0 2024-09-12T03:03:56.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.492+0000 7f4f31102640 1 --2- 172.21.15.116:0/3125507233 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56531a41e4a0 0x56531a428820 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009cd0 tx=0x7f4f2800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9780895f1175b968 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:56.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.492+0000 7f4f177fe640 1 -- 172.21.15.116:0/3125507233 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f28018440 con 0x56531a41e4a0 2024-09-12T03:03:56.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.492+0000 7f4f177fe640 1 -- 172.21.15.116:0/3125507233 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f280178c0 con 0x56531a41e4a0 2024-09-12T03:03:56.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.492+0000 7f4f227fc640 1 --2- 172.21.15.116:0/1899275369 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4f180a2c10 0x7f4f180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:56.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.492+0000 7f4f177fe640 1 -- 172.21.15.116:0/3125507233 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f280169f0 con 0x56531a41e4a0 2024-09-12T03:03:56.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.493+0000 7f4f227fc640 1 -- 172.21.15.116:0/1899275369 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f4f240effd0 con 0x7f4f2c102ef0 2024-09-12T03:03:56.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.493+0000 7f4f30901640 1 --2- 172.21.15.116:0/1899275369 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4f180a2c10 0x7f4f180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:56.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.493+0000 7f4f3573f740 1 -- 172.21.15.116:0/3125507233 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56531a41e4a0 msgr2=0x56531a428820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:56.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.493+0000 7f4f3573f740 1 --2- 172.21.15.116:0/3125507233 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56531a41e4a0 0x56531a428820 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009cd0 tx=0x7f4f2800f730 comp rx=0 tx=0).stop 2024-09-12T03:03:56.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.493+0000 7f4f3573f740 1 -- 172.21.15.116:0/3125507233 shutdown_connections 2024-09-12T03:03:56.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.493+0000 7f4f3573f740 1 --2- 172.21.15.116:0/3125507233 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56531a428d60 0x56531a42b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.493+0000 7f4f3573f740 1 --2- 172.21.15.116:0/3125507233 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56531a41e4a0 0x56531a428820 secure :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009cd0 tx=0x7f4f2800f730 comp rx=0 tx=0).stop 2024-09-12T03:03:56.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.493+0000 7f4f3573f740 1 --2- 172.21.15.116:0/3125507233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56531a41db80 0x56531a41df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.493+0000 7f4f3573f740 1 -- 172.21.15.116:0/3125507233 >> 172.21.15.116:0/3125507233 conn(0x56531a41a110 msgr2=0x56531a41a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:56.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.493+0000 7f4f30901640 1 --2- 172.21.15.116:0/1899275369 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4f180a2c10 0x7f4f180a50d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x56531a41b490 tx=0x7f4f1c00b350 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:56.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.493+0000 7f4f3573f740 1 -- 172.21.15.116:0/3125507233 shutdown_connections 2024-09-12T03:03:56.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.493+0000 7f4f3573f740 1 -- 172.21.15.116:0/3125507233 wait complete. 2024-09-12T03:03:56.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.494+0000 7f4f3573f740 1 -- start start 2024-09-12T03:03:56.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.494+0000 7f4f3573f740 1 Processor -- start 2024-09-12T03:03:56.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.495+0000 7f4f3573f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56531a41db80 0x56531a425ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:56.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.495+0000 7f4f3573f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56531a428d60 0x56531a426220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:56.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.495+0000 7f4f3573f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56531a426760 0x56531a43d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:56.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.495+0000 7f4f3573f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56531a4341e0 con 0x56531a41db80 2024-09-12T03:03:56.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.495+0000 7f4f3573f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56531a33df10 con 0x56531a426760 2024-09-12T03:03:56.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.495+0000 7f4f3573f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56531a434360 con 0x56531a428d60 2024-09-12T03:03:56.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.495+0000 7f4f31102640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56531a41db80 0x56531a425ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:56.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.495+0000 7f4f3573f740 10 client.0 _ll_register_callbacks cb 0x56531a306a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:56.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.495+0000 7f4f31102640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56531a41db80 0x56531a425ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39478/0 (socket says 172.21.15.116:39478) 2024-09-12T03:03:56.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.495+0000 7f4f31102640 1 -- 172.21.15.116:0/2701144817 learned_addr learned my addr 172.21.15.116:0/2701144817 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:56.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.495+0000 7f4f30901640 1 --2- 172.21.15.116:0/2701144817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56531a428d60 0x56531a426220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:56.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.495+0000 7f4f31903640 1 --2- 172.21.15.116:0/2701144817 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56531a426760 0x56531a43d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:56.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.495+0000 7f4f31102640 1 -- 172.21.15.116:0/2701144817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56531a428d60 msgr2=0x56531a426220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:56.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.495+0000 7f4f31102640 1 --2- 172.21.15.116:0/2701144817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56531a428d60 0x56531a426220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.495+0000 7f4f31102640 1 -- 172.21.15.116:0/2701144817 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56531a426760 msgr2=0x56531a43d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:56.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.495+0000 7f4f31102640 1 --2- 172.21.15.116:0/2701144817 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56531a426760 0x56531a43d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.495+0000 7f4f31102640 1 -- 172.21.15.116:0/2701144817 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56531a0f46a0 con 0x56531a41db80 2024-09-12T03:03:56.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.496+0000 7f4f31102640 1 --2- 172.21.15.116:0/2701144817 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56531a41db80 0x56531a425ce0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800ffa0 tx=0x7f4f28016390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e470f887d98d60 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:56.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.496+0000 7f4f14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:03:56.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.496+0000 7f4f14ff9640 1 -- 172.21.15.116:0/2701144817 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f28025020 con 0x56531a41db80 2024-09-12T03:03:56.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.496+0000 7f4f14ff9640 1 -- 172.21.15.116:0/2701144817 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f4f280173e0 con 0x56531a41db80 2024-09-12T03:03:56.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.496+0000 7f4f3573f740 1 -- 172.21.15.116:0/2701144817 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f280178c0 con 0x56531a41db80 2024-09-12T03:03:56.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.497+0000 7f4f14ff9640 1 -- 172.21.15.116:0/2701144817 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4f28017600 con 0x56531a41db80 2024-09-12T03:03:56.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.497+0000 7f4f3573f740 10 client.15483 fetch_fsmap learned FSMap version 25 2024-09-12T03:03:56.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.497+0000 7f4f3573f740 1 -- 172.21.15.116:0/2701144817 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56531a440320 con 0x56531a41db80 2024-09-12T03:03:56.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.497+0000 7f4f14ff9640 1 -- 172.21.15.116:0/2701144817 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f280178c0 con 0x56531a41db80 2024-09-12T03:03:56.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.497+0000 7f4f14ff9640 1 -- 172.21.15.116:0/2701144817 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4f28016070 con 0x56531a41db80 2024-09-12T03:03:56.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.497+0000 7f4f3573f740 10 client.15483 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:03:56.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.497+0000 7f4f3573f740 10 client.15483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:03:56.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.497+0000 7f4f3573f740 20 client.15483 populate_metadata read hostname 'smithi116' 2024-09-12T03:03:56.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.497+0000 7f4f3573f740 1 --2- 172.21.15.116:0/2701144817 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56531a4416d0 0x56531a443ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:56.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.498+0000 7f4f3573f740 4 client.15483 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:03:56.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.498+0000 7f4f3573f740 1 -- 172.21.15.116:0/2701144817 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56531a3907f0 con 0x56531a4416d0 2024-09-12T03:03:56.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.498+0000 7f4f30901640 1 --2- 172.21.15.116:0/2701144817 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56531a4416d0 0x56531a443ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:56.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.498+0000 7f4f30901640 1 --2- 172.21.15.116:0/2701144817 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56531a4416d0 0x56531a443ac0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2efbc8bdec81f46a server_cookie=a064ecf6027112d3 in_seq=0 out_seq=0 2024-09-12T03:03:56.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.498+0000 7f4f14ff9640 10 client.15483 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:56.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.499+0000 7f4f14ff9640 1 -- 172.21.15.116:0/2701144817 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56531a3907f0 con 0x56531a4416d0 2024-09-12T03:03:56.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.499+0000 7f4f14ff9640 10 client.15483 handle_command_reply: tid=0 2024-09-12T03:03:56.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.499+0000 7f4f3573f740 1 client.15483 shutdown 2024-09-12T03:03:56.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.500+0000 7f4f3573f740 1 -- 172.21.15.116:0/2701144817 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56531a41db80 msgr2=0x56531a425ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:56.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.500+0000 7f4f3573f740 1 --2- 172.21.15.116:0/2701144817 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56531a41db80 0x56531a425ce0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800ffa0 tx=0x7f4f28016390 comp rx=0 tx=0).stop 2024-09-12T03:03:56.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.501+0000 7f4f3573f740 1 -- 172.21.15.116:0/2701144817 shutdown_connections 2024-09-12T03:03:56.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.501+0000 7f4f3573f740 1 --2- 172.21.15.116:0/2701144817 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56531a4416d0 0x56531a443ac0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.501+0000 7f4f14ff9640 0 client.15483 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:56.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.501+0000 7f4f3573f740 1 --2- 172.21.15.116:0/2701144817 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56531a426760 0x56531a43d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.501+0000 7f4f3573f740 1 --2- 172.21.15.116:0/2701144817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56531a428d60 0x56531a426220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.501+0000 7f4f3573f740 1 --2- 172.21.15.116:0/2701144817 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56531a41db80 0x56531a425ce0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.501+0000 7f4f3573f740 1 -- 172.21.15.116:0/2701144817 >> 172.21.15.116:0/2701144817 conn(0x56531a41a110 msgr2=0x56531a41fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:56.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.501+0000 7f4f3573f740 1 -- 172.21.15.116:0/2701144817 shutdown_connections 2024-09-12T03:03:56.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.501+0000 7f4f3573f740 1 -- 172.21.15.116:0/2701144817 wait complete. 2024-09-12T03:03:56.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.501+0000 7f4f3573f740 20 client.15483 trim_cache size 0 max 16384 2024-09-12T03:03:56.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.535+0000 7f4f3573f740 1 Processor -- start 2024-09-12T03:03:56.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.535+0000 7f4f3573f740 1 -- start start 2024-09-12T03:03:56.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.535+0000 7f4f3573f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56531a41db80 0x56531a43d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:56.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.535+0000 7f4f3573f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56531a426760 0x56531a449320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:56.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.535+0000 7f4f3573f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56531a428d60 0x56531a449860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:56.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.535+0000 7f4f3573f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c1ad9c0 con 0x56531a426760 2024-09-12T03:03:56.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.535+0000 7f4f3573f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c1ab290 con 0x56531a41db80 2024-09-12T03:03:56.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.535+0000 7f4f3573f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c1a90e0 con 0x56531a428d60 2024-09-12T03:03:56.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.535+0000 7f4f30901640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56531a426760 0x56531a449320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:56.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.535+0000 7f4f30901640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56531a426760 0x56531a449320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39480/0 (socket says 172.21.15.116:39480) 2024-09-12T03:03:56.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.535+0000 7f4f30901640 1 -- 172.21.15.116:0/3029346171 learned_addr learned my addr 172.21.15.116:0/3029346171 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:56.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.535+0000 7f4f31903640 1 --2- 172.21.15.116:0/3029346171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56531a428d60 0x56531a449860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:56.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.535+0000 7f4f31102640 1 --2- 172.21.15.116:0/3029346171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56531a41db80 0x56531a43d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:56.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.536+0000 7f4f30901640 1 -- 172.21.15.116:0/3029346171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56531a428d60 msgr2=0x56531a449860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:56.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.536+0000 7f4f30901640 1 --2- 172.21.15.116:0/3029346171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56531a428d60 0x56531a449860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.536+0000 7f4f30901640 1 -- 172.21.15.116:0/3029346171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56531a41db80 msgr2=0x56531a43d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:56.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.536+0000 7f4f30901640 1 --2- 172.21.15.116:0/3029346171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56531a41db80 0x56531a43d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.536+0000 7f4f30901640 1 -- 172.21.15.116:0/3029346171 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56531a443e30 con 0x56531a426760 2024-09-12T03:03:56.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.536+0000 7f4f30901640 1 --2- 172.21.15.116:0/3029346171 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56531a426760 0x56531a449320 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c10dc20 tx=0x7f4f1c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806b133969cc08be server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:56.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.536+0000 7f4f16ffd640 1 -- 172.21.15.116:0/3029346171 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c0285b0 con 0x56531a426760 2024-09-12T03:03:56.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.536+0000 7f4f16ffd640 1 -- 172.21.15.116:0/3029346171 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f1c028c30 con 0x56531a426760 2024-09-12T03:03:56.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.536+0000 7f4f16ffd640 1 -- 172.21.15.116:0/3029346171 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c011e30 con 0x56531a426760 2024-09-12T03:03:56.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.537+0000 7f4f3573f740 1 -- 172.21.15.116:0/3029346171 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56531a426760 msgr2=0x56531a449320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:56.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.537+0000 7f4f3573f740 1 --2- 172.21.15.116:0/3029346171 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56531a426760 0x56531a449320 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c10dc20 tx=0x7f4f1c015000 comp rx=0 tx=0).stop 2024-09-12T03:03:56.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.537+0000 7f4f3573f740 1 -- 172.21.15.116:0/3029346171 shutdown_connections 2024-09-12T03:03:56.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.537+0000 7f4f3573f740 1 --2- 172.21.15.116:0/3029346171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56531a428d60 0x56531a449860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.537+0000 7f4f3573f740 1 --2- 172.21.15.116:0/3029346171 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56531a426760 0x56531a449320 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.537+0000 7f4f3573f740 1 --2- 172.21.15.116:0/3029346171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56531a41db80 0x56531a43d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.537+0000 7f4f3573f740 1 -- 172.21.15.116:0/3029346171 >> 172.21.15.116:0/3029346171 conn(0x56531a41a110 msgr2=0x56531a427d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:56.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.538+0000 7f4f3573f740 1 -- 172.21.15.116:0/3029346171 shutdown_connections 2024-09-12T03:03:56.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.538+0000 7f4f3573f740 1 -- 172.21.15.116:0/3029346171 wait complete. 2024-09-12T03:03:56.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.538+0000 7f4f3573f740 1 -- start start 2024-09-12T03:03:56.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.538+0000 7f4f3573f740 1 Processor -- start 2024-09-12T03:03:56.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.539+0000 7f4f3573f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56531a41db80 0x56531a435400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:56.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.539+0000 7f4f3573f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56531a426760 0x56531a435940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:56.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.539+0000 7f4f3573f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56531a428d60 0x56531a435e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:56.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.539+0000 7f4f31102640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56531a41db80 0x56531a435400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:56.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.539+0000 7f4f31102640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56531a41db80 0x56531a435400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.116:33202/0 (socket says 172.21.15.116:33202) 2024-09-12T03:03:56.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.539+0000 7f4f30901640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56531a426760 0x56531a435940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:56.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.539+0000 7f4f30901640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56531a426760 0x56531a435940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.116:44964/0 (socket says 172.21.15.116:44964) 2024-09-12T03:03:56.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.539+0000 7f4f31903640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56531a428d60 0x56531a435e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:56.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.539+0000 7f4f31903640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56531a428d60 0x56531a435e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39486/0 (socket says 172.21.15.116:39486) 2024-09-12T03:03:56.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.539+0000 7f4f31102640 1 -- 172.21.15.116:0/581361 learned_addr learned my addr 172.21.15.116:0/581361 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:56.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.539+0000 7f4f3573f740 1 -- 172.21.15.116:0/581361 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c1a9750 con 0x56531a428d60 2024-09-12T03:03:56.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.539+0000 7f4f3573f740 1 -- 172.21.15.116:0/581361 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c1aabc0 con 0x56531a426760 2024-09-12T03:03:56.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.539+0000 7f4f3573f740 1 -- 172.21.15.116:0/581361 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c1a9e20 con 0x56531a41db80 2024-09-12T03:03:56.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.539+0000 7f4f3573f740 10 client.0 _ll_register_callbacks cb 0x7f4f28009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:56.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.540+0000 7f4f30901640 1 -- 172.21.15.116:0/581361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56531a41db80 msgr2=0x56531a435400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:56.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.540+0000 7f4f30901640 1 --2- 172.21.15.116:0/581361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56531a41db80 0x56531a435400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.540+0000 7f4f30901640 1 -- 172.21.15.116:0/581361 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56531a428d60 msgr2=0x56531a435e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:56.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.540+0000 7f4f30901640 1 --2- 172.21.15.116:0/581361 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56531a428d60 0x56531a435e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.540+0000 7f4f30901640 1 -- 172.21.15.116:0/581361 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56531a4ab370 con 0x56531a426760 2024-09-12T03:03:56.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.540+0000 7f4f30901640 1 --2- 172.21.15.116:0/581361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56531a426760 0x56531a435940 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c10dc20 tx=0x7f4f1c00b2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f91d8fbe23260f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:56.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.540+0000 7f4f14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-09-12T03:03:56.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.540+0000 7f4f14ff9640 1 -- 172.21.15.116:0/581361 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c011640 con 0x56531a426760 2024-09-12T03:03:56.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.540+0000 7f4f14ff9640 1 -- 172.21.15.116:0/581361 <== mon.2 v2:172.21.15.177:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f4f1c0117e0 con 0x56531a426760 2024-09-12T03:03:56.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.541+0000 7f4f3573f740 1 -- 172.21.15.116:0/581361 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f1c028c30 con 0x56531a426760 2024-09-12T03:03:56.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.541+0000 7f4f14ff9640 1 -- 172.21.15.116:0/581361 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4f1c0283b0 con 0x56531a426760 2024-09-12T03:03:56.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.541+0000 7f4f3573f740 10 client.24956 fetch_fsmap learned FSMap version 25 2024-09-12T03:03:56.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.541+0000 7f4f3573f740 1 -- 172.21.15.116:0/581361 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56531a4ac5c0 con 0x56531a426760 2024-09-12T03:03:56.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.542+0000 7f4f14ff9640 1 -- 172.21.15.116:0/581361 <== mon.2 v2:172.21.15.177:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f1c028c30 con 0x56531a426760 2024-09-12T03:03:56.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.543+0000 7f4f14ff9640 1 -- 172.21.15.116:0/581361 <== mon.2 v2:172.21.15.177:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4f1c045a30 con 0x56531a426760 2024-09-12T03:03:56.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.543+0000 7f4f3573f740 10 client.24956 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:03:56.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.543+0000 7f4f3573f740 10 client.24956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:03:56.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.543+0000 7f4f3573f740 20 client.24956 populate_metadata read hostname 'smithi116' 2024-09-12T03:03:56.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.543+0000 7f4f3573f740 1 --2- 172.21.15.116:0/581361 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56531a4416d0 0x56531a4aec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:56.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.543+0000 7f4f3573f740 4 client.24956 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:03:56.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.543+0000 7f4f3573f740 1 -- 172.21.15.116:0/581361 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56531a38ad00 con 0x56531a4416d0 2024-09-12T03:03:56.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.543+0000 7f4f31102640 1 --2- 172.21.15.116:0/581361 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56531a4416d0 0x56531a4aec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:56.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.544+0000 7f4f14ff9640 10 client.24956 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:56.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.544+0000 7f4f31102640 1 --2- 172.21.15.116:0/581361 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56531a4416d0 0x56531a4aec20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79cb6971cbc39621 server_cookie=e4acb28179479aa2 in_seq=0 out_seq=0 2024-09-12T03:03:56.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.545+0000 7f4f14ff9640 1 -- 172.21.15.116:0/581361 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56531a38ad00 con 0x56531a4416d0 2024-09-12T03:03:56.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.545+0000 7f4f14ff9640 10 client.24956 handle_command_reply: tid=0 2024-09-12T03:03:56.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.545+0000 7f4f3573f740 1 client.24956 shutdown 2024-09-12T03:03:56.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.545+0000 7f4f3573f740 1 -- 172.21.15.116:0/581361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56531a426760 msgr2=0x56531a435940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:56.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.545+0000 7f4f3573f740 1 --2- 172.21.15.116:0/581361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56531a426760 0x56531a435940 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c10dc20 tx=0x7f4f1c00b2e0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.545+0000 7f4f3573f740 1 -- 172.21.15.116:0/581361 shutdown_connections 2024-09-12T03:03:56.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.545+0000 7f4f3573f740 1 --2- 172.21.15.116:0/581361 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56531a4416d0 0x56531a4aec20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.545+0000 7f4f3573f740 1 --2- 172.21.15.116:0/581361 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56531a428d60 0x56531a435e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.545+0000 7f4f3573f740 1 --2- 172.21.15.116:0/581361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56531a426760 0x56531a435940 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.546+0000 7f4f14ff9640 0 client.24956 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:56.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.545+0000 7f4f3573f740 1 --2- 172.21.15.116:0/581361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56531a41db80 0x56531a435400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.546+0000 7f4f3573f740 1 -- 172.21.15.116:0/581361 >> 172.21.15.116:0/581361 conn(0x56531a41a110 msgr2=0x56531a427540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:56.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.546+0000 7f4f3573f740 1 -- 172.21.15.116:0/581361 shutdown_connections 2024-09-12T03:03:56.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.546+0000 7f4f3573f740 1 -- 172.21.15.116:0/581361 wait complete. 2024-09-12T03:03:56.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.546+0000 7f4f3573f740 20 client.24956 trim_cache size 0 max 16384 2024-09-12T03:03:56.547 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:03:56.547 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:03:56.547 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "73bfb458-4736-4d6b-89c3-f7329f32a5a1", 2024-09-12T03:03:56.548 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:03:56.548 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:03:56.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.547+0000 7f4f33366640 1 -- 172.21.15.116:0/1899275369 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4f180a2c10 msgr2=0x7f4f180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:56.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.547+0000 7f4f33366640 1 --2- 172.21.15.116:0/1899275369 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4f180a2c10 0x7f4f180a50d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x56531a41b490 tx=0x7f4f1c00b350 comp rx=0 tx=0).stop 2024-09-12T03:03:56.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.547+0000 7f4f33366640 1 -- 172.21.15.116:0/1899275369 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f2c102ef0 msgr2=0x7f4f2c198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:56.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.547+0000 7f4f33366640 1 --2- 172.21.15.116:0/1899275369 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f2c102ef0 0x7f4f2c198620 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4f24004af0 tx=0x7f4f2402f790 comp rx=0 tx=0).stop 2024-09-12T03:03:56.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.547+0000 7f4f33366640 1 -- 172.21.15.116:0/1899275369 shutdown_connections 2024-09-12T03:03:56.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.547+0000 7f4f33366640 1 --2- 172.21.15.116:0/1899275369 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4f180a2c10 0x7f4f180a50d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.547+0000 7f4f33366640 1 --2- 172.21.15.116:0/1899275369 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f2c102ef0 0x7f4f2c198620 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.547+0000 7f4f33366640 1 --2- 172.21.15.116:0/1899275369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f2c102570 0x7f4f2c19d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.547+0000 7f4f33366640 1 --2- 172.21.15.116:0/1899275369 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f2c100e20 0x7f4f2c19ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.547+0000 7f4f33366640 1 -- 172.21.15.116:0/1899275369 >> 172.21.15.116:0/1899275369 conn(0x7f4f2c0fc7d0 msgr2=0x7f4f2c103bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:56.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.547+0000 7f4f33366640 1 -- 172.21.15.116:0/1899275369 shutdown_connections 2024-09-12T03:03:56.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:56.547+0000 7f4f33366640 1 -- 172.21.15.116:0/1899275369 wait complete. 2024-09-12T03:03:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:56 smithi183 ceph-mon[35535]: pgmap v366: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-12T03:03:56.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:56 smithi177 ceph-mon[35305]: pgmap v366: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-12T03:03:57.258 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:57 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:03:57.258 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:57 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:57.258 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:57 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:03:57.258 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:57 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:57.353 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:03:57.617 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:57.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:57 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:03:57.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:57 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:57.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:57 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:03:57.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:57 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:57.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:57 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:03:57.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:57 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:57.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:57 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:03:57.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:57 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:58.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.164+0000 7f00f7e43640 1 -- 172.21.15.116:0/702508954 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00f0102570 msgr2=0x7f00f01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:58.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.164+0000 7f00f7e43640 1 --2- 172.21.15.116:0/702508954 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00f0102570 0x7f00f01029b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f00e0009900 tx=0x7f00e002a480 comp rx=0 tx=0).stop 2024-09-12T03:03:58.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.164+0000 7f00f7e43640 1 -- 172.21.15.116:0/702508954 shutdown_connections 2024-09-12T03:03:58.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.164+0000 7f00f7e43640 1 --2- 172.21.15.116:0/702508954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00f0102ef0 0x7f00f010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.164+0000 7f00f7e43640 1 --2- 172.21.15.116:0/702508954 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00f0102570 0x7f00f01029b0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.164+0000 7f00f7e43640 1 --2- 172.21.15.116:0/702508954 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00f0100e20 0x7f00f0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.164+0000 7f00f7e43640 1 -- 172.21.15.116:0/702508954 >> 172.21.15.116:0/702508954 conn(0x7f00f00fc7d0 msgr2=0x7f00f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:58.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.165+0000 7f00f7e43640 1 -- 172.21.15.116:0/702508954 shutdown_connections 2024-09-12T03:03:58.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.165+0000 7f00f7e43640 1 -- 172.21.15.116:0/702508954 wait complete. 2024-09-12T03:03:58.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.165+0000 7f00f7e43640 1 Processor -- start 2024-09-12T03:03:58.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.165+0000 7f00f7e43640 1 -- start start 2024-09-12T03:03:58.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.165+0000 7f00f7e43640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00f0100e20 0x7f00f0198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:58.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.165+0000 7f00f7e43640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00f0102570 0x7f00f0198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:58.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.165+0000 7f00f7e43640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00f0102ef0 0x7f00f019ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:58.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.166+0000 7f00f7e43640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f00f01120a0 con 0x7f00f0102570 2024-09-12T03:03:58.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.166+0000 7f00f7e43640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f00f0111f20 con 0x7f00f0102ef0 2024-09-12T03:03:58.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.166+0000 7f00f7e43640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f00f0112220 con 0x7f00f0100e20 2024-09-12T03:03:58.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.166+0000 7f00f63e0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00f0102ef0 0x7f00f019ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:58.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.166+0000 7f00f63e0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00f0102ef0 0x7f00f019ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.116:44984/0 (socket says 172.21.15.116:44984) 2024-09-12T03:03:58.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.166+0000 7f00f63e0640 1 -- 172.21.15.116:0/473320934 learned_addr learned my addr 172.21.15.116:0/473320934 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:58.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.166+0000 7f00f53de640 1 --2- 172.21.15.116:0/473320934 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00f0102570 0x7f00f0198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:58.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.166+0000 7f00f63e0640 1 -- 172.21.15.116:0/473320934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00f0100e20 msgr2=0x7f00f0198560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:03:58.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.166+0000 7f00f63e0640 1 --2- 172.21.15.116:0/473320934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00f0100e20 0x7f00f0198560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.166+0000 7f00f63e0640 1 -- 172.21.15.116:0/473320934 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00f0102570 msgr2=0x7f00f0198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:58.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.166+0000 7f00f63e0640 1 --2- 172.21.15.116:0/473320934 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00f0102570 0x7f00f0198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.166+0000 7f00f63e0640 1 -- 172.21.15.116:0/473320934 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f019d510 con 0x7f00f0102ef0 2024-09-12T03:03:58.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.166+0000 7f00f53de640 1 --2- 172.21.15.116:0/473320934 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00f0102570 0x7f00f0198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:03:58.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.166+0000 7f00f63e0640 1 --2- 172.21.15.116:0/473320934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00f0102ef0 0x7f00f019ce30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f00e800b970 tx=0x7f00e800be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:58.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.167+0000 7f00e6ffd640 1 -- 172.21.15.116:0/473320934 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e8019070 con 0x7f00f0102ef0 2024-09-12T03:03:58.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.167+0000 7f00e6ffd640 1 -- 172.21.15.116:0/473320934 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e800cb40 con 0x7f00f0102ef0 2024-09-12T03:03:58.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.167+0000 7f00e6ffd640 1 -- 172.21.15.116:0/473320934 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e8007650 con 0x7f00f0102ef0 2024-09-12T03:03:58.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.167+0000 7f00f7e43640 1 -- 172.21.15.116:0/473320934 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f019d800 con 0x7f00f0102ef0 2024-09-12T03:03:58.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.167+0000 7f00f7e43640 1 -- 172.21.15.116:0/473320934 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f010e400 con 0x7f00f0102ef0 2024-09-12T03:03:58.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.170+0000 7f00e6ffd640 1 -- 172.21.15.116:0/473320934 <== mon.2 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f00e801c070 con 0x7f00f0102ef0 2024-09-12T03:03:58.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.171+0000 7f00fa21c740 1 Processor -- start 2024-09-12T03:03:58.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.171+0000 7f00fa21c740 1 -- start start 2024-09-12T03:03:58.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.171+0000 7f00fa21c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5600adf194a0 0x5600adf19880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:58.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.171+0000 7f00fa21c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5600adf19dc0 0x5600adf24140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:58.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.172+0000 7f00e6ffd640 1 --2- 172.21.15.116:0/473320934 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f00dc0a2c90 0x7f00dc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:58.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.172+0000 7f00fa21c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600adf24680 0x5600adf26a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:58.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.172+0000 7f00fa21c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5600ade7f2e0 con 0x5600adf194a0 2024-09-12T03:03:58.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.172+0000 7f00fa21c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5600ade26ce0 con 0x5600adf19dc0 2024-09-12T03:03:58.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.172+0000 7f00fa21c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5600ade7b2a0 con 0x5600adf24680 2024-09-12T03:03:58.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.172+0000 7f00e6ffd640 1 -- 172.21.15.116:0/473320934 <== mon.2 v2:172.21.15.177:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f00e80ce6d0 con 0x7f00f0102ef0 2024-09-12T03:03:58.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.172+0000 7f00f53de640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5600adf194a0 0x5600adf19880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:58.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.172+0000 7f00f53de640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5600adf194a0 0x5600adf19880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39518/0 (socket says 172.21.15.116:39518) 2024-09-12T03:03:58.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.172+0000 7f00f53de640 1 -- 172.21.15.116:0/3870708141 learned_addr learned my addr 172.21.15.116:0/3870708141 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:58.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.172+0000 7f00f5bdf640 1 --2- 172.21.15.116:0/3870708141 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5600adf19dc0 0x5600adf24140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:58.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.172+0000 7f00f63e0640 1 --2- 172.21.15.116:0/3870708141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600adf24680 0x5600adf26a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:58.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.173+0000 7f00f53de640 1 -- 172.21.15.116:0/3870708141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600adf24680 msgr2=0x5600adf26a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:58.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.173+0000 7f00f53de640 1 --2- 172.21.15.116:0/3870708141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600adf24680 0x5600adf26a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.173+0000 7f00f53de640 1 -- 172.21.15.116:0/3870708141 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5600adf19dc0 msgr2=0x5600adf24140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:58.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.173+0000 7f00f53de640 1 --2- 172.21.15.116:0/3870708141 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5600adf19dc0 0x5600adf24140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.173+0000 7f00f53de640 1 -- 172.21.15.116:0/3870708141 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600ade0b440 con 0x5600adf194a0 2024-09-12T03:03:58.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.173+0000 7f00f53de640 1 --2- 172.21.15.116:0/3870708141 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5600adf194a0 0x5600adf19880 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f00e0005ab0 tx=0x7f00e0004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=309dea4bdc8b5323 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:58.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.173+0000 7f00f53de640 1 --2- 172.21.15.116:0/473320934 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f00dc0a2c90 0x7f00dc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:58.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.174+0000 7f00dbfff640 1 -- 172.21.15.116:0/3870708141 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e003f070 con 0x5600adf194a0 2024-09-12T03:03:58.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.174+0000 7f00dbfff640 1 -- 172.21.15.116:0/3870708141 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e00046c0 con 0x5600adf194a0 2024-09-12T03:03:58.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.174+0000 7f00dbfff640 1 -- 172.21.15.116:0/3870708141 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0035030 con 0x5600adf194a0 2024-09-12T03:03:58.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.174+0000 7f00f53de640 1 --2- 172.21.15.116:0/473320934 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f00dc0a2c90 0x7f00dc0a5150 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f00e0042000 tx=0x7f00e0003900 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:58.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.175+0000 7f00fa21c740 1 -- 172.21.15.116:0/3870708141 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5600adf194a0 msgr2=0x5600adf19880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:58.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.175+0000 7f00fa21c740 1 --2- 172.21.15.116:0/3870708141 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5600adf194a0 0x5600adf19880 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f00e0005ab0 tx=0x7f00e0004200 comp rx=0 tx=0).stop 2024-09-12T03:03:58.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.175+0000 7f00fa21c740 1 -- 172.21.15.116:0/3870708141 shutdown_connections 2024-09-12T03:03:58.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.175+0000 7f00fa21c740 1 --2- 172.21.15.116:0/3870708141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600adf24680 0x5600adf26a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.175+0000 7f00fa21c740 1 --2- 172.21.15.116:0/3870708141 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5600adf19dc0 0x5600adf24140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.175+0000 7f00fa21c740 1 --2- 172.21.15.116:0/3870708141 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5600adf194a0 0x5600adf19880 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.175+0000 7f00fa21c740 1 -- 172.21.15.116:0/3870708141 >> 172.21.15.116:0/3870708141 conn(0x5600adf15a30 msgr2=0x5600adf15e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:58.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.176+0000 7f00fa21c740 1 -- 172.21.15.116:0/3870708141 shutdown_connections 2024-09-12T03:03:58.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.176+0000 7f00fa21c740 1 -- 172.21.15.116:0/3870708141 wait complete. 2024-09-12T03:03:58.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.176+0000 7f00fa21c740 1 -- start start 2024-09-12T03:03:58.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.176+0000 7f00fa21c740 1 Processor -- start 2024-09-12T03:03:58.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.177+0000 7f00fa21c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5600adf194a0 0x5600adf23f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:58.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.177+0000 7f00fa21c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5600adf19dc0 0x5600adf21200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:58.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.177+0000 7f00fa21c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600adf24680 0x5600adf21740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:58.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.178+0000 7f00fa21c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5600adf27250 con 0x5600adf194a0 2024-09-12T03:03:58.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.178+0000 7f00fa21c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5600ade3a050 con 0x5600adf19dc0 2024-09-12T03:03:58.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.178+0000 7f00fa21c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5600adf273d0 con 0x5600adf24680 2024-09-12T03:03:58.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.178+0000 7f00f63e0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600adf24680 0x5600adf21740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:58.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.178+0000 7f00f63e0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600adf24680 0x5600adf21740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.116:33242/0 (socket says 172.21.15.116:33242) 2024-09-12T03:03:58.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.178+0000 7f00f5bdf640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5600adf194a0 0x5600adf23f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:58.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.178+0000 7f00f63e0640 1 -- 172.21.15.116:0/1581465242 learned_addr learned my addr 172.21.15.116:0/1581465242 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:58.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.178+0000 7f00f53de640 1 --2- 172.21.15.116:0/1581465242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5600adf19dc0 0x5600adf21200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:58.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.178+0000 7f00f63e0640 1 -- 172.21.15.116:0/1581465242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5600adf19dc0 msgr2=0x5600adf21200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:58.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.178+0000 7f00f63e0640 1 --2- 172.21.15.116:0/1581465242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5600adf19dc0 0x5600adf21200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.178+0000 7f00f63e0640 1 -- 172.21.15.116:0/1581465242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5600adf194a0 msgr2=0x5600adf23f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:58.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.178+0000 7f00fa21c740 10 client.0 _ll_register_callbacks cb 0x5600ade7a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:58.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.178+0000 7f00f63e0640 1 --2- 172.21.15.116:0/1581465242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5600adf194a0 0x5600adf23f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.178+0000 7f00f63e0640 1 -- 172.21.15.116:0/1581465242 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600adbf03c0 con 0x5600adf24680 2024-09-12T03:03:58.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.178+0000 7f00fa21c740 1 -- 172.21.15.116:0/1581465242 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00e00046c0 con 0x5600adf24680 2024-09-12T03:03:58.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.178+0000 7f00f5bdf640 1 --2- 172.21.15.116:0/1581465242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5600adf194a0 0x5600adf23f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:58.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.179+0000 7f00f63e0640 1 --2- 172.21.15.116:0/1581465242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600adf24680 0x5600adf21740 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f00e80cc190 tx=0x7f00e809f260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15779314e4d49abe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:58.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.179+0000 7f00d97fa640 10 client.24994 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T03:03:58.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.179+0000 7f00d97fa640 1 -- 172.21.15.116:0/1581465242 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e80cc360 con 0x5600adf24680 2024-09-12T03:03:58.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.179+0000 7f00d97fa640 1 -- 172.21.15.116:0/1581465242 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f00e80a3690 con 0x5600adf24680 2024-09-12T03:03:58.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.179+0000 7f00d97fa640 1 -- 172.21.15.116:0/1581465242 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f00e80a38b0 con 0x5600adf24680 2024-09-12T03:03:58.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.179+0000 7f00fa21c740 10 client.24994 fetch_fsmap learned FSMap version 25 2024-09-12T03:03:58.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.179+0000 7f00fa21c740 1 -- 172.21.15.116:0/1581465242 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5600adf32aa0 con 0x5600adf24680 2024-09-12T03:03:58.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.180+0000 7f00d97fa640 1 -- 172.21.15.116:0/1581465242 <== mon.1 v2:172.21.15.183:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f00e809a300 con 0x5600adf24680 2024-09-12T03:03:58.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.180+0000 7f00fa21c740 10 client.24994 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:03:58.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.180+0000 7f00fa21c740 10 client.24994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:03:58.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.180+0000 7f00fa21c740 20 client.24994 populate_metadata read hostname 'smithi116' 2024-09-12T03:03:58.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.180+0000 7f00fa21c740 1 --2- 172.21.15.116:0/1581465242 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5600adf33e50 0x5600adf362c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:58.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.180+0000 7f00fa21c740 4 client.24994 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:03:58.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.180+0000 7f00fa21c740 1 -- 172.21.15.116:0/1581465242 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600ade8c850 con 0x5600adf33e50 2024-09-12T03:03:58.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.180+0000 7f00f5bdf640 1 --2- 172.21.15.116:0/1581465242 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5600adf33e50 0x5600adf362c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:58.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.181+0000 7f00d97fa640 10 client.24994 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:58.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.181+0000 7f00f5bdf640 1 --2- 172.21.15.116:0/1581465242 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5600adf33e50 0x5600adf362c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2afc762e1d7462b0 server_cookie=c9a0b202038dd9e3 in_seq=0 out_seq=0 2024-09-12T03:03:58.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.181+0000 7f00d97fa640 1 -- 172.21.15.116:0/1581465242 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5600ade8c850 con 0x5600adf33e50 2024-09-12T03:03:58.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.181+0000 7f00d97fa640 10 client.24994 handle_command_reply: tid=0 2024-09-12T03:03:58.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.181+0000 7f00fa21c740 1 client.24994 shutdown 2024-09-12T03:03:58.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.182+0000 7f00fa21c740 1 -- 172.21.15.116:0/1581465242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600adf24680 msgr2=0x5600adf21740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:58.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.182+0000 7f00fa21c740 1 --2- 172.21.15.116:0/1581465242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600adf24680 0x5600adf21740 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f00e80cc190 tx=0x7f00e809f260 comp rx=0 tx=0).stop 2024-09-12T03:03:58.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.182+0000 7f00fa21c740 1 -- 172.21.15.116:0/1581465242 shutdown_connections 2024-09-12T03:03:58.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.182+0000 7f00fa21c740 1 --2- 172.21.15.116:0/1581465242 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5600adf33e50 0x5600adf362c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.182+0000 7f00fa21c740 1 --2- 172.21.15.116:0/1581465242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600adf24680 0x5600adf21740 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.182+0000 7f00d97fa640 0 client.24994 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:58.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.182+0000 7f00fa21c740 1 --2- 172.21.15.116:0/1581465242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5600adf19dc0 0x5600adf21200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.182+0000 7f00fa21c740 1 --2- 172.21.15.116:0/1581465242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5600adf194a0 0x5600adf23f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.182+0000 7f00fa21c740 1 -- 172.21.15.116:0/1581465242 >> 172.21.15.116:0/1581465242 conn(0x5600adf15a30 msgr2=0x5600adf14b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:58.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.182+0000 7f00fa21c740 1 -- 172.21.15.116:0/1581465242 shutdown_connections 2024-09-12T03:03:58.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.182+0000 7f00fa21c740 1 -- 172.21.15.116:0/1581465242 wait complete. 2024-09-12T03:03:58.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.182+0000 7f00fa21c740 20 client.24994 trim_cache size 0 max 16384 2024-09-12T03:03:58.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.216+0000 7f00fa21c740 1 Processor -- start 2024-09-12T03:03:58.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.216+0000 7f00fa21c740 1 -- start start 2024-09-12T03:03:58.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.216+0000 7f00fa21c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600adf15a30 0x5600adf23170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:58.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.216+0000 7f00fa21c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5600adf194a0 0x5600adf3bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:58.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.216+0000 7f00fa21c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5600adf19dc0 0x5600adf3c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:58.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.216+0000 7f00fa21c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f00f01b2090 con 0x5600adf194a0 2024-09-12T03:03:58.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.216+0000 7f00fa21c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f00f01af960 con 0x5600adf19dc0 2024-09-12T03:03:58.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.216+0000 7f00fa21c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f00f01ad7b0 con 0x5600adf15a30 2024-09-12T03:03:58.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.216+0000 7f00f53de640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5600adf194a0 0x5600adf3bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:58.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.216+0000 7f00f53de640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5600adf194a0 0x5600adf3bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39540/0 (socket says 172.21.15.116:39540) 2024-09-12T03:03:58.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.216+0000 7f00f53de640 1 -- 172.21.15.116:0/880463258 learned_addr learned my addr 172.21.15.116:0/880463258 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:58.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.217+0000 7f00f5bdf640 1 --2- 172.21.15.116:0/880463258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600adf15a30 0x5600adf23170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:58.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.217+0000 7f00f63e0640 1 --2- 172.21.15.116:0/880463258 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5600adf19dc0 0x5600adf3c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:58.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.217+0000 7f00f53de640 1 -- 172.21.15.116:0/880463258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600adf15a30 msgr2=0x5600adf23170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:58.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.217+0000 7f00f53de640 1 --2- 172.21.15.116:0/880463258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600adf15a30 0x5600adf23170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.217+0000 7f00f53de640 1 -- 172.21.15.116:0/880463258 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5600adf19dc0 msgr2=0x5600adf3c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:58.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.217+0000 7f00f53de640 1 --2- 172.21.15.116:0/880463258 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5600adf19dc0 0x5600adf3c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.217+0000 7f00f53de640 1 -- 172.21.15.116:0/880463258 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600adf192d0 con 0x5600adf194a0 2024-09-12T03:03:58.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.218+0000 7f00f53de640 1 --2- 172.21.15.116:0/880463258 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5600adf194a0 0x5600adf3bb90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f00e0005ab0 tx=0x7f00e004f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d323de7c633b6e8e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:58.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.218+0000 7f00db7fe640 1 -- 172.21.15.116:0/880463258 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0035030 con 0x5600adf194a0 2024-09-12T03:03:58.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.218+0000 7f00db7fe640 1 -- 172.21.15.116:0/880463258 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e0057070 con 0x5600adf194a0 2024-09-12T03:03:58.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.218+0000 7f00db7fe640 1 -- 172.21.15.116:0/880463258 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e003f070 con 0x5600adf194a0 2024-09-12T03:03:58.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.219+0000 7f00fa21c740 1 -- 172.21.15.116:0/880463258 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5600adf194a0 msgr2=0x5600adf3bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:58.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.219+0000 7f00fa21c740 1 --2- 172.21.15.116:0/880463258 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5600adf194a0 0x5600adf3bb90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f00e0005ab0 tx=0x7f00e004f000 comp rx=0 tx=0).stop 2024-09-12T03:03:58.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.219+0000 7f00fa21c740 1 -- 172.21.15.116:0/880463258 shutdown_connections 2024-09-12T03:03:58.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.219+0000 7f00fa21c740 1 --2- 172.21.15.116:0/880463258 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5600adf19dc0 0x5600adf3c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.219+0000 7f00fa21c740 1 --2- 172.21.15.116:0/880463258 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5600adf194a0 0x5600adf3bb90 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.219+0000 7f00fa21c740 1 --2- 172.21.15.116:0/880463258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600adf15a30 0x5600adf23170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.219+0000 7f00fa21c740 1 -- 172.21.15.116:0/880463258 >> 172.21.15.116:0/880463258 conn(0x5600adf218a0 msgr2=0x5600adf22a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:58.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.219+0000 7f00fa21c740 1 -- 172.21.15.116:0/880463258 shutdown_connections 2024-09-12T03:03:58.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.219+0000 7f00fa21c740 1 -- 172.21.15.116:0/880463258 wait complete. 2024-09-12T03:03:58.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.220+0000 7f00fa21c740 1 -- start start 2024-09-12T03:03:58.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.220+0000 7f00fa21c740 1 Processor -- start 2024-09-12T03:03:58.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.220+0000 7f00fa21c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5600adf15a30 0x5600adf2a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:58.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.220+0000 7f00fa21c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5600adf194a0 0x5600adf2a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:58.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.220+0000 7f00fa21c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600adf19dc0 0x5600adf2aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:58.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.220+0000 7f00fa21c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f00f01ade20 con 0x5600adf194a0 2024-09-12T03:03:58.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.220+0000 7f00fa21c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f00f01af290 con 0x5600adf15a30 2024-09-12T03:03:58.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.220+0000 7f00fa21c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f00f01ae4f0 con 0x5600adf19dc0 2024-09-12T03:03:58.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.220+0000 7f00fa21c740 10 client.0 _ll_register_callbacks cb 0x7f00e809aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:58.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.221+0000 7f00f5bdf640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5600adf15a30 0x5600adf2a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:58.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.221+0000 7f00f5bdf640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5600adf15a30 0x5600adf2a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.116:45022/0 (socket says 172.21.15.116:45022) 2024-09-12T03:03:58.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.221+0000 7f00f5bdf640 1 -- 172.21.15.116:0/399271858 learned_addr learned my addr 172.21.15.116:0/399271858 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:03:58.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.221+0000 7f00f53de640 1 --2- 172.21.15.116:0/399271858 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5600adf194a0 0x5600adf2a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:58.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.221+0000 7f00f63e0640 1 --2- 172.21.15.116:0/399271858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600adf19dc0 0x5600adf2aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:58.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.221+0000 7f00f5bdf640 1 -- 172.21.15.116:0/399271858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600adf19dc0 msgr2=0x5600adf2aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:58.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.221+0000 7f00f5bdf640 1 --2- 172.21.15.116:0/399271858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600adf19dc0 0x5600adf2aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.221+0000 7f00f5bdf640 1 -- 172.21.15.116:0/399271858 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5600adf194a0 msgr2=0x5600adf2a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:58.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.221+0000 7f00f5bdf640 1 --2- 172.21.15.116:0/399271858 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5600adf194a0 0x5600adf2a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.221+0000 7f00f5bdf640 1 -- 172.21.15.116:0/399271858 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600adf9dc20 con 0x5600adf15a30 2024-09-12T03:03:58.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.221+0000 7f00f63e0640 1 --2- 172.21.15.116:0/399271858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600adf19dc0 0x5600adf2aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:03:58.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.221+0000 7f00f5bdf640 1 --2- 172.21.15.116:0/399271858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5600adf15a30 0x5600adf2a430 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x5600adf36b00 tx=0x7f00ec01db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b2b4336dde8ed32 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:58.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.221+0000 7f00d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-09-12T03:03:58.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.221+0000 7f00d97fa640 1 -- 172.21.15.116:0/399271858 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec004020 con 0x5600adf15a30 2024-09-12T03:03:58.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.222+0000 7f00d97fa640 1 -- 172.21.15.116:0/399271858 <== mon.2 v2:172.21.15.177:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f00ec0041c0 con 0x5600adf15a30 2024-09-12T03:03:58.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.222+0000 7f00fa21c740 1 -- 172.21.15.116:0/399271858 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00e0057070 con 0x5600adf15a30 2024-09-12T03:03:58.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.222+0000 7f00d97fa640 1 -- 172.21.15.116:0/399271858 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f00ec0043c0 con 0x5600adf15a30 2024-09-12T03:03:58.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.222+0000 7f00fa21c740 10 client.24971 fetch_fsmap learned FSMap version 25 2024-09-12T03:03:58.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.222+0000 7f00fa21c740 1 -- 172.21.15.116:0/399271858 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5600adf9eef0 con 0x5600adf15a30 2024-09-12T03:03:58.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.222+0000 7f00d97fa640 1 -- 172.21.15.116:0/399271858 <== mon.2 v2:172.21.15.177:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e0057070 con 0x5600adf15a30 2024-09-12T03:03:58.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.222+0000 7f00d97fa640 1 -- 172.21.15.116:0/399271858 <== mon.2 v2:172.21.15.177:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f00ec003a30 con 0x5600adf15a30 2024-09-12T03:03:58.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.222+0000 7f00fa21c740 10 client.24971 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:03:58.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.222+0000 7f00fa21c740 10 client.24971 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:03:58.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.222+0000 7f00fa21c740 20 client.24971 populate_metadata read hostname 'smithi116' 2024-09-12T03:03:58.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.222+0000 7f00fa21c740 1 --2- 172.21.15.116:0/399271858 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5600adf24680 0x5600adfa1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:58.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.222+0000 7f00fa21c740 4 client.24971 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:03:58.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.222+0000 7f00fa21c740 1 -- 172.21.15.116:0/399271858 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5600ade86d60 con 0x5600adf24680 2024-09-12T03:03:58.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.222+0000 7f00f53de640 1 --2- 172.21.15.116:0/399271858 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5600adf24680 0x5600adfa1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:58.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.223+0000 7f00f53de640 1 --2- 172.21.15.116:0/399271858 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5600adf24680 0x5600adfa1620 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec44369b33ac9cb server_cookie=5c2e7576eb8b3e8d in_seq=0 out_seq=0 2024-09-12T03:03:58.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.223+0000 7f00d97fa640 10 client.24971 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:58.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.223+0000 7f00d97fa640 1 -- 172.21.15.116:0/399271858 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5600ade86d60 con 0x5600adf24680 2024-09-12T03:03:58.228 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:03:58.228 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:03:58.228 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": { 2024-09-12T03:03:58.228 INFO:teuthology.orchestra.run.smithi116.stdout: "73bfb458-4736-4d6b-89c3-f7329f32a5a1": { 2024-09-12T03:03:58.229 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/", 2024-09-12T03:03:58.229 INFO:teuthology.orchestra.run.smithi116.stdout: "tag": "73bfb458-4736-4d6b-89c3-f7329f32a5a1", 2024-09-12T03:03:58.229 INFO:teuthology.orchestra.run.smithi116.stdout: "options": "recursive,force" 2024-09-12T03:03:58.229 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:03:58.229 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:03:58.229 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:03:58.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.223+0000 7f00d97fa640 10 client.24971 handle_command_reply: tid=0 2024-09-12T03:03:58.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.223+0000 7f00fa21c740 1 client.24971 shutdown 2024-09-12T03:03:58.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.223+0000 7f00fa21c740 1 -- 172.21.15.116:0/399271858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5600adf15a30 msgr2=0x5600adf2a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:58.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.223+0000 7f00fa21c740 1 --2- 172.21.15.116:0/399271858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5600adf15a30 0x5600adf2a430 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x5600adf36b00 tx=0x7f00ec01db90 comp rx=0 tx=0).stop 2024-09-12T03:03:58.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.223+0000 7f00fa21c740 1 -- 172.21.15.116:0/399271858 shutdown_connections 2024-09-12T03:03:58.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.223+0000 7f00fa21c740 1 --2- 172.21.15.116:0/399271858 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5600adf24680 0x5600adfa1620 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.224+0000 7f00fa21c740 1 --2- 172.21.15.116:0/399271858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5600adf19dc0 0x5600adf2aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.224+0000 7f00fa21c740 1 --2- 172.21.15.116:0/399271858 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5600adf194a0 0x5600adf2a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.224+0000 7f00d97fa640 0 client.24971 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:03:58.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.224+0000 7f00fa21c740 1 --2- 172.21.15.116:0/399271858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5600adf15a30 0x5600adf2a430 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.224+0000 7f00fa21c740 1 -- 172.21.15.116:0/399271858 >> 172.21.15.116:0/399271858 conn(0x5600adf218a0 msgr2=0x5600adf23280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:58.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.224+0000 7f00fa21c740 1 -- 172.21.15.116:0/399271858 shutdown_connections 2024-09-12T03:03:58.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.224+0000 7f00fa21c740 1 -- 172.21.15.116:0/399271858 wait complete. 2024-09-12T03:03:58.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.224+0000 7f00fa21c740 20 client.24971 trim_cache size 0 max 16384 2024-09-12T03:03:58.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.224+0000 7f00f7e43640 1 -- 172.21.15.116:0/473320934 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f00dc0a2c90 msgr2=0x7f00dc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:58.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.224+0000 7f00f7e43640 1 --2- 172.21.15.116:0/473320934 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f00dc0a2c90 0x7f00dc0a5150 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f00e0042000 tx=0x7f00e0003900 comp rx=0 tx=0).stop 2024-09-12T03:03:58.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.224+0000 7f00f7e43640 1 -- 172.21.15.116:0/473320934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00f0102ef0 msgr2=0x7f00f019ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:58.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.224+0000 7f00f7e43640 1 --2- 172.21.15.116:0/473320934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00f0102ef0 0x7f00f019ce30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f00e800b970 tx=0x7f00e800be30 comp rx=0 tx=0).stop 2024-09-12T03:03:58.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.225+0000 7f00f7e43640 1 -- 172.21.15.116:0/473320934 shutdown_connections 2024-09-12T03:03:58.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.225+0000 7f00f7e43640 1 --2- 172.21.15.116:0/473320934 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f00dc0a2c90 0x7f00dc0a5150 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.225+0000 7f00f7e43640 1 --2- 172.21.15.116:0/473320934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f00f0102ef0 0x7f00f019ce30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.225+0000 7f00f7e43640 1 --2- 172.21.15.116:0/473320934 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00f0102570 0x7f00f0198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.225+0000 7f00f7e43640 1 --2- 172.21.15.116:0/473320934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00f0100e20 0x7f00f0198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.225+0000 7f00f7e43640 1 -- 172.21.15.116:0/473320934 >> 172.21.15.116:0/473320934 conn(0x7f00f00fc7d0 msgr2=0x7f00f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:58.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.225+0000 7f00f7e43640 1 -- 172.21.15.116:0/473320934 shutdown_connections 2024-09-12T03:03:58.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:03:58.225+0000 7f00f7e43640 1 -- 172.21.15.116:0/473320934 wait complete. 2024-09-12T03:03:58.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:58 smithi116 ceph-mon[31410]: pgmap v367: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-12T03:03:58.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:58 smithi183 ceph-mon[35535]: pgmap v367: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-12T03:03:58.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:58 smithi177 ceph-mon[35305]: pgmap v367: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-12T03:03:59.042 INFO:tasks.cephfs.filesystem:scrub status for tag:73bfb458-4736-4d6b-89c3-f7329f32a5a1 - {'path': '/', 'tag': '73bfb458-4736-4d6b-89c3-f7329f32a5a1', 'options': 'recursive,force'} 2024-09-12T03:03:59.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:03:59 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:03:59.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:03:59 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:03:59.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:03:59 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:04:00.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:00 smithi116 ceph-mon[31410]: pgmap v368: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:04:00.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:00 smithi183 ceph-mon[35535]: pgmap v368: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:04:00.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:00 smithi177 ceph-mon[35305]: pgmap v368: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:04:02.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:02 smithi116 ceph-mon[31410]: pgmap v369: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-12T03:04:02.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:02 smithi183 ceph-mon[35535]: pgmap v369: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-12T03:04:02.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:02 smithi177 ceph-mon[35305]: pgmap v369: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-12T03:04:04.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:04 smithi116 ceph-mon[31410]: pgmap v370: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:04:04.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:04 smithi116 ceph-mon[31410]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-12T03:04:04.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:04 smithi183 ceph-mon[35535]: pgmap v370: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:04:04.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:04 smithi183 ceph-mon[35535]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-12T03:04:04.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:04 smithi177 ceph-mon[35305]: pgmap v370: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:04:04.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:04 smithi177 ceph-mon[35305]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-12T03:04:05.435 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:04:05.705 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:06.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.261+0000 7f0f2d6d9640 1 -- 172.21.15.116:0/2216306730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f28101290 msgr2=0x7f0f28107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:06.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.261+0000 7f0f2d6d9640 1 --2- 172.21.15.116:0/2216306730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f28101290 0x7f0f28107a50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c009930 tx=0x7f0f1c02a420 comp rx=0 tx=0).stop 2024-09-12T03:04:06.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.261+0000 7f0f2d6d9640 1 -- 172.21.15.116:0/2216306730 shutdown_connections 2024-09-12T03:04:06.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.261+0000 7f0f2d6d9640 1 --2- 172.21.15.116:0/2216306730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f28108160 0x7f0f2810eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:06.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.261+0000 7f0f2d6d9640 1 --2- 172.21.15.116:0/2216306730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f28101290 0x7f0f28107a50 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:06.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.261+0000 7f0f2d6d9640 1 --2- 172.21.15.116:0/2216306730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f281008e0 0x7f0f28100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:06.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.261+0000 7f0f2d6d9640 1 -- 172.21.15.116:0/2216306730 >> 172.21.15.116:0/2216306730 conn(0x7f0f28077210 msgr2=0x7f0f28077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:06.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.262+0000 7f0f2d6d9640 1 -- 172.21.15.116:0/2216306730 shutdown_connections 2024-09-12T03:04:06.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.262+0000 7f0f2d6d9640 1 -- 172.21.15.116:0/2216306730 wait complete. 2024-09-12T03:04:06.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.263+0000 7f0f2d6d9640 1 Processor -- start 2024-09-12T03:04:06.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.263+0000 7f0f2d6d9640 1 -- start start 2024-09-12T03:04:06.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.264+0000 7f0f2d6d9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f281008e0 0x7f0f28198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:06.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.264+0000 7f0f2d6d9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f28101290 0x7f0f28198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:06.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.264+0000 7f0f267fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f28101290 0x7f0f28198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:06.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.264+0000 7f0f267fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f28101290 0x7f0f28198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45856/0 (socket says 172.21.15.116:45856) 2024-09-12T03:04:06.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.264+0000 7f0f267fc640 1 -- 172.21.15.116:0/1106884501 learned_addr learned my addr 172.21.15.116:0/1106884501 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:06.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.264+0000 7f0f2d6d9640 1 --2- 172.21.15.116:0/1106884501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f28108160 0x7f0f2819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:06.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.264+0000 7f0f2d6d9640 1 -- 172.21.15.116:0/1106884501 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0f28111ff0 con 0x7f0f28101290 2024-09-12T03:04:06.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.264+0000 7f0f2d6d9640 1 -- 172.21.15.116:0/1106884501 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0f28111e70 con 0x7f0f28108160 2024-09-12T03:04:06.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.264+0000 7f0f2d6d9640 1 -- 172.21.15.116:0/1106884501 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0f28112170 con 0x7f0f281008e0 2024-09-12T03:04:06.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.264+0000 7f0f26ffd640 1 --2- 172.21.15.116:0/1106884501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f281008e0 0x7f0f28198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:06.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.264+0000 7f0f267fc640 1 -- 172.21.15.116:0/1106884501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f281008e0 msgr2=0x7f0f28198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:06.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.264+0000 7f0f267fc640 1 --2- 172.21.15.116:0/1106884501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f281008e0 0x7f0f28198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:06.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.264+0000 7f0f267fc640 1 -- 172.21.15.116:0/1106884501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f28108160 msgr2=0x7f0f2819ce20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T03:04:06.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.264+0000 7f0f267fc640 1 --2- 172.21.15.116:0/1106884501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f28108160 0x7f0f2819ce20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:06.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.264+0000 7f0f267fc640 1 -- 172.21.15.116:0/1106884501 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2819d500 con 0x7f0f28101290 2024-09-12T03:04:06.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.265+0000 7f0f267fc640 1 --2- 172.21.15.116:0/1106884501 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f28101290 0x7f0f28198a90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c005940 tx=0x7f0f1c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:06.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.265+0000 7f0f26ffd640 1 --2- 172.21.15.116:0/1106884501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f281008e0 0x7f0f28198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:06.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.265+0000 7f0f0ffff640 1 -- 172.21.15.116:0/1106884501 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c03f030 con 0x7f0f28101290 2024-09-12T03:04:06.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.265+0000 7f0f2d6d9640 1 -- 172.21.15.116:0/1106884501 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f2819d790 con 0x7f0f28101290 2024-09-12T03:04:06.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.265+0000 7f0f2d6d9640 1 -- 172.21.15.116:0/1106884501 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f281a50d0 con 0x7f0f28101290 2024-09-12T03:04:06.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.265+0000 7f0f0ffff640 1 -- 172.21.15.116:0/1106884501 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f1c003ce0 con 0x7f0f28101290 2024-09-12T03:04:06.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.265+0000 7f0f0ffff640 1 -- 172.21.15.116:0/1106884501 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c035070 con 0x7f0f28101290 2024-09-12T03:04:06.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.267+0000 7f0f2d6d9640 1 -- 172.21.15.116:0/1106884501 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f14005290 con 0x7f0f28101290 2024-09-12T03:04:06.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.268+0000 7f0f0ffff640 1 -- 172.21.15.116:0/1106884501 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f0f1c042030 con 0x7f0f28101290 2024-09-12T03:04:06.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.269+0000 7f0f0ffff640 1 --2- 172.21.15.116:0/1106884501 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0f100a2c00 0x7f0f100a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:06.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.270+0000 7f0f26ffd640 1 --2- 172.21.15.116:0/1106884501 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0f100a2c00 0x7f0f100a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:06.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.270+0000 7f0f0ffff640 1 -- 172.21.15.116:0/1106884501 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f0f1c0eef30 con 0x7f0f28101290 2024-09-12T03:04:06.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.270+0000 7f0f26ffd640 1 --2- 172.21.15.116:0/1106884501 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0f100a2c00 0x7f0f100a50c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0f180058c0 tx=0x7f0f18005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:06.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.277+0000 7f0f0ffff640 1 -- 172.21.15.116:0/1106884501 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0f1c0bbec0 con 0x7f0f28101290 2024-09-12T03:04:06.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.513+0000 7f0f2d6d9640 1 -- 172.21.15.116:0/1106884501 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f14005580 con 0x7f0f28101290 2024-09-12T03:04:06.514 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:06 smithi116 ceph-mon[31410]: pgmap v371: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-12T03:04:06.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.514+0000 7f0f0ffff640 1 -- 172.21.15.116:0/1106884501 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f0f1c0bb7c0 con 0x7f0f28101290 2024-09-12T03:04:06.515 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:04:06.515 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:04:06.515 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:04:06.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.517+0000 7f0f2d6d9640 1 -- 172.21.15.116:0/1106884501 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0f100a2c00 msgr2=0x7f0f100a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:06.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.517+0000 7f0f2d6d9640 1 --2- 172.21.15.116:0/1106884501 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0f100a2c00 0x7f0f100a50c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0f180058c0 tx=0x7f0f18005850 comp rx=0 tx=0).stop 2024-09-12T03:04:06.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.518+0000 7f0f2d6d9640 1 -- 172.21.15.116:0/1106884501 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f28101290 msgr2=0x7f0f28198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:06.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.518+0000 7f0f2d6d9640 1 --2- 172.21.15.116:0/1106884501 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f28101290 0x7f0f28198a90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c005940 tx=0x7f0f1c0042f0 comp rx=0 tx=0).stop 2024-09-12T03:04:06.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.518+0000 7f0f2d6d9640 1 -- 172.21.15.116:0/1106884501 shutdown_connections 2024-09-12T03:04:06.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.518+0000 7f0f2d6d9640 1 --2- 172.21.15.116:0/1106884501 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0f100a2c00 0x7f0f100a50c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:06.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.518+0000 7f0f2d6d9640 1 --2- 172.21.15.116:0/1106884501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f28108160 0x7f0f2819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:06.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.518+0000 7f0f2d6d9640 1 --2- 172.21.15.116:0/1106884501 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f28101290 0x7f0f28198a90 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:06.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.518+0000 7f0f2d6d9640 1 --2- 172.21.15.116:0/1106884501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f281008e0 0x7f0f28198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:06.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.518+0000 7f0f2d6d9640 1 -- 172.21.15.116:0/1106884501 >> 172.21.15.116:0/1106884501 conn(0x7f0f28077210 msgr2=0x7f0f28105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:06.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.518+0000 7f0f2d6d9640 1 -- 172.21.15.116:0/1106884501 shutdown_connections 2024-09-12T03:04:06.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:06.518+0000 7f0f2d6d9640 1 -- 172.21.15.116:0/1106884501 wait complete. 2024-09-12T03:04:06.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:06 smithi183 ceph-mon[35535]: pgmap v371: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-12T03:04:06.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:06 smithi177 ceph-mon[35305]: pgmap v371: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-12T03:04:07.374 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id fc4784bc --release --await' 2024-09-12T03:04:07.374 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id fc4784bc --release --await 2024-09-12T03:04:07.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:07 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1106884501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:04:07.696 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:07 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1106884501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:04:07.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:07 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1106884501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:04:08.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.226+0000 7fe60222f640 1 -- 172.21.15.116:0/2634346580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5fc1082a0 msgr2=0x7fe5fc10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:08.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.226+0000 7fe60222f640 1 --2- 172.21.15.116:0/2634346580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5fc1082a0 0x7fe5fc10ec40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009cd0 tx=0x7fe5f402a8e0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.226+0000 7fe60222f640 1 -- 172.21.15.116:0/2634346580 shutdown_connections 2024-09-12T03:04:08.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.226+0000 7fe60222f640 1 --2- 172.21.15.116:0/2634346580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5fc1082a0 0x7fe5fc10ec40 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.226+0000 7fe60222f640 1 --2- 172.21.15.116:0/2634346580 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5fc1036e0 0x7fe5fc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.226+0000 7fe60222f640 1 --2- 172.21.15.116:0/2634346580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5fc102d30 0x7fe5fc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.226+0000 7fe60222f640 1 -- 172.21.15.116:0/2634346580 >> 172.21.15.116:0/2634346580 conn(0x7fe5fc0fc7d0 msgr2=0x7fe5fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:08.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.226+0000 7fe60222f640 1 -- 172.21.15.116:0/2634346580 shutdown_connections 2024-09-12T03:04:08.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.226+0000 7fe60222f640 1 -- 172.21.15.116:0/2634346580 wait complete. 2024-09-12T03:04:08.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.227+0000 7fe60222f640 1 Processor -- start 2024-09-12T03:04:08.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.227+0000 7fe60222f640 1 -- start start 2024-09-12T03:04:08.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.228+0000 7fe60222f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5fc102d30 0x7fe5fc198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:08.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.228+0000 7fe60222f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5fc1036e0 0x7fe5fc198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:08.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.228+0000 7fe60222f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5fc1082a0 0x7fe5fc19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:08.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.228+0000 7fe5faffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5fc1036e0 0x7fe5fc198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:08.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.228+0000 7fe60222f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc1120a0 con 0x7fe5fc1036e0 2024-09-12T03:04:08.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.228+0000 7fe60222f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc111f20 con 0x7fe5fc1082a0 2024-09-12T03:04:08.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.228+0000 7fe5faffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5fc1036e0 0x7fe5fc198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45876/0 (socket says 172.21.15.116:45876) 2024-09-12T03:04:08.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.228+0000 7fe60222f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc112220 con 0x7fe5fc102d30 2024-09-12T03:04:08.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.228+0000 7fe5faffd640 1 -- 172.21.15.116:0/1714187285 learned_addr learned my addr 172.21.15.116:0/1714187285 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:08.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.228+0000 7fe5fb7fe640 1 --2- 172.21.15.116:0/1714187285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5fc102d30 0x7fe5fc198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:08.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.228+0000 7fe5fbfff640 1 --2- 172.21.15.116:0/1714187285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5fc1082a0 0x7fe5fc19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:08.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.228+0000 7fe5faffd640 1 -- 172.21.15.116:0/1714187285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5fc102d30 msgr2=0x7fe5fc198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:08.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.228+0000 7fe5faffd640 1 --2- 172.21.15.116:0/1714187285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5fc102d30 0x7fe5fc198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.228+0000 7fe5faffd640 1 -- 172.21.15.116:0/1714187285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5fc1082a0 msgr2=0x7fe5fc19cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:08.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.228+0000 7fe5faffd640 1 --2- 172.21.15.116:0/1714187285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5fc1082a0 0x7fe5fc19cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.228+0000 7fe5faffd640 1 -- 172.21.15.116:0/1714187285 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5fc19d4d0 con 0x7fe5fc1036e0 2024-09-12T03:04:08.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.228+0000 7fe5fbfff640 1 --2- 172.21.15.116:0/1714187285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5fc1082a0 0x7fe5fc19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:08.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.229+0000 7fe5faffd640 1 --2- 172.21.15.116:0/1714187285 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5fc1036e0 0x7fe5fc198a60 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000d950 tx=0x7fe5f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:08.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.229+0000 7fe5f8ff9640 1 -- 172.21.15.116:0/1714187285 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f001c070 con 0x7fe5fc1036e0 2024-09-12T03:04:08.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.229+0000 7fe60222f640 1 -- 172.21.15.116:0/1714187285 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5fc19d7c0 con 0x7fe5fc1036e0 2024-09-12T03:04:08.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.229+0000 7fe5f8ff9640 1 -- 172.21.15.116:0/1714187285 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5f0009d00 con 0x7fe5fc1036e0 2024-09-12T03:04:08.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.229+0000 7fe5f8ff9640 1 -- 172.21.15.116:0/1714187285 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0020d60 con 0x7fe5fc1036e0 2024-09-12T03:04:08.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.229+0000 7fe60222f640 1 -- 172.21.15.116:0/1714187285 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5fc1008c0 con 0x7fe5fc1036e0 2024-09-12T03:04:08.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.232+0000 7fe5f8ff9640 1 -- 172.21.15.116:0/1714187285 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe5f0009820 con 0x7fe5fc1036e0 2024-09-12T03:04:08.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.233+0000 7fe604608740 1 Processor -- start 2024-09-12T03:04:08.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.233+0000 7fe604608740 1 -- start start 2024-09-12T03:04:08.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.233+0000 7fe5f8ff9640 1 --2- 172.21.15.116:0/1714187285 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe5e40a2c00 0x7fe5e40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:08.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.233+0000 7fe604608740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ae79cb060 0x557ae79cb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:08.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.233+0000 7fe604608740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae79cb980 0x557ae79d5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:08.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.233+0000 7fe604608740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ae79d6240 0x557ae79d8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:08.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.233+0000 7fe604608740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557ae7930620 con 0x557ae79cb980 2024-09-12T03:04:08.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.233+0000 7fe604608740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557ae78d7e50 con 0x557ae79cb060 2024-09-12T03:04:08.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.233+0000 7fe604608740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557ae792c5e0 con 0x557ae79d6240 2024-09-12T03:04:08.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.233+0000 7fe5f8ff9640 1 -- 172.21.15.116:0/1714187285 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe5f00d6f00 con 0x7fe5fc1036e0 2024-09-12T03:04:08.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.234+0000 7fe5fbfff640 1 --2- 172.21.15.116:0/1714187285 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe5e40a2c00 0x7fe5e40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:08.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.234+0000 7fe5fbfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ae79d6240 0x557ae79d8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:08.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.234+0000 7fe5faffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae79cb980 0x557ae79d5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:08.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.234+0000 7fe5faffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae79cb980 0x557ae79d5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45888/0 (socket says 172.21.15.116:45888) 2024-09-12T03:04:08.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.234+0000 7fe5faffd640 1 -- 172.21.15.116:0/1640274961 learned_addr learned my addr 172.21.15.116:0/1640274961 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:08.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.234+0000 7fe5fb7fe640 1 --2- 172.21.15.116:0/1640274961 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ae79cb060 0x557ae79cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:08.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.234+0000 7fe5faffd640 1 -- 172.21.15.116:0/1640274961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ae79d6240 msgr2=0x557ae79d8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:08.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.234+0000 7fe5faffd640 1 --2- 172.21.15.116:0/1640274961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ae79d6240 0x557ae79d8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.234+0000 7fe5faffd640 1 -- 172.21.15.116:0/1640274961 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ae79cb060 msgr2=0x557ae79cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:08.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.234+0000 7fe5faffd640 1 --2- 172.21.15.116:0/1640274961 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ae79cb060 0x557ae79cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.234+0000 7fe5faffd640 1 -- 172.21.15.116:0/1640274961 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ae78c8d40 con 0x557ae79cb980 2024-09-12T03:04:08.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.234+0000 7fe5faffd640 1 --2- 172.21.15.116:0/1640274961 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae79cb980 0x557ae79d5d00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00a3490 tx=0x7fe5f000bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec2fb284ae61a41 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:08.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.235+0000 7fe5e1ffb640 1 -- 172.21.15.116:0/1640274961 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f00a3e90 con 0x557ae79cb980 2024-09-12T03:04:08.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.235+0000 7fe5e1ffb640 1 -- 172.21.15.116:0/1640274961 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5f00a3790 con 0x557ae79cb980 2024-09-12T03:04:08.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.235+0000 7fe5e1ffb640 1 -- 172.21.15.116:0/1640274961 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0020400 con 0x557ae79cb980 2024-09-12T03:04:08.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.236+0000 7fe604608740 1 -- 172.21.15.116:0/1640274961 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae79cb980 msgr2=0x557ae79d5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:08.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.236+0000 7fe5fbfff640 1 --2- 172.21.15.116:0/1714187285 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe5e40a2c00 0x7fe5e40a50c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x557ae79c8970 tx=0x7fe5f4005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:08.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.236+0000 7fe604608740 1 --2- 172.21.15.116:0/1640274961 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae79cb980 0x557ae79d5d00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00a3490 tx=0x7fe5f000bf70 comp rx=0 tx=0).stop 2024-09-12T03:04:08.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.236+0000 7fe604608740 1 -- 172.21.15.116:0/1640274961 shutdown_connections 2024-09-12T03:04:08.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.236+0000 7fe604608740 1 --2- 172.21.15.116:0/1640274961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ae79d6240 0x557ae79d8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.236+0000 7fe604608740 1 --2- 172.21.15.116:0/1640274961 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae79cb980 0x557ae79d5d00 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.236+0000 7fe604608740 1 --2- 172.21.15.116:0/1640274961 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ae79cb060 0x557ae79cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.236+0000 7fe604608740 1 -- 172.21.15.116:0/1640274961 >> 172.21.15.116:0/1640274961 conn(0x557ae79c75f0 msgr2=0x557ae79c79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:08.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.236+0000 7fe604608740 1 -- 172.21.15.116:0/1640274961 shutdown_connections 2024-09-12T03:04:08.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.236+0000 7fe604608740 1 -- 172.21.15.116:0/1640274961 wait complete. 2024-09-12T03:04:08.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.236+0000 7fe604608740 1 -- start start 2024-09-12T03:04:08.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.236+0000 7fe604608740 1 Processor -- start 2024-09-12T03:04:08.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.237+0000 7fe604608740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ae79cb060 0x557ae79d3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:08.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.237+0000 7fe604608740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae79cb980 0x557ae79d3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:08.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.237+0000 7fe5faffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae79cb980 0x557ae79d3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:08.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.237+0000 7fe5faffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae79cb980 0x557ae79d3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45896/0 (socket says 172.21.15.116:45896) 2024-09-12T03:04:08.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.237+0000 7fe5faffd640 1 -- 172.21.15.116:0/1897602281 learned_addr learned my addr 172.21.15.116:0/1897602281 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:08.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.237+0000 7fe604608740 1 --2- 172.21.15.116:0/1897602281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ae79d6240 0x557ae79d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:08.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.237+0000 7fe604608740 1 -- 172.21.15.116:0/1897602281 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557ae79d8e10 con 0x557ae79cb980 2024-09-12T03:04:08.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.237+0000 7fe5fb7fe640 1 --2- 172.21.15.116:0/1897602281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ae79cb060 0x557ae79d3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:08.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.237+0000 7fe604608740 1 -- 172.21.15.116:0/1897602281 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557ae78b8f10 con 0x557ae79cb060 2024-09-12T03:04:08.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.237+0000 7fe604608740 1 -- 172.21.15.116:0/1897602281 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557ae79d8f90 con 0x557ae79d6240 2024-09-12T03:04:08.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.238+0000 7fe5fbfff640 1 --2- 172.21.15.116:0/1897602281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ae79d6240 0x557ae79d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:08.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.238+0000 7fe5fb7fe640 1 -- 172.21.15.116:0/1897602281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ae79d6240 msgr2=0x557ae79d3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:08.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.238+0000 7fe5fb7fe640 1 --2- 172.21.15.116:0/1897602281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ae79d6240 0x557ae79d3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.238+0000 7fe5fb7fe640 1 -- 172.21.15.116:0/1897602281 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae79cb980 msgr2=0x557ae79d3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:08.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.238+0000 7fe5fb7fe640 1 --2- 172.21.15.116:0/1897602281 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae79cb980 0x557ae79d3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.238+0000 7fe5fb7fe640 1 -- 172.21.15.116:0/1897602281 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ae763f580 con 0x557ae79cb060 2024-09-12T03:04:08.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.238+0000 7fe5faffd640 1 --2- 172.21.15.116:0/1897602281 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae79cb980 0x557ae79d3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:04:08.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.238+0000 7fe5fb7fe640 1 --2- 172.21.15.116:0/1897602281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ae79cb060 0x557ae79d3200 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00b8f0 tx=0x7fe5ec00bcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d881ecff11f2122 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:08.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.238+0000 7fe604608740 10 client.0 _ll_register_callbacks cb 0x557ae78dcdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:08.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.238+0000 7fe604608740 1 -- 172.21.15.116:0/1897602281 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5f00a3790 con 0x557ae79cb060 2024-09-12T03:04:08.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.238+0000 7fe5daffd640 10 client.24986 ms_handle_connect on v2:172.21.15.177:3300/0 2024-09-12T03:04:08.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.238+0000 7fe5daffd640 1 -- 172.21.15.116:0/1897602281 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec01b440 con 0x557ae79cb060 2024-09-12T03:04:08.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.238+0000 7fe5daffd640 1 -- 172.21.15.116:0/1897602281 <== mon.2 v2:172.21.15.177:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe5ec01b5e0 con 0x557ae79cb060 2024-09-12T03:04:08.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.238+0000 7fe5daffd640 1 -- 172.21.15.116:0/1897602281 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe5ec01b820 con 0x557ae79cb060 2024-09-12T03:04:08.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.238+0000 7fe604608740 10 client.24986 fetch_fsmap learned FSMap version 25 2024-09-12T03:04:08.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.238+0000 7fe604608740 1 -- 172.21.15.116:0/1897602281 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({fsmap=25}) -- 0x557ae7709960 con 0x557ae79cb060 2024-09-12T03:04:08.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.239+0000 7fe5daffd640 1 -- 172.21.15.116:0/1897602281 <== mon.2 v2:172.21.15.177:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe5ec019020 con 0x557ae79cb060 2024-09-12T03:04:08.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.239+0000 7fe604608740 10 client.24986 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:04:08.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.239+0000 7fe604608740 10 client.24986 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:04:08.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.239+0000 7fe604608740 20 client.24986 populate_metadata read hostname 'smithi116' 2024-09-12T03:04:08.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.239+0000 7fe604608740 1 --2- 172.21.15.116:0/1897602281 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557ae79e5c90 0x557ae79e80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:08.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.239+0000 7fe604608740 4 client.24986 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:04:08.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.239+0000 7fe604608740 1 -- 172.21.15.116:0/1897602281 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe5f00a3790 con 0x557ae79e5c90 2024-09-12T03:04:08.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.239+0000 7fe5faffd640 1 --2- 172.21.15.116:0/1897602281 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557ae79e5c90 0x557ae79e80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:08.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.239+0000 7fe5faffd640 1 --2- 172.21.15.116:0/1897602281 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557ae79e5c90 0x557ae79e80a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f194d817ff9e88e0 server_cookie=6fb70bdef5fe464c in_seq=0 out_seq=0 2024-09-12T03:04:08.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.239+0000 7fe5daffd640 10 client.24986 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:08.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.240+0000 7fe5daffd640 1 -- 172.21.15.116:0/1897602281 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fe5f00a3790 con 0x557ae79e5c90 2024-09-12T03:04:08.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.240+0000 7fe5daffd640 10 client.24986 handle_command_reply: tid=0 2024-09-12T03:04:08.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.240+0000 7fe604608740 1 client.24986 shutdown 2024-09-12T03:04:08.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.240+0000 7fe604608740 1 -- 172.21.15.116:0/1897602281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ae79cb060 msgr2=0x557ae79d3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:08.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.240+0000 7fe604608740 1 --2- 172.21.15.116:0/1897602281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ae79cb060 0x557ae79d3200 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00b8f0 tx=0x7fe5ec00bcf0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.240+0000 7fe604608740 1 -- 172.21.15.116:0/1897602281 shutdown_connections 2024-09-12T03:04:08.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.240+0000 7fe604608740 1 --2- 172.21.15.116:0/1897602281 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557ae79e5c90 0x557ae79e80a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.240+0000 7fe604608740 1 --2- 172.21.15.116:0/1897602281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ae79d6240 0x557ae79d3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.240+0000 7fe5daffd640 0 client.24986 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:08.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.241+0000 7fe604608740 1 --2- 172.21.15.116:0/1897602281 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae79cb980 0x557ae79d3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.241+0000 7fe604608740 1 --2- 172.21.15.116:0/1897602281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ae79cb060 0x557ae79d3200 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.241+0000 7fe604608740 1 -- 172.21.15.116:0/1897602281 >> 172.21.15.116:0/1897602281 conn(0x557ae79c75f0 msgr2=0x557ae79cd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:08.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.241+0000 7fe604608740 1 -- 172.21.15.116:0/1897602281 shutdown_connections 2024-09-12T03:04:08.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.241+0000 7fe604608740 1 -- 172.21.15.116:0/1897602281 wait complete. 2024-09-12T03:04:08.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.241+0000 7fe604608740 20 client.24986 trim_cache size 0 max 16384 2024-09-12T03:04:08.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.273+0000 7fe604608740 1 Processor -- start 2024-09-12T03:04:08.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.273+0000 7fe604608740 1 -- start start 2024-09-12T03:04:08.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.274+0000 7fe604608740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae79cb060 0x557ae79cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:08.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.274+0000 7fe604608740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ae79cb980 0x557ae79d5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:08.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.274+0000 7fe604608740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ae79d6240 0x557ae79ed9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:08.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.274+0000 7fe5fb7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae79cb060 0x557ae79cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:08.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.274+0000 7fe5fb7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae79cb060 0x557ae79cdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45898/0 (socket says 172.21.15.116:45898) 2024-09-12T03:04:08.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.274+0000 7fe5fb7fe640 1 -- 172.21.15.116:0/2957045816 learned_addr learned my addr 172.21.15.116:0/2957045816 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:08.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.274+0000 7fe604608740 1 -- 172.21.15.116:0/2957045816 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc1adbd0 con 0x557ae79cb060 2024-09-12T03:04:08.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.274+0000 7fe604608740 1 -- 172.21.15.116:0/2957045816 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc1ab4a0 con 0x557ae79cb980 2024-09-12T03:04:08.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.274+0000 7fe604608740 1 -- 172.21.15.116:0/2957045816 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc1a92f0 con 0x557ae79d6240 2024-09-12T03:04:08.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.274+0000 7fe5faffd640 1 --2- 172.21.15.116:0/2957045816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ae79cb980 0x557ae79d5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:08.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.274+0000 7fe5fbfff640 1 --2- 172.21.15.116:0/2957045816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ae79d6240 0x557ae79ed9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:08.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.274+0000 7fe5fb7fe640 1 -- 172.21.15.116:0/2957045816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ae79d6240 msgr2=0x557ae79ed9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:08.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.274+0000 7fe5fb7fe640 1 --2- 172.21.15.116:0/2957045816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ae79d6240 0x557ae79ed9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.274+0000 7fe5fb7fe640 1 -- 172.21.15.116:0/2957045816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ae79cb980 msgr2=0x557ae79d5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:08.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.274+0000 7fe5fb7fe640 1 --2- 172.21.15.116:0/2957045816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ae79cb980 0x557ae79d5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.274+0000 7fe5fb7fe640 1 -- 172.21.15.116:0/2957045816 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ae79d21b0 con 0x557ae79cb060 2024-09-12T03:04:08.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.275+0000 7fe5fb7fe640 1 --2- 172.21.15.116:0/2957045816 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae79cb060 0x557ae79cdac0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec0039c0 tx=0x7fe5ec015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9702d51f19e2de67 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:08.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.275+0000 7fe5e17fa640 1 -- 172.21.15.116:0/2957045816 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec01b440 con 0x557ae79cb060 2024-09-12T03:04:08.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.275+0000 7fe5e17fa640 1 -- 172.21.15.116:0/2957045816 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5ec01b5e0 con 0x557ae79cb060 2024-09-12T03:04:08.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.275+0000 7fe5e17fa640 1 -- 172.21.15.116:0/2957045816 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec025420 con 0x557ae79cb060 2024-09-12T03:04:08.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.276+0000 7fe604608740 1 -- 172.21.15.116:0/2957045816 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae79cb060 msgr2=0x557ae79cdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:08.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.276+0000 7fe604608740 1 --2- 172.21.15.116:0/2957045816 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae79cb060 0x557ae79cdac0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec0039c0 tx=0x7fe5ec015000 comp rx=0 tx=0).stop 2024-09-12T03:04:08.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.276+0000 7fe604608740 1 -- 172.21.15.116:0/2957045816 shutdown_connections 2024-09-12T03:04:08.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.276+0000 7fe604608740 1 --2- 172.21.15.116:0/2957045816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ae79d6240 0x557ae79ed9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.276+0000 7fe604608740 1 --2- 172.21.15.116:0/2957045816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ae79cb980 0x557ae79d5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.276+0000 7fe604608740 1 --2- 172.21.15.116:0/2957045816 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae79cb060 0x557ae79cdac0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.276+0000 7fe604608740 1 -- 172.21.15.116:0/2957045816 >> 172.21.15.116:0/2957045816 conn(0x557ae79c75f0 msgr2=0x557ae79d4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:08.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.277+0000 7fe604608740 1 -- 172.21.15.116:0/2957045816 shutdown_connections 2024-09-12T03:04:08.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.277+0000 7fe604608740 1 -- 172.21.15.116:0/2957045816 wait complete. 2024-09-12T03:04:08.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.277+0000 7fe604608740 1 -- start start 2024-09-12T03:04:08.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.278+0000 7fe604608740 1 Processor -- start 2024-09-12T03:04:08.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.278+0000 7fe604608740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ae79cb060 0x557ae79de920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:08.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.278+0000 7fe604608740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ae79cb980 0x557ae79d9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:08.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.278+0000 7fe604608740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae79d6240 0x557ae79da1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:08.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.279+0000 7fe5fb7fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ae79cb060 0x557ae79de920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:08.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.279+0000 7fe5fb7fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ae79cb060 0x557ae79de920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.116:60492/0 (socket says 172.21.15.116:60492) 2024-09-12T03:04:08.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.279+0000 7fe5fbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae79d6240 0x557ae79da1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:08.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.279+0000 7fe5faffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ae79cb980 0x557ae79d9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:08.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.279+0000 7fe5faffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ae79cb980 0x557ae79d9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.116:59066/0 (socket says 172.21.15.116:59066) 2024-09-12T03:04:08.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.279+0000 7fe5fbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae79d6240 0x557ae79da1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45914/0 (socket says 172.21.15.116:45914) 2024-09-12T03:04:08.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.279+0000 7fe5fb7fe640 1 -- 172.21.15.116:0/499282219 learned_addr learned my addr 172.21.15.116:0/499282219 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:08.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.279+0000 7fe604608740 1 -- 172.21.15.116:0/499282219 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc1a9960 con 0x557ae79d6240 2024-09-12T03:04:08.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.279+0000 7fe604608740 1 -- 172.21.15.116:0/499282219 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc1aadd0 con 0x557ae79cb060 2024-09-12T03:04:08.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.279+0000 7fe604608740 1 -- 172.21.15.116:0/499282219 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc1aa030 con 0x557ae79cb980 2024-09-12T03:04:08.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.280+0000 7fe5faffd640 1 -- 172.21.15.116:0/499282219 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ae79cb060 msgr2=0x557ae79de920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:08.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.280+0000 7fe5faffd640 1 --2- 172.21.15.116:0/499282219 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ae79cb060 0x557ae79de920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.280+0000 7fe5faffd640 1 -- 172.21.15.116:0/499282219 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae79d6240 msgr2=0x557ae79da1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:08.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.280+0000 7fe5faffd640 1 --2- 172.21.15.116:0/499282219 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae79d6240 0x557ae79da1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.280+0000 7fe5faffd640 1 -- 172.21.15.116:0/499282219 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ae79daa10 con 0x557ae79cb980 2024-09-12T03:04:08.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.280+0000 7fe5fbfff640 1 --2- 172.21.15.116:0/499282219 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae79d6240 0x557ae79da1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:04:08.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.280+0000 7fe604608740 10 client.0 _ll_register_callbacks cb 0x7fe5ec004480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:08.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.280+0000 7fe5daffd640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T03:04:08.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.280+0000 7fe5faffd640 1 --2- 172.21.15.116:0/499282219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ae79cb980 0x557ae79d9ca0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00d4da0 tx=0x7fe5f00d6040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ebd753115bcf877 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:08.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.280+0000 7fe604608740 1 -- 172.21.15.116:0/499282219 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5ec01b5e0 con 0x557ae79cb980 2024-09-12T03:04:08.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.280+0000 7fe5daffd640 1 -- 172.21.15.116:0/499282219 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f00a3e90 con 0x557ae79cb980 2024-09-12T03:04:08.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.281+0000 7fe5daffd640 1 -- 172.21.15.116:0/499282219 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe5f0020400 con 0x557ae79cb980 2024-09-12T03:04:08.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.281+0000 7fe5daffd640 1 -- 172.21.15.116:0/499282219 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe5f0009b10 con 0x557ae79cb980 2024-09-12T03:04:08.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.281+0000 7fe604608740 10 client.25021 fetch_fsmap learned FSMap version 25 2024-09-12T03:04:08.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.281+0000 7fe604608740 1 -- 172.21.15.116:0/499282219 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({fsmap=25}) -- 0x557ae7a50700 con 0x557ae79cb980 2024-09-12T03:04:08.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.281+0000 7fe5daffd640 1 -- 172.21.15.116:0/499282219 <== mon.1 v2:172.21.15.183:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe5f009fe00 con 0x557ae79cb980 2024-09-12T03:04:08.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.282+0000 7fe604608740 10 client.25021 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:04:08.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.282+0000 7fe604608740 10 client.25021 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:04:08.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.282+0000 7fe604608740 20 client.25021 populate_metadata read hostname 'smithi116' 2024-09-12T03:04:08.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.282+0000 7fe604608740 1 --2- 172.21.15.116:0/499282219 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557ae79e5c90 0x557ae7a52ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:08.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.282+0000 7fe604608740 4 client.25021 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "fc4784bc", "release": true, "await": true}] 2024-09-12T03:04:08.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.282+0000 7fe604608740 1 -- 172.21.15.116:0/499282219 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "quiesce db", "set_id": "fc4784bc", "release": true, "await": true}) -- 0x7fe5ec01b5e0 con 0x557ae79e5c90 2024-09-12T03:04:08.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.282+0000 7fe5fb7fe640 1 --2- 172.21.15.116:0/499282219 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557ae79e5c90 0x557ae7a52ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:08.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.282+0000 7fe5fb7fe640 1 --2- 172.21.15.116:0/499282219 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557ae79e5c90 0x557ae7a52ab0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbda8dcd23c0fa61 server_cookie=7c205d8b713b3464 in_seq=0 out_seq=0 2024-09-12T03:04:08.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.283+0000 7fe5daffd640 10 client.25021 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:08.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.284+0000 7fe5daffd640 1 -- 172.21.15.116:0/499282219 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7fe5ec01b5e0 con 0x557ae79e5c90 2024-09-12T03:04:08.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.284+0000 7fe5daffd640 10 client.25021 handle_command_reply: tid=0 2024-09-12T03:04:08.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.284+0000 7fe604608740 1 client.25021 shutdown 2024-09-12T03:04:08.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.284+0000 7fe604608740 1 -- 172.21.15.116:0/499282219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ae79cb980 msgr2=0x557ae79d9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:08.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.284+0000 7fe604608740 1 --2- 172.21.15.116:0/499282219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ae79cb980 0x557ae79d9ca0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00d4da0 tx=0x7fe5f00d6040 comp rx=0 tx=0).stop 2024-09-12T03:04:08.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.285+0000 7fe604608740 1 -- 172.21.15.116:0/499282219 shutdown_connections 2024-09-12T03:04:08.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.285+0000 7fe604608740 1 --2- 172.21.15.116:0/499282219 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557ae79e5c90 0x557ae7a52ab0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.285+0000 7fe5daffd640 0 client.25021 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:08.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.285+0000 7fe604608740 1 --2- 172.21.15.116:0/499282219 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae79d6240 0x557ae79da1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.285+0000 7fe604608740 1 --2- 172.21.15.116:0/499282219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ae79cb980 0x557ae79d9ca0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.285+0000 7fe604608740 1 --2- 172.21.15.116:0/499282219 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ae79cb060 0x557ae79de920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.285+0000 7fe604608740 1 -- 172.21.15.116:0/499282219 >> 172.21.15.116:0/499282219 conn(0x557ae79c75f0 msgr2=0x557ae79c6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:08.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.285+0000 7fe604608740 1 -- 172.21.15.116:0/499282219 shutdown_connections 2024-09-12T03:04:08.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.285+0000 7fe604608740 1 -- 172.21.15.116:0/499282219 wait complete. 2024-09-12T03:04:08.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.285+0000 7fe604608740 20 client.25021 trim_cache size 0 max 16384 2024-09-12T03:04:08.287 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:04:08.287 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 25, 2024-09-12T03:04:08.287 INFO:teuthology.orchestra.run.smithi116.stdout: "leader": 24494, 2024-09-12T03:04:08.287 INFO:teuthology.orchestra.run.smithi116.stdout: "set_version": 4, 2024-09-12T03:04:08.287 INFO:teuthology.orchestra.run.smithi116.stdout: "sets": { 2024-09-12T03:04:08.288 INFO:teuthology.orchestra.run.smithi116.stdout: "fc4784bc": { 2024-09-12T03:04:08.288 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 4, 2024-09-12T03:04:08.288 INFO:teuthology.orchestra.run.smithi116.stdout: "age_ref": 0.0, 2024-09-12T03:04:08.288 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:04:08.288 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RELEASED", 2024-09-12T03:04:08.288 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:04:08.288 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:04:08.288 INFO:teuthology.orchestra.run.smithi116.stdout: "timeout": 90.0, 2024-09-12T03:04:08.288 INFO:teuthology.orchestra.run.smithi116.stdout: "expiration": 144.5, 2024-09-12T03:04:08.288 INFO:teuthology.orchestra.run.smithi116.stdout: "members": { 2024-09-12T03:04:08.288 INFO:teuthology.orchestra.run.smithi116.stdout: "file:/": { 2024-09-12T03:04:08.288 INFO:teuthology.orchestra.run.smithi116.stdout: "excluded": false, 2024-09-12T03:04:08.288 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:04:08.289 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RELEASED", 2024-09-12T03:04:08.289 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:04:08.289 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:04:08.289 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:04:08.289 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:04:08.289 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:04:08.289 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:04:08.289 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:04:08.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.287+0000 7fe60222f640 1 -- 172.21.15.116:0/1714187285 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe5e40a2c00 msgr2=0x7fe5e40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:08.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.287+0000 7fe60222f640 1 --2- 172.21.15.116:0/1714187285 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe5e40a2c00 0x7fe5e40a50c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x557ae79c8970 tx=0x7fe5f4005ef0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.287+0000 7fe60222f640 1 -- 172.21.15.116:0/1714187285 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5fc1036e0 msgr2=0x7fe5fc198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:08.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.287+0000 7fe60222f640 1 --2- 172.21.15.116:0/1714187285 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5fc1036e0 0x7fe5fc198a60 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000d950 tx=0x7fe5f000de10 comp rx=0 tx=0).stop 2024-09-12T03:04:08.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.287+0000 7fe60222f640 1 -- 172.21.15.116:0/1714187285 shutdown_connections 2024-09-12T03:04:08.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.287+0000 7fe60222f640 1 --2- 172.21.15.116:0/1714187285 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe5e40a2c00 0x7fe5e40a50c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.287+0000 7fe60222f640 1 --2- 172.21.15.116:0/1714187285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5fc1082a0 0x7fe5fc19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.287+0000 7fe60222f640 1 --2- 172.21.15.116:0/1714187285 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5fc1036e0 0x7fe5fc198a60 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.287+0000 7fe60222f640 1 --2- 172.21.15.116:0/1714187285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5fc102d30 0x7fe5fc198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.287+0000 7fe60222f640 1 -- 172.21.15.116:0/1714187285 >> 172.21.15.116:0/1714187285 conn(0x7fe5fc0fc7d0 msgr2=0x7fe5fc106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:08.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.288+0000 7fe60222f640 1 -- 172.21.15.116:0/1714187285 shutdown_connections 2024-09-12T03:04:08.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:08.288+0000 7fe60222f640 1 -- 172.21.15.116:0/1714187285 wait complete. 2024-09-12T03:04:08.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:08 smithi116 ceph-mon[31410]: pgmap v372: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-12T03:04:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:08 smithi183 ceph-mon[35535]: pgmap v372: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-12T03:04:08.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:08 smithi177 ceph-mon[35305]: pgmap v372: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-12T03:04:09.006 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'fc4784bc', total seconds elapsed: 28.1 2024-09-12T03:04:10.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:10 smithi116 ceph-mon[31410]: pgmap v373: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-12T03:04:10.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:10 smithi183 ceph-mon[35535]: pgmap v373: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-12T03:04:10.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:10 smithi177 ceph-mon[35305]: pgmap v373: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-12T03:04:11.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:11 smithi183 ceph-mon[35535]: pgmap v374: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-12T03:04:11.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:11 smithi177 ceph-mon[35305]: pgmap v374: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-12T03:04:11.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:11 smithi116 ceph-mon[31410]: pgmap v374: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-12T03:04:12.646 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:12 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:04:12.646 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:12 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:04:12.646 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:12 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:04:12.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:12 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:04:12.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:12 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:04:12.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:12 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:04:12.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:12 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:04:12.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:12 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:04:12.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:12 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:04:13.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:13 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:04:13.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:13 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:04:13.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:13 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:04:13.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:13 smithi183 ceph-mon[35535]: pgmap v375: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T03:04:13.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:13 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:04:13.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:13 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:04:13.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:13 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:04:13.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:13 smithi177 ceph-mon[35305]: pgmap v375: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T03:04:13.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:13 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:04:13.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:13 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:04:13.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:13 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:04:13.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:13 smithi116 ceph-mon[31410]: pgmap v375: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T03:04:16.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:16 smithi177 ceph-mon[35305]: pgmap v376: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T03:04:16.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:16 smithi116 ceph-mon[31410]: pgmap v376: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T03:04:16.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:16 smithi183 ceph-mon[35535]: pgmap v376: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T03:04:17.825 INFO:tasks.workunit.client.0.smithi116.stderr:Note: switching to '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'. 2024-09-12T03:04:17.825 INFO:tasks.workunit.client.0.smithi116.stderr: 2024-09-12T03:04:17.825 INFO:tasks.workunit.client.0.smithi116.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-12T03:04:17.825 INFO:tasks.workunit.client.0.smithi116.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-12T03:04:17.826 INFO:tasks.workunit.client.0.smithi116.stderr:state without impacting any branches by switching back to a branch. 2024-09-12T03:04:17.826 INFO:tasks.workunit.client.0.smithi116.stderr: 2024-09-12T03:04:17.826 INFO:tasks.workunit.client.0.smithi116.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-12T03:04:17.826 INFO:tasks.workunit.client.0.smithi116.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-12T03:04:17.826 INFO:tasks.workunit.client.0.smithi116.stderr: 2024-09-12T03:04:17.826 INFO:tasks.workunit.client.0.smithi116.stderr: git switch -c 2024-09-12T03:04:17.826 INFO:tasks.workunit.client.0.smithi116.stderr: 2024-09-12T03:04:17.826 INFO:tasks.workunit.client.0.smithi116.stderr:Or undo this operation with: 2024-09-12T03:04:17.826 INFO:tasks.workunit.client.0.smithi116.stderr: 2024-09-12T03:04:17.826 INFO:tasks.workunit.client.0.smithi116.stderr: git switch - 2024-09-12T03:04:17.826 INFO:tasks.workunit.client.0.smithi116.stderr: 2024-09-12T03:04:17.826 INFO:tasks.workunit.client.0.smithi116.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-12T03:04:17.827 INFO:tasks.workunit.client.0.smithi116.stderr: 2024-09-12T03:04:17.827 INFO:tasks.workunit.client.0.smithi116.stderr:HEAD is now at 086c737fa75 Merge pull request #59733 from zdover23/wip-doc-2024-09-11-backport-59635-to-squid 2024-09-12T03:04:17.831 DEBUG:teuthology.orchestra.run.smithi116:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-12T03:04:17.852 INFO:tasks.workunit.client.0.smithi116.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-12T03:04:17.854 INFO:tasks.workunit.client.0.smithi116.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-12T03:04:17.855 INFO:tasks.workunit.client.0.smithi116.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-12T03:04:18.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:18 smithi177 ceph-mon[35305]: pgmap v377: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-12T03:04:18.518 INFO:tasks.workunit.client.0.smithi116.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-12T03:04:18.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:18 smithi116 ceph-mon[31410]: pgmap v377: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-12T03:04:18.597 INFO:tasks.workunit.client.0.smithi116.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-12T03:04:18.637 INFO:tasks.workunit.client.0.smithi116.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-12T03:04:18.639 INFO:tasks.workunit.client.0.smithi116.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-12T03:04:18.639 INFO:tasks.workunit.client.0.smithi116.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-12T03:04:18.677 INFO:tasks.workunit.client.0.smithi116.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-12T03:04:18.681 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-12T03:04:18.681 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-12T03:04:18.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:18 smithi183 ceph-mon[35535]: pgmap v377: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-12T03:04:18.737 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-09-12T03:04:18.738 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-09-12T03:04:18.739 DEBUG:teuthology.orchestra.run.smithi116:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-09-12T03:04:18.799 INFO:tasks.workunit.client.0.smithi116.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-09-12T03:04:18.800 INFO:tasks.workunit.client.0.smithi116.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-09-12T03:04:18.800 INFO:tasks.workunit.client.0.smithi116.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-09-12T03:04:20.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:20 smithi177 ceph-mon[35305]: pgmap v378: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T03:04:20.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:20 smithi116 ceph-mon[31410]: pgmap v378: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T03:04:20.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:20 smithi183 ceph-mon[35535]: pgmap v378: 97 pgs: 97 active+clean; 660 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T03:04:22.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:22 smithi177 ceph-mon[35305]: pgmap v379: 97 pgs: 97 active+clean; 691 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-09-12T03:04:22.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:22 smithi116 ceph-mon[31410]: pgmap v379: 97 pgs: 97 active+clean; 691 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-09-12T03:04:22.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:22 smithi183 ceph-mon[35535]: pgmap v379: 97 pgs: 97 active+clean; 691 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-09-12T03:04:24.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:24 smithi177 ceph-mon[35305]: pgmap v380: 97 pgs: 97 active+clean; 691 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-09-12T03:04:24.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:24 smithi116 ceph-mon[31410]: pgmap v380: 97 pgs: 97 active+clean; 691 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-09-12T03:04:24.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:24 smithi183 ceph-mon[35535]: pgmap v380: 97 pgs: 97 active+clean; 691 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-09-12T03:04:26.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:26 smithi177 ceph-mon[35305]: pgmap v381: 97 pgs: 97 active+clean; 691 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-12T03:04:26.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:26 smithi116 ceph-mon[31410]: pgmap v381: 97 pgs: 97 active+clean; 691 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-12T03:04:26.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:26 smithi183 ceph-mon[35535]: pgmap v381: 97 pgs: 97 active+clean; 691 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-12T03:04:28.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:28 smithi177 ceph-mon[35305]: pgmap v382: 97 pgs: 97 active+clean; 748 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.9 KiB/s wr, 9 op/s 2024-09-12T03:04:28.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:28 smithi116 ceph-mon[31410]: pgmap v382: 97 pgs: 97 active+clean; 748 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.9 KiB/s wr, 9 op/s 2024-09-12T03:04:28.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:28 smithi183 ceph-mon[35535]: pgmap v382: 97 pgs: 97 active+clean; 748 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.9 KiB/s wr, 9 op/s 2024-09-12T03:04:29.043 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:04:29.293 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:29.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.825+0000 7f6ab59d6640 1 -- 172.21.15.116:0/2747127814 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ab0101fc0 msgr2=0x7f6ab0105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:29.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.825+0000 7f6ab59d6640 1 --2- 172.21.15.116:0/2747127814 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ab0101fc0 0x7f6ab0105710 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4009900 tx=0x7f6aa402a410 comp rx=0 tx=0).stop 2024-09-12T03:04:29.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.825+0000 7f6ab59d6640 1 -- 172.21.15.116:0/2747127814 shutdown_connections 2024-09-12T03:04:29.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.825+0000 7f6ab59d6640 1 --2- 172.21.15.116:0/2747127814 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ab0105e40 0x7f6ab010d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.825+0000 7f6ab59d6640 1 --2- 172.21.15.116:0/2747127814 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ab0101fc0 0x7f6ab0105710 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.825+0000 7f6ab59d6640 1 --2- 172.21.15.116:0/2747127814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ab0101610 0x7f6ab01019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.825+0000 7f6ab59d6640 1 -- 172.21.15.116:0/2747127814 >> 172.21.15.116:0/2747127814 conn(0x7f6ab00fc5c0 msgr2=0x7f6ab00fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:29.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.825+0000 7f6ab59d6640 1 -- 172.21.15.116:0/2747127814 shutdown_connections 2024-09-12T03:04:29.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.825+0000 7f6ab59d6640 1 -- 172.21.15.116:0/2747127814 wait complete. 2024-09-12T03:04:29.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.826+0000 7f6ab59d6640 1 Processor -- start 2024-09-12T03:04:29.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.826+0000 7f6ab59d6640 1 -- start start 2024-09-12T03:04:29.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.826+0000 7f6ab59d6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ab0101610 0x7f6ab0199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.826+0000 7f6ab59d6640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ab0101fc0 0x7f6ab0199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.826+0000 7f6ab59d6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ab0105e40 0x7f6ab01a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.826+0000 7f6ab59d6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab0110ea0 con 0x7f6ab0101610 2024-09-12T03:04:29.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.826+0000 7f6ab59d6640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab0110d20 con 0x7f6ab0101fc0 2024-09-12T03:04:29.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.826+0000 7f6ab59d6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab0111020 con 0x7f6ab0105e40 2024-09-12T03:04:29.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.826+0000 7f6aaeffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ab0101610 0x7f6ab0199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:29.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.826+0000 7f6aaeffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ab0101610 0x7f6ab0199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49674/0 (socket says 172.21.15.116:49674) 2024-09-12T03:04:29.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.826+0000 7f6aaeffd640 1 -- 172.21.15.116:0/1544405936 learned_addr learned my addr 172.21.15.116:0/1544405936 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:29.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.826+0000 7f6aae7fc640 1 --2- 172.21.15.116:0/1544405936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ab0101fc0 0x7f6ab0199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:29.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.827+0000 7f6aaeffd640 1 -- 172.21.15.116:0/1544405936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ab0105e40 msgr2=0x7f6ab01a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:29.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.827+0000 7f6aaf7fe640 1 --2- 172.21.15.116:0/1544405936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ab0105e40 0x7f6ab01a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:29.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.827+0000 7f6aaeffd640 1 --2- 172.21.15.116:0/1544405936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ab0105e40 0x7f6ab01a4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.827+0000 7f6aaeffd640 1 -- 172.21.15.116:0/1544405936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ab0101fc0 msgr2=0x7f6ab0199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:29.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.827+0000 7f6aaeffd640 1 --2- 172.21.15.116:0/1544405936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ab0101fc0 0x7f6ab0199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.827+0000 7f6aaeffd640 1 -- 172.21.15.116:0/1544405936 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab01a48d0 con 0x7f6ab0101610 2024-09-12T03:04:29.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.827+0000 7f6aae7fc640 1 --2- 172.21.15.116:0/1544405936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ab0101fc0 0x7f6ab0199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:04:29.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.827+0000 7f6aaeffd640 1 --2- 172.21.15.116:0/1544405936 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ab0101610 0x7f6ab0199310 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000d770 tx=0x7f6aa000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:29.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.827+0000 7f6ab49d4640 1 -- 172.21.15.116:0/1544405936 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0017070 con 0x7f6ab0101610 2024-09-12T03:04:29.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.827+0000 7f6ab49d4640 1 -- 172.21.15.116:0/1544405936 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6aa0005500 con 0x7f6ab0101610 2024-09-12T03:04:29.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.827+0000 7f6ab59d6640 1 -- 172.21.15.116:0/1544405936 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ab01a4aa0 con 0x7f6ab0101610 2024-09-12T03:04:29.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.827+0000 7f6ab49d4640 1 -- 172.21.15.116:0/1544405936 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0004e00 con 0x7f6ab0101610 2024-09-12T03:04:29.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.827+0000 7f6ab59d6640 1 -- 172.21.15.116:0/1544405936 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ab01a4f60 con 0x7f6ab0101610 2024-09-12T03:04:29.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.829+0000 7f6ab49d4640 1 -- 172.21.15.116:0/1544405936 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6aa0004fa0 con 0x7f6ab0101610 2024-09-12T03:04:29.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.830+0000 7f6ab7daf740 1 Processor -- start 2024-09-12T03:04:29.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.830+0000 7f6ab7daf740 1 -- start start 2024-09-12T03:04:29.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.830+0000 7f6ab7daf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5604689b74a0 0x5604689b7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.830+0000 7f6ab7daf740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604689b7dc0 0x5604689c2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.830+0000 7f6ab7daf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604689c2680 0x5604689c4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.830+0000 7f6ab7daf740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56046891d2e0 con 0x5604689b7dc0 2024-09-12T03:04:29.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.830+0000 7f6ab7daf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5604688c4ce0 con 0x5604689b74a0 2024-09-12T03:04:29.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.830+0000 7f6ab7daf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5604689192a0 con 0x5604689c2680 2024-09-12T03:04:29.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.830+0000 7f6aaeffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604689b7dc0 0x5604689c2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:29.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.830+0000 7f6ab49d4640 1 --2- 172.21.15.116:0/1544405936 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6a980a2b80 0x7f6a980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.830+0000 7f6aaeffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604689b7dc0 0x5604689c2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49690/0 (socket says 172.21.15.116:49690) 2024-09-12T03:04:29.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.830+0000 7f6aaeffd640 1 -- 172.21.15.116:0/2844941609 learned_addr learned my addr 172.21.15.116:0/2844941609 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:29.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.830+0000 7f6aae7fc640 1 --2- 172.21.15.116:0/2844941609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604689c2680 0x5604689c4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:29.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.830+0000 7f6aaf7fe640 1 --2- 172.21.15.116:0/2844941609 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5604689b74a0 0x5604689b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:29.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.830+0000 7f6aaf7fe640 1 --2- 172.21.15.116:0/1544405936 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6a980a2b80 0x7f6a980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:29.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.830+0000 7f6ab49d4640 1 -- 172.21.15.116:0/1544405936 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f6aa00ceb40 con 0x7f6ab0101610 2024-09-12T03:04:29.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.830+0000 7f6aaeffd640 1 -- 172.21.15.116:0/2844941609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604689c2680 msgr2=0x5604689c4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:29.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.830+0000 7f6aaeffd640 1 --2- 172.21.15.116:0/2844941609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604689c2680 0x5604689c4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.830+0000 7f6aaeffd640 1 -- 172.21.15.116:0/2844941609 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5604689b74a0 msgr2=0x5604689b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:29.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.830+0000 7f6aaeffd640 1 --2- 172.21.15.116:0/2844941609 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5604689b74a0 0x5604689b7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.830+0000 7f6aaeffd640 1 -- 172.21.15.116:0/2844941609 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604688a9440 con 0x5604689b7dc0 2024-09-12T03:04:29.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.831+0000 7f6aaeffd640 1 --2- 172.21.15.116:0/2844941609 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604689b7dc0 0x5604689c2140 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6aa009b360 tx=0x7f6aa0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea74fe2423e1a509 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:29.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.831+0000 7f6aaf7fe640 1 --2- 172.21.15.116:0/1544405936 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6a980a2b80 0x7f6a980a5040 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8012920 tx=0x7f6aa800a730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:29.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.831+0000 7f6a95ffb640 1 -- 172.21.15.116:0/2844941609 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0020070 con 0x5604689b7dc0 2024-09-12T03:04:29.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.831+0000 7f6a95ffb640 1 -- 172.21.15.116:0/2844941609 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6aa009b3d0 con 0x5604689b7dc0 2024-09-12T03:04:29.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.831+0000 7f6a95ffb640 1 -- 172.21.15.116:0/2844941609 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0097a40 con 0x5604689b7dc0 2024-09-12T03:04:29.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.831+0000 7f6ab7daf740 1 -- 172.21.15.116:0/2844941609 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604689b7dc0 msgr2=0x5604689c2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:29.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.831+0000 7f6ab7daf740 1 --2- 172.21.15.116:0/2844941609 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604689b7dc0 0x5604689c2140 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6aa009b360 tx=0x7f6aa0009c30 comp rx=0 tx=0).stop 2024-09-12T03:04:29.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.832+0000 7f6ab7daf740 1 -- 172.21.15.116:0/2844941609 shutdown_connections 2024-09-12T03:04:29.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.832+0000 7f6ab7daf740 1 --2- 172.21.15.116:0/2844941609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604689c2680 0x5604689c4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.832+0000 7f6ab7daf740 1 --2- 172.21.15.116:0/2844941609 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604689b7dc0 0x5604689c2140 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.832+0000 7f6ab7daf740 1 --2- 172.21.15.116:0/2844941609 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5604689b74a0 0x5604689b7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.832+0000 7f6ab7daf740 1 -- 172.21.15.116:0/2844941609 >> 172.21.15.116:0/2844941609 conn(0x5604689b3a30 msgr2=0x5604689b3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:29.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.832+0000 7f6ab7daf740 1 -- 172.21.15.116:0/2844941609 shutdown_connections 2024-09-12T03:04:29.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.832+0000 7f6ab7daf740 1 -- 172.21.15.116:0/2844941609 wait complete. 2024-09-12T03:04:29.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.832+0000 7f6ab7daf740 1 -- start start 2024-09-12T03:04:29.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.832+0000 7f6ab7daf740 1 Processor -- start 2024-09-12T03:04:29.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.832+0000 7f6ab7daf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5604689b74a0 0x5604689c2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.832+0000 7f6ab7daf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604689b7dc0 0x5604689bf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.832+0000 7f6ab7daf740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604689c2680 0x5604689bf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.832+0000 7f6ab7daf740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5604689c51f0 con 0x5604689c2680 2024-09-12T03:04:29.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.832+0000 7f6ab7daf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5604688d8050 con 0x5604689b74a0 2024-09-12T03:04:29.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.832+0000 7f6ab7daf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5604689c5370 con 0x5604689b7dc0 2024-09-12T03:04:29.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.832+0000 7f6ab7daf740 10 client.0 _ll_register_callbacks cb 0x560468918160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:29.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.833+0000 7f6aaf7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604689c2680 0x5604689bf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:29.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.833+0000 7f6aaf7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604689c2680 0x5604689bf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49706/0 (socket says 172.21.15.116:49706) 2024-09-12T03:04:29.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.833+0000 7f6aaf7fe640 1 -- 172.21.15.116:0/1252607021 learned_addr learned my addr 172.21.15.116:0/1252607021 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:29.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.833+0000 7f6aae7fc640 1 --2- 172.21.15.116:0/1252607021 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604689b7dc0 0x5604689bf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:29.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.833+0000 7f6aaeffd640 1 --2- 172.21.15.116:0/1252607021 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5604689b74a0 0x5604689c2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:29.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.833+0000 7f6aaf7fe640 1 -- 172.21.15.116:0/1252607021 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604689b7dc0 msgr2=0x5604689bf2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:29.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.833+0000 7f6aaf7fe640 1 --2- 172.21.15.116:0/1252607021 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604689b7dc0 0x5604689bf2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.833+0000 7f6aaf7fe640 1 -- 172.21.15.116:0/1252607021 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5604689b74a0 msgr2=0x5604689c2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:29.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.833+0000 7f6aaf7fe640 1 --2- 172.21.15.116:0/1252607021 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5604689b74a0 0x5604689c2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.833+0000 7f6aaf7fe640 1 -- 172.21.15.116:0/1252607021 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56046868e3c0 con 0x5604689c2680 2024-09-12T03:04:29.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.833+0000 7f6aaf7fe640 1 --2- 172.21.15.116:0/1252607021 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604689c2680 0x5604689bf870 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6aa800ff10 tx=0x7f6aa800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad58b660812ee71 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:29.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.833+0000 7f6a8effd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:04:29.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.833+0000 7f6a8effd640 1 -- 172.21.15.116:0/1252607021 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8028630 con 0x5604689c2680 2024-09-12T03:04:29.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.833+0000 7f6a8effd640 1 -- 172.21.15.116:0/1252607021 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f6aa801e3f0 con 0x5604689c2680 2024-09-12T03:04:29.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.833+0000 7f6ab7daf740 1 -- 172.21.15.116:0/1252607021 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6aa009b3d0 con 0x5604689c2680 2024-09-12T03:04:29.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.833+0000 7f6a8effd640 1 -- 172.21.15.116:0/1252607021 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6aa80287d0 con 0x5604689c2680 2024-09-12T03:04:29.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.833+0000 7f6ab7daf740 10 client.15540 fetch_fsmap learned FSMap version 25 2024-09-12T03:04:29.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.833+0000 7f6ab7daf740 1 -- 172.21.15.116:0/1252607021 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5604689d0c00 con 0x5604689c2680 2024-09-12T03:04:29.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.834+0000 7f6a8effd640 1 -- 172.21.15.116:0/1252607021 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6aa009b3d0 con 0x5604689c2680 2024-09-12T03:04:29.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.834+0000 7f6a8effd640 1 -- 172.21.15.116:0/1252607021 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6aa801eb60 con 0x5604689c2680 2024-09-12T03:04:29.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.834+0000 7f6ab7daf740 10 client.15540 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:04:29.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.834+0000 7f6ab7daf740 10 client.15540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:04:29.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.834+0000 7f6ab7daf740 20 client.15540 populate_metadata read hostname 'smithi116' 2024-09-12T03:04:29.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.834+0000 7f6ab7daf740 1 --2- 172.21.15.116:0/1252607021 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5604689d20c0 0x5604689d44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.834+0000 7f6ab7daf740 4 client.15540 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:04:29.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.834+0000 7f6ab7daf740 1 -- 172.21.15.116:0/1252607021 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56046892a850 con 0x5604689d20c0 2024-09-12T03:04:29.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.834+0000 7f6aaeffd640 1 --2- 172.21.15.116:0/1252607021 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5604689d20c0 0x5604689d44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:29.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.835+0000 7f6a8effd640 10 client.15540 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:29.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.835+0000 7f6aaeffd640 1 --2- 172.21.15.116:0/1252607021 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5604689d20c0 0x5604689d44f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be250d396c7d0f5a server_cookie=7497196985b014d5 in_seq=0 out_seq=0 2024-09-12T03:04:29.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.835+0000 7f6a8effd640 1 -- 172.21.15.116:0/1252607021 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56046892a850 con 0x5604689d20c0 2024-09-12T03:04:29.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.835+0000 7f6a8effd640 10 client.15540 handle_command_reply: tid=0 2024-09-12T03:04:29.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.835+0000 7f6ab7daf740 1 client.15540 shutdown 2024-09-12T03:04:29.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.835+0000 7f6ab7daf740 1 -- 172.21.15.116:0/1252607021 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604689c2680 msgr2=0x5604689bf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:29.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.835+0000 7f6ab7daf740 1 --2- 172.21.15.116:0/1252607021 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604689c2680 0x5604689bf870 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6aa800ff10 tx=0x7f6aa800a430 comp rx=0 tx=0).stop 2024-09-12T03:04:29.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.835+0000 7f6ab7daf740 1 -- 172.21.15.116:0/1252607021 shutdown_connections 2024-09-12T03:04:29.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.835+0000 7f6ab7daf740 1 --2- 172.21.15.116:0/1252607021 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5604689d20c0 0x5604689d44f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.835+0000 7f6ab7daf740 1 --2- 172.21.15.116:0/1252607021 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604689c2680 0x5604689bf870 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.835+0000 7f6ab7daf740 1 --2- 172.21.15.116:0/1252607021 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604689b7dc0 0x5604689bf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.835+0000 7f6ab7daf740 1 --2- 172.21.15.116:0/1252607021 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5604689b74a0 0x5604689c2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.835+0000 7f6a8effd640 0 client.15540 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:29.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.835+0000 7f6ab7daf740 1 -- 172.21.15.116:0/1252607021 >> 172.21.15.116:0/1252607021 conn(0x5604689b3a30 msgr2=0x5604689b9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:29.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.836+0000 7f6ab7daf740 1 -- 172.21.15.116:0/1252607021 shutdown_connections 2024-09-12T03:04:29.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.836+0000 7f6ab7daf740 1 -- 172.21.15.116:0/1252607021 wait complete. 2024-09-12T03:04:29.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.836+0000 7f6ab7daf740 20 client.15540 trim_cache size 0 max 16384 2024-09-12T03:04:29.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.853+0000 7f6ab7daf740 1 Processor -- start 2024-09-12T03:04:29.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.853+0000 7f6ab7daf740 1 -- start start 2024-09-12T03:04:29.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.853+0000 7f6ab7daf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5604689b74a0 0x5604689ce450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.853+0000 7f6ab7daf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604689b7dc0 0x5604689c1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.853+0000 7f6ab7daf740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604689c2680 0x5604689d9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.853+0000 7f6ab7daf740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab01afe80 con 0x5604689c2680 2024-09-12T03:04:29.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.853+0000 7f6ab7daf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab01ad750 con 0x5604689b74a0 2024-09-12T03:04:29.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.853+0000 7f6ab7daf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab01ab5a0 con 0x5604689b7dc0 2024-09-12T03:04:29.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.853+0000 7f6aaf7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604689c2680 0x5604689d9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:29.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.853+0000 7f6aaf7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604689c2680 0x5604689d9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49710/0 (socket says 172.21.15.116:49710) 2024-09-12T03:04:29.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.853+0000 7f6aaf7fe640 1 -- 172.21.15.116:0/3048674709 learned_addr learned my addr 172.21.15.116:0/3048674709 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:29.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.853+0000 7f6aaf7fe640 1 -- 172.21.15.116:0/3048674709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604689b7dc0 msgr2=0x5604689c1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:29.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.853+0000 7f6aaf7fe640 1 --2- 172.21.15.116:0/3048674709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604689b7dc0 0x5604689c1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.853+0000 7f6aaf7fe640 1 -- 172.21.15.116:0/3048674709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5604689b74a0 msgr2=0x5604689ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:29.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.853+0000 7f6aaf7fe640 1 --2- 172.21.15.116:0/3048674709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5604689b74a0 0x5604689ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.853+0000 7f6aaf7fe640 1 -- 172.21.15.116:0/3048674709 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604689b46c0 con 0x5604689c2680 2024-09-12T03:04:29.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.853+0000 7f6aaf7fe640 1 --2- 172.21.15.116:0/3048674709 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604689c2680 0x5604689d9dc0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8012600 tx=0x7f6aa800a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0d23f2b3342bef server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:29.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.853+0000 7f6a957fa640 1 -- 172.21.15.116:0/3048674709 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8028630 con 0x5604689c2680 2024-09-12T03:04:29.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.853+0000 7f6a957fa640 1 -- 172.21.15.116:0/3048674709 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6aa8037030 con 0x5604689c2680 2024-09-12T03:04:29.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.853+0000 7f6a957fa640 1 -- 172.21.15.116:0/3048674709 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8032540 con 0x5604689c2680 2024-09-12T03:04:29.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.854+0000 7f6ab7daf740 1 -- 172.21.15.116:0/3048674709 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604689c2680 msgr2=0x5604689d9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:29.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.854+0000 7f6ab7daf740 1 --2- 172.21.15.116:0/3048674709 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604689c2680 0x5604689d9dc0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8012600 tx=0x7f6aa800a8c0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.854+0000 7f6ab7daf740 1 -- 172.21.15.116:0/3048674709 shutdown_connections 2024-09-12T03:04:29.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.854+0000 7f6ab7daf740 1 --2- 172.21.15.116:0/3048674709 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604689c2680 0x5604689d9dc0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.854+0000 7f6ab7daf740 1 --2- 172.21.15.116:0/3048674709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604689b7dc0 0x5604689c1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.854+0000 7f6ab7daf740 1 --2- 172.21.15.116:0/3048674709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5604689b74a0 0x5604689ce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.854+0000 7f6ab7daf740 1 -- 172.21.15.116:0/3048674709 >> 172.21.15.116:0/3048674709 conn(0x5604689b3a30 msgr2=0x5604689c0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:29.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.854+0000 7f6ab7daf740 1 -- 172.21.15.116:0/3048674709 shutdown_connections 2024-09-12T03:04:29.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.854+0000 7f6ab7daf740 1 -- 172.21.15.116:0/3048674709 wait complete. 2024-09-12T03:04:29.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.854+0000 7f6ab7daf740 1 -- start start 2024-09-12T03:04:29.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.855+0000 7f6ab7daf740 1 Processor -- start 2024-09-12T03:04:29.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.855+0000 7f6ab7daf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5604689b74a0 0x5604689bc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.855+0000 7f6ab7daf740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604689b7dc0 0x5604689bc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.855+0000 7f6ab7daf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604689c2680 0x5604689bcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.855+0000 7f6ab7daf740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab01abc10 con 0x5604689b7dc0 2024-09-12T03:04:29.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.855+0000 7f6ab7daf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab01ad080 con 0x5604689b74a0 2024-09-12T03:04:29.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.855+0000 7f6ab7daf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab01ac2e0 con 0x5604689c2680 2024-09-12T03:04:29.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.855+0000 7f6ab7daf740 10 client.0 _ll_register_callbacks cb 0x7f6aa8012630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:29.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.855+0000 7f6aae7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604689b7dc0 0x5604689bc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:29.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.855+0000 7f6aae7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604689b7dc0 0x5604689bc830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49716/0 (socket says 172.21.15.116:49716) 2024-09-12T03:04:29.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.855+0000 7f6aae7fc640 1 -- 172.21.15.116:0/3114142543 learned_addr learned my addr 172.21.15.116:0/3114142543 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:29.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.855+0000 7f6aaf7fe640 1 --2- 172.21.15.116:0/3114142543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604689c2680 0x5604689bcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:29.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.855+0000 7f6aae7fc640 1 -- 172.21.15.116:0/3114142543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604689c2680 msgr2=0x5604689bcd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:29.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.855+0000 7f6aaeffd640 1 --2- 172.21.15.116:0/3114142543 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5604689b74a0 0x5604689bc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:29.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.855+0000 7f6aae7fc640 1 --2- 172.21.15.116:0/3114142543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604689c2680 0x5604689bcd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.855+0000 7f6aae7fc640 1 -- 172.21.15.116:0/3114142543 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5604689b74a0 msgr2=0x5604689bc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:29.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.855+0000 7f6aae7fc640 1 --2- 172.21.15.116:0/3114142543 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5604689b74a0 0x5604689bc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.855+0000 7f6aae7fc640 1 -- 172.21.15.116:0/3114142543 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560468a3bd90 con 0x5604689b7dc0 2024-09-12T03:04:29.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.855+0000 7f6aae7fc640 1 --2- 172.21.15.116:0/3114142543 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604689b7dc0 0x5604689bc830 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4009900 tx=0x7f6aa40317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38afc061d2407fc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:29.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.855+0000 7f6a8effd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:04:29.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.856+0000 7f6a8effd640 1 -- 172.21.15.116:0/3114142543 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4035040 con 0x5604689b7dc0 2024-09-12T03:04:29.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.856+0000 7f6a8effd640 1 -- 172.21.15.116:0/3114142543 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f6aa4004450 con 0x5604689b7dc0 2024-09-12T03:04:29.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.856+0000 7f6ab7daf740 1 -- 172.21.15.116:0/3114142543 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6aa8037030 con 0x5604689b7dc0 2024-09-12T03:04:29.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.856+0000 7f6a8effd640 1 -- 172.21.15.116:0/3114142543 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6aa4031a10 con 0x5604689b7dc0 2024-09-12T03:04:29.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.856+0000 7f6ab7daf740 10 client.15546 fetch_fsmap learned FSMap version 25 2024-09-12T03:04:29.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.856+0000 7f6ab7daf740 1 -- 172.21.15.116:0/3114142543 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560468a3d040 con 0x5604689b7dc0 2024-09-12T03:04:29.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.856+0000 7f6a8effd640 1 -- 172.21.15.116:0/3114142543 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6aa8037030 con 0x5604689b7dc0 2024-09-12T03:04:29.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.856+0000 7f6a8effd640 1 -- 172.21.15.116:0/3114142543 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6aa4004c60 con 0x5604689b7dc0 2024-09-12T03:04:29.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.856+0000 7f6ab7daf740 10 client.15546 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:04:29.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.856+0000 7f6ab7daf740 10 client.15546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:04:29.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.856+0000 7f6ab7daf740 20 client.15546 populate_metadata read hostname 'smithi116' 2024-09-12T03:04:29.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.856+0000 7f6ab7daf740 1 --2- 172.21.15.116:0/3114142543 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560468a3d7c0 0x560468a3fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.856+0000 7f6ab7daf740 4 client.15546 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:04:29.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.856+0000 7f6ab7daf740 1 -- 172.21.15.116:0/3114142543 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560468924d60 con 0x560468a3d7c0 2024-09-12T03:04:29.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.857+0000 7f6aaeffd640 1 --2- 172.21.15.116:0/3114142543 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560468a3d7c0 0x560468a3fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:29.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.857+0000 7f6a8effd640 10 client.15546 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:29.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.857+0000 7f6aaeffd640 1 --2- 172.21.15.116:0/3114142543 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560468a3d7c0 0x560468a3fbe0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8f0aec67f05d9f4 server_cookie=5d75bd11c3ca43ab in_seq=0 out_seq=0 2024-09-12T03:04:29.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.857+0000 7f6a8effd640 1 -- 172.21.15.116:0/3114142543 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560468924d60 con 0x560468a3d7c0 2024-09-12T03:04:29.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.857+0000 7f6a8effd640 10 client.15546 handle_command_reply: tid=0 2024-09-12T03:04:29.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.857+0000 7f6ab7daf740 1 client.15546 shutdown 2024-09-12T03:04:29.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.857+0000 7f6ab7daf740 1 -- 172.21.15.116:0/3114142543 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604689b7dc0 msgr2=0x5604689bc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:29.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.857+0000 7f6ab7daf740 1 --2- 172.21.15.116:0/3114142543 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604689b7dc0 0x5604689bc830 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4009900 tx=0x7f6aa40317e0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.857+0000 7f6ab7daf740 1 -- 172.21.15.116:0/3114142543 shutdown_connections 2024-09-12T03:04:29.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.857+0000 7f6ab7daf740 1 --2- 172.21.15.116:0/3114142543 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560468a3d7c0 0x560468a3fbe0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.857+0000 7f6ab7daf740 1 --2- 172.21.15.116:0/3114142543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604689c2680 0x5604689bcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.857+0000 7f6ab7daf740 1 --2- 172.21.15.116:0/3114142543 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604689b7dc0 0x5604689bc830 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.857+0000 7f6a8effd640 0 client.15546 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:29.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.857+0000 7f6ab7daf740 1 --2- 172.21.15.116:0/3114142543 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5604689b74a0 0x5604689bc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.857+0000 7f6ab7daf740 1 -- 172.21.15.116:0/3114142543 >> 172.21.15.116:0/3114142543 conn(0x5604689b3a30 msgr2=0x5604689b9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:29.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.857+0000 7f6ab7daf740 1 -- 172.21.15.116:0/3114142543 shutdown_connections 2024-09-12T03:04:29.898 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:04:29.898 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:04:29.898 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:04:29.899 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:04:29.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.857+0000 7f6ab7daf740 1 -- 172.21.15.116:0/3114142543 wait complete. 2024-09-12T03:04:29.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.857+0000 7f6ab7daf740 20 client.15546 trim_cache size 0 max 16384 2024-09-12T03:04:29.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.858+0000 7f6ab59d6640 1 -- 172.21.15.116:0/1544405936 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6a980a2b80 msgr2=0x7f6a980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:29.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.858+0000 7f6ab59d6640 1 --2- 172.21.15.116:0/1544405936 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6a980a2b80 0x7f6a980a5040 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8012920 tx=0x7f6aa800a730 comp rx=0 tx=0).stop 2024-09-12T03:04:29.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.858+0000 7f6ab59d6640 1 -- 172.21.15.116:0/1544405936 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ab0101610 msgr2=0x7f6ab0199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:29.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.858+0000 7f6ab59d6640 1 --2- 172.21.15.116:0/1544405936 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ab0101610 0x7f6ab0199310 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000d770 tx=0x7f6aa000dc30 comp rx=0 tx=0).stop 2024-09-12T03:04:29.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.858+0000 7f6ab59d6640 1 -- 172.21.15.116:0/1544405936 shutdown_connections 2024-09-12T03:04:29.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.858+0000 7f6ab59d6640 1 --2- 172.21.15.116:0/1544405936 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6a980a2b80 0x7f6a980a5040 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.858+0000 7f6ab59d6640 1 --2- 172.21.15.116:0/1544405936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ab0105e40 0x7f6ab01a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.858+0000 7f6ab59d6640 1 --2- 172.21.15.116:0/1544405936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ab0101fc0 0x7f6ab0199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.858+0000 7f6ab59d6640 1 --2- 172.21.15.116:0/1544405936 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ab0101610 0x7f6ab0199310 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.858+0000 7f6ab59d6640 1 -- 172.21.15.116:0/1544405936 >> 172.21.15.116:0/1544405936 conn(0x7f6ab00fc5c0 msgr2=0x7f6ab01040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:29.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.858+0000 7f6ab59d6640 1 -- 172.21.15.116:0/1544405936 shutdown_connections 2024-09-12T03:04:29.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:29.858+0000 7f6ab59d6640 1 -- 172.21.15.116:0/1544405936 wait complete. 2024-09-12T03:04:30.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:30 smithi177 ceph-mon[35305]: pgmap v383: 97 pgs: 97 active+clean; 748 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.9 KiB/s wr, 8 op/s 2024-09-12T03:04:30.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:30 smithi116 ceph-mon[31410]: pgmap v383: 97 pgs: 97 active+clean; 748 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.9 KiB/s wr, 8 op/s 2024-09-12T03:04:30.689 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:04:30.689 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:04:30.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:30 smithi183 ceph-mon[35535]: pgmap v383: 97 pgs: 97 active+clean; 748 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.9 KiB/s wr, 8 op/s 2024-09-12T03:04:30.937 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:31.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.439+0000 7f766c3d7640 1 -- 172.21.15.116:0/3909951661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7664101420 msgr2=0x7f766410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:31.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.439+0000 7f766c3d7640 1 --2- 172.21.15.116:0/3909951661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7664101420 0x7f766410d950 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f765c009cd0 tx=0x7f765c02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:04:31.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.439+0000 7f766c3d7640 1 -- 172.21.15.116:0/3909951661 shutdown_connections 2024-09-12T03:04:31.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.439+0000 7f766c3d7640 1 --2- 172.21.15.116:0/3909951661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7664101420 0x7f766410d950 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:31.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.439+0000 7f766c3d7640 1 --2- 172.21.15.116:0/3909951661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7664100a80 0x7f7664100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:31.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.439+0000 7f766c3d7640 1 --2- 172.21.15.116:0/3909951661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7664106a80 0x7f7664106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:31.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.439+0000 7f766c3d7640 1 -- 172.21.15.116:0/3909951661 >> 172.21.15.116:0/3909951661 conn(0x7f76640fc7a0 msgr2=0x7f76640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:31.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.439+0000 7f766c3d7640 1 -- 172.21.15.116:0/3909951661 shutdown_connections 2024-09-12T03:04:31.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.439+0000 7f766c3d7640 1 -- 172.21.15.116:0/3909951661 wait complete. 2024-09-12T03:04:31.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.440+0000 7f766c3d7640 1 Processor -- start 2024-09-12T03:04:31.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.440+0000 7f766c3d7640 1 -- start start 2024-09-12T03:04:31.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.440+0000 7f766c3d7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7664100a80 0x7f7664199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:31.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.440+0000 7f766c3d7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7664101420 0x7f76641997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:31.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.440+0000 7f766c3d7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7664106a80 0x7f76641a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:31.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.440+0000 7f766c3d7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7664110e80 con 0x7f7664106a80 2024-09-12T03:04:31.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.440+0000 7f766c3d7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7664110d00 con 0x7f7664100a80 2024-09-12T03:04:31.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.440+0000 7f766c3d7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7664111000 con 0x7f7664101420 2024-09-12T03:04:31.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.440+0000 7f766a974640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7664106a80 0x7f76641a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:31.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.440+0000 7f766a974640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7664106a80 0x7f76641a4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39134/0 (socket says 172.21.15.116:39134) 2024-09-12T03:04:31.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.440+0000 7f766a974640 1 -- 172.21.15.116:0/2813406234 learned_addr learned my addr 172.21.15.116:0/2813406234 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:31.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.440+0000 7f766a173640 1 --2- 172.21.15.116:0/2813406234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7664100a80 0x7f7664199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:31.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.440+0000 7f7669972640 1 --2- 172.21.15.116:0/2813406234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7664101420 0x7f76641997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:31.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.440+0000 7f766a974640 1 -- 172.21.15.116:0/2813406234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7664101420 msgr2=0x7f76641997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:31.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.440+0000 7f766a974640 1 --2- 172.21.15.116:0/2813406234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7664101420 0x7f76641997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:31.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.440+0000 7f766a974640 1 -- 172.21.15.116:0/2813406234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7664100a80 msgr2=0x7f7664199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:31.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.440+0000 7f766a974640 1 --2- 172.21.15.116:0/2813406234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7664100a80 0x7f7664199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:31.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.440+0000 7f766a974640 1 -- 172.21.15.116:0/2813406234 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76641a4850 con 0x7f7664106a80 2024-09-12T03:04:31.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.440+0000 7f7669972640 1 --2- 172.21.15.116:0/2813406234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7664101420 0x7f76641997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:04:31.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.440+0000 7f766a974640 1 --2- 172.21.15.116:0/2813406234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7664106a80 0x7f76641a4310 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f765c005cc0 tx=0x7f765c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:31.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.441+0000 7f765b7fe640 1 -- 172.21.15.116:0/2813406234 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c004040 con 0x7f7664106a80 2024-09-12T03:04:31.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.441+0000 7f765b7fe640 1 -- 172.21.15.116:0/2813406234 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f765c0046c0 con 0x7f7664106a80 2024-09-12T03:04:31.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.441+0000 7f766c3d7640 1 -- 172.21.15.116:0/2813406234 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76641a4a20 con 0x7f7664106a80 2024-09-12T03:04:31.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.441+0000 7f765b7fe640 1 -- 172.21.15.116:0/2813406234 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c046480 con 0x7f7664106a80 2024-09-12T03:04:31.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.441+0000 7f766c3d7640 1 -- 172.21.15.116:0/2813406234 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76641a4ed0 con 0x7f7664106a80 2024-09-12T03:04:31.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.442+0000 7f766c3d7640 1 -- 172.21.15.116:0/2813406234 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7664100ee0 con 0x7f7664106a80 2024-09-12T03:04:31.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.443+0000 7f765b7fe640 1 -- 172.21.15.116:0/2813406234 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f765c0305a0 con 0x7f7664106a80 2024-09-12T03:04:31.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.443+0000 7f765b7fe640 1 --2- 172.21.15.116:0/2813406234 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f76500a2b80 0x7f76500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:31.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.444+0000 7f766a173640 1 --2- 172.21.15.116:0/2813406234 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f76500a2b80 0x7f76500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:31.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.444+0000 7f765b7fe640 1 -- 172.21.15.116:0/2813406234 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f765c02e030 con 0x7f7664106a80 2024-09-12T03:04:31.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.444+0000 7f766a173640 1 --2- 172.21.15.116:0/2813406234 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f76500a2b80 0x7f76500a5040 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7654005fd0 tx=0x7f765400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:31.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.447+0000 7f765b7fe640 1 -- 172.21.15.116:0/2813406234 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f765c04b060 con 0x7f7664106a80 2024-09-12T03:04:31.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.646+0000 7f766c3d7640 1 -- 172.21.15.116:0/2813406234 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7664108ff0 con 0x7f7664106a80 2024-09-12T03:04:31.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.647+0000 7f765b7fe640 1 -- 172.21.15.116:0/2813406234 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f765c0bd940 con 0x7f7664106a80 2024-09-12T03:04:31.647 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:04:31.648 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:04:31.648 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:04:31.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.650+0000 7f766c3d7640 1 -- 172.21.15.116:0/2813406234 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f76500a2b80 msgr2=0x7f76500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:31.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.650+0000 7f766c3d7640 1 --2- 172.21.15.116:0/2813406234 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f76500a2b80 0x7f76500a5040 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7654005fd0 tx=0x7f765400b390 comp rx=0 tx=0).stop 2024-09-12T03:04:31.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.650+0000 7f766c3d7640 1 -- 172.21.15.116:0/2813406234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7664106a80 msgr2=0x7f76641a4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:31.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.650+0000 7f766c3d7640 1 --2- 172.21.15.116:0/2813406234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7664106a80 0x7f76641a4310 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f765c005cc0 tx=0x7f765c003970 comp rx=0 tx=0).stop 2024-09-12T03:04:31.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.650+0000 7f766c3d7640 1 -- 172.21.15.116:0/2813406234 shutdown_connections 2024-09-12T03:04:31.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.650+0000 7f766c3d7640 1 --2- 172.21.15.116:0/2813406234 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f76500a2b80 0x7f76500a5040 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:31.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.650+0000 7f766c3d7640 1 --2- 172.21.15.116:0/2813406234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7664106a80 0x7f76641a4310 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:31.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.650+0000 7f766c3d7640 1 --2- 172.21.15.116:0/2813406234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7664101420 0x7f76641997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:31.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.650+0000 7f766c3d7640 1 --2- 172.21.15.116:0/2813406234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7664100a80 0x7f7664199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:31.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.650+0000 7f766c3d7640 1 -- 172.21.15.116:0/2813406234 >> 172.21.15.116:0/2813406234 conn(0x7f76640fc7a0 msgr2=0x7f76640fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:31.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.650+0000 7f766c3d7640 1 -- 172.21.15.116:0/2813406234 shutdown_connections 2024-09-12T03:04:31.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:31.650+0000 7f766c3d7640 1 -- 172.21.15.116:0/2813406234 wait complete. 2024-09-12T03:04:32.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:32 smithi116 ceph-mon[31410]: pgmap v384: 97 pgs: 97 active+clean; 803 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T03:04:32.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:32 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2813406234' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:04:32.377 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:04:32.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:32 smithi177 ceph-mon[35305]: pgmap v384: 97 pgs: 97 active+clean; 803 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T03:04:32.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:32 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2813406234' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:04:32.625 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:32.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:32 smithi183 ceph-mon[35535]: pgmap v384: 97 pgs: 97 active+clean; 803 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T03:04:32.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:32 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2813406234' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:04:33.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.116+0000 7f1b1acb1640 1 -- 172.21.15.116:0/1745618931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b14102ef0 msgr2=0x7f1b1410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:33.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.116+0000 7f1b1acb1640 1 --2- 172.21.15.116:0/1745618931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b14102ef0 0x7f1b1410da40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c009900 tx=0x7f1b0c02a480 comp rx=0 tx=0).stop 2024-09-12T03:04:33.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.116+0000 7f1b1acb1640 1 -- 172.21.15.116:0/1745618931 shutdown_connections 2024-09-12T03:04:33.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.116+0000 7f1b1acb1640 1 --2- 172.21.15.116:0/1745618931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b14102ef0 0x7f1b1410da40 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.116+0000 7f1b1acb1640 1 --2- 172.21.15.116:0/1745618931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b14102570 0x7f1b141029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.116+0000 7f1b1acb1640 1 --2- 172.21.15.116:0/1745618931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b14100e20 0x7f1b14101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.116+0000 7f1b1acb1640 1 -- 172.21.15.116:0/1745618931 >> 172.21.15.116:0/1745618931 conn(0x7f1b140fc7d0 msgr2=0x7f1b140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:33.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.116+0000 7f1b1acb1640 1 -- 172.21.15.116:0/1745618931 shutdown_connections 2024-09-12T03:04:33.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.116+0000 7f1b1acb1640 1 -- 172.21.15.116:0/1745618931 wait complete. 2024-09-12T03:04:33.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.116+0000 7f1b1acb1640 1 Processor -- start 2024-09-12T03:04:33.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.116+0000 7f1b1acb1640 1 -- start start 2024-09-12T03:04:33.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.116+0000 7f1b1acb1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b14100e20 0x7f1b141985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:33.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.116+0000 7f1b1acb1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b14102570 0x7f1b14198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:33.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.116+0000 7f1b1acb1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b14102ef0 0x7f1b1419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:33.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.116+0000 7f1b1acb1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1b14111020 con 0x7f1b14102ef0 2024-09-12T03:04:33.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.116+0000 7f1b1acb1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1b14110ea0 con 0x7f1b14100e20 2024-09-12T03:04:33.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.116+0000 7f1b1acb1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1b141111a0 con 0x7f1b14102570 2024-09-12T03:04:33.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.117+0000 7f1b1924e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b14102ef0 0x7f1b1419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:33.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.117+0000 7f1b1924e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b14102ef0 0x7f1b1419ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39162/0 (socket says 172.21.15.116:39162) 2024-09-12T03:04:33.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.117+0000 7f1b1924e640 1 -- 172.21.15.116:0/2645595185 learned_addr learned my addr 172.21.15.116:0/2645595185 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:33.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.117+0000 7f1b18a4d640 1 --2- 172.21.15.116:0/2645595185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b14100e20 0x7f1b141985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:33.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.117+0000 7f1b1924e640 1 -- 172.21.15.116:0/2645595185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b14102570 msgr2=0x7f1b14198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:33.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.117+0000 7f1b1924e640 1 --2- 172.21.15.116:0/2645595185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b14102570 0x7f1b14198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.117+0000 7f1b1924e640 1 -- 172.21.15.116:0/2645595185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b14100e20 msgr2=0x7f1b141985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:33.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.117+0000 7f1b1924e640 1 --2- 172.21.15.116:0/2645595185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b14100e20 0x7f1b141985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.117+0000 7f1b1924e640 1 -- 172.21.15.116:0/2645595185 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b1419d590 con 0x7f1b14102ef0 2024-09-12T03:04:33.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.117+0000 7f1b18a4d640 1 --2- 172.21.15.116:0/2645595185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b14100e20 0x7f1b141985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:04:33.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.117+0000 7f1b1924e640 1 --2- 172.21.15.116:0/2645595185 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b14102ef0 0x7f1b1419ceb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c005ab0 tx=0x7f1b0c003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:33.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.117+0000 7f1b11ffb640 1 -- 172.21.15.116:0/2645595185 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c03f070 con 0x7f1b14102ef0 2024-09-12T03:04:33.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.117+0000 7f1b11ffb640 1 -- 172.21.15.116:0/2645595185 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b0c031e30 con 0x7f1b14102ef0 2024-09-12T03:04:33.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.117+0000 7f1b1acb1640 1 -- 172.21.15.116:0/2645595185 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b1419d820 con 0x7f1b14102ef0 2024-09-12T03:04:33.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.117+0000 7f1b11ffb640 1 -- 172.21.15.116:0/2645595185 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c035030 con 0x7f1b14102ef0 2024-09-12T03:04:33.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.117+0000 7f1b1acb1640 1 -- 172.21.15.116:0/2645595185 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b141a5160 con 0x7f1b14102ef0 2024-09-12T03:04:33.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.120+0000 7f1b11ffb640 1 -- 172.21.15.116:0/2645595185 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1b0c042070 con 0x7f1b14102ef0 2024-09-12T03:04:33.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.120+0000 7f1b1d08a740 1 Processor -- start 2024-09-12T03:04:33.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.120+0000 7f1b1d08a740 1 -- start start 2024-09-12T03:04:33.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.120+0000 7f1b1d08a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56251b905b40 0x56251b904090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:33.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.120+0000 7f1b1d08a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56251b9045d0 0x56251b904a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:33.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.120+0000 7f1b1d08a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56251b913780 0x56251b915b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:33.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.120+0000 7f1b1d08a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56251b86a2f0 con 0x56251b905b40 2024-09-12T03:04:33.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.120+0000 7f1b1d08a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56251b811cf0 con 0x56251b913780 2024-09-12T03:04:33.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.120+0000 7f1b1d08a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56251b8662b0 con 0x56251b9045d0 2024-09-12T03:04:33.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.120+0000 7f1b11ffb640 1 --2- 172.21.15.116:0/2645595185 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1afc0a2b80 0x7f1afc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:33.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.120+0000 7f1b13fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56251b905b40 0x56251b904090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:33.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.120+0000 7f1b13fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56251b905b40 0x56251b904090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39164/0 (socket says 172.21.15.116:39164) 2024-09-12T03:04:33.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.120+0000 7f1b13fff640 1 -- 172.21.15.116:0/771609513 learned_addr learned my addr 172.21.15.116:0/771609513 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:33.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.120+0000 7f1b1924e640 1 --2- 172.21.15.116:0/2645595185 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1afc0a2b80 0x7f1afc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:33.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.120+0000 7f1b18a4d640 1 --2- 172.21.15.116:0/771609513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56251b9045d0 0x56251b904a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:33.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.121+0000 7f1b13fff640 1 --2- 172.21.15.116:0/771609513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56251b913780 0x56251b915b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:33.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.121+0000 7f1b11ffb640 1 -- 172.21.15.116:0/2645595185 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1b0c0ef950 con 0x7f1b14102ef0 2024-09-12T03:04:33.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.121+0000 7f1b13fff640 1 -- 172.21.15.116:0/771609513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56251b9045d0 msgr2=0x56251b904a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:33.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.121+0000 7f1b13fff640 1 --2- 172.21.15.116:0/771609513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56251b9045d0 0x56251b904a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.121+0000 7f1b13fff640 1 -- 172.21.15.116:0/771609513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56251b913780 msgr2=0x56251b915b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:33.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.121+0000 7f1b13fff640 1 --2- 172.21.15.116:0/771609513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56251b913780 0x56251b915b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.121+0000 7f1b13fff640 1 -- 172.21.15.116:0/771609513 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56251b7f63a0 con 0x56251b905b40 2024-09-12T03:04:33.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.121+0000 7f1b13fff640 1 --2- 172.21.15.116:0/771609513 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56251b905b40 0x56251b904090 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1b08009a90 tx=0x7f1b080092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdbce389010cb702 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:33.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.121+0000 7f1b1924e640 1 --2- 172.21.15.116:0/2645595185 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1afc0a2b80 0x7f1afc0a5040 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0bc170 tx=0x7f1b0c02a480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:33.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.121+0000 7f1afaffd640 1 -- 172.21.15.116:0/771609513 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b08019070 con 0x56251b905b40 2024-09-12T03:04:33.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.121+0000 7f1afaffd640 1 -- 172.21.15.116:0/771609513 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b08010040 con 0x56251b905b40 2024-09-12T03:04:33.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.121+0000 7f1afaffd640 1 -- 172.21.15.116:0/771609513 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0800c050 con 0x56251b905b40 2024-09-12T03:04:33.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.122+0000 7f1b1d08a740 1 -- 172.21.15.116:0/771609513 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56251b905b40 msgr2=0x56251b904090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:33.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.122+0000 7f1b1d08a740 1 --2- 172.21.15.116:0/771609513 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56251b905b40 0x56251b904090 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1b08009a90 tx=0x7f1b080092d0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.122+0000 7f1b1d08a740 1 -- 172.21.15.116:0/771609513 shutdown_connections 2024-09-12T03:04:33.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.122+0000 7f1b1d08a740 1 --2- 172.21.15.116:0/771609513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56251b913780 0x56251b915b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.122+0000 7f1b1d08a740 1 --2- 172.21.15.116:0/771609513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56251b9045d0 0x56251b904a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.122+0000 7f1b1d08a740 1 --2- 172.21.15.116:0/771609513 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56251b905b40 0x56251b904090 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.122+0000 7f1b1d08a740 1 -- 172.21.15.116:0/771609513 >> 172.21.15.116:0/771609513 conn(0x56251b8cd310 msgr2=0x56251b900c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:33.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.122+0000 7f1b1d08a740 1 -- 172.21.15.116:0/771609513 shutdown_connections 2024-09-12T03:04:33.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.122+0000 7f1b1d08a740 1 -- 172.21.15.116:0/771609513 wait complete. 2024-09-12T03:04:33.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.122+0000 7f1b1d08a740 1 -- start start 2024-09-12T03:04:33.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.122+0000 7f1b1d08a740 1 Processor -- start 2024-09-12T03:04:33.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.122+0000 7f1b1d08a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56251b9045d0 0x56251b913200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:33.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.123+0000 7f1b1d08a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56251b905b40 0x56251b9104b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:33.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.123+0000 7f1b1d08a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56251b913780 0x56251b910bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:33.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.123+0000 7f1b1d08a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56251b916480 con 0x56251b9045d0 2024-09-12T03:04:33.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.123+0000 7f1b1d08a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56251b825060 con 0x56251b905b40 2024-09-12T03:04:33.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.123+0000 7f1b1d08a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56251b916600 con 0x56251b913780 2024-09-12T03:04:33.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.123+0000 7f1b1d08a740 10 client.0 _ll_register_callbacks cb 0x56251b5788c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:33.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.123+0000 7f1b13fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56251b9045d0 0x56251b913200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:33.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.123+0000 7f1b13fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56251b9045d0 0x56251b913200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39174/0 (socket says 172.21.15.116:39174) 2024-09-12T03:04:33.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.123+0000 7f1b13fff640 1 -- 172.21.15.116:0/589026638 learned_addr learned my addr 172.21.15.116:0/589026638 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:33.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.123+0000 7f1b18a4d640 1 --2- 172.21.15.116:0/589026638 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56251b905b40 0x56251b9104b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:33.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.123+0000 7f1b13fff640 1 -- 172.21.15.116:0/589026638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56251b913780 msgr2=0x56251b910bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:33.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.123+0000 7f1b13fff640 1 --2- 172.21.15.116:0/589026638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56251b913780 0x56251b910bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.123+0000 7f1b13fff640 1 -- 172.21.15.116:0/589026638 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56251b905b40 msgr2=0x56251b9104b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:33.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.123+0000 7f1b13fff640 1 --2- 172.21.15.116:0/589026638 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56251b905b40 0x56251b9104b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.123+0000 7f1b13fff640 1 -- 172.21.15.116:0/589026638 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56251b5db3c0 con 0x56251b9045d0 2024-09-12T03:04:33.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.123+0000 7f1b13fff640 1 --2- 172.21.15.116:0/589026638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56251b9045d0 0x56251b913200 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1b08017040 tx=0x7f1b080038c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783f08a503a6b248 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:33.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.123+0000 7f1af3fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:04:33.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.123+0000 7f1af3fff640 1 -- 172.21.15.116:0/589026638 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b08019040 con 0x56251b9045d0 2024-09-12T03:04:33.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.123+0000 7f1af3fff640 1 -- 172.21.15.116:0/589026638 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1b08014630 con 0x56251b9045d0 2024-09-12T03:04:33.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.123+0000 7f1b1d08a740 1 -- 172.21.15.116:0/589026638 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b08010040 con 0x56251b9045d0 2024-09-12T03:04:33.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.123+0000 7f1af3fff640 1 -- 172.21.15.116:0/589026638 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1b0800c040 con 0x56251b9045d0 2024-09-12T03:04:33.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.123+0000 7f1b1d08a740 10 client.15564 fetch_fsmap learned FSMap version 25 2024-09-12T03:04:33.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.123+0000 7f1b1d08a740 1 -- 172.21.15.116:0/589026638 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56251b921fb0 con 0x56251b9045d0 2024-09-12T03:04:33.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.124+0000 7f1af3fff640 1 -- 172.21.15.116:0/589026638 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b08010040 con 0x56251b9045d0 2024-09-12T03:04:33.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.124+0000 7f1af3fff640 1 -- 172.21.15.116:0/589026638 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1b0801e9e0 con 0x56251b9045d0 2024-09-12T03:04:33.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.124+0000 7f1b1d08a740 10 client.15564 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:04:33.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.124+0000 7f1b1d08a740 10 client.15564 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:04:33.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.124+0000 7f1b1d08a740 20 client.15564 populate_metadata read hostname 'smithi116' 2024-09-12T03:04:33.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.124+0000 7f1b1d08a740 1 --2- 172.21.15.116:0/589026638 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x56251b9233b0 0x56251b9257a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:33.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.124+0000 7f1b1d08a740 4 client.15564 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:04:33.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.124+0000 7f1b1d08a740 1 -- 172.21.15.116:0/589026638 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56251b877860 con 0x56251b9233b0 2024-09-12T03:04:33.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.124+0000 7f1b18a4d640 1 --2- 172.21.15.116:0/589026638 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x56251b9233b0 0x56251b9257a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:33.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.124+0000 7f1b18a4d640 1 --2- 172.21.15.116:0/589026638 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x56251b9233b0 0x56251b9257a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e7e50fd83f55dc70 server_cookie=2ae112599173c9a7 in_seq=0 out_seq=0 2024-09-12T03:04:33.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.124+0000 7f1af3fff640 10 client.15564 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:04:33.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.125+0000 7f1af3fff640 1 -- 172.21.15.116:0/589026638 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56251b877860 con 0x56251b9233b0 2024-09-12T03:04:33.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.125+0000 7f1af3fff640 10 client.15564 handle_command_reply: tid=0 2024-09-12T03:04:33.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.125+0000 7f1b1d08a740 1 client.15564 shutdown 2024-09-12T03:04:33.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.125+0000 7f1b1d08a740 1 -- 172.21.15.116:0/589026638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56251b9045d0 msgr2=0x56251b913200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:33.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.125+0000 7f1b1d08a740 1 --2- 172.21.15.116:0/589026638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56251b9045d0 0x56251b913200 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1b08017040 tx=0x7f1b080038c0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.125+0000 7f1b1d08a740 1 -- 172.21.15.116:0/589026638 shutdown_connections 2024-09-12T03:04:33.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.125+0000 7f1b1d08a740 1 --2- 172.21.15.116:0/589026638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56251b913780 0x56251b910bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.125+0000 7f1b1d08a740 1 --2- 172.21.15.116:0/589026638 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56251b905b40 0x56251b9104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.125+0000 7f1b1d08a740 1 --2- 172.21.15.116:0/589026638 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x56251b9233b0 0x56251b9257a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.125+0000 7f1b1d08a740 1 --2- 172.21.15.116:0/589026638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56251b9045d0 0x56251b913200 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.125+0000 7f1af3fff640 0 client.15564 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:04:33.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.125+0000 7f1b1d08a740 1 -- 172.21.15.116:0/589026638 >> 172.21.15.116:0/589026638 conn(0x56251b8cd310 msgr2=0x56251b906960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:33.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.125+0000 7f1b1d08a740 1 -- 172.21.15.116:0/589026638 shutdown_connections 2024-09-12T03:04:33.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.126+0000 7f1b1d08a740 1 -- 172.21.15.116:0/589026638 wait complete. 2024-09-12T03:04:33.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.126+0000 7f1b1d08a740 20 client.15564 trim_cache size 0 max 16384 2024-09-12T03:04:33.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.138+0000 7f1b1d08a740 1 Processor -- start 2024-09-12T03:04:33.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.138+0000 7f1b1d08a740 1 -- start start 2024-09-12T03:04:33.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.138+0000 7f1b1d08a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56251b9045d0 0x56251b90fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:33.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.138+0000 7f1b1d08a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56251b905b40 0x56251b900510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:33.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.138+0000 7f1b1d08a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56251b913780 0x56251b900a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:33.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.138+0000 7f1b1d08a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1b141adba0 con 0x56251b9045d0 2024-09-12T03:04:33.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.138+0000 7f1b1d08a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1b141ab470 con 0x56251b905b40 2024-09-12T03:04:33.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.138+0000 7f1b1d08a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1b141a92e0 con 0x56251b913780 2024-09-12T03:04:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.139+0000 7f1b13fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56251b9045d0 0x56251b90fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.139+0000 7f1b13fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56251b9045d0 0x56251b90fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39182/0 (socket says 172.21.15.116:39182) 2024-09-12T03:04:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.139+0000 7f1b13fff640 1 -- 172.21.15.116:0/3837551786 learned_addr learned my addr 172.21.15.116:0/3837551786 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.139+0000 7f1b18a4d640 1 --2- 172.21.15.116:0/3837551786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56251b905b40 0x56251b900510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.139+0000 7f1b13fff640 1 -- 172.21.15.116:0/3837551786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56251b913780 msgr2=0x56251b900a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.139+0000 7f1b13fff640 1 --2- 172.21.15.116:0/3837551786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56251b913780 0x56251b900a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.139+0000 7f1b13fff640 1 -- 172.21.15.116:0/3837551786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56251b905b40 msgr2=0x56251b900510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.139+0000 7f1b13fff640 1 --2- 172.21.15.116:0/3837551786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56251b905b40 0x56251b900510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.139+0000 7f1b13fff640 1 -- 172.21.15.116:0/3837551786 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56251b9067f0 con 0x56251b9045d0 2024-09-12T03:04:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.139+0000 7f1b13fff640 1 --2- 172.21.15.116:0/3837551786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56251b9045d0 0x56251b90fd70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1b08009650 tx=0x7f1b080047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518c57cf6381eaba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.139+0000 7f1afa7fc640 1 -- 172.21.15.116:0/3837551786 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b08019070 con 0x56251b9045d0 2024-09-12T03:04:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.139+0000 7f1afa7fc640 1 -- 172.21.15.116:0/3837551786 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b08023070 con 0x56251b9045d0 2024-09-12T03:04:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.139+0000 7f1afa7fc640 1 -- 172.21.15.116:0/3837551786 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0800f040 con 0x56251b9045d0 2024-09-12T03:04:33.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.140+0000 7f1b1d08a740 1 -- 172.21.15.116:0/3837551786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56251b9045d0 msgr2=0x56251b90fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:33.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.140+0000 7f1b1d08a740 1 --2- 172.21.15.116:0/3837551786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56251b9045d0 0x56251b90fd70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1b08009650 tx=0x7f1b080047d0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.140+0000 7f1b1d08a740 1 -- 172.21.15.116:0/3837551786 shutdown_connections 2024-09-12T03:04:33.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.140+0000 7f1b1d08a740 1 --2- 172.21.15.116:0/3837551786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56251b913780 0x56251b900a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.140+0000 7f1b1d08a740 1 --2- 172.21.15.116:0/3837551786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56251b905b40 0x56251b900510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.140+0000 7f1b1d08a740 1 --2- 172.21.15.116:0/3837551786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56251b9045d0 0x56251b90fd70 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.140+0000 7f1b1d08a740 1 -- 172.21.15.116:0/3837551786 >> 172.21.15.116:0/3837551786 conn(0x56251b8cd310 msgr2=0x56251b911d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:33.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.140+0000 7f1b1d08a740 1 -- 172.21.15.116:0/3837551786 shutdown_connections 2024-09-12T03:04:33.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.140+0000 7f1b1d08a740 1 -- 172.21.15.116:0/3837551786 wait complete. 2024-09-12T03:04:33.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.140+0000 7f1b1d08a740 1 -- start start 2024-09-12T03:04:33.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.140+0000 7f1b1d08a740 1 Processor -- start 2024-09-12T03:04:33.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.140+0000 7f1b1d08a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56251b9045d0 0x56251b91bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:33.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.140+0000 7f1b1d08a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56251b905b40 0x56251b91c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:33.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.140+0000 7f1b1d08a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56251b913780 0x56251b90d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:33.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.140+0000 7f1b1d08a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1b141a9930 con 0x56251b905b40 2024-09-12T03:04:33.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.140+0000 7f1b1d08a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1b141aada0 con 0x56251b9045d0 2024-09-12T03:04:33.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.140+0000 7f1b1d08a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1b141aa000 con 0x56251b913780 2024-09-12T03:04:33.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.141+0000 7f1b1d08a740 10 client.0 _ll_register_callbacks cb 0x7f1b08017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:33.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.141+0000 7f1b18a4d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56251b905b40 0x56251b91c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:33.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.141+0000 7f1b18a4d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56251b905b40 0x56251b91c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39186/0 (socket says 172.21.15.116:39186) 2024-09-12T03:04:33.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.141+0000 7f1b18a4d640 1 -- 172.21.15.116:0/883513314 learned_addr learned my addr 172.21.15.116:0/883513314 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:33.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.141+0000 7f1b13fff640 1 --2- 172.21.15.116:0/883513314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56251b9045d0 0x56251b91bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:33.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.141+0000 7f1b13fff640 1 --2- 172.21.15.116:0/883513314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56251b913780 0x56251b90d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:33.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.141+0000 7f1b18a4d640 1 -- 172.21.15.116:0/883513314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56251b913780 msgr2=0x56251b90d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:33.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.141+0000 7f1b18a4d640 1 --2- 172.21.15.116:0/883513314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56251b913780 0x56251b90d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.141+0000 7f1b18a4d640 1 -- 172.21.15.116:0/883513314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56251b9045d0 msgr2=0x56251b91bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:33.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.141+0000 7f1b18a4d640 1 --2- 172.21.15.116:0/883513314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56251b9045d0 0x56251b91bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.141+0000 7f1b18a4d640 1 -- 172.21.15.116:0/883513314 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56251b90d8e0 con 0x56251b905b40 2024-09-12T03:04:33.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.141+0000 7f1b18a4d640 1 --2- 172.21.15.116:0/883513314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56251b905b40 0x56251b91c140 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1b04007fd0 tx=0x7f1b04010c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866e8a996ce2f24f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:33.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.141+0000 7f1af3fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:04:33.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.141+0000 7f1af3fff640 1 -- 172.21.15.116:0/883513314 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b04010ea0 con 0x56251b905b40 2024-09-12T03:04:33.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.141+0000 7f1af3fff640 1 -- 172.21.15.116:0/883513314 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1b04011d70 con 0x56251b905b40 2024-09-12T03:04:33.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.141+0000 7f1b1d08a740 1 -- 172.21.15.116:0/883513314 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b08023070 con 0x56251b905b40 2024-09-12T03:04:33.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.141+0000 7f1af3fff640 1 -- 172.21.15.116:0/883513314 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1b0402a070 con 0x56251b905b40 2024-09-12T03:04:33.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.141+0000 7f1b1d08a740 10 client.15570 fetch_fsmap learned FSMap version 25 2024-09-12T03:04:33.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.141+0000 7f1b1d08a740 1 -- 172.21.15.116:0/883513314 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56251b98daf0 con 0x56251b905b40 2024-09-12T03:04:33.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.142+0000 7f1af3fff640 1 -- 172.21.15.116:0/883513314 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b08023070 con 0x56251b905b40 2024-09-12T03:04:33.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.142+0000 7f1af3fff640 1 -- 172.21.15.116:0/883513314 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1b04023730 con 0x56251b905b40 2024-09-12T03:04:33.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.142+0000 7f1b1d08a740 10 client.15570 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:04:33.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.142+0000 7f1b1d08a740 10 client.15570 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:04:33.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.142+0000 7f1b1d08a740 20 client.15570 populate_metadata read hostname 'smithi116' 2024-09-12T03:04:33.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.142+0000 7f1b1d08a740 1 --2- 172.21.15.116:0/883513314 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x56251b98e250 0x56251b990670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:33.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.142+0000 7f1b1d08a740 4 client.15570 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:04:33.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.142+0000 7f1b1d08a740 1 -- 172.21.15.116:0/883513314 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56251b871d70 con 0x56251b98e250 2024-09-12T03:04:33.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.142+0000 7f1b13fff640 1 --2- 172.21.15.116:0/883513314 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x56251b98e250 0x56251b990670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:33.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.142+0000 7f1b13fff640 1 --2- 172.21.15.116:0/883513314 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x56251b98e250 0x56251b990670 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=833113d409578965 server_cookie=80353225c80bfe64 in_seq=0 out_seq=0 2024-09-12T03:04:33.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.142+0000 7f1af3fff640 10 client.15570 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:04:33.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.143+0000 7f1af3fff640 1 -- 172.21.15.116:0/883513314 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56251b871d70 con 0x56251b98e250 2024-09-12T03:04:33.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.143+0000 7f1af3fff640 10 client.15570 handle_command_reply: tid=0 2024-09-12T03:04:33.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.143+0000 7f1b1d08a740 1 client.15570 shutdown 2024-09-12T03:04:33.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.143+0000 7f1b1d08a740 1 -- 172.21.15.116:0/883513314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56251b905b40 msgr2=0x56251b91c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:33.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.143+0000 7f1b1d08a740 1 --2- 172.21.15.116:0/883513314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56251b905b40 0x56251b91c140 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1b04007fd0 tx=0x7f1b04010c50 comp rx=0 tx=0).stop 2024-09-12T03:04:33.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.143+0000 7f1b1d08a740 1 -- 172.21.15.116:0/883513314 shutdown_connections 2024-09-12T03:04:33.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.143+0000 7f1b1d08a740 1 --2- 172.21.15.116:0/883513314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56251b913780 0x56251b90d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.143+0000 7f1b1d08a740 1 --2- 172.21.15.116:0/883513314 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x56251b98e250 0x56251b990670 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.143+0000 7f1af3fff640 0 client.15570 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:04:33.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.143+0000 7f1b1d08a740 1 --2- 172.21.15.116:0/883513314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56251b905b40 0x56251b91c140 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.143+0000 7f1b1d08a740 1 --2- 172.21.15.116:0/883513314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56251b9045d0 0x56251b91bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.143+0000 7f1b1d08a740 1 -- 172.21.15.116:0/883513314 >> 172.21.15.116:0/883513314 conn(0x56251b8cd310 msgr2=0x56251b90f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:33.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.143+0000 7f1b1d08a740 1 -- 172.21.15.116:0/883513314 shutdown_connections 2024-09-12T03:04:33.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.143+0000 7f1b1d08a740 1 -- 172.21.15.116:0/883513314 wait complete. 2024-09-12T03:04:33.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.143+0000 7f1b1d08a740 20 client.15570 trim_cache size 0 max 16384 2024-09-12T03:04:33.180 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:04:33.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.143+0000 7f1b1acb1640 1 -- 172.21.15.116:0/2645595185 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1afc0a2b80 msgr2=0x7f1afc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:33.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.143+0000 7f1b1acb1640 1 --2- 172.21.15.116:0/2645595185 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1afc0a2b80 0x7f1afc0a5040 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0bc170 tx=0x7f1b0c02a480 comp rx=0 tx=0).stop 2024-09-12T03:04:33.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.143+0000 7f1b1acb1640 1 -- 172.21.15.116:0/2645595185 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b14102ef0 msgr2=0x7f1b1419ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:33.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.144+0000 7f1b1acb1640 1 --2- 172.21.15.116:0/2645595185 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b14102ef0 0x7f1b1419ceb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c005ab0 tx=0x7f1b0c003c30 comp rx=0 tx=0).stop 2024-09-12T03:04:33.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.144+0000 7f1b1acb1640 1 -- 172.21.15.116:0/2645595185 shutdown_connections 2024-09-12T03:04:33.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.144+0000 7f1b1acb1640 1 --2- 172.21.15.116:0/2645595185 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1afc0a2b80 0x7f1afc0a5040 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.144+0000 7f1b1acb1640 1 --2- 172.21.15.116:0/2645595185 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b14102ef0 0x7f1b1419ceb0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.144+0000 7f1b1acb1640 1 --2- 172.21.15.116:0/2645595185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b14102570 0x7f1b14198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.144+0000 7f1b1acb1640 1 --2- 172.21.15.116:0/2645595185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b14100e20 0x7f1b141985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.144+0000 7f1b1acb1640 1 -- 172.21.15.116:0/2645595185 >> 172.21.15.116:0/2645595185 conn(0x7f1b140fc7d0 msgr2=0x7f1b1406a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:33.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.144+0000 7f1b1acb1640 1 -- 172.21.15.116:0/2645595185 shutdown_connections 2024-09-12T03:04:33.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:33.144+0000 7f1b1acb1640 1 -- 172.21.15.116:0/2645595185 wait complete. 2024-09-12T03:04:33.859 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:04:34.105 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:34.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:34 smithi177 ceph-mon[35305]: pgmap v385: 97 pgs: 97 active+clean; 803 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:04:34.519 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:34 smithi116 ceph-mon[31410]: pgmap v385: 97 pgs: 97 active+clean; 803 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:04:34.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.599+0000 7f24021b9640 1 -- 172.21.15.116:0/956688289 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23fc100e20 msgr2=0x7f23fc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:34.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.599+0000 7f24021b9640 1 --2- 172.21.15.116:0/956688289 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23fc100e20 0x7f23fc101200 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f23ec009900 tx=0x7f23ec02a480 comp rx=0 tx=0).stop 2024-09-12T03:04:34.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.599+0000 7f24021b9640 1 -- 172.21.15.116:0/956688289 shutdown_connections 2024-09-12T03:04:34.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.599+0000 7f24021b9640 1 --2- 172.21.15.116:0/956688289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23fc102ef0 0x7f23fc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.599+0000 7f24021b9640 1 --2- 172.21.15.116:0/956688289 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23fc102570 0x7f23fc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.599+0000 7f24021b9640 1 --2- 172.21.15.116:0/956688289 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23fc100e20 0x7f23fc101200 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.599+0000 7f24021b9640 1 -- 172.21.15.116:0/956688289 >> 172.21.15.116:0/956688289 conn(0x7f23fc0fc7d0 msgr2=0x7f23fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:34.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.599+0000 7f24021b9640 1 -- 172.21.15.116:0/956688289 shutdown_connections 2024-09-12T03:04:34.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.599+0000 7f24021b9640 1 -- 172.21.15.116:0/956688289 wait complete. 2024-09-12T03:04:34.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.599+0000 7f24021b9640 1 Processor -- start 2024-09-12T03:04:34.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.599+0000 7f24021b9640 1 -- start start 2024-09-12T03:04:34.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.600+0000 7f24021b9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23fc100e20 0x7f23fc0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:34.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.600+0000 7f24021b9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23fc102570 0x7f23fc0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:34.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.600+0000 7f24021b9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23fc102ef0 0x7f23fc0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:34.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.600+0000 7f24021b9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc111020 con 0x7f23fc102570 2024-09-12T03:04:34.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.600+0000 7f24021b9640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc110ea0 con 0x7f23fc100e20 2024-09-12T03:04:34.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.600+0000 7f24021b9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc1111a0 con 0x7f23fc102ef0 2024-09-12T03:04:34.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.600+0000 7f23f2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23fc102570 0x7f23fc0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:34.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.600+0000 7f23f2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23fc102570 0x7f23fc0fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39206/0 (socket says 172.21.15.116:39206) 2024-09-12T03:04:34.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.600+0000 7f23f2ffd640 1 -- 172.21.15.116:0/631374896 learned_addr learned my addr 172.21.15.116:0/631374896 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:34.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.600+0000 7f23fbfff640 1 --2- 172.21.15.116:0/631374896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23fc102ef0 0x7f23fc0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:34.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.600+0000 7f23fb7fe640 1 --2- 172.21.15.116:0/631374896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23fc100e20 0x7f23fc0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:34.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.600+0000 7f23f2ffd640 1 -- 172.21.15.116:0/631374896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23fc102ef0 msgr2=0x7f23fc0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:34.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.600+0000 7f23f2ffd640 1 --2- 172.21.15.116:0/631374896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23fc102ef0 0x7f23fc0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.600+0000 7f23f2ffd640 1 -- 172.21.15.116:0/631374896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23fc100e20 msgr2=0x7f23fc0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:34.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.600+0000 7f23f2ffd640 1 --2- 172.21.15.116:0/631374896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23fc100e20 0x7f23fc0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.600+0000 7f23f2ffd640 1 -- 172.21.15.116:0/631374896 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23fc1a6b30 con 0x7f23fc102570 2024-09-12T03:04:34.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.601+0000 7f23f2ffd640 1 --2- 172.21.15.116:0/631374896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23fc102570 0x7f23fc0fd1c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f23e400b4a0 tx=0x7f23e400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:34.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.601+0000 7f23f97fa640 1 -- 172.21.15.116:0/631374896 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4014070 con 0x7f23fc102570 2024-09-12T03:04:34.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.601+0000 7f23f97fa640 1 -- 172.21.15.116:0/631374896 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23e4010af0 con 0x7f23fc102570 2024-09-12T03:04:34.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.601+0000 7f24021b9640 1 -- 172.21.15.116:0/631374896 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23fc1a6e20 con 0x7f23fc102570 2024-09-12T03:04:34.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.601+0000 7f23f97fa640 1 -- 172.21.15.116:0/631374896 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4017d10 con 0x7f23fc102570 2024-09-12T03:04:34.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.601+0000 7f24021b9640 1 -- 172.21.15.116:0/631374896 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23fc1a72d0 con 0x7f23fc102570 2024-09-12T03:04:34.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.603+0000 7f23f97fa640 1 -- 172.21.15.116:0/631374896 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f23e4017070 con 0x7f23fc102570 2024-09-12T03:04:34.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.603+0000 7f2404592740 1 Processor -- start 2024-09-12T03:04:34.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.603+0000 7f2404592740 1 -- start start 2024-09-12T03:04:34.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.604+0000 7f2404592740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c9802b40 0x55c2c9801090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:34.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.604+0000 7f2404592740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c98015d0 0x55c2c9801a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:34.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.604+0000 7f2404592740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c980c370 0x55c2c980e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:34.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.604+0000 7f2404592740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c2c97672f0 con 0x55c2c980c370 2024-09-12T03:04:34.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.604+0000 7f2404592740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c2c970ecf0 con 0x55c2c98015d0 2024-09-12T03:04:34.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.604+0000 7f2404592740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c2c97632b0 con 0x55c2c9802b40 2024-09-12T03:04:34.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.604+0000 7f23f2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c980c370 0x55c2c980e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:34.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.604+0000 7f23f2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c980c370 0x55c2c980e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39212/0 (socket says 172.21.15.116:39212) 2024-09-12T03:04:34.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.604+0000 7f23f2ffd640 1 -- 172.21.15.116:0/577609081 learned_addr learned my addr 172.21.15.116:0/577609081 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:34.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.604+0000 7f23fb7fe640 1 --2- 172.21.15.116:0/577609081 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c98015d0 0x55c2c9801a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:34.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.604+0000 7f23fbfff640 1 --2- 172.21.15.116:0/577609081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c9802b40 0x55c2c9801090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:34.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.604+0000 7f23f97fa640 1 --2- 172.21.15.116:0/631374896 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f23e80a2b80 0x7f23e80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:34.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.604+0000 7f23f2ffd640 1 -- 172.21.15.116:0/577609081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c9802b40 msgr2=0x55c2c9801090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:34.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.604+0000 7f23f2ffd640 1 --2- 172.21.15.116:0/577609081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c9802b40 0x55c2c9801090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.604+0000 7f23f2ffd640 1 -- 172.21.15.116:0/577609081 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c98015d0 msgr2=0x55c2c9801a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:34.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.604+0000 7f23f2ffd640 1 --2- 172.21.15.116:0/577609081 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c98015d0 0x55c2c9801a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.604+0000 7f23f2ffd640 1 -- 172.21.15.116:0/577609081 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2c96f33a0 con 0x55c2c980c370 2024-09-12T03:04:34.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.604+0000 7f23fbfff640 1 --2- 172.21.15.116:0/577609081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c9802b40 0x55c2c9801090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:34.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.604+0000 7f23fb7fe640 1 --2- 172.21.15.116:0/577609081 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c98015d0 0x55c2c9801a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:34.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.604+0000 7f23fbfff640 1 --2- 172.21.15.116:0/631374896 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f23e80a2b80 0x7f23e80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:34.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.604+0000 7f23f97fa640 1 -- 172.21.15.116:0/631374896 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f23e40ce370 con 0x7f23fc102570 2024-09-12T03:04:34.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.604+0000 7f23f2ffd640 1 --2- 172.21.15.116:0/577609081 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c980c370 0x55c2c980e760 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f23e404d1c0 tx=0x7f23e4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e1041d1626c506 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:34.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.605+0000 7f23fbfff640 1 --2- 172.21.15.116:0/631374896 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f23e80a2b80 0x7f23e80a5040 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f23f400a5a0 tx=0x7f23f4004b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:34.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.605+0000 7f23f1ffb640 1 -- 172.21.15.116:0/577609081 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e409a740 con 0x55c2c980c370 2024-09-12T03:04:34.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.605+0000 7f23f1ffb640 1 -- 172.21.15.116:0/577609081 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23e404a0f0 con 0x55c2c980c370 2024-09-12T03:04:34.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.605+0000 7f23f1ffb640 1 -- 172.21.15.116:0/577609081 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4013170 con 0x55c2c980c370 2024-09-12T03:04:34.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.605+0000 7f2404592740 1 -- 172.21.15.116:0/577609081 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c980c370 msgr2=0x55c2c980e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:34.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.605+0000 7f2404592740 1 --2- 172.21.15.116:0/577609081 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c980c370 0x55c2c980e760 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f23e404d1c0 tx=0x7f23e4020000 comp rx=0 tx=0).stop 2024-09-12T03:04:34.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.605+0000 7f2404592740 1 -- 172.21.15.116:0/577609081 shutdown_connections 2024-09-12T03:04:34.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.605+0000 7f2404592740 1 --2- 172.21.15.116:0/577609081 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c980c370 0x55c2c980e760 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.605+0000 7f2404592740 1 --2- 172.21.15.116:0/577609081 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c98015d0 0x55c2c9801a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.605+0000 7f2404592740 1 --2- 172.21.15.116:0/577609081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c9802b40 0x55c2c9801090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.605+0000 7f2404592740 1 -- 172.21.15.116:0/577609081 >> 172.21.15.116:0/577609081 conn(0x55c2c97ca310 msgr2=0x55c2c97fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:34.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.605+0000 7f2404592740 1 -- 172.21.15.116:0/577609081 shutdown_connections 2024-09-12T03:04:34.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.605+0000 7f2404592740 1 -- 172.21.15.116:0/577609081 wait complete. 2024-09-12T03:04:34.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.606+0000 7f2404592740 1 -- start start 2024-09-12T03:04:34.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.606+0000 7f2404592740 1 Processor -- start 2024-09-12T03:04:34.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.606+0000 7f2404592740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c98015d0 0x55c2c980bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:34.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.606+0000 7f2404592740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c9802b40 0x55c2c9808f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:34.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.606+0000 7f2404592740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c980c370 0x55c2c98094f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:34.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.606+0000 7f2404592740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c2c980f010 con 0x55c2c9802b40 2024-09-12T03:04:34.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.606+0000 7f2404592740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c2c9722060 con 0x55c2c980c370 2024-09-12T03:04:34.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.606+0000 7f2404592740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c2c980f190 con 0x55c2c98015d0 2024-09-12T03:04:34.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.606+0000 7f2404592740 10 client.0 _ll_register_callbacks cb 0x55c2c94758c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:34.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.606+0000 7f23f2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c9802b40 0x55c2c9808f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:34.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.606+0000 7f23f2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c9802b40 0x55c2c9808f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39214/0 (socket says 172.21.15.116:39214) 2024-09-12T03:04:34.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.606+0000 7f23f2ffd640 1 -- 172.21.15.116:0/3140791034 learned_addr learned my addr 172.21.15.116:0/3140791034 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:34.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.607+0000 7f23fbfff640 1 --2- 172.21.15.116:0/3140791034 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c980c370 0x55c2c98094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:34.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.607+0000 7f23fb7fe640 1 --2- 172.21.15.116:0/3140791034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c98015d0 0x55c2c980bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:34.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.607+0000 7f23f2ffd640 1 -- 172.21.15.116:0/3140791034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c98015d0 msgr2=0x55c2c980bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:34.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.607+0000 7f23f2ffd640 1 --2- 172.21.15.116:0/3140791034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c98015d0 0x55c2c980bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.607+0000 7f23f2ffd640 1 -- 172.21.15.116:0/3140791034 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c980c370 msgr2=0x55c2c98094f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:34.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.607+0000 7f23f2ffd640 1 --2- 172.21.15.116:0/3140791034 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c980c370 0x55c2c98094f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.607+0000 7f23f2ffd640 1 -- 172.21.15.116:0/3140791034 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2c94d83c0 con 0x55c2c9802b40 2024-09-12T03:04:34.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.607+0000 7f23fb7fe640 1 --2- 172.21.15.116:0/3140791034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c98015d0 0x55c2c980bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:04:34.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.607+0000 7f23f2ffd640 1 --2- 172.21.15.116:0/3140791034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c9802b40 0x55c2c9808f90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f23e400b470 tx=0x7f23e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba75081548fa232a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:34.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.607+0000 7f23daffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:04:34.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.607+0000 7f23daffd640 1 -- 172.21.15.116:0/3140791034 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e409a560 con 0x55c2c9802b40 2024-09-12T03:04:34.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.607+0000 7f23daffd640 1 -- 172.21.15.116:0/3140791034 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f23e40d1ae0 con 0x55c2c9802b40 2024-09-12T03:04:34.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.607+0000 7f2404592740 1 -- 172.21.15.116:0/3140791034 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c2c9774860 con 0x55c2c9802b40 2024-09-12T03:04:34.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.607+0000 7f23daffd640 1 -- 172.21.15.116:0/3140791034 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f23e409ac00 con 0x55c2c9802b40 2024-09-12T03:04:34.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.607+0000 7f2404592740 10 client.15582 fetch_fsmap learned FSMap version 25 2024-09-12T03:04:34.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.607+0000 7f2404592740 1 -- 172.21.15.116:0/3140791034 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c2c981aa20 con 0x55c2c9802b40 2024-09-12T03:04:34.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.607+0000 7f23daffd640 1 -- 172.21.15.116:0/3140791034 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c2c9774860 con 0x55c2c9802b40 2024-09-12T03:04:34.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.607+0000 7f23daffd640 1 -- 172.21.15.116:0/3140791034 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f23e4097270 con 0x55c2c9802b40 2024-09-12T03:04:34.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.607+0000 7f2404592740 10 client.15582 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:04:34.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.607+0000 7f2404592740 10 client.15582 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:04:34.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.607+0000 7f2404592740 20 client.15582 populate_metadata read hostname 'smithi116' 2024-09-12T03:04:34.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.607+0000 7f2404592740 1 --2- 172.21.15.116:0/3140791034 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c2c981bd70 0x55c2c981e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:34.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.608+0000 7f2404592740 4 client.15582 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:04:34.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.608+0000 7f2404592740 1 -- 172.21.15.116:0/3140791034 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2c976ed70 con 0x55c2c981bd70 2024-09-12T03:04:34.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.608+0000 7f23fb7fe640 1 --2- 172.21.15.116:0/3140791034 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c2c981bd70 0x55c2c981e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:34.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.608+0000 7f23daffd640 10 client.15582 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:04:34.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.608+0000 7f23fb7fe640 1 --2- 172.21.15.116:0/3140791034 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c2c981bd70 0x55c2c981e160 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1efa7dd0a2cab2d8 server_cookie=9acb8fbd6967ef9a in_seq=0 out_seq=0 2024-09-12T03:04:34.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.609+0000 7f23daffd640 1 -- 172.21.15.116:0/3140791034 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c2c976ed70 con 0x55c2c981bd70 2024-09-12T03:04:34.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.609+0000 7f23daffd640 10 client.15582 handle_command_reply: tid=0 2024-09-12T03:04:34.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.609+0000 7f2404592740 1 client.15582 shutdown 2024-09-12T03:04:34.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.609+0000 7f2404592740 1 -- 172.21.15.116:0/3140791034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c9802b40 msgr2=0x55c2c9808f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:34.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.609+0000 7f2404592740 1 --2- 172.21.15.116:0/3140791034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c9802b40 0x55c2c9808f90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f23e400b470 tx=0x7f23e40046c0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.609+0000 7f2404592740 1 -- 172.21.15.116:0/3140791034 shutdown_connections 2024-09-12T03:04:34.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.609+0000 7f2404592740 1 --2- 172.21.15.116:0/3140791034 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c2c981bd70 0x55c2c981e160 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.609+0000 7f2404592740 1 --2- 172.21.15.116:0/3140791034 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c980c370 0x55c2c98094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.609+0000 7f23daffd640 0 client.15582 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:04:34.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.609+0000 7f2404592740 1 --2- 172.21.15.116:0/3140791034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c9802b40 0x55c2c9808f90 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.609+0000 7f2404592740 1 --2- 172.21.15.116:0/3140791034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c98015d0 0x55c2c980bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.609+0000 7f2404592740 1 -- 172.21.15.116:0/3140791034 >> 172.21.15.116:0/3140791034 conn(0x55c2c97ca310 msgr2=0x55c2c9803990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:34.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.609+0000 7f2404592740 1 -- 172.21.15.116:0/3140791034 shutdown_connections 2024-09-12T03:04:34.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.609+0000 7f2404592740 1 -- 172.21.15.116:0/3140791034 wait complete. 2024-09-12T03:04:34.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.609+0000 7f2404592740 20 client.15582 trim_cache size 0 max 16384 2024-09-12T03:04:34.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.622+0000 7f2404592740 1 Processor -- start 2024-09-12T03:04:34.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.622+0000 7f2404592740 1 -- start start 2024-09-12T03:04:34.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.622+0000 7f2404592740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c98015d0 0x55c2c980a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:34.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.622+0000 7f2404592740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c9802b40 0x55c2c9821180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:34.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.622+0000 7f2404592740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c980c370 0x55c2c98216c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:34.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.622+0000 7f2404592740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc1b20e0 con 0x55c2c9802b40 2024-09-12T03:04:34.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.622+0000 7f2404592740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc1af9b0 con 0x55c2c980c370 2024-09-12T03:04:34.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.622+0000 7f2404592740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc1ad820 con 0x55c2c98015d0 2024-09-12T03:04:34.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.622+0000 7f23f2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c9802b40 0x55c2c9821180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:34.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.623+0000 7f23f2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c9802b40 0x55c2c9821180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39220/0 (socket says 172.21.15.116:39220) 2024-09-12T03:04:34.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.623+0000 7f23f2ffd640 1 -- 172.21.15.116:0/3991660425 learned_addr learned my addr 172.21.15.116:0/3991660425 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:34.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.623+0000 7f23fbfff640 1 --2- 172.21.15.116:0/3991660425 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c980c370 0x55c2c98216c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:34.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.623+0000 7f23f2ffd640 1 -- 172.21.15.116:0/3991660425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c98015d0 msgr2=0x55c2c980a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:34.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.623+0000 7f23f2ffd640 1 --2- 172.21.15.116:0/3991660425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c98015d0 0x55c2c980a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.623+0000 7f23f2ffd640 1 -- 172.21.15.116:0/3991660425 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c980c370 msgr2=0x55c2c98216c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:34.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.623+0000 7f23f2ffd640 1 --2- 172.21.15.116:0/3991660425 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c980c370 0x55c2c98216c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.623+0000 7f23f2ffd640 1 -- 172.21.15.116:0/3991660425 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2c97fc310 con 0x55c2c9802b40 2024-09-12T03:04:34.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.623+0000 7f23f2ffd640 1 --2- 172.21.15.116:0/3991660425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c9802b40 0x55c2c9821180 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f23e4017460 tx=0x7f23e40047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c0a0dd30019b9c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:34.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.623+0000 7f23f17fa640 1 -- 172.21.15.116:0/3991660425 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e409a740 con 0x55c2c9802b40 2024-09-12T03:04:34.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.623+0000 7f23f17fa640 1 -- 172.21.15.116:0/3991660425 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23e409efb0 con 0x55c2c9802b40 2024-09-12T03:04:34.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.623+0000 7f23f17fa640 1 -- 172.21.15.116:0/3991660425 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e409a9d0 con 0x55c2c9802b40 2024-09-12T03:04:34.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.624+0000 7f2404592740 1 -- 172.21.15.116:0/3991660425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c9802b40 msgr2=0x55c2c9821180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:34.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.624+0000 7f2404592740 1 --2- 172.21.15.116:0/3991660425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c9802b40 0x55c2c9821180 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f23e4017460 tx=0x7f23e40047d0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.624+0000 7f2404592740 1 -- 172.21.15.116:0/3991660425 shutdown_connections 2024-09-12T03:04:34.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.624+0000 7f2404592740 1 --2- 172.21.15.116:0/3991660425 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c980c370 0x55c2c98216c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.624+0000 7f2404592740 1 --2- 172.21.15.116:0/3991660425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c9802b40 0x55c2c9821180 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.624+0000 7f2404592740 1 --2- 172.21.15.116:0/3991660425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c98015d0 0x55c2c980a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.624+0000 7f2404592740 1 -- 172.21.15.116:0/3991660425 >> 172.21.15.116:0/3991660425 conn(0x55c2c97ca310 msgr2=0x55c2c9807f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:34.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.624+0000 7f2404592740 1 -- 172.21.15.116:0/3991660425 shutdown_connections 2024-09-12T03:04:34.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.624+0000 7f2404592740 1 -- 172.21.15.116:0/3991660425 wait complete. 2024-09-12T03:04:34.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.624+0000 7f2404592740 1 -- start start 2024-09-12T03:04:34.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.624+0000 7f2404592740 1 Processor -- start 2024-09-12T03:04:34.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.625+0000 7f2404592740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c98015d0 0x55c2c981ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:34.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.625+0000 7f2404592740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c9802b40 0x55c2c981f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:34.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.625+0000 7f2404592740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c980c370 0x55c2c981f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.625+0000 7f2404592740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc1ade70 con 0x55c2c980c370 2024-09-12T03:04:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.625+0000 7f2404592740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc1af2e0 con 0x55c2c98015d0 2024-09-12T03:04:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.625+0000 7f2404592740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc1ae540 con 0x55c2c9802b40 2024-09-12T03:04:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.625+0000 7f2404592740 10 client.0 _ll_register_callbacks cb 0x7f23e4096d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.625+0000 7f23fbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c980c370 0x55c2c981f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.625+0000 7f23fbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c980c370 0x55c2c981f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39232/0 (socket says 172.21.15.116:39232) 2024-09-12T03:04:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.625+0000 7f23fbfff640 1 -- 172.21.15.116:0/4180750704 learned_addr learned my addr 172.21.15.116:0/4180750704 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.625+0000 7f23fb7fe640 1 --2- 172.21.15.116:0/4180750704 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c98015d0 0x55c2c981ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.625+0000 7f23f2ffd640 1 --2- 172.21.15.116:0/4180750704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c9802b40 0x55c2c981f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.625+0000 7f23fbfff640 1 -- 172.21.15.116:0/4180750704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c9802b40 msgr2=0x55c2c981f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.625+0000 7f23fbfff640 1 --2- 172.21.15.116:0/4180750704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c9802b40 0x55c2c981f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.625+0000 7f23fbfff640 1 -- 172.21.15.116:0/4180750704 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c98015d0 msgr2=0x55c2c981ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.625+0000 7f23fbfff640 1 --2- 172.21.15.116:0/4180750704 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c98015d0 0x55c2c981ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.625+0000 7f23fbfff640 1 -- 172.21.15.116:0/4180750704 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2c97fd510 con 0x55c2c980c370 2024-09-12T03:04:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.625+0000 7f23fbfff640 1 --2- 172.21.15.116:0/4180750704 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c980c370 0x55c2c981f910 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f23f4007eb0 tx=0x7f23f400d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f9951aef734cab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.625+0000 7f23daffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:04:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.625+0000 7f23daffd640 1 -- 172.21.15.116:0/4180750704 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4027070 con 0x55c2c980c370 2024-09-12T03:04:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.625+0000 7f23daffd640 1 -- 172.21.15.116:0/4180750704 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f23f4023610 con 0x55c2c980c370 2024-09-12T03:04:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.625+0000 7f2404592740 1 -- 172.21.15.116:0/4180750704 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23e409efb0 con 0x55c2c980c370 2024-09-12T03:04:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.626+0000 7f23daffd640 1 -- 172.21.15.116:0/4180750704 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f23f4033050 con 0x55c2c980c370 2024-09-12T03:04:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.626+0000 7f2404592740 10 client.15588 fetch_fsmap learned FSMap version 25 2024-09-12T03:04:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.626+0000 7f2404592740 1 -- 172.21.15.116:0/4180750704 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c2c98867a0 con 0x55c2c980c370 2024-09-12T03:04:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.626+0000 7f23daffd640 1 -- 172.21.15.116:0/4180750704 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23e409efb0 con 0x55c2c980c370 2024-09-12T03:04:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.626+0000 7f23daffd640 1 -- 172.21.15.116:0/4180750704 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f23f4022040 con 0x55c2c980c370 2024-09-12T03:04:34.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.626+0000 7f2404592740 10 client.15588 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:04:34.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.626+0000 7f2404592740 10 client.15588 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:04:34.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.626+0000 7f2404592740 20 client.15588 populate_metadata read hostname 'smithi116' 2024-09-12T03:04:34.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.626+0000 7f2404592740 1 --2- 172.21.15.116:0/4180750704 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c2c9886f20 0x55c2c9889340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:34.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.626+0000 7f2404592740 4 client.15588 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:04:34.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.626+0000 7f2404592740 1 -- 172.21.15.116:0/4180750704 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c2c96eb170 con 0x55c2c9886f20 2024-09-12T03:04:34.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.626+0000 7f23fb7fe640 1 --2- 172.21.15.116:0/4180750704 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c2c9886f20 0x55c2c9889340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:34.645 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:04:34.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.626+0000 7f23fb7fe640 1 --2- 172.21.15.116:0/4180750704 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c2c9886f20 0x55c2c9889340 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=56bf4205721345ab server_cookie=d31052735cec58e0 in_seq=0 out_seq=0 2024-09-12T03:04:34.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.626+0000 7f23daffd640 10 client.15588 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:04:34.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.627+0000 7f23daffd640 1 -- 172.21.15.116:0/4180750704 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c2c96eb170 con 0x55c2c9886f20 2024-09-12T03:04:34.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.627+0000 7f23daffd640 10 client.15588 handle_command_reply: tid=0 2024-09-12T03:04:34.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.627+0000 7f2404592740 1 client.15588 shutdown 2024-09-12T03:04:34.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.627+0000 7f2404592740 1 -- 172.21.15.116:0/4180750704 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c980c370 msgr2=0x55c2c981f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:34.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.627+0000 7f2404592740 1 --2- 172.21.15.116:0/4180750704 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c980c370 0x55c2c981f910 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f23f4007eb0 tx=0x7f23f400d040 comp rx=0 tx=0).stop 2024-09-12T03:04:34.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.627+0000 7f2404592740 1 -- 172.21.15.116:0/4180750704 shutdown_connections 2024-09-12T03:04:34.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.627+0000 7f2404592740 1 --2- 172.21.15.116:0/4180750704 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c2c9886f20 0x55c2c9889340 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.627+0000 7f2404592740 1 --2- 172.21.15.116:0/4180750704 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c980c370 0x55c2c981f910 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.627+0000 7f2404592740 1 --2- 172.21.15.116:0/4180750704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c9802b40 0x55c2c981f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.627+0000 7f2404592740 1 --2- 172.21.15.116:0/4180750704 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c98015d0 0x55c2c981ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.627+0000 7f23daffd640 0 client.15588 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:04:34.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.627+0000 7f2404592740 1 -- 172.21.15.116:0/4180750704 >> 172.21.15.116:0/4180750704 conn(0x55c2c97ca310 msgr2=0x55c2c97fc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:34.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.627+0000 7f2404592740 1 -- 172.21.15.116:0/4180750704 shutdown_connections 2024-09-12T03:04:34.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.627+0000 7f2404592740 1 -- 172.21.15.116:0/4180750704 wait complete. 2024-09-12T03:04:34.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.627+0000 7f2404592740 20 client.15588 trim_cache size 0 max 16384 2024-09-12T03:04:34.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.627+0000 7f24021b9640 1 -- 172.21.15.116:0/631374896 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f23e80a2b80 msgr2=0x7f23e80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:34.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.627+0000 7f24021b9640 1 --2- 172.21.15.116:0/631374896 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f23e80a2b80 0x7f23e80a5040 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f23f400a5a0 tx=0x7f23f4004b50 comp rx=0 tx=0).stop 2024-09-12T03:04:34.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.627+0000 7f24021b9640 1 -- 172.21.15.116:0/631374896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23fc102570 msgr2=0x7f23fc0fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:34.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.627+0000 7f24021b9640 1 --2- 172.21.15.116:0/631374896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23fc102570 0x7f23fc0fd1c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f23e400b4a0 tx=0x7f23e400b960 comp rx=0 tx=0).stop 2024-09-12T03:04:34.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.628+0000 7f24021b9640 1 -- 172.21.15.116:0/631374896 shutdown_connections 2024-09-12T03:04:34.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.628+0000 7f24021b9640 1 --2- 172.21.15.116:0/631374896 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f23e80a2b80 0x7f23e80a5040 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.628+0000 7f24021b9640 1 --2- 172.21.15.116:0/631374896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23fc102ef0 0x7f23fc0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.628+0000 7f24021b9640 1 --2- 172.21.15.116:0/631374896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23fc102570 0x7f23fc0fd1c0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.628+0000 7f24021b9640 1 --2- 172.21.15.116:0/631374896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23fc100e20 0x7f23fc0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:34.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.628+0000 7f24021b9640 1 -- 172.21.15.116:0/631374896 >> 172.21.15.116:0/631374896 conn(0x7f23fc0fc7d0 msgr2=0x7f23fc06a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:34.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.628+0000 7f24021b9640 1 -- 172.21.15.116:0/631374896 shutdown_connections 2024-09-12T03:04:34.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:34.628+0000 7f24021b9640 1 -- 172.21.15.116:0/631374896 wait complete. 2024-09-12T03:04:34.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:34 smithi183 ceph-mon[35535]: pgmap v385: 97 pgs: 97 active+clean; 803 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:04:35.353 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:04:35.597 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:36.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.085+0000 7f28d488b640 1 -- 172.21.15.116:0/4177350077 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f28d0106a80 msgr2=0x7f28d0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:36.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.085+0000 7f28d488b640 1 --2- 172.21.15.116:0/4177350077 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f28d0106a80 0x7f28d0106e60 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f28c0009930 tx=0x7f28c002a420 comp rx=0 tx=0).stop 2024-09-12T03:04:36.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.085+0000 7f28d488b640 1 -- 172.21.15.116:0/4177350077 shutdown_connections 2024-09-12T03:04:36.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.085+0000 7f28d488b640 1 --2- 172.21.15.116:0/4177350077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28d0101420 0x7f28d010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.085+0000 7f28d488b640 1 --2- 172.21.15.116:0/4177350077 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28d0100a80 0x7f28d0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.085+0000 7f28d488b640 1 --2- 172.21.15.116:0/4177350077 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f28d0106a80 0x7f28d0106e60 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.085+0000 7f28d488b640 1 -- 172.21.15.116:0/4177350077 >> 172.21.15.116:0/4177350077 conn(0x7f28d00fc7a0 msgr2=0x7f28d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:36.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.085+0000 7f28d488b640 1 -- 172.21.15.116:0/4177350077 shutdown_connections 2024-09-12T03:04:36.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.085+0000 7f28d488b640 1 -- 172.21.15.116:0/4177350077 wait complete. 2024-09-12T03:04:36.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.086+0000 7f28d488b640 1 Processor -- start 2024-09-12T03:04:36.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.086+0000 7f28d488b640 1 -- start start 2024-09-12T03:04:36.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.086+0000 7f28d488b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28d0100a80 0x7f28d0197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:36.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.086+0000 7f28d488b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28d0101420 0x7f28d01975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:36.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.086+0000 7f28d488b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f28d0106a80 0x7f28d0197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:36.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.086+0000 7f28d488b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f28d0110e80 con 0x7f28d0106a80 2024-09-12T03:04:36.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.086+0000 7f28d488b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f28d0110d00 con 0x7f28d0101420 2024-09-12T03:04:36.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.086+0000 7f28d488b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f28d0111000 con 0x7f28d0100a80 2024-09-12T03:04:36.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.086+0000 7f28ced9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f28d0106a80 0x7f28d0197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:36.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.086+0000 7f28ced9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f28d0106a80 0x7f28d0197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39260/0 (socket says 172.21.15.116:39260) 2024-09-12T03:04:36.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.086+0000 7f28ced9d640 1 -- 172.21.15.116:0/1114441000 learned_addr learned my addr 172.21.15.116:0/1114441000 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:36.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.086+0000 7f28ce59c640 1 --2- 172.21.15.116:0/1114441000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28d0100a80 0x7f28d0197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:36.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.086+0000 7f28c7fff640 1 --2- 172.21.15.116:0/1114441000 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28d0101420 0x7f28d01975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:36.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.086+0000 7f28ced9d640 1 -- 172.21.15.116:0/1114441000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28d0100a80 msgr2=0x7f28d0197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:36.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.086+0000 7f28ced9d640 1 --2- 172.21.15.116:0/1114441000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28d0100a80 0x7f28d0197080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.086+0000 7f28ced9d640 1 -- 172.21.15.116:0/1114441000 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28d0101420 msgr2=0x7f28d01975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:36.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.086+0000 7f28ced9d640 1 --2- 172.21.15.116:0/1114441000 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28d0101420 0x7f28d01975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.086+0000 7f28ced9d640 1 -- 172.21.15.116:0/1114441000 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d01a68f0 con 0x7f28d0106a80 2024-09-12T03:04:36.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.087+0000 7f28ced9d640 1 --2- 172.21.15.116:0/1114441000 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f28d0106a80 0x7f28d0197b00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f28c800eb10 tx=0x7f28c800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:36.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.087+0000 7f28c77fe640 1 -- 172.21.15.116:0/1114441000 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c8004b20 con 0x7f28d0106a80 2024-09-12T03:04:36.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.087+0000 7f28d488b640 1 -- 172.21.15.116:0/1114441000 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d01a6be0 con 0x7f28d0106a80 2024-09-12T03:04:36.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.087+0000 7f28c77fe640 1 -- 172.21.15.116:0/1114441000 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28c8004cc0 con 0x7f28d0106a80 2024-09-12T03:04:36.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.087+0000 7f28c77fe640 1 -- 172.21.15.116:0/1114441000 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c8010760 con 0x7f28d0106a80 2024-09-12T03:04:36.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.087+0000 7f28d488b640 1 -- 172.21.15.116:0/1114441000 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d01a7120 con 0x7f28d0106a80 2024-09-12T03:04:36.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.089+0000 7f28c77fe640 1 -- 172.21.15.116:0/1114441000 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f28c8005020 con 0x7f28d0106a80 2024-09-12T03:04:36.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.089+0000 7f28d6c64740 1 Processor -- start 2024-09-12T03:04:36.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.090+0000 7f28d6c64740 1 -- start start 2024-09-12T03:04:36.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.090+0000 7f28d6c64740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e9a65b40 0x5614e9a64090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:36.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.090+0000 7f28d6c64740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614e9a645d0 0x5614e9a64a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:36.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.090+0000 7f28d6c64740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614e9a6f370 0x5614e9a71760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:36.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.090+0000 7f28d6c64740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5614e99ca2f0 con 0x5614e9a6f370 2024-09-12T03:04:36.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.090+0000 7f28d6c64740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5614e9971cf0 con 0x5614e9a65b40 2024-09-12T03:04:36.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.090+0000 7f28d6c64740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5614e99c62b0 con 0x5614e9a645d0 2024-09-12T03:04:36.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.090+0000 7f28c77fe640 1 --2- 172.21.15.116:0/1114441000 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f28bc0a2b80 0x7f28bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:36.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.090+0000 7f28c7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614e9a6f370 0x5614e9a71760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:36.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.090+0000 7f28c7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614e9a6f370 0x5614e9a71760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39272/0 (socket says 172.21.15.116:39272) 2024-09-12T03:04:36.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.090+0000 7f28c7fff640 1 -- 172.21.15.116:0/1454041607 learned_addr learned my addr 172.21.15.116:0/1454041607 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:36.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.090+0000 7f28ced9d640 1 --2- 172.21.15.116:0/1114441000 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f28bc0a2b80 0x7f28bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:36.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.090+0000 7f28ce59c640 1 --2- 172.21.15.116:0/1454041607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614e9a645d0 0x5614e9a64a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:36.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.090+0000 7f28c7fff640 1 --2- 172.21.15.116:0/1454041607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e9a65b40 0x5614e9a64090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:36.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.090+0000 7f28c77fe640 1 -- 172.21.15.116:0/1114441000 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f28c80cf6f0 con 0x7f28d0106a80 2024-09-12T03:04:36.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.090+0000 7f28c7fff640 1 -- 172.21.15.116:0/1454041607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614e9a645d0 msgr2=0x5614e9a64a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:36.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.090+0000 7f28c7fff640 1 --2- 172.21.15.116:0/1454041607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614e9a645d0 0x5614e9a64a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.090+0000 7f28c7fff640 1 -- 172.21.15.116:0/1454041607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e9a65b40 msgr2=0x5614e9a64090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:36.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.090+0000 7f28c7fff640 1 --2- 172.21.15.116:0/1454041607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e9a65b40 0x5614e9a64090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.090+0000 7f28c7fff640 1 -- 172.21.15.116:0/1454041607 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614e99563a0 con 0x5614e9a6f370 2024-09-12T03:04:36.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.090+0000 7f28ce59c640 1 --2- 172.21.15.116:0/1454041607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614e9a645d0 0x5614e9a64a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:36.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.091+0000 7f28c7fff640 1 --2- 172.21.15.116:0/1454041607 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614e9a6f370 0x5614e9a71760 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f28b8013580 tx=0x7f28b800b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16c43868fdc0b26 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:36.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.091+0000 7f28ced9d640 1 --2- 172.21.15.116:0/1114441000 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f28bc0a2b80 0x7f28bc0a5040 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f28c809bc80 tx=0x7f28c8066ff0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:36.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.091+0000 7f28c4ff9640 1 -- 172.21.15.116:0/1454041607 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8019030 con 0x5614e9a6f370 2024-09-12T03:04:36.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.091+0000 7f28c4ff9640 1 -- 172.21.15.116:0/1454041607 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28b8011920 con 0x5614e9a6f370 2024-09-12T03:04:36.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.091+0000 7f28c4ff9640 1 -- 172.21.15.116:0/1454041607 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b801d880 con 0x5614e9a6f370 2024-09-12T03:04:36.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.091+0000 7f28d6c64740 1 -- 172.21.15.116:0/1454041607 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614e9a6f370 msgr2=0x5614e9a71760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:36.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.091+0000 7f28d6c64740 1 --2- 172.21.15.116:0/1454041607 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614e9a6f370 0x5614e9a71760 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f28b8013580 tx=0x7f28b800b690 comp rx=0 tx=0).stop 2024-09-12T03:04:36.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.092+0000 7f28d6c64740 1 -- 172.21.15.116:0/1454041607 shutdown_connections 2024-09-12T03:04:36.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.092+0000 7f28d6c64740 1 --2- 172.21.15.116:0/1454041607 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614e9a6f370 0x5614e9a71760 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.092+0000 7f28d6c64740 1 --2- 172.21.15.116:0/1454041607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614e9a645d0 0x5614e9a64a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.092+0000 7f28d6c64740 1 --2- 172.21.15.116:0/1454041607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e9a65b40 0x5614e9a64090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.092+0000 7f28d6c64740 1 -- 172.21.15.116:0/1454041607 >> 172.21.15.116:0/1454041607 conn(0x5614e9a2d310 msgr2=0x5614e9a60c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:36.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.092+0000 7f28d6c64740 1 -- 172.21.15.116:0/1454041607 shutdown_connections 2024-09-12T03:04:36.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.092+0000 7f28d6c64740 1 -- 172.21.15.116:0/1454041607 wait complete. 2024-09-12T03:04:36.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.092+0000 7f28d6c64740 1 -- start start 2024-09-12T03:04:36.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.092+0000 7f28d6c64740 1 Processor -- start 2024-09-12T03:04:36.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.092+0000 7f28d6c64740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614e9a645d0 0x5614e9a6ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:36.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.092+0000 7f28d6c64740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614e9a65b40 0x5614e9a6bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:36.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.092+0000 7f28d6c64740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e9a6f370 0x5614e9a6c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:36.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.092+0000 7f28d6c64740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5614e9a72070 con 0x5614e9a65b40 2024-09-12T03:04:36.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.092+0000 7f28d6c64740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5614e9985060 con 0x5614e9a6f370 2024-09-12T03:04:36.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.092+0000 7f28d6c64740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5614e9a721f0 con 0x5614e9a645d0 2024-09-12T03:04:36.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.092+0000 7f28d6c64740 10 client.0 _ll_register_callbacks cb 0x5614e96d88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:36.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.092+0000 7f28ce59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614e9a65b40 0x5614e9a6bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:36.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.092+0000 7f28ce59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614e9a65b40 0x5614e9a6bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39278/0 (socket says 172.21.15.116:39278) 2024-09-12T03:04:36.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.092+0000 7f28ce59c640 1 -- 172.21.15.116:0/1305968675 learned_addr learned my addr 172.21.15.116:0/1305968675 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:36.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.093+0000 7f28c7fff640 1 --2- 172.21.15.116:0/1305968675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614e9a645d0 0x5614e9a6ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:36.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.093+0000 7f28c7fff640 1 --2- 172.21.15.116:0/1305968675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e9a6f370 0x5614e9a6c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:36.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.093+0000 7f28ce59c640 1 -- 172.21.15.116:0/1305968675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614e9a645d0 msgr2=0x5614e9a6ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:36.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.093+0000 7f28ce59c640 1 --2- 172.21.15.116:0/1305968675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614e9a645d0 0x5614e9a6ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.093+0000 7f28ce59c640 1 -- 172.21.15.116:0/1305968675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e9a6f370 msgr2=0x5614e9a6c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:36.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.093+0000 7f28ce59c640 1 --2- 172.21.15.116:0/1305968675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e9a6f370 0x5614e9a6c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.093+0000 7f28ce59c640 1 -- 172.21.15.116:0/1305968675 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614e973b3c0 con 0x5614e9a65b40 2024-09-12T03:04:36.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.093+0000 7f28c7fff640 1 --2- 172.21.15.116:0/1305968675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e9a6f370 0x5614e9a6c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:36.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.093+0000 7f28ce59c640 1 --2- 172.21.15.116:0/1305968675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614e9a65b40 0x5614e9a6bff0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f28c002a940 tx=0x7f28c00317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff50d3582987391c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:36.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.093+0000 7f28b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:04:36.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.093+0000 7f28b5ffb640 1 -- 172.21.15.116:0/1305968675 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c0035040 con 0x5614e9a65b40 2024-09-12T03:04:36.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.093+0000 7f28b5ffb640 1 -- 172.21.15.116:0/1305968675 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f28c0004240 con 0x5614e9a65b40 2024-09-12T03:04:36.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.093+0000 7f28d6c64740 1 -- 172.21.15.116:0/1305968675 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28b8011920 con 0x5614e9a65b40 2024-09-12T03:04:36.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.093+0000 7f28b5ffb640 1 -- 172.21.15.116:0/1305968675 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f28c004c070 con 0x5614e9a65b40 2024-09-12T03:04:36.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.093+0000 7f28d6c64740 10 client.15600 fetch_fsmap learned FSMap version 25 2024-09-12T03:04:36.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.093+0000 7f28d6c64740 1 -- 172.21.15.116:0/1305968675 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5614e9a7dae0 con 0x5614e9a65b40 2024-09-12T03:04:36.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.093+0000 7f28b5ffb640 1 -- 172.21.15.116:0/1305968675 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28b8011920 con 0x5614e9a65b40 2024-09-12T03:04:36.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.094+0000 7f28b5ffb640 1 -- 172.21.15.116:0/1305968675 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f28c0045510 con 0x5614e9a65b40 2024-09-12T03:04:36.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.094+0000 7f28d6c64740 10 client.15600 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:04:36.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.094+0000 7f28d6c64740 10 client.15600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:04:36.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.094+0000 7f28d6c64740 20 client.15600 populate_metadata read hostname 'smithi116' 2024-09-12T03:04:36.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.094+0000 7f28d6c64740 1 --2- 172.21.15.116:0/1305968675 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5614e9a7efa0 0x5614e9a813e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:36.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.094+0000 7f28d6c64740 4 client.15600 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:04:36.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.094+0000 7f28d6c64740 1 -- 172.21.15.116:0/1305968675 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614e99d7860 con 0x5614e9a7efa0 2024-09-12T03:04:36.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.094+0000 7f28c7fff640 1 --2- 172.21.15.116:0/1305968675 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5614e9a7efa0 0x5614e9a813e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:36.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.094+0000 7f28c7fff640 1 --2- 172.21.15.116:0/1305968675 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5614e9a7efa0 0x5614e9a813e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecb008eeff78a701 server_cookie=3c1204c695a7f78 in_seq=0 out_seq=0 2024-09-12T03:04:36.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.094+0000 7f28b5ffb640 10 client.15600 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:36.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.095+0000 7f28b5ffb640 1 -- 172.21.15.116:0/1305968675 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5614e99d7860 con 0x5614e9a7efa0 2024-09-12T03:04:36.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.095+0000 7f28b5ffb640 10 client.15600 handle_command_reply: tid=0 2024-09-12T03:04:36.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.095+0000 7f28d6c64740 1 client.15600 shutdown 2024-09-12T03:04:36.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.095+0000 7f28d6c64740 1 -- 172.21.15.116:0/1305968675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614e9a65b40 msgr2=0x5614e9a6bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:36.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.095+0000 7f28d6c64740 1 --2- 172.21.15.116:0/1305968675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614e9a65b40 0x5614e9a6bff0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f28c002a940 tx=0x7f28c00317e0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.095+0000 7f28d6c64740 1 -- 172.21.15.116:0/1305968675 shutdown_connections 2024-09-12T03:04:36.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.095+0000 7f28d6c64740 1 --2- 172.21.15.116:0/1305968675 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5614e9a7efa0 0x5614e9a813e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.095+0000 7f28d6c64740 1 --2- 172.21.15.116:0/1305968675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e9a6f370 0x5614e9a6c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.095+0000 7f28b5ffb640 0 client.15600 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:36.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.095+0000 7f28d6c64740 1 --2- 172.21.15.116:0/1305968675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614e9a65b40 0x5614e9a6bff0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.095+0000 7f28d6c64740 1 --2- 172.21.15.116:0/1305968675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614e9a645d0 0x5614e9a6ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.095+0000 7f28d6c64740 1 -- 172.21.15.116:0/1305968675 >> 172.21.15.116:0/1305968675 conn(0x5614e9a2d310 msgr2=0x5614e9a66960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:36.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.095+0000 7f28d6c64740 1 -- 172.21.15.116:0/1305968675 shutdown_connections 2024-09-12T03:04:36.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.095+0000 7f28d6c64740 1 -- 172.21.15.116:0/1305968675 wait complete. 2024-09-12T03:04:36.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.096+0000 7f28d6c64740 20 client.15600 trim_cache size 0 max 16384 2024-09-12T03:04:36.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.108+0000 7f28d6c64740 1 Processor -- start 2024-09-12T03:04:36.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.108+0000 7f28d6c64740 1 -- start start 2024-09-12T03:04:36.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.108+0000 7f28d6c64740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614e9a645d0 0x5614e9a60d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:36.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.108+0000 7f28d6c64740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614e9a65b40 0x5614e9a7b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:36.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.108+0000 7f28d6c64740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e9a6f370 0x5614e9a6afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:36.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.108+0000 7f28d6c64740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f28d01b20a0 con 0x5614e9a645d0 2024-09-12T03:04:36.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.108+0000 7f28d6c64740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f28d01af970 con 0x5614e9a6f370 2024-09-12T03:04:36.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.108+0000 7f28d6c64740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f28d01ad7c0 con 0x5614e9a65b40 2024-09-12T03:04:36.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.109+0000 7f28c7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614e9a645d0 0x5614e9a60d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:36.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.109+0000 7f28c7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614e9a645d0 0x5614e9a60d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39284/0 (socket says 172.21.15.116:39284) 2024-09-12T03:04:36.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.109+0000 7f28c7fff640 1 -- 172.21.15.116:0/1871577378 learned_addr learned my addr 172.21.15.116:0/1871577378 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:36.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.109+0000 7f28c7fff640 1 -- 172.21.15.116:0/1871577378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614e9a65b40 msgr2=0x5614e9a7b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:36.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.109+0000 7f28c7fff640 1 --2- 172.21.15.116:0/1871577378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614e9a65b40 0x5614e9a7b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.109+0000 7f28c7fff640 1 -- 172.21.15.116:0/1871577378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e9a6f370 msgr2=0x5614e9a6afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:36.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.109+0000 7f28c7fff640 1 --2- 172.21.15.116:0/1871577378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e9a6f370 0x5614e9a6afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.109+0000 7f28c7fff640 1 -- 172.21.15.116:0/1871577378 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614e9a67120 con 0x5614e9a645d0 2024-09-12T03:04:36.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.109+0000 7f28c7fff640 1 --2- 172.21.15.116:0/1871577378 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614e9a645d0 0x5614e9a60d70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f28b8013580 tx=0x7f28b8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b562c035fade65 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:36.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.109+0000 7f28b7fff640 1 -- 172.21.15.116:0/1871577378 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8031070 con 0x5614e9a645d0 2024-09-12T03:04:36.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.109+0000 7f28b7fff640 1 -- 172.21.15.116:0/1871577378 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28b8005040 con 0x5614e9a645d0 2024-09-12T03:04:36.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.109+0000 7f28b7fff640 1 -- 172.21.15.116:0/1871577378 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8019040 con 0x5614e9a645d0 2024-09-12T03:04:36.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.110+0000 7f28d6c64740 1 -- 172.21.15.116:0/1871577378 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614e9a645d0 msgr2=0x5614e9a60d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:36.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.110+0000 7f28d6c64740 1 --2- 172.21.15.116:0/1871577378 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614e9a645d0 0x5614e9a60d70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f28b8013580 tx=0x7f28b8005850 comp rx=0 tx=0).stop 2024-09-12T03:04:36.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.110+0000 7f28d6c64740 1 -- 172.21.15.116:0/1871577378 shutdown_connections 2024-09-12T03:04:36.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.110+0000 7f28d6c64740 1 --2- 172.21.15.116:0/1871577378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e9a6f370 0x5614e9a6afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.110+0000 7f28d6c64740 1 --2- 172.21.15.116:0/1871577378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614e9a65b40 0x5614e9a7b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.110+0000 7f28d6c64740 1 --2- 172.21.15.116:0/1871577378 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614e9a645d0 0x5614e9a60d70 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.110+0000 7f28d6c64740 1 -- 172.21.15.116:0/1871577378 >> 172.21.15.116:0/1871577378 conn(0x5614e9a2d310 msgr2=0x5614e9a6a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:36.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.110+0000 7f28d6c64740 1 -- 172.21.15.116:0/1871577378 shutdown_connections 2024-09-12T03:04:36.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.110+0000 7f28d6c64740 1 -- 172.21.15.116:0/1871577378 wait complete. 2024-09-12T03:04:36.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.110+0000 7f28d6c64740 1 -- start start 2024-09-12T03:04:36.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.110+0000 7f28d6c64740 1 Processor -- start 2024-09-12T03:04:36.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.110+0000 7f28d6c64740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614e9a645d0 0x5614e9a77a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:36.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.110+0000 7f28d6c64740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614e9a65b40 0x5614e9a68bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:36.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.110+0000 7f28d6c64740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e9a6f370 0x5614e9a69130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:36.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.110+0000 7f28d6c64740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f28d01ade30 con 0x5614e9a645d0 2024-09-12T03:04:36.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.110+0000 7f28d6c64740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f28d01af2a0 con 0x5614e9a6f370 2024-09-12T03:04:36.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.110+0000 7f28d6c64740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f28d01ae500 con 0x5614e9a65b40 2024-09-12T03:04:36.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.110+0000 7f28d6c64740 10 client.0 _ll_register_callbacks cb 0x7f28c0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:36.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.110+0000 7f28c7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614e9a645d0 0x5614e9a77a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:36.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.111+0000 7f28c7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614e9a645d0 0x5614e9a77a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39290/0 (socket says 172.21.15.116:39290) 2024-09-12T03:04:36.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.111+0000 7f28c7fff640 1 -- 172.21.15.116:0/619898252 learned_addr learned my addr 172.21.15.116:0/619898252 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:36.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.111+0000 7f28c7fff640 1 --2- 172.21.15.116:0/619898252 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e9a6f370 0x5614e9a69130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:36.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.111+0000 7f28ce59c640 1 --2- 172.21.15.116:0/619898252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614e9a65b40 0x5614e9a68bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:36.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.111+0000 7f28c7fff640 1 -- 172.21.15.116:0/619898252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614e9a65b40 msgr2=0x5614e9a68bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:36.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.111+0000 7f28c7fff640 1 --2- 172.21.15.116:0/619898252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614e9a65b40 0x5614e9a68bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.111+0000 7f28c7fff640 1 -- 172.21.15.116:0/619898252 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e9a6f370 msgr2=0x5614e9a69130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:36.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.111+0000 7f28c7fff640 1 --2- 172.21.15.116:0/619898252 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e9a6f370 0x5614e9a69130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.111+0000 7f28c7fff640 1 -- 172.21.15.116:0/619898252 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614e9a697d0 con 0x5614e9a645d0 2024-09-12T03:04:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.111+0000 7f28c7fff640 1 --2- 172.21.15.116:0/619898252 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614e9a645d0 0x5614e9a77a50 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f28b80045f0 tx=0x7f28b8005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c65dcbbf567b9c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.111+0000 7f28b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:04:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.111+0000 7f28b5ffb640 1 -- 172.21.15.116:0/619898252 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8019050 con 0x5614e9a645d0 2024-09-12T03:04:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.111+0000 7f28b5ffb640 1 -- 172.21.15.116:0/619898252 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f28b8004a20 con 0x5614e9a645d0 2024-09-12T03:04:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.111+0000 7f28d6c64740 1 -- 172.21.15.116:0/619898252 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28b8005040 con 0x5614e9a645d0 2024-09-12T03:04:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.111+0000 7f28b5ffb640 1 -- 172.21.15.116:0/619898252 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f28b8031040 con 0x5614e9a645d0 2024-09-12T03:04:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.111+0000 7f28d6c64740 10 client.15606 fetch_fsmap learned FSMap version 25 2024-09-12T03:04:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.111+0000 7f28d6c64740 1 -- 172.21.15.116:0/619898252 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5614e9ae9b30 con 0x5614e9a645d0 2024-09-12T03:04:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.112+0000 7f28b5ffb640 1 -- 172.21.15.116:0/619898252 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28b8005040 con 0x5614e9a645d0 2024-09-12T03:04:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.112+0000 7f28b5ffb640 1 -- 172.21.15.116:0/619898252 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f28b8037070 con 0x5614e9a645d0 2024-09-12T03:04:36.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.112+0000 7f28d6c64740 10 client.15606 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:04:36.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.112+0000 7f28d6c64740 10 client.15606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:04:36.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.112+0000 7f28d6c64740 20 client.15606 populate_metadata read hostname 'smithi116' 2024-09-12T03:04:36.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.112+0000 7f28d6c64740 1 --2- 172.21.15.116:0/619898252 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5614e9a7efa0 0x5614e9aec050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:36.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.112+0000 7f28d6c64740 4 client.15606 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:04:36.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.112+0000 7f28d6c64740 1 -- 172.21.15.116:0/619898252 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5614e99d1d70 con 0x5614e9a7efa0 2024-09-12T03:04:36.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.112+0000 7f28ce59c640 1 --2- 172.21.15.116:0/619898252 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5614e9a7efa0 0x5614e9aec050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:36.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.112+0000 7f28ce59c640 1 --2- 172.21.15.116:0/619898252 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5614e9a7efa0 0x5614e9aec050 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92a34f84f4e86d67 server_cookie=79ed823c484e604d in_seq=0 out_seq=0 2024-09-12T03:04:36.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.112+0000 7f28b5ffb640 10 client.15606 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:36.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.113+0000 7f28b5ffb640 1 -- 172.21.15.116:0/619898252 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5614e99d1d70 con 0x5614e9a7efa0 2024-09-12T03:04:36.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.113+0000 7f28b5ffb640 10 client.15606 handle_command_reply: tid=0 2024-09-12T03:04:36.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.113+0000 7f28d6c64740 1 client.15606 shutdown 2024-09-12T03:04:36.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.113+0000 7f28d6c64740 1 -- 172.21.15.116:0/619898252 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614e9a645d0 msgr2=0x5614e9a77a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:36.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.113+0000 7f28d6c64740 1 --2- 172.21.15.116:0/619898252 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614e9a645d0 0x5614e9a77a50 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f28b80045f0 tx=0x7f28b8005cc0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.113+0000 7f28d6c64740 1 -- 172.21.15.116:0/619898252 shutdown_connections 2024-09-12T03:04:36.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.113+0000 7f28d6c64740 1 --2- 172.21.15.116:0/619898252 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5614e9a7efa0 0x5614e9aec050 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.113+0000 7f28d6c64740 1 --2- 172.21.15.116:0/619898252 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e9a6f370 0x5614e9a69130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.113+0000 7f28b5ffb640 0 client.15606 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:36.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.113+0000 7f28d6c64740 1 --2- 172.21.15.116:0/619898252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614e9a65b40 0x5614e9a68bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.113+0000 7f28d6c64740 1 --2- 172.21.15.116:0/619898252 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614e9a645d0 0x5614e9a77a50 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.113+0000 7f28d6c64740 1 -- 172.21.15.116:0/619898252 >> 172.21.15.116:0/619898252 conn(0x5614e9a2d310 msgr2=0x5614e9a6b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:36.138 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:04:36.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.113+0000 7f28d6c64740 1 -- 172.21.15.116:0/619898252 shutdown_connections 2024-09-12T03:04:36.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.113+0000 7f28d6c64740 1 -- 172.21.15.116:0/619898252 wait complete. 2024-09-12T03:04:36.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.113+0000 7f28d6c64740 20 client.15606 trim_cache size 0 max 16384 2024-09-12T03:04:36.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.113+0000 7f28d488b640 1 -- 172.21.15.116:0/1114441000 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f28bc0a2b80 msgr2=0x7f28bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:36.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.113+0000 7f28d488b640 1 --2- 172.21.15.116:0/1114441000 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f28bc0a2b80 0x7f28bc0a5040 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f28c809bc80 tx=0x7f28c8066ff0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.113+0000 7f28d488b640 1 -- 172.21.15.116:0/1114441000 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f28d0106a80 msgr2=0x7f28d0197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:36.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.113+0000 7f28d488b640 1 --2- 172.21.15.116:0/1114441000 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f28d0106a80 0x7f28d0197b00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f28c800eb10 tx=0x7f28c800c2d0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.113+0000 7f28d488b640 1 -- 172.21.15.116:0/1114441000 shutdown_connections 2024-09-12T03:04:36.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.113+0000 7f28d488b640 1 --2- 172.21.15.116:0/1114441000 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f28bc0a2b80 0x7f28bc0a5040 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.113+0000 7f28d488b640 1 --2- 172.21.15.116:0/1114441000 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f28d0106a80 0x7f28d0197b00 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.113+0000 7f28d488b640 1 --2- 172.21.15.116:0/1114441000 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28d0101420 0x7f28d01975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.113+0000 7f28d488b640 1 --2- 172.21.15.116:0/1114441000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28d0100a80 0x7f28d0197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:36.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.113+0000 7f28d488b640 1 -- 172.21.15.116:0/1114441000 >> 172.21.15.116:0/1114441000 conn(0x7f28d00fc7a0 msgr2=0x7f28d00fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:36.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.114+0000 7f28d488b640 1 -- 172.21.15.116:0/1114441000 shutdown_connections 2024-09-12T03:04:36.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:36.114+0000 7f28d488b640 1 -- 172.21.15.116:0/1114441000 wait complete. 2024-09-12T03:04:36.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:36 smithi116 ceph-mon[31410]: pgmap v386: 97 pgs: 97 active+clean; 803 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:04:36.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:36 smithi177 ceph-mon[35305]: pgmap v386: 97 pgs: 97 active+clean; 803 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:04:36.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:36 smithi183 ceph-mon[35535]: pgmap v386: 97 pgs: 97 active+clean; 803 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:04:37.837 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:04:37.838 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:04:38.098 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:38.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:38 smithi177 ceph-mon[35305]: pgmap v387: 97 pgs: 97 active+clean; 858 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-12T03:04:38.495 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:38 smithi116 ceph-mon[31410]: pgmap v387: 97 pgs: 97 active+clean; 858 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-12T03:04:38.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.578+0000 7fcd761a6640 1 -- 172.21.15.116:0/1587681980 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd70101de0 msgr2=0x7fcd701021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:38.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.578+0000 7fcd761a6640 1 --2- 172.21.15.116:0/1587681980 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd70101de0 0x7fcd701021c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fcd60009900 tx=0x7fcd6002a410 comp rx=0 tx=0).stop 2024-09-12T03:04:38.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.578+0000 7fcd761a6640 1 -- 172.21.15.116:0/1587681980 shutdown_connections 2024-09-12T03:04:38.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.578+0000 7fcd761a6640 1 --2- 172.21.15.116:0/1587681980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd70106f10 0x7fcd7010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.578+0000 7fcd761a6640 1 --2- 172.21.15.116:0/1587681980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd70102790 0x7fcd701067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.578+0000 7fcd761a6640 1 --2- 172.21.15.116:0/1587681980 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd70101de0 0x7fcd701021c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.578+0000 7fcd761a6640 1 -- 172.21.15.116:0/1587681980 >> 172.21.15.116:0/1587681980 conn(0x7fcd700fd8c0 msgr2=0x7fcd700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:38.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.578+0000 7fcd761a6640 1 -- 172.21.15.116:0/1587681980 shutdown_connections 2024-09-12T03:04:38.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.578+0000 7fcd761a6640 1 -- 172.21.15.116:0/1587681980 wait complete. 2024-09-12T03:04:38.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.579+0000 7fcd761a6640 1 Processor -- start 2024-09-12T03:04:38.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.579+0000 7fcd761a6640 1 -- start start 2024-09-12T03:04:38.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.580+0000 7fcd761a6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd70101de0 0x7fcd70198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:38.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.580+0000 7fcd761a6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd70102790 0x7fcd701988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:38.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.580+0000 7fcd761a6640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd70106f10 0x7fcd7019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:38.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.580+0000 7fcd761a6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcd70111f30 con 0x7fcd70102790 2024-09-12T03:04:38.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.580+0000 7fcd761a6640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcd70111db0 con 0x7fcd70106f10 2024-09-12T03:04:38.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.580+0000 7fcd761a6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701120b0 con 0x7fcd70101de0 2024-09-12T03:04:38.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.580+0000 7fcd6effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd70102790 0x7fcd701988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:38.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.580+0000 7fcd6f7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd70101de0 0x7fcd70198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:38.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.580+0000 7fcd6effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd70102790 0x7fcd701988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39306/0 (socket says 172.21.15.116:39306) 2024-09-12T03:04:38.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.580+0000 7fcd6effd640 1 -- 172.21.15.116:0/993584403 learned_addr learned my addr 172.21.15.116:0/993584403 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:38.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.580+0000 7fcd6ffff640 1 --2- 172.21.15.116:0/993584403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd70106f10 0x7fcd7019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:38.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.580+0000 7fcd6effd640 1 -- 172.21.15.116:0/993584403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd70101de0 msgr2=0x7fcd70198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:38.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.580+0000 7fcd6effd640 1 --2- 172.21.15.116:0/993584403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd70101de0 0x7fcd70198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.580+0000 7fcd6effd640 1 -- 172.21.15.116:0/993584403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd70106f10 msgr2=0x7fcd7019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:38.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.580+0000 7fcd6effd640 1 --2- 172.21.15.116:0/993584403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd70106f10 0x7fcd7019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.580+0000 7fcd6effd640 1 -- 172.21.15.116:0/993584403 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7019d340 con 0x7fcd70102790 2024-09-12T03:04:38.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.580+0000 7fcd6effd640 1 --2- 172.21.15.116:0/993584403 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd70102790 0x7fcd701988d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fcd6400c920 tx=0x7fcd6400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:38.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.581+0000 7fcd6cff9640 1 -- 172.21.15.116:0/993584403 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd64007c70 con 0x7fcd70102790 2024-09-12T03:04:38.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.581+0000 7fcd761a6640 1 -- 172.21.15.116:0/993584403 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7019d630 con 0x7fcd70102790 2024-09-12T03:04:38.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.581+0000 7fcd6cff9640 1 -- 172.21.15.116:0/993584403 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd64007e10 con 0x7fcd70102790 2024-09-12T03:04:38.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.581+0000 7fcd6cff9640 1 -- 172.21.15.116:0/993584403 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd64018890 con 0x7fcd70102790 2024-09-12T03:04:38.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.581+0000 7fcd761a6640 1 -- 172.21.15.116:0/993584403 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd701a4f10 con 0x7fcd70102790 2024-09-12T03:04:38.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.583+0000 7fcd6cff9640 1 -- 172.21.15.116:0/993584403 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fcd64018a70 con 0x7fcd70102790 2024-09-12T03:04:38.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.583+0000 7fcd7857f740 1 Processor -- start 2024-09-12T03:04:38.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.583+0000 7fcd7857f740 1 -- start start 2024-09-12T03:04:38.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.583+0000 7fcd6cff9640 1 --2- 172.21.15.116:0/993584403 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcd580a2b80 0x7fcd580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:38.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.583+0000 7fcd7857f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e742482b60 0x55e742482f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:38.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.583+0000 7fcd7857f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e742483480 0x55e74248d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:38.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.583+0000 7fcd7857f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e74248dd40 0x55e742490130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:38.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.584+0000 7fcd7857f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e7423e8280 con 0x55e742482b60 2024-09-12T03:04:38.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.584+0000 7fcd7857f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e74238fb10 con 0x55e742483480 2024-09-12T03:04:38.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.584+0000 7fcd7857f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e7423e4240 con 0x55e74248dd40 2024-09-12T03:04:38.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.584+0000 7fcd6ffff640 1 --2- 172.21.15.116:0/993584403 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcd580a2b80 0x7fcd580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:38.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.584+0000 7fcd6cff9640 1 -- 172.21.15.116:0/993584403 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fcd640cf0b0 con 0x7fcd70102790 2024-09-12T03:04:38.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.584+0000 7fcd6f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e742482b60 0x55e742482f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:38.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.584+0000 7fcd6f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e742482b60 0x55e742482f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39314/0 (socket says 172.21.15.116:39314) 2024-09-12T03:04:38.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.584+0000 7fcd6f7fe640 1 -- 172.21.15.116:0/2109624883 learned_addr learned my addr 172.21.15.116:0/2109624883 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:38.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.584+0000 7fcd6effd640 1 --2- 172.21.15.116:0/2109624883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e742483480 0x55e74248d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:38.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.584+0000 7fcd6f7fe640 1 -- 172.21.15.116:0/2109624883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e74248dd40 msgr2=0x55e742490130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:04:38.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.584+0000 7fcd6f7fe640 1 --2- 172.21.15.116:0/2109624883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e74248dd40 0x55e742490130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.584+0000 7fcd6f7fe640 1 -- 172.21.15.116:0/2109624883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e742483480 msgr2=0x55e74248d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:38.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.584+0000 7fcd6f7fe640 1 --2- 172.21.15.116:0/2109624883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e742483480 0x55e74248d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.584+0000 7fcd6f7fe640 1 -- 172.21.15.116:0/2109624883 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e74239c1f0 con 0x55e742482b60 2024-09-12T03:04:38.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.584+0000 7fcd6ffff640 1 --2- 172.21.15.116:0/993584403 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcd580a2b80 0x7fcd580a5040 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcd68005fd0 tx=0x7fcd68009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:38.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.584+0000 7fcd6f7fe640 1 --2- 172.21.15.116:0/2109624883 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e742482b60 0x55e742482f40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fcd60009900 tx=0x7fcd60031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c02f8995e1329ba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:38.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.584+0000 7fcd55ffb640 1 -- 172.21.15.116:0/2109624883 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60040070 con 0x55e742482b60 2024-09-12T03:04:38.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.584+0000 7fcd55ffb640 1 -- 172.21.15.116:0/2109624883 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd60005020 con 0x55e742482b60 2024-09-12T03:04:38.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.584+0000 7fcd55ffb640 1 -- 172.21.15.116:0/2109624883 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60035040 con 0x55e742482b60 2024-09-12T03:04:38.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.585+0000 7fcd7857f740 1 -- 172.21.15.116:0/2109624883 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e742482b60 msgr2=0x55e742482f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:38.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.585+0000 7fcd7857f740 1 --2- 172.21.15.116:0/2109624883 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e742482b60 0x55e742482f40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fcd60009900 tx=0x7fcd60031630 comp rx=0 tx=0).stop 2024-09-12T03:04:38.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.585+0000 7fcd7857f740 1 -- 172.21.15.116:0/2109624883 shutdown_connections 2024-09-12T03:04:38.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.585+0000 7fcd7857f740 1 --2- 172.21.15.116:0/2109624883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e74248dd40 0x55e742490130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.585+0000 7fcd7857f740 1 --2- 172.21.15.116:0/2109624883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e742483480 0x55e74248d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.585+0000 7fcd7857f740 1 --2- 172.21.15.116:0/2109624883 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e742482b60 0x55e742482f40 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.585+0000 7fcd7857f740 1 -- 172.21.15.116:0/2109624883 >> 172.21.15.116:0/2109624883 conn(0x55e74247f0f0 msgr2=0x55e74247f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:38.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.585+0000 7fcd7857f740 1 -- 172.21.15.116:0/2109624883 shutdown_connections 2024-09-12T03:04:38.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.585+0000 7fcd7857f740 1 -- 172.21.15.116:0/2109624883 wait complete. 2024-09-12T03:04:38.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.585+0000 7fcd7857f740 1 -- start start 2024-09-12T03:04:38.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.586+0000 7fcd7857f740 1 Processor -- start 2024-09-12T03:04:38.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.586+0000 7fcd7857f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e742482b60 0x55e74248d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:38.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.586+0000 7fcd7857f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e742483480 0x55e74248ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:38.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.586+0000 7fcd7857f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e74248dd40 0x55e74248b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:38.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.586+0000 7fcd7857f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e742490910 con 0x55e74248dd40 2024-09-12T03:04:38.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.586+0000 7fcd7857f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e7423a2f10 con 0x55e742482b60 2024-09-12T03:04:38.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.586+0000 7fcd7857f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e742490a90 con 0x55e742483480 2024-09-12T03:04:38.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.586+0000 7fcd7857f740 10 client.0 _ll_register_callbacks cb 0x55e74236ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:38.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.586+0000 7fcd6ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e74248dd40 0x55e74248b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:38.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.586+0000 7fcd6ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e74248dd40 0x55e74248b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39330/0 (socket says 172.21.15.116:39330) 2024-09-12T03:04:38.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.586+0000 7fcd6ffff640 1 -- 172.21.15.116:0/3975850733 learned_addr learned my addr 172.21.15.116:0/3975850733 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:38.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.586+0000 7fcd6f7fe640 1 --2- 172.21.15.116:0/3975850733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e742482b60 0x55e74248d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:38.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.586+0000 7fcd6effd640 1 --2- 172.21.15.116:0/3975850733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e742483480 0x55e74248ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:38.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.586+0000 7fcd6ffff640 1 -- 172.21.15.116:0/3975850733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e742483480 msgr2=0x55e74248ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:38.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.586+0000 7fcd6ffff640 1 --2- 172.21.15.116:0/3975850733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e742483480 0x55e74248ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.586+0000 7fcd6ffff640 1 -- 172.21.15.116:0/3975850733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e742482b60 msgr2=0x55e74248d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:38.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.586+0000 7fcd6ffff640 1 --2- 172.21.15.116:0/3975850733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e742482b60 0x55e74248d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.586+0000 7fcd6ffff640 1 -- 172.21.15.116:0/3975850733 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7421596a0 con 0x55e74248dd40 2024-09-12T03:04:38.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.587+0000 7fcd4effd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:04:38.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.587+0000 7fcd6ffff640 1 --2- 172.21.15.116:0/3975850733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e74248dd40 0x55e74248b2e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fcd68012bf0 tx=0x7fcd68009b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7cbd2c50de3c4d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:38.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.587+0000 7fcd4effd640 1 -- 172.21.15.116:0/3975850733 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68023070 con 0x55e74248dd40 2024-09-12T03:04:38.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.587+0000 7fcd4effd640 1 -- 172.21.15.116:0/3975850733 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fcd68011e30 con 0x55e74248dd40 2024-09-12T03:04:38.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.587+0000 7fcd7857f740 1 -- 172.21.15.116:0/3975850733 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd60005020 con 0x55e74248dd40 2024-09-12T03:04:38.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.587+0000 7fcd4effd640 1 -- 172.21.15.116:0/3975850733 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcd6802e050 con 0x55e74248dd40 2024-09-12T03:04:38.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.587+0000 7fcd7857f740 10 client.15618 fetch_fsmap learned FSMap version 25 2024-09-12T03:04:38.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.587+0000 7fcd7857f740 1 -- 172.21.15.116:0/3975850733 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e74249c4a0 con 0x55e74248dd40 2024-09-12T03:04:38.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.587+0000 7fcd4effd640 1 -- 172.21.15.116:0/3975850733 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd60005020 con 0x55e74248dd40 2024-09-12T03:04:38.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.587+0000 7fcd4effd640 1 -- 172.21.15.116:0/3975850733 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcd6800f040 con 0x55e74248dd40 2024-09-12T03:04:38.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.587+0000 7fcd7857f740 10 client.15618 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:04:38.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.587+0000 7fcd7857f740 10 client.15618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:04:38.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.587+0000 7fcd7857f740 20 client.15618 populate_metadata read hostname 'smithi116' 2024-09-12T03:04:38.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.587+0000 7fcd7857f740 1 --2- 172.21.15.116:0/3975850733 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e74249d8b0 0x55e74249fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:38.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.587+0000 7fcd7857f740 4 client.15618 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:04:38.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.587+0000 7fcd7857f740 1 -- 172.21.15.116:0/3975850733 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7423f57f0 con 0x55e74249d8b0 2024-09-12T03:04:38.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.587+0000 7fcd6f7fe640 1 --2- 172.21.15.116:0/3975850733 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e74249d8b0 0x55e74249fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:38.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.588+0000 7fcd6f7fe640 1 --2- 172.21.15.116:0/3975850733 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e74249d8b0 0x55e74249fd50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70c8a6c8c75acd09 server_cookie=699c4497fe475083 in_seq=0 out_seq=0 2024-09-12T03:04:38.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.588+0000 7fcd4effd640 10 client.15618 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:38.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.588+0000 7fcd4effd640 1 -- 172.21.15.116:0/3975850733 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e7423f57f0 con 0x55e74249d8b0 2024-09-12T03:04:38.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.588+0000 7fcd4effd640 10 client.15618 handle_command_reply: tid=0 2024-09-12T03:04:38.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.589+0000 7fcd7857f740 1 client.15618 shutdown 2024-09-12T03:04:38.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.589+0000 7fcd7857f740 1 -- 172.21.15.116:0/3975850733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e74248dd40 msgr2=0x55e74248b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:38.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.589+0000 7fcd7857f740 1 --2- 172.21.15.116:0/3975850733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e74248dd40 0x55e74248b2e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fcd68012bf0 tx=0x7fcd68009b40 comp rx=0 tx=0).stop 2024-09-12T03:04:38.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.589+0000 7fcd7857f740 1 -- 172.21.15.116:0/3975850733 shutdown_connections 2024-09-12T03:04:38.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.589+0000 7fcd7857f740 1 --2- 172.21.15.116:0/3975850733 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e74249d8b0 0x55e74249fd50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.589+0000 7fcd7857f740 1 --2- 172.21.15.116:0/3975850733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e74248dd40 0x55e74248b2e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.589+0000 7fcd4effd640 0 client.15618 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:38.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.589+0000 7fcd7857f740 1 --2- 172.21.15.116:0/3975850733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e742483480 0x55e74248ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.589+0000 7fcd7857f740 1 --2- 172.21.15.116:0/3975850733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e742482b60 0x55e74248d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.589+0000 7fcd7857f740 1 -- 172.21.15.116:0/3975850733 >> 172.21.15.116:0/3975850733 conn(0x55e74247f0f0 msgr2=0x55e742484e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:38.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.589+0000 7fcd7857f740 1 -- 172.21.15.116:0/3975850733 shutdown_connections 2024-09-12T03:04:38.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.589+0000 7fcd7857f740 1 -- 172.21.15.116:0/3975850733 wait complete. 2024-09-12T03:04:38.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.589+0000 7fcd7857f740 20 client.15618 trim_cache size 0 max 16384 2024-09-12T03:04:38.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.606+0000 7fcd7857f740 1 Processor -- start 2024-09-12T03:04:38.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.606+0000 7fcd7857f740 1 -- start start 2024-09-12T03:04:38.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.606+0000 7fcd7857f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e742482b60 0x55e74247e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:38.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.606+0000 7fcd7857f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e742483480 0x55e7424a5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:38.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.606+0000 7fcd7857f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e74248dd40 0x55e7424a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:38.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.606+0000 7fcd7857f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701afcc0 con 0x55e74248dd40 2024-09-12T03:04:38.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.606+0000 7fcd7857f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701ad590 con 0x55e742483480 2024-09-12T03:04:38.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.606+0000 7fcd7857f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701ab3e0 con 0x55e742482b60 2024-09-12T03:04:38.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.606+0000 7fcd6ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e74248dd40 0x55e7424a5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:38.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.606+0000 7fcd6ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e74248dd40 0x55e7424a5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39346/0 (socket says 172.21.15.116:39346) 2024-09-12T03:04:38.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.606+0000 7fcd6ffff640 1 -- 172.21.15.116:0/841919808 learned_addr learned my addr 172.21.15.116:0/841919808 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:38.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.606+0000 7fcd6effd640 1 --2- 172.21.15.116:0/841919808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e742483480 0x55e7424a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:38.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.606+0000 7fcd6f7fe640 1 --2- 172.21.15.116:0/841919808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e742482b60 0x55e74247e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:38.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.606+0000 7fcd6ffff640 1 -- 172.21.15.116:0/841919808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e742482b60 msgr2=0x55e74247e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:38.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.606+0000 7fcd6ffff640 1 --2- 172.21.15.116:0/841919808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e742482b60 0x55e74247e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.606+0000 7fcd6ffff640 1 -- 172.21.15.116:0/841919808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e742483480 msgr2=0x55e7424a5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:38.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.606+0000 7fcd6ffff640 1 --2- 172.21.15.116:0/841919808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e742483480 0x55e7424a5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.606+0000 7fcd6ffff640 1 -- 172.21.15.116:0/841919808 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e742489b00 con 0x55e74248dd40 2024-09-12T03:04:38.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.606+0000 7fcd6f7fe640 1 --2- 172.21.15.116:0/841919808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e742482b60 0x55e74247e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:04:38.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.606+0000 7fcd6ffff640 1 --2- 172.21.15.116:0/841919808 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e74248dd40 0x55e7424a5bb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009870 tx=0x7fcd68009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e9c6b6f444c9c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:38.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.607+0000 7fcd557fa640 1 -- 172.21.15.116:0/841919808 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68037070 con 0x55e74248dd40 2024-09-12T03:04:38.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.607+0000 7fcd557fa640 1 -- 172.21.15.116:0/841919808 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd6801f770 con 0x55e74248dd40 2024-09-12T03:04:38.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.607+0000 7fcd557fa640 1 -- 172.21.15.116:0/841919808 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68023070 con 0x55e74248dd40 2024-09-12T03:04:38.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.607+0000 7fcd7857f740 1 -- 172.21.15.116:0/841919808 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e74248dd40 msgr2=0x55e7424a5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:38.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.607+0000 7fcd7857f740 1 --2- 172.21.15.116:0/841919808 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e74248dd40 0x55e7424a5bb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009870 tx=0x7fcd68009d30 comp rx=0 tx=0).stop 2024-09-12T03:04:38.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.607+0000 7fcd7857f740 1 -- 172.21.15.116:0/841919808 shutdown_connections 2024-09-12T03:04:38.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.607+0000 7fcd7857f740 1 --2- 172.21.15.116:0/841919808 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e74248dd40 0x55e7424a5bb0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.607+0000 7fcd7857f740 1 --2- 172.21.15.116:0/841919808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e742483480 0x55e7424a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.607+0000 7fcd7857f740 1 --2- 172.21.15.116:0/841919808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e742482b60 0x55e74247e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.607+0000 7fcd7857f740 1 -- 172.21.15.116:0/841919808 >> 172.21.15.116:0/841919808 conn(0x55e74247f0f0 msgr2=0x55e74247da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:38.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.607+0000 7fcd7857f740 1 -- 172.21.15.116:0/841919808 shutdown_connections 2024-09-12T03:04:38.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.607+0000 7fcd7857f740 1 -- 172.21.15.116:0/841919808 wait complete. 2024-09-12T03:04:38.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.608+0000 7fcd7857f740 1 -- start start 2024-09-12T03:04:38.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.608+0000 7fcd7857f740 1 Processor -- start 2024-09-12T03:04:38.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.608+0000 7fcd7857f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e742482b60 0x55e742496060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:38.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.608+0000 7fcd7857f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e742483480 0x55e7424965a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:38.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.608+0000 7fcd7857f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e74248dd40 0x55e742490ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:38.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.608+0000 7fcd7857f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701aba50 con 0x55e742483480 2024-09-12T03:04:38.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.608+0000 7fcd7857f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701acec0 con 0x55e74248dd40 2024-09-12T03:04:38.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.608+0000 7fcd7857f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701ac120 con 0x55e742482b60 2024-09-12T03:04:38.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.608+0000 7fcd7857f740 10 client.0 _ll_register_callbacks cb 0x7fcd68016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:38.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.608+0000 7fcd6effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e742483480 0x55e7424965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:38.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.608+0000 7fcd6effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e742483480 0x55e7424965a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39360/0 (socket says 172.21.15.116:39360) 2024-09-12T03:04:38.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.608+0000 7fcd6effd640 1 -- 172.21.15.116:0/2116930577 learned_addr learned my addr 172.21.15.116:0/2116930577 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:38.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.608+0000 7fcd6f7fe640 1 --2- 172.21.15.116:0/2116930577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e742482b60 0x55e742496060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:38.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.609+0000 7fcd6ffff640 1 --2- 172.21.15.116:0/2116930577 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e74248dd40 0x55e742490ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:38.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.609+0000 7fcd6effd640 1 -- 172.21.15.116:0/2116930577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e742482b60 msgr2=0x55e742496060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:38.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.609+0000 7fcd6effd640 1 --2- 172.21.15.116:0/2116930577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e742482b60 0x55e742496060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.609+0000 7fcd6effd640 1 -- 172.21.15.116:0/2116930577 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e74248dd40 msgr2=0x55e742490ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:38.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.609+0000 7fcd6effd640 1 --2- 172.21.15.116:0/2116930577 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e74248dd40 0x55e742490ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.609+0000 7fcd6effd640 1 -- 172.21.15.116:0/2116930577 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7424916f0 con 0x55e742483480 2024-09-12T03:04:38.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.609+0000 7fcd6ffff640 1 --2- 172.21.15.116:0/2116930577 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e74248dd40 0x55e742490ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:38.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.609+0000 7fcd6effd640 1 --2- 172.21.15.116:0/2116930577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e742483480 0x55e7424965a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fcd640ccc30 tx=0x7fcd6409ba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe805bf2c05c7db server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:38.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.609+0000 7fcd4effd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:04:38.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.609+0000 7fcd4effd640 1 -- 172.21.15.116:0/2116930577 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6400fb30 con 0x55e742483480 2024-09-12T03:04:38.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.609+0000 7fcd4effd640 1 -- 172.21.15.116:0/2116930577 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fcd64052300 con 0x55e742483480 2024-09-12T03:04:38.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.609+0000 7fcd7857f740 1 -- 172.21.15.116:0/2116930577 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd6801f770 con 0x55e742483480 2024-09-12T03:04:38.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.609+0000 7fcd4effd640 1 -- 172.21.15.116:0/2116930577 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcd640afe60 con 0x55e742483480 2024-09-12T03:04:38.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.609+0000 7fcd7857f740 10 client.15624 fetch_fsmap learned FSMap version 25 2024-09-12T03:04:38.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.609+0000 7fcd7857f740 1 -- 172.21.15.116:0/2116930577 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e742509330 con 0x55e742483480 2024-09-12T03:04:38.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.609+0000 7fcd4effd640 1 -- 172.21.15.116:0/2116930577 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd6801f770 con 0x55e742483480 2024-09-12T03:04:38.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.610+0000 7fcd4effd640 1 -- 172.21.15.116:0/2116930577 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcd640a4220 con 0x55e742483480 2024-09-12T03:04:38.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.610+0000 7fcd7857f740 10 client.15624 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:04:38.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.610+0000 7fcd7857f740 10 client.15624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:04:38.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.610+0000 7fcd7857f740 20 client.15624 populate_metadata read hostname 'smithi116' 2024-09-12T03:04:38.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.610+0000 7fcd7857f740 1 --2- 172.21.15.116:0/2116930577 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e742509a80 0x55e74250bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:38.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.610+0000 7fcd7857f740 4 client.15624 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:04:38.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.610+0000 7fcd7857f740 1 -- 172.21.15.116:0/2116930577 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e7423efd00 con 0x55e742509a80 2024-09-12T03:04:38.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.610+0000 7fcd6f7fe640 1 --2- 172.21.15.116:0/2116930577 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e742509a80 0x55e74250bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:38.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.610+0000 7fcd6f7fe640 1 --2- 172.21.15.116:0/2116930577 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e742509a80 0x55e74250bea0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe42889e8b28b1e8 server_cookie=8dcc346d7bdd8d25 in_seq=0 out_seq=0 2024-09-12T03:04:38.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.610+0000 7fcd4effd640 10 client.15624 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:38.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.611+0000 7fcd4effd640 1 -- 172.21.15.116:0/2116930577 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e7423efd00 con 0x55e742509a80 2024-09-12T03:04:38.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.611+0000 7fcd4effd640 10 client.15624 handle_command_reply: tid=0 2024-09-12T03:04:38.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.611+0000 7fcd7857f740 1 client.15624 shutdown 2024-09-12T03:04:38.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.611+0000 7fcd7857f740 1 -- 172.21.15.116:0/2116930577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e742483480 msgr2=0x55e7424965a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:38.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.611+0000 7fcd7857f740 1 --2- 172.21.15.116:0/2116930577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e742483480 0x55e7424965a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fcd640ccc30 tx=0x7fcd6409ba00 comp rx=0 tx=0).stop 2024-09-12T03:04:38.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.611+0000 7fcd7857f740 1 -- 172.21.15.116:0/2116930577 shutdown_connections 2024-09-12T03:04:38.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.611+0000 7fcd7857f740 1 --2- 172.21.15.116:0/2116930577 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e742509a80 0x55e74250bea0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.611+0000 7fcd7857f740 1 --2- 172.21.15.116:0/2116930577 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e74248dd40 0x55e742490ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.611+0000 7fcd7857f740 1 --2- 172.21.15.116:0/2116930577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e742483480 0x55e7424965a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.611+0000 7fcd7857f740 1 --2- 172.21.15.116:0/2116930577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e742482b60 0x55e742496060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.611+0000 7fcd4effd640 0 client.15624 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:38.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.611+0000 7fcd7857f740 1 -- 172.21.15.116:0/2116930577 >> 172.21.15.116:0/2116930577 conn(0x55e74247f0f0 msgr2=0x55e74247e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:38.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.611+0000 7fcd7857f740 1 -- 172.21.15.116:0/2116930577 shutdown_connections 2024-09-12T03:04:38.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.611+0000 7fcd7857f740 1 -- 172.21.15.116:0/2116930577 wait complete. 2024-09-12T03:04:38.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.612+0000 7fcd7857f740 20 client.15624 trim_cache size 0 max 16384 2024-09-12T03:04:38.625 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:04:38.625 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:04:38.625 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "fe147efb-3fb1-4afb-8d71-5d281f24d4e1", 2024-09-12T03:04:38.625 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:04:38.626 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:04:38.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.612+0000 7fcd761a6640 1 -- 172.21.15.116:0/993584403 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcd580a2b80 msgr2=0x7fcd580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:38.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.612+0000 7fcd761a6640 1 --2- 172.21.15.116:0/993584403 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcd580a2b80 0x7fcd580a5040 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcd68005fd0 tx=0x7fcd68009600 comp rx=0 tx=0).stop 2024-09-12T03:04:38.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.612+0000 7fcd761a6640 1 -- 172.21.15.116:0/993584403 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd70102790 msgr2=0x7fcd701988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:38.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.612+0000 7fcd761a6640 1 --2- 172.21.15.116:0/993584403 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd70102790 0x7fcd701988d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fcd6400c920 tx=0x7fcd6400cde0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.612+0000 7fcd761a6640 1 -- 172.21.15.116:0/993584403 shutdown_connections 2024-09-12T03:04:38.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.612+0000 7fcd761a6640 1 --2- 172.21.15.116:0/993584403 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcd580a2b80 0x7fcd580a5040 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.612+0000 7fcd761a6640 1 --2- 172.21.15.116:0/993584403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd70106f10 0x7fcd7019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.612+0000 7fcd761a6640 1 --2- 172.21.15.116:0/993584403 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd70102790 0x7fcd701988d0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.612+0000 7fcd761a6640 1 --2- 172.21.15.116:0/993584403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd70101de0 0x7fcd70198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.612+0000 7fcd761a6640 1 -- 172.21.15.116:0/993584403 >> 172.21.15.116:0/993584403 conn(0x7fcd700fd8c0 msgr2=0x7fcd700fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:38.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.612+0000 7fcd761a6640 1 -- 172.21.15.116:0/993584403 shutdown_connections 2024-09-12T03:04:38.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:38.612+0000 7fcd761a6640 1 -- 172.21.15.116:0/993584403 wait complete. 2024-09-12T03:04:38.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:38 smithi183 ceph-mon[35535]: pgmap v387: 97 pgs: 97 active+clean; 858 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-12T03:04:39.320 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:04:39.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:39 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:04:39.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:39 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:04:39.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:39 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:04:39.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:39 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:04:39.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:39 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:04:39.563 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:39.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:39 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:04:39.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:39 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:04:39.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:39 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:04:39.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:39 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:04:39.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:39 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:04:39.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:39 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:04:39.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:39 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:04:39.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:39 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:04:39.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:39 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:04:39.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:39 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:04:40.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.067+0000 7fe98d8a9640 1 -- 172.21.15.116:0/3230092475 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe988100350 msgr2=0x7fe988100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:40.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.067+0000 7fe98d8a9640 1 --2- 172.21.15.116:0/3230092475 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe988100350 0x7fe988100730 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe97c009930 tx=0x7fe97c02a420 comp rx=0 tx=0).stop 2024-09-12T03:04:40.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.067+0000 7fe98d8a9640 1 -- 172.21.15.116:0/3230092475 shutdown_connections 2024-09-12T03:04:40.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.067+0000 7fe98d8a9640 1 --2- 172.21.15.116:0/3230092475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe988106f50 0x7fe98810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.067+0000 7fe98d8a9640 1 --2- 172.21.15.116:0/3230092475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe988100d00 0x7fe988106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.067+0000 7fe98d8a9640 1 --2- 172.21.15.116:0/3230092475 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe988100350 0x7fe988100730 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.067+0000 7fe98d8a9640 1 -- 172.21.15.116:0/3230092475 >> 172.21.15.116:0/3230092475 conn(0x7fe988077210 msgr2=0x7fe9880fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:40.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.067+0000 7fe98d8a9640 1 -- 172.21.15.116:0/3230092475 shutdown_connections 2024-09-12T03:04:40.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.067+0000 7fe98d8a9640 1 -- 172.21.15.116:0/3230092475 wait complete. 2024-09-12T03:04:40.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.068+0000 7fe98d8a9640 1 Processor -- start 2024-09-12T03:04:40.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.068+0000 7fe98d8a9640 1 -- start start 2024-09-12T03:04:40.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.068+0000 7fe98d8a9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe988100350 0x7fe9881984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:40.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.068+0000 7fe98d8a9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe988100d00 0x7fe988198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:40.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.068+0000 7fe98d8a9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe988106f50 0x7fe98819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:40.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.068+0000 7fe98d8a9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe988111fa0 con 0x7fe988106f50 2024-09-12T03:04:40.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.068+0000 7fe98d8a9640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe988111e20 con 0x7fe988100d00 2024-09-12T03:04:40.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.068+0000 7fe98d8a9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe988112120 con 0x7fe988100350 2024-09-12T03:04:40.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.068+0000 7fe9877fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe988106f50 0x7fe98819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:40.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.068+0000 7fe9877fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe988106f50 0x7fe98819cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39386/0 (socket says 172.21.15.116:39386) 2024-09-12T03:04:40.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.068+0000 7fe9877fe640 1 -- 172.21.15.116:0/531039921 learned_addr learned my addr 172.21.15.116:0/531039921 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:40.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.068+0000 7fe9867fc640 1 --2- 172.21.15.116:0/531039921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe988100d00 0x7fe988198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:40.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.068+0000 7fe986ffd640 1 --2- 172.21.15.116:0/531039921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe988100350 0x7fe9881984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:40.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.068+0000 7fe9877fe640 1 -- 172.21.15.116:0/531039921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe988100350 msgr2=0x7fe9881984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:40.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.068+0000 7fe9877fe640 1 --2- 172.21.15.116:0/531039921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe988100350 0x7fe9881984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.068+0000 7fe9877fe640 1 -- 172.21.15.116:0/531039921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe988100d00 msgr2=0x7fe988198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:40.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.068+0000 7fe9877fe640 1 --2- 172.21.15.116:0/531039921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe988100d00 0x7fe988198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.069+0000 7fe9877fe640 1 -- 172.21.15.116:0/531039921 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98819d4a0 con 0x7fe988106f50 2024-09-12T03:04:40.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.069+0000 7fe9877fe640 1 --2- 172.21.15.116:0/531039921 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe988106f50 0x7fe98819cdc0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe98000cad0 tx=0x7fe98000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:40.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.069+0000 7fe98c8a7640 1 -- 172.21.15.116:0/531039921 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98000ae20 con 0x7fe988106f50 2024-09-12T03:04:40.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.069+0000 7fe98c8a7640 1 -- 172.21.15.116:0/531039921 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe980005020 con 0x7fe988106f50 2024-09-12T03:04:40.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.069+0000 7fe98d8a9640 1 -- 172.21.15.116:0/531039921 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe98819d790 con 0x7fe988106f50 2024-09-12T03:04:40.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.069+0000 7fe98c8a7640 1 -- 172.21.15.116:0/531039921 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98000f6f0 con 0x7fe988106f50 2024-09-12T03:04:40.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.069+0000 7fe98d8a9640 1 -- 172.21.15.116:0/531039921 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9881a5070 con 0x7fe988106f50 2024-09-12T03:04:40.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.071+0000 7fe98c8a7640 1 -- 172.21.15.116:0/531039921 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe9800051d0 con 0x7fe988106f50 2024-09-12T03:04:40.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.071+0000 7fe98fc82740 1 Processor -- start 2024-09-12T03:04:40.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.072+0000 7fe98fc82740 1 -- start start 2024-09-12T03:04:40.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.072+0000 7fe98fc82740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563528706360 0x563528706740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:40.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.072+0000 7fe98fc82740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563528706c80 0x563528715550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:40.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.072+0000 7fe98fc82740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563528715a90 0x563528717e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:40.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.072+0000 7fe98fc82740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56352866c2e0 con 0x563528706360 2024-09-12T03:04:40.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.072+0000 7fe98fc82740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x563528613ce0 con 0x563528706c80 2024-09-12T03:04:40.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.072+0000 7fe98fc82740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5635286682a0 con 0x563528715a90 2024-09-12T03:04:40.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.072+0000 7fe98c8a7640 1 --2- 172.21.15.116:0/531039921 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe9700a2b80 0x7fe9700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:40.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.072+0000 7fe9867fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563528706360 0x563528706740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:40.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.072+0000 7fe9867fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563528706360 0x563528706740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39390/0 (socket says 172.21.15.116:39390) 2024-09-12T03:04:40.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.072+0000 7fe9867fc640 1 -- 172.21.15.116:0/2211634658 learned_addr learned my addr 172.21.15.116:0/2211634658 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:40.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.072+0000 7fe9877fe640 1 --2- 172.21.15.116:0/531039921 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe9700a2b80 0x7fe9700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:40.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.072+0000 7fe9867fc640 1 --2- 172.21.15.116:0/2211634658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563528715a90 0x563528717e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:40.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.072+0000 7fe986ffd640 1 --2- 172.21.15.116:0/2211634658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563528706c80 0x563528715550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:40.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.072+0000 7fe98c8a7640 1 -- 172.21.15.116:0/531039921 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe9800cf0f0 con 0x7fe988106f50 2024-09-12T03:04:40.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.072+0000 7fe9867fc640 1 -- 172.21.15.116:0/2211634658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563528715a90 msgr2=0x563528717e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:40.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.072+0000 7fe9867fc640 1 --2- 172.21.15.116:0/2211634658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563528715a90 0x563528717e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.072+0000 7fe9867fc640 1 -- 172.21.15.116:0/2211634658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563528706c80 msgr2=0x563528715550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:40.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.072+0000 7fe9867fc640 1 --2- 172.21.15.116:0/2211634658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563528706c80 0x563528715550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.072+0000 7fe9867fc640 1 -- 172.21.15.116:0/2211634658 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635285f8440 con 0x563528706360 2024-09-12T03:04:40.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.072+0000 7fe9867fc640 1 --2- 172.21.15.116:0/2211634658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563528706360 0x563528706740 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe978013500 tx=0x7fe97800b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72565673bb34170f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:40.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.073+0000 7fe9877fe640 1 --2- 172.21.15.116:0/531039921 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe9700a2b80 0x7fe9700a5040 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe9800ccf90 tx=0x7fe980086040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:40.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.073+0000 7fe96dffb640 1 -- 172.21.15.116:0/2211634658 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe978019030 con 0x563528706360 2024-09-12T03:04:40.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.073+0000 7fe96dffb640 1 -- 172.21.15.116:0/2211634658 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe978011920 con 0x563528706360 2024-09-12T03:04:40.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.073+0000 7fe96dffb640 1 -- 172.21.15.116:0/2211634658 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97801d840 con 0x563528706360 2024-09-12T03:04:40.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.073+0000 7fe98fc82740 1 -- 172.21.15.116:0/2211634658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563528706360 msgr2=0x563528706740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:40.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.073+0000 7fe98fc82740 1 --2- 172.21.15.116:0/2211634658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563528706360 0x563528706740 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe978013500 tx=0x7fe97800b690 comp rx=0 tx=0).stop 2024-09-12T03:04:40.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.073+0000 7fe98fc82740 1 -- 172.21.15.116:0/2211634658 shutdown_connections 2024-09-12T03:04:40.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.073+0000 7fe98fc82740 1 --2- 172.21.15.116:0/2211634658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563528715a90 0x563528717e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.073+0000 7fe98fc82740 1 --2- 172.21.15.116:0/2211634658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563528706c80 0x563528715550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.073+0000 7fe98fc82740 1 --2- 172.21.15.116:0/2211634658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563528706360 0x563528706740 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.073+0000 7fe98fc82740 1 -- 172.21.15.116:0/2211634658 >> 172.21.15.116:0/2211634658 conn(0x563528702a30 msgr2=0x563528702e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:40.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.073+0000 7fe98fc82740 1 -- 172.21.15.116:0/2211634658 shutdown_connections 2024-09-12T03:04:40.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.073+0000 7fe98fc82740 1 -- 172.21.15.116:0/2211634658 wait complete. 2024-09-12T03:04:40.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.074+0000 7fe98fc82740 1 -- start start 2024-09-12T03:04:40.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.074+0000 7fe98fc82740 1 Processor -- start 2024-09-12T03:04:40.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.074+0000 7fe98fc82740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563528706360 0x563528715330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:40.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.074+0000 7fe98fc82740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563528706c80 0x563528712630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:40.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.074+0000 7fe98fc82740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563528715a90 0x563528712b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:40.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.074+0000 7fe98fc82740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x563528718660 con 0x563528715a90 2024-09-12T03:04:40.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.074+0000 7fe98fc82740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x563528627050 con 0x563528706360 2024-09-12T03:04:40.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.074+0000 7fe98fc82740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5635287187e0 con 0x563528706c80 2024-09-12T03:04:40.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.074+0000 7fe98fc82740 10 client.0 _ll_register_callbacks cb 0x563528667160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:40.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.074+0000 7fe9867fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563528715a90 0x563528712b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:40.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.074+0000 7fe9867fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563528715a90 0x563528712b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39408/0 (socket says 172.21.15.116:39408) 2024-09-12T03:04:40.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.074+0000 7fe9867fc640 1 -- 172.21.15.116:0/3870109540 learned_addr learned my addr 172.21.15.116:0/3870109540 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:40.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.075+0000 7fe986ffd640 1 --2- 172.21.15.116:0/3870109540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563528706c80 0x563528712630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:40.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.075+0000 7fe9867fc640 1 --2- 172.21.15.116:0/3870109540 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563528706360 0x563528715330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:40.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.075+0000 7fe9867fc640 1 -- 172.21.15.116:0/3870109540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563528706c80 msgr2=0x563528712630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:40.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.075+0000 7fe9867fc640 1 --2- 172.21.15.116:0/3870109540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563528706c80 0x563528712630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.075+0000 7fe9867fc640 1 -- 172.21.15.116:0/3870109540 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563528706360 msgr2=0x563528715330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:40.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.075+0000 7fe9867fc640 1 --2- 172.21.15.116:0/3870109540 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563528706360 0x563528715330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.075+0000 7fe9867fc640 1 -- 172.21.15.116:0/3870109540 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635283dd3c0 con 0x563528715a90 2024-09-12T03:04:40.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.075+0000 7fe9867fc640 1 --2- 172.21.15.116:0/3870109540 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563528715a90 0x563528712b70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe97800b4a0 tx=0x7fe97801d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee477f6776e8427 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:40.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.075+0000 7fe966ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:04:40.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.075+0000 7fe966ffd640 1 -- 172.21.15.116:0/3870109540 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe978019040 con 0x563528715a90 2024-09-12T03:04:40.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.075+0000 7fe98fc82740 1 -- 172.21.15.116:0/3870109540 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe978011920 con 0x563528715a90 2024-09-12T03:04:40.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.075+0000 7fe966ffd640 1 -- 172.21.15.116:0/3870109540 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe978011ad0 con 0x563528715a90 2024-09-12T03:04:40.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.075+0000 7fe966ffd640 1 -- 172.21.15.116:0/3870109540 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe978011cd0 con 0x563528715a90 2024-09-12T03:04:40.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.075+0000 7fe98fc82740 10 client.15636 fetch_fsmap learned FSMap version 25 2024-09-12T03:04:40.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.075+0000 7fe98fc82740 1 -- 172.21.15.116:0/3870109540 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563528724010 con 0x563528715a90 2024-09-12T03:04:40.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.075+0000 7fe966ffd640 1 -- 172.21.15.116:0/3870109540 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe978011920 con 0x563528715a90 2024-09-12T03:04:40.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.076+0000 7fe966ffd640 1 -- 172.21.15.116:0/3870109540 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe97801e4a0 con 0x563528715a90 2024-09-12T03:04:40.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.076+0000 7fe98fc82740 10 client.15636 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:04:40.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.076+0000 7fe98fc82740 10 client.15636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:04:40.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.076+0000 7fe98fc82740 20 client.15636 populate_metadata read hostname 'smithi116' 2024-09-12T03:04:40.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.076+0000 7fe98fc82740 1 --2- 172.21.15.116:0/3870109540 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x563528725360 0x563528727750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:40.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.076+0000 7fe98fc82740 4 client.15636 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:04:40.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.076+0000 7fe98fc82740 1 -- 172.21.15.116:0/3870109540 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563528679850 con 0x563528725360 2024-09-12T03:04:40.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.076+0000 7fe986ffd640 1 --2- 172.21.15.116:0/3870109540 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x563528725360 0x563528727750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:40.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.076+0000 7fe966ffd640 10 client.15636 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:40.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.076+0000 7fe986ffd640 1 --2- 172.21.15.116:0/3870109540 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x563528725360 0x563528727750 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=425b60be70ec7b95 server_cookie=145bf7b3f2570c91 in_seq=0 out_seq=0 2024-09-12T03:04:40.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.077+0000 7fe966ffd640 1 -- 172.21.15.116:0/3870109540 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563528679850 con 0x563528725360 2024-09-12T03:04:40.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.077+0000 7fe966ffd640 10 client.15636 handle_command_reply: tid=0 2024-09-12T03:04:40.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.077+0000 7fe98fc82740 1 client.15636 shutdown 2024-09-12T03:04:40.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.078+0000 7fe98fc82740 1 -- 172.21.15.116:0/3870109540 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563528715a90 msgr2=0x563528712b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:40.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.078+0000 7fe98fc82740 1 --2- 172.21.15.116:0/3870109540 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563528715a90 0x563528712b70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe97800b4a0 tx=0x7fe97801d3c0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.078+0000 7fe98fc82740 1 -- 172.21.15.116:0/3870109540 shutdown_connections 2024-09-12T03:04:40.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.078+0000 7fe98fc82740 1 --2- 172.21.15.116:0/3870109540 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x563528725360 0x563528727750 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.078+0000 7fe98fc82740 1 --2- 172.21.15.116:0/3870109540 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563528715a90 0x563528712b70 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.078+0000 7fe98fc82740 1 --2- 172.21.15.116:0/3870109540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563528706c80 0x563528712630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.078+0000 7fe98fc82740 1 --2- 172.21.15.116:0/3870109540 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563528706360 0x563528715330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.078+0000 7fe966ffd640 0 client.15636 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:40.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.078+0000 7fe98fc82740 1 -- 172.21.15.116:0/3870109540 >> 172.21.15.116:0/3870109540 conn(0x563528702a30 msgr2=0x563528701b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:40.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.078+0000 7fe98fc82740 1 -- 172.21.15.116:0/3870109540 shutdown_connections 2024-09-12T03:04:40.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.078+0000 7fe98fc82740 1 -- 172.21.15.116:0/3870109540 wait complete. 2024-09-12T03:04:40.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.078+0000 7fe98fc82740 20 client.15636 trim_cache size 0 max 16384 2024-09-12T03:04:40.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.095+0000 7fe98fc82740 1 Processor -- start 2024-09-12T03:04:40.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.095+0000 7fe98fc82740 1 -- start start 2024-09-12T03:04:40.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.095+0000 7fe98fc82740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563528706360 0x563528714960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:40.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.095+0000 7fe98fc82740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563528706c80 0x56352870f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:40.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.095+0000 7fe98fc82740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563528715a90 0x56352870fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:40.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.095+0000 7fe98fc82740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe9881adcc0 con 0x563528715a90 2024-09-12T03:04:40.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.095+0000 7fe98fc82740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe9881ab590 con 0x563528706c80 2024-09-12T03:04:40.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.095+0000 7fe98fc82740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe9881a9400 con 0x563528706360 2024-09-12T03:04:40.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.095+0000 7fe9867fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563528715a90 0x56352870fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:40.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.095+0000 7fe9867fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563528715a90 0x56352870fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39414/0 (socket says 172.21.15.116:39414) 2024-09-12T03:04:40.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.095+0000 7fe9867fc640 1 -- 172.21.15.116:0/1840241566 learned_addr learned my addr 172.21.15.116:0/1840241566 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:40.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.095+0000 7fe9867fc640 1 --2- 172.21.15.116:0/1840241566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563528706360 0x563528714960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:40.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.095+0000 7fe986ffd640 1 --2- 172.21.15.116:0/1840241566 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563528706c80 0x56352870f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:40.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.095+0000 7fe9867fc640 1 -- 172.21.15.116:0/1840241566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563528706360 msgr2=0x563528714960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:40.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.095+0000 7fe9867fc640 1 --2- 172.21.15.116:0/1840241566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563528706360 0x563528714960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.095+0000 7fe9867fc640 1 -- 172.21.15.116:0/1840241566 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563528706c80 msgr2=0x56352870f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:40.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.095+0000 7fe9867fc640 1 --2- 172.21.15.116:0/1840241566 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563528706c80 0x56352870f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.095+0000 7fe9867fc640 1 -- 172.21.15.116:0/1840241566 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563528714f30 con 0x563528715a90 2024-09-12T03:04:40.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.095+0000 7fe9867fc640 1 --2- 172.21.15.116:0/1840241566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563528715a90 0x56352870fdd0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe978001eb0 tx=0x7fe978015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb7f96f71d59b9f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:40.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.096+0000 7fe96d7fa640 1 -- 172.21.15.116:0/1840241566 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe978019020 con 0x563528715a90 2024-09-12T03:04:40.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.096+0000 7fe96d7fa640 1 -- 172.21.15.116:0/1840241566 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe978013e50 con 0x563528715a90 2024-09-12T03:04:40.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.096+0000 7fe96d7fa640 1 -- 172.21.15.116:0/1840241566 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe978005070 con 0x563528715a90 2024-09-12T03:04:40.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.096+0000 7fe98fc82740 1 -- 172.21.15.116:0/1840241566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563528715a90 msgr2=0x56352870fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:40.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.096+0000 7fe98fc82740 1 --2- 172.21.15.116:0/1840241566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563528715a90 0x56352870fdd0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe978001eb0 tx=0x7fe978015040 comp rx=0 tx=0).stop 2024-09-12T03:04:40.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.096+0000 7fe98fc82740 1 -- 172.21.15.116:0/1840241566 shutdown_connections 2024-09-12T03:04:40.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.096+0000 7fe98fc82740 1 --2- 172.21.15.116:0/1840241566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563528715a90 0x56352870fdd0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.096+0000 7fe98fc82740 1 --2- 172.21.15.116:0/1840241566 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563528706c80 0x56352870f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.096+0000 7fe98fc82740 1 --2- 172.21.15.116:0/1840241566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563528706360 0x563528714960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.096+0000 7fe98fc82740 1 -- 172.21.15.116:0/1840241566 >> 172.21.15.116:0/1840241566 conn(0x563528702a30 msgr2=0x5635287115f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:40.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.096+0000 7fe98fc82740 1 -- 172.21.15.116:0/1840241566 shutdown_connections 2024-09-12T03:04:40.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.096+0000 7fe98fc82740 1 -- 172.21.15.116:0/1840241566 wait complete. 2024-09-12T03:04:40.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.096+0000 7fe98fc82740 1 -- start start 2024-09-12T03:04:40.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.097+0000 7fe98fc82740 1 Processor -- start 2024-09-12T03:04:40.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.097+0000 7fe98fc82740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563528706360 0x563528728740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:40.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.097+0000 7fe98fc82740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563528706c80 0x563528728c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:40.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.097+0000 7fe98fc82740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563528715a90 0x5635287291c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:40.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.097+0000 7fe98fc82740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe9881a9a50 con 0x563528706360 2024-09-12T03:04:40.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.097+0000 7fe98fc82740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe9881aaec0 con 0x563528715a90 2024-09-12T03:04:40.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.097+0000 7fe98fc82740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe9881aa120 con 0x563528706c80 2024-09-12T03:04:40.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.097+0000 7fe98fc82740 10 client.0 _ll_register_callbacks cb 0x7fe978013400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:40.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.097+0000 7fe9867fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563528706360 0x563528728740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:40.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.097+0000 7fe9867fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563528706360 0x563528728740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39424/0 (socket says 172.21.15.116:39424) 2024-09-12T03:04:40.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.097+0000 7fe9867fc640 1 -- 172.21.15.116:0/3630455042 learned_addr learned my addr 172.21.15.116:0/3630455042 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:40.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.097+0000 7fe986ffd640 1 --2- 172.21.15.116:0/3630455042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563528706c80 0x563528728c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:40.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.097+0000 7fe9867fc640 1 -- 172.21.15.116:0/3630455042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563528706c80 msgr2=0x563528728c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:40.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.097+0000 7fe9867fc640 1 --2- 172.21.15.116:0/3630455042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563528706c80 0x563528728c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.097+0000 7fe9867fc640 1 -- 172.21.15.116:0/3630455042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563528715a90 msgr2=0x5635287291c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:40.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.097+0000 7fe9867fc640 1 --2- 172.21.15.116:0/3630455042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563528715a90 0x5635287291c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.097+0000 7fe9867fc640 1 -- 172.21.15.116:0/3630455042 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56352878eef0 con 0x563528706360 2024-09-12T03:04:40.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.097+0000 7fe9867fc640 1 --2- 172.21.15.116:0/3630455042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563528706360 0x563528728740 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe978005d00 tx=0x7fe978011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45ba80ac97f9e7f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:40.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.097+0000 7fe966ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:04:40.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.098+0000 7fe966ffd640 1 -- 172.21.15.116:0/3630455042 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe978019070 con 0x563528706360 2024-09-12T03:04:40.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.098+0000 7fe966ffd640 1 -- 172.21.15.116:0/3630455042 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe97801ece0 con 0x563528706360 2024-09-12T03:04:40.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.098+0000 7fe98fc82740 1 -- 172.21.15.116:0/3630455042 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe978013e50 con 0x563528706360 2024-09-12T03:04:40.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.098+0000 7fe966ffd640 1 -- 172.21.15.116:0/3630455042 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe978005040 con 0x563528706360 2024-09-12T03:04:40.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.098+0000 7fe98fc82740 10 client.15642 fetch_fsmap learned FSMap version 25 2024-09-12T03:04:40.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.098+0000 7fe98fc82740 1 -- 172.21.15.116:0/3630455042 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5635287901e0 con 0x563528706360 2024-09-12T03:04:40.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.098+0000 7fe966ffd640 1 -- 172.21.15.116:0/3630455042 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe978013e50 con 0x563528706360 2024-09-12T03:04:40.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.098+0000 7fe966ffd640 1 -- 172.21.15.116:0/3630455042 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe978004bb0 con 0x563528706360 2024-09-12T03:04:40.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.098+0000 7fe98fc82740 10 client.15642 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:04:40.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.098+0000 7fe98fc82740 10 client.15642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:04:40.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.098+0000 7fe98fc82740 20 client.15642 populate_metadata read hostname 'smithi116' 2024-09-12T03:04:40.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.098+0000 7fe98fc82740 1 --2- 172.21.15.116:0/3630455042 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5635287907f0 0x563528792be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:40.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.098+0000 7fe98fc82740 4 client.15642 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:04:40.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.098+0000 7fe98fc82740 1 -- 172.21.15.116:0/3630455042 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563528673d60 con 0x5635287907f0 2024-09-12T03:04:40.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.098+0000 7fe986ffd640 1 --2- 172.21.15.116:0/3630455042 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5635287907f0 0x563528792be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:40.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.099+0000 7fe986ffd640 1 --2- 172.21.15.116:0/3630455042 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5635287907f0 0x563528792be0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fd9c38c64053780 server_cookie=9dcbeebd782068ae in_seq=0 out_seq=0 2024-09-12T03:04:40.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.099+0000 7fe966ffd640 10 client.15642 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:40.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.099+0000 7fe966ffd640 1 -- 172.21.15.116:0/3630455042 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563528673d60 con 0x5635287907f0 2024-09-12T03:04:40.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.099+0000 7fe966ffd640 10 client.15642 handle_command_reply: tid=0 2024-09-12T03:04:40.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.099+0000 7fe98fc82740 1 client.15642 shutdown 2024-09-12T03:04:40.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.099+0000 7fe98fc82740 1 -- 172.21.15.116:0/3630455042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563528706360 msgr2=0x563528728740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:40.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.099+0000 7fe98fc82740 1 --2- 172.21.15.116:0/3630455042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563528706360 0x563528728740 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe978005d00 tx=0x7fe978011750 comp rx=0 tx=0).stop 2024-09-12T03:04:40.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.099+0000 7fe98fc82740 1 -- 172.21.15.116:0/3630455042 shutdown_connections 2024-09-12T03:04:40.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.099+0000 7fe98fc82740 1 --2- 172.21.15.116:0/3630455042 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5635287907f0 0x563528792be0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.099+0000 7fe98fc82740 1 --2- 172.21.15.116:0/3630455042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563528715a90 0x5635287291c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.099+0000 7fe98fc82740 1 --2- 172.21.15.116:0/3630455042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563528706c80 0x563528728c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.099+0000 7fe966ffd640 0 client.15642 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:40.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.099+0000 7fe98fc82740 1 --2- 172.21.15.116:0/3630455042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563528706360 0x563528728740 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.099+0000 7fe98fc82740 1 -- 172.21.15.116:0/3630455042 >> 172.21.15.116:0/3630455042 conn(0x563528702a30 msgr2=0x563528701350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:40.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.099+0000 7fe98fc82740 1 -- 172.21.15.116:0/3630455042 shutdown_connections 2024-09-12T03:04:40.129 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:04:40.130 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:04:40.130 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:04:40.130 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:04:40.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.099+0000 7fe98fc82740 1 -- 172.21.15.116:0/3630455042 wait complete. 2024-09-12T03:04:40.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.099+0000 7fe98fc82740 20 client.15642 trim_cache size 0 max 16384 2024-09-12T03:04:40.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.100+0000 7fe98d8a9640 1 -- 172.21.15.116:0/531039921 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe9700a2b80 msgr2=0x7fe9700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:40.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.100+0000 7fe98d8a9640 1 --2- 172.21.15.116:0/531039921 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe9700a2b80 0x7fe9700a5040 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe9800ccf90 tx=0x7fe980086040 comp rx=0 tx=0).stop 2024-09-12T03:04:40.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.100+0000 7fe98d8a9640 1 -- 172.21.15.116:0/531039921 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe988106f50 msgr2=0x7fe98819cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:40.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.100+0000 7fe98d8a9640 1 --2- 172.21.15.116:0/531039921 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe988106f50 0x7fe98819cdc0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe98000cad0 tx=0x7fe98000cf90 comp rx=0 tx=0).stop 2024-09-12T03:04:40.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.100+0000 7fe98d8a9640 1 -- 172.21.15.116:0/531039921 shutdown_connections 2024-09-12T03:04:40.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.100+0000 7fe98d8a9640 1 --2- 172.21.15.116:0/531039921 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe9700a2b80 0x7fe9700a5040 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.100+0000 7fe98d8a9640 1 --2- 172.21.15.116:0/531039921 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe988106f50 0x7fe98819cdc0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.100+0000 7fe98d8a9640 1 --2- 172.21.15.116:0/531039921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe988100d00 0x7fe988198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.100+0000 7fe98d8a9640 1 --2- 172.21.15.116:0/531039921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe988100350 0x7fe9881984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:40.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.100+0000 7fe98d8a9640 1 -- 172.21.15.116:0/531039921 >> 172.21.15.116:0/531039921 conn(0x7fe988077210 msgr2=0x7fe9880fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:40.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.100+0000 7fe98d8a9640 1 -- 172.21.15.116:0/531039921 shutdown_connections 2024-09-12T03:04:40.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:40.100+0000 7fe98d8a9640 1 -- 172.21.15.116:0/531039921 wait complete. 2024-09-12T03:04:40.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:40 smithi116 ceph-mon[31410]: pgmap v388: 97 pgs: 97 active+clean; 858 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:04:40.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:40 smithi183 ceph-mon[35535]: pgmap v388: 97 pgs: 97 active+clean; 858 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:04:40.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:40 smithi177 ceph-mon[35305]: pgmap v388: 97 pgs: 97 active+clean; 858 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:04:40.895 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:04:40.896 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:04:41.143 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:41.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.652+0000 7fc9fc908640 1 -- 172.21.15.116:0/1612557129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9f8106f10 msgr2=0x7fc9f810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:41.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.652+0000 7fc9fc908640 1 --2- 172.21.15.116:0/1612557129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9f8106f10 0x7fc9f810ea90 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009ca0 tx=0x7fc9f002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:04:41.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.652+0000 7fc9fc908640 1 -- 172.21.15.116:0/1612557129 shutdown_connections 2024-09-12T03:04:41.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.652+0000 7fc9fc908640 1 --2- 172.21.15.116:0/1612557129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9f8106f10 0x7fc9f810ea90 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:41.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.652+0000 7fc9fc908640 1 --2- 172.21.15.116:0/1612557129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9f8102790 0x7fc9f81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:41.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.652+0000 7fc9fc908640 1 --2- 172.21.15.116:0/1612557129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9f8101de0 0x7fc9f81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:41.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.652+0000 7fc9fc908640 1 -- 172.21.15.116:0/1612557129 >> 172.21.15.116:0/1612557129 conn(0x7fc9f80fd8c0 msgr2=0x7fc9f80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:41.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.652+0000 7fc9fc908640 1 -- 172.21.15.116:0/1612557129 shutdown_connections 2024-09-12T03:04:41.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.652+0000 7fc9fc908640 1 -- 172.21.15.116:0/1612557129 wait complete. 2024-09-12T03:04:41.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.653+0000 7fc9fc908640 1 Processor -- start 2024-09-12T03:04:41.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.653+0000 7fc9fc908640 1 -- start start 2024-09-12T03:04:41.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.653+0000 7fc9fc908640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9f8101de0 0x7fc9f81984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:41.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.653+0000 7fc9fc908640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9f8102790 0x7fc9f8198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:41.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.653+0000 7fc9fc908640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9f8106f10 0x7fc9f819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:41.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.653+0000 7fc9fc908640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f8111f90 con 0x7fc9f8101de0 2024-09-12T03:04:41.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.653+0000 7fc9fc908640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f8111e10 con 0x7fc9f8106f10 2024-09-12T03:04:41.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.653+0000 7fc9fc908640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f8112110 con 0x7fc9f8102790 2024-09-12T03:04:41.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.653+0000 7fc9f659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9f8101de0 0x7fc9f81984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:41.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.653+0000 7fc9f659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9f8101de0 0x7fc9f81984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56516/0 (socket says 172.21.15.116:56516) 2024-09-12T03:04:41.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.653+0000 7fc9f659c640 1 -- 172.21.15.116:0/3231387613 learned_addr learned my addr 172.21.15.116:0/3231387613 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:41.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.653+0000 7fc9f6d9d640 1 --2- 172.21.15.116:0/3231387613 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9f8106f10 0x7fc9f819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:41.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.653+0000 7fc9f5d9b640 1 --2- 172.21.15.116:0/3231387613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9f8102790 0x7fc9f8198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:41.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.653+0000 7fc9f659c640 1 -- 172.21.15.116:0/3231387613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9f8102790 msgr2=0x7fc9f8198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:41.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.653+0000 7fc9f659c640 1 --2- 172.21.15.116:0/3231387613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9f8102790 0x7fc9f8198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:41.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.654+0000 7fc9f659c640 1 -- 172.21.15.116:0/3231387613 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9f8106f10 msgr2=0x7fc9f819cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:41.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.654+0000 7fc9f659c640 1 --2- 172.21.15.116:0/3231387613 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9f8106f10 0x7fc9f819cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:41.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.654+0000 7fc9f659c640 1 -- 172.21.15.116:0/3231387613 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f819d470 con 0x7fc9f8101de0 2024-09-12T03:04:41.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.654+0000 7fc9f5d9b640 1 --2- 172.21.15.116:0/3231387613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9f8102790 0x7fc9f8198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:04:41.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.654+0000 7fc9f6d9d640 1 --2- 172.21.15.116:0/3231387613 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9f8106f10 0x7fc9f819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:41.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.654+0000 7fc9f659c640 1 --2- 172.21.15.116:0/3231387613 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9f8101de0 0x7fc9f81984c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc9e800b4a0 tx=0x7fc9e800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:41.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.654+0000 7fc9e77fe640 1 -- 172.21.15.116:0/3231387613 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8014070 con 0x7fc9f8101de0 2024-09-12T03:04:41.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.654+0000 7fc9fc908640 1 -- 172.21.15.116:0/3231387613 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f819d760 con 0x7fc9f8101de0 2024-09-12T03:04:41.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.654+0000 7fc9e77fe640 1 -- 172.21.15.116:0/3231387613 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9e8010af0 con 0x7fc9f8101de0 2024-09-12T03:04:41.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.654+0000 7fc9e77fe640 1 -- 172.21.15.116:0/3231387613 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8017d10 con 0x7fc9f8101de0 2024-09-12T03:04:41.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.654+0000 7fc9fc908640 1 -- 172.21.15.116:0/3231387613 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f81a5040 con 0x7fc9f8101de0 2024-09-12T03:04:41.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.655+0000 7fc9fc908640 1 -- 172.21.15.116:0/3231387613 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9f81021c0 con 0x7fc9f8101de0 2024-09-12T03:04:41.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.656+0000 7fc9e77fe640 1 -- 172.21.15.116:0/3231387613 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fc9e8017070 con 0x7fc9f8101de0 2024-09-12T03:04:41.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.657+0000 7fc9e77fe640 1 --2- 172.21.15.116:0/3231387613 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc9e00a2b80 0x7fc9e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:41.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.657+0000 7fc9f5d9b640 1 --2- 172.21.15.116:0/3231387613 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc9e00a2b80 0x7fc9e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:41.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.657+0000 7fc9e77fe640 1 -- 172.21.15.116:0/3231387613 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fc9e80ce500 con 0x7fc9f8101de0 2024-09-12T03:04:41.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.657+0000 7fc9f5d9b640 1 --2- 172.21.15.116:0/3231387613 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc9e00a2b80 0x7fc9e00a5040 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc9f80fe130 tx=0x7fc9ec005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:41.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.661+0000 7fc9e77fe640 1 -- 172.21.15.116:0/3231387613 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc9e809b490 con 0x7fc9f8101de0 2024-09-12T03:04:41.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.861+0000 7fc9fc908640 1 -- 172.21.15.116:0/3231387613 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9f810a150 con 0x7fc9f8101de0 2024-09-12T03:04:41.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.862+0000 7fc9e77fe640 1 -- 172.21.15.116:0/3231387613 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7fc9e809ad90 con 0x7fc9f8101de0 2024-09-12T03:04:41.862 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:04:41.862 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:04:41.863 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:04:41.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.864+0000 7fc9fc908640 1 -- 172.21.15.116:0/3231387613 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc9e00a2b80 msgr2=0x7fc9e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:41.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.864+0000 7fc9fc908640 1 --2- 172.21.15.116:0/3231387613 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc9e00a2b80 0x7fc9e00a5040 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc9f80fe130 tx=0x7fc9ec005e50 comp rx=0 tx=0).stop 2024-09-12T03:04:41.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.864+0000 7fc9fc908640 1 -- 172.21.15.116:0/3231387613 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9f8101de0 msgr2=0x7fc9f81984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:41.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.864+0000 7fc9fc908640 1 --2- 172.21.15.116:0/3231387613 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9f8101de0 0x7fc9f81984c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc9e800b4a0 tx=0x7fc9e800b960 comp rx=0 tx=0).stop 2024-09-12T03:04:41.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.864+0000 7fc9fc908640 1 -- 172.21.15.116:0/3231387613 shutdown_connections 2024-09-12T03:04:41.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.864+0000 7fc9fc908640 1 --2- 172.21.15.116:0/3231387613 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc9e00a2b80 0x7fc9e00a5040 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:41.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.864+0000 7fc9fc908640 1 --2- 172.21.15.116:0/3231387613 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9f8106f10 0x7fc9f819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:41.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.864+0000 7fc9fc908640 1 --2- 172.21.15.116:0/3231387613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9f8102790 0x7fc9f8198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:41.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.864+0000 7fc9fc908640 1 --2- 172.21.15.116:0/3231387613 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9f8101de0 0x7fc9f81984c0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:41.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.864+0000 7fc9fc908640 1 -- 172.21.15.116:0/3231387613 >> 172.21.15.116:0/3231387613 conn(0x7fc9f80fd8c0 msgr2=0x7fc9f80fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:41.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.865+0000 7fc9fc908640 1 -- 172.21.15.116:0/3231387613 shutdown_connections 2024-09-12T03:04:41.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:41.865+0000 7fc9fc908640 1 -- 172.21.15.116:0/3231387613 wait complete. 2024-09-12T03:04:42.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:42 smithi116 ceph-mon[31410]: pgmap v389: 97 pgs: 97 active+clean; 923 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:04:42.543 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:42 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3231387613' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:04:42.547 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:42 smithi177 ceph-mon[35305]: pgmap v389: 97 pgs: 97 active+clean; 923 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:04:42.547 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:42 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3231387613' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:04:42.573 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:04:42.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:42 smithi183 ceph-mon[35535]: pgmap v389: 97 pgs: 97 active+clean; 923 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:04:42.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:42 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3231387613' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:04:42.819 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:43.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.312+0000 7fe534e01640 1 -- 172.21.15.116:0/1486145780 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe530100e20 msgr2=0x7fe530101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:43.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.312+0000 7fe534e01640 1 --2- 172.21.15.116:0/1486145780 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe530100e20 0x7fe530101200 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe520009900 tx=0x7fe52002a480 comp rx=0 tx=0).stop 2024-09-12T03:04:43.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.312+0000 7fe534e01640 1 -- 172.21.15.116:0/1486145780 shutdown_connections 2024-09-12T03:04:43.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.312+0000 7fe534e01640 1 --2- 172.21.15.116:0/1486145780 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe530102ef0 0x7fe53010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.312+0000 7fe534e01640 1 --2- 172.21.15.116:0/1486145780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe530102570 0x7fe5301029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.312+0000 7fe534e01640 1 --2- 172.21.15.116:0/1486145780 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe530100e20 0x7fe530101200 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.312+0000 7fe534e01640 1 -- 172.21.15.116:0/1486145780 >> 172.21.15.116:0/1486145780 conn(0x7fe5300fc7d0 msgr2=0x7fe5300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:43.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.312+0000 7fe534e01640 1 -- 172.21.15.116:0/1486145780 shutdown_connections 2024-09-12T03:04:43.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.312+0000 7fe534e01640 1 -- 172.21.15.116:0/1486145780 wait complete. 2024-09-12T03:04:43.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.312+0000 7fe534e01640 1 Processor -- start 2024-09-12T03:04:43.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.312+0000 7fe534e01640 1 -- start start 2024-09-12T03:04:43.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.313+0000 7fe534e01640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe530100e20 0x7fe530198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:43.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.313+0000 7fe534e01640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe530102570 0x7fe530198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:43.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.313+0000 7fe534e01640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe530102ef0 0x7fe53019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:43.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.313+0000 7fe534e01640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe5301120a0 con 0x7fe530102ef0 2024-09-12T03:04:43.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.313+0000 7fe534e01640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe530111f20 con 0x7fe530100e20 2024-09-12T03:04:43.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.313+0000 7fe534e01640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe530112220 con 0x7fe530102570 2024-09-12T03:04:43.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.313+0000 7fe52ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe530102ef0 0x7fe53019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:43.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.313+0000 7fe52ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe530102ef0 0x7fe53019ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56528/0 (socket says 172.21.15.116:56528) 2024-09-12T03:04:43.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.313+0000 7fe52ed9d640 1 -- 172.21.15.116:0/3562402645 learned_addr learned my addr 172.21.15.116:0/3562402645 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:43.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.313+0000 7fe527fff640 1 --2- 172.21.15.116:0/3562402645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe530102570 0x7fe530198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:43.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.313+0000 7fe52e59c640 1 --2- 172.21.15.116:0/3562402645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe530100e20 0x7fe530198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:43.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.313+0000 7fe52ed9d640 1 -- 172.21.15.116:0/3562402645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe530102570 msgr2=0x7fe530198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:43.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.313+0000 7fe52ed9d640 1 --2- 172.21.15.116:0/3562402645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe530102570 0x7fe530198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.313+0000 7fe52ed9d640 1 -- 172.21.15.116:0/3562402645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe530100e20 msgr2=0x7fe530198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:43.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.313+0000 7fe52ed9d640 1 --2- 172.21.15.116:0/3562402645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe530100e20 0x7fe530198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.313+0000 7fe52ed9d640 1 -- 172.21.15.116:0/3562402645 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe53019d4f0 con 0x7fe530102ef0 2024-09-12T03:04:43.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.313+0000 7fe52e59c640 1 --2- 172.21.15.116:0/3562402645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe530100e20 0x7fe530198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:43.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.313+0000 7fe52ed9d640 1 --2- 172.21.15.116:0/3562402645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe530102ef0 0x7fe53019ce10 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe52800cad0 tx=0x7fe52800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:43.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.313+0000 7fe5277fe640 1 -- 172.21.15.116:0/3562402645 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52800ae20 con 0x7fe530102ef0 2024-09-12T03:04:43.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.313+0000 7fe5277fe640 1 -- 172.21.15.116:0/3562402645 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe528005020 con 0x7fe530102ef0 2024-09-12T03:04:43.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.313+0000 7fe534e01640 1 -- 172.21.15.116:0/3562402645 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe53019d7e0 con 0x7fe530102ef0 2024-09-12T03:04:43.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.313+0000 7fe5277fe640 1 -- 172.21.15.116:0/3562402645 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52800f6f0 con 0x7fe530102ef0 2024-09-12T03:04:43.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.314+0000 7fe534e01640 1 -- 172.21.15.116:0/3562402645 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5301a50c0 con 0x7fe530102ef0 2024-09-12T03:04:43.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.316+0000 7fe5277fe640 1 -- 172.21.15.116:0/3562402645 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe5280051d0 con 0x7fe530102ef0 2024-09-12T03:04:43.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.316+0000 7fe5371da740 1 Processor -- start 2024-09-12T03:04:43.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.316+0000 7fe5371da740 1 -- start start 2024-09-12T03:04:43.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.317+0000 7fe5371da740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e1edd7b40 0x561e1edd6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:43.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.317+0000 7fe5371da740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e1edd65d0 0x561e1edd6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:43.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.317+0000 7fe5371da740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e1ede1370 0x561e1ede3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:43.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.317+0000 7fe5371da740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561e1ed3c2f0 con 0x561e1ede1370 2024-09-12T03:04:43.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.317+0000 7fe5371da740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561e1ece3cf0 con 0x561e1edd7b40 2024-09-12T03:04:43.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.317+0000 7fe5371da740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561e1ed382b0 con 0x561e1edd65d0 2024-09-12T03:04:43.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.317+0000 7fe527fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e1ede1370 0x561e1ede3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:43.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.317+0000 7fe527fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e1ede1370 0x561e1ede3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56532/0 (socket says 172.21.15.116:56532) 2024-09-12T03:04:43.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.317+0000 7fe527fff640 1 -- 172.21.15.116:0/3879156798 learned_addr learned my addr 172.21.15.116:0/3879156798 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:43.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.317+0000 7fe5277fe640 1 --2- 172.21.15.116:0/3562402645 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe51c0a2b80 0x7fe51c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:43.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.317+0000 7fe527fff640 1 --2- 172.21.15.116:0/3879156798 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e1edd7b40 0x561e1edd6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:43.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.317+0000 7fe52e59c640 1 --2- 172.21.15.116:0/3879156798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e1edd65d0 0x561e1edd6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:43.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.317+0000 7fe52ed9d640 1 --2- 172.21.15.116:0/3562402645 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe51c0a2b80 0x7fe51c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:43.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.317+0000 7fe527fff640 1 -- 172.21.15.116:0/3879156798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e1edd65d0 msgr2=0x561e1edd6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:43.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.317+0000 7fe527fff640 1 --2- 172.21.15.116:0/3879156798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e1edd65d0 0x561e1edd6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.317+0000 7fe527fff640 1 -- 172.21.15.116:0/3879156798 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e1edd7b40 msgr2=0x561e1edd6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:43.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.317+0000 7fe527fff640 1 --2- 172.21.15.116:0/3879156798 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e1edd7b40 0x561e1edd6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.317+0000 7fe5277fe640 1 -- 172.21.15.116:0/3562402645 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe5280cf110 con 0x7fe530102ef0 2024-09-12T03:04:43.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.317+0000 7fe527fff640 1 -- 172.21.15.116:0/3879156798 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e1ecc83a0 con 0x561e1ede1370 2024-09-12T03:04:43.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.317+0000 7fe527fff640 1 --2- 172.21.15.116:0/3879156798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e1ede1370 0x561e1ede3760 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe5180127a0 tx=0x7fe51800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca04ed4d6adea09d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:43.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.318+0000 7fe52ed9d640 1 --2- 172.21.15.116:0/3562402645 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe51c0a2b80 0x7fe51c0a5040 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe52809b8d0 tx=0x7fe5280665c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:43.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.318+0000 7fe524ff9640 1 -- 172.21.15.116:0/3879156798 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe518012d20 con 0x561e1ede1370 2024-09-12T03:04:43.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.318+0000 7fe524ff9640 1 -- 172.21.15.116:0/3879156798 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe518005020 con 0x561e1ede1370 2024-09-12T03:04:43.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.318+0000 7fe524ff9640 1 -- 172.21.15.116:0/3879156798 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe518004ae0 con 0x561e1ede1370 2024-09-12T03:04:43.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.318+0000 7fe5371da740 1 -- 172.21.15.116:0/3879156798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e1ede1370 msgr2=0x561e1ede3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:43.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.318+0000 7fe5371da740 1 --2- 172.21.15.116:0/3879156798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e1ede1370 0x561e1ede3760 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe5180127a0 tx=0x7fe51800b7c0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.318+0000 7fe5371da740 1 -- 172.21.15.116:0/3879156798 shutdown_connections 2024-09-12T03:04:43.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.318+0000 7fe5371da740 1 --2- 172.21.15.116:0/3879156798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e1ede1370 0x561e1ede3760 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.318+0000 7fe5371da740 1 --2- 172.21.15.116:0/3879156798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e1edd65d0 0x561e1edd6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.318+0000 7fe5371da740 1 --2- 172.21.15.116:0/3879156798 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e1edd7b40 0x561e1edd6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.318+0000 7fe5371da740 1 -- 172.21.15.116:0/3879156798 >> 172.21.15.116:0/3879156798 conn(0x561e1ed9f310 msgr2=0x561e1edd2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:43.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.318+0000 7fe5371da740 1 -- 172.21.15.116:0/3879156798 shutdown_connections 2024-09-12T03:04:43.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.318+0000 7fe5371da740 1 -- 172.21.15.116:0/3879156798 wait complete. 2024-09-12T03:04:43.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.319+0000 7fe5371da740 1 -- start start 2024-09-12T03:04:43.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.319+0000 7fe5371da740 1 Processor -- start 2024-09-12T03:04:43.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.319+0000 7fe5371da740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e1edd65d0 0x561e1ede0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:43.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.319+0000 7fe5371da740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e1edd7b40 0x561e1edddff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:43.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.319+0000 7fe5371da740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e1ede1370 0x561e1edde5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:43.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.319+0000 7fe5371da740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561e1ede4070 con 0x561e1edd65d0 2024-09-12T03:04:43.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.319+0000 7fe5371da740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561e1ecf7060 con 0x561e1ede1370 2024-09-12T03:04:43.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.319+0000 7fe5371da740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561e1ede41f0 con 0x561e1edd7b40 2024-09-12T03:04:43.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.319+0000 7fe5371da740 10 client.0 _ll_register_callbacks cb 0x561e1ea4a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:43.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.319+0000 7fe527fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e1edd65d0 0x561e1ede0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:43.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.319+0000 7fe527fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e1edd65d0 0x561e1ede0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56548/0 (socket says 172.21.15.116:56548) 2024-09-12T03:04:43.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.319+0000 7fe527fff640 1 -- 172.21.15.116:0/2427901117 learned_addr learned my addr 172.21.15.116:0/2427901117 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:43.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.319+0000 7fe52e59c640 1 --2- 172.21.15.116:0/2427901117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e1edd7b40 0x561e1edddff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:43.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.320+0000 7fe527fff640 1 --2- 172.21.15.116:0/2427901117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e1ede1370 0x561e1edde5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:43.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.320+0000 7fe527fff640 1 -- 172.21.15.116:0/2427901117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e1edd7b40 msgr2=0x561e1edddff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:43.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.320+0000 7fe527fff640 1 --2- 172.21.15.116:0/2427901117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e1edd7b40 0x561e1edddff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.320+0000 7fe527fff640 1 -- 172.21.15.116:0/2427901117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e1ede1370 msgr2=0x561e1edde5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:43.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.320+0000 7fe527fff640 1 --2- 172.21.15.116:0/2427901117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e1ede1370 0x561e1edde5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.320+0000 7fe527fff640 1 -- 172.21.15.116:0/2427901117 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e1eaad3c0 con 0x561e1edd65d0 2024-09-12T03:04:43.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.320+0000 7fe527fff640 1 --2- 172.21.15.116:0/2427901117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e1edd65d0 0x561e1ede0d40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe518001eb0 tx=0x7fe518005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881c8ee0f560c733 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:43.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.320+0000 7fe515ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:04:43.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.320+0000 7fe515ffb640 1 -- 172.21.15.116:0/2427901117 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51802c060 con 0x561e1edd65d0 2024-09-12T03:04:43.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.320+0000 7fe515ffb640 1 -- 172.21.15.116:0/2427901117 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe518004a80 con 0x561e1edd65d0 2024-09-12T03:04:43.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.320+0000 7fe5371da740 1 -- 172.21.15.116:0/2427901117 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe518005020 con 0x561e1edd65d0 2024-09-12T03:04:43.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.320+0000 7fe515ffb640 1 -- 172.21.15.116:0/2427901117 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe5180054e0 con 0x561e1edd65d0 2024-09-12T03:04:43.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.320+0000 7fe5371da740 10 client.15660 fetch_fsmap learned FSMap version 25 2024-09-12T03:04:43.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.320+0000 7fe5371da740 1 -- 172.21.15.116:0/2427901117 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561e1edefae0 con 0x561e1edd65d0 2024-09-12T03:04:43.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.320+0000 7fe515ffb640 1 -- 172.21.15.116:0/2427901117 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe518005020 con 0x561e1edd65d0 2024-09-12T03:04:43.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.321+0000 7fe515ffb640 1 -- 172.21.15.116:0/2427901117 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe518033480 con 0x561e1edd65d0 2024-09-12T03:04:43.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.321+0000 7fe5371da740 10 client.15660 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:04:43.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.321+0000 7fe5371da740 10 client.15660 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:04:43.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.321+0000 7fe5371da740 20 client.15660 populate_metadata read hostname 'smithi116' 2024-09-12T03:04:43.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.321+0000 7fe5371da740 1 --2- 172.21.15.116:0/2427901117 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x561e1edf0e30 0x561e1edf3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:43.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.321+0000 7fe5371da740 4 client.15660 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:04:43.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.321+0000 7fe5371da740 1 -- 172.21.15.116:0/2427901117 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e1ed49860 con 0x561e1edf0e30 2024-09-12T03:04:43.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.321+0000 7fe52e59c640 1 --2- 172.21.15.116:0/2427901117 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x561e1edf0e30 0x561e1edf3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:43.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.321+0000 7fe515ffb640 10 client.15660 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:04:43.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.321+0000 7fe52e59c640 1 --2- 172.21.15.116:0/2427901117 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x561e1edf0e30 0x561e1edf3220 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b65ee16f44cc00fd server_cookie=46f9ef8c69d57384 in_seq=0 out_seq=0 2024-09-12T03:04:43.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.322+0000 7fe515ffb640 1 -- 172.21.15.116:0/2427901117 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e1ed49860 con 0x561e1edf0e30 2024-09-12T03:04:43.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.322+0000 7fe515ffb640 10 client.15660 handle_command_reply: tid=0 2024-09-12T03:04:43.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.322+0000 7fe5371da740 1 client.15660 shutdown 2024-09-12T03:04:43.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.322+0000 7fe5371da740 1 -- 172.21.15.116:0/2427901117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e1edd65d0 msgr2=0x561e1ede0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:43.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.322+0000 7fe5371da740 1 --2- 172.21.15.116:0/2427901117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e1edd65d0 0x561e1ede0d40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe518001eb0 tx=0x7fe518005c50 comp rx=0 tx=0).stop 2024-09-12T03:04:43.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.322+0000 7fe5371da740 1 -- 172.21.15.116:0/2427901117 shutdown_connections 2024-09-12T03:04:43.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.322+0000 7fe5371da740 1 --2- 172.21.15.116:0/2427901117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e1ede1370 0x561e1edde5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.322+0000 7fe5371da740 1 --2- 172.21.15.116:0/2427901117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e1edd7b40 0x561e1edddff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.322+0000 7fe5371da740 1 --2- 172.21.15.116:0/2427901117 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x561e1edf0e30 0x561e1edf3220 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.322+0000 7fe5371da740 1 --2- 172.21.15.116:0/2427901117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e1edd65d0 0x561e1ede0d40 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.322+0000 7fe515ffb640 0 client.15660 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:04:43.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.322+0000 7fe5371da740 1 -- 172.21.15.116:0/2427901117 >> 172.21.15.116:0/2427901117 conn(0x561e1ed9f310 msgr2=0x561e1edd8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:43.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.322+0000 7fe5371da740 1 -- 172.21.15.116:0/2427901117 shutdown_connections 2024-09-12T03:04:43.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.322+0000 7fe5371da740 1 -- 172.21.15.116:0/2427901117 wait complete. 2024-09-12T03:04:43.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.322+0000 7fe5371da740 20 client.15660 trim_cache size 0 max 16384 2024-09-12T03:04:43.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.335+0000 7fe5371da740 1 Processor -- start 2024-09-12T03:04:43.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.335+0000 7fe5371da740 1 -- start start 2024-09-12T03:04:43.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.335+0000 7fe5371da740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e1edd65d0 0x561e1edd2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:43.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.335+0000 7fe5371da740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e1edd7b40 0x561e1eded310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:43.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.335+0000 7fe5371da740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e1ede1370 0x561e1edf88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:43.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.335+0000 7fe5371da740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe5301afe20 con 0x561e1edd7b40 2024-09-12T03:04:43.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.335+0000 7fe5371da740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe5301ad6f0 con 0x561e1edd65d0 2024-09-12T03:04:43.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.335+0000 7fe5371da740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe5301ab540 con 0x561e1ede1370 2024-09-12T03:04:43.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.336+0000 7fe52e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e1edd7b40 0x561e1eded310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:43.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.336+0000 7fe52e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e1edd7b40 0x561e1eded310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56564/0 (socket says 172.21.15.116:56564) 2024-09-12T03:04:43.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.336+0000 7fe52e59c640 1 -- 172.21.15.116:0/1077285390 learned_addr learned my addr 172.21.15.116:0/1077285390 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:43.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.336+0000 7fe527fff640 1 --2- 172.21.15.116:0/1077285390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e1edd65d0 0x561e1edd2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:43.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.336+0000 7fe527fff640 1 --2- 172.21.15.116:0/1077285390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e1ede1370 0x561e1edf88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:43.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.336+0000 7fe52e59c640 1 -- 172.21.15.116:0/1077285390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e1ede1370 msgr2=0x561e1edf88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:43.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.336+0000 7fe52e59c640 1 --2- 172.21.15.116:0/1077285390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e1ede1370 0x561e1edf88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.336+0000 7fe52e59c640 1 -- 172.21.15.116:0/1077285390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e1edd65d0 msgr2=0x561e1edd2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:43.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.336+0000 7fe52e59c640 1 --2- 172.21.15.116:0/1077285390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e1edd65d0 0x561e1edd2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.336+0000 7fe52e59c640 1 -- 172.21.15.116:0/1077285390 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e1ede3f50 con 0x561e1edd7b40 2024-09-12T03:04:43.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.336+0000 7fe527fff640 1 --2- 172.21.15.116:0/1077285390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e1ede1370 0x561e1edf88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:04:43.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.336+0000 7fe52e59c640 1 --2- 172.21.15.116:0/1077285390 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e1edd7b40 0x561e1eded310 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe5200314e0 tx=0x7fe520005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd305389eaba9324 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:43.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.336+0000 7fe517fff640 1 -- 172.21.15.116:0/1077285390 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520035040 con 0x561e1edd7b40 2024-09-12T03:04:43.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.336+0000 7fe517fff640 1 -- 172.21.15.116:0/1077285390 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe520049070 con 0x561e1edd7b40 2024-09-12T03:04:43.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.336+0000 7fe517fff640 1 -- 172.21.15.116:0/1077285390 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520003850 con 0x561e1edd7b40 2024-09-12T03:04:43.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.337+0000 7fe5371da740 1 -- 172.21.15.116:0/1077285390 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e1edd7b40 msgr2=0x561e1eded310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:43.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.337+0000 7fe5371da740 1 --2- 172.21.15.116:0/1077285390 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e1edd7b40 0x561e1eded310 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe5200314e0 tx=0x7fe520005a20 comp rx=0 tx=0).stop 2024-09-12T03:04:43.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.337+0000 7fe5371da740 1 -- 172.21.15.116:0/1077285390 shutdown_connections 2024-09-12T03:04:43.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.337+0000 7fe5371da740 1 --2- 172.21.15.116:0/1077285390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e1ede1370 0x561e1edf88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.337+0000 7fe5371da740 1 --2- 172.21.15.116:0/1077285390 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e1edd7b40 0x561e1eded310 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.337+0000 7fe5371da740 1 --2- 172.21.15.116:0/1077285390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e1edd65d0 0x561e1edd2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.337+0000 7fe5371da740 1 -- 172.21.15.116:0/1077285390 >> 172.21.15.116:0/1077285390 conn(0x561e1ed9f310 msgr2=0x561e1eddd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:43.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.337+0000 7fe5371da740 1 -- 172.21.15.116:0/1077285390 shutdown_connections 2024-09-12T03:04:43.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.337+0000 7fe5371da740 1 -- 172.21.15.116:0/1077285390 wait complete. 2024-09-12T03:04:43.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.337+0000 7fe5371da740 1 -- start start 2024-09-12T03:04:43.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.337+0000 7fe5371da740 1 Processor -- start 2024-09-12T03:04:43.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.338+0000 7fe5371da740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e1edd65d0 0x561e1eddaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:43.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.338+0000 7fe5371da740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e1edd7b40 0x561e1eddb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:43.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.338+0000 7fe5371da740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e1ede1370 0x561e1eddb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:43.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.338+0000 7fe5371da740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe5301abbb0 con 0x561e1edd7b40 2024-09-12T03:04:43.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.338+0000 7fe5371da740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe5301ad020 con 0x561e1edd65d0 2024-09-12T03:04:43.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.338+0000 7fe5371da740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe5301ac280 con 0x561e1ede1370 2024-09-12T03:04:43.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.338+0000 7fe5371da740 10 client.0 _ll_register_callbacks cb 0x7fe51800be70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:43.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.338+0000 7fe52e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e1edd7b40 0x561e1eddb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:43.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.338+0000 7fe52e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e1edd7b40 0x561e1eddb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56568/0 (socket says 172.21.15.116:56568) 2024-09-12T03:04:43.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.338+0000 7fe52e59c640 1 -- 172.21.15.116:0/2877383660 learned_addr learned my addr 172.21.15.116:0/2877383660 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:43.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.338+0000 7fe527fff640 1 --2- 172.21.15.116:0/2877383660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e1ede1370 0x561e1eddb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:43.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.338+0000 7fe527fff640 1 --2- 172.21.15.116:0/2877383660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e1edd65d0 0x561e1eddaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:43.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.338+0000 7fe52e59c640 1 -- 172.21.15.116:0/2877383660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e1ede1370 msgr2=0x561e1eddb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:43.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.338+0000 7fe52e59c640 1 --2- 172.21.15.116:0/2877383660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e1ede1370 0x561e1eddb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.338+0000 7fe52e59c640 1 -- 172.21.15.116:0/2877383660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e1edd65d0 msgr2=0x561e1eddaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:43.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.338+0000 7fe52e59c640 1 --2- 172.21.15.116:0/2877383660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e1edd65d0 0x561e1eddaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.338+0000 7fe52e59c640 1 -- 172.21.15.116:0/2877383660 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e1ee5a8a0 con 0x561e1edd7b40 2024-09-12T03:04:43.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.338+0000 7fe527fff640 1 --2- 172.21.15.116:0/2877383660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e1edd65d0 0x561e1eddaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:43.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.338+0000 7fe52e59c640 1 --2- 172.21.15.116:0/2877383660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e1edd7b40 0x561e1eddb470 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe520005ab0 tx=0x7fe520004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db57e43be1361ed3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:43.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.338+0000 7fe515ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:04:43.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.338+0000 7fe515ffb640 1 -- 172.21.15.116:0/2877383660 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520051070 con 0x561e1edd7b40 2024-09-12T03:04:43.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.339+0000 7fe515ffb640 1 -- 172.21.15.116:0/2877383660 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe520004fb0 con 0x561e1edd7b40 2024-09-12T03:04:43.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.339+0000 7fe5371da740 1 -- 172.21.15.116:0/2877383660 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe520049070 con 0x561e1edd7b40 2024-09-12T03:04:43.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.339+0000 7fe515ffb640 1 -- 172.21.15.116:0/2877383660 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe520035050 con 0x561e1edd7b40 2024-09-12T03:04:43.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.339+0000 7fe5371da740 10 client.15666 fetch_fsmap learned FSMap version 25 2024-09-12T03:04:43.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.339+0000 7fe5371da740 1 -- 172.21.15.116:0/2877383660 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561e1ee5ba60 con 0x561e1edd7b40 2024-09-12T03:04:43.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.339+0000 7fe515ffb640 1 -- 172.21.15.116:0/2877383660 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe520049070 con 0x561e1edd7b40 2024-09-12T03:04:43.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.339+0000 7fe515ffb640 1 -- 172.21.15.116:0/2877383660 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe520031540 con 0x561e1edd7b40 2024-09-12T03:04:43.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.339+0000 7fe5371da740 10 client.15666 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:04:43.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.339+0000 7fe5371da740 10 client.15666 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:04:43.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.339+0000 7fe5371da740 20 client.15666 populate_metadata read hostname 'smithi116' 2024-09-12T03:04:43.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.339+0000 7fe5371da740 1 --2- 172.21.15.116:0/2877383660 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x561e1ee5c210 0x561e1ee5e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:43.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.339+0000 7fe5371da740 4 client.15666 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:04:43.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.339+0000 7fe5371da740 1 -- 172.21.15.116:0/2877383660 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561e1ed43d70 con 0x561e1ee5c210 2024-09-12T03:04:43.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.339+0000 7fe527fff640 1 --2- 172.21.15.116:0/2877383660 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x561e1ee5c210 0x561e1ee5e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:43.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.340+0000 7fe527fff640 1 --2- 172.21.15.116:0/2877383660 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x561e1ee5c210 0x561e1ee5e630 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9d26c0bd41d71c05 server_cookie=bb4e9a9cddd9d5b4 in_seq=0 out_seq=0 2024-09-12T03:04:43.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.340+0000 7fe515ffb640 10 client.15666 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:04:43.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.340+0000 7fe515ffb640 1 -- 172.21.15.116:0/2877383660 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561e1ed43d70 con 0x561e1ee5c210 2024-09-12T03:04:43.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.340+0000 7fe515ffb640 10 client.15666 handle_command_reply: tid=0 2024-09-12T03:04:43.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.340+0000 7fe5371da740 1 client.15666 shutdown 2024-09-12T03:04:43.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.340+0000 7fe5371da740 1 -- 172.21.15.116:0/2877383660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e1edd7b40 msgr2=0x561e1eddb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:43.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.340+0000 7fe5371da740 1 --2- 172.21.15.116:0/2877383660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e1edd7b40 0x561e1eddb470 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe520005ab0 tx=0x7fe520004770 comp rx=0 tx=0).stop 2024-09-12T03:04:43.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.340+0000 7fe5371da740 1 -- 172.21.15.116:0/2877383660 shutdown_connections 2024-09-12T03:04:43.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.340+0000 7fe5371da740 1 --2- 172.21.15.116:0/2877383660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e1ede1370 0x561e1eddb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.340+0000 7fe5371da740 1 --2- 172.21.15.116:0/2877383660 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x561e1ee5c210 0x561e1ee5e630 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.340+0000 7fe5371da740 1 --2- 172.21.15.116:0/2877383660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e1edd7b40 0x561e1eddb470 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.340+0000 7fe5371da740 1 --2- 172.21.15.116:0/2877383660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e1edd65d0 0x561e1eddaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.340+0000 7fe515ffb640 0 client.15666 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:04:43.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.340+0000 7fe5371da740 1 -- 172.21.15.116:0/2877383660 >> 172.21.15.116:0/2877383660 conn(0x561e1ed9f310 msgr2=0x561e1edd14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:43.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.340+0000 7fe5371da740 1 -- 172.21.15.116:0/2877383660 shutdown_connections 2024-09-12T03:04:43.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.340+0000 7fe5371da740 1 -- 172.21.15.116:0/2877383660 wait complete. 2024-09-12T03:04:43.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.340+0000 7fe5371da740 20 client.15666 trim_cache size 0 max 16384 2024-09-12T03:04:43.353 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:04:43.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.341+0000 7fe534e01640 1 -- 172.21.15.116:0/3562402645 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe51c0a2b80 msgr2=0x7fe51c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:43.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.341+0000 7fe534e01640 1 --2- 172.21.15.116:0/3562402645 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe51c0a2b80 0x7fe51c0a5040 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe52809b8d0 tx=0x7fe5280665c0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.341+0000 7fe534e01640 1 -- 172.21.15.116:0/3562402645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe530102ef0 msgr2=0x7fe53019ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:43.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.341+0000 7fe534e01640 1 --2- 172.21.15.116:0/3562402645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe530102ef0 0x7fe53019ce10 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe52800cad0 tx=0x7fe52800cf90 comp rx=0 tx=0).stop 2024-09-12T03:04:43.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.341+0000 7fe534e01640 1 -- 172.21.15.116:0/3562402645 shutdown_connections 2024-09-12T03:04:43.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.341+0000 7fe534e01640 1 --2- 172.21.15.116:0/3562402645 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe51c0a2b80 0x7fe51c0a5040 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.341+0000 7fe534e01640 1 --2- 172.21.15.116:0/3562402645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe530102ef0 0x7fe53019ce10 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.341+0000 7fe534e01640 1 --2- 172.21.15.116:0/3562402645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe530102570 0x7fe530198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.341+0000 7fe534e01640 1 --2- 172.21.15.116:0/3562402645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe530100e20 0x7fe530198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:43.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.341+0000 7fe534e01640 1 -- 172.21.15.116:0/3562402645 >> 172.21.15.116:0/3562402645 conn(0x7fe5300fc7d0 msgr2=0x7fe5300fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:43.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.341+0000 7fe534e01640 1 -- 172.21.15.116:0/3562402645 shutdown_connections 2024-09-12T03:04:43.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:43.341+0000 7fe534e01640 1 -- 172.21.15.116:0/3562402645 wait complete. 2024-09-12T03:04:44.084 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:04:44.332 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:44.452 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:44 smithi116 ceph-mon[31410]: pgmap v390: 97 pgs: 97 active+clean; 923 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:04:44.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:44 smithi183 ceph-mon[35535]: pgmap v390: 97 pgs: 97 active+clean; 923 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:04:44.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:44 smithi177 ceph-mon[35305]: pgmap v390: 97 pgs: 97 active+clean; 923 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:04:44.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.836+0000 7f2a94584640 1 -- 172.21.15.116:0/1186518902 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a8c102570 msgr2=0x7f2a8c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:44.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.836+0000 7f2a94584640 1 --2- 172.21.15.116:0/1186518902 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a8c102570 0x7f2a8c1029b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009900 tx=0x7f2a8002a480 comp rx=0 tx=0).stop 2024-09-12T03:04:44.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.837+0000 7f2a94584640 1 -- 172.21.15.116:0/1186518902 shutdown_connections 2024-09-12T03:04:44.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.837+0000 7f2a94584640 1 --2- 172.21.15.116:0/1186518902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a8c102ef0 0x7f2a8c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.837+0000 7f2a94584640 1 --2- 172.21.15.116:0/1186518902 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a8c102570 0x7f2a8c1029b0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.837+0000 7f2a94584640 1 --2- 172.21.15.116:0/1186518902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a8c100e20 0x7f2a8c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.837+0000 7f2a94584640 1 -- 172.21.15.116:0/1186518902 >> 172.21.15.116:0/1186518902 conn(0x7f2a8c0fc7d0 msgr2=0x7f2a8c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:44.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.837+0000 7f2a94584640 1 -- 172.21.15.116:0/1186518902 shutdown_connections 2024-09-12T03:04:44.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.837+0000 7f2a94584640 1 -- 172.21.15.116:0/1186518902 wait complete. 2024-09-12T03:04:44.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.837+0000 7f2a94584640 1 Processor -- start 2024-09-12T03:04:44.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.837+0000 7f2a94584640 1 -- start start 2024-09-12T03:04:44.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.837+0000 7f2a94584640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a8c100e20 0x7f2a8c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:44.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.837+0000 7f2a94584640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a8c102570 0x7f2a8c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:44.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.837+0000 7f2a94584640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a8c102ef0 0x7f2a8c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:44.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.837+0000 7f2a94584640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c110fa0 con 0x7f2a8c100e20 2024-09-12T03:04:44.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.837+0000 7f2a94584640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c110e20 con 0x7f2a8c102570 2024-09-12T03:04:44.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.837+0000 7f2a94584640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c111120 con 0x7f2a8c102ef0 2024-09-12T03:04:44.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.837+0000 7f2a92320640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a8c100e20 0x7f2a8c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:44.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.838+0000 7f2a92320640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a8c100e20 0x7f2a8c198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56594/0 (socket says 172.21.15.116:56594) 2024-09-12T03:04:44.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.838+0000 7f2a92320640 1 -- 172.21.15.116:0/1985452813 learned_addr learned my addr 172.21.15.116:0/1985452813 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:44.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.838+0000 7f2a91b1f640 1 --2- 172.21.15.116:0/1985452813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a8c102570 0x7f2a8c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:44.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.838+0000 7f2a92b21640 1 --2- 172.21.15.116:0/1985452813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a8c102ef0 0x7f2a8c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:44.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.838+0000 7f2a92320640 1 -- 172.21.15.116:0/1985452813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a8c102ef0 msgr2=0x7f2a8c19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:44.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.838+0000 7f2a92320640 1 --2- 172.21.15.116:0/1985452813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a8c102ef0 0x7f2a8c19cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.838+0000 7f2a92320640 1 -- 172.21.15.116:0/1985452813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a8c102570 msgr2=0x7f2a8c198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:44.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.838+0000 7f2a92320640 1 --2- 172.21.15.116:0/1985452813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a8c102570 0x7f2a8c198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.838+0000 7f2a92320640 1 -- 172.21.15.116:0/1985452813 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8c19d4c0 con 0x7f2a8c100e20 2024-09-12T03:04:44.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.838+0000 7f2a92320640 1 --2- 172.21.15.116:0/1985452813 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a8c100e20 0x7f2a8c198510 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800b4a0 tx=0x7f2a8800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:44.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.838+0000 7f2a7f7fe640 1 -- 172.21.15.116:0/1985452813 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a88014070 con 0x7f2a8c100e20 2024-09-12T03:04:44.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.838+0000 7f2a7f7fe640 1 -- 172.21.15.116:0/1985452813 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a88010af0 con 0x7f2a8c100e20 2024-09-12T03:04:44.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.838+0000 7f2a94584640 1 -- 172.21.15.116:0/1985452813 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a8c19d7b0 con 0x7f2a8c100e20 2024-09-12T03:04:44.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.838+0000 7f2a7f7fe640 1 -- 172.21.15.116:0/1985452813 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a88017d10 con 0x7f2a8c100e20 2024-09-12T03:04:44.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.838+0000 7f2a94584640 1 -- 172.21.15.116:0/1985452813 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a8c1a5090 con 0x7f2a8c100e20 2024-09-12T03:04:44.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.840+0000 7f2a7f7fe640 1 -- 172.21.15.116:0/1985452813 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2a88017070 con 0x7f2a8c100e20 2024-09-12T03:04:44.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.841+0000 7f2a9695d740 1 Processor -- start 2024-09-12T03:04:44.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.841+0000 7f2a9695d740 1 -- start start 2024-09-12T03:04:44.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.841+0000 7f2a9695d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c2889b40 0x5555c2888090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:44.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.841+0000 7f2a9695d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555c28885d0 0x5555c2888a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:44.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.841+0000 7f2a9695d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555c2897780 0x5555c2899b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:44.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.841+0000 7f2a9695d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5555c27ee2f0 con 0x5555c28885d0 2024-09-12T03:04:44.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.841+0000 7f2a9695d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5555c2795cf0 con 0x5555c2897780 2024-09-12T03:04:44.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.841+0000 7f2a9695d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5555c27ea2b0 con 0x5555c2889b40 2024-09-12T03:04:44.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.841+0000 7f2a92320640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555c28885d0 0x5555c2888a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:44.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.841+0000 7f2a92320640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555c28885d0 0x5555c2888a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56604/0 (socket says 172.21.15.116:56604) 2024-09-12T03:04:44.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.841+0000 7f2a92320640 1 -- 172.21.15.116:0/1318889732 learned_addr learned my addr 172.21.15.116:0/1318889732 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:44.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.841+0000 7f2a7f7fe640 1 --2- 172.21.15.116:0/1985452813 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2a780a2b80 0x7f2a780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:44.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.842+0000 7f2a91b1f640 1 --2- 172.21.15.116:0/1318889732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555c2897780 0x5555c2899b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:44.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.842+0000 7f2a92b21640 1 --2- 172.21.15.116:0/1318889732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c2889b40 0x5555c2888090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:44.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.842+0000 7f2a92b21640 1 --2- 172.21.15.116:0/1985452813 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2a780a2b80 0x7f2a780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:44.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.842+0000 7f2a7f7fe640 1 -- 172.21.15.116:0/1985452813 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f2a880ce500 con 0x7f2a8c100e20 2024-09-12T03:04:44.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.842+0000 7f2a92320640 1 -- 172.21.15.116:0/1318889732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c2889b40 msgr2=0x5555c2888090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:44.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.842+0000 7f2a92320640 1 --2- 172.21.15.116:0/1318889732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c2889b40 0x5555c2888090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.842+0000 7f2a92320640 1 -- 172.21.15.116:0/1318889732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555c2897780 msgr2=0x5555c2899b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:44.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.842+0000 7f2a92320640 1 --2- 172.21.15.116:0/1318889732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555c2897780 0x5555c2899b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.842+0000 7f2a92320640 1 -- 172.21.15.116:0/1318889732 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555c277a3a0 con 0x5555c28885d0 2024-09-12T03:04:44.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.842+0000 7f2a91b1f640 1 --2- 172.21.15.116:0/1318889732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555c2897780 0x5555c2899b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:44.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.842+0000 7f2a92b21640 1 --2- 172.21.15.116:0/1985452813 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2a780a2b80 0x7f2a780a5040 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400afd0 tx=0x7f2a84005e90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:44.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.842+0000 7f2a92320640 1 --2- 172.21.15.116:0/1318889732 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555c28885d0 0x5555c2888a10 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2a88097760 tx=0x7f2a880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852db3a9e227ffa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:44.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.842+0000 7f2a7cff9640 1 -- 172.21.15.116:0/1318889732 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8801e070 con 0x5555c28885d0 2024-09-12T03:04:44.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.842+0000 7f2a7cff9640 1 -- 172.21.15.116:0/1318889732 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a8809b490 con 0x5555c28885d0 2024-09-12T03:04:44.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.842+0000 7f2a7cff9640 1 -- 172.21.15.116:0/1318889732 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a88097400 con 0x5555c28885d0 2024-09-12T03:04:44.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.843+0000 7f2a9695d740 1 -- 172.21.15.116:0/1318889732 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555c28885d0 msgr2=0x5555c2888a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:44.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.843+0000 7f2a9695d740 1 --2- 172.21.15.116:0/1318889732 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555c28885d0 0x5555c2888a10 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2a88097760 tx=0x7f2a880046c0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.843+0000 7f2a9695d740 1 -- 172.21.15.116:0/1318889732 shutdown_connections 2024-09-12T03:04:44.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.843+0000 7f2a9695d740 1 --2- 172.21.15.116:0/1318889732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555c2897780 0x5555c2899b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.843+0000 7f2a9695d740 1 --2- 172.21.15.116:0/1318889732 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555c28885d0 0x5555c2888a10 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.843+0000 7f2a9695d740 1 --2- 172.21.15.116:0/1318889732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c2889b40 0x5555c2888090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.843+0000 7f2a9695d740 1 -- 172.21.15.116:0/1318889732 >> 172.21.15.116:0/1318889732 conn(0x5555c2851310 msgr2=0x5555c2884c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:44.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.843+0000 7f2a9695d740 1 -- 172.21.15.116:0/1318889732 shutdown_connections 2024-09-12T03:04:44.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.843+0000 7f2a9695d740 1 -- 172.21.15.116:0/1318889732 wait complete. 2024-09-12T03:04:44.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.843+0000 7f2a9695d740 1 -- start start 2024-09-12T03:04:44.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.843+0000 7f2a9695d740 1 Processor -- start 2024-09-12T03:04:44.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.844+0000 7f2a9695d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555c28885d0 0x5555c28971c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:44.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.844+0000 7f2a9695d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555c2889b40 0x5555c2894470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:44.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.844+0000 7f2a9695d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c2897780 0x5555c2894c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:44.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.844+0000 7f2a9695d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5555c289a420 con 0x5555c2889b40 2024-09-12T03:04:44.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.844+0000 7f2a9695d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5555c27a9060 con 0x5555c28885d0 2024-09-12T03:04:44.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.844+0000 7f2a9695d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5555c289a5a0 con 0x5555c2897780 2024-09-12T03:04:44.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.844+0000 7f2a9695d740 10 client.0 _ll_register_callbacks cb 0x5555c24fc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:44.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.844+0000 7f2a91b1f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555c2889b40 0x5555c2894470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:44.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.844+0000 7f2a91b1f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555c2889b40 0x5555c2894470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56608/0 (socket says 172.21.15.116:56608) 2024-09-12T03:04:44.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.844+0000 7f2a91b1f640 1 -- 172.21.15.116:0/187116967 learned_addr learned my addr 172.21.15.116:0/187116967 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:44.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.844+0000 7f2a92b21640 1 --2- 172.21.15.116:0/187116967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c2897780 0x5555c2894c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:44.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.844+0000 7f2a92320640 1 --2- 172.21.15.116:0/187116967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555c28885d0 0x5555c28971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:44.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.844+0000 7f2a91b1f640 1 -- 172.21.15.116:0/187116967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c2897780 msgr2=0x5555c2894c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:44.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.844+0000 7f2a91b1f640 1 --2- 172.21.15.116:0/187116967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c2897780 0x5555c2894c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.844+0000 7f2a91b1f640 1 -- 172.21.15.116:0/187116967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555c28885d0 msgr2=0x5555c28971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:44.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.844+0000 7f2a91b1f640 1 --2- 172.21.15.116:0/187116967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555c28885d0 0x5555c28971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.844+0000 7f2a91b1f640 1 -- 172.21.15.116:0/187116967 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555c255f3c0 con 0x5555c2889b40 2024-09-12T03:04:44.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.844+0000 7f2a91b1f640 1 --2- 172.21.15.116:0/187116967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555c2889b40 0x5555c2894470 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2a800314e0 tx=0x7f2a800044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4eda03865b0009c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:44.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.844+0000 7f2a75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:04:44.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.845+0000 7f2a75ffb640 1 -- 172.21.15.116:0/187116967 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80035030 con 0x5555c2889b40 2024-09-12T03:04:44.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.845+0000 7f2a9695d740 1 -- 172.21.15.116:0/187116967 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a8809b490 con 0x5555c2889b40 2024-09-12T03:04:44.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.845+0000 7f2a75ffb640 1 -- 172.21.15.116:0/187116967 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f2a80003a80 con 0x5555c2889b40 2024-09-12T03:04:44.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.845+0000 7f2a75ffb640 1 -- 172.21.15.116:0/187116967 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2a8004b070 con 0x5555c2889b40 2024-09-12T03:04:44.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.845+0000 7f2a9695d740 10 client.15678 fetch_fsmap learned FSMap version 25 2024-09-12T03:04:44.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.845+0000 7f2a9695d740 1 -- 172.21.15.116:0/187116967 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5555c28a5f10 con 0x5555c2889b40 2024-09-12T03:04:44.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.845+0000 7f2a75ffb640 1 -- 172.21.15.116:0/187116967 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a8809b490 con 0x5555c2889b40 2024-09-12T03:04:44.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.845+0000 7f2a75ffb640 1 -- 172.21.15.116:0/187116967 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2a8003c440 con 0x5555c2889b40 2024-09-12T03:04:44.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.845+0000 7f2a9695d740 10 client.15678 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:04:44.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.845+0000 7f2a9695d740 10 client.15678 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:04:44.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.845+0000 7f2a9695d740 20 client.15678 populate_metadata read hostname 'smithi116' 2024-09-12T03:04:44.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.845+0000 7f2a9695d740 1 --2- 172.21.15.116:0/187116967 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x5555c28a73d0 0x5555c28a9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:44.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.845+0000 7f2a9695d740 4 client.15678 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:04:44.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.845+0000 7f2a9695d740 1 -- 172.21.15.116:0/187116967 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555c27fb860 con 0x5555c28a73d0 2024-09-12T03:04:44.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.845+0000 7f2a92320640 1 --2- 172.21.15.116:0/187116967 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x5555c28a73d0 0x5555c28a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:44.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.846+0000 7f2a92320640 1 --2- 172.21.15.116:0/187116967 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x5555c28a73d0 0x5555c28a9800 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f5ecd234c609b0b7 server_cookie=94c12b545905280e in_seq=0 out_seq=0 2024-09-12T03:04:44.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.846+0000 7f2a75ffb640 10 client.15678 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:04:44.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.846+0000 7f2a75ffb640 1 -- 172.21.15.116:0/187116967 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5555c27fb860 con 0x5555c28a73d0 2024-09-12T03:04:44.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.847+0000 7f2a75ffb640 10 client.15678 handle_command_reply: tid=0 2024-09-12T03:04:44.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.847+0000 7f2a9695d740 1 client.15678 shutdown 2024-09-12T03:04:44.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.847+0000 7f2a9695d740 1 -- 172.21.15.116:0/187116967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555c2889b40 msgr2=0x5555c2894470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:44.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.847+0000 7f2a9695d740 1 --2- 172.21.15.116:0/187116967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555c2889b40 0x5555c2894470 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2a800314e0 tx=0x7f2a800044c0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.847+0000 7f2a9695d740 1 -- 172.21.15.116:0/187116967 shutdown_connections 2024-09-12T03:04:44.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.847+0000 7f2a9695d740 1 --2- 172.21.15.116:0/187116967 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x5555c28a73d0 0x5555c28a9800 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.847+0000 7f2a9695d740 1 --2- 172.21.15.116:0/187116967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c2897780 0x5555c2894c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.847+0000 7f2a75ffb640 0 client.15678 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:04:44.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.847+0000 7f2a9695d740 1 --2- 172.21.15.116:0/187116967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555c2889b40 0x5555c2894470 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.847+0000 7f2a9695d740 1 --2- 172.21.15.116:0/187116967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555c28885d0 0x5555c28971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.847+0000 7f2a9695d740 1 -- 172.21.15.116:0/187116967 >> 172.21.15.116:0/187116967 conn(0x5555c2851310 msgr2=0x5555c288a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:44.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.847+0000 7f2a9695d740 1 -- 172.21.15.116:0/187116967 shutdown_connections 2024-09-12T03:04:44.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.847+0000 7f2a9695d740 1 -- 172.21.15.116:0/187116967 wait complete. 2024-09-12T03:04:44.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.847+0000 7f2a9695d740 20 client.15678 trim_cache size 0 max 16384 2024-09-12T03:04:44.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.860+0000 7f2a9695d740 1 Processor -- start 2024-09-12T03:04:44.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.860+0000 7f2a9695d740 1 -- start start 2024-09-12T03:04:44.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.860+0000 7f2a9695d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c28885d0 0x5555c2884e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:44.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.860+0000 7f2a9695d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555c2889b40 0x5555c28af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:44.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.860+0000 7f2a9695d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555c2897780 0x5555c28af610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:44.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.860+0000 7f2a9695d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1adb50 con 0x5555c2889b40 2024-09-12T03:04:44.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.860+0000 7f2a9695d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1ab420 con 0x5555c2897780 2024-09-12T03:04:44.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.860+0000 7f2a9695d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1a9270 con 0x5555c28885d0 2024-09-12T03:04:44.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.860+0000 7f2a91b1f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555c2889b40 0x5555c28af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:44.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.860+0000 7f2a91b1f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555c2889b40 0x5555c28af0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56612/0 (socket says 172.21.15.116:56612) 2024-09-12T03:04:44.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.860+0000 7f2a91b1f640 1 -- 172.21.15.116:0/3445767371 learned_addr learned my addr 172.21.15.116:0/3445767371 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:44.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.861+0000 7f2a91b1f640 1 -- 172.21.15.116:0/3445767371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c28885d0 msgr2=0x5555c2884e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:44.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.861+0000 7f2a92b21640 1 --2- 172.21.15.116:0/3445767371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555c2897780 0x5555c28af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:44.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.861+0000 7f2a92320640 1 --2- 172.21.15.116:0/3445767371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c28885d0 0x5555c2884e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:44.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.861+0000 7f2a91b1f640 1 --2- 172.21.15.116:0/3445767371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c28885d0 0x5555c2884e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.861+0000 7f2a91b1f640 1 -- 172.21.15.116:0/3445767371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555c2897780 msgr2=0x5555c28af610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:44.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.861+0000 7f2a91b1f640 1 --2- 172.21.15.116:0/3445767371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555c2897780 0x5555c28af610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.861+0000 7f2a91b1f640 1 -- 172.21.15.116:0/3445767371 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555c28a9e80 con 0x5555c2889b40 2024-09-12T03:04:44.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.861+0000 7f2a92b21640 1 --2- 172.21.15.116:0/3445767371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555c2897780 0x5555c28af610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:04:44.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.861+0000 7f2a91b1f640 1 --2- 172.21.15.116:0/3445767371 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555c2889b40 0x5555c28af0d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2a8003c9d0 tx=0x7f2a80004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2590370ba7dd414 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:44.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.861+0000 7f2a77fff640 1 -- 172.21.15.116:0/3445767371 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80035030 con 0x5555c2889b40 2024-09-12T03:04:44.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.861+0000 7f2a77fff640 1 -- 172.21.15.116:0/3445767371 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a80040070 con 0x5555c2889b40 2024-09-12T03:04:44.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.861+0000 7f2a77fff640 1 -- 172.21.15.116:0/3445767371 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a800317a0 con 0x5555c2889b40 2024-09-12T03:04:44.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.862+0000 7f2a9695d740 1 -- 172.21.15.116:0/3445767371 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555c2889b40 msgr2=0x5555c28af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:44.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.862+0000 7f2a9695d740 1 --2- 172.21.15.116:0/3445767371 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555c2889b40 0x5555c28af0d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2a8003c9d0 tx=0x7f2a80004440 comp rx=0 tx=0).stop 2024-09-12T03:04:44.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.862+0000 7f2a9695d740 1 -- 172.21.15.116:0/3445767371 shutdown_connections 2024-09-12T03:04:44.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.862+0000 7f2a9695d740 1 --2- 172.21.15.116:0/3445767371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555c2897780 0x5555c28af610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.862+0000 7f2a9695d740 1 --2- 172.21.15.116:0/3445767371 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555c2889b40 0x5555c28af0d0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.862+0000 7f2a9695d740 1 --2- 172.21.15.116:0/3445767371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c28885d0 0x5555c2884e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.862+0000 7f2a9695d740 1 -- 172.21.15.116:0/3445767371 >> 172.21.15.116:0/3445767371 conn(0x5555c2851310 msgr2=0x5555c2884700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:44.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.862+0000 7f2a9695d740 1 -- 172.21.15.116:0/3445767371 shutdown_connections 2024-09-12T03:04:44.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.862+0000 7f2a9695d740 1 -- 172.21.15.116:0/3445767371 wait complete. 2024-09-12T03:04:44.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.862+0000 7f2a9695d740 1 -- start start 2024-09-12T03:04:44.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.862+0000 7f2a9695d740 1 Processor -- start 2024-09-12T03:04:44.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.862+0000 7f2a9695d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c28885d0 0x5555c289be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:44.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.862+0000 7f2a9695d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555c2889b40 0x5555c289c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:44.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.862+0000 7f2a9695d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555c2897780 0x5555c290b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:44.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.862+0000 7f2a9695d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1a98e0 con 0x5555c2889b40 2024-09-12T03:04:44.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.862+0000 7f2a9695d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1aad50 con 0x5555c2897780 2024-09-12T03:04:44.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.862+0000 7f2a9695d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1a9fb0 con 0x5555c28885d0 2024-09-12T03:04:44.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.862+0000 7f2a9695d740 10 client.0 _ll_register_callbacks cb 0x7f2a80009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:44.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.863+0000 7f2a91b1f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555c2889b40 0x5555c289c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:44.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.863+0000 7f2a91b1f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555c2889b40 0x5555c289c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56614/0 (socket says 172.21.15.116:56614) 2024-09-12T03:04:44.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.863+0000 7f2a91b1f640 1 -- 172.21.15.116:0/1135471288 learned_addr learned my addr 172.21.15.116:0/1135471288 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:44.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.863+0000 7f2a92b21640 1 --2- 172.21.15.116:0/1135471288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555c2897780 0x5555c290b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:44.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.863+0000 7f2a92320640 1 --2- 172.21.15.116:0/1135471288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c28885d0 0x5555c289be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:44.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.863+0000 7f2a91b1f640 1 -- 172.21.15.116:0/1135471288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c28885d0 msgr2=0x5555c289be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:44.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.863+0000 7f2a91b1f640 1 --2- 172.21.15.116:0/1135471288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c28885d0 0x5555c289be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.863+0000 7f2a91b1f640 1 -- 172.21.15.116:0/1135471288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555c2897780 msgr2=0x5555c290b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:44.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.863+0000 7f2a91b1f640 1 --2- 172.21.15.116:0/1135471288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555c2897780 0x5555c290b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.863+0000 7f2a91b1f640 1 -- 172.21.15.116:0/1135471288 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555c290b750 con 0x5555c2889b40 2024-09-12T03:04:44.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.863+0000 7f2a92320640 1 --2- 172.21.15.116:0/1135471288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c28885d0 0x5555c289be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:44.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.863+0000 7f2a91b1f640 1 --2- 172.21.15.116:0/1135471288 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555c2889b40 0x5555c289c3a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2a8003cfd0 tx=0x7f2a80000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a28337561faa56 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:44.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.863+0000 7f2a75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:04:44.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.863+0000 7f2a75ffb640 1 -- 172.21.15.116:0/1135471288 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8005d070 con 0x5555c2889b40 2024-09-12T03:04:44.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.863+0000 7f2a75ffb640 1 -- 172.21.15.116:0/1135471288 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f2a80003a80 con 0x5555c2889b40 2024-09-12T03:04:44.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.863+0000 7f2a9695d740 1 -- 172.21.15.116:0/1135471288 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a80040070 con 0x5555c2889b40 2024-09-12T03:04:44.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.863+0000 7f2a75ffb640 1 -- 172.21.15.116:0/1135471288 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2a80069050 con 0x5555c2889b40 2024-09-12T03:04:44.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.863+0000 7f2a9695d740 10 client.15684 fetch_fsmap learned FSMap version 25 2024-09-12T03:04:44.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.863+0000 7f2a9695d740 1 -- 172.21.15.116:0/1135471288 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5555c2912a40 con 0x5555c2889b40 2024-09-12T03:04:44.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.864+0000 7f2a75ffb640 1 -- 172.21.15.116:0/1135471288 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a80040070 con 0x5555c2889b40 2024-09-12T03:04:44.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.864+0000 7f2a75ffb640 1 -- 172.21.15.116:0/1135471288 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2a80050480 con 0x5555c2889b40 2024-09-12T03:04:44.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.864+0000 7f2a9695d740 10 client.15684 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:04:44.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.864+0000 7f2a9695d740 10 client.15684 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:04:44.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.864+0000 7f2a9695d740 20 client.15684 populate_metadata read hostname 'smithi116' 2024-09-12T03:04:44.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.864+0000 7f2a9695d740 1 --2- 172.21.15.116:0/1135471288 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x5555c29130a0 0x5555c29154c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:44.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.864+0000 7f2a9695d740 4 client.15684 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:04:44.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.864+0000 7f2a9695d740 1 -- 172.21.15.116:0/1135471288 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5555c27f5d70 con 0x5555c29130a0 2024-09-12T03:04:44.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.864+0000 7f2a92320640 1 --2- 172.21.15.116:0/1135471288 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x5555c29130a0 0x5555c29154c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:44.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.865+0000 7f2a75ffb640 10 client.15684 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:04:44.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.865+0000 7f2a92320640 1 --2- 172.21.15.116:0/1135471288 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x5555c29130a0 0x5555c29154c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c4a183e8fd33b689 server_cookie=8d2b3321e9a98a2d in_seq=0 out_seq=0 2024-09-12T03:04:44.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.865+0000 7f2a75ffb640 1 -- 172.21.15.116:0/1135471288 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5555c27f5d70 con 0x5555c29130a0 2024-09-12T03:04:44.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.865+0000 7f2a75ffb640 10 client.15684 handle_command_reply: tid=0 2024-09-12T03:04:44.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.865+0000 7f2a9695d740 1 client.15684 shutdown 2024-09-12T03:04:44.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.865+0000 7f2a9695d740 1 -- 172.21.15.116:0/1135471288 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555c2889b40 msgr2=0x5555c289c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:44.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.865+0000 7f2a9695d740 1 --2- 172.21.15.116:0/1135471288 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555c2889b40 0x5555c289c3a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2a8003cfd0 tx=0x7f2a80000bc0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.865+0000 7f2a9695d740 1 -- 172.21.15.116:0/1135471288 shutdown_connections 2024-09-12T03:04:44.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.865+0000 7f2a9695d740 1 --2- 172.21.15.116:0/1135471288 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x5555c29130a0 0x5555c29154c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.865+0000 7f2a9695d740 1 --2- 172.21.15.116:0/1135471288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555c2897780 0x5555c290b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.865+0000 7f2a9695d740 1 --2- 172.21.15.116:0/1135471288 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555c2889b40 0x5555c289c3a0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.865+0000 7f2a9695d740 1 --2- 172.21.15.116:0/1135471288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555c28885d0 0x5555c289be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.865+0000 7f2a9695d740 1 -- 172.21.15.116:0/1135471288 >> 172.21.15.116:0/1135471288 conn(0x5555c2851310 msgr2=0x5555c2895cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:44.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.865+0000 7f2a75ffb640 0 client.15684 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:04:44.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.865+0000 7f2a9695d740 1 -- 172.21.15.116:0/1135471288 shutdown_connections 2024-09-12T03:04:44.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.865+0000 7f2a9695d740 1 -- 172.21.15.116:0/1135471288 wait complete. 2024-09-12T03:04:44.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.865+0000 7f2a9695d740 20 client.15684 trim_cache size 0 max 16384 2024-09-12T03:04:44.880 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:04:44.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.866+0000 7f2a94584640 1 -- 172.21.15.116:0/1985452813 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2a780a2b80 msgr2=0x7f2a780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:44.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.866+0000 7f2a94584640 1 --2- 172.21.15.116:0/1985452813 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2a780a2b80 0x7f2a780a5040 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400afd0 tx=0x7f2a84005e90 comp rx=0 tx=0).stop 2024-09-12T03:04:44.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.866+0000 7f2a94584640 1 -- 172.21.15.116:0/1985452813 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a8c100e20 msgr2=0x7f2a8c198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:44.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.866+0000 7f2a94584640 1 --2- 172.21.15.116:0/1985452813 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a8c100e20 0x7f2a8c198510 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800b4a0 tx=0x7f2a8800b960 comp rx=0 tx=0).stop 2024-09-12T03:04:44.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.866+0000 7f2a94584640 1 -- 172.21.15.116:0/1985452813 shutdown_connections 2024-09-12T03:04:44.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.866+0000 7f2a94584640 1 --2- 172.21.15.116:0/1985452813 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2a780a2b80 0x7f2a780a5040 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.866+0000 7f2a94584640 1 --2- 172.21.15.116:0/1985452813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a8c102ef0 0x7f2a8c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.866+0000 7f2a94584640 1 --2- 172.21.15.116:0/1985452813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a8c102570 0x7f2a8c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.866+0000 7f2a94584640 1 --2- 172.21.15.116:0/1985452813 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a8c100e20 0x7f2a8c198510 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:44.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.866+0000 7f2a94584640 1 -- 172.21.15.116:0/1985452813 >> 172.21.15.116:0/1985452813 conn(0x7f2a8c0fc7d0 msgr2=0x7f2a8c06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:44.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.866+0000 7f2a94584640 1 -- 172.21.15.116:0/1985452813 shutdown_connections 2024-09-12T03:04:44.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:44.866+0000 7f2a94584640 1 -- 172.21.15.116:0/1985452813 wait complete. 2024-09-12T03:04:45.580 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:04:45.826 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:46.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.320+0000 7fdc38090640 1 -- 172.21.15.116:0/283346636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc30101420 msgr2=0x7fdc3010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:46.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.320+0000 7fdc38090640 1 --2- 172.21.15.116:0/283346636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc30101420 0x7fdc3010d950 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fdc28009cd0 tx=0x7fdc2802a8e0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.320+0000 7fdc38090640 1 -- 172.21.15.116:0/283346636 shutdown_connections 2024-09-12T03:04:46.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.320+0000 7fdc38090640 1 --2- 172.21.15.116:0/283346636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc30101420 0x7fdc3010d950 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.320+0000 7fdc38090640 1 --2- 172.21.15.116:0/283346636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc30100a80 0x7fdc30100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.320+0000 7fdc38090640 1 --2- 172.21.15.116:0/283346636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc30106a80 0x7fdc30106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.320+0000 7fdc38090640 1 -- 172.21.15.116:0/283346636 >> 172.21.15.116:0/283346636 conn(0x7fdc300fc7a0 msgr2=0x7fdc300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:46.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.320+0000 7fdc38090640 1 -- 172.21.15.116:0/283346636 shutdown_connections 2024-09-12T03:04:46.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.320+0000 7fdc38090640 1 -- 172.21.15.116:0/283346636 wait complete. 2024-09-12T03:04:46.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.320+0000 7fdc38090640 1 Processor -- start 2024-09-12T03:04:46.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.320+0000 7fdc38090640 1 -- start start 2024-09-12T03:04:46.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.321+0000 7fdc38090640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc30100a80 0x7fdc30199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:46.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.321+0000 7fdc38090640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc30101420 0x7fdc301997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:46.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.321+0000 7fdc38090640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc30106a80 0x7fdc301a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:46.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.321+0000 7fdc38090640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdc30110e80 con 0x7fdc30100a80 2024-09-12T03:04:46.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.321+0000 7fdc38090640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdc30110d00 con 0x7fdc30106a80 2024-09-12T03:04:46.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.321+0000 7fdc38090640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdc30111000 con 0x7fdc30101420 2024-09-12T03:04:46.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.321+0000 7fdc35e2c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc30100a80 0x7fdc30199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:46.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.321+0000 7fdc35e2c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc30100a80 0x7fdc30199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56640/0 (socket says 172.21.15.116:56640) 2024-09-12T03:04:46.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.321+0000 7fdc35e2c640 1 -- 172.21.15.116:0/3600145007 learned_addr learned my addr 172.21.15.116:0/3600145007 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:46.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.321+0000 7fdc3562b640 1 --2- 172.21.15.116:0/3600145007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc30101420 0x7fdc301997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:46.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.321+0000 7fdc3662d640 1 --2- 172.21.15.116:0/3600145007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc30106a80 0x7fdc301a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:46.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.321+0000 7fdc35e2c640 1 -- 172.21.15.116:0/3600145007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc30101420 msgr2=0x7fdc301997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:46.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.321+0000 7fdc35e2c640 1 --2- 172.21.15.116:0/3600145007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc30101420 0x7fdc301997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.321+0000 7fdc35e2c640 1 -- 172.21.15.116:0/3600145007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc30106a80 msgr2=0x7fdc301a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:46.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.321+0000 7fdc35e2c640 1 --2- 172.21.15.116:0/3600145007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc30106a80 0x7fdc301a4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.321+0000 7fdc35e2c640 1 -- 172.21.15.116:0/3600145007 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc301a4850 con 0x7fdc30100a80 2024-09-12T03:04:46.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.321+0000 7fdc3662d640 1 --2- 172.21.15.116:0/3600145007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc30106a80 0x7fdc301a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:04:46.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.321+0000 7fdc35e2c640 1 --2- 172.21.15.116:0/3600145007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc30100a80 0x7fdc30199290 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00c970 tx=0x7fdc2c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:46.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.321+0000 7fdc26ffd640 1 -- 172.21.15.116:0/3600145007 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c007bb0 con 0x7fdc30100a80 2024-09-12T03:04:46.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.322+0000 7fdc26ffd640 1 -- 172.21.15.116:0/3600145007 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc2c007d50 con 0x7fdc30100a80 2024-09-12T03:04:46.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.322+0000 7fdc38090640 1 -- 172.21.15.116:0/3600145007 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc301a4a20 con 0x7fdc30100a80 2024-09-12T03:04:46.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.322+0000 7fdc26ffd640 1 -- 172.21.15.116:0/3600145007 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c00f450 con 0x7fdc30100a80 2024-09-12T03:04:46.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.322+0000 7fdc38090640 1 -- 172.21.15.116:0/3600145007 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc301a4f30 con 0x7fdc30100a80 2024-09-12T03:04:46.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.324+0000 7fdc26ffd640 1 -- 172.21.15.116:0/3600145007 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fdc2c005020 con 0x7fdc30100a80 2024-09-12T03:04:46.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.324+0000 7fdc3a469740 1 Processor -- start 2024-09-12T03:04:46.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.324+0000 7fdc3a469740 1 -- start start 2024-09-12T03:04:46.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.324+0000 7fdc3a469740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d63313bb40 0x55d63313a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:46.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.324+0000 7fdc3a469740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63313a5d0 0x55d63313aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:46.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.324+0000 7fdc3a469740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d633145370 0x55d633147760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:46.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.324+0000 7fdc3a469740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d6330a02f0 con 0x55d633145370 2024-09-12T03:04:46.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.324+0000 7fdc3a469740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d633047cf0 con 0x55d63313bb40 2024-09-12T03:04:46.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.324+0000 7fdc3a469740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d63309c2b0 con 0x55d63313a5d0 2024-09-12T03:04:46.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.325+0000 7fdc26ffd640 1 --2- 172.21.15.116:0/3600145007 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fdc1c0a2c00 0x7fdc1c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:46.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.325+0000 7fdc3562b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d633145370 0x55d633147760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:46.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.325+0000 7fdc3562b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d633145370 0x55d633147760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56652/0 (socket says 172.21.15.116:56652) 2024-09-12T03:04:46.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.325+0000 7fdc3562b640 1 -- 172.21.15.116:0/3346285144 learned_addr learned my addr 172.21.15.116:0/3346285144 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:46.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.325+0000 7fdc3662d640 1 --2- 172.21.15.116:0/3600145007 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fdc1c0a2c00 0x7fdc1c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:46.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.325+0000 7fdc3662d640 1 --2- 172.21.15.116:0/3346285144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d63313bb40 0x55d63313a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:46.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.325+0000 7fdc26ffd640 1 -- 172.21.15.116:0/3600145007 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fdc2c0ced30 con 0x7fdc30100a80 2024-09-12T03:04:46.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.325+0000 7fdc35e2c640 1 --2- 172.21.15.116:0/3346285144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63313a5d0 0x55d63313aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:46.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.325+0000 7fdc3562b640 1 -- 172.21.15.116:0/3346285144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63313a5d0 msgr2=0x55d63313aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:46.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.325+0000 7fdc3562b640 1 --2- 172.21.15.116:0/3346285144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63313a5d0 0x55d63313aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.325+0000 7fdc3562b640 1 -- 172.21.15.116:0/3346285144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d63313bb40 msgr2=0x55d63313a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:46.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.325+0000 7fdc3562b640 1 --2- 172.21.15.116:0/3346285144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d63313bb40 0x55d63313a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.325+0000 7fdc3562b640 1 -- 172.21.15.116:0/3346285144 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d63302c3a0 con 0x55d633145370 2024-09-12T03:04:46.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.325+0000 7fdc35e2c640 1 --2- 172.21.15.116:0/3346285144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63313a5d0 0x55d63313aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:46.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.325+0000 7fdc3562b640 1 --2- 172.21.15.116:0/3346285144 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d633145370 0x55d633147760 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fdc2000bc90 tx=0x7fdc200107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec1a5023b6ccb15 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:46.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.325+0000 7fdc1bfff640 1 -- 172.21.15.116:0/3346285144 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20005220 con 0x55d633145370 2024-09-12T03:04:46.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.325+0000 7fdc3662d640 1 --2- 172.21.15.116:0/3600145007 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fdc1c0a2c00 0x7fdc1c0a50c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdc28005e30 tx=0x7fdc280095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:46.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.325+0000 7fdc1bfff640 1 -- 172.21.15.116:0/3346285144 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc2000be40 con 0x55d633145370 2024-09-12T03:04:46.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.325+0000 7fdc1bfff640 1 -- 172.21.15.116:0/3346285144 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20011790 con 0x55d633145370 2024-09-12T03:04:46.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.326+0000 7fdc3a469740 1 -- 172.21.15.116:0/3346285144 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d633145370 msgr2=0x55d633147760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:46.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.326+0000 7fdc3a469740 1 --2- 172.21.15.116:0/3346285144 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d633145370 0x55d633147760 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fdc2000bc90 tx=0x7fdc200107a0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.326+0000 7fdc3a469740 1 -- 172.21.15.116:0/3346285144 shutdown_connections 2024-09-12T03:04:46.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.326+0000 7fdc3a469740 1 --2- 172.21.15.116:0/3346285144 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d633145370 0x55d633147760 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.326+0000 7fdc3a469740 1 --2- 172.21.15.116:0/3346285144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63313a5d0 0x55d63313aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.326+0000 7fdc3a469740 1 --2- 172.21.15.116:0/3346285144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d63313bb40 0x55d63313a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.326+0000 7fdc3a469740 1 -- 172.21.15.116:0/3346285144 >> 172.21.15.116:0/3346285144 conn(0x55d633103310 msgr2=0x55d633136c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:46.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.326+0000 7fdc3a469740 1 -- 172.21.15.116:0/3346285144 shutdown_connections 2024-09-12T03:04:46.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.326+0000 7fdc3a469740 1 -- 172.21.15.116:0/3346285144 wait complete. 2024-09-12T03:04:46.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.326+0000 7fdc3a469740 1 -- start start 2024-09-12T03:04:46.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.327+0000 7fdc3a469740 1 Processor -- start 2024-09-12T03:04:46.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.327+0000 7fdc3a469740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d63313a5d0 0x55d633144d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:46.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.327+0000 7fdc3a469740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63313bb40 0x55d633141fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:46.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.327+0000 7fdc3a469740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d633145370 0x55d6331425d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:46.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.327+0000 7fdc3a469740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d633148070 con 0x55d633145370 2024-09-12T03:04:46.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.327+0000 7fdc3a469740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d63305b060 con 0x55d63313a5d0 2024-09-12T03:04:46.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.327+0000 7fdc3a469740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d6331481f0 con 0x55d63313bb40 2024-09-12T03:04:46.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.327+0000 7fdc3a469740 10 client.0 _ll_register_callbacks cb 0x55d632dae8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:46.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.327+0000 7fdc3662d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d633145370 0x55d6331425d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:46.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.327+0000 7fdc3662d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d633145370 0x55d6331425d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56656/0 (socket says 172.21.15.116:56656) 2024-09-12T03:04:46.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.327+0000 7fdc3662d640 1 -- 172.21.15.116:0/2218119128 learned_addr learned my addr 172.21.15.116:0/2218119128 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:46.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.327+0000 7fdc3562b640 1 --2- 172.21.15.116:0/2218119128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63313bb40 0x55d633141fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:46.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.327+0000 7fdc35e2c640 1 --2- 172.21.15.116:0/2218119128 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d63313a5d0 0x55d633144d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:46.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.327+0000 7fdc3662d640 1 -- 172.21.15.116:0/2218119128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63313bb40 msgr2=0x55d633141fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:46.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.327+0000 7fdc3662d640 1 --2- 172.21.15.116:0/2218119128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63313bb40 0x55d633141fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.327+0000 7fdc3662d640 1 -- 172.21.15.116:0/2218119128 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d63313a5d0 msgr2=0x55d633144d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:46.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.327+0000 7fdc3662d640 1 --2- 172.21.15.116:0/2218119128 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d63313a5d0 0x55d633144d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.327+0000 7fdc3662d640 1 -- 172.21.15.116:0/2218119128 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d632e113c0 con 0x55d633145370 2024-09-12T03:04:46.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.328+0000 7fdc3662d640 1 --2- 172.21.15.116:0/2218119128 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d633145370 0x55d6331425d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fdc28003bf0 tx=0x7fdc28009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6752d9f1437b04e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:46.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.328+0000 7fdc197fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:04:46.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.328+0000 7fdc197fa640 1 -- 172.21.15.116:0/2218119128 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc2804f070 con 0x55d633145370 2024-09-12T03:04:46.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.328+0000 7fdc197fa640 1 -- 172.21.15.116:0/2218119128 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fdc280047c0 con 0x55d633145370 2024-09-12T03:04:46.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.328+0000 7fdc3a469740 1 -- 172.21.15.116:0/2218119128 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d6330ad860 con 0x55d633145370 2024-09-12T03:04:46.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.328+0000 7fdc197fa640 1 -- 172.21.15.116:0/2218119128 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdc2805b050 con 0x55d633145370 2024-09-12T03:04:46.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.328+0000 7fdc3a469740 10 client.15696 fetch_fsmap learned FSMap version 25 2024-09-12T03:04:46.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.328+0000 7fdc3a469740 1 -- 172.21.15.116:0/2218119128 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d633153a40 con 0x55d633145370 2024-09-12T03:04:46.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.328+0000 7fdc197fa640 1 -- 172.21.15.116:0/2218119128 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d6330ad860 con 0x55d633145370 2024-09-12T03:04:46.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.328+0000 7fdc197fa640 1 -- 172.21.15.116:0/2218119128 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdc2804a070 con 0x55d633145370 2024-09-12T03:04:46.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.328+0000 7fdc3a469740 10 client.15696 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:04:46.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.328+0000 7fdc3a469740 10 client.15696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:04:46.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.328+0000 7fdc3a469740 20 client.15696 populate_metadata read hostname 'smithi116' 2024-09-12T03:04:46.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.328+0000 7fdc3a469740 1 --2- 172.21.15.116:0/2218119128 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d633154f10 0x55d6331573d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:46.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.328+0000 7fdc3a469740 4 client.15696 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:04:46.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.328+0000 7fdc3a469740 1 -- 172.21.15.116:0/2218119128 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6330a7d70 con 0x55d633154f10 2024-09-12T03:04:46.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.329+0000 7fdc35e2c640 1 --2- 172.21.15.116:0/2218119128 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d633154f10 0x55d6331573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:46.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.329+0000 7fdc35e2c640 1 --2- 172.21.15.116:0/2218119128 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d633154f10 0x55d6331573d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da05d72582cbd177 server_cookie=343dd622be748801 in_seq=0 out_seq=0 2024-09-12T03:04:46.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.329+0000 7fdc197fa640 10 client.15696 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:46.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.330+0000 7fdc197fa640 1 -- 172.21.15.116:0/2218119128 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d6330a7d70 con 0x55d633154f10 2024-09-12T03:04:46.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.330+0000 7fdc197fa640 10 client.15696 handle_command_reply: tid=0 2024-09-12T03:04:46.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.330+0000 7fdc3a469740 1 client.15696 shutdown 2024-09-12T03:04:46.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.330+0000 7fdc3a469740 1 -- 172.21.15.116:0/2218119128 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d633145370 msgr2=0x55d6331425d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:46.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.330+0000 7fdc3a469740 1 --2- 172.21.15.116:0/2218119128 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d633145370 0x55d6331425d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fdc28003bf0 tx=0x7fdc28009890 comp rx=0 tx=0).stop 2024-09-12T03:04:46.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.330+0000 7fdc3a469740 1 -- 172.21.15.116:0/2218119128 shutdown_connections 2024-09-12T03:04:46.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.330+0000 7fdc3a469740 1 --2- 172.21.15.116:0/2218119128 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d633154f10 0x55d6331573d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.330+0000 7fdc3a469740 1 --2- 172.21.15.116:0/2218119128 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d633145370 0x55d6331425d0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.330+0000 7fdc3a469740 1 --2- 172.21.15.116:0/2218119128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63313bb40 0x55d633141fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.330+0000 7fdc3a469740 1 --2- 172.21.15.116:0/2218119128 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d63313a5d0 0x55d633144d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.330+0000 7fdc197fa640 0 client.15696 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:46.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.330+0000 7fdc3a469740 1 -- 172.21.15.116:0/2218119128 >> 172.21.15.116:0/2218119128 conn(0x55d633103310 msgr2=0x55d63313c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:46.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.330+0000 7fdc3a469740 1 -- 172.21.15.116:0/2218119128 shutdown_connections 2024-09-12T03:04:46.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.330+0000 7fdc3a469740 1 -- 172.21.15.116:0/2218119128 wait complete. 2024-09-12T03:04:46.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.330+0000 7fdc3a469740 20 client.15696 trim_cache size 0 max 16384 2024-09-12T03:04:46.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.343+0000 7fdc3a469740 1 Processor -- start 2024-09-12T03:04:46.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.343+0000 7fdc3a469740 1 -- start start 2024-09-12T03:04:46.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.343+0000 7fdc3a469740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63313a5d0 0x55d633151310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:46.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.343+0000 7fdc3a469740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d63313bb40 0x55d633146760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:46.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.343+0000 7fdc3a469740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d633145370 0x55d633146ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:46.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.343+0000 7fdc3a469740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdc301adbd0 con 0x55d63313bb40 2024-09-12T03:04:46.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.343+0000 7fdc3a469740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdc301ab4a0 con 0x55d633145370 2024-09-12T03:04:46.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.343+0000 7fdc3a469740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdc301a92f0 con 0x55d63313a5d0 2024-09-12T03:04:46.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.343+0000 7fdc3562b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d63313bb40 0x55d633146760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:46.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.343+0000 7fdc3562b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d63313bb40 0x55d633146760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56666/0 (socket says 172.21.15.116:56666) 2024-09-12T03:04:46.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.343+0000 7fdc3562b640 1 -- 172.21.15.116:0/193636235 learned_addr learned my addr 172.21.15.116:0/193636235 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:46.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.343+0000 7fdc3562b640 1 -- 172.21.15.116:0/193636235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63313a5d0 msgr2=0x55d633151310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:46.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.343+0000 7fdc3562b640 1 --2- 172.21.15.116:0/193636235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63313a5d0 0x55d633151310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.343+0000 7fdc3562b640 1 -- 172.21.15.116:0/193636235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d633145370 msgr2=0x55d633146ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:46.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.343+0000 7fdc3562b640 1 --2- 172.21.15.116:0/193636235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d633145370 0x55d633146ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.343+0000 7fdc3562b640 1 -- 172.21.15.116:0/193636235 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d633136ed0 con 0x55d63313bb40 2024-09-12T03:04:46.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.343+0000 7fdc3562b640 1 --2- 172.21.15.116:0/193636235 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d63313bb40 0x55d633146760 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdc2000bbb0 tx=0x7fdc200050f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bbc094bdd1d4421 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:46.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.343+0000 7fdc1b7fe640 1 -- 172.21.15.116:0/193636235 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20005160 con 0x55d63313bb40 2024-09-12T03:04:46.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.344+0000 7fdc1b7fe640 1 -- 172.21.15.116:0/193636235 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc20004660 con 0x55d63313bb40 2024-09-12T03:04:46.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.344+0000 7fdc1b7fe640 1 -- 172.21.15.116:0/193636235 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc200118d0 con 0x55d63313bb40 2024-09-12T03:04:46.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.344+0000 7fdc3a469740 1 -- 172.21.15.116:0/193636235 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d63313bb40 msgr2=0x55d633146760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:46.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.344+0000 7fdc3a469740 1 --2- 172.21.15.116:0/193636235 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d63313bb40 0x55d633146760 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdc2000bbb0 tx=0x7fdc200050f0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.344+0000 7fdc3a469740 1 -- 172.21.15.116:0/193636235 shutdown_connections 2024-09-12T03:04:46.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.344+0000 7fdc3a469740 1 --2- 172.21.15.116:0/193636235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d633145370 0x55d633146ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.344+0000 7fdc3a469740 1 --2- 172.21.15.116:0/193636235 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d63313bb40 0x55d633146760 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.344+0000 7fdc3a469740 1 --2- 172.21.15.116:0/193636235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63313a5d0 0x55d633151310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.344+0000 7fdc3a469740 1 -- 172.21.15.116:0/193636235 >> 172.21.15.116:0/193636235 conn(0x55d633103310 msgr2=0x55d6331410d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:46.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.344+0000 7fdc3a469740 1 -- 172.21.15.116:0/193636235 shutdown_connections 2024-09-12T03:04:46.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.344+0000 7fdc3a469740 1 -- 172.21.15.116:0/193636235 wait complete. 2024-09-12T03:04:46.369 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:04:46.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.344+0000 7fdc3a469740 1 -- start start 2024-09-12T03:04:46.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.344+0000 7fdc3a469740 1 Processor -- start 2024-09-12T03:04:46.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.345+0000 7fdc3a469740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63313a5d0 0x55d63314d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:46.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.345+0000 7fdc3a469740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d63313bb40 0x55d63314d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:46.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.345+0000 7fdc3a469740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d633145370 0x55d633148440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:46.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.345+0000 7fdc3a469740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdc301a9960 con 0x55d63313bb40 2024-09-12T03:04:46.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.345+0000 7fdc3a469740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdc301aadd0 con 0x55d633145370 2024-09-12T03:04:46.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.345+0000 7fdc3a469740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdc301aa030 con 0x55d63313a5d0 2024-09-12T03:04:46.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.345+0000 7fdc3a469740 10 client.0 _ll_register_callbacks cb 0x7fdc28009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:46.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.345+0000 7fdc3562b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d63313bb40 0x55d63314d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:46.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.345+0000 7fdc3562b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d63313bb40 0x55d63314d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56674/0 (socket says 172.21.15.116:56674) 2024-09-12T03:04:46.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.345+0000 7fdc3562b640 1 -- 172.21.15.116:0/4108005188 learned_addr learned my addr 172.21.15.116:0/4108005188 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:46.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.345+0000 7fdc35e2c640 1 --2- 172.21.15.116:0/4108005188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63313a5d0 0x55d63314d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:46.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.345+0000 7fdc3562b640 1 -- 172.21.15.116:0/4108005188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63313a5d0 msgr2=0x55d63314d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:46.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.345+0000 7fdc3562b640 1 --2- 172.21.15.116:0/4108005188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63313a5d0 0x55d63314d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.345+0000 7fdc3562b640 1 -- 172.21.15.116:0/4108005188 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d633145370 msgr2=0x55d633148440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:46.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.345+0000 7fdc3562b640 1 --2- 172.21.15.116:0/4108005188 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d633145370 0x55d633148440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.345+0000 7fdc3562b640 1 -- 172.21.15.116:0/4108005188 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6331450a0 con 0x55d63313bb40 2024-09-12T03:04:46.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.345+0000 7fdc3562b640 1 --2- 172.21.15.116:0/4108005188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d63313bb40 0x55d63314d9d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fdc20010ca0 tx=0x7fdc20005370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8344fba83ef4c24 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:46.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.345+0000 7fdc197fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:04:46.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.346+0000 7fdc197fa640 1 -- 172.21.15.116:0/4108005188 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20004810 con 0x55d63313bb40 2024-09-12T03:04:46.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.346+0000 7fdc197fa640 1 -- 172.21.15.116:0/4108005188 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fdc2002a070 con 0x55d63313bb40 2024-09-12T03:04:46.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.346+0000 7fdc3a469740 1 -- 172.21.15.116:0/4108005188 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc20004660 con 0x55d63313bb40 2024-09-12T03:04:46.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.346+0000 7fdc197fa640 1 -- 172.21.15.116:0/4108005188 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdc200049b0 con 0x55d63313bb40 2024-09-12T03:04:46.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.346+0000 7fdc3a469740 10 client.15702 fetch_fsmap learned FSMap version 25 2024-09-12T03:04:46.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.346+0000 7fdc3a469740 1 -- 172.21.15.116:0/4108005188 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d6331c0390 con 0x55d63313bb40 2024-09-12T03:04:46.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.346+0000 7fdc197fa640 1 -- 172.21.15.116:0/4108005188 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc20004660 con 0x55d63313bb40 2024-09-12T03:04:46.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.346+0000 7fdc197fa640 1 -- 172.21.15.116:0/4108005188 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdc2002d410 con 0x55d63313bb40 2024-09-12T03:04:46.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.346+0000 7fdc3a469740 10 client.15702 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:04:46.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.346+0000 7fdc3a469740 10 client.15702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:04:46.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.346+0000 7fdc3a469740 20 client.15702 populate_metadata read hostname 'smithi116' 2024-09-12T03:04:46.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.346+0000 7fdc3a469740 1 --2- 172.21.15.116:0/4108005188 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d6331c0860 0x55d6331c2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:46.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.346+0000 7fdc3a469740 4 client.15702 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:04:46.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.346+0000 7fdc3a469740 1 -- 172.21.15.116:0/4108005188 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d633024170 con 0x55d6331c0860 2024-09-12T03:04:46.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.346+0000 7fdc35e2c640 1 --2- 172.21.15.116:0/4108005188 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d6331c0860 0x55d6331c2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:46.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.347+0000 7fdc35e2c640 1 --2- 172.21.15.116:0/4108005188 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d6331c0860 0x55d6331c2c80 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da6286ae0b6968cb server_cookie=a4339e97e11613e9 in_seq=0 out_seq=0 2024-09-12T03:04:46.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.347+0000 7fdc197fa640 10 client.15702 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:46.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.347+0000 7fdc197fa640 1 -- 172.21.15.116:0/4108005188 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d633024170 con 0x55d6331c0860 2024-09-12T03:04:46.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.347+0000 7fdc197fa640 10 client.15702 handle_command_reply: tid=0 2024-09-12T03:04:46.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.347+0000 7fdc3a469740 1 client.15702 shutdown 2024-09-12T03:04:46.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.347+0000 7fdc3a469740 1 -- 172.21.15.116:0/4108005188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d63313bb40 msgr2=0x55d63314d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:46.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.347+0000 7fdc3a469740 1 --2- 172.21.15.116:0/4108005188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d63313bb40 0x55d63314d9d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fdc20010ca0 tx=0x7fdc20005370 comp rx=0 tx=0).stop 2024-09-12T03:04:46.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.347+0000 7fdc3a469740 1 -- 172.21.15.116:0/4108005188 shutdown_connections 2024-09-12T03:04:46.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.347+0000 7fdc3a469740 1 --2- 172.21.15.116:0/4108005188 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d6331c0860 0x55d6331c2c80 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.347+0000 7fdc3a469740 1 --2- 172.21.15.116:0/4108005188 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d633145370 0x55d633148440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.347+0000 7fdc3a469740 1 --2- 172.21.15.116:0/4108005188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d63313bb40 0x55d63314d9d0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.347+0000 7fdc3a469740 1 --2- 172.21.15.116:0/4108005188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d63313a5d0 0x55d63314d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.347+0000 7fdc197fa640 0 client.15702 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:46.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.347+0000 7fdc3a469740 1 -- 172.21.15.116:0/4108005188 >> 172.21.15.116:0/4108005188 conn(0x55d633103310 msgr2=0x55d6331354e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:46.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.347+0000 7fdc3a469740 1 -- 172.21.15.116:0/4108005188 shutdown_connections 2024-09-12T03:04:46.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.347+0000 7fdc3a469740 1 -- 172.21.15.116:0/4108005188 wait complete. 2024-09-12T03:04:46.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.347+0000 7fdc3a469740 20 client.15702 trim_cache size 0 max 16384 2024-09-12T03:04:46.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.348+0000 7fdc38090640 1 -- 172.21.15.116:0/3600145007 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fdc1c0a2c00 msgr2=0x7fdc1c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:46.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.348+0000 7fdc38090640 1 --2- 172.21.15.116:0/3600145007 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fdc1c0a2c00 0x7fdc1c0a50c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdc28005e30 tx=0x7fdc280095c0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.348+0000 7fdc38090640 1 -- 172.21.15.116:0/3600145007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc30100a80 msgr2=0x7fdc30199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:46.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.348+0000 7fdc38090640 1 --2- 172.21.15.116:0/3600145007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc30100a80 0x7fdc30199290 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00c970 tx=0x7fdc2c00ce30 comp rx=0 tx=0).stop 2024-09-12T03:04:46.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.348+0000 7fdc38090640 1 -- 172.21.15.116:0/3600145007 shutdown_connections 2024-09-12T03:04:46.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.348+0000 7fdc38090640 1 --2- 172.21.15.116:0/3600145007 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fdc1c0a2c00 0x7fdc1c0a50c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.348+0000 7fdc38090640 1 --2- 172.21.15.116:0/3600145007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc30106a80 0x7fdc301a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.348+0000 7fdc38090640 1 --2- 172.21.15.116:0/3600145007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc30101420 0x7fdc301997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.348+0000 7fdc38090640 1 --2- 172.21.15.116:0/3600145007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc30100a80 0x7fdc30199290 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.348+0000 7fdc38090640 1 -- 172.21.15.116:0/3600145007 >> 172.21.15.116:0/3600145007 conn(0x7fdc300fc7a0 msgr2=0x7fdc300fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:46.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.348+0000 7fdc38090640 1 -- 172.21.15.116:0/3600145007 shutdown_connections 2024-09-12T03:04:46.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:46.348+0000 7fdc38090640 1 -- 172.21.15.116:0/3600145007 wait complete. 2024-09-12T03:04:46.505 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:46 smithi183 ceph-mon[35535]: pgmap v391: 97 pgs: 97 active+clean; 923 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:04:46.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:46 smithi116 ceph-mon[31410]: pgmap v391: 97 pgs: 97 active+clean; 923 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:04:46.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:46 smithi177 ceph-mon[35305]: pgmap v391: 97 pgs: 97 active+clean; 923 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:04:48.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:48 smithi116 ceph-mon[31410]: pgmap v392: 97 pgs: 97 active+clean; 980 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:04:48.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:48 smithi183 ceph-mon[35535]: pgmap v392: 97 pgs: 97 active+clean; 980 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:04:48.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:48 smithi177 ceph-mon[35305]: pgmap v392: 97 pgs: 97 active+clean; 980 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:04:49.999 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:04:49.999 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:04:50.258 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:50.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:50 smithi116 ceph-mon[31410]: pgmap v393: 97 pgs: 97 active+clean; 980 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:04:50.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:50 smithi183 ceph-mon[35535]: pgmap v393: 97 pgs: 97 active+clean; 980 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:04:50.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:50 smithi177 ceph-mon[35305]: pgmap v393: 97 pgs: 97 active+clean; 980 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:04:50.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.770+0000 7fa3965c5640 1 -- 172.21.15.116:0/494423851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa390101de0 msgr2=0x7fa3901021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:50.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.770+0000 7fa3965c5640 1 --2- 172.21.15.116:0/494423851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa390101de0 0x7fa3901021c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa380009900 tx=0x7fa38002a410 comp rx=0 tx=0).stop 2024-09-12T03:04:50.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.770+0000 7fa3965c5640 1 -- 172.21.15.116:0/494423851 shutdown_connections 2024-09-12T03:04:50.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.770+0000 7fa3965c5640 1 --2- 172.21.15.116:0/494423851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa390106f10 0x7fa39010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.770+0000 7fa3965c5640 1 --2- 172.21.15.116:0/494423851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa390102790 0x7fa3901067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.770+0000 7fa3965c5640 1 --2- 172.21.15.116:0/494423851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa390101de0 0x7fa3901021c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.770+0000 7fa3965c5640 1 -- 172.21.15.116:0/494423851 >> 172.21.15.116:0/494423851 conn(0x7fa3900fd8c0 msgr2=0x7fa3900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:50.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.770+0000 7fa3965c5640 1 -- 172.21.15.116:0/494423851 shutdown_connections 2024-09-12T03:04:50.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.770+0000 7fa3965c5640 1 -- 172.21.15.116:0/494423851 wait complete. 2024-09-12T03:04:50.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.771+0000 7fa3965c5640 1 Processor -- start 2024-09-12T03:04:50.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.771+0000 7fa3965c5640 1 -- start start 2024-09-12T03:04:50.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.771+0000 7fa3965c5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa390101de0 0x7fa390198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.771+0000 7fa3965c5640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa390102790 0x7fa390198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.771+0000 7fa3965c5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa390106f10 0x7fa39019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.771+0000 7fa3965c5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa390111f30 con 0x7fa390101de0 2024-09-12T03:04:50.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.771+0000 7fa3965c5640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa390111db0 con 0x7fa390102790 2024-09-12T03:04:50.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.771+0000 7fa3965c5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa3901120b0 con 0x7fa390106f10 2024-09-12T03:04:50.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.771+0000 7fa38ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa390101de0 0x7fa390198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.771+0000 7fa38ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa390101de0 0x7fa390198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56696/0 (socket says 172.21.15.116:56696) 2024-09-12T03:04:50.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.771+0000 7fa38ffff640 1 -- 172.21.15.116:0/1415102926 learned_addr learned my addr 172.21.15.116:0/1415102926 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:50.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.772+0000 7fa394b62640 1 --2- 172.21.15.116:0/1415102926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa390106f10 0x7fa39019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.772+0000 7fa38f7fe640 1 --2- 172.21.15.116:0/1415102926 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa390102790 0x7fa390198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.772+0000 7fa38ffff640 1 -- 172.21.15.116:0/1415102926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa390106f10 msgr2=0x7fa39019cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:50.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.772+0000 7fa38ffff640 1 --2- 172.21.15.116:0/1415102926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa390106f10 0x7fa39019cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.772+0000 7fa38ffff640 1 -- 172.21.15.116:0/1415102926 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa390102790 msgr2=0x7fa390198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:50.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.772+0000 7fa38ffff640 1 --2- 172.21.15.116:0/1415102926 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa390102790 0x7fa390198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.772+0000 7fa38ffff640 1 -- 172.21.15.116:0/1415102926 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa39019d4c0 con 0x7fa390101de0 2024-09-12T03:04:50.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.772+0000 7fa38f7fe640 1 --2- 172.21.15.116:0/1415102926 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa390102790 0x7fa390198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:04:50.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.772+0000 7fa38ffff640 1 --2- 172.21.15.116:0/1415102926 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa390101de0 0x7fa390198510 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa380006010 tx=0x7fa3800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:50.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.772+0000 7fa38d7fa640 1 -- 172.21.15.116:0/1415102926 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38003f030 con 0x7fa390101de0 2024-09-12T03:04:50.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.772+0000 7fa38d7fa640 1 -- 172.21.15.116:0/1415102926 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa380003ce0 con 0x7fa390101de0 2024-09-12T03:04:50.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.772+0000 7fa3965c5640 1 -- 172.21.15.116:0/1415102926 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa39019d750 con 0x7fa390101de0 2024-09-12T03:04:50.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.772+0000 7fa38d7fa640 1 -- 172.21.15.116:0/1415102926 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa380035070 con 0x7fa390101de0 2024-09-12T03:04:50.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.772+0000 7fa3965c5640 1 -- 172.21.15.116:0/1415102926 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3901a5090 con 0x7fa390101de0 2024-09-12T03:04:50.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.774+0000 7fa38d7fa640 1 -- 172.21.15.116:0/1415102926 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa380042030 con 0x7fa390101de0 2024-09-12T03:04:50.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.775+0000 7fa39899e740 1 Processor -- start 2024-09-12T03:04:50.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.775+0000 7fa39899e740 1 -- start start 2024-09-12T03:04:50.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.775+0000 7fa39899e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e62a033a40 0x55e62a033e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.775+0000 7fa39899e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e62a034360 0x55e62a042c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.775+0000 7fa39899e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e62a043150 0x55e62a045540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.775+0000 7fa39899e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e629f99280 con 0x55e62a034360 2024-09-12T03:04:50.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.775+0000 7fa39899e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e629f40b10 con 0x55e62a033a40 2024-09-12T03:04:50.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.775+0000 7fa38d7fa640 1 --2- 172.21.15.116:0/1415102926 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa3780a2b80 0x7fa3780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.775+0000 7fa39899e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e629f95240 con 0x55e62a043150 2024-09-12T03:04:50.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.775+0000 7fa38ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e62a034360 0x55e62a042c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.775+0000 7fa394b62640 1 --2- 172.21.15.116:0/1415102926 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa3780a2b80 0x7fa3780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.775+0000 7fa38ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e62a034360 0x55e62a042c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56702/0 (socket says 172.21.15.116:56702) 2024-09-12T03:04:50.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.775+0000 7fa38ffff640 1 -- 172.21.15.116:0/1950505723 learned_addr learned my addr 172.21.15.116:0/1950505723 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:50.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.775+0000 7fa394b62640 1 --2- 172.21.15.116:0/1950505723 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e62a033a40 0x55e62a033e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.775+0000 7fa38f7fe640 1 --2- 172.21.15.116:0/1950505723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e62a043150 0x55e62a045540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.775+0000 7fa38d7fa640 1 -- 172.21.15.116:0/1415102926 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa3800ef6d0 con 0x7fa390101de0 2024-09-12T03:04:50.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.775+0000 7fa38ffff640 1 -- 172.21.15.116:0/1950505723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e62a043150 msgr2=0x55e62a045540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:50.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.775+0000 7fa38ffff640 1 --2- 172.21.15.116:0/1950505723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e62a043150 0x55e62a045540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.775+0000 7fa38ffff640 1 -- 172.21.15.116:0/1950505723 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e62a033a40 msgr2=0x55e62a033e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:50.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.775+0000 7fa38ffff640 1 --2- 172.21.15.116:0/1950505723 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e62a033a40 0x55e62a033e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.775+0000 7fa38ffff640 1 -- 172.21.15.116:0/1950505723 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e629f4d1f0 con 0x55e62a034360 2024-09-12T03:04:50.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.776+0000 7fa394b62640 1 --2- 172.21.15.116:0/1415102926 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa3780a2b80 0x7fa3780a5040 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa3880098b0 tx=0x7fa388009d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:50.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.776+0000 7fa38ffff640 1 --2- 172.21.15.116:0/1950505723 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e62a034360 0x55e62a042c10 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa3800b89f0 tx=0x7fa38008f1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9edb38921953c057 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:50.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.776+0000 7fa3767fc640 1 -- 172.21.15.116:0/1950505723 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa380049070 con 0x55e62a034360 2024-09-12T03:04:50.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.776+0000 7fa3767fc640 1 -- 172.21.15.116:0/1950505723 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa38003b580 con 0x55e62a034360 2024-09-12T03:04:50.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.776+0000 7fa3767fc640 1 -- 172.21.15.116:0/1950505723 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3800d8040 con 0x55e62a034360 2024-09-12T03:04:50.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.776+0000 7fa39899e740 1 -- 172.21.15.116:0/1950505723 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e62a034360 msgr2=0x55e62a042c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:50.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.776+0000 7fa39899e740 1 --2- 172.21.15.116:0/1950505723 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e62a034360 0x55e62a042c10 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa3800b89f0 tx=0x7fa38008f1a0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.777+0000 7fa39899e740 1 -- 172.21.15.116:0/1950505723 shutdown_connections 2024-09-12T03:04:50.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.777+0000 7fa39899e740 1 --2- 172.21.15.116:0/1950505723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e62a043150 0x55e62a045540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.777+0000 7fa39899e740 1 --2- 172.21.15.116:0/1950505723 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e62a034360 0x55e62a042c10 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.777+0000 7fa39899e740 1 --2- 172.21.15.116:0/1950505723 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e62a033a40 0x55e62a033e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.777+0000 7fa39899e740 1 -- 172.21.15.116:0/1950505723 >> 172.21.15.116:0/1950505723 conn(0x55e62a0300f0 msgr2=0x55e62a0304d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:50.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.777+0000 7fa39899e740 1 -- 172.21.15.116:0/1950505723 shutdown_connections 2024-09-12T03:04:50.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.777+0000 7fa39899e740 1 -- 172.21.15.116:0/1950505723 wait complete. 2024-09-12T03:04:50.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.777+0000 7fa39899e740 1 -- start start 2024-09-12T03:04:50.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.777+0000 7fa39899e740 1 Processor -- start 2024-09-12T03:04:50.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.777+0000 7fa39899e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e62a033a40 0x55e62a040090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.777+0000 7fa39899e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e62a034360 0x55e62a0405d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.777+0000 7fa39899e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e62a043150 0x55e62a040b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.777+0000 7fa39899e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e62a045ae0 con 0x55e62a033a40 2024-09-12T03:04:50.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.777+0000 7fa39899e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e629f53f10 con 0x55e62a043150 2024-09-12T03:04:50.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.777+0000 7fa39899e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e62a045c60 con 0x55e62a034360 2024-09-12T03:04:50.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.777+0000 7fa39899e740 10 client.0 _ll_register_callbacks cb 0x55e629f1ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:50.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.777+0000 7fa38ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e62a033a40 0x55e62a040090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.778+0000 7fa38ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e62a033a40 0x55e62a040090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56718/0 (socket says 172.21.15.116:56718) 2024-09-12T03:04:50.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.778+0000 7fa38ffff640 1 -- 172.21.15.116:0/3452782659 learned_addr learned my addr 172.21.15.116:0/3452782659 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:50.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.778+0000 7fa394b62640 1 --2- 172.21.15.116:0/3452782659 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e62a043150 0x55e62a040b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.778+0000 7fa38f7fe640 1 --2- 172.21.15.116:0/3452782659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e62a034360 0x55e62a0405d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.778+0000 7fa38ffff640 1 -- 172.21.15.116:0/3452782659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e62a034360 msgr2=0x55e62a0405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:50.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.778+0000 7fa38ffff640 1 --2- 172.21.15.116:0/3452782659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e62a034360 0x55e62a0405d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.778+0000 7fa38ffff640 1 -- 172.21.15.116:0/3452782659 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e62a043150 msgr2=0x55e62a040b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:50.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.778+0000 7fa38ffff640 1 --2- 172.21.15.116:0/3452782659 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e62a043150 0x55e62a040b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.778+0000 7fa38ffff640 1 -- 172.21.15.116:0/3452782659 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e629d0a6a0 con 0x55e62a033a40 2024-09-12T03:04:50.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.778+0000 7fa394b62640 1 --2- 172.21.15.116:0/3452782659 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e62a043150 0x55e62a040b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:50.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.778+0000 7fa38f7fe640 1 --2- 172.21.15.116:0/3452782659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e62a034360 0x55e62a0405d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:04:50.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.778+0000 7fa38ffff640 1 --2- 172.21.15.116:0/3452782659 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e62a033a40 0x55e62a040090 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa3800c4760 tx=0x7fa3800bbef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f57330090e6d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:50.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.778+0000 7fa36f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:04:50.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.778+0000 7fa36f7fe640 1 -- 172.21.15.116:0/3452782659 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa380049070 con 0x55e62a033a40 2024-09-12T03:04:50.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.778+0000 7fa36f7fe640 1 -- 172.21.15.116:0/3452782659 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa3800b8690 con 0x55e62a033a40 2024-09-12T03:04:50.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.778+0000 7fa39899e740 1 -- 172.21.15.116:0/3452782659 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e629fa67f0 con 0x55e62a033a40 2024-09-12T03:04:50.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.778+0000 7fa36f7fe640 1 -- 172.21.15.116:0/3452782659 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa3800d8040 con 0x55e62a033a40 2024-09-12T03:04:50.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.778+0000 7fa39899e740 10 client.15714 fetch_fsmap learned FSMap version 25 2024-09-12T03:04:50.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.778+0000 7fa39899e740 1 -- 172.21.15.116:0/3452782659 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e62a045ae0 con 0x55e62a033a40 2024-09-12T03:04:50.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.778+0000 7fa36f7fe640 1 -- 172.21.15.116:0/3452782659 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e629fa67f0 con 0x55e62a033a40 2024-09-12T03:04:50.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.779+0000 7fa36f7fe640 1 -- 172.21.15.116:0/3452782659 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa3800bc280 con 0x55e62a033a40 2024-09-12T03:04:50.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.779+0000 7fa39899e740 10 client.15714 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:04:50.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.779+0000 7fa39899e740 10 client.15714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:04:50.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.779+0000 7fa39899e740 20 client.15714 populate_metadata read hostname 'smithi116' 2024-09-12T03:04:50.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.779+0000 7fa39899e740 1 --2- 172.21.15.116:0/3452782659 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e62a0529a0 0x55e62a054e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.779+0000 7fa39899e740 4 client.15714 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:04:50.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.779+0000 7fa39899e740 1 -- 172.21.15.116:0/3452782659 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e629fa0d00 con 0x55e62a0529a0 2024-09-12T03:04:50.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.779+0000 7fa38f7fe640 1 --2- 172.21.15.116:0/3452782659 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e62a0529a0 0x55e62a054e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.779+0000 7fa36f7fe640 10 client.15714 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:50.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.779+0000 7fa38f7fe640 1 --2- 172.21.15.116:0/3452782659 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e62a0529a0 0x55e62a054e40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e1a58e5cf1d1dab server_cookie=cb8cc246de1ddece in_seq=0 out_seq=0 2024-09-12T03:04:50.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.780+0000 7fa36f7fe640 1 -- 172.21.15.116:0/3452782659 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e629fa0d00 con 0x55e62a0529a0 2024-09-12T03:04:50.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.780+0000 7fa36f7fe640 10 client.15714 handle_command_reply: tid=0 2024-09-12T03:04:50.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.780+0000 7fa39899e740 1 client.15714 shutdown 2024-09-12T03:04:50.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.780+0000 7fa39899e740 1 -- 172.21.15.116:0/3452782659 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e62a033a40 msgr2=0x55e62a040090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:50.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.780+0000 7fa39899e740 1 --2- 172.21.15.116:0/3452782659 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e62a033a40 0x55e62a040090 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa3800c4760 tx=0x7fa3800bbef0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.780+0000 7fa39899e740 1 -- 172.21.15.116:0/3452782659 shutdown_connections 2024-09-12T03:04:50.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.780+0000 7fa39899e740 1 --2- 172.21.15.116:0/3452782659 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e62a0529a0 0x55e62a054e40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.780+0000 7fa39899e740 1 --2- 172.21.15.116:0/3452782659 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e62a043150 0x55e62a040b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.780+0000 7fa39899e740 1 --2- 172.21.15.116:0/3452782659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e62a034360 0x55e62a0405d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.780+0000 7fa36f7fe640 0 client.15714 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:50.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.780+0000 7fa39899e740 1 --2- 172.21.15.116:0/3452782659 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e62a033a40 0x55e62a040090 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.780+0000 7fa39899e740 1 -- 172.21.15.116:0/3452782659 >> 172.21.15.116:0/3452782659 conn(0x55e62a0300f0 msgr2=0x55e62a035e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:50.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.780+0000 7fa39899e740 1 -- 172.21.15.116:0/3452782659 shutdown_connections 2024-09-12T03:04:50.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.780+0000 7fa39899e740 1 -- 172.21.15.116:0/3452782659 wait complete. 2024-09-12T03:04:50.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.780+0000 7fa39899e740 20 client.15714 trim_cache size 0 max 16384 2024-09-12T03:04:50.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.797+0000 7fa39899e740 1 Processor -- start 2024-09-12T03:04:50.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.797+0000 7fa39899e740 1 -- start start 2024-09-12T03:04:50.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.797+0000 7fa39899e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e62a033a40 0x55e62a02f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.797+0000 7fa39899e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e62a034360 0x55e62a05a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.797+0000 7fa39899e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e62a043150 0x55e62a05ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.797+0000 7fa39899e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa3901adb40 con 0x55e62a033a40 2024-09-12T03:04:50.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.797+0000 7fa39899e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa3901ab410 con 0x55e62a043150 2024-09-12T03:04:50.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.797+0000 7fa39899e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa3901a9260 con 0x55e62a034360 2024-09-12T03:04:50.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.797+0000 7fa38ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e62a033a40 0x55e62a02f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.797+0000 7fa38ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e62a033a40 0x55e62a02f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56720/0 (socket says 172.21.15.116:56720) 2024-09-12T03:04:50.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.797+0000 7fa38ffff640 1 -- 172.21.15.116:0/1175096921 learned_addr learned my addr 172.21.15.116:0/1175096921 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:50.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.798+0000 7fa38ffff640 1 -- 172.21.15.116:0/1175096921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e62a034360 msgr2=0x55e62a05a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:50.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.798+0000 7fa38ffff640 1 --2- 172.21.15.116:0/1175096921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e62a034360 0x55e62a05a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.798+0000 7fa38ffff640 1 -- 172.21.15.116:0/1175096921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e62a043150 msgr2=0x55e62a05ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:50.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.798+0000 7fa38ffff640 1 --2- 172.21.15.116:0/1175096921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e62a043150 0x55e62a05ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.798+0000 7fa38ffff640 1 -- 172.21.15.116:0/1175096921 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e62a03f040 con 0x55e62a033a40 2024-09-12T03:04:50.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.798+0000 7fa38ffff640 1 --2- 172.21.15.116:0/1175096921 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e62a033a40 0x55e62a02f160 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa3800b89f0 tx=0x7fa38003b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce385ae572d11adc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:50.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.798+0000 7fa375ffb640 1 -- 172.21.15.116:0/1175096921 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa380049030 con 0x55e62a033a40 2024-09-12T03:04:50.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.798+0000 7fa375ffb640 1 -- 172.21.15.116:0/1175096921 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3800c03d0 con 0x55e62a033a40 2024-09-12T03:04:50.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.798+0000 7fa375ffb640 1 -- 172.21.15.116:0/1175096921 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3800bc280 con 0x55e62a033a40 2024-09-12T03:04:50.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.799+0000 7fa39899e740 1 -- 172.21.15.116:0/1175096921 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e62a033a40 msgr2=0x55e62a02f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:50.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.799+0000 7fa39899e740 1 --2- 172.21.15.116:0/1175096921 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e62a033a40 0x55e62a02f160 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa3800b89f0 tx=0x7fa38003b4a0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.799+0000 7fa39899e740 1 -- 172.21.15.116:0/1175096921 shutdown_connections 2024-09-12T03:04:50.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.799+0000 7fa39899e740 1 --2- 172.21.15.116:0/1175096921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e62a043150 0x55e62a05ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.799+0000 7fa39899e740 1 --2- 172.21.15.116:0/1175096921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e62a034360 0x55e62a05a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.799+0000 7fa39899e740 1 --2- 172.21.15.116:0/1175096921 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e62a033a40 0x55e62a02f160 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.799+0000 7fa39899e740 1 -- 172.21.15.116:0/1175096921 >> 172.21.15.116:0/1175096921 conn(0x55e62a0300f0 msgr2=0x55e62a02ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:50.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.799+0000 7fa39899e740 1 -- 172.21.15.116:0/1175096921 shutdown_connections 2024-09-12T03:04:50.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.799+0000 7fa39899e740 1 -- 172.21.15.116:0/1175096921 wait complete. 2024-09-12T03:04:50.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.799+0000 7fa39899e740 1 -- start start 2024-09-12T03:04:50.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.799+0000 7fa39899e740 1 Processor -- start 2024-09-12T03:04:50.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.799+0000 7fa39899e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e62a033a40 0x55e62a046b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.799+0000 7fa39899e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e62a034360 0x55e62a047080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.799+0000 7fa39899e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e62a043150 0x55e62a0475c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.799+0000 7fa39899e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa3901a98d0 con 0x55e62a034360 2024-09-12T03:04:50.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.799+0000 7fa39899e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa3901aad40 con 0x55e62a043150 2024-09-12T03:04:50.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.799+0000 7fa39899e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa3901a9fa0 con 0x55e62a033a40 2024-09-12T03:04:50.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.799+0000 7fa39899e740 10 client.0 _ll_register_callbacks cb 0x7fa3880126d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:50.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.799+0000 7fa38f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e62a034360 0x55e62a047080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.799+0000 7fa38f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e62a034360 0x55e62a047080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56722/0 (socket says 172.21.15.116:56722) 2024-09-12T03:04:50.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.800+0000 7fa38f7fe640 1 -- 172.21.15.116:0/2820417522 learned_addr learned my addr 172.21.15.116:0/2820417522 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:50.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.800+0000 7fa394b62640 1 --2- 172.21.15.116:0/2820417522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e62a043150 0x55e62a0475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.800+0000 7fa38f7fe640 1 -- 172.21.15.116:0/2820417522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e62a033a40 msgr2=0x55e62a046b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:50.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.800+0000 7fa38f7fe640 1 --2- 172.21.15.116:0/2820417522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e62a033a40 0x55e62a046b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.800+0000 7fa38f7fe640 1 -- 172.21.15.116:0/2820417522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e62a043150 msgr2=0x55e62a0475c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:50.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.800+0000 7fa38f7fe640 1 --2- 172.21.15.116:0/2820417522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e62a043150 0x55e62a0475c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.800+0000 7fa38f7fe640 1 -- 172.21.15.116:0/2820417522 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e62a0bc690 con 0x55e62a034360 2024-09-12T03:04:50.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.800+0000 7fa38f7fe640 1 --2- 172.21.15.116:0/2820417522 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e62a034360 0x55e62a047080 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa384005fd0 tx=0x7fa384012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda1b83b120906c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:50.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.800+0000 7fa36f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:04:50.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.800+0000 7fa36f7fe640 1 -- 172.21.15.116:0/2820417522 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38401e070 con 0x55e62a034360 2024-09-12T03:04:50.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.800+0000 7fa36f7fe640 1 -- 172.21.15.116:0/2820417522 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa38401a6d0 con 0x55e62a034360 2024-09-12T03:04:50.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.800+0000 7fa39899e740 1 -- 172.21.15.116:0/2820417522 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3800c03d0 con 0x55e62a034360 2024-09-12T03:04:50.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.800+0000 7fa36f7fe640 1 -- 172.21.15.116:0/2820417522 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa38402a050 con 0x55e62a034360 2024-09-12T03:04:50.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.800+0000 7fa39899e740 10 client.15720 fetch_fsmap learned FSMap version 25 2024-09-12T03:04:50.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.800+0000 7fa39899e740 1 -- 172.21.15.116:0/2820417522 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e62a0bda20 con 0x55e62a034360 2024-09-12T03:04:50.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.800+0000 7fa36f7fe640 1 -- 172.21.15.116:0/2820417522 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3800c03d0 con 0x55e62a034360 2024-09-12T03:04:50.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.800+0000 7fa36f7fe640 1 -- 172.21.15.116:0/2820417522 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa38400f040 con 0x55e62a034360 2024-09-12T03:04:50.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.801+0000 7fa39899e740 10 client.15720 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:04:50.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.801+0000 7fa39899e740 10 client.15720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:04:50.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.801+0000 7fa39899e740 20 client.15720 populate_metadata read hostname 'smithi116' 2024-09-12T03:04:50.823 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:04:50.823 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:04:50.823 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "c7ccc8b5-0ea8-4065-a472-9869b1e833ae", 2024-09-12T03:04:50.823 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:04:50.823 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:04:50.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.801+0000 7fa39899e740 1 --2- 172.21.15.116:0/2820417522 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e62a0be1d0 0x55e62a0c05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:50.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.801+0000 7fa39899e740 4 client.15720 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:04:50.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.801+0000 7fa39899e740 1 -- 172.21.15.116:0/2820417522 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e629f1d110 con 0x55e62a0be1d0 2024-09-12T03:04:50.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.801+0000 7fa38ffff640 1 --2- 172.21.15.116:0/2820417522 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e62a0be1d0 0x55e62a0c05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:50.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.801+0000 7fa36f7fe640 10 client.15720 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:50.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.801+0000 7fa38ffff640 1 --2- 172.21.15.116:0/2820417522 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e62a0be1d0 0x55e62a0c05f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f44e4dfb308e09ef server_cookie=1afd6a417aa098c5 in_seq=0 out_seq=0 2024-09-12T03:04:50.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.802+0000 7fa36f7fe640 1 -- 172.21.15.116:0/2820417522 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e629f1d110 con 0x55e62a0be1d0 2024-09-12T03:04:50.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.802+0000 7fa36f7fe640 10 client.15720 handle_command_reply: tid=0 2024-09-12T03:04:50.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.802+0000 7fa39899e740 1 client.15720 shutdown 2024-09-12T03:04:50.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.802+0000 7fa39899e740 1 -- 172.21.15.116:0/2820417522 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e62a034360 msgr2=0x55e62a047080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:50.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.802+0000 7fa39899e740 1 --2- 172.21.15.116:0/2820417522 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e62a034360 0x55e62a047080 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa384005fd0 tx=0x7fa384012820 comp rx=0 tx=0).stop 2024-09-12T03:04:50.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.802+0000 7fa39899e740 1 -- 172.21.15.116:0/2820417522 shutdown_connections 2024-09-12T03:04:50.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.802+0000 7fa39899e740 1 --2- 172.21.15.116:0/2820417522 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e62a0be1d0 0x55e62a0c05f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.802+0000 7fa39899e740 1 --2- 172.21.15.116:0/2820417522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e62a043150 0x55e62a0475c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.802+0000 7fa39899e740 1 --2- 172.21.15.116:0/2820417522 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e62a034360 0x55e62a047080 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.802+0000 7fa39899e740 1 --2- 172.21.15.116:0/2820417522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e62a033a40 0x55e62a046b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.802+0000 7fa39899e740 1 -- 172.21.15.116:0/2820417522 >> 172.21.15.116:0/2820417522 conn(0x55e62a0300f0 msgr2=0x55e62a040800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:50.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.802+0000 7fa36f7fe640 0 client.15720 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:50.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.803+0000 7fa39899e740 1 -- 172.21.15.116:0/2820417522 shutdown_connections 2024-09-12T03:04:50.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.803+0000 7fa39899e740 1 -- 172.21.15.116:0/2820417522 wait complete. 2024-09-12T03:04:50.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.803+0000 7fa39899e740 20 client.15720 trim_cache size 0 max 16384 2024-09-12T03:04:50.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.803+0000 7fa3965c5640 1 -- 172.21.15.116:0/1415102926 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa3780a2b80 msgr2=0x7fa3780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:50.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.803+0000 7fa3965c5640 1 --2- 172.21.15.116:0/1415102926 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa3780a2b80 0x7fa3780a5040 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa3880098b0 tx=0x7fa388009d90 comp rx=0 tx=0).stop 2024-09-12T03:04:50.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.803+0000 7fa3965c5640 1 -- 172.21.15.116:0/1415102926 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa390101de0 msgr2=0x7fa390198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:50.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.803+0000 7fa3965c5640 1 --2- 172.21.15.116:0/1415102926 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa390101de0 0x7fa390198510 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa380006010 tx=0x7fa3800042f0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.803+0000 7fa3965c5640 1 -- 172.21.15.116:0/1415102926 shutdown_connections 2024-09-12T03:04:50.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.803+0000 7fa3965c5640 1 --2- 172.21.15.116:0/1415102926 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa3780a2b80 0x7fa3780a5040 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.803+0000 7fa3965c5640 1 --2- 172.21.15.116:0/1415102926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa390106f10 0x7fa39019cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.803+0000 7fa3965c5640 1 --2- 172.21.15.116:0/1415102926 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa390102790 0x7fa390198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.803+0000 7fa3965c5640 1 --2- 172.21.15.116:0/1415102926 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa390101de0 0x7fa390198510 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.803+0000 7fa3965c5640 1 -- 172.21.15.116:0/1415102926 >> 172.21.15.116:0/1415102926 conn(0x7fa3900fd8c0 msgr2=0x7fa3900fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:50.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.803+0000 7fa3965c5640 1 -- 172.21.15.116:0/1415102926 shutdown_connections 2024-09-12T03:04:50.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:50.803+0000 7fa3965c5640 1 -- 172.21.15.116:0/1415102926 wait complete. 2024-09-12T03:04:51.541 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:04:51.786 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:52.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.271+0000 7f5b60178640 1 -- 172.21.15.116:0/2124229944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b58101430 msgr2=0x7f5b5810d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:52.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.271+0000 7f5b60178640 1 --2- 172.21.15.116:0/2124229944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b58101430 0x7f5b5810d960 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5b50009cd0 tx=0x7f5b5002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.271+0000 7f5b60178640 1 -- 172.21.15.116:0/2124229944 shutdown_connections 2024-09-12T03:04:52.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.271+0000 7f5b60178640 1 --2- 172.21.15.116:0/2124229944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b58101430 0x7f5b5810d960 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.271+0000 7f5b60178640 1 --2- 172.21.15.116:0/2124229944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b58100a90 0x7f5b58100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.271+0000 7f5b60178640 1 --2- 172.21.15.116:0/2124229944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b58106a90 0x7f5b58106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.271+0000 7f5b60178640 1 -- 172.21.15.116:0/2124229944 >> 172.21.15.116:0/2124229944 conn(0x7f5b580fc7d0 msgr2=0x7f5b580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:52.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.271+0000 7f5b60178640 1 -- 172.21.15.116:0/2124229944 shutdown_connections 2024-09-12T03:04:52.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.271+0000 7f5b60178640 1 -- 172.21.15.116:0/2124229944 wait complete. 2024-09-12T03:04:52.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.272+0000 7f5b60178640 1 Processor -- start 2024-09-12T03:04:52.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.272+0000 7f5b60178640 1 -- start start 2024-09-12T03:04:52.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.272+0000 7f5b60178640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b58100a90 0x7f5b58197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:52.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.272+0000 7f5b60178640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b58101430 0x7f5b58197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:52.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.272+0000 7f5b60178640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b58106a90 0x7f5b5819bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:52.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.272+0000 7f5b60178640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5b58110e90 con 0x7f5b58106a90 2024-09-12T03:04:52.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.272+0000 7f5b60178640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5b58110d10 con 0x7f5b58100a90 2024-09-12T03:04:52.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.272+0000 7f5b60178640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5b58111010 con 0x7f5b58101430 2024-09-12T03:04:52.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.272+0000 7f5b5e715640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b58106a90 0x7f5b5819bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:52.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.272+0000 7f5b5e715640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b58106a90 0x7f5b5819bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46652/0 (socket says 172.21.15.116:46652) 2024-09-12T03:04:52.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.272+0000 7f5b5e715640 1 -- 172.21.15.116:0/1495516575 learned_addr learned my addr 172.21.15.116:0/1495516575 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:52.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.273+0000 7f5b5d713640 1 --2- 172.21.15.116:0/1495516575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b58101430 0x7f5b58197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:52.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.273+0000 7f5b5df14640 1 --2- 172.21.15.116:0/1495516575 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b58100a90 0x7f5b58197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:52.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.273+0000 7f5b5e715640 1 -- 172.21.15.116:0/1495516575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b58101430 msgr2=0x7f5b58197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:52.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.273+0000 7f5b5e715640 1 --2- 172.21.15.116:0/1495516575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b58101430 0x7f5b58197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.273+0000 7f5b5e715640 1 -- 172.21.15.116:0/1495516575 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b58100a90 msgr2=0x7f5b58197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:52.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.273+0000 7f5b5e715640 1 --2- 172.21.15.116:0/1495516575 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b58100a90 0x7f5b58197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.273+0000 7f5b5e715640 1 -- 172.21.15.116:0/1495516575 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5819c2b0 con 0x7f5b58106a90 2024-09-12T03:04:52.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.273+0000 7f5b5df14640 1 --2- 172.21.15.116:0/1495516575 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b58100a90 0x7f5b58197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:52.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.273+0000 7f5b5e715640 1 --2- 172.21.15.116:0/1495516575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b58106a90 0x7f5b5819bbd0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5b50005cc0 tx=0x7f5b50032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:52.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.273+0000 7f5b4effd640 1 -- 172.21.15.116:0/1495516575 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b50004040 con 0x7f5b58106a90 2024-09-12T03:04:52.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.273+0000 7f5b4effd640 1 -- 172.21.15.116:0/1495516575 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b500046c0 con 0x7f5b58106a90 2024-09-12T03:04:52.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.273+0000 7f5b4effd640 1 -- 172.21.15.116:0/1495516575 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b50030e00 con 0x7f5b58106a90 2024-09-12T03:04:52.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.273+0000 7f5b60178640 1 -- 172.21.15.116:0/1495516575 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b5819c540 con 0x7f5b58106a90 2024-09-12T03:04:52.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.273+0000 7f5b60178640 1 -- 172.21.15.116:0/1495516575 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b581a3e80 con 0x7f5b58106a90 2024-09-12T03:04:52.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.275+0000 7f5b4effd640 1 -- 172.21.15.116:0/1495516575 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f5b500305a0 con 0x7f5b58106a90 2024-09-12T03:04:52.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.276+0000 7f5b62551740 1 Processor -- start 2024-09-12T03:04:52.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.276+0000 7f5b62551740 1 -- start start 2024-09-12T03:04:52.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.276+0000 7f5b62551740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d573f8e320 0x55d573f8e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:52.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.276+0000 7f5b62551740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d573f8ec40 0x55d573f9b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:52.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.276+0000 7f5b62551740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d573f9b890 0x55d573f9dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:52.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.276+0000 7f5b4effd640 1 --2- 172.21.15.116:0/1495516575 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5b440a2c00 0x7f5b440a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:52.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.276+0000 7f5b62551740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d573ef42e0 con 0x55d573f8e320 2024-09-12T03:04:52.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.276+0000 7f5b62551740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d573e9bce0 con 0x55d573f8ec40 2024-09-12T03:04:52.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.276+0000 7f5b62551740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d573ef02a0 con 0x55d573f9b890 2024-09-12T03:04:52.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.276+0000 7f5b5d713640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d573f8e320 0x55d573f8e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:52.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.276+0000 7f5b5e715640 1 --2- 172.21.15.116:0/1495516575 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5b440a2c00 0x7f5b440a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:52.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.276+0000 7f5b5d713640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d573f8e320 0x55d573f8e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46664/0 (socket says 172.21.15.116:46664) 2024-09-12T03:04:52.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.276+0000 7f5b5d713640 1 -- 172.21.15.116:0/4294862826 learned_addr learned my addr 172.21.15.116:0/4294862826 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:52.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.276+0000 7f5b4effd640 1 -- 172.21.15.116:0/1495516575 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f5b5002e030 con 0x7f5b58106a90 2024-09-12T03:04:52.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.277+0000 7f5b5df14640 1 --2- 172.21.15.116:0/4294862826 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d573f8ec40 0x55d573f9b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:52.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.277+0000 7f5b5d713640 1 -- 172.21.15.116:0/4294862826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d573f9b890 msgr2=0x55d573f9dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:52.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.277+0000 7f5b5d713640 1 --2- 172.21.15.116:0/4294862826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d573f9b890 0x55d573f9dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.277+0000 7f5b5d713640 1 -- 172.21.15.116:0/4294862826 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d573f8ec40 msgr2=0x55d573f9b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:52.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.277+0000 7f5b5d713640 1 --2- 172.21.15.116:0/4294862826 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d573f8ec40 0x55d573f9b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.277+0000 7f5b5d713640 1 -- 172.21.15.116:0/4294862826 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d573e80440 con 0x55d573f8e320 2024-09-12T03:04:52.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.277+0000 7f5b5e715640 1 --2- 172.21.15.116:0/1495516575 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5b440a2c00 0x7f5b440a50c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5b500c5780 tx=0x7f5b500b2520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:52.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.277+0000 7f5b5d713640 1 --2- 172.21.15.116:0/4294862826 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d573f8e320 0x55d573f8e700 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800bd40 tx=0x7f5b480127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d45de86f737594 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:52.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.277+0000 7f5b43fff640 1 -- 172.21.15.116:0/4294862826 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b480120d0 con 0x55d573f8e320 2024-09-12T03:04:52.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.277+0000 7f5b43fff640 1 -- 172.21.15.116:0/4294862826 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b48016070 con 0x55d573f8e320 2024-09-12T03:04:52.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.277+0000 7f5b43fff640 1 -- 172.21.15.116:0/4294862826 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b480106d0 con 0x55d573f8e320 2024-09-12T03:04:52.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.278+0000 7f5b62551740 1 -- 172.21.15.116:0/4294862826 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d573f8e320 msgr2=0x55d573f8e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:52.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.278+0000 7f5b62551740 1 --2- 172.21.15.116:0/4294862826 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d573f8e320 0x55d573f8e700 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800bd40 tx=0x7f5b480127c0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.278+0000 7f5b62551740 1 -- 172.21.15.116:0/4294862826 shutdown_connections 2024-09-12T03:04:52.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.278+0000 7f5b62551740 1 --2- 172.21.15.116:0/4294862826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d573f9b890 0x55d573f9dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.278+0000 7f5b62551740 1 --2- 172.21.15.116:0/4294862826 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d573f8ec40 0x55d573f9b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.278+0000 7f5b62551740 1 --2- 172.21.15.116:0/4294862826 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d573f8e320 0x55d573f8e700 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.278+0000 7f5b62551740 1 -- 172.21.15.116:0/4294862826 >> 172.21.15.116:0/4294862826 conn(0x55d573f8aa30 msgr2=0x55d573f8ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:52.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.278+0000 7f5b62551740 1 -- 172.21.15.116:0/4294862826 shutdown_connections 2024-09-12T03:04:52.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.278+0000 7f5b62551740 1 -- 172.21.15.116:0/4294862826 wait complete. 2024-09-12T03:04:52.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.278+0000 7f5b62551740 1 -- start start 2024-09-12T03:04:52.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.278+0000 7f5b62551740 1 Processor -- start 2024-09-12T03:04:52.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.279+0000 7f5b62551740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d573f8e320 0x55d573f9b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:52.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.279+0000 7f5b62551740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d573f8ec40 0x55d573f98480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:52.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.279+0000 7f5b62551740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d573f9b890 0x55d573f989c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:52.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.279+0000 7f5b62551740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d573f9e4c0 con 0x55d573f8ec40 2024-09-12T03:04:52.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.279+0000 7f5b62551740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d573eaf050 con 0x55d573f9b890 2024-09-12T03:04:52.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.279+0000 7f5b62551740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d573f9e640 con 0x55d573f8e320 2024-09-12T03:04:52.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.279+0000 7f5b62551740 10 client.0 _ll_register_callbacks cb 0x55d573eef160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:52.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.279+0000 7f5b5df14640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d573f8ec40 0x55d573f98480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:52.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.279+0000 7f5b5df14640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d573f8ec40 0x55d573f98480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46674/0 (socket says 172.21.15.116:46674) 2024-09-12T03:04:52.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.279+0000 7f5b5df14640 1 -- 172.21.15.116:0/287684804 learned_addr learned my addr 172.21.15.116:0/287684804 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:52.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.279+0000 7f5b5d713640 1 --2- 172.21.15.116:0/287684804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d573f8e320 0x55d573f9b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:52.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.279+0000 7f5b5d713640 1 --2- 172.21.15.116:0/287684804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d573f9b890 0x55d573f989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:52.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.279+0000 7f5b5df14640 1 -- 172.21.15.116:0/287684804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d573f8e320 msgr2=0x55d573f9b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:52.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.279+0000 7f5b5df14640 1 --2- 172.21.15.116:0/287684804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d573f8e320 0x55d573f9b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.279+0000 7f5b5df14640 1 -- 172.21.15.116:0/287684804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d573f9b890 msgr2=0x55d573f989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:52.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.279+0000 7f5b5df14640 1 --2- 172.21.15.116:0/287684804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d573f9b890 0x55d573f989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.279+0000 7f5b5df14640 1 -- 172.21.15.116:0/287684804 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d573c653c0 con 0x55d573f8ec40 2024-09-12T03:04:52.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.279+0000 7f5b5d713640 1 --2- 172.21.15.116:0/287684804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d573f9b890 0x55d573f989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:52.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.279+0000 7f5b5df14640 1 --2- 172.21.15.116:0/287684804 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d573f8ec40 0x55d573f98480 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5b54013790 tx=0x7f5b54013c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a033b41f44209f7a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:52.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.279+0000 7f5b417fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:04:52.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.280+0000 7f5b417fa640 1 -- 172.21.15.116:0/287684804 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b54011dd0 con 0x55d573f8ec40 2024-09-12T03:04:52.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.280+0000 7f5b62551740 1 -- 172.21.15.116:0/287684804 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b48016070 con 0x55d573f8ec40 2024-09-12T03:04:52.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.280+0000 7f5b417fa640 1 -- 172.21.15.116:0/287684804 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f5b54004020 con 0x55d573f8ec40 2024-09-12T03:04:52.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.280+0000 7f5b417fa640 1 -- 172.21.15.116:0/287684804 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5b54004220 con 0x55d573f8ec40 2024-09-12T03:04:52.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.280+0000 7f5b62551740 10 client.15732 fetch_fsmap learned FSMap version 25 2024-09-12T03:04:52.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.280+0000 7f5b62551740 1 -- 172.21.15.116:0/287684804 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d573fa9f60 con 0x55d573f8ec40 2024-09-12T03:04:52.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.280+0000 7f5b417fa640 1 -- 172.21.15.116:0/287684804 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b48016070 con 0x55d573f8ec40 2024-09-12T03:04:52.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.280+0000 7f5b417fa640 1 -- 172.21.15.116:0/287684804 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5b5401a030 con 0x55d573f8ec40 2024-09-12T03:04:52.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.280+0000 7f5b62551740 10 client.15732 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:04:52.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.280+0000 7f5b62551740 10 client.15732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:04:52.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.280+0000 7f5b62551740 20 client.15732 populate_metadata read hostname 'smithi116' 2024-09-12T03:04:52.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.280+0000 7f5b62551740 1 --2- 172.21.15.116:0/287684804 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d573fab3c0 0x55d573fad7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:52.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.280+0000 7f5b62551740 4 client.15732 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:04:52.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.280+0000 7f5b62551740 1 -- 172.21.15.116:0/287684804 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d573f01850 con 0x55d573fab3c0 2024-09-12T03:04:52.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.281+0000 7f5b5d713640 1 --2- 172.21.15.116:0/287684804 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d573fab3c0 0x55d573fad7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:52.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.281+0000 7f5b417fa640 10 client.15732 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:52.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.281+0000 7f5b5d713640 1 --2- 172.21.15.116:0/287684804 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d573fab3c0 0x55d573fad7e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=245437cc98e66f2d server_cookie=2dc63462a81a3e39 in_seq=0 out_seq=0 2024-09-12T03:04:52.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.282+0000 7f5b417fa640 1 -- 172.21.15.116:0/287684804 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d573f01850 con 0x55d573fab3c0 2024-09-12T03:04:52.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.282+0000 7f5b417fa640 10 client.15732 handle_command_reply: tid=0 2024-09-12T03:04:52.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.282+0000 7f5b62551740 1 client.15732 shutdown 2024-09-12T03:04:52.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.282+0000 7f5b62551740 1 -- 172.21.15.116:0/287684804 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d573f8ec40 msgr2=0x55d573f98480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:52.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.282+0000 7f5b62551740 1 --2- 172.21.15.116:0/287684804 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d573f8ec40 0x55d573f98480 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5b54013790 tx=0x7f5b54013c50 comp rx=0 tx=0).stop 2024-09-12T03:04:52.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.282+0000 7f5b62551740 1 -- 172.21.15.116:0/287684804 shutdown_connections 2024-09-12T03:04:52.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.282+0000 7f5b62551740 1 --2- 172.21.15.116:0/287684804 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d573fab3c0 0x55d573fad7e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.282+0000 7f5b62551740 1 --2- 172.21.15.116:0/287684804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d573f9b890 0x55d573f989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.282+0000 7f5b62551740 1 --2- 172.21.15.116:0/287684804 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d573f8ec40 0x55d573f98480 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.282+0000 7f5b417fa640 0 client.15732 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:52.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.282+0000 7f5b62551740 1 --2- 172.21.15.116:0/287684804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d573f8e320 0x55d573f9b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.282+0000 7f5b62551740 1 -- 172.21.15.116:0/287684804 >> 172.21.15.116:0/287684804 conn(0x55d573f8aa30 msgr2=0x55d573f906f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:52.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.282+0000 7f5b62551740 1 -- 172.21.15.116:0/287684804 shutdown_connections 2024-09-12T03:04:52.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.282+0000 7f5b62551740 1 -- 172.21.15.116:0/287684804 wait complete. 2024-09-12T03:04:52.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.282+0000 7f5b62551740 20 client.15732 trim_cache size 0 max 16384 2024-09-12T03:04:52.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.299+0000 7f5b62551740 1 Processor -- start 2024-09-12T03:04:52.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.299+0000 7f5b62551740 1 -- start start 2024-09-12T03:04:52.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.299+0000 7f5b62551740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d573f8e320 0x55d573fa7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:52.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.299+0000 7f5b62551740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d573f8ec40 0x55d573fb3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:52.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.299+0000 7f5b62551740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d573f9b890 0x55d573fb35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:52.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.299+0000 7f5b62551740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5b581adab0 con 0x55d573f9b890 2024-09-12T03:04:52.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.299+0000 7f5b62551740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5b581ab380 con 0x55d573f8ec40 2024-09-12T03:04:52.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.299+0000 7f5b62551740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5b581a91f0 con 0x55d573f8e320 2024-09-12T03:04:52.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.299+0000 7f5b5d713640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d573f9b890 0x55d573fb35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:52.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.299+0000 7f5b5d713640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d573f9b890 0x55d573fb35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46676/0 (socket says 172.21.15.116:46676) 2024-09-12T03:04:52.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.299+0000 7f5b5d713640 1 -- 172.21.15.116:0/513266586 learned_addr learned my addr 172.21.15.116:0/513266586 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:52.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.299+0000 7f5b5d713640 1 --2- 172.21.15.116:0/513266586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d573f8e320 0x55d573fa7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:52.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.299+0000 7f5b5df14640 1 --2- 172.21.15.116:0/513266586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d573f8ec40 0x55d573fb3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:52.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.299+0000 7f5b5d713640 1 -- 172.21.15.116:0/513266586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d573f8e320 msgr2=0x55d573fa7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:52.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.299+0000 7f5b5d713640 1 --2- 172.21.15.116:0/513266586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d573f8e320 0x55d573fa7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.299+0000 7f5b5d713640 1 -- 172.21.15.116:0/513266586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d573f8ec40 msgr2=0x55d573fb3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:52.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.299+0000 7f5b5d713640 1 --2- 172.21.15.116:0/513266586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d573f8ec40 0x55d573fb3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.299+0000 7f5b5d713640 1 -- 172.21.15.116:0/513266586 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d573f994c0 con 0x55d573f9b890 2024-09-12T03:04:52.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.299+0000 7f5b5df14640 1 --2- 172.21.15.116:0/513266586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d573f8ec40 0x55d573fb3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:52.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.300+0000 7f5b5d713640 1 --2- 172.21.15.116:0/513266586 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d573f9b890 0x55d573fb35d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800b6c0 tx=0x7f5b480127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823acd253bfb8cd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:52.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.300+0000 7f5b437fe640 1 -- 172.21.15.116:0/513266586 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48005480 con 0x55d573f9b890 2024-09-12T03:04:52.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.300+0000 7f5b437fe640 1 -- 172.21.15.116:0/513266586 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b48016070 con 0x55d573f9b890 2024-09-12T03:04:52.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.300+0000 7f5b437fe640 1 -- 172.21.15.116:0/513266586 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48004a80 con 0x55d573f9b890 2024-09-12T03:04:52.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.300+0000 7f5b62551740 1 -- 172.21.15.116:0/513266586 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d573f9b890 msgr2=0x55d573fb35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:52.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.300+0000 7f5b62551740 1 --2- 172.21.15.116:0/513266586 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d573f9b890 0x55d573fb35d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800b6c0 tx=0x7f5b480127c0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.300+0000 7f5b62551740 1 -- 172.21.15.116:0/513266586 shutdown_connections 2024-09-12T03:04:52.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.300+0000 7f5b62551740 1 --2- 172.21.15.116:0/513266586 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d573f9b890 0x55d573fb35d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.300+0000 7f5b62551740 1 --2- 172.21.15.116:0/513266586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d573f8ec40 0x55d573fb3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.300+0000 7f5b62551740 1 --2- 172.21.15.116:0/513266586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d573f8e320 0x55d573fa7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.300+0000 7f5b62551740 1 -- 172.21.15.116:0/513266586 >> 172.21.15.116:0/513266586 conn(0x55d573f8aa30 msgr2=0x55d573f9a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:52.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.301+0000 7f5b62551740 1 -- 172.21.15.116:0/513266586 shutdown_connections 2024-09-12T03:04:52.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.301+0000 7f5b62551740 1 -- 172.21.15.116:0/513266586 wait complete. 2024-09-12T03:04:52.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.301+0000 7f5b62551740 1 -- start start 2024-09-12T03:04:52.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.301+0000 7f5b62551740 1 Processor -- start 2024-09-12T03:04:52.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.301+0000 7f5b62551740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d573f8e320 0x55d573f9fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:52.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.301+0000 7f5b62551740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d573f8ec40 0x55d573f9ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:52.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.301+0000 7f5b62551740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d573f9b890 0x55d574015060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:52.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.301+0000 7f5b62551740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5b581a9840 con 0x55d573f9b890 2024-09-12T03:04:52.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.301+0000 7f5b62551740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5b581aacb0 con 0x55d573f8e320 2024-09-12T03:04:52.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.301+0000 7f5b62551740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5b581a9f10 con 0x55d573f8ec40 2024-09-12T03:04:52.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.301+0000 7f5b62551740 10 client.0 _ll_register_callbacks cb 0x7f5b54009690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:52.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.302+0000 7f5b5d713640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d573f9b890 0x55d574015060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:52.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.302+0000 7f5b5d713640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d573f9b890 0x55d574015060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46690/0 (socket says 172.21.15.116:46690) 2024-09-12T03:04:52.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.302+0000 7f5b5d713640 1 -- 172.21.15.116:0/3238405818 learned_addr learned my addr 172.21.15.116:0/3238405818 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:52.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.302+0000 7f5b5df14640 1 --2- 172.21.15.116:0/3238405818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d573f8ec40 0x55d573f9ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:52.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.302+0000 7f5b5d713640 1 --2- 172.21.15.116:0/3238405818 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d573f8e320 0x55d573f9fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:52.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.302+0000 7f5b5d713640 1 -- 172.21.15.116:0/3238405818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d573f8ec40 msgr2=0x55d573f9ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:52.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.302+0000 7f5b5d713640 1 --2- 172.21.15.116:0/3238405818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d573f8ec40 0x55d573f9ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.302+0000 7f5b5d713640 1 -- 172.21.15.116:0/3238405818 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d573f8e320 msgr2=0x55d573f9fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:52.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.302+0000 7f5b5d713640 1 --2- 172.21.15.116:0/3238405818 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d573f8e320 0x55d573f9fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.302+0000 7f5b5d713640 1 -- 172.21.15.116:0/3238405818 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5740155a0 con 0x55d573f9b890 2024-09-12T03:04:52.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.302+0000 7f5b5d713640 1 --2- 172.21.15.116:0/3238405818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d573f9b890 0x55d574015060 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5b48004760 tx=0x7f5b48014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9382ee9fa3f6fb75 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:52.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.302+0000 7f5b417fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:04:52.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.302+0000 7f5b417fa640 1 -- 172.21.15.116:0/3238405818 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4803f060 con 0x55d573f9b890 2024-09-12T03:04:52.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.302+0000 7f5b417fa640 1 -- 172.21.15.116:0/3238405818 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f5b48004800 con 0x55d573f9b890 2024-09-12T03:04:52.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.302+0000 7f5b62551740 1 -- 172.21.15.116:0/3238405818 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b48016070 con 0x55d573f9b890 2024-09-12T03:04:52.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.302+0000 7f5b417fa640 1 -- 172.21.15.116:0/3238405818 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5b48004020 con 0x55d573f9b890 2024-09-12T03:04:52.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.302+0000 7f5b62551740 10 client.15738 fetch_fsmap learned FSMap version 25 2024-09-12T03:04:52.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.302+0000 7f5b62551740 1 -- 172.21.15.116:0/3238405818 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d574016730 con 0x55d573f9b890 2024-09-12T03:04:52.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.302+0000 7f5b417fa640 1 -- 172.21.15.116:0/3238405818 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b48016070 con 0x55d573f9b890 2024-09-12T03:04:52.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.303+0000 7f5b417fa640 1 -- 172.21.15.116:0/3238405818 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5b48003860 con 0x55d573f9b890 2024-09-12T03:04:52.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.303+0000 7f5b62551740 10 client.15738 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:04:52.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.303+0000 7f5b62551740 10 client.15738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:04:52.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.303+0000 7f5b62551740 20 client.15738 populate_metadata read hostname 'smithi116' 2024-09-12T03:04:52.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.303+0000 7f5b62551740 1 --2- 172.21.15.116:0/3238405818 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d574016c00 0x55d574018ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:52.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.303+0000 7f5b62551740 4 client.15738 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:04:52.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.303+0000 7f5b62551740 1 -- 172.21.15.116:0/3238405818 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d573efbd60 con 0x55d574016c00 2024-09-12T03:04:52.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.303+0000 7f5b5df14640 1 --2- 172.21.15.116:0/3238405818 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d574016c00 0x55d574018ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:52.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.303+0000 7f5b5df14640 1 --2- 172.21.15.116:0/3238405818 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d574016c00 0x55d574018ff0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfb06cf7f0c5dda0 server_cookie=1dec18d076f23e1d in_seq=0 out_seq=0 2024-09-12T03:04:52.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.303+0000 7f5b417fa640 10 client.15738 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:52.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.303+0000 7f5b417fa640 1 -- 172.21.15.116:0/3238405818 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d573efbd60 con 0x55d574016c00 2024-09-12T03:04:52.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.303+0000 7f5b417fa640 10 client.15738 handle_command_reply: tid=0 2024-09-12T03:04:52.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.303+0000 7f5b62551740 1 client.15738 shutdown 2024-09-12T03:04:52.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.304+0000 7f5b62551740 1 -- 172.21.15.116:0/3238405818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d573f9b890 msgr2=0x55d574015060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:52.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.304+0000 7f5b62551740 1 --2- 172.21.15.116:0/3238405818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d573f9b890 0x55d574015060 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5b48004760 tx=0x7f5b48014040 comp rx=0 tx=0).stop 2024-09-12T03:04:52.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.304+0000 7f5b62551740 1 -- 172.21.15.116:0/3238405818 shutdown_connections 2024-09-12T03:04:52.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.304+0000 7f5b62551740 1 --2- 172.21.15.116:0/3238405818 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d574016c00 0x55d574018ff0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.304+0000 7f5b62551740 1 --2- 172.21.15.116:0/3238405818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d573f9b890 0x55d574015060 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.304+0000 7f5b62551740 1 --2- 172.21.15.116:0/3238405818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d573f8ec40 0x55d573f9ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.304+0000 7f5b62551740 1 --2- 172.21.15.116:0/3238405818 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d573f8e320 0x55d573f9fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.304+0000 7f5b417fa640 0 client.15738 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:52.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.304+0000 7f5b62551740 1 -- 172.21.15.116:0/3238405818 >> 172.21.15.116:0/3238405818 conn(0x55d573f8aa30 msgr2=0x55d573f89350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:52.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.304+0000 7f5b62551740 1 -- 172.21.15.116:0/3238405818 shutdown_connections 2024-09-12T03:04:52.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.304+0000 7f5b62551740 1 -- 172.21.15.116:0/3238405818 wait complete. 2024-09-12T03:04:52.325 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:04:52.325 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:04:52.325 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": { 2024-09-12T03:04:52.325 INFO:teuthology.orchestra.run.smithi116.stdout: "c7ccc8b5-0ea8-4065-a472-9869b1e833ae": { 2024-09-12T03:04:52.325 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/", 2024-09-12T03:04:52.325 INFO:teuthology.orchestra.run.smithi116.stdout: "tag": "c7ccc8b5-0ea8-4065-a472-9869b1e833ae", 2024-09-12T03:04:52.325 INFO:teuthology.orchestra.run.smithi116.stdout: "options": "recursive,force" 2024-09-12T03:04:52.325 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:04:52.325 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:04:52.325 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:04:52.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.304+0000 7f5b62551740 20 client.15738 trim_cache size 0 max 16384 2024-09-12T03:04:52.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.304+0000 7f5b60178640 1 -- 172.21.15.116:0/1495516575 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5b440a2c00 msgr2=0x7f5b440a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:52.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.304+0000 7f5b60178640 1 --2- 172.21.15.116:0/1495516575 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5b440a2c00 0x7f5b440a50c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5b500c5780 tx=0x7f5b500b2520 comp rx=0 tx=0).stop 2024-09-12T03:04:52.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.304+0000 7f5b60178640 1 -- 172.21.15.116:0/1495516575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b58106a90 msgr2=0x7f5b5819bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:52.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.304+0000 7f5b60178640 1 --2- 172.21.15.116:0/1495516575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b58106a90 0x7f5b5819bbd0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5b50005cc0 tx=0x7f5b50032f40 comp rx=0 tx=0).stop 2024-09-12T03:04:52.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.304+0000 7f5b60178640 1 -- 172.21.15.116:0/1495516575 shutdown_connections 2024-09-12T03:04:52.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.304+0000 7f5b60178640 1 --2- 172.21.15.116:0/1495516575 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5b440a2c00 0x7f5b440a50c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.304+0000 7f5b60178640 1 --2- 172.21.15.116:0/1495516575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b58106a90 0x7f5b5819bbd0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.304+0000 7f5b60178640 1 --2- 172.21.15.116:0/1495516575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b58101430 0x7f5b58197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.304+0000 7f5b60178640 1 --2- 172.21.15.116:0/1495516575 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b58100a90 0x7f5b58197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.304+0000 7f5b60178640 1 -- 172.21.15.116:0/1495516575 >> 172.21.15.116:0/1495516575 conn(0x7f5b580fc7d0 msgr2=0x7f5b580fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:52.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.305+0000 7f5b60178640 1 -- 172.21.15.116:0/1495516575 shutdown_connections 2024-09-12T03:04:52.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:52.305+0000 7f5b60178640 1 -- 172.21.15.116:0/1495516575 wait complete. 2024-09-12T03:04:52.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:52 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:04:52.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:52 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:04:52.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:52 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:04:52.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:52 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:04:52.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:52 smithi116 ceph-mon[31410]: pgmap v394: 97 pgs: 97 active+clean; 1.0 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T03:04:52.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:52 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:04:52.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:52 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:04:52.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:52 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:04:52.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:52 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:04:52.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:52 smithi183 ceph-mon[35535]: pgmap v394: 97 pgs: 97 active+clean; 1.0 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T03:04:52.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:52 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:04:52.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:52 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:04:52.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:52 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:04:52.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:52 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:04:52.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:52 smithi177 ceph-mon[35305]: pgmap v394: 97 pgs: 97 active+clean; 1.0 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T03:04:53.058 INFO:tasks.cephfs.filesystem:scrub status for tag:c7ccc8b5-0ea8-4065-a472-9869b1e833ae - {'path': '/', 'tag': 'c7ccc8b5-0ea8-4065-a472-9869b1e833ae', 'options': 'recursive,force'} 2024-09-12T03:04:54.508 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.4 2024-09-12T03:04:54.508 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:04:54.536 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:54 smithi116 ceph-mon[31410]: pgmap v395: 97 pgs: 97 active+clean; 1.0 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:04:54.536 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:54 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:04:54.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:54 smithi183 ceph-mon[35535]: pgmap v395: 97 pgs: 97 active+clean; 1.0 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:04:54.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:54 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:04:54.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:54 smithi177 ceph-mon[35305]: pgmap v395: 97 pgs: 97 active+clean; 1.0 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:04:54.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:54 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:04:54.769 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:55.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.246+0000 7fa05e728640 1 -- 172.21.15.116:0/3188688546 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa058100e20 msgr2=0x7fa058101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:55.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.246+0000 7fa05e728640 1 --2- 172.21.15.116:0/3188688546 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa058100e20 0x7fa058101200 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa048009900 tx=0x7fa04802a480 comp rx=0 tx=0).stop 2024-09-12T03:04:55.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.246+0000 7fa05e728640 1 -- 172.21.15.116:0/3188688546 shutdown_connections 2024-09-12T03:04:55.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.246+0000 7fa05e728640 1 --2- 172.21.15.116:0/3188688546 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa058102ef0 0x7fa05810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.246+0000 7fa05e728640 1 --2- 172.21.15.116:0/3188688546 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa058102570 0x7fa0581029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.246+0000 7fa05e728640 1 --2- 172.21.15.116:0/3188688546 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa058100e20 0x7fa058101200 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.246+0000 7fa05e728640 1 -- 172.21.15.116:0/3188688546 >> 172.21.15.116:0/3188688546 conn(0x7fa0580fc7d0 msgr2=0x7fa0580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:55.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.246+0000 7fa05e728640 1 -- 172.21.15.116:0/3188688546 shutdown_connections 2024-09-12T03:04:55.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.246+0000 7fa05e728640 1 -- 172.21.15.116:0/3188688546 wait complete. 2024-09-12T03:04:55.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.247+0000 7fa05e728640 1 Processor -- start 2024-09-12T03:04:55.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.247+0000 7fa05e728640 1 -- start start 2024-09-12T03:04:55.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.247+0000 7fa05e728640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa058100e20 0x7fa0581984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:55.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.247+0000 7fa05e728640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa058102570 0x7fa0581989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:55.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.247+0000 7fa05e728640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa058102ef0 0x7fa05819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:55.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.247+0000 7fa05e728640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa058110fa0 con 0x7fa058102570 2024-09-12T03:04:55.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.247+0000 7fa05e728640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa058110e20 con 0x7fa058102ef0 2024-09-12T03:04:55.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.247+0000 7fa05e728640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa058111120 con 0x7fa058100e20 2024-09-12T03:04:55.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.247+0000 7fa04ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa058102570 0x7fa0581989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:55.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.247+0000 7fa04ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa058102570 0x7fa0581989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46710/0 (socket says 172.21.15.116:46710) 2024-09-12T03:04:55.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.247+0000 7fa04ffff640 1 -- 172.21.15.116:0/2389458174 learned_addr learned my addr 172.21.15.116:0/2389458174 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:55.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.247+0000 7fa04ffff640 1 -- 172.21.15.116:0/2389458174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa058100e20 msgr2=0x7fa0581984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:55.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.247+0000 7fa04ffff640 1 --2- 172.21.15.116:0/2389458174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa058100e20 0x7fa0581984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.248+0000 7fa04ffff640 1 -- 172.21.15.116:0/2389458174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa058102ef0 msgr2=0x7fa05819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:55.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.248+0000 7fa04ffff640 1 --2- 172.21.15.116:0/2389458174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa058102ef0 0x7fa05819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.248+0000 7fa04ffff640 1 -- 172.21.15.116:0/2389458174 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05819d460 con 0x7fa058102570 2024-09-12T03:04:55.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.248+0000 7fa04ffff640 1 --2- 172.21.15.116:0/2389458174 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa058102570 0x7fa0581989f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa04000b4a0 tx=0x7fa04000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:55.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.248+0000 7fa055ffb640 1 -- 172.21.15.116:0/2389458174 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040014070 con 0x7fa058102570 2024-09-12T03:04:55.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.248+0000 7fa055ffb640 1 -- 172.21.15.116:0/2389458174 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa040010af0 con 0x7fa058102570 2024-09-12T03:04:55.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.248+0000 7fa05e728640 1 -- 172.21.15.116:0/2389458174 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa05819d750 con 0x7fa058102570 2024-09-12T03:04:55.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.248+0000 7fa055ffb640 1 -- 172.21.15.116:0/2389458174 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040017d10 con 0x7fa058102570 2024-09-12T03:04:55.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.248+0000 7fa05e728640 1 -- 172.21.15.116:0/2389458174 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0581a5030 con 0x7fa058102570 2024-09-12T03:04:55.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.249+0000 7fa05e728640 1 -- 172.21.15.116:0/2389458174 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa058101200 con 0x7fa058102570 2024-09-12T03:04:55.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.250+0000 7fa055ffb640 1 -- 172.21.15.116:0/2389458174 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa040017070 con 0x7fa058102570 2024-09-12T03:04:55.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.250+0000 7fa055ffb640 1 --2- 172.21.15.116:0/2389458174 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa0440a2b80 0x7fa0440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:55.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.251+0000 7fa057fff640 1 --2- 172.21.15.116:0/2389458174 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa0440a2b80 0x7fa0440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:55.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.251+0000 7fa055ffb640 1 -- 172.21.15.116:0/2389458174 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa0400ce490 con 0x7fa058102570 2024-09-12T03:04:55.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.251+0000 7fa057fff640 1 --2- 172.21.15.116:0/2389458174 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa0440a2b80 0x7fa0440a5040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa0480096c0 tx=0x7fa048000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:55.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.255+0000 7fa055ffb640 1 -- 172.21.15.116:0/2389458174 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa04009b420 con 0x7fa058102570 2024-09-12T03:04:55.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.454+0000 7fa05e728640 1 -- 172.21.15.116:0/2389458174 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa05810a0a0 con 0x7fa058102570 2024-09-12T03:04:55.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.454+0000 7fa055ffb640 1 -- 172.21.15.116:0/2389458174 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7fa04009ad20 con 0x7fa058102570 2024-09-12T03:04:55.455 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:04:55.455 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:04:55.455 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:04:55.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.456+0000 7fa05e728640 1 -- 172.21.15.116:0/2389458174 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa0440a2b80 msgr2=0x7fa0440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:55.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.456+0000 7fa05e728640 1 --2- 172.21.15.116:0/2389458174 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa0440a2b80 0x7fa0440a5040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa0480096c0 tx=0x7fa048000bc0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.456+0000 7fa05e728640 1 -- 172.21.15.116:0/2389458174 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa058102570 msgr2=0x7fa0581989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:55.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.456+0000 7fa05e728640 1 --2- 172.21.15.116:0/2389458174 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa058102570 0x7fa0581989f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa04000b4a0 tx=0x7fa04000b960 comp rx=0 tx=0).stop 2024-09-12T03:04:55.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.457+0000 7fa05e728640 1 -- 172.21.15.116:0/2389458174 shutdown_connections 2024-09-12T03:04:55.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.457+0000 7fa05e728640 1 --2- 172.21.15.116:0/2389458174 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa0440a2b80 0x7fa0440a5040 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.457+0000 7fa05e728640 1 --2- 172.21.15.116:0/2389458174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa058102ef0 0x7fa05819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.457+0000 7fa05e728640 1 --2- 172.21.15.116:0/2389458174 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa058102570 0x7fa0581989f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.457+0000 7fa05e728640 1 --2- 172.21.15.116:0/2389458174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa058100e20 0x7fa0581984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.457+0000 7fa05e728640 1 -- 172.21.15.116:0/2389458174 >> 172.21.15.116:0/2389458174 conn(0x7fa0580fc7d0 msgr2=0x7fa05806a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:55.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.457+0000 7fa05e728640 1 -- 172.21.15.116:0/2389458174 shutdown_connections 2024-09-12T03:04:55.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:55.457+0000 7fa05e728640 1 -- 172.21.15.116:0/2389458174 wait complete. 2024-09-12T03:04:56.169 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0' 2024-09-12T03:04:56.169 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0 2024-09-12T03:04:56.462 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:56.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:56 smithi116 ceph-mon[31410]: pgmap v396: 97 pgs: 97 active+clean; 1.0 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:04:56.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:56 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2389458174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:04:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:56 smithi183 ceph-mon[35535]: pgmap v396: 97 pgs: 97 active+clean; 1.0 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:04:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:56 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2389458174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:04:56.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:56 smithi177 ceph-mon[35305]: pgmap v396: 97 pgs: 97 active+clean; 1.0 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:04:56.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:56 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2389458174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:04:56.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.961+0000 7fe166a3a640 1 -- 172.21.15.116:0/558301272 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe160106a80 msgr2=0x7fe160106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:56.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.961+0000 7fe166a3a640 1 --2- 172.21.15.116:0/558301272 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe160106a80 0x7fe160106e60 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe150009930 tx=0x7fe15002a420 comp rx=0 tx=0).stop 2024-09-12T03:04:56.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.961+0000 7fe166a3a640 1 -- 172.21.15.116:0/558301272 shutdown_connections 2024-09-12T03:04:56.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.961+0000 7fe166a3a640 1 --2- 172.21.15.116:0/558301272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe160101420 0x7fe16010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:56.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.961+0000 7fe166a3a640 1 --2- 172.21.15.116:0/558301272 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe160100a80 0x7fe160100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:56.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.961+0000 7fe166a3a640 1 --2- 172.21.15.116:0/558301272 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe160106a80 0x7fe160106e60 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:56.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.961+0000 7fe166a3a640 1 -- 172.21.15.116:0/558301272 >> 172.21.15.116:0/558301272 conn(0x7fe1600fc7a0 msgr2=0x7fe1600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:56.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.961+0000 7fe166a3a640 1 -- 172.21.15.116:0/558301272 shutdown_connections 2024-09-12T03:04:56.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.961+0000 7fe166a3a640 1 -- 172.21.15.116:0/558301272 wait complete. 2024-09-12T03:04:56.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.962+0000 7fe166a3a640 1 Processor -- start 2024-09-12T03:04:56.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.962+0000 7fe166a3a640 1 -- start start 2024-09-12T03:04:56.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.962+0000 7fe166a3a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe160100a80 0x7fe160198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:56.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.962+0000 7fe166a3a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe160101420 0x7fe160198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:56.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.962+0000 7fe166a3a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe160106a80 0x7fe16019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:56.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.962+0000 7fe166a3a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe160110e80 con 0x7fe160100a80 2024-09-12T03:04:56.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.962+0000 7fe166a3a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe160110d00 con 0x7fe160101420 2024-09-12T03:04:56.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.962+0000 7fe166a3a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe160111000 con 0x7fe160106a80 2024-09-12T03:04:56.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.963+0000 7fe15ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe160100a80 0x7fe160198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:56.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.963+0000 7fe15ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe160100a80 0x7fe160198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46732/0 (socket says 172.21.15.116:46732) 2024-09-12T03:04:56.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.963+0000 7fe15ffff640 1 -- 172.21.15.116:0/3722513166 learned_addr learned my addr 172.21.15.116:0/3722513166 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:56.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.963+0000 7fe15f7fe640 1 --2- 172.21.15.116:0/3722513166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe160101420 0x7fe160198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:56.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.963+0000 7fe164fd7640 1 --2- 172.21.15.116:0/3722513166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe160106a80 0x7fe16019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:56.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.963+0000 7fe15ffff640 1 -- 172.21.15.116:0/3722513166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe160106a80 msgr2=0x7fe16019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:56.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.963+0000 7fe15ffff640 1 --2- 172.21.15.116:0/3722513166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe160106a80 0x7fe16019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:56.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.963+0000 7fe15ffff640 1 -- 172.21.15.116:0/3722513166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe160101420 msgr2=0x7fe160198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:56.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.963+0000 7fe15ffff640 1 --2- 172.21.15.116:0/3722513166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe160101420 0x7fe160198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:56.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.963+0000 7fe15ffff640 1 -- 172.21.15.116:0/3722513166 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe16019d3e0 con 0x7fe160100a80 2024-09-12T03:04:56.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.963+0000 7fe15ffff640 1 --2- 172.21.15.116:0/3722513166 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe160100a80 0x7fe160198430 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe150009230 tx=0x7fe150004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:56.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.963+0000 7fe15d7fa640 1 -- 172.21.15.116:0/3722513166 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15003f030 con 0x7fe160100a80 2024-09-12T03:04:56.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.963+0000 7fe166a3a640 1 -- 172.21.15.116:0/3722513166 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe16019d670 con 0x7fe160100a80 2024-09-12T03:04:56.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.963+0000 7fe15d7fa640 1 -- 172.21.15.116:0/3722513166 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe150003ad0 con 0x7fe160100a80 2024-09-12T03:04:56.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.963+0000 7fe15d7fa640 1 -- 172.21.15.116:0/3722513166 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe150035070 con 0x7fe160100a80 2024-09-12T03:04:56.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.963+0000 7fe166a3a640 1 -- 172.21.15.116:0/3722513166 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1601a4fb0 con 0x7fe160100a80 2024-09-12T03:04:56.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.966+0000 7fe15d7fa640 1 -- 172.21.15.116:0/3722513166 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe150042030 con 0x7fe160100a80 2024-09-12T03:04:56.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.966+0000 7fe168e13740 1 Processor -- start 2024-09-12T03:04:56.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.966+0000 7fe168e13740 1 -- start start 2024-09-12T03:04:56.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.966+0000 7fe168e13740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55625861a640 0x55625861aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:56.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.966+0000 7fe168e13740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55625861af60 0x556258629810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:56.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.966+0000 7fe168e13740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556258629d50 0x55625862c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:56.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.966+0000 7fe168e13740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5562585802a0 con 0x55625861a640 2024-09-12T03:04:56.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.966+0000 7fe168e13740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556258527b10 con 0x556258629d50 2024-09-12T03:04:56.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.966+0000 7fe168e13740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55625857c260 con 0x55625861af60 2024-09-12T03:04:56.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.966+0000 7fe164fd7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55625861a640 0x55625861aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:56.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.966+0000 7fe164fd7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55625861a640 0x55625861aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46744/0 (socket says 172.21.15.116:46744) 2024-09-12T03:04:56.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.966+0000 7fe164fd7640 1 -- 172.21.15.116:0/3022258489 learned_addr learned my addr 172.21.15.116:0/3022258489 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:56.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.966+0000 7fe15ffff640 1 --2- 172.21.15.116:0/3022258489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55625861af60 0x556258629810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:56.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.966+0000 7fe15d7fa640 1 --2- 172.21.15.116:0/3722513166 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe1480a2b80 0x7fe1480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:56.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.966+0000 7fe15f7fe640 1 --2- 172.21.15.116:0/3022258489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556258629d50 0x55625862c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:56.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.967+0000 7fe164fd7640 1 -- 172.21.15.116:0/3022258489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55625861af60 msgr2=0x556258629810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:56.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.967+0000 7fe164fd7640 1 --2- 172.21.15.116:0/3022258489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55625861af60 0x556258629810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:56.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.967+0000 7fe164fd7640 1 -- 172.21.15.116:0/3022258489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556258629d50 msgr2=0x55625862c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:56.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.967+0000 7fe164fd7640 1 --2- 172.21.15.116:0/3022258489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556258629d50 0x55625862c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:56.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.967+0000 7fe164fd7640 1 -- 172.21.15.116:0/3022258489 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562582f1b90 con 0x55625861a640 2024-09-12T03:04:56.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.967+0000 7fe15f7fe640 1 --2- 172.21.15.116:0/3022258489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556258629d50 0x55625862c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:04:56.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.967+0000 7fe15ffff640 1 --2- 172.21.15.116:0/3022258489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55625861af60 0x556258629810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:56.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.967+0000 7fe164fd7640 1 --2- 172.21.15.116:0/3722513166 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe1480a2b80 0x7fe1480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:56.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.967+0000 7fe15d7fa640 1 -- 172.21.15.116:0/3722513166 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe1500ef450 con 0x7fe160100a80 2024-09-12T03:04:56.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.967+0000 7fe164fd7640 1 --2- 172.21.15.116:0/3022258489 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55625861a640 0x55625861aa20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe158010440 tx=0x7fe158010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec68c4a3f317dea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:56.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.967+0000 7fe1467fc640 1 -- 172.21.15.116:0/3022258489 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15801d400 con 0x55625861a640 2024-09-12T03:04:56.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.967+0000 7fe1467fc640 1 -- 172.21.15.116:0/3022258489 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe158013520 con 0x55625861a640 2024-09-12T03:04:56.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.967+0000 7fe1467fc640 1 -- 172.21.15.116:0/3022258489 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158011920 con 0x55625861a640 2024-09-12T03:04:56.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.968+0000 7fe168e13740 1 -- 172.21.15.116:0/3022258489 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55625861a640 msgr2=0x55625861aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:56.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.968+0000 7fe168e13740 1 --2- 172.21.15.116:0/3022258489 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55625861a640 0x55625861aa20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe158010440 tx=0x7fe158010900 comp rx=0 tx=0).stop 2024-09-12T03:04:56.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.968+0000 7fe164fd7640 1 --2- 172.21.15.116:0/3722513166 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe1480a2b80 0x7fe1480a5040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe158021000 tx=0x7fe15801e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:56.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.968+0000 7fe168e13740 1 -- 172.21.15.116:0/3022258489 shutdown_connections 2024-09-12T03:04:56.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.968+0000 7fe168e13740 1 --2- 172.21.15.116:0/3022258489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556258629d50 0x55625862c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:56.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.968+0000 7fe168e13740 1 --2- 172.21.15.116:0/3022258489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55625861af60 0x556258629810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:56.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.968+0000 7fe168e13740 1 --2- 172.21.15.116:0/3022258489 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55625861a640 0x55625861aa20 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:56.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.968+0000 7fe168e13740 1 -- 172.21.15.116:0/3022258489 >> 172.21.15.116:0/3022258489 conn(0x556258616cf0 msgr2=0x5562586170d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:56.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.968+0000 7fe168e13740 1 -- 172.21.15.116:0/3022258489 shutdown_connections 2024-09-12T03:04:56.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.968+0000 7fe168e13740 1 -- 172.21.15.116:0/3022258489 wait complete. 2024-09-12T03:04:56.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.968+0000 7fe168e13740 1 -- start start 2024-09-12T03:04:56.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.969+0000 7fe168e13740 1 Processor -- start 2024-09-12T03:04:56.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.969+0000 7fe168e13740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55625861a640 0x556258626c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:56.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.969+0000 7fe168e13740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55625861af60 0x5562586271d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:56.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.969+0000 7fe168e13740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556258629d50 0x556258627710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:56.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.969+0000 7fe168e13740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55625862c920 con 0x55625861af60 2024-09-12T03:04:56.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.969+0000 7fe168e13740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5562585086e0 con 0x556258629d50 2024-09-12T03:04:56.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.969+0000 7fe168e13740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55625862caa0 con 0x55625861a640 2024-09-12T03:04:56.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.969+0000 7fe168e13740 10 client.0 _ll_register_callbacks cb 0x55625852ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:56.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.969+0000 7fe15f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55625861af60 0x5562586271d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:56.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.969+0000 7fe15f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55625861af60 0x5562586271d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46756/0 (socket says 172.21.15.116:46756) 2024-09-12T03:04:56.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.969+0000 7fe15f7fe640 1 -- 172.21.15.116:0/1726334948 learned_addr learned my addr 172.21.15.116:0/1726334948 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:56.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.969+0000 7fe15ffff640 1 --2- 172.21.15.116:0/1726334948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55625861a640 0x556258626c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:56.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.969+0000 7fe164fd7640 1 --2- 172.21.15.116:0/1726334948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556258629d50 0x556258627710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:56.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.969+0000 7fe15f7fe640 1 -- 172.21.15.116:0/1726334948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55625861a640 msgr2=0x556258626c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:56.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.969+0000 7fe15f7fe640 1 --2- 172.21.15.116:0/1726334948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55625861a640 0x556258626c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:56.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.969+0000 7fe15f7fe640 1 -- 172.21.15.116:0/1726334948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556258629d50 msgr2=0x556258627710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:56.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.969+0000 7fe15f7fe640 1 --2- 172.21.15.116:0/1726334948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556258629d50 0x556258627710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:56.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.969+0000 7fe15f7fe640 1 -- 172.21.15.116:0/1726334948 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556258635ea0 con 0x55625861af60 2024-09-12T03:04:56.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.969+0000 7fe164fd7640 1 --2- 172.21.15.116:0/1726334948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556258629d50 0x556258627710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:04:56.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.969+0000 7fe15f7fe640 1 --2- 172.21.15.116:0/1726334948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55625861af60 0x5562586271d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe15400b940 tx=0x7fe15400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2cd52fc20d4ab9f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:56.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.969+0000 7fe13f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:04:56.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.970+0000 7fe13f7fe640 1 -- 172.21.15.116:0/1726334948 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe154003ea0 con 0x55625861af60 2024-09-12T03:04:56.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.970+0000 7fe13f7fe640 1 -- 172.21.15.116:0/1726334948 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe154005220 con 0x55625861af60 2024-09-12T03:04:56.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.970+0000 7fe168e13740 1 -- 172.21.15.116:0/1726334948 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe158013520 con 0x55625861af60 2024-09-12T03:04:56.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.970+0000 7fe13f7fe640 1 -- 172.21.15.116:0/1726334948 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe154005420 con 0x55625861af60 2024-09-12T03:04:56.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.970+0000 7fe168e13740 10 client.15756 fetch_fsmap learned FSMap version 25 2024-09-12T03:04:56.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.970+0000 7fe168e13740 1 -- 172.21.15.116:0/1726334948 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556258638850 con 0x55625861af60 2024-09-12T03:04:56.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.970+0000 7fe13f7fe640 1 -- 172.21.15.116:0/1726334948 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe158013520 con 0x55625861af60 2024-09-12T03:04:56.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.970+0000 7fe13f7fe640 1 -- 172.21.15.116:0/1726334948 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe154012870 con 0x55625861af60 2024-09-12T03:04:56.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.970+0000 7fe168e13740 10 client.15756 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:04:56.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.970+0000 7fe168e13740 10 client.15756 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:04:56.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.970+0000 7fe168e13740 20 client.15756 populate_metadata read hostname 'smithi116' 2024-09-12T03:04:56.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.970+0000 7fe168e13740 1 --2- 172.21.15.116:0/1726334948 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556258639d60 0x55625863c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:56.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.970+0000 7fe168e13740 4 client.15756 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:04:56.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.970+0000 7fe168e13740 1 -- 172.21.15.116:0/1726334948 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55625858d9d0 con 0x556258639d60 2024-09-12T03:04:56.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.970+0000 7fe15ffff640 1 --2- 172.21.15.116:0/1726334948 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556258639d60 0x55625863c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:56.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.971+0000 7fe15ffff640 1 --2- 172.21.15.116:0/1726334948 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556258639d60 0x55625863c1a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6702db20bcf6aacd server_cookie=cb940d921bc10383 in_seq=0 out_seq=0 2024-09-12T03:04:56.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.971+0000 7fe13f7fe640 10 client.15756 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:56.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.972+0000 7fe13f7fe640 1 -- 172.21.15.116:0/1726334948 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55625858d9d0 con 0x556258639d60 2024-09-12T03:04:56.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.972+0000 7fe13f7fe640 10 client.15756 handle_command_reply: tid=0 2024-09-12T03:04:56.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.972+0000 7fe168e13740 1 client.15756 shutdown 2024-09-12T03:04:56.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.972+0000 7fe168e13740 1 -- 172.21.15.116:0/1726334948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55625861af60 msgr2=0x5562586271d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:56.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.972+0000 7fe168e13740 1 --2- 172.21.15.116:0/1726334948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55625861af60 0x5562586271d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe15400b940 tx=0x7fe15400be00 comp rx=0 tx=0).stop 2024-09-12T03:04:56.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.972+0000 7fe168e13740 1 -- 172.21.15.116:0/1726334948 shutdown_connections 2024-09-12T03:04:56.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.972+0000 7fe168e13740 1 --2- 172.21.15.116:0/1726334948 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556258639d60 0x55625863c1a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:56.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.972+0000 7fe168e13740 1 --2- 172.21.15.116:0/1726334948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556258629d50 0x556258627710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:56.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.972+0000 7fe168e13740 1 --2- 172.21.15.116:0/1726334948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55625861af60 0x5562586271d0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:56.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.972+0000 7fe13f7fe640 0 client.15756 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:56.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.972+0000 7fe168e13740 1 --2- 172.21.15.116:0/1726334948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55625861a640 0x556258626c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:56.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.972+0000 7fe168e13740 1 -- 172.21.15.116:0/1726334948 >> 172.21.15.116:0/1726334948 conn(0x556258616cf0 msgr2=0x55625861ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:56.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.972+0000 7fe168e13740 1 -- 172.21.15.116:0/1726334948 shutdown_connections 2024-09-12T03:04:56.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.972+0000 7fe168e13740 1 -- 172.21.15.116:0/1726334948 wait complete. 2024-09-12T03:04:56.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.972+0000 7fe168e13740 20 client.15756 trim_cache size 0 max 16384 2024-09-12T03:04:56.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.989+0000 7fe168e13740 1 Processor -- start 2024-09-12T03:04:56.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.989+0000 7fe168e13740 1 -- start start 2024-09-12T03:04:56.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.989+0000 7fe168e13740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55625861a640 0x556258641a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:56.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.989+0000 7fe168e13740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55625861af60 0x556258641fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:56.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.989+0000 7fe168e13740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556258629d50 0x556258623ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:56.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.989+0000 7fe168e13740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe1601adad0 con 0x55625861af60 2024-09-12T03:04:56.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.989+0000 7fe168e13740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe1601ab3a0 con 0x55625861a640 2024-09-12T03:04:56.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.989+0000 7fe168e13740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe1601a91f0 con 0x556258629d50 2024-09-12T03:04:56.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.989+0000 7fe15f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55625861af60 0x556258641fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:56.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.989+0000 7fe15f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55625861af60 0x556258641fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46770/0 (socket says 172.21.15.116:46770) 2024-09-12T03:04:56.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.989+0000 7fe15f7fe640 1 -- 172.21.15.116:0/2815026111 learned_addr learned my addr 172.21.15.116:0/2815026111 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:56.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.989+0000 7fe164fd7640 1 --2- 172.21.15.116:0/2815026111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556258629d50 0x556258623ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:56.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.989+0000 7fe15ffff640 1 --2- 172.21.15.116:0/2815026111 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55625861a640 0x556258641a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:56.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.990+0000 7fe15f7fe640 1 -- 172.21.15.116:0/2815026111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556258629d50 msgr2=0x556258623ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:56.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.990+0000 7fe15f7fe640 1 --2- 172.21.15.116:0/2815026111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556258629d50 0x556258623ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:56.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.990+0000 7fe15f7fe640 1 -- 172.21.15.116:0/2815026111 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55625861a640 msgr2=0x556258641a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:56.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.990+0000 7fe15f7fe640 1 --2- 172.21.15.116:0/2815026111 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55625861a640 0x556258641a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:56.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.990+0000 7fe15f7fe640 1 -- 172.21.15.116:0/2815026111 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556258616790 con 0x55625861af60 2024-09-12T03:04:56.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.990+0000 7fe15f7fe640 1 --2- 172.21.15.116:0/2815026111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55625861af60 0x556258641fb0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe154001eb0 tx=0x7fe154005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b28ea1909a03ccd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:56.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.990+0000 7fe145ffb640 1 -- 172.21.15.116:0/2815026111 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1540055c0 con 0x55625861af60 2024-09-12T03:04:56.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.990+0000 7fe145ffb640 1 -- 172.21.15.116:0/2815026111 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe154016070 con 0x55625861af60 2024-09-12T03:04:56.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.990+0000 7fe145ffb640 1 -- 172.21.15.116:0/2815026111 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe154030c60 con 0x55625861af60 2024-09-12T03:04:56.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.991+0000 7fe168e13740 1 -- 172.21.15.116:0/2815026111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55625861af60 msgr2=0x556258641fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:56.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.991+0000 7fe168e13740 1 --2- 172.21.15.116:0/2815026111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55625861af60 0x556258641fb0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe154001eb0 tx=0x7fe154005b90 comp rx=0 tx=0).stop 2024-09-12T03:04:56.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.991+0000 7fe168e13740 1 -- 172.21.15.116:0/2815026111 shutdown_connections 2024-09-12T03:04:57.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.991+0000 7fe168e13740 1 --2- 172.21.15.116:0/2815026111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556258629d50 0x556258623ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:57.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.991+0000 7fe168e13740 1 --2- 172.21.15.116:0/2815026111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55625861af60 0x556258641fb0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:57.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.991+0000 7fe168e13740 1 --2- 172.21.15.116:0/2815026111 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55625861a640 0x556258641a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:57.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.991+0000 7fe168e13740 1 -- 172.21.15.116:0/2815026111 >> 172.21.15.116:0/2815026111 conn(0x556258616cf0 msgr2=0x556258628d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:57.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.991+0000 7fe168e13740 1 -- 172.21.15.116:0/2815026111 shutdown_connections 2024-09-12T03:04:57.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.991+0000 7fe168e13740 1 -- 172.21.15.116:0/2815026111 wait complete. 2024-09-12T03:04:57.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.991+0000 7fe168e13740 1 -- start start 2024-09-12T03:04:57.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.991+0000 7fe168e13740 1 Processor -- start 2024-09-12T03:04:57.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.991+0000 7fe168e13740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55625861a640 0x55625862e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:57.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.991+0000 7fe168e13740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55625861af60 0x55625862e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:57.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.992+0000 7fe168e13740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556258629d50 0x5562586319f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:57.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.992+0000 7fe168e13740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe1601a9860 con 0x55625861a640 2024-09-12T03:04:57.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.992+0000 7fe168e13740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe1601aacd0 con 0x556258629d50 2024-09-12T03:04:57.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.992+0000 7fe168e13740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe1601a9f30 con 0x55625861af60 2024-09-12T03:04:57.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.992+0000 7fe168e13740 10 client.0 _ll_register_callbacks cb 0x7fe154004470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:57.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.992+0000 7fe15ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55625861a640 0x55625862e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:57.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.992+0000 7fe15ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55625861a640 0x55625862e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46780/0 (socket says 172.21.15.116:46780) 2024-09-12T03:04:57.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.992+0000 7fe15ffff640 1 -- 172.21.15.116:0/2171346191 learned_addr learned my addr 172.21.15.116:0/2171346191 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:04:57.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.992+0000 7fe164fd7640 1 --2- 172.21.15.116:0/2171346191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556258629d50 0x5562586319f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:57.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.992+0000 7fe15f7fe640 1 --2- 172.21.15.116:0/2171346191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55625861af60 0x55625862e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:57.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.992+0000 7fe15ffff640 1 -- 172.21.15.116:0/2171346191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55625861af60 msgr2=0x55625862e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:57.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.992+0000 7fe15ffff640 1 --2- 172.21.15.116:0/2171346191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55625861af60 0x55625862e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:57.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.992+0000 7fe15ffff640 1 -- 172.21.15.116:0/2171346191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556258629d50 msgr2=0x5562586319f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:57.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.992+0000 7fe15ffff640 1 --2- 172.21.15.116:0/2171346191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556258629d50 0x5562586319f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:57.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.992+0000 7fe15ffff640 1 -- 172.21.15.116:0/2171346191 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556258632090 con 0x55625861a640 2024-09-12T03:04:57.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.992+0000 7fe15f7fe640 1 --2- 172.21.15.116:0/2171346191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55625861af60 0x55625862e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:04:57.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.992+0000 7fe15ffff640 1 --2- 172.21.15.116:0/2171346191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55625861a640 0x55625862e3f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe1500b8760 tx=0x7fe1500045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76372578641f3fd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:57.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.992+0000 7fe13f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:04:57.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.992+0000 7fe13f7fe640 1 -- 172.21.15.116:0/2171346191 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe150049040 con 0x55625861a640 2024-09-12T03:04:57.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.992+0000 7fe13f7fe640 1 -- 172.21.15.116:0/2171346191 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe15003b460 con 0x55625861a640 2024-09-12T03:04:57.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.992+0000 7fe168e13740 1 -- 172.21.15.116:0/2171346191 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe154016070 con 0x55625861a640 2024-09-12T03:04:57.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.993+0000 7fe13f7fe640 1 -- 172.21.15.116:0/2171346191 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe150003c80 con 0x55625861a640 2024-09-12T03:04:57.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.993+0000 7fe168e13740 10 client.15762 fetch_fsmap learned FSMap version 25 2024-09-12T03:04:57.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.993+0000 7fe168e13740 1 -- 172.21.15.116:0/2171346191 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5562586a54e0 con 0x55625861a640 2024-09-12T03:04:57.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.993+0000 7fe13f7fe640 1 -- 172.21.15.116:0/2171346191 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe154016070 con 0x55625861a640 2024-09-12T03:04:57.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.993+0000 7fe13f7fe640 1 -- 172.21.15.116:0/2171346191 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe150031bd0 con 0x55625861a640 2024-09-12T03:04:57.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.993+0000 7fe168e13740 10 client.15762 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:04:57.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.993+0000 7fe168e13740 10 client.15762 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:04:57.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.993+0000 7fe168e13740 20 client.15762 populate_metadata read hostname 'smithi116' 2024-09-12T03:04:57.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.993+0000 7fe168e13740 1 --2- 172.21.15.116:0/2171346191 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5562586a5c70 0x5562586a8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:57.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.993+0000 7fe168e13740 4 client.15762 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}] 2024-09-12T03:04:57.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.993+0000 7fe168e13740 1 -- 172.21.15.116:0/2171346191 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}) -- 0x556258587d20 con 0x5562586a5c70 2024-09-12T03:04:57.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.993+0000 7fe15f7fe640 1 --2- 172.21.15.116:0/2171346191 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5562586a5c70 0x5562586a8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:57.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.994+0000 7fe15f7fe640 1 --2- 172.21.15.116:0/2171346191 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5562586a5c70 0x5562586a8090 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9b0d5ac0a788990 server_cookie=7a6523c1228a203a in_seq=0 out_seq=0 2024-09-12T03:04:57.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:56.994+0000 7fe13f7fe640 10 client.15762 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:57.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:57.003+0000 7fe13f7fe640 1 -- 172.21.15.116:0/2171346191 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x556258587d20 con 0x5562586a5c70 2024-09-12T03:04:57.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:57.003+0000 7fe13f7fe640 10 client.15762 handle_command_reply: tid=0 2024-09-12T03:04:57.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:57.003+0000 7fe168e13740 1 client.15762 shutdown 2024-09-12T03:04:57.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:57.004+0000 7fe168e13740 1 -- 172.21.15.116:0/2171346191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55625861a640 msgr2=0x55625862e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:57.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:57.004+0000 7fe168e13740 1 --2- 172.21.15.116:0/2171346191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55625861a640 0x55625862e3f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe1500b8760 tx=0x7fe1500045f0 comp rx=0 tx=0).stop 2024-09-12T03:04:57.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:57.004+0000 7fe168e13740 1 -- 172.21.15.116:0/2171346191 shutdown_connections 2024-09-12T03:04:57.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:57.004+0000 7fe168e13740 1 --2- 172.21.15.116:0/2171346191 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5562586a5c70 0x5562586a8090 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:57.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:57.004+0000 7fe168e13740 1 --2- 172.21.15.116:0/2171346191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556258629d50 0x5562586319f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:57.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:57.004+0000 7fe168e13740 1 --2- 172.21.15.116:0/2171346191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55625861af60 0x55625862e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:57.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:57.004+0000 7fe168e13740 1 --2- 172.21.15.116:0/2171346191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55625861a640 0x55625862e3f0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:57.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:57.004+0000 7fe13f7fe640 0 client.15762 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:04:57.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:57.004+0000 7fe168e13740 1 -- 172.21.15.116:0/2171346191 >> 172.21.15.116:0/2171346191 conn(0x556258616cf0 msgr2=0x556258615650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:57.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:57.004+0000 7fe168e13740 1 -- 172.21.15.116:0/2171346191 shutdown_connections 2024-09-12T03:04:57.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:57.004+0000 7fe168e13740 1 -- 172.21.15.116:0/2171346191 wait complete. 2024-09-12T03:04:57.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:57.004+0000 7fe168e13740 20 client.15762 trim_cache size 0 max 16384 2024-09-12T03:04:57.007 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:04:57.007 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 25, 2024-09-12T03:04:57.007 INFO:teuthology.orchestra.run.smithi116.stdout: "leader": 24494, 2024-09-12T03:04:57.007 INFO:teuthology.orchestra.run.smithi116.stdout: "set_version": 6, 2024-09-12T03:04:57.007 INFO:teuthology.orchestra.run.smithi116.stdout: "sets": { 2024-09-12T03:04:57.007 INFO:teuthology.orchestra.run.smithi116.stdout: "97781e19": { 2024-09-12T03:04:57.007 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 6, 2024-09-12T03:04:57.007 INFO:teuthology.orchestra.run.smithi116.stdout: "age_ref": 0.0, 2024-09-12T03:04:57.007 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:04:57.007 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "QUIESCED", 2024-09-12T03:04:57.008 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:04:57.008 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:04:57.008 INFO:teuthology.orchestra.run.smithi116.stdout: "timeout": 90.0, 2024-09-12T03:04:57.008 INFO:teuthology.orchestra.run.smithi116.stdout: "expiration": 133.4, 2024-09-12T03:04:57.008 INFO:teuthology.orchestra.run.smithi116.stdout: "members": { 2024-09-12T03:04:57.008 INFO:teuthology.orchestra.run.smithi116.stdout: "file:/": { 2024-09-12T03:04:57.008 INFO:teuthology.orchestra.run.smithi116.stdout: "excluded": false, 2024-09-12T03:04:57.008 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:04:57.008 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "QUIESCED", 2024-09-12T03:04:57.008 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:04:57.008 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:04:57.008 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:04:57.008 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:04:57.009 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:04:57.009 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:04:57.009 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:04:57.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:57.004+0000 7fe166a3a640 1 -- 172.21.15.116:0/3722513166 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe1480a2b80 msgr2=0x7fe1480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:57.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:57.004+0000 7fe166a3a640 1 --2- 172.21.15.116:0/3722513166 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe1480a2b80 0x7fe1480a5040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe158021000 tx=0x7fe15801e000 comp rx=0 tx=0).stop 2024-09-12T03:04:57.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:57.004+0000 7fe166a3a640 1 -- 172.21.15.116:0/3722513166 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe160100a80 msgr2=0x7fe160198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:57.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:57.004+0000 7fe166a3a640 1 --2- 172.21.15.116:0/3722513166 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe160100a80 0x7fe160198430 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe150009230 tx=0x7fe150004270 comp rx=0 tx=0).stop 2024-09-12T03:04:57.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:57.004+0000 7fe166a3a640 1 -- 172.21.15.116:0/3722513166 shutdown_connections 2024-09-12T03:04:57.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:57.005+0000 7fe166a3a640 1 --2- 172.21.15.116:0/3722513166 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe1480a2b80 0x7fe1480a5040 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:57.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:57.005+0000 7fe166a3a640 1 --2- 172.21.15.116:0/3722513166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe160106a80 0x7fe16019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:57.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:57.005+0000 7fe166a3a640 1 --2- 172.21.15.116:0/3722513166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe160101420 0x7fe160198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:57.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:57.005+0000 7fe166a3a640 1 --2- 172.21.15.116:0/3722513166 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe160100a80 0x7fe160198430 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:57.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:57.005+0000 7fe166a3a640 1 -- 172.21.15.116:0/3722513166 >> 172.21.15.116:0/3722513166 conn(0x7fe1600fc7a0 msgr2=0x7fe16006a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:57.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:57.005+0000 7fe166a3a640 1 -- 172.21.15.116:0/3722513166 shutdown_connections 2024-09-12T03:04:57.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:04:57.005+0000 7fe166a3a640 1 -- 172.21.15.116:0/3722513166 wait complete. 2024-09-12T03:04:57.786 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '97781e19', quiesce took 3.3 seconds. Will release after: 10.100000000000001 2024-09-12T03:04:58.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:04:58 smithi116 ceph-mon[31410]: pgmap v397: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:04:58.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:04:58 smithi183 ceph-mon[35535]: pgmap v397: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:04:58.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:04:58 smithi177 ceph-mon[35305]: pgmap v397: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:05:00.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:00 smithi116 ceph-mon[31410]: pgmap v398: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:05:00.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:00 smithi183 ceph-mon[35535]: pgmap v398: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:05:00.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:00 smithi177 ceph-mon[35305]: pgmap v398: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:05:02.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:02 smithi116 ceph-mon[31410]: pgmap v399: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T03:05:02.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:02 smithi183 ceph-mon[35535]: pgmap v399: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T03:05:02.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:02 smithi177 ceph-mon[35305]: pgmap v399: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-09-12T03:05:04.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:04 smithi116 ceph-mon[31410]: pgmap v400: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.0 KiB/s wr, 8 op/s 2024-09-12T03:05:04.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:04 smithi183 ceph-mon[35535]: pgmap v400: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.0 KiB/s wr, 8 op/s 2024-09-12T03:05:04.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:04 smithi177 ceph-mon[35305]: pgmap v400: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.0 KiB/s wr, 8 op/s 2024-09-12T03:05:06.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:06 smithi116 ceph-mon[31410]: pgmap v401: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.1 KiB/s wr, 9 op/s 2024-09-12T03:05:06.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:06 smithi183 ceph-mon[35535]: pgmap v401: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.1 KiB/s wr, 9 op/s 2024-09-12T03:05:06.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:06 smithi177 ceph-mon[35305]: pgmap v401: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.1 KiB/s wr, 9 op/s 2024-09-12T03:05:07.887 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:05:08.147 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:08.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:08 smithi116 ceph-mon[31410]: pgmap v402: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 9 op/s 2024-09-12T03:05:08.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.650+0000 7fd1af35b640 1 -- 172.21.15.116:0/3193400834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1a8106a80 msgr2=0x7fd1a8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:08.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.650+0000 7fd1af35b640 1 --2- 172.21.15.116:0/3193400834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1a8106a80 0x7fd1a8106e60 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4009930 tx=0x7fd1a402a420 comp rx=0 tx=0).stop 2024-09-12T03:05:08.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.650+0000 7fd1af35b640 1 -- 172.21.15.116:0/3193400834 shutdown_connections 2024-09-12T03:05:08.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.650+0000 7fd1af35b640 1 --2- 172.21.15.116:0/3193400834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1a8101420 0x7fd1a810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:08.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.650+0000 7fd1af35b640 1 --2- 172.21.15.116:0/3193400834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1a8100a80 0x7fd1a8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:08.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.650+0000 7fd1af35b640 1 --2- 172.21.15.116:0/3193400834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1a8106a80 0x7fd1a8106e60 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:08.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.650+0000 7fd1af35b640 1 -- 172.21.15.116:0/3193400834 >> 172.21.15.116:0/3193400834 conn(0x7fd1a80fc7a0 msgr2=0x7fd1a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:08.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.651+0000 7fd1af35b640 1 -- 172.21.15.116:0/3193400834 shutdown_connections 2024-09-12T03:05:08.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.651+0000 7fd1af35b640 1 -- 172.21.15.116:0/3193400834 wait complete. 2024-09-12T03:05:08.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.651+0000 7fd1af35b640 1 Processor -- start 2024-09-12T03:05:08.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.651+0000 7fd1af35b640 1 -- start start 2024-09-12T03:05:08.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.651+0000 7fd1af35b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1a8100a80 0x7fd1a81984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:08.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.651+0000 7fd1af35b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1a8101420 0x7fd1a81989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:08.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.651+0000 7fd1af35b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1a8106a80 0x7fd1a819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:08.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.651+0000 7fd1af35b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a8110ea0 con 0x7fd1a8106a80 2024-09-12T03:05:08.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.651+0000 7fd1af35b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a8110d20 con 0x7fd1a8100a80 2024-09-12T03:05:08.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.651+0000 7fd1af35b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a8111020 con 0x7fd1a8101420 2024-09-12T03:05:08.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.651+0000 7fd1ad8f8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1a8106a80 0x7fd1a819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:08.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.652+0000 7fd1ad8f8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1a8106a80 0x7fd1a819cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59864/0 (socket says 172.21.15.116:59864) 2024-09-12T03:05:08.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.652+0000 7fd1ad8f8640 1 -- 172.21.15.116:0/395359810 learned_addr learned my addr 172.21.15.116:0/395359810 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:08.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.652+0000 7fd1ad0f7640 1 --2- 172.21.15.116:0/395359810 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1a8100a80 0x7fd1a81984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:08.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.652+0000 7fd1ac8f6640 1 --2- 172.21.15.116:0/395359810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1a8101420 0x7fd1a81989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:08.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.652+0000 7fd1ad8f8640 1 -- 172.21.15.116:0/395359810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1a8101420 msgr2=0x7fd1a81989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:08.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.652+0000 7fd1ad8f8640 1 --2- 172.21.15.116:0/395359810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1a8101420 0x7fd1a81989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:08.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.652+0000 7fd1ad8f8640 1 -- 172.21.15.116:0/395359810 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1a8100a80 msgr2=0x7fd1a81984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:08.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.652+0000 7fd1ad8f8640 1 --2- 172.21.15.116:0/395359810 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1a8100a80 0x7fd1a81984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:08.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.652+0000 7fd1ad8f8640 1 -- 172.21.15.116:0/395359810 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1a819d450 con 0x7fd1a8106a80 2024-09-12T03:05:08.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.652+0000 7fd1ac8f6640 1 --2- 172.21.15.116:0/395359810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1a8101420 0x7fd1a81989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:05:08.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.652+0000 7fd1ad8f8640 1 --2- 172.21.15.116:0/395359810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1a8106a80 0x7fd1a819cd70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd1a000b970 tx=0x7fd1a000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:08.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.652+0000 7fd19e7fc640 1 -- 172.21.15.116:0/395359810 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0019070 con 0x7fd1a8106a80 2024-09-12T03:05:08.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.652+0000 7fd19e7fc640 1 -- 172.21.15.116:0/395359810 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1a000cb40 con 0x7fd1a8106a80 2024-09-12T03:05:08.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.652+0000 7fd19e7fc640 1 -- 172.21.15.116:0/395359810 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0007650 con 0x7fd1a8106a80 2024-09-12T03:05:08.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.652+0000 7fd1af35b640 1 -- 172.21.15.116:0/395359810 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1a819d740 con 0x7fd1a8106a80 2024-09-12T03:05:08.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.652+0000 7fd1af35b640 1 -- 172.21.15.116:0/395359810 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1a81a5020 con 0x7fd1a8106a80 2024-09-12T03:05:08.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.653+0000 7fd1af35b640 1 -- 172.21.15.116:0/395359810 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1a8100ee0 con 0x7fd1a8106a80 2024-09-12T03:05:08.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.654+0000 7fd19e7fc640 1 -- 172.21.15.116:0/395359810 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd1a001c070 con 0x7fd1a8106a80 2024-09-12T03:05:08.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.655+0000 7fd19e7fc640 1 --2- 172.21.15.116:0/395359810 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd1940a2b80 0x7fd1940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:08.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.655+0000 7fd1ad0f7640 1 --2- 172.21.15.116:0/395359810 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd1940a2b80 0x7fd1940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:08.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.655+0000 7fd19e7fc640 1 -- 172.21.15.116:0/395359810 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd1a00ce600 con 0x7fd1a8106a80 2024-09-12T03:05:08.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.655+0000 7fd1ad0f7640 1 --2- 172.21.15.116:0/395359810 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd1940a2b80 0x7fd1940a5040 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4009900 tx=0x7fd1a40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:08.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.659+0000 7fd19e7fc640 1 -- 172.21.15.116:0/395359810 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd1a009af40 con 0x7fd1a8106a80 2024-09-12T03:05:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:08 smithi183 ceph-mon[35535]: pgmap v402: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 9 op/s 2024-09-12T03:05:08.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:08 smithi177 ceph-mon[35305]: pgmap v402: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 9 op/s 2024-09-12T03:05:08.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.858+0000 7fd1af35b640 1 -- 172.21.15.116:0/395359810 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1a81a52c0 con 0x7fd1a8106a80 2024-09-12T03:05:08.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.859+0000 7fd19e7fc640 1 -- 172.21.15.116:0/395359810 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7fd1a009f240 con 0x7fd1a8106a80 2024-09-12T03:05:08.859 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:05:08.859 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:05:08.860 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:05:08.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.861+0000 7fd1af35b640 1 -- 172.21.15.116:0/395359810 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd1940a2b80 msgr2=0x7fd1940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:08.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.861+0000 7fd1af35b640 1 --2- 172.21.15.116:0/395359810 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd1940a2b80 0x7fd1940a5040 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4009900 tx=0x7fd1a40058d0 comp rx=0 tx=0).stop 2024-09-12T03:05:08.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.861+0000 7fd1af35b640 1 -- 172.21.15.116:0/395359810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1a8106a80 msgr2=0x7fd1a819cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:08.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.861+0000 7fd1af35b640 1 --2- 172.21.15.116:0/395359810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1a8106a80 0x7fd1a819cd70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd1a000b970 tx=0x7fd1a000be30 comp rx=0 tx=0).stop 2024-09-12T03:05:08.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.861+0000 7fd1af35b640 1 -- 172.21.15.116:0/395359810 shutdown_connections 2024-09-12T03:05:08.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.861+0000 7fd1af35b640 1 --2- 172.21.15.116:0/395359810 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd1940a2b80 0x7fd1940a5040 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:08.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.861+0000 7fd1af35b640 1 --2- 172.21.15.116:0/395359810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1a8106a80 0x7fd1a819cd70 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:08.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.861+0000 7fd1af35b640 1 --2- 172.21.15.116:0/395359810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1a8101420 0x7fd1a81989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:08.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.861+0000 7fd1af35b640 1 --2- 172.21.15.116:0/395359810 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1a8100a80 0x7fd1a81984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:08.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.861+0000 7fd1af35b640 1 -- 172.21.15.116:0/395359810 >> 172.21.15.116:0/395359810 conn(0x7fd1a80fc7a0 msgr2=0x7fd1a8104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:08.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.861+0000 7fd1af35b640 1 -- 172.21.15.116:0/395359810 shutdown_connections 2024-09-12T03:05:08.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:08.861+0000 7fd1af35b640 1 -- 172.21.15.116:0/395359810 wait complete. 2024-09-12T03:05:09.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:09 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/395359810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:05:09.678 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id 97781e19 --release --await' 2024-09-12T03:05:09.678 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id 97781e19 --release --await 2024-09-12T03:05:09.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:09 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/395359810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:05:09.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:09 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/395359810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:05:09.925 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:10.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.491+0000 7fd276ddc640 1 -- 172.21.15.116:0/2414433220 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd270102ef0 msgr2=0x7fd27010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:10.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.491+0000 7fd276ddc640 1 --2- 172.21.15.116:0/2414433220 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd270102ef0 0x7fd27010da40 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd268009ca0 tx=0x7fd26802a950 comp rx=0 tx=0).stop 2024-09-12T03:05:10.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.491+0000 7fd276ddc640 1 -- 172.21.15.116:0/2414433220 shutdown_connections 2024-09-12T03:05:10.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.491+0000 7fd276ddc640 1 --2- 172.21.15.116:0/2414433220 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd270102ef0 0x7fd27010da40 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.491+0000 7fd276ddc640 1 --2- 172.21.15.116:0/2414433220 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd270102570 0x7fd2701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.491+0000 7fd276ddc640 1 --2- 172.21.15.116:0/2414433220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd270100e20 0x7fd270101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.491+0000 7fd276ddc640 1 -- 172.21.15.116:0/2414433220 >> 172.21.15.116:0/2414433220 conn(0x7fd2700fc7d0 msgr2=0x7fd2700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:10.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.491+0000 7fd276ddc640 1 -- 172.21.15.116:0/2414433220 shutdown_connections 2024-09-12T03:05:10.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.491+0000 7fd276ddc640 1 -- 172.21.15.116:0/2414433220 wait complete. 2024-09-12T03:05:10.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.492+0000 7fd276ddc640 1 Processor -- start 2024-09-12T03:05:10.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.492+0000 7fd276ddc640 1 -- start start 2024-09-12T03:05:10.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.492+0000 7fd276ddc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd270100e20 0x7fd2700fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:10.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.492+0000 7fd276ddc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd270102570 0x7fd2700fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:10.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.492+0000 7fd276ddc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd270102ef0 0x7fd2700fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:10.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.492+0000 7fd276ddc640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd270110fa0 con 0x7fd270102ef0 2024-09-12T03:05:10.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.492+0000 7fd276ddc640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd270110e20 con 0x7fd270102570 2024-09-12T03:05:10.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.492+0000 7fd276ddc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd270111120 con 0x7fd270100e20 2024-09-12T03:05:10.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.492+0000 7fd275379640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd270102ef0 0x7fd2700fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:10.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.492+0000 7fd275379640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd270102ef0 0x7fd2700fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59882/0 (socket says 172.21.15.116:59882) 2024-09-12T03:05:10.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.492+0000 7fd275379640 1 -- 172.21.15.116:0/446452885 learned_addr learned my addr 172.21.15.116:0/446452885 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:10.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.492+0000 7fd26ffff640 1 --2- 172.21.15.116:0/446452885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd270102570 0x7fd2700fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:10.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.492+0000 7fd274b78640 1 --2- 172.21.15.116:0/446452885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd270100e20 0x7fd2700fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:10.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.493+0000 7fd275379640 1 -- 172.21.15.116:0/446452885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd270100e20 msgr2=0x7fd2700fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:10.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.493+0000 7fd275379640 1 --2- 172.21.15.116:0/446452885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd270100e20 0x7fd2700fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.493+0000 7fd275379640 1 -- 172.21.15.116:0/446452885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd270102570 msgr2=0x7fd2700fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:10.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.493+0000 7fd275379640 1 --2- 172.21.15.116:0/446452885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd270102570 0x7fd2700fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.493+0000 7fd275379640 1 -- 172.21.15.116:0/446452885 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2701a6990 con 0x7fd270102ef0 2024-09-12T03:05:10.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.493+0000 7fd275379640 1 --2- 172.21.15.116:0/446452885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd270102ef0 0x7fd2700fd700 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd26802ae70 tx=0x7fd26802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:10.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.493+0000 7fd26dffb640 1 -- 172.21.15.116:0/446452885 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd268034040 con 0x7fd270102ef0 2024-09-12T03:05:10.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.493+0000 7fd276ddc640 1 -- 172.21.15.116:0/446452885 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2701a6c20 con 0x7fd270102ef0 2024-09-12T03:05:10.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.493+0000 7fd26dffb640 1 -- 172.21.15.116:0/446452885 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd268030920 con 0x7fd270102ef0 2024-09-12T03:05:10.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.493+0000 7fd26dffb640 1 -- 172.21.15.116:0/446452885 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2680047f0 con 0x7fd270102ef0 2024-09-12T03:05:10.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.493+0000 7fd276ddc640 1 -- 172.21.15.116:0/446452885 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2701a7150 con 0x7fd270102ef0 2024-09-12T03:05:10.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.496+0000 7fd2791b5740 1 Processor -- start 2024-09-12T03:05:10.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.496+0000 7fd26dffb640 1 -- 172.21.15.116:0/446452885 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd268037020 con 0x7fd270102ef0 2024-09-12T03:05:10.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.496+0000 7fd2791b5740 1 -- start start 2024-09-12T03:05:10.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.497+0000 7fd2791b5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e9446d0060 0x55e9446d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:10.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.497+0000 7fd2791b5740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9446d0980 0x55e9446dad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:10.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.497+0000 7fd2791b5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9446db240 0x55e9446dd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:10.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.497+0000 7fd2791b5740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e944635620 con 0x55e9446d0980 2024-09-12T03:05:10.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.497+0000 7fd2791b5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e9445dce50 con 0x55e9446d0060 2024-09-12T03:05:10.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.497+0000 7fd2791b5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e9446315e0 con 0x55e9446db240 2024-09-12T03:05:10.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.497+0000 7fd274b78640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9446d0980 0x55e9446dad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:10.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.497+0000 7fd274b78640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9446d0980 0x55e9446dad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59896/0 (socket says 172.21.15.116:59896) 2024-09-12T03:05:10.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.497+0000 7fd274b78640 1 -- 172.21.15.116:0/453206920 learned_addr learned my addr 172.21.15.116:0/453206920 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:10.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.497+0000 7fd26ffff640 1 --2- 172.21.15.116:0/453206920 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e9446d0060 0x55e9446d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:10.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.497+0000 7fd26ffff640 1 --2- 172.21.15.116:0/453206920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9446db240 0x55e9446dd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:10.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.497+0000 7fd26dffb640 1 --2- 172.21.15.116:0/446452885 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd2600a2b80 0x7fd2600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:10.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.497+0000 7fd274b78640 1 -- 172.21.15.116:0/453206920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9446db240 msgr2=0x55e9446dd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:10.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.497+0000 7fd274b78640 1 --2- 172.21.15.116:0/453206920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9446db240 0x55e9446dd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.497+0000 7fd274b78640 1 -- 172.21.15.116:0/453206920 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e9446d0060 msgr2=0x55e9446d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:10.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.497+0000 7fd274b78640 1 --2- 172.21.15.116:0/453206920 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e9446d0060 0x55e9446d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.497+0000 7fd275379640 1 --2- 172.21.15.116:0/446452885 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd2600a2b80 0x7fd2600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:10.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.497+0000 7fd274b78640 1 -- 172.21.15.116:0/453206920 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9445cdd40 con 0x55e9446d0980 2024-09-12T03:05:10.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.497+0000 7fd26ffff640 1 --2- 172.21.15.116:0/453206920 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e9446d0060 0x55e9446d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:10.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.497+0000 7fd26dffb640 1 -- 172.21.15.116:0/446452885 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd2680efec0 con 0x7fd270102ef0 2024-09-12T03:05:10.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.498+0000 7fd274b78640 1 --2- 172.21.15.116:0/453206920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9446d0980 0x55e9446dad00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd264009a70 tx=0x7fd264009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e20d76c83bc3203 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:10.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.498+0000 7fd275379640 1 --2- 172.21.15.116:0/446452885 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd2600a2b80 0x7fd2600a5040 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd2680bc450 tx=0x7fd268087320 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:10.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.498+0000 7fd25affd640 1 -- 172.21.15.116:0/453206920 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd264017070 con 0x55e9446d0980 2024-09-12T03:05:10.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.498+0000 7fd25affd640 1 -- 172.21.15.116:0/453206920 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd264013470 con 0x55e9446d0980 2024-09-12T03:05:10.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.498+0000 7fd25affd640 1 -- 172.21.15.116:0/453206920 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2640125f0 con 0x55e9446d0980 2024-09-12T03:05:10.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.499+0000 7fd2791b5740 1 -- 172.21.15.116:0/453206920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9446d0980 msgr2=0x55e9446dad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:10.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.499+0000 7fd2791b5740 1 --2- 172.21.15.116:0/453206920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9446d0980 0x55e9446dad00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd264009a70 tx=0x7fd264009f30 comp rx=0 tx=0).stop 2024-09-12T03:05:10.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.499+0000 7fd2791b5740 1 -- 172.21.15.116:0/453206920 shutdown_connections 2024-09-12T03:05:10.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.499+0000 7fd2791b5740 1 --2- 172.21.15.116:0/453206920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9446db240 0x55e9446dd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.499+0000 7fd2791b5740 1 --2- 172.21.15.116:0/453206920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9446d0980 0x55e9446dad00 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.499+0000 7fd2791b5740 1 --2- 172.21.15.116:0/453206920 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e9446d0060 0x55e9446d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.499+0000 7fd2791b5740 1 -- 172.21.15.116:0/453206920 >> 172.21.15.116:0/453206920 conn(0x55e9446cc5f0 msgr2=0x55e9446cc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:10.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.499+0000 7fd2791b5740 1 -- 172.21.15.116:0/453206920 shutdown_connections 2024-09-12T03:05:10.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.499+0000 7fd2791b5740 1 -- 172.21.15.116:0/453206920 wait complete. 2024-09-12T03:05:10.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.499+0000 7fd2791b5740 1 -- start start 2024-09-12T03:05:10.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.499+0000 7fd2791b5740 1 Processor -- start 2024-09-12T03:05:10.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.500+0000 7fd2791b5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9446d0060 0x55e9446d8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:10.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.500+0000 7fd2791b5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e9446d0980 0x55e9446d8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:10.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.500+0000 7fd2791b5740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9446db240 0x55e9446d8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:10.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.500+0000 7fd2791b5740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e9446dde10 con 0x55e9446db240 2024-09-12T03:05:10.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.500+0000 7fd2791b5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e9445bdf10 con 0x55e9446d0980 2024-09-12T03:05:10.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.500+0000 7fd2791b5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e9446ddf90 con 0x55e9446d0060 2024-09-12T03:05:10.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.500+0000 7fd2791b5740 10 client.0 _ll_register_callbacks cb 0x55e9445e1db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:10.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.500+0000 7fd26ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9446db240 0x55e9446d8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:10.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.500+0000 7fd26ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9446db240 0x55e9446d8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59900/0 (socket says 172.21.15.116:59900) 2024-09-12T03:05:10.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.500+0000 7fd26ffff640 1 -- 172.21.15.116:0/2211032078 learned_addr learned my addr 172.21.15.116:0/2211032078 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:10.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.500+0000 7fd26ffff640 1 --2- 172.21.15.116:0/2211032078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9446d0060 0x55e9446d8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:10.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.500+0000 7fd274b78640 1 --2- 172.21.15.116:0/2211032078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e9446d0980 0x55e9446d8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:10.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.500+0000 7fd26ffff640 1 -- 172.21.15.116:0/2211032078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9446d0060 msgr2=0x55e9446d8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:10.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.500+0000 7fd26ffff640 1 --2- 172.21.15.116:0/2211032078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9446d0060 0x55e9446d8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.500+0000 7fd26ffff640 1 -- 172.21.15.116:0/2211032078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e9446d0980 msgr2=0x55e9446d8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:10.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.500+0000 7fd26ffff640 1 --2- 172.21.15.116:0/2211032078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e9446d0980 0x55e9446d8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.500+0000 7fd26ffff640 1 -- 172.21.15.116:0/2211032078 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e944344580 con 0x55e9446db240 2024-09-12T03:05:10.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.500+0000 7fd26ffff640 1 --2- 172.21.15.116:0/2211032078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9446db240 0x55e9446d8bd0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd25c012a30 tx=0x7fd25c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0a575d83b5f145 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:10.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.500+0000 7fd24ffff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:05:10.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.501+0000 7fd24ffff640 1 -- 172.21.15.116:0/2211032078 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c00dd10 con 0x55e9446db240 2024-09-12T03:05:10.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.501+0000 7fd24ffff640 1 -- 172.21.15.116:0/2211032078 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd25c004030 con 0x55e9446db240 2024-09-12T03:05:10.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.501+0000 7fd2791b5740 1 -- 172.21.15.116:0/2211032078 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd264013470 con 0x55e9446db240 2024-09-12T03:05:10.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.501+0000 7fd24ffff640 1 -- 172.21.15.116:0/2211032078 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd25c004230 con 0x55e9446db240 2024-09-12T03:05:10.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.501+0000 7fd2791b5740 10 client.15780 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:10.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.501+0000 7fd2791b5740 1 -- 172.21.15.116:0/2211032078 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e94440e960 con 0x55e9446db240 2024-09-12T03:05:10.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.501+0000 7fd24ffff640 1 -- 172.21.15.116:0/2211032078 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd264013470 con 0x55e9446db240 2024-09-12T03:05:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.501+0000 7fd24ffff640 1 -- 172.21.15.116:0/2211032078 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd25c0049a0 con 0x55e9446db240 2024-09-12T03:05:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.501+0000 7fd2791b5740 10 client.15780 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.501+0000 7fd2791b5740 10 client.15780 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:05:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.501+0000 7fd2791b5740 20 client.15780 populate_metadata read hostname 'smithi116' 2024-09-12T03:05:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.501+0000 7fd2791b5740 1 --2- 172.21.15.116:0/2211032078 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e9446eae70 0x55e9446ed260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.501+0000 7fd2791b5740 4 client.15780 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:05:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.501+0000 7fd2791b5740 1 -- 172.21.15.116:0/2211032078 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e944642b90 con 0x55e9446eae70 2024-09-12T03:05:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.502+0000 7fd274b78640 1 --2- 172.21.15.116:0/2211032078 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e9446eae70 0x55e9446ed260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.502+0000 7fd24ffff640 10 client.15780 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.502+0000 7fd274b78640 1 --2- 172.21.15.116:0/2211032078 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e9446eae70 0x55e9446ed260 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51b20e4abac9dd14 server_cookie=615a7dd5728e0aac in_seq=0 out_seq=0 2024-09-12T03:05:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.503+0000 7fd24ffff640 1 -- 172.21.15.116:0/2211032078 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e944642b90 con 0x55e9446eae70 2024-09-12T03:05:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.503+0000 7fd24ffff640 10 client.15780 handle_command_reply: tid=0 2024-09-12T03:05:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.503+0000 7fd2791b5740 1 client.15780 shutdown 2024-09-12T03:05:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.503+0000 7fd2791b5740 1 -- 172.21.15.116:0/2211032078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9446db240 msgr2=0x55e9446d8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.503+0000 7fd2791b5740 1 --2- 172.21.15.116:0/2211032078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9446db240 0x55e9446d8bd0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd25c012a30 tx=0x7fd25c004730 comp rx=0 tx=0).stop 2024-09-12T03:05:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.503+0000 7fd2791b5740 1 -- 172.21.15.116:0/2211032078 shutdown_connections 2024-09-12T03:05:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.503+0000 7fd2791b5740 1 --2- 172.21.15.116:0/2211032078 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e9446eae70 0x55e9446ed260 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.503+0000 7fd2791b5740 1 --2- 172.21.15.116:0/2211032078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9446db240 0x55e9446d8bd0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.503+0000 7fd2791b5740 1 --2- 172.21.15.116:0/2211032078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e9446d0980 0x55e9446d8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.503+0000 7fd2791b5740 1 --2- 172.21.15.116:0/2211032078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9446d0060 0x55e9446d8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.503+0000 7fd2791b5740 1 -- 172.21.15.116:0/2211032078 >> 172.21.15.116:0/2211032078 conn(0x55e9446cc5f0 msgr2=0x55e9446cb750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.503+0000 7fd24ffff640 0 client.15780 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.503+0000 7fd2791b5740 1 -- 172.21.15.116:0/2211032078 shutdown_connections 2024-09-12T03:05:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.503+0000 7fd2791b5740 1 -- 172.21.15.116:0/2211032078 wait complete. 2024-09-12T03:05:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.503+0000 7fd2791b5740 20 client.15780 trim_cache size 0 max 16384 2024-09-12T03:05:10.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.520+0000 7fd2791b5740 1 Processor -- start 2024-09-12T03:05:10.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.520+0000 7fd2791b5740 1 -- start start 2024-09-12T03:05:10.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.520+0000 7fd2791b5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9446d0060 0x55e9446d2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:10.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.520+0000 7fd2791b5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e9446d0980 0x55e9446f2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:10.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.520+0000 7fd2791b5740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9446db240 0x55e9446f3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:10.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.520+0000 7fd2791b5740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd2701b1f30 con 0x55e9446db240 2024-09-12T03:05:10.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.520+0000 7fd2791b5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd2701af800 con 0x55e9446d0980 2024-09-12T03:05:10.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.520+0000 7fd2791b5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd2701ad670 con 0x55e9446d0060 2024-09-12T03:05:10.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.520+0000 7fd26ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9446db240 0x55e9446f3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:10.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.521+0000 7fd26ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9446db240 0x55e9446f3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59908/0 (socket says 172.21.15.116:59908) 2024-09-12T03:05:10.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.521+0000 7fd26ffff640 1 -- 172.21.15.116:0/2891938943 learned_addr learned my addr 172.21.15.116:0/2891938943 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:10.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.521+0000 7fd26ffff640 1 --2- 172.21.15.116:0/2891938943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9446d0060 0x55e9446d2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:10.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.521+0000 7fd274b78640 1 --2- 172.21.15.116:0/2891938943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e9446d0980 0x55e9446f2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:10.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.521+0000 7fd26ffff640 1 -- 172.21.15.116:0/2891938943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9446d0060 msgr2=0x55e9446d2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:10.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.521+0000 7fd26ffff640 1 --2- 172.21.15.116:0/2891938943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9446d0060 0x55e9446d2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.521+0000 7fd26ffff640 1 -- 172.21.15.116:0/2891938943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e9446d0980 msgr2=0x55e9446f2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:10.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.521+0000 7fd26ffff640 1 --2- 172.21.15.116:0/2891938943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e9446d0980 0x55e9446f2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.521+0000 7fd26ffff640 1 -- 172.21.15.116:0/2891938943 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9446daac0 con 0x55e9446db240 2024-09-12T03:05:10.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.521+0000 7fd26ffff640 1 --2- 172.21.15.116:0/2891938943 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9446db240 0x55e9446f3070 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd25c005d00 tx=0x7fd25c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6957a60bcd4f961f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:10.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.521+0000 7fd25a7fc640 1 -- 172.21.15.116:0/2891938943 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c00dd10 con 0x55e9446db240 2024-09-12T03:05:10.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.521+0000 7fd25a7fc640 1 -- 172.21.15.116:0/2891938943 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd25c004f60 con 0x55e9446db240 2024-09-12T03:05:10.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.521+0000 7fd25a7fc640 1 -- 172.21.15.116:0/2891938943 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c003b60 con 0x55e9446db240 2024-09-12T03:05:10.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.522+0000 7fd2791b5740 1 -- 172.21.15.116:0/2891938943 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9446db240 msgr2=0x55e9446f3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:10.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.522+0000 7fd2791b5740 1 --2- 172.21.15.116:0/2891938943 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9446db240 0x55e9446f3070 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd25c005d00 tx=0x7fd25c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.522+0000 7fd2791b5740 1 -- 172.21.15.116:0/2891938943 shutdown_connections 2024-09-12T03:05:10.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.522+0000 7fd2791b5740 1 --2- 172.21.15.116:0/2891938943 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9446db240 0x55e9446f3070 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.522+0000 7fd2791b5740 1 --2- 172.21.15.116:0/2891938943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e9446d0980 0x55e9446f2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.522+0000 7fd2791b5740 1 --2- 172.21.15.116:0/2891938943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9446d0060 0x55e9446d2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.522+0000 7fd2791b5740 1 -- 172.21.15.116:0/2891938943 >> 172.21.15.116:0/2891938943 conn(0x55e9446cc5f0 msgr2=0x55e9446d1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:10.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.522+0000 7fd2791b5740 1 -- 172.21.15.116:0/2891938943 shutdown_connections 2024-09-12T03:05:10.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.522+0000 7fd2791b5740 1 -- 172.21.15.116:0/2891938943 wait complete. 2024-09-12T03:05:10.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.522+0000 7fd2791b5740 1 -- start start 2024-09-12T03:05:10.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.523+0000 7fd2791b5740 1 Processor -- start 2024-09-12T03:05:10.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.523+0000 7fd2791b5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e9446d0060 0x55e9446e3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:10.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.523+0000 7fd2791b5740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9446d0980 0x55e9446df9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:10.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.523+0000 7fd2791b5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9446db240 0x55e9446dff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:10.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.523+0000 7fd2791b5740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd2701adcc0 con 0x55e9446d0980 2024-09-12T03:05:10.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.523+0000 7fd2791b5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd2701af130 con 0x55e9446d0060 2024-09-12T03:05:10.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.523+0000 7fd2791b5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd2701ae390 con 0x55e9446db240 2024-09-12T03:05:10.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.523+0000 7fd2791b5740 10 client.0 _ll_register_callbacks cb 0x7fd25c0125c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:10.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.523+0000 7fd274b78640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9446d0980 0x55e9446df9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:10.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.523+0000 7fd274b78640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9446d0980 0x55e9446df9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59924/0 (socket says 172.21.15.116:59924) 2024-09-12T03:05:10.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.523+0000 7fd274b78640 1 -- 172.21.15.116:0/987410838 learned_addr learned my addr 172.21.15.116:0/987410838 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:10.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.523+0000 7fd26ffff640 1 --2- 172.21.15.116:0/987410838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e9446d0060 0x55e9446e3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:10.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.523+0000 7fd26ffff640 1 --2- 172.21.15.116:0/987410838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9446db240 0x55e9446dff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:10.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.523+0000 7fd274b78640 1 -- 172.21.15.116:0/987410838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9446db240 msgr2=0x55e9446dff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:10.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.523+0000 7fd274b78640 1 --2- 172.21.15.116:0/987410838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9446db240 0x55e9446dff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.524+0000 7fd274b78640 1 -- 172.21.15.116:0/987410838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e9446d0060 msgr2=0x55e9446e3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:10.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.524+0000 7fd274b78640 1 --2- 172.21.15.116:0/987410838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e9446d0060 0x55e9446e3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.524+0000 7fd274b78640 1 -- 172.21.15.116:0/987410838 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9446e0470 con 0x55e9446d0980 2024-09-12T03:05:10.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.524+0000 7fd274b78640 1 --2- 172.21.15.116:0/987410838 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9446d0980 0x55e9446df9f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd264000c00 tx=0x7fd264009180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=facfb8eccbb08eea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:10.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.524+0000 7fd24ffff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:05:10.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.524+0000 7fd24ffff640 1 -- 172.21.15.116:0/987410838 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd264017040 con 0x55e9446d0980 2024-09-12T03:05:10.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.524+0000 7fd24ffff640 1 -- 172.21.15.116:0/987410838 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd264013470 con 0x55e9446d0980 2024-09-12T03:05:10.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.524+0000 7fd2791b5740 1 -- 172.21.15.116:0/987410838 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd25c004f60 con 0x55e9446d0980 2024-09-12T03:05:10.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.524+0000 7fd24ffff640 1 -- 172.21.15.116:0/987410838 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd264034070 con 0x55e9446d0980 2024-09-12T03:05:10.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.524+0000 7fd2791b5740 10 client.15786 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:10.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.524+0000 7fd2791b5740 1 -- 172.21.15.116:0/987410838 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e944756800 con 0x55e9446d0980 2024-09-12T03:05:10.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.524+0000 7fd24ffff640 1 -- 172.21.15.116:0/987410838 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd25c004f60 con 0x55e9446d0980 2024-09-12T03:05:10.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.525+0000 7fd24ffff640 1 -- 172.21.15.116:0/987410838 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd2640257a0 con 0x55e9446d0980 2024-09-12T03:05:10.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.525+0000 7fd2791b5740 10 client.15786 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:10.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.525+0000 7fd2791b5740 10 client.15786 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:05:10.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.525+0000 7fd2791b5740 20 client.15786 populate_metadata read hostname 'smithi116' 2024-09-12T03:05:10.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.525+0000 7fd2791b5740 1 --2- 172.21.15.116:0/987410838 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e944756cd0 0x55e9447590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:10.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.525+0000 7fd2791b5740 4 client.15786 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "97781e19", "release": true, "await": true}] 2024-09-12T03:05:10.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.525+0000 7fd2791b5740 1 -- 172.21.15.116:0/987410838 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "quiesce db", "set_id": "97781e19", "release": true, "await": true}) -- 0x55e94463d0a0 con 0x55e944756cd0 2024-09-12T03:05:10.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.525+0000 7fd26ffff640 1 --2- 172.21.15.116:0/987410838 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e944756cd0 0x55e9447590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:10.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.525+0000 7fd24ffff640 10 client.15786 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:10.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.525+0000 7fd26ffff640 1 --2- 172.21.15.116:0/987410838 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e944756cd0 0x55e9447590f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84460e8215af0acd server_cookie=a0dd86df5c0fcba8 in_seq=0 out_seq=0 2024-09-12T03:05:10.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.527+0000 7fd24ffff640 1 -- 172.21.15.116:0/987410838 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55e94463d0a0 con 0x55e944756cd0 2024-09-12T03:05:10.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.527+0000 7fd24ffff640 10 client.15786 handle_command_reply: tid=0 2024-09-12T03:05:10.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.527+0000 7fd2791b5740 1 client.15786 shutdown 2024-09-12T03:05:10.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.527+0000 7fd2791b5740 1 -- 172.21.15.116:0/987410838 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9446d0980 msgr2=0x55e9446df9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:10.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.527+0000 7fd2791b5740 1 --2- 172.21.15.116:0/987410838 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9446d0980 0x55e9446df9f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd264000c00 tx=0x7fd264009180 comp rx=0 tx=0).stop 2024-09-12T03:05:10.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.527+0000 7fd2791b5740 1 -- 172.21.15.116:0/987410838 shutdown_connections 2024-09-12T03:05:10.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.527+0000 7fd2791b5740 1 --2- 172.21.15.116:0/987410838 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e944756cd0 0x55e9447590f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.527+0000 7fd2791b5740 1 --2- 172.21.15.116:0/987410838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9446db240 0x55e9446dff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.527+0000 7fd2791b5740 1 --2- 172.21.15.116:0/987410838 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9446d0980 0x55e9446df9f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.527+0000 7fd24ffff640 0 client.15786 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:10.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.527+0000 7fd2791b5740 1 --2- 172.21.15.116:0/987410838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e9446d0060 0x55e9446e3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.527+0000 7fd2791b5740 1 -- 172.21.15.116:0/987410838 >> 172.21.15.116:0/987410838 conn(0x55e9446cc5f0 msgr2=0x55e9446d8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:10.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.527+0000 7fd2791b5740 1 -- 172.21.15.116:0/987410838 shutdown_connections 2024-09-12T03:05:10.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.527+0000 7fd2791b5740 1 -- 172.21.15.116:0/987410838 wait complete. 2024-09-12T03:05:10.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.527+0000 7fd2791b5740 20 client.15786 trim_cache size 0 max 16384 2024-09-12T03:05:10.532 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:05:10.532 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 25, 2024-09-12T03:05:10.532 INFO:teuthology.orchestra.run.smithi116.stdout: "leader": 24494, 2024-09-12T03:05:10.532 INFO:teuthology.orchestra.run.smithi116.stdout: "set_version": 8, 2024-09-12T03:05:10.533 INFO:teuthology.orchestra.run.smithi116.stdout: "sets": { 2024-09-12T03:05:10.533 INFO:teuthology.orchestra.run.smithi116.stdout: "97781e19": { 2024-09-12T03:05:10.533 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 8, 2024-09-12T03:05:10.533 INFO:teuthology.orchestra.run.smithi116.stdout: "age_ref": 0.0, 2024-09-12T03:05:10.533 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:05:10.533 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RELEASED", 2024-09-12T03:05:10.533 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:05:10.533 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:05:10.533 INFO:teuthology.orchestra.run.smithi116.stdout: "timeout": 90.0, 2024-09-12T03:05:10.533 INFO:teuthology.orchestra.run.smithi116.stdout: "expiration": 133.4, 2024-09-12T03:05:10.533 INFO:teuthology.orchestra.run.smithi116.stdout: "members": { 2024-09-12T03:05:10.533 INFO:teuthology.orchestra.run.smithi116.stdout: "file:/": { 2024-09-12T03:05:10.533 INFO:teuthology.orchestra.run.smithi116.stdout: "excluded": false, 2024-09-12T03:05:10.533 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:05:10.534 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RELEASED", 2024-09-12T03:05:10.534 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:05:10.534 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:05:10.534 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:05:10.534 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:05:10.534 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:05:10.534 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:05:10.534 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:05:10.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.528+0000 7fd276ddc640 1 -- 172.21.15.116:0/446452885 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd2600a2b80 msgr2=0x7fd2600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:10.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.528+0000 7fd276ddc640 1 --2- 172.21.15.116:0/446452885 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd2600a2b80 0x7fd2600a5040 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd2680bc450 tx=0x7fd268087320 comp rx=0 tx=0).stop 2024-09-12T03:05:10.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.528+0000 7fd276ddc640 1 -- 172.21.15.116:0/446452885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd270102ef0 msgr2=0x7fd2700fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:10.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.528+0000 7fd276ddc640 1 --2- 172.21.15.116:0/446452885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd270102ef0 0x7fd2700fd700 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd26802ae70 tx=0x7fd26802f5d0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.528+0000 7fd276ddc640 1 -- 172.21.15.116:0/446452885 shutdown_connections 2024-09-12T03:05:10.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.528+0000 7fd276ddc640 1 --2- 172.21.15.116:0/446452885 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd2600a2b80 0x7fd2600a5040 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.528+0000 7fd276ddc640 1 --2- 172.21.15.116:0/446452885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd270102ef0 0x7fd2700fd700 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.528+0000 7fd276ddc640 1 --2- 172.21.15.116:0/446452885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd270102570 0x7fd2700fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.528+0000 7fd276ddc640 1 --2- 172.21.15.116:0/446452885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd270100e20 0x7fd2700fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.528+0000 7fd276ddc640 1 -- 172.21.15.116:0/446452885 >> 172.21.15.116:0/446452885 conn(0x7fd2700fc7d0 msgr2=0x7fd27006a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:10.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.528+0000 7fd276ddc640 1 -- 172.21.15.116:0/446452885 shutdown_connections 2024-09-12T03:05:10.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:10.528+0000 7fd276ddc640 1 -- 172.21.15.116:0/446452885 wait complete. 2024-09-12T03:05:10.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:10 smithi183 ceph-mon[35535]: pgmap v403: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-09-12T03:05:10.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:10 smithi177 ceph-mon[35305]: pgmap v403: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-09-12T03:05:10.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:10 smithi116 ceph-mon[31410]: pgmap v403: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-09-12T03:05:11.436 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '97781e19', total seconds elapsed: 16.9 2024-09-12T03:05:11.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:11 smithi183 ceph-mon[35535]: pgmap v404: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-09-12T03:05:11.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:11 smithi177 ceph-mon[35305]: pgmap v404: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-09-12T03:05:11.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:11 smithi116 ceph-mon[31410]: pgmap v404: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-09-12T03:05:13.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:12 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:05:13.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:12 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:05:13.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:12 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:05:14.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:14 smithi177 ceph-mon[35305]: pgmap v405: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-12T03:05:14.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:14 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:05:14.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:14 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:05:14.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:14 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:05:14.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:14 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:05:14.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:14 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:05:14.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:14 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:05:14.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:14 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:05:14.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:14 smithi116 ceph-mon[31410]: pgmap v405: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-12T03:05:14.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:14 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:05:14.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:14 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:05:14.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:14 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:05:14.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:14 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:05:14.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:14 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:05:14.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:14 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:05:14.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:14 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:05:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:14 smithi183 ceph-mon[35535]: pgmap v405: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-12T03:05:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:14 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:05:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:14 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:05:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:14 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:05:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:14 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:05:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:14 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:05:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:14 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:05:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:14 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:05:16.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:16 smithi177 ceph-mon[35305]: pgmap v406: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-12T03:05:16.504 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:16 smithi183 ceph-mon[35535]: pgmap v406: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-12T03:05:16.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:16 smithi116 ceph-mon[31410]: pgmap v406: 97 pgs: 97 active+clean; 1.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-12T03:05:18.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:18 smithi177 ceph-mon[35305]: pgmap v407: 97 pgs: 97 active+clean; 1.2 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-09-12T03:05:18.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:18 smithi116 ceph-mon[31410]: pgmap v407: 97 pgs: 97 active+clean; 1.2 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-09-12T03:05:18.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:18 smithi183 ceph-mon[35535]: pgmap v407: 97 pgs: 97 active+clean; 1.2 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-09-12T03:05:20.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:20 smithi177 ceph-mon[35305]: pgmap v408: 97 pgs: 97 active+clean; 1.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.2 KiB/s wr, 7 op/s 2024-09-12T03:05:20.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:20 smithi116 ceph-mon[31410]: pgmap v408: 97 pgs: 97 active+clean; 1.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.2 KiB/s wr, 7 op/s 2024-09-12T03:05:20.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:20 smithi183 ceph-mon[35535]: pgmap v408: 97 pgs: 97 active+clean; 1.2 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.2 KiB/s wr, 7 op/s 2024-09-12T03:05:22.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:22 smithi177 ceph-mon[35305]: pgmap v409: 97 pgs: 97 active+clean; 1.2 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T03:05:22.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:22 smithi116 ceph-mon[31410]: pgmap v409: 97 pgs: 97 active+clean; 1.2 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T03:05:22.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:22 smithi183 ceph-mon[35535]: pgmap v409: 97 pgs: 97 active+clean; 1.2 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T03:05:23.059 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:05:23.323 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:23.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.793+0000 7f4787448640 1 -- 172.21.15.116:0/999563265 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4780101420 msgr2=0x7f478010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:23.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.793+0000 7f4787448640 1 --2- 172.21.15.116:0/999563265 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4780101420 0x7f478010d950 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4778009cd0 tx=0x7f477802a8e0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.793+0000 7f4787448640 1 -- 172.21.15.116:0/999563265 shutdown_connections 2024-09-12T03:05:23.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.793+0000 7f4787448640 1 --2- 172.21.15.116:0/999563265 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4780101420 0x7f478010d950 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.793+0000 7f4787448640 1 --2- 172.21.15.116:0/999563265 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4780100a80 0x7f4780100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.793+0000 7f4787448640 1 --2- 172.21.15.116:0/999563265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4780106a80 0x7f4780106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.793+0000 7f4787448640 1 -- 172.21.15.116:0/999563265 >> 172.21.15.116:0/999563265 conn(0x7f47800fc7a0 msgr2=0x7f47800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:23.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.793+0000 7f4787448640 1 -- 172.21.15.116:0/999563265 shutdown_connections 2024-09-12T03:05:23.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.793+0000 7f4787448640 1 -- 172.21.15.116:0/999563265 wait complete. 2024-09-12T03:05:23.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.794+0000 7f4787448640 1 Processor -- start 2024-09-12T03:05:23.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.794+0000 7f4787448640 1 -- start start 2024-09-12T03:05:23.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.794+0000 7f4787448640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4780100a80 0x7f4780198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:23.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.794+0000 7f4787448640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4780101420 0x7f4780198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:23.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.794+0000 7f4787448640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4780106a80 0x7f478019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:23.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.794+0000 7f4787448640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4780111fa0 con 0x7f4780101420 2024-09-12T03:05:23.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.794+0000 7f4787448640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4780111e20 con 0x7f4780100a80 2024-09-12T03:05:23.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.794+0000 7f4787448640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4780112120 con 0x7f4780106a80 2024-09-12T03:05:23.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.794+0000 7f47849e3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4780101420 0x7f4780198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:23.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.794+0000 7f47849e3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4780101420 0x7f4780198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36508/0 (socket says 172.21.15.116:36508) 2024-09-12T03:05:23.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.794+0000 7f47849e3640 1 -- 172.21.15.116:0/223760246 learned_addr learned my addr 172.21.15.116:0/223760246 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:23.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.794+0000 7f47859e5640 1 --2- 172.21.15.116:0/223760246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4780106a80 0x7f478019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:23.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.794+0000 7f47851e4640 1 --2- 172.21.15.116:0/223760246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4780100a80 0x7f4780198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:23.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.794+0000 7f47849e3640 1 -- 172.21.15.116:0/223760246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4780106a80 msgr2=0x7f478019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:23.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.794+0000 7f47849e3640 1 --2- 172.21.15.116:0/223760246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4780106a80 0x7f478019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.794+0000 7f47849e3640 1 -- 172.21.15.116:0/223760246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4780100a80 msgr2=0x7f4780198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:23.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.794+0000 7f47849e3640 1 --2- 172.21.15.116:0/223760246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4780100a80 0x7f4780198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.794+0000 7f47849e3640 1 -- 172.21.15.116:0/223760246 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f478019d3b0 con 0x7f4780101420 2024-09-12T03:05:23.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.794+0000 7f47851e4640 1 --2- 172.21.15.116:0/223760246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4780100a80 0x7f4780198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:05:23.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.795+0000 7f47849e3640 1 --2- 172.21.15.116:0/223760246 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4780101420 0x7f4780198940 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f477000c920 tx=0x7f477000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:23.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.795+0000 7f47767fc640 1 -- 172.21.15.116:0/223760246 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770007c70 con 0x7f4780101420 2024-09-12T03:05:23.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.795+0000 7f4787448640 1 -- 172.21.15.116:0/223760246 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f478019d6a0 con 0x7f4780101420 2024-09-12T03:05:23.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.795+0000 7f47767fc640 1 -- 172.21.15.116:0/223760246 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4770007e10 con 0x7f4780101420 2024-09-12T03:05:23.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.795+0000 7f47767fc640 1 -- 172.21.15.116:0/223760246 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770018890 con 0x7f4780101420 2024-09-12T03:05:23.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.795+0000 7f4787448640 1 -- 172.21.15.116:0/223760246 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47801a4f80 con 0x7f4780101420 2024-09-12T03:05:23.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.797+0000 7f47767fc640 1 -- 172.21.15.116:0/223760246 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4770018a70 con 0x7f4780101420 2024-09-12T03:05:23.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.798+0000 7f4789821740 1 Processor -- start 2024-09-12T03:05:23.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.798+0000 7f4789821740 1 -- start start 2024-09-12T03:05:23.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.798+0000 7f4789821740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558516ec24a0 0x558516ec2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:23.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.798+0000 7f4789821740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558516ec2dc0 0x558516ecd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:23.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.798+0000 7f47767fc640 1 --2- 172.21.15.116:0/223760246 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f476c0a2b80 0x7f476c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:23.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.798+0000 7f4789821740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558516ecd680 0x558516ecfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:23.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.798+0000 7f4789821740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x558516e282e0 con 0x558516ec24a0 2024-09-12T03:05:23.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.798+0000 7f4789821740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558516dcfce0 con 0x558516ecd680 2024-09-12T03:05:23.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.798+0000 7f4789821740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558516e242a0 con 0x558516ec2dc0 2024-09-12T03:05:23.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.798+0000 7f47859e5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558516ec24a0 0x558516ec2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:23.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.798+0000 7f47859e5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558516ec24a0 0x558516ec2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36510/0 (socket says 172.21.15.116:36510) 2024-09-12T03:05:23.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.798+0000 7f47859e5640 1 -- 172.21.15.116:0/1372158515 learned_addr learned my addr 172.21.15.116:0/1372158515 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:23.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.798+0000 7f47849e3640 1 --2- 172.21.15.116:0/223760246 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f476c0a2b80 0x7f476c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:23.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.798+0000 7f47767fc640 1 -- 172.21.15.116:0/223760246 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f47700cf230 con 0x7f4780101420 2024-09-12T03:05:23.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.798+0000 7f47851e4640 1 --2- 172.21.15.116:0/1372158515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558516ec2dc0 0x558516ecd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:23.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.798+0000 7f47859e5640 1 -- 172.21.15.116:0/1372158515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558516ec2dc0 msgr2=0x558516ecd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:23.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.798+0000 7f47859e5640 1 --2- 172.21.15.116:0/1372158515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558516ec2dc0 0x558516ecd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.798+0000 7f47859e5640 1 -- 172.21.15.116:0/1372158515 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558516ecd680 msgr2=0x558516ecfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:23.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.798+0000 7f47859e5640 1 --2- 172.21.15.116:0/1372158515 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558516ecd680 0x558516ecfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.798+0000 7f47859e5640 1 -- 172.21.15.116:0/1372158515 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558516db4440 con 0x558516ec24a0 2024-09-12T03:05:23.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.798+0000 7f47851e4640 1 --2- 172.21.15.116:0/1372158515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558516ec2dc0 0x558516ecd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:23.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.799+0000 7f47849e3640 1 --2- 172.21.15.116:0/223760246 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f476c0a2b80 0x7f476c0a5040 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f477000cfd0 tx=0x7f4770090f80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:23.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.799+0000 7f47859e5640 1 --2- 172.21.15.116:0/1372158515 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558516ec24a0 0x558516ec2880 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f4778005cc0 tx=0x7f4778031570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed26dede714c0d36 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:23.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.799+0000 7f476b7fe640 1 -- 172.21.15.116:0/1372158515 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4778031ac0 con 0x558516ec24a0 2024-09-12T03:05:23.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.799+0000 7f476b7fe640 1 -- 172.21.15.116:0/1372158515 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4778036040 con 0x558516ec24a0 2024-09-12T03:05:23.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.799+0000 7f476b7fe640 1 -- 172.21.15.116:0/1372158515 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4778004680 con 0x558516ec24a0 2024-09-12T03:05:23.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.799+0000 7f4789821740 1 -- 172.21.15.116:0/1372158515 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558516ec24a0 msgr2=0x558516ec2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:23.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.799+0000 7f4789821740 1 --2- 172.21.15.116:0/1372158515 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558516ec24a0 0x558516ec2880 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f4778005cc0 tx=0x7f4778031570 comp rx=0 tx=0).stop 2024-09-12T03:05:23.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.799+0000 7f4789821740 1 -- 172.21.15.116:0/1372158515 shutdown_connections 2024-09-12T03:05:23.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.799+0000 7f4789821740 1 --2- 172.21.15.116:0/1372158515 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558516ecd680 0x558516ecfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.799+0000 7f4789821740 1 --2- 172.21.15.116:0/1372158515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558516ec2dc0 0x558516ecd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.799+0000 7f4789821740 1 --2- 172.21.15.116:0/1372158515 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558516ec24a0 0x558516ec2880 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.800+0000 7f4789821740 1 -- 172.21.15.116:0/1372158515 >> 172.21.15.116:0/1372158515 conn(0x558516ebea30 msgr2=0x558516ebee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:23.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.800+0000 7f4789821740 1 -- 172.21.15.116:0/1372158515 shutdown_connections 2024-09-12T03:05:23.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.800+0000 7f4789821740 1 -- 172.21.15.116:0/1372158515 wait complete. 2024-09-12T03:05:23.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.800+0000 7f4789821740 1 -- start start 2024-09-12T03:05:23.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.800+0000 7f4789821740 1 Processor -- start 2024-09-12T03:05:23.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.800+0000 7f4789821740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558516ec24a0 0x558516eccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:23.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.800+0000 7f4789821740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558516ec2dc0 0x558516eca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:23.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.800+0000 7f4789821740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558516ecd680 0x558516eca720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:23.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.800+0000 7f4789821740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x558516ed0250 con 0x558516ec2dc0 2024-09-12T03:05:23.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.800+0000 7f4789821740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558516de3050 con 0x558516ecd680 2024-09-12T03:05:23.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.800+0000 7f4789821740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558516ed03d0 con 0x558516ec24a0 2024-09-12T03:05:23.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.800+0000 7f4789821740 10 client.0 _ll_register_callbacks cb 0x558516e23160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:23.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.800+0000 7f47851e4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558516ec2dc0 0x558516eca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:23.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.800+0000 7f47851e4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558516ec2dc0 0x558516eca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36522/0 (socket says 172.21.15.116:36522) 2024-09-12T03:05:23.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.800+0000 7f47851e4640 1 -- 172.21.15.116:0/158114956 learned_addr learned my addr 172.21.15.116:0/158114956 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:23.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.801+0000 7f47859e5640 1 --2- 172.21.15.116:0/158114956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558516ec24a0 0x558516eccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:23.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.801+0000 7f47851e4640 1 -- 172.21.15.116:0/158114956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558516ec24a0 msgr2=0x558516eccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:23.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.801+0000 7f47859e5640 1 --2- 172.21.15.116:0/158114956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558516ecd680 0x558516eca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:23.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.801+0000 7f47851e4640 1 --2- 172.21.15.116:0/158114956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558516ec24a0 0x558516eccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.801+0000 7f47851e4640 1 -- 172.21.15.116:0/158114956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558516ecd680 msgr2=0x558516eca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:23.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.801+0000 7f47851e4640 1 --2- 172.21.15.116:0/158114956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558516ecd680 0x558516eca720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.801+0000 7f47851e4640 1 -- 172.21.15.116:0/158114956 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558516b993c0 con 0x558516ec2dc0 2024-09-12T03:05:23.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.801+0000 7f47851e4640 1 --2- 172.21.15.116:0/158114956 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558516ec2dc0 0x558516eca1e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f477c00e2d0 tx=0x7f477c00e790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55848abf8b7bc83 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:23.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.801+0000 7f4768ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:05:23.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.801+0000 7f4768ff9640 1 -- 172.21.15.116:0/158114956 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c018020 con 0x558516ec2dc0 2024-09-12T03:05:23.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.801+0000 7f4789821740 1 -- 172.21.15.116:0/158114956 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4778036040 con 0x558516ec2dc0 2024-09-12T03:05:23.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.801+0000 7f4768ff9640 1 -- 172.21.15.116:0/158114956 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f477c013660 con 0x558516ec2dc0 2024-09-12T03:05:23.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.801+0000 7f4768ff9640 1 -- 172.21.15.116:0/158114956 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f477c013860 con 0x558516ec2dc0 2024-09-12T03:05:23.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.801+0000 7f4789821740 10 client.15798 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:23.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.801+0000 7f4789821740 1 -- 172.21.15.116:0/158114956 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558516edbc00 con 0x558516ec2dc0 2024-09-12T03:05:23.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.802+0000 7f4768ff9640 1 -- 172.21.15.116:0/158114956 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4778036040 con 0x558516ec2dc0 2024-09-12T03:05:23.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.802+0000 7f4768ff9640 1 -- 172.21.15.116:0/158114956 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f477c0123b0 con 0x558516ec2dc0 2024-09-12T03:05:23.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.802+0000 7f4789821740 10 client.15798 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:23.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.802+0000 7f4789821740 10 client.15798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:05:23.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.802+0000 7f4789821740 20 client.15798 populate_metadata read hostname 'smithi116' 2024-09-12T03:05:23.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.802+0000 7f4789821740 1 --2- 172.21.15.116:0/158114956 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x558516edd060 0x558516edf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:23.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.802+0000 7f4789821740 4 client.15798 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:05:23.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.802+0000 7f4789821740 1 -- 172.21.15.116:0/158114956 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558516e35850 con 0x558516edd060 2024-09-12T03:05:23.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.802+0000 7f47859e5640 1 --2- 172.21.15.116:0/158114956 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x558516edd060 0x558516edf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:23.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.803+0000 7f4768ff9640 10 client.15798 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:23.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.803+0000 7f47859e5640 1 --2- 172.21.15.116:0/158114956 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x558516edd060 0x558516edf4a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2be194401e124acf server_cookie=f839d57fd8fc07c1 in_seq=0 out_seq=0 2024-09-12T03:05:23.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.803+0000 7f4768ff9640 1 -- 172.21.15.116:0/158114956 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558516e35850 con 0x558516edd060 2024-09-12T03:05:23.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.803+0000 7f4768ff9640 10 client.15798 handle_command_reply: tid=0 2024-09-12T03:05:23.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.803+0000 7f4789821740 1 client.15798 shutdown 2024-09-12T03:05:23.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.803+0000 7f4789821740 1 -- 172.21.15.116:0/158114956 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558516ec2dc0 msgr2=0x558516eca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:23.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.803+0000 7f4789821740 1 --2- 172.21.15.116:0/158114956 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558516ec2dc0 0x558516eca1e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f477c00e2d0 tx=0x7f477c00e790 comp rx=0 tx=0).stop 2024-09-12T03:05:23.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.803+0000 7f4789821740 1 -- 172.21.15.116:0/158114956 shutdown_connections 2024-09-12T03:05:23.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.803+0000 7f4789821740 1 --2- 172.21.15.116:0/158114956 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x558516edd060 0x558516edf4a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.803+0000 7f4789821740 1 --2- 172.21.15.116:0/158114956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558516ecd680 0x558516eca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.803+0000 7f4789821740 1 --2- 172.21.15.116:0/158114956 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558516ec2dc0 0x558516eca1e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.803+0000 7f4789821740 1 --2- 172.21.15.116:0/158114956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558516ec24a0 0x558516eccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.803+0000 7f4768ff9640 0 client.15798 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:23.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.803+0000 7f4789821740 1 -- 172.21.15.116:0/158114956 >> 172.21.15.116:0/158114956 conn(0x558516ebea30 msgr2=0x558516ec4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:23.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.804+0000 7f4789821740 1 -- 172.21.15.116:0/158114956 shutdown_connections 2024-09-12T03:05:23.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.804+0000 7f4789821740 1 -- 172.21.15.116:0/158114956 wait complete. 2024-09-12T03:05:23.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.804+0000 7f4789821740 20 client.15798 trim_cache size 0 max 16384 2024-09-12T03:05:23.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.821+0000 7f4789821740 1 Processor -- start 2024-09-12T03:05:23.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.821+0000 7f4789821740 1 -- start start 2024-09-12T03:05:23.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.821+0000 7f4789821740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558516ec24a0 0x558516ed94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:23.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.821+0000 7f4789821740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558516ec2dc0 0x558516ee4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:23.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.821+0000 7f4789821740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558516ecd680 0x558516ee52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:23.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.821+0000 7f4789821740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f47801afd70 con 0x558516ecd680 2024-09-12T03:05:23.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.821+0000 7f4789821740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f47801ad640 con 0x558516ec24a0 2024-09-12T03:05:23.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.821+0000 7f4789821740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f47801ab490 con 0x558516ec2dc0 2024-09-12T03:05:23.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.822+0000 7f47859e5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558516ecd680 0x558516ee52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:23.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.822+0000 7f47859e5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558516ecd680 0x558516ee52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36528/0 (socket says 172.21.15.116:36528) 2024-09-12T03:05:23.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.822+0000 7f47859e5640 1 -- 172.21.15.116:0/1586134792 learned_addr learned my addr 172.21.15.116:0/1586134792 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:23.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.822+0000 7f47851e4640 1 --2- 172.21.15.116:0/1586134792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558516ec2dc0 0x558516ee4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:23.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.822+0000 7f47859e5640 1 --2- 172.21.15.116:0/1586134792 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558516ec24a0 0x558516ed94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:23.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.822+0000 7f47859e5640 1 -- 172.21.15.116:0/1586134792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558516ec2dc0 msgr2=0x558516ee4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:23.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.822+0000 7f47859e5640 1 --2- 172.21.15.116:0/1586134792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558516ec2dc0 0x558516ee4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.822+0000 7f47859e5640 1 -- 172.21.15.116:0/1586134792 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558516ec24a0 msgr2=0x558516ed94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:23.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.822+0000 7f47859e5640 1 --2- 172.21.15.116:0/1586134792 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558516ec24a0 0x558516ed94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.822+0000 7f47859e5640 1 -- 172.21.15.116:0/1586134792 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558516ebe860 con 0x558516ecd680 2024-09-12T03:05:23.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.822+0000 7f47851e4640 1 --2- 172.21.15.116:0/1586134792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558516ec2dc0 0x558516ee4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:23.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.822+0000 7f47859e5640 1 --2- 172.21.15.116:0/1586134792 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558516ecd680 0x558516ee52b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f4778004290 tx=0x7f4778000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9cdec82959c402 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:23.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.822+0000 7f476affd640 1 -- 172.21.15.116:0/1586134792 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47780044f0 con 0x558516ecd680 2024-09-12T03:05:23.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.822+0000 7f476affd640 1 -- 172.21.15.116:0/1586134792 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4778036040 con 0x558516ecd680 2024-09-12T03:05:23.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.822+0000 7f476affd640 1 -- 172.21.15.116:0/1586134792 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47780305a0 con 0x558516ecd680 2024-09-12T03:05:23.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.823+0000 7f4789821740 1 -- 172.21.15.116:0/1586134792 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558516ecd680 msgr2=0x558516ee52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:23.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.823+0000 7f4789821740 1 --2- 172.21.15.116:0/1586134792 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558516ecd680 0x558516ee52b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f4778004290 tx=0x7f4778000f40 comp rx=0 tx=0).stop 2024-09-12T03:05:23.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.823+0000 7f4789821740 1 -- 172.21.15.116:0/1586134792 shutdown_connections 2024-09-12T03:05:23.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.823+0000 7f4789821740 1 --2- 172.21.15.116:0/1586134792 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558516ecd680 0x558516ee52b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.823+0000 7f4789821740 1 --2- 172.21.15.116:0/1586134792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558516ec2dc0 0x558516ee4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.823+0000 7f4789821740 1 --2- 172.21.15.116:0/1586134792 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558516ec24a0 0x558516ed94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.823+0000 7f4789821740 1 -- 172.21.15.116:0/1586134792 >> 172.21.15.116:0/1586134792 conn(0x558516ebea30 msgr2=0x558516ec92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:23.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.823+0000 7f4789821740 1 -- 172.21.15.116:0/1586134792 shutdown_connections 2024-09-12T03:05:23.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.823+0000 7f4789821740 1 -- 172.21.15.116:0/1586134792 wait complete. 2024-09-12T03:05:23.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.823+0000 7f4789821740 1 -- start start 2024-09-12T03:05:23.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.824+0000 7f4789821740 1 Processor -- start 2024-09-12T03:05:23.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.824+0000 7f4789821740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558516ec24a0 0x558516ed5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:23.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.824+0000 7f4789821740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558516ec2dc0 0x558516ed1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:23.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.824+0000 7f4789821740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558516ecd680 0x558516ed1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:23.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.824+0000 7f4789821740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f47801abb00 con 0x558516ec2dc0 2024-09-12T03:05:23.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.824+0000 7f4789821740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f47801acf70 con 0x558516ecd680 2024-09-12T03:05:23.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.824+0000 7f4789821740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f47801ac1d0 con 0x558516ec24a0 2024-09-12T03:05:23.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.824+0000 7f4789821740 10 client.0 _ll_register_callbacks cb 0x7f477c00c170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:23.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.824+0000 7f47851e4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558516ec2dc0 0x558516ed1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:23.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.824+0000 7f47851e4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558516ec2dc0 0x558516ed1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36534/0 (socket says 172.21.15.116:36534) 2024-09-12T03:05:23.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.824+0000 7f47851e4640 1 -- 172.21.15.116:0/3861834210 learned_addr learned my addr 172.21.15.116:0/3861834210 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:23.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.824+0000 7f47859e5640 1 --2- 172.21.15.116:0/3861834210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558516ec24a0 0x558516ed5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:23.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.824+0000 7f47859e5640 1 --2- 172.21.15.116:0/3861834210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558516ecd680 0x558516ed1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:23.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.824+0000 7f47851e4640 1 -- 172.21.15.116:0/3861834210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558516ec24a0 msgr2=0x558516ed5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:23.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.824+0000 7f47851e4640 1 --2- 172.21.15.116:0/3861834210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558516ec24a0 0x558516ed5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.824+0000 7f47851e4640 1 -- 172.21.15.116:0/3861834210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558516ecd680 msgr2=0x558516ed1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:23.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.824+0000 7f47851e4640 1 --2- 172.21.15.116:0/3861834210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558516ecd680 0x558516ed1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.824+0000 7f47851e4640 1 -- 172.21.15.116:0/3861834210 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558516ed1d90 con 0x558516ec2dc0 2024-09-12T03:05:23.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.824+0000 7f47859e5640 1 --2- 172.21.15.116:0/3861834210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558516ecd680 0x558516ed1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:05:23.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.825+0000 7f47851e4640 1 --2- 172.21.15.116:0/3861834210 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558516ec2dc0 0x558516ed1040 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f477c012850 tx=0x7f477c00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7332d4234f413b2c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:23.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.825+0000 7f4768ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:05:23.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.825+0000 7f4768ff9640 1 -- 172.21.15.116:0/3861834210 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c02c060 con 0x558516ec2dc0 2024-09-12T03:05:23.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.825+0000 7f4768ff9640 1 -- 172.21.15.116:0/3861834210 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f477c013a00 con 0x558516ec2dc0 2024-09-12T03:05:23.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.825+0000 7f4789821740 1 -- 172.21.15.116:0/3861834210 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4778036040 con 0x558516ec2dc0 2024-09-12T03:05:23.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.825+0000 7f4768ff9640 1 -- 172.21.15.116:0/3861834210 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f477c018070 con 0x558516ec2dc0 2024-09-12T03:05:23.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.825+0000 7f4789821740 10 client.15804 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:23.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.825+0000 7f4789821740 1 -- 172.21.15.116:0/3861834210 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558516f47cc0 con 0x558516ec2dc0 2024-09-12T03:05:23.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.825+0000 7f4768ff9640 1 -- 172.21.15.116:0/3861834210 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4778036040 con 0x558516ec2dc0 2024-09-12T03:05:23.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.825+0000 7f4768ff9640 1 -- 172.21.15.116:0/3861834210 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f477c026070 con 0x558516ec2dc0 2024-09-12T03:05:23.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.825+0000 7f4789821740 10 client.15804 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:23.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.825+0000 7f4789821740 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:05:23.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.825+0000 7f4789821740 20 client.15804 populate_metadata read hostname 'smithi116' 2024-09-12T03:05:23.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.825+0000 7f4789821740 1 --2- 172.21.15.116:0/3861834210 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x558516f483f0 0x558516f4a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:23.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.825+0000 7f4789821740 4 client.15804 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:05:23.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.825+0000 7f4789821740 1 -- 172.21.15.116:0/3861834210 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558516e2fd60 con 0x558516f483f0 2024-09-12T03:05:23.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.826+0000 7f47859e5640 1 --2- 172.21.15.116:0/3861834210 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x558516f483f0 0x558516f4a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:23.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.826+0000 7f4768ff9640 10 client.15804 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:23.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.826+0000 7f47859e5640 1 --2- 172.21.15.116:0/3861834210 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x558516f483f0 0x558516f4a810 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e8dab9f13cb188 server_cookie=7dcd7fba05c4e30d in_seq=0 out_seq=0 2024-09-12T03:05:23.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.826+0000 7f4768ff9640 1 -- 172.21.15.116:0/3861834210 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558516e2fd60 con 0x558516f483f0 2024-09-12T03:05:23.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.826+0000 7f4768ff9640 10 client.15804 handle_command_reply: tid=0 2024-09-12T03:05:23.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.826+0000 7f4789821740 1 client.15804 shutdown 2024-09-12T03:05:23.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.826+0000 7f4789821740 1 -- 172.21.15.116:0/3861834210 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558516ec2dc0 msgr2=0x558516ed1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:23.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.826+0000 7f4789821740 1 --2- 172.21.15.116:0/3861834210 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558516ec2dc0 0x558516ed1040 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f477c012850 tx=0x7f477c00ed10 comp rx=0 tx=0).stop 2024-09-12T03:05:23.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.826+0000 7f4789821740 1 -- 172.21.15.116:0/3861834210 shutdown_connections 2024-09-12T03:05:23.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.826+0000 7f4789821740 1 --2- 172.21.15.116:0/3861834210 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x558516f483f0 0x558516f4a810 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.826+0000 7f4789821740 1 --2- 172.21.15.116:0/3861834210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558516ecd680 0x558516ed1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.826+0000 7f4789821740 1 --2- 172.21.15.116:0/3861834210 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558516ec2dc0 0x558516ed1040 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.826+0000 7f4768ff9640 0 client.15804 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:23.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.826+0000 7f4789821740 1 --2- 172.21.15.116:0/3861834210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558516ec24a0 0x558516ed5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.826+0000 7f4789821740 1 -- 172.21.15.116:0/3861834210 >> 172.21.15.116:0/3861834210 conn(0x558516ebea30 msgr2=0x558516ecba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:23.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.826+0000 7f4789821740 1 -- 172.21.15.116:0/3861834210 shutdown_connections 2024-09-12T03:05:23.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.826+0000 7f4789821740 1 -- 172.21.15.116:0/3861834210 wait complete. 2024-09-12T03:05:23.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.827+0000 7f4789821740 20 client.15804 trim_cache size 0 max 16384 2024-09-12T03:05:23.835 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:05:23.835 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:05:23.835 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:05:23.835 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:05:23.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.827+0000 7f4787448640 1 -- 172.21.15.116:0/223760246 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f476c0a2b80 msgr2=0x7f476c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:23.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.827+0000 7f4787448640 1 --2- 172.21.15.116:0/223760246 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f476c0a2b80 0x7f476c0a5040 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f477000cfd0 tx=0x7f4770090f80 comp rx=0 tx=0).stop 2024-09-12T03:05:23.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.827+0000 7f4787448640 1 -- 172.21.15.116:0/223760246 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4780101420 msgr2=0x7f4780198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:23.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.827+0000 7f4787448640 1 --2- 172.21.15.116:0/223760246 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4780101420 0x7f4780198940 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f477000c920 tx=0x7f477000cde0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.827+0000 7f4787448640 1 -- 172.21.15.116:0/223760246 shutdown_connections 2024-09-12T03:05:23.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.827+0000 7f4787448640 1 --2- 172.21.15.116:0/223760246 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f476c0a2b80 0x7f476c0a5040 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.827+0000 7f4787448640 1 --2- 172.21.15.116:0/223760246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4780106a80 0x7f478019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.827+0000 7f4787448640 1 --2- 172.21.15.116:0/223760246 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4780101420 0x7f4780198940 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.827+0000 7f4787448640 1 --2- 172.21.15.116:0/223760246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4780100a80 0x7f4780198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.827+0000 7f4787448640 1 -- 172.21.15.116:0/223760246 >> 172.21.15.116:0/223760246 conn(0x7f47800fc7a0 msgr2=0x7f47800fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:23.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.827+0000 7f4787448640 1 -- 172.21.15.116:0/223760246 shutdown_connections 2024-09-12T03:05:23.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:23.827+0000 7f4787448640 1 -- 172.21.15.116:0/223760246 wait complete. 2024-09-12T03:05:24.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:24 smithi177 ceph-mon[35305]: pgmap v410: 97 pgs: 97 active+clean; 1.2 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:05:24.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:24 smithi116 ceph-mon[31410]: pgmap v410: 97 pgs: 97 active+clean; 1.2 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:05:24.662 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:05:24.662 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:05:24.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:24 smithi183 ceph-mon[35535]: pgmap v410: 97 pgs: 97 active+clean; 1.2 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:05:24.908 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:25.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.574+0000 7f2b93b67640 1 -- 172.21.15.116:0/4146492955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b8c074590 msgr2=0x7f2b8c074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:25.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.574+0000 7f2b93b67640 1 --2- 172.21.15.116:0/4146492955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b8c074590 0x7f2b8c074970 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2b88009900 tx=0x7f2b8802a480 comp rx=0 tx=0).stop 2024-09-12T03:05:25.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.574+0000 7f2b93b67640 1 -- 172.21.15.116:0/4146492955 shutdown_connections 2024-09-12T03:05:25.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.574+0000 7f2b93b67640 1 --2- 172.21.15.116:0/4146492955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b8c112b90 0x7f2b8c114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:25.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.574+0000 7f2b93b67640 1 --2- 172.21.15.116:0/4146492955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b8c074eb0 0x7f2b8c112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:25.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.574+0000 7f2b93b67640 1 --2- 172.21.15.116:0/4146492955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b8c074590 0x7f2b8c074970 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:25.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.574+0000 7f2b93b67640 1 -- 172.21.15.116:0/4146492955 >> 172.21.15.116:0/4146492955 conn(0x7f2b8c0fd8c0 msgr2=0x7f2b8c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:25.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.574+0000 7f2b93b67640 1 -- 172.21.15.116:0/4146492955 shutdown_connections 2024-09-12T03:05:25.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.574+0000 7f2b93b67640 1 -- 172.21.15.116:0/4146492955 wait complete. 2024-09-12T03:05:25.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.575+0000 7f2b93b67640 1 Processor -- start 2024-09-12T03:05:25.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.575+0000 7f2b93b67640 1 -- start start 2024-09-12T03:05:25.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.575+0000 7f2b93b67640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b8c074590 0x7f2b8c1a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:25.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.575+0000 7f2b93b67640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b8c074eb0 0x7f2b8c1a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:25.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.575+0000 7f2b93b67640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b8c112b90 0x7f2b8c19e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:25.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.575+0000 7f2b93b67640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c1185c0 con 0x7f2b8c112b90 2024-09-12T03:05:25.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.575+0000 7f2b93b67640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c118440 con 0x7f2b8c074590 2024-09-12T03:05:25.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.575+0000 7f2b93b67640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c118740 con 0x7f2b8c074eb0 2024-09-12T03:05:25.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.575+0000 7f2b92104640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b8c112b90 0x7f2b8c19e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:25.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.575+0000 7f2b92104640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b8c112b90 0x7f2b8c19e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36554/0 (socket says 172.21.15.116:36554) 2024-09-12T03:05:25.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.575+0000 7f2b92104640 1 -- 172.21.15.116:0/2571327847 learned_addr learned my addr 172.21.15.116:0/2571327847 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:25.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.575+0000 7f2b91102640 1 --2- 172.21.15.116:0/2571327847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b8c074eb0 0x7f2b8c1a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:25.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.576+0000 7f2b91903640 1 --2- 172.21.15.116:0/2571327847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b8c074590 0x7f2b8c1a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:25.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.576+0000 7f2b92104640 1 -- 172.21.15.116:0/2571327847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b8c074eb0 msgr2=0x7f2b8c1a4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:25.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.576+0000 7f2b92104640 1 --2- 172.21.15.116:0/2571327847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b8c074eb0 0x7f2b8c1a4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:25.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.576+0000 7f2b92104640 1 -- 172.21.15.116:0/2571327847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b8c074590 msgr2=0x7f2b8c1a4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:25.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.576+0000 7f2b92104640 1 --2- 172.21.15.116:0/2571327847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b8c074590 0x7f2b8c1a4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:25.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.576+0000 7f2b92104640 1 -- 172.21.15.116:0/2571327847 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b8c19f0f0 con 0x7f2b8c112b90 2024-09-12T03:05:25.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.576+0000 7f2b92104640 1 --2- 172.21.15.116:0/2571327847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b8c112b90 0x7f2b8c19e9c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2b8400dac0 tx=0x7f2b8400df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:25.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.576+0000 7f2b82ffd640 1 -- 172.21.15.116:0/2571327847 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b84017070 con 0x7f2b8c112b90 2024-09-12T03:05:25.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.576+0000 7f2b82ffd640 1 -- 172.21.15.116:0/2571327847 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b8400be30 con 0x7f2b8c112b90 2024-09-12T03:05:25.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.576+0000 7f2b93b67640 1 -- 172.21.15.116:0/2571327847 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b8c19f3e0 con 0x7f2b8c112b90 2024-09-12T03:05:25.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.576+0000 7f2b82ffd640 1 -- 172.21.15.116:0/2571327847 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b8400f890 con 0x7f2b8c112b90 2024-09-12T03:05:25.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.576+0000 7f2b93b67640 1 -- 172.21.15.116:0/2571327847 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b8c1ab6d0 con 0x7f2b8c112b90 2024-09-12T03:05:25.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.577+0000 7f2b93b67640 1 -- 172.21.15.116:0/2571327847 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b8c074970 con 0x7f2b8c112b90 2024-09-12T03:05:25.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.578+0000 7f2b82ffd640 1 -- 172.21.15.116:0/2571327847 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2b84004a40 con 0x7f2b8c112b90 2024-09-12T03:05:25.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.579+0000 7f2b82ffd640 1 --2- 172.21.15.116:0/2571327847 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2b780a2b80 0x7f2b780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:25.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.579+0000 7f2b91903640 1 --2- 172.21.15.116:0/2571327847 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2b780a2b80 0x7f2b780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:25.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.579+0000 7f2b82ffd640 1 -- 172.21.15.116:0/2571327847 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f2b840cef70 con 0x7f2b8c112b90 2024-09-12T03:05:25.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.579+0000 7f2b91903640 1 --2- 172.21.15.116:0/2571327847 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2b780a2b80 0x7f2b780a5040 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2b88000c00 tx=0x7f2b88005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:25.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.582+0000 7f2b82ffd640 1 -- 172.21.15.116:0/2571327847 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2b8409bfa0 con 0x7f2b8c112b90 2024-09-12T03:05:25.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.782+0000 7f2b93b67640 1 -- 172.21.15.116:0/2571327847 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b8c19fef0 con 0x7f2b8c112b90 2024-09-12T03:05:25.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.782+0000 7f2b82ffd640 1 -- 172.21.15.116:0/2571327847 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f2b8409b8a0 con 0x7f2b8c112b90 2024-09-12T03:05:25.783 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:05:25.783 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:05:25.783 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:05:25.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.784+0000 7f2b93b67640 1 -- 172.21.15.116:0/2571327847 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2b780a2b80 msgr2=0x7f2b780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:25.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.784+0000 7f2b93b67640 1 --2- 172.21.15.116:0/2571327847 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2b780a2b80 0x7f2b780a5040 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2b88000c00 tx=0x7f2b88005a20 comp rx=0 tx=0).stop 2024-09-12T03:05:25.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.784+0000 7f2b93b67640 1 -- 172.21.15.116:0/2571327847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b8c112b90 msgr2=0x7f2b8c19e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:25.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.784+0000 7f2b93b67640 1 --2- 172.21.15.116:0/2571327847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b8c112b90 0x7f2b8c19e9c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2b8400dac0 tx=0x7f2b8400df80 comp rx=0 tx=0).stop 2024-09-12T03:05:25.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.784+0000 7f2b93b67640 1 -- 172.21.15.116:0/2571327847 shutdown_connections 2024-09-12T03:05:25.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.784+0000 7f2b93b67640 1 --2- 172.21.15.116:0/2571327847 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2b780a2b80 0x7f2b780a5040 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:25.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.784+0000 7f2b93b67640 1 --2- 172.21.15.116:0/2571327847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b8c112b90 0x7f2b8c19e9c0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:25.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.784+0000 7f2b93b67640 1 --2- 172.21.15.116:0/2571327847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b8c074eb0 0x7f2b8c1a4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:25.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.784+0000 7f2b93b67640 1 --2- 172.21.15.116:0/2571327847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b8c074590 0x7f2b8c1a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:25.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.784+0000 7f2b93b67640 1 -- 172.21.15.116:0/2571327847 >> 172.21.15.116:0/2571327847 conn(0x7f2b8c0fd8c0 msgr2=0x7f2b8c0ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:25.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.785+0000 7f2b93b67640 1 -- 172.21.15.116:0/2571327847 shutdown_connections 2024-09-12T03:05:25.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:25.785+0000 7f2b93b67640 1 -- 172.21.15.116:0/2571327847 wait complete. 2024-09-12T03:05:26.468 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:26 smithi116 ceph-mon[31410]: pgmap v411: 97 pgs: 97 active+clean; 1.2 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:05:26.468 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:26 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2571327847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:05:26.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:26 smithi177 ceph-mon[35305]: pgmap v411: 97 pgs: 97 active+clean; 1.2 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:05:26.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:26 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2571327847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:05:26.496 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:05:26.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:26 smithi183 ceph-mon[35535]: pgmap v411: 97 pgs: 97 active+clean; 1.2 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:05:26.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:26 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2571327847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:05:26.741 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:27.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.237+0000 7fc54443b640 1 -- 172.21.15.116:0/725659966 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc53c102d30 msgr2=0x7fc53c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:27.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.237+0000 7fc54443b640 1 --2- 172.21.15.116:0/725659966 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc53c102d30 0x7fc53c103110 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009cd0 tx=0x7fc52c02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.237+0000 7fc54443b640 1 -- 172.21.15.116:0/725659966 shutdown_connections 2024-09-12T03:05:27.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.237+0000 7fc54443b640 1 --2- 172.21.15.116:0/725659966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc53c1082a0 0x7fc53c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.237+0000 7fc54443b640 1 --2- 172.21.15.116:0/725659966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc53c1036e0 0x7fc53c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.237+0000 7fc54443b640 1 --2- 172.21.15.116:0/725659966 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc53c102d30 0x7fc53c103110 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.237+0000 7fc54443b640 1 -- 172.21.15.116:0/725659966 >> 172.21.15.116:0/725659966 conn(0x7fc53c0fc7d0 msgr2=0x7fc53c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:27.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.237+0000 7fc54443b640 1 -- 172.21.15.116:0/725659966 shutdown_connections 2024-09-12T03:05:27.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.237+0000 7fc54443b640 1 -- 172.21.15.116:0/725659966 wait complete. 2024-09-12T03:05:27.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.238+0000 7fc54443b640 1 Processor -- start 2024-09-12T03:05:27.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.238+0000 7fc54443b640 1 -- start start 2024-09-12T03:05:27.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.238+0000 7fc54443b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc53c102d30 0x7fc53c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:27.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.238+0000 7fc54443b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc53c1036e0 0x7fc53c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:27.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.238+0000 7fc54443b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc53c1082a0 0x7fc53c19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:27.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.238+0000 7fc54443b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c1120e0 con 0x7fc53c1036e0 2024-09-12T03:05:27.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.238+0000 7fc54443b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c111f60 con 0x7fc53c102d30 2024-09-12T03:05:27.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.238+0000 7fc54443b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c112260 con 0x7fc53c1082a0 2024-09-12T03:05:27.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.238+0000 7fc5419d6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc53c1036e0 0x7fc53c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:27.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.238+0000 7fc5419d6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc53c1036e0 0x7fc53c198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36570/0 (socket says 172.21.15.116:36570) 2024-09-12T03:05:27.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.238+0000 7fc5419d6640 1 -- 172.21.15.116:0/170482346 learned_addr learned my addr 172.21.15.116:0/170482346 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:27.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.238+0000 7fc5421d7640 1 --2- 172.21.15.116:0/170482346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc53c102d30 0x7fc53c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:27.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.238+0000 7fc5429d8640 1 --2- 172.21.15.116:0/170482346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc53c1082a0 0x7fc53c19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:27.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.238+0000 7fc5419d6640 1 -- 172.21.15.116:0/170482346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc53c1082a0 msgr2=0x7fc53c19cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:27.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.238+0000 7fc5419d6640 1 --2- 172.21.15.116:0/170482346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc53c1082a0 0x7fc53c19cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.238+0000 7fc5419d6640 1 -- 172.21.15.116:0/170482346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc53c102d30 msgr2=0x7fc53c198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:27.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.238+0000 7fc5419d6640 1 --2- 172.21.15.116:0/170482346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc53c102d30 0x7fc53c198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.238+0000 7fc5419d6640 1 -- 172.21.15.116:0/170482346 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53c19d4d0 con 0x7fc53c1036e0 2024-09-12T03:05:27.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.238+0000 7fc5429d8640 1 --2- 172.21.15.116:0/170482346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc53c1082a0 0x7fc53c19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:27.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.239+0000 7fc5419d6640 1 --2- 172.21.15.116:0/170482346 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc53c1036e0 0x7fc53c198a60 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc53000c8c0 tx=0x7fc53000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:27.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.239+0000 7fc53b7fe640 1 -- 172.21.15.116:0/170482346 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc530007bb0 con 0x7fc53c1036e0 2024-09-12T03:05:27.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.239+0000 7fc53b7fe640 1 -- 172.21.15.116:0/170482346 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc530007d50 con 0x7fc53c1036e0 2024-09-12T03:05:27.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.239+0000 7fc54443b640 1 -- 172.21.15.116:0/170482346 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc53c19d7c0 con 0x7fc53c1036e0 2024-09-12T03:05:27.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.239+0000 7fc53b7fe640 1 -- 172.21.15.116:0/170482346 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc530018830 con 0x7fc53c1036e0 2024-09-12T03:05:27.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.239+0000 7fc54443b640 1 -- 172.21.15.116:0/170482346 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc53c1a50a0 con 0x7fc53c1036e0 2024-09-12T03:05:27.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.241+0000 7fc53b7fe640 1 -- 172.21.15.116:0/170482346 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fc530018a10 con 0x7fc53c1036e0 2024-09-12T03:05:27.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.242+0000 7fc546814740 1 Processor -- start 2024-09-12T03:05:27.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.242+0000 7fc546814740 1 -- start start 2024-09-12T03:05:27.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.242+0000 7fc546814740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d473c62b40 0x55d473c61090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:27.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.242+0000 7fc546814740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d473c615d0 0x55d473c61a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:27.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.242+0000 7fc546814740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d473c6c370 0x55d473c6e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:27.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.242+0000 7fc53b7fe640 1 --2- 172.21.15.116:0/170482346 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc5240a2b80 0x7fc5240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:27.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.242+0000 7fc546814740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d473bc72f0 con 0x55d473c6c370 2024-09-12T03:05:27.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.242+0000 7fc546814740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d473b6ecf0 con 0x55d473c615d0 2024-09-12T03:05:27.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.242+0000 7fc546814740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d473bc32b0 con 0x55d473c62b40 2024-09-12T03:05:27.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.242+0000 7fc5429d8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d473c6c370 0x55d473c6e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:27.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.242+0000 7fc5419d6640 1 --2- 172.21.15.116:0/170482346 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc5240a2b80 0x7fc5240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:27.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.242+0000 7fc5429d8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d473c6c370 0x55d473c6e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36574/0 (socket says 172.21.15.116:36574) 2024-09-12T03:05:27.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.242+0000 7fc5429d8640 1 -- 172.21.15.116:0/1384254055 learned_addr learned my addr 172.21.15.116:0/1384254055 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:27.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.242+0000 7fc5421d7640 1 --2- 172.21.15.116:0/1384254055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d473c615d0 0x55d473c61a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:27.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.242+0000 7fc5429d8640 1 --2- 172.21.15.116:0/1384254055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d473c62b40 0x55d473c61090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:27.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.242+0000 7fc53b7fe640 1 -- 172.21.15.116:0/170482346 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fc5300cf110 con 0x7fc53c1036e0 2024-09-12T03:05:27.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.243+0000 7fc5429d8640 1 -- 172.21.15.116:0/1384254055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d473c62b40 msgr2=0x55d473c61090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:27.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.243+0000 7fc5429d8640 1 --2- 172.21.15.116:0/1384254055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d473c62b40 0x55d473c61090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.243+0000 7fc5429d8640 1 -- 172.21.15.116:0/1384254055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d473c615d0 msgr2=0x55d473c61a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:27.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.243+0000 7fc5429d8640 1 --2- 172.21.15.116:0/1384254055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d473c615d0 0x55d473c61a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.243+0000 7fc5429d8640 1 -- 172.21.15.116:0/1384254055 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d473b533a0 con 0x55d473c6c370 2024-09-12T03:05:27.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.243+0000 7fc5421d7640 1 --2- 172.21.15.116:0/1384254055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d473c615d0 0x55d473c61a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:27.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.243+0000 7fc5419d6640 1 --2- 172.21.15.116:0/170482346 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc5240a2b80 0x7fc5240a5040 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc53009b6a0 tx=0x7fc530013b80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:27.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.243+0000 7fc5429d8640 1 --2- 172.21.15.116:0/1384254055 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d473c6c370 0x55d473c6e760 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc534014990 tx=0x7fc53400c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdbc5ec43cf5c8b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:27.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.243+0000 7fc538dea640 1 -- 172.21.15.116:0/1384254055 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5340142d0 con 0x55d473c6c370 2024-09-12T03:05:27.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.243+0000 7fc538dea640 1 -- 172.21.15.116:0/1384254055 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc534005500 con 0x55d473c6c370 2024-09-12T03:05:27.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.243+0000 7fc538dea640 1 -- 172.21.15.116:0/1384254055 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53400f760 con 0x55d473c6c370 2024-09-12T03:05:27.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.244+0000 7fc546814740 1 -- 172.21.15.116:0/1384254055 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d473c6c370 msgr2=0x55d473c6e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:27.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.244+0000 7fc546814740 1 --2- 172.21.15.116:0/1384254055 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d473c6c370 0x55d473c6e760 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc534014990 tx=0x7fc53400c4b0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.244+0000 7fc546814740 1 -- 172.21.15.116:0/1384254055 shutdown_connections 2024-09-12T03:05:27.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.244+0000 7fc546814740 1 --2- 172.21.15.116:0/1384254055 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d473c6c370 0x55d473c6e760 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.244+0000 7fc546814740 1 --2- 172.21.15.116:0/1384254055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d473c615d0 0x55d473c61a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.244+0000 7fc546814740 1 --2- 172.21.15.116:0/1384254055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d473c62b40 0x55d473c61090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.244+0000 7fc546814740 1 -- 172.21.15.116:0/1384254055 >> 172.21.15.116:0/1384254055 conn(0x55d473c2a310 msgr2=0x55d473c5dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:27.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.244+0000 7fc546814740 1 -- 172.21.15.116:0/1384254055 shutdown_connections 2024-09-12T03:05:27.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.244+0000 7fc546814740 1 -- 172.21.15.116:0/1384254055 wait complete. 2024-09-12T03:05:27.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.244+0000 7fc546814740 1 -- start start 2024-09-12T03:05:27.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.244+0000 7fc546814740 1 Processor -- start 2024-09-12T03:05:27.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.244+0000 7fc546814740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d473c615d0 0x55d473c6bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:27.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.244+0000 7fc546814740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d473c62b40 0x55d473c68ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:27.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.245+0000 7fc546814740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d473c6c370 0x55d473c695e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:27.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.245+0000 7fc546814740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d473c6f070 con 0x55d473c615d0 2024-09-12T03:05:27.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.245+0000 7fc546814740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d473b82060 con 0x55d473c6c370 2024-09-12T03:05:27.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.245+0000 7fc546814740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d473c6f1f0 con 0x55d473c62b40 2024-09-12T03:05:27.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.245+0000 7fc546814740 10 client.0 _ll_register_callbacks cb 0x55d4738d58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:27.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.245+0000 7fc5429d8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d473c615d0 0x55d473c6bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:27.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.245+0000 7fc5429d8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d473c615d0 0x55d473c6bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36578/0 (socket says 172.21.15.116:36578) 2024-09-12T03:05:27.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.245+0000 7fc5429d8640 1 -- 172.21.15.116:0/1967949088 learned_addr learned my addr 172.21.15.116:0/1967949088 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:27.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.245+0000 7fc5421d7640 1 --2- 172.21.15.116:0/1967949088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d473c62b40 0x55d473c68ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:27.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.245+0000 7fc5429d8640 1 --2- 172.21.15.116:0/1967949088 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d473c6c370 0x55d473c695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:27.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.245+0000 7fc5429d8640 1 -- 172.21.15.116:0/1967949088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d473c62b40 msgr2=0x55d473c68ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:27.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.245+0000 7fc5429d8640 1 --2- 172.21.15.116:0/1967949088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d473c62b40 0x55d473c68ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.245+0000 7fc5429d8640 1 -- 172.21.15.116:0/1967949088 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d473c6c370 msgr2=0x55d473c695e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:27.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.245+0000 7fc5429d8640 1 --2- 172.21.15.116:0/1967949088 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d473c6c370 0x55d473c695e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.245+0000 7fc5429d8640 1 -- 172.21.15.116:0/1967949088 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4739383c0 con 0x55d473c615d0 2024-09-12T03:05:27.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.245+0000 7fc5429d8640 1 --2- 172.21.15.116:0/1967949088 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d473c615d0 0x55d473c6bd40 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc53400cea0 tx=0x7fc534005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c2cdd0f3a184a8a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:27.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.245+0000 7fc521ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:05:27.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.245+0000 7fc521ffb640 1 -- 172.21.15.116:0/1967949088 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53402a020 con 0x55d473c615d0 2024-09-12T03:05:27.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.245+0000 7fc521ffb640 1 -- 172.21.15.116:0/1967949088 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fc534004fc0 con 0x55d473c615d0 2024-09-12T03:05:27.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.245+0000 7fc546814740 1 -- 172.21.15.116:0/1967949088 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc534005500 con 0x55d473c615d0 2024-09-12T03:05:27.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.245+0000 7fc521ffb640 1 -- 172.21.15.116:0/1967949088 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc534014620 con 0x55d473c615d0 2024-09-12T03:05:27.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.246+0000 7fc546814740 10 client.15822 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:27.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.246+0000 7fc546814740 1 -- 172.21.15.116:0/1967949088 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d473c7aae0 con 0x55d473c615d0 2024-09-12T03:05:27.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.246+0000 7fc521ffb640 1 -- 172.21.15.116:0/1967949088 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc534005500 con 0x55d473c615d0 2024-09-12T03:05:27.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.246+0000 7fc521ffb640 1 -- 172.21.15.116:0/1967949088 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc534012570 con 0x55d473c615d0 2024-09-12T03:05:27.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.246+0000 7fc546814740 10 client.15822 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:27.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.246+0000 7fc546814740 10 client.15822 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:05:27.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.246+0000 7fc546814740 20 client.15822 populate_metadata read hostname 'smithi116' 2024-09-12T03:05:27.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.246+0000 7fc546814740 1 --2- 172.21.15.116:0/1967949088 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55d473c7be30 0x55d473c7e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:27.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.246+0000 7fc546814740 4 client.15822 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:05:27.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.246+0000 7fc546814740 1 -- 172.21.15.116:0/1967949088 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d473bd4860 con 0x55d473c7be30 2024-09-12T03:05:27.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.246+0000 7fc5421d7640 1 --2- 172.21.15.116:0/1967949088 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55d473c7be30 0x55d473c7e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:27.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.247+0000 7fc521ffb640 10 client.15822 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:05:27.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.247+0000 7fc5421d7640 1 --2- 172.21.15.116:0/1967949088 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55d473c7be30 0x55d473c7e220 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a58d57f62ee1061d server_cookie=4c23f9c06b3507d8 in_seq=0 out_seq=0 2024-09-12T03:05:27.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.247+0000 7fc521ffb640 1 -- 172.21.15.116:0/1967949088 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d473bd4860 con 0x55d473c7be30 2024-09-12T03:05:27.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.247+0000 7fc521ffb640 10 client.15822 handle_command_reply: tid=0 2024-09-12T03:05:27.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.248+0000 7fc546814740 1 client.15822 shutdown 2024-09-12T03:05:27.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.248+0000 7fc546814740 1 -- 172.21.15.116:0/1967949088 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d473c615d0 msgr2=0x55d473c6bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:27.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.248+0000 7fc546814740 1 --2- 172.21.15.116:0/1967949088 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d473c615d0 0x55d473c6bd40 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc53400cea0 tx=0x7fc534005d10 comp rx=0 tx=0).stop 2024-09-12T03:05:27.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.248+0000 7fc546814740 1 -- 172.21.15.116:0/1967949088 shutdown_connections 2024-09-12T03:05:27.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.248+0000 7fc546814740 1 --2- 172.21.15.116:0/1967949088 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d473c6c370 0x55d473c695e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.248+0000 7fc546814740 1 --2- 172.21.15.116:0/1967949088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d473c62b40 0x55d473c68ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.248+0000 7fc546814740 1 --2- 172.21.15.116:0/1967949088 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55d473c7be30 0x55d473c7e220 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.248+0000 7fc546814740 1 --2- 172.21.15.116:0/1967949088 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d473c615d0 0x55d473c6bd40 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.248+0000 7fc521ffb640 0 client.15822 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:05:27.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.248+0000 7fc546814740 1 -- 172.21.15.116:0/1967949088 >> 172.21.15.116:0/1967949088 conn(0x55d473c2a310 msgr2=0x55d473c63960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:27.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.248+0000 7fc546814740 1 -- 172.21.15.116:0/1967949088 shutdown_connections 2024-09-12T03:05:27.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.248+0000 7fc546814740 1 -- 172.21.15.116:0/1967949088 wait complete. 2024-09-12T03:05:27.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.248+0000 7fc546814740 20 client.15822 trim_cache size 0 max 16384 2024-09-12T03:05:27.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.260+0000 7fc546814740 1 Processor -- start 2024-09-12T03:05:27.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.260+0000 7fc546814740 1 -- start start 2024-09-12T03:05:27.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.261+0000 7fc546814740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d473c615d0 0x55d473c5dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:27.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.261+0000 7fc546814740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d473c62b40 0x55d473c81630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:27.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.261+0000 7fc546814740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d473c6c370 0x55d473c81b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:27.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.261+0000 7fc546814740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c1afd90 con 0x55d473c615d0 2024-09-12T03:05:27.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.261+0000 7fc546814740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c1ad660 con 0x55d473c62b40 2024-09-12T03:05:27.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.261+0000 7fc546814740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c1ab4b0 con 0x55d473c6c370 2024-09-12T03:05:27.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.261+0000 7fc5429d8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d473c615d0 0x55d473c5dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:27.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.261+0000 7fc5429d8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d473c615d0 0x55d473c5dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36586/0 (socket says 172.21.15.116:36586) 2024-09-12T03:05:27.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.261+0000 7fc5429d8640 1 -- 172.21.15.116:0/2460175696 learned_addr learned my addr 172.21.15.116:0/2460175696 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:27.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.261+0000 7fc5429d8640 1 --2- 172.21.15.116:0/2460175696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d473c6c370 0x55d473c81b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:27.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.261+0000 7fc5429d8640 1 -- 172.21.15.116:0/2460175696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d473c6c370 msgr2=0x55d473c81b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:27.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.261+0000 7fc5429d8640 1 --2- 172.21.15.116:0/2460175696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d473c6c370 0x55d473c81b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.261+0000 7fc5429d8640 1 -- 172.21.15.116:0/2460175696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d473c62b40 msgr2=0x55d473c81630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:27.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.261+0000 7fc5429d8640 1 --2- 172.21.15.116:0/2460175696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d473c62b40 0x55d473c81630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.261+0000 7fc5429d8640 1 -- 172.21.15.116:0/2460175696 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d473c69950 con 0x55d473c615d0 2024-09-12T03:05:27.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.261+0000 7fc5429d8640 1 --2- 172.21.15.116:0/2460175696 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d473c615d0 0x55d473c5dcd0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc534005d80 tx=0x7fc53400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7663800b2383123 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:27.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.261+0000 7fc523fff640 1 -- 172.21.15.116:0/2460175696 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534012cb0 con 0x55d473c615d0 2024-09-12T03:05:27.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.261+0000 7fc523fff640 1 -- 172.21.15.116:0/2460175696 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5340048d0 con 0x55d473c615d0 2024-09-12T03:05:27.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.261+0000 7fc523fff640 1 -- 172.21.15.116:0/2460175696 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534005580 con 0x55d473c615d0 2024-09-12T03:05:27.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.262+0000 7fc546814740 1 -- 172.21.15.116:0/2460175696 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d473c615d0 msgr2=0x55d473c5dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:27.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.262+0000 7fc546814740 1 --2- 172.21.15.116:0/2460175696 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d473c615d0 0x55d473c5dcd0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc534005d80 tx=0x7fc53400cb30 comp rx=0 tx=0).stop 2024-09-12T03:05:27.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.262+0000 7fc546814740 1 -- 172.21.15.116:0/2460175696 shutdown_connections 2024-09-12T03:05:27.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.262+0000 7fc546814740 1 --2- 172.21.15.116:0/2460175696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d473c6c370 0x55d473c81b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.262+0000 7fc546814740 1 --2- 172.21.15.116:0/2460175696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d473c62b40 0x55d473c81630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.262+0000 7fc546814740 1 --2- 172.21.15.116:0/2460175696 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d473c615d0 0x55d473c5dcd0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.262+0000 7fc546814740 1 -- 172.21.15.116:0/2460175696 >> 172.21.15.116:0/2460175696 conn(0x55d473c2a310 msgr2=0x55d473c680e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:27.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.262+0000 7fc546814740 1 -- 172.21.15.116:0/2460175696 shutdown_connections 2024-09-12T03:05:27.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.262+0000 7fc546814740 1 -- 172.21.15.116:0/2460175696 wait complete. 2024-09-12T03:05:27.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.262+0000 7fc546814740 1 -- start start 2024-09-12T03:05:27.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.262+0000 7fc546814740 1 Processor -- start 2024-09-12T03:05:27.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.263+0000 7fc546814740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d473c615d0 0x55d473cde720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:27.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.263+0000 7fc546814740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d473c62b40 0x55d473cdec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:27.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.263+0000 7fc546814740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d473c6c370 0x55d473cdf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:27.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.263+0000 7fc546814740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c1abb20 con 0x55d473c615d0 2024-09-12T03:05:27.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.263+0000 7fc546814740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c1acf90 con 0x55d473c6c370 2024-09-12T03:05:27.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.263+0000 7fc546814740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c1ac1f0 con 0x55d473c62b40 2024-09-12T03:05:27.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.263+0000 7fc546814740 10 client.0 _ll_register_callbacks cb 0x7fc52c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:27.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.263+0000 7fc5429d8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d473c615d0 0x55d473cde720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:27.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.263+0000 7fc5429d8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d473c615d0 0x55d473cde720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36588/0 (socket says 172.21.15.116:36588) 2024-09-12T03:05:27.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.263+0000 7fc5429d8640 1 -- 172.21.15.116:0/1674826078 learned_addr learned my addr 172.21.15.116:0/1674826078 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:27.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.263+0000 7fc5421d7640 1 --2- 172.21.15.116:0/1674826078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d473c62b40 0x55d473cdec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:27.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.263+0000 7fc5429d8640 1 -- 172.21.15.116:0/1674826078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d473c62b40 msgr2=0x55d473cdec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:27.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.263+0000 7fc5429d8640 1 --2- 172.21.15.116:0/1674826078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d473c62b40 0x55d473cdec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.263+0000 7fc5429d8640 1 -- 172.21.15.116:0/1674826078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d473c6c370 msgr2=0x55d473cdf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:27.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.263+0000 7fc5429d8640 1 --2- 172.21.15.116:0/1674826078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d473c6c370 0x55d473cdf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.263+0000 7fc5429d8640 1 -- 172.21.15.116:0/1674826078 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d473ce9970 con 0x55d473c615d0 2024-09-12T03:05:27.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.263+0000 7fc5429d8640 1 --2- 172.21.15.116:0/1674826078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d473c615d0 0x55d473cde720 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc53400d010 tx=0x7fc53400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5fa13a77285ae0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:27.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.263+0000 7fc521ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:05:27.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.263+0000 7fc521ffb640 1 -- 172.21.15.116:0/1674826078 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534042060 con 0x55d473c615d0 2024-09-12T03:05:27.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.263+0000 7fc521ffb640 1 -- 172.21.15.116:0/1674826078 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fc534012450 con 0x55d473c615d0 2024-09-12T03:05:27.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.263+0000 7fc546814740 1 -- 172.21.15.116:0/1674826078 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5340048d0 con 0x55d473c615d0 2024-09-12T03:05:27.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.264+0000 7fc521ffb640 1 -- 172.21.15.116:0/1674826078 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc53400c950 con 0x55d473c615d0 2024-09-12T03:05:27.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.264+0000 7fc546814740 10 client.15828 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:27.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.264+0000 7fc546814740 1 -- 172.21.15.116:0/1674826078 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d473ceac50 con 0x55d473c615d0 2024-09-12T03:05:27.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.264+0000 7fc521ffb640 1 -- 172.21.15.116:0/1674826078 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5340048d0 con 0x55d473c615d0 2024-09-12T03:05:27.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.264+0000 7fc521ffb640 1 -- 172.21.15.116:0/1674826078 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc534031480 con 0x55d473c615d0 2024-09-12T03:05:27.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.264+0000 7fc546814740 10 client.15828 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:27.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.264+0000 7fc546814740 10 client.15828 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:05:27.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.264+0000 7fc546814740 20 client.15828 populate_metadata read hostname 'smithi116' 2024-09-12T03:05:27.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.264+0000 7fc546814740 1 --2- 172.21.15.116:0/1674826078 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55d473ceb260 0x55d473ced650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:27.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.264+0000 7fc546814740 4 client.15828 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:05:27.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.264+0000 7fc546814740 1 -- 172.21.15.116:0/1674826078 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d473bced70 con 0x55d473ceb260 2024-09-12T03:05:27.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.264+0000 7fc5421d7640 1 --2- 172.21.15.116:0/1674826078 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55d473ceb260 0x55d473ced650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:27.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.265+0000 7fc5421d7640 1 --2- 172.21.15.116:0/1674826078 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55d473ceb260 0x55d473ced650 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ff73681ee5a7ce20 server_cookie=69df216e489ca8a3 in_seq=0 out_seq=0 2024-09-12T03:05:27.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.265+0000 7fc521ffb640 10 client.15828 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:05:27.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.265+0000 7fc521ffb640 1 -- 172.21.15.116:0/1674826078 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d473bced70 con 0x55d473ceb260 2024-09-12T03:05:27.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.265+0000 7fc521ffb640 10 client.15828 handle_command_reply: tid=0 2024-09-12T03:05:27.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.265+0000 7fc546814740 1 client.15828 shutdown 2024-09-12T03:05:27.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.265+0000 7fc546814740 1 -- 172.21.15.116:0/1674826078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d473c615d0 msgr2=0x55d473cde720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:27.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.265+0000 7fc546814740 1 --2- 172.21.15.116:0/1674826078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d473c615d0 0x55d473cde720 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc53400d010 tx=0x7fc53400cd80 comp rx=0 tx=0).stop 2024-09-12T03:05:27.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.265+0000 7fc546814740 1 -- 172.21.15.116:0/1674826078 shutdown_connections 2024-09-12T03:05:27.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.265+0000 7fc546814740 1 --2- 172.21.15.116:0/1674826078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d473c6c370 0x55d473cdf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.265+0000 7fc546814740 1 --2- 172.21.15.116:0/1674826078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d473c62b40 0x55d473cdec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.265+0000 7fc546814740 1 --2- 172.21.15.116:0/1674826078 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55d473ceb260 0x55d473ced650 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.265+0000 7fc546814740 1 --2- 172.21.15.116:0/1674826078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d473c615d0 0x55d473cde720 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.265+0000 7fc521ffb640 0 client.15828 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:05:27.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.265+0000 7fc546814740 1 -- 172.21.15.116:0/1674826078 >> 172.21.15.116:0/1674826078 conn(0x55d473c2a310 msgr2=0x55d473c5c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:27.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.265+0000 7fc546814740 1 -- 172.21.15.116:0/1674826078 shutdown_connections 2024-09-12T03:05:27.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.265+0000 7fc546814740 1 -- 172.21.15.116:0/1674826078 wait complete. 2024-09-12T03:05:27.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.265+0000 7fc546814740 20 client.15828 trim_cache size 0 max 16384 2024-09-12T03:05:27.282 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:05:27.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.266+0000 7fc54443b640 1 -- 172.21.15.116:0/170482346 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc5240a2b80 msgr2=0x7fc5240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:27.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.266+0000 7fc54443b640 1 --2- 172.21.15.116:0/170482346 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc5240a2b80 0x7fc5240a5040 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc53009b6a0 tx=0x7fc530013b80 comp rx=0 tx=0).stop 2024-09-12T03:05:27.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.266+0000 7fc54443b640 1 -- 172.21.15.116:0/170482346 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc53c1036e0 msgr2=0x7fc53c198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:27.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.266+0000 7fc54443b640 1 --2- 172.21.15.116:0/170482346 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc53c1036e0 0x7fc53c198a60 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc53000c8c0 tx=0x7fc53000cd80 comp rx=0 tx=0).stop 2024-09-12T03:05:27.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.266+0000 7fc54443b640 1 -- 172.21.15.116:0/170482346 shutdown_connections 2024-09-12T03:05:27.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.266+0000 7fc54443b640 1 --2- 172.21.15.116:0/170482346 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc5240a2b80 0x7fc5240a5040 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.266+0000 7fc54443b640 1 --2- 172.21.15.116:0/170482346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc53c1082a0 0x7fc53c19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.266+0000 7fc54443b640 1 --2- 172.21.15.116:0/170482346 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc53c1036e0 0x7fc53c198a60 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.266+0000 7fc54443b640 1 --2- 172.21.15.116:0/170482346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc53c102d30 0x7fc53c198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.266+0000 7fc54443b640 1 -- 172.21.15.116:0/170482346 >> 172.21.15.116:0/170482346 conn(0x7fc53c0fc7d0 msgr2=0x7fc53c105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:27.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.266+0000 7fc54443b640 1 -- 172.21.15.116:0/170482346 shutdown_connections 2024-09-12T03:05:27.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:27.266+0000 7fc54443b640 1 -- 172.21.15.116:0/170482346 wait complete. 2024-09-12T03:05:28.006 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:05:28.250 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:28.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:28 smithi177 ceph-mon[35305]: pgmap v412: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:05:28.516 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:28 smithi116 ceph-mon[31410]: pgmap v412: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:05:28.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:28 smithi183 ceph-mon[35535]: pgmap v412: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:05:28.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.731+0000 7f18b5ebc640 1 -- 172.21.15.116:0/3565953605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18b01082a0 msgr2=0x7f18b010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:28.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.731+0000 7f18b5ebc640 1 --2- 172.21.15.116:0/3565953605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18b01082a0 0x7f18b010ec40 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009cd0 tx=0x7f18a002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.731+0000 7f18b5ebc640 1 -- 172.21.15.116:0/3565953605 shutdown_connections 2024-09-12T03:05:28.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.731+0000 7f18b5ebc640 1 --2- 172.21.15.116:0/3565953605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18b01082a0 0x7f18b010ec40 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.731+0000 7f18b5ebc640 1 --2- 172.21.15.116:0/3565953605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18b01036e0 0x7f18b0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.731+0000 7f18b5ebc640 1 --2- 172.21.15.116:0/3565953605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18b0102d30 0x7f18b0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.731+0000 7f18b5ebc640 1 -- 172.21.15.116:0/3565953605 >> 172.21.15.116:0/3565953605 conn(0x7f18b00fc7d0 msgr2=0x7f18b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:28.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.731+0000 7f18b5ebc640 1 -- 172.21.15.116:0/3565953605 shutdown_connections 2024-09-12T03:05:28.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.731+0000 7f18b5ebc640 1 -- 172.21.15.116:0/3565953605 wait complete. 2024-09-12T03:05:28.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.731+0000 7f18b5ebc640 1 Processor -- start 2024-09-12T03:05:28.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.731+0000 7f18b5ebc640 1 -- start start 2024-09-12T03:05:28.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.732+0000 7f18b5ebc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18b0102d30 0x7f18b0198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:28.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.732+0000 7f18b5ebc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18b01036e0 0x7f18b0198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:28.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.732+0000 7f18b5ebc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18b01082a0 0x7f18b019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:28.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.732+0000 7f18b5ebc640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f18b0112100 con 0x7f18b01082a0 2024-09-12T03:05:28.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.732+0000 7f18b5ebc640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f18b0111f80 con 0x7f18b01036e0 2024-09-12T03:05:28.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.732+0000 7f18b5ebc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f18b0112280 con 0x7f18b0102d30 2024-09-12T03:05:28.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.732+0000 7f18affff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18b01082a0 0x7f18b019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:28.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.732+0000 7f18affff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18b01082a0 0x7f18b019ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36616/0 (socket says 172.21.15.116:36616) 2024-09-12T03:05:28.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.732+0000 7f18affff640 1 -- 172.21.15.116:0/653877463 learned_addr learned my addr 172.21.15.116:0/653877463 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:28.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.732+0000 7f18af7fe640 1 --2- 172.21.15.116:0/653877463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18b0102d30 0x7f18b0198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:28.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.732+0000 7f18aeffd640 1 --2- 172.21.15.116:0/653877463 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18b01036e0 0x7f18b0198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:28.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.732+0000 7f18affff640 1 -- 172.21.15.116:0/653877463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18b0102d30 msgr2=0x7f18b0198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:28.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.732+0000 7f18affff640 1 --2- 172.21.15.116:0/653877463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18b0102d30 0x7f18b0198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.732+0000 7f18affff640 1 -- 172.21.15.116:0/653877463 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18b01036e0 msgr2=0x7f18b0198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:28.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.732+0000 7f18affff640 1 --2- 172.21.15.116:0/653877463 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18b01036e0 0x7f18b0198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.732+0000 7f18affff640 1 -- 172.21.15.116:0/653877463 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b019d540 con 0x7f18b01082a0 2024-09-12T03:05:28.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.732+0000 7f18aeffd640 1 --2- 172.21.15.116:0/653877463 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18b01036e0 0x7f18b0198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:05:28.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.732+0000 7f18affff640 1 --2- 172.21.15.116:0/653877463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18b01082a0 0x7f18b019ce60 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f18a0006010 tx=0x7f18a0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:28.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.732+0000 7f18acff9640 1 -- 172.21.15.116:0/653877463 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a0004450 con 0x7f18b01082a0 2024-09-12T03:05:28.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.732+0000 7f18acff9640 1 -- 172.21.15.116:0/653877463 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18a0003e10 con 0x7f18b01082a0 2024-09-12T03:05:28.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.733+0000 7f18acff9640 1 -- 172.21.15.116:0/653877463 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a00466e0 con 0x7f18b01082a0 2024-09-12T03:05:28.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.733+0000 7f18b5ebc640 1 -- 172.21.15.116:0/653877463 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b019d7d0 con 0x7f18b01082a0 2024-09-12T03:05:28.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.733+0000 7f18b5ebc640 1 -- 172.21.15.116:0/653877463 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b01a5110 con 0x7f18b01082a0 2024-09-12T03:05:28.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.735+0000 7f18acff9640 1 -- 172.21.15.116:0/653877463 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f18a00305a0 con 0x7f18b01082a0 2024-09-12T03:05:28.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.735+0000 7f18b8295740 1 Processor -- start 2024-09-12T03:05:28.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.735+0000 7f18b8295740 1 -- start start 2024-09-12T03:05:28.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.735+0000 7f18b8295740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562bef89fb40 0x562bef89e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:28.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.735+0000 7f18b8295740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562bef89e5d0 0x562bef89ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:28.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.735+0000 7f18b8295740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562bef8ad780 0x562bef8afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:28.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.735+0000 7f18b8295740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562bef8042f0 con 0x562bef89e5d0 2024-09-12T03:05:28.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.735+0000 7f18b8295740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562bef7abcf0 con 0x562bef89fb40 2024-09-12T03:05:28.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.735+0000 7f18b8295740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562bef8002b0 con 0x562bef8ad780 2024-09-12T03:05:28.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.735+0000 7f18af7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562bef89e5d0 0x562bef89ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:28.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.736+0000 7f18af7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562bef89e5d0 0x562bef89ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36628/0 (socket says 172.21.15.116:36628) 2024-09-12T03:05:28.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.736+0000 7f18af7fe640 1 -- 172.21.15.116:0/2334353981 learned_addr learned my addr 172.21.15.116:0/2334353981 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:28.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.736+0000 7f18aeffd640 1 --2- 172.21.15.116:0/2334353981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562bef89fb40 0x562bef89e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:28.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.736+0000 7f18aeffd640 1 --2- 172.21.15.116:0/2334353981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562bef8ad780 0x562bef8afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:28.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.736+0000 7f18af7fe640 1 -- 172.21.15.116:0/2334353981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562bef8ad780 msgr2=0x562bef8afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:28.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.736+0000 7f18af7fe640 1 --2- 172.21.15.116:0/2334353981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562bef8ad780 0x562bef8afb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.736+0000 7f18af7fe640 1 -- 172.21.15.116:0/2334353981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562bef89fb40 msgr2=0x562bef89e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:28.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.736+0000 7f18af7fe640 1 --2- 172.21.15.116:0/2334353981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562bef89fb40 0x562bef89e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.736+0000 7f18af7fe640 1 -- 172.21.15.116:0/2334353981 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bef7903a0 con 0x562bef89e5d0 2024-09-12T03:05:28.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.736+0000 7f18aeffd640 1 --2- 172.21.15.116:0/2334353981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562bef89fb40 0x562bef89e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:28.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.736+0000 7f18af7fe640 1 --2- 172.21.15.116:0/2334353981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562bef89e5d0 0x562bef89ea10 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f18a800b820 tx=0x7f18a800bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a161cf4db35d47a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:28.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.736+0000 7f18acff9640 1 --2- 172.21.15.116:0/653877463 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f189c0a2b80 0x7f189c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:28.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.736+0000 7f18a5ffb640 1 -- 172.21.15.116:0/2334353981 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a800d950 con 0x562bef89e5d0 2024-09-12T03:05:28.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.736+0000 7f18a5ffb640 1 -- 172.21.15.116:0/2334353981 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18a8019070 con 0x562bef89e5d0 2024-09-12T03:05:28.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.736+0000 7f18aeffd640 1 --2- 172.21.15.116:0/653877463 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f189c0a2b80 0x7f189c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:28.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.736+0000 7f18a5ffb640 1 -- 172.21.15.116:0/2334353981 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a80123e0 con 0x562bef89e5d0 2024-09-12T03:05:28.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.736+0000 7f18acff9640 1 -- 172.21.15.116:0/653877463 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f18a002e030 con 0x7f18b01082a0 2024-09-12T03:05:28.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.737+0000 7f18b8295740 1 -- 172.21.15.116:0/2334353981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562bef89e5d0 msgr2=0x562bef89ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:28.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.737+0000 7f18b8295740 1 --2- 172.21.15.116:0/2334353981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562bef89e5d0 0x562bef89ea10 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f18a800b820 tx=0x7f18a800bce0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.737+0000 7f18aeffd640 1 --2- 172.21.15.116:0/653877463 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f189c0a2b80 0x7f189c0a5040 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1898005fd0 tx=0x7f189800d360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:28.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.737+0000 7f18b8295740 1 -- 172.21.15.116:0/2334353981 shutdown_connections 2024-09-12T03:05:28.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.737+0000 7f18b8295740 1 --2- 172.21.15.116:0/2334353981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562bef8ad780 0x562bef8afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.737+0000 7f18b8295740 1 --2- 172.21.15.116:0/2334353981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562bef89e5d0 0x562bef89ea10 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.737+0000 7f18b8295740 1 --2- 172.21.15.116:0/2334353981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562bef89fb40 0x562bef89e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.737+0000 7f18b8295740 1 -- 172.21.15.116:0/2334353981 >> 172.21.15.116:0/2334353981 conn(0x562bef867310 msgr2=0x562bef89ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:28.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.737+0000 7f18b8295740 1 -- 172.21.15.116:0/2334353981 shutdown_connections 2024-09-12T03:05:28.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.737+0000 7f18b8295740 1 -- 172.21.15.116:0/2334353981 wait complete. 2024-09-12T03:05:28.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.737+0000 7f18b8295740 1 -- start start 2024-09-12T03:05:28.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.738+0000 7f18b8295740 1 Processor -- start 2024-09-12T03:05:28.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.738+0000 7f18b8295740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562bef89e5d0 0x562bef8ad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:28.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.738+0000 7f18b8295740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562bef89fb40 0x562bef8aa380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:28.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.738+0000 7f18b8295740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562bef8ad780 0x562bef8aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:28.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.738+0000 7f18b8295740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562bef8b0480 con 0x562bef89fb40 2024-09-12T03:05:28.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.738+0000 7f18b8295740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562bef7bf060 con 0x562bef89e5d0 2024-09-12T03:05:28.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.738+0000 7f18b8295740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562bef8b0600 con 0x562bef8ad780 2024-09-12T03:05:28.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.738+0000 7f18b8295740 10 client.0 _ll_register_callbacks cb 0x562bef5128c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:28.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.738+0000 7f18aeffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562bef89fb40 0x562bef8aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:28.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.738+0000 7f18aeffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562bef89fb40 0x562bef8aa380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36640/0 (socket says 172.21.15.116:36640) 2024-09-12T03:05:28.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.738+0000 7f18aeffd640 1 -- 172.21.15.116:0/1377667065 learned_addr learned my addr 172.21.15.116:0/1377667065 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:28.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.738+0000 7f18af7fe640 1 --2- 172.21.15.116:0/1377667065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562bef89e5d0 0x562bef8ad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:28.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.738+0000 7f18aeffd640 1 -- 172.21.15.116:0/1377667065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562bef8ad780 msgr2=0x562bef8aa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:28.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.738+0000 7f18aeffd640 1 --2- 172.21.15.116:0/1377667065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562bef8ad780 0x562bef8aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.738+0000 7f18aeffd640 1 -- 172.21.15.116:0/1377667065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562bef89e5d0 msgr2=0x562bef8ad0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:28.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.738+0000 7f18aeffd640 1 --2- 172.21.15.116:0/1377667065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562bef89e5d0 0x562bef8ad0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.738+0000 7f18aeffd640 1 -- 172.21.15.116:0/1377667065 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bef5753c0 con 0x562bef89fb40 2024-09-12T03:05:28.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.739+0000 7f18aeffd640 1 --2- 172.21.15.116:0/1377667065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562bef89fb40 0x562bef8aa380 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f189800f920 tx=0x7f1898005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5de569d189acc3f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:28.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.739+0000 7f188effd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:05:28.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.739+0000 7f188effd640 1 -- 172.21.15.116:0/1377667065 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1898018070 con 0x562bef89fb40 2024-09-12T03:05:28.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.739+0000 7f188effd640 1 -- 172.21.15.116:0/1377667065 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1898010d60 con 0x562bef89fb40 2024-09-12T03:05:28.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.739+0000 7f18b8295740 1 -- 172.21.15.116:0/1377667065 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18a8019070 con 0x562bef89fb40 2024-09-12T03:05:28.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.739+0000 7f188effd640 1 -- 172.21.15.116:0/1377667065 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f189802e070 con 0x562bef89fb40 2024-09-12T03:05:28.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.739+0000 7f18b8295740 10 client.15840 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:28.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.739+0000 7f18b8295740 1 -- 172.21.15.116:0/1377667065 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562bef8bbec0 con 0x562bef89fb40 2024-09-12T03:05:28.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.739+0000 7f188effd640 1 -- 172.21.15.116:0/1377667065 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18a8019070 con 0x562bef89fb40 2024-09-12T03:05:28.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.739+0000 7f188effd640 1 -- 172.21.15.116:0/1377667065 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f189801d4e0 con 0x562bef89fb40 2024-09-12T03:05:28.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.739+0000 7f18b8295740 10 client.15840 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:28.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.739+0000 7f18b8295740 10 client.15840 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:05:28.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.739+0000 7f18b8295740 20 client.15840 populate_metadata read hostname 'smithi116' 2024-09-12T03:05:28.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.739+0000 7f18b8295740 1 --2- 172.21.15.116:0/1377667065 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x562bef8bd3d0 0x562bef8bf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:28.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.739+0000 7f18b8295740 4 client.15840 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:05:28.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.739+0000 7f18b8295740 1 -- 172.21.15.116:0/1377667065 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bef811860 con 0x562bef8bd3d0 2024-09-12T03:05:28.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.740+0000 7f18af7fe640 1 --2- 172.21.15.116:0/1377667065 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x562bef8bd3d0 0x562bef8bf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:28.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.740+0000 7f188effd640 10 client.15840 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:05:28.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.740+0000 7f18af7fe640 1 --2- 172.21.15.116:0/1377667065 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x562bef8bd3d0 0x562bef8bf890 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4066ed54c9342095 server_cookie=ba75058547a9b58a in_seq=0 out_seq=0 2024-09-12T03:05:28.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.741+0000 7f188effd640 1 -- 172.21.15.116:0/1377667065 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562bef811860 con 0x562bef8bd3d0 2024-09-12T03:05:28.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.741+0000 7f188effd640 10 client.15840 handle_command_reply: tid=0 2024-09-12T03:05:28.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.741+0000 7f18b8295740 1 client.15840 shutdown 2024-09-12T03:05:28.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.741+0000 7f18b8295740 1 -- 172.21.15.116:0/1377667065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562bef89fb40 msgr2=0x562bef8aa380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:28.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.741+0000 7f18b8295740 1 --2- 172.21.15.116:0/1377667065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562bef89fb40 0x562bef8aa380 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f189800f920 tx=0x7f1898005b70 comp rx=0 tx=0).stop 2024-09-12T03:05:28.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.741+0000 7f18b8295740 1 -- 172.21.15.116:0/1377667065 shutdown_connections 2024-09-12T03:05:28.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.741+0000 7f18b8295740 1 --2- 172.21.15.116:0/1377667065 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x562bef8bd3d0 0x562bef8bf890 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.741+0000 7f18b8295740 1 --2- 172.21.15.116:0/1377667065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562bef8ad780 0x562bef8aa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.741+0000 7f18b8295740 1 --2- 172.21.15.116:0/1377667065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562bef89fb40 0x562bef8aa380 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.741+0000 7f18b8295740 1 --2- 172.21.15.116:0/1377667065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562bef89e5d0 0x562bef8ad0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.741+0000 7f188effd640 0 client.15840 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:05:28.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.741+0000 7f18b8295740 1 -- 172.21.15.116:0/1377667065 >> 172.21.15.116:0/1377667065 conn(0x562bef867310 msgr2=0x562bef8a0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:28.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.741+0000 7f18b8295740 1 -- 172.21.15.116:0/1377667065 shutdown_connections 2024-09-12T03:05:28.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.741+0000 7f18b8295740 1 -- 172.21.15.116:0/1377667065 wait complete. 2024-09-12T03:05:28.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.741+0000 7f18b8295740 20 client.15840 trim_cache size 0 max 16384 2024-09-12T03:05:28.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.754+0000 7f18b8295740 1 Processor -- start 2024-09-12T03:05:28.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.754+0000 7f18b8295740 1 -- start start 2024-09-12T03:05:28.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.754+0000 7f18b8295740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562bef89e5d0 0x562bef89ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:28.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.754+0000 7f18b8295740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562bef89fb40 0x562bef8b9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:28.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.754+0000 7f18b8295740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562bef8ad780 0x562bef8c5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:28.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.754+0000 7f18b8295740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01adb60 con 0x562bef89fb40 2024-09-12T03:05:28.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.754+0000 7f18b8295740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01ab430 con 0x562bef8ad780 2024-09-12T03:05:28.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.754+0000 7f18b8295740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01a92a0 con 0x562bef89e5d0 2024-09-12T03:05:28.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.754+0000 7f18aeffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562bef89fb40 0x562bef8b9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:28.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.754+0000 7f18aeffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562bef89fb40 0x562bef8b9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36656/0 (socket says 172.21.15.116:36656) 2024-09-12T03:05:28.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.754+0000 7f18aeffd640 1 -- 172.21.15.116:0/3755485425 learned_addr learned my addr 172.21.15.116:0/3755485425 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:28.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.755+0000 7f18affff640 1 --2- 172.21.15.116:0/3755485425 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562bef8ad780 0x562bef8c5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:28.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.755+0000 7f18af7fe640 1 --2- 172.21.15.116:0/3755485425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562bef89e5d0 0x562bef89ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:28.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.755+0000 7f18aeffd640 1 -- 172.21.15.116:0/3755485425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562bef89e5d0 msgr2=0x562bef89ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:28.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.755+0000 7f18aeffd640 1 --2- 172.21.15.116:0/3755485425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562bef89e5d0 0x562bef89ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.755+0000 7f18aeffd640 1 -- 172.21.15.116:0/3755485425 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562bef8ad780 msgr2=0x562bef8c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:28.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.755+0000 7f18aeffd640 1 --2- 172.21.15.116:0/3755485425 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562bef8ad780 0x562bef8c5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.755+0000 7f18aeffd640 1 -- 172.21.15.116:0/3755485425 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bef8a1120 con 0x562bef89fb40 2024-09-12T03:05:28.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.755+0000 7f18aeffd640 1 --2- 172.21.15.116:0/3755485425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562bef89fb40 0x562bef8b9750 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f189801d950 tx=0x7f189801dfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a19b977b55a6e4d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:28.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.755+0000 7f18a57fa640 1 -- 172.21.15.116:0/3755485425 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1898018070 con 0x562bef89fb40 2024-09-12T03:05:28.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.755+0000 7f18a57fa640 1 -- 172.21.15.116:0/3755485425 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18980107a0 con 0x562bef89fb40 2024-09-12T03:05:28.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.755+0000 7f18a57fa640 1 -- 172.21.15.116:0/3755485425 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1898033a50 con 0x562bef89fb40 2024-09-12T03:05:28.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.756+0000 7f18b8295740 1 -- 172.21.15.116:0/3755485425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562bef89fb40 msgr2=0x562bef8b9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:28.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.756+0000 7f18b8295740 1 --2- 172.21.15.116:0/3755485425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562bef89fb40 0x562bef8b9750 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f189801d950 tx=0x7f189801dfd0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.756+0000 7f18b8295740 1 -- 172.21.15.116:0/3755485425 shutdown_connections 2024-09-12T03:05:28.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.756+0000 7f18b8295740 1 --2- 172.21.15.116:0/3755485425 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562bef8ad780 0x562bef8c5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.756+0000 7f18b8295740 1 --2- 172.21.15.116:0/3755485425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562bef89fb40 0x562bef8b9750 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.756+0000 7f18b8295740 1 --2- 172.21.15.116:0/3755485425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562bef89e5d0 0x562bef89ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.756+0000 7f18b8295740 1 -- 172.21.15.116:0/3755485425 >> 172.21.15.116:0/3755485425 conn(0x562bef867310 msgr2=0x562bef89a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:28.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.756+0000 7f18b8295740 1 -- 172.21.15.116:0/3755485425 shutdown_connections 2024-09-12T03:05:28.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.756+0000 7f18b8295740 1 -- 172.21.15.116:0/3755485425 wait complete. 2024-09-12T03:05:28.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.756+0000 7f18b8295740 1 -- start start 2024-09-12T03:05:28.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.756+0000 7f18b8295740 1 Processor -- start 2024-09-12T03:05:28.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.757+0000 7f18b8295740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562bef89e5d0 0x562bef8a73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:28.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.757+0000 7f18b8295740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562bef89fb40 0x562bef8a78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:28.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.757+0000 7f18b8295740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562bef8ad780 0x562bef8a7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:28.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.757+0000 7f18b8295740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01a98f0 con 0x562bef89fb40 2024-09-12T03:05:28.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.757+0000 7f18b8295740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01aad60 con 0x562bef8ad780 2024-09-12T03:05:28.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.757+0000 7f18b8295740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01a9fc0 con 0x562bef89e5d0 2024-09-12T03:05:28.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.757+0000 7f18b8295740 10 client.0 _ll_register_callbacks cb 0x7f189800f5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:28.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.757+0000 7f18aeffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562bef89fb40 0x562bef8a78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:28.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.757+0000 7f18aeffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562bef89fb40 0x562bef8a78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36666/0 (socket says 172.21.15.116:36666) 2024-09-12T03:05:28.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.757+0000 7f18aeffd640 1 -- 172.21.15.116:0/3196770847 learned_addr learned my addr 172.21.15.116:0/3196770847 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:28.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.757+0000 7f18af7fe640 1 --2- 172.21.15.116:0/3196770847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562bef89e5d0 0x562bef8a73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:28.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.757+0000 7f18aeffd640 1 -- 172.21.15.116:0/3196770847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562bef89e5d0 msgr2=0x562bef8a73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:28.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.757+0000 7f18aeffd640 1 --2- 172.21.15.116:0/3196770847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562bef89e5d0 0x562bef8a73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.757+0000 7f18aeffd640 1 -- 172.21.15.116:0/3196770847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562bef8ad780 msgr2=0x562bef8a7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:28.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.757+0000 7f18aeffd640 1 --2- 172.21.15.116:0/3196770847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562bef8ad780 0x562bef8a7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.757+0000 7f18aeffd640 1 -- 172.21.15.116:0/3196770847 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bef927130 con 0x562bef89fb40 2024-09-12T03:05:28.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.757+0000 7f18aeffd640 1 --2- 172.21.15.116:0/3196770847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562bef89fb40 0x562bef8a78f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f189800f920 tx=0x7f189801de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e21e6b30edbf18b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:28.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.757+0000 7f188effd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:05:28.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.757+0000 7f188effd640 1 -- 172.21.15.116:0/3196770847 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1898018040 con 0x562bef89fb40 2024-09-12T03:05:28.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.757+0000 7f188effd640 1 -- 172.21.15.116:0/3196770847 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1898010d60 con 0x562bef89fb40 2024-09-12T03:05:28.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.757+0000 7f18b8295740 1 -- 172.21.15.116:0/3196770847 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18980107a0 con 0x562bef89fb40 2024-09-12T03:05:28.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.757+0000 7f188effd640 1 -- 172.21.15.116:0/3196770847 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f189801d6d0 con 0x562bef89fb40 2024-09-12T03:05:28.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.758+0000 7f18b8295740 10 client.15846 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:28.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.758+0000 7f18b8295740 1 -- 172.21.15.116:0/3196770847 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562bef9283f0 con 0x562bef89fb40 2024-09-12T03:05:28.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.758+0000 7f188effd640 1 -- 172.21.15.116:0/3196770847 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18980107a0 con 0x562bef89fb40 2024-09-12T03:05:28.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.758+0000 7f188effd640 1 -- 172.21.15.116:0/3196770847 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f189803b020 con 0x562bef89fb40 2024-09-12T03:05:28.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.758+0000 7f18b8295740 10 client.15846 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:28.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.758+0000 7f18b8295740 10 client.15846 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:05:28.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.758+0000 7f18b8295740 20 client.15846 populate_metadata read hostname 'smithi116' 2024-09-12T03:05:28.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.758+0000 7f18b8295740 1 --2- 172.21.15.116:0/3196770847 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x562bef928aa0 0x562bef92aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:28.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.758+0000 7f18b8295740 4 client.15846 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:05:28.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.758+0000 7f18b8295740 1 -- 172.21.15.116:0/3196770847 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562bef80bd70 con 0x562bef928aa0 2024-09-12T03:05:28.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.758+0000 7f18af7fe640 1 --2- 172.21.15.116:0/3196770847 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x562bef928aa0 0x562bef92aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:28.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.758+0000 7f18af7fe640 1 --2- 172.21.15.116:0/3196770847 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x562bef928aa0 0x562bef92aec0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4aa60b638bd4886e server_cookie=bf17e6c778039d5f in_seq=0 out_seq=0 2024-09-12T03:05:28.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.758+0000 7f188effd640 10 client.15846 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:05:28.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.759+0000 7f188effd640 1 -- 172.21.15.116:0/3196770847 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562bef80bd70 con 0x562bef928aa0 2024-09-12T03:05:28.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.759+0000 7f188effd640 10 client.15846 handle_command_reply: tid=0 2024-09-12T03:05:28.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.759+0000 7f18b8295740 1 client.15846 shutdown 2024-09-12T03:05:28.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.759+0000 7f18b8295740 1 -- 172.21.15.116:0/3196770847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562bef89fb40 msgr2=0x562bef8a78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:28.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.759+0000 7f18b8295740 1 --2- 172.21.15.116:0/3196770847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562bef89fb40 0x562bef8a78f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f189800f920 tx=0x7f189801de60 comp rx=0 tx=0).stop 2024-09-12T03:05:28.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.759+0000 7f18b8295740 1 -- 172.21.15.116:0/3196770847 shutdown_connections 2024-09-12T03:05:28.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.759+0000 7f18b8295740 1 --2- 172.21.15.116:0/3196770847 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x562bef928aa0 0x562bef92aec0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.759+0000 7f18b8295740 1 --2- 172.21.15.116:0/3196770847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562bef8ad780 0x562bef8a7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.759+0000 7f18b8295740 1 --2- 172.21.15.116:0/3196770847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562bef89fb40 0x562bef8a78f0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.759+0000 7f18b8295740 1 --2- 172.21.15.116:0/3196770847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562bef89e5d0 0x562bef8a73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.759+0000 7f18b8295740 1 -- 172.21.15.116:0/3196770847 >> 172.21.15.116:0/3196770847 conn(0x562bef867310 msgr2=0x562bef8abbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:28.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.759+0000 7f188effd640 0 client.15846 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:05:28.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.759+0000 7f18b8295740 1 -- 172.21.15.116:0/3196770847 shutdown_connections 2024-09-12T03:05:28.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.759+0000 7f18b8295740 1 -- 172.21.15.116:0/3196770847 wait complete. 2024-09-12T03:05:28.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.759+0000 7f18b8295740 20 client.15846 trim_cache size 0 max 16384 2024-09-12T03:05:28.770 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:05:28.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.760+0000 7f18b5ebc640 1 -- 172.21.15.116:0/653877463 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f189c0a2b80 msgr2=0x7f189c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:28.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.760+0000 7f18b5ebc640 1 --2- 172.21.15.116:0/653877463 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f189c0a2b80 0x7f189c0a5040 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1898005fd0 tx=0x7f189800d360 comp rx=0 tx=0).stop 2024-09-12T03:05:28.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.760+0000 7f18b5ebc640 1 -- 172.21.15.116:0/653877463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18b01082a0 msgr2=0x7f18b019ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:28.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.760+0000 7f18b5ebc640 1 --2- 172.21.15.116:0/653877463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18b01082a0 0x7f18b019ce60 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f18a0006010 tx=0x7f18a0003c20 comp rx=0 tx=0).stop 2024-09-12T03:05:28.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.760+0000 7f18b5ebc640 1 -- 172.21.15.116:0/653877463 shutdown_connections 2024-09-12T03:05:28.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.760+0000 7f18b5ebc640 1 --2- 172.21.15.116:0/653877463 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f189c0a2b80 0x7f189c0a5040 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.760+0000 7f18b5ebc640 1 --2- 172.21.15.116:0/653877463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18b01082a0 0x7f18b019ce60 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.760+0000 7f18b5ebc640 1 --2- 172.21.15.116:0/653877463 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18b01036e0 0x7f18b0198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.760+0000 7f18b5ebc640 1 --2- 172.21.15.116:0/653877463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18b0102d30 0x7f18b0198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.760+0000 7f18b5ebc640 1 -- 172.21.15.116:0/653877463 >> 172.21.15.116:0/653877463 conn(0x7f18b00fc7d0 msgr2=0x7f18b0105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:28.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.760+0000 7f18b5ebc640 1 -- 172.21.15.116:0/653877463 shutdown_connections 2024-09-12T03:05:28.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:28.760+0000 7f18b5ebc640 1 -- 172.21.15.116:0/653877463 wait complete. 2024-09-12T03:05:29.715 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:05:29.963 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:30.342 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:30 smithi116 ceph-mon[31410]: pgmap v413: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:05:30.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.425+0000 7f94baabb640 1 -- 172.21.15.116:0/2439208896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94b4100a80 msgr2=0x7f94b4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:30.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.425+0000 7f94baabb640 1 --2- 172.21.15.116:0/2439208896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94b4100a80 0x7f94b4100ee0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009930 tx=0x7f94a002a420 comp rx=0 tx=0).stop 2024-09-12T03:05:30.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.425+0000 7f94baabb640 1 -- 172.21.15.116:0/2439208896 shutdown_connections 2024-09-12T03:05:30.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.425+0000 7f94baabb640 1 --2- 172.21.15.116:0/2439208896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94b4101420 0x7f94b410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.425+0000 7f94baabb640 1 --2- 172.21.15.116:0/2439208896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94b4100a80 0x7f94b4100ee0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.425+0000 7f94baabb640 1 --2- 172.21.15.116:0/2439208896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94b4106a80 0x7f94b4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.425+0000 7f94baabb640 1 -- 172.21.15.116:0/2439208896 >> 172.21.15.116:0/2439208896 conn(0x7f94b40fc7a0 msgr2=0x7f94b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:30.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.425+0000 7f94baabb640 1 -- 172.21.15.116:0/2439208896 shutdown_connections 2024-09-12T03:05:30.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.425+0000 7f94baabb640 1 -- 172.21.15.116:0/2439208896 wait complete. 2024-09-12T03:05:30.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.426+0000 7f94baabb640 1 Processor -- start 2024-09-12T03:05:30.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.426+0000 7f94baabb640 1 -- start start 2024-09-12T03:05:30.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.426+0000 7f94baabb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94b4100a80 0x7f94b41992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.426+0000 7f94baabb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94b4101420 0x7f94b4199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.426+0000 7f94baabb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94b4106a80 0x7f94b41a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.426+0000 7f94baabb640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f94b4110ea0 con 0x7f94b4106a80 2024-09-12T03:05:30.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.426+0000 7f94baabb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f94b4110d20 con 0x7f94b4100a80 2024-09-12T03:05:30.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.426+0000 7f94baabb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f94b4111020 con 0x7f94b4101420 2024-09-12T03:05:30.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.426+0000 7f94b9058640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94b4106a80 0x7f94b41a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.426+0000 7f94b9058640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94b4106a80 0x7f94b41a4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36686/0 (socket says 172.21.15.116:36686) 2024-09-12T03:05:30.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.426+0000 7f94b9058640 1 -- 172.21.15.116:0/3068735811 learned_addr learned my addr 172.21.15.116:0/3068735811 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:30.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.426+0000 7f94b8857640 1 --2- 172.21.15.116:0/3068735811 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94b4100a80 0x7f94b41992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.426+0000 7f94b3fff640 1 --2- 172.21.15.116:0/3068735811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94b4101420 0x7f94b4199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.426+0000 7f94b9058640 1 -- 172.21.15.116:0/3068735811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94b4101420 msgr2=0x7f94b4199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:30.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.426+0000 7f94b9058640 1 --2- 172.21.15.116:0/3068735811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94b4101420 0x7f94b4199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.426+0000 7f94b9058640 1 -- 172.21.15.116:0/3068735811 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94b4100a80 msgr2=0x7f94b41992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:30.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.426+0000 7f94b9058640 1 --2- 172.21.15.116:0/3068735811 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94b4100a80 0x7f94b41992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.426+0000 7f94b9058640 1 -- 172.21.15.116:0/3068735811 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b41a48b0 con 0x7f94b4106a80 2024-09-12T03:05:30.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.426+0000 7f94b8857640 1 --2- 172.21.15.116:0/3068735811 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94b4100a80 0x7f94b41992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:30.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.427+0000 7f94b9058640 1 --2- 172.21.15.116:0/3068735811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94b4106a80 0x7f94b41a4370 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00cce0 tx=0x7f94ac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:30.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.427+0000 7f94b1ffb640 1 -- 172.21.15.116:0/3068735811 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac00f450 con 0x7f94b4106a80 2024-09-12T03:05:30.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.427+0000 7f94b1ffb640 1 -- 172.21.15.116:0/3068735811 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94ac00fad0 con 0x7f94b4106a80 2024-09-12T03:05:30.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.427+0000 7f94baabb640 1 -- 172.21.15.116:0/3068735811 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94b41a4a80 con 0x7f94b4106a80 2024-09-12T03:05:30.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.427+0000 7f94b1ffb640 1 -- 172.21.15.116:0/3068735811 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac018c10 con 0x7f94b4106a80 2024-09-12T03:05:30.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.427+0000 7f94baabb640 1 -- 172.21.15.116:0/3068735811 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94b41a4f40 con 0x7f94b4106a80 2024-09-12T03:05:30.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.429+0000 7f94b1ffb640 1 -- 172.21.15.116:0/3068735811 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f94ac00f5f0 con 0x7f94b4106a80 2024-09-12T03:05:30.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.430+0000 7f94bce94740 1 Processor -- start 2024-09-12T03:05:30.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.430+0000 7f94bce94740 1 -- start start 2024-09-12T03:05:30.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.430+0000 7f94b1ffb640 1 --2- 172.21.15.116:0/3068735811 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f94a40a2c00 0x7f94a40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.430+0000 7f94bce94740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bf022e0b40 0x55bf022df090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.430+0000 7f94bce94740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf022df5d0 0x55bf022dfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.430+0000 7f94bce94740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf022ea370 0x55bf022ec760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.430+0000 7f94bce94740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bf022452f0 con 0x55bf022df5d0 2024-09-12T03:05:30.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.430+0000 7f94bce94740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bf021eccf0 con 0x55bf022e0b40 2024-09-12T03:05:30.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.430+0000 7f94bce94740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bf022412b0 con 0x55bf022ea370 2024-09-12T03:05:30.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.430+0000 7f94b1ffb640 1 -- 172.21.15.116:0/3068735811 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f94ac0cf700 con 0x7f94b4106a80 2024-09-12T03:05:30.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.430+0000 7f94b3fff640 1 --2- 172.21.15.116:0/3068735811 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f94a40a2c00 0x7f94a40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.430+0000 7f94b3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf022df5d0 0x55bf022dfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.430+0000 7f94b3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf022df5d0 0x55bf022dfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36690/0 (socket says 172.21.15.116:36690) 2024-09-12T03:05:30.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.430+0000 7f94b3fff640 1 -- 172.21.15.116:0/2203956750 learned_addr learned my addr 172.21.15.116:0/2203956750 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:30.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.430+0000 7f94b8857640 1 --2- 172.21.15.116:0/2203956750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bf022e0b40 0x55bf022df090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.430+0000 7f94b3fff640 1 -- 172.21.15.116:0/2203956750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf022ea370 msgr2=0x55bf022ec760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:30.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.430+0000 7f94b3fff640 1 --2- 172.21.15.116:0/2203956750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf022ea370 0x55bf022ec760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.430+0000 7f94b3fff640 1 -- 172.21.15.116:0/2203956750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bf022e0b40 msgr2=0x55bf022df090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:30.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.430+0000 7f94b3fff640 1 --2- 172.21.15.116:0/2203956750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bf022e0b40 0x55bf022df090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.430+0000 7f94b3fff640 1 -- 172.21.15.116:0/2203956750 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf021d13a0 con 0x55bf022df5d0 2024-09-12T03:05:30.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.431+0000 7f94b3fff640 1 --2- 172.21.15.116:0/2203956750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf022df5d0 0x55bf022dfa10 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f94a0031550 tx=0x7f94a0004140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20e4134a38070b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:30.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.431+0000 7f94b3fff640 1 --2- 172.21.15.116:0/3068735811 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f94a40a2c00 0x7f94a40a50c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f94a0005ab0 tx=0x7f94a0040000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:30.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.431+0000 7f949effd640 1 -- 172.21.15.116:0/2203956750 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0035040 con 0x55bf022df5d0 2024-09-12T03:05:30.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.431+0000 7f949effd640 1 -- 172.21.15.116:0/2203956750 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a004d070 con 0x55bf022df5d0 2024-09-12T03:05:30.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.431+0000 7f949effd640 1 -- 172.21.15.116:0/2203956750 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0003e20 con 0x55bf022df5d0 2024-09-12T03:05:30.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.431+0000 7f94bce94740 1 -- 172.21.15.116:0/2203956750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf022df5d0 msgr2=0x55bf022dfa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:30.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.431+0000 7f94bce94740 1 --2- 172.21.15.116:0/2203956750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf022df5d0 0x55bf022dfa10 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f94a0031550 tx=0x7f94a0004140 comp rx=0 tx=0).stop 2024-09-12T03:05:30.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.431+0000 7f94bce94740 1 -- 172.21.15.116:0/2203956750 shutdown_connections 2024-09-12T03:05:30.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.431+0000 7f94bce94740 1 --2- 172.21.15.116:0/2203956750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf022ea370 0x55bf022ec760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.432+0000 7f94bce94740 1 --2- 172.21.15.116:0/2203956750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf022df5d0 0x55bf022dfa10 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.432+0000 7f94bce94740 1 --2- 172.21.15.116:0/2203956750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bf022e0b40 0x55bf022df090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.432+0000 7f94bce94740 1 -- 172.21.15.116:0/2203956750 >> 172.21.15.116:0/2203956750 conn(0x55bf022a8310 msgr2=0x55bf022dbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:30.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.432+0000 7f94bce94740 1 -- 172.21.15.116:0/2203956750 shutdown_connections 2024-09-12T03:05:30.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.432+0000 7f94bce94740 1 -- 172.21.15.116:0/2203956750 wait complete. 2024-09-12T03:05:30.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.432+0000 7f94bce94740 1 -- start start 2024-09-12T03:05:30.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.432+0000 7f94bce94740 1 Processor -- start 2024-09-12T03:05:30.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.432+0000 7f94bce94740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf022df5d0 0x55bf022e9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.432+0000 7f94bce94740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bf022e0b40 0x55bf022e7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.432+0000 7f94bce94740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf022ea370 0x55bf022e77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.432+0000 7f94bce94740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bf022ed010 con 0x55bf022df5d0 2024-09-12T03:05:30.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.432+0000 7f94bce94740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bf02200060 con 0x55bf022e0b40 2024-09-12T03:05:30.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.432+0000 7f94bce94740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bf022ed190 con 0x55bf022ea370 2024-09-12T03:05:30.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.432+0000 7f94bce94740 10 client.0 _ll_register_callbacks cb 0x55bf01f538c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:30.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.432+0000 7f94b8857640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf022df5d0 0x55bf022e9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.432+0000 7f94b8857640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf022df5d0 0x55bf022e9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36698/0 (socket says 172.21.15.116:36698) 2024-09-12T03:05:30.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.432+0000 7f94b8857640 1 -- 172.21.15.116:0/1881216623 learned_addr learned my addr 172.21.15.116:0/1881216623 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:30.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.433+0000 7f94b9058640 1 --2- 172.21.15.116:0/1881216623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf022ea370 0x55bf022e77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.433+0000 7f94b3fff640 1 --2- 172.21.15.116:0/1881216623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bf022e0b40 0x55bf022e7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.433+0000 7f94b8857640 1 -- 172.21.15.116:0/1881216623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf022ea370 msgr2=0x55bf022e77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:30.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.433+0000 7f94b8857640 1 --2- 172.21.15.116:0/1881216623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf022ea370 0x55bf022e77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.433+0000 7f94b8857640 1 -- 172.21.15.116:0/1881216623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bf022e0b40 msgr2=0x55bf022e7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:30.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.433+0000 7f94b8857640 1 --2- 172.21.15.116:0/1881216623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bf022e0b40 0x55bf022e7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.433+0000 7f94b8857640 1 -- 172.21.15.116:0/1881216623 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf01fb63c0 con 0x55bf022df5d0 2024-09-12T03:05:30.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.433+0000 7f94b8857640 1 --2- 172.21.15.116:0/1881216623 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf022df5d0 0x55bf022e9db0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f94a8010500 tx=0x7f94a8010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5d863b62bc938 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:30.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.433+0000 7f9493fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:05:30.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.433+0000 7f9493fff640 1 -- 172.21.15.116:0/1881216623 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a8018070 con 0x55bf022df5d0 2024-09-12T03:05:30.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.433+0000 7f9493fff640 1 -- 172.21.15.116:0/1881216623 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f94a8014640 con 0x55bf022df5d0 2024-09-12T03:05:30.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.433+0000 7f94bce94740 1 -- 172.21.15.116:0/1881216623 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94a004d070 con 0x55bf022df5d0 2024-09-12T03:05:30.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.433+0000 7f9493fff640 1 -- 172.21.15.116:0/1881216623 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f94a8024050 con 0x55bf022df5d0 2024-09-12T03:05:30.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.433+0000 7f94bce94740 10 client.15858 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:30.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.433+0000 7f94bce94740 1 -- 172.21.15.116:0/1881216623 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bf022f8bc0 con 0x55bf022df5d0 2024-09-12T03:05:30.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.433+0000 7f9493fff640 1 -- 172.21.15.116:0/1881216623 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a004d070 con 0x55bf022df5d0 2024-09-12T03:05:30.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.434+0000 7f9493fff640 1 -- 172.21.15.116:0/1881216623 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f94a80129f0 con 0x55bf022df5d0 2024-09-12T03:05:30.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.434+0000 7f94bce94740 10 client.15858 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:30.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.434+0000 7f94bce94740 10 client.15858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:05:30.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.434+0000 7f94bce94740 20 client.15858 populate_metadata read hostname 'smithi116' 2024-09-12T03:05:30.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.434+0000 7f94bce94740 1 --2- 172.21.15.116:0/1881216623 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bf022fa020 0x55bf022fc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.434+0000 7f94bce94740 4 client.15858 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:05:30.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.434+0000 7f94bce94740 1 -- 172.21.15.116:0/1881216623 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf02252860 con 0x55bf022fa020 2024-09-12T03:05:30.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.434+0000 7f94b3fff640 1 --2- 172.21.15.116:0/1881216623 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bf022fa020 0x55bf022fc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.434+0000 7f9493fff640 10 client.15858 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:30.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.434+0000 7f94b3fff640 1 --2- 172.21.15.116:0/1881216623 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bf022fa020 0x55bf022fc470 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d2d54b6e175ea8b server_cookie=93d2123fba1269ac in_seq=0 out_seq=0 2024-09-12T03:05:30.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.435+0000 7f9493fff640 1 -- 172.21.15.116:0/1881216623 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf02252860 con 0x55bf022fa020 2024-09-12T03:05:30.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.435+0000 7f9493fff640 10 client.15858 handle_command_reply: tid=0 2024-09-12T03:05:30.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.435+0000 7f94bce94740 1 client.15858 shutdown 2024-09-12T03:05:30.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.435+0000 7f94bce94740 1 -- 172.21.15.116:0/1881216623 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf022df5d0 msgr2=0x55bf022e9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:30.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.435+0000 7f94bce94740 1 --2- 172.21.15.116:0/1881216623 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf022df5d0 0x55bf022e9db0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f94a8010500 tx=0x7f94a8010900 comp rx=0 tx=0).stop 2024-09-12T03:05:30.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.435+0000 7f94bce94740 1 -- 172.21.15.116:0/1881216623 shutdown_connections 2024-09-12T03:05:30.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.435+0000 7f94bce94740 1 --2- 172.21.15.116:0/1881216623 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bf022fa020 0x55bf022fc470 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.435+0000 7f94bce94740 1 --2- 172.21.15.116:0/1881216623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf022ea370 0x55bf022e77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.435+0000 7f94bce94740 1 --2- 172.21.15.116:0/1881216623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bf022e0b40 0x55bf022e7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.435+0000 7f9493fff640 0 client.15858 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:30.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.435+0000 7f94bce94740 1 --2- 172.21.15.116:0/1881216623 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf022df5d0 0x55bf022e9db0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.435+0000 7f94bce94740 1 -- 172.21.15.116:0/1881216623 >> 172.21.15.116:0/1881216623 conn(0x55bf022a8310 msgr2=0x55bf022e1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:30.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.435+0000 7f94bce94740 1 -- 172.21.15.116:0/1881216623 shutdown_connections 2024-09-12T03:05:30.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.435+0000 7f94bce94740 1 -- 172.21.15.116:0/1881216623 wait complete. 2024-09-12T03:05:30.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.436+0000 7f94bce94740 20 client.15858 trim_cache size 0 max 16384 2024-09-12T03:05:30.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.448+0000 7f94bce94740 1 Processor -- start 2024-09-12T03:05:30.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.448+0000 7f94bce94740 1 -- start start 2024-09-12T03:05:30.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.448+0000 7f94bce94740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bf022df5d0 0x55bf022dbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.448+0000 7f94bce94740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf022e0b40 0x55bf02301d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.448+0000 7f94bce94740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf022ea370 0x55bf02302280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.449+0000 7f94bce94740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f94b41afcd0 con 0x55bf022e0b40 2024-09-12T03:05:30.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.449+0000 7f94bce94740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f94b41ad5a0 con 0x55bf022df5d0 2024-09-12T03:05:30.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.449+0000 7f94bce94740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f94b41ab3f0 con 0x55bf022ea370 2024-09-12T03:05:30.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.449+0000 7f94b3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf022e0b40 0x55bf02301d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.449+0000 7f94b3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf022e0b40 0x55bf02301d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36704/0 (socket says 172.21.15.116:36704) 2024-09-12T03:05:30.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.449+0000 7f94b3fff640 1 -- 172.21.15.116:0/3460151354 learned_addr learned my addr 172.21.15.116:0/3460151354 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:30.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.449+0000 7f94b9058640 1 --2- 172.21.15.116:0/3460151354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf022ea370 0x55bf02302280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.449+0000 7f94b3fff640 1 -- 172.21.15.116:0/3460151354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf022ea370 msgr2=0x55bf02302280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:30.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.449+0000 7f94b3fff640 1 --2- 172.21.15.116:0/3460151354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf022ea370 0x55bf02302280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.449+0000 7f94b3fff640 1 -- 172.21.15.116:0/3460151354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bf022df5d0 msgr2=0x55bf022dbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:30.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.449+0000 7f94b3fff640 1 --2- 172.21.15.116:0/3460151354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bf022df5d0 0x55bf022dbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.449+0000 7f94b3fff640 1 -- 172.21.15.116:0/3460151354 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf022ed1d0 con 0x55bf022e0b40 2024-09-12T03:05:30.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.449+0000 7f94b3fff640 1 --2- 172.21.15.116:0/3460151354 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf022e0b40 0x55bf02301d40 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f94a0005640 tx=0x7f94a0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ade8757e395e71f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:30.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.449+0000 7f949e7fc640 1 -- 172.21.15.116:0/3460151354 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0035040 con 0x55bf022e0b40 2024-09-12T03:05:30.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.449+0000 7f949e7fc640 1 -- 172.21.15.116:0/3460151354 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a004d070 con 0x55bf022e0b40 2024-09-12T03:05:30.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.449+0000 7f949e7fc640 1 -- 172.21.15.116:0/3460151354 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a005a640 con 0x55bf022e0b40 2024-09-12T03:05:30.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.450+0000 7f94bce94740 1 -- 172.21.15.116:0/3460151354 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf022e0b40 msgr2=0x55bf02301d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:30.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.450+0000 7f94bce94740 1 --2- 172.21.15.116:0/3460151354 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf022e0b40 0x55bf02301d40 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f94a0005640 tx=0x7f94a0000bc0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.450+0000 7f94bce94740 1 -- 172.21.15.116:0/3460151354 shutdown_connections 2024-09-12T03:05:30.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.450+0000 7f94bce94740 1 --2- 172.21.15.116:0/3460151354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf022ea370 0x55bf02302280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.450+0000 7f94bce94740 1 --2- 172.21.15.116:0/3460151354 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf022e0b40 0x55bf02301d40 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.450+0000 7f94bce94740 1 --2- 172.21.15.116:0/3460151354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bf022df5d0 0x55bf022dbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.450+0000 7f94bce94740 1 -- 172.21.15.116:0/3460151354 >> 172.21.15.116:0/3460151354 conn(0x55bf022a8310 msgr2=0x55bf022db700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:30.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.450+0000 7f94bce94740 1 -- 172.21.15.116:0/3460151354 shutdown_connections 2024-09-12T03:05:30.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.450+0000 7f94bce94740 1 -- 172.21.15.116:0/3460151354 wait complete. 2024-09-12T03:05:30.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.450+0000 7f94bce94740 1 -- start start 2024-09-12T03:05:30.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.450+0000 7f94bce94740 1 Processor -- start 2024-09-12T03:05:30.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.451+0000 7f94bce94740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bf022df5d0 0x55bf022f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.451+0000 7f94bce94740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf022e0b40 0x55bf022ede20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.451+0000 7f94bce94740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf022ea370 0x55bf022ee360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.451+0000 7f94bce94740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f94b41aba60 con 0x55bf022ea370 2024-09-12T03:05:30.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.451+0000 7f94bce94740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f94b41aced0 con 0x55bf022df5d0 2024-09-12T03:05:30.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.451+0000 7f94bce94740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f94b41ac130 con 0x55bf022e0b40 2024-09-12T03:05:30.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.451+0000 7f94bce94740 10 client.0 _ll_register_callbacks cb 0x7f94a8007520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:30.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.451+0000 7f94b9058640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf022ea370 0x55bf022ee360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.451+0000 7f94b9058640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf022ea370 0x55bf022ee360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36708/0 (socket says 172.21.15.116:36708) 2024-09-12T03:05:30.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.451+0000 7f94b9058640 1 -- 172.21.15.116:0/223930930 learned_addr learned my addr 172.21.15.116:0/223930930 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:30.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.451+0000 7f94b3fff640 1 --2- 172.21.15.116:0/223930930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf022e0b40 0x55bf022ede20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.451+0000 7f94b8857640 1 --2- 172.21.15.116:0/223930930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bf022df5d0 0x55bf022f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.451+0000 7f94b9058640 1 -- 172.21.15.116:0/223930930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf022e0b40 msgr2=0x55bf022ede20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:30.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.451+0000 7f94b9058640 1 --2- 172.21.15.116:0/223930930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf022e0b40 0x55bf022ede20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.451+0000 7f94b9058640 1 -- 172.21.15.116:0/223930930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bf022df5d0 msgr2=0x55bf022f2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:30.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.451+0000 7f94b9058640 1 --2- 172.21.15.116:0/223930930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bf022df5d0 0x55bf022f2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.451+0000 7f94b9058640 1 -- 172.21.15.116:0/223930930 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf022eeb90 con 0x55bf022ea370 2024-09-12T03:05:30.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.451+0000 7f94b9058640 1 --2- 172.21.15.116:0/223930930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf022ea370 0x55bf022ee360 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f94ac09bf20 tx=0x7f94ac09bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d00a059145720d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:30.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.451+0000 7f9493fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:05:30.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.451+0000 7f9493fff640 1 -- 172.21.15.116:0/223930930 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac0b4070 con 0x55bf022ea370 2024-09-12T03:05:30.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.452+0000 7f9493fff640 1 -- 172.21.15.116:0/223930930 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f94ac00fc80 con 0x55bf022ea370 2024-09-12T03:05:30.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.452+0000 7f94bce94740 1 -- 172.21.15.116:0/223930930 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94a004d070 con 0x55bf022ea370 2024-09-12T03:05:30.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.452+0000 7f9493fff640 1 -- 172.21.15.116:0/223930930 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f94ac09bf90 con 0x55bf022ea370 2024-09-12T03:05:30.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.452+0000 7f94bce94740 10 client.15864 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:30.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.452+0000 7f94bce94740 1 -- 172.21.15.116:0/223930930 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bf023690b0 con 0x55bf022ea370 2024-09-12T03:05:30.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.452+0000 7f9493fff640 1 -- 172.21.15.116:0/223930930 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a004d070 con 0x55bf022ea370 2024-09-12T03:05:30.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.452+0000 7f9493fff640 1 -- 172.21.15.116:0/223930930 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f94ac0a4770 con 0x55bf022ea370 2024-09-12T03:05:30.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.452+0000 7f94bce94740 10 client.15864 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:30.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.452+0000 7f94bce94740 10 client.15864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:05:30.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.452+0000 7f94bce94740 20 client.15864 populate_metadata read hostname 'smithi116' 2024-09-12T03:05:30.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.452+0000 7f94bce94740 1 --2- 172.21.15.116:0/223930930 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bf02369820 0x55bf0236bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.452+0000 7f94bce94740 4 client.15864 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:05:30.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.452+0000 7f94bce94740 1 -- 172.21.15.116:0/223930930 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf0224cd70 con 0x55bf02369820 2024-09-12T03:05:30.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.452+0000 7f94b8857640 1 --2- 172.21.15.116:0/223930930 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bf02369820 0x55bf0236bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.453+0000 7f94b8857640 1 --2- 172.21.15.116:0/223930930 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bf02369820 0x55bf0236bc10 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aba9576346ada8e3 server_cookie=7f4900da268bd978 in_seq=0 out_seq=0 2024-09-12T03:05:30.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.453+0000 7f9493fff640 10 client.15864 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:30.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.453+0000 7f9493fff640 1 -- 172.21.15.116:0/223930930 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf0224cd70 con 0x55bf02369820 2024-09-12T03:05:30.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.453+0000 7f9493fff640 10 client.15864 handle_command_reply: tid=0 2024-09-12T03:05:30.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.453+0000 7f94bce94740 1 client.15864 shutdown 2024-09-12T03:05:30.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.453+0000 7f94bce94740 1 -- 172.21.15.116:0/223930930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf022ea370 msgr2=0x55bf022ee360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:30.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.453+0000 7f94bce94740 1 --2- 172.21.15.116:0/223930930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf022ea370 0x55bf022ee360 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f94ac09bf20 tx=0x7f94ac09bc00 comp rx=0 tx=0).stop 2024-09-12T03:05:30.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.453+0000 7f94bce94740 1 -- 172.21.15.116:0/223930930 shutdown_connections 2024-09-12T03:05:30.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.453+0000 7f94bce94740 1 --2- 172.21.15.116:0/223930930 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bf02369820 0x55bf0236bc10 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.453+0000 7f94bce94740 1 --2- 172.21.15.116:0/223930930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf022ea370 0x55bf022ee360 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.453+0000 7f94bce94740 1 --2- 172.21.15.116:0/223930930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf022e0b40 0x55bf022ede20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.453+0000 7f9493fff640 0 client.15864 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:30.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.453+0000 7f94bce94740 1 --2- 172.21.15.116:0/223930930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bf022df5d0 0x55bf022f2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.453+0000 7f94bce94740 1 -- 172.21.15.116:0/223930930 >> 172.21.15.116:0/223930930 conn(0x55bf022a8310 msgr2=0x55bf022e1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:30.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.453+0000 7f94bce94740 1 -- 172.21.15.116:0/223930930 shutdown_connections 2024-09-12T03:05:30.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.453+0000 7f94bce94740 1 -- 172.21.15.116:0/223930930 wait complete. 2024-09-12T03:05:30.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.453+0000 7f94bce94740 20 client.15864 trim_cache size 0 max 16384 2024-09-12T03:05:30.465 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:05:30.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.454+0000 7f94baabb640 1 -- 172.21.15.116:0/3068735811 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f94a40a2c00 msgr2=0x7f94a40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:30.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.454+0000 7f94baabb640 1 --2- 172.21.15.116:0/3068735811 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f94a40a2c00 0x7f94a40a50c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f94a0005ab0 tx=0x7f94a0040000 comp rx=0 tx=0).stop 2024-09-12T03:05:30.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.454+0000 7f94baabb640 1 -- 172.21.15.116:0/3068735811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94b4106a80 msgr2=0x7f94b41a4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:30.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.454+0000 7f94baabb640 1 --2- 172.21.15.116:0/3068735811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94b4106a80 0x7f94b41a4370 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00cce0 tx=0x7f94ac00a620 comp rx=0 tx=0).stop 2024-09-12T03:05:30.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.454+0000 7f94baabb640 1 -- 172.21.15.116:0/3068735811 shutdown_connections 2024-09-12T03:05:30.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.454+0000 7f94baabb640 1 --2- 172.21.15.116:0/3068735811 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f94a40a2c00 0x7f94a40a50c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.454+0000 7f94baabb640 1 --2- 172.21.15.116:0/3068735811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94b4106a80 0x7f94b41a4370 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.454+0000 7f94baabb640 1 --2- 172.21.15.116:0/3068735811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94b4101420 0x7f94b4199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.454+0000 7f94baabb640 1 --2- 172.21.15.116:0/3068735811 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94b4100a80 0x7f94b41992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.454+0000 7f94baabb640 1 -- 172.21.15.116:0/3068735811 >> 172.21.15.116:0/3068735811 conn(0x7f94b40fc7a0 msgr2=0x7f94b40fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:30.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.454+0000 7f94baabb640 1 -- 172.21.15.116:0/3068735811 shutdown_connections 2024-09-12T03:05:30.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:30.454+0000 7f94baabb640 1 -- 172.21.15.116:0/3068735811 wait complete. 2024-09-12T03:05:30.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:30 smithi177 ceph-mon[35305]: pgmap v413: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:05:30.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:30 smithi183 ceph-mon[35535]: pgmap v413: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:05:32.175 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:05:32.176 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:05:32.436 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:32.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:32 smithi177 ceph-mon[35305]: pgmap v414: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:05:32.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:32 smithi116 ceph-mon[31410]: pgmap v414: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:05:32.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:32 smithi183 ceph-mon[35535]: pgmap v414: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:05:32.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.927+0000 7fc4ca6db640 1 -- 172.21.15.116:0/2538204770 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4c4106f10 msgr2=0x7fc4c410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:32.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.927+0000 7fc4ca6db640 1 --2- 172.21.15.116:0/2538204770 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4c4106f10 0x7fc4c410ea90 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009ca0 tx=0x7fc4bc02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.927+0000 7fc4ca6db640 1 -- 172.21.15.116:0/2538204770 shutdown_connections 2024-09-12T03:05:32.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.927+0000 7fc4ca6db640 1 --2- 172.21.15.116:0/2538204770 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4c4106f10 0x7fc4c410ea90 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.927+0000 7fc4ca6db640 1 --2- 172.21.15.116:0/2538204770 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4c4102790 0x7fc4c41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.927+0000 7fc4ca6db640 1 --2- 172.21.15.116:0/2538204770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4c4101de0 0x7fc4c41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.927+0000 7fc4ca6db640 1 -- 172.21.15.116:0/2538204770 >> 172.21.15.116:0/2538204770 conn(0x7fc4c40fd8c0 msgr2=0x7fc4c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:32.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.927+0000 7fc4ca6db640 1 -- 172.21.15.116:0/2538204770 shutdown_connections 2024-09-12T03:05:32.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.927+0000 7fc4ca6db640 1 -- 172.21.15.116:0/2538204770 wait complete. 2024-09-12T03:05:32.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.928+0000 7fc4ca6db640 1 Processor -- start 2024-09-12T03:05:32.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.928+0000 7fc4ca6db640 1 -- start start 2024-09-12T03:05:32.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.928+0000 7fc4ca6db640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4c4101de0 0x7fc4c40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:32.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.928+0000 7fc4ca6db640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4c4102790 0x7fc4c4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:32.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.928+0000 7fc4ca6db640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4c4106f10 0x7fc4c4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:32.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.928+0000 7fc4ca6db640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c4112010 con 0x7fc4c4106f10 2024-09-12T03:05:32.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.928+0000 7fc4ca6db640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c4111e90 con 0x7fc4c4102790 2024-09-12T03:05:32.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.928+0000 7fc4ca6db640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c4112190 con 0x7fc4c4101de0 2024-09-12T03:05:32.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.928+0000 7fc4c8c78640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4c4106f10 0x7fc4c4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:32.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.928+0000 7fc4c8c78640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4c4106f10 0x7fc4c4074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57940/0 (socket says 172.21.15.116:57940) 2024-09-12T03:05:32.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.928+0000 7fc4c8c78640 1 -- 172.21.15.116:0/3219482307 learned_addr learned my addr 172.21.15.116:0/3219482307 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:32.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.928+0000 7fc4c37fe640 1 --2- 172.21.15.116:0/3219482307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4c4102790 0x7fc4c4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:32.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.928+0000 7fc4c3fff640 1 --2- 172.21.15.116:0/3219482307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4c4101de0 0x7fc4c40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:32.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.928+0000 7fc4c8c78640 1 -- 172.21.15.116:0/3219482307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4c4101de0 msgr2=0x7fc4c40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:32.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.928+0000 7fc4c8c78640 1 --2- 172.21.15.116:0/3219482307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4c4101de0 0x7fc4c40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.928+0000 7fc4c8c78640 1 -- 172.21.15.116:0/3219482307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4c4102790 msgr2=0x7fc4c4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:32.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.928+0000 7fc4c8c78640 1 --2- 172.21.15.116:0/3219482307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4c4102790 0x7fc4c4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.928+0000 7fc4c8c78640 1 -- 172.21.15.116:0/3219482307 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c41a6ab0 con 0x7fc4c4106f10 2024-09-12T03:05:32.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.929+0000 7fc4c8c78640 1 --2- 172.21.15.116:0/3219482307 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4c4106f10 0x7fc4c4074f70 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc031a60 tx=0x7fc4bc031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:32.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.929+0000 7fc4c17fa640 1 -- 172.21.15.116:0/3219482307 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc003b70 con 0x7fc4c4106f10 2024-09-12T03:05:32.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.929+0000 7fc4ca6db640 1 -- 172.21.15.116:0/3219482307 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c41a6d40 con 0x7fc4c4106f10 2024-09-12T03:05:32.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.929+0000 7fc4c17fa640 1 -- 172.21.15.116:0/3219482307 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4bc003d10 con 0x7fc4c4106f10 2024-09-12T03:05:32.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.929+0000 7fc4c17fa640 1 -- 172.21.15.116:0/3219482307 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc004770 con 0x7fc4c4106f10 2024-09-12T03:05:32.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.929+0000 7fc4ca6db640 1 -- 172.21.15.116:0/3219482307 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c41a7280 con 0x7fc4c4106f10 2024-09-12T03:05:32.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.931+0000 7fc4c17fa640 1 -- 172.21.15.116:0/3219482307 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fc4bc02f790 con 0x7fc4c4106f10 2024-09-12T03:05:32.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.931+0000 7fc4ccab4740 1 Processor -- start 2024-09-12T03:05:32.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.932+0000 7fc4ccab4740 1 -- start start 2024-09-12T03:05:32.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.932+0000 7fc4c17fa640 1 --2- 172.21.15.116:0/3219482307 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc4ac0a2b80 0x7fc4ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:32.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.932+0000 7fc4ccab4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d2b8d82b60 0x55d2b8d82f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:32.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.932+0000 7fc4ccab4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2b8d83480 0x55d2b8d8d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:32.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.932+0000 7fc4ccab4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2b8d8dd40 0x55d2b8d90130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:32.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.932+0000 7fc4ccab4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d2b8ce8280 con 0x55d2b8d82b60 2024-09-12T03:05:32.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.932+0000 7fc4ccab4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d2b8c8fb10 con 0x55d2b8d8dd40 2024-09-12T03:05:32.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.932+0000 7fc4ccab4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d2b8ce4240 con 0x55d2b8d83480 2024-09-12T03:05:32.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.932+0000 7fc4c37fe640 1 --2- 172.21.15.116:0/3219482307 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc4ac0a2b80 0x7fc4ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:32.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.932+0000 7fc4c17fa640 1 -- 172.21.15.116:0/3219482307 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fc4bc0f04b0 con 0x7fc4c4106f10 2024-09-12T03:05:32.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.932+0000 7fc4c3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d2b8d82b60 0x55d2b8d82f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:32.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.932+0000 7fc4c3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d2b8d82b60 0x55d2b8d82f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57956/0 (socket says 172.21.15.116:57956) 2024-09-12T03:05:32.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.932+0000 7fc4c3fff640 1 -- 172.21.15.116:0/3708555811 learned_addr learned my addr 172.21.15.116:0/3708555811 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:32.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.932+0000 7fc4c3fff640 1 -- 172.21.15.116:0/3708555811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2b8d83480 msgr2=0x55d2b8d8d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:32.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.932+0000 7fc4c3fff640 1 --2- 172.21.15.116:0/3708555811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2b8d83480 0x55d2b8d8d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.932+0000 7fc4c8c78640 1 --2- 172.21.15.116:0/3708555811 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2b8d8dd40 0x55d2b8d90130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:32.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.932+0000 7fc4c3fff640 1 -- 172.21.15.116:0/3708555811 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2b8d8dd40 msgr2=0x55d2b8d90130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:32.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.932+0000 7fc4c3fff640 1 --2- 172.21.15.116:0/3708555811 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2b8d8dd40 0x55d2b8d90130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.932+0000 7fc4c3fff640 1 -- 172.21.15.116:0/3708555811 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2b8c9c1f0 con 0x55d2b8d82b60 2024-09-12T03:05:32.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.932+0000 7fc4c37fe640 1 --2- 172.21.15.116:0/3219482307 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc4ac0a2b80 0x7fc4ac0a5040 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4075f80 tx=0x7fc4b8009db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:32.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.932+0000 7fc4c3fff640 1 --2- 172.21.15.116:0/3708555811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d2b8d82b60 0x55d2b8d82f40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40099f0 tx=0x7fc4b4009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05eb0edefef066 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:32.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.933+0000 7fc4aa7fc640 1 -- 172.21.15.116:0/3708555811 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4017070 con 0x55d2b8d82b60 2024-09-12T03:05:32.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.933+0000 7fc4aa7fc640 1 -- 172.21.15.116:0/3708555811 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4b4013470 con 0x55d2b8d82b60 2024-09-12T03:05:32.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.933+0000 7fc4aa7fc640 1 -- 172.21.15.116:0/3708555811 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b40125f0 con 0x55d2b8d82b60 2024-09-12T03:05:32.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.933+0000 7fc4ccab4740 1 -- 172.21.15.116:0/3708555811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d2b8d82b60 msgr2=0x55d2b8d82f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:32.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.933+0000 7fc4ccab4740 1 --2- 172.21.15.116:0/3708555811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d2b8d82b60 0x55d2b8d82f40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40099f0 tx=0x7fc4b4009eb0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.933+0000 7fc4ccab4740 1 -- 172.21.15.116:0/3708555811 shutdown_connections 2024-09-12T03:05:32.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.933+0000 7fc4ccab4740 1 --2- 172.21.15.116:0/3708555811 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2b8d8dd40 0x55d2b8d90130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.933+0000 7fc4ccab4740 1 --2- 172.21.15.116:0/3708555811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2b8d83480 0x55d2b8d8d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.933+0000 7fc4ccab4740 1 --2- 172.21.15.116:0/3708555811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d2b8d82b60 0x55d2b8d82f40 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.933+0000 7fc4ccab4740 1 -- 172.21.15.116:0/3708555811 >> 172.21.15.116:0/3708555811 conn(0x55d2b8d7f0f0 msgr2=0x55d2b8d7f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:32.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.933+0000 7fc4ccab4740 1 -- 172.21.15.116:0/3708555811 shutdown_connections 2024-09-12T03:05:32.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.933+0000 7fc4ccab4740 1 -- 172.21.15.116:0/3708555811 wait complete. 2024-09-12T03:05:32.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.934+0000 7fc4ccab4740 1 -- start start 2024-09-12T03:05:32.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.934+0000 7fc4ccab4740 1 Processor -- start 2024-09-12T03:05:32.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.934+0000 7fc4ccab4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2b8d82b60 0x55d2b8d8d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:32.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.934+0000 7fc4ccab4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2b8d83480 0x55d2b8d8ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:32.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.934+0000 7fc4ccab4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d2b8d8dd40 0x55d2b8d8b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:32.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.934+0000 7fc4ccab4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d2b8d90910 con 0x55d2b8d8dd40 2024-09-12T03:05:32.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.934+0000 7fc4ccab4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d2b8ca2f10 con 0x55d2b8d82b60 2024-09-12T03:05:32.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.934+0000 7fc4ccab4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d2b8d90a90 con 0x55d2b8d83480 2024-09-12T03:05:32.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.934+0000 7fc4ccab4740 10 client.0 _ll_register_callbacks cb 0x55d2b8c6ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:32.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.934+0000 7fc4c8c78640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d2b8d8dd40 0x55d2b8d8b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:32.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.934+0000 7fc4c8c78640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d2b8d8dd40 0x55d2b8d8b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57962/0 (socket says 172.21.15.116:57962) 2024-09-12T03:05:32.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.934+0000 7fc4c8c78640 1 -- 172.21.15.116:0/819595480 learned_addr learned my addr 172.21.15.116:0/819595480 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:32.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.934+0000 7fc4c3fff640 1 --2- 172.21.15.116:0/819595480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2b8d82b60 0x55d2b8d8d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:32.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.935+0000 7fc4c8c78640 1 -- 172.21.15.116:0/819595480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2b8d83480 msgr2=0x55d2b8d8ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:32.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.935+0000 7fc4c37fe640 1 --2- 172.21.15.116:0/819595480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2b8d83480 0x55d2b8d8ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:32.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.935+0000 7fc4c8c78640 1 --2- 172.21.15.116:0/819595480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2b8d83480 0x55d2b8d8ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.935+0000 7fc4c8c78640 1 -- 172.21.15.116:0/819595480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2b8d82b60 msgr2=0x55d2b8d8d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:32.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.935+0000 7fc4c8c78640 1 --2- 172.21.15.116:0/819595480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2b8d82b60 0x55d2b8d8d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.935+0000 7fc4c8c78640 1 -- 172.21.15.116:0/819595480 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2b8a596a0 con 0x55d2b8d8dd40 2024-09-12T03:05:32.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.935+0000 7fc4c37fe640 1 --2- 172.21.15.116:0/819595480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2b8d83480 0x55d2b8d8ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:05:32.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.935+0000 7fc4c3fff640 1 --2- 172.21.15.116:0/819595480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2b8d82b60 0x55d2b8d8d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:05:32.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.935+0000 7fc4c8c78640 1 --2- 172.21.15.116:0/819595480 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d2b8d8dd40 0x55d2b8d8b2c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc0c5480 tx=0x7fc4bc0098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6592971ddfb57775 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:32.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.935+0000 7fc4a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:05:32.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.935+0000 7fc4a37fe640 1 -- 172.21.15.116:0/819595480 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc0bd440 con 0x55d2b8d8dd40 2024-09-12T03:05:32.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.935+0000 7fc4a37fe640 1 -- 172.21.15.116:0/819595480 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fc4bc0b93b0 con 0x55d2b8d8dd40 2024-09-12T03:05:32.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.935+0000 7fc4ccab4740 1 -- 172.21.15.116:0/819595480 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4b4013470 con 0x55d2b8d8dd40 2024-09-12T03:05:32.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.935+0000 7fc4a37fe640 1 -- 172.21.15.116:0/819595480 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc4bc0bcd40 con 0x55d2b8d8dd40 2024-09-12T03:05:32.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.935+0000 7fc4ccab4740 10 client.15876 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:32.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.935+0000 7fc4ccab4740 1 -- 172.21.15.116:0/819595480 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d2b8d90910 con 0x55d2b8d8dd40 2024-09-12T03:05:32.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.935+0000 7fc4a37fe640 1 -- 172.21.15.116:0/819595480 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4b4013470 con 0x55d2b8d8dd40 2024-09-12T03:05:32.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.935+0000 7fc4a37fe640 1 -- 172.21.15.116:0/819595480 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc4bc0c5520 con 0x55d2b8d8dd40 2024-09-12T03:05:32.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.935+0000 7fc4ccab4740 10 client.15876 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:32.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.935+0000 7fc4ccab4740 10 client.15876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:05:32.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.935+0000 7fc4ccab4740 20 client.15876 populate_metadata read hostname 'smithi116' 2024-09-12T03:05:32.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.935+0000 7fc4ccab4740 1 --2- 172.21.15.116:0/819595480 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d2b8d9d920 0x55d2b8d9fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:32.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.935+0000 7fc4ccab4740 4 client.15876 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:05:32.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.935+0000 7fc4ccab4740 1 -- 172.21.15.116:0/819595480 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2b8cf57f0 con 0x55d2b8d9d920 2024-09-12T03:05:32.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.936+0000 7fc4c3fff640 1 --2- 172.21.15.116:0/819595480 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d2b8d9d920 0x55d2b8d9fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:32.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.936+0000 7fc4a37fe640 10 client.15876 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:32.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.936+0000 7fc4c3fff640 1 --2- 172.21.15.116:0/819595480 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d2b8d9d920 0x55d2b8d9fda0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a39f62405b5bdb86 server_cookie=dc614361799bd13a in_seq=0 out_seq=0 2024-09-12T03:05:32.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.937+0000 7fc4a37fe640 1 -- 172.21.15.116:0/819595480 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d2b8cf57f0 con 0x55d2b8d9d920 2024-09-12T03:05:32.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.937+0000 7fc4a37fe640 10 client.15876 handle_command_reply: tid=0 2024-09-12T03:05:32.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.937+0000 7fc4ccab4740 1 client.15876 shutdown 2024-09-12T03:05:32.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.937+0000 7fc4ccab4740 1 -- 172.21.15.116:0/819595480 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d2b8d8dd40 msgr2=0x55d2b8d8b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:32.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.937+0000 7fc4ccab4740 1 --2- 172.21.15.116:0/819595480 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d2b8d8dd40 0x55d2b8d8b2c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc0c5480 tx=0x7fc4bc0098d0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.937+0000 7fc4ccab4740 1 -- 172.21.15.116:0/819595480 shutdown_connections 2024-09-12T03:05:32.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.937+0000 7fc4ccab4740 1 --2- 172.21.15.116:0/819595480 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d2b8d9d920 0x55d2b8d9fda0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.937+0000 7fc4ccab4740 1 --2- 172.21.15.116:0/819595480 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d2b8d8dd40 0x55d2b8d8b2c0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.937+0000 7fc4a37fe640 0 client.15876 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:32.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.937+0000 7fc4ccab4740 1 --2- 172.21.15.116:0/819595480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2b8d83480 0x55d2b8d8ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.937+0000 7fc4ccab4740 1 --2- 172.21.15.116:0/819595480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2b8d82b60 0x55d2b8d8d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.937+0000 7fc4ccab4740 1 -- 172.21.15.116:0/819595480 >> 172.21.15.116:0/819595480 conn(0x55d2b8d7f0f0 msgr2=0x55d2b8d7e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:32.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.937+0000 7fc4ccab4740 1 -- 172.21.15.116:0/819595480 shutdown_connections 2024-09-12T03:05:32.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.937+0000 7fc4ccab4740 1 -- 172.21.15.116:0/819595480 wait complete. 2024-09-12T03:05:32.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.937+0000 7fc4ccab4740 20 client.15876 trim_cache size 0 max 16384 2024-09-12T03:05:32.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.954+0000 7fc4ccab4740 1 Processor -- start 2024-09-12T03:05:32.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.954+0000 7fc4ccab4740 1 -- start start 2024-09-12T03:05:32.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.954+0000 7fc4ccab4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2b8d82b60 0x55d2b8d89b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:32.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.954+0000 7fc4ccab4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2b8d83480 0x55d2b8da5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:32.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.954+0000 7fc4ccab4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d2b8d8dd40 0x55d2b8da5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:32.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.954+0000 7fc4ccab4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c41b1fe0 con 0x55d2b8d8dd40 2024-09-12T03:05:32.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.954+0000 7fc4ccab4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c41af8b0 con 0x55d2b8d82b60 2024-09-12T03:05:32.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.954+0000 7fc4ccab4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c41ad720 con 0x55d2b8d83480 2024-09-12T03:05:32.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.955+0000 7fc4c8c78640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d2b8d8dd40 0x55d2b8da5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:32.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.955+0000 7fc4c8c78640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d2b8d8dd40 0x55d2b8da5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57974/0 (socket says 172.21.15.116:57974) 2024-09-12T03:05:32.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.955+0000 7fc4c8c78640 1 -- 172.21.15.116:0/2579903061 learned_addr learned my addr 172.21.15.116:0/2579903061 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:32.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.955+0000 7fc4c37fe640 1 --2- 172.21.15.116:0/2579903061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2b8d83480 0x55d2b8da5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:32.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.955+0000 7fc4c8c78640 1 -- 172.21.15.116:0/2579903061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2b8d83480 msgr2=0x55d2b8da5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:32.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.955+0000 7fc4c8c78640 1 --2- 172.21.15.116:0/2579903061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2b8d83480 0x55d2b8da5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.955+0000 7fc4c8c78640 1 -- 172.21.15.116:0/2579903061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2b8d82b60 msgr2=0x55d2b8d89b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:32.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.955+0000 7fc4c8c78640 1 --2- 172.21.15.116:0/2579903061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2b8d82b60 0x55d2b8d89b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.955+0000 7fc4c8c78640 1 -- 172.21.15.116:0/2579903061 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2b8d832b0 con 0x55d2b8d8dd40 2024-09-12T03:05:32.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.955+0000 7fc4c37fe640 1 --2- 172.21.15.116:0/2579903061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2b8d83480 0x55d2b8da5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:05:32.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.955+0000 7fc4c8c78640 1 --2- 172.21.15.116:0/2579903061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d2b8d8dd40 0x55d2b8da5ba0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc0ee350 tx=0x7fc4bc0bcbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd59d969b63ce60f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:32.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.955+0000 7fc4a9ffb640 1 -- 172.21.15.116:0/2579903061 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc0f02f0 con 0x55d2b8d8dd40 2024-09-12T03:05:32.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.955+0000 7fc4a9ffb640 1 -- 172.21.15.116:0/2579903061 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4bc036020 con 0x55d2b8d8dd40 2024-09-12T03:05:32.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.955+0000 7fc4a9ffb640 1 -- 172.21.15.116:0/2579903061 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc0bb040 con 0x55d2b8d8dd40 2024-09-12T03:05:32.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.956+0000 7fc4ccab4740 1 -- 172.21.15.116:0/2579903061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d2b8d8dd40 msgr2=0x55d2b8da5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:32.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.956+0000 7fc4ccab4740 1 --2- 172.21.15.116:0/2579903061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d2b8d8dd40 0x55d2b8da5ba0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc0ee350 tx=0x7fc4bc0bcbe0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.956+0000 7fc4ccab4740 1 -- 172.21.15.116:0/2579903061 shutdown_connections 2024-09-12T03:05:32.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.956+0000 7fc4ccab4740 1 --2- 172.21.15.116:0/2579903061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d2b8d8dd40 0x55d2b8da5ba0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.956+0000 7fc4ccab4740 1 --2- 172.21.15.116:0/2579903061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2b8d83480 0x55d2b8da5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.956+0000 7fc4ccab4740 1 --2- 172.21.15.116:0/2579903061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2b8d82b60 0x55d2b8d89b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.956+0000 7fc4ccab4740 1 -- 172.21.15.116:0/2579903061 >> 172.21.15.116:0/2579903061 conn(0x55d2b8d7f0f0 msgr2=0x55d2b8d89400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:32.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.956+0000 7fc4ccab4740 1 -- 172.21.15.116:0/2579903061 shutdown_connections 2024-09-12T03:05:32.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.956+0000 7fc4ccab4740 1 -- 172.21.15.116:0/2579903061 wait complete. 2024-09-12T03:05:32.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.956+0000 7fc4ccab4740 1 -- start start 2024-09-12T03:05:32.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.956+0000 7fc4ccab4740 1 Processor -- start 2024-09-12T03:05:32.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.957+0000 7fc4ccab4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d2b8d82b60 0x55d2b8d95cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:32.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.957+0000 7fc4ccab4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2b8d83480 0x55d2b8d96230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:32.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.957+0000 7fc4ccab4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2b8d8dd40 0x55d2b8d90ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:32.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.957+0000 7fc4ccab4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c41add70 con 0x55d2b8d82b60 2024-09-12T03:05:32.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.957+0000 7fc4ccab4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c41af1e0 con 0x55d2b8d83480 2024-09-12T03:05:32.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.957+0000 7fc4ccab4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c41ae440 con 0x55d2b8d8dd40 2024-09-12T03:05:32.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.957+0000 7fc4ccab4740 10 client.0 _ll_register_callbacks cb 0x7fc4bc004910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:32.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.957+0000 7fc4c3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d2b8d82b60 0x55d2b8d95cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:32.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.957+0000 7fc4c3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d2b8d82b60 0x55d2b8d95cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57980/0 (socket says 172.21.15.116:57980) 2024-09-12T03:05:32.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.957+0000 7fc4c3fff640 1 -- 172.21.15.116:0/2373453763 learned_addr learned my addr 172.21.15.116:0/2373453763 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:32.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.957+0000 7fc4c37fe640 1 --2- 172.21.15.116:0/2373453763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2b8d83480 0x55d2b8d96230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:32.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.957+0000 7fc4c8c78640 1 --2- 172.21.15.116:0/2373453763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2b8d8dd40 0x55d2b8d90ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:32.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.957+0000 7fc4c3fff640 1 -- 172.21.15.116:0/2373453763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2b8d8dd40 msgr2=0x55d2b8d90ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:32.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.957+0000 7fc4c3fff640 1 --2- 172.21.15.116:0/2373453763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2b8d8dd40 0x55d2b8d90ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.957+0000 7fc4c3fff640 1 -- 172.21.15.116:0/2373453763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2b8d83480 msgr2=0x55d2b8d96230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:32.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.957+0000 7fc4c3fff640 1 --2- 172.21.15.116:0/2373453763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2b8d83480 0x55d2b8d96230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.957+0000 7fc4c3fff640 1 -- 172.21.15.116:0/2373453763 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2b8d911e0 con 0x55d2b8d82b60 2024-09-12T03:05:32.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.957+0000 7fc4c8c78640 1 --2- 172.21.15.116:0/2373453763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2b8d8dd40 0x55d2b8d90ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:32.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.957+0000 7fc4c37fe640 1 --2- 172.21.15.116:0/2373453763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2b8d83480 0x55d2b8d96230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:32.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.957+0000 7fc4c3fff640 1 --2- 172.21.15.116:0/2373453763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d2b8d82b60 0x55d2b8d95cf0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40099f0 tx=0x7fc4b4013dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=207fc16944e747fc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:32.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.957+0000 7fc4a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:05:32.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.957+0000 7fc4a37fe640 1 -- 172.21.15.116:0/2373453763 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4017040 con 0x55d2b8d82b60 2024-09-12T03:05:32.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.957+0000 7fc4a37fe640 1 -- 172.21.15.116:0/2373453763 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fc4b4013470 con 0x55d2b8d82b60 2024-09-12T03:05:32.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.958+0000 7fc4ccab4740 1 -- 172.21.15.116:0/2373453763 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4bc036020 con 0x55d2b8d82b60 2024-09-12T03:05:32.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.958+0000 7fc4a37fe640 1 -- 172.21.15.116:0/2373453763 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc4b40094a0 con 0x55d2b8d82b60 2024-09-12T03:05:32.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.958+0000 7fc4ccab4740 10 client.15882 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:32.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.958+0000 7fc4ccab4740 1 -- 172.21.15.116:0/2373453763 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d2b8e08d20 con 0x55d2b8d82b60 2024-09-12T03:05:32.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.958+0000 7fc4a37fe640 1 -- 172.21.15.116:0/2373453763 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4bc036020 con 0x55d2b8d82b60 2024-09-12T03:05:32.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.958+0000 7fc4a37fe640 1 -- 172.21.15.116:0/2373453763 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc4b401cb50 con 0x55d2b8d82b60 2024-09-12T03:05:32.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.958+0000 7fc4ccab4740 10 client.15882 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:32.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.958+0000 7fc4ccab4740 10 client.15882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:05:32.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.958+0000 7fc4ccab4740 20 client.15882 populate_metadata read hostname 'smithi116' 2024-09-12T03:05:32.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.958+0000 7fc4ccab4740 1 --2- 172.21.15.116:0/2373453763 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d2b8e091c0 0x55d2b8e0b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:32.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.958+0000 7fc4ccab4740 4 client.15882 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:05:32.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.958+0000 7fc4ccab4740 1 -- 172.21.15.116:0/2373453763 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d2b8cefd00 con 0x55d2b8e091c0 2024-09-12T03:05:32.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.958+0000 7fc4c37fe640 1 --2- 172.21.15.116:0/2373453763 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d2b8e091c0 0x55d2b8e0b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:32.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.959+0000 7fc4a37fe640 10 client.15882 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:32.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.959+0000 7fc4c37fe640 1 --2- 172.21.15.116:0/2373453763 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d2b8e091c0 0x55d2b8e0b5b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf9ab37bac9379a3 server_cookie=ac85af12473d8f5b in_seq=0 out_seq=0 2024-09-12T03:05:32.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.960+0000 7fc4a37fe640 1 -- 172.21.15.116:0/2373453763 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d2b8cefd00 con 0x55d2b8e091c0 2024-09-12T03:05:32.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.960+0000 7fc4a37fe640 10 client.15882 handle_command_reply: tid=0 2024-09-12T03:05:32.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.960+0000 7fc4ccab4740 1 client.15882 shutdown 2024-09-12T03:05:32.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.960+0000 7fc4ccab4740 1 -- 172.21.15.116:0/2373453763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d2b8d82b60 msgr2=0x55d2b8d95cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:32.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.960+0000 7fc4ccab4740 1 --2- 172.21.15.116:0/2373453763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d2b8d82b60 0x55d2b8d95cf0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40099f0 tx=0x7fc4b4013dd0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.960+0000 7fc4ccab4740 1 -- 172.21.15.116:0/2373453763 shutdown_connections 2024-09-12T03:05:32.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.960+0000 7fc4ccab4740 1 --2- 172.21.15.116:0/2373453763 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d2b8e091c0 0x55d2b8e0b5b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.960+0000 7fc4ccab4740 1 --2- 172.21.15.116:0/2373453763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2b8d8dd40 0x55d2b8d90ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.960+0000 7fc4ccab4740 1 --2- 172.21.15.116:0/2373453763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2b8d83480 0x55d2b8d96230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.960+0000 7fc4ccab4740 1 --2- 172.21.15.116:0/2373453763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d2b8d82b60 0x55d2b8d95cf0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.960+0000 7fc4ccab4740 1 -- 172.21.15.116:0/2373453763 >> 172.21.15.116:0/2373453763 conn(0x55d2b8d7f0f0 msgr2=0x55d2b8d89c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:32.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.960+0000 7fc4a37fe640 0 client.15882 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:32.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.960+0000 7fc4ccab4740 1 -- 172.21.15.116:0/2373453763 shutdown_connections 2024-09-12T03:05:32.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.960+0000 7fc4ccab4740 1 -- 172.21.15.116:0/2373453763 wait complete. 2024-09-12T03:05:32.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.960+0000 7fc4ccab4740 20 client.15882 trim_cache size 0 max 16384 2024-09-12T03:05:32.974 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:05:32.974 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:05:32.974 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "05000b93-c671-4157-83cd-b6b8fd74d09c", 2024-09-12T03:05:32.974 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:05:32.974 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:05:32.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.961+0000 7fc4ca6db640 1 -- 172.21.15.116:0/3219482307 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc4ac0a2b80 msgr2=0x7fc4ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:32.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.961+0000 7fc4ca6db640 1 --2- 172.21.15.116:0/3219482307 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc4ac0a2b80 0x7fc4ac0a5040 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4075f80 tx=0x7fc4b8009db0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.961+0000 7fc4ca6db640 1 -- 172.21.15.116:0/3219482307 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4c4106f10 msgr2=0x7fc4c4074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:32.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.961+0000 7fc4ca6db640 1 --2- 172.21.15.116:0/3219482307 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4c4106f10 0x7fc4c4074f70 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc031a60 tx=0x7fc4bc031a90 comp rx=0 tx=0).stop 2024-09-12T03:05:32.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.961+0000 7fc4ca6db640 1 -- 172.21.15.116:0/3219482307 shutdown_connections 2024-09-12T03:05:32.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.961+0000 7fc4ca6db640 1 --2- 172.21.15.116:0/3219482307 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc4ac0a2b80 0x7fc4ac0a5040 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.961+0000 7fc4ca6db640 1 --2- 172.21.15.116:0/3219482307 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4c4106f10 0x7fc4c4074f70 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.961+0000 7fc4ca6db640 1 --2- 172.21.15.116:0/3219482307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4c4102790 0x7fc4c4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.961+0000 7fc4ca6db640 1 --2- 172.21.15.116:0/3219482307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4c4101de0 0x7fc4c40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.961+0000 7fc4ca6db640 1 -- 172.21.15.116:0/3219482307 >> 172.21.15.116:0/3219482307 conn(0x7fc4c40fd8c0 msgr2=0x7fc4c40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:32.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.961+0000 7fc4ca6db640 1 -- 172.21.15.116:0/3219482307 shutdown_connections 2024-09-12T03:05:32.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:32.961+0000 7fc4ca6db640 1 -- 172.21.15.116:0/3219482307 wait complete. 2024-09-12T03:05:33.872 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:05:34.117 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:34.529 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:34 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:05:34.529 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:34 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:05:34.529 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:34 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:05:34.529 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:34 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:05:34.529 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:34 smithi116 ceph-mon[31410]: pgmap v415: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:05:34.529 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:34 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:05:34.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.609+0000 7f32dc6cc640 1 -- 172.21.15.116:0/4066815867 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32d4106a80 msgr2=0x7f32d4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:34.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.609+0000 7f32dc6cc640 1 --2- 172.21.15.116:0/4066815867 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32d4106a80 0x7f32d4106e60 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009930 tx=0x7f32c802a420 comp rx=0 tx=0).stop 2024-09-12T03:05:34.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.609+0000 7f32dc6cc640 1 -- 172.21.15.116:0/4066815867 shutdown_connections 2024-09-12T03:05:34.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.609+0000 7f32dc6cc640 1 --2- 172.21.15.116:0/4066815867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32d4101420 0x7f32d410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.609+0000 7f32dc6cc640 1 --2- 172.21.15.116:0/4066815867 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32d4100a80 0x7f32d4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.609+0000 7f32dc6cc640 1 --2- 172.21.15.116:0/4066815867 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32d4106a80 0x7f32d4106e60 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.609+0000 7f32dc6cc640 1 -- 172.21.15.116:0/4066815867 >> 172.21.15.116:0/4066815867 conn(0x7f32d40fc7a0 msgr2=0x7f32d40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:34.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.609+0000 7f32dc6cc640 1 -- 172.21.15.116:0/4066815867 shutdown_connections 2024-09-12T03:05:34.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.609+0000 7f32dc6cc640 1 -- 172.21.15.116:0/4066815867 wait complete. 2024-09-12T03:05:34.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.609+0000 7f32dc6cc640 1 Processor -- start 2024-09-12T03:05:34.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.610+0000 7f32dc6cc640 1 -- start start 2024-09-12T03:05:34.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.610+0000 7f32dc6cc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32d4100a80 0x7f32d41992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:34.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.610+0000 7f32dc6cc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32d4101420 0x7f32d4199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:34.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.610+0000 7f32dc6cc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32d4106a80 0x7f32d41a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.610+0000 7f32dc6cc640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f32d4110e80 con 0x7f32d4101420 2024-09-12T03:05:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.610+0000 7f32dc6cc640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f32d4110d00 con 0x7f32d4106a80 2024-09-12T03:05:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.610+0000 7f32dc6cc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f32d4111000 con 0x7f32d4100a80 2024-09-12T03:05:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.610+0000 7f32d9c67640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32d4101420 0x7f32d4199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.610+0000 7f32d9c67640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32d4101420 0x7f32d4199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58000/0 (socket says 172.21.15.116:58000) 2024-09-12T03:05:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.610+0000 7f32d9c67640 1 -- 172.21.15.116:0/1051878700 learned_addr learned my addr 172.21.15.116:0/1051878700 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.610+0000 7f32dac69640 1 --2- 172.21.15.116:0/1051878700 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32d4106a80 0x7f32d41a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.610+0000 7f32da468640 1 --2- 172.21.15.116:0/1051878700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32d4100a80 0x7f32d41992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.610+0000 7f32d9c67640 1 -- 172.21.15.116:0/1051878700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32d4100a80 msgr2=0x7f32d41992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.610+0000 7f32d9c67640 1 --2- 172.21.15.116:0/1051878700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32d4100a80 0x7f32d41992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.610+0000 7f32d9c67640 1 -- 172.21.15.116:0/1051878700 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32d4106a80 msgr2=0x7f32d41a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.610+0000 7f32d9c67640 1 --2- 172.21.15.116:0/1051878700 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32d4106a80 0x7f32d41a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.610+0000 7f32d9c67640 1 -- 172.21.15.116:0/1051878700 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d41a48b0 con 0x7f32d4101420 2024-09-12T03:05:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.610+0000 7f32da468640 1 --2- 172.21.15.116:0/1051878700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32d4100a80 0x7f32d41992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.610+0000 7f32dac69640 1 --2- 172.21.15.116:0/1051878700 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32d4106a80 0x7f32d41a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.610+0000 7f32d9c67640 1 --2- 172.21.15.116:0/1051878700 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32d4101420 0x7f32d4199830 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f32d000d950 tx=0x7f32d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.610+0000 7f32c77fe640 1 -- 172.21.15.116:0/1051878700 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d0017030 con 0x7f32d4101420 2024-09-12T03:05:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.610+0000 7f32c77fe640 1 -- 172.21.15.116:0/1051878700 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32d0004020 con 0x7f32d4101420 2024-09-12T03:05:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.610+0000 7f32c77fe640 1 -- 172.21.15.116:0/1051878700 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d00047a0 con 0x7f32d4101420 2024-09-12T03:05:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.610+0000 7f32dc6cc640 1 -- 172.21.15.116:0/1051878700 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32d41a4a80 con 0x7f32d4101420 2024-09-12T03:05:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.611+0000 7f32dc6cc640 1 -- 172.21.15.116:0/1051878700 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32d41a4fc0 con 0x7f32d4101420 2024-09-12T03:05:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.613+0000 7f32c77fe640 1 -- 172.21.15.116:0/1051878700 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f32d00041d0 con 0x7f32d4101420 2024-09-12T03:05:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.613+0000 7f32deaa5740 1 Processor -- start 2024-09-12T03:05:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.614+0000 7f32deaa5740 1 -- start start 2024-09-12T03:05:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.614+0000 7f32deaa5740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9ab92b4a0 0x55c9ab92b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.614+0000 7f32deaa5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9ab92bdc0 0x55c9ab936140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.614+0000 7f32deaa5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9ab936680 0x55c9ab938a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:34.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.614+0000 7f32deaa5740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c9ab8912e0 con 0x55c9ab92b4a0 2024-09-12T03:05:34.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.614+0000 7f32deaa5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c9ab838ce0 con 0x55c9ab92bdc0 2024-09-12T03:05:34.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.614+0000 7f32deaa5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c9ab88d2a0 con 0x55c9ab936680 2024-09-12T03:05:34.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.614+0000 7f32dac69640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9ab92b4a0 0x55c9ab92b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:34.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.614+0000 7f32dac69640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9ab92b4a0 0x55c9ab92b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58008/0 (socket says 172.21.15.116:58008) 2024-09-12T03:05:34.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.614+0000 7f32dac69640 1 -- 172.21.15.116:0/1198774059 learned_addr learned my addr 172.21.15.116:0/1198774059 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:34.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.614+0000 7f32d9c67640 1 --2- 172.21.15.116:0/1198774059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9ab936680 0x55c9ab938a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:34.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.614+0000 7f32da468640 1 --2- 172.21.15.116:0/1198774059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9ab92bdc0 0x55c9ab936140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:34.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.614+0000 7f32dac69640 1 -- 172.21.15.116:0/1198774059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9ab936680 msgr2=0x55c9ab938a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:34.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.614+0000 7f32dac69640 1 --2- 172.21.15.116:0/1198774059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9ab936680 0x55c9ab938a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.614+0000 7f32dac69640 1 -- 172.21.15.116:0/1198774059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9ab92bdc0 msgr2=0x55c9ab936140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:34.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.614+0000 7f32dac69640 1 --2- 172.21.15.116:0/1198774059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9ab92bdc0 0x55c9ab936140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.614+0000 7f32dac69640 1 -- 172.21.15.116:0/1198774059 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9ab81d440 con 0x55c9ab92b4a0 2024-09-12T03:05:34.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.614+0000 7f32c77fe640 1 --2- 172.21.15.116:0/1051878700 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f32c00a2c00 0x7f32c00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:34.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.614+0000 7f32dac69640 1 --2- 172.21.15.116:0/1198774059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9ab92b4a0 0x55c9ab92b880 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f32cc009f30 tx=0x7f32cc010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fc876e3c4ca72c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:34.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.615+0000 7f32da468640 1 --2- 172.21.15.116:0/1051878700 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f32c00a2c00 0x7f32c00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:34.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.615+0000 7f32c4ff9640 1 -- 172.21.15.116:0/1198774059 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32cc016070 con 0x55c9ab92b4a0 2024-09-12T03:05:34.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.615+0000 7f32c77fe640 1 -- 172.21.15.116:0/1051878700 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f32d00cea30 con 0x7f32d4101420 2024-09-12T03:05:34.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.615+0000 7f32c4ff9640 1 -- 172.21.15.116:0/1198774059 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32cc01bb40 con 0x55c9ab92b4a0 2024-09-12T03:05:34.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.615+0000 7f32c4ff9640 1 -- 172.21.15.116:0/1198774059 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32cc012ca0 con 0x55c9ab92b4a0 2024-09-12T03:05:34.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.615+0000 7f32deaa5740 1 -- 172.21.15.116:0/1198774059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9ab92b4a0 msgr2=0x55c9ab92b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:34.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.615+0000 7f32da468640 1 --2- 172.21.15.116:0/1051878700 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f32c00a2c00 0x7f32c00a50c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f32c8000c00 tx=0x7f32c8005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:34.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.615+0000 7f32deaa5740 1 --2- 172.21.15.116:0/1198774059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9ab92b4a0 0x55c9ab92b880 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f32cc009f30 tx=0x7f32cc010800 comp rx=0 tx=0).stop 2024-09-12T03:05:34.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.615+0000 7f32deaa5740 1 -- 172.21.15.116:0/1198774059 shutdown_connections 2024-09-12T03:05:34.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.615+0000 7f32deaa5740 1 --2- 172.21.15.116:0/1198774059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9ab936680 0x55c9ab938a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.615+0000 7f32deaa5740 1 --2- 172.21.15.116:0/1198774059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9ab92bdc0 0x55c9ab936140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.615+0000 7f32deaa5740 1 --2- 172.21.15.116:0/1198774059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9ab92b4a0 0x55c9ab92b880 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.615+0000 7f32deaa5740 1 -- 172.21.15.116:0/1198774059 >> 172.21.15.116:0/1198774059 conn(0x55c9ab927a30 msgr2=0x55c9ab927e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:34.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.616+0000 7f32deaa5740 1 -- 172.21.15.116:0/1198774059 shutdown_connections 2024-09-12T03:05:34.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.616+0000 7f32deaa5740 1 -- 172.21.15.116:0/1198774059 wait complete. 2024-09-12T03:05:34.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.616+0000 7f32deaa5740 1 -- start start 2024-09-12T03:05:34.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.616+0000 7f32deaa5740 1 Processor -- start 2024-09-12T03:05:34.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.616+0000 7f32deaa5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9ab92b4a0 0x55c9ab936010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:34.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.616+0000 7f32deaa5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9ab92bdc0 0x55c9ab9332c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:34.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.616+0000 7f32deaa5740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9ab936680 0x55c9ab933840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:34.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.616+0000 7f32deaa5740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c9ab939250 con 0x55c9ab936680 2024-09-12T03:05:34.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.616+0000 7f32deaa5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c9ab84c050 con 0x55c9ab92bdc0 2024-09-12T03:05:34.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.616+0000 7f32deaa5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c9ab9393d0 con 0x55c9ab92b4a0 2024-09-12T03:05:34.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.616+0000 7f32deaa5740 10 client.0 _ll_register_callbacks cb 0x55c9ab88c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:34.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.616+0000 7f32dac69640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9ab936680 0x55c9ab933840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:34.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.616+0000 7f32dac69640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9ab936680 0x55c9ab933840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58014/0 (socket says 172.21.15.116:58014) 2024-09-12T03:05:34.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.616+0000 7f32dac69640 1 -- 172.21.15.116:0/3866556168 learned_addr learned my addr 172.21.15.116:0/3866556168 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:34.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.616+0000 7f32da468640 1 --2- 172.21.15.116:0/3866556168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9ab92b4a0 0x55c9ab936010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:34.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.617+0000 7f32d9c67640 1 --2- 172.21.15.116:0/3866556168 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9ab92bdc0 0x55c9ab9332c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:34.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.617+0000 7f32dac69640 1 -- 172.21.15.116:0/3866556168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9ab92b4a0 msgr2=0x55c9ab936010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:34.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.617+0000 7f32dac69640 1 --2- 172.21.15.116:0/3866556168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9ab92b4a0 0x55c9ab936010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.617+0000 7f32dac69640 1 -- 172.21.15.116:0/3866556168 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9ab92bdc0 msgr2=0x55c9ab9332c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:34.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.617+0000 7f32dac69640 1 --2- 172.21.15.116:0/3866556168 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9ab92bdc0 0x55c9ab9332c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.617+0000 7f32dac69640 1 -- 172.21.15.116:0/3866556168 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9ab6023c0 con 0x55c9ab936680 2024-09-12T03:05:34.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.617+0000 7f32d9c67640 1 --2- 172.21.15.116:0/3866556168 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9ab92bdc0 0x55c9ab9332c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:05:34.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.617+0000 7f32dac69640 1 --2- 172.21.15.116:0/3866556168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9ab936680 0x55c9ab933840 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f32cc009f00 tx=0x7f32cc009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200675c79023c651 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:34.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.617+0000 7f32bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:05:34.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.617+0000 7f32bdffb640 1 -- 172.21.15.116:0/3866556168 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32cc016070 con 0x55c9ab936680 2024-09-12T03:05:34.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.617+0000 7f32bdffb640 1 -- 172.21.15.116:0/3866556168 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f32cc004c50 con 0x55c9ab936680 2024-09-12T03:05:34.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.617+0000 7f32deaa5740 1 -- 172.21.15.116:0/3866556168 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32cc01bb40 con 0x55c9ab936680 2024-09-12T03:05:34.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.617+0000 7f32bdffb640 1 -- 172.21.15.116:0/3866556168 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f32cc004e50 con 0x55c9ab936680 2024-09-12T03:05:34.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.617+0000 7f32deaa5740 10 client.15894 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:34.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.617+0000 7f32deaa5740 1 -- 172.21.15.116:0/3866556168 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c9ab944c40 con 0x55c9ab936680 2024-09-12T03:05:34.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.617+0000 7f32bdffb640 1 -- 172.21.15.116:0/3866556168 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32cc01bb40 con 0x55c9ab936680 2024-09-12T03:05:34.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.617+0000 7f32bdffb640 1 -- 172.21.15.116:0/3866556168 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f32cc00f040 con 0x55c9ab936680 2024-09-12T03:05:34.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.618+0000 7f32deaa5740 10 client.15894 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:34.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.618+0000 7f32deaa5740 10 client.15894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:05:34.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.618+0000 7f32deaa5740 20 client.15894 populate_metadata read hostname 'smithi116' 2024-09-12T03:05:34.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.618+0000 7f32deaa5740 1 --2- 172.21.15.116:0/3866556168 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c9ab946100 0x55c9ab9484f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:34.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.618+0000 7f32deaa5740 4 client.15894 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:05:34.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.618+0000 7f32deaa5740 1 -- 172.21.15.116:0/3866556168 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9ab89e850 con 0x55c9ab946100 2024-09-12T03:05:34.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.618+0000 7f32da468640 1 --2- 172.21.15.116:0/3866556168 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c9ab946100 0x55c9ab9484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:34.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.618+0000 7f32bdffb640 10 client.15894 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:34.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.618+0000 7f32da468640 1 --2- 172.21.15.116:0/3866556168 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c9ab946100 0x55c9ab9484f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99800a04e425dd71 server_cookie=3a0a050e677ca086 in_seq=0 out_seq=0 2024-09-12T03:05:34.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.619+0000 7f32bdffb640 1 -- 172.21.15.116:0/3866556168 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c9ab89e850 con 0x55c9ab946100 2024-09-12T03:05:34.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.619+0000 7f32bdffb640 10 client.15894 handle_command_reply: tid=0 2024-09-12T03:05:34.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.619+0000 7f32deaa5740 1 client.15894 shutdown 2024-09-12T03:05:34.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.619+0000 7f32deaa5740 1 -- 172.21.15.116:0/3866556168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9ab936680 msgr2=0x55c9ab933840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:34.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.619+0000 7f32deaa5740 1 --2- 172.21.15.116:0/3866556168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9ab936680 0x55c9ab933840 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f32cc009f00 tx=0x7f32cc009c70 comp rx=0 tx=0).stop 2024-09-12T03:05:34.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.619+0000 7f32deaa5740 1 -- 172.21.15.116:0/3866556168 shutdown_connections 2024-09-12T03:05:34.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.619+0000 7f32deaa5740 1 --2- 172.21.15.116:0/3866556168 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c9ab946100 0x55c9ab9484f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.619+0000 7f32deaa5740 1 --2- 172.21.15.116:0/3866556168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9ab936680 0x55c9ab933840 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.619+0000 7f32deaa5740 1 --2- 172.21.15.116:0/3866556168 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9ab92bdc0 0x55c9ab9332c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.619+0000 7f32bdffb640 0 client.15894 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:34.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.619+0000 7f32deaa5740 1 --2- 172.21.15.116:0/3866556168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9ab92b4a0 0x55c9ab936010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.619+0000 7f32deaa5740 1 -- 172.21.15.116:0/3866556168 >> 172.21.15.116:0/3866556168 conn(0x55c9ab927a30 msgr2=0x55c9ab926b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:34.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.619+0000 7f32deaa5740 1 -- 172.21.15.116:0/3866556168 shutdown_connections 2024-09-12T03:05:34.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.619+0000 7f32deaa5740 1 -- 172.21.15.116:0/3866556168 wait complete. 2024-09-12T03:05:34.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.619+0000 7f32deaa5740 20 client.15894 trim_cache size 0 max 16384 2024-09-12T03:05:34.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.636+0000 7f32deaa5740 1 Processor -- start 2024-09-12T03:05:34.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.636+0000 7f32deaa5740 1 -- start start 2024-09-12T03:05:34.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.636+0000 7f32deaa5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9ab92b4a0 0x55c9ab9424b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:34.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.636+0000 7f32deaa5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9ab92bdc0 0x55c9ab94dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:34.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.636+0000 7f32deaa5740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9ab936680 0x55c9ab94e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:34.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.636+0000 7f32deaa5740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f32d41afe30 con 0x55c9ab936680 2024-09-12T03:05:34.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.636+0000 7f32deaa5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f32d41ad700 con 0x55c9ab92b4a0 2024-09-12T03:05:34.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.636+0000 7f32deaa5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f32d41ab570 con 0x55c9ab92bdc0 2024-09-12T03:05:34.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.636+0000 7f32dac69640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9ab936680 0x55c9ab94e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:34.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.636+0000 7f32dac69640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9ab936680 0x55c9ab94e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58020/0 (socket says 172.21.15.116:58020) 2024-09-12T03:05:34.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.636+0000 7f32dac69640 1 -- 172.21.15.116:0/36824773 learned_addr learned my addr 172.21.15.116:0/36824773 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:34.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.637+0000 7f32dac69640 1 -- 172.21.15.116:0/36824773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9ab92bdc0 msgr2=0x55c9ab94dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:34.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.637+0000 7f32dac69640 1 --2- 172.21.15.116:0/36824773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9ab92bdc0 0x55c9ab94dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.637+0000 7f32dac69640 1 -- 172.21.15.116:0/36824773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9ab92b4a0 msgr2=0x55c9ab9424b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:34.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.637+0000 7f32da468640 1 --2- 172.21.15.116:0/36824773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9ab92b4a0 0x55c9ab9424b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:34.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.637+0000 7f32dac69640 1 --2- 172.21.15.116:0/36824773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9ab92b4a0 0x55c9ab9424b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.637+0000 7f32dac69640 1 -- 172.21.15.116:0/36824773 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9ab928ae0 con 0x55c9ab936680 2024-09-12T03:05:34.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.637+0000 7f32dac69640 1 --2- 172.21.15.116:0/36824773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9ab936680 0x55c9ab94e150 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f32cc002230 tx=0x7f32cc0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da50e9be83269ee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:34.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.637+0000 7f32bffff640 1 -- 172.21.15.116:0/36824773 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32cc034030 con 0x55c9ab936680 2024-09-12T03:05:34.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.637+0000 7f32bffff640 1 -- 172.21.15.116:0/36824773 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32cc012c90 con 0x55c9ab936680 2024-09-12T03:05:34.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.637+0000 7f32bffff640 1 -- 172.21.15.116:0/36824773 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32cc005430 con 0x55c9ab936680 2024-09-12T03:05:34.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.638+0000 7f32deaa5740 1 -- 172.21.15.116:0/36824773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9ab936680 msgr2=0x55c9ab94e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:34.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.638+0000 7f32deaa5740 1 --2- 172.21.15.116:0/36824773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9ab936680 0x55c9ab94e150 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f32cc002230 tx=0x7f32cc0096c0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.638+0000 7f32deaa5740 1 -- 172.21.15.116:0/36824773 shutdown_connections 2024-09-12T03:05:34.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.638+0000 7f32deaa5740 1 --2- 172.21.15.116:0/36824773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9ab936680 0x55c9ab94e150 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.638+0000 7f32deaa5740 1 --2- 172.21.15.116:0/36824773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9ab92bdc0 0x55c9ab94dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.638+0000 7f32deaa5740 1 --2- 172.21.15.116:0/36824773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9ab92b4a0 0x55c9ab9424b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.638+0000 7f32deaa5740 1 -- 172.21.15.116:0/36824773 >> 172.21.15.116:0/36824773 conn(0x55c9ab927a30 msgr2=0x55c9ab9323b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:34.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.638+0000 7f32deaa5740 1 -- 172.21.15.116:0/36824773 shutdown_connections 2024-09-12T03:05:34.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.638+0000 7f32deaa5740 1 -- 172.21.15.116:0/36824773 wait complete. 2024-09-12T03:05:34.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.638+0000 7f32deaa5740 1 -- start start 2024-09-12T03:05:34.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.638+0000 7f32deaa5740 1 Processor -- start 2024-09-12T03:05:34.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.638+0000 7f32deaa5740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9ab92b4a0 0x55c9ab93ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:34.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.638+0000 7f32deaa5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9ab92bdc0 0x55c9ab939f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:34.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.638+0000 7f32deaa5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9ab936680 0x55c9ab93a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:34.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.639+0000 7f32deaa5740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f32d41abbc0 con 0x55c9ab92b4a0 2024-09-12T03:05:34.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.639+0000 7f32deaa5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f32d41ad030 con 0x55c9ab936680 2024-09-12T03:05:34.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.639+0000 7f32deaa5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f32d41ac290 con 0x55c9ab92bdc0 2024-09-12T03:05:34.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.639+0000 7f32deaa5740 10 client.0 _ll_register_callbacks cb 0x7f32cc009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:34.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.639+0000 7f32da468640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9ab92b4a0 0x55c9ab93ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:34.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.639+0000 7f32da468640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9ab92b4a0 0x55c9ab93ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58032/0 (socket says 172.21.15.116:58032) 2024-09-12T03:05:34.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.639+0000 7f32da468640 1 -- 172.21.15.116:0/2005303118 learned_addr learned my addr 172.21.15.116:0/2005303118 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:34.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.639+0000 7f32d9c67640 1 --2- 172.21.15.116:0/2005303118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9ab92bdc0 0x55c9ab939f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:34.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.639+0000 7f32dac69640 1 --2- 172.21.15.116:0/2005303118 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9ab936680 0x55c9ab93a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:34.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.639+0000 7f32da468640 1 -- 172.21.15.116:0/2005303118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9ab92bdc0 msgr2=0x55c9ab939f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:34.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.639+0000 7f32da468640 1 --2- 172.21.15.116:0/2005303118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9ab92bdc0 0x55c9ab939f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.639+0000 7f32da468640 1 -- 172.21.15.116:0/2005303118 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9ab936680 msgr2=0x55c9ab93a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:34.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.639+0000 7f32da468640 1 --2- 172.21.15.116:0/2005303118 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9ab936680 0x55c9ab93a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.639+0000 7f32da468640 1 -- 172.21.15.116:0/2005303118 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9ab93ab10 con 0x55c9ab92b4a0 2024-09-12T03:05:34.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.639+0000 7f32da468640 1 --2- 172.21.15.116:0/2005303118 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9ab92b4a0 0x55c9ab93ec00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f32c8031550 tx=0x7f32c8032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b6545e5de324f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:34.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.639+0000 7f32bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:05:34.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.639+0000 7f32bdffb640 1 -- 172.21.15.116:0/2005303118 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c8056060 con 0x55c9ab92b4a0 2024-09-12T03:05:34.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.639+0000 7f32bdffb640 1 -- 172.21.15.116:0/2005303118 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f32c8005040 con 0x55c9ab92b4a0 2024-09-12T03:05:34.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.639+0000 7f32deaa5740 1 -- 172.21.15.116:0/2005303118 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32cc012c90 con 0x55c9ab92b4a0 2024-09-12T03:05:34.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.640+0000 7f32bdffb640 1 -- 172.21.15.116:0/2005303118 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f32c8035050 con 0x55c9ab92b4a0 2024-09-12T03:05:34.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.640+0000 7f32deaa5740 10 client.15900 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:34.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.640+0000 7f32deaa5740 1 -- 172.21.15.116:0/2005303118 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c9ab9b0a80 con 0x55c9ab92b4a0 2024-09-12T03:05:34.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.640+0000 7f32bdffb640 1 -- 172.21.15.116:0/2005303118 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32cc012c90 con 0x55c9ab92b4a0 2024-09-12T03:05:34.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.640+0000 7f32bdffb640 1 -- 172.21.15.116:0/2005303118 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f32c80035f0 con 0x55c9ab92b4a0 2024-09-12T03:05:34.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.640+0000 7f32deaa5740 10 client.15900 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:34.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.640+0000 7f32deaa5740 10 client.15900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:05:34.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.640+0000 7f32deaa5740 20 client.15900 populate_metadata read hostname 'smithi116' 2024-09-12T03:05:34.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.640+0000 7f32deaa5740 1 --2- 172.21.15.116:0/2005303118 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c9ab9b1180 0x55c9ab9b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:34.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.640+0000 7f32deaa5740 4 client.15900 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:05:34.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.640+0000 7f32deaa5740 1 -- 172.21.15.116:0/2005303118 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c9ab898d60 con 0x55c9ab9b1180 2024-09-12T03:05:34.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.640+0000 7f32d9c67640 1 --2- 172.21.15.116:0/2005303118 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c9ab9b1180 0x55c9ab9b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:34.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.641+0000 7f32d9c67640 1 --2- 172.21.15.116:0/2005303118 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c9ab9b1180 0x55c9ab9b35a0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3130b0c422881a69 server_cookie=87d4009e3f9b5e8a in_seq=0 out_seq=0 2024-09-12T03:05:34.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.641+0000 7f32bdffb640 10 client.15900 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:34.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.641+0000 7f32bdffb640 1 -- 172.21.15.116:0/2005303118 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c9ab898d60 con 0x55c9ab9b1180 2024-09-12T03:05:34.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.641+0000 7f32bdffb640 10 client.15900 handle_command_reply: tid=0 2024-09-12T03:05:34.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.641+0000 7f32deaa5740 1 client.15900 shutdown 2024-09-12T03:05:34.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.641+0000 7f32deaa5740 1 -- 172.21.15.116:0/2005303118 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9ab92b4a0 msgr2=0x55c9ab93ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:34.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.641+0000 7f32deaa5740 1 --2- 172.21.15.116:0/2005303118 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9ab92b4a0 0x55c9ab93ec00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f32c8031550 tx=0x7f32c8032f90 comp rx=0 tx=0).stop 2024-09-12T03:05:34.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.641+0000 7f32deaa5740 1 -- 172.21.15.116:0/2005303118 shutdown_connections 2024-09-12T03:05:34.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.641+0000 7f32deaa5740 1 --2- 172.21.15.116:0/2005303118 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c9ab9b1180 0x55c9ab9b35a0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.641+0000 7f32deaa5740 1 --2- 172.21.15.116:0/2005303118 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9ab936680 0x55c9ab93a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.641+0000 7f32deaa5740 1 --2- 172.21.15.116:0/2005303118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9ab92bdc0 0x55c9ab939f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.641+0000 7f32deaa5740 1 --2- 172.21.15.116:0/2005303118 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9ab92b4a0 0x55c9ab93ec00 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.641+0000 7f32bdffb640 0 client.15900 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:34.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.641+0000 7f32deaa5740 1 -- 172.21.15.116:0/2005303118 >> 172.21.15.116:0/2005303118 conn(0x55c9ab927a30 msgr2=0x55c9ab934b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:34.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.641+0000 7f32deaa5740 1 -- 172.21.15.116:0/2005303118 shutdown_connections 2024-09-12T03:05:34.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.641+0000 7f32deaa5740 1 -- 172.21.15.116:0/2005303118 wait complete. 2024-09-12T03:05:34.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.641+0000 7f32deaa5740 20 client.15900 trim_cache size 0 max 16384 2024-09-12T03:05:34.661 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:05:34.661 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:05:34.661 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:05:34.661 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:05:34.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.642+0000 7f32dc6cc640 1 -- 172.21.15.116:0/1051878700 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f32c00a2c00 msgr2=0x7f32c00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:34.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.642+0000 7f32dc6cc640 1 --2- 172.21.15.116:0/1051878700 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f32c00a2c00 0x7f32c00a50c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f32c8000c00 tx=0x7f32c8005a40 comp rx=0 tx=0).stop 2024-09-12T03:05:34.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.642+0000 7f32dc6cc640 1 -- 172.21.15.116:0/1051878700 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32d4101420 msgr2=0x7f32d4199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:34.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.642+0000 7f32dc6cc640 1 --2- 172.21.15.116:0/1051878700 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32d4101420 0x7f32d4199830 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f32d000d950 tx=0x7f32d000de10 comp rx=0 tx=0).stop 2024-09-12T03:05:34.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.642+0000 7f32dc6cc640 1 -- 172.21.15.116:0/1051878700 shutdown_connections 2024-09-12T03:05:34.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.642+0000 7f32dc6cc640 1 --2- 172.21.15.116:0/1051878700 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f32c00a2c00 0x7f32c00a50c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.642+0000 7f32dc6cc640 1 --2- 172.21.15.116:0/1051878700 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32d4106a80 0x7f32d41a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.642+0000 7f32dc6cc640 1 --2- 172.21.15.116:0/1051878700 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32d4101420 0x7f32d4199830 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.642+0000 7f32dc6cc640 1 --2- 172.21.15.116:0/1051878700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32d4100a80 0x7f32d41992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:34.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.642+0000 7f32dc6cc640 1 -- 172.21.15.116:0/1051878700 >> 172.21.15.116:0/1051878700 conn(0x7f32d40fc7a0 msgr2=0x7f32d40fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:34.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.642+0000 7f32dc6cc640 1 -- 172.21.15.116:0/1051878700 shutdown_connections 2024-09-12T03:05:34.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:34.642+0000 7f32dc6cc640 1 -- 172.21.15.116:0/1051878700 wait complete. 2024-09-12T03:05:34.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:34 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:05:34.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:34 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:05:34.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:34 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:05:34.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:34 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:05:34.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:34 smithi183 ceph-mon[35535]: pgmap v415: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:05:34.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:34 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:05:34.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:34 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:05:34.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:34 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:05:34.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:34 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:05:34.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:34 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:05:34.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:34 smithi177 ceph-mon[35305]: pgmap v415: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:05:34.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:34 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:05:35.453 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:05:35.453 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:05:35.697 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:35.715 INFO:tasks.workunit.client.0.smithi116.stdout:1/600: open fd = 3 2024-09-12T03:05:35.715 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.716 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.716 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.716 INFO:tasks.workunit.client.0.smithi116.stdout:2/600: open fd = 3 2024-09-12T03:05:35.716 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.716 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.716 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.716 INFO:tasks.workunit.client.0.smithi116.stdout:3/600: open fd = 3 2024-09-12T03:05:35.716 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.716 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.716 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.716 INFO:tasks.workunit.client.0.smithi116.stdout:4/600: open fd = 3 2024-09-12T03:05:35.716 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.717 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.717 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.717 INFO:tasks.workunit.client.0.smithi116.stdout:5/600: open fd = 3 2024-09-12T03:05:35.717 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.717 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.717 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.717 INFO:tasks.workunit.client.0.smithi116.stdout:6/600: open fd = 3 2024-09-12T03:05:35.717 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.717 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.717 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.717 INFO:tasks.workunit.client.0.smithi116.stdout:7/600: open fd = 3 2024-09-12T03:05:35.717 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.717 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.718 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.718 INFO:tasks.workunit.client.0.smithi116.stdout:8/600: open fd = 3 2024-09-12T03:05:35.718 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.718 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.718 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.718 INFO:tasks.workunit.client.0.smithi116.stdout:9/600: open fd = 3 2024-09-12T03:05:35.718 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.718 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.718 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.718 INFO:tasks.workunit.client.0.smithi116.stdout:10/600: open fd = 3 2024-09-12T03:05:35.718 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.718 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.718 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.718 INFO:tasks.workunit.client.0.smithi116.stdout:11/600: open fd = 3 2024-09-12T03:05:35.719 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.719 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.719 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.719 INFO:tasks.workunit.client.0.smithi116.stdout:12/600: open fd = 3 2024-09-12T03:05:35.719 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.719 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.719 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.719 INFO:tasks.workunit.client.0.smithi116.stdout:13/600: open fd = 3 2024-09-12T03:05:35.719 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.719 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.719 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.719 INFO:tasks.workunit.client.0.smithi116.stdout:14/600: open fd = 3 2024-09-12T03:05:35.719 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.719 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.720 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.720 INFO:tasks.workunit.client.0.smithi116.stdout:15/600: open fd = 3 2024-09-12T03:05:35.720 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.720 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.720 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.720 INFO:tasks.workunit.client.0.smithi116.stdout:16/600: open fd = 3 2024-09-12T03:05:35.720 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.720 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.720 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.720 INFO:tasks.workunit.client.0.smithi116.stdout:17/600: open fd = 3 2024-09-12T03:05:35.720 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.720 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.720 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.721 INFO:tasks.workunit.client.0.smithi116.stdout:18/600: open fd = 3 2024-09-12T03:05:35.721 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.721 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.721 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.721 INFO:tasks.workunit.client.0.smithi116.stdout:19/600: open fd = 3 2024-09-12T03:05:35.721 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.721 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.721 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.721 INFO:tasks.workunit.client.0.smithi116.stdout:20/600: open fd = 3 2024-09-12T03:05:35.721 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.721 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.721 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.721 INFO:tasks.workunit.client.0.smithi116.stdout:21/600: open fd = 3 2024-09-12T03:05:35.722 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.722 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.722 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.722 INFO:tasks.workunit.client.0.smithi116.stdout:22/600: open fd = 3 2024-09-12T03:05:35.722 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.722 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.722 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.722 INFO:tasks.workunit.client.0.smithi116.stdout:23/600: open fd = 3 2024-09-12T03:05:35.722 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.722 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.722 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.722 INFO:tasks.workunit.client.0.smithi116.stdout:24/600: open fd = 3 2024-09-12T03:05:35.722 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.722 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.723 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.723 INFO:tasks.workunit.client.0.smithi116.stdout:25/600: open fd = 3 2024-09-12T03:05:35.723 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.723 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.723 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.723 INFO:tasks.workunit.client.0.smithi116.stdout:26/600: open fd = 3 2024-09-12T03:05:35.723 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.723 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.723 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.723 INFO:tasks.workunit.client.0.smithi116.stdout:27/600: open fd = 3 2024-09-12T03:05:35.723 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.723 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.723 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.724 INFO:tasks.workunit.client.0.smithi116.stdout:28/600: open fd = 3 2024-09-12T03:05:35.724 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.724 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.724 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.724 INFO:tasks.workunit.client.0.smithi116.stdout:29/600: open fd = 3 2024-09-12T03:05:35.724 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.724 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.724 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.724 INFO:tasks.workunit.client.0.smithi116.stdout:30/600: open fd = 3 2024-09-12T03:05:35.724 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.724 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.724 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.725 INFO:tasks.workunit.client.0.smithi116.stdout:31/600: open fd = 3 2024-09-12T03:05:35.725 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.725 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.725 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.725 INFO:tasks.workunit.client.0.smithi116.stdout:32/600: open fd = 3 2024-09-12T03:05:35.725 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.725 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.725 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.725 INFO:tasks.workunit.client.0.smithi116.stdout:33/600: open fd = 3 2024-09-12T03:05:35.725 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.725 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.725 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.725 INFO:tasks.workunit.client.0.smithi116.stdout:34/600: open fd = 3 2024-09-12T03:05:35.726 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.726 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.726 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.726 INFO:tasks.workunit.client.0.smithi116.stdout:35/600: open fd = 3 2024-09-12T03:05:35.726 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.726 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.726 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.726 INFO:tasks.workunit.client.0.smithi116.stdout:36/600: open fd = 3 2024-09-12T03:05:35.726 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.726 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.726 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.726 INFO:tasks.workunit.client.0.smithi116.stdout:37/600: open fd = 3 2024-09-12T03:05:35.726 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.727 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.727 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.727 INFO:tasks.workunit.client.0.smithi116.stdout:38/600: open fd = 3 2024-09-12T03:05:35.727 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.727 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.727 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.727 INFO:tasks.workunit.client.0.smithi116.stdout:39/600: open fd = 3 2024-09-12T03:05:35.727 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.727 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.727 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.727 INFO:tasks.workunit.client.0.smithi116.stdout:40/600: open fd = 3 2024-09-12T03:05:35.727 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.727 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.728 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.728 INFO:tasks.workunit.client.0.smithi116.stdout:41/600: open fd = 3 2024-09-12T03:05:35.728 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.728 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.728 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.728 INFO:tasks.workunit.client.0.smithi116.stdout:42/600: open fd = 3 2024-09-12T03:05:35.728 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.728 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.728 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.728 INFO:tasks.workunit.client.0.smithi116.stdout:43/600: open fd = 3 2024-09-12T03:05:35.728 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.728 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.728 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.728 INFO:tasks.workunit.client.0.smithi116.stdout:44/600: open fd = 3 2024-09-12T03:05:35.729 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.729 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.729 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.729 INFO:tasks.workunit.client.0.smithi116.stdout:45/600: open fd = 3 2024-09-12T03:05:35.729 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.729 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.729 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.729 INFO:tasks.workunit.client.0.smithi116.stdout:46/600: open fd = 3 2024-09-12T03:05:35.729 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.729 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.729 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.729 INFO:tasks.workunit.client.0.smithi116.stdout:47/600: open fd = 3 2024-09-12T03:05:35.729 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.730 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.730 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.730 INFO:tasks.workunit.client.0.smithi116.stdout:48/600: open fd = 3 2024-09-12T03:05:35.730 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.730 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.730 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.730 INFO:tasks.workunit.client.0.smithi116.stdout:49/600: open fd = 3 2024-09-12T03:05:35.730 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.730 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.730 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.730 INFO:tasks.workunit.client.0.smithi116.stdout:50/600: open fd = 3 2024-09-12T03:05:35.730 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.730 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.730 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.731 INFO:tasks.workunit.client.0.smithi116.stdout:51/600: open fd = 3 2024-09-12T03:05:35.731 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.731 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.731 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.731 INFO:tasks.workunit.client.0.smithi116.stdout:52/600: open fd = 3 2024-09-12T03:05:35.731 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.731 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.731 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.731 INFO:tasks.workunit.client.0.smithi116.stdout:53/600: open fd = 3 2024-09-12T03:05:35.731 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.731 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.731 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.731 INFO:tasks.workunit.client.0.smithi116.stdout:54/600: open fd = 3 2024-09-12T03:05:35.731 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.732 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.732 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.732 INFO:tasks.workunit.client.0.smithi116.stdout:55/600: open fd = 3 2024-09-12T03:05:35.732 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.732 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.732 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.732 INFO:tasks.workunit.client.0.smithi116.stdout:56/600: open fd = 3 2024-09-12T03:05:35.732 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.732 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.732 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.732 INFO:tasks.workunit.client.0.smithi116.stdout:57/600: open fd = 3 2024-09-12T03:05:35.732 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.732 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.732 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.733 INFO:tasks.workunit.client.0.smithi116.stdout:58/600: open fd = 3 2024-09-12T03:05:35.733 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.733 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.733 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.733 INFO:tasks.workunit.client.0.smithi116.stdout:59/600: open fd = 3 2024-09-12T03:05:35.733 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.733 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.733 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.733 INFO:tasks.workunit.client.0.smithi116.stdout:60/600: open fd = 3 2024-09-12T03:05:35.734 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.734 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.734 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.734 INFO:tasks.workunit.client.0.smithi116.stdout:61/600: open fd = 3 2024-09-12T03:05:35.734 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.734 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.734 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.734 INFO:tasks.workunit.client.0.smithi116.stdout:62/600: open fd = 3 2024-09-12T03:05:35.734 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.734 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.734 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:35.734 INFO:tasks.workunit.client.0.smithi116.stdout:63/600: open fd = 3 2024-09-12T03:05:35.734 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.734 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:05:35.735 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:05:36.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.167+0000 7fd62c055640 1 -- 172.21.15.116:0/3570063372 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd624106f10 msgr2=0x7fd62410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:36.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.167+0000 7fd62c055640 1 --2- 172.21.15.116:0/3570063372 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd624106f10 0x7fd62410ea90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009ca0 tx=0x7fd61c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:05:36.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.167+0000 7fd62c055640 1 -- 172.21.15.116:0/3570063372 shutdown_connections 2024-09-12T03:05:36.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.167+0000 7fd62c055640 1 --2- 172.21.15.116:0/3570063372 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd624106f10 0x7fd62410ea90 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:36.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.167+0000 7fd62c055640 1 --2- 172.21.15.116:0/3570063372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd624102790 0x7fd6241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:36.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.167+0000 7fd62c055640 1 --2- 172.21.15.116:0/3570063372 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd624101de0 0x7fd6241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:36.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.167+0000 7fd62c055640 1 -- 172.21.15.116:0/3570063372 >> 172.21.15.116:0/3570063372 conn(0x7fd6240fd8c0 msgr2=0x7fd6240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:36.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.167+0000 7fd62c055640 1 -- 172.21.15.116:0/3570063372 shutdown_connections 2024-09-12T03:05:36.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.167+0000 7fd62c055640 1 -- 172.21.15.116:0/3570063372 wait complete. 2024-09-12T03:05:36.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.168+0000 7fd62c055640 1 Processor -- start 2024-09-12T03:05:36.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.168+0000 7fd62c055640 1 -- start start 2024-09-12T03:05:36.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.168+0000 7fd62c055640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd624101de0 0x7fd6240744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:36.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.168+0000 7fd62c055640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd624102790 0x7fd624074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:36.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.168+0000 7fd62c055640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd624106f10 0x7fd624074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:36.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.168+0000 7fd62c055640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd624111fb0 con 0x7fd624102790 2024-09-12T03:05:36.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.168+0000 7fd62c055640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd624111e30 con 0x7fd624106f10 2024-09-12T03:05:36.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.168+0000 7fd62c055640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd624112130 con 0x7fd624101de0 2024-09-12T03:05:36.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.168+0000 7fd6295f0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd624102790 0x7fd624074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:36.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.168+0000 7fd6295f0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd624102790 0x7fd624074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58056/0 (socket says 172.21.15.116:58056) 2024-09-12T03:05:36.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.168+0000 7fd6295f0640 1 -- 172.21.15.116:0/3032041168 learned_addr learned my addr 172.21.15.116:0/3032041168 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:36.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.169+0000 7fd62a5f2640 1 --2- 172.21.15.116:0/3032041168 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd624106f10 0x7fd624074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:36.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.169+0000 7fd629df1640 1 --2- 172.21.15.116:0/3032041168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd624101de0 0x7fd6240744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:36.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.169+0000 7fd6295f0640 1 -- 172.21.15.116:0/3032041168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd624101de0 msgr2=0x7fd6240744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:36.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.169+0000 7fd6295f0640 1 --2- 172.21.15.116:0/3032041168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd624101de0 0x7fd6240744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:36.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.169+0000 7fd6295f0640 1 -- 172.21.15.116:0/3032041168 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd624106f10 msgr2=0x7fd624074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:36.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.169+0000 7fd6295f0640 1 --2- 172.21.15.116:0/3032041168 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd624106f10 0x7fd624074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:36.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.169+0000 7fd6295f0640 1 -- 172.21.15.116:0/3032041168 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6241a6950 con 0x7fd624102790 2024-09-12T03:05:36.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.169+0000 7fd6295f0640 1 --2- 172.21.15.116:0/3032041168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd624102790 0x7fd624074a30 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd61400c970 tx=0x7fd61400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:36.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.169+0000 7fd61affd640 1 -- 172.21.15.116:0/3032041168 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd614007bb0 con 0x7fd624102790 2024-09-12T03:05:36.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.169+0000 7fd61affd640 1 -- 172.21.15.116:0/3032041168 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd614007d50 con 0x7fd624102790 2024-09-12T03:05:36.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.169+0000 7fd62c055640 1 -- 172.21.15.116:0/3032041168 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6241a6c40 con 0x7fd624102790 2024-09-12T03:05:36.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.169+0000 7fd61affd640 1 -- 172.21.15.116:0/3032041168 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61400f450 con 0x7fd624102790 2024-09-12T03:05:36.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.169+0000 7fd62c055640 1 -- 172.21.15.116:0/3032041168 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6241a7130 con 0x7fd624102790 2024-09-12T03:05:36.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.170+0000 7fd62c055640 1 -- 172.21.15.116:0/3032041168 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6241021c0 con 0x7fd624102790 2024-09-12T03:05:36.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.171+0000 7fd61affd640 1 -- 172.21.15.116:0/3032041168 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd614005020 con 0x7fd624102790 2024-09-12T03:05:36.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.172+0000 7fd61affd640 1 --2- 172.21.15.116:0/3032041168 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd6100a2b80 0x7fd6100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:36.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.172+0000 7fd61affd640 1 -- 172.21.15.116:0/3032041168 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd6140ceeb0 con 0x7fd624102790 2024-09-12T03:05:36.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.172+0000 7fd629df1640 1 --2- 172.21.15.116:0/3032041168 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd6100a2b80 0x7fd6100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:36.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.172+0000 7fd629df1640 1 --2- 172.21.15.116:0/3032041168 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd6100a2b80 0x7fd6100a5040 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd620009890 tx=0x7fd620008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:36.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.176+0000 7fd61affd640 1 -- 172.21.15.116:0/3032041168 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd614014030 con 0x7fd624102790 2024-09-12T03:05:36.322 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.7 2024-09-12T03:05:36.322 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:05:36.345 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:36 smithi116 ceph-mon[31410]: pgmap v416: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:05:36.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.383+0000 7fd62c055640 1 -- 172.21.15.116:0/3032041168 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd62410a0f0 con 0x7fd624102790 2024-09-12T03:05:36.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.383+0000 7fd61affd640 1 -- 172.21.15.116:0/3032041168 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7fd61409be40 con 0x7fd624102790 2024-09-12T03:05:36.384 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:05:36.384 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:05:36.384 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:05:36.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.386+0000 7fd62c055640 1 -- 172.21.15.116:0/3032041168 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd6100a2b80 msgr2=0x7fd6100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:36.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.386+0000 7fd62c055640 1 --2- 172.21.15.116:0/3032041168 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd6100a2b80 0x7fd6100a5040 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd620009890 tx=0x7fd620008040 comp rx=0 tx=0).stop 2024-09-12T03:05:36.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.386+0000 7fd62c055640 1 -- 172.21.15.116:0/3032041168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd624102790 msgr2=0x7fd624074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:36.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.386+0000 7fd62c055640 1 --2- 172.21.15.116:0/3032041168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd624102790 0x7fd624074a30 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd61400c970 tx=0x7fd61400ce30 comp rx=0 tx=0).stop 2024-09-12T03:05:36.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.386+0000 7fd62c055640 1 -- 172.21.15.116:0/3032041168 shutdown_connections 2024-09-12T03:05:36.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.386+0000 7fd62c055640 1 --2- 172.21.15.116:0/3032041168 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd6100a2b80 0x7fd6100a5040 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:36.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.386+0000 7fd62c055640 1 --2- 172.21.15.116:0/3032041168 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd624106f10 0x7fd624074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:36.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.386+0000 7fd62c055640 1 --2- 172.21.15.116:0/3032041168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd624102790 0x7fd624074a30 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:36.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.386+0000 7fd62c055640 1 --2- 172.21.15.116:0/3032041168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd624101de0 0x7fd6240744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:36.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.386+0000 7fd62c055640 1 -- 172.21.15.116:0/3032041168 >> 172.21.15.116:0/3032041168 conn(0x7fd6240fd8c0 msgr2=0x7fd6240fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:36.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.386+0000 7fd62c055640 1 -- 172.21.15.116:0/3032041168 shutdown_connections 2024-09-12T03:05:36.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:36.386+0000 7fd62c055640 1 -- 172.21.15.116:0/3032041168 wait complete. 2024-09-12T03:05:36.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:36 smithi183 ceph-mon[35535]: pgmap v416: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:05:36.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:36 smithi177 ceph-mon[35305]: pgmap v416: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:05:37.094 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:05:37.112 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:37.378 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:37 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3032041168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:05:37.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.600+0000 7f40a4647640 1 -- 172.21.15.116:0/590015354 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f409c101240 msgr2=0x7f409c101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:37.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.600+0000 7f40a4647640 1 --2- 172.21.15.116:0/590015354 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f409c101240 0x7f409c101680 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4098009900 tx=0x7f409802a410 comp rx=0 tx=0).stop 2024-09-12T03:05:37.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.600+0000 7f40a4647640 1 -- 172.21.15.116:0/590015354 shutdown_connections 2024-09-12T03:05:37.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.600+0000 7f40a4647640 1 --2- 172.21.15.116:0/590015354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f409c101bc0 0x7f409c10d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:37.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.600+0000 7f40a4647640 1 --2- 172.21.15.116:0/590015354 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f409c101240 0x7f409c101680 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:37.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.600+0000 7f40a4647640 1 --2- 172.21.15.116:0/590015354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f409c105a90 0x7f409c105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:37.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.600+0000 7f40a4647640 1 -- 172.21.15.116:0/590015354 >> 172.21.15.116:0/590015354 conn(0x7f409c077210 msgr2=0x7f409c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:37.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.600+0000 7f40a4647640 1 -- 172.21.15.116:0/590015354 shutdown_connections 2024-09-12T03:05:37.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.600+0000 7f40a4647640 1 -- 172.21.15.116:0/590015354 wait complete. 2024-09-12T03:05:37.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.601+0000 7f40a4647640 1 Processor -- start 2024-09-12T03:05:37.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.601+0000 7f40a4647640 1 -- start start 2024-09-12T03:05:37.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.601+0000 7f40a4647640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f409c101240 0x7f409c0fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:37.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.601+0000 7f40a4647640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f409c101bc0 0x7f409c0fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:37.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.601+0000 7f40a4647640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f409c105a90 0x7f409c0fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:37.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.601+0000 7f40a4647640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f409c111030 con 0x7f409c101bc0 2024-09-12T03:05:37.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.601+0000 7f40a4647640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f409c110eb0 con 0x7f409c105a90 2024-09-12T03:05:37.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.601+0000 7f40a4647640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1111b0 con 0x7f409c101240 2024-09-12T03:05:37.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.601+0000 7f40a1be2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f409c101bc0 0x7f409c0fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:37.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.601+0000 7f40a1be2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f409c101bc0 0x7f409c0fcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58064/0 (socket says 172.21.15.116:58064) 2024-09-12T03:05:37.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.601+0000 7f40a1be2640 1 -- 172.21.15.116:0/3991610941 learned_addr learned my addr 172.21.15.116:0/3991610941 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:37.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.601+0000 7f40a23e3640 1 --2- 172.21.15.116:0/3991610941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f409c101240 0x7f409c0fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:37.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.601+0000 7f40a2be4640 1 --2- 172.21.15.116:0/3991610941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f409c105a90 0x7f409c0fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:37.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.601+0000 7f40a1be2640 1 -- 172.21.15.116:0/3991610941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f409c101240 msgr2=0x7f409c0fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:37.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.601+0000 7f40a1be2640 1 --2- 172.21.15.116:0/3991610941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f409c101240 0x7f409c0fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:37.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.601+0000 7f40a1be2640 1 -- 172.21.15.116:0/3991610941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f409c105a90 msgr2=0x7f409c0fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:37.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.601+0000 7f40a1be2640 1 --2- 172.21.15.116:0/3991610941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f409c105a90 0x7f409c0fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:37.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.601+0000 7f40a1be2640 1 -- 172.21.15.116:0/3991610941 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409c1a6a60 con 0x7f409c101bc0 2024-09-12T03:05:37.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.601+0000 7f40a2be4640 1 --2- 172.21.15.116:0/3991610941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f409c105a90 0x7f409c0fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:37.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.601+0000 7f40a1be2640 1 --2- 172.21.15.116:0/3991610941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f409c101bc0 0x7f409c0fcbd0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4098006010 tx=0x7f4098003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:37.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.602+0000 7f408f7fe640 1 -- 172.21.15.116:0/3991610941 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409803f070 con 0x7f409c101bc0 2024-09-12T03:05:37.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.602+0000 7f408f7fe640 1 -- 172.21.15.116:0/3991610941 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4098031e30 con 0x7f409c101bc0 2024-09-12T03:05:37.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.602+0000 7f40a4647640 1 -- 172.21.15.116:0/3991610941 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f409c1a6cf0 con 0x7f409c101bc0 2024-09-12T03:05:37.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.602+0000 7f408f7fe640 1 -- 172.21.15.116:0/3991610941 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4098035030 con 0x7f409c101bc0 2024-09-12T03:05:37.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.602+0000 7f40a4647640 1 -- 172.21.15.116:0/3991610941 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f409c1a7180 con 0x7f409c101bc0 2024-09-12T03:05:37.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.603+0000 7f40a4647640 1 -- 172.21.15.116:0/3991610941 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f409c101680 con 0x7f409c101bc0 2024-09-12T03:05:37.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.604+0000 7f408f7fe640 1 -- 172.21.15.116:0/3991610941 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4098042070 con 0x7f409c101bc0 2024-09-12T03:05:37.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.604+0000 7f408f7fe640 1 --2- 172.21.15.116:0/3991610941 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f40880a2b80 0x7f40880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:37.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.605+0000 7f40a23e3640 1 --2- 172.21.15.116:0/3991610941 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f40880a2b80 0x7f40880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:37.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.605+0000 7f408f7fe640 1 -- 172.21.15.116:0/3991610941 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f40980ef900 con 0x7f409c101bc0 2024-09-12T03:05:37.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.605+0000 7f40a23e3640 1 --2- 172.21.15.116:0/3991610941 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f40880a2b80 0x7f40880a5040 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4090009990 tx=0x7f4090009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:37.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.609+0000 7f408f7fe640 1 -- 172.21.15.116:0/3991610941 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f40980bc890 con 0x7f409c101bc0 2024-09-12T03:05:37.681 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:37.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:37 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3032041168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:05:37.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:37 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3032041168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:05:37.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.826+0000 7f40a4647640 1 -- 172.21.15.116:0/3991610941 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f409c1a73b0 con 0x7f409c101bc0 2024-09-12T03:05:37.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.827+0000 7f408f7fe640 1 -- 172.21.15.116:0/3991610941 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f40980bc190 con 0x7f409c101bc0 2024-09-12T03:05:37.828 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:05:37.828 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:05:37.828 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:05:37.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.829+0000 7f40a4647640 1 -- 172.21.15.116:0/3991610941 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f40880a2b80 msgr2=0x7f40880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:37.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.829+0000 7f40a4647640 1 --2- 172.21.15.116:0/3991610941 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f40880a2b80 0x7f40880a5040 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4090009990 tx=0x7f4090009110 comp rx=0 tx=0).stop 2024-09-12T03:05:37.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.829+0000 7f40a4647640 1 -- 172.21.15.116:0/3991610941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f409c101bc0 msgr2=0x7f409c0fcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:37.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.829+0000 7f40a4647640 1 --2- 172.21.15.116:0/3991610941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f409c101bc0 0x7f409c0fcbd0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4098006010 tx=0x7f4098003c80 comp rx=0 tx=0).stop 2024-09-12T03:05:37.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.829+0000 7f40a4647640 1 -- 172.21.15.116:0/3991610941 shutdown_connections 2024-09-12T03:05:37.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.829+0000 7f40a4647640 1 --2- 172.21.15.116:0/3991610941 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f40880a2b80 0x7f40880a5040 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:37.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.829+0000 7f40a4647640 1 --2- 172.21.15.116:0/3991610941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f409c105a90 0x7f409c0fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:37.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.829+0000 7f40a4647640 1 --2- 172.21.15.116:0/3991610941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f409c101bc0 0x7f409c0fcbd0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:37.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.829+0000 7f40a4647640 1 --2- 172.21.15.116:0/3991610941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f409c101240 0x7f409c0fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:37.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.829+0000 7f40a4647640 1 -- 172.21.15.116:0/3991610941 >> 172.21.15.116:0/3991610941 conn(0x7f409c077210 msgr2=0x7f409c100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:37.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.829+0000 7f40a4647640 1 -- 172.21.15.116:0/3991610941 shutdown_connections 2024-09-12T03:05:37.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:37.829+0000 7f40a4647640 1 -- 172.21.15.116:0/3991610941 wait complete. 2024-09-12T03:05:38.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:38 smithi116 ceph-mon[31410]: pgmap v417: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:05:38.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:38 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3991610941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:05:38.581 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0' 2024-09-12T03:05:38.581 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0 2024-09-12T03:05:38.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:38 smithi183 ceph-mon[35535]: pgmap v417: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:05:38.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:38 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3991610941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:05:38.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:38 smithi177 ceph-mon[35305]: pgmap v417: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:05:38.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:38 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3991610941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:05:38.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.801+0000 7fcd806e2640 1 -- 172.21.15.116:0/326895189 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd78101df0 msgr2=0x7fcd781021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:38.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.801+0000 7fcd806e2640 1 --2- 172.21.15.116:0/326895189 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd78101df0 0x7fcd781021d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fcd70009900 tx=0x7fcd7002a410 comp rx=0 tx=0).stop 2024-09-12T03:05:38.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.801+0000 7fcd806e2640 1 -- 172.21.15.116:0/326895189 shutdown_connections 2024-09-12T03:05:38.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.801+0000 7fcd806e2640 1 --2- 172.21.15.116:0/326895189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd78106f20 0x7fcd7810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.801+0000 7fcd806e2640 1 --2- 172.21.15.116:0/326895189 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd781027a0 0x7fcd781067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.801+0000 7fcd806e2640 1 --2- 172.21.15.116:0/326895189 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd78101df0 0x7fcd781021d0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.801+0000 7fcd806e2640 1 -- 172.21.15.116:0/326895189 >> 172.21.15.116:0/326895189 conn(0x7fcd780fd8b0 msgr2=0x7fcd780ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:38.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.801+0000 7fcd806e2640 1 -- 172.21.15.116:0/326895189 shutdown_connections 2024-09-12T03:05:38.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.801+0000 7fcd806e2640 1 -- 172.21.15.116:0/326895189 wait complete. 2024-09-12T03:05:38.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.802+0000 7fcd806e2640 1 Processor -- start 2024-09-12T03:05:38.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.802+0000 7fcd806e2640 1 -- start start 2024-09-12T03:05:38.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.802+0000 7fcd806e2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd78101df0 0x7fcd78198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:38.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.802+0000 7fcd806e2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd781027a0 0x7fcd78198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:38.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.802+0000 7fcd806e2640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd78106f20 0x7fcd7819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:38.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.802+0000 7fcd806e2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcd78112020 con 0x7fcd78101df0 2024-09-12T03:05:38.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.802+0000 7fcd806e2640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcd78111ea0 con 0x7fcd78106f20 2024-09-12T03:05:38.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.802+0000 7fcd806e2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcd781121a0 con 0x7fcd781027a0 2024-09-12T03:05:38.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.802+0000 7fcd7e47e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd78101df0 0x7fcd78198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:38.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.802+0000 7fcd7e47e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd78101df0 0x7fcd78198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58082/0 (socket says 172.21.15.116:58082) 2024-09-12T03:05:38.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.802+0000 7fcd7e47e640 1 -- 172.21.15.116:0/3834293601 learned_addr learned my addr 172.21.15.116:0/3834293601 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:38.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.802+0000 7fcd7dc7d640 1 --2- 172.21.15.116:0/3834293601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd781027a0 0x7fcd78198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:38.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.802+0000 7fcd7ec7f640 1 --2- 172.21.15.116:0/3834293601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd78106f20 0x7fcd7819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:38.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.802+0000 7fcd7e47e640 1 -- 172.21.15.116:0/3834293601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd781027a0 msgr2=0x7fcd78198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:38.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.802+0000 7fcd7e47e640 1 --2- 172.21.15.116:0/3834293601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd781027a0 0x7fcd78198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.802+0000 7fcd7e47e640 1 -- 172.21.15.116:0/3834293601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd78106f20 msgr2=0x7fcd7819cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:38.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.802+0000 7fcd7e47e640 1 --2- 172.21.15.116:0/3834293601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd78106f20 0x7fcd7819cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.802+0000 7fcd7e47e640 1 -- 172.21.15.116:0/3834293601 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7819d5f0 con 0x7fcd78101df0 2024-09-12T03:05:38.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.802+0000 7fcd7dc7d640 1 --2- 172.21.15.116:0/3834293601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd781027a0 0x7fcd78198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:38.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.802+0000 7fcd7e47e640 1 --2- 172.21.15.116:0/3834293601 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd78101df0 0x7fcd78198640 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fcd70005ab0 tx=0x7fcd700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:38.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.803+0000 7fcd777fe640 1 -- 172.21.15.116:0/3834293601 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7003f030 con 0x7fcd78101df0 2024-09-12T03:05:38.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.803+0000 7fcd806e2640 1 -- 172.21.15.116:0/3834293601 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7819d880 con 0x7fcd78101df0 2024-09-12T03:05:38.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.803+0000 7fcd777fe640 1 -- 172.21.15.116:0/3834293601 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd70003ce0 con 0x7fcd78101df0 2024-09-12T03:05:38.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.803+0000 7fcd777fe640 1 -- 172.21.15.116:0/3834293601 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd70035070 con 0x7fcd78101df0 2024-09-12T03:05:38.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.803+0000 7fcd806e2640 1 -- 172.21.15.116:0/3834293601 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd781a51c0 con 0x7fcd78101df0 2024-09-12T03:05:38.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.805+0000 7fcd777fe640 1 -- 172.21.15.116:0/3834293601 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fcd70042030 con 0x7fcd78101df0 2024-09-12T03:05:38.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.805+0000 7fcd82abb740 1 Processor -- start 2024-09-12T03:05:38.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.805+0000 7fcd82abb740 1 -- start start 2024-09-12T03:05:38.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.806+0000 7fcd82abb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564e31df2440 0x564e31df2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:38.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.806+0000 7fcd82abb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e31df2df0 0x564e31df7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:38.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.806+0000 7fcd82abb740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564e31df7a40 0x564e31df7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:38.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.806+0000 7fcd82abb740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x564e31d4e2f0 con 0x564e31df7a40 2024-09-12T03:05:38.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.806+0000 7fcd82abb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x564e31cf5cf0 con 0x564e31df2440 2024-09-12T03:05:38.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.806+0000 7fcd82abb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564e31d4a2b0 con 0x564e31df2df0 2024-09-12T03:05:38.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.806+0000 7fcd777fe640 1 --2- 172.21.15.116:0/3834293601 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcd600a2b80 0x7fcd600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:38.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.806+0000 7fcd7ec7f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564e31df2440 0x564e31df2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:38.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.806+0000 7fcd7ec7f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564e31df2440 0x564e31df2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.116:47430/0 (socket says 172.21.15.116:47430) 2024-09-12T03:05:38.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.806+0000 7fcd7ec7f640 1 -- 172.21.15.116:0/2581142473 learned_addr learned my addr 172.21.15.116:0/2581142473 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:38.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.806+0000 7fcd7ec7f640 1 --2- 172.21.15.116:0/3834293601 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcd600a2b80 0x7fcd600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:38.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.806+0000 7fcd7dc7d640 1 --2- 172.21.15.116:0/2581142473 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564e31df7a40 0x564e31df7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:38.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.806+0000 7fcd777fe640 1 -- 172.21.15.116:0/3834293601 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fcd700ef9b0 con 0x7fcd78101df0 2024-09-12T03:05:38.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.806+0000 7fcd7e47e640 1 --2- 172.21.15.116:0/2581142473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e31df2df0 0x564e31df7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:38.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.806+0000 7fcd7ec7f640 1 -- 172.21.15.116:0/2581142473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e31df2df0 msgr2=0x564e31df7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:38.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.806+0000 7fcd7ec7f640 1 --2- 172.21.15.116:0/2581142473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e31df2df0 0x564e31df7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.807+0000 7fcd7ec7f640 1 -- 172.21.15.116:0/2581142473 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564e31df7a40 msgr2=0x564e31df7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:38.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.807+0000 7fcd7ec7f640 1 --2- 172.21.15.116:0/2581142473 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564e31df7a40 0x564e31df7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.807+0000 7fcd7ec7f640 1 -- 172.21.15.116:0/2581142473 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e31cda3a0 con 0x564e31df2440 2024-09-12T03:05:38.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.807+0000 7fcd7e47e640 1 --2- 172.21.15.116:0/2581142473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e31df2df0 0x564e31df7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:38.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.807+0000 7fcd7dc7d640 1 --2- 172.21.15.116:0/2581142473 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564e31df7a40 0x564e31df7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:38.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.807+0000 7fcd7ec7f640 1 --2- 172.21.15.116:0/3834293601 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcd600a2b80 0x7fcd600a5040 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcd68006fd0 tx=0x7fcd6800a520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:38.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.807+0000 7fcd7ec7f640 1 --2- 172.21.15.116:0/2581142473 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564e31df2440 0x564e31df2820 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcd68016fd0 tx=0x7fcd68019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66ba30ca78cb6ba7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:38.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.807+0000 7fcd74ff9640 1 -- 172.21.15.116:0/2581142473 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68018450 con 0x564e31df2440 2024-09-12T03:05:38.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.807+0000 7fcd74ff9640 1 -- 172.21.15.116:0/2581142473 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd6801c070 con 0x564e31df2440 2024-09-12T03:05:38.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.807+0000 7fcd74ff9640 1 -- 172.21.15.116:0/2581142473 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68020750 con 0x564e31df2440 2024-09-12T03:05:38.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.808+0000 7fcd82abb740 1 -- 172.21.15.116:0/2581142473 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564e31df2440 msgr2=0x564e31df2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:38.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.808+0000 7fcd82abb740 1 --2- 172.21.15.116:0/2581142473 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564e31df2440 0x564e31df2820 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcd68016fd0 tx=0x7fcd68019000 comp rx=0 tx=0).stop 2024-09-12T03:05:38.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.808+0000 7fcd82abb740 1 -- 172.21.15.116:0/2581142473 shutdown_connections 2024-09-12T03:05:38.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.808+0000 7fcd82abb740 1 --2- 172.21.15.116:0/2581142473 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564e31df7a40 0x564e31df7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.808+0000 7fcd82abb740 1 --2- 172.21.15.116:0/2581142473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e31df2df0 0x564e31df7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.808+0000 7fcd82abb740 1 --2- 172.21.15.116:0/2581142473 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564e31df2440 0x564e31df2820 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.808+0000 7fcd82abb740 1 -- 172.21.15.116:0/2581142473 >> 172.21.15.116:0/2581142473 conn(0x564e31db1310 msgr2=0x564e31de46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:38.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.808+0000 7fcd82abb740 1 -- 172.21.15.116:0/2581142473 shutdown_connections 2024-09-12T03:05:38.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.808+0000 7fcd82abb740 1 -- 172.21.15.116:0/2581142473 wait complete. 2024-09-12T03:05:38.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.808+0000 7fcd82abb740 1 -- start start 2024-09-12T03:05:38.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.808+0000 7fcd82abb740 1 Processor -- start 2024-09-12T03:05:38.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.809+0000 7fcd82abb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e31df2440 0x564e31dfde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:38.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.809+0000 7fcd82abb740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564e31df2df0 0x564e31dfb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:38.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.809+0000 7fcd82abb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564e31df7a40 0x564e31dfb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:38.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.809+0000 7fcd82abb740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x564e31dfe810 con 0x564e31df2df0 2024-09-12T03:05:38.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.809+0000 7fcd82abb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x564e31d09060 con 0x564e31df7a40 2024-09-12T03:05:38.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.809+0000 7fcd82abb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564e31dfe990 con 0x564e31df2440 2024-09-12T03:05:38.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.809+0000 7fcd82abb740 10 client.0 _ll_register_callbacks cb 0x564e31a5c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:38.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.809+0000 7fcd7dc7d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564e31df2df0 0x564e31dfb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:38.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.809+0000 7fcd7dc7d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564e31df2df0 0x564e31dfb0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58096/0 (socket says 172.21.15.116:58096) 2024-09-12T03:05:38.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.809+0000 7fcd7dc7d640 1 -- 172.21.15.116:0/1450912728 learned_addr learned my addr 172.21.15.116:0/1450912728 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:38.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.809+0000 7fcd7ec7f640 1 --2- 172.21.15.116:0/1450912728 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564e31df7a40 0x564e31dfb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:38.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.809+0000 7fcd7dc7d640 1 -- 172.21.15.116:0/1450912728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e31df2440 msgr2=0x564e31dfde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:38.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.809+0000 7fcd7dc7d640 1 --2- 172.21.15.116:0/1450912728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e31df2440 0x564e31dfde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.809+0000 7fcd7dc7d640 1 -- 172.21.15.116:0/1450912728 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564e31df7a40 msgr2=0x564e31dfb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:38.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.809+0000 7fcd7dc7d640 1 --2- 172.21.15.116:0/1450912728 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564e31df7a40 0x564e31dfb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.809+0000 7fcd7dc7d640 1 -- 172.21.15.116:0/1450912728 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e31abf3c0 con 0x564e31df2df0 2024-09-12T03:05:38.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.809+0000 7fcd7dc7d640 1 --2- 172.21.15.116:0/1450912728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564e31df2df0 0x564e31dfb0e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00cdf0 tx=0x7fcd6c007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3c0e1e58980dc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:38.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.809+0000 7fcd5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:05:38.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.809+0000 7fcd5dffb640 1 -- 172.21.15.116:0/1450912728 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c01c060 con 0x564e31df2df0 2024-09-12T03:05:38.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.809+0000 7fcd82abb740 1 -- 172.21.15.116:0/1450912728 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd6801c070 con 0x564e31df2df0 2024-09-12T03:05:38.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.809+0000 7fcd5dffb640 1 -- 172.21.15.116:0/1450912728 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fcd6c00c0d0 con 0x564e31df2df0 2024-09-12T03:05:38.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.809+0000 7fcd5dffb640 1 -- 172.21.15.116:0/1450912728 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcd6c017420 con 0x564e31df2df0 2024-09-12T03:05:38.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.809+0000 7fcd82abb740 10 client.15924 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:38.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.809+0000 7fcd82abb740 1 -- 172.21.15.116:0/1450912728 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564e31e0a2d0 con 0x564e31df2df0 2024-09-12T03:05:38.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.810+0000 7fcd5dffb640 1 -- 172.21.15.116:0/1450912728 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd6801c070 con 0x564e31df2df0 2024-09-12T03:05:38.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.810+0000 7fcd5dffb640 1 -- 172.21.15.116:0/1450912728 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcd6c022440 con 0x564e31df2df0 2024-09-12T03:05:38.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.810+0000 7fcd82abb740 10 client.15924 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:38.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.810+0000 7fcd82abb740 10 client.15924 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:05:38.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.810+0000 7fcd82abb740 20 client.15924 populate_metadata read hostname 'smithi116' 2024-09-12T03:05:38.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.810+0000 7fcd82abb740 1 --2- 172.21.15.116:0/1450912728 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x564e31e0b730 0x564e31e0db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:38.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.810+0000 7fcd82abb740 4 client.15924 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:05:38.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.810+0000 7fcd82abb740 1 -- 172.21.15.116:0/1450912728 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e31d5b860 con 0x564e31e0b730 2024-09-12T03:05:38.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.810+0000 7fcd7e47e640 1 --2- 172.21.15.116:0/1450912728 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x564e31e0b730 0x564e31e0db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:38.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.811+0000 7fcd5dffb640 10 client.15924 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:05:38.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.811+0000 7fcd7e47e640 1 --2- 172.21.15.116:0/1450912728 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x564e31e0b730 0x564e31e0db70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9bc80729637eb9d5 server_cookie=1005b36a0c756cba in_seq=0 out_seq=0 2024-09-12T03:05:38.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.811+0000 7fcd5dffb640 1 -- 172.21.15.116:0/1450912728 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564e31d5b860 con 0x564e31e0b730 2024-09-12T03:05:38.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.811+0000 7fcd5dffb640 10 client.15924 handle_command_reply: tid=0 2024-09-12T03:05:38.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.811+0000 7fcd82abb740 1 client.15924 shutdown 2024-09-12T03:05:38.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.811+0000 7fcd82abb740 1 -- 172.21.15.116:0/1450912728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564e31df2df0 msgr2=0x564e31dfb0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:38.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.811+0000 7fcd82abb740 1 --2- 172.21.15.116:0/1450912728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564e31df2df0 0x564e31dfb0e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00cdf0 tx=0x7fcd6c007680 comp rx=0 tx=0).stop 2024-09-12T03:05:38.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.811+0000 7fcd82abb740 1 -- 172.21.15.116:0/1450912728 shutdown_connections 2024-09-12T03:05:38.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.811+0000 7fcd82abb740 1 --2- 172.21.15.116:0/1450912728 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564e31df7a40 0x564e31dfb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.811+0000 7fcd82abb740 1 --2- 172.21.15.116:0/1450912728 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x564e31e0b730 0x564e31e0db70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.811+0000 7fcd82abb740 1 --2- 172.21.15.116:0/1450912728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564e31df2df0 0x564e31dfb0e0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.811+0000 7fcd82abb740 1 --2- 172.21.15.116:0/1450912728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e31df2440 0x564e31dfde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.811+0000 7fcd82abb740 1 -- 172.21.15.116:0/1450912728 >> 172.21.15.116:0/1450912728 conn(0x564e31db1310 msgr2=0x564e31df1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:38.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.811+0000 7fcd5dffb640 0 client.15924 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:05:38.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.812+0000 7fcd82abb740 1 -- 172.21.15.116:0/1450912728 shutdown_connections 2024-09-12T03:05:38.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.812+0000 7fcd82abb740 1 -- 172.21.15.116:0/1450912728 wait complete. 2024-09-12T03:05:38.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.812+0000 7fcd82abb740 20 client.15924 trim_cache size 0 max 16384 2024-09-12T03:05:38.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.825+0000 7fcd82abb740 1 Processor -- start 2024-09-12T03:05:38.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.825+0000 7fcd82abb740 1 -- start start 2024-09-12T03:05:38.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.825+0000 7fcd82abb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e31df2440 0x564e31e07b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:38.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.825+0000 7fcd82abb740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564e31df2df0 0x564e31df8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:38.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.825+0000 7fcd82abb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564e31df7a40 0x564e31df1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:38.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.825+0000 7fcd82abb740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcd781adbd0 con 0x564e31df2df0 2024-09-12T03:05:38.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.825+0000 7fcd82abb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcd781ab4a0 con 0x564e31df7a40 2024-09-12T03:05:38.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.825+0000 7fcd82abb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcd781a9310 con 0x564e31df2440 2024-09-12T03:05:38.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.825+0000 7fcd7dc7d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564e31df2df0 0x564e31df8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:38.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.825+0000 7fcd7dc7d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564e31df2df0 0x564e31df8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58108/0 (socket says 172.21.15.116:58108) 2024-09-12T03:05:38.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.825+0000 7fcd7dc7d640 1 -- 172.21.15.116:0/2207779309 learned_addr learned my addr 172.21.15.116:0/2207779309 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:38.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.825+0000 7fcd7e47e640 1 --2- 172.21.15.116:0/2207779309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e31df2440 0x564e31e07b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:38.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.825+0000 7fcd7ec7f640 1 --2- 172.21.15.116:0/2207779309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564e31df7a40 0x564e31df1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:38.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.825+0000 7fcd7dc7d640 1 -- 172.21.15.116:0/2207779309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e31df2440 msgr2=0x564e31e07b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:38.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.825+0000 7fcd7dc7d640 1 --2- 172.21.15.116:0/2207779309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e31df2440 0x564e31e07b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.825+0000 7fcd7dc7d640 1 -- 172.21.15.116:0/2207779309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564e31df7a40 msgr2=0x564e31df1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:38.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.825+0000 7fcd7dc7d640 1 --2- 172.21.15.116:0/2207779309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564e31df7a40 0x564e31df1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.825+0000 7fcd7dc7d640 1 -- 172.21.15.116:0/2207779309 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e31de7f00 con 0x564e31df2df0 2024-09-12T03:05:38.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.825+0000 7fcd7ec7f640 1 --2- 172.21.15.116:0/2207779309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564e31df7a40 0x564e31df1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:05:38.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.825+0000 7fcd7e47e640 1 --2- 172.21.15.116:0/2207779309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e31df2440 0x564e31e07b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:38.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.825+0000 7fcd7dc7d640 1 --2- 172.21.15.116:0/2207779309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564e31df2df0 0x564e31df8a00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c009380 tx=0x7fcd6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee4bf8d9c288e95 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:38.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.825+0000 7fcd5ffff640 1 -- 172.21.15.116:0/2207779309 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c027040 con 0x564e31df2df0 2024-09-12T03:05:38.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.825+0000 7fcd5ffff640 1 -- 172.21.15.116:0/2207779309 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd6c009ce0 con 0x564e31df2df0 2024-09-12T03:05:38.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.825+0000 7fcd5ffff640 1 -- 172.21.15.116:0/2207779309 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c017aa0 con 0x564e31df2df0 2024-09-12T03:05:38.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.826+0000 7fcd82abb740 1 -- 172.21.15.116:0/2207779309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564e31df2df0 msgr2=0x564e31df8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:38.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.826+0000 7fcd82abb740 1 --2- 172.21.15.116:0/2207779309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564e31df2df0 0x564e31df8a00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c009380 tx=0x7fcd6c004a40 comp rx=0 tx=0).stop 2024-09-12T03:05:38.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.826+0000 7fcd82abb740 1 -- 172.21.15.116:0/2207779309 shutdown_connections 2024-09-12T03:05:38.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.826+0000 7fcd82abb740 1 --2- 172.21.15.116:0/2207779309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564e31df7a40 0x564e31df1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.826+0000 7fcd82abb740 1 --2- 172.21.15.116:0/2207779309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564e31df2df0 0x564e31df8a00 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.826+0000 7fcd82abb740 1 --2- 172.21.15.116:0/2207779309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e31df2440 0x564e31e07b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.826+0000 7fcd82abb740 1 -- 172.21.15.116:0/2207779309 >> 172.21.15.116:0/2207779309 conn(0x564e31db1310 msgr2=0x564e31df82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:38.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.826+0000 7fcd82abb740 1 -- 172.21.15.116:0/2207779309 shutdown_connections 2024-09-12T03:05:38.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.826+0000 7fcd82abb740 1 -- 172.21.15.116:0/2207779309 wait complete. 2024-09-12T03:05:38.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.826+0000 7fcd82abb740 1 -- start start 2024-09-12T03:05:38.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.826+0000 7fcd82abb740 1 Processor -- start 2024-09-12T03:05:38.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.827+0000 7fcd82abb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e31df2440 0x564e31e040b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:38.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.827+0000 7fcd82abb740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564e31df2df0 0x564e31e12a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:38.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.827+0000 7fcd82abb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564e31df7a40 0x564e31e12f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:38.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.827+0000 7fcd82abb740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcd781a9960 con 0x564e31df2df0 2024-09-12T03:05:38.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.827+0000 7fcd82abb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcd781aadd0 con 0x564e31df7a40 2024-09-12T03:05:38.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.827+0000 7fcd82abb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcd781aa030 con 0x564e31df2440 2024-09-12T03:05:38.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.827+0000 7fcd82abb740 10 client.0 _ll_register_callbacks cb 0x7fcd6c009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:38.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.827+0000 7fcd7dc7d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564e31df2df0 0x564e31e12a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:38.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.827+0000 7fcd7dc7d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564e31df2df0 0x564e31e12a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58114/0 (socket says 172.21.15.116:58114) 2024-09-12T03:05:38.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.827+0000 7fcd7dc7d640 1 -- 172.21.15.116:0/3312079609 learned_addr learned my addr 172.21.15.116:0/3312079609 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:38.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.827+0000 7fcd7ec7f640 1 --2- 172.21.15.116:0/3312079609 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564e31df7a40 0x564e31e12f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:38.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.827+0000 7fcd7e47e640 1 --2- 172.21.15.116:0/3312079609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e31df2440 0x564e31e040b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:38.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.827+0000 7fcd7dc7d640 1 -- 172.21.15.116:0/3312079609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e31df2440 msgr2=0x564e31e040b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:38.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.827+0000 7fcd7dc7d640 1 --2- 172.21.15.116:0/3312079609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e31df2440 0x564e31e040b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.827+0000 7fcd7dc7d640 1 -- 172.21.15.116:0/3312079609 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564e31df7a40 msgr2=0x564e31e12f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:38.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.827+0000 7fcd7dc7d640 1 --2- 172.21.15.116:0/3312079609 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564e31df7a40 0x564e31e12f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.827+0000 7fcd7dc7d640 1 -- 172.21.15.116:0/3312079609 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e31e13490 con 0x564e31df2df0 2024-09-12T03:05:38.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.827+0000 7fcd7dc7d640 1 --2- 172.21.15.116:0/3312079609 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564e31df2df0 0x564e31e12a10 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c009380 tx=0x7fcd6c007100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3399adcb0e5bd5e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:38.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.827+0000 7fcd5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:05:38.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.827+0000 7fcd5dffb640 1 -- 172.21.15.116:0/3312079609 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c038060 con 0x564e31df2df0 2024-09-12T03:05:38.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.827+0000 7fcd5dffb640 1 -- 172.21.15.116:0/3312079609 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fcd6c00c070 con 0x564e31df2df0 2024-09-12T03:05:38.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.827+0000 7fcd82abb740 1 -- 172.21.15.116:0/3312079609 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd6c009ce0 con 0x564e31df2df0 2024-09-12T03:05:38.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.828+0000 7fcd5dffb640 1 -- 172.21.15.116:0/3312079609 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcd6c027040 con 0x564e31df2df0 2024-09-12T03:05:38.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.828+0000 7fcd82abb740 10 client.15930 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:38.846 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:05:38.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.828+0000 7fcd82abb740 1 -- 172.21.15.116:0/3312079609 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564e31e76050 con 0x564e31df2df0 2024-09-12T03:05:38.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.828+0000 7fcd5dffb640 1 -- 172.21.15.116:0/3312079609 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd6c009ce0 con 0x564e31df2df0 2024-09-12T03:05:38.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.828+0000 7fcd5dffb640 1 -- 172.21.15.116:0/3312079609 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcd6c02a070 con 0x564e31df2df0 2024-09-12T03:05:38.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.828+0000 7fcd82abb740 10 client.15930 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:38.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.828+0000 7fcd82abb740 10 client.15930 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:05:38.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.828+0000 7fcd82abb740 20 client.15930 populate_metadata read hostname 'smithi116' 2024-09-12T03:05:38.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.828+0000 7fcd82abb740 1 --2- 172.21.15.116:0/3312079609 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x564e31e76730 0x564e31e78b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:38.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.828+0000 7fcd82abb740 4 client.15930 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:05:38.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.828+0000 7fcd82abb740 1 -- 172.21.15.116:0/3312079609 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564e31d55d70 con 0x564e31e76730 2024-09-12T03:05:38.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.828+0000 7fcd7e47e640 1 --2- 172.21.15.116:0/3312079609 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x564e31e76730 0x564e31e78b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:38.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.829+0000 7fcd5dffb640 10 client.15930 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:05:38.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.829+0000 7fcd7e47e640 1 --2- 172.21.15.116:0/3312079609 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x564e31e76730 0x564e31e78b50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8bb9d87a7d819dc7 server_cookie=269d679eb586ba39 in_seq=0 out_seq=0 2024-09-12T03:05:38.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.829+0000 7fcd5dffb640 1 -- 172.21.15.116:0/3312079609 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564e31d55d70 con 0x564e31e76730 2024-09-12T03:05:38.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.829+0000 7fcd5dffb640 10 client.15930 handle_command_reply: tid=0 2024-09-12T03:05:38.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.829+0000 7fcd82abb740 1 client.15930 shutdown 2024-09-12T03:05:38.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.829+0000 7fcd82abb740 1 -- 172.21.15.116:0/3312079609 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564e31df2df0 msgr2=0x564e31e12a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:38.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.829+0000 7fcd82abb740 1 --2- 172.21.15.116:0/3312079609 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564e31df2df0 0x564e31e12a10 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c009380 tx=0x7fcd6c007100 comp rx=0 tx=0).stop 2024-09-12T03:05:38.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.829+0000 7fcd82abb740 1 -- 172.21.15.116:0/3312079609 shutdown_connections 2024-09-12T03:05:38.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.829+0000 7fcd82abb740 1 --2- 172.21.15.116:0/3312079609 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564e31df7a40 0x564e31e12f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.829+0000 7fcd82abb740 1 --2- 172.21.15.116:0/3312079609 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x564e31e76730 0x564e31e78b50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.829+0000 7fcd82abb740 1 --2- 172.21.15.116:0/3312079609 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564e31df2df0 0x564e31e12a10 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.829+0000 7fcd82abb740 1 --2- 172.21.15.116:0/3312079609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e31df2440 0x564e31e040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.829+0000 7fcd82abb740 1 -- 172.21.15.116:0/3312079609 >> 172.21.15.116:0/3312079609 conn(0x564e31db1310 msgr2=0x564e31dfd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:38.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.829+0000 7fcd5dffb640 0 client.15930 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:05:38.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.829+0000 7fcd82abb740 1 -- 172.21.15.116:0/3312079609 shutdown_connections 2024-09-12T03:05:38.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.829+0000 7fcd82abb740 1 -- 172.21.15.116:0/3312079609 wait complete. 2024-09-12T03:05:38.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.829+0000 7fcd82abb740 20 client.15930 trim_cache size 0 max 16384 2024-09-12T03:05:38.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.830+0000 7fcd806e2640 1 -- 172.21.15.116:0/3834293601 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcd600a2b80 msgr2=0x7fcd600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:38.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.830+0000 7fcd806e2640 1 --2- 172.21.15.116:0/3834293601 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcd600a2b80 0x7fcd600a5040 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcd68006fd0 tx=0x7fcd6800a520 comp rx=0 tx=0).stop 2024-09-12T03:05:38.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.830+0000 7fcd806e2640 1 -- 172.21.15.116:0/3834293601 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd78101df0 msgr2=0x7fcd78198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:38.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.830+0000 7fcd806e2640 1 --2- 172.21.15.116:0/3834293601 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd78101df0 0x7fcd78198640 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fcd70005ab0 tx=0x7fcd700042f0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.830+0000 7fcd806e2640 1 -- 172.21.15.116:0/3834293601 shutdown_connections 2024-09-12T03:05:38.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.830+0000 7fcd806e2640 1 --2- 172.21.15.116:0/3834293601 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcd600a2b80 0x7fcd600a5040 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.830+0000 7fcd806e2640 1 --2- 172.21.15.116:0/3834293601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd78106f20 0x7fcd7819cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.830+0000 7fcd806e2640 1 --2- 172.21.15.116:0/3834293601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd781027a0 0x7fcd78198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.830+0000 7fcd806e2640 1 --2- 172.21.15.116:0/3834293601 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd78101df0 0x7fcd78198640 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:38.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.830+0000 7fcd806e2640 1 -- 172.21.15.116:0/3834293601 >> 172.21.15.116:0/3834293601 conn(0x7fcd780fd8b0 msgr2=0x7fcd780fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:38.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.830+0000 7fcd806e2640 1 -- 172.21.15.116:0/3834293601 shutdown_connections 2024-09-12T03:05:38.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:38.830+0000 7fcd806e2640 1 -- 172.21.15.116:0/3834293601 wait complete. 2024-09-12T03:05:39.598 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:05:39.667 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:40.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.198+0000 7f6a5b7ae640 1 -- 172.21.15.116:0/147436670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a54102570 msgr2=0x7f6a541029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:40.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.198+0000 7f6a5b7ae640 1 --2- 172.21.15.116:0/147436670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a54102570 0x7f6a541029b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6a44009900 tx=0x7f6a4402a480 comp rx=0 tx=0).stop 2024-09-12T03:05:40.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.198+0000 7f6a5b7ae640 1 -- 172.21.15.116:0/147436670 shutdown_connections 2024-09-12T03:05:40.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.198+0000 7f6a5b7ae640 1 --2- 172.21.15.116:0/147436670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a54102ef0 0x7f6a5410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.198+0000 7f6a5b7ae640 1 --2- 172.21.15.116:0/147436670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a54102570 0x7f6a541029b0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.198+0000 7f6a5b7ae640 1 --2- 172.21.15.116:0/147436670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a54100e20 0x7f6a54101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.198+0000 7f6a5b7ae640 1 -- 172.21.15.116:0/147436670 >> 172.21.15.116:0/147436670 conn(0x7f6a540fc7d0 msgr2=0x7f6a540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:40.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.198+0000 7f6a5b7ae640 1 -- 172.21.15.116:0/147436670 shutdown_connections 2024-09-12T03:05:40.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.198+0000 7f6a5b7ae640 1 -- 172.21.15.116:0/147436670 wait complete. 2024-09-12T03:05:40.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.199+0000 7f6a5b7ae640 1 Processor -- start 2024-09-12T03:05:40.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.199+0000 7f6a5b7ae640 1 -- start start 2024-09-12T03:05:40.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.199+0000 7f6a5b7ae640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a54100e20 0x7f6a54198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:40.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.199+0000 7f6a5b7ae640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a54102570 0x7f6a54198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:40.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.199+0000 7f6a5b7ae640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a54102ef0 0x7f6a5419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:40.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.199+0000 7f6a5b7ae640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6a54110fc0 con 0x7f6a54100e20 2024-09-12T03:05:40.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.199+0000 7f6a5b7ae640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6a54110e40 con 0x7f6a54102570 2024-09-12T03:05:40.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.199+0000 7f6a5b7ae640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6a54111140 con 0x7f6a54102ef0 2024-09-12T03:05:40.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.199+0000 7f6a5954a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a54100e20 0x7f6a54198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:40.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.199+0000 7f6a5954a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a54100e20 0x7f6a54198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58132/0 (socket says 172.21.15.116:58132) 2024-09-12T03:05:40.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.199+0000 7f6a5954a640 1 -- 172.21.15.116:0/1681710495 learned_addr learned my addr 172.21.15.116:0/1681710495 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:40.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.199+0000 7f6a59d4b640 1 --2- 172.21.15.116:0/1681710495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a54102ef0 0x7f6a5419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:40.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.199+0000 7f6a58d49640 1 --2- 172.21.15.116:0/1681710495 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a54102570 0x7f6a54198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:40.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.199+0000 7f6a5954a640 1 -- 172.21.15.116:0/1681710495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a54102ef0 msgr2=0x7f6a5419ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:40.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.200+0000 7f6a5954a640 1 --2- 172.21.15.116:0/1681710495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a54102ef0 0x7f6a5419ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.200+0000 7f6a5954a640 1 -- 172.21.15.116:0/1681710495 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a54102570 msgr2=0x7f6a54198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:40.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.200+0000 7f6a5954a640 1 --2- 172.21.15.116:0/1681710495 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a54102570 0x7f6a54198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.200+0000 7f6a5954a640 1 -- 172.21.15.116:0/1681710495 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5419d530 con 0x7f6a54100e20 2024-09-12T03:05:40.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.200+0000 7f6a58d49640 1 --2- 172.21.15.116:0/1681710495 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a54102570 0x7f6a54198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:05:40.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.200+0000 7f6a5954a640 1 --2- 172.21.15.116:0/1681710495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a54100e20 0x7f6a54198580 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6a5000b4a0 tx=0x7f6a5000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:40.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.200+0000 7f6a4a7fc640 1 -- 172.21.15.116:0/1681710495 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a50014070 con 0x7f6a54100e20 2024-09-12T03:05:40.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.200+0000 7f6a4a7fc640 1 -- 172.21.15.116:0/1681710495 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a50010af0 con 0x7f6a54100e20 2024-09-12T03:05:40.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.200+0000 7f6a4a7fc640 1 -- 172.21.15.116:0/1681710495 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a50017d10 con 0x7f6a54100e20 2024-09-12T03:05:40.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.200+0000 7f6a5b7ae640 1 -- 172.21.15.116:0/1681710495 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a5419d820 con 0x7f6a54100e20 2024-09-12T03:05:40.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.200+0000 7f6a5b7ae640 1 -- 172.21.15.116:0/1681710495 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a540fcbb0 con 0x7f6a54100e20 2024-09-12T03:05:40.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.203+0000 7f6a4a7fc640 1 -- 172.21.15.116:0/1681710495 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6a50017070 con 0x7f6a54100e20 2024-09-12T03:05:40.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.203+0000 7f6a5db87740 1 Processor -- start 2024-09-12T03:05:40.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.203+0000 7f6a4a7fc640 1 --2- 172.21.15.116:0/1681710495 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6a400a2c00 0x7f6a400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:40.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.203+0000 7f6a5db87740 1 -- start start 2024-09-12T03:05:40.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.204+0000 7f6a59d4b640 1 --2- 172.21.15.116:0/1681710495 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6a400a2c00 0x7f6a400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:40.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.204+0000 7f6a4a7fc640 1 -- 172.21.15.116:0/1681710495 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f6a500cdcd0 con 0x7f6a54100e20 2024-09-12T03:05:40.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.204+0000 7f6a5db87740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576efee0760 0x5576efee0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:40.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.204+0000 7f6a5db87740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5576efee1080 0x5576efeeb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:40.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.204+0000 7f6a5db87740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576efeeb940 0x5576efeedd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:40.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.204+0000 7f6a5db87740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5576efe462a0 con 0x5576efee0760 2024-09-12T03:05:40.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.204+0000 7f6a5db87740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5576efdedb10 con 0x5576efeeb940 2024-09-12T03:05:40.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.204+0000 7f6a5db87740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5576efe42260 con 0x5576efee1080 2024-09-12T03:05:40.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.204+0000 7f6a5954a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576efee0760 0x5576efee0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:40.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.204+0000 7f6a5954a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576efee0760 0x5576efee0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58136/0 (socket says 172.21.15.116:58136) 2024-09-12T03:05:40.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.204+0000 7f6a5954a640 1 -- 172.21.15.116:0/2236369058 learned_addr learned my addr 172.21.15.116:0/2236369058 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:40.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.204+0000 7f6a58d49640 1 --2- 172.21.15.116:0/2236369058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5576efee1080 0x5576efeeb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:40.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.204+0000 7f6a59d4b640 1 --2- 172.21.15.116:0/1681710495 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6a400a2c00 0x7f6a400a50c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c005fd0 tx=0x7f6a4c010740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:40.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.204+0000 7f6a5954a640 1 -- 172.21.15.116:0/2236369058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5576efee1080 msgr2=0x5576efeeb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:40.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.204+0000 7f6a59d4b640 1 --2- 172.21.15.116:0/2236369058 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576efeeb940 0x5576efeedd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:40.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.204+0000 7f6a5954a640 1 --2- 172.21.15.116:0/2236369058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5576efee1080 0x5576efeeb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.204+0000 7f6a5954a640 1 -- 172.21.15.116:0/2236369058 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576efeeb940 msgr2=0x5576efeedd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:40.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.204+0000 7f6a5954a640 1 --2- 172.21.15.116:0/2236369058 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576efeeb940 0x5576efeedd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.204+0000 7f6a5954a640 1 -- 172.21.15.116:0/2236369058 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576efbb7b90 con 0x5576efee0760 2024-09-12T03:05:40.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.204+0000 7f6a58d49640 1 --2- 172.21.15.116:0/2236369058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5576efee1080 0x5576efeeb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:05:40.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.204+0000 7f6a59d4b640 1 --2- 172.21.15.116:0/2236369058 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576efeeb940 0x5576efeedd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:05:40.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.204+0000 7f6a5954a640 1 --2- 172.21.15.116:0/2236369058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576efee0760 0x5576efee0b40 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6a5009a260 tx=0x7f6a50017730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60565041040620da server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:40.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.204+0000 7f6a3f7fe640 1 -- 172.21.15.116:0/2236369058 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5001e050 con 0x5576efee0760 2024-09-12T03:05:40.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.204+0000 7f6a3f7fe640 1 -- 172.21.15.116:0/2236369058 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a5009e910 con 0x5576efee0760 2024-09-12T03:05:40.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.204+0000 7f6a3f7fe640 1 -- 172.21.15.116:0/2236369058 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a500999b0 con 0x5576efee0760 2024-09-12T03:05:40.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.205+0000 7f6a5db87740 1 -- 172.21.15.116:0/2236369058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576efee0760 msgr2=0x5576efee0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:40.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.205+0000 7f6a5db87740 1 --2- 172.21.15.116:0/2236369058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576efee0760 0x5576efee0b40 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6a5009a260 tx=0x7f6a50017730 comp rx=0 tx=0).stop 2024-09-12T03:05:40.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.205+0000 7f6a5db87740 1 -- 172.21.15.116:0/2236369058 shutdown_connections 2024-09-12T03:05:40.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.205+0000 7f6a5db87740 1 --2- 172.21.15.116:0/2236369058 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576efeeb940 0x5576efeedd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.205+0000 7f6a5db87740 1 --2- 172.21.15.116:0/2236369058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5576efee1080 0x5576efeeb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.205+0000 7f6a5db87740 1 --2- 172.21.15.116:0/2236369058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576efee0760 0x5576efee0b40 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.205+0000 7f6a5db87740 1 -- 172.21.15.116:0/2236369058 >> 172.21.15.116:0/2236369058 conn(0x5576efedccf0 msgr2=0x5576efedd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:40.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.205+0000 7f6a5db87740 1 -- 172.21.15.116:0/2236369058 shutdown_connections 2024-09-12T03:05:40.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.205+0000 7f6a5db87740 1 -- 172.21.15.116:0/2236369058 wait complete. 2024-09-12T03:05:40.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.205+0000 7f6a5db87740 1 -- start start 2024-09-12T03:05:40.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.206+0000 7f6a5db87740 1 Processor -- start 2024-09-12T03:05:40.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.206+0000 7f6a5db87740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576efee0760 0x5576efee8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:40.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.206+0000 7f6a5db87740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5576efee1080 0x5576efee8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:40.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.206+0000 7f6a5db87740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576efeeb940 0x5576efee92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:40.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.206+0000 7f6a5db87740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5576efeee510 con 0x5576efee0760 2024-09-12T03:05:40.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.206+0000 7f6a5db87740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5576efdce6e0 con 0x5576efeeb940 2024-09-12T03:05:40.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.206+0000 7f6a5db87740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5576efeee690 con 0x5576efee1080 2024-09-12T03:05:40.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.206+0000 7f6a5db87740 10 client.0 _ll_register_callbacks cb 0x5576efdf2a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:40.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.206+0000 7f6a5954a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576efee0760 0x5576efee8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:40.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.206+0000 7f6a5954a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576efee0760 0x5576efee8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58148/0 (socket says 172.21.15.116:58148) 2024-09-12T03:05:40.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.206+0000 7f6a5954a640 1 -- 172.21.15.116:0/2851548397 learned_addr learned my addr 172.21.15.116:0/2851548397 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:40.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.206+0000 7f6a59d4b640 1 --2- 172.21.15.116:0/2851548397 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576efeeb940 0x5576efee92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:40.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.206+0000 7f6a5954a640 1 -- 172.21.15.116:0/2851548397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5576efee1080 msgr2=0x5576efee8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:40.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.206+0000 7f6a5954a640 1 --2- 172.21.15.116:0/2851548397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5576efee1080 0x5576efee8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.206+0000 7f6a5954a640 1 -- 172.21.15.116:0/2851548397 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576efeeb940 msgr2=0x5576efee92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:40.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.206+0000 7f6a5954a640 1 --2- 172.21.15.116:0/2851548397 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576efeeb940 0x5576efee92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.206+0000 7f6a5954a640 1 -- 172.21.15.116:0/2851548397 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576efef7a30 con 0x5576efee0760 2024-09-12T03:05:40.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.206+0000 7f6a59d4b640 1 --2- 172.21.15.116:0/2851548397 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576efeeb940 0x5576efee92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:05:40.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.207+0000 7f6a5954a640 1 --2- 172.21.15.116:0/2851548397 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576efee0760 0x5576efee8840 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6a500a2ca0 tx=0x7f6a50098670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7539fb3690e13b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:40.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.207+0000 7f6a3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:05:40.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.207+0000 7f6a3cff9640 1 -- 172.21.15.116:0/2851548397 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a500b2070 con 0x5576efee0760 2024-09-12T03:05:40.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.207+0000 7f6a3cff9640 1 -- 172.21.15.116:0/2851548397 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f6a500b3460 con 0x5576efee0760 2024-09-12T03:05:40.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.207+0000 7f6a5db87740 1 -- 172.21.15.116:0/2851548397 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a5009e910 con 0x5576efee0760 2024-09-12T03:05:40.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.207+0000 7f6a3cff9640 1 -- 172.21.15.116:0/2851548397 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6a500999b0 con 0x5576efee0760 2024-09-12T03:05:40.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.207+0000 7f6a5db87740 10 client.15942 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:40.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.207+0000 7f6a5db87740 1 -- 172.21.15.116:0/2851548397 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5576efeee510 con 0x5576efee0760 2024-09-12T03:05:40.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.207+0000 7f6a3cff9640 1 -- 172.21.15.116:0/2851548397 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a5009e910 con 0x5576efee0760 2024-09-12T03:05:40.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.207+0000 7f6a3cff9640 1 -- 172.21.15.116:0/2851548397 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6a50083c60 con 0x5576efee0760 2024-09-12T03:05:40.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.207+0000 7f6a5db87740 10 client.15942 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:40.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.207+0000 7f6a5db87740 10 client.15942 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:05:40.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.207+0000 7f6a5db87740 20 client.15942 populate_metadata read hostname 'smithi116' 2024-09-12T03:05:40.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.207+0000 7f6a5db87740 1 --2- 172.21.15.116:0/2851548397 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5576efefb490 0x5576efefd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:40.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.207+0000 7f6a5db87740 4 client.15942 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:05:40.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.207+0000 7f6a5db87740 1 -- 172.21.15.116:0/2851548397 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576efe539d0 con 0x5576efefb490 2024-09-12T03:05:40.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.208+0000 7f6a58d49640 1 --2- 172.21.15.116:0/2851548397 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5576efefb490 0x5576efefd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:40.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.208+0000 7f6a58d49640 1 --2- 172.21.15.116:0/2851548397 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5576efefb490 0x5576efefd880 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4211e265c006ac35 server_cookie=7d3573df8bc0e76e in_seq=0 out_seq=0 2024-09-12T03:05:40.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.208+0000 7f6a3cff9640 10 client.15942 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:40.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.209+0000 7f6a3cff9640 1 -- 172.21.15.116:0/2851548397 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5576efe539d0 con 0x5576efefb490 2024-09-12T03:05:40.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.209+0000 7f6a3cff9640 10 client.15942 handle_command_reply: tid=0 2024-09-12T03:05:40.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.209+0000 7f6a5db87740 1 client.15942 shutdown 2024-09-12T03:05:40.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.209+0000 7f6a5db87740 1 -- 172.21.15.116:0/2851548397 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576efee0760 msgr2=0x5576efee8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:40.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.209+0000 7f6a5db87740 1 --2- 172.21.15.116:0/2851548397 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576efee0760 0x5576efee8840 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6a500a2ca0 tx=0x7f6a50098670 comp rx=0 tx=0).stop 2024-09-12T03:05:40.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.209+0000 7f6a5db87740 1 -- 172.21.15.116:0/2851548397 shutdown_connections 2024-09-12T03:05:40.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.209+0000 7f6a5db87740 1 --2- 172.21.15.116:0/2851548397 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5576efefb490 0x5576efefd880 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.209+0000 7f6a5db87740 1 --2- 172.21.15.116:0/2851548397 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576efeeb940 0x5576efee92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.209+0000 7f6a3cff9640 0 client.15942 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:40.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.209+0000 7f6a5db87740 1 --2- 172.21.15.116:0/2851548397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5576efee1080 0x5576efee8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.209+0000 7f6a5db87740 1 --2- 172.21.15.116:0/2851548397 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576efee0760 0x5576efee8840 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.209+0000 7f6a5db87740 1 -- 172.21.15.116:0/2851548397 >> 172.21.15.116:0/2851548397 conn(0x5576efedccf0 msgr2=0x5576efedbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:40.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.209+0000 7f6a5db87740 1 -- 172.21.15.116:0/2851548397 shutdown_connections 2024-09-12T03:05:40.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.209+0000 7f6a5db87740 1 -- 172.21.15.116:0/2851548397 wait complete. 2024-09-12T03:05:40.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.209+0000 7f6a5db87740 20 client.15942 trim_cache size 0 max 16384 2024-09-12T03:05:40.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.227+0000 7f6a5db87740 1 Processor -- start 2024-09-12T03:05:40.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.227+0000 7f6a5db87740 1 -- start start 2024-09-12T03:05:40.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.227+0000 7f6a5db87740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576efee0760 0x5576efee7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:40.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.227+0000 7f6a5db87740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5576efee1080 0x5576efeea710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:40.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.227+0000 7f6a5db87740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576efeeb940 0x5576efee31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:40.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.227+0000 7f6a5db87740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6a541b20b0 con 0x5576efee0760 2024-09-12T03:05:40.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.227+0000 7f6a5db87740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6a541af980 con 0x5576efeeb940 2024-09-12T03:05:40.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.227+0000 7f6a5db87740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6a541ad7d0 con 0x5576efee1080 2024-09-12T03:05:40.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.227+0000 7f6a5954a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576efee0760 0x5576efee7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:40.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.227+0000 7f6a5954a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576efee0760 0x5576efee7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58152/0 (socket says 172.21.15.116:58152) 2024-09-12T03:05:40.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.227+0000 7f6a5954a640 1 -- 172.21.15.116:0/3728640626 learned_addr learned my addr 172.21.15.116:0/3728640626 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:40.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.227+0000 7f6a59d4b640 1 --2- 172.21.15.116:0/3728640626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576efeeb940 0x5576efee31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:40.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.228+0000 7f6a58d49640 1 --2- 172.21.15.116:0/3728640626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5576efee1080 0x5576efeea710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:40.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.228+0000 7f6a59d4b640 1 -- 172.21.15.116:0/3728640626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5576efee1080 msgr2=0x5576efeea710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:40.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.228+0000 7f6a59d4b640 1 --2- 172.21.15.116:0/3728640626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5576efee1080 0x5576efeea710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.228+0000 7f6a59d4b640 1 -- 172.21.15.116:0/3728640626 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576efee0760 msgr2=0x5576efee7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:40.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.228+0000 7f6a59d4b640 1 --2- 172.21.15.116:0/3728640626 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576efee0760 0x5576efee7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.228+0000 7f6a59d4b640 1 -- 172.21.15.116:0/3728640626 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576efedd980 con 0x5576efeeb940 2024-09-12T03:05:40.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.228+0000 7f6a59d4b640 1 --2- 172.21.15.116:0/3728640626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576efeeb940 0x5576efee31c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x5576efede070 tx=0x7f6a4c015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=620f86fc4a91a5ac server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:40.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.228+0000 7f6a3effd640 1 -- 172.21.15.116:0/3728640626 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c003970 con 0x5576efeeb940 2024-09-12T03:05:40.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.228+0000 7f6a3effd640 1 -- 172.21.15.116:0/3728640626 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a4c0118b0 con 0x5576efeeb940 2024-09-12T03:05:40.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.228+0000 7f6a3effd640 1 -- 172.21.15.116:0/3728640626 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c004610 con 0x5576efeeb940 2024-09-12T03:05:40.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.229+0000 7f6a5db87740 1 -- 172.21.15.116:0/3728640626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576efeeb940 msgr2=0x5576efee31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:40.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.229+0000 7f6a5db87740 1 --2- 172.21.15.116:0/3728640626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576efeeb940 0x5576efee31c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x5576efede070 tx=0x7f6a4c015000 comp rx=0 tx=0).stop 2024-09-12T03:05:40.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.229+0000 7f6a5db87740 1 -- 172.21.15.116:0/3728640626 shutdown_connections 2024-09-12T03:05:40.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.229+0000 7f6a5db87740 1 --2- 172.21.15.116:0/3728640626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576efeeb940 0x5576efee31c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.229+0000 7f6a5db87740 1 --2- 172.21.15.116:0/3728640626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5576efee1080 0x5576efeea710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.229+0000 7f6a5db87740 1 --2- 172.21.15.116:0/3728640626 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576efee0760 0x5576efee7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.229+0000 7f6a5db87740 1 -- 172.21.15.116:0/3728640626 >> 172.21.15.116:0/3728640626 conn(0x5576efedccf0 msgr2=0x5576efedb650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:40.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.229+0000 7f6a5db87740 1 -- 172.21.15.116:0/3728640626 shutdown_connections 2024-09-12T03:05:40.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.229+0000 7f6a5db87740 1 -- 172.21.15.116:0/3728640626 wait complete. 2024-09-12T03:05:40.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.230+0000 7f6a5db87740 1 -- start start 2024-09-12T03:05:40.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.230+0000 7f6a5db87740 1 Processor -- start 2024-09-12T03:05:40.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.231+0000 7f6a5db87740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576efee0760 0x5576efeefe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:40.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.231+0000 7f6a5db87740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576efee1080 0x5576efef0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:40.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.231+0000 7f6a5db87740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5576efeeb940 0x5576eff64f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:40.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.231+0000 7f6a5db87740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6a541ade40 con 0x5576efee0760 2024-09-12T03:05:40.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.231+0000 7f6a5db87740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6a541af2b0 con 0x5576efee1080 2024-09-12T03:05:40.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.231+0000 7f6a5db87740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6a541ae510 con 0x5576efeeb940 2024-09-12T03:05:40.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.231+0000 7f6a5db87740 10 client.0 _ll_register_callbacks cb 0x7f6a5009a130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:40.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.231+0000 7f6a5954a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576efee0760 0x5576efeefe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:40.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.231+0000 7f6a5954a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576efee0760 0x5576efeefe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58164/0 (socket says 172.21.15.116:58164) 2024-09-12T03:05:40.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.231+0000 7f6a5954a640 1 -- 172.21.15.116:0/1006171749 learned_addr learned my addr 172.21.15.116:0/1006171749 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:40.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.231+0000 7f6a58d49640 1 --2- 172.21.15.116:0/1006171749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576efee1080 0x5576efef0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:40.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.231+0000 7f6a59d4b640 1 --2- 172.21.15.116:0/1006171749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5576efeeb940 0x5576eff64f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:40.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.231+0000 7f6a58d49640 1 -- 172.21.15.116:0/1006171749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5576efeeb940 msgr2=0x5576eff64f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:40.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.231+0000 7f6a58d49640 1 --2- 172.21.15.116:0/1006171749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5576efeeb940 0x5576eff64f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.231+0000 7f6a58d49640 1 -- 172.21.15.116:0/1006171749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576efee0760 msgr2=0x5576efeefe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:40.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.231+0000 7f6a58d49640 1 --2- 172.21.15.116:0/1006171749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576efee0760 0x5576efeefe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.231+0000 7f6a58d49640 1 -- 172.21.15.116:0/1006171749 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576eff65580 con 0x5576efee1080 2024-09-12T03:05:40.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.231+0000 7f6a5954a640 1 --2- 172.21.15.116:0/1006171749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576efee0760 0x5576efeefe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:40.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.232+0000 7f6a58d49640 1 --2- 172.21.15.116:0/1006171749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576efee1080 0x5576efef0370 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6a540fff50 tx=0x7f6a44004380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed5f4c05e176d15b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:40.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.232+0000 7f6a3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-09-12T03:05:40.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.232+0000 7f6a3cff9640 1 -- 172.21.15.116:0/1006171749 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a44035040 con 0x5576efee1080 2024-09-12T03:05:40.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.232+0000 7f6a3cff9640 1 -- 172.21.15.116:0/1006171749 <== mon.2 v2:172.21.15.177:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f6a44005460 con 0x5576efee1080 2024-09-12T03:05:40.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.232+0000 7f6a5db87740 1 -- 172.21.15.116:0/1006171749 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a4c0118b0 con 0x5576efee1080 2024-09-12T03:05:40.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.232+0000 7f6a3cff9640 1 -- 172.21.15.116:0/1006171749 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6a440046c0 con 0x5576efee1080 2024-09-12T03:05:40.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.232+0000 7f6a5db87740 10 client.25247 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:40.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.232+0000 7f6a5db87740 1 -- 172.21.15.116:0/1006171749 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5576eff66880 con 0x5576efee1080 2024-09-12T03:05:40.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.232+0000 7f6a3cff9640 1 -- 172.21.15.116:0/1006171749 <== mon.2 v2:172.21.15.177:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a4c0118b0 con 0x5576efee1080 2024-09-12T03:05:40.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.232+0000 7f6a3cff9640 1 -- 172.21.15.116:0/1006171749 <== mon.2 v2:172.21.15.177:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6a44031890 con 0x5576efee1080 2024-09-12T03:05:40.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.233+0000 7f6a5db87740 10 client.25247 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:40.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.233+0000 7f6a5db87740 10 client.25247 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:05:40.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.233+0000 7f6a5db87740 20 client.25247 populate_metadata read hostname 'smithi116' 2024-09-12T03:05:40.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.233+0000 7f6a5db87740 1 --2- 172.21.15.116:0/1006171749 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5576efefb490 0x5576eff690b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:40.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.233+0000 7f6a5db87740 4 client.25247 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}] 2024-09-12T03:05:40.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.233+0000 7f6a5db87740 1 -- 172.21.15.116:0/1006171749 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}) -- 0x5576efe4dd20 con 0x5576efefb490 2024-09-12T03:05:40.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.233+0000 7f6a5954a640 1 --2- 172.21.15.116:0/1006171749 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5576efefb490 0x5576eff690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:40.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.233+0000 7f6a5954a640 1 --2- 172.21.15.116:0/1006171749 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5576efefb490 0x5576eff690b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59a9020366e3c214 server_cookie=71b0ab6544c838b4 in_seq=0 out_seq=0 2024-09-12T03:05:40.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.233+0000 7f6a3cff9640 10 client.25247 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:40.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.247+0000 7f6a3cff9640 1 -- 172.21.15.116:0/1006171749 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5576efe4dd20 con 0x5576efefb490 2024-09-12T03:05:40.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.247+0000 7f6a3cff9640 10 client.25247 handle_command_reply: tid=0 2024-09-12T03:05:40.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.247+0000 7f6a5db87740 1 client.25247 shutdown 2024-09-12T03:05:40.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.247+0000 7f6a5db87740 1 -- 172.21.15.116:0/1006171749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576efee1080 msgr2=0x5576efef0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:40.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.247+0000 7f6a5db87740 1 --2- 172.21.15.116:0/1006171749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576efee1080 0x5576efef0370 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6a540fff50 tx=0x7f6a44004380 comp rx=0 tx=0).stop 2024-09-12T03:05:40.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.247+0000 7f6a5db87740 1 -- 172.21.15.116:0/1006171749 shutdown_connections 2024-09-12T03:05:40.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.247+0000 7f6a5db87740 1 --2- 172.21.15.116:0/1006171749 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5576efefb490 0x5576eff690b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.247+0000 7f6a5db87740 1 --2- 172.21.15.116:0/1006171749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5576efeeb940 0x5576eff64f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.247+0000 7f6a5db87740 1 --2- 172.21.15.116:0/1006171749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5576efee1080 0x5576efef0370 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.247+0000 7f6a5db87740 1 --2- 172.21.15.116:0/1006171749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5576efee0760 0x5576efeefe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.247+0000 7f6a3cff9640 0 client.25247 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:40.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.247+0000 7f6a5db87740 1 -- 172.21.15.116:0/1006171749 >> 172.21.15.116:0/1006171749 conn(0x5576efedccf0 msgr2=0x5576efee8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:40.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.248+0000 7f6a5db87740 1 -- 172.21.15.116:0/1006171749 shutdown_connections 2024-09-12T03:05:40.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.248+0000 7f6a5db87740 1 -- 172.21.15.116:0/1006171749 wait complete. 2024-09-12T03:05:40.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.248+0000 7f6a5db87740 20 client.25247 trim_cache size 0 max 16384 2024-09-12T03:05:40.250 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:05:40.250 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 25, 2024-09-12T03:05:40.250 INFO:teuthology.orchestra.run.smithi116.stdout: "leader": 24494, 2024-09-12T03:05:40.250 INFO:teuthology.orchestra.run.smithi116.stdout: "set_version": 10, 2024-09-12T03:05:40.250 INFO:teuthology.orchestra.run.smithi116.stdout: "sets": { 2024-09-12T03:05:40.250 INFO:teuthology.orchestra.run.smithi116.stdout: "ff1640ad": { 2024-09-12T03:05:40.250 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 10, 2024-09-12T03:05:40.250 INFO:teuthology.orchestra.run.smithi116.stdout: "age_ref": 0.0, 2024-09-12T03:05:40.250 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:05:40.251 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "QUIESCED", 2024-09-12T03:05:40.251 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:05:40.251 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:05:40.251 INFO:teuthology.orchestra.run.smithi116.stdout: "timeout": 90.0, 2024-09-12T03:05:40.251 INFO:teuthology.orchestra.run.smithi116.stdout: "expiration": 144.7, 2024-09-12T03:05:40.251 INFO:teuthology.orchestra.run.smithi116.stdout: "members": { 2024-09-12T03:05:40.251 INFO:teuthology.orchestra.run.smithi116.stdout: "file:/": { 2024-09-12T03:05:40.251 INFO:teuthology.orchestra.run.smithi116.stdout: "excluded": false, 2024-09-12T03:05:40.251 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:05:40.251 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "QUIESCED", 2024-09-12T03:05:40.251 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:05:40.251 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:05:40.251 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:05:40.252 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:05:40.252 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:05:40.252 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:05:40.252 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:05:40.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.249+0000 7f6a5b7ae640 1 -- 172.21.15.116:0/1681710495 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6a400a2c00 msgr2=0x7f6a400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:40.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.249+0000 7f6a5b7ae640 1 --2- 172.21.15.116:0/1681710495 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6a400a2c00 0x7f6a400a50c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c005fd0 tx=0x7f6a4c010740 comp rx=0 tx=0).stop 2024-09-12T03:05:40.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.249+0000 7f6a5b7ae640 1 -- 172.21.15.116:0/1681710495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a54100e20 msgr2=0x7f6a54198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:40.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.249+0000 7f6a5b7ae640 1 --2- 172.21.15.116:0/1681710495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a54100e20 0x7f6a54198580 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6a5000b4a0 tx=0x7f6a5000b960 comp rx=0 tx=0).stop 2024-09-12T03:05:40.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.249+0000 7f6a5b7ae640 1 -- 172.21.15.116:0/1681710495 shutdown_connections 2024-09-12T03:05:40.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.249+0000 7f6a5b7ae640 1 --2- 172.21.15.116:0/1681710495 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6a400a2c00 0x7f6a400a50c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.249+0000 7f6a5b7ae640 1 --2- 172.21.15.116:0/1681710495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a54102ef0 0x7f6a5419ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.249+0000 7f6a5b7ae640 1 --2- 172.21.15.116:0/1681710495 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a54102570 0x7f6a54198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.249+0000 7f6a5b7ae640 1 --2- 172.21.15.116:0/1681710495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a54100e20 0x7f6a54198580 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:40.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.249+0000 7f6a5b7ae640 1 -- 172.21.15.116:0/1681710495 >> 172.21.15.116:0/1681710495 conn(0x7f6a540fc7d0 msgr2=0x7f6a5406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:40.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.249+0000 7f6a5b7ae640 1 -- 172.21.15.116:0/1681710495 shutdown_connections 2024-09-12T03:05:40.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:40.249+0000 7f6a5b7ae640 1 -- 172.21.15.116:0/1681710495 wait complete. 2024-09-12T03:05:40.254 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:40.477 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:40 smithi116 ceph-mon[31410]: pgmap v418: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:05:40.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:40 smithi183 ceph-mon[35535]: pgmap v418: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:05:40.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:40 smithi177 ceph-mon[35305]: pgmap v418: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:05:41.126 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ff1640ad', quiesce took 4.8 seconds. Will release after: 19.9 2024-09-12T03:05:41.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.548+0000 7f4ceeefa640 1 -- 172.21.15.116:0/4239843995 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ce8101de0 msgr2=0x7f4ce81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:41.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.548+0000 7f4ceeefa640 1 --2- 172.21.15.116:0/4239843995 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ce8101de0 0x7f4ce81021c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8009900 tx=0x7f4cd802a410 comp rx=0 tx=0).stop 2024-09-12T03:05:41.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.548+0000 7f4ceeefa640 1 -- 172.21.15.116:0/4239843995 shutdown_connections 2024-09-12T03:05:41.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.548+0000 7f4ceeefa640 1 --2- 172.21.15.116:0/4239843995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ce8106f10 0x7f4ce810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.548+0000 7f4ceeefa640 1 --2- 172.21.15.116:0/4239843995 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ce8102790 0x7f4ce81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.548+0000 7f4ceeefa640 1 --2- 172.21.15.116:0/4239843995 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ce8101de0 0x7f4ce81021c0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.548+0000 7f4ceeefa640 1 -- 172.21.15.116:0/4239843995 >> 172.21.15.116:0/4239843995 conn(0x7f4ce80fd8c0 msgr2=0x7f4ce80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:41.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.549+0000 7f4ceeefa640 1 -- 172.21.15.116:0/4239843995 shutdown_connections 2024-09-12T03:05:41.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.549+0000 7f4ceeefa640 1 -- 172.21.15.116:0/4239843995 wait complete. 2024-09-12T03:05:41.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.549+0000 7f4ceeefa640 1 Processor -- start 2024-09-12T03:05:41.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.549+0000 7f4ceeefa640 1 -- start start 2024-09-12T03:05:41.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.549+0000 7f4ceeefa640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ce8101de0 0x7f4ce81983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:41.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.549+0000 7f4ceeefa640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ce8102790 0x7f4ce8198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:41.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.549+0000 7f4ceeefa640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ce8106f10 0x7f4ce819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:41.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.549+0000 7f4ceeefa640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce8111f70 con 0x7f4ce8102790 2024-09-12T03:05:41.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.549+0000 7f4ceeefa640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce8111df0 con 0x7f4ce8101de0 2024-09-12T03:05:41.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.549+0000 7f4ceeefa640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81120f0 con 0x7f4ce8106f10 2024-09-12T03:05:41.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.549+0000 7f4ce7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ce8102790 0x7f4ce8198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:41.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.549+0000 7f4ce7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ce8102790 0x7f4ce8198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40568/0 (socket says 172.21.15.116:40568) 2024-09-12T03:05:41.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.549+0000 7f4ce7fff640 1 -- 172.21.15.116:0/2755686353 learned_addr learned my addr 172.21.15.116:0/2755686353 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:41.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.550+0000 7f4ced497640 1 --2- 172.21.15.116:0/2755686353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ce8106f10 0x7f4ce819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:41.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.550+0000 7f4cecc96640 1 --2- 172.21.15.116:0/2755686353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ce8101de0 0x7f4ce81983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:41.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.550+0000 7f4ce7fff640 1 -- 172.21.15.116:0/2755686353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ce8106f10 msgr2=0x7f4ce819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:41.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.550+0000 7f4ce7fff640 1 --2- 172.21.15.116:0/2755686353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ce8106f10 0x7f4ce819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.550+0000 7f4ce7fff640 1 -- 172.21.15.116:0/2755686353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ce8101de0 msgr2=0x7f4ce81983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:41.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.550+0000 7f4ce7fff640 1 --2- 172.21.15.116:0/2755686353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ce8101de0 0x7f4ce81983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.550+0000 7f4ce7fff640 1 -- 172.21.15.116:0/2755686353 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce819d380 con 0x7f4ce8102790 2024-09-12T03:05:41.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.550+0000 7f4cecc96640 1 --2- 172.21.15.116:0/2755686353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ce8101de0 0x7f4ce81983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:41.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.550+0000 7f4ce7fff640 1 --2- 172.21.15.116:0/2755686353 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ce8102790 0x7f4ce8198910 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc00d770 tx=0x7f4cdc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:41.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.550+0000 7f4ce5ffb640 1 -- 172.21.15.116:0/2755686353 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc017070 con 0x7f4ce8102790 2024-09-12T03:05:41.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.550+0000 7f4ce5ffb640 1 -- 172.21.15.116:0/2755686353 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cdc005500 con 0x7f4ce8102790 2024-09-12T03:05:41.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.550+0000 7f4ceeefa640 1 -- 172.21.15.116:0/2755686353 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ce819d670 con 0x7f4ce8102790 2024-09-12T03:05:41.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.550+0000 7f4ce5ffb640 1 -- 172.21.15.116:0/2755686353 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc004e00 con 0x7f4ce8102790 2024-09-12T03:05:41.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.550+0000 7f4ceeefa640 1 -- 172.21.15.116:0/2755686353 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ce81a4f50 con 0x7f4ce8102790 2024-09-12T03:05:41.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.553+0000 7f4ce5ffb640 1 -- 172.21.15.116:0/2755686353 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4cdc004fa0 con 0x7f4ce8102790 2024-09-12T03:05:41.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.553+0000 7f4cf12d3740 1 Processor -- start 2024-09-12T03:05:41.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.553+0000 7f4cf12d3740 1 -- start start 2024-09-12T03:05:41.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.554+0000 7f4cf12d3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562b8558af50 0x562b855894a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:41.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.554+0000 7f4cf12d3740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b855899e0 0x562b85589e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:41.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.554+0000 7f4cf12d3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b85594780 0x562b85596b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:41.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.554+0000 7f4cf12d3740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562b854eb2f0 con 0x562b855899e0 2024-09-12T03:05:41.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.554+0000 7f4cf12d3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562b85492cf0 con 0x562b8558af50 2024-09-12T03:05:41.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.554+0000 7f4cf12d3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562b854e72b0 con 0x562b85594780 2024-09-12T03:05:41.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.554+0000 7f4cecc96640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b855899e0 0x562b85589e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:41.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.554+0000 7f4cecc96640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b855899e0 0x562b85589e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40582/0 (socket says 172.21.15.116:40582) 2024-09-12T03:05:41.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.554+0000 7f4cecc96640 1 -- 172.21.15.116:0/1430158711 learned_addr learned my addr 172.21.15.116:0/1430158711 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:41.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.554+0000 7f4ced497640 1 --2- 172.21.15.116:0/1430158711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562b8558af50 0x562b855894a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:41.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.554+0000 7f4ce5ffb640 1 --2- 172.21.15.116:0/2755686353 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4cd00a2b80 0x7f4cd00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:41.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.554+0000 7f4ce7fff640 1 --2- 172.21.15.116:0/1430158711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b85594780 0x562b85596b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:41.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.554+0000 7f4cecc96640 1 -- 172.21.15.116:0/1430158711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b85594780 msgr2=0x562b85596b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:41.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.554+0000 7f4cecc96640 1 --2- 172.21.15.116:0/1430158711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b85594780 0x562b85596b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.554+0000 7f4cecc96640 1 -- 172.21.15.116:0/1430158711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562b8558af50 msgr2=0x562b855894a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:41.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.554+0000 7f4cecc96640 1 --2- 172.21.15.116:0/1430158711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562b8558af50 0x562b855894a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.554+0000 7f4cecc96640 1 -- 172.21.15.116:0/1430158711 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b854773a0 con 0x562b855899e0 2024-09-12T03:05:41.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.554+0000 7f4ced497640 1 --2- 172.21.15.116:0/1430158711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562b8558af50 0x562b855894a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:41.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.554+0000 7f4ced497640 1 --2- 172.21.15.116:0/2755686353 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4cd00a2b80 0x7f4cd00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:41.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.554+0000 7f4ce5ffb640 1 -- 172.21.15.116:0/2755686353 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f4cdc0cea90 con 0x7f4ce8102790 2024-09-12T03:05:41.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.554+0000 7f4cecc96640 1 --2- 172.21.15.116:0/1430158711 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b855899e0 0x562b85589e20 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8033000 tx=0x7f4cd8032d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf546b520154cd26 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:41.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.554+0000 7f4cceffd640 1 -- 172.21.15.116:0/1430158711 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd803f070 con 0x562b855899e0 2024-09-12T03:05:41.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.555+0000 7f4ced497640 1 --2- 172.21.15.116:0/2755686353 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4cd00a2b80 0x7f4cd00a5040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00125f0 tx=0x7f4ce0004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:41.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.555+0000 7f4cceffd640 1 -- 172.21.15.116:0/1430158711 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cd80317b0 con 0x562b855899e0 2024-09-12T03:05:41.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.555+0000 7f4cceffd640 1 -- 172.21.15.116:0/1430158711 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8035070 con 0x562b855899e0 2024-09-12T03:05:41.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.555+0000 7f4cf12d3740 1 -- 172.21.15.116:0/1430158711 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b855899e0 msgr2=0x562b85589e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:41.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.555+0000 7f4cf12d3740 1 --2- 172.21.15.116:0/1430158711 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b855899e0 0x562b85589e20 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8033000 tx=0x7f4cd8032d60 comp rx=0 tx=0).stop 2024-09-12T03:05:41.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.555+0000 7f4cf12d3740 1 -- 172.21.15.116:0/1430158711 shutdown_connections 2024-09-12T03:05:41.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.555+0000 7f4cf12d3740 1 --2- 172.21.15.116:0/1430158711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b85594780 0x562b85596b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.555+0000 7f4cf12d3740 1 --2- 172.21.15.116:0/1430158711 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b855899e0 0x562b85589e20 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.555+0000 7f4cf12d3740 1 --2- 172.21.15.116:0/1430158711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562b8558af50 0x562b855894a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.555+0000 7f4cf12d3740 1 -- 172.21.15.116:0/1430158711 >> 172.21.15.116:0/1430158711 conn(0x562b8554e310 msgr2=0x562b85586070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.555+0000 7f4cf12d3740 1 -- 172.21.15.116:0/1430158711 shutdown_connections 2024-09-12T03:05:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.555+0000 7f4cf12d3740 1 -- 172.21.15.116:0/1430158711 wait complete. 2024-09-12T03:05:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.556+0000 7f4cf12d3740 1 -- start start 2024-09-12T03:05:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.556+0000 7f4cf12d3740 1 Processor -- start 2024-09-12T03:05:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.556+0000 7f4cf12d3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562b855899e0 0x562b85594190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.556+0000 7f4cf12d3740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b8558af50 0x562b85591440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.556+0000 7f4cf12d3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b85594780 0x562b85591a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.556+0000 7f4cf12d3740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562b85597420 con 0x562b8558af50 2024-09-12T03:05:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.556+0000 7f4cf12d3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562b854a6060 con 0x562b855899e0 2024-09-12T03:05:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.556+0000 7f4cf12d3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562b855975a0 con 0x562b85594780 2024-09-12T03:05:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.556+0000 7f4cf12d3740 10 client.0 _ll_register_callbacks cb 0x562b851f98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.556+0000 7f4ce7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b8558af50 0x562b85591440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.556+0000 7f4ce7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b8558af50 0x562b85591440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40590/0 (socket says 172.21.15.116:40590) 2024-09-12T03:05:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.556+0000 7f4ce7fff640 1 -- 172.21.15.116:0/2370774215 learned_addr learned my addr 172.21.15.116:0/2370774215 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.556+0000 7f4ced497640 1 --2- 172.21.15.116:0/2370774215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b85594780 0x562b85591a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.557+0000 7f4ce7fff640 1 -- 172.21.15.116:0/2370774215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b85594780 msgr2=0x562b85591a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.557+0000 7f4ce7fff640 1 --2- 172.21.15.116:0/2370774215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b85594780 0x562b85591a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.557+0000 7f4cecc96640 1 --2- 172.21.15.116:0/2370774215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562b855899e0 0x562b85594190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.557+0000 7f4ce7fff640 1 -- 172.21.15.116:0/2370774215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562b855899e0 msgr2=0x562b85594190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.557+0000 7f4ce7fff640 1 --2- 172.21.15.116:0/2370774215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562b855899e0 0x562b85594190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.557+0000 7f4ce7fff640 1 -- 172.21.15.116:0/2370774215 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b8525c3c0 con 0x562b8558af50 2024-09-12T03:05:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.557+0000 7f4ced497640 1 --2- 172.21.15.116:0/2370774215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b85594780 0x562b85591a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:05:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.557+0000 7f4cecc96640 1 --2- 172.21.15.116:0/2370774215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562b855899e0 0x562b85594190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:05:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.557+0000 7f4ce7fff640 1 --2- 172.21.15.116:0/2370774215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b8558af50 0x562b85591440 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc097cf0 tx=0x7f4cdc09ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e8501f8e2aaf68 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.557+0000 7f4cc7fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:05:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.557+0000 7f4cc7fff640 1 -- 172.21.15.116:0/2370774215 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc020040 con 0x562b8558af50 2024-09-12T03:05:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.557+0000 7f4cc7fff640 1 -- 172.21.15.116:0/2370774215 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f4cdc009cc0 con 0x562b8558af50 2024-09-12T03:05:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.557+0000 7f4cf12d3740 1 -- 172.21.15.116:0/2370774215 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cd80317b0 con 0x562b8558af50 2024-09-12T03:05:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.557+0000 7f4cc7fff640 1 -- 172.21.15.116:0/2370774215 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4cdc09b320 con 0x562b8558af50 2024-09-12T03:05:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.557+0000 7f4cf12d3740 10 client.15960 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.557+0000 7f4cf12d3740 1 -- 172.21.15.116:0/2370774215 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562b855a2ef0 con 0x562b8558af50 2024-09-12T03:05:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.557+0000 7f4cc7fff640 1 -- 172.21.15.116:0/2370774215 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cd80317b0 con 0x562b8558af50 2024-09-12T03:05:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.557+0000 7f4cc7fff640 1 -- 172.21.15.116:0/2370774215 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4cdc0b4170 con 0x562b8558af50 2024-09-12T03:05:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.557+0000 7f4cf12d3740 10 client.15960 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.557+0000 7f4cf12d3740 10 client.15960 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:05:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.557+0000 7f4cf12d3740 20 client.15960 populate_metadata read hostname 'smithi116' 2024-09-12T03:05:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.557+0000 7f4cf12d3740 1 --2- 172.21.15.116:0/2370774215 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x562b855a4400 0x562b855a6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.557+0000 7f4cf12d3740 4 client.15960 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:05:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.557+0000 7f4cf12d3740 1 -- 172.21.15.116:0/2370774215 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b854f8860 con 0x562b855a4400 2024-09-12T03:05:41.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.558+0000 7f4cecc96640 1 --2- 172.21.15.116:0/2370774215 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x562b855a4400 0x562b855a6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:41.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.558+0000 7f4cc7fff640 10 client.15960 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:05:41.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.558+0000 7f4cecc96640 1 --2- 172.21.15.116:0/2370774215 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x562b855a4400 0x562b855a6810 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f1b8a16f59fe82d0 server_cookie=662fffce19c80c9c in_seq=0 out_seq=0 2024-09-12T03:05:41.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.559+0000 7f4cc7fff640 1 -- 172.21.15.116:0/2370774215 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562b854f8860 con 0x562b855a4400 2024-09-12T03:05:41.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.559+0000 7f4cc7fff640 10 client.15960 handle_command_reply: tid=0 2024-09-12T03:05:41.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.559+0000 7f4cf12d3740 1 client.15960 shutdown 2024-09-12T03:05:41.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.559+0000 7f4cf12d3740 1 -- 172.21.15.116:0/2370774215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b8558af50 msgr2=0x562b85591440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:41.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.559+0000 7f4cf12d3740 1 --2- 172.21.15.116:0/2370774215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b8558af50 0x562b85591440 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc097cf0 tx=0x7f4cdc09ac80 comp rx=0 tx=0).stop 2024-09-12T03:05:41.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.559+0000 7f4cf12d3740 1 -- 172.21.15.116:0/2370774215 shutdown_connections 2024-09-12T03:05:41.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.559+0000 7f4cf12d3740 1 --2- 172.21.15.116:0/2370774215 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x562b855a4400 0x562b855a6810 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.559+0000 7f4cf12d3740 1 --2- 172.21.15.116:0/2370774215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b85594780 0x562b85591a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.559+0000 7f4cf12d3740 1 --2- 172.21.15.116:0/2370774215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b8558af50 0x562b85591440 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.559+0000 7f4cc7fff640 0 client.15960 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:05:41.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.559+0000 7f4cf12d3740 1 --2- 172.21.15.116:0/2370774215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562b855899e0 0x562b85594190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.559+0000 7f4cf12d3740 1 -- 172.21.15.116:0/2370774215 >> 172.21.15.116:0/2370774215 conn(0x562b8554e310 msgr2=0x562b8558bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:41.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.559+0000 7f4cf12d3740 1 -- 172.21.15.116:0/2370774215 shutdown_connections 2024-09-12T03:05:41.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.559+0000 7f4cf12d3740 1 -- 172.21.15.116:0/2370774215 wait complete. 2024-09-12T03:05:41.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.559+0000 7f4cf12d3740 20 client.15960 trim_cache size 0 max 16384 2024-09-12T03:05:41.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.572+0000 7f4cf12d3740 1 Processor -- start 2024-09-12T03:05:41.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.572+0000 7f4cf12d3740 1 -- start start 2024-09-12T03:05:41.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.572+0000 7f4cf12d3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562b855899e0 0x562b85590d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:41.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.572+0000 7f4cf12d3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b8558af50 0x562b855933b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:41.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.572+0000 7f4cf12d3740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85594780 0x562b8558c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:41.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.572+0000 7f4cf12d3740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81ada00 con 0x562b85594780 2024-09-12T03:05:41.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.572+0000 7f4cf12d3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81ab2d0 con 0x562b855899e0 2024-09-12T03:05:41.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.572+0000 7f4cf12d3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81a9140 con 0x562b8558af50 2024-09-12T03:05:41.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.572+0000 7f4ced497640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85594780 0x562b8558c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:41.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.572+0000 7f4ced497640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85594780 0x562b8558c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40602/0 (socket says 172.21.15.116:40602) 2024-09-12T03:05:41.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.572+0000 7f4ced497640 1 -- 172.21.15.116:0/2668618120 learned_addr learned my addr 172.21.15.116:0/2668618120 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:41.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.573+0000 7f4cecc96640 1 --2- 172.21.15.116:0/2668618120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562b855899e0 0x562b85590d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:41.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.573+0000 7f4ce7fff640 1 --2- 172.21.15.116:0/2668618120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b8558af50 0x562b855933b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:41.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.573+0000 7f4ced497640 1 -- 172.21.15.116:0/2668618120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b8558af50 msgr2=0x562b855933b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:41.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.573+0000 7f4ced497640 1 --2- 172.21.15.116:0/2668618120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b8558af50 0x562b855933b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.573+0000 7f4ced497640 1 -- 172.21.15.116:0/2668618120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562b855899e0 msgr2=0x562b85590d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:41.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.573+0000 7f4ced497640 1 --2- 172.21.15.116:0/2668618120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562b855899e0 0x562b85590d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.573+0000 7f4ced497640 1 -- 172.21.15.116:0/2668618120 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b855a6e90 con 0x562b85594780 2024-09-12T03:05:41.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.573+0000 7f4cecc96640 1 --2- 172.21.15.116:0/2668618120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562b855899e0 0x562b85590d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:41.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.573+0000 7f4ced497640 1 --2- 172.21.15.116:0/2668618120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85594780 0x562b8558c530 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0012240 tx=0x7f4ce0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251b66593f4bca16 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:41.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.573+0000 7f4cce7fc640 1 -- 172.21.15.116:0/2668618120 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce001b680 con 0x562b85594780 2024-09-12T03:05:41.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.573+0000 7f4cce7fc640 1 -- 172.21.15.116:0/2668618120 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ce001bd00 con 0x562b85594780 2024-09-12T03:05:41.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.573+0000 7f4cce7fc640 1 -- 172.21.15.116:0/2668618120 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0010430 con 0x562b85594780 2024-09-12T03:05:41.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.574+0000 7f4cf12d3740 1 -- 172.21.15.116:0/2668618120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85594780 msgr2=0x562b8558c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:41.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.574+0000 7f4cf12d3740 1 --2- 172.21.15.116:0/2668618120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85594780 0x562b8558c530 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0012240 tx=0x7f4ce0015000 comp rx=0 tx=0).stop 2024-09-12T03:05:41.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.574+0000 7f4cf12d3740 1 -- 172.21.15.116:0/2668618120 shutdown_connections 2024-09-12T03:05:41.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.574+0000 7f4cf12d3740 1 --2- 172.21.15.116:0/2668618120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85594780 0x562b8558c530 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.574+0000 7f4cf12d3740 1 --2- 172.21.15.116:0/2668618120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b8558af50 0x562b855933b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.574+0000 7f4cf12d3740 1 --2- 172.21.15.116:0/2668618120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562b855899e0 0x562b85590d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.574+0000 7f4cf12d3740 1 -- 172.21.15.116:0/2668618120 >> 172.21.15.116:0/2668618120 conn(0x562b8554e310 msgr2=0x562b85592ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:41.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.574+0000 7f4cf12d3740 1 -- 172.21.15.116:0/2668618120 shutdown_connections 2024-09-12T03:05:41.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.574+0000 7f4cf12d3740 1 -- 172.21.15.116:0/2668618120 wait complete. 2024-09-12T03:05:41.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.574+0000 7f4cf12d3740 1 -- start start 2024-09-12T03:05:41.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.574+0000 7f4cf12d3740 1 Processor -- start 2024-09-12T03:05:41.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.574+0000 7f4cf12d3740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b855899e0 0x562b8559cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:41.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.575+0000 7f4cf12d3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562b8558af50 0x562b85598280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:41.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.575+0000 7f4cf12d3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b85594780 0x562b855987c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:41.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.575+0000 7f4cf12d3740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81a9790 con 0x562b855899e0 2024-09-12T03:05:41.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.575+0000 7f4cf12d3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81aac00 con 0x562b8558af50 2024-09-12T03:05:41.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.575+0000 7f4cf12d3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81a9e60 con 0x562b85594780 2024-09-12T03:05:41.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.575+0000 7f4cf12d3740 10 client.0 _ll_register_callbacks cb 0x7f4cdc0976b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:41.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.575+0000 7f4cecc96640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b855899e0 0x562b8559cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:41.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.575+0000 7f4cecc96640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b855899e0 0x562b8559cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40608/0 (socket says 172.21.15.116:40608) 2024-09-12T03:05:41.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.575+0000 7f4cecc96640 1 -- 172.21.15.116:0/3864674571 learned_addr learned my addr 172.21.15.116:0/3864674571 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:41.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.575+0000 7f4ce7fff640 1 --2- 172.21.15.116:0/3864674571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562b8558af50 0x562b85598280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:41.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.575+0000 7f4ced497640 1 --2- 172.21.15.116:0/3864674571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b85594780 0x562b855987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:41.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.575+0000 7f4cecc96640 1 -- 172.21.15.116:0/3864674571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b85594780 msgr2=0x562b855987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:41.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.575+0000 7f4cecc96640 1 --2- 172.21.15.116:0/3864674571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b85594780 0x562b855987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.575+0000 7f4cecc96640 1 -- 172.21.15.116:0/3864674571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562b8558af50 msgr2=0x562b85598280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:41.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.575+0000 7f4cecc96640 1 --2- 172.21.15.116:0/3864674571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562b8558af50 0x562b85598280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.575+0000 7f4cecc96640 1 -- 172.21.15.116:0/3864674571 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b85598fc0 con 0x562b855899e0 2024-09-12T03:05:41.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.575+0000 7f4ced497640 1 --2- 172.21.15.116:0/3864674571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b85594780 0x562b855987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:41.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.575+0000 7f4cecc96640 1 --2- 172.21.15.116:0/3864674571 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b855899e0 0x562b8559cf50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8005420 tx=0x7f4cd8005450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729ee2bdeff682bd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:41.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.575+0000 7f4cc7fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:05:41.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.575+0000 7f4cc7fff640 1 -- 172.21.15.116:0/3864674571 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8050060 con 0x562b855899e0 2024-09-12T03:05:41.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.575+0000 7f4cf12d3740 1 -- 172.21.15.116:0/3864674571 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ce001bd00 con 0x562b855899e0 2024-09-12T03:05:41.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.575+0000 7f4cc7fff640 1 -- 172.21.15.116:0/3864674571 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f4cd8004040 con 0x562b855899e0 2024-09-12T03:05:41.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.575+0000 7f4cc7fff640 1 -- 172.21.15.116:0/3864674571 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4cd803f050 con 0x562b855899e0 2024-09-12T03:05:41.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.575+0000 7f4cf12d3740 10 client.15966 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:41.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.575+0000 7f4cf12d3740 1 -- 172.21.15.116:0/3864674571 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562b8560f0e0 con 0x562b855899e0 2024-09-12T03:05:41.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.576+0000 7f4cc7fff640 1 -- 172.21.15.116:0/3864674571 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ce001bd00 con 0x562b855899e0 2024-09-12T03:05:41.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.576+0000 7f4cc7fff640 1 -- 172.21.15.116:0/3864674571 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4cd804b410 con 0x562b855899e0 2024-09-12T03:05:41.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.576+0000 7f4cf12d3740 10 client.15966 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:41.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.576+0000 7f4cf12d3740 10 client.15966 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:05:41.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.576+0000 7f4cf12d3740 20 client.15966 populate_metadata read hostname 'smithi116' 2024-09-12T03:05:41.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.576+0000 7f4cf12d3740 1 --2- 172.21.15.116:0/3864674571 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x562b8560f3d0 0x562b856117c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:41.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.576+0000 7f4cf12d3740 4 client.15966 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:05:41.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.576+0000 7f4cf12d3740 1 -- 172.21.15.116:0/3864674571 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b854f2d70 con 0x562b8560f3d0 2024-09-12T03:05:41.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.576+0000 7f4ce7fff640 1 --2- 172.21.15.116:0/3864674571 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x562b8560f3d0 0x562b856117c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:41.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.577+0000 7f4cc7fff640 10 client.15966 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:05:41.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.577+0000 7f4ce7fff640 1 --2- 172.21.15.116:0/3864674571 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x562b8560f3d0 0x562b856117c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9e8ec0b2e1e1baed server_cookie=39d210353504ce93 in_seq=0 out_seq=0 2024-09-12T03:05:41.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.577+0000 7f4cc7fff640 1 -- 172.21.15.116:0/3864674571 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b854f2d70 con 0x562b8560f3d0 2024-09-12T03:05:41.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.577+0000 7f4cc7fff640 10 client.15966 handle_command_reply: tid=0 2024-09-12T03:05:41.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.577+0000 7f4cf12d3740 1 client.15966 shutdown 2024-09-12T03:05:41.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.577+0000 7f4cf12d3740 1 -- 172.21.15.116:0/3864674571 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b855899e0 msgr2=0x562b8559cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:41.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.577+0000 7f4cf12d3740 1 --2- 172.21.15.116:0/3864674571 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b855899e0 0x562b8559cf50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8005420 tx=0x7f4cd8005450 comp rx=0 tx=0).stop 2024-09-12T03:05:41.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.577+0000 7f4cf12d3740 1 -- 172.21.15.116:0/3864674571 shutdown_connections 2024-09-12T03:05:41.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.577+0000 7f4cf12d3740 1 --2- 172.21.15.116:0/3864674571 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x562b8560f3d0 0x562b856117c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.577+0000 7f4cf12d3740 1 --2- 172.21.15.116:0/3864674571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b85594780 0x562b855987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.577+0000 7f4cf12d3740 1 --2- 172.21.15.116:0/3864674571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562b8558af50 0x562b85598280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.577+0000 7f4cc7fff640 0 client.15966 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:05:41.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.577+0000 7f4cf12d3740 1 --2- 172.21.15.116:0/3864674571 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b855899e0 0x562b8559cf50 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.577+0000 7f4cf12d3740 1 -- 172.21.15.116:0/3864674571 >> 172.21.15.116:0/3864674571 conn(0x562b8554e310 msgr2=0x562b855934c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:41.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.577+0000 7f4cf12d3740 1 -- 172.21.15.116:0/3864674571 shutdown_connections 2024-09-12T03:05:41.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.577+0000 7f4cf12d3740 1 -- 172.21.15.116:0/3864674571 wait complete. 2024-09-12T03:05:41.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.577+0000 7f4cf12d3740 20 client.15966 trim_cache size 0 max 16384 2024-09-12T03:05:41.594 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:05:41.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.578+0000 7f4ceeefa640 1 -- 172.21.15.116:0/2755686353 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4cd00a2b80 msgr2=0x7f4cd00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:41.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.578+0000 7f4ceeefa640 1 --2- 172.21.15.116:0/2755686353 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4cd00a2b80 0x7f4cd00a5040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00125f0 tx=0x7f4ce0004a40 comp rx=0 tx=0).stop 2024-09-12T03:05:41.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.578+0000 7f4ceeefa640 1 -- 172.21.15.116:0/2755686353 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ce8102790 msgr2=0x7f4ce8198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:41.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.578+0000 7f4ceeefa640 1 --2- 172.21.15.116:0/2755686353 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ce8102790 0x7f4ce8198910 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc00d770 tx=0x7f4cdc00dc30 comp rx=0 tx=0).stop 2024-09-12T03:05:41.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.578+0000 7f4ceeefa640 1 -- 172.21.15.116:0/2755686353 shutdown_connections 2024-09-12T03:05:41.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.578+0000 7f4ceeefa640 1 --2- 172.21.15.116:0/2755686353 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4cd00a2b80 0x7f4cd00a5040 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.578+0000 7f4ceeefa640 1 --2- 172.21.15.116:0/2755686353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ce8106f10 0x7f4ce819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.578+0000 7f4ceeefa640 1 --2- 172.21.15.116:0/2755686353 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ce8102790 0x7f4ce8198910 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.578+0000 7f4ceeefa640 1 --2- 172.21.15.116:0/2755686353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ce8101de0 0x7f4ce81983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.578+0000 7f4ceeefa640 1 -- 172.21.15.116:0/2755686353 >> 172.21.15.116:0/2755686353 conn(0x7f4ce80fd8c0 msgr2=0x7f4ce80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:41.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.578+0000 7f4ceeefa640 1 -- 172.21.15.116:0/2755686353 shutdown_connections 2024-09-12T03:05:41.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:41.578+0000 7f4ceeefa640 1 -- 172.21.15.116:0/2755686353 wait complete. 2024-09-12T03:05:42.292 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:05:42.542 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:42.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:42 smithi116 ceph-mon[31410]: pgmap v419: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:05:42.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:42 smithi183 ceph-mon[35535]: pgmap v419: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:05:42.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:42 smithi177 ceph-mon[35305]: pgmap v419: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:05:43.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.065+0000 7fe12b0a6640 1 -- 172.21.15.116:0/3469115960 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe124102790 msgr2=0x7fe1241067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:43.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.065+0000 7fe12b0a6640 1 --2- 172.21.15.116:0/3469115960 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe124102790 0x7fe1241067e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe114009ca0 tx=0x7fe11402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.066+0000 7fe12b0a6640 1 -- 172.21.15.116:0/3469115960 shutdown_connections 2024-09-12T03:05:43.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.066+0000 7fe12b0a6640 1 --2- 172.21.15.116:0/3469115960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe124106f10 0x7fe12410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.066+0000 7fe12b0a6640 1 --2- 172.21.15.116:0/3469115960 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe124102790 0x7fe1241067e0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.066+0000 7fe12b0a6640 1 --2- 172.21.15.116:0/3469115960 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe124101de0 0x7fe1241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.066+0000 7fe12b0a6640 1 -- 172.21.15.116:0/3469115960 >> 172.21.15.116:0/3469115960 conn(0x7fe1240fd8c0 msgr2=0x7fe1240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:43.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.066+0000 7fe12b0a6640 1 -- 172.21.15.116:0/3469115960 shutdown_connections 2024-09-12T03:05:43.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.066+0000 7fe12b0a6640 1 -- 172.21.15.116:0/3469115960 wait complete. 2024-09-12T03:05:43.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.066+0000 7fe12b0a6640 1 Processor -- start 2024-09-12T03:05:43.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.066+0000 7fe12b0a6640 1 -- start start 2024-09-12T03:05:43.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.066+0000 7fe12b0a6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe124101de0 0x7fe124198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.066+0000 7fe12b0a6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe124102790 0x7fe124198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.066+0000 7fe12b0a6640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe124106f10 0x7fe12419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.066+0000 7fe12b0a6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe124111f90 con 0x7fe124102790 2024-09-12T03:05:43.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.066+0000 7fe12b0a6640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe124111e10 con 0x7fe124106f10 2024-09-12T03:05:43.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.066+0000 7fe12b0a6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe124112110 con 0x7fe124101de0 2024-09-12T03:05:43.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.066+0000 7fe123fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe124102790 0x7fe124198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.067+0000 7fe123fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe124102790 0x7fe124198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40628/0 (socket says 172.21.15.116:40628) 2024-09-12T03:05:43.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.067+0000 7fe123fff640 1 -- 172.21.15.116:0/855760952 learned_addr learned my addr 172.21.15.116:0/855760952 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:43.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.067+0000 7fe129643640 1 --2- 172.21.15.116:0/855760952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe124106f10 0x7fe12419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.067+0000 7fe123fff640 1 -- 172.21.15.116:0/855760952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe124101de0 msgr2=0x7fe124198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:43.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.067+0000 7fe123fff640 1 --2- 172.21.15.116:0/855760952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe124101de0 0x7fe124198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.067+0000 7fe123fff640 1 -- 172.21.15.116:0/855760952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe124106f10 msgr2=0x7fe12419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:43.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.067+0000 7fe123fff640 1 --2- 172.21.15.116:0/855760952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe124106f10 0x7fe12419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.067+0000 7fe123fff640 1 -- 172.21.15.116:0/855760952 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe12419d520 con 0x7fe124102790 2024-09-12T03:05:43.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.067+0000 7fe123fff640 1 --2- 172.21.15.116:0/855760952 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe124102790 0x7fe124198ab0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe1140097b0 tx=0x7fe114031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:43.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.067+0000 7fe121ffb640 1 -- 172.21.15.116:0/855760952 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11402fba0 con 0x7fe124102790 2024-09-12T03:05:43.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.067+0000 7fe12b0a6640 1 -- 172.21.15.116:0/855760952 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe12419d7b0 con 0x7fe124102790 2024-09-12T03:05:43.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.067+0000 7fe121ffb640 1 -- 172.21.15.116:0/855760952 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe11402fd40 con 0x7fe124102790 2024-09-12T03:05:43.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.067+0000 7fe121ffb640 1 -- 172.21.15.116:0/855760952 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11403b930 con 0x7fe124102790 2024-09-12T03:05:43.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.067+0000 7fe12b0a6640 1 -- 172.21.15.116:0/855760952 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1241a50f0 con 0x7fe124102790 2024-09-12T03:05:43.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.070+0000 7fe121ffb640 1 -- 172.21.15.116:0/855760952 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe114030cc0 con 0x7fe124102790 2024-09-12T03:05:43.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.070+0000 7fe12d47f740 1 Processor -- start 2024-09-12T03:05:43.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.070+0000 7fe12d47f740 1 -- start start 2024-09-12T03:05:43.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.070+0000 7fe12d47f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ed8b81cb40 0x55ed8b81b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.070+0000 7fe12d47f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8b81b5d0 0x55ed8b81ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.070+0000 7fe12d47f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed8b826370 0x55ed8b828760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.070+0000 7fe12d47f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ed8b7812f0 con 0x55ed8b81cb40 2024-09-12T03:05:43.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.070+0000 7fe12d47f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ed8b728cf0 con 0x55ed8b826370 2024-09-12T03:05:43.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.070+0000 7fe12d47f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ed8b77d2b0 con 0x55ed8b81b5d0 2024-09-12T03:05:43.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.070+0000 7fe121ffb640 1 --2- 172.21.15.116:0/855760952 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe1100a2b80 0x7fe1100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.070+0000 7fe129643640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ed8b81cb40 0x55ed8b81b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.070+0000 7fe129643640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ed8b81cb40 0x55ed8b81b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40632/0 (socket says 172.21.15.116:40632) 2024-09-12T03:05:43.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.070+0000 7fe129643640 1 -- 172.21.15.116:0/521710907 learned_addr learned my addr 172.21.15.116:0/521710907 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:43.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.070+0000 7fe129643640 1 --2- 172.21.15.116:0/855760952 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe1100a2b80 0x7fe1100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.071+0000 7fe121ffb640 1 -- 172.21.15.116:0/855760952 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe1140f07e0 con 0x7fe124102790 2024-09-12T03:05:43.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.071+0000 7fe123fff640 1 --2- 172.21.15.116:0/521710907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed8b826370 0x55ed8b828760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.071+0000 7fe128e42640 1 --2- 172.21.15.116:0/521710907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8b81b5d0 0x55ed8b81ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.071+0000 7fe129643640 1 -- 172.21.15.116:0/521710907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8b81b5d0 msgr2=0x55ed8b81ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:43.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.071+0000 7fe129643640 1 --2- 172.21.15.116:0/521710907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8b81b5d0 0x55ed8b81ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.071+0000 7fe129643640 1 -- 172.21.15.116:0/521710907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed8b826370 msgr2=0x55ed8b828760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:43.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.071+0000 7fe129643640 1 --2- 172.21.15.116:0/521710907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed8b826370 0x55ed8b828760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.071+0000 7fe129643640 1 -- 172.21.15.116:0/521710907 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed8b70d3a0 con 0x55ed8b81cb40 2024-09-12T03:05:43.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.071+0000 7fe128e42640 1 --2- 172.21.15.116:0/521710907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8b81b5d0 0x55ed8b81ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:43.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.071+0000 7fe129643640 1 --2- 172.21.15.116:0/521710907 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ed8b81cb40 0x55ed8b81b090 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe10c016740 tx=0x7fe10c016c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4149c4779edd3a9c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:43.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.071+0000 7fe129643640 1 --2- 172.21.15.116:0/855760952 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe1100a2b80 0x7fe1100a5040 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe10c006aa0 tx=0x7fe10c01a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:43.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.071+0000 7fe11affd640 1 -- 172.21.15.116:0/521710907 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c024070 con 0x55ed8b81cb40 2024-09-12T03:05:43.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.071+0000 7fe11affd640 1 -- 172.21.15.116:0/521710907 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe10c00a750 con 0x55ed8b81cb40 2024-09-12T03:05:43.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.071+0000 7fe11affd640 1 -- 172.21.15.116:0/521710907 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c01f650 con 0x55ed8b81cb40 2024-09-12T03:05:43.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.072+0000 7fe12d47f740 1 -- 172.21.15.116:0/521710907 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ed8b81cb40 msgr2=0x55ed8b81b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:43.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.072+0000 7fe12d47f740 1 --2- 172.21.15.116:0/521710907 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ed8b81cb40 0x55ed8b81b090 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe10c016740 tx=0x7fe10c016c00 comp rx=0 tx=0).stop 2024-09-12T03:05:43.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.072+0000 7fe12d47f740 1 -- 172.21.15.116:0/521710907 shutdown_connections 2024-09-12T03:05:43.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.072+0000 7fe12d47f740 1 --2- 172.21.15.116:0/521710907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed8b826370 0x55ed8b828760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.072+0000 7fe12d47f740 1 --2- 172.21.15.116:0/521710907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8b81b5d0 0x55ed8b81ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.072+0000 7fe12d47f740 1 --2- 172.21.15.116:0/521710907 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ed8b81cb40 0x55ed8b81b090 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.072+0000 7fe12d47f740 1 -- 172.21.15.116:0/521710907 >> 172.21.15.116:0/521710907 conn(0x55ed8b7e4310 msgr2=0x55ed8b817c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:43.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.072+0000 7fe12d47f740 1 -- 172.21.15.116:0/521710907 shutdown_connections 2024-09-12T03:05:43.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.072+0000 7fe12d47f740 1 -- 172.21.15.116:0/521710907 wait complete. 2024-09-12T03:05:43.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.072+0000 7fe12d47f740 1 -- start start 2024-09-12T03:05:43.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.072+0000 7fe12d47f740 1 Processor -- start 2024-09-12T03:05:43.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.073+0000 7fe12d47f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8b81b5d0 0x55ed8b825d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.073+0000 7fe12d47f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed8b81cb40 0x55ed8b823040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.073+0000 7fe12d47f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ed8b826370 0x55ed8b823650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.073+0000 7fe12d47f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ed8b829010 con 0x55ed8b826370 2024-09-12T03:05:43.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.073+0000 7fe12d47f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ed8b73c060 con 0x55ed8b81cb40 2024-09-12T03:05:43.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.073+0000 7fe12d47f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ed8b829190 con 0x55ed8b81b5d0 2024-09-12T03:05:43.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.073+0000 7fe12d47f740 10 client.0 _ll_register_callbacks cb 0x55ed8b48f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:43.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.073+0000 7fe129643640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ed8b826370 0x55ed8b823650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.073+0000 7fe129643640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ed8b826370 0x55ed8b823650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40640/0 (socket says 172.21.15.116:40640) 2024-09-12T03:05:43.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.073+0000 7fe129643640 1 -- 172.21.15.116:0/854534233 learned_addr learned my addr 172.21.15.116:0/854534233 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:43.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.073+0000 7fe128e42640 1 --2- 172.21.15.116:0/854534233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8b81b5d0 0x55ed8b825d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.073+0000 7fe123fff640 1 --2- 172.21.15.116:0/854534233 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed8b81cb40 0x55ed8b823040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.073+0000 7fe129643640 1 -- 172.21.15.116:0/854534233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8b81b5d0 msgr2=0x55ed8b825d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:43.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.073+0000 7fe129643640 1 --2- 172.21.15.116:0/854534233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8b81b5d0 0x55ed8b825d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.073+0000 7fe129643640 1 -- 172.21.15.116:0/854534233 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed8b81cb40 msgr2=0x55ed8b823040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:43.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.073+0000 7fe129643640 1 --2- 172.21.15.116:0/854534233 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed8b81cb40 0x55ed8b823040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.073+0000 7fe129643640 1 -- 172.21.15.116:0/854534233 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed8b4f23c0 con 0x55ed8b826370 2024-09-12T03:05:43.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.073+0000 7fe129643640 1 --2- 172.21.15.116:0/854534233 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ed8b826370 0x55ed8b823650 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe10c014720 tx=0x7fe10c016370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c61597ed4eff8d2e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:43.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.073+0000 7fe103fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:05:43.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.074+0000 7fe103fff640 1 -- 172.21.15.116:0/854534233 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c024040 con 0x55ed8b826370 2024-09-12T03:05:43.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.074+0000 7fe103fff640 1 -- 172.21.15.116:0/854534233 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe10c028430 con 0x55ed8b826370 2024-09-12T03:05:43.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.074+0000 7fe12d47f740 1 -- 172.21.15.116:0/854534233 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe10c00a750 con 0x55ed8b826370 2024-09-12T03:05:43.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.074+0000 7fe103fff640 1 -- 172.21.15.116:0/854534233 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe10c018c00 con 0x55ed8b826370 2024-09-12T03:05:43.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.074+0000 7fe12d47f740 10 client.15978 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:43.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.074+0000 7fe12d47f740 1 -- 172.21.15.116:0/854534233 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ed8b834b30 con 0x55ed8b826370 2024-09-12T03:05:43.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.074+0000 7fe103fff640 1 -- 172.21.15.116:0/854534233 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe10c00a750 con 0x55ed8b826370 2024-09-12T03:05:43.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.074+0000 7fe103fff640 1 -- 172.21.15.116:0/854534233 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe10c027070 con 0x55ed8b826370 2024-09-12T03:05:43.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.074+0000 7fe12d47f740 10 client.15978 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:43.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.074+0000 7fe12d47f740 10 client.15978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:05:43.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.074+0000 7fe12d47f740 20 client.15978 populate_metadata read hostname 'smithi116' 2024-09-12T03:05:43.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.074+0000 7fe12d47f740 1 --2- 172.21.15.116:0/854534233 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ed8b835f90 0x55ed8b838380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.074+0000 7fe12d47f740 4 client.15978 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:05:43.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.074+0000 7fe12d47f740 1 -- 172.21.15.116:0/854534233 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed8b78e860 con 0x55ed8b835f90 2024-09-12T03:05:43.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.074+0000 7fe128e42640 1 --2- 172.21.15.116:0/854534233 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ed8b835f90 0x55ed8b838380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.075+0000 7fe103fff640 10 client.15978 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:43.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.075+0000 7fe128e42640 1 --2- 172.21.15.116:0/854534233 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ed8b835f90 0x55ed8b838380 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=180d7a6fd98d0eb3 server_cookie=af2acfcc8ab8777b in_seq=0 out_seq=0 2024-09-12T03:05:43.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.076+0000 7fe103fff640 1 -- 172.21.15.116:0/854534233 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed8b78e860 con 0x55ed8b835f90 2024-09-12T03:05:43.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.076+0000 7fe103fff640 10 client.15978 handle_command_reply: tid=0 2024-09-12T03:05:43.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.076+0000 7fe12d47f740 1 client.15978 shutdown 2024-09-12T03:05:43.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.076+0000 7fe12d47f740 1 -- 172.21.15.116:0/854534233 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ed8b826370 msgr2=0x55ed8b823650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:43.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.076+0000 7fe12d47f740 1 --2- 172.21.15.116:0/854534233 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ed8b826370 0x55ed8b823650 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe10c014720 tx=0x7fe10c016370 comp rx=0 tx=0).stop 2024-09-12T03:05:43.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.076+0000 7fe12d47f740 1 -- 172.21.15.116:0/854534233 shutdown_connections 2024-09-12T03:05:43.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.076+0000 7fe12d47f740 1 --2- 172.21.15.116:0/854534233 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ed8b835f90 0x55ed8b838380 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.076+0000 7fe12d47f740 1 --2- 172.21.15.116:0/854534233 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ed8b826370 0x55ed8b823650 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.076+0000 7fe12d47f740 1 --2- 172.21.15.116:0/854534233 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed8b81cb40 0x55ed8b823040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.076+0000 7fe103fff640 0 client.15978 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:43.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.076+0000 7fe12d47f740 1 --2- 172.21.15.116:0/854534233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8b81b5d0 0x55ed8b825d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.076+0000 7fe12d47f740 1 -- 172.21.15.116:0/854534233 >> 172.21.15.116:0/854534233 conn(0x55ed8b7e4310 msgr2=0x55ed8b81d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:43.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.076+0000 7fe12d47f740 1 -- 172.21.15.116:0/854534233 shutdown_connections 2024-09-12T03:05:43.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.076+0000 7fe12d47f740 1 -- 172.21.15.116:0/854534233 wait complete. 2024-09-12T03:05:43.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.076+0000 7fe12d47f740 20 client.15978 trim_cache size 0 max 16384 2024-09-12T03:05:43.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.089+0000 7fe12d47f740 1 Processor -- start 2024-09-12T03:05:43.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.089+0000 7fe12d47f740 1 -- start start 2024-09-12T03:05:43.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.089+0000 7fe12d47f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8b81b5d0 0x55ed8b8322b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.089+0000 7fe12d47f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ed8b81cb40 0x55ed8b824fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.089+0000 7fe12d47f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed8b826370 0x55ed8b83d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.089+0000 7fe12d47f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe1241afc80 con 0x55ed8b81cb40 2024-09-12T03:05:43.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.089+0000 7fe12d47f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe1241ad550 con 0x55ed8b826370 2024-09-12T03:05:43.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.089+0000 7fe12d47f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe1241ab3c0 con 0x55ed8b81b5d0 2024-09-12T03:05:43.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.089+0000 7fe123fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ed8b81cb40 0x55ed8b824fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.089+0000 7fe123fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ed8b81cb40 0x55ed8b824fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40642/0 (socket says 172.21.15.116:40642) 2024-09-12T03:05:43.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.089+0000 7fe123fff640 1 -- 172.21.15.116:0/1352445279 learned_addr learned my addr 172.21.15.116:0/1352445279 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:43.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.089+0000 7fe123fff640 1 -- 172.21.15.116:0/1352445279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8b81b5d0 msgr2=0x55ed8b8322b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:43.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.089+0000 7fe128e42640 1 --2- 172.21.15.116:0/1352445279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8b81b5d0 0x55ed8b8322b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.089+0000 7fe123fff640 1 --2- 172.21.15.116:0/1352445279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8b81b5d0 0x55ed8b8322b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.089+0000 7fe123fff640 1 -- 172.21.15.116:0/1352445279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed8b826370 msgr2=0x55ed8b83d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:43.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.089+0000 7fe123fff640 1 --2- 172.21.15.116:0/1352445279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed8b826370 0x55ed8b83d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.089+0000 7fe123fff640 1 -- 172.21.15.116:0/1352445279 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed8b823fe0 con 0x55ed8b81cb40 2024-09-12T03:05:43.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.090+0000 7fe123fff640 1 --2- 172.21.15.116:0/1352445279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ed8b81cb40 0x55ed8b824fb0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe1140ee310 tx=0x7fe1140ee1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34886e9d00330ac server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:43.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.090+0000 7fe11a7fc640 1 -- 172.21.15.116:0/1352445279 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe114044380 con 0x55ed8b81cb40 2024-09-12T03:05:43.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.090+0000 7fe11a7fc640 1 -- 172.21.15.116:0/1352445279 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe114036040 con 0x55ed8b81cb40 2024-09-12T03:05:43.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.090+0000 7fe11a7fc640 1 -- 172.21.15.116:0/1352445279 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1140c10d0 con 0x55ed8b81cb40 2024-09-12T03:05:43.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.090+0000 7fe12d47f740 1 -- 172.21.15.116:0/1352445279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ed8b81cb40 msgr2=0x55ed8b824fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:43.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.090+0000 7fe12d47f740 1 --2- 172.21.15.116:0/1352445279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ed8b81cb40 0x55ed8b824fb0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe1140ee310 tx=0x7fe1140ee1e0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.090+0000 7fe12d47f740 1 -- 172.21.15.116:0/1352445279 shutdown_connections 2024-09-12T03:05:43.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.090+0000 7fe12d47f740 1 --2- 172.21.15.116:0/1352445279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed8b826370 0x55ed8b83d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.090+0000 7fe12d47f740 1 --2- 172.21.15.116:0/1352445279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ed8b81cb40 0x55ed8b824fb0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.090+0000 7fe12d47f740 1 --2- 172.21.15.116:0/1352445279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8b81b5d0 0x55ed8b8322b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.090+0000 7fe12d47f740 1 -- 172.21.15.116:0/1352445279 >> 172.21.15.116:0/1352445279 conn(0x55ed8b7e4310 msgr2=0x55ed8b8248a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:43.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.091+0000 7fe12d47f740 1 -- 172.21.15.116:0/1352445279 shutdown_connections 2024-09-12T03:05:43.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.091+0000 7fe12d47f740 1 -- 172.21.15.116:0/1352445279 wait complete. 2024-09-12T03:05:43.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.091+0000 7fe12d47f740 1 -- start start 2024-09-12T03:05:43.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.091+0000 7fe12d47f740 1 Processor -- start 2024-09-12T03:05:43.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.091+0000 7fe12d47f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8b81b5d0 0x55ed8b82ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.091+0000 7fe12d47f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ed8b81cb40 0x55ed8b82b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.091+0000 7fe12d47f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed8b826370 0x55ed8b89cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.091+0000 7fe12d47f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe1241aba10 con 0x55ed8b81cb40 2024-09-12T03:05:43.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.091+0000 7fe12d47f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe1241ace80 con 0x55ed8b826370 2024-09-12T03:05:43.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.091+0000 7fe12d47f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe1241ac0e0 con 0x55ed8b81b5d0 2024-09-12T03:05:43.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.091+0000 7fe12d47f740 10 client.0 _ll_register_callbacks cb 0x7fe10c022070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:43.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.091+0000 7fe123fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ed8b81cb40 0x55ed8b82b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.091+0000 7fe123fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ed8b81cb40 0x55ed8b82b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40658/0 (socket says 172.21.15.116:40658) 2024-09-12T03:05:43.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.091+0000 7fe123fff640 1 -- 172.21.15.116:0/675576405 learned_addr learned my addr 172.21.15.116:0/675576405 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:43.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.092+0000 7fe129643640 1 --2- 172.21.15.116:0/675576405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed8b826370 0x55ed8b89cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.092+0000 7fe128e42640 1 --2- 172.21.15.116:0/675576405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8b81b5d0 0x55ed8b82ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.092+0000 7fe123fff640 1 -- 172.21.15.116:0/675576405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8b81b5d0 msgr2=0x55ed8b82ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:43.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.092+0000 7fe123fff640 1 --2- 172.21.15.116:0/675576405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8b81b5d0 0x55ed8b82ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.092+0000 7fe123fff640 1 -- 172.21.15.116:0/675576405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed8b826370 msgr2=0x55ed8b89cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:43.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.092+0000 7fe123fff640 1 --2- 172.21.15.116:0/675576405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed8b826370 0x55ed8b89cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.092+0000 7fe123fff640 1 -- 172.21.15.116:0/675576405 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed8b89d410 con 0x55ed8b81cb40 2024-09-12T03:05:43.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.092+0000 7fe128e42640 1 --2- 172.21.15.116:0/675576405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8b81b5d0 0x55ed8b82ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:05:43.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.092+0000 7fe129643640 1 --2- 172.21.15.116:0/675576405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed8b826370 0x55ed8b89cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:43.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.092+0000 7fe123fff640 1 --2- 172.21.15.116:0/675576405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ed8b81cb40 0x55ed8b82b280 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe1140c57b0 tx=0x7fe1140c1540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae1448c8b321356 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:43.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.092+0000 7fe103fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:05:43.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.092+0000 7fe103fff640 1 -- 172.21.15.116:0/675576405 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1140d5030 con 0x55ed8b81cb40 2024-09-12T03:05:43.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.092+0000 7fe12d47f740 1 -- 172.21.15.116:0/675576405 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe114036040 con 0x55ed8b81cb40 2024-09-12T03:05:43.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.092+0000 7fe103fff640 1 -- 172.21.15.116:0/675576405 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe1140c10d0 con 0x55ed8b81cb40 2024-09-12T03:05:43.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.092+0000 7fe103fff640 1 -- 172.21.15.116:0/675576405 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe114088020 con 0x55ed8b81cb40 2024-09-12T03:05:43.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.092+0000 7fe12d47f740 10 client.15984 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:43.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.092+0000 7fe12d47f740 1 -- 172.21.15.116:0/675576405 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ed8b8a5910 con 0x55ed8b81cb40 2024-09-12T03:05:43.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.092+0000 7fe103fff640 1 -- 172.21.15.116:0/675576405 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe114036040 con 0x55ed8b81cb40 2024-09-12T03:05:43.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.093+0000 7fe103fff640 1 -- 172.21.15.116:0/675576405 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe1140c5850 con 0x55ed8b81cb40 2024-09-12T03:05:43.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.093+0000 7fe12d47f740 10 client.15984 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:43.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.093+0000 7fe12d47f740 10 client.15984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:05:43.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.093+0000 7fe12d47f740 20 client.15984 populate_metadata read hostname 'smithi116' 2024-09-12T03:05:43.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.093+0000 7fe12d47f740 1 --2- 172.21.15.116:0/675576405 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ed8b8a5fa0 0x55ed8b8a83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.093+0000 7fe12d47f740 4 client.15984 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:05:43.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.093+0000 7fe12d47f740 1 -- 172.21.15.116:0/675576405 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed8b788d70 con 0x55ed8b8a5fa0 2024-09-12T03:05:43.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.093+0000 7fe128e42640 1 --2- 172.21.15.116:0/675576405 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ed8b8a5fa0 0x55ed8b8a83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.093+0000 7fe103fff640 10 client.15984 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:43.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.093+0000 7fe128e42640 1 --2- 172.21.15.116:0/675576405 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ed8b8a5fa0 0x55ed8b8a83c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a487eed71e9fed4 server_cookie=c15135536fc639c6 in_seq=0 out_seq=0 2024-09-12T03:05:43.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.094+0000 7fe103fff640 1 -- 172.21.15.116:0/675576405 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed8b788d70 con 0x55ed8b8a5fa0 2024-09-12T03:05:43.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.094+0000 7fe103fff640 10 client.15984 handle_command_reply: tid=0 2024-09-12T03:05:43.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.094+0000 7fe12d47f740 1 client.15984 shutdown 2024-09-12T03:05:43.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.094+0000 7fe12d47f740 1 -- 172.21.15.116:0/675576405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ed8b81cb40 msgr2=0x55ed8b82b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:43.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.094+0000 7fe12d47f740 1 --2- 172.21.15.116:0/675576405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ed8b81cb40 0x55ed8b82b280 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe1140c57b0 tx=0x7fe1140c1540 comp rx=0 tx=0).stop 2024-09-12T03:05:43.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.094+0000 7fe12d47f740 1 -- 172.21.15.116:0/675576405 shutdown_connections 2024-09-12T03:05:43.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.094+0000 7fe12d47f740 1 --2- 172.21.15.116:0/675576405 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ed8b8a5fa0 0x55ed8b8a83c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.094+0000 7fe12d47f740 1 --2- 172.21.15.116:0/675576405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ed8b826370 0x55ed8b89cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.094+0000 7fe12d47f740 1 --2- 172.21.15.116:0/675576405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ed8b81cb40 0x55ed8b82b280 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.094+0000 7fe12d47f740 1 --2- 172.21.15.116:0/675576405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed8b81b5d0 0x55ed8b82ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.094+0000 7fe12d47f740 1 -- 172.21.15.116:0/675576405 >> 172.21.15.116:0/675576405 conn(0x55ed8b7e4310 msgr2=0x55ed8b816d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:43.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.094+0000 7fe103fff640 0 client.15984 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:43.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.094+0000 7fe12d47f740 1 -- 172.21.15.116:0/675576405 shutdown_connections 2024-09-12T03:05:43.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.094+0000 7fe12d47f740 1 -- 172.21.15.116:0/675576405 wait complete. 2024-09-12T03:05:43.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.094+0000 7fe12d47f740 20 client.15984 trim_cache size 0 max 16384 2024-09-12T03:05:43.106 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:05:43.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.094+0000 7fe12b0a6640 1 -- 172.21.15.116:0/855760952 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe1100a2b80 msgr2=0x7fe1100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:43.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.094+0000 7fe12b0a6640 1 --2- 172.21.15.116:0/855760952 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe1100a2b80 0x7fe1100a5040 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe10c006aa0 tx=0x7fe10c01a000 comp rx=0 tx=0).stop 2024-09-12T03:05:43.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.094+0000 7fe12b0a6640 1 -- 172.21.15.116:0/855760952 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe124102790 msgr2=0x7fe124198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:43.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.094+0000 7fe12b0a6640 1 --2- 172.21.15.116:0/855760952 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe124102790 0x7fe124198ab0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe1140097b0 tx=0x7fe114031b10 comp rx=0 tx=0).stop 2024-09-12T03:05:43.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.095+0000 7fe12b0a6640 1 -- 172.21.15.116:0/855760952 shutdown_connections 2024-09-12T03:05:43.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.095+0000 7fe12b0a6640 1 --2- 172.21.15.116:0/855760952 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe1100a2b80 0x7fe1100a5040 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.095+0000 7fe12b0a6640 1 --2- 172.21.15.116:0/855760952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe124106f10 0x7fe12419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.095+0000 7fe12b0a6640 1 --2- 172.21.15.116:0/855760952 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe124102790 0x7fe124198ab0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.095+0000 7fe12b0a6640 1 --2- 172.21.15.116:0/855760952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe124101de0 0x7fe124198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.095+0000 7fe12b0a6640 1 -- 172.21.15.116:0/855760952 >> 172.21.15.116:0/855760952 conn(0x7fe1240fd8c0 msgr2=0x7fe1240fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:43.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.095+0000 7fe12b0a6640 1 -- 172.21.15.116:0/855760952 shutdown_connections 2024-09-12T03:05:43.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:43.095+0000 7fe12b0a6640 1 -- 172.21.15.116:0/855760952 wait complete. 2024-09-12T03:05:44.547 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:44 smithi177 ceph-mon[35305]: pgmap v420: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:05:44.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:44 smithi116 ceph-mon[31410]: pgmap v420: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:05:44.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:44 smithi183 ceph-mon[35535]: pgmap v420: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:05:44.934 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:05:44.934 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:05:45.194 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:45.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.718+0000 7f960f76c640 1 -- 172.21.15.116:0/1346334551 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9608101430 msgr2=0x7f960810d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:45.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.718+0000 7f960f76c640 1 --2- 172.21.15.116:0/1346334551 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9608101430 0x7f960810d960 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9600009cd0 tx=0x7f960002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.718+0000 7f960f76c640 1 -- 172.21.15.116:0/1346334551 shutdown_connections 2024-09-12T03:05:45.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.718+0000 7f960f76c640 1 --2- 172.21.15.116:0/1346334551 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9608101430 0x7f960810d960 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.718+0000 7f960f76c640 1 --2- 172.21.15.116:0/1346334551 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9608100a90 0x7f9608100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.718+0000 7f960f76c640 1 --2- 172.21.15.116:0/1346334551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9608106a90 0x7f9608106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.718+0000 7f960f76c640 1 -- 172.21.15.116:0/1346334551 >> 172.21.15.116:0/1346334551 conn(0x7f96080fc7d0 msgr2=0x7f96080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:45.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.718+0000 7f960f76c640 1 -- 172.21.15.116:0/1346334551 shutdown_connections 2024-09-12T03:05:45.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.718+0000 7f960f76c640 1 -- 172.21.15.116:0/1346334551 wait complete. 2024-09-12T03:05:45.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.719+0000 7f960f76c640 1 Processor -- start 2024-09-12T03:05:45.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.719+0000 7f960f76c640 1 -- start start 2024-09-12T03:05:45.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.719+0000 7f960f76c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9608100a90 0x7f96081973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.719+0000 7f960f76c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9608101430 0x7f96081978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.719+0000 7f960f76c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9608106a90 0x7f960819bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.719+0000 7f960f76c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9608110ed0 con 0x7f9608100a90 2024-09-12T03:05:45.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.719+0000 7f960f76c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9608110d50 con 0x7f9608101430 2024-09-12T03:05:45.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.719+0000 7f960f76c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9608111050 con 0x7f9608106a90 2024-09-12T03:05:45.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.719+0000 7f960d508640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9608100a90 0x7f96081973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.719+0000 7f960d508640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9608100a90 0x7f96081973b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40674/0 (socket says 172.21.15.116:40674) 2024-09-12T03:05:45.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.719+0000 7f960d508640 1 -- 172.21.15.116:0/3606590886 learned_addr learned my addr 172.21.15.116:0/3606590886 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:45.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.719+0000 7f960cd07640 1 --2- 172.21.15.116:0/3606590886 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9608101430 0x7f96081978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.719+0000 7f960dd09640 1 --2- 172.21.15.116:0/3606590886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9608106a90 0x7f960819bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.719+0000 7f960d508640 1 -- 172.21.15.116:0/3606590886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9608106a90 msgr2=0x7f960819bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:45.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.719+0000 7f960d508640 1 --2- 172.21.15.116:0/3606590886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9608106a90 0x7f960819bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.719+0000 7f960d508640 1 -- 172.21.15.116:0/3606590886 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9608101430 msgr2=0x7f96081978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.719+0000 7f960d508640 1 --2- 172.21.15.116:0/3606590886 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9608101430 0x7f96081978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.719+0000 7f960d508640 1 -- 172.21.15.116:0/3606590886 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960819c360 con 0x7f9608100a90 2024-09-12T03:05:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.719+0000 7f960dd09640 1 --2- 172.21.15.116:0/3606590886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9608106a90 0x7f960819bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:05:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.720+0000 7f960d508640 1 --2- 172.21.15.116:0/3606590886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9608100a90 0x7f96081973b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00d770 tx=0x7f95fc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.720+0000 7f95fa7fc640 1 -- 172.21.15.116:0/3606590886 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc017070 con 0x7f9608100a90 2024-09-12T03:05:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.720+0000 7f95fa7fc640 1 -- 172.21.15.116:0/3606590886 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95fc005500 con 0x7f9608100a90 2024-09-12T03:05:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.720+0000 7f960f76c640 1 -- 172.21.15.116:0/3606590886 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f960819c650 con 0x7f9608100a90 2024-09-12T03:05:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.720+0000 7f95fa7fc640 1 -- 172.21.15.116:0/3606590886 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc004e00 con 0x7f9608100a90 2024-09-12T03:05:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.720+0000 7f960f76c640 1 -- 172.21.15.116:0/3606590886 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96081a3f30 con 0x7f9608100a90 2024-09-12T03:05:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.722+0000 7f95fa7fc640 1 -- 172.21.15.116:0/3606590886 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f95fc004fa0 con 0x7f9608100a90 2024-09-12T03:05:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.723+0000 7f9611b45740 1 Processor -- start 2024-09-12T03:05:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.723+0000 7f9611b45740 1 -- start start 2024-09-12T03:05:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.723+0000 7f95fa7fc640 1 --2- 172.21.15.116:0/3606590886 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f95f40a2b80 0x7f95f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.723+0000 7f9611b45740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564109374200 0x564109372a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.723+0000 7f9611b45740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564109372fd0 0x564109373410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.723+0000 7f9611b45740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564109381e00 0x5641093841f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.723+0000 7f9611b45740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5641092d8280 con 0x564109374200 2024-09-12T03:05:45.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.723+0000 7f9611b45740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56410927fb10 con 0x564109372fd0 2024-09-12T03:05:45.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.723+0000 7f9611b45740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5641092d4240 con 0x564109381e00 2024-09-12T03:05:45.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.723+0000 7f95fa7fc640 1 -- 172.21.15.116:0/3606590886 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f95fc0ceaf0 con 0x7f9608100a90 2024-09-12T03:05:45.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.723+0000 7f960dd09640 1 --2- 172.21.15.116:0/3606590886 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f95f40a2b80 0x7f95f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.723+0000 7f960d508640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564109374200 0x564109372a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.723+0000 7f960d508640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564109374200 0x564109372a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40678/0 (socket says 172.21.15.116:40678) 2024-09-12T03:05:45.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.723+0000 7f960d508640 1 -- 172.21.15.116:0/1595803905 learned_addr learned my addr 172.21.15.116:0/1595803905 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:45.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.723+0000 7f960cd07640 1 --2- 172.21.15.116:0/1595803905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564109372fd0 0x564109373410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.723+0000 7f960d508640 1 -- 172.21.15.116:0/1595803905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564109381e00 msgr2=0x5641093841f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:45.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.723+0000 7f960d508640 1 --2- 172.21.15.116:0/1595803905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564109381e00 0x5641093841f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.723+0000 7f960d508640 1 -- 172.21.15.116:0/1595803905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564109372fd0 msgr2=0x564109373410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:45.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.723+0000 7f960d508640 1 --2- 172.21.15.116:0/1595803905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564109372fd0 0x564109373410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.723+0000 7f960d508640 1 -- 172.21.15.116:0/1595803905 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56410928c1f0 con 0x564109374200 2024-09-12T03:05:45.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.723+0000 7f960dd09640 1 --2- 172.21.15.116:0/3606590886 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f95f40a2b80 0x7f95f40a5040 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9600009ca0 tx=0x7f96000099e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:45.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.724+0000 7f960d508640 1 --2- 172.21.15.116:0/1595803905 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564109374200 0x564109372a00 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f95fc09b080 tx=0x7f95fc009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492e8a97f8643d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:45.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.724+0000 7f95f37fe640 1 -- 172.21.15.116:0/1595803905 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc020070 con 0x564109374200 2024-09-12T03:05:45.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.724+0000 7f95f37fe640 1 -- 172.21.15.116:0/1595803905 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95fc09b380 con 0x564109374200 2024-09-12T03:05:45.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.724+0000 7f95f37fe640 1 -- 172.21.15.116:0/1595803905 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc0979f0 con 0x564109374200 2024-09-12T03:05:45.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.724+0000 7f9611b45740 1 -- 172.21.15.116:0/1595803905 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564109374200 msgr2=0x564109372a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:45.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.724+0000 7f9611b45740 1 --2- 172.21.15.116:0/1595803905 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564109374200 0x564109372a00 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f95fc09b080 tx=0x7f95fc009c30 comp rx=0 tx=0).stop 2024-09-12T03:05:45.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.724+0000 7f9611b45740 1 -- 172.21.15.116:0/1595803905 shutdown_connections 2024-09-12T03:05:45.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.724+0000 7f9611b45740 1 --2- 172.21.15.116:0/1595803905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564109381e00 0x5641093841f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.724+0000 7f9611b45740 1 --2- 172.21.15.116:0/1595803905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564109372fd0 0x564109373410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.724+0000 7f9611b45740 1 --2- 172.21.15.116:0/1595803905 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564109374200 0x564109372a00 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.724+0000 7f9611b45740 1 -- 172.21.15.116:0/1595803905 >> 172.21.15.116:0/1595803905 conn(0x56410936f0f0 msgr2=0x56410936f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:45.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.725+0000 7f9611b45740 1 -- 172.21.15.116:0/1595803905 shutdown_connections 2024-09-12T03:05:45.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.725+0000 7f9611b45740 1 -- 172.21.15.116:0/1595803905 wait complete. 2024-09-12T03:05:45.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.725+0000 7f9611b45740 1 -- start start 2024-09-12T03:05:45.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.725+0000 7f9611b45740 1 Processor -- start 2024-09-12T03:05:45.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.725+0000 7f9611b45740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564109372fd0 0x56410937ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.725+0000 7f9611b45740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564109374200 0x56410937f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.725+0000 7f9611b45740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564109381e00 0x56410937f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.725+0000 7f9611b45740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x564109384c90 con 0x564109381e00 2024-09-12T03:05:45.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.725+0000 7f9611b45740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x564109292f10 con 0x564109372fd0 2024-09-12T03:05:45.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.725+0000 7f9611b45740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564109384e10 con 0x564109374200 2024-09-12T03:05:45.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.725+0000 7f9611b45740 10 client.0 _ll_register_callbacks cb 0x56410925ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:45.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.726+0000 7f960dd09640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564109381e00 0x56410937f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.726+0000 7f960d508640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564109372fd0 0x56410937ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.726+0000 7f960dd09640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564109381e00 0x56410937f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40680/0 (socket says 172.21.15.116:40680) 2024-09-12T03:05:45.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.726+0000 7f960d508640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564109372fd0 0x56410937ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.116:49562/0 (socket says 172.21.15.116:49562) 2024-09-12T03:05:45.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.726+0000 7f960dd09640 1 -- 172.21.15.116:0/3852278831 learned_addr learned my addr 172.21.15.116:0/3852278831 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:45.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.726+0000 7f960cd07640 1 --2- 172.21.15.116:0/3852278831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564109374200 0x56410937f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.726+0000 7f960dd09640 1 -- 172.21.15.116:0/3852278831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564109374200 msgr2=0x56410937f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:45.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.726+0000 7f960dd09640 1 --2- 172.21.15.116:0/3852278831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564109374200 0x56410937f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.726+0000 7f960dd09640 1 -- 172.21.15.116:0/3852278831 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564109372fd0 msgr2=0x56410937ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:45.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.726+0000 7f960dd09640 1 --2- 172.21.15.116:0/3852278831 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564109372fd0 0x56410937ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.726+0000 7f960dd09640 1 -- 172.21.15.116:0/3852278831 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641090496a0 con 0x564109381e00 2024-09-12T03:05:45.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.726+0000 7f960d508640 1 --2- 172.21.15.116:0/3852278831 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564109372fd0 0x56410937ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:45.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.726+0000 7f960dd09640 1 --2- 172.21.15.116:0/3852278831 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564109381e00 0x56410937f860 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9600031830 tx=0x7f96000095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ffbe0bd23018f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:45.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.726+0000 7f95f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:05:45.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.726+0000 7f95f0ff9640 1 -- 172.21.15.116:0/3852278831 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960004c070 con 0x564109381e00 2024-09-12T03:05:45.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.726+0000 7f9611b45740 1 -- 172.21.15.116:0/3852278831 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95fc09b380 con 0x564109381e00 2024-09-12T03:05:45.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.726+0000 7f95f0ff9640 1 -- 172.21.15.116:0/3852278831 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f9600004510 con 0x564109381e00 2024-09-12T03:05:45.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.726+0000 7f95f0ff9640 1 -- 172.21.15.116:0/3852278831 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9600058050 con 0x564109381e00 2024-09-12T03:05:45.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.726+0000 7f9611b45740 10 client.15996 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:45.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.726+0000 7f9611b45740 1 -- 172.21.15.116:0/3852278831 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5641093909d0 con 0x564109381e00 2024-09-12T03:05:45.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.727+0000 7f95f0ff9640 1 -- 172.21.15.116:0/3852278831 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95fc09b380 con 0x564109381e00 2024-09-12T03:05:45.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.727+0000 7f95f0ff9640 1 -- 172.21.15.116:0/3852278831 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9600030070 con 0x564109381e00 2024-09-12T03:05:45.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.727+0000 7f9611b45740 10 client.15996 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:45.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.727+0000 7f9611b45740 10 client.15996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:05:45.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.727+0000 7f9611b45740 20 client.15996 populate_metadata read hostname 'smithi116' 2024-09-12T03:05:45.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.727+0000 7f9611b45740 1 --2- 172.21.15.116:0/3852278831 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x564109391e30 0x564109394290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.727+0000 7f9611b45740 4 client.15996 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:05:45.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.727+0000 7f9611b45740 1 -- 172.21.15.116:0/3852278831 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641092e57f0 con 0x564109391e30 2024-09-12T03:05:45.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.727+0000 7f960d508640 1 --2- 172.21.15.116:0/3852278831 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x564109391e30 0x564109394290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.728+0000 7f95f0ff9640 10 client.15996 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:45.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.728+0000 7f960d508640 1 --2- 172.21.15.116:0/3852278831 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x564109391e30 0x564109394290 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b31db6287ad02620 server_cookie=61a7a6bf04cb9ca0 in_seq=0 out_seq=0 2024-09-12T03:05:45.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.728+0000 7f95f0ff9640 1 -- 172.21.15.116:0/3852278831 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5641092e57f0 con 0x564109391e30 2024-09-12T03:05:45.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.728+0000 7f95f0ff9640 10 client.15996 handle_command_reply: tid=0 2024-09-12T03:05:45.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.728+0000 7f9611b45740 1 client.15996 shutdown 2024-09-12T03:05:45.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.728+0000 7f9611b45740 1 -- 172.21.15.116:0/3852278831 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564109381e00 msgr2=0x56410937f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:45.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.728+0000 7f9611b45740 1 --2- 172.21.15.116:0/3852278831 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564109381e00 0x56410937f860 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9600031830 tx=0x7f96000095c0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.728+0000 7f9611b45740 1 -- 172.21.15.116:0/3852278831 shutdown_connections 2024-09-12T03:05:45.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.728+0000 7f9611b45740 1 --2- 172.21.15.116:0/3852278831 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x564109391e30 0x564109394290 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.728+0000 7f9611b45740 1 --2- 172.21.15.116:0/3852278831 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564109381e00 0x56410937f860 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.728+0000 7f9611b45740 1 --2- 172.21.15.116:0/3852278831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564109374200 0x56410937f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.728+0000 7f95f0ff9640 0 client.15996 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:45.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.728+0000 7f9611b45740 1 --2- 172.21.15.116:0/3852278831 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564109372fd0 0x56410937ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.728+0000 7f9611b45740 1 -- 172.21.15.116:0/3852278831 >> 172.21.15.116:0/3852278831 conn(0x56410936f0f0 msgr2=0x564109374e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:45.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.729+0000 7f9611b45740 1 -- 172.21.15.116:0/3852278831 shutdown_connections 2024-09-12T03:05:45.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.729+0000 7f9611b45740 1 -- 172.21.15.116:0/3852278831 wait complete. 2024-09-12T03:05:45.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.729+0000 7f9611b45740 20 client.15996 trim_cache size 0 max 16384 2024-09-12T03:05:45.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.746+0000 7f9611b45740 1 Processor -- start 2024-09-12T03:05:45.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.746+0000 7f9611b45740 1 -- start start 2024-09-12T03:05:45.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.746+0000 7f9611b45740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564109372fd0 0x5641093841a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.746+0000 7f9611b45740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564109374200 0x564109399b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.746+0000 7f9611b45740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564109381e00 0x56410939a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.746+0000 7f9611b45740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f96081adbd0 con 0x564109374200 2024-09-12T03:05:45.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.746+0000 7f9611b45740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f96081ab4a0 con 0x564109381e00 2024-09-12T03:05:45.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.746+0000 7f9611b45740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f96081a9310 con 0x564109372fd0 2024-09-12T03:05:45.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.746+0000 7f960cd07640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564109374200 0x564109399b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.746+0000 7f960cd07640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564109374200 0x564109399b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40692/0 (socket says 172.21.15.116:40692) 2024-09-12T03:05:45.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.746+0000 7f960cd07640 1 -- 172.21.15.116:0/2085641304 learned_addr learned my addr 172.21.15.116:0/2085641304 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:45.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.746+0000 7f960d508640 1 --2- 172.21.15.116:0/2085641304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564109372fd0 0x5641093841a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.746+0000 7f960cd07640 1 -- 172.21.15.116:0/2085641304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564109372fd0 msgr2=0x5641093841a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:45.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.746+0000 7f960cd07640 1 --2- 172.21.15.116:0/2085641304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564109372fd0 0x5641093841a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.746+0000 7f960dd09640 1 --2- 172.21.15.116:0/2085641304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564109381e00 0x56410939a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.746+0000 7f960cd07640 1 -- 172.21.15.116:0/2085641304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564109381e00 msgr2=0x56410939a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:45.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.746+0000 7f960cd07640 1 --2- 172.21.15.116:0/2085641304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564109381e00 0x56410939a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.746+0000 7f960cd07640 1 -- 172.21.15.116:0/2085641304 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564109370210 con 0x564109374200 2024-09-12T03:05:45.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.746+0000 7f960dd09640 1 --2- 172.21.15.116:0/2085641304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564109381e00 0x56410939a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:05:45.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.746+0000 7f960cd07640 1 --2- 172.21.15.116:0/2085641304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564109374200 0x564109399b60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f960400b950 tx=0x7f960400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97f1db92af8562e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:45.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.747+0000 7f95f2ffd640 1 -- 172.21.15.116:0/2085641304 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9604013b70 con 0x564109374200 2024-09-12T03:05:45.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.747+0000 7f95f2ffd640 1 -- 172.21.15.116:0/2085641304 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9604013d10 con 0x564109374200 2024-09-12T03:05:45.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.747+0000 7f95f2ffd640 1 -- 172.21.15.116:0/2085641304 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96040147a0 con 0x564109374200 2024-09-12T03:05:45.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.747+0000 7f9611b45740 1 -- 172.21.15.116:0/2085641304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564109374200 msgr2=0x564109399b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:45.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.747+0000 7f9611b45740 1 --2- 172.21.15.116:0/2085641304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564109374200 0x564109399b60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f960400b950 tx=0x7f960400be10 comp rx=0 tx=0).stop 2024-09-12T03:05:45.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.747+0000 7f9611b45740 1 -- 172.21.15.116:0/2085641304 shutdown_connections 2024-09-12T03:05:45.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.747+0000 7f9611b45740 1 --2- 172.21.15.116:0/2085641304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564109381e00 0x56410939a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.747+0000 7f9611b45740 1 --2- 172.21.15.116:0/2085641304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564109374200 0x564109399b60 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.747+0000 7f9611b45740 1 --2- 172.21.15.116:0/2085641304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564109372fd0 0x5641093841a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.747+0000 7f9611b45740 1 -- 172.21.15.116:0/2085641304 >> 172.21.15.116:0/2085641304 conn(0x56410936f0f0 msgr2=0x564109383a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:45.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.747+0000 7f9611b45740 1 -- 172.21.15.116:0/2085641304 shutdown_connections 2024-09-12T03:05:45.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.747+0000 7f9611b45740 1 -- 172.21.15.116:0/2085641304 wait complete. 2024-09-12T03:05:45.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.748+0000 7f9611b45740 1 -- start start 2024-09-12T03:05:45.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.748+0000 7f9611b45740 1 Processor -- start 2024-09-12T03:05:45.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.748+0000 7f9611b45740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564109372fd0 0x56410938a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.748+0000 7f9611b45740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564109374200 0x56410938a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.748+0000 7f9611b45740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564109381e00 0x564109387090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.748+0000 7f9611b45740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f96081a9960 con 0x564109381e00 2024-09-12T03:05:45.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.748+0000 7f9611b45740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f96081aadd0 con 0x564109372fd0 2024-09-12T03:05:45.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.748+0000 7f9611b45740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f96081aa030 con 0x564109374200 2024-09-12T03:05:45.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.748+0000 7f9611b45740 10 client.0 _ll_register_callbacks cb 0x7f9600041050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:45.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.748+0000 7f960dd09640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564109381e00 0x564109387090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.748+0000 7f960dd09640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564109381e00 0x564109387090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40698/0 (socket says 172.21.15.116:40698) 2024-09-12T03:05:45.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.748+0000 7f960dd09640 1 -- 172.21.15.116:0/4157820453 learned_addr learned my addr 172.21.15.116:0/4157820453 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:45.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.748+0000 7f960d508640 1 --2- 172.21.15.116:0/4157820453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564109372fd0 0x56410938a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.749+0000 7f960cd07640 1 --2- 172.21.15.116:0/4157820453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564109374200 0x56410938a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.749+0000 7f960dd09640 1 -- 172.21.15.116:0/4157820453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564109374200 msgr2=0x56410938a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:45.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.749+0000 7f960dd09640 1 --2- 172.21.15.116:0/4157820453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564109374200 0x56410938a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.749+0000 7f960dd09640 1 -- 172.21.15.116:0/4157820453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564109372fd0 msgr2=0x56410938a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:45.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.749+0000 7f960dd09640 1 --2- 172.21.15.116:0/4157820453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564109372fd0 0x56410938a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.749+0000 7f960dd09640 1 -- 172.21.15.116:0/4157820453 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641093878c0 con 0x564109381e00 2024-09-12T03:05:45.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.749+0000 7f960d508640 1 --2- 172.21.15.116:0/4157820453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564109372fd0 0x56410938a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:45.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.749+0000 7f960cd07640 1 --2- 172.21.15.116:0/4157820453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564109374200 0x56410938a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:05:45.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.749+0000 7f960dd09640 1 --2- 172.21.15.116:0/4157820453 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564109381e00 0x564109387090 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9600048a00 tx=0x7f9600048a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54103038955c36a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:45.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.749+0000 7f95f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:05:45.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.749+0000 7f95f0ff9640 1 -- 172.21.15.116:0/4157820453 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9600061060 con 0x564109381e00 2024-09-12T03:05:45.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.749+0000 7f95f0ff9640 1 -- 172.21.15.116:0/4157820453 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f9600048d40 con 0x564109381e00 2024-09-12T03:05:45.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.749+0000 7f9611b45740 1 -- 172.21.15.116:0/4157820453 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9604013d10 con 0x564109381e00 2024-09-12T03:05:45.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.749+0000 7f95f0ff9640 1 -- 172.21.15.116:0/4157820453 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f960004c070 con 0x564109381e00 2024-09-12T03:05:45.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.749+0000 7f9611b45740 10 client.16002 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:45.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.749+0000 7f9611b45740 1 -- 172.21.15.116:0/4157820453 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5641093fd630 con 0x564109381e00 2024-09-12T03:05:45.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.749+0000 7f95f0ff9640 1 -- 172.21.15.116:0/4157820453 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9604013d10 con 0x564109381e00 2024-09-12T03:05:45.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.749+0000 7f95f0ff9640 1 -- 172.21.15.116:0/4157820453 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9600030a60 con 0x564109381e00 2024-09-12T03:05:45.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.750+0000 7f9611b45740 10 client.16002 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:45.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.750+0000 7f9611b45740 10 client.16002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:05:45.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.750+0000 7f9611b45740 20 client.16002 populate_metadata read hostname 'smithi116' 2024-09-12T03:05:45.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.750+0000 7f9611b45740 1 --2- 172.21.15.116:0/4157820453 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5641093fdcf0 0x564109400130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.750+0000 7f9611b45740 4 client.16002 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:05:45.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.750+0000 7f9611b45740 1 -- 172.21.15.116:0/4157820453 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5641092dfd00 con 0x5641093fdcf0 2024-09-12T03:05:45.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.750+0000 7f960d508640 1 --2- 172.21.15.116:0/4157820453 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5641093fdcf0 0x564109400130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.750+0000 7f95f0ff9640 10 client.16002 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:45.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.750+0000 7f960d508640 1 --2- 172.21.15.116:0/4157820453 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5641093fdcf0 0x564109400130 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bbc20b3891e0ef5 server_cookie=34677f879c2decb8 in_seq=0 out_seq=0 2024-09-12T03:05:45.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.751+0000 7f95f0ff9640 1 -- 172.21.15.116:0/4157820453 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5641092dfd00 con 0x5641093fdcf0 2024-09-12T03:05:45.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.751+0000 7f95f0ff9640 10 client.16002 handle_command_reply: tid=0 2024-09-12T03:05:45.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.751+0000 7f9611b45740 1 client.16002 shutdown 2024-09-12T03:05:45.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.752+0000 7f9611b45740 1 -- 172.21.15.116:0/4157820453 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564109381e00 msgr2=0x564109387090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:45.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.752+0000 7f9611b45740 1 --2- 172.21.15.116:0/4157820453 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564109381e00 0x564109387090 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9600048a00 tx=0x7f9600048a30 comp rx=0 tx=0).stop 2024-09-12T03:05:45.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.752+0000 7f9611b45740 1 -- 172.21.15.116:0/4157820453 shutdown_connections 2024-09-12T03:05:45.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.752+0000 7f9611b45740 1 --2- 172.21.15.116:0/4157820453 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5641093fdcf0 0x564109400130 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.752+0000 7f9611b45740 1 --2- 172.21.15.116:0/4157820453 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564109381e00 0x564109387090 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.752+0000 7f9611b45740 1 --2- 172.21.15.116:0/4157820453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564109374200 0x56410938a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.752+0000 7f9611b45740 1 --2- 172.21.15.116:0/4157820453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564109372fd0 0x56410938a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.752+0000 7f95f0ff9640 0 client.16002 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:45.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.752+0000 7f9611b45740 1 -- 172.21.15.116:0/4157820453 >> 172.21.15.116:0/4157820453 conn(0x56410936f0f0 msgr2=0x56410936da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:45.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.752+0000 7f9611b45740 1 -- 172.21.15.116:0/4157820453 shutdown_connections 2024-09-12T03:05:45.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.752+0000 7f9611b45740 1 -- 172.21.15.116:0/4157820453 wait complete. 2024-09-12T03:05:45.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.752+0000 7f9611b45740 20 client.16002 trim_cache size 0 max 16384 2024-09-12T03:05:45.758 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:05:45.758 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:05:45.758 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "92a01a02-7630-431c-8a26-094739a23fe1", 2024-09-12T03:05:45.758 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:05:45.759 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:05:45.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.752+0000 7f960f76c640 1 -- 172.21.15.116:0/3606590886 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f95f40a2b80 msgr2=0x7f95f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:45.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.752+0000 7f960f76c640 1 --2- 172.21.15.116:0/3606590886 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f95f40a2b80 0x7f95f40a5040 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9600009ca0 tx=0x7f96000099e0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.752+0000 7f960f76c640 1 -- 172.21.15.116:0/3606590886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9608100a90 msgr2=0x7f96081973b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:45.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.752+0000 7f960f76c640 1 --2- 172.21.15.116:0/3606590886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9608100a90 0x7f96081973b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00d770 tx=0x7f95fc00dc30 comp rx=0 tx=0).stop 2024-09-12T03:05:45.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.753+0000 7f960f76c640 1 -- 172.21.15.116:0/3606590886 shutdown_connections 2024-09-12T03:05:45.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.753+0000 7f960f76c640 1 --2- 172.21.15.116:0/3606590886 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f95f40a2b80 0x7f95f40a5040 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.753+0000 7f960f76c640 1 --2- 172.21.15.116:0/3606590886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9608106a90 0x7f960819bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.753+0000 7f960f76c640 1 --2- 172.21.15.116:0/3606590886 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9608101430 0x7f96081978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.753+0000 7f960f76c640 1 --2- 172.21.15.116:0/3606590886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9608100a90 0x7f96081973b0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.753+0000 7f960f76c640 1 -- 172.21.15.116:0/3606590886 >> 172.21.15.116:0/3606590886 conn(0x7f96080fc7d0 msgr2=0x7f96080fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:45.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.753+0000 7f960f76c640 1 -- 172.21.15.116:0/3606590886 shutdown_connections 2024-09-12T03:05:45.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:45.753+0000 7f960f76c640 1 -- 172.21.15.116:0/3606590886 wait complete. 2024-09-12T03:05:46.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:46 smithi116 ceph-mon[31410]: pgmap v421: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:05:46.661 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:05:46.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:46 smithi183 ceph-mon[35535]: pgmap v421: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:05:46.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:46 smithi177 ceph-mon[35305]: pgmap v421: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:05:46.906 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:47.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:47 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:05:47.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:47 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:05:47.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:47 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:05:47.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:47 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:05:47.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.428+0000 7f4cc2fab640 1 -- 172.21.15.116:0/485881091 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cbc100e20 msgr2=0x7f4cbc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:47.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.428+0000 7f4cc2fab640 1 --2- 172.21.15.116:0/485881091 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cbc100e20 0x7f4cbc101200 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4cac009ca0 tx=0x7f4cac02a950 comp rx=0 tx=0).stop 2024-09-12T03:05:47.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.428+0000 7f4cc2fab640 1 -- 172.21.15.116:0/485881091 shutdown_connections 2024-09-12T03:05:47.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.428+0000 7f4cc2fab640 1 --2- 172.21.15.116:0/485881091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cbc102ef0 0x7f4cbc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.428+0000 7f4cc2fab640 1 --2- 172.21.15.116:0/485881091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cbc102570 0x7f4cbc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.428+0000 7f4cc2fab640 1 --2- 172.21.15.116:0/485881091 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cbc100e20 0x7f4cbc101200 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.428+0000 7f4cc2fab640 1 -- 172.21.15.116:0/485881091 >> 172.21.15.116:0/485881091 conn(0x7f4cbc0fc7d0 msgr2=0x7f4cbc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:47.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.428+0000 7f4cc2fab640 1 -- 172.21.15.116:0/485881091 shutdown_connections 2024-09-12T03:05:47.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.429+0000 7f4cc2fab640 1 -- 172.21.15.116:0/485881091 wait complete. 2024-09-12T03:05:47.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.429+0000 7f4cc2fab640 1 Processor -- start 2024-09-12T03:05:47.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.429+0000 7f4cc2fab640 1 -- start start 2024-09-12T03:05:47.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.429+0000 7f4cc2fab640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cbc100e20 0x7f4cbc1993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.429+0000 7f4cc2fab640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cbc102570 0x7f4cbc1998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.429+0000 7f4cc2fab640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cbc102ef0 0x7f4cbc1a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.429+0000 7f4cc2fab640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4cbc110fa0 con 0x7f4cbc102570 2024-09-12T03:05:47.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.429+0000 7f4cc2fab640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4cbc110e20 con 0x7f4cbc100e20 2024-09-12T03:05:47.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.429+0000 7f4cc2fab640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4cbc111120 con 0x7f4cbc102ef0 2024-09-12T03:05:47.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.429+0000 7f4cbbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cbc102570 0x7f4cbc1998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.429+0000 7f4cbbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cbc102570 0x7f4cbc1998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40712/0 (socket says 172.21.15.116:40712) 2024-09-12T03:05:47.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.429+0000 7f4cbbfff640 1 -- 172.21.15.116:0/216843206 learned_addr learned my addr 172.21.15.116:0/216843206 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:47.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.430+0000 7f4cc0d47640 1 --2- 172.21.15.116:0/216843206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cbc100e20 0x7f4cbc1993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.430+0000 7f4cc1548640 1 --2- 172.21.15.116:0/216843206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cbc102ef0 0x7f4cbc1a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.430+0000 7f4cbbfff640 1 -- 172.21.15.116:0/216843206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cbc102ef0 msgr2=0x7f4cbc1a4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:47.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.430+0000 7f4cbbfff640 1 --2- 172.21.15.116:0/216843206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cbc102ef0 0x7f4cbc1a4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.430+0000 7f4cbbfff640 1 -- 172.21.15.116:0/216843206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cbc100e20 msgr2=0x7f4cbc1993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:47.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.430+0000 7f4cbbfff640 1 --2- 172.21.15.116:0/216843206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cbc100e20 0x7f4cbc1993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.430+0000 7f4cbbfff640 1 -- 172.21.15.116:0/216843206 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cbc1a4970 con 0x7f4cbc102570 2024-09-12T03:05:47.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.430+0000 7f4cbbfff640 1 --2- 172.21.15.116:0/216843206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cbc102570 0x7f4cbc1998f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4ca400e9f0 tx=0x7f4ca400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:47.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.430+0000 7f4cb9ffb640 1 -- 172.21.15.116:0/216843206 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca400cd80 con 0x7f4cbc102570 2024-09-12T03:05:47.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.430+0000 7f4cb9ffb640 1 -- 172.21.15.116:0/216843206 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ca4010910 con 0x7f4cbc102570 2024-09-12T03:05:47.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.430+0000 7f4cb9ffb640 1 -- 172.21.15.116:0/216843206 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4018990 con 0x7f4cbc102570 2024-09-12T03:05:47.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.430+0000 7f4cc2fab640 1 -- 172.21.15.116:0/216843206 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cbc1a4b40 con 0x7f4cbc102570 2024-09-12T03:05:47.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.430+0000 7f4cc2fab640 1 -- 172.21.15.116:0/216843206 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cbc1a5050 con 0x7f4cbc102570 2024-09-12T03:05:47.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.432+0000 7f4cb9ffb640 1 -- 172.21.15.116:0/216843206 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4ca4010ac0 con 0x7f4cbc102570 2024-09-12T03:05:47.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.433+0000 7f4cc5384740 1 Processor -- start 2024-09-12T03:05:47.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.433+0000 7f4cc5384740 1 -- start start 2024-09-12T03:05:47.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.433+0000 7f4cb9ffb640 1 --2- 172.21.15.116:0/216843206 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4ca80a2b80 0x7f4ca80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.433+0000 7f4cb9ffb640 1 -- 172.21.15.116:0/216843206 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f4ca40cf410 con 0x7f4cbc102570 2024-09-12T03:05:47.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.433+0000 7f4cc5384740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55612edbb360 0x55612edbb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.433+0000 7f4cc1548640 1 --2- 172.21.15.116:0/216843206 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4ca80a2b80 0x7f4ca80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.433+0000 7f4cc5384740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55612edbbc80 0x55612edca550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.433+0000 7f4cc5384740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55612edcaa90 0x55612edcce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.433+0000 7f4cc5384740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55612ed212e0 con 0x55612edcaa90 2024-09-12T03:05:47.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.433+0000 7f4cc5384740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55612ecc8ce0 con 0x55612edbbc80 2024-09-12T03:05:47.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.433+0000 7f4cc5384740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55612ed1d2a0 con 0x55612edbb360 2024-09-12T03:05:47.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.434+0000 7f4cc1548640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55612edcaa90 0x55612edcce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.434+0000 7f4cc1548640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55612edcaa90 0x55612edcce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40724/0 (socket says 172.21.15.116:40724) 2024-09-12T03:05:47.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.434+0000 7f4cc1548640 1 -- 172.21.15.116:0/3592425103 learned_addr learned my addr 172.21.15.116:0/3592425103 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:47.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.434+0000 7f4cbbfff640 1 --2- 172.21.15.116:0/3592425103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55612edbbc80 0x55612edca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.434+0000 7f4cc0d47640 1 --2- 172.21.15.116:0/3592425103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55612edbb360 0x55612edbb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.434+0000 7f4cc1548640 1 --2- 172.21.15.116:0/216843206 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4ca80a2b80 0x7f4ca80a5040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400ad40 tx=0x7f4cb4013650 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:47.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.434+0000 7f4cc1548640 1 -- 172.21.15.116:0/3592425103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55612edbb360 msgr2=0x55612edbb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:47.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.434+0000 7f4cc1548640 1 --2- 172.21.15.116:0/3592425103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55612edbb360 0x55612edbb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.434+0000 7f4cc1548640 1 -- 172.21.15.116:0/3592425103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55612edbbc80 msgr2=0x55612edca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:47.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.434+0000 7f4cc1548640 1 --2- 172.21.15.116:0/3592425103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55612edbbc80 0x55612edca550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.434+0000 7f4cc1548640 1 -- 172.21.15.116:0/3592425103 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55612ecad440 con 0x55612edcaa90 2024-09-12T03:05:47.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.434+0000 7f4cc1548640 1 --2- 172.21.15.116:0/3592425103 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55612edcaa90 0x55612edcce80 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4013b50 tx=0x7f4cb401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f438ea1fb867f53 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:47.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.434+0000 7f4cb2ffd640 1 -- 172.21.15.116:0/3592425103 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4026070 con 0x55612edcaa90 2024-09-12T03:05:47.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.434+0000 7f4cb2ffd640 1 -- 172.21.15.116:0/3592425103 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cb4021b40 con 0x55612edcaa90 2024-09-12T03:05:47.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.434+0000 7f4cb2ffd640 1 -- 172.21.15.116:0/3592425103 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb400f040 con 0x55612edcaa90 2024-09-12T03:05:47.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.435+0000 7f4cc5384740 1 -- 172.21.15.116:0/3592425103 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55612edcaa90 msgr2=0x55612edcce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:47.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.435+0000 7f4cc5384740 1 --2- 172.21.15.116:0/3592425103 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55612edcaa90 0x55612edcce80 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4013b50 tx=0x7f4cb401b000 comp rx=0 tx=0).stop 2024-09-12T03:05:47.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.435+0000 7f4cc5384740 1 -- 172.21.15.116:0/3592425103 shutdown_connections 2024-09-12T03:05:47.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.435+0000 7f4cc5384740 1 --2- 172.21.15.116:0/3592425103 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55612edcaa90 0x55612edcce80 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.435+0000 7f4cc5384740 1 --2- 172.21.15.116:0/3592425103 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55612edbbc80 0x55612edca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.435+0000 7f4cc5384740 1 --2- 172.21.15.116:0/3592425103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55612edbb360 0x55612edbb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.435+0000 7f4cc5384740 1 -- 172.21.15.116:0/3592425103 >> 172.21.15.116:0/3592425103 conn(0x55612edb7a30 msgr2=0x55612edb7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:47.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.435+0000 7f4cc5384740 1 -- 172.21.15.116:0/3592425103 shutdown_connections 2024-09-12T03:05:47.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.435+0000 7f4cc5384740 1 -- 172.21.15.116:0/3592425103 wait complete. 2024-09-12T03:05:47.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.435+0000 7f4cc5384740 1 -- start start 2024-09-12T03:05:47.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.436+0000 7f4cc5384740 1 Processor -- start 2024-09-12T03:05:47.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.436+0000 7f4cc5384740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55612edbb360 0x55612edca440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.436+0000 7f4cc5384740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55612edbbc80 0x55612edc76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.436+0000 7f4cc5384740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55612edcaa90 0x55612edc7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.436+0000 7f4cc5384740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55612edcd600 con 0x55612edbb360 2024-09-12T03:05:47.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.436+0000 7f4cc5384740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55612ecdc050 con 0x55612edbbc80 2024-09-12T03:05:47.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.436+0000 7f4cc5384740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55612edcd780 con 0x55612edcaa90 2024-09-12T03:05:47.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.436+0000 7f4cc5384740 10 client.0 _ll_register_callbacks cb 0x55612ed1c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:47.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.436+0000 7f4cc0d47640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55612edbb360 0x55612edca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.436+0000 7f4cc0d47640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55612edbb360 0x55612edca440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40740/0 (socket says 172.21.15.116:40740) 2024-09-12T03:05:47.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.436+0000 7f4cc0d47640 1 -- 172.21.15.116:0/267583275 learned_addr learned my addr 172.21.15.116:0/267583275 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:47.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.436+0000 7f4cc1548640 1 --2- 172.21.15.116:0/267583275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55612edcaa90 0x55612edc7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.436+0000 7f4cbbfff640 1 --2- 172.21.15.116:0/267583275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55612edbbc80 0x55612edc76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.436+0000 7f4cc0d47640 1 -- 172.21.15.116:0/267583275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55612edcaa90 msgr2=0x55612edc7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:47.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.436+0000 7f4cc0d47640 1 --2- 172.21.15.116:0/267583275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55612edcaa90 0x55612edc7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.436+0000 7f4cc0d47640 1 -- 172.21.15.116:0/267583275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55612edbbc80 msgr2=0x55612edc76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:47.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.436+0000 7f4cc0d47640 1 --2- 172.21.15.116:0/267583275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55612edbbc80 0x55612edc76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.436+0000 7f4cc0d47640 1 -- 172.21.15.116:0/267583275 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55612ea923c0 con 0x55612edbb360 2024-09-12T03:05:47.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.436+0000 7f4cbbfff640 1 --2- 172.21.15.116:0/267583275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55612edbbc80 0x55612edc76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:47.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.437+0000 7f4cc0d47640 1 --2- 172.21.15.116:0/267583275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55612edbb360 0x55612edca440 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4cac006010 tx=0x7f4cac02f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37de899a86be92c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:47.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.437+0000 7f4c9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:05:47.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.437+0000 7f4c9bfff640 1 -- 172.21.15.116:0/267583275 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac049060 con 0x55612edbb360 2024-09-12T03:05:47.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.437+0000 7f4c9bfff640 1 -- 172.21.15.116:0/267583275 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f4cac003970 con 0x55612edbb360 2024-09-12T03:05:47.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.437+0000 7f4cc5384740 1 -- 172.21.15.116:0/267583275 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cb4021b40 con 0x55612edbb360 2024-09-12T03:05:47.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.437+0000 7f4c9bfff640 1 -- 172.21.15.116:0/267583275 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4cac034040 con 0x55612edbb360 2024-09-12T03:05:47.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.437+0000 7f4cc5384740 10 client.16014 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:47.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.437+0000 7f4cc5384740 1 -- 172.21.15.116:0/267583275 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55612edd9080 con 0x55612edbb360 2024-09-12T03:05:47.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.437+0000 7f4c9bfff640 1 -- 172.21.15.116:0/267583275 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cb4021b40 con 0x55612edbb360 2024-09-12T03:05:47.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.437+0000 7f4c9bfff640 1 -- 172.21.15.116:0/267583275 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4cac030560 con 0x55612edbb360 2024-09-12T03:05:47.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.437+0000 7f4cc5384740 10 client.16014 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:47.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.437+0000 7f4cc5384740 10 client.16014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:05:47.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.437+0000 7f4cc5384740 20 client.16014 populate_metadata read hostname 'smithi116' 2024-09-12T03:05:47.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.437+0000 7f4cc5384740 1 --2- 172.21.15.116:0/267583275 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55612edda540 0x55612eddc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.437+0000 7f4cc5384740 4 client.16014 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:05:47.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.437+0000 7f4cc5384740 1 -- 172.21.15.116:0/267583275 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55612ed2e850 con 0x55612edda540 2024-09-12T03:05:47.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.438+0000 7f4cbbfff640 1 --2- 172.21.15.116:0/267583275 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55612edda540 0x55612eddc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.438+0000 7f4c9bfff640 10 client.16014 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:47.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.438+0000 7f4cbbfff640 1 --2- 172.21.15.116:0/267583275 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55612edda540 0x55612eddc930 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2ebcf5a83489817 server_cookie=9fc52d4c952feeb0 in_seq=0 out_seq=0 2024-09-12T03:05:47.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.439+0000 7f4c9bfff640 1 -- 172.21.15.116:0/267583275 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55612ed2e850 con 0x55612edda540 2024-09-12T03:05:47.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.439+0000 7f4c9bfff640 10 client.16014 handle_command_reply: tid=0 2024-09-12T03:05:47.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.439+0000 7f4cc5384740 1 client.16014 shutdown 2024-09-12T03:05:47.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.439+0000 7f4cc5384740 1 -- 172.21.15.116:0/267583275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55612edbb360 msgr2=0x55612edca440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:47.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.439+0000 7f4cc5384740 1 --2- 172.21.15.116:0/267583275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55612edbb360 0x55612edca440 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4cac006010 tx=0x7f4cac02f3c0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.439+0000 7f4cc5384740 1 -- 172.21.15.116:0/267583275 shutdown_connections 2024-09-12T03:05:47.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.439+0000 7f4cc5384740 1 --2- 172.21.15.116:0/267583275 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55612edda540 0x55612eddc930 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.439+0000 7f4cc5384740 1 --2- 172.21.15.116:0/267583275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55612edcaa90 0x55612edc7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.439+0000 7f4cc5384740 1 --2- 172.21.15.116:0/267583275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55612edbbc80 0x55612edc76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.439+0000 7f4c9bfff640 0 client.16014 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:47.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.439+0000 7f4cc5384740 1 --2- 172.21.15.116:0/267583275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55612edbb360 0x55612edca440 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.439+0000 7f4cc5384740 1 -- 172.21.15.116:0/267583275 >> 172.21.15.116:0/267583275 conn(0x55612edb7a30 msgr2=0x55612edbd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:47.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.439+0000 7f4cc5384740 1 -- 172.21.15.116:0/267583275 shutdown_connections 2024-09-12T03:05:47.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.439+0000 7f4cc5384740 1 -- 172.21.15.116:0/267583275 wait complete. 2024-09-12T03:05:47.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.439+0000 7f4cc5384740 20 client.16014 trim_cache size 0 max 16384 2024-09-12T03:05:47.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.456+0000 7f4cc5384740 1 Processor -- start 2024-09-12T03:05:47.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.456+0000 7f4cc5384740 1 -- start start 2024-09-12T03:05:47.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.456+0000 7f4cc5384740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55612edbb360 0x55612edd68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.456+0000 7f4cc5384740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55612edbbc80 0x55612ede21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.456+0000 7f4cc5384740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55612edcaa90 0x55612ede2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.456+0000 7f4cc5384740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4cbc1adbc0 con 0x55612edcaa90 2024-09-12T03:05:47.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.456+0000 7f4cc5384740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4cbc1ab490 con 0x55612edbbc80 2024-09-12T03:05:47.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.456+0000 7f4cc5384740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4cbc1a9300 con 0x55612edbb360 2024-09-12T03:05:47.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.456+0000 7f4cc1548640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55612edcaa90 0x55612ede2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.456+0000 7f4cc1548640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55612edcaa90 0x55612ede2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40750/0 (socket says 172.21.15.116:40750) 2024-09-12T03:05:47.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.456+0000 7f4cc1548640 1 -- 172.21.15.116:0/2071831789 learned_addr learned my addr 172.21.15.116:0/2071831789 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:47.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.457+0000 7f4cc0d47640 1 --2- 172.21.15.116:0/2071831789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55612edbb360 0x55612edd68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.457+0000 7f4cc1548640 1 -- 172.21.15.116:0/2071831789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55612edbb360 msgr2=0x55612edd68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:47.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.457+0000 7f4cc1548640 1 --2- 172.21.15.116:0/2071831789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55612edbb360 0x55612edd68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.457+0000 7f4cbbfff640 1 --2- 172.21.15.116:0/2071831789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55612edbbc80 0x55612ede21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.457+0000 7f4cc1548640 1 -- 172.21.15.116:0/2071831789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55612edbbc80 msgr2=0x55612ede21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:47.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.457+0000 7f4cc1548640 1 --2- 172.21.15.116:0/2071831789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55612edbbc80 0x55612ede21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.457+0000 7f4cc1548640 1 -- 172.21.15.116:0/2071831789 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55612edb74d0 con 0x55612edcaa90 2024-09-12T03:05:47.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.457+0000 7f4cbbfff640 1 --2- 172.21.15.116:0/2071831789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55612edbbc80 0x55612ede21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:05:47.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.457+0000 7f4cc1548640 1 --2- 172.21.15.116:0/2071831789 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55612edcaa90 0x55612ede2720 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4004810 tx=0x7f4cb4006a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4748caccdad29454 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:47.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.457+0000 7f4cb27fc640 1 -- 172.21.15.116:0/2071831789 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4031020 con 0x55612edcaa90 2024-09-12T03:05:47.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.457+0000 7f4cb27fc640 1 -- 172.21.15.116:0/2071831789 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cb4021b40 con 0x55612edcaa90 2024-09-12T03:05:47.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.457+0000 7f4cb27fc640 1 -- 172.21.15.116:0/2071831789 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4026040 con 0x55612edcaa90 2024-09-12T03:05:47.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.458+0000 7f4cc5384740 1 -- 172.21.15.116:0/2071831789 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55612edcaa90 msgr2=0x55612ede2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:47.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.458+0000 7f4cc5384740 1 --2- 172.21.15.116:0/2071831789 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55612edcaa90 0x55612ede2720 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4004810 tx=0x7f4cb4006a70 comp rx=0 tx=0).stop 2024-09-12T03:05:47.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.458+0000 7f4cc5384740 1 -- 172.21.15.116:0/2071831789 shutdown_connections 2024-09-12T03:05:47.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.458+0000 7f4cc5384740 1 --2- 172.21.15.116:0/2071831789 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55612edcaa90 0x55612ede2720 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.458+0000 7f4cc5384740 1 --2- 172.21.15.116:0/2071831789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55612edbbc80 0x55612ede21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.458+0000 7f4cc5384740 1 --2- 172.21.15.116:0/2071831789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55612edbb360 0x55612edd68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.458+0000 7f4cc5384740 1 -- 172.21.15.116:0/2071831789 >> 172.21.15.116:0/2071831789 conn(0x55612edb7a30 msgr2=0x55612edc2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:47.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.458+0000 7f4cc5384740 1 -- 172.21.15.116:0/2071831789 shutdown_connections 2024-09-12T03:05:47.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.458+0000 7f4cc5384740 1 -- 172.21.15.116:0/2071831789 wait complete. 2024-09-12T03:05:47.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.458+0000 7f4cc5384740 1 -- start start 2024-09-12T03:05:47.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.458+0000 7f4cc5384740 1 Processor -- start 2024-09-12T03:05:47.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.459+0000 7f4cc5384740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55612edbb360 0x55612edcea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.459+0000 7f4cc5384740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55612edbbc80 0x55612edcef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.459+0000 7f4cc5384740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55612edcaa90 0x55612edcf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.459+0000 7f4cc5384740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4cbc1a9950 con 0x55612edcaa90 2024-09-12T03:05:47.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.459+0000 7f4cc5384740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4cbc1aadc0 con 0x55612edbbc80 2024-09-12T03:05:47.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.459+0000 7f4cc5384740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4cbc1aa020 con 0x55612edbb360 2024-09-12T03:05:47.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.459+0000 7f4cc5384740 10 client.0 _ll_register_callbacks cb 0x7f4cac02fcd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:47.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.459+0000 7f4cc1548640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55612edcaa90 0x55612edcf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.459+0000 7f4cc1548640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55612edcaa90 0x55612edcf490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40756/0 (socket says 172.21.15.116:40756) 2024-09-12T03:05:47.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.459+0000 7f4cc1548640 1 -- 172.21.15.116:0/2040209229 learned_addr learned my addr 172.21.15.116:0/2040209229 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:05:47.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.459+0000 7f4cc0d47640 1 --2- 172.21.15.116:0/2040209229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55612edbb360 0x55612edcea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.459+0000 7f4cbbfff640 1 --2- 172.21.15.116:0/2040209229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55612edbbc80 0x55612edcef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.459+0000 7f4cc1548640 1 -- 172.21.15.116:0/2040209229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55612edbb360 msgr2=0x55612edcea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:47.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.459+0000 7f4cc1548640 1 --2- 172.21.15.116:0/2040209229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55612edbb360 0x55612edcea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.459+0000 7f4cc1548640 1 -- 172.21.15.116:0/2040209229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55612edbbc80 msgr2=0x55612edcef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:47.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.459+0000 7f4cc1548640 1 --2- 172.21.15.116:0/2040209229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55612edbbc80 0x55612edcef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.459+0000 7f4cc1548640 1 -- 172.21.15.116:0/2040209229 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55612edcd250 con 0x55612edcaa90 2024-09-12T03:05:47.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.459+0000 7f4cc0d47640 1 --2- 172.21.15.116:0/2040209229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55612edbb360 0x55612edcea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:47.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.459+0000 7f4cc1548640 1 --2- 172.21.15.116:0/2040209229 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55612edcaa90 0x55612edcf490 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4006ae0 tx=0x7f4cb402cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540507ef5c853683 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:47.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.459+0000 7f4c9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:05:47.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.460+0000 7f4c9bfff640 1 -- 172.21.15.116:0/2040209229 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4031030 con 0x55612edcaa90 2024-09-12T03:05:47.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.460+0000 7f4cc5384740 1 -- 172.21.15.116:0/2040209229 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cb4021b40 con 0x55612edcaa90 2024-09-12T03:05:47.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.460+0000 7f4c9bfff640 1 -- 172.21.15.116:0/2040209229 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f4cb402c880 con 0x55612edcaa90 2024-09-12T03:05:47.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.460+0000 7f4c9bfff640 1 -- 172.21.15.116:0/2040209229 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4cb4047050 con 0x55612edcaa90 2024-09-12T03:05:47.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.460+0000 7f4cc5384740 10 client.16020 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:47.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.460+0000 7f4cc5384740 1 -- 172.21.15.116:0/2040209229 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55612ee454b0 con 0x55612edcaa90 2024-09-12T03:05:47.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.460+0000 7f4c9bfff640 1 -- 172.21.15.116:0/2040209229 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cb4021b40 con 0x55612edcaa90 2024-09-12T03:05:47.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.460+0000 7f4c9bfff640 1 -- 172.21.15.116:0/2040209229 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4cb4015040 con 0x55612edcaa90 2024-09-12T03:05:47.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.460+0000 7f4cc5384740 10 client.16020 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:47.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.460+0000 7f4cc5384740 10 client.16020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:05:47.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.460+0000 7f4cc5384740 20 client.16020 populate_metadata read hostname 'smithi116' 2024-09-12T03:05:47.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.460+0000 7f4cc5384740 1 --2- 172.21.15.116:0/2040209229 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55612ee459e0 0x55612ee47dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.460+0000 7f4cc5384740 4 client.16020 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:05:47.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.460+0000 7f4cc5384740 1 -- 172.21.15.116:0/2040209229 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55612ed28d60 con 0x55612ee459e0 2024-09-12T03:05:47.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.460+0000 7f4cc0d47640 1 --2- 172.21.15.116:0/2040209229 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55612ee459e0 0x55612ee47dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.461+0000 7f4c9bfff640 10 client.16020 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:47.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.461+0000 7f4cc0d47640 1 --2- 172.21.15.116:0/2040209229 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55612ee459e0 0x55612ee47dd0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fca9fcb5057780a0 server_cookie=75da24dcb1ed8004 in_seq=0 out_seq=0 2024-09-12T03:05:47.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.461+0000 7f4c9bfff640 1 -- 172.21.15.116:0/2040209229 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55612ed28d60 con 0x55612ee459e0 2024-09-12T03:05:47.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.461+0000 7f4c9bfff640 10 client.16020 handle_command_reply: tid=0 2024-09-12T03:05:47.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.461+0000 7f4cc5384740 1 client.16020 shutdown 2024-09-12T03:05:47.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.461+0000 7f4cc5384740 1 -- 172.21.15.116:0/2040209229 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55612edcaa90 msgr2=0x55612edcf490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:47.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.461+0000 7f4cc5384740 1 --2- 172.21.15.116:0/2040209229 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55612edcaa90 0x55612edcf490 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4006ae0 tx=0x7f4cb402cbe0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.461+0000 7f4cc5384740 1 -- 172.21.15.116:0/2040209229 shutdown_connections 2024-09-12T03:05:47.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.461+0000 7f4cc5384740 1 --2- 172.21.15.116:0/2040209229 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55612ee459e0 0x55612ee47dd0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.461+0000 7f4cc5384740 1 --2- 172.21.15.116:0/2040209229 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55612edcaa90 0x55612edcf490 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.461+0000 7f4cc5384740 1 --2- 172.21.15.116:0/2040209229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55612edbbc80 0x55612edcef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.461+0000 7f4cc5384740 1 --2- 172.21.15.116:0/2040209229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55612edbb360 0x55612edcea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.461+0000 7f4cc5384740 1 -- 172.21.15.116:0/2040209229 >> 172.21.15.116:0/2040209229 conn(0x55612edb7a30 msgr2=0x55612edb6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:47.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.461+0000 7f4c9bfff640 0 client.16020 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:05:47.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.461+0000 7f4cc5384740 1 -- 172.21.15.116:0/2040209229 shutdown_connections 2024-09-12T03:05:47.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.461+0000 7f4cc5384740 1 -- 172.21.15.116:0/2040209229 wait complete. 2024-09-12T03:05:47.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.462+0000 7f4cc5384740 20 client.16020 trim_cache size 0 max 16384 2024-09-12T03:05:47.468 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:05:47.468 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:05:47.468 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": { 2024-09-12T03:05:47.468 INFO:teuthology.orchestra.run.smithi116.stdout: "92a01a02-7630-431c-8a26-094739a23fe1": { 2024-09-12T03:05:47.468 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/", 2024-09-12T03:05:47.468 INFO:teuthology.orchestra.run.smithi116.stdout: "tag": "92a01a02-7630-431c-8a26-094739a23fe1", 2024-09-12T03:05:47.468 INFO:teuthology.orchestra.run.smithi116.stdout: "options": "recursive,force" 2024-09-12T03:05:47.468 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:05:47.468 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:05:47.469 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:05:47.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.462+0000 7f4cc2fab640 1 -- 172.21.15.116:0/216843206 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4ca80a2b80 msgr2=0x7f4ca80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:47.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.462+0000 7f4cc2fab640 1 --2- 172.21.15.116:0/216843206 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4ca80a2b80 0x7f4ca80a5040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400ad40 tx=0x7f4cb4013650 comp rx=0 tx=0).stop 2024-09-12T03:05:47.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.462+0000 7f4cc2fab640 1 -- 172.21.15.116:0/216843206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cbc102570 msgr2=0x7f4cbc1998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:47.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.462+0000 7f4cc2fab640 1 --2- 172.21.15.116:0/216843206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cbc102570 0x7f4cbc1998f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4ca400e9f0 tx=0x7f4ca400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.462+0000 7f4cc2fab640 1 -- 172.21.15.116:0/216843206 shutdown_connections 2024-09-12T03:05:47.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.462+0000 7f4cc2fab640 1 --2- 172.21.15.116:0/216843206 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4ca80a2b80 0x7f4ca80a5040 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.462+0000 7f4cc2fab640 1 --2- 172.21.15.116:0/216843206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cbc102ef0 0x7f4cbc1a4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.462+0000 7f4cc2fab640 1 --2- 172.21.15.116:0/216843206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cbc102570 0x7f4cbc1998f0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.462+0000 7f4cc2fab640 1 --2- 172.21.15.116:0/216843206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cbc100e20 0x7f4cbc1993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.462+0000 7f4cc2fab640 1 -- 172.21.15.116:0/216843206 >> 172.21.15.116:0/216843206 conn(0x7f4cbc0fc7d0 msgr2=0x7f4cbc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:47.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.462+0000 7f4cc2fab640 1 -- 172.21.15.116:0/216843206 shutdown_connections 2024-09-12T03:05:47.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:05:47.462+0000 7f4cc2fab640 1 -- 172.21.15.116:0/216843206 wait complete. 2024-09-12T03:05:47.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:47 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:05:47.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:47 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:05:47.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:47 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:05:47.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:47 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:05:47.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:47 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:05:47.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:47 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:05:47.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:47 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:05:47.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:47 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:05:48.347 INFO:tasks.cephfs.filesystem:scrub status for tag:92a01a02-7630-431c-8a26-094739a23fe1 - {'path': '/', 'tag': '92a01a02-7630-431c-8a26-094739a23fe1', 'options': 'recursive,force'} 2024-09-12T03:05:48.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:48 smithi116 ceph-mon[31410]: pgmap v422: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T03:05:48.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:48 smithi183 ceph-mon[35535]: pgmap v422: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T03:05:48.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:48 smithi177 ceph-mon[35305]: pgmap v422: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T03:05:49.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:49 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:05:49.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:49 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:05:49.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:49 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:05:50.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:50 smithi116 ceph-mon[31410]: pgmap v423: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.7 KiB/s wr, 7 op/s 2024-09-12T03:05:50.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:50 smithi183 ceph-mon[35535]: pgmap v423: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.7 KiB/s wr, 7 op/s 2024-09-12T03:05:50.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:50 smithi177 ceph-mon[35305]: pgmap v423: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.7 KiB/s wr, 7 op/s 2024-09-12T03:05:52.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:52 smithi116 ceph-mon[31410]: pgmap v424: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-09-12T03:05:52.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:52 smithi183 ceph-mon[35535]: pgmap v424: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-09-12T03:05:52.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:52 smithi177 ceph-mon[35305]: pgmap v424: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-09-12T03:05:54.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:54 smithi116 ceph-mon[31410]: pgmap v425: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:05:54.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:54 smithi183 ceph-mon[35535]: pgmap v425: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:05:54.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:54 smithi177 ceph-mon[35305]: pgmap v425: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:05:56.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:56 smithi116 ceph-mon[31410]: pgmap v426: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:05:56.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:56 smithi183 ceph-mon[35535]: pgmap v426: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:05:56.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:56 smithi177 ceph-mon[35305]: pgmap v426: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:05:58.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:05:58 smithi116 ceph-mon[31410]: pgmap v427: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-12T03:05:58.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:05:58 smithi183 ceph-mon[35535]: pgmap v427: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-12T03:05:58.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:05:58 smithi177 ceph-mon[35305]: pgmap v427: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-12T03:06:00.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:00 smithi116 ceph-mon[31410]: pgmap v428: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-12T03:06:00.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:00 smithi183 ceph-mon[35535]: pgmap v428: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-12T03:06:00.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:00 smithi177 ceph-mon[35305]: pgmap v428: 97 pgs: 97 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-12T03:06:01.027 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:06:01.289 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:06:01.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.842+0000 7ff1a5be5640 1 -- 172.21.15.116:0/3543847538 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1a0102570 msgr2=0x7ff1a01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:01.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.842+0000 7ff1a5be5640 1 --2- 172.21.15.116:0/3543847538 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1a0102570 0x7ff1a01029b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff190009900 tx=0x7ff19002a480 comp rx=0 tx=0).stop 2024-09-12T03:06:01.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.842+0000 7ff1a5be5640 1 -- 172.21.15.116:0/3543847538 shutdown_connections 2024-09-12T03:06:01.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.842+0000 7ff1a5be5640 1 --2- 172.21.15.116:0/3543847538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1a0102ef0 0x7ff1a010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:01.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.842+0000 7ff1a5be5640 1 --2- 172.21.15.116:0/3543847538 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1a0102570 0x7ff1a01029b0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:01.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.842+0000 7ff1a5be5640 1 --2- 172.21.15.116:0/3543847538 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1a0100e20 0x7ff1a0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:01.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.842+0000 7ff1a5be5640 1 -- 172.21.15.116:0/3543847538 >> 172.21.15.116:0/3543847538 conn(0x7ff1a00fc7d0 msgr2=0x7ff1a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:01.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.842+0000 7ff1a5be5640 1 -- 172.21.15.116:0/3543847538 shutdown_connections 2024-09-12T03:06:01.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.842+0000 7ff1a5be5640 1 -- 172.21.15.116:0/3543847538 wait complete. 2024-09-12T03:06:01.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.843+0000 7ff1a5be5640 1 Processor -- start 2024-09-12T03:06:01.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.843+0000 7ff1a5be5640 1 -- start start 2024-09-12T03:06:01.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.843+0000 7ff1a5be5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1a0100e20 0x7ff1a019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:01.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.843+0000 7ff1a5be5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1a0102570 0x7ff1a019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:01.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.843+0000 7ff1a5be5640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1a0102ef0 0x7ff1a019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:01.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.843+0000 7ff1a5be5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a0110f80 con 0x7ff1a0100e20 2024-09-12T03:06:01.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.843+0000 7ff1a5be5640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a0110e00 con 0x7ff1a0102ef0 2024-09-12T03:06:01.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.843+0000 7ff1a5be5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a0111100 con 0x7ff1a0102570 2024-09-12T03:06:01.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.843+0000 7ff19f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1a0100e20 0x7ff1a019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:01.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.843+0000 7ff19f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1a0100e20 0x7ff1a019a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33806/0 (socket says 172.21.15.116:33806) 2024-09-12T03:06:01.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.843+0000 7ff19f7fe640 1 -- 172.21.15.116:0/915870486 learned_addr learned my addr 172.21.15.116:0/915870486 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:01.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.843+0000 7ff19ffff640 1 --2- 172.21.15.116:0/915870486 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1a0102ef0 0x7ff1a019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:01.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.843+0000 7ff19effd640 1 --2- 172.21.15.116:0/915870486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1a0102570 0x7ff1a019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:01.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.843+0000 7ff19f7fe640 1 -- 172.21.15.116:0/915870486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1a0102570 msgr2=0x7ff1a019abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:01.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.843+0000 7ff19f7fe640 1 --2- 172.21.15.116:0/915870486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1a0102570 0x7ff1a019abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:01.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.843+0000 7ff19f7fe640 1 -- 172.21.15.116:0/915870486 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1a0102ef0 msgr2=0x7ff1a019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:01.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.843+0000 7ff19f7fe640 1 --2- 172.21.15.116:0/915870486 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1a0102ef0 0x7ff1a019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:01.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.843+0000 7ff19f7fe640 1 -- 172.21.15.116:0/915870486 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a019f660 con 0x7ff1a0100e20 2024-09-12T03:06:01.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.844+0000 7ff19f7fe640 1 --2- 172.21.15.116:0/915870486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1a0100e20 0x7ff1a019a6b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff19400d950 tx=0x7ff19400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:01.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.844+0000 7ff19cff9640 1 -- 172.21.15.116:0/915870486 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff194017030 con 0x7ff1a0100e20 2024-09-12T03:06:01.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.844+0000 7ff19cff9640 1 -- 172.21.15.116:0/915870486 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff194004020 con 0x7ff1a0100e20 2024-09-12T03:06:01.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.844+0000 7ff1a5be5640 1 -- 172.21.15.116:0/915870486 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1a019f950 con 0x7ff1a0100e20 2024-09-12T03:06:01.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.844+0000 7ff19cff9640 1 -- 172.21.15.116:0/915870486 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1940047a0 con 0x7ff1a0100e20 2024-09-12T03:06:01.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.844+0000 7ff1a5be5640 1 -- 172.21.15.116:0/915870486 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1a01a7230 con 0x7ff1a0100e20 2024-09-12T03:06:01.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.845+0000 7ff1a5be5640 1 -- 172.21.15.116:0/915870486 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1a0101200 con 0x7ff1a0100e20 2024-09-12T03:06:01.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.846+0000 7ff19cff9640 1 -- 172.21.15.116:0/915870486 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ff1940041d0 con 0x7ff1a0100e20 2024-09-12T03:06:01.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.847+0000 7ff19cff9640 1 --2- 172.21.15.116:0/915870486 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff1880a2b80 0x7ff1880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:01.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.847+0000 7ff19cff9640 1 -- 172.21.15.116:0/915870486 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff1940cf1c0 con 0x7ff1a0100e20 2024-09-12T03:06:01.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.847+0000 7ff19effd640 1 --2- 172.21.15.116:0/915870486 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff1880a2b80 0x7ff1880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:01.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.847+0000 7ff19effd640 1 --2- 172.21.15.116:0/915870486 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff1880a2b80 0x7ff1880a5040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff1a010ddf0 tx=0x7ff190005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:01.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:01.850+0000 7ff19cff9640 1 -- 172.21.15.116:0/915870486 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff19409c150 con 0x7ff1a0100e20 2024-09-12T03:06:02.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:02.049+0000 7ff1a5be5640 1 -- 172.21.15.116:0/915870486 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1a019ba10 con 0x7ff1a0100e20 2024-09-12T03:06:02.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:02.049+0000 7ff19cff9640 1 -- 172.21.15.116:0/915870486 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7ff1940ce090 con 0x7ff1a0100e20 2024-09-12T03:06:02.050 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:06:02.050 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:06:02.050 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:06:02.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:02.052+0000 7ff1a5be5640 1 -- 172.21.15.116:0/915870486 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff1880a2b80 msgr2=0x7ff1880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:02.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:02.052+0000 7ff1a5be5640 1 --2- 172.21.15.116:0/915870486 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff1880a2b80 0x7ff1880a5040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff1a010ddf0 tx=0x7ff190005a20 comp rx=0 tx=0).stop 2024-09-12T03:06:02.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:02.052+0000 7ff1a5be5640 1 -- 172.21.15.116:0/915870486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1a0100e20 msgr2=0x7ff1a019a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:02.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:02.052+0000 7ff1a5be5640 1 --2- 172.21.15.116:0/915870486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1a0100e20 0x7ff1a019a6b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff19400d950 tx=0x7ff19400de10 comp rx=0 tx=0).stop 2024-09-12T03:06:02.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:02.052+0000 7ff1a5be5640 1 -- 172.21.15.116:0/915870486 shutdown_connections 2024-09-12T03:06:02.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:02.052+0000 7ff1a5be5640 1 --2- 172.21.15.116:0/915870486 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff1880a2b80 0x7ff1880a5040 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:02.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:02.052+0000 7ff1a5be5640 1 --2- 172.21.15.116:0/915870486 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1a0102ef0 0x7ff1a019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:02.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:02.052+0000 7ff1a5be5640 1 --2- 172.21.15.116:0/915870486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1a0102570 0x7ff1a019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:02.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:02.052+0000 7ff1a5be5640 1 --2- 172.21.15.116:0/915870486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1a0100e20 0x7ff1a019a6b0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:02.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:02.052+0000 7ff1a5be5640 1 -- 172.21.15.116:0/915870486 >> 172.21.15.116:0/915870486 conn(0x7ff1a00fc7d0 msgr2=0x7ff1a006a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:02.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:02.052+0000 7ff1a5be5640 1 -- 172.21.15.116:0/915870486 shutdown_connections 2024-09-12T03:06:02.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:02.052+0000 7ff1a5be5640 1 -- 172.21.15.116:0/915870486 wait complete. 2024-09-12T03:06:02.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:02 smithi116 ceph-mon[31410]: pgmap v429: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-12T03:06:02.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:02 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/915870486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:06:02.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:02 smithi183 ceph-mon[35535]: pgmap v429: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-12T03:06:02.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:02 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/915870486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:06:02.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:02 smithi177 ceph-mon[35305]: pgmap v429: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-09-12T03:06:02.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:02 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/915870486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:06:02.865 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id ff1640ad --release --await' 2024-09-12T03:06:02.865 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id ff1640ad --release --await 2024-09-12T03:06:03.111 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:06:03.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.608+0000 7fb770435640 1 -- 172.21.15.116:0/1977129295 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb768102780 msgr2=0x7fb7681067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:03.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.608+0000 7fb770435640 1 --2- 172.21.15.116:0/1977129295 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb768102780 0x7fb7681067d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb758009900 tx=0x7fb75802a410 comp rx=0 tx=0).stop 2024-09-12T03:06:03.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.608+0000 7fb770435640 1 -- 172.21.15.116:0/1977129295 shutdown_connections 2024-09-12T03:06:03.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.608+0000 7fb770435640 1 --2- 172.21.15.116:0/1977129295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb768106f00 0x7fb76810ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.608+0000 7fb770435640 1 --2- 172.21.15.116:0/1977129295 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb768102780 0x7fb7681067d0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.608+0000 7fb770435640 1 --2- 172.21.15.116:0/1977129295 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb768101dd0 0x7fb7681021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.608+0000 7fb770435640 1 -- 172.21.15.116:0/1977129295 >> 172.21.15.116:0/1977129295 conn(0x7fb7680fd8b0 msgr2=0x7fb7680ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:03.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.608+0000 7fb770435640 1 -- 172.21.15.116:0/1977129295 shutdown_connections 2024-09-12T03:06:03.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.608+0000 7fb770435640 1 -- 172.21.15.116:0/1977129295 wait complete. 2024-09-12T03:06:03.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.609+0000 7fb770435640 1 Processor -- start 2024-09-12T03:06:03.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.609+0000 7fb770435640 1 -- start start 2024-09-12T03:06:03.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.609+0000 7fb770435640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb768101dd0 0x7fb768198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:03.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.609+0000 7fb770435640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb768102780 0x7fb768198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:03.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.609+0000 7fb770435640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb768106f00 0x7fb76819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:03.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.609+0000 7fb770435640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb768111f60 con 0x7fb768102780 2024-09-12T03:06:03.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.609+0000 7fb770435640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb768111de0 con 0x7fb768101dd0 2024-09-12T03:06:03.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.609+0000 7fb770435640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb7681120e0 con 0x7fb768106f00 2024-09-12T03:06:03.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.609+0000 7fb76d9d0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb768102780 0x7fb768198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:03.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.609+0000 7fb76d9d0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb768102780 0x7fb768198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33818/0 (socket says 172.21.15.116:33818) 2024-09-12T03:06:03.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.609+0000 7fb76d9d0640 1 -- 172.21.15.116:0/1607517519 learned_addr learned my addr 172.21.15.116:0/1607517519 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:03.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.609+0000 7fb76e1d1640 1 --2- 172.21.15.116:0/1607517519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb768101dd0 0x7fb768198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:03.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.609+0000 7fb76e9d2640 1 --2- 172.21.15.116:0/1607517519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb768106f00 0x7fb76819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:03.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.609+0000 7fb76d9d0640 1 -- 172.21.15.116:0/1607517519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb768106f00 msgr2=0x7fb76819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:03.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.609+0000 7fb76d9d0640 1 --2- 172.21.15.116:0/1607517519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb768106f00 0x7fb76819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.609+0000 7fb76d9d0640 1 -- 172.21.15.116:0/1607517519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb768101dd0 msgr2=0x7fb768198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:03.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.609+0000 7fb76d9d0640 1 --2- 172.21.15.116:0/1607517519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb768101dd0 0x7fb768198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.609+0000 7fb76d9d0640 1 -- 172.21.15.116:0/1607517519 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76819d3d0 con 0x7fb768102780 2024-09-12T03:06:03.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.609+0000 7fb76d9d0640 1 --2- 172.21.15.116:0/1607517519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb768102780 0x7fb768198960 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb758006010 tx=0x7fb7580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:03.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.610+0000 7fb75f7fe640 1 -- 172.21.15.116:0/1607517519 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75803f030 con 0x7fb768102780 2024-09-12T03:06:03.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.610+0000 7fb770435640 1 -- 172.21.15.116:0/1607517519 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb76819d660 con 0x7fb768102780 2024-09-12T03:06:03.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.610+0000 7fb75f7fe640 1 -- 172.21.15.116:0/1607517519 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb758003ce0 con 0x7fb768102780 2024-09-12T03:06:03.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.610+0000 7fb75f7fe640 1 -- 172.21.15.116:0/1607517519 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb758035070 con 0x7fb768102780 2024-09-12T03:06:03.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.610+0000 7fb770435640 1 -- 172.21.15.116:0/1607517519 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7681a4fa0 con 0x7fb768102780 2024-09-12T03:06:03.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.612+0000 7fb75f7fe640 1 -- 172.21.15.116:0/1607517519 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb758042030 con 0x7fb768102780 2024-09-12T03:06:03.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.612+0000 7fb77280e740 1 Processor -- start 2024-09-12T03:06:03.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.612+0000 7fb77280e740 1 -- start start 2024-09-12T03:06:03.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.613+0000 7fb75f7fe640 1 --2- 172.21.15.116:0/1607517519 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb7540a2b80 0x7fb7540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:03.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.613+0000 7fb77280e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572b2752f40 0x5572b2753320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:03.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.613+0000 7fb77280e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572b2753860 0x5572b2762110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:03.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.613+0000 7fb77280e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572b2762650 0x5572b2764a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:03.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.613+0000 7fb77280e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5572b26b8620 con 0x5572b2753860 2024-09-12T03:06:03.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.613+0000 7fb77280e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5572b265fe50 con 0x5572b2762650 2024-09-12T03:06:03.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.613+0000 7fb77280e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5572b26b45e0 con 0x5572b2752f40 2024-09-12T03:06:03.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.613+0000 7fb76d9d0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572b2753860 0x5572b2762110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:03.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.613+0000 7fb76d9d0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572b2753860 0x5572b2762110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33820/0 (socket says 172.21.15.116:33820) 2024-09-12T03:06:03.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.613+0000 7fb76d9d0640 1 -- 172.21.15.116:0/3474155082 learned_addr learned my addr 172.21.15.116:0/3474155082 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:03.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.613+0000 7fb76e9d2640 1 --2- 172.21.15.116:0/1607517519 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb7540a2b80 0x7fb7540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:03.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.613+0000 7fb76e1d1640 1 --2- 172.21.15.116:0/3474155082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572b2752f40 0x5572b2753320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:03.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.613+0000 7fb75f7fe640 1 -- 172.21.15.116:0/1607517519 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fb7580ef1e0 con 0x7fb768102780 2024-09-12T03:06:03.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.613+0000 7fb76e9d2640 1 --2- 172.21.15.116:0/3474155082 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572b2762650 0x5572b2764a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:03.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.613+0000 7fb76d9d0640 1 -- 172.21.15.116:0/3474155082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572b2752f40 msgr2=0x5572b2753320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:03.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.613+0000 7fb76d9d0640 1 --2- 172.21.15.116:0/3474155082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572b2752f40 0x5572b2753320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.613+0000 7fb76d9d0640 1 -- 172.21.15.116:0/3474155082 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572b2762650 msgr2=0x5572b2764a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:03.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.613+0000 7fb76d9d0640 1 --2- 172.21.15.116:0/3474155082 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572b2762650 0x5572b2764a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.613+0000 7fb76d9d0640 1 -- 172.21.15.116:0/3474155082 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572b2650d40 con 0x5572b2753860 2024-09-12T03:06:03.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.613+0000 7fb76d9d0640 1 --2- 172.21.15.116:0/3474155082 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572b2753860 0x5572b2762110 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb7580b7f90 tx=0x7fb758005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f574aa6ba863dd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:03.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.613+0000 7fb76e9d2640 1 --2- 172.21.15.116:0/1607517519 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb7540a2b80 0x7fb7540a5040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb760005c40 tx=0x7fb760014a50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:03.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.614+0000 7fb75cdea640 1 -- 172.21.15.116:0/3474155082 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb758049030 con 0x5572b2753860 2024-09-12T03:06:03.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.614+0000 7fb75cdea640 1 -- 172.21.15.116:0/3474155082 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb758039710 con 0x5572b2753860 2024-09-12T03:06:03.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.614+0000 7fb75cdea640 1 -- 172.21.15.116:0/3474155082 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7580bbb30 con 0x5572b2753860 2024-09-12T03:06:03.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.614+0000 7fb77280e740 1 -- 172.21.15.116:0/3474155082 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572b2753860 msgr2=0x5572b2762110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:03.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.614+0000 7fb77280e740 1 --2- 172.21.15.116:0/3474155082 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572b2753860 0x5572b2762110 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb7580b7f90 tx=0x7fb758005c00 comp rx=0 tx=0).stop 2024-09-12T03:06:03.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.614+0000 7fb77280e740 1 -- 172.21.15.116:0/3474155082 shutdown_connections 2024-09-12T03:06:03.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.614+0000 7fb77280e740 1 --2- 172.21.15.116:0/3474155082 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572b2762650 0x5572b2764a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.614+0000 7fb77280e740 1 --2- 172.21.15.116:0/3474155082 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572b2753860 0x5572b2762110 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.614+0000 7fb77280e740 1 --2- 172.21.15.116:0/3474155082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572b2752f40 0x5572b2753320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.614+0000 7fb77280e740 1 -- 172.21.15.116:0/3474155082 >> 172.21.15.116:0/3474155082 conn(0x5572b274f5f0 msgr2=0x5572b274f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:03.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.614+0000 7fb77280e740 1 -- 172.21.15.116:0/3474155082 shutdown_connections 2024-09-12T03:06:03.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.614+0000 7fb77280e740 1 -- 172.21.15.116:0/3474155082 wait complete. 2024-09-12T03:06:03.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.615+0000 7fb77280e740 1 -- start start 2024-09-12T03:06:03.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.615+0000 7fb77280e740 1 Processor -- start 2024-09-12T03:06:03.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.615+0000 7fb77280e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572b2752f40 0x5572b275f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:03.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.615+0000 7fb77280e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572b2753860 0x5572b275fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:03.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.615+0000 7fb77280e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572b2762650 0x5572b2760090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:03.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.615+0000 7fb77280e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5572b2764fe0 con 0x5572b2762650 2024-09-12T03:06:03.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.615+0000 7fb77280e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5572b2640f10 con 0x5572b2752f40 2024-09-12T03:06:03.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.615+0000 7fb77280e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5572b2765160 con 0x5572b2753860 2024-09-12T03:06:03.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.615+0000 7fb77280e740 10 client.0 _ll_register_callbacks cb 0x5572b2664db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:03.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.615+0000 7fb76e9d2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572b2762650 0x5572b2760090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:03.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.615+0000 7fb76e9d2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572b2762650 0x5572b2760090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33832/0 (socket says 172.21.15.116:33832) 2024-09-12T03:06:03.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.615+0000 7fb76e9d2640 1 -- 172.21.15.116:0/2244720653 learned_addr learned my addr 172.21.15.116:0/2244720653 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:03.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.615+0000 7fb76d9d0640 1 --2- 172.21.15.116:0/2244720653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572b2753860 0x5572b275fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:03.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.616+0000 7fb76e1d1640 1 --2- 172.21.15.116:0/2244720653 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572b2752f40 0x5572b275f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:03.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.616+0000 7fb76e9d2640 1 -- 172.21.15.116:0/2244720653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572b2753860 msgr2=0x5572b275fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:03.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.616+0000 7fb76e9d2640 1 --2- 172.21.15.116:0/2244720653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572b2753860 0x5572b275fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.616+0000 7fb76e9d2640 1 -- 172.21.15.116:0/2244720653 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572b2752f40 msgr2=0x5572b275f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:03.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.616+0000 7fb76e9d2640 1 --2- 172.21.15.116:0/2244720653 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572b2752f40 0x5572b275f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.616+0000 7fb76e9d2640 1 -- 172.21.15.116:0/2244720653 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572b23c7580 con 0x5572b2762650 2024-09-12T03:06:03.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.616+0000 7fb76e1d1640 1 --2- 172.21.15.116:0/2244720653 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572b2752f40 0x5572b275f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:06:03.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.616+0000 7fb76d9d0640 1 --2- 172.21.15.116:0/2244720653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572b2753860 0x5572b275fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:06:03.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.616+0000 7fb76e9d2640 1 --2- 172.21.15.116:0/2244720653 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572b2762650 0x5572b2760090 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb760012bd0 tx=0x7fb76001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f6e5fe1b133478 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:03.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.616+0000 7fb751ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:06:03.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.616+0000 7fb751ffb640 1 -- 172.21.15.116:0/2244720653 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb760026070 con 0x5572b2762650 2024-09-12T03:06:03.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.616+0000 7fb751ffb640 1 -- 172.21.15.116:0/2244720653 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fb760004b20 con 0x5572b2762650 2024-09-12T03:06:03.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.616+0000 7fb77280e740 1 -- 172.21.15.116:0/2244720653 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb758039710 con 0x5572b2762650 2024-09-12T03:06:03.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.616+0000 7fb751ffb640 1 -- 172.21.15.116:0/2244720653 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb760022430 con 0x5572b2762650 2024-09-12T03:06:03.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.616+0000 7fb77280e740 10 client.16038 fetch_fsmap learned FSMap version 25 2024-09-12T03:06:03.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.616+0000 7fb77280e740 1 -- 172.21.15.116:0/2244720653 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5572b2491960 con 0x5572b2762650 2024-09-12T03:06:03.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.616+0000 7fb751ffb640 1 -- 172.21.15.116:0/2244720653 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb758039710 con 0x5572b2762650 2024-09-12T03:06:03.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.616+0000 7fb751ffb640 1 -- 172.21.15.116:0/2244720653 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb760020630 con 0x5572b2762650 2024-09-12T03:06:03.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.617+0000 7fb77280e740 10 client.16038 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:06:03.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.617+0000 7fb77280e740 10 client.16038 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:06:03.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.617+0000 7fb77280e740 20 client.16038 populate_metadata read hostname 'smithi116' 2024-09-12T03:06:03.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.617+0000 7fb77280e740 1 --2- 172.21.15.116:0/2244720653 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5572b2771f10 0x5572b2774390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:03.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.617+0000 7fb77280e740 4 client.16038 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:06:03.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.617+0000 7fb77280e740 1 -- 172.21.15.116:0/2244720653 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572b26c5b90 con 0x5572b2771f10 2024-09-12T03:06:03.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.617+0000 7fb76e1d1640 1 --2- 172.21.15.116:0/2244720653 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5572b2771f10 0x5572b2774390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:03.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.617+0000 7fb76e1d1640 1 --2- 172.21.15.116:0/2244720653 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5572b2771f10 0x5572b2774390 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab59b56d6fc8371b server_cookie=5f51e8cc77cf58fc in_seq=0 out_seq=0 2024-09-12T03:06:03.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.617+0000 7fb751ffb640 10 client.16038 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:03.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.618+0000 7fb751ffb640 1 -- 172.21.15.116:0/2244720653 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5572b26c5b90 con 0x5572b2771f10 2024-09-12T03:06:03.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.618+0000 7fb751ffb640 10 client.16038 handle_command_reply: tid=0 2024-09-12T03:06:03.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.618+0000 7fb77280e740 1 client.16038 shutdown 2024-09-12T03:06:03.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.618+0000 7fb77280e740 1 -- 172.21.15.116:0/2244720653 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572b2762650 msgr2=0x5572b2760090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:03.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.618+0000 7fb77280e740 1 --2- 172.21.15.116:0/2244720653 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572b2762650 0x5572b2760090 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb760012bd0 tx=0x7fb76001a000 comp rx=0 tx=0).stop 2024-09-12T03:06:03.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.618+0000 7fb77280e740 1 -- 172.21.15.116:0/2244720653 shutdown_connections 2024-09-12T03:06:03.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.618+0000 7fb77280e740 1 --2- 172.21.15.116:0/2244720653 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5572b2771f10 0x5572b2774390 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.618+0000 7fb751ffb640 0 client.16038 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:03.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.618+0000 7fb77280e740 1 --2- 172.21.15.116:0/2244720653 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572b2762650 0x5572b2760090 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.618+0000 7fb77280e740 1 --2- 172.21.15.116:0/2244720653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572b2753860 0x5572b275fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.618+0000 7fb77280e740 1 --2- 172.21.15.116:0/2244720653 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572b2752f40 0x5572b275f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.618+0000 7fb77280e740 1 -- 172.21.15.116:0/2244720653 >> 172.21.15.116:0/2244720653 conn(0x5572b274f5f0 msgr2=0x5572b2755350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:03.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.618+0000 7fb77280e740 1 -- 172.21.15.116:0/2244720653 shutdown_connections 2024-09-12T03:06:03.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.618+0000 7fb77280e740 1 -- 172.21.15.116:0/2244720653 wait complete. 2024-09-12T03:06:03.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.618+0000 7fb77280e740 20 client.16038 trim_cache size 0 max 16384 2024-09-12T03:06:03.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.636+0000 7fb77280e740 1 Processor -- start 2024-09-12T03:06:03.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.636+0000 7fb77280e740 1 -- start start 2024-09-12T03:06:03.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.636+0000 7fb77280e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572b2752f40 0x5572b274a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:03.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.636+0000 7fb77280e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572b2753860 0x5572b2761580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:03.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.636+0000 7fb77280e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572b2762650 0x5572b2779cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:03.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.636+0000 7fb77280e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb7681adb80 con 0x5572b2762650 2024-09-12T03:06:03.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.636+0000 7fb77280e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb7681ab450 con 0x5572b2753860 2024-09-12T03:06:03.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.636+0000 7fb77280e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb7681a92a0 con 0x5572b2752f40 2024-09-12T03:06:03.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.636+0000 7fb76e9d2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572b2762650 0x5572b2779cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:03.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.636+0000 7fb76e9d2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572b2762650 0x5572b2779cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33844/0 (socket says 172.21.15.116:33844) 2024-09-12T03:06:03.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.636+0000 7fb76e9d2640 1 -- 172.21.15.116:0/387935433 learned_addr learned my addr 172.21.15.116:0/387935433 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:03.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.636+0000 7fb76e1d1640 1 --2- 172.21.15.116:0/387935433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572b2752f40 0x5572b274a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:03.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.636+0000 7fb76d9d0640 1 --2- 172.21.15.116:0/387935433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572b2753860 0x5572b2761580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:03.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.636+0000 7fb76e9d2640 1 -- 172.21.15.116:0/387935433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572b2752f40 msgr2=0x5572b274a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:03.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.636+0000 7fb76e9d2640 1 --2- 172.21.15.116:0/387935433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572b2752f40 0x5572b274a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.636+0000 7fb76e9d2640 1 -- 172.21.15.116:0/387935433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572b2753860 msgr2=0x5572b2761580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:03.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.636+0000 7fb76e9d2640 1 --2- 172.21.15.116:0/387935433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572b2753860 0x5572b2761580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.636+0000 7fb76e9d2640 1 -- 172.21.15.116:0/387935433 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572b275a420 con 0x5572b2762650 2024-09-12T03:06:03.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.636+0000 7fb76e9d2640 1 --2- 172.21.15.116:0/387935433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572b2762650 0x5572b2779cb0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb760012ba0 tx=0x7fb760012fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1fb013cc4340aec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:03.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.637+0000 7fb753fff640 1 -- 172.21.15.116:0/387935433 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb760026070 con 0x5572b2762650 2024-09-12T03:06:03.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.637+0000 7fb753fff640 1 -- 172.21.15.116:0/387935433 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb760020400 con 0x5572b2762650 2024-09-12T03:06:03.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.637+0000 7fb753fff640 1 -- 172.21.15.116:0/387935433 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7600225d0 con 0x5572b2762650 2024-09-12T03:06:03.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.637+0000 7fb77280e740 1 -- 172.21.15.116:0/387935433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572b2762650 msgr2=0x5572b2779cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:03.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.637+0000 7fb77280e740 1 --2- 172.21.15.116:0/387935433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572b2762650 0x5572b2779cb0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb760012ba0 tx=0x7fb760012fd0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.637+0000 7fb77280e740 1 -- 172.21.15.116:0/387935433 shutdown_connections 2024-09-12T03:06:03.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.637+0000 7fb77280e740 1 --2- 172.21.15.116:0/387935433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572b2762650 0x5572b2779cb0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.637+0000 7fb77280e740 1 --2- 172.21.15.116:0/387935433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572b2753860 0x5572b2761580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.637+0000 7fb77280e740 1 --2- 172.21.15.116:0/387935433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572b2752f40 0x5572b274a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.637+0000 7fb77280e740 1 -- 172.21.15.116:0/387935433 >> 172.21.15.116:0/387935433 conn(0x5572b274f5f0 msgr2=0x5572b2760e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:03.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.637+0000 7fb77280e740 1 -- 172.21.15.116:0/387935433 shutdown_connections 2024-09-12T03:06:03.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.637+0000 7fb77280e740 1 -- 172.21.15.116:0/387935433 wait complete. 2024-09-12T03:06:03.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.638+0000 7fb77280e740 1 -- start start 2024-09-12T03:06:03.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.638+0000 7fb77280e740 1 Processor -- start 2024-09-12T03:06:03.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.638+0000 7fb77280e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572b2752f40 0x5572b276ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:03.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.638+0000 7fb77280e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572b2753860 0x5572b2767380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:03.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.638+0000 7fb77280e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572b2762650 0x5572b27678c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:03.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.638+0000 7fb77280e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb7681a9910 con 0x5572b2752f40 2024-09-12T03:06:03.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.638+0000 7fb77280e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb7681aad80 con 0x5572b2753860 2024-09-12T03:06:03.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.638+0000 7fb77280e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb7681a9fe0 con 0x5572b2762650 2024-09-12T03:06:03.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.638+0000 7fb77280e740 10 client.0 _ll_register_callbacks cb 0x7fb760012a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:03.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.638+0000 7fb76e1d1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572b2752f40 0x5572b276ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:03.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.638+0000 7fb76e1d1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572b2752f40 0x5572b276ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33850/0 (socket says 172.21.15.116:33850) 2024-09-12T03:06:03.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.638+0000 7fb76e1d1640 1 -- 172.21.15.116:0/187323722 learned_addr learned my addr 172.21.15.116:0/187323722 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:03.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.638+0000 7fb76e9d2640 1 --2- 172.21.15.116:0/187323722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572b2762650 0x5572b27678c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:03.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.638+0000 7fb76d9d0640 1 --2- 172.21.15.116:0/187323722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572b2753860 0x5572b2767380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:03.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.638+0000 7fb76e1d1640 1 -- 172.21.15.116:0/187323722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572b2762650 msgr2=0x5572b27678c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:03.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.638+0000 7fb76e1d1640 1 --2- 172.21.15.116:0/187323722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572b2762650 0x5572b27678c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.638+0000 7fb76e1d1640 1 -- 172.21.15.116:0/187323722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572b2753860 msgr2=0x5572b2767380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:03.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.638+0000 7fb76e1d1640 1 --2- 172.21.15.116:0/187323722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572b2753860 0x5572b2767380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.638+0000 7fb76e1d1640 1 -- 172.21.15.116:0/187323722 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572b2768110 con 0x5572b2752f40 2024-09-12T03:06:03.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.638+0000 7fb76d9d0640 1 --2- 172.21.15.116:0/187323722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572b2753860 0x5572b2767380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:06:03.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.639+0000 7fb76e1d1640 1 --2- 172.21.15.116:0/187323722 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572b2752f40 0x5572b276ab90 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb76400ba50 tx=0x7fb76400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3d65e288e3b1d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:03.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.639+0000 7fb751ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:06:03.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.639+0000 7fb751ffb640 1 -- 172.21.15.116:0/187323722 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76401d5b0 con 0x5572b2752f40 2024-09-12T03:06:03.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.639+0000 7fb77280e740 1 -- 172.21.15.116:0/187323722 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb760020400 con 0x5572b2752f40 2024-09-12T03:06:03.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.639+0000 7fb751ffb640 1 -- 172.21.15.116:0/187323722 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fb76401d750 con 0x5572b2752f40 2024-09-12T03:06:03.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.639+0000 7fb751ffb640 1 -- 172.21.15.116:0/187323722 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb76401d950 con 0x5572b2752f40 2024-09-12T03:06:03.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.639+0000 7fb77280e740 10 client.16044 fetch_fsmap learned FSMap version 25 2024-09-12T03:06:03.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.639+0000 7fb77280e740 1 -- 172.21.15.116:0/187323722 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5572b27ddd60 con 0x5572b2752f40 2024-09-12T03:06:03.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.639+0000 7fb751ffb640 1 -- 172.21.15.116:0/187323722 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb760020400 con 0x5572b2752f40 2024-09-12T03:06:03.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.639+0000 7fb751ffb640 1 -- 172.21.15.116:0/187323722 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb764024030 con 0x5572b2752f40 2024-09-12T03:06:03.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.639+0000 7fb77280e740 10 client.16044 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:06:03.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.639+0000 7fb77280e740 10 client.16044 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:06:03.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.639+0000 7fb77280e740 20 client.16044 populate_metadata read hostname 'smithi116' 2024-09-12T03:06:03.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.639+0000 7fb77280e740 1 --2- 172.21.15.116:0/187323722 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5572b2771f10 0x5572b27e0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:03.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.639+0000 7fb77280e740 4 client.16044 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ff1640ad", "release": true, "await": true}] 2024-09-12T03:06:03.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.639+0000 7fb77280e740 1 -- 172.21.15.116:0/187323722 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ff1640ad", "release": true, "await": true}) -- 0x5572b26c00a0 con 0x5572b2771f10 2024-09-12T03:06:03.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.640+0000 7fb76d9d0640 1 --2- 172.21.15.116:0/187323722 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5572b2771f10 0x5572b27e0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:03.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.640+0000 7fb76d9d0640 1 --2- 172.21.15.116:0/187323722 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5572b2771f10 0x5572b27e0470 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4c7f4e854232f75 server_cookie=b87ccb72b03172ac in_seq=0 out_seq=0 2024-09-12T03:06:03.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.640+0000 7fb751ffb640 10 client.16044 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:03.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.642+0000 7fb751ffb640 1 -- 172.21.15.116:0/187323722 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5572b26c00a0 con 0x5572b2771f10 2024-09-12T03:06:03.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.642+0000 7fb751ffb640 10 client.16044 handle_command_reply: tid=0 2024-09-12T03:06:03.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.642+0000 7fb77280e740 1 client.16044 shutdown 2024-09-12T03:06:03.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.642+0000 7fb77280e740 1 -- 172.21.15.116:0/187323722 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572b2752f40 msgr2=0x5572b276ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:03.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.642+0000 7fb77280e740 1 --2- 172.21.15.116:0/187323722 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572b2752f40 0x5572b276ab90 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb76400ba50 tx=0x7fb76400bf10 comp rx=0 tx=0).stop 2024-09-12T03:06:03.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.642+0000 7fb77280e740 1 -- 172.21.15.116:0/187323722 shutdown_connections 2024-09-12T03:06:03.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.642+0000 7fb77280e740 1 --2- 172.21.15.116:0/187323722 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5572b2771f10 0x5572b27e0470 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.642+0000 7fb751ffb640 0 client.16044 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:03.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.642+0000 7fb77280e740 1 --2- 172.21.15.116:0/187323722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572b2762650 0x5572b27678c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.642+0000 7fb77280e740 1 --2- 172.21.15.116:0/187323722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572b2753860 0x5572b2767380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.642+0000 7fb77280e740 1 --2- 172.21.15.116:0/187323722 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572b2752f40 0x5572b276ab90 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.642+0000 7fb77280e740 1 -- 172.21.15.116:0/187323722 >> 172.21.15.116:0/187323722 conn(0x5572b274f5f0 msgr2=0x5572b274e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:03.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.643+0000 7fb77280e740 1 -- 172.21.15.116:0/187323722 shutdown_connections 2024-09-12T03:06:03.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.643+0000 7fb77280e740 1 -- 172.21.15.116:0/187323722 wait complete. 2024-09-12T03:06:03.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.643+0000 7fb77280e740 20 client.16044 trim_cache size 0 max 16384 2024-09-12T03:06:03.648 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:06:03.648 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 25, 2024-09-12T03:06:03.648 INFO:teuthology.orchestra.run.smithi116.stdout: "leader": 24494, 2024-09-12T03:06:03.648 INFO:teuthology.orchestra.run.smithi116.stdout: "set_version": 12, 2024-09-12T03:06:03.648 INFO:teuthology.orchestra.run.smithi116.stdout: "sets": { 2024-09-12T03:06:03.648 INFO:teuthology.orchestra.run.smithi116.stdout: "ff1640ad": { 2024-09-12T03:06:03.648 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 12, 2024-09-12T03:06:03.649 INFO:teuthology.orchestra.run.smithi116.stdout: "age_ref": 0.0, 2024-09-12T03:06:03.649 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:06:03.649 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RELEASED", 2024-09-12T03:06:03.649 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:06:03.649 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:06:03.649 INFO:teuthology.orchestra.run.smithi116.stdout: "timeout": 90.0, 2024-09-12T03:06:03.649 INFO:teuthology.orchestra.run.smithi116.stdout: "expiration": 144.7, 2024-09-12T03:06:03.649 INFO:teuthology.orchestra.run.smithi116.stdout: "members": { 2024-09-12T03:06:03.649 INFO:teuthology.orchestra.run.smithi116.stdout: "file:/": { 2024-09-12T03:06:03.649 INFO:teuthology.orchestra.run.smithi116.stdout: "excluded": false, 2024-09-12T03:06:03.650 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:06:03.650 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RELEASED", 2024-09-12T03:06:03.650 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:06:03.650 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:06:03.650 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:06:03.650 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:06:03.650 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:06:03.650 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:06:03.650 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:06:03.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.643+0000 7fb770435640 1 -- 172.21.15.116:0/1607517519 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb7540a2b80 msgr2=0x7fb7540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:03.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.643+0000 7fb770435640 1 --2- 172.21.15.116:0/1607517519 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb7540a2b80 0x7fb7540a5040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb760005c40 tx=0x7fb760014a50 comp rx=0 tx=0).stop 2024-09-12T03:06:03.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.643+0000 7fb770435640 1 -- 172.21.15.116:0/1607517519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb768102780 msgr2=0x7fb768198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:03.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.643+0000 7fb770435640 1 --2- 172.21.15.116:0/1607517519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb768102780 0x7fb768198960 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb758006010 tx=0x7fb7580042f0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.643+0000 7fb770435640 1 -- 172.21.15.116:0/1607517519 shutdown_connections 2024-09-12T03:06:03.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.643+0000 7fb770435640 1 --2- 172.21.15.116:0/1607517519 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb7540a2b80 0x7fb7540a5040 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.643+0000 7fb770435640 1 --2- 172.21.15.116:0/1607517519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb768106f00 0x7fb76819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.643+0000 7fb770435640 1 --2- 172.21.15.116:0/1607517519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb768102780 0x7fb768198960 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.643+0000 7fb770435640 1 --2- 172.21.15.116:0/1607517519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb768101dd0 0x7fb768198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:03.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.643+0000 7fb770435640 1 -- 172.21.15.116:0/1607517519 >> 172.21.15.116:0/1607517519 conn(0x7fb7680fd8b0 msgr2=0x7fb7680fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:03.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.643+0000 7fb770435640 1 -- 172.21.15.116:0/1607517519 shutdown_connections 2024-09-12T03:06:03.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:03.643+0000 7fb770435640 1 -- 172.21.15.116:0/1607517519 wait complete. 2024-09-12T03:06:04.473 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ff1640ad', total seconds elapsed: 28.2 2024-09-12T03:06:04.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:04 smithi116 ceph-mon[31410]: pgmap v430: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-12T03:06:04.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:04 smithi183 ceph-mon[35535]: pgmap v430: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-12T03:06:04.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:04 smithi177 ceph-mon[35305]: pgmap v430: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-12T03:06:05.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:05 smithi183 ceph-mon[35535]: pgmap v431: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-12T03:06:05.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:05 smithi177 ceph-mon[35305]: pgmap v431: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-12T03:06:05.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:05 smithi116 ceph-mon[31410]: pgmap v431: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-12T03:06:08.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:08 smithi177 ceph-mon[35305]: pgmap v432: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-09-12T03:06:08.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:08 smithi116 ceph-mon[31410]: pgmap v432: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-09-12T03:06:08.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:08 smithi183 ceph-mon[35535]: pgmap v432: 97 pgs: 97 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-09-12T03:06:10.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:10 smithi177 ceph-mon[35305]: pgmap v433: 97 pgs: 97 active+clean; 1.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2024-09-12T03:06:10.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:10 smithi116 ceph-mon[31410]: pgmap v433: 97 pgs: 97 active+clean; 1.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2024-09-12T03:06:10.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:10 smithi183 ceph-mon[35535]: pgmap v433: 97 pgs: 97 active+clean; 1.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2024-09-12T03:06:12.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:12 smithi116 ceph-mon[31410]: pgmap v434: 97 pgs: 97 active+clean; 1.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 KiB/s wr, 10 op/s 2024-09-12T03:06:12.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:12 smithi183 ceph-mon[35535]: pgmap v434: 97 pgs: 97 active+clean; 1.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 KiB/s wr, 10 op/s 2024-09-12T03:06:12.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:12 smithi177 ceph-mon[35305]: pgmap v434: 97 pgs: 97 active+clean; 1.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 KiB/s wr, 10 op/s 2024-09-12T03:06:14.546 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:14 smithi177 ceph-mon[35305]: pgmap v435: 97 pgs: 97 active+clean; 1.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 8 op/s 2024-09-12T03:06:14.546 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:14 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:06:14.547 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:14 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:06:14.547 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:14 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:06:14.547 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:14 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:06:14.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:14 smithi116 ceph-mon[31410]: pgmap v435: 97 pgs: 97 active+clean; 1.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 8 op/s 2024-09-12T03:06:14.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:14 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:06:14.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:14 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:06:14.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:14 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:06:14.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:14 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:06:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:14 smithi183 ceph-mon[35535]: pgmap v435: 97 pgs: 97 active+clean; 1.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 8 op/s 2024-09-12T03:06:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:14 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:06:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:14 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:06:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:14 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:06:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:14 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:06:16.505 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:16 smithi183 ceph-mon[35535]: pgmap v436: 97 pgs: 97 active+clean; 1.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.1 KiB/s wr, 9 op/s 2024-09-12T03:06:16.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:16 smithi116 ceph-mon[31410]: pgmap v436: 97 pgs: 97 active+clean; 1.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.1 KiB/s wr, 9 op/s 2024-09-12T03:06:16.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:16 smithi177 ceph-mon[35305]: pgmap v436: 97 pgs: 97 active+clean; 1.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.1 KiB/s wr, 9 op/s 2024-09-12T03:06:18.348 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:06:18.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:18 smithi116 ceph-mon[31410]: pgmap v437: 97 pgs: 97 active+clean; 1.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-09-12T03:06:18.608 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:06:18.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:18 smithi183 ceph-mon[35535]: pgmap v437: 97 pgs: 97 active+clean; 1.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-09-12T03:06:18.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:18 smithi177 ceph-mon[35305]: pgmap v437: 97 pgs: 97 active+clean; 1.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-09-12T03:06:19.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.125+0000 7fee4a1a1640 1 -- 172.21.15.116:0/2768506437 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee441027c0 msgr2=0x7fee44106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:19.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.125+0000 7fee4a1a1640 1 --2- 172.21.15.116:0/2768506437 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee441027c0 0x7fee44106810 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fee38009900 tx=0x7fee3802a410 comp rx=0 tx=0).stop 2024-09-12T03:06:19.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.125+0000 7fee4a1a1640 1 -- 172.21.15.116:0/2768506437 shutdown_connections 2024-09-12T03:06:19.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.125+0000 7fee4a1a1640 1 --2- 172.21.15.116:0/2768506437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee44106f40 0x7fee4410eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.125+0000 7fee4a1a1640 1 --2- 172.21.15.116:0/2768506437 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee441027c0 0x7fee44106810 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.125+0000 7fee4a1a1640 1 --2- 172.21.15.116:0/2768506437 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee44101e10 0x7fee441021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.125+0000 7fee4a1a1640 1 -- 172.21.15.116:0/2768506437 >> 172.21.15.116:0/2768506437 conn(0x7fee440fd8d0 msgr2=0x7fee440ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:19.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.125+0000 7fee4a1a1640 1 -- 172.21.15.116:0/2768506437 shutdown_connections 2024-09-12T03:06:19.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.125+0000 7fee4a1a1640 1 -- 172.21.15.116:0/2768506437 wait complete. 2024-09-12T03:06:19.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.126+0000 7fee4a1a1640 1 Processor -- start 2024-09-12T03:06:19.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.126+0000 7fee4a1a1640 1 -- start start 2024-09-12T03:06:19.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.126+0000 7fee4a1a1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee44101e10 0x7fee44198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:19.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.126+0000 7fee4a1a1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee441027c0 0x7fee44198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:19.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.126+0000 7fee4a1a1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee44106f40 0x7fee4419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:19.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.126+0000 7fee4a1a1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fee44111fc0 con 0x7fee44101e10 2024-09-12T03:06:19.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.126+0000 7fee4a1a1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fee44111e40 con 0x7fee441027c0 2024-09-12T03:06:19.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.126+0000 7fee4a1a1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fee44112140 con 0x7fee44106f40 2024-09-12T03:06:19.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.126+0000 7fee437fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee44101e10 0x7fee44198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:19.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.126+0000 7fee437fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee44101e10 0x7fee44198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36770/0 (socket says 172.21.15.116:36770) 2024-09-12T03:06:19.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.126+0000 7fee437fe640 1 -- 172.21.15.116:0/2708287400 learned_addr learned my addr 172.21.15.116:0/2708287400 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:19.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.127+0000 7fee42ffd640 1 --2- 172.21.15.116:0/2708287400 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee441027c0 0x7fee44198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:19.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.127+0000 7fee43fff640 1 --2- 172.21.15.116:0/2708287400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee44106f40 0x7fee4419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:19.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.127+0000 7fee437fe640 1 -- 172.21.15.116:0/2708287400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee44106f40 msgr2=0x7fee4419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:19.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.127+0000 7fee437fe640 1 --2- 172.21.15.116:0/2708287400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee44106f40 0x7fee4419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.127+0000 7fee437fe640 1 -- 172.21.15.116:0/2708287400 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee441027c0 msgr2=0x7fee44198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:19.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.127+0000 7fee437fe640 1 --2- 172.21.15.116:0/2708287400 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee441027c0 0x7fee44198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.127+0000 7fee437fe640 1 -- 172.21.15.116:0/2708287400 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee4419d4b0 con 0x7fee44101e10 2024-09-12T03:06:19.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.127+0000 7fee42ffd640 1 --2- 172.21.15.116:0/2708287400 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee441027c0 0x7fee44198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:06:19.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.127+0000 7fee437fe640 1 --2- 172.21.15.116:0/2708287400 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee44101e10 0x7fee44198500 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fee3400c890 tx=0x7fee3400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:19.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.127+0000 7fee40ff9640 1 -- 172.21.15.116:0/2708287400 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee34004220 con 0x7fee44101e10 2024-09-12T03:06:19.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.127+0000 7fee40ff9640 1 -- 172.21.15.116:0/2708287400 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee340043c0 con 0x7fee44101e10 2024-09-12T03:06:19.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.127+0000 7fee4a1a1640 1 -- 172.21.15.116:0/2708287400 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee4419d7a0 con 0x7fee44101e10 2024-09-12T03:06:19.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.127+0000 7fee40ff9640 1 -- 172.21.15.116:0/2708287400 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee340049e0 con 0x7fee44101e10 2024-09-12T03:06:19.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.127+0000 7fee4a1a1640 1 -- 172.21.15.116:0/2708287400 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee441a5080 con 0x7fee44101e10 2024-09-12T03:06:19.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.130+0000 7fee40ff9640 1 -- 172.21.15.116:0/2708287400 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fee340079b0 con 0x7fee44101e10 2024-09-12T03:06:19.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.130+0000 7fee4c57a740 1 Processor -- start 2024-09-12T03:06:19.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.130+0000 7fee4c57a740 1 -- start start 2024-09-12T03:06:19.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.131+0000 7fee4c57a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643d882a4a0 0x5643d882a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:19.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.131+0000 7fee4c57a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643d882adc0 0x5643d8835140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:19.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.131+0000 7fee4c57a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d8835680 0x5643d8837a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:19.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.131+0000 7fee4c57a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5643d87902e0 con 0x5643d882a4a0 2024-09-12T03:06:19.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.131+0000 7fee4c57a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5643d8737ce0 con 0x5643d8835680 2024-09-12T03:06:19.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.131+0000 7fee4c57a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5643d878c2a0 con 0x5643d882adc0 2024-09-12T03:06:19.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.131+0000 7fee43fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643d882a4a0 0x5643d882a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:19.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.131+0000 7fee43fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643d882a4a0 0x5643d882a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36784/0 (socket says 172.21.15.116:36784) 2024-09-12T03:06:19.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.131+0000 7fee43fff640 1 -- 172.21.15.116:0/1055308183 learned_addr learned my addr 172.21.15.116:0/1055308183 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:19.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.131+0000 7fee437fe640 1 --2- 172.21.15.116:0/1055308183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643d882adc0 0x5643d8835140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:19.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.131+0000 7fee42ffd640 1 --2- 172.21.15.116:0/1055308183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d8835680 0x5643d8837a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:19.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.131+0000 7fee40ff9640 1 --2- 172.21.15.116:0/2708287400 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fee2c0a2b80 0x7fee2c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:19.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.131+0000 7fee43fff640 1 -- 172.21.15.116:0/1055308183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643d882adc0 msgr2=0x5643d8835140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:19.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.131+0000 7fee43fff640 1 --2- 172.21.15.116:0/1055308183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643d882adc0 0x5643d8835140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.131+0000 7fee43fff640 1 -- 172.21.15.116:0/1055308183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d8835680 msgr2=0x5643d8837a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:19.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.131+0000 7fee43fff640 1 --2- 172.21.15.116:0/1055308183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d8835680 0x5643d8837a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.131+0000 7fee43fff640 1 -- 172.21.15.116:0/1055308183 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643d871c440 con 0x5643d882a4a0 2024-09-12T03:06:19.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.131+0000 7fee40ff9640 1 -- 172.21.15.116:0/2708287400 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fee340cf120 con 0x7fee44101e10 2024-09-12T03:06:19.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.131+0000 7fee43fff640 1 --2- 172.21.15.116:0/2708287400 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fee2c0a2b80 0x7fee2c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:19.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.131+0000 7fee43fff640 1 --2- 172.21.15.116:0/1055308183 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643d882a4a0 0x5643d882a880 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00c4a0 tx=0x7fee3c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fd6ab1dcdcb75f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:19.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.132+0000 7fee29ffb640 1 -- 172.21.15.116:0/1055308183 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c018640 con 0x5643d882a4a0 2024-09-12T03:06:19.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.132+0000 7fee29ffb640 1 -- 172.21.15.116:0/1055308183 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee3c01e070 con 0x5643d882a4a0 2024-09-12T03:06:19.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.132+0000 7fee29ffb640 1 -- 172.21.15.116:0/1055308183 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c00a970 con 0x5643d882a4a0 2024-09-12T03:06:19.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.132+0000 7fee4c57a740 1 -- 172.21.15.116:0/1055308183 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643d882a4a0 msgr2=0x5643d882a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:19.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.132+0000 7fee43fff640 1 --2- 172.21.15.116:0/2708287400 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fee2c0a2b80 0x7fee2c0a5040 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fee3c01d000 tx=0x7fee3c017000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:19.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.132+0000 7fee4c57a740 1 --2- 172.21.15.116:0/1055308183 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643d882a4a0 0x5643d882a880 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00c4a0 tx=0x7fee3c00c960 comp rx=0 tx=0).stop 2024-09-12T03:06:19.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.132+0000 7fee4c57a740 1 -- 172.21.15.116:0/1055308183 shutdown_connections 2024-09-12T03:06:19.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.132+0000 7fee4c57a740 1 --2- 172.21.15.116:0/1055308183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d8835680 0x5643d8837a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.132+0000 7fee4c57a740 1 --2- 172.21.15.116:0/1055308183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643d882adc0 0x5643d8835140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.132+0000 7fee4c57a740 1 --2- 172.21.15.116:0/1055308183 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643d882a4a0 0x5643d882a880 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.132+0000 7fee4c57a740 1 -- 172.21.15.116:0/1055308183 >> 172.21.15.116:0/1055308183 conn(0x5643d8826a30 msgr2=0x5643d8826e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:19.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.133+0000 7fee4c57a740 1 -- 172.21.15.116:0/1055308183 shutdown_connections 2024-09-12T03:06:19.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.133+0000 7fee4c57a740 1 -- 172.21.15.116:0/1055308183 wait complete. 2024-09-12T03:06:19.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.133+0000 7fee4c57a740 1 -- start start 2024-09-12T03:06:19.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.133+0000 7fee4c57a740 1 Processor -- start 2024-09-12T03:06:19.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.133+0000 7fee4c57a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643d882a4a0 0x5643d8834fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:19.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.133+0000 7fee4c57a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d882adc0 0x5643d8832270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:19.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.133+0000 7fee4c57a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643d8835680 0x5643d88327b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:19.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.133+0000 7fee4c57a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5643d8838250 con 0x5643d8835680 2024-09-12T03:06:19.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.133+0000 7fee4c57a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5643d874b050 con 0x5643d882adc0 2024-09-12T03:06:19.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.133+0000 7fee4c57a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5643d88383d0 con 0x5643d882a4a0 2024-09-12T03:06:19.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.133+0000 7fee4c57a740 10 client.0 _ll_register_callbacks cb 0x5643d878b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:19.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.133+0000 7fee43fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643d8835680 0x5643d88327b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:19.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.134+0000 7fee43fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643d8835680 0x5643d88327b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36792/0 (socket says 172.21.15.116:36792) 2024-09-12T03:06:19.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.134+0000 7fee43fff640 1 -- 172.21.15.116:0/112303895 learned_addr learned my addr 172.21.15.116:0/112303895 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:19.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.134+0000 7fee42ffd640 1 --2- 172.21.15.116:0/112303895 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d882adc0 0x5643d8832270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:19.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.134+0000 7fee43fff640 1 -- 172.21.15.116:0/112303895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643d882a4a0 msgr2=0x5643d8834fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:19.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.134+0000 7fee43fff640 1 --2- 172.21.15.116:0/112303895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643d882a4a0 0x5643d8834fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.134+0000 7fee43fff640 1 -- 172.21.15.116:0/112303895 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d882adc0 msgr2=0x5643d8832270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:19.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.134+0000 7fee43fff640 1 --2- 172.21.15.116:0/112303895 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d882adc0 0x5643d8832270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.134+0000 7fee43fff640 1 -- 172.21.15.116:0/112303895 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643d85013c0 con 0x5643d8835680 2024-09-12T03:06:19.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.134+0000 7fee42ffd640 1 --2- 172.21.15.116:0/112303895 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d882adc0 0x5643d8832270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:06:19.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.134+0000 7fee43fff640 1 --2- 172.21.15.116:0/112303895 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643d8835680 0x5643d88327b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00a550 tx=0x7fee3c018a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe5c3fe83b944b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:19.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.134+0000 7fee22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:06:19.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.134+0000 7fee22ffd640 1 -- 172.21.15.116:0/112303895 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c018bf0 con 0x5643d8835680 2024-09-12T03:06:19.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.134+0000 7fee22ffd640 1 -- 172.21.15.116:0/112303895 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fee3c022430 con 0x5643d8835680 2024-09-12T03:06:19.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.134+0000 7fee4c57a740 1 -- 172.21.15.116:0/112303895 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee3c01e070 con 0x5643d8835680 2024-09-12T03:06:19.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.134+0000 7fee22ffd640 1 -- 172.21.15.116:0/112303895 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fee3c018d90 con 0x5643d8835680 2024-09-12T03:06:19.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.134+0000 7fee4c57a740 10 client.16056 fetch_fsmap learned FSMap version 25 2024-09-12T03:06:19.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.134+0000 7fee4c57a740 1 -- 172.21.15.116:0/112303895 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5643d8843c00 con 0x5643d8835680 2024-09-12T03:06:19.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.134+0000 7fee22ffd640 1 -- 172.21.15.116:0/112303895 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee3c01e070 con 0x5643d8835680 2024-09-12T03:06:19.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.135+0000 7fee22ffd640 1 -- 172.21.15.116:0/112303895 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fee3c029070 con 0x5643d8835680 2024-09-12T03:06:19.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.135+0000 7fee4c57a740 10 client.16056 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:06:19.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.135+0000 7fee4c57a740 10 client.16056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:06:19.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.135+0000 7fee4c57a740 20 client.16056 populate_metadata read hostname 'smithi116' 2024-09-12T03:06:19.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.135+0000 7fee4c57a740 1 --2- 172.21.15.116:0/112303895 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5643d8844f50 0x5643d8847340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:19.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.135+0000 7fee4c57a740 4 client.16056 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:06:19.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.135+0000 7fee4c57a740 1 -- 172.21.15.116:0/112303895 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643d879d850 con 0x5643d8844f50 2024-09-12T03:06:19.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.135+0000 7fee437fe640 1 --2- 172.21.15.116:0/112303895 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5643d8844f50 0x5643d8847340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:19.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.135+0000 7fee437fe640 1 --2- 172.21.15.116:0/112303895 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5643d8844f50 0x5643d8847340 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e50b93eda20f083b server_cookie=57f3c001a7b7e765 in_seq=0 out_seq=0 2024-09-12T03:06:19.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.135+0000 7fee22ffd640 10 client.16056 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:19.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.136+0000 7fee22ffd640 1 -- 172.21.15.116:0/112303895 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5643d879d850 con 0x5643d8844f50 2024-09-12T03:06:19.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.136+0000 7fee22ffd640 10 client.16056 handle_command_reply: tid=0 2024-09-12T03:06:19.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.136+0000 7fee4c57a740 1 client.16056 shutdown 2024-09-12T03:06:19.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.136+0000 7fee4c57a740 1 -- 172.21.15.116:0/112303895 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643d8835680 msgr2=0x5643d88327b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:19.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.136+0000 7fee4c57a740 1 --2- 172.21.15.116:0/112303895 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643d8835680 0x5643d88327b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00a550 tx=0x7fee3c018a00 comp rx=0 tx=0).stop 2024-09-12T03:06:19.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.136+0000 7fee4c57a740 1 -- 172.21.15.116:0/112303895 shutdown_connections 2024-09-12T03:06:19.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.136+0000 7fee4c57a740 1 --2- 172.21.15.116:0/112303895 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5643d8844f50 0x5643d8847340 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.136+0000 7fee4c57a740 1 --2- 172.21.15.116:0/112303895 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643d8835680 0x5643d88327b0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.136+0000 7fee4c57a740 1 --2- 172.21.15.116:0/112303895 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d882adc0 0x5643d8832270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.136+0000 7fee22ffd640 0 client.16056 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:19.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.136+0000 7fee4c57a740 1 --2- 172.21.15.116:0/112303895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643d882a4a0 0x5643d8834fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.136+0000 7fee4c57a740 1 -- 172.21.15.116:0/112303895 >> 172.21.15.116:0/112303895 conn(0x5643d8826a30 msgr2=0x5643d882c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:19.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.136+0000 7fee4c57a740 1 -- 172.21.15.116:0/112303895 shutdown_connections 2024-09-12T03:06:19.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.136+0000 7fee4c57a740 1 -- 172.21.15.116:0/112303895 wait complete. 2024-09-12T03:06:19.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.136+0000 7fee4c57a740 20 client.16056 trim_cache size 0 max 16384 2024-09-12T03:06:19.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.154+0000 7fee4c57a740 1 Processor -- start 2024-09-12T03:06:19.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.154+0000 7fee4c57a740 1 -- start start 2024-09-12T03:06:19.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.154+0000 7fee4c57a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643d8826a30 0x5643d8830b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:19.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.154+0000 7fee4c57a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d882a4a0 0x5643d884cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:19.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.154+0000 7fee4c57a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643d882adc0 0x5643d884d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:19.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.154+0000 7fee4c57a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fee441add20 con 0x5643d882adc0 2024-09-12T03:06:19.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.154+0000 7fee4c57a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fee441ab5f0 con 0x5643d882a4a0 2024-09-12T03:06:19.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.154+0000 7fee4c57a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fee441a9460 con 0x5643d8826a30 2024-09-12T03:06:19.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.154+0000 7fee43fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643d882adc0 0x5643d884d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:19.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.154+0000 7fee43fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643d882adc0 0x5643d884d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36796/0 (socket says 172.21.15.116:36796) 2024-09-12T03:06:19.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.154+0000 7fee43fff640 1 -- 172.21.15.116:0/2914154824 learned_addr learned my addr 172.21.15.116:0/2914154824 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:19.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.154+0000 7fee437fe640 1 --2- 172.21.15.116:0/2914154824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643d8826a30 0x5643d8830b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:19.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.154+0000 7fee42ffd640 1 --2- 172.21.15.116:0/2914154824 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d882a4a0 0x5643d884cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:19.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.154+0000 7fee43fff640 1 -- 172.21.15.116:0/2914154824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643d8826a30 msgr2=0x5643d8830b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:19.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.154+0000 7fee43fff640 1 --2- 172.21.15.116:0/2914154824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643d8826a30 0x5643d8830b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.154+0000 7fee43fff640 1 -- 172.21.15.116:0/2914154824 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d882a4a0 msgr2=0x5643d884cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:19.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.154+0000 7fee43fff640 1 --2- 172.21.15.116:0/2914154824 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d882a4a0 0x5643d884cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.154+0000 7fee43fff640 1 -- 172.21.15.116:0/2914154824 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643d88476b0 con 0x5643d882adc0 2024-09-12T03:06:19.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.154+0000 7fee437fe640 1 --2- 172.21.15.116:0/2914154824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643d8826a30 0x5643d8830b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:06:19.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.155+0000 7fee43fff640 1 --2- 172.21.15.116:0/2914154824 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643d882adc0 0x5643d884d040 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fee3c002230 tx=0x7fee3c00ab60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db264bdd71e7b7cf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:19.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.155+0000 7fee297fa640 1 -- 172.21.15.116:0/2914154824 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c009c20 con 0x5643d882adc0 2024-09-12T03:06:19.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.155+0000 7fee297fa640 1 -- 172.21.15.116:0/2914154824 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee3c03a020 con 0x5643d882adc0 2024-09-12T03:06:19.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.155+0000 7fee297fa640 1 -- 172.21.15.116:0/2914154824 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c00a700 con 0x5643d882adc0 2024-09-12T03:06:19.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.156+0000 7fee4c57a740 1 -- 172.21.15.116:0/2914154824 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643d882adc0 msgr2=0x5643d884d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:19.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.156+0000 7fee4c57a740 1 --2- 172.21.15.116:0/2914154824 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643d882adc0 0x5643d884d040 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fee3c002230 tx=0x7fee3c00ab60 comp rx=0 tx=0).stop 2024-09-12T03:06:19.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.156+0000 7fee4c57a740 1 -- 172.21.15.116:0/2914154824 shutdown_connections 2024-09-12T03:06:19.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.156+0000 7fee4c57a740 1 --2- 172.21.15.116:0/2914154824 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643d882adc0 0x5643d884d040 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.156+0000 7fee4c57a740 1 --2- 172.21.15.116:0/2914154824 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d882a4a0 0x5643d884cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.156+0000 7fee4c57a740 1 --2- 172.21.15.116:0/2914154824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643d8826a30 0x5643d8830b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.156+0000 7fee4c57a740 1 -- 172.21.15.116:0/2914154824 >> 172.21.15.116:0/2914154824 conn(0x5643d8832910 msgr2=0x5643d88342f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:19.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.156+0000 7fee4c57a740 1 -- 172.21.15.116:0/2914154824 shutdown_connections 2024-09-12T03:06:19.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.156+0000 7fee4c57a740 1 -- 172.21.15.116:0/2914154824 wait complete. 2024-09-12T03:06:19.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.156+0000 7fee4c57a740 1 -- start start 2024-09-12T03:06:19.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.156+0000 7fee4c57a740 1 Processor -- start 2024-09-12T03:06:19.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.157+0000 7fee4c57a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643d8826a30 0x5643d883b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:19.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.157+0000 7fee4c57a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d882a4a0 0x5643d883b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:19.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.157+0000 7fee4c57a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643d882adc0 0x5643d883bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:19.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.157+0000 7fee4c57a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fee441a9ab0 con 0x5643d8826a30 2024-09-12T03:06:19.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.157+0000 7fee4c57a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fee441aaf20 con 0x5643d882a4a0 2024-09-12T03:06:19.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.157+0000 7fee4c57a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fee441aa180 con 0x5643d882adc0 2024-09-12T03:06:19.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.157+0000 7fee437fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643d8826a30 0x5643d883b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:19.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.157+0000 7fee4c57a740 10 client.0 _ll_register_callbacks cb 0x7fee38009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:19.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.157+0000 7fee437fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643d8826a30 0x5643d883b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36808/0 (socket says 172.21.15.116:36808) 2024-09-12T03:06:19.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.157+0000 7fee437fe640 1 -- 172.21.15.116:0/2584538888 learned_addr learned my addr 172.21.15.116:0/2584538888 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:19.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.157+0000 7fee42ffd640 1 --2- 172.21.15.116:0/2584538888 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d882a4a0 0x5643d883b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:19.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.157+0000 7fee437fe640 1 -- 172.21.15.116:0/2584538888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643d882adc0 msgr2=0x5643d883bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:19.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.157+0000 7fee437fe640 1 --2- 172.21.15.116:0/2584538888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643d882adc0 0x5643d883bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.157+0000 7fee437fe640 1 -- 172.21.15.116:0/2584538888 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d882a4a0 msgr2=0x5643d883b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:19.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.157+0000 7fee437fe640 1 --2- 172.21.15.116:0/2584538888 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d882a4a0 0x5643d883b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.157+0000 7fee437fe640 1 -- 172.21.15.116:0/2584538888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643d88aeb90 con 0x5643d8826a30 2024-09-12T03:06:19.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.157+0000 7fee437fe640 1 --2- 172.21.15.116:0/2584538888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643d8826a30 0x5643d883b440 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fee340ccfc0 tx=0x7fee34097d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8cf61700aeb2ba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:19.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.157+0000 7fee22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:06:19.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.157+0000 7fee22ffd640 1 -- 172.21.15.116:0/2584538888 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3401d040 con 0x5643d8826a30 2024-09-12T03:06:19.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.158+0000 7fee22ffd640 1 -- 172.21.15.116:0/2584538888 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fee34099cb0 con 0x5643d8826a30 2024-09-12T03:06:19.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.158+0000 7fee4c57a740 1 -- 172.21.15.116:0/2584538888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee3c03a020 con 0x5643d8826a30 2024-09-12T03:06:19.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.158+0000 7fee22ffd640 1 -- 172.21.15.116:0/2584538888 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fee3409b310 con 0x5643d8826a30 2024-09-12T03:06:19.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.158+0000 7fee4c57a740 10 client.16062 fetch_fsmap learned FSMap version 25 2024-09-12T03:06:19.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.158+0000 7fee4c57a740 1 -- 172.21.15.116:0/2584538888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5643d88afe80 con 0x5643d8826a30 2024-09-12T03:06:19.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.158+0000 7fee22ffd640 1 -- 172.21.15.116:0/2584538888 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee3c03a020 con 0x5643d8826a30 2024-09-12T03:06:19.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.158+0000 7fee22ffd640 1 -- 172.21.15.116:0/2584538888 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fee3409cce0 con 0x5643d8826a30 2024-09-12T03:06:19.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.158+0000 7fee4c57a740 10 client.16062 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:06:19.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.158+0000 7fee4c57a740 10 client.16062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:06:19.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.158+0000 7fee4c57a740 20 client.16062 populate_metadata read hostname 'smithi116' 2024-09-12T03:06:19.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.158+0000 7fee4c57a740 1 --2- 172.21.15.116:0/2584538888 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5643d8835680 0x5643d88b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:19.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.158+0000 7fee4c57a740 4 client.16062 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:06:19.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.158+0000 7fee4c57a740 1 -- 172.21.15.116:0/2584538888 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5643d8797d60 con 0x5643d8835680 2024-09-12T03:06:19.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.158+0000 7fee42ffd640 1 --2- 172.21.15.116:0/2584538888 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5643d8835680 0x5643d88b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:19.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.159+0000 7fee42ffd640 1 --2- 172.21.15.116:0/2584538888 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5643d8835680 0x5643d88b2560 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3e8594c46df6fe1 server_cookie=57f178a5fc8b32ca in_seq=0 out_seq=0 2024-09-12T03:06:19.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.159+0000 7fee22ffd640 10 client.16062 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:19.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.159+0000 7fee22ffd640 1 -- 172.21.15.116:0/2584538888 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5643d8797d60 con 0x5643d8835680 2024-09-12T03:06:19.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.159+0000 7fee22ffd640 10 client.16062 handle_command_reply: tid=0 2024-09-12T03:06:19.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.159+0000 7fee4c57a740 1 client.16062 shutdown 2024-09-12T03:06:19.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.159+0000 7fee4c57a740 1 -- 172.21.15.116:0/2584538888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643d8826a30 msgr2=0x5643d883b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:19.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.159+0000 7fee4c57a740 1 --2- 172.21.15.116:0/2584538888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643d8826a30 0x5643d883b440 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fee340ccfc0 tx=0x7fee34097d20 comp rx=0 tx=0).stop 2024-09-12T03:06:19.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.159+0000 7fee4c57a740 1 -- 172.21.15.116:0/2584538888 shutdown_connections 2024-09-12T03:06:19.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.159+0000 7fee4c57a740 1 --2- 172.21.15.116:0/2584538888 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5643d8835680 0x5643d88b2560 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.159+0000 7fee4c57a740 1 --2- 172.21.15.116:0/2584538888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643d882adc0 0x5643d883bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.159+0000 7fee4c57a740 1 --2- 172.21.15.116:0/2584538888 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d882a4a0 0x5643d883b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.159+0000 7fee4c57a740 1 --2- 172.21.15.116:0/2584538888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643d8826a30 0x5643d883b440 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.159+0000 7fee4c57a740 1 -- 172.21.15.116:0/2584538888 >> 172.21.15.116:0/2584538888 conn(0x5643d8832910 msgr2=0x5643d8825220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:19.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.159+0000 7fee22ffd640 0 client.16062 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:19.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.159+0000 7fee4c57a740 1 -- 172.21.15.116:0/2584538888 shutdown_connections 2024-09-12T03:06:19.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.159+0000 7fee4c57a740 1 -- 172.21.15.116:0/2584538888 wait complete. 2024-09-12T03:06:19.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.159+0000 7fee4c57a740 20 client.16062 trim_cache size 0 max 16384 2024-09-12T03:06:19.170 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:06:19.171 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:06:19.171 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:06:19.171 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:06:19.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.160+0000 7fee4a1a1640 1 -- 172.21.15.116:0/2708287400 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fee2c0a2b80 msgr2=0x7fee2c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:19.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.160+0000 7fee4a1a1640 1 --2- 172.21.15.116:0/2708287400 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fee2c0a2b80 0x7fee2c0a5040 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fee3c01d000 tx=0x7fee3c017000 comp rx=0 tx=0).stop 2024-09-12T03:06:19.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.160+0000 7fee4a1a1640 1 -- 172.21.15.116:0/2708287400 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee44101e10 msgr2=0x7fee44198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:19.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.160+0000 7fee4a1a1640 1 --2- 172.21.15.116:0/2708287400 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee44101e10 0x7fee44198500 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fee3400c890 tx=0x7fee3400cd50 comp rx=0 tx=0).stop 2024-09-12T03:06:19.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.160+0000 7fee4a1a1640 1 -- 172.21.15.116:0/2708287400 shutdown_connections 2024-09-12T03:06:19.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.160+0000 7fee4a1a1640 1 --2- 172.21.15.116:0/2708287400 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fee2c0a2b80 0x7fee2c0a5040 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.160+0000 7fee4a1a1640 1 --2- 172.21.15.116:0/2708287400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee44106f40 0x7fee4419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.160+0000 7fee4a1a1640 1 --2- 172.21.15.116:0/2708287400 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fee441027c0 0x7fee44198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.160+0000 7fee4a1a1640 1 --2- 172.21.15.116:0/2708287400 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee44101e10 0x7fee44198500 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:19.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.160+0000 7fee4a1a1640 1 -- 172.21.15.116:0/2708287400 >> 172.21.15.116:0/2708287400 conn(0x7fee440fd8d0 msgr2=0x7fee440fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:19.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.160+0000 7fee4a1a1640 1 -- 172.21.15.116:0/2708287400 shutdown_connections 2024-09-12T03:06:19.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:19.160+0000 7fee4a1a1640 1 -- 172.21.15.116:0/2708287400 wait complete. 2024-09-12T03:06:20.074 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:06:20.074 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:06:20.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:20 smithi116 ceph-mon[31410]: pgmap v438: 97 pgs: 97 active+clean; 1.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:06:20.330 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:06:20.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:20 smithi183 ceph-mon[35535]: pgmap v438: 97 pgs: 97 active+clean; 1.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:06:20.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:20 smithi177 ceph-mon[35305]: pgmap v438: 97 pgs: 97 active+clean; 1.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:06:20.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.839+0000 7fb08c9bc640 1 -- 172.21.15.116:0/2669258192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb088101420 msgr2=0x7fb08810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:20.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.839+0000 7fb08c9bc640 1 --2- 172.21.15.116:0/2669258192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb088101420 0x7fb08810d950 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb080009cd0 tx=0x7fb08002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:06:20.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.839+0000 7fb08c9bc640 1 -- 172.21.15.116:0/2669258192 shutdown_connections 2024-09-12T03:06:20.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.839+0000 7fb08c9bc640 1 --2- 172.21.15.116:0/2669258192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb088101420 0x7fb08810d950 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:20.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.839+0000 7fb08c9bc640 1 --2- 172.21.15.116:0/2669258192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb088100a80 0x7fb088100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:20.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.839+0000 7fb08c9bc640 1 --2- 172.21.15.116:0/2669258192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb088106a80 0x7fb088106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:20.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.839+0000 7fb08c9bc640 1 -- 172.21.15.116:0/2669258192 >> 172.21.15.116:0/2669258192 conn(0x7fb0880fc7a0 msgr2=0x7fb0880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:20.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.839+0000 7fb08c9bc640 1 -- 172.21.15.116:0/2669258192 shutdown_connections 2024-09-12T03:06:20.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.839+0000 7fb08c9bc640 1 -- 172.21.15.116:0/2669258192 wait complete. 2024-09-12T03:06:20.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.840+0000 7fb08c9bc640 1 Processor -- start 2024-09-12T03:06:20.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.840+0000 7fb08c9bc640 1 -- start start 2024-09-12T03:06:20.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.840+0000 7fb08c9bc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb088100a80 0x7fb088197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:20.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.840+0000 7fb08c9bc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb088101420 0x7fb088197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:20.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.840+0000 7fb08c9bc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb088106a80 0x7fb08819bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:20.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.840+0000 7fb08c9bc640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb088110f20 con 0x7fb088106a80 2024-09-12T03:06:20.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.840+0000 7fb08c9bc640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb088110da0 con 0x7fb088101420 2024-09-12T03:06:20.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.840+0000 7fb08c9bc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb0881110a0 con 0x7fb088100a80 2024-09-12T03:06:20.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.840+0000 7fb086d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb088106a80 0x7fb08819bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:20.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.840+0000 7fb086d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb088106a80 0x7fb08819bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36822/0 (socket says 172.21.15.116:36822) 2024-09-12T03:06:20.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.840+0000 7fb086d9d640 1 -- 172.21.15.116:0/1977995433 learned_addr learned my addr 172.21.15.116:0/1977995433 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:20.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.841+0000 7fb08659c640 1 --2- 172.21.15.116:0/1977995433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb088100a80 0x7fb088197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:20.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.841+0000 7fb085d9b640 1 --2- 172.21.15.116:0/1977995433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb088101420 0x7fb088197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:20.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.841+0000 7fb086d9d640 1 -- 172.21.15.116:0/1977995433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb088100a80 msgr2=0x7fb088197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:20.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.841+0000 7fb086d9d640 1 --2- 172.21.15.116:0/1977995433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb088100a80 0x7fb088197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:20.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.841+0000 7fb086d9d640 1 -- 172.21.15.116:0/1977995433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb088101420 msgr2=0x7fb088197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:20.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.841+0000 7fb086d9d640 1 --2- 172.21.15.116:0/1977995433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb088101420 0x7fb088197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:20.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.841+0000 7fb086d9d640 1 -- 172.21.15.116:0/1977995433 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb08819c3f0 con 0x7fb088106a80 2024-09-12T03:06:20.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.841+0000 7fb085d9b640 1 --2- 172.21.15.116:0/1977995433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb088101420 0x7fb088197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:06:20.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.841+0000 7fb08659c640 1 --2- 172.21.15.116:0/1977995433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb088100a80 0x7fb088197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:06:20.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.841+0000 7fb086d9d640 1 --2- 172.21.15.116:0/1977995433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb088106a80 0x7fb08819bd10 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb080006010 tx=0x7fb080003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:20.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.841+0000 7fb07f7fe640 1 -- 172.21.15.116:0/1977995433 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb080004450 con 0x7fb088106a80 2024-09-12T03:06:20.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.841+0000 7fb07f7fe640 1 -- 172.21.15.116:0/1977995433 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb080003e10 con 0x7fb088106a80 2024-09-12T03:06:20.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.841+0000 7fb08c9bc640 1 -- 172.21.15.116:0/1977995433 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb08819c680 con 0x7fb088106a80 2024-09-12T03:06:20.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.841+0000 7fb07f7fe640 1 -- 172.21.15.116:0/1977995433 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0800466e0 con 0x7fb088106a80 2024-09-12T03:06:20.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.841+0000 7fb08c9bc640 1 -- 172.21.15.116:0/1977995433 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0881a3fc0 con 0x7fb088106a80 2024-09-12T03:06:20.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.842+0000 7fb08c9bc640 1 -- 172.21.15.116:0/1977995433 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb088100f60 con 0x7fb088106a80 2024-09-12T03:06:20.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.843+0000 7fb07f7fe640 1 -- 172.21.15.116:0/1977995433 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb0800305a0 con 0x7fb088106a80 2024-09-12T03:06:20.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.844+0000 7fb07f7fe640 1 --2- 172.21.15.116:0/1977995433 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb0740a2b80 0x7fb0740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:20.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.844+0000 7fb08659c640 1 --2- 172.21.15.116:0/1977995433 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb0740a2b80 0x7fb0740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:20.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.844+0000 7fb07f7fe640 1 -- 172.21.15.116:0/1977995433 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fb08002e030 con 0x7fb088106a80 2024-09-12T03:06:20.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.845+0000 7fb08659c640 1 --2- 172.21.15.116:0/1977995433 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb0740a2b80 0x7fb0740a5040 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb078009930 tx=0x7fb078008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:20.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:20.848+0000 7fb07f7fe640 1 -- 172.21.15.116:0/1977995433 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb08004b060 con 0x7fb088106a80 2024-09-12T03:06:21.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:21.049+0000 7fb08c9bc640 1 -- 172.21.15.116:0/1977995433 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb088109010 con 0x7fb088106a80 2024-09-12T03:06:21.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:21.049+0000 7fb07f7fe640 1 -- 172.21.15.116:0/1977995433 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7fb080039380 con 0x7fb088106a80 2024-09-12T03:06:21.050 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:06:21.050 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:06:21.050 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:06:21.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:21.052+0000 7fb08c9bc640 1 -- 172.21.15.116:0/1977995433 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb0740a2b80 msgr2=0x7fb0740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:21.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:21.052+0000 7fb08c9bc640 1 --2- 172.21.15.116:0/1977995433 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb0740a2b80 0x7fb0740a5040 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb078009930 tx=0x7fb078008040 comp rx=0 tx=0).stop 2024-09-12T03:06:21.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:21.052+0000 7fb08c9bc640 1 -- 172.21.15.116:0/1977995433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb088106a80 msgr2=0x7fb08819bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:21.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:21.052+0000 7fb08c9bc640 1 --2- 172.21.15.116:0/1977995433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb088106a80 0x7fb08819bd10 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb080006010 tx=0x7fb080003c20 comp rx=0 tx=0).stop 2024-09-12T03:06:21.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:21.052+0000 7fb08c9bc640 1 -- 172.21.15.116:0/1977995433 shutdown_connections 2024-09-12T03:06:21.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:21.052+0000 7fb08c9bc640 1 --2- 172.21.15.116:0/1977995433 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb0740a2b80 0x7fb0740a5040 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:21.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:21.052+0000 7fb08c9bc640 1 --2- 172.21.15.116:0/1977995433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb088106a80 0x7fb08819bd10 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:21.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:21.052+0000 7fb08c9bc640 1 --2- 172.21.15.116:0/1977995433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb088101420 0x7fb088197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:21.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:21.052+0000 7fb08c9bc640 1 --2- 172.21.15.116:0/1977995433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb088100a80 0x7fb088197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:21.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:21.052+0000 7fb08c9bc640 1 -- 172.21.15.116:0/1977995433 >> 172.21.15.116:0/1977995433 conn(0x7fb0880fc7a0 msgr2=0x7fb0880fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:21.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:21.052+0000 7fb08c9bc640 1 -- 172.21.15.116:0/1977995433 shutdown_connections 2024-09-12T03:06:21.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:21.052+0000 7fb08c9bc640 1 -- 172.21.15.116:0/1977995433 wait complete. 2024-09-12T03:06:21.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:21 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1977995433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:06:21.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:21 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1977995433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:06:21.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:21 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1977995433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:06:21.838 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:06:22.090 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:06:22.525 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:22 smithi116 ceph-mon[31410]: pgmap v439: 97 pgs: 97 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:06:22.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.605+0000 7f24463a2640 1 -- 172.21.15.116:0/3863372663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2440101420 msgr2=0x7f244010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:22.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.605+0000 7f24463a2640 1 --2- 172.21.15.116:0/3863372663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2440101420 0x7f244010d950 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2438009cd0 tx=0x7f243802a8e0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.605+0000 7f24463a2640 1 -- 172.21.15.116:0/3863372663 shutdown_connections 2024-09-12T03:06:22.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.605+0000 7f24463a2640 1 --2- 172.21.15.116:0/3863372663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2440101420 0x7f244010d950 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.605+0000 7f24463a2640 1 --2- 172.21.15.116:0/3863372663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2440100a80 0x7f2440100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.605+0000 7f24463a2640 1 --2- 172.21.15.116:0/3863372663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2440106a80 0x7f2440106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.605+0000 7f24463a2640 1 -- 172.21.15.116:0/3863372663 >> 172.21.15.116:0/3863372663 conn(0x7f24400fc7a0 msgr2=0x7f24400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:22.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.605+0000 7f24463a2640 1 -- 172.21.15.116:0/3863372663 shutdown_connections 2024-09-12T03:06:22.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.605+0000 7f24463a2640 1 -- 172.21.15.116:0/3863372663 wait complete. 2024-09-12T03:06:22.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.606+0000 7f24463a2640 1 Processor -- start 2024-09-12T03:06:22.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.606+0000 7f24463a2640 1 -- start start 2024-09-12T03:06:22.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.606+0000 7f24463a2640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2440100a80 0x7f2440102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:22.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.606+0000 7f24463a2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2440101420 0x7f2440102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:22.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.606+0000 7f24463a2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2440106a80 0x7f24401030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:22.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.606+0000 7f24463a2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2440110f20 con 0x7f2440101420 2024-09-12T03:06:22.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.606+0000 7f24463a2640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2440110da0 con 0x7f2440100a80 2024-09-12T03:06:22.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.606+0000 7f24463a2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f24401110a0 con 0x7f2440106a80 2024-09-12T03:06:22.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.606+0000 7f243f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2440101420 0x7f2440102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:22.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.606+0000 7f243f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2440101420 0x7f2440102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52682/0 (socket says 172.21.15.116:52682) 2024-09-12T03:06:22.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.606+0000 7f243f7fe640 1 -- 172.21.15.116:0/3641946239 learned_addr learned my addr 172.21.15.116:0/3641946239 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:22.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.606+0000 7f243ffff640 1 --2- 172.21.15.116:0/3641946239 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2440100a80 0x7f2440102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:22.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.606+0000 7f244493f640 1 --2- 172.21.15.116:0/3641946239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2440106a80 0x7f24401030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:22.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.607+0000 7f243f7fe640 1 -- 172.21.15.116:0/3641946239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2440106a80 msgr2=0x7f24401030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:22.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.607+0000 7f243f7fe640 1 --2- 172.21.15.116:0/3641946239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2440106a80 0x7f24401030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.607+0000 7f243f7fe640 1 -- 172.21.15.116:0/3641946239 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2440100a80 msgr2=0x7f2440102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:22.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.607+0000 7f243f7fe640 1 --2- 172.21.15.116:0/3641946239 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2440100a80 0x7f2440102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.607+0000 7f243f7fe640 1 -- 172.21.15.116:0/3641946239 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24401a6a60 con 0x7f2440101420 2024-09-12T03:06:22.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.607+0000 7f243f7fe640 1 --2- 172.21.15.116:0/3641946239 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2440101420 0x7f2440102b90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f243400b4a0 tx=0x7f243400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:22.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.607+0000 7f243d7fa640 1 -- 172.21.15.116:0/3641946239 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2434014070 con 0x7f2440101420 2024-09-12T03:06:22.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.607+0000 7f24463a2640 1 -- 172.21.15.116:0/3641946239 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24401a6d50 con 0x7f2440101420 2024-09-12T03:06:22.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.607+0000 7f243d7fa640 1 -- 172.21.15.116:0/3641946239 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2434010af0 con 0x7f2440101420 2024-09-12T03:06:22.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.607+0000 7f243d7fa640 1 -- 172.21.15.116:0/3641946239 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2434017d10 con 0x7f2440101420 2024-09-12T03:06:22.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.607+0000 7f24463a2640 1 -- 172.21.15.116:0/3641946239 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24401a7230 con 0x7f2440101420 2024-09-12T03:06:22.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.609+0000 7f243d7fa640 1 -- 172.21.15.116:0/3641946239 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2434017070 con 0x7f2440101420 2024-09-12T03:06:22.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.609+0000 7f244877b740 1 Processor -- start 2024-09-12T03:06:22.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.609+0000 7f244877b740 1 -- start start 2024-09-12T03:06:22.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.610+0000 7f244877b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614574a0b40 0x56145749f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:22.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.610+0000 7f244877b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145749f5d0 0x56145749fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:22.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.610+0000 7f244877b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614574aa370 0x5614574ac760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:22.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.610+0000 7f244877b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5614574052f0 con 0x5614574a0b40 2024-09-12T03:06:22.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.610+0000 7f244877b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5614573accf0 con 0x5614574aa370 2024-09-12T03:06:22.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.610+0000 7f244877b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5614574012b0 con 0x56145749f5d0 2024-09-12T03:06:22.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.610+0000 7f243d7fa640 1 --2- 172.21.15.116:0/3641946239 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f24280a2b80 0x7f24280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:22.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.610+0000 7f244493f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614574a0b40 0x56145749f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:22.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.610+0000 7f244493f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614574a0b40 0x56145749f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52694/0 (socket says 172.21.15.116:52694) 2024-09-12T03:06:22.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.610+0000 7f244493f640 1 -- 172.21.15.116:0/3773244583 learned_addr learned my addr 172.21.15.116:0/3773244583 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:22.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.610+0000 7f244493f640 1 --2- 172.21.15.116:0/3641946239 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f24280a2b80 0x7f24280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:22.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.610+0000 7f243f7fe640 1 --2- 172.21.15.116:0/3773244583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614574aa370 0x5614574ac760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:22.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.610+0000 7f243ffff640 1 --2- 172.21.15.116:0/3773244583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145749f5d0 0x56145749fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:22.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.610+0000 7f243d7fa640 1 -- 172.21.15.116:0/3641946239 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f24340ce450 con 0x7f2440101420 2024-09-12T03:06:22.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.610+0000 7f244493f640 1 -- 172.21.15.116:0/3773244583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145749f5d0 msgr2=0x56145749fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:22.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.610+0000 7f244493f640 1 --2- 172.21.15.116:0/3773244583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145749f5d0 0x56145749fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.610+0000 7f244493f640 1 -- 172.21.15.116:0/3773244583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614574aa370 msgr2=0x5614574ac760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:22.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.610+0000 7f244493f640 1 --2- 172.21.15.116:0/3773244583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614574aa370 0x5614574ac760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.610+0000 7f244493f640 1 -- 172.21.15.116:0/3773244583 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614573913a0 con 0x5614574a0b40 2024-09-12T03:06:22.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.610+0000 7f244493f640 1 --2- 172.21.15.116:0/3773244583 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614574a0b40 0x56145749f090 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2438009430 tx=0x7f2438031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1993f21e19a9b28f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:22.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.610+0000 7f244493f640 1 --2- 172.21.15.116:0/3641946239 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f24280a2b80 0x7f24280a5040 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2438009a60 tx=0x7f2438048000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:22.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.610+0000 7f24267fc640 1 -- 172.21.15.116:0/3773244583 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243804e070 con 0x5614574a0b40 2024-09-12T03:06:22.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.611+0000 7f24267fc640 1 -- 172.21.15.116:0/3773244583 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2438036040 con 0x5614574a0b40 2024-09-12T03:06:22.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.611+0000 7f24267fc640 1 -- 172.21.15.116:0/3773244583 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438030800 con 0x5614574a0b40 2024-09-12T03:06:22.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.611+0000 7f244877b740 1 -- 172.21.15.116:0/3773244583 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614574a0b40 msgr2=0x56145749f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:22.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.611+0000 7f244877b740 1 --2- 172.21.15.116:0/3773244583 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614574a0b40 0x56145749f090 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2438009430 tx=0x7f2438031cc0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.611+0000 7f244877b740 1 -- 172.21.15.116:0/3773244583 shutdown_connections 2024-09-12T03:06:22.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.611+0000 7f244877b740 1 --2- 172.21.15.116:0/3773244583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614574aa370 0x5614574ac760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.611+0000 7f244877b740 1 --2- 172.21.15.116:0/3773244583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145749f5d0 0x56145749fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.611+0000 7f244877b740 1 --2- 172.21.15.116:0/3773244583 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614574a0b40 0x56145749f090 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.611+0000 7f244877b740 1 -- 172.21.15.116:0/3773244583 >> 172.21.15.116:0/3773244583 conn(0x561457468310 msgr2=0x56145749bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:22.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.611+0000 7f244877b740 1 -- 172.21.15.116:0/3773244583 shutdown_connections 2024-09-12T03:06:22.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.611+0000 7f244877b740 1 -- 172.21.15.116:0/3773244583 wait complete. 2024-09-12T03:06:22.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.612+0000 7f244877b740 1 -- start start 2024-09-12T03:06:22.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.612+0000 7f244877b740 1 Processor -- start 2024-09-12T03:06:22.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.612+0000 7f244877b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56145749f5d0 0x5614574a9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:22.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.612+0000 7f244877b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614574a0b40 0x5614574a7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:22.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.612+0000 7f244877b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614574aa370 0x5614574a7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:22.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.612+0000 7f244877b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5614574ad010 con 0x5614574aa370 2024-09-12T03:06:22.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.612+0000 7f244877b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5614573c0060 con 0x56145749f5d0 2024-09-12T03:06:22.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.612+0000 7f244877b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5614574ad190 con 0x5614574a0b40 2024-09-12T03:06:22.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.612+0000 7f244877b740 10 client.0 _ll_register_callbacks cb 0x5614571138c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:22.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.612+0000 7f244493f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614574aa370 0x5614574a7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:22.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.612+0000 7f244493f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614574aa370 0x5614574a7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52704/0 (socket says 172.21.15.116:52704) 2024-09-12T03:06:22.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.612+0000 7f244493f640 1 -- 172.21.15.116:0/1650757029 learned_addr learned my addr 172.21.15.116:0/1650757029 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:22.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.613+0000 7f243ffff640 1 --2- 172.21.15.116:0/1650757029 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56145749f5d0 0x5614574a9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:22.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.613+0000 7f243f7fe640 1 --2- 172.21.15.116:0/1650757029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614574a0b40 0x5614574a7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:22.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.613+0000 7f244493f640 1 -- 172.21.15.116:0/1650757029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614574a0b40 msgr2=0x5614574a7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:22.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.613+0000 7f244493f640 1 --2- 172.21.15.116:0/1650757029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614574a0b40 0x5614574a7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.613+0000 7f244493f640 1 -- 172.21.15.116:0/1650757029 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56145749f5d0 msgr2=0x5614574a9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:22.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.613+0000 7f244493f640 1 --2- 172.21.15.116:0/1650757029 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56145749f5d0 0x5614574a9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.613+0000 7f244493f640 1 -- 172.21.15.116:0/1650757029 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614571763c0 con 0x5614574aa370 2024-09-12T03:06:22.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.613+0000 7f243ffff640 1 --2- 172.21.15.116:0/1650757029 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56145749f5d0 0x5614574a9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:06:22.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.613+0000 7f244493f640 1 --2- 172.21.15.116:0/1650757029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614574aa370 0x5614574a7640 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f24380042f0 tx=0x7f2438004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f2f81a95330c7e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:22.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.613+0000 7f241f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:06:22.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.613+0000 7f241f7fe640 1 -- 172.21.15.116:0/1650757029 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438057020 con 0x5614574aa370 2024-09-12T03:06:22.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.613+0000 7f241f7fe640 1 -- 172.21.15.116:0/1650757029 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f24380307b0 con 0x5614574aa370 2024-09-12T03:06:22.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.613+0000 7f244877b740 1 -- 172.21.15.116:0/1650757029 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2438036040 con 0x5614574aa370 2024-09-12T03:06:22.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.613+0000 7f241f7fe640 1 -- 172.21.15.116:0/1650757029 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f243804e050 con 0x5614574aa370 2024-09-12T03:06:22.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.613+0000 7f244877b740 10 client.16080 fetch_fsmap learned FSMap version 25 2024-09-12T03:06:22.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.613+0000 7f244877b740 1 -- 172.21.15.116:0/1650757029 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5614574b8ae0 con 0x5614574aa370 2024-09-12T03:06:22.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.613+0000 7f241f7fe640 1 -- 172.21.15.116:0/1650757029 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2438036040 con 0x5614574aa370 2024-09-12T03:06:22.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.613+0000 7f241f7fe640 1 -- 172.21.15.116:0/1650757029 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2438030070 con 0x5614574aa370 2024-09-12T03:06:22.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.614+0000 7f244877b740 10 client.16080 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:06:22.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.614+0000 7f244877b740 10 client.16080 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:06:22.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.614+0000 7f244877b740 20 client.16080 populate_metadata read hostname 'smithi116' 2024-09-12T03:06:22.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.614+0000 7f244877b740 1 --2- 172.21.15.116:0/1650757029 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5614574b9e30 0x5614574bc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:22.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.614+0000 7f244877b740 4 client.16080 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:06:22.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.614+0000 7f244877b740 1 -- 172.21.15.116:0/1650757029 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561457412860 con 0x5614574b9e30 2024-09-12T03:06:22.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.614+0000 7f243ffff640 1 --2- 172.21.15.116:0/1650757029 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5614574b9e30 0x5614574bc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:22.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.614+0000 7f241f7fe640 10 client.16080 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:06:22.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.614+0000 7f243ffff640 1 --2- 172.21.15.116:0/1650757029 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5614574b9e30 0x5614574bc220 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e013db5b6f00b149 server_cookie=bd3df714e95bfa60 in_seq=0 out_seq=0 2024-09-12T03:06:22.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.615+0000 7f241f7fe640 1 -- 172.21.15.116:0/1650757029 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561457412860 con 0x5614574b9e30 2024-09-12T03:06:22.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.615+0000 7f241f7fe640 10 client.16080 handle_command_reply: tid=0 2024-09-12T03:06:22.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.615+0000 7f244877b740 1 client.16080 shutdown 2024-09-12T03:06:22.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.615+0000 7f244877b740 1 -- 172.21.15.116:0/1650757029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614574aa370 msgr2=0x5614574a7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:22.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.615+0000 7f244877b740 1 --2- 172.21.15.116:0/1650757029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614574aa370 0x5614574a7640 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f24380042f0 tx=0x7f2438004320 comp rx=0 tx=0).stop 2024-09-12T03:06:22.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.615+0000 7f244877b740 1 -- 172.21.15.116:0/1650757029 shutdown_connections 2024-09-12T03:06:22.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.615+0000 7f244877b740 1 --2- 172.21.15.116:0/1650757029 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5614574b9e30 0x5614574bc220 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.615+0000 7f244877b740 1 --2- 172.21.15.116:0/1650757029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614574aa370 0x5614574a7640 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.615+0000 7f244877b740 1 --2- 172.21.15.116:0/1650757029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614574a0b40 0x5614574a7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.615+0000 7f244877b740 1 --2- 172.21.15.116:0/1650757029 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56145749f5d0 0x5614574a9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.615+0000 7f244877b740 1 -- 172.21.15.116:0/1650757029 >> 172.21.15.116:0/1650757029 conn(0x561457468310 msgr2=0x5614574a1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:22.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.615+0000 7f241f7fe640 0 client.16080 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:06:22.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.615+0000 7f244877b740 1 -- 172.21.15.116:0/1650757029 shutdown_connections 2024-09-12T03:06:22.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.615+0000 7f244877b740 1 -- 172.21.15.116:0/1650757029 wait complete. 2024-09-12T03:06:22.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.616+0000 7f244877b740 20 client.16080 trim_cache size 0 max 16384 2024-09-12T03:06:22.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.628+0000 7f244877b740 1 Processor -- start 2024-09-12T03:06:22.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.628+0000 7f244877b740 1 -- start start 2024-09-12T03:06:22.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.628+0000 7f244877b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145749f5d0 0x5614574b6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:22.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.628+0000 7f244877b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614574a0b40 0x5614574a8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:22.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.628+0000 7f244877b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614574aa370 0x5614574a2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:22.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.628+0000 7f244877b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f24401b20d0 con 0x5614574a0b40 2024-09-12T03:06:22.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.628+0000 7f244877b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f24401af9a0 con 0x5614574aa370 2024-09-12T03:06:22.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.628+0000 7f244877b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f24401ad7f0 con 0x56145749f5d0 2024-09-12T03:06:22.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.628+0000 7f243f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614574a0b40 0x5614574a8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:22.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.628+0000 7f243f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614574a0b40 0x5614574a8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52718/0 (socket says 172.21.15.116:52718) 2024-09-12T03:06:22.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.628+0000 7f243f7fe640 1 -- 172.21.15.116:0/2592458566 learned_addr learned my addr 172.21.15.116:0/2592458566 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:22.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.629+0000 7f243ffff640 1 --2- 172.21.15.116:0/2592458566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145749f5d0 0x5614574b6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:22.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.629+0000 7f243f7fe640 1 -- 172.21.15.116:0/2592458566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145749f5d0 msgr2=0x5614574b6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:22.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.629+0000 7f243f7fe640 1 --2- 172.21.15.116:0/2592458566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145749f5d0 0x5614574b6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.629+0000 7f243f7fe640 1 -- 172.21.15.116:0/2592458566 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614574aa370 msgr2=0x5614574a2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:22.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.629+0000 7f243f7fe640 1 --2- 172.21.15.116:0/2592458566 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614574aa370 0x5614574a2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.629+0000 7f243f7fe640 1 -- 172.21.15.116:0/2592458566 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614574a0780 con 0x5614574a0b40 2024-09-12T03:06:22.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.629+0000 7f243ffff640 1 --2- 172.21.15.116:0/2592458566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145749f5d0 0x5614574b6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:06:22.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.629+0000 7f243f7fe640 1 --2- 172.21.15.116:0/2592458566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614574a0b40 0x5614574a8fa0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f24340176e0 tx=0x7f243401de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a720d9619ebe734e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:22.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.629+0000 7f2425ffb640 1 -- 172.21.15.116:0/2592458566 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243409ace0 con 0x5614574a0b40 2024-09-12T03:06:22.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.629+0000 7f2425ffb640 1 -- 172.21.15.116:0/2592458566 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f243409f090 con 0x5614574a0b40 2024-09-12T03:06:22.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.629+0000 7f2425ffb640 1 -- 172.21.15.116:0/2592458566 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24340a34c0 con 0x5614574a0b40 2024-09-12T03:06:22.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.630+0000 7f244877b740 1 -- 172.21.15.116:0/2592458566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614574a0b40 msgr2=0x5614574a8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:22.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.630+0000 7f244877b740 1 --2- 172.21.15.116:0/2592458566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614574a0b40 0x5614574a8fa0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f24340176e0 tx=0x7f243401de80 comp rx=0 tx=0).stop 2024-09-12T03:06:22.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.630+0000 7f244877b740 1 -- 172.21.15.116:0/2592458566 shutdown_connections 2024-09-12T03:06:22.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.630+0000 7f244877b740 1 --2- 172.21.15.116:0/2592458566 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614574aa370 0x5614574a2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.630+0000 7f244877b740 1 --2- 172.21.15.116:0/2592458566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614574a0b40 0x5614574a8fa0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.630+0000 7f244877b740 1 --2- 172.21.15.116:0/2592458566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145749f5d0 0x5614574b6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.630+0000 7f244877b740 1 -- 172.21.15.116:0/2592458566 >> 172.21.15.116:0/2592458566 conn(0x561457468310 msgr2=0x5614574a8890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:22.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.630+0000 7f244877b740 1 -- 172.21.15.116:0/2592458566 shutdown_connections 2024-09-12T03:06:22.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.630+0000 7f244877b740 1 -- 172.21.15.116:0/2592458566 wait complete. 2024-09-12T03:06:22.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.630+0000 7f244877b740 1 -- start start 2024-09-12T03:06:22.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.630+0000 7f244877b740 1 Processor -- start 2024-09-12T03:06:22.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.630+0000 7f244877b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145749f5d0 0x5614574bc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:22.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.630+0000 7f244877b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614574a0b40 0x5614574bcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:22.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.630+0000 7f244877b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614574aa370 0x56145751c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:22.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.630+0000 7f244877b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f24401ade60 con 0x5614574a0b40 2024-09-12T03:06:22.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.630+0000 7f244877b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f24401af2d0 con 0x5614574aa370 2024-09-12T03:06:22.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.630+0000 7f244877b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f24401ae530 con 0x56145749f5d0 2024-09-12T03:06:22.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.631+0000 7f244877b740 10 client.0 _ll_register_callbacks cb 0x7f2434017630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:22.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.631+0000 7f243f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614574a0b40 0x5614574bcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:22.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.631+0000 7f243f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614574a0b40 0x5614574bcb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52728/0 (socket says 172.21.15.116:52728) 2024-09-12T03:06:22.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.631+0000 7f243f7fe640 1 -- 172.21.15.116:0/3662799608 learned_addr learned my addr 172.21.15.116:0/3662799608 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:22.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.631+0000 7f243ffff640 1 --2- 172.21.15.116:0/3662799608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145749f5d0 0x5614574bc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:22.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.631+0000 7f244493f640 1 --2- 172.21.15.116:0/3662799608 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614574aa370 0x56145751c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:22.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.631+0000 7f243f7fe640 1 -- 172.21.15.116:0/3662799608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145749f5d0 msgr2=0x5614574bc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:22.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.631+0000 7f243f7fe640 1 --2- 172.21.15.116:0/3662799608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145749f5d0 0x5614574bc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.631+0000 7f243f7fe640 1 -- 172.21.15.116:0/3662799608 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614574aa370 msgr2=0x56145751c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:22.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.631+0000 7f243f7fe640 1 --2- 172.21.15.116:0/3662799608 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614574aa370 0x56145751c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.631+0000 7f243f7fe640 1 -- 172.21.15.116:0/3662799608 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56145751cd20 con 0x5614574a0b40 2024-09-12T03:06:22.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.631+0000 7f244493f640 1 --2- 172.21.15.116:0/3662799608 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614574aa370 0x56145751c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:06:22.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.631+0000 7f243f7fe640 1 --2- 172.21.15.116:0/3662799608 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614574a0b40 0x5614574bcb20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f24340cc2f0 tx=0x7f24340047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c3690cc13e690b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:22.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.631+0000 7f241f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:06:22.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.631+0000 7f241f7fe640 1 -- 172.21.15.116:0/3662799608 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24340b7020 con 0x5614574a0b40 2024-09-12T03:06:22.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.631+0000 7f241f7fe640 1 -- 172.21.15.116:0/3662799608 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f24340953b0 con 0x5614574a0b40 2024-09-12T03:06:22.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.631+0000 7f244877b740 1 -- 172.21.15.116:0/3662799608 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f243409f090 con 0x5614574a0b40 2024-09-12T03:06:22.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.631+0000 7f241f7fe640 1 -- 172.21.15.116:0/3662799608 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f24340a3950 con 0x5614574a0b40 2024-09-12T03:06:22.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.631+0000 7f244877b740 10 client.16086 fetch_fsmap learned FSMap version 25 2024-09-12T03:06:22.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.631+0000 7f244877b740 1 -- 172.21.15.116:0/3662799608 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561457523ff0 con 0x5614574a0b40 2024-09-12T03:06:22.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.631+0000 7f241f7fe640 1 -- 172.21.15.116:0/3662799608 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f243409f090 con 0x5614574a0b40 2024-09-12T03:06:22.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.632+0000 7f241f7fe640 1 -- 172.21.15.116:0/3662799608 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2434079be0 con 0x5614574a0b40 2024-09-12T03:06:22.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.632+0000 7f244877b740 10 client.16086 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:06:22.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.632+0000 7f244877b740 10 client.16086 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:06:22.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.632+0000 7f244877b740 20 client.16086 populate_metadata read hostname 'smithi116' 2024-09-12T03:06:22.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.632+0000 7f244877b740 1 --2- 172.21.15.116:0/3662799608 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5614575247c0 0x561457526be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:22.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.632+0000 7f244877b740 4 client.16086 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:06:22.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.632+0000 7f244877b740 1 -- 172.21.15.116:0/3662799608 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56145740cd70 con 0x5614575247c0 2024-09-12T03:06:22.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.632+0000 7f243ffff640 1 --2- 172.21.15.116:0/3662799608 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5614575247c0 0x561457526be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:22.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.632+0000 7f241f7fe640 10 client.16086 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:06:22.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.632+0000 7f243ffff640 1 --2- 172.21.15.116:0/3662799608 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5614575247c0 0x561457526be0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c6dadaa2f927f895 server_cookie=dd6847a15ccfccc8 in_seq=0 out_seq=0 2024-09-12T03:06:22.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.633+0000 7f241f7fe640 1 -- 172.21.15.116:0/3662799608 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56145740cd70 con 0x5614575247c0 2024-09-12T03:06:22.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.633+0000 7f241f7fe640 10 client.16086 handle_command_reply: tid=0 2024-09-12T03:06:22.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.633+0000 7f244877b740 1 client.16086 shutdown 2024-09-12T03:06:22.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.633+0000 7f244877b740 1 -- 172.21.15.116:0/3662799608 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614574a0b40 msgr2=0x5614574bcb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:22.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.633+0000 7f244877b740 1 --2- 172.21.15.116:0/3662799608 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614574a0b40 0x5614574bcb20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f24340cc2f0 tx=0x7f24340047d0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.633+0000 7f244877b740 1 -- 172.21.15.116:0/3662799608 shutdown_connections 2024-09-12T03:06:22.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.633+0000 7f244877b740 1 --2- 172.21.15.116:0/3662799608 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614574aa370 0x56145751c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.633+0000 7f244877b740 1 --2- 172.21.15.116:0/3662799608 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5614575247c0 0x561457526be0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.633+0000 7f244877b740 1 --2- 172.21.15.116:0/3662799608 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614574a0b40 0x5614574bcb20 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.633+0000 7f244877b740 1 --2- 172.21.15.116:0/3662799608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56145749f5d0 0x5614574bc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.633+0000 7f241f7fe640 0 client.16086 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:06:22.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.633+0000 7f244877b740 1 -- 172.21.15.116:0/3662799608 >> 172.21.15.116:0/3662799608 conn(0x561457468310 msgr2=0x5614574a90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:22.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.633+0000 7f244877b740 1 -- 172.21.15.116:0/3662799608 shutdown_connections 2024-09-12T03:06:22.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.633+0000 7f244877b740 1 -- 172.21.15.116:0/3662799608 wait complete. 2024-09-12T03:06:22.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.633+0000 7f244877b740 20 client.16086 trim_cache size 0 max 16384 2024-09-12T03:06:22.653 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:06:22.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.633+0000 7f24463a2640 1 -- 172.21.15.116:0/3641946239 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f24280a2b80 msgr2=0x7f24280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:22.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.633+0000 7f24463a2640 1 --2- 172.21.15.116:0/3641946239 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f24280a2b80 0x7f24280a5040 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2438009a60 tx=0x7f2438048000 comp rx=0 tx=0).stop 2024-09-12T03:06:22.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.633+0000 7f24463a2640 1 -- 172.21.15.116:0/3641946239 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2440101420 msgr2=0x7f2440102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:22.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.633+0000 7f24463a2640 1 --2- 172.21.15.116:0/3641946239 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2440101420 0x7f2440102b90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f243400b4a0 tx=0x7f243400b960 comp rx=0 tx=0).stop 2024-09-12T03:06:22.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.634+0000 7f24463a2640 1 -- 172.21.15.116:0/3641946239 shutdown_connections 2024-09-12T03:06:22.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.634+0000 7f24463a2640 1 --2- 172.21.15.116:0/3641946239 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f24280a2b80 0x7f24280a5040 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.634+0000 7f24463a2640 1 --2- 172.21.15.116:0/3641946239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2440106a80 0x7f24401030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.634+0000 7f24463a2640 1 --2- 172.21.15.116:0/3641946239 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2440101420 0x7f2440102b90 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.634+0000 7f24463a2640 1 --2- 172.21.15.116:0/3641946239 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2440100a80 0x7f2440102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:22.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.634+0000 7f24463a2640 1 -- 172.21.15.116:0/3641946239 >> 172.21.15.116:0/3641946239 conn(0x7f24400fc7a0 msgr2=0x7f2440104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:22.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.634+0000 7f24463a2640 1 -- 172.21.15.116:0/3641946239 shutdown_connections 2024-09-12T03:06:22.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:22.634+0000 7f24463a2640 1 -- 172.21.15.116:0/3641946239 wait complete. 2024-09-12T03:06:22.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:22 smithi183 ceph-mon[35535]: pgmap v439: 97 pgs: 97 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:06:22.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:22 smithi177 ceph-mon[35305]: pgmap v439: 97 pgs: 97 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:06:23.460 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:06:23.706 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:06:24.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.334+0000 7f1f86c08640 1 -- 172.21.15.116:0/3717586270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f80101240 msgr2=0x7f1f80101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:24.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.334+0000 7f1f86c08640 1 --2- 172.21.15.116:0/3717586270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f80101240 0x7f1f80101680 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009900 tx=0x7f1f6c02a410 comp rx=0 tx=0).stop 2024-09-12T03:06:24.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.334+0000 7f1f86c08640 1 -- 172.21.15.116:0/3717586270 shutdown_connections 2024-09-12T03:06:24.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.334+0000 7f1f86c08640 1 --2- 172.21.15.116:0/3717586270 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f80101bc0 0x7f1f8010d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.334+0000 7f1f86c08640 1 --2- 172.21.15.116:0/3717586270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f80101240 0x7f1f80101680 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.334+0000 7f1f86c08640 1 --2- 172.21.15.116:0/3717586270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f80105a90 0x7f1f80105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.334+0000 7f1f86c08640 1 -- 172.21.15.116:0/3717586270 >> 172.21.15.116:0/3717586270 conn(0x7f1f80077210 msgr2=0x7f1f800fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:24.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.334+0000 7f1f86c08640 1 -- 172.21.15.116:0/3717586270 shutdown_connections 2024-09-12T03:06:24.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.334+0000 7f1f86c08640 1 -- 172.21.15.116:0/3717586270 wait complete. 2024-09-12T03:06:24.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.335+0000 7f1f86c08640 1 Processor -- start 2024-09-12T03:06:24.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.335+0000 7f1f86c08640 1 -- start start 2024-09-12T03:06:24.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.335+0000 7f1f86c08640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f80101240 0x7f1f801974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:24.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.335+0000 7f1f86c08640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f80101bc0 0x7f1f80197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:24.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.335+0000 7f1f86c08640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f80105a90 0x7f1f8019bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:24.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.335+0000 7f1f86c08640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1f80111030 con 0x7f1f80101bc0 2024-09-12T03:06:24.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.335+0000 7f1f86c08640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1f80110eb0 con 0x7f1f80105a90 2024-09-12T03:06:24.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.335+0000 7f1f86c08640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1f801111b0 con 0x7f1f80101240 2024-09-12T03:06:24.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.335+0000 7f1f7ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f80101bc0 0x7f1f80197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:24.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.335+0000 7f1f7ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f80101bc0 0x7f1f80197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52752/0 (socket says 172.21.15.116:52752) 2024-09-12T03:06:24.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.335+0000 7f1f7ffff640 1 -- 172.21.15.116:0/3763910218 learned_addr learned my addr 172.21.15.116:0/3763910218 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:24.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.335+0000 7f1f849a4640 1 --2- 172.21.15.116:0/3763910218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f80101240 0x7f1f801974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:24.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.335+0000 7f1f851a5640 1 --2- 172.21.15.116:0/3763910218 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f80105a90 0x7f1f8019bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:24.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.336+0000 7f1f7ffff640 1 -- 172.21.15.116:0/3763910218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f80101240 msgr2=0x7f1f801974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:24.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.336+0000 7f1f7ffff640 1 --2- 172.21.15.116:0/3763910218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f80101240 0x7f1f801974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.336+0000 7f1f7ffff640 1 -- 172.21.15.116:0/3763910218 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f80105a90 msgr2=0x7f1f8019bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:24.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.336+0000 7f1f7ffff640 1 --2- 172.21.15.116:0/3763910218 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f80105a90 0x7f1f8019bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.336+0000 7f1f7ffff640 1 -- 172.21.15.116:0/3763910218 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8019c4a0 con 0x7f1f80101bc0 2024-09-12T03:06:24.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.336+0000 7f1f851a5640 1 --2- 172.21.15.116:0/3763910218 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f80105a90 0x7f1f8019bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:06:24.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.336+0000 7f1f7ffff640 1 --2- 172.21.15.116:0/3763910218 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f80101bc0 0x7f1f80197a30 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c006010 tx=0x7f1f6c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:24.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.336+0000 7f1f7dffb640 1 -- 172.21.15.116:0/3763910218 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c03f070 con 0x7f1f80101bc0 2024-09-12T03:06:24.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.336+0000 7f1f86c08640 1 -- 172.21.15.116:0/3763910218 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f8019c730 con 0x7f1f80101bc0 2024-09-12T03:06:24.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.336+0000 7f1f7dffb640 1 -- 172.21.15.116:0/3763910218 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f6c031e30 con 0x7f1f80101bc0 2024-09-12T03:06:24.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.336+0000 7f1f7dffb640 1 -- 172.21.15.116:0/3763910218 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c035030 con 0x7f1f80101bc0 2024-09-12T03:06:24.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.336+0000 7f1f86c08640 1 -- 172.21.15.116:0/3763910218 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f801a4070 con 0x7f1f80101bc0 2024-09-12T03:06:24.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.338+0000 7f1f7dffb640 1 -- 172.21.15.116:0/3763910218 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1f6c042070 con 0x7f1f80101bc0 2024-09-12T03:06:24.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.339+0000 7f1f88fe1740 1 Processor -- start 2024-09-12T03:06:24.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.339+0000 7f1f88fe1740 1 -- start start 2024-09-12T03:06:24.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.339+0000 7f1f88fe1740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab3747cd50 0x55ab3747b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:24.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.339+0000 7f1f88fe1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab3747b7e0 0x55ab3747bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:24.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.339+0000 7f1f88fe1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab37486580 0x55ab37488970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:24.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.339+0000 7f1f88fe1740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ab373df2f0 con 0x55ab3747cd50 2024-09-12T03:06:24.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.339+0000 7f1f88fe1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ab37386cf0 con 0x55ab37486580 2024-09-12T03:06:24.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.339+0000 7f1f88fe1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ab373db2b0 con 0x55ab3747b7e0 2024-09-12T03:06:24.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.339+0000 7f1f7dffb640 1 --2- 172.21.15.116:0/3763910218 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1f700a2b80 0x7f1f700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:24.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.339+0000 7f1f851a5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab3747cd50 0x55ab3747b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:24.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.339+0000 7f1f851a5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab3747cd50 0x55ab3747b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52768/0 (socket says 172.21.15.116:52768) 2024-09-12T03:06:24.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.339+0000 7f1f851a5640 1 -- 172.21.15.116:0/2571212147 learned_addr learned my addr 172.21.15.116:0/2571212147 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:24.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.339+0000 7f1f849a4640 1 --2- 172.21.15.116:0/2571212147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab3747b7e0 0x55ab3747bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:24.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.339+0000 7f1f7dffb640 1 -- 172.21.15.116:0/3763910218 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1f6c0efa10 con 0x7f1f80101bc0 2024-09-12T03:06:24.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.339+0000 7f1f7ffff640 1 --2- 172.21.15.116:0/2571212147 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab37486580 0x55ab37488970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:24.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.339+0000 7f1f851a5640 1 --2- 172.21.15.116:0/3763910218 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1f700a2b80 0x7f1f700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:24.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.339+0000 7f1f849a4640 1 -- 172.21.15.116:0/2571212147 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab37486580 msgr2=0x55ab37488970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:24.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.339+0000 7f1f849a4640 1 --2- 172.21.15.116:0/2571212147 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab37486580 0x55ab37488970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.340+0000 7f1f849a4640 1 -- 172.21.15.116:0/2571212147 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab3747cd50 msgr2=0x55ab3747b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:24.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.340+0000 7f1f849a4640 1 --2- 172.21.15.116:0/2571212147 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab3747cd50 0x55ab3747b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.340+0000 7f1f849a4640 1 -- 172.21.15.116:0/2571212147 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab3736b3a0 con 0x55ab3747b7e0 2024-09-12T03:06:24.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.340+0000 7f1f851a5640 1 --2- 172.21.15.116:0/2571212147 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab3747cd50 0x55ab3747b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:06:24.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.340+0000 7f1f849a4640 1 --2- 172.21.15.116:0/2571212147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab3747b7e0 0x55ab3747bc20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1f7400b760 tx=0x7f1f7400bc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ad8ca04320d1d2e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:24.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.340+0000 7f1f851a5640 1 --2- 172.21.15.116:0/3763910218 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1f700a2b80 0x7f1f700a5040 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1f78010af0 tx=0x7f1f78004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:24.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.340+0000 7f1f6affd640 1 -- 172.21.15.116:0/2571212147 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7400d950 con 0x55ab3747b7e0 2024-09-12T03:06:24.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.340+0000 7f1f6affd640 1 -- 172.21.15.116:0/2571212147 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f74019070 con 0x55ab3747b7e0 2024-09-12T03:06:24.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.340+0000 7f1f6affd640 1 -- 172.21.15.116:0/2571212147 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f74014e00 con 0x55ab3747b7e0 2024-09-12T03:06:24.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.341+0000 7f1f88fe1740 1 -- 172.21.15.116:0/2571212147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab3747b7e0 msgr2=0x55ab3747bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:24.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.341+0000 7f1f88fe1740 1 --2- 172.21.15.116:0/2571212147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab3747b7e0 0x55ab3747bc20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1f7400b760 tx=0x7f1f7400bc20 comp rx=0 tx=0).stop 2024-09-12T03:06:24.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.341+0000 7f1f88fe1740 1 -- 172.21.15.116:0/2571212147 shutdown_connections 2024-09-12T03:06:24.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.341+0000 7f1f88fe1740 1 --2- 172.21.15.116:0/2571212147 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab37486580 0x55ab37488970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.341+0000 7f1f88fe1740 1 --2- 172.21.15.116:0/2571212147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab3747b7e0 0x55ab3747bc20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.341+0000 7f1f88fe1740 1 --2- 172.21.15.116:0/2571212147 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab3747cd50 0x55ab3747b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.341+0000 7f1f88fe1740 1 -- 172.21.15.116:0/2571212147 >> 172.21.15.116:0/2571212147 conn(0x55ab37442310 msgr2=0x55ab37477e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:24.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.341+0000 7f1f88fe1740 1 -- 172.21.15.116:0/2571212147 shutdown_connections 2024-09-12T03:06:24.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.341+0000 7f1f88fe1740 1 -- 172.21.15.116:0/2571212147 wait complete. 2024-09-12T03:06:24.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.341+0000 7f1f88fe1740 1 -- start start 2024-09-12T03:06:24.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.341+0000 7f1f88fe1740 1 Processor -- start 2024-09-12T03:06:24.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.342+0000 7f1f88fe1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab3747b7e0 0x55ab37485ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:24.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.342+0000 7f1f88fe1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab3747cd50 0x55ab374831a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:24.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.342+0000 7f1f88fe1740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab37486580 0x55ab37483700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:24.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.342+0000 7f1f88fe1740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ab37489280 con 0x55ab37486580 2024-09-12T03:06:24.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.342+0000 7f1f88fe1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ab3739a060 con 0x55ab3747b7e0 2024-09-12T03:06:24.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.342+0000 7f1f88fe1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ab37489400 con 0x55ab3747cd50 2024-09-12T03:06:24.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.342+0000 7f1f88fe1740 10 client.0 _ll_register_callbacks cb 0x55ab370ed8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:24.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.342+0000 7f1f851a5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab37486580 0x55ab37483700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:24.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.342+0000 7f1f851a5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab37486580 0x55ab37483700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52774/0 (socket says 172.21.15.116:52774) 2024-09-12T03:06:24.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.342+0000 7f1f851a5640 1 -- 172.21.15.116:0/1023234904 learned_addr learned my addr 172.21.15.116:0/1023234904 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:24.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.342+0000 7f1f7ffff640 1 --2- 172.21.15.116:0/1023234904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab3747cd50 0x55ab374831a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:24.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.342+0000 7f1f849a4640 1 --2- 172.21.15.116:0/1023234904 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab3747b7e0 0x55ab37485ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:24.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.342+0000 7f1f851a5640 1 -- 172.21.15.116:0/1023234904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab3747cd50 msgr2=0x55ab374831a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:24.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.342+0000 7f1f851a5640 1 --2- 172.21.15.116:0/1023234904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab3747cd50 0x55ab374831a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.342+0000 7f1f851a5640 1 -- 172.21.15.116:0/1023234904 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab3747b7e0 msgr2=0x55ab37485ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:24.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.342+0000 7f1f851a5640 1 --2- 172.21.15.116:0/1023234904 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab3747b7e0 0x55ab37485ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.342+0000 7f1f851a5640 1 -- 172.21.15.116:0/1023234904 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab371503c0 con 0x55ab37486580 2024-09-12T03:06:24.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.342+0000 7f1f851a5640 1 --2- 172.21.15.116:0/1023234904 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab37486580 0x55ab37483700 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1f78013cd0 tx=0x7f1f78004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af7a9dfdf84d669c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:24.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.342+0000 7f1f5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:06:24.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.342+0000 7f1f5ffff640 1 -- 172.21.15.116:0/1023234904 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78018070 con 0x55ab37486580 2024-09-12T03:06:24.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.342+0000 7f1f88fe1740 1 -- 172.21.15.116:0/1023234904 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f74019070 con 0x55ab37486580 2024-09-12T03:06:24.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.343+0000 7f1f5ffff640 1 -- 172.21.15.116:0/1023234904 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1f7801f670 con 0x55ab37486580 2024-09-12T03:06:24.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.343+0000 7f1f5ffff640 1 -- 172.21.15.116:0/1023234904 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1f7801f870 con 0x55ab37486580 2024-09-12T03:06:24.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.343+0000 7f1f88fe1740 10 client.16098 fetch_fsmap learned FSMap version 25 2024-09-12T03:06:24.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.343+0000 7f1f88fe1740 1 -- 172.21.15.116:0/1023234904 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ab37494d40 con 0x55ab37486580 2024-09-12T03:06:24.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.343+0000 7f1f5ffff640 1 -- 172.21.15.116:0/1023234904 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f74019070 con 0x55ab37486580 2024-09-12T03:06:24.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.343+0000 7f1f5ffff640 1 -- 172.21.15.116:0/1023234904 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1f78028ab0 con 0x55ab37486580 2024-09-12T03:06:24.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.343+0000 7f1f88fe1740 10 client.16098 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:06:24.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.343+0000 7f1f88fe1740 10 client.16098 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:06:24.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.343+0000 7f1f88fe1740 20 client.16098 populate_metadata read hostname 'smithi116' 2024-09-12T03:06:24.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.343+0000 7f1f88fe1740 1 --2- 172.21.15.116:0/1023234904 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55ab37496150 0x55ab374985d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:24.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.343+0000 7f1f88fe1740 4 client.16098 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:06:24.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.343+0000 7f1f88fe1740 1 -- 172.21.15.116:0/1023234904 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab373ec860 con 0x55ab37496150 2024-09-12T03:06:24.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.343+0000 7f1f849a4640 1 --2- 172.21.15.116:0/1023234904 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55ab37496150 0x55ab374985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:24.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.344+0000 7f1f5ffff640 10 client.16098 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:06:24.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.344+0000 7f1f849a4640 1 --2- 172.21.15.116:0/1023234904 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55ab37496150 0x55ab374985d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8a396b555687dee5 server_cookie=189a60c29c23da9c in_seq=0 out_seq=0 2024-09-12T03:06:24.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.344+0000 7f1f5ffff640 1 -- 172.21.15.116:0/1023234904 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ab373ec860 con 0x55ab37496150 2024-09-12T03:06:24.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.344+0000 7f1f5ffff640 10 client.16098 handle_command_reply: tid=0 2024-09-12T03:06:24.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.344+0000 7f1f88fe1740 1 client.16098 shutdown 2024-09-12T03:06:24.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.345+0000 7f1f88fe1740 1 -- 172.21.15.116:0/1023234904 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab37486580 msgr2=0x55ab37483700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:24.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.345+0000 7f1f88fe1740 1 --2- 172.21.15.116:0/1023234904 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab37486580 0x55ab37483700 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1f78013cd0 tx=0x7f1f78004b50 comp rx=0 tx=0).stop 2024-09-12T03:06:24.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.345+0000 7f1f88fe1740 1 -- 172.21.15.116:0/1023234904 shutdown_connections 2024-09-12T03:06:24.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.345+0000 7f1f88fe1740 1 --2- 172.21.15.116:0/1023234904 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55ab37496150 0x55ab374985d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.345+0000 7f1f88fe1740 1 --2- 172.21.15.116:0/1023234904 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab37486580 0x55ab37483700 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.345+0000 7f1f88fe1740 1 --2- 172.21.15.116:0/1023234904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab3747cd50 0x55ab374831a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.345+0000 7f1f88fe1740 1 --2- 172.21.15.116:0/1023234904 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab3747b7e0 0x55ab37485ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.345+0000 7f1f5ffff640 0 client.16098 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:06:24.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.345+0000 7f1f88fe1740 1 -- 172.21.15.116:0/1023234904 >> 172.21.15.116:0/1023234904 conn(0x55ab37442310 msgr2=0x55ab3747db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:24.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.345+0000 7f1f88fe1740 1 -- 172.21.15.116:0/1023234904 shutdown_connections 2024-09-12T03:06:24.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.345+0000 7f1f88fe1740 1 -- 172.21.15.116:0/1023234904 wait complete. 2024-09-12T03:06:24.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.345+0000 7f1f88fe1740 20 client.16098 trim_cache size 0 max 16384 2024-09-12T03:06:24.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.358+0000 7f1f88fe1740 1 Processor -- start 2024-09-12T03:06:24.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.358+0000 7f1f88fe1740 1 -- start start 2024-09-12T03:06:24.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.358+0000 7f1f88fe1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab3747b7e0 0x55ab37492520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:24.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.358+0000 7f1f88fe1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab3747cd50 0x55ab3747e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:24.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.358+0000 7f1f88fe1740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab37486580 0x55ab3747e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:24.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.358+0000 7f1f88fe1740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1f801adc80 con 0x55ab37486580 2024-09-12T03:06:24.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.358+0000 7f1f88fe1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1f801ab550 con 0x55ab3747cd50 2024-09-12T03:06:24.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.358+0000 7f1f88fe1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1f801a93c0 con 0x55ab3747b7e0 2024-09-12T03:06:24.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.358+0000 7f1f851a5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab37486580 0x55ab3747e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:24.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.358+0000 7f1f851a5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab37486580 0x55ab3747e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52782/0 (socket says 172.21.15.116:52782) 2024-09-12T03:06:24.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.358+0000 7f1f851a5640 1 -- 172.21.15.116:0/110182939 learned_addr learned my addr 172.21.15.116:0/110182939 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:24.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.358+0000 7f1f849a4640 1 --2- 172.21.15.116:0/110182939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab3747b7e0 0x55ab37492520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:24.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.358+0000 7f1f851a5640 1 -- 172.21.15.116:0/110182939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab3747b7e0 msgr2=0x55ab37492520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:24.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.358+0000 7f1f851a5640 1 --2- 172.21.15.116:0/110182939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab3747b7e0 0x55ab37492520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.358+0000 7f1f851a5640 1 -- 172.21.15.116:0/110182939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab3747cd50 msgr2=0x55ab3747e4a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:06:24.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.358+0000 7f1f851a5640 1 --2- 172.21.15.116:0/110182939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab3747cd50 0x55ab3747e4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.358+0000 7f1f851a5640 1 -- 172.21.15.116:0/110182939 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab3747c740 con 0x55ab37486580 2024-09-12T03:06:24.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.359+0000 7f1f851a5640 1 --2- 172.21.15.116:0/110182939 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab37486580 0x55ab3747e9e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1f78029000 tx=0x7f1f780109b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d15dc8a786203e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:24.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.359+0000 7f1f6a7fc640 1 -- 172.21.15.116:0/110182939 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78038070 con 0x55ab37486580 2024-09-12T03:06:24.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.359+0000 7f1f6a7fc640 1 -- 172.21.15.116:0/110182939 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f78024070 con 0x55ab37486580 2024-09-12T03:06:24.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.359+0000 7f1f6a7fc640 1 -- 172.21.15.116:0/110182939 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78018050 con 0x55ab37486580 2024-09-12T03:06:24.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.359+0000 7f1f88fe1740 1 -- 172.21.15.116:0/110182939 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab37486580 msgr2=0x55ab3747e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:24.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.359+0000 7f1f88fe1740 1 --2- 172.21.15.116:0/110182939 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab37486580 0x55ab3747e9e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1f78029000 tx=0x7f1f780109b0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.359+0000 7f1f88fe1740 1 -- 172.21.15.116:0/110182939 shutdown_connections 2024-09-12T03:06:24.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.359+0000 7f1f88fe1740 1 --2- 172.21.15.116:0/110182939 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab37486580 0x55ab3747e9e0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.359+0000 7f1f88fe1740 1 --2- 172.21.15.116:0/110182939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab3747cd50 0x55ab3747e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.359+0000 7f1f88fe1740 1 --2- 172.21.15.116:0/110182939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab3747b7e0 0x55ab37492520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.359+0000 7f1f88fe1740 1 -- 172.21.15.116:0/110182939 >> 172.21.15.116:0/110182939 conn(0x55ab37442310 msgr2=0x55ab37483160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:24.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.359+0000 7f1f88fe1740 1 -- 172.21.15.116:0/110182939 shutdown_connections 2024-09-12T03:06:24.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.359+0000 7f1f88fe1740 1 -- 172.21.15.116:0/110182939 wait complete. 2024-09-12T03:06:24.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.360+0000 7f1f88fe1740 1 -- start start 2024-09-12T03:06:24.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.360+0000 7f1f88fe1740 1 Processor -- start 2024-09-12T03:06:24.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.360+0000 7f1f88fe1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab3747b7e0 0x55ab3747fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:24.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.360+0000 7f1f88fe1740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab3747cd50 0x55ab374800d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:24.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.360+0000 7f1f88fe1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab37486580 0x55ab3748d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:24.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.360+0000 7f1f88fe1740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1f801a9a10 con 0x55ab3747cd50 2024-09-12T03:06:24.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.360+0000 7f1f88fe1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1f801aae80 con 0x55ab37486580 2024-09-12T03:06:24.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.360+0000 7f1f88fe1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1f801aa0e0 con 0x55ab3747b7e0 2024-09-12T03:06:24.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.360+0000 7f1f88fe1740 10 client.0 _ll_register_callbacks cb 0x7f1f78007be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:24.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.360+0000 7f1f7ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab3747cd50 0x55ab374800d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:24.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.360+0000 7f1f7ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab3747cd50 0x55ab374800d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52796/0 (socket says 172.21.15.116:52796) 2024-09-12T03:06:24.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.360+0000 7f1f7ffff640 1 -- 172.21.15.116:0/741878475 learned_addr learned my addr 172.21.15.116:0/741878475 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:24.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.361+0000 7f1f7ffff640 1 -- 172.21.15.116:0/741878475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab3747b7e0 msgr2=0x55ab3747fb90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:06:24.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.361+0000 7f1f7ffff640 1 --2- 172.21.15.116:0/741878475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab3747b7e0 0x55ab3747fb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.361+0000 7f1f7ffff640 1 -- 172.21.15.116:0/741878475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab37486580 msgr2=0x55ab3748d690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:06:24.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.361+0000 7f1f7ffff640 1 --2- 172.21.15.116:0/741878475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab37486580 0x55ab3748d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.361+0000 7f1f7ffff640 1 -- 172.21.15.116:0/741878475 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab3748dc80 con 0x55ab3747cd50 2024-09-12T03:06:24.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.361+0000 7f1f7ffff640 1 --2- 172.21.15.116:0/741878475 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab3747cd50 0x55ab374800d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c0ed540 tx=0x7f1f6c0bc070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=579e569b5a17fbec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:24.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.361+0000 7f1f5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:06:24.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.361+0000 7f1f5ffff640 1 -- 172.21.15.116:0/741878475 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c0d5060 con 0x55ab3747cd50 2024-09-12T03:06:24.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.361+0000 7f1f5ffff640 1 -- 172.21.15.116:0/741878475 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1f6c0b8910 con 0x55ab3747cd50 2024-09-12T03:06:24.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.361+0000 7f1f88fe1740 1 -- 172.21.15.116:0/741878475 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f78024070 con 0x55ab3747cd50 2024-09-12T03:06:24.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.361+0000 7f1f5ffff640 1 -- 172.21.15.116:0/741878475 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1f6c049040 con 0x55ab3747cd50 2024-09-12T03:06:24.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.361+0000 7f1f88fe1740 10 client.16104 fetch_fsmap learned FSMap version 25 2024-09-12T03:06:24.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.361+0000 7f1f88fe1740 1 -- 172.21.15.116:0/741878475 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ab3748efe0 con 0x55ab3747cd50 2024-09-12T03:06:24.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.361+0000 7f1f5ffff640 1 -- 172.21.15.116:0/741878475 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f78024070 con 0x55ab3747cd50 2024-09-12T03:06:24.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.362+0000 7f1f5ffff640 1 -- 172.21.15.116:0/741878475 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1f6c03b5f0 con 0x55ab3747cd50 2024-09-12T03:06:24.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.362+0000 7f1f88fe1740 10 client.16104 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:06:24.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.362+0000 7f1f88fe1740 10 client.16104 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:06:24.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.362+0000 7f1f88fe1740 20 client.16104 populate_metadata read hostname 'smithi116' 2024-09-12T03:06:24.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.362+0000 7f1f88fe1740 1 --2- 172.21.15.116:0/741878475 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55ab37475890 0x55ab37475ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:24.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.362+0000 7f1f88fe1740 4 client.16104 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:06:24.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.362+0000 7f1f88fe1740 1 -- 172.21.15.116:0/741878475 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ab373e6d70 con 0x55ab37475890 2024-09-12T03:06:24.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.363+0000 7f1f849a4640 1 --2- 172.21.15.116:0/741878475 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55ab37475890 0x55ab37475ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:24.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.366+0000 7f1f849a4640 1 --2- 172.21.15.116:0/741878475 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55ab37475890 0x55ab37475ca0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5479779b935282b2 server_cookie=f7ff19c4b9f11277 in_seq=0 out_seq=0 2024-09-12T03:06:24.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.366+0000 7f1f5ffff640 10 client.16104 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:06:24.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.367+0000 7f1f5ffff640 1 -- 172.21.15.116:0/741878475 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ab373e6d70 con 0x55ab37475890 2024-09-12T03:06:24.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.367+0000 7f1f5ffff640 10 client.16104 handle_command_reply: tid=0 2024-09-12T03:06:24.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.367+0000 7f1f88fe1740 1 client.16104 shutdown 2024-09-12T03:06:24.372 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:24 smithi116 ceph-mon[31410]: pgmap v440: 97 pgs: 97 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:06:24.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.367+0000 7f1f88fe1740 1 -- 172.21.15.116:0/741878475 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab3747cd50 msgr2=0x55ab374800d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:24.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.367+0000 7f1f88fe1740 1 --2- 172.21.15.116:0/741878475 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab3747cd50 0x55ab374800d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c0ed540 tx=0x7f1f6c0bc070 comp rx=0 tx=0).stop 2024-09-12T03:06:24.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.368+0000 7f1f88fe1740 1 -- 172.21.15.116:0/741878475 shutdown_connections 2024-09-12T03:06:24.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.368+0000 7f1f88fe1740 1 --2- 172.21.15.116:0/741878475 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55ab37475890 0x55ab37475ca0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.368+0000 7f1f88fe1740 1 --2- 172.21.15.116:0/741878475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab37486580 0x55ab3748d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.368+0000 7f1f5ffff640 0 client.16104 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:06:24.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.368+0000 7f1f88fe1740 1 --2- 172.21.15.116:0/741878475 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab3747cd50 0x55ab374800d0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.368+0000 7f1f88fe1740 1 --2- 172.21.15.116:0/741878475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab3747b7e0 0x55ab3747fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.368+0000 7f1f88fe1740 1 -- 172.21.15.116:0/741878475 >> 172.21.15.116:0/741878475 conn(0x55ab37442310 msgr2=0x55ab374819f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:24.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.368+0000 7f1f88fe1740 1 -- 172.21.15.116:0/741878475 shutdown_connections 2024-09-12T03:06:24.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.368+0000 7f1f88fe1740 1 -- 172.21.15.116:0/741878475 wait complete. 2024-09-12T03:06:24.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.368+0000 7f1f88fe1740 20 client.16104 trim_cache size 0 max 16384 2024-09-12T03:06:24.375 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:06:24.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.368+0000 7f1f86c08640 1 -- 172.21.15.116:0/3763910218 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1f700a2b80 msgr2=0x7f1f700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:24.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.368+0000 7f1f86c08640 1 --2- 172.21.15.116:0/3763910218 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1f700a2b80 0x7f1f700a5040 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1f78010af0 tx=0x7f1f78004a40 comp rx=0 tx=0).stop 2024-09-12T03:06:24.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.368+0000 7f1f86c08640 1 -- 172.21.15.116:0/3763910218 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f80101bc0 msgr2=0x7f1f80197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:24.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.368+0000 7f1f86c08640 1 --2- 172.21.15.116:0/3763910218 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f80101bc0 0x7f1f80197a30 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c006010 tx=0x7f1f6c003c80 comp rx=0 tx=0).stop 2024-09-12T03:06:24.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.369+0000 7f1f86c08640 1 -- 172.21.15.116:0/3763910218 shutdown_connections 2024-09-12T03:06:24.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.369+0000 7f1f86c08640 1 --2- 172.21.15.116:0/3763910218 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1f700a2b80 0x7f1f700a5040 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.369+0000 7f1f86c08640 1 --2- 172.21.15.116:0/3763910218 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f80105a90 0x7f1f8019bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.369+0000 7f1f86c08640 1 --2- 172.21.15.116:0/3763910218 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f80101bc0 0x7f1f80197a30 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.369+0000 7f1f86c08640 1 --2- 172.21.15.116:0/3763910218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f80101240 0x7f1f801974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:24.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.369+0000 7f1f86c08640 1 -- 172.21.15.116:0/3763910218 >> 172.21.15.116:0/3763910218 conn(0x7f1f80077210 msgr2=0x7f1f80103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:24.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.369+0000 7f1f86c08640 1 -- 172.21.15.116:0/3763910218 shutdown_connections 2024-09-12T03:06:24.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:24.369+0000 7f1f86c08640 1 -- 172.21.15.116:0/3763910218 wait complete. 2024-09-12T03:06:24.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:24 smithi183 ceph-mon[35535]: pgmap v440: 97 pgs: 97 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:06:24.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:24 smithi177 ceph-mon[35305]: pgmap v440: 97 pgs: 97 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:06:25.196 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:06:25.446 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:06:25.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.959+0000 7f3af69d9640 1 -- 172.21.15.116:0/575803072 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3af0100a80 msgr2=0x7f3af0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:25.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.959+0000 7f3af69d9640 1 --2- 172.21.15.116:0/575803072 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3af0100a80 0x7f3af0100ee0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4009930 tx=0x7f3ae402a420 comp rx=0 tx=0).stop 2024-09-12T03:06:25.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.959+0000 7f3af69d9640 1 -- 172.21.15.116:0/575803072 shutdown_connections 2024-09-12T03:06:25.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.959+0000 7f3af69d9640 1 --2- 172.21.15.116:0/575803072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3af0101420 0x7f3af010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:25.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.959+0000 7f3af69d9640 1 --2- 172.21.15.116:0/575803072 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3af0100a80 0x7f3af0100ee0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:25.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.959+0000 7f3af69d9640 1 --2- 172.21.15.116:0/575803072 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3af0106a80 0x7f3af0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:25.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.959+0000 7f3af69d9640 1 -- 172.21.15.116:0/575803072 >> 172.21.15.116:0/575803072 conn(0x7f3af00fc7a0 msgr2=0x7f3af00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:25.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.959+0000 7f3af69d9640 1 -- 172.21.15.116:0/575803072 shutdown_connections 2024-09-12T03:06:25.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.959+0000 7f3af69d9640 1 -- 172.21.15.116:0/575803072 wait complete. 2024-09-12T03:06:25.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.960+0000 7f3af69d9640 1 Processor -- start 2024-09-12T03:06:25.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.960+0000 7f3af69d9640 1 -- start start 2024-09-12T03:06:25.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.960+0000 7f3af69d9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3af0100a80 0x7f3af010e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:25.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.960+0000 7f3af69d9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3af0101420 0x7f3af010e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:25.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.960+0000 7f3af69d9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3af0106a80 0x7f3af010ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:25.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.960+0000 7f3af69d9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3af0111fa0 con 0x7f3af0106a80 2024-09-12T03:06:25.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.960+0000 7f3af69d9640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3af0111e20 con 0x7f3af0101420 2024-09-12T03:06:25.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.960+0000 7f3af69d9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3af0112120 con 0x7f3af0100a80 2024-09-12T03:06:25.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.960+0000 7f3af4f76640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3af0106a80 0x7f3af010ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:25.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.960+0000 7f3af4f76640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3af0106a80 0x7f3af010ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52814/0 (socket says 172.21.15.116:52814) 2024-09-12T03:06:25.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.960+0000 7f3af4f76640 1 -- 172.21.15.116:0/3071144571 learned_addr learned my addr 172.21.15.116:0/3071144571 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:25.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.960+0000 7f3aef7fe640 1 --2- 172.21.15.116:0/3071144571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3af0101420 0x7f3af010e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:25.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.960+0000 7f3aeffff640 1 --2- 172.21.15.116:0/3071144571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3af0100a80 0x7f3af010e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:25.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.960+0000 7f3af4f76640 1 -- 172.21.15.116:0/3071144571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3af0100a80 msgr2=0x7f3af010e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:25.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.960+0000 7f3af4f76640 1 --2- 172.21.15.116:0/3071144571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3af0100a80 0x7f3af010e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:25.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.960+0000 7f3af4f76640 1 -- 172.21.15.116:0/3071144571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3af0101420 msgr2=0x7f3af010e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:25.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.960+0000 7f3af4f76640 1 --2- 172.21.15.116:0/3071144571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3af0101420 0x7f3af010e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:25.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.960+0000 7f3af4f76640 1 -- 172.21.15.116:0/3071144571 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af01a6930 con 0x7f3af0106a80 2024-09-12T03:06:25.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.961+0000 7f3af4f76640 1 --2- 172.21.15.116:0/3071144571 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3af0106a80 0x7f3af010ee60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800eaa0 tx=0x7f3ae800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:25.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.961+0000 7f3aed7fa640 1 -- 172.21.15.116:0/3071144571 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae800cdb0 con 0x7f3af0106a80 2024-09-12T03:06:25.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.961+0000 7f3af69d9640 1 -- 172.21.15.116:0/3071144571 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3af01a6c20 con 0x7f3af0106a80 2024-09-12T03:06:25.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.961+0000 7f3aed7fa640 1 -- 172.21.15.116:0/3071144571 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae8005020 con 0x7f3af0106a80 2024-09-12T03:06:25.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.961+0000 7f3aed7fa640 1 -- 172.21.15.116:0/3071144571 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae80116b0 con 0x7f3af0106a80 2024-09-12T03:06:25.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.961+0000 7f3af69d9640 1 -- 172.21.15.116:0/3071144571 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3af01a7160 con 0x7f3af0106a80 2024-09-12T03:06:25.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.963+0000 7f3aed7fa640 1 -- 172.21.15.116:0/3071144571 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3ae8005560 con 0x7f3af0106a80 2024-09-12T03:06:25.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.963+0000 7f3af8db2740 1 Processor -- start 2024-09-12T03:06:25.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.963+0000 7f3af8db2740 1 -- start start 2024-09-12T03:06:25.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.964+0000 7f3aed7fa640 1 --2- 172.21.15.116:0/3071144571 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3ad80a2b80 0x7f3ad80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:25.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.964+0000 7f3af8db2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee89d50b40 0x55ee89d4f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:25.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.964+0000 7f3af8db2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee89d4f5d0 0x55ee89d4fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:25.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.964+0000 7f3af8db2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee89d5a370 0x55ee89d5c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:25.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.964+0000 7f3af8db2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ee89cb52f0 con 0x55ee89d50b40 2024-09-12T03:06:25.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.964+0000 7f3af8db2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ee89c5ccf0 con 0x55ee89d5a370 2024-09-12T03:06:25.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.964+0000 7f3af8db2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ee89cb12b0 con 0x55ee89d4f5d0 2024-09-12T03:06:25.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.964+0000 7f3aef7fe640 1 --2- 172.21.15.116:0/3071144571 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3ad80a2b80 0x7f3ad80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:25.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.964+0000 7f3aeffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee89d50b40 0x55ee89d4f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:25.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.964+0000 7f3aeffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee89d50b40 0x55ee89d4f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52820/0 (socket says 172.21.15.116:52820) 2024-09-12T03:06:25.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.964+0000 7f3aeffff640 1 -- 172.21.15.116:0/2927614871 learned_addr learned my addr 172.21.15.116:0/2927614871 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:25.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.964+0000 7f3aed7fa640 1 -- 172.21.15.116:0/3071144571 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f3ae80cf310 con 0x7f3af0106a80 2024-09-12T03:06:25.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.964+0000 7f3aef7fe640 1 --2- 172.21.15.116:0/2927614871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee89d4f5d0 0x55ee89d4fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:25.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.964+0000 7f3af4f76640 1 --2- 172.21.15.116:0/2927614871 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee89d5a370 0x55ee89d5c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:25.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.964+0000 7f3aeffff640 1 -- 172.21.15.116:0/2927614871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee89d4f5d0 msgr2=0x55ee89d4fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:25.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.964+0000 7f3aeffff640 1 --2- 172.21.15.116:0/2927614871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee89d4f5d0 0x55ee89d4fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:25.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.964+0000 7f3aeffff640 1 -- 172.21.15.116:0/2927614871 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee89d5a370 msgr2=0x55ee89d5c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:25.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.964+0000 7f3aeffff640 1 --2- 172.21.15.116:0/2927614871 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee89d5a370 0x55ee89d5c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:25.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.964+0000 7f3aeffff640 1 -- 172.21.15.116:0/2927614871 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee89c413a0 con 0x55ee89d50b40 2024-09-12T03:06:25.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.964+0000 7f3aeffff640 1 --2- 172.21.15.116:0/2927614871 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee89d50b40 0x55ee89d4f090 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3ae000ba50 tx=0x7f3ae000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd206ba66278b487 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:25.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.964+0000 7f3aef7fe640 1 --2- 172.21.15.116:0/3071144571 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3ad80a2b80 0x7f3ad80a5040 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40096c0 tx=0x7f3ae4005e60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:25.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.965+0000 7f3ad67fc640 1 -- 172.21.15.116:0/2927614871 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0017420 con 0x55ee89d50b40 2024-09-12T03:06:25.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.965+0000 7f3ad67fc640 1 -- 172.21.15.116:0/2927614871 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae0017aa0 con 0x55ee89d50b40 2024-09-12T03:06:25.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.965+0000 7f3ad67fc640 1 -- 172.21.15.116:0/2927614871 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae000fa70 con 0x55ee89d50b40 2024-09-12T03:06:25.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.965+0000 7f3af8db2740 1 -- 172.21.15.116:0/2927614871 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee89d50b40 msgr2=0x55ee89d4f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:25.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.965+0000 7f3af8db2740 1 --2- 172.21.15.116:0/2927614871 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee89d50b40 0x55ee89d4f090 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3ae000ba50 tx=0x7f3ae000bf10 comp rx=0 tx=0).stop 2024-09-12T03:06:25.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.965+0000 7f3af8db2740 1 -- 172.21.15.116:0/2927614871 shutdown_connections 2024-09-12T03:06:25.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.965+0000 7f3af8db2740 1 --2- 172.21.15.116:0/2927614871 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee89d5a370 0x55ee89d5c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:25.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.965+0000 7f3af8db2740 1 --2- 172.21.15.116:0/2927614871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee89d4f5d0 0x55ee89d4fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:25.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.965+0000 7f3af8db2740 1 --2- 172.21.15.116:0/2927614871 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee89d50b40 0x55ee89d4f090 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:25.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.965+0000 7f3af8db2740 1 -- 172.21.15.116:0/2927614871 >> 172.21.15.116:0/2927614871 conn(0x55ee89d18310 msgr2=0x55ee89d4bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:25.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.965+0000 7f3af8db2740 1 -- 172.21.15.116:0/2927614871 shutdown_connections 2024-09-12T03:06:25.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.965+0000 7f3af8db2740 1 -- 172.21.15.116:0/2927614871 wait complete. 2024-09-12T03:06:25.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.966+0000 7f3af8db2740 1 -- start start 2024-09-12T03:06:25.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.966+0000 7f3af8db2740 1 Processor -- start 2024-09-12T03:06:25.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.966+0000 7f3af8db2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee89d4f5d0 0x55ee89d59c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:25.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.966+0000 7f3af8db2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee89d50b40 0x55ee89d56f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:25.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.966+0000 7f3af8db2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee89d5a370 0x55ee89d57460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:25.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.966+0000 7f3af8db2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ee89d5d070 con 0x55ee89d50b40 2024-09-12T03:06:25.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.966+0000 7f3af8db2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ee89c70060 con 0x55ee89d5a370 2024-09-12T03:06:25.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.966+0000 7f3af8db2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ee89d5d1f0 con 0x55ee89d4f5d0 2024-09-12T03:06:25.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.966+0000 7f3af8db2740 10 client.0 _ll_register_callbacks cb 0x55ee899c38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:25.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.966+0000 7f3aef7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee89d50b40 0x55ee89d56f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:25.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.966+0000 7f3aef7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee89d50b40 0x55ee89d56f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52836/0 (socket says 172.21.15.116:52836) 2024-09-12T03:06:25.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.966+0000 7f3aef7fe640 1 -- 172.21.15.116:0/3431914401 learned_addr learned my addr 172.21.15.116:0/3431914401 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:25.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.966+0000 7f3aeffff640 1 --2- 172.21.15.116:0/3431914401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee89d4f5d0 0x55ee89d59c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:25.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.967+0000 7f3af4f76640 1 --2- 172.21.15.116:0/3431914401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee89d5a370 0x55ee89d57460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:25.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.967+0000 7f3aef7fe640 1 -- 172.21.15.116:0/3431914401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee89d4f5d0 msgr2=0x55ee89d59c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:25.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.967+0000 7f3aef7fe640 1 --2- 172.21.15.116:0/3431914401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee89d4f5d0 0x55ee89d59c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:25.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.967+0000 7f3aef7fe640 1 -- 172.21.15.116:0/3431914401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee89d5a370 msgr2=0x55ee89d57460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:25.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.967+0000 7f3aef7fe640 1 --2- 172.21.15.116:0/3431914401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee89d5a370 0x55ee89d57460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:25.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.967+0000 7f3aef7fe640 1 -- 172.21.15.116:0/3431914401 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee89a263c0 con 0x55ee89d50b40 2024-09-12T03:06:25.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.967+0000 7f3af4f76640 1 --2- 172.21.15.116:0/3431914401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee89d5a370 0x55ee89d57460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:06:25.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.967+0000 7f3aef7fe640 1 --2- 172.21.15.116:0/3431914401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee89d50b40 0x55ee89d56f20 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4009930 tx=0x7f3ae402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f8823d667cdc74 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:25.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.967+0000 7f3acf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:06:25.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.967+0000 7f3acf7fe640 1 -- 172.21.15.116:0/3431914401 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae404f060 con 0x55ee89d50b40 2024-09-12T03:06:25.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.967+0000 7f3acf7fe640 1 -- 172.21.15.116:0/3431914401 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f3ae40046c0 con 0x55ee89d50b40 2024-09-12T03:06:25.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.967+0000 7f3af8db2740 1 -- 172.21.15.116:0/3431914401 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ae0017aa0 con 0x55ee89d50b40 2024-09-12T03:06:25.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.967+0000 7f3acf7fe640 1 -- 172.21.15.116:0/3431914401 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3ae4035050 con 0x55ee89d50b40 2024-09-12T03:06:25.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.967+0000 7f3af8db2740 10 client.16116 fetch_fsmap learned FSMap version 25 2024-09-12T03:06:25.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.967+0000 7f3af8db2740 1 -- 172.21.15.116:0/3431914401 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ee89d68a20 con 0x55ee89d50b40 2024-09-12T03:06:25.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.967+0000 7f3acf7fe640 1 -- 172.21.15.116:0/3431914401 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae0017aa0 con 0x55ee89d50b40 2024-09-12T03:06:25.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.968+0000 7f3acf7fe640 1 -- 172.21.15.116:0/3431914401 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3ae40039a0 con 0x55ee89d50b40 2024-09-12T03:06:25.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.968+0000 7f3af8db2740 10 client.16116 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:06:25.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.968+0000 7f3af8db2740 10 client.16116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:06:25.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.968+0000 7f3af8db2740 20 client.16116 populate_metadata read hostname 'smithi116' 2024-09-12T03:06:25.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.968+0000 7f3af8db2740 1 --2- 172.21.15.116:0/3431914401 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ee89d69ee0 0x55ee89d6c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:25.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.968+0000 7f3af8db2740 4 client.16116 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:06:25.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.968+0000 7f3af8db2740 1 -- 172.21.15.116:0/3431914401 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee89cc2860 con 0x55ee89d69ee0 2024-09-12T03:06:25.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.968+0000 7f3aeffff640 1 --2- 172.21.15.116:0/3431914401 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ee89d69ee0 0x55ee89d6c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:25.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.968+0000 7f3acf7fe640 10 client.16116 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:25.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.968+0000 7f3aeffff640 1 --2- 172.21.15.116:0/3431914401 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ee89d69ee0 0x55ee89d6c340 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1f33ca4285adb40 server_cookie=cb4b72dbdf99289d in_seq=0 out_seq=0 2024-09-12T03:06:25.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.969+0000 7f3acf7fe640 1 -- 172.21.15.116:0/3431914401 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee89cc2860 con 0x55ee89d69ee0 2024-09-12T03:06:25.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.969+0000 7f3acf7fe640 10 client.16116 handle_command_reply: tid=0 2024-09-12T03:06:25.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.969+0000 7f3af8db2740 1 client.16116 shutdown 2024-09-12T03:06:25.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.969+0000 7f3af8db2740 1 -- 172.21.15.116:0/3431914401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee89d50b40 msgr2=0x55ee89d56f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:25.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.969+0000 7f3af8db2740 1 --2- 172.21.15.116:0/3431914401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee89d50b40 0x55ee89d56f20 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4009930 tx=0x7f3ae402a420 comp rx=0 tx=0).stop 2024-09-12T03:06:25.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.969+0000 7f3af8db2740 1 -- 172.21.15.116:0/3431914401 shutdown_connections 2024-09-12T03:06:25.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.969+0000 7f3af8db2740 1 --2- 172.21.15.116:0/3431914401 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ee89d69ee0 0x55ee89d6c340 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:25.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.969+0000 7f3af8db2740 1 --2- 172.21.15.116:0/3431914401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee89d5a370 0x55ee89d57460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:25.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.969+0000 7f3af8db2740 1 --2- 172.21.15.116:0/3431914401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee89d50b40 0x55ee89d56f20 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:25.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.969+0000 7f3acf7fe640 0 client.16116 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:25.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.969+0000 7f3af8db2740 1 --2- 172.21.15.116:0/3431914401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee89d4f5d0 0x55ee89d59c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:25.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.969+0000 7f3af8db2740 1 -- 172.21.15.116:0/3431914401 >> 172.21.15.116:0/3431914401 conn(0x55ee89d18310 msgr2=0x55ee89d51960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:25.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.970+0000 7f3af8db2740 1 -- 172.21.15.116:0/3431914401 shutdown_connections 2024-09-12T03:06:25.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.970+0000 7f3af8db2740 1 -- 172.21.15.116:0/3431914401 wait complete. 2024-09-12T03:06:25.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.970+0000 7f3af8db2740 20 client.16116 trim_cache size 0 max 16384 2024-09-12T03:06:25.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.982+0000 7f3af8db2740 1 Processor -- start 2024-09-12T03:06:25.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.982+0000 7f3af8db2740 1 -- start start 2024-09-12T03:06:25.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.982+0000 7f3af8db2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee89d4f5d0 0x55ee89d66310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:25.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.982+0000 7f3af8db2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee89d50b40 0x55ee89d557f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:25.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.982+0000 7f3af8db2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee89d5a370 0x55ee89d55d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:25.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.982+0000 7f3af8db2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3af01b1fe0 con 0x55ee89d4f5d0 2024-09-12T03:06:25.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.982+0000 7f3af8db2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3af01af8b0 con 0x55ee89d50b40 2024-09-12T03:06:25.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.982+0000 7f3af8db2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3af01ad720 con 0x55ee89d5a370 2024-09-12T03:06:25.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.983+0000 7f3aeffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee89d4f5d0 0x55ee89d66310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:25.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.983+0000 7f3aeffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee89d4f5d0 0x55ee89d66310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52848/0 (socket says 172.21.15.116:52848) 2024-09-12T03:06:25.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.983+0000 7f3aeffff640 1 -- 172.21.15.116:0/3144562377 learned_addr learned my addr 172.21.15.116:0/3144562377 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:25.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.983+0000 7f3af4f76640 1 --2- 172.21.15.116:0/3144562377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee89d5a370 0x55ee89d55d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:25.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.983+0000 7f3aef7fe640 1 --2- 172.21.15.116:0/3144562377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee89d50b40 0x55ee89d557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:25.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.983+0000 7f3aeffff640 1 -- 172.21.15.116:0/3144562377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee89d5a370 msgr2=0x55ee89d55d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:25.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.983+0000 7f3aeffff640 1 --2- 172.21.15.116:0/3144562377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee89d5a370 0x55ee89d55d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:25.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.983+0000 7f3aeffff640 1 -- 172.21.15.116:0/3144562377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee89d50b40 msgr2=0x55ee89d557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:25.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.983+0000 7f3aeffff640 1 --2- 172.21.15.116:0/3144562377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee89d50b40 0x55ee89d557f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:25.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.983+0000 7f3aeffff640 1 -- 172.21.15.116:0/3144562377 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee89d55470 con 0x55ee89d4f5d0 2024-09-12T03:06:25.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.983+0000 7f3aeffff640 1 --2- 172.21.15.116:0/3144562377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee89d4f5d0 0x55ee89d66310 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f3ae000b970 tx=0x7f3ae000b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9dc18689a5dc4f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:25.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.983+0000 7f3ad5ffb640 1 -- 172.21.15.116:0/3144562377 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae002f070 con 0x55ee89d4f5d0 2024-09-12T03:06:25.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.983+0000 7f3ad5ffb640 1 -- 172.21.15.116:0/3144562377 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae002aa90 con 0x55ee89d4f5d0 2024-09-12T03:06:25.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.983+0000 7f3ad5ffb640 1 -- 172.21.15.116:0/3144562377 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae000f9c0 con 0x55ee89d4f5d0 2024-09-12T03:06:25.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.984+0000 7f3af8db2740 1 -- 172.21.15.116:0/3144562377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee89d4f5d0 msgr2=0x55ee89d66310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:25.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.984+0000 7f3af8db2740 1 --2- 172.21.15.116:0/3144562377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee89d4f5d0 0x55ee89d66310 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f3ae000b970 tx=0x7f3ae000b340 comp rx=0 tx=0).stop 2024-09-12T03:06:25.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.984+0000 7f3af8db2740 1 -- 172.21.15.116:0/3144562377 shutdown_connections 2024-09-12T03:06:25.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.984+0000 7f3af8db2740 1 --2- 172.21.15.116:0/3144562377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee89d5a370 0x55ee89d55d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:25.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.984+0000 7f3af8db2740 1 --2- 172.21.15.116:0/3144562377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee89d50b40 0x55ee89d557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:25.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.984+0000 7f3af8db2740 1 --2- 172.21.15.116:0/3144562377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee89d4f5d0 0x55ee89d66310 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:25.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.984+0000 7f3af8db2740 1 -- 172.21.15.116:0/3144562377 >> 172.21.15.116:0/3144562377 conn(0x55ee89d18310 msgr2=0x55ee89d56ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:25.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.984+0000 7f3af8db2740 1 -- 172.21.15.116:0/3144562377 shutdown_connections 2024-09-12T03:06:25.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.984+0000 7f3af8db2740 1 -- 172.21.15.116:0/3144562377 wait complete. 2024-09-12T03:06:25.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.984+0000 7f3af8db2740 1 -- start start 2024-09-12T03:06:25.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.985+0000 7f3af8db2740 1 Processor -- start 2024-09-12T03:06:25.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.985+0000 7f3af8db2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee89d4f5d0 0x55ee89d62ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:25.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.985+0000 7f3af8db2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee89d50b40 0x55ee89d53d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:25.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.985+0000 7f3af8db2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee89d5a370 0x55ee89d54380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:25.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.985+0000 7f3af8db2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3af01add70 con 0x55ee89d5a370 2024-09-12T03:06:25.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.985+0000 7f3af8db2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3af01af1e0 con 0x55ee89d4f5d0 2024-09-12T03:06:25.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.985+0000 7f3af8db2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3af01ae440 con 0x55ee89d50b40 2024-09-12T03:06:25.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.985+0000 7f3af8db2740 10 client.0 _ll_register_callbacks cb 0x7f3ae4004440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:25.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.985+0000 7f3af4f76640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee89d5a370 0x55ee89d54380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:25.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.985+0000 7f3aef7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee89d50b40 0x55ee89d53d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:25.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.985+0000 7f3aeffff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee89d4f5d0 0x55ee89d62ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:25.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.985+0000 7f3aeffff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee89d4f5d0 0x55ee89d62ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.116:58990/0 (socket says 172.21.15.116:58990) 2024-09-12T03:06:25.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.985+0000 7f3aeffff640 1 -- 172.21.15.116:0/2079195129 learned_addr learned my addr 172.21.15.116:0/2079195129 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:25.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.985+0000 7f3af4f76640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee89d5a370 0x55ee89d54380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52864/0 (socket says 172.21.15.116:52864) 2024-09-12T03:06:25.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.985+0000 7f3af4f76640 1 -- 172.21.15.116:0/2079195129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee89d50b40 msgr2=0x55ee89d53d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:25.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.985+0000 7f3af4f76640 1 --2- 172.21.15.116:0/2079195129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee89d50b40 0x55ee89d53d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:25.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.985+0000 7f3af4f76640 1 -- 172.21.15.116:0/2079195129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee89d4f5d0 msgr2=0x55ee89d62ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:25.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.985+0000 7f3af4f76640 1 --2- 172.21.15.116:0/2079195129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee89d4f5d0 0x55ee89d62ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:25.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.985+0000 7f3af4f76640 1 -- 172.21.15.116:0/2079195129 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee89d54b80 con 0x55ee89d5a370 2024-09-12T03:06:25.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.985+0000 7f3aef7fe640 1 --2- 172.21.15.116:0/2079195129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee89d50b40 0x55ee89d53d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:06:25.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.985+0000 7f3af4f76640 1 --2- 172.21.15.116:0/2079195129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee89d5a370 0x55ee89d54380 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x55ee89d50070 tx=0x7f3ae80a0090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2d9fa211e0ad15 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:25.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.985+0000 7f3acf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:06:25.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.986+0000 7f3acf7fe640 1 -- 172.21.15.116:0/2079195129 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae801f040 con 0x55ee89d5a370 2024-09-12T03:06:25.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.986+0000 7f3acf7fe640 1 -- 172.21.15.116:0/2079195129 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f3ae8086410 con 0x55ee89d5a370 2024-09-12T03:06:25.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.986+0000 7f3af8db2740 1 -- 172.21.15.116:0/2079195129 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ae002aa90 con 0x55ee89d5a370 2024-09-12T03:06:25.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.986+0000 7f3acf7fe640 1 -- 172.21.15.116:0/2079195129 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3ae809aff0 con 0x55ee89d5a370 2024-09-12T03:06:25.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.986+0000 7f3af8db2740 10 client.16122 fetch_fsmap learned FSMap version 25 2024-09-12T03:06:25.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.986+0000 7f3af8db2740 1 -- 172.21.15.116:0/2079195129 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ee89dd4d40 con 0x55ee89d5a370 2024-09-12T03:06:25.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.986+0000 7f3acf7fe640 1 -- 172.21.15.116:0/2079195129 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae002aa90 con 0x55ee89d5a370 2024-09-12T03:06:25.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.986+0000 7f3acf7fe640 1 -- 172.21.15.116:0/2079195129 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3ae800f040 con 0x55ee89d5a370 2024-09-12T03:06:25.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.986+0000 7f3af8db2740 10 client.16122 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:06:25.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.986+0000 7f3af8db2740 10 client.16122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:06:25.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.986+0000 7f3af8db2740 20 client.16122 populate_metadata read hostname 'smithi116' 2024-09-12T03:06:25.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.986+0000 7f3af8db2740 1 --2- 172.21.15.116:0/2079195129 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ee89dd54e0 0x55ee89dd7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:25.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.986+0000 7f3af8db2740 4 client.16122 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:06:25.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.986+0000 7f3af8db2740 1 -- 172.21.15.116:0/2079195129 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee89cbcd70 con 0x55ee89dd54e0 2024-09-12T03:06:25.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.986+0000 7f3aeffff640 1 --2- 172.21.15.116:0/2079195129 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ee89dd54e0 0x55ee89dd7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:25.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.987+0000 7f3acf7fe640 10 client.16122 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:25.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.987+0000 7f3aeffff640 1 --2- 172.21.15.116:0/2079195129 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ee89dd54e0 0x55ee89dd7900 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b02e47f61c367104 server_cookie=dc96db4075d0d4e7 in_seq=0 out_seq=0 2024-09-12T03:06:25.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.987+0000 7f3acf7fe640 1 -- 172.21.15.116:0/2079195129 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee89cbcd70 con 0x55ee89dd54e0 2024-09-12T03:06:25.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.987+0000 7f3acf7fe640 10 client.16122 handle_command_reply: tid=0 2024-09-12T03:06:25.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.987+0000 7f3af8db2740 1 client.16122 shutdown 2024-09-12T03:06:25.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.987+0000 7f3af8db2740 1 -- 172.21.15.116:0/2079195129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee89d5a370 msgr2=0x55ee89d54380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:25.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.987+0000 7f3af8db2740 1 --2- 172.21.15.116:0/2079195129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee89d5a370 0x55ee89d54380 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x55ee89d50070 tx=0x7f3ae80a0090 comp rx=0 tx=0).stop 2024-09-12T03:06:25.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.987+0000 7f3af8db2740 1 -- 172.21.15.116:0/2079195129 shutdown_connections 2024-09-12T03:06:25.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.987+0000 7f3af8db2740 1 --2- 172.21.15.116:0/2079195129 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ee89dd54e0 0x55ee89dd7900 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:25.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.987+0000 7f3af8db2740 1 --2- 172.21.15.116:0/2079195129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee89d5a370 0x55ee89d54380 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:25.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.987+0000 7f3af8db2740 1 --2- 172.21.15.116:0/2079195129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee89d50b40 0x55ee89d53d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:25.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.987+0000 7f3af8db2740 1 --2- 172.21.15.116:0/2079195129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee89d4f5d0 0x55ee89d62ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:25.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.987+0000 7f3af8db2740 1 -- 172.21.15.116:0/2079195129 >> 172.21.15.116:0/2079195129 conn(0x55ee89d18310 msgr2=0x55ee89d557b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:25.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.987+0000 7f3acf7fe640 0 client.16122 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:25.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.987+0000 7f3af8db2740 1 -- 172.21.15.116:0/2079195129 shutdown_connections 2024-09-12T03:06:25.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.987+0000 7f3af8db2740 1 -- 172.21.15.116:0/2079195129 wait complete. 2024-09-12T03:06:25.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.987+0000 7f3af8db2740 20 client.16122 trim_cache size 0 max 16384 2024-09-12T03:06:26.000 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:06:26.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.988+0000 7f3af69d9640 1 -- 172.21.15.116:0/3071144571 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3ad80a2b80 msgr2=0x7f3ad80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:26.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.988+0000 7f3af69d9640 1 --2- 172.21.15.116:0/3071144571 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3ad80a2b80 0x7f3ad80a5040 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40096c0 tx=0x7f3ae4005e60 comp rx=0 tx=0).stop 2024-09-12T03:06:26.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.988+0000 7f3af69d9640 1 -- 172.21.15.116:0/3071144571 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3af0106a80 msgr2=0x7f3af010ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:26.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.988+0000 7f3af69d9640 1 --2- 172.21.15.116:0/3071144571 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3af0106a80 0x7f3af010ee60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800eaa0 tx=0x7f3ae800ef60 comp rx=0 tx=0).stop 2024-09-12T03:06:26.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.988+0000 7f3af69d9640 1 -- 172.21.15.116:0/3071144571 shutdown_connections 2024-09-12T03:06:26.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.988+0000 7f3af69d9640 1 --2- 172.21.15.116:0/3071144571 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3ad80a2b80 0x7f3ad80a5040 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.988+0000 7f3af69d9640 1 --2- 172.21.15.116:0/3071144571 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3af0106a80 0x7f3af010ee60 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.988+0000 7f3af69d9640 1 --2- 172.21.15.116:0/3071144571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3af0101420 0x7f3af010e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.988+0000 7f3af69d9640 1 --2- 172.21.15.116:0/3071144571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3af0100a80 0x7f3af010e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.988+0000 7f3af69d9640 1 -- 172.21.15.116:0/3071144571 >> 172.21.15.116:0/3071144571 conn(0x7f3af00fc7a0 msgr2=0x7f3af00fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:26.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.988+0000 7f3af69d9640 1 -- 172.21.15.116:0/3071144571 shutdown_connections 2024-09-12T03:06:26.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:25.988+0000 7f3af69d9640 1 -- 172.21.15.116:0/3071144571 wait complete. 2024-09-12T03:06:26.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:26 smithi116 ceph-mon[31410]: pgmap v441: 97 pgs: 97 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:06:26.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:26 smithi183 ceph-mon[35535]: pgmap v441: 97 pgs: 97 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:06:26.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:26 smithi177 ceph-mon[35305]: pgmap v441: 97 pgs: 97 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:06:27.875 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:06:27.875 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:06:28.133 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:06:28.537 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:28 smithi116 ceph-mon[31410]: pgmap v442: 97 pgs: 97 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:06:28.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.620+0000 7ff07f0f9640 1 -- 172.21.15.116:0/2728093213 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff078102790 msgr2=0x7ff0781067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:28.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.620+0000 7ff07f0f9640 1 --2- 172.21.15.116:0/2728093213 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff078102790 0x7ff0781067e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff064009900 tx=0x7ff06402a410 comp rx=0 tx=0).stop 2024-09-12T03:06:28.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.620+0000 7ff07f0f9640 1 -- 172.21.15.116:0/2728093213 shutdown_connections 2024-09-12T03:06:28.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.620+0000 7ff07f0f9640 1 --2- 172.21.15.116:0/2728093213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff078106f10 0x7ff07810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.620+0000 7ff07f0f9640 1 --2- 172.21.15.116:0/2728093213 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff078102790 0x7ff0781067e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.620+0000 7ff07f0f9640 1 --2- 172.21.15.116:0/2728093213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff078101de0 0x7ff0781021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.620+0000 7ff07f0f9640 1 -- 172.21.15.116:0/2728093213 >> 172.21.15.116:0/2728093213 conn(0x7ff0780fd8c0 msgr2=0x7ff0780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:28.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.620+0000 7ff07f0f9640 1 -- 172.21.15.116:0/2728093213 shutdown_connections 2024-09-12T03:06:28.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.620+0000 7ff07f0f9640 1 -- 172.21.15.116:0/2728093213 wait complete. 2024-09-12T03:06:28.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.621+0000 7ff07f0f9640 1 Processor -- start 2024-09-12T03:06:28.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.621+0000 7ff07f0f9640 1 -- start start 2024-09-12T03:06:28.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.621+0000 7ff07f0f9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff078101de0 0x7ff078198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:28.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.621+0000 7ff07f0f9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff078102790 0x7ff078198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:28.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.621+0000 7ff07f0f9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff078106f10 0x7ff07819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:28.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.621+0000 7ff07f0f9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff078111f70 con 0x7ff078102790 2024-09-12T03:06:28.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.621+0000 7ff07f0f9640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff078111df0 con 0x7ff078106f10 2024-09-12T03:06:28.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.621+0000 7ff07f0f9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781120f0 con 0x7ff078101de0 2024-09-12T03:06:28.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.621+0000 7ff077fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff078102790 0x7ff078198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:28.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.621+0000 7ff077fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff078102790 0x7ff078198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52880/0 (socket says 172.21.15.116:52880) 2024-09-12T03:06:28.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.621+0000 7ff077fff640 1 -- 172.21.15.116:0/1831521331 learned_addr learned my addr 172.21.15.116:0/1831521331 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:28.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.621+0000 7ff07d696640 1 --2- 172.21.15.116:0/1831521331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff078106f10 0x7ff07819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:28.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.621+0000 7ff07ce95640 1 --2- 172.21.15.116:0/1831521331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff078101de0 0x7ff078198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:28.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.621+0000 7ff077fff640 1 -- 172.21.15.116:0/1831521331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff078101de0 msgr2=0x7ff078198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:28.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.621+0000 7ff077fff640 1 --2- 172.21.15.116:0/1831521331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff078101de0 0x7ff078198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.621+0000 7ff077fff640 1 -- 172.21.15.116:0/1831521331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff078106f10 msgr2=0x7ff07819cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:28.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.621+0000 7ff077fff640 1 --2- 172.21.15.116:0/1831521331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff078106f10 0x7ff07819cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.621+0000 7ff077fff640 1 -- 172.21.15.116:0/1831521331 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff07819d4c0 con 0x7ff078102790 2024-09-12T03:06:28.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.621+0000 7ff07d696640 1 --2- 172.21.15.116:0/1831521331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff078106f10 0x7ff07819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:06:28.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.622+0000 7ff077fff640 1 --2- 172.21.15.116:0/1831521331 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff078102790 0x7ff078198a50 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff064006010 tx=0x7ff0640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:28.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.622+0000 7ff075ffb640 1 -- 172.21.15.116:0/1831521331 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06403f030 con 0x7ff078102790 2024-09-12T03:06:28.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.622+0000 7ff075ffb640 1 -- 172.21.15.116:0/1831521331 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff064003ce0 con 0x7ff078102790 2024-09-12T03:06:28.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.622+0000 7ff07f0f9640 1 -- 172.21.15.116:0/1831521331 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff07819d750 con 0x7ff078102790 2024-09-12T03:06:28.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.622+0000 7ff075ffb640 1 -- 172.21.15.116:0/1831521331 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff064035070 con 0x7ff078102790 2024-09-12T03:06:28.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.622+0000 7ff07f0f9640 1 -- 172.21.15.116:0/1831521331 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0781a5090 con 0x7ff078102790 2024-09-12T03:06:28.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.624+0000 7ff075ffb640 1 -- 172.21.15.116:0/1831521331 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ff064042030 con 0x7ff078102790 2024-09-12T03:06:28.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.625+0000 7ff0814d2740 1 Processor -- start 2024-09-12T03:06:28.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.625+0000 7ff0814d2740 1 -- start start 2024-09-12T03:06:28.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.625+0000 7ff075ffb640 1 --2- 172.21.15.116:0/1831521331 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff0680a2b80 0x7ff0680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:28.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.625+0000 7ff0814d2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614d7c23a20 0x5614d7c23e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:28.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.625+0000 7ff0814d2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614d7c24340 0x5614d7c32c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:28.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.625+0000 7ff0814d2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614d7c33150 0x5614d7c35540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:28.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.625+0000 7ff0814d2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5614d7b89280 con 0x5614d7c24340 2024-09-12T03:06:28.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.625+0000 7ff0814d2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5614d7b30b10 con 0x5614d7c33150 2024-09-12T03:06:28.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.625+0000 7ff0814d2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5614d7b85240 con 0x5614d7c23a20 2024-09-12T03:06:28.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.625+0000 7ff07d696640 1 --2- 172.21.15.116:0/1831521331 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff0680a2b80 0x7ff0680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:28.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.625+0000 7ff075ffb640 1 -- 172.21.15.116:0/1831521331 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff0640ef780 con 0x7ff078102790 2024-09-12T03:06:28.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.625+0000 7ff077fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614d7c24340 0x5614d7c32c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:28.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.625+0000 7ff077fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614d7c24340 0x5614d7c32c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52890/0 (socket says 172.21.15.116:52890) 2024-09-12T03:06:28.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.625+0000 7ff077fff640 1 -- 172.21.15.116:0/3643844317 learned_addr learned my addr 172.21.15.116:0/3643844317 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:28.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.625+0000 7ff07ce95640 1 --2- 172.21.15.116:0/3643844317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614d7c23a20 0x5614d7c23e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:28.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.625+0000 7ff07d696640 1 --2- 172.21.15.116:0/3643844317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614d7c33150 0x5614d7c35540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:28.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.625+0000 7ff077fff640 1 -- 172.21.15.116:0/3643844317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614d7c23a20 msgr2=0x5614d7c23e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:28.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.625+0000 7ff077fff640 1 --2- 172.21.15.116:0/3643844317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614d7c23a20 0x5614d7c23e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.625+0000 7ff077fff640 1 -- 172.21.15.116:0/3643844317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614d7c33150 msgr2=0x5614d7c35540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:28.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.625+0000 7ff077fff640 1 --2- 172.21.15.116:0/3643844317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614d7c33150 0x5614d7c35540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.625+0000 7ff077fff640 1 -- 172.21.15.116:0/3643844317 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614d7b3d1f0 con 0x5614d7c24340 2024-09-12T03:06:28.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.626+0000 7ff07d696640 1 --2- 172.21.15.116:0/1831521331 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff0680a2b80 0x7ff0680a5040 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff07000afd0 tx=0x7ff07000ac80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:28.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.626+0000 7ff077fff640 1 --2- 172.21.15.116:0/3643844317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614d7c24340 0x5614d7c32c10 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff0640c4810 tx=0x7ff0640b83f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d13db27911f747 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:28.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.626+0000 7ff062ffd640 1 -- 172.21.15.116:0/3643844317 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff064049040 con 0x5614d7c24340 2024-09-12T03:06:28.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.626+0000 7ff062ffd640 1 -- 172.21.15.116:0/3643844317 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff06403b580 con 0x5614d7c24340 2024-09-12T03:06:28.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.626+0000 7ff062ffd640 1 -- 172.21.15.116:0/3643844317 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0640dc0d0 con 0x5614d7c24340 2024-09-12T03:06:28.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.626+0000 7ff0814d2740 1 -- 172.21.15.116:0/3643844317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614d7c24340 msgr2=0x5614d7c32c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:28.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.626+0000 7ff0814d2740 1 --2- 172.21.15.116:0/3643844317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614d7c24340 0x5614d7c32c10 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff0640c4810 tx=0x7ff0640b83f0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.626+0000 7ff0814d2740 1 -- 172.21.15.116:0/3643844317 shutdown_connections 2024-09-12T03:06:28.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.626+0000 7ff0814d2740 1 --2- 172.21.15.116:0/3643844317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614d7c33150 0x5614d7c35540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.626+0000 7ff0814d2740 1 --2- 172.21.15.116:0/3643844317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614d7c24340 0x5614d7c32c10 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.626+0000 7ff0814d2740 1 --2- 172.21.15.116:0/3643844317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614d7c23a20 0x5614d7c23e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.626+0000 7ff0814d2740 1 -- 172.21.15.116:0/3643844317 >> 172.21.15.116:0/3643844317 conn(0x5614d7c200f0 msgr2=0x5614d7c204d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:28.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.626+0000 7ff0814d2740 1 -- 172.21.15.116:0/3643844317 shutdown_connections 2024-09-12T03:06:28.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.627+0000 7ff0814d2740 1 -- 172.21.15.116:0/3643844317 wait complete. 2024-09-12T03:06:28.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.627+0000 7ff0814d2740 1 -- start start 2024-09-12T03:06:28.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.627+0000 7ff0814d2740 1 Processor -- start 2024-09-12T03:06:28.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.627+0000 7ff0814d2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614d7c23a20 0x5614d7c30100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:28.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.627+0000 7ff0814d2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614d7c24340 0x5614d7c30640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:28.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.627+0000 7ff0814d2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614d7c33150 0x5614d7c30b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:28.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.627+0000 7ff0814d2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5614d7c35ae0 con 0x5614d7c33150 2024-09-12T03:06:28.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.627+0000 7ff0814d2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5614d7b43f10 con 0x5614d7c23a20 2024-09-12T03:06:28.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.627+0000 7ff0814d2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5614d7c35c60 con 0x5614d7c24340 2024-09-12T03:06:28.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.627+0000 7ff0814d2740 10 client.0 _ll_register_callbacks cb 0x5614d7b0ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:28.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.627+0000 7ff07d696640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614d7c33150 0x5614d7c30b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:28.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.627+0000 7ff07d696640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614d7c33150 0x5614d7c30b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52894/0 (socket says 172.21.15.116:52894) 2024-09-12T03:06:28.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.627+0000 7ff07d696640 1 -- 172.21.15.116:0/1951009993 learned_addr learned my addr 172.21.15.116:0/1951009993 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:28.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.628+0000 7ff07ce95640 1 --2- 172.21.15.116:0/1951009993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614d7c23a20 0x5614d7c30100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:28.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.628+0000 7ff077fff640 1 --2- 172.21.15.116:0/1951009993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614d7c24340 0x5614d7c30640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:28.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.628+0000 7ff07d696640 1 -- 172.21.15.116:0/1951009993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614d7c24340 msgr2=0x5614d7c30640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:28.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.628+0000 7ff07d696640 1 --2- 172.21.15.116:0/1951009993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614d7c24340 0x5614d7c30640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.628+0000 7ff07d696640 1 -- 172.21.15.116:0/1951009993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614d7c23a20 msgr2=0x5614d7c30100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:28.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.628+0000 7ff07d696640 1 --2- 172.21.15.116:0/1951009993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614d7c23a20 0x5614d7c30100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.628+0000 7ff07d696640 1 -- 172.21.15.116:0/1951009993 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614d78fa6a0 con 0x5614d7c33150 2024-09-12T03:06:28.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.628+0000 7ff077fff640 1 --2- 172.21.15.116:0/1951009993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614d7c24340 0x5614d7c30640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:06:28.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.628+0000 7ff07ce95640 1 --2- 172.21.15.116:0/1951009993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614d7c23a20 0x5614d7c30100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:06:28.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.628+0000 7ff07d696640 1 --2- 172.21.15.116:0/1951009993 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614d7c33150 0x5614d7c30b80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff070010fc0 tx=0x7ff070005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d169a7ba996252b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:28.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.628+0000 7ff057fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:06:28.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.628+0000 7ff057fff640 1 -- 172.21.15.116:0/1951009993 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070026070 con 0x5614d7c33150 2024-09-12T03:06:28.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.628+0000 7ff057fff640 1 -- 172.21.15.116:0/1951009993 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff070004d20 con 0x5614d7c33150 2024-09-12T03:06:28.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.628+0000 7ff0814d2740 1 -- 172.21.15.116:0/1951009993 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5614d7b967f0 con 0x5614d7c33150 2024-09-12T03:06:28.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.628+0000 7ff057fff640 1 -- 172.21.15.116:0/1951009993 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff070031050 con 0x5614d7c33150 2024-09-12T03:06:28.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.628+0000 7ff0814d2740 10 client.16134 fetch_fsmap learned FSMap version 25 2024-09-12T03:06:28.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.628+0000 7ff0814d2740 1 -- 172.21.15.116:0/1951009993 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5614d7c41740 con 0x5614d7c33150 2024-09-12T03:06:28.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.628+0000 7ff057fff640 1 -- 172.21.15.116:0/1951009993 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5614d7b967f0 con 0x5614d7c33150 2024-09-12T03:06:28.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.629+0000 7ff057fff640 1 -- 172.21.15.116:0/1951009993 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff070005020 con 0x5614d7c33150 2024-09-12T03:06:28.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.629+0000 7ff0814d2740 10 client.16134 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:06:28.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.629+0000 7ff0814d2740 10 client.16134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:06:28.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.629+0000 7ff0814d2740 20 client.16134 populate_metadata read hostname 'smithi116' 2024-09-12T03:06:28.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.629+0000 7ff0814d2740 1 --2- 172.21.15.116:0/1951009993 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5614d7c42bf0 0x5614d7c45050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:28.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.629+0000 7ff0814d2740 4 client.16134 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:06:28.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.629+0000 7ff0814d2740 1 -- 172.21.15.116:0/1951009993 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614d7b90d00 con 0x5614d7c42bf0 2024-09-12T03:06:28.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.629+0000 7ff07ce95640 1 --2- 172.21.15.116:0/1951009993 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5614d7c42bf0 0x5614d7c45050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:28.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.629+0000 7ff07ce95640 1 --2- 172.21.15.116:0/1951009993 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5614d7c42bf0 0x5614d7c45050 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6ca5dec0492469e server_cookie=4c8ebdeaac733a69 in_seq=0 out_seq=0 2024-09-12T03:06:28.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.629+0000 7ff057fff640 10 client.16134 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:28.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.630+0000 7ff057fff640 1 -- 172.21.15.116:0/1951009993 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5614d7b90d00 con 0x5614d7c42bf0 2024-09-12T03:06:28.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.630+0000 7ff057fff640 10 client.16134 handle_command_reply: tid=0 2024-09-12T03:06:28.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.630+0000 7ff0814d2740 1 client.16134 shutdown 2024-09-12T03:06:28.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.630+0000 7ff0814d2740 1 -- 172.21.15.116:0/1951009993 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614d7c33150 msgr2=0x5614d7c30b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:28.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.630+0000 7ff0814d2740 1 --2- 172.21.15.116:0/1951009993 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614d7c33150 0x5614d7c30b80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff070010fc0 tx=0x7ff070005c60 comp rx=0 tx=0).stop 2024-09-12T03:06:28.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.630+0000 7ff0814d2740 1 -- 172.21.15.116:0/1951009993 shutdown_connections 2024-09-12T03:06:28.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.630+0000 7ff0814d2740 1 --2- 172.21.15.116:0/1951009993 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5614d7c42bf0 0x5614d7c45050 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.630+0000 7ff0814d2740 1 --2- 172.21.15.116:0/1951009993 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614d7c33150 0x5614d7c30b80 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.630+0000 7ff0814d2740 1 --2- 172.21.15.116:0/1951009993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614d7c24340 0x5614d7c30640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.630+0000 7ff0814d2740 1 --2- 172.21.15.116:0/1951009993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614d7c23a20 0x5614d7c30100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.630+0000 7ff0814d2740 1 -- 172.21.15.116:0/1951009993 >> 172.21.15.116:0/1951009993 conn(0x5614d7c200f0 msgr2=0x5614d7c25e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:28.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.630+0000 7ff057fff640 0 client.16134 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:28.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.631+0000 7ff0814d2740 1 -- 172.21.15.116:0/1951009993 shutdown_connections 2024-09-12T03:06:28.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.631+0000 7ff0814d2740 1 -- 172.21.15.116:0/1951009993 wait complete. 2024-09-12T03:06:28.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.631+0000 7ff0814d2740 20 client.16134 trim_cache size 0 max 16384 2024-09-12T03:06:28.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.648+0000 7ff0814d2740 1 Processor -- start 2024-09-12T03:06:28.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.648+0000 7ff0814d2740 1 -- start start 2024-09-12T03:06:28.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.648+0000 7ff0814d2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614d7c23a20 0x5614d7c1f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:28.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.648+0000 7ff0814d2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614d7c24340 0x5614d7c4a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:28.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.648+0000 7ff0814d2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614d7c33150 0x5614d7c4ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:28.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.648+0000 7ff0814d2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781adb10 con 0x5614d7c24340 2024-09-12T03:06:28.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.648+0000 7ff0814d2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781ab3e0 con 0x5614d7c23a20 2024-09-12T03:06:28.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.648+0000 7ff0814d2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781a9230 con 0x5614d7c33150 2024-09-12T03:06:28.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.648+0000 7ff077fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614d7c24340 0x5614d7c4a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:28.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.648+0000 7ff077fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614d7c24340 0x5614d7c4a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52898/0 (socket says 172.21.15.116:52898) 2024-09-12T03:06:28.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.648+0000 7ff077fff640 1 -- 172.21.15.116:0/3298199663 learned_addr learned my addr 172.21.15.116:0/3298199663 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:28.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.648+0000 7ff07ce95640 1 --2- 172.21.15.116:0/3298199663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614d7c23a20 0x5614d7c1f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:28.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.648+0000 7ff07d696640 1 --2- 172.21.15.116:0/3298199663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614d7c33150 0x5614d7c4ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:28.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.648+0000 7ff077fff640 1 -- 172.21.15.116:0/3298199663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614d7c33150 msgr2=0x5614d7c4ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:28.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.648+0000 7ff077fff640 1 --2- 172.21.15.116:0/3298199663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614d7c33150 0x5614d7c4ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.648+0000 7ff077fff640 1 -- 172.21.15.116:0/3298199663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614d7c23a20 msgr2=0x5614d7c1f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:28.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.648+0000 7ff077fff640 1 --2- 172.21.15.116:0/3298199663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614d7c23a20 0x5614d7c1f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.648+0000 7ff077fff640 1 -- 172.21.15.116:0/3298199663 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614d7c2f0b0 con 0x5614d7c24340 2024-09-12T03:06:28.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.648+0000 7ff07d696640 1 --2- 172.21.15.116:0/3298199663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614d7c33150 0x5614d7c4ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:06:28.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.649+0000 7ff077fff640 1 --2- 172.21.15.116:0/3298199663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614d7c24340 0x5614d7c4a920 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff0640bc060 tx=0x7ff0640bbcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6149a05ecc5a62da server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:28.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.649+0000 7ff0627fc640 1 -- 172.21.15.116:0/3298199663 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff064049070 con 0x5614d7c24340 2024-09-12T03:06:28.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.649+0000 7ff0627fc640 1 -- 172.21.15.116:0/3298199663 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0640dc0d0 con 0x5614d7c24340 2024-09-12T03:06:28.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.649+0000 7ff0627fc640 1 -- 172.21.15.116:0/3298199663 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0640c4d40 con 0x5614d7c24340 2024-09-12T03:06:28.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.649+0000 7ff0814d2740 1 -- 172.21.15.116:0/3298199663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614d7c24340 msgr2=0x5614d7c4a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:28.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.649+0000 7ff0814d2740 1 --2- 172.21.15.116:0/3298199663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614d7c24340 0x5614d7c4a920 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff0640bc060 tx=0x7ff0640bbcd0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.649+0000 7ff0814d2740 1 -- 172.21.15.116:0/3298199663 shutdown_connections 2024-09-12T03:06:28.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.649+0000 7ff0814d2740 1 --2- 172.21.15.116:0/3298199663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614d7c33150 0x5614d7c4ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.649+0000 7ff0814d2740 1 --2- 172.21.15.116:0/3298199663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614d7c24340 0x5614d7c4a920 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.649+0000 7ff0814d2740 1 --2- 172.21.15.116:0/3298199663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614d7c23a20 0x5614d7c1f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.649+0000 7ff0814d2740 1 -- 172.21.15.116:0/3298199663 >> 172.21.15.116:0/3298199663 conn(0x5614d7c200f0 msgr2=0x5614d7c1ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:28.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.649+0000 7ff0814d2740 1 -- 172.21.15.116:0/3298199663 shutdown_connections 2024-09-12T03:06:28.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.649+0000 7ff0814d2740 1 -- 172.21.15.116:0/3298199663 wait complete. 2024-09-12T03:06:28.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.650+0000 7ff0814d2740 1 -- start start 2024-09-12T03:06:28.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.650+0000 7ff0814d2740 1 Processor -- start 2024-09-12T03:06:28.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.650+0000 7ff0814d2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614d7c23a20 0x5614d7c3b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:28.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.650+0000 7ff0814d2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614d7c24340 0x5614d7c36a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:28.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.650+0000 7ff0814d2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614d7c33150 0x5614d7c37040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:28.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.650+0000 7ff0814d2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781a98a0 con 0x5614d7c24340 2024-09-12T03:06:28.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.650+0000 7ff0814d2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781aad10 con 0x5614d7c33150 2024-09-12T03:06:28.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.650+0000 7ff0814d2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781a9f70 con 0x5614d7c23a20 2024-09-12T03:06:28.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.650+0000 7ff0814d2740 10 client.0 _ll_register_callbacks cb 0x7ff070013800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:28.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.650+0000 7ff077fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614d7c24340 0x5614d7c36a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:28.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.650+0000 7ff077fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614d7c24340 0x5614d7c36a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52914/0 (socket says 172.21.15.116:52914) 2024-09-12T03:06:28.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.650+0000 7ff077fff640 1 -- 172.21.15.116:0/3355626050 learned_addr learned my addr 172.21.15.116:0/3355626050 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:28.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.651+0000 7ff07d696640 1 --2- 172.21.15.116:0/3355626050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614d7c33150 0x5614d7c37040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:28.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.651+0000 7ff07ce95640 1 --2- 172.21.15.116:0/3355626050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614d7c23a20 0x5614d7c3b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:28.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.651+0000 7ff077fff640 1 -- 172.21.15.116:0/3355626050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614d7c23a20 msgr2=0x5614d7c3b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:28.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.651+0000 7ff077fff640 1 --2- 172.21.15.116:0/3355626050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614d7c23a20 0x5614d7c3b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.651+0000 7ff077fff640 1 -- 172.21.15.116:0/3355626050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614d7c33150 msgr2=0x5614d7c37040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:28.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.651+0000 7ff077fff640 1 --2- 172.21.15.116:0/3355626050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614d7c33150 0x5614d7c37040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.651+0000 7ff077fff640 1 -- 172.21.15.116:0/3355626050 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614d7c37870 con 0x5614d7c24340 2024-09-12T03:06:28.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.651+0000 7ff07d696640 1 --2- 172.21.15.116:0/3355626050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614d7c33150 0x5614d7c37040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:06:28.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.651+0000 7ff077fff640 1 --2- 172.21.15.116:0/3355626050 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614d7c24340 0x5614d7c36a50 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff0640ed300 tx=0x7ff0640bc110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c81164d6bbbd272 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:28.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.651+0000 7ff057fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:06:28.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.651+0000 7ff057fff640 1 -- 172.21.15.116:0/3355626050 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff064049070 con 0x5614d7c24340 2024-09-12T03:06:28.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.651+0000 7ff057fff640 1 -- 172.21.15.116:0/3355626050 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff0640c0170 con 0x5614d7c24340 2024-09-12T03:06:28.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.651+0000 7ff0814d2740 1 -- 172.21.15.116:0/3355626050 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0640dc0d0 con 0x5614d7c24340 2024-09-12T03:06:28.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.651+0000 7ff057fff640 1 -- 172.21.15.116:0/3355626050 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff0640bbea0 con 0x5614d7c24340 2024-09-12T03:06:28.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.651+0000 7ff0814d2740 10 client.16140 fetch_fsmap learned FSMap version 25 2024-09-12T03:06:28.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.651+0000 7ff0814d2740 1 -- 172.21.15.116:0/3355626050 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5614d7cad9b0 con 0x5614d7c24340 2024-09-12T03:06:28.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.651+0000 7ff057fff640 1 -- 172.21.15.116:0/3355626050 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0640dc0d0 con 0x5614d7c24340 2024-09-12T03:06:28.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.652+0000 7ff057fff640 1 -- 172.21.15.116:0/3355626050 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff06409c030 con 0x5614d7c24340 2024-09-12T03:06:28.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.652+0000 7ff0814d2740 10 client.16140 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:06:28.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.652+0000 7ff0814d2740 10 client.16140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:06:28.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.652+0000 7ff0814d2740 20 client.16140 populate_metadata read hostname 'smithi116' 2024-09-12T03:06:28.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.652+0000 7ff0814d2740 1 --2- 172.21.15.116:0/3355626050 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5614d7cadc60 0x5614d7cb0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:28.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.652+0000 7ff0814d2740 4 client.16140 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:06:28.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.652+0000 7ff0814d2740 1 -- 172.21.15.116:0/3355626050 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5614d7b0d110 con 0x5614d7cadc60 2024-09-12T03:06:28.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.652+0000 7ff07ce95640 1 --2- 172.21.15.116:0/3355626050 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5614d7cadc60 0x5614d7cb0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:28.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.653+0000 7ff07ce95640 1 --2- 172.21.15.116:0/3355626050 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5614d7cadc60 0x5614d7cb0080 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=970279c1eb07de85 server_cookie=3f4e92e401e1e624 in_seq=0 out_seq=0 2024-09-12T03:06:28.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.653+0000 7ff057fff640 10 client.16140 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:28.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.653+0000 7ff057fff640 1 -- 172.21.15.116:0/3355626050 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5614d7b0d110 con 0x5614d7cadc60 2024-09-12T03:06:28.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.653+0000 7ff057fff640 10 client.16140 handle_command_reply: tid=0 2024-09-12T03:06:28.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.654+0000 7ff0814d2740 1 client.16140 shutdown 2024-09-12T03:06:28.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.654+0000 7ff0814d2740 1 -- 172.21.15.116:0/3355626050 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614d7c24340 msgr2=0x5614d7c36a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:28.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.654+0000 7ff0814d2740 1 --2- 172.21.15.116:0/3355626050 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614d7c24340 0x5614d7c36a50 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff0640ed300 tx=0x7ff0640bc110 comp rx=0 tx=0).stop 2024-09-12T03:06:28.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.654+0000 7ff0814d2740 1 -- 172.21.15.116:0/3355626050 shutdown_connections 2024-09-12T03:06:28.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.654+0000 7ff0814d2740 1 --2- 172.21.15.116:0/3355626050 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5614d7cadc60 0x5614d7cb0080 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.654+0000 7ff057fff640 0 client.16140 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:28.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.654+0000 7ff0814d2740 1 --2- 172.21.15.116:0/3355626050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614d7c33150 0x5614d7c37040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.654+0000 7ff0814d2740 1 --2- 172.21.15.116:0/3355626050 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614d7c24340 0x5614d7c36a50 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.654+0000 7ff0814d2740 1 --2- 172.21.15.116:0/3355626050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614d7c23a20 0x5614d7c3b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.654+0000 7ff0814d2740 1 -- 172.21.15.116:0/3355626050 >> 172.21.15.116:0/3355626050 conn(0x5614d7c200f0 msgr2=0x5614d7c1f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:28.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.654+0000 7ff0814d2740 1 -- 172.21.15.116:0/3355626050 shutdown_connections 2024-09-12T03:06:28.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.654+0000 7ff0814d2740 1 -- 172.21.15.116:0/3355626050 wait complete. 2024-09-12T03:06:28.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.654+0000 7ff0814d2740 20 client.16140 trim_cache size 0 max 16384 2024-09-12T03:06:28.660 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:06:28.661 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:06:28.661 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "d000eeee-f0dc-4e01-be63-bb6189391f95", 2024-09-12T03:06:28.661 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:06:28.661 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:06:28.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.655+0000 7ff07f0f9640 1 -- 172.21.15.116:0/1831521331 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff0680a2b80 msgr2=0x7ff0680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:28.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.655+0000 7ff07f0f9640 1 --2- 172.21.15.116:0/1831521331 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff0680a2b80 0x7ff0680a5040 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff07000afd0 tx=0x7ff07000ac80 comp rx=0 tx=0).stop 2024-09-12T03:06:28.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.655+0000 7ff07f0f9640 1 -- 172.21.15.116:0/1831521331 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff078102790 msgr2=0x7ff078198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:28.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.655+0000 7ff07f0f9640 1 --2- 172.21.15.116:0/1831521331 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff078102790 0x7ff078198a50 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff064006010 tx=0x7ff0640042f0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.655+0000 7ff07f0f9640 1 -- 172.21.15.116:0/1831521331 shutdown_connections 2024-09-12T03:06:28.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.655+0000 7ff07f0f9640 1 --2- 172.21.15.116:0/1831521331 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff0680a2b80 0x7ff0680a5040 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.655+0000 7ff07f0f9640 1 --2- 172.21.15.116:0/1831521331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff078106f10 0x7ff07819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.655+0000 7ff07f0f9640 1 --2- 172.21.15.116:0/1831521331 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff078102790 0x7ff078198a50 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.655+0000 7ff07f0f9640 1 --2- 172.21.15.116:0/1831521331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff078101de0 0x7ff078198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:28.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.655+0000 7ff07f0f9640 1 -- 172.21.15.116:0/1831521331 >> 172.21.15.116:0/1831521331 conn(0x7ff0780fd8c0 msgr2=0x7ff0780fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:28.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.655+0000 7ff07f0f9640 1 -- 172.21.15.116:0/1831521331 shutdown_connections 2024-09-12T03:06:28.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:28.655+0000 7ff07f0f9640 1 -- 172.21.15.116:0/1831521331 wait complete. 2024-09-12T03:06:28.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:28 smithi183 ceph-mon[35535]: pgmap v442: 97 pgs: 97 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:06:28.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:28 smithi177 ceph-mon[35305]: pgmap v442: 97 pgs: 97 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:06:29.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:29 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:06:29.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:29 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:06:29.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:29 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:06:29.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:29 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:06:29.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:29 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:06:29.586 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:06:29.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:29 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:06:29.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:29 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:06:29.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:29 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:06:29.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:29 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:06:29.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:29 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:06:29.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:29 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:06:29.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:29 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:06:29.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:29 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:06:29.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:29 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:06:29.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:29 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:06:29.834 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:06:30.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.321+0000 7f234e46f640 1 -- 172.21.15.116:0/967859890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2348100e20 msgr2=0x7f2348101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:30.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.321+0000 7f234e46f640 1 --2- 172.21.15.116:0/967859890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2348100e20 0x7f2348101200 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f2338009900 tx=0x7f233802a480 comp rx=0 tx=0).stop 2024-09-12T03:06:30.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.321+0000 7f234e46f640 1 -- 172.21.15.116:0/967859890 shutdown_connections 2024-09-12T03:06:30.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.321+0000 7f234e46f640 1 --2- 172.21.15.116:0/967859890 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2348102ef0 0x7f234810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.321+0000 7f234e46f640 1 --2- 172.21.15.116:0/967859890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2348102570 0x7f23481029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.321+0000 7f234e46f640 1 --2- 172.21.15.116:0/967859890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2348100e20 0x7f2348101200 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.321+0000 7f234e46f640 1 -- 172.21.15.116:0/967859890 >> 172.21.15.116:0/967859890 conn(0x7f23480fc7d0 msgr2=0x7f23480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:30.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.321+0000 7f234e46f640 1 -- 172.21.15.116:0/967859890 shutdown_connections 2024-09-12T03:06:30.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.321+0000 7f234e46f640 1 -- 172.21.15.116:0/967859890 wait complete. 2024-09-12T03:06:30.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.322+0000 7f234e46f640 1 Processor -- start 2024-09-12T03:06:30.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.322+0000 7f234e46f640 1 -- start start 2024-09-12T03:06:30.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.322+0000 7f234e46f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2348100e20 0x7f2348199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:30.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.322+0000 7f234e46f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2348102570 0x7f2348199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:30.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.322+0000 7f234e46f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2348102ef0 0x7f23481a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:30.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.322+0000 7f234e46f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2348110f80 con 0x7f2348100e20 2024-09-12T03:06:30.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.322+0000 7f234e46f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2348110e00 con 0x7f2348102570 2024-09-12T03:06:30.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.322+0000 7f234e46f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2348111100 con 0x7f2348102ef0 2024-09-12T03:06:30.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.322+0000 7f2347fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2348100e20 0x7f2348199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:30.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.322+0000 7f2347fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2348100e20 0x7f2348199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52930/0 (socket says 172.21.15.116:52930) 2024-09-12T03:06:30.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.322+0000 7f2347fff640 1 -- 172.21.15.116:0/2625624148 learned_addr learned my addr 172.21.15.116:0/2625624148 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:30.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.322+0000 7f233ffff640 1 --2- 172.21.15.116:0/2625624148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2348102570 0x7f2348199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:30.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.323+0000 7f234ca0c640 1 --2- 172.21.15.116:0/2625624148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2348102ef0 0x7f23481a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:30.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.323+0000 7f2347fff640 1 -- 172.21.15.116:0/2625624148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2348102ef0 msgr2=0x7f23481a43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:30.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.323+0000 7f2347fff640 1 --2- 172.21.15.116:0/2625624148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2348102ef0 0x7f23481a43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.323+0000 7f2347fff640 1 -- 172.21.15.116:0/2625624148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2348102570 msgr2=0x7f2348199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:30.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.323+0000 7f2347fff640 1 --2- 172.21.15.116:0/2625624148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2348102570 0x7f2348199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.323+0000 7f2347fff640 1 -- 172.21.15.116:0/2625624148 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23481a4900 con 0x7f2348100e20 2024-09-12T03:06:30.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.323+0000 7f233ffff640 1 --2- 172.21.15.116:0/2625624148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2348102570 0x7f2348199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:06:30.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.323+0000 7f2347fff640 1 --2- 172.21.15.116:0/2625624148 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2348100e20 0x7f2348199340 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2338006010 tx=0x7f2338003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:30.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.323+0000 7f2345ffb640 1 -- 172.21.15.116:0/2625624148 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233803f070 con 0x7f2348100e20 2024-09-12T03:06:30.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.323+0000 7f234e46f640 1 -- 172.21.15.116:0/2625624148 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23481a4ad0 con 0x7f2348100e20 2024-09-12T03:06:30.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.323+0000 7f2345ffb640 1 -- 172.21.15.116:0/2625624148 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2338031c30 con 0x7f2348100e20 2024-09-12T03:06:30.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.323+0000 7f2345ffb640 1 -- 172.21.15.116:0/2625624148 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2338035030 con 0x7f2348100e20 2024-09-12T03:06:30.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.323+0000 7f234e46f640 1 -- 172.21.15.116:0/2625624148 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23481a4f80 con 0x7f2348100e20 2024-09-12T03:06:30.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.325+0000 7f2345ffb640 1 -- 172.21.15.116:0/2625624148 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2338042070 con 0x7f2348100e20 2024-09-12T03:06:30.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.326+0000 7f2350848740 1 Processor -- start 2024-09-12T03:06:30.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.326+0000 7f2350848740 1 -- start start 2024-09-12T03:06:30.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.326+0000 7f2350848740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1c1f4d8b0 0x55d1c1f4dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:30.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.326+0000 7f2345ffb640 1 --2- 172.21.15.116:0/2625624148 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f23340a2b80 0x7f23340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:30.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.326+0000 7f2350848740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1c1f4e1d0 0x55d1c1f58550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:30.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.326+0000 7f2350848740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1c1f58a90 0x55d1c1f5ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:30.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.326+0000 7f2350848740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d1c1eaf2e0 con 0x55d1c1f58a90 2024-09-12T03:06:30.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.326+0000 7f2350848740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d1c1e56ce0 con 0x55d1c1f4d8b0 2024-09-12T03:06:30.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.326+0000 7f2350848740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d1c1eab2a0 con 0x55d1c1f4e1d0 2024-09-12T03:06:30.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.326+0000 7f2347fff640 1 --2- 172.21.15.116:0/2625624148 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f23340a2b80 0x7f23340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:30.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.326+0000 7f234ca0c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1c1f58a90 0x55d1c1f5ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:30.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.326+0000 7f2345ffb640 1 -- 172.21.15.116:0/2625624148 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f23380ef770 con 0x7f2348100e20 2024-09-12T03:06:30.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.326+0000 7f234ca0c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1c1f58a90 0x55d1c1f5ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52932/0 (socket says 172.21.15.116:52932) 2024-09-12T03:06:30.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.327+0000 7f234ca0c640 1 -- 172.21.15.116:0/1699791748 learned_addr learned my addr 172.21.15.116:0/1699791748 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:30.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.327+0000 7f234ca0c640 1 --2- 172.21.15.116:0/1699791748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1c1f4d8b0 0x55d1c1f4dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:30.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.327+0000 7f234ca0c640 1 -- 172.21.15.116:0/1699791748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1c1f4e1d0 msgr2=0x55d1c1f58550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:06:30.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.327+0000 7f234ca0c640 1 --2- 172.21.15.116:0/1699791748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1c1f4e1d0 0x55d1c1f58550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.327+0000 7f234ca0c640 1 -- 172.21.15.116:0/1699791748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1c1f4d8b0 msgr2=0x55d1c1f4dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:30.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.327+0000 7f234ca0c640 1 --2- 172.21.15.116:0/1699791748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1c1f4d8b0 0x55d1c1f4dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.327+0000 7f234ca0c640 1 -- 172.21.15.116:0/1699791748 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1c1e3b440 con 0x55d1c1f58a90 2024-09-12T03:06:30.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.327+0000 7f2347fff640 1 --2- 172.21.15.116:0/2625624148 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f23340a2b80 0x7f23340a5040 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f23380bbf30 tx=0x7f2338049dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:30.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.327+0000 7f234ca0c640 1 --2- 172.21.15.116:0/1699791748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1c1f58a90 0x55d1c1f5ae80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f2340012950 tx=0x7f2340009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3cbaeebb36c831 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:30.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.327+0000 7f233e7fc640 1 -- 172.21.15.116:0/1699791748 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2340010770 con 0x55d1c1f58a90 2024-09-12T03:06:30.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.327+0000 7f233e7fc640 1 -- 172.21.15.116:0/1699791748 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2340010df0 con 0x55d1c1f58a90 2024-09-12T03:06:30.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.327+0000 7f233e7fc640 1 -- 172.21.15.116:0/1699791748 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234001edb0 con 0x55d1c1f58a90 2024-09-12T03:06:30.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.328+0000 7f2350848740 1 -- 172.21.15.116:0/1699791748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1c1f58a90 msgr2=0x55d1c1f5ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:30.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.328+0000 7f2350848740 1 --2- 172.21.15.116:0/1699791748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1c1f58a90 0x55d1c1f5ae80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f2340012950 tx=0x7f2340009710 comp rx=0 tx=0).stop 2024-09-12T03:06:30.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.328+0000 7f2350848740 1 -- 172.21.15.116:0/1699791748 shutdown_connections 2024-09-12T03:06:30.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.328+0000 7f2350848740 1 --2- 172.21.15.116:0/1699791748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1c1f58a90 0x55d1c1f5ae80 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.328+0000 7f2350848740 1 --2- 172.21.15.116:0/1699791748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1c1f4e1d0 0x55d1c1f58550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.328+0000 7f2350848740 1 --2- 172.21.15.116:0/1699791748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1c1f4d8b0 0x55d1c1f4dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.328+0000 7f2350848740 1 -- 172.21.15.116:0/1699791748 >> 172.21.15.116:0/1699791748 conn(0x55d1c1f49e40 msgr2=0x55d1c1f4a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:30.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.328+0000 7f2350848740 1 -- 172.21.15.116:0/1699791748 shutdown_connections 2024-09-12T03:06:30.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.328+0000 7f2350848740 1 -- 172.21.15.116:0/1699791748 wait complete. 2024-09-12T03:06:30.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.328+0000 7f2350848740 1 -- start start 2024-09-12T03:06:30.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.329+0000 7f2350848740 1 Processor -- start 2024-09-12T03:06:30.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.329+0000 7f2350848740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1c1f4d8b0 0x55d1c1f58380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:30.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.329+0000 7f2350848740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1c1f4e1d0 0x55d1c1f55630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:30.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.329+0000 7f2350848740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1c1f58a90 0x55d1c1f55b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:30.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.329+0000 7f2350848740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d1c1f5b660 con 0x55d1c1f4d8b0 2024-09-12T03:06:30.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.329+0000 7f2350848740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d1c1e6a050 con 0x55d1c1f58a90 2024-09-12T03:06:30.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.329+0000 7f2350848740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d1c1f5b7e0 con 0x55d1c1f4e1d0 2024-09-12T03:06:30.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.329+0000 7f2350848740 10 client.0 _ll_register_callbacks cb 0x55d1c1eaa160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:30.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.329+0000 7f234ca0c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1c1f4d8b0 0x55d1c1f58380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:30.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.329+0000 7f234ca0c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1c1f4d8b0 0x55d1c1f58380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52946/0 (socket says 172.21.15.116:52946) 2024-09-12T03:06:30.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.329+0000 7f234ca0c640 1 -- 172.21.15.116:0/1789979636 learned_addr learned my addr 172.21.15.116:0/1789979636 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:30.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.329+0000 7f233ffff640 1 --2- 172.21.15.116:0/1789979636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1c1f4e1d0 0x55d1c1f55630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:30.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.329+0000 7f234ca0c640 1 -- 172.21.15.116:0/1789979636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1c1f4e1d0 msgr2=0x55d1c1f55630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:30.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.329+0000 7f234ca0c640 1 --2- 172.21.15.116:0/1789979636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1c1f4e1d0 0x55d1c1f55630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.329+0000 7f234ca0c640 1 -- 172.21.15.116:0/1789979636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1c1f58a90 msgr2=0x55d1c1f55b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:30.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.329+0000 7f234ca0c640 1 --2- 172.21.15.116:0/1789979636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1c1f58a90 0x55d1c1f55b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.329+0000 7f234ca0c640 1 -- 172.21.15.116:0/1789979636 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1c1c203c0 con 0x55d1c1f4d8b0 2024-09-12T03:06:30.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.329+0000 7f234ca0c640 1 --2- 172.21.15.116:0/1789979636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1c1f4d8b0 0x55d1c1f58380 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2340002230 tx=0x7f23400126f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e22b90ddabcfecb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:30.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.329+0000 7f23277fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:06:30.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.330+0000 7f23277fe640 1 -- 172.21.15.116:0/1789979636 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234001ed30 con 0x55d1c1f4d8b0 2024-09-12T03:06:30.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.330+0000 7f23277fe640 1 -- 172.21.15.116:0/1789979636 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f2340004a40 con 0x55d1c1f4d8b0 2024-09-12T03:06:30.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.330+0000 7f2350848740 1 -- 172.21.15.116:0/1789979636 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2340010df0 con 0x55d1c1f4d8b0 2024-09-12T03:06:30.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.330+0000 7f23277fe640 1 -- 172.21.15.116:0/1789979636 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f234002f070 con 0x55d1c1f4d8b0 2024-09-12T03:06:30.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.330+0000 7f2350848740 10 client.16152 fetch_fsmap learned FSMap version 25 2024-09-12T03:06:30.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.330+0000 7f2350848740 1 -- 172.21.15.116:0/1789979636 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d1c1f67000 con 0x55d1c1f4d8b0 2024-09-12T03:06:30.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.330+0000 7f23277fe640 1 -- 172.21.15.116:0/1789979636 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2340010df0 con 0x55d1c1f4d8b0 2024-09-12T03:06:30.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.330+0000 7f23277fe640 1 -- 172.21.15.116:0/1789979636 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f234000c070 con 0x55d1c1f4d8b0 2024-09-12T03:06:30.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.330+0000 7f2350848740 10 client.16152 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:06:30.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.330+0000 7f2350848740 10 client.16152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:06:30.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.330+0000 7f2350848740 20 client.16152 populate_metadata read hostname 'smithi116' 2024-09-12T03:06:30.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.330+0000 7f2350848740 1 --2- 172.21.15.116:0/1789979636 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d1c1f68400 0x55d1c1f6a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:30.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.330+0000 7f2350848740 4 client.16152 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:06:30.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.330+0000 7f2350848740 1 -- 172.21.15.116:0/1789979636 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1c1ebc850 con 0x55d1c1f68400 2024-09-12T03:06:30.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.331+0000 7f233ffff640 1 --2- 172.21.15.116:0/1789979636 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d1c1f68400 0x55d1c1f6a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:30.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.331+0000 7f233ffff640 1 --2- 172.21.15.116:0/1789979636 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d1c1f68400 0x55d1c1f6a7f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ed5518dd26600cc server_cookie=5d5261bc21d5e15 in_seq=0 out_seq=0 2024-09-12T03:06:30.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.331+0000 7f23277fe640 10 client.16152 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:30.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.332+0000 7f23277fe640 1 -- 172.21.15.116:0/1789979636 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1c1ebc850 con 0x55d1c1f68400 2024-09-12T03:06:30.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.332+0000 7f23277fe640 10 client.16152 handle_command_reply: tid=0 2024-09-12T03:06:30.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.332+0000 7f2350848740 1 client.16152 shutdown 2024-09-12T03:06:30.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.332+0000 7f2350848740 1 -- 172.21.15.116:0/1789979636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1c1f4d8b0 msgr2=0x55d1c1f58380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:30.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.332+0000 7f2350848740 1 --2- 172.21.15.116:0/1789979636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1c1f4d8b0 0x55d1c1f58380 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2340002230 tx=0x7f23400126f0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.332+0000 7f2350848740 1 -- 172.21.15.116:0/1789979636 shutdown_connections 2024-09-12T03:06:30.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.332+0000 7f2350848740 1 --2- 172.21.15.116:0/1789979636 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d1c1f68400 0x55d1c1f6a7f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.332+0000 7f2350848740 1 --2- 172.21.15.116:0/1789979636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1c1f58a90 0x55d1c1f55b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.332+0000 7f2350848740 1 --2- 172.21.15.116:0/1789979636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1c1f4e1d0 0x55d1c1f55630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.332+0000 7f2350848740 1 --2- 172.21.15.116:0/1789979636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1c1f4d8b0 0x55d1c1f58380 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.332+0000 7f2350848740 1 -- 172.21.15.116:0/1789979636 >> 172.21.15.116:0/1789979636 conn(0x55d1c1f49e40 msgr2=0x55d1c1f44b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:30.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.332+0000 7f23277fe640 0 client.16152 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:30.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.332+0000 7f2350848740 1 -- 172.21.15.116:0/1789979636 shutdown_connections 2024-09-12T03:06:30.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.332+0000 7f2350848740 1 -- 172.21.15.116:0/1789979636 wait complete. 2024-09-12T03:06:30.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.332+0000 7f2350848740 20 client.16152 trim_cache size 0 max 16384 2024-09-12T03:06:30.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.349+0000 7f2350848740 1 Processor -- start 2024-09-12T03:06:30.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.349+0000 7f2350848740 1 -- start start 2024-09-12T03:06:30.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.349+0000 7f2350848740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1c1f4d8b0 0x55d1c1f648f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:30.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.349+0000 7f2350848740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1c1f4e1d0 0x55d1c1f6ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:30.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.349+0000 7f2350848740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1c1f58a90 0x55d1c1f70450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:30.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.349+0000 7f2350848740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f23481adbe0 con 0x55d1c1f4d8b0 2024-09-12T03:06:30.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.349+0000 7f2350848740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f23481ab4b0 con 0x55d1c1f58a90 2024-09-12T03:06:30.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.349+0000 7f2350848740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f23481a9300 con 0x55d1c1f4e1d0 2024-09-12T03:06:30.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.350+0000 7f234ca0c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1c1f4d8b0 0x55d1c1f648f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:30.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.350+0000 7f234ca0c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1c1f4d8b0 0x55d1c1f648f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52960/0 (socket says 172.21.15.116:52960) 2024-09-12T03:06:30.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.350+0000 7f234ca0c640 1 -- 172.21.15.116:0/2656477903 learned_addr learned my addr 172.21.15.116:0/2656477903 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:30.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.350+0000 7f233ffff640 1 --2- 172.21.15.116:0/2656477903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1c1f4e1d0 0x55d1c1f6ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:30.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.350+0000 7f234ca0c640 1 -- 172.21.15.116:0/2656477903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1c1f4e1d0 msgr2=0x55d1c1f6ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:30.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.350+0000 7f234ca0c640 1 --2- 172.21.15.116:0/2656477903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1c1f4e1d0 0x55d1c1f6ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.350+0000 7f234ca0c640 1 -- 172.21.15.116:0/2656477903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1c1f58a90 msgr2=0x55d1c1f70450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:30.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.350+0000 7f234ca0c640 1 --2- 172.21.15.116:0/2656477903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1c1f58a90 0x55d1c1f70450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.350+0000 7f234ca0c640 1 -- 172.21.15.116:0/2656477903 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1c1f49c70 con 0x55d1c1f4d8b0 2024-09-12T03:06:30.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.350+0000 7f234ca0c640 1 --2- 172.21.15.116:0/2656477903 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1c1f4d8b0 0x55d1c1f648f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2340012760 tx=0x7f234001e6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4433fc27511024d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:30.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.350+0000 7f233dffb640 1 -- 172.21.15.116:0/2656477903 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2340039070 con 0x55d1c1f4d8b0 2024-09-12T03:06:30.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.350+0000 7f233dffb640 1 -- 172.21.15.116:0/2656477903 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2340023070 con 0x55d1c1f4d8b0 2024-09-12T03:06:30.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.350+0000 7f233dffb640 1 -- 172.21.15.116:0/2656477903 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23400109a0 con 0x55d1c1f4d8b0 2024-09-12T03:06:30.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.351+0000 7f2350848740 1 -- 172.21.15.116:0/2656477903 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1c1f4d8b0 msgr2=0x55d1c1f648f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:30.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.351+0000 7f2350848740 1 --2- 172.21.15.116:0/2656477903 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1c1f4d8b0 0x55d1c1f648f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2340012760 tx=0x7f234001e6b0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.351+0000 7f2350848740 1 -- 172.21.15.116:0/2656477903 shutdown_connections 2024-09-12T03:06:30.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.351+0000 7f2350848740 1 --2- 172.21.15.116:0/2656477903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1c1f58a90 0x55d1c1f70450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.351+0000 7f2350848740 1 --2- 172.21.15.116:0/2656477903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1c1f4e1d0 0x55d1c1f6ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.351+0000 7f2350848740 1 --2- 172.21.15.116:0/2656477903 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1c1f4d8b0 0x55d1c1f648f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.351+0000 7f2350848740 1 -- 172.21.15.116:0/2656477903 >> 172.21.15.116:0/2656477903 conn(0x55d1c1f49e40 msgr2=0x55d1c1f53f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:30.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.351+0000 7f2350848740 1 -- 172.21.15.116:0/2656477903 shutdown_connections 2024-09-12T03:06:30.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.351+0000 7f2350848740 1 -- 172.21.15.116:0/2656477903 wait complete. 2024-09-12T03:06:30.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.351+0000 7f2350848740 1 -- start start 2024-09-12T03:06:30.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.352+0000 7f2350848740 1 Processor -- start 2024-09-12T03:06:30.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.352+0000 7f2350848740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1c1f4d8b0 0x55d1c1f60f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:30.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.352+0000 7f2350848740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1c1f4e1d0 0x55d1c1f5c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:30.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.352+0000 7f2350848740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1c1f58a90 0x55d1c1f5c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:30.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.352+0000 7f2350848740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f23481a9970 con 0x55d1c1f4e1d0 2024-09-12T03:06:30.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.352+0000 7f2350848740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f23481aade0 con 0x55d1c1f4d8b0 2024-09-12T03:06:30.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.352+0000 7f2350848740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f23481aa040 con 0x55d1c1f58a90 2024-09-12T03:06:30.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.352+0000 7f2350848740 10 client.0 _ll_register_callbacks cb 0x7f2340012410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:30.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.352+0000 7f233ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1c1f4e1d0 0x55d1c1f5c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:30.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.352+0000 7f233ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1c1f4e1d0 0x55d1c1f5c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52976/0 (socket says 172.21.15.116:52976) 2024-09-12T03:06:30.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.352+0000 7f233ffff640 1 -- 172.21.15.116:0/620817302 learned_addr learned my addr 172.21.15.116:0/620817302 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:30.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.352+0000 7f234ca0c640 1 --2- 172.21.15.116:0/620817302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1c1f58a90 0x55d1c1f5c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:30.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.352+0000 7f234ca0c640 1 --2- 172.21.15.116:0/620817302 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1c1f4d8b0 0x55d1c1f60f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:30.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.352+0000 7f233ffff640 1 -- 172.21.15.116:0/620817302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1c1f58a90 msgr2=0x55d1c1f5c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:30.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.352+0000 7f233ffff640 1 --2- 172.21.15.116:0/620817302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1c1f58a90 0x55d1c1f5c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.352+0000 7f233ffff640 1 -- 172.21.15.116:0/620817302 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1c1f4d8b0 msgr2=0x55d1c1f60f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:30.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.352+0000 7f233ffff640 1 --2- 172.21.15.116:0/620817302 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1c1f4d8b0 0x55d1c1f60f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.352+0000 7f233ffff640 1 -- 172.21.15.116:0/620817302 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1c1f5cdb0 con 0x55d1c1f4e1d0 2024-09-12T03:06:30.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.352+0000 7f234ca0c640 1 --2- 172.21.15.116:0/620817302 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1c1f4d8b0 0x55d1c1f60f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:06:30.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.353+0000 7f233ffff640 1 --2- 172.21.15.116:0/620817302 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1c1f4e1d0 0x55d1c1f5c2b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f234806ab00 tx=0x7f2330013930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=993784efb315a615 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:30.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.353+0000 7f23277fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:06:30.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.353+0000 7f23277fe640 1 -- 172.21.15.116:0/620817302 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2330023030 con 0x55d1c1f4e1d0 2024-09-12T03:06:30.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.353+0000 7f23277fe640 1 -- 172.21.15.116:0/620817302 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f2330004260 con 0x55d1c1f4e1d0 2024-09-12T03:06:30.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.353+0000 7f2350848740 1 -- 172.21.15.116:0/620817302 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2340023070 con 0x55d1c1f4e1d0 2024-09-12T03:06:30.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.353+0000 7f23277fe640 1 -- 172.21.15.116:0/620817302 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2330004460 con 0x55d1c1f4e1d0 2024-09-12T03:06:30.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.353+0000 7f2350848740 10 client.16158 fetch_fsmap learned FSMap version 25 2024-09-12T03:06:30.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.353+0000 7f2350848740 1 -- 172.21.15.116:0/620817302 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d1c1fd2d50 con 0x55d1c1f4e1d0 2024-09-12T03:06:30.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.353+0000 7f23277fe640 1 -- 172.21.15.116:0/620817302 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2340023070 con 0x55d1c1f4e1d0 2024-09-12T03:06:30.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.353+0000 7f23277fe640 1 -- 172.21.15.116:0/620817302 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f233001f7e0 con 0x55d1c1f4e1d0 2024-09-12T03:06:30.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.353+0000 7f2350848740 10 client.16158 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:06:30.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.353+0000 7f2350848740 10 client.16158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:06:30.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.353+0000 7f2350848740 20 client.16158 populate_metadata read hostname 'smithi116' 2024-09-12T03:06:30.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.353+0000 7f2350848740 1 --2- 172.21.15.116:0/620817302 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d1c1fd3550 0x55d1c1fd5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:30.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.353+0000 7f2350848740 4 client.16158 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:06:30.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.353+0000 7f2350848740 1 -- 172.21.15.116:0/620817302 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d1c1f55f00 con 0x55d1c1fd3550 2024-09-12T03:06:30.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.354+0000 7f234ca0c640 1 --2- 172.21.15.116:0/620817302 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d1c1fd3550 0x55d1c1fd5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:30.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.354+0000 7f23277fe640 10 client.16158 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:30.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.354+0000 7f234ca0c640 1 --2- 172.21.15.116:0/620817302 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d1c1fd3550 0x55d1c1fd5970 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=413cf0318a880c53 server_cookie=a9b458e8d7a367f8 in_seq=0 out_seq=0 2024-09-12T03:06:30.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.354+0000 7f23277fe640 1 -- 172.21.15.116:0/620817302 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d1c1f55f00 con 0x55d1c1fd3550 2024-09-12T03:06:30.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.354+0000 7f23277fe640 10 client.16158 handle_command_reply: tid=0 2024-09-12T03:06:30.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.354+0000 7f2350848740 1 client.16158 shutdown 2024-09-12T03:06:30.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.355+0000 7f2350848740 1 -- 172.21.15.116:0/620817302 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1c1f4e1d0 msgr2=0x55d1c1f5c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:30.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.355+0000 7f2350848740 1 --2- 172.21.15.116:0/620817302 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1c1f4e1d0 0x55d1c1f5c2b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f234806ab00 tx=0x7f2330013930 comp rx=0 tx=0).stop 2024-09-12T03:06:30.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.355+0000 7f2350848740 1 -- 172.21.15.116:0/620817302 shutdown_connections 2024-09-12T03:06:30.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.355+0000 7f2350848740 1 --2- 172.21.15.116:0/620817302 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d1c1fd3550 0x55d1c1fd5970 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.355+0000 7f2350848740 1 --2- 172.21.15.116:0/620817302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1c1f58a90 0x55d1c1f5c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.355+0000 7f2350848740 1 --2- 172.21.15.116:0/620817302 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d1c1f4e1d0 0x55d1c1f5c2b0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.355+0000 7f2350848740 1 --2- 172.21.15.116:0/620817302 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d1c1f4d8b0 0x55d1c1f60f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.355+0000 7f2350848740 1 -- 172.21.15.116:0/620817302 >> 172.21.15.116:0/620817302 conn(0x55d1c1f49e40 msgr2=0x55d1c1f44b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:30.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.355+0000 7f23277fe640 0 client.16158 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:30.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.355+0000 7f2350848740 1 -- 172.21.15.116:0/620817302 shutdown_connections 2024-09-12T03:06:30.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.355+0000 7f2350848740 1 -- 172.21.15.116:0/620817302 wait complete. 2024-09-12T03:06:30.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.355+0000 7f2350848740 20 client.16158 trim_cache size 0 max 16384 2024-09-12T03:06:30.370 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:06:30.370 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:06:30.370 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:06:30.371 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:06:30.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.355+0000 7f234e46f640 1 -- 172.21.15.116:0/2625624148 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f23340a2b80 msgr2=0x7f23340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:30.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.355+0000 7f234e46f640 1 --2- 172.21.15.116:0/2625624148 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f23340a2b80 0x7f23340a5040 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f23380bbf30 tx=0x7f2338049dd0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.355+0000 7f234e46f640 1 -- 172.21.15.116:0/2625624148 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2348100e20 msgr2=0x7f2348199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:30.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.355+0000 7f234e46f640 1 --2- 172.21.15.116:0/2625624148 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2348100e20 0x7f2348199340 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2338006010 tx=0x7f2338003c10 comp rx=0 tx=0).stop 2024-09-12T03:06:30.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.355+0000 7f234e46f640 1 -- 172.21.15.116:0/2625624148 shutdown_connections 2024-09-12T03:06:30.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.355+0000 7f234e46f640 1 --2- 172.21.15.116:0/2625624148 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f23340a2b80 0x7f23340a5040 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.355+0000 7f234e46f640 1 --2- 172.21.15.116:0/2625624148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2348102ef0 0x7f23481a43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.356+0000 7f234e46f640 1 --2- 172.21.15.116:0/2625624148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2348102570 0x7f2348199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.356+0000 7f234e46f640 1 --2- 172.21.15.116:0/2625624148 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2348100e20 0x7f2348199340 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:30.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.356+0000 7f234e46f640 1 -- 172.21.15.116:0/2625624148 >> 172.21.15.116:0/2625624148 conn(0x7f23480fc7d0 msgr2=0x7f23480fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:30.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.356+0000 7f234e46f640 1 -- 172.21.15.116:0/2625624148 shutdown_connections 2024-09-12T03:06:30.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:30.356+0000 7f234e46f640 1 -- 172.21.15.116:0/2625624148 wait complete. 2024-09-12T03:06:30.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:30 smithi116 ceph-mon[31410]: pgmap v443: 97 pgs: 97 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:06:30.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:30 smithi183 ceph-mon[35535]: pgmap v443: 97 pgs: 97 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:06:30.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:30 smithi177 ceph-mon[35305]: pgmap v443: 97 pgs: 97 active+clean; 1.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:06:31.082 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:06:31.083 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:06:31.330 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:06:31.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.809+0000 7fddcdbed640 1 -- 172.21.15.116:0/1337704886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddc8102da0 msgr2=0x7fddc8103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:31.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.809+0000 7fddcdbed640 1 --2- 172.21.15.116:0/1337704886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddc8102da0 0x7fddc8103180 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fddb8009900 tx=0x7fddb802a480 comp rx=0 tx=0).stop 2024-09-12T03:06:31.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.809+0000 7fddcdbed640 1 -- 172.21.15.116:0/1337704886 shutdown_connections 2024-09-12T03:06:31.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.809+0000 7fddcdbed640 1 --2- 172.21.15.116:0/1337704886 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddc810c180 0x7fddc810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:31.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.809+0000 7fddcdbed640 1 --2- 172.21.15.116:0/1337704886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fddc81036c0 0x7fddc810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:31.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.809+0000 7fddcdbed640 1 --2- 172.21.15.116:0/1337704886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddc8102da0 0x7fddc8103180 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:31.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.809+0000 7fddcdbed640 1 -- 172.21.15.116:0/1337704886 >> 172.21.15.116:0/1337704886 conn(0x7fddc80fc7a0 msgr2=0x7fddc80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:31.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.809+0000 7fddcdbed640 1 -- 172.21.15.116:0/1337704886 shutdown_connections 2024-09-12T03:06:31.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.809+0000 7fddcdbed640 1 -- 172.21.15.116:0/1337704886 wait complete. 2024-09-12T03:06:31.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.810+0000 7fddcdbed640 1 Processor -- start 2024-09-12T03:06:31.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.810+0000 7fddcdbed640 1 -- start start 2024-09-12T03:06:31.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.810+0000 7fddcdbed640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddc8102da0 0x7fddc8198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:31.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.810+0000 7fddcdbed640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddc81036c0 0x7fddc81989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:31.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.810+0000 7fddcdbed640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fddc810c180 0x7fddc819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:31.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.810+0000 7fddcdbed640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fddc8111fc0 con 0x7fddc81036c0 2024-09-12T03:06:31.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.810+0000 7fddcdbed640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fddc8111e40 con 0x7fddc8102da0 2024-09-12T03:06:31.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.810+0000 7fddcdbed640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fddc8112140 con 0x7fddc810c180 2024-09-12T03:06:31.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.810+0000 7fddc6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddc81036c0 0x7fddc81989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:31.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.810+0000 7fddc6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddc81036c0 0x7fddc81989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42800/0 (socket says 172.21.15.116:42800) 2024-09-12T03:06:31.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.810+0000 7fddc6ffd640 1 -- 172.21.15.116:0/3933861425 learned_addr learned my addr 172.21.15.116:0/3933861425 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:31.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.810+0000 7fddc77fe640 1 --2- 172.21.15.116:0/3933861425 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddc8102da0 0x7fddc8198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:31.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.810+0000 7fddc7fff640 1 --2- 172.21.15.116:0/3933861425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fddc810c180 0x7fddc819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:31.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.810+0000 7fddc6ffd640 1 -- 172.21.15.116:0/3933861425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fddc810c180 msgr2=0x7fddc819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:31.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.810+0000 7fddc6ffd640 1 --2- 172.21.15.116:0/3933861425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fddc810c180 0x7fddc819cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:31.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.810+0000 7fddc6ffd640 1 -- 172.21.15.116:0/3933861425 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddc8102da0 msgr2=0x7fddc8198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:31.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.810+0000 7fddc6ffd640 1 --2- 172.21.15.116:0/3933861425 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddc8102da0 0x7fddc8198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:31.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.810+0000 7fddc6ffd640 1 -- 172.21.15.116:0/3933861425 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc819d430 con 0x7fddc81036c0 2024-09-12T03:06:31.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.810+0000 7fddc77fe640 1 --2- 172.21.15.116:0/3933861425 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddc8102da0 0x7fddc8198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:06:31.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.811+0000 7fddc6ffd640 1 --2- 172.21.15.116:0/3933861425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddc81036c0 0x7fddc81989c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fddbc00c970 tx=0x7fddbc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:31.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.811+0000 7fddc4ff9640 1 -- 172.21.15.116:0/3933861425 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddbc007bb0 con 0x7fddc81036c0 2024-09-12T03:06:31.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.811+0000 7fddcdbed640 1 -- 172.21.15.116:0/3933861425 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddc819d720 con 0x7fddc81036c0 2024-09-12T03:06:31.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.811+0000 7fddc4ff9640 1 -- 172.21.15.116:0/3933861425 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddbc007d50 con 0x7fddc81036c0 2024-09-12T03:06:31.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.811+0000 7fddc4ff9640 1 -- 172.21.15.116:0/3933861425 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddbc00f450 con 0x7fddc81036c0 2024-09-12T03:06:31.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.811+0000 7fddcdbed640 1 -- 172.21.15.116:0/3933861425 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddc81a5000 con 0x7fddc81036c0 2024-09-12T03:06:31.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.812+0000 7fddcdbed640 1 -- 172.21.15.116:0/3933861425 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddc8103180 con 0x7fddc81036c0 2024-09-12T03:06:31.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.813+0000 7fddc4ff9640 1 -- 172.21.15.116:0/3933861425 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fddbc005020 con 0x7fddc81036c0 2024-09-12T03:06:31.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.813+0000 7fddc4ff9640 1 --2- 172.21.15.116:0/3933861425 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fddb00a2c00 0x7fddb00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:31.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.814+0000 7fddc77fe640 1 --2- 172.21.15.116:0/3933861425 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fddb00a2c00 0x7fddb00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:31.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.814+0000 7fddc4ff9640 1 -- 172.21.15.116:0/3933861425 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fddbc0cee20 con 0x7fddc81036c0 2024-09-12T03:06:31.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.814+0000 7fddc77fe640 1 --2- 172.21.15.116:0/3933861425 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fddb00a2c00 0x7fddb00a50c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fddb8000c00 tx=0x7fddb8032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:31.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:31.818+0000 7fddc4ff9640 1 -- 172.21.15.116:0/3933861425 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fddbc014030 con 0x7fddc81036c0 2024-09-12T03:06:32.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:32.017+0000 7fddcdbed640 1 -- 172.21.15.116:0/3933861425 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fddc8107280 con 0x7fddc81036c0 2024-09-12T03:06:32.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:32.018+0000 7fddc4ff9640 1 -- 172.21.15.116:0/3933861425 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7fddbc09bdb0 con 0x7fddc81036c0 2024-09-12T03:06:32.018 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:06:32.019 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:06:32.019 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:06:32.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:32.020+0000 7fddcdbed640 1 -- 172.21.15.116:0/3933861425 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fddb00a2c00 msgr2=0x7fddb00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:32.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:32.020+0000 7fddcdbed640 1 --2- 172.21.15.116:0/3933861425 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fddb00a2c00 0x7fddb00a50c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fddb8000c00 tx=0x7fddb8032d50 comp rx=0 tx=0).stop 2024-09-12T03:06:32.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:32.020+0000 7fddcdbed640 1 -- 172.21.15.116:0/3933861425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddc81036c0 msgr2=0x7fddc81989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:32.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:32.020+0000 7fddcdbed640 1 --2- 172.21.15.116:0/3933861425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddc81036c0 0x7fddc81989c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fddbc00c970 tx=0x7fddbc00ce30 comp rx=0 tx=0).stop 2024-09-12T03:06:32.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:32.020+0000 7fddcdbed640 1 -- 172.21.15.116:0/3933861425 shutdown_connections 2024-09-12T03:06:32.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:32.020+0000 7fddcdbed640 1 --2- 172.21.15.116:0/3933861425 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fddb00a2c00 0x7fddb00a50c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:32.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:32.020+0000 7fddcdbed640 1 --2- 172.21.15.116:0/3933861425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fddc810c180 0x7fddc819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:32.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:32.020+0000 7fddcdbed640 1 --2- 172.21.15.116:0/3933861425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddc81036c0 0x7fddc81989c0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:32.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:32.020+0000 7fddcdbed640 1 --2- 172.21.15.116:0/3933861425 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fddc8102da0 0x7fddc8198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:32.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:32.020+0000 7fddcdbed640 1 -- 172.21.15.116:0/3933861425 >> 172.21.15.116:0/3933861425 conn(0x7fddc80fc7a0 msgr2=0x7fddc80fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:32.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:32.020+0000 7fddcdbed640 1 -- 172.21.15.116:0/3933861425 shutdown_connections 2024-09-12T03:06:32.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:32.020+0000 7fddcdbed640 1 -- 172.21.15.116:0/3933861425 wait complete. 2024-09-12T03:06:32.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:32 smithi116 ceph-mon[31410]: pgmap v444: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T03:06:32.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:32 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3933861425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:06:32.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:32 smithi183 ceph-mon[35535]: pgmap v444: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T03:06:32.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:32 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3933861425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:06:32.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:32 smithi177 ceph-mon[35305]: pgmap v444: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T03:06:32.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:32 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3933861425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:06:32.847 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:06:33.092 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:06:33.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.604+0000 7fcf658e9640 1 -- 172.21.15.116:0/2681830686 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf60106a80 msgr2=0x7fcf60106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:33.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.604+0000 7fcf658e9640 1 --2- 172.21.15.116:0/2681830686 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf60106a80 0x7fcf60106e60 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fcf48009930 tx=0x7fcf4802a420 comp rx=0 tx=0).stop 2024-09-12T03:06:33.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.604+0000 7fcf658e9640 1 -- 172.21.15.116:0/2681830686 shutdown_connections 2024-09-12T03:06:33.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.604+0000 7fcf658e9640 1 --2- 172.21.15.116:0/2681830686 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf60101420 0x7fcf6010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.604+0000 7fcf658e9640 1 --2- 172.21.15.116:0/2681830686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf60100a80 0x7fcf60100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.604+0000 7fcf658e9640 1 --2- 172.21.15.116:0/2681830686 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf60106a80 0x7fcf60106e60 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.604+0000 7fcf658e9640 1 -- 172.21.15.116:0/2681830686 >> 172.21.15.116:0/2681830686 conn(0x7fcf600fc7a0 msgr2=0x7fcf600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:33.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.604+0000 7fcf658e9640 1 -- 172.21.15.116:0/2681830686 shutdown_connections 2024-09-12T03:06:33.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.604+0000 7fcf658e9640 1 -- 172.21.15.116:0/2681830686 wait complete. 2024-09-12T03:06:33.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.604+0000 7fcf658e9640 1 Processor -- start 2024-09-12T03:06:33.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.605+0000 7fcf658e9640 1 -- start start 2024-09-12T03:06:33.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.605+0000 7fcf658e9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf60100a80 0x7fcf601974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:33.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.605+0000 7fcf658e9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf60101420 0x7fcf60197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:33.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.605+0000 7fcf658e9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf60106a80 0x7fcf6019bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:33.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.605+0000 7fcf658e9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcf60110f20 con 0x7fcf60101420 2024-09-12T03:06:33.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.605+0000 7fcf658e9640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcf60110da0 con 0x7fcf60106a80 2024-09-12T03:06:33.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.605+0000 7fcf658e9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601110a0 con 0x7fcf60100a80 2024-09-12T03:06:33.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.605+0000 7fcf5e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf60101420 0x7fcf60197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:33.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.605+0000 7fcf5e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf60101420 0x7fcf60197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42828/0 (socket says 172.21.15.116:42828) 2024-09-12T03:06:33.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.605+0000 7fcf5e7fc640 1 -- 172.21.15.116:0/2745170730 learned_addr learned my addr 172.21.15.116:0/2745170730 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:33.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.605+0000 7fcf5effd640 1 --2- 172.21.15.116:0/2745170730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf60100a80 0x7fcf601974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:33.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.605+0000 7fcf5f7fe640 1 --2- 172.21.15.116:0/2745170730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf60106a80 0x7fcf6019bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:33.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.605+0000 7fcf5e7fc640 1 -- 172.21.15.116:0/2745170730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf60100a80 msgr2=0x7fcf601974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:33.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.605+0000 7fcf5e7fc640 1 --2- 172.21.15.116:0/2745170730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf60100a80 0x7fcf601974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.605+0000 7fcf5e7fc640 1 -- 172.21.15.116:0/2745170730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf60106a80 msgr2=0x7fcf6019bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:33.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.605+0000 7fcf5e7fc640 1 --2- 172.21.15.116:0/2745170730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf60106a80 0x7fcf6019bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.605+0000 7fcf5e7fc640 1 -- 172.21.15.116:0/2745170730 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf6019c490 con 0x7fcf60101420 2024-09-12T03:06:33.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.606+0000 7fcf5e7fc640 1 --2- 172.21.15.116:0/2745170730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf60101420 0x7fcf60197a20 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000b4a0 tx=0x7fcf5000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:33.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.606+0000 7fcf648e7640 1 -- 172.21.15.116:0/2745170730 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50014070 con 0x7fcf60101420 2024-09-12T03:06:33.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.606+0000 7fcf648e7640 1 -- 172.21.15.116:0/2745170730 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf50010af0 con 0x7fcf60101420 2024-09-12T03:06:33.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.606+0000 7fcf658e9640 1 -- 172.21.15.116:0/2745170730 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf6019c780 con 0x7fcf60101420 2024-09-12T03:06:33.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.606+0000 7fcf648e7640 1 -- 172.21.15.116:0/2745170730 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50017d10 con 0x7fcf60101420 2024-09-12T03:06:33.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.606+0000 7fcf658e9640 1 -- 172.21.15.116:0/2745170730 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf601a4060 con 0x7fcf60101420 2024-09-12T03:06:33.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.608+0000 7fcf648e7640 1 -- 172.21.15.116:0/2745170730 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fcf50017070 con 0x7fcf60101420 2024-09-12T03:06:33.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.608+0000 7fcf67cc2740 1 Processor -- start 2024-09-12T03:06:33.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.608+0000 7fcf67cc2740 1 -- start start 2024-09-12T03:06:33.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.608+0000 7fcf67cc2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632fe76fd50 0x5632fe76e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:33.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.608+0000 7fcf67cc2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632fe76e7e0 0x5632fe76ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:33.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.608+0000 7fcf67cc2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632fe779580 0x5632fe77b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:33.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.609+0000 7fcf67cc2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5632fe6d22f0 con 0x5632fe76e7e0 2024-09-12T03:06:33.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.609+0000 7fcf67cc2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5632fe679cf0 con 0x5632fe779580 2024-09-12T03:06:33.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.609+0000 7fcf67cc2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5632fe6ce2b0 con 0x5632fe76fd50 2024-09-12T03:06:33.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.609+0000 7fcf5effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632fe76e7e0 0x5632fe76ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:33.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.609+0000 7fcf5effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632fe76e7e0 0x5632fe76ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42842/0 (socket says 172.21.15.116:42842) 2024-09-12T03:06:33.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.609+0000 7fcf5effd640 1 -- 172.21.15.116:0/3160864344 learned_addr learned my addr 172.21.15.116:0/3160864344 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:33.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.609+0000 7fcf648e7640 1 --2- 172.21.15.116:0/2745170730 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcf4c0a2b80 0x7fcf4c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:33.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.609+0000 7fcf5e7fc640 1 --2- 172.21.15.116:0/3160864344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632fe779580 0x5632fe77b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:33.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.609+0000 7fcf5f7fe640 1 --2- 172.21.15.116:0/3160864344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632fe76fd50 0x5632fe76e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:33.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.609+0000 7fcf5f7fe640 1 --2- 172.21.15.116:0/2745170730 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcf4c0a2b80 0x7fcf4c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:33.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.609+0000 7fcf5effd640 1 -- 172.21.15.116:0/3160864344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632fe76fd50 msgr2=0x5632fe76e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:33.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.609+0000 7fcf5effd640 1 --2- 172.21.15.116:0/3160864344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632fe76fd50 0x5632fe76e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.609+0000 7fcf5effd640 1 -- 172.21.15.116:0/3160864344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632fe779580 msgr2=0x5632fe77b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:33.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.609+0000 7fcf5effd640 1 --2- 172.21.15.116:0/3160864344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632fe779580 0x5632fe77b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.609+0000 7fcf5effd640 1 -- 172.21.15.116:0/3160864344 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632fe65e3a0 con 0x5632fe76e7e0 2024-09-12T03:06:33.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.609+0000 7fcf5f7fe640 1 --2- 172.21.15.116:0/3160864344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632fe76fd50 0x5632fe76e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:06:33.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.609+0000 7fcf648e7640 1 -- 172.21.15.116:0/2745170730 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fcf500ce450 con 0x7fcf60101420 2024-09-12T03:06:33.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.609+0000 7fcf5effd640 1 --2- 172.21.15.116:0/3160864344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632fe76e7e0 0x5632fe76ec20 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fcf4802a940 tx=0x7fcf480316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f924503d99950c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:33.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.609+0000 7fcf5f7fe640 1 --2- 172.21.15.116:0/2745170730 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcf4c0a2b80 0x7fcf4c0a5040 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcf580070b0 tx=0x7fcf5800c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:33.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.609+0000 7fcf55ffb640 1 -- 172.21.15.116:0/3160864344 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf48040070 con 0x5632fe76e7e0 2024-09-12T03:06:33.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.609+0000 7fcf55ffb640 1 -- 172.21.15.116:0/3160864344 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf48005020 con 0x5632fe76e7e0 2024-09-12T03:06:33.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.609+0000 7fcf55ffb640 1 -- 172.21.15.116:0/3160864344 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf48035040 con 0x5632fe76e7e0 2024-09-12T03:06:33.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.610+0000 7fcf67cc2740 1 -- 172.21.15.116:0/3160864344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632fe76e7e0 msgr2=0x5632fe76ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:33.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.610+0000 7fcf67cc2740 1 --2- 172.21.15.116:0/3160864344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632fe76e7e0 0x5632fe76ec20 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fcf4802a940 tx=0x7fcf480316e0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.610+0000 7fcf67cc2740 1 -- 172.21.15.116:0/3160864344 shutdown_connections 2024-09-12T03:06:33.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.610+0000 7fcf67cc2740 1 --2- 172.21.15.116:0/3160864344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632fe779580 0x5632fe77b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.610+0000 7fcf67cc2740 1 --2- 172.21.15.116:0/3160864344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632fe76e7e0 0x5632fe76ec20 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.610+0000 7fcf67cc2740 1 --2- 172.21.15.116:0/3160864344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632fe76fd50 0x5632fe76e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.610+0000 7fcf67cc2740 1 -- 172.21.15.116:0/3160864344 >> 172.21.15.116:0/3160864344 conn(0x5632fe735310 msgr2=0x5632fe76ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:33.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.610+0000 7fcf67cc2740 1 -- 172.21.15.116:0/3160864344 shutdown_connections 2024-09-12T03:06:33.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.610+0000 7fcf67cc2740 1 -- 172.21.15.116:0/3160864344 wait complete. 2024-09-12T03:06:33.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.610+0000 7fcf67cc2740 1 -- start start 2024-09-12T03:06:33.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.611+0000 7fcf67cc2740 1 Processor -- start 2024-09-12T03:06:33.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.611+0000 7fcf67cc2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632fe76e7e0 0x5632fe778f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:33.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.611+0000 7fcf67cc2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632fe76fd50 0x5632fe776220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:33.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.611+0000 7fcf67cc2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632fe779580 0x5632fe776810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:33.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.611+0000 7fcf67cc2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5632fe77c220 con 0x5632fe76e7e0 2024-09-12T03:06:33.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.611+0000 7fcf67cc2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5632fe68d060 con 0x5632fe76fd50 2024-09-12T03:06:33.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.611+0000 7fcf67cc2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5632fe77c3a0 con 0x5632fe779580 2024-09-12T03:06:33.672 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:06:33.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.611+0000 7fcf5effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632fe76e7e0 0x5632fe778f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:33.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.611+0000 7fcf67cc2740 10 client.0 _ll_register_callbacks cb 0x5632fe3e08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:33.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.611+0000 7fcf5effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632fe76e7e0 0x5632fe778f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42856/0 (socket says 172.21.15.116:42856) 2024-09-12T03:06:33.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.611+0000 7fcf5effd640 1 -- 172.21.15.116:0/566644086 learned_addr learned my addr 172.21.15.116:0/566644086 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:33.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.611+0000 7fcf5f7fe640 1 --2- 172.21.15.116:0/566644086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632fe779580 0x5632fe776810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:33.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.611+0000 7fcf5effd640 1 -- 172.21.15.116:0/566644086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632fe779580 msgr2=0x5632fe776810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:33.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.611+0000 7fcf5effd640 1 --2- 172.21.15.116:0/566644086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632fe779580 0x5632fe776810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.611+0000 7fcf5effd640 1 -- 172.21.15.116:0/566644086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632fe76fd50 msgr2=0x5632fe776220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:33.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.611+0000 7fcf5effd640 1 --2- 172.21.15.116:0/566644086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632fe76fd50 0x5632fe776220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.611+0000 7fcf5effd640 1 -- 172.21.15.116:0/566644086 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632fe4433c0 con 0x5632fe76e7e0 2024-09-12T03:06:33.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.612+0000 7fcf5effd640 1 --2- 172.21.15.116:0/566644086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632fe76e7e0 0x5632fe778f70 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fcf48009540 tx=0x7fcf480314b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d88f950f12cb31f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:33.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.612+0000 7fcf3effd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:06:33.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.612+0000 7fcf3effd640 1 -- 172.21.15.116:0/566644086 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4804d060 con 0x5632fe76e7e0 2024-09-12T03:06:33.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.612+0000 7fcf3effd640 1 -- 172.21.15.116:0/566644086 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fcf48004030 con 0x5632fe76e7e0 2024-09-12T03:06:33.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.612+0000 7fcf67cc2740 1 -- 172.21.15.116:0/566644086 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf48005020 con 0x5632fe76e7e0 2024-09-12T03:06:33.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.612+0000 7fcf3effd640 1 -- 172.21.15.116:0/566644086 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcf48040070 con 0x5632fe76e7e0 2024-09-12T03:06:33.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.612+0000 7fcf67cc2740 10 client.16176 fetch_fsmap learned FSMap version 25 2024-09-12T03:06:33.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.612+0000 7fcf67cc2740 1 -- 172.21.15.116:0/566644086 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5632fe787cc0 con 0x5632fe76e7e0 2024-09-12T03:06:33.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.612+0000 7fcf3effd640 1 -- 172.21.15.116:0/566644086 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf48005020 con 0x5632fe76e7e0 2024-09-12T03:06:33.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.612+0000 7fcf3effd640 1 -- 172.21.15.116:0/566644086 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcf480038f0 con 0x5632fe76e7e0 2024-09-12T03:06:33.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.612+0000 7fcf67cc2740 10 client.16176 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:06:33.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.612+0000 7fcf67cc2740 10 client.16176 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:06:33.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.612+0000 7fcf67cc2740 20 client.16176 populate_metadata read hostname 'smithi116' 2024-09-12T03:06:33.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.612+0000 7fcf67cc2740 1 --2- 172.21.15.116:0/566644086 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5632fe789010 0x5632fe78b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:33.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.612+0000 7fcf67cc2740 4 client.16176 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:06:33.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.612+0000 7fcf67cc2740 1 -- 172.21.15.116:0/566644086 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632fe6df860 con 0x5632fe789010 2024-09-12T03:06:33.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.613+0000 7fcf5e7fc640 1 --2- 172.21.15.116:0/566644086 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5632fe789010 0x5632fe78b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:33.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.613+0000 7fcf3effd640 10 client.16176 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:06:33.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.613+0000 7fcf5e7fc640 1 --2- 172.21.15.116:0/566644086 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5632fe789010 0x5632fe78b400 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d18976bd6982b2b3 server_cookie=bf65cffa6cde1f6d in_seq=0 out_seq=0 2024-09-12T03:06:33.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.614+0000 7fcf3effd640 1 -- 172.21.15.116:0/566644086 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5632fe6df860 con 0x5632fe789010 2024-09-12T03:06:33.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.614+0000 7fcf3effd640 10 client.16176 handle_command_reply: tid=0 2024-09-12T03:06:33.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.614+0000 7fcf67cc2740 1 client.16176 shutdown 2024-09-12T03:06:33.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.614+0000 7fcf67cc2740 1 -- 172.21.15.116:0/566644086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632fe76e7e0 msgr2=0x5632fe778f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:33.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.614+0000 7fcf67cc2740 1 --2- 172.21.15.116:0/566644086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632fe76e7e0 0x5632fe778f70 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fcf48009540 tx=0x7fcf480314b0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.614+0000 7fcf67cc2740 1 -- 172.21.15.116:0/566644086 shutdown_connections 2024-09-12T03:06:33.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.614+0000 7fcf67cc2740 1 --2- 172.21.15.116:0/566644086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632fe779580 0x5632fe776810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.614+0000 7fcf67cc2740 1 --2- 172.21.15.116:0/566644086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632fe76fd50 0x5632fe776220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.614+0000 7fcf67cc2740 1 --2- 172.21.15.116:0/566644086 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5632fe789010 0x5632fe78b400 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.614+0000 7fcf67cc2740 1 --2- 172.21.15.116:0/566644086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632fe76e7e0 0x5632fe778f70 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.614+0000 7fcf67cc2740 1 -- 172.21.15.116:0/566644086 >> 172.21.15.116:0/566644086 conn(0x5632fe735310 msgr2=0x5632fe770b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:33.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.614+0000 7fcf3effd640 0 client.16176 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:06:33.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.614+0000 7fcf67cc2740 1 -- 172.21.15.116:0/566644086 shutdown_connections 2024-09-12T03:06:33.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.614+0000 7fcf67cc2740 1 -- 172.21.15.116:0/566644086 wait complete. 2024-09-12T03:06:33.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.614+0000 7fcf67cc2740 20 client.16176 trim_cache size 0 max 16384 2024-09-12T03:06:33.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.627+0000 7fcf67cc2740 1 Processor -- start 2024-09-12T03:06:33.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.627+0000 7fcf67cc2740 1 -- start start 2024-09-12T03:06:33.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.627+0000 7fcf67cc2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632fe76e7e0 0x5632fe76b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:33.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.627+0000 7fcf67cc2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632fe76fd50 0x5632fe78e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:33.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.627+0000 7fcf67cc2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632fe779580 0x5632fe78e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:33.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.627+0000 7fcf67cc2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601adba0 con 0x5632fe76e7e0 2024-09-12T03:06:33.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.627+0000 7fcf67cc2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601ab470 con 0x5632fe76fd50 2024-09-12T03:06:33.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.627+0000 7fcf67cc2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601a92c0 con 0x5632fe779580 2024-09-12T03:06:33.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.627+0000 7fcf5effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632fe76e7e0 0x5632fe76b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:33.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.627+0000 7fcf5effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632fe76e7e0 0x5632fe76b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42860/0 (socket says 172.21.15.116:42860) 2024-09-12T03:06:33.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.627+0000 7fcf5effd640 1 -- 172.21.15.116:0/749624124 learned_addr learned my addr 172.21.15.116:0/749624124 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:33.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.627+0000 7fcf5effd640 1 -- 172.21.15.116:0/749624124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632fe779580 msgr2=0x5632fe78e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:33.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.627+0000 7fcf5f7fe640 1 --2- 172.21.15.116:0/749624124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632fe779580 0x5632fe78e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:33.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.627+0000 7fcf5e7fc640 1 --2- 172.21.15.116:0/749624124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632fe76fd50 0x5632fe78e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:33.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.627+0000 7fcf5effd640 1 --2- 172.21.15.116:0/749624124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632fe779580 0x5632fe78e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.627+0000 7fcf5effd640 1 -- 172.21.15.116:0/749624124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632fe76fd50 msgr2=0x5632fe78e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:33.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.627+0000 7fcf5effd640 1 --2- 172.21.15.116:0/749624124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632fe76fd50 0x5632fe78e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.627+0000 7fcf5effd640 1 -- 172.21.15.116:0/749624124 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632fe774770 con 0x5632fe76e7e0 2024-09-12T03:06:33.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.627+0000 7fcf5f7fe640 1 --2- 172.21.15.116:0/749624124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632fe779580 0x5632fe78e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:06:33.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.627+0000 7fcf5e7fc640 1 --2- 172.21.15.116:0/749624124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632fe76fd50 0x5632fe78e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:06:33.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.628+0000 7fcf5effd640 1 --2- 172.21.15.116:0/749624124 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632fe76e7e0 0x5632fe76b020 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fcf480047e0 tx=0x7fcf48005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3028117c6cc01df server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:33.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.628+0000 7fcf557fa640 1 -- 172.21.15.116:0/749624124 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf48058030 con 0x5632fe76e7e0 2024-09-12T03:06:33.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.628+0000 7fcf557fa640 1 -- 172.21.15.116:0/749624124 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf480055e0 con 0x5632fe76e7e0 2024-09-12T03:06:33.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.628+0000 7fcf557fa640 1 -- 172.21.15.116:0/749624124 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4804d040 con 0x5632fe76e7e0 2024-09-12T03:06:33.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.628+0000 7fcf67cc2740 1 -- 172.21.15.116:0/749624124 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632fe76e7e0 msgr2=0x5632fe76b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:33.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.628+0000 7fcf67cc2740 1 --2- 172.21.15.116:0/749624124 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632fe76e7e0 0x5632fe76b020 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fcf480047e0 tx=0x7fcf48005fd0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.628+0000 7fcf67cc2740 1 -- 172.21.15.116:0/749624124 shutdown_connections 2024-09-12T03:06:33.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.628+0000 7fcf67cc2740 1 --2- 172.21.15.116:0/749624124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632fe779580 0x5632fe78e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.628+0000 7fcf67cc2740 1 --2- 172.21.15.116:0/749624124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632fe76fd50 0x5632fe78e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.628+0000 7fcf67cc2740 1 --2- 172.21.15.116:0/749624124 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632fe76e7e0 0x5632fe76b020 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.628+0000 7fcf67cc2740 1 -- 172.21.15.116:0/749624124 >> 172.21.15.116:0/749624124 conn(0x5632fe735310 msgr2=0x5632fe76a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:33.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.628+0000 7fcf67cc2740 1 -- 172.21.15.116:0/749624124 shutdown_connections 2024-09-12T03:06:33.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.628+0000 7fcf67cc2740 1 -- 172.21.15.116:0/749624124 wait complete. 2024-09-12T03:06:33.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.629+0000 7fcf67cc2740 1 -- start start 2024-09-12T03:06:33.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.629+0000 7fcf67cc2740 1 Processor -- start 2024-09-12T03:06:33.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.629+0000 7fcf67cc2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632fe76e7e0 0x5632fe78eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:33.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.629+0000 7fcf67cc2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632fe76fd50 0x5632fe78bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:33.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.629+0000 7fcf67cc2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632fe779580 0x5632fe78c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:33.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.629+0000 7fcf67cc2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601a9930 con 0x5632fe76fd50 2024-09-12T03:06:33.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.629+0000 7fcf67cc2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601aada0 con 0x5632fe76e7e0 2024-09-12T03:06:33.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.629+0000 7fcf67cc2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf601aa000 con 0x5632fe779580 2024-09-12T03:06:33.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.629+0000 7fcf67cc2740 10 client.0 _ll_register_callbacks cb 0x7fcf500956a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:33.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.629+0000 7fcf5e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632fe76fd50 0x5632fe78bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:33.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.629+0000 7fcf5e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632fe76fd50 0x5632fe78bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42874/0 (socket says 172.21.15.116:42874) 2024-09-12T03:06:33.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.629+0000 7fcf5e7fc640 1 -- 172.21.15.116:0/2538875559 learned_addr learned my addr 172.21.15.116:0/2538875559 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:33.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.629+0000 7fcf5effd640 1 --2- 172.21.15.116:0/2538875559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632fe76e7e0 0x5632fe78eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:33.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.629+0000 7fcf5f7fe640 1 --2- 172.21.15.116:0/2538875559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632fe779580 0x5632fe78c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:33.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.629+0000 7fcf5e7fc640 1 -- 172.21.15.116:0/2538875559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632fe779580 msgr2=0x5632fe78c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:33.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.629+0000 7fcf5e7fc640 1 --2- 172.21.15.116:0/2538875559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632fe779580 0x5632fe78c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.629+0000 7fcf5e7fc640 1 -- 172.21.15.116:0/2538875559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632fe76e7e0 msgr2=0x5632fe78eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:33.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.629+0000 7fcf5e7fc640 1 --2- 172.21.15.116:0/2538875559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632fe76e7e0 0x5632fe78eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.629+0000 7fcf5e7fc640 1 -- 172.21.15.116:0/2538875559 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632fe78ccd0 con 0x5632fe76fd50 2024-09-12T03:06:33.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.629+0000 7fcf5effd640 1 --2- 172.21.15.116:0/2538875559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632fe76e7e0 0x5632fe78eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:06:33.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.629+0000 7fcf5f7fe640 1 --2- 172.21.15.116:0/2538875559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632fe779580 0x5632fe78c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:06:33.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.630+0000 7fcf5e7fc640 1 --2- 172.21.15.116:0/2538875559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632fe76fd50 0x5632fe78bf40 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fcf500cc2f0 tx=0x7fcf5000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef6b384712d75af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:33.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.630+0000 7fcf3effd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:06:33.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.630+0000 7fcf3effd640 1 -- 172.21.15.116:0/2538875559 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5001e040 con 0x5632fe76fd50 2024-09-12T03:06:33.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.630+0000 7fcf3effd640 1 -- 172.21.15.116:0/2538875559 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fcf50017360 con 0x5632fe76fd50 2024-09-12T03:06:33.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.630+0000 7fcf67cc2740 1 -- 172.21.15.116:0/2538875559 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf480055e0 con 0x5632fe76fd50 2024-09-12T03:06:33.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.630+0000 7fcf3effd640 1 -- 172.21.15.116:0/2538875559 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcf5009f090 con 0x5632fe76fd50 2024-09-12T03:06:33.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.630+0000 7fcf67cc2740 10 client.16182 fetch_fsmap learned FSMap version 25 2024-09-12T03:06:33.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.630+0000 7fcf67cc2740 1 -- 172.21.15.116:0/2538875559 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5632fe7f38f0 con 0x5632fe76fd50 2024-09-12T03:06:33.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.630+0000 7fcf3effd640 1 -- 172.21.15.116:0/2538875559 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf480055e0 con 0x5632fe76fd50 2024-09-12T03:06:33.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.630+0000 7fcf3effd640 1 -- 172.21.15.116:0/2538875559 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcf50063030 con 0x5632fe76fd50 2024-09-12T03:06:33.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.630+0000 7fcf67cc2740 10 client.16182 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:06:33.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.630+0000 7fcf67cc2740 10 client.16182 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:06:33.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.630+0000 7fcf67cc2740 20 client.16182 populate_metadata read hostname 'smithi116' 2024-09-12T03:06:33.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.630+0000 7fcf67cc2740 1 --2- 172.21.15.116:0/2538875559 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5632fe7f4020 0x5632fe7f6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:33.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.630+0000 7fcf67cc2740 4 client.16182 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:06:33.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.630+0000 7fcf67cc2740 1 -- 172.21.15.116:0/2538875559 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5632fe6d9d70 con 0x5632fe7f4020 2024-09-12T03:06:33.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.631+0000 7fcf5effd640 1 --2- 172.21.15.116:0/2538875559 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5632fe7f4020 0x5632fe7f6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:33.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.631+0000 7fcf5effd640 1 --2- 172.21.15.116:0/2538875559 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5632fe7f4020 0x5632fe7f6440 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=570a98ef1553e1f7 server_cookie=9498cebaa5f79560 in_seq=0 out_seq=0 2024-09-12T03:06:33.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.631+0000 7fcf3effd640 10 client.16182 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:06:33.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.631+0000 7fcf3effd640 1 -- 172.21.15.116:0/2538875559 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5632fe6d9d70 con 0x5632fe7f4020 2024-09-12T03:06:33.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.631+0000 7fcf3effd640 10 client.16182 handle_command_reply: tid=0 2024-09-12T03:06:33.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.631+0000 7fcf67cc2740 1 client.16182 shutdown 2024-09-12T03:06:33.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.631+0000 7fcf67cc2740 1 -- 172.21.15.116:0/2538875559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632fe76fd50 msgr2=0x5632fe78bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:33.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.631+0000 7fcf67cc2740 1 --2- 172.21.15.116:0/2538875559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632fe76fd50 0x5632fe78bf40 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fcf500cc2f0 tx=0x7fcf5000bf90 comp rx=0 tx=0).stop 2024-09-12T03:06:33.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.631+0000 7fcf67cc2740 1 -- 172.21.15.116:0/2538875559 shutdown_connections 2024-09-12T03:06:33.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.631+0000 7fcf67cc2740 1 --2- 172.21.15.116:0/2538875559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632fe779580 0x5632fe78c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.631+0000 7fcf67cc2740 1 --2- 172.21.15.116:0/2538875559 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5632fe7f4020 0x5632fe7f6440 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.631+0000 7fcf67cc2740 1 --2- 172.21.15.116:0/2538875559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632fe76fd50 0x5632fe78bf40 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.631+0000 7fcf67cc2740 1 --2- 172.21.15.116:0/2538875559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632fe76e7e0 0x5632fe78eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.631+0000 7fcf67cc2740 1 -- 172.21.15.116:0/2538875559 >> 172.21.15.116:0/2538875559 conn(0x5632fe735310 msgr2=0x5632fe7674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:33.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.631+0000 7fcf3effd640 0 client.16182 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:06:33.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.631+0000 7fcf67cc2740 1 -- 172.21.15.116:0/2538875559 shutdown_connections 2024-09-12T03:06:33.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.631+0000 7fcf67cc2740 1 -- 172.21.15.116:0/2538875559 wait complete. 2024-09-12T03:06:33.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.632+0000 7fcf67cc2740 20 client.16182 trim_cache size 0 max 16384 2024-09-12T03:06:33.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.632+0000 7fcf658e9640 1 -- 172.21.15.116:0/2745170730 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcf4c0a2b80 msgr2=0x7fcf4c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:33.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.632+0000 7fcf658e9640 1 --2- 172.21.15.116:0/2745170730 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcf4c0a2b80 0x7fcf4c0a5040 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcf580070b0 tx=0x7fcf5800c040 comp rx=0 tx=0).stop 2024-09-12T03:06:33.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.632+0000 7fcf658e9640 1 -- 172.21.15.116:0/2745170730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf60101420 msgr2=0x7fcf60197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:33.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.632+0000 7fcf658e9640 1 --2- 172.21.15.116:0/2745170730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf60101420 0x7fcf60197a20 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000b4a0 tx=0x7fcf5000b960 comp rx=0 tx=0).stop 2024-09-12T03:06:33.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.632+0000 7fcf658e9640 1 -- 172.21.15.116:0/2745170730 shutdown_connections 2024-09-12T03:06:33.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.632+0000 7fcf658e9640 1 --2- 172.21.15.116:0/2745170730 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcf4c0a2b80 0x7fcf4c0a5040 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.632+0000 7fcf658e9640 1 --2- 172.21.15.116:0/2745170730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf60106a80 0x7fcf6019bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.632+0000 7fcf658e9640 1 --2- 172.21.15.116:0/2745170730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf60101420 0x7fcf60197a20 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.632+0000 7fcf658e9640 1 --2- 172.21.15.116:0/2745170730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf60100a80 0x7fcf601974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:33.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.632+0000 7fcf658e9640 1 -- 172.21.15.116:0/2745170730 >> 172.21.15.116:0/2745170730 conn(0x7fcf600fc7a0 msgr2=0x7fcf600fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:33.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.632+0000 7fcf658e9640 1 -- 172.21.15.116:0/2745170730 shutdown_connections 2024-09-12T03:06:33.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:33.632+0000 7fcf658e9640 1 -- 172.21.15.116:0/2745170730 wait complete. 2024-09-12T03:06:34.485 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:06:34.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:34 smithi116 ceph-mon[31410]: pgmap v445: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:06:34.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:34 smithi183 ceph-mon[35535]: pgmap v445: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:06:34.732 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:06:34.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:34 smithi177 ceph-mon[35305]: pgmap v445: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:06:35.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.239+0000 7f3d14ae3640 1 -- 172.21.15.116:0/56050386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d1010c170 msgr2=0x7f3d1010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:35.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.239+0000 7f3d14ae3640 1 --2- 172.21.15.116:0/56050386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d1010c170 0x7f3d1010e610 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3d08009ca0 tx=0x7f3d0802a950 comp rx=0 tx=0).stop 2024-09-12T03:06:35.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.239+0000 7f3d14ae3640 1 -- 172.21.15.116:0/56050386 shutdown_connections 2024-09-12T03:06:35.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.239+0000 7f3d14ae3640 1 --2- 172.21.15.116:0/56050386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d1010c170 0x7f3d1010e610 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.239+0000 7f3d14ae3640 1 --2- 172.21.15.116:0/56050386 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d10103660 0x7f3d1010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.239+0000 7f3d14ae3640 1 --2- 172.21.15.116:0/56050386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d10102d40 0x7f3d10103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.239+0000 7f3d14ae3640 1 -- 172.21.15.116:0/56050386 >> 172.21.15.116:0/56050386 conn(0x7f3d100fc7a0 msgr2=0x7f3d100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:35.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.239+0000 7f3d14ae3640 1 -- 172.21.15.116:0/56050386 shutdown_connections 2024-09-12T03:06:35.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.239+0000 7f3d14ae3640 1 -- 172.21.15.116:0/56050386 wait complete. 2024-09-12T03:06:35.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.240+0000 7f3d14ae3640 1 Processor -- start 2024-09-12T03:06:35.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.240+0000 7f3d14ae3640 1 -- start start 2024-09-12T03:06:35.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.240+0000 7f3d14ae3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d10102d40 0x7f3d10198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:35.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.240+0000 7f3d14ae3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d10103660 0x7f3d101988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:35.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.240+0000 7f3d14ae3640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d1010c170 0x7f3d1019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:35.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.240+0000 7f3d14ae3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3d10111f70 con 0x7f3d10102d40 2024-09-12T03:06:35.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.240+0000 7f3d14ae3640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3d10111df0 con 0x7f3d1010c170 2024-09-12T03:06:35.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.240+0000 7f3d14ae3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3d101120f0 con 0x7f3d10103660 2024-09-12T03:06:35.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.240+0000 7f3d0e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d10102d40 0x7f3d10198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:35.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.241+0000 7f3d0e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d10102d40 0x7f3d10198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42900/0 (socket says 172.21.15.116:42900) 2024-09-12T03:06:35.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.241+0000 7f3d0e59c640 1 -- 172.21.15.116:0/2469212763 learned_addr learned my addr 172.21.15.116:0/2469212763 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:35.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.241+0000 7f3d0dd9b640 1 --2- 172.21.15.116:0/2469212763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d10103660 0x7f3d101988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:35.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.241+0000 7f3d0ed9d640 1 --2- 172.21.15.116:0/2469212763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d1010c170 0x7f3d1019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:35.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.241+0000 7f3d0e59c640 1 -- 172.21.15.116:0/2469212763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d10103660 msgr2=0x7f3d101988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:35.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.241+0000 7f3d0e59c640 1 --2- 172.21.15.116:0/2469212763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d10103660 0x7f3d101988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.241+0000 7f3d0e59c640 1 -- 172.21.15.116:0/2469212763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d1010c170 msgr2=0x7f3d1019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:35.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.241+0000 7f3d0e59c640 1 --2- 172.21.15.116:0/2469212763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d1010c170 0x7f3d1019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.241+0000 7f3d0e59c640 1 -- 172.21.15.116:0/2469212763 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d1019d320 con 0x7f3d10102d40 2024-09-12T03:06:35.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.241+0000 7f3d0dd9b640 1 --2- 172.21.15.116:0/2469212763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d10103660 0x7f3d101988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:06:35.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.241+0000 7f3d0ed9d640 1 --2- 172.21.15.116:0/2469212763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d1010c170 0x7f3d1019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:06:35.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.241+0000 7f3d0e59c640 1 --2- 172.21.15.116:0/2469212763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d10102d40 0x7f3d10198370 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3cf800d770 tx=0x7f3cf800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:35.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.241+0000 7f3d077fe640 1 -- 172.21.15.116:0/2469212763 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8017070 con 0x7f3d10102d40 2024-09-12T03:06:35.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.241+0000 7f3d077fe640 1 -- 172.21.15.116:0/2469212763 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cf8005500 con 0x7f3d10102d40 2024-09-12T03:06:35.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.241+0000 7f3d14ae3640 1 -- 172.21.15.116:0/2469212763 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d1019d610 con 0x7f3d10102d40 2024-09-12T03:06:35.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.241+0000 7f3d077fe640 1 -- 172.21.15.116:0/2469212763 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8004e00 con 0x7f3d10102d40 2024-09-12T03:06:35.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.241+0000 7f3d14ae3640 1 -- 172.21.15.116:0/2469212763 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d101a4ef0 con 0x7f3d10102d40 2024-09-12T03:06:35.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.244+0000 7f3d077fe640 1 -- 172.21.15.116:0/2469212763 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3cf8004fa0 con 0x7f3d10102d40 2024-09-12T03:06:35.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.244+0000 7f3d16ebc740 1 Processor -- start 2024-09-12T03:06:35.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.244+0000 7f3d16ebc740 1 -- start start 2024-09-12T03:06:35.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.244+0000 7f3d16ebc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1be4a8b40 0x55a1be4a7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:35.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.244+0000 7f3d16ebc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1be4a75d0 0x55a1be4a7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:35.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.244+0000 7f3d16ebc740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1be4b6780 0x55a1be4b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:35.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.244+0000 7f3d16ebc740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a1be40d2f0 con 0x55a1be4b6780 2024-09-12T03:06:35.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.244+0000 7f3d16ebc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a1be3b4cf0 con 0x55a1be4a75d0 2024-09-12T03:06:35.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.244+0000 7f3d16ebc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a1be4092b0 con 0x55a1be4a8b40 2024-09-12T03:06:35.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.244+0000 7f3d077fe640 1 --2- 172.21.15.116:0/2469212763 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3cfc0a2c00 0x7f3cfc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:35.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.244+0000 7f3d0dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1be4b6780 0x55a1be4b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:35.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.245+0000 7f3d0dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1be4b6780 0x55a1be4b8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42902/0 (socket says 172.21.15.116:42902) 2024-09-12T03:06:35.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.245+0000 7f3d0dd9b640 1 -- 172.21.15.116:0/2256858223 learned_addr learned my addr 172.21.15.116:0/2256858223 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:35.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.245+0000 7f3d0e59c640 1 --2- 172.21.15.116:0/2256858223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1be4a75d0 0x55a1be4a7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:35.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.245+0000 7f3d0ed9d640 1 --2- 172.21.15.116:0/2469212763 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3cfc0a2c00 0x7f3cfc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:35.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.245+0000 7f3d077fe640 1 -- 172.21.15.116:0/2469212763 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f3cf80cead0 con 0x7f3d10102d40 2024-09-12T03:06:35.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.245+0000 7f3d0ed9d640 1 --2- 172.21.15.116:0/2256858223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1be4a8b40 0x55a1be4a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:35.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.245+0000 7f3d0dd9b640 1 -- 172.21.15.116:0/2256858223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1be4a8b40 msgr2=0x55a1be4a7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:35.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.245+0000 7f3d0dd9b640 1 --2- 172.21.15.116:0/2256858223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1be4a8b40 0x55a1be4a7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.245+0000 7f3d0dd9b640 1 -- 172.21.15.116:0/2256858223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1be4a75d0 msgr2=0x55a1be4a7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:35.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.245+0000 7f3d0dd9b640 1 --2- 172.21.15.116:0/2256858223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1be4a75d0 0x55a1be4a7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.245+0000 7f3d0dd9b640 1 -- 172.21.15.116:0/2256858223 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1be3993a0 con 0x55a1be4b6780 2024-09-12T03:06:35.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.245+0000 7f3d0dd9b640 1 --2- 172.21.15.116:0/2256858223 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1be4b6780 0x55a1be4b8b70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3d00008bb0 tx=0x7f3d0000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11768812b40fb21 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:35.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.245+0000 7f3d0ed9d640 1 --2- 172.21.15.116:0/2469212763 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3cfc0a2c00 0x7f3cfc0a50c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3d08005e30 tx=0x7f3d08005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:35.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.245+0000 7f3d04ff9640 1 -- 172.21.15.116:0/2256858223 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d00014070 con 0x55a1be4b6780 2024-09-12T03:06:35.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.245+0000 7f3d04ff9640 1 -- 172.21.15.116:0/2256858223 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d000047a0 con 0x55a1be4b6780 2024-09-12T03:06:35.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.245+0000 7f3d04ff9640 1 -- 172.21.15.116:0/2256858223 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d00010670 con 0x55a1be4b6780 2024-09-12T03:06:35.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.246+0000 7f3d16ebc740 1 -- 172.21.15.116:0/2256858223 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1be4b6780 msgr2=0x55a1be4b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:35.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.246+0000 7f3d16ebc740 1 --2- 172.21.15.116:0/2256858223 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1be4b6780 0x55a1be4b8b70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3d00008bb0 tx=0x7f3d0000e720 comp rx=0 tx=0).stop 2024-09-12T03:06:35.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.246+0000 7f3d16ebc740 1 -- 172.21.15.116:0/2256858223 shutdown_connections 2024-09-12T03:06:35.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.246+0000 7f3d16ebc740 1 --2- 172.21.15.116:0/2256858223 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1be4b6780 0x55a1be4b8b70 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.246+0000 7f3d16ebc740 1 --2- 172.21.15.116:0/2256858223 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1be4a75d0 0x55a1be4a7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.246+0000 7f3d16ebc740 1 --2- 172.21.15.116:0/2256858223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1be4a8b40 0x55a1be4a7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.246+0000 7f3d16ebc740 1 -- 172.21.15.116:0/2256858223 >> 172.21.15.116:0/2256858223 conn(0x55a1be470310 msgr2=0x55a1be4a3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:35.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.246+0000 7f3d16ebc740 1 -- 172.21.15.116:0/2256858223 shutdown_connections 2024-09-12T03:06:35.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.246+0000 7f3d16ebc740 1 -- 172.21.15.116:0/2256858223 wait complete. 2024-09-12T03:06:35.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.246+0000 7f3d16ebc740 1 -- start start 2024-09-12T03:06:35.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.247+0000 7f3d16ebc740 1 Processor -- start 2024-09-12T03:06:35.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.247+0000 7f3d16ebc740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1be4a75d0 0x55a1be4b6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:35.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.247+0000 7f3d16ebc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1be4a8b40 0x55a1be4b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:35.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.247+0000 7f3d16ebc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1be4b6780 0x55a1be4b3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:35.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.247+0000 7f3d16ebc740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a1be4b9480 con 0x55a1be4a75d0 2024-09-12T03:06:35.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.247+0000 7f3d16ebc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a1be3c8060 con 0x55a1be4b6780 2024-09-12T03:06:35.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.247+0000 7f3d16ebc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a1be4b9600 con 0x55a1be4a8b40 2024-09-12T03:06:35.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.247+0000 7f3d16ebc740 10 client.0 _ll_register_callbacks cb 0x55a1be11b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:35.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.247+0000 7f3d0e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1be4a75d0 0x55a1be4b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:35.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.247+0000 7f3d0e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1be4a75d0 0x55a1be4b6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42908/0 (socket says 172.21.15.116:42908) 2024-09-12T03:06:35.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.247+0000 7f3d0e59c640 1 -- 172.21.15.116:0/439790095 learned_addr learned my addr 172.21.15.116:0/439790095 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:35.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.247+0000 7f3d0e59c640 1 -- 172.21.15.116:0/439790095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1be4a8b40 msgr2=0x55a1be4b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:35.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.247+0000 7f3d0dd9b640 1 --2- 172.21.15.116:0/439790095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1be4a8b40 0x55a1be4b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:35.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.247+0000 7f3d0e59c640 1 --2- 172.21.15.116:0/439790095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1be4a8b40 0x55a1be4b33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.247+0000 7f3d0e59c640 1 -- 172.21.15.116:0/439790095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1be4b6780 msgr2=0x55a1be4b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:35.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.247+0000 7f3d0e59c640 1 --2- 172.21.15.116:0/439790095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1be4b6780 0x55a1be4b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.247+0000 7f3d0e59c640 1 -- 172.21.15.116:0/439790095 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1be17e3c0 con 0x55a1be4a75d0 2024-09-12T03:06:35.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.247+0000 7f3d0e59c640 1 --2- 172.21.15.116:0/439790095 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1be4a75d0 0x55a1be4b6130 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8097d30 tx=0x7f3cf806b210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a564b03c56c6de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:35.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.247+0000 7f3cf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:06:35.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.247+0000 7f3cf5ffb640 1 -- 172.21.15.116:0/439790095 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8020040 con 0x55a1be4a75d0 2024-09-12T03:06:35.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.248+0000 7f3cf5ffb640 1 -- 172.21.15.116:0/439790095 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f3cf8009cc0 con 0x55a1be4a75d0 2024-09-12T03:06:35.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.248+0000 7f3d16ebc740 1 -- 172.21.15.116:0/439790095 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d000047a0 con 0x55a1be4a75d0 2024-09-12T03:06:35.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.248+0000 7f3cf5ffb640 1 -- 172.21.15.116:0/439790095 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3cf8089040 con 0x55a1be4a75d0 2024-09-12T03:06:35.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.248+0000 7f3d16ebc740 10 client.16194 fetch_fsmap learned FSMap version 25 2024-09-12T03:06:35.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.248+0000 7f3d16ebc740 1 -- 172.21.15.116:0/439790095 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a1be4c4f20 con 0x55a1be4a75d0 2024-09-12T03:06:35.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.248+0000 7f3cf5ffb640 1 -- 172.21.15.116:0/439790095 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d000047a0 con 0x55a1be4a75d0 2024-09-12T03:06:35.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.248+0000 7f3cf5ffb640 1 -- 172.21.15.116:0/439790095 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3cf80979d0 con 0x55a1be4a75d0 2024-09-12T03:06:35.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.248+0000 7f3d16ebc740 10 client.16194 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:06:35.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.248+0000 7f3d16ebc740 10 client.16194 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:06:35.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.248+0000 7f3d16ebc740 20 client.16194 populate_metadata read hostname 'smithi116' 2024-09-12T03:06:35.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.248+0000 7f3d16ebc740 1 --2- 172.21.15.116:0/439790095 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55a1be4c63e0 0x55a1be4c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:35.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.248+0000 7f3d16ebc740 4 client.16194 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:06:35.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.248+0000 7f3d16ebc740 1 -- 172.21.15.116:0/439790095 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1be41a860 con 0x55a1be4c63e0 2024-09-12T03:06:35.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.248+0000 7f3d0dd9b640 1 --2- 172.21.15.116:0/439790095 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55a1be4c63e0 0x55a1be4c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:35.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.249+0000 7f3cf5ffb640 10 client.16194 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:06:35.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.249+0000 7f3d0dd9b640 1 --2- 172.21.15.116:0/439790095 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55a1be4c63e0 0x55a1be4c87d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=33f3a1bbb0306b49 server_cookie=b7b59b94891ec00e in_seq=0 out_seq=0 2024-09-12T03:06:35.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.250+0000 7f3cf5ffb640 1 -- 172.21.15.116:0/439790095 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a1be41a860 con 0x55a1be4c63e0 2024-09-12T03:06:35.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.250+0000 7f3cf5ffb640 10 client.16194 handle_command_reply: tid=0 2024-09-12T03:06:35.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.250+0000 7f3d16ebc740 1 client.16194 shutdown 2024-09-12T03:06:35.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.250+0000 7f3d16ebc740 1 -- 172.21.15.116:0/439790095 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1be4a75d0 msgr2=0x55a1be4b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:35.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.250+0000 7f3d16ebc740 1 --2- 172.21.15.116:0/439790095 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1be4a75d0 0x55a1be4b6130 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8097d30 tx=0x7f3cf806b210 comp rx=0 tx=0).stop 2024-09-12T03:06:35.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.250+0000 7f3d16ebc740 1 -- 172.21.15.116:0/439790095 shutdown_connections 2024-09-12T03:06:35.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.250+0000 7f3d16ebc740 1 --2- 172.21.15.116:0/439790095 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55a1be4c63e0 0x55a1be4c87d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.250+0000 7f3d16ebc740 1 --2- 172.21.15.116:0/439790095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1be4b6780 0x55a1be4b3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.250+0000 7f3d16ebc740 1 --2- 172.21.15.116:0/439790095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1be4a8b40 0x55a1be4b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.250+0000 7f3cf5ffb640 0 client.16194 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:06:35.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.250+0000 7f3d16ebc740 1 --2- 172.21.15.116:0/439790095 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1be4a75d0 0x55a1be4b6130 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.250+0000 7f3d16ebc740 1 -- 172.21.15.116:0/439790095 >> 172.21.15.116:0/439790095 conn(0x55a1be470310 msgr2=0x55a1be4a9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:35.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.250+0000 7f3d16ebc740 1 -- 172.21.15.116:0/439790095 shutdown_connections 2024-09-12T03:06:35.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.250+0000 7f3d16ebc740 1 -- 172.21.15.116:0/439790095 wait complete. 2024-09-12T03:06:35.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.250+0000 7f3d16ebc740 20 client.16194 trim_cache size 0 max 16384 2024-09-12T03:06:35.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.263+0000 7f3d16ebc740 1 Processor -- start 2024-09-12T03:06:35.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.263+0000 7f3d16ebc740 1 -- start start 2024-09-12T03:06:35.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.263+0000 7f3d16ebc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1be4a75d0 0x55a1be4c2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:35.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.263+0000 7f3d16ebc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1be4a8b40 0x55a1be4a2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:35.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.263+0000 7f3d16ebc740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1be4b6780 0x55a1be4ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:35.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.263+0000 7f3d16ebc740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3d101ada80 con 0x55a1be4b6780 2024-09-12T03:06:35.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.263+0000 7f3d16ebc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3d101ab350 con 0x55a1be4a75d0 2024-09-12T03:06:35.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.263+0000 7f3d16ebc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3d101a91c0 con 0x55a1be4a8b40 2024-09-12T03:06:35.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.263+0000 7f3d0ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1be4b6780 0x55a1be4ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:35.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.263+0000 7f3d0ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1be4b6780 0x55a1be4ce020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42914/0 (socket says 172.21.15.116:42914) 2024-09-12T03:06:35.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.263+0000 7f3d0ed9d640 1 -- 172.21.15.116:0/3977536491 learned_addr learned my addr 172.21.15.116:0/3977536491 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:35.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.264+0000 7f3d0e59c640 1 --2- 172.21.15.116:0/3977536491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1be4a75d0 0x55a1be4c2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:35.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.264+0000 7f3d0dd9b640 1 --2- 172.21.15.116:0/3977536491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1be4a8b40 0x55a1be4a2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:35.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.264+0000 7f3d0ed9d640 1 -- 172.21.15.116:0/3977536491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1be4a8b40 msgr2=0x55a1be4a2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:35.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.264+0000 7f3d0ed9d640 1 --2- 172.21.15.116:0/3977536491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1be4a8b40 0x55a1be4a2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.264+0000 7f3d0ed9d640 1 -- 172.21.15.116:0/3977536491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1be4a75d0 msgr2=0x55a1be4c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:35.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.264+0000 7f3d0ed9d640 1 --2- 172.21.15.116:0/3977536491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1be4a75d0 0x55a1be4c2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.264+0000 7f3d0ed9d640 1 -- 172.21.15.116:0/3977536491 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1be4a4960 con 0x55a1be4b6780 2024-09-12T03:06:35.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.264+0000 7f3d0e59c640 1 --2- 172.21.15.116:0/3977536491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1be4a75d0 0x55a1be4c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:06:35.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.264+0000 7f3d0ed9d640 1 --2- 172.21.15.116:0/3977536491 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1be4b6780 0x55a1be4ce020 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f3d08004ab0 tx=0x7f3d0802a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec364a480fbf851 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:35.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.264+0000 7f3cf7fff640 1 -- 172.21.15.116:0/3977536491 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0804e030 con 0x55a1be4b6780 2024-09-12T03:06:35.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.264+0000 7f3cf7fff640 1 -- 172.21.15.116:0/3977536491 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d08003e50 con 0x55a1be4b6780 2024-09-12T03:06:35.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.264+0000 7f3cf7fff640 1 -- 172.21.15.116:0/3977536491 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d08034040 con 0x55a1be4b6780 2024-09-12T03:06:35.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.265+0000 7f3d16ebc740 1 -- 172.21.15.116:0/3977536491 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1be4b6780 msgr2=0x55a1be4ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:35.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.265+0000 7f3d16ebc740 1 --2- 172.21.15.116:0/3977536491 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1be4b6780 0x55a1be4ce020 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f3d08004ab0 tx=0x7f3d0802a950 comp rx=0 tx=0).stop 2024-09-12T03:06:35.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.265+0000 7f3d16ebc740 1 -- 172.21.15.116:0/3977536491 shutdown_connections 2024-09-12T03:06:35.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.265+0000 7f3d16ebc740 1 --2- 172.21.15.116:0/3977536491 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1be4b6780 0x55a1be4ce020 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.265+0000 7f3d16ebc740 1 --2- 172.21.15.116:0/3977536491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1be4a8b40 0x55a1be4a2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.265+0000 7f3d16ebc740 1 --2- 172.21.15.116:0/3977536491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1be4a75d0 0x55a1be4c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.265+0000 7f3d16ebc740 1 -- 172.21.15.116:0/3977536491 >> 172.21.15.116:0/3977536491 conn(0x55a1be470310 msgr2=0x55a1be4a24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:35.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.265+0000 7f3d16ebc740 1 -- 172.21.15.116:0/3977536491 shutdown_connections 2024-09-12T03:06:35.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.265+0000 7f3d16ebc740 1 -- 172.21.15.116:0/3977536491 wait complete. 2024-09-12T03:06:35.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.265+0000 7f3d16ebc740 1 -- start start 2024-09-12T03:06:35.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.265+0000 7f3d16ebc740 1 Processor -- start 2024-09-12T03:06:35.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.266+0000 7f3d16ebc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1be4a75d0 0x55a1be4bf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:35.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.266+0000 7f3d16ebc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1be4a8b40 0x55a1be4bb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:35.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.266+0000 7f3d16ebc740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1be4b6780 0x55a1be4bbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:35.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.266+0000 7f3d16ebc740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3d101a9810 con 0x55a1be4b6780 2024-09-12T03:06:35.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.266+0000 7f3d16ebc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3d101aac80 con 0x55a1be4a8b40 2024-09-12T03:06:35.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.266+0000 7f3d16ebc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3d101a9ee0 con 0x55a1be4a75d0 2024-09-12T03:06:35.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.266+0000 7f3d16ebc740 10 client.0 _ll_register_callbacks cb 0x7f3cf800de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:35.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.266+0000 7f3d0ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1be4b6780 0x55a1be4bbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:35.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.266+0000 7f3d0ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1be4b6780 0x55a1be4bbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42916/0 (socket says 172.21.15.116:42916) 2024-09-12T03:06:35.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.266+0000 7f3d0ed9d640 1 -- 172.21.15.116:0/2772240224 learned_addr learned my addr 172.21.15.116:0/2772240224 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:35.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.266+0000 7f3d0e59c640 1 --2- 172.21.15.116:0/2772240224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1be4a75d0 0x55a1be4bf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:35.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.266+0000 7f3d0dd9b640 1 --2- 172.21.15.116:0/2772240224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1be4a8b40 0x55a1be4bb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:35.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.266+0000 7f3d0ed9d640 1 -- 172.21.15.116:0/2772240224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1be4a75d0 msgr2=0x55a1be4bf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:35.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.266+0000 7f3d0ed9d640 1 --2- 172.21.15.116:0/2772240224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1be4a75d0 0x55a1be4bf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.266+0000 7f3d0ed9d640 1 -- 172.21.15.116:0/2772240224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1be4a8b40 msgr2=0x55a1be4bb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:35.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.266+0000 7f3d0ed9d640 1 --2- 172.21.15.116:0/2772240224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1be4a8b40 0x55a1be4bb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.266+0000 7f3d0ed9d640 1 -- 172.21.15.116:0/2772240224 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1be4bc670 con 0x55a1be4b6780 2024-09-12T03:06:35.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.266+0000 7f3d0dd9b640 1 --2- 172.21.15.116:0/2772240224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1be4a8b40 0x55a1be4bb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:06:35.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.266+0000 7f3d0ed9d640 1 --2- 172.21.15.116:0/2772240224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1be4b6780 0x55a1be4bbe70 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3d08006010 tx=0x7f3d08009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5322286a82c2924 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:35.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.266+0000 7f3cf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:06:35.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.266+0000 7f3cf5ffb640 1 -- 172.21.15.116:0/2772240224 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d08057020 con 0x55a1be4b6780 2024-09-12T03:06:35.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.266+0000 7f3d16ebc740 1 -- 172.21.15.116:0/2772240224 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d08003e50 con 0x55a1be4b6780 2024-09-12T03:06:35.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.266+0000 7f3cf5ffb640 1 -- 172.21.15.116:0/2772240224 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f3d080042d0 con 0x55a1be4b6780 2024-09-12T03:06:35.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.267+0000 7f3cf5ffb640 1 -- 172.21.15.116:0/2772240224 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3d0804e070 con 0x55a1be4b6780 2024-09-12T03:06:35.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.267+0000 7f3d16ebc740 10 client.16200 fetch_fsmap learned FSMap version 25 2024-09-12T03:06:35.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.267+0000 7f3d16ebc740 1 -- 172.21.15.116:0/2772240224 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a1be532100 con 0x55a1be4b6780 2024-09-12T03:06:35.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.267+0000 7f3cf5ffb640 1 -- 172.21.15.116:0/2772240224 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d08003e50 con 0x55a1be4b6780 2024-09-12T03:06:35.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.267+0000 7f3cf5ffb640 1 -- 172.21.15.116:0/2772240224 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3d08047050 con 0x55a1be4b6780 2024-09-12T03:06:35.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.267+0000 7f3d16ebc740 10 client.16200 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:06:35.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.267+0000 7f3d16ebc740 10 client.16200 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:06:35.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.267+0000 7f3d16ebc740 20 client.16200 populate_metadata read hostname 'smithi116' 2024-09-12T03:06:35.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.267+0000 7f3d16ebc740 1 --2- 172.21.15.116:0/2772240224 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55a1be532770 0x55a1be534b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:35.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.267+0000 7f3d16ebc740 4 client.16200 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:06:35.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.267+0000 7f3d16ebc740 1 -- 172.21.15.116:0/2772240224 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a1be414d70 con 0x55a1be532770 2024-09-12T03:06:35.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.267+0000 7f3d0e59c640 1 --2- 172.21.15.116:0/2772240224 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55a1be532770 0x55a1be534b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:35.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.268+0000 7f3cf5ffb640 10 client.16200 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:06:35.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.268+0000 7f3d0e59c640 1 --2- 172.21.15.116:0/2772240224 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55a1be532770 0x55a1be534b60 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=14bde788ac601de2 server_cookie=324eacae4d1becd8 in_seq=0 out_seq=0 2024-09-12T03:06:35.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.268+0000 7f3cf5ffb640 1 -- 172.21.15.116:0/2772240224 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a1be414d70 con 0x55a1be532770 2024-09-12T03:06:35.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.268+0000 7f3cf5ffb640 10 client.16200 handle_command_reply: tid=0 2024-09-12T03:06:35.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.268+0000 7f3d16ebc740 1 client.16200 shutdown 2024-09-12T03:06:35.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.268+0000 7f3d16ebc740 1 -- 172.21.15.116:0/2772240224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1be4b6780 msgr2=0x55a1be4bbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:35.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.268+0000 7f3d16ebc740 1 --2- 172.21.15.116:0/2772240224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1be4b6780 0x55a1be4bbe70 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3d08006010 tx=0x7f3d08009880 comp rx=0 tx=0).stop 2024-09-12T03:06:35.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.268+0000 7f3d16ebc740 1 -- 172.21.15.116:0/2772240224 shutdown_connections 2024-09-12T03:06:35.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.268+0000 7f3d16ebc740 1 --2- 172.21.15.116:0/2772240224 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55a1be532770 0x55a1be534b60 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.268+0000 7f3d16ebc740 1 --2- 172.21.15.116:0/2772240224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1be4b6780 0x55a1be4bbe70 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.268+0000 7f3d16ebc740 1 --2- 172.21.15.116:0/2772240224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1be4a8b40 0x55a1be4bb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.268+0000 7f3d16ebc740 1 --2- 172.21.15.116:0/2772240224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1be4a75d0 0x55a1be4bf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.268+0000 7f3d16ebc740 1 -- 172.21.15.116:0/2772240224 >> 172.21.15.116:0/2772240224 conn(0x55a1be470310 msgr2=0x55a1be4a2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:35.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.268+0000 7f3cf5ffb640 0 client.16200 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:06:35.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.268+0000 7f3d16ebc740 1 -- 172.21.15.116:0/2772240224 shutdown_connections 2024-09-12T03:06:35.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.268+0000 7f3d16ebc740 1 -- 172.21.15.116:0/2772240224 wait complete. 2024-09-12T03:06:35.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.268+0000 7f3d16ebc740 20 client.16200 trim_cache size 0 max 16384 2024-09-12T03:06:35.279 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:06:35.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.269+0000 7f3d14ae3640 1 -- 172.21.15.116:0/2469212763 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3cfc0a2c00 msgr2=0x7f3cfc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:35.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.269+0000 7f3d14ae3640 1 --2- 172.21.15.116:0/2469212763 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3cfc0a2c00 0x7f3cfc0a50c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3d08005e30 tx=0x7f3d08005d80 comp rx=0 tx=0).stop 2024-09-12T03:06:35.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.269+0000 7f3d14ae3640 1 -- 172.21.15.116:0/2469212763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d10102d40 msgr2=0x7f3d10198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:35.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.269+0000 7f3d14ae3640 1 --2- 172.21.15.116:0/2469212763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d10102d40 0x7f3d10198370 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3cf800d770 tx=0x7f3cf800dc30 comp rx=0 tx=0).stop 2024-09-12T03:06:35.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.269+0000 7f3d14ae3640 1 -- 172.21.15.116:0/2469212763 shutdown_connections 2024-09-12T03:06:35.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.269+0000 7f3d14ae3640 1 --2- 172.21.15.116:0/2469212763 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3cfc0a2c00 0x7f3cfc0a50c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.269+0000 7f3d14ae3640 1 --2- 172.21.15.116:0/2469212763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d1010c170 0x7f3d1019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.269+0000 7f3d14ae3640 1 --2- 172.21.15.116:0/2469212763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d10103660 0x7f3d101988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.269+0000 7f3d14ae3640 1 --2- 172.21.15.116:0/2469212763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d10102d40 0x7f3d10198370 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:35.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.269+0000 7f3d14ae3640 1 -- 172.21.15.116:0/2469212763 >> 172.21.15.116:0/2469212763 conn(0x7f3d100fc7a0 msgr2=0x7f3d100fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:35.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.269+0000 7f3d14ae3640 1 -- 172.21.15.116:0/2469212763 shutdown_connections 2024-09-12T03:06:35.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:35.269+0000 7f3d14ae3640 1 -- 172.21.15.116:0/2469212763 wait complete. 2024-09-12T03:06:36.153 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:06:36.400 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:06:36.521 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:36 smithi116 ceph-mon[31410]: pgmap v446: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:06:36.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:36 smithi183 ceph-mon[35535]: pgmap v446: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:06:36.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:36 smithi177 ceph-mon[35305]: pgmap v446: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:06:36.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.873+0000 7f772424f640 1 -- 172.21.15.116:0/2909328807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f771c106a80 msgr2=0x7f771c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:36.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.873+0000 7f772424f640 1 --2- 172.21.15.116:0/2909328807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f771c106a80 0x7f771c106e60 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7718009930 tx=0x7f771802a420 comp rx=0 tx=0).stop 2024-09-12T03:06:36.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.873+0000 7f772424f640 1 -- 172.21.15.116:0/2909328807 shutdown_connections 2024-09-12T03:06:36.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.873+0000 7f772424f640 1 --2- 172.21.15.116:0/2909328807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f771c101420 0x7f771c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.873+0000 7f772424f640 1 --2- 172.21.15.116:0/2909328807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f771c100a80 0x7f771c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.873+0000 7f772424f640 1 --2- 172.21.15.116:0/2909328807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f771c106a80 0x7f771c106e60 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.873+0000 7f772424f640 1 -- 172.21.15.116:0/2909328807 >> 172.21.15.116:0/2909328807 conn(0x7f771c0fc7a0 msgr2=0x7f771c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:36.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.873+0000 7f772424f640 1 -- 172.21.15.116:0/2909328807 shutdown_connections 2024-09-12T03:06:36.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.873+0000 7f772424f640 1 -- 172.21.15.116:0/2909328807 wait complete. 2024-09-12T03:06:36.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.874+0000 7f772424f640 1 Processor -- start 2024-09-12T03:06:36.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.874+0000 7f772424f640 1 -- start start 2024-09-12T03:06:36.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.874+0000 7f772424f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f771c100a80 0x7f771c199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:36.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.874+0000 7f772424f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f771c101420 0x7f771c199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:36.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.874+0000 7f772424f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f771c106a80 0x7f771c1a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:36.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.874+0000 7f772424f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f771c110ea0 con 0x7f771c100a80 2024-09-12T03:06:36.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.874+0000 7f772424f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f771c110d20 con 0x7f771c101420 2024-09-12T03:06:36.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.874+0000 7f772424f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f771c111020 con 0x7f771c106a80 2024-09-12T03:06:36.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.874+0000 7f7721feb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f771c100a80 0x7f771c199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:36.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.874+0000 7f7721feb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f771c100a80 0x7f771c199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42936/0 (socket says 172.21.15.116:42936) 2024-09-12T03:06:36.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.874+0000 7f7721feb640 1 -- 172.21.15.116:0/2781612775 learned_addr learned my addr 172.21.15.116:0/2781612775 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:36.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.874+0000 7f77227ec640 1 --2- 172.21.15.116:0/2781612775 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f771c106a80 0x7f771c1a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:36.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.874+0000 7f7721feb640 1 -- 172.21.15.116:0/2781612775 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f771c106a80 msgr2=0x7f771c1a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:36.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.874+0000 7f77217ea640 1 --2- 172.21.15.116:0/2781612775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f771c101420 0x7f771c199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:36.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.874+0000 7f7721feb640 1 --2- 172.21.15.116:0/2781612775 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f771c106a80 0x7f771c1a43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.874+0000 7f7721feb640 1 -- 172.21.15.116:0/2781612775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f771c101420 msgr2=0x7f771c199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:36.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.874+0000 7f7721feb640 1 --2- 172.21.15.116:0/2781612775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f771c101420 0x7f771c199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.874+0000 7f7721feb640 1 -- 172.21.15.116:0/2781612775 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f771c1a48f0 con 0x7f771c100a80 2024-09-12T03:06:36.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.875+0000 7f7721feb640 1 --2- 172.21.15.116:0/2781612775 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f771c100a80 0x7f771c199330 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f7718009230 tx=0x7f7718004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:36.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.875+0000 7f7712ffd640 1 -- 172.21.15.116:0/2781612775 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771803f030 con 0x7f771c100a80 2024-09-12T03:06:36.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.875+0000 7f772424f640 1 -- 172.21.15.116:0/2781612775 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f771c1a4ac0 con 0x7f771c100a80 2024-09-12T03:06:36.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.875+0000 7f7712ffd640 1 -- 172.21.15.116:0/2781612775 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7718003ad0 con 0x7f771c100a80 2024-09-12T03:06:36.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.875+0000 7f7712ffd640 1 -- 172.21.15.116:0/2781612775 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7718035070 con 0x7f771c100a80 2024-09-12T03:06:36.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.875+0000 7f772424f640 1 -- 172.21.15.116:0/2781612775 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f771c1a4f70 con 0x7f771c100a80 2024-09-12T03:06:36.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.877+0000 7f7712ffd640 1 -- 172.21.15.116:0/2781612775 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f7718042030 con 0x7f771c100a80 2024-09-12T03:06:36.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.877+0000 7f7726628740 1 Processor -- start 2024-09-12T03:06:36.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.877+0000 7f7726628740 1 -- start start 2024-09-12T03:06:36.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.878+0000 7f7726628740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555d9fcfb40 0x5555d9fce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:36.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.878+0000 7f7726628740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d9fce5d0 0x5555d9fcea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:36.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.878+0000 7f7726628740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555d9fd9370 0x5555d9fdb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:36.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.878+0000 7f7726628740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5555d9f342f0 con 0x5555d9fd9370 2024-09-12T03:06:36.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.878+0000 7f7726628740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5555d9edbcf0 con 0x5555d9fcfb40 2024-09-12T03:06:36.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.878+0000 7f7726628740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5555d9f302b0 con 0x5555d9fce5d0 2024-09-12T03:06:36.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.878+0000 7f7712ffd640 1 --2- 172.21.15.116:0/2781612775 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f77080a2b80 0x7f77080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:36.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.878+0000 7f77217ea640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555d9fd9370 0x5555d9fdb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:36.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.878+0000 7f7721feb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d9fce5d0 0x5555d9fcea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:36.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.878+0000 7f77217ea640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555d9fd9370 0x5555d9fdb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42944/0 (socket says 172.21.15.116:42944) 2024-09-12T03:06:36.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.878+0000 7f77217ea640 1 -- 172.21.15.116:0/3991317004 learned_addr learned my addr 172.21.15.116:0/3991317004 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:36.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.878+0000 7f77227ec640 1 --2- 172.21.15.116:0/2781612775 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f77080a2b80 0x7f77080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:36.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.878+0000 7f77227ec640 1 --2- 172.21.15.116:0/3991317004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555d9fcfb40 0x5555d9fce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:36.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.878+0000 7f7712ffd640 1 -- 172.21.15.116:0/2781612775 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f77180ef5b0 con 0x7f771c100a80 2024-09-12T03:06:36.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.878+0000 7f77217ea640 1 -- 172.21.15.116:0/3991317004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d9fce5d0 msgr2=0x5555d9fcea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:36.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.878+0000 7f77217ea640 1 --2- 172.21.15.116:0/3991317004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d9fce5d0 0x5555d9fcea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.878+0000 7f77217ea640 1 -- 172.21.15.116:0/3991317004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555d9fcfb40 msgr2=0x5555d9fce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:36.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.878+0000 7f77217ea640 1 --2- 172.21.15.116:0/3991317004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555d9fcfb40 0x5555d9fce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.878+0000 7f77217ea640 1 -- 172.21.15.116:0/3991317004 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555d9ec03a0 con 0x5555d9fd9370 2024-09-12T03:06:36.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.879+0000 7f77217ea640 1 --2- 172.21.15.116:0/3991317004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555d9fd9370 0x5555d9fdb760 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f770c00a8e0 tx=0x7f770c00ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0dc9b3a43373524 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:36.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.879+0000 7f77227ec640 1 --2- 172.21.15.116:0/2781612775 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f77080a2b80 0x7f77080a5040 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7714009fd0 tx=0x7f7714009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:36.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.879+0000 7f7707fff640 1 -- 172.21.15.116:0/3991317004 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c015070 con 0x5555d9fd9370 2024-09-12T03:06:36.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.879+0000 7f7707fff640 1 -- 172.21.15.116:0/3991317004 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f770c00cb40 con 0x5555d9fd9370 2024-09-12T03:06:36.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.879+0000 7f7707fff640 1 -- 172.21.15.116:0/3991317004 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c010590 con 0x5555d9fd9370 2024-09-12T03:06:36.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.879+0000 7f7726628740 1 -- 172.21.15.116:0/3991317004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555d9fd9370 msgr2=0x5555d9fdb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:36.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.879+0000 7f7726628740 1 --2- 172.21.15.116:0/3991317004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555d9fd9370 0x5555d9fdb760 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f770c00a8e0 tx=0x7f770c00ada0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.880+0000 7f7726628740 1 -- 172.21.15.116:0/3991317004 shutdown_connections 2024-09-12T03:06:36.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.880+0000 7f7726628740 1 --2- 172.21.15.116:0/3991317004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555d9fd9370 0x5555d9fdb760 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.880+0000 7f7726628740 1 --2- 172.21.15.116:0/3991317004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d9fce5d0 0x5555d9fcea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.880+0000 7f7726628740 1 --2- 172.21.15.116:0/3991317004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555d9fcfb40 0x5555d9fce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.880+0000 7f7726628740 1 -- 172.21.15.116:0/3991317004 >> 172.21.15.116:0/3991317004 conn(0x5555d9f97310 msgr2=0x5555d9fcac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:36.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.880+0000 7f7726628740 1 -- 172.21.15.116:0/3991317004 shutdown_connections 2024-09-12T03:06:36.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.880+0000 7f7726628740 1 -- 172.21.15.116:0/3991317004 wait complete. 2024-09-12T03:06:36.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.880+0000 7f7726628740 1 -- start start 2024-09-12T03:06:36.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.880+0000 7f7726628740 1 Processor -- start 2024-09-12T03:06:36.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.880+0000 7f7726628740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555d9fce5d0 0x5555d9fd8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:36.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.880+0000 7f7726628740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555d9fcfb40 0x5555d9fd5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:36.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.880+0000 7f7726628740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d9fd9370 0x5555d9fd6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:36.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.880+0000 7f7726628740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5555d9fdc070 con 0x5555d9fcfb40 2024-09-12T03:06:36.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.880+0000 7f7726628740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5555d9eef060 con 0x5555d9fce5d0 2024-09-12T03:06:36.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.880+0000 7f7726628740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5555d9fdc1f0 con 0x5555d9fd9370 2024-09-12T03:06:36.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.880+0000 7f7726628740 10 client.0 _ll_register_callbacks cb 0x5555d9c428c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:36.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.881+0000 7f77217ea640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555d9fcfb40 0x5555d9fd5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:36.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.881+0000 7f77217ea640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555d9fcfb40 0x5555d9fd5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42958/0 (socket says 172.21.15.116:42958) 2024-09-12T03:06:36.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.881+0000 7f77217ea640 1 -- 172.21.15.116:0/3747556863 learned_addr learned my addr 172.21.15.116:0/3747556863 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:36.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.881+0000 7f7721feb640 1 --2- 172.21.15.116:0/3747556863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555d9fce5d0 0x5555d9fd8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:36.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.881+0000 7f77227ec640 1 --2- 172.21.15.116:0/3747556863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d9fd9370 0x5555d9fd6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:36.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.881+0000 7f77217ea640 1 -- 172.21.15.116:0/3747556863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d9fd9370 msgr2=0x5555d9fd6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:36.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.881+0000 7f77217ea640 1 --2- 172.21.15.116:0/3747556863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d9fd9370 0x5555d9fd6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.881+0000 7f77217ea640 1 -- 172.21.15.116:0/3747556863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555d9fce5d0 msgr2=0x5555d9fd8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:36.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.881+0000 7f77217ea640 1 --2- 172.21.15.116:0/3747556863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555d9fce5d0 0x5555d9fd8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.881+0000 7f77217ea640 1 -- 172.21.15.116:0/3747556863 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555d9ca53c0 con 0x5555d9fcfb40 2024-09-12T03:06:36.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.881+0000 7f7721feb640 1 --2- 172.21.15.116:0/3747556863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555d9fce5d0 0x5555d9fd8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:06:36.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.881+0000 7f77217ea640 1 --2- 172.21.15.116:0/3747556863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555d9fcfb40 0x5555d9fd5f40 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f770c011e20 tx=0x7f770c011620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feeeb7c4d90ec594 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:36.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.881+0000 7f77057fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:06:36.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.881+0000 7f77057fa640 1 -- 172.21.15.116:0/3747556863 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c029060 con 0x5555d9fcfb40 2024-09-12T03:06:36.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.881+0000 7f77057fa640 1 -- 172.21.15.116:0/3747556863 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f770c011790 con 0x5555d9fcfb40 2024-09-12T03:06:36.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.881+0000 7f7726628740 1 -- 172.21.15.116:0/3747556863 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f770c00cb40 con 0x5555d9fcfb40 2024-09-12T03:06:36.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.881+0000 7f77057fa640 1 -- 172.21.15.116:0/3747556863 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f770c015040 con 0x5555d9fcfb40 2024-09-12T03:06:36.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.881+0000 7f7726628740 10 client.16212 fetch_fsmap learned FSMap version 25 2024-09-12T03:06:36.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.881+0000 7f7726628740 1 -- 172.21.15.116:0/3747556863 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5555d9fe7a60 con 0x5555d9fcfb40 2024-09-12T03:06:36.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.882+0000 7f77057fa640 1 -- 172.21.15.116:0/3747556863 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f770c00cb40 con 0x5555d9fcfb40 2024-09-12T03:06:36.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.882+0000 7f77057fa640 1 -- 172.21.15.116:0/3747556863 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f770c0244b0 con 0x5555d9fcfb40 2024-09-12T03:06:36.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.882+0000 7f7726628740 10 client.16212 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:06:36.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.882+0000 7f7726628740 10 client.16212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:06:36.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.882+0000 7f7726628740 20 client.16212 populate_metadata read hostname 'smithi116' 2024-09-12T03:06:36.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.882+0000 7f7726628740 1 --2- 172.21.15.116:0/3747556863 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5555d9fe8db0 0x5555d9feb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:36.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.882+0000 7f7726628740 4 client.16212 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:06:36.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.882+0000 7f7726628740 1 -- 172.21.15.116:0/3747556863 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555d9f41860 con 0x5555d9fe8db0 2024-09-12T03:06:36.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.882+0000 7f7721feb640 1 --2- 172.21.15.116:0/3747556863 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5555d9fe8db0 0x5555d9feb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:36.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.882+0000 7f7721feb640 1 --2- 172.21.15.116:0/3747556863 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5555d9fe8db0 0x5555d9feb1a0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddefd90d636a1a8b server_cookie=46e88e297ff9a3d4 in_seq=0 out_seq=0 2024-09-12T03:06:36.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.882+0000 7f77057fa640 10 client.16212 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:36.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.883+0000 7f77057fa640 1 -- 172.21.15.116:0/3747556863 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5555d9f41860 con 0x5555d9fe8db0 2024-09-12T03:06:36.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.883+0000 7f77057fa640 10 client.16212 handle_command_reply: tid=0 2024-09-12T03:06:36.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.883+0000 7f7726628740 1 client.16212 shutdown 2024-09-12T03:06:36.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.883+0000 7f7726628740 1 -- 172.21.15.116:0/3747556863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555d9fcfb40 msgr2=0x5555d9fd5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:36.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.883+0000 7f7726628740 1 --2- 172.21.15.116:0/3747556863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555d9fcfb40 0x5555d9fd5f40 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f770c011e20 tx=0x7f770c011620 comp rx=0 tx=0).stop 2024-09-12T03:06:36.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.883+0000 7f7726628740 1 -- 172.21.15.116:0/3747556863 shutdown_connections 2024-09-12T03:06:36.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.883+0000 7f7726628740 1 --2- 172.21.15.116:0/3747556863 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5555d9fe8db0 0x5555d9feb1a0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.883+0000 7f7726628740 1 --2- 172.21.15.116:0/3747556863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d9fd9370 0x5555d9fd6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.883+0000 7f7726628740 1 --2- 172.21.15.116:0/3747556863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555d9fcfb40 0x5555d9fd5f40 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.883+0000 7f7726628740 1 --2- 172.21.15.116:0/3747556863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555d9fce5d0 0x5555d9fd8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.883+0000 7f77057fa640 0 client.16212 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:36.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.883+0000 7f7726628740 1 -- 172.21.15.116:0/3747556863 >> 172.21.15.116:0/3747556863 conn(0x5555d9f97310 msgr2=0x5555d9fd0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:36.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.884+0000 7f7726628740 1 -- 172.21.15.116:0/3747556863 shutdown_connections 2024-09-12T03:06:36.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.884+0000 7f7726628740 1 -- 172.21.15.116:0/3747556863 wait complete. 2024-09-12T03:06:36.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.884+0000 7f7726628740 20 client.16212 trim_cache size 0 max 16384 2024-09-12T03:06:36.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.897+0000 7f7726628740 1 Processor -- start 2024-09-12T03:06:36.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.897+0000 7f7726628740 1 -- start start 2024-09-12T03:06:36.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.897+0000 7f7726628740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d9fce5d0 0x5555d9fe5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:36.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.897+0000 7f7726628740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555d9fcfb40 0x5555d9ff08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:36.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.897+0000 7f7726628740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555d9fd9370 0x5555d9ff0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:36.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.897+0000 7f7726628740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f771c1adb00 con 0x5555d9fcfb40 2024-09-12T03:06:36.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.897+0000 7f7726628740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f771c1ab3d0 con 0x5555d9fd9370 2024-09-12T03:06:36.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.897+0000 7f7726628740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f771c1a9220 con 0x5555d9fce5d0 2024-09-12T03:06:36.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.898+0000 7f77217ea640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555d9fcfb40 0x5555d9ff08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:36.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.898+0000 7f77217ea640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555d9fcfb40 0x5555d9ff08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42970/0 (socket says 172.21.15.116:42970) 2024-09-12T03:06:36.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.898+0000 7f77217ea640 1 -- 172.21.15.116:0/3052109692 learned_addr learned my addr 172.21.15.116:0/3052109692 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:36.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.898+0000 7f77217ea640 1 -- 172.21.15.116:0/3052109692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d9fce5d0 msgr2=0x5555d9fe5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:36.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.898+0000 7f77217ea640 1 --2- 172.21.15.116:0/3052109692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d9fce5d0 0x5555d9fe5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.898+0000 7f77217ea640 1 -- 172.21.15.116:0/3052109692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555d9fd9370 msgr2=0x5555d9ff0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:36.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.898+0000 7f77217ea640 1 --2- 172.21.15.116:0/3052109692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555d9fd9370 0x5555d9ff0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.898+0000 7f77217ea640 1 -- 172.21.15.116:0/3052109692 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555d9fd88b0 con 0x5555d9fcfb40 2024-09-12T03:06:36.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.898+0000 7f77217ea640 1 --2- 172.21.15.116:0/3052109692 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555d9fcfb40 0x5555d9ff08c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f770c00a530 tx=0x7f770c00c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f92e6514c54b749 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:36.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.898+0000 7f77077fe640 1 -- 172.21.15.116:0/3052109692 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c034020 con 0x5555d9fcfb40 2024-09-12T03:06:36.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.898+0000 7f77077fe640 1 -- 172.21.15.116:0/3052109692 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f770c011e50 con 0x5555d9fcfb40 2024-09-12T03:06:36.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.898+0000 7f77077fe640 1 -- 172.21.15.116:0/3052109692 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c00a070 con 0x5555d9fcfb40 2024-09-12T03:06:36.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.899+0000 7f7726628740 1 -- 172.21.15.116:0/3052109692 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555d9fcfb40 msgr2=0x5555d9ff08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:36.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.899+0000 7f7726628740 1 --2- 172.21.15.116:0/3052109692 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555d9fcfb40 0x5555d9ff08c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f770c00a530 tx=0x7f770c00c690 comp rx=0 tx=0).stop 2024-09-12T03:06:36.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.899+0000 7f7726628740 1 -- 172.21.15.116:0/3052109692 shutdown_connections 2024-09-12T03:06:36.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.899+0000 7f7726628740 1 --2- 172.21.15.116:0/3052109692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555d9fd9370 0x5555d9ff0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.899+0000 7f7726628740 1 --2- 172.21.15.116:0/3052109692 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555d9fcfb40 0x5555d9ff08c0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.899+0000 7f7726628740 1 --2- 172.21.15.116:0/3052109692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d9fce5d0 0x5555d9fe5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.899+0000 7f7726628740 1 -- 172.21.15.116:0/3052109692 >> 172.21.15.116:0/3052109692 conn(0x5555d9f97310 msgr2=0x5555d9fd7fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:36.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.899+0000 7f7726628740 1 -- 172.21.15.116:0/3052109692 shutdown_connections 2024-09-12T03:06:36.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.899+0000 7f7726628740 1 -- 172.21.15.116:0/3052109692 wait complete. 2024-09-12T03:06:36.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.899+0000 7f7726628740 1 -- start start 2024-09-12T03:06:36.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.900+0000 7f7726628740 1 Processor -- start 2024-09-12T03:06:36.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.900+0000 7f7726628740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d9fce5d0 0x5555d9fdd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:36.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.900+0000 7f7726628740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555d9fcfb40 0x5555d9fdd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:36.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.900+0000 7f7726628740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555d9fd9370 0x5555d9fddc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:36.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.900+0000 7f7726628740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f771c1a9890 con 0x5555d9fcfb40 2024-09-12T03:06:36.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.900+0000 7f7726628740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f771c1aad00 con 0x5555d9fd9370 2024-09-12T03:06:36.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.900+0000 7f7726628740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f771c1a9f60 con 0x5555d9fce5d0 2024-09-12T03:06:36.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.900+0000 7f7726628740 10 client.0 _ll_register_callbacks cb 0x7f77180bbc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:36.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.900+0000 7f77217ea640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555d9fcfb40 0x5555d9fdd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:36.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.900+0000 7f77217ea640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555d9fcfb40 0x5555d9fdd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42980/0 (socket says 172.21.15.116:42980) 2024-09-12T03:06:36.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.900+0000 7f77217ea640 1 -- 172.21.15.116:0/4051410630 learned_addr learned my addr 172.21.15.116:0/4051410630 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:36.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.900+0000 7f77227ec640 1 --2- 172.21.15.116:0/4051410630 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555d9fd9370 0x5555d9fddc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:36.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.900+0000 7f7721feb640 1 --2- 172.21.15.116:0/4051410630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d9fce5d0 0x5555d9fdd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:36.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.900+0000 7f77217ea640 1 -- 172.21.15.116:0/4051410630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d9fce5d0 msgr2=0x5555d9fdd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:36.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.900+0000 7f77217ea640 1 --2- 172.21.15.116:0/4051410630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d9fce5d0 0x5555d9fdd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.900+0000 7f77217ea640 1 -- 172.21.15.116:0/4051410630 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555d9fd9370 msgr2=0x5555d9fddc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:36.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.900+0000 7f77217ea640 1 --2- 172.21.15.116:0/4051410630 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555d9fd9370 0x5555d9fddc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.901+0000 7f77217ea640 1 -- 172.21.15.116:0/4051410630 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555da052890 con 0x5555d9fcfb40 2024-09-12T03:06:36.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.901+0000 7f77217ea640 1 --2- 172.21.15.116:0/4051410630 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555d9fcfb40 0x5555d9fdd700 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f770c024850 tx=0x7f770c011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d7026c6a6492f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:36.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.901+0000 7f77057fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:06:36.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.901+0000 7f77057fa640 1 -- 172.21.15.116:0/4051410630 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c034050 con 0x5555d9fcfb40 2024-09-12T03:06:36.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.901+0000 7f77057fa640 1 -- 172.21.15.116:0/4051410630 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f770c011800 con 0x5555d9fcfb40 2024-09-12T03:06:36.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.901+0000 7f7726628740 1 -- 172.21.15.116:0/4051410630 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f770c011e50 con 0x5555d9fcfb40 2024-09-12T03:06:36.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.901+0000 7f77057fa640 1 -- 172.21.15.116:0/4051410630 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f770c00a040 con 0x5555d9fcfb40 2024-09-12T03:06:36.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.901+0000 7f7726628740 10 client.16218 fetch_fsmap learned FSMap version 25 2024-09-12T03:06:36.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.901+0000 7f7726628740 1 -- 172.21.15.116:0/4051410630 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5555da053a90 con 0x5555d9fcfb40 2024-09-12T03:06:36.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.901+0000 7f77057fa640 1 -- 172.21.15.116:0/4051410630 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f770c011e50 con 0x5555d9fcfb40 2024-09-12T03:06:36.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.901+0000 7f77057fa640 1 -- 172.21.15.116:0/4051410630 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f770c030430 con 0x5555d9fcfb40 2024-09-12T03:06:36.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.901+0000 7f7726628740 10 client.16218 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:06:36.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.901+0000 7f7726628740 10 client.16218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:06:36.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.901+0000 7f7726628740 20 client.16218 populate_metadata read hostname 'smithi116' 2024-09-12T03:06:36.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.901+0000 7f7726628740 1 --2- 172.21.15.116:0/4051410630 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5555da054070 0x5555da056490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:36.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.901+0000 7f7726628740 4 client.16218 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:06:36.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.901+0000 7f7726628740 1 -- 172.21.15.116:0/4051410630 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5555d9fd6810 con 0x5555da054070 2024-09-12T03:06:36.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.902+0000 7f7721feb640 1 --2- 172.21.15.116:0/4051410630 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5555da054070 0x5555da056490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:36.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.902+0000 7f7721feb640 1 --2- 172.21.15.116:0/4051410630 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5555da054070 0x5555da056490 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f42b1824a1ed198 server_cookie=15267bc0ce7735a in_seq=0 out_seq=0 2024-09-12T03:06:36.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.902+0000 7f77057fa640 10 client.16218 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:36.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.902+0000 7f77057fa640 1 -- 172.21.15.116:0/4051410630 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5555d9fd6810 con 0x5555da054070 2024-09-12T03:06:36.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.902+0000 7f77057fa640 10 client.16218 handle_command_reply: tid=0 2024-09-12T03:06:36.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.902+0000 7f7726628740 1 client.16218 shutdown 2024-09-12T03:06:36.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.902+0000 7f7726628740 1 -- 172.21.15.116:0/4051410630 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555d9fcfb40 msgr2=0x5555d9fdd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:36.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.902+0000 7f7726628740 1 --2- 172.21.15.116:0/4051410630 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555d9fcfb40 0x5555d9fdd700 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f770c024850 tx=0x7f770c011dc0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.902+0000 7f7726628740 1 -- 172.21.15.116:0/4051410630 shutdown_connections 2024-09-12T03:06:36.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.902+0000 7f7726628740 1 --2- 172.21.15.116:0/4051410630 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5555da054070 0x5555da056490 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.902+0000 7f7726628740 1 --2- 172.21.15.116:0/4051410630 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5555d9fd9370 0x5555d9fddc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.902+0000 7f7726628740 1 --2- 172.21.15.116:0/4051410630 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5555d9fcfb40 0x5555d9fdd700 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.902+0000 7f77057fa640 0 client.16218 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:36.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.902+0000 7f7726628740 1 --2- 172.21.15.116:0/4051410630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5555d9fce5d0 0x5555d9fdd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.902+0000 7f7726628740 1 -- 172.21.15.116:0/4051410630 >> 172.21.15.116:0/4051410630 conn(0x5555d9f97310 msgr2=0x5555d9fd77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:36.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.903+0000 7f7726628740 1 -- 172.21.15.116:0/4051410630 shutdown_connections 2024-09-12T03:06:36.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.903+0000 7f7726628740 1 -- 172.21.15.116:0/4051410630 wait complete. 2024-09-12T03:06:36.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.903+0000 7f7726628740 20 client.16218 trim_cache size 0 max 16384 2024-09-12T03:06:36.927 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:06:36.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.903+0000 7f772424f640 1 -- 172.21.15.116:0/2781612775 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f77080a2b80 msgr2=0x7f77080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:36.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.903+0000 7f772424f640 1 --2- 172.21.15.116:0/2781612775 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f77080a2b80 0x7f77080a5040 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7714009fd0 tx=0x7f7714009560 comp rx=0 tx=0).stop 2024-09-12T03:06:36.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.903+0000 7f772424f640 1 -- 172.21.15.116:0/2781612775 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f771c100a80 msgr2=0x7f771c199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:36.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.903+0000 7f772424f640 1 --2- 172.21.15.116:0/2781612775 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f771c100a80 0x7f771c199330 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f7718009230 tx=0x7f7718004270 comp rx=0 tx=0).stop 2024-09-12T03:06:36.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.903+0000 7f772424f640 1 -- 172.21.15.116:0/2781612775 shutdown_connections 2024-09-12T03:06:36.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.903+0000 7f772424f640 1 --2- 172.21.15.116:0/2781612775 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f77080a2b80 0x7f77080a5040 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.903+0000 7f772424f640 1 --2- 172.21.15.116:0/2781612775 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f771c106a80 0x7f771c1a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.903+0000 7f772424f640 1 --2- 172.21.15.116:0/2781612775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f771c101420 0x7f771c199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.903+0000 7f772424f640 1 --2- 172.21.15.116:0/2781612775 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f771c100a80 0x7f771c199330 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:36.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.903+0000 7f772424f640 1 -- 172.21.15.116:0/2781612775 >> 172.21.15.116:0/2781612775 conn(0x7f771c0fc7a0 msgr2=0x7f771c0fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:36.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.903+0000 7f772424f640 1 -- 172.21.15.116:0/2781612775 shutdown_connections 2024-09-12T03:06:36.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:36.903+0000 7f772424f640 1 -- 172.21.15.116:0/2781612775 wait complete. 2024-09-12T03:06:38.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:38 smithi116 ceph-mon[31410]: pgmap v447: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:06:38.702 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:06:38.703 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:06:38.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:38 smithi183 ceph-mon[35535]: pgmap v447: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:06:38.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:38 smithi177 ceph-mon[35305]: pgmap v447: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:06:38.961 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:06:39.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.470+0000 7fa90a3f4640 1 -- 172.21.15.116:0/4207762207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa904101de0 msgr2=0x7fa9041021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:39.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.470+0000 7fa90a3f4640 1 --2- 172.21.15.116:0/4207762207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa904101de0 0x7fa9041021c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4009900 tx=0x7fa8f402a410 comp rx=0 tx=0).stop 2024-09-12T03:06:39.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.470+0000 7fa90a3f4640 1 -- 172.21.15.116:0/4207762207 shutdown_connections 2024-09-12T03:06:39.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.470+0000 7fa90a3f4640 1 --2- 172.21.15.116:0/4207762207 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa904106f10 0x7fa90410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.470+0000 7fa90a3f4640 1 --2- 172.21.15.116:0/4207762207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa904102790 0x7fa9041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.470+0000 7fa90a3f4640 1 --2- 172.21.15.116:0/4207762207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa904101de0 0x7fa9041021c0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.470+0000 7fa90a3f4640 1 -- 172.21.15.116:0/4207762207 >> 172.21.15.116:0/4207762207 conn(0x7fa9040fd8c0 msgr2=0x7fa9040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:39.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.470+0000 7fa90a3f4640 1 -- 172.21.15.116:0/4207762207 shutdown_connections 2024-09-12T03:06:39.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.470+0000 7fa90a3f4640 1 -- 172.21.15.116:0/4207762207 wait complete. 2024-09-12T03:06:39.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.471+0000 7fa90a3f4640 1 Processor -- start 2024-09-12T03:06:39.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.471+0000 7fa90a3f4640 1 -- start start 2024-09-12T03:06:39.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.471+0000 7fa90a3f4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa904101de0 0x7fa904198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.471+0000 7fa90a3f4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa904102790 0x7fa9041989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.471+0000 7fa90a3f4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa904106f10 0x7fa90419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.471+0000 7fa90a3f4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa904111f90 con 0x7fa904102790 2024-09-12T03:06:39.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.471+0000 7fa90a3f4640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa904111e10 con 0x7fa904101de0 2024-09-12T03:06:39.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.471+0000 7fa90a3f4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa904112110 con 0x7fa904106f10 2024-09-12T03:06:39.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.471+0000 7fa903fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa904101de0 0x7fa904198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.471+0000 7fa8fbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa904102790 0x7fa9041989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.471+0000 7fa903fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa904101de0 0x7fa904198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.116:59192/0 (socket says 172.21.15.116:59192) 2024-09-12T03:06:39.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.471+0000 7fa903fff640 1 -- 172.21.15.116:0/108735566 learned_addr learned my addr 172.21.15.116:0/108735566 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:39.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.471+0000 7fa908991640 1 --2- 172.21.15.116:0/108735566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa904106f10 0x7fa90419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.471+0000 7fa8fbfff640 1 -- 172.21.15.116:0/108735566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa904106f10 msgr2=0x7fa90419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:39.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.471+0000 7fa8fbfff640 1 --2- 172.21.15.116:0/108735566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa904106f10 0x7fa90419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.471+0000 7fa8fbfff640 1 -- 172.21.15.116:0/108735566 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa904101de0 msgr2=0x7fa904198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:39.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.471+0000 7fa8fbfff640 1 --2- 172.21.15.116:0/108735566 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa904101de0 0x7fa904198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.471+0000 7fa8fbfff640 1 -- 172.21.15.116:0/108735566 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa90419d440 con 0x7fa904102790 2024-09-12T03:06:39.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.472+0000 7fa8fbfff640 1 --2- 172.21.15.116:0/108735566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa904102790 0x7fa9041989d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00b3f0 tx=0x7fa8ec00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:39.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.472+0000 7fa901ffb640 1 -- 172.21.15.116:0/108735566 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec015020 con 0x7fa904102790 2024-09-12T03:06:39.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.472+0000 7fa901ffb640 1 -- 172.21.15.116:0/108735566 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8ec011d10 con 0x7fa904102790 2024-09-12T03:06:39.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.472+0000 7fa901ffb640 1 -- 172.21.15.116:0/108735566 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec00f2e0 con 0x7fa904102790 2024-09-12T03:06:39.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.472+0000 7fa90a3f4640 1 -- 172.21.15.116:0/108735566 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa90419d730 con 0x7fa904102790 2024-09-12T03:06:39.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.472+0000 7fa90a3f4640 1 -- 172.21.15.116:0/108735566 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9041a5010 con 0x7fa904102790 2024-09-12T03:06:39.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.474+0000 7fa901ffb640 1 -- 172.21.15.116:0/108735566 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa8ec018020 con 0x7fa904102790 2024-09-12T03:06:39.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.474+0000 7fa90c7cd740 1 Processor -- start 2024-09-12T03:06:39.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.474+0000 7fa90c7cd740 1 -- start start 2024-09-12T03:06:39.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.475+0000 7fa90c7cd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2ba32b60 0x55fc2ba32f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.475+0000 7fa90c7cd740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc2ba33480 0x55fc2ba3d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.475+0000 7fa901ffb640 1 --2- 172.21.15.116:0/108735566 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa8f00a2b80 0x7fa8f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.475+0000 7fa90c7cd740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc2ba3dd40 0x55fc2ba40130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.475+0000 7fa90c7cd740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55fc2b998280 con 0x55fc2ba3dd40 2024-09-12T03:06:39.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.475+0000 7fa90c7cd740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55fc2b93fb10 con 0x55fc2ba33480 2024-09-12T03:06:39.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.475+0000 7fa90c7cd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fc2b994240 con 0x55fc2ba32b60 2024-09-12T03:06:39.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.475+0000 7fa908991640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc2ba3dd40 0x55fc2ba40130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.475+0000 7fa8fbfff640 1 --2- 172.21.15.116:0/108735566 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa8f00a2b80 0x7fa8f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.475+0000 7fa908991640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc2ba3dd40 0x55fc2ba40130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43004/0 (socket says 172.21.15.116:43004) 2024-09-12T03:06:39.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.475+0000 7fa908991640 1 -- 172.21.15.116:0/3004937192 learned_addr learned my addr 172.21.15.116:0/3004937192 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:39.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.475+0000 7fa903fff640 1 --2- 172.21.15.116:0/3004937192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc2ba33480 0x55fc2ba3d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.475+0000 7fa908991640 1 --2- 172.21.15.116:0/3004937192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2ba32b60 0x55fc2ba32f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.475+0000 7fa901ffb640 1 -- 172.21.15.116:0/108735566 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa8ec0ce860 con 0x7fa904102790 2024-09-12T03:06:39.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.475+0000 7fa908991640 1 -- 172.21.15.116:0/3004937192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2ba32b60 msgr2=0x55fc2ba32f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:39.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.475+0000 7fa908991640 1 --2- 172.21.15.116:0/3004937192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2ba32b60 0x55fc2ba32f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.475+0000 7fa908991640 1 -- 172.21.15.116:0/3004937192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc2ba33480 msgr2=0x55fc2ba3d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:39.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.475+0000 7fa908991640 1 --2- 172.21.15.116:0/3004937192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc2ba33480 0x55fc2ba3d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.475+0000 7fa908991640 1 -- 172.21.15.116:0/3004937192 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc2b94c1f0 con 0x55fc2ba3dd40 2024-09-12T03:06:39.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.475+0000 7fa8fbfff640 1 --2- 172.21.15.116:0/108735566 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa8f00a2b80 0x7fa8f00a5040 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec0cc3e0 tx=0x7fa8ec0bb040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:39.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.475+0000 7fa908991640 1 --2- 172.21.15.116:0/3004937192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc2ba3dd40 0x55fc2ba40130 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc012a10 tx=0x7fa8fc0097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f67521cb300520 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:39.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.476+0000 7fa8fa7fc640 1 -- 172.21.15.116:0/3004937192 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc010830 con 0x55fc2ba3dd40 2024-09-12T03:06:39.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.476+0000 7fa8fa7fc640 1 -- 172.21.15.116:0/3004937192 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8fc01e3c0 con 0x55fc2ba3dd40 2024-09-12T03:06:39.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.476+0000 7fa8fa7fc640 1 -- 172.21.15.116:0/3004937192 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc011640 con 0x55fc2ba3dd40 2024-09-12T03:06:39.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.476+0000 7fa90c7cd740 1 -- 172.21.15.116:0/3004937192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc2ba3dd40 msgr2=0x55fc2ba40130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:39.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.476+0000 7fa90c7cd740 1 --2- 172.21.15.116:0/3004937192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc2ba3dd40 0x55fc2ba40130 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc012a10 tx=0x7fa8fc0097b0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.476+0000 7fa90c7cd740 1 -- 172.21.15.116:0/3004937192 shutdown_connections 2024-09-12T03:06:39.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.476+0000 7fa90c7cd740 1 --2- 172.21.15.116:0/3004937192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc2ba3dd40 0x55fc2ba40130 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.476+0000 7fa90c7cd740 1 --2- 172.21.15.116:0/3004937192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc2ba33480 0x55fc2ba3d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.476+0000 7fa90c7cd740 1 --2- 172.21.15.116:0/3004937192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2ba32b60 0x55fc2ba32f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.476+0000 7fa90c7cd740 1 -- 172.21.15.116:0/3004937192 >> 172.21.15.116:0/3004937192 conn(0x55fc2ba2f0f0 msgr2=0x55fc2ba2f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:39.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.476+0000 7fa90c7cd740 1 -- 172.21.15.116:0/3004937192 shutdown_connections 2024-09-12T03:06:39.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.476+0000 7fa90c7cd740 1 -- 172.21.15.116:0/3004937192 wait complete. 2024-09-12T03:06:39.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.477+0000 7fa90c7cd740 1 -- start start 2024-09-12T03:06:39.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.477+0000 7fa90c7cd740 1 Processor -- start 2024-09-12T03:06:39.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.477+0000 7fa90c7cd740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc2ba32b60 0x55fc2ba3ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.477+0000 7fa90c7cd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2ba33480 0x55fc2ba3b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.477+0000 7fa90c7cd740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc2ba3dd40 0x55fc2ba3b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.477+0000 7fa90c7cd740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55fc2ba40910 con 0x55fc2ba32b60 2024-09-12T03:06:39.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.477+0000 7fa90c7cd740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55fc2b952f10 con 0x55fc2ba3dd40 2024-09-12T03:06:39.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.477+0000 7fa90c7cd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fc2ba40a90 con 0x55fc2ba33480 2024-09-12T03:06:39.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.477+0000 7fa90c7cd740 10 client.0 _ll_register_callbacks cb 0x55fc2b91ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:39.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.477+0000 7fa908991640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc2ba32b60 0x55fc2ba3ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.477+0000 7fa908991640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc2ba32b60 0x55fc2ba3ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43018/0 (socket says 172.21.15.116:43018) 2024-09-12T03:06:39.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.477+0000 7fa908991640 1 -- 172.21.15.116:0/197903170 learned_addr learned my addr 172.21.15.116:0/197903170 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:39.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.478+0000 7fa908991640 1 --2- 172.21.15.116:0/197903170 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc2ba3dd40 0x55fc2ba3b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.478+0000 7fa908991640 1 -- 172.21.15.116:0/197903170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2ba33480 msgr2=0x55fc2ba3b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:39.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.478+0000 7fa908991640 1 --2- 172.21.15.116:0/197903170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2ba33480 0x55fc2ba3b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.478+0000 7fa908991640 1 -- 172.21.15.116:0/197903170 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc2ba3dd40 msgr2=0x55fc2ba3b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:39.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.478+0000 7fa908991640 1 --2- 172.21.15.116:0/197903170 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc2ba3dd40 0x55fc2ba3b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.478+0000 7fa908991640 1 -- 172.21.15.116:0/197903170 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc2b7096a0 con 0x55fc2ba32b60 2024-09-12T03:06:39.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.478+0000 7fa908991640 1 --2- 172.21.15.116:0/197903170 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc2ba32b60 0x55fc2ba3ac40 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc01ea70 tx=0x7fa8fc01ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ebd41b49b55b30a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:39.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.478+0000 7fa8e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:06:39.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.478+0000 7fa8e37fe640 1 -- 172.21.15.116:0/197903170 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc004a40 con 0x55fc2ba32b60 2024-09-12T03:06:39.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.478+0000 7fa8e37fe640 1 -- 172.21.15.116:0/197903170 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa8fc004be0 con 0x55fc2ba32b60 2024-09-12T03:06:39.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.478+0000 7fa90c7cd740 1 -- 172.21.15.116:0/197903170 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8fc01e3c0 con 0x55fc2ba32b60 2024-09-12T03:06:39.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.478+0000 7fa8e37fe640 1 -- 172.21.15.116:0/197903170 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa8fc02c070 con 0x55fc2ba32b60 2024-09-12T03:06:39.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.478+0000 7fa90c7cd740 10 client.16230 fetch_fsmap learned FSMap version 25 2024-09-12T03:06:39.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.478+0000 7fa90c7cd740 1 -- 172.21.15.116:0/197903170 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fc2ba4c540 con 0x55fc2ba32b60 2024-09-12T03:06:39.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.478+0000 7fa8e37fe640 1 -- 172.21.15.116:0/197903170 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8fc01e3c0 con 0x55fc2ba32b60 2024-09-12T03:06:39.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.479+0000 7fa8e37fe640 1 -- 172.21.15.116:0/197903170 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa8fc00c030 con 0x55fc2ba32b60 2024-09-12T03:06:39.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.479+0000 7fa90c7cd740 10 client.16230 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:06:39.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.479+0000 7fa90c7cd740 10 client.16230 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:06:39.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.479+0000 7fa90c7cd740 20 client.16230 populate_metadata read hostname 'smithi116' 2024-09-12T03:06:39.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.479+0000 7fa90c7cd740 1 --2- 172.21.15.116:0/197903170 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55fc2ba4d8e0 0x55fc2ba4fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.479+0000 7fa90c7cd740 4 client.16230 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:06:39.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.479+0000 7fa90c7cd740 1 -- 172.21.15.116:0/197903170 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc2b9a57f0 con 0x55fc2ba4d8e0 2024-09-12T03:06:39.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.479+0000 7fa903fff640 1 --2- 172.21.15.116:0/197903170 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55fc2ba4d8e0 0x55fc2ba4fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.479+0000 7fa903fff640 1 --2- 172.21.15.116:0/197903170 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55fc2ba4d8e0 0x55fc2ba4fcd0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19cdba0dbbad4e2b server_cookie=d54bc8286f58007c in_seq=0 out_seq=0 2024-09-12T03:06:39.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.479+0000 7fa8e37fe640 10 client.16230 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:39.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.480+0000 7fa8e37fe640 1 -- 172.21.15.116:0/197903170 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fc2b9a57f0 con 0x55fc2ba4d8e0 2024-09-12T03:06:39.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.480+0000 7fa8e37fe640 10 client.16230 handle_command_reply: tid=0 2024-09-12T03:06:39.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.480+0000 7fa90c7cd740 1 client.16230 shutdown 2024-09-12T03:06:39.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.480+0000 7fa90c7cd740 1 -- 172.21.15.116:0/197903170 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc2ba32b60 msgr2=0x55fc2ba3ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:39.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.480+0000 7fa90c7cd740 1 --2- 172.21.15.116:0/197903170 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc2ba32b60 0x55fc2ba3ac40 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc01ea70 tx=0x7fa8fc01ed30 comp rx=0 tx=0).stop 2024-09-12T03:06:39.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.480+0000 7fa90c7cd740 1 -- 172.21.15.116:0/197903170 shutdown_connections 2024-09-12T03:06:39.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.480+0000 7fa90c7cd740 1 --2- 172.21.15.116:0/197903170 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55fc2ba4d8e0 0x55fc2ba4fcd0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.480+0000 7fa90c7cd740 1 --2- 172.21.15.116:0/197903170 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc2ba3dd40 0x55fc2ba3b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.480+0000 7fa90c7cd740 1 --2- 172.21.15.116:0/197903170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2ba33480 0x55fc2ba3b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.480+0000 7fa90c7cd740 1 --2- 172.21.15.116:0/197903170 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc2ba32b60 0x55fc2ba3ac40 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.480+0000 7fa8e37fe640 0 client.16230 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:39.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.480+0000 7fa90c7cd740 1 -- 172.21.15.116:0/197903170 >> 172.21.15.116:0/197903170 conn(0x55fc2ba2f0f0 msgr2=0x55fc2ba34dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:39.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.480+0000 7fa90c7cd740 1 -- 172.21.15.116:0/197903170 shutdown_connections 2024-09-12T03:06:39.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.480+0000 7fa90c7cd740 1 -- 172.21.15.116:0/197903170 wait complete. 2024-09-12T03:06:39.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.480+0000 7fa90c7cd740 20 client.16230 trim_cache size 0 max 16384 2024-09-12T03:06:39.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.497+0000 7fa90c7cd740 1 Processor -- start 2024-09-12T03:06:39.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.497+0000 7fa90c7cd740 1 -- start start 2024-09-12T03:06:39.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.498+0000 7fa90c7cd740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc2ba32b60 0x55fc2ba2e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.498+0000 7fa90c7cd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2ba33480 0x55fc2ba3a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.498+0000 7fa90c7cd740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc2ba3dd40 0x55fc2ba460c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.498+0000 7fa90c7cd740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa9041afde0 con 0x55fc2ba32b60 2024-09-12T03:06:39.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.498+0000 7fa90c7cd740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa9041ad6b0 con 0x55fc2ba3dd40 2024-09-12T03:06:39.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.498+0000 7fa90c7cd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa9041ab500 con 0x55fc2ba33480 2024-09-12T03:06:39.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.498+0000 7fa908991640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc2ba32b60 0x55fc2ba2e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.498+0000 7fa908991640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc2ba32b60 0x55fc2ba2e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43028/0 (socket says 172.21.15.116:43028) 2024-09-12T03:06:39.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.498+0000 7fa908991640 1 -- 172.21.15.116:0/3813207503 learned_addr learned my addr 172.21.15.116:0/3813207503 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:39.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.498+0000 7fa903fff640 1 --2- 172.21.15.116:0/3813207503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2ba33480 0x55fc2ba3a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.498+0000 7fa908991640 1 --2- 172.21.15.116:0/3813207503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc2ba3dd40 0x55fc2ba460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.498+0000 7fa908991640 1 -- 172.21.15.116:0/3813207503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2ba33480 msgr2=0x55fc2ba3a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:39.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.498+0000 7fa908991640 1 --2- 172.21.15.116:0/3813207503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2ba33480 0x55fc2ba3a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.498+0000 7fa908991640 1 -- 172.21.15.116:0/3813207503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc2ba3dd40 msgr2=0x55fc2ba460c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:39.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.498+0000 7fa908991640 1 --2- 172.21.15.116:0/3813207503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc2ba3dd40 0x55fc2ba460c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.498+0000 7fa908991640 1 -- 172.21.15.116:0/3813207503 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc2ba3d220 con 0x55fc2ba32b60 2024-09-12T03:06:39.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.498+0000 7fa908991640 1 --2- 172.21.15.116:0/3813207503 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc2ba32b60 0x55fc2ba2e160 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc01ede0 tx=0x7fa8fc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a8999ecf58c1890 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:39.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.498+0000 7fa8f9ffb640 1 -- 172.21.15.116:0/3813207503 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc009670 con 0x55fc2ba32b60 2024-09-12T03:06:39.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.499+0000 7fa8f9ffb640 1 -- 172.21.15.116:0/3813207503 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8fc016030 con 0x55fc2ba32b60 2024-09-12T03:06:39.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.499+0000 7fa8f9ffb640 1 -- 172.21.15.116:0/3813207503 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc010830 con 0x55fc2ba32b60 2024-09-12T03:06:39.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.499+0000 7fa90c7cd740 1 -- 172.21.15.116:0/3813207503 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc2ba32b60 msgr2=0x55fc2ba2e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:39.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.499+0000 7fa90c7cd740 1 --2- 172.21.15.116:0/3813207503 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc2ba32b60 0x55fc2ba2e160 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc01ede0 tx=0x7fa8fc005fd0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.499+0000 7fa90c7cd740 1 -- 172.21.15.116:0/3813207503 shutdown_connections 2024-09-12T03:06:39.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.499+0000 7fa90c7cd740 1 --2- 172.21.15.116:0/3813207503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc2ba3dd40 0x55fc2ba460c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.499+0000 7fa90c7cd740 1 --2- 172.21.15.116:0/3813207503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2ba33480 0x55fc2ba3a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.499+0000 7fa90c7cd740 1 --2- 172.21.15.116:0/3813207503 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc2ba32b60 0x55fc2ba2e160 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.499+0000 7fa90c7cd740 1 -- 172.21.15.116:0/3813207503 >> 172.21.15.116:0/3813207503 conn(0x55fc2ba2f0f0 msgr2=0x55fc2ba2da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:39.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.499+0000 7fa90c7cd740 1 -- 172.21.15.116:0/3813207503 shutdown_connections 2024-09-12T03:06:39.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.499+0000 7fa90c7cd740 1 -- 172.21.15.116:0/3813207503 wait complete. 2024-09-12T03:06:39.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.499+0000 7fa90c7cd740 1 -- start start 2024-09-12T03:06:39.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.500+0000 7fa90c7cd740 1 Processor -- start 2024-09-12T03:06:39.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.500+0000 7fa90c7cd740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc2ba32b60 0x55fc2bab03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.500+0000 7fa90c7cd740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc2ba33480 0x55fc2bab0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.500+0000 7fa90c7cd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2ba3dd40 0x55fc2babb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.500+0000 7fa90c7cd740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa9041abb70 con 0x55fc2ba33480 2024-09-12T03:06:39.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.500+0000 7fa90c7cd740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa9041acfe0 con 0x55fc2ba32b60 2024-09-12T03:06:39.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.500+0000 7fa90c7cd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa9041ac240 con 0x55fc2ba3dd40 2024-09-12T03:06:39.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.500+0000 7fa90c7cd740 10 client.0 _ll_register_callbacks cb 0x7fa8f4009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:39.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.500+0000 7fa903fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc2ba33480 0x55fc2bab0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.500+0000 7fa903fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc2ba33480 0x55fc2bab0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43036/0 (socket says 172.21.15.116:43036) 2024-09-12T03:06:39.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.500+0000 7fa903fff640 1 -- 172.21.15.116:0/1697646127 learned_addr learned my addr 172.21.15.116:0/1697646127 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:39.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.500+0000 7fa908991640 1 --2- 172.21.15.116:0/1697646127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2ba3dd40 0x55fc2babb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.500+0000 7fa908991640 1 --2- 172.21.15.116:0/1697646127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc2ba32b60 0x55fc2bab03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.500+0000 7fa903fff640 1 -- 172.21.15.116:0/1697646127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2ba3dd40 msgr2=0x55fc2babb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:39.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.500+0000 7fa903fff640 1 --2- 172.21.15.116:0/1697646127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2ba3dd40 0x55fc2babb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.500+0000 7fa903fff640 1 -- 172.21.15.116:0/1697646127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc2ba32b60 msgr2=0x55fc2bab03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:39.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.500+0000 7fa903fff640 1 --2- 172.21.15.116:0/1697646127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc2ba32b60 0x55fc2bab03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.500+0000 7fa903fff640 1 -- 172.21.15.116:0/1697646127 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc2babb990 con 0x55fc2ba33480 2024-09-12T03:06:39.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.500+0000 7fa908991640 1 --2- 172.21.15.116:0/1697646127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2ba3dd40 0x55fc2babb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:06:39.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.501+0000 7fa903fff640 1 --2- 172.21.15.116:0/1697646127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc2ba33480 0x55fc2bab0910 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa8f40092a0 tx=0x7fa8f4004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64363f44ba1089b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:39.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.501+0000 7fa8e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:06:39.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.501+0000 7fa8e37fe640 1 -- 172.21.15.116:0/1697646127 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4035040 con 0x55fc2ba33480 2024-09-12T03:06:39.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.501+0000 7fa8e37fe640 1 -- 172.21.15.116:0/1697646127 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa8f4005240 con 0x55fc2ba33480 2024-09-12T03:06:39.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.501+0000 7fa90c7cd740 1 -- 172.21.15.116:0/1697646127 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8fc016030 con 0x55fc2ba33480 2024-09-12T03:06:39.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.501+0000 7fa8e37fe640 1 -- 172.21.15.116:0/1697646127 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa8f4054050 con 0x55fc2ba33480 2024-09-12T03:06:39.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.501+0000 7fa90c7cd740 10 client.16236 fetch_fsmap learned FSMap version 25 2024-09-12T03:06:39.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.501+0000 7fa90c7cd740 1 -- 172.21.15.116:0/1697646127 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fc2babcb10 con 0x55fc2ba33480 2024-09-12T03:06:39.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.501+0000 7fa8e37fe640 1 -- 172.21.15.116:0/1697646127 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8fc016030 con 0x55fc2ba33480 2024-09-12T03:06:39.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.501+0000 7fa8e37fe640 1 -- 172.21.15.116:0/1697646127 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa8f4003710 con 0x55fc2ba33480 2024-09-12T03:06:39.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.501+0000 7fa90c7cd740 10 client.16236 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:06:39.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.501+0000 7fa90c7cd740 10 client.16236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:06:39.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.501+0000 7fa90c7cd740 20 client.16236 populate_metadata read hostname 'smithi116' 2024-09-12T03:06:39.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.501+0000 7fa90c7cd740 1 --2- 172.21.15.116:0/1697646127 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55fc2babd270 0x55fc2babf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.501+0000 7fa90c7cd740 4 client.16236 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:06:39.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.501+0000 7fa90c7cd740 1 -- 172.21.15.116:0/1697646127 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fc2b99fd00 con 0x55fc2babd270 2024-09-12T03:06:39.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.502+0000 7fa908991640 1 --2- 172.21.15.116:0/1697646127 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55fc2babd270 0x55fc2babf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.502+0000 7fa8e37fe640 10 client.16236 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:39.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.502+0000 7fa908991640 1 --2- 172.21.15.116:0/1697646127 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55fc2babd270 0x55fc2babf690 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43b386b86fd9791 server_cookie=16ca91403ee5ee5 in_seq=0 out_seq=0 2024-09-12T03:06:39.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.503+0000 7fa8e37fe640 1 -- 172.21.15.116:0/1697646127 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fc2b99fd00 con 0x55fc2babd270 2024-09-12T03:06:39.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.503+0000 7fa8e37fe640 10 client.16236 handle_command_reply: tid=0 2024-09-12T03:06:39.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.503+0000 7fa90c7cd740 1 client.16236 shutdown 2024-09-12T03:06:39.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.503+0000 7fa90c7cd740 1 -- 172.21.15.116:0/1697646127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc2ba33480 msgr2=0x55fc2bab0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:39.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.503+0000 7fa90c7cd740 1 --2- 172.21.15.116:0/1697646127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc2ba33480 0x55fc2bab0910 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa8f40092a0 tx=0x7fa8f4004180 comp rx=0 tx=0).stop 2024-09-12T03:06:39.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.503+0000 7fa90c7cd740 1 -- 172.21.15.116:0/1697646127 shutdown_connections 2024-09-12T03:06:39.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.503+0000 7fa90c7cd740 1 --2- 172.21.15.116:0/1697646127 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55fc2babd270 0x55fc2babf690 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.503+0000 7fa90c7cd740 1 --2- 172.21.15.116:0/1697646127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc2ba3dd40 0x55fc2babb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.503+0000 7fa90c7cd740 1 --2- 172.21.15.116:0/1697646127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc2ba33480 0x55fc2bab0910 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.503+0000 7fa90c7cd740 1 --2- 172.21.15.116:0/1697646127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc2ba32b60 0x55fc2bab03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.503+0000 7fa90c7cd740 1 -- 172.21.15.116:0/1697646127 >> 172.21.15.116:0/1697646127 conn(0x55fc2ba2f0f0 msgr2=0x55fc2ba2e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:39.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.503+0000 7fa8e37fe640 0 client.16236 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:39.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.504+0000 7fa90c7cd740 1 -- 172.21.15.116:0/1697646127 shutdown_connections 2024-09-12T03:06:39.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.504+0000 7fa90c7cd740 1 -- 172.21.15.116:0/1697646127 wait complete. 2024-09-12T03:06:39.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.504+0000 7fa90c7cd740 20 client.16236 trim_cache size 0 max 16384 2024-09-12T03:06:39.516 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:06:39.516 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:06:39.516 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "6ad7a7a0-2dcc-4482-9b31-9d77ead6510e", 2024-09-12T03:06:39.516 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:06:39.516 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:06:39.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.504+0000 7fa90a3f4640 1 -- 172.21.15.116:0/108735566 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa8f00a2b80 msgr2=0x7fa8f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:39.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.504+0000 7fa90a3f4640 1 --2- 172.21.15.116:0/108735566 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa8f00a2b80 0x7fa8f00a5040 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec0cc3e0 tx=0x7fa8ec0bb040 comp rx=0 tx=0).stop 2024-09-12T03:06:39.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.504+0000 7fa90a3f4640 1 -- 172.21.15.116:0/108735566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa904102790 msgr2=0x7fa9041989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:39.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.504+0000 7fa90a3f4640 1 --2- 172.21.15.116:0/108735566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa904102790 0x7fa9041989d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00b3f0 tx=0x7fa8ec00b8b0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.504+0000 7fa90a3f4640 1 -- 172.21.15.116:0/108735566 shutdown_connections 2024-09-12T03:06:39.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.504+0000 7fa90a3f4640 1 --2- 172.21.15.116:0/108735566 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa8f00a2b80 0x7fa8f00a5040 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.504+0000 7fa90a3f4640 1 --2- 172.21.15.116:0/108735566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa904106f10 0x7fa90419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.504+0000 7fa90a3f4640 1 --2- 172.21.15.116:0/108735566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa904102790 0x7fa9041989d0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.504+0000 7fa90a3f4640 1 --2- 172.21.15.116:0/108735566 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa904101de0 0x7fa904198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.504+0000 7fa90a3f4640 1 -- 172.21.15.116:0/108735566 >> 172.21.15.116:0/108735566 conn(0x7fa9040fd8c0 msgr2=0x7fa9040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:39.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.504+0000 7fa90a3f4640 1 -- 172.21.15.116:0/108735566 shutdown_connections 2024-09-12T03:06:39.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:39.504+0000 7fa90a3f4640 1 -- 172.21.15.116:0/108735566 wait complete. 2024-09-12T03:06:40.354 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:06:40.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:40 smithi116 ceph-mon[31410]: pgmap v448: 97 pgs: 97 active+clean; 1.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:06:40.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:40 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:06:40.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:40 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:06:40.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:40 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:06:40.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:40 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:06:40.598 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:06:40.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:40 smithi183 ceph-mon[35535]: pgmap v448: 97 pgs: 97 active+clean; 1.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:06:40.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:40 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:06:40.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:40 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:06:40.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:40 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:06:40.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:40 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:06:40.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:40 smithi177 ceph-mon[35305]: pgmap v448: 97 pgs: 97 active+clean; 1.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:06:40.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:40 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:06:40.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:40 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:06:40.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:40 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:06:40.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:40 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:06:41.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.094+0000 7f028ae5b640 1 -- 172.21.15.116:0/3047992604 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0284101df0 msgr2=0x7f02841021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:41.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.094+0000 7f028ae5b640 1 --2- 172.21.15.116:0/3047992604 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0284101df0 0x7f02841021d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f0278009900 tx=0x7f027802a410 comp rx=0 tx=0).stop 2024-09-12T03:06:41.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.094+0000 7f028ae5b640 1 -- 172.21.15.116:0/3047992604 shutdown_connections 2024-09-12T03:06:41.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.094+0000 7f028ae5b640 1 --2- 172.21.15.116:0/3047992604 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0284106f20 0x7f028410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.094+0000 7f028ae5b640 1 --2- 172.21.15.116:0/3047992604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02841027a0 0x7f02841067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.094+0000 7f028ae5b640 1 --2- 172.21.15.116:0/3047992604 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0284101df0 0x7f02841021d0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.094+0000 7f028ae5b640 1 -- 172.21.15.116:0/3047992604 >> 172.21.15.116:0/3047992604 conn(0x7f02840fd8b0 msgr2=0x7f02840ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:41.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.095+0000 7f028ae5b640 1 -- 172.21.15.116:0/3047992604 shutdown_connections 2024-09-12T03:06:41.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.095+0000 7f028ae5b640 1 -- 172.21.15.116:0/3047992604 wait complete. 2024-09-12T03:06:41.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.095+0000 7f028ae5b640 1 Processor -- start 2024-09-12T03:06:41.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.095+0000 7f028ae5b640 1 -- start start 2024-09-12T03:06:41.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.095+0000 7f028ae5b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0284101df0 0x7f02840744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:41.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.095+0000 7f028ae5b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02841027a0 0x7f0284074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:41.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.095+0000 7f028ae5b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0284106f20 0x7f0284074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:41.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.095+0000 7f028ae5b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0284111fa0 con 0x7f0284106f20 2024-09-12T03:06:41.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.095+0000 7f028ae5b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0284111e20 con 0x7f0284101df0 2024-09-12T03:06:41.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.095+0000 7f028ae5b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0284112120 con 0x7f02841027a0 2024-09-12T03:06:41.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.095+0000 7f02893f8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0284106f20 0x7f0284074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:41.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.096+0000 7f02893f8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0284106f20 0x7f0284074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41112/0 (socket says 172.21.15.116:41112) 2024-09-12T03:06:41.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.096+0000 7f02893f8640 1 -- 172.21.15.116:0/653395678 learned_addr learned my addr 172.21.15.116:0/653395678 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:41.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.096+0000 7f0283fff640 1 --2- 172.21.15.116:0/653395678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02841027a0 0x7f0284074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:41.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.096+0000 7f0288bf7640 1 --2- 172.21.15.116:0/653395678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0284101df0 0x7f02840744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:41.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.096+0000 7f02893f8640 1 -- 172.21.15.116:0/653395678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02841027a0 msgr2=0x7f0284074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:41.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.096+0000 7f02893f8640 1 --2- 172.21.15.116:0/653395678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02841027a0 0x7f0284074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.096+0000 7f02893f8640 1 -- 172.21.15.116:0/653395678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0284101df0 msgr2=0x7f02840744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:41.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.096+0000 7f02893f8640 1 --2- 172.21.15.116:0/653395678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0284101df0 0x7f02840744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.096+0000 7f02893f8640 1 -- 172.21.15.116:0/653395678 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02841a6990 con 0x7f0284106f20 2024-09-12T03:06:41.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.096+0000 7f02893f8640 1 --2- 172.21.15.116:0/653395678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0284106f20 0x7f0284074f70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f027c00cce0 tx=0x7f027c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:41.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.096+0000 7f0281ffb640 1 -- 172.21.15.116:0/653395678 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c00f450 con 0x7f0284106f20 2024-09-12T03:06:41.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.096+0000 7f0281ffb640 1 -- 172.21.15.116:0/653395678 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f027c00fad0 con 0x7f0284106f20 2024-09-12T03:06:41.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.096+0000 7f028ae5b640 1 -- 172.21.15.116:0/653395678 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02841a6c80 con 0x7f0284106f20 2024-09-12T03:06:41.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.096+0000 7f0281ffb640 1 -- 172.21.15.116:0/653395678 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c018c10 con 0x7f0284106f20 2024-09-12T03:06:41.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.096+0000 7f028ae5b640 1 -- 172.21.15.116:0/653395678 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02841a71c0 con 0x7f0284106f20 2024-09-12T03:06:41.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.098+0000 7f0281ffb640 1 -- 172.21.15.116:0/653395678 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f027c00f5f0 con 0x7f0284106f20 2024-09-12T03:06:41.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.099+0000 7f028d234740 1 Processor -- start 2024-09-12T03:06:41.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.099+0000 7f028d234740 1 -- start start 2024-09-12T03:06:41.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.099+0000 7f0281ffb640 1 --2- 172.21.15.116:0/653395678 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f02740a2c00 0x7f02740a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:41.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.099+0000 7f028d234740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642cff534a0 0x5642cff53880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:41.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.099+0000 7f028d234740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642cff53dc0 0x5642cff5e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:41.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.099+0000 7f028d234740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5642cff5e680 0x5642cff60a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:41.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.099+0000 7f028d234740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5642cfeb92e0 con 0x5642cff53dc0 2024-09-12T03:06:41.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.099+0000 7f028d234740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5642cfe60ce0 con 0x5642cff534a0 2024-09-12T03:06:41.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.099+0000 7f028d234740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5642cfeb52a0 con 0x5642cff5e680 2024-09-12T03:06:41.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.099+0000 7f0281ffb640 1 -- 172.21.15.116:0/653395678 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f027c0cf700 con 0x7f0284106f20 2024-09-12T03:06:41.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.099+0000 7f0283fff640 1 --2- 172.21.15.116:0/653395678 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f02740a2c00 0x7f02740a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:41.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.099+0000 7f02893f8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5642cff5e680 0x5642cff60a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:41.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.099+0000 7f0288bf7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642cff534a0 0x5642cff53880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:41.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.099+0000 7f02893f8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5642cff5e680 0x5642cff60a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.116:47170/0 (socket says 172.21.15.116:47170) 2024-09-12T03:06:41.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.099+0000 7f0288bf7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642cff534a0 0x5642cff53880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.116:33532/0 (socket says 172.21.15.116:33532) 2024-09-12T03:06:41.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.099+0000 7f02893f8640 1 -- 172.21.15.116:0/1093128107 learned_addr learned my addr 172.21.15.116:0/1093128107 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:41.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.099+0000 7f0283fff640 1 --2- 172.21.15.116:0/1093128107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642cff53dc0 0x5642cff5e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:41.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.100+0000 7f02893f8640 1 -- 172.21.15.116:0/1093128107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642cff534a0 msgr2=0x5642cff53880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:41.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.100+0000 7f02893f8640 1 --2- 172.21.15.116:0/1093128107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642cff534a0 0x5642cff53880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.100+0000 7f02893f8640 1 -- 172.21.15.116:0/1093128107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642cff53dc0 msgr2=0x5642cff5e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:41.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.100+0000 7f02893f8640 1 --2- 172.21.15.116:0/1093128107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642cff53dc0 0x5642cff5e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.100+0000 7f02893f8640 1 -- 172.21.15.116:0/1093128107 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642cfe45440 con 0x5642cff5e680 2024-09-12T03:06:41.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.100+0000 7f0283fff640 1 --2- 172.21.15.116:0/1093128107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642cff53dc0 0x5642cff5e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:06:41.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.100+0000 7f02893f8640 1 --2- 172.21.15.116:0/1093128107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5642cff5e680 0x5642cff60a70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f027c09bf20 tx=0x7f027c09bb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfa4d2bcdd7f071d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:41.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.100+0000 7f0283fff640 1 --2- 172.21.15.116:0/653395678 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f02740a2c00 0x7f02740a50c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5642cff50890 tx=0x7f027000a0d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:41.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.100+0000 7f026effd640 1 -- 172.21.15.116:0/1093128107 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c0641a0 con 0x5642cff5e680 2024-09-12T03:06:41.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.100+0000 7f026effd640 1 -- 172.21.15.116:0/1093128107 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f027c09c690 con 0x5642cff5e680 2024-09-12T03:06:41.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.100+0000 7f026effd640 1 -- 172.21.15.116:0/1093128107 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c016de0 con 0x5642cff5e680 2024-09-12T03:06:41.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.101+0000 7f028d234740 1 -- 172.21.15.116:0/1093128107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5642cff5e680 msgr2=0x5642cff60a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:41.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.101+0000 7f028d234740 1 --2- 172.21.15.116:0/1093128107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5642cff5e680 0x5642cff60a70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f027c09bf20 tx=0x7f027c09bb90 comp rx=0 tx=0).stop 2024-09-12T03:06:41.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.101+0000 7f028d234740 1 -- 172.21.15.116:0/1093128107 shutdown_connections 2024-09-12T03:06:41.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.101+0000 7f028d234740 1 --2- 172.21.15.116:0/1093128107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5642cff5e680 0x5642cff60a70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.101+0000 7f028d234740 1 --2- 172.21.15.116:0/1093128107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642cff53dc0 0x5642cff5e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.101+0000 7f028d234740 1 --2- 172.21.15.116:0/1093128107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642cff534a0 0x5642cff53880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.101+0000 7f028d234740 1 -- 172.21.15.116:0/1093128107 >> 172.21.15.116:0/1093128107 conn(0x5642cff4fa30 msgr2=0x5642cff4fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:41.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.101+0000 7f028d234740 1 -- 172.21.15.116:0/1093128107 shutdown_connections 2024-09-12T03:06:41.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.101+0000 7f028d234740 1 -- 172.21.15.116:0/1093128107 wait complete. 2024-09-12T03:06:41.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.101+0000 7f028d234740 1 -- start start 2024-09-12T03:06:41.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.101+0000 7f028d234740 1 Processor -- start 2024-09-12T03:06:41.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.102+0000 7f028d234740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5642cff534a0 0x5642cff5e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:41.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.102+0000 7f028d234740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642cff53dc0 0x5642cff5b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:41.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.102+0000 7f028d234740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642cff5e680 0x5642cff5b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:41.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.102+0000 7f028d234740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5642cff61250 con 0x5642cff5e680 2024-09-12T03:06:41.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.102+0000 7f028d234740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5642cfe74050 con 0x5642cff53dc0 2024-09-12T03:06:41.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.102+0000 7f028d234740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5642cff613d0 con 0x5642cff534a0 2024-09-12T03:06:41.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.102+0000 7f028d234740 10 client.0 _ll_register_callbacks cb 0x5642cfeb4160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:41.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.102+0000 7f02893f8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642cff5e680 0x5642cff5b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:41.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.102+0000 7f02893f8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642cff5e680 0x5642cff5b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41122/0 (socket says 172.21.15.116:41122) 2024-09-12T03:06:41.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.102+0000 7f02893f8640 1 -- 172.21.15.116:0/2006025759 learned_addr learned my addr 172.21.15.116:0/2006025759 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:41.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.102+0000 7f0288bf7640 1 --2- 172.21.15.116:0/2006025759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5642cff534a0 0x5642cff5e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:41.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.102+0000 7f0283fff640 1 --2- 172.21.15.116:0/2006025759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642cff53dc0 0x5642cff5b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:41.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.102+0000 7f02893f8640 1 -- 172.21.15.116:0/2006025759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5642cff534a0 msgr2=0x5642cff5e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:41.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.102+0000 7f02893f8640 1 --2- 172.21.15.116:0/2006025759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5642cff534a0 0x5642cff5e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.102+0000 7f02893f8640 1 -- 172.21.15.116:0/2006025759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642cff53dc0 msgr2=0x5642cff5b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:41.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.102+0000 7f02893f8640 1 --2- 172.21.15.116:0/2006025759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642cff53dc0 0x5642cff5b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.102+0000 7f02893f8640 1 -- 172.21.15.116:0/2006025759 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642cfc2a3c0 con 0x5642cff5e680 2024-09-12T03:06:41.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.102+0000 7f02893f8640 1 --2- 172.21.15.116:0/2006025759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642cff5e680 0x5642cff5b830 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f027c0cd280 tx=0x7f027c00f950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5172392b7e472337 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:41.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.102+0000 7f0263fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:06:41.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.102+0000 7f0263fff640 1 -- 172.21.15.116:0/2006025759 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c0a4c00 con 0x5642cff5e680 2024-09-12T03:06:41.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.102+0000 7f0263fff640 1 -- 172.21.15.116:0/2006025759 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f027c00fc80 con 0x5642cff5e680 2024-09-12T03:06:41.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.102+0000 7f028d234740 1 -- 172.21.15.116:0/2006025759 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f027c09c690 con 0x5642cff5e680 2024-09-12T03:06:41.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.103+0000 7f0263fff640 1 -- 172.21.15.116:0/2006025759 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f027c0a0340 con 0x5642cff5e680 2024-09-12T03:06:41.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.103+0000 7f028d234740 10 client.16248 fetch_fsmap learned FSMap version 25 2024-09-12T03:06:41.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.103+0000 7f028d234740 1 -- 172.21.15.116:0/2006025759 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5642cff6cc40 con 0x5642cff5e680 2024-09-12T03:06:41.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.103+0000 7f0263fff640 1 -- 172.21.15.116:0/2006025759 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f027c09c690 con 0x5642cff5e680 2024-09-12T03:06:41.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.103+0000 7f0263fff640 1 -- 172.21.15.116:0/2006025759 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f027c0889a0 con 0x5642cff5e680 2024-09-12T03:06:41.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.103+0000 7f028d234740 10 client.16248 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:06:41.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.103+0000 7f028d234740 10 client.16248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:06:41.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.103+0000 7f028d234740 20 client.16248 populate_metadata read hostname 'smithi116' 2024-09-12T03:06:41.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.103+0000 7f028d234740 1 --2- 172.21.15.116:0/2006025759 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5642cff6df90 0x5642cff70380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:41.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.103+0000 7f028d234740 4 client.16248 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:06:41.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.103+0000 7f028d234740 1 -- 172.21.15.116:0/2006025759 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642cfec6850 con 0x5642cff6df90 2024-09-12T03:06:41.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.103+0000 7f0288bf7640 1 --2- 172.21.15.116:0/2006025759 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5642cff6df90 0x5642cff70380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:41.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.104+0000 7f0288bf7640 1 --2- 172.21.15.116:0/2006025759 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5642cff6df90 0x5642cff70380 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa966f07ea51037d server_cookie=45702273740c5db6 in_seq=0 out_seq=0 2024-09-12T03:06:41.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.104+0000 7f0263fff640 10 client.16248 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:41.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.104+0000 7f0263fff640 1 -- 172.21.15.116:0/2006025759 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5642cfec6850 con 0x5642cff6df90 2024-09-12T03:06:41.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.104+0000 7f0263fff640 10 client.16248 handle_command_reply: tid=0 2024-09-12T03:06:41.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.104+0000 7f028d234740 1 client.16248 shutdown 2024-09-12T03:06:41.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.105+0000 7f028d234740 1 -- 172.21.15.116:0/2006025759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642cff5e680 msgr2=0x5642cff5b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:41.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.105+0000 7f028d234740 1 --2- 172.21.15.116:0/2006025759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642cff5e680 0x5642cff5b830 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f027c0cd280 tx=0x7f027c00f950 comp rx=0 tx=0).stop 2024-09-12T03:06:41.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.105+0000 7f028d234740 1 -- 172.21.15.116:0/2006025759 shutdown_connections 2024-09-12T03:06:41.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.105+0000 7f028d234740 1 --2- 172.21.15.116:0/2006025759 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5642cff6df90 0x5642cff70380 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.105+0000 7f028d234740 1 --2- 172.21.15.116:0/2006025759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642cff5e680 0x5642cff5b830 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.105+0000 7f028d234740 1 --2- 172.21.15.116:0/2006025759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642cff53dc0 0x5642cff5b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.105+0000 7f028d234740 1 --2- 172.21.15.116:0/2006025759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5642cff534a0 0x5642cff5e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.105+0000 7f028d234740 1 -- 172.21.15.116:0/2006025759 >> 172.21.15.116:0/2006025759 conn(0x5642cff4fa30 msgr2=0x5642cff55710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:41.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.105+0000 7f0263fff640 0 client.16248 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:41.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.105+0000 7f028d234740 1 -- 172.21.15.116:0/2006025759 shutdown_connections 2024-09-12T03:06:41.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.105+0000 7f028d234740 1 -- 172.21.15.116:0/2006025759 wait complete. 2024-09-12T03:06:41.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.105+0000 7f028d234740 20 client.16248 trim_cache size 0 max 16384 2024-09-12T03:06:41.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.123+0000 7f028d234740 1 Processor -- start 2024-09-12T03:06:41.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.123+0000 7f028d234740 1 -- start start 2024-09-12T03:06:41.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.123+0000 7f028d234740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5642cff534a0 0x5642cff72fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:41.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.123+0000 7f028d234740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642cff53dc0 0x5642cff73500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:41.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.123+0000 7f028d234740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642cff5e680 0x5642cff58510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:41.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.123+0000 7f028d234740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f02841b1e80 con 0x5642cff53dc0 2024-09-12T03:06:41.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.123+0000 7f028d234740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f02841af750 con 0x5642cff5e680 2024-09-12T03:06:41.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.123+0000 7f028d234740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f02841ad5c0 con 0x5642cff534a0 2024-09-12T03:06:41.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.123+0000 7f0283fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642cff53dc0 0x5642cff73500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:41.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.123+0000 7f0283fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642cff53dc0 0x5642cff73500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41136/0 (socket says 172.21.15.116:41136) 2024-09-12T03:06:41.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.123+0000 7f0283fff640 1 -- 172.21.15.116:0/151847064 learned_addr learned my addr 172.21.15.116:0/151847064 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:41.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.123+0000 7f0288bf7640 1 --2- 172.21.15.116:0/151847064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5642cff534a0 0x5642cff72fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:41.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.124+0000 7f02893f8640 1 --2- 172.21.15.116:0/151847064 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642cff5e680 0x5642cff58510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:41.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.124+0000 7f0283fff640 1 -- 172.21.15.116:0/151847064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5642cff534a0 msgr2=0x5642cff72fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:41.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.124+0000 7f0283fff640 1 --2- 172.21.15.116:0/151847064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5642cff534a0 0x5642cff72fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.124+0000 7f0283fff640 1 -- 172.21.15.116:0/151847064 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642cff5e680 msgr2=0x5642cff58510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:41.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.124+0000 7f0283fff640 1 --2- 172.21.15.116:0/151847064 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642cff5e680 0x5642cff58510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.124+0000 7f0283fff640 1 -- 172.21.15.116:0/151847064 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642cff59800 con 0x5642cff53dc0 2024-09-12T03:06:41.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.124+0000 7f02893f8640 1 --2- 172.21.15.116:0/151847064 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642cff5e680 0x5642cff58510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:06:41.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.124+0000 7f0283fff640 1 --2- 172.21.15.116:0/151847064 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642cff53dc0 0x5642cff73500 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f02841087f0 tx=0x7f0270006a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a66047865c70f695 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:41.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.124+0000 7f026e7fc640 1 -- 172.21.15.116:0/151847064 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0270014d30 con 0x5642cff53dc0 2024-09-12T03:06:41.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.124+0000 7f026e7fc640 1 -- 172.21.15.116:0/151847064 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0270027070 con 0x5642cff53dc0 2024-09-12T03:06:41.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.124+0000 7f026e7fc640 1 -- 172.21.15.116:0/151847064 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0270022570 con 0x5642cff53dc0 2024-09-12T03:06:41.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.125+0000 7f028d234740 1 -- 172.21.15.116:0/151847064 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642cff53dc0 msgr2=0x5642cff73500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:41.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.125+0000 7f028d234740 1 --2- 172.21.15.116:0/151847064 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642cff53dc0 0x5642cff73500 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f02841087f0 tx=0x7f0270006a30 comp rx=0 tx=0).stop 2024-09-12T03:06:41.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.125+0000 7f028d234740 1 -- 172.21.15.116:0/151847064 shutdown_connections 2024-09-12T03:06:41.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.125+0000 7f028d234740 1 --2- 172.21.15.116:0/151847064 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642cff5e680 0x5642cff58510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.125+0000 7f028d234740 1 --2- 172.21.15.116:0/151847064 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642cff53dc0 0x5642cff73500 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.125+0000 7f028d234740 1 --2- 172.21.15.116:0/151847064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5642cff534a0 0x5642cff72fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.125+0000 7f028d234740 1 -- 172.21.15.116:0/151847064 >> 172.21.15.116:0/151847064 conn(0x5642cff4fa30 msgr2=0x5642cff5a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:41.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.125+0000 7f028d234740 1 -- 172.21.15.116:0/151847064 shutdown_connections 2024-09-12T03:06:41.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.125+0000 7f028d234740 1 -- 172.21.15.116:0/151847064 wait complete. 2024-09-12T03:06:41.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.125+0000 7f028d234740 1 -- start start 2024-09-12T03:06:41.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.125+0000 7f028d234740 1 Processor -- start 2024-09-12T03:06:41.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.125+0000 7f028d234740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642cff534a0 0x5642cff737f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:41.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.125+0000 7f028d234740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5642cff53dc0 0x5642cff70b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:41.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.126+0000 7f028d234740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642cff5e680 0x5642cff710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:41.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.126+0000 7f028d234740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f02841adc10 con 0x5642cff5e680 2024-09-12T03:06:41.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.126+0000 7f028d234740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f02841af080 con 0x5642cff534a0 2024-09-12T03:06:41.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.126+0000 7f028d234740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f02841ae2e0 con 0x5642cff53dc0 2024-09-12T03:06:41.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.126+0000 7f028d234740 10 client.0 _ll_register_callbacks cb 0x7f027000a820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:41.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.126+0000 7f02893f8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642cff5e680 0x5642cff710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:41.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.126+0000 7f02893f8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642cff5e680 0x5642cff710e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41148/0 (socket says 172.21.15.116:41148) 2024-09-12T03:06:41.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.126+0000 7f02893f8640 1 -- 172.21.15.116:0/2495454618 learned_addr learned my addr 172.21.15.116:0/2495454618 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:41.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.126+0000 7f0283fff640 1 --2- 172.21.15.116:0/2495454618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5642cff53dc0 0x5642cff70b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:41.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.126+0000 7f0288bf7640 1 --2- 172.21.15.116:0/2495454618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642cff534a0 0x5642cff737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:41.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.126+0000 7f02893f8640 1 -- 172.21.15.116:0/2495454618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5642cff53dc0 msgr2=0x5642cff70b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:41.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.126+0000 7f02893f8640 1 --2- 172.21.15.116:0/2495454618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5642cff53dc0 0x5642cff70b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.126+0000 7f02893f8640 1 -- 172.21.15.116:0/2495454618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642cff534a0 msgr2=0x5642cff737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:41.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.126+0000 7f02893f8640 1 --2- 172.21.15.116:0/2495454618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642cff534a0 0x5642cff737f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.126+0000 7f02893f8640 1 -- 172.21.15.116:0/2495454618 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642cff71910 con 0x5642cff5e680 2024-09-12T03:06:41.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.126+0000 7f02893f8640 1 --2- 172.21.15.116:0/2495454618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642cff5e680 0x5642cff710e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f027c09bc90 tx=0x7f027c09a4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39554fc43a7b5a69 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:41.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.126+0000 7f0263fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:06:41.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.126+0000 7f0263fff640 1 -- 172.21.15.116:0/2495454618 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c09c690 con 0x5642cff5e680 2024-09-12T03:06:41.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.126+0000 7f0263fff640 1 -- 172.21.15.116:0/2495454618 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f027c0a4770 con 0x5642cff5e680 2024-09-12T03:06:41.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.126+0000 7f028d234740 1 -- 172.21.15.116:0/2495454618 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0270027070 con 0x5642cff5e680 2024-09-12T03:06:41.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.126+0000 7f0263fff640 1 -- 172.21.15.116:0/2495454618 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f027c0a4c00 con 0x5642cff5e680 2024-09-12T03:06:41.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.127+0000 7f028d234740 10 client.16254 fetch_fsmap learned FSMap version 25 2024-09-12T03:06:41.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.127+0000 7f028d234740 1 -- 172.21.15.116:0/2495454618 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5642cff71910 con 0x5642cff5e680 2024-09-12T03:06:41.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.127+0000 7f0263fff640 1 -- 172.21.15.116:0/2495454618 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0270027070 con 0x5642cff5e680 2024-09-12T03:06:41.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.127+0000 7f0263fff640 1 -- 172.21.15.116:0/2495454618 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f027c0bacd0 con 0x5642cff5e680 2024-09-12T03:06:41.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.127+0000 7f028d234740 10 client.16254 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:06:41.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.127+0000 7f028d234740 10 client.16254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:06:41.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.127+0000 7f028d234740 20 client.16254 populate_metadata read hostname 'smithi116' 2024-09-12T03:06:41.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.127+0000 7f028d234740 1 --2- 172.21.15.116:0/2495454618 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5642cffd8ce0 0x5642cffdb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:41.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.127+0000 7f028d234740 4 client.16254 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:06:41.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.127+0000 7f028d234740 1 -- 172.21.15.116:0/2495454618 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5642cfec0d60 con 0x5642cffd8ce0 2024-09-12T03:06:41.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.127+0000 7f0288bf7640 1 --2- 172.21.15.116:0/2495454618 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5642cffd8ce0 0x5642cffdb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:41.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.127+0000 7f0288bf7640 1 --2- 172.21.15.116:0/2495454618 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5642cffd8ce0 0x5642cffdb0d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb47952d240ae97b server_cookie=1f0f88f2ed9250c1 in_seq=0 out_seq=0 2024-09-12T03:06:41.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.127+0000 7f0263fff640 10 client.16254 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:41.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.128+0000 7f0263fff640 1 -- 172.21.15.116:0/2495454618 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5642cfec0d60 con 0x5642cffd8ce0 2024-09-12T03:06:41.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.128+0000 7f0263fff640 10 client.16254 handle_command_reply: tid=0 2024-09-12T03:06:41.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.128+0000 7f028d234740 1 client.16254 shutdown 2024-09-12T03:06:41.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.128+0000 7f028d234740 1 -- 172.21.15.116:0/2495454618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642cff5e680 msgr2=0x5642cff710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:41.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.128+0000 7f028d234740 1 --2- 172.21.15.116:0/2495454618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642cff5e680 0x5642cff710e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f027c09bc90 tx=0x7f027c09a4a0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.128+0000 7f028d234740 1 -- 172.21.15.116:0/2495454618 shutdown_connections 2024-09-12T03:06:41.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.128+0000 7f028d234740 1 --2- 172.21.15.116:0/2495454618 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5642cffd8ce0 0x5642cffdb0d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.128+0000 7f028d234740 1 --2- 172.21.15.116:0/2495454618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642cff5e680 0x5642cff710e0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.128+0000 7f028d234740 1 --2- 172.21.15.116:0/2495454618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5642cff53dc0 0x5642cff70b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.128+0000 7f028d234740 1 --2- 172.21.15.116:0/2495454618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642cff534a0 0x5642cff737f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.128+0000 7f028d234740 1 -- 172.21.15.116:0/2495454618 >> 172.21.15.116:0/2495454618 conn(0x5642cff4fa30 msgr2=0x5642cff56090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:41.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.128+0000 7f0263fff640 0 client.16254 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:41.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.128+0000 7f028d234740 1 -- 172.21.15.116:0/2495454618 shutdown_connections 2024-09-12T03:06:41.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.128+0000 7f028d234740 1 -- 172.21.15.116:0/2495454618 wait complete. 2024-09-12T03:06:41.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.128+0000 7f028d234740 20 client.16254 trim_cache size 0 max 16384 2024-09-12T03:06:41.135 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:06:41.135 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:06:41.135 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": { 2024-09-12T03:06:41.135 INFO:teuthology.orchestra.run.smithi116.stdout: "6ad7a7a0-2dcc-4482-9b31-9d77ead6510e": { 2024-09-12T03:06:41.135 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/", 2024-09-12T03:06:41.135 INFO:teuthology.orchestra.run.smithi116.stdout: "tag": "6ad7a7a0-2dcc-4482-9b31-9d77ead6510e", 2024-09-12T03:06:41.135 INFO:teuthology.orchestra.run.smithi116.stdout: "options": "recursive,force" 2024-09-12T03:06:41.135 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:06:41.135 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:06:41.135 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:06:41.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.129+0000 7f028ae5b640 1 -- 172.21.15.116:0/653395678 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f02740a2c00 msgr2=0x7f02740a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:41.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.129+0000 7f028ae5b640 1 --2- 172.21.15.116:0/653395678 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f02740a2c00 0x7f02740a50c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5642cff50890 tx=0x7f027000a0d0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.129+0000 7f028ae5b640 1 -- 172.21.15.116:0/653395678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0284106f20 msgr2=0x7f0284074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:41.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.129+0000 7f028ae5b640 1 --2- 172.21.15.116:0/653395678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0284106f20 0x7f0284074f70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f027c00cce0 tx=0x7f027c00a620 comp rx=0 tx=0).stop 2024-09-12T03:06:41.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.129+0000 7f028ae5b640 1 -- 172.21.15.116:0/653395678 shutdown_connections 2024-09-12T03:06:41.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.129+0000 7f028ae5b640 1 --2- 172.21.15.116:0/653395678 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f02740a2c00 0x7f02740a50c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.129+0000 7f028ae5b640 1 --2- 172.21.15.116:0/653395678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0284106f20 0x7f0284074f70 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.129+0000 7f028ae5b640 1 --2- 172.21.15.116:0/653395678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02841027a0 0x7f0284074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.129+0000 7f028ae5b640 1 --2- 172.21.15.116:0/653395678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0284101df0 0x7f02840744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:41.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.129+0000 7f028ae5b640 1 -- 172.21.15.116:0/653395678 >> 172.21.15.116:0/653395678 conn(0x7f02840fd8b0 msgr2=0x7f02840fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:41.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.129+0000 7f028ae5b640 1 -- 172.21.15.116:0/653395678 shutdown_connections 2024-09-12T03:06:41.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:41.129+0000 7f028ae5b640 1 -- 172.21.15.116:0/653395678 wait complete. 2024-09-12T03:06:41.947 INFO:tasks.cephfs.filesystem:scrub status for tag:6ad7a7a0-2dcc-4482-9b31-9d77ead6510e - {'path': '/', 'tag': '6ad7a7a0-2dcc-4482-9b31-9d77ead6510e', 'options': 'recursive,force'} 2024-09-12T03:06:42.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:42 smithi116 ceph-mon[31410]: pgmap v449: 97 pgs: 97 active+clean; 1.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T03:06:42.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:42 smithi183 ceph-mon[35535]: pgmap v449: 97 pgs: 97 active+clean; 1.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T03:06:42.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:42 smithi177 ceph-mon[35305]: pgmap v449: 97 pgs: 97 active+clean; 1.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T03:06:44.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:44 smithi116 ceph-mon[31410]: pgmap v450: 97 pgs: 97 active+clean; 1.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:06:44.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:44 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:06:44.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:44 smithi183 ceph-mon[35535]: pgmap v450: 97 pgs: 97 active+clean; 1.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:06:44.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:44 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:06:44.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:44 smithi177 ceph-mon[35305]: pgmap v450: 97 pgs: 97 active+clean; 1.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:06:44.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:44 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:06:46.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:46 smithi183 ceph-mon[35535]: pgmap v451: 97 pgs: 97 active+clean; 1.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:06:46.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:46 smithi177 ceph-mon[35305]: pgmap v451: 97 pgs: 97 active+clean; 1.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:06:46.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:46 smithi116 ceph-mon[31410]: pgmap v451: 97 pgs: 97 active+clean; 1.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:06:48.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:48 smithi183 ceph-mon[35535]: pgmap v452: 97 pgs: 97 active+clean; 1.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:06:48.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:48 smithi177 ceph-mon[35305]: pgmap v452: 97 pgs: 97 active+clean; 1.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:06:48.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:48 smithi116 ceph-mon[31410]: pgmap v452: 97 pgs: 97 active+clean; 1.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:06:50.345 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.3) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T03:06:50.345 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.15 2024-09-12T03:06:50.346 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:06:50.609 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:06:50.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:50 smithi183 ceph-mon[35535]: pgmap v453: 97 pgs: 97 active+clean; 1.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:06:50.730 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:50 smithi116 ceph-mon[31410]: pgmap v453: 97 pgs: 97 active+clean; 1.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:06:50.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:50 smithi177 ceph-mon[35305]: pgmap v453: 97 pgs: 97 active+clean; 1.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:06:51.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.112+0000 7f2a06039640 1 -- 172.21.15.116:0/2675252691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a00100a80 msgr2=0x7f2a00100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:51.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.112+0000 7f2a06039640 1 --2- 172.21.15.116:0/2675252691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a00100a80 0x7f2a00100ee0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009930 tx=0x7f29f402a420 comp rx=0 tx=0).stop 2024-09-12T03:06:51.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.112+0000 7f2a06039640 1 -- 172.21.15.116:0/2675252691 shutdown_connections 2024-09-12T03:06:51.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.112+0000 7f2a06039640 1 --2- 172.21.15.116:0/2675252691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a00101420 0x7f2a0010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:51.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.112+0000 7f2a06039640 1 --2- 172.21.15.116:0/2675252691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a00100a80 0x7f2a00100ee0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:51.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.112+0000 7f2a06039640 1 --2- 172.21.15.116:0/2675252691 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a00106a80 0x7f2a00106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:51.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.112+0000 7f2a06039640 1 -- 172.21.15.116:0/2675252691 >> 172.21.15.116:0/2675252691 conn(0x7f2a000fc7a0 msgr2=0x7f2a000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:51.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.112+0000 7f2a06039640 1 -- 172.21.15.116:0/2675252691 shutdown_connections 2024-09-12T03:06:51.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.112+0000 7f2a06039640 1 -- 172.21.15.116:0/2675252691 wait complete. 2024-09-12T03:06:51.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.112+0000 7f2a06039640 1 Processor -- start 2024-09-12T03:06:51.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.112+0000 7f2a06039640 1 -- start start 2024-09-12T03:06:51.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.113+0000 7f2a06039640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a00100a80 0x7f2a00198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:51.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.113+0000 7f2a06039640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a00101420 0x7f2a001989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:51.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.113+0000 7f2a06039640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a00106a80 0x7f2a0019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:51.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.113+0000 7f2a06039640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2a00110e40 con 0x7f2a00100a80 2024-09-12T03:06:51.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.113+0000 7f2a06039640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2a00110cc0 con 0x7f2a00106a80 2024-09-12T03:06:51.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.113+0000 7f2a06039640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a00110fc0 con 0x7f2a00101420 2024-09-12T03:06:51.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.113+0000 7f29ff7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a00100a80 0x7f2a00198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:51.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.113+0000 7f29ff7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a00100a80 0x7f2a00198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53348/0 (socket says 172.21.15.116:53348) 2024-09-12T03:06:51.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.113+0000 7f29ff7fe640 1 -- 172.21.15.116:0/1055149114 learned_addr learned my addr 172.21.15.116:0/1055149114 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:51.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.113+0000 7f29fffff640 1 --2- 172.21.15.116:0/1055149114 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a00106a80 0x7f2a0019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:51.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.113+0000 7f29feffd640 1 --2- 172.21.15.116:0/1055149114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a00101420 0x7f2a001989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:51.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.113+0000 7f29ff7fe640 1 -- 172.21.15.116:0/1055149114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a00101420 msgr2=0x7f2a001989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:51.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.113+0000 7f29ff7fe640 1 --2- 172.21.15.116:0/1055149114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a00101420 0x7f2a001989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:51.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.113+0000 7f29ff7fe640 1 -- 172.21.15.116:0/1055149114 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a00106a80 msgr2=0x7f2a0019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:51.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.113+0000 7f29ff7fe640 1 --2- 172.21.15.116:0/1055149114 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a00106a80 0x7f2a0019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:51.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.113+0000 7f29ff7fe640 1 -- 172.21.15.116:0/1055149114 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0019d410 con 0x7f2a00100a80 2024-09-12T03:06:51.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.113+0000 7f29ff7fe640 1 --2- 172.21.15.116:0/1055149114 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a00100a80 0x7f2a00198460 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f29f000eab0 tx=0x7f29f000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:51.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.113+0000 7f29fcff9640 1 -- 172.21.15.116:0/1055149114 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f000ce40 con 0x7f2a00100a80 2024-09-12T03:06:51.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.113+0000 7f2a06039640 1 -- 172.21.15.116:0/1055149114 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a0019d700 con 0x7f2a00100a80 2024-09-12T03:06:51.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.113+0000 7f29fcff9640 1 -- 172.21.15.116:0/1055149114 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29f0011950 con 0x7f2a00100a80 2024-09-12T03:06:51.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.113+0000 7f29fcff9640 1 -- 172.21.15.116:0/1055149114 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f0010ae0 con 0x7f2a00100a80 2024-09-12T03:06:51.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.113+0000 7f2a06039640 1 -- 172.21.15.116:0/1055149114 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a001a4fe0 con 0x7f2a00100a80 2024-09-12T03:06:51.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.114+0000 7f2a06039640 1 -- 172.21.15.116:0/1055149114 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a00100ee0 con 0x7f2a00100a80 2024-09-12T03:06:51.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.116+0000 7f29fcff9640 1 -- 172.21.15.116:0/1055149114 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f29f0011b00 con 0x7f2a00100a80 2024-09-12T03:06:51.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.116+0000 7f29fcff9640 1 --2- 172.21.15.116:0/1055149114 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f29e80a2b80 0x7f29e80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:51.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.116+0000 7f29feffd640 1 --2- 172.21.15.116:0/1055149114 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f29e80a2b80 0x7f29e80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:51.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.116+0000 7f29fcff9640 1 -- 172.21.15.116:0/1055149114 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f29f00cf560 con 0x7f2a00100a80 2024-09-12T03:06:51.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.117+0000 7f29feffd640 1 --2- 172.21.15.116:0/1055149114 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f29e80a2b80 0x7f29e80a5040 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009900 tx=0x7f29f40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:51.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.120+0000 7f29fcff9640 1 -- 172.21.15.116:0/1055149114 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f29f0016030 con 0x7f2a00100a80 2024-09-12T03:06:51.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.324+0000 7f2a06039640 1 -- 172.21.15.116:0/1055149114 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a001997c0 con 0x7f2a00100a80 2024-09-12T03:06:51.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.324+0000 7f29fcff9640 1 -- 172.21.15.116:0/1055149114 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f2a001997c0 con 0x7f2a00100a80 2024-09-12T03:06:51.325 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:06:51.325 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:06:51.325 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:06:51.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.326+0000 7f2a06039640 1 -- 172.21.15.116:0/1055149114 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f29e80a2b80 msgr2=0x7f29e80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:51.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.326+0000 7f2a06039640 1 --2- 172.21.15.116:0/1055149114 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f29e80a2b80 0x7f29e80a5040 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009900 tx=0x7f29f40058d0 comp rx=0 tx=0).stop 2024-09-12T03:06:51.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.327+0000 7f2a06039640 1 -- 172.21.15.116:0/1055149114 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a00100a80 msgr2=0x7f2a00198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:51.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.327+0000 7f2a06039640 1 --2- 172.21.15.116:0/1055149114 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a00100a80 0x7f2a00198460 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f29f000eab0 tx=0x7f29f000ef70 comp rx=0 tx=0).stop 2024-09-12T03:06:51.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.327+0000 7f2a06039640 1 -- 172.21.15.116:0/1055149114 shutdown_connections 2024-09-12T03:06:51.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.327+0000 7f2a06039640 1 --2- 172.21.15.116:0/1055149114 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f29e80a2b80 0x7f29e80a5040 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:51.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.327+0000 7f2a06039640 1 --2- 172.21.15.116:0/1055149114 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a00106a80 0x7f2a0019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:51.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.327+0000 7f2a06039640 1 --2- 172.21.15.116:0/1055149114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a00101420 0x7f2a001989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:51.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.327+0000 7f2a06039640 1 --2- 172.21.15.116:0/1055149114 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a00100a80 0x7f2a00198460 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:51.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.327+0000 7f2a06039640 1 -- 172.21.15.116:0/1055149114 >> 172.21.15.116:0/1055149114 conn(0x7f2a000fc7a0 msgr2=0x7f2a00104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:51.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.327+0000 7f2a06039640 1 -- 172.21.15.116:0/1055149114 shutdown_connections 2024-09-12T03:06:51.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:51.327+0000 7f2a06039640 1 -- 172.21.15.116:0/1055149114 wait complete. 2024-09-12T03:06:52.149 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0' 2024-09-12T03:06:52.149 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0 2024-09-12T03:06:52.401 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:06:52.521 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:52 smithi116 ceph-mon[31410]: pgmap v454: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T03:06:52.522 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:52 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1055149114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:06:52.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:52 smithi183 ceph-mon[35535]: pgmap v454: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T03:06:52.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:52 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1055149114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:06:52.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:52 smithi177 ceph-mon[35305]: pgmap v454: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T03:06:52.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:52 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1055149114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:06:52.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.927+0000 7f5b09e6a640 1 -- 172.21.15.116:0/2255982163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b04100e20 msgr2=0x7f5b04101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:52.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.927+0000 7f5b09e6a640 1 --2- 172.21.15.116:0/2255982163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b04100e20 0x7f5b04101200 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5af4009ca0 tx=0x7f5af402a950 comp rx=0 tx=0).stop 2024-09-12T03:06:52.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.928+0000 7f5b09e6a640 1 -- 172.21.15.116:0/2255982163 shutdown_connections 2024-09-12T03:06:52.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.928+0000 7f5b09e6a640 1 --2- 172.21.15.116:0/2255982163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b04102ef0 0x7f5b0410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.928+0000 7f5b09e6a640 1 --2- 172.21.15.116:0/2255982163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b04102570 0x7f5b041029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.928+0000 7f5b09e6a640 1 --2- 172.21.15.116:0/2255982163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b04100e20 0x7f5b04101200 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.928+0000 7f5b09e6a640 1 -- 172.21.15.116:0/2255982163 >> 172.21.15.116:0/2255982163 conn(0x7f5b040fc7d0 msgr2=0x7f5b040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:52.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.928+0000 7f5b09e6a640 1 -- 172.21.15.116:0/2255982163 shutdown_connections 2024-09-12T03:06:52.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.928+0000 7f5b09e6a640 1 -- 172.21.15.116:0/2255982163 wait complete. 2024-09-12T03:06:52.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.928+0000 7f5b09e6a640 1 Processor -- start 2024-09-12T03:06:52.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.928+0000 7f5b09e6a640 1 -- start start 2024-09-12T03:06:52.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.929+0000 7f5b09e6a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b04100e20 0x7f5b040fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:52.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.929+0000 7f5b09e6a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b04102570 0x7f5b040fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:52.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.929+0000 7f5b09e6a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b04102ef0 0x7f5b040fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:52.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.929+0000 7f5b09e6a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5b04111020 con 0x7f5b04102570 2024-09-12T03:06:52.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.929+0000 7f5b09e6a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5b04110ea0 con 0x7f5b04100e20 2024-09-12T03:06:52.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.929+0000 7f5b09e6a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5b041111a0 con 0x7f5b04102ef0 2024-09-12T03:06:52.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.929+0000 7f5b02ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b04102570 0x7f5b040fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:52.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.929+0000 7f5b02ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b04102570 0x7f5b040fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53370/0 (socket says 172.21.15.116:53370) 2024-09-12T03:06:52.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.929+0000 7f5b02ffd640 1 -- 172.21.15.116:0/3449354888 learned_addr learned my addr 172.21.15.116:0/3449354888 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:52.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.929+0000 7f5b037fe640 1 --2- 172.21.15.116:0/3449354888 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b04100e20 0x7f5b040fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:52.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.929+0000 7f5b03fff640 1 --2- 172.21.15.116:0/3449354888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b04102ef0 0x7f5b040fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:52.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.929+0000 7f5b02ffd640 1 -- 172.21.15.116:0/3449354888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b04102ef0 msgr2=0x7f5b040fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:52.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.929+0000 7f5b02ffd640 1 --2- 172.21.15.116:0/3449354888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b04102ef0 0x7f5b040fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.929+0000 7f5b02ffd640 1 -- 172.21.15.116:0/3449354888 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b04100e20 msgr2=0x7f5b040fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:52.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.929+0000 7f5b02ffd640 1 --2- 172.21.15.116:0/3449354888 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b04100e20 0x7f5b040fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.929+0000 7f5b02ffd640 1 -- 172.21.15.116:0/3449354888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b041a6b30 con 0x7f5b04102570 2024-09-12T03:06:52.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.929+0000 7f5b03fff640 1 --2- 172.21.15.116:0/3449354888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b04102ef0 0x7f5b040fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:06:52.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.929+0000 7f5b037fe640 1 --2- 172.21.15.116:0/3449354888 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b04100e20 0x7f5b040fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:06:52.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.929+0000 7f5b02ffd640 1 --2- 172.21.15.116:0/3449354888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b04102570 0x7f5b040fd1c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5af800b4a0 tx=0x7f5af800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:52.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.929+0000 7f5b00ff9640 1 -- 172.21.15.116:0/3449354888 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af8014070 con 0x7f5b04102570 2024-09-12T03:06:52.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.929+0000 7f5b09e6a640 1 -- 172.21.15.116:0/3449354888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b041a6e20 con 0x7f5b04102570 2024-09-12T03:06:52.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.929+0000 7f5b00ff9640 1 -- 172.21.15.116:0/3449354888 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5af8010af0 con 0x7f5b04102570 2024-09-12T03:06:52.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.929+0000 7f5b00ff9640 1 -- 172.21.15.116:0/3449354888 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af8017d10 con 0x7f5b04102570 2024-09-12T03:06:52.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.929+0000 7f5b09e6a640 1 -- 172.21.15.116:0/3449354888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b041a7320 con 0x7f5b04102570 2024-09-12T03:06:52.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.931+0000 7f5b00ff9640 1 -- 172.21.15.116:0/3449354888 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f5af8017070 con 0x7f5b04102570 2024-09-12T03:06:52.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.932+0000 7f5b0c243740 1 Processor -- start 2024-09-12T03:06:52.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.932+0000 7f5b0c243740 1 -- start start 2024-09-12T03:06:52.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.932+0000 7f5b0c243740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582aa739850 0x5582aa739c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:52.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.932+0000 7f5b0c243740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582aa73a170 0x5582aa7444f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:52.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.932+0000 7f5b0c243740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582aa744a30 0x5582aa746e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:52.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.932+0000 7f5b0c243740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5582aa69e160 con 0x5582aa744a30 2024-09-12T03:06:52.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.932+0000 7f5b0c243740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5582aa645720 con 0x5582aa73a170 2024-09-12T03:06:52.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.932+0000 7f5b0c243740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5582aa69a120 con 0x5582aa739850 2024-09-12T03:06:52.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.932+0000 7f5b02ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582aa744a30 0x5582aa746e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:52.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.932+0000 7f5b03fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582aa739850 0x5582aa739c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:52.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.932+0000 7f5b02ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582aa744a30 0x5582aa746e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53382/0 (socket says 172.21.15.116:53382) 2024-09-12T03:06:52.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.932+0000 7f5b02ffd640 1 -- 172.21.15.116:0/3911124896 learned_addr learned my addr 172.21.15.116:0/3911124896 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:52.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.932+0000 7f5b03fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582aa739850 0x5582aa739c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.116:32972/0 (socket says 172.21.15.116:32972) 2024-09-12T03:06:52.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.932+0000 7f5b00ff9640 1 --2- 172.21.15.116:0/3449354888 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5aec0a2b80 0x7f5aec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:52.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.933+0000 7f5b037fe640 1 --2- 172.21.15.116:0/3911124896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582aa73a170 0x5582aa7444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:52.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.933+0000 7f5b03fff640 1 --2- 172.21.15.116:0/3449354888 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5aec0a2b80 0x7f5aec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:52.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.933+0000 7f5b00ff9640 1 -- 172.21.15.116:0/3449354888 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f5af80ce480 con 0x7f5b04102570 2024-09-12T03:06:52.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.933+0000 7f5b02ffd640 1 -- 172.21.15.116:0/3911124896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582aa739850 msgr2=0x5582aa739c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:52.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.933+0000 7f5b02ffd640 1 --2- 172.21.15.116:0/3911124896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582aa739850 0x5582aa739c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.933+0000 7f5b02ffd640 1 -- 172.21.15.116:0/3911124896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582aa73a170 msgr2=0x5582aa7444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:52.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.933+0000 7f5b02ffd640 1 --2- 172.21.15.116:0/3911124896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582aa73a170 0x5582aa7444f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.933+0000 7f5b02ffd640 1 -- 172.21.15.116:0/3911124896 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582aa40fba0 con 0x5582aa744a30 2024-09-12T03:06:52.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.933+0000 7f5b02ffd640 1 --2- 172.21.15.116:0/3911124896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582aa744a30 0x5582aa746e20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5af80cbf50 tx=0x7f5af80047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab06c40de64da68 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:52.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.933+0000 7f5b03fff640 1 --2- 172.21.15.116:0/3449354888 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5aec0a2b80 0x7f5aec0a5040 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5afc015600 tx=0x7f5afc0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:52.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.933+0000 7f5ae9ffb640 1 -- 172.21.15.116:0/3911124896 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af801e030 con 0x5582aa744a30 2024-09-12T03:06:52.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.933+0000 7f5ae9ffb640 1 -- 172.21.15.116:0/3911124896 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5af809acb0 con 0x5582aa744a30 2024-09-12T03:06:52.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.933+0000 7f5ae9ffb640 1 -- 172.21.15.116:0/3911124896 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af8010ca0 con 0x5582aa744a30 2024-09-12T03:06:52.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.934+0000 7f5b0c243740 1 -- 172.21.15.116:0/3911124896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582aa744a30 msgr2=0x5582aa746e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:52.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.934+0000 7f5b0c243740 1 --2- 172.21.15.116:0/3911124896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582aa744a30 0x5582aa746e20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5af80cbf50 tx=0x7f5af80047d0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.934+0000 7f5b0c243740 1 -- 172.21.15.116:0/3911124896 shutdown_connections 2024-09-12T03:06:52.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.934+0000 7f5b0c243740 1 --2- 172.21.15.116:0/3911124896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582aa744a30 0x5582aa746e20 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.934+0000 7f5b0c243740 1 --2- 172.21.15.116:0/3911124896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582aa73a170 0x5582aa7444f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.934+0000 7f5b0c243740 1 --2- 172.21.15.116:0/3911124896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582aa739850 0x5582aa739c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.934+0000 7f5b0c243740 1 -- 172.21.15.116:0/3911124896 >> 172.21.15.116:0/3911124896 conn(0x5582aa735de0 msgr2=0x5582aa7361c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:52.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.934+0000 7f5b0c243740 1 -- 172.21.15.116:0/3911124896 shutdown_connections 2024-09-12T03:06:52.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.934+0000 7f5b0c243740 1 -- 172.21.15.116:0/3911124896 wait complete. 2024-09-12T03:06:52.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.934+0000 7f5b0c243740 1 -- start start 2024-09-12T03:06:52.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.934+0000 7f5b0c243740 1 Processor -- start 2024-09-12T03:06:52.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.935+0000 7f5b0c243740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582aa739850 0x5582aa741890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:52.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.935+0000 7f5b0c243740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582aa73a170 0x5582aa741dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:52.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.935+0000 7f5b0c243740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582aa744a30 0x5582aa742310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:52.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.935+0000 7f5b0c243740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5582aa747600 con 0x5582aa744a30 2024-09-12T03:06:52.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.935+0000 7f5b0c243740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5582aa626900 con 0x5582aa739850 2024-09-12T03:06:52.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.935+0000 7f5b0c243740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5582aa747780 con 0x5582aa73a170 2024-09-12T03:06:52.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.935+0000 7f5b0c243740 10 client.0 _ll_register_callbacks cb 0x5582aa624780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:52.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.935+0000 7f5b03fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582aa744a30 0x5582aa742310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:52.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.935+0000 7f5b03fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582aa744a30 0x5582aa742310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53388/0 (socket says 172.21.15.116:53388) 2024-09-12T03:06:52.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.935+0000 7f5b03fff640 1 -- 172.21.15.116:0/2199815882 learned_addr learned my addr 172.21.15.116:0/2199815882 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:52.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.935+0000 7f5b02ffd640 1 --2- 172.21.15.116:0/2199815882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582aa73a170 0x5582aa741dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:52.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.935+0000 7f5b03fff640 1 -- 172.21.15.116:0/2199815882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582aa73a170 msgr2=0x5582aa741dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:52.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.935+0000 7f5b03fff640 1 --2- 172.21.15.116:0/2199815882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582aa73a170 0x5582aa741dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.935+0000 7f5b037fe640 1 --2- 172.21.15.116:0/2199815882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582aa739850 0x5582aa741890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:52.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.935+0000 7f5b03fff640 1 -- 172.21.15.116:0/2199815882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582aa739850 msgr2=0x5582aa741890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:52.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.935+0000 7f5b03fff640 1 --2- 172.21.15.116:0/2199815882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582aa739850 0x5582aa741890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.935+0000 7f5b03fff640 1 -- 172.21.15.116:0/2199815882 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582aa666c50 con 0x5582aa744a30 2024-09-12T03:06:52.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.935+0000 7f5b03fff640 1 --2- 172.21.15.116:0/2199815882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582aa744a30 0x5582aa742310 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5afc017b40 tx=0x7f5afc0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd5a7c4209cccfa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:52.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.935+0000 7f5ae2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:06:52.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.935+0000 7f5ae2ffd640 1 -- 172.21.15.116:0/2199815882 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc027060 con 0x5582aa744a30 2024-09-12T03:06:52.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.935+0000 7f5ae2ffd640 1 -- 172.21.15.116:0/2199815882 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f5afc016d00 con 0x5582aa744a30 2024-09-12T03:06:52.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.935+0000 7f5b0c243740 1 -- 172.21.15.116:0/2199815882 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5af809acb0 con 0x5582aa744a30 2024-09-12T03:06:52.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.936+0000 7f5ae2ffd640 1 -- 172.21.15.116:0/2199815882 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5afc00f040 con 0x5582aa744a30 2024-09-12T03:06:52.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.936+0000 7f5b0c243740 10 client.16272 fetch_fsmap learned FSMap version 25 2024-09-12T03:06:52.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.936+0000 7f5b0c243740 1 -- 172.21.15.116:0/2199815882 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5582aa753290 con 0x5582aa744a30 2024-09-12T03:06:52.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.936+0000 7f5ae2ffd640 1 -- 172.21.15.116:0/2199815882 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5af809acb0 con 0x5582aa744a30 2024-09-12T03:06:52.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.936+0000 7f5ae2ffd640 1 -- 172.21.15.116:0/2199815882 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5afc02d3b0 con 0x5582aa744a30 2024-09-12T03:06:52.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.936+0000 7f5b0c243740 10 client.16272 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:06:52.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.936+0000 7f5b0c243740 10 client.16272 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:06:52.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.936+0000 7f5b0c243740 20 client.16272 populate_metadata read hostname 'smithi116' 2024-09-12T03:06:52.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.936+0000 7f5b0c243740 1 --2- 172.21.15.116:0/2199815882 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5582aa754750 0x5582aa756bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:52.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.936+0000 7f5b0c243740 4 client.16272 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:06:52.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.936+0000 7f5b0c243740 1 -- 172.21.15.116:0/2199815882 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582aa6ab890 con 0x5582aa754750 2024-09-12T03:06:52.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.936+0000 7f5b037fe640 1 --2- 172.21.15.116:0/2199815882 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5582aa754750 0x5582aa756bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:52.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.937+0000 7f5ae2ffd640 10 client.16272 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:52.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.937+0000 7f5b037fe640 1 --2- 172.21.15.116:0/2199815882 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5582aa754750 0x5582aa756bf0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71128dfeae08034b server_cookie=95f16d5de2c31c85 in_seq=0 out_seq=0 2024-09-12T03:06:52.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.937+0000 7f5ae2ffd640 1 -- 172.21.15.116:0/2199815882 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5582aa6ab890 con 0x5582aa754750 2024-09-12T03:06:52.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.937+0000 7f5ae2ffd640 10 client.16272 handle_command_reply: tid=0 2024-09-12T03:06:52.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.938+0000 7f5b0c243740 1 client.16272 shutdown 2024-09-12T03:06:52.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.938+0000 7f5b0c243740 1 -- 172.21.15.116:0/2199815882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582aa744a30 msgr2=0x5582aa742310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:52.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.938+0000 7f5b0c243740 1 --2- 172.21.15.116:0/2199815882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582aa744a30 0x5582aa742310 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5afc017b40 tx=0x7f5afc0047d0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.938+0000 7f5b0c243740 1 -- 172.21.15.116:0/2199815882 shutdown_connections 2024-09-12T03:06:52.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.938+0000 7f5b0c243740 1 --2- 172.21.15.116:0/2199815882 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5582aa754750 0x5582aa756bf0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.938+0000 7f5b0c243740 1 --2- 172.21.15.116:0/2199815882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582aa744a30 0x5582aa742310 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.938+0000 7f5b0c243740 1 --2- 172.21.15.116:0/2199815882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582aa73a170 0x5582aa741dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.938+0000 7f5ae2ffd640 0 client.16272 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:52.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.938+0000 7f5b0c243740 1 --2- 172.21.15.116:0/2199815882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582aa739850 0x5582aa741890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.938+0000 7f5b0c243740 1 -- 172.21.15.116:0/2199815882 >> 172.21.15.116:0/2199815882 conn(0x5582aa735de0 msgr2=0x5582aa73bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:52.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.938+0000 7f5b0c243740 1 -- 172.21.15.116:0/2199815882 shutdown_connections 2024-09-12T03:06:52.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.938+0000 7f5b0c243740 1 -- 172.21.15.116:0/2199815882 wait complete. 2024-09-12T03:06:52.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.938+0000 7f5b0c243740 20 client.16272 trim_cache size 0 max 16384 2024-09-12T03:06:52.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.955+0000 7f5b0c243740 1 Processor -- start 2024-09-12T03:06:52.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.955+0000 7f5b0c243740 1 -- start start 2024-09-12T03:06:52.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.955+0000 7f5b0c243740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582aa739850 0x5582aa743760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:52.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.955+0000 7f5b0c243740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582aa73a170 0x5582aa740160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:52.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.955+0000 7f5b0c243740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582aa744a30 0x5582aa75c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:52.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.955+0000 7f5b0c243740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5b041b2190 con 0x5582aa73a170 2024-09-12T03:06:52.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.955+0000 7f5b0c243740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5b041afa60 con 0x5582aa739850 2024-09-12T03:06:52.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.955+0000 7f5b0c243740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5b041ad8d0 con 0x5582aa744a30 2024-09-12T03:06:52.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.955+0000 7f5b02ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582aa73a170 0x5582aa740160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:52.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.955+0000 7f5b02ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582aa73a170 0x5582aa740160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53396/0 (socket says 172.21.15.116:53396) 2024-09-12T03:06:52.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.955+0000 7f5b02ffd640 1 -- 172.21.15.116:0/4244936620 learned_addr learned my addr 172.21.15.116:0/4244936620 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:52.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.955+0000 7f5b02ffd640 1 -- 172.21.15.116:0/4244936620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582aa744a30 msgr2=0x5582aa75c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:52.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.955+0000 7f5b02ffd640 1 --2- 172.21.15.116:0/4244936620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582aa744a30 0x5582aa75c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.955+0000 7f5b037fe640 1 --2- 172.21.15.116:0/4244936620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582aa739850 0x5582aa743760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:52.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.955+0000 7f5b02ffd640 1 -- 172.21.15.116:0/4244936620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582aa739850 msgr2=0x5582aa743760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:52.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.955+0000 7f5b02ffd640 1 --2- 172.21.15.116:0/4244936620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582aa739850 0x5582aa743760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.955+0000 7f5b02ffd640 1 -- 172.21.15.116:0/4244936620 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582aa735c10 con 0x5582aa73a170 2024-09-12T03:06:52.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.955+0000 7f5b037fe640 1 --2- 172.21.15.116:0/4244936620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582aa739850 0x5582aa743760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:06:52.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.955+0000 7f5b02ffd640 1 --2- 172.21.15.116:0/4244936620 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582aa73a170 0x5582aa740160 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5af8009420 tx=0x7f5af8017730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe4845ddfcdbece server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:52.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.955+0000 7f5ae97fa640 1 -- 172.21.15.116:0/4244936620 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af801e040 con 0x5582aa73a170 2024-09-12T03:06:52.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.956+0000 7f5ae97fa640 1 -- 172.21.15.116:0/4244936620 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5af8098fb0 con 0x5582aa73a170 2024-09-12T03:06:52.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.956+0000 7f5ae97fa640 1 -- 172.21.15.116:0/4244936620 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af80ce110 con 0x5582aa73a170 2024-09-12T03:06:52.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.956+0000 7f5b0c243740 1 -- 172.21.15.116:0/4244936620 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582aa73a170 msgr2=0x5582aa740160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:52.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.956+0000 7f5b0c243740 1 --2- 172.21.15.116:0/4244936620 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582aa73a170 0x5582aa740160 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5af8009420 tx=0x7f5af8017730 comp rx=0 tx=0).stop 2024-09-12T03:06:52.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.956+0000 7f5b0c243740 1 -- 172.21.15.116:0/4244936620 shutdown_connections 2024-09-12T03:06:52.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.956+0000 7f5b0c243740 1 --2- 172.21.15.116:0/4244936620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582aa744a30 0x5582aa75c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.956+0000 7f5b0c243740 1 --2- 172.21.15.116:0/4244936620 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582aa73a170 0x5582aa740160 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.956+0000 7f5b0c243740 1 --2- 172.21.15.116:0/4244936620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582aa739850 0x5582aa743760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.956+0000 7f5b0c243740 1 -- 172.21.15.116:0/4244936620 >> 172.21.15.116:0/4244936620 conn(0x5582aa735de0 msgr2=0x5582aa734ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:52.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.956+0000 7f5b0c243740 1 -- 172.21.15.116:0/4244936620 shutdown_connections 2024-09-12T03:06:52.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.956+0000 7f5b0c243740 1 -- 172.21.15.116:0/4244936620 wait complete. 2024-09-12T03:06:52.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.956+0000 7f5b0c243740 1 -- start start 2024-09-12T03:06:52.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.957+0000 7f5b0c243740 1 Processor -- start 2024-09-12T03:06:52.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.957+0000 7f5b0c243740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582aa739850 0x5582aa749a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:52.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.957+0000 7f5b0c243740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582aa73a170 0x5582aa749f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:52.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.957+0000 7f5b0c243740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582aa744a30 0x5582aa74a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:52.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.957+0000 7f5b0c243740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5b041adf20 con 0x5582aa744a30 2024-09-12T03:06:52.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.957+0000 7f5b0c243740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5b041af390 con 0x5582aa73a170 2024-09-12T03:06:52.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.957+0000 7f5b0c243740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5b041ae5f0 con 0x5582aa739850 2024-09-12T03:06:52.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.957+0000 7f5b0c243740 10 client.0 _ll_register_callbacks cb 0x7f5afc015f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:52.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.957+0000 7f5b03fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582aa744a30 0x5582aa74a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:52.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.957+0000 7f5b03fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582aa744a30 0x5582aa74a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53412/0 (socket says 172.21.15.116:53412) 2024-09-12T03:06:52.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.957+0000 7f5b03fff640 1 -- 172.21.15.116:0/920947182 learned_addr learned my addr 172.21.15.116:0/920947182 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:06:52.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.957+0000 7f5b037fe640 1 --2- 172.21.15.116:0/920947182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582aa739850 0x5582aa749a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:52.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.957+0000 7f5b02ffd640 1 --2- 172.21.15.116:0/920947182 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582aa73a170 0x5582aa749f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:52.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.957+0000 7f5b03fff640 1 -- 172.21.15.116:0/920947182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582aa739850 msgr2=0x5582aa749a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:52.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.957+0000 7f5b03fff640 1 --2- 172.21.15.116:0/920947182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582aa739850 0x5582aa749a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.957+0000 7f5b03fff640 1 -- 172.21.15.116:0/920947182 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582aa73a170 msgr2=0x5582aa749f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:52.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.957+0000 7f5b03fff640 1 --2- 172.21.15.116:0/920947182 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582aa73a170 0x5582aa749f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.957+0000 7f5b03fff640 1 -- 172.21.15.116:0/920947182 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582aa7313f0 con 0x5582aa744a30 2024-09-12T03:06:52.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.957+0000 7f5b02ffd640 1 --2- 172.21.15.116:0/920947182 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582aa73a170 0x5582aa749f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:06:52.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.957+0000 7f5b037fe640 1 --2- 172.21.15.116:0/920947182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582aa739850 0x5582aa749a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:06:52.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.958+0000 7f5b03fff640 1 --2- 172.21.15.116:0/920947182 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582aa744a30 0x5582aa74a480 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5afc015f30 tx=0x7f5afc00af60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458432463df1a823 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:52.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.958+0000 7f5ae2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:06:52.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.958+0000 7f5ae2ffd640 1 -- 172.21.15.116:0/920947182 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc032020 con 0x5582aa744a30 2024-09-12T03:06:52.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.958+0000 7f5ae2ffd640 1 -- 172.21.15.116:0/920947182 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f5afc02d3b0 con 0x5582aa744a30 2024-09-12T03:06:52.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.958+0000 7f5b0c243740 1 -- 172.21.15.116:0/920947182 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5af8098fb0 con 0x5582aa744a30 2024-09-12T03:06:52.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.958+0000 7f5ae2ffd640 1 -- 172.21.15.116:0/920947182 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5afc027070 con 0x5582aa744a30 2024-09-12T03:06:52.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.958+0000 7f5b0c243740 10 client.16278 fetch_fsmap learned FSMap version 25 2024-09-12T03:06:52.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.958+0000 7f5b0c243740 1 -- 172.21.15.116:0/920947182 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5582aa7c0470 con 0x5582aa744a30 2024-09-12T03:06:52.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.958+0000 7f5ae2ffd640 1 -- 172.21.15.116:0/920947182 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5af8098fb0 con 0x5582aa744a30 2024-09-12T03:06:52.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.958+0000 7f5ae2ffd640 1 -- 172.21.15.116:0/920947182 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5afc02e730 con 0x5582aa744a30 2024-09-12T03:06:52.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.958+0000 7f5b0c243740 10 client.16278 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:06:52.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.958+0000 7f5b0c243740 10 client.16278 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:06:52.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.958+0000 7f5b0c243740 20 client.16278 populate_metadata read hostname 'smithi116' 2024-09-12T03:06:52.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.958+0000 7f5b0c243740 1 --2- 172.21.15.116:0/920947182 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5582aa7c0ba0 0x5582aa7c2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:52.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.958+0000 7f5b0c243740 4 client.16278 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}] 2024-09-12T03:06:52.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.958+0000 7f5b0c243740 1 -- 172.21.15.116:0/920947182 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}) -- 0x5582aa6a5be0 con 0x5582aa7c0ba0 2024-09-12T03:06:52.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.959+0000 7f5b037fe640 1 --2- 172.21.15.116:0/920947182 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5582aa7c0ba0 0x5582aa7c2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:52.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.959+0000 7f5ae2ffd640 10 client.16278 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:52.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.959+0000 7f5b037fe640 1 --2- 172.21.15.116:0/920947182 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5582aa7c0ba0 0x5582aa7c2f90 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d5d9a27e2f16756 server_cookie=9f563d7db067af68 in_seq=0 out_seq=0 2024-09-12T03:06:52.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.968+0000 7f5ae2ffd640 1 -- 172.21.15.116:0/920947182 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5582aa6a5be0 con 0x5582aa7c0ba0 2024-09-12T03:06:52.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.968+0000 7f5ae2ffd640 10 client.16278 handle_command_reply: tid=0 2024-09-12T03:06:52.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.968+0000 7f5b0c243740 1 client.16278 shutdown 2024-09-12T03:06:52.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.968+0000 7f5b0c243740 1 -- 172.21.15.116:0/920947182 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582aa744a30 msgr2=0x5582aa74a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:52.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.968+0000 7f5b0c243740 1 --2- 172.21.15.116:0/920947182 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582aa744a30 0x5582aa74a480 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5afc015f30 tx=0x7f5afc00af60 comp rx=0 tx=0).stop 2024-09-12T03:06:52.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.968+0000 7f5b0c243740 1 -- 172.21.15.116:0/920947182 shutdown_connections 2024-09-12T03:06:52.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.968+0000 7f5b0c243740 1 --2- 172.21.15.116:0/920947182 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5582aa7c0ba0 0x5582aa7c2f90 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.968+0000 7f5b0c243740 1 --2- 172.21.15.116:0/920947182 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582aa744a30 0x5582aa74a480 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.968+0000 7f5b0c243740 1 --2- 172.21.15.116:0/920947182 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582aa73a170 0x5582aa749f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.968+0000 7f5b0c243740 1 --2- 172.21.15.116:0/920947182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582aa739850 0x5582aa749a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.968+0000 7f5b0c243740 1 -- 172.21.15.116:0/920947182 >> 172.21.15.116:0/920947182 conn(0x5582aa735de0 msgr2=0x5582aa734ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:52.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.968+0000 7f5ae2ffd640 0 client.16278 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:06:52.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.968+0000 7f5b0c243740 1 -- 172.21.15.116:0/920947182 shutdown_connections 2024-09-12T03:06:52.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.968+0000 7f5b0c243740 1 -- 172.21.15.116:0/920947182 wait complete. 2024-09-12T03:06:52.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.968+0000 7f5b0c243740 20 client.16278 trim_cache size 0 max 16384 2024-09-12T03:06:52.971 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:06:52.971 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 25, 2024-09-12T03:06:52.971 INFO:teuthology.orchestra.run.smithi116.stdout: "leader": 24494, 2024-09-12T03:06:52.971 INFO:teuthology.orchestra.run.smithi116.stdout: "set_version": 14, 2024-09-12T03:06:52.971 INFO:teuthology.orchestra.run.smithi116.stdout: "sets": { 2024-09-12T03:06:52.971 INFO:teuthology.orchestra.run.smithi116.stdout: "34c5c87e": { 2024-09-12T03:06:52.971 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 14, 2024-09-12T03:06:52.971 INFO:teuthology.orchestra.run.smithi116.stdout: "age_ref": 0.0, 2024-09-12T03:06:52.972 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:06:52.972 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "QUIESCED", 2024-09-12T03:06:52.972 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:06:52.972 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:06:52.972 INFO:teuthology.orchestra.run.smithi116.stdout: "timeout": 90.0, 2024-09-12T03:06:52.972 INFO:teuthology.orchestra.run.smithi116.stdout: "expiration": 140.2, 2024-09-12T03:06:52.972 INFO:teuthology.orchestra.run.smithi116.stdout: "members": { 2024-09-12T03:06:52.972 INFO:teuthology.orchestra.run.smithi116.stdout: "file:/": { 2024-09-12T03:06:52.972 INFO:teuthology.orchestra.run.smithi116.stdout: "excluded": false, 2024-09-12T03:06:52.972 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:06:52.972 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "QUIESCED", 2024-09-12T03:06:52.972 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:06:52.972 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:06:52.973 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:06:52.973 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:06:52.973 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:06:52.973 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:06:52.973 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:06:52.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.969+0000 7f5b09e6a640 1 -- 172.21.15.116:0/3449354888 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5aec0a2b80 msgr2=0x7f5aec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:52.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.969+0000 7f5b09e6a640 1 --2- 172.21.15.116:0/3449354888 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5aec0a2b80 0x7f5aec0a5040 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5afc015600 tx=0x7f5afc0046c0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.969+0000 7f5b09e6a640 1 -- 172.21.15.116:0/3449354888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b04102570 msgr2=0x7f5b040fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:52.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.969+0000 7f5b09e6a640 1 --2- 172.21.15.116:0/3449354888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b04102570 0x7f5b040fd1c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5af800b4a0 tx=0x7f5af800b960 comp rx=0 tx=0).stop 2024-09-12T03:06:52.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.969+0000 7f5b09e6a640 1 -- 172.21.15.116:0/3449354888 shutdown_connections 2024-09-12T03:06:52.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.969+0000 7f5b09e6a640 1 --2- 172.21.15.116:0/3449354888 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5aec0a2b80 0x7f5aec0a5040 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.969+0000 7f5b09e6a640 1 --2- 172.21.15.116:0/3449354888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b04102ef0 0x7f5b040fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.969+0000 7f5b09e6a640 1 --2- 172.21.15.116:0/3449354888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b04102570 0x7f5b040fd1c0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.969+0000 7f5b09e6a640 1 --2- 172.21.15.116:0/3449354888 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b04100e20 0x7f5b040fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:52.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.969+0000 7f5b09e6a640 1 -- 172.21.15.116:0/3449354888 >> 172.21.15.116:0/3449354888 conn(0x7f5b040fc7d0 msgr2=0x7f5b0406a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:52.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.969+0000 7f5b09e6a640 1 -- 172.21.15.116:0/3449354888 shutdown_connections 2024-09-12T03:06:52.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:06:52.969+0000 7f5b09e6a640 1 -- 172.21.15.116:0/3449354888 wait complete. 2024-09-12T03:06:53.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:53 smithi183 ceph-mon[35535]: pgmap v455: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:06:53.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:53 smithi177 ceph-mon[35305]: pgmap v455: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:06:53.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:53 smithi116 ceph-mon[31410]: pgmap v455: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:06:53.900 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '34c5c87e', quiesce took 3.6 seconds. Will release after: 16.549999999999997 2024-09-12T03:06:56.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:56 smithi116 ceph-mon[31410]: pgmap v456: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:06:56.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:56 smithi183 ceph-mon[35535]: pgmap v456: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:06:56.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:56 smithi177 ceph-mon[35305]: pgmap v456: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:06:58.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:06:58 smithi116 ceph-mon[31410]: pgmap v457: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-12T03:06:58.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:06:58 smithi183 ceph-mon[35535]: pgmap v457: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-12T03:06:58.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:06:58 smithi177 ceph-mon[35305]: pgmap v457: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-12T03:07:00.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:00 smithi116 ceph-mon[31410]: pgmap v458: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.9 KiB/s wr, 8 op/s 2024-09-12T03:07:00.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:00 smithi183 ceph-mon[35535]: pgmap v458: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.9 KiB/s wr, 8 op/s 2024-09-12T03:07:00.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:00 smithi177 ceph-mon[35305]: pgmap v458: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.9 KiB/s wr, 8 op/s 2024-09-12T03:07:02.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:02 smithi116 ceph-mon[31410]: pgmap v459: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 KiB/s wr, 11 op/s 2024-09-12T03:07:02.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:02 smithi183 ceph-mon[35535]: pgmap v459: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 KiB/s wr, 11 op/s 2024-09-12T03:07:02.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:02 smithi177 ceph-mon[35305]: pgmap v459: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 KiB/s wr, 11 op/s 2024-09-12T03:07:04.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:04 smithi116 ceph-mon[31410]: pgmap v460: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.4 KiB/s wr, 6 op/s 2024-09-12T03:07:04.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:04 smithi183 ceph-mon[35535]: pgmap v460: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.4 KiB/s wr, 6 op/s 2024-09-12T03:07:04.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:04 smithi177 ceph-mon[35305]: pgmap v460: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.4 KiB/s wr, 6 op/s 2024-09-12T03:07:06.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:06 smithi116 ceph-mon[31410]: pgmap v461: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2024-09-12T03:07:06.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:06 smithi183 ceph-mon[35535]: pgmap v461: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2024-09-12T03:07:06.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:06 smithi177 ceph-mon[35305]: pgmap v461: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2024-09-12T03:07:08.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:08 smithi116 ceph-mon[31410]: pgmap v462: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-09-12T03:07:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:08 smithi183 ceph-mon[35535]: pgmap v462: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-09-12T03:07:08.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:08 smithi177 ceph-mon[35305]: pgmap v462: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-09-12T03:07:10.451 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:07:10.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:10 smithi116 ceph-mon[31410]: pgmap v463: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-12T03:07:10.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:10 smithi183 ceph-mon[35535]: pgmap v463: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-12T03:07:10.713 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:07:10.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:10 smithi177 ceph-mon[35305]: pgmap v463: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-12T03:07:11.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.212+0000 7fcc599e8640 1 -- 172.21.15.116:0/2310336893 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc54100a80 msgr2=0x7fcc54100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:11.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.212+0000 7fcc599e8640 1 --2- 172.21.15.116:0/2310336893 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc54100a80 0x7fcc54100ee0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c009930 tx=0x7fcc3c02a420 comp rx=0 tx=0).stop 2024-09-12T03:07:11.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.212+0000 7fcc599e8640 1 -- 172.21.15.116:0/2310336893 shutdown_connections 2024-09-12T03:07:11.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.212+0000 7fcc599e8640 1 --2- 172.21.15.116:0/2310336893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc54101420 0x7fcc5410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:11.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.212+0000 7fcc599e8640 1 --2- 172.21.15.116:0/2310336893 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc54100a80 0x7fcc54100ee0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:11.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.212+0000 7fcc599e8640 1 --2- 172.21.15.116:0/2310336893 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc54106a80 0x7fcc54106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:11.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.212+0000 7fcc599e8640 1 -- 172.21.15.116:0/2310336893 >> 172.21.15.116:0/2310336893 conn(0x7fcc540fc7a0 msgr2=0x7fcc540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:11.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.212+0000 7fcc599e8640 1 -- 172.21.15.116:0/2310336893 shutdown_connections 2024-09-12T03:07:11.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.212+0000 7fcc599e8640 1 -- 172.21.15.116:0/2310336893 wait complete. 2024-09-12T03:07:11.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.213+0000 7fcc599e8640 1 Processor -- start 2024-09-12T03:07:11.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.213+0000 7fcc599e8640 1 -- start start 2024-09-12T03:07:11.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.213+0000 7fcc599e8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc54100a80 0x7fcc540fcd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:11.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.213+0000 7fcc599e8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc54101420 0x7fcc540fd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:11.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.213+0000 7fcc599e8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc54106a80 0x7fcc540fd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:11.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.213+0000 7fcc599e8640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcc54110ea0 con 0x7fcc54106a80 2024-09-12T03:07:11.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.213+0000 7fcc599e8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcc54110d20 con 0x7fcc54101420 2024-09-12T03:07:11.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.213+0000 7fcc599e8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcc54111020 con 0x7fcc54100a80 2024-09-12T03:07:11.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.213+0000 7fcc537fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc54106a80 0x7fcc540fd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:11.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.213+0000 7fcc537fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc54106a80 0x7fcc540fd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46554/0 (socket says 172.21.15.116:46554) 2024-09-12T03:07:11.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.213+0000 7fcc537fe640 1 -- 172.21.15.116:0/3942484702 learned_addr learned my addr 172.21.15.116:0/3942484702 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:11.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.214+0000 7fcc52ffd640 1 --2- 172.21.15.116:0/3942484702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc54100a80 0x7fcc540fcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:11.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.214+0000 7fcc4a7fc640 1 --2- 172.21.15.116:0/3942484702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc54101420 0x7fcc540fd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:11.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.214+0000 7fcc537fe640 1 -- 172.21.15.116:0/3942484702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc54100a80 msgr2=0x7fcc540fcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:11.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.214+0000 7fcc537fe640 1 --2- 172.21.15.116:0/3942484702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc54100a80 0x7fcc540fcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:11.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.214+0000 7fcc537fe640 1 -- 172.21.15.116:0/3942484702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc54101420 msgr2=0x7fcc540fd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:11.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.214+0000 7fcc537fe640 1 --2- 172.21.15.116:0/3942484702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc54101420 0x7fcc540fd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:11.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.214+0000 7fcc537fe640 1 -- 172.21.15.116:0/3942484702 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc541a6900 con 0x7fcc54106a80 2024-09-12T03:07:11.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.214+0000 7fcc52ffd640 1 --2- 172.21.15.116:0/3942484702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc54100a80 0x7fcc540fcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:07:11.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.214+0000 7fcc537fe640 1 --2- 172.21.15.116:0/3942484702 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc54106a80 0x7fcc540fd7b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c00cce0 tx=0x7fcc4c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:11.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.214+0000 7fcc589e6640 1 -- 172.21.15.116:0/3942484702 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c00f450 con 0x7fcc54106a80 2024-09-12T03:07:11.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.214+0000 7fcc589e6640 1 -- 172.21.15.116:0/3942484702 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc4c00fad0 con 0x7fcc54106a80 2024-09-12T03:07:11.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.214+0000 7fcc599e8640 1 -- 172.21.15.116:0/3942484702 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc541a6bf0 con 0x7fcc54106a80 2024-09-12T03:07:11.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.214+0000 7fcc589e6640 1 -- 172.21.15.116:0/3942484702 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c018c10 con 0x7fcc54106a80 2024-09-12T03:07:11.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.214+0000 7fcc599e8640 1 -- 172.21.15.116:0/3942484702 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc541a70b0 con 0x7fcc54106a80 2024-09-12T03:07:11.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.215+0000 7fcc599e8640 1 -- 172.21.15.116:0/3942484702 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc541021c0 con 0x7fcc54106a80 2024-09-12T03:07:11.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.216+0000 7fcc589e6640 1 -- 172.21.15.116:0/3942484702 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fcc4c00f5f0 con 0x7fcc54106a80 2024-09-12T03:07:11.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.217+0000 7fcc589e6640 1 --2- 172.21.15.116:0/3942484702 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcc400a2b80 0x7fcc400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:11.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.217+0000 7fcc52ffd640 1 --2- 172.21.15.116:0/3942484702 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcc400a2b80 0x7fcc400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:11.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.217+0000 7fcc589e6640 1 -- 172.21.15.116:0/3942484702 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fcc4c0cf780 con 0x7fcc54106a80 2024-09-12T03:07:11.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.217+0000 7fcc52ffd640 1 --2- 172.21.15.116:0/3942484702 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcc400a2b80 0x7fcc400a5040 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcc44009930 tx=0x7fcc44008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:11.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.221+0000 7fcc589e6640 1 -- 172.21.15.116:0/3942484702 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcc4c014030 con 0x7fcc54106a80 2024-09-12T03:07:11.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.423+0000 7fcc599e8640 1 -- 172.21.15.116:0/3942484702 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc5410a050 con 0x7fcc54106a80 2024-09-12T03:07:11.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.424+0000 7fcc589e6640 1 -- 172.21.15.116:0/3942484702 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7fcc4c09c710 con 0x7fcc54106a80 2024-09-12T03:07:11.425 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:07:11.425 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:07:11.425 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:07:11.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.427+0000 7fcc599e8640 1 -- 172.21.15.116:0/3942484702 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcc400a2b80 msgr2=0x7fcc400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:11.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.427+0000 7fcc599e8640 1 --2- 172.21.15.116:0/3942484702 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcc400a2b80 0x7fcc400a5040 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcc44009930 tx=0x7fcc44008040 comp rx=0 tx=0).stop 2024-09-12T03:07:11.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.427+0000 7fcc599e8640 1 -- 172.21.15.116:0/3942484702 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc54106a80 msgr2=0x7fcc540fd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:11.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.427+0000 7fcc599e8640 1 --2- 172.21.15.116:0/3942484702 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc54106a80 0x7fcc540fd7b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c00cce0 tx=0x7fcc4c00a620 comp rx=0 tx=0).stop 2024-09-12T03:07:11.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.427+0000 7fcc599e8640 1 -- 172.21.15.116:0/3942484702 shutdown_connections 2024-09-12T03:07:11.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.427+0000 7fcc599e8640 1 --2- 172.21.15.116:0/3942484702 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcc400a2b80 0x7fcc400a5040 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:11.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.427+0000 7fcc599e8640 1 --2- 172.21.15.116:0/3942484702 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc54106a80 0x7fcc540fd7b0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:11.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.427+0000 7fcc599e8640 1 --2- 172.21.15.116:0/3942484702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc54101420 0x7fcc540fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:11.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.427+0000 7fcc599e8640 1 --2- 172.21.15.116:0/3942484702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc54100a80 0x7fcc540fcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:11.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.427+0000 7fcc599e8640 1 -- 172.21.15.116:0/3942484702 >> 172.21.15.116:0/3942484702 conn(0x7fcc540fc7a0 msgr2=0x7fcc5406aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:11.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.427+0000 7fcc599e8640 1 -- 172.21.15.116:0/3942484702 shutdown_connections 2024-09-12T03:07:11.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:11.427+0000 7fcc599e8640 1 -- 172.21.15.116:0/3942484702 wait complete. 2024-09-12T03:07:11.948 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:07:12.172 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id 34c5c87e --release --await' 2024-09-12T03:07:12.172 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id 34c5c87e --release --await 2024-09-12T03:07:12.206 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:07:12.474 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:12 smithi116 ceph-mon[31410]: pgmap v464: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-12T03:07:12.474 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:12 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3942484702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:07:12.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:12 smithi183 ceph-mon[35535]: pgmap v464: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-12T03:07:12.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:12 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3942484702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:07:12.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.696+0000 7f020f63b640 1 -- 172.21.15.116:0/3789558734 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0208102790 msgr2=0x7f02081067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:12.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.696+0000 7f020f63b640 1 --2- 172.21.15.116:0/3789558734 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0208102790 0x7f02081067e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0204009900 tx=0x7f020402a410 comp rx=0 tx=0).stop 2024-09-12T03:07:12.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.696+0000 7f020f63b640 1 -- 172.21.15.116:0/3789558734 shutdown_connections 2024-09-12T03:07:12.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.696+0000 7f020f63b640 1 --2- 172.21.15.116:0/3789558734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0208106f10 0x7f020810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.696+0000 7f020f63b640 1 --2- 172.21.15.116:0/3789558734 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0208102790 0x7f02081067e0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.696+0000 7f020f63b640 1 --2- 172.21.15.116:0/3789558734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0208101de0 0x7f02081021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.696+0000 7f020f63b640 1 -- 172.21.15.116:0/3789558734 >> 172.21.15.116:0/3789558734 conn(0x7f02080fd8c0 msgr2=0x7f02080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:12.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.697+0000 7f020f63b640 1 -- 172.21.15.116:0/3789558734 shutdown_connections 2024-09-12T03:07:12.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.697+0000 7f020f63b640 1 -- 172.21.15.116:0/3789558734 wait complete. 2024-09-12T03:07:12.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.697+0000 7f020f63b640 1 Processor -- start 2024-09-12T03:07:12.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.697+0000 7f020f63b640 1 -- start start 2024-09-12T03:07:12.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.697+0000 7f020f63b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0208101de0 0x7f02080744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:12.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.697+0000 7f020f63b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0208102790 0x7f0208074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:12.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.697+0000 7f020f63b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0208106f10 0x7f0208074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:12.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.697+0000 7f020f63b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0208111f70 con 0x7f0208102790 2024-09-12T03:07:12.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.697+0000 7f020f63b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0208111df0 con 0x7f0208106f10 2024-09-12T03:07:12.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.697+0000 7f020f63b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f02081120f0 con 0x7f0208101de0 2024-09-12T03:07:12.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.698+0000 7f020cbd6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0208102790 0x7f0208074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:12.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.698+0000 7f020cbd6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0208102790 0x7f0208074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46578/0 (socket says 172.21.15.116:46578) 2024-09-12T03:07:12.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.698+0000 7f020cbd6640 1 -- 172.21.15.116:0/270659648 learned_addr learned my addr 172.21.15.116:0/270659648 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:12.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.698+0000 7f020d3d7640 1 --2- 172.21.15.116:0/270659648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0208101de0 0x7f02080744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:12.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.698+0000 7f020dbd8640 1 --2- 172.21.15.116:0/270659648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0208106f10 0x7f0208074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:12.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.698+0000 7f020cbd6640 1 -- 172.21.15.116:0/270659648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0208101de0 msgr2=0x7f02080744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:12.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.698+0000 7f020cbd6640 1 --2- 172.21.15.116:0/270659648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0208101de0 0x7f02080744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.698+0000 7f020cbd6640 1 -- 172.21.15.116:0/270659648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0208106f10 msgr2=0x7f0208074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:12.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.698+0000 7f020cbd6640 1 --2- 172.21.15.116:0/270659648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0208106f10 0x7f0208074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.698+0000 7f020cbd6640 1 -- 172.21.15.116:0/270659648 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02081a6870 con 0x7f0208102790 2024-09-12T03:07:12.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.698+0000 7f020d3d7640 1 --2- 172.21.15.116:0/270659648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0208101de0 0x7f02080744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:07:12.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.698+0000 7f020cbd6640 1 --2- 172.21.15.116:0/270659648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0208102790 0x7f0208074a30 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0204006010 tx=0x7f02040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:12.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.698+0000 7f01fe7fc640 1 -- 172.21.15.116:0/270659648 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020403f030 con 0x7f0208102790 2024-09-12T03:07:12.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.698+0000 7f020f63b640 1 -- 172.21.15.116:0/270659648 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02081a6b00 con 0x7f0208102790 2024-09-12T03:07:12.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.698+0000 7f01fe7fc640 1 -- 172.21.15.116:0/270659648 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0204003ce0 con 0x7f0208102790 2024-09-12T03:07:12.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.698+0000 7f01fe7fc640 1 -- 172.21.15.116:0/270659648 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0204035070 con 0x7f0208102790 2024-09-12T03:07:12.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.698+0000 7f020f63b640 1 -- 172.21.15.116:0/270659648 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02081a7030 con 0x7f0208102790 2024-09-12T03:07:12.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.700+0000 7f01fe7fc640 1 -- 172.21.15.116:0/270659648 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f0204042030 con 0x7f0208102790 2024-09-12T03:07:12.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.701+0000 7f0211a14740 1 Processor -- start 2024-09-12T03:07:12.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.701+0000 7f0211a14740 1 -- start start 2024-09-12T03:07:12.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.701+0000 7f01fe7fc640 1 --2- 172.21.15.116:0/270659648 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f01f40a2b80 0x7f01f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:12.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.701+0000 7f0211a14740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56428942d4a0 0x56428942d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:12.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.701+0000 7f0211a14740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56428942ddc0 0x564289438140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:12.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.701+0000 7f0211a14740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564289438680 0x56428943aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:12.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.701+0000 7f0211a14740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5642893932e0 con 0x56428942ddc0 2024-09-12T03:07:12.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.701+0000 7f0211a14740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56428933ace0 con 0x564289438680 2024-09-12T03:07:12.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.701+0000 7f0211a14740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56428938f2a0 con 0x56428942d4a0 2024-09-12T03:07:12.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.701+0000 7f020dbd8640 1 --2- 172.21.15.116:0/270659648 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f01f40a2b80 0x7f01f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:12.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.701+0000 7f020cbd6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56428942ddc0 0x564289438140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:12.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.701+0000 7f020cbd6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56428942ddc0 0x564289438140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46584/0 (socket says 172.21.15.116:46584) 2024-09-12T03:07:12.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.701+0000 7f020cbd6640 1 -- 172.21.15.116:0/2786388215 learned_addr learned my addr 172.21.15.116:0/2786388215 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:12.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.702+0000 7f020d3d7640 1 --2- 172.21.15.116:0/2786388215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56428942d4a0 0x56428942d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:12.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.702+0000 7f020dbd8640 1 --2- 172.21.15.116:0/2786388215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564289438680 0x56428943aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:12.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.702+0000 7f01fe7fc640 1 -- 172.21.15.116:0/270659648 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f02040ef600 con 0x7f0208102790 2024-09-12T03:07:12.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.702+0000 7f020cbd6640 1 -- 172.21.15.116:0/2786388215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56428942d4a0 msgr2=0x56428942d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:12.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.702+0000 7f020cbd6640 1 --2- 172.21.15.116:0/2786388215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56428942d4a0 0x56428942d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.702+0000 7f020cbd6640 1 -- 172.21.15.116:0/2786388215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564289438680 msgr2=0x56428943aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:12.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.702+0000 7f020cbd6640 1 --2- 172.21.15.116:0/2786388215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564289438680 0x56428943aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.702+0000 7f020cbd6640 1 -- 172.21.15.116:0/2786388215 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56428931f440 con 0x56428942ddc0 2024-09-12T03:07:12.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.702+0000 7f020dbd8640 1 --2- 172.21.15.116:0/2786388215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564289438680 0x56428943aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:07:12.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.702+0000 7f020dbd8640 1 --2- 172.21.15.116:0/270659648 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f01f40a2b80 0x7f01f40a5040 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0200008fd0 tx=0x7f0200014950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:12.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.702+0000 7f020cbd6640 1 --2- 172.21.15.116:0/2786388215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56428942ddc0 0x564289438140 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f02040b83b0 tx=0x7f0204049bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=468b6dbd3d76be97 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:12.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.702+0000 7f01f37fe640 1 -- 172.21.15.116:0/2786388215 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0204049070 con 0x56428942ddc0 2024-09-12T03:07:12.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.702+0000 7f01f37fe640 1 -- 172.21.15.116:0/2786388215 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f020403b580 con 0x56428942ddc0 2024-09-12T03:07:12.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.702+0000 7f01f37fe640 1 -- 172.21.15.116:0/2786388215 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020403f990 con 0x56428942ddc0 2024-09-12T03:07:12.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.703+0000 7f0211a14740 1 -- 172.21.15.116:0/2786388215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56428942ddc0 msgr2=0x564289438140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:12.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.703+0000 7f0211a14740 1 --2- 172.21.15.116:0/2786388215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56428942ddc0 0x564289438140 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f02040b83b0 tx=0x7f0204049bd0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.703+0000 7f0211a14740 1 -- 172.21.15.116:0/2786388215 shutdown_connections 2024-09-12T03:07:12.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.703+0000 7f0211a14740 1 --2- 172.21.15.116:0/2786388215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564289438680 0x56428943aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.703+0000 7f0211a14740 1 --2- 172.21.15.116:0/2786388215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56428942ddc0 0x564289438140 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.703+0000 7f0211a14740 1 --2- 172.21.15.116:0/2786388215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56428942d4a0 0x56428942d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.703+0000 7f0211a14740 1 -- 172.21.15.116:0/2786388215 >> 172.21.15.116:0/2786388215 conn(0x564289429a30 msgr2=0x564289429e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:12.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.703+0000 7f0211a14740 1 -- 172.21.15.116:0/2786388215 shutdown_connections 2024-09-12T03:07:12.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.703+0000 7f0211a14740 1 -- 172.21.15.116:0/2786388215 wait complete. 2024-09-12T03:07:12.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.703+0000 7f0211a14740 1 -- start start 2024-09-12T03:07:12.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.703+0000 7f0211a14740 1 Processor -- start 2024-09-12T03:07:12.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.704+0000 7f0211a14740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56428942d4a0 0x564289437f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:12.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.704+0000 7f0211a14740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56428942ddc0 0x564289435220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:12.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.704+0000 7f0211a14740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564289438680 0x564289435760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:12.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.704+0000 7f0211a14740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56428943b010 con 0x564289438680 2024-09-12T03:07:12.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.704+0000 7f0211a14740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56428934e050 con 0x56428942d4a0 2024-09-12T03:07:12.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.704+0000 7f0211a14740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56428943b190 con 0x56428942ddc0 2024-09-12T03:07:12.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.704+0000 7f0211a14740 10 client.0 _ll_register_callbacks cb 0x56428938e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:07:12.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.704+0000 7f020dbd8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564289438680 0x564289435760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:12.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.704+0000 7f020dbd8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564289438680 0x564289435760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46588/0 (socket says 172.21.15.116:46588) 2024-09-12T03:07:12.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.704+0000 7f020dbd8640 1 -- 172.21.15.116:0/3626461120 learned_addr learned my addr 172.21.15.116:0/3626461120 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:12.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.704+0000 7f020cbd6640 1 --2- 172.21.15.116:0/3626461120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56428942ddc0 0x564289435220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:12.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.704+0000 7f020d3d7640 1 --2- 172.21.15.116:0/3626461120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56428942d4a0 0x564289437f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:12.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.704+0000 7f020dbd8640 1 -- 172.21.15.116:0/3626461120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56428942ddc0 msgr2=0x564289435220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:12.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.704+0000 7f020dbd8640 1 --2- 172.21.15.116:0/3626461120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56428942ddc0 0x564289435220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.704+0000 7f020dbd8640 1 -- 172.21.15.116:0/3626461120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56428942d4a0 msgr2=0x564289437f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:12.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.704+0000 7f020dbd8640 1 --2- 172.21.15.116:0/3626461120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56428942d4a0 0x564289437f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.704+0000 7f020dbd8640 1 -- 172.21.15.116:0/3626461120 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642891043c0 con 0x564289438680 2024-09-12T03:07:12.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.705+0000 7f020dbd8640 1 --2- 172.21.15.116:0/3626461120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564289438680 0x564289435760 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0200011c10 tx=0x7f0200004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31be2510ae2a9eb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:12.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.705+0000 7f01f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:07:12.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.705+0000 7f01f0ff9640 1 -- 172.21.15.116:0/3626461120 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0200023070 con 0x564289438680 2024-09-12T03:07:12.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.705+0000 7f01f0ff9640 1 -- 172.21.15.116:0/3626461120 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f020001f5f0 con 0x564289438680 2024-09-12T03:07:12.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.705+0000 7f0211a14740 1 -- 172.21.15.116:0/3626461120 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5642893a0850 con 0x564289438680 2024-09-12T03:07:12.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.705+0000 7f01f0ff9640 1 -- 172.21.15.116:0/3626461120 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f020002f050 con 0x564289438680 2024-09-12T03:07:12.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.705+0000 7f0211a14740 10 client.16296 fetch_fsmap learned FSMap version 25 2024-09-12T03:07:12.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.705+0000 7f0211a14740 1 -- 172.21.15.116:0/3626461120 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5642894468b0 con 0x564289438680 2024-09-12T03:07:12.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.705+0000 7f01f0ff9640 1 -- 172.21.15.116:0/3626461120 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5642893a0850 con 0x564289438680 2024-09-12T03:07:12.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.705+0000 7f01f0ff9640 1 -- 172.21.15.116:0/3626461120 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f020001e070 con 0x564289438680 2024-09-12T03:07:12.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.705+0000 7f0211a14740 10 client.16296 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:07:12.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.705+0000 7f0211a14740 10 client.16296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:07:12.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.705+0000 7f0211a14740 20 client.16296 populate_metadata read hostname 'smithi116' 2024-09-12T03:07:12.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.705+0000 7f0211a14740 1 --2- 172.21.15.116:0/3626461120 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x564289447cc0 0x56428944a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:12.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.705+0000 7f0211a14740 4 client.16296 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:07:12.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.705+0000 7f0211a14740 1 -- 172.21.15.116:0/3626461120 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56428939ad60 con 0x564289447cc0 2024-09-12T03:07:12.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.706+0000 7f020d3d7640 1 --2- 172.21.15.116:0/3626461120 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x564289447cc0 0x56428944a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:12.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.706+0000 7f020d3d7640 1 --2- 172.21.15.116:0/3626461120 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x564289447cc0 0x56428944a140 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1708c3e3a4c6a530 server_cookie=8164f75911c45aff in_seq=0 out_seq=0 2024-09-12T03:07:12.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.706+0000 7f01f0ff9640 10 client.16296 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:12.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.707+0000 7f01f0ff9640 1 -- 172.21.15.116:0/3626461120 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56428939ad60 con 0x564289447cc0 2024-09-12T03:07:12.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.707+0000 7f01f0ff9640 10 client.16296 handle_command_reply: tid=0 2024-09-12T03:07:12.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.707+0000 7f0211a14740 1 client.16296 shutdown 2024-09-12T03:07:12.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.707+0000 7f0211a14740 1 -- 172.21.15.116:0/3626461120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564289438680 msgr2=0x564289435760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:12.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.707+0000 7f0211a14740 1 --2- 172.21.15.116:0/3626461120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564289438680 0x564289435760 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0200011c10 tx=0x7f0200004ab0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.707+0000 7f0211a14740 1 -- 172.21.15.116:0/3626461120 shutdown_connections 2024-09-12T03:07:12.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.707+0000 7f0211a14740 1 --2- 172.21.15.116:0/3626461120 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x564289447cc0 0x56428944a140 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.707+0000 7f0211a14740 1 --2- 172.21.15.116:0/3626461120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564289438680 0x564289435760 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.707+0000 7f0211a14740 1 --2- 172.21.15.116:0/3626461120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56428942ddc0 0x564289435220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.707+0000 7f0211a14740 1 --2- 172.21.15.116:0/3626461120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56428942d4a0 0x564289437f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.707+0000 7f0211a14740 1 -- 172.21.15.116:0/3626461120 >> 172.21.15.116:0/3626461120 conn(0x564289429a30 msgr2=0x56428942f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:12.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.707+0000 7f01f0ff9640 0 client.16296 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:12.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.707+0000 7f0211a14740 1 -- 172.21.15.116:0/3626461120 shutdown_connections 2024-09-12T03:07:12.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.707+0000 7f0211a14740 1 -- 172.21.15.116:0/3626461120 wait complete. 2024-09-12T03:07:12.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.707+0000 7f0211a14740 20 client.16296 trim_cache size 0 max 16384 2024-09-12T03:07:12.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.727+0000 7f0211a14740 1 Processor -- start 2024-09-12T03:07:12.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.727+0000 7f0211a14740 1 -- start start 2024-09-12T03:07:12.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.727+0000 7f0211a14740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564289429a30 0x564289437190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:12.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.727+0000 7f0211a14740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56428942d4a0 0x56428944fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:12.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.727+0000 7f0211a14740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56428942ddc0 0x56428944ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:12.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.727+0000 7f0211a14740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f02081b1ef0 con 0x56428942d4a0 2024-09-12T03:07:12.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.727+0000 7f0211a14740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f02081af7c0 con 0x56428942ddc0 2024-09-12T03:07:12.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.727+0000 7f0211a14740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f02081ad610 con 0x564289429a30 2024-09-12T03:07:12.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.727+0000 7f020cbd6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56428942d4a0 0x56428944fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:12.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.727+0000 7f020cbd6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56428942d4a0 0x56428944fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46598/0 (socket says 172.21.15.116:46598) 2024-09-12T03:07:12.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.727+0000 7f020cbd6640 1 -- 172.21.15.116:0/3637198454 learned_addr learned my addr 172.21.15.116:0/3637198454 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:12.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.728+0000 7f020d3d7640 1 --2- 172.21.15.116:0/3637198454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564289429a30 0x564289437190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:12.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.728+0000 7f020dbd8640 1 --2- 172.21.15.116:0/3637198454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56428942ddc0 0x56428944ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:12.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.728+0000 7f020cbd6640 1 -- 172.21.15.116:0/3637198454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564289429a30 msgr2=0x564289437190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:12.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.728+0000 7f020cbd6640 1 --2- 172.21.15.116:0/3637198454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564289429a30 0x564289437190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.728+0000 7f020cbd6640 1 -- 172.21.15.116:0/3637198454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56428942ddc0 msgr2=0x56428944ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:12.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.728+0000 7f020cbd6640 1 --2- 172.21.15.116:0/3637198454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56428942ddc0 0x56428944ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.728+0000 7f020cbd6640 1 -- 172.21.15.116:0/3637198454 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56428942d220 con 0x56428942d4a0 2024-09-12T03:07:12.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.728+0000 7f020cbd6640 1 --2- 172.21.15.116:0/3637198454 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56428942d4a0 0x56428944fa60 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f02040045a0 tx=0x7f02040b82f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f934d86e167055 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:12.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.728+0000 7f01f2ffd640 1 -- 172.21.15.116:0/3637198454 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0204049040 con 0x56428942d4a0 2024-09-12T03:07:12.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.728+0000 7f01f2ffd640 1 -- 172.21.15.116:0/3637198454 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02040bc650 con 0x56428942d4a0 2024-09-12T03:07:12.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.728+0000 7f01f2ffd640 1 -- 172.21.15.116:0/3637198454 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02040ba250 con 0x56428942d4a0 2024-09-12T03:07:12.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.729+0000 7f0211a14740 1 -- 172.21.15.116:0/3637198454 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56428942d4a0 msgr2=0x56428944fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:12.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.729+0000 7f0211a14740 1 --2- 172.21.15.116:0/3637198454 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56428942d4a0 0x56428944fa60 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f02040045a0 tx=0x7f02040b82f0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.729+0000 7f0211a14740 1 -- 172.21.15.116:0/3637198454 shutdown_connections 2024-09-12T03:07:12.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.729+0000 7f0211a14740 1 --2- 172.21.15.116:0/3637198454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56428942ddc0 0x56428944ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.729+0000 7f0211a14740 1 --2- 172.21.15.116:0/3637198454 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56428942d4a0 0x56428944fa60 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.729+0000 7f0211a14740 1 --2- 172.21.15.116:0/3637198454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564289429a30 0x564289437190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.729+0000 7f0211a14740 1 -- 172.21.15.116:0/3637198454 >> 172.21.15.116:0/3637198454 conn(0x5642894358c0 msgr2=0x564289436a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:12.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.730+0000 7f0211a14740 1 -- 172.21.15.116:0/3637198454 shutdown_connections 2024-09-12T03:07:12.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.730+0000 7f0211a14740 1 -- 172.21.15.116:0/3637198454 wait complete. 2024-09-12T03:07:12.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.730+0000 7f0211a14740 1 -- start start 2024-09-12T03:07:12.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.730+0000 7f0211a14740 1 Processor -- start 2024-09-12T03:07:12.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.730+0000 7f0211a14740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564289429a30 0x564289440970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:12.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.730+0000 7f0211a14740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56428942d4a0 0x56428943dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:12.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.730+0000 7f0211a14740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56428942ddc0 0x56428943e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:12.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.730+0000 7f0211a14740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f02081adc80 con 0x564289429a30 2024-09-12T03:07:12.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.730+0000 7f0211a14740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f02081af0f0 con 0x56428942d4a0 2024-09-12T03:07:12.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.730+0000 7f0211a14740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f02081ae350 con 0x56428942ddc0 2024-09-12T03:07:12.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.731+0000 7f0211a14740 10 client.0 _ll_register_callbacks cb 0x7f02000118e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:07:12.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.731+0000 7f020cbd6640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56428942d4a0 0x56428943dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:12.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.731+0000 7f020cbd6640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56428942d4a0 0x56428943dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.116:34434/0 (socket says 172.21.15.116:34434) 2024-09-12T03:07:12.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.731+0000 7f020d3d7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564289429a30 0x564289440970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:12.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.731+0000 7f020cbd6640 1 -- 172.21.15.116:0/3601553771 learned_addr learned my addr 172.21.15.116:0/3601553771 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:12.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.731+0000 7f020dbd8640 1 --2- 172.21.15.116:0/3601553771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56428942ddc0 0x56428943e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:12.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.731+0000 7f020cbd6640 1 -- 172.21.15.116:0/3601553771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56428942ddc0 msgr2=0x56428943e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:12.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.731+0000 7f020cbd6640 1 --2- 172.21.15.116:0/3601553771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56428942ddc0 0x56428943e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.731+0000 7f020cbd6640 1 -- 172.21.15.116:0/3601553771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564289429a30 msgr2=0x564289440970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:12.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.731+0000 7f020cbd6640 1 --2- 172.21.15.116:0/3601553771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564289429a30 0x564289440970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.731+0000 7f020cbd6640 1 -- 172.21.15.116:0/3601553771 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56428943e990 con 0x56428942d4a0 2024-09-12T03:07:12.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.731+0000 7f020d3d7640 1 --2- 172.21.15.116:0/3601553771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564289429a30 0x564289440970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:07:12.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.731+0000 7f020cbd6640 1 --2- 172.21.15.116:0/3601553771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56428942d4a0 0x56428943dd60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f02040b8920 tx=0x7f02040057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d5506f51d24139e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:12.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.731+0000 7f01f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-09-12T03:07:12.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.731+0000 7f01f0ff9640 1 -- 172.21.15.116:0/3601553771 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0204049070 con 0x56428942d4a0 2024-09-12T03:07:12.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.732+0000 7f0211a14740 1 -- 172.21.15.116:0/3601553771 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02040bc650 con 0x56428942d4a0 2024-09-12T03:07:12.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.732+0000 7f01f0ff9640 1 -- 172.21.15.116:0/3601553771 <== mon.2 v2:172.21.15.177:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f02040b85c0 con 0x56428942d4a0 2024-09-12T03:07:12.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.732+0000 7f01f0ff9640 1 -- 172.21.15.116:0/3601553771 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f02040bbf50 con 0x56428942d4a0 2024-09-12T03:07:12.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.732+0000 7f0211a14740 10 client.25466 fetch_fsmap learned FSMap version 25 2024-09-12T03:07:12.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.732+0000 7f0211a14740 1 -- 172.21.15.116:0/3601553771 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5642894b2910 con 0x56428942d4a0 2024-09-12T03:07:12.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.732+0000 7f01f0ff9640 1 -- 172.21.15.116:0/3601553771 <== mon.2 v2:172.21.15.177:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02040bc650 con 0x56428942d4a0 2024-09-12T03:07:12.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.732+0000 7f01f0ff9640 1 -- 172.21.15.116:0/3601553771 <== mon.2 v2:172.21.15.177:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f02040de070 con 0x56428942d4a0 2024-09-12T03:07:12.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.732+0000 7f0211a14740 10 client.25466 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:07:12.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.732+0000 7f0211a14740 10 client.25466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:07:12.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.732+0000 7f0211a14740 20 client.25466 populate_metadata read hostname 'smithi116' 2024-09-12T03:07:12.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.732+0000 7f0211a14740 1 --2- 172.21.15.116:0/3601553771 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x564289438680 0x5642894b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:12.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.732+0000 7f0211a14740 4 client.25466 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:07:12.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.732+0000 7f0211a14740 1 -- 172.21.15.116:0/3601553771 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564289317210 con 0x564289438680 2024-09-12T03:07:12.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.732+0000 7f020d3d7640 1 --2- 172.21.15.116:0/3601553771 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x564289438680 0x5642894b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:12.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.733+0000 7f01f0ff9640 10 client.25466 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:12.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.733+0000 7f020d3d7640 1 --2- 172.21.15.116:0/3601553771 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x564289438680 0x5642894b4ca0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10b7b3e35627bd10 server_cookie=69acb9ffef1d78a4 in_seq=0 out_seq=0 2024-09-12T03:07:12.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.733+0000 7f01f0ff9640 1 -- 172.21.15.116:0/3601553771 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564289317210 con 0x564289438680 2024-09-12T03:07:12.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.733+0000 7f01f0ff9640 10 client.25466 handle_command_reply: tid=0 2024-09-12T03:07:12.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.733+0000 7f0211a14740 1 client.25466 shutdown 2024-09-12T03:07:12.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.733+0000 7f0211a14740 1 -- 172.21.15.116:0/3601553771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56428942d4a0 msgr2=0x56428943dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:12.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.733+0000 7f0211a14740 1 --2- 172.21.15.116:0/3601553771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56428942d4a0 0x56428943dd60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f02040b8920 tx=0x7f02040057c0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.733+0000 7f0211a14740 1 -- 172.21.15.116:0/3601553771 shutdown_connections 2024-09-12T03:07:12.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.733+0000 7f0211a14740 1 --2- 172.21.15.116:0/3601553771 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x564289438680 0x5642894b4ca0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.733+0000 7f0211a14740 1 --2- 172.21.15.116:0/3601553771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56428942ddc0 0x56428943e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.733+0000 7f0211a14740 1 --2- 172.21.15.116:0/3601553771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56428942d4a0 0x56428943dd60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.733+0000 7f0211a14740 1 --2- 172.21.15.116:0/3601553771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564289429a30 0x564289440970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.733+0000 7f0211a14740 1 -- 172.21.15.116:0/3601553771 >> 172.21.15.116:0/3601553771 conn(0x5642894358c0 msgr2=0x5642894372a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:12.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.733+0000 7f01f0ff9640 0 client.25466 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:12.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.734+0000 7f0211a14740 1 -- 172.21.15.116:0/3601553771 shutdown_connections 2024-09-12T03:07:12.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.734+0000 7f0211a14740 1 -- 172.21.15.116:0/3601553771 wait complete. 2024-09-12T03:07:12.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.734+0000 7f0211a14740 20 client.25466 trim_cache size 0 max 16384 2024-09-12T03:07:12.738 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:07:12.739 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:07:12.739 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:07:12.739 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:07:12.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.734+0000 7f020f63b640 1 -- 172.21.15.116:0/270659648 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f01f40a2b80 msgr2=0x7f01f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:12.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.734+0000 7f020f63b640 1 --2- 172.21.15.116:0/270659648 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f01f40a2b80 0x7f01f40a5040 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0200008fd0 tx=0x7f0200014950 comp rx=0 tx=0).stop 2024-09-12T03:07:12.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.734+0000 7f020f63b640 1 -- 172.21.15.116:0/270659648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0208102790 msgr2=0x7f0208074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:12.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.734+0000 7f020f63b640 1 --2- 172.21.15.116:0/270659648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0208102790 0x7f0208074a30 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0204006010 tx=0x7f02040042f0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.735+0000 7f020f63b640 1 -- 172.21.15.116:0/270659648 shutdown_connections 2024-09-12T03:07:12.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.735+0000 7f020f63b640 1 --2- 172.21.15.116:0/270659648 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f01f40a2b80 0x7f01f40a5040 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.735+0000 7f020f63b640 1 --2- 172.21.15.116:0/270659648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0208106f10 0x7f0208074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.735+0000 7f020f63b640 1 --2- 172.21.15.116:0/270659648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0208102790 0x7f0208074a30 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.735+0000 7f020f63b640 1 --2- 172.21.15.116:0/270659648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0208101de0 0x7f02080744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:12.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.735+0000 7f020f63b640 1 -- 172.21.15.116:0/270659648 >> 172.21.15.116:0/270659648 conn(0x7f02080fd8c0 msgr2=0x7f02080fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:12.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.735+0000 7f020f63b640 1 -- 172.21.15.116:0/270659648 shutdown_connections 2024-09-12T03:07:12.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:12.735+0000 7f020f63b640 1 -- 172.21.15.116:0/270659648 wait complete. 2024-09-12T03:07:12.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:12 smithi177 ceph-mon[35305]: pgmap v464: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-12T03:07:12.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:12 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3942484702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:07:12.986 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:07:13.451 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:07:13.452 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:07:13.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.910+0000 7f3f7dd2a640 1 -- 172.21.15.116:0/1036457914 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f7810c1a0 msgr2=0x7f3f7810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:13.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.910+0000 7f3f7dd2a640 1 --2- 172.21.15.116:0/1036457914 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f7810c1a0 0x7f3f7810e610 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f3f70009ca0 tx=0x7f3f7002a950 comp rx=0 tx=0).stop 2024-09-12T03:07:13.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.910+0000 7f3f7dd2a640 1 -- 172.21.15.116:0/1036457914 shutdown_connections 2024-09-12T03:07:13.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.910+0000 7f3f7dd2a640 1 --2- 172.21.15.116:0/1036457914 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f7810c1a0 0x7f3f7810e610 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.910+0000 7f3f7dd2a640 1 --2- 172.21.15.116:0/1036457914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f781036e0 0x7f3f7810bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.910+0000 7f3f7dd2a640 1 --2- 172.21.15.116:0/1036457914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f78102dc0 0x7f3f781031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.910+0000 7f3f7dd2a640 1 -- 172.21.15.116:0/1036457914 >> 172.21.15.116:0/1036457914 conn(0x7f3f780fc7a0 msgr2=0x7f3f780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:13.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.910+0000 7f3f7dd2a640 1 -- 172.21.15.116:0/1036457914 shutdown_connections 2024-09-12T03:07:13.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.910+0000 7f3f7dd2a640 1 -- 172.21.15.116:0/1036457914 wait complete. 2024-09-12T03:07:13.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.911+0000 7f3f7dd2a640 1 Processor -- start 2024-09-12T03:07:13.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.911+0000 7f3f7dd2a640 1 -- start start 2024-09-12T03:07:13.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.912+0000 7f3f7dd2a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f78102dc0 0x7f3f78198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:13.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.912+0000 7f3f7dd2a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f781036e0 0x7f3f78198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:13.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.912+0000 7f3f7dd2a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f7810c1a0 0x7f3f7819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:13.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.912+0000 7f3f7dd2a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3f78111f70 con 0x7f3f78102dc0 2024-09-12T03:07:13.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.912+0000 7f3f7dd2a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3f78111df0 con 0x7f3f7810c1a0 2024-09-12T03:07:13.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.912+0000 7f3f7dd2a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3f781120f0 con 0x7f3f781036e0 2024-09-12T03:07:13.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.912+0000 7f3f777fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f78102dc0 0x7f3f78198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:13.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.912+0000 7f3f777fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f78102dc0 0x7f3f78198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46620/0 (socket says 172.21.15.116:46620) 2024-09-12T03:07:13.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.912+0000 7f3f777fe640 1 -- 172.21.15.116:0/3255435121 learned_addr learned my addr 172.21.15.116:0/3255435121 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:13.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.912+0000 7f3f77fff640 1 --2- 172.21.15.116:0/3255435121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f7810c1a0 0x7f3f7819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:13.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.912+0000 7f3f76ffd640 1 --2- 172.21.15.116:0/3255435121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f781036e0 0x7f3f78198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:13.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.912+0000 7f3f777fe640 1 -- 172.21.15.116:0/3255435121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f781036e0 msgr2=0x7f3f78198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:13.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.912+0000 7f3f777fe640 1 --2- 172.21.15.116:0/3255435121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f781036e0 0x7f3f78198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.912+0000 7f3f777fe640 1 -- 172.21.15.116:0/3255435121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f7810c1a0 msgr2=0x7f3f7819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:13.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.912+0000 7f3f777fe640 1 --2- 172.21.15.116:0/3255435121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f7810c1a0 0x7f3f7819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.912+0000 7f3f777fe640 1 -- 172.21.15.116:0/3255435121 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f7819d400 con 0x7f3f78102dc0 2024-09-12T03:07:13.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.912+0000 7f3f76ffd640 1 --2- 172.21.15.116:0/3255435121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f781036e0 0x7f3f78198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:07:13.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.912+0000 7f3f777fe640 1 --2- 172.21.15.116:0/3255435121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f78102dc0 0x7f3f78198450 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f3f6800d770 tx=0x7f3f6800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:13.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.912+0000 7f3f74ff9640 1 -- 172.21.15.116:0/3255435121 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f68017070 con 0x7f3f78102dc0 2024-09-12T03:07:13.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.912+0000 7f3f74ff9640 1 -- 172.21.15.116:0/3255435121 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f68005500 con 0x7f3f78102dc0 2024-09-12T03:07:13.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.912+0000 7f3f7dd2a640 1 -- 172.21.15.116:0/3255435121 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f7819d6f0 con 0x7f3f78102dc0 2024-09-12T03:07:13.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.913+0000 7f3f74ff9640 1 -- 172.21.15.116:0/3255435121 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f68004e00 con 0x7f3f78102dc0 2024-09-12T03:07:13.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.913+0000 7f3f7dd2a640 1 -- 172.21.15.116:0/3255435121 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f781a4fd0 con 0x7f3f78102dc0 2024-09-12T03:07:13.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.915+0000 7f3f74ff9640 1 -- 172.21.15.116:0/3255435121 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3f68004fa0 con 0x7f3f78102dc0 2024-09-12T03:07:13.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.915+0000 7f3f80103740 1 Processor -- start 2024-09-12T03:07:13.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.915+0000 7f3f80103740 1 -- start start 2024-09-12T03:07:13.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.916+0000 7f3f80103740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581c42e9060 0x5581c42e9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:13.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.916+0000 7f3f80103740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581c42e9980 0x5581c42f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:13.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.916+0000 7f3f80103740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581c42f4240 0x5581c42f6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:13.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.916+0000 7f3f80103740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5581c424e620 con 0x5581c42e9060 2024-09-12T03:07:13.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.916+0000 7f3f80103740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5581c41f5e50 con 0x5581c42f4240 2024-09-12T03:07:13.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.916+0000 7f3f80103740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5581c424a5e0 con 0x5581c42e9980 2024-09-12T03:07:13.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.916+0000 7f3f77fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581c42e9060 0x5581c42e9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:13.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.916+0000 7f3f77fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581c42e9060 0x5581c42e9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46626/0 (socket says 172.21.15.116:46626) 2024-09-12T03:07:13.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.916+0000 7f3f77fff640 1 -- 172.21.15.116:0/1813110407 learned_addr learned my addr 172.21.15.116:0/1813110407 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:13.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.916+0000 7f3f777fe640 1 --2- 172.21.15.116:0/1813110407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581c42e9980 0x5581c42f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:13.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.916+0000 7f3f76ffd640 1 --2- 172.21.15.116:0/1813110407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581c42f4240 0x5581c42f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:13.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.916+0000 7f3f77fff640 1 -- 172.21.15.116:0/1813110407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581c42e9980 msgr2=0x5581c42f3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:13.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.916+0000 7f3f77fff640 1 --2- 172.21.15.116:0/1813110407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581c42e9980 0x5581c42f3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.916+0000 7f3f77fff640 1 -- 172.21.15.116:0/1813110407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581c42f4240 msgr2=0x5581c42f6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:13.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.916+0000 7f3f77fff640 1 --2- 172.21.15.116:0/1813110407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581c42f4240 0x5581c42f6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.916+0000 7f3f77fff640 1 -- 172.21.15.116:0/1813110407 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581c41e6d40 con 0x5581c42e9060 2024-09-12T03:07:13.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.916+0000 7f3f76ffd640 1 --2- 172.21.15.116:0/1813110407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581c42f4240 0x5581c42f6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:07:13.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.916+0000 7f3f74ff9640 1 --2- 172.21.15.116:0/3255435121 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3f600a2b80 0x7f3f600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:13.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.916+0000 7f3f77fff640 1 --2- 172.21.15.116:0/1813110407 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581c42e9060 0x5581c42e9440 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f3f70035000 tx=0x7f3f700098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e04a57c4e0854ff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:13.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.916+0000 7f3f777fe640 1 --2- 172.21.15.116:0/3255435121 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3f600a2b80 0x7f3f600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:13.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.917+0000 7f3f5dffb640 1 -- 172.21.15.116:0/1813110407 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70048070 con 0x5581c42e9060 2024-09-12T03:07:13.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.917+0000 7f3f5dffb640 1 -- 172.21.15.116:0/1813110407 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f7002fa80 con 0x5581c42e9060 2024-09-12T03:07:13.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.917+0000 7f3f5dffb640 1 -- 172.21.15.116:0/1813110407 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70034040 con 0x5581c42e9060 2024-09-12T03:07:13.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.917+0000 7f3f74ff9640 1 -- 172.21.15.116:0/3255435121 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f3f680cea40 con 0x7f3f78102dc0 2024-09-12T03:07:13.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.917+0000 7f3f80103740 1 -- 172.21.15.116:0/1813110407 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581c42e9060 msgr2=0x5581c42e9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:13.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.917+0000 7f3f777fe640 1 --2- 172.21.15.116:0/3255435121 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3f600a2b80 0x7f3f600a5040 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3f680cc8e0 tx=0x7f3f680048a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:13.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.917+0000 7f3f80103740 1 --2- 172.21.15.116:0/1813110407 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581c42e9060 0x5581c42e9440 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f3f70035000 tx=0x7f3f700098d0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.917+0000 7f3f80103740 1 -- 172.21.15.116:0/1813110407 shutdown_connections 2024-09-12T03:07:13.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.917+0000 7f3f80103740 1 --2- 172.21.15.116:0/1813110407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581c42f4240 0x5581c42f6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.917+0000 7f3f80103740 1 --2- 172.21.15.116:0/1813110407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581c42e9980 0x5581c42f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.917+0000 7f3f80103740 1 --2- 172.21.15.116:0/1813110407 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581c42e9060 0x5581c42e9440 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.917+0000 7f3f80103740 1 -- 172.21.15.116:0/1813110407 >> 172.21.15.116:0/1813110407 conn(0x5581c42e55f0 msgr2=0x5581c42e59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:13.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.917+0000 7f3f80103740 1 -- 172.21.15.116:0/1813110407 shutdown_connections 2024-09-12T03:07:13.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.917+0000 7f3f80103740 1 -- 172.21.15.116:0/1813110407 wait complete. 2024-09-12T03:07:13.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.918+0000 7f3f80103740 1 -- start start 2024-09-12T03:07:13.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.918+0000 7f3f80103740 1 Processor -- start 2024-09-12T03:07:13.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.918+0000 7f3f80103740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581c42e9060 0x5581c42f1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:13.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.918+0000 7f3f80103740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581c42e9980 0x5581c42f16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:13.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.918+0000 7f3f80103740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581c42f4240 0x5581c42f1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:13.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.918+0000 7f3f80103740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5581c42f6e10 con 0x5581c42e9980 2024-09-12T03:07:13.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.918+0000 7f3f80103740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5581c41d6f10 con 0x5581c42e9060 2024-09-12T03:07:13.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.918+0000 7f3f80103740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5581c42f6f90 con 0x5581c42f4240 2024-09-12T03:07:13.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.918+0000 7f3f80103740 10 client.0 _ll_register_callbacks cb 0x5581c41fadb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:07:13.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.918+0000 7f3f76ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581c42e9980 0x5581c42f16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:13.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.918+0000 7f3f76ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581c42e9980 0x5581c42f16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46632/0 (socket says 172.21.15.116:46632) 2024-09-12T03:07:13.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.918+0000 7f3f76ffd640 1 -- 172.21.15.116:0/1436956917 learned_addr learned my addr 172.21.15.116:0/1436956917 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:13.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.918+0000 7f3f77fff640 1 --2- 172.21.15.116:0/1436956917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581c42f4240 0x5581c42f1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:13.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.919+0000 7f3f77fff640 1 --2- 172.21.15.116:0/1436956917 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581c42e9060 0x5581c42f1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:13.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.919+0000 7f3f76ffd640 1 -- 172.21.15.116:0/1436956917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581c42f4240 msgr2=0x5581c42f1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:13.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.919+0000 7f3f76ffd640 1 --2- 172.21.15.116:0/1436956917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581c42f4240 0x5581c42f1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.919+0000 7f3f76ffd640 1 -- 172.21.15.116:0/1436956917 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581c42e9060 msgr2=0x5581c42f1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:13.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.919+0000 7f3f76ffd640 1 --2- 172.21.15.116:0/1436956917 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581c42e9060 0x5581c42f1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.919+0000 7f3f76ffd640 1 -- 172.21.15.116:0/1436956917 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581c3f5d580 con 0x5581c42e9980 2024-09-12T03:07:13.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.919+0000 7f3f77fff640 1 --2- 172.21.15.116:0/1436956917 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581c42e9060 0x5581c42f1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:07:13.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.919+0000 7f3f76ffd640 1 --2- 172.21.15.116:0/1436956917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581c42e9980 0x5581c42f16c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00e6e0 tx=0x7f3f6c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bde8e6571358544 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:13.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.919+0000 7f3f56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:07:13.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.919+0000 7f3f56ffd640 1 -- 172.21.15.116:0/1436956917 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c01c060 con 0x5581c42e9980 2024-09-12T03:07:13.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.919+0000 7f3f56ffd640 1 -- 172.21.15.116:0/1436956917 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f3f6c009a80 con 0x5581c42e9980 2024-09-12T03:07:13.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.919+0000 7f3f80103740 1 -- 172.21.15.116:0/1436956917 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f7002fa80 con 0x5581c42e9980 2024-09-12T03:07:13.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.919+0000 7f3f56ffd640 1 -- 172.21.15.116:0/1436956917 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3f6c009c80 con 0x5581c42e9980 2024-09-12T03:07:13.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.919+0000 7f3f80103740 10 client.16314 fetch_fsmap learned FSMap version 25 2024-09-12T03:07:13.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.919+0000 7f3f80103740 1 -- 172.21.15.116:0/1436956917 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5581c4027960 con 0x5581c42e9980 2024-09-12T03:07:13.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.919+0000 7f3f56ffd640 1 -- 172.21.15.116:0/1436956917 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f7002fa80 con 0x5581c42e9980 2024-09-12T03:07:13.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.919+0000 7f3f56ffd640 1 -- 172.21.15.116:0/1436956917 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3f6c022440 con 0x5581c42e9980 2024-09-12T03:07:13.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.919+0000 7f3f80103740 10 client.16314 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:07:13.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.919+0000 7f3f80103740 10 client.16314 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:07:13.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.920+0000 7f3f80103740 20 client.16314 populate_metadata read hostname 'smithi116' 2024-09-12T03:07:13.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.920+0000 7f3f80103740 1 --2- 172.21.15.116:0/1436956917 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5581c4303db0 0x5581c4306220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:13.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.920+0000 7f3f80103740 4 client.16314 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:07:13.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.920+0000 7f3f80103740 1 -- 172.21.15.116:0/1436956917 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581c425bb90 con 0x5581c4303db0 2024-09-12T03:07:13.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.920+0000 7f3f77fff640 1 --2- 172.21.15.116:0/1436956917 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5581c4303db0 0x5581c4306220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:13.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.920+0000 7f3f77fff640 1 --2- 172.21.15.116:0/1436956917 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5581c4303db0 0x5581c4306220 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e22ed95d5ad3265 server_cookie=58060943887d98e6 in_seq=0 out_seq=0 2024-09-12T03:07:13.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.920+0000 7f3f56ffd640 10 client.16314 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:13.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.921+0000 7f3f56ffd640 1 -- 172.21.15.116:0/1436956917 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5581c425bb90 con 0x5581c4303db0 2024-09-12T03:07:13.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.921+0000 7f3f56ffd640 10 client.16314 handle_command_reply: tid=0 2024-09-12T03:07:13.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.921+0000 7f3f80103740 1 client.16314 shutdown 2024-09-12T03:07:13.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.921+0000 7f3f80103740 1 -- 172.21.15.116:0/1436956917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581c42e9980 msgr2=0x5581c42f16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:13.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.921+0000 7f3f80103740 1 --2- 172.21.15.116:0/1436956917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581c42e9980 0x5581c42f16c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00e6e0 tx=0x7f3f6c00cf90 comp rx=0 tx=0).stop 2024-09-12T03:07:13.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.921+0000 7f3f80103740 1 -- 172.21.15.116:0/1436956917 shutdown_connections 2024-09-12T03:07:13.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.921+0000 7f3f80103740 1 --2- 172.21.15.116:0/1436956917 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5581c4303db0 0x5581c4306220 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.921+0000 7f3f80103740 1 --2- 172.21.15.116:0/1436956917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581c42f4240 0x5581c42f1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.921+0000 7f3f80103740 1 --2- 172.21.15.116:0/1436956917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581c42e9980 0x5581c42f16c0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.921+0000 7f3f56ffd640 0 client.16314 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:13.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.921+0000 7f3f80103740 1 --2- 172.21.15.116:0/1436956917 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581c42e9060 0x5581c42f1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.921+0000 7f3f80103740 1 -- 172.21.15.116:0/1436956917 >> 172.21.15.116:0/1436956917 conn(0x5581c42e55f0 msgr2=0x5581c42eb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:13.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.921+0000 7f3f80103740 1 -- 172.21.15.116:0/1436956917 shutdown_connections 2024-09-12T03:07:13.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.921+0000 7f3f80103740 1 -- 172.21.15.116:0/1436956917 wait complete. 2024-09-12T03:07:13.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.921+0000 7f3f80103740 20 client.16314 trim_cache size 0 max 16384 2024-09-12T03:07:13.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.938+0000 7f3f80103740 1 Processor -- start 2024-09-12T03:07:13.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.938+0000 7f3f80103740 1 -- start start 2024-09-12T03:07:13.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.939+0000 7f3f80103740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581c42e9060 0x5581c42eb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:13.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.939+0000 7f3f80103740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581c42e9980 0x5581c430bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:13.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.939+0000 7f3f80103740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581c42f4240 0x5581c430c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:13.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.939+0000 7f3f80103740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3f781afd00 con 0x5581c42e9980 2024-09-12T03:07:13.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.939+0000 7f3f80103740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3f781ad5d0 con 0x5581c42f4240 2024-09-12T03:07:13.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.939+0000 7f3f80103740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3f781ab420 con 0x5581c42e9060 2024-09-12T03:07:13.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.939+0000 7f3f76ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581c42e9980 0x5581c430bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:13.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.939+0000 7f3f76ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581c42e9980 0x5581c430bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46634/0 (socket says 172.21.15.116:46634) 2024-09-12T03:07:13.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.939+0000 7f3f76ffd640 1 -- 172.21.15.116:0/2059198848 learned_addr learned my addr 172.21.15.116:0/2059198848 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:13.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.939+0000 7f3f77fff640 1 --2- 172.21.15.116:0/2059198848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581c42e9060 0x5581c42eb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:13.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.939+0000 7f3f77fff640 1 --2- 172.21.15.116:0/2059198848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581c42f4240 0x5581c430c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:13.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.939+0000 7f3f76ffd640 1 -- 172.21.15.116:0/2059198848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581c42e9060 msgr2=0x5581c42eb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:13.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.939+0000 7f3f76ffd640 1 --2- 172.21.15.116:0/2059198848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581c42e9060 0x5581c42eb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.939+0000 7f3f76ffd640 1 -- 172.21.15.116:0/2059198848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581c42f4240 msgr2=0x5581c430c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:13.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.939+0000 7f3f76ffd640 1 --2- 172.21.15.116:0/2059198848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581c42f4240 0x5581c430c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.939+0000 7f3f76ffd640 1 -- 172.21.15.116:0/2059198848 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581c42f0120 con 0x5581c42e9980 2024-09-12T03:07:13.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.939+0000 7f3f77fff640 1 --2- 172.21.15.116:0/2059198848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581c42f4240 0x5581c430c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:07:13.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.939+0000 7f3f76ffd640 1 --2- 172.21.15.116:0/2059198848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581c42e9980 0x5581c430bb30 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c001eb0 tx=0x7f3f6c005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381f1c5d2482c865 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:13.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.939+0000 7f3f5d7fa640 1 -- 172.21.15.116:0/2059198848 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c027070 con 0x5581c42e9980 2024-09-12T03:07:13.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.940+0000 7f3f5d7fa640 1 -- 172.21.15.116:0/2059198848 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f6c009e20 con 0x5581c42e9980 2024-09-12T03:07:13.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.940+0000 7f3f5d7fa640 1 -- 172.21.15.116:0/2059198848 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0176a0 con 0x5581c42e9980 2024-09-12T03:07:13.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.940+0000 7f3f80103740 1 -- 172.21.15.116:0/2059198848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581c42e9980 msgr2=0x5581c430bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:13.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.940+0000 7f3f80103740 1 --2- 172.21.15.116:0/2059198848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581c42e9980 0x5581c430bb30 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c001eb0 tx=0x7f3f6c005c40 comp rx=0 tx=0).stop 2024-09-12T03:07:13.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.940+0000 7f3f80103740 1 -- 172.21.15.116:0/2059198848 shutdown_connections 2024-09-12T03:07:13.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.940+0000 7f3f80103740 1 --2- 172.21.15.116:0/2059198848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581c42f4240 0x5581c430c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.940+0000 7f3f80103740 1 --2- 172.21.15.116:0/2059198848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581c42e9980 0x5581c430bb30 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.940+0000 7f3f80103740 1 --2- 172.21.15.116:0/2059198848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581c42e9060 0x5581c42eb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.940+0000 7f3f80103740 1 -- 172.21.15.116:0/2059198848 >> 172.21.15.116:0/2059198848 conn(0x5581c42e55f0 msgr2=0x5581c42eaab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:13.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.940+0000 7f3f80103740 1 -- 172.21.15.116:0/2059198848 shutdown_connections 2024-09-12T03:07:13.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.940+0000 7f3f80103740 1 -- 172.21.15.116:0/2059198848 wait complete. 2024-09-12T03:07:13.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.941+0000 7f3f80103740 1 -- start start 2024-09-12T03:07:13.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.941+0000 7f3f80103740 1 Processor -- start 2024-09-12T03:07:13.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.941+0000 7f3f80103740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581c42e9060 0x5581c42ee6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:13.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.941+0000 7f3f80103740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581c42e9980 0x5581c42eec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:13.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.941+0000 7f3f80103740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581c42f4240 0x5581c42ed450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:13.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.941+0000 7f3f80103740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3f781aba90 con 0x5581c42f4240 2024-09-12T03:07:13.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.941+0000 7f3f80103740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3f781acf00 con 0x5581c42e9060 2024-09-12T03:07:13.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.941+0000 7f3f80103740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3f781ac160 con 0x5581c42e9980 2024-09-12T03:07:13.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.941+0000 7f3f80103740 10 client.0 _ll_register_callbacks cb 0x7f3f6c00ce50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:07:13.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.941+0000 7f3f77fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581c42f4240 0x5581c42ed450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:13.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.942+0000 7f3f77fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581c42f4240 0x5581c42ed450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46640/0 (socket says 172.21.15.116:46640) 2024-09-12T03:07:13.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.942+0000 7f3f77fff640 1 -- 172.21.15.116:0/3067050625 learned_addr learned my addr 172.21.15.116:0/3067050625 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:13.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.942+0000 7f3f76ffd640 1 --2- 172.21.15.116:0/3067050625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581c42e9980 0x5581c42eec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:13.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.942+0000 7f3f77fff640 1 --2- 172.21.15.116:0/3067050625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581c42e9060 0x5581c42ee6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:13.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.942+0000 7f3f77fff640 1 -- 172.21.15.116:0/3067050625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581c42e9980 msgr2=0x5581c42eec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:13.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.942+0000 7f3f77fff640 1 --2- 172.21.15.116:0/3067050625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581c42e9980 0x5581c42eec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.942+0000 7f3f77fff640 1 -- 172.21.15.116:0/3067050625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581c42e9060 msgr2=0x5581c42ee6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:13.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.942+0000 7f3f77fff640 1 --2- 172.21.15.116:0/3067050625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581c42e9060 0x5581c42ee6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.942+0000 7f3f77fff640 1 -- 172.21.15.116:0/3067050625 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581c42edb90 con 0x5581c42f4240 2024-09-12T03:07:13.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.942+0000 7f3f77fff640 1 --2- 172.21.15.116:0/3067050625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581c42f4240 0x5581c42ed450 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3f7002fd90 tx=0x7f3f6c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765095a06fde2380 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:13.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.942+0000 7f3f56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:07:13.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.942+0000 7f3f56ffd640 1 -- 172.21.15.116:0/3067050625 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70048040 con 0x5581c42f4240 2024-09-12T03:07:13.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.942+0000 7f3f56ffd640 1 -- 172.21.15.116:0/3067050625 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f3f7002f460 con 0x5581c42f4240 2024-09-12T03:07:13.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.942+0000 7f3f80103740 1 -- 172.21.15.116:0/3067050625 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f6c009e20 con 0x5581c42f4240 2024-09-12T03:07:13.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.942+0000 7f3f56ffd640 1 -- 172.21.15.116:0/3067050625 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3f7002f660 con 0x5581c42f4240 2024-09-12T03:07:13.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.942+0000 7f3f80103740 10 client.16320 fetch_fsmap learned FSMap version 25 2024-09-12T03:07:13.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.942+0000 7f3f80103740 1 -- 172.21.15.116:0/3067050625 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5581c43734d0 con 0x5581c42f4240 2024-09-12T03:07:13.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.943+0000 7f3f56ffd640 1 -- 172.21.15.116:0/3067050625 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f6c009e20 con 0x5581c42f4240 2024-09-12T03:07:13.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.943+0000 7f3f56ffd640 1 -- 172.21.15.116:0/3067050625 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3f7004e070 con 0x5581c42f4240 2024-09-12T03:07:13.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.943+0000 7f3f80103740 10 client.16320 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:07:13.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.943+0000 7f3f80103740 10 client.16320 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:07:13.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.943+0000 7f3f80103740 20 client.16320 populate_metadata read hostname 'smithi116' 2024-09-12T03:07:13.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.943+0000 7f3f80103740 1 --2- 172.21.15.116:0/3067050625 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5581c43737c0 0x5581c4375bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:13.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.943+0000 7f3f80103740 4 client.16320 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "34c5c87e", "release": true, "await": true}] 2024-09-12T03:07:13.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.943+0000 7f3f80103740 1 -- 172.21.15.116:0/3067050625 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "quiesce db", "set_id": "34c5c87e", "release": true, "await": true}) -- 0x5581c42560a0 con 0x5581c43737c0 2024-09-12T03:07:13.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.943+0000 7f3f76ffd640 1 --2- 172.21.15.116:0/3067050625 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5581c43737c0 0x5581c4375bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:13.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.944+0000 7f3f76ffd640 1 --2- 172.21.15.116:0/3067050625 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5581c43737c0 0x5581c4375bb0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2f77265bb13b631 server_cookie=9e89e457e3b7779a in_seq=0 out_seq=0 2024-09-12T03:07:13.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.944+0000 7f3f56ffd640 10 client.16320 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:13.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.946+0000 7f3f56ffd640 1 -- 172.21.15.116:0/3067050625 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5581c42560a0 con 0x5581c43737c0 2024-09-12T03:07:13.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.946+0000 7f3f56ffd640 10 client.16320 handle_command_reply: tid=0 2024-09-12T03:07:13.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.946+0000 7f3f80103740 1 client.16320 shutdown 2024-09-12T03:07:13.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.946+0000 7f3f80103740 1 -- 172.21.15.116:0/3067050625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581c42f4240 msgr2=0x5581c42ed450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:13.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.946+0000 7f3f80103740 1 --2- 172.21.15.116:0/3067050625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581c42f4240 0x5581c42ed450 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3f7002fd90 tx=0x7f3f6c00cbb0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.946+0000 7f3f80103740 1 -- 172.21.15.116:0/3067050625 shutdown_connections 2024-09-12T03:07:13.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.946+0000 7f3f80103740 1 --2- 172.21.15.116:0/3067050625 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5581c43737c0 0x5581c4375bb0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.947+0000 7f3f80103740 1 --2- 172.21.15.116:0/3067050625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581c42f4240 0x5581c42ed450 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.947+0000 7f3f56ffd640 0 client.16320 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:13.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.947+0000 7f3f80103740 1 --2- 172.21.15.116:0/3067050625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581c42e9980 0x5581c42eec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.947+0000 7f3f80103740 1 --2- 172.21.15.116:0/3067050625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5581c42e9060 0x5581c42ee6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.947+0000 7f3f80103740 1 -- 172.21.15.116:0/3067050625 >> 172.21.15.116:0/3067050625 conn(0x5581c42e55f0 msgr2=0x5581c42f1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:13.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.947+0000 7f3f80103740 1 -- 172.21.15.116:0/3067050625 shutdown_connections 2024-09-12T03:07:13.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.947+0000 7f3f80103740 1 -- 172.21.15.116:0/3067050625 wait complete. 2024-09-12T03:07:13.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.947+0000 7f3f80103740 20 client.16320 trim_cache size 0 max 16384 2024-09-12T03:07:13.962 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:07:13.962 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 25, 2024-09-12T03:07:13.962 INFO:teuthology.orchestra.run.smithi116.stdout: "leader": 24494, 2024-09-12T03:07:13.962 INFO:teuthology.orchestra.run.smithi116.stdout: "set_version": 16, 2024-09-12T03:07:13.963 INFO:teuthology.orchestra.run.smithi116.stdout: "sets": { 2024-09-12T03:07:13.963 INFO:teuthology.orchestra.run.smithi116.stdout: "34c5c87e": { 2024-09-12T03:07:13.963 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 16, 2024-09-12T03:07:13.963 INFO:teuthology.orchestra.run.smithi116.stdout: "age_ref": 0.0, 2024-09-12T03:07:13.963 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:07:13.963 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RELEASED", 2024-09-12T03:07:13.964 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:07:13.964 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:07:13.964 INFO:teuthology.orchestra.run.smithi116.stdout: "timeout": 90.0, 2024-09-12T03:07:13.964 INFO:teuthology.orchestra.run.smithi116.stdout: "expiration": 140.2, 2024-09-12T03:07:13.964 INFO:teuthology.orchestra.run.smithi116.stdout: "members": { 2024-09-12T03:07:13.964 INFO:teuthology.orchestra.run.smithi116.stdout: "file:/": { 2024-09-12T03:07:13.965 INFO:teuthology.orchestra.run.smithi116.stdout: "excluded": false, 2024-09-12T03:07:13.965 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:07:13.965 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RELEASED", 2024-09-12T03:07:13.965 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:07:13.965 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:07:13.966 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:07:13.966 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:07:13.966 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:07:13.966 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:07:13.966 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:07:13.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.948+0000 7f3f7dd2a640 1 -- 172.21.15.116:0/3255435121 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3f600a2b80 msgr2=0x7f3f600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:13.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.948+0000 7f3f7dd2a640 1 --2- 172.21.15.116:0/3255435121 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3f600a2b80 0x7f3f600a5040 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3f680cc8e0 tx=0x7f3f680048a0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.948+0000 7f3f7dd2a640 1 -- 172.21.15.116:0/3255435121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f78102dc0 msgr2=0x7f3f78198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:13.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.948+0000 7f3f7dd2a640 1 --2- 172.21.15.116:0/3255435121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f78102dc0 0x7f3f78198450 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f3f6800d770 tx=0x7f3f6800dc30 comp rx=0 tx=0).stop 2024-09-12T03:07:13.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.948+0000 7f3f7dd2a640 1 -- 172.21.15.116:0/3255435121 shutdown_connections 2024-09-12T03:07:13.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.948+0000 7f3f7dd2a640 1 --2- 172.21.15.116:0/3255435121 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3f600a2b80 0x7f3f600a5040 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.948+0000 7f3f7dd2a640 1 --2- 172.21.15.116:0/3255435121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f7810c1a0 0x7f3f7819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.948+0000 7f3f7dd2a640 1 --2- 172.21.15.116:0/3255435121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f781036e0 0x7f3f78198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.948+0000 7f3f7dd2a640 1 --2- 172.21.15.116:0/3255435121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f78102dc0 0x7f3f78198450 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:13.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.948+0000 7f3f7dd2a640 1 -- 172.21.15.116:0/3255435121 >> 172.21.15.116:0/3255435121 conn(0x7f3f780fc7a0 msgr2=0x7f3f780fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:13.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.948+0000 7f3f7dd2a640 1 -- 172.21.15.116:0/3255435121 shutdown_connections 2024-09-12T03:07:13.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:13.948+0000 7f3f7dd2a640 1 -- 172.21.15.116:0/3255435121 wait complete. 2024-09-12T03:07:14.194 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:07:14.546 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:14 smithi177 ceph-mon[35305]: pgmap v465: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-12T03:07:14.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:14 smithi116 ceph-mon[31410]: pgmap v465: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-12T03:07:14.663 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '34c5c87e', total seconds elapsed: 24.3 2024-09-12T03:07:14.663 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.15 2024-09-12T03:07:14.663 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:07:14.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:14 smithi183 ceph-mon[35535]: pgmap v465: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-12T03:07:15.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.122+0000 7fe125341640 1 -- 172.21.15.116:0/903263570 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe120102d30 msgr2=0x7fe120103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:15.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.122+0000 7fe125341640 1 --2- 172.21.15.116:0/903263570 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe120102d30 0x7fe120103110 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe110009930 tx=0x7fe11002a420 comp rx=0 tx=0).stop 2024-09-12T03:07:15.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.122+0000 7fe125341640 1 -- 172.21.15.116:0/903263570 shutdown_connections 2024-09-12T03:07:15.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.122+0000 7fe125341640 1 --2- 172.21.15.116:0/903263570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1201082a0 0x7fe12010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:15.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.122+0000 7fe125341640 1 --2- 172.21.15.116:0/903263570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1201036e0 0x7fe120107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:15.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.122+0000 7fe125341640 1 --2- 172.21.15.116:0/903263570 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe120102d30 0x7fe120103110 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:15.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.122+0000 7fe125341640 1 -- 172.21.15.116:0/903263570 >> 172.21.15.116:0/903263570 conn(0x7fe1200fc7d0 msgr2=0x7fe1200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:15.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.122+0000 7fe125341640 1 -- 172.21.15.116:0/903263570 shutdown_connections 2024-09-12T03:07:15.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.122+0000 7fe125341640 1 -- 172.21.15.116:0/903263570 wait complete. 2024-09-12T03:07:15.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.123+0000 7fe125341640 1 Processor -- start 2024-09-12T03:07:15.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.123+0000 7fe125341640 1 -- start start 2024-09-12T03:07:15.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.123+0000 7fe125341640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe120102d30 0x7fe1201985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:15.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.123+0000 7fe125341640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1201036e0 0x7fe120198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:15.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.123+0000 7fe125341640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1201082a0 0x7fe12019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:15.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.123+0000 7fe125341640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe1201120a0 con 0x7fe1201082a0 2024-09-12T03:07:15.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.123+0000 7fe125341640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe120111f20 con 0x7fe120102d30 2024-09-12T03:07:15.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.123+0000 7fe125341640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe120112220 con 0x7fe1201036e0 2024-09-12T03:07:15.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.123+0000 7fe11f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1201082a0 0x7fe12019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:15.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.123+0000 7fe11f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1201082a0 0x7fe12019ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46668/0 (socket says 172.21.15.116:46668) 2024-09-12T03:07:15.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.123+0000 7fe11f7fe640 1 -- 172.21.15.116:0/554934347 learned_addr learned my addr 172.21.15.116:0/554934347 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:15.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.124+0000 7fe11e7fc640 1 --2- 172.21.15.116:0/554934347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1201036e0 0x7fe120198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:15.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.124+0000 7fe11f7fe640 1 -- 172.21.15.116:0/554934347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1201036e0 msgr2=0x7fe120198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:15.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.124+0000 7fe11f7fe640 1 --2- 172.21.15.116:0/554934347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1201036e0 0x7fe120198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:15.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.124+0000 7fe11f7fe640 1 -- 172.21.15.116:0/554934347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe120102d30 msgr2=0x7fe1201985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:15.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.124+0000 7fe11f7fe640 1 --2- 172.21.15.116:0/554934347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe120102d30 0x7fe1201985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:15.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.124+0000 7fe11f7fe640 1 -- 172.21.15.116:0/554934347 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe12019d570 con 0x7fe1201082a0 2024-09-12T03:07:15.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.124+0000 7fe11f7fe640 1 --2- 172.21.15.116:0/554934347 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1201082a0 0x7fe12019ce90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe11800eb10 tx=0x7fe11800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:15.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.124+0000 7fe107fff640 1 -- 172.21.15.116:0/554934347 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe118004b20 con 0x7fe1201082a0 2024-09-12T03:07:15.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.124+0000 7fe125341640 1 -- 172.21.15.116:0/554934347 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe12019d860 con 0x7fe1201082a0 2024-09-12T03:07:15.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.124+0000 7fe107fff640 1 -- 172.21.15.116:0/554934347 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe118004cc0 con 0x7fe1201082a0 2024-09-12T03:07:15.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.124+0000 7fe107fff640 1 -- 172.21.15.116:0/554934347 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe118010760 con 0x7fe1201082a0 2024-09-12T03:07:15.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.124+0000 7fe125341640 1 -- 172.21.15.116:0/554934347 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1201a5140 con 0x7fe1201082a0 2024-09-12T03:07:15.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.125+0000 7fe125341640 1 -- 172.21.15.116:0/554934347 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe12006a850 con 0x7fe1201082a0 2024-09-12T03:07:15.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.126+0000 7fe107fff640 1 -- 172.21.15.116:0/554934347 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe118005020 con 0x7fe1201082a0 2024-09-12T03:07:15.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.127+0000 7fe107fff640 1 --2- 172.21.15.116:0/554934347 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe1080a2b80 0x7fe1080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:15.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.127+0000 7fe11effd640 1 --2- 172.21.15.116:0/554934347 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe1080a2b80 0x7fe1080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:15.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.127+0000 7fe107fff640 1 -- 172.21.15.116:0/554934347 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe1180cf7b0 con 0x7fe1201082a0 2024-09-12T03:07:15.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.127+0000 7fe11effd640 1 --2- 172.21.15.116:0/554934347 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe1080a2b80 0x7fe1080a5040 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe11002a940 tx=0x7fe110000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:15.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.131+0000 7fe107fff640 1 -- 172.21.15.116:0/554934347 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe11809c740 con 0x7fe1201082a0 2024-09-12T03:07:15.212 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:07:15.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.341+0000 7fe125341640 1 -- 172.21.15.116:0/554934347 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe12010a2b0 con 0x7fe1201082a0 2024-09-12T03:07:15.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.341+0000 7fe107fff640 1 -- 172.21.15.116:0/554934347 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7fe11809c040 con 0x7fe1201082a0 2024-09-12T03:07:15.342 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:07:15.342 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:07:15.343 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:07:15.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.344+0000 7fe125341640 1 -- 172.21.15.116:0/554934347 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe1080a2b80 msgr2=0x7fe1080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:15.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.344+0000 7fe125341640 1 --2- 172.21.15.116:0/554934347 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe1080a2b80 0x7fe1080a5040 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe11002a940 tx=0x7fe110000bc0 comp rx=0 tx=0).stop 2024-09-12T03:07:15.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.344+0000 7fe125341640 1 -- 172.21.15.116:0/554934347 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1201082a0 msgr2=0x7fe12019ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:15.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.344+0000 7fe125341640 1 --2- 172.21.15.116:0/554934347 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1201082a0 0x7fe12019ce90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe11800eb10 tx=0x7fe11800c2d0 comp rx=0 tx=0).stop 2024-09-12T03:07:15.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.344+0000 7fe125341640 1 -- 172.21.15.116:0/554934347 shutdown_connections 2024-09-12T03:07:15.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.344+0000 7fe125341640 1 --2- 172.21.15.116:0/554934347 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe1080a2b80 0x7fe1080a5040 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:15.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.344+0000 7fe125341640 1 --2- 172.21.15.116:0/554934347 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1201082a0 0x7fe12019ce90 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:15.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.344+0000 7fe125341640 1 --2- 172.21.15.116:0/554934347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1201036e0 0x7fe120198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:15.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.344+0000 7fe125341640 1 --2- 172.21.15.116:0/554934347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe120102d30 0x7fe1201985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:15.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.344+0000 7fe125341640 1 -- 172.21.15.116:0/554934347 >> 172.21.15.116:0/554934347 conn(0x7fe1200fc7d0 msgr2=0x7fe120105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:15.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.344+0000 7fe125341640 1 -- 172.21.15.116:0/554934347 shutdown_connections 2024-09-12T03:07:15.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:15.344+0000 7fe125341640 1 -- 172.21.15.116:0/554934347 wait complete. 2024-09-12T03:07:15.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:07:15.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:07:15.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:07:15.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:07:15.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:15 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:07:15.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:15 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:07:15.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:15 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:07:15.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:15 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:07:15.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:15 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:07:15.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:15 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:07:15.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:15 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:07:15.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:15 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:07:16.037 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:07:16.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.439+0000 7f859783b640 1 -- 172.21.15.116:0/1712596598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8590100a80 msgr2=0x7f8590100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:16.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.439+0000 7f859783b640 1 --2- 172.21.15.116:0/1712596598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8590100a80 0x7f8590100ee0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8580009930 tx=0x7f858002a420 comp rx=0 tx=0).stop 2024-09-12T03:07:16.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.439+0000 7f859783b640 1 -- 172.21.15.116:0/1712596598 shutdown_connections 2024-09-12T03:07:16.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.439+0000 7f859783b640 1 --2- 172.21.15.116:0/1712596598 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8590101420 0x7f859010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:16.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.439+0000 7f859783b640 1 --2- 172.21.15.116:0/1712596598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8590100a80 0x7f8590100ee0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:16.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.439+0000 7f859783b640 1 --2- 172.21.15.116:0/1712596598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8590106a80 0x7f8590106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:16.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.439+0000 7f859783b640 1 -- 172.21.15.116:0/1712596598 >> 172.21.15.116:0/1712596598 conn(0x7f85900fc7a0 msgr2=0x7f85900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:16.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.440+0000 7f859783b640 1 -- 172.21.15.116:0/1712596598 shutdown_connections 2024-09-12T03:07:16.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.440+0000 7f859783b640 1 -- 172.21.15.116:0/1712596598 wait complete. 2024-09-12T03:07:16.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.440+0000 7f859783b640 1 Processor -- start 2024-09-12T03:07:16.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.440+0000 7f859783b640 1 -- start start 2024-09-12T03:07:16.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.440+0000 7f859783b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8590100a80 0x7f8590199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:16.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.440+0000 7f859783b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8590101420 0x7f8590199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:16.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.440+0000 7f859783b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8590106a80 0x7f85901a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:16.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.440+0000 7f859783b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8590110e80 con 0x7f8590101420 2024-09-12T03:07:16.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.440+0000 7f859783b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8590110d00 con 0x7f8590100a80 2024-09-12T03:07:16.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.440+0000 7f859783b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8590111000 con 0x7f8590106a80 2024-09-12T03:07:16.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.441+0000 7f8594dd6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8590101420 0x7f8590199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:16.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.441+0000 7f8594dd6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8590101420 0x7f8590199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46684/0 (socket says 172.21.15.116:46684) 2024-09-12T03:07:16.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.441+0000 7f8594dd6640 1 -- 172.21.15.116:0/1478061169 learned_addr learned my addr 172.21.15.116:0/1478061169 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:16.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.441+0000 7f8595dd8640 1 --2- 172.21.15.116:0/1478061169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8590106a80 0x7f85901a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:16.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.441+0000 7f85955d7640 1 --2- 172.21.15.116:0/1478061169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8590100a80 0x7f8590199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:16.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.441+0000 7f8594dd6640 1 -- 172.21.15.116:0/1478061169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8590106a80 msgr2=0x7f85901a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:16.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.441+0000 7f8594dd6640 1 --2- 172.21.15.116:0/1478061169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8590106a80 0x7f85901a43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:16.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.441+0000 7f8594dd6640 1 -- 172.21.15.116:0/1478061169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8590100a80 msgr2=0x7f8590199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:16.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.441+0000 7f8594dd6640 1 --2- 172.21.15.116:0/1478061169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8590100a80 0x7f8590199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:16.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.441+0000 7f8594dd6640 1 -- 172.21.15.116:0/1478061169 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85901a48f0 con 0x7f8590101420 2024-09-12T03:07:16.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.441+0000 7f8595dd8640 1 --2- 172.21.15.116:0/1478061169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8590106a80 0x7f85901a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:07:16.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.441+0000 7f85955d7640 1 --2- 172.21.15.116:0/1478061169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8590100a80 0x7f8590199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:07:16.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.441+0000 7f8594dd6640 1 --2- 172.21.15.116:0/1478061169 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8590101420 0x7f8590199870 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f8580009230 tx=0x7f8580004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:16.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.441+0000 7f85867fc640 1 -- 172.21.15.116:0/1478061169 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858003f030 con 0x7f8590101420 2024-09-12T03:07:16.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.441+0000 7f85867fc640 1 -- 172.21.15.116:0/1478061169 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8580003ad0 con 0x7f8590101420 2024-09-12T03:07:16.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.441+0000 7f859783b640 1 -- 172.21.15.116:0/1478061169 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85901a4ac0 con 0x7f8590101420 2024-09-12T03:07:16.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.441+0000 7f85867fc640 1 -- 172.21.15.116:0/1478061169 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8580035070 con 0x7f8590101420 2024-09-12T03:07:16.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.441+0000 7f859783b640 1 -- 172.21.15.116:0/1478061169 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85901a4f10 con 0x7f8590101420 2024-09-12T03:07:16.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.442+0000 7f859783b640 1 -- 172.21.15.116:0/1478061169 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8590100ee0 con 0x7f8590101420 2024-09-12T03:07:16.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.443+0000 7f85867fc640 1 -- 172.21.15.116:0/1478061169 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8580042030 con 0x7f8590101420 2024-09-12T03:07:16.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.444+0000 7f85867fc640 1 --2- 172.21.15.116:0/1478061169 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f857c0a2b80 0x7f857c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:16.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.444+0000 7f85955d7640 1 --2- 172.21.15.116:0/1478061169 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f857c0a2b80 0x7f857c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:16.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.445+0000 7f85867fc640 1 -- 172.21.15.116:0/1478061169 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f85800efa00 con 0x7f8590101420 2024-09-12T03:07:16.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.445+0000 7f85955d7640 1 --2- 172.21.15.116:0/1478061169 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f857c0a2b80 0x7f857c0a5040 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f858c005fd0 tx=0x7f858c00b5d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:16.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.448+0000 7f85867fc640 1 -- 172.21.15.116:0/1478061169 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f85800bc990 con 0x7f8590101420 2024-09-12T03:07:16.517 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:07:16.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:16 smithi116 ceph-mon[31410]: pgmap v466: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-12T03:07:16.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:16 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/554934347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:07:16.604 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:16 smithi183 ceph-mon[35535]: pgmap v466: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-12T03:07:16.604 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:16 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/554934347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:07:16.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.660+0000 7f859783b640 1 -- 172.21.15.116:0/1478061169 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f859006ad80 con 0x7f8590101420 2024-09-12T03:07:16.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.660+0000 7f85867fc640 1 -- 172.21.15.116:0/1478061169 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f859006ad80 con 0x7f8590101420 2024-09-12T03:07:16.661 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:07:16.661 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:07:16.661 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:07:16.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.663+0000 7f859783b640 1 -- 172.21.15.116:0/1478061169 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f857c0a2b80 msgr2=0x7f857c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:16.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.663+0000 7f859783b640 1 --2- 172.21.15.116:0/1478061169 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f857c0a2b80 0x7f857c0a5040 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f858c005fd0 tx=0x7f858c00b5d0 comp rx=0 tx=0).stop 2024-09-12T03:07:16.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.663+0000 7f859783b640 1 -- 172.21.15.116:0/1478061169 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8590101420 msgr2=0x7f8590199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:16.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.663+0000 7f859783b640 1 --2- 172.21.15.116:0/1478061169 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8590101420 0x7f8590199870 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f8580009230 tx=0x7f8580004270 comp rx=0 tx=0).stop 2024-09-12T03:07:16.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.663+0000 7f859783b640 1 -- 172.21.15.116:0/1478061169 shutdown_connections 2024-09-12T03:07:16.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.663+0000 7f859783b640 1 --2- 172.21.15.116:0/1478061169 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f857c0a2b80 0x7f857c0a5040 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:16.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.663+0000 7f859783b640 1 --2- 172.21.15.116:0/1478061169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8590106a80 0x7f85901a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:16.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.663+0000 7f859783b640 1 --2- 172.21.15.116:0/1478061169 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8590101420 0x7f8590199870 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:16.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.663+0000 7f859783b640 1 --2- 172.21.15.116:0/1478061169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8590100a80 0x7f8590199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:16.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.663+0000 7f859783b640 1 -- 172.21.15.116:0/1478061169 >> 172.21.15.116:0/1478061169 conn(0x7f85900fc7a0 msgr2=0x7f85900fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:16.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.663+0000 7f859783b640 1 -- 172.21.15.116:0/1478061169 shutdown_connections 2024-09-12T03:07:16.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:16.663+0000 7f859783b640 1 -- 172.21.15.116:0/1478061169 wait complete. 2024-09-12T03:07:16.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:16 smithi177 ceph-mon[35305]: pgmap v466: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-12T03:07:16.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:16 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/554934347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:07:17.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:17 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1478061169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:07:17.596 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0' 2024-09-12T03:07:17.596 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0 2024-09-12T03:07:17.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:17 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1478061169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:07:17.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:17 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1478061169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:07:17.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.770+0000 7f623892a640 1 -- 172.21.15.116:0/3394342744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6234100e20 msgr2=0x7f6234101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:17.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.770+0000 7f623892a640 1 --2- 172.21.15.116:0/3394342744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6234100e20 0x7f6234101200 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f6224009900 tx=0x7f622402a480 comp rx=0 tx=0).stop 2024-09-12T03:07:17.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.770+0000 7f623892a640 1 -- 172.21.15.116:0/3394342744 shutdown_connections 2024-09-12T03:07:17.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.770+0000 7f623892a640 1 --2- 172.21.15.116:0/3394342744 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6234102ef0 0x7f623410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.770+0000 7f623892a640 1 --2- 172.21.15.116:0/3394342744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6234102570 0x7f62341029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.770+0000 7f623892a640 1 --2- 172.21.15.116:0/3394342744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6234100e20 0x7f6234101200 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.770+0000 7f623892a640 1 -- 172.21.15.116:0/3394342744 >> 172.21.15.116:0/3394342744 conn(0x7f62340fc7d0 msgr2=0x7f62340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:17.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.770+0000 7f623892a640 1 -- 172.21.15.116:0/3394342744 shutdown_connections 2024-09-12T03:07:17.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.770+0000 7f623892a640 1 -- 172.21.15.116:0/3394342744 wait complete. 2024-09-12T03:07:17.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.771+0000 7f623892a640 1 Processor -- start 2024-09-12T03:07:17.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.771+0000 7f623892a640 1 -- start start 2024-09-12T03:07:17.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.771+0000 7f623892a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6234100e20 0x7f62340fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:17.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.771+0000 7f623892a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6234102570 0x7f62340fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:17.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.771+0000 7f623892a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6234102ef0 0x7f62340fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:17.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.771+0000 7f623892a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6234110f80 con 0x7f6234102570 2024-09-12T03:07:17.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.771+0000 7f623892a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6234110e00 con 0x7f6234100e20 2024-09-12T03:07:17.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.771+0000 7f623892a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6234111100 con 0x7f6234102ef0 2024-09-12T03:07:17.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.771+0000 7f6229d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6234102570 0x7f62340fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:17.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.771+0000 7f6229d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6234102570 0x7f62340fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46708/0 (socket says 172.21.15.116:46708) 2024-09-12T03:07:17.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.771+0000 7f6229d9b640 1 -- 172.21.15.116:0/3646581645 learned_addr learned my addr 172.21.15.116:0/3646581645 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:17.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.772+0000 7f623259c640 1 --2- 172.21.15.116:0/3646581645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6234100e20 0x7f62340fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:17.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.772+0000 7f6232d9d640 1 --2- 172.21.15.116:0/3646581645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6234102ef0 0x7f62340fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:17.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.772+0000 7f6229d9b640 1 -- 172.21.15.116:0/3646581645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6234102ef0 msgr2=0x7f62340fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:17.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.772+0000 7f6229d9b640 1 --2- 172.21.15.116:0/3646581645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6234102ef0 0x7f62340fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.772+0000 7f6229d9b640 1 -- 172.21.15.116:0/3646581645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6234100e20 msgr2=0x7f62340fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:17.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.772+0000 7f6229d9b640 1 --2- 172.21.15.116:0/3646581645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6234100e20 0x7f62340fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.772+0000 7f6229d9b640 1 -- 172.21.15.116:0/3646581645 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62341a69f0 con 0x7f6234102570 2024-09-12T03:07:17.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.772+0000 7f6229d9b640 1 --2- 172.21.15.116:0/3646581645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6234102570 0x7f62340fd1c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f621c00d770 tx=0x7f621c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:17.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.772+0000 7f622bfff640 1 -- 172.21.15.116:0/3646581645 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c017070 con 0x7f6234102570 2024-09-12T03:07:17.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.772+0000 7f623892a640 1 -- 172.21.15.116:0/3646581645 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62341a6ce0 con 0x7f6234102570 2024-09-12T03:07:17.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.772+0000 7f622bfff640 1 -- 172.21.15.116:0/3646581645 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f621c005500 con 0x7f6234102570 2024-09-12T03:07:17.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.772+0000 7f622bfff640 1 -- 172.21.15.116:0/3646581645 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c004e00 con 0x7f6234102570 2024-09-12T03:07:17.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.772+0000 7f623892a640 1 -- 172.21.15.116:0/3646581645 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62341a71c0 con 0x7f6234102570 2024-09-12T03:07:17.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.775+0000 7f623ad03740 1 Processor -- start 2024-09-12T03:07:17.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.775+0000 7f623ad03740 1 -- start start 2024-09-12T03:07:17.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.775+0000 7f623ad03740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d510a6b40 0x555d510a5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:17.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.775+0000 7f623ad03740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d510a55d0 0x555d510a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:17.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.775+0000 7f623ad03740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d510b0370 0x555d510b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:17.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.775+0000 7f623ad03740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555d5100b2f0 con 0x555d510a55d0 2024-09-12T03:07:17.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.775+0000 7f623ad03740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x555d50fb2cf0 con 0x555d510b0370 2024-09-12T03:07:17.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.775+0000 7f623ad03740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555d510072b0 con 0x555d510a6b40 2024-09-12T03:07:17.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.776+0000 7f623259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d510a55d0 0x555d510a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:17.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.776+0000 7f622bfff640 1 -- 172.21.15.116:0/3646581645 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f621c004fa0 con 0x7f6234102570 2024-09-12T03:07:17.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.776+0000 7f623259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d510a55d0 0x555d510a5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46712/0 (socket says 172.21.15.116:46712) 2024-09-12T03:07:17.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.776+0000 7f623259c640 1 -- 172.21.15.116:0/4184263052 learned_addr learned my addr 172.21.15.116:0/4184263052 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:17.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.776+0000 7f6232d9d640 1 --2- 172.21.15.116:0/4184263052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d510a6b40 0x555d510a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:17.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.776+0000 7f623259c640 1 -- 172.21.15.116:0/4184263052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d510a6b40 msgr2=0x555d510a5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:17.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.776+0000 7f623259c640 1 --2- 172.21.15.116:0/4184263052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d510a6b40 0x555d510a5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.776+0000 7f623259c640 1 -- 172.21.15.116:0/4184263052 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d510b0370 msgr2=0x555d510b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:17.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.776+0000 7f623259c640 1 --2- 172.21.15.116:0/4184263052 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d510b0370 0x555d510b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.776+0000 7f623259c640 1 -- 172.21.15.116:0/4184263052 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d50f973a0 con 0x555d510a55d0 2024-09-12T03:07:17.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.776+0000 7f623259c640 1 --2- 172.21.15.116:0/4184263052 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d510a55d0 0x555d510a5a10 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f622402a9a0 tx=0x7f62240040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a4196f06768041 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:17.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.776+0000 7f6228d99640 1 -- 172.21.15.116:0/4184263052 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6224035030 con 0x555d510a55d0 2024-09-12T03:07:17.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.776+0000 7f6228d99640 1 -- 172.21.15.116:0/4184263052 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6224040070 con 0x555d510a55d0 2024-09-12T03:07:17.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.777+0000 7f622bfff640 1 --2- 172.21.15.116:0/3646581645 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f62200a2b80 0x7f62200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:17.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.777+0000 7f6228d99640 1 -- 172.21.15.116:0/4184263052 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6224031a70 con 0x555d510a55d0 2024-09-12T03:07:17.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.777+0000 7f6232d9d640 1 --2- 172.21.15.116:0/3646581645 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f62200a2b80 0x7f62200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:17.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.777+0000 7f622bfff640 1 -- 172.21.15.116:0/3646581645 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f621c0ce9e0 con 0x7f6234102570 2024-09-12T03:07:17.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.777+0000 7f623ad03740 1 -- 172.21.15.116:0/4184263052 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d510a55d0 msgr2=0x555d510a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:17.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.777+0000 7f623ad03740 1 --2- 172.21.15.116:0/4184263052 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d510a55d0 0x555d510a5a10 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f622402a9a0 tx=0x7f62240040c0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.777+0000 7f6232d9d640 1 --2- 172.21.15.116:0/3646581645 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f62200a2b80 0x7f62200a5040 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f622c005fd0 tx=0x7f622c009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:17.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.777+0000 7f623ad03740 1 -- 172.21.15.116:0/4184263052 shutdown_connections 2024-09-12T03:07:17.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.777+0000 7f623ad03740 1 --2- 172.21.15.116:0/4184263052 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d510b0370 0x555d510b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.777+0000 7f623ad03740 1 --2- 172.21.15.116:0/4184263052 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d510a55d0 0x555d510a5a10 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.777+0000 7f623ad03740 1 --2- 172.21.15.116:0/4184263052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d510a6b40 0x555d510a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.777+0000 7f623ad03740 1 -- 172.21.15.116:0/4184263052 >> 172.21.15.116:0/4184263052 conn(0x555d5106e310 msgr2=0x555d510a1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:17.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.777+0000 7f623ad03740 1 -- 172.21.15.116:0/4184263052 shutdown_connections 2024-09-12T03:07:17.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.777+0000 7f623ad03740 1 -- 172.21.15.116:0/4184263052 wait complete. 2024-09-12T03:07:17.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.778+0000 7f623ad03740 1 -- start start 2024-09-12T03:07:17.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.778+0000 7f623ad03740 1 Processor -- start 2024-09-12T03:07:17.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.778+0000 7f623ad03740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d510a6b40 0x555d510ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:17.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.778+0000 7f623ad03740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d510b0370 0x555d510ad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:17.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.778+0000 7f623ad03740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d510add30 0x555d510bc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:17.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.778+0000 7f623ad03740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555d510b30b0 con 0x555d510b0370 2024-09-12T03:07:17.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.778+0000 7f623ad03740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x555d50fc6060 con 0x555d510a6b40 2024-09-12T03:07:17.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.778+0000 7f623ad03740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555d510b3230 con 0x555d510add30 2024-09-12T03:07:17.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.778+0000 7f623ad03740 10 client.0 _ll_register_callbacks cb 0x555d50d198c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:07:17.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.778+0000 7f6229d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d510b0370 0x555d510ad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:17.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.778+0000 7f6229d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d510b0370 0x555d510ad7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46714/0 (socket says 172.21.15.116:46714) 2024-09-12T03:07:17.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.779+0000 7f6229d9b640 1 -- 172.21.15.116:0/474922974 learned_addr learned my addr 172.21.15.116:0/474922974 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:17.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.779+0000 7f6232d9d640 1 --2- 172.21.15.116:0/474922974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d510add30 0x555d510bc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:17.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.779+0000 7f623259c640 1 --2- 172.21.15.116:0/474922974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d510a6b40 0x555d510ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:17.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.779+0000 7f6229d9b640 1 -- 172.21.15.116:0/474922974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d510add30 msgr2=0x555d510bc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:17.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.779+0000 7f6229d9b640 1 --2- 172.21.15.116:0/474922974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d510add30 0x555d510bc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.779+0000 7f6229d9b640 1 -- 172.21.15.116:0/474922974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d510a6b40 msgr2=0x555d510ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:17.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.779+0000 7f6229d9b640 1 --2- 172.21.15.116:0/474922974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d510a6b40 0x555d510ad2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.779+0000 7f6229d9b640 1 -- 172.21.15.116:0/474922974 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d50d7c3c0 con 0x555d510b0370 2024-09-12T03:07:17.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.779+0000 7f6229d9b640 1 --2- 172.21.15.116:0/474922974 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d510b0370 0x555d510ad7f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f621c00d740 tx=0x7f621c09ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29502180f24f5ff7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:17.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.779+0000 7f6219ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:07:17.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.779+0000 7f6219ffb640 1 -- 172.21.15.116:0/474922974 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c020070 con 0x555d510b0370 2024-09-12T03:07:17.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.779+0000 7f6219ffb640 1 -- 172.21.15.116:0/474922974 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f621c009cc0 con 0x555d510b0370 2024-09-12T03:07:17.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.779+0000 7f623ad03740 1 -- 172.21.15.116:0/474922974 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6224040070 con 0x555d510b0370 2024-09-12T03:07:17.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.779+0000 7f6219ffb640 1 -- 172.21.15.116:0/474922974 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f621c0a3ee0 con 0x555d510b0370 2024-09-12T03:07:17.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.779+0000 7f623ad03740 10 client.16344 fetch_fsmap learned FSMap version 25 2024-09-12T03:07:17.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.779+0000 7f623ad03740 1 -- 172.21.15.116:0/474922974 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555d510bf1f0 con 0x555d510b0370 2024-09-12T03:07:17.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.780+0000 7f6219ffb640 1 -- 172.21.15.116:0/474922974 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6224040070 con 0x555d510b0370 2024-09-12T03:07:17.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.780+0000 7f6219ffb640 1 -- 172.21.15.116:0/474922974 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f621c0bb140 con 0x555d510b0370 2024-09-12T03:07:17.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.780+0000 7f623ad03740 10 client.16344 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:07:17.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.780+0000 7f623ad03740 10 client.16344 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:07:17.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.780+0000 7f623ad03740 20 client.16344 populate_metadata read hostname 'smithi116' 2024-09-12T03:07:17.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.780+0000 7f623ad03740 1 --2- 172.21.15.116:0/474922974 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x555d510c0540 0x555d510c2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:17.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.780+0000 7f623ad03740 4 client.16344 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:07:17.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.780+0000 7f623ad03740 1 -- 172.21.15.116:0/474922974 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d51018860 con 0x555d510c0540 2024-09-12T03:07:17.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.780+0000 7f623259c640 1 --2- 172.21.15.116:0/474922974 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x555d510c0540 0x555d510c2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:17.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.781+0000 7f623259c640 1 --2- 172.21.15.116:0/474922974 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x555d510c0540 0x555d510c2930 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fef603b3970605b2 server_cookie=1a88bd3d245d8926 in_seq=0 out_seq=0 2024-09-12T03:07:17.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.781+0000 7f6219ffb640 10 client.16344 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:07:17.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.781+0000 7f6219ffb640 1 -- 172.21.15.116:0/474922974 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555d51018860 con 0x555d510c0540 2024-09-12T03:07:17.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.781+0000 7f6219ffb640 10 client.16344 handle_command_reply: tid=0 2024-09-12T03:07:17.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.781+0000 7f623ad03740 1 client.16344 shutdown 2024-09-12T03:07:17.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.782+0000 7f623ad03740 1 -- 172.21.15.116:0/474922974 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d510b0370 msgr2=0x555d510ad7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:17.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.782+0000 7f623ad03740 1 --2- 172.21.15.116:0/474922974 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d510b0370 0x555d510ad7f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f621c00d740 tx=0x7f621c09ad40 comp rx=0 tx=0).stop 2024-09-12T03:07:17.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.782+0000 7f623ad03740 1 -- 172.21.15.116:0/474922974 shutdown_connections 2024-09-12T03:07:17.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.782+0000 7f623ad03740 1 --2- 172.21.15.116:0/474922974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d510add30 0x555d510bc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.782+0000 7f623ad03740 1 --2- 172.21.15.116:0/474922974 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x555d510c0540 0x555d510c2930 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.782+0000 7f6219ffb640 0 client.16344 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:07:17.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.782+0000 7f623ad03740 1 --2- 172.21.15.116:0/474922974 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d510b0370 0x555d510ad7f0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.782+0000 7f623ad03740 1 --2- 172.21.15.116:0/474922974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d510a6b40 0x555d510ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.782+0000 7f623ad03740 1 -- 172.21.15.116:0/474922974 >> 172.21.15.116:0/474922974 conn(0x555d5106e310 msgr2=0x555d510a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:17.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.782+0000 7f623ad03740 1 -- 172.21.15.116:0/474922974 shutdown_connections 2024-09-12T03:07:17.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.782+0000 7f623ad03740 1 -- 172.21.15.116:0/474922974 wait complete. 2024-09-12T03:07:17.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.782+0000 7f623ad03740 20 client.16344 trim_cache size 0 max 16384 2024-09-12T03:07:17.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.795+0000 7f623ad03740 1 Processor -- start 2024-09-12T03:07:17.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.795+0000 7f623ad03740 1 -- start start 2024-09-12T03:07:17.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.795+0000 7f623ad03740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d510a6b40 0x555d510a7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:17.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.795+0000 7f623ad03740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d510add30 0x555d510bc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:17.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.795+0000 7f623ad03740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d510b0370 0x555d510bc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:17.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.795+0000 7f623ad03740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f62341b1e30 con 0x555d510b0370 2024-09-12T03:07:17.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.795+0000 7f623ad03740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f62341af700 con 0x555d510add30 2024-09-12T03:07:17.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.795+0000 7f623ad03740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f62341ad570 con 0x555d510a6b40 2024-09-12T03:07:17.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.796+0000 7f6232d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d510b0370 0x555d510bc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:17.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.796+0000 7f6232d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d510b0370 0x555d510bc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46724/0 (socket says 172.21.15.116:46724) 2024-09-12T03:07:17.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.796+0000 7f6232d9d640 1 -- 172.21.15.116:0/2070655715 learned_addr learned my addr 172.21.15.116:0/2070655715 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:17.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.796+0000 7f6229d9b640 1 --2- 172.21.15.116:0/2070655715 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d510add30 0x555d510bc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:17.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.796+0000 7f623259c640 1 --2- 172.21.15.116:0/2070655715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d510a6b40 0x555d510a7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:17.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.796+0000 7f6232d9d640 1 -- 172.21.15.116:0/2070655715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d510a6b40 msgr2=0x555d510a7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:17.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.796+0000 7f6232d9d640 1 --2- 172.21.15.116:0/2070655715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d510a6b40 0x555d510a7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.796+0000 7f6232d9d640 1 -- 172.21.15.116:0/2070655715 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d510add30 msgr2=0x555d510bc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:17.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.796+0000 7f6232d9d640 1 --2- 172.21.15.116:0/2070655715 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d510add30 0x555d510bc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.796+0000 7f6232d9d640 1 -- 172.21.15.116:0/2070655715 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d510a14f0 con 0x555d510b0370 2024-09-12T03:07:17.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.796+0000 7f623259c640 1 --2- 172.21.15.116:0/2070655715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d510a6b40 0x555d510a7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:07:17.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.796+0000 7f6229d9b640 1 --2- 172.21.15.116:0/2070655715 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d510add30 0x555d510bc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:07:17.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.796+0000 7f6232d9d640 1 --2- 172.21.15.116:0/2070655715 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d510b0370 0x555d510bc8c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f622c01f5e0 tx=0x7f622c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eaba91055240a09 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:17.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.797+0000 7f621bfff640 1 -- 172.21.15.116:0/2070655715 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c004d20 con 0x555d510b0370 2024-09-12T03:07:17.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.797+0000 7f621bfff640 1 -- 172.21.15.116:0/2070655715 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f622c005500 con 0x555d510b0370 2024-09-12T03:07:17.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.797+0000 7f621bfff640 1 -- 172.21.15.116:0/2070655715 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c027990 con 0x555d510b0370 2024-09-12T03:07:17.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.797+0000 7f623ad03740 1 -- 172.21.15.116:0/2070655715 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d510b0370 msgr2=0x555d510bc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:17.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.797+0000 7f623ad03740 1 --2- 172.21.15.116:0/2070655715 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d510b0370 0x555d510bc8c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f622c01f5e0 tx=0x7f622c015000 comp rx=0 tx=0).stop 2024-09-12T03:07:17.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.797+0000 7f623ad03740 1 -- 172.21.15.116:0/2070655715 shutdown_connections 2024-09-12T03:07:17.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.797+0000 7f623ad03740 1 --2- 172.21.15.116:0/2070655715 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d510b0370 0x555d510bc8c0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.797+0000 7f623ad03740 1 --2- 172.21.15.116:0/2070655715 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d510add30 0x555d510bc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.797+0000 7f623ad03740 1 --2- 172.21.15.116:0/2070655715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d510a6b40 0x555d510a7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.797+0000 7f623ad03740 1 -- 172.21.15.116:0/2070655715 >> 172.21.15.116:0/2070655715 conn(0x555d5106e310 msgr2=0x555d510a7110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:17.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.797+0000 7f623ad03740 1 -- 172.21.15.116:0/2070655715 shutdown_connections 2024-09-12T03:07:17.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.798+0000 7f623ad03740 1 -- 172.21.15.116:0/2070655715 wait complete. 2024-09-12T03:07:17.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.798+0000 7f623ad03740 1 -- start start 2024-09-12T03:07:17.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.798+0000 7f623ad03740 1 Processor -- start 2024-09-12T03:07:17.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.798+0000 7f623ad03740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d510a6b40 0x555d510b84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:17.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.798+0000 7f623ad03740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d510add30 0x555d510b8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:17.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.798+0000 7f623ad03740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d510b0370 0x555d510a9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:17.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.798+0000 7f623ad03740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f62341adbc0 con 0x555d510a6b40 2024-09-12T03:07:17.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.798+0000 7f623ad03740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f62341af030 con 0x555d510b0370 2024-09-12T03:07:17.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.798+0000 7f623ad03740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f62341ae290 con 0x555d510add30 2024-09-12T03:07:17.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.798+0000 7f623ad03740 10 client.0 _ll_register_callbacks cb 0x555d510b02c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:07:17.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.799+0000 7f623259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d510a6b40 0x555d510b84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:17.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.799+0000 7f623259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d510a6b40 0x555d510b84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46730/0 (socket says 172.21.15.116:46730) 2024-09-12T03:07:17.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.799+0000 7f623259c640 1 -- 172.21.15.116:0/234263387 learned_addr learned my addr 172.21.15.116:0/234263387 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:17.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.799+0000 7f6232d9d640 1 --2- 172.21.15.116:0/234263387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d510b0370 0x555d510a9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:17.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.799+0000 7f6229d9b640 1 --2- 172.21.15.116:0/234263387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d510add30 0x555d510b8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:17.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.799+0000 7f6232d9d640 1 -- 172.21.15.116:0/234263387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d510add30 msgr2=0x555d510b8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:17.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.799+0000 7f6232d9d640 1 --2- 172.21.15.116:0/234263387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d510add30 0x555d510b8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.799+0000 7f6232d9d640 1 -- 172.21.15.116:0/234263387 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d510a6b40 msgr2=0x555d510b84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:17.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.799+0000 7f6232d9d640 1 --2- 172.21.15.116:0/234263387 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d510a6b40 0x555d510b84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.799+0000 7f6232d9d640 1 -- 172.21.15.116:0/234263387 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d510a9a60 con 0x555d510b0370 2024-09-12T03:07:17.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.799+0000 7f623259c640 1 --2- 172.21.15.116:0/234263387 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d510a6b40 0x555d510b84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:07:17.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.799+0000 7f6232d9d640 1 --2- 172.21.15.116:0/234263387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d510b0370 0x555d510a9520 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f622c029040 tx=0x7f622c0059d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c3bf091f571c31 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:17.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.799+0000 7f6219ffb640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-09-12T03:07:17.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.800+0000 7f6219ffb640 1 -- 172.21.15.116:0/234263387 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c037070 con 0x555d510b0370 2024-09-12T03:07:17.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.800+0000 7f6219ffb640 1 -- 172.21.15.116:0/234263387 <== mon.2 v2:172.21.15.177:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f622c005200 con 0x555d510b0370 2024-09-12T03:07:17.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.800+0000 7f623ad03740 1 -- 172.21.15.116:0/234263387 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f622c005500 con 0x555d510b0370 2024-09-12T03:07:17.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.800+0000 7f6219ffb640 1 -- 172.21.15.116:0/234263387 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f622c042050 con 0x555d510b0370 2024-09-12T03:07:17.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.800+0000 7f623ad03740 10 client.25493 fetch_fsmap learned FSMap version 25 2024-09-12T03:07:17.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.800+0000 7f623ad03740 1 -- 172.21.15.116:0/234263387 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555d510aad20 con 0x555d510b0370 2024-09-12T03:07:17.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.800+0000 7f6219ffb640 1 -- 172.21.15.116:0/234263387 <== mon.2 v2:172.21.15.177:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f622c005500 con 0x555d510b0370 2024-09-12T03:07:17.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.800+0000 7f6219ffb640 1 -- 172.21.15.116:0/234263387 <== mon.2 v2:172.21.15.177:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f622c026030 con 0x555d510b0370 2024-09-12T03:07:17.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.800+0000 7f623ad03740 10 client.25493 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:07:17.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.800+0000 7f623ad03740 10 client.25493 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:07:17.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.800+0000 7f623ad03740 20 client.25493 populate_metadata read hostname 'smithi116' 2024-09-12T03:07:17.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.800+0000 7f623ad03740 1 --2- 172.21.15.116:0/234263387 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x555d510c0540 0x555d5112e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:17.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.800+0000 7f623ad03740 4 client.25493 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:07:17.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.800+0000 7f623ad03740 1 -- 172.21.15.116:0/234263387 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555d51012d70 con 0x555d510c0540 2024-09-12T03:07:17.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.801+0000 7f623259c640 1 --2- 172.21.15.116:0/234263387 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x555d510c0540 0x555d5112e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:17.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.801+0000 7f6219ffb640 10 client.25493 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:07:17.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.801+0000 7f623259c640 1 --2- 172.21.15.116:0/234263387 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x555d510c0540 0x555d5112e180 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=878b5809fc56e778 server_cookie=386cb2ce6eec185e in_seq=0 out_seq=0 2024-09-12T03:07:17.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.801+0000 7f6219ffb640 1 -- 172.21.15.116:0/234263387 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555d51012d70 con 0x555d510c0540 2024-09-12T03:07:17.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.801+0000 7f6219ffb640 10 client.25493 handle_command_reply: tid=0 2024-09-12T03:07:17.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.801+0000 7f623ad03740 1 client.25493 shutdown 2024-09-12T03:07:17.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.801+0000 7f623ad03740 1 -- 172.21.15.116:0/234263387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d510b0370 msgr2=0x555d510a9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:17.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.801+0000 7f623ad03740 1 --2- 172.21.15.116:0/234263387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d510b0370 0x555d510a9520 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f622c029040 tx=0x7f622c0059d0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.801+0000 7f623ad03740 1 -- 172.21.15.116:0/234263387 shutdown_connections 2024-09-12T03:07:17.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.801+0000 7f623ad03740 1 --2- 172.21.15.116:0/234263387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d510b0370 0x555d510a9520 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.801+0000 7f623ad03740 1 --2- 172.21.15.116:0/234263387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d510add30 0x555d510b8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.801+0000 7f623ad03740 1 --2- 172.21.15.116:0/234263387 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x555d510c0540 0x555d5112e180 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.801+0000 7f623ad03740 1 --2- 172.21.15.116:0/234263387 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d510a6b40 0x555d510b84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.801+0000 7f623ad03740 1 -- 172.21.15.116:0/234263387 >> 172.21.15.116:0/234263387 conn(0x555d5106e310 msgr2=0x555d510a0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:17.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.802+0000 7f6219ffb640 0 client.25493 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:07:17.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.802+0000 7f623ad03740 1 -- 172.21.15.116:0/234263387 shutdown_connections 2024-09-12T03:07:17.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.802+0000 7f623ad03740 1 -- 172.21.15.116:0/234263387 wait complete. 2024-09-12T03:07:17.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.802+0000 7f623ad03740 20 client.25493 trim_cache size 0 max 16384 2024-09-12T03:07:17.818 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:07:17.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.802+0000 7f623892a640 1 -- 172.21.15.116:0/3646581645 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f62200a2b80 msgr2=0x7f62200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:17.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.802+0000 7f623892a640 1 --2- 172.21.15.116:0/3646581645 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f62200a2b80 0x7f62200a5040 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f622c005fd0 tx=0x7f622c009600 comp rx=0 tx=0).stop 2024-09-12T03:07:17.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.802+0000 7f623892a640 1 -- 172.21.15.116:0/3646581645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6234102570 msgr2=0x7f62340fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:17.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.802+0000 7f623892a640 1 --2- 172.21.15.116:0/3646581645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6234102570 0x7f62340fd1c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f621c00d770 tx=0x7f621c00dc30 comp rx=0 tx=0).stop 2024-09-12T03:07:17.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.803+0000 7f623892a640 1 -- 172.21.15.116:0/3646581645 shutdown_connections 2024-09-12T03:07:17.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.803+0000 7f623892a640 1 --2- 172.21.15.116:0/3646581645 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f62200a2b80 0x7f62200a5040 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.803+0000 7f623892a640 1 --2- 172.21.15.116:0/3646581645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6234102ef0 0x7f62340fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.803+0000 7f623892a640 1 --2- 172.21.15.116:0/3646581645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6234102570 0x7f62340fd1c0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.803+0000 7f623892a640 1 --2- 172.21.15.116:0/3646581645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6234100e20 0x7f62340fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:17.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.803+0000 7f623892a640 1 -- 172.21.15.116:0/3646581645 >> 172.21.15.116:0/3646581645 conn(0x7f62340fc7d0 msgr2=0x7f623406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:17.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.803+0000 7f623892a640 1 -- 172.21.15.116:0/3646581645 shutdown_connections 2024-09-12T03:07:17.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:17.803+0000 7f623892a640 1 -- 172.21.15.116:0/3646581645 wait complete. 2024-09-12T03:07:18.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:18 smithi116 ceph-mon[31410]: pgmap v467: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-09-12T03:07:18.615 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:07:18.630 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:07:18.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:18 smithi183 ceph-mon[35535]: pgmap v467: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-09-12T03:07:18.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:18 smithi177 ceph-mon[35305]: pgmap v467: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-09-12T03:07:19.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.134+0000 7f4ab6b7a640 1 -- 172.21.15.116:0/449334240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ab0106a80 msgr2=0x7f4ab0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:19.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.134+0000 7f4ab6b7a640 1 --2- 172.21.15.116:0/449334240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ab0106a80 0x7f4ab0106e60 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f4aac009930 tx=0x7f4aac02a420 comp rx=0 tx=0).stop 2024-09-12T03:07:19.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.134+0000 7f4ab6b7a640 1 -- 172.21.15.116:0/449334240 shutdown_connections 2024-09-12T03:07:19.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.134+0000 7f4ab6b7a640 1 --2- 172.21.15.116:0/449334240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ab0101420 0x7f4ab010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.134+0000 7f4ab6b7a640 1 --2- 172.21.15.116:0/449334240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ab0100a80 0x7f4ab0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.134+0000 7f4ab6b7a640 1 --2- 172.21.15.116:0/449334240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ab0106a80 0x7f4ab0106e60 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.134+0000 7f4ab6b7a640 1 -- 172.21.15.116:0/449334240 >> 172.21.15.116:0/449334240 conn(0x7f4ab00fc7a0 msgr2=0x7f4ab00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:19.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.134+0000 7f4ab6b7a640 1 -- 172.21.15.116:0/449334240 shutdown_connections 2024-09-12T03:07:19.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.134+0000 7f4ab6b7a640 1 -- 172.21.15.116:0/449334240 wait complete. 2024-09-12T03:07:19.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.135+0000 7f4ab6b7a640 1 Processor -- start 2024-09-12T03:07:19.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.135+0000 7f4ab6b7a640 1 -- start start 2024-09-12T03:07:19.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.135+0000 7f4ab6b7a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ab0100a80 0x7f4ab01984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:19.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.135+0000 7f4ab6b7a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ab0101420 0x7f4ab0198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:19.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.135+0000 7f4ab6b7a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ab0106a80 0x7f4ab019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:19.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.135+0000 7f4ab6b7a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab0110ea0 con 0x7f4ab0106a80 2024-09-12T03:07:19.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.135+0000 7f4ab6b7a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab0110d20 con 0x7f4ab0101420 2024-09-12T03:07:19.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.135+0000 7f4ab6b7a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab0111020 con 0x7f4ab0100a80 2024-09-12T03:07:19.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.135+0000 7f4ab5117640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ab0106a80 0x7f4ab019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:19.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.135+0000 7f4ab5117640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ab0106a80 0x7f4ab019cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46736/0 (socket says 172.21.15.116:46736) 2024-09-12T03:07:19.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.135+0000 7f4ab5117640 1 -- 172.21.15.116:0/2980734120 learned_addr learned my addr 172.21.15.116:0/2980734120 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:19.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.135+0000 7f4ab4916640 1 --2- 172.21.15.116:0/2980734120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ab0100a80 0x7f4ab01984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:19.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.135+0000 7f4aa7fff640 1 --2- 172.21.15.116:0/2980734120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ab0101420 0x7f4ab0198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:19.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.136+0000 7f4ab5117640 1 -- 172.21.15.116:0/2980734120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ab0100a80 msgr2=0x7f4ab01984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:19.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.136+0000 7f4ab5117640 1 --2- 172.21.15.116:0/2980734120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ab0100a80 0x7f4ab01984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.136+0000 7f4ab5117640 1 -- 172.21.15.116:0/2980734120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ab0101420 msgr2=0x7f4ab0198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:19.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.136+0000 7f4ab5117640 1 --2- 172.21.15.116:0/2980734120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ab0101420 0x7f4ab0198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.136+0000 7f4ab5117640 1 -- 172.21.15.116:0/2980734120 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ab019d480 con 0x7f4ab0106a80 2024-09-12T03:07:19.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.136+0000 7f4ab5117640 1 --2- 172.21.15.116:0/2980734120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ab0106a80 0x7f4ab019cda0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4aa800b970 tx=0x7f4aa800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:19.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.136+0000 7f4aa5ffb640 1 -- 172.21.15.116:0/2980734120 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa8019070 con 0x7f4ab0106a80 2024-09-12T03:07:19.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.136+0000 7f4aa5ffb640 1 -- 172.21.15.116:0/2980734120 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aa800cb40 con 0x7f4ab0106a80 2024-09-12T03:07:19.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.136+0000 7f4ab6b7a640 1 -- 172.21.15.116:0/2980734120 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ab019d770 con 0x7f4ab0106a80 2024-09-12T03:07:19.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.136+0000 7f4aa5ffb640 1 -- 172.21.15.116:0/2980734120 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa8007650 con 0x7f4ab0106a80 2024-09-12T03:07:19.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.136+0000 7f4ab6b7a640 1 -- 172.21.15.116:0/2980734120 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ab01a5050 con 0x7f4ab0106a80 2024-09-12T03:07:19.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.139+0000 7f4aa5ffb640 1 -- 172.21.15.116:0/2980734120 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4aa801c070 con 0x7f4ab0106a80 2024-09-12T03:07:19.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.139+0000 7f4ab8f53740 1 Processor -- start 2024-09-12T03:07:19.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.139+0000 7f4ab8f53740 1 -- start start 2024-09-12T03:07:19.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.139+0000 7f4ab8f53740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b8e417730 0x556b8e417b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:19.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.139+0000 7f4ab8f53740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8e418050 0x556b8e426900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:19.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.139+0000 7f4ab8f53740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b8e426e40 0x556b8e429230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:19.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.139+0000 7f4ab8f53740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556b8e37c160 con 0x556b8e426e40 2024-09-12T03:07:19.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.139+0000 7f4ab8f53740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556b8e323720 con 0x556b8e418050 2024-09-12T03:07:19.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.139+0000 7f4ab8f53740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556b8e378120 con 0x556b8e417730 2024-09-12T03:07:19.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.139+0000 7f4aa5ffb640 1 --2- 172.21.15.116:0/2980734120 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4a980a2c00 0x7f4a980a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:19.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.140+0000 7f4ab4916640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8e418050 0x556b8e426900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:19.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.140+0000 7f4ab4916640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8e418050 0x556b8e426900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.116:34618/0 (socket says 172.21.15.116:34618) 2024-09-12T03:07:19.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.140+0000 7f4ab4916640 1 -- 172.21.15.116:0/626026813 learned_addr learned my addr 172.21.15.116:0/626026813 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:19.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.140+0000 7f4ab5117640 1 --2- 172.21.15.116:0/2980734120 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4a980a2c00 0x7f4a980a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:19.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.140+0000 7f4aa7fff640 1 --2- 172.21.15.116:0/626026813 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b8e426e40 0x556b8e429230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:19.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.140+0000 7f4aa5ffb640 1 -- 172.21.15.116:0/2980734120 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f4aa80ce5a0 con 0x7f4ab0106a80 2024-09-12T03:07:19.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.140+0000 7f4aa7fff640 1 --2- 172.21.15.116:0/626026813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b8e417730 0x556b8e417b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:19.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.140+0000 7f4ab4916640 1 -- 172.21.15.116:0/626026813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b8e417730 msgr2=0x556b8e417b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:19.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.140+0000 7f4ab4916640 1 --2- 172.21.15.116:0/626026813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b8e417730 0x556b8e417b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.140+0000 7f4ab4916640 1 -- 172.21.15.116:0/626026813 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b8e426e40 msgr2=0x556b8e429230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:19.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.140+0000 7f4ab4916640 1 --2- 172.21.15.116:0/626026813 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b8e426e40 0x556b8e429230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.140+0000 7f4ab4916640 1 -- 172.21.15.116:0/626026813 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b8e0edba0 con 0x556b8e418050 2024-09-12T03:07:19.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.140+0000 7f4ab4916640 1 --2- 172.21.15.116:0/626026813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8e418050 0x556b8e426900 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4aac02a940 tx=0x7f4aac0316e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdd3170683d05efe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:19.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.140+0000 7f4ab5117640 1 --2- 172.21.15.116:0/2980734120 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4a980a2c00 0x7f4a980a50c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4aa80a3510 tx=0x7f4aa8065b00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:19.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.141+0000 7f4a96ffd640 1 -- 172.21.15.116:0/626026813 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac040070 con 0x556b8e418050 2024-09-12T03:07:19.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.141+0000 7f4a96ffd640 1 -- 172.21.15.116:0/626026813 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aac005020 con 0x556b8e418050 2024-09-12T03:07:19.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.141+0000 7f4a96ffd640 1 -- 172.21.15.116:0/626026813 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac035040 con 0x556b8e418050 2024-09-12T03:07:19.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.141+0000 7f4ab8f53740 1 -- 172.21.15.116:0/626026813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8e418050 msgr2=0x556b8e426900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:19.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.141+0000 7f4ab8f53740 1 --2- 172.21.15.116:0/626026813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8e418050 0x556b8e426900 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4aac02a940 tx=0x7f4aac0316e0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.141+0000 7f4ab8f53740 1 -- 172.21.15.116:0/626026813 shutdown_connections 2024-09-12T03:07:19.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.141+0000 7f4ab8f53740 1 --2- 172.21.15.116:0/626026813 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b8e426e40 0x556b8e429230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.141+0000 7f4ab8f53740 1 --2- 172.21.15.116:0/626026813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8e418050 0x556b8e426900 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.141+0000 7f4ab8f53740 1 --2- 172.21.15.116:0/626026813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b8e417730 0x556b8e417b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.141+0000 7f4ab8f53740 1 -- 172.21.15.116:0/626026813 >> 172.21.15.116:0/626026813 conn(0x556b8e413de0 msgr2=0x556b8e4141c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:19.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.142+0000 7f4ab8f53740 1 -- 172.21.15.116:0/626026813 shutdown_connections 2024-09-12T03:07:19.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.142+0000 7f4ab8f53740 1 -- 172.21.15.116:0/626026813 wait complete. 2024-09-12T03:07:19.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.142+0000 7f4ab8f53740 1 -- start start 2024-09-12T03:07:19.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.142+0000 7f4ab8f53740 1 Processor -- start 2024-09-12T03:07:19.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.142+0000 7f4ab8f53740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b8e417730 0x556b8e423d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:19.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.142+0000 7f4ab8f53740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b8e418050 0x556b8e4242c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:19.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.142+0000 7f4ab8f53740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8e426e40 0x556b8e424800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:19.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.142+0000 7f4ab8f53740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556b8e429a10 con 0x556b8e417730 2024-09-12T03:07:19.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.142+0000 7f4ab8f53740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556b8e304900 con 0x556b8e426e40 2024-09-12T03:07:19.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.142+0000 7f4ab8f53740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556b8e429b90 con 0x556b8e418050 2024-09-12T03:07:19.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.143+0000 7f4ab8f53740 10 client.0 _ll_register_callbacks cb 0x556b8e302780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:07:19.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.143+0000 7f4aa7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b8e417730 0x556b8e423d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:19.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.143+0000 7f4aa7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b8e417730 0x556b8e423d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46746/0 (socket says 172.21.15.116:46746) 2024-09-12T03:07:19.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.143+0000 7f4aa7fff640 1 -- 172.21.15.116:0/2493486874 learned_addr learned my addr 172.21.15.116:0/2493486874 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:19.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.143+0000 7f4ab4916640 1 --2- 172.21.15.116:0/2493486874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b8e418050 0x556b8e4242c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:19.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.143+0000 7f4aa7fff640 1 --2- 172.21.15.116:0/2493486874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8e426e40 0x556b8e424800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:19.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.143+0000 7f4aa7fff640 1 -- 172.21.15.116:0/2493486874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b8e418050 msgr2=0x556b8e4242c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:19.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.143+0000 7f4aa7fff640 1 --2- 172.21.15.116:0/2493486874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b8e418050 0x556b8e4242c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.143+0000 7f4aa7fff640 1 -- 172.21.15.116:0/2493486874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8e426e40 msgr2=0x556b8e424800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:19.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.143+0000 7f4aa7fff640 1 --2- 172.21.15.116:0/2493486874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8e426e40 0x556b8e424800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.143+0000 7f4aa7fff640 1 -- 172.21.15.116:0/2493486874 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b8e344c50 con 0x556b8e417730 2024-09-12T03:07:19.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.143+0000 7f4aa7fff640 1 --2- 172.21.15.116:0/2493486874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b8e417730 0x556b8e423d80 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0010fb0 tx=0x7f4aa000a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb2f5d1a988f89a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:19.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.143+0000 7f4a8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:07:19.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.143+0000 7f4a8ffff640 1 -- 172.21.15.116:0/2493486874 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0014070 con 0x556b8e417730 2024-09-12T03:07:19.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.143+0000 7f4ab8f53740 1 -- 172.21.15.116:0/2493486874 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4aac005020 con 0x556b8e417730 2024-09-12T03:07:19.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.143+0000 7f4a8ffff640 1 -- 172.21.15.116:0/2493486874 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f4aa000fdb0 con 0x556b8e417730 2024-09-12T03:07:19.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.143+0000 7f4a8ffff640 1 -- 172.21.15.116:0/2493486874 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4aa0004020 con 0x556b8e417730 2024-09-12T03:07:19.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.143+0000 7f4ab8f53740 10 client.16362 fetch_fsmap learned FSMap version 25 2024-09-12T03:07:19.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.143+0000 7f4ab8f53740 1 -- 172.21.15.116:0/2493486874 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556b8e435790 con 0x556b8e417730 2024-09-12T03:07:19.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.144+0000 7f4a8ffff640 1 -- 172.21.15.116:0/2493486874 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aac005020 con 0x556b8e417730 2024-09-12T03:07:19.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.144+0000 7f4a8ffff640 1 -- 172.21.15.116:0/2493486874 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4aa0022410 con 0x556b8e417730 2024-09-12T03:07:19.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.144+0000 7f4ab8f53740 10 client.16362 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:07:19.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.144+0000 7f4ab8f53740 10 client.16362 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:07:19.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.144+0000 7f4ab8f53740 20 client.16362 populate_metadata read hostname 'smithi116' 2024-09-12T03:07:19.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.144+0000 7f4ab8f53740 1 --2- 172.21.15.116:0/2493486874 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556b8e436c50 0x556b8e439040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:19.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.144+0000 7f4ab8f53740 4 client.16362 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:07:19.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.144+0000 7f4ab8f53740 1 -- 172.21.15.116:0/2493486874 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b8e389890 con 0x556b8e436c50 2024-09-12T03:07:19.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.144+0000 7f4ab4916640 1 --2- 172.21.15.116:0/2493486874 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556b8e436c50 0x556b8e439040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:19.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.145+0000 7f4ab4916640 1 --2- 172.21.15.116:0/2493486874 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556b8e436c50 0x556b8e439040 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47dffeb666185494 server_cookie=cd8376092d6a1ce5 in_seq=0 out_seq=0 2024-09-12T03:07:19.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.145+0000 7f4a8ffff640 10 client.16362 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:19.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.146+0000 7f4a8ffff640 1 -- 172.21.15.116:0/2493486874 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556b8e389890 con 0x556b8e436c50 2024-09-12T03:07:19.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.146+0000 7f4a8ffff640 10 client.16362 handle_command_reply: tid=0 2024-09-12T03:07:19.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.146+0000 7f4ab8f53740 1 client.16362 shutdown 2024-09-12T03:07:19.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.146+0000 7f4ab8f53740 1 -- 172.21.15.116:0/2493486874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b8e417730 msgr2=0x556b8e423d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:19.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.146+0000 7f4ab8f53740 1 --2- 172.21.15.116:0/2493486874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b8e417730 0x556b8e423d80 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0010fb0 tx=0x7f4aa000a2d0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.146+0000 7f4ab8f53740 1 -- 172.21.15.116:0/2493486874 shutdown_connections 2024-09-12T03:07:19.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.146+0000 7f4ab8f53740 1 --2- 172.21.15.116:0/2493486874 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556b8e436c50 0x556b8e439040 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.146+0000 7f4ab8f53740 1 --2- 172.21.15.116:0/2493486874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8e426e40 0x556b8e424800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.146+0000 7f4ab8f53740 1 --2- 172.21.15.116:0/2493486874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b8e418050 0x556b8e4242c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.146+0000 7f4ab8f53740 1 --2- 172.21.15.116:0/2493486874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b8e417730 0x556b8e423d80 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.146+0000 7f4ab8f53740 1 -- 172.21.15.116:0/2493486874 >> 172.21.15.116:0/2493486874 conn(0x556b8e413de0 msgr2=0x556b8e419b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:19.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.146+0000 7f4a8ffff640 0 client.16362 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:19.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.146+0000 7f4ab8f53740 1 -- 172.21.15.116:0/2493486874 shutdown_connections 2024-09-12T03:07:19.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.146+0000 7f4ab8f53740 1 -- 172.21.15.116:0/2493486874 wait complete. 2024-09-12T03:07:19.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.146+0000 7f4ab8f53740 20 client.16362 trim_cache size 0 max 16384 2024-09-12T03:07:19.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.169+0000 7f4ab8f53740 1 Processor -- start 2024-09-12T03:07:19.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.169+0000 7f4ab8f53740 1 -- start start 2024-09-12T03:07:19.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.169+0000 7f4ab8f53740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8e417730 0x556b8e43e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:19.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.169+0000 7f4ab8f53740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b8e418050 0x556b8e43ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:19.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.169+0000 7f4ab8f53740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b8e426e40 0x556b8e423d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:19.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.169+0000 7f4ab8f53740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab01adc30 con 0x556b8e418050 2024-09-12T03:07:19.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.169+0000 7f4ab8f53740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab01ab500 con 0x556b8e417730 2024-09-12T03:07:19.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.169+0000 7f4ab8f53740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab01a9350 con 0x556b8e426e40 2024-09-12T03:07:19.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.169+0000 7f4ab4916640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b8e418050 0x556b8e43ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:19.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.169+0000 7f4ab4916640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b8e418050 0x556b8e43ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46758/0 (socket says 172.21.15.116:46758) 2024-09-12T03:07:19.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.169+0000 7f4ab4916640 1 -- 172.21.15.116:0/3554332441 learned_addr learned my addr 172.21.15.116:0/3554332441 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:19.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.169+0000 7f4aa7fff640 1 --2- 172.21.15.116:0/3554332441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8e417730 0x556b8e43e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:19.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.170+0000 7f4aa7fff640 1 --2- 172.21.15.116:0/3554332441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b8e426e40 0x556b8e423d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:19.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.170+0000 7f4ab4916640 1 -- 172.21.15.116:0/3554332441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b8e426e40 msgr2=0x556b8e423d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:19.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.170+0000 7f4ab4916640 1 --2- 172.21.15.116:0/3554332441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b8e426e40 0x556b8e423d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.170+0000 7f4ab4916640 1 -- 172.21.15.116:0/3554332441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8e417730 msgr2=0x556b8e43e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:19.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.170+0000 7f4ab4916640 1 --2- 172.21.15.116:0/3554332441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8e417730 0x556b8e43e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.170+0000 7f4ab4916640 1 -- 172.21.15.116:0/3554332441 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b8e417e80 con 0x556b8e418050 2024-09-12T03:07:19.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.170+0000 7f4ab4916640 1 --2- 172.21.15.116:0/3554332441 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b8e418050 0x556b8e43ee50 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4aac031520 tx=0x7f4aac005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf7f63578aaea4e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:19.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.170+0000 7f4a967fc640 1 -- 172.21.15.116:0/3554332441 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac040040 con 0x556b8e418050 2024-09-12T03:07:19.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.170+0000 7f4a967fc640 1 -- 172.21.15.116:0/3554332441 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aac03dcc0 con 0x556b8e418050 2024-09-12T03:07:19.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.170+0000 7f4a967fc640 1 -- 172.21.15.116:0/3554332441 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac035040 con 0x556b8e418050 2024-09-12T03:07:19.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.171+0000 7f4ab8f53740 1 -- 172.21.15.116:0/3554332441 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b8e418050 msgr2=0x556b8e43ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:19.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.171+0000 7f4ab8f53740 1 --2- 172.21.15.116:0/3554332441 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b8e418050 0x556b8e43ee50 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4aac031520 tx=0x7f4aac005fd0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.171+0000 7f4ab8f53740 1 -- 172.21.15.116:0/3554332441 shutdown_connections 2024-09-12T03:07:19.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.171+0000 7f4ab8f53740 1 --2- 172.21.15.116:0/3554332441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b8e426e40 0x556b8e423d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.171+0000 7f4ab8f53740 1 --2- 172.21.15.116:0/3554332441 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b8e418050 0x556b8e43ee50 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.171+0000 7f4ab8f53740 1 --2- 172.21.15.116:0/3554332441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8e417730 0x556b8e43e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.171+0000 7f4ab8f53740 1 -- 172.21.15.116:0/3554332441 >> 172.21.15.116:0/3554332441 conn(0x556b8e413de0 msgr2=0x556b8e422e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:19.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.171+0000 7f4ab8f53740 1 -- 172.21.15.116:0/3554332441 shutdown_connections 2024-09-12T03:07:19.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.171+0000 7f4ab8f53740 1 -- 172.21.15.116:0/3554332441 wait complete. 2024-09-12T03:07:19.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.172+0000 7f4ab8f53740 1 -- start start 2024-09-12T03:07:19.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.172+0000 7f4ab8f53740 1 Processor -- start 2024-09-12T03:07:19.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.172+0000 7f4ab8f53740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b8e417730 0x556b8e49c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:19.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.172+0000 7f4ab8f53740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8e418050 0x556b8e49cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:19.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.172+0000 7f4ab8f53740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b8e426e40 0x556b8e499f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:19.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.172+0000 7f4ab8f53740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab01a99c0 con 0x556b8e417730 2024-09-12T03:07:19.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.172+0000 7f4ab8f53740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab01aae30 con 0x556b8e418050 2024-09-12T03:07:19.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.172+0000 7f4aa7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b8e417730 0x556b8e49c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:19.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.172+0000 7f4ab8f53740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab01aa090 con 0x556b8e426e40 2024-09-12T03:07:19.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.172+0000 7f4aa7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b8e417730 0x556b8e49c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46772/0 (socket says 172.21.15.116:46772) 2024-09-12T03:07:19.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.172+0000 7f4aa7fff640 1 -- 172.21.15.116:0/1517522086 learned_addr learned my addr 172.21.15.116:0/1517522086 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:19.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.173+0000 7f4ab8f53740 10 client.0 _ll_register_callbacks cb 0x7f4aa000a710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:07:19.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.173+0000 7f4ab4916640 1 --2- 172.21.15.116:0/1517522086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8e418050 0x556b8e49cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:19.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.173+0000 7f4aa7fff640 1 -- 172.21.15.116:0/1517522086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b8e426e40 msgr2=0x556b8e499f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:19.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.173+0000 7f4aa7fff640 1 --2- 172.21.15.116:0/1517522086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b8e426e40 0x556b8e499f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.173+0000 7f4aa7fff640 1 -- 172.21.15.116:0/1517522086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8e418050 msgr2=0x556b8e49cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:19.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.173+0000 7f4aa7fff640 1 --2- 172.21.15.116:0/1517522086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8e418050 0x556b8e49cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.173+0000 7f4aa7fff640 1 -- 172.21.15.116:0/1517522086 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b8e49a4a0 con 0x556b8e417730 2024-09-12T03:07:19.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.173+0000 7f4aa7fff640 1 --2- 172.21.15.116:0/1517522086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b8e417730 0x556b8e49c880 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f4aa00228b0 tx=0x7f4aa000ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=207bf750d3645363 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:19.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.173+0000 7f4a8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:07:19.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.173+0000 7f4a8ffff640 1 -- 172.21.15.116:0/1517522086 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0033020 con 0x556b8e417730 2024-09-12T03:07:19.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.173+0000 7f4a8ffff640 1 -- 172.21.15.116:0/1517522086 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f4aa0022aa0 con 0x556b8e417730 2024-09-12T03:07:19.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.173+0000 7f4ab8f53740 1 -- 172.21.15.116:0/1517522086 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4aac03dcc0 con 0x556b8e417730 2024-09-12T03:07:19.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.173+0000 7f4a8ffff640 1 -- 172.21.15.116:0/1517522086 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4aa0014070 con 0x556b8e417730 2024-09-12T03:07:19.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.174+0000 7f4ab8f53740 10 client.16368 fetch_fsmap learned FSMap version 25 2024-09-12T03:07:19.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.174+0000 7f4ab8f53740 1 -- 172.21.15.116:0/1517522086 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556b8e41dad0 con 0x556b8e417730 2024-09-12T03:07:19.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.174+0000 7f4a8ffff640 1 -- 172.21.15.116:0/1517522086 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aac03dcc0 con 0x556b8e417730 2024-09-12T03:07:19.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.174+0000 7f4a8ffff640 1 -- 172.21.15.116:0/1517522086 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4aa0003ad0 con 0x556b8e417730 2024-09-12T03:07:19.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.174+0000 7f4ab8f53740 10 client.16368 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:07:19.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.174+0000 7f4ab8f53740 10 client.16368 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:07:19.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.174+0000 7f4ab8f53740 20 client.16368 populate_metadata read hostname 'smithi116' 2024-09-12T03:07:19.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.174+0000 7f4ab8f53740 1 --2- 172.21.15.116:0/1517522086 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556b8e42de00 0x556b8e42e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:19.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.174+0000 7f4ab8f53740 4 client.16368 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}] 2024-09-12T03:07:19.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.174+0000 7f4ab8f53740 1 -- 172.21.15.116:0/1517522086 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}) -- 0x556b8e383be0 con 0x556b8e42de00 2024-09-12T03:07:19.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.174+0000 7f4ab4916640 1 --2- 172.21.15.116:0/1517522086 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556b8e42de00 0x556b8e42e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:19.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.175+0000 7f4a8ffff640 10 client.16368 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:19.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.175+0000 7f4ab4916640 1 --2- 172.21.15.116:0/1517522086 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556b8e42de00 0x556b8e42e210 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c28393a78eb4202 server_cookie=c70bf9db2a54a6f7 in_seq=0 out_seq=0 2024-09-12T03:07:19.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.181+0000 7f4a8ffff640 1 -- 172.21.15.116:0/1517522086 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556b8e383be0 con 0x556b8e42de00 2024-09-12T03:07:19.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.181+0000 7f4a8ffff640 10 client.16368 handle_command_reply: tid=0 2024-09-12T03:07:19.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.181+0000 7f4ab8f53740 1 client.16368 shutdown 2024-09-12T03:07:19.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.181+0000 7f4ab8f53740 1 -- 172.21.15.116:0/1517522086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b8e417730 msgr2=0x556b8e49c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:19.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.181+0000 7f4ab8f53740 1 --2- 172.21.15.116:0/1517522086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b8e417730 0x556b8e49c880 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f4aa00228b0 tx=0x7f4aa000ada0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.181+0000 7f4ab8f53740 1 -- 172.21.15.116:0/1517522086 shutdown_connections 2024-09-12T03:07:19.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.181+0000 7f4ab8f53740 1 --2- 172.21.15.116:0/1517522086 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556b8e42de00 0x556b8e42e210 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.181+0000 7f4ab8f53740 1 --2- 172.21.15.116:0/1517522086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556b8e426e40 0x556b8e499f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.181+0000 7f4ab8f53740 1 --2- 172.21.15.116:0/1517522086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8e418050 0x556b8e49cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.181+0000 7f4ab8f53740 1 --2- 172.21.15.116:0/1517522086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b8e417730 0x556b8e49c880 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.181+0000 7f4ab8f53740 1 -- 172.21.15.116:0/1517522086 >> 172.21.15.116:0/1517522086 conn(0x556b8e413de0 msgr2=0x556b8e419240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:19.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.181+0000 7f4a8ffff640 0 client.16368 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:19.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.182+0000 7f4ab8f53740 1 -- 172.21.15.116:0/1517522086 shutdown_connections 2024-09-12T03:07:19.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.182+0000 7f4ab8f53740 1 -- 172.21.15.116:0/1517522086 wait complete. 2024-09-12T03:07:19.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.182+0000 7f4ab8f53740 20 client.16368 trim_cache size 0 max 16384 2024-09-12T03:07:19.183 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:07:19.183 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 25, 2024-09-12T03:07:19.183 INFO:teuthology.orchestra.run.smithi116.stdout: "leader": 24494, 2024-09-12T03:07:19.184 INFO:teuthology.orchestra.run.smithi116.stdout: "set_version": 18, 2024-09-12T03:07:19.184 INFO:teuthology.orchestra.run.smithi116.stdout: "sets": { 2024-09-12T03:07:19.184 INFO:teuthology.orchestra.run.smithi116.stdout: "1c00527e": { 2024-09-12T03:07:19.184 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 18, 2024-09-12T03:07:19.184 INFO:teuthology.orchestra.run.smithi116.stdout: "age_ref": 0.0, 2024-09-12T03:07:19.184 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:07:19.184 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "QUIESCED", 2024-09-12T03:07:19.184 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:07:19.184 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:07:19.184 INFO:teuthology.orchestra.run.smithi116.stdout: "timeout": 90.0, 2024-09-12T03:07:19.184 INFO:teuthology.orchestra.run.smithi116.stdout: "expiration": 140.2, 2024-09-12T03:07:19.184 INFO:teuthology.orchestra.run.smithi116.stdout: "members": { 2024-09-12T03:07:19.184 INFO:teuthology.orchestra.run.smithi116.stdout: "file:/": { 2024-09-12T03:07:19.185 INFO:teuthology.orchestra.run.smithi116.stdout: "excluded": false, 2024-09-12T03:07:19.185 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:07:19.185 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "QUIESCED", 2024-09-12T03:07:19.185 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:07:19.185 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:07:19.185 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:07:19.185 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:07:19.185 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:07:19.185 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:07:19.185 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:07:19.186 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:07:19.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.182+0000 7f4ab6b7a640 1 -- 172.21.15.116:0/2980734120 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4a980a2c00 msgr2=0x7f4a980a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:19.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.182+0000 7f4ab6b7a640 1 --2- 172.21.15.116:0/2980734120 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4a980a2c00 0x7f4a980a50c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4aa80a3510 tx=0x7f4aa8065b00 comp rx=0 tx=0).stop 2024-09-12T03:07:19.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.182+0000 7f4ab6b7a640 1 -- 172.21.15.116:0/2980734120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ab0106a80 msgr2=0x7f4ab019cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:19.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.182+0000 7f4ab6b7a640 1 --2- 172.21.15.116:0/2980734120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ab0106a80 0x7f4ab019cda0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4aa800b970 tx=0x7f4aa800be30 comp rx=0 tx=0).stop 2024-09-12T03:07:19.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.182+0000 7f4ab6b7a640 1 -- 172.21.15.116:0/2980734120 shutdown_connections 2024-09-12T03:07:19.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.182+0000 7f4ab6b7a640 1 --2- 172.21.15.116:0/2980734120 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4a980a2c00 0x7f4a980a50c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.182+0000 7f4ab6b7a640 1 --2- 172.21.15.116:0/2980734120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ab0106a80 0x7f4ab019cda0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.182+0000 7f4ab6b7a640 1 --2- 172.21.15.116:0/2980734120 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ab0101420 0x7f4ab0198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.182+0000 7f4ab6b7a640 1 --2- 172.21.15.116:0/2980734120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ab0100a80 0x7f4ab01984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:19.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.182+0000 7f4ab6b7a640 1 -- 172.21.15.116:0/2980734120 >> 172.21.15.116:0/2980734120 conn(0x7f4ab00fc7a0 msgr2=0x7f4ab0104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:19.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.183+0000 7f4ab6b7a640 1 -- 172.21.15.116:0/2980734120 shutdown_connections 2024-09-12T03:07:19.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:19.183+0000 7f4ab6b7a640 1 -- 172.21.15.116:0/2980734120 wait complete. 2024-09-12T03:07:20.129 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1c00527e', quiesce took 5.5 seconds. Will release after: 14.649999999999999 2024-09-12T03:07:20.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.327+0000 7fbf86a1f640 1 -- 172.21.15.116:0/1047575401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf80106f50 msgr2=0x7fbf8010eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:20.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.327+0000 7fbf86a1f640 1 --2- 172.21.15.116:0/1047575401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf80106f50 0x7fbf8010eaf0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009cd0 tx=0x7fbf7802a8e0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.327+0000 7fbf86a1f640 1 -- 172.21.15.116:0/1047575401 shutdown_connections 2024-09-12T03:07:20.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.327+0000 7fbf86a1f640 1 --2- 172.21.15.116:0/1047575401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf80106f50 0x7fbf8010eaf0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.327+0000 7fbf86a1f640 1 --2- 172.21.15.116:0/1047575401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf80100ca0 0x7fbf80106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.327+0000 7fbf86a1f640 1 --2- 172.21.15.116:0/1047575401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf801002f0 0x7fbf801006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.327+0000 7fbf86a1f640 1 -- 172.21.15.116:0/1047575401 >> 172.21.15.116:0/1047575401 conn(0x7fbf80077210 msgr2=0x7fbf800fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:20.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.327+0000 7fbf86a1f640 1 -- 172.21.15.116:0/1047575401 shutdown_connections 2024-09-12T03:07:20.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.327+0000 7fbf86a1f640 1 -- 172.21.15.116:0/1047575401 wait complete. 2024-09-12T03:07:20.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.327+0000 7fbf86a1f640 1 Processor -- start 2024-09-12T03:07:20.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.327+0000 7fbf86a1f640 1 -- start start 2024-09-12T03:07:20.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.328+0000 7fbf86a1f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf801002f0 0x7fbf80198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.328+0000 7fbf86a1f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf80100ca0 0x7fbf801989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.328+0000 7fbf86a1f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf80106f50 0x7fbf8019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.328+0000 7fbf86a1f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbf80111fa0 con 0x7fbf80106f50 2024-09-12T03:07:20.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.328+0000 7fbf86a1f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbf80111e20 con 0x7fbf80100ca0 2024-09-12T03:07:20.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.328+0000 7fbf86a1f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbf80112120 con 0x7fbf801002f0 2024-09-12T03:07:20.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.328+0000 7fbf84fbc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf80106f50 0x7fbf8019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.328+0000 7fbf7ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf801002f0 0x7fbf80198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.328+0000 7fbf84fbc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf80106f50 0x7fbf8019cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46794/0 (socket says 172.21.15.116:46794) 2024-09-12T03:07:20.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.328+0000 7fbf7ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf801002f0 0x7fbf80198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.116:59402/0 (socket says 172.21.15.116:59402) 2024-09-12T03:07:20.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.328+0000 7fbf84fbc640 1 -- 172.21.15.116:0/2329026740 learned_addr learned my addr 172.21.15.116:0/2329026740 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:20.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.328+0000 7fbf7f7fe640 1 --2- 172.21.15.116:0/2329026740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf80100ca0 0x7fbf801989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.328+0000 7fbf84fbc640 1 -- 172.21.15.116:0/2329026740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf801002f0 msgr2=0x7fbf80198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:20.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.328+0000 7fbf84fbc640 1 --2- 172.21.15.116:0/2329026740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf801002f0 0x7fbf80198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.328+0000 7fbf84fbc640 1 -- 172.21.15.116:0/2329026740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf80100ca0 msgr2=0x7fbf801989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:20.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.328+0000 7fbf84fbc640 1 --2- 172.21.15.116:0/2329026740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf80100ca0 0x7fbf801989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.328+0000 7fbf84fbc640 1 -- 172.21.15.116:0/2329026740 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8019d410 con 0x7fbf80106f50 2024-09-12T03:07:20.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.328+0000 7fbf7f7fe640 1 --2- 172.21.15.116:0/2329026740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf80100ca0 0x7fbf801989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:07:20.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.328+0000 7fbf84fbc640 1 --2- 172.21.15.116:0/2329026740 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf80106f50 0x7fbf8019cd30 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbf78005cc0 tx=0x7fbf780093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:20.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.329+0000 7fbf7d7fa640 1 -- 172.21.15.116:0/2329026740 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf78003e00 con 0x7fbf80106f50 2024-09-12T03:07:20.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.329+0000 7fbf86a1f640 1 -- 172.21.15.116:0/2329026740 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf8019d6a0 con 0x7fbf80106f50 2024-09-12T03:07:20.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.329+0000 7fbf7d7fa640 1 -- 172.21.15.116:0/2329026740 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf78004520 con 0x7fbf80106f50 2024-09-12T03:07:20.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.329+0000 7fbf7d7fa640 1 -- 172.21.15.116:0/2329026740 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf78030db0 con 0x7fbf80106f50 2024-09-12T03:07:20.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.329+0000 7fbf86a1f640 1 -- 172.21.15.116:0/2329026740 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf801a4fe0 con 0x7fbf80106f50 2024-09-12T03:07:20.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.331+0000 7fbf7d7fa640 1 -- 172.21.15.116:0/2329026740 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fbf780046d0 con 0x7fbf80106f50 2024-09-12T03:07:20.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.331+0000 7fbf88df8740 1 Processor -- start 2024-09-12T03:07:20.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.331+0000 7fbf88df8740 1 -- start start 2024-09-12T03:07:20.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.332+0000 7fbf88df8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960e2c0b40 0x55960e2bf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.332+0000 7fbf88df8740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55960e2bf5d0 0x55960e2bfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.332+0000 7fbf88df8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55960e2ca370 0x55960e2cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.332+0000 7fbf88df8740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55960e2252f0 con 0x55960e2bf5d0 2024-09-12T03:07:20.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.332+0000 7fbf88df8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55960e1cccf0 con 0x55960e2ca370 2024-09-12T03:07:20.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.332+0000 7fbf88df8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55960e2212b0 con 0x55960e2c0b40 2024-09-12T03:07:20.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.332+0000 7fbf7d7fa640 1 --2- 172.21.15.116:0/2329026740 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbf680a2b80 0x7fbf680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.332+0000 7fbf7ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55960e2bf5d0 0x55960e2bfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.332+0000 7fbf7f7fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55960e2ca370 0x55960e2cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.332+0000 7fbf7ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55960e2bf5d0 0x55960e2bfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46806/0 (socket says 172.21.15.116:46806) 2024-09-12T03:07:20.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.332+0000 7fbf7ffff640 1 -- 172.21.15.116:0/3831170157 learned_addr learned my addr 172.21.15.116:0/3831170157 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:20.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.332+0000 7fbf7f7fe640 1 --2- 172.21.15.116:0/3831170157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960e2c0b40 0x55960e2bf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.332+0000 7fbf7d7fa640 1 -- 172.21.15.116:0/2329026740 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fbf7802e030 con 0x7fbf80106f50 2024-09-12T03:07:20.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.332+0000 7fbf84fbc640 1 --2- 172.21.15.116:0/2329026740 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbf680a2b80 0x7fbf680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.332+0000 7fbf7ffff640 1 -- 172.21.15.116:0/3831170157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960e2c0b40 msgr2=0x55960e2bf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:20.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.332+0000 7fbf7ffff640 1 --2- 172.21.15.116:0/3831170157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960e2c0b40 0x55960e2bf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.332+0000 7fbf7ffff640 1 -- 172.21.15.116:0/3831170157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55960e2ca370 msgr2=0x55960e2cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:20.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.332+0000 7fbf7ffff640 1 --2- 172.21.15.116:0/3831170157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55960e2ca370 0x55960e2cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.332+0000 7fbf7ffff640 1 -- 172.21.15.116:0/3831170157 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55960e1b13a0 con 0x55960e2bf5d0 2024-09-12T03:07:20.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.332+0000 7fbf84fbc640 1 --2- 172.21.15.116:0/2329026740 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbf680a2b80 0x7fbf680a5040 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbf780c56c0 tx=0x7fbf780095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:20.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.332+0000 7fbf7ffff640 1 --2- 172.21.15.116:0/3831170157 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55960e2bf5d0 0x55960e2bfa10 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fbf70009cf0 tx=0x7fbf70011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e3ed68ea637de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:20.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.333+0000 7fbf667fc640 1 -- 172.21.15.116:0/3831170157 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7001e070 con 0x55960e2bf5d0 2024-09-12T03:07:20.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.333+0000 7fbf667fc640 1 -- 172.21.15.116:0/3831170157 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf70009de0 con 0x55960e2bf5d0 2024-09-12T03:07:20.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.333+0000 7fbf667fc640 1 -- 172.21.15.116:0/3831170157 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf700196b0 con 0x55960e2bf5d0 2024-09-12T03:07:20.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.333+0000 7fbf88df8740 1 -- 172.21.15.116:0/3831170157 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55960e2bf5d0 msgr2=0x55960e2bfa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:20.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.333+0000 7fbf88df8740 1 --2- 172.21.15.116:0/3831170157 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55960e2bf5d0 0x55960e2bfa10 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fbf70009cf0 tx=0x7fbf70011810 comp rx=0 tx=0).stop 2024-09-12T03:07:20.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.333+0000 7fbf88df8740 1 -- 172.21.15.116:0/3831170157 shutdown_connections 2024-09-12T03:07:20.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.333+0000 7fbf88df8740 1 --2- 172.21.15.116:0/3831170157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55960e2ca370 0x55960e2cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.333+0000 7fbf88df8740 1 --2- 172.21.15.116:0/3831170157 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55960e2bf5d0 0x55960e2bfa10 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.333+0000 7fbf88df8740 1 --2- 172.21.15.116:0/3831170157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960e2c0b40 0x55960e2bf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.333+0000 7fbf88df8740 1 -- 172.21.15.116:0/3831170157 >> 172.21.15.116:0/3831170157 conn(0x55960e288310 msgr2=0x55960e2bbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:20.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.333+0000 7fbf88df8740 1 -- 172.21.15.116:0/3831170157 shutdown_connections 2024-09-12T03:07:20.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.333+0000 7fbf88df8740 1 -- 172.21.15.116:0/3831170157 wait complete. 2024-09-12T03:07:20.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.334+0000 7fbf88df8740 1 -- start start 2024-09-12T03:07:20.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.334+0000 7fbf88df8740 1 Processor -- start 2024-09-12T03:07:20.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.334+0000 7fbf88df8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55960e2bf5d0 0x55960e2c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.334+0000 7fbf88df8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960e2c0b40 0x55960e2c7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.334+0000 7fbf88df8740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55960e2ca370 0x55960e2c7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.334+0000 7fbf88df8740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55960e2cd070 con 0x55960e2ca370 2024-09-12T03:07:20.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.334+0000 7fbf88df8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55960e1e0060 con 0x55960e2bf5d0 2024-09-12T03:07:20.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.334+0000 7fbf88df8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55960e2cd1f0 con 0x55960e2c0b40 2024-09-12T03:07:20.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.334+0000 7fbf88df8740 10 client.0 _ll_register_callbacks cb 0x55960df338c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:07:20.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.334+0000 7fbf7f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55960e2ca370 0x55960e2c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.334+0000 7fbf7f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55960e2ca370 0x55960e2c7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46812/0 (socket says 172.21.15.116:46812) 2024-09-12T03:07:20.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.334+0000 7fbf7f7fe640 1 -- 172.21.15.116:0/3333587175 learned_addr learned my addr 172.21.15.116:0/3333587175 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:20.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.334+0000 7fbf7f7fe640 1 --2- 172.21.15.116:0/3333587175 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55960e2bf5d0 0x55960e2c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.335+0000 7fbf7ffff640 1 --2- 172.21.15.116:0/3333587175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960e2c0b40 0x55960e2c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.335+0000 7fbf7f7fe640 1 -- 172.21.15.116:0/3333587175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960e2c0b40 msgr2=0x55960e2c7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:20.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.335+0000 7fbf7f7fe640 1 --2- 172.21.15.116:0/3333587175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960e2c0b40 0x55960e2c7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.335+0000 7fbf7f7fe640 1 -- 172.21.15.116:0/3333587175 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55960e2bf5d0 msgr2=0x55960e2c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:20.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.335+0000 7fbf7f7fe640 1 --2- 172.21.15.116:0/3333587175 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55960e2bf5d0 0x55960e2c9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.335+0000 7fbf7f7fe640 1 -- 172.21.15.116:0/3333587175 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55960df963c0 con 0x55960e2ca370 2024-09-12T03:07:20.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.335+0000 7fbf7ffff640 1 --2- 172.21.15.116:0/3333587175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960e2c0b40 0x55960e2c7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:07:20.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.335+0000 7fbf7f7fe640 1 --2- 172.21.15.116:0/3333587175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55960e2ca370 0x55960e2c7600 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fbf74012d20 tx=0x7fbf74004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc4d8cb95d7d1a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:20.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.335+0000 7fbf5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:07:20.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.335+0000 7fbf5f7fe640 1 -- 172.21.15.116:0/3333587175 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7400ee50 con 0x55960e2ca370 2024-09-12T03:07:20.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.335+0000 7fbf5f7fe640 1 -- 172.21.15.116:0/3333587175 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fbf74012050 con 0x55960e2ca370 2024-09-12T03:07:20.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.335+0000 7fbf88df8740 1 -- 172.21.15.116:0/3333587175 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf70009de0 con 0x55960e2ca370 2024-09-12T03:07:20.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.335+0000 7fbf5f7fe640 1 -- 172.21.15.116:0/3333587175 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbf74012250 con 0x55960e2ca370 2024-09-12T03:07:20.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.335+0000 7fbf88df8740 10 client.16380 fetch_fsmap learned FSMap version 25 2024-09-12T03:07:20.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.335+0000 7fbf88df8740 1 -- 172.21.15.116:0/3333587175 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55960e2d8b70 con 0x55960e2ca370 2024-09-12T03:07:20.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.335+0000 7fbf5f7fe640 1 -- 172.21.15.116:0/3333587175 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf70009de0 con 0x55960e2ca370 2024-09-12T03:07:20.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.336+0000 7fbf5f7fe640 1 -- 172.21.15.116:0/3333587175 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbf74004b70 con 0x55960e2ca370 2024-09-12T03:07:20.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.336+0000 7fbf88df8740 10 client.16380 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:07:20.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.336+0000 7fbf88df8740 10 client.16380 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:07:20.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.336+0000 7fbf88df8740 20 client.16380 populate_metadata read hostname 'smithi116' 2024-09-12T03:07:20.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.336+0000 7fbf88df8740 1 --2- 172.21.15.116:0/3333587175 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55960e2da030 0x55960e2dc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.336+0000 7fbf88df8740 4 client.16380 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:07:20.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.336+0000 7fbf88df8740 1 -- 172.21.15.116:0/3333587175 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55960e232860 con 0x55960e2da030 2024-09-12T03:07:20.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.336+0000 7fbf7ffff640 1 --2- 172.21.15.116:0/3333587175 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55960e2da030 0x55960e2dc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.336+0000 7fbf5f7fe640 10 client.16380 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:07:20.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.336+0000 7fbf7ffff640 1 --2- 172.21.15.116:0/3333587175 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55960e2da030 0x55960e2dc420 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5757975391c2d8d3 server_cookie=282dfcb02599c763 in_seq=0 out_seq=0 2024-09-12T03:07:20.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.337+0000 7fbf5f7fe640 1 -- 172.21.15.116:0/3333587175 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55960e232860 con 0x55960e2da030 2024-09-12T03:07:20.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.337+0000 7fbf5f7fe640 10 client.16380 handle_command_reply: tid=0 2024-09-12T03:07:20.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.337+0000 7fbf88df8740 1 client.16380 shutdown 2024-09-12T03:07:20.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.337+0000 7fbf88df8740 1 -- 172.21.15.116:0/3333587175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55960e2ca370 msgr2=0x55960e2c7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:20.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.337+0000 7fbf88df8740 1 --2- 172.21.15.116:0/3333587175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55960e2ca370 0x55960e2c7600 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fbf74012d20 tx=0x7fbf74004730 comp rx=0 tx=0).stop 2024-09-12T03:07:20.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.337+0000 7fbf88df8740 1 -- 172.21.15.116:0/3333587175 shutdown_connections 2024-09-12T03:07:20.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.337+0000 7fbf88df8740 1 --2- 172.21.15.116:0/3333587175 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55960e2da030 0x55960e2dc420 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.337+0000 7fbf88df8740 1 --2- 172.21.15.116:0/3333587175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55960e2ca370 0x55960e2c7600 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.337+0000 7fbf88df8740 1 --2- 172.21.15.116:0/3333587175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960e2c0b40 0x55960e2c7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.337+0000 7fbf88df8740 1 --2- 172.21.15.116:0/3333587175 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55960e2bf5d0 0x55960e2c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.337+0000 7fbf5f7fe640 0 client.16380 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:07:20.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.337+0000 7fbf88df8740 1 -- 172.21.15.116:0/3333587175 >> 172.21.15.116:0/3333587175 conn(0x55960e288310 msgr2=0x55960e2c1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:20.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.337+0000 7fbf88df8740 1 -- 172.21.15.116:0/3333587175 shutdown_connections 2024-09-12T03:07:20.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.338+0000 7fbf88df8740 1 -- 172.21.15.116:0/3333587175 wait complete. 2024-09-12T03:07:20.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.338+0000 7fbf88df8740 20 client.16380 trim_cache size 0 max 16384 2024-09-12T03:07:20.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.350+0000 7fbf88df8740 1 Processor -- start 2024-09-12T03:07:20.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.350+0000 7fbf88df8740 1 -- start start 2024-09-12T03:07:20.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.351+0000 7fbf88df8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55960e2bf5d0 0x55960e2babf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.351+0000 7fbf88df8740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55960e2c0b40 0x55960e2c2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.351+0000 7fbf88df8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960e2ca370 0x55960e2c2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.351+0000 7fbf88df8740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbf801afd50 con 0x55960e2c0b40 2024-09-12T03:07:20.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.351+0000 7fbf88df8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbf801ad620 con 0x55960e2bf5d0 2024-09-12T03:07:20.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.351+0000 7fbf88df8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbf801ab490 con 0x55960e2ca370 2024-09-12T03:07:20.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.351+0000 7fbf7ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55960e2c0b40 0x55960e2c2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.351+0000 7fbf7ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55960e2c0b40 0x55960e2c2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46824/0 (socket says 172.21.15.116:46824) 2024-09-12T03:07:20.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.351+0000 7fbf7ffff640 1 -- 172.21.15.116:0/941985510 learned_addr learned my addr 172.21.15.116:0/941985510 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:20.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.351+0000 7fbf7f7fe640 1 --2- 172.21.15.116:0/941985510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55960e2bf5d0 0x55960e2babf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.351+0000 7fbf7f7fe640 1 --2- 172.21.15.116:0/941985510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960e2ca370 0x55960e2c2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.351+0000 7fbf7ffff640 1 -- 172.21.15.116:0/941985510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960e2ca370 msgr2=0x55960e2c2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:20.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.351+0000 7fbf7ffff640 1 --2- 172.21.15.116:0/941985510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960e2ca370 0x55960e2c2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.351+0000 7fbf7ffff640 1 -- 172.21.15.116:0/941985510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55960e2bf5d0 msgr2=0x55960e2babf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:20.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.351+0000 7fbf7ffff640 1 --2- 172.21.15.116:0/941985510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55960e2bf5d0 0x55960e2babf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.351+0000 7fbf7ffff640 1 -- 172.21.15.116:0/941985510 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55960e2ca0d0 con 0x55960e2c0b40 2024-09-12T03:07:20.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.351+0000 7fbf7ffff640 1 --2- 172.21.15.116:0/941985510 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55960e2c0b40 0x55960e2c2120 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fbf70009ab0 tx=0x7fbf70009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269f3491a1ec5179 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:20.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.351+0000 7fbf65ffb640 1 -- 172.21.15.116:0/941985510 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf70029070 con 0x55960e2c0b40 2024-09-12T03:07:20.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.351+0000 7fbf65ffb640 1 -- 172.21.15.116:0/941985510 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf70005520 con 0x55960e2c0b40 2024-09-12T03:07:20.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.351+0000 7fbf65ffb640 1 -- 172.21.15.116:0/941985510 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7001e040 con 0x55960e2c0b40 2024-09-12T03:07:20.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.352+0000 7fbf88df8740 1 -- 172.21.15.116:0/941985510 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55960e2c0b40 msgr2=0x55960e2c2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:20.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.352+0000 7fbf88df8740 1 --2- 172.21.15.116:0/941985510 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55960e2c0b40 0x55960e2c2120 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fbf70009ab0 tx=0x7fbf70009910 comp rx=0 tx=0).stop 2024-09-12T03:07:20.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.352+0000 7fbf88df8740 1 -- 172.21.15.116:0/941985510 shutdown_connections 2024-09-12T03:07:20.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.352+0000 7fbf88df8740 1 --2- 172.21.15.116:0/941985510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960e2ca370 0x55960e2c2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.352+0000 7fbf88df8740 1 --2- 172.21.15.116:0/941985510 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55960e2c0b40 0x55960e2c2120 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.352+0000 7fbf88df8740 1 --2- 172.21.15.116:0/941985510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55960e2bf5d0 0x55960e2babf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.352+0000 7fbf88df8740 1 -- 172.21.15.116:0/941985510 >> 172.21.15.116:0/941985510 conn(0x55960e288310 msgr2=0x55960e2ba4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:20.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.352+0000 7fbf88df8740 1 -- 172.21.15.116:0/941985510 shutdown_connections 2024-09-12T03:07:20.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.352+0000 7fbf88df8740 1 -- 172.21.15.116:0/941985510 wait complete. 2024-09-12T03:07:20.356 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:20 smithi116 ceph-mon[31410]: pgmap v468: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2024-09-12T03:07:20.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.352+0000 7fbf88df8740 1 -- start start 2024-09-12T03:07:20.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.353+0000 7fbf88df8740 1 Processor -- start 2024-09-12T03:07:20.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.353+0000 7fbf88df8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55960e2bf5d0 0x55960e2d2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.353+0000 7fbf88df8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960e2c0b40 0x55960e2c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.353+0000 7fbf88df8740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55960e2ca370 0x55960e2c4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.353+0000 7fbf88df8740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbf801abae0 con 0x55960e2ca370 2024-09-12T03:07:20.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.353+0000 7fbf88df8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbf801acf50 con 0x55960e2bf5d0 2024-09-12T03:07:20.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.353+0000 7fbf88df8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbf801ac1b0 con 0x55960e2c0b40 2024-09-12T03:07:20.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.353+0000 7fbf88df8740 10 client.0 _ll_register_callbacks cb 0x7fbf74012560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:07:20.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.353+0000 7fbf7f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55960e2ca370 0x55960e2c4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.353+0000 7fbf7f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55960e2ca370 0x55960e2c4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46828/0 (socket says 172.21.15.116:46828) 2024-09-12T03:07:20.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.353+0000 7fbf7f7fe640 1 -- 172.21.15.116:0/2976953472 learned_addr learned my addr 172.21.15.116:0/2976953472 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:20.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.353+0000 7fbf7f7fe640 1 --2- 172.21.15.116:0/2976953472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55960e2bf5d0 0x55960e2d2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.353+0000 7fbf7ffff640 1 --2- 172.21.15.116:0/2976953472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960e2c0b40 0x55960e2c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.354+0000 7fbf7f7fe640 1 -- 172.21.15.116:0/2976953472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960e2c0b40 msgr2=0x55960e2c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:20.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.354+0000 7fbf7f7fe640 1 --2- 172.21.15.116:0/2976953472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960e2c0b40 0x55960e2c3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.354+0000 7fbf7f7fe640 1 -- 172.21.15.116:0/2976953472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55960e2bf5d0 msgr2=0x55960e2d2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:20.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.354+0000 7fbf7f7fe640 1 --2- 172.21.15.116:0/2976953472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55960e2bf5d0 0x55960e2d2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.354+0000 7fbf7f7fe640 1 -- 172.21.15.116:0/2976953472 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55960e2c4a90 con 0x55960e2ca370 2024-09-12T03:07:20.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.354+0000 7fbf7f7fe640 1 --2- 172.21.15.116:0/2976953472 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55960e2ca370 0x55960e2c4260 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009680 tx=0x7fbf74014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0defccc1bd52fc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:20.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.354+0000 7fbf5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:07:20.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.354+0000 7fbf5f7fe640 1 -- 172.21.15.116:0/2976953472 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf74012800 con 0x55960e2ca370 2024-09-12T03:07:20.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.354+0000 7fbf5f7fe640 1 -- 172.21.15.116:0/2976953472 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fbf7401b040 con 0x55960e2ca370 2024-09-12T03:07:20.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.354+0000 7fbf88df8740 1 -- 172.21.15.116:0/2976953472 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf70005520 con 0x55960e2ca370 2024-09-12T03:07:20.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.354+0000 7fbf5f7fe640 1 -- 172.21.15.116:0/2976953472 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbf74044050 con 0x55960e2ca370 2024-09-12T03:07:20.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.354+0000 7fbf88df8740 10 client.16386 fetch_fsmap learned FSMap version 25 2024-09-12T03:07:20.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.354+0000 7fbf88df8740 1 -- 172.21.15.116:0/2976953472 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55960e344c10 con 0x55960e2ca370 2024-09-12T03:07:20.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.354+0000 7fbf5f7fe640 1 -- 172.21.15.116:0/2976953472 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf70005520 con 0x55960e2ca370 2024-09-12T03:07:20.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.355+0000 7fbf5f7fe640 1 -- 172.21.15.116:0/2976953472 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbf740053a0 con 0x55960e2ca370 2024-09-12T03:07:20.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.355+0000 7fbf88df8740 10 client.16386 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:07:20.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.355+0000 7fbf88df8740 10 client.16386 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:07:20.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.355+0000 7fbf88df8740 20 client.16386 populate_metadata read hostname 'smithi116' 2024-09-12T03:07:20.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.355+0000 7fbf88df8740 1 --2- 172.21.15.116:0/2976953472 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55960e345280 0x55960e347670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.355+0000 7fbf88df8740 4 client.16386 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:07:20.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.355+0000 7fbf88df8740 1 -- 172.21.15.116:0/2976953472 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55960e22cd70 con 0x55960e345280 2024-09-12T03:07:20.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.355+0000 7fbf7ffff640 1 --2- 172.21.15.116:0/2976953472 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55960e345280 0x55960e347670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.355+0000 7fbf7ffff640 1 --2- 172.21.15.116:0/2976953472 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55960e345280 0x55960e347670 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8313e30110eb7349 server_cookie=55a5438c5357ca90 in_seq=0 out_seq=0 2024-09-12T03:07:20.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.355+0000 7fbf5f7fe640 10 client.16386 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:07:20.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.356+0000 7fbf5f7fe640 1 -- 172.21.15.116:0/2976953472 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55960e22cd70 con 0x55960e345280 2024-09-12T03:07:20.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.356+0000 7fbf5f7fe640 10 client.16386 handle_command_reply: tid=0 2024-09-12T03:07:20.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.356+0000 7fbf88df8740 1 client.16386 shutdown 2024-09-12T03:07:20.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.356+0000 7fbf88df8740 1 -- 172.21.15.116:0/2976953472 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55960e2ca370 msgr2=0x55960e2c4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:20.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.356+0000 7fbf88df8740 1 --2- 172.21.15.116:0/2976953472 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55960e2ca370 0x55960e2c4260 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009680 tx=0x7fbf74014040 comp rx=0 tx=0).stop 2024-09-12T03:07:20.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.356+0000 7fbf88df8740 1 -- 172.21.15.116:0/2976953472 shutdown_connections 2024-09-12T03:07:20.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.356+0000 7fbf88df8740 1 --2- 172.21.15.116:0/2976953472 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55960e345280 0x55960e347670 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.356+0000 7fbf88df8740 1 --2- 172.21.15.116:0/2976953472 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55960e2ca370 0x55960e2c4260 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.356+0000 7fbf88df8740 1 --2- 172.21.15.116:0/2976953472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960e2c0b40 0x55960e2c3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.356+0000 7fbf5f7fe640 0 client.16386 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:07:20.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.356+0000 7fbf88df8740 1 --2- 172.21.15.116:0/2976953472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55960e2bf5d0 0x55960e2d2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.356+0000 7fbf88df8740 1 -- 172.21.15.116:0/2976953472 >> 172.21.15.116:0/2976953472 conn(0x55960e288310 msgr2=0x55960e2c9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:20.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.356+0000 7fbf88df8740 1 -- 172.21.15.116:0/2976953472 shutdown_connections 2024-09-12T03:07:20.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.356+0000 7fbf88df8740 1 -- 172.21.15.116:0/2976953472 wait complete. 2024-09-12T03:07:20.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.356+0000 7fbf88df8740 20 client.16386 trim_cache size 0 max 16384 2024-09-12T03:07:20.363 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:07:20.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.357+0000 7fbf86a1f640 1 -- 172.21.15.116:0/2329026740 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbf680a2b80 msgr2=0x7fbf680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:20.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.357+0000 7fbf86a1f640 1 --2- 172.21.15.116:0/2329026740 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbf680a2b80 0x7fbf680a5040 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbf780c56c0 tx=0x7fbf780095c0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.357+0000 7fbf86a1f640 1 -- 172.21.15.116:0/2329026740 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf80106f50 msgr2=0x7fbf8019cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:20.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.357+0000 7fbf86a1f640 1 --2- 172.21.15.116:0/2329026740 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf80106f50 0x7fbf8019cd30 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbf78005cc0 tx=0x7fbf780093d0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.357+0000 7fbf86a1f640 1 -- 172.21.15.116:0/2329026740 shutdown_connections 2024-09-12T03:07:20.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.357+0000 7fbf86a1f640 1 --2- 172.21.15.116:0/2329026740 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbf680a2b80 0x7fbf680a5040 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.357+0000 7fbf86a1f640 1 --2- 172.21.15.116:0/2329026740 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf80106f50 0x7fbf8019cd30 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.357+0000 7fbf86a1f640 1 --2- 172.21.15.116:0/2329026740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf80100ca0 0x7fbf801989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.357+0000 7fbf86a1f640 1 --2- 172.21.15.116:0/2329026740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf801002f0 0x7fbf80198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.357+0000 7fbf86a1f640 1 -- 172.21.15.116:0/2329026740 >> 172.21.15.116:0/2329026740 conn(0x7fbf80077210 msgr2=0x7fbf80105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:20.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.357+0000 7fbf86a1f640 1 -- 172.21.15.116:0/2329026740 shutdown_connections 2024-09-12T03:07:20.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:20.357+0000 7fbf86a1f640 1 -- 172.21.15.116:0/2329026740 wait complete. 2024-09-12T03:07:20.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:20 smithi183 ceph-mon[35535]: pgmap v468: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2024-09-12T03:07:20.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:20 smithi177 ceph-mon[35305]: pgmap v468: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2024-09-12T03:07:21.272 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:07:21.518 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:07:22.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.010+0000 7f0640fe6640 1 -- 172.21.15.116:0/4068784540 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f063c102d40 msgr2=0x7f063c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:22.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.010+0000 7f0640fe6640 1 --2- 172.21.15.116:0/4068784540 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f063c102d40 0x7f063c103120 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f062c009900 tx=0x7f062c02a480 comp rx=0 tx=0).stop 2024-09-12T03:07:22.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.010+0000 7f0640fe6640 1 -- 172.21.15.116:0/4068784540 shutdown_connections 2024-09-12T03:07:22.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.010+0000 7f0640fe6640 1 --2- 172.21.15.116:0/4068784540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f063c10c170 0x7f063c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.010+0000 7f0640fe6640 1 --2- 172.21.15.116:0/4068784540 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f063c103660 0x7f063c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.010+0000 7f0640fe6640 1 --2- 172.21.15.116:0/4068784540 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f063c102d40 0x7f063c103120 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.010+0000 7f0640fe6640 1 -- 172.21.15.116:0/4068784540 >> 172.21.15.116:0/4068784540 conn(0x7f063c0fc7a0 msgr2=0x7f063c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:22.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.010+0000 7f0640fe6640 1 -- 172.21.15.116:0/4068784540 shutdown_connections 2024-09-12T03:07:22.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.010+0000 7f0640fe6640 1 -- 172.21.15.116:0/4068784540 wait complete. 2024-09-12T03:07:22.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.011+0000 7f0640fe6640 1 Processor -- start 2024-09-12T03:07:22.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.011+0000 7f0640fe6640 1 -- start start 2024-09-12T03:07:22.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.011+0000 7f0640fe6640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f063c102d40 0x7f063c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:22.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.011+0000 7f0640fe6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f063c103660 0x7f063c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:22.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.011+0000 7f0640fe6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f063c10c170 0x7f063c19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:22.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.011+0000 7f0640fe6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f063c111f70 con 0x7f063c10c170 2024-09-12T03:07:22.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.011+0000 7f0640fe6640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f063c111df0 con 0x7f063c102d40 2024-09-12T03:07:22.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.011+0000 7f0640fe6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f063c1120f0 con 0x7f063c103660 2024-09-12T03:07:22.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.012+0000 7f063ad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f063c10c170 0x7f063c19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:22.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.012+0000 7f063ad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f063c10c170 0x7f063c19cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58764/0 (socket says 172.21.15.116:58764) 2024-09-12T03:07:22.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.012+0000 7f063ad9d640 1 -- 172.21.15.116:0/4197880053 learned_addr learned my addr 172.21.15.116:0/4197880053 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:22.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.012+0000 7f063a59c640 1 --2- 172.21.15.116:0/4197880053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f063c102d40 0x7f063c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:22.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.012+0000 7f0633fff640 1 --2- 172.21.15.116:0/4197880053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f063c103660 0x7f063c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:22.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.012+0000 7f063ad9d640 1 -- 172.21.15.116:0/4197880053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f063c103660 msgr2=0x7f063c198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:22.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.012+0000 7f063ad9d640 1 --2- 172.21.15.116:0/4197880053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f063c103660 0x7f063c198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.012+0000 7f063ad9d640 1 -- 172.21.15.116:0/4197880053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f063c102d40 msgr2=0x7f063c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:22.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.012+0000 7f063ad9d640 1 --2- 172.21.15.116:0/4197880053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f063c102d40 0x7f063c198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.012+0000 7f063ad9d640 1 -- 172.21.15.116:0/4197880053 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063c19d3c0 con 0x7f063c10c170 2024-09-12T03:07:22.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.012+0000 7f063ad9d640 1 --2- 172.21.15.116:0/4197880053 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f063c10c170 0x7f063c19cce0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f063400cad0 tx=0x7f063400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:22.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.012+0000 7f06337fe640 1 -- 172.21.15.116:0/4197880053 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063400ae20 con 0x7f063c10c170 2024-09-12T03:07:22.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.012+0000 7f06337fe640 1 -- 172.21.15.116:0/4197880053 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0634005020 con 0x7f063c10c170 2024-09-12T03:07:22.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.012+0000 7f06337fe640 1 -- 172.21.15.116:0/4197880053 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063400f6f0 con 0x7f063c10c170 2024-09-12T03:07:22.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.012+0000 7f0640fe6640 1 -- 172.21.15.116:0/4197880053 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f063c19d6b0 con 0x7f063c10c170 2024-09-12T03:07:22.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.012+0000 7f0640fe6640 1 -- 172.21.15.116:0/4197880053 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f063c1a4f90 con 0x7f063c10c170 2024-09-12T03:07:22.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.015+0000 7f06337fe640 1 -- 172.21.15.116:0/4197880053 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f06340051d0 con 0x7f063c10c170 2024-09-12T03:07:22.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.015+0000 7f06433bf740 1 Processor -- start 2024-09-12T03:07:22.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.015+0000 7f06433bf740 1 -- start start 2024-09-12T03:07:22.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.015+0000 7f06433bf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e764d72b40 0x55e764d71090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:22.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.015+0000 7f06433bf740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e764d715d0 0x55e764d71a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:22.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.015+0000 7f06433bf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e764d80780 0x55e764d82b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:22.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.015+0000 7f06433bf740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e764cd72f0 con 0x55e764d715d0 2024-09-12T03:07:22.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.015+0000 7f06433bf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e764c7ecf0 con 0x55e764d80780 2024-09-12T03:07:22.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.015+0000 7f06433bf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e764cd32b0 con 0x55e764d72b40 2024-09-12T03:07:22.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.016+0000 7f06337fe640 1 --2- 172.21.15.116:0/4197880053 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f06280a2c00 0x7f06280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:22.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.016+0000 7f063a59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e764d715d0 0x55e764d71a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:22.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.016+0000 7f063a59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e764d715d0 0x55e764d71a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58776/0 (socket says 172.21.15.116:58776) 2024-09-12T03:07:22.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.016+0000 7f063a59c640 1 -- 172.21.15.116:0/1316828862 learned_addr learned my addr 172.21.15.116:0/1316828862 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:22.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.016+0000 7f0633fff640 1 --2- 172.21.15.116:0/1316828862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e764d72b40 0x55e764d71090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:22.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.016+0000 7f0633fff640 1 --2- 172.21.15.116:0/1316828862 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e764d80780 0x55e764d82b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:22.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.016+0000 7f063ad9d640 1 --2- 172.21.15.116:0/4197880053 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f06280a2c00 0x7f06280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:22.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.016+0000 7f06337fe640 1 -- 172.21.15.116:0/4197880053 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f06340cf2b0 con 0x7f063c10c170 2024-09-12T03:07:22.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.016+0000 7f0633fff640 1 -- 172.21.15.116:0/1316828862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e764d72b40 msgr2=0x55e764d71090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:22.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.016+0000 7f0633fff640 1 --2- 172.21.15.116:0/1316828862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e764d72b40 0x55e764d71090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.016+0000 7f0633fff640 1 -- 172.21.15.116:0/1316828862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e764d715d0 msgr2=0x55e764d71a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:22.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.016+0000 7f0633fff640 1 --2- 172.21.15.116:0/1316828862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e764d715d0 0x55e764d71a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.016+0000 7f0633fff640 1 -- 172.21.15.116:0/1316828862 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e764c633a0 con 0x55e764d80780 2024-09-12T03:07:22.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.016+0000 7f063a59c640 1 --2- 172.21.15.116:0/1316828862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e764d715d0 0x55e764d71a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:07:22.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.016+0000 7f063ad9d640 1 --2- 172.21.15.116:0/4197880053 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f06280a2c00 0x7f06280a50c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0634098510 tx=0x7f0634000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:22.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.016+0000 7f0633fff640 1 --2- 172.21.15.116:0/1316828862 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e764d80780 0x55e764d82b70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f063c10e020 tx=0x7f06240046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5690ebe92e9e0d64 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:22.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.016+0000 7f0630ff9640 1 -- 172.21.15.116:0/1316828862 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624013310 con 0x55e764d80780 2024-09-12T03:07:22.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.016+0000 7f0630ff9640 1 -- 172.21.15.116:0/1316828862 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0624005020 con 0x55e764d80780 2024-09-12T03:07:22.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.016+0000 7f0630ff9640 1 -- 172.21.15.116:0/1316828862 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624004a00 con 0x55e764d80780 2024-09-12T03:07:22.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.017+0000 7f06433bf740 1 -- 172.21.15.116:0/1316828862 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e764d80780 msgr2=0x55e764d82b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:22.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.017+0000 7f06433bf740 1 --2- 172.21.15.116:0/1316828862 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e764d80780 0x55e764d82b70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f063c10e020 tx=0x7f06240046c0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.017+0000 7f06433bf740 1 -- 172.21.15.116:0/1316828862 shutdown_connections 2024-09-12T03:07:22.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.017+0000 7f06433bf740 1 --2- 172.21.15.116:0/1316828862 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e764d80780 0x55e764d82b70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.017+0000 7f06433bf740 1 --2- 172.21.15.116:0/1316828862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e764d715d0 0x55e764d71a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.017+0000 7f06433bf740 1 --2- 172.21.15.116:0/1316828862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e764d72b40 0x55e764d71090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.017+0000 7f06433bf740 1 -- 172.21.15.116:0/1316828862 >> 172.21.15.116:0/1316828862 conn(0x55e764d3a310 msgr2=0x55e764d6dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:22.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.017+0000 7f06433bf740 1 -- 172.21.15.116:0/1316828862 shutdown_connections 2024-09-12T03:07:22.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.017+0000 7f06433bf740 1 -- 172.21.15.116:0/1316828862 wait complete. 2024-09-12T03:07:22.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.018+0000 7f06433bf740 1 -- start start 2024-09-12T03:07:22.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.018+0000 7f06433bf740 1 Processor -- start 2024-09-12T03:07:22.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.018+0000 7f06433bf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e764d715d0 0x55e764d80190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:22.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.018+0000 7f06433bf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e764d72b40 0x55e764d7d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:22.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.018+0000 7f06433bf740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e764d80780 0x55e764d7da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:22.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.018+0000 7f06433bf740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e764d83480 con 0x55e764d80780 2024-09-12T03:07:22.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.018+0000 7f06433bf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e764c92060 con 0x55e764d715d0 2024-09-12T03:07:22.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.018+0000 7f06433bf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e764d83600 con 0x55e764d72b40 2024-09-12T03:07:22.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.018+0000 7f06433bf740 10 client.0 _ll_register_callbacks cb 0x55e7649e58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:07:22.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.018+0000 7f0633fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e764d80780 0x55e764d7da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:22.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.018+0000 7f0633fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e764d80780 0x55e764d7da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58782/0 (socket says 172.21.15.116:58782) 2024-09-12T03:07:22.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.018+0000 7f0633fff640 1 -- 172.21.15.116:0/3751183748 learned_addr learned my addr 172.21.15.116:0/3751183748 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:22.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.018+0000 7f063a59c640 1 --2- 172.21.15.116:0/3751183748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e764d72b40 0x55e764d7d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:22.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.018+0000 7f0633fff640 1 --2- 172.21.15.116:0/3751183748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e764d715d0 0x55e764d80190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:22.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.018+0000 7f0633fff640 1 -- 172.21.15.116:0/3751183748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e764d72b40 msgr2=0x55e764d7d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:22.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.018+0000 7f0633fff640 1 --2- 172.21.15.116:0/3751183748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e764d72b40 0x55e764d7d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.018+0000 7f0633fff640 1 -- 172.21.15.116:0/3751183748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e764d715d0 msgr2=0x55e764d80190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:22.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.018+0000 7f0633fff640 1 --2- 172.21.15.116:0/3751183748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e764d715d0 0x55e764d80190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.018+0000 7f0633fff640 1 -- 172.21.15.116:0/3751183748 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e764a483c0 con 0x55e764d80780 2024-09-12T03:07:22.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.019+0000 7f0633fff640 1 --2- 172.21.15.116:0/3751183748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e764d80780 0x55e764d7da50 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f062400b420 tx=0x7f0624010f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6983151893eb4ba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:22.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.019+0000 7f0621ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:07:22.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.019+0000 7f0621ffb640 1 -- 172.21.15.116:0/3751183748 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06240109e0 con 0x55e764d80780 2024-09-12T03:07:22.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.019+0000 7f0621ffb640 1 -- 172.21.15.116:0/3751183748 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f0624010b80 con 0x55e764d80780 2024-09-12T03:07:22.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.019+0000 7f06433bf740 1 -- 172.21.15.116:0/3751183748 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0624005020 con 0x55e764d80780 2024-09-12T03:07:22.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.019+0000 7f0621ffb640 1 -- 172.21.15.116:0/3751183748 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0624010d80 con 0x55e764d80780 2024-09-12T03:07:22.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.019+0000 7f06433bf740 10 client.16398 fetch_fsmap learned FSMap version 25 2024-09-12T03:07:22.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.019+0000 7f06433bf740 1 -- 172.21.15.116:0/3751183748 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e764d8ef50 con 0x55e764d80780 2024-09-12T03:07:22.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.019+0000 7f0621ffb640 1 -- 172.21.15.116:0/3751183748 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0624005020 con 0x55e764d80780 2024-09-12T03:07:22.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.019+0000 7f0621ffb640 1 -- 172.21.15.116:0/3751183748 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f06240035f0 con 0x55e764d80780 2024-09-12T03:07:22.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.019+0000 7f06433bf740 10 client.16398 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:07:22.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.019+0000 7f06433bf740 10 client.16398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:07:22.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.019+0000 7f06433bf740 20 client.16398 populate_metadata read hostname 'smithi116' 2024-09-12T03:07:22.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.019+0000 7f06433bf740 1 --2- 172.21.15.116:0/3751183748 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e764d902a0 0x55e764d92690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:22.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.019+0000 7f06433bf740 4 client.16398 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:07:22.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.019+0000 7f06433bf740 1 -- 172.21.15.116:0/3751183748 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e764ce4860 con 0x55e764d902a0 2024-09-12T03:07:22.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.020+0000 7f063a59c640 1 --2- 172.21.15.116:0/3751183748 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e764d902a0 0x55e764d92690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:22.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.020+0000 7f0621ffb640 10 client.16398 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:22.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.020+0000 7f063a59c640 1 --2- 172.21.15.116:0/3751183748 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e764d902a0 0x55e764d92690 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c0c913a29f3593f server_cookie=7a8fea2e927142e7 in_seq=0 out_seq=0 2024-09-12T03:07:22.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.021+0000 7f0621ffb640 1 -- 172.21.15.116:0/3751183748 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e764ce4860 con 0x55e764d902a0 2024-09-12T03:07:22.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.021+0000 7f0621ffb640 10 client.16398 handle_command_reply: tid=0 2024-09-12T03:07:22.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.021+0000 7f06433bf740 1 client.16398 shutdown 2024-09-12T03:07:22.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.021+0000 7f06433bf740 1 -- 172.21.15.116:0/3751183748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e764d80780 msgr2=0x55e764d7da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:22.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.021+0000 7f06433bf740 1 --2- 172.21.15.116:0/3751183748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e764d80780 0x55e764d7da50 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f062400b420 tx=0x7f0624010f60 comp rx=0 tx=0).stop 2024-09-12T03:07:22.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.021+0000 7f06433bf740 1 -- 172.21.15.116:0/3751183748 shutdown_connections 2024-09-12T03:07:22.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.021+0000 7f06433bf740 1 --2- 172.21.15.116:0/3751183748 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e764d902a0 0x55e764d92690 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.021+0000 7f06433bf740 1 --2- 172.21.15.116:0/3751183748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e764d80780 0x55e764d7da50 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.021+0000 7f0621ffb640 0 client.16398 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:22.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.021+0000 7f06433bf740 1 --2- 172.21.15.116:0/3751183748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e764d72b40 0x55e764d7d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.021+0000 7f06433bf740 1 --2- 172.21.15.116:0/3751183748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e764d715d0 0x55e764d80190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.021+0000 7f06433bf740 1 -- 172.21.15.116:0/3751183748 >> 172.21.15.116:0/3751183748 conn(0x55e764d3a310 msgr2=0x55e764d73960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:22.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.021+0000 7f06433bf740 1 -- 172.21.15.116:0/3751183748 shutdown_connections 2024-09-12T03:07:22.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.021+0000 7f06433bf740 1 -- 172.21.15.116:0/3751183748 wait complete. 2024-09-12T03:07:22.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.021+0000 7f06433bf740 20 client.16398 trim_cache size 0 max 16384 2024-09-12T03:07:22.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.034+0000 7f06433bf740 1 Processor -- start 2024-09-12T03:07:22.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.034+0000 7f06433bf740 1 -- start start 2024-09-12T03:07:22.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.034+0000 7f06433bf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e764d715d0 0x55e764d8c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:22.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.034+0000 7f06433bf740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e764d72b40 0x55e764d7f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:22.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.034+0000 7f06433bf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e764d80780 0x55e764d97d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:22.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.034+0000 7f06433bf740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f063c1adc70 con 0x55e764d72b40 2024-09-12T03:07:22.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.034+0000 7f06433bf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f063c1ab540 con 0x55e764d715d0 2024-09-12T03:07:22.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.034+0000 7f06433bf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f063c1a93b0 con 0x55e764d80780 2024-09-12T03:07:22.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.034+0000 7f063a59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e764d72b40 0x55e764d7f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:22.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.034+0000 7f063a59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e764d72b40 0x55e764d7f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58784/0 (socket says 172.21.15.116:58784) 2024-09-12T03:07:22.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.034+0000 7f063a59c640 1 -- 172.21.15.116:0/399931240 learned_addr learned my addr 172.21.15.116:0/399931240 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:22.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.034+0000 7f0633fff640 1 --2- 172.21.15.116:0/399931240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e764d715d0 0x55e764d8c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:22.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.034+0000 7f0633fff640 1 --2- 172.21.15.116:0/399931240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e764d80780 0x55e764d97d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:22.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.034+0000 7f063a59c640 1 -- 172.21.15.116:0/399931240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e764d80780 msgr2=0x55e764d97d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:22.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.034+0000 7f063a59c640 1 --2- 172.21.15.116:0/399931240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e764d80780 0x55e764d97d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.034+0000 7f063a59c640 1 -- 172.21.15.116:0/399931240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e764d715d0 msgr2=0x55e764d8c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:22.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.034+0000 7f063a59c640 1 --2- 172.21.15.116:0/399931240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e764d715d0 0x55e764d8c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.034+0000 7f063a59c640 1 -- 172.21.15.116:0/399931240 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e764d72300 con 0x55e764d72b40 2024-09-12T03:07:22.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.034+0000 7f063a59c640 1 --2- 172.21.15.116:0/399931240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e764d72b40 0x55e764d7f3b0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f062c009900 tx=0x7f062c031bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c284cfb7b0b2498f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:22.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.035+0000 7f0623fff640 1 -- 172.21.15.116:0/399931240 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c035040 con 0x55e764d72b40 2024-09-12T03:07:22.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.035+0000 7f0623fff640 1 -- 172.21.15.116:0/399931240 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f062c005500 con 0x55e764d72b40 2024-09-12T03:07:22.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.035+0000 7f0623fff640 1 -- 172.21.15.116:0/399931240 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c004cd0 con 0x55e764d72b40 2024-09-12T03:07:22.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.035+0000 7f06433bf740 1 -- 172.21.15.116:0/399931240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e764d72b40 msgr2=0x55e764d7f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:22.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.035+0000 7f06433bf740 1 --2- 172.21.15.116:0/399931240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e764d72b40 0x55e764d7f3b0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f062c009900 tx=0x7f062c031bb0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.035+0000 7f06433bf740 1 -- 172.21.15.116:0/399931240 shutdown_connections 2024-09-12T03:07:22.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.035+0000 7f06433bf740 1 --2- 172.21.15.116:0/399931240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e764d80780 0x55e764d97d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.035+0000 7f06433bf740 1 --2- 172.21.15.116:0/399931240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e764d72b40 0x55e764d7f3b0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.035+0000 7f06433bf740 1 --2- 172.21.15.116:0/399931240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e764d715d0 0x55e764d8c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.035+0000 7f06433bf740 1 -- 172.21.15.116:0/399931240 >> 172.21.15.116:0/399931240 conn(0x55e764d3a310 msgr2=0x55e764d7eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:22.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.035+0000 7f06433bf740 1 -- 172.21.15.116:0/399931240 shutdown_connections 2024-09-12T03:07:22.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.035+0000 7f06433bf740 1 -- 172.21.15.116:0/399931240 wait complete. 2024-09-12T03:07:22.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.035+0000 7f06433bf740 1 -- start start 2024-09-12T03:07:22.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.036+0000 7f06433bf740 1 Processor -- start 2024-09-12T03:07:22.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.036+0000 7f06433bf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e764d715d0 0x55e764d83930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:22.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.036+0000 7f06433bf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e764d72b40 0x55e764d83e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:22.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.036+0000 7f06433bf740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e764d80780 0x55e764df8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:22.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.036+0000 7f06433bf740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f063c1a9a00 con 0x55e764d80780 2024-09-12T03:07:22.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.036+0000 7f06433bf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f063c1aae70 con 0x55e764d72b40 2024-09-12T03:07:22.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.036+0000 7f06433bf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f063c1aa0d0 con 0x55e764d715d0 2024-09-12T03:07:22.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.036+0000 7f06433bf740 10 client.0 _ll_register_callbacks cb 0x7f062c0314d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:07:22.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.036+0000 7f0633fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e764d80780 0x55e764df8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:22.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.036+0000 7f0633fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e764d80780 0x55e764df8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58786/0 (socket says 172.21.15.116:58786) 2024-09-12T03:07:22.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.036+0000 7f0633fff640 1 -- 172.21.15.116:0/4176530395 learned_addr learned my addr 172.21.15.116:0/4176530395 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:22.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.036+0000 7f0633fff640 1 --2- 172.21.15.116:0/4176530395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e764d715d0 0x55e764d83930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:22.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.036+0000 7f063a59c640 1 --2- 172.21.15.116:0/4176530395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e764d72b40 0x55e764d83e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:22.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.036+0000 7f0633fff640 1 -- 172.21.15.116:0/4176530395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e764d715d0 msgr2=0x55e764d83930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:22.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.036+0000 7f0633fff640 1 --2- 172.21.15.116:0/4176530395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e764d715d0 0x55e764d83930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.036+0000 7f0633fff640 1 -- 172.21.15.116:0/4176530395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e764d72b40 msgr2=0x55e764d83e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:22.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.036+0000 7f0633fff640 1 --2- 172.21.15.116:0/4176530395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e764d72b40 0x55e764d83e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.036+0000 7f0633fff640 1 -- 172.21.15.116:0/4176530395 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e764df9240 con 0x55e764d80780 2024-09-12T03:07:22.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.036+0000 7f063a59c640 1 --2- 172.21.15.116:0/4176530395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e764d72b40 0x55e764d83e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:07:22.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.036+0000 7f0633fff640 1 --2- 172.21.15.116:0/4176530395 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e764d80780 0x55e764df8d00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f062400bbf0 tx=0x7f0624031040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df72460210def9ea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:22.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.036+0000 7f0621ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:07:22.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.036+0000 7f0621ffb640 1 -- 172.21.15.116:0/4176530395 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06240037f0 con 0x55e764d80780 2024-09-12T03:07:22.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.037+0000 7f0621ffb640 1 -- 172.21.15.116:0/4176530395 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f0624010b80 con 0x55e764d80780 2024-09-12T03:07:22.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.037+0000 7f06433bf740 1 -- 172.21.15.116:0/4176530395 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f062c005500 con 0x55e764d80780 2024-09-12T03:07:22.054 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:07:22.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.037+0000 7f0621ffb640 1 -- 172.21.15.116:0/4176530395 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0624010d80 con 0x55e764d80780 2024-09-12T03:07:22.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.037+0000 7f06433bf740 10 client.16404 fetch_fsmap learned FSMap version 25 2024-09-12T03:07:22.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.037+0000 7f06433bf740 1 -- 172.21.15.116:0/4176530395 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e764dfa300 con 0x55e764d80780 2024-09-12T03:07:22.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.037+0000 7f0621ffb640 1 -- 172.21.15.116:0/4176530395 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f062c005500 con 0x55e764d80780 2024-09-12T03:07:22.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.037+0000 7f0621ffb640 1 -- 172.21.15.116:0/4176530395 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0624003c30 con 0x55e764d80780 2024-09-12T03:07:22.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.037+0000 7f06433bf740 10 client.16404 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:07:22.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.037+0000 7f06433bf740 10 client.16404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:07:22.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.037+0000 7f06433bf740 20 client.16404 populate_metadata read hostname 'smithi116' 2024-09-12T03:07:22.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.037+0000 7f06433bf740 1 --2- 172.21.15.116:0/4176530395 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e764dfaa20 0x55e764dfce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:22.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.037+0000 7f06433bf740 4 client.16404 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:07:22.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.037+0000 7f06433bf740 1 -- 172.21.15.116:0/4176530395 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e764cded70 con 0x55e764dfaa20 2024-09-12T03:07:22.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.037+0000 7f063a59c640 1 --2- 172.21.15.116:0/4176530395 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e764dfaa20 0x55e764dfce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:22.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.038+0000 7f063a59c640 1 --2- 172.21.15.116:0/4176530395 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e764dfaa20 0x55e764dfce40 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74a7d9499cb72ba6 server_cookie=baf93e6a96ff98e9 in_seq=0 out_seq=0 2024-09-12T03:07:22.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.038+0000 7f0621ffb640 10 client.16404 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:22.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.038+0000 7f0621ffb640 1 -- 172.21.15.116:0/4176530395 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e764cded70 con 0x55e764dfaa20 2024-09-12T03:07:22.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.038+0000 7f0621ffb640 10 client.16404 handle_command_reply: tid=0 2024-09-12T03:07:22.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.038+0000 7f06433bf740 1 client.16404 shutdown 2024-09-12T03:07:22.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.038+0000 7f06433bf740 1 -- 172.21.15.116:0/4176530395 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e764d80780 msgr2=0x55e764df8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:22.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.038+0000 7f06433bf740 1 --2- 172.21.15.116:0/4176530395 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e764d80780 0x55e764df8d00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f062400bbf0 tx=0x7f0624031040 comp rx=0 tx=0).stop 2024-09-12T03:07:22.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.038+0000 7f06433bf740 1 -- 172.21.15.116:0/4176530395 shutdown_connections 2024-09-12T03:07:22.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.038+0000 7f06433bf740 1 --2- 172.21.15.116:0/4176530395 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e764dfaa20 0x55e764dfce40 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.038+0000 7f06433bf740 1 --2- 172.21.15.116:0/4176530395 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e764d80780 0x55e764df8d00 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.038+0000 7f0621ffb640 0 client.16404 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:22.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.038+0000 7f06433bf740 1 --2- 172.21.15.116:0/4176530395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e764d72b40 0x55e764d83e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.038+0000 7f06433bf740 1 --2- 172.21.15.116:0/4176530395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e764d715d0 0x55e764d83930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.038+0000 7f06433bf740 1 -- 172.21.15.116:0/4176530395 >> 172.21.15.116:0/4176530395 conn(0x55e764d3a310 msgr2=0x55e764d7f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:22.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.038+0000 7f06433bf740 1 -- 172.21.15.116:0/4176530395 shutdown_connections 2024-09-12T03:07:22.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.038+0000 7f06433bf740 1 -- 172.21.15.116:0/4176530395 wait complete. 2024-09-12T03:07:22.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.038+0000 7f06433bf740 20 client.16404 trim_cache size 0 max 16384 2024-09-12T03:07:22.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.039+0000 7f0640fe6640 1 -- 172.21.15.116:0/4197880053 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f06280a2c00 msgr2=0x7f06280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:22.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.039+0000 7f0640fe6640 1 --2- 172.21.15.116:0/4197880053 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f06280a2c00 0x7f06280a50c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0634098510 tx=0x7f0634000f40 comp rx=0 tx=0).stop 2024-09-12T03:07:22.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.039+0000 7f0640fe6640 1 -- 172.21.15.116:0/4197880053 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f063c10c170 msgr2=0x7f063c19cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:22.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.039+0000 7f0640fe6640 1 --2- 172.21.15.116:0/4197880053 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f063c10c170 0x7f063c19cce0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f063400cad0 tx=0x7f063400cf90 comp rx=0 tx=0).stop 2024-09-12T03:07:22.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.039+0000 7f0640fe6640 1 -- 172.21.15.116:0/4197880053 shutdown_connections 2024-09-12T03:07:22.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.039+0000 7f0640fe6640 1 --2- 172.21.15.116:0/4197880053 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f06280a2c00 0x7f06280a50c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.039+0000 7f0640fe6640 1 --2- 172.21.15.116:0/4197880053 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f063c10c170 0x7f063c19cce0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.039+0000 7f0640fe6640 1 --2- 172.21.15.116:0/4197880053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f063c103660 0x7f063c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.039+0000 7f0640fe6640 1 --2- 172.21.15.116:0/4197880053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f063c102d40 0x7f063c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:22.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.039+0000 7f0640fe6640 1 -- 172.21.15.116:0/4197880053 >> 172.21.15.116:0/4197880053 conn(0x7f063c0fc7a0 msgr2=0x7f063c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:22.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.039+0000 7f0640fe6640 1 -- 172.21.15.116:0/4197880053 shutdown_connections 2024-09-12T03:07:22.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:22.039+0000 7f0640fe6640 1 -- 172.21.15.116:0/4197880053 wait complete. 2024-09-12T03:07:22.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:22 smithi116 ceph-mon[31410]: pgmap v469: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.5 KiB/s wr, 10 op/s 2024-09-12T03:07:22.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:22 smithi183 ceph-mon[35535]: pgmap v469: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.5 KiB/s wr, 10 op/s 2024-09-12T03:07:22.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:22 smithi177 ceph-mon[35305]: pgmap v469: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.5 KiB/s wr, 10 op/s 2024-09-12T03:07:23.931 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:07:23.932 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:07:24.182 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:07:24.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:24 smithi116 ceph-mon[31410]: pgmap v470: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.0 KiB/s wr, 8 op/s 2024-09-12T03:07:24.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.676+0000 7f883cc5c640 1 -- 172.21.15.116:0/1305901038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f883810c170 msgr2=0x7f883810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:24.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.676+0000 7f883cc5c640 1 --2- 172.21.15.116:0/1305901038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f883810c170 0x7f883810e610 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f8830009900 tx=0x7f883002a480 comp rx=0 tx=0).stop 2024-09-12T03:07:24.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.676+0000 7f883cc5c640 1 -- 172.21.15.116:0/1305901038 shutdown_connections 2024-09-12T03:07:24.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.676+0000 7f883cc5c640 1 --2- 172.21.15.116:0/1305901038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f883810c170 0x7f883810e610 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.676+0000 7f883cc5c640 1 --2- 172.21.15.116:0/1305901038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8838103660 0x7f883810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.676+0000 7f883cc5c640 1 --2- 172.21.15.116:0/1305901038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8838102d40 0x7f8838103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.676+0000 7f883cc5c640 1 -- 172.21.15.116:0/1305901038 >> 172.21.15.116:0/1305901038 conn(0x7f88380fc7a0 msgr2=0x7f88380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:24.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.676+0000 7f883cc5c640 1 -- 172.21.15.116:0/1305901038 shutdown_connections 2024-09-12T03:07:24.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.676+0000 7f883cc5c640 1 -- 172.21.15.116:0/1305901038 wait complete. 2024-09-12T03:07:24.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.677+0000 7f883cc5c640 1 Processor -- start 2024-09-12T03:07:24.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.677+0000 7f883cc5c640 1 -- start start 2024-09-12T03:07:24.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.677+0000 7f883cc5c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8838102d40 0x7f8838198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:24.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.677+0000 7f883cc5c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8838103660 0x7f8838198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:24.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.677+0000 7f883cc5c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f883810c170 0x7f883819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:24.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.677+0000 7f883cc5c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8838111f70 con 0x7f883810c170 2024-09-12T03:07:24.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.677+0000 7f883cc5c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8838111df0 con 0x7f8838103660 2024-09-12T03:07:24.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.677+0000 7f883cc5c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f88381120f0 con 0x7f8838102d40 2024-09-12T03:07:24.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.677+0000 7f8836d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f883810c170 0x7f883819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:24.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.677+0000 7f8836d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f883810c170 0x7f883819cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58798/0 (socket says 172.21.15.116:58798) 2024-09-12T03:07:24.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.677+0000 7f8836d9d640 1 -- 172.21.15.116:0/3055805106 learned_addr learned my addr 172.21.15.116:0/3055805106 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:24.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.677+0000 7f883659c640 1 --2- 172.21.15.116:0/3055805106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8838102d40 0x7f8838198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:24.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.677+0000 7f8835d9b640 1 --2- 172.21.15.116:0/3055805106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8838103660 0x7f8838198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:24.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.677+0000 7f8836d9d640 1 -- 172.21.15.116:0/3055805106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8838102d40 msgr2=0x7f8838198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:24.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.677+0000 7f8836d9d640 1 --2- 172.21.15.116:0/3055805106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8838102d40 0x7f8838198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.677+0000 7f8836d9d640 1 -- 172.21.15.116:0/3055805106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8838103660 msgr2=0x7f8838198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:24.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.677+0000 7f8836d9d640 1 --2- 172.21.15.116:0/3055805106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8838103660 0x7f8838198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.677+0000 7f8836d9d640 1 -- 172.21.15.116:0/3055805106 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883819d3c0 con 0x7f883810c170 2024-09-12T03:07:24.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.677+0000 7f8836d9d640 1 --2- 172.21.15.116:0/3055805106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f883810c170 0x7f883819cce0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f8830005ab0 tx=0x7f8830004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:24.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.678+0000 7f88277fe640 1 -- 172.21.15.116:0/3055805106 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883003f030 con 0x7f883810c170 2024-09-12T03:07:24.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.678+0000 7f883cc5c640 1 -- 172.21.15.116:0/3055805106 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f883819d650 con 0x7f883810c170 2024-09-12T03:07:24.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.678+0000 7f88277fe640 1 -- 172.21.15.116:0/3055805106 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8830003ad0 con 0x7f883810c170 2024-09-12T03:07:24.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.678+0000 7f88277fe640 1 -- 172.21.15.116:0/3055805106 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8830035070 con 0x7f883810c170 2024-09-12T03:07:24.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.678+0000 7f883cc5c640 1 -- 172.21.15.116:0/3055805106 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88381a4f90 con 0x7f883810c170 2024-09-12T03:07:24.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.680+0000 7f88277fe640 1 -- 172.21.15.116:0/3055805106 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8830042030 con 0x7f883810c170 2024-09-12T03:07:24.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.680+0000 7f883f035740 1 Processor -- start 2024-09-12T03:07:24.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.680+0000 7f883f035740 1 -- start start 2024-09-12T03:07:24.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.681+0000 7f883f035740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c53ae8a40 0x555c53ae8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:24.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.681+0000 7f88277fe640 1 --2- 172.21.15.116:0/3055805106 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f88200a2b80 0x7f88200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:24.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.681+0000 7f883f035740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c53ae9360 0x555c53af7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:24.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.681+0000 7f883f035740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c53af8150 0x555c53afa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:24.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.681+0000 7f883f035740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555c53a4e280 con 0x555c53ae8a40 2024-09-12T03:07:24.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.681+0000 7f883f035740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x555c539f5b10 con 0x555c53af8150 2024-09-12T03:07:24.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.681+0000 7f883f035740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555c53a4a240 con 0x555c53ae9360 2024-09-12T03:07:24.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.681+0000 7f883659c640 1 --2- 172.21.15.116:0/3055805106 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f88200a2b80 0x7f88200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:24.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.681+0000 7f8835d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c53ae8a40 0x555c53ae8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:24.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.681+0000 7f8835d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c53ae8a40 0x555c53ae8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58808/0 (socket says 172.21.15.116:58808) 2024-09-12T03:07:24.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.681+0000 7f8835d9b640 1 -- 172.21.15.116:0/83329217 learned_addr learned my addr 172.21.15.116:0/83329217 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:24.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.681+0000 7f88277fe640 1 -- 172.21.15.116:0/3055805106 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f88300ef9a0 con 0x7f883810c170 2024-09-12T03:07:24.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.681+0000 7f8836d9d640 1 --2- 172.21.15.116:0/83329217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c53af8150 0x555c53afa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:24.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.681+0000 7f8835d9b640 1 --2- 172.21.15.116:0/83329217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c53ae9360 0x555c53af7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:24.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.681+0000 7f8835d9b640 1 -- 172.21.15.116:0/83329217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c53ae9360 msgr2=0x555c53af7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:24.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.681+0000 7f8835d9b640 1 --2- 172.21.15.116:0/83329217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c53ae9360 0x555c53af7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.681+0000 7f8835d9b640 1 -- 172.21.15.116:0/83329217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c53af8150 msgr2=0x555c53afa540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:24.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.681+0000 7f8835d9b640 1 --2- 172.21.15.116:0/83329217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c53af8150 0x555c53afa540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.681+0000 7f8835d9b640 1 -- 172.21.15.116:0/83329217 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c53a021f0 con 0x555c53ae8a40 2024-09-12T03:07:24.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.681+0000 7f883659c640 1 --2- 172.21.15.116:0/3055805106 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f88200a2b80 0x7f88200a5040 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8828005fd0 tx=0x7f8828009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:24.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.681+0000 7f8835d9b640 1 --2- 172.21.15.116:0/83329217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c53ae8a40 0x555c53ae8e20 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f882c0058c0 tx=0x7f882c00b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb4d19093859c44 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:24.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.682+0000 7f8824ff9640 1 -- 172.21.15.116:0/83329217 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c019030 con 0x555c53ae8a40 2024-09-12T03:07:24.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.682+0000 7f8824ff9640 1 -- 172.21.15.116:0/83329217 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f882c011920 con 0x555c53ae8a40 2024-09-12T03:07:24.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.682+0000 7f8824ff9640 1 -- 172.21.15.116:0/83329217 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c01d840 con 0x555c53ae8a40 2024-09-12T03:07:24.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.682+0000 7f883f035740 1 -- 172.21.15.116:0/83329217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c53ae8a40 msgr2=0x555c53ae8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:24.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.682+0000 7f883f035740 1 --2- 172.21.15.116:0/83329217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c53ae8a40 0x555c53ae8e20 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f882c0058c0 tx=0x7f882c00b620 comp rx=0 tx=0).stop 2024-09-12T03:07:24.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.682+0000 7f883f035740 1 -- 172.21.15.116:0/83329217 shutdown_connections 2024-09-12T03:07:24.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.682+0000 7f883f035740 1 --2- 172.21.15.116:0/83329217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c53af8150 0x555c53afa540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.682+0000 7f883f035740 1 --2- 172.21.15.116:0/83329217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c53ae9360 0x555c53af7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.682+0000 7f883f035740 1 --2- 172.21.15.116:0/83329217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c53ae8a40 0x555c53ae8e20 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.682+0000 7f883f035740 1 -- 172.21.15.116:0/83329217 >> 172.21.15.116:0/83329217 conn(0x555c53ae50f0 msgr2=0x555c53ae54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:24.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.682+0000 7f883f035740 1 -- 172.21.15.116:0/83329217 shutdown_connections 2024-09-12T03:07:24.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.682+0000 7f883f035740 1 -- 172.21.15.116:0/83329217 wait complete. 2024-09-12T03:07:24.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.683+0000 7f883f035740 1 -- start start 2024-09-12T03:07:24.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.683+0000 7f883f035740 1 Processor -- start 2024-09-12T03:07:24.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.683+0000 7f883f035740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c53ae8a40 0x555c53af5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:24.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.683+0000 7f883f035740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c53ae9360 0x555c53af5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:24.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.683+0000 7f883f035740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c53af8150 0x555c53af5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:24.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.683+0000 7f883f035740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555c53afad20 con 0x555c53ae8a40 2024-09-12T03:07:24.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.683+0000 7f883f035740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x555c53a08f10 con 0x555c53ae9360 2024-09-12T03:07:24.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.683+0000 7f883f035740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555c53afaea0 con 0x555c53af8150 2024-09-12T03:07:24.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.683+0000 7f883f035740 10 client.0 _ll_register_callbacks cb 0x555c539d1a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:07:24.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.683+0000 7f883659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c53ae8a40 0x555c53af5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:24.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.683+0000 7f883659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c53ae8a40 0x555c53af5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58816/0 (socket says 172.21.15.116:58816) 2024-09-12T03:07:24.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.683+0000 7f883659c640 1 -- 172.21.15.116:0/597146153 learned_addr learned my addr 172.21.15.116:0/597146153 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:24.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.683+0000 7f883659c640 1 -- 172.21.15.116:0/597146153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c53af8150 msgr2=0x555c53af5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:24.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.683+0000 7f883659c640 1 --2- 172.21.15.116:0/597146153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c53af8150 0x555c53af5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.683+0000 7f8835d9b640 1 --2- 172.21.15.116:0/597146153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c53ae9360 0x555c53af5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:24.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.683+0000 7f883659c640 1 -- 172.21.15.116:0/597146153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c53ae9360 msgr2=0x555c53af5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:24.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.683+0000 7f883659c640 1 --2- 172.21.15.116:0/597146153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c53ae9360 0x555c53af5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.684+0000 7f883659c640 1 -- 172.21.15.116:0/597146153 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c537bf6a0 con 0x555c53ae8a40 2024-09-12T03:07:24.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.684+0000 7f8835d9b640 1 --2- 172.21.15.116:0/597146153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c53ae9360 0x555c53af5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:07:24.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.684+0000 7f883659c640 1 --2- 172.21.15.116:0/597146153 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c53ae8a40 0x555c53af5000 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f882800ec00 tx=0x7f8828004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6d21729ba9f16a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:24.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.684+0000 7f881dffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:07:24.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.684+0000 7f881dffb640 1 -- 172.21.15.116:0/597146153 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882801dc50 con 0x555c53ae8a40 2024-09-12T03:07:24.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.684+0000 7f883f035740 1 -- 172.21.15.116:0/597146153 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f882c011920 con 0x555c53ae8a40 2024-09-12T03:07:24.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.684+0000 7f881dffb640 1 -- 172.21.15.116:0/597146153 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f882801ddf0 con 0x555c53ae8a40 2024-09-12T03:07:24.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.684+0000 7f881dffb640 1 -- 172.21.15.116:0/597146153 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8828003970 con 0x555c53ae8a40 2024-09-12T03:07:24.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.684+0000 7f883f035740 10 client.16416 fetch_fsmap learned FSMap version 25 2024-09-12T03:07:24.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.684+0000 7f883f035740 1 -- 172.21.15.116:0/597146153 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555c53b06990 con 0x555c53ae8a40 2024-09-12T03:07:24.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.684+0000 7f881dffb640 1 -- 172.21.15.116:0/597146153 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f882c011920 con 0x555c53ae8a40 2024-09-12T03:07:24.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.684+0000 7f881dffb640 1 -- 172.21.15.116:0/597146153 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f882801e4c0 con 0x555c53ae8a40 2024-09-12T03:07:24.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.684+0000 7f883f035740 10 client.16416 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:07:24.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.684+0000 7f883f035740 10 client.16416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:07:24.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.684+0000 7f883f035740 20 client.16416 populate_metadata read hostname 'smithi116' 2024-09-12T03:07:24.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.684+0000 7f883f035740 1 --2- 172.21.15.116:0/597146153 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x555c53b07ef0 0x555c53b0a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:24.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.684+0000 7f883f035740 4 client.16416 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:07:24.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.684+0000 7f883f035740 1 -- 172.21.15.116:0/597146153 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c53a5b7f0 con 0x555c53b07ef0 2024-09-12T03:07:24.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.685+0000 7f8835d9b640 1 --2- 172.21.15.116:0/597146153 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x555c53b07ef0 0x555c53b0a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:24.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.685+0000 7f881dffb640 10 client.16416 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:24.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.685+0000 7f8835d9b640 1 --2- 172.21.15.116:0/597146153 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x555c53b07ef0 0x555c53b0a390 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b39e8382892090a8 server_cookie=5de50cf6d9d98b4f in_seq=0 out_seq=0 2024-09-12T03:07:24.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.686+0000 7f881dffb640 1 -- 172.21.15.116:0/597146153 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555c53a5b7f0 con 0x555c53b07ef0 2024-09-12T03:07:24.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.686+0000 7f881dffb640 10 client.16416 handle_command_reply: tid=0 2024-09-12T03:07:24.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.686+0000 7f883f035740 1 client.16416 shutdown 2024-09-12T03:07:24.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.686+0000 7f883f035740 1 -- 172.21.15.116:0/597146153 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c53ae8a40 msgr2=0x555c53af5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:24.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.686+0000 7f883f035740 1 --2- 172.21.15.116:0/597146153 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c53ae8a40 0x555c53af5000 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f882800ec00 tx=0x7f8828004ab0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.686+0000 7f883f035740 1 -- 172.21.15.116:0/597146153 shutdown_connections 2024-09-12T03:07:24.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.686+0000 7f883f035740 1 --2- 172.21.15.116:0/597146153 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x555c53b07ef0 0x555c53b0a390 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.686+0000 7f883f035740 1 --2- 172.21.15.116:0/597146153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c53af8150 0x555c53af5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.686+0000 7f883f035740 1 --2- 172.21.15.116:0/597146153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c53ae9360 0x555c53af5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.686+0000 7f881dffb640 0 client.16416 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:24.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.686+0000 7f883f035740 1 --2- 172.21.15.116:0/597146153 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c53ae8a40 0x555c53af5000 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.686+0000 7f883f035740 1 -- 172.21.15.116:0/597146153 >> 172.21.15.116:0/597146153 conn(0x555c53ae50f0 msgr2=0x555c53aeae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:24.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.686+0000 7f883f035740 1 -- 172.21.15.116:0/597146153 shutdown_connections 2024-09-12T03:07:24.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.686+0000 7f883f035740 1 -- 172.21.15.116:0/597146153 wait complete. 2024-09-12T03:07:24.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.686+0000 7f883f035740 20 client.16416 trim_cache size 0 max 16384 2024-09-12T03:07:24.709 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:24 smithi183 ceph-mon[35535]: pgmap v470: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.0 KiB/s wr, 8 op/s 2024-09-12T03:07:24.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.703+0000 7f883f035740 1 Processor -- start 2024-09-12T03:07:24.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.703+0000 7f883f035740 1 -- start start 2024-09-12T03:07:24.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.703+0000 7f883f035740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c53ae8a40 0x555c53af6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:24.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.703+0000 7f883f035740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c53ae9360 0x555c53b0fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:24.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.703+0000 7f883f035740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c53af8150 0x555c53b101a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:24.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.703+0000 7f883f035740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f88381adb70 con 0x555c53ae9360 2024-09-12T03:07:24.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.703+0000 7f883f035740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f88381ab440 con 0x555c53ae8a40 2024-09-12T03:07:24.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.703+0000 7f883f035740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f88381a92b0 con 0x555c53af8150 2024-09-12T03:07:24.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.704+0000 7f8835d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c53ae9360 0x555c53b0fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:24.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.704+0000 7f8835d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c53ae9360 0x555c53b0fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58828/0 (socket says 172.21.15.116:58828) 2024-09-12T03:07:24.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.704+0000 7f8835d9b640 1 -- 172.21.15.116:0/1721117251 learned_addr learned my addr 172.21.15.116:0/1721117251 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:24.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.704+0000 7f883659c640 1 --2- 172.21.15.116:0/1721117251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c53ae8a40 0x555c53af6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:24.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.704+0000 7f8836d9d640 1 --2- 172.21.15.116:0/1721117251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c53af8150 0x555c53b101a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:24.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.704+0000 7f8835d9b640 1 -- 172.21.15.116:0/1721117251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c53af8150 msgr2=0x555c53b101a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:24.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.704+0000 7f8835d9b640 1 --2- 172.21.15.116:0/1721117251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c53af8150 0x555c53b101a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.704+0000 7f8835d9b640 1 -- 172.21.15.116:0/1721117251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c53ae8a40 msgr2=0x555c53af6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:24.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.704+0000 7f8835d9b640 1 --2- 172.21.15.116:0/1721117251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c53ae8a40 0x555c53af6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.704+0000 7f8835d9b640 1 -- 172.21.15.116:0/1721117251 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c53ae4f20 con 0x555c53ae9360 2024-09-12T03:07:24.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.704+0000 7f8835d9b640 1 --2- 172.21.15.116:0/1721117251 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c53ae9360 0x555c53b0fc60 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f882c01d4d0 tx=0x7f882c01d500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04930d8e929ebbe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:24.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.704+0000 7f881ffff640 1 -- 172.21.15.116:0/1721117251 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c019040 con 0x555c53ae9360 2024-09-12T03:07:24.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.704+0000 7f881ffff640 1 -- 172.21.15.116:0/1721117251 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f882c01a040 con 0x555c53ae9360 2024-09-12T03:07:24.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.704+0000 7f881ffff640 1 -- 172.21.15.116:0/1721117251 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c0118b0 con 0x555c53ae9360 2024-09-12T03:07:24.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.705+0000 7f883f035740 1 -- 172.21.15.116:0/1721117251 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c53ae9360 msgr2=0x555c53b0fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:24.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.705+0000 7f883f035740 1 --2- 172.21.15.116:0/1721117251 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c53ae9360 0x555c53b0fc60 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f882c01d4d0 tx=0x7f882c01d500 comp rx=0 tx=0).stop 2024-09-12T03:07:24.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.705+0000 7f883f035740 1 -- 172.21.15.116:0/1721117251 shutdown_connections 2024-09-12T03:07:24.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.705+0000 7f883f035740 1 --2- 172.21.15.116:0/1721117251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c53af8150 0x555c53b101a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.705+0000 7f883f035740 1 --2- 172.21.15.116:0/1721117251 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c53ae9360 0x555c53b0fc60 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.705+0000 7f883f035740 1 --2- 172.21.15.116:0/1721117251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c53ae8a40 0x555c53af6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.705+0000 7f883f035740 1 -- 172.21.15.116:0/1721117251 >> 172.21.15.116:0/1721117251 conn(0x555c53ae50f0 msgr2=0x555c53af6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:24.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.705+0000 7f883f035740 1 -- 172.21.15.116:0/1721117251 shutdown_connections 2024-09-12T03:07:24.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.705+0000 7f883f035740 1 -- 172.21.15.116:0/1721117251 wait complete. 2024-09-12T03:07:24.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.705+0000 7f883f035740 1 -- start start 2024-09-12T03:07:24.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.705+0000 7f883f035740 1 Processor -- start 2024-09-12T03:07:24.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.706+0000 7f883f035740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c53ae8a40 0x555c53b008e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:24.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.706+0000 7f883f035740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c53ae9360 0x555c53afbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:24.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.706+0000 7f883f035740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c53af8150 0x555c53afc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:24.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.706+0000 7f883f035740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f88381a9900 con 0x555c53ae8a40 2024-09-12T03:07:24.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.706+0000 7f883f035740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f88381aad70 con 0x555c53ae9360 2024-09-12T03:07:24.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.706+0000 7f883f035740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f88381a9fd0 con 0x555c53af8150 2024-09-12T03:07:24.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.706+0000 7f883f035740 10 client.0 _ll_register_callbacks cb 0x7f882800e410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:07:24.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.706+0000 7f883659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c53ae8a40 0x555c53b008e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:24.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.706+0000 7f883659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c53ae8a40 0x555c53b008e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58832/0 (socket says 172.21.15.116:58832) 2024-09-12T03:07:24.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.706+0000 7f883659c640 1 -- 172.21.15.116:0/1000886793 learned_addr learned my addr 172.21.15.116:0/1000886793 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:24.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.706+0000 7f8836d9d640 1 --2- 172.21.15.116:0/1000886793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c53af8150 0x555c53afc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:24.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.706+0000 7f8835d9b640 1 --2- 172.21.15.116:0/1000886793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c53ae9360 0x555c53afbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:24.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.706+0000 7f883659c640 1 -- 172.21.15.116:0/1000886793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c53af8150 msgr2=0x555c53afc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:24.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.706+0000 7f883659c640 1 --2- 172.21.15.116:0/1000886793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c53af8150 0x555c53afc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.706+0000 7f883659c640 1 -- 172.21.15.116:0/1000886793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c53ae9360 msgr2=0x555c53afbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:24.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.706+0000 7f883659c640 1 --2- 172.21.15.116:0/1000886793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c53ae9360 0x555c53afbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.706+0000 7f883659c640 1 -- 172.21.15.116:0/1000886793 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c53afc9d0 con 0x555c53ae8a40 2024-09-12T03:07:24.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.706+0000 7f883659c640 1 --2- 172.21.15.116:0/1000886793 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c53ae8a40 0x555c53b008e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f882801e900 tx=0x7f882801dee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e629cd5deda9f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:24.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.706+0000 7f881dffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:07:24.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.706+0000 7f881dffb640 1 -- 172.21.15.116:0/1000886793 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828040060 con 0x555c53ae8a40 2024-09-12T03:07:24.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.706+0000 7f881dffb640 1 -- 172.21.15.116:0/1000886793 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f8828031420 con 0x555c53ae8a40 2024-09-12T03:07:24.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.706+0000 7f883f035740 1 -- 172.21.15.116:0/1000886793 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f882c01a040 con 0x555c53ae8a40 2024-09-12T03:07:24.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.707+0000 7f881dffb640 1 -- 172.21.15.116:0/1000886793 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8828036070 con 0x555c53ae8a40 2024-09-12T03:07:24.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.707+0000 7f883f035740 10 client.16422 fetch_fsmap learned FSMap version 25 2024-09-12T03:07:24.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.707+0000 7f883f035740 1 -- 172.21.15.116:0/1000886793 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555c53b72cd0 con 0x555c53ae8a40 2024-09-12T03:07:24.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.707+0000 7f881dffb640 1 -- 172.21.15.116:0/1000886793 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f882c01a040 con 0x555c53ae8a40 2024-09-12T03:07:24.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.707+0000 7f881dffb640 1 -- 172.21.15.116:0/1000886793 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f882803b600 con 0x555c53ae8a40 2024-09-12T03:07:24.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.707+0000 7f883f035740 10 client.16422 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:07:24.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.707+0000 7f883f035740 10 client.16422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:07:24.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.707+0000 7f883f035740 20 client.16422 populate_metadata read hostname 'smithi116' 2024-09-12T03:07:24.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.707+0000 7f883f035740 1 --2- 172.21.15.116:0/1000886793 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x555c53b733f0 0x555c53b75810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:24.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.707+0000 7f883f035740 4 client.16422 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:07:24.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.707+0000 7f883f035740 1 -- 172.21.15.116:0/1000886793 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555c53a55d00 con 0x555c53b733f0 2024-09-12T03:07:24.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.707+0000 7f8835d9b640 1 --2- 172.21.15.116:0/1000886793 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x555c53b733f0 0x555c53b75810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:24.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.708+0000 7f8835d9b640 1 --2- 172.21.15.116:0/1000886793 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x555c53b733f0 0x555c53b75810 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=491d21670a88458 server_cookie=bc45bc9c9bafaab2 in_seq=0 out_seq=0 2024-09-12T03:07:24.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.708+0000 7f881dffb640 10 client.16422 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:24.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.709+0000 7f881dffb640 1 -- 172.21.15.116:0/1000886793 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555c53a55d00 con 0x555c53b733f0 2024-09-12T03:07:24.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.709+0000 7f881dffb640 10 client.16422 handle_command_reply: tid=0 2024-09-12T03:07:24.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.709+0000 7f883f035740 1 client.16422 shutdown 2024-09-12T03:07:24.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.709+0000 7f883f035740 1 -- 172.21.15.116:0/1000886793 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c53ae8a40 msgr2=0x555c53b008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:24.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.709+0000 7f883f035740 1 --2- 172.21.15.116:0/1000886793 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c53ae8a40 0x555c53b008e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f882801e900 tx=0x7f882801dee0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.709+0000 7f883f035740 1 -- 172.21.15.116:0/1000886793 shutdown_connections 2024-09-12T03:07:24.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.709+0000 7f883f035740 1 --2- 172.21.15.116:0/1000886793 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x555c53b733f0 0x555c53b75810 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.709+0000 7f883f035740 1 --2- 172.21.15.116:0/1000886793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c53af8150 0x555c53afc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.709+0000 7f881dffb640 0 client.16422 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:24.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.709+0000 7f883f035740 1 --2- 172.21.15.116:0/1000886793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c53ae9360 0x555c53afbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.709+0000 7f883f035740 1 --2- 172.21.15.116:0/1000886793 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555c53ae8a40 0x555c53b008e0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.709+0000 7f883f035740 1 -- 172.21.15.116:0/1000886793 >> 172.21.15.116:0/1000886793 conn(0x555c53ae50f0 msgr2=0x555c53ae4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:24.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.709+0000 7f883f035740 1 -- 172.21.15.116:0/1000886793 shutdown_connections 2024-09-12T03:07:24.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.709+0000 7f883f035740 1 -- 172.21.15.116:0/1000886793 wait complete. 2024-09-12T03:07:24.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.709+0000 7f883f035740 20 client.16422 trim_cache size 0 max 16384 2024-09-12T03:07:24.720 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:07:24.720 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:07:24.720 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "9bd5be35-4116-46c9-82c6-a5656c2c6f00", 2024-09-12T03:07:24.720 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:07:24.720 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:07:24.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.710+0000 7f883cc5c640 1 -- 172.21.15.116:0/3055805106 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f88200a2b80 msgr2=0x7f88200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:24.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.710+0000 7f883cc5c640 1 --2- 172.21.15.116:0/3055805106 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f88200a2b80 0x7f88200a5040 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8828005fd0 tx=0x7f8828009500 comp rx=0 tx=0).stop 2024-09-12T03:07:24.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.710+0000 7f883cc5c640 1 -- 172.21.15.116:0/3055805106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f883810c170 msgr2=0x7f883819cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:24.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.710+0000 7f883cc5c640 1 --2- 172.21.15.116:0/3055805106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f883810c170 0x7f883819cce0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f8830005ab0 tx=0x7f8830004270 comp rx=0 tx=0).stop 2024-09-12T03:07:24.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.710+0000 7f883cc5c640 1 -- 172.21.15.116:0/3055805106 shutdown_connections 2024-09-12T03:07:24.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.710+0000 7f883cc5c640 1 --2- 172.21.15.116:0/3055805106 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f88200a2b80 0x7f88200a5040 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.710+0000 7f883cc5c640 1 --2- 172.21.15.116:0/3055805106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f883810c170 0x7f883819cce0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.710+0000 7f883cc5c640 1 --2- 172.21.15.116:0/3055805106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8838103660 0x7f8838198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.710+0000 7f883cc5c640 1 --2- 172.21.15.116:0/3055805106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8838102d40 0x7f8838198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:24.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.710+0000 7f883cc5c640 1 -- 172.21.15.116:0/3055805106 >> 172.21.15.116:0/3055805106 conn(0x7f88380fc7a0 msgr2=0x7f88380fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:24.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.710+0000 7f883cc5c640 1 -- 172.21.15.116:0/3055805106 shutdown_connections 2024-09-12T03:07:24.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:24.710+0000 7f883cc5c640 1 -- 172.21.15.116:0/3055805106 wait complete. 2024-09-12T03:07:24.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:24 smithi177 ceph-mon[35305]: pgmap v470: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.0 KiB/s wr, 8 op/s 2024-09-12T03:07:25.587 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:25 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:07:25.588 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:25 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:07:25.588 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:25 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:07:25.588 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:25 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:07:25.617 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:07:25.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:25 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:07:25.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:25 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:07:25.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:25 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:07:25.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:25 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:07:25.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:25 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:07:25.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:25 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:07:25.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:25 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:07:25.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:25 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:07:25.867 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:07:26.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.322+0000 7f7d2fa3e640 1 -- 172.21.15.116:0/598614027 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d28108290 msgr2=0x7f7d2810ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:26.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.322+0000 7f7d2fa3e640 1 --2- 172.21.15.116:0/598614027 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d28108290 0x7f7d2810ec30 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009930 tx=0x7f7d2002a420 comp rx=0 tx=0).stop 2024-09-12T03:07:26.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.323+0000 7f7d2fa3e640 1 -- 172.21.15.116:0/598614027 shutdown_connections 2024-09-12T03:07:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.323+0000 7f7d2fa3e640 1 --2- 172.21.15.116:0/598614027 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d28108290 0x7f7d2810ec30 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.323+0000 7f7d2fa3e640 1 --2- 172.21.15.116:0/598614027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d28103740 0x7f7d28107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.323+0000 7f7d2fa3e640 1 --2- 172.21.15.116:0/598614027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d28102d90 0x7f7d28103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.323+0000 7f7d2fa3e640 1 -- 172.21.15.116:0/598614027 >> 172.21.15.116:0/598614027 conn(0x7f7d280fc7d0 msgr2=0x7f7d280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.323+0000 7f7d2fa3e640 1 -- 172.21.15.116:0/598614027 shutdown_connections 2024-09-12T03:07:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.323+0000 7f7d2fa3e640 1 -- 172.21.15.116:0/598614027 wait complete. 2024-09-12T03:07:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.324+0000 7f7d2fa3e640 1 Processor -- start 2024-09-12T03:07:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.324+0000 7f7d2fa3e640 1 -- start start 2024-09-12T03:07:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.324+0000 7f7d2fa3e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d28102d90 0x7f7d281985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.324+0000 7f7d2fa3e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d28103740 0x7f7d28198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:26.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.324+0000 7f7d2fa3e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d28108290 0x7f7d2819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:26.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.324+0000 7f7d2fa3e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7d281120d0 con 0x7f7d28108290 2024-09-12T03:07:26.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.324+0000 7f7d2fa3e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7d28111f50 con 0x7f7d28103740 2024-09-12T03:07:26.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.324+0000 7f7d2fa3e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d28112250 con 0x7f7d28102d90 2024-09-12T03:07:26.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.324+0000 7f7d2dfdb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d28108290 0x7f7d2819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:26.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.324+0000 7f7d2dfdb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d28108290 0x7f7d2819ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58852/0 (socket says 172.21.15.116:58852) 2024-09-12T03:07:26.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.324+0000 7f7d2dfdb640 1 -- 172.21.15.116:0/2546374660 learned_addr learned my addr 172.21.15.116:0/2546374660 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:26.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.324+0000 7f7d2d7da640 1 --2- 172.21.15.116:0/2546374660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d28102d90 0x7f7d281985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:26.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.324+0000 7f7d2cfd9640 1 --2- 172.21.15.116:0/2546374660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d28103740 0x7f7d28198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:26.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.324+0000 7f7d2dfdb640 1 -- 172.21.15.116:0/2546374660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d28102d90 msgr2=0x7f7d281985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:26.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.324+0000 7f7d2dfdb640 1 --2- 172.21.15.116:0/2546374660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d28102d90 0x7f7d281985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.324+0000 7f7d2dfdb640 1 -- 172.21.15.116:0/2546374660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d28103740 msgr2=0x7f7d28198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:26.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.324+0000 7f7d2dfdb640 1 --2- 172.21.15.116:0/2546374660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d28103740 0x7f7d28198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.324+0000 7f7d2dfdb640 1 -- 172.21.15.116:0/2546374660 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2819d560 con 0x7f7d28108290 2024-09-12T03:07:26.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.324+0000 7f7d2d7da640 1 --2- 172.21.15.116:0/2546374660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d28102d90 0x7f7d281985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:07:26.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.325+0000 7f7d2dfdb640 1 --2- 172.21.15.116:0/2546374660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d28108290 0x7f7d2819ce80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f7d20005ab0 tx=0x7f7d200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:26.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.325+0000 7f7d267fc640 1 -- 172.21.15.116:0/2546374660 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2003f030 con 0x7f7d28108290 2024-09-12T03:07:26.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.325+0000 7f7d2fa3e640 1 -- 172.21.15.116:0/2546374660 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d2819d7f0 con 0x7f7d28108290 2024-09-12T03:07:26.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.325+0000 7f7d267fc640 1 -- 172.21.15.116:0/2546374660 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d20003ce0 con 0x7f7d28108290 2024-09-12T03:07:26.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.325+0000 7f7d267fc640 1 -- 172.21.15.116:0/2546374660 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20035070 con 0x7f7d28108290 2024-09-12T03:07:26.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.325+0000 7f7d2fa3e640 1 -- 172.21.15.116:0/2546374660 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d281a5130 con 0x7f7d28108290 2024-09-12T03:07:26.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.328+0000 7f7d267fc640 1 -- 172.21.15.116:0/2546374660 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f7d20042030 con 0x7f7d28108290 2024-09-12T03:07:26.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.328+0000 7f7d31e17740 1 Processor -- start 2024-09-12T03:07:26.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.328+0000 7f7d31e17740 1 -- start start 2024-09-12T03:07:26.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.328+0000 7f7d31e17740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562619f7b380 0x562619f7b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:26.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.328+0000 7f7d31e17740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562619f7bca0 0x562619f8a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:26.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.328+0000 7f7d31e17740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562619f8aa90 0x562619f8ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:26.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.328+0000 7f7d31e17740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562619ee12e0 con 0x562619f7b380 2024-09-12T03:07:26.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.328+0000 7f7d31e17740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562619e88ce0 con 0x562619f8aa90 2024-09-12T03:07:26.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.328+0000 7f7d31e17740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562619edd2a0 con 0x562619f7bca0 2024-09-12T03:07:26.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.328+0000 7f7d2cfd9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562619f7b380 0x562619f7b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:26.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.329+0000 7f7d2cfd9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562619f7b380 0x562619f7b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58862/0 (socket says 172.21.15.116:58862) 2024-09-12T03:07:26.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.329+0000 7f7d2cfd9640 1 -- 172.21.15.116:0/3871238116 learned_addr learned my addr 172.21.15.116:0/3871238116 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:26.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.329+0000 7f7d2d7da640 1 --2- 172.21.15.116:0/3871238116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562619f7bca0 0x562619f8a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:26.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.329+0000 7f7d2cfd9640 1 --2- 172.21.15.116:0/3871238116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562619f8aa90 0x562619f8ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:26.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.329+0000 7f7d2cfd9640 1 -- 172.21.15.116:0/3871238116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562619f7bca0 msgr2=0x562619f8a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:26.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.329+0000 7f7d2cfd9640 1 --2- 172.21.15.116:0/3871238116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562619f7bca0 0x562619f8a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.329+0000 7f7d2cfd9640 1 -- 172.21.15.116:0/3871238116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562619f8aa90 msgr2=0x562619f8ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:26.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.329+0000 7f7d2cfd9640 1 --2- 172.21.15.116:0/3871238116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562619f8aa90 0x562619f8ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.329+0000 7f7d2cfd9640 1 -- 172.21.15.116:0/3871238116 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562619e6d440 con 0x562619f7b380 2024-09-12T03:07:26.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.329+0000 7f7d2d7da640 1 --2- 172.21.15.116:0/3871238116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562619f7bca0 0x562619f8a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:07:26.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.329+0000 7f7d2cfd9640 1 --2- 172.21.15.116:0/3871238116 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562619f7b380 0x562619f7b760 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c0058a0 tx=0x7f7d1c00b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6dbd373d8c34ae2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:26.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.329+0000 7f7d267fc640 1 --2- 172.21.15.116:0/2546374660 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7d100a2b80 0x7f7d100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:26.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.329+0000 7f7d0f7fe640 1 -- 172.21.15.116:0/3871238116 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c004030 con 0x562619f7b380 2024-09-12T03:07:26.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.329+0000 7f7d2d7da640 1 --2- 172.21.15.116:0/2546374660 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7d100a2b80 0x7f7d100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:26.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.329+0000 7f7d267fc640 1 -- 172.21.15.116:0/2546374660 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f7d200ef4b0 con 0x7f7d28108290 2024-09-12T03:07:26.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.329+0000 7f7d0f7fe640 1 -- 172.21.15.116:0/3871238116 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d1c005500 con 0x562619f7b380 2024-09-12T03:07:26.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.329+0000 7f7d0f7fe640 1 -- 172.21.15.116:0/3871238116 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c004be0 con 0x562619f7b380 2024-09-12T03:07:26.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.330+0000 7f7d31e17740 1 -- 172.21.15.116:0/3871238116 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562619f7b380 msgr2=0x562619f7b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:26.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.330+0000 7f7d2d7da640 1 --2- 172.21.15.116:0/2546374660 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7d100a2b80 0x7f7d100a5040 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x562619f7e090 tx=0x7f7d18009770 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:26.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.330+0000 7f7d31e17740 1 --2- 172.21.15.116:0/3871238116 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562619f7b380 0x562619f7b760 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c0058a0 tx=0x7f7d1c00b820 comp rx=0 tx=0).stop 2024-09-12T03:07:26.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.330+0000 7f7d31e17740 1 -- 172.21.15.116:0/3871238116 shutdown_connections 2024-09-12T03:07:26.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.330+0000 7f7d31e17740 1 --2- 172.21.15.116:0/3871238116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562619f8aa90 0x562619f8ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.330+0000 7f7d31e17740 1 --2- 172.21.15.116:0/3871238116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562619f7bca0 0x562619f8a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.330+0000 7f7d31e17740 1 --2- 172.21.15.116:0/3871238116 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562619f7b380 0x562619f7b760 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.330+0000 7f7d31e17740 1 -- 172.21.15.116:0/3871238116 >> 172.21.15.116:0/3871238116 conn(0x562619f77a30 msgr2=0x562619f77e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:26.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.330+0000 7f7d31e17740 1 -- 172.21.15.116:0/3871238116 shutdown_connections 2024-09-12T03:07:26.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.330+0000 7f7d31e17740 1 -- 172.21.15.116:0/3871238116 wait complete. 2024-09-12T03:07:26.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.331+0000 7f7d31e17740 1 -- start start 2024-09-12T03:07:26.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.331+0000 7f7d31e17740 1 Processor -- start 2024-09-12T03:07:26.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.331+0000 7f7d31e17740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562619f7b380 0x562619f8a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:26.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.331+0000 7f7d31e17740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562619f7bca0 0x562619f87640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:26.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.331+0000 7f7d31e17740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562619f8aa90 0x562619f87b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:26.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.331+0000 7f7d31e17740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562619f8d660 con 0x562619f7bca0 2024-09-12T03:07:26.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.331+0000 7f7d31e17740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562619e9c050 con 0x562619f8aa90 2024-09-12T03:07:26.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.331+0000 7f7d31e17740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562619f8d7e0 con 0x562619f7b380 2024-09-12T03:07:26.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.331+0000 7f7d31e17740 10 client.0 _ll_register_callbacks cb 0x562619edc160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:07:26.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.331+0000 7f7d2cfd9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562619f7bca0 0x562619f87640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:26.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.331+0000 7f7d2cfd9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562619f7bca0 0x562619f87640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58864/0 (socket says 172.21.15.116:58864) 2024-09-12T03:07:26.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.331+0000 7f7d2cfd9640 1 -- 172.21.15.116:0/3960364133 learned_addr learned my addr 172.21.15.116:0/3960364133 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:26.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.331+0000 7f7d2d7da640 1 --2- 172.21.15.116:0/3960364133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562619f7b380 0x562619f8a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:26.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.331+0000 7f7d2dfdb640 1 --2- 172.21.15.116:0/3960364133 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562619f8aa90 0x562619f87b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:26.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.332+0000 7f7d2cfd9640 1 -- 172.21.15.116:0/3960364133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562619f7b380 msgr2=0x562619f8a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:26.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.332+0000 7f7d2cfd9640 1 --2- 172.21.15.116:0/3960364133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562619f7b380 0x562619f8a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.332+0000 7f7d2cfd9640 1 -- 172.21.15.116:0/3960364133 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562619f8aa90 msgr2=0x562619f87b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:26.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.332+0000 7f7d2cfd9640 1 --2- 172.21.15.116:0/3960364133 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562619f8aa90 0x562619f87b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.332+0000 7f7d2cfd9640 1 -- 172.21.15.116:0/3960364133 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562619c523c0 con 0x562619f7bca0 2024-09-12T03:07:26.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.332+0000 7f7d2dfdb640 1 --2- 172.21.15.116:0/3960364133 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562619f8aa90 0x562619f87b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:07:26.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.332+0000 7f7d2cfd9640 1 --2- 172.21.15.116:0/3960364133 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562619f7bca0 0x562619f87640 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c005c90 tx=0x7f7d1c012300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8fc6cf2a4c14e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:26.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.332+0000 7f7d0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:07:26.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.332+0000 7f7d0cff9640 1 -- 172.21.15.116:0/3960364133 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c005230 con 0x562619f7bca0 2024-09-12T03:07:26.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.332+0000 7f7d0cff9640 1 -- 172.21.15.116:0/3960364133 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f7d1c004920 con 0x562619f7bca0 2024-09-12T03:07:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.332+0000 7f7d31e17740 1 -- 172.21.15.116:0/3960364133 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d1c005500 con 0x562619f7bca0 2024-09-12T03:07:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.332+0000 7f7d0cff9640 1 -- 172.21.15.116:0/3960364133 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7d1c004b20 con 0x562619f7bca0 2024-09-12T03:07:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.332+0000 7f7d31e17740 10 client.16434 fetch_fsmap learned FSMap version 25 2024-09-12T03:07:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.332+0000 7f7d31e17740 1 -- 172.21.15.116:0/3960364133 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562619f99040 con 0x562619f7bca0 2024-09-12T03:07:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.332+0000 7f7d0cff9640 1 -- 172.21.15.116:0/3960364133 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d1c005500 con 0x562619f7bca0 2024-09-12T03:07:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.332+0000 7f7d0cff9640 1 -- 172.21.15.116:0/3960364133 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7d1c031420 con 0x562619f7bca0 2024-09-12T03:07:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.332+0000 7f7d31e17740 10 client.16434 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:07:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.332+0000 7f7d31e17740 10 client.16434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:07:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.333+0000 7f7d31e17740 20 client.16434 populate_metadata read hostname 'smithi116' 2024-09-12T03:07:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.333+0000 7f7d31e17740 1 --2- 172.21.15.116:0/3960364133 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562619f9a4f0 0x562619f9c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.333+0000 7f7d31e17740 4 client.16434 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:07:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.333+0000 7f7d31e17740 1 -- 172.21.15.116:0/3960364133 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562619eee850 con 0x562619f9a4f0 2024-09-12T03:07:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.333+0000 7f7d2d7da640 1 --2- 172.21.15.116:0/3960364133 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562619f9a4f0 0x562619f9c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:26.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.333+0000 7f7d2d7da640 1 --2- 172.21.15.116:0/3960364133 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562619f9a4f0 0x562619f9c8e0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f1f616f0dee9a4f server_cookie=b73287ddd024a947 in_seq=0 out_seq=0 2024-09-12T03:07:26.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.333+0000 7f7d0cff9640 10 client.16434 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:26.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.334+0000 7f7d0cff9640 1 -- 172.21.15.116:0/3960364133 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562619eee850 con 0x562619f9a4f0 2024-09-12T03:07:26.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.334+0000 7f7d0cff9640 10 client.16434 handle_command_reply: tid=0 2024-09-12T03:07:26.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.334+0000 7f7d31e17740 1 client.16434 shutdown 2024-09-12T03:07:26.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.334+0000 7f7d31e17740 1 -- 172.21.15.116:0/3960364133 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562619f7bca0 msgr2=0x562619f87640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:26.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.334+0000 7f7d31e17740 1 --2- 172.21.15.116:0/3960364133 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562619f7bca0 0x562619f87640 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c005c90 tx=0x7f7d1c012300 comp rx=0 tx=0).stop 2024-09-12T03:07:26.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.334+0000 7f7d31e17740 1 -- 172.21.15.116:0/3960364133 shutdown_connections 2024-09-12T03:07:26.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.334+0000 7f7d31e17740 1 --2- 172.21.15.116:0/3960364133 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562619f9a4f0 0x562619f9c8e0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.334+0000 7f7d31e17740 1 --2- 172.21.15.116:0/3960364133 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562619f8aa90 0x562619f87b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.334+0000 7f7d31e17740 1 --2- 172.21.15.116:0/3960364133 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562619f7bca0 0x562619f87640 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.334+0000 7f7d31e17740 1 --2- 172.21.15.116:0/3960364133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562619f7b380 0x562619f8a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.334+0000 7f7d0cff9640 0 client.16434 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:26.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.334+0000 7f7d31e17740 1 -- 172.21.15.116:0/3960364133 >> 172.21.15.116:0/3960364133 conn(0x562619f77a30 msgr2=0x562619f7d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:26.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.334+0000 7f7d31e17740 1 -- 172.21.15.116:0/3960364133 shutdown_connections 2024-09-12T03:07:26.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.334+0000 7f7d31e17740 1 -- 172.21.15.116:0/3960364133 wait complete. 2024-09-12T03:07:26.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.334+0000 7f7d31e17740 20 client.16434 trim_cache size 0 max 16384 2024-09-12T03:07:26.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.351+0000 7f7d31e17740 1 Processor -- start 2024-09-12T03:07:26.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.351+0000 7f7d31e17740 1 -- start start 2024-09-12T03:07:26.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.351+0000 7f7d31e17740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562619f7b380 0x562619f96930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:26.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.351+0000 7f7d31e17740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562619f7bca0 0x562619fa2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:26.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.351+0000 7f7d31e17740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562619f8aa90 0x562619fa2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:26.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.351+0000 7f7d31e17740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7d281adb30 con 0x562619f8aa90 2024-09-12T03:07:26.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.351+0000 7f7d31e17740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7d281ab400 con 0x562619f7b380 2024-09-12T03:07:26.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.351+0000 7f7d31e17740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d281a9250 con 0x562619f7bca0 2024-09-12T03:07:26.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.351+0000 7f7d2dfdb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562619f8aa90 0x562619fa2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:26.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.352+0000 7f7d2dfdb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562619f8aa90 0x562619fa2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58868/0 (socket says 172.21.15.116:58868) 2024-09-12T03:07:26.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.352+0000 7f7d2dfdb640 1 -- 172.21.15.116:0/984197038 learned_addr learned my addr 172.21.15.116:0/984197038 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:26.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.352+0000 7f7d2d7da640 1 --2- 172.21.15.116:0/984197038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562619f7b380 0x562619f96930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:26.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.352+0000 7f7d2cfd9640 1 --2- 172.21.15.116:0/984197038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562619f7bca0 0x562619fa2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:26.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.352+0000 7f7d2dfdb640 1 -- 172.21.15.116:0/984197038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562619f7bca0 msgr2=0x562619fa2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:26.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.352+0000 7f7d2dfdb640 1 --2- 172.21.15.116:0/984197038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562619f7bca0 0x562619fa2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.352+0000 7f7d2dfdb640 1 -- 172.21.15.116:0/984197038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562619f7b380 msgr2=0x562619f96930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:26.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.352+0000 7f7d2dfdb640 1 --2- 172.21.15.116:0/984197038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562619f7b380 0x562619f96930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.352+0000 7f7d2dfdb640 1 -- 172.21.15.116:0/984197038 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562619f77860 con 0x562619f8aa90 2024-09-12T03:07:26.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.352+0000 7f7d2dfdb640 1 --2- 172.21.15.116:0/984197038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562619f8aa90 0x562619fa2540 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f7d200ecfe0 tx=0x7f7d200bb880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f6091567c4fcff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:26.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.352+0000 7f7d0effd640 1 -- 172.21.15.116:0/984197038 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20049050 con 0x562619f8aa90 2024-09-12T03:07:26.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.352+0000 7f7d0effd640 1 -- 172.21.15.116:0/984197038 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d2003b460 con 0x562619f8aa90 2024-09-12T03:07:26.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.352+0000 7f7d0effd640 1 -- 172.21.15.116:0/984197038 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d200ba0d0 con 0x562619f8aa90 2024-09-12T03:07:26.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.353+0000 7f7d31e17740 1 -- 172.21.15.116:0/984197038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562619f8aa90 msgr2=0x562619fa2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:26.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.353+0000 7f7d31e17740 1 --2- 172.21.15.116:0/984197038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562619f8aa90 0x562619fa2540 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f7d200ecfe0 tx=0x7f7d200bb880 comp rx=0 tx=0).stop 2024-09-12T03:07:26.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.353+0000 7f7d31e17740 1 -- 172.21.15.116:0/984197038 shutdown_connections 2024-09-12T03:07:26.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.353+0000 7f7d31e17740 1 --2- 172.21.15.116:0/984197038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562619f8aa90 0x562619fa2540 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.353+0000 7f7d31e17740 1 --2- 172.21.15.116:0/984197038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562619f7bca0 0x562619fa2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.353+0000 7f7d31e17740 1 --2- 172.21.15.116:0/984197038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562619f7b380 0x562619f96930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.353+0000 7f7d31e17740 1 -- 172.21.15.116:0/984197038 >> 172.21.15.116:0/984197038 conn(0x562619f77a30 msgr2=0x562619f86730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:26.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.353+0000 7f7d31e17740 1 -- 172.21.15.116:0/984197038 shutdown_connections 2024-09-12T03:07:26.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.353+0000 7f7d31e17740 1 -- 172.21.15.116:0/984197038 wait complete. 2024-09-12T03:07:26.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.353+0000 7f7d31e17740 1 -- start start 2024-09-12T03:07:26.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.353+0000 7f7d31e17740 1 Processor -- start 2024-09-12T03:07:26.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.353+0000 7f7d31e17740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562619f7b380 0x562619f84650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:26.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.353+0000 7f7d31e17740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562619f7bca0 0x562619f84b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:26.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.353+0000 7f7d31e17740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562619f8aa90 0x562619f850d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:26.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.353+0000 7f7d31e17740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7d281a98c0 con 0x562619f7bca0 2024-09-12T03:07:26.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.353+0000 7f7d31e17740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7d281aad30 con 0x562619f7b380 2024-09-12T03:07:26.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.353+0000 7f7d31e17740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d281a9f90 con 0x562619f8aa90 2024-09-12T03:07:26.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.353+0000 7f7d31e17740 10 client.0 _ll_register_callbacks cb 0x7f7d1c00bd80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:07:26.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.354+0000 7f7d2cfd9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562619f7bca0 0x562619f84b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:26.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.354+0000 7f7d2cfd9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562619f7bca0 0x562619f84b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58880/0 (socket says 172.21.15.116:58880) 2024-09-12T03:07:26.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.354+0000 7f7d2cfd9640 1 -- 172.21.15.116:0/3151980785 learned_addr learned my addr 172.21.15.116:0/3151980785 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:26.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.354+0000 7f7d2d7da640 1 --2- 172.21.15.116:0/3151980785 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562619f7b380 0x562619f84650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:26.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.354+0000 7f7d2dfdb640 1 --2- 172.21.15.116:0/3151980785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562619f8aa90 0x562619f850d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:26.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.354+0000 7f7d2cfd9640 1 -- 172.21.15.116:0/3151980785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562619f8aa90 msgr2=0x562619f850d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:26.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.354+0000 7f7d2cfd9640 1 --2- 172.21.15.116:0/3151980785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562619f8aa90 0x562619f850d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.354+0000 7f7d2cfd9640 1 -- 172.21.15.116:0/3151980785 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562619f7b380 msgr2=0x562619f84650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:26.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.354+0000 7f7d2cfd9640 1 --2- 172.21.15.116:0/3151980785 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562619f7b380 0x562619f84650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.354+0000 7f7d2cfd9640 1 -- 172.21.15.116:0/3151980785 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56261a003fd0 con 0x562619f7bca0 2024-09-12T03:07:26.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.354+0000 7f7d2dfdb640 1 --2- 172.21.15.116:0/3151980785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562619f8aa90 0x562619f850d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:07:26.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.354+0000 7f7d2cfd9640 1 --2- 172.21.15.116:0/3151980785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562619f7bca0 0x562619f84b90 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c031870 tx=0x7f7d1c00b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea0a3ad4e33e961 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:26.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.354+0000 7f7d0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:07:26.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.354+0000 7f7d0cff9640 1 -- 172.21.15.116:0/3151980785 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c031420 con 0x562619f7bca0 2024-09-12T03:07:26.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.354+0000 7f7d0cff9640 1 -- 172.21.15.116:0/3151980785 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f7d1c022040 con 0x562619f7bca0 2024-09-12T03:07:26.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.354+0000 7f7d31e17740 1 -- 172.21.15.116:0/3151980785 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562619ee8d60 con 0x562619f7bca0 2024-09-12T03:07:26.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.354+0000 7f7d0cff9640 1 -- 172.21.15.116:0/3151980785 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7d1c0315c0 con 0x562619f7bca0 2024-09-12T03:07:26.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.354+0000 7f7d31e17740 10 client.16440 fetch_fsmap learned FSMap version 25 2024-09-12T03:07:26.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.354+0000 7f7d31e17740 1 -- 172.21.15.116:0/3151980785 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56261a0052a0 con 0x562619f7bca0 2024-09-12T03:07:26.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.354+0000 7f7d0cff9640 1 -- 172.21.15.116:0/3151980785 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562619ee8d60 con 0x562619f7bca0 2024-09-12T03:07:26.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.355+0000 7f7d0cff9640 1 -- 172.21.15.116:0/3151980785 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7d1c0104f0 con 0x562619f7bca0 2024-09-12T03:07:26.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.355+0000 7f7d31e17740 10 client.16440 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:07:26.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.355+0000 7f7d31e17740 10 client.16440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:07:26.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.355+0000 7f7d31e17740 20 client.16440 populate_metadata read hostname 'smithi116' 2024-09-12T03:07:26.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.355+0000 7f7d31e17740 1 --2- 172.21.15.116:0/3151980785 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56261a005960 0x56261a007d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:26.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.355+0000 7f7d31e17740 4 client.16440 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:07:26.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.355+0000 7f7d31e17740 1 -- 172.21.15.116:0/3151980785 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562619e65210 con 0x56261a005960 2024-09-12T03:07:26.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.355+0000 7f7d2d7da640 1 --2- 172.21.15.116:0/3151980785 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56261a005960 0x56261a007d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:26.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.355+0000 7f7d0cff9640 10 client.16440 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:26.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.355+0000 7f7d2d7da640 1 --2- 172.21.15.116:0/3151980785 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56261a005960 0x56261a007d80 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9e06322cd9b0118 server_cookie=12a78ef0a294b7ce in_seq=0 out_seq=0 2024-09-12T03:07:26.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.355+0000 7f7d0cff9640 1 -- 172.21.15.116:0/3151980785 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562619e65210 con 0x56261a005960 2024-09-12T03:07:26.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.355+0000 7f7d0cff9640 10 client.16440 handle_command_reply: tid=0 2024-09-12T03:07:26.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.355+0000 7f7d31e17740 1 client.16440 shutdown 2024-09-12T03:07:26.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.356+0000 7f7d31e17740 1 -- 172.21.15.116:0/3151980785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562619f7bca0 msgr2=0x562619f84b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:26.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.356+0000 7f7d31e17740 1 --2- 172.21.15.116:0/3151980785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562619f7bca0 0x562619f84b90 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c031870 tx=0x7f7d1c00b7f0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.356+0000 7f7d31e17740 1 -- 172.21.15.116:0/3151980785 shutdown_connections 2024-09-12T03:07:26.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.356+0000 7f7d31e17740 1 --2- 172.21.15.116:0/3151980785 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56261a005960 0x56261a007d80 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.356+0000 7f7d31e17740 1 --2- 172.21.15.116:0/3151980785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562619f8aa90 0x562619f850d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.356+0000 7f7d0cff9640 0 client.16440 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:26.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.356+0000 7f7d31e17740 1 --2- 172.21.15.116:0/3151980785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562619f7bca0 0x562619f84b90 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.356+0000 7f7d31e17740 1 --2- 172.21.15.116:0/3151980785 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562619f7b380 0x562619f84650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.356+0000 7f7d31e17740 1 -- 172.21.15.116:0/3151980785 >> 172.21.15.116:0/3151980785 conn(0x562619f77a30 msgr2=0x562619f76350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:26.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.356+0000 7f7d31e17740 1 -- 172.21.15.116:0/3151980785 shutdown_connections 2024-09-12T03:07:26.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.356+0000 7f7d31e17740 1 -- 172.21.15.116:0/3151980785 wait complete. 2024-09-12T03:07:26.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.356+0000 7f7d31e17740 20 client.16440 trim_cache size 0 max 16384 2024-09-12T03:07:26.379 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:07:26.379 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:07:26.379 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": { 2024-09-12T03:07:26.379 INFO:teuthology.orchestra.run.smithi116.stdout: "9bd5be35-4116-46c9-82c6-a5656c2c6f00": { 2024-09-12T03:07:26.379 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/", 2024-09-12T03:07:26.379 INFO:teuthology.orchestra.run.smithi116.stdout: "tag": "9bd5be35-4116-46c9-82c6-a5656c2c6f00", 2024-09-12T03:07:26.379 INFO:teuthology.orchestra.run.smithi116.stdout: "options": "recursive,force" 2024-09-12T03:07:26.379 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:07:26.379 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:07:26.379 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:07:26.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.356+0000 7f7d2fa3e640 1 -- 172.21.15.116:0/2546374660 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7d100a2b80 msgr2=0x7f7d100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:26.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.356+0000 7f7d2fa3e640 1 --2- 172.21.15.116:0/2546374660 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7d100a2b80 0x7f7d100a5040 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x562619f7e090 tx=0x7f7d18009770 comp rx=0 tx=0).stop 2024-09-12T03:07:26.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.356+0000 7f7d2fa3e640 1 -- 172.21.15.116:0/2546374660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d28108290 msgr2=0x7f7d2819ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:26.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.356+0000 7f7d2fa3e640 1 --2- 172.21.15.116:0/2546374660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d28108290 0x7f7d2819ce80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f7d20005ab0 tx=0x7f7d200042f0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.356+0000 7f7d2fa3e640 1 -- 172.21.15.116:0/2546374660 shutdown_connections 2024-09-12T03:07:26.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.356+0000 7f7d2fa3e640 1 --2- 172.21.15.116:0/2546374660 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7d100a2b80 0x7f7d100a5040 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.356+0000 7f7d2fa3e640 1 --2- 172.21.15.116:0/2546374660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d28108290 0x7f7d2819ce80 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.356+0000 7f7d2fa3e640 1 --2- 172.21.15.116:0/2546374660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d28103740 0x7f7d28198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.356+0000 7f7d2fa3e640 1 --2- 172.21.15.116:0/2546374660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d28102d90 0x7f7d281985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:26.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.356+0000 7f7d2fa3e640 1 -- 172.21.15.116:0/2546374660 >> 172.21.15.116:0/2546374660 conn(0x7f7d280fc7d0 msgr2=0x7f7d28106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:26.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.357+0000 7f7d2fa3e640 1 -- 172.21.15.116:0/2546374660 shutdown_connections 2024-09-12T03:07:26.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:26.357+0000 7f7d2fa3e640 1 -- 172.21.15.116:0/2546374660 wait complete. 2024-09-12T03:07:26.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:26 smithi116 ceph-mon[31410]: pgmap v471: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.0 KiB/s wr, 8 op/s 2024-09-12T03:07:26.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:26 smithi183 ceph-mon[35535]: pgmap v471: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.0 KiB/s wr, 8 op/s 2024-09-12T03:07:26.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:26 smithi177 ceph-mon[35305]: pgmap v471: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.0 KiB/s wr, 8 op/s 2024-09-12T03:07:27.174 INFO:tasks.cephfs.filesystem:scrub status for tag:9bd5be35-4116-46c9-82c6-a5656c2c6f00 - {'path': '/', 'tag': '9bd5be35-4116-46c9-82c6-a5656c2c6f00', 'options': 'recursive,force'} 2024-09-12T03:07:28.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:28 smithi183 ceph-mon[35535]: pgmap v472: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 KiB/s wr, 9 op/s 2024-09-12T03:07:28.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:28 smithi177 ceph-mon[35305]: pgmap v472: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 KiB/s wr, 9 op/s 2024-09-12T03:07:28.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:28 smithi116 ceph-mon[31410]: pgmap v472: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 KiB/s wr, 9 op/s 2024-09-12T03:07:29.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:29 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:07:29.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:29 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:07:29.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:29 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:07:30.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:30 smithi183 ceph-mon[35535]: pgmap v473: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.5 KiB/s wr, 7 op/s 2024-09-12T03:07:30.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:30 smithi177 ceph-mon[35305]: pgmap v473: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.5 KiB/s wr, 7 op/s 2024-09-12T03:07:30.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:30 smithi116 ceph-mon[31410]: pgmap v473: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.5 KiB/s wr, 7 op/s 2024-09-12T03:07:32.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:32 smithi183 ceph-mon[35535]: pgmap v474: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-09-12T03:07:32.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:32 smithi177 ceph-mon[35305]: pgmap v474: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-09-12T03:07:32.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:32 smithi116 ceph-mon[31410]: pgmap v474: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-09-12T03:07:33.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:33 smithi183 ceph-mon[35535]: pgmap v475: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-12T03:07:33.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:33 smithi177 ceph-mon[35305]: pgmap v475: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-12T03:07:33.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:33 smithi116 ceph-mon[31410]: pgmap v475: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-12T03:07:34.780 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:07:35.041 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:07:35.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.496+0000 7f8fd75d5640 1 -- 172.21.15.116:0/3474183558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8fd0102ef0 msgr2=0x7f8fd010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:35.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.496+0000 7f8fd75d5640 1 --2- 172.21.15.116:0/3474183558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8fd0102ef0 0x7f8fd010da40 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009ca0 tx=0x7f8fc802a950 comp rx=0 tx=0).stop 2024-09-12T03:07:35.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.496+0000 7f8fd75d5640 1 -- 172.21.15.116:0/3474183558 shutdown_connections 2024-09-12T03:07:35.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.496+0000 7f8fd75d5640 1 --2- 172.21.15.116:0/3474183558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8fd0102ef0 0x7f8fd010da40 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:35.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.496+0000 7f8fd75d5640 1 --2- 172.21.15.116:0/3474183558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fd0102570 0x7f8fd01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:35.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.496+0000 7f8fd75d5640 1 --2- 172.21.15.116:0/3474183558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fd0100e20 0x7f8fd0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:35.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.496+0000 7f8fd75d5640 1 -- 172.21.15.116:0/3474183558 >> 172.21.15.116:0/3474183558 conn(0x7f8fd00fc7d0 msgr2=0x7f8fd00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:35.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.496+0000 7f8fd75d5640 1 -- 172.21.15.116:0/3474183558 shutdown_connections 2024-09-12T03:07:35.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.496+0000 7f8fd75d5640 1 -- 172.21.15.116:0/3474183558 wait complete. 2024-09-12T03:07:35.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.496+0000 7f8fd75d5640 1 Processor -- start 2024-09-12T03:07:35.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.496+0000 7f8fd75d5640 1 -- start start 2024-09-12T03:07:35.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.497+0000 7f8fd75d5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fd0100e20 0x7f8fd01987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:35.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.497+0000 7f8fd75d5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8fd0102570 0x7f8fd0198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:35.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.497+0000 7f8fd75d5640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fd0102ef0 0x7f8fd019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:35.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.497+0000 7f8fd75d5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd0110f80 con 0x7f8fd0102570 2024-09-12T03:07:35.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.497+0000 7f8fd75d5640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd0110e00 con 0x7f8fd0102ef0 2024-09-12T03:07:35.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.497+0000 7f8fd75d5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd0111100 con 0x7f8fd0100e20 2024-09-12T03:07:35.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.497+0000 7f8fd4b70640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8fd0102570 0x7f8fd0198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:35.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.497+0000 7f8fd4b70640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8fd0102570 0x7f8fd0198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33950/0 (socket says 172.21.15.116:33950) 2024-09-12T03:07:35.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.497+0000 7f8fd4b70640 1 -- 172.21.15.116:0/3570050552 learned_addr learned my addr 172.21.15.116:0/3570050552 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:35.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.497+0000 7f8fd5371640 1 --2- 172.21.15.116:0/3570050552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fd0100e20 0x7f8fd01987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:35.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.497+0000 7f8fd5b72640 1 --2- 172.21.15.116:0/3570050552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fd0102ef0 0x7f8fd019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:35.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.497+0000 7f8fd4b70640 1 -- 172.21.15.116:0/3570050552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fd0100e20 msgr2=0x7f8fd01987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:35.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.497+0000 7f8fd4b70640 1 --2- 172.21.15.116:0/3570050552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fd0100e20 0x7f8fd01987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:35.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.497+0000 7f8fd4b70640 1 -- 172.21.15.116:0/3570050552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fd0102ef0 msgr2=0x7f8fd019cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:35.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.497+0000 7f8fd4b70640 1 --2- 172.21.15.116:0/3570050552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fd0102ef0 0x7f8fd019cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:35.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.497+0000 7f8fd4b70640 1 -- 172.21.15.116:0/3570050552 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd019d2d0 con 0x7f8fd0102570 2024-09-12T03:07:35.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.497+0000 7f8fd4b70640 1 --2- 172.21.15.116:0/3570050552 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8fd0102570 0x7f8fd0198d10 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000c920 tx=0x7f8fc000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:35.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.497+0000 7f8fc67fc640 1 -- 172.21.15.116:0/3570050552 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0007c70 con 0x7f8fd0102570 2024-09-12T03:07:35.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.497+0000 7f8fc67fc640 1 -- 172.21.15.116:0/3570050552 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fc0007e10 con 0x7f8fd0102570 2024-09-12T03:07:35.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.497+0000 7f8fd75d5640 1 -- 172.21.15.116:0/3570050552 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd019d5c0 con 0x7f8fd0102570 2024-09-12T03:07:35.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.497+0000 7f8fc67fc640 1 -- 172.21.15.116:0/3570050552 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0018890 con 0x7f8fd0102570 2024-09-12T03:07:35.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.498+0000 7f8fd75d5640 1 -- 172.21.15.116:0/3570050552 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd01a4e80 con 0x7f8fd0102570 2024-09-12T03:07:35.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.498+0000 7f8fd75d5640 1 -- 172.21.15.116:0/3570050552 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fd0101200 con 0x7f8fd0102570 2024-09-12T03:07:35.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.500+0000 7f8fc67fc640 1 -- 172.21.15.116:0/3570050552 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8fc0018a50 con 0x7f8fd0102570 2024-09-12T03:07:35.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.500+0000 7f8fc67fc640 1 --2- 172.21.15.116:0/3570050552 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8fbc0a2b80 0x7f8fbc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:35.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.500+0000 7f8fd5371640 1 --2- 172.21.15.116:0/3570050552 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8fbc0a2b80 0x7f8fbc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:35.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.500+0000 7f8fc67fc640 1 -- 172.21.15.116:0/3570050552 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f8fc00cf4b0 con 0x7f8fd0102570 2024-09-12T03:07:35.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.501+0000 7f8fd5371640 1 --2- 172.21.15.116:0/3570050552 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8fbc0a2b80 0x7f8fbc0a5040 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc005e10 tx=0x7f8fcc005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:35.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.504+0000 7f8fc67fc640 1 -- 172.21.15.116:0/3570050552 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8fc0014030 con 0x7f8fd0102570 2024-09-12T03:07:35.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.702+0000 7f8fd75d5640 1 -- 172.21.15.116:0/3570050552 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8fd0199680 con 0x7f8fd0102570 2024-09-12T03:07:35.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.703+0000 7f8fc67fc640 1 -- 172.21.15.116:0/3570050552 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f8fd0199680 con 0x7f8fd0102570 2024-09-12T03:07:35.705 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:07:35.705 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:07:35.705 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:07:35.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.706+0000 7f8fd75d5640 1 -- 172.21.15.116:0/3570050552 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8fbc0a2b80 msgr2=0x7f8fbc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:35.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.706+0000 7f8fd75d5640 1 --2- 172.21.15.116:0/3570050552 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8fbc0a2b80 0x7f8fbc0a5040 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc005e10 tx=0x7f8fcc005da0 comp rx=0 tx=0).stop 2024-09-12T03:07:35.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.706+0000 7f8fd75d5640 1 -- 172.21.15.116:0/3570050552 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8fd0102570 msgr2=0x7f8fd0198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:35.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.706+0000 7f8fd75d5640 1 --2- 172.21.15.116:0/3570050552 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8fd0102570 0x7f8fd0198d10 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000c920 tx=0x7f8fc000cde0 comp rx=0 tx=0).stop 2024-09-12T03:07:35.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.706+0000 7f8fd75d5640 1 -- 172.21.15.116:0/3570050552 shutdown_connections 2024-09-12T03:07:35.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.706+0000 7f8fd75d5640 1 --2- 172.21.15.116:0/3570050552 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8fbc0a2b80 0x7f8fbc0a5040 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:35.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.706+0000 7f8fd75d5640 1 --2- 172.21.15.116:0/3570050552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8fd0102ef0 0x7f8fd019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:35.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.706+0000 7f8fd75d5640 1 --2- 172.21.15.116:0/3570050552 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8fd0102570 0x7f8fd0198d10 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:35.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.706+0000 7f8fd75d5640 1 --2- 172.21.15.116:0/3570050552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fd0100e20 0x7f8fd01987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:35.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.706+0000 7f8fd75d5640 1 -- 172.21.15.116:0/3570050552 >> 172.21.15.116:0/3570050552 conn(0x7f8fd00fc7d0 msgr2=0x7f8fd0103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:35.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.706+0000 7f8fd75d5640 1 -- 172.21.15.116:0/3570050552 shutdown_connections 2024-09-12T03:07:35.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:35.706+0000 7f8fd75d5640 1 -- 172.21.15.116:0/3570050552 wait complete. 2024-09-12T03:07:36.514 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:36 smithi116 ceph-mon[31410]: pgmap v476: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T03:07:36.514 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:36 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3570050552' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:07:36.545 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id 1c00527e --release --await' 2024-09-12T03:07:36.545 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id 1c00527e --release --await 2024-09-12T03:07:36.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:36 smithi183 ceph-mon[35535]: pgmap v476: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T03:07:36.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:36 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3570050552' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:07:36.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:36 smithi177 ceph-mon[35305]: pgmap v476: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T03:07:36.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:36 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3570050552' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:07:36.791 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:07:37.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.287+0000 7f5ba994f640 1 -- 172.21.15.116:0/3774670249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ba4102ef0 msgr2=0x7f5ba410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:37.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.287+0000 7f5ba994f640 1 --2- 172.21.15.116:0/3774670249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ba4102ef0 0x7f5ba410da40 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c009ca0 tx=0x7f5b9c02a950 comp rx=0 tx=0).stop 2024-09-12T03:07:37.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.287+0000 7f5ba994f640 1 -- 172.21.15.116:0/3774670249 shutdown_connections 2024-09-12T03:07:37.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.287+0000 7f5ba994f640 1 --2- 172.21.15.116:0/3774670249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ba4102ef0 0x7f5ba410da40 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.287+0000 7f5ba994f640 1 --2- 172.21.15.116:0/3774670249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ba4102570 0x7f5ba41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.287+0000 7f5ba994f640 1 --2- 172.21.15.116:0/3774670249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ba4100e20 0x7f5ba4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.287+0000 7f5ba994f640 1 -- 172.21.15.116:0/3774670249 >> 172.21.15.116:0/3774670249 conn(0x7f5ba40fc7d0 msgr2=0x7f5ba40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:37.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.287+0000 7f5ba994f640 1 -- 172.21.15.116:0/3774670249 shutdown_connections 2024-09-12T03:07:37.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.287+0000 7f5ba994f640 1 -- 172.21.15.116:0/3774670249 wait complete. 2024-09-12T03:07:37.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.288+0000 7f5ba994f640 1 Processor -- start 2024-09-12T03:07:37.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.288+0000 7f5ba994f640 1 -- start start 2024-09-12T03:07:37.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.288+0000 7f5ba994f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ba4100e20 0x7f5ba40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:37.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.288+0000 7f5ba994f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ba4102570 0x7f5ba40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:37.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.288+0000 7f5ba994f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ba4102ef0 0x7f5ba40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:37.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.288+0000 7f5ba994f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba4110fa0 con 0x7f5ba4102570 2024-09-12T03:07:37.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.288+0000 7f5ba994f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba4110e20 con 0x7f5ba4102ef0 2024-09-12T03:07:37.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.288+0000 7f5ba994f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba4111120 con 0x7f5ba4100e20 2024-09-12T03:07:37.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.288+0000 7f5ba27fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ba4102570 0x7f5ba40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:37.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.288+0000 7f5ba27fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ba4102570 0x7f5ba40fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33970/0 (socket says 172.21.15.116:33970) 2024-09-12T03:07:37.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.288+0000 7f5ba27fc640 1 -- 172.21.15.116:0/877967190 learned_addr learned my addr 172.21.15.116:0/877967190 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:37.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.289+0000 7f5ba2ffd640 1 --2- 172.21.15.116:0/877967190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ba4100e20 0x7f5ba40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:37.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.289+0000 7f5ba37fe640 1 --2- 172.21.15.116:0/877967190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ba4102ef0 0x7f5ba40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:37.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.289+0000 7f5ba27fc640 1 -- 172.21.15.116:0/877967190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ba4100e20 msgr2=0x7f5ba40fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:37.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.289+0000 7f5ba27fc640 1 --2- 172.21.15.116:0/877967190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ba4100e20 0x7f5ba40fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.289+0000 7f5ba27fc640 1 -- 172.21.15.116:0/877967190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ba4102ef0 msgr2=0x7f5ba40fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:37.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.289+0000 7f5ba27fc640 1 --2- 172.21.15.116:0/877967190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ba4102ef0 0x7f5ba40fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.289+0000 7f5ba27fc640 1 -- 172.21.15.116:0/877967190 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba41a69b0 con 0x7f5ba4102570 2024-09-12T03:07:37.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.289+0000 7f5ba2ffd640 1 --2- 172.21.15.116:0/877967190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ba4100e20 0x7f5ba40fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:07:37.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.289+0000 7f5ba37fe640 1 --2- 172.21.15.116:0/877967190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ba4102ef0 0x7f5ba40fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:07:37.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.289+0000 7f5ba27fc640 1 --2- 172.21.15.116:0/877967190 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ba4102570 0x7f5ba40fd1c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5b9800b4a0 tx=0x7f5b9800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:37.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.289+0000 7f5ba894d640 1 -- 172.21.15.116:0/877967190 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b98014070 con 0x7f5ba4102570 2024-09-12T03:07:37.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.289+0000 7f5ba994f640 1 -- 172.21.15.116:0/877967190 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ba41a6ca0 con 0x7f5ba4102570 2024-09-12T03:07:37.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.289+0000 7f5ba894d640 1 -- 172.21.15.116:0/877967190 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b98010af0 con 0x7f5ba4102570 2024-09-12T03:07:37.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.289+0000 7f5ba894d640 1 -- 172.21.15.116:0/877967190 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b98017d10 con 0x7f5ba4102570 2024-09-12T03:07:37.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.289+0000 7f5ba994f640 1 -- 172.21.15.116:0/877967190 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ba41a7190 con 0x7f5ba4102570 2024-09-12T03:07:37.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.291+0000 7f5ba894d640 1 -- 172.21.15.116:0/877967190 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f5b98017070 con 0x7f5ba4102570 2024-09-12T03:07:37.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.291+0000 7f5babd28740 1 Processor -- start 2024-09-12T03:07:37.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.292+0000 7f5babd28740 1 -- start start 2024-09-12T03:07:37.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.292+0000 7f5babd28740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56050f6f2060 0x56050f6f2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:37.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.292+0000 7f5babd28740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56050f6f2980 0x56050f6fcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:37.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.292+0000 7f5babd28740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56050f6fd240 0x56050f6ff630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:37.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.292+0000 7f5babd28740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56050f657620 con 0x56050f6f2980 2024-09-12T03:07:37.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.292+0000 7f5babd28740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56050f5fee50 con 0x56050f6f2060 2024-09-12T03:07:37.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.292+0000 7f5babd28740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56050f6535e0 con 0x56050f6fd240 2024-09-12T03:07:37.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.292+0000 7f5ba2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56050f6f2980 0x56050f6fcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:37.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.292+0000 7f5ba2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56050f6f2980 0x56050f6fcd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33984/0 (socket says 172.21.15.116:33984) 2024-09-12T03:07:37.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.292+0000 7f5ba894d640 1 --2- 172.21.15.116:0/877967190 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5b8c0a2b80 0x7f5b8c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:37.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.292+0000 7f5ba2ffd640 1 -- 172.21.15.116:0/1587224317 learned_addr learned my addr 172.21.15.116:0/1587224317 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:37.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.292+0000 7f5ba27fc640 1 --2- 172.21.15.116:0/1587224317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56050f6fd240 0x56050f6ff630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:37.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.292+0000 7f5ba37fe640 1 --2- 172.21.15.116:0/1587224317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56050f6f2060 0x56050f6f2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:37.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.292+0000 7f5ba37fe640 1 --2- 172.21.15.116:0/877967190 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5b8c0a2b80 0x7f5b8c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:37.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.292+0000 7f5ba2ffd640 1 -- 172.21.15.116:0/1587224317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56050f6fd240 msgr2=0x56050f6ff630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:37.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.292+0000 7f5ba2ffd640 1 --2- 172.21.15.116:0/1587224317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56050f6fd240 0x56050f6ff630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.292+0000 7f5ba2ffd640 1 -- 172.21.15.116:0/1587224317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56050f6f2060 msgr2=0x56050f6f2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:37.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.292+0000 7f5ba2ffd640 1 --2- 172.21.15.116:0/1587224317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56050f6f2060 0x56050f6f2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.292+0000 7f5ba2ffd640 1 -- 172.21.15.116:0/1587224317 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56050f5efd40 con 0x56050f6f2980 2024-09-12T03:07:37.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.292+0000 7f5ba894d640 1 -- 172.21.15.116:0/877967190 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f5b980ce490 con 0x7f5ba4102570 2024-09-12T03:07:37.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.292+0000 7f5ba2ffd640 1 --2- 172.21.15.116:0/1587224317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56050f6f2980 0x56050f6fcd00 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5b9400bb70 tx=0x7f5b9400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cde2f81ca8cd58 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:37.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.293+0000 7f5b89ffb640 1 -- 172.21.15.116:0/1587224317 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b94017620 con 0x56050f6f2980 2024-09-12T03:07:37.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.293+0000 7f5b89ffb640 1 -- 172.21.15.116:0/1587224317 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b94017ca0 con 0x56050f6f2980 2024-09-12T03:07:37.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.293+0000 7f5b89ffb640 1 -- 172.21.15.116:0/1587224317 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9400fc90 con 0x56050f6f2980 2024-09-12T03:07:37.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.293+0000 7f5ba37fe640 1 --2- 172.21.15.116:0/877967190 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5b8c0a2b80 0x7f5b8c0a5040 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c009a60 tx=0x7f5b9c0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:37.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.293+0000 7f5babd28740 1 -- 172.21.15.116:0/1587224317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56050f6f2980 msgr2=0x56050f6fcd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:37.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.293+0000 7f5babd28740 1 --2- 172.21.15.116:0/1587224317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56050f6f2980 0x56050f6fcd00 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5b9400bb70 tx=0x7f5b9400d710 comp rx=0 tx=0).stop 2024-09-12T03:07:37.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.293+0000 7f5babd28740 1 -- 172.21.15.116:0/1587224317 shutdown_connections 2024-09-12T03:07:37.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.293+0000 7f5babd28740 1 --2- 172.21.15.116:0/1587224317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56050f6fd240 0x56050f6ff630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.293+0000 7f5babd28740 1 --2- 172.21.15.116:0/1587224317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56050f6f2980 0x56050f6fcd00 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.293+0000 7f5babd28740 1 --2- 172.21.15.116:0/1587224317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56050f6f2060 0x56050f6f2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.293+0000 7f5babd28740 1 -- 172.21.15.116:0/1587224317 >> 172.21.15.116:0/1587224317 conn(0x56050f6ee5f0 msgr2=0x56050f6ee9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:37.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.293+0000 7f5babd28740 1 -- 172.21.15.116:0/1587224317 shutdown_connections 2024-09-12T03:07:37.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.293+0000 7f5babd28740 1 -- 172.21.15.116:0/1587224317 wait complete. 2024-09-12T03:07:37.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.294+0000 7f5babd28740 1 -- start start 2024-09-12T03:07:37.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.294+0000 7f5babd28740 1 Processor -- start 2024-09-12T03:07:37.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.294+0000 7f5babd28740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56050f6f2060 0x56050f6fa080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:37.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.294+0000 7f5babd28740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56050f6f2980 0x56050f6fa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:37.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.294+0000 7f5babd28740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56050f6fd240 0x56050f6fab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:37.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.294+0000 7f5babd28740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56050f6ffe10 con 0x56050f6fd240 2024-09-12T03:07:37.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.294+0000 7f5babd28740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56050f5dff10 con 0x56050f6f2980 2024-09-12T03:07:37.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.294+0000 7f5babd28740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56050f6fff90 con 0x56050f6f2060 2024-09-12T03:07:37.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.294+0000 7f5babd28740 10 client.0 _ll_register_callbacks cb 0x56050f603db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:07:37.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.294+0000 7f5ba37fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56050f6fd240 0x56050f6fab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:37.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.294+0000 7f5ba37fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56050f6fd240 0x56050f6fab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33986/0 (socket says 172.21.15.116:33986) 2024-09-12T03:07:37.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.294+0000 7f5ba37fe640 1 -- 172.21.15.116:0/329293649 learned_addr learned my addr 172.21.15.116:0/329293649 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:37.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.294+0000 7f5ba2ffd640 1 --2- 172.21.15.116:0/329293649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56050f6f2060 0x56050f6fa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:37.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.295+0000 7f5ba27fc640 1 --2- 172.21.15.116:0/329293649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56050f6f2980 0x56050f6fa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:37.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.295+0000 7f5ba37fe640 1 -- 172.21.15.116:0/329293649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56050f6f2060 msgr2=0x56050f6fa080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:37.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.295+0000 7f5ba37fe640 1 --2- 172.21.15.116:0/329293649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56050f6f2060 0x56050f6fa080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.295+0000 7f5ba37fe640 1 -- 172.21.15.116:0/329293649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56050f6f2980 msgr2=0x56050f6fa5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:37.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.295+0000 7f5ba37fe640 1 --2- 172.21.15.116:0/329293649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56050f6f2980 0x56050f6fa5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.295+0000 7f5ba37fe640 1 -- 172.21.15.116:0/329293649 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56050f366580 con 0x56050f6fd240 2024-09-12T03:07:37.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.295+0000 7f5ba37fe640 1 --2- 172.21.15.116:0/329293649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56050f6fd240 0x56050f6fab00 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c02ff30 tx=0x7f5b9c02fe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f9c8061392b648 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:37.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.295+0000 7f5b82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:07:37.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.295+0000 7f5b82ffd640 1 -- 172.21.15.116:0/329293649 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c04c070 con 0x56050f6fd240 2024-09-12T03:07:37.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.295+0000 7f5b82ffd640 1 -- 172.21.15.116:0/329293649 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f5b9c004450 con 0x56050f6fd240 2024-09-12T03:07:37.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.295+0000 7f5babd28740 1 -- 172.21.15.116:0/329293649 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b94017ca0 con 0x56050f6fd240 2024-09-12T03:07:37.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.295+0000 7f5b82ffd640 1 -- 172.21.15.116:0/329293649 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5b9c042070 con 0x56050f6fd240 2024-09-12T03:07:37.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.295+0000 7f5babd28740 10 client.16458 fetch_fsmap learned FSMap version 25 2024-09-12T03:07:37.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.295+0000 7f5babd28740 1 -- 172.21.15.116:0/329293649 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56050f430960 con 0x56050f6fd240 2024-09-12T03:07:37.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.295+0000 7f5b82ffd640 1 -- 172.21.15.116:0/329293649 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b94017ca0 con 0x56050f6fd240 2024-09-12T03:07:37.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.296+0000 7f5b82ffd640 1 -- 172.21.15.116:0/329293649 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5b9c030440 con 0x56050f6fd240 2024-09-12T03:07:37.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.296+0000 7f5babd28740 10 client.16458 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:07:37.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.296+0000 7f5babd28740 10 client.16458 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:07:37.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.296+0000 7f5babd28740 20 client.16458 populate_metadata read hostname 'smithi116' 2024-09-12T03:07:37.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.296+0000 7f5babd28740 1 --2- 172.21.15.116:0/329293649 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56050f70cd30 0x56050f70f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:37.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.296+0000 7f5babd28740 4 client.16458 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:07:37.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.296+0000 7f5babd28740 1 -- 172.21.15.116:0/329293649 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56050f664b90 con 0x56050f70cd30 2024-09-12T03:07:37.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.296+0000 7f5ba2ffd640 1 --2- 172.21.15.116:0/329293649 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56050f70cd30 0x56050f70f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:37.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.297+0000 7f5b82ffd640 10 client.16458 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:37.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.297+0000 7f5ba2ffd640 1 --2- 172.21.15.116:0/329293649 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56050f70cd30 0x56050f70f180 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ada963c261061d0c server_cookie=4071863ab882051c in_seq=0 out_seq=0 2024-09-12T03:07:37.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.297+0000 7f5b82ffd640 1 -- 172.21.15.116:0/329293649 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56050f664b90 con 0x56050f70cd30 2024-09-12T03:07:37.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.297+0000 7f5b82ffd640 10 client.16458 handle_command_reply: tid=0 2024-09-12T03:07:37.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.297+0000 7f5babd28740 1 client.16458 shutdown 2024-09-12T03:07:37.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.297+0000 7f5babd28740 1 -- 172.21.15.116:0/329293649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56050f6fd240 msgr2=0x56050f6fab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:37.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.297+0000 7f5babd28740 1 --2- 172.21.15.116:0/329293649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56050f6fd240 0x56050f6fab00 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c02ff30 tx=0x7f5b9c02fe00 comp rx=0 tx=0).stop 2024-09-12T03:07:37.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.297+0000 7f5babd28740 1 -- 172.21.15.116:0/329293649 shutdown_connections 2024-09-12T03:07:37.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.297+0000 7f5babd28740 1 --2- 172.21.15.116:0/329293649 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56050f70cd30 0x56050f70f180 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.297+0000 7f5babd28740 1 --2- 172.21.15.116:0/329293649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56050f6fd240 0x56050f6fab00 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.297+0000 7f5babd28740 1 --2- 172.21.15.116:0/329293649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56050f6f2980 0x56050f6fa5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.297+0000 7f5babd28740 1 --2- 172.21.15.116:0/329293649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56050f6f2060 0x56050f6fa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.297+0000 7f5b82ffd640 0 client.16458 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:37.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.297+0000 7f5babd28740 1 -- 172.21.15.116:0/329293649 >> 172.21.15.116:0/329293649 conn(0x56050f6ee5f0 msgr2=0x56050f6f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:37.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.298+0000 7f5babd28740 1 -- 172.21.15.116:0/329293649 shutdown_connections 2024-09-12T03:07:37.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.298+0000 7f5babd28740 1 -- 172.21.15.116:0/329293649 wait complete. 2024-09-12T03:07:37.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.298+0000 7f5babd28740 20 client.16458 trim_cache size 0 max 16384 2024-09-12T03:07:37.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.314+0000 7f5babd28740 1 Processor -- start 2024-09-12T03:07:37.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.314+0000 7f5babd28740 1 -- start start 2024-09-12T03:07:37.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.314+0000 7f5babd28740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56050f6f2060 0x56050f6f4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:37.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.314+0000 7f5babd28740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56050f6f2980 0x56050f714a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:37.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.314+0000 7f5babd28740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56050f6fd240 0x56050f714f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:37.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.314+0000 7f5babd28740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba41b1ff0 con 0x56050f6f2060 2024-09-12T03:07:37.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.314+0000 7f5babd28740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba41af8c0 con 0x56050f6fd240 2024-09-12T03:07:37.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.314+0000 7f5babd28740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba41ad710 con 0x56050f6f2980 2024-09-12T03:07:37.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.315+0000 7f5ba2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56050f6f2060 0x56050f6f4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:37.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.315+0000 7f5ba2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56050f6f2060 0x56050f6f4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33992/0 (socket says 172.21.15.116:33992) 2024-09-12T03:07:37.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.315+0000 7f5ba2ffd640 1 -- 172.21.15.116:0/3087576053 learned_addr learned my addr 172.21.15.116:0/3087576053 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:37.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.315+0000 7f5ba37fe640 1 --2- 172.21.15.116:0/3087576053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56050f6fd240 0x56050f714f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:37.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.315+0000 7f5ba27fc640 1 --2- 172.21.15.116:0/3087576053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56050f6f2980 0x56050f714a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:37.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.315+0000 7f5ba2ffd640 1 -- 172.21.15.116:0/3087576053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56050f6f2980 msgr2=0x56050f714a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:37.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.315+0000 7f5ba2ffd640 1 --2- 172.21.15.116:0/3087576053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56050f6f2980 0x56050f714a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.315+0000 7f5ba2ffd640 1 -- 172.21.15.116:0/3087576053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56050f6fd240 msgr2=0x56050f714f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:37.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.315+0000 7f5ba2ffd640 1 --2- 172.21.15.116:0/3087576053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56050f6fd240 0x56050f714f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.315+0000 7f5ba2ffd640 1 -- 172.21.15.116:0/3087576053 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56050f6ee420 con 0x56050f6f2060 2024-09-12T03:07:37.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.315+0000 7f5ba37fe640 1 --2- 172.21.15.116:0/3087576053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56050f6fd240 0x56050f714f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:07:37.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.315+0000 7f5ba2ffd640 1 --2- 172.21.15.116:0/3087576053 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56050f6f2060 0x56050f6f4ac0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f5b9400bb70 tx=0x7f5b94022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80807132df37e0a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:37.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.315+0000 7f5b897fa640 1 -- 172.21.15.116:0/3087576053 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9400be70 con 0x56050f6f2060 2024-09-12T03:07:37.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.315+0000 7f5b897fa640 1 -- 172.21.15.116:0/3087576053 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b9402f070 con 0x56050f6f2060 2024-09-12T03:07:37.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.315+0000 7f5b897fa640 1 -- 172.21.15.116:0/3087576053 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9402aa10 con 0x56050f6f2060 2024-09-12T03:07:37.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.316+0000 7f5babd28740 1 -- 172.21.15.116:0/3087576053 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56050f6f2060 msgr2=0x56050f6f4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:37.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.316+0000 7f5babd28740 1 --2- 172.21.15.116:0/3087576053 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56050f6f2060 0x56050f6f4ac0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f5b9400bb70 tx=0x7f5b94022040 comp rx=0 tx=0).stop 2024-09-12T03:07:37.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.316+0000 7f5babd28740 1 -- 172.21.15.116:0/3087576053 shutdown_connections 2024-09-12T03:07:37.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.316+0000 7f5babd28740 1 --2- 172.21.15.116:0/3087576053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56050f6fd240 0x56050f714f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.316+0000 7f5babd28740 1 --2- 172.21.15.116:0/3087576053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56050f6f2980 0x56050f714a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.316+0000 7f5babd28740 1 --2- 172.21.15.116:0/3087576053 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56050f6f2060 0x56050f6f4ac0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.316+0000 7f5babd28740 1 -- 172.21.15.116:0/3087576053 >> 172.21.15.116:0/3087576053 conn(0x56050f6ee5f0 msgr2=0x56050f6f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:37.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.316+0000 7f5babd28740 1 -- 172.21.15.116:0/3087576053 shutdown_connections 2024-09-12T03:07:37.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.316+0000 7f5babd28740 1 -- 172.21.15.116:0/3087576053 wait complete. 2024-09-12T03:07:37.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.316+0000 7f5babd28740 1 -- start start 2024-09-12T03:07:37.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.316+0000 7f5babd28740 1 Processor -- start 2024-09-12T03:07:37.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.317+0000 7f5babd28740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56050f6f2060 0x56050f700ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:37.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.317+0000 7f5babd28740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56050f6f2980 0x56050f701530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:37.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.317+0000 7f5babd28740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56050f6fd240 0x56050f701a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:37.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.317+0000 7f5babd28740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba41add80 con 0x56050f6fd240 2024-09-12T03:07:37.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.317+0000 7f5babd28740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba41af1f0 con 0x56050f6f2980 2024-09-12T03:07:37.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.317+0000 7f5babd28740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba41ae450 con 0x56050f6f2060 2024-09-12T03:07:37.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.317+0000 7f5babd28740 10 client.0 _ll_register_callbacks cb 0x7f5b9c02f7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:07:37.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.317+0000 7f5ba37fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56050f6fd240 0x56050f701a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:37.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.317+0000 7f5ba2ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56050f6f2060 0x56050f700ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:37.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.317+0000 7f5ba37fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56050f6fd240 0x56050f701a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34004/0 (socket says 172.21.15.116:34004) 2024-09-12T03:07:37.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.317+0000 7f5ba37fe640 1 -- 172.21.15.116:0/703104871 learned_addr learned my addr 172.21.15.116:0/703104871 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:37.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.317+0000 7f5ba27fc640 1 --2- 172.21.15.116:0/703104871 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56050f6f2980 0x56050f701530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:37.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.317+0000 7f5ba2ffd640 1 -- 172.21.15.116:0/703104871 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56050f6f2980 msgr2=0x56050f701530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:37.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.317+0000 7f5ba2ffd640 1 --2- 172.21.15.116:0/703104871 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56050f6f2980 0x56050f701530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.317+0000 7f5ba2ffd640 1 -- 172.21.15.116:0/703104871 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56050f6fd240 msgr2=0x56050f701a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:37.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.317+0000 7f5ba2ffd640 1 --2- 172.21.15.116:0/703104871 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56050f6fd240 0x56050f701a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.317+0000 7f5ba2ffd640 1 -- 172.21.15.116:0/703104871 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56050f776aa0 con 0x56050f6f2060 2024-09-12T03:07:37.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.317+0000 7f5ba37fe640 1 --2- 172.21.15.116:0/703104871 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56050f6fd240 0x56050f701a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:07:37.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.317+0000 7f5ba2ffd640 1 --2- 172.21.15.116:0/703104871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56050f6f2060 0x56050f700ff0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5b9400bb40 tx=0x7f5b9400b620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8351446d8b842265 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:37.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.317+0000 7f5b82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T03:07:37.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.318+0000 7f5b82ffd640 1 -- 172.21.15.116:0/703104871 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9403a070 con 0x56050f6f2060 2024-09-12T03:07:37.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.318+0000 7f5babd28740 1 -- 172.21.15.116:0/703104871 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b9402f070 con 0x56050f6f2060 2024-09-12T03:07:37.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.318+0000 7f5b82ffd640 1 -- 172.21.15.116:0/703104871 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f5b9402a990 con 0x56050f6f2060 2024-09-12T03:07:37.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.318+0000 7f5b82ffd640 1 -- 172.21.15.116:0/703104871 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5b9402ab90 con 0x56050f6f2060 2024-09-12T03:07:37.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.318+0000 7f5babd28740 10 client.25633 fetch_fsmap learned FSMap version 25 2024-09-12T03:07:37.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.318+0000 7f5babd28740 1 -- 172.21.15.116:0/703104871 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56050f777dc0 con 0x56050f6f2060 2024-09-12T03:07:37.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.318+0000 7f5b82ffd640 1 -- 172.21.15.116:0/703104871 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b9402f070 con 0x56050f6f2060 2024-09-12T03:07:37.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.318+0000 7f5b82ffd640 1 -- 172.21.15.116:0/703104871 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5b9403d020 con 0x56050f6f2060 2024-09-12T03:07:37.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.318+0000 7f5babd28740 10 client.25633 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:07:37.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.318+0000 7f5babd28740 10 client.25633 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:07:37.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.318+0000 7f5babd28740 20 client.25633 populate_metadata read hostname 'smithi116' 2024-09-12T03:07:37.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.318+0000 7f5babd28740 1 --2- 172.21.15.116:0/703104871 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56050f778280 0x56050f77a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:37.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.318+0000 7f5babd28740 4 client.25633 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1c00527e", "release": true, "await": true}] 2024-09-12T03:07:37.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.318+0000 7f5babd28740 1 -- 172.21.15.116:0/703104871 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1c00527e", "release": true, "await": true}) -- 0x56050f65f0a0 con 0x56050f778280 2024-09-12T03:07:37.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.319+0000 7f5ba27fc640 1 --2- 172.21.15.116:0/703104871 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56050f778280 0x56050f77a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:37.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.319+0000 7f5b82ffd640 10 client.25633 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:37.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.319+0000 7f5ba27fc640 1 --2- 172.21.15.116:0/703104871 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56050f778280 0x56050f77a670 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b111a0c93464a87 server_cookie=fc5c8d424f0db1b8 in_seq=0 out_seq=0 2024-09-12T03:07:37.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.321+0000 7f5b82ffd640 1 -- 172.21.15.116:0/703104871 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56050f65f0a0 con 0x56050f778280 2024-09-12T03:07:37.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.321+0000 7f5b82ffd640 10 client.25633 handle_command_reply: tid=0 2024-09-12T03:07:37.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.321+0000 7f5babd28740 1 client.25633 shutdown 2024-09-12T03:07:37.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.321+0000 7f5babd28740 1 -- 172.21.15.116:0/703104871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56050f6f2060 msgr2=0x56050f700ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:37.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.321+0000 7f5babd28740 1 --2- 172.21.15.116:0/703104871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56050f6f2060 0x56050f700ff0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5b9400bb40 tx=0x7f5b9400b620 comp rx=0 tx=0).stop 2024-09-12T03:07:37.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.321+0000 7f5babd28740 1 -- 172.21.15.116:0/703104871 shutdown_connections 2024-09-12T03:07:37.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.321+0000 7f5babd28740 1 --2- 172.21.15.116:0/703104871 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56050f778280 0x56050f77a670 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.321+0000 7f5babd28740 1 --2- 172.21.15.116:0/703104871 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56050f6fd240 0x56050f701a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.321+0000 7f5babd28740 1 --2- 172.21.15.116:0/703104871 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56050f6f2980 0x56050f701530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.321+0000 7f5b82ffd640 0 client.25633 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:37.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.321+0000 7f5babd28740 1 --2- 172.21.15.116:0/703104871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56050f6f2060 0x56050f700ff0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.321+0000 7f5babd28740 1 -- 172.21.15.116:0/703104871 >> 172.21.15.116:0/703104871 conn(0x56050f6ee5f0 msgr2=0x56050f6fa040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:37.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.322+0000 7f5babd28740 1 -- 172.21.15.116:0/703104871 shutdown_connections 2024-09-12T03:07:37.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.322+0000 7f5babd28740 1 -- 172.21.15.116:0/703104871 wait complete. 2024-09-12T03:07:37.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.322+0000 7f5babd28740 20 client.25633 trim_cache size 0 max 16384 2024-09-12T03:07:37.326 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:07:37.326 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 25, 2024-09-12T03:07:37.327 INFO:teuthology.orchestra.run.smithi116.stdout: "leader": 24494, 2024-09-12T03:07:37.327 INFO:teuthology.orchestra.run.smithi116.stdout: "set_version": 20, 2024-09-12T03:07:37.327 INFO:teuthology.orchestra.run.smithi116.stdout: "sets": { 2024-09-12T03:07:37.327 INFO:teuthology.orchestra.run.smithi116.stdout: "1c00527e": { 2024-09-12T03:07:37.327 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 20, 2024-09-12T03:07:37.327 INFO:teuthology.orchestra.run.smithi116.stdout: "age_ref": 0.0, 2024-09-12T03:07:37.327 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:07:37.327 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RELEASED", 2024-09-12T03:07:37.327 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:07:37.327 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:07:37.327 INFO:teuthology.orchestra.run.smithi116.stdout: "timeout": 90.0, 2024-09-12T03:07:37.327 INFO:teuthology.orchestra.run.smithi116.stdout: "expiration": 140.2, 2024-09-12T03:07:37.327 INFO:teuthology.orchestra.run.smithi116.stdout: "members": { 2024-09-12T03:07:37.328 INFO:teuthology.orchestra.run.smithi116.stdout: "file:/": { 2024-09-12T03:07:37.328 INFO:teuthology.orchestra.run.smithi116.stdout: "excluded": false, 2024-09-12T03:07:37.328 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:07:37.328 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RELEASED", 2024-09-12T03:07:37.328 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:07:37.328 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:07:37.328 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:07:37.328 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:07:37.328 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:07:37.328 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:07:37.328 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:07:37.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.322+0000 7f5ba994f640 1 -- 172.21.15.116:0/877967190 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5b8c0a2b80 msgr2=0x7f5b8c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:37.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.322+0000 7f5ba994f640 1 --2- 172.21.15.116:0/877967190 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5b8c0a2b80 0x7f5b8c0a5040 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c009a60 tx=0x7f5b9c0095c0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.322+0000 7f5ba994f640 1 -- 172.21.15.116:0/877967190 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ba4102570 msgr2=0x7f5ba40fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:37.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.322+0000 7f5ba994f640 1 --2- 172.21.15.116:0/877967190 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ba4102570 0x7f5ba40fd1c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5b9800b4a0 tx=0x7f5b9800b960 comp rx=0 tx=0).stop 2024-09-12T03:07:37.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.322+0000 7f5ba994f640 1 -- 172.21.15.116:0/877967190 shutdown_connections 2024-09-12T03:07:37.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.322+0000 7f5ba994f640 1 --2- 172.21.15.116:0/877967190 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5b8c0a2b80 0x7f5b8c0a5040 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.322+0000 7f5ba994f640 1 --2- 172.21.15.116:0/877967190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ba4102ef0 0x7f5ba40fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.322+0000 7f5ba994f640 1 --2- 172.21.15.116:0/877967190 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ba4102570 0x7f5ba40fd1c0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.322+0000 7f5ba994f640 1 --2- 172.21.15.116:0/877967190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ba4100e20 0x7f5ba40fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:37.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.322+0000 7f5ba994f640 1 -- 172.21.15.116:0/877967190 >> 172.21.15.116:0/877967190 conn(0x7f5ba40fc7d0 msgr2=0x7f5ba406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:37.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.322+0000 7f5ba994f640 1 -- 172.21.15.116:0/877967190 shutdown_connections 2024-09-12T03:07:37.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:37.322+0000 7f5ba994f640 1 -- 172.21.15.116:0/877967190 wait complete. 2024-09-12T03:07:38.032 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1c00527e', total seconds elapsed: 23.4 2024-09-12T03:07:38.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:38 smithi116 ceph-mon[31410]: pgmap v477: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-12T03:07:38.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:38 smithi183 ceph-mon[35535]: pgmap v477: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-12T03:07:38.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:38 smithi177 ceph-mon[35305]: pgmap v477: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-12T03:07:40.342 INFO:tasks.workunit.client.0.smithi116.stdout:64/600: open fd = 3 2024-09-12T03:07:40.342 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.342 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.342 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.342 INFO:tasks.workunit.client.0.smithi116.stdout:65/600: open fd = 3 2024-09-12T03:07:40.342 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.342 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.343 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.343 INFO:tasks.workunit.client.0.smithi116.stdout:66/600: open fd = 3 2024-09-12T03:07:40.343 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.343 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.343 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.343 INFO:tasks.workunit.client.0.smithi116.stdout:67/600: open fd = 3 2024-09-12T03:07:40.343 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.343 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.343 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.343 INFO:tasks.workunit.client.0.smithi116.stdout:68/600: open fd = 3 2024-09-12T03:07:40.343 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.343 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.343 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.343 INFO:tasks.workunit.client.0.smithi116.stdout:69/600: open fd = 3 2024-09-12T03:07:40.343 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.344 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.344 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.344 INFO:tasks.workunit.client.0.smithi116.stdout:70/600: open fd = 3 2024-09-12T03:07:40.344 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.344 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.344 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.344 INFO:tasks.workunit.client.0.smithi116.stdout:71/600: open fd = 3 2024-09-12T03:07:40.344 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.344 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.344 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.344 INFO:tasks.workunit.client.0.smithi116.stdout:72/600: open fd = 3 2024-09-12T03:07:40.344 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.344 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.344 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.345 INFO:tasks.workunit.client.0.smithi116.stdout:73/600: open fd = 3 2024-09-12T03:07:40.345 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.345 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.345 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.345 INFO:tasks.workunit.client.0.smithi116.stdout:74/600: open fd = 3 2024-09-12T03:07:40.345 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.345 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.345 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.345 INFO:tasks.workunit.client.0.smithi116.stdout:75/600: open fd = 3 2024-09-12T03:07:40.345 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.345 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.345 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.345 INFO:tasks.workunit.client.0.smithi116.stdout:76/600: open fd = 3 2024-09-12T03:07:40.346 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.346 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.346 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.346 INFO:tasks.workunit.client.0.smithi116.stdout:77/600: open fd = 3 2024-09-12T03:07:40.346 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.346 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.346 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.346 INFO:tasks.workunit.client.0.smithi116.stdout:78/600: open fd = 3 2024-09-12T03:07:40.346 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.346 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.346 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.346 INFO:tasks.workunit.client.0.smithi116.stdout:79/600: open fd = 3 2024-09-12T03:07:40.346 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.347 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.347 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.347 INFO:tasks.workunit.client.0.smithi116.stdout:80/600: open fd = 3 2024-09-12T03:07:40.347 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.347 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.347 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.347 INFO:tasks.workunit.client.0.smithi116.stdout:81/600: open fd = 3 2024-09-12T03:07:40.347 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.347 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.347 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.348 INFO:tasks.workunit.client.0.smithi116.stdout:82/600: open fd = 3 2024-09-12T03:07:40.348 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.348 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.348 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.348 INFO:tasks.workunit.client.0.smithi116.stdout:83/600: open fd = 3 2024-09-12T03:07:40.348 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.348 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.348 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.348 INFO:tasks.workunit.client.0.smithi116.stdout:84/600: open fd = 3 2024-09-12T03:07:40.348 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.348 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.348 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.348 INFO:tasks.workunit.client.0.smithi116.stdout:85/600: open fd = 3 2024-09-12T03:07:40.349 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.350 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.350 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.350 INFO:tasks.workunit.client.0.smithi116.stdout:86/600: open fd = 3 2024-09-12T03:07:40.350 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.350 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.350 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.350 INFO:tasks.workunit.client.0.smithi116.stdout:87/600: open fd = 3 2024-09-12T03:07:40.350 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.350 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.350 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.350 INFO:tasks.workunit.client.0.smithi116.stdout:88/600: open fd = 3 2024-09-12T03:07:40.350 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.351 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.351 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.351 INFO:tasks.workunit.client.0.smithi116.stdout:89/600: open fd = 3 2024-09-12T03:07:40.351 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.351 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.351 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.351 INFO:tasks.workunit.client.0.smithi116.stdout:90/600: open fd = 3 2024-09-12T03:07:40.351 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.351 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.351 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.351 INFO:tasks.workunit.client.0.smithi116.stdout:91/600: open fd = 3 2024-09-12T03:07:40.351 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.351 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.351 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.352 INFO:tasks.workunit.client.0.smithi116.stdout:92/600: open fd = 3 2024-09-12T03:07:40.352 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.352 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.352 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.352 INFO:tasks.workunit.client.0.smithi116.stdout:93/600: open fd = 3 2024-09-12T03:07:40.352 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.352 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.352 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.352 INFO:tasks.workunit.client.0.smithi116.stdout:94/600: open fd = 3 2024-09-12T03:07:40.352 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.352 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.352 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.352 INFO:tasks.workunit.client.0.smithi116.stdout:95/600: open fd = 3 2024-09-12T03:07:40.353 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.353 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.353 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.353 INFO:tasks.workunit.client.0.smithi116.stdout:96/600: open fd = 3 2024-09-12T03:07:40.353 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.353 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.353 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.353 INFO:tasks.workunit.client.0.smithi116.stdout:97/600: open fd = 3 2024-09-12T03:07:40.354 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.354 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.354 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.354 INFO:tasks.workunit.client.0.smithi116.stdout:98/600: open fd = 3 2024-09-12T03:07:40.354 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.354 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.355 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.355 INFO:tasks.workunit.client.0.smithi116.stdout:99/600: open fd = 3 2024-09-12T03:07:40.355 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.355 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.355 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.355 INFO:tasks.workunit.client.0.smithi116.stdout:100/600: open fd = 3 2024-09-12T03:07:40.356 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.356 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.356 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.356 INFO:tasks.workunit.client.0.smithi116.stdout:101/600: open fd = 3 2024-09-12T03:07:40.356 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.357 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.357 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.357 INFO:tasks.workunit.client.0.smithi116.stdout:102/600: open fd = 3 2024-09-12T03:07:40.357 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.357 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.357 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.357 INFO:tasks.workunit.client.0.smithi116.stdout:103/600: open fd = 3 2024-09-12T03:07:40.357 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.357 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.357 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.357 INFO:tasks.workunit.client.0.smithi116.stdout:104/600: open fd = 3 2024-09-12T03:07:40.357 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.358 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.358 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.358 INFO:tasks.workunit.client.0.smithi116.stdout:105/600: open fd = 3 2024-09-12T03:07:40.358 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.358 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.358 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.358 INFO:tasks.workunit.client.0.smithi116.stdout:106/600: open fd = 3 2024-09-12T03:07:40.358 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.358 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.358 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.358 INFO:tasks.workunit.client.0.smithi116.stdout:107/600: open fd = 3 2024-09-12T03:07:40.358 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.358 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.358 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.359 INFO:tasks.workunit.client.0.smithi116.stdout:108/600: open fd = 3 2024-09-12T03:07:40.359 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.359 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.359 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.359 INFO:tasks.workunit.client.0.smithi116.stdout:109/600: open fd = 3 2024-09-12T03:07:40.359 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.359 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.359 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.359 INFO:tasks.workunit.client.0.smithi116.stdout:110/600: open fd = 3 2024-09-12T03:07:40.359 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.359 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.359 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.359 INFO:tasks.workunit.client.0.smithi116.stdout:111/600: open fd = 3 2024-09-12T03:07:40.359 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.359 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.360 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.360 INFO:tasks.workunit.client.0.smithi116.stdout:112/600: open fd = 3 2024-09-12T03:07:40.360 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.360 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.360 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.360 INFO:tasks.workunit.client.0.smithi116.stdout:113/600: open fd = 3 2024-09-12T03:07:40.360 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.360 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.360 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.360 INFO:tasks.workunit.client.0.smithi116.stdout:114/600: open fd = 3 2024-09-12T03:07:40.360 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.360 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.360 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.360 INFO:tasks.workunit.client.0.smithi116.stdout:115/600: open fd = 3 2024-09-12T03:07:40.361 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.361 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.361 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.361 INFO:tasks.workunit.client.0.smithi116.stdout:116/600: open fd = 3 2024-09-12T03:07:40.361 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.361 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.361 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.361 INFO:tasks.workunit.client.0.smithi116.stdout:117/600: open fd = 3 2024-09-12T03:07:40.361 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.361 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.361 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.361 INFO:tasks.workunit.client.0.smithi116.stdout:118/600: open fd = 3 2024-09-12T03:07:40.361 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.361 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.362 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.362 INFO:tasks.workunit.client.0.smithi116.stdout:119/600: open fd = 3 2024-09-12T03:07:40.362 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.362 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.362 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.362 INFO:tasks.workunit.client.0.smithi116.stdout:120/600: open fd = 3 2024-09-12T03:07:40.362 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.362 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.362 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.362 INFO:tasks.workunit.client.0.smithi116.stdout:121/600: open fd = 3 2024-09-12T03:07:40.362 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.362 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.362 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.362 INFO:tasks.workunit.client.0.smithi116.stdout:122/600: open fd = 3 2024-09-12T03:07:40.363 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.363 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.363 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.363 INFO:tasks.workunit.client.0.smithi116.stdout:123/600: open fd = 3 2024-09-12T03:07:40.363 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.363 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.363 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.363 INFO:tasks.workunit.client.0.smithi116.stdout:124/600: open fd = 3 2024-09-12T03:07:40.363 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.363 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.363 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.363 INFO:tasks.workunit.client.0.smithi116.stdout:125/600: open fd = 3 2024-09-12T03:07:40.363 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.363 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.364 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:07:40.364 INFO:tasks.workunit.client.0.smithi116.stdout:126/600: open fd = 3 2024-09-12T03:07:40.364 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:07:40.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:40 smithi116 ceph-mon[31410]: pgmap v478: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:07:40.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:40 smithi183 ceph-mon[35535]: pgmap v478: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:07:40.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:40 smithi177 ceph-mon[35305]: pgmap v478: 97 pgs: 97 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:07:42.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:42 smithi116 ceph-mon[31410]: pgmap v479: 97 pgs: 97 active+clean; 2.2 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-09-12T03:07:42.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:42 smithi183 ceph-mon[35535]: pgmap v479: 97 pgs: 97 active+clean; 2.2 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-09-12T03:07:42.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:42 smithi177 ceph-mon[35305]: pgmap v479: 97 pgs: 97 active+clean; 2.2 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-09-12T03:07:44.547 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:44 smithi177 ceph-mon[35305]: pgmap v480: 97 pgs: 97 active+clean; 2.2 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-09-12T03:07:44.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:44 smithi116 ceph-mon[31410]: pgmap v480: 97 pgs: 97 active+clean; 2.2 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-09-12T03:07:44.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:44 smithi183 ceph-mon[35535]: pgmap v480: 97 pgs: 97 active+clean; 2.2 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-09-12T03:07:46.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:46 smithi116 ceph-mon[31410]: pgmap v481: 97 pgs: 97 active+clean; 2.2 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.5 KiB/s wr, 7 op/s 2024-09-12T03:07:46.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:46 smithi183 ceph-mon[35535]: pgmap v481: 97 pgs: 97 active+clean; 2.2 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.5 KiB/s wr, 7 op/s 2024-09-12T03:07:46.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:46 smithi177 ceph-mon[35305]: pgmap v481: 97 pgs: 97 active+clean; 2.2 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.5 KiB/s wr, 7 op/s 2024-09-12T03:07:48.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:48 smithi116 ceph-mon[31410]: pgmap v482: 97 pgs: 97 active+clean; 2.2 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-09-12T03:07:48.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:48 smithi183 ceph-mon[35535]: pgmap v482: 97 pgs: 97 active+clean; 2.2 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-09-12T03:07:48.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:48 smithi177 ceph-mon[35305]: pgmap v482: 97 pgs: 97 active+clean; 2.2 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-09-12T03:07:50.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:50 smithi116 ceph-mon[31410]: pgmap v483: 97 pgs: 97 active+clean; 2.2 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 8 op/s 2024-09-12T03:07:50.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:50 smithi183 ceph-mon[35535]: pgmap v483: 97 pgs: 97 active+clean; 2.2 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 8 op/s 2024-09-12T03:07:50.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:50 smithi177 ceph-mon[35305]: pgmap v483: 97 pgs: 97 active+clean; 2.2 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 8 op/s 2024-09-12T03:07:52.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:52 smithi116 ceph-mon[31410]: pgmap v484: 97 pgs: 97 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-12T03:07:52.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:52 smithi183 ceph-mon[35535]: pgmap v484: 97 pgs: 97 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-12T03:07:52.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:52 smithi177 ceph-mon[35305]: pgmap v484: 97 pgs: 97 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-12T03:07:54.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:54 smithi116 ceph-mon[31410]: pgmap v485: 97 pgs: 97 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:07:54.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:54 smithi183 ceph-mon[35535]: pgmap v485: 97 pgs: 97 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:07:54.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:54 smithi177 ceph-mon[35305]: pgmap v485: 97 pgs: 97 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:07:56.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:56 smithi116 ceph-mon[31410]: pgmap v486: 97 pgs: 97 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:07:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:56 smithi183 ceph-mon[35535]: pgmap v486: 97 pgs: 97 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:07:56.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:56 smithi177 ceph-mon[35305]: pgmap v486: 97 pgs: 97 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:07:57.176 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:07:57.441 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:07:57.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.932+0000 7f50ff532640 1 -- 172.21.15.116:0/3110831912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50f8100a80 msgr2=0x7f50f8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:57.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.932+0000 7f50ff532640 1 --2- 172.21.15.116:0/3110831912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50f8100a80 0x7f50f8100ee0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009930 tx=0x7f50f402a420 comp rx=0 tx=0).stop 2024-09-12T03:07:57.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.932+0000 7f50ff532640 1 -- 172.21.15.116:0/3110831912 shutdown_connections 2024-09-12T03:07:57.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.932+0000 7f50ff532640 1 --2- 172.21.15.116:0/3110831912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50f8101420 0x7f50f810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.932+0000 7f50ff532640 1 --2- 172.21.15.116:0/3110831912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50f8100a80 0x7f50f8100ee0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.932+0000 7f50ff532640 1 --2- 172.21.15.116:0/3110831912 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50f8106a80 0x7f50f8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.932+0000 7f50ff532640 1 -- 172.21.15.116:0/3110831912 >> 172.21.15.116:0/3110831912 conn(0x7f50f80fc7a0 msgr2=0x7f50f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:57.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.932+0000 7f50ff532640 1 -- 172.21.15.116:0/3110831912 shutdown_connections 2024-09-12T03:07:57.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.932+0000 7f50ff532640 1 -- 172.21.15.116:0/3110831912 wait complete. 2024-09-12T03:07:57.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.932+0000 7f50ff532640 1 Processor -- start 2024-09-12T03:07:57.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.932+0000 7f50ff532640 1 -- start start 2024-09-12T03:07:57.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.933+0000 7f50ff532640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50f8100a80 0x7f50f81985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:57.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.933+0000 7f50ff532640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50f8101420 0x7f50f8198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:57.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.933+0000 7f50ff532640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50f8106a80 0x7f50f819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:57.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.933+0000 7f50ff532640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f50f8110f20 con 0x7f50f8106a80 2024-09-12T03:07:57.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.933+0000 7f50ff532640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f50f8110da0 con 0x7f50f8101420 2024-09-12T03:07:57.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.933+0000 7f50ff532640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81110a0 con 0x7f50f8100a80 2024-09-12T03:07:57.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.933+0000 7f50fdacf640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50f8106a80 0x7f50f819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:57.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.933+0000 7f50fdacf640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50f8106a80 0x7f50f819ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33660/0 (socket says 172.21.15.116:33660) 2024-09-12T03:07:57.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.933+0000 7f50fdacf640 1 -- 172.21.15.116:0/4068483558 learned_addr learned my addr 172.21.15.116:0/4068483558 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:57.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.933+0000 7f50fcacd640 1 --2- 172.21.15.116:0/4068483558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50f8101420 0x7f50f8198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:57.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.933+0000 7f50fd2ce640 1 --2- 172.21.15.116:0/4068483558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50f8100a80 0x7f50f81985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:57.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.933+0000 7f50fdacf640 1 -- 172.21.15.116:0/4068483558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50f8100a80 msgr2=0x7f50f81985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:57.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.933+0000 7f50fdacf640 1 --2- 172.21.15.116:0/4068483558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50f8100a80 0x7f50f81985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.933+0000 7f50fdacf640 1 -- 172.21.15.116:0/4068483558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50f8101420 msgr2=0x7f50f8198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:57.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.933+0000 7f50fdacf640 1 --2- 172.21.15.116:0/4068483558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50f8101420 0x7f50f8198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.933+0000 7f50fdacf640 1 -- 172.21.15.116:0/4068483558 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f819d560 con 0x7f50f8106a80 2024-09-12T03:07:57.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.933+0000 7f50fdacf640 1 --2- 172.21.15.116:0/4068483558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50f8106a80 0x7f50f819ce80 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f50f000b970 tx=0x7f50f000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:57.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.933+0000 7f50ea7fc640 1 -- 172.21.15.116:0/4068483558 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0019070 con 0x7f50f8106a80 2024-09-12T03:07:57.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.933+0000 7f50ea7fc640 1 -- 172.21.15.116:0/4068483558 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50f0014440 con 0x7f50f8106a80 2024-09-12T03:07:57.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.933+0000 7f50ff532640 1 -- 172.21.15.116:0/4068483558 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f819d850 con 0x7f50f8106a80 2024-09-12T03:07:57.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.933+0000 7f50ea7fc640 1 -- 172.21.15.116:0/4068483558 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f00075d0 con 0x7f50f8106a80 2024-09-12T03:07:57.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.933+0000 7f50ff532640 1 -- 172.21.15.116:0/4068483558 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f81a5130 con 0x7f50f8106a80 2024-09-12T03:07:57.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.936+0000 7f50ea7fc640 1 -- 172.21.15.116:0/4068483558 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f50f001c070 con 0x7f50f8106a80 2024-09-12T03:07:57.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.936+0000 7f510190b740 1 Processor -- start 2024-09-12T03:07:57.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.936+0000 7f510190b740 1 -- start start 2024-09-12T03:07:57.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.936+0000 7f510190b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e919624a0 0x561e91962880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:57.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.936+0000 7f510190b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e91962dc0 0x561e9196d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:57.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.936+0000 7f510190b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e9196d680 0x561e9196fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:57.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.936+0000 7f510190b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561e918c82e0 con 0x561e919624a0 2024-09-12T03:07:57.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.936+0000 7f510190b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561e9186fce0 con 0x561e9196d680 2024-09-12T03:07:57.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.936+0000 7f510190b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561e918c42a0 con 0x561e91962dc0 2024-09-12T03:07:57.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.937+0000 7f50fcacd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e919624a0 0x561e91962880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:57.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.937+0000 7f50fcacd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e919624a0 0x561e91962880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33666/0 (socket says 172.21.15.116:33666) 2024-09-12T03:07:57.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.937+0000 7f50fcacd640 1 -- 172.21.15.116:0/3987119926 learned_addr learned my addr 172.21.15.116:0/3987119926 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:57.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.937+0000 7f50ea7fc640 1 --2- 172.21.15.116:0/4068483558 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f50e40a2b80 0x7f50e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:57.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.937+0000 7f50fcacd640 1 --2- 172.21.15.116:0/3987119926 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e9196d680 0x561e9196fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:57.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.937+0000 7f50fd2ce640 1 --2- 172.21.15.116:0/3987119926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e91962dc0 0x561e9196d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:57.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.937+0000 7f50fdacf640 1 --2- 172.21.15.116:0/4068483558 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f50e40a2b80 0x7f50e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:57.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.937+0000 7f50fcacd640 1 -- 172.21.15.116:0/3987119926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e91962dc0 msgr2=0x561e9196d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:57.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.937+0000 7f50fcacd640 1 --2- 172.21.15.116:0/3987119926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e91962dc0 0x561e9196d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.937+0000 7f50fcacd640 1 -- 172.21.15.116:0/3987119926 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e9196d680 msgr2=0x561e9196fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:57.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.937+0000 7f50fcacd640 1 --2- 172.21.15.116:0/3987119926 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e9196d680 0x561e9196fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.937+0000 7f50fcacd640 1 -- 172.21.15.116:0/3987119926 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e91854440 con 0x561e919624a0 2024-09-12T03:07:57.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.937+0000 7f50ea7fc640 1 -- 172.21.15.116:0/4068483558 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f50f00ce5b0 con 0x7f50f8106a80 2024-09-12T03:07:57.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.937+0000 7f50fcacd640 1 --2- 172.21.15.116:0/3987119926 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e919624a0 0x561e91962880 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f50f4005ec0 tx=0x7f50f4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f15cc0d83b93c1f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:57.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.937+0000 7f50e37fe640 1 -- 172.21.15.116:0/3987119926 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4035040 con 0x561e919624a0 2024-09-12T03:07:57.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.937+0000 7f50fdacf640 1 --2- 172.21.15.116:0/4068483558 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f50e40a2b80 0x7f50e40a5040 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f50f006df80 tx=0x7f50f0087040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:57.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.937+0000 7f50e37fe640 1 -- 172.21.15.116:0/3987119926 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50f4005020 con 0x561e919624a0 2024-09-12T03:07:57.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.937+0000 7f50e37fe640 1 -- 172.21.15.116:0/3987119926 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4004bc0 con 0x561e919624a0 2024-09-12T03:07:57.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.938+0000 7f510190b740 1 -- 172.21.15.116:0/3987119926 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e919624a0 msgr2=0x561e91962880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:57.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.938+0000 7f510190b740 1 --2- 172.21.15.116:0/3987119926 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e919624a0 0x561e91962880 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f50f4005ec0 tx=0x7f50f4005fd0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.938+0000 7f510190b740 1 -- 172.21.15.116:0/3987119926 shutdown_connections 2024-09-12T03:07:57.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.938+0000 7f510190b740 1 --2- 172.21.15.116:0/3987119926 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e9196d680 0x561e9196fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.938+0000 7f510190b740 1 --2- 172.21.15.116:0/3987119926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e91962dc0 0x561e9196d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.938+0000 7f510190b740 1 --2- 172.21.15.116:0/3987119926 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e919624a0 0x561e91962880 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.938+0000 7f510190b740 1 -- 172.21.15.116:0/3987119926 >> 172.21.15.116:0/3987119926 conn(0x561e9195ea30 msgr2=0x561e9195ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:57.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.938+0000 7f510190b740 1 -- 172.21.15.116:0/3987119926 shutdown_connections 2024-09-12T03:07:57.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.938+0000 7f510190b740 1 -- 172.21.15.116:0/3987119926 wait complete. 2024-09-12T03:07:57.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.939+0000 7f510190b740 1 -- start start 2024-09-12T03:07:57.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.939+0000 7f510190b740 1 Processor -- start 2024-09-12T03:07:57.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.939+0000 7f510190b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e919624a0 0x561e9196d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:57.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.939+0000 7f510190b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e91962dc0 0x561e9196a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:57.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.939+0000 7f510190b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e9196d680 0x561e9196a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:57.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.939+0000 7f510190b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561e91970250 con 0x561e91962dc0 2024-09-12T03:07:57.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.939+0000 7f510190b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561e91883050 con 0x561e9196d680 2024-09-12T03:07:57.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.939+0000 7f510190b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561e919703d0 con 0x561e919624a0 2024-09-12T03:07:57.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.939+0000 7f510190b740 10 client.0 _ll_register_callbacks cb 0x561e918c3160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:07:57.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.939+0000 7f50fd2ce640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e91962dc0 0x561e9196a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:57.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.939+0000 7f50fd2ce640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e91962dc0 0x561e9196a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33670/0 (socket says 172.21.15.116:33670) 2024-09-12T03:07:57.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.939+0000 7f50fd2ce640 1 -- 172.21.15.116:0/581161605 learned_addr learned my addr 172.21.15.116:0/581161605 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:57.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.939+0000 7f50fcacd640 1 --2- 172.21.15.116:0/581161605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e919624a0 0x561e9196d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:57.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.939+0000 7f50fcacd640 1 --2- 172.21.15.116:0/581161605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e9196d680 0x561e9196a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:57.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.940+0000 7f50fd2ce640 1 -- 172.21.15.116:0/581161605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e919624a0 msgr2=0x561e9196d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:57.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.940+0000 7f50fd2ce640 1 --2- 172.21.15.116:0/581161605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e919624a0 0x561e9196d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.940+0000 7f50fd2ce640 1 -- 172.21.15.116:0/581161605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e9196d680 msgr2=0x561e9196a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:57.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.940+0000 7f50fd2ce640 1 --2- 172.21.15.116:0/581161605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e9196d680 0x561e9196a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.940+0000 7f50fd2ce640 1 -- 172.21.15.116:0/581161605 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e916393c0 con 0x561e91962dc0 2024-09-12T03:07:57.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.940+0000 7f50fcacd640 1 --2- 172.21.15.116:0/581161605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e9196d680 0x561e9196a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:07:57.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.940+0000 7f50fd2ce640 1 --2- 172.21.15.116:0/581161605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e91962dc0 0x561e9196a2d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f50ec011540 tx=0x7f50ec011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634512040a30ef3a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:57.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.940+0000 7f50e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:07:57.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.940+0000 7f50e0ff9640 1 -- 172.21.15.116:0/581161605 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec00f6d0 con 0x561e91962dc0 2024-09-12T03:07:57.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.940+0000 7f510190b740 1 -- 172.21.15.116:0/581161605 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50f4005020 con 0x561e91962dc0 2024-09-12T03:07:57.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.940+0000 7f50e0ff9640 1 -- 172.21.15.116:0/581161605 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f50ec00f870 con 0x561e91962dc0 2024-09-12T03:07:57.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.940+0000 7f50e0ff9640 1 -- 172.21.15.116:0/581161605 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f50ec024070 con 0x561e91962dc0 2024-09-12T03:07:57.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.940+0000 7f510190b740 10 client.16476 fetch_fsmap learned FSMap version 25 2024-09-12T03:07:57.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.940+0000 7f510190b740 1 -- 172.21.15.116:0/581161605 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561e9197bcc0 con 0x561e91962dc0 2024-09-12T03:07:57.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.940+0000 7f50e0ff9640 1 -- 172.21.15.116:0/581161605 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50f4005020 con 0x561e91962dc0 2024-09-12T03:07:57.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.940+0000 7f50e0ff9640 1 -- 172.21.15.116:0/581161605 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f50ec014cb0 con 0x561e91962dc0 2024-09-12T03:07:57.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.940+0000 7f510190b740 10 client.16476 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:07:57.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.940+0000 7f510190b740 10 client.16476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:07:57.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.941+0000 7f510190b740 20 client.16476 populate_metadata read hostname 'smithi116' 2024-09-12T03:07:57.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.941+0000 7f510190b740 1 --2- 172.21.15.116:0/581161605 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x561e9197d1d0 0x561e9197f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:57.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.941+0000 7f510190b740 4 client.16476 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:07:57.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.941+0000 7f510190b740 1 -- 172.21.15.116:0/581161605 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e918d5850 con 0x561e9197d1d0 2024-09-12T03:07:57.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.941+0000 7f50fcacd640 1 --2- 172.21.15.116:0/581161605 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x561e9197d1d0 0x561e9197f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:57.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.941+0000 7f50fcacd640 1 --2- 172.21.15.116:0/581161605 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x561e9197d1d0 0x561e9197f5f0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f05aab1fdfca9fee server_cookie=1687ad18f0cc2cfd in_seq=0 out_seq=0 2024-09-12T03:07:57.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.941+0000 7f50e0ff9640 10 client.16476 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:57.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.942+0000 7f50e0ff9640 1 -- 172.21.15.116:0/581161605 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e918d5850 con 0x561e9197d1d0 2024-09-12T03:07:57.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.942+0000 7f50e0ff9640 10 client.16476 handle_command_reply: tid=0 2024-09-12T03:07:57.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.942+0000 7f510190b740 1 client.16476 shutdown 2024-09-12T03:07:57.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.942+0000 7f510190b740 1 -- 172.21.15.116:0/581161605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e91962dc0 msgr2=0x561e9196a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:57.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.942+0000 7f510190b740 1 --2- 172.21.15.116:0/581161605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e91962dc0 0x561e9196a2d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f50ec011540 tx=0x7f50ec011a00 comp rx=0 tx=0).stop 2024-09-12T03:07:57.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.942+0000 7f510190b740 1 -- 172.21.15.116:0/581161605 shutdown_connections 2024-09-12T03:07:57.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.942+0000 7f510190b740 1 --2- 172.21.15.116:0/581161605 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x561e9197d1d0 0x561e9197f5f0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.942+0000 7f510190b740 1 --2- 172.21.15.116:0/581161605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e9196d680 0x561e9196a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.942+0000 7f510190b740 1 --2- 172.21.15.116:0/581161605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e91962dc0 0x561e9196a2d0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.942+0000 7f510190b740 1 --2- 172.21.15.116:0/581161605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e919624a0 0x561e9196d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.942+0000 7f50e0ff9640 0 client.16476 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:57.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.942+0000 7f510190b740 1 -- 172.21.15.116:0/581161605 >> 172.21.15.116:0/581161605 conn(0x561e9195ea30 msgr2=0x561e91964760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:57.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.942+0000 7f510190b740 1 -- 172.21.15.116:0/581161605 shutdown_connections 2024-09-12T03:07:57.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.942+0000 7f510190b740 1 -- 172.21.15.116:0/581161605 wait complete. 2024-09-12T03:07:57.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.942+0000 7f510190b740 20 client.16476 trim_cache size 0 max 16384 2024-09-12T03:07:57.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.959+0000 7f510190b740 1 Processor -- start 2024-09-12T03:07:57.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.959+0000 7f510190b740 1 -- start start 2024-09-12T03:07:57.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.959+0000 7f510190b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e919624a0 0x561e919794b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:57.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.959+0000 7f510190b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e91962dc0 0x561e91984ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:57.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.959+0000 7f510190b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e9196d680 0x561e91985400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:57.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.959+0000 7f510190b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81afd80 con 0x561e9196d680 2024-09-12T03:07:57.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.959+0000 7f510190b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81ad650 con 0x561e919624a0 2024-09-12T03:07:57.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.959+0000 7f510190b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81ab4c0 con 0x561e91962dc0 2024-09-12T03:07:57.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.960+0000 7f50fcacd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e9196d680 0x561e91985400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:57.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.960+0000 7f50fcacd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e9196d680 0x561e91985400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33676/0 (socket says 172.21.15.116:33676) 2024-09-12T03:07:57.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.960+0000 7f50fcacd640 1 -- 172.21.15.116:0/2228663498 learned_addr learned my addr 172.21.15.116:0/2228663498 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:57.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.960+0000 7f50fd2ce640 1 --2- 172.21.15.116:0/2228663498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e91962dc0 0x561e91984ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:57.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.960+0000 7f50fcacd640 1 --2- 172.21.15.116:0/2228663498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e919624a0 0x561e919794b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:57.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.960+0000 7f50fcacd640 1 -- 172.21.15.116:0/2228663498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e91962dc0 msgr2=0x561e91984ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:57.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.960+0000 7f50fcacd640 1 --2- 172.21.15.116:0/2228663498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e91962dc0 0x561e91984ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.960+0000 7f50fcacd640 1 -- 172.21.15.116:0/2228663498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e919624a0 msgr2=0x561e919794b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:57.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.960+0000 7f50fcacd640 1 --2- 172.21.15.116:0/2228663498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e919624a0 0x561e919794b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.960+0000 7f50fcacd640 1 -- 172.21.15.116:0/2228663498 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e919622d0 con 0x561e9196d680 2024-09-12T03:07:57.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.960+0000 7f50fcacd640 1 --2- 172.21.15.116:0/2228663498 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e9196d680 0x561e91985400 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f50f4005e90 tx=0x7f50f4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1558cbfe9d65a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:57.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.960+0000 7f50e2ffd640 1 -- 172.21.15.116:0/2228663498 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4035040 con 0x561e9196d680 2024-09-12T03:07:57.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.960+0000 7f50e2ffd640 1 -- 172.21.15.116:0/2228663498 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50f4004380 con 0x561e9196d680 2024-09-12T03:07:57.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.960+0000 7f50e2ffd640 1 -- 172.21.15.116:0/2228663498 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f404a630 con 0x561e9196d680 2024-09-12T03:07:57.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.961+0000 7f510190b740 1 -- 172.21.15.116:0/2228663498 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e9196d680 msgr2=0x561e91985400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:57.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.961+0000 7f510190b740 1 --2- 172.21.15.116:0/2228663498 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e9196d680 0x561e91985400 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f50f4005e90 tx=0x7f50f4000bc0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.961+0000 7f510190b740 1 -- 172.21.15.116:0/2228663498 shutdown_connections 2024-09-12T03:07:57.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.961+0000 7f510190b740 1 --2- 172.21.15.116:0/2228663498 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e9196d680 0x561e91985400 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.961+0000 7f510190b740 1 --2- 172.21.15.116:0/2228663498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e91962dc0 0x561e91984ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.961+0000 7f510190b740 1 --2- 172.21.15.116:0/2228663498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e919624a0 0x561e919794b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.961+0000 7f510190b740 1 -- 172.21.15.116:0/2228663498 >> 172.21.15.116:0/2228663498 conn(0x561e9195ea30 msgr2=0x561e919693c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:57.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.961+0000 7f510190b740 1 -- 172.21.15.116:0/2228663498 shutdown_connections 2024-09-12T03:07:57.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.961+0000 7f510190b740 1 -- 172.21.15.116:0/2228663498 wait complete. 2024-09-12T03:07:57.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.961+0000 7f510190b740 1 -- start start 2024-09-12T03:07:57.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.961+0000 7f510190b740 1 Processor -- start 2024-09-12T03:07:57.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.962+0000 7f510190b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e919624a0 0x561e919713e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:57.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.962+0000 7f510190b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e91962dc0 0x561e91971920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:57.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.962+0000 7f510190b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e9196d680 0x561e91971e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:57.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.962+0000 7f510190b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81abb10 con 0x561e91962dc0 2024-09-12T03:07:57.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.962+0000 7f510190b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81acf80 con 0x561e919624a0 2024-09-12T03:07:57.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.962+0000 7f510190b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81ac1e0 con 0x561e9196d680 2024-09-12T03:07:57.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.962+0000 7f510190b740 10 client.0 _ll_register_callbacks cb 0x7f50ec009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:07:57.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.962+0000 7f50fd2ce640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e91962dc0 0x561e91971920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:57.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.962+0000 7f50fd2ce640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e91962dc0 0x561e91971920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33690/0 (socket says 172.21.15.116:33690) 2024-09-12T03:07:57.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.962+0000 7f50fd2ce640 1 -- 172.21.15.116:0/4159606806 learned_addr learned my addr 172.21.15.116:0/4159606806 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:57.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.962+0000 7f50fcacd640 1 --2- 172.21.15.116:0/4159606806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e919624a0 0x561e919713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:57.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.962+0000 7f50fcacd640 1 --2- 172.21.15.116:0/4159606806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e9196d680 0x561e91971e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:57.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.962+0000 7f50fd2ce640 1 -- 172.21.15.116:0/4159606806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e9196d680 msgr2=0x561e91971e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:57.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.962+0000 7f50fd2ce640 1 --2- 172.21.15.116:0/4159606806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e9196d680 0x561e91971e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.962+0000 7f50fd2ce640 1 -- 172.21.15.116:0/4159606806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e919624a0 msgr2=0x561e919713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:57.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.962+0000 7f50fd2ce640 1 --2- 172.21.15.116:0/4159606806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e919624a0 0x561e919713e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.962+0000 7f50fd2ce640 1 -- 172.21.15.116:0/4159606806 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e919e6f10 con 0x561e91962dc0 2024-09-12T03:07:57.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.962+0000 7f50fcacd640 1 --2- 172.21.15.116:0/4159606806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e919624a0 0x561e919713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:07:57.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.962+0000 7f50fd2ce640 1 --2- 172.21.15.116:0/4159606806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e91962dc0 0x561e91971920 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x561e9196b910 tx=0x7f50ec00f270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d786f4705ed674 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:57.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.962+0000 7f50e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:07:57.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.963+0000 7f50e0ff9640 1 -- 172.21.15.116:0/4159606806 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec01db50 con 0x561e91962dc0 2024-09-12T03:07:57.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.963+0000 7f50e0ff9640 1 -- 172.21.15.116:0/4159606806 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f50ec01dcf0 con 0x561e91962dc0 2024-09-12T03:07:57.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.963+0000 7f510190b740 1 -- 172.21.15.116:0/4159606806 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50f4004380 con 0x561e91962dc0 2024-09-12T03:07:57.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.963+0000 7f50e0ff9640 1 -- 172.21.15.116:0/4159606806 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f50ec006d60 con 0x561e91962dc0 2024-09-12T03:07:57.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.963+0000 7f510190b740 10 client.16482 fetch_fsmap learned FSMap version 25 2024-09-12T03:07:57.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.963+0000 7f510190b740 1 -- 172.21.15.116:0/4159606806 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561e919e8210 con 0x561e91962dc0 2024-09-12T03:07:57.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.963+0000 7f50e0ff9640 1 -- 172.21.15.116:0/4159606806 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50f4004380 con 0x561e91962dc0 2024-09-12T03:07:57.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.963+0000 7f50e0ff9640 1 -- 172.21.15.116:0/4159606806 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f50ec029800 con 0x561e91962dc0 2024-09-12T03:07:57.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.963+0000 7f510190b740 10 client.16482 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:07:57.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.963+0000 7f510190b740 10 client.16482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:07:57.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.963+0000 7f510190b740 20 client.16482 populate_metadata read hostname 'smithi116' 2024-09-12T03:07:57.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.963+0000 7f510190b740 1 --2- 172.21.15.116:0/4159606806 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x561e919e89c0 0x561e919eade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:57.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.963+0000 7f510190b740 4 client.16482 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:07:57.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.963+0000 7f510190b740 1 -- 172.21.15.116:0/4159606806 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561e918cfd60 con 0x561e919e89c0 2024-09-12T03:07:57.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.963+0000 7f50fcacd640 1 --2- 172.21.15.116:0/4159606806 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x561e919e89c0 0x561e919eade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:57.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.964+0000 7f50fcacd640 1 --2- 172.21.15.116:0/4159606806 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x561e919e89c0 0x561e919eade0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca5a05e10f0b3617 server_cookie=1f8f72d5873b15e3 in_seq=0 out_seq=0 2024-09-12T03:07:57.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.964+0000 7f50e0ff9640 10 client.16482 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:57.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.964+0000 7f50e0ff9640 1 -- 172.21.15.116:0/4159606806 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561e918cfd60 con 0x561e919e89c0 2024-09-12T03:07:57.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.964+0000 7f50e0ff9640 10 client.16482 handle_command_reply: tid=0 2024-09-12T03:07:57.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.964+0000 7f510190b740 1 client.16482 shutdown 2024-09-12T03:07:57.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.964+0000 7f510190b740 1 -- 172.21.15.116:0/4159606806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e91962dc0 msgr2=0x561e91971920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:57.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.964+0000 7f510190b740 1 --2- 172.21.15.116:0/4159606806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e91962dc0 0x561e91971920 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x561e9196b910 tx=0x7f50ec00f270 comp rx=0 tx=0).stop 2024-09-12T03:07:57.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.964+0000 7f510190b740 1 -- 172.21.15.116:0/4159606806 shutdown_connections 2024-09-12T03:07:57.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.964+0000 7f510190b740 1 --2- 172.21.15.116:0/4159606806 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x561e919e89c0 0x561e919eade0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.964+0000 7f510190b740 1 --2- 172.21.15.116:0/4159606806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e9196d680 0x561e91971e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.964+0000 7f510190b740 1 --2- 172.21.15.116:0/4159606806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e91962dc0 0x561e91971920 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.964+0000 7f50e0ff9640 0 client.16482 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:07:57.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.964+0000 7f510190b740 1 --2- 172.21.15.116:0/4159606806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e919624a0 0x561e919713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.964+0000 7f510190b740 1 -- 172.21.15.116:0/4159606806 >> 172.21.15.116:0/4159606806 conn(0x561e9195ea30 msgr2=0x561e9196bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:57.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.964+0000 7f510190b740 1 -- 172.21.15.116:0/4159606806 shutdown_connections 2024-09-12T03:07:57.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.964+0000 7f510190b740 1 -- 172.21.15.116:0/4159606806 wait complete. 2024-09-12T03:07:57.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.964+0000 7f510190b740 20 client.16482 trim_cache size 0 max 16384 2024-09-12T03:07:57.986 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:07:57.986 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:07:57.987 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:07:57.987 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:07:57.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.965+0000 7f50ff532640 1 -- 172.21.15.116:0/4068483558 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f50e40a2b80 msgr2=0x7f50e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:57.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.965+0000 7f50ff532640 1 --2- 172.21.15.116:0/4068483558 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f50e40a2b80 0x7f50e40a5040 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f50f006df80 tx=0x7f50f0087040 comp rx=0 tx=0).stop 2024-09-12T03:07:57.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.965+0000 7f50ff532640 1 -- 172.21.15.116:0/4068483558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50f8106a80 msgr2=0x7f50f819ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:57.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.965+0000 7f50ff532640 1 --2- 172.21.15.116:0/4068483558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50f8106a80 0x7f50f819ce80 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f50f000b970 tx=0x7f50f000bd70 comp rx=0 tx=0).stop 2024-09-12T03:07:57.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.965+0000 7f50ff532640 1 -- 172.21.15.116:0/4068483558 shutdown_connections 2024-09-12T03:07:57.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.965+0000 7f50ff532640 1 --2- 172.21.15.116:0/4068483558 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f50e40a2b80 0x7f50e40a5040 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.965+0000 7f50ff532640 1 --2- 172.21.15.116:0/4068483558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50f8106a80 0x7f50f819ce80 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.965+0000 7f50ff532640 1 --2- 172.21.15.116:0/4068483558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50f8101420 0x7f50f8198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.965+0000 7f50ff532640 1 --2- 172.21.15.116:0/4068483558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50f8100a80 0x7f50f81985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:57.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.965+0000 7f50ff532640 1 -- 172.21.15.116:0/4068483558 >> 172.21.15.116:0/4068483558 conn(0x7f50f80fc7a0 msgr2=0x7f50f806aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:57.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.965+0000 7f50ff532640 1 -- 172.21.15.116:0/4068483558 shutdown_connections 2024-09-12T03:07:57.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:57.965+0000 7f50ff532640 1 -- 172.21.15.116:0/4068483558 wait complete. 2024-09-12T03:07:58.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:07:58 smithi116 ceph-mon[31410]: pgmap v487: 97 pgs: 97 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:07:58.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:07:58 smithi183 ceph-mon[35535]: pgmap v487: 97 pgs: 97 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:07:58.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:07:58 smithi177 ceph-mon[35305]: pgmap v487: 97 pgs: 97 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:07:58.867 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:07:58.868 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:07:59.126 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:07:59.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.631+0000 7f1ed9447640 1 -- 172.21.15.116:0/4250444471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ed4101420 msgr2=0x7f1ed410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:59.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.631+0000 7f1ed9447640 1 --2- 172.21.15.116:0/4250444471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ed4101420 0x7f1ed410d950 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc009cd0 tx=0x7f1ecc02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:07:59.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.631+0000 7f1ed9447640 1 -- 172.21.15.116:0/4250444471 shutdown_connections 2024-09-12T03:07:59.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.631+0000 7f1ed9447640 1 --2- 172.21.15.116:0/4250444471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ed4101420 0x7f1ed410d950 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:59.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.631+0000 7f1ed9447640 1 --2- 172.21.15.116:0/4250444471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ed4100a80 0x7f1ed4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:59.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.631+0000 7f1ed9447640 1 --2- 172.21.15.116:0/4250444471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ed4106a80 0x7f1ed4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:59.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.631+0000 7f1ed9447640 1 -- 172.21.15.116:0/4250444471 >> 172.21.15.116:0/4250444471 conn(0x7f1ed40fc7a0 msgr2=0x7f1ed40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:59.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.631+0000 7f1ed9447640 1 -- 172.21.15.116:0/4250444471 shutdown_connections 2024-09-12T03:07:59.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.631+0000 7f1ed9447640 1 -- 172.21.15.116:0/4250444471 wait complete. 2024-09-12T03:07:59.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.632+0000 7f1ed9447640 1 Processor -- start 2024-09-12T03:07:59.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.632+0000 7f1ed9447640 1 -- start start 2024-09-12T03:07:59.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.632+0000 7f1ed9447640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ed4100a80 0x7f1ed4198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:59.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.632+0000 7f1ed9447640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ed4101420 0x7f1ed4198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:59.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.632+0000 7f1ed9447640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ed4106a80 0x7f1ed419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:59.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.632+0000 7f1ed9447640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed4110f20 con 0x7f1ed4100a80 2024-09-12T03:07:59.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.632+0000 7f1ed9447640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed4110da0 con 0x7f1ed4106a80 2024-09-12T03:07:59.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.632+0000 7f1ed9447640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed41110a0 con 0x7f1ed4101420 2024-09-12T03:07:59.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.632+0000 7f1ed2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ed4100a80 0x7f1ed4198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:59.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.632+0000 7f1ed2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ed4100a80 0x7f1ed4198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33718/0 (socket says 172.21.15.116:33718) 2024-09-12T03:07:59.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.632+0000 7f1ed2ffd640 1 -- 172.21.15.116:0/1487809556 learned_addr learned my addr 172.21.15.116:0/1487809556 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:07:59.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.633+0000 7f1ed37fe640 1 --2- 172.21.15.116:0/1487809556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ed4106a80 0x7f1ed419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:59.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.633+0000 7f1ed2ffd640 1 -- 172.21.15.116:0/1487809556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ed4101420 msgr2=0x7f1ed4198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:59.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.633+0000 7f1ed2ffd640 1 --2- 172.21.15.116:0/1487809556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ed4101420 0x7f1ed4198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:59.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.633+0000 7f1ed2ffd640 1 -- 172.21.15.116:0/1487809556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ed4106a80 msgr2=0x7f1ed419ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:59.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.633+0000 7f1ed2ffd640 1 --2- 172.21.15.116:0/1487809556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ed4106a80 0x7f1ed419ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:59.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.633+0000 7f1ed2ffd640 1 -- 172.21.15.116:0/1487809556 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed419d4e0 con 0x7f1ed4100a80 2024-09-12T03:07:59.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.633+0000 7f1ed2ffd640 1 --2- 172.21.15.116:0/1487809556 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ed4100a80 0x7f1ed4198530 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400b4a0 tx=0x7f1ec400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:59.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.633+0000 7f1ecbfff640 1 -- 172.21.15.116:0/1487809556 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4014070 con 0x7f1ed4100a80 2024-09-12T03:07:59.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.633+0000 7f1ecbfff640 1 -- 172.21.15.116:0/1487809556 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ec4010af0 con 0x7f1ed4100a80 2024-09-12T03:07:59.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.633+0000 7f1ed9447640 1 -- 172.21.15.116:0/1487809556 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ed419d7d0 con 0x7f1ed4100a80 2024-09-12T03:07:59.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.633+0000 7f1ecbfff640 1 -- 172.21.15.116:0/1487809556 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4017d10 con 0x7f1ed4100a80 2024-09-12T03:07:59.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.633+0000 7f1ed9447640 1 -- 172.21.15.116:0/1487809556 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ed41a50b0 con 0x7f1ed4100a80 2024-09-12T03:07:59.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.634+0000 7f1ed9447640 1 -- 172.21.15.116:0/1487809556 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ed4100f60 con 0x7f1ed4100a80 2024-09-12T03:07:59.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.636+0000 7f1ecbfff640 1 -- 172.21.15.116:0/1487809556 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1ec4017070 con 0x7f1ed4100a80 2024-09-12T03:07:59.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.636+0000 7f1ecbfff640 1 --2- 172.21.15.116:0/1487809556 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1ec00a2b80 0x7f1ec00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:59.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.637+0000 7f1ed27fc640 1 --2- 172.21.15.116:0/1487809556 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1ec00a2b80 0x7f1ec00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:59.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.637+0000 7f1ecbfff640 1 -- 172.21.15.116:0/1487809556 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1ec40ce820 con 0x7f1ed4100a80 2024-09-12T03:07:59.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.637+0000 7f1ed27fc640 1 --2- 172.21.15.116:0/1487809556 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1ec00a2b80 0x7f1ec00a5040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4104a60 tx=0x7f1ebc006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:59.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.640+0000 7f1ecbfff640 1 -- 172.21.15.116:0/1487809556 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1ec409b7b0 con 0x7f1ed4100a80 2024-09-12T03:07:59.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.841+0000 7f1ed9447640 1 -- 172.21.15.116:0/1487809556 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ed4199890 con 0x7f1ed4100a80 2024-09-12T03:07:59.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.842+0000 7f1ecbfff640 1 -- 172.21.15.116:0/1487809556 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f1ed4199890 con 0x7f1ed4100a80 2024-09-12T03:07:59.843 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:07:59.843 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:07:59.843 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:07:59.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.844+0000 7f1ed9447640 1 -- 172.21.15.116:0/1487809556 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1ec00a2b80 msgr2=0x7f1ec00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:59.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.844+0000 7f1ed9447640 1 --2- 172.21.15.116:0/1487809556 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1ec00a2b80 0x7f1ec00a5040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4104a60 tx=0x7f1ebc006c90 comp rx=0 tx=0).stop 2024-09-12T03:07:59.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.844+0000 7f1ed9447640 1 -- 172.21.15.116:0/1487809556 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ed4100a80 msgr2=0x7f1ed4198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:59.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.844+0000 7f1ed9447640 1 --2- 172.21.15.116:0/1487809556 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ed4100a80 0x7f1ed4198530 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400b4a0 tx=0x7f1ec400b960 comp rx=0 tx=0).stop 2024-09-12T03:07:59.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.844+0000 7f1ed9447640 1 -- 172.21.15.116:0/1487809556 shutdown_connections 2024-09-12T03:07:59.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.844+0000 7f1ed9447640 1 --2- 172.21.15.116:0/1487809556 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1ec00a2b80 0x7f1ec00a5040 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:59.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.844+0000 7f1ed9447640 1 --2- 172.21.15.116:0/1487809556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ed4106a80 0x7f1ed419ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:59.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.844+0000 7f1ed9447640 1 --2- 172.21.15.116:0/1487809556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ed4101420 0x7f1ed4198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:59.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.844+0000 7f1ed9447640 1 --2- 172.21.15.116:0/1487809556 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ed4100a80 0x7f1ed4198530 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:59.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.844+0000 7f1ed9447640 1 -- 172.21.15.116:0/1487809556 >> 172.21.15.116:0/1487809556 conn(0x7f1ed40fc7a0 msgr2=0x7f1ed406aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:59.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.844+0000 7f1ed9447640 1 -- 172.21.15.116:0/1487809556 shutdown_connections 2024-09-12T03:07:59.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:07:59.844+0000 7f1ed9447640 1 -- 172.21.15.116:0/1487809556 wait complete. 2024-09-12T03:08:00.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:00 smithi116 ceph-mon[31410]: pgmap v488: 97 pgs: 97 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:08:00.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:00 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1487809556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:08:00.619 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:08:00.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:00 smithi183 ceph-mon[35535]: pgmap v488: 97 pgs: 97 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:08:00.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:00 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1487809556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:08:00.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:00 smithi177 ceph-mon[35305]: pgmap v488: 97 pgs: 97 active+clean; 2.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:08:00.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:00 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1487809556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:08:00.866 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:08:01.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.358+0000 7f34af31a640 1 -- 172.21.15.116:0/2362396341 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34a8100090 msgr2=0x7f34a8100510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:01.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.358+0000 7f34af31a640 1 --2- 172.21.15.116:0/2362396341 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34a8100090 0x7f34a8100510 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f3498009900 tx=0x7f349802a410 comp rx=0 tx=0).stop 2024-09-12T03:08:01.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.358+0000 7f34af31a640 1 -- 172.21.15.116:0/2362396341 shutdown_connections 2024-09-12T03:08:01.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.358+0000 7f34af31a640 1 --2- 172.21.15.116:0/2362396341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34a8100a50 0x7f34a81057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.358+0000 7f34af31a640 1 --2- 172.21.15.116:0/2362396341 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34a8100090 0x7f34a8100510 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.358+0000 7f34af31a640 1 --2- 172.21.15.116:0/2362396341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34a8106b70 0x7f34a8106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.358+0000 7f34af31a640 1 -- 172.21.15.116:0/2362396341 >> 172.21.15.116:0/2362396341 conn(0x7f34a8077210 msgr2=0x7f34a80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:01.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.358+0000 7f34af31a640 1 -- 172.21.15.116:0/2362396341 shutdown_connections 2024-09-12T03:08:01.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.358+0000 7f34af31a640 1 -- 172.21.15.116:0/2362396341 wait complete. 2024-09-12T03:08:01.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.359+0000 7f34af31a640 1 Processor -- start 2024-09-12T03:08:01.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.359+0000 7f34af31a640 1 -- start start 2024-09-12T03:08:01.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.359+0000 7f34af31a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34a8100090 0x7f34a81a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:01.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.359+0000 7f34af31a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34a8100a50 0x7f34a81a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:01.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.359+0000 7f34af31a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34a8106b70 0x7f34a819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:01.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.359+0000 7f34af31a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f34a81185c0 con 0x7f34a8100a50 2024-09-12T03:08:01.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.359+0000 7f34af31a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f34a8118440 con 0x7f34a8100090 2024-09-12T03:08:01.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.359+0000 7f34af31a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f34a8118740 con 0x7f34a8106b70 2024-09-12T03:08:01.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.359+0000 7f34ac8b5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34a8100a50 0x7f34a81a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:01.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.359+0000 7f34ac8b5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34a8100a50 0x7f34a81a4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45480/0 (socket says 172.21.15.116:45480) 2024-09-12T03:08:01.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.359+0000 7f34ac8b5640 1 -- 172.21.15.116:0/1750624547 learned_addr learned my addr 172.21.15.116:0/1750624547 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:01.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.359+0000 7f34ad0b6640 1 --2- 172.21.15.116:0/1750624547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34a8100090 0x7f34a81a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:01.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.360+0000 7f34ad8b7640 1 --2- 172.21.15.116:0/1750624547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34a8106b70 0x7f34a819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:01.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.360+0000 7f34ac8b5640 1 -- 172.21.15.116:0/1750624547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34a8106b70 msgr2=0x7f34a819e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:01.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.360+0000 7f34ac8b5640 1 --2- 172.21.15.116:0/1750624547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34a8106b70 0x7f34a819e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.360+0000 7f34ac8b5640 1 -- 172.21.15.116:0/1750624547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34a8100090 msgr2=0x7f34a81a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:01.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.360+0000 7f34ac8b5640 1 --2- 172.21.15.116:0/1750624547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34a8100090 0x7f34a81a4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.360+0000 7f34ac8b5640 1 -- 172.21.15.116:0/1750624547 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a819ef90 con 0x7f34a8100a50 2024-09-12T03:08:01.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.360+0000 7f34ac8b5640 1 --2- 172.21.15.116:0/1750624547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34a8100a50 0x7f34a81a4bb0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3498005ab0 tx=0x7f34980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:01.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.360+0000 7f349e7fc640 1 -- 172.21.15.116:0/1750624547 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349803f030 con 0x7f34a8100a50 2024-09-12T03:08:01.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.360+0000 7f349e7fc640 1 -- 172.21.15.116:0/1750624547 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3498003ce0 con 0x7f34a8100a50 2024-09-12T03:08:01.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.360+0000 7f34af31a640 1 -- 172.21.15.116:0/1750624547 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a819f220 con 0x7f34a8100a50 2024-09-12T03:08:01.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.360+0000 7f349e7fc640 1 -- 172.21.15.116:0/1750624547 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498035070 con 0x7f34a8100a50 2024-09-12T03:08:01.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.360+0000 7f34af31a640 1 -- 172.21.15.116:0/1750624547 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a81ab4d0 con 0x7f34a8100a50 2024-09-12T03:08:01.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.362+0000 7f349e7fc640 1 -- 172.21.15.116:0/1750624547 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3498042030 con 0x7f34a8100a50 2024-09-12T03:08:01.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.363+0000 7f34b16f3740 1 Processor -- start 2024-09-12T03:08:01.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.363+0000 7f34b16f3740 1 -- start start 2024-09-12T03:08:01.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.363+0000 7f34b16f3740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557dd8329b40 0x557dd8328090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:01.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.363+0000 7f34b16f3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dd83285d0 0x557dd8328a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:01.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.363+0000 7f34b16f3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557dd8333370 0x557dd8335760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:01.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.363+0000 7f34b16f3740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557dd828e2f0 con 0x557dd8329b40 2024-09-12T03:08:01.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.363+0000 7f34b16f3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557dd8235cf0 con 0x557dd8333370 2024-09-12T03:08:01.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.363+0000 7f34b16f3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557dd828a2b0 con 0x557dd83285d0 2024-09-12T03:08:01.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.363+0000 7f349e7fc640 1 --2- 172.21.15.116:0/1750624547 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f34940a2b80 0x7f34940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:01.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.363+0000 7f34ad8b7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557dd8329b40 0x557dd8328090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:01.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.363+0000 7f34ad8b7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557dd8329b40 0x557dd8328090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45484/0 (socket says 172.21.15.116:45484) 2024-09-12T03:08:01.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.363+0000 7f34ad8b7640 1 -- 172.21.15.116:0/561726230 learned_addr learned my addr 172.21.15.116:0/561726230 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:01.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.363+0000 7f34ad8b7640 1 --2- 172.21.15.116:0/1750624547 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f34940a2b80 0x7f34940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:01.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.363+0000 7f34ad0b6640 1 --2- 172.21.15.116:0/561726230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dd83285d0 0x557dd8328a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:01.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.363+0000 7f349e7fc640 1 -- 172.21.15.116:0/1750624547 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f34980ef890 con 0x7f34a8100a50 2024-09-12T03:08:01.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.363+0000 7f34ac8b5640 1 --2- 172.21.15.116:0/561726230 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557dd8333370 0x557dd8335760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:01.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.364+0000 7f34ad8b7640 1 -- 172.21.15.116:0/561726230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dd83285d0 msgr2=0x557dd8328a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:01.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.364+0000 7f34ad8b7640 1 --2- 172.21.15.116:0/561726230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dd83285d0 0x557dd8328a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.364+0000 7f34ad8b7640 1 -- 172.21.15.116:0/561726230 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557dd8333370 msgr2=0x557dd8335760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:01.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.364+0000 7f34ad8b7640 1 --2- 172.21.15.116:0/561726230 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557dd8333370 0x557dd8335760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.364+0000 7f34ad8b7640 1 -- 172.21.15.116:0/561726230 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dd821a3a0 con 0x557dd8329b40 2024-09-12T03:08:01.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.364+0000 7f34ad8b7640 1 --2- 172.21.15.116:0/561726230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557dd8329b40 0x557dd8328090 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f34a0014e10 tx=0x7f34a0018750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72f52000aa8bb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:01.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.364+0000 7f34ad8b7640 1 --2- 172.21.15.116:0/1750624547 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f34940a2b80 0x7f34940a5040 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f34a0018c50 tx=0x7f34a0009000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:01.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.364+0000 7f34937fe640 1 -- 172.21.15.116:0/561726230 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0026070 con 0x557dd8329b40 2024-09-12T03:08:01.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.364+0000 7f34937fe640 1 -- 172.21.15.116:0/561726230 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34a001d070 con 0x557dd8329b40 2024-09-12T03:08:01.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.364+0000 7f34937fe640 1 -- 172.21.15.116:0/561726230 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0010040 con 0x557dd8329b40 2024-09-12T03:08:01.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.365+0000 7f34b16f3740 1 -- 172.21.15.116:0/561726230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557dd8329b40 msgr2=0x557dd8328090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:01.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.365+0000 7f34b16f3740 1 --2- 172.21.15.116:0/561726230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557dd8329b40 0x557dd8328090 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f34a0014e10 tx=0x7f34a0018750 comp rx=0 tx=0).stop 2024-09-12T03:08:01.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.365+0000 7f34b16f3740 1 -- 172.21.15.116:0/561726230 shutdown_connections 2024-09-12T03:08:01.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.365+0000 7f34b16f3740 1 --2- 172.21.15.116:0/561726230 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557dd8333370 0x557dd8335760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.365+0000 7f34b16f3740 1 --2- 172.21.15.116:0/561726230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dd83285d0 0x557dd8328a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.365+0000 7f34b16f3740 1 --2- 172.21.15.116:0/561726230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557dd8329b40 0x557dd8328090 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.365+0000 7f34b16f3740 1 -- 172.21.15.116:0/561726230 >> 172.21.15.116:0/561726230 conn(0x557dd82f1310 msgr2=0x557dd8324c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:01.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.365+0000 7f34b16f3740 1 -- 172.21.15.116:0/561726230 shutdown_connections 2024-09-12T03:08:01.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.365+0000 7f34b16f3740 1 -- 172.21.15.116:0/561726230 wait complete. 2024-09-12T03:08:01.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.365+0000 7f34b16f3740 1 -- start start 2024-09-12T03:08:01.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.365+0000 7f34b16f3740 1 Processor -- start 2024-09-12T03:08:01.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.366+0000 7f34b16f3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557dd83285d0 0x557dd8332d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:01.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.366+0000 7f34b16f3740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557dd8329b40 0x557dd832ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:01.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.366+0000 7f34b16f3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dd8333370 0x557dd83305d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:01.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.366+0000 7f34b16f3740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557dd8336010 con 0x557dd8329b40 2024-09-12T03:08:01.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.366+0000 7f34b16f3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557dd8249060 con 0x557dd83285d0 2024-09-12T03:08:01.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.366+0000 7f34b16f3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557dd8336190 con 0x557dd8333370 2024-09-12T03:08:01.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.366+0000 7f34b16f3740 10 client.0 _ll_register_callbacks cb 0x557dd7f9c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:01.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.366+0000 7f34ac8b5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557dd8329b40 0x557dd832ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:01.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.366+0000 7f34ac8b5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557dd8329b40 0x557dd832ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45494/0 (socket says 172.21.15.116:45494) 2024-09-12T03:08:01.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.366+0000 7f34ac8b5640 1 -- 172.21.15.116:0/3158775378 learned_addr learned my addr 172.21.15.116:0/3158775378 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:01.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.366+0000 7f34ad0b6640 1 --2- 172.21.15.116:0/3158775378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557dd83285d0 0x557dd8332d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:01.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.366+0000 7f34ad8b7640 1 --2- 172.21.15.116:0/3158775378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dd8333370 0x557dd83305d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:01.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.366+0000 7f34ac8b5640 1 -- 172.21.15.116:0/3158775378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dd8333370 msgr2=0x557dd83305d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:01.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.366+0000 7f34ac8b5640 1 --2- 172.21.15.116:0/3158775378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dd8333370 0x557dd83305d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.366+0000 7f34ac8b5640 1 -- 172.21.15.116:0/3158775378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557dd83285d0 msgr2=0x557dd8332d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:01.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.366+0000 7f34ac8b5640 1 --2- 172.21.15.116:0/3158775378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557dd83285d0 0x557dd8332d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.366+0000 7f34ac8b5640 1 -- 172.21.15.116:0/3158775378 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dd7fff3c0 con 0x557dd8329b40 2024-09-12T03:08:01.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.366+0000 7f34ad0b6640 1 --2- 172.21.15.116:0/3158775378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557dd83285d0 0x557dd8332d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:01.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.366+0000 7f34ac8b5640 1 --2- 172.21.15.116:0/3158775378 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557dd8329b40 0x557dd832ffe0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3498005ec0 tx=0x7f3498009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa279f0015d4c8bb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:01.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.366+0000 7f3490ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:01.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.366+0000 7f3490ff9640 1 -- 172.21.15.116:0/3158775378 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498049070 con 0x557dd8329b40 2024-09-12T03:08:01.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.366+0000 7f3490ff9640 1 -- 172.21.15.116:0/3158775378 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f349803b460 con 0x557dd8329b40 2024-09-12T03:08:01.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.366+0000 7f34b16f3740 1 -- 172.21.15.116:0/3158775378 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34a001d070 con 0x557dd8329b40 2024-09-12T03:08:01.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.366+0000 7f3490ff9640 1 -- 172.21.15.116:0/3158775378 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f34980bc820 con 0x557dd8329b40 2024-09-12T03:08:01.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.366+0000 7f34b16f3740 10 client.16500 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:01.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.367+0000 7f34b16f3740 1 -- 172.21.15.116:0/3158775378 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557dd8341a70 con 0x557dd8329b40 2024-09-12T03:08:01.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.367+0000 7f3490ff9640 1 -- 172.21.15.116:0/3158775378 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34a001d070 con 0x557dd8329b40 2024-09-12T03:08:01.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.367+0000 7f3490ff9640 1 -- 172.21.15.116:0/3158775378 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f34980b8790 con 0x557dd8329b40 2024-09-12T03:08:01.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.367+0000 7f34b16f3740 10 client.16500 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:01.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.367+0000 7f34b16f3740 10 client.16500 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:08:01.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.367+0000 7f34b16f3740 20 client.16500 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:01.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.367+0000 7f34b16f3740 1 --2- 172.21.15.116:0/3158775378 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x557dd8342eb0 0x557dd83452c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:01.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.367+0000 7f34b16f3740 4 client.16500 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:08:01.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.367+0000 7f34b16f3740 1 -- 172.21.15.116:0/3158775378 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dd829b860 con 0x557dd8342eb0 2024-09-12T03:08:01.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.367+0000 7f34ad0b6640 1 --2- 172.21.15.116:0/3158775378 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x557dd8342eb0 0x557dd83452c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:01.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.368+0000 7f3490ff9640 10 client.16500 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:08:01.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.368+0000 7f34ad0b6640 1 --2- 172.21.15.116:0/3158775378 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x557dd8342eb0 0x557dd83452c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=59ae9ec8c7bc23b1 server_cookie=90cc4b59e59f3616 in_seq=0 out_seq=0 2024-09-12T03:08:01.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.368+0000 7f3490ff9640 1 -- 172.21.15.116:0/3158775378 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557dd829b860 con 0x557dd8342eb0 2024-09-12T03:08:01.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.368+0000 7f3490ff9640 10 client.16500 handle_command_reply: tid=0 2024-09-12T03:08:01.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.368+0000 7f34b16f3740 1 client.16500 shutdown 2024-09-12T03:08:01.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.368+0000 7f34b16f3740 1 -- 172.21.15.116:0/3158775378 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557dd8329b40 msgr2=0x557dd832ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:01.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.368+0000 7f34b16f3740 1 --2- 172.21.15.116:0/3158775378 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557dd8329b40 0x557dd832ffe0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3498005ec0 tx=0x7f3498009660 comp rx=0 tx=0).stop 2024-09-12T03:08:01.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.368+0000 7f34b16f3740 1 -- 172.21.15.116:0/3158775378 shutdown_connections 2024-09-12T03:08:01.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.368+0000 7f34b16f3740 1 --2- 172.21.15.116:0/3158775378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dd8333370 0x557dd83305d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.368+0000 7f34b16f3740 1 --2- 172.21.15.116:0/3158775378 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x557dd8342eb0 0x557dd83452c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.368+0000 7f34b16f3740 1 --2- 172.21.15.116:0/3158775378 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557dd8329b40 0x557dd832ffe0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.368+0000 7f34b16f3740 1 --2- 172.21.15.116:0/3158775378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557dd83285d0 0x557dd8332d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.368+0000 7f3490ff9640 0 client.16500 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:08:01.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.368+0000 7f34b16f3740 1 -- 172.21.15.116:0/3158775378 >> 172.21.15.116:0/3158775378 conn(0x557dd82f1310 msgr2=0x557dd832a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:01.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.369+0000 7f34b16f3740 1 -- 172.21.15.116:0/3158775378 shutdown_connections 2024-09-12T03:08:01.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.369+0000 7f34b16f3740 1 -- 172.21.15.116:0/3158775378 wait complete. 2024-09-12T03:08:01.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.369+0000 7f34b16f3740 20 client.16500 trim_cache size 0 max 16384 2024-09-12T03:08:01.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.381+0000 7f34b16f3740 1 Processor -- start 2024-09-12T03:08:01.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.381+0000 7f34b16f3740 1 -- start start 2024-09-12T03:08:01.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.382+0000 7f34b16f3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dd83285d0 0x557dd832b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:01.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.382+0000 7f34b16f3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557dd8329b40 0x557dd832b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:01.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.382+0000 7f34b16f3740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557dd8333370 0x557dd834abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:01.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.382+0000 7f34b16f3740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f34a81b63a0 con 0x557dd8333370 2024-09-12T03:08:01.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.382+0000 7f34b16f3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f34a81b3c70 con 0x557dd8329b40 2024-09-12T03:08:01.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.382+0000 7f34b16f3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f34a81b1ac0 con 0x557dd83285d0 2024-09-12T03:08:01.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.382+0000 7f34ad8b7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557dd8333370 0x557dd834abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:01.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.382+0000 7f34ad8b7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557dd8333370 0x557dd834abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45496/0 (socket says 172.21.15.116:45496) 2024-09-12T03:08:01.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.382+0000 7f34ad8b7640 1 -- 172.21.15.116:0/2031928887 learned_addr learned my addr 172.21.15.116:0/2031928887 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:01.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.382+0000 7f34ad0b6640 1 --2- 172.21.15.116:0/2031928887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dd83285d0 0x557dd832b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:01.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.382+0000 7f34ac8b5640 1 --2- 172.21.15.116:0/2031928887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557dd8329b40 0x557dd832b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:01.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.382+0000 7f34ad8b7640 1 -- 172.21.15.116:0/2031928887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dd83285d0 msgr2=0x557dd832b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:01.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.382+0000 7f34ad8b7640 1 --2- 172.21.15.116:0/2031928887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dd83285d0 0x557dd832b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.382+0000 7f34ad8b7640 1 -- 172.21.15.116:0/2031928887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557dd8329b40 msgr2=0x557dd832b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:01.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.382+0000 7f34ad8b7640 1 --2- 172.21.15.116:0/2031928887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557dd8329b40 0x557dd832b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.382+0000 7f34ad8b7640 1 -- 172.21.15.116:0/2031928887 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dd83330a0 con 0x557dd8333370 2024-09-12T03:08:01.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.382+0000 7f34ad8b7640 1 --2- 172.21.15.116:0/2031928887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557dd8333370 0x557dd834abd0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f34a0001dc0 tx=0x7f34a0014280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7a1a150aa654b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:01.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.382+0000 7f3492ffd640 1 -- 172.21.15.116:0/2031928887 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0026040 con 0x557dd8333370 2024-09-12T03:08:01.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.383+0000 7f3492ffd640 1 -- 172.21.15.116:0/2031928887 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34a001d040 con 0x557dd8333370 2024-09-12T03:08:01.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.383+0000 7f3492ffd640 1 -- 172.21.15.116:0/2031928887 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a00042b0 con 0x557dd8333370 2024-09-12T03:08:01.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.383+0000 7f34b16f3740 1 -- 172.21.15.116:0/2031928887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557dd8333370 msgr2=0x557dd834abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:01.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.383+0000 7f34b16f3740 1 --2- 172.21.15.116:0/2031928887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557dd8333370 0x557dd834abd0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f34a0001dc0 tx=0x7f34a0014280 comp rx=0 tx=0).stop 2024-09-12T03:08:01.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.383+0000 7f34b16f3740 1 -- 172.21.15.116:0/2031928887 shutdown_connections 2024-09-12T03:08:01.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.383+0000 7f34b16f3740 1 --2- 172.21.15.116:0/2031928887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557dd8333370 0x557dd834abd0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.383+0000 7f34b16f3740 1 --2- 172.21.15.116:0/2031928887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557dd8329b40 0x557dd832b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.383+0000 7f34b16f3740 1 --2- 172.21.15.116:0/2031928887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dd83285d0 0x557dd832b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.383+0000 7f34b16f3740 1 -- 172.21.15.116:0/2031928887 >> 172.21.15.116:0/2031928887 conn(0x557dd82f1310 msgr2=0x557dd8332060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:01.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.383+0000 7f34b16f3740 1 -- 172.21.15.116:0/2031928887 shutdown_connections 2024-09-12T03:08:01.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.383+0000 7f34b16f3740 1 -- 172.21.15.116:0/2031928887 wait complete. 2024-09-12T03:08:01.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.384+0000 7f34b16f3740 1 -- start start 2024-09-12T03:08:01.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.384+0000 7f34b16f3740 1 Processor -- start 2024-09-12T03:08:01.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.384+0000 7f34b16f3740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557dd83285d0 0x557dd833b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:01.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.384+0000 7f34b16f3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dd8329b40 0x557dd833bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:01.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.384+0000 7f34b16f3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557dd8333370 0x557dd83383f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:01.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.384+0000 7f34b16f3740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f34a81b2130 con 0x557dd83285d0 2024-09-12T03:08:01.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.384+0000 7f34b16f3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f34a81b35a0 con 0x557dd8333370 2024-09-12T03:08:01.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.384+0000 7f34b16f3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f34a81b2800 con 0x557dd8329b40 2024-09-12T03:08:01.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.384+0000 7f34b16f3740 10 client.0 _ll_register_callbacks cb 0x7f3498003e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:01.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.384+0000 7f34ad0b6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557dd83285d0 0x557dd833b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:01.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.384+0000 7f34ad0b6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557dd83285d0 0x557dd833b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45510/0 (socket says 172.21.15.116:45510) 2024-09-12T03:08:01.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.384+0000 7f34ad0b6640 1 -- 172.21.15.116:0/1899522636 learned_addr learned my addr 172.21.15.116:0/1899522636 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:01.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.384+0000 7f34ac8b5640 1 --2- 172.21.15.116:0/1899522636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dd8329b40 0x557dd833bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:01.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.384+0000 7f34ad8b7640 1 --2- 172.21.15.116:0/1899522636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557dd8333370 0x557dd83383f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:01.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.384+0000 7f34ad0b6640 1 -- 172.21.15.116:0/1899522636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dd8329b40 msgr2=0x557dd833bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:01.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.384+0000 7f34ad0b6640 1 --2- 172.21.15.116:0/1899522636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dd8329b40 0x557dd833bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.384+0000 7f34ad0b6640 1 -- 172.21.15.116:0/1899522636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557dd8333370 msgr2=0x557dd83383f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:01.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.384+0000 7f34ad0b6640 1 --2- 172.21.15.116:0/1899522636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557dd8333370 0x557dd83383f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.385+0000 7f34ad0b6640 1 -- 172.21.15.116:0/1899522636 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dd8338c20 con 0x557dd83285d0 2024-09-12T03:08:01.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.385+0000 7f34ad0b6640 1 --2- 172.21.15.116:0/1899522636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557dd83285d0 0x557dd833b600 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f34a4000c00 tx=0x7f34a401fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90cd7e3a8300cc7c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:01.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.385+0000 7f3490ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:01.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.385+0000 7f3490ff9640 1 -- 172.21.15.116:0/1899522636 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4011b80 con 0x557dd83285d0 2024-09-12T03:08:01.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.385+0000 7f3490ff9640 1 -- 172.21.15.116:0/1899522636 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f34a4011d20 con 0x557dd83285d0 2024-09-12T03:08:01.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.385+0000 7f34b16f3740 1 -- 172.21.15.116:0/1899522636 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34a001d040 con 0x557dd83285d0 2024-09-12T03:08:01.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.385+0000 7f3490ff9640 1 -- 172.21.15.116:0/1899522636 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f34a4005020 con 0x557dd83285d0 2024-09-12T03:08:01.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.385+0000 7f34b16f3740 10 client.16506 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:01.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.385+0000 7f34b16f3740 1 -- 172.21.15.116:0/1899522636 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557dd83ae730 con 0x557dd83285d0 2024-09-12T03:08:01.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.385+0000 7f3490ff9640 1 -- 172.21.15.116:0/1899522636 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34a001d040 con 0x557dd83285d0 2024-09-12T03:08:01.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.385+0000 7f3490ff9640 1 -- 172.21.15.116:0/1899522636 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f34a4030410 con 0x557dd83285d0 2024-09-12T03:08:01.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.385+0000 7f34b16f3740 10 client.16506 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:01.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.385+0000 7f34b16f3740 10 client.16506 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:08:01.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.385+0000 7f34b16f3740 20 client.16506 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:01.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.385+0000 7f34b16f3740 1 --2- 172.21.15.116:0/1899522636 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x557dd83aee50 0x557dd83b1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:01.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.385+0000 7f34b16f3740 4 client.16506 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:08:01.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.385+0000 7f34b16f3740 1 -- 172.21.15.116:0/1899522636 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557dd8295d70 con 0x557dd83aee50 2024-09-12T03:08:01.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.386+0000 7f34ac8b5640 1 --2- 172.21.15.116:0/1899522636 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x557dd83aee50 0x557dd83b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:01.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.386+0000 7f3490ff9640 10 client.16506 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:08:01.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.386+0000 7f34ac8b5640 1 --2- 172.21.15.116:0/1899522636 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x557dd83aee50 0x557dd83b1240 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b3b7035999a089e7 server_cookie=1d0e34632aea6eb8 in_seq=0 out_seq=0 2024-09-12T03:08:01.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.386+0000 7f3490ff9640 1 -- 172.21.15.116:0/1899522636 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557dd8295d70 con 0x557dd83aee50 2024-09-12T03:08:01.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.386+0000 7f3490ff9640 10 client.16506 handle_command_reply: tid=0 2024-09-12T03:08:01.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.386+0000 7f34b16f3740 1 client.16506 shutdown 2024-09-12T03:08:01.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.386+0000 7f34b16f3740 1 -- 172.21.15.116:0/1899522636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557dd83285d0 msgr2=0x557dd833b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:01.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.386+0000 7f34b16f3740 1 --2- 172.21.15.116:0/1899522636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557dd83285d0 0x557dd833b600 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f34a4000c00 tx=0x7f34a401fae0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.387+0000 7f34b16f3740 1 -- 172.21.15.116:0/1899522636 shutdown_connections 2024-09-12T03:08:01.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.387+0000 7f34b16f3740 1 --2- 172.21.15.116:0/1899522636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557dd8333370 0x557dd83383f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.387+0000 7f34b16f3740 1 --2- 172.21.15.116:0/1899522636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dd8329b40 0x557dd833bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.387+0000 7f34b16f3740 1 --2- 172.21.15.116:0/1899522636 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x557dd83aee50 0x557dd83b1240 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.387+0000 7f34b16f3740 1 --2- 172.21.15.116:0/1899522636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557dd83285d0 0x557dd833b600 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.387+0000 7f34b16f3740 1 -- 172.21.15.116:0/1899522636 >> 172.21.15.116:0/1899522636 conn(0x557dd82f1310 msgr2=0x557dd83234e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:01.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.387+0000 7f3490ff9640 0 client.16506 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:08:01.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.387+0000 7f34b16f3740 1 -- 172.21.15.116:0/1899522636 shutdown_connections 2024-09-12T03:08:01.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.387+0000 7f34b16f3740 1 -- 172.21.15.116:0/1899522636 wait complete. 2024-09-12T03:08:01.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.387+0000 7f34b16f3740 20 client.16506 trim_cache size 0 max 16384 2024-09-12T03:08:01.400 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:08:01.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.387+0000 7f34af31a640 1 -- 172.21.15.116:0/1750624547 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f34940a2b80 msgr2=0x7f34940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:01.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.387+0000 7f34af31a640 1 --2- 172.21.15.116:0/1750624547 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f34940a2b80 0x7f34940a5040 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f34a0018c50 tx=0x7f34a0009000 comp rx=0 tx=0).stop 2024-09-12T03:08:01.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.387+0000 7f34af31a640 1 -- 172.21.15.116:0/1750624547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34a8100a50 msgr2=0x7f34a81a4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:01.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.387+0000 7f34af31a640 1 --2- 172.21.15.116:0/1750624547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34a8100a50 0x7f34a81a4bb0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3498005ab0 tx=0x7f34980042f0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.387+0000 7f34af31a640 1 -- 172.21.15.116:0/1750624547 shutdown_connections 2024-09-12T03:08:01.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.387+0000 7f34af31a640 1 --2- 172.21.15.116:0/1750624547 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f34940a2b80 0x7f34940a5040 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.387+0000 7f34af31a640 1 --2- 172.21.15.116:0/1750624547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34a8106b70 0x7f34a819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.387+0000 7f34af31a640 1 --2- 172.21.15.116:0/1750624547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34a8100a50 0x7f34a81a4bb0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.387+0000 7f34af31a640 1 --2- 172.21.15.116:0/1750624547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34a8100090 0x7f34a81a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:01.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.387+0000 7f34af31a640 1 -- 172.21.15.116:0/1750624547 >> 172.21.15.116:0/1750624547 conn(0x7f34a8077210 msgr2=0x7f34a81037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:01.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.388+0000 7f34af31a640 1 -- 172.21.15.116:0/1750624547 shutdown_connections 2024-09-12T03:08:01.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:01.388+0000 7f34af31a640 1 -- 172.21.15.116:0/1750624547 wait complete. 2024-09-12T03:08:02.089 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:08:02.339 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:08:02.461 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:02 smithi116 ceph-mon[31410]: pgmap v489: 97 pgs: 97 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:08:02.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:02 smithi183 ceph-mon[35535]: pgmap v489: 97 pgs: 97 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:08:02.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:02 smithi177 ceph-mon[35305]: pgmap v489: 97 pgs: 97 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:08:02.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.852+0000 7f40d5807640 1 -- 172.21.15.116:0/2262112018 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40d0102ef0 msgr2=0x7f40d010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:02.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.852+0000 7f40d5807640 1 --2- 172.21.15.116:0/2262112018 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40d0102ef0 0x7f40d010da40 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f40c8009ca0 tx=0x7f40c802a950 comp rx=0 tx=0).stop 2024-09-12T03:08:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.852+0000 7f40d5807640 1 -- 172.21.15.116:0/2262112018 shutdown_connections 2024-09-12T03:08:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.852+0000 7f40d5807640 1 --2- 172.21.15.116:0/2262112018 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40d0102ef0 0x7f40d010da40 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.852+0000 7f40d5807640 1 --2- 172.21.15.116:0/2262112018 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40d0102570 0x7f40d01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.852+0000 7f40d5807640 1 --2- 172.21.15.116:0/2262112018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40d0100e20 0x7f40d0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.852+0000 7f40d5807640 1 -- 172.21.15.116:0/2262112018 >> 172.21.15.116:0/2262112018 conn(0x7f40d00fc7d0 msgr2=0x7f40d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.852+0000 7f40d5807640 1 -- 172.21.15.116:0/2262112018 shutdown_connections 2024-09-12T03:08:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.852+0000 7f40d5807640 1 -- 172.21.15.116:0/2262112018 wait complete. 2024-09-12T03:08:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.852+0000 7f40d5807640 1 Processor -- start 2024-09-12T03:08:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.853+0000 7f40d5807640 1 -- start start 2024-09-12T03:08:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.853+0000 7f40d5807640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40d0100e20 0x7f40d01993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.853+0000 7f40d5807640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40d0102570 0x7f40d01998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.853+0000 7f40d5807640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40d0102ef0 0x7f40d0199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.853+0000 7f40d5807640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f40d0110f80 con 0x7f40d0102570 2024-09-12T03:08:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.853+0000 7f40d5807640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f40d0110e00 con 0x7f40d0102ef0 2024-09-12T03:08:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.853+0000 7f40d5807640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f40d0111100 con 0x7f40d0100e20 2024-09-12T03:08:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.853+0000 7f40ce7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40d0102570 0x7f40d01998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.853+0000 7f40ce7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40d0102570 0x7f40d01998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45534/0 (socket says 172.21.15.116:45534) 2024-09-12T03:08:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.853+0000 7f40ce7fc640 1 -- 172.21.15.116:0/3392708245 learned_addr learned my addr 172.21.15.116:0/3392708245 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.853+0000 7f40ceffd640 1 --2- 172.21.15.116:0/3392708245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40d0100e20 0x7f40d01993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.853+0000 7f40cf7fe640 1 --2- 172.21.15.116:0/3392708245 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40d0102ef0 0x7f40d0199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.853+0000 7f40ce7fc640 1 -- 172.21.15.116:0/3392708245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40d0100e20 msgr2=0x7f40d01993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.853+0000 7f40ce7fc640 1 --2- 172.21.15.116:0/3392708245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40d0100e20 0x7f40d01993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.853+0000 7f40ce7fc640 1 -- 172.21.15.116:0/3392708245 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40d0102ef0 msgr2=0x7f40d0199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.853+0000 7f40ce7fc640 1 --2- 172.21.15.116:0/3392708245 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40d0102ef0 0x7f40d0199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.853+0000 7f40ce7fc640 1 -- 172.21.15.116:0/3392708245 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d01a47b0 con 0x7f40d0102570 2024-09-12T03:08:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.853+0000 7f40cf7fe640 1 --2- 172.21.15.116:0/3392708245 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40d0102ef0 0x7f40d0199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:08:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.853+0000 7f40ceffd640 1 --2- 172.21.15.116:0/3392708245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40d0100e20 0x7f40d01993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.854+0000 7f40ce7fc640 1 --2- 172.21.15.116:0/3392708245 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40d0102570 0x7f40d01998f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f40b800e9f0 tx=0x7f40b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.854+0000 7f40d4805640 1 -- 172.21.15.116:0/3392708245 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b800cd80 con 0x7f40d0102570 2024-09-12T03:08:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.854+0000 7f40d5807640 1 -- 172.21.15.116:0/3392708245 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40d01a4aa0 con 0x7f40d0102570 2024-09-12T03:08:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.854+0000 7f40d4805640 1 -- 172.21.15.116:0/3392708245 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40b8010910 con 0x7f40d0102570 2024-09-12T03:08:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.854+0000 7f40d4805640 1 -- 172.21.15.116:0/3392708245 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b80188f0 con 0x7f40d0102570 2024-09-12T03:08:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.854+0000 7f40d5807640 1 -- 172.21.15.116:0/3392708245 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40d01a5030 con 0x7f40d0102570 2024-09-12T03:08:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.856+0000 7f40d4805640 1 -- 172.21.15.116:0/3392708245 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f40b8010ac0 con 0x7f40d0102570 2024-09-12T03:08:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.856+0000 7f40d7be0740 1 Processor -- start 2024-09-12T03:08:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.856+0000 7f40d7be0740 1 -- start start 2024-09-12T03:08:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.856+0000 7f40d7be0740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560f3a84ab40 0x560f3a849090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.856+0000 7f40d7be0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3a8495d0 0x560f3a849a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.856+0000 7f40d7be0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560f3a854370 0x560f3a856760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.856+0000 7f40d7be0740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560f3a7af2f0 con 0x560f3a84ab40 2024-09-12T03:08:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.856+0000 7f40d7be0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560f3a756cf0 con 0x560f3a854370 2024-09-12T03:08:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.856+0000 7f40d7be0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560f3a7ab2b0 con 0x560f3a8495d0 2024-09-12T03:08:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.857+0000 7f40cf7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560f3a84ab40 0x560f3a849090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.857+0000 7f40cf7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560f3a84ab40 0x560f3a849090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45546/0 (socket says 172.21.15.116:45546) 2024-09-12T03:08:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.857+0000 7f40cf7fe640 1 -- 172.21.15.116:0/1470046801 learned_addr learned my addr 172.21.15.116:0/1470046801 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.857+0000 7f40d4805640 1 --2- 172.21.15.116:0/3392708245 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f40bc0a2b80 0x7f40bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.857+0000 7f40ce7fc640 1 --2- 172.21.15.116:0/1470046801 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560f3a854370 0x560f3a856760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.857+0000 7f40ceffd640 1 --2- 172.21.15.116:0/1470046801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3a8495d0 0x560f3a849a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.857+0000 7f40cf7fe640 1 --2- 172.21.15.116:0/3392708245 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f40bc0a2b80 0x7f40bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.857+0000 7f40cf7fe640 1 -- 172.21.15.116:0/1470046801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3a8495d0 msgr2=0x560f3a849a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.857+0000 7f40cf7fe640 1 --2- 172.21.15.116:0/1470046801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3a8495d0 0x560f3a849a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.857+0000 7f40cf7fe640 1 -- 172.21.15.116:0/1470046801 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560f3a854370 msgr2=0x560f3a856760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.857+0000 7f40cf7fe640 1 --2- 172.21.15.116:0/1470046801 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560f3a854370 0x560f3a856760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.857+0000 7f40d4805640 1 -- 172.21.15.116:0/3392708245 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f40b80cf4b0 con 0x7f40d0102570 2024-09-12T03:08:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.857+0000 7f40cf7fe640 1 -- 172.21.15.116:0/1470046801 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f3a73b3a0 con 0x560f3a84ab40 2024-09-12T03:08:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.857+0000 7f40cf7fe640 1 --2- 172.21.15.116:0/1470046801 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560f3a84ab40 0x560f3a849090 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f40c8004af0 tx=0x7f40c802fc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d654a13fc2fdc8d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.857+0000 7f40c5ffb640 1 -- 172.21.15.116:0/1470046801 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c8034070 con 0x560f3a84ab40 2024-09-12T03:08:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.857+0000 7f40c5ffb640 1 -- 172.21.15.116:0/1470046801 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40c804d070 con 0x560f3a84ab40 2024-09-12T03:08:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.858+0000 7f40c5ffb640 1 -- 172.21.15.116:0/1470046801 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c8048470 con 0x560f3a84ab40 2024-09-12T03:08:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.858+0000 7f40d7be0740 1 -- 172.21.15.116:0/1470046801 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560f3a84ab40 msgr2=0x560f3a849090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.858+0000 7f40cf7fe640 1 --2- 172.21.15.116:0/3392708245 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f40bc0a2b80 0x7f40bc0a5040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f40c804c000 tx=0x7f40c804a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.858+0000 7f40d7be0740 1 --2- 172.21.15.116:0/1470046801 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560f3a84ab40 0x560f3a849090 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f40c8004af0 tx=0x7f40c802fc40 comp rx=0 tx=0).stop 2024-09-12T03:08:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.858+0000 7f40d7be0740 1 -- 172.21.15.116:0/1470046801 shutdown_connections 2024-09-12T03:08:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.858+0000 7f40d7be0740 1 --2- 172.21.15.116:0/1470046801 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560f3a854370 0x560f3a856760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.858+0000 7f40d7be0740 1 --2- 172.21.15.116:0/1470046801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3a8495d0 0x560f3a849a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.858+0000 7f40d7be0740 1 --2- 172.21.15.116:0/1470046801 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560f3a84ab40 0x560f3a849090 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.858+0000 7f40d7be0740 1 -- 172.21.15.116:0/1470046801 >> 172.21.15.116:0/1470046801 conn(0x560f3a812310 msgr2=0x560f3a845c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:02.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.858+0000 7f40d7be0740 1 -- 172.21.15.116:0/1470046801 shutdown_connections 2024-09-12T03:08:02.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.858+0000 7f40d7be0740 1 -- 172.21.15.116:0/1470046801 wait complete. 2024-09-12T03:08:02.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.859+0000 7f40d7be0740 1 -- start start 2024-09-12T03:08:02.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.859+0000 7f40d7be0740 1 Processor -- start 2024-09-12T03:08:02.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.859+0000 7f40d7be0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3a8495d0 0x560f3a853d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:02.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.859+0000 7f40d7be0740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560f3a84ab40 0x560f3a851000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:02.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.859+0000 7f40d7be0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560f3a854370 0x560f3a8515d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:02.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.859+0000 7f40d7be0740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560f3a857070 con 0x560f3a84ab40 2024-09-12T03:08:02.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.859+0000 7f40d7be0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560f3a76a060 con 0x560f3a854370 2024-09-12T03:08:02.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.859+0000 7f40d7be0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560f3a8571f0 con 0x560f3a8495d0 2024-09-12T03:08:02.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.859+0000 7f40d7be0740 10 client.0 _ll_register_callbacks cb 0x560f3a4bd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:02.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.859+0000 7f40ce7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560f3a84ab40 0x560f3a851000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:02.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.859+0000 7f40ce7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560f3a84ab40 0x560f3a851000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45560/0 (socket says 172.21.15.116:45560) 2024-09-12T03:08:02.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.859+0000 7f40ce7fc640 1 -- 172.21.15.116:0/2481051592 learned_addr learned my addr 172.21.15.116:0/2481051592 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:02.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.859+0000 7f40ceffd640 1 --2- 172.21.15.116:0/2481051592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3a8495d0 0x560f3a853d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:02.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.859+0000 7f40ce7fc640 1 -- 172.21.15.116:0/2481051592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3a8495d0 msgr2=0x560f3a853d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:02.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.859+0000 7f40ce7fc640 1 --2- 172.21.15.116:0/2481051592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3a8495d0 0x560f3a853d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.859+0000 7f40ce7fc640 1 -- 172.21.15.116:0/2481051592 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560f3a854370 msgr2=0x560f3a8515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:02.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.859+0000 7f40ce7fc640 1 --2- 172.21.15.116:0/2481051592 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560f3a854370 0x560f3a8515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.859+0000 7f40ce7fc640 1 -- 172.21.15.116:0/2481051592 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f3a5203c0 con 0x560f3a84ab40 2024-09-12T03:08:02.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.859+0000 7f40ceffd640 1 --2- 172.21.15.116:0/2481051592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3a8495d0 0x560f3a853d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:02.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.860+0000 7f40ce7fc640 1 --2- 172.21.15.116:0/2481051592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560f3a84ab40 0x560f3a851000 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f40b8098710 tx=0x7f40b809b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18c64baca23e46e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:02.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.860+0000 7f40aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:02.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.860+0000 7f40aeffd640 1 -- 172.21.15.116:0/2481051592 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b80cf310 con 0x560f3a84ab40 2024-09-12T03:08:02.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.860+0000 7f40aeffd640 1 -- 172.21.15.116:0/2481051592 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f40b80a4520 con 0x560f3a84ab40 2024-09-12T03:08:02.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.860+0000 7f40d7be0740 1 -- 172.21.15.116:0/2481051592 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40c804d070 con 0x560f3a84ab40 2024-09-12T03:08:02.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.860+0000 7f40aeffd640 1 -- 172.21.15.116:0/2481051592 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f40b80a00f0 con 0x560f3a84ab40 2024-09-12T03:08:02.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.860+0000 7f40d7be0740 10 client.16518 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:02.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.860+0000 7f40d7be0740 1 -- 172.21.15.116:0/2481051592 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560f3a862bd0 con 0x560f3a84ab40 2024-09-12T03:08:02.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.860+0000 7f40aeffd640 1 -- 172.21.15.116:0/2481051592 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40c804d070 con 0x560f3a84ab40 2024-09-12T03:08:02.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.860+0000 7f40aeffd640 1 -- 172.21.15.116:0/2481051592 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f40b8097b20 con 0x560f3a84ab40 2024-09-12T03:08:02.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.860+0000 7f40d7be0740 10 client.16518 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:02.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.860+0000 7f40d7be0740 10 client.16518 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:08:02.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.860+0000 7f40d7be0740 20 client.16518 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:02.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.860+0000 7f40d7be0740 1 --2- 172.21.15.116:0/2481051592 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x560f3a864030 0x560f3a866480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:02.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.860+0000 7f40d7be0740 4 client.16518 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:08:02.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.860+0000 7f40d7be0740 1 -- 172.21.15.116:0/2481051592 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f3a7bc860 con 0x560f3a864030 2024-09-12T03:08:02.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.861+0000 7f40ceffd640 1 --2- 172.21.15.116:0/2481051592 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x560f3a864030 0x560f3a866480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:02.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.861+0000 7f40aeffd640 10 client.16518 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:08:02.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.861+0000 7f40ceffd640 1 --2- 172.21.15.116:0/2481051592 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x560f3a864030 0x560f3a866480 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b34a6227e3bd9ca9 server_cookie=171ebbcd68fccb45 in_seq=0 out_seq=0 2024-09-12T03:08:02.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.862+0000 7f40aeffd640 1 -- 172.21.15.116:0/2481051592 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f3a7bc860 con 0x560f3a864030 2024-09-12T03:08:02.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.862+0000 7f40aeffd640 10 client.16518 handle_command_reply: tid=0 2024-09-12T03:08:02.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.862+0000 7f40d7be0740 1 client.16518 shutdown 2024-09-12T03:08:02.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.862+0000 7f40d7be0740 1 -- 172.21.15.116:0/2481051592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560f3a84ab40 msgr2=0x560f3a851000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:02.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.862+0000 7f40d7be0740 1 --2- 172.21.15.116:0/2481051592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560f3a84ab40 0x560f3a851000 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f40b8098710 tx=0x7f40b809b6a0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.862+0000 7f40d7be0740 1 -- 172.21.15.116:0/2481051592 shutdown_connections 2024-09-12T03:08:02.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.862+0000 7f40d7be0740 1 --2- 172.21.15.116:0/2481051592 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x560f3a864030 0x560f3a866480 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.862+0000 7f40d7be0740 1 --2- 172.21.15.116:0/2481051592 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560f3a854370 0x560f3a8515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.862+0000 7f40d7be0740 1 --2- 172.21.15.116:0/2481051592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560f3a84ab40 0x560f3a851000 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.862+0000 7f40aeffd640 0 client.16518 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:08:02.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.862+0000 7f40d7be0740 1 --2- 172.21.15.116:0/2481051592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3a8495d0 0x560f3a853d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.862+0000 7f40d7be0740 1 -- 172.21.15.116:0/2481051592 >> 172.21.15.116:0/2481051592 conn(0x560f3a812310 msgr2=0x560f3a84b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:02.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.862+0000 7f40d7be0740 1 -- 172.21.15.116:0/2481051592 shutdown_connections 2024-09-12T03:08:02.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.862+0000 7f40d7be0740 1 -- 172.21.15.116:0/2481051592 wait complete. 2024-09-12T03:08:02.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.862+0000 7f40d7be0740 20 client.16518 trim_cache size 0 max 16384 2024-09-12T03:08:02.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.875+0000 7f40d7be0740 1 Processor -- start 2024-09-12T03:08:02.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.875+0000 7f40d7be0740 1 -- start start 2024-09-12T03:08:02.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.876+0000 7f40d7be0740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560f3a8495d0 0x560f3a860340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:02.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.876+0000 7f40d7be0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560f3a84ab40 0x560f3a84c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:02.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.876+0000 7f40d7be0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3a854370 0x560f3a84c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.876+0000 7f40d7be0740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f40d01afcb0 con 0x560f3a8495d0 2024-09-12T03:08:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.876+0000 7f40d7be0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f40d01ad580 con 0x560f3a84ab40 2024-09-12T03:08:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.876+0000 7f40d7be0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f40d01ab3f0 con 0x560f3a854370 2024-09-12T03:08:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.876+0000 7f40ceffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560f3a8495d0 0x560f3a860340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.876+0000 7f40ceffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560f3a8495d0 0x560f3a860340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45564/0 (socket says 172.21.15.116:45564) 2024-09-12T03:08:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.876+0000 7f40ceffd640 1 -- 172.21.15.116:0/475840059 learned_addr learned my addr 172.21.15.116:0/475840059 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.876+0000 7f40ce7fc640 1 --2- 172.21.15.116:0/475840059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560f3a84ab40 0x560f3a84c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.876+0000 7f40ceffd640 1 -- 172.21.15.116:0/475840059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3a854370 msgr2=0x560f3a84c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.876+0000 7f40ceffd640 1 --2- 172.21.15.116:0/475840059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3a854370 0x560f3a84c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.876+0000 7f40ceffd640 1 -- 172.21.15.116:0/475840059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560f3a84ab40 msgr2=0x560f3a84c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.876+0000 7f40ceffd640 1 --2- 172.21.15.116:0/475840059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560f3a84ab40 0x560f3a84c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.876+0000 7f40ceffd640 1 -- 172.21.15.116:0/475840059 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f3a84f550 con 0x560f3a8495d0 2024-09-12T03:08:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.876+0000 7f40ce7fc640 1 --2- 172.21.15.116:0/475840059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560f3a84ab40 0x560f3a84c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:08:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.876+0000 7f40ceffd640 1 --2- 172.21.15.116:0/475840059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560f3a8495d0 0x560f3a860340 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x560f3a866cf0 tx=0x7f40c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e0009c5d104de50 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.876+0000 7f40c57fa640 1 -- 172.21.15.116:0/475840059 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c0022020 con 0x560f3a8495d0 2024-09-12T03:08:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.876+0000 7f40c57fa640 1 -- 172.21.15.116:0/475840059 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40c0010d60 con 0x560f3a8495d0 2024-09-12T03:08:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.876+0000 7f40c57fa640 1 -- 172.21.15.116:0/475840059 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c001c3a0 con 0x560f3a8495d0 2024-09-12T03:08:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.877+0000 7f40d7be0740 1 -- 172.21.15.116:0/475840059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560f3a8495d0 msgr2=0x560f3a860340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.877+0000 7f40d7be0740 1 --2- 172.21.15.116:0/475840059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560f3a8495d0 0x560f3a860340 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x560f3a866cf0 tx=0x7f40c00046c0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.877+0000 7f40d7be0740 1 -- 172.21.15.116:0/475840059 shutdown_connections 2024-09-12T03:08:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.877+0000 7f40d7be0740 1 --2- 172.21.15.116:0/475840059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3a854370 0x560f3a84c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.877+0000 7f40d7be0740 1 --2- 172.21.15.116:0/475840059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560f3a84ab40 0x560f3a84c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.877+0000 7f40d7be0740 1 --2- 172.21.15.116:0/475840059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560f3a8495d0 0x560f3a860340 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.877+0000 7f40d7be0740 1 -- 172.21.15.116:0/475840059 >> 172.21.15.116:0/475840059 conn(0x560f3a812310 msgr2=0x560f3a8500f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.877+0000 7f40d7be0740 1 -- 172.21.15.116:0/475840059 shutdown_connections 2024-09-12T03:08:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.877+0000 7f40d7be0740 1 -- 172.21.15.116:0/475840059 wait complete. 2024-09-12T03:08:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.877+0000 7f40d7be0740 1 -- start start 2024-09-12T03:08:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.878+0000 7f40d7be0740 1 Processor -- start 2024-09-12T03:08:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.878+0000 7f40d7be0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560f3a8495d0 0x560f3a85cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.878+0000 7f40d7be0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3a84ab40 0x560f3a84dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.878+0000 7f40d7be0740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560f3a854370 0x560f3a84e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.878+0000 7f40d7be0740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f40d01aba40 con 0x560f3a854370 2024-09-12T03:08:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.878+0000 7f40d7be0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f40d01aceb0 con 0x560f3a8495d0 2024-09-12T03:08:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.878+0000 7f40d7be0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f40d01ac110 con 0x560f3a84ab40 2024-09-12T03:08:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.878+0000 7f40d7be0740 10 client.0 _ll_register_callbacks cb 0x7f40b8009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.878+0000 7f40cf7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560f3a854370 0x560f3a84e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.878+0000 7f40cf7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560f3a854370 0x560f3a84e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45566/0 (socket says 172.21.15.116:45566) 2024-09-12T03:08:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.878+0000 7f40cf7fe640 1 -- 172.21.15.116:0/1155228951 learned_addr learned my addr 172.21.15.116:0/1155228951 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.878+0000 7f40ceffd640 1 --2- 172.21.15.116:0/1155228951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560f3a8495d0 0x560f3a85cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:02.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.878+0000 7f40ce7fc640 1 --2- 172.21.15.116:0/1155228951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3a84ab40 0x560f3a84dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:02.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.878+0000 7f40cf7fe640 1 -- 172.21.15.116:0/1155228951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3a84ab40 msgr2=0x560f3a84dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:02.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.878+0000 7f40cf7fe640 1 --2- 172.21.15.116:0/1155228951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3a84ab40 0x560f3a84dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.878+0000 7f40cf7fe640 1 -- 172.21.15.116:0/1155228951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560f3a8495d0 msgr2=0x560f3a85cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:02.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.878+0000 7f40cf7fe640 1 --2- 172.21.15.116:0/1155228951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560f3a8495d0 0x560f3a85cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.878+0000 7f40cf7fe640 1 -- 172.21.15.116:0/1155228951 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f3a84e840 con 0x560f3a854370 2024-09-12T03:08:02.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.878+0000 7f40ce7fc640 1 --2- 172.21.15.116:0/1155228951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3a84ab40 0x560f3a84dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:02.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.879+0000 7f40cf7fe640 1 --2- 172.21.15.116:0/1155228951 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560f3a854370 0x560f3a84e150 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f40c802ffb0 tx=0x7f40c802fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63dccff25c3ea0a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:02.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.879+0000 7f40aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:02.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.879+0000 7f40aeffd640 1 -- 172.21.15.116:0/1155228951 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c8034040 con 0x560f3a854370 2024-09-12T03:08:02.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.879+0000 7f40aeffd640 1 -- 172.21.15.116:0/1155228951 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f40c8048920 con 0x560f3a854370 2024-09-12T03:08:02.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.879+0000 7f40d7be0740 1 -- 172.21.15.116:0/1155228951 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40c0010d60 con 0x560f3a854370 2024-09-12T03:08:02.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.879+0000 7f40aeffd640 1 -- 172.21.15.116:0/1155228951 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f40c8060070 con 0x560f3a854370 2024-09-12T03:08:02.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.879+0000 7f40d7be0740 10 client.16524 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:02.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.879+0000 7f40d7be0740 1 -- 172.21.15.116:0/1155228951 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560f3a8ceb50 con 0x560f3a854370 2024-09-12T03:08:02.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.879+0000 7f40aeffd640 1 -- 172.21.15.116:0/1155228951 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40c0010d60 con 0x560f3a854370 2024-09-12T03:08:02.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.879+0000 7f40aeffd640 1 -- 172.21.15.116:0/1155228951 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f40c8004520 con 0x560f3a854370 2024-09-12T03:08:02.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.879+0000 7f40d7be0740 10 client.16524 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:02.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.879+0000 7f40d7be0740 10 client.16524 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:08:02.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.879+0000 7f40d7be0740 20 client.16524 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:02.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.879+0000 7f40d7be0740 1 --2- 172.21.15.116:0/1155228951 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x560f3a8cedd0 0x560f3a8d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:02.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.879+0000 7f40d7be0740 4 client.16524 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:08:02.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.879+0000 7f40d7be0740 1 -- 172.21.15.116:0/1155228951 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560f3a7b6d70 con 0x560f3a8cedd0 2024-09-12T03:08:02.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.880+0000 7f40ceffd640 1 --2- 172.21.15.116:0/1155228951 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x560f3a8cedd0 0x560f3a8d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:02.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.880+0000 7f40ceffd640 1 --2- 172.21.15.116:0/1155228951 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x560f3a8cedd0 0x560f3a8d11c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6acab5ca99a0e1d2 server_cookie=90f5d39d325ae29e in_seq=0 out_seq=0 2024-09-12T03:08:02.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.880+0000 7f40aeffd640 10 client.16524 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:08:02.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.880+0000 7f40aeffd640 1 -- 172.21.15.116:0/1155228951 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560f3a7b6d70 con 0x560f3a8cedd0 2024-09-12T03:08:02.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.880+0000 7f40aeffd640 10 client.16524 handle_command_reply: tid=0 2024-09-12T03:08:02.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.880+0000 7f40d7be0740 1 client.16524 shutdown 2024-09-12T03:08:02.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.880+0000 7f40d7be0740 1 -- 172.21.15.116:0/1155228951 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560f3a854370 msgr2=0x560f3a84e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:02.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.880+0000 7f40d7be0740 1 --2- 172.21.15.116:0/1155228951 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560f3a854370 0x560f3a84e150 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f40c802ffb0 tx=0x7f40c802fbe0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.880+0000 7f40d7be0740 1 -- 172.21.15.116:0/1155228951 shutdown_connections 2024-09-12T03:08:02.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.880+0000 7f40d7be0740 1 --2- 172.21.15.116:0/1155228951 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x560f3a8cedd0 0x560f3a8d11c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.880+0000 7f40d7be0740 1 --2- 172.21.15.116:0/1155228951 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560f3a854370 0x560f3a84e150 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.880+0000 7f40d7be0740 1 --2- 172.21.15.116:0/1155228951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3a84ab40 0x560f3a84dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.880+0000 7f40d7be0740 1 --2- 172.21.15.116:0/1155228951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560f3a8495d0 0x560f3a85cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.880+0000 7f40aeffd640 0 client.16524 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:08:02.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.880+0000 7f40d7be0740 1 -- 172.21.15.116:0/1155228951 >> 172.21.15.116:0/1155228951 conn(0x560f3a812310 msgr2=0x560f3a852860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:02.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.880+0000 7f40d7be0740 1 -- 172.21.15.116:0/1155228951 shutdown_connections 2024-09-12T03:08:02.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.881+0000 7f40d7be0740 1 -- 172.21.15.116:0/1155228951 wait complete. 2024-09-12T03:08:02.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.881+0000 7f40d7be0740 20 client.16524 trim_cache size 0 max 16384 2024-09-12T03:08:02.895 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:08:02.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.881+0000 7f40d5807640 1 -- 172.21.15.116:0/3392708245 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f40bc0a2b80 msgr2=0x7f40bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:02.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.881+0000 7f40d5807640 1 --2- 172.21.15.116:0/3392708245 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f40bc0a2b80 0x7f40bc0a5040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f40c804c000 tx=0x7f40c804a000 comp rx=0 tx=0).stop 2024-09-12T03:08:02.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.881+0000 7f40d5807640 1 -- 172.21.15.116:0/3392708245 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40d0102570 msgr2=0x7f40d01998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:02.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.881+0000 7f40d5807640 1 --2- 172.21.15.116:0/3392708245 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40d0102570 0x7f40d01998f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f40b800e9f0 tx=0x7f40b800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.881+0000 7f40d5807640 1 -- 172.21.15.116:0/3392708245 shutdown_connections 2024-09-12T03:08:02.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.881+0000 7f40d5807640 1 --2- 172.21.15.116:0/3392708245 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f40bc0a2b80 0x7f40bc0a5040 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.881+0000 7f40d5807640 1 --2- 172.21.15.116:0/3392708245 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40d0102ef0 0x7f40d0199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.881+0000 7f40d5807640 1 --2- 172.21.15.116:0/3392708245 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40d0102570 0x7f40d01998f0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.881+0000 7f40d5807640 1 --2- 172.21.15.116:0/3392708245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40d0100e20 0x7f40d01993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:02.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.881+0000 7f40d5807640 1 -- 172.21.15.116:0/3392708245 >> 172.21.15.116:0/3392708245 conn(0x7f40d00fc7d0 msgr2=0x7f40d01068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:02.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.881+0000 7f40d5807640 1 -- 172.21.15.116:0/3392708245 shutdown_connections 2024-09-12T03:08:02.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:02.881+0000 7f40d5807640 1 -- 172.21.15.116:0/3392708245 wait complete. 2024-09-12T03:08:03.720 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:08:03.968 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:08:04.387 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:04 smithi116 ceph-mon[31410]: pgmap v490: 97 pgs: 97 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:08:04.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.466+0000 7fd7a7483640 1 -- 172.21.15.116:0/1598828131 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7a0101420 msgr2=0x7fd7a010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:04.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.466+0000 7fd7a7483640 1 --2- 172.21.15.116:0/1598828131 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7a0101420 0x7fd7a010d950 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd798009cd0 tx=0x7fd79802a8e0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.466+0000 7fd7a7483640 1 -- 172.21.15.116:0/1598828131 shutdown_connections 2024-09-12T03:08:04.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.466+0000 7fd7a7483640 1 --2- 172.21.15.116:0/1598828131 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7a0101420 0x7fd7a010d950 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.466+0000 7fd7a7483640 1 --2- 172.21.15.116:0/1598828131 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7a0100a80 0x7fd7a0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.466+0000 7fd7a7483640 1 --2- 172.21.15.116:0/1598828131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7a0106a80 0x7fd7a0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.466+0000 7fd7a7483640 1 -- 172.21.15.116:0/1598828131 >> 172.21.15.116:0/1598828131 conn(0x7fd7a00fc7a0 msgr2=0x7fd7a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:04.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.466+0000 7fd7a7483640 1 -- 172.21.15.116:0/1598828131 shutdown_connections 2024-09-12T03:08:04.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.466+0000 7fd7a7483640 1 -- 172.21.15.116:0/1598828131 wait complete. 2024-09-12T03:08:04.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.467+0000 7fd7a7483640 1 Processor -- start 2024-09-12T03:08:04.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.467+0000 7fd7a7483640 1 -- start start 2024-09-12T03:08:04.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.467+0000 7fd7a7483640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7a0100a80 0x7fd7a0198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:04.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.467+0000 7fd7a7483640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7a0101420 0x7fd7a0198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:04.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.467+0000 7fd7a7483640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7a0106a80 0x7fd7a019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:04.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.467+0000 7fd7a7483640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a0111fa0 con 0x7fd7a0106a80 2024-09-12T03:08:04.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.467+0000 7fd7a7483640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a0111e20 con 0x7fd7a0101420 2024-09-12T03:08:04.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.467+0000 7fd7a7483640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a0112120 con 0x7fd7a0100a80 2024-09-12T03:08:04.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.467+0000 7fd7a5a20640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7a0106a80 0x7fd7a019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:04.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.467+0000 7fd7a5a20640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7a0106a80 0x7fd7a019ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45576/0 (socket says 172.21.15.116:45576) 2024-09-12T03:08:04.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.467+0000 7fd7a5a20640 1 -- 172.21.15.116:0/3226990543 learned_addr learned my addr 172.21.15.116:0/3226990543 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:04.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.467+0000 7fd7a521f640 1 --2- 172.21.15.116:0/3226990543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7a0100a80 0x7fd7a0198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:04.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.467+0000 7fd7a4a1e640 1 --2- 172.21.15.116:0/3226990543 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7a0101420 0x7fd7a0198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:04.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.467+0000 7fd7a5a20640 1 -- 172.21.15.116:0/3226990543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7a0100a80 msgr2=0x7fd7a0198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:04.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.467+0000 7fd7a5a20640 1 --2- 172.21.15.116:0/3226990543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7a0100a80 0x7fd7a0198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.467+0000 7fd7a5a20640 1 -- 172.21.15.116:0/3226990543 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7a0101420 msgr2=0x7fd7a0198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:04.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.467+0000 7fd7a5a20640 1 --2- 172.21.15.116:0/3226990543 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7a0101420 0x7fd7a0198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.467+0000 7fd7a5a20640 1 -- 172.21.15.116:0/3226990543 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a019d3b0 con 0x7fd7a0106a80 2024-09-12T03:08:04.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.468+0000 7fd7a5a20640 1 --2- 172.21.15.116:0/3226990543 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7a0106a80 0x7fd7a019ccd0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd798005cc0 tx=0x7fd798003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:04.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.468+0000 7fd7967fc640 1 -- 172.21.15.116:0/3226990543 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd798004040 con 0x7fd7a0106a80 2024-09-12T03:08:04.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.468+0000 7fd7967fc640 1 -- 172.21.15.116:0/3226990543 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7980046c0 con 0x7fd7a0106a80 2024-09-12T03:08:04.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.468+0000 7fd7a7483640 1 -- 172.21.15.116:0/3226990543 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7a019d640 con 0x7fd7a0106a80 2024-09-12T03:08:04.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.468+0000 7fd7967fc640 1 -- 172.21.15.116:0/3226990543 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd798046480 con 0x7fd7a0106a80 2024-09-12T03:08:04.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.468+0000 7fd7a7483640 1 -- 172.21.15.116:0/3226990543 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7a01a4f80 con 0x7fd7a0106a80 2024-09-12T03:08:04.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.471+0000 7fd7967fc640 1 -- 172.21.15.116:0/3226990543 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd7980305a0 con 0x7fd7a0106a80 2024-09-12T03:08:04.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.471+0000 7fd7a985c740 1 Processor -- start 2024-09-12T03:08:04.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.471+0000 7fd7a985c740 1 -- start start 2024-09-12T03:08:04.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.471+0000 7fd7a985c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d03022eb40 0x55d03022d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:04.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.471+0000 7fd7a985c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d03022d5d0 0x55d03022da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:04.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.471+0000 7fd7a985c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d030238370 0x55d03023a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:04.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.471+0000 7fd7a985c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d0301932f0 con 0x55d03022d5d0 2024-09-12T03:08:04.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.471+0000 7fd7a985c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d03013acf0 con 0x55d03022eb40 2024-09-12T03:08:04.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.471+0000 7fd7a985c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d03018f2b0 con 0x55d030238370 2024-09-12T03:08:04.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.471+0000 7fd7a521f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d03022d5d0 0x55d03022da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:04.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.472+0000 7fd7a521f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d03022d5d0 0x55d03022da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45592/0 (socket says 172.21.15.116:45592) 2024-09-12T03:08:04.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.472+0000 7fd7a521f640 1 -- 172.21.15.116:0/2820987121 learned_addr learned my addr 172.21.15.116:0/2820987121 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:04.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.472+0000 7fd7967fc640 1 --2- 172.21.15.116:0/3226990543 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd78c0a2c00 0x7fd78c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:04.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.472+0000 7fd7a4a1e640 1 --2- 172.21.15.116:0/2820987121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d03022eb40 0x55d03022d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:04.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.472+0000 7fd7a4a1e640 1 --2- 172.21.15.116:0/2820987121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d030238370 0x55d03023a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:04.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.472+0000 7fd7a5a20640 1 --2- 172.21.15.116:0/3226990543 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd78c0a2c00 0x7fd78c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:04.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.472+0000 7fd7a521f640 1 -- 172.21.15.116:0/2820987121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d030238370 msgr2=0x55d03023a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:04.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.472+0000 7fd7a521f640 1 --2- 172.21.15.116:0/2820987121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d030238370 0x55d03023a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.472+0000 7fd7a521f640 1 -- 172.21.15.116:0/2820987121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d03022eb40 msgr2=0x55d03022d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:04.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.472+0000 7fd7a521f640 1 --2- 172.21.15.116:0/2820987121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d03022eb40 0x55d03022d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.472+0000 7fd7a521f640 1 -- 172.21.15.116:0/2820987121 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d03011f3a0 con 0x55d03022d5d0 2024-09-12T03:08:04.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.472+0000 7fd7a4a1e640 1 --2- 172.21.15.116:0/2820987121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d03022eb40 0x55d03022d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:04.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.472+0000 7fd7967fc640 1 -- 172.21.15.116:0/3226990543 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd79802e030 con 0x7fd7a0106a80 2024-09-12T03:08:04.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.472+0000 7fd7a521f640 1 --2- 172.21.15.116:0/2820987121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d03022d5d0 0x55d03022da10 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00aba0 tx=0x7fd79c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3b434a7093f711 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:04.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.472+0000 7fd7a5a20640 1 --2- 172.21.15.116:0/3226990543 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd78c0a2c00 0x7fd78c0a50c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd7980b7b20 tx=0x7fd7980dd040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:04.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.472+0000 7fd78b7fe640 1 -- 172.21.15.116:0/2820987121 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c01d070 con 0x55d03022d5d0 2024-09-12T03:08:04.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.472+0000 7fd78b7fe640 1 -- 172.21.15.116:0/2820987121 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd79c018b20 con 0x55d03022d5d0 2024-09-12T03:08:04.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.472+0000 7fd78b7fe640 1 -- 172.21.15.116:0/2820987121 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c010b50 con 0x55d03022d5d0 2024-09-12T03:08:04.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.473+0000 7fd7a985c740 1 -- 172.21.15.116:0/2820987121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d03022d5d0 msgr2=0x55d03022da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:04.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.473+0000 7fd7a985c740 1 --2- 172.21.15.116:0/2820987121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d03022d5d0 0x55d03022da10 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00aba0 tx=0x7fd79c00f790 comp rx=0 tx=0).stop 2024-09-12T03:08:04.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.473+0000 7fd7a985c740 1 -- 172.21.15.116:0/2820987121 shutdown_connections 2024-09-12T03:08:04.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.473+0000 7fd7a985c740 1 --2- 172.21.15.116:0/2820987121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d030238370 0x55d03023a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.473+0000 7fd7a985c740 1 --2- 172.21.15.116:0/2820987121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d03022d5d0 0x55d03022da10 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.473+0000 7fd7a985c740 1 --2- 172.21.15.116:0/2820987121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d03022eb40 0x55d03022d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.473+0000 7fd7a985c740 1 -- 172.21.15.116:0/2820987121 >> 172.21.15.116:0/2820987121 conn(0x55d0301f6310 msgr2=0x55d030229c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:04.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.473+0000 7fd7a985c740 1 -- 172.21.15.116:0/2820987121 shutdown_connections 2024-09-12T03:08:04.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.473+0000 7fd7a985c740 1 -- 172.21.15.116:0/2820987121 wait complete. 2024-09-12T03:08:04.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.473+0000 7fd7a985c740 1 -- start start 2024-09-12T03:08:04.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.473+0000 7fd7a985c740 1 Processor -- start 2024-09-12T03:08:04.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.474+0000 7fd7a985c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d03022d5d0 0x55d030237d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:04.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.474+0000 7fd7a985c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03022eb40 0x55d030235030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:04.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.474+0000 7fd7a985c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d030238370 0x55d030235640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:04.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.474+0000 7fd7a985c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d03023b070 con 0x55d03022d5d0 2024-09-12T03:08:04.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.474+0000 7fd7a985c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d03014e060 con 0x55d030238370 2024-09-12T03:08:04.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.474+0000 7fd7a985c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d03023b1f0 con 0x55d03022eb40 2024-09-12T03:08:04.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.474+0000 7fd7a985c740 10 client.0 _ll_register_callbacks cb 0x55d02fea18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:04.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.474+0000 7fd7a4a1e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d03022d5d0 0x55d030237d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:04.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.474+0000 7fd7a4a1e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d03022d5d0 0x55d030237d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45600/0 (socket says 172.21.15.116:45600) 2024-09-12T03:08:04.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.474+0000 7fd7a4a1e640 1 -- 172.21.15.116:0/1917208281 learned_addr learned my addr 172.21.15.116:0/1917208281 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:04.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.474+0000 7fd7a521f640 1 --2- 172.21.15.116:0/1917208281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03022eb40 0x55d030235030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:04.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.474+0000 7fd7a4a1e640 1 --2- 172.21.15.116:0/1917208281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d030238370 0x55d030235640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:04.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.474+0000 7fd7a4a1e640 1 -- 172.21.15.116:0/1917208281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03022eb40 msgr2=0x55d030235030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:04.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.474+0000 7fd7a4a1e640 1 --2- 172.21.15.116:0/1917208281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03022eb40 0x55d030235030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.474+0000 7fd7a4a1e640 1 -- 172.21.15.116:0/1917208281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d030238370 msgr2=0x55d030235640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:04.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.474+0000 7fd7a4a1e640 1 --2- 172.21.15.116:0/1917208281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d030238370 0x55d030235640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.474+0000 7fd7a4a1e640 1 -- 172.21.15.116:0/1917208281 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d02ff043c0 con 0x55d03022d5d0 2024-09-12T03:08:04.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.474+0000 7fd7a521f640 1 --2- 172.21.15.116:0/1917208281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03022eb40 0x55d030235030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:08:04.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.475+0000 7fd7a4a1e640 1 --2- 172.21.15.116:0/1917208281 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d03022d5d0 0x55d030237d80 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fd790013d80 tx=0x7fd790015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418f4e36c873c919 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:04.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.475+0000 7fd788ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:04.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.475+0000 7fd788ff9640 1 -- 172.21.15.116:0/1917208281 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79000ed70 con 0x55d03022d5d0 2024-09-12T03:08:04.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.475+0000 7fd788ff9640 1 -- 172.21.15.116:0/1917208281 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd790005020 con 0x55d03022d5d0 2024-09-12T03:08:04.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.475+0000 7fd7a985c740 1 -- 172.21.15.116:0/1917208281 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd79c018b20 con 0x55d03022d5d0 2024-09-12T03:08:04.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.475+0000 7fd788ff9640 1 -- 172.21.15.116:0/1917208281 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd790005220 con 0x55d03022d5d0 2024-09-12T03:08:04.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.475+0000 7fd7a985c740 10 client.16536 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:04.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.475+0000 7fd7a985c740 1 -- 172.21.15.116:0/1917208281 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d030246ba0 con 0x55d03022d5d0 2024-09-12T03:08:04.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.475+0000 7fd788ff9640 1 -- 172.21.15.116:0/1917208281 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd79c018b20 con 0x55d03022d5d0 2024-09-12T03:08:04.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.475+0000 7fd788ff9640 1 -- 172.21.15.116:0/1917208281 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd79000f040 con 0x55d03022d5d0 2024-09-12T03:08:04.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.475+0000 7fd7a985c740 10 client.16536 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:04.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.475+0000 7fd7a985c740 10 client.16536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:08:04.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.475+0000 7fd7a985c740 20 client.16536 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:04.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.475+0000 7fd7a985c740 1 --2- 172.21.15.116:0/1917208281 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d030247fb0 0x55d03024a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:04.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.475+0000 7fd7a985c740 4 client.16536 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:08:04.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.475+0000 7fd7a985c740 1 -- 172.21.15.116:0/1917208281 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0301a0860 con 0x55d030247fb0 2024-09-12T03:08:04.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.476+0000 7fd7a521f640 1 --2- 172.21.15.116:0/1917208281 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d030247fb0 0x55d03024a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:04.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.476+0000 7fd7a521f640 1 --2- 172.21.15.116:0/1917208281 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d030247fb0 0x55d03024a3a0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a34b1b8ea312f6 server_cookie=80eed5a37c5ebb8f in_seq=0 out_seq=0 2024-09-12T03:08:04.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.476+0000 7fd788ff9640 10 client.16536 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:04.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.477+0000 7fd788ff9640 1 -- 172.21.15.116:0/1917208281 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d0301a0860 con 0x55d030247fb0 2024-09-12T03:08:04.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.477+0000 7fd788ff9640 10 client.16536 handle_command_reply: tid=0 2024-09-12T03:08:04.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.477+0000 7fd7a985c740 1 client.16536 shutdown 2024-09-12T03:08:04.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.477+0000 7fd7a985c740 1 -- 172.21.15.116:0/1917208281 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d03022d5d0 msgr2=0x55d030237d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:04.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.477+0000 7fd7a985c740 1 --2- 172.21.15.116:0/1917208281 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d03022d5d0 0x55d030237d80 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fd790013d80 tx=0x7fd790015040 comp rx=0 tx=0).stop 2024-09-12T03:08:04.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.477+0000 7fd7a985c740 1 -- 172.21.15.116:0/1917208281 shutdown_connections 2024-09-12T03:08:04.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.477+0000 7fd7a985c740 1 --2- 172.21.15.116:0/1917208281 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d030247fb0 0x55d03024a3a0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.477+0000 7fd7a985c740 1 --2- 172.21.15.116:0/1917208281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d030238370 0x55d030235640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.477+0000 7fd7a985c740 1 --2- 172.21.15.116:0/1917208281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03022eb40 0x55d030235030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.477+0000 7fd7a985c740 1 --2- 172.21.15.116:0/1917208281 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d03022d5d0 0x55d030237d80 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.477+0000 7fd7a985c740 1 -- 172.21.15.116:0/1917208281 >> 172.21.15.116:0/1917208281 conn(0x55d0301f6310 msgr2=0x55d03022f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:04.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.477+0000 7fd788ff9640 0 client.16536 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:04.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.477+0000 7fd7a985c740 1 -- 172.21.15.116:0/1917208281 shutdown_connections 2024-09-12T03:08:04.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.477+0000 7fd7a985c740 1 -- 172.21.15.116:0/1917208281 wait complete. 2024-09-12T03:08:04.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.477+0000 7fd7a985c740 20 client.16536 trim_cache size 0 max 16384 2024-09-12T03:08:04.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.490+0000 7fd7a985c740 1 Processor -- start 2024-09-12T03:08:04.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.490+0000 7fd7a985c740 1 -- start start 2024-09-12T03:08:04.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.490+0000 7fd7a985c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d03022d5d0 0x55d030233f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:04.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.490+0000 7fd7a985c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d03022eb40 0x55d030228c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:04.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.490+0000 7fd7a985c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d030238370 0x55d030240860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:04.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.490+0000 7fd7a985c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a01afd30 con 0x55d03022d5d0 2024-09-12T03:08:04.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.490+0000 7fd7a985c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a01ad600 con 0x55d03022eb40 2024-09-12T03:08:04.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.490+0000 7fd7a985c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a01ab450 con 0x55d030238370 2024-09-12T03:08:04.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.490+0000 7fd7a4a1e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d03022d5d0 0x55d030233f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:04.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.490+0000 7fd7a4a1e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d03022d5d0 0x55d030233f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45602/0 (socket says 172.21.15.116:45602) 2024-09-12T03:08:04.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.490+0000 7fd7a4a1e640 1 -- 172.21.15.116:0/1268633583 learned_addr learned my addr 172.21.15.116:0/1268633583 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:04.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.491+0000 7fd7a4a1e640 1 -- 172.21.15.116:0/1268633583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d030238370 msgr2=0x55d030240860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:04.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.491+0000 7fd7a521f640 1 --2- 172.21.15.116:0/1268633583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d03022eb40 0x55d030228c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:04.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.491+0000 7fd7a4a1e640 1 --2- 172.21.15.116:0/1268633583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d030238370 0x55d030240860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.491+0000 7fd7a4a1e640 1 -- 172.21.15.116:0/1268633583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d03022eb40 msgr2=0x55d030228c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:04.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.491+0000 7fd7a4a1e640 1 --2- 172.21.15.116:0/1268633583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d03022eb40 0x55d030228c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.491+0000 7fd7a4a1e640 1 -- 172.21.15.116:0/1268633583 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d030233580 con 0x55d03022d5d0 2024-09-12T03:08:04.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.491+0000 7fd7a521f640 1 --2- 172.21.15.116:0/1268633583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d03022eb40 0x55d030228c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:08:04.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.491+0000 7fd7a4a1e640 1 --2- 172.21.15.116:0/1268633583 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d03022d5d0 0x55d030233f70 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd79000e3d0 tx=0x7fd790005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523f15040dfe7a1e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:04.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.491+0000 7fd78affd640 1 -- 172.21.15.116:0/1268633583 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79000ed70 con 0x55d03022d5d0 2024-09-12T03:08:04.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.491+0000 7fd78affd640 1 -- 172.21.15.116:0/1268633583 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7900053c0 con 0x55d03022d5d0 2024-09-12T03:08:04.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.491+0000 7fd78affd640 1 -- 172.21.15.116:0/1268633583 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79002a6b0 con 0x55d03022d5d0 2024-09-12T03:08:04.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.492+0000 7fd7a985c740 1 -- 172.21.15.116:0/1268633583 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d03022d5d0 msgr2=0x55d030233f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:04.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.492+0000 7fd7a985c740 1 --2- 172.21.15.116:0/1268633583 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d03022d5d0 0x55d030233f70 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd79000e3d0 tx=0x7fd790005850 comp rx=0 tx=0).stop 2024-09-12T03:08:04.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.492+0000 7fd7a985c740 1 -- 172.21.15.116:0/1268633583 shutdown_connections 2024-09-12T03:08:04.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.492+0000 7fd7a985c740 1 --2- 172.21.15.116:0/1268633583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d030238370 0x55d030240860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.492+0000 7fd7a985c740 1 --2- 172.21.15.116:0/1268633583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d03022eb40 0x55d030228c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.492+0000 7fd7a985c740 1 --2- 172.21.15.116:0/1268633583 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d03022d5d0 0x55d030233f70 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.492+0000 7fd7a985c740 1 -- 172.21.15.116:0/1268633583 >> 172.21.15.116:0/1268633583 conn(0x55d0301f6310 msgr2=0x55d030228500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:04.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.492+0000 7fd7a985c740 1 -- 172.21.15.116:0/1268633583 shutdown_connections 2024-09-12T03:08:04.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.492+0000 7fd7a985c740 1 -- 172.21.15.116:0/1268633583 wait complete. 2024-09-12T03:08:04.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.492+0000 7fd7a985c740 1 -- start start 2024-09-12T03:08:04.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.492+0000 7fd7a985c740 1 Processor -- start 2024-09-12T03:08:04.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.492+0000 7fd7a985c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d03022d5d0 0x55d030232b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:04.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.492+0000 7fd7a985c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03022eb40 0x55d03024f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:04.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.493+0000 7fd7a985c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d030238370 0x55d03024fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:04.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.493+0000 7fd7a985c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a01abac0 con 0x55d03022d5d0 2024-09-12T03:08:04.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.493+0000 7fd7a985c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a01acf30 con 0x55d030238370 2024-09-12T03:08:04.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.493+0000 7fd7a985c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a01ac190 con 0x55d03022eb40 2024-09-12T03:08:04.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.493+0000 7fd7a985c740 10 client.0 _ll_register_callbacks cb 0x7fd79000e870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:04.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.493+0000 7fd7a4a1e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d03022d5d0 0x55d030232b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:04.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.493+0000 7fd7a4a1e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d03022d5d0 0x55d030232b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45618/0 (socket says 172.21.15.116:45618) 2024-09-12T03:08:04.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.493+0000 7fd7a4a1e640 1 -- 172.21.15.116:0/3643254326 learned_addr learned my addr 172.21.15.116:0/3643254326 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:04.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.493+0000 7fd7a521f640 1 --2- 172.21.15.116:0/3643254326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03022eb40 0x55d03024f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:04.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.493+0000 7fd7a4a1e640 1 -- 172.21.15.116:0/3643254326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03022eb40 msgr2=0x55d03024f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:04.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.493+0000 7fd7a4a1e640 1 --2- 172.21.15.116:0/3643254326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03022eb40 0x55d03024f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.493+0000 7fd7a4a1e640 1 -- 172.21.15.116:0/3643254326 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d030238370 msgr2=0x55d03024fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:04.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.493+0000 7fd7a4a1e640 1 --2- 172.21.15.116:0/3643254326 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d030238370 0x55d03024fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.493+0000 7fd7a4a1e640 1 -- 172.21.15.116:0/3643254326 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0302503f0 con 0x55d03022d5d0 2024-09-12T03:08:04.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.493+0000 7fd7a521f640 1 --2- 172.21.15.116:0/3643254326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03022eb40 0x55d03024f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:04.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.493+0000 7fd7a4a1e640 1 --2- 172.21.15.116:0/3643254326 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d03022d5d0 0x55d030232b40 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd7900046a0 tx=0x7fd7900139b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80573f01be79d41 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:04.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.493+0000 7fd788ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:04.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.493+0000 7fd788ff9640 1 -- 172.21.15.116:0/3643254326 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd790005630 con 0x55d03022d5d0 2024-09-12T03:08:04.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.493+0000 7fd788ff9640 1 -- 172.21.15.116:0/3643254326 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd790038070 con 0x55d03022d5d0 2024-09-12T03:08:04.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.493+0000 7fd7a985c740 1 -- 172.21.15.116:0/3643254326 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7900053c0 con 0x55d03022d5d0 2024-09-12T03:08:04.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.493+0000 7fd788ff9640 1 -- 172.21.15.116:0/3643254326 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd790043070 con 0x55d03022d5d0 2024-09-12T03:08:04.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.493+0000 7fd7a985c740 10 client.16542 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:04.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.493+0000 7fd7a985c740 1 -- 172.21.15.116:0/3643254326 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d0302b6b60 con 0x55d03022d5d0 2024-09-12T03:08:04.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.494+0000 7fd788ff9640 1 -- 172.21.15.116:0/3643254326 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7900053c0 con 0x55d03022d5d0 2024-09-12T03:08:04.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.494+0000 7fd788ff9640 1 -- 172.21.15.116:0/3643254326 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd79001f840 con 0x55d03022d5d0 2024-09-12T03:08:04.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.494+0000 7fd7a985c740 10 client.16542 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:04.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.494+0000 7fd7a985c740 10 client.16542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:08:04.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.494+0000 7fd7a985c740 20 client.16542 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:04.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.494+0000 7fd7a985c740 1 --2- 172.21.15.116:0/3643254326 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d0302b71c0 0x55d0302b95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:04.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.494+0000 7fd7a985c740 4 client.16542 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:08:04.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.494+0000 7fd7a985c740 1 -- 172.21.15.116:0/3643254326 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d03019ad70 con 0x55d0302b71c0 2024-09-12T03:08:04.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.494+0000 7fd7a521f640 1 --2- 172.21.15.116:0/3643254326 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d0302b71c0 0x55d0302b95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:04.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.495+0000 7fd788ff9640 10 client.16542 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:04.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.495+0000 7fd7a521f640 1 --2- 172.21.15.116:0/3643254326 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d0302b71c0 0x55d0302b95b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c06c582a10725002 server_cookie=7dcc725a7b9f26f1 in_seq=0 out_seq=0 2024-09-12T03:08:04.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.495+0000 7fd788ff9640 1 -- 172.21.15.116:0/3643254326 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d03019ad70 con 0x55d0302b71c0 2024-09-12T03:08:04.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.495+0000 7fd788ff9640 10 client.16542 handle_command_reply: tid=0 2024-09-12T03:08:04.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.495+0000 7fd7a985c740 1 client.16542 shutdown 2024-09-12T03:08:04.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.495+0000 7fd7a985c740 1 -- 172.21.15.116:0/3643254326 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d03022d5d0 msgr2=0x55d030232b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:04.528 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:08:04.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.495+0000 7fd7a985c740 1 --2- 172.21.15.116:0/3643254326 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d03022d5d0 0x55d030232b40 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd7900046a0 tx=0x7fd7900139b0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.495+0000 7fd7a985c740 1 -- 172.21.15.116:0/3643254326 shutdown_connections 2024-09-12T03:08:04.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.495+0000 7fd7a985c740 1 --2- 172.21.15.116:0/3643254326 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d0302b71c0 0x55d0302b95b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.495+0000 7fd7a985c740 1 --2- 172.21.15.116:0/3643254326 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d030238370 0x55d03024fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.495+0000 7fd788ff9640 0 client.16542 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:04.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.495+0000 7fd7a985c740 1 --2- 172.21.15.116:0/3643254326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03022eb40 0x55d03024f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.495+0000 7fd7a985c740 1 --2- 172.21.15.116:0/3643254326 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d03022d5d0 0x55d030232b40 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.495+0000 7fd7a985c740 1 -- 172.21.15.116:0/3643254326 >> 172.21.15.116:0/3643254326 conn(0x55d0301f6310 msgr2=0x55d030228d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:04.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.495+0000 7fd7a985c740 1 -- 172.21.15.116:0/3643254326 shutdown_connections 2024-09-12T03:08:04.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.495+0000 7fd7a985c740 1 -- 172.21.15.116:0/3643254326 wait complete. 2024-09-12T03:08:04.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.495+0000 7fd7a985c740 20 client.16542 trim_cache size 0 max 16384 2024-09-12T03:08:04.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.496+0000 7fd7a7483640 1 -- 172.21.15.116:0/3226990543 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd78c0a2c00 msgr2=0x7fd78c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:04.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.496+0000 7fd7a7483640 1 --2- 172.21.15.116:0/3226990543 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd78c0a2c00 0x7fd78c0a50c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd7980b7b20 tx=0x7fd7980dd040 comp rx=0 tx=0).stop 2024-09-12T03:08:04.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.496+0000 7fd7a7483640 1 -- 172.21.15.116:0/3226990543 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7a0106a80 msgr2=0x7fd7a019ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:04.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.496+0000 7fd7a7483640 1 --2- 172.21.15.116:0/3226990543 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7a0106a80 0x7fd7a019ccd0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd798005cc0 tx=0x7fd798003970 comp rx=0 tx=0).stop 2024-09-12T03:08:04.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.496+0000 7fd7a7483640 1 -- 172.21.15.116:0/3226990543 shutdown_connections 2024-09-12T03:08:04.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.496+0000 7fd7a7483640 1 --2- 172.21.15.116:0/3226990543 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd78c0a2c00 0x7fd78c0a50c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.496+0000 7fd7a7483640 1 --2- 172.21.15.116:0/3226990543 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7a0106a80 0x7fd7a019ccd0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.496+0000 7fd7a7483640 1 --2- 172.21.15.116:0/3226990543 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7a0101420 0x7fd7a0198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.496+0000 7fd7a7483640 1 --2- 172.21.15.116:0/3226990543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7a0100a80 0x7fd7a0198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:04.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.496+0000 7fd7a7483640 1 -- 172.21.15.116:0/3226990543 >> 172.21.15.116:0/3226990543 conn(0x7fd7a00fc7a0 msgr2=0x7fd7a00fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:04.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.496+0000 7fd7a7483640 1 -- 172.21.15.116:0/3226990543 shutdown_connections 2024-09-12T03:08:04.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:04.496+0000 7fd7a7483640 1 -- 172.21.15.116:0/3226990543 wait complete. 2024-09-12T03:08:04.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:04 smithi183 ceph-mon[35535]: pgmap v490: 97 pgs: 97 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:08:04.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:04 smithi177 ceph-mon[35305]: pgmap v490: 97 pgs: 97 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:08:06.357 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:08:06.357 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:08:06.615 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:08:06.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:06 smithi183 ceph-mon[35535]: pgmap v491: 97 pgs: 97 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:08:06.736 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:06 smithi116 ceph-mon[31410]: pgmap v491: 97 pgs: 97 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:08:06.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:06 smithi177 ceph-mon[35305]: pgmap v491: 97 pgs: 97 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:08:07.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.105+0000 7f5a4565f640 1 -- 172.21.15.116:0/3887372912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a40106f20 msgr2=0x7f5a4010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:07.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.105+0000 7f5a4565f640 1 --2- 172.21.15.116:0/3887372912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a40106f20 0x7f5a4010eaa0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009ca0 tx=0x7f5a3802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.105+0000 7f5a4565f640 1 -- 172.21.15.116:0/3887372912 shutdown_connections 2024-09-12T03:08:07.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.105+0000 7f5a4565f640 1 --2- 172.21.15.116:0/3887372912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a40106f20 0x7f5a4010eaa0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.105+0000 7f5a4565f640 1 --2- 172.21.15.116:0/3887372912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a401027a0 0x7f5a401067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.105+0000 7f5a4565f640 1 --2- 172.21.15.116:0/3887372912 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a40101df0 0x7f5a401021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.105+0000 7f5a4565f640 1 -- 172.21.15.116:0/3887372912 >> 172.21.15.116:0/3887372912 conn(0x7f5a400fd8b0 msgr2=0x7f5a400ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:07.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.105+0000 7f5a4565f640 1 -- 172.21.15.116:0/3887372912 shutdown_connections 2024-09-12T03:08:07.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.105+0000 7f5a4565f640 1 -- 172.21.15.116:0/3887372912 wait complete. 2024-09-12T03:08:07.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.105+0000 7f5a4565f640 1 Processor -- start 2024-09-12T03:08:07.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.105+0000 7f5a4565f640 1 -- start start 2024-09-12T03:08:07.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.106+0000 7f5a4565f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a40101df0 0x7f5a40198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:07.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.106+0000 7f5a4565f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a401027a0 0x7f5a40198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:07.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.106+0000 7f5a4565f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a40106f20 0x7f5a4019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:07.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.106+0000 7f5a4565f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5a40111f80 con 0x7f5a401027a0 2024-09-12T03:08:07.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.106+0000 7f5a4565f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5a40111e00 con 0x7f5a40106f20 2024-09-12T03:08:07.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.106+0000 7f5a4565f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5a40112100 con 0x7f5a40101df0 2024-09-12T03:08:07.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.106+0000 7f5a3e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a401027a0 0x7f5a40198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:07.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.106+0000 7f5a3e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a401027a0 0x7f5a40198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45632/0 (socket says 172.21.15.116:45632) 2024-09-12T03:08:07.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.106+0000 7f5a3e7fc640 1 -- 172.21.15.116:0/1281490857 learned_addr learned my addr 172.21.15.116:0/1281490857 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:07.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.106+0000 7f5a3effd640 1 --2- 172.21.15.116:0/1281490857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a40101df0 0x7f5a40198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:07.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.106+0000 7f5a3f7fe640 1 --2- 172.21.15.116:0/1281490857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a40106f20 0x7f5a4019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:07.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.106+0000 7f5a3e7fc640 1 -- 172.21.15.116:0/1281490857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a40101df0 msgr2=0x7f5a40198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:07.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.106+0000 7f5a3e7fc640 1 --2- 172.21.15.116:0/1281490857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a40101df0 0x7f5a40198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.106+0000 7f5a3e7fc640 1 -- 172.21.15.116:0/1281490857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a40106f20 msgr2=0x7f5a4019cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:07.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.106+0000 7f5a3e7fc640 1 --2- 172.21.15.116:0/1281490857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a40106f20 0x7f5a4019cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.106+0000 7f5a3e7fc640 1 -- 172.21.15.116:0/1281490857 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4019d3f0 con 0x7f5a401027a0 2024-09-12T03:08:07.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.106+0000 7f5a3e7fc640 1 --2- 172.21.15.116:0/1281490857 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a401027a0 0x7f5a40198980 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5a3400d770 tx=0x7f5a3400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:07.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.106+0000 7f5a27fff640 1 -- 172.21.15.116:0/1281490857 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34017070 con 0x7f5a401027a0 2024-09-12T03:08:07.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.106+0000 7f5a4565f640 1 -- 172.21.15.116:0/1281490857 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4019d6e0 con 0x7f5a401027a0 2024-09-12T03:08:07.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.106+0000 7f5a27fff640 1 -- 172.21.15.116:0/1281490857 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a34005500 con 0x7f5a401027a0 2024-09-12T03:08:07.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.106+0000 7f5a27fff640 1 -- 172.21.15.116:0/1281490857 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34004e00 con 0x7f5a401027a0 2024-09-12T03:08:07.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.107+0000 7f5a4565f640 1 -- 172.21.15.116:0/1281490857 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a401a4fc0 con 0x7f5a401027a0 2024-09-12T03:08:07.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.109+0000 7f5a27fff640 1 -- 172.21.15.116:0/1281490857 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f5a34004fa0 con 0x7f5a401027a0 2024-09-12T03:08:07.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.109+0000 7f5a47a38740 1 Processor -- start 2024-09-12T03:08:07.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.109+0000 7f5a47a38740 1 -- start start 2024-09-12T03:08:07.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.109+0000 7f5a47a38740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55962c1e9b60 0x55962c1e9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:07.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.109+0000 7f5a47a38740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55962c1ea480 0x55962c1f4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:07.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.109+0000 7f5a47a38740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55962c1f4d40 0x55962c1f7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:07.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.109+0000 7f5a47a38740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55962c14f280 con 0x55962c1e9b60 2024-09-12T03:08:07.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.109+0000 7f5a47a38740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55962c0f6b10 con 0x55962c1ea480 2024-09-12T03:08:07.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.109+0000 7f5a47a38740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55962c14b240 con 0x55962c1f4d40 2024-09-12T03:08:07.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.109+0000 7f5a27fff640 1 --2- 172.21.15.116:0/1281490857 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5a280a2b80 0x7f5a280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:07.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.109+0000 7f5a3f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55962c1e9b60 0x55962c1e9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:07.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.109+0000 7f5a3f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55962c1e9b60 0x55962c1e9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45644/0 (socket says 172.21.15.116:45644) 2024-09-12T03:08:07.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.109+0000 7f5a3f7fe640 1 -- 172.21.15.116:0/3285973826 learned_addr learned my addr 172.21.15.116:0/3285973826 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:07.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.110+0000 7f5a27fff640 1 -- 172.21.15.116:0/1281490857 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f5a340ce980 con 0x7f5a401027a0 2024-09-12T03:08:07.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.110+0000 7f5a3f7fe640 1 --2- 172.21.15.116:0/1281490857 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5a280a2b80 0x7f5a280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:07.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.110+0000 7f5a3effd640 1 --2- 172.21.15.116:0/3285973826 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55962c1ea480 0x55962c1f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:07.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.110+0000 7f5a3e7fc640 1 --2- 172.21.15.116:0/3285973826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55962c1f4d40 0x55962c1f7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:07.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.110+0000 7f5a3f7fe640 1 -- 172.21.15.116:0/3285973826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55962c1f4d40 msgr2=0x55962c1f7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:07.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.110+0000 7f5a3f7fe640 1 --2- 172.21.15.116:0/3285973826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55962c1f4d40 0x55962c1f7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.110+0000 7f5a3f7fe640 1 -- 172.21.15.116:0/3285973826 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55962c1ea480 msgr2=0x55962c1f4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:07.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.110+0000 7f5a3e7fc640 1 --2- 172.21.15.116:0/3285973826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55962c1f4d40 0x55962c1f7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:08:07.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.110+0000 7f5a3f7fe640 1 --2- 172.21.15.116:0/3285973826 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55962c1ea480 0x55962c1f4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.110+0000 7f5a3f7fe640 1 -- 172.21.15.116:0/3285973826 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55962c1031f0 con 0x55962c1e9b60 2024-09-12T03:08:07.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.110+0000 7f5a3effd640 1 --2- 172.21.15.116:0/3285973826 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55962c1ea480 0x55962c1f4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:07.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.110+0000 7f5a3f7fe640 1 --2- 172.21.15.116:0/3285973826 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55962c1e9b60 0x55962c1e9f40 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009430 tx=0x7f5a380039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f8963e22bdc3cf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:07.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.110+0000 7f5a257fa640 1 -- 172.21.15.116:0/3285973826 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a380319a0 con 0x55962c1e9b60 2024-09-12T03:08:07.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.110+0000 7f5a257fa640 1 -- 172.21.15.116:0/3285973826 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a3804f070 con 0x55962c1e9b60 2024-09-12T03:08:07.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.110+0000 7f5a257fa640 1 -- 172.21.15.116:0/3285973826 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3802fbf0 con 0x55962c1e9b60 2024-09-12T03:08:07.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.110+0000 7f5a3f7fe640 1 --2- 172.21.15.116:0/1281490857 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5a280a2b80 0x7f5a280a5040 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5a38004af0 tx=0x7f5a3804c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:07.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.111+0000 7f5a47a38740 1 -- 172.21.15.116:0/3285973826 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55962c1e9b60 msgr2=0x55962c1e9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:07.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.111+0000 7f5a47a38740 1 --2- 172.21.15.116:0/3285973826 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55962c1e9b60 0x55962c1e9f40 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009430 tx=0x7f5a380039f0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.111+0000 7f5a47a38740 1 -- 172.21.15.116:0/3285973826 shutdown_connections 2024-09-12T03:08:07.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.111+0000 7f5a47a38740 1 --2- 172.21.15.116:0/3285973826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55962c1f4d40 0x55962c1f7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.111+0000 7f5a47a38740 1 --2- 172.21.15.116:0/3285973826 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55962c1ea480 0x55962c1f4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.111+0000 7f5a47a38740 1 --2- 172.21.15.116:0/3285973826 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55962c1e9b60 0x55962c1e9f40 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.111+0000 7f5a47a38740 1 -- 172.21.15.116:0/3285973826 >> 172.21.15.116:0/3285973826 conn(0x55962c1e60f0 msgr2=0x55962c1e64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:07.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.111+0000 7f5a47a38740 1 -- 172.21.15.116:0/3285973826 shutdown_connections 2024-09-12T03:08:07.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.111+0000 7f5a47a38740 1 -- 172.21.15.116:0/3285973826 wait complete. 2024-09-12T03:08:07.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.111+0000 7f5a47a38740 1 -- start start 2024-09-12T03:08:07.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.111+0000 7f5a47a38740 1 Processor -- start 2024-09-12T03:08:07.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.112+0000 7f5a47a38740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55962c1e9b60 0x55962c1f1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:07.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.112+0000 7f5a47a38740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55962c1ea480 0x55962c1f2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:07.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.112+0000 7f5a47a38740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55962c1f4d40 0x55962c1f2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:07.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.112+0000 7f5a47a38740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55962c1f7790 con 0x55962c1f4d40 2024-09-12T03:08:07.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.112+0000 7f5a47a38740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55962c109f10 con 0x55962c1e9b60 2024-09-12T03:08:07.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.112+0000 7f5a47a38740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55962c1f7910 con 0x55962c1ea480 2024-09-12T03:08:07.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.112+0000 7f5a47a38740 10 client.0 _ll_register_callbacks cb 0x55962c0d2a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:07.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.112+0000 7f5a3f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55962c1f4d40 0x55962c1f2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:07.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.112+0000 7f5a3f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55962c1f4d40 0x55962c1f2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45646/0 (socket says 172.21.15.116:45646) 2024-09-12T03:08:07.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.112+0000 7f5a3f7fe640 1 -- 172.21.15.116:0/4263923536 learned_addr learned my addr 172.21.15.116:0/4263923536 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:07.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.112+0000 7f5a3effd640 1 --2- 172.21.15.116:0/4263923536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55962c1e9b60 0x55962c1f1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:07.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.112+0000 7f5a3e7fc640 1 --2- 172.21.15.116:0/4263923536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55962c1ea480 0x55962c1f2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:07.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.112+0000 7f5a3f7fe640 1 -- 172.21.15.116:0/4263923536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55962c1ea480 msgr2=0x55962c1f2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:07.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.112+0000 7f5a3f7fe640 1 --2- 172.21.15.116:0/4263923536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55962c1ea480 0x55962c1f2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.112+0000 7f5a3f7fe640 1 -- 172.21.15.116:0/4263923536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55962c1e9b60 msgr2=0x55962c1f1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:07.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.112+0000 7f5a3f7fe640 1 --2- 172.21.15.116:0/4263923536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55962c1e9b60 0x55962c1f1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.112+0000 7f5a3f7fe640 1 -- 172.21.15.116:0/4263923536 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55962bec06a0 con 0x55962c1f4d40 2024-09-12T03:08:07.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.112+0000 7f5a3effd640 1 --2- 172.21.15.116:0/4263923536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55962c1e9b60 0x55962c1f1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:08:07.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.112+0000 7f5a3e7fc640 1 --2- 172.21.15.116:0/4263923536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55962c1ea480 0x55962c1f2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:08:07.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.112+0000 7f5a3f7fe640 1 --2- 172.21.15.116:0/4263923536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55962c1f4d40 0x55962c1f2670 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5a380099b0 tx=0x7f5a38004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e0b0aeab600e8a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:07.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.112+0000 7f5a22ff5640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:07.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.112+0000 7f5a22ff5640 1 -- 172.21.15.116:0/4263923536 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a380319a0 con 0x55962c1f4d40 2024-09-12T03:08:07.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.112+0000 7f5a22ff5640 1 -- 172.21.15.116:0/4263923536 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f5a38004410 con 0x55962c1f4d40 2024-09-12T03:08:07.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.112+0000 7f5a47a38740 1 -- 172.21.15.116:0/4263923536 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a3804f070 con 0x55962c1f4d40 2024-09-12T03:08:07.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.113+0000 7f5a22ff5640 1 -- 172.21.15.116:0/4263923536 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5a38004610 con 0x55962c1f4d40 2024-09-12T03:08:07.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.113+0000 7f5a47a38740 10 client.16554 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:07.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.113+0000 7f5a47a38740 1 -- 172.21.15.116:0/4263923536 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55962c1f7790 con 0x55962c1f4d40 2024-09-12T03:08:07.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.113+0000 7f5a22ff5640 1 -- 172.21.15.116:0/4263923536 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a3804f070 con 0x55962c1f4d40 2024-09-12T03:08:07.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.113+0000 7f5a22ff5640 1 -- 172.21.15.116:0/4263923536 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5a3804a590 con 0x55962c1f4d40 2024-09-12T03:08:07.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.113+0000 7f5a47a38740 10 client.16554 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:07.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.113+0000 7f5a47a38740 10 client.16554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:08:07.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.113+0000 7f5a47a38740 20 client.16554 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:07.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.113+0000 7f5a47a38740 1 --2- 172.21.15.116:0/4263923536 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55962c2044a0 0x55962c2068b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:07.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.113+0000 7f5a47a38740 4 client.16554 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:08:07.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.113+0000 7f5a47a38740 1 -- 172.21.15.116:0/4263923536 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55962c15c7f0 con 0x55962c2044a0 2024-09-12T03:08:07.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.113+0000 7f5a3effd640 1 --2- 172.21.15.116:0/4263923536 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55962c2044a0 0x55962c2068b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:07.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.114+0000 7f5a22ff5640 10 client.16554 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:07.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.114+0000 7f5a3effd640 1 --2- 172.21.15.116:0/4263923536 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55962c2044a0 0x55962c2068b0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=843d7a032f37210e server_cookie=a2650794a417e231 in_seq=0 out_seq=0 2024-09-12T03:08:07.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.114+0000 7f5a22ff5640 1 -- 172.21.15.116:0/4263923536 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55962c15c7f0 con 0x55962c2044a0 2024-09-12T03:08:07.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.114+0000 7f5a22ff5640 10 client.16554 handle_command_reply: tid=0 2024-09-12T03:08:07.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.115+0000 7f5a47a38740 1 client.16554 shutdown 2024-09-12T03:08:07.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.115+0000 7f5a47a38740 1 -- 172.21.15.116:0/4263923536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55962c1f4d40 msgr2=0x55962c1f2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:07.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.115+0000 7f5a47a38740 1 --2- 172.21.15.116:0/4263923536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55962c1f4d40 0x55962c1f2670 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5a380099b0 tx=0x7f5a38004a80 comp rx=0 tx=0).stop 2024-09-12T03:08:07.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.115+0000 7f5a47a38740 1 -- 172.21.15.116:0/4263923536 shutdown_connections 2024-09-12T03:08:07.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.115+0000 7f5a47a38740 1 --2- 172.21.15.116:0/4263923536 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55962c2044a0 0x55962c2068b0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.115+0000 7f5a47a38740 1 --2- 172.21.15.116:0/4263923536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55962c1f4d40 0x55962c1f2670 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.115+0000 7f5a47a38740 1 --2- 172.21.15.116:0/4263923536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55962c1ea480 0x55962c1f2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.115+0000 7f5a47a38740 1 --2- 172.21.15.116:0/4263923536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55962c1e9b60 0x55962c1f1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.115+0000 7f5a22ff5640 0 client.16554 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:07.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.115+0000 7f5a47a38740 1 -- 172.21.15.116:0/4263923536 >> 172.21.15.116:0/4263923536 conn(0x55962c1e60f0 msgr2=0x55962c1e5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:07.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.115+0000 7f5a47a38740 1 -- 172.21.15.116:0/4263923536 shutdown_connections 2024-09-12T03:08:07.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.115+0000 7f5a47a38740 1 -- 172.21.15.116:0/4263923536 wait complete. 2024-09-12T03:08:07.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.115+0000 7f5a47a38740 20 client.16554 trim_cache size 0 max 16384 2024-09-12T03:08:07.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.132+0000 7f5a47a38740 1 Processor -- start 2024-09-12T03:08:07.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.132+0000 7f5a47a38740 1 -- start start 2024-09-12T03:08:07.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.132+0000 7f5a47a38740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55962c1e9b60 0x55962c1ec5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:07.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.132+0000 7f5a47a38740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55962c1ea480 0x55962c1e5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:07.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.132+0000 7f5a47a38740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55962c1f4d40 0x55962c20c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:07.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.132+0000 7f5a47a38740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5a401afca0 con 0x55962c1ea480 2024-09-12T03:08:07.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.132+0000 7f5a47a38740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5a401ad570 con 0x55962c1f4d40 2024-09-12T03:08:07.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.132+0000 7f5a47a38740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5a401ab3c0 con 0x55962c1e9b60 2024-09-12T03:08:07.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.132+0000 7f5a3e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55962c1ea480 0x55962c1e5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:07.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.133+0000 7f5a3e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55962c1ea480 0x55962c1e5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45654/0 (socket says 172.21.15.116:45654) 2024-09-12T03:08:07.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.133+0000 7f5a3e7fc640 1 -- 172.21.15.116:0/3868785948 learned_addr learned my addr 172.21.15.116:0/3868785948 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:07.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.133+0000 7f5a3f7fe640 1 --2- 172.21.15.116:0/3868785948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55962c1f4d40 0x55962c20c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:07.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.133+0000 7f5a3effd640 1 --2- 172.21.15.116:0/3868785948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55962c1e9b60 0x55962c1ec5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:07.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.133+0000 7f5a3e7fc640 1 -- 172.21.15.116:0/3868785948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55962c1e9b60 msgr2=0x55962c1ec5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:07.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.133+0000 7f5a3e7fc640 1 --2- 172.21.15.116:0/3868785948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55962c1e9b60 0x55962c1ec5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.133+0000 7f5a3e7fc640 1 -- 172.21.15.116:0/3868785948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55962c1f4d40 msgr2=0x55962c20c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:07.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.133+0000 7f5a3e7fc640 1 --2- 172.21.15.116:0/3868785948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55962c1f4d40 0x55962c20c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.133+0000 7f5a3e7fc640 1 -- 172.21.15.116:0/3868785948 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55962c1ea2b0 con 0x55962c1ea480 2024-09-12T03:08:07.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.133+0000 7f5a3f7fe640 1 --2- 172.21.15.116:0/3868785948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55962c1f4d40 0x55962c20c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:07.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.133+0000 7f5a3e7fc640 1 --2- 172.21.15.116:0/3868785948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55962c1ea480 0x55962c1e5160 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5a3409b250 tx=0x7f5a34009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9531ea31ab792c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:07.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.133+0000 7f5a24ff9640 1 -- 172.21.15.116:0/3868785948 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34020040 con 0x55962c1ea480 2024-09-12T03:08:07.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.133+0000 7f5a24ff9640 1 -- 172.21.15.116:0/3868785948 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a340b9e50 con 0x55962c1ea480 2024-09-12T03:08:07.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.133+0000 7f5a24ff9640 1 -- 172.21.15.116:0/3868785948 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34009cc0 con 0x55962c1ea480 2024-09-12T03:08:07.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.134+0000 7f5a47a38740 1 -- 172.21.15.116:0/3868785948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55962c1ea480 msgr2=0x55962c1e5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:07.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.134+0000 7f5a47a38740 1 --2- 172.21.15.116:0/3868785948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55962c1ea480 0x55962c1e5160 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5a3409b250 tx=0x7f5a34009c30 comp rx=0 tx=0).stop 2024-09-12T03:08:07.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.134+0000 7f5a47a38740 1 -- 172.21.15.116:0/3868785948 shutdown_connections 2024-09-12T03:08:07.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.134+0000 7f5a47a38740 1 --2- 172.21.15.116:0/3868785948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55962c1f4d40 0x55962c20c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.134+0000 7f5a47a38740 1 --2- 172.21.15.116:0/3868785948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55962c1ea480 0x55962c1e5160 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.134+0000 7f5a47a38740 1 --2- 172.21.15.116:0/3868785948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55962c1e9b60 0x55962c1ec5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.134+0000 7f5a47a38740 1 -- 172.21.15.116:0/3868785948 >> 172.21.15.116:0/3868785948 conn(0x55962c1e60f0 msgr2=0x55962c1e4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:07.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.134+0000 7f5a47a38740 1 -- 172.21.15.116:0/3868785948 shutdown_connections 2024-09-12T03:08:07.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.134+0000 7f5a47a38740 1 -- 172.21.15.116:0/3868785948 wait complete. 2024-09-12T03:08:07.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.134+0000 7f5a47a38740 1 -- start start 2024-09-12T03:08:07.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.134+0000 7f5a47a38740 1 Processor -- start 2024-09-12T03:08:07.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.135+0000 7f5a47a38740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55962c1e9b60 0x55962c1f8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:07.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.135+0000 7f5a47a38740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55962c1ea480 0x55962c1f8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:07.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.135+0000 7f5a47a38740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55962c1f4d40 0x55962c1f92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:07.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.135+0000 7f5a47a38740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5a401aba30 con 0x55962c1ea480 2024-09-12T03:08:07.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.135+0000 7f5a47a38740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5a401acea0 con 0x55962c1f4d40 2024-09-12T03:08:07.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.135+0000 7f5a47a38740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5a401ac100 con 0x55962c1e9b60 2024-09-12T03:08:07.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.135+0000 7f5a47a38740 10 client.0 _ll_register_callbacks cb 0x7f5a38031dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:07.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.135+0000 7f5a3e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55962c1ea480 0x55962c1f8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:07.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.135+0000 7f5a3e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55962c1ea480 0x55962c1f8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45662/0 (socket says 172.21.15.116:45662) 2024-09-12T03:08:07.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.135+0000 7f5a3e7fc640 1 -- 172.21.15.116:0/2428169138 learned_addr learned my addr 172.21.15.116:0/2428169138 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:07.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.135+0000 7f5a3effd640 1 --2- 172.21.15.116:0/2428169138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55962c1e9b60 0x55962c1f8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:07.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.135+0000 7f5a3e7fc640 1 -- 172.21.15.116:0/2428169138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55962c1e9b60 msgr2=0x55962c1f8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:07.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.135+0000 7f5a3f7fe640 1 --2- 172.21.15.116:0/2428169138 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55962c1f4d40 0x55962c1f92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:07.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.135+0000 7f5a3e7fc640 1 --2- 172.21.15.116:0/2428169138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55962c1e9b60 0x55962c1f8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.135+0000 7f5a3e7fc640 1 -- 172.21.15.116:0/2428169138 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55962c1f4d40 msgr2=0x55962c1f92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:07.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.135+0000 7f5a3e7fc640 1 --2- 172.21.15.116:0/2428169138 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55962c1f4d40 0x55962c1f92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.135+0000 7f5a3e7fc640 1 -- 172.21.15.116:0/2428169138 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55962c272560 con 0x55962c1ea480 2024-09-12T03:08:07.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.135+0000 7f5a3e7fc640 1 --2- 172.21.15.116:0/2428169138 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55962c1ea480 0x55962c1f8d80 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f5a3409afc0 tx=0x7f5a3409f630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed18f1e24d009850 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:07.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.135+0000 7f5a22ff5640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:07.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.135+0000 7f5a22ff5640 1 -- 172.21.15.116:0/2428169138 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34020050 con 0x55962c1ea480 2024-09-12T03:08:07.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.135+0000 7f5a22ff5640 1 -- 172.21.15.116:0/2428169138 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f5a340cc500 con 0x55962c1ea480 2024-09-12T03:08:07.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.135+0000 7f5a47a38740 1 -- 172.21.15.116:0/2428169138 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a340b9e50 con 0x55962c1ea480 2024-09-12T03:08:07.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.136+0000 7f5a22ff5640 1 -- 172.21.15.116:0/2428169138 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5a34097930 con 0x55962c1ea480 2024-09-12T03:08:07.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.136+0000 7f5a47a38740 10 client.16560 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:07.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.136+0000 7f5a47a38740 1 -- 172.21.15.116:0/2428169138 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55962c273800 con 0x55962c1ea480 2024-09-12T03:08:07.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.136+0000 7f5a22ff5640 1 -- 172.21.15.116:0/2428169138 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a340b9e50 con 0x55962c1ea480 2024-09-12T03:08:07.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.136+0000 7f5a22ff5640 1 -- 172.21.15.116:0/2428169138 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5a3409a710 con 0x55962c1ea480 2024-09-12T03:08:07.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.136+0000 7f5a47a38740 10 client.16560 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:07.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.136+0000 7f5a47a38740 10 client.16560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:08:07.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.136+0000 7f5a47a38740 20 client.16560 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:07.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.136+0000 7f5a47a38740 1 --2- 172.21.15.116:0/2428169138 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55962c273f10 0x55962c276330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:07.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.136+0000 7f5a47a38740 4 client.16560 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:08:07.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.136+0000 7f5a47a38740 1 -- 172.21.15.116:0/2428169138 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55962c156d00 con 0x55962c273f10 2024-09-12T03:08:07.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.136+0000 7f5a3effd640 1 --2- 172.21.15.116:0/2428169138 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55962c273f10 0x55962c276330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:07.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.137+0000 7f5a22ff5640 10 client.16560 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:07.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.137+0000 7f5a3effd640 1 --2- 172.21.15.116:0/2428169138 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55962c273f10 0x55962c276330 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29aa689c3ecabf21 server_cookie=7c711e755b4ab65b in_seq=0 out_seq=0 2024-09-12T03:08:07.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.138+0000 7f5a22ff5640 1 -- 172.21.15.116:0/2428169138 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55962c156d00 con 0x55962c273f10 2024-09-12T03:08:07.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.138+0000 7f5a22ff5640 10 client.16560 handle_command_reply: tid=0 2024-09-12T03:08:07.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.138+0000 7f5a47a38740 1 client.16560 shutdown 2024-09-12T03:08:07.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.138+0000 7f5a47a38740 1 -- 172.21.15.116:0/2428169138 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55962c1ea480 msgr2=0x55962c1f8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:07.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.138+0000 7f5a47a38740 1 --2- 172.21.15.116:0/2428169138 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55962c1ea480 0x55962c1f8d80 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f5a3409afc0 tx=0x7f5a3409f630 comp rx=0 tx=0).stop 2024-09-12T03:08:07.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.138+0000 7f5a47a38740 1 -- 172.21.15.116:0/2428169138 shutdown_connections 2024-09-12T03:08:07.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.138+0000 7f5a47a38740 1 --2- 172.21.15.116:0/2428169138 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55962c273f10 0x55962c276330 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.138+0000 7f5a47a38740 1 --2- 172.21.15.116:0/2428169138 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55962c1f4d40 0x55962c1f92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.138+0000 7f5a47a38740 1 --2- 172.21.15.116:0/2428169138 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55962c1ea480 0x55962c1f8d80 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.138+0000 7f5a47a38740 1 --2- 172.21.15.116:0/2428169138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55962c1e9b60 0x55962c1f8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.138+0000 7f5a47a38740 1 -- 172.21.15.116:0/2428169138 >> 172.21.15.116:0/2428169138 conn(0x55962c1e60f0 msgr2=0x55962c1f23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:07.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.138+0000 7f5a22ff5640 0 client.16560 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:07.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.138+0000 7f5a47a38740 1 -- 172.21.15.116:0/2428169138 shutdown_connections 2024-09-12T03:08:07.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.138+0000 7f5a47a38740 1 -- 172.21.15.116:0/2428169138 wait complete. 2024-09-12T03:08:07.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.138+0000 7f5a47a38740 20 client.16560 trim_cache size 0 max 16384 2024-09-12T03:08:07.145 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:08:07.145 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:08:07.145 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "9ea76c4f-6222-49ed-b973-04efd15f0624", 2024-09-12T03:08:07.145 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:08:07.145 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:08:07.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.139+0000 7f5a4565f640 1 -- 172.21.15.116:0/1281490857 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5a280a2b80 msgr2=0x7f5a280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:07.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.139+0000 7f5a4565f640 1 --2- 172.21.15.116:0/1281490857 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5a280a2b80 0x7f5a280a5040 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5a38004af0 tx=0x7f5a3804c000 comp rx=0 tx=0).stop 2024-09-12T03:08:07.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.139+0000 7f5a4565f640 1 -- 172.21.15.116:0/1281490857 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a401027a0 msgr2=0x7f5a40198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:07.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.139+0000 7f5a4565f640 1 --2- 172.21.15.116:0/1281490857 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a401027a0 0x7f5a40198980 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5a3400d770 tx=0x7f5a3400dc30 comp rx=0 tx=0).stop 2024-09-12T03:08:07.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.139+0000 7f5a4565f640 1 -- 172.21.15.116:0/1281490857 shutdown_connections 2024-09-12T03:08:07.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.139+0000 7f5a4565f640 1 --2- 172.21.15.116:0/1281490857 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5a280a2b80 0x7f5a280a5040 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.139+0000 7f5a4565f640 1 --2- 172.21.15.116:0/1281490857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5a40106f20 0x7f5a4019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.139+0000 7f5a4565f640 1 --2- 172.21.15.116:0/1281490857 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a401027a0 0x7f5a40198980 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.139+0000 7f5a4565f640 1 --2- 172.21.15.116:0/1281490857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a40101df0 0x7f5a40198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:07.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.139+0000 7f5a4565f640 1 -- 172.21.15.116:0/1281490857 >> 172.21.15.116:0/1281490857 conn(0x7f5a400fd8b0 msgr2=0x7f5a400feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:07.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.139+0000 7f5a4565f640 1 -- 172.21.15.116:0/1281490857 shutdown_connections 2024-09-12T03:08:07.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:07.139+0000 7f5a4565f640 1 -- 172.21.15.116:0/1281490857 wait complete. 2024-09-12T03:08:07.990 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:08:08.236 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:08:08.648 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:08 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:08:08.648 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:08 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:08:08.648 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:08 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:08:08.648 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:08 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:08:08.648 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:08 smithi116 ceph-mon[31410]: pgmap v492: 97 pgs: 97 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-12T03:08:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:08 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:08:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:08 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:08:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:08 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:08:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:08 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:08:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:08 smithi183 ceph-mon[35535]: pgmap v492: 97 pgs: 97 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-12T03:08:08.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:08 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:08:08.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:08 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:08:08.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:08 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:08:08.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:08 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:08:08.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:08 smithi177 ceph-mon[35305]: pgmap v492: 97 pgs: 97 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-12T03:08:08.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.732+0000 7f2ec49de640 1 -- 172.21.15.116:0/4133313476 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ec0106a80 msgr2=0x7f2ec0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:08.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.732+0000 7f2ec49de640 1 --2- 172.21.15.116:0/4133313476 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ec0106a80 0x7f2ec0106e60 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0009930 tx=0x7f2eb002a420 comp rx=0 tx=0).stop 2024-09-12T03:08:08.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.732+0000 7f2ec49de640 1 -- 172.21.15.116:0/4133313476 shutdown_connections 2024-09-12T03:08:08.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.732+0000 7f2ec49de640 1 --2- 172.21.15.116:0/4133313476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ec0101420 0x7f2ec010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.732+0000 7f2ec49de640 1 --2- 172.21.15.116:0/4133313476 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ec0100a80 0x7f2ec0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.732+0000 7f2ec49de640 1 --2- 172.21.15.116:0/4133313476 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ec0106a80 0x7f2ec0106e60 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.732+0000 7f2ec49de640 1 -- 172.21.15.116:0/4133313476 >> 172.21.15.116:0/4133313476 conn(0x7f2ec00fc7a0 msgr2=0x7f2ec00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:08.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.732+0000 7f2ec49de640 1 -- 172.21.15.116:0/4133313476 shutdown_connections 2024-09-12T03:08:08.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.732+0000 7f2ec49de640 1 -- 172.21.15.116:0/4133313476 wait complete. 2024-09-12T03:08:08.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.732+0000 7f2ec49de640 1 Processor -- start 2024-09-12T03:08:08.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.732+0000 7f2ec49de640 1 -- start start 2024-09-12T03:08:08.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.733+0000 7f2ec49de640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ec0100a80 0x7f2ec0199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:08.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.733+0000 7f2ec49de640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ec0101420 0x7f2ec0199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:08.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.733+0000 7f2ec49de640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ec0106a80 0x7f2ec01a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:08.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.733+0000 7f2ec49de640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec0110e80 con 0x7f2ec0101420 2024-09-12T03:08:08.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.733+0000 7f2ec49de640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec0110d00 con 0x7f2ec0100a80 2024-09-12T03:08:08.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.733+0000 7f2ec49de640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec0111000 con 0x7f2ec0106a80 2024-09-12T03:08:08.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.733+0000 7f2eb7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ec0101420 0x7f2ec0199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:08.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.733+0000 7f2eb7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ec0101420 0x7f2ec0199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45680/0 (socket says 172.21.15.116:45680) 2024-09-12T03:08:08.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.733+0000 7f2eb7fff640 1 -- 172.21.15.116:0/3075138403 learned_addr learned my addr 172.21.15.116:0/3075138403 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:08.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.733+0000 7f2ebed9d640 1 --2- 172.21.15.116:0/3075138403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ec0106a80 0x7f2ec01a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:08.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.733+0000 7f2ebe59c640 1 --2- 172.21.15.116:0/3075138403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ec0100a80 0x7f2ec0199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:08.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.733+0000 7f2eb7fff640 1 -- 172.21.15.116:0/3075138403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ec0106a80 msgr2=0x7f2ec01a42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:08.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.733+0000 7f2eb7fff640 1 --2- 172.21.15.116:0/3075138403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ec0106a80 0x7f2ec01a42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.733+0000 7f2eb7fff640 1 -- 172.21.15.116:0/3075138403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ec0100a80 msgr2=0x7f2ec0199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:08.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.733+0000 7f2eb7fff640 1 --2- 172.21.15.116:0/3075138403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ec0100a80 0x7f2ec0199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.733+0000 7f2eb7fff640 1 -- 172.21.15.116:0/3075138403 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec01a47f0 con 0x7f2ec0101420 2024-09-12T03:08:08.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.733+0000 7f2eb7fff640 1 --2- 172.21.15.116:0/3075138403 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ec0101420 0x7f2ec0199770 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f2ea800c920 tx=0x7f2ea800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:08.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.733+0000 7f2eb77fe640 1 -- 172.21.15.116:0/3075138403 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8007c70 con 0x7f2ec0101420 2024-09-12T03:08:08.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.733+0000 7f2eb77fe640 1 -- 172.21.15.116:0/3075138403 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ea8007e10 con 0x7f2ec0101420 2024-09-12T03:08:08.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.734+0000 7f2ec49de640 1 -- 172.21.15.116:0/3075138403 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec01a49c0 con 0x7f2ec0101420 2024-09-12T03:08:08.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.734+0000 7f2eb77fe640 1 -- 172.21.15.116:0/3075138403 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8018890 con 0x7f2ec0101420 2024-09-12T03:08:08.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.734+0000 7f2ec49de640 1 -- 172.21.15.116:0/3075138403 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec01a4ed0 con 0x7f2ec0101420 2024-09-12T03:08:08.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.736+0000 7f2ec6db7740 1 Processor -- start 2024-09-12T03:08:08.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.736+0000 7f2eb77fe640 1 -- 172.21.15.116:0/3075138403 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2ea8018a70 con 0x7f2ec0101420 2024-09-12T03:08:08.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.736+0000 7f2ec6db7740 1 -- start start 2024-09-12T03:08:08.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.736+0000 7f2ec6db7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56442a757360 0x56442a757740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:08.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.736+0000 7f2ec6db7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56442a757c80 0x56442a766550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:08.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.736+0000 7f2ec6db7740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56442a766a90 0x56442a768e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:08.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.736+0000 7f2ec6db7740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56442a6bd2e0 con 0x56442a766a90 2024-09-12T03:08:08.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.736+0000 7f2ec6db7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56442a664ce0 con 0x56442a757c80 2024-09-12T03:08:08.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.736+0000 7f2ec6db7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56442a6b92a0 con 0x56442a757360 2024-09-12T03:08:08.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.737+0000 7f2eb7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56442a766a90 0x56442a768e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:08.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.737+0000 7f2eb7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56442a766a90 0x56442a768e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45692/0 (socket says 172.21.15.116:45692) 2024-09-12T03:08:08.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.737+0000 7f2eb7fff640 1 -- 172.21.15.116:0/2252089832 learned_addr learned my addr 172.21.15.116:0/2252089832 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:08.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.737+0000 7f2ebed9d640 1 --2- 172.21.15.116:0/2252089832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56442a757360 0x56442a757740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:08.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.737+0000 7f2ebe59c640 1 --2- 172.21.15.116:0/2252089832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56442a757c80 0x56442a766550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:08.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.737+0000 7f2eb7fff640 1 -- 172.21.15.116:0/2252089832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56442a757360 msgr2=0x56442a757740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:08.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.737+0000 7f2eb7fff640 1 --2- 172.21.15.116:0/2252089832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56442a757360 0x56442a757740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.737+0000 7f2eb7fff640 1 -- 172.21.15.116:0/2252089832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56442a757c80 msgr2=0x56442a766550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:08.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.737+0000 7f2eb77fe640 1 --2- 172.21.15.116:0/3075138403 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2eac0a2b80 0x7f2eac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:08.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.737+0000 7f2eb7fff640 1 --2- 172.21.15.116:0/2252089832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56442a757c80 0x56442a766550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.737+0000 7f2eb7fff640 1 -- 172.21.15.116:0/2252089832 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56442a649440 con 0x56442a766a90 2024-09-12T03:08:08.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.737+0000 7f2ebe59c640 1 --2- 172.21.15.116:0/2252089832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56442a757c80 0x56442a766550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:08.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.737+0000 7f2ebed9d640 1 --2- 172.21.15.116:0/3075138403 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2eac0a2b80 0x7f2eac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:08.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.737+0000 7f2eb7fff640 1 --2- 172.21.15.116:0/2252089832 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56442a766a90 0x56442a768e80 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f2ea809ea90 tx=0x7f2ea8097ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26e2a7ad398b14e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:08.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.737+0000 7f2eb77fe640 1 -- 172.21.15.116:0/3075138403 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f2ea80cf0b0 con 0x7f2ec0101420 2024-09-12T03:08:08.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.737+0000 7f2eb4ff9640 1 -- 172.21.15.116:0/2252089832 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea809b710 con 0x56442a766a90 2024-09-12T03:08:08.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.737+0000 7f2eb4ff9640 1 -- 172.21.15.116:0/2252089832 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ea809eac0 con 0x56442a766a90 2024-09-12T03:08:08.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.737+0000 7f2eb4ff9640 1 -- 172.21.15.116:0/2252089832 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea809ed10 con 0x56442a766a90 2024-09-12T03:08:08.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.738+0000 7f2ec6db7740 1 -- 172.21.15.116:0/2252089832 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56442a766a90 msgr2=0x56442a768e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:08.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.738+0000 7f2ec6db7740 1 --2- 172.21.15.116:0/2252089832 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56442a766a90 0x56442a768e80 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f2ea809ea90 tx=0x7f2ea8097ca0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.738+0000 7f2ebed9d640 1 --2- 172.21.15.116:0/3075138403 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2eac0a2b80 0x7f2eac0a5040 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8005fd0 tx=0x7f2eb8009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:08.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.738+0000 7f2ec6db7740 1 -- 172.21.15.116:0/2252089832 shutdown_connections 2024-09-12T03:08:08.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.738+0000 7f2ec6db7740 1 --2- 172.21.15.116:0/2252089832 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56442a766a90 0x56442a768e80 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.738+0000 7f2ec6db7740 1 --2- 172.21.15.116:0/2252089832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56442a757c80 0x56442a766550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.738+0000 7f2ec6db7740 1 --2- 172.21.15.116:0/2252089832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56442a757360 0x56442a757740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.738+0000 7f2ec6db7740 1 -- 172.21.15.116:0/2252089832 >> 172.21.15.116:0/2252089832 conn(0x56442a753a30 msgr2=0x56442a753e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:08.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.738+0000 7f2ec6db7740 1 -- 172.21.15.116:0/2252089832 shutdown_connections 2024-09-12T03:08:08.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.738+0000 7f2ec6db7740 1 -- 172.21.15.116:0/2252089832 wait complete. 2024-09-12T03:08:08.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.739+0000 7f2ec6db7740 1 -- start start 2024-09-12T03:08:08.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.739+0000 7f2ec6db7740 1 Processor -- start 2024-09-12T03:08:08.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.739+0000 7f2ec6db7740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56442a757360 0x56442a7664e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:08.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.739+0000 7f2ec6db7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56442a757c80 0x56442a763790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:08.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.739+0000 7f2ec6db7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56442a766a90 0x56442a763f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:08.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.739+0000 7f2ec6db7740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56442a769600 con 0x56442a757360 2024-09-12T03:08:08.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.739+0000 7f2ec6db7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56442a678050 con 0x56442a757c80 2024-09-12T03:08:08.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.739+0000 7f2ec6db7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56442a769780 con 0x56442a766a90 2024-09-12T03:08:08.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.739+0000 7f2ec6db7740 10 client.0 _ll_register_callbacks cb 0x56442a6b8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:08.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.739+0000 7f2ebe59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56442a757360 0x56442a7664e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:08.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.739+0000 7f2ebe59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56442a757360 0x56442a7664e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45696/0 (socket says 172.21.15.116:45696) 2024-09-12T03:08:08.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.739+0000 7f2ebe59c640 1 -- 172.21.15.116:0/1785162356 learned_addr learned my addr 172.21.15.116:0/1785162356 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:08.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.739+0000 7f2ebed9d640 1 --2- 172.21.15.116:0/1785162356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56442a766a90 0x56442a763f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:08.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.739+0000 7f2eb7fff640 1 --2- 172.21.15.116:0/1785162356 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56442a757c80 0x56442a763790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:08.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.739+0000 7f2ebe59c640 1 -- 172.21.15.116:0/1785162356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56442a766a90 msgr2=0x56442a763f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:08.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.739+0000 7f2ebe59c640 1 --2- 172.21.15.116:0/1785162356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56442a766a90 0x56442a763f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.739+0000 7f2ebe59c640 1 -- 172.21.15.116:0/1785162356 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56442a757c80 msgr2=0x56442a763790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:08.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.739+0000 7f2ebe59c640 1 --2- 172.21.15.116:0/1785162356 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56442a757c80 0x56442a763790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.739+0000 7f2ebe59c640 1 -- 172.21.15.116:0/1785162356 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56442a42e3c0 con 0x56442a757360 2024-09-12T03:08:08.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.739+0000 7f2eb7fff640 1 --2- 172.21.15.116:0/1785162356 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56442a757c80 0x56442a763790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:08:08.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.739+0000 7f2ebe59c640 1 --2- 172.21.15.116:0/1785162356 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56442a757360 0x56442a7664e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0009930 tx=0x7f2eb00318d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d6aa8e2757b804 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:08.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.739+0000 7f2ea5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:08.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.740+0000 7f2ea5ffb640 1 -- 172.21.15.116:0/1785162356 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0035040 con 0x56442a757360 2024-09-12T03:08:08.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.740+0000 7f2ec6db7740 1 -- 172.21.15.116:0/1785162356 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56442a6ca850 con 0x56442a757360 2024-09-12T03:08:08.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.740+0000 7f2ea5ffb640 1 -- 172.21.15.116:0/1785162356 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f2eb0004240 con 0x56442a757360 2024-09-12T03:08:08.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.740+0000 7f2ea5ffb640 1 -- 172.21.15.116:0/1785162356 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2eb004c070 con 0x56442a757360 2024-09-12T03:08:08.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.740+0000 7f2ec6db7740 10 client.16572 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:08.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.740+0000 7f2ec6db7740 1 -- 172.21.15.116:0/1785162356 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56442a775110 con 0x56442a757360 2024-09-12T03:08:08.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.740+0000 7f2ea5ffb640 1 -- 172.21.15.116:0/1785162356 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56442a6ca850 con 0x56442a757360 2024-09-12T03:08:08.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.740+0000 7f2ea5ffb640 1 -- 172.21.15.116:0/1785162356 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2eb0004c50 con 0x56442a757360 2024-09-12T03:08:08.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.740+0000 7f2ec6db7740 10 client.16572 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:08.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.740+0000 7f2ec6db7740 10 client.16572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:08:08.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.740+0000 7f2ec6db7740 20 client.16572 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:08.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.740+0000 7f2ec6db7740 1 --2- 172.21.15.116:0/1785162356 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56442a7765d0 0x56442a7789e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:08.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.740+0000 7f2ec6db7740 4 client.16572 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:08:08.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.740+0000 7f2ec6db7740 1 -- 172.21.15.116:0/1785162356 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56442a6c4d60 con 0x56442a7765d0 2024-09-12T03:08:08.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.740+0000 7f2eb7fff640 1 --2- 172.21.15.116:0/1785162356 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56442a7765d0 0x56442a7789e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:08.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.741+0000 7f2ea5ffb640 10 client.16572 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:08.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.741+0000 7f2eb7fff640 1 --2- 172.21.15.116:0/1785162356 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56442a7765d0 0x56442a7789e0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8791a586b80c89 server_cookie=622adc544f245938 in_seq=0 out_seq=0 2024-09-12T03:08:08.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.741+0000 7f2ea5ffb640 1 -- 172.21.15.116:0/1785162356 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56442a6c4d60 con 0x56442a7765d0 2024-09-12T03:08:08.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.741+0000 7f2ea5ffb640 10 client.16572 handle_command_reply: tid=0 2024-09-12T03:08:08.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.742+0000 7f2ec6db7740 1 client.16572 shutdown 2024-09-12T03:08:08.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.742+0000 7f2ec6db7740 1 -- 172.21.15.116:0/1785162356 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56442a757360 msgr2=0x56442a7664e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:08.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.742+0000 7f2ec6db7740 1 --2- 172.21.15.116:0/1785162356 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56442a757360 0x56442a7664e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0009930 tx=0x7f2eb00318d0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.742+0000 7f2ec6db7740 1 -- 172.21.15.116:0/1785162356 shutdown_connections 2024-09-12T03:08:08.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.742+0000 7f2ec6db7740 1 --2- 172.21.15.116:0/1785162356 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56442a7765d0 0x56442a7789e0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.742+0000 7f2ec6db7740 1 --2- 172.21.15.116:0/1785162356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56442a766a90 0x56442a763f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.742+0000 7f2ec6db7740 1 --2- 172.21.15.116:0/1785162356 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56442a757c80 0x56442a763790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.742+0000 7f2ec6db7740 1 --2- 172.21.15.116:0/1785162356 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56442a757360 0x56442a7664e0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.742+0000 7f2ec6db7740 1 -- 172.21.15.116:0/1785162356 >> 172.21.15.116:0/1785162356 conn(0x56442a753a30 msgr2=0x56442a759790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:08.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.742+0000 7f2ea5ffb640 0 client.16572 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:08.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.742+0000 7f2ec6db7740 1 -- 172.21.15.116:0/1785162356 shutdown_connections 2024-09-12T03:08:08.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.742+0000 7f2ec6db7740 1 -- 172.21.15.116:0/1785162356 wait complete. 2024-09-12T03:08:08.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.742+0000 7f2ec6db7740 20 client.16572 trim_cache size 0 max 16384 2024-09-12T03:08:08.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.759+0000 7f2ec6db7740 1 Processor -- start 2024-09-12T03:08:08.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.759+0000 7f2ec6db7740 1 -- start start 2024-09-12T03:08:08.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.759+0000 7f2ec6db7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56442a757360 0x56442a77e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:08.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.759+0000 7f2ec6db7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56442a757c80 0x56442a77e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:08.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.759+0000 7f2ec6db7740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56442a766a90 0x56442a771890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:08.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.759+0000 7f2ec6db7740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec01adbc0 con 0x56442a766a90 2024-09-12T03:08:08.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.759+0000 7f2ec6db7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec01ab490 con 0x56442a757360 2024-09-12T03:08:08.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.759+0000 7f2ec6db7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec01a9300 con 0x56442a757c80 2024-09-12T03:08:08.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.759+0000 7f2ebed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56442a766a90 0x56442a771890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:08.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.759+0000 7f2ebed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56442a766a90 0x56442a771890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45704/0 (socket says 172.21.15.116:45704) 2024-09-12T03:08:08.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.759+0000 7f2ebed9d640 1 -- 172.21.15.116:0/3478432573 learned_addr learned my addr 172.21.15.116:0/3478432573 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:08.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.760+0000 7f2eb7fff640 1 --2- 172.21.15.116:0/3478432573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56442a757c80 0x56442a77e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:08.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.760+0000 7f2ebe59c640 1 --2- 172.21.15.116:0/3478432573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56442a757360 0x56442a77e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:08.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.760+0000 7f2ebed9d640 1 -- 172.21.15.116:0/3478432573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56442a757c80 msgr2=0x56442a77e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:08.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.760+0000 7f2ebed9d640 1 --2- 172.21.15.116:0/3478432573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56442a757c80 0x56442a77e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.760+0000 7f2ebed9d640 1 -- 172.21.15.116:0/3478432573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56442a757360 msgr2=0x56442a77e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:08.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.760+0000 7f2ebed9d640 1 --2- 172.21.15.116:0/3478432573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56442a757360 0x56442a77e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.760+0000 7f2ebed9d640 1 -- 172.21.15.116:0/3478432573 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56442a757ab0 con 0x56442a766a90 2024-09-12T03:08:08.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.760+0000 7f2ebe59c640 1 --2- 172.21.15.116:0/3478432573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56442a757360 0x56442a77e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:08:08.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.760+0000 7f2eb7fff640 1 --2- 172.21.15.116:0/3478432573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56442a757c80 0x56442a77e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:08:08.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.760+0000 7f2ebed9d640 1 --2- 172.21.15.116:0/3478432573 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56442a766a90 0x56442a771890 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2eb80092f0 tx=0x7f2eb8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de312b50e3646cd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:08.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.760+0000 7f2ea7fff640 1 -- 172.21.15.116:0/3478432573 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8003b70 con 0x56442a766a90 2024-09-12T03:08:08.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.760+0000 7f2ea7fff640 1 -- 172.21.15.116:0/3478432573 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eb8003d10 con 0x56442a766a90 2024-09-12T03:08:08.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.760+0000 7f2ea7fff640 1 -- 172.21.15.116:0/3478432573 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb80046c0 con 0x56442a766a90 2024-09-12T03:08:08.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.761+0000 7f2ec6db7740 1 -- 172.21.15.116:0/3478432573 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56442a766a90 msgr2=0x56442a771890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:08.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.761+0000 7f2ec6db7740 1 --2- 172.21.15.116:0/3478432573 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56442a766a90 0x56442a771890 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2eb80092f0 tx=0x7f2eb8016000 comp rx=0 tx=0).stop 2024-09-12T03:08:08.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.761+0000 7f2ec6db7740 1 -- 172.21.15.116:0/3478432573 shutdown_connections 2024-09-12T03:08:08.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.761+0000 7f2ec6db7740 1 --2- 172.21.15.116:0/3478432573 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56442a766a90 0x56442a771890 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.761+0000 7f2ec6db7740 1 --2- 172.21.15.116:0/3478432573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56442a757c80 0x56442a77e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.761+0000 7f2ec6db7740 1 --2- 172.21.15.116:0/3478432573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56442a757360 0x56442a77e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.761+0000 7f2ec6db7740 1 -- 172.21.15.116:0/3478432573 >> 172.21.15.116:0/3478432573 conn(0x56442a753a30 msgr2=0x56442a762880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:08.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.761+0000 7f2ec6db7740 1 -- 172.21.15.116:0/3478432573 shutdown_connections 2024-09-12T03:08:08.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.761+0000 7f2ec6db7740 1 -- 172.21.15.116:0/3478432573 wait complete. 2024-09-12T03:08:08.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.761+0000 7f2ec6db7740 1 -- start start 2024-09-12T03:08:08.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.761+0000 7f2ec6db7740 1 Processor -- start 2024-09-12T03:08:08.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.762+0000 7f2ec6db7740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56442a757360 0x56442a769db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:08.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.762+0000 7f2ec6db7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56442a757c80 0x56442a76a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:08.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.762+0000 7f2ec6db7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56442a766a90 0x56442a7df270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:08.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.762+0000 7f2ec6db7740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec01a9950 con 0x56442a757360 2024-09-12T03:08:08.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.762+0000 7f2ec6db7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec01aadc0 con 0x56442a757c80 2024-09-12T03:08:08.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.762+0000 7f2ec6db7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec01aa020 con 0x56442a766a90 2024-09-12T03:08:08.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.762+0000 7f2ec6db7740 10 client.0 _ll_register_callbacks cb 0x7f2eb0005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:08.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.762+0000 7f2ebe59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56442a757360 0x56442a769db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:08.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.762+0000 7f2ebe59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56442a757360 0x56442a769db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45706/0 (socket says 172.21.15.116:45706) 2024-09-12T03:08:08.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.762+0000 7f2ebe59c640 1 -- 172.21.15.116:0/3744577744 learned_addr learned my addr 172.21.15.116:0/3744577744 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:08.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.762+0000 7f2ebed9d640 1 --2- 172.21.15.116:0/3744577744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56442a766a90 0x56442a7df270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:08.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.762+0000 7f2eb7fff640 1 --2- 172.21.15.116:0/3744577744 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56442a757c80 0x56442a76a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:08.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.762+0000 7f2ebe59c640 1 -- 172.21.15.116:0/3744577744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56442a766a90 msgr2=0x56442a7df270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:08.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.762+0000 7f2ebe59c640 1 --2- 172.21.15.116:0/3744577744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56442a766a90 0x56442a7df270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.762+0000 7f2ebe59c640 1 -- 172.21.15.116:0/3744577744 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56442a757c80 msgr2=0x56442a76a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:08.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.762+0000 7f2ebe59c640 1 --2- 172.21.15.116:0/3744577744 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56442a757c80 0x56442a76a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.762+0000 7f2ebe59c640 1 -- 172.21.15.116:0/3744577744 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56442a764290 con 0x56442a757360 2024-09-12T03:08:08.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.762+0000 7f2ebe59c640 1 --2- 172.21.15.116:0/3744577744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56442a757360 0x56442a769db0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0003da0 tx=0x7f2eb002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9674b959389a503 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:08.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.762+0000 7f2ea5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:08.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.762+0000 7f2ea5ffb640 1 -- 172.21.15.116:0/3744577744 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0055020 con 0x56442a757360 2024-09-12T03:08:08.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.762+0000 7f2ea5ffb640 1 -- 172.21.15.116:0/3744577744 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f2eb0004290 con 0x56442a757360 2024-09-12T03:08:08.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.762+0000 7f2ec6db7740 1 -- 172.21.15.116:0/3744577744 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2eb8003d10 con 0x56442a757360 2024-09-12T03:08:08.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.763+0000 7f2ea5ffb640 1 -- 172.21.15.116:0/3744577744 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2eb0035030 con 0x56442a757360 2024-09-12T03:08:08.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.763+0000 7f2ec6db7740 10 client.16578 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:08.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.763+0000 7f2ec6db7740 1 -- 172.21.15.116:0/3744577744 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56442a7e0700 con 0x56442a757360 2024-09-12T03:08:08.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.763+0000 7f2ea5ffb640 1 -- 172.21.15.116:0/3744577744 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eb8003d10 con 0x56442a757360 2024-09-12T03:08:08.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.763+0000 7f2ea5ffb640 1 -- 172.21.15.116:0/3744577744 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2eb0045cd0 con 0x56442a757360 2024-09-12T03:08:08.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.763+0000 7f2ec6db7740 10 client.16578 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:08.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.763+0000 7f2ec6db7740 10 client.16578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:08:08.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.763+0000 7f2ec6db7740 20 client.16578 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:08.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.763+0000 7f2ec6db7740 1 --2- 172.21.15.116:0/3744577744 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56442a7e0e80 0x56442a7e32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:08.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.763+0000 7f2ec6db7740 4 client.16578 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:08:08.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.763+0000 7f2ec6db7740 1 -- 172.21.15.116:0/3744577744 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56442a641210 con 0x56442a7e0e80 2024-09-12T03:08:08.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.763+0000 7f2eb7fff640 1 --2- 172.21.15.116:0/3744577744 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56442a7e0e80 0x56442a7e32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:08.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.764+0000 7f2ea5ffb640 10 client.16578 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:08.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.764+0000 7f2eb7fff640 1 --2- 172.21.15.116:0/3744577744 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56442a7e0e80 0x56442a7e32a0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edf2d4c78b175e61 server_cookie=3e6166695f905a48 in_seq=0 out_seq=0 2024-09-12T03:08:08.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.764+0000 7f2ea5ffb640 1 -- 172.21.15.116:0/3744577744 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56442a641210 con 0x56442a7e0e80 2024-09-12T03:08:08.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.764+0000 7f2ea5ffb640 10 client.16578 handle_command_reply: tid=0 2024-09-12T03:08:08.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.764+0000 7f2ec6db7740 1 client.16578 shutdown 2024-09-12T03:08:08.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.764+0000 7f2ec6db7740 1 -- 172.21.15.116:0/3744577744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56442a757360 msgr2=0x56442a769db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:08.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.764+0000 7f2ec6db7740 1 --2- 172.21.15.116:0/3744577744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56442a757360 0x56442a769db0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0003da0 tx=0x7f2eb002a420 comp rx=0 tx=0).stop 2024-09-12T03:08:08.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.764+0000 7f2ec6db7740 1 -- 172.21.15.116:0/3744577744 shutdown_connections 2024-09-12T03:08:08.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.764+0000 7f2ec6db7740 1 --2- 172.21.15.116:0/3744577744 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56442a7e0e80 0x56442a7e32a0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.764+0000 7f2ec6db7740 1 --2- 172.21.15.116:0/3744577744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56442a766a90 0x56442a7df270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.764+0000 7f2ec6db7740 1 --2- 172.21.15.116:0/3744577744 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56442a757c80 0x56442a76a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.764+0000 7f2ec6db7740 1 --2- 172.21.15.116:0/3744577744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56442a757360 0x56442a769db0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.764+0000 7f2ec6db7740 1 -- 172.21.15.116:0/3744577744 >> 172.21.15.116:0/3744577744 conn(0x56442a753a30 msgr2=0x56442a75e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:08.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.764+0000 7f2ea5ffb640 0 client.16578 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:08.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.764+0000 7f2ec6db7740 1 -- 172.21.15.116:0/3744577744 shutdown_connections 2024-09-12T03:08:08.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.764+0000 7f2ec6db7740 1 -- 172.21.15.116:0/3744577744 wait complete. 2024-09-12T03:08:08.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.764+0000 7f2ec6db7740 20 client.16578 trim_cache size 0 max 16384 2024-09-12T03:08:08.778 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:08:08.778 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:08:08.778 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:08:08.779 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:08:08.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.765+0000 7f2ec49de640 1 -- 172.21.15.116:0/3075138403 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2eac0a2b80 msgr2=0x7f2eac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:08.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.765+0000 7f2ec49de640 1 --2- 172.21.15.116:0/3075138403 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2eac0a2b80 0x7f2eac0a5040 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8005fd0 tx=0x7f2eb8009600 comp rx=0 tx=0).stop 2024-09-12T03:08:08.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.765+0000 7f2ec49de640 1 -- 172.21.15.116:0/3075138403 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ec0101420 msgr2=0x7f2ec0199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:08.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.765+0000 7f2ec49de640 1 --2- 172.21.15.116:0/3075138403 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ec0101420 0x7f2ec0199770 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f2ea800c920 tx=0x7f2ea800cde0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.765+0000 7f2ec49de640 1 -- 172.21.15.116:0/3075138403 shutdown_connections 2024-09-12T03:08:08.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.765+0000 7f2ec49de640 1 --2- 172.21.15.116:0/3075138403 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2eac0a2b80 0x7f2eac0a5040 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.765+0000 7f2ec49de640 1 --2- 172.21.15.116:0/3075138403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ec0106a80 0x7f2ec01a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.765+0000 7f2ec49de640 1 --2- 172.21.15.116:0/3075138403 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ec0101420 0x7f2ec0199770 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.765+0000 7f2ec49de640 1 --2- 172.21.15.116:0/3075138403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ec0100a80 0x7f2ec0199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:08.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.765+0000 7f2ec49de640 1 -- 172.21.15.116:0/3075138403 >> 172.21.15.116:0/3075138403 conn(0x7f2ec00fc7a0 msgr2=0x7f2ec00fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:08.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.765+0000 7f2ec49de640 1 -- 172.21.15.116:0/3075138403 shutdown_connections 2024-09-12T03:08:08.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:08.765+0000 7f2ec49de640 1 -- 172.21.15.116:0/3075138403 wait complete. 2024-09-12T03:08:09.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:09 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:08:09.648 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:08:09.648 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:08:09.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:09 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:08:09.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:09 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:08:09.893 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:08:10.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.367+0000 7f66a85ef640 1 -- 172.21.15.116:0/1146337058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66a0106b80 msgr2=0x7f66a0106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:10.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.367+0000 7f66a85ef640 1 --2- 172.21.15.116:0/1146337058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66a0106b80 0x7f66a0106f60 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f669c009900 tx=0x7f669c02a410 comp rx=0 tx=0).stop 2024-09-12T03:08:10.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.367+0000 7f66a85ef640 1 -- 172.21.15.116:0/1146337058 shutdown_connections 2024-09-12T03:08:10.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.367+0000 7f66a85ef640 1 --2- 172.21.15.116:0/1146337058 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66a0100aa0 0x7f66a010ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:10.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.367+0000 7f66a85ef640 1 --2- 172.21.15.116:0/1146337058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66a0100100 0x7f66a0100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:10.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.367+0000 7f66a85ef640 1 --2- 172.21.15.116:0/1146337058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66a0106b80 0x7f66a0106f60 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:10.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.367+0000 7f66a85ef640 1 -- 172.21.15.116:0/1146337058 >> 172.21.15.116:0/1146337058 conn(0x7f66a0077230 msgr2=0x7f66a00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:10.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.367+0000 7f66a85ef640 1 -- 172.21.15.116:0/1146337058 shutdown_connections 2024-09-12T03:08:10.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.367+0000 7f66a85ef640 1 -- 172.21.15.116:0/1146337058 wait complete. 2024-09-12T03:08:10.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.368+0000 7f66a85ef640 1 Processor -- start 2024-09-12T03:08:10.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.368+0000 7f66a85ef640 1 -- start start 2024-09-12T03:08:10.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.368+0000 7f66a85ef640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66a0100100 0x7f66a01012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:10.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.368+0000 7f66a85ef640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66a0100aa0 0x7f66a0101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:10.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.368+0000 7f66a85ef640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66a0106b80 0x7f66a0101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:10.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.368+0000 7f66a85ef640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f66a0112050 con 0x7f66a0100aa0 2024-09-12T03:08:10.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.368+0000 7f66a85ef640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f66a0111ed0 con 0x7f66a0106b80 2024-09-12T03:08:10.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.368+0000 7f66a85ef640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f66a01121d0 con 0x7f66a0100100 2024-09-12T03:08:10.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.368+0000 7f66a5b8a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66a0100aa0 0x7f66a0101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:10.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.368+0000 7f66a5b8a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66a0100aa0 0x7f66a0101820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45730/0 (socket says 172.21.15.116:45730) 2024-09-12T03:08:10.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.368+0000 7f66a5b8a640 1 -- 172.21.15.116:0/1069754536 learned_addr learned my addr 172.21.15.116:0/1069754536 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:10.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.368+0000 7f66a638b640 1 --2- 172.21.15.116:0/1069754536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66a0100100 0x7f66a01012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:10.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.368+0000 7f66a6b8c640 1 --2- 172.21.15.116:0/1069754536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66a0106b80 0x7f66a0101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:10.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.368+0000 7f66a5b8a640 1 -- 172.21.15.116:0/1069754536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66a0100100 msgr2=0x7f66a01012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:10.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.368+0000 7f66a5b8a640 1 --2- 172.21.15.116:0/1069754536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66a0100100 0x7f66a01012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:10.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.368+0000 7f66a5b8a640 1 -- 172.21.15.116:0/1069754536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66a0106b80 msgr2=0x7f66a0101d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:10.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.368+0000 7f66a5b8a640 1 --2- 172.21.15.116:0/1069754536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66a0106b80 0x7f66a0101d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:10.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.368+0000 7f66a5b8a640 1 -- 172.21.15.116:0/1069754536 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a01a6990 con 0x7f66a0100aa0 2024-09-12T03:08:10.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.368+0000 7f66a6b8c640 1 --2- 172.21.15.116:0/1069754536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66a0106b80 0x7f66a0101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:10.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.369+0000 7f66a5b8a640 1 --2- 172.21.15.116:0/1069754536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66a0100aa0 0x7f66a0101820 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f669000e9f0 tx=0x7f669000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:10.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.369+0000 7f66977fe640 1 -- 172.21.15.116:0/1069754536 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669000cd80 con 0x7f66a0100aa0 2024-09-12T03:08:10.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.369+0000 7f66977fe640 1 -- 172.21.15.116:0/1069754536 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6690005500 con 0x7f66a0100aa0 2024-09-12T03:08:10.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.369+0000 7f66a85ef640 1 -- 172.21.15.116:0/1069754536 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a01a6c80 con 0x7f66a0100aa0 2024-09-12T03:08:10.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.369+0000 7f66977fe640 1 -- 172.21.15.116:0/1069754536 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690010690 con 0x7f66a0100aa0 2024-09-12T03:08:10.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.369+0000 7f66a85ef640 1 -- 172.21.15.116:0/1069754536 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a01a7170 con 0x7f66a0100aa0 2024-09-12T03:08:10.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.370+0000 7f66a85ef640 1 -- 172.21.15.116:0/1069754536 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66a0100560 con 0x7f66a0100aa0 2024-09-12T03:08:10.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.371+0000 7f66977fe640 1 -- 172.21.15.116:0/1069754536 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f66900046c0 con 0x7f66a0100aa0 2024-09-12T03:08:10.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.372+0000 7f66977fe640 1 --2- 172.21.15.116:0/1069754536 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f668c0a2b80 0x7f668c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:10.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.372+0000 7f66a638b640 1 --2- 172.21.15.116:0/1069754536 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f668c0a2b80 0x7f668c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:10.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.372+0000 7f66977fe640 1 -- 172.21.15.116:0/1069754536 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f66900cf670 con 0x7f66a0100aa0 2024-09-12T03:08:10.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.372+0000 7f66a638b640 1 --2- 172.21.15.116:0/1069754536 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f668c0a2b80 0x7f668c0a5040 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f669c009230 tx=0x7f669c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:10.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.375+0000 7f66977fe640 1 -- 172.21.15.116:0/1069754536 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f66900541b0 con 0x7f66a0100aa0 2024-09-12T03:08:10.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:10 smithi116 ceph-mon[31410]: pgmap v493: 97 pgs: 97 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:08:10.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.586+0000 7f66a85ef640 1 -- 172.21.15.116:0/1069754536 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66a0102810 con 0x7f66a0100aa0 2024-09-12T03:08:10.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.587+0000 7f66977fe640 1 -- 172.21.15.116:0/1069754536 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f66a0102810 con 0x7f66a0100aa0 2024-09-12T03:08:10.588 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:08:10.588 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:08:10.588 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:08:10.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.589+0000 7f66a85ef640 1 -- 172.21.15.116:0/1069754536 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f668c0a2b80 msgr2=0x7f668c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:10.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.589+0000 7f66a85ef640 1 --2- 172.21.15.116:0/1069754536 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f668c0a2b80 0x7f668c0a5040 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f669c009230 tx=0x7f669c0058d0 comp rx=0 tx=0).stop 2024-09-12T03:08:10.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.589+0000 7f66a85ef640 1 -- 172.21.15.116:0/1069754536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66a0100aa0 msgr2=0x7f66a0101820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:10.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.589+0000 7f66a85ef640 1 --2- 172.21.15.116:0/1069754536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66a0100aa0 0x7f66a0101820 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f669000e9f0 tx=0x7f669000edf0 comp rx=0 tx=0).stop 2024-09-12T03:08:10.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.589+0000 7f66a85ef640 1 -- 172.21.15.116:0/1069754536 shutdown_connections 2024-09-12T03:08:10.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.589+0000 7f66a85ef640 1 --2- 172.21.15.116:0/1069754536 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f668c0a2b80 0x7f668c0a5040 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:10.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.589+0000 7f66a85ef640 1 --2- 172.21.15.116:0/1069754536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66a0106b80 0x7f66a0101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:10.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.589+0000 7f66a85ef640 1 --2- 172.21.15.116:0/1069754536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66a0100aa0 0x7f66a0101820 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:10.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.589+0000 7f66a85ef640 1 --2- 172.21.15.116:0/1069754536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66a0100100 0x7f66a01012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:10.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.589+0000 7f66a85ef640 1 -- 172.21.15.116:0/1069754536 >> 172.21.15.116:0/1069754536 conn(0x7f66a0077230 msgr2=0x7f66a0104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:10.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.589+0000 7f66a85ef640 1 -- 172.21.15.116:0/1069754536 shutdown_connections 2024-09-12T03:08:10.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:10.589+0000 7f66a85ef640 1 -- 172.21.15.116:0/1069754536 wait complete. 2024-09-12T03:08:10.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:10 smithi183 ceph-mon[35535]: pgmap v493: 97 pgs: 97 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:08:10.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:10 smithi177 ceph-mon[35305]: pgmap v493: 97 pgs: 97 active+clean; 2.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:08:11.357 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:08:11.605 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:08:11.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:11 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1069754536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:08:11.726 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:11 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1069754536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:08:11.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:11 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1069754536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:08:12.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.119+0000 7f71ffb83640 1 -- 172.21.15.116:0/4027066514 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71f8101420 msgr2=0x7f71f810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:12.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.119+0000 7f71ffb83640 1 --2- 172.21.15.116:0/4027066514 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71f8101420 0x7f71f810d950 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f71f0009cd0 tx=0x7f71f002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.119+0000 7f71ffb83640 1 -- 172.21.15.116:0/4027066514 shutdown_connections 2024-09-12T03:08:12.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.119+0000 7f71ffb83640 1 --2- 172.21.15.116:0/4027066514 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71f8101420 0x7f71f810d950 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.119+0000 7f71ffb83640 1 --2- 172.21.15.116:0/4027066514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71f8100a80 0x7f71f8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.119+0000 7f71ffb83640 1 --2- 172.21.15.116:0/4027066514 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71f8106a80 0x7f71f8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.119+0000 7f71ffb83640 1 -- 172.21.15.116:0/4027066514 >> 172.21.15.116:0/4027066514 conn(0x7f71f80fc7a0 msgr2=0x7f71f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:12.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.119+0000 7f71ffb83640 1 -- 172.21.15.116:0/4027066514 shutdown_connections 2024-09-12T03:08:12.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.119+0000 7f71ffb83640 1 -- 172.21.15.116:0/4027066514 wait complete. 2024-09-12T03:08:12.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.120+0000 7f71ffb83640 1 Processor -- start 2024-09-12T03:08:12.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.120+0000 7f71ffb83640 1 -- start start 2024-09-12T03:08:12.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.120+0000 7f71ffb83640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71f8100a80 0x7f71f810e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:12.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.120+0000 7f71ffb83640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71f8101420 0x7f71f810e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:12.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.120+0000 7f71ffb83640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71f8106a80 0x7f71f810ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:12.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.120+0000 7f71ffb83640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f71f8111fa0 con 0x7f71f8106a80 2024-09-12T03:08:12.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.120+0000 7f71ffb83640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f71f8111e20 con 0x7f71f8101420 2024-09-12T03:08:12.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.120+0000 7f71ffb83640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71f8112120 con 0x7f71f8100a80 2024-09-12T03:08:12.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.120+0000 7f71fe120640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71f8106a80 0x7f71f810ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:12.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.120+0000 7f71fe120640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71f8106a80 0x7f71f810ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57642/0 (socket says 172.21.15.116:57642) 2024-09-12T03:08:12.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.120+0000 7f71fe120640 1 -- 172.21.15.116:0/3842660004 learned_addr learned my addr 172.21.15.116:0/3842660004 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:12.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.120+0000 7f71fd91f640 1 --2- 172.21.15.116:0/3842660004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71f8100a80 0x7f71f810e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:12.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.120+0000 7f71fe120640 1 -- 172.21.15.116:0/3842660004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71f8100a80 msgr2=0x7f71f810e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:12.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.120+0000 7f71fe120640 1 --2- 172.21.15.116:0/3842660004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71f8100a80 0x7f71f810e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.120+0000 7f71fd11e640 1 --2- 172.21.15.116:0/3842660004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71f8101420 0x7f71f810e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:12.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.120+0000 7f71fe120640 1 -- 172.21.15.116:0/3842660004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71f8101420 msgr2=0x7f71f810e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:12.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.120+0000 7f71fe120640 1 --2- 172.21.15.116:0/3842660004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71f8101420 0x7f71f810e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.120+0000 7f71fe120640 1 -- 172.21.15.116:0/3842660004 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f81a6920 con 0x7f71f8106a80 2024-09-12T03:08:12.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.120+0000 7f71fd11e640 1 --2- 172.21.15.116:0/3842660004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71f8101420 0x7f71f810e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:08:12.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.120+0000 7f71fe120640 1 --2- 172.21.15.116:0/3842660004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71f8106a80 0x7f71f810ee60 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f71f0006010 tx=0x7f71f0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:12.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.121+0000 7f71eaffd640 1 -- 172.21.15.116:0/3842660004 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f0004450 con 0x7f71f8106a80 2024-09-12T03:08:12.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.121+0000 7f71ffb83640 1 -- 172.21.15.116:0/3842660004 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f81a6bb0 con 0x7f71f8106a80 2024-09-12T03:08:12.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.121+0000 7f71eaffd640 1 -- 172.21.15.116:0/3842660004 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f0003e10 con 0x7f71f8106a80 2024-09-12T03:08:12.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.121+0000 7f71eaffd640 1 -- 172.21.15.116:0/3842660004 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f00466e0 con 0x7f71f8106a80 2024-09-12T03:08:12.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.121+0000 7f71ffb83640 1 -- 172.21.15.116:0/3842660004 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f81a7140 con 0x7f71f8106a80 2024-09-12T03:08:12.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.124+0000 7f71eaffd640 1 -- 172.21.15.116:0/3842660004 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f71f00305a0 con 0x7f71f8106a80 2024-09-12T03:08:12.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.124+0000 7f7201f5c740 1 Processor -- start 2024-09-12T03:08:12.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.124+0000 7f7201f5c740 1 -- start start 2024-09-12T03:08:12.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.124+0000 7f7201f5c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624706f5b40 0x5624706f4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:12.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.124+0000 7f7201f5c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624706f45d0 0x5624706f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:12.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.124+0000 7f7201f5c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624706ff370 0x562470701760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:12.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.124+0000 7f7201f5c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56247065a2f0 con 0x5624706ff370 2024-09-12T03:08:12.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.124+0000 7f7201f5c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562470601cf0 con 0x5624706f45d0 2024-09-12T03:08:12.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.124+0000 7f7201f5c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5624706562b0 con 0x5624706f5b40 2024-09-12T03:08:12.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.124+0000 7f71fd11e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624706ff370 0x562470701760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:12.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.124+0000 7f71fd11e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624706ff370 0x562470701760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57648/0 (socket says 172.21.15.116:57648) 2024-09-12T03:08:12.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.124+0000 7f71fd11e640 1 -- 172.21.15.116:0/1608588277 learned_addr learned my addr 172.21.15.116:0/1608588277 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:12.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.124+0000 7f71fd11e640 1 --2- 172.21.15.116:0/1608588277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624706f5b40 0x5624706f4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:12.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.124+0000 7f71fd91f640 1 --2- 172.21.15.116:0/1608588277 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624706f45d0 0x5624706f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:12.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.124+0000 7f71eaffd640 1 --2- 172.21.15.116:0/3842660004 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f71e40a2b80 0x7f71e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:12.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.124+0000 7f71fd11e640 1 -- 172.21.15.116:0/1608588277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624706f5b40 msgr2=0x5624706f4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:12.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.124+0000 7f71fd11e640 1 --2- 172.21.15.116:0/1608588277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624706f5b40 0x5624706f4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.124+0000 7f71fd11e640 1 -- 172.21.15.116:0/1608588277 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624706f45d0 msgr2=0x5624706f4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:12.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.124+0000 7f71fd11e640 1 --2- 172.21.15.116:0/1608588277 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624706f45d0 0x5624706f4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.125+0000 7f71fd11e640 1 -- 172.21.15.116:0/1608588277 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624705e63a0 con 0x5624706ff370 2024-09-12T03:08:12.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.125+0000 7f71fd91f640 1 --2- 172.21.15.116:0/1608588277 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624706f45d0 0x5624706f4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:08:12.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.125+0000 7f71fe120640 1 --2- 172.21.15.116:0/3842660004 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f71e40a2b80 0x7f71e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:12.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.125+0000 7f71fd11e640 1 --2- 172.21.15.116:0/1608588277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624706ff370 0x562470701760 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f71f4014510 tx=0x7f71f400d5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3850c6179cf449c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:12.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.125+0000 7f71eaffd640 1 -- 172.21.15.116:0/3842660004 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f71f002e030 con 0x7f71f8106a80 2024-09-12T03:08:12.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.125+0000 7f71e3fff640 1 -- 172.21.15.116:0/1608588277 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f4012710 con 0x5624706ff370 2024-09-12T03:08:12.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.125+0000 7f71e3fff640 1 -- 172.21.15.116:0/1608588277 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f401a070 con 0x5624706ff370 2024-09-12T03:08:12.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.125+0000 7f71e3fff640 1 -- 172.21.15.116:0/1608588277 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f4011b90 con 0x5624706ff370 2024-09-12T03:08:12.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.126+0000 7f7201f5c740 1 -- 172.21.15.116:0/1608588277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624706ff370 msgr2=0x562470701760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:12.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.126+0000 7f7201f5c740 1 --2- 172.21.15.116:0/1608588277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624706ff370 0x562470701760 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f71f4014510 tx=0x7f71f400d5a0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.126+0000 7f71fe120640 1 --2- 172.21.15.116:0/3842660004 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f71e40a2b80 0x7f71e40a5040 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f71f00b94c0 tx=0x7f71f0087c30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:12.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.126+0000 7f7201f5c740 1 -- 172.21.15.116:0/1608588277 shutdown_connections 2024-09-12T03:08:12.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.126+0000 7f7201f5c740 1 --2- 172.21.15.116:0/1608588277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624706ff370 0x562470701760 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.126+0000 7f7201f5c740 1 --2- 172.21.15.116:0/1608588277 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624706f45d0 0x5624706f4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.126+0000 7f7201f5c740 1 --2- 172.21.15.116:0/1608588277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624706f5b40 0x5624706f4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.126+0000 7f7201f5c740 1 -- 172.21.15.116:0/1608588277 >> 172.21.15.116:0/1608588277 conn(0x5624706bd310 msgr2=0x5624706f0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:12.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.126+0000 7f7201f5c740 1 -- 172.21.15.116:0/1608588277 shutdown_connections 2024-09-12T03:08:12.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.126+0000 7f7201f5c740 1 -- 172.21.15.116:0/1608588277 wait complete. 2024-09-12T03:08:12.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.126+0000 7f7201f5c740 1 -- start start 2024-09-12T03:08:12.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.126+0000 7f7201f5c740 1 Processor -- start 2024-09-12T03:08:12.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.127+0000 7f7201f5c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624706f45d0 0x5624706fed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:12.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.127+0000 7f7201f5c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624706f5b40 0x5624706fbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:12.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.127+0000 7f7201f5c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624706ff370 0x5624706fc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:12.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.127+0000 7f7201f5c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562470702070 con 0x5624706ff370 2024-09-12T03:08:12.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.127+0000 7f7201f5c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562470615060 con 0x5624706f45d0 2024-09-12T03:08:12.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.127+0000 7f7201f5c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5624707021f0 con 0x5624706f5b40 2024-09-12T03:08:12.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.127+0000 7f7201f5c740 10 client.0 _ll_register_callbacks cb 0x5624703688c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:12.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.127+0000 7f71fd11e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624706ff370 0x5624706fc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:12.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.127+0000 7f71fd11e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624706ff370 0x5624706fc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57652/0 (socket says 172.21.15.116:57652) 2024-09-12T03:08:12.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.127+0000 7f71fd11e640 1 -- 172.21.15.116:0/3279221467 learned_addr learned my addr 172.21.15.116:0/3279221467 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:12.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.127+0000 7f71fd11e640 1 --2- 172.21.15.116:0/3279221467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624706f45d0 0x5624706fed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:12.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.127+0000 7f71fd91f640 1 --2- 172.21.15.116:0/3279221467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624706f5b40 0x5624706fbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:12.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.127+0000 7f71fd11e640 1 -- 172.21.15.116:0/3279221467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624706f5b40 msgr2=0x5624706fbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:12.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.127+0000 7f71fd11e640 1 --2- 172.21.15.116:0/3279221467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624706f5b40 0x5624706fbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.127+0000 7f71fd11e640 1 -- 172.21.15.116:0/3279221467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624706f45d0 msgr2=0x5624706fed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:12.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.127+0000 7f71fd11e640 1 --2- 172.21.15.116:0/3279221467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624706f45d0 0x5624706fed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.127+0000 7f71fd11e640 1 -- 172.21.15.116:0/3279221467 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624703cb3c0 con 0x5624706ff370 2024-09-12T03:08:12.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.127+0000 7f71fd11e640 1 --2- 172.21.15.116:0/3279221467 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624706ff370 0x5624706fc5d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f71f4001eb0 tx=0x7f71f4016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4a34c5da775334 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:12.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.127+0000 7f71e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:12.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.127+0000 7f71e17fa640 1 -- 172.21.15.116:0/3279221467 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f4025020 con 0x5624706ff370 2024-09-12T03:08:12.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.127+0000 7f71e17fa640 1 -- 172.21.15.116:0/3279221467 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f71f4012510 con 0x5624706ff370 2024-09-12T03:08:12.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.127+0000 7f7201f5c740 1 -- 172.21.15.116:0/3279221467 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71f401a070 con 0x5624706ff370 2024-09-12T03:08:12.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.128+0000 7f71e17fa640 1 -- 172.21.15.116:0/3279221467 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f71f4005020 con 0x5624706ff370 2024-09-12T03:08:12.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.128+0000 7f7201f5c740 10 client.16596 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:12.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.128+0000 7f7201f5c740 1 -- 172.21.15.116:0/3279221467 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56247070da70 con 0x5624706ff370 2024-09-12T03:08:12.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.128+0000 7f71e17fa640 1 -- 172.21.15.116:0/3279221467 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f401a070 con 0x5624706ff370 2024-09-12T03:08:12.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.128+0000 7f71e17fa640 1 -- 172.21.15.116:0/3279221467 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f71f4030420 con 0x5624706ff370 2024-09-12T03:08:12.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.128+0000 7f7201f5c740 10 client.16596 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:12.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.128+0000 7f7201f5c740 10 client.16596 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:08:12.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.128+0000 7f7201f5c740 20 client.16596 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:12.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.128+0000 7f7201f5c740 1 --2- 172.21.15.116:0/3279221467 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x56247070ee70 0x562470711260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:12.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.128+0000 7f7201f5c740 4 client.16596 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:08:12.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.128+0000 7f7201f5c740 1 -- 172.21.15.116:0/3279221467 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562470667860 con 0x56247070ee70 2024-09-12T03:08:12.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.128+0000 7f71fd91f640 1 --2- 172.21.15.116:0/3279221467 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x56247070ee70 0x562470711260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:12.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.129+0000 7f71e17fa640 10 client.16596 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:08:12.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.129+0000 7f71fd91f640 1 --2- 172.21.15.116:0/3279221467 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x56247070ee70 0x562470711260 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3fa21340d4b4985c server_cookie=b5adf19506e9db9e in_seq=0 out_seq=0 2024-09-12T03:08:12.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.130+0000 7f71e17fa640 1 -- 172.21.15.116:0/3279221467 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562470667860 con 0x56247070ee70 2024-09-12T03:08:12.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.130+0000 7f71e17fa640 10 client.16596 handle_command_reply: tid=0 2024-09-12T03:08:12.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.130+0000 7f7201f5c740 1 client.16596 shutdown 2024-09-12T03:08:12.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.130+0000 7f7201f5c740 1 -- 172.21.15.116:0/3279221467 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624706ff370 msgr2=0x5624706fc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:12.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.130+0000 7f7201f5c740 1 --2- 172.21.15.116:0/3279221467 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624706ff370 0x5624706fc5d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f71f4001eb0 tx=0x7f71f4016040 comp rx=0 tx=0).stop 2024-09-12T03:08:12.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.130+0000 7f7201f5c740 1 -- 172.21.15.116:0/3279221467 shutdown_connections 2024-09-12T03:08:12.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.130+0000 7f7201f5c740 1 --2- 172.21.15.116:0/3279221467 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x56247070ee70 0x562470711260 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.130+0000 7f7201f5c740 1 --2- 172.21.15.116:0/3279221467 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624706ff370 0x5624706fc5d0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.130+0000 7f7201f5c740 1 --2- 172.21.15.116:0/3279221467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624706f5b40 0x5624706fbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.130+0000 7f7201f5c740 1 --2- 172.21.15.116:0/3279221467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624706f45d0 0x5624706fed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.130+0000 7f71e17fa640 0 client.16596 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:08:12.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.130+0000 7f7201f5c740 1 -- 172.21.15.116:0/3279221467 >> 172.21.15.116:0/3279221467 conn(0x5624706bd310 msgr2=0x5624706f6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:12.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.130+0000 7f7201f5c740 1 -- 172.21.15.116:0/3279221467 shutdown_connections 2024-09-12T03:08:12.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.130+0000 7f7201f5c740 1 -- 172.21.15.116:0/3279221467 wait complete. 2024-09-12T03:08:12.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.130+0000 7f7201f5c740 20 client.16596 trim_cache size 0 max 16384 2024-09-12T03:08:12.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.143+0000 7f7201f5c740 1 Processor -- start 2024-09-12T03:08:12.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.143+0000 7f7201f5c740 1 -- start start 2024-09-12T03:08:12.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.143+0000 7f7201f5c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624706f45d0 0x56247070b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:12.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.143+0000 7f7201f5c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624706f5b40 0x5624706fafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:12.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.143+0000 7f7201f5c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624706ff370 0x562470716800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:12.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.143+0000 7f7201f5c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f71f81b1f10 con 0x5624706ff370 2024-09-12T03:08:12.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.143+0000 7f7201f5c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f71f81af7e0 con 0x5624706f45d0 2024-09-12T03:08:12.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.143+0000 7f7201f5c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71f81ad650 con 0x5624706f5b40 2024-09-12T03:08:12.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.143+0000 7f71fd11e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624706ff370 0x562470716800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:12.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.143+0000 7f71fd11e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624706ff370 0x562470716800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57662/0 (socket says 172.21.15.116:57662) 2024-09-12T03:08:12.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.143+0000 7f71fd11e640 1 -- 172.21.15.116:0/1835214944 learned_addr learned my addr 172.21.15.116:0/1835214944 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:12.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.144+0000 7f71fd91f640 1 --2- 172.21.15.116:0/1835214944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624706f5b40 0x5624706fafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:12.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.144+0000 7f71fd11e640 1 -- 172.21.15.116:0/1835214944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624706f5b40 msgr2=0x5624706fafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:12.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.144+0000 7f71fd11e640 1 --2- 172.21.15.116:0/1835214944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624706f5b40 0x5624706fafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.144+0000 7f71fd11e640 1 -- 172.21.15.116:0/1835214944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624706f45d0 msgr2=0x56247070b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:12.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.144+0000 7f71fd11e640 1 --2- 172.21.15.116:0/1835214944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624706f45d0 0x56247070b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.144+0000 7f71fd11e640 1 -- 172.21.15.116:0/1835214944 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624706f42d0 con 0x5624706ff370 2024-09-12T03:08:12.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.144+0000 7f71fd11e640 1 --2- 172.21.15.116:0/1835214944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624706ff370 0x562470716800 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f71f400d4c0 tx=0x7f71f4011fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef755abe455f7e54 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:12.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.144+0000 7f71e37fe640 1 -- 172.21.15.116:0/1835214944 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f4025040 con 0x5624706ff370 2024-09-12T03:08:12.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.144+0000 7f71e37fe640 1 -- 172.21.15.116:0/1835214944 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f401a030 con 0x5624706ff370 2024-09-12T03:08:12.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.144+0000 7f71e37fe640 1 -- 172.21.15.116:0/1835214944 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f4030420 con 0x5624706ff370 2024-09-12T03:08:12.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.145+0000 7f7201f5c740 1 -- 172.21.15.116:0/1835214944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624706ff370 msgr2=0x562470716800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:12.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.145+0000 7f7201f5c740 1 --2- 172.21.15.116:0/1835214944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624706ff370 0x562470716800 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f71f400d4c0 tx=0x7f71f4011fd0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.145+0000 7f7201f5c740 1 -- 172.21.15.116:0/1835214944 shutdown_connections 2024-09-12T03:08:12.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.145+0000 7f7201f5c740 1 --2- 172.21.15.116:0/1835214944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624706ff370 0x562470716800 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.145+0000 7f7201f5c740 1 --2- 172.21.15.116:0/1835214944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624706f5b40 0x5624706fafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.145+0000 7f7201f5c740 1 --2- 172.21.15.116:0/1835214944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624706f45d0 0x56247070b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.145+0000 7f7201f5c740 1 -- 172.21.15.116:0/1835214944 >> 172.21.15.116:0/1835214944 conn(0x5624706bd310 msgr2=0x5624706fa8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:12.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.145+0000 7f7201f5c740 1 -- 172.21.15.116:0/1835214944 shutdown_connections 2024-09-12T03:08:12.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.145+0000 7f7201f5c740 1 -- 172.21.15.116:0/1835214944 wait complete. 2024-09-12T03:08:12.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.145+0000 7f7201f5c740 1 -- start start 2024-09-12T03:08:12.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.145+0000 7f7201f5c740 1 Processor -- start 2024-09-12T03:08:12.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.146+0000 7f7201f5c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624706f45d0 0x562470707a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:12.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.146+0000 7f7201f5c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624706f5b40 0x5624706f8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:12.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.146+0000 7f7201f5c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624706ff370 0x5624706f9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:12.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.146+0000 7f7201f5c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f71f81adca0 con 0x5624706f5b40 2024-09-12T03:08:12.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.146+0000 7f7201f5c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f71f81af110 con 0x5624706ff370 2024-09-12T03:08:12.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.146+0000 7f7201f5c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71f81ae370 con 0x5624706f45d0 2024-09-12T03:08:12.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.146+0000 7f7201f5c740 10 client.0 _ll_register_callbacks cb 0x7f71f4011550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:12.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.146+0000 7f71fd91f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624706f5b40 0x5624706f8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:12.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.146+0000 7f71fd91f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624706f5b40 0x5624706f8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57672/0 (socket says 172.21.15.116:57672) 2024-09-12T03:08:12.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.146+0000 7f71fd91f640 1 -- 172.21.15.116:0/3665484805 learned_addr learned my addr 172.21.15.116:0/3665484805 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:12.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.146+0000 7f71fd11e640 1 --2- 172.21.15.116:0/3665484805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624706f45d0 0x562470707a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:12.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.146+0000 7f71fd11e640 1 --2- 172.21.15.116:0/3665484805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624706ff370 0x5624706f9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:12.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.146+0000 7f71fd91f640 1 -- 172.21.15.116:0/3665484805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624706f45d0 msgr2=0x562470707a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:12.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.146+0000 7f71fd91f640 1 --2- 172.21.15.116:0/3665484805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624706f45d0 0x562470707a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.146+0000 7f71fd91f640 1 -- 172.21.15.116:0/3665484805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624706ff370 msgr2=0x5624706f9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:12.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.146+0000 7f71fd91f640 1 --2- 172.21.15.116:0/3665484805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624706ff370 0x5624706f9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.146+0000 7f71fd91f640 1 -- 172.21.15.116:0/3665484805 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624706f9830 con 0x5624706f5b40 2024-09-12T03:08:12.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.146+0000 7f71fd11e640 1 --2- 172.21.15.116:0/3665484805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624706ff370 0x5624706f9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:12.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.146+0000 7f71fd91f640 1 --2- 172.21.15.116:0/3665484805 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624706f5b40 0x5624706f8c50 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f71ec009850 tx=0x7f71ec015420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34801c213b6dff56 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:12.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.146+0000 7f71e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:12.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.146+0000 7f71e17fa640 1 -- 172.21.15.116:0/3665484805 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec00f040 con 0x5624706f5b40 2024-09-12T03:08:12.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.146+0000 7f71e17fa640 1 -- 172.21.15.116:0/3665484805 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f71ec024480 con 0x5624706f5b40 2024-09-12T03:08:12.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.146+0000 7f7201f5c740 1 -- 172.21.15.116:0/3665484805 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71f401a030 con 0x5624706f5b40 2024-09-12T03:08:12.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.147+0000 7f71e17fa640 1 -- 172.21.15.116:0/3665484805 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f71ec02b070 con 0x5624706f5b40 2024-09-12T03:08:12.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.147+0000 7f7201f5c740 10 client.16602 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:12.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.147+0000 7f7201f5c740 1 -- 172.21.15.116:0/3665484805 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5624707796e0 con 0x5624706f5b40 2024-09-12T03:08:12.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.147+0000 7f71e17fa640 1 -- 172.21.15.116:0/3665484805 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f401a030 con 0x5624706f5b40 2024-09-12T03:08:12.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.147+0000 7f71e17fa640 1 -- 172.21.15.116:0/3665484805 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f71ec02f8b0 con 0x5624706f5b40 2024-09-12T03:08:12.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.147+0000 7f7201f5c740 10 client.16602 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:12.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.147+0000 7f7201f5c740 10 client.16602 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:08:12.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.147+0000 7f7201f5c740 20 client.16602 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:12.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.147+0000 7f7201f5c740 1 --2- 172.21.15.116:0/3665484805 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x562470779db0 0x56247077c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:12.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.147+0000 7f7201f5c740 4 client.16602 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:08:12.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.147+0000 7f7201f5c740 1 -- 172.21.15.116:0/3665484805 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562470661d70 con 0x562470779db0 2024-09-12T03:08:12.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.147+0000 7f71fd11e640 1 --2- 172.21.15.116:0/3665484805 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x562470779db0 0x56247077c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:12.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.148+0000 7f71e17fa640 10 client.16602 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:08:12.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.148+0000 7f71fd11e640 1 --2- 172.21.15.116:0/3665484805 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x562470779db0 0x56247077c1d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c8004fbef8e927da server_cookie=7c6abfe6171033ff in_seq=0 out_seq=0 2024-09-12T03:08:12.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.148+0000 7f71e17fa640 1 -- 172.21.15.116:0/3665484805 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562470661d70 con 0x562470779db0 2024-09-12T03:08:12.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.148+0000 7f71e17fa640 10 client.16602 handle_command_reply: tid=0 2024-09-12T03:08:12.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.148+0000 7f7201f5c740 1 client.16602 shutdown 2024-09-12T03:08:12.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.148+0000 7f7201f5c740 1 -- 172.21.15.116:0/3665484805 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624706f5b40 msgr2=0x5624706f8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:12.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.148+0000 7f7201f5c740 1 --2- 172.21.15.116:0/3665484805 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624706f5b40 0x5624706f8c50 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f71ec009850 tx=0x7f71ec015420 comp rx=0 tx=0).stop 2024-09-12T03:08:12.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.148+0000 7f7201f5c740 1 -- 172.21.15.116:0/3665484805 shutdown_connections 2024-09-12T03:08:12.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.148+0000 7f7201f5c740 1 --2- 172.21.15.116:0/3665484805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624706ff370 0x5624706f9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.148+0000 7f7201f5c740 1 --2- 172.21.15.116:0/3665484805 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x562470779db0 0x56247077c1d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.148+0000 7f7201f5c740 1 --2- 172.21.15.116:0/3665484805 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624706f5b40 0x5624706f8c50 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.148+0000 7f7201f5c740 1 --2- 172.21.15.116:0/3665484805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624706f45d0 0x562470707a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.148+0000 7f71e17fa640 0 client.16602 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:08:12.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.148+0000 7f7201f5c740 1 -- 172.21.15.116:0/3665484805 >> 172.21.15.116:0/3665484805 conn(0x5624706bd310 msgr2=0x5624706fb0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:12.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.148+0000 7f7201f5c740 1 -- 172.21.15.116:0/3665484805 shutdown_connections 2024-09-12T03:08:12.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.148+0000 7f7201f5c740 1 -- 172.21.15.116:0/3665484805 wait complete. 2024-09-12T03:08:12.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.148+0000 7f7201f5c740 20 client.16602 trim_cache size 0 max 16384 2024-09-12T03:08:12.157 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:08:12.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.149+0000 7f71ffb83640 1 -- 172.21.15.116:0/3842660004 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f71e40a2b80 msgr2=0x7f71e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:12.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.149+0000 7f71ffb83640 1 --2- 172.21.15.116:0/3842660004 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f71e40a2b80 0x7f71e40a5040 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f71f00b94c0 tx=0x7f71f0087c30 comp rx=0 tx=0).stop 2024-09-12T03:08:12.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.149+0000 7f71ffb83640 1 -- 172.21.15.116:0/3842660004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71f8106a80 msgr2=0x7f71f810ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:12.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.149+0000 7f71ffb83640 1 --2- 172.21.15.116:0/3842660004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71f8106a80 0x7f71f810ee60 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f71f0006010 tx=0x7f71f0003c20 comp rx=0 tx=0).stop 2024-09-12T03:08:12.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.149+0000 7f71ffb83640 1 -- 172.21.15.116:0/3842660004 shutdown_connections 2024-09-12T03:08:12.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.149+0000 7f71ffb83640 1 --2- 172.21.15.116:0/3842660004 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f71e40a2b80 0x7f71e40a5040 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.149+0000 7f71ffb83640 1 --2- 172.21.15.116:0/3842660004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71f8106a80 0x7f71f810ee60 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.149+0000 7f71ffb83640 1 --2- 172.21.15.116:0/3842660004 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71f8101420 0x7f71f810e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.149+0000 7f71ffb83640 1 --2- 172.21.15.116:0/3842660004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71f8100a80 0x7f71f810e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:12.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.149+0000 7f71ffb83640 1 -- 172.21.15.116:0/3842660004 >> 172.21.15.116:0/3842660004 conn(0x7f71f80fc7a0 msgr2=0x7f71f80fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:12.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.149+0000 7f71ffb83640 1 -- 172.21.15.116:0/3842660004 shutdown_connections 2024-09-12T03:08:12.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:12.149+0000 7f71ffb83640 1 -- 172.21.15.116:0/3842660004 wait complete. 2024-09-12T03:08:12.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:12 smithi183 ceph-mon[35535]: pgmap v494: 97 pgs: 97 active+clean; 2.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:08:12.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:12 smithi177 ceph-mon[35305]: pgmap v494: 97 pgs: 97 active+clean; 2.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:08:12.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:12 smithi116 ceph-mon[31410]: pgmap v494: 97 pgs: 97 active+clean; 2.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:08:12.968 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:08:13.215 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:08:13.629 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:13 smithi116 ceph-mon[31410]: pgmap v495: 97 pgs: 97 active+clean; 2.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:08:13.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:13 smithi183 ceph-mon[35535]: pgmap v495: 97 pgs: 97 active+clean; 2.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:08:13.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.708+0000 7fd6d34fb640 1 -- 172.21.15.116:0/648602939 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6cc112b90 msgr2=0x7fd6cc114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:13.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.708+0000 7fd6d34fb640 1 --2- 172.21.15.116:0/648602939 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6cc112b90 0x7fd6cc114f80 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4009ca0 tx=0x7fd6c402a950 comp rx=0 tx=0).stop 2024-09-12T03:08:13.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.708+0000 7fd6d34fb640 1 -- 172.21.15.116:0/648602939 shutdown_connections 2024-09-12T03:08:13.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.708+0000 7fd6d34fb640 1 --2- 172.21.15.116:0/648602939 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6cc112b90 0x7fd6cc114f80 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.708+0000 7fd6d34fb640 1 --2- 172.21.15.116:0/648602939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6cc074eb0 0x7fd6cc112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.708+0000 7fd6d34fb640 1 --2- 172.21.15.116:0/648602939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6cc074590 0x7fd6cc074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.708+0000 7fd6d34fb640 1 -- 172.21.15.116:0/648602939 >> 172.21.15.116:0/648602939 conn(0x7fd6cc0fd8c0 msgr2=0x7fd6cc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:13.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.708+0000 7fd6d34fb640 1 -- 172.21.15.116:0/648602939 shutdown_connections 2024-09-12T03:08:13.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.708+0000 7fd6d34fb640 1 -- 172.21.15.116:0/648602939 wait complete. 2024-09-12T03:08:13.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.709+0000 7fd6d34fb640 1 Processor -- start 2024-09-12T03:08:13.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.709+0000 7fd6d34fb640 1 -- start start 2024-09-12T03:08:13.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.709+0000 7fd6d34fb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6cc074590 0x7fd6cc1a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.709+0000 7fd6d34fb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6cc074eb0 0x7fd6cc1a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.709+0000 7fd6d34fb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6cc112b90 0x7fd6cc19e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.709+0000 7fd6d34fb640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc1185e0 con 0x7fd6cc074590 2024-09-12T03:08:13.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.709+0000 7fd6d34fb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc118460 con 0x7fd6cc112b90 2024-09-12T03:08:13.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.709+0000 7fd6d34fb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc118760 con 0x7fd6cc074eb0 2024-09-12T03:08:13.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.709+0000 7fd6d1297640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6cc074590 0x7fd6cc1a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.710+0000 7fd6d1297640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6cc074590 0x7fd6cc1a4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57678/0 (socket says 172.21.15.116:57678) 2024-09-12T03:08:13.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.710+0000 7fd6d1297640 1 -- 172.21.15.116:0/134152510 learned_addr learned my addr 172.21.15.116:0/134152510 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:13.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.710+0000 7fd6d0a96640 1 --2- 172.21.15.116:0/134152510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6cc074eb0 0x7fd6cc1a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.710+0000 7fd6d1a98640 1 --2- 172.21.15.116:0/134152510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6cc112b90 0x7fd6cc19e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.710+0000 7fd6d1297640 1 -- 172.21.15.116:0/134152510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6cc074eb0 msgr2=0x7fd6cc1a4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:13.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.710+0000 7fd6d1297640 1 --2- 172.21.15.116:0/134152510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6cc074eb0 0x7fd6cc1a4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.710+0000 7fd6d1297640 1 -- 172.21.15.116:0/134152510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6cc112b90 msgr2=0x7fd6cc19e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:13.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.710+0000 7fd6d1297640 1 --2- 172.21.15.116:0/134152510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6cc112b90 0x7fd6cc19e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.710+0000 7fd6d1297640 1 -- 172.21.15.116:0/134152510 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6cc19f030 con 0x7fd6cc074590 2024-09-12T03:08:13.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.710+0000 7fd6d1a98640 1 --2- 172.21.15.116:0/134152510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6cc112b90 0x7fd6cc19e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:13.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.710+0000 7fd6d1297640 1 --2- 172.21.15.116:0/134152510 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6cc074590 0x7fd6cc1a4740 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800b4a0 tx=0x7fd6c800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:13.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.710+0000 7fd6c27fc640 1 -- 172.21.15.116:0/134152510 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8014070 con 0x7fd6cc074590 2024-09-12T03:08:13.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.710+0000 7fd6c27fc640 1 -- 172.21.15.116:0/134152510 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6c8010af0 con 0x7fd6cc074590 2024-09-12T03:08:13.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.710+0000 7fd6d34fb640 1 -- 172.21.15.116:0/134152510 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6cc19f320 con 0x7fd6cc074590 2024-09-12T03:08:13.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.710+0000 7fd6c27fc640 1 -- 172.21.15.116:0/134152510 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8017d10 con 0x7fd6cc074590 2024-09-12T03:08:13.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.710+0000 7fd6d34fb640 1 -- 172.21.15.116:0/134152510 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6cc1ab610 con 0x7fd6cc074590 2024-09-12T03:08:13.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.713+0000 7fd6c27fc640 1 -- 172.21.15.116:0/134152510 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd6c8017070 con 0x7fd6cc074590 2024-09-12T03:08:13.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.713+0000 7fd6d58d4740 1 Processor -- start 2024-09-12T03:08:13.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.713+0000 7fd6d58d4740 1 -- start start 2024-09-12T03:08:13.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.713+0000 7fd6d58d4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56494b661b40 0x56494b660090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.713+0000 7fd6d58d4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56494b6605d0 0x56494b660a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.713+0000 7fd6d58d4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56494b66b370 0x56494b66d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.713+0000 7fd6c27fc640 1 --2- 172.21.15.116:0/134152510 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd6b80a2b80 0x7fd6b80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.713+0000 7fd6d58d4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56494b5c62f0 con 0x56494b661b40 2024-09-12T03:08:13.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.713+0000 7fd6d58d4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56494b56dcf0 con 0x56494b6605d0 2024-09-12T03:08:13.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.713+0000 7fd6d58d4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56494b5c22b0 con 0x56494b66b370 2024-09-12T03:08:13.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.713+0000 7fd6d1a98640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56494b661b40 0x56494b660090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.713+0000 7fd6d1a98640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56494b661b40 0x56494b660090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57684/0 (socket says 172.21.15.116:57684) 2024-09-12T03:08:13.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.714+0000 7fd6d1a98640 1 -- 172.21.15.116:0/1089306164 learned_addr learned my addr 172.21.15.116:0/1089306164 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:13.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.714+0000 7fd6d1297640 1 --2- 172.21.15.116:0/1089306164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56494b6605d0 0x56494b660a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.714+0000 7fd6d1a98640 1 --2- 172.21.15.116:0/134152510 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd6b80a2b80 0x7fd6b80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.714+0000 7fd6d0a96640 1 --2- 172.21.15.116:0/1089306164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56494b66b370 0x56494b66d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.714+0000 7fd6c27fc640 1 -- 172.21.15.116:0/134152510 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd6c80ce3e0 con 0x7fd6cc074590 2024-09-12T03:08:13.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.714+0000 7fd6d1a98640 1 -- 172.21.15.116:0/1089306164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56494b66b370 msgr2=0x56494b66d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:13.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.714+0000 7fd6d1a98640 1 --2- 172.21.15.116:0/1089306164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56494b66b370 0x56494b66d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.714+0000 7fd6d1a98640 1 -- 172.21.15.116:0/1089306164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56494b6605d0 msgr2=0x56494b660a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:13.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.714+0000 7fd6d1a98640 1 --2- 172.21.15.116:0/1089306164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56494b6605d0 0x56494b660a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.714+0000 7fd6d1a98640 1 -- 172.21.15.116:0/1089306164 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56494b5523a0 con 0x56494b661b40 2024-09-12T03:08:13.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.714+0000 7fd6d1a98640 1 --2- 172.21.15.116:0/1089306164 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56494b661b40 0x56494b660090 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4030040 tx=0x7fd6c402f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63055191d848b65f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:13.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.714+0000 7fd6d1a98640 1 --2- 172.21.15.116:0/134152510 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd6b80a2b80 0x7fd6b80a5040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4035000 tx=0x7fd6c4042000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:13.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.714+0000 7fd6b77fe640 1 -- 172.21.15.116:0/1089306164 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c404c070 con 0x56494b661b40 2024-09-12T03:08:13.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.714+0000 7fd6b77fe640 1 -- 172.21.15.116:0/1089306164 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6c4004520 con 0x56494b661b40 2024-09-12T03:08:13.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.714+0000 7fd6b77fe640 1 -- 172.21.15.116:0/1089306164 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4034040 con 0x56494b661b40 2024-09-12T03:08:13.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.715+0000 7fd6d58d4740 1 -- 172.21.15.116:0/1089306164 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56494b661b40 msgr2=0x56494b660090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:13.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.715+0000 7fd6d58d4740 1 --2- 172.21.15.116:0/1089306164 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56494b661b40 0x56494b660090 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4030040 tx=0x7fd6c402f740 comp rx=0 tx=0).stop 2024-09-12T03:08:13.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.715+0000 7fd6d58d4740 1 -- 172.21.15.116:0/1089306164 shutdown_connections 2024-09-12T03:08:13.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.715+0000 7fd6d58d4740 1 --2- 172.21.15.116:0/1089306164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56494b66b370 0x56494b66d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.715+0000 7fd6d58d4740 1 --2- 172.21.15.116:0/1089306164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56494b6605d0 0x56494b660a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.715+0000 7fd6d58d4740 1 --2- 172.21.15.116:0/1089306164 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56494b661b40 0x56494b660090 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.715+0000 7fd6d58d4740 1 -- 172.21.15.116:0/1089306164 >> 172.21.15.116:0/1089306164 conn(0x56494b629310 msgr2=0x56494b65cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:13.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.715+0000 7fd6d58d4740 1 -- 172.21.15.116:0/1089306164 shutdown_connections 2024-09-12T03:08:13.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.715+0000 7fd6d58d4740 1 -- 172.21.15.116:0/1089306164 wait complete. 2024-09-12T03:08:13.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.715+0000 7fd6d58d4740 1 -- start start 2024-09-12T03:08:13.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.715+0000 7fd6d58d4740 1 Processor -- start 2024-09-12T03:08:13.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.716+0000 7fd6d58d4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56494b6605d0 0x56494b66adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.716+0000 7fd6d58d4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56494b661b40 0x56494b668070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.716+0000 7fd6d58d4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56494b66b370 0x56494b668800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.716+0000 7fd6d58d4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56494b66e010 con 0x56494b661b40 2024-09-12T03:08:13.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.716+0000 7fd6d58d4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56494b581060 con 0x56494b66b370 2024-09-12T03:08:13.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.716+0000 7fd6d58d4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56494b66e190 con 0x56494b6605d0 2024-09-12T03:08:13.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.716+0000 7fd6d58d4740 10 client.0 _ll_register_callbacks cb 0x56494b2d48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:13.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.716+0000 7fd6d0a96640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56494b661b40 0x56494b668070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.716+0000 7fd6d0a96640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56494b661b40 0x56494b668070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57694/0 (socket says 172.21.15.116:57694) 2024-09-12T03:08:13.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.716+0000 7fd6d0a96640 1 -- 172.21.15.116:0/1040721639 learned_addr learned my addr 172.21.15.116:0/1040721639 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:13.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.716+0000 7fd6d1a98640 1 --2- 172.21.15.116:0/1040721639 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56494b66b370 0x56494b668800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.716+0000 7fd6d1297640 1 --2- 172.21.15.116:0/1040721639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56494b6605d0 0x56494b66adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.716+0000 7fd6d0a96640 1 -- 172.21.15.116:0/1040721639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56494b6605d0 msgr2=0x56494b66adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:13.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.716+0000 7fd6d0a96640 1 --2- 172.21.15.116:0/1040721639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56494b6605d0 0x56494b66adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.716+0000 7fd6d0a96640 1 -- 172.21.15.116:0/1040721639 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56494b66b370 msgr2=0x56494b668800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:13.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.716+0000 7fd6d0a96640 1 --2- 172.21.15.116:0/1040721639 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56494b66b370 0x56494b668800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.716+0000 7fd6d0a96640 1 -- 172.21.15.116:0/1040721639 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56494b3373c0 con 0x56494b661b40 2024-09-12T03:08:13.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.716+0000 7fd6d1297640 1 --2- 172.21.15.116:0/1040721639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56494b6605d0 0x56494b66adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:13.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.716+0000 7fd6d0a96640 1 --2- 172.21.15.116:0/1040721639 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56494b661b40 0x56494b668070 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00b920 tx=0x7fd6bc00bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57dba185c925a374 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:13.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.716+0000 7fd6b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:13.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.717+0000 7fd6b4ff9640 1 -- 172.21.15.116:0/1040721639 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc003ea0 con 0x56494b661b40 2024-09-12T03:08:13.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.717+0000 7fd6b4ff9640 1 -- 172.21.15.116:0/1040721639 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd6bc01b640 con 0x56494b661b40 2024-09-12T03:08:13.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.717+0000 7fd6d58d4740 1 -- 172.21.15.116:0/1040721639 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6c4004520 con 0x56494b661b40 2024-09-12T03:08:13.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.717+0000 7fd6b4ff9640 1 -- 172.21.15.116:0/1040721639 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd6bc022070 con 0x56494b661b40 2024-09-12T03:08:13.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.717+0000 7fd6d58d4740 10 client.16614 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:13.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.717+0000 7fd6d58d4740 1 -- 172.21.15.116:0/1040721639 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56494b679b80 con 0x56494b661b40 2024-09-12T03:08:13.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.717+0000 7fd6b4ff9640 1 -- 172.21.15.116:0/1040721639 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6c4004520 con 0x56494b661b40 2024-09-12T03:08:13.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.717+0000 7fd6b4ff9640 1 -- 172.21.15.116:0/1040721639 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd6bc019020 con 0x56494b661b40 2024-09-12T03:08:13.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.717+0000 7fd6d58d4740 10 client.16614 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:13.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.717+0000 7fd6d58d4740 10 client.16614 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:08:13.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.717+0000 7fd6d58d4740 20 client.16614 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:13.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.717+0000 7fd6d58d4740 1 --2- 172.21.15.116:0/1040721639 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x56494b67b040 0x56494b67d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.717+0000 7fd6d58d4740 4 client.16614 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:08:13.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.717+0000 7fd6d58d4740 1 -- 172.21.15.116:0/1040721639 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56494b5d3860 con 0x56494b67b040 2024-09-12T03:08:13.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.718+0000 7fd6d1297640 1 --2- 172.21.15.116:0/1040721639 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x56494b67b040 0x56494b67d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.718+0000 7fd6b4ff9640 10 client.16614 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:08:13.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.718+0000 7fd6d1297640 1 --2- 172.21.15.116:0/1040721639 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x56494b67b040 0x56494b67d450 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6276e4dacd7ba3e7 server_cookie=efd41af2dd1deb62 in_seq=0 out_seq=0 2024-09-12T03:08:13.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.719+0000 7fd6b4ff9640 1 -- 172.21.15.116:0/1040721639 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56494b5d3860 con 0x56494b67b040 2024-09-12T03:08:13.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.719+0000 7fd6b4ff9640 10 client.16614 handle_command_reply: tid=0 2024-09-12T03:08:13.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.719+0000 7fd6d58d4740 1 client.16614 shutdown 2024-09-12T03:08:13.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.719+0000 7fd6d58d4740 1 -- 172.21.15.116:0/1040721639 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56494b661b40 msgr2=0x56494b668070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:13.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.719+0000 7fd6d58d4740 1 --2- 172.21.15.116:0/1040721639 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56494b661b40 0x56494b668070 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00b920 tx=0x7fd6bc00bde0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.719+0000 7fd6d58d4740 1 -- 172.21.15.116:0/1040721639 shutdown_connections 2024-09-12T03:08:13.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.719+0000 7fd6d58d4740 1 --2- 172.21.15.116:0/1040721639 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x56494b67b040 0x56494b67d450 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.719+0000 7fd6d58d4740 1 --2- 172.21.15.116:0/1040721639 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56494b66b370 0x56494b668800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.719+0000 7fd6b4ff9640 0 client.16614 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:08:13.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.719+0000 7fd6d58d4740 1 --2- 172.21.15.116:0/1040721639 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56494b661b40 0x56494b668070 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.719+0000 7fd6d58d4740 1 --2- 172.21.15.116:0/1040721639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56494b6605d0 0x56494b66adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.719+0000 7fd6d58d4740 1 -- 172.21.15.116:0/1040721639 >> 172.21.15.116:0/1040721639 conn(0x56494b629310 msgr2=0x56494b662960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:13.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.719+0000 7fd6d58d4740 1 -- 172.21.15.116:0/1040721639 shutdown_connections 2024-09-12T03:08:13.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.719+0000 7fd6d58d4740 1 -- 172.21.15.116:0/1040721639 wait complete. 2024-09-12T03:08:13.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.719+0000 7fd6d58d4740 20 client.16614 trim_cache size 0 max 16384 2024-09-12T03:08:13.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.732+0000 7fd6d58d4740 1 Processor -- start 2024-09-12T03:08:13.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.732+0000 7fd6d58d4740 1 -- start start 2024-09-12T03:08:13.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.732+0000 7fd6d58d4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56494b6605d0 0x56494b669fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.732+0000 7fd6d58d4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56494b661b40 0x56494b663120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.732+0000 7fd6d58d4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56494b66b370 0x56494b663660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.732+0000 7fd6d58d4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc1b64e0 con 0x56494b661b40 2024-09-12T03:08:13.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.732+0000 7fd6d58d4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc1b3db0 con 0x56494b6605d0 2024-09-12T03:08:13.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.732+0000 7fd6d58d4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc1b1c00 con 0x56494b66b370 2024-09-12T03:08:13.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.732+0000 7fd6d0a96640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56494b661b40 0x56494b663120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.732+0000 7fd6d0a96640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56494b661b40 0x56494b663120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57704/0 (socket says 172.21.15.116:57704) 2024-09-12T03:08:13.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.732+0000 7fd6d0a96640 1 -- 172.21.15.116:0/38350255 learned_addr learned my addr 172.21.15.116:0/38350255 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:13.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.732+0000 7fd6d1297640 1 --2- 172.21.15.116:0/38350255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56494b6605d0 0x56494b669fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.732+0000 7fd6d1a98640 1 --2- 172.21.15.116:0/38350255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56494b66b370 0x56494b663660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.732+0000 7fd6d0a96640 1 -- 172.21.15.116:0/38350255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56494b66b370 msgr2=0x56494b663660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:13.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.732+0000 7fd6d0a96640 1 --2- 172.21.15.116:0/38350255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56494b66b370 0x56494b663660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.732+0000 7fd6d0a96640 1 -- 172.21.15.116:0/38350255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56494b6605d0 msgr2=0x56494b669fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:13.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.732+0000 7fd6d0a96640 1 --2- 172.21.15.116:0/38350255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56494b6605d0 0x56494b669fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.732+0000 7fd6d0a96640 1 -- 172.21.15.116:0/38350255 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56494b65c510 con 0x56494b661b40 2024-09-12T03:08:13.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.733+0000 7fd6d0a96640 1 --2- 172.21.15.116:0/38350255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56494b661b40 0x56494b663120 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc004340 tx=0x7fd6bc0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6790c2098324a5b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:13.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.733+0000 7fd6b6ffd640 1 -- 172.21.15.116:0/38350255 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc011640 con 0x56494b661b40 2024-09-12T03:08:13.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.733+0000 7fd6b6ffd640 1 -- 172.21.15.116:0/38350255 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6bc005520 con 0x56494b661b40 2024-09-12T03:08:13.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.733+0000 7fd6b6ffd640 1 -- 172.21.15.116:0/38350255 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc004d50 con 0x56494b661b40 2024-09-12T03:08:13.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.733+0000 7fd6d58d4740 1 -- 172.21.15.116:0/38350255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56494b661b40 msgr2=0x56494b663120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:13.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.733+0000 7fd6d58d4740 1 --2- 172.21.15.116:0/38350255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56494b661b40 0x56494b663120 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc004340 tx=0x7fd6bc0039c0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.733+0000 7fd6d58d4740 1 -- 172.21.15.116:0/38350255 shutdown_connections 2024-09-12T03:08:13.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.734+0000 7fd6d58d4740 1 --2- 172.21.15.116:0/38350255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56494b66b370 0x56494b663660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.734+0000 7fd6d58d4740 1 --2- 172.21.15.116:0/38350255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56494b661b40 0x56494b663120 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.734+0000 7fd6d58d4740 1 --2- 172.21.15.116:0/38350255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56494b6605d0 0x56494b669fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.734+0000 7fd6d58d4740 1 -- 172.21.15.116:0/38350255 >> 172.21.15.116:0/38350255 conn(0x56494b629310 msgr2=0x56494b6698d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:13.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.734+0000 7fd6d58d4740 1 -- 172.21.15.116:0/38350255 shutdown_connections 2024-09-12T03:08:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.734+0000 7fd6d58d4740 1 -- 172.21.15.116:0/38350255 wait complete. 2024-09-12T03:08:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.734+0000 7fd6d58d4740 1 -- start start 2024-09-12T03:08:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.734+0000 7fd6d58d4740 1 Processor -- start 2024-09-12T03:08:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.734+0000 7fd6d58d4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56494b6605d0 0x56494b673420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.734+0000 7fd6d58d4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56494b661b40 0x56494b673960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.734+0000 7fd6d58d4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56494b66b370 0x56494b6703f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.734+0000 7fd6d58d4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc1b2270 con 0x56494b66b370 2024-09-12T03:08:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.735+0000 7fd6d58d4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc1b36e0 con 0x56494b661b40 2024-09-12T03:08:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.735+0000 7fd6d58d4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc1b2940 con 0x56494b6605d0 2024-09-12T03:08:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.735+0000 7fd6d58d4740 10 client.0 _ll_register_callbacks cb 0x7fd6bc004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.735+0000 7fd6d1a98640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56494b66b370 0x56494b6703f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.735+0000 7fd6d1a98640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56494b66b370 0x56494b6703f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57710/0 (socket says 172.21.15.116:57710) 2024-09-12T03:08:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.735+0000 7fd6d1a98640 1 -- 172.21.15.116:0/2115462066 learned_addr learned my addr 172.21.15.116:0/2115462066 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:13.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.735+0000 7fd6d0a96640 1 --2- 172.21.15.116:0/2115462066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56494b661b40 0x56494b673960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.735+0000 7fd6d1297640 1 --2- 172.21.15.116:0/2115462066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56494b6605d0 0x56494b673420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.735+0000 7fd6d1a98640 1 -- 172.21.15.116:0/2115462066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56494b6605d0 msgr2=0x56494b673420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:13.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.735+0000 7fd6d1a98640 1 --2- 172.21.15.116:0/2115462066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56494b6605d0 0x56494b673420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.735+0000 7fd6d1a98640 1 -- 172.21.15.116:0/2115462066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56494b661b40 msgr2=0x56494b673960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:13.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.735+0000 7fd6d1a98640 1 --2- 172.21.15.116:0/2115462066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56494b661b40 0x56494b673960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.735+0000 7fd6d1a98640 1 -- 172.21.15.116:0/2115462066 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56494b670930 con 0x56494b66b370 2024-09-12T03:08:13.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.735+0000 7fd6d1a98640 1 --2- 172.21.15.116:0/2115462066 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56494b66b370 0x56494b6703f0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4009870 tx=0x7fd6c40098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3049d40fbcdbefcf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:13.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.735+0000 7fd6b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:13.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.735+0000 7fd6b4ff9640 1 -- 172.21.15.116:0/2115462066 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c405f060 con 0x56494b66b370 2024-09-12T03:08:13.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.735+0000 7fd6b4ff9640 1 -- 172.21.15.116:0/2115462066 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd6c4003dc0 con 0x56494b66b370 2024-09-12T03:08:13.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.735+0000 7fd6d58d4740 1 -- 172.21.15.116:0/2115462066 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6bc005520 con 0x56494b66b370 2024-09-12T03:08:13.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.736+0000 7fd6b4ff9640 1 -- 172.21.15.116:0/2115462066 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd6c404c040 con 0x56494b66b370 2024-09-12T03:08:13.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.736+0000 7fd6d58d4740 10 client.16620 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:13.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.736+0000 7fd6d58d4740 1 -- 172.21.15.116:0/2115462066 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56494b6e64e0 con 0x56494b66b370 2024-09-12T03:08:13.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.736+0000 7fd6b4ff9640 1 -- 172.21.15.116:0/2115462066 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6bc005520 con 0x56494b66b370 2024-09-12T03:08:13.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.736+0000 7fd6b4ff9640 1 -- 172.21.15.116:0/2115462066 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd6c4030c90 con 0x56494b66b370 2024-09-12T03:08:13.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.736+0000 7fd6d58d4740 10 client.16620 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:13.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.736+0000 7fd6d58d4740 10 client.16620 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:08:13.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.736+0000 7fd6d58d4740 20 client.16620 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:13.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.736+0000 7fd6d58d4740 1 --2- 172.21.15.116:0/2115462066 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x56494b6e6cb0 0x56494b6e90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.736+0000 7fd6d58d4740 4 client.16620 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:08:13.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.736+0000 7fd6d58d4740 1 -- 172.21.15.116:0/2115462066 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56494b5cdd70 con 0x56494b6e6cb0 2024-09-12T03:08:13.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.736+0000 7fd6d1297640 1 --2- 172.21.15.116:0/2115462066 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x56494b6e6cb0 0x56494b6e90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.737+0000 7fd6b4ff9640 10 client.16620 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:08:13.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.737+0000 7fd6d1297640 1 --2- 172.21.15.116:0/2115462066 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x56494b6e6cb0 0x56494b6e90a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=da8bea6945692fa9 server_cookie=39c48bf8c6719f88 in_seq=0 out_seq=0 2024-09-12T03:08:13.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.737+0000 7fd6b4ff9640 1 -- 172.21.15.116:0/2115462066 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56494b5cdd70 con 0x56494b6e6cb0 2024-09-12T03:08:13.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.737+0000 7fd6b4ff9640 10 client.16620 handle_command_reply: tid=0 2024-09-12T03:08:13.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.737+0000 7fd6d58d4740 1 client.16620 shutdown 2024-09-12T03:08:13.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.737+0000 7fd6d58d4740 1 -- 172.21.15.116:0/2115462066 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56494b66b370 msgr2=0x56494b6703f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:13.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.737+0000 7fd6d58d4740 1 --2- 172.21.15.116:0/2115462066 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56494b66b370 0x56494b6703f0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4009870 tx=0x7fd6c40098a0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.737+0000 7fd6d58d4740 1 -- 172.21.15.116:0/2115462066 shutdown_connections 2024-09-12T03:08:13.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.737+0000 7fd6d58d4740 1 --2- 172.21.15.116:0/2115462066 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x56494b6e6cb0 0x56494b6e90a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.737+0000 7fd6d58d4740 1 --2- 172.21.15.116:0/2115462066 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56494b66b370 0x56494b6703f0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.737+0000 7fd6d58d4740 1 --2- 172.21.15.116:0/2115462066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56494b661b40 0x56494b673960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.737+0000 7fd6d58d4740 1 --2- 172.21.15.116:0/2115462066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56494b6605d0 0x56494b673420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.737+0000 7fd6d58d4740 1 -- 172.21.15.116:0/2115462066 >> 172.21.15.116:0/2115462066 conn(0x56494b629310 msgr2=0x56494b66a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:13.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.737+0000 7fd6b4ff9640 0 client.16620 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:08:13.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.737+0000 7fd6d58d4740 1 -- 172.21.15.116:0/2115462066 shutdown_connections 2024-09-12T03:08:13.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.737+0000 7fd6d58d4740 1 -- 172.21.15.116:0/2115462066 wait complete. 2024-09-12T03:08:13.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.737+0000 7fd6d58d4740 20 client.16620 trim_cache size 0 max 16384 2024-09-12T03:08:13.750 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:08:13.751 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:13 smithi177 ceph-mon[35305]: pgmap v495: 97 pgs: 97 active+clean; 2.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:08:13.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.738+0000 7fd6d34fb640 1 -- 172.21.15.116:0/134152510 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd6b80a2b80 msgr2=0x7fd6b80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:13.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.738+0000 7fd6d34fb640 1 --2- 172.21.15.116:0/134152510 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd6b80a2b80 0x7fd6b80a5040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4035000 tx=0x7fd6c4042000 comp rx=0 tx=0).stop 2024-09-12T03:08:13.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.738+0000 7fd6d34fb640 1 -- 172.21.15.116:0/134152510 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6cc074590 msgr2=0x7fd6cc1a4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:13.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.738+0000 7fd6d34fb640 1 --2- 172.21.15.116:0/134152510 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6cc074590 0x7fd6cc1a4740 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800b4a0 tx=0x7fd6c800b960 comp rx=0 tx=0).stop 2024-09-12T03:08:13.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.738+0000 7fd6d34fb640 1 -- 172.21.15.116:0/134152510 shutdown_connections 2024-09-12T03:08:13.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.738+0000 7fd6d34fb640 1 --2- 172.21.15.116:0/134152510 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd6b80a2b80 0x7fd6b80a5040 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.738+0000 7fd6d34fb640 1 --2- 172.21.15.116:0/134152510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6cc112b90 0x7fd6cc19e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.738+0000 7fd6d34fb640 1 --2- 172.21.15.116:0/134152510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6cc074eb0 0x7fd6cc1a4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.738+0000 7fd6d34fb640 1 --2- 172.21.15.116:0/134152510 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6cc074590 0x7fd6cc1a4740 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.738+0000 7fd6d34fb640 1 -- 172.21.15.116:0/134152510 >> 172.21.15.116:0/134152510 conn(0x7fd6cc0fd8c0 msgr2=0x7fd6cc0ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:13.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.738+0000 7fd6d34fb640 1 -- 172.21.15.116:0/134152510 shutdown_connections 2024-09-12T03:08:13.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:13.738+0000 7fd6d34fb640 1 -- 172.21.15.116:0/134152510 wait complete. 2024-09-12T03:08:14.604 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:08:14.858 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:08:15.148 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:14 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:08:15.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:14 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:08:15.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:14 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:08:15.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.360+0000 7f102779f640 1 -- 172.21.15.116:0/1448961017 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1020102db0 msgr2=0x7f1020103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:15.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.360+0000 7f102779f640 1 --2- 172.21.15.116:0/1448961017 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1020102db0 0x7f1020103190 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f1018009cd0 tx=0x7f101802a8e0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.360+0000 7f102779f640 1 -- 172.21.15.116:0/1448961017 shutdown_connections 2024-09-12T03:08:15.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.360+0000 7f102779f640 1 --2- 172.21.15.116:0/1448961017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10201082e0 0x7f102010ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.360+0000 7f102779f640 1 --2- 172.21.15.116:0/1448961017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1020103760 0x7f1020107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.360+0000 7f102779f640 1 --2- 172.21.15.116:0/1448961017 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1020102db0 0x7f1020103190 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.360+0000 7f102779f640 1 -- 172.21.15.116:0/1448961017 >> 172.21.15.116:0/1448961017 conn(0x7f10200fc7d0 msgr2=0x7f10200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:15.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.360+0000 7f102779f640 1 -- 172.21.15.116:0/1448961017 shutdown_connections 2024-09-12T03:08:15.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.360+0000 7f102779f640 1 -- 172.21.15.116:0/1448961017 wait complete. 2024-09-12T03:08:15.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.361+0000 7f102779f640 1 Processor -- start 2024-09-12T03:08:15.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.361+0000 7f102779f640 1 -- start start 2024-09-12T03:08:15.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.361+0000 7f102779f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1020102db0 0x7f10201985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:15.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.361+0000 7f102779f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1020103760 0x7f1020198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:15.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.361+0000 7f102779f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10201082e0 0x7f102019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:15.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.361+0000 7f102779f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1020112090 con 0x7f1020103760 2024-09-12T03:08:15.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.361+0000 7f102779f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1020111f10 con 0x7f1020102db0 2024-09-12T03:08:15.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.361+0000 7f102779f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1020112210 con 0x7f10201082e0 2024-09-12T03:08:15.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.362+0000 7f1024d3a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1020103760 0x7f1020198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:15.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.362+0000 7f1024d3a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1020103760 0x7f1020198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57730/0 (socket says 172.21.15.116:57730) 2024-09-12T03:08:15.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.362+0000 7f1024d3a640 1 -- 172.21.15.116:0/3788694141 learned_addr learned my addr 172.21.15.116:0/3788694141 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:15.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.362+0000 7f102553b640 1 --2- 172.21.15.116:0/3788694141 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1020102db0 0x7f10201985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:15.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.362+0000 7f1025d3c640 1 --2- 172.21.15.116:0/3788694141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10201082e0 0x7f102019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:15.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.362+0000 7f1025d3c640 1 -- 172.21.15.116:0/3788694141 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1020102db0 msgr2=0x7f10201985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:15.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.362+0000 7f1025d3c640 1 --2- 172.21.15.116:0/3788694141 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1020102db0 0x7f10201985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.362+0000 7f1025d3c640 1 -- 172.21.15.116:0/3788694141 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1020103760 msgr2=0x7f1020198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:15.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.362+0000 7f1025d3c640 1 --2- 172.21.15.116:0/3788694141 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1020103760 0x7f1020198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.362+0000 7f1025d3c640 1 -- 172.21.15.116:0/3788694141 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f102019d550 con 0x7f10201082e0 2024-09-12T03:08:15.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.362+0000 7f1024d3a640 1 --2- 172.21.15.116:0/3788694141 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1020103760 0x7f1020198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:08:15.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.362+0000 7f102553b640 1 --2- 172.21.15.116:0/3788694141 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1020102db0 0x7f10201985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:08:15.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.362+0000 7f1025d3c640 1 --2- 172.21.15.116:0/3788694141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10201082e0 0x7f102019ce70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1010009b50 tx=0x7f101000b690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:15.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.362+0000 7f101e7fc640 1 -- 172.21.15.116:0/3788694141 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1010009e50 con 0x7f10201082e0 2024-09-12T03:08:15.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.362+0000 7f101e7fc640 1 -- 172.21.15.116:0/3788694141 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f101000be40 con 0x7f10201082e0 2024-09-12T03:08:15.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.362+0000 7f101e7fc640 1 -- 172.21.15.116:0/3788694141 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1010019670 con 0x7f10201082e0 2024-09-12T03:08:15.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.362+0000 7f102779f640 1 -- 172.21.15.116:0/3788694141 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f102019d840 con 0x7f10201082e0 2024-09-12T03:08:15.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.363+0000 7f102779f640 1 -- 172.21.15.116:0/3788694141 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1020100940 con 0x7f10201082e0 2024-09-12T03:08:15.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.366+0000 7f1029b78740 1 Processor -- start 2024-09-12T03:08:15.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.366+0000 7f101e7fc640 1 -- 172.21.15.116:0/3788694141 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1010020020 con 0x7f10201082e0 2024-09-12T03:08:15.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.366+0000 7f1029b78740 1 -- start start 2024-09-12T03:08:15.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.366+0000 7f1029b78740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087ff17b40 0x56087ff16090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:15.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.366+0000 7f1029b78740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56087ff165d0 0x56087ff16a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:15.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.366+0000 7f1029b78740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56087ff21370 0x56087ff23760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:15.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.366+0000 7f1029b78740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56087fe7c2f0 con 0x56087ff17b40 2024-09-12T03:08:15.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.366+0000 7f1029b78740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56087fe23cf0 con 0x56087ff165d0 2024-09-12T03:08:15.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.366+0000 7f1029b78740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56087fe782b0 con 0x56087ff21370 2024-09-12T03:08:15.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.366+0000 7f1024d3a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087ff17b40 0x56087ff16090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:15.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.366+0000 7f1024d3a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087ff17b40 0x56087ff16090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57742/0 (socket says 172.21.15.116:57742) 2024-09-12T03:08:15.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.366+0000 7f1024d3a640 1 -- 172.21.15.116:0/2181204988 learned_addr learned my addr 172.21.15.116:0/2181204988 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:15.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.366+0000 7f101e7fc640 1 --2- 172.21.15.116:0/3788694141 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f10080a2da0 0x7f10080a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:15.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.366+0000 7f102553b640 1 --2- 172.21.15.116:0/2181204988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56087ff165d0 0x56087ff16a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:15.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.367+0000 7f1024d3a640 1 --2- 172.21.15.116:0/2181204988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56087ff21370 0x56087ff23760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:15.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.367+0000 7f1025d3c640 1 --2- 172.21.15.116:0/3788694141 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f10080a2da0 0x7f10080a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:15.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.367+0000 7f102553b640 1 -- 172.21.15.116:0/2181204988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56087ff21370 msgr2=0x56087ff23760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:15.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.367+0000 7f102553b640 1 --2- 172.21.15.116:0/2181204988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56087ff21370 0x56087ff23760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.367+0000 7f102553b640 1 -- 172.21.15.116:0/2181204988 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087ff17b40 msgr2=0x56087ff16090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:15.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.367+0000 7f102553b640 1 --2- 172.21.15.116:0/2181204988 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087ff17b40 0x56087ff16090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.367+0000 7f102553b640 1 -- 172.21.15.116:0/2181204988 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56087fe083a0 con 0x56087ff165d0 2024-09-12T03:08:15.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.367+0000 7f101e7fc640 1 -- 172.21.15.116:0/3788694141 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f10100d26a0 con 0x7f10201082e0 2024-09-12T03:08:15.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.367+0000 7f1024d3a640 1 --2- 172.21.15.116:0/2181204988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56087ff21370 0x56087ff23760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:08:15.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.367+0000 7f102553b640 1 --2- 172.21.15.116:0/2181204988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56087ff165d0 0x56087ff16a10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1018005e30 tx=0x7f1018031800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d34fc611e76de072 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:15.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.367+0000 7f10077fe640 1 -- 172.21.15.116:0/2181204988 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1018048070 con 0x56087ff165d0 2024-09-12T03:08:15.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.367+0000 7f1025d3c640 1 --2- 172.21.15.116:0/3788694141 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f10080a2da0 0x7f10080a5260 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f101009ec30 tx=0x7f1010069bb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:15.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.367+0000 7f10077fe640 1 -- 172.21.15.116:0/2181204988 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1018036070 con 0x56087ff165d0 2024-09-12T03:08:15.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.367+0000 7f10077fe640 1 -- 172.21.15.116:0/2181204988 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10180045a0 con 0x56087ff165d0 2024-09-12T03:08:15.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.368+0000 7f1029b78740 1 -- 172.21.15.116:0/2181204988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56087ff165d0 msgr2=0x56087ff16a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:15.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.368+0000 7f1029b78740 1 --2- 172.21.15.116:0/2181204988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56087ff165d0 0x56087ff16a10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1018005e30 tx=0x7f1018031800 comp rx=0 tx=0).stop 2024-09-12T03:08:15.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.368+0000 7f1029b78740 1 -- 172.21.15.116:0/2181204988 shutdown_connections 2024-09-12T03:08:15.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.368+0000 7f1029b78740 1 --2- 172.21.15.116:0/2181204988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56087ff21370 0x56087ff23760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.368+0000 7f1029b78740 1 --2- 172.21.15.116:0/2181204988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56087ff165d0 0x56087ff16a10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.368+0000 7f1029b78740 1 --2- 172.21.15.116:0/2181204988 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087ff17b40 0x56087ff16090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.368+0000 7f1029b78740 1 -- 172.21.15.116:0/2181204988 >> 172.21.15.116:0/2181204988 conn(0x56087fedf310 msgr2=0x56087ff12c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:15.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.368+0000 7f1029b78740 1 -- 172.21.15.116:0/2181204988 shutdown_connections 2024-09-12T03:08:15.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.368+0000 7f1029b78740 1 -- 172.21.15.116:0/2181204988 wait complete. 2024-09-12T03:08:15.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.369+0000 7f1029b78740 1 -- start start 2024-09-12T03:08:15.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.369+0000 7f1029b78740 1 Processor -- start 2024-09-12T03:08:15.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.369+0000 7f1029b78740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56087ff165d0 0x56087ff20d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:15.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.369+0000 7f1029b78740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087ff17b40 0x56087ff1dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:15.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.369+0000 7f1029b78740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56087ff21370 0x56087ff1e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:15.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.369+0000 7f1029b78740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56087ff240b0 con 0x56087ff17b40 2024-09-12T03:08:15.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.369+0000 7f1029b78740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56087fe37060 con 0x56087ff21370 2024-09-12T03:08:15.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.369+0000 7f1029b78740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56087ff24230 con 0x56087ff165d0 2024-09-12T03:08:15.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.369+0000 7f1029b78740 10 client.0 _ll_register_callbacks cb 0x56087fb8a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:15.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.369+0000 7f102553b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087ff17b40 0x56087ff1dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:15.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.369+0000 7f102553b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087ff17b40 0x56087ff1dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57754/0 (socket says 172.21.15.116:57754) 2024-09-12T03:08:15.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.369+0000 7f102553b640 1 -- 172.21.15.116:0/327875826 learned_addr learned my addr 172.21.15.116:0/327875826 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:15.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.369+0000 7f1024d3a640 1 --2- 172.21.15.116:0/327875826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56087ff165d0 0x56087ff20d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:15.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.370+0000 7f1024d3a640 1 --2- 172.21.15.116:0/327875826 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56087ff21370 0x56087ff1e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:15.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.370+0000 7f102553b640 1 -- 172.21.15.116:0/327875826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56087ff165d0 msgr2=0x56087ff20d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:15.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.370+0000 7f102553b640 1 --2- 172.21.15.116:0/327875826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56087ff165d0 0x56087ff20d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.370+0000 7f102553b640 1 -- 172.21.15.116:0/327875826 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56087ff21370 msgr2=0x56087ff1e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:15.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.370+0000 7f102553b640 1 --2- 172.21.15.116:0/327875826 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56087ff21370 0x56087ff1e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.370+0000 7f102553b640 1 -- 172.21.15.116:0/327875826 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56087fbed3c0 con 0x56087ff17b40 2024-09-12T03:08:15.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.370+0000 7f102553b640 1 --2- 172.21.15.116:0/327875826 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087ff17b40 0x56087ff1dfe0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f101802a920 tx=0x7f1018005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd11c380be32e5f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:15.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.370+0000 7f1004ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:15.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.370+0000 7f1004ff9640 1 -- 172.21.15.116:0/327875826 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1018053060 con 0x56087ff17b40 2024-09-12T03:08:15.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.370+0000 7f1004ff9640 1 -- 172.21.15.116:0/327875826 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1018004320 con 0x56087ff17b40 2024-09-12T03:08:15.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.370+0000 7f1029b78740 1 -- 172.21.15.116:0/327875826 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1018036070 con 0x56087ff17b40 2024-09-12T03:08:15.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.370+0000 7f1004ff9640 1 -- 172.21.15.116:0/327875826 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1018048040 con 0x56087ff17b40 2024-09-12T03:08:15.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.370+0000 7f1029b78740 10 client.16632 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:15.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.370+0000 7f1029b78740 1 -- 172.21.15.116:0/327875826 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56087ff240b0 con 0x56087ff17b40 2024-09-12T03:08:15.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.370+0000 7f1004ff9640 1 -- 172.21.15.116:0/327875826 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1018036070 con 0x56087ff17b40 2024-09-12T03:08:15.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.371+0000 7f1004ff9640 1 -- 172.21.15.116:0/327875826 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1018042030 con 0x56087ff17b40 2024-09-12T03:08:15.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.371+0000 7f1029b78740 10 client.16632 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:15.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.371+0000 7f1029b78740 10 client.16632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:08:15.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.371+0000 7f1029b78740 20 client.16632 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:15.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.371+0000 7f1029b78740 1 --2- 172.21.15.116:0/327875826 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56087ff30c00 0x56087ff32ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:15.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.371+0000 7f1029b78740 4 client.16632 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:08:15.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.371+0000 7f1029b78740 1 -- 172.21.15.116:0/327875826 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56087fe89860 con 0x56087ff30c00 2024-09-12T03:08:15.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.371+0000 7f1024d3a640 1 --2- 172.21.15.116:0/327875826 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56087ff30c00 0x56087ff32ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:15.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.371+0000 7f1024d3a640 1 --2- 172.21.15.116:0/327875826 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56087ff30c00 0x56087ff32ff0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=294575d3f96038ea server_cookie=66a36e90ccd398cb in_seq=0 out_seq=0 2024-09-12T03:08:15.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.371+0000 7f1004ff9640 10 client.16632 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:15.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.372+0000 7f1004ff9640 1 -- 172.21.15.116:0/327875826 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56087fe89860 con 0x56087ff30c00 2024-09-12T03:08:15.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.372+0000 7f1004ff9640 10 client.16632 handle_command_reply: tid=0 2024-09-12T03:08:15.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.372+0000 7f1029b78740 1 client.16632 shutdown 2024-09-12T03:08:15.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.372+0000 7f1029b78740 1 -- 172.21.15.116:0/327875826 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087ff17b40 msgr2=0x56087ff1dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:15.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.372+0000 7f1029b78740 1 --2- 172.21.15.116:0/327875826 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087ff17b40 0x56087ff1dfe0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f101802a920 tx=0x7f1018005f70 comp rx=0 tx=0).stop 2024-09-12T03:08:15.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.372+0000 7f1029b78740 1 -- 172.21.15.116:0/327875826 shutdown_connections 2024-09-12T03:08:15.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.372+0000 7f1029b78740 1 --2- 172.21.15.116:0/327875826 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56087ff30c00 0x56087ff32ff0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.372+0000 7f1029b78740 1 --2- 172.21.15.116:0/327875826 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56087ff21370 0x56087ff1e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.372+0000 7f1029b78740 1 --2- 172.21.15.116:0/327875826 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087ff17b40 0x56087ff1dfe0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.372+0000 7f1029b78740 1 --2- 172.21.15.116:0/327875826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56087ff165d0 0x56087ff20d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.372+0000 7f1029b78740 1 -- 172.21.15.116:0/327875826 >> 172.21.15.116:0/327875826 conn(0x56087fedf310 msgr2=0x56087ff18960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:15.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.372+0000 7f1004ff9640 0 client.16632 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:15.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.372+0000 7f1029b78740 1 -- 172.21.15.116:0/327875826 shutdown_connections 2024-09-12T03:08:15.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.372+0000 7f1029b78740 1 -- 172.21.15.116:0/327875826 wait complete. 2024-09-12T03:08:15.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.372+0000 7f1029b78740 20 client.16632 trim_cache size 0 max 16384 2024-09-12T03:08:15.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.386+0000 7f1029b78740 1 Processor -- start 2024-09-12T03:08:15.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.386+0000 7f1029b78740 1 -- start start 2024-09-12T03:08:15.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.386+0000 7f1029b78740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087ff165d0 0x56087ff1dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:15.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.386+0000 7f1029b78740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56087ff17b40 0x56087ff2d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:15.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.386+0000 7f1029b78740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56087ff21370 0x56087ff22760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:15.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.386+0000 7f1029b78740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f10201adbc0 con 0x56087ff165d0 2024-09-12T03:08:15.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.386+0000 7f1029b78740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f10201ab490 con 0x56087ff21370 2024-09-12T03:08:15.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.386+0000 7f1029b78740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f10201a92e0 con 0x56087ff17b40 2024-09-12T03:08:15.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.386+0000 7f1024d3a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087ff165d0 0x56087ff1dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:15.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.386+0000 7f1024d3a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087ff165d0 0x56087ff1dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57764/0 (socket says 172.21.15.116:57764) 2024-09-12T03:08:15.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.386+0000 7f1024d3a640 1 -- 172.21.15.116:0/691110628 learned_addr learned my addr 172.21.15.116:0/691110628 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:15.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.386+0000 7f1024d3a640 1 --2- 172.21.15.116:0/691110628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56087ff21370 0x56087ff22760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:15.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.387+0000 7f102553b640 1 --2- 172.21.15.116:0/691110628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56087ff17b40 0x56087ff2d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:15.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.387+0000 7f1024d3a640 1 -- 172.21.15.116:0/691110628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56087ff17b40 msgr2=0x56087ff2d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:15.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.387+0000 7f1024d3a640 1 --2- 172.21.15.116:0/691110628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56087ff17b40 0x56087ff2d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.387+0000 7f1024d3a640 1 -- 172.21.15.116:0/691110628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56087ff21370 msgr2=0x56087ff22760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:15.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.387+0000 7f1024d3a640 1 --2- 172.21.15.116:0/691110628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56087ff21370 0x56087ff22760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.387+0000 7f1024d3a640 1 -- 172.21.15.116:0/691110628 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56087ff1e940 con 0x56087ff165d0 2024-09-12T03:08:15.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.387+0000 7f102553b640 1 --2- 172.21.15.116:0/691110628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56087ff17b40 0x56087ff2d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:08:15.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.387+0000 7f1024d3a640 1 --2- 172.21.15.116:0/691110628 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087ff165d0 0x56087ff1dfb0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f1014005eb0 tx=0x7f1014009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca2d7f75b97dc22 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:15.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.387+0000 7f1006ffd640 1 -- 172.21.15.116:0/691110628 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101401a040 con 0x56087ff165d0 2024-09-12T03:08:15.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.387+0000 7f1006ffd640 1 -- 172.21.15.116:0/691110628 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1014025040 con 0x56087ff165d0 2024-09-12T03:08:15.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.387+0000 7f1006ffd640 1 -- 172.21.15.116:0/691110628 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1014003e50 con 0x56087ff165d0 2024-09-12T03:08:15.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.388+0000 7f1029b78740 1 -- 172.21.15.116:0/691110628 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087ff165d0 msgr2=0x56087ff1dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:15.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.388+0000 7f1029b78740 1 --2- 172.21.15.116:0/691110628 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087ff165d0 0x56087ff1dfb0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f1014005eb0 tx=0x7f1014009da0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.388+0000 7f1029b78740 1 -- 172.21.15.116:0/691110628 shutdown_connections 2024-09-12T03:08:15.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.388+0000 7f1029b78740 1 --2- 172.21.15.116:0/691110628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56087ff21370 0x56087ff22760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.388+0000 7f1029b78740 1 --2- 172.21.15.116:0/691110628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56087ff17b40 0x56087ff2d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.388+0000 7f1029b78740 1 --2- 172.21.15.116:0/691110628 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087ff165d0 0x56087ff1dfb0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.388+0000 7f1029b78740 1 -- 172.21.15.116:0/691110628 >> 172.21.15.116:0/691110628 conn(0x56087fedf310 msgr2=0x56087ff1d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:15.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.388+0000 7f1029b78740 1 -- 172.21.15.116:0/691110628 shutdown_connections 2024-09-12T03:08:15.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.388+0000 7f1029b78740 1 -- 172.21.15.116:0/691110628 wait complete. 2024-09-12T03:08:15.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.388+0000 7f1029b78740 1 -- start start 2024-09-12T03:08:15.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.388+0000 7f1029b78740 1 Processor -- start 2024-09-12T03:08:15.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.389+0000 7f1029b78740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56087ff165d0 0x56087ff29760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:15.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.389+0000 7f1029b78740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56087ff17b40 0x56087ff29ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:15.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.389+0000 7f1029b78740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087ff21370 0x56087ff37c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:15.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.389+0000 7f1029b78740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f10201a9950 con 0x56087ff21370 2024-09-12T03:08:15.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.389+0000 7f1029b78740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f10201aadc0 con 0x56087ff165d0 2024-09-12T03:08:15.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.389+0000 7f1029b78740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f10201aa020 con 0x56087ff17b40 2024-09-12T03:08:15.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.389+0000 7f1029b78740 10 client.0 _ll_register_callbacks cb 0x7f1018046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:15.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.389+0000 7f1024d3a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087ff21370 0x56087ff37c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:15.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.389+0000 7f1024d3a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087ff21370 0x56087ff37c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57780/0 (socket says 172.21.15.116:57780) 2024-09-12T03:08:15.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.389+0000 7f1024d3a640 1 -- 172.21.15.116:0/496371750 learned_addr learned my addr 172.21.15.116:0/496371750 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:15.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.389+0000 7f102553b640 1 --2- 172.21.15.116:0/496371750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56087ff17b40 0x56087ff29ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:15.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.389+0000 7f1024d3a640 1 --2- 172.21.15.116:0/496371750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56087ff165d0 0x56087ff29760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:15.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.389+0000 7f1024d3a640 1 -- 172.21.15.116:0/496371750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56087ff17b40 msgr2=0x56087ff29ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:15.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.389+0000 7f1024d3a640 1 --2- 172.21.15.116:0/496371750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56087ff17b40 0x56087ff29ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.389+0000 7f1024d3a640 1 -- 172.21.15.116:0/496371750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56087ff165d0 msgr2=0x56087ff29760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:15.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.389+0000 7f1024d3a640 1 --2- 172.21.15.116:0/496371750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56087ff165d0 0x56087ff29760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.389+0000 7f1024d3a640 1 -- 172.21.15.116:0/496371750 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56087ff38460 con 0x56087ff21370 2024-09-12T03:08:15.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.389+0000 7f1024d3a640 1 --2- 172.21.15.116:0/496371750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087ff21370 0x56087ff37c30 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f101400a010 tx=0x7f1014003a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3503ae14366d03 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:15.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.389+0000 7f1004ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:15.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.390+0000 7f1004ff9640 1 -- 172.21.15.116:0/496371750 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1014023680 con 0x56087ff21370 2024-09-12T03:08:15.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.390+0000 7f1029b78740 1 -- 172.21.15.116:0/496371750 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1014025040 con 0x56087ff21370 2024-09-12T03:08:15.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.390+0000 7f1004ff9640 1 -- 172.21.15.116:0/496371750 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1014023820 con 0x56087ff21370 2024-09-12T03:08:15.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.390+0000 7f1004ff9640 1 -- 172.21.15.116:0/496371750 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1014023a20 con 0x56087ff21370 2024-09-12T03:08:15.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.390+0000 7f1029b78740 10 client.16638 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:15.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.390+0000 7f1029b78740 1 -- 172.21.15.116:0/496371750 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56087ff28f70 con 0x56087ff21370 2024-09-12T03:08:15.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.390+0000 7f1004ff9640 1 -- 172.21.15.116:0/496371750 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1014025040 con 0x56087ff21370 2024-09-12T03:08:15.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.390+0000 7f1004ff9640 1 -- 172.21.15.116:0/496371750 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1014039030 con 0x56087ff21370 2024-09-12T03:08:15.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.390+0000 7f1029b78740 10 client.16638 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:15.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.390+0000 7f1029b78740 10 client.16638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:08:15.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.390+0000 7f1029b78740 20 client.16638 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:15.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.390+0000 7f1029b78740 1 --2- 172.21.15.116:0/496371750 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56087ff9b830 0x56087ff9dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:15.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.390+0000 7f1029b78740 4 client.16638 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:08:15.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.390+0000 7f1029b78740 1 -- 172.21.15.116:0/496371750 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56087fe83d70 con 0x56087ff9b830 2024-09-12T03:08:15.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.390+0000 7f102553b640 1 --2- 172.21.15.116:0/496371750 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56087ff9b830 0x56087ff9dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:15.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.391+0000 7f102553b640 1 --2- 172.21.15.116:0/496371750 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56087ff9b830 0x56087ff9dc50 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ad064a603bfbfcb server_cookie=477659c938291f00 in_seq=0 out_seq=0 2024-09-12T03:08:15.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.391+0000 7f1004ff9640 10 client.16638 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:15.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.391+0000 7f1004ff9640 1 -- 172.21.15.116:0/496371750 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56087fe83d70 con 0x56087ff9b830 2024-09-12T03:08:15.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.391+0000 7f1004ff9640 10 client.16638 handle_command_reply: tid=0 2024-09-12T03:08:15.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.391+0000 7f1029b78740 1 client.16638 shutdown 2024-09-12T03:08:15.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.391+0000 7f1029b78740 1 -- 172.21.15.116:0/496371750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087ff21370 msgr2=0x56087ff37c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:15.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.391+0000 7f1029b78740 1 --2- 172.21.15.116:0/496371750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087ff21370 0x56087ff37c30 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f101400a010 tx=0x7f1014003a30 comp rx=0 tx=0).stop 2024-09-12T03:08:15.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.391+0000 7f1029b78740 1 -- 172.21.15.116:0/496371750 shutdown_connections 2024-09-12T03:08:15.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.391+0000 7f1029b78740 1 --2- 172.21.15.116:0/496371750 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56087ff9b830 0x56087ff9dc50 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.391+0000 7f1029b78740 1 --2- 172.21.15.116:0/496371750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56087ff21370 0x56087ff37c30 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.391+0000 7f1029b78740 1 --2- 172.21.15.116:0/496371750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56087ff17b40 0x56087ff29ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.391+0000 7f1004ff9640 0 client.16638 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:15.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.391+0000 7f1029b78740 1 --2- 172.21.15.116:0/496371750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56087ff165d0 0x56087ff29760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.391+0000 7f1029b78740 1 -- 172.21.15.116:0/496371750 >> 172.21.15.116:0/496371750 conn(0x56087fedf310 msgr2=0x56087ff20060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:15.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.391+0000 7f1029b78740 1 -- 172.21.15.116:0/496371750 shutdown_connections 2024-09-12T03:08:15.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.391+0000 7f1029b78740 1 -- 172.21.15.116:0/496371750 wait complete. 2024-09-12T03:08:15.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.391+0000 7f1029b78740 20 client.16638 trim_cache size 0 max 16384 2024-09-12T03:08:15.397 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:08:15.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.392+0000 7f102779f640 1 -- 172.21.15.116:0/3788694141 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f10080a2da0 msgr2=0x7f10080a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:15.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.392+0000 7f102779f640 1 --2- 172.21.15.116:0/3788694141 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f10080a2da0 0x7f10080a5260 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f101009ec30 tx=0x7f1010069bb0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.392+0000 7f102779f640 1 -- 172.21.15.116:0/3788694141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10201082e0 msgr2=0x7f102019ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:15.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.392+0000 7f102779f640 1 --2- 172.21.15.116:0/3788694141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10201082e0 0x7f102019ce70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1010009b50 tx=0x7f101000b690 comp rx=0 tx=0).stop 2024-09-12T03:08:15.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.392+0000 7f102779f640 1 -- 172.21.15.116:0/3788694141 shutdown_connections 2024-09-12T03:08:15.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.392+0000 7f102779f640 1 --2- 172.21.15.116:0/3788694141 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f10080a2da0 0x7f10080a5260 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.392+0000 7f102779f640 1 --2- 172.21.15.116:0/3788694141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10201082e0 0x7f102019ce70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.392+0000 7f102779f640 1 --2- 172.21.15.116:0/3788694141 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1020103760 0x7f1020198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.392+0000 7f102779f640 1 --2- 172.21.15.116:0/3788694141 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1020102db0 0x7f10201985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:15.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.392+0000 7f102779f640 1 -- 172.21.15.116:0/3788694141 >> 172.21.15.116:0/3788694141 conn(0x7f10200fc7d0 msgr2=0x7f1020105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:15.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.392+0000 7f102779f640 1 -- 172.21.15.116:0/3788694141 shutdown_connections 2024-09-12T03:08:15.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:15.392+0000 7f102779f640 1 -- 172.21.15.116:0/3788694141 wait complete. 2024-09-12T03:08:16.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:15 smithi183 ceph-mon[35535]: pgmap v496: 97 pgs: 97 active+clean; 2.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:08:16.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:15 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:08:16.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:15 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:08:16.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:15 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:08:16.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:15 smithi177 ceph-mon[35305]: pgmap v496: 97 pgs: 97 active+clean; 2.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:08:16.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:15 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:08:16.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:15 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:08:16.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:15 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:08:16.288 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:15 smithi116 ceph-mon[31410]: pgmap v496: 97 pgs: 97 active+clean; 2.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:08:16.288 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:08:16.288 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:08:16.288 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:15 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:08:17.317 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:08:17.317 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:08:17.578 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:08:18.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.059+0000 7fb1ad4a5640 1 -- 172.21.15.116:0/1165806325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1a8101420 msgr2=0x7fb1a810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:18.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.059+0000 7fb1ad4a5640 1 --2- 172.21.15.116:0/1165806325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1a8101420 0x7fb1a810d950 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0009cd0 tx=0x7fb1a002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.059+0000 7fb1ad4a5640 1 -- 172.21.15.116:0/1165806325 shutdown_connections 2024-09-12T03:08:18.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.059+0000 7fb1ad4a5640 1 --2- 172.21.15.116:0/1165806325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1a8101420 0x7fb1a810d950 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.059+0000 7fb1ad4a5640 1 --2- 172.21.15.116:0/1165806325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1a8100a80 0x7fb1a8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.059+0000 7fb1ad4a5640 1 --2- 172.21.15.116:0/1165806325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1a8106a80 0x7fb1a8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.059+0000 7fb1ad4a5640 1 -- 172.21.15.116:0/1165806325 >> 172.21.15.116:0/1165806325 conn(0x7fb1a80fc7a0 msgr2=0x7fb1a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:18.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.059+0000 7fb1ad4a5640 1 -- 172.21.15.116:0/1165806325 shutdown_connections 2024-09-12T03:08:18.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.059+0000 7fb1ad4a5640 1 -- 172.21.15.116:0/1165806325 wait complete. 2024-09-12T03:08:18.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.060+0000 7fb1ad4a5640 1 Processor -- start 2024-09-12T03:08:18.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.060+0000 7fb1ad4a5640 1 -- start start 2024-09-12T03:08:18.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.060+0000 7fb1ad4a5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1a8100a80 0x7fb1a8197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:18.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.060+0000 7fb1ad4a5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1a8101420 0x7fb1a8197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:18.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.060+0000 7fb1ad4a5640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1a8106a80 0x7fb1a819bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:18.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.060+0000 7fb1ad4a5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a8110e80 con 0x7fb1a8101420 2024-09-12T03:08:18.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.060+0000 7fb1ad4a5640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a8110d00 con 0x7fb1a8106a80 2024-09-12T03:08:18.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.060+0000 7fb1ad4a5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a8111000 con 0x7fb1a8100a80 2024-09-12T03:08:18.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.060+0000 7fb1a67fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1a8101420 0x7fb1a8197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:18.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.060+0000 7fb1a67fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1a8101420 0x7fb1a8197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57794/0 (socket says 172.21.15.116:57794) 2024-09-12T03:08:18.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.060+0000 7fb1a67fc640 1 -- 172.21.15.116:0/2759517967 learned_addr learned my addr 172.21.15.116:0/2759517967 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:18.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.060+0000 7fb1a77fe640 1 --2- 172.21.15.116:0/2759517967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1a8106a80 0x7fb1a819bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:18.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.060+0000 7fb1a6ffd640 1 --2- 172.21.15.116:0/2759517967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1a8100a80 0x7fb1a8197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:18.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.060+0000 7fb1a67fc640 1 -- 172.21.15.116:0/2759517967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1a8100a80 msgr2=0x7fb1a8197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:18.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.060+0000 7fb1a67fc640 1 --2- 172.21.15.116:0/2759517967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1a8100a80 0x7fb1a8197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.060+0000 7fb1a67fc640 1 -- 172.21.15.116:0/2759517967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1a8106a80 msgr2=0x7fb1a819bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:18.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.060+0000 7fb1a67fc640 1 --2- 172.21.15.116:0/2759517967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1a8106a80 0x7fb1a819bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.060+0000 7fb1a67fc640 1 -- 172.21.15.116:0/2759517967 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a819c2b0 con 0x7fb1a8101420 2024-09-12T03:08:18.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.060+0000 7fb1a77fe640 1 --2- 172.21.15.116:0/2759517967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1a8106a80 0x7fb1a819bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:18.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.060+0000 7fb1a6ffd640 1 --2- 172.21.15.116:0/2759517967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1a8100a80 0x7fb1a8197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:18.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.061+0000 7fb1a67fc640 1 --2- 172.21.15.116:0/2759517967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1a8101420 0x7fb1a8197840 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00d950 tx=0x7fb19c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:18.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.061+0000 7fb18ffff640 1 -- 172.21.15.116:0/2759517967 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19c017070 con 0x7fb1a8101420 2024-09-12T03:08:18.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.061+0000 7fb18ffff640 1 -- 172.21.15.116:0/2759517967 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb19c009c30 con 0x7fb1a8101420 2024-09-12T03:08:18.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.061+0000 7fb1ad4a5640 1 -- 172.21.15.116:0/2759517967 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a819c5a0 con 0x7fb1a8101420 2024-09-12T03:08:18.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.061+0000 7fb18ffff640 1 -- 172.21.15.116:0/2759517967 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19c00f6b0 con 0x7fb1a8101420 2024-09-12T03:08:18.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.061+0000 7fb1ad4a5640 1 -- 172.21.15.116:0/2759517967 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a81a3e80 con 0x7fb1a8101420 2024-09-12T03:08:18.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.063+0000 7fb18ffff640 1 -- 172.21.15.116:0/2759517967 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb19c0109d0 con 0x7fb1a8101420 2024-09-12T03:08:18.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.063+0000 7fb1af87e740 1 Processor -- start 2024-09-12T03:08:18.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.063+0000 7fb1af87e740 1 -- start start 2024-09-12T03:08:18.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.064+0000 7fb1af87e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cf6250b60 0x562cf6250f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:18.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.064+0000 7fb1af87e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562cf6251480 0x562cf625b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:18.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.064+0000 7fb1af87e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562cf625bd40 0x562cf625e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:18.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.064+0000 7fb1af87e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562cf61b6280 con 0x562cf625bd40 2024-09-12T03:08:18.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.064+0000 7fb1af87e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562cf615db10 con 0x562cf6251480 2024-09-12T03:08:18.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.064+0000 7fb1af87e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562cf61b2240 con 0x562cf6250b60 2024-09-12T03:08:18.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.064+0000 7fb1a67fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562cf625bd40 0x562cf625e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:18.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.064+0000 7fb18ffff640 1 --2- 172.21.15.116:0/2759517967 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb1900a2b80 0x7fb1900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:18.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.064+0000 7fb1a67fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562cf625bd40 0x562cf625e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57796/0 (socket says 172.21.15.116:57796) 2024-09-12T03:08:18.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.064+0000 7fb1a67fc640 1 -- 172.21.15.116:0/1546883745 learned_addr learned my addr 172.21.15.116:0/1546883745 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:18.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.064+0000 7fb1a77fe640 1 --2- 172.21.15.116:0/1546883745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cf6250b60 0x562cf6250f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:18.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.064+0000 7fb1a6ffd640 1 --2- 172.21.15.116:0/1546883745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562cf6251480 0x562cf625b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:18.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.064+0000 7fb1a77fe640 1 --2- 172.21.15.116:0/2759517967 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb1900a2b80 0x7fb1900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:18.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.064+0000 7fb18ffff640 1 -- 172.21.15.116:0/2759517967 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fb19c0ced50 con 0x7fb1a8101420 2024-09-12T03:08:18.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.064+0000 7fb1a67fc640 1 -- 172.21.15.116:0/1546883745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cf6250b60 msgr2=0x562cf6250f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:18.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.064+0000 7fb1a67fc640 1 --2- 172.21.15.116:0/1546883745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cf6250b60 0x562cf6250f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.064+0000 7fb1a67fc640 1 -- 172.21.15.116:0/1546883745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562cf6251480 msgr2=0x562cf625b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:18.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.064+0000 7fb1a67fc640 1 --2- 172.21.15.116:0/1546883745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562cf6251480 0x562cf625b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.064+0000 7fb1a67fc640 1 -- 172.21.15.116:0/1546883745 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cf616a1f0 con 0x562cf625bd40 2024-09-12T03:08:18.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.064+0000 7fb1a6ffd640 1 --2- 172.21.15.116:0/1546883745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562cf6251480 0x562cf625b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:08:18.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.064+0000 7fb1a77fe640 1 --2- 172.21.15.116:0/1546883745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cf6250b60 0x562cf6250f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:18.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.064+0000 7fb1a67fc640 1 --2- 172.21.15.116:0/1546883745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562cf625bd40 0x562cf625e130 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb19c097a40 tx=0x7fb19c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e450bcc74ff422da server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:18.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.065+0000 7fb1a77fe640 1 --2- 172.21.15.116:0/2759517967 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb1900a2b80 0x7fb1900a5040 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0033040 tx=0x7fb1a00095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:18.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.065+0000 7fb18d7fa640 1 -- 172.21.15.116:0/1546883745 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19c020070 con 0x562cf625bd40 2024-09-12T03:08:18.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.065+0000 7fb18d7fa640 1 -- 172.21.15.116:0/1546883745 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb19c009de0 con 0x562cf625bd40 2024-09-12T03:08:18.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.065+0000 7fb18d7fa640 1 -- 172.21.15.116:0/1546883745 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19c09b840 con 0x562cf625bd40 2024-09-12T03:08:18.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.065+0000 7fb1af87e740 1 -- 172.21.15.116:0/1546883745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562cf625bd40 msgr2=0x562cf625e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:18.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.065+0000 7fb1af87e740 1 --2- 172.21.15.116:0/1546883745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562cf625bd40 0x562cf625e130 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb19c097a40 tx=0x7fb19c000bc0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.065+0000 7fb1af87e740 1 -- 172.21.15.116:0/1546883745 shutdown_connections 2024-09-12T03:08:18.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.065+0000 7fb1af87e740 1 --2- 172.21.15.116:0/1546883745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562cf625bd40 0x562cf625e130 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.065+0000 7fb1af87e740 1 --2- 172.21.15.116:0/1546883745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562cf6251480 0x562cf625b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.065+0000 7fb1af87e740 1 --2- 172.21.15.116:0/1546883745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cf6250b60 0x562cf6250f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.065+0000 7fb1af87e740 1 -- 172.21.15.116:0/1546883745 >> 172.21.15.116:0/1546883745 conn(0x562cf624d0f0 msgr2=0x562cf624d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:18.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.065+0000 7fb1af87e740 1 -- 172.21.15.116:0/1546883745 shutdown_connections 2024-09-12T03:08:18.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.065+0000 7fb1af87e740 1 -- 172.21.15.116:0/1546883745 wait complete. 2024-09-12T03:08:18.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.066+0000 7fb1af87e740 1 -- start start 2024-09-12T03:08:18.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.066+0000 7fb1af87e740 1 Processor -- start 2024-09-12T03:08:18.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.066+0000 7fb1af87e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562cf6250b60 0x562cf6258c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:18.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.066+0000 7fb1af87e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562cf6251480 0x562cf62591c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:18.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.066+0000 7fb1af87e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cf625bd40 0x562cf6259700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:18.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.066+0000 7fb1af87e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562cf625e910 con 0x562cf6250b60 2024-09-12T03:08:18.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.066+0000 7fb1af87e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562cf6170f10 con 0x562cf6251480 2024-09-12T03:08:18.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.066+0000 7fb1af87e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562cf625ea90 con 0x562cf625bd40 2024-09-12T03:08:18.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.066+0000 7fb1af87e740 10 client.0 _ll_register_callbacks cb 0x562cf6139a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:18.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.066+0000 7fb1a6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562cf6250b60 0x562cf6258c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:18.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.066+0000 7fb1a6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562cf6250b60 0x562cf6258c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57806/0 (socket says 172.21.15.116:57806) 2024-09-12T03:08:18.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.066+0000 7fb1a6ffd640 1 -- 172.21.15.116:0/2207127813 learned_addr learned my addr 172.21.15.116:0/2207127813 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:18.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.066+0000 7fb1a67fc640 1 --2- 172.21.15.116:0/2207127813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562cf6251480 0x562cf62591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:18.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.066+0000 7fb1a6ffd640 1 -- 172.21.15.116:0/2207127813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cf625bd40 msgr2=0x562cf6259700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:18.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.066+0000 7fb1a6ffd640 1 --2- 172.21.15.116:0/2207127813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cf625bd40 0x562cf6259700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.066+0000 7fb1a6ffd640 1 -- 172.21.15.116:0/2207127813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562cf6251480 msgr2=0x562cf62591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:18.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.066+0000 7fb1a6ffd640 1 --2- 172.21.15.116:0/2207127813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562cf6251480 0x562cf62591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.066+0000 7fb1a6ffd640 1 -- 172.21.15.116:0/2207127813 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cf5f276a0 con 0x562cf6250b60 2024-09-12T03:08:18.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.066+0000 7fb1a67fc640 1 --2- 172.21.15.116:0/2207127813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562cf6251480 0x562cf62591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:08:18.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.067+0000 7fb1a6ffd640 1 --2- 172.21.15.116:0/2207127813 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562cf6250b60 0x562cf6258c80 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb1980106a0 tx=0x7fb198010b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac51f4673fc418b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:18.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.067+0000 7fb18aff5640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:18.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.067+0000 7fb18aff5640 1 -- 172.21.15.116:0/2207127813 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19800ead0 con 0x562cf6250b60 2024-09-12T03:08:18.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.067+0000 7fb18aff5640 1 -- 172.21.15.116:0/2207127813 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fb19800ec70 con 0x562cf6250b60 2024-09-12T03:08:18.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.067+0000 7fb1af87e740 1 -- 172.21.15.116:0/2207127813 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb19c009de0 con 0x562cf6250b60 2024-09-12T03:08:18.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.067+0000 7fb18aff5640 1 -- 172.21.15.116:0/2207127813 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb19800ee70 con 0x562cf6250b60 2024-09-12T03:08:18.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.067+0000 7fb1af87e740 10 client.16650 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:18.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.067+0000 7fb1af87e740 1 -- 172.21.15.116:0/2207127813 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562cf626a540 con 0x562cf6250b60 2024-09-12T03:08:18.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.067+0000 7fb18aff5640 1 -- 172.21.15.116:0/2207127813 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb19c009de0 con 0x562cf6250b60 2024-09-12T03:08:18.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.067+0000 7fb18aff5640 1 -- 172.21.15.116:0/2207127813 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb19800f040 con 0x562cf6250b60 2024-09-12T03:08:18.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.067+0000 7fb1af87e740 10 client.16650 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:18.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.067+0000 7fb1af87e740 10 client.16650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:08:18.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.067+0000 7fb1af87e740 20 client.16650 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:18.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.067+0000 7fb1af87e740 1 --2- 172.21.15.116:0/2207127813 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562cf626ba00 0x562cf626dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:18.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.067+0000 7fb1af87e740 4 client.16650 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:08:18.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.067+0000 7fb1af87e740 1 -- 172.21.15.116:0/2207127813 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cf61c37f0 con 0x562cf626ba00 2024-09-12T03:08:18.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.068+0000 7fb1a67fc640 1 --2- 172.21.15.116:0/2207127813 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562cf626ba00 0x562cf626dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:18.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.068+0000 7fb1a67fc640 1 --2- 172.21.15.116:0/2207127813 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562cf626ba00 0x562cf626dec0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b294ee525503dfb8 server_cookie=9dc2d01b4ad0a7d1 in_seq=0 out_seq=0 2024-09-12T03:08:18.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.068+0000 7fb18aff5640 10 client.16650 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:18.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.069+0000 7fb18aff5640 1 -- 172.21.15.116:0/2207127813 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562cf61c37f0 con 0x562cf626ba00 2024-09-12T03:08:18.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.069+0000 7fb18aff5640 10 client.16650 handle_command_reply: tid=0 2024-09-12T03:08:18.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.069+0000 7fb1af87e740 1 client.16650 shutdown 2024-09-12T03:08:18.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.069+0000 7fb1af87e740 1 -- 172.21.15.116:0/2207127813 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562cf6250b60 msgr2=0x562cf6258c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:18.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.069+0000 7fb1af87e740 1 --2- 172.21.15.116:0/2207127813 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562cf6250b60 0x562cf6258c80 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb1980106a0 tx=0x7fb198010b60 comp rx=0 tx=0).stop 2024-09-12T03:08:18.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.069+0000 7fb1af87e740 1 -- 172.21.15.116:0/2207127813 shutdown_connections 2024-09-12T03:08:18.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.069+0000 7fb1af87e740 1 --2- 172.21.15.116:0/2207127813 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562cf626ba00 0x562cf626dec0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.069+0000 7fb1af87e740 1 --2- 172.21.15.116:0/2207127813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cf625bd40 0x562cf6259700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.069+0000 7fb1af87e740 1 --2- 172.21.15.116:0/2207127813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562cf6251480 0x562cf62591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.069+0000 7fb1af87e740 1 --2- 172.21.15.116:0/2207127813 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562cf6250b60 0x562cf6258c80 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.069+0000 7fb1af87e740 1 -- 172.21.15.116:0/2207127813 >> 172.21.15.116:0/2207127813 conn(0x562cf624d0f0 msgr2=0x562cf624c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:18.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.069+0000 7fb18aff5640 0 client.16650 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:18.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.069+0000 7fb1af87e740 1 -- 172.21.15.116:0/2207127813 shutdown_connections 2024-09-12T03:08:18.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.069+0000 7fb1af87e740 1 -- 172.21.15.116:0/2207127813 wait complete. 2024-09-12T03:08:18.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.069+0000 7fb1af87e740 20 client.16650 trim_cache size 0 max 16384 2024-09-12T03:08:18.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.087+0000 7fb1af87e740 1 Processor -- start 2024-09-12T03:08:18.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.087+0000 7fb1af87e740 1 -- start start 2024-09-12T03:08:18.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.087+0000 7fb1af87e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562cf6250b60 0x562cf625aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:18.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.087+0000 7fb1af87e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562cf6251480 0x562cf6263ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:18.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.087+0000 7fb1af87e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cf625bd40 0x562cf6264410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:18.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.087+0000 7fb1af87e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a81aed50 con 0x562cf6250b60 2024-09-12T03:08:18.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.087+0000 7fb1af87e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a81ac620 con 0x562cf6251480 2024-09-12T03:08:18.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.087+0000 7fb1af87e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a81aa470 con 0x562cf625bd40 2024-09-12T03:08:18.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.087+0000 7fb1a6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562cf6250b60 0x562cf625aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:18.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.087+0000 7fb1a6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562cf6250b60 0x562cf625aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57816/0 (socket says 172.21.15.116:57816) 2024-09-12T03:08:18.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.087+0000 7fb1a6ffd640 1 -- 172.21.15.116:0/3728309222 learned_addr learned my addr 172.21.15.116:0/3728309222 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:18.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.088+0000 7fb1a6ffd640 1 -- 172.21.15.116:0/3728309222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cf625bd40 msgr2=0x562cf6264410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:18.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.088+0000 7fb1a6ffd640 1 --2- 172.21.15.116:0/3728309222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cf625bd40 0x562cf6264410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.088+0000 7fb1a6ffd640 1 -- 172.21.15.116:0/3728309222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562cf6251480 msgr2=0x562cf6263ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:18.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.088+0000 7fb1a6ffd640 1 --2- 172.21.15.116:0/3728309222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562cf6251480 0x562cf6263ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.088+0000 7fb1a6ffd640 1 -- 172.21.15.116:0/3728309222 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cf6257c30 con 0x562cf6250b60 2024-09-12T03:08:18.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.088+0000 7fb1a6ffd640 1 --2- 172.21.15.116:0/3728309222 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562cf6250b60 0x562cf625aae0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb1980057a0 tx=0x7fb198026430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6739685c3ae8cc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:18.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.088+0000 7fb18cff9640 1 -- 172.21.15.116:0/3728309222 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb198010070 con 0x562cf6250b60 2024-09-12T03:08:18.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.088+0000 7fb18cff9640 1 -- 172.21.15.116:0/3728309222 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb198004020 con 0x562cf6250b60 2024-09-12T03:08:18.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.088+0000 7fb18cff9640 1 -- 172.21.15.116:0/3728309222 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1980035f0 con 0x562cf6250b60 2024-09-12T03:08:18.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.089+0000 7fb1af87e740 1 -- 172.21.15.116:0/3728309222 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562cf6250b60 msgr2=0x562cf625aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:18.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.089+0000 7fb1af87e740 1 --2- 172.21.15.116:0/3728309222 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562cf6250b60 0x562cf625aae0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb1980057a0 tx=0x7fb198026430 comp rx=0 tx=0).stop 2024-09-12T03:08:18.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.089+0000 7fb1af87e740 1 -- 172.21.15.116:0/3728309222 shutdown_connections 2024-09-12T03:08:18.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.089+0000 7fb1af87e740 1 --2- 172.21.15.116:0/3728309222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cf625bd40 0x562cf6264410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.089+0000 7fb1af87e740 1 --2- 172.21.15.116:0/3728309222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562cf6251480 0x562cf6263ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.089+0000 7fb1af87e740 1 --2- 172.21.15.116:0/3728309222 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562cf6250b60 0x562cf625aae0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.089+0000 7fb1af87e740 1 -- 172.21.15.116:0/3728309222 >> 172.21.15.116:0/3728309222 conn(0x562cf624d0f0 msgr2=0x562cf625a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:18.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.089+0000 7fb1af87e740 1 -- 172.21.15.116:0/3728309222 shutdown_connections 2024-09-12T03:08:18.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.089+0000 7fb1af87e740 1 -- 172.21.15.116:0/3728309222 wait complete. 2024-09-12T03:08:18.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.089+0000 7fb1af87e740 1 -- start start 2024-09-12T03:08:18.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.089+0000 7fb1af87e740 1 Processor -- start 2024-09-12T03:08:18.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.090+0000 7fb1af87e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cf6251480 0x562cf6256230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:18.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.090+0000 7fb1af87e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562cf625bd40 0x562cf6256770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:18.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.090+0000 7fb1af87e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562cf6254ed0 0x562cf6255350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:18.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.090+0000 7fb1af87e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a81aaae0 con 0x562cf6254ed0 2024-09-12T03:08:18.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.090+0000 7fb1af87e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a81abf50 con 0x562cf625bd40 2024-09-12T03:08:18.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.090+0000 7fb1af87e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a81ab1b0 con 0x562cf6251480 2024-09-12T03:08:18.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.090+0000 7fb1af87e740 10 client.0 _ll_register_callbacks cb 0x7fb19800b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:18.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.090+0000 7fb1a77fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562cf6254ed0 0x562cf6255350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:18.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.090+0000 7fb1a77fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562cf6254ed0 0x562cf6255350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57822/0 (socket says 172.21.15.116:57822) 2024-09-12T03:08:18.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.090+0000 7fb1a77fe640 1 -- 172.21.15.116:0/2906724675 learned_addr learned my addr 172.21.15.116:0/2906724675 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:18.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.090+0000 7fb1a67fc640 1 --2- 172.21.15.116:0/2906724675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562cf625bd40 0x562cf6256770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:18.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.090+0000 7fb1a6ffd640 1 --2- 172.21.15.116:0/2906724675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cf6251480 0x562cf6256230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:18.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.090+0000 7fb1a77fe640 1 -- 172.21.15.116:0/2906724675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cf6251480 msgr2=0x562cf6256230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:18.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.090+0000 7fb1a77fe640 1 --2- 172.21.15.116:0/2906724675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cf6251480 0x562cf6256230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.090+0000 7fb1a77fe640 1 -- 172.21.15.116:0/2906724675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562cf625bd40 msgr2=0x562cf6256770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:18.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.090+0000 7fb1a77fe640 1 --2- 172.21.15.116:0/2906724675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562cf625bd40 0x562cf6256770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.090+0000 7fb1a77fe640 1 -- 172.21.15.116:0/2906724675 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cf6255a90 con 0x562cf6254ed0 2024-09-12T03:08:18.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.090+0000 7fb1a6ffd640 1 --2- 172.21.15.116:0/2906724675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cf6251480 0x562cf6256230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:18.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.090+0000 7fb1a77fe640 1 --2- 172.21.15.116:0/2906724675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562cf6254ed0 0x562cf6255350 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0009cd0 tx=0x7fb1a0031dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0208818cf271fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:18.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.090+0000 7fb18aff5640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:18.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.091+0000 7fb18aff5640 1 -- 172.21.15.116:0/2906724675 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a004e030 con 0x562cf6254ed0 2024-09-12T03:08:18.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.091+0000 7fb18aff5640 1 -- 172.21.15.116:0/2906724675 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fb1a0044070 con 0x562cf6254ed0 2024-09-12T03:08:18.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.091+0000 7fb1af87e740 1 -- 172.21.15.116:0/2906724675 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb198004020 con 0x562cf6254ed0 2024-09-12T03:08:18.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.091+0000 7fb18aff5640 1 -- 172.21.15.116:0/2906724675 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb1a0059020 con 0x562cf6254ed0 2024-09-12T03:08:18.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.091+0000 7fb1af87e740 10 client.16656 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:18.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.091+0000 7fb1af87e740 1 -- 172.21.15.116:0/2906724675 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562cf62d96e0 con 0x562cf6254ed0 2024-09-12T03:08:18.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.091+0000 7fb18aff5640 1 -- 172.21.15.116:0/2906724675 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb198004020 con 0x562cf6254ed0 2024-09-12T03:08:18.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.091+0000 7fb18aff5640 1 -- 172.21.15.116:0/2906724675 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb1a0047050 con 0x562cf6254ed0 2024-09-12T03:08:18.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.091+0000 7fb1af87e740 10 client.16656 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:18.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.091+0000 7fb1af87e740 10 client.16656 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:08:18.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.091+0000 7fb1af87e740 20 client.16656 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:18.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.091+0000 7fb1af87e740 1 --2- 172.21.15.116:0/2906724675 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562cf62d9eb0 0x562cf62dc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:18.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.091+0000 7fb1af87e740 4 client.16656 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:08:18.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.091+0000 7fb1af87e740 1 -- 172.21.15.116:0/2906724675 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562cf61bdd00 con 0x562cf62d9eb0 2024-09-12T03:08:18.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.091+0000 7fb1a6ffd640 1 --2- 172.21.15.116:0/2906724675 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562cf62d9eb0 0x562cf62dc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:18.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.092+0000 7fb1a6ffd640 1 --2- 172.21.15.116:0/2906724675 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562cf62d9eb0 0x562cf62dc2a0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8df2106b66621eb8 server_cookie=bb54bbe882a9180d in_seq=0 out_seq=0 2024-09-12T03:08:18.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.092+0000 7fb18aff5640 10 client.16656 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:18.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.093+0000 7fb18aff5640 1 -- 172.21.15.116:0/2906724675 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562cf61bdd00 con 0x562cf62d9eb0 2024-09-12T03:08:18.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.093+0000 7fb18aff5640 10 client.16656 handle_command_reply: tid=0 2024-09-12T03:08:18.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.093+0000 7fb1af87e740 1 client.16656 shutdown 2024-09-12T03:08:18.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.093+0000 7fb1af87e740 1 -- 172.21.15.116:0/2906724675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562cf6254ed0 msgr2=0x562cf6255350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:18.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.093+0000 7fb1af87e740 1 --2- 172.21.15.116:0/2906724675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562cf6254ed0 0x562cf6255350 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0009cd0 tx=0x7fb1a0031dc0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.093+0000 7fb1af87e740 1 -- 172.21.15.116:0/2906724675 shutdown_connections 2024-09-12T03:08:18.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.093+0000 7fb1af87e740 1 --2- 172.21.15.116:0/2906724675 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562cf62d9eb0 0x562cf62dc2a0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.093+0000 7fb1af87e740 1 --2- 172.21.15.116:0/2906724675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562cf6254ed0 0x562cf6255350 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.093+0000 7fb1af87e740 1 --2- 172.21.15.116:0/2906724675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562cf625bd40 0x562cf6256770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.093+0000 7fb1af87e740 1 --2- 172.21.15.116:0/2906724675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562cf6251480 0x562cf6256230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.093+0000 7fb1af87e740 1 -- 172.21.15.116:0/2906724675 >> 172.21.15.116:0/2906724675 conn(0x562cf624d0f0 msgr2=0x562cf624c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:18.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.093+0000 7fb18aff5640 0 client.16656 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:18.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.093+0000 7fb1af87e740 1 -- 172.21.15.116:0/2906724675 shutdown_connections 2024-09-12T03:08:18.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.093+0000 7fb1af87e740 1 -- 172.21.15.116:0/2906724675 wait complete. 2024-09-12T03:08:18.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.093+0000 7fb1af87e740 20 client.16656 trim_cache size 0 max 16384 2024-09-12T03:08:18.099 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:08:18.099 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:08:18.099 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "dc8997ad-33f8-4e93-8001-593c4d77cf10", 2024-09-12T03:08:18.099 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:08:18.099 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:08:18.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.094+0000 7fb1ad4a5640 1 -- 172.21.15.116:0/2759517967 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb1900a2b80 msgr2=0x7fb1900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:18.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.094+0000 7fb1ad4a5640 1 --2- 172.21.15.116:0/2759517967 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb1900a2b80 0x7fb1900a5040 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0033040 tx=0x7fb1a00095c0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.094+0000 7fb1ad4a5640 1 -- 172.21.15.116:0/2759517967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1a8101420 msgr2=0x7fb1a8197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:18.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.094+0000 7fb1ad4a5640 1 --2- 172.21.15.116:0/2759517967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1a8101420 0x7fb1a8197840 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00d950 tx=0x7fb19c00de10 comp rx=0 tx=0).stop 2024-09-12T03:08:18.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.094+0000 7fb1ad4a5640 1 -- 172.21.15.116:0/2759517967 shutdown_connections 2024-09-12T03:08:18.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.094+0000 7fb1ad4a5640 1 --2- 172.21.15.116:0/2759517967 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb1900a2b80 0x7fb1900a5040 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.094+0000 7fb1ad4a5640 1 --2- 172.21.15.116:0/2759517967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1a8106a80 0x7fb1a819bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.094+0000 7fb1ad4a5640 1 --2- 172.21.15.116:0/2759517967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1a8101420 0x7fb1a8197840 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.094+0000 7fb1ad4a5640 1 --2- 172.21.15.116:0/2759517967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1a8100a80 0x7fb1a8197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:18.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.094+0000 7fb1ad4a5640 1 -- 172.21.15.116:0/2759517967 >> 172.21.15.116:0/2759517967 conn(0x7fb1a80fc7a0 msgr2=0x7fb1a80fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:18.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.094+0000 7fb1ad4a5640 1 -- 172.21.15.116:0/2759517967 shutdown_connections 2024-09-12T03:08:18.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:18.094+0000 7fb1ad4a5640 1 -- 172.21.15.116:0/2759517967 wait complete. 2024-09-12T03:08:18.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:18 smithi116 ceph-mon[31410]: pgmap v497: 97 pgs: 97 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:08:18.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:18 smithi183 ceph-mon[35535]: pgmap v497: 97 pgs: 97 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:08:18.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:18 smithi177 ceph-mon[35305]: pgmap v497: 97 pgs: 97 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:08:18.924 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:08:19.176 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:08:19.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:19 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:08:19.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:19 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:08:19.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:19 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:08:19.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:19 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:08:19.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:19 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:08:19.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.675+0000 7fa2ff557640 1 -- 172.21.15.116:0/1906264487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2f8106a80 msgr2=0x7fa2f8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:19.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.675+0000 7fa2ff557640 1 --2- 172.21.15.116:0/1906264487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2f8106a80 0x7fa2f8106e60 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec009930 tx=0x7fa2ec02a420 comp rx=0 tx=0).stop 2024-09-12T03:08:19.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.675+0000 7fa2ff557640 1 -- 172.21.15.116:0/1906264487 shutdown_connections 2024-09-12T03:08:19.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.675+0000 7fa2ff557640 1 --2- 172.21.15.116:0/1906264487 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2f8101420 0x7fa2f810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.675+0000 7fa2ff557640 1 --2- 172.21.15.116:0/1906264487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2f8100a80 0x7fa2f8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.675+0000 7fa2ff557640 1 --2- 172.21.15.116:0/1906264487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2f8106a80 0x7fa2f8106e60 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.675+0000 7fa2ff557640 1 -- 172.21.15.116:0/1906264487 >> 172.21.15.116:0/1906264487 conn(0x7fa2f80fc7a0 msgr2=0x7fa2f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:19.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.675+0000 7fa2ff557640 1 -- 172.21.15.116:0/1906264487 shutdown_connections 2024-09-12T03:08:19.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.675+0000 7fa2ff557640 1 -- 172.21.15.116:0/1906264487 wait complete. 2024-09-12T03:08:19.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.676+0000 7fa2ff557640 1 Processor -- start 2024-09-12T03:08:19.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.676+0000 7fa2ff557640 1 -- start start 2024-09-12T03:08:19.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.676+0000 7fa2ff557640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2f8100a80 0x7fa2f8197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:19.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.676+0000 7fa2ff557640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2f8101420 0x7fa2f8197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:19.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.676+0000 7fa2ff557640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2f8106a80 0x7fa2f819bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:19.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.676+0000 7fa2ff557640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f8110e90 con 0x7fa2f8101420 2024-09-12T03:08:19.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.676+0000 7fa2ff557640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f8110d10 con 0x7fa2f8106a80 2024-09-12T03:08:19.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.676+0000 7fa2ff557640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f8111010 con 0x7fa2f8100a80 2024-09-12T03:08:19.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.676+0000 7fa2fcaf2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2f8101420 0x7fa2f8197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:19.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.676+0000 7fa2fcaf2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2f8101420 0x7fa2f8197850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57844/0 (socket says 172.21.15.116:57844) 2024-09-12T03:08:19.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.676+0000 7fa2fcaf2640 1 -- 172.21.15.116:0/1970561620 learned_addr learned my addr 172.21.15.116:0/1970561620 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:19.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.676+0000 7fa2fd2f3640 1 --2- 172.21.15.116:0/1970561620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2f8100a80 0x7fa2f8197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:19.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.676+0000 7fa2fdaf4640 1 --2- 172.21.15.116:0/1970561620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2f8106a80 0x7fa2f819bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:19.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.676+0000 7fa2fcaf2640 1 -- 172.21.15.116:0/1970561620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2f8100a80 msgr2=0x7fa2f8197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:19.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.676+0000 7fa2fcaf2640 1 --2- 172.21.15.116:0/1970561620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2f8100a80 0x7fa2f8197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.676+0000 7fa2fcaf2640 1 -- 172.21.15.116:0/1970561620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2f8106a80 msgr2=0x7fa2f819bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:19.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.676+0000 7fa2fcaf2640 1 --2- 172.21.15.116:0/1970561620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2f8106a80 0x7fa2f819bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.676+0000 7fa2fcaf2640 1 -- 172.21.15.116:0/1970561620 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f819c2c0 con 0x7fa2f8101420 2024-09-12T03:08:19.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.676+0000 7fa2fdaf4640 1 --2- 172.21.15.116:0/1970561620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2f8106a80 0x7fa2f819bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:08:19.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.677+0000 7fa2fcaf2640 1 --2- 172.21.15.116:0/1970561620 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2f8101420 0x7fa2f8197850 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa2f400c920 tx=0x7fa2f400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:19.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.677+0000 7fa2ea7fc640 1 -- 172.21.15.116:0/1970561620 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4007c70 con 0x7fa2f8101420 2024-09-12T03:08:19.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.677+0000 7fa2ea7fc640 1 -- 172.21.15.116:0/1970561620 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2f4007e10 con 0x7fa2f8101420 2024-09-12T03:08:19.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.677+0000 7fa2ea7fc640 1 -- 172.21.15.116:0/1970561620 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4018890 con 0x7fa2f8101420 2024-09-12T03:08:19.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.677+0000 7fa2ff557640 1 -- 172.21.15.116:0/1970561620 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2f819c5b0 con 0x7fa2f8101420 2024-09-12T03:08:19.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.677+0000 7fa2ff557640 1 -- 172.21.15.116:0/1970561620 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2f81a3e90 con 0x7fa2f8101420 2024-09-12T03:08:19.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.679+0000 7fa2ea7fc640 1 -- 172.21.15.116:0/1970561620 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa2f4018a70 con 0x7fa2f8101420 2024-09-12T03:08:19.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.680+0000 7fa301930740 1 Processor -- start 2024-09-12T03:08:19.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.680+0000 7fa301930740 1 -- start start 2024-09-12T03:08:19.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.680+0000 7fa301930740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7fa5d76b0 0x55e7fa5d7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:19.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.680+0000 7fa301930740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e7fa5d7fd0 0x55e7fa5e2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:19.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.680+0000 7fa301930740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7fa5e2890 0x55e7fa5e4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:19.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.680+0000 7fa301930740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e7fa53b2e0 con 0x55e7fa5d76b0 2024-09-12T03:08:19.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.680+0000 7fa301930740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e7fa4e2ce0 con 0x55e7fa5e2890 2024-09-12T03:08:19.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.680+0000 7fa301930740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e7fa5372a0 con 0x55e7fa5d7fd0 2024-09-12T03:08:19.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.680+0000 7fa2ea7fc640 1 --2- 172.21.15.116:0/1970561620 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa2e40a2c00 0x7fa2e40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:19.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.680+0000 7fa2fdaf4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7fa5d76b0 0x55e7fa5d7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:19.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.680+0000 7fa2fdaf4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7fa5d76b0 0x55e7fa5d7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57848/0 (socket says 172.21.15.116:57848) 2024-09-12T03:08:19.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.680+0000 7fa2fdaf4640 1 -- 172.21.15.116:0/2281093176 learned_addr learned my addr 172.21.15.116:0/2281093176 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:19.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.680+0000 7fa2fcaf2640 1 --2- 172.21.15.116:0/2281093176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7fa5e2890 0x55e7fa5e4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:19.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.680+0000 7fa2fdaf4640 1 --2- 172.21.15.116:0/1970561620 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa2e40a2c00 0x7fa2e40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:19.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.680+0000 7fa2fd2f3640 1 --2- 172.21.15.116:0/2281093176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e7fa5d7fd0 0x55e7fa5e2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:19.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.680+0000 7fa2ea7fc640 1 -- 172.21.15.116:0/1970561620 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa2f40cf1e0 con 0x7fa2f8101420 2024-09-12T03:08:19.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.680+0000 7fa2fdaf4640 1 -- 172.21.15.116:0/2281093176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e7fa5d7fd0 msgr2=0x55e7fa5e2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:19.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.680+0000 7fa2fdaf4640 1 --2- 172.21.15.116:0/2281093176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e7fa5d7fd0 0x55e7fa5e2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.680+0000 7fa2fdaf4640 1 -- 172.21.15.116:0/2281093176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7fa5e2890 msgr2=0x55e7fa5e4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:19.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.681+0000 7fa2fdaf4640 1 --2- 172.21.15.116:0/2281093176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7fa5e2890 0x55e7fa5e4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.681+0000 7fa2fdaf4640 1 -- 172.21.15.116:0/2281093176 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7fa4c7440 con 0x55e7fa5d76b0 2024-09-12T03:08:19.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.681+0000 7fa2fcaf2640 1 --2- 172.21.15.116:0/2281093176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7fa5e2890 0x55e7fa5e4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:08:19.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.681+0000 7fa2fdaf4640 1 --2- 172.21.15.116:0/2281093176 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7fa5d76b0 0x55e7fa5d7a90 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0012d00 tx=0x7fa2f000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae643857b1d21ec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:19.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.681+0000 7fa2fdaf4640 1 --2- 172.21.15.116:0/1970561620 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa2e40a2c00 0x7fa2e40a50c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000fc30 tx=0x7fa2f001a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:19.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.681+0000 7fa2e37fe640 1 -- 172.21.15.116:0/2281093176 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0020070 con 0x55e7fa5d76b0 2024-09-12T03:08:19.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.681+0000 7fa2e37fe640 1 -- 172.21.15.116:0/2281093176 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2f000ab20 con 0x55e7fa5d76b0 2024-09-12T03:08:19.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.681+0000 7fa2e37fe640 1 -- 172.21.15.116:0/2281093176 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f000d630 con 0x55e7fa5d76b0 2024-09-12T03:08:19.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.682+0000 7fa301930740 1 -- 172.21.15.116:0/2281093176 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7fa5d76b0 msgr2=0x55e7fa5d7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:19.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.682+0000 7fa301930740 1 --2- 172.21.15.116:0/2281093176 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7fa5d76b0 0x55e7fa5d7a90 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0012d00 tx=0x7fa2f000f730 comp rx=0 tx=0).stop 2024-09-12T03:08:19.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.682+0000 7fa301930740 1 -- 172.21.15.116:0/2281093176 shutdown_connections 2024-09-12T03:08:19.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.682+0000 7fa301930740 1 --2- 172.21.15.116:0/2281093176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7fa5e2890 0x55e7fa5e4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.682+0000 7fa301930740 1 --2- 172.21.15.116:0/2281093176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e7fa5d7fd0 0x55e7fa5e2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.682+0000 7fa301930740 1 --2- 172.21.15.116:0/2281093176 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7fa5d76b0 0x55e7fa5d7a90 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.682+0000 7fa301930740 1 -- 172.21.15.116:0/2281093176 >> 172.21.15.116:0/2281093176 conn(0x55e7fa5d3c40 msgr2=0x55e7fa5d4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:19.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.682+0000 7fa301930740 1 -- 172.21.15.116:0/2281093176 shutdown_connections 2024-09-12T03:08:19.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.682+0000 7fa301930740 1 -- 172.21.15.116:0/2281093176 wait complete. 2024-09-12T03:08:19.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.682+0000 7fa301930740 1 -- start start 2024-09-12T03:08:19.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.682+0000 7fa301930740 1 Processor -- start 2024-09-12T03:08:19.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.683+0000 7fa301930740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e7fa5d76b0 0x55e7fa5e2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:19.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.683+0000 7fa301930740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7fa5d7fd0 0x55e7fa5df4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:19.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.683+0000 7fa301930740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7fa5e2890 0x55e7fa5dfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:19.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.683+0000 7fa301930740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e7fa5e5460 con 0x55e7fa5d7fd0 2024-09-12T03:08:19.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.683+0000 7fa301930740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e7fa4f6050 con 0x55e7fa5e2890 2024-09-12T03:08:19.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.683+0000 7fa301930740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e7fa5e55e0 con 0x55e7fa5d76b0 2024-09-12T03:08:19.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.683+0000 7fa301930740 10 client.0 _ll_register_callbacks cb 0x55e7fa536160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:19.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.683+0000 7fa2fcaf2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7fa5d7fd0 0x55e7fa5df4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:19.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.683+0000 7fa2fcaf2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7fa5d7fd0 0x55e7fa5df4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57862/0 (socket says 172.21.15.116:57862) 2024-09-12T03:08:19.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.683+0000 7fa2fcaf2640 1 -- 172.21.15.116:0/4073160361 learned_addr learned my addr 172.21.15.116:0/4073160361 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:19.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.683+0000 7fa2fd2f3640 1 --2- 172.21.15.116:0/4073160361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e7fa5d76b0 0x55e7fa5e2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:19.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.683+0000 7fa2fdaf4640 1 --2- 172.21.15.116:0/4073160361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7fa5e2890 0x55e7fa5dfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:19.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.683+0000 7fa2fcaf2640 1 -- 172.21.15.116:0/4073160361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e7fa5d76b0 msgr2=0x55e7fa5e2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:19.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.683+0000 7fa2fcaf2640 1 --2- 172.21.15.116:0/4073160361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e7fa5d76b0 0x55e7fa5e2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.683+0000 7fa2fcaf2640 1 -- 172.21.15.116:0/4073160361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7fa5e2890 msgr2=0x55e7fa5dfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:19.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.683+0000 7fa2fcaf2640 1 --2- 172.21.15.116:0/4073160361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7fa5e2890 0x55e7fa5dfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.683+0000 7fa2fcaf2640 1 -- 172.21.15.116:0/4073160361 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7fa2ac3c0 con 0x55e7fa5d7fd0 2024-09-12T03:08:19.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.683+0000 7fa2fd2f3640 1 --2- 172.21.15.116:0/4073160361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e7fa5d76b0 0x55e7fa5e2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:19.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.683+0000 7fa2fcaf2640 1 --2- 172.21.15.116:0/4073160361 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7fa5d7fd0 0x55e7fa5df4b0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4097ed0 tx=0x7fa2f4015aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b639c1fe3b1ec2f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:19.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.683+0000 7fa2e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:19.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.683+0000 7fa2e0ff9640 1 -- 172.21.15.116:0/4073160361 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f40b3020 con 0x55e7fa5d7fd0 2024-09-12T03:08:19.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.683+0000 7fa2e0ff9640 1 -- 172.21.15.116:0/4073160361 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa2f400de00 con 0x55e7fa5d7fd0 2024-09-12T03:08:19.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.683+0000 7fa301930740 1 -- 172.21.15.116:0/4073160361 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2f000ab20 con 0x55e7fa5d7fd0 2024-09-12T03:08:19.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.683+0000 7fa2e0ff9640 1 -- 172.21.15.116:0/4073160361 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa2f409ba70 con 0x55e7fa5d7fd0 2024-09-12T03:08:19.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.683+0000 7fa301930740 10 client.16668 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:19.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.683+0000 7fa301930740 1 -- 172.21.15.116:0/4073160361 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e7fa5f0e90 con 0x55e7fa5d7fd0 2024-09-12T03:08:19.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.684+0000 7fa2e0ff9640 1 -- 172.21.15.116:0/4073160361 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2f000ab20 con 0x55e7fa5d7fd0 2024-09-12T03:08:19.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.684+0000 7fa2e0ff9640 1 -- 172.21.15.116:0/4073160361 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa2f40a4250 con 0x55e7fa5d7fd0 2024-09-12T03:08:19.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.684+0000 7fa301930740 10 client.16668 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:19.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.684+0000 7fa301930740 10 client.16668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:08:19.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.684+0000 7fa301930740 20 client.16668 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:19.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.684+0000 7fa301930740 1 --2- 172.21.15.116:0/4073160361 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e7fa5f2350 0x55e7fa5f4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:19.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.684+0000 7fa301930740 4 client.16668 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:08:19.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.684+0000 7fa301930740 1 -- 172.21.15.116:0/4073160361 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7fa548850 con 0x55e7fa5f2350 2024-09-12T03:08:19.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.684+0000 7fa2fd2f3640 1 --2- 172.21.15.116:0/4073160361 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e7fa5f2350 0x55e7fa5f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:19.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.684+0000 7fa2e0ff9640 10 client.16668 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:19.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.684+0000 7fa2fd2f3640 1 --2- 172.21.15.116:0/4073160361 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e7fa5f2350 0x55e7fa5f4740 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26dfc3e6bdee3f4d server_cookie=cb20e86c8e7550d in_seq=0 out_seq=0 2024-09-12T03:08:19.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.685+0000 7fa2e0ff9640 1 -- 172.21.15.116:0/4073160361 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e7fa548850 con 0x55e7fa5f2350 2024-09-12T03:08:19.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.685+0000 7fa2e0ff9640 10 client.16668 handle_command_reply: tid=0 2024-09-12T03:08:19.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.685+0000 7fa301930740 1 client.16668 shutdown 2024-09-12T03:08:19.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.685+0000 7fa301930740 1 -- 172.21.15.116:0/4073160361 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7fa5d7fd0 msgr2=0x55e7fa5df4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:19.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.685+0000 7fa301930740 1 --2- 172.21.15.116:0/4073160361 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7fa5d7fd0 0x55e7fa5df4b0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4097ed0 tx=0x7fa2f4015aa0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.685+0000 7fa301930740 1 -- 172.21.15.116:0/4073160361 shutdown_connections 2024-09-12T03:08:19.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.685+0000 7fa301930740 1 --2- 172.21.15.116:0/4073160361 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e7fa5f2350 0x55e7fa5f4740 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.685+0000 7fa301930740 1 --2- 172.21.15.116:0/4073160361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7fa5e2890 0x55e7fa5dfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.685+0000 7fa301930740 1 --2- 172.21.15.116:0/4073160361 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7fa5d7fd0 0x55e7fa5df4b0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.685+0000 7fa301930740 1 --2- 172.21.15.116:0/4073160361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e7fa5d76b0 0x55e7fa5e2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.685+0000 7fa2e0ff9640 0 client.16668 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:19.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.685+0000 7fa301930740 1 -- 172.21.15.116:0/4073160361 >> 172.21.15.116:0/4073160361 conn(0x55e7fa5d3c40 msgr2=0x55e7fa5d0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:19.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.685+0000 7fa301930740 1 -- 172.21.15.116:0/4073160361 shutdown_connections 2024-09-12T03:08:19.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.685+0000 7fa301930740 1 -- 172.21.15.116:0/4073160361 wait complete. 2024-09-12T03:08:19.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.686+0000 7fa301930740 20 client.16668 trim_cache size 0 max 16384 2024-09-12T03:08:19.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:19 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:08:19.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:19 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:08:19.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:19 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:08:19.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:19 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:08:19.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:19 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:08:19.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.703+0000 7fa301930740 1 Processor -- start 2024-09-12T03:08:19.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.703+0000 7fa301930740 1 -- start start 2024-09-12T03:08:19.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.703+0000 7fa301930740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e7fa5d76b0 0x55e7fa5ee6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:19.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.703+0000 7fa301930740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7fa5d7fd0 0x55e7fa5f9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:19.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.703+0000 7fa301930740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7fa5e2890 0x55e7fa5fa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:19.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.703+0000 7fa301930740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f81adb90 con 0x55e7fa5e2890 2024-09-12T03:08:19.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.703+0000 7fa301930740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f81ab460 con 0x55e7fa5d7fd0 2024-09-12T03:08:19.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.703+0000 7fa301930740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f81a92d0 con 0x55e7fa5d76b0 2024-09-12T03:08:19.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.703+0000 7fa2fdaf4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7fa5e2890 0x55e7fa5fa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:19.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.703+0000 7fa2fdaf4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7fa5e2890 0x55e7fa5fa440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57864/0 (socket says 172.21.15.116:57864) 2024-09-12T03:08:19.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.703+0000 7fa2fdaf4640 1 -- 172.21.15.116:0/745108842 learned_addr learned my addr 172.21.15.116:0/745108842 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:19.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.703+0000 7fa2fd2f3640 1 --2- 172.21.15.116:0/745108842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e7fa5d76b0 0x55e7fa5ee6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:19.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.703+0000 7fa2fcaf2640 1 --2- 172.21.15.116:0/745108842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7fa5d7fd0 0x55e7fa5f9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:19.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.703+0000 7fa2fdaf4640 1 -- 172.21.15.116:0/745108842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e7fa5d76b0 msgr2=0x55e7fa5ee6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:19.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.703+0000 7fa2fdaf4640 1 --2- 172.21.15.116:0/745108842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e7fa5d76b0 0x55e7fa5ee6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.703+0000 7fa2fdaf4640 1 -- 172.21.15.116:0/745108842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7fa5d7fd0 msgr2=0x55e7fa5f9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:19.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.703+0000 7fa2fdaf4640 1 --2- 172.21.15.116:0/745108842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7fa5d7fd0 0x55e7fa5f9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.703+0000 7fa2fdaf4640 1 -- 172.21.15.116:0/745108842 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7fa5f4ab0 con 0x55e7fa5e2890 2024-09-12T03:08:19.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.703+0000 7fa2fcaf2640 1 --2- 172.21.15.116:0/745108842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7fa5d7fd0 0x55e7fa5f9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:08:19.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.703+0000 7fa2fdaf4640 1 --2- 172.21.15.116:0/745108842 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7fa5e2890 0x55e7fa5fa440 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000fff0 tx=0x7fa2f0004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ebd0dae00ddf4d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:19.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.704+0000 7fa2e2ffd640 1 -- 172.21.15.116:0/745108842 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0032070 con 0x55e7fa5e2890 2024-09-12T03:08:19.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.704+0000 7fa2e2ffd640 1 -- 172.21.15.116:0/745108842 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2f000ab20 con 0x55e7fa5e2890 2024-09-12T03:08:19.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.704+0000 7fa2e2ffd640 1 -- 172.21.15.116:0/745108842 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0020040 con 0x55e7fa5e2890 2024-09-12T03:08:19.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.704+0000 7fa301930740 1 -- 172.21.15.116:0/745108842 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7fa5e2890 msgr2=0x55e7fa5fa440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:19.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.704+0000 7fa301930740 1 --2- 172.21.15.116:0/745108842 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7fa5e2890 0x55e7fa5fa440 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000fff0 tx=0x7fa2f0004a00 comp rx=0 tx=0).stop 2024-09-12T03:08:19.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.704+0000 7fa301930740 1 -- 172.21.15.116:0/745108842 shutdown_connections 2024-09-12T03:08:19.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.704+0000 7fa301930740 1 --2- 172.21.15.116:0/745108842 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7fa5e2890 0x55e7fa5fa440 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.704+0000 7fa301930740 1 --2- 172.21.15.116:0/745108842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7fa5d7fd0 0x55e7fa5f9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.704+0000 7fa301930740 1 --2- 172.21.15.116:0/745108842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e7fa5d76b0 0x55e7fa5ee6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.704+0000 7fa301930740 1 -- 172.21.15.116:0/745108842 >> 172.21.15.116:0/745108842 conn(0x55e7fa5d3c40 msgr2=0x55e7fa5dab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:19.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.704+0000 7fa301930740 1 -- 172.21.15.116:0/745108842 shutdown_connections 2024-09-12T03:08:19.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.704+0000 7fa301930740 1 -- 172.21.15.116:0/745108842 wait complete. 2024-09-12T03:08:19.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.705+0000 7fa301930740 1 -- start start 2024-09-12T03:08:19.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.705+0000 7fa301930740 1 Processor -- start 2024-09-12T03:08:19.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.705+0000 7fa301930740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7fa5d76b0 0x55e7fa5eae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:19.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.705+0000 7fa301930740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7fa5d7fd0 0x55e7fa5e61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:19.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.705+0000 7fa301930740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e7fa5e2890 0x55e7fa5e66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:19.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.705+0000 7fa301930740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f81a9920 con 0x55e7fa5d76b0 2024-09-12T03:08:19.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.705+0000 7fa301930740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f81aad90 con 0x55e7fa5d7fd0 2024-09-12T03:08:19.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.705+0000 7fa301930740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f81a9ff0 con 0x55e7fa5e2890 2024-09-12T03:08:19.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.705+0000 7fa301930740 10 client.0 _ll_register_callbacks cb 0x7fa2f400c870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:19.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.705+0000 7fa2fd2f3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7fa5d76b0 0x55e7fa5eae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:19.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.705+0000 7fa2fd2f3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7fa5d76b0 0x55e7fa5eae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57872/0 (socket says 172.21.15.116:57872) 2024-09-12T03:08:19.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.705+0000 7fa2fd2f3640 1 -- 172.21.15.116:0/2926938922 learned_addr learned my addr 172.21.15.116:0/2926938922 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:19.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.706+0000 7fa2fdaf4640 1 --2- 172.21.15.116:0/2926938922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e7fa5e2890 0x55e7fa5e66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:19.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.706+0000 7fa2fcaf2640 1 --2- 172.21.15.116:0/2926938922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7fa5d7fd0 0x55e7fa5e61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:19.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.706+0000 7fa2fd2f3640 1 -- 172.21.15.116:0/2926938922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e7fa5e2890 msgr2=0x55e7fa5e66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:19.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.706+0000 7fa2fd2f3640 1 --2- 172.21.15.116:0/2926938922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e7fa5e2890 0x55e7fa5e66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.706+0000 7fa2fd2f3640 1 -- 172.21.15.116:0/2926938922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7fa5d7fd0 msgr2=0x55e7fa5e61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:19.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.706+0000 7fa2fd2f3640 1 --2- 172.21.15.116:0/2926938922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7fa5d7fd0 0x55e7fa5e61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.706+0000 7fa2fd2f3640 1 -- 172.21.15.116:0/2926938922 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7fa5e2570 con 0x55e7fa5d76b0 2024-09-12T03:08:19.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.706+0000 7fa2fdaf4640 1 --2- 172.21.15.116:0/2926938922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e7fa5e2890 0x55e7fa5e66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:19.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.706+0000 7fa2fcaf2640 1 --2- 172.21.15.116:0/2926938922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7fa5d7fd0 0x55e7fa5e61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:19.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.706+0000 7fa2fd2f3640 1 --2- 172.21.15.116:0/2926938922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7fa5d76b0 0x55e7fa5eae80 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec005ab0 tx=0x7fa2ec02aae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ae78dc0c48e214 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:19.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.706+0000 7fa2e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:19.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.706+0000 7fa2e0ff9640 1 -- 172.21.15.116:0/2926938922 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec048070 con 0x55e7fa5d76b0 2024-09-12T03:08:19.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.706+0000 7fa2e0ff9640 1 -- 172.21.15.116:0/2926938922 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa2ec031a80 con 0x55e7fa5d76b0 2024-09-12T03:08:19.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.706+0000 7fa301930740 1 -- 172.21.15.116:0/2926938922 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2f000ab20 con 0x55e7fa5d76b0 2024-09-12T03:08:19.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.706+0000 7fa2e0ff9640 1 -- 172.21.15.116:0/2926938922 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa2ec035040 con 0x55e7fa5d76b0 2024-09-12T03:08:19.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.706+0000 7fa301930740 10 client.16674 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:19.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.706+0000 7fa301930740 1 -- 172.21.15.116:0/2926938922 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e7fa65cbf0 con 0x55e7fa5d76b0 2024-09-12T03:08:19.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.706+0000 7fa2e0ff9640 1 -- 172.21.15.116:0/2926938922 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2f000ab20 con 0x55e7fa5d76b0 2024-09-12T03:08:19.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.707+0000 7fa2e0ff9640 1 -- 172.21.15.116:0/2926938922 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa2ec004b00 con 0x55e7fa5d76b0 2024-09-12T03:08:19.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.707+0000 7fa301930740 10 client.16674 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:19.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.707+0000 7fa301930740 10 client.16674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:08:19.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.707+0000 7fa301930740 20 client.16674 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:19.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.707+0000 7fa301930740 1 --2- 172.21.15.116:0/2926938922 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e7fa65d2c0 0x55e7fa65f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:19.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.707+0000 7fa301930740 4 client.16674 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:08:19.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.707+0000 7fa301930740 1 -- 172.21.15.116:0/2926938922 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e7fa542d60 con 0x55e7fa65d2c0 2024-09-12T03:08:19.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.707+0000 7fa2fcaf2640 1 --2- 172.21.15.116:0/2926938922 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e7fa65d2c0 0x55e7fa65f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:19.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.707+0000 7fa2e0ff9640 10 client.16674 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:19.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.707+0000 7fa2fcaf2640 1 --2- 172.21.15.116:0/2926938922 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e7fa65d2c0 0x55e7fa65f6b0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c721636b2589daad server_cookie=b0c9c7045b2d9755 in_seq=0 out_seq=0 2024-09-12T03:08:19.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.707+0000 7fa2e0ff9640 1 -- 172.21.15.116:0/2926938922 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e7fa542d60 con 0x55e7fa65d2c0 2024-09-12T03:08:19.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.707+0000 7fa2e0ff9640 10 client.16674 handle_command_reply: tid=0 2024-09-12T03:08:19.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.707+0000 7fa301930740 1 client.16674 shutdown 2024-09-12T03:08:19.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.708+0000 7fa301930740 1 -- 172.21.15.116:0/2926938922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7fa5d76b0 msgr2=0x55e7fa5eae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:19.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.708+0000 7fa301930740 1 --2- 172.21.15.116:0/2926938922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7fa5d76b0 0x55e7fa5eae80 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec005ab0 tx=0x7fa2ec02aae0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.708+0000 7fa301930740 1 -- 172.21.15.116:0/2926938922 shutdown_connections 2024-09-12T03:08:19.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.708+0000 7fa301930740 1 --2- 172.21.15.116:0/2926938922 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e7fa65d2c0 0x55e7fa65f6b0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.708+0000 7fa301930740 1 --2- 172.21.15.116:0/2926938922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e7fa5e2890 0x55e7fa5e66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.708+0000 7fa301930740 1 --2- 172.21.15.116:0/2926938922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e7fa5d7fd0 0x55e7fa5e61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.708+0000 7fa301930740 1 --2- 172.21.15.116:0/2926938922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7fa5d76b0 0x55e7fa5eae80 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.708+0000 7fa301930740 1 -- 172.21.15.116:0/2926938922 >> 172.21.15.116:0/2926938922 conn(0x55e7fa5d3c40 msgr2=0x55e7fa5da280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:19.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.708+0000 7fa2e0ff9640 0 client.16674 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:19.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.708+0000 7fa301930740 1 -- 172.21.15.116:0/2926938922 shutdown_connections 2024-09-12T03:08:19.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.708+0000 7fa301930740 1 -- 172.21.15.116:0/2926938922 wait complete. 2024-09-12T03:08:19.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.708+0000 7fa301930740 20 client.16674 trim_cache size 0 max 16384 2024-09-12T03:08:19.715 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:08:19.715 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:08:19.715 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:08:19.715 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:08:19.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.708+0000 7fa2ff557640 1 -- 172.21.15.116:0/1970561620 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa2e40a2c00 msgr2=0x7fa2e40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:19.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.708+0000 7fa2ff557640 1 --2- 172.21.15.116:0/1970561620 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa2e40a2c00 0x7fa2e40a50c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000fc30 tx=0x7fa2f001a000 comp rx=0 tx=0).stop 2024-09-12T03:08:19.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.708+0000 7fa2ff557640 1 -- 172.21.15.116:0/1970561620 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2f8101420 msgr2=0x7fa2f8197850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:19.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.708+0000 7fa2ff557640 1 --2- 172.21.15.116:0/1970561620 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2f8101420 0x7fa2f8197850 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa2f400c920 tx=0x7fa2f400cde0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.708+0000 7fa2ff557640 1 -- 172.21.15.116:0/1970561620 shutdown_connections 2024-09-12T03:08:19.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.708+0000 7fa2ff557640 1 --2- 172.21.15.116:0/1970561620 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa2e40a2c00 0x7fa2e40a50c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.708+0000 7fa2ff557640 1 --2- 172.21.15.116:0/1970561620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2f8106a80 0x7fa2f819bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.708+0000 7fa2ff557640 1 --2- 172.21.15.116:0/1970561620 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2f8101420 0x7fa2f8197850 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.708+0000 7fa2ff557640 1 --2- 172.21.15.116:0/1970561620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2f8100a80 0x7fa2f8197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:19.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.708+0000 7fa2ff557640 1 -- 172.21.15.116:0/1970561620 >> 172.21.15.116:0/1970561620 conn(0x7fa2f80fc7a0 msgr2=0x7fa2f80fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:19.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.709+0000 7fa2ff557640 1 -- 172.21.15.116:0/1970561620 shutdown_connections 2024-09-12T03:08:19.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:19.709+0000 7fa2ff557640 1 -- 172.21.15.116:0/1970561620 wait complete. 2024-09-12T03:08:19.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:19 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:08:19.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:19 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:08:19.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:19 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:08:19.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:19 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:08:19.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:19 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:08:20.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:20 smithi116 ceph-mon[31410]: pgmap v498: 97 pgs: 97 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:08:20.626 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:08:20.626 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:08:20.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:20 smithi183 ceph-mon[35535]: pgmap v498: 97 pgs: 97 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:08:20.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:20 smithi177 ceph-mon[35305]: pgmap v498: 97 pgs: 97 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:08:20.875 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:08:21.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.326+0000 7f5b0c86d640 1 -- 172.21.15.116:0/2100834624 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b0810c170 msgr2=0x7f5b0810e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:21.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.326+0000 7f5b0c86d640 1 --2- 172.21.15.116:0/2100834624 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b0810c170 0x7f5b0810e5f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009ca0 tx=0x7f5b0002a950 comp rx=0 tx=0).stop 2024-09-12T03:08:21.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.326+0000 7f5b0c86d640 1 -- 172.21.15.116:0/2100834624 shutdown_connections 2024-09-12T03:08:21.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.326+0000 7f5b0c86d640 1 --2- 172.21.15.116:0/2100834624 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b0810c170 0x7f5b0810e5f0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:21.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.326+0000 7f5b0c86d640 1 --2- 172.21.15.116:0/2100834624 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b081036b0 0x7f5b0810bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:21.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.326+0000 7f5b0c86d640 1 --2- 172.21.15.116:0/2100834624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b08102d90 0x7f5b08103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:21.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.326+0000 7f5b0c86d640 1 -- 172.21.15.116:0/2100834624 >> 172.21.15.116:0/2100834624 conn(0x7f5b080fc790 msgr2=0x7f5b080febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:21.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.326+0000 7f5b0c86d640 1 -- 172.21.15.116:0/2100834624 shutdown_connections 2024-09-12T03:08:21.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.326+0000 7f5b0c86d640 1 -- 172.21.15.116:0/2100834624 wait complete. 2024-09-12T03:08:21.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.327+0000 7f5b0c86d640 1 Processor -- start 2024-09-12T03:08:21.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.327+0000 7f5b0c86d640 1 -- start start 2024-09-12T03:08:21.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.327+0000 7f5b0c86d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b08102d90 0x7f5b08198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:21.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.327+0000 7f5b0c86d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b081036b0 0x7f5b081988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:21.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.327+0000 7f5b0c86d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b0810c170 0x7f5b0819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:21.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.327+0000 7f5b0c86d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5b08111f10 con 0x7f5b0810c170 2024-09-12T03:08:21.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.327+0000 7f5b0c86d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5b08111d90 con 0x7f5b081036b0 2024-09-12T03:08:21.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.327+0000 7f5b0c86d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5b08112090 con 0x7f5b08102d90 2024-09-12T03:08:21.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.327+0000 7f5b06d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b0810c170 0x7f5b0819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:21.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.328+0000 7f5b06d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b0810c170 0x7f5b0819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52730/0 (socket says 172.21.15.116:52730) 2024-09-12T03:08:21.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.328+0000 7f5b06d9d640 1 -- 172.21.15.116:0/933706229 learned_addr learned my addr 172.21.15.116:0/933706229 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:21.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.328+0000 7f5afffff640 1 --2- 172.21.15.116:0/933706229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b081036b0 0x7f5b081988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:21.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.328+0000 7f5b0659c640 1 --2- 172.21.15.116:0/933706229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b08102d90 0x7f5b08198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:21.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.328+0000 7f5b06d9d640 1 -- 172.21.15.116:0/933706229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b08102d90 msgr2=0x7f5b08198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:21.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.328+0000 7f5b06d9d640 1 --2- 172.21.15.116:0/933706229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b08102d90 0x7f5b08198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:21.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.328+0000 7f5b06d9d640 1 -- 172.21.15.116:0/933706229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b081036b0 msgr2=0x7f5b081988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:21.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.328+0000 7f5b06d9d640 1 --2- 172.21.15.116:0/933706229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b081036b0 0x7f5b081988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:21.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.328+0000 7f5b06d9d640 1 -- 172.21.15.116:0/933706229 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b0819d320 con 0x7f5b0810c170 2024-09-12T03:08:21.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.328+0000 7f5b0659c640 1 --2- 172.21.15.116:0/933706229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b08102d90 0x7f5b08198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:08:21.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.328+0000 7f5afffff640 1 --2- 172.21.15.116:0/933706229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b081036b0 0x7f5b081988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:21.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.328+0000 7f5b06d9d640 1 --2- 172.21.15.116:0/933706229 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b0810c170 0x7f5b0819cc40 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f5b00006010 tx=0x7f5b0002f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:21.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.328+0000 7f5aff7fe640 1 -- 172.21.15.116:0/933706229 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00034040 con 0x7f5b0810c170 2024-09-12T03:08:21.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.328+0000 7f5b0c86d640 1 -- 172.21.15.116:0/933706229 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b0819d5b0 con 0x7f5b0810c170 2024-09-12T03:08:21.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.328+0000 7f5aff7fe640 1 -- 172.21.15.116:0/933706229 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b00003e00 con 0x7f5b0810c170 2024-09-12T03:08:21.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.328+0000 7f5aff7fe640 1 -- 172.21.15.116:0/933706229 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b000456e0 con 0x7f5b0810c170 2024-09-12T03:08:21.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.328+0000 7f5b0c86d640 1 -- 172.21.15.116:0/933706229 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b081a4ef0 con 0x7f5b0810c170 2024-09-12T03:08:21.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.329+0000 7f5b0c86d640 1 -- 172.21.15.116:0/933706229 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b08103170 con 0x7f5b0810c170 2024-09-12T03:08:21.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.331+0000 7f5aff7fe640 1 -- 172.21.15.116:0/933706229 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f5b00037020 con 0x7f5b0810c170 2024-09-12T03:08:21.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.331+0000 7f5aff7fe640 1 --2- 172.21.15.116:0/933706229 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5af40a2c00 0x7f5af40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:21.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.331+0000 7f5aff7fe640 1 -- 172.21.15.116:0/933706229 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f5b000f01f0 con 0x7f5b0810c170 2024-09-12T03:08:21.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.331+0000 7f5b0659c640 1 --2- 172.21.15.116:0/933706229 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5af40a2c00 0x7f5af40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:21.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.332+0000 7f5b0659c640 1 --2- 172.21.15.116:0/933706229 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5af40a2c00 0x7f5af40a50c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5af800baa0 tx=0x7f5af8005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:21.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.335+0000 7f5aff7fe640 1 -- 172.21.15.116:0/933706229 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5b000bd180 con 0x7f5b0810c170 2024-09-12T03:08:21.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.541+0000 7f5b0c86d640 1 -- 172.21.15.116:0/933706229 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b081a5150 con 0x7f5b0810c170 2024-09-12T03:08:21.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.542+0000 7f5aff7fe640 1 -- 172.21.15.116:0/933706229 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f5b000bca80 con 0x7f5b0810c170 2024-09-12T03:08:21.542 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:08:21.542 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:08:21.542 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:08:21.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.544+0000 7f5b0c86d640 1 -- 172.21.15.116:0/933706229 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5af40a2c00 msgr2=0x7f5af40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:21.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.544+0000 7f5b0c86d640 1 --2- 172.21.15.116:0/933706229 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5af40a2c00 0x7f5af40a50c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5af800baa0 tx=0x7f5af8005d90 comp rx=0 tx=0).stop 2024-09-12T03:08:21.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.544+0000 7f5b0c86d640 1 -- 172.21.15.116:0/933706229 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b0810c170 msgr2=0x7f5b0819cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:21.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.544+0000 7f5b0c86d640 1 --2- 172.21.15.116:0/933706229 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b0810c170 0x7f5b0819cc40 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f5b00006010 tx=0x7f5b0002f710 comp rx=0 tx=0).stop 2024-09-12T03:08:21.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.544+0000 7f5b0c86d640 1 -- 172.21.15.116:0/933706229 shutdown_connections 2024-09-12T03:08:21.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.544+0000 7f5b0c86d640 1 --2- 172.21.15.116:0/933706229 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5af40a2c00 0x7f5af40a50c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:21.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.544+0000 7f5b0c86d640 1 --2- 172.21.15.116:0/933706229 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b0810c170 0x7f5b0819cc40 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:21.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.544+0000 7f5b0c86d640 1 --2- 172.21.15.116:0/933706229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b081036b0 0x7f5b081988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:21.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.544+0000 7f5b0c86d640 1 --2- 172.21.15.116:0/933706229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b08102d90 0x7f5b08198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:21.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.544+0000 7f5b0c86d640 1 -- 172.21.15.116:0/933706229 >> 172.21.15.116:0/933706229 conn(0x7f5b080fc790 msgr2=0x7f5b080fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:21.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.544+0000 7f5b0c86d640 1 -- 172.21.15.116:0/933706229 shutdown_connections 2024-09-12T03:08:21.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:21.544+0000 7f5b0c86d640 1 -- 172.21.15.116:0/933706229 wait complete. 2024-09-12T03:08:22.371 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:08:22.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:22 smithi116 ceph-mon[31410]: pgmap v499: 97 pgs: 97 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:08:22.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:22 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/933706229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:08:22.618 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:08:22.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:22 smithi183 ceph-mon[35535]: pgmap v499: 97 pgs: 97 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:08:22.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:22 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/933706229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:08:22.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:22 smithi177 ceph-mon[35305]: pgmap v499: 97 pgs: 97 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:08:22.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:22 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/933706229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:08:23.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.127+0000 7f1caa2ac640 1 -- 172.21.15.116:0/59987696 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ca4100e20 msgr2=0x7f1ca4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:23.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.127+0000 7f1caa2ac640 1 --2- 172.21.15.116:0/59987696 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ca4100e20 0x7f1ca4101200 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f1c94009900 tx=0x7f1c9402a480 comp rx=0 tx=0).stop 2024-09-12T03:08:23.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.127+0000 7f1caa2ac640 1 -- 172.21.15.116:0/59987696 shutdown_connections 2024-09-12T03:08:23.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.127+0000 7f1caa2ac640 1 --2- 172.21.15.116:0/59987696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ca4102ef0 0x7f1ca410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.127+0000 7f1caa2ac640 1 --2- 172.21.15.116:0/59987696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ca4102570 0x7f1ca41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.127+0000 7f1caa2ac640 1 --2- 172.21.15.116:0/59987696 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ca4100e20 0x7f1ca4101200 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.127+0000 7f1caa2ac640 1 -- 172.21.15.116:0/59987696 >> 172.21.15.116:0/59987696 conn(0x7f1ca40fc7d0 msgr2=0x7f1ca40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:23.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.127+0000 7f1caa2ac640 1 -- 172.21.15.116:0/59987696 shutdown_connections 2024-09-12T03:08:23.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.127+0000 7f1caa2ac640 1 -- 172.21.15.116:0/59987696 wait complete. 2024-09-12T03:08:23.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.127+0000 7f1caa2ac640 1 Processor -- start 2024-09-12T03:08:23.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.128+0000 7f1caa2ac640 1 -- start start 2024-09-12T03:08:23.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.128+0000 7f1caa2ac640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ca4100e20 0x7f1ca41994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:23.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.128+0000 7f1caa2ac640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ca4102570 0x7f1ca4199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:23.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.128+0000 7f1caa2ac640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ca4102ef0 0x7f1ca41a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:23.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.128+0000 7f1caa2ac640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca4111020 con 0x7f1ca4100e20 2024-09-12T03:08:23.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.128+0000 7f1caa2ac640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca4110ea0 con 0x7f1ca4102ef0 2024-09-12T03:08:23.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.128+0000 7f1caa2ac640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca41111a0 con 0x7f1ca4102570 2024-09-12T03:08:23.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.128+0000 7f1ca3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ca4100e20 0x7f1ca41994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:23.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.128+0000 7f1ca3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ca4100e20 0x7f1ca41994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52752/0 (socket says 172.21.15.116:52752) 2024-09-12T03:08:23.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.128+0000 7f1ca3fff640 1 -- 172.21.15.116:0/3978443376 learned_addr learned my addr 172.21.15.116:0/3978443376 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:23.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.128+0000 7f1ca8849640 1 --2- 172.21.15.116:0/3978443376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ca4102ef0 0x7f1ca41a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:23.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.128+0000 7f1ca3fff640 1 -- 172.21.15.116:0/3978443376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ca4102570 msgr2=0x7f1ca4199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:23.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.128+0000 7f1ca3fff640 1 --2- 172.21.15.116:0/3978443376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ca4102570 0x7f1ca4199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.128+0000 7f1ca3fff640 1 -- 172.21.15.116:0/3978443376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ca4102ef0 msgr2=0x7f1ca41a4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:23.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.128+0000 7f1ca3fff640 1 --2- 172.21.15.116:0/3978443376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ca4102ef0 0x7f1ca41a4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.128+0000 7f1ca3fff640 1 -- 172.21.15.116:0/3978443376 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca41a4ab0 con 0x7f1ca4100e20 2024-09-12T03:08:23.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.128+0000 7f1ca8849640 1 --2- 172.21.15.116:0/3978443376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ca4102ef0 0x7f1ca41a4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:08:23.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.129+0000 7f1ca3fff640 1 --2- 172.21.15.116:0/3978443376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ca4100e20 0x7f1ca41994f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f1c94006010 tx=0x7f1c94003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:23.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.129+0000 7f1ca1ffb640 1 -- 172.21.15.116:0/3978443376 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9403f070 con 0x7f1ca4100e20 2024-09-12T03:08:23.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.129+0000 7f1ca1ffb640 1 -- 172.21.15.116:0/3978443376 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c94031e30 con 0x7f1ca4100e20 2024-09-12T03:08:23.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.129+0000 7f1caa2ac640 1 -- 172.21.15.116:0/3978443376 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca41a4c80 con 0x7f1ca4100e20 2024-09-12T03:08:23.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.129+0000 7f1ca1ffb640 1 -- 172.21.15.116:0/3978443376 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c94035030 con 0x7f1ca4100e20 2024-09-12T03:08:23.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.129+0000 7f1caa2ac640 1 -- 172.21.15.116:0/3978443376 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca41a50d0 con 0x7f1ca4100e20 2024-09-12T03:08:23.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.131+0000 7f1ca1ffb640 1 -- 172.21.15.116:0/3978443376 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1c94042070 con 0x7f1ca4100e20 2024-09-12T03:08:23.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.131+0000 7f1cac685740 1 Processor -- start 2024-09-12T03:08:23.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.131+0000 7f1cac685740 1 -- start start 2024-09-12T03:08:23.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.131+0000 7f1cac685740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac7a226f50 0x55ac7a2254a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:23.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.131+0000 7f1cac685740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac7a2259e0 0x55ac7a225e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:23.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.131+0000 7f1cac685740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac7a230780 0x55ac7a232b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:23.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.131+0000 7f1cac685740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ac7a1872f0 con 0x55ac7a2259e0 2024-09-12T03:08:23.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.131+0000 7f1cac685740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ac7a12ecf0 con 0x55ac7a226f50 2024-09-12T03:08:23.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.131+0000 7f1cac685740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ac7a1832b0 con 0x55ac7a230780 2024-09-12T03:08:23.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.132+0000 7f1ca3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac7a2259e0 0x55ac7a225e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:23.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.132+0000 7f1ca8849640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac7a226f50 0x55ac7a2254a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:23.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.132+0000 7f1ca3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac7a2259e0 0x55ac7a225e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52762/0 (socket says 172.21.15.116:52762) 2024-09-12T03:08:23.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.132+0000 7f1ca3fff640 1 -- 172.21.15.116:0/3874232276 learned_addr learned my addr 172.21.15.116:0/3874232276 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:23.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.132+0000 7f1ca1ffb640 1 --2- 172.21.15.116:0/3978443376 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1c900a2b80 0x7f1c900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:23.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.132+0000 7f1c9bfff640 1 --2- 172.21.15.116:0/3874232276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac7a230780 0x55ac7a232b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:23.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.132+0000 7f1ca8849640 1 --2- 172.21.15.116:0/3978443376 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1c900a2b80 0x7f1c900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:23.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.132+0000 7f1ca1ffb640 1 -- 172.21.15.116:0/3978443376 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1c940efa10 con 0x7f1ca4100e20 2024-09-12T03:08:23.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.132+0000 7f1ca8849640 1 -- 172.21.15.116:0/3874232276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac7a230780 msgr2=0x55ac7a232b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:23.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.132+0000 7f1ca8849640 1 --2- 172.21.15.116:0/3874232276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac7a230780 0x55ac7a232b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.132+0000 7f1ca8849640 1 -- 172.21.15.116:0/3874232276 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac7a2259e0 msgr2=0x55ac7a225e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:23.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.132+0000 7f1ca8849640 1 --2- 172.21.15.116:0/3874232276 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac7a2259e0 0x55ac7a225e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.132+0000 7f1ca8849640 1 -- 172.21.15.116:0/3874232276 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac7a1133a0 con 0x55ac7a226f50 2024-09-12T03:08:23.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.132+0000 7f1ca3fff640 1 --2- 172.21.15.116:0/3874232276 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac7a2259e0 0x55ac7a225e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:08:23.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.132+0000 7f1ca8849640 1 --2- 172.21.15.116:0/3874232276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac7a226f50 0x55ac7a2254a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c0109b0 tx=0x7f1c9c010e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af9e2fbab628fcbd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:23.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.133+0000 7f1ca8849640 1 --2- 172.21.15.116:0/3978443376 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1c900a2b80 0x7f1c900a5040 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c00a000 tx=0x7f1c9c019000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:23.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.133+0000 7f1c9a7fc640 1 -- 172.21.15.116:0/3874232276 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c020070 con 0x55ac7a226f50 2024-09-12T03:08:23.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.133+0000 7f1c9a7fc640 1 -- 172.21.15.116:0/3874232276 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c9c00cc90 con 0x55ac7a226f50 2024-09-12T03:08:23.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.133+0000 7f1c9a7fc640 1 -- 172.21.15.116:0/3874232276 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c00f690 con 0x55ac7a226f50 2024-09-12T03:08:23.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.133+0000 7f1cac685740 1 -- 172.21.15.116:0/3874232276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac7a226f50 msgr2=0x55ac7a2254a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:23.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.133+0000 7f1cac685740 1 --2- 172.21.15.116:0/3874232276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac7a226f50 0x55ac7a2254a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c0109b0 tx=0x7f1c9c010e70 comp rx=0 tx=0).stop 2024-09-12T03:08:23.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.133+0000 7f1cac685740 1 -- 172.21.15.116:0/3874232276 shutdown_connections 2024-09-12T03:08:23.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.133+0000 7f1cac685740 1 --2- 172.21.15.116:0/3874232276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac7a230780 0x55ac7a232b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.133+0000 7f1cac685740 1 --2- 172.21.15.116:0/3874232276 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac7a2259e0 0x55ac7a225e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.133+0000 7f1cac685740 1 --2- 172.21.15.116:0/3874232276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac7a226f50 0x55ac7a2254a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.133+0000 7f1cac685740 1 -- 172.21.15.116:0/3874232276 >> 172.21.15.116:0/3874232276 conn(0x55ac7a1ea310 msgr2=0x55ac7a222070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:23.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.133+0000 7f1cac685740 1 -- 172.21.15.116:0/3874232276 shutdown_connections 2024-09-12T03:08:23.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.133+0000 7f1cac685740 1 -- 172.21.15.116:0/3874232276 wait complete. 2024-09-12T03:08:23.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.134+0000 7f1cac685740 1 -- start start 2024-09-12T03:08:23.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.134+0000 7f1cac685740 1 Processor -- start 2024-09-12T03:08:23.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.134+0000 7f1cac685740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac7a2259e0 0x55ac7a230140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:23.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.134+0000 7f1cac685740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac7a226f50 0x55ac7a22d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:23.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.134+0000 7f1cac685740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac7a230780 0x55ac7a22d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:23.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.134+0000 7f1cac685740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ac7a233420 con 0x55ac7a226f50 2024-09-12T03:08:23.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.134+0000 7f1cac685740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ac7a142060 con 0x55ac7a230780 2024-09-12T03:08:23.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.134+0000 7f1cac685740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ac7a2335a0 con 0x55ac7a2259e0 2024-09-12T03:08:23.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.134+0000 7f1cac685740 10 client.0 _ll_register_callbacks cb 0x55ac79e958c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:23.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.134+0000 7f1c9bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac7a226f50 0x55ac7a22d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:23.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.134+0000 7f1c9bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac7a226f50 0x55ac7a22d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52778/0 (socket says 172.21.15.116:52778) 2024-09-12T03:08:23.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.134+0000 7f1c9bfff640 1 -- 172.21.15.116:0/1785196858 learned_addr learned my addr 172.21.15.116:0/1785196858 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:23.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.135+0000 7f1ca8849640 1 --2- 172.21.15.116:0/1785196858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac7a230780 0x55ac7a22d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:23.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.135+0000 7f1ca3fff640 1 --2- 172.21.15.116:0/1785196858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac7a2259e0 0x55ac7a230140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:23.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.135+0000 7f1c9bfff640 1 -- 172.21.15.116:0/1785196858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac7a2259e0 msgr2=0x55ac7a230140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:23.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.135+0000 7f1c9bfff640 1 --2- 172.21.15.116:0/1785196858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac7a2259e0 0x55ac7a230140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.135+0000 7f1c9bfff640 1 -- 172.21.15.116:0/1785196858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac7a230780 msgr2=0x55ac7a22d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:23.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.135+0000 7f1c9bfff640 1 --2- 172.21.15.116:0/1785196858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac7a230780 0x55ac7a22d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.135+0000 7f1c9bfff640 1 -- 172.21.15.116:0/1785196858 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac79ef83c0 con 0x55ac7a226f50 2024-09-12T03:08:23.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.135+0000 7f1ca3fff640 1 --2- 172.21.15.116:0/1785196858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac7a2259e0 0x55ac7a230140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:23.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.135+0000 7f1ca8849640 1 --2- 172.21.15.116:0/1785196858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac7a230780 0x55ac7a22d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:23.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.135+0000 7f1c9bfff640 1 --2- 172.21.15.116:0/1785196858 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac7a226f50 0x55ac7a22d3f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c00e200 tx=0x7f1c8c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2dcc2c72ba392f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:23.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.135+0000 7f1c837fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:23.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.135+0000 7f1c837fe640 1 -- 172.21.15.116:0/1785196858 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c017070 con 0x55ac7a226f50 2024-09-12T03:08:23.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.135+0000 7f1c837fe640 1 -- 172.21.15.116:0/1785196858 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1c8c013680 con 0x55ac7a226f50 2024-09-12T03:08:23.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.135+0000 7f1cac685740 1 -- 172.21.15.116:0/1785196858 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c9c00cc90 con 0x55ac7a226f50 2024-09-12T03:08:23.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.135+0000 7f1c837fe640 1 -- 172.21.15.116:0/1785196858 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1c8c023050 con 0x55ac7a226f50 2024-09-12T03:08:23.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.135+0000 7f1cac685740 10 client.16692 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:23.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.135+0000 7f1cac685740 1 -- 172.21.15.116:0/1785196858 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ac7a23ef60 con 0x55ac7a226f50 2024-09-12T03:08:23.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.135+0000 7f1c837fe640 1 -- 172.21.15.116:0/1785196858 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c9c00cc90 con 0x55ac7a226f50 2024-09-12T03:08:23.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.136+0000 7f1c837fe640 1 -- 172.21.15.116:0/1785196858 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1c8c012a10 con 0x55ac7a226f50 2024-09-12T03:08:23.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.136+0000 7f1cac685740 10 client.16692 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:23.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.136+0000 7f1cac685740 10 client.16692 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:08:23.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.136+0000 7f1cac685740 20 client.16692 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:23.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.136+0000 7f1cac685740 1 --2- 172.21.15.116:0/1785196858 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55ac7a240410 0x55ac7a242830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:23.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.136+0000 7f1cac685740 4 client.16692 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:08:23.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.136+0000 7f1cac685740 1 -- 172.21.15.116:0/1785196858 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac7a194860 con 0x55ac7a240410 2024-09-12T03:08:23.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.136+0000 7f1ca3fff640 1 --2- 172.21.15.116:0/1785196858 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55ac7a240410 0x55ac7a242830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:23.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.136+0000 7f1c837fe640 10 client.16692 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:08:23.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.136+0000 7f1ca3fff640 1 --2- 172.21.15.116:0/1785196858 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55ac7a240410 0x55ac7a242830 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fc5356c49581b8e8 server_cookie=9cfed10c46a43fb2 in_seq=0 out_seq=0 2024-09-12T03:08:23.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.137+0000 7f1c837fe640 1 -- 172.21.15.116:0/1785196858 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ac7a194860 con 0x55ac7a240410 2024-09-12T03:08:23.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.137+0000 7f1c837fe640 10 client.16692 handle_command_reply: tid=0 2024-09-12T03:08:23.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.137+0000 7f1cac685740 1 client.16692 shutdown 2024-09-12T03:08:23.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.137+0000 7f1cac685740 1 -- 172.21.15.116:0/1785196858 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac7a226f50 msgr2=0x55ac7a22d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:23.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.137+0000 7f1cac685740 1 --2- 172.21.15.116:0/1785196858 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac7a226f50 0x55ac7a22d3f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c00e200 tx=0x7f1c8c00e6c0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.137+0000 7f1cac685740 1 -- 172.21.15.116:0/1785196858 shutdown_connections 2024-09-12T03:08:23.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.137+0000 7f1cac685740 1 --2- 172.21.15.116:0/1785196858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac7a230780 0x55ac7a22d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.137+0000 7f1cac685740 1 --2- 172.21.15.116:0/1785196858 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55ac7a240410 0x55ac7a242830 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.137+0000 7f1cac685740 1 --2- 172.21.15.116:0/1785196858 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac7a226f50 0x55ac7a22d3f0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.137+0000 7f1cac685740 1 --2- 172.21.15.116:0/1785196858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac7a2259e0 0x55ac7a230140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.137+0000 7f1c837fe640 0 client.16692 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:08:23.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.137+0000 7f1cac685740 1 -- 172.21.15.116:0/1785196858 >> 172.21.15.116:0/1785196858 conn(0x55ac7a1ea310 msgr2=0x55ac7a227d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:23.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.137+0000 7f1cac685740 1 -- 172.21.15.116:0/1785196858 shutdown_connections 2024-09-12T03:08:23.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.137+0000 7f1cac685740 1 -- 172.21.15.116:0/1785196858 wait complete. 2024-09-12T03:08:23.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.137+0000 7f1cac685740 20 client.16692 trim_cache size 0 max 16384 2024-09-12T03:08:23.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.150+0000 7f1cac685740 1 Processor -- start 2024-09-12T03:08:23.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.150+0000 7f1cac685740 1 -- start start 2024-09-12T03:08:23.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.150+0000 7f1cac685740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac7a2259e0 0x55ac7a23c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:23.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.150+0000 7f1cac685740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac7a226f50 0x55ac7a248100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:23.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.150+0000 7f1cac685740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac7a230780 0x55ac7a248640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:23.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.150+0000 7f1cac685740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca41adc40 con 0x55ac7a2259e0 2024-09-12T03:08:23.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.150+0000 7f1cac685740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca41ab510 con 0x55ac7a226f50 2024-09-12T03:08:23.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.150+0000 7f1cac685740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca41a9380 con 0x55ac7a230780 2024-09-12T03:08:23.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.150+0000 7f1ca3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac7a2259e0 0x55ac7a23c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:23.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.150+0000 7f1ca3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac7a2259e0 0x55ac7a23c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52782/0 (socket says 172.21.15.116:52782) 2024-09-12T03:08:23.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.151+0000 7f1ca3fff640 1 -- 172.21.15.116:0/3252573925 learned_addr learned my addr 172.21.15.116:0/3252573925 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:23.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.151+0000 7f1ca8849640 1 --2- 172.21.15.116:0/3252573925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac7a230780 0x55ac7a248640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:23.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.151+0000 7f1c9bfff640 1 --2- 172.21.15.116:0/3252573925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac7a226f50 0x55ac7a248100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:23.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.151+0000 7f1ca3fff640 1 -- 172.21.15.116:0/3252573925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac7a230780 msgr2=0x55ac7a248640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:23.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.151+0000 7f1ca3fff640 1 --2- 172.21.15.116:0/3252573925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac7a230780 0x55ac7a248640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.151+0000 7f1ca3fff640 1 -- 172.21.15.116:0/3252573925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac7a226f50 msgr2=0x55ac7a248100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:23.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.151+0000 7f1ca3fff640 1 --2- 172.21.15.116:0/3252573925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac7a226f50 0x55ac7a248100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.151+0000 7f1ca3fff640 1 -- 172.21.15.116:0/3252573925 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac7a228530 con 0x55ac7a2259e0 2024-09-12T03:08:23.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.151+0000 7f1ca3fff640 1 --2- 172.21.15.116:0/3252573925 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac7a2259e0 0x55ac7a23c6f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f1c940ed590 tx=0x7f1c940b8670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf02fd5b3e85e012 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:23.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.151+0000 7f1c99ffb640 1 -- 172.21.15.116:0/3252573925 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c94049040 con 0x55ac7a2259e0 2024-09-12T03:08:23.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.151+0000 7f1c99ffb640 1 -- 172.21.15.116:0/3252573925 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c940bc9a0 con 0x55ac7a2259e0 2024-09-12T03:08:23.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.151+0000 7f1c99ffb640 1 -- 172.21.15.116:0/3252573925 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c940ba5a0 con 0x55ac7a2259e0 2024-09-12T03:08:23.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.152+0000 7f1cac685740 1 -- 172.21.15.116:0/3252573925 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac7a2259e0 msgr2=0x55ac7a23c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:23.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.152+0000 7f1cac685740 1 --2- 172.21.15.116:0/3252573925 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac7a2259e0 0x55ac7a23c6f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f1c940ed590 tx=0x7f1c940b8670 comp rx=0 tx=0).stop 2024-09-12T03:08:23.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.152+0000 7f1cac685740 1 -- 172.21.15.116:0/3252573925 shutdown_connections 2024-09-12T03:08:23.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.152+0000 7f1cac685740 1 --2- 172.21.15.116:0/3252573925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac7a230780 0x55ac7a248640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.152+0000 7f1cac685740 1 --2- 172.21.15.116:0/3252573925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac7a226f50 0x55ac7a248100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.152+0000 7f1cac685740 1 --2- 172.21.15.116:0/3252573925 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac7a2259e0 0x55ac7a23c6f0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.152+0000 7f1cac685740 1 -- 172.21.15.116:0/3252573925 >> 172.21.15.116:0/3252573925 conn(0x55ac7a1ea310 msgr2=0x55ac7a22c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:23.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.152+0000 7f1cac685740 1 -- 172.21.15.116:0/3252573925 shutdown_connections 2024-09-12T03:08:23.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.152+0000 7f1cac685740 1 -- 172.21.15.116:0/3252573925 wait complete. 2024-09-12T03:08:23.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.152+0000 7f1cac685740 1 -- start start 2024-09-12T03:08:23.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.152+0000 7f1cac685740 1 Processor -- start 2024-09-12T03:08:23.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.152+0000 7f1cac685740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac7a2259e0 0x55ac7a234750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:23.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.152+0000 7f1cac685740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac7a226f50 0x55ac7a234c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:23.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.152+0000 7f1cac685740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac7a230780 0x55ac7a2351d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:23.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.152+0000 7f1cac685740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca41a99d0 con 0x55ac7a226f50 2024-09-12T03:08:23.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.152+0000 7f1cac685740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca41aae40 con 0x55ac7a230780 2024-09-12T03:08:23.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.152+0000 7f1cac685740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca41aa0a0 con 0x55ac7a2259e0 2024-09-12T03:08:23.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.153+0000 7f1cac685740 10 client.0 _ll_register_callbacks cb 0x7f1c8c009340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:23.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.153+0000 7f1c9bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac7a226f50 0x55ac7a234c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:23.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.153+0000 7f1c9bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac7a226f50 0x55ac7a234c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52796/0 (socket says 172.21.15.116:52796) 2024-09-12T03:08:23.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.153+0000 7f1c9bfff640 1 -- 172.21.15.116:0/587447567 learned_addr learned my addr 172.21.15.116:0/587447567 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:23.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.153+0000 7f1ca8849640 1 --2- 172.21.15.116:0/587447567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac7a230780 0x55ac7a2351d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:23.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.153+0000 7f1ca3fff640 1 --2- 172.21.15.116:0/587447567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac7a2259e0 0x55ac7a234750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:23.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.153+0000 7f1c9bfff640 1 -- 172.21.15.116:0/587447567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac7a2259e0 msgr2=0x55ac7a234750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:23.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.153+0000 7f1c9bfff640 1 --2- 172.21.15.116:0/587447567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac7a2259e0 0x55ac7a234750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.153+0000 7f1c9bfff640 1 -- 172.21.15.116:0/587447567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac7a230780 msgr2=0x55ac7a2351d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:23.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.153+0000 7f1c9bfff640 1 --2- 172.21.15.116:0/587447567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac7a230780 0x55ac7a2351d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.153+0000 7f1c9bfff640 1 -- 172.21.15.116:0/587447567 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac7a2aa230 con 0x55ac7a226f50 2024-09-12T03:08:23.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.153+0000 7f1ca3fff640 1 --2- 172.21.15.116:0/587447567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac7a2259e0 0x55ac7a234750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:08:23.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.153+0000 7f1c9bfff640 1 --2- 172.21.15.116:0/587447567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac7a226f50 0x55ac7a234c90 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x55ac7a22ea30 tx=0x7f1c8c01b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be9de127e3dd5e47 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:23.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.153+0000 7f1c837fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:23.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.153+0000 7f1c837fe640 1 -- 172.21.15.116:0/587447567 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c017070 con 0x55ac7a226f50 2024-09-12T03:08:23.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.153+0000 7f1c837fe640 1 -- 172.21.15.116:0/587447567 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1c8c01b820 con 0x55ac7a226f50 2024-09-12T03:08:23.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.153+0000 7f1cac685740 1 -- 172.21.15.116:0/587447567 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c940bc9a0 con 0x55ac7a226f50 2024-09-12T03:08:23.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.154+0000 7f1c837fe640 1 -- 172.21.15.116:0/587447567 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1c8c00ebc0 con 0x55ac7a226f50 2024-09-12T03:08:23.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.154+0000 7f1cac685740 10 client.16698 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:23.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.154+0000 7f1cac685740 1 -- 172.21.15.116:0/587447567 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ac7a2ab4f0 con 0x55ac7a226f50 2024-09-12T03:08:23.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.154+0000 7f1c837fe640 1 -- 172.21.15.116:0/587447567 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c940bc9a0 con 0x55ac7a226f50 2024-09-12T03:08:23.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.154+0000 7f1c837fe640 1 -- 172.21.15.116:0/587447567 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1c8c013b40 con 0x55ac7a226f50 2024-09-12T03:08:23.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.154+0000 7f1cac685740 10 client.16698 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:23.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.154+0000 7f1cac685740 10 client.16698 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:08:23.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.154+0000 7f1cac685740 20 client.16698 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:23.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.154+0000 7f1cac685740 1 --2- 172.21.15.116:0/587447567 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55ac7a2abc40 0x55ac7a2ae060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:23.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.154+0000 7f1cac685740 4 client.16698 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:08:23.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.154+0000 7f1cac685740 1 -- 172.21.15.116:0/587447567 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac7a18ed70 con 0x55ac7a2abc40 2024-09-12T03:08:23.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.154+0000 7f1ca3fff640 1 --2- 172.21.15.116:0/587447567 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55ac7a2abc40 0x55ac7a2ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:23.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.155+0000 7f1c837fe640 10 client.16698 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:08:23.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.155+0000 7f1ca3fff640 1 --2- 172.21.15.116:0/587447567 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55ac7a2abc40 0x55ac7a2ae060 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ab5dc62d05870cfa server_cookie=b1865ba711bfb68f in_seq=0 out_seq=0 2024-09-12T03:08:23.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.155+0000 7f1c837fe640 1 -- 172.21.15.116:0/587447567 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac7a18ed70 con 0x55ac7a2abc40 2024-09-12T03:08:23.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.155+0000 7f1c837fe640 10 client.16698 handle_command_reply: tid=0 2024-09-12T03:08:23.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.155+0000 7f1cac685740 1 client.16698 shutdown 2024-09-12T03:08:23.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.155+0000 7f1cac685740 1 -- 172.21.15.116:0/587447567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac7a226f50 msgr2=0x55ac7a234c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:23.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.155+0000 7f1cac685740 1 --2- 172.21.15.116:0/587447567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac7a226f50 0x55ac7a234c90 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x55ac7a22ea30 tx=0x7f1c8c01b6c0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.155+0000 7f1cac685740 1 -- 172.21.15.116:0/587447567 shutdown_connections 2024-09-12T03:08:23.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.155+0000 7f1cac685740 1 --2- 172.21.15.116:0/587447567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac7a230780 0x55ac7a2351d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.155+0000 7f1cac685740 1 --2- 172.21.15.116:0/587447567 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55ac7a2abc40 0x55ac7a2ae060 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.155+0000 7f1cac685740 1 --2- 172.21.15.116:0/587447567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac7a226f50 0x55ac7a234c90 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.155+0000 7f1cac685740 1 --2- 172.21.15.116:0/587447567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac7a2259e0 0x55ac7a234750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.155+0000 7f1cac685740 1 -- 172.21.15.116:0/587447567 >> 172.21.15.116:0/587447567 conn(0x55ac7a1ea310 msgr2=0x55ac7a21c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:23.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.155+0000 7f1c837fe640 0 client.16698 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:08:23.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.155+0000 7f1cac685740 1 -- 172.21.15.116:0/587447567 shutdown_connections 2024-09-12T03:08:23.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.155+0000 7f1cac685740 1 -- 172.21.15.116:0/587447567 wait complete. 2024-09-12T03:08:23.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.155+0000 7f1cac685740 20 client.16698 trim_cache size 0 max 16384 2024-09-12T03:08:23.173 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:08:23.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.156+0000 7f1caa2ac640 1 -- 172.21.15.116:0/3978443376 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1c900a2b80 msgr2=0x7f1c900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:23.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.156+0000 7f1caa2ac640 1 --2- 172.21.15.116:0/3978443376 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1c900a2b80 0x7f1c900a5040 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c00a000 tx=0x7f1c9c019000 comp rx=0 tx=0).stop 2024-09-12T03:08:23.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.156+0000 7f1caa2ac640 1 -- 172.21.15.116:0/3978443376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ca4100e20 msgr2=0x7f1ca41994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:23.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.156+0000 7f1caa2ac640 1 --2- 172.21.15.116:0/3978443376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ca4100e20 0x7f1ca41994f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f1c94006010 tx=0x7f1c94003c80 comp rx=0 tx=0).stop 2024-09-12T03:08:23.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.156+0000 7f1caa2ac640 1 -- 172.21.15.116:0/3978443376 shutdown_connections 2024-09-12T03:08:23.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.156+0000 7f1caa2ac640 1 --2- 172.21.15.116:0/3978443376 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1c900a2b80 0x7f1c900a5040 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.156+0000 7f1caa2ac640 1 --2- 172.21.15.116:0/3978443376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ca4102ef0 0x7f1ca41a4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.156+0000 7f1caa2ac640 1 --2- 172.21.15.116:0/3978443376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ca4102570 0x7f1ca4199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.156+0000 7f1caa2ac640 1 --2- 172.21.15.116:0/3978443376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ca4100e20 0x7f1ca41994f0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:23.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.156+0000 7f1caa2ac640 1 -- 172.21.15.116:0/3978443376 >> 172.21.15.116:0/3978443376 conn(0x7f1ca40fc7d0 msgr2=0x7f1ca40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:23.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.156+0000 7f1caa2ac640 1 -- 172.21.15.116:0/3978443376 shutdown_connections 2024-09-12T03:08:23.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:23.156+0000 7f1caa2ac640 1 -- 172.21.15.116:0/3978443376 wait complete. 2024-09-12T03:08:23.955 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:08:24.203 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:08:24.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:24 smithi116 ceph-mon[31410]: pgmap v500: 97 pgs: 97 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:08:24.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:24 smithi183 ceph-mon[35535]: pgmap v500: 97 pgs: 97 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:08:24.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:24 smithi177 ceph-mon[35305]: pgmap v500: 97 pgs: 97 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:08:24.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.734+0000 7f61af410640 1 -- 172.21.15.116:0/2790174213 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61a8100a80 msgr2=0x7f61a8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:24.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.734+0000 7f61af410640 1 --2- 172.21.15.116:0/2790174213 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61a8100a80 0x7f61a8100ee0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6198009930 tx=0x7f619802a420 comp rx=0 tx=0).stop 2024-09-12T03:08:24.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.734+0000 7f61af410640 1 -- 172.21.15.116:0/2790174213 shutdown_connections 2024-09-12T03:08:24.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.734+0000 7f61af410640 1 --2- 172.21.15.116:0/2790174213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61a8101420 0x7f61a810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.734+0000 7f61af410640 1 --2- 172.21.15.116:0/2790174213 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61a8100a80 0x7f61a8100ee0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.734+0000 7f61af410640 1 --2- 172.21.15.116:0/2790174213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61a8106a80 0x7f61a8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.734+0000 7f61af410640 1 -- 172.21.15.116:0/2790174213 >> 172.21.15.116:0/2790174213 conn(0x7f61a80fc7a0 msgr2=0x7f61a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:24.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.735+0000 7f61af410640 1 -- 172.21.15.116:0/2790174213 shutdown_connections 2024-09-12T03:08:24.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.735+0000 7f61af410640 1 -- 172.21.15.116:0/2790174213 wait complete. 2024-09-12T03:08:24.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.735+0000 7f61af410640 1 Processor -- start 2024-09-12T03:08:24.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.735+0000 7f61af410640 1 -- start start 2024-09-12T03:08:24.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.735+0000 7f61af410640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61a8100a80 0x7f61a8102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:24.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.735+0000 7f61af410640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61a8101420 0x7f61a8102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:24.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.735+0000 7f61af410640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61a8106a80 0x7f61a81030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:24.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.735+0000 7f61af410640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f61a8110e80 con 0x7f61a8106a80 2024-09-12T03:08:24.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.735+0000 7f61af410640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f61a8110d00 con 0x7f61a8100a80 2024-09-12T03:08:24.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.735+0000 7f61af410640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f61a8111000 con 0x7f61a8101420 2024-09-12T03:08:24.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.735+0000 7f61ad9ad640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61a8106a80 0x7f61a81030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:24.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.736+0000 7f61ad9ad640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61a8106a80 0x7f61a81030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52824/0 (socket says 172.21.15.116:52824) 2024-09-12T03:08:24.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.736+0000 7f61ad9ad640 1 -- 172.21.15.116:0/3517760862 learned_addr learned my addr 172.21.15.116:0/3517760862 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:24.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.736+0000 7f61ad1ac640 1 --2- 172.21.15.116:0/3517760862 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61a8100a80 0x7f61a8102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:24.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.736+0000 7f61ac9ab640 1 --2- 172.21.15.116:0/3517760862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61a8101420 0x7f61a8102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:24.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.736+0000 7f61ad9ad640 1 -- 172.21.15.116:0/3517760862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61a8101420 msgr2=0x7f61a8102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:24.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.736+0000 7f61ad9ad640 1 --2- 172.21.15.116:0/3517760862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61a8101420 0x7f61a8102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.736+0000 7f61ad9ad640 1 -- 172.21.15.116:0/3517760862 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61a8100a80 msgr2=0x7f61a8102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:24.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.736+0000 7f61ad9ad640 1 --2- 172.21.15.116:0/3517760862 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61a8100a80 0x7f61a8102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.736+0000 7f61ad9ad640 1 -- 172.21.15.116:0/3517760862 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a81a6930 con 0x7f61a8106a80 2024-09-12T03:08:24.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.736+0000 7f61ad9ad640 1 --2- 172.21.15.116:0/3517760862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61a8106a80 0x7f61a81030d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f61a000eaa0 tx=0x7f61a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:24.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.736+0000 7f619e7fc640 1 -- 172.21.15.116:0/3517760862 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a000cdb0 con 0x7f61a8106a80 2024-09-12T03:08:24.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.736+0000 7f619e7fc640 1 -- 172.21.15.116:0/3517760862 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61a0005020 con 0x7f61a8106a80 2024-09-12T03:08:24.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.736+0000 7f61af410640 1 -- 172.21.15.116:0/3517760862 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61a81a6c20 con 0x7f61a8106a80 2024-09-12T03:08:24.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.736+0000 7f619e7fc640 1 -- 172.21.15.116:0/3517760862 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a00116b0 con 0x7f61a8106a80 2024-09-12T03:08:24.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.736+0000 7f61af410640 1 -- 172.21.15.116:0/3517760862 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61a81a70e0 con 0x7f61a8106a80 2024-09-12T03:08:24.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.738+0000 7f619e7fc640 1 -- 172.21.15.116:0/3517760862 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f61a0005560 con 0x7f61a8106a80 2024-09-12T03:08:24.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.739+0000 7f61b17e9740 1 Processor -- start 2024-09-12T03:08:24.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.739+0000 7f61b17e9740 1 -- start start 2024-09-12T03:08:24.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.739+0000 7f61b17e9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd4e3dfb40 0x55fd4e3de090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:24.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.739+0000 7f61b17e9740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fd4e3de5d0 0x55fd4e3dea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:24.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.739+0000 7f61b17e9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4e3e9370 0x55fd4e3eb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:24.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.739+0000 7f61b17e9740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55fd4e3442f0 con 0x55fd4e3de5d0 2024-09-12T03:08:24.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.739+0000 7f61b17e9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55fd4e2ebcf0 con 0x55fd4e3dfb40 2024-09-12T03:08:24.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.739+0000 7f61b17e9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fd4e3402b0 con 0x55fd4e3e9370 2024-09-12T03:08:24.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.739+0000 7f61ad1ac640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fd4e3de5d0 0x55fd4e3dea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:24.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.739+0000 7f61ad1ac640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fd4e3de5d0 0x55fd4e3dea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52826/0 (socket says 172.21.15.116:52826) 2024-09-12T03:08:24.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.739+0000 7f61ad1ac640 1 -- 172.21.15.116:0/2878144065 learned_addr learned my addr 172.21.15.116:0/2878144065 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:24.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.739+0000 7f619e7fc640 1 --2- 172.21.15.116:0/3517760862 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f61940a2b80 0x7f61940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:24.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.740+0000 7f61ac9ab640 1 --2- 172.21.15.116:0/2878144065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4e3e9370 0x55fd4e3eb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:24.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.740+0000 7f61ad9ad640 1 --2- 172.21.15.116:0/3517760862 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f61940a2b80 0x7f61940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:24.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.740+0000 7f61ac9ab640 1 --2- 172.21.15.116:0/2878144065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd4e3dfb40 0x55fd4e3de090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:24.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.740+0000 7f61ad1ac640 1 -- 172.21.15.116:0/2878144065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4e3e9370 msgr2=0x55fd4e3eb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:24.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.740+0000 7f61ad1ac640 1 --2- 172.21.15.116:0/2878144065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4e3e9370 0x55fd4e3eb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.740+0000 7f61ad1ac640 1 -- 172.21.15.116:0/2878144065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd4e3dfb40 msgr2=0x55fd4e3de090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:24.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.740+0000 7f61ad1ac640 1 --2- 172.21.15.116:0/2878144065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd4e3dfb40 0x55fd4e3de090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.740+0000 7f61ad1ac640 1 -- 172.21.15.116:0/2878144065 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd4e2d03a0 con 0x55fd4e3de5d0 2024-09-12T03:08:24.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.740+0000 7f61ac9ab640 1 --2- 172.21.15.116:0/2878144065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd4e3dfb40 0x55fd4e3de090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:08:24.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.740+0000 7f619e7fc640 1 -- 172.21.15.116:0/3517760862 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f61a00cf210 con 0x7f61a8106a80 2024-09-12T03:08:24.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.740+0000 7f61ad1ac640 1 --2- 172.21.15.116:0/2878144065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fd4e3de5d0 0x55fd4e3dea10 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f61a400b710 tx=0x7f61a4004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650fb79ec182df39 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:24.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.740+0000 7f61ad9ad640 1 --2- 172.21.15.116:0/3517760862 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f61940a2b80 0x7f61940a5040 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f61a0096590 tx=0x7f61a00c65d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:24.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.740+0000 7f61937fe640 1 -- 172.21.15.116:0/2878144065 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a4003bd0 con 0x55fd4e3de5d0 2024-09-12T03:08:24.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.740+0000 7f61937fe640 1 -- 172.21.15.116:0/2878144065 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61a4015070 con 0x55fd4e3de5d0 2024-09-12T03:08:24.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.740+0000 7f61937fe640 1 -- 172.21.15.116:0/2878144065 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a4004e40 con 0x55fd4e3de5d0 2024-09-12T03:08:24.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.741+0000 7f61b17e9740 1 -- 172.21.15.116:0/2878144065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fd4e3de5d0 msgr2=0x55fd4e3dea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:24.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.741+0000 7f61b17e9740 1 --2- 172.21.15.116:0/2878144065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fd4e3de5d0 0x55fd4e3dea10 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f61a400b710 tx=0x7f61a4004610 comp rx=0 tx=0).stop 2024-09-12T03:08:24.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.741+0000 7f61b17e9740 1 -- 172.21.15.116:0/2878144065 shutdown_connections 2024-09-12T03:08:24.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.741+0000 7f61b17e9740 1 --2- 172.21.15.116:0/2878144065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4e3e9370 0x55fd4e3eb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.741+0000 7f61b17e9740 1 --2- 172.21.15.116:0/2878144065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fd4e3de5d0 0x55fd4e3dea10 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.741+0000 7f61b17e9740 1 --2- 172.21.15.116:0/2878144065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd4e3dfb40 0x55fd4e3de090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.741+0000 7f61b17e9740 1 -- 172.21.15.116:0/2878144065 >> 172.21.15.116:0/2878144065 conn(0x55fd4e3a7310 msgr2=0x55fd4e3dac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:24.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.741+0000 7f61b17e9740 1 -- 172.21.15.116:0/2878144065 shutdown_connections 2024-09-12T03:08:24.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.741+0000 7f61b17e9740 1 -- 172.21.15.116:0/2878144065 wait complete. 2024-09-12T03:08:24.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.741+0000 7f61b17e9740 1 -- start start 2024-09-12T03:08:24.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.741+0000 7f61b17e9740 1 Processor -- start 2024-09-12T03:08:24.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.742+0000 7f61b17e9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4e3de5d0 0x55fd4e3e8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:24.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.742+0000 7f61b17e9740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fd4e3dfb40 0x55fd4e3e5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:24.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.742+0000 7f61b17e9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd4e3e9370 0x55fd4e3e6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:24.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.742+0000 7f61b17e9740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55fd4e3ec070 con 0x55fd4e3dfb40 2024-09-12T03:08:24.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.742+0000 7f61b17e9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55fd4e2ff060 con 0x55fd4e3e9370 2024-09-12T03:08:24.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.742+0000 7f61b17e9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fd4e3ec1f0 con 0x55fd4e3de5d0 2024-09-12T03:08:24.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.742+0000 7f61b17e9740 10 client.0 _ll_register_callbacks cb 0x55fd4e0528c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:24.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.742+0000 7f61ad1ac640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fd4e3dfb40 0x55fd4e3e5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:24.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.742+0000 7f61ad1ac640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fd4e3dfb40 0x55fd4e3e5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52838/0 (socket says 172.21.15.116:52838) 2024-09-12T03:08:24.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.742+0000 7f61ad1ac640 1 -- 172.21.15.116:0/1124940394 learned_addr learned my addr 172.21.15.116:0/1124940394 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:24.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.742+0000 7f61ac9ab640 1 --2- 172.21.15.116:0/1124940394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4e3de5d0 0x55fd4e3e8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:24.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.742+0000 7f61ac9ab640 1 --2- 172.21.15.116:0/1124940394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd4e3e9370 0x55fd4e3e6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:24.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.742+0000 7f61ad1ac640 1 -- 172.21.15.116:0/1124940394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4e3de5d0 msgr2=0x55fd4e3e8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:24.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.742+0000 7f61ad1ac640 1 --2- 172.21.15.116:0/1124940394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4e3de5d0 0x55fd4e3e8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.742+0000 7f61ad1ac640 1 -- 172.21.15.116:0/1124940394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd4e3e9370 msgr2=0x55fd4e3e6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:24.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.742+0000 7f61ad1ac640 1 --2- 172.21.15.116:0/1124940394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd4e3e9370 0x55fd4e3e6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.742+0000 7f61ad1ac640 1 -- 172.21.15.116:0/1124940394 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd4e0b53c0 con 0x55fd4e3dfb40 2024-09-12T03:08:24.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.742+0000 7f61ac9ab640 1 --2- 172.21.15.116:0/1124940394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd4e3e9370 0x55fd4e3e6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:08:24.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.742+0000 7f61ad1ac640 1 --2- 172.21.15.116:0/1124940394 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fd4e3dfb40 0x55fd4e3e5fb0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f61a40039c0 tx=0x7f61a4004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d68d6965e1d3160d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:24.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.742+0000 7f6190ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:24.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.742+0000 7f6190ff9640 1 -- 172.21.15.116:0/1124940394 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a4028060 con 0x55fd4e3dfb40 2024-09-12T03:08:24.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.742+0000 7f6190ff9640 1 -- 172.21.15.116:0/1124940394 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f61a4011640 con 0x55fd4e3dfb40 2024-09-12T03:08:24.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.742+0000 7f61b17e9740 1 -- 172.21.15.116:0/1124940394 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61a4015070 con 0x55fd4e3dfb40 2024-09-12T03:08:24.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.743+0000 7f6190ff9640 1 -- 172.21.15.116:0/1124940394 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f61a4003ee0 con 0x55fd4e3dfb40 2024-09-12T03:08:24.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.743+0000 7f61b17e9740 10 client.16710 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:24.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.743+0000 7f61b17e9740 1 -- 172.21.15.116:0/1124940394 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fd4e3f7a10 con 0x55fd4e3dfb40 2024-09-12T03:08:24.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.743+0000 7f6190ff9640 1 -- 172.21.15.116:0/1124940394 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61a4015070 con 0x55fd4e3dfb40 2024-09-12T03:08:24.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.743+0000 7f6190ff9640 1 -- 172.21.15.116:0/1124940394 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f61a4022070 con 0x55fd4e3dfb40 2024-09-12T03:08:24.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.743+0000 7f61b17e9740 10 client.16710 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:24.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.743+0000 7f61b17e9740 10 client.16710 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:08:24.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.743+0000 7f61b17e9740 20 client.16710 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:24.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.743+0000 7f61b17e9740 1 --2- 172.21.15.116:0/1124940394 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55fd4e3f8e10 0x55fd4e3fb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:24.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.743+0000 7f61b17e9740 4 client.16710 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:08:24.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.743+0000 7f61b17e9740 1 -- 172.21.15.116:0/1124940394 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd4e351860 con 0x55fd4e3f8e10 2024-09-12T03:08:24.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.743+0000 7f61ac9ab640 1 --2- 172.21.15.116:0/1124940394 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55fd4e3f8e10 0x55fd4e3fb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:24.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.744+0000 7f6190ff9640 10 client.16710 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:08:24.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.744+0000 7f61ac9ab640 1 --2- 172.21.15.116:0/1124940394 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55fd4e3f8e10 0x55fd4e3fb200 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5a2069d5a3887ce8 server_cookie=4102ece5ca13341f in_seq=0 out_seq=0 2024-09-12T03:08:24.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.744+0000 7f6190ff9640 1 -- 172.21.15.116:0/1124940394 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fd4e351860 con 0x55fd4e3f8e10 2024-09-12T03:08:24.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.744+0000 7f6190ff9640 10 client.16710 handle_command_reply: tid=0 2024-09-12T03:08:24.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.745+0000 7f61b17e9740 1 client.16710 shutdown 2024-09-12T03:08:24.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.745+0000 7f61b17e9740 1 -- 172.21.15.116:0/1124940394 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fd4e3dfb40 msgr2=0x55fd4e3e5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:24.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.745+0000 7f61b17e9740 1 --2- 172.21.15.116:0/1124940394 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fd4e3dfb40 0x55fd4e3e5fb0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f61a40039c0 tx=0x7f61a4004800 comp rx=0 tx=0).stop 2024-09-12T03:08:24.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.745+0000 7f61b17e9740 1 -- 172.21.15.116:0/1124940394 shutdown_connections 2024-09-12T03:08:24.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.745+0000 7f61b17e9740 1 --2- 172.21.15.116:0/1124940394 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55fd4e3f8e10 0x55fd4e3fb200 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.745+0000 7f61b17e9740 1 --2- 172.21.15.116:0/1124940394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd4e3e9370 0x55fd4e3e6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.745+0000 7f61b17e9740 1 --2- 172.21.15.116:0/1124940394 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fd4e3dfb40 0x55fd4e3e5fb0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.745+0000 7f61b17e9740 1 --2- 172.21.15.116:0/1124940394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4e3de5d0 0x55fd4e3e8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.745+0000 7f61b17e9740 1 -- 172.21.15.116:0/1124940394 >> 172.21.15.116:0/1124940394 conn(0x55fd4e3a7310 msgr2=0x55fd4e3e0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:24.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.745+0000 7f6190ff9640 0 client.16710 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:08:24.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.745+0000 7f61b17e9740 1 -- 172.21.15.116:0/1124940394 shutdown_connections 2024-09-12T03:08:24.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.745+0000 7f61b17e9740 1 -- 172.21.15.116:0/1124940394 wait complete. 2024-09-12T03:08:24.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.745+0000 7f61b17e9740 20 client.16710 trim_cache size 0 max 16384 2024-09-12T03:08:24.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.758+0000 7f61b17e9740 1 Processor -- start 2024-09-12T03:08:24.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.758+0000 7f61b17e9740 1 -- start start 2024-09-12T03:08:24.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.758+0000 7f61b17e9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd4e3de5d0 0x55fd4e3dae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:24.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.758+0000 7f61b17e9740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fd4e3dfb40 0x55fd4e3f5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:24.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.758+0000 7f61b17e9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4e3e9370 0x55fd4e4007d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:24.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.758+0000 7f61b17e9740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f61a81b1fb0 con 0x55fd4e3dfb40 2024-09-12T03:08:24.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.759+0000 7f61b17e9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f61a81af880 con 0x55fd4e3de5d0 2024-09-12T03:08:24.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.759+0000 7f61b17e9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f61a81ad6d0 con 0x55fd4e3e9370 2024-09-12T03:08:24.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.759+0000 7f61ad1ac640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fd4e3dfb40 0x55fd4e3f5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:24.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.759+0000 7f61ad1ac640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fd4e3dfb40 0x55fd4e3f5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52840/0 (socket says 172.21.15.116:52840) 2024-09-12T03:08:24.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.759+0000 7f61ad1ac640 1 -- 172.21.15.116:0/697815558 learned_addr learned my addr 172.21.15.116:0/697815558 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:24.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.759+0000 7f61ac9ab640 1 --2- 172.21.15.116:0/697815558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd4e3de5d0 0x55fd4e3dae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:24.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.759+0000 7f61ac9ab640 1 --2- 172.21.15.116:0/697815558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4e3e9370 0x55fd4e4007d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:24.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.759+0000 7f61ad1ac640 1 -- 172.21.15.116:0/697815558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4e3e9370 msgr2=0x55fd4e4007d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:24.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.759+0000 7f61ad1ac640 1 --2- 172.21.15.116:0/697815558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4e3e9370 0x55fd4e4007d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.759+0000 7f61ad1ac640 1 -- 172.21.15.116:0/697815558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd4e3de5d0 msgr2=0x55fd4e3dae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:24.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.759+0000 7f61ad1ac640 1 --2- 172.21.15.116:0/697815558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd4e3de5d0 0x55fd4e3dae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.759+0000 7f61ad1ac640 1 -- 172.21.15.116:0/697815558 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd4e3d9310 con 0x55fd4e3dfb40 2024-09-12T03:08:24.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.759+0000 7f61ad1ac640 1 --2- 172.21.15.116:0/697815558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fd4e3dfb40 0x55fd4e3f5310 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f61a4004450 tx=0x7f61a4004080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65bbe5fd8f02e460 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:24.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.759+0000 7f6192ffd640 1 -- 172.21.15.116:0/697815558 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a4033070 con 0x55fd4e3dfb40 2024-09-12T03:08:24.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.759+0000 7f6192ffd640 1 -- 172.21.15.116:0/697815558 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61a4015070 con 0x55fd4e3dfb40 2024-09-12T03:08:24.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.759+0000 7f6192ffd640 1 -- 172.21.15.116:0/697815558 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a4028040 con 0x55fd4e3dfb40 2024-09-12T03:08:24.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.760+0000 7f61b17e9740 1 -- 172.21.15.116:0/697815558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fd4e3dfb40 msgr2=0x55fd4e3f5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:24.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.760+0000 7f61b17e9740 1 --2- 172.21.15.116:0/697815558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fd4e3dfb40 0x55fd4e3f5310 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f61a4004450 tx=0x7f61a4004080 comp rx=0 tx=0).stop 2024-09-12T03:08:24.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.760+0000 7f61b17e9740 1 -- 172.21.15.116:0/697815558 shutdown_connections 2024-09-12T03:08:24.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.760+0000 7f61b17e9740 1 --2- 172.21.15.116:0/697815558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4e3e9370 0x55fd4e4007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.760+0000 7f61b17e9740 1 --2- 172.21.15.116:0/697815558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fd4e3dfb40 0x55fd4e3f5310 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.760+0000 7f61b17e9740 1 --2- 172.21.15.116:0/697815558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd4e3de5d0 0x55fd4e3dae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.760+0000 7f61b17e9740 1 -- 172.21.15.116:0/697815558 >> 172.21.15.116:0/697815558 conn(0x55fd4e3a7310 msgr2=0x55fd4e3da700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:24.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.760+0000 7f61b17e9740 1 -- 172.21.15.116:0/697815558 shutdown_connections 2024-09-12T03:08:24.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.760+0000 7f61b17e9740 1 -- 172.21.15.116:0/697815558 wait complete. 2024-09-12T03:08:24.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.760+0000 7f61b17e9740 1 -- start start 2024-09-12T03:08:24.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.760+0000 7f61b17e9740 1 Processor -- start 2024-09-12T03:08:24.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.761+0000 7f61b17e9740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fd4e3de5d0 0x55fd4e3ec5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:24.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.761+0000 7f61b17e9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4e3dfb40 0x55fd4e3ecae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:24.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.761+0000 7f61b17e9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd4e3e9370 0x55fd4e461790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:24.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.761+0000 7f61b17e9740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f61a81add40 con 0x55fd4e3de5d0 2024-09-12T03:08:24.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.761+0000 7f61b17e9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f61a81af1b0 con 0x55fd4e3e9370 2024-09-12T03:08:24.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.761+0000 7f61b17e9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f61a81ae410 con 0x55fd4e3dfb40 2024-09-12T03:08:24.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.761+0000 7f61b17e9740 10 client.0 _ll_register_callbacks cb 0x7f61a4004510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:24.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.761+0000 7f61ac9ab640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fd4e3de5d0 0x55fd4e3ec5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:24.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.761+0000 7f61ac9ab640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fd4e3de5d0 0x55fd4e3ec5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52844/0 (socket says 172.21.15.116:52844) 2024-09-12T03:08:24.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.761+0000 7f61ac9ab640 1 -- 172.21.15.116:0/4280701279 learned_addr learned my addr 172.21.15.116:0/4280701279 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:24.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.761+0000 7f61ad1ac640 1 --2- 172.21.15.116:0/4280701279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4e3dfb40 0x55fd4e3ecae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:24.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.761+0000 7f61ac9ab640 1 -- 172.21.15.116:0/4280701279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4e3dfb40 msgr2=0x55fd4e3ecae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:24.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.761+0000 7f61ac9ab640 1 --2- 172.21.15.116:0/4280701279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4e3dfb40 0x55fd4e3ecae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.761+0000 7f61ac9ab640 1 -- 172.21.15.116:0/4280701279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd4e3e9370 msgr2=0x55fd4e461790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:24.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.761+0000 7f61ac9ab640 1 --2- 172.21.15.116:0/4280701279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd4e3e9370 0x55fd4e461790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.761+0000 7f61ac9ab640 1 -- 172.21.15.116:0/4280701279 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd4e461cd0 con 0x55fd4e3de5d0 2024-09-12T03:08:24.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.762+0000 7f61ac9ab640 1 --2- 172.21.15.116:0/4280701279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fd4e3de5d0 0x55fd4e3ec5a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f619802a940 tx=0x7f61980092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff87725bacaffad7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:24.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.762+0000 7f6190ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:24.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.762+0000 7f6190ff9640 1 -- 172.21.15.116:0/4280701279 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6198035040 con 0x55fd4e3de5d0 2024-09-12T03:08:24.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.762+0000 7f6190ff9640 1 -- 172.21.15.116:0/4280701279 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f6198005020 con 0x55fd4e3de5d0 2024-09-12T03:08:24.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.762+0000 7f61b17e9740 1 -- 172.21.15.116:0/4280701279 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61a4015070 con 0x55fd4e3de5d0 2024-09-12T03:08:24.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.762+0000 7f6190ff9640 1 -- 172.21.15.116:0/4280701279 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6198005220 con 0x55fd4e3de5d0 2024-09-12T03:08:24.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.762+0000 7f61b17e9740 10 client.16716 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:24.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.762+0000 7f61b17e9740 1 -- 172.21.15.116:0/4280701279 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fd4e462eb0 con 0x55fd4e3de5d0 2024-09-12T03:08:24.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.762+0000 7f6190ff9640 1 -- 172.21.15.116:0/4280701279 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61a4015070 con 0x55fd4e3de5d0 2024-09-12T03:08:24.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.762+0000 7f6190ff9640 1 -- 172.21.15.116:0/4280701279 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6198052030 con 0x55fd4e3de5d0 2024-09-12T03:08:24.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.762+0000 7f61b17e9740 10 client.16716 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:24.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.762+0000 7f61b17e9740 10 client.16716 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:08:24.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.762+0000 7f61b17e9740 20 client.16716 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:24.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.762+0000 7f61b17e9740 1 --2- 172.21.15.116:0/4280701279 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55fd4e463600 0x55fd4e465a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:24.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.762+0000 7f61b17e9740 4 client.16716 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:08:24.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.762+0000 7f61b17e9740 1 -- 172.21.15.116:0/4280701279 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fd4e34bd70 con 0x55fd4e463600 2024-09-12T03:08:24.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.763+0000 7f61ad1ac640 1 --2- 172.21.15.116:0/4280701279 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55fd4e463600 0x55fd4e465a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:24.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.763+0000 7f6190ff9640 10 client.16716 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:08:24.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.763+0000 7f61ad1ac640 1 --2- 172.21.15.116:0/4280701279 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55fd4e463600 0x55fd4e465a20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4e947ca0b2b37f18 server_cookie=ed4255db071fa7cf in_seq=0 out_seq=0 2024-09-12T03:08:24.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.763+0000 7f6190ff9640 1 -- 172.21.15.116:0/4280701279 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fd4e34bd70 con 0x55fd4e463600 2024-09-12T03:08:24.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.763+0000 7f6190ff9640 10 client.16716 handle_command_reply: tid=0 2024-09-12T03:08:24.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.763+0000 7f61b17e9740 1 client.16716 shutdown 2024-09-12T03:08:24.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.763+0000 7f61b17e9740 1 -- 172.21.15.116:0/4280701279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fd4e3de5d0 msgr2=0x55fd4e3ec5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:24.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.763+0000 7f61b17e9740 1 --2- 172.21.15.116:0/4280701279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fd4e3de5d0 0x55fd4e3ec5a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f619802a940 tx=0x7f61980092d0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.763+0000 7f61b17e9740 1 -- 172.21.15.116:0/4280701279 shutdown_connections 2024-09-12T03:08:24.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.763+0000 7f61b17e9740 1 --2- 172.21.15.116:0/4280701279 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55fd4e463600 0x55fd4e465a20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.763+0000 7f61b17e9740 1 --2- 172.21.15.116:0/4280701279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd4e3e9370 0x55fd4e461790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.763+0000 7f61b17e9740 1 --2- 172.21.15.116:0/4280701279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4e3dfb40 0x55fd4e3ecae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.763+0000 7f6190ff9640 0 client.16716 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:08:24.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.763+0000 7f61b17e9740 1 --2- 172.21.15.116:0/4280701279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fd4e3de5d0 0x55fd4e3ec5a0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.763+0000 7f61b17e9740 1 -- 172.21.15.116:0/4280701279 >> 172.21.15.116:0/4280701279 conn(0x55fd4e3a7310 msgr2=0x55fd4e3e50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:24.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.764+0000 7f61b17e9740 1 -- 172.21.15.116:0/4280701279 shutdown_connections 2024-09-12T03:08:24.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.764+0000 7f61b17e9740 1 -- 172.21.15.116:0/4280701279 wait complete. 2024-09-12T03:08:24.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.764+0000 7f61b17e9740 20 client.16716 trim_cache size 0 max 16384 2024-09-12T03:08:24.776 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:08:24.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.764+0000 7f61af410640 1 -- 172.21.15.116:0/3517760862 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f61940a2b80 msgr2=0x7f61940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:24.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.764+0000 7f61af410640 1 --2- 172.21.15.116:0/3517760862 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f61940a2b80 0x7f61940a5040 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f61a0096590 tx=0x7f61a00c65d0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.764+0000 7f61af410640 1 -- 172.21.15.116:0/3517760862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61a8106a80 msgr2=0x7f61a81030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:24.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.764+0000 7f61af410640 1 --2- 172.21.15.116:0/3517760862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61a8106a80 0x7f61a81030d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f61a000eaa0 tx=0x7f61a000ef60 comp rx=0 tx=0).stop 2024-09-12T03:08:24.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.764+0000 7f61af410640 1 -- 172.21.15.116:0/3517760862 shutdown_connections 2024-09-12T03:08:24.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.764+0000 7f61af410640 1 --2- 172.21.15.116:0/3517760862 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f61940a2b80 0x7f61940a5040 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.764+0000 7f61af410640 1 --2- 172.21.15.116:0/3517760862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61a8106a80 0x7f61a81030d0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.764+0000 7f61af410640 1 --2- 172.21.15.116:0/3517760862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61a8101420 0x7f61a8102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.764+0000 7f61af410640 1 --2- 172.21.15.116:0/3517760862 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61a8100a80 0x7f61a8102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:24.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.764+0000 7f61af410640 1 -- 172.21.15.116:0/3517760862 >> 172.21.15.116:0/3517760862 conn(0x7f61a80fc7a0 msgr2=0x7f61a8104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:24.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.764+0000 7f61af410640 1 -- 172.21.15.116:0/3517760862 shutdown_connections 2024-09-12T03:08:24.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:24.764+0000 7f61af410640 1 -- 172.21.15.116:0/3517760862 wait complete. 2024-09-12T03:08:25.577 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:08:25.825 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:08:26.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.329+0000 7fab04c29640 1 -- 172.21.15.116:0/2875865202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab00102570 msgr2=0x7fab001029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:26.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.329+0000 7fab04c29640 1 --2- 172.21.15.116:0/2875865202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab00102570 0x7fab001029b0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7faaf0009ca0 tx=0x7faaf002a950 comp rx=0 tx=0).stop 2024-09-12T03:08:26.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.329+0000 7fab04c29640 1 -- 172.21.15.116:0/2875865202 shutdown_connections 2024-09-12T03:08:26.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.329+0000 7fab04c29640 1 --2- 172.21.15.116:0/2875865202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab00102ef0 0x7fab0010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.329+0000 7fab04c29640 1 --2- 172.21.15.116:0/2875865202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab00102570 0x7fab001029b0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.329+0000 7fab04c29640 1 --2- 172.21.15.116:0/2875865202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab00100e20 0x7fab00101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.329+0000 7fab04c29640 1 -- 172.21.15.116:0/2875865202 >> 172.21.15.116:0/2875865202 conn(0x7fab000fc7d0 msgr2=0x7fab000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:26.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.329+0000 7fab04c29640 1 -- 172.21.15.116:0/2875865202 shutdown_connections 2024-09-12T03:08:26.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.329+0000 7fab04c29640 1 -- 172.21.15.116:0/2875865202 wait complete. 2024-09-12T03:08:26.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.329+0000 7fab04c29640 1 Processor -- start 2024-09-12T03:08:26.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.329+0000 7fab04c29640 1 -- start start 2024-09-12T03:08:26.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.330+0000 7fab04c29640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab00100e20 0x7fab00199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:26.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.330+0000 7fab04c29640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab00102570 0x7fab00199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:26.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.330+0000 7fab04c29640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab00102ef0 0x7fab001a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:26.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.330+0000 7fab04c29640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fab00110fa0 con 0x7fab00102570 2024-09-12T03:08:26.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.330+0000 7fab04c29640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fab00110e20 con 0x7fab00100e20 2024-09-12T03:08:26.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.330+0000 7fab04c29640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fab00111120 con 0x7fab00102ef0 2024-09-12T03:08:26.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.330+0000 7faafdd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab00102570 0x7fab00199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:26.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.330+0000 7faafdd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab00102570 0x7fab00199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52858/0 (socket says 172.21.15.116:52858) 2024-09-12T03:08:26.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.330+0000 7faafdd9b640 1 -- 172.21.15.116:0/2700368485 learned_addr learned my addr 172.21.15.116:0/2700368485 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:26.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.330+0000 7faafe59c640 1 --2- 172.21.15.116:0/2700368485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab00100e20 0x7fab00199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:26.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.330+0000 7faafed9d640 1 --2- 172.21.15.116:0/2700368485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab00102ef0 0x7fab001a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:26.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.330+0000 7faafdd9b640 1 -- 172.21.15.116:0/2700368485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab00102ef0 msgr2=0x7fab001a44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:26.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.330+0000 7faafdd9b640 1 --2- 172.21.15.116:0/2700368485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab00102ef0 0x7fab001a44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.330+0000 7faafdd9b640 1 -- 172.21.15.116:0/2700368485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab00100e20 msgr2=0x7fab00199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:26.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.330+0000 7faafdd9b640 1 --2- 172.21.15.116:0/2700368485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab00100e20 0x7fab00199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.330+0000 7faafdd9b640 1 -- 172.21.15.116:0/2700368485 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab001a49f0 con 0x7fab00102570 2024-09-12T03:08:26.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.330+0000 7faafdd9b640 1 --2- 172.21.15.116:0/2700368485 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab00102570 0x7fab00199970 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7faaf0005cc0 tx=0x7faaf002f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:26.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.330+0000 7faaef7fe640 1 -- 172.21.15.116:0/2700368485 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf0034040 con 0x7fab00102570 2024-09-12T03:08:26.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.331+0000 7fab04c29640 1 -- 172.21.15.116:0/2700368485 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab001a4bc0 con 0x7fab00102570 2024-09-12T03:08:26.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.331+0000 7faaef7fe640 1 -- 172.21.15.116:0/2700368485 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaf0030920 con 0x7fab00102570 2024-09-12T03:08:26.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.331+0000 7faaef7fe640 1 -- 172.21.15.116:0/2700368485 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf00047c0 con 0x7fab00102570 2024-09-12T03:08:26.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.331+0000 7fab04c29640 1 -- 172.21.15.116:0/2700368485 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab001a5070 con 0x7fab00102570 2024-09-12T03:08:26.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.333+0000 7faaef7fe640 1 -- 172.21.15.116:0/2700368485 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7faaf0037020 con 0x7fab00102570 2024-09-12T03:08:26.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.333+0000 7fab07002740 1 Processor -- start 2024-09-12T03:08:26.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.333+0000 7fab07002740 1 -- start start 2024-09-12T03:08:26.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.333+0000 7fab07002740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f6c107b40 0x558f6c106090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:26.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.333+0000 7fab07002740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558f6c1065d0 0x558f6c106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:26.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.334+0000 7fab07002740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f6c115780 0x558f6c117b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:26.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.334+0000 7fab07002740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x558f6c06c2f0 con 0x558f6c1065d0 2024-09-12T03:08:26.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.334+0000 7fab07002740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558f6c013cf0 con 0x558f6c115780 2024-09-12T03:08:26.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.334+0000 7fab07002740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558f6c0682b0 con 0x558f6c107b40 2024-09-12T03:08:26.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.334+0000 7faaef7fe640 1 --2- 172.21.15.116:0/2700368485 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7faae80a2b80 0x7faae80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:26.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.334+0000 7faafe59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558f6c1065d0 0x558f6c106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:26.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.334+0000 7faafe59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558f6c1065d0 0x558f6c106a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52868/0 (socket says 172.21.15.116:52868) 2024-09-12T03:08:26.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.334+0000 7faafe59c640 1 -- 172.21.15.116:0/921478342 learned_addr learned my addr 172.21.15.116:0/921478342 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:26.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.334+0000 7faafdd9b640 1 --2- 172.21.15.116:0/921478342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f6c115780 0x558f6c117b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:26.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.334+0000 7faafed9d640 1 --2- 172.21.15.116:0/2700368485 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7faae80a2b80 0x7faae80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:26.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.334+0000 7faafed9d640 1 --2- 172.21.15.116:0/921478342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f6c107b40 0x558f6c106090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:26.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.334+0000 7faaef7fe640 1 -- 172.21.15.116:0/2700368485 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7faaf00efff0 con 0x7fab00102570 2024-09-12T03:08:26.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.334+0000 7faafe59c640 1 -- 172.21.15.116:0/921478342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f6c107b40 msgr2=0x558f6c106090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:26.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.334+0000 7faafe59c640 1 --2- 172.21.15.116:0/921478342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f6c107b40 0x558f6c106090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.334+0000 7faafe59c640 1 -- 172.21.15.116:0/921478342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f6c115780 msgr2=0x558f6c117b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:26.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.334+0000 7faafe59c640 1 --2- 172.21.15.116:0/921478342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f6c115780 0x558f6c117b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.334+0000 7faafe59c640 1 -- 172.21.15.116:0/921478342 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f6bff83a0 con 0x558f6c1065d0 2024-09-12T03:08:26.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.334+0000 7faafdd9b640 1 --2- 172.21.15.116:0/921478342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f6c115780 0x558f6c117b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:26.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.334+0000 7faafe59c640 1 --2- 172.21.15.116:0/921478342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558f6c1065d0 0x558f6c106a10 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7faaf4009a70 tx=0x7faaf4009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e2164f2ec735bc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:26.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.334+0000 7faafed9d640 1 --2- 172.21.15.116:0/2700368485 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7faae80a2b80 0x7faae80a5040 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faaf80146b0 tx=0x7faaf800c2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:26.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.334+0000 7faaecff9640 1 -- 172.21.15.116:0/921478342 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf4017070 con 0x558f6c1065d0 2024-09-12T03:08:26.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.334+0000 7faaecff9640 1 -- 172.21.15.116:0/921478342 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaf4013470 con 0x558f6c1065d0 2024-09-12T03:08:26.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.334+0000 7faaecff9640 1 -- 172.21.15.116:0/921478342 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf4012630 con 0x558f6c1065d0 2024-09-12T03:08:26.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.335+0000 7fab07002740 1 -- 172.21.15.116:0/921478342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558f6c1065d0 msgr2=0x558f6c106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:26.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.335+0000 7fab07002740 1 --2- 172.21.15.116:0/921478342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558f6c1065d0 0x558f6c106a10 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7faaf4009a70 tx=0x7faaf4009f30 comp rx=0 tx=0).stop 2024-09-12T03:08:26.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.335+0000 7fab07002740 1 -- 172.21.15.116:0/921478342 shutdown_connections 2024-09-12T03:08:26.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.335+0000 7fab07002740 1 --2- 172.21.15.116:0/921478342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f6c115780 0x558f6c117b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.335+0000 7fab07002740 1 --2- 172.21.15.116:0/921478342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558f6c1065d0 0x558f6c106a10 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.335+0000 7fab07002740 1 --2- 172.21.15.116:0/921478342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f6c107b40 0x558f6c106090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.335+0000 7fab07002740 1 -- 172.21.15.116:0/921478342 >> 172.21.15.116:0/921478342 conn(0x558f6c0cf310 msgr2=0x558f6c102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:26.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.335+0000 7fab07002740 1 -- 172.21.15.116:0/921478342 shutdown_connections 2024-09-12T03:08:26.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.335+0000 7fab07002740 1 -- 172.21.15.116:0/921478342 wait complete. 2024-09-12T03:08:26.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.336+0000 7fab07002740 1 -- start start 2024-09-12T03:08:26.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.336+0000 7fab07002740 1 Processor -- start 2024-09-12T03:08:26.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.336+0000 7fab07002740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f6c1065d0 0x558f6c1150b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:26.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.336+0000 7fab07002740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558f6c107b40 0x558f6c112360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:26.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.336+0000 7fab07002740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f6c115780 0x558f6c1128a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:26.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.336+0000 7fab07002740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x558f6c118480 con 0x558f6c107b40 2024-09-12T03:08:26.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.336+0000 7fab07002740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558f6c027060 con 0x558f6c115780 2024-09-12T03:08:26.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.336+0000 7fab07002740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558f6c118600 con 0x558f6c1065d0 2024-09-12T03:08:26.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.336+0000 7fab07002740 10 client.0 _ll_register_callbacks cb 0x558f6bd7a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:26.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.336+0000 7faafdd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558f6c107b40 0x558f6c112360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:26.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.336+0000 7faafdd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558f6c107b40 0x558f6c112360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52882/0 (socket says 172.21.15.116:52882) 2024-09-12T03:08:26.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.336+0000 7faafdd9b640 1 -- 172.21.15.116:0/2653944158 learned_addr learned my addr 172.21.15.116:0/2653944158 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:26.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.336+0000 7faafe59c640 1 --2- 172.21.15.116:0/2653944158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f6c1065d0 0x558f6c1150b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:26.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.337+0000 7faafdd9b640 1 -- 172.21.15.116:0/2653944158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f6c1065d0 msgr2=0x558f6c1150b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:26.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.337+0000 7faafdd9b640 1 --2- 172.21.15.116:0/2653944158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f6c1065d0 0x558f6c1150b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.337+0000 7faafdd9b640 1 -- 172.21.15.116:0/2653944158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f6c115780 msgr2=0x558f6c1128a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:26.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.337+0000 7faafdd9b640 1 --2- 172.21.15.116:0/2653944158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f6c115780 0x558f6c1128a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.337+0000 7faafdd9b640 1 -- 172.21.15.116:0/2653944158 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f6bddd3c0 con 0x558f6c107b40 2024-09-12T03:08:26.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.337+0000 7faafe59c640 1 --2- 172.21.15.116:0/2653944158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f6c1065d0 0x558f6c1150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:26.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.337+0000 7faafdd9b640 1 --2- 172.21.15.116:0/2653944158 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558f6c107b40 0x558f6c112360 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7faaf00b8d20 tx=0x7faaf008b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f0e36784007126 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:26.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.337+0000 7faae5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:26.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.337+0000 7faae5ffb640 1 -- 172.21.15.116:0/2653944158 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf0049980 con 0x558f6c107b40 2024-09-12T03:08:26.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.337+0000 7faae5ffb640 1 -- 172.21.15.116:0/2653944158 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7faaf0009800 con 0x558f6c107b40 2024-09-12T03:08:26.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.337+0000 7fab07002740 1 -- 172.21.15.116:0/2653944158 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaf4013470 con 0x558f6c107b40 2024-09-12T03:08:26.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.337+0000 7faae5ffb640 1 -- 172.21.15.116:0/2653944158 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7faaf00879a0 con 0x558f6c107b40 2024-09-12T03:08:26.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.337+0000 7fab07002740 10 client.16728 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:26.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.337+0000 7fab07002740 1 -- 172.21.15.116:0/2653944158 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558f6c118480 con 0x558f6c107b40 2024-09-12T03:08:26.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.337+0000 7faae5ffb640 1 -- 172.21.15.116:0/2653944158 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaf4013470 con 0x558f6c107b40 2024-09-12T03:08:26.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.337+0000 7faae5ffb640 1 -- 172.21.15.116:0/2653944158 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7faaf004d030 con 0x558f6c107b40 2024-09-12T03:08:26.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.337+0000 7fab07002740 10 client.16728 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:26.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.337+0000 7fab07002740 10 client.16728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:08:26.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.338+0000 7fab07002740 20 client.16728 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:26.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.338+0000 7fab07002740 1 --2- 172.21.15.116:0/2653944158 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x558f6c125350 0x558f6c1277c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:26.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.338+0000 7fab07002740 4 client.16728 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:08:26.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.338+0000 7fab07002740 1 -- 172.21.15.116:0/2653944158 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f6c079860 con 0x558f6c125350 2024-09-12T03:08:26.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.338+0000 7faafe59c640 1 --2- 172.21.15.116:0/2653944158 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x558f6c125350 0x558f6c1277c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:26.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.338+0000 7faafe59c640 1 --2- 172.21.15.116:0/2653944158 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x558f6c125350 0x558f6c1277c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4788f80b4486a427 server_cookie=a96bf0ac355e50b1 in_seq=0 out_seq=0 2024-09-12T03:08:26.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.338+0000 7faae5ffb640 10 client.16728 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:26.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.339+0000 7faae5ffb640 1 -- 172.21.15.116:0/2653944158 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558f6c079860 con 0x558f6c125350 2024-09-12T03:08:26.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.339+0000 7faae5ffb640 10 client.16728 handle_command_reply: tid=0 2024-09-12T03:08:26.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.339+0000 7fab07002740 1 client.16728 shutdown 2024-09-12T03:08:26.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.339+0000 7fab07002740 1 -- 172.21.15.116:0/2653944158 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558f6c107b40 msgr2=0x558f6c112360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:26.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.339+0000 7fab07002740 1 --2- 172.21.15.116:0/2653944158 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558f6c107b40 0x558f6c112360 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7faaf00b8d20 tx=0x7faaf008b840 comp rx=0 tx=0).stop 2024-09-12T03:08:26.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.339+0000 7fab07002740 1 -- 172.21.15.116:0/2653944158 shutdown_connections 2024-09-12T03:08:26.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.339+0000 7fab07002740 1 --2- 172.21.15.116:0/2653944158 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x558f6c125350 0x558f6c1277c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.339+0000 7fab07002740 1 --2- 172.21.15.116:0/2653944158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f6c115780 0x558f6c1128a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.339+0000 7fab07002740 1 --2- 172.21.15.116:0/2653944158 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558f6c107b40 0x558f6c112360 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.339+0000 7fab07002740 1 --2- 172.21.15.116:0/2653944158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f6c1065d0 0x558f6c1150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.339+0000 7fab07002740 1 -- 172.21.15.116:0/2653944158 >> 172.21.15.116:0/2653944158 conn(0x558f6c0cf310 msgr2=0x558f6c108930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:26.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.339+0000 7faae5ffb640 0 client.16728 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:26.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.339+0000 7fab07002740 1 -- 172.21.15.116:0/2653944158 shutdown_connections 2024-09-12T03:08:26.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.339+0000 7fab07002740 1 -- 172.21.15.116:0/2653944158 wait complete. 2024-09-12T03:08:26.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.339+0000 7fab07002740 20 client.16728 trim_cache size 0 max 16384 2024-09-12T03:08:26.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.352+0000 7fab07002740 1 Processor -- start 2024-09-12T03:08:26.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.353+0000 7fab07002740 1 -- start start 2024-09-12T03:08:26.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.353+0000 7fab07002740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558f6c1065d0 0x558f6c121750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:26.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.353+0000 7fab07002740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f6c107b40 0x558f6c117740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:26.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.353+0000 7fab07002740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f6c115780 0x558f6c116b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:26.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.353+0000 7fab07002740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fab001ada80 con 0x558f6c1065d0 2024-09-12T03:08:26.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.353+0000 7fab07002740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fab001ab350 con 0x558f6c115780 2024-09-12T03:08:26.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.353+0000 7fab07002740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fab001a91c0 con 0x558f6c107b40 2024-09-12T03:08:26.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.353+0000 7faafe59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558f6c1065d0 0x558f6c121750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:26.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.353+0000 7faafe59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558f6c1065d0 0x558f6c121750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52894/0 (socket says 172.21.15.116:52894) 2024-09-12T03:08:26.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.353+0000 7faafe59c640 1 -- 172.21.15.116:0/700939110 learned_addr learned my addr 172.21.15.116:0/700939110 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:26.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.353+0000 7faafed9d640 1 --2- 172.21.15.116:0/700939110 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f6c115780 0x558f6c116b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:26.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.353+0000 7faafdd9b640 1 --2- 172.21.15.116:0/700939110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f6c107b40 0x558f6c117740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:26.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.353+0000 7faafe59c640 1 -- 172.21.15.116:0/700939110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f6c107b40 msgr2=0x558f6c117740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:26.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.353+0000 7faafe59c640 1 --2- 172.21.15.116:0/700939110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f6c107b40 0x558f6c117740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.353+0000 7faafe59c640 1 -- 172.21.15.116:0/700939110 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f6c115780 msgr2=0x558f6c116b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:26.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.353+0000 7faafe59c640 1 --2- 172.21.15.116:0/700939110 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f6c115780 0x558f6c116b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.353+0000 7faafe59c640 1 -- 172.21.15.116:0/700939110 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f6c109120 con 0x558f6c1065d0 2024-09-12T03:08:26.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.353+0000 7faafe59c640 1 --2- 172.21.15.116:0/700939110 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558f6c1065d0 0x558f6c121750 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7faaf4025fd0 tx=0x7faaf4009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5234a71934bb7325 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:26.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.354+0000 7faae7fff640 1 -- 172.21.15.116:0/700939110 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf402b070 con 0x558f6c1065d0 2024-09-12T03:08:26.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.354+0000 7faae7fff640 1 -- 172.21.15.116:0/700939110 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaf4025440 con 0x558f6c1065d0 2024-09-12T03:08:26.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.354+0000 7faae7fff640 1 -- 172.21.15.116:0/700939110 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf4017040 con 0x558f6c1065d0 2024-09-12T03:08:26.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.354+0000 7fab07002740 1 -- 172.21.15.116:0/700939110 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558f6c1065d0 msgr2=0x558f6c121750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:26.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.354+0000 7fab07002740 1 --2- 172.21.15.116:0/700939110 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558f6c1065d0 0x558f6c121750 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7faaf4025fd0 tx=0x7faaf4009110 comp rx=0 tx=0).stop 2024-09-12T03:08:26.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.354+0000 7fab07002740 1 -- 172.21.15.116:0/700939110 shutdown_connections 2024-09-12T03:08:26.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.354+0000 7fab07002740 1 --2- 172.21.15.116:0/700939110 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f6c115780 0x558f6c116b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.354+0000 7fab07002740 1 --2- 172.21.15.116:0/700939110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f6c107b40 0x558f6c117740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.354+0000 7fab07002740 1 --2- 172.21.15.116:0/700939110 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558f6c1065d0 0x558f6c121750 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.354+0000 7fab07002740 1 -- 172.21.15.116:0/700939110 >> 172.21.15.116:0/700939110 conn(0x558f6c0cf310 msgr2=0x558f6c110c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:26.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.354+0000 7fab07002740 1 -- 172.21.15.116:0/700939110 shutdown_connections 2024-09-12T03:08:26.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.354+0000 7fab07002740 1 -- 172.21.15.116:0/700939110 wait complete. 2024-09-12T03:08:26.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.355+0000 7fab07002740 1 -- start start 2024-09-12T03:08:26.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.355+0000 7fab07002740 1 Processor -- start 2024-09-12T03:08:26.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.355+0000 7fab07002740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f6c1065d0 0x558f6c11dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:26.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.355+0000 7fab07002740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558f6c107b40 0x558f6c11dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:26.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.356+0000 7fab07002740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f6c115780 0x558f6c118880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:26.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.356+0000 7fab07002740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fab001a9810 con 0x558f6c107b40 2024-09-12T03:08:26.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.356+0000 7fab07002740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fab001aac80 con 0x558f6c1065d0 2024-09-12T03:08:26.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.356+0000 7fab07002740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fab001a9ee0 con 0x558f6c115780 2024-09-12T03:08:26.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.356+0000 7fab07002740 10 client.0 _ll_register_callbacks cb 0x7faaf00b8b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:26.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.356+0000 7faafdd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558f6c107b40 0x558f6c11dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:26.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.356+0000 7faafdd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558f6c107b40 0x558f6c11dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52910/0 (socket says 172.21.15.116:52910) 2024-09-12T03:08:26.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.356+0000 7faafdd9b640 1 -- 172.21.15.116:0/1586971719 learned_addr learned my addr 172.21.15.116:0/1586971719 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:26.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.356+0000 7faafe59c640 1 --2- 172.21.15.116:0/1586971719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f6c1065d0 0x558f6c11dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:26.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.356+0000 7faafed9d640 1 --2- 172.21.15.116:0/1586971719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f6c115780 0x558f6c118880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:26.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.356+0000 7faafdd9b640 1 -- 172.21.15.116:0/1586971719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f6c115780 msgr2=0x558f6c118880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:26.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.356+0000 7faafdd9b640 1 --2- 172.21.15.116:0/1586971719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f6c115780 0x558f6c118880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.356+0000 7faafdd9b640 1 -- 172.21.15.116:0/1586971719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f6c1065d0 msgr2=0x558f6c11dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:26.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.356+0000 7faafdd9b640 1 --2- 172.21.15.116:0/1586971719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f6c1065d0 0x558f6c11dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.356+0000 7faafdd9b640 1 -- 172.21.15.116:0/1586971719 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f6c118f90 con 0x558f6c107b40 2024-09-12T03:08:26.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.357+0000 7faafdd9b640 1 --2- 172.21.15.116:0/1586971719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558f6c107b40 0x558f6c11dff0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7faaf0043fb0 tx=0x7faaf00f20d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8259215449d3ef87 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:26.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.357+0000 7faae5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:26.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.357+0000 7faae5ffb640 1 -- 172.21.15.116:0/1586971719 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf00bcd20 con 0x558f6c107b40 2024-09-12T03:08:26.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.357+0000 7faae5ffb640 1 -- 172.21.15.116:0/1586971719 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7faaf00bbd10 con 0x558f6c107b40 2024-09-12T03:08:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.357+0000 7fab07002740 1 -- 172.21.15.116:0/1586971719 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaf4025440 con 0x558f6c107b40 2024-09-12T03:08:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.357+0000 7faae5ffb640 1 -- 172.21.15.116:0/1586971719 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7faaf00bcfc0 con 0x558f6c107b40 2024-09-12T03:08:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.357+0000 7fab07002740 10 client.16734 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.357+0000 7fab07002740 1 -- 172.21.15.116:0/1586971719 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558f6c1094d0 con 0x558f6c107b40 2024-09-12T03:08:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.358+0000 7faae5ffb640 1 -- 172.21.15.116:0/1586971719 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaf4025440 con 0x558f6c107b40 2024-09-12T03:08:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.358+0000 7faae5ffb640 1 -- 172.21.15.116:0/1586971719 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7faaf004d070 con 0x558f6c107b40 2024-09-12T03:08:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.358+0000 7fab07002740 10 client.16734 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.358+0000 7fab07002740 10 client.16734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:08:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.358+0000 7fab07002740 20 client.16734 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.358+0000 7fab07002740 1 --2- 172.21.15.116:0/1586971719 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x558f6c109b10 0x558f6c10bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.358+0000 7fab07002740 4 client.16734 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:08:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.358+0000 7fab07002740 1 -- 172.21.15.116:0/1586971719 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f6c073d70 con 0x558f6c109b10 2024-09-12T03:08:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.358+0000 7faafe59c640 1 --2- 172.21.15.116:0/1586971719 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x558f6c109b10 0x558f6c10bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:26.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.358+0000 7faae5ffb640 10 client.16734 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:26.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.358+0000 7faafe59c640 1 --2- 172.21.15.116:0/1586971719 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x558f6c109b10 0x558f6c10bf30 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7394f8b0a453f352 server_cookie=6dfd1f0dd210dd1d in_seq=0 out_seq=0 2024-09-12T03:08:26.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.359+0000 7faae5ffb640 1 -- 172.21.15.116:0/1586971719 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f6c073d70 con 0x558f6c109b10 2024-09-12T03:08:26.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.359+0000 7faae5ffb640 10 client.16734 handle_command_reply: tid=0 2024-09-12T03:08:26.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.359+0000 7fab07002740 1 client.16734 shutdown 2024-09-12T03:08:26.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.359+0000 7fab07002740 1 -- 172.21.15.116:0/1586971719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558f6c107b40 msgr2=0x558f6c11dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:26.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.359+0000 7fab07002740 1 --2- 172.21.15.116:0/1586971719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558f6c107b40 0x558f6c11dff0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7faaf0043fb0 tx=0x7faaf00f20d0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.359+0000 7fab07002740 1 -- 172.21.15.116:0/1586971719 shutdown_connections 2024-09-12T03:08:26.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.359+0000 7fab07002740 1 --2- 172.21.15.116:0/1586971719 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x558f6c109b10 0x558f6c10bf30 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.359+0000 7fab07002740 1 --2- 172.21.15.116:0/1586971719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f6c115780 0x558f6c118880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.359+0000 7faae5ffb640 0 client.16734 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:26.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.359+0000 7fab07002740 1 --2- 172.21.15.116:0/1586971719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558f6c107b40 0x558f6c11dff0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.359+0000 7fab07002740 1 --2- 172.21.15.116:0/1586971719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558f6c1065d0 0x558f6c11dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.359+0000 7fab07002740 1 -- 172.21.15.116:0/1586971719 >> 172.21.15.116:0/1586971719 conn(0x558f6c0cf310 msgr2=0x558f6c113bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:26.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.359+0000 7fab07002740 1 -- 172.21.15.116:0/1586971719 shutdown_connections 2024-09-12T03:08:26.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.359+0000 7fab07002740 1 -- 172.21.15.116:0/1586971719 wait complete. 2024-09-12T03:08:26.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.359+0000 7fab07002740 20 client.16734 trim_cache size 0 max 16384 2024-09-12T03:08:26.364 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:08:26.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.360+0000 7fab04c29640 1 -- 172.21.15.116:0/2700368485 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7faae80a2b80 msgr2=0x7faae80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:26.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.360+0000 7fab04c29640 1 --2- 172.21.15.116:0/2700368485 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7faae80a2b80 0x7faae80a5040 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faaf80146b0 tx=0x7faaf800c2d0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.360+0000 7fab04c29640 1 -- 172.21.15.116:0/2700368485 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab00102570 msgr2=0x7fab00199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:26.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.360+0000 7fab04c29640 1 --2- 172.21.15.116:0/2700368485 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab00102570 0x7fab00199970 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7faaf0005cc0 tx=0x7faaf002f650 comp rx=0 tx=0).stop 2024-09-12T03:08:26.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.360+0000 7fab04c29640 1 -- 172.21.15.116:0/2700368485 shutdown_connections 2024-09-12T03:08:26.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.360+0000 7fab04c29640 1 --2- 172.21.15.116:0/2700368485 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7faae80a2b80 0x7faae80a5040 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.360+0000 7fab04c29640 1 --2- 172.21.15.116:0/2700368485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab00102ef0 0x7fab001a44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.360+0000 7fab04c29640 1 --2- 172.21.15.116:0/2700368485 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab00102570 0x7fab00199970 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.360+0000 7fab04c29640 1 --2- 172.21.15.116:0/2700368485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab00100e20 0x7fab00199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.360+0000 7fab04c29640 1 -- 172.21.15.116:0/2700368485 >> 172.21.15.116:0/2700368485 conn(0x7fab000fc7d0 msgr2=0x7fab000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:26.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.360+0000 7fab04c29640 1 -- 172.21.15.116:0/2700368485 shutdown_connections 2024-09-12T03:08:26.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:26.360+0000 7fab04c29640 1 -- 172.21.15.116:0/2700368485 wait complete. 2024-09-12T03:08:26.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:26 smithi116 ceph-mon[31410]: pgmap v501: 97 pgs: 97 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:08:26.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:26 smithi183 ceph-mon[35535]: pgmap v501: 97 pgs: 97 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:08:26.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:26 smithi177 ceph-mon[35305]: pgmap v501: 97 pgs: 97 active+clean; 2.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:08:28.123 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:08:28.123 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:08:28.383 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:08:28.503 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:28 smithi116 ceph-mon[31410]: pgmap v502: 97 pgs: 97 active+clean; 2.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:08:28.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:28 smithi183 ceph-mon[35535]: pgmap v502: 97 pgs: 97 active+clean; 2.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:08:28.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:28 smithi177 ceph-mon[35305]: pgmap v502: 97 pgs: 97 active+clean; 2.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:08:28.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.872+0000 7f38dcec4640 1 -- 172.21.15.116:0/2706977220 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38d8106a80 msgr2=0x7f38d8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:28.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.872+0000 7f38dcec4640 1 --2- 172.21.15.116:0/2706977220 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38d8106a80 0x7f38d8106e60 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f38c8009930 tx=0x7f38c802a420 comp rx=0 tx=0).stop 2024-09-12T03:08:28.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.872+0000 7f38dcec4640 1 -- 172.21.15.116:0/2706977220 shutdown_connections 2024-09-12T03:08:28.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.872+0000 7f38dcec4640 1 --2- 172.21.15.116:0/2706977220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38d8101420 0x7f38d810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.872+0000 7f38dcec4640 1 --2- 172.21.15.116:0/2706977220 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38d8100a80 0x7f38d8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.872+0000 7f38dcec4640 1 --2- 172.21.15.116:0/2706977220 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38d8106a80 0x7f38d8106e60 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.872+0000 7f38dcec4640 1 -- 172.21.15.116:0/2706977220 >> 172.21.15.116:0/2706977220 conn(0x7f38d80fc7a0 msgr2=0x7f38d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:28.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.872+0000 7f38dcec4640 1 -- 172.21.15.116:0/2706977220 shutdown_connections 2024-09-12T03:08:28.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.872+0000 7f38dcec4640 1 -- 172.21.15.116:0/2706977220 wait complete. 2024-09-12T03:08:28.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.873+0000 7f38dcec4640 1 Processor -- start 2024-09-12T03:08:28.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.873+0000 7f38dcec4640 1 -- start start 2024-09-12T03:08:28.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.873+0000 7f38dcec4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38d8100a80 0x7f38d81992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:28.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.873+0000 7f38dcec4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38d8101420 0x7f38d8199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:28.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.873+0000 7f38dcec4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38d8106a80 0x7f38d81a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:28.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.873+0000 7f38dcec4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f38d8110e80 con 0x7f38d8100a80 2024-09-12T03:08:28.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.873+0000 7f38dcec4640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f38d8110d00 con 0x7f38d8106a80 2024-09-12T03:08:28.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.873+0000 7f38dcec4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f38d8111000 con 0x7f38d8101420 2024-09-12T03:08:28.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.873+0000 7f38d659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38d8100a80 0x7f38d81992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:28.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.873+0000 7f38d659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38d8100a80 0x7f38d81992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52930/0 (socket says 172.21.15.116:52930) 2024-09-12T03:08:28.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.873+0000 7f38d659c640 1 -- 172.21.15.116:0/340667178 learned_addr learned my addr 172.21.15.116:0/340667178 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:28.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.873+0000 7f38d5d9b640 1 --2- 172.21.15.116:0/340667178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38d8101420 0x7f38d8199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:28.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.873+0000 7f38d6d9d640 1 --2- 172.21.15.116:0/340667178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38d8106a80 0x7f38d81a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:28.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.873+0000 7f38d659c640 1 -- 172.21.15.116:0/340667178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38d8101420 msgr2=0x7f38d8199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:28.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.873+0000 7f38d659c640 1 --2- 172.21.15.116:0/340667178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38d8101420 0x7f38d8199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.873+0000 7f38d659c640 1 -- 172.21.15.116:0/340667178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38d8106a80 msgr2=0x7f38d81a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:28.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.873+0000 7f38d659c640 1 --2- 172.21.15.116:0/340667178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38d8106a80 0x7f38d81a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.873+0000 7f38d659c640 1 -- 172.21.15.116:0/340667178 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d81a48b0 con 0x7f38d8100a80 2024-09-12T03:08:28.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.873+0000 7f38d659c640 1 --2- 172.21.15.116:0/340667178 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38d8100a80 0x7f38d81992f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f38c8009230 tx=0x7f38c8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:28.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.874+0000 7f38c77fe640 1 -- 172.21.15.116:0/340667178 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c803f030 con 0x7f38d8100a80 2024-09-12T03:08:28.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.874+0000 7f38c77fe640 1 -- 172.21.15.116:0/340667178 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38c8003ad0 con 0x7f38d8100a80 2024-09-12T03:08:28.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.874+0000 7f38dcec4640 1 -- 172.21.15.116:0/340667178 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38d81a4a80 con 0x7f38d8100a80 2024-09-12T03:08:28.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.874+0000 7f38c77fe640 1 -- 172.21.15.116:0/340667178 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8035070 con 0x7f38d8100a80 2024-09-12T03:08:28.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.874+0000 7f38dcec4640 1 -- 172.21.15.116:0/340667178 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38d81a4f30 con 0x7f38d8100a80 2024-09-12T03:08:28.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.876+0000 7f38c77fe640 1 -- 172.21.15.116:0/340667178 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f38c8042030 con 0x7f38d8100a80 2024-09-12T03:08:28.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.876+0000 7f38df29d740 1 Processor -- start 2024-09-12T03:08:28.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.876+0000 7f38df29d740 1 -- start start 2024-09-12T03:08:28.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.877+0000 7f38df29d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5585b6d17b60 0x5585b6d17f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:28.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.877+0000 7f38df29d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5585b6d18480 0x5585b6d22800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:28.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.877+0000 7f38df29d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585b6d22d40 0x5585b6d25130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:28.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.877+0000 7f38df29d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5585b6c7d280 con 0x5585b6d18480 2024-09-12T03:08:28.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.877+0000 7f38df29d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5585b6c24b10 con 0x5585b6d22d40 2024-09-12T03:08:28.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.877+0000 7f38df29d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5585b6c79240 con 0x5585b6d17b60 2024-09-12T03:08:28.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.877+0000 7f38d659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5585b6d18480 0x5585b6d22800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:28.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.877+0000 7f38c77fe640 1 --2- 172.21.15.116:0/340667178 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f38c00a2c00 0x7f38c00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:28.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.877+0000 7f38d659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5585b6d18480 0x5585b6d22800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52938/0 (socket says 172.21.15.116:52938) 2024-09-12T03:08:28.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.877+0000 7f38d659c640 1 -- 172.21.15.116:0/3610668645 learned_addr learned my addr 172.21.15.116:0/3610668645 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:28.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.877+0000 7f38d5d9b640 1 --2- 172.21.15.116:0/3610668645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585b6d22d40 0x5585b6d25130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:28.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.877+0000 7f38d6d9d640 1 --2- 172.21.15.116:0/3610668645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5585b6d17b60 0x5585b6d17f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:28.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.877+0000 7f38d6d9d640 1 --2- 172.21.15.116:0/340667178 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f38c00a2c00 0x7f38c00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:28.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.877+0000 7f38c77fe640 1 -- 172.21.15.116:0/340667178 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f38c80ef4a0 con 0x7f38d8100a80 2024-09-12T03:08:28.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.877+0000 7f38d659c640 1 -- 172.21.15.116:0/3610668645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5585b6d17b60 msgr2=0x5585b6d17f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:28.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.877+0000 7f38d659c640 1 --2- 172.21.15.116:0/3610668645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5585b6d17b60 0x5585b6d17f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.877+0000 7f38d659c640 1 -- 172.21.15.116:0/3610668645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585b6d22d40 msgr2=0x5585b6d25130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:28.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.877+0000 7f38d659c640 1 --2- 172.21.15.116:0/3610668645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585b6d22d40 0x5585b6d25130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.877+0000 7f38d659c640 1 -- 172.21.15.116:0/3610668645 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585b6c311f0 con 0x5585b6d18480 2024-09-12T03:08:28.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.877+0000 7f38d6d9d640 1 --2- 172.21.15.116:0/3610668645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5585b6d17b60 0x5585b6d17f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:28.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.877+0000 7f38d659c640 1 --2- 172.21.15.116:0/3610668645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5585b6d18480 0x5585b6d22800 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f38c80c02a0 tx=0x7f38c8005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8f36b33464bee5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:28.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.877+0000 7f38d6d9d640 1 --2- 172.21.15.116:0/340667178 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f38c00a2c00 0x7f38c00a50c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f38d0009fd0 tx=0x7f38d0005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:28.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.877+0000 7f38c4ff9640 1 -- 172.21.15.116:0/3610668645 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8049020 con 0x5585b6d18480 2024-09-12T03:08:28.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.878+0000 7f38c4ff9640 1 -- 172.21.15.116:0/3610668645 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38c80bc430 con 0x5585b6d18480 2024-09-12T03:08:28.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.878+0000 7f38c4ff9640 1 -- 172.21.15.116:0/3610668645 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c80b83a0 con 0x5585b6d18480 2024-09-12T03:08:28.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.878+0000 7f38df29d740 1 -- 172.21.15.116:0/3610668645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5585b6d18480 msgr2=0x5585b6d22800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:28.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.878+0000 7f38df29d740 1 --2- 172.21.15.116:0/3610668645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5585b6d18480 0x5585b6d22800 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f38c80c02a0 tx=0x7f38c8005e50 comp rx=0 tx=0).stop 2024-09-12T03:08:28.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.878+0000 7f38df29d740 1 -- 172.21.15.116:0/3610668645 shutdown_connections 2024-09-12T03:08:28.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.878+0000 7f38df29d740 1 --2- 172.21.15.116:0/3610668645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585b6d22d40 0x5585b6d25130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.878+0000 7f38df29d740 1 --2- 172.21.15.116:0/3610668645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5585b6d18480 0x5585b6d22800 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.878+0000 7f38df29d740 1 --2- 172.21.15.116:0/3610668645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5585b6d17b60 0x5585b6d17f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.878+0000 7f38df29d740 1 -- 172.21.15.116:0/3610668645 >> 172.21.15.116:0/3610668645 conn(0x5585b6d140f0 msgr2=0x5585b6d144d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:28.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.878+0000 7f38df29d740 1 -- 172.21.15.116:0/3610668645 shutdown_connections 2024-09-12T03:08:28.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.878+0000 7f38df29d740 1 -- 172.21.15.116:0/3610668645 wait complete. 2024-09-12T03:08:28.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.879+0000 7f38df29d740 1 -- start start 2024-09-12T03:08:28.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.879+0000 7f38df29d740 1 Processor -- start 2024-09-12T03:08:28.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.879+0000 7f38df29d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5585b6d17b60 0x5585b6d1fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:28.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.879+0000 7f38df29d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5585b6d18480 0x5585b6d200e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:28.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.879+0000 7f38df29d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585b6d22d40 0x5585b6d20620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:28.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.879+0000 7f38df29d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5585b6d256d0 con 0x5585b6d18480 2024-09-12T03:08:28.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.879+0000 7f38df29d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5585b6c37f10 con 0x5585b6d22d40 2024-09-12T03:08:28.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.879+0000 7f38df29d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5585b6d25850 con 0x5585b6d17b60 2024-09-12T03:08:28.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.879+0000 7f38df29d740 10 client.0 _ll_register_callbacks cb 0x5585b6c00a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:28.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.879+0000 7f38d5d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5585b6d18480 0x5585b6d200e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:28.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.879+0000 7f38d5d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5585b6d18480 0x5585b6d200e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52940/0 (socket says 172.21.15.116:52940) 2024-09-12T03:08:28.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.879+0000 7f38d5d9b640 1 -- 172.21.15.116:0/1144760193 learned_addr learned my addr 172.21.15.116:0/1144760193 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:28.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.879+0000 7f38d659c640 1 --2- 172.21.15.116:0/1144760193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5585b6d17b60 0x5585b6d1fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:28.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.879+0000 7f38d6d9d640 1 --2- 172.21.15.116:0/1144760193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585b6d22d40 0x5585b6d20620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:28.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.879+0000 7f38d5d9b640 1 -- 172.21.15.116:0/1144760193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5585b6d17b60 msgr2=0x5585b6d1fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:28.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.879+0000 7f38d5d9b640 1 --2- 172.21.15.116:0/1144760193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5585b6d17b60 0x5585b6d1fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.880+0000 7f38d5d9b640 1 -- 172.21.15.116:0/1144760193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585b6d22d40 msgr2=0x5585b6d20620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:28.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.880+0000 7f38d5d9b640 1 --2- 172.21.15.116:0/1144760193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585b6d22d40 0x5585b6d20620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.880+0000 7f38d5d9b640 1 -- 172.21.15.116:0/1144760193 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585b69ee6a0 con 0x5585b6d18480 2024-09-12T03:08:28.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.880+0000 7f38d5d9b640 1 --2- 172.21.15.116:0/1144760193 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5585b6d18480 0x5585b6d200e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f38cc010770 tx=0x7f38cc010c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fee006fc5073bd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:28.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.880+0000 7f38bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:28.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.880+0000 7f38bdffb640 1 -- 172.21.15.116:0/1144760193 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc00fb90 con 0x5585b6d18480 2024-09-12T03:08:28.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.880+0000 7f38bdffb640 1 -- 172.21.15.116:0/1144760193 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f38cc00fd30 con 0x5585b6d18480 2024-09-12T03:08:28.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.880+0000 7f38df29d740 1 -- 172.21.15.116:0/1144760193 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38c80bc430 con 0x5585b6d18480 2024-09-12T03:08:28.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.880+0000 7f38bdffb640 1 -- 172.21.15.116:0/1144760193 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f38cc004020 con 0x5585b6d18480 2024-09-12T03:08:28.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.880+0000 7f38df29d740 10 client.16746 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:28.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.880+0000 7f38df29d740 1 -- 172.21.15.116:0/1144760193 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5585b6d31230 con 0x5585b6d18480 2024-09-12T03:08:28.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.880+0000 7f38bdffb640 1 -- 172.21.15.116:0/1144760193 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38c80bc430 con 0x5585b6d18480 2024-09-12T03:08:28.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.880+0000 7f38bdffb640 1 -- 172.21.15.116:0/1144760193 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f38cc025480 con 0x5585b6d18480 2024-09-12T03:08:28.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.880+0000 7f38df29d740 10 client.16746 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:28.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.880+0000 7f38df29d740 10 client.16746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:08:28.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.880+0000 7f38df29d740 20 client.16746 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:28.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.880+0000 7f38df29d740 1 --2- 172.21.15.116:0/1144760193 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5585b6d32700 0x5585b6d34b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:28.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.880+0000 7f38df29d740 4 client.16746 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:08:28.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.880+0000 7f38df29d740 1 -- 172.21.15.116:0/1144760193 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585b6c8a7f0 con 0x5585b6d32700 2024-09-12T03:08:28.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.881+0000 7f38d659c640 1 --2- 172.21.15.116:0/1144760193 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5585b6d32700 0x5585b6d34b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:28.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.881+0000 7f38bdffb640 10 client.16746 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:28.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.881+0000 7f38d659c640 1 --2- 172.21.15.116:0/1144760193 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5585b6d32700 0x5585b6d34b60 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=860cf66761db160 server_cookie=c45596ac0cdaeb99 in_seq=0 out_seq=0 2024-09-12T03:08:28.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.882+0000 7f38bdffb640 1 -- 172.21.15.116:0/1144760193 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5585b6c8a7f0 con 0x5585b6d32700 2024-09-12T03:08:28.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.882+0000 7f38bdffb640 10 client.16746 handle_command_reply: tid=0 2024-09-12T03:08:28.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.882+0000 7f38df29d740 1 client.16746 shutdown 2024-09-12T03:08:28.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.882+0000 7f38df29d740 1 -- 172.21.15.116:0/1144760193 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5585b6d18480 msgr2=0x5585b6d200e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:28.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.882+0000 7f38df29d740 1 --2- 172.21.15.116:0/1144760193 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5585b6d18480 0x5585b6d200e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f38cc010770 tx=0x7f38cc010c30 comp rx=0 tx=0).stop 2024-09-12T03:08:28.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.882+0000 7f38df29d740 1 -- 172.21.15.116:0/1144760193 shutdown_connections 2024-09-12T03:08:28.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.882+0000 7f38df29d740 1 --2- 172.21.15.116:0/1144760193 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5585b6d32700 0x5585b6d34b60 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.882+0000 7f38df29d740 1 --2- 172.21.15.116:0/1144760193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585b6d22d40 0x5585b6d20620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.882+0000 7f38df29d740 1 --2- 172.21.15.116:0/1144760193 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5585b6d18480 0x5585b6d200e0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.882+0000 7f38df29d740 1 --2- 172.21.15.116:0/1144760193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5585b6d17b60 0x5585b6d1fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.882+0000 7f38df29d740 1 -- 172.21.15.116:0/1144760193 >> 172.21.15.116:0/1144760193 conn(0x5585b6d140f0 msgr2=0x5585b6d19dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:28.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.882+0000 7f38bdffb640 0 client.16746 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:28.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.882+0000 7f38df29d740 1 -- 172.21.15.116:0/1144760193 shutdown_connections 2024-09-12T03:08:28.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.882+0000 7f38df29d740 1 -- 172.21.15.116:0/1144760193 wait complete. 2024-09-12T03:08:28.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.882+0000 7f38df29d740 20 client.16746 trim_cache size 0 max 16384 2024-09-12T03:08:28.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.899+0000 7f38df29d740 1 Processor -- start 2024-09-12T03:08:28.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.899+0000 7f38df29d740 1 -- start start 2024-09-12T03:08:28.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.900+0000 7f38df29d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5585b6d17b60 0x5585b6d21a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:28.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.900+0000 7f38df29d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5585b6d18480 0x5585b6d3a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:28.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.900+0000 7f38df29d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585b6d22d40 0x5585b6d3a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:28.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.900+0000 7f38df29d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f38d81afdd0 con 0x5585b6d17b60 2024-09-12T03:08:28.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.900+0000 7f38df29d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f38d81ad6a0 con 0x5585b6d22d40 2024-09-12T03:08:28.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.900+0000 7f38df29d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f38d81ab510 con 0x5585b6d18480 2024-09-12T03:08:28.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.900+0000 7f38d659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5585b6d17b60 0x5585b6d21a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:28.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.900+0000 7f38d659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5585b6d17b60 0x5585b6d21a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52956/0 (socket says 172.21.15.116:52956) 2024-09-12T03:08:28.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.900+0000 7f38d659c640 1 -- 172.21.15.116:0/4135473570 learned_addr learned my addr 172.21.15.116:0/4135473570 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:28.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.900+0000 7f38d5d9b640 1 --2- 172.21.15.116:0/4135473570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5585b6d18480 0x5585b6d3a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:28.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.900+0000 7f38d6d9d640 1 --2- 172.21.15.116:0/4135473570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585b6d22d40 0x5585b6d3a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:28.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.900+0000 7f38d659c640 1 -- 172.21.15.116:0/4135473570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5585b6d18480 msgr2=0x5585b6d3a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:28.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.900+0000 7f38d659c640 1 --2- 172.21.15.116:0/4135473570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5585b6d18480 0x5585b6d3a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.900+0000 7f38d659c640 1 -- 172.21.15.116:0/4135473570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585b6d22d40 msgr2=0x5585b6d3a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:28.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.900+0000 7f38d659c640 1 --2- 172.21.15.116:0/4135473570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585b6d22d40 0x5585b6d3a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.900+0000 7f38d659c640 1 -- 172.21.15.116:0/4135473570 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585b6d0f830 con 0x5585b6d17b60 2024-09-12T03:08:28.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.900+0000 7f38d5d9b640 1 --2- 172.21.15.116:0/4135473570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5585b6d18480 0x5585b6d3a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:28.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.900+0000 7f38d659c640 1 --2- 172.21.15.116:0/4135473570 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5585b6d17b60 0x5585b6d21a00 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f38c80ed020 tx=0x7f38c8005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7f5eb17e4694be server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:28.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.900+0000 7f38bffff640 1 -- 172.21.15.116:0/4135473570 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8049040 con 0x5585b6d17b60 2024-09-12T03:08:28.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.900+0000 7f38bffff640 1 -- 172.21.15.116:0/4135473570 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38c80bc430 con 0x5585b6d17b60 2024-09-12T03:08:28.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.900+0000 7f38bffff640 1 -- 172.21.15.116:0/4135473570 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8003de0 con 0x5585b6d17b60 2024-09-12T03:08:28.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.901+0000 7f38df29d740 1 -- 172.21.15.116:0/4135473570 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5585b6d17b60 msgr2=0x5585b6d21a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:28.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.901+0000 7f38df29d740 1 --2- 172.21.15.116:0/4135473570 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5585b6d17b60 0x5585b6d21a00 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f38c80ed020 tx=0x7f38c8005c00 comp rx=0 tx=0).stop 2024-09-12T03:08:28.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.901+0000 7f38df29d740 1 -- 172.21.15.116:0/4135473570 shutdown_connections 2024-09-12T03:08:28.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.901+0000 7f38df29d740 1 --2- 172.21.15.116:0/4135473570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585b6d22d40 0x5585b6d3a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.901+0000 7f38df29d740 1 --2- 172.21.15.116:0/4135473570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5585b6d18480 0x5585b6d3a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.901+0000 7f38df29d740 1 --2- 172.21.15.116:0/4135473570 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5585b6d17b60 0x5585b6d21a00 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.901+0000 7f38df29d740 1 -- 172.21.15.116:0/4135473570 >> 172.21.15.116:0/4135473570 conn(0x5585b6d140f0 msgr2=0x5585b6d212f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:28.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.901+0000 7f38df29d740 1 -- 172.21.15.116:0/4135473570 shutdown_connections 2024-09-12T03:08:28.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.901+0000 7f38df29d740 1 -- 172.21.15.116:0/4135473570 wait complete. 2024-09-12T03:08:28.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.901+0000 7f38df29d740 1 -- start start 2024-09-12T03:08:28.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.901+0000 7f38df29d740 1 Processor -- start 2024-09-12T03:08:28.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.902+0000 7f38df29d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5585b6d17b60 0x5585b6d266d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:28.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.902+0000 7f38df29d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585b6d18480 0x5585b6d26c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:28.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.902+0000 7f38df29d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5585b6d22d40 0x5585b6d27150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:28.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.902+0000 7f38df29d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f38d81abb60 con 0x5585b6d17b60 2024-09-12T03:08:28.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.902+0000 7f38df29d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f38d81acfd0 con 0x5585b6d18480 2024-09-12T03:08:28.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.902+0000 7f38df29d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f38d81ac230 con 0x5585b6d22d40 2024-09-12T03:08:28.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.902+0000 7f38df29d740 10 client.0 _ll_register_callbacks cb 0x7f38cc00a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:28.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.902+0000 7f38d659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5585b6d17b60 0x5585b6d266d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:28.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.902+0000 7f38d659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5585b6d17b60 0x5585b6d266d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52972/0 (socket says 172.21.15.116:52972) 2024-09-12T03:08:28.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.902+0000 7f38d659c640 1 -- 172.21.15.116:0/319700313 learned_addr learned my addr 172.21.15.116:0/319700313 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:28.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.902+0000 7f38d5d9b640 1 --2- 172.21.15.116:0/319700313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585b6d18480 0x5585b6d26c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:28.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.902+0000 7f38d6d9d640 1 --2- 172.21.15.116:0/319700313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5585b6d22d40 0x5585b6d27150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:28.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.902+0000 7f38d659c640 1 -- 172.21.15.116:0/319700313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5585b6d22d40 msgr2=0x5585b6d27150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:28.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.902+0000 7f38d659c640 1 --2- 172.21.15.116:0/319700313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5585b6d22d40 0x5585b6d27150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.902+0000 7f38d659c640 1 -- 172.21.15.116:0/319700313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585b6d18480 msgr2=0x5585b6d26c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:28.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.902+0000 7f38d659c640 1 --2- 172.21.15.116:0/319700313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585b6d18480 0x5585b6d26c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.902+0000 7f38d659c640 1 -- 172.21.15.116:0/319700313 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585b6d9c400 con 0x5585b6d17b60 2024-09-12T03:08:28.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.902+0000 7f38d659c640 1 --2- 172.21.15.116:0/319700313 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5585b6d17b60 0x5585b6d266d0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f38c80b8700 tx=0x7f38c80c4a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0ddfcdd3a3c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:28.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.902+0000 7f38bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:28.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.902+0000 7f38bdffb640 1 -- 172.21.15.116:0/319700313 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c80d8020 con 0x5585b6d17b60 2024-09-12T03:08:28.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.902+0000 7f38bdffb640 1 -- 172.21.15.116:0/319700313 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f38c8003c80 con 0x5585b6d17b60 2024-09-12T03:08:28.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.903+0000 7f38df29d740 1 -- 172.21.15.116:0/319700313 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38c80bc430 con 0x5585b6d17b60 2024-09-12T03:08:28.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.903+0000 7f38bdffb640 1 -- 172.21.15.116:0/319700313 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f38c80bbd30 con 0x5585b6d17b60 2024-09-12T03:08:28.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.903+0000 7f38df29d740 10 client.16752 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:28.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.903+0000 7f38df29d740 1 -- 172.21.15.116:0/319700313 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5585b6d9d6c0 con 0x5585b6d17b60 2024-09-12T03:08:28.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.903+0000 7f38bdffb640 1 -- 172.21.15.116:0/319700313 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38c80bc430 con 0x5585b6d17b60 2024-09-12T03:08:28.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.903+0000 7f38bdffb640 1 -- 172.21.15.116:0/319700313 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f38c803bb90 con 0x5585b6d17b60 2024-09-12T03:08:28.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.903+0000 7f38df29d740 10 client.16752 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:28.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.903+0000 7f38df29d740 10 client.16752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:08:28.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.903+0000 7f38df29d740 20 client.16752 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:28.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.903+0000 7f38df29d740 1 --2- 172.21.15.116:0/319700313 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5585b6d9da50 0x5585b6d9fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:28.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.903+0000 7f38df29d740 4 client.16752 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:08:28.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.903+0000 7f38df29d740 1 -- 172.21.15.116:0/319700313 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5585b6c84d00 con 0x5585b6d9da50 2024-09-12T03:08:28.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.903+0000 7f38d5d9b640 1 --2- 172.21.15.116:0/319700313 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5585b6d9da50 0x5585b6d9fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:28.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.904+0000 7f38d5d9b640 1 --2- 172.21.15.116:0/319700313 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5585b6d9da50 0x5585b6d9fe40 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d40b4619104b72d server_cookie=9aac948785e6afde in_seq=0 out_seq=0 2024-09-12T03:08:28.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.904+0000 7f38bdffb640 10 client.16752 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:28.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.905+0000 7f38bdffb640 1 -- 172.21.15.116:0/319700313 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5585b6c84d00 con 0x5585b6d9da50 2024-09-12T03:08:28.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.905+0000 7f38bdffb640 10 client.16752 handle_command_reply: tid=0 2024-09-12T03:08:28.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.905+0000 7f38df29d740 1 client.16752 shutdown 2024-09-12T03:08:28.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.905+0000 7f38df29d740 1 -- 172.21.15.116:0/319700313 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5585b6d17b60 msgr2=0x5585b6d266d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:28.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.905+0000 7f38df29d740 1 --2- 172.21.15.116:0/319700313 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5585b6d17b60 0x5585b6d266d0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f38c80b8700 tx=0x7f38c80c4a90 comp rx=0 tx=0).stop 2024-09-12T03:08:28.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.905+0000 7f38df29d740 1 -- 172.21.15.116:0/319700313 shutdown_connections 2024-09-12T03:08:28.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.905+0000 7f38df29d740 1 --2- 172.21.15.116:0/319700313 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5585b6d9da50 0x5585b6d9fe40 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.905+0000 7f38df29d740 1 --2- 172.21.15.116:0/319700313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5585b6d22d40 0x5585b6d27150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.905+0000 7f38df29d740 1 --2- 172.21.15.116:0/319700313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5585b6d18480 0x5585b6d26c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.905+0000 7f38bdffb640 0 client.16752 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:28.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.905+0000 7f38df29d740 1 --2- 172.21.15.116:0/319700313 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5585b6d17b60 0x5585b6d266d0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.905+0000 7f38df29d740 1 -- 172.21.15.116:0/319700313 >> 172.21.15.116:0/319700313 conn(0x5585b6d140f0 msgr2=0x5585b6d13270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:28.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.905+0000 7f38df29d740 1 -- 172.21.15.116:0/319700313 shutdown_connections 2024-09-12T03:08:28.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.905+0000 7f38df29d740 1 -- 172.21.15.116:0/319700313 wait complete. 2024-09-12T03:08:28.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.905+0000 7f38df29d740 20 client.16752 trim_cache size 0 max 16384 2024-09-12T03:08:28.911 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:08:28.911 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:08:28.911 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "c169cb46-7d2d-4565-88cf-d56b89070d99", 2024-09-12T03:08:28.911 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:08:28.911 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:08:28.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.906+0000 7f38dcec4640 1 -- 172.21.15.116:0/340667178 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f38c00a2c00 msgr2=0x7f38c00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:28.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.906+0000 7f38dcec4640 1 --2- 172.21.15.116:0/340667178 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f38c00a2c00 0x7f38c00a50c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f38d0009fd0 tx=0x7f38d0005bd0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.906+0000 7f38dcec4640 1 -- 172.21.15.116:0/340667178 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38d8100a80 msgr2=0x7f38d81992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:28.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.906+0000 7f38dcec4640 1 --2- 172.21.15.116:0/340667178 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38d8100a80 0x7f38d81992f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f38c8009230 tx=0x7f38c8004270 comp rx=0 tx=0).stop 2024-09-12T03:08:28.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.906+0000 7f38dcec4640 1 -- 172.21.15.116:0/340667178 shutdown_connections 2024-09-12T03:08:28.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.906+0000 7f38dcec4640 1 --2- 172.21.15.116:0/340667178 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f38c00a2c00 0x7f38c00a50c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.906+0000 7f38dcec4640 1 --2- 172.21.15.116:0/340667178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38d8106a80 0x7f38d81a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.906+0000 7f38dcec4640 1 --2- 172.21.15.116:0/340667178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38d8101420 0x7f38d8199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.906+0000 7f38dcec4640 1 --2- 172.21.15.116:0/340667178 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38d8100a80 0x7f38d81992f0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:28.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.906+0000 7f38dcec4640 1 -- 172.21.15.116:0/340667178 >> 172.21.15.116:0/340667178 conn(0x7f38d80fc7a0 msgr2=0x7f38d80fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:28.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.906+0000 7f38dcec4640 1 -- 172.21.15.116:0/340667178 shutdown_connections 2024-09-12T03:08:28.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:28.906+0000 7f38dcec4640 1 -- 172.21.15.116:0/340667178 wait complete. 2024-09-12T03:08:29.642 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:08:29.905 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:08:30.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.436+0000 7fdb1ee31640 1 -- 172.21.15.116:0/3149901127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb18106a80 msgr2=0x7fdb18106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:30.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.436+0000 7fdb1ee31640 1 --2- 172.21.15.116:0/3149901127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb18106a80 0x7fdb18106e60 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fdb10009930 tx=0x7fdb1002a420 comp rx=0 tx=0).stop 2024-09-12T03:08:30.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.436+0000 7fdb1ee31640 1 -- 172.21.15.116:0/3149901127 shutdown_connections 2024-09-12T03:08:30.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.436+0000 7fdb1ee31640 1 --2- 172.21.15.116:0/3149901127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb18101420 0x7fdb1810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.436+0000 7fdb1ee31640 1 --2- 172.21.15.116:0/3149901127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb18100a80 0x7fdb18100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.436+0000 7fdb1ee31640 1 --2- 172.21.15.116:0/3149901127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb18106a80 0x7fdb18106e60 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.436+0000 7fdb1ee31640 1 -- 172.21.15.116:0/3149901127 >> 172.21.15.116:0/3149901127 conn(0x7fdb180fc7a0 msgr2=0x7fdb180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:30.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.436+0000 7fdb1ee31640 1 -- 172.21.15.116:0/3149901127 shutdown_connections 2024-09-12T03:08:30.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.436+0000 7fdb1ee31640 1 -- 172.21.15.116:0/3149901127 wait complete. 2024-09-12T03:08:30.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.437+0000 7fdb1ee31640 1 Processor -- start 2024-09-12T03:08:30.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.437+0000 7fdb1ee31640 1 -- start start 2024-09-12T03:08:30.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.437+0000 7fdb1ee31640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb18100a80 0x7fdb181984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:30.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.437+0000 7fdb1ee31640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb18101420 0x7fdb181989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:30.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.437+0000 7fdb1ee31640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb18106a80 0x7fdb1819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:30.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.437+0000 7fdb1ee31640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdb18110e40 con 0x7fdb18101420 2024-09-12T03:08:30.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.437+0000 7fdb1ee31640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdb18110cc0 con 0x7fdb18106a80 2024-09-12T03:08:30.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.437+0000 7fdb1ee31640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdb18110fc0 con 0x7fdb18100a80 2024-09-12T03:08:30.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.437+0000 7fdb17fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb18101420 0x7fdb181989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:30.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.437+0000 7fdb17fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb18101420 0x7fdb181989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52996/0 (socket says 172.21.15.116:52996) 2024-09-12T03:08:30.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.437+0000 7fdb17fff640 1 -- 172.21.15.116:0/3468730371 learned_addr learned my addr 172.21.15.116:0/3468730371 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:30.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.437+0000 7fdb1cbcd640 1 --2- 172.21.15.116:0/3468730371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb18100a80 0x7fdb181984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:30.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.437+0000 7fdb1d3ce640 1 --2- 172.21.15.116:0/3468730371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb18106a80 0x7fdb1819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:30.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.437+0000 7fdb17fff640 1 -- 172.21.15.116:0/3468730371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb18100a80 msgr2=0x7fdb181984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:30.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.437+0000 7fdb17fff640 1 --2- 172.21.15.116:0/3468730371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb18100a80 0x7fdb181984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.437+0000 7fdb17fff640 1 -- 172.21.15.116:0/3468730371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb18106a80 msgr2=0x7fdb1819cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:30.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.437+0000 7fdb17fff640 1 --2- 172.21.15.116:0/3468730371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb18106a80 0x7fdb1819cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.437+0000 7fdb17fff640 1 -- 172.21.15.116:0/3468730371 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1819d450 con 0x7fdb18101420 2024-09-12T03:08:30.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.438+0000 7fdb17fff640 1 --2- 172.21.15.116:0/3468730371 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb18101420 0x7fdb181989e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800cce0 tx=0x7fdb0800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:30.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.438+0000 7fdb15ffb640 1 -- 172.21.15.116:0/3468730371 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb08004b20 con 0x7fdb18101420 2024-09-12T03:08:30.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.438+0000 7fdb1ee31640 1 -- 172.21.15.116:0/3468730371 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb1819d740 con 0x7fdb18101420 2024-09-12T03:08:30.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.438+0000 7fdb15ffb640 1 -- 172.21.15.116:0/3468730371 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb08004cc0 con 0x7fdb18101420 2024-09-12T03:08:30.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.438+0000 7fdb15ffb640 1 -- 172.21.15.116:0/3468730371 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0800f7d0 con 0x7fdb18101420 2024-09-12T03:08:30.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.438+0000 7fdb1ee31640 1 -- 172.21.15.116:0/3468730371 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb181a5020 con 0x7fdb18101420 2024-09-12T03:08:30.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.441+0000 7fdb15ffb640 1 -- 172.21.15.116:0/3468730371 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fdb0800f970 con 0x7fdb18101420 2024-09-12T03:08:30.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.441+0000 7fdb2120a740 1 Processor -- start 2024-09-12T03:08:30.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.441+0000 7fdb2120a740 1 -- start start 2024-09-12T03:08:30.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.441+0000 7fdb2120a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1a3bb94a0 0x55e1a3bb9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:30.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.441+0000 7fdb2120a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a3bb9dc0 0x55e1a3bc4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:30.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.441+0000 7fdb2120a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e1a3bc4680 0x55e1a3bc6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:30.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.441+0000 7fdb2120a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e1a3b1f2e0 con 0x55e1a3bc4680 2024-09-12T03:08:30.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.441+0000 7fdb2120a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e1a3ac6ce0 con 0x55e1a3bb94a0 2024-09-12T03:08:30.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.441+0000 7fdb2120a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e1a3b1b2a0 con 0x55e1a3bb9dc0 2024-09-12T03:08:30.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.441+0000 7fdb15ffb640 1 --2- 172.21.15.116:0/3468730371 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fdb040a2b80 0x7fdb040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:30.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.441+0000 7fdb17fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e1a3bc4680 0x55e1a3bc6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:30.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.441+0000 7fdb17fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e1a3bc4680 0x55e1a3bc6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53010/0 (socket says 172.21.15.116:53010) 2024-09-12T03:08:30.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.441+0000 7fdb17fff640 1 -- 172.21.15.116:0/1763703385 learned_addr learned my addr 172.21.15.116:0/1763703385 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:30.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.442+0000 7fdb1d3ce640 1 --2- 172.21.15.116:0/1763703385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1a3bb94a0 0x55e1a3bb9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:30.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.442+0000 7fdb1d3ce640 1 --2- 172.21.15.116:0/3468730371 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fdb040a2b80 0x7fdb040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:30.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.442+0000 7fdb1cbcd640 1 --2- 172.21.15.116:0/1763703385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a3bb9dc0 0x55e1a3bc4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:30.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.442+0000 7fdb15ffb640 1 -- 172.21.15.116:0/3468730371 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fdb080cf190 con 0x7fdb18101420 2024-09-12T03:08:30.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.442+0000 7fdb17fff640 1 -- 172.21.15.116:0/1763703385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a3bb9dc0 msgr2=0x55e1a3bc4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:30.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.442+0000 7fdb17fff640 1 --2- 172.21.15.116:0/1763703385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a3bb9dc0 0x55e1a3bc4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.442+0000 7fdb17fff640 1 -- 172.21.15.116:0/1763703385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1a3bb94a0 msgr2=0x55e1a3bb9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:30.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.442+0000 7fdb17fff640 1 --2- 172.21.15.116:0/1763703385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1a3bb94a0 0x55e1a3bb9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.442+0000 7fdb17fff640 1 -- 172.21.15.116:0/1763703385 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1a3aab440 con 0x55e1a3bc4680 2024-09-12T03:08:30.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.442+0000 7fdb1d3ce640 1 --2- 172.21.15.116:0/1763703385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1a3bb94a0 0x55e1a3bb9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:08:30.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.442+0000 7fdb17fff640 1 --2- 172.21.15.116:0/1763703385 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e1a3bc4680 0x55e1a3bc6a70 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800ab20 tx=0x7fdb0801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6a229d2667166d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:30.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.442+0000 7fdb1d3ce640 1 --2- 172.21.15.116:0/3468730371 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fdb040a2b80 0x7fdb040a5040 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdb00005fd0 tx=0x7fdb00005900 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:30.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.442+0000 7fdb0effd640 1 -- 172.21.15.116:0/1763703385 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0801d070 con 0x55e1a3bc4680 2024-09-12T03:08:30.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.442+0000 7fdb0effd640 1 -- 172.21.15.116:0/1763703385 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb080ce030 con 0x55e1a3bc4680 2024-09-12T03:08:30.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.442+0000 7fdb0effd640 1 -- 172.21.15.116:0/1763703385 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0800fc60 con 0x55e1a3bc4680 2024-09-12T03:08:30.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.443+0000 7fdb2120a740 1 -- 172.21.15.116:0/1763703385 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e1a3bc4680 msgr2=0x55e1a3bc6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:30.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.443+0000 7fdb2120a740 1 --2- 172.21.15.116:0/1763703385 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e1a3bc4680 0x55e1a3bc6a70 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800ab20 tx=0x7fdb0801b000 comp rx=0 tx=0).stop 2024-09-12T03:08:30.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.443+0000 7fdb2120a740 1 -- 172.21.15.116:0/1763703385 shutdown_connections 2024-09-12T03:08:30.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.443+0000 7fdb2120a740 1 --2- 172.21.15.116:0/1763703385 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e1a3bc4680 0x55e1a3bc6a70 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.443+0000 7fdb2120a740 1 --2- 172.21.15.116:0/1763703385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a3bb9dc0 0x55e1a3bc4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.443+0000 7fdb2120a740 1 --2- 172.21.15.116:0/1763703385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1a3bb94a0 0x55e1a3bb9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.443+0000 7fdb2120a740 1 -- 172.21.15.116:0/1763703385 >> 172.21.15.116:0/1763703385 conn(0x55e1a3bb5a30 msgr2=0x55e1a3bb5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:30.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.443+0000 7fdb2120a740 1 -- 172.21.15.116:0/1763703385 shutdown_connections 2024-09-12T03:08:30.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.443+0000 7fdb2120a740 1 -- 172.21.15.116:0/1763703385 wait complete. 2024-09-12T03:08:30.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.443+0000 7fdb2120a740 1 -- start start 2024-09-12T03:08:30.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.443+0000 7fdb2120a740 1 Processor -- start 2024-09-12T03:08:30.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.444+0000 7fdb2120a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1a3bb94a0 0x55e1a3bc4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:30.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.444+0000 7fdb2120a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e1a3bb9dc0 0x55e1a3bc1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:30.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.444+0000 7fdb2120a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a3bc4680 0x55e1a3bc18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:30.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.444+0000 7fdb2120a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e1a3bc71f0 con 0x55e1a3bb9dc0 2024-09-12T03:08:30.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.444+0000 7fdb2120a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e1a3ada050 con 0x55e1a3bb94a0 2024-09-12T03:08:30.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.444+0000 7fdb2120a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e1a3bc7370 con 0x55e1a3bc4680 2024-09-12T03:08:30.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.444+0000 7fdb2120a740 10 client.0 _ll_register_callbacks cb 0x55e1a3b1a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:30.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.444+0000 7fdb17fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e1a3bb9dc0 0x55e1a3bc1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:30.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.444+0000 7fdb17fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e1a3bb9dc0 0x55e1a3bc1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53018/0 (socket says 172.21.15.116:53018) 2024-09-12T03:08:30.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.444+0000 7fdb17fff640 1 -- 172.21.15.116:0/2603995231 learned_addr learned my addr 172.21.15.116:0/2603995231 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:30.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.444+0000 7fdb1cbcd640 1 --2- 172.21.15.116:0/2603995231 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1a3bb94a0 0x55e1a3bc4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:30.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.444+0000 7fdb1d3ce640 1 --2- 172.21.15.116:0/2603995231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a3bc4680 0x55e1a3bc18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:30.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.444+0000 7fdb17fff640 1 -- 172.21.15.116:0/2603995231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a3bc4680 msgr2=0x55e1a3bc18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:30.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.444+0000 7fdb17fff640 1 --2- 172.21.15.116:0/2603995231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a3bc4680 0x55e1a3bc18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.444+0000 7fdb17fff640 1 -- 172.21.15.116:0/2603995231 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1a3bb94a0 msgr2=0x55e1a3bc4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:30.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.444+0000 7fdb17fff640 1 --2- 172.21.15.116:0/2603995231 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1a3bb94a0 0x55e1a3bc4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.444+0000 7fdb17fff640 1 -- 172.21.15.116:0/2603995231 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1a38903c0 con 0x55e1a3bb9dc0 2024-09-12T03:08:30.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.444+0000 7fdb17fff640 1 --2- 172.21.15.116:0/2603995231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e1a3bb9dc0 0x55e1a3bc1300 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fdb0809b720 tx=0x7fdb0809fe50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5196d757c5cda1ba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:30.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.444+0000 7fdaf7fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:30.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.444+0000 7fdaf7fff640 1 -- 172.21.15.116:0/2603995231 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0801d040 con 0x55e1a3bb9dc0 2024-09-12T03:08:30.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.445+0000 7fdaf7fff640 1 -- 172.21.15.116:0/2603995231 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fdb0809ae70 con 0x55e1a3bb9dc0 2024-09-12T03:08:30.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.445+0000 7fdb2120a740 1 -- 172.21.15.116:0/2603995231 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb080ce030 con 0x55e1a3bb9dc0 2024-09-12T03:08:30.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.445+0000 7fdaf7fff640 1 -- 172.21.15.116:0/2603995231 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdb080ba050 con 0x55e1a3bb9dc0 2024-09-12T03:08:30.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.445+0000 7fdb2120a740 10 client.16764 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:30.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.445+0000 7fdb2120a740 1 -- 172.21.15.116:0/2603995231 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e1a3bd2c20 con 0x55e1a3bb9dc0 2024-09-12T03:08:30.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.445+0000 7fdaf7fff640 1 -- 172.21.15.116:0/2603995231 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb080ce030 con 0x55e1a3bb9dc0 2024-09-12T03:08:30.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.445+0000 7fdaf7fff640 1 -- 172.21.15.116:0/2603995231 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdb080860c0 con 0x55e1a3bb9dc0 2024-09-12T03:08:30.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.445+0000 7fdb2120a740 10 client.16764 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:30.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.445+0000 7fdb2120a740 10 client.16764 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:08:30.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.445+0000 7fdb2120a740 20 client.16764 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:30.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.445+0000 7fdb2120a740 1 --2- 172.21.15.116:0/2603995231 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e1a3bd3f70 0x55e1a3bd6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:30.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.445+0000 7fdb2120a740 4 client.16764 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:08:30.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.445+0000 7fdb2120a740 1 -- 172.21.15.116:0/2603995231 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1a3b2c850 con 0x55e1a3bd3f70 2024-09-12T03:08:30.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.445+0000 7fdb1cbcd640 1 --2- 172.21.15.116:0/2603995231 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e1a3bd3f70 0x55e1a3bd6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:30.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.446+0000 7fdb1cbcd640 1 --2- 172.21.15.116:0/2603995231 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e1a3bd3f70 0x55e1a3bd6360 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=918924e8bf8a472e server_cookie=d453b0cfb3aef669 in_seq=0 out_seq=0 2024-09-12T03:08:30.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.446+0000 7fdaf7fff640 10 client.16764 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:30.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.447+0000 7fdaf7fff640 1 -- 172.21.15.116:0/2603995231 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e1a3b2c850 con 0x55e1a3bd3f70 2024-09-12T03:08:30.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.447+0000 7fdaf7fff640 10 client.16764 handle_command_reply: tid=0 2024-09-12T03:08:30.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.447+0000 7fdb2120a740 1 client.16764 shutdown 2024-09-12T03:08:30.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.447+0000 7fdb2120a740 1 -- 172.21.15.116:0/2603995231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e1a3bb9dc0 msgr2=0x55e1a3bc1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:30.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.447+0000 7fdb2120a740 1 --2- 172.21.15.116:0/2603995231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e1a3bb9dc0 0x55e1a3bc1300 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fdb0809b720 tx=0x7fdb0809fe50 comp rx=0 tx=0).stop 2024-09-12T03:08:30.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.447+0000 7fdb2120a740 1 -- 172.21.15.116:0/2603995231 shutdown_connections 2024-09-12T03:08:30.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.447+0000 7fdb2120a740 1 --2- 172.21.15.116:0/2603995231 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e1a3bd3f70 0x55e1a3bd6360 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.447+0000 7fdb2120a740 1 --2- 172.21.15.116:0/2603995231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a3bc4680 0x55e1a3bc18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.447+0000 7fdb2120a740 1 --2- 172.21.15.116:0/2603995231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e1a3bb9dc0 0x55e1a3bc1300 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.447+0000 7fdaf7fff640 0 client.16764 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:30.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.447+0000 7fdb2120a740 1 --2- 172.21.15.116:0/2603995231 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1a3bb94a0 0x55e1a3bc4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.447+0000 7fdb2120a740 1 -- 172.21.15.116:0/2603995231 >> 172.21.15.116:0/2603995231 conn(0x55e1a3bb5a30 msgr2=0x55e1a3bbb760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:30.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.447+0000 7fdb2120a740 1 -- 172.21.15.116:0/2603995231 shutdown_connections 2024-09-12T03:08:30.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.447+0000 7fdb2120a740 1 -- 172.21.15.116:0/2603995231 wait complete. 2024-09-12T03:08:30.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.447+0000 7fdb2120a740 20 client.16764 trim_cache size 0 max 16384 2024-09-12T03:08:30.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.464+0000 7fdb2120a740 1 Processor -- start 2024-09-12T03:08:30.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.464+0000 7fdb2120a740 1 -- start start 2024-09-12T03:08:30.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.464+0000 7fdb2120a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1a3bb94a0 0x55e1a3bd9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:30.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.464+0000 7fdb2120a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e1a3bb9dc0 0x55e1a3bd98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:30.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.464+0000 7fdb2120a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a3bc4680 0x55e1a3bbe560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:30.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.464+0000 7fdb2120a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdb181afea0 con 0x55e1a3bb9dc0 2024-09-12T03:08:30.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.464+0000 7fdb2120a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdb181ad770 con 0x55e1a3bb94a0 2024-09-12T03:08:30.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.464+0000 7fdb2120a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdb181ab5c0 con 0x55e1a3bc4680 2024-09-12T03:08:30.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.464+0000 7fdb17fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e1a3bb9dc0 0x55e1a3bd98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:30.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.464+0000 7fdb17fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e1a3bb9dc0 0x55e1a3bd98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53024/0 (socket says 172.21.15.116:53024) 2024-09-12T03:08:30.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.464+0000 7fdb17fff640 1 -- 172.21.15.116:0/131268342 learned_addr learned my addr 172.21.15.116:0/131268342 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:30.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.465+0000 7fdb1d3ce640 1 --2- 172.21.15.116:0/131268342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a3bc4680 0x55e1a3bbe560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:30.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.465+0000 7fdb17fff640 1 -- 172.21.15.116:0/131268342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a3bc4680 msgr2=0x55e1a3bbe560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:30.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.465+0000 7fdb1cbcd640 1 --2- 172.21.15.116:0/131268342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1a3bb94a0 0x55e1a3bd9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:30.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.465+0000 7fdb17fff640 1 --2- 172.21.15.116:0/131268342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a3bc4680 0x55e1a3bbe560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.465+0000 7fdb17fff640 1 -- 172.21.15.116:0/131268342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1a3bb94a0 msgr2=0x55e1a3bd9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:30.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.465+0000 7fdb17fff640 1 --2- 172.21.15.116:0/131268342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1a3bb94a0 0x55e1a3bd9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.465+0000 7fdb17fff640 1 -- 172.21.15.116:0/131268342 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1a3bc02b0 con 0x55e1a3bb9dc0 2024-09-12T03:08:30.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.465+0000 7fdb1d3ce640 1 --2- 172.21.15.116:0/131268342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a3bc4680 0x55e1a3bbe560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:08:30.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.465+0000 7fdb1cbcd640 1 --2- 172.21.15.116:0/131268342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1a3bb94a0 0x55e1a3bd9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:08:30.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.465+0000 7fdb17fff640 1 --2- 172.21.15.116:0/131268342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e1a3bb9dc0 0x55e1a3bd98c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fdb080a4160 tx=0x7fdb0800ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67343fef11d37a31 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:30.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.465+0000 7fdb0e7fc640 1 -- 172.21.15.116:0/131268342 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0801d070 con 0x55e1a3bb9dc0 2024-09-12T03:08:30.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.465+0000 7fdb0e7fc640 1 -- 172.21.15.116:0/131268342 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb080ce070 con 0x55e1a3bb9dc0 2024-09-12T03:08:30.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.465+0000 7fdb0e7fc640 1 -- 172.21.15.116:0/131268342 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb080860d0 con 0x55e1a3bb9dc0 2024-09-12T03:08:30.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.466+0000 7fdb2120a740 1 -- 172.21.15.116:0/131268342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e1a3bb9dc0 msgr2=0x55e1a3bd98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:30.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.466+0000 7fdb2120a740 1 --2- 172.21.15.116:0/131268342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e1a3bb9dc0 0x55e1a3bd98c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fdb080a4160 tx=0x7fdb0800ff90 comp rx=0 tx=0).stop 2024-09-12T03:08:30.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.466+0000 7fdb2120a740 1 -- 172.21.15.116:0/131268342 shutdown_connections 2024-09-12T03:08:30.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.466+0000 7fdb2120a740 1 --2- 172.21.15.116:0/131268342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a3bc4680 0x55e1a3bbe560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.466+0000 7fdb2120a740 1 --2- 172.21.15.116:0/131268342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e1a3bb9dc0 0x55e1a3bd98c0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.466+0000 7fdb2120a740 1 --2- 172.21.15.116:0/131268342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1a3bb94a0 0x55e1a3bd9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.466+0000 7fdb2120a740 1 -- 172.21.15.116:0/131268342 >> 172.21.15.116:0/131268342 conn(0x55e1a3bb5a30 msgr2=0x55e1a3bbc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:30.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.466+0000 7fdb2120a740 1 -- 172.21.15.116:0/131268342 shutdown_connections 2024-09-12T03:08:30.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.466+0000 7fdb2120a740 1 -- 172.21.15.116:0/131268342 wait complete. 2024-09-12T03:08:30.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.466+0000 7fdb2120a740 1 -- start start 2024-09-12T03:08:30.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.466+0000 7fdb2120a740 1 Processor -- start 2024-09-12T03:08:30.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.467+0000 7fdb2120a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1a3bb94a0 0x55e1a3bd7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:30.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.467+0000 7fdb2120a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a3bb9dc0 0x55e1a3bd76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:30.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.467+0000 7fdb2120a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e1a3bc4680 0x55e1a3bd7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:30.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.467+0000 7fdb2120a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdb181abc30 con 0x55e1a3bc4680 2024-09-12T03:08:30.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.467+0000 7fdb2120a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdb181ad0a0 con 0x55e1a3bb94a0 2024-09-12T03:08:30.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.467+0000 7fdb2120a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdb181ac300 con 0x55e1a3bb9dc0 2024-09-12T03:08:30.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.467+0000 7fdb2120a740 10 client.0 _ll_register_callbacks cb 0x7fdb0000f960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:30.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.467+0000 7fdb1d3ce640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e1a3bc4680 0x55e1a3bd7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:30.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.467+0000 7fdb1d3ce640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e1a3bc4680 0x55e1a3bd7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53038/0 (socket says 172.21.15.116:53038) 2024-09-12T03:08:30.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.467+0000 7fdb1d3ce640 1 -- 172.21.15.116:0/156532673 learned_addr learned my addr 172.21.15.116:0/156532673 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:30.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.467+0000 7fdb1cbcd640 1 --2- 172.21.15.116:0/156532673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1a3bb94a0 0x55e1a3bd7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:30.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.467+0000 7fdb17fff640 1 --2- 172.21.15.116:0/156532673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a3bb9dc0 0x55e1a3bd76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:30.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.467+0000 7fdb1d3ce640 1 -- 172.21.15.116:0/156532673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a3bb9dc0 msgr2=0x55e1a3bd76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:30.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.467+0000 7fdb1d3ce640 1 --2- 172.21.15.116:0/156532673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a3bb9dc0 0x55e1a3bd76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.467+0000 7fdb1d3ce640 1 -- 172.21.15.116:0/156532673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1a3bb94a0 msgr2=0x55e1a3bd7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:30.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.467+0000 7fdb1d3ce640 1 --2- 172.21.15.116:0/156532673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1a3bb94a0 0x55e1a3bd7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.467+0000 7fdb1d3ce640 1 -- 172.21.15.116:0/156532673 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1a3bc1c60 con 0x55e1a3bc4680 2024-09-12T03:08:30.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.467+0000 7fdb1d3ce640 1 --2- 172.21.15.116:0/156532673 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e1a3bc4680 0x55e1a3bd7c10 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fdb00007910 tx=0x7fdb000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10f6d2bf3568a10 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:30.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.467+0000 7fdaf7fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:30.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.468+0000 7fdaf7fff640 1 -- 172.21.15.116:0/156532673 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb00024070 con 0x55e1a3bc4680 2024-09-12T03:08:30.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.468+0000 7fdaf7fff640 1 -- 172.21.15.116:0/156532673 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fdb0000dd90 con 0x55e1a3bc4680 2024-09-12T03:08:30.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.468+0000 7fdb2120a740 1 -- 172.21.15.116:0/156532673 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb080ce070 con 0x55e1a3bc4680 2024-09-12T03:08:30.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.468+0000 7fdaf7fff640 1 -- 172.21.15.116:0/156532673 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdb0002f070 con 0x55e1a3bc4680 2024-09-12T03:08:30.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.468+0000 7fdb2120a740 10 client.16770 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:30.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.468+0000 7fdb2120a740 1 -- 172.21.15.116:0/156532673 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e1a3c3e980 con 0x55e1a3bc4680 2024-09-12T03:08:30.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.468+0000 7fdaf7fff640 1 -- 172.21.15.116:0/156532673 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb080ce070 con 0x55e1a3bc4680 2024-09-12T03:08:30.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.468+0000 7fdaf7fff640 1 -- 172.21.15.116:0/156532673 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdb00004bc0 con 0x55e1a3bc4680 2024-09-12T03:08:30.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.468+0000 7fdb2120a740 10 client.16770 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:30.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.468+0000 7fdb2120a740 10 client.16770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:08:30.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.468+0000 7fdb2120a740 20 client.16770 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:30.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.468+0000 7fdb2120a740 1 --2- 172.21.15.116:0/156532673 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e1a3c3f0b0 0x55e1a3c414d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:30.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.468+0000 7fdb2120a740 4 client.16770 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:08:30.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.468+0000 7fdb2120a740 1 -- 172.21.15.116:0/156532673 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e1a3b26d60 con 0x55e1a3c3f0b0 2024-09-12T03:08:30.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.469+0000 7fdb1cbcd640 1 --2- 172.21.15.116:0/156532673 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e1a3c3f0b0 0x55e1a3c414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:30.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.469+0000 7fdaf7fff640 10 client.16770 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:30.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.469+0000 7fdb1cbcd640 1 --2- 172.21.15.116:0/156532673 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e1a3c3f0b0 0x55e1a3c414d0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97ff0901f1570808 server_cookie=5e72738a5e36ad03 in_seq=0 out_seq=0 2024-09-12T03:08:30.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.469+0000 7fdaf7fff640 1 -- 172.21.15.116:0/156532673 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e1a3b26d60 con 0x55e1a3c3f0b0 2024-09-12T03:08:30.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.469+0000 7fdaf7fff640 10 client.16770 handle_command_reply: tid=0 2024-09-12T03:08:30.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.469+0000 7fdb2120a740 1 client.16770 shutdown 2024-09-12T03:08:30.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.469+0000 7fdb2120a740 1 -- 172.21.15.116:0/156532673 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e1a3bc4680 msgr2=0x55e1a3bd7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:30.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.469+0000 7fdb2120a740 1 --2- 172.21.15.116:0/156532673 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e1a3bc4680 0x55e1a3bd7c10 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fdb00007910 tx=0x7fdb000046c0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.470+0000 7fdb2120a740 1 -- 172.21.15.116:0/156532673 shutdown_connections 2024-09-12T03:08:30.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.470+0000 7fdb2120a740 1 --2- 172.21.15.116:0/156532673 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e1a3c3f0b0 0x55e1a3c414d0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.470+0000 7fdb2120a740 1 --2- 172.21.15.116:0/156532673 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e1a3bc4680 0x55e1a3bd7c10 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.470+0000 7fdb2120a740 1 --2- 172.21.15.116:0/156532673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a3bb9dc0 0x55e1a3bd76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.470+0000 7fdb2120a740 1 --2- 172.21.15.116:0/156532673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e1a3bb94a0 0x55e1a3bd7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.470+0000 7fdb2120a740 1 -- 172.21.15.116:0/156532673 >> 172.21.15.116:0/156532673 conn(0x55e1a3bb5a30 msgr2=0x55e1a3bc2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:30.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.470+0000 7fdaf7fff640 0 client.16770 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:30.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.470+0000 7fdb2120a740 1 -- 172.21.15.116:0/156532673 shutdown_connections 2024-09-12T03:08:30.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.470+0000 7fdb2120a740 1 -- 172.21.15.116:0/156532673 wait complete. 2024-09-12T03:08:30.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.470+0000 7fdb2120a740 20 client.16770 trim_cache size 0 max 16384 2024-09-12T03:08:30.480 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:08:30.481 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:08:30.481 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": { 2024-09-12T03:08:30.481 INFO:teuthology.orchestra.run.smithi116.stdout: "c169cb46-7d2d-4565-88cf-d56b89070d99": { 2024-09-12T03:08:30.481 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/", 2024-09-12T03:08:30.481 INFO:teuthology.orchestra.run.smithi116.stdout: "tag": "c169cb46-7d2d-4565-88cf-d56b89070d99", 2024-09-12T03:08:30.481 INFO:teuthology.orchestra.run.smithi116.stdout: "options": "recursive,force" 2024-09-12T03:08:30.481 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:08:30.481 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:08:30.481 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:08:30.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.470+0000 7fdb1ee31640 1 -- 172.21.15.116:0/3468730371 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fdb040a2b80 msgr2=0x7fdb040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:30.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.470+0000 7fdb1ee31640 1 --2- 172.21.15.116:0/3468730371 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fdb040a2b80 0x7fdb040a5040 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdb00005fd0 tx=0x7fdb00005900 comp rx=0 tx=0).stop 2024-09-12T03:08:30.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.470+0000 7fdb1ee31640 1 -- 172.21.15.116:0/3468730371 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb18101420 msgr2=0x7fdb181989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:30.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.470+0000 7fdb1ee31640 1 --2- 172.21.15.116:0/3468730371 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb18101420 0x7fdb181989e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800cce0 tx=0x7fdb0800a620 comp rx=0 tx=0).stop 2024-09-12T03:08:30.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.470+0000 7fdb1ee31640 1 -- 172.21.15.116:0/3468730371 shutdown_connections 2024-09-12T03:08:30.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.470+0000 7fdb1ee31640 1 --2- 172.21.15.116:0/3468730371 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fdb040a2b80 0x7fdb040a5040 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.470+0000 7fdb1ee31640 1 --2- 172.21.15.116:0/3468730371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb18106a80 0x7fdb1819cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.470+0000 7fdb1ee31640 1 --2- 172.21.15.116:0/3468730371 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb18101420 0x7fdb181989e0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.470+0000 7fdb1ee31640 1 --2- 172.21.15.116:0/3468730371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb18100a80 0x7fdb181984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:30.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.470+0000 7fdb1ee31640 1 -- 172.21.15.116:0/3468730371 >> 172.21.15.116:0/3468730371 conn(0x7fdb180fc7a0 msgr2=0x7fdb1806aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:30.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.471+0000 7fdb1ee31640 1 -- 172.21.15.116:0/3468730371 shutdown_connections 2024-09-12T03:08:30.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:30.471+0000 7fdb1ee31640 1 -- 172.21.15.116:0/3468730371 wait complete. 2024-09-12T03:08:30.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:30 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:08:30.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:30 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:08:30.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:30 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:08:30.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:30 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:08:30.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:30 smithi116 ceph-mon[31410]: pgmap v503: 97 pgs: 97 active+clean; 2.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:08:30.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:30 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:08:30.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:30 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:08:30.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:30 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:08:30.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:30 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:08:30.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:30 smithi183 ceph-mon[35535]: pgmap v503: 97 pgs: 97 active+clean; 2.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:08:30.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:30 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:08:30.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:30 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:08:30.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:30 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:08:30.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:30 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:08:30.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:30 smithi177 ceph-mon[35305]: pgmap v503: 97 pgs: 97 active+clean; 2.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:08:31.274 INFO:tasks.cephfs.filesystem:scrub status for tag:c169cb46-7d2d-4565-88cf-d56b89070d99 - {'path': '/', 'tag': 'c169cb46-7d2d-4565-88cf-d56b89070d99', 'options': 'recursive,force'} 2024-09-12T03:08:32.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:32 smithi116 ceph-mon[31410]: pgmap v504: 97 pgs: 97 active+clean; 2.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:08:32.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:32 smithi183 ceph-mon[35535]: pgmap v504: 97 pgs: 97 active+clean; 2.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:08:32.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:32 smithi177 ceph-mon[35305]: pgmap v504: 97 pgs: 97 active+clean; 2.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:08:34.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:34 smithi116 ceph-mon[31410]: pgmap v505: 97 pgs: 97 active+clean; 2.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:08:34.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:34 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:08:34.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:34 smithi183 ceph-mon[35535]: pgmap v505: 97 pgs: 97 active+clean; 2.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:08:34.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:34 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:08:34.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:34 smithi177 ceph-mon[35305]: pgmap v505: 97 pgs: 97 active+clean; 2.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:08:34.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:34 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:08:36.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:36 smithi183 ceph-mon[35535]: pgmap v506: 97 pgs: 97 active+clean; 2.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:08:36.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:36 smithi177 ceph-mon[35305]: pgmap v506: 97 pgs: 97 active+clean; 2.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:08:36.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:36 smithi116 ceph-mon[31410]: pgmap v506: 97 pgs: 97 active+clean; 2.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:08:38.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:38 smithi183 ceph-mon[35535]: pgmap v507: 97 pgs: 97 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:08:38.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:38 smithi177 ceph-mon[35305]: pgmap v507: 97 pgs: 97 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:08:38.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:38 smithi116 ceph-mon[31410]: pgmap v507: 97 pgs: 97 active+clean; 2.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:08:40.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:40 smithi116 ceph-mon[31410]: pgmap v508: 97 pgs: 97 active+clean; 2.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:08:40.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:40 smithi183 ceph-mon[35535]: pgmap v508: 97 pgs: 97 active+clean; 2.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:08:40.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:40 smithi177 ceph-mon[35305]: pgmap v508: 97 pgs: 97 active+clean; 2.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:08:42.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:42 smithi183 ceph-mon[35535]: pgmap v509: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:08:42.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:42 smithi177 ceph-mon[35305]: pgmap v509: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:08:42.795 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.795 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.795 INFO:tasks.workunit.client.0.smithi116.stdout:127/600: open fd = 3 2024-09-12T03:08:42.795 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.795 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.795 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.795 INFO:tasks.workunit.client.0.smithi116.stdout:128/600: open fd = 3 2024-09-12T03:08:42.795 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.795 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.795 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.796 INFO:tasks.workunit.client.0.smithi116.stdout:129/600: open fd = 3 2024-09-12T03:08:42.796 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.796 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.796 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.796 INFO:tasks.workunit.client.0.smithi116.stdout:130/600: open fd = 3 2024-09-12T03:08:42.796 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.796 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.796 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.796 INFO:tasks.workunit.client.0.smithi116.stdout:131/600: open fd = 3 2024-09-12T03:08:42.796 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.796 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.796 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.796 INFO:tasks.workunit.client.0.smithi116.stdout:132/600: open fd = 3 2024-09-12T03:08:42.797 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.797 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.797 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.797 INFO:tasks.workunit.client.0.smithi116.stdout:133/600: open fd = 3 2024-09-12T03:08:42.797 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.797 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.797 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.797 INFO:tasks.workunit.client.0.smithi116.stdout:134/600: open fd = 3 2024-09-12T03:08:42.797 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.797 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.797 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.797 INFO:tasks.workunit.client.0.smithi116.stdout:135/600: open fd = 3 2024-09-12T03:08:42.797 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.797 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.798 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.798 INFO:tasks.workunit.client.0.smithi116.stdout:136/600: open fd = 3 2024-09-12T03:08:42.798 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.798 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.798 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.798 INFO:tasks.workunit.client.0.smithi116.stdout:137/600: open fd = 3 2024-09-12T03:08:42.798 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.798 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.798 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.798 INFO:tasks.workunit.client.0.smithi116.stdout:138/600: open fd = 3 2024-09-12T03:08:42.798 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.798 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.798 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.798 INFO:tasks.workunit.client.0.smithi116.stdout:139/600: open fd = 3 2024-09-12T03:08:42.798 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.799 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.799 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.799 INFO:tasks.workunit.client.0.smithi116.stdout:140/600: open fd = 3 2024-09-12T03:08:42.799 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.799 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.799 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.799 INFO:tasks.workunit.client.0.smithi116.stdout:141/600: open fd = 3 2024-09-12T03:08:42.799 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.799 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.799 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.799 INFO:tasks.workunit.client.0.smithi116.stdout:142/600: open fd = 3 2024-09-12T03:08:42.799 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.799 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.799 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.799 INFO:tasks.workunit.client.0.smithi116.stdout:143/600: open fd = 3 2024-09-12T03:08:42.799 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.800 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.800 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.800 INFO:tasks.workunit.client.0.smithi116.stdout:144/600: open fd = 3 2024-09-12T03:08:42.800 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.800 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.800 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.800 INFO:tasks.workunit.client.0.smithi116.stdout:145/600: open fd = 3 2024-09-12T03:08:42.800 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.800 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.800 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.800 INFO:tasks.workunit.client.0.smithi116.stdout:146/600: open fd = 3 2024-09-12T03:08:42.800 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.800 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.800 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.801 INFO:tasks.workunit.client.0.smithi116.stdout:147/600: open fd = 3 2024-09-12T03:08:42.801 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.801 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.801 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.801 INFO:tasks.workunit.client.0.smithi116.stdout:148/600: open fd = 3 2024-09-12T03:08:42.801 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.801 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.801 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.801 INFO:tasks.workunit.client.0.smithi116.stdout:149/600: open fd = 3 2024-09-12T03:08:42.801 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.801 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.801 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.801 INFO:tasks.workunit.client.0.smithi116.stdout:150/600: open fd = 3 2024-09-12T03:08:42.801 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.802 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.802 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.802 INFO:tasks.workunit.client.0.smithi116.stdout:151/600: open fd = 3 2024-09-12T03:08:42.802 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.802 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.802 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.802 INFO:tasks.workunit.client.0.smithi116.stdout:152/600: open fd = 3 2024-09-12T03:08:42.802 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.802 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.802 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.802 INFO:tasks.workunit.client.0.smithi116.stdout:153/600: open fd = 3 2024-09-12T03:08:42.802 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.802 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.802 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.802 INFO:tasks.workunit.client.0.smithi116.stdout:154/600: open fd = 3 2024-09-12T03:08:42.803 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.803 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.803 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.803 INFO:tasks.workunit.client.0.smithi116.stdout:155/600: open fd = 3 2024-09-12T03:08:42.803 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.803 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.803 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.803 INFO:tasks.workunit.client.0.smithi116.stdout:156/600: open fd = 3 2024-09-12T03:08:42.803 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.803 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.803 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.803 INFO:tasks.workunit.client.0.smithi116.stdout:157/600: open fd = 3 2024-09-12T03:08:42.803 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.803 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.803 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.804 INFO:tasks.workunit.client.0.smithi116.stdout:158/600: open fd = 3 2024-09-12T03:08:42.804 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.804 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.804 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.804 INFO:tasks.workunit.client.0.smithi116.stdout:159/600: open fd = 3 2024-09-12T03:08:42.804 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.804 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.804 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.804 INFO:tasks.workunit.client.0.smithi116.stdout:160/600: open fd = 3 2024-09-12T03:08:42.804 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.804 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.804 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.804 INFO:tasks.workunit.client.0.smithi116.stdout:161/600: open fd = 3 2024-09-12T03:08:42.804 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.805 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.805 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.805 INFO:tasks.workunit.client.0.smithi116.stdout:162/600: open fd = 3 2024-09-12T03:08:42.805 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.805 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.805 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.805 INFO:tasks.workunit.client.0.smithi116.stdout:163/600: open fd = 3 2024-09-12T03:08:42.805 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.805 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.805 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.805 INFO:tasks.workunit.client.0.smithi116.stdout:164/600: open fd = 3 2024-09-12T03:08:42.805 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.805 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.805 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.806 INFO:tasks.workunit.client.0.smithi116.stdout:165/600: open fd = 3 2024-09-12T03:08:42.806 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.806 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.806 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.806 INFO:tasks.workunit.client.0.smithi116.stdout:166/600: open fd = 3 2024-09-12T03:08:42.806 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.806 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.806 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.806 INFO:tasks.workunit.client.0.smithi116.stdout:167/600: open fd = 3 2024-09-12T03:08:42.806 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.806 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.806 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.806 INFO:tasks.workunit.client.0.smithi116.stdout:168/600: open fd = 3 2024-09-12T03:08:42.806 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.806 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.807 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.807 INFO:tasks.workunit.client.0.smithi116.stdout:169/600: open fd = 3 2024-09-12T03:08:42.807 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.807 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.807 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.807 INFO:tasks.workunit.client.0.smithi116.stdout:170/600: open fd = 3 2024-09-12T03:08:42.807 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.807 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.807 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.807 INFO:tasks.workunit.client.0.smithi116.stdout:171/600: open fd = 3 2024-09-12T03:08:42.807 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.807 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.807 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.807 INFO:tasks.workunit.client.0.smithi116.stdout:172/600: open fd = 3 2024-09-12T03:08:42.808 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.808 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.808 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.808 INFO:tasks.workunit.client.0.smithi116.stdout:173/600: open fd = 3 2024-09-12T03:08:42.808 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.808 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.808 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.808 INFO:tasks.workunit.client.0.smithi116.stdout:174/600: open fd = 3 2024-09-12T03:08:42.808 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.808 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.808 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.808 INFO:tasks.workunit.client.0.smithi116.stdout:175/600: open fd = 3 2024-09-12T03:08:42.808 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.809 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.809 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.809 INFO:tasks.workunit.client.0.smithi116.stdout:176/600: open fd = 3 2024-09-12T03:08:42.809 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.809 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.809 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.809 INFO:tasks.workunit.client.0.smithi116.stdout:177/600: open fd = 3 2024-09-12T03:08:42.809 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.809 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.809 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.809 INFO:tasks.workunit.client.0.smithi116.stdout:178/600: open fd = 3 2024-09-12T03:08:42.809 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.809 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.810 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.810 INFO:tasks.workunit.client.0.smithi116.stdout:179/600: open fd = 3 2024-09-12T03:08:42.810 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.810 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.810 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.810 INFO:tasks.workunit.client.0.smithi116.stdout:180/600: open fd = 3 2024-09-12T03:08:42.810 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.810 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.810 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.810 INFO:tasks.workunit.client.0.smithi116.stdout:181/600: open fd = 3 2024-09-12T03:08:42.810 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.810 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.810 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.811 INFO:tasks.workunit.client.0.smithi116.stdout:182/600: open fd = 3 2024-09-12T03:08:42.811 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.811 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.811 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.811 INFO:tasks.workunit.client.0.smithi116.stdout:183/600: open fd = 3 2024-09-12T03:08:42.811 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.811 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.811 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.811 INFO:tasks.workunit.client.0.smithi116.stdout:184/600: open fd = 3 2024-09-12T03:08:42.811 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.811 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.811 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.811 INFO:tasks.workunit.client.0.smithi116.stdout:185/600: open fd = 3 2024-09-12T03:08:42.811 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.812 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.812 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.812 INFO:tasks.workunit.client.0.smithi116.stdout:186/600: open fd = 3 2024-09-12T03:08:42.812 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.812 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.812 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.812 INFO:tasks.workunit.client.0.smithi116.stdout:187/600: open fd = 3 2024-09-12T03:08:42.812 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.812 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.812 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:08:42.812 INFO:tasks.workunit.client.0.smithi116.stdout:188/600: open fd = 3 2024-09-12T03:08:42.812 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.812 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:08:42.813 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:42 smithi116 ceph-mon[31410]: pgmap v509: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:08:44.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:44 smithi183 ceph-mon[35535]: pgmap v510: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:08:44.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:44 smithi177 ceph-mon[35305]: pgmap v510: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:08:44.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:44 smithi116 ceph-mon[31410]: pgmap v510: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:08:46.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:46 smithi183 ceph-mon[35535]: pgmap v511: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:08:46.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:46 smithi177 ceph-mon[35305]: pgmap v511: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:08:46.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:46 smithi116 ceph-mon[31410]: pgmap v511: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:08:47.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:47 smithi183 ceph-mon[35535]: pgmap v512: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:08:47.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:47 smithi177 ceph-mon[35305]: pgmap v512: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:08:47.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:47 smithi116 ceph-mon[31410]: pgmap v512: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:08:50.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:50 smithi116 ceph-mon[31410]: pgmap v513: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:08:50.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:50 smithi183 ceph-mon[35535]: pgmap v513: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:08:50.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:50 smithi177 ceph-mon[35305]: pgmap v513: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:08:52.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:52 smithi116 ceph-mon[31410]: pgmap v514: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:08:52.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:52 smithi183 ceph-mon[35535]: pgmap v514: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:08:52.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:52 smithi177 ceph-mon[35305]: pgmap v514: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:08:52.876 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.1 2024-09-12T03:08:52.876 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:08:53.141 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:08:53.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.680+0000 7f7b97b1c640 1 -- 172.21.15.116:0/3178331848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b90100a80 msgr2=0x7f7b90100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:53.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.680+0000 7f7b97b1c640 1 --2- 172.21.15.116:0/3178331848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b90100a80 0x7f7b90100ee0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f7b84009cd0 tx=0x7f7b8402a8e0 comp rx=0 tx=0).stop 2024-09-12T03:08:53.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.680+0000 7f7b97b1c640 1 -- 172.21.15.116:0/3178331848 shutdown_connections 2024-09-12T03:08:53.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.680+0000 7f7b97b1c640 1 --2- 172.21.15.116:0/3178331848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b90101420 0x7f7b9010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:53.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.680+0000 7f7b97b1c640 1 --2- 172.21.15.116:0/3178331848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b90100a80 0x7f7b90100ee0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:53.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.680+0000 7f7b97b1c640 1 --2- 172.21.15.116:0/3178331848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b90106a80 0x7f7b90106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:53.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.680+0000 7f7b97b1c640 1 -- 172.21.15.116:0/3178331848 >> 172.21.15.116:0/3178331848 conn(0x7f7b900fc7a0 msgr2=0x7f7b900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:53.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.680+0000 7f7b97b1c640 1 -- 172.21.15.116:0/3178331848 shutdown_connections 2024-09-12T03:08:53.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.680+0000 7f7b97b1c640 1 -- 172.21.15.116:0/3178331848 wait complete. 2024-09-12T03:08:53.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.681+0000 7f7b97b1c640 1 Processor -- start 2024-09-12T03:08:53.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.681+0000 7f7b97b1c640 1 -- start start 2024-09-12T03:08:53.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.681+0000 7f7b97b1c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b90100a80 0x7f7b90197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:53.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.681+0000 7f7b97b1c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b90101420 0x7f7b901978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:53.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.681+0000 7f7b97b1c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b90106a80 0x7f7b9019bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:53.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.681+0000 7f7b97b1c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7b90110ea0 con 0x7f7b90100a80 2024-09-12T03:08:53.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.681+0000 7f7b97b1c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7b90110d20 con 0x7f7b90101420 2024-09-12T03:08:53.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.681+0000 7f7b97b1c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7b90111020 con 0x7f7b90106a80 2024-09-12T03:08:53.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.681+0000 7f7b958b8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b90100a80 0x7f7b90197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:53.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.681+0000 7f7b958b8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b90100a80 0x7f7b90197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60356/0 (socket says 172.21.15.116:60356) 2024-09-12T03:08:53.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.681+0000 7f7b958b8640 1 -- 172.21.15.116:0/1683540009 learned_addr learned my addr 172.21.15.116:0/1683540009 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:53.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.681+0000 7f7b960b9640 1 --2- 172.21.15.116:0/1683540009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b90106a80 0x7f7b9019bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:53.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.681+0000 7f7b958b8640 1 -- 172.21.15.116:0/1683540009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b90106a80 msgr2=0x7f7b9019bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:53.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.681+0000 7f7b950b7640 1 --2- 172.21.15.116:0/1683540009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b90101420 0x7f7b901978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:53.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.681+0000 7f7b958b8640 1 --2- 172.21.15.116:0/1683540009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b90106a80 0x7f7b9019bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:53.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.681+0000 7f7b958b8640 1 -- 172.21.15.116:0/1683540009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b90101420 msgr2=0x7f7b901978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:53.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.681+0000 7f7b958b8640 1 --2- 172.21.15.116:0/1683540009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b90101420 0x7f7b901978c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:53.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.681+0000 7f7b958b8640 1 -- 172.21.15.116:0/1683540009 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9019c330 con 0x7f7b90100a80 2024-09-12T03:08:53.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.682+0000 7f7b958b8640 1 --2- 172.21.15.116:0/1683540009 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b90100a80 0x7f7b90197380 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800b4a0 tx=0x7f7b8800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:53.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.682+0000 7f7b8effd640 1 -- 172.21.15.116:0/1683540009 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88014070 con 0x7f7b90100a80 2024-09-12T03:08:53.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.682+0000 7f7b8effd640 1 -- 172.21.15.116:0/1683540009 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b88010af0 con 0x7f7b90100a80 2024-09-12T03:08:53.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.682+0000 7f7b97b1c640 1 -- 172.21.15.116:0/1683540009 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b9019c620 con 0x7f7b90100a80 2024-09-12T03:08:53.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.682+0000 7f7b8effd640 1 -- 172.21.15.116:0/1683540009 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88017d10 con 0x7f7b90100a80 2024-09-12T03:08:53.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.682+0000 7f7b97b1c640 1 -- 172.21.15.116:0/1683540009 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b9006a790 con 0x7f7b90100a80 2024-09-12T03:08:53.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.683+0000 7f7b97b1c640 1 -- 172.21.15.116:0/1683540009 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b90100ee0 con 0x7f7b90100a80 2024-09-12T03:08:53.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.684+0000 7f7b8effd640 1 -- 172.21.15.116:0/1683540009 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f7b88017070 con 0x7f7b90100a80 2024-09-12T03:08:53.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.685+0000 7f7b8effd640 1 --2- 172.21.15.116:0/1683540009 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7b780a2c00 0x7f7b780a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:53.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.685+0000 7f7b950b7640 1 --2- 172.21.15.116:0/1683540009 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7b780a2c00 0x7f7b780a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:53.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.685+0000 7f7b8effd640 1 -- 172.21.15.116:0/1683540009 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f7b880ce3b0 con 0x7f7b90100a80 2024-09-12T03:08:53.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.685+0000 7f7b950b7640 1 --2- 172.21.15.116:0/1683540009 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7b780a2c00 0x7f7b780a50c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7b84006010 tx=0x7f7b84000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:53.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.690+0000 7f7b8effd640 1 -- 172.21.15.116:0/1683540009 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7b8809b340 con 0x7f7b90100a80 2024-09-12T03:08:53.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.889+0000 7f7b97b1c640 1 -- 172.21.15.116:0/1683540009 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b9006b210 con 0x7f7b90100a80 2024-09-12T03:08:53.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.890+0000 7f7b8effd640 1 -- 172.21.15.116:0/1683540009 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f7b8809ac40 con 0x7f7b90100a80 2024-09-12T03:08:53.890 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:08:53.890 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:08:53.891 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:08:53.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.892+0000 7f7b97b1c640 1 -- 172.21.15.116:0/1683540009 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7b780a2c00 msgr2=0x7f7b780a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:53.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.892+0000 7f7b97b1c640 1 --2- 172.21.15.116:0/1683540009 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7b780a2c00 0x7f7b780a50c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7b84006010 tx=0x7f7b84000f40 comp rx=0 tx=0).stop 2024-09-12T03:08:53.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.892+0000 7f7b97b1c640 1 -- 172.21.15.116:0/1683540009 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b90100a80 msgr2=0x7f7b90197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:53.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.892+0000 7f7b97b1c640 1 --2- 172.21.15.116:0/1683540009 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b90100a80 0x7f7b90197380 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800b4a0 tx=0x7f7b8800b960 comp rx=0 tx=0).stop 2024-09-12T03:08:53.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.893+0000 7f7b97b1c640 1 -- 172.21.15.116:0/1683540009 shutdown_connections 2024-09-12T03:08:53.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.893+0000 7f7b97b1c640 1 --2- 172.21.15.116:0/1683540009 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7b780a2c00 0x7f7b780a50c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:53.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.893+0000 7f7b97b1c640 1 --2- 172.21.15.116:0/1683540009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b90106a80 0x7f7b9019bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:53.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.893+0000 7f7b97b1c640 1 --2- 172.21.15.116:0/1683540009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b90101420 0x7f7b901978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:53.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.893+0000 7f7b97b1c640 1 --2- 172.21.15.116:0/1683540009 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b90100a80 0x7f7b90197380 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:53.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.893+0000 7f7b97b1c640 1 -- 172.21.15.116:0/1683540009 >> 172.21.15.116:0/1683540009 conn(0x7f7b900fc7a0 msgr2=0x7f7b900fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:53.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.893+0000 7f7b97b1c640 1 -- 172.21.15.116:0/1683540009 shutdown_connections 2024-09-12T03:08:53.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:53.893+0000 7f7b97b1c640 1 -- 172.21.15.116:0/1683540009 wait complete. 2024-09-12T03:08:54.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:54 smithi116 ceph-mon[31410]: pgmap v515: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:08:54.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:54 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1683540009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:08:54.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:54 smithi183 ceph-mon[35535]: pgmap v515: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:08:54.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:54 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1683540009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:08:54.712 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0' 2024-09-12T03:08:54.712 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0 2024-09-12T03:08:54.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:54 smithi177 ceph-mon[35305]: pgmap v515: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:08:54.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:54 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1683540009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:08:54.964 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:08:55.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.455+0000 7f919508f640 1 -- 172.21.15.116:0/2726273139 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9190101df0 msgr2=0x7f91901021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:55.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.455+0000 7f919508f640 1 --2- 172.21.15.116:0/2726273139 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9190101df0 0x7f91901021d0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f9180009900 tx=0x7f918002a410 comp rx=0 tx=0).stop 2024-09-12T03:08:55.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.455+0000 7f919508f640 1 -- 172.21.15.116:0/2726273139 shutdown_connections 2024-09-12T03:08:55.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.455+0000 7f919508f640 1 --2- 172.21.15.116:0/2726273139 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9190106f20 0x7f919010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.455+0000 7f919508f640 1 --2- 172.21.15.116:0/2726273139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91901027a0 0x7f91901067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.455+0000 7f919508f640 1 --2- 172.21.15.116:0/2726273139 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9190101df0 0x7f91901021d0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.455+0000 7f919508f640 1 -- 172.21.15.116:0/2726273139 >> 172.21.15.116:0/2726273139 conn(0x7f91900fd8b0 msgr2=0x7f91900ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:55.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.455+0000 7f919508f640 1 -- 172.21.15.116:0/2726273139 shutdown_connections 2024-09-12T03:08:55.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.455+0000 7f919508f640 1 -- 172.21.15.116:0/2726273139 wait complete. 2024-09-12T03:08:55.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.456+0000 7f919508f640 1 Processor -- start 2024-09-12T03:08:55.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.456+0000 7f919508f640 1 -- start start 2024-09-12T03:08:55.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.456+0000 7f919508f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9190101df0 0x7f91900744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:55.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.456+0000 7f919508f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91901027a0 0x7f9190074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:55.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.456+0000 7f919508f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9190106f20 0x7f9190074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:55.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.456+0000 7f919508f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9190112020 con 0x7f91901027a0 2024-09-12T03:08:55.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.456+0000 7f919508f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9190111ea0 con 0x7f9190101df0 2024-09-12T03:08:55.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.456+0000 7f919508f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f91901121a0 con 0x7f9190106f20 2024-09-12T03:08:55.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.457+0000 7f9187fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91901027a0 0x7f9190074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:55.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.457+0000 7f9187fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91901027a0 0x7f9190074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60372/0 (socket says 172.21.15.116:60372) 2024-09-12T03:08:55.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.457+0000 7f9187fff640 1 -- 172.21.15.116:0/540478203 learned_addr learned my addr 172.21.15.116:0/540478203 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:55.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.457+0000 7f918ed9d640 1 --2- 172.21.15.116:0/540478203 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9190101df0 0x7f91900744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:55.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.457+0000 7f918f59e640 1 --2- 172.21.15.116:0/540478203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9190106f20 0x7f9190074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:55.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.457+0000 7f9187fff640 1 -- 172.21.15.116:0/540478203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9190106f20 msgr2=0x7f9190074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:55.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.457+0000 7f9187fff640 1 --2- 172.21.15.116:0/540478203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9190106f20 0x7f9190074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.457+0000 7f9187fff640 1 -- 172.21.15.116:0/540478203 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9190101df0 msgr2=0x7f91900744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:55.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.457+0000 7f9187fff640 1 --2- 172.21.15.116:0/540478203 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9190101df0 0x7f91900744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.457+0000 7f9187fff640 1 -- 172.21.15.116:0/540478203 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91901a6ab0 con 0x7f91901027a0 2024-09-12T03:08:55.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.457+0000 7f918f59e640 1 --2- 172.21.15.116:0/540478203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9190106f20 0x7f9190074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:08:55.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.457+0000 7f9187fff640 1 --2- 172.21.15.116:0/540478203 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91901027a0 0x7f9190074a30 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f917800b4a0 tx=0x7f917800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:55.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.457+0000 7f918cd99640 1 -- 172.21.15.116:0/540478203 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9178014070 con 0x7f91901027a0 2024-09-12T03:08:55.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.457+0000 7f918cd99640 1 -- 172.21.15.116:0/540478203 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9178010af0 con 0x7f91901027a0 2024-09-12T03:08:55.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.457+0000 7f919508f640 1 -- 172.21.15.116:0/540478203 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91901a6da0 con 0x7f91901027a0 2024-09-12T03:08:55.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.457+0000 7f918cd99640 1 -- 172.21.15.116:0/540478203 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9178017d10 con 0x7f91901027a0 2024-09-12T03:08:55.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.457+0000 7f919508f640 1 -- 172.21.15.116:0/540478203 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91901a7250 con 0x7f91901027a0 2024-09-12T03:08:55.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.460+0000 7f918cd99640 1 -- 172.21.15.116:0/540478203 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f9178017070 con 0x7f91901027a0 2024-09-12T03:08:55.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.460+0000 7f9197468740 1 Processor -- start 2024-09-12T03:08:55.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.460+0000 7f9197468740 1 -- start start 2024-09-12T03:08:55.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.461+0000 7f9197468740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564a7bb16760 0x564a7bb16b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:55.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.461+0000 7f9197468740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a7bb17080 0x564a7bb21400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:55.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.461+0000 7f9197468740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564a7bb21940 0x564a7bb23d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:55.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.461+0000 7f9197468740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x564a7ba7c2a0 con 0x564a7bb21940 2024-09-12T03:08:55.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.461+0000 7f9197468740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x564a7ba23b10 con 0x564a7bb16760 2024-09-12T03:08:55.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.461+0000 7f9197468740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564a7ba78260 con 0x564a7bb17080 2024-09-12T03:08:55.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.461+0000 7f9187fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564a7bb21940 0x564a7bb23d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:55.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.461+0000 7f9187fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564a7bb21940 0x564a7bb23d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60380/0 (socket says 172.21.15.116:60380) 2024-09-12T03:08:55.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.461+0000 7f9187fff640 1 -- 172.21.15.116:0/3229418914 learned_addr learned my addr 172.21.15.116:0/3229418914 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:55.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.461+0000 7f918ed9d640 1 --2- 172.21.15.116:0/3229418914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a7bb17080 0x564a7bb21400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:55.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.461+0000 7f918f59e640 1 --2- 172.21.15.116:0/3229418914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564a7bb16760 0x564a7bb16b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:55.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.461+0000 7f918cd99640 1 --2- 172.21.15.116:0/540478203 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f917c0a2b80 0x7f917c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:55.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.461+0000 7f9187fff640 1 -- 172.21.15.116:0/3229418914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a7bb17080 msgr2=0x564a7bb21400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:55.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.461+0000 7f9187fff640 1 --2- 172.21.15.116:0/3229418914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a7bb17080 0x564a7bb21400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.461+0000 7f9187fff640 1 -- 172.21.15.116:0/3229418914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564a7bb16760 msgr2=0x564a7bb16b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:55.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.461+0000 7f9187fff640 1 --2- 172.21.15.116:0/3229418914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564a7bb16760 0x564a7bb16b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.461+0000 7f9187fff640 1 -- 172.21.15.116:0/3229418914 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a7b7edb90 con 0x564a7bb21940 2024-09-12T03:08:55.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.461+0000 7f918f59e640 1 --2- 172.21.15.116:0/3229418914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564a7bb16760 0x564a7bb16b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:55.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.461+0000 7f918f59e640 1 --2- 172.21.15.116:0/540478203 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f917c0a2b80 0x7f917c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:55.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.461+0000 7f918cd99640 1 -- 172.21.15.116:0/540478203 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f91780ce420 con 0x7f91901027a0 2024-09-12T03:08:55.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.462+0000 7f9187fff640 1 --2- 172.21.15.116:0/3229418914 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564a7bb21940 0x564a7bb23d30 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f9178013740 tx=0x7f917801de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af98becf6bce17ca server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:55.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.462+0000 7f918f59e640 1 --2- 172.21.15.116:0/540478203 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f917c0a2b80 0x7f917c0a5040 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f918800a3c0 tx=0x7f9188004b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:55.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.462+0000 7f91857fa640 1 -- 172.21.15.116:0/3229418914 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917809a7f0 con 0x564a7bb21940 2024-09-12T03:08:55.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.462+0000 7f91857fa640 1 -- 172.21.15.116:0/3229418914 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9178013580 con 0x564a7bb21940 2024-09-12T03:08:55.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.462+0000 7f91857fa640 1 -- 172.21.15.116:0/3229418914 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9178013170 con 0x564a7bb21940 2024-09-12T03:08:55.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.463+0000 7f9197468740 1 -- 172.21.15.116:0/3229418914 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564a7bb21940 msgr2=0x564a7bb23d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:55.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.463+0000 7f9197468740 1 --2- 172.21.15.116:0/3229418914 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564a7bb21940 0x564a7bb23d30 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f9178013740 tx=0x7f917801de80 comp rx=0 tx=0).stop 2024-09-12T03:08:55.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.463+0000 7f9197468740 1 -- 172.21.15.116:0/3229418914 shutdown_connections 2024-09-12T03:08:55.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.463+0000 7f9197468740 1 --2- 172.21.15.116:0/3229418914 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564a7bb21940 0x564a7bb23d30 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.463+0000 7f9197468740 1 --2- 172.21.15.116:0/3229418914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a7bb17080 0x564a7bb21400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.463+0000 7f9197468740 1 --2- 172.21.15.116:0/3229418914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564a7bb16760 0x564a7bb16b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.463+0000 7f9197468740 1 -- 172.21.15.116:0/3229418914 >> 172.21.15.116:0/3229418914 conn(0x564a7bb12cf0 msgr2=0x564a7bb130d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:55.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.463+0000 7f9197468740 1 -- 172.21.15.116:0/3229418914 shutdown_connections 2024-09-12T03:08:55.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.463+0000 7f9197468740 1 -- 172.21.15.116:0/3229418914 wait complete. 2024-09-12T03:08:55.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.463+0000 7f9197468740 1 -- start start 2024-09-12T03:08:55.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.463+0000 7f9197468740 1 Processor -- start 2024-09-12T03:08:55.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.463+0000 7f9197468740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564a7bb16760 0x564a7bb1e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:55.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.463+0000 7f9197468740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a7bb17080 0x564a7bb1ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:55.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.463+0000 7f9197468740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564a7bb21940 0x564a7bb1f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:55.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.463+0000 7f9197468740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x564a7bb24510 con 0x564a7bb21940 2024-09-12T03:08:55.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.463+0000 7f9197468740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x564a7ba046e0 con 0x564a7bb16760 2024-09-12T03:08:55.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.463+0000 7f9197468740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564a7bb24690 con 0x564a7bb17080 2024-09-12T03:08:55.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.464+0000 7f9197468740 10 client.0 _ll_register_callbacks cb 0x564a7ba28a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:55.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.464+0000 7f918f59e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564a7bb21940 0x564a7bb1f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:55.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.464+0000 7f918f59e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564a7bb21940 0x564a7bb1f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60382/0 (socket says 172.21.15.116:60382) 2024-09-12T03:08:55.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.464+0000 7f918f59e640 1 -- 172.21.15.116:0/1463777404 learned_addr learned my addr 172.21.15.116:0/1463777404 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:55.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.464+0000 7f918ed9d640 1 --2- 172.21.15.116:0/1463777404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564a7bb16760 0x564a7bb1e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:55.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.464+0000 7f9187fff640 1 --2- 172.21.15.116:0/1463777404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a7bb17080 0x564a7bb1ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:55.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.464+0000 7f918f59e640 1 -- 172.21.15.116:0/1463777404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a7bb17080 msgr2=0x564a7bb1ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:55.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.464+0000 7f918f59e640 1 --2- 172.21.15.116:0/1463777404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a7bb17080 0x564a7bb1ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.464+0000 7f918f59e640 1 -- 172.21.15.116:0/1463777404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564a7bb16760 msgr2=0x564a7bb1e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:55.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.464+0000 7f918f59e640 1 --2- 172.21.15.116:0/1463777404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564a7bb16760 0x564a7bb1e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.464+0000 7f918f59e640 1 -- 172.21.15.116:0/1463777404 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a7bb2d9b0 con 0x564a7bb21940 2024-09-12T03:08:55.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.464+0000 7f918f59e640 1 --2- 172.21.15.116:0/1463777404 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564a7bb21940 0x564a7bb1f250 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f9188011af0 tx=0x7f9188004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970fa502abf8a25d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:55.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.464+0000 7f91767fc640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:55.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.464+0000 7f91767fc640 1 -- 172.21.15.116:0/1463777404 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188022020 con 0x564a7bb21940 2024-09-12T03:08:55.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.464+0000 7f91767fc640 1 -- 172.21.15.116:0/1463777404 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f918801e450 con 0x564a7bb21940 2024-09-12T03:08:55.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.464+0000 7f9197468740 1 -- 172.21.15.116:0/1463777404 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564a7ba899d0 con 0x564a7bb21940 2024-09-12T03:08:55.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.464+0000 7f91767fc640 1 -- 172.21.15.116:0/1463777404 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f918801e650 con 0x564a7bb21940 2024-09-12T03:08:55.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.464+0000 7f9197468740 10 client.16788 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:55.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.464+0000 7f9197468740 1 -- 172.21.15.116:0/1463777404 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564a7bb30380 con 0x564a7bb21940 2024-09-12T03:08:55.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.465+0000 7f91767fc640 1 -- 172.21.15.116:0/1463777404 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564a7ba899d0 con 0x564a7bb21940 2024-09-12T03:08:55.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.465+0000 7f91767fc640 1 -- 172.21.15.116:0/1463777404 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f918801d890 con 0x564a7bb21940 2024-09-12T03:08:55.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.465+0000 7f9197468740 10 client.16788 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:55.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.465+0000 7f9197468740 10 client.16788 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:08:55.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.465+0000 7f9197468740 20 client.16788 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:55.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.465+0000 7f9197468740 1 --2- 172.21.15.116:0/1463777404 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x564a7bb31840 0x564a7bb33c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:55.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.465+0000 7f9197468740 4 client.16788 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:08:55.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.465+0000 7f9197468740 1 -- 172.21.15.116:0/1463777404 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a7ba83d20 con 0x564a7bb31840 2024-09-12T03:08:55.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.465+0000 7f918ed9d640 1 --2- 172.21.15.116:0/1463777404 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x564a7bb31840 0x564a7bb33c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:55.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.465+0000 7f918ed9d640 1 --2- 172.21.15.116:0/1463777404 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x564a7bb31840 0x564a7bb33c30 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ea2feac8371ad81 server_cookie=57335ed07e624c0 in_seq=0 out_seq=0 2024-09-12T03:08:55.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.465+0000 7f91767fc640 10 client.16788 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:55.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.466+0000 7f91767fc640 1 -- 172.21.15.116:0/1463777404 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564a7ba83d20 con 0x564a7bb31840 2024-09-12T03:08:55.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.466+0000 7f91767fc640 10 client.16788 handle_command_reply: tid=0 2024-09-12T03:08:55.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.466+0000 7f9197468740 1 client.16788 shutdown 2024-09-12T03:08:55.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.466+0000 7f9197468740 1 -- 172.21.15.116:0/1463777404 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564a7bb21940 msgr2=0x564a7bb1f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:55.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.466+0000 7f9197468740 1 --2- 172.21.15.116:0/1463777404 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564a7bb21940 0x564a7bb1f250 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f9188011af0 tx=0x7f9188004a40 comp rx=0 tx=0).stop 2024-09-12T03:08:55.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.466+0000 7f9197468740 1 -- 172.21.15.116:0/1463777404 shutdown_connections 2024-09-12T03:08:55.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.466+0000 7f9197468740 1 --2- 172.21.15.116:0/1463777404 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x564a7bb31840 0x564a7bb33c30 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.466+0000 7f9197468740 1 --2- 172.21.15.116:0/1463777404 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564a7bb21940 0x564a7bb1f250 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.466+0000 7f9197468740 1 --2- 172.21.15.116:0/1463777404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a7bb17080 0x564a7bb1ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.466+0000 7f9197468740 1 --2- 172.21.15.116:0/1463777404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564a7bb16760 0x564a7bb1e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.466+0000 7f9197468740 1 -- 172.21.15.116:0/1463777404 >> 172.21.15.116:0/1463777404 conn(0x564a7bb12cf0 msgr2=0x564a7bb18a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:55.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.467+0000 7f91767fc640 0 client.16788 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:55.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.467+0000 7f9197468740 1 -- 172.21.15.116:0/1463777404 shutdown_connections 2024-09-12T03:08:55.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.467+0000 7f9197468740 1 -- 172.21.15.116:0/1463777404 wait complete. 2024-09-12T03:08:55.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.467+0000 7f9197468740 20 client.16788 trim_cache size 0 max 16384 2024-09-12T03:08:55.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.483+0000 7f9197468740 1 Processor -- start 2024-09-12T03:08:55.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.483+0000 7f9197468740 1 -- start start 2024-09-12T03:08:55.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.484+0000 7f9197468740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564a7bb16760 0x564a7bb1d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:55.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.484+0000 7f9197468740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564a7bb17080 0x564a7bb394e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:55.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.484+0000 7f9197468740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a7bb21940 0x564a7bb39a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:55.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.484+0000 7f9197468740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f91901b2100 con 0x564a7bb17080 2024-09-12T03:08:55.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.484+0000 7f9197468740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f91901af9d0 con 0x564a7bb16760 2024-09-12T03:08:55.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.484+0000 7f9197468740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f91901ad820 con 0x564a7bb21940 2024-09-12T03:08:55.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.484+0000 7f9187fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564a7bb17080 0x564a7bb394e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:55.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.484+0000 7f9187fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564a7bb17080 0x564a7bb394e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60384/0 (socket says 172.21.15.116:60384) 2024-09-12T03:08:55.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.484+0000 7f9187fff640 1 -- 172.21.15.116:0/939733244 learned_addr learned my addr 172.21.15.116:0/939733244 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:55.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.484+0000 7f918f59e640 1 --2- 172.21.15.116:0/939733244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a7bb21940 0x564a7bb39a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:55.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.484+0000 7f918ed9d640 1 --2- 172.21.15.116:0/939733244 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564a7bb16760 0x564a7bb1d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:55.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.484+0000 7f9187fff640 1 -- 172.21.15.116:0/939733244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a7bb21940 msgr2=0x564a7bb39a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:55.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.484+0000 7f9187fff640 1 --2- 172.21.15.116:0/939733244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a7bb21940 0x564a7bb39a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.484+0000 7f9187fff640 1 -- 172.21.15.116:0/939733244 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564a7bb16760 msgr2=0x564a7bb1d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:55.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.484+0000 7f9187fff640 1 --2- 172.21.15.116:0/939733244 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564a7bb16760 0x564a7bb1d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.484+0000 7f9187fff640 1 -- 172.21.15.116:0/939733244 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a7bb1a160 con 0x564a7bb17080 2024-09-12T03:08:55.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.484+0000 7f918f59e640 1 --2- 172.21.15.116:0/939733244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a7bb21940 0x564a7bb39a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:08:55.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.484+0000 7f918ed9d640 1 --2- 172.21.15.116:0/939733244 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564a7bb16760 0x564a7bb1d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:08:55.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.484+0000 7f9187fff640 1 --2- 172.21.15.116:0/939733244 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564a7bb17080 0x564a7bb394e0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f917800b470 tx=0x7f91780047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa91a860f769ba3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:55.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.485+0000 7f9184ff9640 1 -- 172.21.15.116:0/939733244 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9178013380 con 0x564a7bb17080 2024-09-12T03:08:55.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.485+0000 7f9184ff9640 1 -- 172.21.15.116:0/939733244 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f917809b3b0 con 0x564a7bb17080 2024-09-12T03:08:55.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.485+0000 7f9184ff9640 1 -- 172.21.15.116:0/939733244 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9178020300 con 0x564a7bb17080 2024-09-12T03:08:55.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.485+0000 7f9197468740 1 -- 172.21.15.116:0/939733244 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564a7bb17080 msgr2=0x564a7bb394e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:55.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.485+0000 7f9197468740 1 --2- 172.21.15.116:0/939733244 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564a7bb17080 0x564a7bb394e0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f917800b470 tx=0x7f91780047d0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.485+0000 7f9197468740 1 -- 172.21.15.116:0/939733244 shutdown_connections 2024-09-12T03:08:55.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.485+0000 7f9197468740 1 --2- 172.21.15.116:0/939733244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a7bb21940 0x564a7bb39a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.485+0000 7f9197468740 1 --2- 172.21.15.116:0/939733244 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564a7bb17080 0x564a7bb394e0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.485+0000 7f9197468740 1 --2- 172.21.15.116:0/939733244 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564a7bb16760 0x564a7bb1d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.485+0000 7f9197468740 1 -- 172.21.15.116:0/939733244 >> 172.21.15.116:0/939733244 conn(0x564a7bb12cf0 msgr2=0x564a7bb20850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:55.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.485+0000 7f9197468740 1 -- 172.21.15.116:0/939733244 shutdown_connections 2024-09-12T03:08:55.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.485+0000 7f9197468740 1 -- 172.21.15.116:0/939733244 wait complete. 2024-09-12T03:08:55.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.486+0000 7f9197468740 1 -- start start 2024-09-12T03:08:55.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.486+0000 7f9197468740 1 Processor -- start 2024-09-12T03:08:55.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.486+0000 7f9197468740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564a7bb16760 0x564a7bb25680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:55.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.486+0000 7f9197468740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564a7bb17080 0x564a7bb25bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:55.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.486+0000 7f9197468740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a7bb21940 0x564a7bb26100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:55.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.486+0000 7f9197468740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f91901ade90 con 0x564a7bb16760 2024-09-12T03:08:55.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.486+0000 7f9197468740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f91901af300 con 0x564a7bb17080 2024-09-12T03:08:55.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.486+0000 7f9197468740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f91901ae560 con 0x564a7bb21940 2024-09-12T03:08:55.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.486+0000 7f9197468740 10 client.0 _ll_register_callbacks cb 0x7f91880117f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:55.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.486+0000 7f918ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564a7bb16760 0x564a7bb25680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:55.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.486+0000 7f918ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564a7bb16760 0x564a7bb25680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60398/0 (socket says 172.21.15.116:60398) 2024-09-12T03:08:55.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.486+0000 7f918ed9d640 1 -- 172.21.15.116:0/2215192242 learned_addr learned my addr 172.21.15.116:0/2215192242 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:08:55.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.486+0000 7f9187fff640 1 --2- 172.21.15.116:0/2215192242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564a7bb17080 0x564a7bb25bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:55.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.487+0000 7f918f59e640 1 --2- 172.21.15.116:0/2215192242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a7bb21940 0x564a7bb26100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:55.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.487+0000 7f918ed9d640 1 -- 172.21.15.116:0/2215192242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a7bb21940 msgr2=0x564a7bb26100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:55.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.487+0000 7f918ed9d640 1 --2- 172.21.15.116:0/2215192242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a7bb21940 0x564a7bb26100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.487+0000 7f918ed9d640 1 -- 172.21.15.116:0/2215192242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564a7bb17080 msgr2=0x564a7bb25bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:55.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.487+0000 7f918ed9d640 1 --2- 172.21.15.116:0/2215192242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564a7bb17080 0x564a7bb25bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.487+0000 7f918ed9d640 1 -- 172.21.15.116:0/2215192242 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a7bb1d780 con 0x564a7bb16760 2024-09-12T03:08:55.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.487+0000 7f9187fff640 1 --2- 172.21.15.116:0/2215192242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564a7bb17080 0x564a7bb25bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:55.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.487+0000 7f918ed9d640 1 --2- 172.21.15.116:0/2215192242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564a7bb16760 0x564a7bb25680 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f918002a930 tx=0x7f9180031860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=740c6e8268267881 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:55.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.487+0000 7f91767fc640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:08:55.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.487+0000 7f91767fc640 1 -- 172.21.15.116:0/2215192242 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918004c070 con 0x564a7bb16760 2024-09-12T03:08:55.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.487+0000 7f91767fc640 1 -- 172.21.15.116:0/2215192242 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f9180004450 con 0x564a7bb16760 2024-09-12T03:08:55.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.487+0000 7f9197468740 1 -- 172.21.15.116:0/2215192242 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f917809b3b0 con 0x564a7bb16760 2024-09-12T03:08:55.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.487+0000 7f91767fc640 1 -- 172.21.15.116:0/2215192242 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9180035040 con 0x564a7bb16760 2024-09-12T03:08:55.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.487+0000 7f9197468740 10 client.16794 fetch_fsmap learned FSMap version 25 2024-09-12T03:08:55.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.487+0000 7f9197468740 1 -- 172.21.15.116:0/2215192242 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564a7bb9c5c0 con 0x564a7bb16760 2024-09-12T03:08:55.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.487+0000 7f91767fc640 1 -- 172.21.15.116:0/2215192242 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f917809b3b0 con 0x564a7bb16760 2024-09-12T03:08:55.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.488+0000 7f91767fc640 1 -- 172.21.15.116:0/2215192242 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9180004c50 con 0x564a7bb16760 2024-09-12T03:08:55.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.488+0000 7f9197468740 10 client.16794 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:08:55.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.488+0000 7f9197468740 10 client.16794 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:08:55.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.488+0000 7f9197468740 20 client.16794 populate_metadata read hostname 'smithi116' 2024-09-12T03:08:55.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.488+0000 7f9197468740 1 --2- 172.21.15.116:0/2215192242 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x564a7bb9cdb0 0x564a7bb9f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:55.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.488+0000 7f9197468740 4 client.16794 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}] 2024-09-12T03:08:55.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.488+0000 7f9197468740 1 -- 172.21.15.116:0/2215192242 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}) -- 0x564a7b9ffec0 con 0x564a7bb9cdb0 2024-09-12T03:08:55.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.488+0000 7f9187fff640 1 --2- 172.21.15.116:0/2215192242 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x564a7bb9cdb0 0x564a7bb9f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:55.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.488+0000 7f9187fff640 1 --2- 172.21.15.116:0/2215192242 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x564a7bb9cdb0 0x564a7bb9f1a0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9035a317222d5e6 server_cookie=e7c902cfdc8ebe5e in_seq=0 out_seq=0 2024-09-12T03:08:55.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.488+0000 7f91767fc640 10 client.16794 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:55.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.497+0000 7f91767fc640 1 -- 172.21.15.116:0/2215192242 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564a7b9ffec0 con 0x564a7bb9cdb0 2024-09-12T03:08:55.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.497+0000 7f91767fc640 10 client.16794 handle_command_reply: tid=0 2024-09-12T03:08:55.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.497+0000 7f9197468740 1 client.16794 shutdown 2024-09-12T03:08:55.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.498+0000 7f9197468740 1 -- 172.21.15.116:0/2215192242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564a7bb16760 msgr2=0x564a7bb25680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:55.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.498+0000 7f9197468740 1 --2- 172.21.15.116:0/2215192242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564a7bb16760 0x564a7bb25680 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f918002a930 tx=0x7f9180031860 comp rx=0 tx=0).stop 2024-09-12T03:08:55.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.498+0000 7f9197468740 1 -- 172.21.15.116:0/2215192242 shutdown_connections 2024-09-12T03:08:55.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.498+0000 7f9197468740 1 --2- 172.21.15.116:0/2215192242 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x564a7bb9cdb0 0x564a7bb9f1a0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.498+0000 7f9197468740 1 --2- 172.21.15.116:0/2215192242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a7bb21940 0x564a7bb26100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.498+0000 7f9197468740 1 --2- 172.21.15.116:0/2215192242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564a7bb17080 0x564a7bb25bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.498+0000 7f91767fc640 0 client.16794 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:08:55.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.498+0000 7f9197468740 1 --2- 172.21.15.116:0/2215192242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564a7bb16760 0x564a7bb25680 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.498+0000 7f9197468740 1 -- 172.21.15.116:0/2215192242 >> 172.21.15.116:0/2215192242 conn(0x564a7bb12cf0 msgr2=0x564a7bb1e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:55.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.498+0000 7f9197468740 1 -- 172.21.15.116:0/2215192242 shutdown_connections 2024-09-12T03:08:55.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.498+0000 7f9197468740 1 -- 172.21.15.116:0/2215192242 wait complete. 2024-09-12T03:08:55.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.498+0000 7f9197468740 20 client.16794 trim_cache size 0 max 16384 2024-09-12T03:08:55.499 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:08:55.500 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 25, 2024-09-12T03:08:55.500 INFO:teuthology.orchestra.run.smithi116.stdout: "leader": 24494, 2024-09-12T03:08:55.500 INFO:teuthology.orchestra.run.smithi116.stdout: "set_version": 22, 2024-09-12T03:08:55.500 INFO:teuthology.orchestra.run.smithi116.stdout: "sets": { 2024-09-12T03:08:55.500 INFO:teuthology.orchestra.run.smithi116.stdout: "d2a1b25e": { 2024-09-12T03:08:55.500 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 22, 2024-09-12T03:08:55.500 INFO:teuthology.orchestra.run.smithi116.stdout: "age_ref": 0.0, 2024-09-12T03:08:55.500 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:08:55.500 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "QUIESCED", 2024-09-12T03:08:55.500 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:08:55.500 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:08:55.501 INFO:teuthology.orchestra.run.smithi116.stdout: "timeout": 90.0, 2024-09-12T03:08:55.501 INFO:teuthology.orchestra.run.smithi116.stdout: "expiration": 140.1, 2024-09-12T03:08:55.501 INFO:teuthology.orchestra.run.smithi116.stdout: "members": { 2024-09-12T03:08:55.501 INFO:teuthology.orchestra.run.smithi116.stdout: "file:/": { 2024-09-12T03:08:55.501 INFO:teuthology.orchestra.run.smithi116.stdout: "excluded": false, 2024-09-12T03:08:55.501 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:08:55.501 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "QUIESCED", 2024-09-12T03:08:55.501 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:08:55.501 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:08:55.501 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:08:55.501 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:08:55.501 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:08:55.501 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:08:55.501 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:08:55.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.498+0000 7f919508f640 1 -- 172.21.15.116:0/540478203 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f917c0a2b80 msgr2=0x7f917c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:55.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.498+0000 7f919508f640 1 --2- 172.21.15.116:0/540478203 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f917c0a2b80 0x7f917c0a5040 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f918800a3c0 tx=0x7f9188004b50 comp rx=0 tx=0).stop 2024-09-12T03:08:55.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.498+0000 7f919508f640 1 -- 172.21.15.116:0/540478203 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91901027a0 msgr2=0x7f9190074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:55.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.498+0000 7f919508f640 1 --2- 172.21.15.116:0/540478203 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91901027a0 0x7f9190074a30 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f917800b4a0 tx=0x7f917800b960 comp rx=0 tx=0).stop 2024-09-12T03:08:55.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.498+0000 7f919508f640 1 -- 172.21.15.116:0/540478203 shutdown_connections 2024-09-12T03:08:55.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.498+0000 7f919508f640 1 --2- 172.21.15.116:0/540478203 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f917c0a2b80 0x7f917c0a5040 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.498+0000 7f919508f640 1 --2- 172.21.15.116:0/540478203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9190106f20 0x7f9190074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.498+0000 7f919508f640 1 --2- 172.21.15.116:0/540478203 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91901027a0 0x7f9190074a30 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.498+0000 7f919508f640 1 --2- 172.21.15.116:0/540478203 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9190101df0 0x7f91900744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:55.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.498+0000 7f919508f640 1 -- 172.21.15.116:0/540478203 >> 172.21.15.116:0/540478203 conn(0x7f91900fd8b0 msgr2=0x7f91900fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:55.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.499+0000 7f919508f640 1 -- 172.21.15.116:0/540478203 shutdown_connections 2024-09-12T03:08:55.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:08:55.499+0000 7f919508f640 1 -- 172.21.15.116:0/540478203 wait complete. 2024-09-12T03:08:56.211 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd2a1b25e', quiesce took 3.3 seconds. Will release after: 16.8 2024-09-12T03:08:56.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:56 smithi116 ceph-mon[31410]: pgmap v516: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:08:56.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:56 smithi183 ceph-mon[35535]: pgmap v516: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:08:56.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:56 smithi177 ceph-mon[35305]: pgmap v516: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:08:58.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:08:58 smithi116 ceph-mon[31410]: pgmap v517: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:08:58.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:08:58 smithi183 ceph-mon[35535]: pgmap v517: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:08:58.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:08:58 smithi177 ceph-mon[35305]: pgmap v517: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:09:00.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:00 smithi116 ceph-mon[31410]: pgmap v518: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:09:00.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:00 smithi183 ceph-mon[35535]: pgmap v518: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:09:00.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:00 smithi177 ceph-mon[35305]: pgmap v518: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:09:01.275 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:09:01.535 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:09:02.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.131+0000 7f283af37640 1 -- 172.21.15.116:0/3485775034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2834102570 msgr2=0x7f28341029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:02.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.131+0000 7f283af37640 1 --2- 172.21.15.116:0/3485775034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2834102570 0x7f28341029b0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f2824009ca0 tx=0x7f282402a950 comp rx=0 tx=0).stop 2024-09-12T03:09:02.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.131+0000 7f283af37640 1 -- 172.21.15.116:0/3485775034 shutdown_connections 2024-09-12T03:09:02.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.131+0000 7f283af37640 1 --2- 172.21.15.116:0/3485775034 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2834102ef0 0x7f283410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.131+0000 7f283af37640 1 --2- 172.21.15.116:0/3485775034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2834102570 0x7f28341029b0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.131+0000 7f283af37640 1 --2- 172.21.15.116:0/3485775034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2834100e20 0x7f2834101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.131+0000 7f283af37640 1 -- 172.21.15.116:0/3485775034 >> 172.21.15.116:0/3485775034 conn(0x7f28340fc7d0 msgr2=0x7f28340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:02.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.131+0000 7f283af37640 1 -- 172.21.15.116:0/3485775034 shutdown_connections 2024-09-12T03:09:02.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.131+0000 7f283af37640 1 -- 172.21.15.116:0/3485775034 wait complete. 2024-09-12T03:09:02.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.132+0000 7f283af37640 1 Processor -- start 2024-09-12T03:09:02.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.132+0000 7f283af37640 1 -- start start 2024-09-12T03:09:02.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.132+0000 7f283af37640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2834100e20 0x7f2834197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:02.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.132+0000 7f283af37640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2834102570 0x7f2834197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:02.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.132+0000 7f283af37640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2834102ef0 0x7f283419be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:02.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.132+0000 7f283af37640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2834111020 con 0x7f2834102570 2024-09-12T03:09:02.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.132+0000 7f283af37640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2834110ea0 con 0x7f2834102ef0 2024-09-12T03:09:02.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.132+0000 7f283af37640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f28341111a0 con 0x7f2834100e20 2024-09-12T03:09:02.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.132+0000 7f2833fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2834102570 0x7f2834197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:02.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.132+0000 7f2833fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2834102570 0x7f2834197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56190/0 (socket says 172.21.15.116:56190) 2024-09-12T03:09:02.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.132+0000 7f2833fff640 1 -- 172.21.15.116:0/55711533 learned_addr learned my addr 172.21.15.116:0/55711533 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:02.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.132+0000 7f2838cd3640 1 --2- 172.21.15.116:0/55711533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2834100e20 0x7f2834197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:02.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.132+0000 7f28394d4640 1 --2- 172.21.15.116:0/55711533 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2834102ef0 0x7f283419be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:02.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.132+0000 7f2833fff640 1 -- 172.21.15.116:0/55711533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2834100e20 msgr2=0x7f2834197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:02.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.132+0000 7f2833fff640 1 --2- 172.21.15.116:0/55711533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2834100e20 0x7f2834197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.132+0000 7f2833fff640 1 -- 172.21.15.116:0/55711533 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2834102ef0 msgr2=0x7f283419be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:02.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.132+0000 7f2833fff640 1 --2- 172.21.15.116:0/55711533 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2834102ef0 0x7f283419be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.132+0000 7f2833fff640 1 -- 172.21.15.116:0/55711533 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f283419c4f0 con 0x7f2834102570 2024-09-12T03:09:02.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.132+0000 7f2833fff640 1 --2- 172.21.15.116:0/55711533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2834102570 0x7f2834197a80 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f2824005cc0 tx=0x7f282402f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:02.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.133+0000 7f2831ffb640 1 -- 172.21.15.116:0/55711533 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824034040 con 0x7f2834102570 2024-09-12T03:09:02.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.133+0000 7f2831ffb640 1 -- 172.21.15.116:0/55711533 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2824030920 con 0x7f2834102570 2024-09-12T03:09:02.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.133+0000 7f2831ffb640 1 -- 172.21.15.116:0/55711533 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282403a5c0 con 0x7f2834102570 2024-09-12T03:09:02.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.133+0000 7f283af37640 1 -- 172.21.15.116:0/55711533 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f283419c780 con 0x7f2834102570 2024-09-12T03:09:02.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.133+0000 7f283af37640 1 -- 172.21.15.116:0/55711533 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2834069f70 con 0x7f2834102570 2024-09-12T03:09:02.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.135+0000 7f2831ffb640 1 -- 172.21.15.116:0/55711533 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2824037020 con 0x7f2834102570 2024-09-12T03:09:02.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.135+0000 7f283d310740 1 Processor -- start 2024-09-12T03:09:02.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.135+0000 7f283d310740 1 -- start start 2024-09-12T03:09:02.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.135+0000 7f283d310740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee9c5d33a0 0x55ee9c5d3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:02.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.135+0000 7f283d310740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee9c5d3cc0 0x55ee9c5e2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:02.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.136+0000 7f283d310740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee9c5e2a90 0x55ee9c5e4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:02.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.136+0000 7f283d310740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ee9c5392e0 con 0x55ee9c5d33a0 2024-09-12T03:09:02.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.136+0000 7f283d310740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ee9c4e0ce0 con 0x55ee9c5d3cc0 2024-09-12T03:09:02.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.136+0000 7f283d310740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ee9c5352a0 con 0x55ee9c5e2a90 2024-09-12T03:09:02.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.136+0000 7f2831ffb640 1 --2- 172.21.15.116:0/55711533 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f28200a2b80 0x7f28200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:02.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.136+0000 7f28394d4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee9c5d33a0 0x55ee9c5d3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:02.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.136+0000 7f28394d4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee9c5d33a0 0x55ee9c5d3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56206/0 (socket says 172.21.15.116:56206) 2024-09-12T03:09:02.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.136+0000 7f28394d4640 1 -- 172.21.15.116:0/3144615771 learned_addr learned my addr 172.21.15.116:0/3144615771 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:02.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.136+0000 7f28394d4640 1 --2- 172.21.15.116:0/55711533 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f28200a2b80 0x7f28200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:02.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.136+0000 7f2831ffb640 1 -- 172.21.15.116:0/55711533 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f28240f1a00 con 0x7f2834102570 2024-09-12T03:09:02.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.136+0000 7f2838cd3640 1 --2- 172.21.15.116:0/3144615771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee9c5d3cc0 0x55ee9c5e2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:02.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.136+0000 7f28394d4640 1 -- 172.21.15.116:0/3144615771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee9c5e2a90 msgr2=0x55ee9c5e4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:02.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.136+0000 7f2833fff640 1 --2- 172.21.15.116:0/3144615771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee9c5e2a90 0x55ee9c5e4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:02.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.136+0000 7f28394d4640 1 --2- 172.21.15.116:0/3144615771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee9c5e2a90 0x55ee9c5e4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.136+0000 7f28394d4640 1 -- 172.21.15.116:0/3144615771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee9c5d3cc0 msgr2=0x55ee9c5e2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:02.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.136+0000 7f28394d4640 1 --2- 172.21.15.116:0/3144615771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee9c5d3cc0 0x55ee9c5e2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.136+0000 7f28394d4640 1 -- 172.21.15.116:0/3144615771 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee9c4c5440 con 0x55ee9c5d33a0 2024-09-12T03:09:02.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.136+0000 7f28394d4640 1 --2- 172.21.15.116:0/3144615771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee9c5d33a0 0x55ee9c5d3780 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f281c017980 tx=0x7f281c017e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4121c8d19d888a77 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:02.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.136+0000 7f28394d4640 1 --2- 172.21.15.116:0/55711533 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f28200a2b80 0x7f28200a5040 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f281c015790 tx=0x7f281c01b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:02.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.136+0000 7f282affd640 1 -- 172.21.15.116:0/3144615771 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c0097d0 con 0x55ee9c5d33a0 2024-09-12T03:09:02.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.136+0000 7f282affd640 1 -- 172.21.15.116:0/3144615771 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f281c025070 con 0x55ee9c5d33a0 2024-09-12T03:09:02.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.136+0000 7f282affd640 1 -- 172.21.15.116:0/3144615771 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c020aa0 con 0x55ee9c5d33a0 2024-09-12T03:09:02.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.137+0000 7f283d310740 1 -- 172.21.15.116:0/3144615771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee9c5d33a0 msgr2=0x55ee9c5d3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:02.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.137+0000 7f283d310740 1 --2- 172.21.15.116:0/3144615771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee9c5d33a0 0x55ee9c5d3780 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f281c017980 tx=0x7f281c017e40 comp rx=0 tx=0).stop 2024-09-12T03:09:02.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.137+0000 7f283d310740 1 -- 172.21.15.116:0/3144615771 shutdown_connections 2024-09-12T03:09:02.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.137+0000 7f283d310740 1 --2- 172.21.15.116:0/3144615771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee9c5e2a90 0x55ee9c5e4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.137+0000 7f283d310740 1 --2- 172.21.15.116:0/3144615771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee9c5d3cc0 0x55ee9c5e2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.137+0000 7f283d310740 1 --2- 172.21.15.116:0/3144615771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee9c5d33a0 0x55ee9c5d3780 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.137+0000 7f283d310740 1 -- 172.21.15.116:0/3144615771 >> 172.21.15.116:0/3144615771 conn(0x55ee9c5cfa30 msgr2=0x55ee9c5cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:02.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.137+0000 7f283d310740 1 -- 172.21.15.116:0/3144615771 shutdown_connections 2024-09-12T03:09:02.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.137+0000 7f283d310740 1 -- 172.21.15.116:0/3144615771 wait complete. 2024-09-12T03:09:02.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.138+0000 7f283d310740 1 -- start start 2024-09-12T03:09:02.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.138+0000 7f283d310740 1 Processor -- start 2024-09-12T03:09:02.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.138+0000 7f283d310740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee9c5d33a0 0x55ee9c5e24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:02.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.138+0000 7f283d310740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee9c5d3cc0 0x55ee9c5df770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:02.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.138+0000 7f283d310740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee9c5e2a90 0x55ee9c5dff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:02.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.138+0000 7f283d310740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ee9c5e5660 con 0x55ee9c5d3cc0 2024-09-12T03:09:02.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.138+0000 7f283d310740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ee9c4f4050 con 0x55ee9c5d33a0 2024-09-12T03:09:02.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.138+0000 7f283d310740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ee9c5e57e0 con 0x55ee9c5e2a90 2024-09-12T03:09:02.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.138+0000 7f283d310740 10 client.0 _ll_register_callbacks cb 0x55ee9c534160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:02.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.138+0000 7f2833fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee9c5d3cc0 0x55ee9c5df770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:02.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.138+0000 7f2833fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee9c5d3cc0 0x55ee9c5df770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56210/0 (socket says 172.21.15.116:56210) 2024-09-12T03:09:02.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.138+0000 7f2833fff640 1 -- 172.21.15.116:0/1526294339 learned_addr learned my addr 172.21.15.116:0/1526294339 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:02.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.138+0000 7f28394d4640 1 --2- 172.21.15.116:0/1526294339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee9c5e2a90 0x55ee9c5dff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:02.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.138+0000 7f2838cd3640 1 --2- 172.21.15.116:0/1526294339 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee9c5d33a0 0x55ee9c5e24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:02.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.138+0000 7f2833fff640 1 -- 172.21.15.116:0/1526294339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee9c5e2a90 msgr2=0x55ee9c5dff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:02.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.138+0000 7f2833fff640 1 --2- 172.21.15.116:0/1526294339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee9c5e2a90 0x55ee9c5dff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.138+0000 7f2833fff640 1 -- 172.21.15.116:0/1526294339 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee9c5d33a0 msgr2=0x55ee9c5e24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:02.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.138+0000 7f2833fff640 1 --2- 172.21.15.116:0/1526294339 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee9c5d33a0 0x55ee9c5e24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.138+0000 7f2833fff640 1 -- 172.21.15.116:0/1526294339 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee9c2aa3c0 con 0x55ee9c5d3cc0 2024-09-12T03:09:02.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.139+0000 7f2838cd3640 1 --2- 172.21.15.116:0/1526294339 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee9c5d33a0 0x55ee9c5e24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:09:02.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.139+0000 7f2833fff640 1 --2- 172.21.15.116:0/1526294339 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee9c5d3cc0 0x55ee9c5df770 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f282402fdd0 tx=0x7f28240bddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c660cfb0e5f5a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:02.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.139+0000 7f2813fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:09:02.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.139+0000 7f2813fff640 1 -- 172.21.15.116:0/1526294339 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282402c040 con 0x55ee9c5d3cc0 2024-09-12T03:09:02.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.139+0000 7f2813fff640 1 -- 172.21.15.116:0/1526294339 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f28240bc590 con 0x55ee9c5d3cc0 2024-09-12T03:09:02.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.139+0000 7f283d310740 1 -- 172.21.15.116:0/1526294339 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f281c025070 con 0x55ee9c5d3cc0 2024-09-12T03:09:02.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.139+0000 7f2813fff640 1 -- 172.21.15.116:0/1526294339 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2824003d80 con 0x55ee9c5d3cc0 2024-09-12T03:09:02.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.139+0000 7f283d310740 10 client.16806 fetch_fsmap learned FSMap version 25 2024-09-12T03:09:02.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.139+0000 7f283d310740 1 -- 172.21.15.116:0/1526294339 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ee9c5e5660 con 0x55ee9c5d3cc0 2024-09-12T03:09:02.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.139+0000 7f2813fff640 1 -- 172.21.15.116:0/1526294339 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f281c025070 con 0x55ee9c5d3cc0 2024-09-12T03:09:02.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.139+0000 7f2813fff640 1 -- 172.21.15.116:0/1526294339 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f28240be4f0 con 0x55ee9c5d3cc0 2024-09-12T03:09:02.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.139+0000 7f283d310740 10 client.16806 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:09:02.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.139+0000 7f283d310740 10 client.16806 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:09:02.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.139+0000 7f283d310740 20 client.16806 populate_metadata read hostname 'smithi116' 2024-09-12T03:09:02.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.139+0000 7f283d310740 1 --2- 172.21.15.116:0/1526294339 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ee9c5f2510 0x55ee9c5f4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:02.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.139+0000 7f283d310740 4 client.16806 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:09:02.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.139+0000 7f283d310740 1 -- 172.21.15.116:0/1526294339 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee9c546850 con 0x55ee9c5f2510 2024-09-12T03:09:02.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.140+0000 7f2838cd3640 1 --2- 172.21.15.116:0/1526294339 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ee9c5f2510 0x55ee9c5f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:02.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.140+0000 7f2838cd3640 1 --2- 172.21.15.116:0/1526294339 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ee9c5f2510 0x55ee9c5f4900 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8566634bf4a18b8 server_cookie=1cb0df0e4d4544c6 in_seq=0 out_seq=0 2024-09-12T03:09:02.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.140+0000 7f2813fff640 10 client.16806 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:02.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.141+0000 7f2813fff640 1 -- 172.21.15.116:0/1526294339 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee9c546850 con 0x55ee9c5f2510 2024-09-12T03:09:02.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.141+0000 7f2813fff640 10 client.16806 handle_command_reply: tid=0 2024-09-12T03:09:02.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.141+0000 7f283d310740 1 client.16806 shutdown 2024-09-12T03:09:02.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.141+0000 7f283d310740 1 -- 172.21.15.116:0/1526294339 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee9c5d3cc0 msgr2=0x55ee9c5df770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:02.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.141+0000 7f283d310740 1 --2- 172.21.15.116:0/1526294339 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee9c5d3cc0 0x55ee9c5df770 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f282402fdd0 tx=0x7f28240bddd0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.141+0000 7f283d310740 1 -- 172.21.15.116:0/1526294339 shutdown_connections 2024-09-12T03:09:02.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.141+0000 7f283d310740 1 --2- 172.21.15.116:0/1526294339 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ee9c5f2510 0x55ee9c5f4900 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.141+0000 7f283d310740 1 --2- 172.21.15.116:0/1526294339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee9c5e2a90 0x55ee9c5dff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.141+0000 7f283d310740 1 --2- 172.21.15.116:0/1526294339 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee9c5d3cc0 0x55ee9c5df770 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.141+0000 7f2813fff640 0 client.16806 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:02.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.141+0000 7f283d310740 1 --2- 172.21.15.116:0/1526294339 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee9c5d33a0 0x55ee9c5e24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.141+0000 7f283d310740 1 -- 172.21.15.116:0/1526294339 >> 172.21.15.116:0/1526294339 conn(0x55ee9c5cfa30 msgr2=0x55ee9c5d5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:02.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.141+0000 7f283d310740 1 -- 172.21.15.116:0/1526294339 shutdown_connections 2024-09-12T03:09:02.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.141+0000 7f283d310740 1 -- 172.21.15.116:0/1526294339 wait complete. 2024-09-12T03:09:02.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.141+0000 7f283d310740 20 client.16806 trim_cache size 0 max 16384 2024-09-12T03:09:02.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.158+0000 7f283d310740 1 Processor -- start 2024-09-12T03:09:02.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.158+0000 7f283d310740 1 -- start start 2024-09-12T03:09:02.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.158+0000 7f283d310740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee9c5d33a0 0x55ee9c5dac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:02.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.158+0000 7f283d310740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee9c5d3cc0 0x55ee9c5fa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:02.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.158+0000 7f283d310740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee9c5e2a90 0x55ee9c5fa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:02.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.158+0000 7f283d310740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f28341adb40 con 0x55ee9c5e2a90 2024-09-12T03:09:02.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.158+0000 7f283d310740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f28341ab410 con 0x55ee9c5d3cc0 2024-09-12T03:09:02.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.158+0000 7f283d310740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f28341a9280 con 0x55ee9c5d33a0 2024-09-12T03:09:02.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.159+0000 7f28394d4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee9c5e2a90 0x55ee9c5fa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:02.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.159+0000 7f28394d4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee9c5e2a90 0x55ee9c5fa6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56226/0 (socket says 172.21.15.116:56226) 2024-09-12T03:09:02.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.159+0000 7f28394d4640 1 -- 172.21.15.116:0/2686326675 learned_addr learned my addr 172.21.15.116:0/2686326675 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:02.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.159+0000 7f2833fff640 1 --2- 172.21.15.116:0/2686326675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee9c5d3cc0 0x55ee9c5fa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:02.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.159+0000 7f2838cd3640 1 --2- 172.21.15.116:0/2686326675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee9c5d33a0 0x55ee9c5dac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:02.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.159+0000 7f28394d4640 1 -- 172.21.15.116:0/2686326675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee9c5d33a0 msgr2=0x55ee9c5dac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:02.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.159+0000 7f28394d4640 1 --2- 172.21.15.116:0/2686326675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee9c5d33a0 0x55ee9c5dac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.159+0000 7f28394d4640 1 -- 172.21.15.116:0/2686326675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee9c5d3cc0 msgr2=0x55ee9c5fa180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:02.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.159+0000 7f28394d4640 1 --2- 172.21.15.116:0/2686326675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee9c5d3cc0 0x55ee9c5fa180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.159+0000 7f28394d4640 1 -- 172.21.15.116:0/2686326675 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee9c5de720 con 0x55ee9c5e2a90 2024-09-12T03:09:02.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.159+0000 7f2833fff640 1 --2- 172.21.15.116:0/2686326675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee9c5d3cc0 0x55ee9c5fa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:09:02.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.159+0000 7f28394d4640 1 --2- 172.21.15.116:0/2686326675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee9c5e2a90 0x55ee9c5fa6c0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f281c020670 tx=0x7f281c0206a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f43fe6ad4ec6ca server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:02.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.159+0000 7f282a7fc640 1 -- 172.21.15.116:0/2686326675 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c0097d0 con 0x55ee9c5e2a90 2024-09-12T03:09:02.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.159+0000 7f282a7fc640 1 -- 172.21.15.116:0/2686326675 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f281c025070 con 0x55ee9c5e2a90 2024-09-12T03:09:02.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.159+0000 7f282a7fc640 1 -- 172.21.15.116:0/2686326675 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c020770 con 0x55ee9c5e2a90 2024-09-12T03:09:02.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.160+0000 7f283d310740 1 -- 172.21.15.116:0/2686326675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee9c5e2a90 msgr2=0x55ee9c5fa6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:02.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.160+0000 7f283d310740 1 --2- 172.21.15.116:0/2686326675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee9c5e2a90 0x55ee9c5fa6c0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f281c020670 tx=0x7f281c0206a0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.160+0000 7f283d310740 1 -- 172.21.15.116:0/2686326675 shutdown_connections 2024-09-12T03:09:02.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.160+0000 7f283d310740 1 --2- 172.21.15.116:0/2686326675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee9c5e2a90 0x55ee9c5fa6c0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.160+0000 7f283d310740 1 --2- 172.21.15.116:0/2686326675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee9c5d3cc0 0x55ee9c5fa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.160+0000 7f283d310740 1 --2- 172.21.15.116:0/2686326675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee9c5d33a0 0x55ee9c5dac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.160+0000 7f283d310740 1 -- 172.21.15.116:0/2686326675 >> 172.21.15.116:0/2686326675 conn(0x55ee9c5cfa30 msgr2=0x55ee9c5da4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:02.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.160+0000 7f283d310740 1 -- 172.21.15.116:0/2686326675 shutdown_connections 2024-09-12T03:09:02.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.160+0000 7f283d310740 1 -- 172.21.15.116:0/2686326675 wait complete. 2024-09-12T03:09:02.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.160+0000 7f283d310740 1 -- start start 2024-09-12T03:09:02.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.160+0000 7f283d310740 1 Processor -- start 2024-09-12T03:09:02.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.161+0000 7f283d310740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee9c5d33a0 0x55ee9c5e67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:02.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.161+0000 7f283d310740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee9c5d3cc0 0x55ee9c5e6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:02.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.161+0000 7f283d310740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee9c5e2a90 0x55ee9c5e7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:02.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.161+0000 7f283d310740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f28341a98d0 con 0x55ee9c5d3cc0 2024-09-12T03:09:02.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.161+0000 7f283d310740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f28341aad40 con 0x55ee9c5d33a0 2024-09-12T03:09:02.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.161+0000 7f283d310740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f28341a9fa0 con 0x55ee9c5e2a90 2024-09-12T03:09:02.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.161+0000 7f283d310740 10 client.0 _ll_register_callbacks cb 0x7f28240be060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:02.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.161+0000 7f2833fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee9c5d3cc0 0x55ee9c5e6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:02.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.161+0000 7f2833fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee9c5d3cc0 0x55ee9c5e6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56234/0 (socket says 172.21.15.116:56234) 2024-09-12T03:09:02.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.161+0000 7f2833fff640 1 -- 172.21.15.116:0/2686191471 learned_addr learned my addr 172.21.15.116:0/2686191471 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:02.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.161+0000 7f28394d4640 1 --2- 172.21.15.116:0/2686191471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee9c5e2a90 0x55ee9c5e7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:02.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.161+0000 7f2838cd3640 1 --2- 172.21.15.116:0/2686191471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee9c5d33a0 0x55ee9c5e67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:02.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.161+0000 7f2833fff640 1 -- 172.21.15.116:0/2686191471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee9c5e2a90 msgr2=0x55ee9c5e7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:02.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.161+0000 7f2833fff640 1 --2- 172.21.15.116:0/2686191471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee9c5e2a90 0x55ee9c5e7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.161+0000 7f2833fff640 1 -- 172.21.15.116:0/2686191471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee9c5d33a0 msgr2=0x55ee9c5e67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:02.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.161+0000 7f2833fff640 1 --2- 172.21.15.116:0/2686191471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee9c5d33a0 0x55ee9c5e67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.161+0000 7f2833fff640 1 -- 172.21.15.116:0/2686191471 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee9c5e0270 con 0x55ee9c5d3cc0 2024-09-12T03:09:02.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.161+0000 7f2833fff640 1 --2- 172.21.15.116:0/2686191471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee9c5d3cc0 0x55ee9c5e6cf0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f28240bac10 tx=0x7f28240093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b94433ff746e718 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:02.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.161+0000 7f2813fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:09:02.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.162+0000 7f2813fff640 1 -- 172.21.15.116:0/2686191471 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28240c2640 con 0x55ee9c5d3cc0 2024-09-12T03:09:02.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.162+0000 7f2813fff640 1 -- 172.21.15.116:0/2686191471 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f28240de2c0 con 0x55ee9c5d3cc0 2024-09-12T03:09:02.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.162+0000 7f283d310740 1 -- 172.21.15.116:0/2686191471 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f281c025070 con 0x55ee9c5d3cc0 2024-09-12T03:09:02.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.162+0000 7f2813fff640 1 -- 172.21.15.116:0/2686191471 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f28240c6f00 con 0x55ee9c5d3cc0 2024-09-12T03:09:02.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.162+0000 7f283d310740 10 client.16812 fetch_fsmap learned FSMap version 25 2024-09-12T03:09:02.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.162+0000 7f283d310740 1 -- 172.21.15.116:0/2686191471 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ee9c65d250 con 0x55ee9c5d3cc0 2024-09-12T03:09:02.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.162+0000 7f2813fff640 1 -- 172.21.15.116:0/2686191471 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f281c025070 con 0x55ee9c5d3cc0 2024-09-12T03:09:02.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.162+0000 7f2813fff640 1 -- 172.21.15.116:0/2686191471 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f28240bd800 con 0x55ee9c5d3cc0 2024-09-12T03:09:02.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.162+0000 7f283d310740 10 client.16812 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:09:02.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.162+0000 7f283d310740 10 client.16812 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:09:02.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.162+0000 7f283d310740 20 client.16812 populate_metadata read hostname 'smithi116' 2024-09-12T03:09:02.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.162+0000 7f283d310740 1 --2- 172.21.15.116:0/2686191471 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ee9c65d830 0x55ee9c65fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:02.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.162+0000 7f283d310740 4 client.16812 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:09:02.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.162+0000 7f283d310740 1 -- 172.21.15.116:0/2686191471 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ee9c540d60 con 0x55ee9c65d830 2024-09-12T03:09:02.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.162+0000 7f2838cd3640 1 --2- 172.21.15.116:0/2686191471 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ee9c65d830 0x55ee9c65fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:02.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.163+0000 7f2813fff640 10 client.16812 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:02.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.163+0000 7f2838cd3640 1 --2- 172.21.15.116:0/2686191471 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ee9c65d830 0x55ee9c65fc50 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4d8db789f01d735 server_cookie=fe6d19eddfd21fe9 in_seq=0 out_seq=0 2024-09-12T03:09:02.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.163+0000 7f2813fff640 1 -- 172.21.15.116:0/2686191471 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ee9c540d60 con 0x55ee9c65d830 2024-09-12T03:09:02.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.163+0000 7f2813fff640 10 client.16812 handle_command_reply: tid=0 2024-09-12T03:09:02.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.163+0000 7f283d310740 1 client.16812 shutdown 2024-09-12T03:09:02.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.163+0000 7f283d310740 1 -- 172.21.15.116:0/2686191471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee9c5d3cc0 msgr2=0x55ee9c5e6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:02.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.163+0000 7f283d310740 1 --2- 172.21.15.116:0/2686191471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee9c5d3cc0 0x55ee9c5e6cf0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f28240bac10 tx=0x7f28240093f0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.163+0000 7f283d310740 1 -- 172.21.15.116:0/2686191471 shutdown_connections 2024-09-12T03:09:02.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.163+0000 7f283d310740 1 --2- 172.21.15.116:0/2686191471 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ee9c65d830 0x55ee9c65fc50 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.163+0000 7f283d310740 1 --2- 172.21.15.116:0/2686191471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee9c5e2a90 0x55ee9c5e7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.163+0000 7f283d310740 1 --2- 172.21.15.116:0/2686191471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee9c5d3cc0 0x55ee9c5e6cf0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.163+0000 7f2813fff640 0 client.16812 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:02.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.163+0000 7f283d310740 1 --2- 172.21.15.116:0/2686191471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee9c5d33a0 0x55ee9c5e67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.163+0000 7f283d310740 1 -- 172.21.15.116:0/2686191471 >> 172.21.15.116:0/2686191471 conn(0x55ee9c5cfa30 msgr2=0x55ee9c5dad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:02.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.163+0000 7f283d310740 1 -- 172.21.15.116:0/2686191471 shutdown_connections 2024-09-12T03:09:02.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.163+0000 7f283d310740 1 -- 172.21.15.116:0/2686191471 wait complete. 2024-09-12T03:09:02.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.164+0000 7f283d310740 20 client.16812 trim_cache size 0 max 16384 2024-09-12T03:09:02.180 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:09:02.180 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:09:02.181 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:09:02.181 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:09:02.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.164+0000 7f283af37640 1 -- 172.21.15.116:0/55711533 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f28200a2b80 msgr2=0x7f28200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:02.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.164+0000 7f283af37640 1 --2- 172.21.15.116:0/55711533 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f28200a2b80 0x7f28200a5040 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f281c015790 tx=0x7f281c01b000 comp rx=0 tx=0).stop 2024-09-12T03:09:02.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.164+0000 7f283af37640 1 -- 172.21.15.116:0/55711533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2834102570 msgr2=0x7f2834197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:02.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.164+0000 7f283af37640 1 --2- 172.21.15.116:0/55711533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2834102570 0x7f2834197a80 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f2824005cc0 tx=0x7f282402f6c0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.164+0000 7f283af37640 1 -- 172.21.15.116:0/55711533 shutdown_connections 2024-09-12T03:09:02.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.164+0000 7f283af37640 1 --2- 172.21.15.116:0/55711533 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f28200a2b80 0x7f28200a5040 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.164+0000 7f283af37640 1 --2- 172.21.15.116:0/55711533 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2834102ef0 0x7f283419be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.164+0000 7f283af37640 1 --2- 172.21.15.116:0/55711533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2834102570 0x7f2834197a80 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.164+0000 7f283af37640 1 --2- 172.21.15.116:0/55711533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2834100e20 0x7f2834197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:02.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.164+0000 7f283af37640 1 -- 172.21.15.116:0/55711533 >> 172.21.15.116:0/55711533 conn(0x7f28340fc7d0 msgr2=0x7f28340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:02.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.164+0000 7f283af37640 1 -- 172.21.15.116:0/55711533 shutdown_connections 2024-09-12T03:09:02.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:02.164+0000 7f283af37640 1 -- 172.21.15.116:0/55711533 wait complete. 2024-09-12T03:09:02.444 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:02 smithi116 ceph-mon[31410]: pgmap v519: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-12T03:09:02.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:02 smithi183 ceph-mon[35535]: pgmap v519: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-12T03:09:02.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:02 smithi177 ceph-mon[35305]: pgmap v519: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-12T03:09:03.078 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:09:03.078 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:09:03.336 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:09:03.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.833+0000 7f1fdd6cf640 1 -- 172.21.15.116:0/827759822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fd81036e0 msgr2=0x7f1fd8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:03.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.833+0000 7f1fdd6cf640 1 --2- 172.21.15.116:0/827759822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fd81036e0 0x7f1fd8107b90 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc009930 tx=0x7f1fcc02a420 comp rx=0 tx=0).stop 2024-09-12T03:09:03.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.833+0000 7f1fdd6cf640 1 -- 172.21.15.116:0/827759822 shutdown_connections 2024-09-12T03:09:03.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.833+0000 7f1fdd6cf640 1 --2- 172.21.15.116:0/827759822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fd81082a0 0x7f1fd810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:03.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.833+0000 7f1fdd6cf640 1 --2- 172.21.15.116:0/827759822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fd81036e0 0x7f1fd8107b90 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:03.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.833+0000 7f1fdd6cf640 1 --2- 172.21.15.116:0/827759822 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fd8102d30 0x7f1fd8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:03.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.833+0000 7f1fdd6cf640 1 -- 172.21.15.116:0/827759822 >> 172.21.15.116:0/827759822 conn(0x7f1fd80fc7d0 msgr2=0x7f1fd80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:03.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.833+0000 7f1fdd6cf640 1 -- 172.21.15.116:0/827759822 shutdown_connections 2024-09-12T03:09:03.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.833+0000 7f1fdd6cf640 1 -- 172.21.15.116:0/827759822 wait complete. 2024-09-12T03:09:03.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.833+0000 7f1fdd6cf640 1 Processor -- start 2024-09-12T03:09:03.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.834+0000 7f1fdd6cf640 1 -- start start 2024-09-12T03:09:03.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.834+0000 7f1fdd6cf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fd8102d30 0x7f1fd81a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:03.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.834+0000 7f1fdd6cf640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fd81036e0 0x7f1fd81a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:03.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.834+0000 7f1fdd6cf640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fd81082a0 0x7f1fd819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:03.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.834+0000 7f1fdd6cf640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd81120c0 con 0x7f1fd81036e0 2024-09-12T03:09:03.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.834+0000 7f1fdd6cf640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd8111f40 con 0x7f1fd81082a0 2024-09-12T03:09:03.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.834+0000 7f1fdd6cf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd8112240 con 0x7f1fd8102d30 2024-09-12T03:09:03.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.834+0000 7f1fd67fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fd81036e0 0x7f1fd81a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:03.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.834+0000 7f1fd67fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fd81036e0 0x7f1fd81a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56250/0 (socket says 172.21.15.116:56250) 2024-09-12T03:09:03.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.834+0000 7f1fd6ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fd8102d30 0x7f1fd81a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:03.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.834+0000 7f1fd67fc640 1 -- 172.21.15.116:0/3679017670 learned_addr learned my addr 172.21.15.116:0/3679017670 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:03.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.834+0000 7f1fd77fe640 1 --2- 172.21.15.116:0/3679017670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fd81082a0 0x7f1fd819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:03.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.834+0000 7f1fd67fc640 1 -- 172.21.15.116:0/3679017670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fd8102d30 msgr2=0x7f1fd81a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:03.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.834+0000 7f1fd67fc640 1 --2- 172.21.15.116:0/3679017670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fd8102d30 0x7f1fd81a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:03.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.834+0000 7f1fd67fc640 1 -- 172.21.15.116:0/3679017670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fd81082a0 msgr2=0x7f1fd819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:03.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.834+0000 7f1fd67fc640 1 --2- 172.21.15.116:0/3679017670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fd81082a0 0x7f1fd819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:03.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.834+0000 7f1fd67fc640 1 -- 172.21.15.116:0/3679017670 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd819aee0 con 0x7f1fd81036e0 2024-09-12T03:09:03.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.835+0000 7f1fd67fc640 1 --2- 172.21.15.116:0/3679017670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fd81036e0 0x7f1fd81a0ae0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc005940 tx=0x7f1fcc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:03.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.835+0000 7f1fbffff640 1 -- 172.21.15.116:0/3679017670 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc03f030 con 0x7f1fd81036e0 2024-09-12T03:09:03.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.835+0000 7f1fdd6cf640 1 -- 172.21.15.116:0/3679017670 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fd819b170 con 0x7f1fd81036e0 2024-09-12T03:09:03.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.835+0000 7f1fbffff640 1 -- 172.21.15.116:0/3679017670 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fcc003ce0 con 0x7f1fd81036e0 2024-09-12T03:09:03.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.835+0000 7f1fbffff640 1 -- 172.21.15.116:0/3679017670 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc035070 con 0x7f1fd81036e0 2024-09-12T03:09:03.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.835+0000 7f1fdd6cf640 1 -- 172.21.15.116:0/3679017670 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fd81a7350 con 0x7f1fd81036e0 2024-09-12T03:09:03.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.836+0000 7f1fdd6cf640 1 -- 172.21.15.116:0/3679017670 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fd806a850 con 0x7f1fd81036e0 2024-09-12T03:09:03.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.837+0000 7f1fbffff640 1 -- 172.21.15.116:0/3679017670 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1fcc042030 con 0x7f1fd81036e0 2024-09-12T03:09:03.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.838+0000 7f1fbffff640 1 --2- 172.21.15.116:0/3679017670 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1fc00a2b80 0x7f1fc00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:03.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.838+0000 7f1fd6ffd640 1 --2- 172.21.15.116:0/3679017670 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1fc00a2b80 0x7f1fc00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:03.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.838+0000 7f1fbffff640 1 -- 172.21.15.116:0/3679017670 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1fcc0ef520 con 0x7f1fd81036e0 2024-09-12T03:09:03.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.838+0000 7f1fd6ffd640 1 --2- 172.21.15.116:0/3679017670 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1fc00a2b80 0x7f1fc00a5040 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8007ec0 tx=0x7f1fc80075b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:03.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:03.842+0000 7f1fbffff640 1 -- 172.21.15.116:0/3679017670 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1fcc0bbe60 con 0x7f1fd81036e0 2024-09-12T03:09:04.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:04.043+0000 7f1fdd6cf640 1 -- 172.21.15.116:0/3679017670 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1fd819bba0 con 0x7f1fd81036e0 2024-09-12T03:09:04.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:04.044+0000 7f1fbffff640 1 -- 172.21.15.116:0/3679017670 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f1fcc0c0160 con 0x7f1fd81036e0 2024-09-12T03:09:04.044 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:09:04.045 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:09:04.045 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:09:04.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:04.046+0000 7f1fdd6cf640 1 -- 172.21.15.116:0/3679017670 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1fc00a2b80 msgr2=0x7f1fc00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:04.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:04.046+0000 7f1fdd6cf640 1 --2- 172.21.15.116:0/3679017670 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1fc00a2b80 0x7f1fc00a5040 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8007ec0 tx=0x7f1fc80075b0 comp rx=0 tx=0).stop 2024-09-12T03:09:04.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:04.046+0000 7f1fdd6cf640 1 -- 172.21.15.116:0/3679017670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fd81036e0 msgr2=0x7f1fd81a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:04.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:04.046+0000 7f1fdd6cf640 1 --2- 172.21.15.116:0/3679017670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fd81036e0 0x7f1fd81a0ae0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc005940 tx=0x7f1fcc0042f0 comp rx=0 tx=0).stop 2024-09-12T03:09:04.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:04.046+0000 7f1fdd6cf640 1 -- 172.21.15.116:0/3679017670 shutdown_connections 2024-09-12T03:09:04.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:04.046+0000 7f1fdd6cf640 1 --2- 172.21.15.116:0/3679017670 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1fc00a2b80 0x7f1fc00a5040 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:04.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:04.046+0000 7f1fdd6cf640 1 --2- 172.21.15.116:0/3679017670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fd81082a0 0x7f1fd819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:04.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:04.046+0000 7f1fdd6cf640 1 --2- 172.21.15.116:0/3679017670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fd81036e0 0x7f1fd81a0ae0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:04.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:04.046+0000 7f1fdd6cf640 1 --2- 172.21.15.116:0/3679017670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fd8102d30 0x7f1fd81a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:04.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:04.046+0000 7f1fdd6cf640 1 -- 172.21.15.116:0/3679017670 >> 172.21.15.116:0/3679017670 conn(0x7f1fd80fc7d0 msgr2=0x7f1fd80fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:04.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:04.046+0000 7f1fdd6cf640 1 -- 172.21.15.116:0/3679017670 shutdown_connections 2024-09-12T03:09:04.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:04.046+0000 7f1fdd6cf640 1 -- 172.21.15.116:0/3679017670 wait complete. 2024-09-12T03:09:04.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:04 smithi183 ceph-mon[35535]: pgmap v520: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.8 KiB/s wr, 8 op/s 2024-09-12T03:09:04.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:04 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3679017670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:09:04.732 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:04 smithi116 ceph-mon[31410]: pgmap v520: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.8 KiB/s wr, 8 op/s 2024-09-12T03:09:04.733 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:04 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3679017670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:09:04.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:04 smithi177 ceph-mon[35305]: pgmap v520: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.8 KiB/s wr, 8 op/s 2024-09-12T03:09:04.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:04 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3679017670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:09:04.761 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:09:05.008 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:09:05.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.493+0000 7efe73016640 1 -- 172.21.15.116:0/691047434 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe6c106f10 msgr2=0x7efe6c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:05.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.493+0000 7efe73016640 1 --2- 172.21.15.116:0/691047434 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe6c106f10 0x7efe6c10ea90 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7efe64009ca0 tx=0x7efe6402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.493+0000 7efe73016640 1 -- 172.21.15.116:0/691047434 shutdown_connections 2024-09-12T03:09:05.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.493+0000 7efe73016640 1 --2- 172.21.15.116:0/691047434 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe6c106f10 0x7efe6c10ea90 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.493+0000 7efe73016640 1 --2- 172.21.15.116:0/691047434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe6c102790 0x7efe6c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.493+0000 7efe73016640 1 --2- 172.21.15.116:0/691047434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe6c101de0 0x7efe6c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.493+0000 7efe73016640 1 -- 172.21.15.116:0/691047434 >> 172.21.15.116:0/691047434 conn(0x7efe6c0fd8c0 msgr2=0x7efe6c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:05.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.493+0000 7efe73016640 1 -- 172.21.15.116:0/691047434 shutdown_connections 2024-09-12T03:09:05.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.493+0000 7efe73016640 1 -- 172.21.15.116:0/691047434 wait complete. 2024-09-12T03:09:05.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.494+0000 7efe73016640 1 Processor -- start 2024-09-12T03:09:05.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.494+0000 7efe73016640 1 -- start start 2024-09-12T03:09:05.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.494+0000 7efe73016640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe6c101de0 0x7efe6c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:05.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.494+0000 7efe73016640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe6c102790 0x7efe6c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:05.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.494+0000 7efe73016640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe6c106f10 0x7efe6c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:05.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.494+0000 7efe73016640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c111f70 con 0x7efe6c102790 2024-09-12T03:09:05.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.494+0000 7efe73016640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c111df0 con 0x7efe6c106f10 2024-09-12T03:09:05.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.494+0000 7efe73016640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c1120f0 con 0x7efe6c101de0 2024-09-12T03:09:05.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.494+0000 7efe6bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe6c102790 0x7efe6c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:05.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.494+0000 7efe70db2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe6c101de0 0x7efe6c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:05.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.494+0000 7efe6bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe6c102790 0x7efe6c198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56274/0 (socket says 172.21.15.116:56274) 2024-09-12T03:09:05.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.494+0000 7efe6bfff640 1 -- 172.21.15.116:0/518819922 learned_addr learned my addr 172.21.15.116:0/518819922 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:05.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.494+0000 7efe715b3640 1 --2- 172.21.15.116:0/518819922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe6c106f10 0x7efe6c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:05.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.494+0000 7efe6bfff640 1 -- 172.21.15.116:0/518819922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe6c101de0 msgr2=0x7efe6c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:05.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.494+0000 7efe6bfff640 1 --2- 172.21.15.116:0/518819922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe6c101de0 0x7efe6c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.494+0000 7efe6bfff640 1 -- 172.21.15.116:0/518819922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe6c106f10 msgr2=0x7efe6c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:05.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.494+0000 7efe6bfff640 1 --2- 172.21.15.116:0/518819922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe6c106f10 0x7efe6c19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.494+0000 7efe6bfff640 1 -- 172.21.15.116:0/518819922 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6c19d380 con 0x7efe6c102790 2024-09-12T03:09:05.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.495+0000 7efe6bfff640 1 --2- 172.21.15.116:0/518819922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe6c102790 0x7efe6c198910 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7efe5800d950 tx=0x7efe5800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:05.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.495+0000 7efe69ffb640 1 -- 172.21.15.116:0/518819922 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe58017070 con 0x7efe6c102790 2024-09-12T03:09:05.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.495+0000 7efe69ffb640 1 -- 172.21.15.116:0/518819922 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe58009c30 con 0x7efe6c102790 2024-09-12T03:09:05.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.495+0000 7efe73016640 1 -- 172.21.15.116:0/518819922 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe6c19d670 con 0x7efe6c102790 2024-09-12T03:09:05.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.495+0000 7efe69ffb640 1 -- 172.21.15.116:0/518819922 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5800f6b0 con 0x7efe6c102790 2024-09-12T03:09:05.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.495+0000 7efe73016640 1 -- 172.21.15.116:0/518819922 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe6c1a4f50 con 0x7efe6c102790 2024-09-12T03:09:05.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.497+0000 7efe69ffb640 1 -- 172.21.15.116:0/518819922 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7efe580109d0 con 0x7efe6c102790 2024-09-12T03:09:05.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.497+0000 7efe753ef740 1 Processor -- start 2024-09-12T03:09:05.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.497+0000 7efe753ef740 1 -- start start 2024-09-12T03:09:05.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.498+0000 7efe69ffb640 1 --2- 172.21.15.116:0/518819922 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7efe5c0a2b80 0x7efe5c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:05.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.498+0000 7efe753ef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b68ccdb40 0x564b68ccc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:05.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.498+0000 7efe753ef740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b68ccc5d0 0x564b68ccca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:05.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.498+0000 7efe753ef740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564b68cd7370 0x564b68cd9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:05.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.498+0000 7efe753ef740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x564b68c322f0 con 0x564b68cd7370 2024-09-12T03:09:05.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.498+0000 7efe753ef740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x564b68bd9cf0 con 0x564b68ccc5d0 2024-09-12T03:09:05.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.498+0000 7efe753ef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564b68c2e2b0 con 0x564b68ccdb40 2024-09-12T03:09:05.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.498+0000 7efe715b3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564b68cd7370 0x564b68cd9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:05.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.498+0000 7efe715b3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564b68cd7370 0x564b68cd9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56276/0 (socket says 172.21.15.116:56276) 2024-09-12T03:09:05.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.498+0000 7efe70db2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b68ccdb40 0x564b68ccc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:05.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.498+0000 7efe715b3640 1 -- 172.21.15.116:0/936159496 learned_addr learned my addr 172.21.15.116:0/936159496 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:05.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.498+0000 7efe69ffb640 1 -- 172.21.15.116:0/518819922 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7efe580ceaf0 con 0x7efe6c102790 2024-09-12T03:09:05.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.498+0000 7efe715b3640 1 --2- 172.21.15.116:0/518819922 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7efe5c0a2b80 0x7efe5c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:05.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.498+0000 7efe6bfff640 1 --2- 172.21.15.116:0/936159496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b68ccc5d0 0x564b68ccca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:05.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.498+0000 7efe70db2640 1 -- 172.21.15.116:0/936159496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b68ccc5d0 msgr2=0x564b68ccca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:05.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.498+0000 7efe70db2640 1 --2- 172.21.15.116:0/936159496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b68ccc5d0 0x564b68ccca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.498+0000 7efe70db2640 1 -- 172.21.15.116:0/936159496 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564b68cd7370 msgr2=0x564b68cd9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:05.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.498+0000 7efe70db2640 1 --2- 172.21.15.116:0/936159496 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564b68cd7370 0x564b68cd9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.498+0000 7efe70db2640 1 -- 172.21.15.116:0/936159496 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b68bbe3a0 con 0x564b68ccdb40 2024-09-12T03:09:05.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.499+0000 7efe715b3640 1 --2- 172.21.15.116:0/518819922 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7efe5c0a2b80 0x7efe5c0a5040 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x564b68ccd070 tx=0x7efe64004a80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:05.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.499+0000 7efe70db2640 1 --2- 172.21.15.116:0/936159496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b68ccdb40 0x564b68ccc090 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efe60009c00 tx=0x7efe60011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdbda33f635e8544 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:05.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.499+0000 7efe56ffd640 1 -- 172.21.15.116:0/936159496 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe60004230 con 0x564b68ccdb40 2024-09-12T03:09:05.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.499+0000 7efe56ffd640 1 -- 172.21.15.116:0/936159496 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe600043d0 con 0x564b68ccdb40 2024-09-12T03:09:05.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.499+0000 7efe56ffd640 1 -- 172.21.15.116:0/936159496 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe60004d90 con 0x564b68ccdb40 2024-09-12T03:09:05.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.500+0000 7efe753ef740 1 -- 172.21.15.116:0/936159496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b68ccdb40 msgr2=0x564b68ccc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:05.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.500+0000 7efe753ef740 1 --2- 172.21.15.116:0/936159496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b68ccdb40 0x564b68ccc090 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efe60009c00 tx=0x7efe60011810 comp rx=0 tx=0).stop 2024-09-12T03:09:05.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.500+0000 7efe753ef740 1 -- 172.21.15.116:0/936159496 shutdown_connections 2024-09-12T03:09:05.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.500+0000 7efe753ef740 1 --2- 172.21.15.116:0/936159496 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564b68cd7370 0x564b68cd9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.500+0000 7efe753ef740 1 --2- 172.21.15.116:0/936159496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b68ccc5d0 0x564b68ccca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.500+0000 7efe753ef740 1 --2- 172.21.15.116:0/936159496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b68ccdb40 0x564b68ccc090 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.500+0000 7efe753ef740 1 -- 172.21.15.116:0/936159496 >> 172.21.15.116:0/936159496 conn(0x564b68c95310 msgr2=0x564b68cc8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:05.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.500+0000 7efe753ef740 1 -- 172.21.15.116:0/936159496 shutdown_connections 2024-09-12T03:09:05.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.500+0000 7efe753ef740 1 -- 172.21.15.116:0/936159496 wait complete. 2024-09-12T03:09:05.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.500+0000 7efe753ef740 1 -- start start 2024-09-12T03:09:05.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.500+0000 7efe753ef740 1 Processor -- start 2024-09-12T03:09:05.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.500+0000 7efe753ef740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564b68ccc5d0 0x564b68cd6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:05.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.500+0000 7efe753ef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b68ccdb40 0x564b68cd3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:05.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.501+0000 7efe753ef740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b68cd7370 0x564b68cd44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:05.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.501+0000 7efe753ef740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x564b68cda070 con 0x564b68ccc5d0 2024-09-12T03:09:05.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.501+0000 7efe753ef740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x564b68bed060 con 0x564b68cd7370 2024-09-12T03:09:05.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.501+0000 7efe753ef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564b68cda1f0 con 0x564b68ccdb40 2024-09-12T03:09:05.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.501+0000 7efe753ef740 10 client.0 _ll_register_callbacks cb 0x564b689408c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:05.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.501+0000 7efe70db2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564b68ccc5d0 0x564b68cd6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:05.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.501+0000 7efe70db2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564b68ccc5d0 0x564b68cd6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56284/0 (socket says 172.21.15.116:56284) 2024-09-12T03:09:05.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.501+0000 7efe70db2640 1 -- 172.21.15.116:0/48415910 learned_addr learned my addr 172.21.15.116:0/48415910 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:05.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.501+0000 7efe715b3640 1 --2- 172.21.15.116:0/48415910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b68cd7370 0x564b68cd44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:05.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.501+0000 7efe6bfff640 1 --2- 172.21.15.116:0/48415910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b68ccdb40 0x564b68cd3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:05.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.501+0000 7efe70db2640 1 -- 172.21.15.116:0/48415910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b68ccdb40 msgr2=0x564b68cd3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:05.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.501+0000 7efe70db2640 1 --2- 172.21.15.116:0/48415910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b68ccdb40 0x564b68cd3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.501+0000 7efe70db2640 1 -- 172.21.15.116:0/48415910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b68cd7370 msgr2=0x564b68cd44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:05.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.501+0000 7efe70db2640 1 --2- 172.21.15.116:0/48415910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b68cd7370 0x564b68cd44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.501+0000 7efe70db2640 1 -- 172.21.15.116:0/48415910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b689a33c0 con 0x564b68ccc5d0 2024-09-12T03:09:05.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.501+0000 7efe70db2640 1 --2- 172.21.15.116:0/48415910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564b68ccc5d0 0x564b68cd6cd0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7efe60019ad0 tx=0x7efe60019b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=105d62360440580c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:05.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.501+0000 7efe4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:09:05.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.501+0000 7efe4bfff640 1 -- 172.21.15.116:0/48415910 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe60028020 con 0x564b68ccc5d0 2024-09-12T03:09:05.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.501+0000 7efe4bfff640 1 -- 172.21.15.116:0/48415910 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7efe60024620 con 0x564b68ccc5d0 2024-09-12T03:09:05.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.501+0000 7efe753ef740 1 -- 172.21.15.116:0/48415910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe600043d0 con 0x564b68ccc5d0 2024-09-12T03:09:05.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.501+0000 7efe4bfff640 1 -- 172.21.15.116:0/48415910 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efe60009e70 con 0x564b68ccc5d0 2024-09-12T03:09:05.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.502+0000 7efe753ef740 10 client.16830 fetch_fsmap learned FSMap version 25 2024-09-12T03:09:05.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.502+0000 7efe753ef740 1 -- 172.21.15.116:0/48415910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564b68ce5a80 con 0x564b68ccc5d0 2024-09-12T03:09:05.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.502+0000 7efe4bfff640 1 -- 172.21.15.116:0/48415910 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe600043d0 con 0x564b68ccc5d0 2024-09-12T03:09:05.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.502+0000 7efe4bfff640 1 -- 172.21.15.116:0/48415910 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7efe60018070 con 0x564b68ccc5d0 2024-09-12T03:09:05.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.502+0000 7efe753ef740 10 client.16830 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:09:05.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.502+0000 7efe753ef740 10 client.16830 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:09:05.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.502+0000 7efe753ef740 20 client.16830 populate_metadata read hostname 'smithi116' 2024-09-12T03:09:05.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.502+0000 7efe753ef740 1 --2- 172.21.15.116:0/48415910 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x564b68ce6dd0 0x564b68ce91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:05.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.502+0000 7efe753ef740 4 client.16830 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:09:05.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.502+0000 7efe753ef740 1 -- 172.21.15.116:0/48415910 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b68c3f860 con 0x564b68ce6dd0 2024-09-12T03:09:05.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.502+0000 7efe6bfff640 1 --2- 172.21.15.116:0/48415910 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x564b68ce6dd0 0x564b68ce91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:05.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.503+0000 7efe6bfff640 1 --2- 172.21.15.116:0/48415910 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x564b68ce6dd0 0x564b68ce91c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=321dfcf2551bec4e server_cookie=8111e7a978bc8d6 in_seq=0 out_seq=0 2024-09-12T03:09:05.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.503+0000 7efe4bfff640 10 client.16830 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:09:05.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.503+0000 7efe4bfff640 1 -- 172.21.15.116:0/48415910 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564b68c3f860 con 0x564b68ce6dd0 2024-09-12T03:09:05.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.503+0000 7efe4bfff640 10 client.16830 handle_command_reply: tid=0 2024-09-12T03:09:05.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.503+0000 7efe753ef740 1 client.16830 shutdown 2024-09-12T03:09:05.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.503+0000 7efe753ef740 1 -- 172.21.15.116:0/48415910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564b68ccc5d0 msgr2=0x564b68cd6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:05.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.503+0000 7efe753ef740 1 --2- 172.21.15.116:0/48415910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564b68ccc5d0 0x564b68cd6cd0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7efe60019ad0 tx=0x7efe60019b00 comp rx=0 tx=0).stop 2024-09-12T03:09:05.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.503+0000 7efe753ef740 1 -- 172.21.15.116:0/48415910 shutdown_connections 2024-09-12T03:09:05.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.503+0000 7efe753ef740 1 --2- 172.21.15.116:0/48415910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b68cd7370 0x564b68cd44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.503+0000 7efe753ef740 1 --2- 172.21.15.116:0/48415910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b68ccdb40 0x564b68cd3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.503+0000 7efe753ef740 1 --2- 172.21.15.116:0/48415910 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x564b68ce6dd0 0x564b68ce91c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.503+0000 7efe753ef740 1 --2- 172.21.15.116:0/48415910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564b68ccc5d0 0x564b68cd6cd0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.503+0000 7efe4bfff640 0 client.16830 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:09:05.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.503+0000 7efe753ef740 1 -- 172.21.15.116:0/48415910 >> 172.21.15.116:0/48415910 conn(0x564b68c95310 msgr2=0x564b68cce960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:05.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.504+0000 7efe753ef740 1 -- 172.21.15.116:0/48415910 shutdown_connections 2024-09-12T03:09:05.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.504+0000 7efe753ef740 1 -- 172.21.15.116:0/48415910 wait complete. 2024-09-12T03:09:05.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.504+0000 7efe753ef740 20 client.16830 trim_cache size 0 max 16384 2024-09-12T03:09:05.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.516+0000 7efe753ef740 1 Processor -- start 2024-09-12T03:09:05.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.517+0000 7efe753ef740 1 -- start start 2024-09-12T03:09:05.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.517+0000 7efe753ef740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b68ccc5d0 0x564b68ccf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:05.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.517+0000 7efe753ef740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564b68ccdb40 0x564b68ccf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:05.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.517+0000 7efe753ef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b68cd7370 0x564b68cee970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:05.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.517+0000 7efe753ef740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c1afd20 con 0x564b68ccdb40 2024-09-12T03:09:05.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.517+0000 7efe753ef740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c1ad5f0 con 0x564b68ccc5d0 2024-09-12T03:09:05.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.517+0000 7efe753ef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c1ab440 con 0x564b68cd7370 2024-09-12T03:09:05.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.517+0000 7efe6bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564b68ccdb40 0x564b68ccf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:05.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.517+0000 7efe6bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564b68ccdb40 0x564b68ccf660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56290/0 (socket says 172.21.15.116:56290) 2024-09-12T03:09:05.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.517+0000 7efe6bfff640 1 -- 172.21.15.116:0/970762572 learned_addr learned my addr 172.21.15.116:0/970762572 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:05.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.517+0000 7efe70db2640 1 --2- 172.21.15.116:0/970762572 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b68ccc5d0 0x564b68ccf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:05.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.517+0000 7efe715b3640 1 --2- 172.21.15.116:0/970762572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b68cd7370 0x564b68cee970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:05.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.517+0000 7efe6bfff640 1 -- 172.21.15.116:0/970762572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b68cd7370 msgr2=0x564b68cee970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:05.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.517+0000 7efe6bfff640 1 --2- 172.21.15.116:0/970762572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b68cd7370 0x564b68cee970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.517+0000 7efe6bfff640 1 -- 172.21.15.116:0/970762572 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b68ccc5d0 msgr2=0x564b68ccf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:05.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.517+0000 7efe6bfff640 1 --2- 172.21.15.116:0/970762572 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b68ccc5d0 0x564b68ccf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.517+0000 7efe6bfff640 1 -- 172.21.15.116:0/970762572 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b68cc9960 con 0x564b68ccdb40 2024-09-12T03:09:05.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.517+0000 7efe715b3640 1 --2- 172.21.15.116:0/970762572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b68cd7370 0x564b68cee970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:09:05.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.517+0000 7efe6bfff640 1 --2- 172.21.15.116:0/970762572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564b68ccdb40 0x564b68ccf660 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7efe58009610 tx=0x7efe58010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b83379ff966484e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:05.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.518+0000 7efe567fc640 1 -- 172.21.15.116:0/970762572 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe58020070 con 0x564b68ccdb40 2024-09-12T03:09:05.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.518+0000 7efe567fc640 1 -- 172.21.15.116:0/970762572 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe580a3ff0 con 0x564b68ccdb40 2024-09-12T03:09:05.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.518+0000 7efe567fc640 1 -- 172.21.15.116:0/970762572 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe580af110 con 0x564b68ccdb40 2024-09-12T03:09:05.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.518+0000 7efe753ef740 1 -- 172.21.15.116:0/970762572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564b68ccdb40 msgr2=0x564b68ccf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:05.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.518+0000 7efe753ef740 1 --2- 172.21.15.116:0/970762572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564b68ccdb40 0x564b68ccf660 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7efe58009610 tx=0x7efe58010d70 comp rx=0 tx=0).stop 2024-09-12T03:09:05.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.518+0000 7efe753ef740 1 -- 172.21.15.116:0/970762572 shutdown_connections 2024-09-12T03:09:05.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.518+0000 7efe753ef740 1 --2- 172.21.15.116:0/970762572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b68cd7370 0x564b68cee970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.518+0000 7efe753ef740 1 --2- 172.21.15.116:0/970762572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564b68ccdb40 0x564b68ccf660 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.518+0000 7efe753ef740 1 --2- 172.21.15.116:0/970762572 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b68ccc5d0 0x564b68ccf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.518+0000 7efe753ef740 1 -- 172.21.15.116:0/970762572 >> 172.21.15.116:0/970762572 conn(0x564b68c95310 msgr2=0x564b68cc7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:05.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.518+0000 7efe753ef740 1 -- 172.21.15.116:0/970762572 shutdown_connections 2024-09-12T03:09:05.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.518+0000 7efe753ef740 1 -- 172.21.15.116:0/970762572 wait complete. 2024-09-12T03:09:05.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.519+0000 7efe753ef740 1 -- start start 2024-09-12T03:09:05.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.519+0000 7efe753ef740 1 Processor -- start 2024-09-12T03:09:05.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.519+0000 7efe753ef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b68ccc5d0 0x564b68d49900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:05.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.519+0000 7efe753ef740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b68ccdb40 0x564b68d49e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:05.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.519+0000 7efe753ef740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564b68cd7370 0x564b68d54980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:05.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.519+0000 7efe753ef740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c1abab0 con 0x564b68cd7370 2024-09-12T03:09:05.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.519+0000 7efe753ef740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c1acf20 con 0x564b68ccdb40 2024-09-12T03:09:05.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.519+0000 7efe753ef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c1ac180 con 0x564b68ccc5d0 2024-09-12T03:09:05.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.519+0000 7efe753ef740 10 client.0 _ll_register_callbacks cb 0x7efe64009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:05.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.519+0000 7efe715b3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564b68cd7370 0x564b68d54980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:05.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.519+0000 7efe715b3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564b68cd7370 0x564b68d54980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56304/0 (socket says 172.21.15.116:56304) 2024-09-12T03:09:05.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.519+0000 7efe715b3640 1 -- 172.21.15.116:0/3033244698 learned_addr learned my addr 172.21.15.116:0/3033244698 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:05.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.519+0000 7efe6bfff640 1 --2- 172.21.15.116:0/3033244698 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b68ccdb40 0x564b68d49e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:05.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.519+0000 7efe70db2640 1 --2- 172.21.15.116:0/3033244698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b68ccc5d0 0x564b68d49900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:05.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.519+0000 7efe715b3640 1 -- 172.21.15.116:0/3033244698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b68ccc5d0 msgr2=0x564b68d49900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:05.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.519+0000 7efe715b3640 1 --2- 172.21.15.116:0/3033244698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b68ccc5d0 0x564b68d49900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.520+0000 7efe715b3640 1 -- 172.21.15.116:0/3033244698 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b68ccdb40 msgr2=0x564b68d49e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:05.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.520+0000 7efe715b3640 1 --2- 172.21.15.116:0/3033244698 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b68ccdb40 0x564b68d49e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.520+0000 7efe715b3640 1 -- 172.21.15.116:0/3033244698 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b68d54ec0 con 0x564b68cd7370 2024-09-12T03:09:05.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.520+0000 7efe715b3640 1 --2- 172.21.15.116:0/3033244698 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564b68cd7370 0x564b68d54980 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7efe6402ade0 tx=0x7efe640095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6140e000cf62ada5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:05.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.520+0000 7efe4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:09:05.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.520+0000 7efe4bfff640 1 -- 172.21.15.116:0/3033244698 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe640305c0 con 0x564b68cd7370 2024-09-12T03:09:05.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.520+0000 7efe4bfff640 1 -- 172.21.15.116:0/3033244698 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7efe64030760 con 0x564b68cd7370 2024-09-12T03:09:05.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.520+0000 7efe753ef740 1 -- 172.21.15.116:0/3033244698 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe580a3ff0 con 0x564b68cd7370 2024-09-12T03:09:05.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.520+0000 7efe4bfff640 1 -- 172.21.15.116:0/3033244698 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efe64030960 con 0x564b68cd7370 2024-09-12T03:09:05.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.520+0000 7efe753ef740 10 client.16836 fetch_fsmap learned FSMap version 25 2024-09-12T03:09:05.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.520+0000 7efe753ef740 1 -- 172.21.15.116:0/3033244698 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564b68d56090 con 0x564b68cd7370 2024-09-12T03:09:05.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.520+0000 7efe4bfff640 1 -- 172.21.15.116:0/3033244698 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe580a3ff0 con 0x564b68cd7370 2024-09-12T03:09:05.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.520+0000 7efe4bfff640 1 -- 172.21.15.116:0/3033244698 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7efe6404aad0 con 0x564b68cd7370 2024-09-12T03:09:05.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.520+0000 7efe753ef740 10 client.16836 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:09:05.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.520+0000 7efe753ef740 10 client.16836 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:09:05.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.520+0000 7efe753ef740 20 client.16836 populate_metadata read hostname 'smithi116' 2024-09-12T03:09:05.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.520+0000 7efe753ef740 1 --2- 172.21.15.116:0/3033244698 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x564b68d56860 0x564b68d58c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:05.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.520+0000 7efe753ef740 4 client.16836 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:09:05.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.520+0000 7efe753ef740 1 -- 172.21.15.116:0/3033244698 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564b68c39d70 con 0x564b68d56860 2024-09-12T03:09:05.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.521+0000 7efe70db2640 1 --2- 172.21.15.116:0/3033244698 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x564b68d56860 0x564b68d58c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:05.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.521+0000 7efe4bfff640 10 client.16836 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:09:05.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.521+0000 7efe70db2640 1 --2- 172.21.15.116:0/3033244698 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x564b68d56860 0x564b68d58c80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9573716446679965 server_cookie=8702e47901c24c02 in_seq=0 out_seq=0 2024-09-12T03:09:05.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.521+0000 7efe4bfff640 1 -- 172.21.15.116:0/3033244698 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564b68c39d70 con 0x564b68d56860 2024-09-12T03:09:05.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.521+0000 7efe4bfff640 10 client.16836 handle_command_reply: tid=0 2024-09-12T03:09:05.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.521+0000 7efe753ef740 1 client.16836 shutdown 2024-09-12T03:09:05.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.521+0000 7efe753ef740 1 -- 172.21.15.116:0/3033244698 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564b68cd7370 msgr2=0x564b68d54980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:05.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.521+0000 7efe753ef740 1 --2- 172.21.15.116:0/3033244698 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564b68cd7370 0x564b68d54980 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7efe6402ade0 tx=0x7efe640095c0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.521+0000 7efe753ef740 1 -- 172.21.15.116:0/3033244698 shutdown_connections 2024-09-12T03:09:05.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.521+0000 7efe753ef740 1 --2- 172.21.15.116:0/3033244698 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x564b68d56860 0x564b68d58c80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.522+0000 7efe753ef740 1 --2- 172.21.15.116:0/3033244698 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564b68cd7370 0x564b68d54980 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.522+0000 7efe753ef740 1 --2- 172.21.15.116:0/3033244698 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b68ccdb40 0x564b68d49e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.522+0000 7efe753ef740 1 --2- 172.21.15.116:0/3033244698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b68ccc5d0 0x564b68d49900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.522+0000 7efe753ef740 1 -- 172.21.15.116:0/3033244698 >> 172.21.15.116:0/3033244698 conn(0x564b68c95310 msgr2=0x564b68cd57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:05.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.522+0000 7efe4bfff640 0 client.16836 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:09:05.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.522+0000 7efe753ef740 1 -- 172.21.15.116:0/3033244698 shutdown_connections 2024-09-12T03:09:05.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.522+0000 7efe753ef740 1 -- 172.21.15.116:0/3033244698 wait complete. 2024-09-12T03:09:05.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.522+0000 7efe753ef740 20 client.16836 trim_cache size 0 max 16384 2024-09-12T03:09:05.528 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:09:05.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.522+0000 7efe73016640 1 -- 172.21.15.116:0/518819922 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7efe5c0a2b80 msgr2=0x7efe5c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:05.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.522+0000 7efe73016640 1 --2- 172.21.15.116:0/518819922 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7efe5c0a2b80 0x7efe5c0a5040 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x564b68ccd070 tx=0x7efe64004a80 comp rx=0 tx=0).stop 2024-09-12T03:09:05.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.522+0000 7efe73016640 1 -- 172.21.15.116:0/518819922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe6c102790 msgr2=0x7efe6c198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:05.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.522+0000 7efe73016640 1 --2- 172.21.15.116:0/518819922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe6c102790 0x7efe6c198910 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7efe5800d950 tx=0x7efe5800de10 comp rx=0 tx=0).stop 2024-09-12T03:09:05.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.522+0000 7efe73016640 1 -- 172.21.15.116:0/518819922 shutdown_connections 2024-09-12T03:09:05.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.522+0000 7efe73016640 1 --2- 172.21.15.116:0/518819922 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7efe5c0a2b80 0x7efe5c0a5040 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.522+0000 7efe73016640 1 --2- 172.21.15.116:0/518819922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe6c106f10 0x7efe6c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.522+0000 7efe73016640 1 --2- 172.21.15.116:0/518819922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe6c102790 0x7efe6c198910 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.522+0000 7efe73016640 1 --2- 172.21.15.116:0/518819922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe6c101de0 0x7efe6c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:05.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.522+0000 7efe73016640 1 -- 172.21.15.116:0/518819922 >> 172.21.15.116:0/518819922 conn(0x7efe6c0fd8c0 msgr2=0x7efe6c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:05.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.523+0000 7efe73016640 1 -- 172.21.15.116:0/518819922 shutdown_connections 2024-09-12T03:09:05.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:05.523+0000 7efe73016640 1 -- 172.21.15.116:0/518819922 wait complete. 2024-09-12T03:09:06.380 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:09:06.625 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:09:06.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:06 smithi183 ceph-mon[35535]: pgmap v521: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.9 KiB/s wr, 8 op/s 2024-09-12T03:09:06.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:06 smithi177 ceph-mon[35305]: pgmap v521: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.9 KiB/s wr, 8 op/s 2024-09-12T03:09:06.746 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:06 smithi116 ceph-mon[31410]: pgmap v521: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.9 KiB/s wr, 8 op/s 2024-09-12T03:09:07.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.077+0000 7f1ef26b4640 1 -- 172.21.15.116:0/3993811528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1eec106f50 msgr2=0x7f1eec10eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:07.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.077+0000 7f1ef26b4640 1 --2- 172.21.15.116:0/3993811528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1eec106f50 0x7f1eec10eaf0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4009cd0 tx=0x7f1ee402a8e0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.077+0000 7f1ef26b4640 1 -- 172.21.15.116:0/3993811528 shutdown_connections 2024-09-12T03:09:07.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.077+0000 7f1ef26b4640 1 --2- 172.21.15.116:0/3993811528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1eec106f50 0x7f1eec10eaf0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.077+0000 7f1ef26b4640 1 --2- 172.21.15.116:0/3993811528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1eec100ca0 0x7f1eec106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.077+0000 7f1ef26b4640 1 --2- 172.21.15.116:0/3993811528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eec1002f0 0x7f1eec1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.077+0000 7f1ef26b4640 1 -- 172.21.15.116:0/3993811528 >> 172.21.15.116:0/3993811528 conn(0x7f1eec077210 msgr2=0x7f1eec0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:07.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.077+0000 7f1ef26b4640 1 -- 172.21.15.116:0/3993811528 shutdown_connections 2024-09-12T03:09:07.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.077+0000 7f1ef26b4640 1 -- 172.21.15.116:0/3993811528 wait complete. 2024-09-12T03:09:07.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.078+0000 7f1ef26b4640 1 Processor -- start 2024-09-12T03:09:07.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.078+0000 7f1ef26b4640 1 -- start start 2024-09-12T03:09:07.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.078+0000 7f1ef26b4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1eec1002f0 0x7f1eec198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.078+0000 7f1ef26b4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1eec100ca0 0x7f1eec198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.078+0000 7f1ef26b4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eec106f50 0x7f1eec19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.078+0000 7f1ef26b4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1eec112000 con 0x7f1eec100ca0 2024-09-12T03:09:07.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.078+0000 7f1ef26b4640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1eec111e80 con 0x7f1eec1002f0 2024-09-12T03:09:07.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.078+0000 7f1ef26b4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1eec112180 con 0x7f1eec106f50 2024-09-12T03:09:07.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.078+0000 7f1ee3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1eec100ca0 0x7f1eec198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.078+0000 7f1ee3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1eec100ca0 0x7f1eec198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56328/0 (socket says 172.21.15.116:56328) 2024-09-12T03:09:07.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.078+0000 7f1ee3fff640 1 -- 172.21.15.116:0/239317108 learned_addr learned my addr 172.21.15.116:0/239317108 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:07.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.078+0000 7f1eebfff640 1 --2- 172.21.15.116:0/239317108 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1eec1002f0 0x7f1eec198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.078+0000 7f1ef0c51640 1 --2- 172.21.15.116:0/239317108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eec106f50 0x7f1eec19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.078+0000 7f1ee3fff640 1 -- 172.21.15.116:0/239317108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eec106f50 msgr2=0x7f1eec19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:07.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.079+0000 7f1ee3fff640 1 --2- 172.21.15.116:0/239317108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eec106f50 0x7f1eec19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.079+0000 7f1ee3fff640 1 -- 172.21.15.116:0/239317108 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1eec1002f0 msgr2=0x7f1eec198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:07.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.079+0000 7f1ee3fff640 1 --2- 172.21.15.116:0/239317108 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1eec1002f0 0x7f1eec198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.079+0000 7f1ee3fff640 1 -- 172.21.15.116:0/239317108 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eec19d3f0 con 0x7f1eec100ca0 2024-09-12T03:09:07.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.079+0000 7f1eebfff640 1 --2- 172.21.15.116:0/239317108 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1eec1002f0 0x7f1eec198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:09:07.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.079+0000 7f1ef0c51640 1 --2- 172.21.15.116:0/239317108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eec106f50 0x7f1eec19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:09:07.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.079+0000 7f1ee3fff640 1 --2- 172.21.15.116:0/239317108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1eec100ca0 0x7f1eec198980 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f1ed400c970 tx=0x7f1ed400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:07.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.079+0000 7f1ee9ffb640 1 -- 172.21.15.116:0/239317108 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4014070 con 0x7f1eec100ca0 2024-09-12T03:09:07.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.079+0000 7f1ef26b4640 1 -- 172.21.15.116:0/239317108 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eec19d6e0 con 0x7f1eec100ca0 2024-09-12T03:09:07.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.079+0000 7f1ee9ffb640 1 -- 172.21.15.116:0/239317108 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ed4004440 con 0x7f1eec100ca0 2024-09-12T03:09:07.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.079+0000 7f1ee9ffb640 1 -- 172.21.15.116:0/239317108 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed400f450 con 0x7f1eec100ca0 2024-09-12T03:09:07.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.079+0000 7f1ef26b4640 1 -- 172.21.15.116:0/239317108 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eec1a4fc0 con 0x7f1eec100ca0 2024-09-12T03:09:07.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.081+0000 7f1ee9ffb640 1 -- 172.21.15.116:0/239317108 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1ed4017070 con 0x7f1eec100ca0 2024-09-12T03:09:07.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.082+0000 7f1ef4a8d740 1 Processor -- start 2024-09-12T03:09:07.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.082+0000 7f1ef4a8d740 1 -- start start 2024-09-12T03:09:07.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.082+0000 7f1ef4a8d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93a53fb40 0x55b93a53e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.082+0000 7f1ef4a8d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b93a53e5d0 0x55b93a53ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.082+0000 7f1ef4a8d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b93a549370 0x55b93a54b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.082+0000 7f1ef4a8d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b93a4a42f0 con 0x55b93a53e5d0 2024-09-12T03:09:07.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.082+0000 7f1ef4a8d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b93a44bcf0 con 0x55b93a549370 2024-09-12T03:09:07.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.082+0000 7f1ef4a8d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b93a4a02b0 con 0x55b93a53fb40 2024-09-12T03:09:07.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.082+0000 7f1eebfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b93a53e5d0 0x55b93a53ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.082+0000 7f1eebfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b93a53e5d0 0x55b93a53ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56338/0 (socket says 172.21.15.116:56338) 2024-09-12T03:09:07.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.082+0000 7f1eebfff640 1 -- 172.21.15.116:0/4056612079 learned_addr learned my addr 172.21.15.116:0/4056612079 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:07.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.082+0000 7f1ef0c51640 1 --2- 172.21.15.116:0/4056612079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93a53fb40 0x55b93a53e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.082+0000 7f1ee3fff640 1 --2- 172.21.15.116:0/4056612079 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b93a549370 0x55b93a54b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.082+0000 7f1eebfff640 1 -- 172.21.15.116:0/4056612079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93a53fb40 msgr2=0x55b93a53e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:07.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.082+0000 7f1eebfff640 1 --2- 172.21.15.116:0/4056612079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93a53fb40 0x55b93a53e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.082+0000 7f1eebfff640 1 -- 172.21.15.116:0/4056612079 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b93a549370 msgr2=0x55b93a54b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:07.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.082+0000 7f1ee9ffb640 1 --2- 172.21.15.116:0/239317108 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1ed80a2b80 0x7f1ed80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.082+0000 7f1eebfff640 1 --2- 172.21.15.116:0/4056612079 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b93a549370 0x55b93a54b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.082+0000 7f1eebfff640 1 -- 172.21.15.116:0/4056612079 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b93a4303a0 con 0x55b93a53e5d0 2024-09-12T03:09:07.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.082+0000 7f1ee3fff640 1 --2- 172.21.15.116:0/4056612079 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b93a549370 0x55b93a54b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:09:07.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.082+0000 7f1ef0c51640 1 --2- 172.21.15.116:0/239317108 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1ed80a2b80 0x7f1ed80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.083+0000 7f1eebfff640 1 --2- 172.21.15.116:0/4056612079 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b93a53e5d0 0x55b93a53ea10 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f1edc007850 tx=0x7f1edc007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26b2c4d0aaca10 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:07.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.083+0000 7f1ee9ffb640 1 -- 172.21.15.116:0/239317108 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1ed40cef20 con 0x7f1eec100ca0 2024-09-12T03:09:07.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.083+0000 7f1ee27fc640 1 -- 172.21.15.116:0/4056612079 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc00d950 con 0x55b93a53e5d0 2024-09-12T03:09:07.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.083+0000 7f1ee27fc640 1 -- 172.21.15.116:0/4056612079 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1edc018070 con 0x55b93a53e5d0 2024-09-12T03:09:07.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.083+0000 7f1ee27fc640 1 -- 172.21.15.116:0/4056612079 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc0123e0 con 0x55b93a53e5d0 2024-09-12T03:09:07.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.083+0000 7f1ef4a8d740 1 -- 172.21.15.116:0/4056612079 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b93a53e5d0 msgr2=0x55b93a53ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:07.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.083+0000 7f1ef0c51640 1 --2- 172.21.15.116:0/239317108 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1ed80a2b80 0x7f1ed80a5040 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4009ca0 tx=0x7f1ee40093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:07.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.083+0000 7f1ef4a8d740 1 --2- 172.21.15.116:0/4056612079 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b93a53e5d0 0x55b93a53ea10 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f1edc007850 tx=0x7f1edc007d10 comp rx=0 tx=0).stop 2024-09-12T03:09:07.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.083+0000 7f1ef4a8d740 1 -- 172.21.15.116:0/4056612079 shutdown_connections 2024-09-12T03:09:07.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.083+0000 7f1ef4a8d740 1 --2- 172.21.15.116:0/4056612079 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b93a549370 0x55b93a54b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.083+0000 7f1ef4a8d740 1 --2- 172.21.15.116:0/4056612079 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b93a53e5d0 0x55b93a53ea10 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.083+0000 7f1ef4a8d740 1 --2- 172.21.15.116:0/4056612079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93a53fb40 0x55b93a53e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.083+0000 7f1ef4a8d740 1 -- 172.21.15.116:0/4056612079 >> 172.21.15.116:0/4056612079 conn(0x55b93a507310 msgr2=0x55b93a53ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:07.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.083+0000 7f1ef4a8d740 1 -- 172.21.15.116:0/4056612079 shutdown_connections 2024-09-12T03:09:07.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.083+0000 7f1ef4a8d740 1 -- 172.21.15.116:0/4056612079 wait complete. 2024-09-12T03:09:07.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.084+0000 7f1ef4a8d740 1 -- start start 2024-09-12T03:09:07.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.084+0000 7f1ef4a8d740 1 Processor -- start 2024-09-12T03:09:07.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.084+0000 7f1ef4a8d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b93a53e5d0 0x55b93a548d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.084+0000 7f1ef4a8d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b93a53fb40 0x55b93a545fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.084+0000 7f1ef4a8d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93a549370 0x55b93a546530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.084+0000 7f1ef4a8d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b93a54c070 con 0x55b93a53fb40 2024-09-12T03:09:07.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.084+0000 7f1ef4a8d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b93a45f060 con 0x55b93a53e5d0 2024-09-12T03:09:07.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.084+0000 7f1ef4a8d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b93a54c1f0 con 0x55b93a549370 2024-09-12T03:09:07.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.084+0000 7f1ef4a8d740 10 client.0 _ll_register_callbacks cb 0x55b93a1b28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:07.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.084+0000 7f1ee3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b93a53fb40 0x55b93a545fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.084+0000 7f1ee3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b93a53fb40 0x55b93a545fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56350/0 (socket says 172.21.15.116:56350) 2024-09-12T03:09:07.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.084+0000 7f1ee3fff640 1 -- 172.21.15.116:0/1468495129 learned_addr learned my addr 172.21.15.116:0/1468495129 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:07.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.085+0000 7f1ef0c51640 1 --2- 172.21.15.116:0/1468495129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93a549370 0x55b93a546530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.085+0000 7f1eebfff640 1 --2- 172.21.15.116:0/1468495129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b93a53e5d0 0x55b93a548d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.085+0000 7f1ee3fff640 1 -- 172.21.15.116:0/1468495129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93a549370 msgr2=0x55b93a546530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:07.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.085+0000 7f1ee3fff640 1 --2- 172.21.15.116:0/1468495129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93a549370 0x55b93a546530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.085+0000 7f1ee3fff640 1 -- 172.21.15.116:0/1468495129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b93a53e5d0 msgr2=0x55b93a548d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:07.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.085+0000 7f1ee3fff640 1 --2- 172.21.15.116:0/1468495129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b93a53e5d0 0x55b93a548d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.085+0000 7f1ee3fff640 1 -- 172.21.15.116:0/1468495129 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b93a2153c0 con 0x55b93a53fb40 2024-09-12T03:09:07.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.085+0000 7f1eebfff640 1 --2- 172.21.15.116:0/1468495129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b93a53e5d0 0x55b93a548d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:09:07.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.085+0000 7f1ee3fff640 1 --2- 172.21.15.116:0/1468495129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b93a53fb40 0x55b93a545fb0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4007590 tx=0x7f1ed409b2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3111fca265c3d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:07.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.085+0000 7f1ecb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:09:07.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.085+0000 7f1ecb7fe640 1 -- 172.21.15.116:0/1468495129 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed40a6a70 con 0x55b93a53fb40 2024-09-12T03:09:07.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.085+0000 7f1ecb7fe640 1 -- 172.21.15.116:0/1468495129 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1ed4097e20 con 0x55b93a53fb40 2024-09-12T03:09:07.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.085+0000 7f1ef4a8d740 1 -- 172.21.15.116:0/1468495129 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1edc018070 con 0x55b93a53fb40 2024-09-12T03:09:07.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.085+0000 7f1ecb7fe640 1 -- 172.21.15.116:0/1468495129 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1ed409beb0 con 0x55b93a53fb40 2024-09-12T03:09:07.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.085+0000 7f1ef4a8d740 10 client.16848 fetch_fsmap learned FSMap version 25 2024-09-12T03:09:07.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.085+0000 7f1ef4a8d740 1 -- 172.21.15.116:0/1468495129 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b93a557ac0 con 0x55b93a53fb40 2024-09-12T03:09:07.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.085+0000 7f1ecb7fe640 1 -- 172.21.15.116:0/1468495129 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1edc018070 con 0x55b93a53fb40 2024-09-12T03:09:07.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.085+0000 7f1ecb7fe640 1 -- 172.21.15.116:0/1468495129 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1ed40a3f90 con 0x55b93a53fb40 2024-09-12T03:09:07.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.085+0000 7f1ef4a8d740 10 client.16848 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:09:07.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.085+0000 7f1ef4a8d740 10 client.16848 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:09:07.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.085+0000 7f1ef4a8d740 20 client.16848 populate_metadata read hostname 'smithi116' 2024-09-12T03:09:07.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.086+0000 7f1ef4a8d740 1 --2- 172.21.15.116:0/1468495129 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55b93a558f80 0x55b93a55b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.086+0000 7f1ef4a8d740 4 client.16848 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:09:07.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.086+0000 7f1ef4a8d740 1 -- 172.21.15.116:0/1468495129 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b93a4b1860 con 0x55b93a558f80 2024-09-12T03:09:07.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.086+0000 7f1eebfff640 1 --2- 172.21.15.116:0/1468495129 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55b93a558f80 0x55b93a55b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.086+0000 7f1eebfff640 1 --2- 172.21.15.116:0/1468495129 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55b93a558f80 0x55b93a55b370 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a0415ed1c66d4ea1 server_cookie=75af56ecbe2c2878 in_seq=0 out_seq=0 2024-09-12T03:09:07.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.086+0000 7f1ecb7fe640 10 client.16848 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:09:07.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.087+0000 7f1ecb7fe640 1 -- 172.21.15.116:0/1468495129 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b93a4b1860 con 0x55b93a558f80 2024-09-12T03:09:07.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.087+0000 7f1ecb7fe640 10 client.16848 handle_command_reply: tid=0 2024-09-12T03:09:07.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.087+0000 7f1ef4a8d740 1 client.16848 shutdown 2024-09-12T03:09:07.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.087+0000 7f1ef4a8d740 1 -- 172.21.15.116:0/1468495129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b93a53fb40 msgr2=0x55b93a545fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:07.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.087+0000 7f1ef4a8d740 1 --2- 172.21.15.116:0/1468495129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b93a53fb40 0x55b93a545fb0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4007590 tx=0x7f1ed409b2f0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.087+0000 7f1ef4a8d740 1 -- 172.21.15.116:0/1468495129 shutdown_connections 2024-09-12T03:09:07.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.087+0000 7f1ef4a8d740 1 --2- 172.21.15.116:0/1468495129 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55b93a558f80 0x55b93a55b370 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.087+0000 7f1ef4a8d740 1 --2- 172.21.15.116:0/1468495129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93a549370 0x55b93a546530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.087+0000 7f1ef4a8d740 1 --2- 172.21.15.116:0/1468495129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b93a53fb40 0x55b93a545fb0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.087+0000 7f1ecb7fe640 0 client.16848 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:09:07.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.087+0000 7f1ef4a8d740 1 --2- 172.21.15.116:0/1468495129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b93a53e5d0 0x55b93a548d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.087+0000 7f1ef4a8d740 1 -- 172.21.15.116:0/1468495129 >> 172.21.15.116:0/1468495129 conn(0x55b93a507310 msgr2=0x55b93a540960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:07.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.087+0000 7f1ef4a8d740 1 -- 172.21.15.116:0/1468495129 shutdown_connections 2024-09-12T03:09:07.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.087+0000 7f1ef4a8d740 1 -- 172.21.15.116:0/1468495129 wait complete. 2024-09-12T03:09:07.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.087+0000 7f1ef4a8d740 20 client.16848 trim_cache size 0 max 16384 2024-09-12T03:09:07.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.100+0000 7f1ef4a8d740 1 Processor -- start 2024-09-12T03:09:07.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.100+0000 7f1ef4a8d740 1 -- start start 2024-09-12T03:09:07.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.100+0000 7f1ef4a8d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b93a53e5d0 0x55b93a547f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.100+0000 7f1ef4a8d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b93a53fb40 0x55b93a541120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.100+0000 7f1ef4a8d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93a549370 0x55b93a541660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.100+0000 7f1ef4a8d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1eec1b1fc0 con 0x55b93a53e5d0 2024-09-12T03:09:07.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.100+0000 7f1ef4a8d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1eec1af890 con 0x55b93a53fb40 2024-09-12T03:09:07.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.100+0000 7f1ef4a8d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1eec1ad6e0 con 0x55b93a549370 2024-09-12T03:09:07.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.100+0000 7f1eebfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b93a53e5d0 0x55b93a547f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.100+0000 7f1eebfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b93a53e5d0 0x55b93a547f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56362/0 (socket says 172.21.15.116:56362) 2024-09-12T03:09:07.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.100+0000 7f1eebfff640 1 -- 172.21.15.116:0/1146388319 learned_addr learned my addr 172.21.15.116:0/1146388319 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:07.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.101+0000 7f1ef0c51640 1 --2- 172.21.15.116:0/1146388319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93a549370 0x55b93a541660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.101+0000 7f1ee3fff640 1 --2- 172.21.15.116:0/1146388319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b93a53fb40 0x55b93a541120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.101+0000 7f1eebfff640 1 -- 172.21.15.116:0/1146388319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93a549370 msgr2=0x55b93a541660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:07.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.101+0000 7f1eebfff640 1 --2- 172.21.15.116:0/1146388319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93a549370 0x55b93a541660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.101+0000 7f1eebfff640 1 -- 172.21.15.116:0/1146388319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b93a53fb40 msgr2=0x55b93a541120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:07.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.101+0000 7f1eebfff640 1 --2- 172.21.15.116:0/1146388319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b93a53fb40 0x55b93a541120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.101+0000 7f1eebfff640 1 -- 172.21.15.116:0/1146388319 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b93a5468a0 con 0x55b93a53e5d0 2024-09-12T03:09:07.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.101+0000 7f1eebfff640 1 --2- 172.21.15.116:0/1146388319 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b93a53e5d0 0x55b93a547f20 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f1edc0135c0 tx=0x7f1edc00f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ebc7826940cfe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:07.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.101+0000 7f1ee1ffb640 1 -- 172.21.15.116:0/1146388319 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc02b020 con 0x55b93a53e5d0 2024-09-12T03:09:07.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.101+0000 7f1ee1ffb640 1 -- 172.21.15.116:0/1146388319 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1edc018040 con 0x55b93a53e5d0 2024-09-12T03:09:07.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.101+0000 7f1ee1ffb640 1 -- 172.21.15.116:0/1146388319 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc027a10 con 0x55b93a53e5d0 2024-09-12T03:09:07.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.102+0000 7f1ef4a8d740 1 -- 172.21.15.116:0/1146388319 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b93a53e5d0 msgr2=0x55b93a547f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:07.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.102+0000 7f1ef4a8d740 1 --2- 172.21.15.116:0/1146388319 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b93a53e5d0 0x55b93a547f20 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f1edc0135c0 tx=0x7f1edc00f040 comp rx=0 tx=0).stop 2024-09-12T03:09:07.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.102+0000 7f1ef4a8d740 1 -- 172.21.15.116:0/1146388319 shutdown_connections 2024-09-12T03:09:07.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.102+0000 7f1ef4a8d740 1 --2- 172.21.15.116:0/1146388319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93a549370 0x55b93a541660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.102+0000 7f1ef4a8d740 1 --2- 172.21.15.116:0/1146388319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b93a53fb40 0x55b93a541120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.102+0000 7f1ef4a8d740 1 --2- 172.21.15.116:0/1146388319 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b93a53e5d0 0x55b93a547f20 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.102+0000 7f1ef4a8d740 1 -- 172.21.15.116:0/1146388319 >> 172.21.15.116:0/1146388319 conn(0x55b93a507310 msgr2=0x55b93a547810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:07.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.102+0000 7f1ef4a8d740 1 -- 172.21.15.116:0/1146388319 shutdown_connections 2024-09-12T03:09:07.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.102+0000 7f1ef4a8d740 1 -- 172.21.15.116:0/1146388319 wait complete. 2024-09-12T03:09:07.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.102+0000 7f1ef4a8d740 1 -- start start 2024-09-12T03:09:07.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.102+0000 7f1ef4a8d740 1 Processor -- start 2024-09-12T03:09:07.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.102+0000 7f1ef4a8d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93a53e5d0 0x55b93a54d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.102+0000 7f1ef4a8d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b93a53fb40 0x55b93a54d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.102+0000 7f1ef4a8d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b93a549370 0x55b93a54dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.102+0000 7f1ef4a8d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1eec1add50 con 0x55b93a549370 2024-09-12T03:09:07.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.102+0000 7f1ef4a8d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1eec1af1c0 con 0x55b93a53fb40 2024-09-12T03:09:07.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.102+0000 7f1ef4a8d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1eec1ae420 con 0x55b93a53e5d0 2024-09-12T03:09:07.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.103+0000 7f1ef4a8d740 10 client.0 _ll_register_callbacks cb 0x7f1ed400c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:07.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.103+0000 7f1ef0c51640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b93a549370 0x55b93a54dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.103+0000 7f1ef0c51640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b93a549370 0x55b93a54dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56378/0 (socket says 172.21.15.116:56378) 2024-09-12T03:09:07.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.103+0000 7f1ef0c51640 1 -- 172.21.15.116:0/822512462 learned_addr learned my addr 172.21.15.116:0/822512462 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:07.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.103+0000 7f1eebfff640 1 --2- 172.21.15.116:0/822512462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93a53e5d0 0x55b93a54d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.103+0000 7f1ee3fff640 1 --2- 172.21.15.116:0/822512462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b93a53fb40 0x55b93a54d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.103+0000 7f1ef0c51640 1 -- 172.21.15.116:0/822512462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93a53e5d0 msgr2=0x55b93a54d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:07.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.103+0000 7f1ef0c51640 1 --2- 172.21.15.116:0/822512462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93a53e5d0 0x55b93a54d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.103+0000 7f1ef0c51640 1 -- 172.21.15.116:0/822512462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b93a53fb40 msgr2=0x55b93a54d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:07.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.103+0000 7f1ef0c51640 1 --2- 172.21.15.116:0/822512462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b93a53fb40 0x55b93a54d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.103+0000 7f1ef0c51640 1 -- 172.21.15.116:0/822512462 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b93a5c2cd0 con 0x55b93a549370 2024-09-12T03:09:07.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.103+0000 7f1ee3fff640 1 --2- 172.21.15.116:0/822512462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b93a53fb40 0x55b93a54d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:09:07.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.103+0000 7f1ef0c51640 1 --2- 172.21.15.116:0/822512462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b93a549370 0x55b93a54dd50 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4003c20 tx=0x7f1ee40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f96e4d3b8b941d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:07.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.103+0000 7f1ecb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:09:07.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.103+0000 7f1ecb7fe640 1 -- 172.21.15.116:0/822512462 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4031430 con 0x55b93a549370 2024-09-12T03:09:07.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.103+0000 7f1ecb7fe640 1 -- 172.21.15.116:0/822512462 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1ee4004540 con 0x55b93a549370 2024-09-12T03:09:07.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.103+0000 7f1ef4a8d740 1 -- 172.21.15.116:0/822512462 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1edc018040 con 0x55b93a549370 2024-09-12T03:09:07.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.103+0000 7f1ecb7fe640 1 -- 172.21.15.116:0/822512462 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1ee4004740 con 0x55b93a549370 2024-09-12T03:09:07.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.103+0000 7f1ef4a8d740 10 client.16854 fetch_fsmap learned FSMap version 25 2024-09-12T03:09:07.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.103+0000 7f1ef4a8d740 1 -- 172.21.15.116:0/822512462 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b93a5c3fb0 con 0x55b93a549370 2024-09-12T03:09:07.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.104+0000 7f1ecb7fe640 1 -- 172.21.15.116:0/822512462 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1edc018040 con 0x55b93a549370 2024-09-12T03:09:07.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.104+0000 7f1ecb7fe640 1 -- 172.21.15.116:0/822512462 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1ee4030070 con 0x55b93a549370 2024-09-12T03:09:07.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.104+0000 7f1ef4a8d740 10 client.16854 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:09:07.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.104+0000 7f1ef4a8d740 10 client.16854 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:09:07.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.104+0000 7f1ef4a8d740 20 client.16854 populate_metadata read hostname 'smithi116' 2024-09-12T03:09:07.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.104+0000 7f1ef4a8d740 1 --2- 172.21.15.116:0/822512462 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55b93a5c4780 0x55b93a5c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.104+0000 7f1ef4a8d740 4 client.16854 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:09:07.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.104+0000 7f1ef4a8d740 1 -- 172.21.15.116:0/822512462 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b93a4abd70 con 0x55b93a5c4780 2024-09-12T03:09:07.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.104+0000 7f1eebfff640 1 --2- 172.21.15.116:0/822512462 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55b93a5c4780 0x55b93a5c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.105+0000 7f1ecb7fe640 10 client.16854 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:09:07.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.105+0000 7f1eebfff640 1 --2- 172.21.15.116:0/822512462 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55b93a5c4780 0x55b93a5c6ba0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8372b26030710220 server_cookie=63610752e0cd0387 in_seq=0 out_seq=0 2024-09-12T03:09:07.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.105+0000 7f1ecb7fe640 1 -- 172.21.15.116:0/822512462 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b93a4abd70 con 0x55b93a5c4780 2024-09-12T03:09:07.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.105+0000 7f1ecb7fe640 10 client.16854 handle_command_reply: tid=0 2024-09-12T03:09:07.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.105+0000 7f1ef4a8d740 1 client.16854 shutdown 2024-09-12T03:09:07.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.105+0000 7f1ef4a8d740 1 -- 172.21.15.116:0/822512462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b93a549370 msgr2=0x55b93a54dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:07.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.105+0000 7f1ef4a8d740 1 --2- 172.21.15.116:0/822512462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b93a549370 0x55b93a54dd50 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4003c20 tx=0x7f1ee40095c0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.105+0000 7f1ef4a8d740 1 -- 172.21.15.116:0/822512462 shutdown_connections 2024-09-12T03:09:07.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.105+0000 7f1ef4a8d740 1 --2- 172.21.15.116:0/822512462 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55b93a5c4780 0x55b93a5c6ba0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.105+0000 7f1ef4a8d740 1 --2- 172.21.15.116:0/822512462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b93a549370 0x55b93a54dd50 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.105+0000 7f1ef4a8d740 1 --2- 172.21.15.116:0/822512462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b93a53fb40 0x55b93a54d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.105+0000 7f1ef4a8d740 1 --2- 172.21.15.116:0/822512462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93a53e5d0 0x55b93a54d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.105+0000 7f1ef4a8d740 1 -- 172.21.15.116:0/822512462 >> 172.21.15.116:0/822512462 conn(0x55b93a507310 msgr2=0x55b93a548030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:07.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.105+0000 7f1ecb7fe640 0 client.16854 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:09:07.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.105+0000 7f1ef4a8d740 1 -- 172.21.15.116:0/822512462 shutdown_connections 2024-09-12T03:09:07.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.105+0000 7f1ef4a8d740 1 -- 172.21.15.116:0/822512462 wait complete. 2024-09-12T03:09:07.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.105+0000 7f1ef4a8d740 20 client.16854 trim_cache size 0 max 16384 2024-09-12T03:09:07.117 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:09:07.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.106+0000 7f1ef26b4640 1 -- 172.21.15.116:0/239317108 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1ed80a2b80 msgr2=0x7f1ed80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:07.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.106+0000 7f1ef26b4640 1 --2- 172.21.15.116:0/239317108 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1ed80a2b80 0x7f1ed80a5040 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4009ca0 tx=0x7f1ee40093f0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.106+0000 7f1ef26b4640 1 -- 172.21.15.116:0/239317108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1eec100ca0 msgr2=0x7f1eec198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:07.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.106+0000 7f1ef26b4640 1 --2- 172.21.15.116:0/239317108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1eec100ca0 0x7f1eec198980 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f1ed400c970 tx=0x7f1ed400ce30 comp rx=0 tx=0).stop 2024-09-12T03:09:07.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.106+0000 7f1ef26b4640 1 -- 172.21.15.116:0/239317108 shutdown_connections 2024-09-12T03:09:07.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.106+0000 7f1ef26b4640 1 --2- 172.21.15.116:0/239317108 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1ed80a2b80 0x7f1ed80a5040 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.106+0000 7f1ef26b4640 1 --2- 172.21.15.116:0/239317108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eec106f50 0x7f1eec19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.106+0000 7f1ef26b4640 1 --2- 172.21.15.116:0/239317108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1eec100ca0 0x7f1eec198980 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.106+0000 7f1ef26b4640 1 --2- 172.21.15.116:0/239317108 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1eec1002f0 0x7f1eec198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.106+0000 7f1ef26b4640 1 -- 172.21.15.116:0/239317108 >> 172.21.15.116:0/239317108 conn(0x7f1eec077210 msgr2=0x7f1eec105750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:07.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.106+0000 7f1ef26b4640 1 -- 172.21.15.116:0/239317108 shutdown_connections 2024-09-12T03:09:07.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:07.106+0000 7f1ef26b4640 1 -- 172.21.15.116:0/239317108 wait complete. 2024-09-12T03:09:07.820 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:09:08.064 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:09:08.489 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:08 smithi116 ceph-mon[31410]: pgmap v522: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-09-12T03:09:08.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.569+0000 7f0888c86640 1 -- 172.21.15.116:0/3041227233 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0884100e20 msgr2=0x7f0884101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:08.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.569+0000 7f0888c86640 1 --2- 172.21.15.116:0/3041227233 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0884100e20 0x7f0884101200 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f0874009900 tx=0x7f087402a480 comp rx=0 tx=0).stop 2024-09-12T03:09:08.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.569+0000 7f0888c86640 1 -- 172.21.15.116:0/3041227233 shutdown_connections 2024-09-12T03:09:08.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.569+0000 7f0888c86640 1 --2- 172.21.15.116:0/3041227233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0884102ef0 0x7f088410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.569+0000 7f0888c86640 1 --2- 172.21.15.116:0/3041227233 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0884102570 0x7f08841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.569+0000 7f0888c86640 1 --2- 172.21.15.116:0/3041227233 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0884100e20 0x7f0884101200 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.569+0000 7f0888c86640 1 -- 172.21.15.116:0/3041227233 >> 172.21.15.116:0/3041227233 conn(0x7f08840fc7d0 msgr2=0x7f08840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:08.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.569+0000 7f0888c86640 1 -- 172.21.15.116:0/3041227233 shutdown_connections 2024-09-12T03:09:08.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.569+0000 7f0888c86640 1 -- 172.21.15.116:0/3041227233 wait complete. 2024-09-12T03:09:08.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.570+0000 7f0888c86640 1 Processor -- start 2024-09-12T03:09:08.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.570+0000 7f0888c86640 1 -- start start 2024-09-12T03:09:08.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.570+0000 7f0888c86640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0884100e20 0x7f08841985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:08.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.570+0000 7f0888c86640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0884102570 0x7f0884198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:08.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.570+0000 7f0888c86640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0884102ef0 0x7f088419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:08.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.570+0000 7f0888c86640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0884110fc0 con 0x7f0884102570 2024-09-12T03:09:08.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.570+0000 7f0888c86640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0884110e40 con 0x7f0884100e20 2024-09-12T03:09:08.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.570+0000 7f0888c86640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0884111140 con 0x7f0884102ef0 2024-09-12T03:09:08.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.570+0000 7f0879d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0884102570 0x7f0884198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:08.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.570+0000 7f0879d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0884102570 0x7f0884198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56396/0 (socket says 172.21.15.116:56396) 2024-09-12T03:09:08.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.570+0000 7f0879d9b640 1 -- 172.21.15.116:0/364190109 learned_addr learned my addr 172.21.15.116:0/364190109 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:08.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.571+0000 7f088259c640 1 --2- 172.21.15.116:0/364190109 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0884100e20 0x7f08841985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:08.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.571+0000 7f0882d9d640 1 --2- 172.21.15.116:0/364190109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0884102ef0 0x7f088419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:08.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.571+0000 7f0879d9b640 1 -- 172.21.15.116:0/364190109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0884102ef0 msgr2=0x7f088419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:08.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.571+0000 7f0879d9b640 1 --2- 172.21.15.116:0/364190109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0884102ef0 0x7f088419ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.571+0000 7f0879d9b640 1 -- 172.21.15.116:0/364190109 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0884100e20 msgr2=0x7f08841985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:08.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.571+0000 7f0879d9b640 1 --2- 172.21.15.116:0/364190109 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0884100e20 0x7f08841985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.571+0000 7f0879d9b640 1 -- 172.21.15.116:0/364190109 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088419d560 con 0x7f0884102570 2024-09-12T03:09:08.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.571+0000 7f0882d9d640 1 --2- 172.21.15.116:0/364190109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0884102ef0 0x7f088419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:09:08.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.571+0000 7f0879d9b640 1 --2- 172.21.15.116:0/364190109 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0884102570 0x7f0884198af0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f086c00b4a0 tx=0x7f086c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:08.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.571+0000 7f087bfff640 1 -- 172.21.15.116:0/364190109 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c014070 con 0x7f0884102570 2024-09-12T03:09:08.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.571+0000 7f0888c86640 1 -- 172.21.15.116:0/364190109 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f088419d850 con 0x7f0884102570 2024-09-12T03:09:08.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.571+0000 7f087bfff640 1 -- 172.21.15.116:0/364190109 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f086c010af0 con 0x7f0884102570 2024-09-12T03:09:08.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.571+0000 7f087bfff640 1 -- 172.21.15.116:0/364190109 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c017d10 con 0x7f0884102570 2024-09-12T03:09:08.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.571+0000 7f0888c86640 1 -- 172.21.15.116:0/364190109 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08841a5130 con 0x7f0884102570 2024-09-12T03:09:08.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.573+0000 7f087bfff640 1 -- 172.21.15.116:0/364190109 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f086c017070 con 0x7f0884102570 2024-09-12T03:09:08.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.574+0000 7f088b05f740 1 Processor -- start 2024-09-12T03:09:08.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.574+0000 7f088b05f740 1 -- start start 2024-09-12T03:09:08.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.574+0000 7f088b05f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f01a09b40 0x559f01a08090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:08.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.574+0000 7f088b05f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f01a085d0 0x559f01a08a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:08.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.574+0000 7f088b05f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f01a13370 0x559f01a15760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:08.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.574+0000 7f088b05f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559f0196e2f0 con 0x559f01a09b40 2024-09-12T03:09:08.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.574+0000 7f088b05f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559f01915cf0 con 0x559f01a13370 2024-09-12T03:09:08.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.574+0000 7f088b05f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559f0196a2b0 con 0x559f01a085d0 2024-09-12T03:09:08.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.574+0000 7f0882d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f01a09b40 0x559f01a08090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:08.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.574+0000 7f0882d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f01a09b40 0x559f01a08090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56406/0 (socket says 172.21.15.116:56406) 2024-09-12T03:09:08.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.574+0000 7f0882d9d640 1 -- 172.21.15.116:0/1380882897 learned_addr learned my addr 172.21.15.116:0/1380882897 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:08.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.574+0000 7f088259c640 1 --2- 172.21.15.116:0/1380882897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f01a085d0 0x559f01a08a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:08.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.574+0000 7f087bfff640 1 --2- 172.21.15.116:0/364190109 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f08700a2b80 0x7f08700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:08.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.575+0000 7f0879d9b640 1 --2- 172.21.15.116:0/1380882897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f01a13370 0x559f01a15760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:08.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.575+0000 7f0882d9d640 1 -- 172.21.15.116:0/1380882897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f01a085d0 msgr2=0x559f01a08a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:08.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.575+0000 7f0882d9d640 1 --2- 172.21.15.116:0/1380882897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f01a085d0 0x559f01a08a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.575+0000 7f0882d9d640 1 -- 172.21.15.116:0/1380882897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f01a13370 msgr2=0x559f01a15760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:08.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.575+0000 7f0882d9d640 1 --2- 172.21.15.116:0/1380882897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f01a13370 0x559f01a15760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.575+0000 7f0882d9d640 1 -- 172.21.15.116:0/1380882897 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f018fa3a0 con 0x559f01a09b40 2024-09-12T03:09:08.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.575+0000 7f088259c640 1 --2- 172.21.15.116:0/1380882897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f01a085d0 0x559f01a08a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:09:08.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.575+0000 7f0879d9b640 1 --2- 172.21.15.116:0/1380882897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f01a13370 0x559f01a15760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:09:08.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.575+0000 7f0882d9d640 1 --2- 172.21.15.116:0/364190109 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f08700a2b80 0x7f08700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:08.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.575+0000 7f087bfff640 1 -- 172.21.15.116:0/364190109 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f086c0ce490 con 0x7f0884102570 2024-09-12T03:09:08.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.575+0000 7f0882d9d640 1 --2- 172.21.15.116:0/1380882897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f01a09b40 0x559f01a08090 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f087c010540 tx=0x7f087c010a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747745a7b320da46 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:08.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.575+0000 7f0878d99640 1 -- 172.21.15.116:0/1380882897 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c01d400 con 0x559f01a09b40 2024-09-12T03:09:08.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.575+0000 7f0878d99640 1 -- 172.21.15.116:0/1380882897 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f087c01da80 con 0x559f01a09b40 2024-09-12T03:09:08.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.575+0000 7f0878d99640 1 -- 172.21.15.116:0/1380882897 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c01c840 con 0x559f01a09b40 2024-09-12T03:09:08.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.576+0000 7f088b05f740 1 -- 172.21.15.116:0/1380882897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f01a09b40 msgr2=0x559f01a08090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:08.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.576+0000 7f0882d9d640 1 --2- 172.21.15.116:0/364190109 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f08700a2b80 0x7f08700a5040 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f087c021000 tx=0x7f087c01e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:08.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.576+0000 7f088b05f740 1 --2- 172.21.15.116:0/1380882897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f01a09b40 0x559f01a08090 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f087c010540 tx=0x7f087c010a00 comp rx=0 tx=0).stop 2024-09-12T03:09:08.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.576+0000 7f088b05f740 1 -- 172.21.15.116:0/1380882897 shutdown_connections 2024-09-12T03:09:08.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.576+0000 7f088b05f740 1 --2- 172.21.15.116:0/1380882897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f01a13370 0x559f01a15760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.576+0000 7f088b05f740 1 --2- 172.21.15.116:0/1380882897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f01a085d0 0x559f01a08a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.576+0000 7f088b05f740 1 --2- 172.21.15.116:0/1380882897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f01a09b40 0x559f01a08090 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.576+0000 7f088b05f740 1 -- 172.21.15.116:0/1380882897 >> 172.21.15.116:0/1380882897 conn(0x559f019d1310 msgr2=0x559f01a04c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:08.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.576+0000 7f088b05f740 1 -- 172.21.15.116:0/1380882897 shutdown_connections 2024-09-12T03:09:08.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.576+0000 7f088b05f740 1 -- 172.21.15.116:0/1380882897 wait complete. 2024-09-12T03:09:08.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.576+0000 7f088b05f740 1 -- start start 2024-09-12T03:09:08.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.576+0000 7f088b05f740 1 Processor -- start 2024-09-12T03:09:08.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.577+0000 7f088b05f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f01a085d0 0x559f01a12d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:08.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.577+0000 7f088b05f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f01a09b40 0x559f01a10000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:08.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.577+0000 7f088b05f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f01a13370 0x559f01a105d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:08.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.577+0000 7f088b05f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559f01a16070 con 0x559f01a13370 2024-09-12T03:09:08.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.577+0000 7f088b05f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559f01929060 con 0x559f01a09b40 2024-09-12T03:09:08.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.577+0000 7f088b05f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559f01a161f0 con 0x559f01a085d0 2024-09-12T03:09:08.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.577+0000 7f088b05f740 10 client.0 _ll_register_callbacks cb 0x559f0167c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:08.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.577+0000 7f0882d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f01a13370 0x559f01a105d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:08.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.577+0000 7f0882d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f01a13370 0x559f01a105d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56416/0 (socket says 172.21.15.116:56416) 2024-09-12T03:09:08.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.577+0000 7f0882d9d640 1 -- 172.21.15.116:0/1600396431 learned_addr learned my addr 172.21.15.116:0/1600396431 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:08.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.577+0000 7f088259c640 1 --2- 172.21.15.116:0/1600396431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f01a085d0 0x559f01a12d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:08.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.577+0000 7f0882d9d640 1 -- 172.21.15.116:0/1600396431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f01a085d0 msgr2=0x559f01a12d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:08.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.577+0000 7f0882d9d640 1 --2- 172.21.15.116:0/1600396431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f01a085d0 0x559f01a12d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.577+0000 7f0882d9d640 1 -- 172.21.15.116:0/1600396431 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f01a09b40 msgr2=0x559f01a10000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:08.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.577+0000 7f0882d9d640 1 --2- 172.21.15.116:0/1600396431 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f01a09b40 0x559f01a10000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.577+0000 7f0882d9d640 1 -- 172.21.15.116:0/1600396431 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f016df3c0 con 0x559f01a13370 2024-09-12T03:09:08.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.577+0000 7f0882d9d640 1 --2- 172.21.15.116:0/1600396431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f01a13370 0x559f01a105d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f087c011010 tx=0x7f087c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3000f2185c29391f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:08.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.577+0000 7f0869ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:09:08.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.577+0000 7f0869ffb640 1 -- 172.21.15.116:0/1600396431 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c01c7d0 con 0x559f01a13370 2024-09-12T03:09:08.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.577+0000 7f0869ffb640 1 -- 172.21.15.116:0/1600396431 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f087c01ccf0 con 0x559f01a13370 2024-09-12T03:09:08.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.577+0000 7f088b05f740 1 -- 172.21.15.116:0/1600396431 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f087c01da80 con 0x559f01a13370 2024-09-12T03:09:08.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.578+0000 7f0869ffb640 1 -- 172.21.15.116:0/1600396431 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f087c004a00 con 0x559f01a13370 2024-09-12T03:09:08.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.578+0000 7f088b05f740 10 client.16866 fetch_fsmap learned FSMap version 25 2024-09-12T03:09:08.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.578+0000 7f088b05f740 1 -- 172.21.15.116:0/1600396431 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559f01a21a60 con 0x559f01a13370 2024-09-12T03:09:08.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.578+0000 7f0869ffb640 1 -- 172.21.15.116:0/1600396431 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f087c01da80 con 0x559f01a13370 2024-09-12T03:09:08.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.578+0000 7f0869ffb640 1 -- 172.21.15.116:0/1600396431 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f087c00f040 con 0x559f01a13370 2024-09-12T03:09:08.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.578+0000 7f088b05f740 10 client.16866 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:09:08.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.578+0000 7f088b05f740 10 client.16866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:09:08.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.578+0000 7f088b05f740 20 client.16866 populate_metadata read hostname 'smithi116' 2024-09-12T03:09:08.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.578+0000 7f088b05f740 1 --2- 172.21.15.116:0/1600396431 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559f01a22e60 0x559f01a25250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:08.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.578+0000 7f088b05f740 4 client.16866 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:09:08.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.578+0000 7f088b05f740 1 -- 172.21.15.116:0/1600396431 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f0197b860 con 0x559f01a22e60 2024-09-12T03:09:08.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.578+0000 7f088259c640 1 --2- 172.21.15.116:0/1600396431 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559f01a22e60 0x559f01a25250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:08.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.579+0000 7f088259c640 1 --2- 172.21.15.116:0/1600396431 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559f01a22e60 0x559f01a25250 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6a2f5c3aee225c8 server_cookie=153a046bd308e2a5 in_seq=0 out_seq=0 2024-09-12T03:09:08.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.579+0000 7f0869ffb640 10 client.16866 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:08.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.579+0000 7f0869ffb640 1 -- 172.21.15.116:0/1600396431 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559f0197b860 con 0x559f01a22e60 2024-09-12T03:09:08.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.579+0000 7f0869ffb640 10 client.16866 handle_command_reply: tid=0 2024-09-12T03:09:08.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.579+0000 7f088b05f740 1 client.16866 shutdown 2024-09-12T03:09:08.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.580+0000 7f088b05f740 1 -- 172.21.15.116:0/1600396431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f01a13370 msgr2=0x559f01a105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:08.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.580+0000 7f088b05f740 1 --2- 172.21.15.116:0/1600396431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f01a13370 0x559f01a105d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f087c011010 tx=0x7f087c01c4e0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.580+0000 7f088b05f740 1 -- 172.21.15.116:0/1600396431 shutdown_connections 2024-09-12T03:09:08.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.580+0000 7f088b05f740 1 --2- 172.21.15.116:0/1600396431 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559f01a22e60 0x559f01a25250 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.580+0000 7f088b05f740 1 --2- 172.21.15.116:0/1600396431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f01a13370 0x559f01a105d0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.580+0000 7f088b05f740 1 --2- 172.21.15.116:0/1600396431 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f01a09b40 0x559f01a10000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.580+0000 7f088b05f740 1 --2- 172.21.15.116:0/1600396431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f01a085d0 0x559f01a12d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.580+0000 7f0869ffb640 0 client.16866 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:08.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.580+0000 7f088b05f740 1 -- 172.21.15.116:0/1600396431 >> 172.21.15.116:0/1600396431 conn(0x559f019d1310 msgr2=0x559f01a0a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:08.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.580+0000 7f088b05f740 1 -- 172.21.15.116:0/1600396431 shutdown_connections 2024-09-12T03:09:08.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.580+0000 7f088b05f740 1 -- 172.21.15.116:0/1600396431 wait complete. 2024-09-12T03:09:08.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.580+0000 7f088b05f740 20 client.16866 trim_cache size 0 max 16384 2024-09-12T03:09:08.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.592+0000 7f088b05f740 1 Processor -- start 2024-09-12T03:09:08.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.592+0000 7f088b05f740 1 -- start start 2024-09-12T03:09:08.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.593+0000 7f088b05f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f01a085d0 0x559f01a1f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:08.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.593+0000 7f088b05f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f01a09b40 0x559f01a0efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:08.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.593+0000 7f088b05f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f01a13370 0x559f01a2aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:08.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.593+0000 7f088b05f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f08841afe50 con 0x559f01a09b40 2024-09-12T03:09:08.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.593+0000 7f088b05f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f08841ad720 con 0x559f01a13370 2024-09-12T03:09:08.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.593+0000 7f088b05f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f08841ab570 con 0x559f01a085d0 2024-09-12T03:09:08.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.593+0000 7f0879d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f01a09b40 0x559f01a0efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:08.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.593+0000 7f0879d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f01a09b40 0x559f01a0efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56418/0 (socket says 172.21.15.116:56418) 2024-09-12T03:09:08.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.593+0000 7f0879d9b640 1 -- 172.21.15.116:0/755708807 learned_addr learned my addr 172.21.15.116:0/755708807 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:08.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.593+0000 7f0882d9d640 1 --2- 172.21.15.116:0/755708807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f01a13370 0x559f01a2aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:08.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.593+0000 7f0879d9b640 1 -- 172.21.15.116:0/755708807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f01a085d0 msgr2=0x559f01a1f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:08.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.593+0000 7f0879d9b640 1 --2- 172.21.15.116:0/755708807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f01a085d0 0x559f01a1f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.593+0000 7f0879d9b640 1 -- 172.21.15.116:0/755708807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f01a13370 msgr2=0x559f01a2aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:08.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.593+0000 7f0879d9b640 1 --2- 172.21.15.116:0/755708807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f01a13370 0x559f01a2aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.593+0000 7f0879d9b640 1 -- 172.21.15.116:0/755708807 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f01a25880 con 0x559f01a09b40 2024-09-12T03:09:08.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.593+0000 7f0879d9b640 1 --2- 172.21.15.116:0/755708807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f01a09b40 0x559f01a0efe0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f086c0cc330 tx=0x7f086c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a42f06cd88441b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:08.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.593+0000 7f086bfff640 1 -- 172.21.15.116:0/755708807 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c01e040 con 0x559f01a09b40 2024-09-12T03:09:08.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.593+0000 7f086bfff640 1 -- 172.21.15.116:0/755708807 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f086c09b420 con 0x559f01a09b40 2024-09-12T03:09:08.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.593+0000 7f086bfff640 1 -- 172.21.15.116:0/755708807 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c010ca0 con 0x559f01a09b40 2024-09-12T03:09:08.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.594+0000 7f088b05f740 1 -- 172.21.15.116:0/755708807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f01a09b40 msgr2=0x559f01a0efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:08.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.594+0000 7f088b05f740 1 --2- 172.21.15.116:0/755708807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f01a09b40 0x559f01a0efe0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f086c0cc330 tx=0x7f086c0047d0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.594+0000 7f088b05f740 1 -- 172.21.15.116:0/755708807 shutdown_connections 2024-09-12T03:09:08.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.594+0000 7f088b05f740 1 --2- 172.21.15.116:0/755708807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f01a13370 0x559f01a2aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.594+0000 7f088b05f740 1 --2- 172.21.15.116:0/755708807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f01a09b40 0x559f01a0efe0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.594+0000 7f088b05f740 1 --2- 172.21.15.116:0/755708807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f01a085d0 0x559f01a1f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.594+0000 7f088b05f740 1 -- 172.21.15.116:0/755708807 >> 172.21.15.116:0/755708807 conn(0x559f019d1310 msgr2=0x559f01a0e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:08.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.594+0000 7f088b05f740 1 -- 172.21.15.116:0/755708807 shutdown_connections 2024-09-12T03:09:08.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.594+0000 7f088b05f740 1 -- 172.21.15.116:0/755708807 wait complete. 2024-09-12T03:09:08.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.594+0000 7f088b05f740 1 -- start start 2024-09-12T03:09:08.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.594+0000 7f088b05f740 1 Processor -- start 2024-09-12T03:09:08.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.595+0000 7f088b05f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f01a085d0 0x559f01a169e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:08.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.595+0000 7f088b05f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f01a09b40 0x559f01a16f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:08.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.595+0000 7f088b05f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f01a13370 0x559f01a8ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:08.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.595+0000 7f088b05f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f08841abbe0 con 0x559f01a085d0 2024-09-12T03:09:08.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.595+0000 7f088b05f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f08841ad050 con 0x559f01a09b40 2024-09-12T03:09:08.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.595+0000 7f088b05f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f08841ac2b0 con 0x559f01a13370 2024-09-12T03:09:08.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.595+0000 7f088b05f740 10 client.0 _ll_register_callbacks cb 0x7f087c020020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:08.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.595+0000 7f088259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f01a085d0 0x559f01a169e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:08.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.595+0000 7f088259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f01a085d0 0x559f01a169e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56432/0 (socket says 172.21.15.116:56432) 2024-09-12T03:09:08.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.595+0000 7f088259c640 1 -- 172.21.15.116:0/2007241898 learned_addr learned my addr 172.21.15.116:0/2007241898 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:08.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.595+0000 7f0879d9b640 1 --2- 172.21.15.116:0/2007241898 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f01a09b40 0x559f01a16f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:08.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.595+0000 7f088259c640 1 -- 172.21.15.116:0/2007241898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f01a13370 msgr2=0x559f01a8ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:08.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.595+0000 7f088259c640 1 --2- 172.21.15.116:0/2007241898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f01a13370 0x559f01a8ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.595+0000 7f088259c640 1 -- 172.21.15.116:0/2007241898 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f01a09b40 msgr2=0x559f01a16f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:08.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.595+0000 7f088259c640 1 --2- 172.21.15.116:0/2007241898 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f01a09b40 0x559f01a16f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.595+0000 7f088259c640 1 -- 172.21.15.116:0/2007241898 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f01a8c060 con 0x559f01a085d0 2024-09-12T03:09:08.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.595+0000 7f0879d9b640 1 --2- 172.21.15.116:0/2007241898 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f01a09b40 0x559f01a16f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:09:08.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.595+0000 7f088259c640 1 --2- 172.21.15.116:0/2007241898 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f01a085d0 0x559f01a169e0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f087402a4c0 tx=0x7f08740044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7757775f416022c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:08.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.595+0000 7f0869ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:09:08.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.595+0000 7f0869ffb640 1 -- 172.21.15.116:0/2007241898 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874035040 con 0x559f01a085d0 2024-09-12T03:09:08.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.595+0000 7f0869ffb640 1 -- 172.21.15.116:0/2007241898 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f0874005450 con 0x559f01a085d0 2024-09-12T03:09:08.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.595+0000 7f088b05f740 1 -- 172.21.15.116:0/2007241898 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f086c09b420 con 0x559f01a085d0 2024-09-12T03:09:08.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.595+0000 7f0869ffb640 1 -- 172.21.15.116:0/2007241898 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0874005650 con 0x559f01a085d0 2024-09-12T03:09:08.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.595+0000 7f088b05f740 10 client.16872 fetch_fsmap learned FSMap version 25 2024-09-12T03:09:08.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.595+0000 7f088b05f740 1 -- 172.21.15.116:0/2007241898 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559f01a8d340 con 0x559f01a085d0 2024-09-12T03:09:08.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.596+0000 7f0869ffb640 1 -- 172.21.15.116:0/2007241898 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f086c09b420 con 0x559f01a085d0 2024-09-12T03:09:08.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.596+0000 7f0869ffb640 1 -- 172.21.15.116:0/2007241898 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0874003b10 con 0x559f01a085d0 2024-09-12T03:09:08.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.596+0000 7f088b05f740 10 client.16872 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:09:08.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.596+0000 7f088b05f740 10 client.16872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:09:08.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.596+0000 7f088b05f740 20 client.16872 populate_metadata read hostname 'smithi116' 2024-09-12T03:09:08.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.596+0000 7f088b05f740 1 --2- 172.21.15.116:0/2007241898 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559f01a8db10 0x559f01a8ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:08.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.596+0000 7f088b05f740 4 client.16872 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:09:08.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.596+0000 7f088b05f740 1 -- 172.21.15.116:0/2007241898 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559f01975d70 con 0x559f01a8db10 2024-09-12T03:09:08.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.596+0000 7f0879d9b640 1 --2- 172.21.15.116:0/2007241898 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559f01a8db10 0x559f01a8ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:08.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.596+0000 7f0879d9b640 1 --2- 172.21.15.116:0/2007241898 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559f01a8db10 0x559f01a8ff30 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ed3d5b08f992fc server_cookie=a2c01ee15b58c42b in_seq=0 out_seq=0 2024-09-12T03:09:08.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.596+0000 7f0869ffb640 10 client.16872 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:08.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.597+0000 7f0869ffb640 1 -- 172.21.15.116:0/2007241898 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559f01975d70 con 0x559f01a8db10 2024-09-12T03:09:08.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.597+0000 7f0869ffb640 10 client.16872 handle_command_reply: tid=0 2024-09-12T03:09:08.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.597+0000 7f088b05f740 1 client.16872 shutdown 2024-09-12T03:09:08.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.597+0000 7f088b05f740 1 -- 172.21.15.116:0/2007241898 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f01a085d0 msgr2=0x559f01a169e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:08.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.597+0000 7f088b05f740 1 --2- 172.21.15.116:0/2007241898 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f01a085d0 0x559f01a169e0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f087402a4c0 tx=0x7f08740044b0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.597+0000 7f088b05f740 1 -- 172.21.15.116:0/2007241898 shutdown_connections 2024-09-12T03:09:08.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.597+0000 7f088b05f740 1 --2- 172.21.15.116:0/2007241898 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559f01a8db10 0x559f01a8ff30 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.597+0000 7f088b05f740 1 --2- 172.21.15.116:0/2007241898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f01a13370 0x559f01a8ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.597+0000 7f0869ffb640 0 client.16872 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:08.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.597+0000 7f088b05f740 1 --2- 172.21.15.116:0/2007241898 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f01a09b40 0x559f01a16f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.597+0000 7f088b05f740 1 --2- 172.21.15.116:0/2007241898 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f01a085d0 0x559f01a169e0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.597+0000 7f088b05f740 1 -- 172.21.15.116:0/2007241898 >> 172.21.15.116:0/2007241898 conn(0x559f019d1310 msgr2=0x559f01a0f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:08.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.597+0000 7f088b05f740 1 -- 172.21.15.116:0/2007241898 shutdown_connections 2024-09-12T03:09:08.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.597+0000 7f088b05f740 1 -- 172.21.15.116:0/2007241898 wait complete. 2024-09-12T03:09:08.621 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:09:08.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.597+0000 7f088b05f740 20 client.16872 trim_cache size 0 max 16384 2024-09-12T03:09:08.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.597+0000 7f0888c86640 1 -- 172.21.15.116:0/364190109 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f08700a2b80 msgr2=0x7f08700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:08.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.597+0000 7f0888c86640 1 --2- 172.21.15.116:0/364190109 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f08700a2b80 0x7f08700a5040 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f087c021000 tx=0x7f087c01e000 comp rx=0 tx=0).stop 2024-09-12T03:09:08.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.597+0000 7f0888c86640 1 -- 172.21.15.116:0/364190109 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0884102570 msgr2=0x7f0884198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:08.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.597+0000 7f0888c86640 1 --2- 172.21.15.116:0/364190109 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0884102570 0x7f0884198af0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f086c00b4a0 tx=0x7f086c00b960 comp rx=0 tx=0).stop 2024-09-12T03:09:08.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.597+0000 7f0888c86640 1 -- 172.21.15.116:0/364190109 shutdown_connections 2024-09-12T03:09:08.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.597+0000 7f0888c86640 1 --2- 172.21.15.116:0/364190109 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f08700a2b80 0x7f08700a5040 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.597+0000 7f0888c86640 1 --2- 172.21.15.116:0/364190109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0884102ef0 0x7f088419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.597+0000 7f0888c86640 1 --2- 172.21.15.116:0/364190109 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0884102570 0x7f0884198af0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.597+0000 7f0888c86640 1 --2- 172.21.15.116:0/364190109 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0884100e20 0x7f08841985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:08.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.597+0000 7f0888c86640 1 -- 172.21.15.116:0/364190109 >> 172.21.15.116:0/364190109 conn(0x7f08840fc7d0 msgr2=0x7f088406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:08.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.598+0000 7f0888c86640 1 -- 172.21.15.116:0/364190109 shutdown_connections 2024-09-12T03:09:08.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:08.598+0000 7f0888c86640 1 -- 172.21.15.116:0/364190109 wait complete. 2024-09-12T03:09:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:08 smithi183 ceph-mon[35535]: pgmap v522: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-09-12T03:09:08.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:08 smithi177 ceph-mon[35305]: pgmap v522: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-09-12T03:09:10.406 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:09:10.407 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:09:10.665 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:09:10.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:10 smithi183 ceph-mon[35535]: pgmap v523: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-12T03:09:10.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:10 smithi177 ceph-mon[35305]: pgmap v523: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-12T03:09:10.785 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:10 smithi116 ceph-mon[31410]: pgmap v523: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-12T03:09:11.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.166+0000 7ff14ff44640 1 -- 172.21.15.116:0/1245550565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff148100e10 msgr2=0x7ff148104e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:11.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.166+0000 7ff14ff44640 1 --2- 172.21.15.116:0/1245550565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff148100e10 0x7ff148104e60 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff138009900 tx=0x7ff13802a410 comp rx=0 tx=0).stop 2024-09-12T03:09:11.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.166+0000 7ff14ff44640 1 -- 172.21.15.116:0/1245550565 shutdown_connections 2024-09-12T03:09:11.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.166+0000 7ff14ff44640 1 --2- 172.21.15.116:0/1245550565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff148105590 0x7ff14810d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.166+0000 7ff14ff44640 1 --2- 172.21.15.116:0/1245550565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff148100e10 0x7ff148104e60 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.166+0000 7ff14ff44640 1 --2- 172.21.15.116:0/1245550565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff148100460 0x7ff148100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.166+0000 7ff14ff44640 1 -- 172.21.15.116:0/1245550565 >> 172.21.15.116:0/1245550565 conn(0x7ff1480fbee0 msgr2=0x7ff1480fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:11.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.166+0000 7ff14ff44640 1 -- 172.21.15.116:0/1245550565 shutdown_connections 2024-09-12T03:09:11.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.166+0000 7ff14ff44640 1 -- 172.21.15.116:0/1245550565 wait complete. 2024-09-12T03:09:11.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.166+0000 7ff14ff44640 1 Processor -- start 2024-09-12T03:09:11.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.167+0000 7ff14ff44640 1 -- start start 2024-09-12T03:09:11.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.167+0000 7ff14ff44640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff148100460 0x7ff148198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:11.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.167+0000 7ff14ff44640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff148100e10 0x7ff1481989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:11.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.167+0000 7ff14ff44640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff148105590 0x7ff14819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:11.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.167+0000 7ff14ff44640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff148111f30 con 0x7ff148100460 2024-09-12T03:09:11.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.167+0000 7ff14ff44640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff148111db0 con 0x7ff148100e10 2024-09-12T03:09:11.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.167+0000 7ff14ff44640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481120b0 con 0x7ff148105590 2024-09-12T03:09:11.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.167+0000 7ff14dce0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff148100460 0x7ff148198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:11.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.167+0000 7ff14dce0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff148100460 0x7ff148198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50516/0 (socket says 172.21.15.116:50516) 2024-09-12T03:09:11.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.167+0000 7ff14dce0640 1 -- 172.21.15.116:0/1525174481 learned_addr learned my addr 172.21.15.116:0/1525174481 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:11.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.167+0000 7ff14e4e1640 1 --2- 172.21.15.116:0/1525174481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff148105590 0x7ff14819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:11.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.167+0000 7ff14dce0640 1 -- 172.21.15.116:0/1525174481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff148105590 msgr2=0x7ff14819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:11.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.167+0000 7ff14dce0640 1 --2- 172.21.15.116:0/1525174481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff148105590 0x7ff14819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.167+0000 7ff14dce0640 1 -- 172.21.15.116:0/1525174481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff148100e10 msgr2=0x7ff1481989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:11.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.167+0000 7ff14dce0640 1 --2- 172.21.15.116:0/1525174481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff148100e10 0x7ff1481989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.167+0000 7ff14dce0640 1 -- 172.21.15.116:0/1525174481 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff14819d420 con 0x7ff148100460 2024-09-12T03:09:11.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.167+0000 7ff14dce0640 1 --2- 172.21.15.116:0/1525174481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff148100460 0x7ff148198470 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff14400c920 tx=0x7ff14400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:11.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.168+0000 7ff13effd640 1 -- 172.21.15.116:0/1525174481 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff144007c70 con 0x7ff148100460 2024-09-12T03:09:11.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.168+0000 7ff13effd640 1 -- 172.21.15.116:0/1525174481 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff144007e10 con 0x7ff148100460 2024-09-12T03:09:11.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.168+0000 7ff14ff44640 1 -- 172.21.15.116:0/1525174481 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff14819d710 con 0x7ff148100460 2024-09-12T03:09:11.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.168+0000 7ff13effd640 1 -- 172.21.15.116:0/1525174481 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff144018890 con 0x7ff148100460 2024-09-12T03:09:11.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.168+0000 7ff14ff44640 1 -- 172.21.15.116:0/1525174481 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1481a4ff0 con 0x7ff148100460 2024-09-12T03:09:11.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.170+0000 7ff13effd640 1 -- 172.21.15.116:0/1525174481 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ff144018a70 con 0x7ff148100460 2024-09-12T03:09:11.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.170+0000 7ff15231d740 1 Processor -- start 2024-09-12T03:09:11.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.170+0000 7ff15231d740 1 -- start start 2024-09-12T03:09:11.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.171+0000 7ff15231d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56097452ab60 0x56097452af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:11.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.171+0000 7ff15231d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56097452b480 0x560974535800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:11.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.171+0000 7ff13effd640 1 --2- 172.21.15.116:0/1525174481 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff1340a2c00 0x7ff1340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:11.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.171+0000 7ff15231d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560974535d40 0x560974538130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:11.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.171+0000 7ff15231d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560974490280 con 0x560974535d40 2024-09-12T03:09:11.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.171+0000 7ff15231d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560974437b10 con 0x56097452b480 2024-09-12T03:09:11.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.171+0000 7ff15231d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56097448c240 con 0x56097452ab60 2024-09-12T03:09:11.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.171+0000 7ff14e4e1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560974535d40 0x560974538130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:11.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.171+0000 7ff13effd640 1 -- 172.21.15.116:0/1525174481 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff1440cf1e0 con 0x7ff148100460 2024-09-12T03:09:11.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.171+0000 7ff14e4e1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560974535d40 0x560974538130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50520/0 (socket says 172.21.15.116:50520) 2024-09-12T03:09:11.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.171+0000 7ff14e4e1640 1 -- 172.21.15.116:0/3102380019 learned_addr learned my addr 172.21.15.116:0/3102380019 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:11.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.171+0000 7ff14d4df640 1 --2- 172.21.15.116:0/1525174481 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff1340a2c00 0x7ff1340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:11.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.171+0000 7ff14e4e1640 1 --2- 172.21.15.116:0/3102380019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56097452ab60 0x56097452af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:11.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.171+0000 7ff14dce0640 1 --2- 172.21.15.116:0/3102380019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56097452b480 0x560974535800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:11.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.171+0000 7ff14e4e1640 1 -- 172.21.15.116:0/3102380019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56097452ab60 msgr2=0x56097452af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:11.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.171+0000 7ff14e4e1640 1 --2- 172.21.15.116:0/3102380019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56097452ab60 0x56097452af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.171+0000 7ff14e4e1640 1 -- 172.21.15.116:0/3102380019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56097452b480 msgr2=0x560974535800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:11.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.171+0000 7ff14e4e1640 1 --2- 172.21.15.116:0/3102380019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56097452b480 0x560974535800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.171+0000 7ff14e4e1640 1 -- 172.21.15.116:0/3102380019 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609744441f0 con 0x560974535d40 2024-09-12T03:09:11.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.171+0000 7ff14dce0640 1 --2- 172.21.15.116:0/3102380019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56097452b480 0x560974535800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:09:11.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.171+0000 7ff14d4df640 1 --2- 172.21.15.116:0/1525174481 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff1340a2c00 0x7ff1340a50c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff13802a930 tx=0x7ff138000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:11.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.171+0000 7ff14e4e1640 1 --2- 172.21.15.116:0/3102380019 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560974535d40 0x560974538130 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff140013a80 tx=0x7ff14000a590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a9765cce909777 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:11.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.172+0000 7ff133fff640 1 -- 172.21.15.116:0/3102380019 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140004b20 con 0x560974535d40 2024-09-12T03:09:11.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.172+0000 7ff133fff640 1 -- 172.21.15.116:0/3102380019 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff14000ee40 con 0x560974535d40 2024-09-12T03:09:11.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.172+0000 7ff133fff640 1 -- 172.21.15.116:0/3102380019 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140005720 con 0x560974535d40 2024-09-12T03:09:11.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.172+0000 7ff15231d740 1 -- 172.21.15.116:0/3102380019 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560974535d40 msgr2=0x560974538130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:11.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.172+0000 7ff15231d740 1 --2- 172.21.15.116:0/3102380019 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560974535d40 0x560974538130 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff140013a80 tx=0x7ff14000a590 comp rx=0 tx=0).stop 2024-09-12T03:09:11.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.172+0000 7ff15231d740 1 -- 172.21.15.116:0/3102380019 shutdown_connections 2024-09-12T03:09:11.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.172+0000 7ff15231d740 1 --2- 172.21.15.116:0/3102380019 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560974535d40 0x560974538130 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.172+0000 7ff15231d740 1 --2- 172.21.15.116:0/3102380019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56097452b480 0x560974535800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.172+0000 7ff15231d740 1 --2- 172.21.15.116:0/3102380019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56097452ab60 0x56097452af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.172+0000 7ff15231d740 1 -- 172.21.15.116:0/3102380019 >> 172.21.15.116:0/3102380019 conn(0x5609745270f0 msgr2=0x5609745274d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:11.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.173+0000 7ff15231d740 1 -- 172.21.15.116:0/3102380019 shutdown_connections 2024-09-12T03:09:11.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.173+0000 7ff15231d740 1 -- 172.21.15.116:0/3102380019 wait complete. 2024-09-12T03:09:11.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.173+0000 7ff15231d740 1 -- start start 2024-09-12T03:09:11.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.173+0000 7ff15231d740 1 Processor -- start 2024-09-12T03:09:11.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.173+0000 7ff15231d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56097452ab60 0x560974532c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:11.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.173+0000 7ff15231d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56097452b480 0x560974533180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:11.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.173+0000 7ff15231d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560974535d40 0x5609745336c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:11.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.173+0000 7ff15231d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560974538910 con 0x560974535d40 2024-09-12T03:09:11.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.173+0000 7ff15231d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56097444af10 con 0x56097452b480 2024-09-12T03:09:11.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.173+0000 7ff15231d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560974538a90 con 0x56097452ab60 2024-09-12T03:09:11.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.173+0000 7ff15231d740 10 client.0 _ll_register_callbacks cb 0x560974413a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:11.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.173+0000 7ff14e4e1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560974535d40 0x5609745336c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:11.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.173+0000 7ff14e4e1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560974535d40 0x5609745336c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50534/0 (socket says 172.21.15.116:50534) 2024-09-12T03:09:11.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.173+0000 7ff14e4e1640 1 -- 172.21.15.116:0/539568038 learned_addr learned my addr 172.21.15.116:0/539568038 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:11.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.174+0000 7ff14d4df640 1 --2- 172.21.15.116:0/539568038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56097452b480 0x560974533180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:11.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.174+0000 7ff14dce0640 1 --2- 172.21.15.116:0/539568038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56097452ab60 0x560974532c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:11.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.174+0000 7ff14e4e1640 1 -- 172.21.15.116:0/539568038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56097452ab60 msgr2=0x560974532c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:11.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.174+0000 7ff14e4e1640 1 --2- 172.21.15.116:0/539568038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56097452ab60 0x560974532c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.174+0000 7ff14e4e1640 1 -- 172.21.15.116:0/539568038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56097452b480 msgr2=0x560974533180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:11.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.174+0000 7ff14e4e1640 1 --2- 172.21.15.116:0/539568038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56097452b480 0x560974533180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.174+0000 7ff14e4e1640 1 -- 172.21.15.116:0/539568038 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609742016a0 con 0x560974535d40 2024-09-12T03:09:11.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.174+0000 7ff14dce0640 1 --2- 172.21.15.116:0/539568038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56097452ab60 0x560974532c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:09:11.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.174+0000 7ff14e4e1640 1 --2- 172.21.15.116:0/539568038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560974535d40 0x5609745336c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff14000f010 tx=0x7ff14000a450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c05c4922d5b9be server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:11.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.174+0000 7ff1317fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:09:11.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.174+0000 7ff1317fa640 1 -- 172.21.15.116:0/539568038 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14000ec30 con 0x560974535d40 2024-09-12T03:09:11.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.174+0000 7ff15231d740 1 -- 172.21.15.116:0/539568038 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff14000ee40 con 0x560974535d40 2024-09-12T03:09:11.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.174+0000 7ff1317fa640 1 -- 172.21.15.116:0/539568038 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff140004a40 con 0x560974535d40 2024-09-12T03:09:11.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.174+0000 7ff1317fa640 1 -- 172.21.15.116:0/539568038 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff140004c40 con 0x560974535d40 2024-09-12T03:09:11.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.174+0000 7ff15231d740 10 client.16884 fetch_fsmap learned FSMap version 25 2024-09-12T03:09:11.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.174+0000 7ff15231d740 1 -- 172.21.15.116:0/539568038 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560974544540 con 0x560974535d40 2024-09-12T03:09:11.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.174+0000 7ff1317fa640 1 -- 172.21.15.116:0/539568038 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff14000ee40 con 0x560974535d40 2024-09-12T03:09:11.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.175+0000 7ff1317fa640 1 -- 172.21.15.116:0/539568038 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff14001e8f0 con 0x560974535d40 2024-09-12T03:09:11.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.175+0000 7ff15231d740 10 client.16884 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:09:11.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.175+0000 7ff15231d740 10 client.16884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:09:11.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.175+0000 7ff15231d740 20 client.16884 populate_metadata read hostname 'smithi116' 2024-09-12T03:09:11.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.175+0000 7ff15231d740 1 --2- 172.21.15.116:0/539568038 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560974545890 0x560974547c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:11.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.175+0000 7ff15231d740 4 client.16884 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:09:11.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.175+0000 7ff15231d740 1 -- 172.21.15.116:0/539568038 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56097449d7f0 con 0x560974545890 2024-09-12T03:09:11.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.175+0000 7ff14dce0640 1 --2- 172.21.15.116:0/539568038 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560974545890 0x560974547c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:11.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.175+0000 7ff14dce0640 1 --2- 172.21.15.116:0/539568038 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560974545890 0x560974547c80 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f1491166232d250 server_cookie=e35e29c4e89b5ba8 in_seq=0 out_seq=0 2024-09-12T03:09:11.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.175+0000 7ff1317fa640 10 client.16884 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:11.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.176+0000 7ff1317fa640 1 -- 172.21.15.116:0/539568038 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56097449d7f0 con 0x560974545890 2024-09-12T03:09:11.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.176+0000 7ff1317fa640 10 client.16884 handle_command_reply: tid=0 2024-09-12T03:09:11.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.176+0000 7ff15231d740 1 client.16884 shutdown 2024-09-12T03:09:11.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.176+0000 7ff15231d740 1 -- 172.21.15.116:0/539568038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560974535d40 msgr2=0x5609745336c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:11.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.176+0000 7ff15231d740 1 --2- 172.21.15.116:0/539568038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560974535d40 0x5609745336c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff14000f010 tx=0x7ff14000a450 comp rx=0 tx=0).stop 2024-09-12T03:09:11.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.176+0000 7ff15231d740 1 -- 172.21.15.116:0/539568038 shutdown_connections 2024-09-12T03:09:11.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.176+0000 7ff15231d740 1 --2- 172.21.15.116:0/539568038 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560974545890 0x560974547c80 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.176+0000 7ff15231d740 1 --2- 172.21.15.116:0/539568038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560974535d40 0x5609745336c0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.176+0000 7ff15231d740 1 --2- 172.21.15.116:0/539568038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56097452b480 0x560974533180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.176+0000 7ff15231d740 1 --2- 172.21.15.116:0/539568038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56097452ab60 0x560974532c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.176+0000 7ff1317fa640 0 client.16884 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:11.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.176+0000 7ff15231d740 1 -- 172.21.15.116:0/539568038 >> 172.21.15.116:0/539568038 conn(0x5609745270f0 msgr2=0x56097452cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:11.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.176+0000 7ff15231d740 1 -- 172.21.15.116:0/539568038 shutdown_connections 2024-09-12T03:09:11.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.176+0000 7ff15231d740 1 -- 172.21.15.116:0/539568038 wait complete. 2024-09-12T03:09:11.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.177+0000 7ff15231d740 20 client.16884 trim_cache size 0 max 16384 2024-09-12T03:09:11.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.194+0000 7ff15231d740 1 Processor -- start 2024-09-12T03:09:11.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.194+0000 7ff15231d740 1 -- start start 2024-09-12T03:09:11.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.194+0000 7ff15231d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56097452ab60 0x560974531af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:11.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.194+0000 7ff15231d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56097452b480 0x56097454d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:11.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.194+0000 7ff15231d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560974535d40 0x56097454d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:11.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.194+0000 7ff15231d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481afd10 con 0x56097452b480 2024-09-12T03:09:11.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.194+0000 7ff15231d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481ad5e0 con 0x56097452ab60 2024-09-12T03:09:11.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.194+0000 7ff15231d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481ab430 con 0x560974535d40 2024-09-12T03:09:11.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.194+0000 7ff14d4df640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56097452b480 0x56097454d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:11.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.194+0000 7ff14d4df640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56097452b480 0x56097454d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50548/0 (socket says 172.21.15.116:50548) 2024-09-12T03:09:11.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.194+0000 7ff14d4df640 1 -- 172.21.15.116:0/3003651088 learned_addr learned my addr 172.21.15.116:0/3003651088 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:11.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.194+0000 7ff14dce0640 1 --2- 172.21.15.116:0/3003651088 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56097452ab60 0x560974531af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:11.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.194+0000 7ff14e4e1640 1 --2- 172.21.15.116:0/3003651088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560974535d40 0x56097454d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:11.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.194+0000 7ff14d4df640 1 -- 172.21.15.116:0/3003651088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560974535d40 msgr2=0x56097454d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:11.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.194+0000 7ff14d4df640 1 --2- 172.21.15.116:0/3003651088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560974535d40 0x56097454d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.194+0000 7ff14d4df640 1 -- 172.21.15.116:0/3003651088 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56097452ab60 msgr2=0x560974531af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:11.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.194+0000 7ff14d4df640 1 --2- 172.21.15.116:0/3003651088 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56097452ab60 0x560974531af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.194+0000 7ff14d4df640 1 -- 172.21.15.116:0/3003651088 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560974535220 con 0x56097452b480 2024-09-12T03:09:11.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.195+0000 7ff14d4df640 1 --2- 172.21.15.116:0/3003651088 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56097452b480 0x56097454d3f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7ff138009900 tx=0x7ff13803a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7662a47832aef603 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:11.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.195+0000 7ff1337fe640 1 -- 172.21.15.116:0/3003651088 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13804c070 con 0x56097452b480 2024-09-12T03:09:11.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.195+0000 7ff1337fe640 1 -- 172.21.15.116:0/3003651088 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1380035f0 con 0x56097452b480 2024-09-12T03:09:11.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.195+0000 7ff1337fe640 1 -- 172.21.15.116:0/3003651088 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff138035040 con 0x56097452b480 2024-09-12T03:09:11.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.195+0000 7ff15231d740 1 -- 172.21.15.116:0/3003651088 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56097452b480 msgr2=0x56097454d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:11.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.195+0000 7ff15231d740 1 --2- 172.21.15.116:0/3003651088 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56097452b480 0x56097454d3f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7ff138009900 tx=0x7ff13803a000 comp rx=0 tx=0).stop 2024-09-12T03:09:11.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.195+0000 7ff15231d740 1 -- 172.21.15.116:0/3003651088 shutdown_connections 2024-09-12T03:09:11.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.195+0000 7ff15231d740 1 --2- 172.21.15.116:0/3003651088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560974535d40 0x56097454d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.195+0000 7ff15231d740 1 --2- 172.21.15.116:0/3003651088 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56097452b480 0x56097454d3f0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.195+0000 7ff15231d740 1 --2- 172.21.15.116:0/3003651088 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56097452ab60 0x560974531af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.195+0000 7ff15231d740 1 -- 172.21.15.116:0/3003651088 >> 172.21.15.116:0/3003651088 conn(0x5609745270f0 msgr2=0x5609745313e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:11.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.196+0000 7ff15231d740 1 -- 172.21.15.116:0/3003651088 shutdown_connections 2024-09-12T03:09:11.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.196+0000 7ff15231d740 1 -- 172.21.15.116:0/3003651088 wait complete. 2024-09-12T03:09:11.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.196+0000 7ff15231d740 1 -- start start 2024-09-12T03:09:11.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.196+0000 7ff15231d740 1 Processor -- start 2024-09-12T03:09:11.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.196+0000 7ff15231d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56097452ab60 0x560974539ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:11.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.196+0000 7ff15231d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56097452b480 0x56097453a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:11.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.196+0000 7ff15231d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560974535d40 0x56097453a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:11.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.196+0000 7ff15231d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481abaa0 con 0x56097452b480 2024-09-12T03:09:11.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.196+0000 7ff15231d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481acf10 con 0x56097452ab60 2024-09-12T03:09:11.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.196+0000 7ff15231d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481ac170 con 0x560974535d40 2024-09-12T03:09:11.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.196+0000 7ff15231d740 10 client.0 _ll_register_callbacks cb 0x7ff14400a560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:11.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.196+0000 7ff14d4df640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56097452b480 0x56097453a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:11.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.197+0000 7ff14d4df640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56097452b480 0x56097453a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50556/0 (socket says 172.21.15.116:50556) 2024-09-12T03:09:11.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.197+0000 7ff14d4df640 1 -- 172.21.15.116:0/1285250625 learned_addr learned my addr 172.21.15.116:0/1285250625 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:11.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.197+0000 7ff14e4e1640 1 --2- 172.21.15.116:0/1285250625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560974535d40 0x56097453a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:11.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.197+0000 7ff14dce0640 1 --2- 172.21.15.116:0/1285250625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56097452ab60 0x560974539ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:11.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.197+0000 7ff14d4df640 1 -- 172.21.15.116:0/1285250625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560974535d40 msgr2=0x56097453a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:11.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.197+0000 7ff14d4df640 1 --2- 172.21.15.116:0/1285250625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560974535d40 0x56097453a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.197+0000 7ff14d4df640 1 -- 172.21.15.116:0/1285250625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56097452ab60 msgr2=0x560974539ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:11.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.197+0000 7ff14d4df640 1 --2- 172.21.15.116:0/1285250625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56097452ab60 0x560974539ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.197+0000 7ff14d4df640 1 -- 172.21.15.116:0/1285250625 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609745af660 con 0x56097452b480 2024-09-12T03:09:11.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.197+0000 7ff14dce0640 1 --2- 172.21.15.116:0/1285250625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56097452ab60 0x560974539ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:09:11.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.197+0000 7ff14e4e1640 1 --2- 172.21.15.116:0/1285250625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560974535d40 0x56097453a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:09:11.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.197+0000 7ff14d4df640 1 --2- 172.21.15.116:0/1285250625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56097452b480 0x56097453a220 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff1380096c0 tx=0x7ff138031960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ef290ad922e6be server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:11.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.197+0000 7ff1317fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:09:11.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.197+0000 7ff1317fa640 1 -- 172.21.15.116:0/1285250625 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13804c070 con 0x56097452b480 2024-09-12T03:09:11.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.197+0000 7ff1317fa640 1 -- 172.21.15.116:0/1285250625 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff1380037a0 con 0x56097452b480 2024-09-12T03:09:11.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.197+0000 7ff15231d740 1 -- 172.21.15.116:0/1285250625 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1380035f0 con 0x56097452b480 2024-09-12T03:09:11.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.197+0000 7ff1317fa640 1 -- 172.21.15.116:0/1285250625 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff138035040 con 0x56097452b480 2024-09-12T03:09:11.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.197+0000 7ff15231d740 10 client.16890 fetch_fsmap learned FSMap version 25 2024-09-12T03:09:11.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.197+0000 7ff15231d740 1 -- 172.21.15.116:0/1285250625 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5609745b0980 con 0x56097452b480 2024-09-12T03:09:11.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.198+0000 7ff1317fa640 1 -- 172.21.15.116:0/1285250625 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1380035f0 con 0x56097452b480 2024-09-12T03:09:11.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.198+0000 7ff1317fa640 1 -- 172.21.15.116:0/1285250625 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff13805ac40 con 0x56097452b480 2024-09-12T03:09:11.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.198+0000 7ff15231d740 10 client.16890 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:09:11.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.198+0000 7ff15231d740 10 client.16890 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:09:11.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.198+0000 7ff15231d740 20 client.16890 populate_metadata read hostname 'smithi116' 2024-09-12T03:09:11.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.198+0000 7ff15231d740 1 --2- 172.21.15.116:0/1285250625 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5609745b10a0 0x5609745b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:11.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.198+0000 7ff15231d740 4 client.16890 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:09:11.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.198+0000 7ff15231d740 1 -- 172.21.15.116:0/1285250625 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560974497d00 con 0x5609745b10a0 2024-09-12T03:09:11.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.198+0000 7ff14dce0640 1 --2- 172.21.15.116:0/1285250625 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5609745b10a0 0x5609745b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:11.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.199+0000 7ff14dce0640 1 --2- 172.21.15.116:0/1285250625 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5609745b10a0 0x5609745b34c0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6991a9900de16828 server_cookie=4a18073836a510c9 in_seq=0 out_seq=0 2024-09-12T03:09:11.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.199+0000 7ff1317fa640 10 client.16890 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:11.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.200+0000 7ff1317fa640 1 -- 172.21.15.116:0/1285250625 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560974497d00 con 0x5609745b10a0 2024-09-12T03:09:11.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.200+0000 7ff1317fa640 10 client.16890 handle_command_reply: tid=0 2024-09-12T03:09:11.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.200+0000 7ff15231d740 1 client.16890 shutdown 2024-09-12T03:09:11.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.200+0000 7ff15231d740 1 -- 172.21.15.116:0/1285250625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56097452b480 msgr2=0x56097453a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:11.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.200+0000 7ff15231d740 1 --2- 172.21.15.116:0/1285250625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56097452b480 0x56097453a220 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff1380096c0 tx=0x7ff138031960 comp rx=0 tx=0).stop 2024-09-12T03:09:11.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.200+0000 7ff15231d740 1 -- 172.21.15.116:0/1285250625 shutdown_connections 2024-09-12T03:09:11.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.200+0000 7ff15231d740 1 --2- 172.21.15.116:0/1285250625 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5609745b10a0 0x5609745b34c0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.200+0000 7ff15231d740 1 --2- 172.21.15.116:0/1285250625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560974535d40 0x56097453a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.200+0000 7ff15231d740 1 --2- 172.21.15.116:0/1285250625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56097452b480 0x56097453a220 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.200+0000 7ff1317fa640 0 client.16890 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:11.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.200+0000 7ff15231d740 1 --2- 172.21.15.116:0/1285250625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56097452ab60 0x560974539ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.200+0000 7ff15231d740 1 -- 172.21.15.116:0/1285250625 >> 172.21.15.116:0/1285250625 conn(0x5609745270f0 msgr2=0x560974533400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:11.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.200+0000 7ff15231d740 1 -- 172.21.15.116:0/1285250625 shutdown_connections 2024-09-12T03:09:11.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.200+0000 7ff15231d740 1 -- 172.21.15.116:0/1285250625 wait complete. 2024-09-12T03:09:11.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.200+0000 7ff15231d740 20 client.16890 trim_cache size 0 max 16384 2024-09-12T03:09:11.210 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:09:11.210 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:09:11.210 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "0aa2260a-9cf7-4401-95d5-83104ab2700c", 2024-09-12T03:09:11.210 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:09:11.211 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:09:11.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.201+0000 7ff14ff44640 1 -- 172.21.15.116:0/1525174481 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff1340a2c00 msgr2=0x7ff1340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:11.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.201+0000 7ff14ff44640 1 --2- 172.21.15.116:0/1525174481 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff1340a2c00 0x7ff1340a50c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff13802a930 tx=0x7ff138000bc0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.201+0000 7ff14ff44640 1 -- 172.21.15.116:0/1525174481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff148100460 msgr2=0x7ff148198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:11.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.201+0000 7ff14ff44640 1 --2- 172.21.15.116:0/1525174481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff148100460 0x7ff148198470 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff14400c920 tx=0x7ff14400cde0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.201+0000 7ff14ff44640 1 -- 172.21.15.116:0/1525174481 shutdown_connections 2024-09-12T03:09:11.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.201+0000 7ff14ff44640 1 --2- 172.21.15.116:0/1525174481 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff1340a2c00 0x7ff1340a50c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.201+0000 7ff14ff44640 1 --2- 172.21.15.116:0/1525174481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff148105590 0x7ff14819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.201+0000 7ff14ff44640 1 --2- 172.21.15.116:0/1525174481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff148100e10 0x7ff1481989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.201+0000 7ff14ff44640 1 --2- 172.21.15.116:0/1525174481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff148100460 0x7ff148198470 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:11.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.201+0000 7ff14ff44640 1 -- 172.21.15.116:0/1525174481 >> 172.21.15.116:0/1525174481 conn(0x7ff1480fbee0 msgr2=0x7ff1480fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:11.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.201+0000 7ff14ff44640 1 -- 172.21.15.116:0/1525174481 shutdown_connections 2024-09-12T03:09:11.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:11.201+0000 7ff14ff44640 1 -- 172.21.15.116:0/1525174481 wait complete. 2024-09-12T03:09:12.090 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:09:12.339 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:09:12.623 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:12 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:09:12.623 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:12 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:09:12.623 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:12 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:09:12.623 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:12 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:09:12.623 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:12 smithi116 ceph-mon[31410]: pgmap v524: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-12T03:09:12.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:12 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:09:12.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:12 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:09:12.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:12 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:09:12.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:12 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:09:12.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:12 smithi183 ceph-mon[35535]: pgmap v524: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-12T03:09:12.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:12 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:09:12.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:12 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:09:12.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:12 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:09:12.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:12 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:09:12.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:12 smithi177 ceph-mon[35305]: pgmap v524: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-12T03:09:12.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.839+0000 7f1615b6d640 1 -- 172.21.15.116:0/2624171528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1610100e20 msgr2=0x7f1610101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:12.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.839+0000 7f1615b6d640 1 --2- 172.21.15.116:0/2624171528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1610100e20 0x7f1610101200 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f1600009900 tx=0x7f160002a480 comp rx=0 tx=0).stop 2024-09-12T03:09:12.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.839+0000 7f1615b6d640 1 -- 172.21.15.116:0/2624171528 shutdown_connections 2024-09-12T03:09:12.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.839+0000 7f1615b6d640 1 --2- 172.21.15.116:0/2624171528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1610102ef0 0x7f161010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.839+0000 7f1615b6d640 1 --2- 172.21.15.116:0/2624171528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1610102570 0x7f16101029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.839+0000 7f1615b6d640 1 --2- 172.21.15.116:0/2624171528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1610100e20 0x7f1610101200 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.839+0000 7f1615b6d640 1 -- 172.21.15.116:0/2624171528 >> 172.21.15.116:0/2624171528 conn(0x7f16100fc7d0 msgr2=0x7f16100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:12.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.839+0000 7f1615b6d640 1 -- 172.21.15.116:0/2624171528 shutdown_connections 2024-09-12T03:09:12.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.839+0000 7f1615b6d640 1 -- 172.21.15.116:0/2624171528 wait complete. 2024-09-12T03:09:12.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.839+0000 7f1615b6d640 1 Processor -- start 2024-09-12T03:09:12.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.840+0000 7f1615b6d640 1 -- start start 2024-09-12T03:09:12.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.840+0000 7f1615b6d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1610100e20 0x7f1610198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:12.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.840+0000 7f1615b6d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1610102570 0x7f1610198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:12.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.840+0000 7f1615b6d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1610102ef0 0x7f161019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:12.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.840+0000 7f1615b6d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f16101120c0 con 0x7f1610100e20 2024-09-12T03:09:12.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.840+0000 7f1615b6d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1610111f40 con 0x7f1610102ef0 2024-09-12T03:09:12.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.840+0000 7f1615b6d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1610112240 con 0x7f1610102570 2024-09-12T03:09:12.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.840+0000 7f160f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1610100e20 0x7f1610198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:12.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.840+0000 7f160f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1610100e20 0x7f1610198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50566/0 (socket says 172.21.15.116:50566) 2024-09-12T03:09:12.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.840+0000 7f160f7fe640 1 -- 172.21.15.116:0/4174611733 learned_addr learned my addr 172.21.15.116:0/4174611733 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:12.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.840+0000 7f1606ffd640 1 --2- 172.21.15.116:0/4174611733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1610102570 0x7f1610198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:12.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.840+0000 7f160ffff640 1 --2- 172.21.15.116:0/4174611733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1610102ef0 0x7f161019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:12.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.840+0000 7f160f7fe640 1 -- 172.21.15.116:0/4174611733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1610102570 msgr2=0x7f1610198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:12.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.840+0000 7f160f7fe640 1 --2- 172.21.15.116:0/4174611733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1610102570 0x7f1610198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.840+0000 7f160f7fe640 1 -- 172.21.15.116:0/4174611733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1610102ef0 msgr2=0x7f161019ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:12.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.840+0000 7f160f7fe640 1 --2- 172.21.15.116:0/4174611733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1610102ef0 0x7f161019ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.840+0000 7f160f7fe640 1 -- 172.21.15.116:0/4174611733 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f161019d4e0 con 0x7f1610100e20 2024-09-12T03:09:12.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.841+0000 7f160f7fe640 1 --2- 172.21.15.116:0/4174611733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1610100e20 0x7f1610198530 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f1600006010 tx=0x7f1600003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:12.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.841+0000 7f160d7fa640 1 -- 172.21.15.116:0/4174611733 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160003f070 con 0x7f1610100e20 2024-09-12T03:09:12.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.841+0000 7f160d7fa640 1 -- 172.21.15.116:0/4174611733 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1600031c30 con 0x7f1610100e20 2024-09-12T03:09:12.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.841+0000 7f1615b6d640 1 -- 172.21.15.116:0/4174611733 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f161019d770 con 0x7f1610100e20 2024-09-12T03:09:12.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.841+0000 7f160d7fa640 1 -- 172.21.15.116:0/4174611733 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1600035030 con 0x7f1610100e20 2024-09-12T03:09:12.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.841+0000 7f1615b6d640 1 -- 172.21.15.116:0/4174611733 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16101a50b0 con 0x7f1610100e20 2024-09-12T03:09:12.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.843+0000 7f160d7fa640 1 -- 172.21.15.116:0/4174611733 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1600042070 con 0x7f1610100e20 2024-09-12T03:09:12.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.843+0000 7f1617f46740 1 Processor -- start 2024-09-12T03:09:12.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.843+0000 7f1617f46740 1 -- start start 2024-09-12T03:09:12.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.843+0000 7f1617f46740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632de9c24a0 0x5632de9c2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:12.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.843+0000 7f1617f46740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632de9c2dc0 0x5632de9cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:12.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.843+0000 7f1617f46740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632de9cd680 0x5632de9cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:12.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.843+0000 7f1617f46740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5632de9282e0 con 0x5632de9cd680 2024-09-12T03:09:12.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.843+0000 7f1617f46740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5632de8cfce0 con 0x5632de9c24a0 2024-09-12T03:09:12.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.843+0000 7f1617f46740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5632de9242a0 con 0x5632de9c2dc0 2024-09-12T03:09:12.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.844+0000 7f1606ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632de9cd680 0x5632de9cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:12.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.844+0000 7f160ffff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632de9c24a0 0x5632de9c2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:12.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.844+0000 7f1606ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632de9cd680 0x5632de9cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50582/0 (socket says 172.21.15.116:50582) 2024-09-12T03:09:12.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.844+0000 7f1606ffd640 1 -- 172.21.15.116:0/1765411887 learned_addr learned my addr 172.21.15.116:0/1765411887 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:12.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.844+0000 7f160d7fa640 1 --2- 172.21.15.116:0/4174611733 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f15fc0a2c00 0x7f15fc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:12.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.844+0000 7f160f7fe640 1 --2- 172.21.15.116:0/1765411887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632de9c2dc0 0x5632de9cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:12.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.844+0000 7f1606ffd640 1 -- 172.21.15.116:0/1765411887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632de9c2dc0 msgr2=0x5632de9cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:12.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.844+0000 7f1606ffd640 1 --2- 172.21.15.116:0/1765411887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632de9c2dc0 0x5632de9cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.844+0000 7f1606ffd640 1 -- 172.21.15.116:0/1765411887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632de9c24a0 msgr2=0x5632de9c2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:12.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.844+0000 7f1606ffd640 1 --2- 172.21.15.116:0/1765411887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632de9c24a0 0x5632de9c2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.844+0000 7f1606ffd640 1 -- 172.21.15.116:0/1765411887 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632de8b4440 con 0x5632de9cd680 2024-09-12T03:09:12.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.844+0000 7f160ffff640 1 --2- 172.21.15.116:0/4174611733 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f15fc0a2c00 0x7f15fc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:12.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.844+0000 7f160f7fe640 1 --2- 172.21.15.116:0/1765411887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632de9c2dc0 0x5632de9cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:09:12.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.844+0000 7f160d7fa640 1 -- 172.21.15.116:0/4174611733 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f16000ef750 con 0x7f1610100e20 2024-09-12T03:09:12.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.844+0000 7f1606ffd640 1 --2- 172.21.15.116:0/1765411887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632de9cd680 0x5632de9cfa70 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f15f8009a40 tx=0x7f15f8009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc758a5a9fe0723 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:12.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.844+0000 7f1605ffb640 1 -- 172.21.15.116:0/1765411887 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f8017070 con 0x5632de9cd680 2024-09-12T03:09:12.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.844+0000 7f1605ffb640 1 -- 172.21.15.116:0/1765411887 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15f8013470 con 0x5632de9cd680 2024-09-12T03:09:12.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.844+0000 7f1605ffb640 1 -- 172.21.15.116:0/1765411887 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f8012630 con 0x5632de9cd680 2024-09-12T03:09:12.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.845+0000 7f1617f46740 1 -- 172.21.15.116:0/1765411887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632de9cd680 msgr2=0x5632de9cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:12.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.845+0000 7f160ffff640 1 --2- 172.21.15.116:0/4174611733 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f15fc0a2c00 0x7f15fc0a50c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1608005fd0 tx=0x7f1608004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:12.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.845+0000 7f1617f46740 1 --2- 172.21.15.116:0/1765411887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632de9cd680 0x5632de9cfa70 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f15f8009a40 tx=0x7f15f8009f00 comp rx=0 tx=0).stop 2024-09-12T03:09:12.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.845+0000 7f1617f46740 1 -- 172.21.15.116:0/1765411887 shutdown_connections 2024-09-12T03:09:12.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.845+0000 7f1617f46740 1 --2- 172.21.15.116:0/1765411887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632de9cd680 0x5632de9cfa70 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.845+0000 7f1617f46740 1 --2- 172.21.15.116:0/1765411887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632de9c2dc0 0x5632de9cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.845+0000 7f1617f46740 1 --2- 172.21.15.116:0/1765411887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632de9c24a0 0x5632de9c2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.845+0000 7f1617f46740 1 -- 172.21.15.116:0/1765411887 >> 172.21.15.116:0/1765411887 conn(0x5632de9bea30 msgr2=0x5632de9bee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:12.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.845+0000 7f1617f46740 1 -- 172.21.15.116:0/1765411887 shutdown_connections 2024-09-12T03:09:12.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.845+0000 7f1617f46740 1 -- 172.21.15.116:0/1765411887 wait complete. 2024-09-12T03:09:12.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.846+0000 7f1617f46740 1 -- start start 2024-09-12T03:09:12.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.846+0000 7f1617f46740 1 Processor -- start 2024-09-12T03:09:12.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.846+0000 7f1617f46740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632de9c24a0 0x5632de9ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:12.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.846+0000 7f1617f46740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632de9c2dc0 0x5632de9ca210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:12.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.846+0000 7f1617f46740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632de9cd680 0x5632de9ca750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:12.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.846+0000 7f1617f46740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5632de9d0250 con 0x5632de9c24a0 2024-09-12T03:09:12.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.846+0000 7f1617f46740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5632de8e3050 con 0x5632de9c2dc0 2024-09-12T03:09:12.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.846+0000 7f1617f46740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5632de9d03d0 con 0x5632de9cd680 2024-09-12T03:09:12.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.846+0000 7f1617f46740 10 client.0 _ll_register_callbacks cb 0x5632de923160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:12.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.846+0000 7f160f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632de9c24a0 0x5632de9ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:12.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.846+0000 7f160f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632de9c24a0 0x5632de9ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50598/0 (socket says 172.21.15.116:50598) 2024-09-12T03:09:12.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.846+0000 7f160f7fe640 1 -- 172.21.15.116:0/846837286 learned_addr learned my addr 172.21.15.116:0/846837286 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:12.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.846+0000 7f160ffff640 1 --2- 172.21.15.116:0/846837286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632de9cd680 0x5632de9ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:12.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.846+0000 7f1606ffd640 1 --2- 172.21.15.116:0/846837286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632de9c2dc0 0x5632de9ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:12.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.847+0000 7f160f7fe640 1 -- 172.21.15.116:0/846837286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632de9cd680 msgr2=0x5632de9ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:12.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.847+0000 7f160f7fe640 1 --2- 172.21.15.116:0/846837286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632de9cd680 0x5632de9ca750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.847+0000 7f160f7fe640 1 -- 172.21.15.116:0/846837286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632de9c2dc0 msgr2=0x5632de9ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:12.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.847+0000 7f160f7fe640 1 --2- 172.21.15.116:0/846837286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632de9c2dc0 0x5632de9ca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.847+0000 7f160f7fe640 1 -- 172.21.15.116:0/846837286 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632de6993c0 con 0x5632de9c24a0 2024-09-12T03:09:12.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.847+0000 7f160f7fe640 1 --2- 172.21.15.116:0/846837286 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632de9c24a0 0x5632de9ccf60 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f16000ed5f0 tx=0x7f16000bbdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a003e9fbc1e00686 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:12.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.847+0000 7f15eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:09:12.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.847+0000 7f15eeffd640 1 -- 172.21.15.116:0/846837286 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1600049070 con 0x5632de9c24a0 2024-09-12T03:09:12.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.847+0000 7f15eeffd640 1 -- 172.21.15.116:0/846837286 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f16000ba2e0 con 0x5632de9c24a0 2024-09-12T03:09:12.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.847+0000 7f1617f46740 1 -- 172.21.15.116:0/846837286 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15f8013470 con 0x5632de9c24a0 2024-09-12T03:09:12.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.847+0000 7f15eeffd640 1 -- 172.21.15.116:0/846837286 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f16000dc0e0 con 0x5632de9c24a0 2024-09-12T03:09:12.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.847+0000 7f1617f46740 10 client.16902 fetch_fsmap learned FSMap version 25 2024-09-12T03:09:12.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.847+0000 7f1617f46740 1 -- 172.21.15.116:0/846837286 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5632de9dbb30 con 0x5632de9c24a0 2024-09-12T03:09:12.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.847+0000 7f15eeffd640 1 -- 172.21.15.116:0/846837286 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15f8013470 con 0x5632de9c24a0 2024-09-12T03:09:12.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.847+0000 7f15eeffd640 1 -- 172.21.15.116:0/846837286 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f16000b8650 con 0x5632de9c24a0 2024-09-12T03:09:12.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.847+0000 7f1617f46740 10 client.16902 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:09:12.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.847+0000 7f1617f46740 10 client.16902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:09:12.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.847+0000 7f1617f46740 20 client.16902 populate_metadata read hostname 'smithi116' 2024-09-12T03:09:12.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.847+0000 7f1617f46740 1 --2- 172.21.15.116:0/846837286 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5632de9dced0 0x5632de9df300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:12.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.847+0000 7f1617f46740 4 client.16902 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:09:12.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.847+0000 7f1617f46740 1 -- 172.21.15.116:0/846837286 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632de935850 con 0x5632de9dced0 2024-09-12T03:09:12.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.848+0000 7f1606ffd640 1 --2- 172.21.15.116:0/846837286 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5632de9dced0 0x5632de9df300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:12.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.848+0000 7f1606ffd640 1 --2- 172.21.15.116:0/846837286 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5632de9dced0 0x5632de9df300 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d13317079d99df7 server_cookie=d5ab6ff77ddd6fb4 in_seq=0 out_seq=0 2024-09-12T03:09:12.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.848+0000 7f15eeffd640 10 client.16902 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:12.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.849+0000 7f15eeffd640 1 -- 172.21.15.116:0/846837286 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5632de935850 con 0x5632de9dced0 2024-09-12T03:09:12.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.849+0000 7f15eeffd640 10 client.16902 handle_command_reply: tid=0 2024-09-12T03:09:12.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.849+0000 7f1617f46740 1 client.16902 shutdown 2024-09-12T03:09:12.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.849+0000 7f1617f46740 1 -- 172.21.15.116:0/846837286 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632de9c24a0 msgr2=0x5632de9ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:12.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.849+0000 7f1617f46740 1 --2- 172.21.15.116:0/846837286 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632de9c24a0 0x5632de9ccf60 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f16000ed5f0 tx=0x7f16000bbdb0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.849+0000 7f1617f46740 1 -- 172.21.15.116:0/846837286 shutdown_connections 2024-09-12T03:09:12.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.849+0000 7f1617f46740 1 --2- 172.21.15.116:0/846837286 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5632de9dced0 0x5632de9df300 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.849+0000 7f1617f46740 1 --2- 172.21.15.116:0/846837286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632de9cd680 0x5632de9ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.849+0000 7f1617f46740 1 --2- 172.21.15.116:0/846837286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632de9c2dc0 0x5632de9ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.849+0000 7f15eeffd640 0 client.16902 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:12.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.849+0000 7f1617f46740 1 --2- 172.21.15.116:0/846837286 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632de9c24a0 0x5632de9ccf60 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.849+0000 7f1617f46740 1 -- 172.21.15.116:0/846837286 >> 172.21.15.116:0/846837286 conn(0x5632de9bea30 msgr2=0x5632de9c4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:12.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.849+0000 7f1617f46740 1 -- 172.21.15.116:0/846837286 shutdown_connections 2024-09-12T03:09:12.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.849+0000 7f1617f46740 1 -- 172.21.15.116:0/846837286 wait complete. 2024-09-12T03:09:12.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.849+0000 7f1617f46740 20 client.16902 trim_cache size 0 max 16384 2024-09-12T03:09:12.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.866+0000 7f1617f46740 1 Processor -- start 2024-09-12T03:09:12.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.866+0000 7f1617f46740 1 -- start start 2024-09-12T03:09:12.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.867+0000 7f1617f46740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632de9c24a0 0x5632de9d94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:12.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.867+0000 7f1617f46740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632de9c2dc0 0x5632de9e4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:12.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.867+0000 7f1617f46740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632de9cd680 0x5632de9e5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:12.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.867+0000 7f1617f46740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f16101ada80 con 0x5632de9c24a0 2024-09-12T03:09:12.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.867+0000 7f1617f46740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f16101ab350 con 0x5632de9cd680 2024-09-12T03:09:12.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.867+0000 7f1617f46740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f16101a91c0 con 0x5632de9c2dc0 2024-09-12T03:09:12.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.867+0000 7f160f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632de9c24a0 0x5632de9d94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:12.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.867+0000 7f160f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632de9c24a0 0x5632de9d94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50602/0 (socket says 172.21.15.116:50602) 2024-09-12T03:09:12.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.867+0000 7f160f7fe640 1 -- 172.21.15.116:0/1672340606 learned_addr learned my addr 172.21.15.116:0/1672340606 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:12.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.867+0000 7f160ffff640 1 --2- 172.21.15.116:0/1672340606 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632de9cd680 0x5632de9e5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:12.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.867+0000 7f1606ffd640 1 --2- 172.21.15.116:0/1672340606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632de9c2dc0 0x5632de9e4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:12.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.867+0000 7f160f7fe640 1 -- 172.21.15.116:0/1672340606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632de9c2dc0 msgr2=0x5632de9e4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:12.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.867+0000 7f160f7fe640 1 --2- 172.21.15.116:0/1672340606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632de9c2dc0 0x5632de9e4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.867+0000 7f160f7fe640 1 -- 172.21.15.116:0/1672340606 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632de9cd680 msgr2=0x5632de9e5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:12.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.867+0000 7f160f7fe640 1 --2- 172.21.15.116:0/1672340606 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632de9cd680 0x5632de9e5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.867+0000 7f160f7fe640 1 -- 172.21.15.116:0/1672340606 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632de9c2bf0 con 0x5632de9c24a0 2024-09-12T03:09:12.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.867+0000 7f160f7fe640 1 --2- 172.21.15.116:0/1672340606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632de9c24a0 0x5632de9d94b0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f16000b8440 tx=0x7f160003b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2958cf7a65d8569f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:12.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.868+0000 7f16057fa640 1 -- 172.21.15.116:0/1672340606 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1600049070 con 0x5632de9c24a0 2024-09-12T03:09:12.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.868+0000 7f16057fa640 1 -- 172.21.15.116:0/1672340606 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16000bc6e0 con 0x5632de9c24a0 2024-09-12T03:09:12.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.868+0000 7f16057fa640 1 -- 172.21.15.116:0/1672340606 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16000b8650 con 0x5632de9c24a0 2024-09-12T03:09:12.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.868+0000 7f1617f46740 1 -- 172.21.15.116:0/1672340606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632de9c24a0 msgr2=0x5632de9d94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:12.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.868+0000 7f1617f46740 1 --2- 172.21.15.116:0/1672340606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632de9c24a0 0x5632de9d94b0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f16000b8440 tx=0x7f160003b890 comp rx=0 tx=0).stop 2024-09-12T03:09:12.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.868+0000 7f1617f46740 1 -- 172.21.15.116:0/1672340606 shutdown_connections 2024-09-12T03:09:12.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.868+0000 7f1617f46740 1 --2- 172.21.15.116:0/1672340606 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632de9cd680 0x5632de9e5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.868+0000 7f1617f46740 1 --2- 172.21.15.116:0/1672340606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632de9c2dc0 0x5632de9e4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.868+0000 7f1617f46740 1 --2- 172.21.15.116:0/1672340606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632de9c24a0 0x5632de9d94b0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.868+0000 7f1617f46740 1 -- 172.21.15.116:0/1672340606 >> 172.21.15.116:0/1672340606 conn(0x5632de9bea30 msgr2=0x5632de9c8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:12.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.868+0000 7f1617f46740 1 -- 172.21.15.116:0/1672340606 shutdown_connections 2024-09-12T03:09:12.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.868+0000 7f1617f46740 1 -- 172.21.15.116:0/1672340606 wait complete. 2024-09-12T03:09:12.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.869+0000 7f1617f46740 1 -- start start 2024-09-12T03:09:12.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.869+0000 7f1617f46740 1 Processor -- start 2024-09-12T03:09:12.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.869+0000 7f1617f46740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632de9c24a0 0x5632de9d5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:12.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.869+0000 7f1617f46740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632de9c2dc0 0x5632de9d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:12.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.869+0000 7f1617f46740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632de9cd680 0x5632de9d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:12.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.869+0000 7f1617f46740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f16101a9810 con 0x5632de9c24a0 2024-09-12T03:09:12.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.869+0000 7f1617f46740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f16101aac80 con 0x5632de9c2dc0 2024-09-12T03:09:12.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.869+0000 7f1617f46740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f16101a9ee0 con 0x5632de9cd680 2024-09-12T03:09:12.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.869+0000 7f1617f46740 10 client.0 _ll_register_callbacks cb 0x7f1600005e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:12.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.869+0000 7f160f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632de9c24a0 0x5632de9d5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:12.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.869+0000 7f160f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632de9c24a0 0x5632de9d5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50604/0 (socket says 172.21.15.116:50604) 2024-09-12T03:09:12.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.869+0000 7f160f7fe640 1 -- 172.21.15.116:0/4091507127 learned_addr learned my addr 172.21.15.116:0/4091507127 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:12.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.869+0000 7f160ffff640 1 --2- 172.21.15.116:0/4091507127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632de9cd680 0x5632de9d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:12.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.870+0000 7f1606ffd640 1 --2- 172.21.15.116:0/4091507127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632de9c2dc0 0x5632de9d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:12.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.870+0000 7f160f7fe640 1 -- 172.21.15.116:0/4091507127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632de9cd680 msgr2=0x5632de9d17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:12.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.870+0000 7f160f7fe640 1 --2- 172.21.15.116:0/4091507127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632de9cd680 0x5632de9d17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.870+0000 7f160f7fe640 1 -- 172.21.15.116:0/4091507127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632de9c2dc0 msgr2=0x5632de9d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:12.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.870+0000 7f160f7fe640 1 --2- 172.21.15.116:0/4091507127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632de9c2dc0 0x5632de9d11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.870+0000 7f160f7fe640 1 -- 172.21.15.116:0/4091507127 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632de9d2020 con 0x5632de9c24a0 2024-09-12T03:09:12.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.870+0000 7f160ffff640 1 --2- 172.21.15.116:0/4091507127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632de9cd680 0x5632de9d17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:09:12.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.870+0000 7f1606ffd640 1 --2- 172.21.15.116:0/4091507127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632de9c2dc0 0x5632de9d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:09:12.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.870+0000 7f160f7fe640 1 --2- 172.21.15.116:0/4091507127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632de9c24a0 0x5632de9d5eb0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f16000b89b0 tx=0x7f16000bbdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f45b88f77bf803 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:12.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.870+0000 7f15eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:09:12.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.870+0000 7f15eeffd640 1 -- 172.21.15.116:0/4091507127 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1600049070 con 0x5632de9c24a0 2024-09-12T03:09:12.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.870+0000 7f15eeffd640 1 -- 172.21.15.116:0/4091507127 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f160003b570 con 0x5632de9c24a0 2024-09-12T03:09:12.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.870+0000 7f1617f46740 1 -- 172.21.15.116:0/4091507127 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16000bc6e0 con 0x5632de9c24a0 2024-09-12T03:09:12.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.870+0000 7f15eeffd640 1 -- 172.21.15.116:0/4091507127 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f16000bbb20 con 0x5632de9c24a0 2024-09-12T03:09:12.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.870+0000 7f1617f46740 10 client.16908 fetch_fsmap learned FSMap version 25 2024-09-12T03:09:12.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.870+0000 7f1617f46740 1 -- 172.21.15.116:0/4091507127 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5632de9d2020 con 0x5632de9c24a0 2024-09-12T03:09:12.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.870+0000 7f15eeffd640 1 -- 172.21.15.116:0/4091507127 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f160009ae50 con 0x5632de9c24a0 2024-09-12T03:09:12.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.870+0000 7f15eeffd640 1 -- 172.21.15.116:0/4091507127 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f16000b6b50 con 0x5632de9c24a0 2024-09-12T03:09:12.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.870+0000 7f1617f46740 10 client.16908 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:09:12.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.870+0000 7f1617f46740 10 client.16908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:09:12.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.870+0000 7f1617f46740 20 client.16908 populate_metadata read hostname 'smithi116' 2024-09-12T03:09:12.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.870+0000 7f1617f46740 1 --2- 172.21.15.116:0/4091507127 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5632de9dced0 0x5632dea49fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:12.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.870+0000 7f1617f46740 4 client.16908 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:09:12.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.870+0000 7f1617f46740 1 -- 172.21.15.116:0/4091507127 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5632de92fd60 con 0x5632de9dced0 2024-09-12T03:09:12.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.871+0000 7f1606ffd640 1 --2- 172.21.15.116:0/4091507127 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5632de9dced0 0x5632dea49fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:12.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.871+0000 7f1606ffd640 1 --2- 172.21.15.116:0/4091507127 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5632de9dced0 0x5632dea49fe0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ac5ba6e4b8f8117 server_cookie=8146ef28aae52b5b in_seq=0 out_seq=0 2024-09-12T03:09:12.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.871+0000 7f15eeffd640 10 client.16908 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:12.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.871+0000 7f15eeffd640 1 -- 172.21.15.116:0/4091507127 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5632de92fd60 con 0x5632de9dced0 2024-09-12T03:09:12.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.871+0000 7f15eeffd640 10 client.16908 handle_command_reply: tid=0 2024-09-12T03:09:12.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.871+0000 7f1617f46740 1 client.16908 shutdown 2024-09-12T03:09:12.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.871+0000 7f1617f46740 1 -- 172.21.15.116:0/4091507127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632de9c24a0 msgr2=0x5632de9d5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:12.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.871+0000 7f1617f46740 1 --2- 172.21.15.116:0/4091507127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632de9c24a0 0x5632de9d5eb0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f16000b89b0 tx=0x7f16000bbdb0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.871+0000 7f1617f46740 1 -- 172.21.15.116:0/4091507127 shutdown_connections 2024-09-12T03:09:12.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.871+0000 7f1617f46740 1 --2- 172.21.15.116:0/4091507127 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5632de9dced0 0x5632dea49fe0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.871+0000 7f1617f46740 1 --2- 172.21.15.116:0/4091507127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5632de9cd680 0x5632de9d17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.871+0000 7f1617f46740 1 --2- 172.21.15.116:0/4091507127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632de9c2dc0 0x5632de9d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.872+0000 7f1617f46740 1 --2- 172.21.15.116:0/4091507127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5632de9c24a0 0x5632de9d5eb0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.872+0000 7f1617f46740 1 -- 172.21.15.116:0/4091507127 >> 172.21.15.116:0/4091507127 conn(0x5632de9bea30 msgr2=0x5632de9bd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:12.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.872+0000 7f15eeffd640 0 client.16908 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:12.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.872+0000 7f1617f46740 1 -- 172.21.15.116:0/4091507127 shutdown_connections 2024-09-12T03:09:12.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.872+0000 7f1617f46740 1 -- 172.21.15.116:0/4091507127 wait complete. 2024-09-12T03:09:12.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.872+0000 7f1617f46740 20 client.16908 trim_cache size 0 max 16384 2024-09-12T03:09:12.879 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:09:12.879 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:09:12.879 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": { 2024-09-12T03:09:12.880 INFO:teuthology.orchestra.run.smithi116.stdout: "0aa2260a-9cf7-4401-95d5-83104ab2700c": { 2024-09-12T03:09:12.880 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/", 2024-09-12T03:09:12.880 INFO:teuthology.orchestra.run.smithi116.stdout: "tag": "0aa2260a-9cf7-4401-95d5-83104ab2700c", 2024-09-12T03:09:12.880 INFO:teuthology.orchestra.run.smithi116.stdout: "options": "recursive,force" 2024-09-12T03:09:12.880 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:09:12.880 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:09:12.880 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:09:12.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.872+0000 7f1615b6d640 1 -- 172.21.15.116:0/4174611733 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f15fc0a2c00 msgr2=0x7f15fc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:12.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.872+0000 7f1615b6d640 1 --2- 172.21.15.116:0/4174611733 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f15fc0a2c00 0x7f15fc0a50c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1608005fd0 tx=0x7f1608004a40 comp rx=0 tx=0).stop 2024-09-12T03:09:12.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.872+0000 7f1615b6d640 1 -- 172.21.15.116:0/4174611733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1610100e20 msgr2=0x7f1610198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:12.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.872+0000 7f1615b6d640 1 --2- 172.21.15.116:0/4174611733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1610100e20 0x7f1610198530 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f1600006010 tx=0x7f1600003c10 comp rx=0 tx=0).stop 2024-09-12T03:09:12.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.872+0000 7f1615b6d640 1 -- 172.21.15.116:0/4174611733 shutdown_connections 2024-09-12T03:09:12.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.872+0000 7f1615b6d640 1 --2- 172.21.15.116:0/4174611733 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f15fc0a2c00 0x7f15fc0a50c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.873+0000 7f1615b6d640 1 --2- 172.21.15.116:0/4174611733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1610102ef0 0x7f161019ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.873+0000 7f1615b6d640 1 --2- 172.21.15.116:0/4174611733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1610102570 0x7f1610198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.873+0000 7f1615b6d640 1 --2- 172.21.15.116:0/4174611733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1610100e20 0x7f1610198530 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:12.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.873+0000 7f1615b6d640 1 -- 172.21.15.116:0/4174611733 >> 172.21.15.116:0/4174611733 conn(0x7f16100fc7d0 msgr2=0x7f16100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:12.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.873+0000 7f1615b6d640 1 -- 172.21.15.116:0/4174611733 shutdown_connections 2024-09-12T03:09:12.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:12.873+0000 7f1615b6d640 1 -- 172.21.15.116:0/4174611733 wait complete. 2024-09-12T03:09:13.012 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:09:13.580 INFO:tasks.cephfs.filesystem:scrub status for tag:0aa2260a-9cf7-4401-95d5-83104ab2700c - {'path': '/', 'tag': '0aa2260a-9cf7-4401-95d5-83104ab2700c', 'options': 'recursive,force'} 2024-09-12T03:09:13.689 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:09:13.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:13 smithi183 ceph-mon[35535]: pgmap v525: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-12T03:09:13.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:13 smithi177 ceph-mon[35305]: pgmap v525: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-12T03:09:13.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:13 smithi116 ceph-mon[31410]: pgmap v525: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-12T03:09:14.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.169+0000 7f5def006640 1 -- 172.21.15.116:0/2024301817 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5de8102d30 msgr2=0x7f5de8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:14.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.169+0000 7f5def006640 1 --2- 172.21.15.116:0/2024301817 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5de8102d30 0x7f5de8103110 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f5de0009930 tx=0x7f5de002a440 comp rx=0 tx=0).stop 2024-09-12T03:09:14.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.169+0000 7f5def006640 1 -- 172.21.15.116:0/2024301817 shutdown_connections 2024-09-12T03:09:14.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.169+0000 7f5def006640 1 --2- 172.21.15.116:0/2024301817 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5de81082a0 0x7f5de810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:14.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.169+0000 7f5def006640 1 --2- 172.21.15.116:0/2024301817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5de81036e0 0x7f5de8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:14.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.169+0000 7f5def006640 1 --2- 172.21.15.116:0/2024301817 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5de8102d30 0x7f5de8103110 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:14.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.169+0000 7f5def006640 1 -- 172.21.15.116:0/2024301817 >> 172.21.15.116:0/2024301817 conn(0x7f5de80fc7d0 msgr2=0x7f5de80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:14.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.169+0000 7f5def006640 1 -- 172.21.15.116:0/2024301817 shutdown_connections 2024-09-12T03:09:14.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.169+0000 7f5def006640 1 -- 172.21.15.116:0/2024301817 wait complete. 2024-09-12T03:09:14.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.169+0000 7f5def006640 1 Processor -- start 2024-09-12T03:09:14.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.170+0000 7f5def006640 1 -- start start 2024-09-12T03:09:14.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.170+0000 7f5def006640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5de8102d30 0x7f5de8198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:14.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.170+0000 7f5def006640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5de81036e0 0x7f5de8198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:14.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.170+0000 7f5def006640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5de81082a0 0x7f5de819cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:14.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.170+0000 7f5def006640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5de8112160 con 0x7f5de81036e0 2024-09-12T03:09:14.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.170+0000 7f5def006640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5de8111fe0 con 0x7f5de8102d30 2024-09-12T03:09:14.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.170+0000 7f5def006640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5de81122e0 con 0x7f5de81082a0 2024-09-12T03:09:14.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.170+0000 7f5de7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5de81036e0 0x7f5de8198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:14.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.170+0000 7f5de7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5de81036e0 0x7f5de8198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50614/0 (socket says 172.21.15.116:50614) 2024-09-12T03:09:14.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.170+0000 7f5de7fff640 1 -- 172.21.15.116:0/2310266528 learned_addr learned my addr 172.21.15.116:0/2310266528 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:14.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.170+0000 7f5ded5a3640 1 --2- 172.21.15.116:0/2310266528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5de81082a0 0x7f5de819cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:14.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.170+0000 7f5decda2640 1 --2- 172.21.15.116:0/2310266528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5de8102d30 0x7f5de8198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:14.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.170+0000 7f5de7fff640 1 -- 172.21.15.116:0/2310266528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5de81082a0 msgr2=0x7f5de819cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:14.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.170+0000 7f5de7fff640 1 --2- 172.21.15.116:0/2310266528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5de81082a0 0x7f5de819cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:14.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.170+0000 7f5de7fff640 1 -- 172.21.15.116:0/2310266528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5de8102d30 msgr2=0x7f5de8198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:14.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.170+0000 7f5de7fff640 1 --2- 172.21.15.116:0/2310266528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5de8102d30 0x7f5de8198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:14.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.170+0000 7f5de7fff640 1 -- 172.21.15.116:0/2310266528 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de819d6d0 con 0x7f5de81036e0 2024-09-12T03:09:14.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.170+0000 7f5de7fff640 1 --2- 172.21.15.116:0/2310266528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5de81036e0 0x7f5de8198c60 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800b990 tx=0x7f5dd800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:14.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.170+0000 7f5de5ffb640 1 -- 172.21.15.116:0/2310266528 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8019070 con 0x7f5de81036e0 2024-09-12T03:09:14.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.171+0000 7f5de5ffb640 1 -- 172.21.15.116:0/2310266528 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dd800cba0 con 0x7f5de81036e0 2024-09-12T03:09:14.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.171+0000 7f5def006640 1 -- 172.21.15.116:0/2310266528 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de819d9c0 con 0x7f5de81036e0 2024-09-12T03:09:14.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.171+0000 7f5de5ffb640 1 -- 172.21.15.116:0/2310266528 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd80126b0 con 0x7f5de81036e0 2024-09-12T03:09:14.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.171+0000 7f5def006640 1 -- 172.21.15.116:0/2310266528 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de81a52a0 con 0x7f5de81036e0 2024-09-12T03:09:14.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.172+0000 7f5def006640 1 -- 172.21.15.116:0/2310266528 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5de806a850 con 0x7f5de81036e0 2024-09-12T03:09:14.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.173+0000 7f5de5ffb640 1 -- 172.21.15.116:0/2310266528 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f5dd801c070 con 0x7f5de81036e0 2024-09-12T03:09:14.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.173+0000 7f5de5ffb640 1 --2- 172.21.15.116:0/2310266528 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5dd40a2b80 0x7f5dd40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:14.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.174+0000 7f5decda2640 1 --2- 172.21.15.116:0/2310266528 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5dd40a2b80 0x7f5dd40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:14.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.174+0000 7f5de5ffb640 1 -- 172.21.15.116:0/2310266528 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f5dd80ce5e0 con 0x7f5de81036e0 2024-09-12T03:09:14.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.174+0000 7f5decda2640 1 --2- 172.21.15.116:0/2310266528 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5dd40a2b80 0x7f5dd40a5040 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5de00096c0 tx=0x7f5de0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:14.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.178+0000 7f5de5ffb640 1 -- 172.21.15.116:0/2310266528 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5dd809b570 con 0x7f5de81036e0 2024-09-12T03:09:14.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.380+0000 7f5def006640 1 -- 172.21.15.116:0/2310266528 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5de810a2b0 con 0x7f5de81036e0 2024-09-12T03:09:14.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.381+0000 7f5de5ffb640 1 -- 172.21.15.116:0/2310266528 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f5dd809ae70 con 0x7f5de81036e0 2024-09-12T03:09:14.382 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:09:14.382 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:09:14.382 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:09:14.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.383+0000 7f5def006640 1 -- 172.21.15.116:0/2310266528 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5dd40a2b80 msgr2=0x7f5dd40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:14.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.383+0000 7f5def006640 1 --2- 172.21.15.116:0/2310266528 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5dd40a2b80 0x7f5dd40a5040 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5de00096c0 tx=0x7f5de0000bc0 comp rx=0 tx=0).stop 2024-09-12T03:09:14.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.383+0000 7f5def006640 1 -- 172.21.15.116:0/2310266528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5de81036e0 msgr2=0x7f5de8198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:14.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.383+0000 7f5def006640 1 --2- 172.21.15.116:0/2310266528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5de81036e0 0x7f5de8198c60 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800b990 tx=0x7f5dd800be50 comp rx=0 tx=0).stop 2024-09-12T03:09:14.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.383+0000 7f5def006640 1 -- 172.21.15.116:0/2310266528 shutdown_connections 2024-09-12T03:09:14.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.383+0000 7f5def006640 1 --2- 172.21.15.116:0/2310266528 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5dd40a2b80 0x7f5dd40a5040 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:14.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.383+0000 7f5def006640 1 --2- 172.21.15.116:0/2310266528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5de81082a0 0x7f5de819cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:14.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.383+0000 7f5def006640 1 --2- 172.21.15.116:0/2310266528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5de81036e0 0x7f5de8198c60 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:14.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.383+0000 7f5def006640 1 --2- 172.21.15.116:0/2310266528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5de8102d30 0x7f5de8198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:14.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.383+0000 7f5def006640 1 -- 172.21.15.116:0/2310266528 >> 172.21.15.116:0/2310266528 conn(0x7f5de80fc7d0 msgr2=0x7f5de8105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:14.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.383+0000 7f5def006640 1 -- 172.21.15.116:0/2310266528 shutdown_connections 2024-09-12T03:09:14.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:14.383+0000 7f5def006640 1 -- 172.21.15.116:0/2310266528 wait complete. 2024-09-12T03:09:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:14 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:09:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:14 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2310266528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:09:14.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:14 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:09:14.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:14 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2310266528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:09:14.753 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:14 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:09:14.753 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:14 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2310266528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:09:15.149 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id d2a1b25e --release --await' 2024-09-12T03:09:15.149 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id d2a1b25e --release --await 2024-09-12T03:09:15.394 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:09:15.670 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:15 smithi116 ceph-mon[31410]: pgmap v526: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-12T03:09:15.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:15 smithi183 ceph-mon[35535]: pgmap v526: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-12T03:09:15.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:15 smithi177 ceph-mon[35305]: pgmap v526: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-12T03:09:15.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.916+0000 7f891c8a3640 1 -- 172.21.15.116:0/2864655184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8918102ef0 msgr2=0x7f891810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:15.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.916+0000 7f891c8a3640 1 --2- 172.21.15.116:0/2864655184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8918102ef0 0x7f891810da40 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f8910009ca0 tx=0x7f891002a950 comp rx=0 tx=0).stop 2024-09-12T03:09:15.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.916+0000 7f891c8a3640 1 -- 172.21.15.116:0/2864655184 shutdown_connections 2024-09-12T03:09:15.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.916+0000 7f891c8a3640 1 --2- 172.21.15.116:0/2864655184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8918102ef0 0x7f891810da40 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.916+0000 7f891c8a3640 1 --2- 172.21.15.116:0/2864655184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8918102570 0x7f89181029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.916+0000 7f891c8a3640 1 --2- 172.21.15.116:0/2864655184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8918100e20 0x7f8918101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.916+0000 7f891c8a3640 1 -- 172.21.15.116:0/2864655184 >> 172.21.15.116:0/2864655184 conn(0x7f89180fc7d0 msgr2=0x7f89180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:15.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.916+0000 7f891c8a3640 1 -- 172.21.15.116:0/2864655184 shutdown_connections 2024-09-12T03:09:15.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.916+0000 7f891c8a3640 1 -- 172.21.15.116:0/2864655184 wait complete. 2024-09-12T03:09:15.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.917+0000 7f891c8a3640 1 Processor -- start 2024-09-12T03:09:15.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.917+0000 7f891c8a3640 1 -- start start 2024-09-12T03:09:15.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.917+0000 7f891c8a3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8918100e20 0x7f89180fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:15.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.917+0000 7f891c8a3640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8918102570 0x7f89180fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:15.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.917+0000 7f891c8a3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8918102ef0 0x7f89180fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:15.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.917+0000 7f891c8a3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8918110f80 con 0x7f8918102ef0 2024-09-12T03:09:15.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.917+0000 7f891c8a3640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8918110e00 con 0x7f8918102570 2024-09-12T03:09:15.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.917+0000 7f891c8a3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8918111100 con 0x7f8918100e20 2024-09-12T03:09:15.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.917+0000 7f8915d9b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8918102570 0x7f89180fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:15.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.917+0000 7f8915d9b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8918102570 0x7f89180fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.116:57916/0 (socket says 172.21.15.116:57916) 2024-09-12T03:09:15.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.917+0000 7f8915d9b640 1 -- 172.21.15.116:0/561719249 learned_addr learned my addr 172.21.15.116:0/561719249 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:15.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.918+0000 7f891659c640 1 --2- 172.21.15.116:0/561719249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8918100e20 0x7f89180fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:15.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.918+0000 7f8916d9d640 1 --2- 172.21.15.116:0/561719249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8918102ef0 0x7f89180fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:15.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.918+0000 7f8915d9b640 1 -- 172.21.15.116:0/561719249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8918100e20 msgr2=0x7f89180fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:15.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.918+0000 7f8915d9b640 1 --2- 172.21.15.116:0/561719249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8918100e20 0x7f89180fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.918+0000 7f8915d9b640 1 -- 172.21.15.116:0/561719249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8918102ef0 msgr2=0x7f89180fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:15.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.918+0000 7f8915d9b640 1 --2- 172.21.15.116:0/561719249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8918102ef0 0x7f89180fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.918+0000 7f8915d9b640 1 -- 172.21.15.116:0/561719249 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89181aade0 con 0x7f8918102570 2024-09-12T03:09:15.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.918+0000 7f8915d9b640 1 --2- 172.21.15.116:0/561719249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8918102570 0x7f89180fd1c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f890c00c920 tx=0x7f890c00cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:15.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.918+0000 7f89077fe640 1 -- 172.21.15.116:0/561719249 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c007c70 con 0x7f8918102570 2024-09-12T03:09:15.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.918+0000 7f89077fe640 1 -- 172.21.15.116:0/561719249 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f890c007e10 con 0x7f8918102570 2024-09-12T03:09:15.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.918+0000 7f891c8a3640 1 -- 172.21.15.116:0/561719249 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89181aafb0 con 0x7f8918102570 2024-09-12T03:09:15.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.918+0000 7f89077fe640 1 -- 172.21.15.116:0/561719249 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c018890 con 0x7f8918102570 2024-09-12T03:09:15.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.918+0000 7f891c8a3640 1 -- 172.21.15.116:0/561719249 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89181ab4b0 con 0x7f8918102570 2024-09-12T03:09:15.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.920+0000 7f89077fe640 1 -- 172.21.15.116:0/561719249 <== mon.2 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f890c00f450 con 0x7f8918102570 2024-09-12T03:09:15.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.921+0000 7f891ec7c740 1 Processor -- start 2024-09-12T03:09:15.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.921+0000 7f891ec7c740 1 -- start start 2024-09-12T03:09:15.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.921+0000 7f891ec7c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559068157060 0x559068157440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:15.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.921+0000 7f891ec7c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559068157980 0x559068161d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:15.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.921+0000 7f891ec7c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559068162240 0x559068164630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:15.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.921+0000 7f89077fe640 1 --2- 172.21.15.116:0/561719249 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f89000a2c00 0x7f89000a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:15.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.921+0000 7f891ec7c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5590680bc620 con 0x559068157980 2024-09-12T03:09:15.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.921+0000 7f891ec7c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559068063e50 con 0x559068162240 2024-09-12T03:09:15.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.921+0000 7f891ec7c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5590680b85e0 con 0x559068157060 2024-09-12T03:09:15.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.922+0000 7f891659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559068157980 0x559068161d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:15.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.922+0000 7f8915d9b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559068162240 0x559068164630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:15.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.922+0000 7f891659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559068157980 0x559068161d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50642/0 (socket says 172.21.15.116:50642) 2024-09-12T03:09:15.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.922+0000 7f891659c640 1 -- 172.21.15.116:0/2417890970 learned_addr learned my addr 172.21.15.116:0/2417890970 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:15.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.922+0000 7f8916d9d640 1 --2- 172.21.15.116:0/2417890970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559068157060 0x559068157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:15.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.922+0000 7f8916d9d640 1 --2- 172.21.15.116:0/561719249 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f89000a2c00 0x7f89000a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:15.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.922+0000 7f89077fe640 1 -- 172.21.15.116:0/561719249 <== mon.2 v2:172.21.15.177:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f890c0cf050 con 0x7f8918102570 2024-09-12T03:09:15.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.922+0000 7f891659c640 1 -- 172.21.15.116:0/2417890970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559068157060 msgr2=0x559068157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:15.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.922+0000 7f891659c640 1 --2- 172.21.15.116:0/2417890970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559068157060 0x559068157440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.922+0000 7f891659c640 1 -- 172.21.15.116:0/2417890970 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559068162240 msgr2=0x559068164630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:15.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.922+0000 7f891659c640 1 --2- 172.21.15.116:0/2417890970 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559068162240 0x559068164630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.922+0000 7f891659c640 1 -- 172.21.15.116:0/2417890970 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559068054d40 con 0x559068157980 2024-09-12T03:09:15.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.922+0000 7f891659c640 1 --2- 172.21.15.116:0/2417890970 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559068157980 0x559068161d00 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f890800ac20 tx=0x7f890800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfff4f4f2567bc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:15.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.922+0000 7f8916d9d640 1 --2- 172.21.15.116:0/561719249 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f89000a2c00 0x7f89000a50c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8910005fe0 tx=0x7f8910005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:15.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.922+0000 7f8904ff9640 1 -- 172.21.15.116:0/2417890970 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890801d070 con 0x559068157980 2024-09-12T03:09:15.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.923+0000 7f8904ff9640 1 -- 172.21.15.116:0/2417890970 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8908018b20 con 0x559068157980 2024-09-12T03:09:15.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.923+0000 7f8904ff9640 1 -- 172.21.15.116:0/2417890970 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8908010b50 con 0x559068157980 2024-09-12T03:09:15.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.924+0000 7f891ec7c740 1 -- 172.21.15.116:0/2417890970 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559068157980 msgr2=0x559068161d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:15.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.924+0000 7f891ec7c740 1 --2- 172.21.15.116:0/2417890970 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559068157980 0x559068161d00 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f890800ac20 tx=0x7f890800f790 comp rx=0 tx=0).stop 2024-09-12T03:09:15.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.924+0000 7f891ec7c740 1 -- 172.21.15.116:0/2417890970 shutdown_connections 2024-09-12T03:09:15.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.924+0000 7f891ec7c740 1 --2- 172.21.15.116:0/2417890970 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559068162240 0x559068164630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.924+0000 7f891ec7c740 1 --2- 172.21.15.116:0/2417890970 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559068157980 0x559068161d00 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.924+0000 7f891ec7c740 1 --2- 172.21.15.116:0/2417890970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559068157060 0x559068157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.924+0000 7f891ec7c740 1 -- 172.21.15.116:0/2417890970 >> 172.21.15.116:0/2417890970 conn(0x5590681535f0 msgr2=0x5590681539d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:15.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.924+0000 7f891ec7c740 1 -- 172.21.15.116:0/2417890970 shutdown_connections 2024-09-12T03:09:15.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.924+0000 7f891ec7c740 1 -- 172.21.15.116:0/2417890970 wait complete. 2024-09-12T03:09:15.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.924+0000 7f891ec7c740 1 -- start start 2024-09-12T03:09:15.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.924+0000 7f891ec7c740 1 Processor -- start 2024-09-12T03:09:15.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.924+0000 7f891ec7c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559068157060 0x55906815f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:15.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.924+0000 7f891ec7c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559068157980 0x55906815f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:15.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.924+0000 7f891ec7c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559068162240 0x55906815fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:15.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.924+0000 7f891ec7c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559068164e10 con 0x559068157060 2024-09-12T03:09:15.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.924+0000 7f891ec7c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559068044f10 con 0x559068157980 2024-09-12T03:09:15.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.924+0000 7f891ec7c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559068164f90 con 0x559068162240 2024-09-12T03:09:15.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.924+0000 7f891ec7c740 10 client.0 _ll_register_callbacks cb 0x559068068db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:15.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.925+0000 7f8915d9b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559068157980 0x55906815f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:15.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.925+0000 7f8915d9b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559068157980 0x55906815f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.116:57930/0 (socket says 172.21.15.116:57930) 2024-09-12T03:09:15.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.925+0000 7f8915d9b640 1 -- 172.21.15.116:0/852315074 learned_addr learned my addr 172.21.15.116:0/852315074 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:15.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.925+0000 7f8916d9d640 1 --2- 172.21.15.116:0/852315074 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559068162240 0x55906815fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:15.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.925+0000 7f891659c640 1 --2- 172.21.15.116:0/852315074 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559068157060 0x55906815f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:15.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.925+0000 7f8915d9b640 1 -- 172.21.15.116:0/852315074 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559068162240 msgr2=0x55906815fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:15.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.925+0000 7f8915d9b640 1 --2- 172.21.15.116:0/852315074 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559068162240 0x55906815fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.925+0000 7f8915d9b640 1 -- 172.21.15.116:0/852315074 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559068157060 msgr2=0x55906815f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:15.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.925+0000 7f8915d9b640 1 --2- 172.21.15.116:0/852315074 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559068157060 0x55906815f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.925+0000 7f8915d9b640 1 -- 172.21.15.116:0/852315074 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559067dcb580 con 0x559068157980 2024-09-12T03:09:15.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.925+0000 7f891659c640 1 --2- 172.21.15.116:0/852315074 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559068157060 0x55906815f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:09:15.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.925+0000 7f8915d9b640 1 --2- 172.21.15.116:0/852315074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559068157980 0x55906815f620 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f890c00c8f0 tx=0x7f890c09b6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e76f46a8d7783f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:15.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.925+0000 7f88fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-09-12T03:09:15.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.925+0000 7f88fdffb640 1 -- 172.21.15.116:0/852315074 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c09bfe0 con 0x559068157980 2024-09-12T03:09:15.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.925+0000 7f88fdffb640 1 -- 172.21.15.116:0/852315074 <== mon.2 v2:172.21.15.177:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f890c0860b0 con 0x559068157980 2024-09-12T03:09:15.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.925+0000 7f891ec7c740 1 -- 172.21.15.116:0/852315074 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8908018b20 con 0x559068157980 2024-09-12T03:09:15.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.925+0000 7f88fdffb640 1 -- 172.21.15.116:0/852315074 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f890c09b8e0 con 0x559068157980 2024-09-12T03:09:15.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.925+0000 7f891ec7c740 10 client.25859 fetch_fsmap learned FSMap version 25 2024-09-12T03:09:15.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.925+0000 7f891ec7c740 1 -- 172.21.15.116:0/852315074 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559067e95960 con 0x559068157980 2024-09-12T03:09:15.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.926+0000 7f88fdffb640 1 -- 172.21.15.116:0/852315074 <== mon.2 v2:172.21.15.177:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8908018b20 con 0x559068157980 2024-09-12T03:09:15.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.926+0000 7f88fdffb640 1 -- 172.21.15.116:0/852315074 <== mon.2 v2:172.21.15.177:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f890c0665d0 con 0x559068157980 2024-09-12T03:09:15.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.926+0000 7f891ec7c740 10 client.25859 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:09:15.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.926+0000 7f891ec7c740 10 client.25859 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:09:15.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.926+0000 7f891ec7c740 20 client.25859 populate_metadata read hostname 'smithi116' 2024-09-12T03:09:15.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.926+0000 7f891ec7c740 1 --2- 172.21.15.116:0/852315074 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559068171cb0 0x5590681740c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:15.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.926+0000 7f891ec7c740 4 client.25859 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:09:15.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.926+0000 7f891ec7c740 1 -- 172.21.15.116:0/852315074 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590680c9b90 con 0x559068171cb0 2024-09-12T03:09:15.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.926+0000 7f891659c640 1 --2- 172.21.15.116:0/852315074 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559068171cb0 0x5590681740c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:15.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.927+0000 7f88fdffb640 10 client.25859 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:15.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.927+0000 7f891659c640 1 --2- 172.21.15.116:0/852315074 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559068171cb0 0x5590681740c0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d91b45ef36af34eb server_cookie=17f501a94cac91b9 in_seq=0 out_seq=0 2024-09-12T03:09:15.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.927+0000 7f88fdffb640 1 -- 172.21.15.116:0/852315074 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5590680c9b90 con 0x559068171cb0 2024-09-12T03:09:15.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.927+0000 7f88fdffb640 10 client.25859 handle_command_reply: tid=0 2024-09-12T03:09:15.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.927+0000 7f891ec7c740 1 client.25859 shutdown 2024-09-12T03:09:15.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.928+0000 7f891ec7c740 1 -- 172.21.15.116:0/852315074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559068157980 msgr2=0x55906815f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:15.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.928+0000 7f891ec7c740 1 --2- 172.21.15.116:0/852315074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559068157980 0x55906815f620 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f890c00c8f0 tx=0x7f890c09b6b0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.928+0000 7f891ec7c740 1 -- 172.21.15.116:0/852315074 shutdown_connections 2024-09-12T03:09:15.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.928+0000 7f891ec7c740 1 --2- 172.21.15.116:0/852315074 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559068171cb0 0x5590681740c0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.928+0000 7f891ec7c740 1 --2- 172.21.15.116:0/852315074 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559068162240 0x55906815fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.928+0000 7f891ec7c740 1 --2- 172.21.15.116:0/852315074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559068157980 0x55906815f620 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.928+0000 7f891ec7c740 1 --2- 172.21.15.116:0/852315074 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559068157060 0x55906815f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.928+0000 7f891ec7c740 1 -- 172.21.15.116:0/852315074 >> 172.21.15.116:0/852315074 conn(0x5590681535f0 msgr2=0x5590681592d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:15.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.928+0000 7f88fdffb640 0 client.25859 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:15.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.928+0000 7f891ec7c740 1 -- 172.21.15.116:0/852315074 shutdown_connections 2024-09-12T03:09:15.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.928+0000 7f891ec7c740 1 -- 172.21.15.116:0/852315074 wait complete. 2024-09-12T03:09:15.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.928+0000 7f891ec7c740 20 client.25859 trim_cache size 0 max 16384 2024-09-12T03:09:15.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.944+0000 7f891ec7c740 1 Processor -- start 2024-09-12T03:09:15.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.944+0000 7f891ec7c740 1 -- start start 2024-09-12T03:09:15.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.945+0000 7f891ec7c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559068157060 0x559068161050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:15.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.945+0000 7f891ec7c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559068157980 0x559068179990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:15.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.945+0000 7f891ec7c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559068162240 0x559068179ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:15.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.945+0000 7f891ec7c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f89181b6280 con 0x559068157980 2024-09-12T03:09:15.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.945+0000 7f891ec7c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f89181b3b50 con 0x559068162240 2024-09-12T03:09:15.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.945+0000 7f891ec7c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f89181b19a0 con 0x559068157060 2024-09-12T03:09:15.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.945+0000 7f8915d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559068157980 0x559068179990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:15.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.945+0000 7f8915d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559068157980 0x559068179990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50666/0 (socket says 172.21.15.116:50666) 2024-09-12T03:09:15.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.945+0000 7f8915d9b640 1 -- 172.21.15.116:0/3318722024 learned_addr learned my addr 172.21.15.116:0/3318722024 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:15.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.945+0000 7f8916d9d640 1 --2- 172.21.15.116:0/3318722024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559068162240 0x559068179ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:15.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.945+0000 7f8915d9b640 1 -- 172.21.15.116:0/3318722024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559068157060 msgr2=0x559068161050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:15.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.945+0000 7f8915d9b640 1 --2- 172.21.15.116:0/3318722024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559068157060 0x559068161050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.945+0000 7f8915d9b640 1 -- 172.21.15.116:0/3318722024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559068162240 msgr2=0x559068179ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:15.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.945+0000 7f8915d9b640 1 --2- 172.21.15.116:0/3318722024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559068162240 0x559068179ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.945+0000 7f8915d9b640 1 -- 172.21.15.116:0/3318722024 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559068159ed0 con 0x559068157980 2024-09-12T03:09:15.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.945+0000 7f8916d9d640 1 --2- 172.21.15.116:0/3318722024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559068162240 0x559068179ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:09:15.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.945+0000 7f8915d9b640 1 --2- 172.21.15.116:0/3318722024 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559068157980 0x559068179990 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f890c0982b0 tx=0x7f890c099df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fada48753630e5ea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:15.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.945+0000 7f88fffff640 1 -- 172.21.15.116:0/3318722024 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c09b420 con 0x559068157980 2024-09-12T03:09:15.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.945+0000 7f88fffff640 1 -- 172.21.15.116:0/3318722024 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f890c09bfe0 con 0x559068157980 2024-09-12T03:09:15.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.945+0000 7f88fffff640 1 -- 172.21.15.116:0/3318722024 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c0a4320 con 0x559068157980 2024-09-12T03:09:15.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.946+0000 7f891ec7c740 1 -- 172.21.15.116:0/3318722024 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559068157980 msgr2=0x559068179990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:15.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.946+0000 7f891ec7c740 1 --2- 172.21.15.116:0/3318722024 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559068157980 0x559068179990 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f890c0982b0 tx=0x7f890c099df0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.946+0000 7f891ec7c740 1 -- 172.21.15.116:0/3318722024 shutdown_connections 2024-09-12T03:09:15.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.946+0000 7f891ec7c740 1 --2- 172.21.15.116:0/3318722024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559068162240 0x559068179ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.946+0000 7f891ec7c740 1 --2- 172.21.15.116:0/3318722024 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559068157980 0x559068179990 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.946+0000 7f891ec7c740 1 --2- 172.21.15.116:0/3318722024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559068157060 0x559068161050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.946+0000 7f891ec7c740 1 -- 172.21.15.116:0/3318722024 >> 172.21.15.116:0/3318722024 conn(0x5590681535f0 msgr2=0x559068160940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:15.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.946+0000 7f891ec7c740 1 -- 172.21.15.116:0/3318722024 shutdown_connections 2024-09-12T03:09:15.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.946+0000 7f891ec7c740 1 -- 172.21.15.116:0/3318722024 wait complete. 2024-09-12T03:09:15.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.946+0000 7f891ec7c740 1 -- start start 2024-09-12T03:09:15.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.946+0000 7f891ec7c740 1 Processor -- start 2024-09-12T03:09:15.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.947+0000 7f891ec7c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559068157060 0x55906816a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:15.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.947+0000 7f891ec7c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559068157980 0x559068166cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:15.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.947+0000 7f891ec7c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559068162240 0x559068167250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:15.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.947+0000 7f891ec7c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f89181b2010 con 0x559068157980 2024-09-12T03:09:15.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.947+0000 7f891ec7c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f89181b3480 con 0x559068157060 2024-09-12T03:09:15.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.947+0000 7f891ec7c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f89181b26e0 con 0x559068162240 2024-09-12T03:09:15.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.947+0000 7f891ec7c740 10 client.0 _ll_register_callbacks cb 0x7f890c00a560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:15.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.947+0000 7f8915d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559068157980 0x559068166cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:15.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.947+0000 7f891659c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559068157060 0x55906816a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:15.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.947+0000 7f8915d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559068157980 0x559068166cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50678/0 (socket says 172.21.15.116:50678) 2024-09-12T03:09:15.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.947+0000 7f891659c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559068157060 0x55906816a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.116:57950/0 (socket says 172.21.15.116:57950) 2024-09-12T03:09:15.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.947+0000 7f8915d9b640 1 -- 172.21.15.116:0/1951234246 learned_addr learned my addr 172.21.15.116:0/1951234246 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:15.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.947+0000 7f8916d9d640 1 --2- 172.21.15.116:0/1951234246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559068162240 0x559068167250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:15.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.947+0000 7f8915d9b640 1 -- 172.21.15.116:0/1951234246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559068162240 msgr2=0x559068167250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:15.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.947+0000 7f8915d9b640 1 --2- 172.21.15.116:0/1951234246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559068162240 0x559068167250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.947+0000 7f8915d9b640 1 -- 172.21.15.116:0/1951234246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559068157060 msgr2=0x55906816a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:15.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.947+0000 7f8915d9b640 1 --2- 172.21.15.116:0/1951234246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559068157060 0x55906816a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.947+0000 7f8915d9b640 1 -- 172.21.15.116:0/1951234246 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559068167a80 con 0x559068157980 2024-09-12T03:09:15.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.947+0000 7f891659c640 1 --2- 172.21.15.116:0/1951234246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559068157060 0x55906816a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:09:15.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.947+0000 7f8915d9b640 1 --2- 172.21.15.116:0/1951234246 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559068157980 0x559068166cf0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f890c00c8f0 tx=0x7f890c09bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6372c3d579648a78 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:15.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.947+0000 7f88fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:09:15.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.947+0000 7f88fdffb640 1 -- 172.21.15.116:0/1951234246 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c09b420 con 0x559068157980 2024-09-12T03:09:15.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.947+0000 7f88fdffb640 1 -- 172.21.15.116:0/1951234246 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f890c0ab300 con 0x559068157980 2024-09-12T03:09:15.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.947+0000 7f891ec7c740 1 -- 172.21.15.116:0/1951234246 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f890c09bfe0 con 0x559068157980 2024-09-12T03:09:15.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.948+0000 7f88fdffb640 1 -- 172.21.15.116:0/1951234246 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f890c09fc90 con 0x559068157980 2024-09-12T03:09:15.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.948+0000 7f891ec7c740 10 client.16932 fetch_fsmap learned FSMap version 25 2024-09-12T03:09:15.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.948+0000 7f891ec7c740 1 -- 172.21.15.116:0/1951234246 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5590681dda40 con 0x559068157980 2024-09-12T03:09:15.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.948+0000 7f88fdffb640 1 -- 172.21.15.116:0/1951234246 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f890c09bfe0 con 0x559068157980 2024-09-12T03:09:15.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.948+0000 7f88fdffb640 1 -- 172.21.15.116:0/1951234246 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f890c09ad30 con 0x559068157980 2024-09-12T03:09:15.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.948+0000 7f891ec7c740 10 client.16932 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:09:15.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.948+0000 7f891ec7c740 10 client.16932 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:09:15.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.948+0000 7f891ec7c740 20 client.16932 populate_metadata read hostname 'smithi116' 2024-09-12T03:09:15.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.948+0000 7f891ec7c740 1 --2- 172.21.15.116:0/1951234246 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5590681ddf10 0x5590681e0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:15.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.948+0000 7f891ec7c740 4 client.16932 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d2a1b25e", "release": true, "await": true}] 2024-09-12T03:09:15.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.948+0000 7f891ec7c740 1 -- 172.21.15.116:0/1951234246 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d2a1b25e", "release": true, "await": true}) -- 0x5590680c40a0 con 0x5590681ddf10 2024-09-12T03:09:15.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.948+0000 7f891659c640 1 --2- 172.21.15.116:0/1951234246 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5590681ddf10 0x5590681e0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:15.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.949+0000 7f891659c640 1 --2- 172.21.15.116:0/1951234246 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5590681ddf10 0x5590681e0300 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=282573adc505e521 server_cookie=6e6cc2f893373ed8 in_seq=0 out_seq=0 2024-09-12T03:09:15.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.949+0000 7f88fdffb640 10 client.16932 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:15.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.951+0000 7f88fdffb640 1 -- 172.21.15.116:0/1951234246 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5590680c40a0 con 0x5590681ddf10 2024-09-12T03:09:15.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.951+0000 7f88fdffb640 10 client.16932 handle_command_reply: tid=0 2024-09-12T03:09:15.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.951+0000 7f891ec7c740 1 client.16932 shutdown 2024-09-12T03:09:15.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.951+0000 7f891ec7c740 1 -- 172.21.15.116:0/1951234246 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559068157980 msgr2=0x559068166cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:15.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.951+0000 7f891ec7c740 1 --2- 172.21.15.116:0/1951234246 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559068157980 0x559068166cf0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f890c00c8f0 tx=0x7f890c09bee0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.951+0000 7f891ec7c740 1 -- 172.21.15.116:0/1951234246 shutdown_connections 2024-09-12T03:09:15.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.951+0000 7f891ec7c740 1 --2- 172.21.15.116:0/1951234246 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5590681ddf10 0x5590681e0300 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.951+0000 7f88fdffb640 0 client.16932 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:15.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.951+0000 7f891ec7c740 1 --2- 172.21.15.116:0/1951234246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559068162240 0x559068167250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.951+0000 7f891ec7c740 1 --2- 172.21.15.116:0/1951234246 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559068157980 0x559068166cf0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.951+0000 7f891ec7c740 1 --2- 172.21.15.116:0/1951234246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559068157060 0x55906816a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.951+0000 7f891ec7c740 1 -- 172.21.15.116:0/1951234246 >> 172.21.15.116:0/1951234246 conn(0x5590681535f0 msgr2=0x55906815f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:15.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.951+0000 7f891ec7c740 1 -- 172.21.15.116:0/1951234246 shutdown_connections 2024-09-12T03:09:15.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.951+0000 7f891ec7c740 1 -- 172.21.15.116:0/1951234246 wait complete. 2024-09-12T03:09:15.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.951+0000 7f891ec7c740 20 client.16932 trim_cache size 0 max 16384 2024-09-12T03:09:15.956 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:09:15.956 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 25, 2024-09-12T03:09:15.956 INFO:teuthology.orchestra.run.smithi116.stdout: "leader": 24494, 2024-09-12T03:09:15.957 INFO:teuthology.orchestra.run.smithi116.stdout: "set_version": 24, 2024-09-12T03:09:15.957 INFO:teuthology.orchestra.run.smithi116.stdout: "sets": { 2024-09-12T03:09:15.957 INFO:teuthology.orchestra.run.smithi116.stdout: "d2a1b25e": { 2024-09-12T03:09:15.957 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 24, 2024-09-12T03:09:15.957 INFO:teuthology.orchestra.run.smithi116.stdout: "age_ref": 0.0, 2024-09-12T03:09:15.957 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:09:15.957 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RELEASED", 2024-09-12T03:09:15.957 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:09:15.957 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:09:15.957 INFO:teuthology.orchestra.run.smithi116.stdout: "timeout": 90.0, 2024-09-12T03:09:15.957 INFO:teuthology.orchestra.run.smithi116.stdout: "expiration": 140.1, 2024-09-12T03:09:15.957 INFO:teuthology.orchestra.run.smithi116.stdout: "members": { 2024-09-12T03:09:15.957 INFO:teuthology.orchestra.run.smithi116.stdout: "file:/": { 2024-09-12T03:09:15.957 INFO:teuthology.orchestra.run.smithi116.stdout: "excluded": false, 2024-09-12T03:09:15.958 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:09:15.958 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RELEASED", 2024-09-12T03:09:15.958 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:09:15.958 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:09:15.958 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:09:15.958 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:09:15.958 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:09:15.958 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:09:15.958 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:09:15.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.952+0000 7f891c8a3640 1 -- 172.21.15.116:0/561719249 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f89000a2c00 msgr2=0x7f89000a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:15.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.952+0000 7f891c8a3640 1 --2- 172.21.15.116:0/561719249 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f89000a2c00 0x7f89000a50c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8910005fe0 tx=0x7f8910005c50 comp rx=0 tx=0).stop 2024-09-12T03:09:15.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.952+0000 7f891c8a3640 1 -- 172.21.15.116:0/561719249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8918102570 msgr2=0x7f89180fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:15.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.952+0000 7f891c8a3640 1 --2- 172.21.15.116:0/561719249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8918102570 0x7f89180fd1c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f890c00c920 tx=0x7f890c00cde0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.952+0000 7f891c8a3640 1 -- 172.21.15.116:0/561719249 shutdown_connections 2024-09-12T03:09:15.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.952+0000 7f891c8a3640 1 --2- 172.21.15.116:0/561719249 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f89000a2c00 0x7f89000a50c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.952+0000 7f891c8a3640 1 --2- 172.21.15.116:0/561719249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8918102ef0 0x7f89180fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.952+0000 7f891c8a3640 1 --2- 172.21.15.116:0/561719249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8918102570 0x7f89180fd1c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.952+0000 7f891c8a3640 1 --2- 172.21.15.116:0/561719249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8918100e20 0x7f89180fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:15.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.952+0000 7f891c8a3640 1 -- 172.21.15.116:0/561719249 >> 172.21.15.116:0/561719249 conn(0x7f89180fc7d0 msgr2=0x7f891806a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:15.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.952+0000 7f891c8a3640 1 -- 172.21.15.116:0/561719249 shutdown_connections 2024-09-12T03:09:15.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:15.952+0000 7f891c8a3640 1 -- 172.21.15.116:0/561719249 wait complete. 2024-09-12T03:09:16.630 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:16 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:09:16.630 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:16 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:09:16.631 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:16 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:09:16.631 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:16 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:09:16.631 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:16 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:09:16.631 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:16 smithi116 ceph-mon[31410]: Adjusting osd_memory_target on smithi116 to 176.0M 2024-09-12T03:09:16.631 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:16 smithi116 ceph-mon[31410]: Unable to set osd_memory_target on smithi116 to 184577433: error parsing value: Value '184577433' is below minimum 939524096 2024-09-12T03:09:16.631 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:16 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:09:16.631 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:16 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:09:16.631 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:16 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:09:16.645 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:16 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:09:16.646 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:16 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:09:16.646 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:16 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:09:16.646 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:16 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:09:16.646 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:16 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:09:16.646 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:16 smithi177 ceph-mon[35305]: Adjusting osd_memory_target on smithi116 to 176.0M 2024-09-12T03:09:16.646 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:16 smithi177 ceph-mon[35305]: Unable to set osd_memory_target on smithi116 to 184577433: error parsing value: Value '184577433' is below minimum 939524096 2024-09-12T03:09:16.646 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:16 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:09:16.646 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:16 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:09:16.646 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:16 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:09:16.656 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd2a1b25e', total seconds elapsed: 23.8 2024-09-12T03:09:16.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:16 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:09:16.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:16 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:09:16.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:16 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:09:16.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:16 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:09:16.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:16 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:09:16.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:16 smithi183 ceph-mon[35535]: Adjusting osd_memory_target on smithi116 to 176.0M 2024-09-12T03:09:16.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:16 smithi183 ceph-mon[35535]: Unable to set osd_memory_target on smithi116 to 184577433: error parsing value: Value '184577433' is below minimum 939524096 2024-09-12T03:09:16.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:16 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:09:16.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:16 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:09:16.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:16 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:09:17.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:17 smithi183 ceph-mon[35535]: pgmap v527: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-09-12T03:09:17.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:17 smithi177 ceph-mon[35305]: pgmap v527: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-09-12T03:09:17.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:17 smithi116 ceph-mon[31410]: pgmap v527: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-09-12T03:09:20.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:20 smithi116 ceph-mon[31410]: pgmap v528: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-09-12T03:09:20.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:20 smithi183 ceph-mon[35535]: pgmap v528: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-09-12T03:09:20.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:20 smithi177 ceph-mon[35305]: pgmap v528: 97 pgs: 97 active+clean; 3.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-09-12T03:09:22.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:22 smithi116 ceph-mon[31410]: pgmap v529: 97 pgs: 97 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-09-12T03:09:22.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:22 smithi183 ceph-mon[35535]: pgmap v529: 97 pgs: 97 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-09-12T03:09:22.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:22 smithi177 ceph-mon[35305]: pgmap v529: 97 pgs: 97 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-09-12T03:09:24.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:24 smithi116 ceph-mon[31410]: pgmap v530: 97 pgs: 97 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-09-12T03:09:24.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:24 smithi183 ceph-mon[35535]: pgmap v530: 97 pgs: 97 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-09-12T03:09:24.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:24 smithi177 ceph-mon[35305]: pgmap v530: 97 pgs: 97 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-09-12T03:09:26.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:26 smithi116 ceph-mon[31410]: pgmap v531: 97 pgs: 97 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-09-12T03:09:26.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:26 smithi183 ceph-mon[35535]: pgmap v531: 97 pgs: 97 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-09-12T03:09:26.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:26 smithi177 ceph-mon[35305]: pgmap v531: 97 pgs: 97 active+clean; 3.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-09-12T03:09:28.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:28 smithi183 ceph-mon[35535]: pgmap v532: 97 pgs: 97 active+clean; 3.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T03:09:28.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:28 smithi177 ceph-mon[35305]: pgmap v532: 97 pgs: 97 active+clean; 3.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T03:09:28.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:28 smithi116 ceph-mon[31410]: pgmap v532: 97 pgs: 97 active+clean; 3.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T03:09:30.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:30 smithi183 ceph-mon[35535]: pgmap v533: 97 pgs: 97 active+clean; 3.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:09:30.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:30 smithi177 ceph-mon[35305]: pgmap v533: 97 pgs: 97 active+clean; 3.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:09:30.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:30 smithi116 ceph-mon[31410]: pgmap v533: 97 pgs: 97 active+clean; 3.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:09:32.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:32 smithi183 ceph-mon[35535]: pgmap v534: 97 pgs: 97 active+clean; 3.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:09:32.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:32 smithi177 ceph-mon[35305]: pgmap v534: 97 pgs: 97 active+clean; 3.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:09:32.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:32 smithi116 ceph-mon[31410]: pgmap v534: 97 pgs: 97 active+clean; 3.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:09:34.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:34 smithi183 ceph-mon[35535]: pgmap v535: 97 pgs: 97 active+clean; 3.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:09:34.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:34 smithi177 ceph-mon[35305]: pgmap v535: 97 pgs: 97 active+clean; 3.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:09:34.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:34 smithi116 ceph-mon[31410]: pgmap v535: 97 pgs: 97 active+clean; 3.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:09:36.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:36 smithi183 ceph-mon[35535]: pgmap v536: 97 pgs: 97 active+clean; 3.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:09:36.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:36 smithi177 ceph-mon[35305]: pgmap v536: 97 pgs: 97 active+clean; 3.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:09:36.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:36 smithi116 ceph-mon[31410]: pgmap v536: 97 pgs: 97 active+clean; 3.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:09:38.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:38 smithi183 ceph-mon[35535]: pgmap v537: 97 pgs: 97 active+clean; 3.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:09:38.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:38 smithi177 ceph-mon[35305]: pgmap v537: 97 pgs: 97 active+clean; 3.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:09:38.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:38 smithi116 ceph-mon[31410]: pgmap v537: 97 pgs: 97 active+clean; 3.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:09:40.605 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:40 smithi116 ceph-mon[31410]: pgmap v538: 97 pgs: 97 active+clean; 3.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:09:40.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:40 smithi183 ceph-mon[35535]: pgmap v538: 97 pgs: 97 active+clean; 3.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:09:40.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:40 smithi177 ceph-mon[35305]: pgmap v538: 97 pgs: 97 active+clean; 3.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:09:42.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:42 smithi183 ceph-mon[35535]: pgmap v539: 97 pgs: 97 active+clean; 3.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:09:42.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:42 smithi177 ceph-mon[35305]: pgmap v539: 97 pgs: 97 active+clean; 3.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:09:42.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:42 smithi116 ceph-mon[31410]: pgmap v539: 97 pgs: 97 active+clean; 3.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:09:43.581 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:09:43.835 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:09:44.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.325+0000 7f5159e58640 1 -- 172.21.15.116:0/1762979093 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f515410c180 msgr2=0x7f515410e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:44.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.325+0000 7f5159e58640 1 --2- 172.21.15.116:0/1762979093 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f515410c180 0x7f515410e600 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f514c009900 tx=0x7f514c02a480 comp rx=0 tx=0).stop 2024-09-12T03:09:44.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.325+0000 7f5159e58640 1 -- 172.21.15.116:0/1762979093 shutdown_connections 2024-09-12T03:09:44.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.325+0000 7f5159e58640 1 --2- 172.21.15.116:0/1762979093 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f515410c180 0x7f515410e600 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.325+0000 7f5159e58640 1 --2- 172.21.15.116:0/1762979093 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51541036c0 0x7f515410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.325+0000 7f5159e58640 1 --2- 172.21.15.116:0/1762979093 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5154102da0 0x7f5154103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.325+0000 7f5159e58640 1 -- 172.21.15.116:0/1762979093 >> 172.21.15.116:0/1762979093 conn(0x7f51540fc7a0 msgr2=0x7f51540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:44.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.325+0000 7f5159e58640 1 -- 172.21.15.116:0/1762979093 shutdown_connections 2024-09-12T03:09:44.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.325+0000 7f5159e58640 1 -- 172.21.15.116:0/1762979093 wait complete. 2024-09-12T03:09:44.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.326+0000 7f5159e58640 1 Processor -- start 2024-09-12T03:09:44.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.326+0000 7f5159e58640 1 -- start start 2024-09-12T03:09:44.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.326+0000 7f5159e58640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5154102da0 0x7f5154100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:44.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.326+0000 7f5159e58640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51541036c0 0x7f5154100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:44.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.326+0000 7f5159e58640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f515410c180 0x7f5154101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:44.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.326+0000 7f5159e58640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5154111f60 con 0x7f51541036c0 2024-09-12T03:09:44.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.326+0000 7f5159e58640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5154111de0 con 0x7f5154102da0 2024-09-12T03:09:44.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.326+0000 7f5159e58640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f51541120e0 con 0x7f515410c180 2024-09-12T03:09:44.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.326+0000 7f5152ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51541036c0 0x7f5154100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:44.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.326+0000 7f5152ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51541036c0 0x7f5154100f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58338/0 (socket says 172.21.15.116:58338) 2024-09-12T03:09:44.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.326+0000 7f5152ffd640 1 -- 172.21.15.116:0/71552499 learned_addr learned my addr 172.21.15.116:0/71552499 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:44.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.326+0000 7f51537fe640 1 --2- 172.21.15.116:0/71552499 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5154102da0 0x7f5154100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:44.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.326+0000 7f5153fff640 1 --2- 172.21.15.116:0/71552499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f515410c180 0x7f5154101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:44.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.326+0000 7f5152ffd640 1 -- 172.21.15.116:0/71552499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f515410c180 msgr2=0x7f5154101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:44.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.326+0000 7f5152ffd640 1 --2- 172.21.15.116:0/71552499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f515410c180 0x7f5154101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.326+0000 7f5152ffd640 1 -- 172.21.15.116:0/71552499 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5154102da0 msgr2=0x7f5154100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:44.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.326+0000 7f5152ffd640 1 --2- 172.21.15.116:0/71552499 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5154102da0 0x7f5154100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.326+0000 7f5152ffd640 1 -- 172.21.15.116:0/71552499 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51541a6880 con 0x7f51541036c0 2024-09-12T03:09:44.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.326+0000 7f5153fff640 1 --2- 172.21.15.116:0/71552499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f515410c180 0x7f5154101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:09:44.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.327+0000 7f5152ffd640 1 --2- 172.21.15.116:0/71552499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51541036c0 0x7f5154100f40 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f514800d950 tx=0x7f514800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:44.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.327+0000 7f5150ff9640 1 -- 172.21.15.116:0/71552499 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5148017070 con 0x7f51541036c0 2024-09-12T03:09:44.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.327+0000 7f5159e58640 1 -- 172.21.15.116:0/71552499 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51541a6b70 con 0x7f51541036c0 2024-09-12T03:09:44.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.327+0000 7f5150ff9640 1 -- 172.21.15.116:0/71552499 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5148009c30 con 0x7f51541036c0 2024-09-12T03:09:44.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.327+0000 7f5150ff9640 1 -- 172.21.15.116:0/71552499 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514800f6b0 con 0x7f51541036c0 2024-09-12T03:09:44.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.327+0000 7f5159e58640 1 -- 172.21.15.116:0/71552499 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51541a70b0 con 0x7f51541036c0 2024-09-12T03:09:44.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.330+0000 7f515c231740 1 Processor -- start 2024-09-12T03:09:44.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.330+0000 7f515c231740 1 -- start start 2024-09-12T03:09:44.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.330+0000 7f515c231740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb18914a0 0x55bbb1891880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:44.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.330+0000 7f515c231740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb1891dc0 0x55bbb189c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:44.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.330+0000 7f515c231740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb189c680 0x55bbb189ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:44.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.330+0000 7f515c231740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bbb17f72e0 con 0x55bbb18914a0 2024-09-12T03:09:44.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.330+0000 7f515c231740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bbb179ece0 con 0x55bbb189c680 2024-09-12T03:09:44.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.330+0000 7f515c231740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bbb17f32a0 con 0x55bbb1891dc0 2024-09-12T03:09:44.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.330+0000 7f5153fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb18914a0 0x55bbb1891880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:44.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.330+0000 7f5153fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb18914a0 0x55bbb1891880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58348/0 (socket says 172.21.15.116:58348) 2024-09-12T03:09:44.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.330+0000 7f5153fff640 1 -- 172.21.15.116:0/1804089866 learned_addr learned my addr 172.21.15.116:0/1804089866 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:44.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.331+0000 7f51537fe640 1 --2- 172.21.15.116:0/1804089866 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb1891dc0 0x55bbb189c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:44.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.331+0000 7f5152ffd640 1 --2- 172.21.15.116:0/1804089866 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb189c680 0x55bbb189ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:44.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.331+0000 7f5153fff640 1 -- 172.21.15.116:0/1804089866 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb1891dc0 msgr2=0x55bbb189c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:44.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.331+0000 7f5153fff640 1 --2- 172.21.15.116:0/1804089866 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb1891dc0 0x55bbb189c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.331+0000 7f5153fff640 1 -- 172.21.15.116:0/1804089866 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb189c680 msgr2=0x55bbb189ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:44.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.331+0000 7f5153fff640 1 --2- 172.21.15.116:0/1804089866 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb189c680 0x55bbb189ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.331+0000 7f5153fff640 1 -- 172.21.15.116:0/1804089866 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbb1783440 con 0x55bbb18914a0 2024-09-12T03:09:44.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.331+0000 7f51537fe640 1 --2- 172.21.15.116:0/1804089866 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb1891dc0 0x55bbb189c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:09:44.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.331+0000 7f5152ffd640 1 --2- 172.21.15.116:0/1804089866 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb189c680 0x55bbb189ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:09:44.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.331+0000 7f5153fff640 1 --2- 172.21.15.116:0/1804089866 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb18914a0 0x55bbb1891880 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f514c005c40 tx=0x7f514c032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c0ad696e719082 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:44.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.331+0000 7f5139ffb640 1 -- 172.21.15.116:0/1804089866 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c03f070 con 0x55bbb18914a0 2024-09-12T03:09:44.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.331+0000 7f5139ffb640 1 -- 172.21.15.116:0/1804089866 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f514c0317b0 con 0x55bbb18914a0 2024-09-12T03:09:44.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.331+0000 7f5139ffb640 1 -- 172.21.15.116:0/1804089866 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c035070 con 0x55bbb18914a0 2024-09-12T03:09:44.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.332+0000 7f515c231740 1 -- 172.21.15.116:0/1804089866 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb18914a0 msgr2=0x55bbb1891880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:44.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.332+0000 7f515c231740 1 --2- 172.21.15.116:0/1804089866 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb18914a0 0x55bbb1891880 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f514c005c40 tx=0x7f514c032cb0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.332+0000 7f515c231740 1 -- 172.21.15.116:0/1804089866 shutdown_connections 2024-09-12T03:09:44.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.332+0000 7f515c231740 1 --2- 172.21.15.116:0/1804089866 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb189c680 0x55bbb189ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.332+0000 7f515c231740 1 --2- 172.21.15.116:0/1804089866 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb1891dc0 0x55bbb189c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.332+0000 7f515c231740 1 --2- 172.21.15.116:0/1804089866 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb18914a0 0x55bbb1891880 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.332+0000 7f515c231740 1 -- 172.21.15.116:0/1804089866 >> 172.21.15.116:0/1804089866 conn(0x55bbb188da30 msgr2=0x55bbb188de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:44.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.333+0000 7f5150ff9640 1 -- 172.21.15.116:0/71552499 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f51480109d0 con 0x7f51541036c0 2024-09-12T03:09:44.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.333+0000 7f515c231740 1 -- 172.21.15.116:0/1804089866 shutdown_connections 2024-09-12T03:09:44.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.333+0000 7f515c231740 1 -- 172.21.15.116:0/1804089866 wait complete. 2024-09-12T03:09:44.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.333+0000 7f515c231740 1 -- start start 2024-09-12T03:09:44.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.333+0000 7f515c231740 1 Processor -- start 2024-09-12T03:09:44.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.334+0000 7f5150ff9640 1 --2- 172.21.15.116:0/71552499 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f513c0a2a90 0x7f513c0a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:44.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.334+0000 7f515c231740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb18914a0 0x55bbb189bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:44.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.334+0000 7f515c231740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb1891dc0 0x55bbb18991c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:44.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.334+0000 7f515c231740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb189c680 0x55bbb1899700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:44.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.334+0000 7f515c231740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bbb189f250 con 0x55bbb1891dc0 2024-09-12T03:09:44.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.334+0000 7f515c231740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bbb17b2050 con 0x55bbb189c680 2024-09-12T03:09:44.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.334+0000 7f515c231740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bbb189f3d0 con 0x55bbb18914a0 2024-09-12T03:09:44.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.334+0000 7f5153fff640 1 --2- 172.21.15.116:0/71552499 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f513c0a2a90 0x7f513c0a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:44.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.334+0000 7f5150ff9640 1 -- 172.21.15.116:0/71552499 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f51480cebc0 con 0x7f51541036c0 2024-09-12T03:09:44.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.334+0000 7f515c231740 10 client.0 _ll_register_callbacks cb 0x55bbb17f2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:44.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.334+0000 7f51537fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb18914a0 0x55bbb189bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:44.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.334+0000 7f51537fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb18914a0 0x55bbb189bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.116:56406/0 (socket says 172.21.15.116:56406) 2024-09-12T03:09:44.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.334+0000 7f51537fe640 1 -- 172.21.15.116:0/4201581136 learned_addr learned my addr 172.21.15.116:0/4201581136 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:44.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.334+0000 7f5152ffd640 1 --2- 172.21.15.116:0/4201581136 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb1891dc0 0x55bbb18991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:44.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.334+0000 7f5153fff640 1 --2- 172.21.15.116:0/4201581136 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb189c680 0x55bbb1899700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:44.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.334+0000 7f5152ffd640 1 -- 172.21.15.116:0/4201581136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb18914a0 msgr2=0x55bbb189bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:44.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.334+0000 7f5152ffd640 1 --2- 172.21.15.116:0/4201581136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb18914a0 0x55bbb189bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.334+0000 7f5152ffd640 1 -- 172.21.15.116:0/4201581136 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb189c680 msgr2=0x55bbb1899700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:44.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.334+0000 7f5152ffd640 1 --2- 172.21.15.116:0/4201581136 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb189c680 0x55bbb1899700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.334+0000 7f5152ffd640 1 -- 172.21.15.116:0/4201581136 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbb15683c0 con 0x55bbb1891dc0 2024-09-12T03:09:44.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.334+0000 7f5153fff640 1 --2- 172.21.15.116:0/4201581136 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb189c680 0x55bbb1899700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:09:44.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.334+0000 7f51537fe640 1 --2- 172.21.15.116:0/4201581136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb18914a0 0x55bbb189bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:09:44.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.334+0000 7f5153fff640 1 --2- 172.21.15.116:0/71552499 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f513c0a2a90 0x7f513c0a4f50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f514c005270 tx=0x7f514c0052d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:44.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.334+0000 7f5152ffd640 1 --2- 172.21.15.116:0/4201581136 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb1891dc0 0x55bbb18991c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f51480cc6f0 tx=0x7f514809ae70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ba5fa6ced37de9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:44.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.334+0000 7f5132ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:09:44.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.335+0000 7f5132ffd640 1 -- 172.21.15.116:0/4201581136 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5148020070 con 0x55bbb1891dc0 2024-09-12T03:09:44.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.335+0000 7f5132ffd640 1 -- 172.21.15.116:0/4201581136 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f51480ccbb0 con 0x55bbb1891dc0 2024-09-12T03:09:44.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.335+0000 7f515c231740 1 -- 172.21.15.116:0/4201581136 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f514c0317b0 con 0x55bbb1891dc0 2024-09-12T03:09:44.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.335+0000 7f5132ffd640 1 -- 172.21.15.116:0/4201581136 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5148013230 con 0x55bbb1891dc0 2024-09-12T03:09:44.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.335+0000 7f515c231740 10 client.16944 fetch_fsmap learned FSMap version 25 2024-09-12T03:09:44.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.335+0000 7f515c231740 1 -- 172.21.15.116:0/4201581136 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bbb18aaba0 con 0x55bbb1891dc0 2024-09-12T03:09:44.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.335+0000 7f5132ffd640 1 -- 172.21.15.116:0/4201581136 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f514c0317b0 con 0x55bbb1891dc0 2024-09-12T03:09:44.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.335+0000 7f5132ffd640 1 -- 172.21.15.116:0/4201581136 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f51480b4230 con 0x55bbb1891dc0 2024-09-12T03:09:44.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.335+0000 7f515c231740 10 client.16944 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:09:44.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.335+0000 7f515c231740 10 client.16944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:09:44.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.335+0000 7f515c231740 20 client.16944 populate_metadata read hostname 'smithi116' 2024-09-12T03:09:44.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.335+0000 7f515c231740 1 --2- 172.21.15.116:0/4201581136 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bbb18ac060 0x55bbb18ae450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:44.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.335+0000 7f515c231740 4 client.16944 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:09:44.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.335+0000 7f515c231740 1 -- 172.21.15.116:0/4201581136 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbb1804850 con 0x55bbb18ac060 2024-09-12T03:09:44.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.335+0000 7f51537fe640 1 --2- 172.21.15.116:0/4201581136 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bbb18ac060 0x55bbb18ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:44.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.336+0000 7f5132ffd640 10 client.16944 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:44.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.336+0000 7f51537fe640 1 --2- 172.21.15.116:0/4201581136 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bbb18ac060 0x55bbb18ae450 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f047408beac9442 server_cookie=420ef199bdf4ad3a in_seq=0 out_seq=0 2024-09-12T03:09:44.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.337+0000 7f5132ffd640 1 -- 172.21.15.116:0/4201581136 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bbb1804850 con 0x55bbb18ac060 2024-09-12T03:09:44.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.337+0000 7f5132ffd640 10 client.16944 handle_command_reply: tid=0 2024-09-12T03:09:44.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.337+0000 7f515c231740 1 client.16944 shutdown 2024-09-12T03:09:44.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.337+0000 7f515c231740 1 -- 172.21.15.116:0/4201581136 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb1891dc0 msgr2=0x55bbb18991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:44.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.337+0000 7f515c231740 1 --2- 172.21.15.116:0/4201581136 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb1891dc0 0x55bbb18991c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f51480cc6f0 tx=0x7f514809ae70 comp rx=0 tx=0).stop 2024-09-12T03:09:44.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.337+0000 7f515c231740 1 -- 172.21.15.116:0/4201581136 shutdown_connections 2024-09-12T03:09:44.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.337+0000 7f515c231740 1 --2- 172.21.15.116:0/4201581136 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bbb18ac060 0x55bbb18ae450 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.337+0000 7f515c231740 1 --2- 172.21.15.116:0/4201581136 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb189c680 0x55bbb1899700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.337+0000 7f515c231740 1 --2- 172.21.15.116:0/4201581136 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb1891dc0 0x55bbb18991c0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.337+0000 7f5132ffd640 0 client.16944 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:44.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.337+0000 7f515c231740 1 --2- 172.21.15.116:0/4201581136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb18914a0 0x55bbb189bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.337+0000 7f515c231740 1 -- 172.21.15.116:0/4201581136 >> 172.21.15.116:0/4201581136 conn(0x55bbb188da30 msgr2=0x55bbb188cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:44.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.337+0000 7f515c231740 1 -- 172.21.15.116:0/4201581136 shutdown_connections 2024-09-12T03:09:44.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.337+0000 7f515c231740 1 -- 172.21.15.116:0/4201581136 wait complete. 2024-09-12T03:09:44.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.337+0000 7f515c231740 20 client.16944 trim_cache size 0 max 16384 2024-09-12T03:09:44.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.354+0000 7f515c231740 1 Processor -- start 2024-09-12T03:09:44.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.354+0000 7f515c231740 1 -- start start 2024-09-12T03:09:44.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.354+0000 7f515c231740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb18914a0 0x55bbb1897a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:44.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.354+0000 7f515c231740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb1891dc0 0x55bbb18b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:44.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.354+0000 7f515c231740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb189c680 0x55bbb18b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:44.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.354+0000 7f515c231740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f51541b1f10 con 0x55bbb189c680 2024-09-12T03:09:44.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.354+0000 7f515c231740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f51541af7e0 con 0x55bbb1891dc0 2024-09-12T03:09:44.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.354+0000 7f515c231740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f51541ad630 con 0x55bbb18914a0 2024-09-12T03:09:44.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.355+0000 7f5153fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb189c680 0x55bbb18b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:44.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.355+0000 7f5153fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb189c680 0x55bbb18b41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58370/0 (socket says 172.21.15.116:58370) 2024-09-12T03:09:44.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.355+0000 7f5153fff640 1 -- 172.21.15.116:0/1115948178 learned_addr learned my addr 172.21.15.116:0/1115948178 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:44.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.355+0000 7f51537fe640 1 --2- 172.21.15.116:0/1115948178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb18914a0 0x55bbb1897a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:44.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.355+0000 7f5152ffd640 1 --2- 172.21.15.116:0/1115948178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb1891dc0 0x55bbb18b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:44.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.355+0000 7f5153fff640 1 -- 172.21.15.116:0/1115948178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb18914a0 msgr2=0x55bbb1897a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:44.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.355+0000 7f5153fff640 1 --2- 172.21.15.116:0/1115948178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb18914a0 0x55bbb1897a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.355+0000 7f5153fff640 1 -- 172.21.15.116:0/1115948178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb1891dc0 msgr2=0x55bbb18b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:44.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.355+0000 7f5153fff640 1 --2- 172.21.15.116:0/1115948178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb1891dc0 0x55bbb18b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.355+0000 7f5153fff640 1 -- 172.21.15.116:0/1115948178 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbb1898170 con 0x55bbb189c680 2024-09-12T03:09:44.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.355+0000 7f5153fff640 1 --2- 172.21.15.116:0/1115948178 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb189c680 0x55bbb18b41e0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f514c032d20 tx=0x7f514c04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a29f39fefcb6f4b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:44.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.355+0000 7f51397fa640 1 -- 172.21.15.116:0/1115948178 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c03f030 con 0x55bbb189c680 2024-09-12T03:09:44.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.355+0000 7f51397fa640 1 -- 172.21.15.116:0/1115948178 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f514c0054a0 con 0x55bbb189c680 2024-09-12T03:09:44.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.355+0000 7f51397fa640 1 -- 172.21.15.116:0/1115948178 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c0049b0 con 0x55bbb189c680 2024-09-12T03:09:44.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.356+0000 7f515c231740 1 -- 172.21.15.116:0/1115948178 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb189c680 msgr2=0x55bbb18b41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:44.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.356+0000 7f515c231740 1 --2- 172.21.15.116:0/1115948178 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb189c680 0x55bbb18b41e0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f514c032d20 tx=0x7f514c04c000 comp rx=0 tx=0).stop 2024-09-12T03:09:44.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.356+0000 7f515c231740 1 -- 172.21.15.116:0/1115948178 shutdown_connections 2024-09-12T03:09:44.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.356+0000 7f515c231740 1 --2- 172.21.15.116:0/1115948178 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb189c680 0x55bbb18b41e0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.356+0000 7f515c231740 1 --2- 172.21.15.116:0/1115948178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb1891dc0 0x55bbb18b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.356+0000 7f515c231740 1 --2- 172.21.15.116:0/1115948178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb18914a0 0x55bbb1897a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.356+0000 7f515c231740 1 -- 172.21.15.116:0/1115948178 >> 172.21.15.116:0/1115948178 conn(0x55bbb188da30 msgr2=0x55bbb1893710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:44.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.356+0000 7f515c231740 1 -- 172.21.15.116:0/1115948178 shutdown_connections 2024-09-12T03:09:44.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.356+0000 7f515c231740 1 -- 172.21.15.116:0/1115948178 wait complete. 2024-09-12T03:09:44.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.356+0000 7f515c231740 1 -- start start 2024-09-12T03:09:44.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.357+0000 7f515c231740 1 Processor -- start 2024-09-12T03:09:44.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.357+0000 7f515c231740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb18914a0 0x55bbb18a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:44.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.357+0000 7f515c231740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb1891dc0 0x55bbb1893a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:44.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.357+0000 7f515c231740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb189c680 0x55bbb1893f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:44.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.357+0000 7f515c231740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f51541adca0 con 0x55bbb1891dc0 2024-09-12T03:09:44.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.357+0000 7f515c231740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f51541af110 con 0x55bbb189c680 2024-09-12T03:09:44.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.357+0000 7f515c231740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f51541ae370 con 0x55bbb18914a0 2024-09-12T03:09:44.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.357+0000 7f515c231740 10 client.0 _ll_register_callbacks cb 0x7f514809af90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:44.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.357+0000 7f5152ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb1891dc0 0x55bbb1893a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:44.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.357+0000 7f5152ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb1891dc0 0x55bbb1893a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58372/0 (socket says 172.21.15.116:58372) 2024-09-12T03:09:44.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.357+0000 7f5152ffd640 1 -- 172.21.15.116:0/2698464524 learned_addr learned my addr 172.21.15.116:0/2698464524 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:44.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.357+0000 7f51537fe640 1 --2- 172.21.15.116:0/2698464524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb18914a0 0x55bbb18a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:44.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.357+0000 7f5153fff640 1 --2- 172.21.15.116:0/2698464524 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb189c680 0x55bbb1893f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:44.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.357+0000 7f5152ffd640 1 -- 172.21.15.116:0/2698464524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb18914a0 msgr2=0x55bbb18a4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:44.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.357+0000 7f5152ffd640 1 --2- 172.21.15.116:0/2698464524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb18914a0 0x55bbb18a4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.357+0000 7f5152ffd640 1 -- 172.21.15.116:0/2698464524 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb189c680 msgr2=0x55bbb1893f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:44.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.357+0000 7f5152ffd640 1 --2- 172.21.15.116:0/2698464524 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb189c680 0x55bbb1893f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.357+0000 7f5152ffd640 1 -- 172.21.15.116:0/2698464524 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbb1894740 con 0x55bbb1891dc0 2024-09-12T03:09:44.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.357+0000 7f5153fff640 1 --2- 172.21.15.116:0/2698464524 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb189c680 0x55bbb1893f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:09:44.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.357+0000 7f5152ffd640 1 --2- 172.21.15.116:0/2698464524 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb1891dc0 0x55bbb1893a00 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f514809f940 tx=0x7f5148009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcac39086afb48cd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:44.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.357+0000 7f5132ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:09:44.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.358+0000 7f5132ffd640 1 -- 172.21.15.116:0/2698464524 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5148020070 con 0x55bbb1891dc0 2024-09-12T03:09:44.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.358+0000 7f5132ffd640 1 -- 172.21.15.116:0/2698464524 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f5148097ac0 con 0x55bbb1891dc0 2024-09-12T03:09:44.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.358+0000 7f515c231740 1 -- 172.21.15.116:0/2698464524 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f514c0054a0 con 0x55bbb1891dc0 2024-09-12T03:09:44.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.358+0000 7f5132ffd640 1 -- 172.21.15.116:0/2698464524 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f514801ccb0 con 0x55bbb1891dc0 2024-09-12T03:09:44.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.358+0000 7f515c231740 10 client.16950 fetch_fsmap learned FSMap version 25 2024-09-12T03:09:44.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.358+0000 7f515c231740 1 -- 172.21.15.116:0/2698464524 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bbb1916c70 con 0x55bbb1891dc0 2024-09-12T03:09:44.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.358+0000 7f5132ffd640 1 -- 172.21.15.116:0/2698464524 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f514c0054a0 con 0x55bbb1891dc0 2024-09-12T03:09:44.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.358+0000 7f5132ffd640 1 -- 172.21.15.116:0/2698464524 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5148010460 con 0x55bbb1891dc0 2024-09-12T03:09:44.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.358+0000 7f515c231740 10 client.16950 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:09:44.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.358+0000 7f515c231740 10 client.16950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:09:44.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.358+0000 7f515c231740 20 client.16950 populate_metadata read hostname 'smithi116' 2024-09-12T03:09:44.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.358+0000 7f515c231740 1 --2- 172.21.15.116:0/2698464524 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bbb19172f0 0x55bbb1919710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:44.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.358+0000 7f515c231740 4 client.16950 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:09:44.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.358+0000 7f515c231740 1 -- 172.21.15.116:0/2698464524 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bbb17fed60 con 0x55bbb19172f0 2024-09-12T03:09:44.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.359+0000 7f51537fe640 1 --2- 172.21.15.116:0/2698464524 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bbb19172f0 0x55bbb1919710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:44.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.359+0000 7f5132ffd640 10 client.16950 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:44.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.359+0000 7f51537fe640 1 --2- 172.21.15.116:0/2698464524 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bbb19172f0 0x55bbb1919710 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f69cc26f80dbc9e server_cookie=111c695e7c22a76 in_seq=0 out_seq=0 2024-09-12T03:09:44.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.359+0000 7f5132ffd640 1 -- 172.21.15.116:0/2698464524 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bbb17fed60 con 0x55bbb19172f0 2024-09-12T03:09:44.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.359+0000 7f5132ffd640 10 client.16950 handle_command_reply: tid=0 2024-09-12T03:09:44.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.359+0000 7f515c231740 1 client.16950 shutdown 2024-09-12T03:09:44.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.359+0000 7f515c231740 1 -- 172.21.15.116:0/2698464524 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb1891dc0 msgr2=0x55bbb1893a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:44.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.359+0000 7f515c231740 1 --2- 172.21.15.116:0/2698464524 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb1891dc0 0x55bbb1893a00 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f514809f940 tx=0x7f5148009e70 comp rx=0 tx=0).stop 2024-09-12T03:09:44.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.359+0000 7f515c231740 1 -- 172.21.15.116:0/2698464524 shutdown_connections 2024-09-12T03:09:44.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.359+0000 7f515c231740 1 --2- 172.21.15.116:0/2698464524 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bbb19172f0 0x55bbb1919710 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.359+0000 7f515c231740 1 --2- 172.21.15.116:0/2698464524 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb189c680 0x55bbb1893f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.359+0000 7f515c231740 1 --2- 172.21.15.116:0/2698464524 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb1891dc0 0x55bbb1893a00 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.359+0000 7f515c231740 1 --2- 172.21.15.116:0/2698464524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb18914a0 0x55bbb18a4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.359+0000 7f515c231740 1 -- 172.21.15.116:0/2698464524 >> 172.21.15.116:0/2698464524 conn(0x55bbb188da30 msgr2=0x55bbb188cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:44.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.359+0000 7f5132ffd640 0 client.16950 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:44.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.359+0000 7f515c231740 1 -- 172.21.15.116:0/2698464524 shutdown_connections 2024-09-12T03:09:44.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.359+0000 7f515c231740 1 -- 172.21.15.116:0/2698464524 wait complete. 2024-09-12T03:09:44.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.360+0000 7f515c231740 20 client.16950 trim_cache size 0 max 16384 2024-09-12T03:09:44.371 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:09:44.371 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:09:44.371 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:09:44.371 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:09:44.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.360+0000 7f5159e58640 1 -- 172.21.15.116:0/71552499 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f513c0a2a90 msgr2=0x7f513c0a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:44.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.360+0000 7f5159e58640 1 --2- 172.21.15.116:0/71552499 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f513c0a2a90 0x7f513c0a4f50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f514c005270 tx=0x7f514c0052d0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.360+0000 7f5159e58640 1 -- 172.21.15.116:0/71552499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51541036c0 msgr2=0x7f5154100f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:44.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.360+0000 7f5159e58640 1 --2- 172.21.15.116:0/71552499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51541036c0 0x7f5154100f40 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f514800d950 tx=0x7f514800de10 comp rx=0 tx=0).stop 2024-09-12T03:09:44.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.360+0000 7f5159e58640 1 -- 172.21.15.116:0/71552499 shutdown_connections 2024-09-12T03:09:44.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.360+0000 7f5159e58640 1 --2- 172.21.15.116:0/71552499 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f513c0a2a90 0x7f513c0a4f50 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.360+0000 7f5159e58640 1 --2- 172.21.15.116:0/71552499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f515410c180 0x7f5154101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.360+0000 7f5159e58640 1 --2- 172.21.15.116:0/71552499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51541036c0 0x7f5154100f40 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.360+0000 7f5159e58640 1 --2- 172.21.15.116:0/71552499 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5154102da0 0x7f5154100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:44.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.360+0000 7f5159e58640 1 -- 172.21.15.116:0/71552499 >> 172.21.15.116:0/71552499 conn(0x7f51540fc7a0 msgr2=0x7f51540fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:44.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.360+0000 7f5159e58640 1 -- 172.21.15.116:0/71552499 shutdown_connections 2024-09-12T03:09:44.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:44.360+0000 7f5159e58640 1 -- 172.21.15.116:0/71552499 wait complete. 2024-09-12T03:09:44.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:44 smithi116 ceph-mon[31410]: pgmap v540: 97 pgs: 97 active+clean; 3.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:09:44.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:44 smithi183 ceph-mon[35535]: pgmap v540: 97 pgs: 97 active+clean; 3.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:09:44.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:44 smithi177 ceph-mon[35305]: pgmap v540: 97 pgs: 97 active+clean; 3.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:09:45.207 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:09:45.207 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:09:45.454 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:09:45.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.930+0000 7fc420267640 1 -- 172.21.15.116:0/3803589183 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc418100a80 msgr2=0x7fc418100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:45.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.930+0000 7fc420267640 1 --2- 172.21.15.116:0/3803589183 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc418100a80 0x7fc418100ee0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fc414009930 tx=0x7fc41402a420 comp rx=0 tx=0).stop 2024-09-12T03:09:45.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.930+0000 7fc420267640 1 -- 172.21.15.116:0/3803589183 shutdown_connections 2024-09-12T03:09:45.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.930+0000 7fc420267640 1 --2- 172.21.15.116:0/3803589183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc418101420 0x7fc41810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:45.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.930+0000 7fc420267640 1 --2- 172.21.15.116:0/3803589183 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc418100a80 0x7fc418100ee0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:45.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.930+0000 7fc420267640 1 --2- 172.21.15.116:0/3803589183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc418106a80 0x7fc418106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:45.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.930+0000 7fc420267640 1 -- 172.21.15.116:0/3803589183 >> 172.21.15.116:0/3803589183 conn(0x7fc4180fc7a0 msgr2=0x7fc4180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:45.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.930+0000 7fc420267640 1 -- 172.21.15.116:0/3803589183 shutdown_connections 2024-09-12T03:09:45.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.930+0000 7fc420267640 1 -- 172.21.15.116:0/3803589183 wait complete. 2024-09-12T03:09:45.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.931+0000 7fc420267640 1 Processor -- start 2024-09-12T03:09:45.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.931+0000 7fc420267640 1 -- start start 2024-09-12T03:09:45.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.931+0000 7fc420267640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc418100a80 0x7fc41819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:45.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.931+0000 7fc420267640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc418101420 0x7fc41819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:45.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.931+0000 7fc420267640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc418106a80 0x7fc41819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:45.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.931+0000 7fc420267640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc418110ed0 con 0x7fc418100a80 2024-09-12T03:09:45.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.931+0000 7fc420267640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc418110d50 con 0x7fc418106a80 2024-09-12T03:09:45.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.931+0000 7fc420267640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc418111050 con 0x7fc418101420 2024-09-12T03:09:45.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.931+0000 7fc41e003640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc418100a80 0x7fc41819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:45.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.931+0000 7fc41e003640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc418100a80 0x7fc41819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58394/0 (socket says 172.21.15.116:58394) 2024-09-12T03:09:45.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.931+0000 7fc41e003640 1 -- 172.21.15.116:0/2208843806 learned_addr learned my addr 172.21.15.116:0/2208843806 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:45.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.931+0000 7fc41d802640 1 --2- 172.21.15.116:0/2208843806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc418101420 0x7fc41819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:45.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.931+0000 7fc41e003640 1 -- 172.21.15.116:0/2208843806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc418101420 msgr2=0x7fc41819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:45.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.931+0000 7fc41e003640 1 --2- 172.21.15.116:0/2208843806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc418101420 0x7fc41819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:45.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.931+0000 7fc41e003640 1 -- 172.21.15.116:0/2208843806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc418106a80 msgr2=0x7fc41819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:45.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.931+0000 7fc41e003640 1 --2- 172.21.15.116:0/2208843806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc418106a80 0x7fc41819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:45.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.931+0000 7fc41e003640 1 -- 172.21.15.116:0/2208843806 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc41819f5a0 con 0x7fc418100a80 2024-09-12T03:09:45.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.931+0000 7fc41d802640 1 --2- 172.21.15.116:0/2208843806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc418101420 0x7fc41819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:09:45.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.931+0000 7fc41e003640 1 --2- 172.21.15.116:0/2208843806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc418100a80 0x7fc41819a5f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fc40800d950 tx=0x7fc40800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:45.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.932+0000 7fc40f7fe640 1 -- 172.21.15.116:0/2208843806 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408017030 con 0x7fc418100a80 2024-09-12T03:09:45.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.932+0000 7fc420267640 1 -- 172.21.15.116:0/2208843806 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc41819f890 con 0x7fc418100a80 2024-09-12T03:09:45.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.932+0000 7fc40f7fe640 1 -- 172.21.15.116:0/2208843806 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc408004020 con 0x7fc418100a80 2024-09-12T03:09:45.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.932+0000 7fc40f7fe640 1 -- 172.21.15.116:0/2208843806 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4080047a0 con 0x7fc418100a80 2024-09-12T03:09:45.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.932+0000 7fc420267640 1 -- 172.21.15.116:0/2208843806 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4181a7170 con 0x7fc418100a80 2024-09-12T03:09:45.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.933+0000 7fc420267640 1 -- 172.21.15.116:0/2208843806 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc418100ee0 con 0x7fc418100a80 2024-09-12T03:09:45.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.934+0000 7fc40f7fe640 1 -- 172.21.15.116:0/2208843806 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fc4080041d0 con 0x7fc418100a80 2024-09-12T03:09:45.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.934+0000 7fc40f7fe640 1 --2- 172.21.15.116:0/2208843806 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc4040a2b80 0x7fc4040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:45.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.935+0000 7fc41d802640 1 --2- 172.21.15.116:0/2208843806 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc4040a2b80 0x7fc4040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:45.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.935+0000 7fc40f7fe640 1 -- 172.21.15.116:0/2208843806 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fc4080cea80 con 0x7fc418100a80 2024-09-12T03:09:45.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.935+0000 7fc41d802640 1 --2- 172.21.15.116:0/2208843806 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc4040a2b80 0x7fc4040a5040 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc41402a940 tx=0x7fc414005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:45.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:45.939+0000 7fc40f7fe640 1 -- 172.21.15.116:0/2208843806 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc40809ba10 con 0x7fc418100a80 2024-09-12T03:09:46.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:46.141+0000 7fc420267640 1 -- 172.21.15.116:0/2208843806 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4181a74c0 con 0x7fc418100a80 2024-09-12T03:09:46.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:46.141+0000 7fc40f7fe640 1 -- 172.21.15.116:0/2208843806 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7fc40809b310 con 0x7fc418100a80 2024-09-12T03:09:46.142 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:09:46.142 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:09:46.142 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:09:46.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:46.143+0000 7fc420267640 1 -- 172.21.15.116:0/2208843806 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc4040a2b80 msgr2=0x7fc4040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:46.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:46.143+0000 7fc420267640 1 --2- 172.21.15.116:0/2208843806 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc4040a2b80 0x7fc4040a5040 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc41402a940 tx=0x7fc414005fd0 comp rx=0 tx=0).stop 2024-09-12T03:09:46.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:46.143+0000 7fc420267640 1 -- 172.21.15.116:0/2208843806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc418100a80 msgr2=0x7fc41819a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:46.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:46.143+0000 7fc420267640 1 --2- 172.21.15.116:0/2208843806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc418100a80 0x7fc41819a5f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fc40800d950 tx=0x7fc40800de10 comp rx=0 tx=0).stop 2024-09-12T03:09:46.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:46.144+0000 7fc420267640 1 -- 172.21.15.116:0/2208843806 shutdown_connections 2024-09-12T03:09:46.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:46.144+0000 7fc420267640 1 --2- 172.21.15.116:0/2208843806 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc4040a2b80 0x7fc4040a5040 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:46.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:46.144+0000 7fc420267640 1 --2- 172.21.15.116:0/2208843806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc418106a80 0x7fc41819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:46.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:46.144+0000 7fc420267640 1 --2- 172.21.15.116:0/2208843806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc418101420 0x7fc41819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:46.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:46.144+0000 7fc420267640 1 --2- 172.21.15.116:0/2208843806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc418100a80 0x7fc41819a5f0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:46.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:46.144+0000 7fc420267640 1 -- 172.21.15.116:0/2208843806 >> 172.21.15.116:0/2208843806 conn(0x7fc4180fc7a0 msgr2=0x7fc41806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:46.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:46.144+0000 7fc420267640 1 -- 172.21.15.116:0/2208843806 shutdown_connections 2024-09-12T03:09:46.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:46.144+0000 7fc420267640 1 -- 172.21.15.116:0/2208843806 wait complete. 2024-09-12T03:09:46.537 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:46 smithi116 ceph-mon[31410]: pgmap v541: 97 pgs: 97 active+clean; 3.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:09:46.538 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:46 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2208843806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:09:46.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:46 smithi183 ceph-mon[35535]: pgmap v541: 97 pgs: 97 active+clean; 3.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:09:46.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:46 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2208843806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:09:46.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:46 smithi177 ceph-mon[35305]: pgmap v541: 97 pgs: 97 active+clean; 3.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:09:46.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:46 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2208843806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:09:46.925 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:09:47.169 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:09:47.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.664+0000 7f601328d640 1 -- 172.21.15.116:0/1383082444 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f600c102570 msgr2=0x7f600c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:47.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.664+0000 7f601328d640 1 --2- 172.21.15.116:0/1383082444 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f600c102570 0x7f600c1029b0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc009900 tx=0x7f5ffc02a480 comp rx=0 tx=0).stop 2024-09-12T03:09:47.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.664+0000 7f601328d640 1 -- 172.21.15.116:0/1383082444 shutdown_connections 2024-09-12T03:09:47.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.664+0000 7f601328d640 1 --2- 172.21.15.116:0/1383082444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f600c102ef0 0x7f600c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.664+0000 7f601328d640 1 --2- 172.21.15.116:0/1383082444 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f600c102570 0x7f600c1029b0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.664+0000 7f601328d640 1 --2- 172.21.15.116:0/1383082444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f600c100e20 0x7f600c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.664+0000 7f601328d640 1 -- 172.21.15.116:0/1383082444 >> 172.21.15.116:0/1383082444 conn(0x7f600c0fc7d0 msgr2=0x7f600c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:47.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.665+0000 7f601328d640 1 -- 172.21.15.116:0/1383082444 shutdown_connections 2024-09-12T03:09:47.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.665+0000 7f601328d640 1 -- 172.21.15.116:0/1383082444 wait complete. 2024-09-12T03:09:47.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.665+0000 7f601328d640 1 Processor -- start 2024-09-12T03:09:47.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.665+0000 7f601328d640 1 -- start start 2024-09-12T03:09:47.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.665+0000 7f601328d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f600c100e20 0x7f600c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:47.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.665+0000 7f601328d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f600c102570 0x7f600c198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:47.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.665+0000 7f601328d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f600c102ef0 0x7f600c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:47.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.666+0000 7f601328d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f600c110f80 con 0x7f600c100e20 2024-09-12T03:09:47.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.666+0000 7f601328d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f600c110e00 con 0x7f600c102ef0 2024-09-12T03:09:47.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.666+0000 7f601328d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f600c111100 con 0x7f600c102570 2024-09-12T03:09:47.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.666+0000 7f6011029640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f600c100e20 0x7f600c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:47.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.666+0000 7f6011029640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f600c100e20 0x7f600c1984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58422/0 (socket says 172.21.15.116:58422) 2024-09-12T03:09:47.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.666+0000 7f6011029640 1 -- 172.21.15.116:0/1928536559 learned_addr learned my addr 172.21.15.116:0/1928536559 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:47.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.666+0000 7f601182a640 1 --2- 172.21.15.116:0/1928536559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f600c102ef0 0x7f600c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:47.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.666+0000 7f6010828640 1 --2- 172.21.15.116:0/1928536559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f600c102570 0x7f600c198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:47.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.666+0000 7f6011029640 1 -- 172.21.15.116:0/1928536559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f600c102570 msgr2=0x7f600c198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:47.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.666+0000 7f6011029640 1 --2- 172.21.15.116:0/1928536559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f600c102570 0x7f600c198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.666+0000 7f6011029640 1 -- 172.21.15.116:0/1928536559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f600c102ef0 msgr2=0x7f600c19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:47.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.666+0000 7f6011029640 1 --2- 172.21.15.116:0/1928536559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f600c102ef0 0x7f600c19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.666+0000 7f6011029640 1 -- 172.21.15.116:0/1928536559 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f600c19d480 con 0x7f600c100e20 2024-09-12T03:09:47.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.666+0000 7f601182a640 1 --2- 172.21.15.116:0/1928536559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f600c102ef0 0x7f600c19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:09:47.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.666+0000 7f6011029640 1 --2- 172.21.15.116:0/1928536559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f600c100e20 0x7f600c1984d0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f600800d770 tx=0x7f600800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:47.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.666+0000 7f60027fc640 1 -- 172.21.15.116:0/1928536559 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008017070 con 0x7f600c100e20 2024-09-12T03:09:47.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.666+0000 7f60027fc640 1 -- 172.21.15.116:0/1928536559 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6008005500 con 0x7f600c100e20 2024-09-12T03:09:47.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.666+0000 7f601328d640 1 -- 172.21.15.116:0/1928536559 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f600c19d770 con 0x7f600c100e20 2024-09-12T03:09:47.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.666+0000 7f60027fc640 1 -- 172.21.15.116:0/1928536559 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008004e00 con 0x7f600c100e20 2024-09-12T03:09:47.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.666+0000 7f601328d640 1 -- 172.21.15.116:0/1928536559 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f600c1a5050 con 0x7f600c100e20 2024-09-12T03:09:47.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.669+0000 7f60027fc640 1 -- 172.21.15.116:0/1928536559 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6008004fa0 con 0x7f600c100e20 2024-09-12T03:09:47.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.669+0000 7f6015666740 1 Processor -- start 2024-09-12T03:09:47.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.669+0000 7f6015666740 1 -- start start 2024-09-12T03:09:47.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.669+0000 7f6015666740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dbd4370b40 0x55dbd436f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:47.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.669+0000 7f6015666740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbd436f5d0 0x55dbd436fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:47.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.669+0000 7f6015666740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dbd437e780 0x55dbd4380b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:47.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.669+0000 7f60027fc640 1 --2- 172.21.15.116:0/1928536559 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5ff80a2b80 0x7f5ff80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:47.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.669+0000 7f6015666740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55dbd42d52f0 con 0x55dbd436f5d0 2024-09-12T03:09:47.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.669+0000 7f6015666740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55dbd427ccf0 con 0x55dbd437e780 2024-09-12T03:09:47.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.669+0000 7f6015666740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55dbd42d12b0 con 0x55dbd4370b40 2024-09-12T03:09:47.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.669+0000 7f6011029640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbd436f5d0 0x55dbd436fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:47.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.670+0000 7f6010828640 1 --2- 172.21.15.116:0/1928536559 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5ff80a2b80 0x7f5ff80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:47.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.670+0000 7f6011029640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbd436f5d0 0x55dbd436fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58424/0 (socket says 172.21.15.116:58424) 2024-09-12T03:09:47.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.670+0000 7f6011029640 1 -- 172.21.15.116:0/419120700 learned_addr learned my addr 172.21.15.116:0/419120700 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:47.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.670+0000 7f601182a640 1 --2- 172.21.15.116:0/419120700 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dbd437e780 0x55dbd4380b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:47.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.670+0000 7f60027fc640 1 -- 172.21.15.116:0/1928536559 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f60080ceb40 con 0x7f600c100e20 2024-09-12T03:09:47.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.670+0000 7f601182a640 1 --2- 172.21.15.116:0/419120700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dbd4370b40 0x55dbd436f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:47.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.670+0000 7f6011029640 1 -- 172.21.15.116:0/419120700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dbd4370b40 msgr2=0x55dbd436f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:47.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.670+0000 7f6011029640 1 --2- 172.21.15.116:0/419120700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dbd4370b40 0x55dbd436f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.670+0000 7f6011029640 1 -- 172.21.15.116:0/419120700 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dbd437e780 msgr2=0x55dbd4380b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:47.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.670+0000 7f6011029640 1 --2- 172.21.15.116:0/419120700 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dbd437e780 0x55dbd4380b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.670+0000 7f6011029640 1 -- 172.21.15.116:0/419120700 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbd42613a0 con 0x55dbd436f5d0 2024-09-12T03:09:47.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.670+0000 7f6010828640 1 --2- 172.21.15.116:0/1928536559 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5ff80a2b80 0x7f5ff80a5040 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f600c0fffe0 tx=0x7f5ffc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:47.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.670+0000 7f6011029640 1 --2- 172.21.15.116:0/419120700 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbd436f5d0 0x55dbd436fa10 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f6008009610 tx=0x7f600806d3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b801c05f95f5440a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:47.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.670+0000 7f5ff77fe640 1 -- 172.21.15.116:0/419120700 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008020070 con 0x55dbd436f5d0 2024-09-12T03:09:47.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.670+0000 7f5ff77fe640 1 -- 172.21.15.116:0/419120700 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60080046c0 con 0x55dbd436f5d0 2024-09-12T03:09:47.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.670+0000 7f5ff77fe640 1 -- 172.21.15.116:0/419120700 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60080a4040 con 0x55dbd436f5d0 2024-09-12T03:09:47.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.671+0000 7f6015666740 1 -- 172.21.15.116:0/419120700 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbd436f5d0 msgr2=0x55dbd436fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:47.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.671+0000 7f6015666740 1 --2- 172.21.15.116:0/419120700 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbd436f5d0 0x55dbd436fa10 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f6008009610 tx=0x7f600806d3e0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.671+0000 7f6015666740 1 -- 172.21.15.116:0/419120700 shutdown_connections 2024-09-12T03:09:47.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.671+0000 7f6015666740 1 --2- 172.21.15.116:0/419120700 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dbd437e780 0x55dbd4380b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.671+0000 7f6015666740 1 --2- 172.21.15.116:0/419120700 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbd436f5d0 0x55dbd436fa10 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.671+0000 7f6015666740 1 --2- 172.21.15.116:0/419120700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dbd4370b40 0x55dbd436f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.671+0000 7f6015666740 1 -- 172.21.15.116:0/419120700 >> 172.21.15.116:0/419120700 conn(0x55dbd4338310 msgr2=0x55dbd436bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:47.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.671+0000 7f6015666740 1 -- 172.21.15.116:0/419120700 shutdown_connections 2024-09-12T03:09:47.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.671+0000 7f6015666740 1 -- 172.21.15.116:0/419120700 wait complete. 2024-09-12T03:09:47.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.671+0000 7f6015666740 1 -- start start 2024-09-12T03:09:47.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.671+0000 7f6015666740 1 Processor -- start 2024-09-12T03:09:47.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.672+0000 7f6015666740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dbd436f5d0 0x55dbd437e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:47.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.672+0000 7f6015666740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbd4370b40 0x55dbd437b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:47.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.672+0000 7f6015666740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dbd437e780 0x55dbd437ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:47.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.672+0000 7f6015666740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55dbd4381480 con 0x55dbd4370b40 2024-09-12T03:09:47.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.672+0000 7f6015666740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55dbd4290060 con 0x55dbd437e780 2024-09-12T03:09:47.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.672+0000 7f6015666740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55dbd4381600 con 0x55dbd436f5d0 2024-09-12T03:09:47.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.672+0000 7f6015666740 10 client.0 _ll_register_callbacks cb 0x55dbd3fe38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:47.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.672+0000 7f6010828640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbd4370b40 0x55dbd437b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:47.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.672+0000 7f6010828640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbd4370b40 0x55dbd437b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58428/0 (socket says 172.21.15.116:58428) 2024-09-12T03:09:47.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.672+0000 7f6010828640 1 -- 172.21.15.116:0/1683801681 learned_addr learned my addr 172.21.15.116:0/1683801681 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:47.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.672+0000 7f601182a640 1 --2- 172.21.15.116:0/1683801681 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dbd437e780 0x55dbd437ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:47.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.672+0000 7f6011029640 1 --2- 172.21.15.116:0/1683801681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dbd436f5d0 0x55dbd437e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:47.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.672+0000 7f6010828640 1 -- 172.21.15.116:0/1683801681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dbd436f5d0 msgr2=0x55dbd437e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:47.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.672+0000 7f6010828640 1 --2- 172.21.15.116:0/1683801681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dbd436f5d0 0x55dbd437e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.672+0000 7f6010828640 1 -- 172.21.15.116:0/1683801681 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dbd437e780 msgr2=0x55dbd437ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:47.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.672+0000 7f6010828640 1 --2- 172.21.15.116:0/1683801681 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dbd437e780 0x55dbd437ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.672+0000 7f6010828640 1 -- 172.21.15.116:0/1683801681 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbd40463c0 con 0x55dbd4370b40 2024-09-12T03:09:47.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.672+0000 7f6010828640 1 --2- 172.21.15.116:0/1683801681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbd4370b40 0x55dbd437b430 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc005ab0 tx=0x7f5ffc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999f4603e8e0b247 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:47.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.672+0000 7f5ff4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:09:47.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.673+0000 7f5ff4ff9640 1 -- 172.21.15.116:0/1683801681 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc04c020 con 0x55dbd4370b40 2024-09-12T03:09:47.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.673+0000 7f5ff4ff9640 1 -- 172.21.15.116:0/1683801681 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f5ffc004b90 con 0x55dbd4370b40 2024-09-12T03:09:47.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.673+0000 7f6015666740 1 -- 172.21.15.116:0/1683801681 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dbd42e2860 con 0x55dbd4370b40 2024-09-12T03:09:47.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.673+0000 7f5ff4ff9640 1 -- 172.21.15.116:0/1683801681 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5ffc035050 con 0x55dbd4370b40 2024-09-12T03:09:47.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.673+0000 7f6015666740 10 client.16968 fetch_fsmap learned FSMap version 25 2024-09-12T03:09:47.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.673+0000 7f6015666740 1 -- 172.21.15.116:0/1683801681 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dbd438cf20 con 0x55dbd4370b40 2024-09-12T03:09:47.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.673+0000 7f5ff4ff9640 1 -- 172.21.15.116:0/1683801681 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55dbd42e2860 con 0x55dbd4370b40 2024-09-12T03:09:47.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.673+0000 7f5ff4ff9640 1 -- 172.21.15.116:0/1683801681 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5ffc0325e0 con 0x55dbd4370b40 2024-09-12T03:09:47.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.673+0000 7f6015666740 10 client.16968 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:09:47.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.673+0000 7f6015666740 10 client.16968 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:09:47.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.673+0000 7f6015666740 20 client.16968 populate_metadata read hostname 'smithi116' 2024-09-12T03:09:47.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.673+0000 7f6015666740 1 --2- 172.21.15.116:0/1683801681 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55dbd438e380 0x55dbd43907c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:47.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.673+0000 7f6015666740 4 client.16968 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:09:47.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.673+0000 7f6015666740 1 -- 172.21.15.116:0/1683801681 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbd42dcd70 con 0x55dbd438e380 2024-09-12T03:09:47.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.673+0000 7f6011029640 1 --2- 172.21.15.116:0/1683801681 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55dbd438e380 0x55dbd43907c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:47.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.674+0000 7f5ff4ff9640 10 client.16968 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:09:47.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.674+0000 7f6011029640 1 --2- 172.21.15.116:0/1683801681 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55dbd438e380 0x55dbd43907c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e8c99e32766c76 server_cookie=8dfabb5558594465 in_seq=0 out_seq=0 2024-09-12T03:09:47.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.674+0000 7f5ff4ff9640 1 -- 172.21.15.116:0/1683801681 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dbd42dcd70 con 0x55dbd438e380 2024-09-12T03:09:47.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.674+0000 7f5ff4ff9640 10 client.16968 handle_command_reply: tid=0 2024-09-12T03:09:47.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.674+0000 7f6015666740 1 client.16968 shutdown 2024-09-12T03:09:47.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.675+0000 7f6015666740 1 -- 172.21.15.116:0/1683801681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbd4370b40 msgr2=0x55dbd437b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:47.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.675+0000 7f6015666740 1 --2- 172.21.15.116:0/1683801681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbd4370b40 0x55dbd437b430 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc005ab0 tx=0x7f5ffc009260 comp rx=0 tx=0).stop 2024-09-12T03:09:47.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.675+0000 7f6015666740 1 -- 172.21.15.116:0/1683801681 shutdown_connections 2024-09-12T03:09:47.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.675+0000 7f6015666740 1 --2- 172.21.15.116:0/1683801681 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dbd437e780 0x55dbd437ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.675+0000 7f6015666740 1 --2- 172.21.15.116:0/1683801681 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55dbd438e380 0x55dbd43907c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.675+0000 7f6015666740 1 --2- 172.21.15.116:0/1683801681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbd4370b40 0x55dbd437b430 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.675+0000 7f6015666740 1 --2- 172.21.15.116:0/1683801681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dbd436f5d0 0x55dbd437e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.675+0000 7f6015666740 1 -- 172.21.15.116:0/1683801681 >> 172.21.15.116:0/1683801681 conn(0x55dbd4338310 msgr2=0x55dbd4371990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:47.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.675+0000 7f5ff4ff9640 0 client.16968 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:09:47.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.675+0000 7f6015666740 1 -- 172.21.15.116:0/1683801681 shutdown_connections 2024-09-12T03:09:47.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.675+0000 7f6015666740 1 -- 172.21.15.116:0/1683801681 wait complete. 2024-09-12T03:09:47.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.675+0000 7f6015666740 20 client.16968 trim_cache size 0 max 16384 2024-09-12T03:09:47.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.687+0000 7f6015666740 1 Processor -- start 2024-09-12T03:09:47.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.687+0000 7f6015666740 1 -- start start 2024-09-12T03:09:47.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.688+0000 7f6015666740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbd436f5d0 0x55dbd437b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:47.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.688+0000 7f6015666740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dbd4370b40 0x55dbd438a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:47.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.688+0000 7f6015666740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dbd437e780 0x55dbd4396090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:47.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.688+0000 7f6015666740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f600c1adb50 con 0x55dbd436f5d0 2024-09-12T03:09:47.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.688+0000 7f6015666740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f600c1ab420 con 0x55dbd4370b40 2024-09-12T03:09:47.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.688+0000 7f6015666740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f600c1a9290 con 0x55dbd437e780 2024-09-12T03:09:47.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.688+0000 7f6011029640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbd436f5d0 0x55dbd437b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:47.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.688+0000 7f6011029640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbd436f5d0 0x55dbd437b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58440/0 (socket says 172.21.15.116:58440) 2024-09-12T03:09:47.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.688+0000 7f6011029640 1 -- 172.21.15.116:0/2900098779 learned_addr learned my addr 172.21.15.116:0/2900098779 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:47.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.688+0000 7f6010828640 1 --2- 172.21.15.116:0/2900098779 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dbd4370b40 0x55dbd438a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:47.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.688+0000 7f601182a640 1 --2- 172.21.15.116:0/2900098779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dbd437e780 0x55dbd4396090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:47.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.688+0000 7f6011029640 1 -- 172.21.15.116:0/2900098779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dbd437e780 msgr2=0x55dbd4396090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:47.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.688+0000 7f6011029640 1 --2- 172.21.15.116:0/2900098779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dbd437e780 0x55dbd4396090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.688+0000 7f6011029640 1 -- 172.21.15.116:0/2900098779 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dbd4370b40 msgr2=0x55dbd438a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:47.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.688+0000 7f6011029640 1 --2- 172.21.15.116:0/2900098779 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dbd4370b40 0x55dbd438a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.688+0000 7f6011029640 1 -- 172.21.15.116:0/2900098779 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbd4381360 con 0x55dbd436f5d0 2024-09-12T03:09:47.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.688+0000 7f6010828640 1 --2- 172.21.15.116:0/2900098779 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dbd4370b40 0x55dbd438a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:09:47.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.688+0000 7f6011029640 1 --2- 172.21.15.116:0/2900098779 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbd436f5d0 0x55dbd437b400 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f60080970b0 tx=0x7f6008097040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8338e30e0aa41e31 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:47.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.688+0000 7f5ff6ffd640 1 -- 172.21.15.116:0/2900098779 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008020040 con 0x55dbd436f5d0 2024-09-12T03:09:47.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.688+0000 7f5ff6ffd640 1 -- 172.21.15.116:0/2900098779 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60080a4040 con 0x55dbd436f5d0 2024-09-12T03:09:47.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.688+0000 7f5ff6ffd640 1 -- 172.21.15.116:0/2900098779 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008009cc0 con 0x55dbd436f5d0 2024-09-12T03:09:47.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.689+0000 7f6015666740 1 -- 172.21.15.116:0/2900098779 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbd436f5d0 msgr2=0x55dbd437b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:47.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.689+0000 7f6015666740 1 --2- 172.21.15.116:0/2900098779 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbd436f5d0 0x55dbd437b400 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f60080970b0 tx=0x7f6008097040 comp rx=0 tx=0).stop 2024-09-12T03:09:47.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.689+0000 7f6015666740 1 -- 172.21.15.116:0/2900098779 shutdown_connections 2024-09-12T03:09:47.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.689+0000 7f6015666740 1 --2- 172.21.15.116:0/2900098779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dbd437e780 0x55dbd4396090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.689+0000 7f6015666740 1 --2- 172.21.15.116:0/2900098779 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dbd4370b40 0x55dbd438a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.689+0000 7f6015666740 1 --2- 172.21.15.116:0/2900098779 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbd436f5d0 0x55dbd437b400 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.689+0000 7f6015666740 1 -- 172.21.15.116:0/2900098779 >> 172.21.15.116:0/2900098779 conn(0x55dbd4338310 msgr2=0x55dbd437acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:47.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.689+0000 7f6015666740 1 -- 172.21.15.116:0/2900098779 shutdown_connections 2024-09-12T03:09:47.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.689+0000 7f6015666740 1 -- 172.21.15.116:0/2900098779 wait complete. 2024-09-12T03:09:47.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.689+0000 7f6015666740 1 -- start start 2024-09-12T03:09:47.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.689+0000 7f6015666740 1 Processor -- start 2024-09-12T03:09:47.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.690+0000 7f6015666740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbd436f5d0 0x55dbd4386a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:47.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.690+0000 7f6015666740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dbd4370b40 0x55dbd4386fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:47.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.690+0000 7f6015666740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dbd437e780 0x55dbd4377c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:47.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.690+0000 7f6015666740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f600c1a98e0 con 0x55dbd436f5d0 2024-09-12T03:09:47.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.690+0000 7f6015666740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f600c1aad50 con 0x55dbd437e780 2024-09-12T03:09:47.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.690+0000 7f6015666740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f600c1a9fb0 con 0x55dbd4370b40 2024-09-12T03:09:47.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.690+0000 7f6015666740 10 client.0 _ll_register_callbacks cb 0x7f5ffc03f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:47.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.690+0000 7f6011029640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbd436f5d0 0x55dbd4386a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:47.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.690+0000 7f6011029640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbd436f5d0 0x55dbd4386a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58456/0 (socket says 172.21.15.116:58456) 2024-09-12T03:09:47.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.690+0000 7f6011029640 1 -- 172.21.15.116:0/1557447387 learned_addr learned my addr 172.21.15.116:0/1557447387 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:47.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.690+0000 7f6010828640 1 --2- 172.21.15.116:0/1557447387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dbd4370b40 0x55dbd4386fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:47.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.690+0000 7f601182a640 1 --2- 172.21.15.116:0/1557447387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dbd437e780 0x55dbd4377c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:47.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.690+0000 7f6011029640 1 -- 172.21.15.116:0/1557447387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dbd4370b40 msgr2=0x55dbd4386fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:47.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.690+0000 7f6011029640 1 --2- 172.21.15.116:0/1557447387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dbd4370b40 0x55dbd4386fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.690+0000 7f6011029640 1 -- 172.21.15.116:0/1557447387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dbd437e780 msgr2=0x55dbd4377c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:47.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.690+0000 7f6011029640 1 --2- 172.21.15.116:0/1557447387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dbd437e780 0x55dbd4377c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.690+0000 7f6011029640 1 -- 172.21.15.116:0/1557447387 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbd43783d0 con 0x55dbd436f5d0 2024-09-12T03:09:47.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.690+0000 7f601182a640 1 --2- 172.21.15.116:0/1557447387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dbd437e780 0x55dbd4377c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:09:47.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.690+0000 7f6011029640 1 --2- 172.21.15.116:0/1557447387 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbd436f5d0 0x55dbd4386a70 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f6008015010 tx=0x7f6008005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef2c5460d233d00 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:47.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.690+0000 7f5ff4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:09:47.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.690+0000 7f5ff4ff9640 1 -- 172.21.15.116:0/1557447387 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008020070 con 0x55dbd436f5d0 2024-09-12T03:09:47.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.691+0000 7f5ff4ff9640 1 -- 172.21.15.116:0/1557447387 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f60080663e0 con 0x55dbd436f5d0 2024-09-12T03:09:47.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.691+0000 7f6015666740 1 -- 172.21.15.116:0/1557447387 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60080a4040 con 0x55dbd436f5d0 2024-09-12T03:09:47.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.691+0000 7f5ff4ff9640 1 -- 172.21.15.116:0/1557447387 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f600809bad0 con 0x55dbd436f5d0 2024-09-12T03:09:47.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.691+0000 7f6015666740 10 client.16974 fetch_fsmap learned FSMap version 25 2024-09-12T03:09:47.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.691+0000 7f6015666740 1 -- 172.21.15.116:0/1557447387 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dbd43f8a20 con 0x55dbd436f5d0 2024-09-12T03:09:47.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.691+0000 7f5ff4ff9640 1 -- 172.21.15.116:0/1557447387 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60080a4040 con 0x55dbd436f5d0 2024-09-12T03:09:47.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.691+0000 7f5ff4ff9640 1 -- 172.21.15.116:0/1557447387 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f60080b4220 con 0x55dbd436f5d0 2024-09-12T03:09:47.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.691+0000 7f6015666740 10 client.16974 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:09:47.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.691+0000 7f6015666740 10 client.16974 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:09:47.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.691+0000 7f6015666740 20 client.16974 populate_metadata read hostname 'smithi116' 2024-09-12T03:09:47.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.691+0000 7f6015666740 1 --2- 172.21.15.116:0/1557447387 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55dbd43f8c40 0x55dbd43fb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:47.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.691+0000 7f6015666740 4 client.16974 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:09:47.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.691+0000 7f6015666740 1 -- 172.21.15.116:0/1557447387 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dbd4259170 con 0x55dbd43f8c40 2024-09-12T03:09:47.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.691+0000 7f6010828640 1 --2- 172.21.15.116:0/1557447387 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55dbd43f8c40 0x55dbd43fb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:47.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.692+0000 7f5ff4ff9640 10 client.16974 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:09:47.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.692+0000 7f6010828640 1 --2- 172.21.15.116:0/1557447387 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55dbd43f8c40 0x55dbd43fb030 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7de12a349bb11bb0 server_cookie=e0ed1b3b597cc8e8 in_seq=0 out_seq=0 2024-09-12T03:09:47.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.692+0000 7f5ff4ff9640 1 -- 172.21.15.116:0/1557447387 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dbd4259170 con 0x55dbd43f8c40 2024-09-12T03:09:47.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.692+0000 7f5ff4ff9640 10 client.16974 handle_command_reply: tid=0 2024-09-12T03:09:47.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.692+0000 7f6015666740 1 client.16974 shutdown 2024-09-12T03:09:47.729 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:09:47.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.692+0000 7f6015666740 1 -- 172.21.15.116:0/1557447387 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbd436f5d0 msgr2=0x55dbd4386a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:47.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.692+0000 7f6015666740 1 --2- 172.21.15.116:0/1557447387 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbd436f5d0 0x55dbd4386a70 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f6008015010 tx=0x7f6008005ab0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.692+0000 7f6015666740 1 -- 172.21.15.116:0/1557447387 shutdown_connections 2024-09-12T03:09:47.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.692+0000 7f6015666740 1 --2- 172.21.15.116:0/1557447387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dbd437e780 0x55dbd4377c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.692+0000 7f6015666740 1 --2- 172.21.15.116:0/1557447387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dbd4370b40 0x55dbd4386fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.692+0000 7f6015666740 1 --2- 172.21.15.116:0/1557447387 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55dbd43f8c40 0x55dbd43fb030 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.692+0000 7f6015666740 1 --2- 172.21.15.116:0/1557447387 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbd436f5d0 0x55dbd4386a70 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.692+0000 7f5ff4ff9640 0 client.16974 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:09:47.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.692+0000 7f6015666740 1 -- 172.21.15.116:0/1557447387 >> 172.21.15.116:0/1557447387 conn(0x55dbd4338310 msgr2=0x55dbd4379d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:47.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.692+0000 7f6015666740 1 -- 172.21.15.116:0/1557447387 shutdown_connections 2024-09-12T03:09:47.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.692+0000 7f6015666740 1 -- 172.21.15.116:0/1557447387 wait complete. 2024-09-12T03:09:47.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.692+0000 7f6015666740 20 client.16974 trim_cache size 0 max 16384 2024-09-12T03:09:47.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.693+0000 7f601328d640 1 -- 172.21.15.116:0/1928536559 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5ff80a2b80 msgr2=0x7f5ff80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:47.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.693+0000 7f601328d640 1 --2- 172.21.15.116:0/1928536559 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5ff80a2b80 0x7f5ff80a5040 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f600c0fffe0 tx=0x7f5ffc005a20 comp rx=0 tx=0).stop 2024-09-12T03:09:47.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.693+0000 7f601328d640 1 -- 172.21.15.116:0/1928536559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f600c100e20 msgr2=0x7f600c1984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:47.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.693+0000 7f601328d640 1 --2- 172.21.15.116:0/1928536559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f600c100e20 0x7f600c1984d0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f600800d770 tx=0x7f600800dc30 comp rx=0 tx=0).stop 2024-09-12T03:09:47.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.693+0000 7f601328d640 1 -- 172.21.15.116:0/1928536559 shutdown_connections 2024-09-12T03:09:47.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.693+0000 7f601328d640 1 --2- 172.21.15.116:0/1928536559 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5ff80a2b80 0x7f5ff80a5040 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.693+0000 7f601328d640 1 --2- 172.21.15.116:0/1928536559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f600c102ef0 0x7f600c19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.693+0000 7f601328d640 1 --2- 172.21.15.116:0/1928536559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f600c102570 0x7f600c198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.693+0000 7f601328d640 1 --2- 172.21.15.116:0/1928536559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f600c100e20 0x7f600c1984d0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:47.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.693+0000 7f601328d640 1 -- 172.21.15.116:0/1928536559 >> 172.21.15.116:0/1928536559 conn(0x7f600c0fc7d0 msgr2=0x7f600c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:47.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.693+0000 7f601328d640 1 -- 172.21.15.116:0/1928536559 shutdown_connections 2024-09-12T03:09:47.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:47.693+0000 7f601328d640 1 -- 172.21.15.116:0/1928536559 wait complete. 2024-09-12T03:09:48.475 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:09:48.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:48 smithi183 ceph-mon[35535]: pgmap v542: 97 pgs: 97 active+clean; 3.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:09:48.718 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:09:48.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:48 smithi177 ceph-mon[35305]: pgmap v542: 97 pgs: 97 active+clean; 3.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:09:48.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:48 smithi116 ceph-mon[31410]: pgmap v542: 97 pgs: 97 active+clean; 3.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:09:49.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.191+0000 7fa0f9495640 1 -- 172.21.15.116:0/3805670342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0f4102790 msgr2=0x7fa0f41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:49.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.191+0000 7fa0f9495640 1 --2- 172.21.15.116:0/3805670342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0f4102790 0x7fa0f41067e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc009900 tx=0x7fa0dc02a410 comp rx=0 tx=0).stop 2024-09-12T03:09:49.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.191+0000 7fa0f9495640 1 -- 172.21.15.116:0/3805670342 shutdown_connections 2024-09-12T03:09:49.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.191+0000 7fa0f9495640 1 --2- 172.21.15.116:0/3805670342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0f4106f10 0x7fa0f410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.191+0000 7fa0f9495640 1 --2- 172.21.15.116:0/3805670342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0f4102790 0x7fa0f41067e0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.191+0000 7fa0f9495640 1 --2- 172.21.15.116:0/3805670342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0f4101de0 0x7fa0f41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.191+0000 7fa0f9495640 1 -- 172.21.15.116:0/3805670342 >> 172.21.15.116:0/3805670342 conn(0x7fa0f40fd8c0 msgr2=0x7fa0f40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:49.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.191+0000 7fa0f9495640 1 -- 172.21.15.116:0/3805670342 shutdown_connections 2024-09-12T03:09:49.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.191+0000 7fa0f9495640 1 -- 172.21.15.116:0/3805670342 wait complete. 2024-09-12T03:09:49.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.191+0000 7fa0f9495640 1 Processor -- start 2024-09-12T03:09:49.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.191+0000 7fa0f9495640 1 -- start start 2024-09-12T03:09:49.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.192+0000 7fa0f9495640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0f4101de0 0x7fa0f41983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:49.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.192+0000 7fa0f9495640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0f4102790 0x7fa0f41988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:49.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.192+0000 7fa0f9495640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0f4106f10 0x7fa0f419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:49.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.192+0000 7fa0f9495640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f4111f70 con 0x7fa0f4106f10 2024-09-12T03:09:49.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.192+0000 7fa0f9495640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f4111df0 con 0x7fa0f4101de0 2024-09-12T03:09:49.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.192+0000 7fa0f9495640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f41120f0 con 0x7fa0f4102790 2024-09-12T03:09:49.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.192+0000 7fa0f37fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0f4106f10 0x7fa0f419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:49.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.192+0000 7fa0f37fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0f4106f10 0x7fa0f419cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58468/0 (socket says 172.21.15.116:58468) 2024-09-12T03:09:49.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.192+0000 7fa0f37fe640 1 -- 172.21.15.116:0/1434897735 learned_addr learned my addr 172.21.15.116:0/1434897735 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:49.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.192+0000 7fa0f27fc640 1 --2- 172.21.15.116:0/1434897735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0f4102790 0x7fa0f41988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:49.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.192+0000 7fa0f2ffd640 1 --2- 172.21.15.116:0/1434897735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0f4101de0 0x7fa0f41983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:49.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.192+0000 7fa0f37fe640 1 -- 172.21.15.116:0/1434897735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0f4102790 msgr2=0x7fa0f41988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:49.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.192+0000 7fa0f37fe640 1 --2- 172.21.15.116:0/1434897735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0f4102790 0x7fa0f41988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.192+0000 7fa0f37fe640 1 -- 172.21.15.116:0/1434897735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0f4101de0 msgr2=0x7fa0f41983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:49.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.192+0000 7fa0f37fe640 1 --2- 172.21.15.116:0/1434897735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0f4101de0 0x7fa0f41983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.192+0000 7fa0f37fe640 1 -- 172.21.15.116:0/1434897735 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f419d360 con 0x7fa0f4106f10 2024-09-12T03:09:49.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.192+0000 7fa0f37fe640 1 --2- 172.21.15.116:0/1434897735 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0f4106f10 0x7fa0f419cc80 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fa0e400c970 tx=0x7fa0e400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:49.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.192+0000 7fa0ebfff640 1 -- 172.21.15.116:0/1434897735 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4007bb0 con 0x7fa0f4106f10 2024-09-12T03:09:49.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.192+0000 7fa0ebfff640 1 -- 172.21.15.116:0/1434897735 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0e4007d50 con 0x7fa0f4106f10 2024-09-12T03:09:49.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.193+0000 7fa0f9495640 1 -- 172.21.15.116:0/1434897735 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0f419d650 con 0x7fa0f4106f10 2024-09-12T03:09:49.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.193+0000 7fa0ebfff640 1 -- 172.21.15.116:0/1434897735 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4018800 con 0x7fa0f4106f10 2024-09-12T03:09:49.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.193+0000 7fa0f9495640 1 -- 172.21.15.116:0/1434897735 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0f41a4f30 con 0x7fa0f4106f10 2024-09-12T03:09:49.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.195+0000 7fa0ebfff640 1 -- 172.21.15.116:0/1434897735 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa0e400fa50 con 0x7fa0f4106f10 2024-09-12T03:09:49.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.195+0000 7fa0fb86e740 1 Processor -- start 2024-09-12T03:09:49.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.195+0000 7fa0fb86e740 1 -- start start 2024-09-12T03:09:49.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.196+0000 7fa0fb86e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd3c219b40 0x55bd3c218090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:49.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.196+0000 7fa0fb86e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd3c2185d0 0x55bd3c218a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:49.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.196+0000 7fa0fb86e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd3c223370 0x55bd3c225760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:49.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.196+0000 7fa0fb86e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bd3c17e2f0 con 0x55bd3c219b40 2024-09-12T03:09:49.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.196+0000 7fa0fb86e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bd3c125cf0 con 0x55bd3c223370 2024-09-12T03:09:49.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.196+0000 7fa0fb86e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bd3c17a2b0 con 0x55bd3c2185d0 2024-09-12T03:09:49.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.196+0000 7fa0f27fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd3c219b40 0x55bd3c218090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:49.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.196+0000 7fa0f27fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd3c219b40 0x55bd3c218090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58474/0 (socket says 172.21.15.116:58474) 2024-09-12T03:09:49.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.196+0000 7fa0f27fc640 1 -- 172.21.15.116:0/1578733180 learned_addr learned my addr 172.21.15.116:0/1578733180 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:49.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.196+0000 7fa0f2ffd640 1 --2- 172.21.15.116:0/1578733180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd3c2185d0 0x55bd3c218a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:49.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.196+0000 7fa0ebfff640 1 --2- 172.21.15.116:0/1434897735 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa0e00a2b80 0x7fa0e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:49.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.196+0000 7fa0f27fc640 1 --2- 172.21.15.116:0/1578733180 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd3c223370 0x55bd3c225760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:49.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.196+0000 7fa0f27fc640 1 -- 172.21.15.116:0/1578733180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd3c2185d0 msgr2=0x55bd3c218a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:49.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.196+0000 7fa0f27fc640 1 --2- 172.21.15.116:0/1578733180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd3c2185d0 0x55bd3c218a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.196+0000 7fa0f27fc640 1 -- 172.21.15.116:0/1578733180 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd3c223370 msgr2=0x55bd3c225760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:49.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.196+0000 7fa0f27fc640 1 --2- 172.21.15.116:0/1578733180 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd3c223370 0x55bd3c225760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.196+0000 7fa0f27fc640 1 -- 172.21.15.116:0/1578733180 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd3c10a3a0 con 0x55bd3c219b40 2024-09-12T03:09:49.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.196+0000 7fa0f37fe640 1 --2- 172.21.15.116:0/1434897735 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa0e00a2b80 0x7fa0e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:49.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.196+0000 7fa0f27fc640 1 --2- 172.21.15.116:0/1578733180 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd3c219b40 0x55bd3c218090 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc005ec0 tx=0x7fa0dc005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a7572cabfba663 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:49.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.196+0000 7fa0ebfff640 1 -- 172.21.15.116:0/1434897735 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa0e40cef40 con 0x7fa0f4106f10 2024-09-12T03:09:49.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.196+0000 7fa0e97fa640 1 -- 172.21.15.116:0/1578733180 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc035040 con 0x55bd3c219b40 2024-09-12T03:09:49.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.197+0000 7fa0e97fa640 1 -- 172.21.15.116:0/1578733180 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0dc0035f0 con 0x55bd3c219b40 2024-09-12T03:09:49.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.197+0000 7fa0e97fa640 1 -- 172.21.15.116:0/1578733180 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc03a630 con 0x55bd3c219b40 2024-09-12T03:09:49.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.197+0000 7fa0fb86e740 1 -- 172.21.15.116:0/1578733180 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd3c219b40 msgr2=0x55bd3c218090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:49.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.197+0000 7fa0f37fe640 1 --2- 172.21.15.116:0/1434897735 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa0e00a2b80 0x7fa0e00a5040 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa0e409b760 tx=0x7fa0e406e890 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:49.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.197+0000 7fa0fb86e740 1 --2- 172.21.15.116:0/1578733180 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd3c219b40 0x55bd3c218090 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc005ec0 tx=0x7fa0dc005d90 comp rx=0 tx=0).stop 2024-09-12T03:09:49.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.197+0000 7fa0fb86e740 1 -- 172.21.15.116:0/1578733180 shutdown_connections 2024-09-12T03:09:49.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.197+0000 7fa0fb86e740 1 --2- 172.21.15.116:0/1578733180 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd3c223370 0x55bd3c225760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.197+0000 7fa0fb86e740 1 --2- 172.21.15.116:0/1578733180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd3c2185d0 0x55bd3c218a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.197+0000 7fa0fb86e740 1 --2- 172.21.15.116:0/1578733180 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd3c219b40 0x55bd3c218090 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.197+0000 7fa0fb86e740 1 -- 172.21.15.116:0/1578733180 >> 172.21.15.116:0/1578733180 conn(0x55bd3c1e1310 msgr2=0x55bd3c214c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:49.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.197+0000 7fa0fb86e740 1 -- 172.21.15.116:0/1578733180 shutdown_connections 2024-09-12T03:09:49.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.197+0000 7fa0fb86e740 1 -- 172.21.15.116:0/1578733180 wait complete. 2024-09-12T03:09:49.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.198+0000 7fa0fb86e740 1 -- start start 2024-09-12T03:09:49.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.198+0000 7fa0fb86e740 1 Processor -- start 2024-09-12T03:09:49.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.198+0000 7fa0fb86e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd3c2185d0 0x55bd3c222d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:49.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.198+0000 7fa0fb86e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd3c219b40 0x55bd3c21ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:49.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.198+0000 7fa0fb86e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd3c223370 0x55bd3c2205d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:49.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.198+0000 7fa0fb86e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bd3c226010 con 0x55bd3c2185d0 2024-09-12T03:09:49.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.198+0000 7fa0fb86e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bd3c139060 con 0x55bd3c219b40 2024-09-12T03:09:49.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.198+0000 7fa0fb86e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bd3c226190 con 0x55bd3c223370 2024-09-12T03:09:49.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.198+0000 7fa0fb86e740 10 client.0 _ll_register_callbacks cb 0x55bd3be8c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:49.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.198+0000 7fa0f27fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd3c2185d0 0x55bd3c222d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:49.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.198+0000 7fa0f27fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd3c2185d0 0x55bd3c222d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58476/0 (socket says 172.21.15.116:58476) 2024-09-12T03:09:49.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.198+0000 7fa0f27fc640 1 -- 172.21.15.116:0/3436526725 learned_addr learned my addr 172.21.15.116:0/3436526725 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:49.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.198+0000 7fa0f27fc640 1 --2- 172.21.15.116:0/3436526725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd3c223370 0x55bd3c2205d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:49.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.198+0000 7fa0f27fc640 1 -- 172.21.15.116:0/3436526725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd3c223370 msgr2=0x55bd3c2205d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:49.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.198+0000 7fa0f2ffd640 1 --2- 172.21.15.116:0/3436526725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd3c219b40 0x55bd3c21ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:49.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.198+0000 7fa0f27fc640 1 --2- 172.21.15.116:0/3436526725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd3c223370 0x55bd3c2205d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.198+0000 7fa0f27fc640 1 -- 172.21.15.116:0/3436526725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd3c219b40 msgr2=0x55bd3c21ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:49.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.198+0000 7fa0f27fc640 1 --2- 172.21.15.116:0/3436526725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd3c219b40 0x55bd3c21ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.198+0000 7fa0f27fc640 1 -- 172.21.15.116:0/3436526725 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd3beef3c0 con 0x55bd3c2185d0 2024-09-12T03:09:49.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.198+0000 7fa0f2ffd640 1 --2- 172.21.15.116:0/3436526725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd3c219b40 0x55bd3c21ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:09:49.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.199+0000 7fa0f27fc640 1 --2- 172.21.15.116:0/3436526725 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd3c2185d0 0x55bd3c222d30 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc006010 tx=0x7fa0dc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9928b0a398292268 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:49.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.199+0000 7fa0da7fc640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:09:49.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.199+0000 7fa0da7fc640 1 -- 172.21.15.116:0/3436526725 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc035070 con 0x55bd3c2185d0 2024-09-12T03:09:49.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.199+0000 7fa0da7fc640 1 -- 172.21.15.116:0/3436526725 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa0dc0039b0 con 0x55bd3c2185d0 2024-09-12T03:09:49.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.199+0000 7fa0fb86e740 1 -- 172.21.15.116:0/3436526725 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0dc0035f0 con 0x55bd3c2185d0 2024-09-12T03:09:49.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.199+0000 7fa0da7fc640 1 -- 172.21.15.116:0/3436526725 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa0dc004490 con 0x55bd3c2185d0 2024-09-12T03:09:49.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.199+0000 7fa0fb86e740 10 client.16986 fetch_fsmap learned FSMap version 25 2024-09-12T03:09:49.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.199+0000 7fa0fb86e740 1 -- 172.21.15.116:0/3436526725 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bd3c231a70 con 0x55bd3c2185d0 2024-09-12T03:09:49.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.199+0000 7fa0da7fc640 1 -- 172.21.15.116:0/3436526725 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0dc0035f0 con 0x55bd3c2185d0 2024-09-12T03:09:49.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.199+0000 7fa0da7fc640 1 -- 172.21.15.116:0/3436526725 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa0dc03a3f0 con 0x55bd3c2185d0 2024-09-12T03:09:49.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.199+0000 7fa0fb86e740 10 client.16986 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:09:49.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.199+0000 7fa0fb86e740 10 client.16986 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:09:49.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.199+0000 7fa0fb86e740 20 client.16986 populate_metadata read hostname 'smithi116' 2024-09-12T03:09:49.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.199+0000 7fa0fb86e740 1 --2- 172.21.15.116:0/3436526725 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55bd3c232dc0 0x55bd3c2351b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:49.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.199+0000 7fa0fb86e740 4 client.16986 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:09:49.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.199+0000 7fa0fb86e740 1 -- 172.21.15.116:0/3436526725 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd3c18b860 con 0x55bd3c232dc0 2024-09-12T03:09:49.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.200+0000 7fa0f2ffd640 1 --2- 172.21.15.116:0/3436526725 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55bd3c232dc0 0x55bd3c2351b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:49.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.200+0000 7fa0da7fc640 10 client.16986 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:09:49.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.200+0000 7fa0f2ffd640 1 --2- 172.21.15.116:0/3436526725 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55bd3c232dc0 0x55bd3c2351b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d80aa29ae44bbfe6 server_cookie=1706b37815c6a1f5 in_seq=0 out_seq=0 2024-09-12T03:09:49.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.201+0000 7fa0da7fc640 1 -- 172.21.15.116:0/3436526725 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bd3c18b860 con 0x55bd3c232dc0 2024-09-12T03:09:49.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.201+0000 7fa0da7fc640 10 client.16986 handle_command_reply: tid=0 2024-09-12T03:09:49.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.201+0000 7fa0fb86e740 1 client.16986 shutdown 2024-09-12T03:09:49.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.201+0000 7fa0fb86e740 1 -- 172.21.15.116:0/3436526725 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd3c2185d0 msgr2=0x55bd3c222d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:49.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.201+0000 7fa0fb86e740 1 --2- 172.21.15.116:0/3436526725 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd3c2185d0 0x55bd3c222d30 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc006010 tx=0x7fa0dc000bc0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.201+0000 7fa0fb86e740 1 -- 172.21.15.116:0/3436526725 shutdown_connections 2024-09-12T03:09:49.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.201+0000 7fa0fb86e740 1 --2- 172.21.15.116:0/3436526725 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55bd3c232dc0 0x55bd3c2351b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.201+0000 7fa0fb86e740 1 --2- 172.21.15.116:0/3436526725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd3c223370 0x55bd3c2205d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.201+0000 7fa0fb86e740 1 --2- 172.21.15.116:0/3436526725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd3c219b40 0x55bd3c21ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.201+0000 7fa0da7fc640 0 client.16986 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:09:49.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.201+0000 7fa0fb86e740 1 --2- 172.21.15.116:0/3436526725 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd3c2185d0 0x55bd3c222d30 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.201+0000 7fa0fb86e740 1 -- 172.21.15.116:0/3436526725 >> 172.21.15.116:0/3436526725 conn(0x55bd3c1e1310 msgr2=0x55bd3c21a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:49.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.201+0000 7fa0fb86e740 1 -- 172.21.15.116:0/3436526725 shutdown_connections 2024-09-12T03:09:49.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.201+0000 7fa0fb86e740 1 -- 172.21.15.116:0/3436526725 wait complete. 2024-09-12T03:09:49.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.201+0000 7fa0fb86e740 20 client.16986 trim_cache size 0 max 16384 2024-09-12T03:09:49.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.214+0000 7fa0fb86e740 1 Processor -- start 2024-09-12T03:09:49.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.214+0000 7fa0fb86e740 1 -- start start 2024-09-12T03:09:49.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.214+0000 7fa0fb86e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd3c2185d0 0x55bd3c221f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:49.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.214+0000 7fa0fb86e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd3c219b40 0x55bd3c222490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:49.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.214+0000 7fa0fb86e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd3c223370 0x55bd3c2381e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:49.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.214+0000 7fa0fb86e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f41afbb0 con 0x55bd3c2185d0 2024-09-12T03:09:49.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.214+0000 7fa0fb86e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f41ad480 con 0x55bd3c223370 2024-09-12T03:09:49.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.214+0000 7fa0fb86e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f41ab2d0 con 0x55bd3c219b40 2024-09-12T03:09:49.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.214+0000 7fa0f27fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd3c2185d0 0x55bd3c221f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:49.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.214+0000 7fa0f27fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd3c2185d0 0x55bd3c221f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58482/0 (socket says 172.21.15.116:58482) 2024-09-12T03:09:49.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.214+0000 7fa0f27fc640 1 -- 172.21.15.116:0/447113581 learned_addr learned my addr 172.21.15.116:0/447113581 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:49.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.214+0000 7fa0f2ffd640 1 --2- 172.21.15.116:0/447113581 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd3c219b40 0x55bd3c222490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:49.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.215+0000 7fa0f27fc640 1 --2- 172.21.15.116:0/447113581 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd3c223370 0x55bd3c2381e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:49.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.215+0000 7fa0f27fc640 1 -- 172.21.15.116:0/447113581 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd3c219b40 msgr2=0x55bd3c222490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:49.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.215+0000 7fa0f27fc640 1 --2- 172.21.15.116:0/447113581 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd3c219b40 0x55bd3c222490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.215+0000 7fa0f27fc640 1 -- 172.21.15.116:0/447113581 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd3c223370 msgr2=0x55bd3c2381e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:49.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.215+0000 7fa0f27fc640 1 --2- 172.21.15.116:0/447113581 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd3c223370 0x55bd3c2381e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.215+0000 7fa0f27fc640 1 -- 172.21.15.116:0/447113581 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd3c214390 con 0x55bd3c2185d0 2024-09-12T03:09:49.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.215+0000 7fa0f27fc640 1 --2- 172.21.15.116:0/447113581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd3c2185d0 0x55bd3c221f50 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc03a7c0 tx=0x7fa0dc003fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ab52c902e53038 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:49.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.215+0000 7fa0e8ff9640 1 -- 172.21.15.116:0/447113581 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc035040 con 0x55bd3c2185d0 2024-09-12T03:09:49.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.215+0000 7fa0e8ff9640 1 -- 172.21.15.116:0/447113581 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0dc0035f0 con 0x55bd3c2185d0 2024-09-12T03:09:49.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.215+0000 7fa0e8ff9640 1 -- 172.21.15.116:0/447113581 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc0319d0 con 0x55bd3c2185d0 2024-09-12T03:09:49.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.216+0000 7fa0fb86e740 1 -- 172.21.15.116:0/447113581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd3c2185d0 msgr2=0x55bd3c221f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:49.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.216+0000 7fa0fb86e740 1 --2- 172.21.15.116:0/447113581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd3c2185d0 0x55bd3c221f50 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc03a7c0 tx=0x7fa0dc003fe0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.216+0000 7fa0fb86e740 1 -- 172.21.15.116:0/447113581 shutdown_connections 2024-09-12T03:09:49.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.216+0000 7fa0fb86e740 1 --2- 172.21.15.116:0/447113581 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd3c223370 0x55bd3c2381e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.216+0000 7fa0fb86e740 1 --2- 172.21.15.116:0/447113581 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd3c219b40 0x55bd3c222490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.216+0000 7fa0fb86e740 1 --2- 172.21.15.116:0/447113581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd3c2185d0 0x55bd3c221f50 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.216+0000 7fa0fb86e740 1 -- 172.21.15.116:0/447113581 >> 172.21.15.116:0/447113581 conn(0x55bd3c1e1310 msgr2=0x55bd3c221840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:49.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.216+0000 7fa0fb86e740 1 -- 172.21.15.116:0/447113581 shutdown_connections 2024-09-12T03:09:49.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.216+0000 7fa0fb86e740 1 -- 172.21.15.116:0/447113581 wait complete. 2024-09-12T03:09:49.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.216+0000 7fa0fb86e740 1 -- start start 2024-09-12T03:09:49.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.216+0000 7fa0fb86e740 1 Processor -- start 2024-09-12T03:09:49.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.216+0000 7fa0fb86e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd3c2185d0 0x55bd3c236110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:49.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.217+0000 7fa0fb86e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd3c219b40 0x55bd3c236650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:49.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.217+0000 7fa0fb86e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd3c223370 0x55bd3c236b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:49.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.217+0000 7fa0fb86e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f41ab940 con 0x55bd3c223370 2024-09-12T03:09:49.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.217+0000 7fa0fb86e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f41acdb0 con 0x55bd3c2185d0 2024-09-12T03:09:49.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.217+0000 7fa0fb86e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f41ac010 con 0x55bd3c219b40 2024-09-12T03:09:49.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.217+0000 7fa0fb86e740 10 client.0 _ll_register_callbacks cb 0x7fa0ec00a250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:49.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.217+0000 7fa0f27fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd3c223370 0x55bd3c236b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:49.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.217+0000 7fa0f27fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd3c223370 0x55bd3c236b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58484/0 (socket says 172.21.15.116:58484) 2024-09-12T03:09:49.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.217+0000 7fa0f27fc640 1 -- 172.21.15.116:0/1875240422 learned_addr learned my addr 172.21.15.116:0/1875240422 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:49.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.217+0000 7fa0f27fc640 1 --2- 172.21.15.116:0/1875240422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd3c2185d0 0x55bd3c236110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:49.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.217+0000 7fa0f2ffd640 1 --2- 172.21.15.116:0/1875240422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd3c219b40 0x55bd3c236650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:49.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.217+0000 7fa0f27fc640 1 -- 172.21.15.116:0/1875240422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd3c219b40 msgr2=0x55bd3c236650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:49.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.217+0000 7fa0f27fc640 1 --2- 172.21.15.116:0/1875240422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd3c219b40 0x55bd3c236650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.217+0000 7fa0f27fc640 1 -- 172.21.15.116:0/1875240422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd3c2185d0 msgr2=0x55bd3c236110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:49.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.217+0000 7fa0f27fc640 1 --2- 172.21.15.116:0/1875240422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd3c2185d0 0x55bd3c236110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.217+0000 7fa0f27fc640 1 -- 172.21.15.116:0/1875240422 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd3c29c900 con 0x55bd3c223370 2024-09-12T03:09:49.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.217+0000 7fa0f27fc640 1 --2- 172.21.15.116:0/1875240422 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd3c223370 0x55bd3c236b90 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc03a9b0 tx=0x7fa0dc03a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd651d6c1d77c0a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:49.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.217+0000 7fa0da7fc640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:09:49.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.217+0000 7fa0da7fc640 1 -- 172.21.15.116:0/1875240422 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc035070 con 0x55bd3c223370 2024-09-12T03:09:49.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.217+0000 7fa0fb86e740 1 -- 172.21.15.116:0/1875240422 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0dc0035f0 con 0x55bd3c223370 2024-09-12T03:09:49.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.217+0000 7fa0da7fc640 1 -- 172.21.15.116:0/1875240422 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa0dc061040 con 0x55bd3c223370 2024-09-12T03:09:49.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.218+0000 7fa0da7fc640 1 -- 172.21.15.116:0/1875240422 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa0dc0047a0 con 0x55bd3c223370 2024-09-12T03:09:49.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.218+0000 7fa0fb86e740 10 client.16992 fetch_fsmap learned FSMap version 25 2024-09-12T03:09:49.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.218+0000 7fa0fb86e740 1 -- 172.21.15.116:0/1875240422 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bd3c29db80 con 0x55bd3c223370 2024-09-12T03:09:49.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.218+0000 7fa0da7fc640 1 -- 172.21.15.116:0/1875240422 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0dc0035f0 con 0x55bd3c223370 2024-09-12T03:09:49.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.218+0000 7fa0da7fc640 1 -- 172.21.15.116:0/1875240422 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa0dc0054d0 con 0x55bd3c223370 2024-09-12T03:09:49.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.218+0000 7fa0fb86e740 10 client.16992 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:09:49.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.218+0000 7fa0fb86e740 10 client.16992 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:09:49.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.218+0000 7fa0fb86e740 20 client.16992 populate_metadata read hostname 'smithi116' 2024-09-12T03:09:49.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.218+0000 7fa0fb86e740 1 --2- 172.21.15.116:0/1875240422 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55bd3c29e190 0x55bd3c2a0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:49.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.218+0000 7fa0fb86e740 4 client.16992 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:09:49.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.218+0000 7fa0fb86e740 1 -- 172.21.15.116:0/1875240422 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bd3c185d70 con 0x55bd3c29e190 2024-09-12T03:09:49.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.218+0000 7fa0f2ffd640 1 --2- 172.21.15.116:0/1875240422 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55bd3c29e190 0x55bd3c2a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:49.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.219+0000 7fa0f2ffd640 1 --2- 172.21.15.116:0/1875240422 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55bd3c29e190 0x55bd3c2a0580 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2f4589193f422003 server_cookie=11cca86adf57c3d7 in_seq=0 out_seq=0 2024-09-12T03:09:49.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.219+0000 7fa0da7fc640 10 client.16992 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:09:49.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.219+0000 7fa0da7fc640 1 -- 172.21.15.116:0/1875240422 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bd3c185d70 con 0x55bd3c29e190 2024-09-12T03:09:49.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.219+0000 7fa0da7fc640 10 client.16992 handle_command_reply: tid=0 2024-09-12T03:09:49.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.219+0000 7fa0fb86e740 1 client.16992 shutdown 2024-09-12T03:09:49.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.219+0000 7fa0fb86e740 1 -- 172.21.15.116:0/1875240422 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd3c223370 msgr2=0x55bd3c236b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:49.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.219+0000 7fa0fb86e740 1 --2- 172.21.15.116:0/1875240422 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd3c223370 0x55bd3c236b90 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc03a9b0 tx=0x7fa0dc03a6e0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.219+0000 7fa0fb86e740 1 -- 172.21.15.116:0/1875240422 shutdown_connections 2024-09-12T03:09:49.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.219+0000 7fa0fb86e740 1 --2- 172.21.15.116:0/1875240422 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55bd3c29e190 0x55bd3c2a0580 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.219+0000 7fa0fb86e740 1 --2- 172.21.15.116:0/1875240422 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd3c223370 0x55bd3c236b90 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.219+0000 7fa0fb86e740 1 --2- 172.21.15.116:0/1875240422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd3c219b40 0x55bd3c236650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.219+0000 7fa0da7fc640 0 client.16992 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:09:49.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.219+0000 7fa0fb86e740 1 --2- 172.21.15.116:0/1875240422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bd3c2185d0 0x55bd3c236110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.219+0000 7fa0fb86e740 1 -- 172.21.15.116:0/1875240422 >> 172.21.15.116:0/1875240422 conn(0x55bd3c1e1310 msgr2=0x55bd3c222060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:49.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.219+0000 7fa0fb86e740 1 -- 172.21.15.116:0/1875240422 shutdown_connections 2024-09-12T03:09:49.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.219+0000 7fa0fb86e740 1 -- 172.21.15.116:0/1875240422 wait complete. 2024-09-12T03:09:49.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.219+0000 7fa0fb86e740 20 client.16992 trim_cache size 0 max 16384 2024-09-12T03:09:49.231 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:09:49.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.219+0000 7fa0f9495640 1 -- 172.21.15.116:0/1434897735 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa0e00a2b80 msgr2=0x7fa0e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:49.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.219+0000 7fa0f9495640 1 --2- 172.21.15.116:0/1434897735 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa0e00a2b80 0x7fa0e00a5040 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa0e409b760 tx=0x7fa0e406e890 comp rx=0 tx=0).stop 2024-09-12T03:09:49.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.219+0000 7fa0f9495640 1 -- 172.21.15.116:0/1434897735 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0f4106f10 msgr2=0x7fa0f419cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:49.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.219+0000 7fa0f9495640 1 --2- 172.21.15.116:0/1434897735 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0f4106f10 0x7fa0f419cc80 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fa0e400c970 tx=0x7fa0e400ce30 comp rx=0 tx=0).stop 2024-09-12T03:09:49.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.220+0000 7fa0f9495640 1 -- 172.21.15.116:0/1434897735 shutdown_connections 2024-09-12T03:09:49.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.220+0000 7fa0f9495640 1 --2- 172.21.15.116:0/1434897735 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa0e00a2b80 0x7fa0e00a5040 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.220+0000 7fa0f9495640 1 --2- 172.21.15.116:0/1434897735 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0f4106f10 0x7fa0f419cc80 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.220+0000 7fa0f9495640 1 --2- 172.21.15.116:0/1434897735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0f4102790 0x7fa0f41988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.220+0000 7fa0f9495640 1 --2- 172.21.15.116:0/1434897735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0f4101de0 0x7fa0f41983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.220+0000 7fa0f9495640 1 -- 172.21.15.116:0/1434897735 >> 172.21.15.116:0/1434897735 conn(0x7fa0f40fd8c0 msgr2=0x7fa0f40fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:49.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.220+0000 7fa0f9495640 1 -- 172.21.15.116:0/1434897735 shutdown_connections 2024-09-12T03:09:49.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:49.220+0000 7fa0f9495640 1 -- 172.21.15.116:0/1434897735 wait complete. 2024-09-12T03:09:49.954 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:09:50.200 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:09:50.612 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:50 smithi116 ceph-mon[31410]: pgmap v543: 97 pgs: 97 active+clean; 3.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:09:50.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:50 smithi183 ceph-mon[35535]: pgmap v543: 97 pgs: 97 active+clean; 3.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:09:50.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:50 smithi177 ceph-mon[35305]: pgmap v543: 97 pgs: 97 active+clean; 3.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:09:50.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.692+0000 7ff14f6f9640 1 -- 172.21.15.116:0/3352039588 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff148102da0 msgr2=0x7ff148103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:50.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.692+0000 7ff14f6f9640 1 --2- 172.21.15.116:0/3352039588 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff148102da0 0x7ff148103180 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7ff144009900 tx=0x7ff14402a480 comp rx=0 tx=0).stop 2024-09-12T03:09:50.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.692+0000 7ff14f6f9640 1 -- 172.21.15.116:0/3352039588 shutdown_connections 2024-09-12T03:09:50.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.692+0000 7ff14f6f9640 1 --2- 172.21.15.116:0/3352039588 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff14810c180 0x7ff14810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.692+0000 7ff14f6f9640 1 --2- 172.21.15.116:0/3352039588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1481036c0 0x7ff14810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.692+0000 7ff14f6f9640 1 --2- 172.21.15.116:0/3352039588 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff148102da0 0x7ff148103180 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.692+0000 7ff14f6f9640 1 -- 172.21.15.116:0/3352039588 >> 172.21.15.116:0/3352039588 conn(0x7ff1480fc7a0 msgr2=0x7ff1480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:50.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.692+0000 7ff14f6f9640 1 -- 172.21.15.116:0/3352039588 shutdown_connections 2024-09-12T03:09:50.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.692+0000 7ff14f6f9640 1 -- 172.21.15.116:0/3352039588 wait complete. 2024-09-12T03:09:50.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.693+0000 7ff14f6f9640 1 Processor -- start 2024-09-12T03:09:50.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.693+0000 7ff14f6f9640 1 -- start start 2024-09-12T03:09:50.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.693+0000 7ff14f6f9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff148102da0 0x7ff148198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:50.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.693+0000 7ff14f6f9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1481036c0 0x7ff148198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:50.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.693+0000 7ff14f6f9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff14810c180 0x7ff14819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:50.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.693+0000 7ff14f6f9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff148111f60 con 0x7ff14810c180 2024-09-12T03:09:50.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.693+0000 7ff14f6f9640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff148111de0 con 0x7ff1481036c0 2024-09-12T03:09:50.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.693+0000 7ff14f6f9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481120e0 con 0x7ff148102da0 2024-09-12T03:09:50.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.693+0000 7ff14dc96640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff14810c180 0x7ff14819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:50.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.693+0000 7ff14dc96640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff14810c180 0x7ff14819ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58504/0 (socket says 172.21.15.116:58504) 2024-09-12T03:09:50.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.693+0000 7ff14dc96640 1 -- 172.21.15.116:0/1790963933 learned_addr learned my addr 172.21.15.116:0/1790963933 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:50.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.693+0000 7ff14d495640 1 --2- 172.21.15.116:0/1790963933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff148102da0 0x7ff148198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:50.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.694+0000 7ff14cc94640 1 --2- 172.21.15.116:0/1790963933 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1481036c0 0x7ff148198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:50.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.694+0000 7ff14dc96640 1 -- 172.21.15.116:0/1790963933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff148102da0 msgr2=0x7ff148198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:50.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.694+0000 7ff14dc96640 1 --2- 172.21.15.116:0/1790963933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff148102da0 0x7ff148198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.694+0000 7ff14dc96640 1 -- 172.21.15.116:0/1790963933 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1481036c0 msgr2=0x7ff148198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:50.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.694+0000 7ff14dc96640 1 --2- 172.21.15.116:0/1790963933 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1481036c0 0x7ff148198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.694+0000 7ff14dc96640 1 -- 172.21.15.116:0/1790963933 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff14819d3d0 con 0x7ff14810c180 2024-09-12T03:09:50.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.694+0000 7ff14d495640 1 --2- 172.21.15.116:0/1790963933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff148102da0 0x7ff148198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:09:50.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.694+0000 7ff14dc96640 1 --2- 172.21.15.116:0/1790963933 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff14810c180 0x7ff14819ccf0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7ff14000eb10 tx=0x7ff14000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:50.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.694+0000 7ff13e7fc640 1 -- 172.21.15.116:0/1790963933 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140004b20 con 0x7ff14810c180 2024-09-12T03:09:50.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.694+0000 7ff13e7fc640 1 -- 172.21.15.116:0/1790963933 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff140004cc0 con 0x7ff14810c180 2024-09-12T03:09:50.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.694+0000 7ff14f6f9640 1 -- 172.21.15.116:0/1790963933 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff14819d6c0 con 0x7ff14810c180 2024-09-12T03:09:50.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.694+0000 7ff13e7fc640 1 -- 172.21.15.116:0/1790963933 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140010760 con 0x7ff14810c180 2024-09-12T03:09:50.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.694+0000 7ff14f6f9640 1 -- 172.21.15.116:0/1790963933 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1481a4fa0 con 0x7ff14810c180 2024-09-12T03:09:50.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.696+0000 7ff13e7fc640 1 -- 172.21.15.116:0/1790963933 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ff140005020 con 0x7ff14810c180 2024-09-12T03:09:50.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.697+0000 7ff151ad2740 1 Processor -- start 2024-09-12T03:09:50.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.697+0000 7ff151ad2740 1 -- start start 2024-09-12T03:09:50.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.697+0000 7ff151ad2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8cf6bfb40 0x55b8cf6be090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:50.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.697+0000 7ff151ad2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8cf6be5d0 0x55b8cf6bea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:50.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.697+0000 7ff151ad2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b8cf6c9370 0x55b8cf6cb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:50.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.697+0000 7ff151ad2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b8cf6242f0 con 0x55b8cf6c9370 2024-09-12T03:09:50.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.697+0000 7ff151ad2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b8cf5cbcf0 con 0x55b8cf6bfb40 2024-09-12T03:09:50.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.697+0000 7ff151ad2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b8cf6202b0 con 0x55b8cf6be5d0 2024-09-12T03:09:50.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.697+0000 7ff14cc94640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b8cf6c9370 0x55b8cf6cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:50.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.697+0000 7ff13e7fc640 1 --2- 172.21.15.116:0/1790963933 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff1340a2c00 0x7ff1340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:50.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.697+0000 7ff14cc94640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b8cf6c9370 0x55b8cf6cb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58514/0 (socket says 172.21.15.116:58514) 2024-09-12T03:09:50.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.697+0000 7ff14cc94640 1 -- 172.21.15.116:0/2525357126 learned_addr learned my addr 172.21.15.116:0/2525357126 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:50.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.697+0000 7ff14cc94640 1 --2- 172.21.15.116:0/2525357126 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8cf6bfb40 0x55b8cf6be090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:50.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.697+0000 7ff14dc96640 1 --2- 172.21.15.116:0/1790963933 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff1340a2c00 0x7ff1340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:50.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.697+0000 7ff14d495640 1 --2- 172.21.15.116:0/2525357126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8cf6be5d0 0x55b8cf6bea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:50.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.698+0000 7ff14cc94640 1 -- 172.21.15.116:0/2525357126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8cf6be5d0 msgr2=0x55b8cf6bea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:50.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.698+0000 7ff14cc94640 1 --2- 172.21.15.116:0/2525357126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8cf6be5d0 0x55b8cf6bea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.698+0000 7ff14cc94640 1 -- 172.21.15.116:0/2525357126 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8cf6bfb40 msgr2=0x55b8cf6be090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:50.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.698+0000 7ff14cc94640 1 --2- 172.21.15.116:0/2525357126 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8cf6bfb40 0x55b8cf6be090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.698+0000 7ff14cc94640 1 -- 172.21.15.116:0/2525357126 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8cf5b03a0 con 0x55b8cf6c9370 2024-09-12T03:09:50.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.698+0000 7ff14d495640 1 --2- 172.21.15.116:0/2525357126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8cf6be5d0 0x55b8cf6bea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:09:50.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.698+0000 7ff13e7fc640 1 -- 172.21.15.116:0/1790963933 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff1400cf700 con 0x7ff14810c180 2024-09-12T03:09:50.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.698+0000 7ff14cc94640 1 --2- 172.21.15.116:0/2525357126 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b8cf6c9370 0x55b8cf6cb760 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7ff138009fd0 tx=0x7ff1380096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1954b04ba8b2359b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:50.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.698+0000 7ff1337fe640 1 -- 172.21.15.116:0/2525357126 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff138019030 con 0x55b8cf6c9370 2024-09-12T03:09:50.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.698+0000 7ff14dc96640 1 --2- 172.21.15.116:0/1790963933 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff1340a2c00 0x7ff1340a50c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff1400a46d0 tx=0x7ff1400bc040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:50.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.698+0000 7ff1337fe640 1 -- 172.21.15.116:0/2525357126 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1380035f0 con 0x55b8cf6c9370 2024-09-12T03:09:50.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.698+0000 7ff1337fe640 1 -- 172.21.15.116:0/2525357126 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff138011670 con 0x55b8cf6c9370 2024-09-12T03:09:50.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.699+0000 7ff151ad2740 1 -- 172.21.15.116:0/2525357126 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b8cf6c9370 msgr2=0x55b8cf6cb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:50.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.699+0000 7ff151ad2740 1 --2- 172.21.15.116:0/2525357126 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b8cf6c9370 0x55b8cf6cb760 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7ff138009fd0 tx=0x7ff1380096e0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.699+0000 7ff151ad2740 1 -- 172.21.15.116:0/2525357126 shutdown_connections 2024-09-12T03:09:50.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.699+0000 7ff151ad2740 1 --2- 172.21.15.116:0/2525357126 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b8cf6c9370 0x55b8cf6cb760 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.699+0000 7ff151ad2740 1 --2- 172.21.15.116:0/2525357126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8cf6be5d0 0x55b8cf6bea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.699+0000 7ff151ad2740 1 --2- 172.21.15.116:0/2525357126 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8cf6bfb40 0x55b8cf6be090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.699+0000 7ff151ad2740 1 -- 172.21.15.116:0/2525357126 >> 172.21.15.116:0/2525357126 conn(0x55b8cf687310 msgr2=0x55b8cf6bac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:50.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.699+0000 7ff151ad2740 1 -- 172.21.15.116:0/2525357126 shutdown_connections 2024-09-12T03:09:50.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.699+0000 7ff151ad2740 1 -- 172.21.15.116:0/2525357126 wait complete. 2024-09-12T03:09:50.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.699+0000 7ff151ad2740 1 -- start start 2024-09-12T03:09:50.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.699+0000 7ff151ad2740 1 Processor -- start 2024-09-12T03:09:50.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.700+0000 7ff151ad2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b8cf6be5d0 0x55b8cf6c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:50.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.700+0000 7ff151ad2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8cf6bfb40 0x55b8cf6c6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:50.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.700+0000 7ff151ad2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8cf6c9370 0x55b8cf6c6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:50.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.700+0000 7ff151ad2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b8cf6cc070 con 0x55b8cf6be5d0 2024-09-12T03:09:50.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.700+0000 7ff151ad2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b8cf5df060 con 0x55b8cf6c9370 2024-09-12T03:09:50.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.700+0000 7ff151ad2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b8cf6cc1f0 con 0x55b8cf6bfb40 2024-09-12T03:09:50.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.700+0000 7ff151ad2740 10 client.0 _ll_register_callbacks cb 0x55b8cf3328c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:50.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.700+0000 7ff14cc94640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b8cf6be5d0 0x55b8cf6c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:50.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.700+0000 7ff14cc94640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b8cf6be5d0 0x55b8cf6c8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58528/0 (socket says 172.21.15.116:58528) 2024-09-12T03:09:50.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.700+0000 7ff14cc94640 1 -- 172.21.15.116:0/3300885478 learned_addr learned my addr 172.21.15.116:0/3300885478 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:50.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.700+0000 7ff14d495640 1 --2- 172.21.15.116:0/3300885478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8cf6bfb40 0x55b8cf6c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:50.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.700+0000 7ff14cc94640 1 -- 172.21.15.116:0/3300885478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8cf6bfb40 msgr2=0x55b8cf6c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:50.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.700+0000 7ff14cc94640 1 --2- 172.21.15.116:0/3300885478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8cf6bfb40 0x55b8cf6c6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.700+0000 7ff14cc94640 1 -- 172.21.15.116:0/3300885478 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8cf6c9370 msgr2=0x55b8cf6c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:50.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.700+0000 7ff14cc94640 1 --2- 172.21.15.116:0/3300885478 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8cf6c9370 0x55b8cf6c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.700+0000 7ff14cc94640 1 -- 172.21.15.116:0/3300885478 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8cf3953c0 con 0x55b8cf6be5d0 2024-09-12T03:09:50.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.700+0000 7ff14cc94640 1 --2- 172.21.15.116:0/3300885478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b8cf6be5d0 0x55b8cf6c8d70 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ff138006010 tx=0x7ff1380058c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54da5655414c12da server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:50.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.700+0000 7ff130ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:09:50.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.700+0000 7ff130ff9640 1 -- 172.21.15.116:0/3300885478 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff138019030 con 0x55b8cf6be5d0 2024-09-12T03:09:50.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.700+0000 7ff151ad2740 1 -- 172.21.15.116:0/3300885478 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1380035f0 con 0x55b8cf6be5d0 2024-09-12T03:09:50.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.700+0000 7ff130ff9640 1 -- 172.21.15.116:0/3300885478 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff138005040 con 0x55b8cf6be5d0 2024-09-12T03:09:50.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.701+0000 7ff130ff9640 1 -- 172.21.15.116:0/3300885478 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff138031070 con 0x55b8cf6be5d0 2024-09-12T03:09:50.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.701+0000 7ff151ad2740 10 client.17004 fetch_fsmap learned FSMap version 25 2024-09-12T03:09:50.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.701+0000 7ff151ad2740 1 -- 172.21.15.116:0/3300885478 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b8cf6d7a80 con 0x55b8cf6be5d0 2024-09-12T03:09:50.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.701+0000 7ff130ff9640 1 -- 172.21.15.116:0/3300885478 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1380035f0 con 0x55b8cf6be5d0 2024-09-12T03:09:50.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.701+0000 7ff130ff9640 1 -- 172.21.15.116:0/3300885478 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff13802a470 con 0x55b8cf6be5d0 2024-09-12T03:09:50.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.701+0000 7ff151ad2740 10 client.17004 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:09:50.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.701+0000 7ff151ad2740 10 client.17004 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:09:50.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.701+0000 7ff151ad2740 20 client.17004 populate_metadata read hostname 'smithi116' 2024-09-12T03:09:50.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.701+0000 7ff151ad2740 1 --2- 172.21.15.116:0/3300885478 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b8cf6d8e30 0x55b8cf6db220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:50.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.701+0000 7ff151ad2740 4 client.17004 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:09:50.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.701+0000 7ff151ad2740 1 -- 172.21.15.116:0/3300885478 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8cf631860 con 0x55b8cf6d8e30 2024-09-12T03:09:50.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.701+0000 7ff14d495640 1 --2- 172.21.15.116:0/3300885478 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b8cf6d8e30 0x55b8cf6db220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:50.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.702+0000 7ff14d495640 1 --2- 172.21.15.116:0/3300885478 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b8cf6d8e30 0x55b8cf6db220 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f2903a97ebb2d1c server_cookie=cf86faeb8f6e6fef in_seq=0 out_seq=0 2024-09-12T03:09:50.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.702+0000 7ff130ff9640 10 client.17004 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:50.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.702+0000 7ff130ff9640 1 -- 172.21.15.116:0/3300885478 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b8cf631860 con 0x55b8cf6d8e30 2024-09-12T03:09:50.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.702+0000 7ff130ff9640 10 client.17004 handle_command_reply: tid=0 2024-09-12T03:09:50.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.703+0000 7ff151ad2740 1 client.17004 shutdown 2024-09-12T03:09:50.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.703+0000 7ff151ad2740 1 -- 172.21.15.116:0/3300885478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b8cf6be5d0 msgr2=0x55b8cf6c8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:50.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.703+0000 7ff151ad2740 1 --2- 172.21.15.116:0/3300885478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b8cf6be5d0 0x55b8cf6c8d70 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ff138006010 tx=0x7ff1380058c0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.703+0000 7ff151ad2740 1 -- 172.21.15.116:0/3300885478 shutdown_connections 2024-09-12T03:09:50.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.703+0000 7ff151ad2740 1 --2- 172.21.15.116:0/3300885478 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b8cf6d8e30 0x55b8cf6db220 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.703+0000 7ff151ad2740 1 --2- 172.21.15.116:0/3300885478 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8cf6c9370 0x55b8cf6c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.703+0000 7ff130ff9640 0 client.17004 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:50.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.703+0000 7ff151ad2740 1 --2- 172.21.15.116:0/3300885478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8cf6bfb40 0x55b8cf6c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.703+0000 7ff151ad2740 1 --2- 172.21.15.116:0/3300885478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b8cf6be5d0 0x55b8cf6c8d70 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.703+0000 7ff151ad2740 1 -- 172.21.15.116:0/3300885478 >> 172.21.15.116:0/3300885478 conn(0x55b8cf687310 msgr2=0x55b8cf6c0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:50.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.703+0000 7ff151ad2740 1 -- 172.21.15.116:0/3300885478 shutdown_connections 2024-09-12T03:09:50.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.703+0000 7ff151ad2740 1 -- 172.21.15.116:0/3300885478 wait complete. 2024-09-12T03:09:50.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.703+0000 7ff151ad2740 20 client.17004 trim_cache size 0 max 16384 2024-09-12T03:09:50.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.716+0000 7ff151ad2740 1 Processor -- start 2024-09-12T03:09:50.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.716+0000 7ff151ad2740 1 -- start start 2024-09-12T03:09:50.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.716+0000 7ff151ad2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b8cf6be5d0 0x55b8cf6d5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:50.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.716+0000 7ff151ad2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8cf6bfb40 0x55b8cf6c1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:50.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.716+0000 7ff151ad2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8cf6c9370 0x55b8cf6c1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:50.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.716+0000 7ff151ad2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481afdf0 con 0x55b8cf6be5d0 2024-09-12T03:09:50.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.716+0000 7ff151ad2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481ad6c0 con 0x55b8cf6c9370 2024-09-12T03:09:50.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.716+0000 7ff151ad2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481ab510 con 0x55b8cf6bfb40 2024-09-12T03:09:50.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.716+0000 7ff14cc94640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b8cf6be5d0 0x55b8cf6d5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:50.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.716+0000 7ff14cc94640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b8cf6be5d0 0x55b8cf6d5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58536/0 (socket says 172.21.15.116:58536) 2024-09-12T03:09:50.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.716+0000 7ff14cc94640 1 -- 172.21.15.116:0/189523357 learned_addr learned my addr 172.21.15.116:0/189523357 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:50.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.716+0000 7ff14d495640 1 --2- 172.21.15.116:0/189523357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8cf6bfb40 0x55b8cf6c1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:50.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.716+0000 7ff14cc94640 1 -- 172.21.15.116:0/189523357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8cf6bfb40 msgr2=0x55b8cf6c1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:50.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.716+0000 7ff14cc94640 1 --2- 172.21.15.116:0/189523357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8cf6bfb40 0x55b8cf6c1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.716+0000 7ff14cc94640 1 -- 172.21.15.116:0/189523357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8cf6c9370 msgr2=0x55b8cf6c1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:50.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.716+0000 7ff14cc94640 1 --2- 172.21.15.116:0/189523357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8cf6c9370 0x55b8cf6c1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.716+0000 7ff14cc94640 1 -- 172.21.15.116:0/189523357 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8cf6c90e0 con 0x55b8cf6be5d0 2024-09-12T03:09:50.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.716+0000 7ff14cc94640 1 --2- 172.21.15.116:0/189523357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b8cf6be5d0 0x55b8cf6d5310 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7ff1380092e0 tx=0x7ff138009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2bb457edeb4862d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:50.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.717+0000 7ff132ffd640 1 -- 172.21.15.116:0/189523357 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff138019070 con 0x55b8cf6be5d0 2024-09-12T03:09:50.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.717+0000 7ff132ffd640 1 -- 172.21.15.116:0/189523357 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff138025070 con 0x55b8cf6be5d0 2024-09-12T03:09:50.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.717+0000 7ff132ffd640 1 -- 172.21.15.116:0/189523357 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13802ae60 con 0x55b8cf6be5d0 2024-09-12T03:09:50.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.717+0000 7ff151ad2740 1 -- 172.21.15.116:0/189523357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b8cf6be5d0 msgr2=0x55b8cf6d5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:50.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.717+0000 7ff151ad2740 1 --2- 172.21.15.116:0/189523357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b8cf6be5d0 0x55b8cf6d5310 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7ff1380092e0 tx=0x7ff138009670 comp rx=0 tx=0).stop 2024-09-12T03:09:50.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.717+0000 7ff151ad2740 1 -- 172.21.15.116:0/189523357 shutdown_connections 2024-09-12T03:09:50.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.717+0000 7ff151ad2740 1 --2- 172.21.15.116:0/189523357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8cf6c9370 0x55b8cf6c1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.717+0000 7ff151ad2740 1 --2- 172.21.15.116:0/189523357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8cf6bfb40 0x55b8cf6c1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.717+0000 7ff151ad2740 1 --2- 172.21.15.116:0/189523357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b8cf6be5d0 0x55b8cf6d5310 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.717+0000 7ff151ad2740 1 -- 172.21.15.116:0/189523357 >> 172.21.15.116:0/189523357 conn(0x55b8cf687310 msgr2=0x55b8cf6b9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:50.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.717+0000 7ff151ad2740 1 -- 172.21.15.116:0/189523357 shutdown_connections 2024-09-12T03:09:50.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.717+0000 7ff151ad2740 1 -- 172.21.15.116:0/189523357 wait complete. 2024-09-12T03:09:50.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.717+0000 7ff151ad2740 1 -- start start 2024-09-12T03:09:50.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.718+0000 7ff151ad2740 1 Processor -- start 2024-09-12T03:09:50.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.718+0000 7ff151ad2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b8cf6be5d0 0x55b8cf6d1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:50.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.718+0000 7ff151ad2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8cf6bfb40 0x55b8cf6d1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:50.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.718+0000 7ff151ad2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8cf6c9370 0x55b8cf6c2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:50.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.718+0000 7ff151ad2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481abb80 con 0x55b8cf6be5d0 2024-09-12T03:09:50.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.718+0000 7ff151ad2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481acff0 con 0x55b8cf6c9370 2024-09-12T03:09:50.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.718+0000 7ff151ad2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481ac250 con 0x55b8cf6bfb40 2024-09-12T03:09:50.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.718+0000 7ff151ad2740 10 client.0 _ll_register_callbacks cb 0x7ff138009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:50.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.718+0000 7ff14cc94640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b8cf6be5d0 0x55b8cf6d1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:50.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.718+0000 7ff14cc94640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b8cf6be5d0 0x55b8cf6d1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58542/0 (socket says 172.21.15.116:58542) 2024-09-12T03:09:50.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.718+0000 7ff14cc94640 1 -- 172.21.15.116:0/264784486 learned_addr learned my addr 172.21.15.116:0/264784486 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:50.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.718+0000 7ff14d495640 1 --2- 172.21.15.116:0/264784486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8cf6bfb40 0x55b8cf6d1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:50.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.718+0000 7ff14cc94640 1 -- 172.21.15.116:0/264784486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8cf6bfb40 msgr2=0x55b8cf6d1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:50.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.718+0000 7ff14cc94640 1 --2- 172.21.15.116:0/264784486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8cf6bfb40 0x55b8cf6d1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.718+0000 7ff14cc94640 1 -- 172.21.15.116:0/264784486 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8cf6c9370 msgr2=0x55b8cf6c2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:50.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.718+0000 7ff14cc94640 1 --2- 172.21.15.116:0/264784486 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8cf6c9370 0x55b8cf6c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.718+0000 7ff14cc94640 1 -- 172.21.15.116:0/264784486 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8cf6c3050 con 0x55b8cf6be5d0 2024-09-12T03:09:50.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.719+0000 7ff14cc94640 1 --2- 172.21.15.116:0/264784486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b8cf6be5d0 0x55b8cf6d1660 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ff138000c00 tx=0x7ff138009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11352c4b2f0981fb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:50.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.719+0000 7ff130ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:09:50.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.719+0000 7ff130ff9640 1 -- 172.21.15.116:0/264784486 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13803a070 con 0x55b8cf6be5d0 2024-09-12T03:09:50.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.719+0000 7ff130ff9640 1 -- 172.21.15.116:0/264784486 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff138003dd0 con 0x55b8cf6be5d0 2024-09-12T03:09:50.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.719+0000 7ff151ad2740 1 -- 172.21.15.116:0/264784486 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff138025070 con 0x55b8cf6be5d0 2024-09-12T03:09:50.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.719+0000 7ff130ff9640 1 -- 172.21.15.116:0/264784486 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff138046070 con 0x55b8cf6be5d0 2024-09-12T03:09:50.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.719+0000 7ff151ad2740 10 client.17010 fetch_fsmap learned FSMap version 25 2024-09-12T03:09:50.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.719+0000 7ff151ad2740 1 -- 172.21.15.116:0/264784486 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b8cf743210 con 0x55b8cf6be5d0 2024-09-12T03:09:50.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.719+0000 7ff130ff9640 1 -- 172.21.15.116:0/264784486 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff138025070 con 0x55b8cf6be5d0 2024-09-12T03:09:50.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.719+0000 7ff130ff9640 1 -- 172.21.15.116:0/264784486 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff13801e460 con 0x55b8cf6be5d0 2024-09-12T03:09:50.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.719+0000 7ff151ad2740 10 client.17010 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:09:50.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.719+0000 7ff151ad2740 10 client.17010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:09:50.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.719+0000 7ff151ad2740 20 client.17010 populate_metadata read hostname 'smithi116' 2024-09-12T03:09:50.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.719+0000 7ff151ad2740 1 --2- 172.21.15.116:0/264784486 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b8cf7437c0 0x55b8cf745bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:50.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.719+0000 7ff151ad2740 4 client.17010 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:09:50.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.719+0000 7ff151ad2740 1 -- 172.21.15.116:0/264784486 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b8cf62bd70 con 0x55b8cf7437c0 2024-09-12T03:09:50.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.720+0000 7ff14d495640 1 --2- 172.21.15.116:0/264784486 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b8cf7437c0 0x55b8cf745bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:50.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.720+0000 7ff130ff9640 10 client.17010 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:50.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.720+0000 7ff14d495640 1 --2- 172.21.15.116:0/264784486 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b8cf7437c0 0x55b8cf745bb0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e4bcc8dab9742c8 server_cookie=88bed345ad47fa66 in_seq=0 out_seq=0 2024-09-12T03:09:50.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.720+0000 7ff130ff9640 1 -- 172.21.15.116:0/264784486 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b8cf62bd70 con 0x55b8cf7437c0 2024-09-12T03:09:50.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.720+0000 7ff130ff9640 10 client.17010 handle_command_reply: tid=0 2024-09-12T03:09:50.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.720+0000 7ff151ad2740 1 client.17010 shutdown 2024-09-12T03:09:50.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.720+0000 7ff151ad2740 1 -- 172.21.15.116:0/264784486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b8cf6be5d0 msgr2=0x55b8cf6d1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:50.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.720+0000 7ff151ad2740 1 --2- 172.21.15.116:0/264784486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b8cf6be5d0 0x55b8cf6d1660 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ff138000c00 tx=0x7ff138009ce0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.720+0000 7ff151ad2740 1 -- 172.21.15.116:0/264784486 shutdown_connections 2024-09-12T03:09:50.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.720+0000 7ff151ad2740 1 --2- 172.21.15.116:0/264784486 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b8cf7437c0 0x55b8cf745bb0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.720+0000 7ff151ad2740 1 --2- 172.21.15.116:0/264784486 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8cf6c9370 0x55b8cf6c2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.720+0000 7ff151ad2740 1 --2- 172.21.15.116:0/264784486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8cf6bfb40 0x55b8cf6d1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.720+0000 7ff151ad2740 1 --2- 172.21.15.116:0/264784486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b8cf6be5d0 0x55b8cf6d1660 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.769 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:09:50.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.720+0000 7ff151ad2740 1 -- 172.21.15.116:0/264784486 >> 172.21.15.116:0/264784486 conn(0x55b8cf687310 msgr2=0x55b8cf6b94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:50.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.720+0000 7ff130ff9640 0 client.17010 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:50.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.720+0000 7ff151ad2740 1 -- 172.21.15.116:0/264784486 shutdown_connections 2024-09-12T03:09:50.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.721+0000 7ff151ad2740 1 -- 172.21.15.116:0/264784486 wait complete. 2024-09-12T03:09:50.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.721+0000 7ff151ad2740 20 client.17010 trim_cache size 0 max 16384 2024-09-12T03:09:50.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.721+0000 7ff14f6f9640 1 -- 172.21.15.116:0/1790963933 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff1340a2c00 msgr2=0x7ff1340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:50.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.721+0000 7ff14f6f9640 1 --2- 172.21.15.116:0/1790963933 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff1340a2c00 0x7ff1340a50c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff1400a46d0 tx=0x7ff1400bc040 comp rx=0 tx=0).stop 2024-09-12T03:09:50.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.721+0000 7ff14f6f9640 1 -- 172.21.15.116:0/1790963933 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff14810c180 msgr2=0x7ff14819ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:50.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.721+0000 7ff14f6f9640 1 --2- 172.21.15.116:0/1790963933 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff14810c180 0x7ff14819ccf0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7ff14000eb10 tx=0x7ff14000c2d0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.721+0000 7ff14f6f9640 1 -- 172.21.15.116:0/1790963933 shutdown_connections 2024-09-12T03:09:50.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.721+0000 7ff14f6f9640 1 --2- 172.21.15.116:0/1790963933 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff1340a2c00 0x7ff1340a50c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.721+0000 7ff14f6f9640 1 --2- 172.21.15.116:0/1790963933 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff14810c180 0x7ff14819ccf0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.721+0000 7ff14f6f9640 1 --2- 172.21.15.116:0/1790963933 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1481036c0 0x7ff148198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.721+0000 7ff14f6f9640 1 --2- 172.21.15.116:0/1790963933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff148102da0 0x7ff148198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:50.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.721+0000 7ff14f6f9640 1 -- 172.21.15.116:0/1790963933 >> 172.21.15.116:0/1790963933 conn(0x7ff1480fc7a0 msgr2=0x7ff1480fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:50.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.721+0000 7ff14f6f9640 1 -- 172.21.15.116:0/1790963933 shutdown_connections 2024-09-12T03:09:50.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:50.721+0000 7ff14f6f9640 1 -- 172.21.15.116:0/1790963933 wait complete. 2024-09-12T03:09:51.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:51 smithi183 ceph-mon[35535]: pgmap v544: 97 pgs: 97 active+clean; 3.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-12T03:09:51.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:51 smithi177 ceph-mon[35305]: pgmap v544: 97 pgs: 97 active+clean; 3.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-12T03:09:51.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:51 smithi116 ceph-mon[31410]: pgmap v544: 97 pgs: 97 active+clean; 3.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-12T03:09:52.533 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:09:52.533 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:09:52.784 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:09:53.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.291+0000 7f226b35b640 1 -- 172.21.15.116:0/3632720064 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2264101420 msgr2=0x7f226410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:53.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.291+0000 7f226b35b640 1 --2- 172.21.15.116:0/3632720064 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2264101420 0x7f226410d950 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f225c009cd0 tx=0x7f225c02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.291+0000 7f226b35b640 1 -- 172.21.15.116:0/3632720064 shutdown_connections 2024-09-12T03:09:53.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.291+0000 7f226b35b640 1 --2- 172.21.15.116:0/3632720064 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2264101420 0x7f226410d950 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.291+0000 7f226b35b640 1 --2- 172.21.15.116:0/3632720064 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2264100a80 0x7f2264100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.291+0000 7f226b35b640 1 --2- 172.21.15.116:0/3632720064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2264106a80 0x7f2264106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.291+0000 7f226b35b640 1 -- 172.21.15.116:0/3632720064 >> 172.21.15.116:0/3632720064 conn(0x7f22640fc7a0 msgr2=0x7f22640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:53.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.291+0000 7f226b35b640 1 -- 172.21.15.116:0/3632720064 shutdown_connections 2024-09-12T03:09:53.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.291+0000 7f226b35b640 1 -- 172.21.15.116:0/3632720064 wait complete. 2024-09-12T03:09:53.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.292+0000 7f226b35b640 1 Processor -- start 2024-09-12T03:09:53.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.292+0000 7f226b35b640 1 -- start start 2024-09-12T03:09:53.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.292+0000 7f226b35b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2264100a80 0x7f2264102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:53.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.292+0000 7f226b35b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2264101420 0x7f2264102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:53.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.292+0000 7f226b35b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2264106a80 0x7f22641030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:53.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.292+0000 7f226b35b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2264110f20 con 0x7f2264100a80 2024-09-12T03:09:53.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.292+0000 7f226b35b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2264110da0 con 0x7f2264106a80 2024-09-12T03:09:53.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.292+0000 7f226b35b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f22641110a0 con 0x7f2264101420 2024-09-12T03:09:53.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.292+0000 7f22690f7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2264100a80 0x7f2264102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:53.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.292+0000 7f22690f7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2264100a80 0x7f2264102650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40348/0 (socket says 172.21.15.116:40348) 2024-09-12T03:09:53.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.292+0000 7f22690f7640 1 -- 172.21.15.116:0/2343813440 learned_addr learned my addr 172.21.15.116:0/2343813440 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:53.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.292+0000 7f22688f6640 1 --2- 172.21.15.116:0/2343813440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2264101420 0x7f2264102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:53.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.292+0000 7f22690f7640 1 -- 172.21.15.116:0/2343813440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2264101420 msgr2=0x7f2264102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:53.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.292+0000 7f22690f7640 1 --2- 172.21.15.116:0/2343813440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2264101420 0x7f2264102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.292+0000 7f22690f7640 1 -- 172.21.15.116:0/2343813440 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2264106a80 msgr2=0x7f22641030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:53.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.292+0000 7f22698f8640 1 --2- 172.21.15.116:0/2343813440 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2264106a80 0x7f22641030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:53.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.292+0000 7f22690f7640 1 --2- 172.21.15.116:0/2343813440 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2264106a80 0x7f22641030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.292+0000 7f22690f7640 1 -- 172.21.15.116:0/2343813440 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22641a6a10 con 0x7f2264100a80 2024-09-12T03:09:53.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.292+0000 7f22690f7640 1 --2- 172.21.15.116:0/2343813440 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2264100a80 0x7f2264102650 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f226000b4a0 tx=0x7f226000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:53.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.293+0000 7f225a7fc640 1 -- 172.21.15.116:0/2343813440 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2260014070 con 0x7f2264100a80 2024-09-12T03:09:53.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.293+0000 7f225a7fc640 1 -- 172.21.15.116:0/2343813440 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2260010af0 con 0x7f2264100a80 2024-09-12T03:09:53.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.293+0000 7f226b35b640 1 -- 172.21.15.116:0/2343813440 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22641a6d00 con 0x7f2264100a80 2024-09-12T03:09:53.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.293+0000 7f225a7fc640 1 -- 172.21.15.116:0/2343813440 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2260017d10 con 0x7f2264100a80 2024-09-12T03:09:53.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.293+0000 7f226b35b640 1 -- 172.21.15.116:0/2343813440 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22641a7190 con 0x7f2264100a80 2024-09-12T03:09:53.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.295+0000 7f225a7fc640 1 -- 172.21.15.116:0/2343813440 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2260017070 con 0x7f2264100a80 2024-09-12T03:09:53.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.295+0000 7f226d734740 1 Processor -- start 2024-09-12T03:09:53.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.295+0000 7f226d734740 1 -- start start 2024-09-12T03:09:53.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.295+0000 7f226d734740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce0e69fb60 0x55ce0e69ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:53.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.296+0000 7f226d734740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce0e6a0480 0x55ce0e6aa800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:53.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.296+0000 7f226d734740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce0e6aad40 0x55ce0e6ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:53.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.296+0000 7f226d734740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ce0e605280 con 0x55ce0e6a0480 2024-09-12T03:09:53.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.296+0000 7f226d734740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ce0e5acb10 con 0x55ce0e6aad40 2024-09-12T03:09:53.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.296+0000 7f226d734740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ce0e601240 con 0x55ce0e69fb60 2024-09-12T03:09:53.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.296+0000 7f22690f7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce0e6a0480 0x55ce0e6aa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:53.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.296+0000 7f22690f7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce0e6a0480 0x55ce0e6aa800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40364/0 (socket says 172.21.15.116:40364) 2024-09-12T03:09:53.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.296+0000 7f22690f7640 1 -- 172.21.15.116:0/2452811929 learned_addr learned my addr 172.21.15.116:0/2452811929 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:53.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.296+0000 7f22688f6640 1 --2- 172.21.15.116:0/2452811929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce0e6aad40 0x55ce0e6ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:53.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.296+0000 7f22698f8640 1 --2- 172.21.15.116:0/2452811929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce0e69fb60 0x55ce0e69ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:53.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.296+0000 7f225a7fc640 1 --2- 172.21.15.116:0/2343813440 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f22500a2b80 0x7f22500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:53.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.296+0000 7f22690f7640 1 -- 172.21.15.116:0/2452811929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce0e69fb60 msgr2=0x55ce0e69ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:53.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.296+0000 7f22690f7640 1 --2- 172.21.15.116:0/2452811929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce0e69fb60 0x55ce0e69ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.296+0000 7f22690f7640 1 -- 172.21.15.116:0/2452811929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce0e6aad40 msgr2=0x55ce0e6ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:53.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.296+0000 7f22690f7640 1 --2- 172.21.15.116:0/2452811929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce0e6aad40 0x55ce0e6ad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.296+0000 7f22690f7640 1 -- 172.21.15.116:0/2452811929 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce0e5b91f0 con 0x55ce0e6a0480 2024-09-12T03:09:53.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.296+0000 7f22698f8640 1 --2- 172.21.15.116:0/2343813440 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f22500a2b80 0x7f22500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:53.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.296+0000 7f225a7fc640 1 -- 172.21.15.116:0/2343813440 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f22600ce4f0 con 0x7f2264100a80 2024-09-12T03:09:53.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.296+0000 7f22690f7640 1 --2- 172.21.15.116:0/2452811929 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce0e6a0480 0x55ce0e6aa800 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f2260072430 tx=0x7f22600658f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e3e82e27b360b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:53.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.297+0000 7f224f7fe640 1 -- 172.21.15.116:0/2452811929 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226009a860 con 0x55ce0e6a0480 2024-09-12T03:09:53.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.297+0000 7f224f7fe640 1 -- 172.21.15.116:0/2452811929 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2260071cb0 con 0x55ce0e6a0480 2024-09-12T03:09:53.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.297+0000 7f22698f8640 1 --2- 172.21.15.116:0/2343813440 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f22500a2b80 0x7f22500a5040 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f225c006010 tx=0x7f225c0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:53.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.297+0000 7f224f7fe640 1 -- 172.21.15.116:0/2452811929 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2260072090 con 0x55ce0e6a0480 2024-09-12T03:09:53.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.297+0000 7f226d734740 1 -- 172.21.15.116:0/2452811929 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce0e6a0480 msgr2=0x55ce0e6aa800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:53.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.297+0000 7f226d734740 1 --2- 172.21.15.116:0/2452811929 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce0e6a0480 0x55ce0e6aa800 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f2260072430 tx=0x7f22600658f0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.297+0000 7f226d734740 1 -- 172.21.15.116:0/2452811929 shutdown_connections 2024-09-12T03:09:53.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.297+0000 7f226d734740 1 --2- 172.21.15.116:0/2452811929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce0e6aad40 0x55ce0e6ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.297+0000 7f226d734740 1 --2- 172.21.15.116:0/2452811929 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce0e6a0480 0x55ce0e6aa800 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.297+0000 7f226d734740 1 --2- 172.21.15.116:0/2452811929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce0e69fb60 0x55ce0e69ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.297+0000 7f226d734740 1 -- 172.21.15.116:0/2452811929 >> 172.21.15.116:0/2452811929 conn(0x55ce0e69c0f0 msgr2=0x55ce0e69c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:53.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.297+0000 7f226d734740 1 -- 172.21.15.116:0/2452811929 shutdown_connections 2024-09-12T03:09:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.297+0000 7f226d734740 1 -- 172.21.15.116:0/2452811929 wait complete. 2024-09-12T03:09:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.298+0000 7f226d734740 1 -- start start 2024-09-12T03:09:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.298+0000 7f226d734740 1 Processor -- start 2024-09-12T03:09:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.298+0000 7f226d734740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce0e69fb60 0x55ce0e6a7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.298+0000 7f226d734740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce0e6a0480 0x55ce0e6a8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.298+0000 7f226d734740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce0e6aad40 0x55ce0e6a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.298+0000 7f226d734740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ce0e6ad8b0 con 0x55ce0e69fb60 2024-09-12T03:09:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.298+0000 7f226d734740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ce0e5bff10 con 0x55ce0e6aad40 2024-09-12T03:09:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.298+0000 7f226d734740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ce0e6ada30 con 0x55ce0e6a0480 2024-09-12T03:09:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.298+0000 7f226d734740 10 client.0 _ll_register_callbacks cb 0x55ce0e588a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.298+0000 7f22690f7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce0e69fb60 0x55ce0e6a7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.298+0000 7f22690f7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce0e69fb60 0x55ce0e6a7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40374/0 (socket says 172.21.15.116:40374) 2024-09-12T03:09:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.298+0000 7f22690f7640 1 -- 172.21.15.116:0/729742317 learned_addr learned my addr 172.21.15.116:0/729742317 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.299+0000 7f22698f8640 1 --2- 172.21.15.116:0/729742317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce0e6aad40 0x55ce0e6a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:53.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.299+0000 7f22690f7640 1 -- 172.21.15.116:0/729742317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce0e6a0480 msgr2=0x55ce0e6a8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:53.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.299+0000 7f22690f7640 1 --2- 172.21.15.116:0/729742317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce0e6a0480 0x55ce0e6a8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.299+0000 7f22690f7640 1 -- 172.21.15.116:0/729742317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce0e6aad40 msgr2=0x55ce0e6a8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:53.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.299+0000 7f22690f7640 1 --2- 172.21.15.116:0/729742317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce0e6aad40 0x55ce0e6a8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.299+0000 7f22690f7640 1 -- 172.21.15.116:0/729742317 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce0e3766a0 con 0x55ce0e69fb60 2024-09-12T03:09:53.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.299+0000 7f22698f8640 1 --2- 172.21.15.116:0/729742317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce0e6aad40 0x55ce0e6a8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:09:53.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.299+0000 7f22690f7640 1 --2- 172.21.15.116:0/729742317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce0e69fb60 0x55ce0e6a7be0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f226009aa20 tx=0x7f226009b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49fdae74b39b5dab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:53.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.299+0000 7f224cff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:09:53.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.299+0000 7f224cff9640 1 -- 172.21.15.116:0/729742317 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226001e050 con 0x55ce0e69fb60 2024-09-12T03:09:53.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.299+0000 7f224cff9640 1 -- 172.21.15.116:0/729742317 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f2260098dc0 con 0x55ce0e69fb60 2024-09-12T03:09:53.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.299+0000 7f226d734740 1 -- 172.21.15.116:0/729742317 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ce0e6127f0 con 0x55ce0e69fb60 2024-09-12T03:09:53.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.299+0000 7f224cff9640 1 -- 172.21.15.116:0/729742317 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2260072290 con 0x55ce0e69fb60 2024-09-12T03:09:53.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.299+0000 7f226d734740 10 client.17022 fetch_fsmap learned FSMap version 25 2024-09-12T03:09:53.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.299+0000 7f226d734740 1 -- 172.21.15.116:0/729742317 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ce0e6b94b0 con 0x55ce0e69fb60 2024-09-12T03:09:53.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.299+0000 7f224cff9640 1 -- 172.21.15.116:0/729742317 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ce0e6127f0 con 0x55ce0e69fb60 2024-09-12T03:09:53.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.300+0000 7f224cff9640 1 -- 172.21.15.116:0/729742317 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2260097390 con 0x55ce0e69fb60 2024-09-12T03:09:53.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.300+0000 7f226d734740 10 client.17022 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:09:53.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.300+0000 7f226d734740 10 client.17022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:09:53.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.300+0000 7f226d734740 20 client.17022 populate_metadata read hostname 'smithi116' 2024-09-12T03:09:53.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.300+0000 7f226d734740 1 --2- 172.21.15.116:0/729742317 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ce0e6ba800 0x55ce0e6bcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:53.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.300+0000 7f226d734740 4 client.17022 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:09:53.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.300+0000 7f226d734740 1 -- 172.21.15.116:0/729742317 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce0e60cd00 con 0x55ce0e6ba800 2024-09-12T03:09:53.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.300+0000 7f22688f6640 1 --2- 172.21.15.116:0/729742317 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ce0e6ba800 0x55ce0e6bcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:53.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.300+0000 7f224cff9640 10 client.17022 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:53.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.300+0000 7f22688f6640 1 --2- 172.21.15.116:0/729742317 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ce0e6ba800 0x55ce0e6bcbf0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc1747a44867b7c5 server_cookie=37c3f7686640ba35 in_seq=0 out_seq=0 2024-09-12T03:09:53.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.301+0000 7f224cff9640 1 -- 172.21.15.116:0/729742317 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ce0e60cd00 con 0x55ce0e6ba800 2024-09-12T03:09:53.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.301+0000 7f224cff9640 10 client.17022 handle_command_reply: tid=0 2024-09-12T03:09:53.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.301+0000 7f226d734740 1 client.17022 shutdown 2024-09-12T03:09:53.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.301+0000 7f226d734740 1 -- 172.21.15.116:0/729742317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce0e69fb60 msgr2=0x55ce0e6a7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:53.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.301+0000 7f226d734740 1 --2- 172.21.15.116:0/729742317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce0e69fb60 0x55ce0e6a7be0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f226009aa20 tx=0x7f226009b490 comp rx=0 tx=0).stop 2024-09-12T03:09:53.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.301+0000 7f226d734740 1 -- 172.21.15.116:0/729742317 shutdown_connections 2024-09-12T03:09:53.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.301+0000 7f226d734740 1 --2- 172.21.15.116:0/729742317 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ce0e6ba800 0x55ce0e6bcbf0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.301+0000 7f226d734740 1 --2- 172.21.15.116:0/729742317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce0e6aad40 0x55ce0e6a8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.301+0000 7f226d734740 1 --2- 172.21.15.116:0/729742317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce0e6a0480 0x55ce0e6a8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.301+0000 7f226d734740 1 --2- 172.21.15.116:0/729742317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce0e69fb60 0x55ce0e6a7be0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.301+0000 7f226d734740 1 -- 172.21.15.116:0/729742317 >> 172.21.15.116:0/729742317 conn(0x55ce0e69c0f0 msgr2=0x55ce0e6a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:53.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.301+0000 7f224cff9640 0 client.17022 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:53.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.301+0000 7f226d734740 1 -- 172.21.15.116:0/729742317 shutdown_connections 2024-09-12T03:09:53.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.301+0000 7f226d734740 1 -- 172.21.15.116:0/729742317 wait complete. 2024-09-12T03:09:53.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.302+0000 7f226d734740 20 client.17022 trim_cache size 0 max 16384 2024-09-12T03:09:53.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.318+0000 7f226d734740 1 Processor -- start 2024-09-12T03:09:53.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.318+0000 7f226d734740 1 -- start start 2024-09-12T03:09:53.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.318+0000 7f226d734740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce0e69fb60 0x55ce0e6a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:53.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.318+0000 7f226d734740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce0e6a0480 0x55ce0e6a5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:53.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.318+0000 7f226d734740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce0e6aad40 0x55ce0e6a6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:53.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.319+0000 7f226d734740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f22641b2080 con 0x55ce0e6a0480 2024-09-12T03:09:53.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.319+0000 7f226d734740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f22641af950 con 0x55ce0e69fb60 2024-09-12T03:09:53.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.319+0000 7f226d734740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f22641ad7a0 con 0x55ce0e6aad40 2024-09-12T03:09:53.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.319+0000 7f22688f6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce0e6a0480 0x55ce0e6a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:53.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.319+0000 7f22688f6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce0e6a0480 0x55ce0e6a5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40378/0 (socket says 172.21.15.116:40378) 2024-09-12T03:09:53.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.319+0000 7f22688f6640 1 -- 172.21.15.116:0/175784 learned_addr learned my addr 172.21.15.116:0/175784 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:53.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.319+0000 7f22698f8640 1 --2- 172.21.15.116:0/175784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce0e6aad40 0x55ce0e6a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:53.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.319+0000 7f22688f6640 1 -- 172.21.15.116:0/175784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce0e6aad40 msgr2=0x55ce0e6a6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:53.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.319+0000 7f22688f6640 1 --2- 172.21.15.116:0/175784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce0e6aad40 0x55ce0e6a6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.319+0000 7f22688f6640 1 -- 172.21.15.116:0/175784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce0e69fb60 msgr2=0x55ce0e6a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:53.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.319+0000 7f22688f6640 1 --2- 172.21.15.116:0/175784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce0e69fb60 0x55ce0e6a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.319+0000 7f22688f6640 1 -- 172.21.15.116:0/175784 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce0e69bf20 con 0x55ce0e6a0480 2024-09-12T03:09:53.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.319+0000 7f22698f8640 1 --2- 172.21.15.116:0/175784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce0e6aad40 0x55ce0e6a6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:09:53.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.319+0000 7f22688f6640 1 --2- 172.21.15.116:0/175784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce0e6a0480 0x55ce0e6a5380 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f2254004810 tx=0x7f225400b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426558740d90383c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:53.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.319+0000 7f224effd640 1 -- 172.21.15.116:0/175784 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22540144d0 con 0x55ce0e6a0480 2024-09-12T03:09:53.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.319+0000 7f224effd640 1 -- 172.21.15.116:0/175784 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f225401e030 con 0x55ce0e6a0480 2024-09-12T03:09:53.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.319+0000 7f224effd640 1 -- 172.21.15.116:0/175784 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2254022a10 con 0x55ce0e6a0480 2024-09-12T03:09:53.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.320+0000 7f226d734740 1 -- 172.21.15.116:0/175784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce0e6a0480 msgr2=0x55ce0e6a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:53.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.320+0000 7f226d734740 1 --2- 172.21.15.116:0/175784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce0e6a0480 0x55ce0e6a5380 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f2254004810 tx=0x7f225400b800 comp rx=0 tx=0).stop 2024-09-12T03:09:53.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.320+0000 7f226d734740 1 -- 172.21.15.116:0/175784 shutdown_connections 2024-09-12T03:09:53.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.320+0000 7f226d734740 1 --2- 172.21.15.116:0/175784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce0e6aad40 0x55ce0e6a6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.320+0000 7f226d734740 1 --2- 172.21.15.116:0/175784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce0e6a0480 0x55ce0e6a5380 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.320+0000 7f226d734740 1 --2- 172.21.15.116:0/175784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce0e69fb60 0x55ce0e6a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.320+0000 7f226d734740 1 -- 172.21.15.116:0/175784 >> 172.21.15.116:0/175784 conn(0x55ce0e69c0f0 msgr2=0x55ce0e6b64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:53.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.320+0000 7f226d734740 1 -- 172.21.15.116:0/175784 shutdown_connections 2024-09-12T03:09:53.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.320+0000 7f226d734740 1 -- 172.21.15.116:0/175784 wait complete. 2024-09-12T03:09:53.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.320+0000 7f226d734740 1 -- start start 2024-09-12T03:09:53.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.320+0000 7f226d734740 1 Processor -- start 2024-09-12T03:09:53.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.321+0000 7f226d734740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce0e69fb60 0x55ce0e6bff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:53.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.321+0000 7f226d734740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce0e6a0480 0x55ce0e6bd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:53.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.321+0000 7f226d734740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce0e6aad40 0x55ce0e6bd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:53.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.321+0000 7f226d734740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f22641ade10 con 0x55ce0e69fb60 2024-09-12T03:09:53.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.321+0000 7f226d734740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f22641af280 con 0x55ce0e6a0480 2024-09-12T03:09:53.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.321+0000 7f226d734740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f22641ae4e0 con 0x55ce0e6aad40 2024-09-12T03:09:53.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.321+0000 7f226d734740 10 client.0 _ll_register_callbacks cb 0x7f225400b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:53.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.321+0000 7f22690f7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce0e69fb60 0x55ce0e6bff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:53.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.321+0000 7f22690f7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce0e69fb60 0x55ce0e6bff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40384/0 (socket says 172.21.15.116:40384) 2024-09-12T03:09:53.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.321+0000 7f22690f7640 1 -- 172.21.15.116:0/3496938512 learned_addr learned my addr 172.21.15.116:0/3496938512 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:53.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.321+0000 7f22698f8640 1 --2- 172.21.15.116:0/3496938512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce0e6aad40 0x55ce0e6bd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:53.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.321+0000 7f22690f7640 1 -- 172.21.15.116:0/3496938512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce0e6aad40 msgr2=0x55ce0e6bd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:53.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.321+0000 7f22690f7640 1 --2- 172.21.15.116:0/3496938512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce0e6aad40 0x55ce0e6bd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.321+0000 7f22690f7640 1 -- 172.21.15.116:0/3496938512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce0e6a0480 msgr2=0x55ce0e6bd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:53.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.321+0000 7f22688f6640 1 --2- 172.21.15.116:0/3496938512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce0e6a0480 0x55ce0e6bd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:53.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.321+0000 7f22690f7640 1 --2- 172.21.15.116:0/3496938512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce0e6a0480 0x55ce0e6bd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.321+0000 7f22690f7640 1 -- 172.21.15.116:0/3496938512 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce0e6bdff0 con 0x55ce0e69fb60 2024-09-12T03:09:53.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.321+0000 7f22688f6640 1 --2- 172.21.15.116:0/3496938512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce0e6a0480 0x55ce0e6bd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:09:53.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.321+0000 7f22690f7640 1 --2- 172.21.15.116:0/3496938512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce0e69fb60 0x55ce0e6bff60 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f2260097180 tx=0x7f22600092b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1bbaa12684a6af1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:53.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.321+0000 7f224cff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:09:53.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.321+0000 7f224cff9640 1 -- 172.21.15.116:0/3496938512 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226001e070 con 0x55ce0e69fb60 2024-09-12T03:09:53.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.321+0000 7f224cff9640 1 -- 172.21.15.116:0/3496938512 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f2260010ca0 con 0x55ce0e69fb60 2024-09-12T03:09:53.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.321+0000 7f226d734740 1 -- 172.21.15.116:0/3496938512 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f225401e030 con 0x55ce0e69fb60 2024-09-12T03:09:53.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.321+0000 7f224cff9640 1 -- 172.21.15.116:0/3496938512 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f22600d2020 con 0x55ce0e69fb60 2024-09-12T03:09:53.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.322+0000 7f226d734740 10 client.17028 fetch_fsmap learned FSMap version 25 2024-09-12T03:09:53.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.322+0000 7f226d734740 1 -- 172.21.15.116:0/3496938512 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ce0e7250c0 con 0x55ce0e69fb60 2024-09-12T03:09:53.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.322+0000 7f224cff9640 1 -- 172.21.15.116:0/3496938512 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f225401e030 con 0x55ce0e69fb60 2024-09-12T03:09:53.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.322+0000 7f224cff9640 1 -- 172.21.15.116:0/3496938512 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f226009af80 con 0x55ce0e69fb60 2024-09-12T03:09:53.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.322+0000 7f226d734740 10 client.17028 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:09:53.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.322+0000 7f226d734740 10 client.17028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:09:53.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.322+0000 7f226d734740 20 client.17028 populate_metadata read hostname 'smithi116' 2024-09-12T03:09:53.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.322+0000 7f226d734740 1 --2- 172.21.15.116:0/3496938512 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ce0e7254f0 0x55ce0e7278e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:53.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.322+0000 7f226d734740 4 client.17028 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:09:53.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.322+0000 7f226d734740 1 -- 172.21.15.116:0/3496938512 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ce0e589110 con 0x55ce0e7254f0 2024-09-12T03:09:53.348 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:09:53.348 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:09:53.348 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "65449ba8-4acb-4f43-a689-55631aace3c9", 2024-09-12T03:09:53.348 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:09:53.348 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:09:53.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.322+0000 7f22688f6640 1 --2- 172.21.15.116:0/3496938512 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ce0e7254f0 0x55ce0e7278e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:53.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.322+0000 7f22688f6640 1 --2- 172.21.15.116:0/3496938512 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ce0e7254f0 0x55ce0e7278e0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ddc8ce95ddd8e76 server_cookie=45e51973f04105d3 in_seq=0 out_seq=0 2024-09-12T03:09:53.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.322+0000 7f224cff9640 10 client.17028 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:53.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.324+0000 7f224cff9640 1 -- 172.21.15.116:0/3496938512 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ce0e589110 con 0x55ce0e7254f0 2024-09-12T03:09:53.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.324+0000 7f224cff9640 10 client.17028 handle_command_reply: tid=0 2024-09-12T03:09:53.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.324+0000 7f226d734740 1 client.17028 shutdown 2024-09-12T03:09:53.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.324+0000 7f226d734740 1 -- 172.21.15.116:0/3496938512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce0e69fb60 msgr2=0x55ce0e6bff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:53.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.324+0000 7f226d734740 1 --2- 172.21.15.116:0/3496938512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce0e69fb60 0x55ce0e6bff60 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f2260097180 tx=0x7f22600092b0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.324+0000 7f226d734740 1 -- 172.21.15.116:0/3496938512 shutdown_connections 2024-09-12T03:09:53.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.324+0000 7f226d734740 1 --2- 172.21.15.116:0/3496938512 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ce0e7254f0 0x55ce0e7278e0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.324+0000 7f224cff9640 0 client.17028 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:53.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.324+0000 7f226d734740 1 --2- 172.21.15.116:0/3496938512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce0e6aad40 0x55ce0e6bd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.324+0000 7f226d734740 1 --2- 172.21.15.116:0/3496938512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce0e6a0480 0x55ce0e6bd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.324+0000 7f226d734740 1 --2- 172.21.15.116:0/3496938512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce0e69fb60 0x55ce0e6bff60 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.324+0000 7f226d734740 1 -- 172.21.15.116:0/3496938512 >> 172.21.15.116:0/3496938512 conn(0x55ce0e69c0f0 msgr2=0x55ce0e6a15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:53.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.324+0000 7f226d734740 1 -- 172.21.15.116:0/3496938512 shutdown_connections 2024-09-12T03:09:53.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.324+0000 7f226d734740 1 -- 172.21.15.116:0/3496938512 wait complete. 2024-09-12T03:09:53.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.324+0000 7f226d734740 20 client.17028 trim_cache size 0 max 16384 2024-09-12T03:09:53.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.325+0000 7f226b35b640 1 -- 172.21.15.116:0/2343813440 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f22500a2b80 msgr2=0x7f22500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:53.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.325+0000 7f226b35b640 1 --2- 172.21.15.116:0/2343813440 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f22500a2b80 0x7f22500a5040 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f225c006010 tx=0x7f225c0095c0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.325+0000 7f226b35b640 1 -- 172.21.15.116:0/2343813440 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2264100a80 msgr2=0x7f2264102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:53.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.325+0000 7f226b35b640 1 --2- 172.21.15.116:0/2343813440 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2264100a80 0x7f2264102650 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f226000b4a0 tx=0x7f226000b960 comp rx=0 tx=0).stop 2024-09-12T03:09:53.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.325+0000 7f226b35b640 1 -- 172.21.15.116:0/2343813440 shutdown_connections 2024-09-12T03:09:53.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.325+0000 7f226b35b640 1 --2- 172.21.15.116:0/2343813440 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f22500a2b80 0x7f22500a5040 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.325+0000 7f226b35b640 1 --2- 172.21.15.116:0/2343813440 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2264106a80 0x7f22641030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.325+0000 7f226b35b640 1 --2- 172.21.15.116:0/2343813440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2264101420 0x7f2264102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.325+0000 7f226b35b640 1 --2- 172.21.15.116:0/2343813440 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2264100a80 0x7f2264102650 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.325+0000 7f226b35b640 1 -- 172.21.15.116:0/2343813440 >> 172.21.15.116:0/2343813440 conn(0x7f22640fc7a0 msgr2=0x7f22641048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:53.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.325+0000 7f226b35b640 1 -- 172.21.15.116:0/2343813440 shutdown_connections 2024-09-12T03:09:53.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:53.325+0000 7f226b35b640 1 -- 172.21.15.116:0/2343813440 wait complete. 2024-09-12T03:09:53.986 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 16.6 2024-09-12T03:09:53.986 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:09:54.149 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:09:54.241 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:09:54.684 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:54 smithi116 ceph-mon[31410]: pgmap v545: 97 pgs: 97 active+clean; 3.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:09:54.684 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:54 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:09:54.684 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:54 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:09:54.684 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:54 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:09:54.684 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:54 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:09:54.684 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:54 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:09:54.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:54 smithi183 ceph-mon[35535]: pgmap v545: 97 pgs: 97 active+clean; 3.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:09:54.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:54 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:09:54.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:54 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:09:54.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:54 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:09:54.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:54 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:09:54.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:54 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:09:54.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:54 smithi177 ceph-mon[35305]: pgmap v545: 97 pgs: 97 active+clean; 3.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:09:54.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:54 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:09:54.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:54 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:09:54.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:54 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:09:54.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:54 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:09:54.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:54 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:09:54.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.763+0000 7f85a9721640 1 -- 172.21.15.116:0/1238817706 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85a4102790 msgr2=0x7f85a41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:54.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.763+0000 7f85a9721640 1 --2- 172.21.15.116:0/1238817706 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85a4102790 0x7f85a41067e0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f858c009900 tx=0x7f858c02a410 comp rx=0 tx=0).stop 2024-09-12T03:09:54.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.763+0000 7f85a9721640 1 -- 172.21.15.116:0/1238817706 shutdown_connections 2024-09-12T03:09:54.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.763+0000 7f85a9721640 1 --2- 172.21.15.116:0/1238817706 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85a4106f10 0x7f85a410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:54.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.763+0000 7f85a9721640 1 --2- 172.21.15.116:0/1238817706 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85a4102790 0x7f85a41067e0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:54.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.763+0000 7f85a9721640 1 --2- 172.21.15.116:0/1238817706 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85a4101de0 0x7f85a41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:54.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.763+0000 7f85a9721640 1 -- 172.21.15.116:0/1238817706 >> 172.21.15.116:0/1238817706 conn(0x7f85a40fd8c0 msgr2=0x7f85a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:54.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.763+0000 7f85a9721640 1 -- 172.21.15.116:0/1238817706 shutdown_connections 2024-09-12T03:09:54.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.763+0000 7f85a9721640 1 -- 172.21.15.116:0/1238817706 wait complete. 2024-09-12T03:09:54.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.763+0000 7f85a9721640 1 Processor -- start 2024-09-12T03:09:54.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.764+0000 7f85a9721640 1 -- start start 2024-09-12T03:09:54.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.764+0000 7f85a9721640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85a4101de0 0x7f85a4198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:54.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.764+0000 7f85a9721640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85a4102790 0x7f85a4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:54.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.764+0000 7f85a9721640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85a4106f10 0x7f85a419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:54.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.764+0000 7f85a9721640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f85a4111f70 con 0x7f85a4106f10 2024-09-12T03:09:54.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.764+0000 7f85a9721640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f85a4111df0 con 0x7f85a4101de0 2024-09-12T03:09:54.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.764+0000 7f85a9721640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f85a41120f0 con 0x7f85a4102790 2024-09-12T03:09:54.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.764+0000 7f85a37fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85a4106f10 0x7f85a419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:54.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.764+0000 7f85a37fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85a4106f10 0x7f85a419cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40396/0 (socket says 172.21.15.116:40396) 2024-09-12T03:09:54.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.764+0000 7f85a37fe640 1 -- 172.21.15.116:0/702202953 learned_addr learned my addr 172.21.15.116:0/702202953 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:54.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.764+0000 7f85a27fc640 1 --2- 172.21.15.116:0/702202953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85a4102790 0x7f85a4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:54.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.764+0000 7f85a2ffd640 1 --2- 172.21.15.116:0/702202953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85a4101de0 0x7f85a4198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:54.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.764+0000 7f85a37fe640 1 -- 172.21.15.116:0/702202953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85a4102790 msgr2=0x7f85a4198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:54.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.764+0000 7f85a37fe640 1 --2- 172.21.15.116:0/702202953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85a4102790 0x7f85a4198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:54.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.764+0000 7f85a37fe640 1 -- 172.21.15.116:0/702202953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85a4101de0 msgr2=0x7f85a4198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:54.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.764+0000 7f85a37fe640 1 --2- 172.21.15.116:0/702202953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85a4101de0 0x7f85a4198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:54.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.764+0000 7f85a37fe640 1 -- 172.21.15.116:0/702202953 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a419d4c0 con 0x7f85a4106f10 2024-09-12T03:09:54.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.764+0000 7f85a37fe640 1 --2- 172.21.15.116:0/702202953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85a4106f10 0x7f85a419cde0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f859c009fd0 tx=0x7f859c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:54.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.764+0000 7f859bfff640 1 -- 172.21.15.116:0/702202953 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c017070 con 0x7f85a4106f10 2024-09-12T03:09:54.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.764+0000 7f85a9721640 1 -- 172.21.15.116:0/702202953 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a419d7b0 con 0x7f85a4106f10 2024-09-12T03:09:54.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.764+0000 7f859bfff640 1 -- 172.21.15.116:0/702202953 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f859c003e50 con 0x7f85a4106f10 2024-09-12T03:09:54.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.764+0000 7f859bfff640 1 -- 172.21.15.116:0/702202953 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c0106c0 con 0x7f85a4106f10 2024-09-12T03:09:54.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.765+0000 7f85a9721640 1 -- 172.21.15.116:0/702202953 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a41a5090 con 0x7f85a4106f10 2024-09-12T03:09:54.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.766+0000 7f85a9721640 1 -- 172.21.15.116:0/702202953 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85a41021c0 con 0x7f85a4106f10 2024-09-12T03:09:54.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.767+0000 7f859bfff640 1 -- 172.21.15.116:0/702202953 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f859c003970 con 0x7f85a4106f10 2024-09-12T03:09:54.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.768+0000 7f859bfff640 1 --2- 172.21.15.116:0/702202953 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f85900a2b80 0x7f85900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:54.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.768+0000 7f85a2ffd640 1 --2- 172.21.15.116:0/702202953 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f85900a2b80 0x7f85900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:54.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.768+0000 7f859bfff640 1 -- 172.21.15.116:0/702202953 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f859c0cedd0 con 0x7f85a4106f10 2024-09-12T03:09:54.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.769+0000 7f85a2ffd640 1 --2- 172.21.15.116:0/702202953 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f85900a2b80 0x7f85900a5040 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8594005fd0 tx=0x7f8594005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:54.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.771+0000 7f859bfff640 1 -- 172.21.15.116:0/702202953 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f859c09bd60 con 0x7f85a4106f10 2024-09-12T03:09:54.923 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:09:54.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.978+0000 7f85a9721640 1 -- 172.21.15.116:0/702202953 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85a410a150 con 0x7f85a4106f10 2024-09-12T03:09:54.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.979+0000 7f859bfff640 1 -- 172.21.15.116:0/702202953 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f859c09b660 con 0x7f85a4106f10 2024-09-12T03:09:54.980 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:09:54.980 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:09:54.980 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:09:54.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.982+0000 7f85a9721640 1 -- 172.21.15.116:0/702202953 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f85900a2b80 msgr2=0x7f85900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:54.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.982+0000 7f85a9721640 1 --2- 172.21.15.116:0/702202953 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f85900a2b80 0x7f85900a5040 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8594005fd0 tx=0x7f8594005850 comp rx=0 tx=0).stop 2024-09-12T03:09:54.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.982+0000 7f85a9721640 1 -- 172.21.15.116:0/702202953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85a4106f10 msgr2=0x7f85a419cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:54.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.982+0000 7f85a9721640 1 --2- 172.21.15.116:0/702202953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85a4106f10 0x7f85a419cde0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f859c009fd0 tx=0x7f859c00b260 comp rx=0 tx=0).stop 2024-09-12T03:09:54.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.982+0000 7f85a9721640 1 -- 172.21.15.116:0/702202953 shutdown_connections 2024-09-12T03:09:54.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.982+0000 7f85a9721640 1 --2- 172.21.15.116:0/702202953 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f85900a2b80 0x7f85900a5040 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:54.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.982+0000 7f85a9721640 1 --2- 172.21.15.116:0/702202953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85a4106f10 0x7f85a419cde0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:54.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.982+0000 7f85a9721640 1 --2- 172.21.15.116:0/702202953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85a4102790 0x7f85a4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:54.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.982+0000 7f85a9721640 1 --2- 172.21.15.116:0/702202953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85a4101de0 0x7f85a4198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:54.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.982+0000 7f85a9721640 1 -- 172.21.15.116:0/702202953 >> 172.21.15.116:0/702202953 conn(0x7f85a40fd8c0 msgr2=0x7f85a40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:54.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.982+0000 7f85a9721640 1 -- 172.21.15.116:0/702202953 shutdown_connections 2024-09-12T03:09:54.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:54.982+0000 7f85a9721640 1 -- 172.21.15.116:0/702202953 wait complete. 2024-09-12T03:09:55.571 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:55 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/702202953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:09:55.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.570+0000 7fc650432640 1 -- 172.21.15.116:0/1080366246 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc648102d40 msgr2=0x7fc648103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:55.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.570+0000 7fc650432640 1 --2- 172.21.15.116:0/1080366246 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc648102d40 0x7fc648103120 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fc644009900 tx=0x7fc64402a480 comp rx=0 tx=0).stop 2024-09-12T03:09:55.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.570+0000 7fc650432640 1 -- 172.21.15.116:0/1080366246 shutdown_connections 2024-09-12T03:09:55.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.570+0000 7fc650432640 1 --2- 172.21.15.116:0/1080366246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc64810c170 0x7fc64810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.570+0000 7fc650432640 1 --2- 172.21.15.116:0/1080366246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc648103660 0x7fc64810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.570+0000 7fc650432640 1 --2- 172.21.15.116:0/1080366246 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc648102d40 0x7fc648103120 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.570+0000 7fc650432640 1 -- 172.21.15.116:0/1080366246 >> 172.21.15.116:0/1080366246 conn(0x7fc6480fc7a0 msgr2=0x7fc6480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:55.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.570+0000 7fc650432640 1 -- 172.21.15.116:0/1080366246 shutdown_connections 2024-09-12T03:09:55.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.571+0000 7fc650432640 1 -- 172.21.15.116:0/1080366246 wait complete. 2024-09-12T03:09:55.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.571+0000 7fc650432640 1 Processor -- start 2024-09-12T03:09:55.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.571+0000 7fc650432640 1 -- start start 2024-09-12T03:09:55.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.571+0000 7fc650432640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc648102d40 0x7fc6481009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:55.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.571+0000 7fc650432640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc648103660 0x7fc648100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:55.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.571+0000 7fc650432640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc64810c170 0x7fc648101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:55.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.571+0000 7fc650432640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc648111f70 con 0x7fc648103660 2024-09-12T03:09:55.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.571+0000 7fc650432640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc648111df0 con 0x7fc64810c170 2024-09-12T03:09:55.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.571+0000 7fc650432640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc6481120f0 con 0x7fc648102d40 2024-09-12T03:09:55.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.571+0000 7fc64d9cd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc648103660 0x7fc648100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:55.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.571+0000 7fc64d9cd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc648103660 0x7fc648100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40414/0 (socket says 172.21.15.116:40414) 2024-09-12T03:09:55.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.571+0000 7fc64d9cd640 1 -- 172.21.15.116:0/1903224313 learned_addr learned my addr 172.21.15.116:0/1903224313 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:55.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.572+0000 7fc64e1ce640 1 --2- 172.21.15.116:0/1903224313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc648102d40 0x7fc6481009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:55.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.572+0000 7fc64e9cf640 1 --2- 172.21.15.116:0/1903224313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc64810c170 0x7fc648101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:55.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.572+0000 7fc64d9cd640 1 -- 172.21.15.116:0/1903224313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc648102d40 msgr2=0x7fc6481009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:55.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.572+0000 7fc64d9cd640 1 --2- 172.21.15.116:0/1903224313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc648102d40 0x7fc6481009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.572+0000 7fc64d9cd640 1 -- 172.21.15.116:0/1903224313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc64810c170 msgr2=0x7fc648101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:55.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.572+0000 7fc64d9cd640 1 --2- 172.21.15.116:0/1903224313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc64810c170 0x7fc648101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.572+0000 7fc64d9cd640 1 -- 172.21.15.116:0/1903224313 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6481a6890 con 0x7fc648103660 2024-09-12T03:09:55.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.572+0000 7fc64d9cd640 1 --2- 172.21.15.116:0/1903224313 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc648103660 0x7fc648100ee0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc63800c970 tx=0x7fc63800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:55.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.572+0000 7fc63f7fe640 1 -- 172.21.15.116:0/1903224313 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc638007bb0 con 0x7fc648103660 2024-09-12T03:09:55.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.572+0000 7fc63f7fe640 1 -- 172.21.15.116:0/1903224313 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc638007d50 con 0x7fc648103660 2024-09-12T03:09:55.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.572+0000 7fc650432640 1 -- 172.21.15.116:0/1903224313 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6481a6b80 con 0x7fc648103660 2024-09-12T03:09:55.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.572+0000 7fc650432640 1 -- 172.21.15.116:0/1903224313 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6481a70c0 con 0x7fc648103660 2024-09-12T03:09:55.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.573+0000 7fc63f7fe640 1 -- 172.21.15.116:0/1903224313 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc638018820 con 0x7fc648103660 2024-09-12T03:09:55.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.574+0000 7fc63f7fe640 1 -- 172.21.15.116:0/1903224313 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fc638018a80 con 0x7fc648103660 2024-09-12T03:09:55.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.575+0000 7fc65280b740 1 Processor -- start 2024-09-12T03:09:55.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.575+0000 7fc65280b740 1 -- start start 2024-09-12T03:09:55.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.575+0000 7fc65280b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cdabf6c4a0 0x55cdabf6c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:55.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.575+0000 7fc65280b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cdabf6cdc0 0x55cdabf77140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:55.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.575+0000 7fc65280b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cdabf77680 0x55cdabf79a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:55.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.575+0000 7fc65280b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55cdabed22e0 con 0x55cdabf6c4a0 2024-09-12T03:09:55.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.575+0000 7fc65280b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55cdabe79ce0 con 0x55cdabf6cdc0 2024-09-12T03:09:55.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.575+0000 7fc65280b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cdabece2a0 con 0x55cdabf77680 2024-09-12T03:09:55.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.575+0000 7fc63f7fe640 1 --2- 172.21.15.116:0/1903224313 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc6340a2b80 0x7fc6340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:55.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.575+0000 7fc64e9cf640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cdabf6c4a0 0x55cdabf6c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:55.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.576+0000 7fc64e9cf640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cdabf6c4a0 0x55cdabf6c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40424/0 (socket says 172.21.15.116:40424) 2024-09-12T03:09:55.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.576+0000 7fc64e9cf640 1 -- 172.21.15.116:0/2460673281 learned_addr learned my addr 172.21.15.116:0/2460673281 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:55.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.576+0000 7fc63f7fe640 1 -- 172.21.15.116:0/1903224313 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fc6380cf000 con 0x7fc648103660 2024-09-12T03:09:55.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.576+0000 7fc64e9cf640 1 --2- 172.21.15.116:0/1903224313 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc6340a2b80 0x7fc6340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:55.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.576+0000 7fc64e1ce640 1 --2- 172.21.15.116:0/2460673281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cdabf6cdc0 0x55cdabf77140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:55.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.576+0000 7fc64d9cd640 1 --2- 172.21.15.116:0/2460673281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cdabf77680 0x55cdabf79a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:55.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.576+0000 7fc64e9cf640 1 -- 172.21.15.116:0/2460673281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cdabf77680 msgr2=0x55cdabf79a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:55.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.576+0000 7fc64e9cf640 1 --2- 172.21.15.116:0/2460673281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cdabf77680 0x55cdabf79a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.576+0000 7fc64e9cf640 1 -- 172.21.15.116:0/2460673281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cdabf6cdc0 msgr2=0x55cdabf77140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:55.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.576+0000 7fc64e9cf640 1 --2- 172.21.15.116:0/2460673281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cdabf6cdc0 0x55cdabf77140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.576+0000 7fc64e9cf640 1 -- 172.21.15.116:0/2460673281 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdabe5e440 con 0x55cdabf6c4a0 2024-09-12T03:09:55.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.576+0000 7fc64e9cf640 1 --2- 172.21.15.116:0/2460673281 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cdabf6c4a0 0x55cdabf6c880 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc640014a60 tx=0x7fc640014e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1e6aaf67650fe3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:55.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.576+0000 7fc63cdea640 1 -- 172.21.15.116:0/2460673281 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc640004d20 con 0x55cdabf6c4a0 2024-09-12T03:09:55.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.576+0000 7fc63cdea640 1 -- 172.21.15.116:0/2460673281 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc640023070 con 0x55cdabf6c4a0 2024-09-12T03:09:55.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.576+0000 7fc63cdea640 1 -- 172.21.15.116:0/2460673281 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6400055e0 con 0x55cdabf6c4a0 2024-09-12T03:09:55.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.576+0000 7fc64e9cf640 1 --2- 172.21.15.116:0/1903224313 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc6340a2b80 0x7fc6340a5040 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc640022000 tx=0x7fc64001e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:55.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.577+0000 7fc65280b740 1 -- 172.21.15.116:0/2460673281 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cdabf6c4a0 msgr2=0x55cdabf6c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:55.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.577+0000 7fc65280b740 1 --2- 172.21.15.116:0/2460673281 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cdabf6c4a0 0x55cdabf6c880 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc640014a60 tx=0x7fc640014e60 comp rx=0 tx=0).stop 2024-09-12T03:09:55.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.577+0000 7fc65280b740 1 -- 172.21.15.116:0/2460673281 shutdown_connections 2024-09-12T03:09:55.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.577+0000 7fc65280b740 1 --2- 172.21.15.116:0/2460673281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cdabf77680 0x55cdabf79a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.577+0000 7fc65280b740 1 --2- 172.21.15.116:0/2460673281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cdabf6cdc0 0x55cdabf77140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.577+0000 7fc65280b740 1 --2- 172.21.15.116:0/2460673281 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cdabf6c4a0 0x55cdabf6c880 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.577+0000 7fc65280b740 1 -- 172.21.15.116:0/2460673281 >> 172.21.15.116:0/2460673281 conn(0x55cdabf68a30 msgr2=0x55cdabf68e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:55.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.577+0000 7fc65280b740 1 -- 172.21.15.116:0/2460673281 shutdown_connections 2024-09-12T03:09:55.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.577+0000 7fc65280b740 1 -- 172.21.15.116:0/2460673281 wait complete. 2024-09-12T03:09:55.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.578+0000 7fc65280b740 1 -- start start 2024-09-12T03:09:55.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.578+0000 7fc65280b740 1 Processor -- start 2024-09-12T03:09:55.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.578+0000 7fc65280b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cdabf6c4a0 0x55cdabf76f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:55.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.578+0000 7fc65280b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cdabf6cdc0 0x55cdabf74240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:55.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.578+0000 7fc65280b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cdabf77680 0x55cdabf74780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:55.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.578+0000 7fc65280b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55cdabf7a250 con 0x55cdabf6c4a0 2024-09-12T03:09:55.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.578+0000 7fc65280b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55cdabe8d050 con 0x55cdabf6cdc0 2024-09-12T03:09:55.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.578+0000 7fc65280b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cdabf7a3d0 con 0x55cdabf77680 2024-09-12T03:09:55.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.578+0000 7fc65280b740 10 client.0 _ll_register_callbacks cb 0x55cdabecd160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:55.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.578+0000 7fc64e1ce640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cdabf6c4a0 0x55cdabf76f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:55.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.578+0000 7fc64e1ce640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cdabf6c4a0 0x55cdabf76f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40432/0 (socket says 172.21.15.116:40432) 2024-09-12T03:09:55.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.578+0000 7fc64e1ce640 1 -- 172.21.15.116:0/2204980184 learned_addr learned my addr 172.21.15.116:0/2204980184 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:55.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.578+0000 7fc64d9cd640 1 --2- 172.21.15.116:0/2204980184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cdabf6cdc0 0x55cdabf74240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:55.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.578+0000 7fc64e9cf640 1 --2- 172.21.15.116:0/2204980184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cdabf77680 0x55cdabf74780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:55.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.578+0000 7fc64e1ce640 1 -- 172.21.15.116:0/2204980184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cdabf77680 msgr2=0x55cdabf74780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:55.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.578+0000 7fc64e1ce640 1 --2- 172.21.15.116:0/2204980184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cdabf77680 0x55cdabf74780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.578+0000 7fc64e1ce640 1 -- 172.21.15.116:0/2204980184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cdabf6cdc0 msgr2=0x55cdabf74240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:55.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.578+0000 7fc64e1ce640 1 --2- 172.21.15.116:0/2204980184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cdabf6cdc0 0x55cdabf74240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.578+0000 7fc64e1ce640 1 -- 172.21.15.116:0/2204980184 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdabc433c0 con 0x55cdabf6c4a0 2024-09-12T03:09:55.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.578+0000 7fc64d9cd640 1 --2- 172.21.15.116:0/2204980184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cdabf6cdc0 0x55cdabf74240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:09:55.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.578+0000 7fc64e9cf640 1 --2- 172.21.15.116:0/2204980184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cdabf77680 0x55cdabf74780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:09:55.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.579+0000 7fc64e1ce640 1 --2- 172.21.15.116:0/2204980184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cdabf6c4a0 0x55cdabf76f90 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fc644009900 tx=0x7fc6440319b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2dd0a315057f97 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:55.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.579+0000 7fc631ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:09:55.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.579+0000 7fc631ffb640 1 -- 172.21.15.116:0/2204980184 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc644035040 con 0x55cdabf6c4a0 2024-09-12T03:09:55.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.579+0000 7fc631ffb640 1 -- 172.21.15.116:0/2204980184 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fc644004300 con 0x55cdabf6c4a0 2024-09-12T03:09:55.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.579+0000 7fc65280b740 1 -- 172.21.15.116:0/2204980184 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc640023070 con 0x55cdabf6c4a0 2024-09-12T03:09:55.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.579+0000 7fc631ffb640 1 -- 172.21.15.116:0/2204980184 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc64404d070 con 0x55cdabf6c4a0 2024-09-12T03:09:55.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.579+0000 7fc65280b740 10 client.17046 fetch_fsmap learned FSMap version 25 2024-09-12T03:09:55.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.579+0000 7fc65280b740 1 -- 172.21.15.116:0/2204980184 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cdabf85bd0 con 0x55cdabf6c4a0 2024-09-12T03:09:55.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.579+0000 7fc631ffb640 1 -- 172.21.15.116:0/2204980184 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc640023070 con 0x55cdabf6c4a0 2024-09-12T03:09:55.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.579+0000 7fc631ffb640 1 -- 172.21.15.116:0/2204980184 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc644046750 con 0x55cdabf6c4a0 2024-09-12T03:09:55.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.579+0000 7fc65280b740 10 client.17046 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:09:55.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.579+0000 7fc65280b740 10 client.17046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:09:55.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.579+0000 7fc65280b740 20 client.17046 populate_metadata read hostname 'smithi116' 2024-09-12T03:09:55.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.579+0000 7fc65280b740 1 --2- 172.21.15.116:0/2204980184 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55cdabf87090 0x55cdabf894a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:55.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.579+0000 7fc65280b740 4 client.17046 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:09:55.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.579+0000 7fc65280b740 1 -- 172.21.15.116:0/2204980184 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdabedf850 con 0x55cdabf87090 2024-09-12T03:09:55.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.580+0000 7fc64d9cd640 1 --2- 172.21.15.116:0/2204980184 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55cdabf87090 0x55cdabf894a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:55.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.580+0000 7fc631ffb640 10 client.17046 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:55.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.580+0000 7fc64d9cd640 1 --2- 172.21.15.116:0/2204980184 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55cdabf87090 0x55cdabf894a0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efc1114599a956a7 server_cookie=9477fc06d0a8d2dc in_seq=0 out_seq=0 2024-09-12T03:09:55.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.581+0000 7fc631ffb640 1 -- 172.21.15.116:0/2204980184 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cdabedf850 con 0x55cdabf87090 2024-09-12T03:09:55.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.581+0000 7fc631ffb640 10 client.17046 handle_command_reply: tid=0 2024-09-12T03:09:55.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.581+0000 7fc65280b740 1 client.17046 shutdown 2024-09-12T03:09:55.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.581+0000 7fc65280b740 1 -- 172.21.15.116:0/2204980184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cdabf6c4a0 msgr2=0x55cdabf76f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:55.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.581+0000 7fc65280b740 1 --2- 172.21.15.116:0/2204980184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cdabf6c4a0 0x55cdabf76f90 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fc644009900 tx=0x7fc6440319b0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.581+0000 7fc65280b740 1 -- 172.21.15.116:0/2204980184 shutdown_connections 2024-09-12T03:09:55.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.581+0000 7fc65280b740 1 --2- 172.21.15.116:0/2204980184 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55cdabf87090 0x55cdabf894a0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.581+0000 7fc65280b740 1 --2- 172.21.15.116:0/2204980184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cdabf77680 0x55cdabf74780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.581+0000 7fc65280b740 1 --2- 172.21.15.116:0/2204980184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cdabf6cdc0 0x55cdabf74240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.581+0000 7fc631ffb640 0 client.17046 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:55.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.581+0000 7fc65280b740 1 --2- 172.21.15.116:0/2204980184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cdabf6c4a0 0x55cdabf76f90 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.581+0000 7fc65280b740 1 -- 172.21.15.116:0/2204980184 >> 172.21.15.116:0/2204980184 conn(0x55cdabf68a30 msgr2=0x55cdabf67b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:55.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.581+0000 7fc65280b740 1 -- 172.21.15.116:0/2204980184 shutdown_connections 2024-09-12T03:09:55.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.581+0000 7fc65280b740 1 -- 172.21.15.116:0/2204980184 wait complete. 2024-09-12T03:09:55.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.581+0000 7fc65280b740 20 client.17046 trim_cache size 0 max 16384 2024-09-12T03:09:55.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.598+0000 7fc65280b740 1 Processor -- start 2024-09-12T03:09:55.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.598+0000 7fc65280b740 1 -- start start 2024-09-12T03:09:55.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.598+0000 7fc65280b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cdabf6c4a0 0x55cdabf67980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:55.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.598+0000 7fc65280b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cdabf6cdc0 0x55cdabf67ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:55.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.598+0000 7fc65280b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cdabf77680 0x55cdabf8ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:55.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.598+0000 7fc65280b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc6481b1e80 con 0x55cdabf6c4a0 2024-09-12T03:09:55.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.598+0000 7fc65280b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc6481af750 con 0x55cdabf6cdc0 2024-09-12T03:09:55.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.598+0000 7fc65280b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc6481ad5c0 con 0x55cdabf77680 2024-09-12T03:09:55.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.598+0000 7fc64e1ce640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cdabf6c4a0 0x55cdabf67980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:55.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.598+0000 7fc64e1ce640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cdabf6c4a0 0x55cdabf67980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40444/0 (socket says 172.21.15.116:40444) 2024-09-12T03:09:55.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.598+0000 7fc64e1ce640 1 -- 172.21.15.116:0/2389552510 learned_addr learned my addr 172.21.15.116:0/2389552510 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:55.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.598+0000 7fc64e1ce640 1 -- 172.21.15.116:0/2389552510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cdabf77680 msgr2=0x55cdabf8ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:55.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.598+0000 7fc64e1ce640 1 --2- 172.21.15.116:0/2389552510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cdabf77680 0x55cdabf8ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.598+0000 7fc64e1ce640 1 -- 172.21.15.116:0/2389552510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cdabf6cdc0 msgr2=0x55cdabf67ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:55.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.598+0000 7fc64e1ce640 1 --2- 172.21.15.116:0/2389552510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cdabf6cdc0 0x55cdabf67ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.598+0000 7fc64e1ce640 1 -- 172.21.15.116:0/2389552510 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdabf6c2b0 con 0x55cdabf6c4a0 2024-09-12T03:09:55.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.599+0000 7fc64e1ce640 1 --2- 172.21.15.116:0/2389552510 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cdabf6c4a0 0x55cdabf67980 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fc6440096c0 tx=0x7fc644031bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5cc55377cafc665 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:55.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.599+0000 7fc633fff640 1 -- 172.21.15.116:0/2389552510 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc644035070 con 0x55cdabf6c4a0 2024-09-12T03:09:55.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.599+0000 7fc633fff640 1 -- 172.21.15.116:0/2389552510 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc644004300 con 0x55cdabf6c4a0 2024-09-12T03:09:55.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.599+0000 7fc633fff640 1 -- 172.21.15.116:0/2389552510 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6440046c0 con 0x55cdabf6c4a0 2024-09-12T03:09:55.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.599+0000 7fc65280b740 1 -- 172.21.15.116:0/2389552510 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cdabf6c4a0 msgr2=0x55cdabf67980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:55.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.599+0000 7fc65280b740 1 --2- 172.21.15.116:0/2389552510 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cdabf6c4a0 0x55cdabf67980 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fc6440096c0 tx=0x7fc644031bb0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.599+0000 7fc65280b740 1 -- 172.21.15.116:0/2389552510 shutdown_connections 2024-09-12T03:09:55.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.599+0000 7fc65280b740 1 --2- 172.21.15.116:0/2389552510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cdabf77680 0x55cdabf8ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.599+0000 7fc65280b740 1 --2- 172.21.15.116:0/2389552510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cdabf6cdc0 0x55cdabf67ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.599+0000 7fc65280b740 1 --2- 172.21.15.116:0/2389552510 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cdabf6c4a0 0x55cdabf67980 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.599+0000 7fc65280b740 1 -- 172.21.15.116:0/2389552510 >> 172.21.15.116:0/2389552510 conn(0x55cdabf68a30 msgr2=0x55cdabf67270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:55.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.599+0000 7fc65280b740 1 -- 172.21.15.116:0/2389552510 shutdown_connections 2024-09-12T03:09:55.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.600+0000 7fc65280b740 1 -- 172.21.15.116:0/2389552510 wait complete. 2024-09-12T03:09:55.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.600+0000 7fc65280b740 1 -- start start 2024-09-12T03:09:55.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.600+0000 7fc65280b740 1 Processor -- start 2024-09-12T03:09:55.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.600+0000 7fc65280b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cdabf6c4a0 0x55cdabf7fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:55.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.600+0000 7fc65280b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cdabf6cdc0 0x55cdabf7d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:55.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.600+0000 7fc65280b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cdabf77680 0x55cdabf7d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:55.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.600+0000 7fc65280b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc6481adc10 con 0x55cdabf77680 2024-09-12T03:09:55.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.600+0000 7fc65280b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc6481af080 con 0x55cdabf6c4a0 2024-09-12T03:09:55.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.600+0000 7fc65280b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc6481ae2e0 con 0x55cdabf6cdc0 2024-09-12T03:09:55.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.600+0000 7fc65280b740 10 client.0 _ll_register_callbacks cb 0x7fc644031640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:55.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.600+0000 7fc64e9cf640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cdabf77680 0x55cdabf7d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:55.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.600+0000 7fc64e9cf640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cdabf77680 0x55cdabf7d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40448/0 (socket says 172.21.15.116:40448) 2024-09-12T03:09:55.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.600+0000 7fc64e9cf640 1 -- 172.21.15.116:0/4195505292 learned_addr learned my addr 172.21.15.116:0/4195505292 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:55.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.600+0000 7fc64e1ce640 1 --2- 172.21.15.116:0/4195505292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cdabf6c4a0 0x55cdabf7fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:55.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.600+0000 7fc64d9cd640 1 --2- 172.21.15.116:0/4195505292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cdabf6cdc0 0x55cdabf7d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:55.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.601+0000 7fc64e9cf640 1 -- 172.21.15.116:0/4195505292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cdabf6cdc0 msgr2=0x55cdabf7d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:55.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.601+0000 7fc64e9cf640 1 --2- 172.21.15.116:0/4195505292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cdabf6cdc0 0x55cdabf7d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.601+0000 7fc64e9cf640 1 -- 172.21.15.116:0/4195505292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cdabf6c4a0 msgr2=0x55cdabf7fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:55.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.601+0000 7fc64e9cf640 1 --2- 172.21.15.116:0/4195505292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cdabf6c4a0 0x55cdabf7fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.601+0000 7fc64e9cf640 1 -- 172.21.15.116:0/4195505292 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdabf7dee0 con 0x55cdabf77680 2024-09-12T03:09:55.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.601+0000 7fc64e1ce640 1 --2- 172.21.15.116:0/4195505292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cdabf6c4a0 0x55cdabf7fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:09:55.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.601+0000 7fc64d9cd640 1 --2- 172.21.15.116:0/4195505292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cdabf6cdc0 0x55cdabf7d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:09:55.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.601+0000 7fc64e9cf640 1 --2- 172.21.15.116:0/4195505292 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cdabf77680 0x55cdabf7d6b0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fc640009ea0 tx=0x7fc640012810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc8bfbbec25f5b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:55.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.601+0000 7fc631ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:09:55.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.601+0000 7fc631ffb640 1 -- 172.21.15.116:0/4195505292 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc640037060 con 0x55cdabf77680 2024-09-12T03:09:55.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.601+0000 7fc631ffb640 1 -- 172.21.15.116:0/4195505292 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fc640009a00 con 0x55cdabf77680 2024-09-12T03:09:55.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.601+0000 7fc65280b740 1 -- 172.21.15.116:0/4195505292 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc644004300 con 0x55cdabf77680 2024-09-12T03:09:55.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.601+0000 7fc631ffb640 1 -- 172.21.15.116:0/4195505292 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc6400049a0 con 0x55cdabf77680 2024-09-12T03:09:55.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.601+0000 7fc65280b740 10 client.17052 fetch_fsmap learned FSMap version 25 2024-09-12T03:09:55.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.601+0000 7fc65280b740 1 -- 172.21.15.116:0/4195505292 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cdabff1df0 con 0x55cdabf77680 2024-09-12T03:09:55.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.601+0000 7fc631ffb640 1 -- 172.21.15.116:0/4195505292 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc644004300 con 0x55cdabf77680 2024-09-12T03:09:55.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.601+0000 7fc631ffb640 1 -- 172.21.15.116:0/4195505292 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc64000f040 con 0x55cdabf77680 2024-09-12T03:09:55.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.601+0000 7fc65280b740 10 client.17052 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:09:55.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.601+0000 7fc65280b740 10 client.17052 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:09:55.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.601+0000 7fc65280b740 20 client.17052 populate_metadata read hostname 'smithi116' 2024-09-12T03:09:55.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.601+0000 7fc65280b740 1 --2- 172.21.15.116:0/4195505292 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55cdabff21d0 0x55cdabff45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:55.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.601+0000 7fc65280b740 4 client.17052 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:09:55.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.601+0000 7fc65280b740 1 -- 172.21.15.116:0/4195505292 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cdabf74b10 con 0x55cdabff21d0 2024-09-12T03:09:55.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.602+0000 7fc64e1ce640 1 --2- 172.21.15.116:0/4195505292 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55cdabff21d0 0x55cdabff45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:55.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.602+0000 7fc64e1ce640 1 --2- 172.21.15.116:0/4195505292 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55cdabff21d0 0x55cdabff45f0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fb73d134f39fa81 server_cookie=82bc3bf21d187b99 in_seq=0 out_seq=0 2024-09-12T03:09:55.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.602+0000 7fc631ffb640 10 client.17052 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:55.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.602+0000 7fc631ffb640 1 -- 172.21.15.116:0/4195505292 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cdabf74b10 con 0x55cdabff21d0 2024-09-12T03:09:55.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.602+0000 7fc631ffb640 10 client.17052 handle_command_reply: tid=0 2024-09-12T03:09:55.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.602+0000 7fc65280b740 1 client.17052 shutdown 2024-09-12T03:09:55.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.603+0000 7fc65280b740 1 -- 172.21.15.116:0/4195505292 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cdabf77680 msgr2=0x55cdabf7d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:55.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.603+0000 7fc65280b740 1 --2- 172.21.15.116:0/4195505292 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cdabf77680 0x55cdabf7d6b0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fc640009ea0 tx=0x7fc640012810 comp rx=0 tx=0).stop 2024-09-12T03:09:55.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.603+0000 7fc65280b740 1 -- 172.21.15.116:0/4195505292 shutdown_connections 2024-09-12T03:09:55.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.603+0000 7fc65280b740 1 --2- 172.21.15.116:0/4195505292 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55cdabff21d0 0x55cdabff45f0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.603+0000 7fc65280b740 1 --2- 172.21.15.116:0/4195505292 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cdabf77680 0x55cdabf7d6b0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.603+0000 7fc65280b740 1 --2- 172.21.15.116:0/4195505292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cdabf6cdc0 0x55cdabf7d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.603+0000 7fc65280b740 1 --2- 172.21.15.116:0/4195505292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cdabf6c4a0 0x55cdabf7fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.603+0000 7fc65280b740 1 -- 172.21.15.116:0/4195505292 >> 172.21.15.116:0/4195505292 conn(0x55cdabf68a30 msgr2=0x55cdabf67a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:55.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.603+0000 7fc631ffb640 0 client.17052 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:55.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.603+0000 7fc65280b740 1 -- 172.21.15.116:0/4195505292 shutdown_connections 2024-09-12T03:09:55.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.603+0000 7fc65280b740 1 -- 172.21.15.116:0/4195505292 wait complete. 2024-09-12T03:09:55.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.603+0000 7fc65280b740 20 client.17052 trim_cache size 0 max 16384 2024-09-12T03:09:55.609 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:09:55.609 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:09:55.609 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:09:55.609 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:09:55.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.603+0000 7fc650432640 1 -- 172.21.15.116:0/1903224313 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc6340a2b80 msgr2=0x7fc6340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:55.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.603+0000 7fc650432640 1 --2- 172.21.15.116:0/1903224313 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc6340a2b80 0x7fc6340a5040 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc640022000 tx=0x7fc64001e000 comp rx=0 tx=0).stop 2024-09-12T03:09:55.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.603+0000 7fc650432640 1 -- 172.21.15.116:0/1903224313 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc648103660 msgr2=0x7fc648100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:55.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.603+0000 7fc650432640 1 --2- 172.21.15.116:0/1903224313 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc648103660 0x7fc648100ee0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc63800c970 tx=0x7fc63800ce30 comp rx=0 tx=0).stop 2024-09-12T03:09:55.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.603+0000 7fc650432640 1 -- 172.21.15.116:0/1903224313 shutdown_connections 2024-09-12T03:09:55.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.603+0000 7fc650432640 1 --2- 172.21.15.116:0/1903224313 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc6340a2b80 0x7fc6340a5040 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.603+0000 7fc650432640 1 --2- 172.21.15.116:0/1903224313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc64810c170 0x7fc648101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.603+0000 7fc650432640 1 --2- 172.21.15.116:0/1903224313 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc648103660 0x7fc648100ee0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.603+0000 7fc650432640 1 --2- 172.21.15.116:0/1903224313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc648102d40 0x7fc6481009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:55.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.603+0000 7fc650432640 1 -- 172.21.15.116:0/1903224313 >> 172.21.15.116:0/1903224313 conn(0x7fc6480fc7a0 msgr2=0x7fc6480fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:55.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.603+0000 7fc650432640 1 -- 172.21.15.116:0/1903224313 shutdown_connections 2024-09-12T03:09:55.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:55.603+0000 7fc650432640 1 -- 172.21.15.116:0/1903224313 wait complete. 2024-09-12T03:09:55.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:55 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/702202953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:09:55.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:55 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/702202953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:09:56.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:56 smithi116 ceph-mon[31410]: pgmap v546: 97 pgs: 97 active+clean; 3.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:09:56.431 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:09:56.431 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:09:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:56 smithi183 ceph-mon[35535]: pgmap v546: 97 pgs: 97 active+clean; 3.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:09:56.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:56 smithi177 ceph-mon[35305]: pgmap v546: 97 pgs: 97 active+clean; 3.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:09:56.965 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0' 2024-09-12T03:09:56.965 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0 2024-09-12T03:09:57.013 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:09:57.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.523+0000 7f0561b18640 1 -- 172.21.15.116:0/1746790203 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f055c1027a0 msgr2=0x7f055c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:57.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.523+0000 7f0561b18640 1 --2- 172.21.15.116:0/1746790203 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f055c1027a0 0x7f055c1067f0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f054c009900 tx=0x7f054c02a410 comp rx=0 tx=0).stop 2024-09-12T03:09:57.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.523+0000 7f0561b18640 1 -- 172.21.15.116:0/1746790203 shutdown_connections 2024-09-12T03:09:57.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.523+0000 7f0561b18640 1 --2- 172.21.15.116:0/1746790203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f055c106f20 0x7f055c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:57.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.523+0000 7f0561b18640 1 --2- 172.21.15.116:0/1746790203 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f055c1027a0 0x7f055c1067f0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:57.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.523+0000 7f0561b18640 1 --2- 172.21.15.116:0/1746790203 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f055c101df0 0x7f055c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:57.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.523+0000 7f0561b18640 1 -- 172.21.15.116:0/1746790203 >> 172.21.15.116:0/1746790203 conn(0x7f055c0fd8b0 msgr2=0x7f055c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:57.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.523+0000 7f0561b18640 1 -- 172.21.15.116:0/1746790203 shutdown_connections 2024-09-12T03:09:57.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.523+0000 7f0561b18640 1 -- 172.21.15.116:0/1746790203 wait complete. 2024-09-12T03:09:57.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.524+0000 7f0561b18640 1 Processor -- start 2024-09-12T03:09:57.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.524+0000 7f0561b18640 1 -- start start 2024-09-12T03:09:57.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.524+0000 7f0561b18640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f055c101df0 0x7f055c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:57.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.524+0000 7f0561b18640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f055c1027a0 0x7f055c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:57.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.524+0000 7f0561b18640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f055c106f20 0x7f055c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:57.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.524+0000 7f0561b18640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f055c112020 con 0x7f055c1027a0 2024-09-12T03:09:57.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.524+0000 7f0561b18640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f055c111ea0 con 0x7f055c101df0 2024-09-12T03:09:57.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.524+0000 7f0561b18640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f055c1121a0 con 0x7f055c106f20 2024-09-12T03:09:57.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.524+0000 7f055affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f055c1027a0 0x7f055c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:57.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.525+0000 7f055affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f055c1027a0 0x7f055c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40466/0 (socket says 172.21.15.116:40466) 2024-09-12T03:09:57.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.525+0000 7f055affd640 1 -- 172.21.15.116:0/1885762370 learned_addr learned my addr 172.21.15.116:0/1885762370 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:57.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.525+0000 7f055bfff640 1 --2- 172.21.15.116:0/1885762370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f055c106f20 0x7f055c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:57.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.525+0000 7f055b7fe640 1 --2- 172.21.15.116:0/1885762370 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f055c101df0 0x7f055c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:57.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.525+0000 7f055affd640 1 -- 172.21.15.116:0/1885762370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f055c106f20 msgr2=0x7f055c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:57.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.525+0000 7f055affd640 1 --2- 172.21.15.116:0/1885762370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f055c106f20 0x7f055c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:57.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.525+0000 7f055affd640 1 -- 172.21.15.116:0/1885762370 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f055c101df0 msgr2=0x7f055c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:57.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.525+0000 7f055affd640 1 --2- 172.21.15.116:0/1885762370 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f055c101df0 0x7f055c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:57.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.525+0000 7f055affd640 1 -- 172.21.15.116:0/1885762370 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055c1a6a40 con 0x7f055c1027a0 2024-09-12T03:09:57.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.525+0000 7f055b7fe640 1 --2- 172.21.15.116:0/1885762370 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f055c101df0 0x7f055c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:09:57.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.525+0000 7f055affd640 1 --2- 172.21.15.116:0/1885762370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f055c1027a0 0x7f055c074a30 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f054c005ab0 tx=0x7f054c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:57.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.525+0000 7f0558ff9640 1 -- 172.21.15.116:0/1885762370 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c03f030 con 0x7f055c1027a0 2024-09-12T03:09:57.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.525+0000 7f0561b18640 1 -- 172.21.15.116:0/1885762370 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f055c1a6cd0 con 0x7f055c1027a0 2024-09-12T03:09:57.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.525+0000 7f0558ff9640 1 -- 172.21.15.116:0/1885762370 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f054c003ce0 con 0x7f055c1027a0 2024-09-12T03:09:57.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.525+0000 7f0558ff9640 1 -- 172.21.15.116:0/1885762370 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c035070 con 0x7f055c1027a0 2024-09-12T03:09:57.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.525+0000 7f0561b18640 1 -- 172.21.15.116:0/1885762370 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f055c1a71b0 con 0x7f055c1027a0 2024-09-12T03:09:57.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.526+0000 7f0561b18640 1 -- 172.21.15.116:0/1885762370 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f055c102250 con 0x7f055c1027a0 2024-09-12T03:09:57.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.527+0000 7f0558ff9640 1 -- 172.21.15.116:0/1885762370 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f054c042030 con 0x7f055c1027a0 2024-09-12T03:09:57.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.528+0000 7f0558ff9640 1 --2- 172.21.15.116:0/1885762370 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f05480a2b80 0x7f05480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:57.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.528+0000 7f055b7fe640 1 --2- 172.21.15.116:0/1885762370 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f05480a2b80 0x7f05480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:57.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.528+0000 7f0558ff9640 1 -- 172.21.15.116:0/1885762370 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f054c0efb50 con 0x7f055c1027a0 2024-09-12T03:09:57.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.529+0000 7f055b7fe640 1 --2- 172.21.15.116:0/1885762370 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f05480a2b80 0x7f05480a5040 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0544009930 tx=0x7f0544008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:57.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.531+0000 7f0558ff9640 1 -- 172.21.15.116:0/1885762370 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f054c0bcae0 con 0x7f055c1027a0 2024-09-12T03:09:57.587 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:09:57.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.740+0000 7f0561b18640 1 -- 172.21.15.116:0/1885762370 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f055c10a100 con 0x7f055c1027a0 2024-09-12T03:09:57.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.740+0000 7f0558ff9640 1 -- 172.21.15.116:0/1885762370 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f054c0bc3e0 con 0x7f055c1027a0 2024-09-12T03:09:57.741 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:09:57.741 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:09:57.741 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:09:57.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.743+0000 7f0561b18640 1 -- 172.21.15.116:0/1885762370 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f05480a2b80 msgr2=0x7f05480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:57.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.743+0000 7f0561b18640 1 --2- 172.21.15.116:0/1885762370 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f05480a2b80 0x7f05480a5040 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0544009930 tx=0x7f0544008040 comp rx=0 tx=0).stop 2024-09-12T03:09:57.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.743+0000 7f0561b18640 1 -- 172.21.15.116:0/1885762370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f055c1027a0 msgr2=0x7f055c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:57.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.743+0000 7f0561b18640 1 --2- 172.21.15.116:0/1885762370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f055c1027a0 0x7f055c074a30 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f054c005ab0 tx=0x7f054c0042f0 comp rx=0 tx=0).stop 2024-09-12T03:09:57.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.743+0000 7f0561b18640 1 -- 172.21.15.116:0/1885762370 shutdown_connections 2024-09-12T03:09:57.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.743+0000 7f0561b18640 1 --2- 172.21.15.116:0/1885762370 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f05480a2b80 0x7f05480a5040 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:57.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.743+0000 7f0561b18640 1 --2- 172.21.15.116:0/1885762370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f055c106f20 0x7f055c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:57.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.743+0000 7f0561b18640 1 --2- 172.21.15.116:0/1885762370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f055c1027a0 0x7f055c074a30 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:57.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.743+0000 7f0561b18640 1 --2- 172.21.15.116:0/1885762370 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f055c101df0 0x7f055c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:57.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.743+0000 7f0561b18640 1 -- 172.21.15.116:0/1885762370 >> 172.21.15.116:0/1885762370 conn(0x7f055c0fd8b0 msgr2=0x7f055c0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:57.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.743+0000 7f0561b18640 1 -- 172.21.15.116:0/1885762370 shutdown_connections 2024-09-12T03:09:57.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:57.743+0000 7f0561b18640 1 -- 172.21.15.116:0/1885762370 wait complete. 2024-09-12T03:09:58.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.272+0000 7fb93af55640 1 -- 172.21.15.116:0/1245387497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb934106f20 msgr2=0x7fb93410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:58.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.272+0000 7fb93af55640 1 --2- 172.21.15.116:0/1245387497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb934106f20 0x7fb93410eaa0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fb92c009ca0 tx=0x7fb92c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.272+0000 7fb93af55640 1 -- 172.21.15.116:0/1245387497 shutdown_connections 2024-09-12T03:09:58.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.272+0000 7fb93af55640 1 --2- 172.21.15.116:0/1245387497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb934106f20 0x7fb93410eaa0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.272+0000 7fb93af55640 1 --2- 172.21.15.116:0/1245387497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9341027a0 0x7fb9341067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.272+0000 7fb93af55640 1 --2- 172.21.15.116:0/1245387497 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb934101df0 0x7fb9341021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.272+0000 7fb93af55640 1 -- 172.21.15.116:0/1245387497 >> 172.21.15.116:0/1245387497 conn(0x7fb9340fd8b0 msgr2=0x7fb9340ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:58.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.272+0000 7fb93af55640 1 -- 172.21.15.116:0/1245387497 shutdown_connections 2024-09-12T03:09:58.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.272+0000 7fb93af55640 1 -- 172.21.15.116:0/1245387497 wait complete. 2024-09-12T03:09:58.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.273+0000 7fb93af55640 1 Processor -- start 2024-09-12T03:09:58.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.273+0000 7fb93af55640 1 -- start start 2024-09-12T03:09:58.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.273+0000 7fb93af55640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb934101df0 0x7fb934198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:58.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.273+0000 7fb93af55640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9341027a0 0x7fb9341988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:58.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.273+0000 7fb93af55640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb934106f20 0x7fb93419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:58.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.273+0000 7fb93af55640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb934111f80 con 0x7fb934106f20 2024-09-12T03:09:58.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.273+0000 7fb93af55640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb934111e00 con 0x7fb9341027a0 2024-09-12T03:09:58.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.273+0000 7fb93af55640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb934112100 con 0x7fb934101df0 2024-09-12T03:09:58.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.273+0000 7fb9394f2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb934106f20 0x7fb93419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:58.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.273+0000 7fb9394f2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb934106f20 0x7fb93419cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40476/0 (socket says 172.21.15.116:40476) 2024-09-12T03:09:58.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.273+0000 7fb9394f2640 1 -- 172.21.15.116:0/2898837638 learned_addr learned my addr 172.21.15.116:0/2898837638 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:58.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.273+0000 7fb938cf1640 1 --2- 172.21.15.116:0/2898837638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb934101df0 0x7fb934198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:58.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.273+0000 7fb933fff640 1 --2- 172.21.15.116:0/2898837638 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9341027a0 0x7fb9341988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:58.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.273+0000 7fb9394f2640 1 -- 172.21.15.116:0/2898837638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb934101df0 msgr2=0x7fb934198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:58.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.273+0000 7fb9394f2640 1 --2- 172.21.15.116:0/2898837638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb934101df0 0x7fb934198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.273+0000 7fb9394f2640 1 -- 172.21.15.116:0/2898837638 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9341027a0 msgr2=0x7fb9341988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:58.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.273+0000 7fb9394f2640 1 --2- 172.21.15.116:0/2898837638 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9341027a0 0x7fb9341988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.273+0000 7fb9394f2640 1 -- 172.21.15.116:0/2898837638 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93419d310 con 0x7fb934106f20 2024-09-12T03:09:58.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.273+0000 7fb938cf1640 1 --2- 172.21.15.116:0/2898837638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb934101df0 0x7fb934198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:09:58.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.273+0000 7fb9394f2640 1 --2- 172.21.15.116:0/2898837638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb934106f20 0x7fb93419cc30 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fb92c031a20 tx=0x7fb92c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:58.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.274+0000 7fb931ffb640 1 -- 172.21.15.116:0/2898837638 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c003c30 con 0x7fb934106f20 2024-09-12T03:09:58.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.274+0000 7fb93af55640 1 -- 172.21.15.116:0/2898837638 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb93419d5a0 con 0x7fb934106f20 2024-09-12T03:09:58.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.274+0000 7fb931ffb640 1 -- 172.21.15.116:0/2898837638 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb92c003dd0 con 0x7fb934106f20 2024-09-12T03:09:58.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.274+0000 7fb931ffb640 1 -- 172.21.15.116:0/2898837638 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c02fc40 con 0x7fb934106f20 2024-09-12T03:09:58.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.274+0000 7fb93af55640 1 -- 172.21.15.116:0/2898837638 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9341a4ee0 con 0x7fb934106f20 2024-09-12T03:09:58.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.276+0000 7fb931ffb640 1 -- 172.21.15.116:0/2898837638 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb92c004050 con 0x7fb934106f20 2024-09-12T03:09:58.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.276+0000 7fb93d32e740 1 Processor -- start 2024-09-12T03:09:58.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.276+0000 7fb93d32e740 1 -- start start 2024-09-12T03:09:58.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.277+0000 7fb93d32e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c187487760 0x55c187487b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:58.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.277+0000 7fb93d32e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c187488080 0x55c187492400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:58.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.277+0000 7fb93d32e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c187492940 0x55c187494d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:58.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.277+0000 7fb93d32e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c1873ed2a0 con 0x55c187487760 2024-09-12T03:09:58.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.277+0000 7fb93d32e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c187394b10 con 0x55c187488080 2024-09-12T03:09:58.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.277+0000 7fb93d32e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c1873e9260 con 0x55c187492940 2024-09-12T03:09:58.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.277+0000 7fb933fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c187487760 0x55c187487b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:58.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.277+0000 7fb933fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c187487760 0x55c187487b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40486/0 (socket says 172.21.15.116:40486) 2024-09-12T03:09:58.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.277+0000 7fb931ffb640 1 --2- 172.21.15.116:0/2898837638 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb9240a2b80 0x7fb9240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:58.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.277+0000 7fb933fff640 1 -- 172.21.15.116:0/2691632409 learned_addr learned my addr 172.21.15.116:0/2691632409 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:58.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.277+0000 7fb938cf1640 1 --2- 172.21.15.116:0/2691632409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c187488080 0x55c187492400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:58.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.277+0000 7fb933fff640 1 --2- 172.21.15.116:0/2691632409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c187492940 0x55c187494d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:58.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.277+0000 7fb9394f2640 1 --2- 172.21.15.116:0/2898837638 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb9240a2b80 0x7fb9240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:58.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.277+0000 7fb933fff640 1 -- 172.21.15.116:0/2691632409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c187492940 msgr2=0x55c187494d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:58.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.277+0000 7fb933fff640 1 --2- 172.21.15.116:0/2691632409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c187492940 0x55c187494d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.277+0000 7fb933fff640 1 -- 172.21.15.116:0/2691632409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c187488080 msgr2=0x55c187492400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:58.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.277+0000 7fb933fff640 1 --2- 172.21.15.116:0/2691632409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c187488080 0x55c187492400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.277+0000 7fb933fff640 1 -- 172.21.15.116:0/2691632409 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c18715eb90 con 0x55c187487760 2024-09-12T03:09:58.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.277+0000 7fb931ffb640 1 -- 172.21.15.116:0/2898837638 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fb92c0f02e0 con 0x7fb934106f20 2024-09-12T03:09:58.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.277+0000 7fb933fff640 1 --2- 172.21.15.116:0/2691632409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c187487760 0x55c187487b40 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fb920013450 tx=0x7fb920009720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e59e18be49fb8f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:58.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.277+0000 7fb91effd640 1 -- 172.21.15.116:0/2691632409 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb920010490 con 0x55c187487760 2024-09-12T03:09:58.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.277+0000 7fb9394f2640 1 --2- 172.21.15.116:0/2898837638 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb9240a2b80 0x7fb9240a5040 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0319f0 tx=0x7fb92c02a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:58.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.277+0000 7fb91effd640 1 -- 172.21.15.116:0/2691632409 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb920010b10 con 0x55c187487760 2024-09-12T03:09:58.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.277+0000 7fb91effd640 1 -- 172.21.15.116:0/2691632409 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb920011b50 con 0x55c187487760 2024-09-12T03:09:58.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.278+0000 7fb93d32e740 1 -- 172.21.15.116:0/2691632409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c187487760 msgr2=0x55c187487b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:58.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.278+0000 7fb93d32e740 1 --2- 172.21.15.116:0/2691632409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c187487760 0x55c187487b40 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fb920013450 tx=0x7fb920009720 comp rx=0 tx=0).stop 2024-09-12T03:09:58.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.278+0000 7fb93d32e740 1 -- 172.21.15.116:0/2691632409 shutdown_connections 2024-09-12T03:09:58.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.278+0000 7fb93d32e740 1 --2- 172.21.15.116:0/2691632409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c187492940 0x55c187494d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.278+0000 7fb93d32e740 1 --2- 172.21.15.116:0/2691632409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c187488080 0x55c187492400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.278+0000 7fb93d32e740 1 --2- 172.21.15.116:0/2691632409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c187487760 0x55c187487b40 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.278+0000 7fb93d32e740 1 -- 172.21.15.116:0/2691632409 >> 172.21.15.116:0/2691632409 conn(0x55c187483cf0 msgr2=0x55c1874840d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:58.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.278+0000 7fb93d32e740 1 -- 172.21.15.116:0/2691632409 shutdown_connections 2024-09-12T03:09:58.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.278+0000 7fb93d32e740 1 -- 172.21.15.116:0/2691632409 wait complete. 2024-09-12T03:09:58.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.279+0000 7fb93d32e740 1 -- start start 2024-09-12T03:09:58.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.279+0000 7fb93d32e740 1 Processor -- start 2024-09-12T03:09:58.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.279+0000 7fb93d32e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c187487760 0x55c18748f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:58.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.279+0000 7fb93d32e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c187488080 0x55c18748fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:58.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.279+0000 7fb93d32e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c187492940 0x55c187490260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:58.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.279+0000 7fb93d32e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c187495510 con 0x55c187488080 2024-09-12T03:09:58.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.279+0000 7fb93d32e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c1873756e0 con 0x55c187487760 2024-09-12T03:09:58.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.279+0000 7fb93d32e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c187495690 con 0x55c187492940 2024-09-12T03:09:58.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.279+0000 7fb93d32e740 10 client.0 _ll_register_callbacks cb 0x55c187399a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:58.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.279+0000 7fb938cf1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c187488080 0x55c18748fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:58.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.279+0000 7fb938cf1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c187488080 0x55c18748fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40490/0 (socket says 172.21.15.116:40490) 2024-09-12T03:09:58.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.279+0000 7fb938cf1640 1 -- 172.21.15.116:0/1263141027 learned_addr learned my addr 172.21.15.116:0/1263141027 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:58.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.279+0000 7fb933fff640 1 --2- 172.21.15.116:0/1263141027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c187487760 0x55c18748f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:58.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.279+0000 7fb933fff640 1 --2- 172.21.15.116:0/1263141027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c187492940 0x55c187490260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:58.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.280+0000 7fb938cf1640 1 -- 172.21.15.116:0/1263141027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c187492940 msgr2=0x55c187490260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:58.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.280+0000 7fb938cf1640 1 --2- 172.21.15.116:0/1263141027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c187492940 0x55c187490260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.280+0000 7fb938cf1640 1 -- 172.21.15.116:0/1263141027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c187487760 msgr2=0x55c18748f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:58.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.280+0000 7fb938cf1640 1 --2- 172.21.15.116:0/1263141027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c187487760 0x55c18748f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.280+0000 7fb938cf1640 1 -- 172.21.15.116:0/1263141027 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c18749e9d0 con 0x55c187488080 2024-09-12T03:09:58.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.280+0000 7fb933fff640 1 --2- 172.21.15.116:0/1263141027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c187487760 0x55c18748f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:09:58.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.280+0000 7fb938cf1640 1 --2- 172.21.15.116:0/1263141027 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c187488080 0x55c18748fd20 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fb928011a90 tx=0x7fb928011f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d660f32397168a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:58.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.280+0000 7fb913fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:09:58.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.280+0000 7fb913fff640 1 -- 172.21.15.116:0/1263141027 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92800fe70 con 0x55c187488080 2024-09-12T03:09:58.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.280+0000 7fb913fff640 1 -- 172.21.15.116:0/1263141027 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fb92800e3b0 con 0x55c187488080 2024-09-12T03:09:58.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.280+0000 7fb93d32e740 1 -- 172.21.15.116:0/1263141027 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb920010b10 con 0x55c187488080 2024-09-12T03:09:58.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.280+0000 7fb913fff640 1 -- 172.21.15.116:0/1263141027 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb92800e5b0 con 0x55c187488080 2024-09-12T03:09:58.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.280+0000 7fb93d32e740 10 client.17070 fetch_fsmap learned FSMap version 25 2024-09-12T03:09:58.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.280+0000 7fb93d32e740 1 -- 172.21.15.116:0/1263141027 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c1874a13a0 con 0x55c187488080 2024-09-12T03:09:58.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.280+0000 7fb913fff640 1 -- 172.21.15.116:0/1263141027 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb920010b10 con 0x55c187488080 2024-09-12T03:09:58.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.280+0000 7fb913fff640 1 -- 172.21.15.116:0/1263141027 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb9280269a0 con 0x55c187488080 2024-09-12T03:09:58.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.280+0000 7fb93d32e740 10 client.17070 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:09:58.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.280+0000 7fb93d32e740 10 client.17070 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:09:58.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.280+0000 7fb93d32e740 20 client.17070 populate_metadata read hostname 'smithi116' 2024-09-12T03:09:58.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.280+0000 7fb93d32e740 1 --2- 172.21.15.116:0/1263141027 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c1874a2860 0x55c1874a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:58.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.281+0000 7fb93d32e740 4 client.17070 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:09:58.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.281+0000 7fb93d32e740 1 -- 172.21.15.116:0/1263141027 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1873fa9d0 con 0x55c1874a2860 2024-09-12T03:09:58.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.281+0000 7fb933fff640 1 --2- 172.21.15.116:0/1263141027 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c1874a2860 0x55c1874a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:58.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.281+0000 7fb933fff640 1 --2- 172.21.15.116:0/1263141027 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c1874a2860 0x55c1874a4c90 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74c8ba1679410e0a server_cookie=191f0bb9a5f9ed30 in_seq=0 out_seq=0 2024-09-12T03:09:58.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.281+0000 7fb913fff640 10 client.17070 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:58.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.282+0000 7fb913fff640 1 -- 172.21.15.116:0/1263141027 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c1873fa9d0 con 0x55c1874a2860 2024-09-12T03:09:58.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.282+0000 7fb913fff640 10 client.17070 handle_command_reply: tid=0 2024-09-12T03:09:58.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.282+0000 7fb93d32e740 1 client.17070 shutdown 2024-09-12T03:09:58.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.282+0000 7fb93d32e740 1 -- 172.21.15.116:0/1263141027 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c187488080 msgr2=0x55c18748fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:58.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.282+0000 7fb93d32e740 1 --2- 172.21.15.116:0/1263141027 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c187488080 0x55c18748fd20 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fb928011a90 tx=0x7fb928011f50 comp rx=0 tx=0).stop 2024-09-12T03:09:58.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.282+0000 7fb93d32e740 1 -- 172.21.15.116:0/1263141027 shutdown_connections 2024-09-12T03:09:58.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.282+0000 7fb93d32e740 1 --2- 172.21.15.116:0/1263141027 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c1874a2860 0x55c1874a4c90 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.282+0000 7fb93d32e740 1 --2- 172.21.15.116:0/1263141027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c187492940 0x55c187490260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.282+0000 7fb93d32e740 1 --2- 172.21.15.116:0/1263141027 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c187488080 0x55c18748fd20 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.282+0000 7fb913fff640 0 client.17070 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:58.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.282+0000 7fb93d32e740 1 --2- 172.21.15.116:0/1263141027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c187487760 0x55c18748f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.282+0000 7fb93d32e740 1 -- 172.21.15.116:0/1263141027 >> 172.21.15.116:0/1263141027 conn(0x55c187483cf0 msgr2=0x55c187489a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:58.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.282+0000 7fb93d32e740 1 -- 172.21.15.116:0/1263141027 shutdown_connections 2024-09-12T03:09:58.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.282+0000 7fb93d32e740 1 -- 172.21.15.116:0/1263141027 wait complete. 2024-09-12T03:09:58.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.283+0000 7fb93d32e740 20 client.17070 trim_cache size 0 max 16384 2024-09-12T03:09:58.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.299+0000 7fb93d32e740 1 Processor -- start 2024-09-12T03:09:58.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.299+0000 7fb93d32e740 1 -- start start 2024-09-12T03:09:58.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.299+0000 7fb93d32e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c187487760 0x55c18748a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:58.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.299+0000 7fb93d32e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c187488080 0x55c187482d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:58.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.299+0000 7fb93d32e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c187492940 0x55c1874aa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:58.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.299+0000 7fb93d32e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb9341afc80 con 0x55c187488080 2024-09-12T03:09:58.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.299+0000 7fb93d32e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb9341ad550 con 0x55c187492940 2024-09-12T03:09:58.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.299+0000 7fb93d32e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb9341ab3a0 con 0x55c187487760 2024-09-12T03:09:58.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.300+0000 7fb938cf1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c187488080 0x55c187482d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:58.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.300+0000 7fb938cf1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c187488080 0x55c187482d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40506/0 (socket says 172.21.15.116:40506) 2024-09-12T03:09:58.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.300+0000 7fb938cf1640 1 -- 172.21.15.116:0/3043594650 learned_addr learned my addr 172.21.15.116:0/3043594650 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:58.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.300+0000 7fb933fff640 1 --2- 172.21.15.116:0/3043594650 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c187492940 0x55c1874aa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:58.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.300+0000 7fb938cf1640 1 -- 172.21.15.116:0/3043594650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c187487760 msgr2=0x55c18748a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:58.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.300+0000 7fb938cf1640 1 --2- 172.21.15.116:0/3043594650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c187487760 0x55c18748a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.300+0000 7fb938cf1640 1 -- 172.21.15.116:0/3043594650 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c187492940 msgr2=0x55c1874aa5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:58.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.300+0000 7fb938cf1640 1 --2- 172.21.15.116:0/3043594650 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c187492940 0x55c1874aa5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.300+0000 7fb938cf1640 1 -- 172.21.15.116:0/3043594650 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1874a5310 con 0x55c187488080 2024-09-12T03:09:58.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.300+0000 7fb933fff640 1 --2- 172.21.15.116:0/3043594650 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c187492940 0x55c1874aa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:09:58.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.300+0000 7fb938cf1640 1 --2- 172.21.15.116:0/3043594650 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c187488080 0x55c187482d60 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fb9280058c0 tx=0x7fb92800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba4abb10176fb3c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:58.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.300+0000 7fb91e7fc640 1 -- 172.21.15.116:0/3043594650 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92800f790 con 0x55c187488080 2024-09-12T03:09:58.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.300+0000 7fb91e7fc640 1 -- 172.21.15.116:0/3043594650 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb928005520 con 0x55c187488080 2024-09-12T03:09:58.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.300+0000 7fb91e7fc640 1 -- 172.21.15.116:0/3043594650 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92800ed60 con 0x55c187488080 2024-09-12T03:09:58.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.301+0000 7fb93d32e740 1 -- 172.21.15.116:0/3043594650 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c187488080 msgr2=0x55c187482d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:58.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.301+0000 7fb93d32e740 1 --2- 172.21.15.116:0/3043594650 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c187488080 0x55c187482d60 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fb9280058c0 tx=0x7fb92800cf80 comp rx=0 tx=0).stop 2024-09-12T03:09:58.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.301+0000 7fb93d32e740 1 -- 172.21.15.116:0/3043594650 shutdown_connections 2024-09-12T03:09:58.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.301+0000 7fb93d32e740 1 --2- 172.21.15.116:0/3043594650 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c187492940 0x55c1874aa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.301+0000 7fb93d32e740 1 --2- 172.21.15.116:0/3043594650 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c187488080 0x55c187482d60 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.301+0000 7fb93d32e740 1 --2- 172.21.15.116:0/3043594650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c187487760 0x55c18748a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.301+0000 7fb93d32e740 1 -- 172.21.15.116:0/3043594650 >> 172.21.15.116:0/3043594650 conn(0x55c187483cf0 msgr2=0x55c187482650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:58.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.301+0000 7fb93d32e740 1 -- 172.21.15.116:0/3043594650 shutdown_connections 2024-09-12T03:09:58.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.301+0000 7fb93d32e740 1 -- 172.21.15.116:0/3043594650 wait complete. 2024-09-12T03:09:58.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.302+0000 7fb93d32e740 1 -- start start 2024-09-12T03:09:58.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.302+0000 7fb93d32e740 1 Processor -- start 2024-09-12T03:09:58.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.302+0000 7fb93d32e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c187487760 0x55c187497070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:58.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.302+0000 7fb93d32e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c187492940 0x55c1874975b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:58.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.302+0000 7fb93d32e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c18749a730 0x55c18749abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:58.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.302+0000 7fb93d32e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb9341aba10 con 0x55c18749a730 2024-09-12T03:09:58.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.302+0000 7fb93d32e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb9341ace80 con 0x55c187487760 2024-09-12T03:09:58.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.302+0000 7fb93d32e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb9341ac0e0 con 0x55c187492940 2024-09-12T03:09:58.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.302+0000 7fb93d32e740 10 client.0 _ll_register_callbacks cb 0x7fb928011670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:58.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.302+0000 7fb933fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c18749a730 0x55c18749abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:58.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.302+0000 7fb933fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c18749a730 0x55c18749abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40508/0 (socket says 172.21.15.116:40508) 2024-09-12T03:09:58.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.302+0000 7fb933fff640 1 -- 172.21.15.116:0/1336848242 learned_addr learned my addr 172.21.15.116:0/1336848242 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:09:58.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.302+0000 7fb938cf1640 1 --2- 172.21.15.116:0/1336848242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c187492940 0x55c1874975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:58.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.302+0000 7fb933fff640 1 --2- 172.21.15.116:0/1336848242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c187487760 0x55c187497070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:58.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.303+0000 7fb933fff640 1 -- 172.21.15.116:0/1336848242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c187492940 msgr2=0x55c1874975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:58.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.303+0000 7fb933fff640 1 --2- 172.21.15.116:0/1336848242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c187492940 0x55c1874975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.303+0000 7fb933fff640 1 -- 172.21.15.116:0/1336848242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c187487760 msgr2=0x55c187497070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:58.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.303+0000 7fb933fff640 1 --2- 172.21.15.116:0/1336848242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c187487760 0x55c187497070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.303+0000 7fb933fff640 1 -- 172.21.15.116:0/1336848242 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c187487590 con 0x55c18749a730 2024-09-12T03:09:58.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.303+0000 7fb933fff640 1 --2- 172.21.15.116:0/1336848242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c18749a730 0x55c18749abb0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fb9200133d0 tx=0x7fb920009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c308bff7dd1c40 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:58.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.303+0000 7fb913fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:09:58.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.303+0000 7fb913fff640 1 -- 172.21.15.116:0/1336848242 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92002b040 con 0x55c18749a730 2024-09-12T03:09:58.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.303+0000 7fb913fff640 1 -- 172.21.15.116:0/1336848242 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fb9200052f0 con 0x55c18749a730 2024-09-12T03:09:58.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.303+0000 7fb93d32e740 1 -- 172.21.15.116:0/1336848242 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb928005520 con 0x55c18749a730 2024-09-12T03:09:58.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.303+0000 7fb913fff640 1 -- 172.21.15.116:0/1336848242 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb920005510 con 0x55c18749a730 2024-09-12T03:09:58.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.303+0000 7fb93d32e740 10 client.17076 fetch_fsmap learned FSMap version 25 2024-09-12T03:09:58.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.303+0000 7fb93d32e740 1 -- 172.21.15.116:0/1336848242 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c18750e2a0 con 0x55c18749a730 2024-09-12T03:09:58.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.304+0000 7fb913fff640 1 -- 172.21.15.116:0/1336848242 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb928005520 con 0x55c18749a730 2024-09-12T03:09:58.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.304+0000 7fb913fff640 1 -- 172.21.15.116:0/1336848242 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb92001e070 con 0x55c18749a730 2024-09-12T03:09:58.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.304+0000 7fb93d32e740 10 client.17076 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:09:58.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.304+0000 7fb93d32e740 10 client.17076 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:09:58.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.304+0000 7fb93d32e740 20 client.17076 populate_metadata read hostname 'smithi116' 2024-09-12T03:09:58.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.304+0000 7fb93d32e740 1 --2- 172.21.15.116:0/1336848242 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c18750e8a0 0x55c187510c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:58.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.304+0000 7fb93d32e740 4 client.17076 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}] 2024-09-12T03:09:58.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.304+0000 7fb93d32e740 1 -- 172.21.15.116:0/1336848242 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}) -- 0x55c1873f4d20 con 0x55c18750e8a0 2024-09-12T03:09:58.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.304+0000 7fb938cf1640 1 --2- 172.21.15.116:0/1336848242 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c18750e8a0 0x55c187510c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:58.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.304+0000 7fb938cf1640 1 --2- 172.21.15.116:0/1336848242 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c18750e8a0 0x55c187510c90 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5624a3d6a66f7e76 server_cookie=cc7f142b241137ae in_seq=0 out_seq=0 2024-09-12T03:09:58.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.304+0000 7fb913fff640 10 client.17076 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:58.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.313+0000 7fb913fff640 1 -- 172.21.15.116:0/1336848242 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c1873f4d20 con 0x55c18750e8a0 2024-09-12T03:09:58.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.313+0000 7fb913fff640 10 client.17076 handle_command_reply: tid=0 2024-09-12T03:09:58.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.313+0000 7fb93d32e740 1 client.17076 shutdown 2024-09-12T03:09:58.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.313+0000 7fb93d32e740 1 -- 172.21.15.116:0/1336848242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c18749a730 msgr2=0x55c18749abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:58.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.313+0000 7fb93d32e740 1 --2- 172.21.15.116:0/1336848242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c18749a730 0x55c18749abb0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fb9200133d0 tx=0x7fb920009e80 comp rx=0 tx=0).stop 2024-09-12T03:09:58.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.313+0000 7fb93d32e740 1 -- 172.21.15.116:0/1336848242 shutdown_connections 2024-09-12T03:09:58.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.313+0000 7fb93d32e740 1 --2- 172.21.15.116:0/1336848242 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c18750e8a0 0x55c187510c90 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.313+0000 7fb93d32e740 1 --2- 172.21.15.116:0/1336848242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c18749a730 0x55c18749abb0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.313+0000 7fb913fff640 0 client.17076 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:09:58.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.313+0000 7fb93d32e740 1 --2- 172.21.15.116:0/1336848242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c187492940 0x55c1874975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.313+0000 7fb93d32e740 1 --2- 172.21.15.116:0/1336848242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c187487760 0x55c187497070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.313+0000 7fb93d32e740 1 -- 172.21.15.116:0/1336848242 >> 172.21.15.116:0/1336848242 conn(0x55c187483cf0 msgr2=0x55c18748ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:58.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.313+0000 7fb93d32e740 1 -- 172.21.15.116:0/1336848242 shutdown_connections 2024-09-12T03:09:58.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.313+0000 7fb93d32e740 1 -- 172.21.15.116:0/1336848242 wait complete. 2024-09-12T03:09:58.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.313+0000 7fb93d32e740 20 client.17076 trim_cache size 0 max 16384 2024-09-12T03:09:58.315 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:09:58.315 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 25, 2024-09-12T03:09:58.315 INFO:teuthology.orchestra.run.smithi116.stdout: "leader": 24494, 2024-09-12T03:09:58.315 INFO:teuthology.orchestra.run.smithi116.stdout: "set_version": 26, 2024-09-12T03:09:58.315 INFO:teuthology.orchestra.run.smithi116.stdout: "sets": { 2024-09-12T03:09:58.315 INFO:teuthology.orchestra.run.smithi116.stdout: "4007087b": { 2024-09-12T03:09:58.316 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 26, 2024-09-12T03:09:58.316 INFO:teuthology.orchestra.run.smithi116.stdout: "age_ref": 0.0, 2024-09-12T03:09:58.316 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:09:58.316 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "QUIESCED", 2024-09-12T03:09:58.316 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:09:58.316 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:09:58.316 INFO:teuthology.orchestra.run.smithi116.stdout: "timeout": 90.0, 2024-09-12T03:09:58.316 INFO:teuthology.orchestra.run.smithi116.stdout: "expiration": 136.6, 2024-09-12T03:09:58.316 INFO:teuthology.orchestra.run.smithi116.stdout: "members": { 2024-09-12T03:09:58.316 INFO:teuthology.orchestra.run.smithi116.stdout: "file:/": { 2024-09-12T03:09:58.316 INFO:teuthology.orchestra.run.smithi116.stdout: "excluded": false, 2024-09-12T03:09:58.316 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:09:58.316 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "QUIESCED", 2024-09-12T03:09:58.317 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:09:58.317 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:09:58.317 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:09:58.317 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:09:58.317 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:09:58.317 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:09:58.317 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:09:58.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.314+0000 7fb93af55640 1 -- 172.21.15.116:0/2898837638 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb9240a2b80 msgr2=0x7fb9240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:58.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.314+0000 7fb93af55640 1 --2- 172.21.15.116:0/2898837638 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb9240a2b80 0x7fb9240a5040 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0319f0 tx=0x7fb92c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.314+0000 7fb93af55640 1 -- 172.21.15.116:0/2898837638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb934106f20 msgr2=0x7fb93419cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:58.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.314+0000 7fb93af55640 1 --2- 172.21.15.116:0/2898837638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb934106f20 0x7fb93419cc30 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fb92c031a20 tx=0x7fb92c031a50 comp rx=0 tx=0).stop 2024-09-12T03:09:58.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.314+0000 7fb93af55640 1 -- 172.21.15.116:0/2898837638 shutdown_connections 2024-09-12T03:09:58.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.314+0000 7fb93af55640 1 --2- 172.21.15.116:0/2898837638 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb9240a2b80 0x7fb9240a5040 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.314+0000 7fb93af55640 1 --2- 172.21.15.116:0/2898837638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb934106f20 0x7fb93419cc30 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.314+0000 7fb93af55640 1 --2- 172.21.15.116:0/2898837638 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9341027a0 0x7fb9341988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.314+0000 7fb93af55640 1 --2- 172.21.15.116:0/2898837638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb934101df0 0x7fb934198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.314+0000 7fb93af55640 1 -- 172.21.15.116:0/2898837638 >> 172.21.15.116:0/2898837638 conn(0x7fb9340fd8b0 msgr2=0x7fb9340fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:58.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.314+0000 7fb93af55640 1 -- 172.21.15.116:0/2898837638 shutdown_connections 2024-09-12T03:09:58.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:09:58.314+0000 7fb93af55640 1 -- 172.21.15.116:0/2898837638 wait complete. 2024-09-12T03:09:58.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:58 smithi116 ceph-mon[31410]: pgmap v547: 97 pgs: 97 active+clean; 3.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:09:58.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:09:58 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1885762370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:09:58.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:58 smithi183 ceph-mon[35535]: pgmap v547: 97 pgs: 97 active+clean; 3.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:09:58.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:09:58 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1885762370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:09:58.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:58 smithi177 ceph-mon[35305]: pgmap v547: 97 pgs: 97 active+clean; 3.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:09:58.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:09:58 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1885762370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:09:58.923 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:09:59.662 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4007087b', quiesce took 5.7 seconds. Will release after: 10.900000000000002 2024-09-12T03:09:59.755 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:10:00.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.253+0000 7f7eca55c640 1 -- 172.21.15.116:0/1020684988 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ec4101420 msgr2=0x7f7ec410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:00.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.253+0000 7f7eca55c640 1 --2- 172.21.15.116:0/1020684988 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ec4101420 0x7f7ec410d950 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009cd0 tx=0x7f7ebc02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.253+0000 7f7eca55c640 1 -- 172.21.15.116:0/1020684988 shutdown_connections 2024-09-12T03:10:00.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.253+0000 7f7eca55c640 1 --2- 172.21.15.116:0/1020684988 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ec4101420 0x7f7ec410d950 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.253+0000 7f7eca55c640 1 --2- 172.21.15.116:0/1020684988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ec4100a80 0x7f7ec4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.253+0000 7f7eca55c640 1 --2- 172.21.15.116:0/1020684988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ec4106a80 0x7f7ec4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.253+0000 7f7eca55c640 1 -- 172.21.15.116:0/1020684988 >> 172.21.15.116:0/1020684988 conn(0x7f7ec40fc7a0 msgr2=0x7f7ec40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:00.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.253+0000 7f7eca55c640 1 -- 172.21.15.116:0/1020684988 shutdown_connections 2024-09-12T03:10:00.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.253+0000 7f7eca55c640 1 -- 172.21.15.116:0/1020684988 wait complete. 2024-09-12T03:10:00.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.253+0000 7f7eca55c640 1 Processor -- start 2024-09-12T03:10:00.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.254+0000 7f7eca55c640 1 -- start start 2024-09-12T03:10:00.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.254+0000 7f7eca55c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ec4100a80 0x7f7ec40fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:00.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.254+0000 7f7eca55c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ec4101420 0x7f7ec40fd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:00.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.254+0000 7f7eca55c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ec4106a80 0x7f7ec40fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:00.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.254+0000 7f7eca55c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec4110ea0 con 0x7f7ec4106a80 2024-09-12T03:10:00.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.254+0000 7f7eca55c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec4110d20 con 0x7f7ec4101420 2024-09-12T03:10:00.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.254+0000 7f7eca55c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec4111020 con 0x7f7ec4100a80 2024-09-12T03:10:00.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.254+0000 7f7ec8af9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ec4106a80 0x7f7ec40fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:00.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.254+0000 7f7ec8af9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ec4106a80 0x7f7ec40fd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40528/0 (socket says 172.21.15.116:40528) 2024-09-12T03:10:00.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.254+0000 7f7ec8af9640 1 -- 172.21.15.116:0/2994675483 learned_addr learned my addr 172.21.15.116:0/2994675483 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:00.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.254+0000 7f7ebbfff640 1 --2- 172.21.15.116:0/2994675483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ec4101420 0x7f7ec40fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:00.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.254+0000 7f7ec3fff640 1 --2- 172.21.15.116:0/2994675483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ec4100a80 0x7f7ec40fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:00.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.254+0000 7f7ec8af9640 1 -- 172.21.15.116:0/2994675483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ec4100a80 msgr2=0x7f7ec40fcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:00.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.254+0000 7f7ec8af9640 1 --2- 172.21.15.116:0/2994675483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ec4100a80 0x7f7ec40fcde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.254+0000 7f7ec8af9640 1 -- 172.21.15.116:0/2994675483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ec4101420 msgr2=0x7f7ec40fd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:00.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.254+0000 7f7ec8af9640 1 --2- 172.21.15.116:0/2994675483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ec4101420 0x7f7ec40fd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.254+0000 7f7ec8af9640 1 -- 172.21.15.116:0/2994675483 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec41a68f0 con 0x7f7ec4106a80 2024-09-12T03:10:00.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.254+0000 7f7ec8af9640 1 --2- 172.21.15.116:0/2994675483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ec4106a80 0x7f7ec40fd860 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc005cc0 tx=0x7f7ebc003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:00.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.255+0000 7f7ec1ffb640 1 -- 172.21.15.116:0/2994675483 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc004040 con 0x7f7ec4106a80 2024-09-12T03:10:00.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.255+0000 7f7eca55c640 1 -- 172.21.15.116:0/2994675483 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ec41a6b80 con 0x7f7ec4106a80 2024-09-12T03:10:00.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.255+0000 7f7ec1ffb640 1 -- 172.21.15.116:0/2994675483 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ebc0046c0 con 0x7f7ec4106a80 2024-09-12T03:10:00.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.255+0000 7f7ec1ffb640 1 -- 172.21.15.116:0/2994675483 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc030e70 con 0x7f7ec4106a80 2024-09-12T03:10:00.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.255+0000 7f7eca55c640 1 -- 172.21.15.116:0/2994675483 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ec41a7110 con 0x7f7ec4106a80 2024-09-12T03:10:00.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.257+0000 7f7ec1ffb640 1 -- 172.21.15.116:0/2994675483 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f7ebc0305a0 con 0x7f7ec4106a80 2024-09-12T03:10:00.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.257+0000 7f7ecc935740 1 Processor -- start 2024-09-12T03:10:00.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.257+0000 7f7ecc935740 1 -- start start 2024-09-12T03:10:00.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.257+0000 7f7ecc935740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599514c4b40 0x5599514c3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:00.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.257+0000 7f7ecc935740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5599514c35d0 0x5599514c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:00.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.257+0000 7f7ecc935740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5599514ce370 0x5599514d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:00.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.258+0000 7f7ecc935740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5599514292f0 con 0x5599514ce370 2024-09-12T03:10:00.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.258+0000 7f7ecc935740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5599513d0cf0 con 0x5599514c35d0 2024-09-12T03:10:00.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.258+0000 7f7ecc935740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5599514252b0 con 0x5599514c4b40 2024-09-12T03:10:00.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.258+0000 7f7ebbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5599514ce370 0x5599514d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:00.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.258+0000 7f7ebbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5599514ce370 0x5599514d0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40540/0 (socket says 172.21.15.116:40540) 2024-09-12T03:10:00.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.258+0000 7f7ebbfff640 1 -- 172.21.15.116:0/2288737617 learned_addr learned my addr 172.21.15.116:0/2288737617 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:00.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.258+0000 7f7ec1ffb640 1 --2- 172.21.15.116:0/2994675483 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7eb00a2b80 0x7f7eb00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:00.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.258+0000 7f7ec3fff640 1 --2- 172.21.15.116:0/2288737617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5599514c35d0 0x5599514c3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:00.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.258+0000 7f7ebbfff640 1 --2- 172.21.15.116:0/2288737617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599514c4b40 0x5599514c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:00.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.258+0000 7f7ec8af9640 1 --2- 172.21.15.116:0/2994675483 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7eb00a2b80 0x7f7eb00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:00.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.258+0000 7f7ebbfff640 1 -- 172.21.15.116:0/2288737617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599514c4b40 msgr2=0x5599514c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:00.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.258+0000 7f7ebbfff640 1 --2- 172.21.15.116:0/2288737617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599514c4b40 0x5599514c3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.258+0000 7f7ebbfff640 1 -- 172.21.15.116:0/2288737617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5599514c35d0 msgr2=0x5599514c3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:00.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.258+0000 7f7ebbfff640 1 --2- 172.21.15.116:0/2288737617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5599514c35d0 0x5599514c3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.258+0000 7f7ebbfff640 1 -- 172.21.15.116:0/2288737617 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599513b53a0 con 0x5599514ce370 2024-09-12T03:10:00.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.258+0000 7f7ec1ffb640 1 -- 172.21.15.116:0/2994675483 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f7ebc02e030 con 0x7f7ec4106a80 2024-09-12T03:10:00.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.258+0000 7f7ebbfff640 1 --2- 172.21.15.116:0/2288737617 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5599514ce370 0x5599514d0760 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f7eac009f60 tx=0x7f7eac0095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdfc0bb964dfaa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:00.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.258+0000 7f7ec8af9640 1 --2- 172.21.15.116:0/2994675483 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7eb00a2b80 0x7f7eb00a5040 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc0c4ec0 tx=0x7f7ebc0877f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:00.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.258+0000 7f7eba7fc640 1 -- 172.21.15.116:0/2288737617 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac017610 con 0x5599514ce370 2024-09-12T03:10:00.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.258+0000 7f7eba7fc640 1 -- 172.21.15.116:0/2288737617 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eac01c070 con 0x5599514ce370 2024-09-12T03:10:00.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.258+0000 7f7eba7fc640 1 -- 172.21.15.116:0/2288737617 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac015b50 con 0x5599514ce370 2024-09-12T03:10:00.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.259+0000 7f7ecc935740 1 -- 172.21.15.116:0/2288737617 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5599514ce370 msgr2=0x5599514d0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:00.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.259+0000 7f7ecc935740 1 --2- 172.21.15.116:0/2288737617 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5599514ce370 0x5599514d0760 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f7eac009f60 tx=0x7f7eac0095a0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.259+0000 7f7ecc935740 1 -- 172.21.15.116:0/2288737617 shutdown_connections 2024-09-12T03:10:00.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.259+0000 7f7ecc935740 1 --2- 172.21.15.116:0/2288737617 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5599514ce370 0x5599514d0760 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.259+0000 7f7ecc935740 1 --2- 172.21.15.116:0/2288737617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5599514c35d0 0x5599514c3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.259+0000 7f7ecc935740 1 --2- 172.21.15.116:0/2288737617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599514c4b40 0x5599514c3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.259+0000 7f7ecc935740 1 -- 172.21.15.116:0/2288737617 >> 172.21.15.116:0/2288737617 conn(0x55995148c310 msgr2=0x5599514bfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:00.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.259+0000 7f7ecc935740 1 -- 172.21.15.116:0/2288737617 shutdown_connections 2024-09-12T03:10:00.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.259+0000 7f7ecc935740 1 -- 172.21.15.116:0/2288737617 wait complete. 2024-09-12T03:10:00.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.259+0000 7f7ecc935740 1 -- start start 2024-09-12T03:10:00.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.260+0000 7f7ecc935740 1 Processor -- start 2024-09-12T03:10:00.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.260+0000 7f7ecc935740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5599514c35d0 0x5599514cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:00.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.260+0000 7f7ecc935740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5599514c4b40 0x5599514cb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:00.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.260+0000 7f7ecc935740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599514ce370 0x5599514cb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:00.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.260+0000 7f7ecc935740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5599514d1070 con 0x5599514c4b40 2024-09-12T03:10:00.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.260+0000 7f7ecc935740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5599513e4060 con 0x5599514c35d0 2024-09-12T03:10:00.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.260+0000 7f7ecc935740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5599514d11f0 con 0x5599514ce370 2024-09-12T03:10:00.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.260+0000 7f7ecc935740 10 client.0 _ll_register_callbacks cb 0x5599511378c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:00.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.260+0000 7f7ec3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5599514c4b40 0x5599514cb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:00.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.260+0000 7f7ec3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5599514c4b40 0x5599514cb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40556/0 (socket says 172.21.15.116:40556) 2024-09-12T03:10:00.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.260+0000 7f7ec3fff640 1 -- 172.21.15.116:0/4202892522 learned_addr learned my addr 172.21.15.116:0/4202892522 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:00.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.260+0000 7f7ebbfff640 1 --2- 172.21.15.116:0/4202892522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599514ce370 0x5599514cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:00.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.260+0000 7f7ebbfff640 1 --2- 172.21.15.116:0/4202892522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5599514c35d0 0x5599514cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:00.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.260+0000 7f7ec3fff640 1 -- 172.21.15.116:0/4202892522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599514ce370 msgr2=0x5599514cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:00.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.260+0000 7f7ec3fff640 1 --2- 172.21.15.116:0/4202892522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599514ce370 0x5599514cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.260+0000 7f7ec3fff640 1 -- 172.21.15.116:0/4202892522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5599514c35d0 msgr2=0x5599514cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:00.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.260+0000 7f7ec3fff640 1 --2- 172.21.15.116:0/4202892522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5599514c35d0 0x5599514cdd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.260+0000 7f7ec3fff640 1 -- 172.21.15.116:0/4202892522 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55995119a3c0 con 0x5599514c4b40 2024-09-12T03:10:00.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.261+0000 7f7ec3fff640 1 --2- 172.21.15.116:0/4202892522 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5599514c4b40 0x5599514cb030 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400bad0 tx=0x7f7eb400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=524bec9e24089aaf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:00.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.261+0000 7f7ea37fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:00.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.261+0000 7f7ea37fe640 1 -- 172.21.15.116:0/4202892522 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4005020 con 0x5599514c4b40 2024-09-12T03:10:00.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.261+0000 7f7ea37fe640 1 -- 172.21.15.116:0/4202892522 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f7eb40051c0 con 0x5599514c4b40 2024-09-12T03:10:00.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.261+0000 7f7ecc935740 1 -- 172.21.15.116:0/4202892522 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7eac01c070 con 0x5599514c4b40 2024-09-12T03:10:00.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.261+0000 7f7ea37fe640 1 -- 172.21.15.116:0/4202892522 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7eb4005400 con 0x5599514c4b40 2024-09-12T03:10:00.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.261+0000 7f7ecc935740 10 client.17088 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:00.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.261+0000 7f7ecc935740 1 -- 172.21.15.116:0/4202892522 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5599514dcc20 con 0x5599514c4b40 2024-09-12T03:10:00.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.261+0000 7f7ea37fe640 1 -- 172.21.15.116:0/4202892522 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eac01c070 con 0x5599514c4b40 2024-09-12T03:10:00.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.261+0000 7f7ea37fe640 1 -- 172.21.15.116:0/4202892522 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7eb401b6d0 con 0x5599514c4b40 2024-09-12T03:10:00.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.261+0000 7f7ecc935740 10 client.17088 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:00.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.261+0000 7f7ecc935740 10 client.17088 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:10:00.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.261+0000 7f7ecc935740 20 client.17088 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:00.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.261+0000 7f7ecc935740 1 --2- 172.21.15.116:0/4202892522 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5599514de0e0 0x5599514e04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:00.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.261+0000 7f7ecc935740 4 client.17088 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:10:00.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.261+0000 7f7ecc935740 1 -- 172.21.15.116:0/4202892522 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559951436860 con 0x5599514de0e0 2024-09-12T03:10:00.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.262+0000 7f7ebbfff640 1 --2- 172.21.15.116:0/4202892522 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5599514de0e0 0x5599514e04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:00.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.262+0000 7f7ea37fe640 10 client.17088 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:10:00.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.262+0000 7f7ebbfff640 1 --2- 172.21.15.116:0/4202892522 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5599514de0e0 0x5599514e04f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c8b473d85b76f35 server_cookie=8c5b86d268750bdd in_seq=0 out_seq=0 2024-09-12T03:10:00.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.263+0000 7f7ea37fe640 1 -- 172.21.15.116:0/4202892522 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559951436860 con 0x5599514de0e0 2024-09-12T03:10:00.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.263+0000 7f7ea37fe640 10 client.17088 handle_command_reply: tid=0 2024-09-12T03:10:00.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.263+0000 7f7ecc935740 1 client.17088 shutdown 2024-09-12T03:10:00.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.263+0000 7f7ecc935740 1 -- 172.21.15.116:0/4202892522 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5599514c4b40 msgr2=0x5599514cb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:00.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.263+0000 7f7ecc935740 1 --2- 172.21.15.116:0/4202892522 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5599514c4b40 0x5599514cb030 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400bad0 tx=0x7f7eb400bed0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.263+0000 7f7ecc935740 1 -- 172.21.15.116:0/4202892522 shutdown_connections 2024-09-12T03:10:00.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.263+0000 7f7ecc935740 1 --2- 172.21.15.116:0/4202892522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599514ce370 0x5599514cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.263+0000 7f7ecc935740 1 --2- 172.21.15.116:0/4202892522 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5599514de0e0 0x5599514e04f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.263+0000 7f7ecc935740 1 --2- 172.21.15.116:0/4202892522 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5599514c4b40 0x5599514cb030 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.263+0000 7f7ecc935740 1 --2- 172.21.15.116:0/4202892522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5599514c35d0 0x5599514cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.263+0000 7f7ecc935740 1 -- 172.21.15.116:0/4202892522 >> 172.21.15.116:0/4202892522 conn(0x55995148c310 msgr2=0x5599514c5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:00.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.263+0000 7f7ea37fe640 0 client.17088 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:10:00.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.263+0000 7f7ecc935740 1 -- 172.21.15.116:0/4202892522 shutdown_connections 2024-09-12T03:10:00.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.263+0000 7f7ecc935740 1 -- 172.21.15.116:0/4202892522 wait complete. 2024-09-12T03:10:00.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.263+0000 7f7ecc935740 20 client.17088 trim_cache size 0 max 16384 2024-09-12T03:10:00.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.276+0000 7f7ecc935740 1 Processor -- start 2024-09-12T03:10:00.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.276+0000 7f7ecc935740 1 -- start start 2024-09-12T03:10:00.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.276+0000 7f7ecc935740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5599514c35d0 0x5599514bfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:00.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.276+0000 7f7ecc935740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599514c4b40 0x5599514da340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:00.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.276+0000 7f7ecc935740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5599514ce370 0x5599514e5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:00.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.276+0000 7f7ecc935740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec41b1e60 con 0x5599514c35d0 2024-09-12T03:10:00.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.276+0000 7f7ecc935740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec41af730 con 0x5599514ce370 2024-09-12T03:10:00.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.276+0000 7f7ecc935740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec41ad5a0 con 0x5599514c4b40 2024-09-12T03:10:00.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.276+0000 7f7ebbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5599514c35d0 0x5599514bfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:00.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.276+0000 7f7ebbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5599514c35d0 0x5599514bfe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40568/0 (socket says 172.21.15.116:40568) 2024-09-12T03:10:00.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.276+0000 7f7ebbfff640 1 -- 172.21.15.116:0/316153882 learned_addr learned my addr 172.21.15.116:0/316153882 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:00.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.277+0000 7f7ebbfff640 1 -- 172.21.15.116:0/316153882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599514c4b40 msgr2=0x5599514da340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:00.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.277+0000 7f7ebbfff640 1 --2- 172.21.15.116:0/316153882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599514c4b40 0x5599514da340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.277+0000 7f7ebbfff640 1 -- 172.21.15.116:0/316153882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5599514ce370 msgr2=0x5599514e5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:00.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.277+0000 7f7ebbfff640 1 --2- 172.21.15.116:0/316153882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5599514ce370 0x5599514e5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.277+0000 7f7ebbfff640 1 -- 172.21.15.116:0/316153882 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599514c0960 con 0x5599514c35d0 2024-09-12T03:10:00.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.277+0000 7f7ebbfff640 1 --2- 172.21.15.116:0/316153882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5599514c35d0 0x5599514bfe10 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f7eac009f60 tx=0x7f7eac010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d98884c2bd6998fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:00.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.277+0000 7f7eb9ffb640 1 -- 172.21.15.116:0/316153882 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac005c40 con 0x5599514c35d0 2024-09-12T03:10:00.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.277+0000 7f7eb9ffb640 1 -- 172.21.15.116:0/316153882 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eac01c070 con 0x5599514c35d0 2024-09-12T03:10:00.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.277+0000 7f7eb9ffb640 1 -- 172.21.15.116:0/316153882 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac0062b0 con 0x5599514c35d0 2024-09-12T03:10:00.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.278+0000 7f7ecc935740 1 -- 172.21.15.116:0/316153882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5599514c35d0 msgr2=0x5599514bfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:00.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.278+0000 7f7ecc935740 1 --2- 172.21.15.116:0/316153882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5599514c35d0 0x5599514bfe10 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f7eac009f60 tx=0x7f7eac010f90 comp rx=0 tx=0).stop 2024-09-12T03:10:00.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.278+0000 7f7ecc935740 1 -- 172.21.15.116:0/316153882 shutdown_connections 2024-09-12T03:10:00.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.278+0000 7f7ecc935740 1 --2- 172.21.15.116:0/316153882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5599514ce370 0x5599514e5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.278+0000 7f7ecc935740 1 --2- 172.21.15.116:0/316153882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599514c4b40 0x5599514da340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.278+0000 7f7ecc935740 1 --2- 172.21.15.116:0/316153882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5599514c35d0 0x5599514bfe10 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.278+0000 7f7ecc935740 1 -- 172.21.15.116:0/316153882 >> 172.21.15.116:0/316153882 conn(0x55995148c310 msgr2=0x5599514bf700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:00.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.278+0000 7f7ecc935740 1 -- 172.21.15.116:0/316153882 shutdown_connections 2024-09-12T03:10:00.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.278+0000 7f7ecc935740 1 -- 172.21.15.116:0/316153882 wait complete. 2024-09-12T03:10:00.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.278+0000 7f7ecc935740 1 -- start start 2024-09-12T03:10:00.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.278+0000 7f7ecc935740 1 Processor -- start 2024-09-12T03:10:00.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.279+0000 7f7ecc935740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5599514c35d0 0x5599514d6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:00.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.279+0000 7f7ecc935740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599514c4b40 0x5599514c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:00.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.279+0000 7f7ecc935740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5599514ce370 0x5599514c8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:00.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.279+0000 7f7ecc935740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec41adbf0 con 0x5599514c35d0 2024-09-12T03:10:00.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.279+0000 7f7ecc935740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec41af060 con 0x5599514ce370 2024-09-12T03:10:00.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.279+0000 7f7ecc935740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec41ae2c0 con 0x5599514c4b40 2024-09-12T03:10:00.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.279+0000 7f7ecc935740 10 client.0 _ll_register_callbacks cb 0x7f7eb40040a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:00.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.279+0000 7f7ebbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5599514c35d0 0x5599514d6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:00.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.279+0000 7f7ebbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5599514c35d0 0x5599514d6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40582/0 (socket says 172.21.15.116:40582) 2024-09-12T03:10:00.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.279+0000 7f7ebbfff640 1 -- 172.21.15.116:0/2873738114 learned_addr learned my addr 172.21.15.116:0/2873738114 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:00.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.279+0000 7f7ec3fff640 1 --2- 172.21.15.116:0/2873738114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599514c4b40 0x5599514c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:00.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.279+0000 7f7ebbfff640 1 --2- 172.21.15.116:0/2873738114 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5599514ce370 0x5599514c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:00.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.279+0000 7f7ebbfff640 1 -- 172.21.15.116:0/2873738114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599514c4b40 msgr2=0x5599514c7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:00.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.279+0000 7f7ebbfff640 1 --2- 172.21.15.116:0/2873738114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599514c4b40 0x5599514c7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.279+0000 7f7ebbfff640 1 -- 172.21.15.116:0/2873738114 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5599514ce370 msgr2=0x5599514c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:00.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.279+0000 7f7ebbfff640 1 --2- 172.21.15.116:0/2873738114 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5599514ce370 0x5599514c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.279+0000 7f7ebbfff640 1 -- 172.21.15.116:0/2873738114 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599514c8ca0 con 0x5599514c35d0 2024-09-12T03:10:00.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.279+0000 7f7ebbfff640 1 --2- 172.21.15.116:0/2873738114 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5599514c35d0 0x5599514d6d00 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f7eac010b90 tx=0x7f7eac009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200b8f81adc7641a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:00.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.279+0000 7f7ea37fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:00.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.279+0000 7f7ea37fe640 1 -- 172.21.15.116:0/2873738114 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac02f040 con 0x5599514c35d0 2024-09-12T03:10:00.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.279+0000 7f7ea37fe640 1 -- 172.21.15.116:0/2873738114 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f7eac0059c0 con 0x5599514c35d0 2024-09-12T03:10:00.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.279+0000 7f7ecc935740 1 -- 172.21.15.116:0/2873738114 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7eac01c070 con 0x5599514c35d0 2024-09-12T03:10:00.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.279+0000 7f7ea37fe640 1 -- 172.21.15.116:0/2873738114 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7eac015c90 con 0x5599514c35d0 2024-09-12T03:10:00.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.280+0000 7f7ecc935740 10 client.17094 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:00.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.280+0000 7f7ecc935740 1 -- 172.21.15.116:0/2873738114 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5599515516a0 con 0x5599514c35d0 2024-09-12T03:10:00.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.280+0000 7f7ea37fe640 1 -- 172.21.15.116:0/2873738114 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eac01c070 con 0x5599514c35d0 2024-09-12T03:10:00.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.280+0000 7f7ea37fe640 1 -- 172.21.15.116:0/2873738114 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7eac035070 con 0x5599514c35d0 2024-09-12T03:10:00.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.280+0000 7f7ecc935740 10 client.17094 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:00.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.280+0000 7f7ecc935740 10 client.17094 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:10:00.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.280+0000 7f7ecc935740 20 client.17094 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:00.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.280+0000 7f7ecc935740 1 --2- 172.21.15.116:0/2873738114 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x559951551a90 0x559951553e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:00.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.280+0000 7f7ecc935740 4 client.17094 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:10:00.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.280+0000 7f7ecc935740 1 -- 172.21.15.116:0/2873738114 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559951430d70 con 0x559951551a90 2024-09-12T03:10:00.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.280+0000 7f7ec3fff640 1 --2- 172.21.15.116:0/2873738114 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x559951551a90 0x559951553e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:00.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.281+0000 7f7ea37fe640 10 client.17094 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:10:00.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.281+0000 7f7ec3fff640 1 --2- 172.21.15.116:0/2873738114 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x559951551a90 0x559951553e80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=109620fb9e468ddd server_cookie=53520eab133ca47 in_seq=0 out_seq=0 2024-09-12T03:10:00.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.281+0000 7f7ea37fe640 1 -- 172.21.15.116:0/2873738114 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559951430d70 con 0x559951551a90 2024-09-12T03:10:00.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.281+0000 7f7ea37fe640 10 client.17094 handle_command_reply: tid=0 2024-09-12T03:10:00.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.281+0000 7f7ecc935740 1 client.17094 shutdown 2024-09-12T03:10:00.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.281+0000 7f7ecc935740 1 -- 172.21.15.116:0/2873738114 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5599514c35d0 msgr2=0x5599514d6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:00.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.281+0000 7f7ecc935740 1 --2- 172.21.15.116:0/2873738114 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5599514c35d0 0x5599514d6d00 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f7eac010b90 tx=0x7f7eac009530 comp rx=0 tx=0).stop 2024-09-12T03:10:00.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.281+0000 7f7ecc935740 1 -- 172.21.15.116:0/2873738114 shutdown_connections 2024-09-12T03:10:00.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.281+0000 7f7ecc935740 1 --2- 172.21.15.116:0/2873738114 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5599514ce370 0x5599514c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.281+0000 7f7ecc935740 1 --2- 172.21.15.116:0/2873738114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599514c4b40 0x5599514c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.281+0000 7f7ecc935740 1 --2- 172.21.15.116:0/2873738114 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x559951551a90 0x559951553e80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.281+0000 7f7ecc935740 1 --2- 172.21.15.116:0/2873738114 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5599514c35d0 0x5599514d6d00 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.281+0000 7f7ecc935740 1 -- 172.21.15.116:0/2873738114 >> 172.21.15.116:0/2873738114 conn(0x55995148c310 msgr2=0x5599514cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:00.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.281+0000 7f7ea37fe640 0 client.17094 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:10:00.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.281+0000 7f7ecc935740 1 -- 172.21.15.116:0/2873738114 shutdown_connections 2024-09-12T03:10:00.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.281+0000 7f7ecc935740 1 -- 172.21.15.116:0/2873738114 wait complete. 2024-09-12T03:10:00.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.281+0000 7f7ecc935740 20 client.17094 trim_cache size 0 max 16384 2024-09-12T03:10:00.295 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:10:00.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.282+0000 7f7eca55c640 1 -- 172.21.15.116:0/2994675483 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7eb00a2b80 msgr2=0x7f7eb00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:00.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.282+0000 7f7eca55c640 1 --2- 172.21.15.116:0/2994675483 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7eb00a2b80 0x7f7eb00a5040 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc0c4ec0 tx=0x7f7ebc0877f0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.282+0000 7f7eca55c640 1 -- 172.21.15.116:0/2994675483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ec4106a80 msgr2=0x7f7ec40fd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:00.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.282+0000 7f7eca55c640 1 --2- 172.21.15.116:0/2994675483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ec4106a80 0x7f7ec40fd860 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc005cc0 tx=0x7f7ebc003970 comp rx=0 tx=0).stop 2024-09-12T03:10:00.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.282+0000 7f7eca55c640 1 -- 172.21.15.116:0/2994675483 shutdown_connections 2024-09-12T03:10:00.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.282+0000 7f7eca55c640 1 --2- 172.21.15.116:0/2994675483 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7eb00a2b80 0x7f7eb00a5040 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.282+0000 7f7eca55c640 1 --2- 172.21.15.116:0/2994675483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ec4106a80 0x7f7ec40fd860 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.282+0000 7f7eca55c640 1 --2- 172.21.15.116:0/2994675483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ec4101420 0x7f7ec40fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.282+0000 7f7eca55c640 1 --2- 172.21.15.116:0/2994675483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ec4100a80 0x7f7ec40fcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.282+0000 7f7eca55c640 1 -- 172.21.15.116:0/2994675483 >> 172.21.15.116:0/2994675483 conn(0x7f7ec40fc7a0 msgr2=0x7f7ec406a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:00.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.282+0000 7f7eca55c640 1 -- 172.21.15.116:0/2994675483 shutdown_connections 2024-09-12T03:10:00.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:00.282+0000 7f7eca55c640 1 -- 172.21.15.116:0/2994675483 wait complete. 2024-09-12T03:10:00.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:00 smithi116 ceph-mon[31410]: pgmap v548: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:10:00.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:00 smithi116 ceph-mon[31410]: overall HEALTH_OK 2024-09-12T03:10:00.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:00 smithi183 ceph-mon[35535]: pgmap v548: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:10:00.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:00 smithi183 ceph-mon[35535]: overall HEALTH_OK 2024-09-12T03:10:00.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:00 smithi177 ceph-mon[35305]: pgmap v548: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:10:00.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:00 smithi177 ceph-mon[35305]: overall HEALTH_OK 2024-09-12T03:10:01.023 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:10:01.270 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:10:01.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.770+0000 7f4c6ec55640 1 -- 172.21.15.116:0/1085047728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c68106f10 msgr2=0x7f4c6810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:01.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.770+0000 7f4c6ec55640 1 --2- 172.21.15.116:0/1085047728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c68106f10 0x7f4c6810ea90 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f4c60009ca0 tx=0x7f4c6002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.770+0000 7f4c6ec55640 1 -- 172.21.15.116:0/1085047728 shutdown_connections 2024-09-12T03:10:01.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.770+0000 7f4c6ec55640 1 --2- 172.21.15.116:0/1085047728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c68106f10 0x7f4c6810ea90 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.770+0000 7f4c6ec55640 1 --2- 172.21.15.116:0/1085047728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c68102790 0x7f4c681067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.770+0000 7f4c6ec55640 1 --2- 172.21.15.116:0/1085047728 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c68101de0 0x7f4c681021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.770+0000 7f4c6ec55640 1 -- 172.21.15.116:0/1085047728 >> 172.21.15.116:0/1085047728 conn(0x7f4c680fd8c0 msgr2=0x7f4c680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:01.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.770+0000 7f4c6ec55640 1 -- 172.21.15.116:0/1085047728 shutdown_connections 2024-09-12T03:10:01.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.770+0000 7f4c6ec55640 1 -- 172.21.15.116:0/1085047728 wait complete. 2024-09-12T03:10:01.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.771+0000 7f4c6ec55640 1 Processor -- start 2024-09-12T03:10:01.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.771+0000 7f4c6ec55640 1 -- start start 2024-09-12T03:10:01.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.771+0000 7f4c6ec55640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c68101de0 0x7f4c681985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.771+0000 7f4c6ec55640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c68102790 0x7f4c68198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.771+0000 7f4c6ec55640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c68106f10 0x7f4c6819cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.771+0000 7f4c6ec55640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4c68112010 con 0x7f4c68106f10 2024-09-12T03:10:01.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.771+0000 7f4c6ec55640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4c68111e90 con 0x7f4c68102790 2024-09-12T03:10:01.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.771+0000 7f4c6ec55640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c68112190 con 0x7f4c68101de0 2024-09-12T03:10:01.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.771+0000 7f4c6d1f2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c68106f10 0x7f4c6819cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.771+0000 7f4c6d1f2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c68106f10 0x7f4c6819cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58840/0 (socket says 172.21.15.116:58840) 2024-09-12T03:10:01.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.771+0000 7f4c6d1f2640 1 -- 172.21.15.116:0/1146658055 learned_addr learned my addr 172.21.15.116:0/1146658055 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:01.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.771+0000 7f4c67fff640 1 --2- 172.21.15.116:0/1146658055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c68102790 0x7f4c68198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.771+0000 7f4c6c9f1640 1 --2- 172.21.15.116:0/1146658055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c68101de0 0x7f4c681985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.771+0000 7f4c6d1f2640 1 -- 172.21.15.116:0/1146658055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c68101de0 msgr2=0x7f4c681985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:01.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.771+0000 7f4c6d1f2640 1 --2- 172.21.15.116:0/1146658055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c68101de0 0x7f4c681985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.771+0000 7f4c6d1f2640 1 -- 172.21.15.116:0/1146658055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c68102790 msgr2=0x7f4c68198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:01.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.772+0000 7f4c6d1f2640 1 --2- 172.21.15.116:0/1146658055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c68102790 0x7f4c68198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.772+0000 7f4c6d1f2640 1 -- 172.21.15.116:0/1146658055 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c6819d5a0 con 0x7f4c68106f10 2024-09-12T03:10:01.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.772+0000 7f4c6c9f1640 1 --2- 172.21.15.116:0/1146658055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c68101de0 0x7f4c681985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:01.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.772+0000 7f4c6d1f2640 1 --2- 172.21.15.116:0/1146658055 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c68106f10 0x7f4c6819cec0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4c60031a60 tx=0x7f4c60031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:01.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.772+0000 7f4c65ffb640 1 -- 172.21.15.116:0/1146658055 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c60003c30 con 0x7f4c68106f10 2024-09-12T03:10:01.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.772+0000 7f4c6ec55640 1 -- 172.21.15.116:0/1146658055 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c6819d830 con 0x7f4c68106f10 2024-09-12T03:10:01.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.772+0000 7f4c65ffb640 1 -- 172.21.15.116:0/1146658055 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c60003dd0 con 0x7f4c68106f10 2024-09-12T03:10:01.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.772+0000 7f4c65ffb640 1 -- 172.21.15.116:0/1146658055 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c600047e0 con 0x7f4c68106f10 2024-09-12T03:10:01.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.772+0000 7f4c6ec55640 1 -- 172.21.15.116:0/1146658055 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c681a5170 con 0x7f4c68106f10 2024-09-12T03:10:01.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.774+0000 7f4c65ffb640 1 -- 172.21.15.116:0/1146658055 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4c6002f790 con 0x7f4c68106f10 2024-09-12T03:10:01.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.775+0000 7f4c7102e740 1 Processor -- start 2024-09-12T03:10:01.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.775+0000 7f4c7102e740 1 -- start start 2024-09-12T03:10:01.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.775+0000 7f4c7102e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d2656b40 0x55c3d2655090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.775+0000 7f4c7102e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3d26555d0 0x55c3d2655a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.775+0000 7f4c7102e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c3d2660370 0x55c3d2662760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.775+0000 7f4c7102e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c3d25bb2f0 con 0x55c3d2660370 2024-09-12T03:10:01.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.775+0000 7f4c7102e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c3d2562cf0 con 0x55c3d2656b40 2024-09-12T03:10:01.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.775+0000 7f4c7102e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c3d25b72b0 con 0x55c3d26555d0 2024-09-12T03:10:01.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.775+0000 7f4c67fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c3d2660370 0x55c3d2662760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.775+0000 7f4c65ffb640 1 --2- 172.21.15.116:0/1146658055 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4c580a2b80 0x7f4c580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.775+0000 7f4c67fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c3d2660370 0x55c3d2662760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58848/0 (socket says 172.21.15.116:58848) 2024-09-12T03:10:01.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.775+0000 7f4c67fff640 1 -- 172.21.15.116:0/3549875190 learned_addr learned my addr 172.21.15.116:0/3549875190 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:01.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.775+0000 7f4c6c9f1640 1 --2- 172.21.15.116:0/3549875190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3d26555d0 0x55c3d2655a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.775+0000 7f4c67fff640 1 --2- 172.21.15.116:0/3549875190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d2656b40 0x55c3d2655090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.775+0000 7f4c6d1f2640 1 --2- 172.21.15.116:0/1146658055 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4c580a2b80 0x7f4c580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.775+0000 7f4c65ffb640 1 -- 172.21.15.116:0/1146658055 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f4c600f0550 con 0x7f4c68106f10 2024-09-12T03:10:01.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.775+0000 7f4c67fff640 1 -- 172.21.15.116:0/3549875190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3d26555d0 msgr2=0x55c3d2655a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:01.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.775+0000 7f4c67fff640 1 --2- 172.21.15.116:0/3549875190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3d26555d0 0x55c3d2655a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.775+0000 7f4c67fff640 1 -- 172.21.15.116:0/3549875190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d2656b40 msgr2=0x55c3d2655090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:01.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.775+0000 7f4c67fff640 1 --2- 172.21.15.116:0/3549875190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d2656b40 0x55c3d2655090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.775+0000 7f4c67fff640 1 -- 172.21.15.116:0/3549875190 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3d25473a0 con 0x55c3d2660370 2024-09-12T03:10:01.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.776+0000 7f4c67fff640 1 --2- 172.21.15.116:0/3549875190 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c3d2660370 0x55c3d2662760 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f4c54016530 tx=0x7f4c5400a2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3fec6fd47a05d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:01.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.776+0000 7f4c6d1f2640 1 --2- 172.21.15.116:0/1146658055 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4c580a2b80 0x7f4c580a5040 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4c600b91e0 tx=0x7f4c60087950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:01.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.776+0000 7f4c52ffd640 1 -- 172.21.15.116:0/3549875190 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c54018430 con 0x55c3d2660370 2024-09-12T03:10:01.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.776+0000 7f4c52ffd640 1 -- 172.21.15.116:0/3549875190 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c54018ab0 con 0x55c3d2660370 2024-09-12T03:10:01.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.776+0000 7f4c52ffd640 1 -- 172.21.15.116:0/3549875190 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c540179d0 con 0x55c3d2660370 2024-09-12T03:10:01.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.776+0000 7f4c7102e740 1 -- 172.21.15.116:0/3549875190 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c3d2660370 msgr2=0x55c3d2662760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:01.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.776+0000 7f4c7102e740 1 --2- 172.21.15.116:0/3549875190 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c3d2660370 0x55c3d2662760 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f4c54016530 tx=0x7f4c5400a2b0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.777+0000 7f4c7102e740 1 -- 172.21.15.116:0/3549875190 shutdown_connections 2024-09-12T03:10:01.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.777+0000 7f4c7102e740 1 --2- 172.21.15.116:0/3549875190 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c3d2660370 0x55c3d2662760 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.777+0000 7f4c7102e740 1 --2- 172.21.15.116:0/3549875190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3d26555d0 0x55c3d2655a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.777+0000 7f4c7102e740 1 --2- 172.21.15.116:0/3549875190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d2656b40 0x55c3d2655090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.777+0000 7f4c7102e740 1 -- 172.21.15.116:0/3549875190 >> 172.21.15.116:0/3549875190 conn(0x55c3d261e310 msgr2=0x55c3d2651c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:01.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.777+0000 7f4c7102e740 1 -- 172.21.15.116:0/3549875190 shutdown_connections 2024-09-12T03:10:01.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.777+0000 7f4c7102e740 1 -- 172.21.15.116:0/3549875190 wait complete. 2024-09-12T03:10:01.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.777+0000 7f4c7102e740 1 -- start start 2024-09-12T03:10:01.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.777+0000 7f4c7102e740 1 Processor -- start 2024-09-12T03:10:01.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.777+0000 7f4c7102e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d26555d0 0x55c3d265fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.777+0000 7f4c7102e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3d2656b40 0x55c3d265d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.777+0000 7f4c7102e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c3d2660370 0x55c3d265d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.777+0000 7f4c7102e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c3d2662fb0 con 0x55c3d2660370 2024-09-12T03:10:01.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.777+0000 7f4c7102e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c3d2576060 con 0x55c3d26555d0 2024-09-12T03:10:01.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.777+0000 7f4c7102e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c3d2663130 con 0x55c3d2656b40 2024-09-12T03:10:01.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.777+0000 7f4c7102e740 10 client.0 _ll_register_callbacks cb 0x55c3d22c98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:01.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.778+0000 7f4c67fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c3d2660370 0x55c3d265d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.778+0000 7f4c67fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c3d2660370 0x55c3d265d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58854/0 (socket says 172.21.15.116:58854) 2024-09-12T03:10:01.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.778+0000 7f4c67fff640 1 -- 172.21.15.116:0/3441169982 learned_addr learned my addr 172.21.15.116:0/3441169982 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:01.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.778+0000 7f4c67fff640 1 --2- 172.21.15.116:0/3441169982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d26555d0 0x55c3d265fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.778+0000 7f4c6c9f1640 1 --2- 172.21.15.116:0/3441169982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3d2656b40 0x55c3d265d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.778+0000 7f4c67fff640 1 -- 172.21.15.116:0/3441169982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3d2656b40 msgr2=0x55c3d265d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:01.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.778+0000 7f4c67fff640 1 --2- 172.21.15.116:0/3441169982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3d2656b40 0x55c3d265d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.778+0000 7f4c67fff640 1 -- 172.21.15.116:0/3441169982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d26555d0 msgr2=0x55c3d265fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:01.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.778+0000 7f4c67fff640 1 --2- 172.21.15.116:0/3441169982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d26555d0 0x55c3d265fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.778+0000 7f4c67fff640 1 -- 172.21.15.116:0/3441169982 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3d232c3c0 con 0x55c3d2660370 2024-09-12T03:10:01.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.778+0000 7f4c67fff640 1 --2- 172.21.15.116:0/3441169982 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c3d2660370 0x55c3d265d610 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400a250 tx=0x7f4c540175b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5667cb77172014a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:01.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.778+0000 7f4c47fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:01.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.778+0000 7f4c47fff640 1 -- 172.21.15.116:0/3441169982 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5402b060 con 0x55c3d2660370 2024-09-12T03:10:01.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.778+0000 7f4c7102e740 1 -- 172.21.15.116:0/3441169982 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c54018ab0 con 0x55c3d2660370 2024-09-12T03:10:01.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.778+0000 7f4c47fff640 1 -- 172.21.15.116:0/3441169982 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f4c54017750 con 0x55c3d2660370 2024-09-12T03:10:01.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.778+0000 7f4c47fff640 1 -- 172.21.15.116:0/3441169982 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4c54017950 con 0x55c3d2660370 2024-09-12T03:10:01.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.778+0000 7f4c7102e740 10 client.17106 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:01.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.778+0000 7f4c7102e740 1 -- 172.21.15.116:0/3441169982 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c3d266ea50 con 0x55c3d2660370 2024-09-12T03:10:01.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.778+0000 7f4c47fff640 1 -- 172.21.15.116:0/3441169982 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c54018ab0 con 0x55c3d2660370 2024-09-12T03:10:01.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.779+0000 7f4c47fff640 1 -- 172.21.15.116:0/3441169982 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4c54012050 con 0x55c3d2660370 2024-09-12T03:10:01.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.779+0000 7f4c7102e740 10 client.17106 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:01.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.779+0000 7f4c7102e740 10 client.17106 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:10:01.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.779+0000 7f4c7102e740 20 client.17106 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:01.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.779+0000 7f4c7102e740 1 --2- 172.21.15.116:0/3441169982 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c3d266fe60 0x55c3d2672250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.779+0000 7f4c7102e740 4 client.17106 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:10:01.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.779+0000 7f4c7102e740 1 -- 172.21.15.116:0/3441169982 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3d25c8860 con 0x55c3d266fe60 2024-09-12T03:10:01.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.779+0000 7f4c6c9f1640 1 --2- 172.21.15.116:0/3441169982 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c3d266fe60 0x55c3d2672250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.779+0000 7f4c6c9f1640 1 --2- 172.21.15.116:0/3441169982 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c3d266fe60 0x55c3d2672250 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=36b5ebcbe8404863 server_cookie=99ee860d8d8e11d7 in_seq=0 out_seq=0 2024-09-12T03:10:01.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.779+0000 7f4c47fff640 10 client.17106 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:10:01.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.780+0000 7f4c47fff640 1 -- 172.21.15.116:0/3441169982 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c3d25c8860 con 0x55c3d266fe60 2024-09-12T03:10:01.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.780+0000 7f4c47fff640 10 client.17106 handle_command_reply: tid=0 2024-09-12T03:10:01.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.780+0000 7f4c7102e740 1 client.17106 shutdown 2024-09-12T03:10:01.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.780+0000 7f4c7102e740 1 -- 172.21.15.116:0/3441169982 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c3d2660370 msgr2=0x55c3d265d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:01.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.780+0000 7f4c7102e740 1 --2- 172.21.15.116:0/3441169982 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c3d2660370 0x55c3d265d610 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400a250 tx=0x7f4c540175b0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.780+0000 7f4c7102e740 1 -- 172.21.15.116:0/3441169982 shutdown_connections 2024-09-12T03:10:01.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.780+0000 7f4c7102e740 1 --2- 172.21.15.116:0/3441169982 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c3d266fe60 0x55c3d2672250 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.780+0000 7f4c7102e740 1 --2- 172.21.15.116:0/3441169982 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c3d2660370 0x55c3d265d610 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.780+0000 7f4c7102e740 1 --2- 172.21.15.116:0/3441169982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3d2656b40 0x55c3d265d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.780+0000 7f4c7102e740 1 --2- 172.21.15.116:0/3441169982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d26555d0 0x55c3d265fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.780+0000 7f4c47fff640 0 client.17106 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:10:01.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.780+0000 7f4c7102e740 1 -- 172.21.15.116:0/3441169982 >> 172.21.15.116:0/3441169982 conn(0x55c3d261e310 msgr2=0x55c3d2657960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:01.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.780+0000 7f4c7102e740 1 -- 172.21.15.116:0/3441169982 shutdown_connections 2024-09-12T03:10:01.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.780+0000 7f4c7102e740 1 -- 172.21.15.116:0/3441169982 wait complete. 2024-09-12T03:10:01.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.781+0000 7f4c7102e740 20 client.17106 trim_cache size 0 max 16384 2024-09-12T03:10:01.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.793+0000 7f4c7102e740 1 Processor -- start 2024-09-12T03:10:01.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.793+0000 7f4c7102e740 1 -- start start 2024-09-12T03:10:01.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.794+0000 7f4c7102e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3d26555d0 0x55c3d265f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.794+0000 7f4c7102e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c3d2656b40 0x55c3d2658120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.794+0000 7f4c7102e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d2660370 0x55c3d2658660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.794+0000 7f4c7102e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681afeb0 con 0x55c3d2656b40 2024-09-12T03:10:01.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.794+0000 7f4c7102e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681ad780 con 0x55c3d2660370 2024-09-12T03:10:01.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.794+0000 7f4c7102e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681ab5d0 con 0x55c3d26555d0 2024-09-12T03:10:01.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.794+0000 7f4c6c9f1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c3d2656b40 0x55c3d2658120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.794+0000 7f4c6c9f1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c3d2656b40 0x55c3d2658120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58856/0 (socket says 172.21.15.116:58856) 2024-09-12T03:10:01.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.794+0000 7f4c6c9f1640 1 -- 172.21.15.116:0/1428781318 learned_addr learned my addr 172.21.15.116:0/1428781318 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:01.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.794+0000 7f4c67fff640 1 --2- 172.21.15.116:0/1428781318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3d26555d0 0x55c3d265f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.794+0000 7f4c67fff640 1 --2- 172.21.15.116:0/1428781318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d2660370 0x55c3d2658660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.794+0000 7f4c6c9f1640 1 -- 172.21.15.116:0/1428781318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3d26555d0 msgr2=0x55c3d265f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:01.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.794+0000 7f4c6c9f1640 1 --2- 172.21.15.116:0/1428781318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3d26555d0 0x55c3d265f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.794+0000 7f4c6c9f1640 1 -- 172.21.15.116:0/1428781318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d2660370 msgr2=0x55c3d2658660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:01.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.794+0000 7f4c6c9f1640 1 --2- 172.21.15.116:0/1428781318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d2660370 0x55c3d2658660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.794+0000 7f4c6c9f1640 1 -- 172.21.15.116:0/1428781318 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3d26514f0 con 0x55c3d2656b40 2024-09-12T03:10:01.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.794+0000 7f4c6c9f1640 1 --2- 172.21.15.116:0/1428781318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c3d2656b40 0x55c3d2658120 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c000c00 tx=0x7f4c5c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b583c65ce71543 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:01.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.794+0000 7f4c527fc640 1 -- 172.21.15.116:0/1428781318 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c00ad40 con 0x55c3d2656b40 2024-09-12T03:10:01.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.794+0000 7f4c527fc640 1 -- 172.21.15.116:0/1428781318 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c5c020030 con 0x55c3d2656b40 2024-09-12T03:10:01.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.794+0000 7f4c527fc640 1 -- 172.21.15.116:0/1428781318 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c01c540 con 0x55c3d2656b40 2024-09-12T03:10:01.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.795+0000 7f4c7102e740 1 -- 172.21.15.116:0/1428781318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c3d2656b40 msgr2=0x55c3d2658120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:01.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.795+0000 7f4c7102e740 1 --2- 172.21.15.116:0/1428781318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c3d2656b40 0x55c3d2658120 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c000c00 tx=0x7f4c5c008040 comp rx=0 tx=0).stop 2024-09-12T03:10:01.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.795+0000 7f4c7102e740 1 -- 172.21.15.116:0/1428781318 shutdown_connections 2024-09-12T03:10:01.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.795+0000 7f4c7102e740 1 --2- 172.21.15.116:0/1428781318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d2660370 0x55c3d2658660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.795+0000 7f4c7102e740 1 --2- 172.21.15.116:0/1428781318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c3d2656b40 0x55c3d2658120 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.795+0000 7f4c7102e740 1 --2- 172.21.15.116:0/1428781318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3d26555d0 0x55c3d265f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.795+0000 7f4c7102e740 1 -- 172.21.15.116:0/1428781318 >> 172.21.15.116:0/1428781318 conn(0x55c3d261e310 msgr2=0x55c3d265c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:01.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.795+0000 7f4c7102e740 1 -- 172.21.15.116:0/1428781318 shutdown_connections 2024-09-12T03:10:01.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.795+0000 7f4c7102e740 1 -- 172.21.15.116:0/1428781318 wait complete. 2024-09-12T03:10:01.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.795+0000 7f4c7102e740 1 -- start start 2024-09-12T03:10:01.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.795+0000 7f4c7102e740 1 Processor -- start 2024-09-12T03:10:01.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.796+0000 7f4c7102e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c3d26555d0 0x55c3d26683c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.796+0000 7f4c7102e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d2656b40 0x55c3d2668900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.796+0000 7f4c7102e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3d2660370 0x55c3d2665390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.796+0000 7f4c7102e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681abc40 con 0x55c3d26555d0 2024-09-12T03:10:01.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.796+0000 7f4c7102e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681ad0b0 con 0x55c3d2656b40 2024-09-12T03:10:01.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.796+0000 7f4c7102e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c681ac310 con 0x55c3d2660370 2024-09-12T03:10:01.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.796+0000 7f4c7102e740 10 client.0 _ll_register_callbacks cb 0x7f4c5400ac60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:01.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.796+0000 7f4c67fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c3d26555d0 0x55c3d26683c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.796+0000 7f4c67fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c3d26555d0 0x55c3d26683c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58864/0 (socket says 172.21.15.116:58864) 2024-09-12T03:10:01.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.796+0000 7f4c67fff640 1 -- 172.21.15.116:0/1861478041 learned_addr learned my addr 172.21.15.116:0/1861478041 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:01.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.796+0000 7f4c6c9f1640 1 --2- 172.21.15.116:0/1861478041 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d2656b40 0x55c3d2668900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.796+0000 7f4c67fff640 1 --2- 172.21.15.116:0/1861478041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3d2660370 0x55c3d2665390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.796+0000 7f4c67fff640 1 -- 172.21.15.116:0/1861478041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3d2660370 msgr2=0x55c3d2665390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:01.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.796+0000 7f4c67fff640 1 --2- 172.21.15.116:0/1861478041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3d2660370 0x55c3d2665390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.796+0000 7f4c67fff640 1 -- 172.21.15.116:0/1861478041 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d2656b40 msgr2=0x55c3d2668900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:01.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.796+0000 7f4c67fff640 1 --2- 172.21.15.116:0/1861478041 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d2656b40 0x55c3d2668900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.796+0000 7f4c67fff640 1 -- 172.21.15.116:0/1861478041 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3d2668ff0 con 0x55c3d26555d0 2024-09-12T03:10:01.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.797+0000 7f4c67fff640 1 --2- 172.21.15.116:0/1861478041 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c3d26555d0 0x55c3d26683c0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x55c3d2652ce0 tx=0x7f4c540175b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781d55943b477de1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:01.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.797+0000 7f4c47fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:01.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.797+0000 7f4c47fff640 1 -- 172.21.15.116:0/1861478041 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c54036040 con 0x55c3d26555d0 2024-09-12T03:10:01.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.797+0000 7f4c47fff640 1 -- 172.21.15.116:0/1861478041 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f4c5400a380 con 0x55c3d26555d0 2024-09-12T03:10:01.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.797+0000 7f4c7102e740 1 -- 172.21.15.116:0/1861478041 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c5c020030 con 0x55c3d26555d0 2024-09-12T03:10:01.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.797+0000 7f4c47fff640 1 -- 172.21.15.116:0/1861478041 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4c5402b040 con 0x55c3d26555d0 2024-09-12T03:10:01.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.797+0000 7f4c7102e740 10 client.17112 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:01.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.797+0000 7f4c7102e740 1 -- 172.21.15.116:0/1861478041 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c3d26dafa0 con 0x55c3d26555d0 2024-09-12T03:10:01.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.797+0000 7f4c47fff640 1 -- 172.21.15.116:0/1861478041 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c5c020030 con 0x55c3d26555d0 2024-09-12T03:10:01.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.797+0000 7f4c47fff640 1 -- 172.21.15.116:0/1861478041 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4c54012040 con 0x55c3d26555d0 2024-09-12T03:10:01.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.797+0000 7f4c7102e740 10 client.17112 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:01.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.797+0000 7f4c7102e740 10 client.17112 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:10:01.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.797+0000 7f4c7102e740 20 client.17112 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:01.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.797+0000 7f4c7102e740 1 --2- 172.21.15.116:0/1861478041 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c3d26db6d0 0x55c3d26ddaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.797+0000 7f4c7102e740 4 client.17112 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:10:01.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.797+0000 7f4c7102e740 1 -- 172.21.15.116:0/1861478041 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c3d25c2d70 con 0x55c3d26db6d0 2024-09-12T03:10:01.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.798+0000 7f4c6c9f1640 1 --2- 172.21.15.116:0/1861478041 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c3d26db6d0 0x55c3d26ddaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.798+0000 7f4c47fff640 10 client.17112 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:10:01.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.798+0000 7f4c6c9f1640 1 --2- 172.21.15.116:0/1861478041 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c3d26db6d0 0x55c3d26ddaf0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b623ea97078a04d server_cookie=e7b9d7d0952bd399 in_seq=0 out_seq=0 2024-09-12T03:10:01.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.798+0000 7f4c47fff640 1 -- 172.21.15.116:0/1861478041 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c3d25c2d70 con 0x55c3d26db6d0 2024-09-12T03:10:01.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.798+0000 7f4c47fff640 10 client.17112 handle_command_reply: tid=0 2024-09-12T03:10:01.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.798+0000 7f4c7102e740 1 client.17112 shutdown 2024-09-12T03:10:01.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.798+0000 7f4c7102e740 1 -- 172.21.15.116:0/1861478041 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c3d26555d0 msgr2=0x55c3d26683c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:01.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.798+0000 7f4c7102e740 1 --2- 172.21.15.116:0/1861478041 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c3d26555d0 0x55c3d26683c0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x55c3d2652ce0 tx=0x7f4c540175b0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.798+0000 7f4c7102e740 1 -- 172.21.15.116:0/1861478041 shutdown_connections 2024-09-12T03:10:01.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.798+0000 7f4c7102e740 1 --2- 172.21.15.116:0/1861478041 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c3d26db6d0 0x55c3d26ddaf0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.798+0000 7f4c7102e740 1 --2- 172.21.15.116:0/1861478041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3d2660370 0x55c3d2665390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.798+0000 7f4c7102e740 1 --2- 172.21.15.116:0/1861478041 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3d2656b40 0x55c3d2668900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.798+0000 7f4c7102e740 1 --2- 172.21.15.116:0/1861478041 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c3d26555d0 0x55c3d26683c0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.798+0000 7f4c7102e740 1 -- 172.21.15.116:0/1861478041 >> 172.21.15.116:0/1861478041 conn(0x55c3d261e310 msgr2=0x55c3d265b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:01.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.798+0000 7f4c47fff640 0 client.17112 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:10:01.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.798+0000 7f4c7102e740 1 -- 172.21.15.116:0/1861478041 shutdown_connections 2024-09-12T03:10:01.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.798+0000 7f4c7102e740 1 -- 172.21.15.116:0/1861478041 wait complete. 2024-09-12T03:10:01.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.799+0000 7f4c7102e740 20 client.17112 trim_cache size 0 max 16384 2024-09-12T03:10:01.813 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:10:01.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.799+0000 7f4c6ec55640 1 -- 172.21.15.116:0/1146658055 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4c580a2b80 msgr2=0x7f4c580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:01.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.799+0000 7f4c6ec55640 1 --2- 172.21.15.116:0/1146658055 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4c580a2b80 0x7f4c580a5040 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4c600b91e0 tx=0x7f4c60087950 comp rx=0 tx=0).stop 2024-09-12T03:10:01.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.799+0000 7f4c6ec55640 1 -- 172.21.15.116:0/1146658055 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c68106f10 msgr2=0x7f4c6819cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:01.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.799+0000 7f4c6ec55640 1 --2- 172.21.15.116:0/1146658055 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c68106f10 0x7f4c6819cec0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4c60031a60 tx=0x7f4c60031a90 comp rx=0 tx=0).stop 2024-09-12T03:10:01.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.799+0000 7f4c6ec55640 1 -- 172.21.15.116:0/1146658055 shutdown_connections 2024-09-12T03:10:01.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.799+0000 7f4c6ec55640 1 --2- 172.21.15.116:0/1146658055 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4c580a2b80 0x7f4c580a5040 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.799+0000 7f4c6ec55640 1 --2- 172.21.15.116:0/1146658055 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c68106f10 0x7f4c6819cec0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.799+0000 7f4c6ec55640 1 --2- 172.21.15.116:0/1146658055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c68102790 0x7f4c68198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.799+0000 7f4c6ec55640 1 --2- 172.21.15.116:0/1146658055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c68101de0 0x7f4c681985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.799+0000 7f4c6ec55640 1 -- 172.21.15.116:0/1146658055 >> 172.21.15.116:0/1146658055 conn(0x7f4c680fd8c0 msgr2=0x7f4c680fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:01.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.799+0000 7f4c6ec55640 1 -- 172.21.15.116:0/1146658055 shutdown_connections 2024-09-12T03:10:01.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:01.799+0000 7f4c6ec55640 1 -- 172.21.15.116:0/1146658055 wait complete. 2024-09-12T03:10:02.572 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:10:02.595 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:02 smithi116 ceph-mon[31410]: pgmap v549: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-12T03:10:02.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:02 smithi183 ceph-mon[35535]: pgmap v549: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-12T03:10:02.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:02 smithi177 ceph-mon[35305]: pgmap v549: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-12T03:10:02.819 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:10:03.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.355+0000 7f71d3e93640 1 -- 172.21.15.116:0/2934447585 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71cc106f10 msgr2=0x7f71cc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:03.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.355+0000 7f71d3e93640 1 --2- 172.21.15.116:0/2934447585 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71cc106f10 0x7f71cc10ea90 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009ca0 tx=0x7f71c402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.355+0000 7f71d3e93640 1 -- 172.21.15.116:0/2934447585 shutdown_connections 2024-09-12T03:10:03.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.355+0000 7f71d3e93640 1 --2- 172.21.15.116:0/2934447585 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71cc106f10 0x7f71cc10ea90 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.355+0000 7f71d3e93640 1 --2- 172.21.15.116:0/2934447585 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71cc102790 0x7f71cc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.355+0000 7f71d3e93640 1 --2- 172.21.15.116:0/2934447585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71cc101de0 0x7f71cc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.355+0000 7f71d3e93640 1 -- 172.21.15.116:0/2934447585 >> 172.21.15.116:0/2934447585 conn(0x7f71cc0fd8c0 msgr2=0x7f71cc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:03.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.355+0000 7f71d3e93640 1 -- 172.21.15.116:0/2934447585 shutdown_connections 2024-09-12T03:10:03.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.355+0000 7f71d3e93640 1 -- 172.21.15.116:0/2934447585 wait complete. 2024-09-12T03:10:03.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.356+0000 7f71d3e93640 1 Processor -- start 2024-09-12T03:10:03.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.356+0000 7f71d3e93640 1 -- start start 2024-09-12T03:10:03.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.356+0000 7f71d3e93640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71cc101de0 0x7f71cc198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:03.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.356+0000 7f71d3e93640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71cc102790 0x7f71cc198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:03.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.356+0000 7f71d3e93640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71cc106f10 0x7f71cc19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.356+0000 7f71d3e93640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc111f90 con 0x7f71cc102790 2024-09-12T03:10:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.356+0000 7f71d3e93640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc111e10 con 0x7f71cc101de0 2024-09-12T03:10:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.356+0000 7f71d3e93640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc112110 con 0x7f71cc106f10 2024-09-12T03:10:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.356+0000 7f71d142e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71cc102790 0x7f71cc198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.356+0000 7f71d142e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71cc102790 0x7f71cc198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58886/0 (socket says 172.21.15.116:58886) 2024-09-12T03:10:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.356+0000 7f71d142e640 1 -- 172.21.15.116:0/1849194490 learned_addr learned my addr 172.21.15.116:0/1849194490 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.356+0000 7f71d2430640 1 --2- 172.21.15.116:0/1849194490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71cc106f10 0x7f71cc19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.356+0000 7f71d1c2f640 1 --2- 172.21.15.116:0/1849194490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71cc101de0 0x7f71cc198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.357+0000 7f71d142e640 1 -- 172.21.15.116:0/1849194490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71cc106f10 msgr2=0x7f71cc19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.357+0000 7f71d142e640 1 --2- 172.21.15.116:0/1849194490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71cc106f10 0x7f71cc19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.357+0000 7f71d142e640 1 -- 172.21.15.116:0/1849194490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71cc101de0 msgr2=0x7f71cc198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.357+0000 7f71d142e640 1 --2- 172.21.15.116:0/1849194490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71cc101de0 0x7f71cc198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.357+0000 7f71d142e640 1 -- 172.21.15.116:0/1849194490 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71cc19d3c0 con 0x7f71cc102790 2024-09-12T03:10:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.357+0000 7f71d2430640 1 --2- 172.21.15.116:0/1849194490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71cc106f10 0x7f71cc19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:10:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.357+0000 7f71d1c2f640 1 --2- 172.21.15.116:0/1849194490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71cc101de0 0x7f71cc198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.357+0000 7f71d142e640 1 --2- 172.21.15.116:0/1849194490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71cc102790 0x7f71cc198950 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00b4a0 tx=0x7f71bc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.357+0000 7f71c2ffd640 1 -- 172.21.15.116:0/1849194490 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc014070 con 0x7f71cc102790 2024-09-12T03:10:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.357+0000 7f71d3e93640 1 -- 172.21.15.116:0/1849194490 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71cc19d6b0 con 0x7f71cc102790 2024-09-12T03:10:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.357+0000 7f71c2ffd640 1 -- 172.21.15.116:0/1849194490 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71bc010af0 con 0x7f71cc102790 2024-09-12T03:10:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.357+0000 7f71c2ffd640 1 -- 172.21.15.116:0/1849194490 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc017d10 con 0x7f71cc102790 2024-09-12T03:10:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.357+0000 7f71d3e93640 1 -- 172.21.15.116:0/1849194490 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71cc1a4f90 con 0x7f71cc102790 2024-09-12T03:10:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.359+0000 7f71c2ffd640 1 -- 172.21.15.116:0/1849194490 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f71bc017070 con 0x7f71cc102790 2024-09-12T03:10:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.360+0000 7f71d626c740 1 Processor -- start 2024-09-12T03:10:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.360+0000 7f71d626c740 1 -- start start 2024-09-12T03:10:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.360+0000 7f71c2ffd640 1 --2- 172.21.15.116:0/1849194490 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f71b80a2b80 0x7f71b80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:03.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.360+0000 7f71d626c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cbfcabb40 0x560cbfcaa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:03.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.360+0000 7f71d626c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cbfcaa5d0 0x560cbfcaaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:03.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.360+0000 7f71d626c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560cbfcb5370 0x560cbfcb7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:03.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.360+0000 7f71d626c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560cbfc102f0 con 0x560cbfcb5370 2024-09-12T03:10:03.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.360+0000 7f71d626c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560cbfbb7cf0 con 0x560cbfcaa5d0 2024-09-12T03:10:03.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.360+0000 7f71d626c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560cbfc0c2b0 con 0x560cbfcabb40 2024-09-12T03:10:03.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.360+0000 7f71d2430640 1 --2- 172.21.15.116:0/1849194490 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f71b80a2b80 0x7f71b80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:03.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.360+0000 7f71c2ffd640 1 -- 172.21.15.116:0/1849194490 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f71bc0ce4f0 con 0x7f71cc102790 2024-09-12T03:10:03.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.360+0000 7f71d2430640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560cbfcb5370 0x560cbfcb7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:03.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.360+0000 7f71d2430640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560cbfcb5370 0x560cbfcb7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58898/0 (socket says 172.21.15.116:58898) 2024-09-12T03:10:03.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.360+0000 7f71d2430640 1 -- 172.21.15.116:0/3926556153 learned_addr learned my addr 172.21.15.116:0/3926556153 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:03.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.360+0000 7f71d1c2f640 1 --2- 172.21.15.116:0/3926556153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cbfcabb40 0x560cbfcaa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:03.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.360+0000 7f71d142e640 1 --2- 172.21.15.116:0/3926556153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cbfcaa5d0 0x560cbfcaaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:03.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.361+0000 7f71d1c2f640 1 -- 172.21.15.116:0/3926556153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cbfcaa5d0 msgr2=0x560cbfcaaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:03.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.361+0000 7f71d1c2f640 1 --2- 172.21.15.116:0/3926556153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cbfcaa5d0 0x560cbfcaaa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.361+0000 7f71d1c2f640 1 -- 172.21.15.116:0/3926556153 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560cbfcb5370 msgr2=0x560cbfcb7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:03.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.361+0000 7f71d1c2f640 1 --2- 172.21.15.116:0/3926556153 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560cbfcb5370 0x560cbfcb7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.361+0000 7f71d1c2f640 1 -- 172.21.15.116:0/3926556153 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cbfb9c3a0 con 0x560cbfcabb40 2024-09-12T03:10:03.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.361+0000 7f71d2430640 1 --2- 172.21.15.116:0/3926556153 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560cbfcb5370 0x560cbfcb7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:10:03.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.361+0000 7f71d2430640 1 --2- 172.21.15.116:0/1849194490 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f71b80a2b80 0x7f71b80a5040 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f71c402f540 tx=0x7f71c4009880 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:03.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.361+0000 7f71d1c2f640 1 --2- 172.21.15.116:0/3926556153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cbfcabb40 0x560cbfcaa090 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f71c800bab0 tx=0x7f71c800bf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0a5a7fcb4db7676 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:03.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.361+0000 7f71b7fff640 1 -- 172.21.15.116:0/3926556153 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8017420 con 0x560cbfcabb40 2024-09-12T03:10:03.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.361+0000 7f71b7fff640 1 -- 172.21.15.116:0/3926556153 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71c8017aa0 con 0x560cbfcabb40 2024-09-12T03:10:03.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.361+0000 7f71b7fff640 1 -- 172.21.15.116:0/3926556153 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c800fb20 con 0x560cbfcabb40 2024-09-12T03:10:03.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.362+0000 7f71d626c740 1 -- 172.21.15.116:0/3926556153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cbfcabb40 msgr2=0x560cbfcaa090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:03.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.362+0000 7f71d626c740 1 --2- 172.21.15.116:0/3926556153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cbfcabb40 0x560cbfcaa090 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f71c800bab0 tx=0x7f71c800bf70 comp rx=0 tx=0).stop 2024-09-12T03:10:03.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.362+0000 7f71d626c740 1 -- 172.21.15.116:0/3926556153 shutdown_connections 2024-09-12T03:10:03.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.362+0000 7f71d626c740 1 --2- 172.21.15.116:0/3926556153 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560cbfcb5370 0x560cbfcb7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.362+0000 7f71d626c740 1 --2- 172.21.15.116:0/3926556153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cbfcaa5d0 0x560cbfcaaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.362+0000 7f71d626c740 1 --2- 172.21.15.116:0/3926556153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cbfcabb40 0x560cbfcaa090 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.362+0000 7f71d626c740 1 -- 172.21.15.116:0/3926556153 >> 172.21.15.116:0/3926556153 conn(0x560cbfc73310 msgr2=0x560cbfca6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:03.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.362+0000 7f71d626c740 1 -- 172.21.15.116:0/3926556153 shutdown_connections 2024-09-12T03:10:03.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.362+0000 7f71d626c740 1 -- 172.21.15.116:0/3926556153 wait complete. 2024-09-12T03:10:03.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.362+0000 7f71d626c740 1 -- start start 2024-09-12T03:10:03.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.362+0000 7f71d626c740 1 Processor -- start 2024-09-12T03:10:03.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.363+0000 7f71d626c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cbfcaa5d0 0x560cbfcb4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:03.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.363+0000 7f71d626c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cbfcabb40 0x560cbfcb1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:03.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.363+0000 7f71d626c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560cbfcb5370 0x560cbfcb24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:03.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.363+0000 7f71d626c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560cbfcb8070 con 0x560cbfcb5370 2024-09-12T03:10:03.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.363+0000 7f71d626c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560cbfbcb060 con 0x560cbfcabb40 2024-09-12T03:10:03.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.363+0000 7f71d626c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560cbfcb81f0 con 0x560cbfcaa5d0 2024-09-12T03:10:03.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.363+0000 7f71d626c740 10 client.0 _ll_register_callbacks cb 0x560cbf91e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:03.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.363+0000 7f71d2430640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560cbfcb5370 0x560cbfcb24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:03.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.363+0000 7f71d1c2f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cbfcaa5d0 0x560cbfcb4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:03.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.363+0000 7f71d2430640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560cbfcb5370 0x560cbfcb24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58910/0 (socket says 172.21.15.116:58910) 2024-09-12T03:10:03.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.363+0000 7f71d2430640 1 -- 172.21.15.116:0/3421466607 learned_addr learned my addr 172.21.15.116:0/3421466607 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:03.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.363+0000 7f71d1c2f640 1 --2- 172.21.15.116:0/3421466607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cbfcaa5d0 0x560cbfcb4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.116:58156/0 (socket says 172.21.15.116:58156) 2024-09-12T03:10:03.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.363+0000 7f71d142e640 1 --2- 172.21.15.116:0/3421466607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cbfcabb40 0x560cbfcb1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:03.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.363+0000 7f71d2430640 1 -- 172.21.15.116:0/3421466607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cbfcaa5d0 msgr2=0x560cbfcb4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:03.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.363+0000 7f71d2430640 1 --2- 172.21.15.116:0/3421466607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cbfcaa5d0 0x560cbfcb4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.363+0000 7f71d2430640 1 -- 172.21.15.116:0/3421466607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cbfcabb40 msgr2=0x560cbfcb1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:03.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.363+0000 7f71d2430640 1 --2- 172.21.15.116:0/3421466607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cbfcabb40 0x560cbfcb1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.363+0000 7f71d2430640 1 -- 172.21.15.116:0/3421466607 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cbf9813c0 con 0x560cbfcb5370 2024-09-12T03:10:03.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.363+0000 7f71d2430640 1 --2- 172.21.15.116:0/3421466607 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560cbfcb5370 0x560cbfcb24e0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f71c4005cc0 tx=0x7f71c4047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168a17c9ae17972a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:03.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.363+0000 7f71b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:03.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.364+0000 7f71b57fa640 1 -- 172.21.15.116:0/3421466607 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c4051070 con 0x560cbfcb5370 2024-09-12T03:10:03.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.364+0000 7f71b57fa640 1 -- 172.21.15.116:0/3421466607 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f71c402fd70 con 0x560cbfcb5370 2024-09-12T03:10:03.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.364+0000 7f71d626c740 1 -- 172.21.15.116:0/3421466607 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71c8017aa0 con 0x560cbfcb5370 2024-09-12T03:10:03.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.364+0000 7f71b57fa640 1 -- 172.21.15.116:0/3421466607 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f71c405c050 con 0x560cbfcb5370 2024-09-12T03:10:03.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.364+0000 7f71d626c740 10 client.17124 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:03.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.364+0000 7f71d626c740 1 -- 172.21.15.116:0/3421466607 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560cbfcc3a80 con 0x560cbfcb5370 2024-09-12T03:10:03.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.364+0000 7f71b57fa640 1 -- 172.21.15.116:0/3421466607 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71c8017aa0 con 0x560cbfcb5370 2024-09-12T03:10:03.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.364+0000 7f71b57fa640 1 -- 172.21.15.116:0/3421466607 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f71c404a070 con 0x560cbfcb5370 2024-09-12T03:10:03.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.364+0000 7f71d626c740 10 client.17124 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:03.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.364+0000 7f71d626c740 10 client.17124 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:10:03.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.364+0000 7f71d626c740 20 client.17124 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:03.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.364+0000 7f71d626c740 1 --2- 172.21.15.116:0/3421466607 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560cbfcc4e90 0x560cbfcc7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:03.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.364+0000 7f71d626c740 4 client.17124 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:10:03.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.364+0000 7f71d626c740 1 -- 172.21.15.116:0/3421466607 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cbfc1d860 con 0x560cbfcc4e90 2024-09-12T03:10:03.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.365+0000 7f71d1c2f640 1 --2- 172.21.15.116:0/3421466607 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560cbfcc4e90 0x560cbfcc7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:03.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.365+0000 7f71b57fa640 10 client.17124 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:03.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.365+0000 7f71d1c2f640 1 --2- 172.21.15.116:0/3421466607 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560cbfcc4e90 0x560cbfcc7330 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3ab8477b460782d server_cookie=21d1da9029851ac in_seq=0 out_seq=0 2024-09-12T03:10:03.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.366+0000 7f71b57fa640 1 -- 172.21.15.116:0/3421466607 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560cbfc1d860 con 0x560cbfcc4e90 2024-09-12T03:10:03.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.366+0000 7f71b57fa640 10 client.17124 handle_command_reply: tid=0 2024-09-12T03:10:03.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.366+0000 7f71d626c740 1 client.17124 shutdown 2024-09-12T03:10:03.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.366+0000 7f71d626c740 1 -- 172.21.15.116:0/3421466607 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560cbfcb5370 msgr2=0x560cbfcb24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:03.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.366+0000 7f71d626c740 1 --2- 172.21.15.116:0/3421466607 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560cbfcb5370 0x560cbfcb24e0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f71c4005cc0 tx=0x7f71c4047000 comp rx=0 tx=0).stop 2024-09-12T03:10:03.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.366+0000 7f71d626c740 1 -- 172.21.15.116:0/3421466607 shutdown_connections 2024-09-12T03:10:03.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.366+0000 7f71d626c740 1 --2- 172.21.15.116:0/3421466607 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560cbfcc4e90 0x560cbfcc7330 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.366+0000 7f71d626c740 1 --2- 172.21.15.116:0/3421466607 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560cbfcb5370 0x560cbfcb24e0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.366+0000 7f71d626c740 1 --2- 172.21.15.116:0/3421466607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cbfcabb40 0x560cbfcb1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.366+0000 7f71d626c740 1 --2- 172.21.15.116:0/3421466607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cbfcaa5d0 0x560cbfcb4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.366+0000 7f71b57fa640 0 client.17124 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.366+0000 7f71d626c740 1 -- 172.21.15.116:0/3421466607 >> 172.21.15.116:0/3421466607 conn(0x560cbfc73310 msgr2=0x560cbfcac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.366+0000 7f71d626c740 1 -- 172.21.15.116:0/3421466607 shutdown_connections 2024-09-12T03:10:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.366+0000 7f71d626c740 1 -- 172.21.15.116:0/3421466607 wait complete. 2024-09-12T03:10:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.366+0000 7f71d626c740 20 client.17124 trim_cache size 0 max 16384 2024-09-12T03:10:03.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.379+0000 7f71d626c740 1 Processor -- start 2024-09-12T03:10:03.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.379+0000 7f71d626c740 1 -- start start 2024-09-12T03:10:03.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.379+0000 7f71d626c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cbfcaa5d0 0x560cbfcb3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:03.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.379+0000 7f71d626c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560cbfcabb40 0x560cbfcad120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:03.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.379+0000 7f71d626c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cbfcb5370 0x560cbfcad660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:03.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.379+0000 7f71d626c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1afdb0 con 0x560cbfcabb40 2024-09-12T03:10:03.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.379+0000 7f71d626c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1ad680 con 0x560cbfcaa5d0 2024-09-12T03:10:03.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.379+0000 7f71d626c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1ab4d0 con 0x560cbfcb5370 2024-09-12T03:10:03.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.379+0000 7f71d142e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560cbfcabb40 0x560cbfcad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:03.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.379+0000 7f71d142e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560cbfcabb40 0x560cbfcad120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58926/0 (socket says 172.21.15.116:58926) 2024-09-12T03:10:03.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.379+0000 7f71d142e640 1 -- 172.21.15.116:0/1920963349 learned_addr learned my addr 172.21.15.116:0/1920963349 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:03.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.379+0000 7f71d1c2f640 1 --2- 172.21.15.116:0/1920963349 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cbfcaa5d0 0x560cbfcb3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:03.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.379+0000 7f71d142e640 1 -- 172.21.15.116:0/1920963349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cbfcb5370 msgr2=0x560cbfcad660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:03.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.379+0000 7f71d142e640 1 --2- 172.21.15.116:0/1920963349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cbfcb5370 0x560cbfcad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.379+0000 7f71d142e640 1 -- 172.21.15.116:0/1920963349 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cbfcaa5d0 msgr2=0x560cbfcb3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:03.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.379+0000 7f71d142e640 1 --2- 172.21.15.116:0/1920963349 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cbfcaa5d0 0x560cbfcb3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.379+0000 7f71d142e640 1 -- 172.21.15.116:0/1920963349 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cbfca7960 con 0x560cbfcabb40 2024-09-12T03:10:03.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.379+0000 7f71d142e640 1 --2- 172.21.15.116:0/1920963349 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560cbfcabb40 0x560cbfcad120 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0cbfc0 tx=0x7f71bc017730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d4a7e25764d9c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:03.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.380+0000 7f71b77fe640 1 -- 172.21.15.116:0/1920963349 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc01e040 con 0x560cbfcabb40 2024-09-12T03:10:03.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.380+0000 7f71b77fe640 1 -- 172.21.15.116:0/1920963349 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71bc0a3990 con 0x560cbfcabb40 2024-09-12T03:10:03.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.380+0000 7f71b77fe640 1 -- 172.21.15.116:0/1920963349 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc097390 con 0x560cbfcabb40 2024-09-12T03:10:03.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.380+0000 7f71d626c740 1 -- 172.21.15.116:0/1920963349 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560cbfcabb40 msgr2=0x560cbfcad120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:03.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.380+0000 7f71d626c740 1 --2- 172.21.15.116:0/1920963349 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560cbfcabb40 0x560cbfcad120 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0cbfc0 tx=0x7f71bc017730 comp rx=0 tx=0).stop 2024-09-12T03:10:03.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.380+0000 7f71d626c740 1 -- 172.21.15.116:0/1920963349 shutdown_connections 2024-09-12T03:10:03.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.380+0000 7f71d626c740 1 --2- 172.21.15.116:0/1920963349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cbfcb5370 0x560cbfcad660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.380+0000 7f71d626c740 1 --2- 172.21.15.116:0/1920963349 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560cbfcabb40 0x560cbfcad120 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.380+0000 7f71d626c740 1 --2- 172.21.15.116:0/1920963349 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cbfcaa5d0 0x560cbfcb3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.380+0000 7f71d626c740 1 -- 172.21.15.116:0/1920963349 >> 172.21.15.116:0/1920963349 conn(0x560cbfc73310 msgr2=0x560cbfcb37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:03.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.380+0000 7f71d626c740 1 -- 172.21.15.116:0/1920963349 shutdown_connections 2024-09-12T03:10:03.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.380+0000 7f71d626c740 1 -- 172.21.15.116:0/1920963349 wait complete. 2024-09-12T03:10:03.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.381+0000 7f71d626c740 1 -- start start 2024-09-12T03:10:03.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.381+0000 7f71d626c740 1 Processor -- start 2024-09-12T03:10:03.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.381+0000 7f71d626c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560cbfcaa5d0 0x560cbfcb9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:03.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.381+0000 7f71d626c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cbfcabb40 0x560cbfcb9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:03.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.381+0000 7f71d626c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cbfcb5370 0x560cbfcb9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:03.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.381+0000 7f71d626c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1abb40 con 0x560cbfcaa5d0 2024-09-12T03:10:03.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.381+0000 7f71d626c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1acfb0 con 0x560cbfcabb40 2024-09-12T03:10:03.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.381+0000 7f71d626c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1ac210 con 0x560cbfcb5370 2024-09-12T03:10:03.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.381+0000 7f71d626c740 10 client.0 _ll_register_callbacks cb 0x7f71c4031d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:03.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.381+0000 7f71d1c2f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560cbfcaa5d0 0x560cbfcb9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:03.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.381+0000 7f71d1c2f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560cbfcaa5d0 0x560cbfcb9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58928/0 (socket says 172.21.15.116:58928) 2024-09-12T03:10:03.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.381+0000 7f71d1c2f640 1 -- 172.21.15.116:0/2768291767 learned_addr learned my addr 172.21.15.116:0/2768291767 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:03.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.381+0000 7f71d142e640 1 --2- 172.21.15.116:0/2768291767 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cbfcabb40 0x560cbfcb9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:03.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.381+0000 7f71d2430640 1 --2- 172.21.15.116:0/2768291767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cbfcb5370 0x560cbfcb9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:03.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.382+0000 7f71d1c2f640 1 -- 172.21.15.116:0/2768291767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cbfcb5370 msgr2=0x560cbfcb9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:03.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.382+0000 7f71d1c2f640 1 --2- 172.21.15.116:0/2768291767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cbfcb5370 0x560cbfcb9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.382+0000 7f71d1c2f640 1 -- 172.21.15.116:0/2768291767 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cbfcabb40 msgr2=0x560cbfcb9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:03.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.382+0000 7f71d1c2f640 1 --2- 172.21.15.116:0/2768291767 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cbfcabb40 0x560cbfcb9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.382+0000 7f71d1c2f640 1 -- 172.21.15.116:0/2768291767 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cbfd2ed30 con 0x560cbfcaa5d0 2024-09-12T03:10:03.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.382+0000 7f71d1c2f640 1 --2- 172.21.15.116:0/2768291767 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560cbfcaa5d0 0x560cbfcb9300 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f71c800ba80 tx=0x7f71c800f5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f103cfc89e7dc1f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:03.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.382+0000 7f71b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:03.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.382+0000 7f71b57fa640 1 -- 172.21.15.116:0/2768291767 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8032060 con 0x560cbfcaa5d0 2024-09-12T03:10:03.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.382+0000 7f71b57fa640 1 -- 172.21.15.116:0/2768291767 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f71c801d040 con 0x560cbfcaa5d0 2024-09-12T03:10:03.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.382+0000 7f71d626c740 1 -- 172.21.15.116:0/2768291767 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71bc0a3990 con 0x560cbfcaa5d0 2024-09-12T03:10:03.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.382+0000 7f71b57fa640 1 -- 172.21.15.116:0/2768291767 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f71c800dd30 con 0x560cbfcaa5d0 2024-09-12T03:10:03.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.382+0000 7f71d626c740 10 client.17130 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:03.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.382+0000 7f71d626c740 1 -- 172.21.15.116:0/2768291767 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560cbfd2ffd0 con 0x560cbfcaa5d0 2024-09-12T03:10:03.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.382+0000 7f71b57fa640 1 -- 172.21.15.116:0/2768291767 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71bc0a3990 con 0x560cbfcaa5d0 2024-09-12T03:10:03.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.382+0000 7f71b57fa640 1 -- 172.21.15.116:0/2768291767 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f71c802c070 con 0x560cbfcaa5d0 2024-09-12T03:10:03.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.383+0000 7f71d626c740 10 client.17130 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:03.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.383+0000 7f71d626c740 10 client.17130 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:10:03.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.383+0000 7f71d626c740 20 client.17130 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:03.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.383+0000 7f71d626c740 1 --2- 172.21.15.116:0/2768291767 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560cbfd303c0 0x560cbfd327b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:03.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.383+0000 7f71d626c740 4 client.17130 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:10:03.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.383+0000 7f71d626c740 1 -- 172.21.15.116:0/2768291767 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560cbfc17d70 con 0x560cbfd303c0 2024-09-12T03:10:03.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.383+0000 7f71d142e640 1 --2- 172.21.15.116:0/2768291767 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560cbfd303c0 0x560cbfd327b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:03.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.383+0000 7f71b57fa640 10 client.17130 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:03.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.383+0000 7f71d142e640 1 --2- 172.21.15.116:0/2768291767 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560cbfd303c0 0x560cbfd327b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c76f11237be961c server_cookie=578f047f7ea5ebc4 in_seq=0 out_seq=0 2024-09-12T03:10:03.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.383+0000 7f71b57fa640 1 -- 172.21.15.116:0/2768291767 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560cbfc17d70 con 0x560cbfd303c0 2024-09-12T03:10:03.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.383+0000 7f71b57fa640 10 client.17130 handle_command_reply: tid=0 2024-09-12T03:10:03.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.383+0000 7f71d626c740 1 client.17130 shutdown 2024-09-12T03:10:03.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.383+0000 7f71d626c740 1 -- 172.21.15.116:0/2768291767 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560cbfcaa5d0 msgr2=0x560cbfcb9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:03.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.383+0000 7f71d626c740 1 --2- 172.21.15.116:0/2768291767 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560cbfcaa5d0 0x560cbfcb9300 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f71c800ba80 tx=0x7f71c800f5e0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.384+0000 7f71d626c740 1 -- 172.21.15.116:0/2768291767 shutdown_connections 2024-09-12T03:10:03.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.384+0000 7f71d626c740 1 --2- 172.21.15.116:0/2768291767 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560cbfd303c0 0x560cbfd327b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.384+0000 7f71d626c740 1 --2- 172.21.15.116:0/2768291767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cbfcb5370 0x560cbfcb9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.384+0000 7f71d626c740 1 --2- 172.21.15.116:0/2768291767 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cbfcabb40 0x560cbfcb9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.384+0000 7f71b57fa640 0 client.17130 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:03.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.384+0000 7f71d626c740 1 --2- 172.21.15.116:0/2768291767 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560cbfcaa5d0 0x560cbfcb9300 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.384+0000 7f71d626c740 1 -- 172.21.15.116:0/2768291767 >> 172.21.15.116:0/2768291767 conn(0x560cbfc73310 msgr2=0x560cbfcb0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:03.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.384+0000 7f71d626c740 1 -- 172.21.15.116:0/2768291767 shutdown_connections 2024-09-12T03:10:03.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.384+0000 7f71d626c740 1 -- 172.21.15.116:0/2768291767 wait complete. 2024-09-12T03:10:03.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.384+0000 7f71d626c740 20 client.17130 trim_cache size 0 max 16384 2024-09-12T03:10:03.395 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:10:03.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.384+0000 7f71d3e93640 1 -- 172.21.15.116:0/1849194490 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f71b80a2b80 msgr2=0x7f71b80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:03.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.384+0000 7f71d3e93640 1 --2- 172.21.15.116:0/1849194490 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f71b80a2b80 0x7f71b80a5040 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f71c402f540 tx=0x7f71c4009880 comp rx=0 tx=0).stop 2024-09-12T03:10:03.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.384+0000 7f71d3e93640 1 -- 172.21.15.116:0/1849194490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71cc102790 msgr2=0x7f71cc198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:03.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.384+0000 7f71d3e93640 1 --2- 172.21.15.116:0/1849194490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71cc102790 0x7f71cc198950 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00b4a0 tx=0x7f71bc00b960 comp rx=0 tx=0).stop 2024-09-12T03:10:03.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.384+0000 7f71d3e93640 1 -- 172.21.15.116:0/1849194490 shutdown_connections 2024-09-12T03:10:03.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.384+0000 7f71d3e93640 1 --2- 172.21.15.116:0/1849194490 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f71b80a2b80 0x7f71b80a5040 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.384+0000 7f71d3e93640 1 --2- 172.21.15.116:0/1849194490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71cc106f10 0x7f71cc19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.384+0000 7f71d3e93640 1 --2- 172.21.15.116:0/1849194490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71cc102790 0x7f71cc198950 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.384+0000 7f71d3e93640 1 --2- 172.21.15.116:0/1849194490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71cc101de0 0x7f71cc198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.384+0000 7f71d3e93640 1 -- 172.21.15.116:0/1849194490 >> 172.21.15.116:0/1849194490 conn(0x7f71cc0fd8c0 msgr2=0x7f71cc0ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:03.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.384+0000 7f71d3e93640 1 -- 172.21.15.116:0/1849194490 shutdown_connections 2024-09-12T03:10:03.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:03.384+0000 7f71d3e93640 1 -- 172.21.15.116:0/1849194490 wait complete. 2024-09-12T03:10:04.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:04 smithi183 ceph-mon[35535]: pgmap v550: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 9 op/s 2024-09-12T03:10:04.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:04 smithi177 ceph-mon[35305]: pgmap v550: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 9 op/s 2024-09-12T03:10:04.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:04 smithi116 ceph-mon[31410]: pgmap v550: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 9 op/s 2024-09-12T03:10:05.089 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:10:05.089 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:10:05.341 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:10:05.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.841+0000 7f025568c640 1 -- 172.21.15.116:0/2369313529 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0250102df0 msgr2=0x7f02501031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:05.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.841+0000 7f025568c640 1 --2- 172.21.15.116:0/2369313529 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0250102df0 0x7f02501031d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f0240009900 tx=0x7f024002a410 comp rx=0 tx=0).stop 2024-09-12T03:10:05.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.841+0000 7f025568c640 1 -- 172.21.15.116:0/2369313529 shutdown_connections 2024-09-12T03:10:05.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.841+0000 7f025568c640 1 --2- 172.21.15.116:0/2369313529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f025010c2f0 0x7f025010e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.841+0000 7f025568c640 1 --2- 172.21.15.116:0/2369313529 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02501037a0 0x7f025010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.841+0000 7f025568c640 1 --2- 172.21.15.116:0/2369313529 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0250102df0 0x7f02501031d0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.841+0000 7f025568c640 1 -- 172.21.15.116:0/2369313529 >> 172.21.15.116:0/2369313529 conn(0x7f02500fc7d0 msgr2=0x7f02500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:05.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.841+0000 7f025568c640 1 -- 172.21.15.116:0/2369313529 shutdown_connections 2024-09-12T03:10:05.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.842+0000 7f025568c640 1 -- 172.21.15.116:0/2369313529 wait complete. 2024-09-12T03:10:05.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.842+0000 7f025568c640 1 Processor -- start 2024-09-12T03:10:05.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.842+0000 7f025568c640 1 -- start start 2024-09-12T03:10:05.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.843+0000 7f025568c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0250102df0 0x7f0250198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:05.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.843+0000 7f025568c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02501037a0 0x7f0250198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:05.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.843+0000 7f025568c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f025010c2f0 0x7f025019cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:05.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.843+0000 7f025568c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0250112150 con 0x7f025010c2f0 2024-09-12T03:10:05.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.843+0000 7f025568c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0250111fd0 con 0x7f02501037a0 2024-09-12T03:10:05.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.843+0000 7f025568c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f02501122d0 con 0x7f0250102df0 2024-09-12T03:10:05.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.843+0000 7f024f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f025010c2f0 0x7f025019cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:05.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.843+0000 7f024f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f025010c2f0 0x7f025019cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58952/0 (socket says 172.21.15.116:58952) 2024-09-12T03:10:05.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.843+0000 7f024f7fe640 1 -- 172.21.15.116:0/1943565010 learned_addr learned my addr 172.21.15.116:0/1943565010 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:05.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.843+0000 7f024effd640 1 --2- 172.21.15.116:0/1943565010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0250102df0 0x7f0250198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:05.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.843+0000 7f024e7fc640 1 --2- 172.21.15.116:0/1943565010 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02501037a0 0x7f0250198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:05.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.843+0000 7f024f7fe640 1 -- 172.21.15.116:0/1943565010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0250102df0 msgr2=0x7f0250198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:05.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.843+0000 7f024f7fe640 1 --2- 172.21.15.116:0/1943565010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0250102df0 0x7f0250198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.843+0000 7f024f7fe640 1 -- 172.21.15.116:0/1943565010 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02501037a0 msgr2=0x7f0250198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:05.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.843+0000 7f024f7fe640 1 --2- 172.21.15.116:0/1943565010 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02501037a0 0x7f0250198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.843+0000 7f024f7fe640 1 -- 172.21.15.116:0/1943565010 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025019d5c0 con 0x7f025010c2f0 2024-09-12T03:10:05.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.843+0000 7f024e7fc640 1 --2- 172.21.15.116:0/1943565010 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02501037a0 0x7f0250198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:05.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.843+0000 7f024f7fe640 1 --2- 172.21.15.116:0/1943565010 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f025010c2f0 0x7f025019cee0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f024800cce0 tx=0x7f024800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:05.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.843+0000 7f0237fff640 1 -- 172.21.15.116:0/1943565010 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024800f450 con 0x7f025010c2f0 2024-09-12T03:10:05.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.844+0000 7f025568c640 1 -- 172.21.15.116:0/1943565010 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f025019d8b0 con 0x7f025010c2f0 2024-09-12T03:10:05.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.844+0000 7f0237fff640 1 -- 172.21.15.116:0/1943565010 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f024800fad0 con 0x7f025010c2f0 2024-09-12T03:10:05.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.844+0000 7f0237fff640 1 -- 172.21.15.116:0/1943565010 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248018c10 con 0x7f025010c2f0 2024-09-12T03:10:05.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.844+0000 7f025568c640 1 -- 172.21.15.116:0/1943565010 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02501a5190 con 0x7f025010c2f0 2024-09-12T03:10:05.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.846+0000 7f0237fff640 1 -- 172.21.15.116:0/1943565010 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f024800f5f0 con 0x7f025010c2f0 2024-09-12T03:10:05.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.846+0000 7f0257a65740 1 Processor -- start 2024-09-12T03:10:05.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.846+0000 7f0257a65740 1 -- start start 2024-09-12T03:10:05.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.846+0000 7f0257a65740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56392969ab60 0x56392969af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:05.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.846+0000 7f0257a65740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56392969b480 0x5639296a5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:05.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.846+0000 7f0257a65740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639296a5d40 0x5639296a8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:05.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.846+0000 7f0257a65740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x563929600280 con 0x5639296a5d40 2024-09-12T03:10:05.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.846+0000 7f0257a65740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5639295a7b10 con 0x56392969ab60 2024-09-12T03:10:05.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.846+0000 7f0257a65740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5639295fc240 con 0x56392969b480 2024-09-12T03:10:05.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.846+0000 7f0237fff640 1 --2- 172.21.15.116:0/1943565010 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f02380a2b80 0x7f02380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:05.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.846+0000 7f024e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639296a5d40 0x5639296a8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:05.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.846+0000 7f024e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639296a5d40 0x5639296a8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58962/0 (socket says 172.21.15.116:58962) 2024-09-12T03:10:05.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.846+0000 7f024e7fc640 1 -- 172.21.15.116:0/3662697806 learned_addr learned my addr 172.21.15.116:0/3662697806 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:05.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.847+0000 7f024effd640 1 --2- 172.21.15.116:0/3662697806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56392969b480 0x5639296a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:05.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.847+0000 7f024e7fc640 1 --2- 172.21.15.116:0/3662697806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56392969ab60 0x56392969af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:05.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.847+0000 7f024f7fe640 1 --2- 172.21.15.116:0/1943565010 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f02380a2b80 0x7f02380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:05.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.847+0000 7f0237fff640 1 -- 172.21.15.116:0/1943565010 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f02480cf700 con 0x7f025010c2f0 2024-09-12T03:10:05.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.847+0000 7f024e7fc640 1 -- 172.21.15.116:0/3662697806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56392969b480 msgr2=0x5639296a5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:05.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.847+0000 7f024e7fc640 1 --2- 172.21.15.116:0/3662697806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56392969b480 0x5639296a5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.847+0000 7f024e7fc640 1 -- 172.21.15.116:0/3662697806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56392969ab60 msgr2=0x56392969af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:05.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.847+0000 7f024e7fc640 1 --2- 172.21.15.116:0/3662697806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56392969ab60 0x56392969af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.847+0000 7f024e7fc640 1 -- 172.21.15.116:0/3662697806 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639295b41f0 con 0x5639296a5d40 2024-09-12T03:10:05.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.847+0000 7f024e7fc640 1 --2- 172.21.15.116:0/3662697806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639296a5d40 0x5639296a8130 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f024400cfc0 tx=0x7f024400c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24cecd0e76e2b7f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:05.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.847+0000 7f024f7fe640 1 --2- 172.21.15.116:0/1943565010 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f02380a2b80 0x7f02380a5040 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f02480983f0 tx=0x7f0248066b60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:05.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.847+0000 7f02357fa640 1 -- 172.21.15.116:0/3662697806 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024401d070 con 0x5639296a5d40 2024-09-12T03:10:05.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.847+0000 7f02357fa640 1 -- 172.21.15.116:0/3662697806 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0244018b50 con 0x5639296a5d40 2024-09-12T03:10:05.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.847+0000 7f02357fa640 1 -- 172.21.15.116:0/3662697806 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024400f040 con 0x5639296a5d40 2024-09-12T03:10:05.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.848+0000 7f0257a65740 1 -- 172.21.15.116:0/3662697806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639296a5d40 msgr2=0x5639296a8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:05.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.848+0000 7f0257a65740 1 --2- 172.21.15.116:0/3662697806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639296a5d40 0x5639296a8130 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f024400cfc0 tx=0x7f024400c1d0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.848+0000 7f0257a65740 1 -- 172.21.15.116:0/3662697806 shutdown_connections 2024-09-12T03:10:05.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.848+0000 7f0257a65740 1 --2- 172.21.15.116:0/3662697806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639296a5d40 0x5639296a8130 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.848+0000 7f0257a65740 1 --2- 172.21.15.116:0/3662697806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56392969b480 0x5639296a5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.848+0000 7f0257a65740 1 --2- 172.21.15.116:0/3662697806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56392969ab60 0x56392969af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.848+0000 7f0257a65740 1 -- 172.21.15.116:0/3662697806 >> 172.21.15.116:0/3662697806 conn(0x5639296970f0 msgr2=0x5639296974d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:05.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.848+0000 7f0257a65740 1 -- 172.21.15.116:0/3662697806 shutdown_connections 2024-09-12T03:10:05.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.848+0000 7f0257a65740 1 -- 172.21.15.116:0/3662697806 wait complete. 2024-09-12T03:10:05.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.848+0000 7f0257a65740 1 -- start start 2024-09-12T03:10:05.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.848+0000 7f0257a65740 1 Processor -- start 2024-09-12T03:10:05.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.849+0000 7f0257a65740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56392969ab60 0x5639296a2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:05.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.849+0000 7f0257a65740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56392969b480 0x5639296a31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:05.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.849+0000 7f0257a65740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639296a5d40 0x5639296a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:05.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.849+0000 7f0257a65740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5639296a8910 con 0x5639296a5d40 2024-09-12T03:10:05.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.849+0000 7f0257a65740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5639295baf10 con 0x56392969b480 2024-09-12T03:10:05.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.849+0000 7f0257a65740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5639296a8a90 con 0x56392969ab60 2024-09-12T03:10:05.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.849+0000 7f0257a65740 10 client.0 _ll_register_callbacks cb 0x563929583a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:05.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.849+0000 7f024e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639296a5d40 0x5639296a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:05.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.849+0000 7f024e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639296a5d40 0x5639296a3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58972/0 (socket says 172.21.15.116:58972) 2024-09-12T03:10:05.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.849+0000 7f024e7fc640 1 -- 172.21.15.116:0/115872901 learned_addr learned my addr 172.21.15.116:0/115872901 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:05.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.849+0000 7f024effd640 1 --2- 172.21.15.116:0/115872901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56392969b480 0x5639296a31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:05.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.849+0000 7f024e7fc640 1 --2- 172.21.15.116:0/115872901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56392969ab60 0x5639296a2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:05.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.849+0000 7f024e7fc640 1 -- 172.21.15.116:0/115872901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56392969ab60 msgr2=0x5639296a2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:05.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.849+0000 7f024e7fc640 1 --2- 172.21.15.116:0/115872901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56392969ab60 0x5639296a2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.849+0000 7f024e7fc640 1 -- 172.21.15.116:0/115872901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56392969b480 msgr2=0x5639296a31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:05.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.849+0000 7f024e7fc640 1 --2- 172.21.15.116:0/115872901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56392969b480 0x5639296a31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.849+0000 7f024e7fc640 1 -- 172.21.15.116:0/115872901 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639293716a0 con 0x5639296a5d40 2024-09-12T03:10:05.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.849+0000 7f024effd640 1 --2- 172.21.15.116:0/115872901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56392969b480 0x5639296a31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:10:05.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.849+0000 7f024e7fc640 1 --2- 172.21.15.116:0/115872901 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639296a5d40 0x5639296a3700 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f0244001eb0 tx=0x7f02440167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dcec890ce698353 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:05.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.849+0000 7f0232ff5640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:05.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.849+0000 7f0232ff5640 1 -- 172.21.15.116:0/115872901 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024401d040 con 0x5639296a5d40 2024-09-12T03:10:05.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.849+0000 7f0232ff5640 1 -- 172.21.15.116:0/115872901 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f0244016a90 con 0x5639296a5d40 2024-09-12T03:10:05.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.849+0000 7f0257a65740 1 -- 172.21.15.116:0/115872901 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0244018b50 con 0x5639296a5d40 2024-09-12T03:10:05.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.850+0000 7f0232ff5640 1 -- 172.21.15.116:0/115872901 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0244006bf0 con 0x5639296a5d40 2024-09-12T03:10:05.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.850+0000 7f0257a65740 10 client.17142 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:05.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.850+0000 7f0257a65740 1 -- 172.21.15.116:0/115872901 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5639296b4570 con 0x5639296a5d40 2024-09-12T03:10:05.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.850+0000 7f0232ff5640 1 -- 172.21.15.116:0/115872901 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0244018b50 con 0x5639296a5d40 2024-09-12T03:10:05.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.850+0000 7f0232ff5640 1 -- 172.21.15.116:0/115872901 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0244023040 con 0x5639296a5d40 2024-09-12T03:10:05.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.850+0000 7f0257a65740 10 client.17142 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:05.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.850+0000 7f0257a65740 10 client.17142 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:10:05.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.850+0000 7f0257a65740 20 client.17142 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:05.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.850+0000 7f0257a65740 1 --2- 172.21.15.116:0/115872901 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5639296b5910 0x5639296b7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:05.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.850+0000 7f0257a65740 4 client.17142 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:10:05.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.850+0000 7f0257a65740 1 -- 172.21.15.116:0/115872901 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56392960d7f0 con 0x5639296b5910 2024-09-12T03:10:05.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.850+0000 7f024effd640 1 --2- 172.21.15.116:0/115872901 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5639296b5910 0x5639296b7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:05.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.851+0000 7f0232ff5640 10 client.17142 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:05.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.851+0000 7f024effd640 1 --2- 172.21.15.116:0/115872901 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5639296b5910 0x5639296b7d00 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=335be5f1df3c05ef server_cookie=244858ee026c2750 in_seq=0 out_seq=0 2024-09-12T03:10:05.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.851+0000 7f0232ff5640 1 -- 172.21.15.116:0/115872901 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56392960d7f0 con 0x5639296b5910 2024-09-12T03:10:05.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.851+0000 7f0232ff5640 10 client.17142 handle_command_reply: tid=0 2024-09-12T03:10:05.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.851+0000 7f0257a65740 1 client.17142 shutdown 2024-09-12T03:10:05.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.851+0000 7f0257a65740 1 -- 172.21.15.116:0/115872901 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639296a5d40 msgr2=0x5639296a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:05.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.851+0000 7f0257a65740 1 --2- 172.21.15.116:0/115872901 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639296a5d40 0x5639296a3700 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f0244001eb0 tx=0x7f02440167b0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.852+0000 7f0257a65740 1 -- 172.21.15.116:0/115872901 shutdown_connections 2024-09-12T03:10:05.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.852+0000 7f0257a65740 1 --2- 172.21.15.116:0/115872901 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5639296b5910 0x5639296b7d00 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.852+0000 7f0257a65740 1 --2- 172.21.15.116:0/115872901 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639296a5d40 0x5639296a3700 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.852+0000 7f0257a65740 1 --2- 172.21.15.116:0/115872901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56392969b480 0x5639296a31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.852+0000 7f0232ff5640 0 client.17142 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:05.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.852+0000 7f0257a65740 1 --2- 172.21.15.116:0/115872901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56392969ab60 0x5639296a2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.852+0000 7f0257a65740 1 -- 172.21.15.116:0/115872901 >> 172.21.15.116:0/115872901 conn(0x5639296970f0 msgr2=0x56392969cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:05.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.852+0000 7f0257a65740 1 -- 172.21.15.116:0/115872901 shutdown_connections 2024-09-12T03:10:05.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.852+0000 7f0257a65740 1 -- 172.21.15.116:0/115872901 wait complete. 2024-09-12T03:10:05.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.852+0000 7f0257a65740 20 client.17142 trim_cache size 0 max 16384 2024-09-12T03:10:05.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.869+0000 7f0257a65740 1 Processor -- start 2024-09-12T03:10:05.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.869+0000 7f0257a65740 1 -- start start 2024-09-12T03:10:05.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.869+0000 7f0257a65740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56392969ab60 0x5639296a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:05.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.869+0000 7f0257a65740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56392969b480 0x563929696160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:05.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.869+0000 7f0257a65740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639296a5d40 0x5639296bad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:05.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.869+0000 7f0257a65740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f02501afc50 con 0x5639296a5d40 2024-09-12T03:10:05.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.869+0000 7f0257a65740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f02501ad520 con 0x56392969ab60 2024-09-12T03:10:05.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.869+0000 7f0257a65740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f02501ab390 con 0x56392969b480 2024-09-12T03:10:05.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.869+0000 7f024e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639296a5d40 0x5639296bad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:05.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.869+0000 7f024e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639296a5d40 0x5639296bad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58984/0 (socket says 172.21.15.116:58984) 2024-09-12T03:10:05.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.869+0000 7f024e7fc640 1 -- 172.21.15.116:0/392060817 learned_addr learned my addr 172.21.15.116:0/392060817 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:05.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.869+0000 7f024effd640 1 --2- 172.21.15.116:0/392060817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56392969b480 0x563929696160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:05.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.869+0000 7f024e7fc640 1 -- 172.21.15.116:0/392060817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56392969b480 msgr2=0x563929696160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:05.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.869+0000 7f024e7fc640 1 --2- 172.21.15.116:0/392060817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56392969b480 0x563929696160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.869+0000 7f024e7fc640 1 -- 172.21.15.116:0/392060817 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56392969ab60 msgr2=0x5639296a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:05.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.869+0000 7f024e7fc640 1 --2- 172.21.15.116:0/392060817 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56392969ab60 0x5639296a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.869+0000 7f024e7fc640 1 -- 172.21.15.116:0/392060817 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563929692830 con 0x5639296a5d40 2024-09-12T03:10:05.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.869+0000 7f024e7fc640 1 --2- 172.21.15.116:0/392060817 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639296a5d40 0x5639296bad20 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f0244006bc0 tx=0x7f0244024580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415f31871bddafad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:05.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.869+0000 7f0234ff9640 1 -- 172.21.15.116:0/392060817 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024401d040 con 0x5639296a5d40 2024-09-12T03:10:05.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.870+0000 7f0234ff9640 1 -- 172.21.15.116:0/392060817 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f024400a070 con 0x5639296a5d40 2024-09-12T03:10:05.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.870+0000 7f0234ff9640 1 -- 172.21.15.116:0/392060817 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244018a50 con 0x5639296a5d40 2024-09-12T03:10:05.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.870+0000 7f0257a65740 1 -- 172.21.15.116:0/392060817 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639296a5d40 msgr2=0x5639296bad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:05.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.870+0000 7f0257a65740 1 --2- 172.21.15.116:0/392060817 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639296a5d40 0x5639296bad20 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f0244006bc0 tx=0x7f0244024580 comp rx=0 tx=0).stop 2024-09-12T03:10:05.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.870+0000 7f0257a65740 1 -- 172.21.15.116:0/392060817 shutdown_connections 2024-09-12T03:10:05.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.870+0000 7f0257a65740 1 --2- 172.21.15.116:0/392060817 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639296a5d40 0x5639296bad20 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.870+0000 7f0257a65740 1 --2- 172.21.15.116:0/392060817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56392969b480 0x563929696160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.870+0000 7f0257a65740 1 --2- 172.21.15.116:0/392060817 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56392969ab60 0x5639296a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.870+0000 7f0257a65740 1 -- 172.21.15.116:0/392060817 >> 172.21.15.116:0/392060817 conn(0x5639296970f0 msgr2=0x563929695a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:05.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.870+0000 7f0257a65740 1 -- 172.21.15.116:0/392060817 shutdown_connections 2024-09-12T03:10:05.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.870+0000 7f0257a65740 1 -- 172.21.15.116:0/392060817 wait complete. 2024-09-12T03:10:05.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.870+0000 7f0257a65740 1 -- start start 2024-09-12T03:10:05.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.871+0000 7f0257a65740 1 Processor -- start 2024-09-12T03:10:05.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.871+0000 7f0257a65740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56392969ab60 0x5639296b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:05.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.871+0000 7f0257a65740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56392969b480 0x5639296b90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:05.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.871+0000 7f0257a65740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639296a5d40 0x5639296b95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:05.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.871+0000 7f0257a65740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f02501ab9e0 con 0x56392969ab60 2024-09-12T03:10:05.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.871+0000 7f0257a65740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f02501ace50 con 0x56392969b480 2024-09-12T03:10:05.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.871+0000 7f0257a65740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f02501ac0b0 con 0x5639296a5d40 2024-09-12T03:10:05.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.871+0000 7f0257a65740 10 client.0 _ll_register_callbacks cb 0x7f024401b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:05.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.871+0000 7f024e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56392969ab60 0x5639296b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:05.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.871+0000 7f024e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56392969ab60 0x5639296b8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58998/0 (socket says 172.21.15.116:58998) 2024-09-12T03:10:05.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.871+0000 7f024e7fc640 1 -- 172.21.15.116:0/1465694098 learned_addr learned my addr 172.21.15.116:0/1465694098 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:05.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.871+0000 7f024e7fc640 1 -- 172.21.15.116:0/1465694098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639296a5d40 msgr2=0x5639296b95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:05.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.871+0000 7f024effd640 1 --2- 172.21.15.116:0/1465694098 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56392969b480 0x5639296b90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:05.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.871+0000 7f024e7fc640 1 --2- 172.21.15.116:0/1465694098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639296a5d40 0x5639296b95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.871+0000 7f024e7fc640 1 -- 172.21.15.116:0/1465694098 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56392969b480 msgr2=0x5639296b90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:05.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.871+0000 7f024e7fc640 1 --2- 172.21.15.116:0/1465694098 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56392969b480 0x5639296b90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.871+0000 7f024e7fc640 1 -- 172.21.15.116:0/1465694098 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56392971f320 con 0x56392969ab60 2024-09-12T03:10:05.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.871+0000 7f024effd640 1 --2- 172.21.15.116:0/1465694098 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56392969b480 0x5639296b90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:05.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.871+0000 7f024e7fc640 1 --2- 172.21.15.116:0/1465694098 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56392969ab60 0x5639296b8b70 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f0244000c00 tx=0x7f024400c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1613b00ff4c96cec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:05.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.871+0000 7f0232ff5640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:05.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.871+0000 7f0232ff5640 1 -- 172.21.15.116:0/1465694098 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024401d040 con 0x56392969ab60 2024-09-12T03:10:05.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.872+0000 7f0232ff5640 1 -- 172.21.15.116:0/1465694098 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f024402d600 con 0x56392969ab60 2024-09-12T03:10:05.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.872+0000 7f0257a65740 1 -- 172.21.15.116:0/1465694098 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f024400a070 con 0x56392969ab60 2024-09-12T03:10:05.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.872+0000 7f0232ff5640 1 -- 172.21.15.116:0/1465694098 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0244040050 con 0x56392969ab60 2024-09-12T03:10:05.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.872+0000 7f0257a65740 10 client.17148 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:05.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.872+0000 7f0257a65740 1 -- 172.21.15.116:0/1465694098 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563929720640 con 0x56392969ab60 2024-09-12T03:10:05.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.872+0000 7f0232ff5640 1 -- 172.21.15.116:0/1465694098 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f024400a070 con 0x56392969ab60 2024-09-12T03:10:05.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.872+0000 7f0232ff5640 1 -- 172.21.15.116:0/1465694098 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0244015070 con 0x56392969ab60 2024-09-12T03:10:05.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.872+0000 7f0257a65740 10 client.17148 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:05.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.872+0000 7f0257a65740 10 client.17148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:10:05.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.872+0000 7f0257a65740 20 client.17148 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:05.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.872+0000 7f0257a65740 1 --2- 172.21.15.116:0/1465694098 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x563929720c40 0x563929723030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:05.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.872+0000 7f0257a65740 4 client.17148 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:10:05.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.872+0000 7f0257a65740 1 -- 172.21.15.116:0/1465694098 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563929607d00 con 0x563929720c40 2024-09-12T03:10:05.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.872+0000 7f024effd640 1 --2- 172.21.15.116:0/1465694098 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x563929720c40 0x563929723030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:05.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.873+0000 7f024effd640 1 --2- 172.21.15.116:0/1465694098 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x563929720c40 0x563929723030 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aee8ce29b0cd9519 server_cookie=6b5749b531f4a3ab in_seq=0 out_seq=0 2024-09-12T03:10:05.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.873+0000 7f0232ff5640 10 client.17148 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:05.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.874+0000 7f0232ff5640 1 -- 172.21.15.116:0/1465694098 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563929607d00 con 0x563929720c40 2024-09-12T03:10:05.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.874+0000 7f0232ff5640 10 client.17148 handle_command_reply: tid=0 2024-09-12T03:10:05.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.874+0000 7f0257a65740 1 client.17148 shutdown 2024-09-12T03:10:05.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.874+0000 7f0257a65740 1 -- 172.21.15.116:0/1465694098 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56392969ab60 msgr2=0x5639296b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:05.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.874+0000 7f0257a65740 1 --2- 172.21.15.116:0/1465694098 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56392969ab60 0x5639296b8b70 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f0244000c00 tx=0x7f024400c6b0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.874+0000 7f0257a65740 1 -- 172.21.15.116:0/1465694098 shutdown_connections 2024-09-12T03:10:05.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.874+0000 7f0257a65740 1 --2- 172.21.15.116:0/1465694098 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x563929720c40 0x563929723030 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.874+0000 7f0257a65740 1 --2- 172.21.15.116:0/1465694098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639296a5d40 0x5639296b95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.874+0000 7f0257a65740 1 --2- 172.21.15.116:0/1465694098 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56392969b480 0x5639296b90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.874+0000 7f0257a65740 1 --2- 172.21.15.116:0/1465694098 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56392969ab60 0x5639296b8b70 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.874+0000 7f0257a65740 1 -- 172.21.15.116:0/1465694098 >> 172.21.15.116:0/1465694098 conn(0x5639296970f0 msgr2=0x5639296b1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:05.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.874+0000 7f0232ff5640 0 client.17148 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:05.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.874+0000 7f0257a65740 1 -- 172.21.15.116:0/1465694098 shutdown_connections 2024-09-12T03:10:05.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.874+0000 7f0257a65740 1 -- 172.21.15.116:0/1465694098 wait complete. 2024-09-12T03:10:05.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.874+0000 7f0257a65740 20 client.17148 trim_cache size 0 max 16384 2024-09-12T03:10:05.886 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:10:05.887 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:10:05.887 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "b97c07c7-7361-4f4c-a3d2-3916e8dd20d3", 2024-09-12T03:10:05.887 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:10:05.887 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:10:05.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.875+0000 7f025568c640 1 -- 172.21.15.116:0/1943565010 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f02380a2b80 msgr2=0x7f02380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:05.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.875+0000 7f025568c640 1 --2- 172.21.15.116:0/1943565010 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f02380a2b80 0x7f02380a5040 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f02480983f0 tx=0x7f0248066b60 comp rx=0 tx=0).stop 2024-09-12T03:10:05.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.875+0000 7f025568c640 1 -- 172.21.15.116:0/1943565010 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f025010c2f0 msgr2=0x7f025019cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:05.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.875+0000 7f025568c640 1 --2- 172.21.15.116:0/1943565010 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f025010c2f0 0x7f025019cee0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f024800cce0 tx=0x7f024800a620 comp rx=0 tx=0).stop 2024-09-12T03:10:05.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.875+0000 7f025568c640 1 -- 172.21.15.116:0/1943565010 shutdown_connections 2024-09-12T03:10:05.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.875+0000 7f025568c640 1 --2- 172.21.15.116:0/1943565010 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f02380a2b80 0x7f02380a5040 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.875+0000 7f025568c640 1 --2- 172.21.15.116:0/1943565010 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f025010c2f0 0x7f025019cee0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.875+0000 7f025568c640 1 --2- 172.21.15.116:0/1943565010 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02501037a0 0x7f0250198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.875+0000 7f025568c640 1 --2- 172.21.15.116:0/1943565010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0250102df0 0x7f0250198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:05.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.875+0000 7f025568c640 1 -- 172.21.15.116:0/1943565010 >> 172.21.15.116:0/1943565010 conn(0x7f02500fc7d0 msgr2=0x7f02500fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:05.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.875+0000 7f025568c640 1 -- 172.21.15.116:0/1943565010 shutdown_connections 2024-09-12T03:10:05.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:05.875+0000 7f025568c640 1 -- 172.21.15.116:0/1943565010 wait complete. 2024-09-12T03:10:06.541 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:10:06.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:06 smithi183 ceph-mon[35535]: pgmap v551: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-09-12T03:10:06.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:06 smithi177 ceph-mon[35305]: pgmap v551: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-09-12T03:10:06.785 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:10:06.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:06 smithi116 ceph-mon[31410]: pgmap v551: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-09-12T03:10:07.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.313+0000 7f2207ce7640 1 -- 172.21.15.116:0/1692213190 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f220010c170 msgr2=0x7f220010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:07.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.313+0000 7f2207ce7640 1 --2- 172.21.15.116:0/1692213190 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f220010c170 0x7f220010e610 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f21f8009ca0 tx=0x7f21f802a950 comp rx=0 tx=0).stop 2024-09-12T03:10:07.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.313+0000 7f2207ce7640 1 -- 172.21.15.116:0/1692213190 shutdown_connections 2024-09-12T03:10:07.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.313+0000 7f2207ce7640 1 --2- 172.21.15.116:0/1692213190 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f220010c170 0x7f220010e610 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.313+0000 7f2207ce7640 1 --2- 172.21.15.116:0/1692213190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2200103660 0x7f220010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.313+0000 7f2207ce7640 1 --2- 172.21.15.116:0/1692213190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2200102d40 0x7f2200103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.313+0000 7f2207ce7640 1 -- 172.21.15.116:0/1692213190 >> 172.21.15.116:0/1692213190 conn(0x7f22000fc7a0 msgr2=0x7f22000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:07.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.313+0000 7f2207ce7640 1 -- 172.21.15.116:0/1692213190 shutdown_connections 2024-09-12T03:10:07.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.313+0000 7f2207ce7640 1 -- 172.21.15.116:0/1692213190 wait complete. 2024-09-12T03:10:07.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.314+0000 7f2207ce7640 1 Processor -- start 2024-09-12T03:10:07.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.314+0000 7f2207ce7640 1 -- start start 2024-09-12T03:10:07.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.314+0000 7f2207ce7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2200102d40 0x7f2200198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:07.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.314+0000 7f2207ce7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2200103660 0x7f2200198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:07.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.314+0000 7f2207ce7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f220010c170 0x7f220019cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:07.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.314+0000 7f2207ce7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2200112030 con 0x7f2200103660 2024-09-12T03:10:07.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.314+0000 7f2207ce7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2200111eb0 con 0x7f2200102d40 2024-09-12T03:10:07.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.314+0000 7f2207ce7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f22001121b0 con 0x7f220010c170 2024-09-12T03:10:07.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.314+0000 7f2205282640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2200103660 0x7f2200198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:07.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.314+0000 7f2205282640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2200103660 0x7f2200198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59006/0 (socket says 172.21.15.116:59006) 2024-09-12T03:10:07.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.314+0000 7f2205282640 1 -- 172.21.15.116:0/2927874325 learned_addr learned my addr 172.21.15.116:0/2927874325 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:07.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.314+0000 7f2205a83640 1 --2- 172.21.15.116:0/2927874325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2200102d40 0x7f2200198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:07.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.315+0000 7f2206284640 1 --2- 172.21.15.116:0/2927874325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f220010c170 0x7f220019cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:07.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.315+0000 7f2205282640 1 -- 172.21.15.116:0/2927874325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f220010c170 msgr2=0x7f220019cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:07.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.315+0000 7f2205282640 1 --2- 172.21.15.116:0/2927874325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f220010c170 0x7f220019cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.315+0000 7f2205282640 1 -- 172.21.15.116:0/2927874325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2200102d40 msgr2=0x7f2200198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:07.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.315+0000 7f2205282640 1 --2- 172.21.15.116:0/2927874325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2200102d40 0x7f2200198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.315+0000 7f2205282640 1 -- 172.21.15.116:0/2927874325 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f220019d5f0 con 0x7f2200103660 2024-09-12T03:10:07.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.315+0000 7f2205a83640 1 --2- 172.21.15.116:0/2927874325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2200102d40 0x7f2200198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:07.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.315+0000 7f2206284640 1 --2- 172.21.15.116:0/2927874325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f220010c170 0x7f220019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:07.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.315+0000 7f2205282640 1 --2- 172.21.15.116:0/2927874325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2200103660 0x7f2200198b80 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00b3f0 tx=0x7f21fc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:07.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.315+0000 7f21f6ffd640 1 -- 172.21.15.116:0/2927874325 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc015020 con 0x7f2200103660 2024-09-12T03:10:07.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.315+0000 7f21f6ffd640 1 -- 172.21.15.116:0/2927874325 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21fc011d10 con 0x7f2200103660 2024-09-12T03:10:07.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.315+0000 7f2207ce7640 1 -- 172.21.15.116:0/2927874325 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f220019d8e0 con 0x7f2200103660 2024-09-12T03:10:07.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.315+0000 7f21f6ffd640 1 -- 172.21.15.116:0/2927874325 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc00f2e0 con 0x7f2200103660 2024-09-12T03:10:07.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.315+0000 7f2207ce7640 1 -- 172.21.15.116:0/2927874325 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22001a51c0 con 0x7f2200103660 2024-09-12T03:10:07.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.317+0000 7f21f6ffd640 1 -- 172.21.15.116:0/2927874325 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f21fc018020 con 0x7f2200103660 2024-09-12T03:10:07.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.317+0000 7f220a0c0740 1 Processor -- start 2024-09-12T03:10:07.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.318+0000 7f220a0c0740 1 -- start start 2024-09-12T03:10:07.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.318+0000 7f220a0c0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3a20724a0 0x55b3a2072880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:07.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.318+0000 7f220a0c0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3a2072dc0 0x55b3a207d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:07.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.318+0000 7f220a0c0740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3a207d680 0x55b3a207fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:07.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.318+0000 7f220a0c0740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b3a1fd82e0 con 0x55b3a207d680 2024-09-12T03:10:07.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.318+0000 7f220a0c0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b3a1f7fce0 con 0x55b3a2072dc0 2024-09-12T03:10:07.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.318+0000 7f220a0c0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b3a1fd42a0 con 0x55b3a20724a0 2024-09-12T03:10:07.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.318+0000 7f2205282640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3a207d680 0x55b3a207fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:07.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.318+0000 7f2205282640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3a207d680 0x55b3a207fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59020/0 (socket says 172.21.15.116:59020) 2024-09-12T03:10:07.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.318+0000 7f2205282640 1 -- 172.21.15.116:0/2324981578 learned_addr learned my addr 172.21.15.116:0/2324981578 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:07.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.318+0000 7f2206284640 1 --2- 172.21.15.116:0/2324981578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3a20724a0 0x55b3a2072880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:07.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.318+0000 7f21f6ffd640 1 --2- 172.21.15.116:0/2927874325 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f21ec0a2b80 0x7f21ec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:07.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.318+0000 7f2205a83640 1 --2- 172.21.15.116:0/2324981578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3a2072dc0 0x55b3a207d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:07.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.318+0000 7f2206284640 1 --2- 172.21.15.116:0/2927874325 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f21ec0a2b80 0x7f21ec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:07.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.318+0000 7f21f6ffd640 1 -- 172.21.15.116:0/2927874325 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f21fc0ce920 con 0x7f2200103660 2024-09-12T03:10:07.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.319+0000 7f2205282640 1 -- 172.21.15.116:0/2324981578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3a20724a0 msgr2=0x55b3a2072880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:07.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.319+0000 7f2205282640 1 --2- 172.21.15.116:0/2324981578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3a20724a0 0x55b3a2072880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.319+0000 7f2205282640 1 -- 172.21.15.116:0/2324981578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3a2072dc0 msgr2=0x55b3a207d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:07.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.319+0000 7f2205282640 1 --2- 172.21.15.116:0/2324981578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3a2072dc0 0x55b3a207d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.319+0000 7f2205282640 1 -- 172.21.15.116:0/2324981578 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3a1f64440 con 0x55b3a207d680 2024-09-12T03:10:07.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.319+0000 7f2205a83640 1 --2- 172.21.15.116:0/2324981578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3a2072dc0 0x55b3a207d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:10:07.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.319+0000 7f2205282640 1 --2- 172.21.15.116:0/2324981578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3a207d680 0x55b3a207fa70 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f21fc0cc450 tx=0x7f21fc0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca1088d4ded927b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:07.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.319+0000 7f2206284640 1 --2- 172.21.15.116:0/2927874325 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f21ec0a2b80 0x7f21ec0a5040 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f21f8009ca0 tx=0x7f21f802a950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:07.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.319+0000 7f21ebfff640 1 -- 172.21.15.116:0/2324981578 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc09ab10 con 0x55b3a207d680 2024-09-12T03:10:07.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.319+0000 7f21ebfff640 1 -- 172.21.15.116:0/2324981578 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21fc010eb0 con 0x55b3a207d680 2024-09-12T03:10:07.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.319+0000 7f21ebfff640 1 -- 172.21.15.116:0/2324981578 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc009200 con 0x55b3a207d680 2024-09-12T03:10:07.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.320+0000 7f220a0c0740 1 -- 172.21.15.116:0/2324981578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3a207d680 msgr2=0x55b3a207fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:07.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.320+0000 7f220a0c0740 1 --2- 172.21.15.116:0/2324981578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3a207d680 0x55b3a207fa70 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f21fc0cc450 tx=0x7f21fc0047d0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.320+0000 7f220a0c0740 1 -- 172.21.15.116:0/2324981578 shutdown_connections 2024-09-12T03:10:07.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.320+0000 7f220a0c0740 1 --2- 172.21.15.116:0/2324981578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3a207d680 0x55b3a207fa70 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.320+0000 7f220a0c0740 1 --2- 172.21.15.116:0/2324981578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3a2072dc0 0x55b3a207d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.320+0000 7f220a0c0740 1 --2- 172.21.15.116:0/2324981578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3a20724a0 0x55b3a2072880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.320+0000 7f220a0c0740 1 -- 172.21.15.116:0/2324981578 >> 172.21.15.116:0/2324981578 conn(0x55b3a206ea30 msgr2=0x55b3a206ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:07.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.320+0000 7f220a0c0740 1 -- 172.21.15.116:0/2324981578 shutdown_connections 2024-09-12T03:10:07.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.320+0000 7f220a0c0740 1 -- 172.21.15.116:0/2324981578 wait complete. 2024-09-12T03:10:07.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.320+0000 7f220a0c0740 1 -- start start 2024-09-12T03:10:07.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.320+0000 7f220a0c0740 1 Processor -- start 2024-09-12T03:10:07.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.321+0000 7f220a0c0740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3a20724a0 0x55b3a207cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:07.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.321+0000 7f220a0c0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3a2072dc0 0x55b3a207a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:07.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.321+0000 7f220a0c0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3a207d680 0x55b3a207a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:07.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.321+0000 7f220a0c0740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b3a20801f0 con 0x55b3a20724a0 2024-09-12T03:10:07.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.321+0000 7f220a0c0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b3a1f93050 con 0x55b3a2072dc0 2024-09-12T03:10:07.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.321+0000 7f220a0c0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b3a2080370 con 0x55b3a207d680 2024-09-12T03:10:07.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.321+0000 7f220a0c0740 10 client.0 _ll_register_callbacks cb 0x55b3a1fd3160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:07.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.321+0000 7f2205a83640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3a20724a0 0x55b3a207cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:07.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.321+0000 7f2205a83640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3a20724a0 0x55b3a207cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59030/0 (socket says 172.21.15.116:59030) 2024-09-12T03:10:07.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.321+0000 7f2205a83640 1 -- 172.21.15.116:0/3932034263 learned_addr learned my addr 172.21.15.116:0/3932034263 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:07.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.321+0000 7f2206284640 1 --2- 172.21.15.116:0/3932034263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3a207d680 0x55b3a207a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:07.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.321+0000 7f2205282640 1 --2- 172.21.15.116:0/3932034263 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3a2072dc0 0x55b3a207a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:07.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.321+0000 7f2205a83640 1 -- 172.21.15.116:0/3932034263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3a207d680 msgr2=0x55b3a207a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:07.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.321+0000 7f2205a83640 1 --2- 172.21.15.116:0/3932034263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3a207d680 0x55b3a207a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.321+0000 7f2205a83640 1 -- 172.21.15.116:0/3932034263 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3a2072dc0 msgr2=0x55b3a207a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:07.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.321+0000 7f2205a83640 1 --2- 172.21.15.116:0/3932034263 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3a2072dc0 0x55b3a207a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.321+0000 7f2205a83640 1 -- 172.21.15.116:0/3932034263 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3a1d493c0 con 0x55b3a20724a0 2024-09-12T03:10:07.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.321+0000 7f2205282640 1 --2- 172.21.15.116:0/3932034263 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3a2072dc0 0x55b3a207a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:07.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.321+0000 7f2206284640 1 --2- 172.21.15.116:0/3932034263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3a207d680 0x55b3a207a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:07.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.321+0000 7f2205a83640 1 --2- 172.21.15.116:0/3932034263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3a20724a0 0x55b3a207cf50 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f21f000df00 tx=0x7f21f000f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd1263519058ffa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:07.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.321+0000 7f21e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:07.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.322+0000 7f21e97fa640 1 -- 172.21.15.116:0/3932034263 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f0018070 con 0x55b3a20724a0 2024-09-12T03:10:07.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.322+0000 7f21e97fa640 1 -- 172.21.15.116:0/3932034263 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f21f0014680 con 0x55b3a20724a0 2024-09-12T03:10:07.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.322+0000 7f220a0c0740 1 -- 172.21.15.116:0/3932034263 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b3a1fe5850 con 0x55b3a20724a0 2024-09-12T03:10:07.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.322+0000 7f21e97fa640 1 -- 172.21.15.116:0/3932034263 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f21f0024050 con 0x55b3a20724a0 2024-09-12T03:10:07.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.322+0000 7f220a0c0740 10 client.17160 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:07.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.322+0000 7f220a0c0740 1 -- 172.21.15.116:0/3932034263 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b3a208bba0 con 0x55b3a20724a0 2024-09-12T03:10:07.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.322+0000 7f21e97fa640 1 -- 172.21.15.116:0/3932034263 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b3a1fe5850 con 0x55b3a20724a0 2024-09-12T03:10:07.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.322+0000 7f21e97fa640 1 -- 172.21.15.116:0/3932034263 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f21f0013b70 con 0x55b3a20724a0 2024-09-12T03:10:07.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.322+0000 7f220a0c0740 10 client.17160 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:07.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.322+0000 7f220a0c0740 10 client.17160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:10:07.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.322+0000 7f220a0c0740 20 client.17160 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:07.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.322+0000 7f220a0c0740 1 --2- 172.21.15.116:0/3932034263 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b3a208cfb0 0x55b3a208f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:07.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.322+0000 7f220a0c0740 4 client.17160 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:10:07.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.322+0000 7f220a0c0740 1 -- 172.21.15.116:0/3932034263 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3a1fdfd60 con 0x55b3a208cfb0 2024-09-12T03:10:07.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.322+0000 7f2205282640 1 --2- 172.21.15.116:0/3932034263 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b3a208cfb0 0x55b3a208f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:07.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.323+0000 7f21e97fa640 10 client.17160 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:07.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.323+0000 7f2205282640 1 --2- 172.21.15.116:0/3932034263 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b3a208cfb0 0x55b3a208f440 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75d64fb03abc1ba3 server_cookie=7b552c233c36b384 in_seq=0 out_seq=0 2024-09-12T03:10:07.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.324+0000 7f21e97fa640 1 -- 172.21.15.116:0/3932034263 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b3a1fdfd60 con 0x55b3a208cfb0 2024-09-12T03:10:07.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.324+0000 7f21e97fa640 10 client.17160 handle_command_reply: tid=0 2024-09-12T03:10:07.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.324+0000 7f220a0c0740 1 client.17160 shutdown 2024-09-12T03:10:07.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.324+0000 7f220a0c0740 1 -- 172.21.15.116:0/3932034263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3a20724a0 msgr2=0x55b3a207cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:07.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.324+0000 7f220a0c0740 1 --2- 172.21.15.116:0/3932034263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3a20724a0 0x55b3a207cf50 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f21f000df00 tx=0x7f21f000f5b0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.324+0000 7f220a0c0740 1 -- 172.21.15.116:0/3932034263 shutdown_connections 2024-09-12T03:10:07.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.324+0000 7f220a0c0740 1 --2- 172.21.15.116:0/3932034263 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b3a208cfb0 0x55b3a208f440 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.324+0000 7f220a0c0740 1 --2- 172.21.15.116:0/3932034263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3a207d680 0x55b3a207a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.324+0000 7f220a0c0740 1 --2- 172.21.15.116:0/3932034263 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3a2072dc0 0x55b3a207a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.324+0000 7f220a0c0740 1 --2- 172.21.15.116:0/3932034263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3a20724a0 0x55b3a207cf50 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.324+0000 7f21e97fa640 0 client.17160 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:07.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.324+0000 7f220a0c0740 1 -- 172.21.15.116:0/3932034263 >> 172.21.15.116:0/3932034263 conn(0x55b3a206ea30 msgr2=0x55b3a2074710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:07.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.324+0000 7f220a0c0740 1 -- 172.21.15.116:0/3932034263 shutdown_connections 2024-09-12T03:10:07.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.324+0000 7f220a0c0740 1 -- 172.21.15.116:0/3932034263 wait complete. 2024-09-12T03:10:07.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.324+0000 7f220a0c0740 20 client.17160 trim_cache size 0 max 16384 2024-09-12T03:10:07.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.342+0000 7f220a0c0740 1 Processor -- start 2024-09-12T03:10:07.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.342+0000 7f220a0c0740 1 -- start start 2024-09-12T03:10:07.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.342+0000 7f220a0c0740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3a20724a0 0x55b3a2089450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:07.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.342+0000 7f220a0c0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3a2072dc0 0x55b3a2094d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:07.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.342+0000 7f220a0c0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3a207d680 0x55b3a2095250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:07.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.342+0000 7f220a0c0740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f22001afee0 con 0x55b3a20724a0 2024-09-12T03:10:07.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.342+0000 7f220a0c0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f22001ad7b0 con 0x55b3a2072dc0 2024-09-12T03:10:07.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.342+0000 7f220a0c0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f22001ab600 con 0x55b3a207d680 2024-09-12T03:10:07.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.342+0000 7f2205a83640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3a20724a0 0x55b3a2089450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:07.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.342+0000 7f2205a83640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3a20724a0 0x55b3a2089450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59034/0 (socket says 172.21.15.116:59034) 2024-09-12T03:10:07.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.342+0000 7f2205a83640 1 -- 172.21.15.116:0/2189968401 learned_addr learned my addr 172.21.15.116:0/2189968401 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:07.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.342+0000 7f2205282640 1 --2- 172.21.15.116:0/2189968401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3a2072dc0 0x55b3a2094d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:07.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.342+0000 7f2206284640 1 --2- 172.21.15.116:0/2189968401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3a207d680 0x55b3a2095250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:07.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.342+0000 7f2205a83640 1 -- 172.21.15.116:0/2189968401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3a207d680 msgr2=0x55b3a2095250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:07.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.342+0000 7f2205a83640 1 --2- 172.21.15.116:0/2189968401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3a207d680 0x55b3a2095250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.342+0000 7f2205a83640 1 -- 172.21.15.116:0/2189968401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3a2072dc0 msgr2=0x55b3a2094d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:07.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.342+0000 7f2205a83640 1 --2- 172.21.15.116:0/2189968401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3a2072dc0 0x55b3a2094d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.342+0000 7f2205a83640 1 -- 172.21.15.116:0/2189968401 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3a2072bf0 con 0x55b3a20724a0 2024-09-12T03:10:07.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.342+0000 7f2205a83640 1 --2- 172.21.15.116:0/2189968401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3a20724a0 0x55b3a2089450 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f21f0004810 tx=0x7f21f000de90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d259a31cac9ac4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:07.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.343+0000 7f21eb7fe640 1 -- 172.21.15.116:0/2189968401 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f0018040 con 0x55b3a20724a0 2024-09-12T03:10:07.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.343+0000 7f21eb7fe640 1 -- 172.21.15.116:0/2189968401 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21f001c970 con 0x55b3a20724a0 2024-09-12T03:10:07.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.343+0000 7f21eb7fe640 1 -- 172.21.15.116:0/2189968401 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f0006450 con 0x55b3a20724a0 2024-09-12T03:10:07.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.343+0000 7f220a0c0740 1 -- 172.21.15.116:0/2189968401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3a20724a0 msgr2=0x55b3a2089450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:07.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.343+0000 7f220a0c0740 1 --2- 172.21.15.116:0/2189968401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3a20724a0 0x55b3a2089450 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f21f0004810 tx=0x7f21f000de90 comp rx=0 tx=0).stop 2024-09-12T03:10:07.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.343+0000 7f220a0c0740 1 -- 172.21.15.116:0/2189968401 shutdown_connections 2024-09-12T03:10:07.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.343+0000 7f220a0c0740 1 --2- 172.21.15.116:0/2189968401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3a207d680 0x55b3a2095250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.343+0000 7f220a0c0740 1 --2- 172.21.15.116:0/2189968401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3a2072dc0 0x55b3a2094d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.343+0000 7f220a0c0740 1 --2- 172.21.15.116:0/2189968401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3a20724a0 0x55b3a2089450 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.343+0000 7f220a0c0740 1 -- 172.21.15.116:0/2189968401 >> 172.21.15.116:0/2189968401 conn(0x55b3a206ea30 msgr2=0x55b3a2078ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:07.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.343+0000 7f220a0c0740 1 -- 172.21.15.116:0/2189968401 shutdown_connections 2024-09-12T03:10:07.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.343+0000 7f220a0c0740 1 -- 172.21.15.116:0/2189968401 wait complete. 2024-09-12T03:10:07.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.344+0000 7f220a0c0740 1 -- start start 2024-09-12T03:10:07.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.344+0000 7f220a0c0740 1 Processor -- start 2024-09-12T03:10:07.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.344+0000 7f220a0c0740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3a20724a0 0x55b3a2085e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:07.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.344+0000 7f220a0c0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3a2072dc0 0x55b3a2081210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:07.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.344+0000 7f220a0c0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3a207d680 0x55b3a20819a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:07.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.344+0000 7f220a0c0740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f22001abc70 con 0x55b3a20724a0 2024-09-12T03:10:07.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.344+0000 7f220a0c0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f22001ad0e0 con 0x55b3a2072dc0 2024-09-12T03:10:07.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.344+0000 7f220a0c0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f22001ac340 con 0x55b3a207d680 2024-09-12T03:10:07.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.344+0000 7f220a0c0740 10 client.0 _ll_register_callbacks cb 0x7f21f000d180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:07.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.344+0000 7f2205a83640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3a20724a0 0x55b3a2085e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:07.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.344+0000 7f2205a83640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3a20724a0 0x55b3a2085e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:59040/0 (socket says 172.21.15.116:59040) 2024-09-12T03:10:07.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.344+0000 7f2205a83640 1 -- 172.21.15.116:0/1553352410 learned_addr learned my addr 172.21.15.116:0/1553352410 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:07.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.344+0000 7f2206284640 1 --2- 172.21.15.116:0/1553352410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3a207d680 0x55b3a20819a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:07.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.344+0000 7f2205282640 1 --2- 172.21.15.116:0/1553352410 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3a2072dc0 0x55b3a2081210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:07.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.345+0000 7f2205a83640 1 -- 172.21.15.116:0/1553352410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3a207d680 msgr2=0x55b3a20819a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:07.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.345+0000 7f2205a83640 1 --2- 172.21.15.116:0/1553352410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3a207d680 0x55b3a20819a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.345+0000 7f2205a83640 1 -- 172.21.15.116:0/1553352410 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3a2072dc0 msgr2=0x55b3a2081210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:07.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.345+0000 7f2205a83640 1 --2- 172.21.15.116:0/1553352410 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3a2072dc0 0x55b3a2081210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.345+0000 7f2205a83640 1 -- 172.21.15.116:0/1553352410 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3a20fb0f0 con 0x55b3a20724a0 2024-09-12T03:10:07.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.345+0000 7f2205a83640 1 --2- 172.21.15.116:0/1553352410 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3a20724a0 0x55b3a2085e90 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f21f000dba0 tx=0x7f21f001c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abeb8cf33b3a582 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:07.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.345+0000 7f21e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:07.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.345+0000 7f21e97fa640 1 -- 172.21.15.116:0/1553352410 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f0005bc0 con 0x55b3a20724a0 2024-09-12T03:10:07.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.345+0000 7f21e97fa640 1 -- 172.21.15.116:0/1553352410 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f21f0005f40 con 0x55b3a20724a0 2024-09-12T03:10:07.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.345+0000 7f220a0c0740 1 -- 172.21.15.116:0/1553352410 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21f001c970 con 0x55b3a20724a0 2024-09-12T03:10:07.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.345+0000 7f21e97fa640 1 -- 172.21.15.116:0/1553352410 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f21f0005d60 con 0x55b3a20724a0 2024-09-12T03:10:07.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.345+0000 7f220a0c0740 10 client.17166 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:07.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.345+0000 7f220a0c0740 1 -- 172.21.15.116:0/1553352410 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b3a20fc290 con 0x55b3a20724a0 2024-09-12T03:10:07.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.345+0000 7f21e97fa640 1 -- 172.21.15.116:0/1553352410 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21f001c970 con 0x55b3a20724a0 2024-09-12T03:10:07.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.346+0000 7f21e97fa640 1 -- 172.21.15.116:0/1553352410 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f21f00137b0 con 0x55b3a20724a0 2024-09-12T03:10:07.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.346+0000 7f220a0c0740 10 client.17166 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:07.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.346+0000 7f220a0c0740 10 client.17166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:10:07.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.346+0000 7f220a0c0740 20 client.17166 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:07.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.346+0000 7f220a0c0740 1 --2- 172.21.15.116:0/1553352410 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b3a20fc8f0 0x55b3a20fece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:07.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.346+0000 7f220a0c0740 4 client.17166 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:10:07.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.346+0000 7f220a0c0740 1 -- 172.21.15.116:0/1553352410 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b3a207aad0 con 0x55b3a20fc8f0 2024-09-12T03:10:07.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.346+0000 7f2205282640 1 --2- 172.21.15.116:0/1553352410 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b3a20fc8f0 0x55b3a20fece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:07.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.346+0000 7f21e97fa640 10 client.17166 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:07.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.346+0000 7f2205282640 1 --2- 172.21.15.116:0/1553352410 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b3a20fc8f0 0x55b3a20fece0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8ad96bd5653a7f7 server_cookie=10cabc2c18e53b98 in_seq=0 out_seq=0 2024-09-12T03:10:07.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.346+0000 7f21e97fa640 1 -- 172.21.15.116:0/1553352410 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b3a207aad0 con 0x55b3a20fc8f0 2024-09-12T03:10:07.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.346+0000 7f21e97fa640 10 client.17166 handle_command_reply: tid=0 2024-09-12T03:10:07.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.346+0000 7f220a0c0740 1 client.17166 shutdown 2024-09-12T03:10:07.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.347+0000 7f220a0c0740 1 -- 172.21.15.116:0/1553352410 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3a20724a0 msgr2=0x55b3a2085e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:07.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.347+0000 7f220a0c0740 1 --2- 172.21.15.116:0/1553352410 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3a20724a0 0x55b3a2085e90 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f21f000dba0 tx=0x7f21f001c890 comp rx=0 tx=0).stop 2024-09-12T03:10:07.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.347+0000 7f220a0c0740 1 -- 172.21.15.116:0/1553352410 shutdown_connections 2024-09-12T03:10:07.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.347+0000 7f220a0c0740 1 --2- 172.21.15.116:0/1553352410 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b3a20fc8f0 0x55b3a20fece0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.347+0000 7f220a0c0740 1 --2- 172.21.15.116:0/1553352410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b3a207d680 0x55b3a20819a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.347+0000 7f220a0c0740 1 --2- 172.21.15.116:0/1553352410 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b3a2072dc0 0x55b3a2081210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.347+0000 7f220a0c0740 1 --2- 172.21.15.116:0/1553352410 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3a20724a0 0x55b3a2085e90 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.347+0000 7f21e97fa640 0 client.17166 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:07.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.347+0000 7f220a0c0740 1 -- 172.21.15.116:0/1553352410 >> 172.21.15.116:0/1553352410 conn(0x55b3a206ea30 msgr2=0x55b3a2078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:07.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.347+0000 7f220a0c0740 1 -- 172.21.15.116:0/1553352410 shutdown_connections 2024-09-12T03:10:07.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.347+0000 7f220a0c0740 1 -- 172.21.15.116:0/1553352410 wait complete. 2024-09-12T03:10:07.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.347+0000 7f220a0c0740 20 client.17166 trim_cache size 0 max 16384 2024-09-12T03:10:07.362 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:10:07.362 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:10:07.362 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": { 2024-09-12T03:10:07.362 INFO:teuthology.orchestra.run.smithi116.stdout: "b97c07c7-7361-4f4c-a3d2-3916e8dd20d3": { 2024-09-12T03:10:07.362 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/", 2024-09-12T03:10:07.362 INFO:teuthology.orchestra.run.smithi116.stdout: "tag": "b97c07c7-7361-4f4c-a3d2-3916e8dd20d3", 2024-09-12T03:10:07.362 INFO:teuthology.orchestra.run.smithi116.stdout: "options": "recursive,force" 2024-09-12T03:10:07.363 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:10:07.363 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:10:07.363 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:10:07.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.347+0000 7f2207ce7640 1 -- 172.21.15.116:0/2927874325 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f21ec0a2b80 msgr2=0x7f21ec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:07.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.347+0000 7f2207ce7640 1 --2- 172.21.15.116:0/2927874325 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f21ec0a2b80 0x7f21ec0a5040 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f21f8009ca0 tx=0x7f21f802a950 comp rx=0 tx=0).stop 2024-09-12T03:10:07.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.347+0000 7f2207ce7640 1 -- 172.21.15.116:0/2927874325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2200103660 msgr2=0x7f2200198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:07.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.347+0000 7f2207ce7640 1 --2- 172.21.15.116:0/2927874325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2200103660 0x7f2200198b80 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00b3f0 tx=0x7f21fc00b8b0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.347+0000 7f2207ce7640 1 -- 172.21.15.116:0/2927874325 shutdown_connections 2024-09-12T03:10:07.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.347+0000 7f2207ce7640 1 --2- 172.21.15.116:0/2927874325 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f21ec0a2b80 0x7f21ec0a5040 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.347+0000 7f2207ce7640 1 --2- 172.21.15.116:0/2927874325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f220010c170 0x7f220019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.347+0000 7f2207ce7640 1 --2- 172.21.15.116:0/2927874325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2200103660 0x7f2200198b80 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.347+0000 7f2207ce7640 1 --2- 172.21.15.116:0/2927874325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2200102d40 0x7f2200198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.347+0000 7f2207ce7640 1 -- 172.21.15.116:0/2927874325 >> 172.21.15.116:0/2927874325 conn(0x7f22000fc7a0 msgr2=0x7f22000fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:07.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.348+0000 7f2207ce7640 1 -- 172.21.15.116:0/2927874325 shutdown_connections 2024-09-12T03:10:07.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:07.348+0000 7f2207ce7640 1 -- 172.21.15.116:0/2927874325 wait complete. 2024-09-12T03:10:07.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:07 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:10:07.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:07 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:10:07.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:07 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:10:07.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:07 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:10:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:07 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:10:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:07 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:10:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:07 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:10:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:07 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:10:07.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:07 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:10:07.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:07 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:10:07.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:07 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:10:07.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:07 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:10:08.083 INFO:tasks.cephfs.filesystem:scrub status for tag:b97c07c7-7361-4f4c-a3d2-3916e8dd20d3 - {'path': '/', 'tag': 'b97c07c7-7361-4f4c-a3d2-3916e8dd20d3', 'options': 'recursive,force'} 2024-09-12T03:10:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:08 smithi183 ceph-mon[35535]: pgmap v552: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T03:10:08.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:08 smithi177 ceph-mon[35305]: pgmap v552: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T03:10:08.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:08 smithi116 ceph-mon[31410]: pgmap v552: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T03:10:09.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:09 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:10:09.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:09 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:10:09.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:09 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:10:10.563 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:10:10.592 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:10 smithi116 ceph-mon[31410]: pgmap v553: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.9 KiB/s wr, 7 op/s 2024-09-12T03:10:10.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:10 smithi183 ceph-mon[35535]: pgmap v553: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.9 KiB/s wr, 7 op/s 2024-09-12T03:10:10.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:10 smithi177 ceph-mon[35305]: pgmap v553: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.9 KiB/s wr, 7 op/s 2024-09-12T03:10:10.829 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:10:11.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.373+0000 7f09cac49640 1 -- 172.21.15.116:0/2027868983 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09c4100100 msgr2=0x7f09c4100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:11.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.373+0000 7f09cac49640 1 --2- 172.21.15.116:0/2027868983 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09c4100100 0x7f09c4100560 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009900 tx=0x7f09b002a410 comp rx=0 tx=0).stop 2024-09-12T03:10:11.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.374+0000 7f09cac49640 1 -- 172.21.15.116:0/2027868983 shutdown_connections 2024-09-12T03:10:11.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.374+0000 7f09cac49640 1 --2- 172.21.15.116:0/2027868983 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09c4100aa0 0x7f09c410ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:11.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.374+0000 7f09cac49640 1 --2- 172.21.15.116:0/2027868983 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09c4100100 0x7f09c4100560 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:11.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.374+0000 7f09cac49640 1 --2- 172.21.15.116:0/2027868983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09c4106b80 0x7f09c4106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:11.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.374+0000 7f09cac49640 1 -- 172.21.15.116:0/2027868983 >> 172.21.15.116:0/2027868983 conn(0x7f09c4077230 msgr2=0x7f09c40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:11.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.374+0000 7f09cac49640 1 -- 172.21.15.116:0/2027868983 shutdown_connections 2024-09-12T03:10:11.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.374+0000 7f09cac49640 1 -- 172.21.15.116:0/2027868983 wait complete. 2024-09-12T03:10:11.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.374+0000 7f09cac49640 1 Processor -- start 2024-09-12T03:10:11.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.374+0000 7f09cac49640 1 -- start start 2024-09-12T03:10:11.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.375+0000 7f09cac49640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09c4100100 0x7f09c4198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:11.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.375+0000 7f09cac49640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09c4100aa0 0x7f09c4198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:11.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.375+0000 7f09cac49640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09c4106b80 0x7f09c419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:11.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.375+0000 7f09cac49640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f09c41120a0 con 0x7f09c4100100 2024-09-12T03:10:11.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.375+0000 7f09cac49640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f09c4111f20 con 0x7f09c4106b80 2024-09-12T03:10:11.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.375+0000 7f09cac49640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f09c4112220 con 0x7f09c4100aa0 2024-09-12T03:10:11.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.375+0000 7f09c89e5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09c4100100 0x7f09c4198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:11.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.375+0000 7f09c89e5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09c4100100 0x7f09c4198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33398/0 (socket says 172.21.15.116:33398) 2024-09-12T03:10:11.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.375+0000 7f09c89e5640 1 -- 172.21.15.116:0/3726737598 learned_addr learned my addr 172.21.15.116:0/3726737598 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:11.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.375+0000 7f09c3fff640 1 --2- 172.21.15.116:0/3726737598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09c4100aa0 0x7f09c4198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:11.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.375+0000 7f09c89e5640 1 -- 172.21.15.116:0/3726737598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09c4100aa0 msgr2=0x7f09c4198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:11.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.375+0000 7f09c89e5640 1 --2- 172.21.15.116:0/3726737598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09c4100aa0 0x7f09c4198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:11.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.375+0000 7f09c91e6640 1 --2- 172.21.15.116:0/3726737598 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09c4106b80 0x7f09c419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:11.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.375+0000 7f09c89e5640 1 -- 172.21.15.116:0/3726737598 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09c4106b80 msgr2=0x7f09c419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:11.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.375+0000 7f09c89e5640 1 --2- 172.21.15.116:0/3726737598 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09c4106b80 0x7f09c419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:11.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.375+0000 7f09c89e5640 1 -- 172.21.15.116:0/3726737598 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c419d4b0 con 0x7f09c4100100 2024-09-12T03:10:11.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.375+0000 7f09c91e6640 1 --2- 172.21.15.116:0/3726737598 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09c4106b80 0x7f09c419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:11.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.375+0000 7f09c89e5640 1 --2- 172.21.15.116:0/3726737598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09c4100100 0x7f09c4198500 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f09b800c970 tx=0x7f09b800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:11.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.376+0000 7f09c1ffb640 1 -- 172.21.15.116:0/3726737598 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b8007bb0 con 0x7f09c4100100 2024-09-12T03:10:11.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.376+0000 7f09c1ffb640 1 -- 172.21.15.116:0/3726737598 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09b8007d50 con 0x7f09c4100100 2024-09-12T03:10:11.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.376+0000 7f09cac49640 1 -- 172.21.15.116:0/3726737598 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c419d7a0 con 0x7f09c4100100 2024-09-12T03:10:11.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.376+0000 7f09c1ffb640 1 -- 172.21.15.116:0/3726737598 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b800f450 con 0x7f09c4100100 2024-09-12T03:10:11.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.376+0000 7f09cac49640 1 -- 172.21.15.116:0/3726737598 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c4101210 con 0x7f09c4100100 2024-09-12T03:10:11.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.377+0000 7f09cac49640 1 -- 172.21.15.116:0/3726737598 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09c4100560 con 0x7f09c4100100 2024-09-12T03:10:11.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.379+0000 7f09c1ffb640 1 -- 172.21.15.116:0/3726737598 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f09b8004fe0 con 0x7f09c4100100 2024-09-12T03:10:11.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.379+0000 7f09c1ffb640 1 --2- 172.21.15.116:0/3726737598 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f09b40a2b80 0x7f09b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:11.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.379+0000 7f09c1ffb640 1 -- 172.21.15.116:0/3726737598 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f09b80cf290 con 0x7f09c4100100 2024-09-12T03:10:11.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.379+0000 7f09c3fff640 1 --2- 172.21.15.116:0/3726737598 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f09b40a2b80 0x7f09b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:11.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.380+0000 7f09c3fff640 1 --2- 172.21.15.116:0/3726737598 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f09b40a2b80 0x7f09b40a5040 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f09b0000c00 tx=0x7f09b0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:11.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.382+0000 7f09c1ffb640 1 -- 172.21.15.116:0/3726737598 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f09b8014030 con 0x7f09c4100100 2024-09-12T03:10:11.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:11 smithi116 ceph-mon[31410]: pgmap v554: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.8 KiB/s wr, 8 op/s 2024-09-12T03:10:11.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.590+0000 7f09cac49640 1 -- 172.21.15.116:0/3726737598 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09c4199860 con 0x7f09c4100100 2024-09-12T03:10:11.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.591+0000 7f09c1ffb640 1 -- 172.21.15.116:0/3726737598 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f09c4199860 con 0x7f09c4100100 2024-09-12T03:10:11.591 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:10:11.591 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:10:11.591 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:10:11.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.593+0000 7f09cac49640 1 -- 172.21.15.116:0/3726737598 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f09b40a2b80 msgr2=0x7f09b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:11.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.593+0000 7f09cac49640 1 --2- 172.21.15.116:0/3726737598 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f09b40a2b80 0x7f09b40a5040 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f09b0000c00 tx=0x7f09b0005a20 comp rx=0 tx=0).stop 2024-09-12T03:10:11.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.593+0000 7f09cac49640 1 -- 172.21.15.116:0/3726737598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09c4100100 msgr2=0x7f09c4198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:11.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.593+0000 7f09cac49640 1 --2- 172.21.15.116:0/3726737598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09c4100100 0x7f09c4198500 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f09b800c970 tx=0x7f09b800ce30 comp rx=0 tx=0).stop 2024-09-12T03:10:11.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.593+0000 7f09cac49640 1 -- 172.21.15.116:0/3726737598 shutdown_connections 2024-09-12T03:10:11.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.593+0000 7f09cac49640 1 --2- 172.21.15.116:0/3726737598 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f09b40a2b80 0x7f09b40a5040 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:11.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.593+0000 7f09cac49640 1 --2- 172.21.15.116:0/3726737598 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09c4106b80 0x7f09c419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:11.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.593+0000 7f09cac49640 1 --2- 172.21.15.116:0/3726737598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09c4100aa0 0x7f09c4198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:11.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.593+0000 7f09cac49640 1 --2- 172.21.15.116:0/3726737598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09c4100100 0x7f09c4198500 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:11.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.593+0000 7f09cac49640 1 -- 172.21.15.116:0/3726737598 >> 172.21.15.116:0/3726737598 conn(0x7f09c4077230 msgr2=0x7f09c4104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:11.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.593+0000 7f09cac49640 1 -- 172.21.15.116:0/3726737598 shutdown_connections 2024-09-12T03:10:11.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:11.593+0000 7f09cac49640 1 -- 172.21.15.116:0/3726737598 wait complete. 2024-09-12T03:10:11.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:11 smithi183 ceph-mon[35535]: pgmap v554: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.8 KiB/s wr, 8 op/s 2024-09-12T03:10:11.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:11 smithi177 ceph-mon[35305]: pgmap v554: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.8 KiB/s wr, 8 op/s 2024-09-12T03:10:12.368 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id 4007087b --release --await' 2024-09-12T03:10:12.368 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id 4007087b --release --await 2024-09-12T03:10:12.656 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:10:12.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:12 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3726737598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:10:12.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:12 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3726737598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:10:12.777 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:12 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3726737598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:10:13.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.196+0000 7f8c05a8e640 1 -- 172.21.15.116:0/73258734 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c00100a80 msgr2=0x7f8c00100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:13.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.196+0000 7f8c05a8e640 1 --2- 172.21.15.116:0/73258734 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c00100a80 0x7f8c00100ee0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009930 tx=0x7f8bf402a420 comp rx=0 tx=0).stop 2024-09-12T03:10:13.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.196+0000 7f8c05a8e640 1 -- 172.21.15.116:0/73258734 shutdown_connections 2024-09-12T03:10:13.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.196+0000 7f8c05a8e640 1 --2- 172.21.15.116:0/73258734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c00101420 0x7f8c0010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.196+0000 7f8c05a8e640 1 --2- 172.21.15.116:0/73258734 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c00100a80 0x7f8c00100ee0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.196+0000 7f8c05a8e640 1 --2- 172.21.15.116:0/73258734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c00106a80 0x7f8c00106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.196+0000 7f8c05a8e640 1 -- 172.21.15.116:0/73258734 >> 172.21.15.116:0/73258734 conn(0x7f8c000fc7a0 msgr2=0x7f8c000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:13.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.196+0000 7f8c05a8e640 1 -- 172.21.15.116:0/73258734 shutdown_connections 2024-09-12T03:10:13.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.196+0000 7f8c05a8e640 1 -- 172.21.15.116:0/73258734 wait complete. 2024-09-12T03:10:13.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.196+0000 7f8c05a8e640 1 Processor -- start 2024-09-12T03:10:13.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.197+0000 7f8c05a8e640 1 -- start start 2024-09-12T03:10:13.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.197+0000 7f8c05a8e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c00100a80 0x7f8c00198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:13.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.197+0000 7f8c05a8e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c00101420 0x7f8c00198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:13.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.197+0000 7f8c05a8e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c00106a80 0x7f8c0019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:13.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.197+0000 7f8c05a8e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8c00110e80 con 0x7f8c00100a80 2024-09-12T03:10:13.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.197+0000 7f8c05a8e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8c00110d00 con 0x7f8c00101420 2024-09-12T03:10:13.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.197+0000 7f8c05a8e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8c00111000 con 0x7f8c00106a80 2024-09-12T03:10:13.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.197+0000 7f8bff7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c00100a80 0x7f8c00198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:13.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.197+0000 7f8bff7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c00100a80 0x7f8c00198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33418/0 (socket says 172.21.15.116:33418) 2024-09-12T03:10:13.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.197+0000 7f8bff7fe640 1 -- 172.21.15.116:0/3761568903 learned_addr learned my addr 172.21.15.116:0/3761568903 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:13.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.197+0000 7f8bfffff640 1 --2- 172.21.15.116:0/3761568903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c00106a80 0x7f8c0019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:13.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.197+0000 7f8bfeffd640 1 --2- 172.21.15.116:0/3761568903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c00101420 0x7f8c00198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:13.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.197+0000 7f8bff7fe640 1 -- 172.21.15.116:0/3761568903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c00106a80 msgr2=0x7f8c0019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:13.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.197+0000 7f8bff7fe640 1 --2- 172.21.15.116:0/3761568903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c00106a80 0x7f8c0019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.197+0000 7f8bff7fe640 1 -- 172.21.15.116:0/3761568903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c00101420 msgr2=0x7f8c00198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:13.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.197+0000 7f8bff7fe640 1 --2- 172.21.15.116:0/3761568903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c00101420 0x7f8c00198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.197+0000 7f8bff7fe640 1 -- 172.21.15.116:0/3761568903 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0019d3c0 con 0x7f8c00100a80 2024-09-12T03:10:13.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.197+0000 7f8bfffff640 1 --2- 172.21.15.116:0/3761568903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c00106a80 0x7f8c0019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:13.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.197+0000 7f8bff7fe640 1 --2- 172.21.15.116:0/3761568903 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c00100a80 0x7f8c00198410 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f8bf000cce0 tx=0x7f8bf000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:13.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.197+0000 7f8bfcff9640 1 -- 172.21.15.116:0/3761568903 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0004b20 con 0x7f8c00100a80 2024-09-12T03:10:13.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.198+0000 7f8bfcff9640 1 -- 172.21.15.116:0/3761568903 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bf0004cc0 con 0x7f8c00100a80 2024-09-12T03:10:13.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.198+0000 7f8c05a8e640 1 -- 172.21.15.116:0/3761568903 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c0019d6b0 con 0x7f8c00100a80 2024-09-12T03:10:13.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.198+0000 7f8bfcff9640 1 -- 172.21.15.116:0/3761568903 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf000f7d0 con 0x7f8c00100a80 2024-09-12T03:10:13.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.198+0000 7f8c05a8e640 1 -- 172.21.15.116:0/3761568903 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c001a4f90 con 0x7f8c00100a80 2024-09-12T03:10:13.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.200+0000 7f8bfcff9640 1 -- 172.21.15.116:0/3761568903 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8bf000f970 con 0x7f8c00100a80 2024-09-12T03:10:13.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.200+0000 7f8c07e67740 1 Processor -- start 2024-09-12T03:10:13.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.200+0000 7f8c07e67740 1 -- start start 2024-09-12T03:10:13.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.200+0000 7f8c07e67740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab6210c060 0x55ab6210c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:13.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.200+0000 7f8c07e67740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab6210c980 0x55ab62116d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:13.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.200+0000 7f8c07e67740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab62117240 0x55ab62119630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:13.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.200+0000 7f8c07e67740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ab62071620 con 0x55ab6210c060 2024-09-12T03:10:13.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.200+0000 7f8c07e67740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ab62018e50 con 0x55ab6210c980 2024-09-12T03:10:13.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.200+0000 7f8c07e67740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ab6206d5e0 con 0x55ab62117240 2024-09-12T03:10:13.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.200+0000 7f8bfffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab6210c060 0x55ab6210c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:13.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.200+0000 7f8bfcff9640 1 --2- 172.21.15.116:0/3761568903 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8be80a2c00 0x7f8be80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:13.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.200+0000 7f8bfffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab6210c060 0x55ab6210c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33428/0 (socket says 172.21.15.116:33428) 2024-09-12T03:10:13.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.200+0000 7f8bfffff640 1 -- 172.21.15.116:0/2231560069 learned_addr learned my addr 172.21.15.116:0/2231560069 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:13.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.201+0000 7f8bff7fe640 1 --2- 172.21.15.116:0/2231560069 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab6210c980 0x55ab62116d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:13.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.201+0000 7f8bfeffd640 1 --2- 172.21.15.116:0/2231560069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab62117240 0x55ab62119630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:13.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.201+0000 7f8bfffff640 1 --2- 172.21.15.116:0/3761568903 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8be80a2c00 0x7f8be80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:13.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.201+0000 7f8bfcff9640 1 -- 172.21.15.116:0/3761568903 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f8bf00cf0b0 con 0x7f8c00100a80 2024-09-12T03:10:13.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.201+0000 7f8bfffff640 1 -- 172.21.15.116:0/2231560069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab62117240 msgr2=0x55ab62119630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:13.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.201+0000 7f8bfffff640 1 --2- 172.21.15.116:0/2231560069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab62117240 0x55ab62119630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.201+0000 7f8bfffff640 1 -- 172.21.15.116:0/2231560069 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab6210c980 msgr2=0x55ab62116d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:13.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.201+0000 7f8bfffff640 1 --2- 172.21.15.116:0/2231560069 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab6210c980 0x55ab62116d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.201+0000 7f8bfffff640 1 -- 172.21.15.116:0/2231560069 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab62009d40 con 0x55ab6210c060 2024-09-12T03:10:13.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.201+0000 7f8bfeffd640 1 --2- 172.21.15.116:0/2231560069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab62117240 0x55ab62119630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:10:13.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.201+0000 7f8bfffff640 1 --2- 172.21.15.116:0/2231560069 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab6210c060 0x55ab6210c440 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f8bf80147a0 tx=0x7f8bf8014c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d7f415dc559282 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:13.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.201+0000 7f8be5ffb640 1 -- 172.21.15.116:0/2231560069 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8009c30 con 0x55ab6210c060 2024-09-12T03:10:13.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.201+0000 7f8be5ffb640 1 -- 172.21.15.116:0/2231560069 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bf8012e50 con 0x55ab6210c060 2024-09-12T03:10:13.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.201+0000 7f8be5ffb640 1 -- 172.21.15.116:0/2231560069 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf801e440 con 0x55ab6210c060 2024-09-12T03:10:13.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.201+0000 7f8bfffff640 1 --2- 172.21.15.116:0/3761568903 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8be80a2c00 0x7f8be80a50c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8021000 tx=0x7f8bf8020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:13.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.202+0000 7f8c07e67740 1 -- 172.21.15.116:0/2231560069 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab6210c060 msgr2=0x55ab6210c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:13.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.202+0000 7f8c07e67740 1 --2- 172.21.15.116:0/2231560069 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab6210c060 0x55ab6210c440 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f8bf80147a0 tx=0x7f8bf8014c60 comp rx=0 tx=0).stop 2024-09-12T03:10:13.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.202+0000 7f8c07e67740 1 -- 172.21.15.116:0/2231560069 shutdown_connections 2024-09-12T03:10:13.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.202+0000 7f8c07e67740 1 --2- 172.21.15.116:0/2231560069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab62117240 0x55ab62119630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.202+0000 7f8c07e67740 1 --2- 172.21.15.116:0/2231560069 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab6210c980 0x55ab62116d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.202+0000 7f8c07e67740 1 --2- 172.21.15.116:0/2231560069 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab6210c060 0x55ab6210c440 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.202+0000 7f8c07e67740 1 -- 172.21.15.116:0/2231560069 >> 172.21.15.116:0/2231560069 conn(0x55ab621085f0 msgr2=0x55ab621089d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:13.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.202+0000 7f8c07e67740 1 -- 172.21.15.116:0/2231560069 shutdown_connections 2024-09-12T03:10:13.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.202+0000 7f8c07e67740 1 -- 172.21.15.116:0/2231560069 wait complete. 2024-09-12T03:10:13.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.202+0000 7f8c07e67740 1 -- start start 2024-09-12T03:10:13.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.203+0000 7f8c07e67740 1 Processor -- start 2024-09-12T03:10:13.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.203+0000 7f8c07e67740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab6210c060 0x55ab62114110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:13.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.203+0000 7f8c07e67740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab6210c980 0x55ab62114650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:13.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.203+0000 7f8c07e67740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab62117240 0x55ab62114b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:13.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.203+0000 7f8c07e67740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ab62119e10 con 0x55ab6210c980 2024-09-12T03:10:13.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.203+0000 7f8c07e67740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ab61ff9f10 con 0x55ab6210c060 2024-09-12T03:10:13.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.203+0000 7f8c07e67740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ab62119f90 con 0x55ab62117240 2024-09-12T03:10:13.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.203+0000 7f8c07e67740 10 client.0 _ll_register_callbacks cb 0x55ab6201ddb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:13.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.203+0000 7f8bfeffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab6210c980 0x55ab62114650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:13.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.203+0000 7f8bfeffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab6210c980 0x55ab62114650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33444/0 (socket says 172.21.15.116:33444) 2024-09-12T03:10:13.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.203+0000 7f8bfeffd640 1 -- 172.21.15.116:0/753915091 learned_addr learned my addr 172.21.15.116:0/753915091 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:13.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.203+0000 7f8bfffff640 1 --2- 172.21.15.116:0/753915091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab62117240 0x55ab62114b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:13.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.203+0000 7f8bfeffd640 1 -- 172.21.15.116:0/753915091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab62117240 msgr2=0x55ab62114b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:13.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.203+0000 7f8bfeffd640 1 --2- 172.21.15.116:0/753915091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab62117240 0x55ab62114b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.203+0000 7f8bfeffd640 1 -- 172.21.15.116:0/753915091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab6210c060 msgr2=0x55ab62114110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:13.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.203+0000 7f8bfeffd640 1 --2- 172.21.15.116:0/753915091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab6210c060 0x55ab62114110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.203+0000 7f8bfeffd640 1 -- 172.21.15.116:0/753915091 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab61d80580 con 0x55ab6210c980 2024-09-12T03:10:13.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.203+0000 7f8bfffff640 1 --2- 172.21.15.116:0/753915091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab62117240 0x55ab62114b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:13.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.203+0000 7f8bfeffd640 1 --2- 172.21.15.116:0/753915091 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab6210c980 0x55ab62114650 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009930 tx=0x7f8bf40317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d0372e586f37a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:13.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.203+0000 7f8bdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:13.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.204+0000 7f8bdeffd640 1 -- 172.21.15.116:0/753915091 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4035030 con 0x55ab6210c980 2024-09-12T03:10:13.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.204+0000 7f8bdeffd640 1 -- 172.21.15.116:0/753915091 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f8bf4004260 con 0x55ab6210c980 2024-09-12T03:10:13.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.204+0000 7f8c07e67740 1 -- 172.21.15.116:0/753915091 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bf8012e50 con 0x55ab6210c980 2024-09-12T03:10:13.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.204+0000 7f8bdeffd640 1 -- 172.21.15.116:0/753915091 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8bf4004460 con 0x55ab6210c980 2024-09-12T03:10:13.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.204+0000 7f8c07e67740 10 client.17184 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:13.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.204+0000 7f8c07e67740 1 -- 172.21.15.116:0/753915091 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ab61e4a960 con 0x55ab6210c980 2024-09-12T03:10:13.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.204+0000 7f8bdeffd640 1 -- 172.21.15.116:0/753915091 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bf8012e50 con 0x55ab6210c980 2024-09-12T03:10:13.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.204+0000 7f8bdeffd640 1 -- 172.21.15.116:0/753915091 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8bf40035f0 con 0x55ab6210c980 2024-09-12T03:10:13.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.204+0000 7f8c07e67740 10 client.17184 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:13.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.204+0000 7f8c07e67740 10 client.17184 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:10:13.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.204+0000 7f8c07e67740 20 client.17184 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:13.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.204+0000 7f8c07e67740 1 --2- 172.21.15.116:0/753915091 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ab62126de0 0x55ab62129280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:13.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.204+0000 7f8c07e67740 4 client.17184 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:10:13.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.204+0000 7f8c07e67740 1 -- 172.21.15.116:0/753915091 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab6207eb90 con 0x55ab62126de0 2024-09-12T03:10:13.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.205+0000 7f8bff7fe640 1 --2- 172.21.15.116:0/753915091 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ab62126de0 0x55ab62129280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:13.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.205+0000 7f8bff7fe640 1 --2- 172.21.15.116:0/753915091 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ab62126de0 0x55ab62129280 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63c4b0627d1f128e server_cookie=72c70d5c8961f037 in_seq=0 out_seq=0 2024-09-12T03:10:13.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.205+0000 7f8bdeffd640 10 client.17184 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:13.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.206+0000 7f8bdeffd640 1 -- 172.21.15.116:0/753915091 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ab6207eb90 con 0x55ab62126de0 2024-09-12T03:10:13.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.206+0000 7f8bdeffd640 10 client.17184 handle_command_reply: tid=0 2024-09-12T03:10:13.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.206+0000 7f8c07e67740 1 client.17184 shutdown 2024-09-12T03:10:13.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.206+0000 7f8c07e67740 1 -- 172.21.15.116:0/753915091 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab6210c980 msgr2=0x55ab62114650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:13.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.206+0000 7f8c07e67740 1 --2- 172.21.15.116:0/753915091 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab6210c980 0x55ab62114650 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009930 tx=0x7f8bf40317e0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.206+0000 7f8c07e67740 1 -- 172.21.15.116:0/753915091 shutdown_connections 2024-09-12T03:10:13.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.206+0000 7f8c07e67740 1 --2- 172.21.15.116:0/753915091 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ab62126de0 0x55ab62129280 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.206+0000 7f8c07e67740 1 --2- 172.21.15.116:0/753915091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab62117240 0x55ab62114b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.206+0000 7f8c07e67740 1 --2- 172.21.15.116:0/753915091 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab6210c980 0x55ab62114650 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.206+0000 7f8c07e67740 1 --2- 172.21.15.116:0/753915091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab6210c060 0x55ab62114110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.206+0000 7f8bdeffd640 0 client.17184 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:13.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.206+0000 7f8c07e67740 1 -- 172.21.15.116:0/753915091 >> 172.21.15.116:0/753915091 conn(0x55ab621085f0 msgr2=0x55ab6210e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:13.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.206+0000 7f8c07e67740 1 -- 172.21.15.116:0/753915091 shutdown_connections 2024-09-12T03:10:13.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.206+0000 7f8c07e67740 1 -- 172.21.15.116:0/753915091 wait complete. 2024-09-12T03:10:13.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.206+0000 7f8c07e67740 20 client.17184 trim_cache size 0 max 16384 2024-09-12T03:10:13.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.223+0000 7f8c07e67740 1 Processor -- start 2024-09-12T03:10:13.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.223+0000 7f8c07e67740 1 -- start start 2024-09-12T03:10:13.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.223+0000 7f8c07e67740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab6210c060 0x55ab62113070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:13.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.223+0000 7f8c07e67740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab6210c980 0x55ab6210eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:13.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.223+0000 7f8c07e67740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab62117240 0x55ab6212eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:13.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.223+0000 7f8c07e67740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8c001adbd0 con 0x55ab6210c980 2024-09-12T03:10:13.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.223+0000 7f8c07e67740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8c001ab4a0 con 0x55ab62117240 2024-09-12T03:10:13.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.223+0000 7f8c07e67740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8c001a92f0 con 0x55ab6210c060 2024-09-12T03:10:13.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.223+0000 7f8bfeffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab6210c980 0x55ab6210eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:13.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.223+0000 7f8bfeffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab6210c980 0x55ab6210eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33460/0 (socket says 172.21.15.116:33460) 2024-09-12T03:10:13.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.223+0000 7f8bfeffd640 1 -- 172.21.15.116:0/1426526764 learned_addr learned my addr 172.21.15.116:0/1426526764 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:13.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.223+0000 7f8bff7fe640 1 --2- 172.21.15.116:0/1426526764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab6210c060 0x55ab62113070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:13.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.223+0000 7f8bfffff640 1 --2- 172.21.15.116:0/1426526764 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab62117240 0x55ab6212eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:13.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.224+0000 7f8bfeffd640 1 -- 172.21.15.116:0/1426526764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab6210c060 msgr2=0x55ab62113070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:13.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.224+0000 7f8bfeffd640 1 --2- 172.21.15.116:0/1426526764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab6210c060 0x55ab62113070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.224+0000 7f8bfeffd640 1 -- 172.21.15.116:0/1426526764 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab62117240 msgr2=0x55ab6212eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:13.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.224+0000 7f8bfeffd640 1 --2- 172.21.15.116:0/1426526764 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab62117240 0x55ab6212eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.224+0000 7f8bfeffd640 1 -- 172.21.15.116:0/1426526764 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab62112680 con 0x55ab6210c980 2024-09-12T03:10:13.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.224+0000 7f8bfeffd640 1 --2- 172.21.15.116:0/1426526764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab6210c980 0x55ab6210eb10 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4003bd0 tx=0x7f8bf4005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ce496f8535a4da server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:13.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.224+0000 7f8be57fa640 1 -- 172.21.15.116:0/1426526764 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4035030 con 0x55ab6210c980 2024-09-12T03:10:13.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.224+0000 7f8be57fa640 1 -- 172.21.15.116:0/1426526764 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bf4031d30 con 0x55ab6210c980 2024-09-12T03:10:13.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.224+0000 7f8be57fa640 1 -- 172.21.15.116:0/1426526764 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4032550 con 0x55ab6210c980 2024-09-12T03:10:13.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.225+0000 7f8c07e67740 1 -- 172.21.15.116:0/1426526764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab6210c980 msgr2=0x55ab6210eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:13.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.225+0000 7f8c07e67740 1 --2- 172.21.15.116:0/1426526764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab6210c980 0x55ab6210eb10 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4003bd0 tx=0x7f8bf4005b50 comp rx=0 tx=0).stop 2024-09-12T03:10:13.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.225+0000 7f8c07e67740 1 -- 172.21.15.116:0/1426526764 shutdown_connections 2024-09-12T03:10:13.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.225+0000 7f8c07e67740 1 --2- 172.21.15.116:0/1426526764 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab62117240 0x55ab6212eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.225+0000 7f8c07e67740 1 --2- 172.21.15.116:0/1426526764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab6210c980 0x55ab6210eb10 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.225+0000 7f8c07e67740 1 --2- 172.21.15.116:0/1426526764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab6210c060 0x55ab62113070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.225+0000 7f8c07e67740 1 -- 172.21.15.116:0/1426526764 >> 172.21.15.116:0/1426526764 conn(0x55ab621085f0 msgr2=0x55ab62116190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:13.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.225+0000 7f8c07e67740 1 -- 172.21.15.116:0/1426526764 shutdown_connections 2024-09-12T03:10:13.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.225+0000 7f8c07e67740 1 -- 172.21.15.116:0/1426526764 wait complete. 2024-09-12T03:10:13.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.225+0000 7f8c07e67740 1 -- start start 2024-09-12T03:10:13.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.225+0000 7f8c07e67740 1 Processor -- start 2024-09-12T03:10:13.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.225+0000 7f8c07e67740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab6210c060 0x55ab6211f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:13.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.225+0000 7f8c07e67740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab6210c980 0x55ab6211f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:13.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.225+0000 7f8c07e67740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab62117240 0x55ab6211c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:13.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.225+0000 7f8c07e67740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8c001a9960 con 0x55ab62117240 2024-09-12T03:10:13.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.225+0000 7f8c07e67740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8c001aadd0 con 0x55ab6210c060 2024-09-12T03:10:13.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.226+0000 7f8c07e67740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8c001aa030 con 0x55ab6210c980 2024-09-12T03:10:13.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.226+0000 7f8c07e67740 10 client.0 _ll_register_callbacks cb 0x7f8bf4005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:13.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.226+0000 7f8bfffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab62117240 0x55ab6211c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:13.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.226+0000 7f8bfffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab62117240 0x55ab6211c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33470/0 (socket says 172.21.15.116:33470) 2024-09-12T03:10:13.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.226+0000 7f8bfffff640 1 -- 172.21.15.116:0/623547398 learned_addr learned my addr 172.21.15.116:0/623547398 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:13.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.226+0000 7f8bff7fe640 1 --2- 172.21.15.116:0/623547398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab6210c060 0x55ab6211f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:13.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.226+0000 7f8bfeffd640 1 --2- 172.21.15.116:0/623547398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab6210c980 0x55ab6211f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:13.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.226+0000 7f8bfffff640 1 -- 172.21.15.116:0/623547398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab6210c980 msgr2=0x55ab6211f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:13.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.226+0000 7f8bfffff640 1 --2- 172.21.15.116:0/623547398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab6210c980 0x55ab6211f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.226+0000 7f8bfffff640 1 -- 172.21.15.116:0/623547398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab6210c060 msgr2=0x55ab6211f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:13.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.226+0000 7f8bfffff640 1 --2- 172.21.15.116:0/623547398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab6210c060 0x55ab6211f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.226+0000 7f8bfffff640 1 -- 172.21.15.116:0/623547398 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab6211c6f0 con 0x55ab62117240 2024-09-12T03:10:13.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.226+0000 7f8bfffff640 1 --2- 172.21.15.116:0/623547398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab62117240 0x55ab6211c1b0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8009460 tx=0x7f8bf8014500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff51c3bda4d0a6f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:13.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.226+0000 7f8bdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:13.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.226+0000 7f8bdeffd640 1 -- 172.21.15.116:0/623547398 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8037060 con 0x55ab62117240 2024-09-12T03:10:13.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.226+0000 7f8bdeffd640 1 -- 172.21.15.116:0/623547398 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f8bf8012970 con 0x55ab62117240 2024-09-12T03:10:13.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.226+0000 7f8c07e67740 1 -- 172.21.15.116:0/623547398 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bf4031d30 con 0x55ab62117240 2024-09-12T03:10:13.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.226+0000 7f8bdeffd640 1 -- 172.21.15.116:0/623547398 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8bf8012b70 con 0x55ab62117240 2024-09-12T03:10:13.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.226+0000 7f8c07e67740 10 client.17190 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:13.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.226+0000 7f8c07e67740 1 -- 172.21.15.116:0/623547398 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ab62192220 con 0x55ab62117240 2024-09-12T03:10:13.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.227+0000 7f8bdeffd640 1 -- 172.21.15.116:0/623547398 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bf4031d30 con 0x55ab62117240 2024-09-12T03:10:13.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.227+0000 7f8bdeffd640 1 -- 172.21.15.116:0/623547398 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8bf800f040 con 0x55ab62117240 2024-09-12T03:10:13.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.227+0000 7f8c07e67740 10 client.17190 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:13.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.227+0000 7f8c07e67740 10 client.17190 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:10:13.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.227+0000 7f8c07e67740 20 client.17190 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:13.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.227+0000 7f8c07e67740 1 --2- 172.21.15.116:0/623547398 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ab62192710 0x55ab62194b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:13.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.227+0000 7f8c07e67740 4 client.17190 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4007087b", "release": true, "await": true}] 2024-09-12T03:10:13.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.227+0000 7f8c07e67740 1 -- 172.21.15.116:0/623547398 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4007087b", "release": true, "await": true}) -- 0x55ab620790a0 con 0x55ab62192710 2024-09-12T03:10:13.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.227+0000 7f8bff7fe640 1 --2- 172.21.15.116:0/623547398 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ab62192710 0x55ab62194b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:13.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.228+0000 7f8bdeffd640 10 client.17190 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:13.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.228+0000 7f8bff7fe640 1 --2- 172.21.15.116:0/623547398 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ab62192710 0x55ab62194b00 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a81682be1b90abf server_cookie=4d59271f7acbc38b in_seq=0 out_seq=0 2024-09-12T03:10:13.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.231+0000 7f8bdeffd640 1 -- 172.21.15.116:0/623547398 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ab620790a0 con 0x55ab62192710 2024-09-12T03:10:13.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.231+0000 7f8bdeffd640 10 client.17190 handle_command_reply: tid=0 2024-09-12T03:10:13.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.231+0000 7f8c07e67740 1 client.17190 shutdown 2024-09-12T03:10:13.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.231+0000 7f8c07e67740 1 -- 172.21.15.116:0/623547398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab62117240 msgr2=0x55ab6211c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:13.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.231+0000 7f8c07e67740 1 --2- 172.21.15.116:0/623547398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab62117240 0x55ab6211c1b0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8009460 tx=0x7f8bf8014500 comp rx=0 tx=0).stop 2024-09-12T03:10:13.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.231+0000 7f8c07e67740 1 -- 172.21.15.116:0/623547398 shutdown_connections 2024-09-12T03:10:13.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.231+0000 7f8c07e67740 1 --2- 172.21.15.116:0/623547398 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ab62192710 0x55ab62194b00 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.231+0000 7f8c07e67740 1 --2- 172.21.15.116:0/623547398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ab62117240 0x55ab6211c1b0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.231+0000 7f8c07e67740 1 --2- 172.21.15.116:0/623547398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ab6210c980 0x55ab6211f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.231+0000 7f8bdeffd640 0 client.17190 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:13.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.231+0000 7f8c07e67740 1 --2- 172.21.15.116:0/623547398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ab6210c060 0x55ab6211f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.231+0000 7f8c07e67740 1 -- 172.21.15.116:0/623547398 >> 172.21.15.116:0/623547398 conn(0x55ab621085f0 msgr2=0x55ab62106f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:13.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.231+0000 7f8c07e67740 1 -- 172.21.15.116:0/623547398 shutdown_connections 2024-09-12T03:10:13.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.231+0000 7f8c07e67740 1 -- 172.21.15.116:0/623547398 wait complete. 2024-09-12T03:10:13.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.231+0000 7f8c07e67740 20 client.17190 trim_cache size 0 max 16384 2024-09-12T03:10:13.236 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:10:13.236 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 25, 2024-09-12T03:10:13.236 INFO:teuthology.orchestra.run.smithi116.stdout: "leader": 24494, 2024-09-12T03:10:13.236 INFO:teuthology.orchestra.run.smithi116.stdout: "set_version": 28, 2024-09-12T03:10:13.236 INFO:teuthology.orchestra.run.smithi116.stdout: "sets": { 2024-09-12T03:10:13.236 INFO:teuthology.orchestra.run.smithi116.stdout: "4007087b": { 2024-09-12T03:10:13.236 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 28, 2024-09-12T03:10:13.236 INFO:teuthology.orchestra.run.smithi116.stdout: "age_ref": 0.0, 2024-09-12T03:10:13.237 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:10:13.237 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RELEASED", 2024-09-12T03:10:13.237 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:10:13.237 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:10:13.237 INFO:teuthology.orchestra.run.smithi116.stdout: "timeout": 90.0, 2024-09-12T03:10:13.237 INFO:teuthology.orchestra.run.smithi116.stdout: "expiration": 136.6, 2024-09-12T03:10:13.237 INFO:teuthology.orchestra.run.smithi116.stdout: "members": { 2024-09-12T03:10:13.237 INFO:teuthology.orchestra.run.smithi116.stdout: "file:/": { 2024-09-12T03:10:13.237 INFO:teuthology.orchestra.run.smithi116.stdout: "excluded": false, 2024-09-12T03:10:13.237 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:10:13.237 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RELEASED", 2024-09-12T03:10:13.237 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:10:13.237 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:10:13.237 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:10:13.238 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:10:13.238 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:10:13.238 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:10:13.238 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:10:13.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.232+0000 7f8c05a8e640 1 -- 172.21.15.116:0/3761568903 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8be80a2c00 msgr2=0x7f8be80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:13.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.232+0000 7f8c05a8e640 1 --2- 172.21.15.116:0/3761568903 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8be80a2c00 0x7f8be80a50c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8021000 tx=0x7f8bf8020000 comp rx=0 tx=0).stop 2024-09-12T03:10:13.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.232+0000 7f8c05a8e640 1 -- 172.21.15.116:0/3761568903 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c00100a80 msgr2=0x7f8c00198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:13.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.232+0000 7f8c05a8e640 1 --2- 172.21.15.116:0/3761568903 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c00100a80 0x7f8c00198410 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f8bf000cce0 tx=0x7f8bf000a620 comp rx=0 tx=0).stop 2024-09-12T03:10:13.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.232+0000 7f8c05a8e640 1 -- 172.21.15.116:0/3761568903 shutdown_connections 2024-09-12T03:10:13.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.232+0000 7f8c05a8e640 1 --2- 172.21.15.116:0/3761568903 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8be80a2c00 0x7f8be80a50c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.232+0000 7f8c05a8e640 1 --2- 172.21.15.116:0/3761568903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c00106a80 0x7f8c0019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.232+0000 7f8c05a8e640 1 --2- 172.21.15.116:0/3761568903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c00101420 0x7f8c00198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.232+0000 7f8c05a8e640 1 --2- 172.21.15.116:0/3761568903 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c00100a80 0x7f8c00198410 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.232+0000 7f8c05a8e640 1 -- 172.21.15.116:0/3761568903 >> 172.21.15.116:0/3761568903 conn(0x7f8c000fc7a0 msgr2=0x7f8c0006aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:13.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.232+0000 7f8c05a8e640 1 -- 172.21.15.116:0/3761568903 shutdown_connections 2024-09-12T03:10:13.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:13.232+0000 7f8c05a8e640 1 -- 172.21.15.116:0/3761568903 wait complete. 2024-09-12T03:10:13.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:13 smithi116 ceph-mon[31410]: pgmap v555: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:10:13.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:13 smithi183 ceph-mon[35535]: pgmap v555: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:10:13.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:13 smithi177 ceph-mon[35305]: pgmap v555: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:10:14.059 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4007087b', total seconds elapsed: 20.1 2024-09-12T03:10:16.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:16 smithi183 ceph-mon[35535]: pgmap v556: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:10:16.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:16 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:10:16.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:16 smithi177 ceph-mon[35305]: pgmap v556: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:10:16.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:16 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:10:16.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:16 smithi116 ceph-mon[31410]: pgmap v556: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:10:16.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:16 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:10:17.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:17 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:10:17.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:17 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:10:17.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:17 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:10:17.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:17 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:10:17.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:17 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:10:17.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:17 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:10:17.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:17 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:10:17.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:17 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:10:17.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:17 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:10:17.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:17 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:10:17.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:17 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:10:17.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:17 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:10:17.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:17 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:10:17.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:17 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:10:17.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:17 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:10:17.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:17 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:10:17.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:17 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:10:17.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:17 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:10:17.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:17 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:10:17.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:17 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:10:17.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:17 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:10:17.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:17 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:10:17.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:17 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:10:17.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:17 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:10:17.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:17 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:10:17.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:17 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:10:17.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:17 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:10:17.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:17 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:10:17.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:17 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:10:17.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:17 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:10:17.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:17 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:10:17.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:17 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:10:17.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:17 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:10:18.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:18 smithi183 ceph-mon[35535]: Adjusting osd_memory_target on smithi177 to 175.9M 2024-09-12T03:10:18.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:18 smithi183 ceph-mon[35535]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-09-12T03:10:18.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:18 smithi183 ceph-mon[35535]: Unable to set osd_memory_target on smithi177 to 184530124: error parsing value: Value '184530124' is below minimum 939524096 2024-09-12T03:10:18.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:18 smithi183 ceph-mon[35535]: Unable to set osd_memory_target on smithi183 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-12T03:10:18.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:18 smithi183 ceph-mon[35535]: pgmap v557: 97 pgs: 97 active+clean; 3.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-09-12T03:10:18.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:18 smithi177 ceph-mon[35305]: Adjusting osd_memory_target on smithi177 to 175.9M 2024-09-12T03:10:18.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:18 smithi177 ceph-mon[35305]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-09-12T03:10:18.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:18 smithi177 ceph-mon[35305]: Unable to set osd_memory_target on smithi177 to 184530124: error parsing value: Value '184530124' is below minimum 939524096 2024-09-12T03:10:18.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:18 smithi177 ceph-mon[35305]: Unable to set osd_memory_target on smithi183 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-12T03:10:18.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:18 smithi177 ceph-mon[35305]: pgmap v557: 97 pgs: 97 active+clean; 3.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-09-12T03:10:18.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:18 smithi116 ceph-mon[31410]: Adjusting osd_memory_target on smithi177 to 175.9M 2024-09-12T03:10:18.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:18 smithi116 ceph-mon[31410]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-09-12T03:10:18.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:18 smithi116 ceph-mon[31410]: Unable to set osd_memory_target on smithi177 to 184530124: error parsing value: Value '184530124' is below minimum 939524096 2024-09-12T03:10:18.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:18 smithi116 ceph-mon[31410]: Unable to set osd_memory_target on smithi183 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-12T03:10:18.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:18 smithi116 ceph-mon[31410]: pgmap v557: 97 pgs: 97 active+clean; 3.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-09-12T03:10:19.273 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.273 INFO:tasks.workunit.client.0.smithi116.stdout:189/600: open fd = 3 2024-09-12T03:10:19.274 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.274 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.274 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.274 INFO:tasks.workunit.client.0.smithi116.stdout:190/600: open fd = 3 2024-09-12T03:10:19.274 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.274 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.274 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.274 INFO:tasks.workunit.client.0.smithi116.stdout:191/600: open fd = 3 2024-09-12T03:10:19.274 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.274 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.274 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.274 INFO:tasks.workunit.client.0.smithi116.stdout:192/600: open fd = 3 2024-09-12T03:10:19.274 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.275 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.275 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.275 INFO:tasks.workunit.client.0.smithi116.stdout:193/600: open fd = 3 2024-09-12T03:10:19.275 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.275 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.275 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.275 INFO:tasks.workunit.client.0.smithi116.stdout:194/600: open fd = 3 2024-09-12T03:10:19.275 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.275 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.275 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.275 INFO:tasks.workunit.client.0.smithi116.stdout:195/600: open fd = 3 2024-09-12T03:10:19.275 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.275 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.276 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.276 INFO:tasks.workunit.client.0.smithi116.stdout:196/600: open fd = 3 2024-09-12T03:10:19.276 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.276 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.276 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.276 INFO:tasks.workunit.client.0.smithi116.stdout:197/600: open fd = 3 2024-09-12T03:10:19.276 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.276 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.276 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.276 INFO:tasks.workunit.client.0.smithi116.stdout:198/600: open fd = 3 2024-09-12T03:10:19.276 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.276 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.276 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.277 INFO:tasks.workunit.client.0.smithi116.stdout:199/600: open fd = 3 2024-09-12T03:10:19.277 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.277 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.277 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.277 INFO:tasks.workunit.client.0.smithi116.stdout:200/600: open fd = 3 2024-09-12T03:10:19.277 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.277 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.277 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.277 INFO:tasks.workunit.client.0.smithi116.stdout:201/600: open fd = 3 2024-09-12T03:10:19.277 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.277 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.277 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.278 INFO:tasks.workunit.client.0.smithi116.stdout:202/600: open fd = 3 2024-09-12T03:10:19.278 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.278 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.278 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.278 INFO:tasks.workunit.client.0.smithi116.stdout:203/600: open fd = 3 2024-09-12T03:10:19.278 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.278 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.278 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.278 INFO:tasks.workunit.client.0.smithi116.stdout:204/600: open fd = 3 2024-09-12T03:10:19.278 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.278 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.278 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.278 INFO:tasks.workunit.client.0.smithi116.stdout:205/600: open fd = 3 2024-09-12T03:10:19.279 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.279 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.279 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.279 INFO:tasks.workunit.client.0.smithi116.stdout:206/600: open fd = 3 2024-09-12T03:10:19.279 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.279 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.279 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.279 INFO:tasks.workunit.client.0.smithi116.stdout:207/600: open fd = 3 2024-09-12T03:10:19.279 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.279 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.279 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.279 INFO:tasks.workunit.client.0.smithi116.stdout:208/600: open fd = 3 2024-09-12T03:10:19.279 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.280 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.280 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.280 INFO:tasks.workunit.client.0.smithi116.stdout:209/600: open fd = 3 2024-09-12T03:10:19.280 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.280 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.280 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.280 INFO:tasks.workunit.client.0.smithi116.stdout:210/600: open fd = 3 2024-09-12T03:10:19.280 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.280 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.280 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.280 INFO:tasks.workunit.client.0.smithi116.stdout:211/600: open fd = 3 2024-09-12T03:10:19.280 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.280 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.281 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.281 INFO:tasks.workunit.client.0.smithi116.stdout:212/600: open fd = 3 2024-09-12T03:10:19.281 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.281 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.281 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.281 INFO:tasks.workunit.client.0.smithi116.stdout:213/600: open fd = 3 2024-09-12T03:10:19.281 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.281 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.281 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.281 INFO:tasks.workunit.client.0.smithi116.stdout:214/600: open fd = 3 2024-09-12T03:10:19.281 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.281 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.281 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.282 INFO:tasks.workunit.client.0.smithi116.stdout:215/600: open fd = 3 2024-09-12T03:10:19.282 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.282 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.282 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.282 INFO:tasks.workunit.client.0.smithi116.stdout:216/600: open fd = 3 2024-09-12T03:10:19.282 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.282 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.282 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.282 INFO:tasks.workunit.client.0.smithi116.stdout:217/600: open fd = 3 2024-09-12T03:10:19.282 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.282 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.282 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.282 INFO:tasks.workunit.client.0.smithi116.stdout:218/600: open fd = 3 2024-09-12T03:10:19.282 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.283 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.283 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.283 INFO:tasks.workunit.client.0.smithi116.stdout:219/600: open fd = 3 2024-09-12T03:10:19.283 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.283 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.283 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.283 INFO:tasks.workunit.client.0.smithi116.stdout:220/600: open fd = 3 2024-09-12T03:10:19.283 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.283 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.283 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.283 INFO:tasks.workunit.client.0.smithi116.stdout:221/600: open fd = 3 2024-09-12T03:10:19.283 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.283 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.283 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.284 INFO:tasks.workunit.client.0.smithi116.stdout:222/600: open fd = 3 2024-09-12T03:10:19.284 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.284 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.284 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.284 INFO:tasks.workunit.client.0.smithi116.stdout:223/600: open fd = 3 2024-09-12T03:10:19.284 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.284 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.284 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.284 INFO:tasks.workunit.client.0.smithi116.stdout:224/600: open fd = 3 2024-09-12T03:10:19.284 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.284 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.284 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.284 INFO:tasks.workunit.client.0.smithi116.stdout:225/600: open fd = 3 2024-09-12T03:10:19.285 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.285 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.285 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.285 INFO:tasks.workunit.client.0.smithi116.stdout:226/600: open fd = 3 2024-09-12T03:10:19.285 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.285 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.285 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.285 INFO:tasks.workunit.client.0.smithi116.stdout:227/600: open fd = 3 2024-09-12T03:10:19.285 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.285 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.285 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.285 INFO:tasks.workunit.client.0.smithi116.stdout:228/600: open fd = 3 2024-09-12T03:10:19.285 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.286 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.286 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.286 INFO:tasks.workunit.client.0.smithi116.stdout:229/600: open fd = 3 2024-09-12T03:10:19.286 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.286 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.286 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.286 INFO:tasks.workunit.client.0.smithi116.stdout:230/600: open fd = 3 2024-09-12T03:10:19.286 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.286 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.286 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.286 INFO:tasks.workunit.client.0.smithi116.stdout:231/600: open fd = 3 2024-09-12T03:10:19.286 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.286 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.286 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.287 INFO:tasks.workunit.client.0.smithi116.stdout:232/600: open fd = 3 2024-09-12T03:10:19.287 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.287 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.287 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.287 INFO:tasks.workunit.client.0.smithi116.stdout:233/600: open fd = 3 2024-09-12T03:10:19.287 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.287 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.287 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.287 INFO:tasks.workunit.client.0.smithi116.stdout:234/600: open fd = 3 2024-09-12T03:10:19.287 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.287 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.287 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.287 INFO:tasks.workunit.client.0.smithi116.stdout:235/600: open fd = 3 2024-09-12T03:10:19.288 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.288 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.288 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.288 INFO:tasks.workunit.client.0.smithi116.stdout:236/600: open fd = 3 2024-09-12T03:10:19.288 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.288 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.288 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.288 INFO:tasks.workunit.client.0.smithi116.stdout:237/600: open fd = 3 2024-09-12T03:10:19.288 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.288 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.288 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.288 INFO:tasks.workunit.client.0.smithi116.stdout:238/600: open fd = 3 2024-09-12T03:10:19.288 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.288 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.289 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.289 INFO:tasks.workunit.client.0.smithi116.stdout:239/600: open fd = 3 2024-09-12T03:10:19.289 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.289 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.289 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.289 INFO:tasks.workunit.client.0.smithi116.stdout:240/600: open fd = 3 2024-09-12T03:10:19.289 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.289 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.289 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.289 INFO:tasks.workunit.client.0.smithi116.stdout:241/600: open fd = 3 2024-09-12T03:10:19.289 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.289 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.289 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.290 INFO:tasks.workunit.client.0.smithi116.stdout:242/600: open fd = 3 2024-09-12T03:10:19.290 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.290 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.290 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.293 INFO:tasks.workunit.client.0.smithi116.stdout:243/600: open fd = 3 2024-09-12T03:10:19.293 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.293 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.293 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.293 INFO:tasks.workunit.client.0.smithi116.stdout:244/600: open fd = 3 2024-09-12T03:10:19.293 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.293 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.293 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.293 INFO:tasks.workunit.client.0.smithi116.stdout:245/600: open fd = 3 2024-09-12T03:10:19.293 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.293 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.293 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.293 INFO:tasks.workunit.client.0.smithi116.stdout:246/600: open fd = 3 2024-09-12T03:10:19.294 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.294 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.294 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.294 INFO:tasks.workunit.client.0.smithi116.stdout:247/600: open fd = 3 2024-09-12T03:10:19.294 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.294 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.294 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.294 INFO:tasks.workunit.client.0.smithi116.stdout:248/600: open fd = 3 2024-09-12T03:10:19.294 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.294 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.294 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.294 INFO:tasks.workunit.client.0.smithi116.stdout:249/600: open fd = 3 2024-09-12T03:10:19.294 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.294 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.295 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:10:19.295 INFO:tasks.workunit.client.0.smithi116.stdout:250/600: open fd = 3 2024-09-12T03:10:19.295 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:19.295 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:10:20.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:20 smithi183 ceph-mon[35535]: pgmap v558: 97 pgs: 97 active+clean; 3.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.7 KiB/s wr, 7 op/s 2024-09-12T03:10:20.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:20 smithi177 ceph-mon[35305]: pgmap v558: 97 pgs: 97 active+clean; 3.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.7 KiB/s wr, 7 op/s 2024-09-12T03:10:20.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:20 smithi116 ceph-mon[31410]: pgmap v558: 97 pgs: 97 active+clean; 3.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.7 KiB/s wr, 7 op/s 2024-09-12T03:10:22.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:22 smithi183 ceph-mon[35535]: pgmap v559: 97 pgs: 97 active+clean; 3.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 KiB/s wr, 10 op/s 2024-09-12T03:10:22.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:22 smithi177 ceph-mon[35305]: pgmap v559: 97 pgs: 97 active+clean; 3.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 KiB/s wr, 10 op/s 2024-09-12T03:10:22.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:22 smithi116 ceph-mon[31410]: pgmap v559: 97 pgs: 97 active+clean; 3.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 KiB/s wr, 10 op/s 2024-09-12T03:10:24.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:24 smithi183 ceph-mon[35535]: pgmap v560: 97 pgs: 97 active+clean; 3.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 KiB/s wr, 9 op/s 2024-09-12T03:10:24.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:24 smithi177 ceph-mon[35305]: pgmap v560: 97 pgs: 97 active+clean; 3.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 KiB/s wr, 9 op/s 2024-09-12T03:10:24.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:24 smithi116 ceph-mon[31410]: pgmap v560: 97 pgs: 97 active+clean; 3.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 KiB/s wr, 9 op/s 2024-09-12T03:10:26.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:26 smithi183 ceph-mon[35535]: pgmap v561: 97 pgs: 97 active+clean; 3.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 9 op/s 2024-09-12T03:10:26.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:26 smithi177 ceph-mon[35305]: pgmap v561: 97 pgs: 97 active+clean; 3.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 9 op/s 2024-09-12T03:10:26.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:26 smithi116 ceph-mon[31410]: pgmap v561: 97 pgs: 97 active+clean; 3.6 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 9 op/s 2024-09-12T03:10:28.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:28 smithi183 ceph-mon[35535]: pgmap v562: 97 pgs: 97 active+clean; 3.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-09-12T03:10:28.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:28 smithi177 ceph-mon[35305]: pgmap v562: 97 pgs: 97 active+clean; 3.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-09-12T03:10:28.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:28 smithi116 ceph-mon[31410]: pgmap v562: 97 pgs: 97 active+clean; 3.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-09-12T03:10:30.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:30 smithi183 ceph-mon[35535]: pgmap v563: 97 pgs: 97 active+clean; 3.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:10:30.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:30 smithi177 ceph-mon[35305]: pgmap v563: 97 pgs: 97 active+clean; 3.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:10:30.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:30 smithi116 ceph-mon[31410]: pgmap v563: 97 pgs: 97 active+clean; 3.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:10:32.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:32 smithi183 ceph-mon[35535]: pgmap v564: 97 pgs: 97 active+clean; 3.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:10:32.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:32 smithi177 ceph-mon[35305]: pgmap v564: 97 pgs: 97 active+clean; 3.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:10:32.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:32 smithi116 ceph-mon[31410]: pgmap v564: 97 pgs: 97 active+clean; 3.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:10:34.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:34 smithi183 ceph-mon[35535]: pgmap v565: 97 pgs: 97 active+clean; 3.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-12T03:10:34.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:34 smithi177 ceph-mon[35305]: pgmap v565: 97 pgs: 97 active+clean; 3.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-12T03:10:34.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:34 smithi116 ceph-mon[31410]: pgmap v565: 97 pgs: 97 active+clean; 3.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-12T03:10:36.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:36 smithi183 ceph-mon[35535]: pgmap v566: 97 pgs: 97 active+clean; 3.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T03:10:36.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:36 smithi177 ceph-mon[35305]: pgmap v566: 97 pgs: 97 active+clean; 3.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T03:10:36.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:36 smithi116 ceph-mon[31410]: pgmap v566: 97 pgs: 97 active+clean; 3.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T03:10:38.084 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:10:38.343 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:10:38.464 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:38 smithi116 ceph-mon[31410]: pgmap v567: 97 pgs: 97 active+clean; 3.8 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:10:38.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:38 smithi183 ceph-mon[35535]: pgmap v567: 97 pgs: 97 active+clean; 3.8 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:10:38.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:38 smithi177 ceph-mon[35305]: pgmap v567: 97 pgs: 97 active+clean; 3.8 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:10:38.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.872+0000 7fc010a95640 1 -- 172.21.15.116:0/332926313 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc00c101de0 msgr2=0x7fc00c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:38.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.872+0000 7fc010a95640 1 --2- 172.21.15.116:0/332926313 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc00c101de0 0x7fc00c1021c0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009900 tx=0x7fbffc02a410 comp rx=0 tx=0).stop 2024-09-12T03:10:38.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.872+0000 7fc010a95640 1 -- 172.21.15.116:0/332926313 shutdown_connections 2024-09-12T03:10:38.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.872+0000 7fc010a95640 1 --2- 172.21.15.116:0/332926313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc00c106f10 0x7fc00c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.872+0000 7fc010a95640 1 --2- 172.21.15.116:0/332926313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc00c102790 0x7fc00c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.872+0000 7fc010a95640 1 --2- 172.21.15.116:0/332926313 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc00c101de0 0x7fc00c1021c0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.872+0000 7fc010a95640 1 -- 172.21.15.116:0/332926313 >> 172.21.15.116:0/332926313 conn(0x7fc00c0fd8c0 msgr2=0x7fc00c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:38.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.872+0000 7fc010a95640 1 -- 172.21.15.116:0/332926313 shutdown_connections 2024-09-12T03:10:38.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.872+0000 7fc010a95640 1 -- 172.21.15.116:0/332926313 wait complete. 2024-09-12T03:10:38.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.873+0000 7fc010a95640 1 Processor -- start 2024-09-12T03:10:38.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.873+0000 7fc010a95640 1 -- start start 2024-09-12T03:10:38.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.873+0000 7fc010a95640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc00c101de0 0x7fc00c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:38.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.873+0000 7fc010a95640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc00c102790 0x7fc00c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:38.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.873+0000 7fc010a95640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc00c106f10 0x7fc00c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:38.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.873+0000 7fc010a95640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c111f70 con 0x7fc00c102790 2024-09-12T03:10:38.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.873+0000 7fc010a95640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c111df0 con 0x7fc00c101de0 2024-09-12T03:10:38.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.873+0000 7fc010a95640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1120f0 con 0x7fc00c106f10 2024-09-12T03:10:38.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.873+0000 7fc003fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc00c102790 0x7fc00c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:38.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.873+0000 7fc003fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc00c102790 0x7fc00c198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33144/0 (socket says 172.21.15.116:33144) 2024-09-12T03:10:38.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.873+0000 7fc003fff640 1 -- 172.21.15.116:0/3282857512 learned_addr learned my addr 172.21.15.116:0/3282857512 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:38.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.874+0000 7fc00a59c640 1 --2- 172.21.15.116:0/3282857512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc00c101de0 0x7fc00c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:38.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.874+0000 7fc003fff640 1 -- 172.21.15.116:0/3282857512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc00c106f10 msgr2=0x7fc00c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:38.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.874+0000 7fc00ad9d640 1 --2- 172.21.15.116:0/3282857512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc00c106f10 0x7fc00c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:38.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.874+0000 7fc003fff640 1 --2- 172.21.15.116:0/3282857512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc00c106f10 0x7fc00c19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.874+0000 7fc003fff640 1 -- 172.21.15.116:0/3282857512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc00c101de0 msgr2=0x7fc00c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:38.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.874+0000 7fc003fff640 1 --2- 172.21.15.116:0/3282857512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc00c101de0 0x7fc00c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.874+0000 7fc003fff640 1 -- 172.21.15.116:0/3282857512 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00c19d3e0 con 0x7fc00c102790 2024-09-12T03:10:38.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.874+0000 7fc00a59c640 1 --2- 172.21.15.116:0/3282857512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc00c101de0 0x7fc00c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:38.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.874+0000 7fc00ad9d640 1 --2- 172.21.15.116:0/3282857512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc00c106f10 0x7fc00c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:38.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.874+0000 7fc003fff640 1 --2- 172.21.15.116:0/3282857512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc00c102790 0x7fc00c198970 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fbff400d950 tx=0x7fbff400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:38.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.874+0000 7fc0037fe640 1 -- 172.21.15.116:0/3282857512 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff4017070 con 0x7fc00c102790 2024-09-12T03:10:38.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.874+0000 7fc010a95640 1 -- 172.21.15.116:0/3282857512 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc00c19d6d0 con 0x7fc00c102790 2024-09-12T03:10:38.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.874+0000 7fc0037fe640 1 -- 172.21.15.116:0/3282857512 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbff4009c30 con 0x7fc00c102790 2024-09-12T03:10:38.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.874+0000 7fc0037fe640 1 -- 172.21.15.116:0/3282857512 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff400f6b0 con 0x7fc00c102790 2024-09-12T03:10:38.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.874+0000 7fc010a95640 1 -- 172.21.15.116:0/3282857512 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc00c1a4fb0 con 0x7fc00c102790 2024-09-12T03:10:38.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.876+0000 7fc0037fe640 1 -- 172.21.15.116:0/3282857512 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fbff40109d0 con 0x7fc00c102790 2024-09-12T03:10:38.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.877+0000 7fc012e6e740 1 Processor -- start 2024-09-12T03:10:38.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.877+0000 7fc012e6e740 1 -- start start 2024-09-12T03:10:38.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.877+0000 7fc012e6e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55857f52f4a0 0x55857f52f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:38.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.877+0000 7fc012e6e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55857f52fdc0 0x55857f53a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:38.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.877+0000 7fc012e6e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55857f53a680 0x55857f53ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:38.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.877+0000 7fc012e6e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55857f4952e0 con 0x55857f52fdc0 2024-09-12T03:10:38.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.877+0000 7fc012e6e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55857f43cce0 con 0x55857f52f4a0 2024-09-12T03:10:38.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.877+0000 7fc012e6e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55857f4912a0 con 0x55857f53a680 2024-09-12T03:10:38.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.877+0000 7fc00a59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55857f52fdc0 0x55857f53a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:38.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.877+0000 7fc00a59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55857f52fdc0 0x55857f53a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33160/0 (socket says 172.21.15.116:33160) 2024-09-12T03:10:38.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.877+0000 7fc00a59c640 1 -- 172.21.15.116:0/602034645 learned_addr learned my addr 172.21.15.116:0/602034645 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:38.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.877+0000 7fc0037fe640 1 --2- 172.21.15.116:0/3282857512 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbff80a2b80 0x7fbff80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:38.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.877+0000 7fc00ad9d640 1 --2- 172.21.15.116:0/602034645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55857f52f4a0 0x55857f52f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:38.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.877+0000 7fc00ad9d640 1 --2- 172.21.15.116:0/3282857512 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbff80a2b80 0x7fbff80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:38.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.877+0000 7fc0037fe640 1 -- 172.21.15.116:0/3282857512 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fbff40ced90 con 0x7fc00c102790 2024-09-12T03:10:38.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.878+0000 7fc003fff640 1 --2- 172.21.15.116:0/602034645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55857f53a680 0x55857f53ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:38.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.878+0000 7fc00a59c640 1 -- 172.21.15.116:0/602034645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55857f53a680 msgr2=0x55857f53ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:38.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.878+0000 7fc00a59c640 1 --2- 172.21.15.116:0/602034645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55857f53a680 0x55857f53ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.878+0000 7fc00a59c640 1 -- 172.21.15.116:0/602034645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55857f52f4a0 msgr2=0x55857f52f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:38.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.878+0000 7fc00a59c640 1 --2- 172.21.15.116:0/602034645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55857f52f4a0 0x55857f52f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.878+0000 7fc00a59c640 1 -- 172.21.15.116:0/602034645 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55857f421440 con 0x55857f52fdc0 2024-09-12T03:10:38.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.878+0000 7fc003fff640 1 --2- 172.21.15.116:0/602034645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55857f53a680 0x55857f53ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:38.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.878+0000 7fc00ad9d640 1 --2- 172.21.15.116:0/3282857512 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbff80a2b80 0x7fbff80a5040 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc004005fd0 tx=0x7fc004005c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:38.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.878+0000 7fc00a59c640 1 --2- 172.21.15.116:0/602034645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55857f52fdc0 0x55857f53a140 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fbffc031550 tx=0x7fbffc032cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcbbb2e69ad1c127 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:38.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.878+0000 7fc000ff9640 1 -- 172.21.15.116:0/602034645 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc03f070 con 0x55857f52fdc0 2024-09-12T03:10:38.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.878+0000 7fc000ff9640 1 -- 172.21.15.116:0/602034645 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbffc0317b0 con 0x55857f52fdc0 2024-09-12T03:10:38.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.878+0000 7fc000ff9640 1 -- 172.21.15.116:0/602034645 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc035070 con 0x55857f52fdc0 2024-09-12T03:10:38.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.879+0000 7fc012e6e740 1 -- 172.21.15.116:0/602034645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55857f52fdc0 msgr2=0x55857f53a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:38.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.879+0000 7fc012e6e740 1 --2- 172.21.15.116:0/602034645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55857f52fdc0 0x55857f53a140 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fbffc031550 tx=0x7fbffc032cd0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.879+0000 7fc012e6e740 1 -- 172.21.15.116:0/602034645 shutdown_connections 2024-09-12T03:10:38.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.879+0000 7fc012e6e740 1 --2- 172.21.15.116:0/602034645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55857f53a680 0x55857f53ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.879+0000 7fc012e6e740 1 --2- 172.21.15.116:0/602034645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55857f52fdc0 0x55857f53a140 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.879+0000 7fc012e6e740 1 --2- 172.21.15.116:0/602034645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55857f52f4a0 0x55857f52f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.879+0000 7fc012e6e740 1 -- 172.21.15.116:0/602034645 >> 172.21.15.116:0/602034645 conn(0x55857f52ba30 msgr2=0x55857f52be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:38.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.879+0000 7fc012e6e740 1 -- 172.21.15.116:0/602034645 shutdown_connections 2024-09-12T03:10:38.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.879+0000 7fc012e6e740 1 -- 172.21.15.116:0/602034645 wait complete. 2024-09-12T03:10:38.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.879+0000 7fc012e6e740 1 -- start start 2024-09-12T03:10:38.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.879+0000 7fc012e6e740 1 Processor -- start 2024-09-12T03:10:38.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.880+0000 7fc012e6e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55857f52f4a0 0x55857f539f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:38.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.880+0000 7fc012e6e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55857f52fdc0 0x55857f537210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:38.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.880+0000 7fc012e6e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55857f53a680 0x55857f537750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:38.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.880+0000 7fc012e6e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55857f53d250 con 0x55857f53a680 2024-09-12T03:10:38.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.880+0000 7fc012e6e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55857f450050 con 0x55857f52fdc0 2024-09-12T03:10:38.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.880+0000 7fc012e6e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55857f53d3d0 con 0x55857f52f4a0 2024-09-12T03:10:38.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.880+0000 7fc012e6e740 10 client.0 _ll_register_callbacks cb 0x55857f490160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:38.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.880+0000 7fc00ad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55857f53a680 0x55857f537750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:38.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.880+0000 7fc00ad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55857f53a680 0x55857f537750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33168/0 (socket says 172.21.15.116:33168) 2024-09-12T03:10:38.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.880+0000 7fc00ad9d640 1 -- 172.21.15.116:0/2426164415 learned_addr learned my addr 172.21.15.116:0/2426164415 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:38.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.880+0000 7fc00a59c640 1 --2- 172.21.15.116:0/2426164415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55857f52f4a0 0x55857f539f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:38.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.880+0000 7fc00ad9d640 1 -- 172.21.15.116:0/2426164415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55857f52f4a0 msgr2=0x55857f539f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:38.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.880+0000 7fc00ad9d640 1 --2- 172.21.15.116:0/2426164415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55857f52f4a0 0x55857f539f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.880+0000 7fc003fff640 1 --2- 172.21.15.116:0/2426164415 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55857f52fdc0 0x55857f537210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:38.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.880+0000 7fc00ad9d640 1 -- 172.21.15.116:0/2426164415 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55857f52fdc0 msgr2=0x55857f537210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:38.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.880+0000 7fc00ad9d640 1 --2- 172.21.15.116:0/2426164415 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55857f52fdc0 0x55857f537210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.880+0000 7fc00ad9d640 1 -- 172.21.15.116:0/2426164415 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55857f2063c0 con 0x55857f53a680 2024-09-12T03:10:38.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.880+0000 7fc003fff640 1 --2- 172.21.15.116:0/2426164415 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55857f52fdc0 0x55857f537210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:38.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.880+0000 7fc00ad9d640 1 --2- 172.21.15.116:0/2426164415 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55857f53a680 0x55857f537750 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fc00400fa70 tx=0x7fc004004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f6c2bed2fc72db server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:38.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.880+0000 7fbff1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:38.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.880+0000 7fbff1ffb640 1 -- 172.21.15.116:0/2426164415 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004011dd0 con 0x55857f53a680 2024-09-12T03:10:38.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.880+0000 7fbff1ffb640 1 -- 172.21.15.116:0/2426164415 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fc00401b420 con 0x55857f53a680 2024-09-12T03:10:38.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.880+0000 7fc012e6e740 1 -- 172.21.15.116:0/2426164415 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbffc0317b0 con 0x55857f53a680 2024-09-12T03:10:38.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.880+0000 7fbff1ffb640 1 -- 172.21.15.116:0/2426164415 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc00401b620 con 0x55857f53a680 2024-09-12T03:10:38.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.880+0000 7fc012e6e740 10 client.17202 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:38.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.880+0000 7fc012e6e740 1 -- 172.21.15.116:0/2426164415 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55857f548be0 con 0x55857f53a680 2024-09-12T03:10:38.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.881+0000 7fbff1ffb640 1 -- 172.21.15.116:0/2426164415 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbffc0317b0 con 0x55857f53a680 2024-09-12T03:10:38.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.881+0000 7fbff1ffb640 1 -- 172.21.15.116:0/2426164415 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc004031860 con 0x55857f53a680 2024-09-12T03:10:38.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.881+0000 7fc012e6e740 10 client.17202 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:38.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.881+0000 7fc012e6e740 10 client.17202 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:10:38.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.881+0000 7fc012e6e740 20 client.17202 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:38.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.881+0000 7fc012e6e740 1 --2- 172.21.15.116:0/2426164415 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55857f54a0a0 0x55857f54c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:38.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.881+0000 7fc012e6e740 4 client.17202 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:10:38.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.881+0000 7fc012e6e740 1 -- 172.21.15.116:0/2426164415 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55857f4a2850 con 0x55857f54a0a0 2024-09-12T03:10:38.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.881+0000 7fc00a59c640 1 --2- 172.21.15.116:0/2426164415 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55857f54a0a0 0x55857f54c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:38.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.882+0000 7fbff1ffb640 10 client.17202 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:38.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.882+0000 7fc00a59c640 1 --2- 172.21.15.116:0/2426164415 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55857f54a0a0 0x55857f54c560 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed588046653ced79 server_cookie=ba4108a21e3a72b7 in_seq=0 out_seq=0 2024-09-12T03:10:38.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.882+0000 7fbff1ffb640 1 -- 172.21.15.116:0/2426164415 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55857f4a2850 con 0x55857f54a0a0 2024-09-12T03:10:38.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.882+0000 7fbff1ffb640 10 client.17202 handle_command_reply: tid=0 2024-09-12T03:10:38.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.882+0000 7fc012e6e740 1 client.17202 shutdown 2024-09-12T03:10:38.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.883+0000 7fc012e6e740 1 -- 172.21.15.116:0/2426164415 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55857f53a680 msgr2=0x55857f537750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:38.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.883+0000 7fc012e6e740 1 --2- 172.21.15.116:0/2426164415 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55857f53a680 0x55857f537750 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fc00400fa70 tx=0x7fc004004a40 comp rx=0 tx=0).stop 2024-09-12T03:10:38.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.883+0000 7fc012e6e740 1 -- 172.21.15.116:0/2426164415 shutdown_connections 2024-09-12T03:10:38.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.883+0000 7fc012e6e740 1 --2- 172.21.15.116:0/2426164415 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55857f54a0a0 0x55857f54c560 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.883+0000 7fbff1ffb640 0 client.17202 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:38.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.883+0000 7fc012e6e740 1 --2- 172.21.15.116:0/2426164415 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55857f53a680 0x55857f537750 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.883+0000 7fc012e6e740 1 --2- 172.21.15.116:0/2426164415 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55857f52fdc0 0x55857f537210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.883+0000 7fc012e6e740 1 --2- 172.21.15.116:0/2426164415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55857f52f4a0 0x55857f539f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.883+0000 7fc012e6e740 1 -- 172.21.15.116:0/2426164415 >> 172.21.15.116:0/2426164415 conn(0x55857f52ba30 msgr2=0x55857f531760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:38.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.883+0000 7fc012e6e740 1 -- 172.21.15.116:0/2426164415 shutdown_connections 2024-09-12T03:10:38.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.883+0000 7fc012e6e740 1 -- 172.21.15.116:0/2426164415 wait complete. 2024-09-12T03:10:38.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.883+0000 7fc012e6e740 20 client.17202 trim_cache size 0 max 16384 2024-09-12T03:10:38.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.899+0000 7fc012e6e740 1 Processor -- start 2024-09-12T03:10:38.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.899+0000 7fc012e6e740 1 -- start start 2024-09-12T03:10:38.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.900+0000 7fc012e6e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55857f52ba30 0x55857f539180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:38.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.900+0000 7fc012e6e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55857f52f4a0 0x55857f551e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:38.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.900+0000 7fc012e6e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55857f52fdc0 0x55857f5523c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:38.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.900+0000 7fc012e6e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1afdb0 con 0x55857f52f4a0 2024-09-12T03:10:38.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.900+0000 7fc012e6e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1ad680 con 0x55857f52ba30 2024-09-12T03:10:38.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.900+0000 7fc012e6e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1ab4d0 con 0x55857f52fdc0 2024-09-12T03:10:38.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.900+0000 7fc003fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55857f52f4a0 0x55857f551e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:38.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.900+0000 7fc003fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55857f52f4a0 0x55857f551e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33172/0 (socket says 172.21.15.116:33172) 2024-09-12T03:10:38.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.900+0000 7fc003fff640 1 -- 172.21.15.116:0/920240410 learned_addr learned my addr 172.21.15.116:0/920240410 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:38.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.900+0000 7fc00ad9d640 1 --2- 172.21.15.116:0/920240410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55857f52fdc0 0x55857f5523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:38.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.900+0000 7fc00a59c640 1 --2- 172.21.15.116:0/920240410 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55857f52ba30 0x55857f539180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:38.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.900+0000 7fc003fff640 1 -- 172.21.15.116:0/920240410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55857f52fdc0 msgr2=0x55857f5523c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:38.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.900+0000 7fc003fff640 1 --2- 172.21.15.116:0/920240410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55857f52fdc0 0x55857f5523c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.900+0000 7fc003fff640 1 -- 172.21.15.116:0/920240410 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55857f52ba30 msgr2=0x55857f539180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:38.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.900+0000 7fc003fff640 1 --2- 172.21.15.116:0/920240410 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55857f52ba30 0x55857f539180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.900+0000 7fc003fff640 1 -- 172.21.15.116:0/920240410 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55857f52fbf0 con 0x55857f52f4a0 2024-09-12T03:10:38.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.900+0000 7fc00a59c640 1 --2- 172.21.15.116:0/920240410 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55857f52ba30 0x55857f539180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:38.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.900+0000 7fc003fff640 1 --2- 172.21.15.116:0/920240410 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55857f52f4a0 0x55857f551e80 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fbff40a3d00 tx=0x7fbff4009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4da37c951cc3c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:38.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.900+0000 7fbff3fff640 1 -- 172.21.15.116:0/920240410 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff4020040 con 0x55857f52f4a0 2024-09-12T03:10:38.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.900+0000 7fbff3fff640 1 -- 172.21.15.116:0/920240410 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbff40d0070 con 0x55857f52f4a0 2024-09-12T03:10:38.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.900+0000 7fbff3fff640 1 -- 172.21.15.116:0/920240410 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff4084b50 con 0x55857f52f4a0 2024-09-12T03:10:38.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.901+0000 7fc012e6e740 1 -- 172.21.15.116:0/920240410 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55857f52f4a0 msgr2=0x55857f551e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:38.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.901+0000 7fc012e6e740 1 --2- 172.21.15.116:0/920240410 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55857f52f4a0 0x55857f551e80 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fbff40a3d00 tx=0x7fbff4009de0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.901+0000 7fc012e6e740 1 -- 172.21.15.116:0/920240410 shutdown_connections 2024-09-12T03:10:38.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.901+0000 7fc012e6e740 1 --2- 172.21.15.116:0/920240410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55857f52fdc0 0x55857f5523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.901+0000 7fc012e6e740 1 --2- 172.21.15.116:0/920240410 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55857f52f4a0 0x55857f551e80 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.901+0000 7fc012e6e740 1 --2- 172.21.15.116:0/920240410 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55857f52ba30 0x55857f539180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.901+0000 7fc012e6e740 1 -- 172.21.15.116:0/920240410 >> 172.21.15.116:0/920240410 conn(0x55857f5378b0 msgr2=0x55857f538a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:38.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.901+0000 7fc012e6e740 1 -- 172.21.15.116:0/920240410 shutdown_connections 2024-09-12T03:10:38.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.901+0000 7fc012e6e740 1 -- 172.21.15.116:0/920240410 wait complete. 2024-09-12T03:10:38.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.901+0000 7fc012e6e740 1 -- start start 2024-09-12T03:10:38.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.901+0000 7fc012e6e740 1 Processor -- start 2024-09-12T03:10:38.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.902+0000 7fc012e6e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55857f52ba30 0x55857f5402c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:38.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.902+0000 7fc012e6e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55857f52f4a0 0x55857f540800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:38.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.902+0000 7fc012e6e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55857f52fdc0 0x55857f540d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:38.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.902+0000 7fc012e6e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1abb40 con 0x55857f52f4a0 2024-09-12T03:10:38.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.902+0000 7fc012e6e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1acfb0 con 0x55857f52ba30 2024-09-12T03:10:38.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.902+0000 7fc012e6e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1ac210 con 0x55857f52fdc0 2024-09-12T03:10:38.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.902+0000 7fc012e6e740 10 client.0 _ll_register_callbacks cb 0x7fc00400ce50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:38.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.902+0000 7fc003fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55857f52f4a0 0x55857f540800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:38.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.902+0000 7fc003fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55857f52f4a0 0x55857f540800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33186/0 (socket says 172.21.15.116:33186) 2024-09-12T03:10:38.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.902+0000 7fc003fff640 1 -- 172.21.15.116:0/2492927109 learned_addr learned my addr 172.21.15.116:0/2492927109 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:38.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.902+0000 7fc00ad9d640 1 --2- 172.21.15.116:0/2492927109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55857f52fdc0 0x55857f540d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:38.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.902+0000 7fc00a59c640 1 --2- 172.21.15.116:0/2492927109 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55857f52ba30 0x55857f5402c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:38.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.902+0000 7fc003fff640 1 -- 172.21.15.116:0/2492927109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55857f52fdc0 msgr2=0x55857f540d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:38.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.902+0000 7fc003fff640 1 --2- 172.21.15.116:0/2492927109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55857f52fdc0 0x55857f540d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.902+0000 7fc003fff640 1 -- 172.21.15.116:0/2492927109 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55857f52ba30 msgr2=0x55857f5402c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:38.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.902+0000 7fc003fff640 1 --2- 172.21.15.116:0/2492927109 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55857f52ba30 0x55857f5402c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.902+0000 7fc003fff640 1 -- 172.21.15.116:0/2492927109 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55857f5b3e60 con 0x55857f52f4a0 2024-09-12T03:10:38.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.902+0000 7fc003fff640 1 --2- 172.21.15.116:0/2492927109 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55857f52f4a0 0x55857f540800 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fbff4000c00 tx=0x7fbff409f970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b11f34282e6f2f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:38.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.902+0000 7fbff1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:38.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.902+0000 7fbff1ffb640 1 -- 172.21.15.116:0/2492927109 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff40b4020 con 0x55857f52f4a0 2024-09-12T03:10:38.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.902+0000 7fbff1ffb640 1 -- 172.21.15.116:0/2492927109 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fbff4084d50 con 0x55857f52f4a0 2024-09-12T03:10:38.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.902+0000 7fc012e6e740 1 -- 172.21.15.116:0/2492927109 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbff40d0070 con 0x55857f52f4a0 2024-09-12T03:10:38.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.903+0000 7fbff1ffb640 1 -- 172.21.15.116:0/2492927109 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbff409b5c0 con 0x55857f52f4a0 2024-09-12T03:10:38.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.903+0000 7fc012e6e740 10 client.17208 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:38.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.903+0000 7fc012e6e740 1 -- 172.21.15.116:0/2492927109 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55857f5b4fe0 con 0x55857f52f4a0 2024-09-12T03:10:38.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.903+0000 7fbff1ffb640 1 -- 172.21.15.116:0/2492927109 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbff40d0070 con 0x55857f52f4a0 2024-09-12T03:10:38.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.903+0000 7fbff1ffb640 1 -- 172.21.15.116:0/2492927109 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbff40973a0 con 0x55857f52f4a0 2024-09-12T03:10:38.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.903+0000 7fc012e6e740 10 client.17208 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:38.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.903+0000 7fc012e6e740 10 client.17208 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:10:38.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.903+0000 7fc012e6e740 20 client.17208 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:38.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.903+0000 7fc012e6e740 1 --2- 172.21.15.116:0/2492927109 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55857f53a680 0x55857f5b76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:38.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.903+0000 7fc012e6e740 4 client.17208 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:10:38.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.903+0000 7fc012e6e740 1 -- 172.21.15.116:0/2492927109 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55857f49cd60 con 0x55857f53a680 2024-09-12T03:10:38.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.903+0000 7fc00a59c640 1 --2- 172.21.15.116:0/2492927109 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55857f53a680 0x55857f5b76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:38.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.904+0000 7fc00a59c640 1 --2- 172.21.15.116:0/2492927109 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55857f53a680 0x55857f5b76f0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=102505441c65a3de server_cookie=7eb1be1a13aa4d5d in_seq=0 out_seq=0 2024-09-12T03:10:38.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.904+0000 7fbff1ffb640 10 client.17208 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:38.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.904+0000 7fbff1ffb640 1 -- 172.21.15.116:0/2492927109 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55857f49cd60 con 0x55857f53a680 2024-09-12T03:10:38.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.904+0000 7fbff1ffb640 10 client.17208 handle_command_reply: tid=0 2024-09-12T03:10:38.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.904+0000 7fc012e6e740 1 client.17208 shutdown 2024-09-12T03:10:38.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.904+0000 7fc012e6e740 1 -- 172.21.15.116:0/2492927109 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55857f52f4a0 msgr2=0x55857f540800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:38.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.904+0000 7fc012e6e740 1 --2- 172.21.15.116:0/2492927109 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55857f52f4a0 0x55857f540800 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fbff4000c00 tx=0x7fbff409f970 comp rx=0 tx=0).stop 2024-09-12T03:10:38.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.904+0000 7fc012e6e740 1 -- 172.21.15.116:0/2492927109 shutdown_connections 2024-09-12T03:10:38.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.904+0000 7fc012e6e740 1 --2- 172.21.15.116:0/2492927109 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55857f53a680 0x55857f5b76f0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.904+0000 7fc012e6e740 1 --2- 172.21.15.116:0/2492927109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55857f52fdc0 0x55857f540d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.904+0000 7fc012e6e740 1 --2- 172.21.15.116:0/2492927109 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55857f52f4a0 0x55857f540800 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.904+0000 7fc012e6e740 1 --2- 172.21.15.116:0/2492927109 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55857f52ba30 0x55857f5402c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.904+0000 7fc012e6e740 1 -- 172.21.15.116:0/2492927109 >> 172.21.15.116:0/2492927109 conn(0x55857f5378b0 msgr2=0x55857f538a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:38.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.904+0000 7fbff1ffb640 0 client.17208 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:38.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.904+0000 7fc012e6e740 1 -- 172.21.15.116:0/2492927109 shutdown_connections 2024-09-12T03:10:38.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.904+0000 7fc012e6e740 1 -- 172.21.15.116:0/2492927109 wait complete. 2024-09-12T03:10:38.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.904+0000 7fc012e6e740 20 client.17208 trim_cache size 0 max 16384 2024-09-12T03:10:38.924 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:10:38.925 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:10:38.925 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:10:38.925 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:10:38.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.905+0000 7fc010a95640 1 -- 172.21.15.116:0/3282857512 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbff80a2b80 msgr2=0x7fbff80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:38.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.905+0000 7fc010a95640 1 --2- 172.21.15.116:0/3282857512 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbff80a2b80 0x7fbff80a5040 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc004005fd0 tx=0x7fc004005c80 comp rx=0 tx=0).stop 2024-09-12T03:10:38.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.905+0000 7fc010a95640 1 -- 172.21.15.116:0/3282857512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc00c102790 msgr2=0x7fc00c198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:38.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.905+0000 7fc010a95640 1 --2- 172.21.15.116:0/3282857512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc00c102790 0x7fc00c198970 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fbff400d950 tx=0x7fbff400de10 comp rx=0 tx=0).stop 2024-09-12T03:10:38.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.905+0000 7fc010a95640 1 -- 172.21.15.116:0/3282857512 shutdown_connections 2024-09-12T03:10:38.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.905+0000 7fc010a95640 1 --2- 172.21.15.116:0/3282857512 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbff80a2b80 0x7fbff80a5040 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.905+0000 7fc010a95640 1 --2- 172.21.15.116:0/3282857512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc00c106f10 0x7fc00c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.905+0000 7fc010a95640 1 --2- 172.21.15.116:0/3282857512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc00c102790 0x7fc00c198970 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.905+0000 7fc010a95640 1 --2- 172.21.15.116:0/3282857512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc00c101de0 0x7fc00c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.905+0000 7fc010a95640 1 -- 172.21.15.116:0/3282857512 >> 172.21.15.116:0/3282857512 conn(0x7fc00c0fd8c0 msgr2=0x7fc00c0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:38.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.905+0000 7fc010a95640 1 -- 172.21.15.116:0/3282857512 shutdown_connections 2024-09-12T03:10:38.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:38.905+0000 7fc010a95640 1 -- 172.21.15.116:0/3282857512 wait complete. 2024-09-12T03:10:39.656 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:39 smithi116 ceph-mon[31410]: pgmap v568: 97 pgs: 97 active+clean; 3.8 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:10:39.657 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:10:39.658 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:10:39.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:39 smithi183 ceph-mon[35535]: pgmap v568: 97 pgs: 97 active+clean; 3.8 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:10:39.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:39 smithi177 ceph-mon[35305]: pgmap v568: 97 pgs: 97 active+clean; 3.8 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:10:39.903 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:10:40.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.412+0000 7feacf1e1640 1 -- 172.21.15.116:0/286407142 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feac8101420 msgr2=0x7feac810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:40.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.412+0000 7feacf1e1640 1 --2- 172.21.15.116:0/286407142 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feac8101420 0x7feac810d950 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7feac0009cd0 tx=0x7feac002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:10:40.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.412+0000 7feacf1e1640 1 -- 172.21.15.116:0/286407142 shutdown_connections 2024-09-12T03:10:40.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.412+0000 7feacf1e1640 1 --2- 172.21.15.116:0/286407142 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feac8101420 0x7feac810d950 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:40.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.412+0000 7feacf1e1640 1 --2- 172.21.15.116:0/286407142 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feac8100a80 0x7feac8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:40.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.412+0000 7feacf1e1640 1 --2- 172.21.15.116:0/286407142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feac8106a80 0x7feac8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:40.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.412+0000 7feacf1e1640 1 -- 172.21.15.116:0/286407142 >> 172.21.15.116:0/286407142 conn(0x7feac80fc7a0 msgr2=0x7feac80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:40.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.412+0000 7feacf1e1640 1 -- 172.21.15.116:0/286407142 shutdown_connections 2024-09-12T03:10:40.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.412+0000 7feacf1e1640 1 -- 172.21.15.116:0/286407142 wait complete. 2024-09-12T03:10:40.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.413+0000 7feacf1e1640 1 Processor -- start 2024-09-12T03:10:40.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.413+0000 7feacf1e1640 1 -- start start 2024-09-12T03:10:40.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.413+0000 7feacf1e1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feac8100a80 0x7feac81992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:40.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.413+0000 7feacf1e1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feac8101420 0x7feac81997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:40.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.414+0000 7feacf1e1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feac8106a80 0x7feac81a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:40.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.414+0000 7feacf1e1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7feac8110e80 con 0x7feac8106a80 2024-09-12T03:10:40.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.414+0000 7feacf1e1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7feac8110d00 con 0x7feac8101420 2024-09-12T03:10:40.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.414+0000 7feacf1e1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feac8111000 con 0x7feac8100a80 2024-09-12T03:10:40.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.414+0000 7feacd77e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feac8106a80 0x7feac81a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:40.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.414+0000 7feacd77e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feac8106a80 0x7feac81a4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33196/0 (socket says 172.21.15.116:33196) 2024-09-12T03:10:40.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.414+0000 7feacd77e640 1 -- 172.21.15.116:0/116529578 learned_addr learned my addr 172.21.15.116:0/116529578 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:40.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.414+0000 7feac7fff640 1 --2- 172.21.15.116:0/116529578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feac8101420 0x7feac81997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:40.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.414+0000 7feaccf7d640 1 --2- 172.21.15.116:0/116529578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feac8100a80 0x7feac81992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:40.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.414+0000 7feacd77e640 1 -- 172.21.15.116:0/116529578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feac8100a80 msgr2=0x7feac81992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:40.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.414+0000 7feacd77e640 1 --2- 172.21.15.116:0/116529578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feac8100a80 0x7feac81992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:40.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.414+0000 7feacd77e640 1 -- 172.21.15.116:0/116529578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feac8101420 msgr2=0x7feac81997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:40.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.414+0000 7feacd77e640 1 --2- 172.21.15.116:0/116529578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feac8101420 0x7feac81997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:40.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.414+0000 7feacd77e640 1 -- 172.21.15.116:0/116529578 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac81a4870 con 0x7feac8106a80 2024-09-12T03:10:40.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.414+0000 7feacd77e640 1 --2- 172.21.15.116:0/116529578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feac8106a80 0x7feac81a4330 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7feac0005cc0 tx=0x7feac0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:40.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.414+0000 7feac5ffb640 1 -- 172.21.15.116:0/116529578 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac0004040 con 0x7feac8106a80 2024-09-12T03:10:40.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.414+0000 7feacf1e1640 1 -- 172.21.15.116:0/116529578 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac81a4a40 con 0x7feac8106a80 2024-09-12T03:10:40.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.414+0000 7feac5ffb640 1 -- 172.21.15.116:0/116529578 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feac00046c0 con 0x7feac8106a80 2024-09-12T03:10:40.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.414+0000 7feac5ffb640 1 -- 172.21.15.116:0/116529578 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac0046480 con 0x7feac8106a80 2024-09-12T03:10:40.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.415+0000 7feacf1e1640 1 -- 172.21.15.116:0/116529578 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac81a4ef0 con 0x7feac8106a80 2024-09-12T03:10:40.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.416+0000 7feacf1e1640 1 -- 172.21.15.116:0/116529578 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feac8100ee0 con 0x7feac8106a80 2024-09-12T03:10:40.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.418+0000 7feac5ffb640 1 -- 172.21.15.116:0/116529578 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7feac00305a0 con 0x7feac8106a80 2024-09-12T03:10:40.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.418+0000 7feac5ffb640 1 --2- 172.21.15.116:0/116529578 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7feab80a2b80 0x7feab80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:40.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.418+0000 7feaccf7d640 1 --2- 172.21.15.116:0/116529578 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7feab80a2b80 0x7feab80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:40.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.418+0000 7feac5ffb640 1 -- 172.21.15.116:0/116529578 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7feac002e030 con 0x7feac8106a80 2024-09-12T03:10:40.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.419+0000 7feaccf7d640 1 --2- 172.21.15.116:0/116529578 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7feab80a2b80 0x7feab80a5040 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7feabc005fd0 tx=0x7feabc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:40.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.421+0000 7feac5ffb640 1 -- 172.21.15.116:0/116529578 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feac004b060 con 0x7feac8106a80 2024-09-12T03:10:40.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.624+0000 7feacf1e1640 1 -- 172.21.15.116:0/116529578 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feac8109010 con 0x7feac8106a80 2024-09-12T03:10:40.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.625+0000 7feac5ffb640 1 -- 172.21.15.116:0/116529578 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7feac00bd940 con 0x7feac8106a80 2024-09-12T03:10:40.625 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:10:40.626 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:10:40.626 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:10:40.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.627+0000 7feacf1e1640 1 -- 172.21.15.116:0/116529578 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7feab80a2b80 msgr2=0x7feab80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:40.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.627+0000 7feacf1e1640 1 --2- 172.21.15.116:0/116529578 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7feab80a2b80 0x7feab80a5040 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7feabc005fd0 tx=0x7feabc005850 comp rx=0 tx=0).stop 2024-09-12T03:10:40.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.627+0000 7feacf1e1640 1 -- 172.21.15.116:0/116529578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feac8106a80 msgr2=0x7feac81a4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:40.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.627+0000 7feacf1e1640 1 --2- 172.21.15.116:0/116529578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feac8106a80 0x7feac81a4330 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7feac0005cc0 tx=0x7feac0003970 comp rx=0 tx=0).stop 2024-09-12T03:10:40.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.627+0000 7feacf1e1640 1 -- 172.21.15.116:0/116529578 shutdown_connections 2024-09-12T03:10:40.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.627+0000 7feacf1e1640 1 --2- 172.21.15.116:0/116529578 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7feab80a2b80 0x7feab80a5040 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:40.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.627+0000 7feacf1e1640 1 --2- 172.21.15.116:0/116529578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feac8106a80 0x7feac81a4330 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:40.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.627+0000 7feacf1e1640 1 --2- 172.21.15.116:0/116529578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feac8101420 0x7feac81997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:40.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.627+0000 7feacf1e1640 1 --2- 172.21.15.116:0/116529578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feac8100a80 0x7feac81992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:40.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.627+0000 7feacf1e1640 1 -- 172.21.15.116:0/116529578 >> 172.21.15.116:0/116529578 conn(0x7feac80fc7a0 msgr2=0x7feac80fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:40.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.628+0000 7feacf1e1640 1 -- 172.21.15.116:0/116529578 shutdown_connections 2024-09-12T03:10:40.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:40.628+0000 7feacf1e1640 1 -- 172.21.15.116:0/116529578 wait complete. 2024-09-12T03:10:40.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:40 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/116529578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:10:40.987 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:40 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/116529578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:10:40.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:40 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/116529578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:10:41.440 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:10:41.686 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:10:41.946 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:41 smithi116 ceph-mon[31410]: pgmap v569: 97 pgs: 97 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-12T03:10:41.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:41 smithi183 ceph-mon[35535]: pgmap v569: 97 pgs: 97 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-12T03:10:41.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:41 smithi177 ceph-mon[35305]: pgmap v569: 97 pgs: 97 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-12T03:10:42.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.139+0000 7fd09e222640 1 -- 172.21.15.116:0/4068500631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd098100a80 msgr2=0x7fd098100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:42.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.139+0000 7fd09e222640 1 --2- 172.21.15.116:0/4068500631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd098100a80 0x7fd098100ee0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fd08c009930 tx=0x7fd08c02a420 comp rx=0 tx=0).stop 2024-09-12T03:10:42.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.139+0000 7fd09e222640 1 -- 172.21.15.116:0/4068500631 shutdown_connections 2024-09-12T03:10:42.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.139+0000 7fd09e222640 1 --2- 172.21.15.116:0/4068500631 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd098101420 0x7fd09810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.139+0000 7fd09e222640 1 --2- 172.21.15.116:0/4068500631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd098100a80 0x7fd098100ee0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.139+0000 7fd09e222640 1 --2- 172.21.15.116:0/4068500631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd098106a80 0x7fd098106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.139+0000 7fd09e222640 1 -- 172.21.15.116:0/4068500631 >> 172.21.15.116:0/4068500631 conn(0x7fd0980fc7a0 msgr2=0x7fd0980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:42.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.139+0000 7fd09e222640 1 -- 172.21.15.116:0/4068500631 shutdown_connections 2024-09-12T03:10:42.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.139+0000 7fd09e222640 1 -- 172.21.15.116:0/4068500631 wait complete. 2024-09-12T03:10:42.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.140+0000 7fd09e222640 1 Processor -- start 2024-09-12T03:10:42.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.140+0000 7fd09e222640 1 -- start start 2024-09-12T03:10:42.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.140+0000 7fd09e222640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd098100a80 0x7fd0981985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.140+0000 7fd09e222640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd098101420 0x7fd098198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.140+0000 7fd09e222640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd098106a80 0x7fd09819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.140+0000 7fd09e222640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd098112020 con 0x7fd098100a80 2024-09-12T03:10:42.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.140+0000 7fd09e222640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd098111ea0 con 0x7fd098101420 2024-09-12T03:10:42.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.140+0000 7fd09e222640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd0981121a0 con 0x7fd098106a80 2024-09-12T03:10:42.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.140+0000 7fd0977fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd098100a80 0x7fd0981985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.140+0000 7fd0977fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd098100a80 0x7fd0981985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55684/0 (socket says 172.21.15.116:55684) 2024-09-12T03:10:42.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.140+0000 7fd0977fe640 1 -- 172.21.15.116:0/810530575 learned_addr learned my addr 172.21.15.116:0/810530575 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:42.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.140+0000 7fd096ffd640 1 --2- 172.21.15.116:0/810530575 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd098101420 0x7fd098198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.140+0000 7fd0977fe640 1 -- 172.21.15.116:0/810530575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd098106a80 msgr2=0x7fd09819ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:42.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.140+0000 7fd097fff640 1 --2- 172.21.15.116:0/810530575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd098106a80 0x7fd09819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.141+0000 7fd0977fe640 1 --2- 172.21.15.116:0/810530575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd098106a80 0x7fd09819ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.141+0000 7fd0977fe640 1 -- 172.21.15.116:0/810530575 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd098101420 msgr2=0x7fd098198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:42.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.141+0000 7fd0977fe640 1 --2- 172.21.15.116:0/810530575 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd098101420 0x7fd098198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.141+0000 7fd0977fe640 1 -- 172.21.15.116:0/810530575 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd09819d570 con 0x7fd098100a80 2024-09-12T03:10:42.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.141+0000 7fd096ffd640 1 --2- 172.21.15.116:0/810530575 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd098101420 0x7fd098198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:10:42.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.141+0000 7fd0977fe640 1 --2- 172.21.15.116:0/810530575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd098100a80 0x7fd0981985c0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fd08800ba50 tx=0x7fd08800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:42.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.141+0000 7fd094ff9640 1 -- 172.21.15.116:0/810530575 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd088019070 con 0x7fd098100a80 2024-09-12T03:10:42.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.141+0000 7fd094ff9640 1 -- 172.21.15.116:0/810530575 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd08800cbe0 con 0x7fd098100a80 2024-09-12T03:10:42.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.141+0000 7fd09e222640 1 -- 172.21.15.116:0/810530575 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd09819d860 con 0x7fd098100a80 2024-09-12T03:10:42.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.141+0000 7fd094ff9640 1 -- 172.21.15.116:0/810530575 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0880076e0 con 0x7fd098100a80 2024-09-12T03:10:42.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.141+0000 7fd09e222640 1 -- 172.21.15.116:0/810530575 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0981a5140 con 0x7fd098100a80 2024-09-12T03:10:42.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.143+0000 7fd094ff9640 1 -- 172.21.15.116:0/810530575 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd08801c070 con 0x7fd098100a80 2024-09-12T03:10:42.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.144+0000 7fd0a05fb740 1 Processor -- start 2024-09-12T03:10:42.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.144+0000 7fd0a05fb740 1 -- start start 2024-09-12T03:10:42.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.144+0000 7fd0a05fb740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578f2185b40 0x5578f2184090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.144+0000 7fd0a05fb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5578f21845d0 0x5578f2184a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.144+0000 7fd0a05fb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578f218f370 0x5578f2191760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.144+0000 7fd0a05fb740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5578f20ea2f0 con 0x5578f2185b40 2024-09-12T03:10:42.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.144+0000 7fd0a05fb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5578f2091cf0 con 0x5578f21845d0 2024-09-12T03:10:42.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.144+0000 7fd0a05fb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5578f20e62b0 con 0x5578f218f370 2024-09-12T03:10:42.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.144+0000 7fd097fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578f2185b40 0x5578f2184090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.144+0000 7fd097fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578f2185b40 0x5578f2184090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55698/0 (socket says 172.21.15.116:55698) 2024-09-12T03:10:42.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.144+0000 7fd097fff640 1 -- 172.21.15.116:0/80063013 learned_addr learned my addr 172.21.15.116:0/80063013 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:42.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.144+0000 7fd094ff9640 1 --2- 172.21.15.116:0/810530575 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd0800a2c00 0x7fd0800a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.144+0000 7fd097fff640 1 --2- 172.21.15.116:0/810530575 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd0800a2c00 0x7fd0800a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.144+0000 7fd096ffd640 1 --2- 172.21.15.116:0/80063013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578f218f370 0x5578f2191760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.144+0000 7fd094ff9640 1 -- 172.21.15.116:0/810530575 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd0880ce630 con 0x7fd098100a80 2024-09-12T03:10:42.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.144+0000 7fd0977fe640 1 --2- 172.21.15.116:0/80063013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5578f21845d0 0x5578f2184a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.145+0000 7fd097fff640 1 -- 172.21.15.116:0/80063013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578f218f370 msgr2=0x5578f2191760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:42.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.145+0000 7fd097fff640 1 --2- 172.21.15.116:0/80063013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578f218f370 0x5578f2191760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.145+0000 7fd097fff640 1 -- 172.21.15.116:0/80063013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5578f21845d0 msgr2=0x5578f2184a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:42.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.145+0000 7fd097fff640 1 --2- 172.21.15.116:0/80063013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5578f21845d0 0x5578f2184a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.145+0000 7fd097fff640 1 -- 172.21.15.116:0/80063013 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578f20763a0 con 0x5578f2185b40 2024-09-12T03:10:42.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.145+0000 7fd097fff640 1 --2- 172.21.15.116:0/80063013 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578f2185b40 0x5578f2184090 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fd090013300 tx=0x7fd0900137c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b206c820effb4b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:42.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.145+0000 7fd07dffb640 1 -- 172.21.15.116:0/80063013 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09001a070 con 0x5578f2185b40 2024-09-12T03:10:42.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.145+0000 7fd07dffb640 1 -- 172.21.15.116:0/80063013 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd090016d40 con 0x5578f2185b40 2024-09-12T03:10:42.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.145+0000 7fd07dffb640 1 -- 172.21.15.116:0/80063013 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09000f040 con 0x5578f2185b40 2024-09-12T03:10:42.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.146+0000 7fd0a05fb740 1 -- 172.21.15.116:0/80063013 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578f2185b40 msgr2=0x5578f2184090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:42.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.146+0000 7fd097fff640 1 --2- 172.21.15.116:0/810530575 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd0800a2c00 0x7fd0800a50c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd09001e000 tx=0x7fd09001c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:42.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.146+0000 7fd0a05fb740 1 --2- 172.21.15.116:0/80063013 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578f2185b40 0x5578f2184090 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fd090013300 tx=0x7fd0900137c0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.146+0000 7fd0a05fb740 1 -- 172.21.15.116:0/80063013 shutdown_connections 2024-09-12T03:10:42.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.146+0000 7fd0a05fb740 1 --2- 172.21.15.116:0/80063013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578f218f370 0x5578f2191760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.146+0000 7fd0a05fb740 1 --2- 172.21.15.116:0/80063013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5578f21845d0 0x5578f2184a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.146+0000 7fd0a05fb740 1 --2- 172.21.15.116:0/80063013 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578f2185b40 0x5578f2184090 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.146+0000 7fd0a05fb740 1 -- 172.21.15.116:0/80063013 >> 172.21.15.116:0/80063013 conn(0x5578f214d310 msgr2=0x5578f2180c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:42.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.146+0000 7fd0a05fb740 1 -- 172.21.15.116:0/80063013 shutdown_connections 2024-09-12T03:10:42.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.146+0000 7fd0a05fb740 1 -- 172.21.15.116:0/80063013 wait complete. 2024-09-12T03:10:42.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.146+0000 7fd0a05fb740 1 -- start start 2024-09-12T03:10:42.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.146+0000 7fd0a05fb740 1 Processor -- start 2024-09-12T03:10:42.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.147+0000 7fd0a05fb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578f21845d0 0x5578f218ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.147+0000 7fd0a05fb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5578f2185b40 0x5578f218c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.147+0000 7fd0a05fb740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578f218f370 0x5578f218c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.147+0000 7fd0a05fb740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5578f2192070 con 0x5578f218f370 2024-09-12T03:10:42.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.147+0000 7fd0a05fb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5578f20a5060 con 0x5578f2185b40 2024-09-12T03:10:42.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.147+0000 7fd0a05fb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5578f21921f0 con 0x5578f21845d0 2024-09-12T03:10:42.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.147+0000 7fd0a05fb740 10 client.0 _ll_register_callbacks cb 0x5578f1df88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:42.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.147+0000 7fd097fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578f218f370 0x5578f218c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.147+0000 7fd097fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578f218f370 0x5578f218c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55704/0 (socket says 172.21.15.116:55704) 2024-09-12T03:10:42.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.147+0000 7fd097fff640 1 -- 172.21.15.116:0/4199927236 learned_addr learned my addr 172.21.15.116:0/4199927236 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:42.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.147+0000 7fd096ffd640 1 --2- 172.21.15.116:0/4199927236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5578f2185b40 0x5578f218c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.147+0000 7fd0977fe640 1 --2- 172.21.15.116:0/4199927236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578f21845d0 0x5578f218ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.147+0000 7fd097fff640 1 -- 172.21.15.116:0/4199927236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578f21845d0 msgr2=0x5578f218ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:42.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.147+0000 7fd097fff640 1 --2- 172.21.15.116:0/4199927236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578f21845d0 0x5578f218ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.147+0000 7fd097fff640 1 -- 172.21.15.116:0/4199927236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5578f2185b40 msgr2=0x5578f218c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:42.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.147+0000 7fd097fff640 1 --2- 172.21.15.116:0/4199927236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5578f2185b40 0x5578f218c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.147+0000 7fd097fff640 1 -- 172.21.15.116:0/4199927236 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578f1e5b3c0 con 0x5578f218f370 2024-09-12T03:10:42.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.147+0000 7fd0977fe640 1 --2- 172.21.15.116:0/4199927236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578f21845d0 0x5578f218ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:10:42.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.147+0000 7fd096ffd640 1 --2- 172.21.15.116:0/4199927236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5578f2185b40 0x5578f218c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:10:42.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.147+0000 7fd097fff640 1 --2- 172.21.15.116:0/4199927236 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578f218f370 0x5578f218c650 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd09001d740 tx=0x7fd090013d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d938f9b3724277b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:42.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.147+0000 7fd076ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:42.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.147+0000 7fd076ffd640 1 -- 172.21.15.116:0/4199927236 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09002d070 con 0x5578f218f370 2024-09-12T03:10:42.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.147+0000 7fd0a05fb740 1 -- 172.21.15.116:0/4199927236 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd090016d40 con 0x5578f218f370 2024-09-12T03:10:42.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.147+0000 7fd076ffd640 1 -- 172.21.15.116:0/4199927236 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd09001d880 con 0x5578f218f370 2024-09-12T03:10:42.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.147+0000 7fd076ffd640 1 -- 172.21.15.116:0/4199927236 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd090039050 con 0x5578f218f370 2024-09-12T03:10:42.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.148+0000 7fd0a05fb740 10 client.17226 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:42.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.148+0000 7fd0a05fb740 1 -- 172.21.15.116:0/4199927236 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5578f219db40 con 0x5578f218f370 2024-09-12T03:10:42.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.148+0000 7fd076ffd640 1 -- 172.21.15.116:0/4199927236 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd090016d40 con 0x5578f218f370 2024-09-12T03:10:42.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.148+0000 7fd076ffd640 1 -- 172.21.15.116:0/4199927236 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd090015040 con 0x5578f218f370 2024-09-12T03:10:42.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.148+0000 7fd0a05fb740 10 client.17226 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:42.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.148+0000 7fd0a05fb740 10 client.17226 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:10:42.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.148+0000 7fd0a05fb740 20 client.17226 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:42.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.148+0000 7fd0a05fb740 1 --2- 172.21.15.116:0/4199927236 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5578f219eef0 0x5578f21a12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.148+0000 7fd0a05fb740 4 client.17226 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:10:42.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.148+0000 7fd0a05fb740 1 -- 172.21.15.116:0/4199927236 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578f20f7860 con 0x5578f219eef0 2024-09-12T03:10:42.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.148+0000 7fd0977fe640 1 --2- 172.21.15.116:0/4199927236 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5578f219eef0 0x5578f21a12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.149+0000 7fd076ffd640 10 client.17226 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:10:42.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.149+0000 7fd0977fe640 1 --2- 172.21.15.116:0/4199927236 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5578f219eef0 0x5578f21a12e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2a5be8dec6bbda90 server_cookie=88131cdc1d4d42b7 in_seq=0 out_seq=0 2024-09-12T03:10:42.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.149+0000 7fd076ffd640 1 -- 172.21.15.116:0/4199927236 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5578f20f7860 con 0x5578f219eef0 2024-09-12T03:10:42.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.149+0000 7fd076ffd640 10 client.17226 handle_command_reply: tid=0 2024-09-12T03:10:42.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.149+0000 7fd0a05fb740 1 client.17226 shutdown 2024-09-12T03:10:42.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.149+0000 7fd0a05fb740 1 -- 172.21.15.116:0/4199927236 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578f218f370 msgr2=0x5578f218c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:42.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.149+0000 7fd0a05fb740 1 --2- 172.21.15.116:0/4199927236 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578f218f370 0x5578f218c650 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd09001d740 tx=0x7fd090013d70 comp rx=0 tx=0).stop 2024-09-12T03:10:42.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.149+0000 7fd0a05fb740 1 -- 172.21.15.116:0/4199927236 shutdown_connections 2024-09-12T03:10:42.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.149+0000 7fd0a05fb740 1 --2- 172.21.15.116:0/4199927236 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5578f219eef0 0x5578f21a12e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.150+0000 7fd0a05fb740 1 --2- 172.21.15.116:0/4199927236 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578f218f370 0x5578f218c650 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.150+0000 7fd0a05fb740 1 --2- 172.21.15.116:0/4199927236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5578f2185b40 0x5578f218c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.150+0000 7fd076ffd640 0 client.17226 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:10:42.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.150+0000 7fd0a05fb740 1 --2- 172.21.15.116:0/4199927236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578f21845d0 0x5578f218ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.150+0000 7fd0a05fb740 1 -- 172.21.15.116:0/4199927236 >> 172.21.15.116:0/4199927236 conn(0x5578f214d310 msgr2=0x5578f2186960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:42.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.150+0000 7fd0a05fb740 1 -- 172.21.15.116:0/4199927236 shutdown_connections 2024-09-12T03:10:42.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.150+0000 7fd0a05fb740 1 -- 172.21.15.116:0/4199927236 wait complete. 2024-09-12T03:10:42.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.150+0000 7fd0a05fb740 20 client.17226 trim_cache size 0 max 16384 2024-09-12T03:10:42.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.163+0000 7fd0a05fb740 1 Processor -- start 2024-09-12T03:10:42.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.163+0000 7fd0a05fb740 1 -- start start 2024-09-12T03:10:42.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.163+0000 7fd0a05fb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578f21845d0 0x5578f217fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.163+0000 7fd0a05fb740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578f2185b40 0x5578f2197860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.163+0000 7fd0a05fb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5578f218f370 0x5578f2197da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.163+0000 7fd0a05fb740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd0981afeb0 con 0x5578f2185b40 2024-09-12T03:10:42.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.163+0000 7fd0a05fb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd0981ad780 con 0x5578f218f370 2024-09-12T03:10:42.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.163+0000 7fd0a05fb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd0981ab5d0 con 0x5578f21845d0 2024-09-12T03:10:42.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.163+0000 7fd096ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578f2185b40 0x5578f2197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.163+0000 7fd096ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578f2185b40 0x5578f2197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55712/0 (socket says 172.21.15.116:55712) 2024-09-12T03:10:42.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.163+0000 7fd096ffd640 1 -- 172.21.15.116:0/2333583276 learned_addr learned my addr 172.21.15.116:0/2333583276 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:42.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.163+0000 7fd0977fe640 1 --2- 172.21.15.116:0/2333583276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578f21845d0 0x5578f217fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.163+0000 7fd097fff640 1 --2- 172.21.15.116:0/2333583276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5578f218f370 0x5578f2197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.163+0000 7fd096ffd640 1 -- 172.21.15.116:0/2333583276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578f21845d0 msgr2=0x5578f217fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:42.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.163+0000 7fd096ffd640 1 --2- 172.21.15.116:0/2333583276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578f21845d0 0x5578f217fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.163+0000 7fd096ffd640 1 -- 172.21.15.116:0/2333583276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5578f218f370 msgr2=0x5578f2197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:42.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.163+0000 7fd096ffd640 1 --2- 172.21.15.116:0/2333583276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5578f218f370 0x5578f2197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.163+0000 7fd096ffd640 1 -- 172.21.15.116:0/2333583276 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578f218cf20 con 0x5578f2185b40 2024-09-12T03:10:42.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.163+0000 7fd097fff640 1 --2- 172.21.15.116:0/2333583276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5578f218f370 0x5578f2197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:42.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.164+0000 7fd096ffd640 1 --2- 172.21.15.116:0/2333583276 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578f2185b40 0x5578f2197860 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fd08c0096c0 tx=0x7fd08c031900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e3b249c210e4b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:42.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.164+0000 7fd07d7fa640 1 -- 172.21.15.116:0/2333583276 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c035040 con 0x5578f2185b40 2024-09-12T03:10:42.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.164+0000 7fd07d7fa640 1 -- 172.21.15.116:0/2333583276 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd08c005500 con 0x5578f2185b40 2024-09-12T03:10:42.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.164+0000 7fd07d7fa640 1 -- 172.21.15.116:0/2333583276 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c004c00 con 0x5578f2185b40 2024-09-12T03:10:42.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.164+0000 7fd0a05fb740 1 -- 172.21.15.116:0/2333583276 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578f2185b40 msgr2=0x5578f2197860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:42.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.164+0000 7fd0a05fb740 1 --2- 172.21.15.116:0/2333583276 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578f2185b40 0x5578f2197860 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fd08c0096c0 tx=0x7fd08c031900 comp rx=0 tx=0).stop 2024-09-12T03:10:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.164+0000 7fd0a05fb740 1 -- 172.21.15.116:0/2333583276 shutdown_connections 2024-09-12T03:10:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.164+0000 7fd0a05fb740 1 --2- 172.21.15.116:0/2333583276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5578f218f370 0x5578f2197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.164+0000 7fd0a05fb740 1 --2- 172.21.15.116:0/2333583276 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578f2185b40 0x5578f2197860 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.164+0000 7fd0a05fb740 1 --2- 172.21.15.116:0/2333583276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578f21845d0 0x5578f217fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.164+0000 7fd0a05fb740 1 -- 172.21.15.116:0/2333583276 >> 172.21.15.116:0/2333583276 conn(0x5578f214d310 msgr2=0x5578f217f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.164+0000 7fd0a05fb740 1 -- 172.21.15.116:0/2333583276 shutdown_connections 2024-09-12T03:10:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.164+0000 7fd0a05fb740 1 -- 172.21.15.116:0/2333583276 wait complete. 2024-09-12T03:10:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.165+0000 7fd0a05fb740 1 -- start start 2024-09-12T03:10:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.165+0000 7fd0a05fb740 1 Processor -- start 2024-09-12T03:10:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.165+0000 7fd0a05fb740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578f21845d0 0x5578f2201a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.165+0000 7fd0a05fb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5578f2185b40 0x5578f2201f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.165+0000 7fd0a05fb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578f218f370 0x5578f220ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.165+0000 7fd0a05fb740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd0981abc40 con 0x5578f21845d0 2024-09-12T03:10:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.165+0000 7fd0a05fb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd0981ad0b0 con 0x5578f2185b40 2024-09-12T03:10:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.165+0000 7fd0a05fb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd0981ac310 con 0x5578f218f370 2024-09-12T03:10:42.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.165+0000 7fd0a05fb740 10 client.0 _ll_register_callbacks cb 0x7fd09000ae30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:42.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.165+0000 7fd0977fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578f21845d0 0x5578f2201a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.165+0000 7fd0977fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578f21845d0 0x5578f2201a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55728/0 (socket says 172.21.15.116:55728) 2024-09-12T03:10:42.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.165+0000 7fd0977fe640 1 -- 172.21.15.116:0/3382003130 learned_addr learned my addr 172.21.15.116:0/3382003130 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:42.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.166+0000 7fd096ffd640 1 --2- 172.21.15.116:0/3382003130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5578f2185b40 0x5578f2201f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.166+0000 7fd0977fe640 1 -- 172.21.15.116:0/3382003130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578f218f370 msgr2=0x5578f220ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:42.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.166+0000 7fd0977fe640 1 --2- 172.21.15.116:0/3382003130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578f218f370 0x5578f220ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.166+0000 7fd0977fe640 1 -- 172.21.15.116:0/3382003130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5578f2185b40 msgr2=0x5578f2201f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:42.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.166+0000 7fd0977fe640 1 --2- 172.21.15.116:0/3382003130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5578f2185b40 0x5578f2201f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.166+0000 7fd0977fe640 1 -- 172.21.15.116:0/3382003130 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578f220cfc0 con 0x5578f21845d0 2024-09-12T03:10:42.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.166+0000 7fd0977fe640 1 --2- 172.21.15.116:0/3382003130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578f21845d0 0x5578f2201a00 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fd08809ab60 tx=0x7fd0880bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ff57457379559 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:42.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.166+0000 7fd076ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:42.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.166+0000 7fd076ffd640 1 -- 172.21.15.116:0/3382003130 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08800ce50 con 0x5578f21845d0 2024-09-12T03:10:42.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.166+0000 7fd076ffd640 1 -- 172.21.15.116:0/3382003130 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd0880974d0 con 0x5578f21845d0 2024-09-12T03:10:42.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.166+0000 7fd0a05fb740 1 -- 172.21.15.116:0/3382003130 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd08c005500 con 0x5578f21845d0 2024-09-12T03:10:42.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.166+0000 7fd076ffd640 1 -- 172.21.15.116:0/3382003130 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd0880a3640 con 0x5578f21845d0 2024-09-12T03:10:42.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.166+0000 7fd0a05fb740 10 client.17232 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:42.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.166+0000 7fd0a05fb740 1 -- 172.21.15.116:0/3382003130 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5578f220e1d0 con 0x5578f21845d0 2024-09-12T03:10:42.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.166+0000 7fd076ffd640 1 -- 172.21.15.116:0/3382003130 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd08c005500 con 0x5578f21845d0 2024-09-12T03:10:42.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.166+0000 7fd076ffd640 1 -- 172.21.15.116:0/3382003130 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd088096bb0 con 0x5578f21845d0 2024-09-12T03:10:42.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.167+0000 7fd0a05fb740 10 client.17232 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:42.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.167+0000 7fd0a05fb740 10 client.17232 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:10:42.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.167+0000 7fd0a05fb740 20 client.17232 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:42.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.167+0000 7fd0a05fb740 1 --2- 172.21.15.116:0/3382003130 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5578f220e8f0 0x5578f2210d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.167+0000 7fd0a05fb740 4 client.17232 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:10:42.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.167+0000 7fd0a05fb740 1 -- 172.21.15.116:0/3382003130 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5578f20f1d70 con 0x5578f220e8f0 2024-09-12T03:10:42.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.167+0000 7fd096ffd640 1 --2- 172.21.15.116:0/3382003130 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5578f220e8f0 0x5578f2210d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.167+0000 7fd096ffd640 1 --2- 172.21.15.116:0/3382003130 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5578f220e8f0 0x5578f2210d50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=77be50027ff62b4b server_cookie=8a2fbc2adf18beb8 in_seq=0 out_seq=0 2024-09-12T03:10:42.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.167+0000 7fd076ffd640 10 client.17232 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:10:42.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.167+0000 7fd076ffd640 1 -- 172.21.15.116:0/3382003130 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5578f20f1d70 con 0x5578f220e8f0 2024-09-12T03:10:42.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.167+0000 7fd076ffd640 10 client.17232 handle_command_reply: tid=0 2024-09-12T03:10:42.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.168+0000 7fd0a05fb740 1 client.17232 shutdown 2024-09-12T03:10:42.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.168+0000 7fd0a05fb740 1 -- 172.21.15.116:0/3382003130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578f21845d0 msgr2=0x5578f2201a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:42.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.168+0000 7fd0a05fb740 1 --2- 172.21.15.116:0/3382003130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578f21845d0 0x5578f2201a00 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fd08809ab60 tx=0x7fd0880bb040 comp rx=0 tx=0).stop 2024-09-12T03:10:42.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.168+0000 7fd0a05fb740 1 -- 172.21.15.116:0/3382003130 shutdown_connections 2024-09-12T03:10:42.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.168+0000 7fd0a05fb740 1 --2- 172.21.15.116:0/3382003130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5578f218f370 0x5578f220ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.168+0000 7fd0a05fb740 1 --2- 172.21.15.116:0/3382003130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5578f2185b40 0x5578f2201f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.168+0000 7fd0a05fb740 1 --2- 172.21.15.116:0/3382003130 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5578f220e8f0 0x5578f2210d50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.168+0000 7fd0a05fb740 1 --2- 172.21.15.116:0/3382003130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578f21845d0 0x5578f2201a00 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.168+0000 7fd0a05fb740 1 -- 172.21.15.116:0/3382003130 >> 172.21.15.116:0/3382003130 conn(0x5578f214d310 msgr2=0x5578f218e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:42.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.168+0000 7fd076ffd640 0 client.17232 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:10:42.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.168+0000 7fd0a05fb740 1 -- 172.21.15.116:0/3382003130 shutdown_connections 2024-09-12T03:10:42.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.168+0000 7fd0a05fb740 1 -- 172.21.15.116:0/3382003130 wait complete. 2024-09-12T03:10:42.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.168+0000 7fd0a05fb740 20 client.17232 trim_cache size 0 max 16384 2024-09-12T03:10:42.181 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:10:42.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.168+0000 7fd09e222640 1 -- 172.21.15.116:0/810530575 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd0800a2c00 msgr2=0x7fd0800a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:42.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.168+0000 7fd09e222640 1 --2- 172.21.15.116:0/810530575 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd0800a2c00 0x7fd0800a50c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd09001e000 tx=0x7fd09001c000 comp rx=0 tx=0).stop 2024-09-12T03:10:42.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.168+0000 7fd09e222640 1 -- 172.21.15.116:0/810530575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd098100a80 msgr2=0x7fd0981985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:42.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.168+0000 7fd09e222640 1 --2- 172.21.15.116:0/810530575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd098100a80 0x7fd0981985c0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fd08800ba50 tx=0x7fd08800bf10 comp rx=0 tx=0).stop 2024-09-12T03:10:42.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.168+0000 7fd09e222640 1 -- 172.21.15.116:0/810530575 shutdown_connections 2024-09-12T03:10:42.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.168+0000 7fd09e222640 1 --2- 172.21.15.116:0/810530575 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd0800a2c00 0x7fd0800a50c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.168+0000 7fd09e222640 1 --2- 172.21.15.116:0/810530575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd098106a80 0x7fd09819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.168+0000 7fd09e222640 1 --2- 172.21.15.116:0/810530575 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd098101420 0x7fd098198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.168+0000 7fd09e222640 1 --2- 172.21.15.116:0/810530575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd098100a80 0x7fd0981985c0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.168+0000 7fd09e222640 1 -- 172.21.15.116:0/810530575 >> 172.21.15.116:0/810530575 conn(0x7fd0980fc7a0 msgr2=0x7fd0980fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:42.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.169+0000 7fd09e222640 1 -- 172.21.15.116:0/810530575 shutdown_connections 2024-09-12T03:10:42.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:42.169+0000 7fd09e222640 1 -- 172.21.15.116:0/810530575 wait complete. 2024-09-12T03:10:43.012 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:10:43.271 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:10:43.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.757+0000 7f4f61172640 1 -- 172.21.15.116:0/1295030426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f5c106f10 msgr2=0x7f4f5c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:43.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.757+0000 7f4f61172640 1 --2- 172.21.15.116:0/1295030426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f5c106f10 0x7f4f5c10ea90 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009ca0 tx=0x7f4f5402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.757+0000 7f4f61172640 1 -- 172.21.15.116:0/1295030426 shutdown_connections 2024-09-12T03:10:43.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.757+0000 7f4f61172640 1 --2- 172.21.15.116:0/1295030426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f5c106f10 0x7f4f5c10ea90 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.757+0000 7f4f61172640 1 --2- 172.21.15.116:0/1295030426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f5c102790 0x7f4f5c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.757+0000 7f4f61172640 1 --2- 172.21.15.116:0/1295030426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f5c101de0 0x7f4f5c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.757+0000 7f4f61172640 1 -- 172.21.15.116:0/1295030426 >> 172.21.15.116:0/1295030426 conn(0x7f4f5c0fd8c0 msgr2=0x7f4f5c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:43.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.757+0000 7f4f61172640 1 -- 172.21.15.116:0/1295030426 shutdown_connections 2024-09-12T03:10:43.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.757+0000 7f4f61172640 1 -- 172.21.15.116:0/1295030426 wait complete. 2024-09-12T03:10:43.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.758+0000 7f4f61172640 1 Processor -- start 2024-09-12T03:10:43.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.758+0000 7f4f61172640 1 -- start start 2024-09-12T03:10:43.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.758+0000 7f4f61172640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f5c101de0 0x7f4f5c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:43.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.758+0000 7f4f61172640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f5c102790 0x7f4f5c1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:43.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.758+0000 7f4f61172640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f5c106f10 0x7f4f5c19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:43.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.758+0000 7f4f61172640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5c111f70 con 0x7f4f5c101de0 2024-09-12T03:10:43.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.758+0000 7f4f61172640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5c111df0 con 0x7f4f5c106f10 2024-09-12T03:10:43.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.758+0000 7f4f61172640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5c1120f0 con 0x7f4f5c102790 2024-09-12T03:10:43.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.758+0000 7f4f5ad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f5c101de0 0x7f4f5c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:43.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.758+0000 7f4f5ad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f5c101de0 0x7f4f5c198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55758/0 (socket says 172.21.15.116:55758) 2024-09-12T03:10:43.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.758+0000 7f4f5ad9d640 1 -- 172.21.15.116:0/1528206548 learned_addr learned my addr 172.21.15.116:0/1528206548 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:43.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.759+0000 7f4f5b59e640 1 --2- 172.21.15.116:0/1528206548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f5c106f10 0x7f4f5c19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:43.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.759+0000 7f4f5ad9d640 1 -- 172.21.15.116:0/1528206548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f5c102790 msgr2=0x7f4f5c1987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:43.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.759+0000 7f4f5a59c640 1 --2- 172.21.15.116:0/1528206548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f5c102790 0x7f4f5c1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:43.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.759+0000 7f4f5ad9d640 1 --2- 172.21.15.116:0/1528206548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f5c102790 0x7f4f5c1987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.759+0000 7f4f5ad9d640 1 -- 172.21.15.116:0/1528206548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f5c106f10 msgr2=0x7f4f5c19cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:43.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.759+0000 7f4f5ad9d640 1 --2- 172.21.15.116:0/1528206548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f5c106f10 0x7f4f5c19cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.759+0000 7f4f5ad9d640 1 -- 172.21.15.116:0/1528206548 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f5c19d230 con 0x7f4f5c101de0 2024-09-12T03:10:43.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.759+0000 7f4f5b59e640 1 --2- 172.21.15.116:0/1528206548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f5c106f10 0x7f4f5c19cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:10:43.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.759+0000 7f4f5a59c640 1 --2- 172.21.15.116:0/1528206548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f5c102790 0x7f4f5c1987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:43.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.759+0000 7f4f5ad9d640 1 --2- 172.21.15.116:0/1528206548 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f5c101de0 0x7f4f5c198280 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00c920 tx=0x7f4f4c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:43.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.759+0000 7f4f43fff640 1 -- 172.21.15.116:0/1528206548 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c007c70 con 0x7f4f5c101de0 2024-09-12T03:10:43.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.759+0000 7f4f61172640 1 -- 172.21.15.116:0/1528206548 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f5c19d520 con 0x7f4f5c101de0 2024-09-12T03:10:43.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.759+0000 7f4f43fff640 1 -- 172.21.15.116:0/1528206548 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f4c007e10 con 0x7f4f5c101de0 2024-09-12T03:10:43.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.759+0000 7f4f43fff640 1 -- 172.21.15.116:0/1528206548 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c018890 con 0x7f4f5c101de0 2024-09-12T03:10:43.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.759+0000 7f4f61172640 1 -- 172.21.15.116:0/1528206548 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f5c1a4e00 con 0x7f4f5c101de0 2024-09-12T03:10:43.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.761+0000 7f4f43fff640 1 -- 172.21.15.116:0/1528206548 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4f4c018b30 con 0x7f4f5c101de0 2024-09-12T03:10:43.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.762+0000 7f4f6354b740 1 Processor -- start 2024-09-12T03:10:43.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.762+0000 7f4f6354b740 1 -- start start 2024-09-12T03:10:43.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.762+0000 7f4f6354b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564393e6eb40 0x564393e6d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:43.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.762+0000 7f4f6354b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564393e6d5d0 0x564393e6da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:43.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.762+0000 7f4f6354b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564393e78370 0x564393e7a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:43.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.762+0000 7f4f6354b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x564393dd32f0 con 0x564393e6d5d0 2024-09-12T03:10:43.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.762+0000 7f4f6354b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x564393d7acf0 con 0x564393e6eb40 2024-09-12T03:10:43.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.762+0000 7f4f6354b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564393dcf2b0 con 0x564393e78370 2024-09-12T03:10:43.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.762+0000 7f4f43fff640 1 --2- 172.21.15.116:0/1528206548 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4f440a2b80 0x7f4f440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:43.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.762+0000 7f4f5ad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564393e6d5d0 0x564393e6da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:43.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.762+0000 7f4f5ad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564393e6d5d0 0x564393e6da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55762/0 (socket says 172.21.15.116:55762) 2024-09-12T03:10:43.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.762+0000 7f4f5ad9d640 1 -- 172.21.15.116:0/1429448488 learned_addr learned my addr 172.21.15.116:0/1429448488 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:43.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.762+0000 7f4f5a59c640 1 --2- 172.21.15.116:0/1429448488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564393e78370 0x564393e7a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:43.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.762+0000 7f4f5b59e640 1 --2- 172.21.15.116:0/1429448488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564393e6eb40 0x564393e6d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:43.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.762+0000 7f4f5b59e640 1 --2- 172.21.15.116:0/1528206548 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4f440a2b80 0x7f4f440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:43.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.762+0000 7f4f43fff640 1 -- 172.21.15.116:0/1528206548 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f4f4c0cf230 con 0x7f4f5c101de0 2024-09-12T03:10:43.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.762+0000 7f4f5ad9d640 1 -- 172.21.15.116:0/1429448488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564393e78370 msgr2=0x564393e7a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:43.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.762+0000 7f4f5ad9d640 1 --2- 172.21.15.116:0/1429448488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564393e78370 0x564393e7a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.762+0000 7f4f5ad9d640 1 -- 172.21.15.116:0/1429448488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564393e6eb40 msgr2=0x564393e6d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:43.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.762+0000 7f4f5ad9d640 1 --2- 172.21.15.116:0/1429448488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564393e6eb40 0x564393e6d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.763+0000 7f4f5ad9d640 1 -- 172.21.15.116:0/1429448488 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564393d5f3a0 con 0x564393e6d5d0 2024-09-12T03:10:43.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.763+0000 7f4f5ad9d640 1 --2- 172.21.15.116:0/1429448488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564393e6d5d0 0x564393e6da10 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c0a4200 tx=0x7f4f4c00ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcae54733f9e529e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:43.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.763+0000 7f4f5b59e640 1 --2- 172.21.15.116:0/1528206548 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4f440a2b80 0x7f4f440a5040 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009a60 tx=0x7f4f540095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:43.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.763+0000 7f4f417fa640 1 -- 172.21.15.116:0/1429448488 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c00de20 con 0x564393e6d5d0 2024-09-12T03:10:43.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.763+0000 7f4f417fa640 1 -- 172.21.15.116:0/1429448488 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f4c018e20 con 0x564393e6d5d0 2024-09-12T03:10:43.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.763+0000 7f4f417fa640 1 -- 172.21.15.116:0/1429448488 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c00fb30 con 0x564393e6d5d0 2024-09-12T03:10:43.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.764+0000 7f4f6354b740 1 -- 172.21.15.116:0/1429448488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564393e6d5d0 msgr2=0x564393e6da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:43.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.764+0000 7f4f6354b740 1 --2- 172.21.15.116:0/1429448488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564393e6d5d0 0x564393e6da10 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c0a4200 tx=0x7f4f4c00ff30 comp rx=0 tx=0).stop 2024-09-12T03:10:43.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.764+0000 7f4f6354b740 1 -- 172.21.15.116:0/1429448488 shutdown_connections 2024-09-12T03:10:43.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.764+0000 7f4f6354b740 1 --2- 172.21.15.116:0/1429448488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564393e78370 0x564393e7a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.764+0000 7f4f6354b740 1 --2- 172.21.15.116:0/1429448488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564393e6d5d0 0x564393e6da10 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.764+0000 7f4f6354b740 1 --2- 172.21.15.116:0/1429448488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564393e6eb40 0x564393e6d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.764+0000 7f4f6354b740 1 -- 172.21.15.116:0/1429448488 >> 172.21.15.116:0/1429448488 conn(0x564393e36310 msgr2=0x564393e69c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:43.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.764+0000 7f4f6354b740 1 -- 172.21.15.116:0/1429448488 shutdown_connections 2024-09-12T03:10:43.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.764+0000 7f4f6354b740 1 -- 172.21.15.116:0/1429448488 wait complete. 2024-09-12T03:10:43.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.764+0000 7f4f6354b740 1 -- start start 2024-09-12T03:10:43.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.764+0000 7f4f6354b740 1 Processor -- start 2024-09-12T03:10:43.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.764+0000 7f4f6354b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564393e6d5d0 0x564393e77e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:43.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.764+0000 7f4f6354b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564393e6eb40 0x564393e750d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:43.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.764+0000 7f4f6354b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564393e78370 0x564393e75860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:43.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.764+0000 7f4f6354b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x564393e7b070 con 0x564393e78370 2024-09-12T03:10:43.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.764+0000 7f4f6354b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x564393d8e060 con 0x564393e6d5d0 2024-09-12T03:10:43.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.764+0000 7f4f6354b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564393e7b1f0 con 0x564393e6eb40 2024-09-12T03:10:43.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.764+0000 7f4f6354b740 10 client.0 _ll_register_callbacks cb 0x564393ae18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:43.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.764+0000 7f4f5b59e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564393e78370 0x564393e75860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:43.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.764+0000 7f4f5b59e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564393e78370 0x564393e75860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55778/0 (socket says 172.21.15.116:55778) 2024-09-12T03:10:43.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.764+0000 7f4f5b59e640 1 -- 172.21.15.116:0/1389615102 learned_addr learned my addr 172.21.15.116:0/1389615102 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:43.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.764+0000 7f4f5ad9d640 1 --2- 172.21.15.116:0/1389615102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564393e6d5d0 0x564393e77e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:43.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.765+0000 7f4f5a59c640 1 --2- 172.21.15.116:0/1389615102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564393e6eb40 0x564393e750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:43.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.765+0000 7f4f5b59e640 1 -- 172.21.15.116:0/1389615102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564393e6eb40 msgr2=0x564393e750d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:43.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.765+0000 7f4f5b59e640 1 --2- 172.21.15.116:0/1389615102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564393e6eb40 0x564393e750d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.765+0000 7f4f5b59e640 1 -- 172.21.15.116:0/1389615102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564393e6d5d0 msgr2=0x564393e77e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:43.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.765+0000 7f4f5b59e640 1 --2- 172.21.15.116:0/1389615102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564393e6d5d0 0x564393e77e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.765+0000 7f4f5b59e640 1 -- 172.21.15.116:0/1389615102 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564393b443c0 con 0x564393e78370 2024-09-12T03:10:43.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.765+0000 7f4f5ad9d640 1 --2- 172.21.15.116:0/1389615102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564393e6d5d0 0x564393e77e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:10:43.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.765+0000 7f4f5b59e640 1 --2- 172.21.15.116:0/1389615102 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564393e78370 0x564393e75860 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009ca0 tx=0x7f4f5402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89eb06ff22b0a87 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:43.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.765+0000 7f4f3eff5640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:43.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.765+0000 7f4f3eff5640 1 -- 172.21.15.116:0/1389615102 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f54004100 con 0x564393e78370 2024-09-12T03:10:43.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.765+0000 7f4f3eff5640 1 -- 172.21.15.116:0/1389615102 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f4f540042a0 con 0x564393e78370 2024-09-12T03:10:43.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.765+0000 7f4f6354b740 1 -- 172.21.15.116:0/1389615102 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f4c018e20 con 0x564393e78370 2024-09-12T03:10:43.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.765+0000 7f4f3eff5640 1 -- 172.21.15.116:0/1389615102 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4f540044a0 con 0x564393e78370 2024-09-12T03:10:43.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.765+0000 7f4f6354b740 10 client.17244 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:43.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.765+0000 7f4f6354b740 1 -- 172.21.15.116:0/1389615102 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564393e86be0 con 0x564393e78370 2024-09-12T03:10:43.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.766+0000 7f4f3eff5640 1 -- 172.21.15.116:0/1389615102 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f4c018e20 con 0x564393e78370 2024-09-12T03:10:43.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.766+0000 7f4f3eff5640 1 -- 172.21.15.116:0/1389615102 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4f54030650 con 0x564393e78370 2024-09-12T03:10:43.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.766+0000 7f4f6354b740 10 client.17244 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:43.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.766+0000 7f4f6354b740 10 client.17244 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:10:43.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.766+0000 7f4f6354b740 20 client.17244 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:43.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.766+0000 7f4f6354b740 1 --2- 172.21.15.116:0/1389615102 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x564393e880f0 0x564393e8a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:43.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.766+0000 7f4f6354b740 4 client.17244 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:10:43.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.766+0000 7f4f6354b740 1 -- 172.21.15.116:0/1389615102 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564393de0860 con 0x564393e880f0 2024-09-12T03:10:43.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.766+0000 7f4f5ad9d640 1 --2- 172.21.15.116:0/1389615102 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x564393e880f0 0x564393e8a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:43.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.767+0000 7f4f5ad9d640 1 --2- 172.21.15.116:0/1389615102 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x564393e880f0 0x564393e8a530 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=89ec416bc3c74497 server_cookie=c29da2deebec47ee in_seq=0 out_seq=0 2024-09-12T03:10:43.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.767+0000 7f4f3eff5640 10 client.17244 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:10:43.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.767+0000 7f4f3eff5640 1 -- 172.21.15.116:0/1389615102 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564393de0860 con 0x564393e880f0 2024-09-12T03:10:43.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.767+0000 7f4f3eff5640 10 client.17244 handle_command_reply: tid=0 2024-09-12T03:10:43.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.767+0000 7f4f6354b740 1 client.17244 shutdown 2024-09-12T03:10:43.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.767+0000 7f4f6354b740 1 -- 172.21.15.116:0/1389615102 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564393e78370 msgr2=0x564393e75860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:43.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.767+0000 7f4f6354b740 1 --2- 172.21.15.116:0/1389615102 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564393e78370 0x564393e75860 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009ca0 tx=0x7f4f5402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.767+0000 7f4f6354b740 1 -- 172.21.15.116:0/1389615102 shutdown_connections 2024-09-12T03:10:43.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.767+0000 7f4f6354b740 1 --2- 172.21.15.116:0/1389615102 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x564393e880f0 0x564393e8a530 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.767+0000 7f4f6354b740 1 --2- 172.21.15.116:0/1389615102 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564393e78370 0x564393e75860 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.767+0000 7f4f6354b740 1 --2- 172.21.15.116:0/1389615102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564393e6eb40 0x564393e750d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.767+0000 7f4f6354b740 1 --2- 172.21.15.116:0/1389615102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564393e6d5d0 0x564393e77e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.767+0000 7f4f6354b740 1 -- 172.21.15.116:0/1389615102 >> 172.21.15.116:0/1389615102 conn(0x564393e36310 msgr2=0x564393e6f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:43.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.767+0000 7f4f3eff5640 0 client.17244 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:10:43.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.768+0000 7f4f6354b740 1 -- 172.21.15.116:0/1389615102 shutdown_connections 2024-09-12T03:10:43.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.768+0000 7f4f6354b740 1 -- 172.21.15.116:0/1389615102 wait complete. 2024-09-12T03:10:43.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.768+0000 7f4f6354b740 20 client.17244 trim_cache size 0 max 16384 2024-09-12T03:10:43.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.780+0000 7f4f6354b740 1 Processor -- start 2024-09-12T03:10:43.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.780+0000 7f4f6354b740 1 -- start start 2024-09-12T03:10:43.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.780+0000 7f4f6354b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564393e6d5d0 0x564393e69e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:43.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.780+0000 7f4f6354b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564393e6eb40 0x564393e70180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:43.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.780+0000 7f4f6354b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564393e78370 0x564393e706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:43.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.780+0000 7f4f6354b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5c1afc90 con 0x564393e6d5d0 2024-09-12T03:10:43.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.780+0000 7f4f6354b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5c1ad560 con 0x564393e6eb40 2024-09-12T03:10:43.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.780+0000 7f4f6354b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5c1ab3b0 con 0x564393e78370 2024-09-12T03:10:43.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.781+0000 7f4f5ad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564393e6d5d0 0x564393e69e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:43.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.781+0000 7f4f5ad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564393e6d5d0 0x564393e69e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55782/0 (socket says 172.21.15.116:55782) 2024-09-12T03:10:43.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.781+0000 7f4f5ad9d640 1 -- 172.21.15.116:0/2781677346 learned_addr learned my addr 172.21.15.116:0/2781677346 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:43.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.781+0000 7f4f5ad9d640 1 -- 172.21.15.116:0/2781677346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564393e78370 msgr2=0x564393e706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:43.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.781+0000 7f4f5ad9d640 1 --2- 172.21.15.116:0/2781677346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564393e78370 0x564393e706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.781+0000 7f4f5ad9d640 1 -- 172.21.15.116:0/2781677346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564393e6eb40 msgr2=0x564393e70180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:43.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.781+0000 7f4f5a59c640 1 --2- 172.21.15.116:0/2781677346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564393e6eb40 0x564393e70180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:43.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.781+0000 7f4f5ad9d640 1 --2- 172.21.15.116:0/2781677346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564393e6eb40 0x564393e70180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.781+0000 7f4f5ad9d640 1 -- 172.21.15.116:0/2781677346 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564393e6e300 con 0x564393e6d5d0 2024-09-12T03:10:43.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.781+0000 7f4f5a59c640 1 --2- 172.21.15.116:0/2781677346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564393e6eb40 0x564393e70180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:43.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.781+0000 7f4f5ad9d640 1 --2- 172.21.15.116:0/2781677346 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564393e6d5d0 0x564393e69e10 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c09ba50 tx=0x7f4f4c097d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a25a881f4322c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:43.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.781+0000 7f4f40ff9640 1 -- 172.21.15.116:0/2781677346 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c0b8020 con 0x564393e6d5d0 2024-09-12T03:10:43.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.781+0000 7f4f40ff9640 1 -- 172.21.15.116:0/2781677346 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f4c018e20 con 0x564393e6d5d0 2024-09-12T03:10:43.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.781+0000 7f4f40ff9640 1 -- 172.21.15.116:0/2781677346 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c0a4500 con 0x564393e6d5d0 2024-09-12T03:10:43.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.782+0000 7f4f6354b740 1 -- 172.21.15.116:0/2781677346 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564393e6d5d0 msgr2=0x564393e69e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:43.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.782+0000 7f4f6354b740 1 --2- 172.21.15.116:0/2781677346 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564393e6d5d0 0x564393e69e10 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c09ba50 tx=0x7f4f4c097d60 comp rx=0 tx=0).stop 2024-09-12T03:10:43.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.782+0000 7f4f6354b740 1 -- 172.21.15.116:0/2781677346 shutdown_connections 2024-09-12T03:10:43.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.782+0000 7f4f6354b740 1 --2- 172.21.15.116:0/2781677346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564393e78370 0x564393e706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.782+0000 7f4f6354b740 1 --2- 172.21.15.116:0/2781677346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564393e6eb40 0x564393e70180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.782+0000 7f4f6354b740 1 --2- 172.21.15.116:0/2781677346 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564393e6d5d0 0x564393e69e10 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.782+0000 7f4f6354b740 1 -- 172.21.15.116:0/2781677346 >> 172.21.15.116:0/2781677346 conn(0x564393e36310 msgr2=0x564393e69700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:43.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.782+0000 7f4f6354b740 1 -- 172.21.15.116:0/2781677346 shutdown_connections 2024-09-12T03:10:43.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.782+0000 7f4f6354b740 1 -- 172.21.15.116:0/2781677346 wait complete. 2024-09-12T03:10:43.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.782+0000 7f4f6354b740 1 -- start start 2024-09-12T03:10:43.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.782+0000 7f4f6354b740 1 Processor -- start 2024-09-12T03:10:43.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.782+0000 7f4f6354b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564393e6d5d0 0x564393e80880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:43.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.783+0000 7f4f6354b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564393e6eb40 0x564393e71aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:43.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.783+0000 7f4f6354b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564393e78370 0x564393e71fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:43.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.783+0000 7f4f6354b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5c1aba20 con 0x564393e6eb40 2024-09-12T03:10:43.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.783+0000 7f4f6354b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5c1ace90 con 0x564393e78370 2024-09-12T03:10:43.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.783+0000 7f4f6354b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5c1ac0f0 con 0x564393e6d5d0 2024-09-12T03:10:43.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.783+0000 7f4f6354b740 10 client.0 _ll_register_callbacks cb 0x7f4f54009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:43.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.783+0000 7f4f5a59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564393e6eb40 0x564393e71aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:43.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.783+0000 7f4f5a59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564393e6eb40 0x564393e71aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55784/0 (socket says 172.21.15.116:55784) 2024-09-12T03:10:43.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.783+0000 7f4f5a59c640 1 -- 172.21.15.116:0/1816315953 learned_addr learned my addr 172.21.15.116:0/1816315953 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:43.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.783+0000 7f4f5a59c640 1 -- 172.21.15.116:0/1816315953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564393e6d5d0 msgr2=0x564393e80880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:43.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.783+0000 7f4f5a59c640 1 --2- 172.21.15.116:0/1816315953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564393e6d5d0 0x564393e80880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.783+0000 7f4f5a59c640 1 -- 172.21.15.116:0/1816315953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564393e78370 msgr2=0x564393e71fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:43.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.783+0000 7f4f5b59e640 1 --2- 172.21.15.116:0/1816315953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564393e78370 0x564393e71fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:43.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.783+0000 7f4f5a59c640 1 --2- 172.21.15.116:0/1816315953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564393e78370 0x564393e71fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.783+0000 7f4f5a59c640 1 -- 172.21.15.116:0/1816315953 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564393e72520 con 0x564393e6eb40 2024-09-12T03:10:43.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.783+0000 7f4f5a59c640 1 --2- 172.21.15.116:0/1816315953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564393e6eb40 0x564393e71aa0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x564393e6e070 tx=0x7f4f5000fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e58564e39b4e809 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:43.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.783+0000 7f4f3eff5640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:43.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.783+0000 7f4f3eff5640 1 -- 172.21.15.116:0/1816315953 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5000ecf0 con 0x564393e6eb40 2024-09-12T03:10:43.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.783+0000 7f4f3eff5640 1 -- 172.21.15.116:0/1816315953 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f4f50005020 con 0x564393e6eb40 2024-09-12T03:10:43.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.783+0000 7f4f6354b740 1 -- 172.21.15.116:0/1816315953 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f4c018e20 con 0x564393e6eb40 2024-09-12T03:10:43.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.783+0000 7f4f3eff5640 1 -- 172.21.15.116:0/1816315953 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4f50005260 con 0x564393e6eb40 2024-09-12T03:10:43.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.783+0000 7f4f6354b740 10 client.17250 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:43.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.783+0000 7f4f6354b740 1 -- 172.21.15.116:0/1816315953 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564393ef2ae0 con 0x564393e6eb40 2024-09-12T03:10:43.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.784+0000 7f4f3eff5640 1 -- 172.21.15.116:0/1816315953 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f4c018e20 con 0x564393e6eb40 2024-09-12T03:10:43.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.784+0000 7f4f3eff5640 1 -- 172.21.15.116:0/1816315953 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4f50026480 con 0x564393e6eb40 2024-09-12T03:10:43.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.784+0000 7f4f6354b740 10 client.17250 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:43.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.784+0000 7f4f6354b740 10 client.17250 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:10:43.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.784+0000 7f4f6354b740 20 client.17250 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:43.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.784+0000 7f4f6354b740 1 --2- 172.21.15.116:0/1816315953 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x564393ef32c0 0x564393ef56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:43.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.784+0000 7f4f6354b740 4 client.17250 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:10:43.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.784+0000 7f4f6354b740 1 -- 172.21.15.116:0/1816315953 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564393ddad70 con 0x564393ef32c0 2024-09-12T03:10:43.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.784+0000 7f4f5ad9d640 1 --2- 172.21.15.116:0/1816315953 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x564393ef32c0 0x564393ef56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:43.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.785+0000 7f4f3eff5640 10 client.17250 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:10:43.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.785+0000 7f4f5ad9d640 1 --2- 172.21.15.116:0/1816315953 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x564393ef32c0 0x564393ef56e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fe8f8bf48a20ab13 server_cookie=f4bdeb286e3f8063 in_seq=0 out_seq=0 2024-09-12T03:10:43.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.785+0000 7f4f3eff5640 1 -- 172.21.15.116:0/1816315953 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564393ddad70 con 0x564393ef32c0 2024-09-12T03:10:43.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.785+0000 7f4f3eff5640 10 client.17250 handle_command_reply: tid=0 2024-09-12T03:10:43.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.785+0000 7f4f6354b740 1 client.17250 shutdown 2024-09-12T03:10:43.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.785+0000 7f4f6354b740 1 -- 172.21.15.116:0/1816315953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564393e6eb40 msgr2=0x564393e71aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:43.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.785+0000 7f4f6354b740 1 --2- 172.21.15.116:0/1816315953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564393e6eb40 0x564393e71aa0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x564393e6e070 tx=0x7f4f5000fc70 comp rx=0 tx=0).stop 2024-09-12T03:10:43.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.785+0000 7f4f6354b740 1 -- 172.21.15.116:0/1816315953 shutdown_connections 2024-09-12T03:10:43.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.785+0000 7f4f6354b740 1 --2- 172.21.15.116:0/1816315953 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x564393ef32c0 0x564393ef56e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.785+0000 7f4f6354b740 1 --2- 172.21.15.116:0/1816315953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564393e78370 0x564393e71fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.785+0000 7f4f6354b740 1 --2- 172.21.15.116:0/1816315953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564393e6eb40 0x564393e71aa0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.785+0000 7f4f6354b740 1 --2- 172.21.15.116:0/1816315953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564393e6d5d0 0x564393e80880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.785+0000 7f4f3eff5640 0 client.17250 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:10:43.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.785+0000 7f4f6354b740 1 -- 172.21.15.116:0/1816315953 >> 172.21.15.116:0/1816315953 conn(0x564393e36310 msgr2=0x564393e77150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:43.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.785+0000 7f4f6354b740 1 -- 172.21.15.116:0/1816315953 shutdown_connections 2024-09-12T03:10:43.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.785+0000 7f4f6354b740 1 -- 172.21.15.116:0/1816315953 wait complete. 2024-09-12T03:10:43.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.785+0000 7f4f6354b740 20 client.17250 trim_cache size 0 max 16384 2024-09-12T03:10:43.797 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:10:43.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.786+0000 7f4f61172640 1 -- 172.21.15.116:0/1528206548 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4f440a2b80 msgr2=0x7f4f440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:43.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.786+0000 7f4f61172640 1 --2- 172.21.15.116:0/1528206548 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4f440a2b80 0x7f4f440a5040 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009a60 tx=0x7f4f540095c0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.786+0000 7f4f61172640 1 -- 172.21.15.116:0/1528206548 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f5c101de0 msgr2=0x7f4f5c198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:43.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.786+0000 7f4f61172640 1 --2- 172.21.15.116:0/1528206548 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f5c101de0 0x7f4f5c198280 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00c920 tx=0x7f4f4c00cde0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.786+0000 7f4f61172640 1 -- 172.21.15.116:0/1528206548 shutdown_connections 2024-09-12T03:10:43.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.786+0000 7f4f61172640 1 --2- 172.21.15.116:0/1528206548 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4f440a2b80 0x7f4f440a5040 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.786+0000 7f4f61172640 1 --2- 172.21.15.116:0/1528206548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f5c106f10 0x7f4f5c19cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.786+0000 7f4f61172640 1 --2- 172.21.15.116:0/1528206548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f5c102790 0x7f4f5c1987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.786+0000 7f4f61172640 1 --2- 172.21.15.116:0/1528206548 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f5c101de0 0x7f4f5c198280 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.786+0000 7f4f61172640 1 -- 172.21.15.116:0/1528206548 >> 172.21.15.116:0/1528206548 conn(0x7f4f5c0fd8c0 msgr2=0x7f4f5c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:43.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.786+0000 7f4f61172640 1 -- 172.21.15.116:0/1528206548 shutdown_connections 2024-09-12T03:10:43.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:43.786+0000 7f4f61172640 1 -- 172.21.15.116:0/1528206548 wait complete. 2024-09-12T03:10:44.649 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:44 smithi116 ceph-mon[31410]: pgmap v570: 97 pgs: 97 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:10:44.682 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:10:44.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:44 smithi183 ceph-mon[35535]: pgmap v570: 97 pgs: 97 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:10:44.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:44 smithi177 ceph-mon[35305]: pgmap v570: 97 pgs: 97 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:10:44.888 INFO:tasks.quiescer.fs.[cephfs]:Total duration (38.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T03:10:44.888 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.4 2024-09-12T03:10:44.892 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:10:44.933 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:10:45.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.427+0000 7f3812578640 1 -- 172.21.15.116:0/3971279301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f380c102ef0 msgr2=0x7f380c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:45.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.427+0000 7f3812578640 1 --2- 172.21.15.116:0/3971279301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f380c102ef0 0x7f380c10da40 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f3804009ca0 tx=0x7f380402a950 comp rx=0 tx=0).stop 2024-09-12T03:10:45.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.427+0000 7f3812578640 1 -- 172.21.15.116:0/3971279301 shutdown_connections 2024-09-12T03:10:45.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.427+0000 7f3812578640 1 --2- 172.21.15.116:0/3971279301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f380c102ef0 0x7f380c10da40 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.427+0000 7f3812578640 1 --2- 172.21.15.116:0/3971279301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f380c102570 0x7f380c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.427+0000 7f3812578640 1 --2- 172.21.15.116:0/3971279301 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f380c100e20 0x7f380c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.427+0000 7f3812578640 1 -- 172.21.15.116:0/3971279301 >> 172.21.15.116:0/3971279301 conn(0x7f380c0fc7d0 msgr2=0x7f380c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:45.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.427+0000 7f3812578640 1 -- 172.21.15.116:0/3971279301 shutdown_connections 2024-09-12T03:10:45.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.427+0000 7f3812578640 1 -- 172.21.15.116:0/3971279301 wait complete. 2024-09-12T03:10:45.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.427+0000 7f3812578640 1 Processor -- start 2024-09-12T03:10:45.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.427+0000 7f3812578640 1 -- start start 2024-09-12T03:10:45.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.427+0000 7f3812578640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f380c100e20 0x7f380c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:45.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.427+0000 7f3812578640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f380c102570 0x7f380c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:45.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.427+0000 7f3812578640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f380c102ef0 0x7f380c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:45.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.427+0000 7f3812578640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f380c110fa0 con 0x7f380c100e20 2024-09-12T03:10:45.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.428+0000 7f3812578640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f380c110e20 con 0x7f380c102ef0 2024-09-12T03:10:45.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.428+0000 7f3812578640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f380c111120 con 0x7f380c102570 2024-09-12T03:10:45.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.428+0000 7f380bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f380c100e20 0x7f380c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:45.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.428+0000 7f380bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f380c100e20 0x7f380c0fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55810/0 (socket says 172.21.15.116:55810) 2024-09-12T03:10:45.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.428+0000 7f380bfff640 1 -- 172.21.15.116:0/1261858646 learned_addr learned my addr 172.21.15.116:0/1261858646 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:45.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.428+0000 7f3810b15640 1 --2- 172.21.15.116:0/1261858646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f380c102ef0 0x7f380c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:45.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.428+0000 7f380bfff640 1 -- 172.21.15.116:0/1261858646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f380c102570 msgr2=0x7f380c0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:45.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.428+0000 7f3803fff640 1 --2- 172.21.15.116:0/1261858646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f380c102570 0x7f380c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:45.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.428+0000 7f380bfff640 1 --2- 172.21.15.116:0/1261858646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f380c102570 0x7f380c0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.428+0000 7f380bfff640 1 -- 172.21.15.116:0/1261858646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f380c102ef0 msgr2=0x7f380c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:45.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.428+0000 7f380bfff640 1 --2- 172.21.15.116:0/1261858646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f380c102ef0 0x7f380c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.428+0000 7f380bfff640 1 -- 172.21.15.116:0/1261858646 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f380c1a6a50 con 0x7f380c100e20 2024-09-12T03:10:45.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.428+0000 7f3810b15640 1 --2- 172.21.15.116:0/1261858646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f380c102ef0 0x7f380c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:10:45.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.428+0000 7f3803fff640 1 --2- 172.21.15.116:0/1261858646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f380c102570 0x7f380c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:45.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.428+0000 7f380bfff640 1 --2- 172.21.15.116:0/1261858646 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f380c100e20 0x7f380c0fcc80 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f37fc00d950 tx=0x7f37fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:45.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.428+0000 7f3809ffb640 1 -- 172.21.15.116:0/1261858646 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc017070 con 0x7f380c100e20 2024-09-12T03:10:45.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.428+0000 7f3809ffb640 1 -- 172.21.15.116:0/1261858646 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37fc009c30 con 0x7f380c100e20 2024-09-12T03:10:45.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.428+0000 7f3812578640 1 -- 172.21.15.116:0/1261858646 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f380c1a6d40 con 0x7f380c100e20 2024-09-12T03:10:45.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.428+0000 7f3809ffb640 1 -- 172.21.15.116:0/1261858646 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc00f6b0 con 0x7f380c100e20 2024-09-12T03:10:45.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.428+0000 7f3812578640 1 -- 172.21.15.116:0/1261858646 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f380c1a7270 con 0x7f380c100e20 2024-09-12T03:10:45.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.430+0000 7f3809ffb640 1 -- 172.21.15.116:0/1261858646 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f37fc0109d0 con 0x7f380c100e20 2024-09-12T03:10:45.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.431+0000 7f3814951740 1 Processor -- start 2024-09-12T03:10:45.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.431+0000 7f3814951740 1 -- start start 2024-09-12T03:10:45.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.431+0000 7f3814951740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd99edcb40 0x55cd99edb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:45.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.431+0000 7f3814951740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd99edb5d0 0x55cd99edba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:45.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.431+0000 7f3814951740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cd99ee6370 0x55cd99ee8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:45.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.431+0000 7f3814951740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55cd99e412f0 con 0x55cd99ee6370 2024-09-12T03:10:45.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.431+0000 7f3814951740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55cd99de8cf0 con 0x55cd99edb5d0 2024-09-12T03:10:45.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.431+0000 7f3814951740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cd99e3d2b0 con 0x55cd99edcb40 2024-09-12T03:10:45.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.431+0000 7f3803fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cd99ee6370 0x55cd99ee8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:45.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.431+0000 7f3803fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cd99ee6370 0x55cd99ee8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55824/0 (socket says 172.21.15.116:55824) 2024-09-12T03:10:45.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.431+0000 7f3803fff640 1 -- 172.21.15.116:0/62183838 learned_addr learned my addr 172.21.15.116:0/62183838 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:45.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.431+0000 7f380bfff640 1 --2- 172.21.15.116:0/62183838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd99edb5d0 0x55cd99edba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:45.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.431+0000 7f3810b15640 1 --2- 172.21.15.116:0/62183838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd99edcb40 0x55cd99edb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:45.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.431+0000 7f3803fff640 1 -- 172.21.15.116:0/62183838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd99edcb40 msgr2=0x55cd99edb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:45.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.431+0000 7f3803fff640 1 --2- 172.21.15.116:0/62183838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd99edcb40 0x55cd99edb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.431+0000 7f3803fff640 1 -- 172.21.15.116:0/62183838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd99edb5d0 msgr2=0x55cd99edba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:45.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.431+0000 7f3803fff640 1 --2- 172.21.15.116:0/62183838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd99edb5d0 0x55cd99edba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.431+0000 7f3803fff640 1 -- 172.21.15.116:0/62183838 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd99dcd3a0 con 0x55cd99ee6370 2024-09-12T03:10:45.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.431+0000 7f3810b15640 1 --2- 172.21.15.116:0/62183838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd99edcb40 0x55cd99edb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:45.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.431+0000 7f3809ffb640 1 --2- 172.21.15.116:0/1261858646 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f37f80a2c00 0x7f37f80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:45.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.432+0000 7f3810b15640 1 --2- 172.21.15.116:0/1261858646 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f37f80a2c00 0x7f37f80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:45.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.432+0000 7f3803fff640 1 --2- 172.21.15.116:0/62183838 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cd99ee6370 0x55cd99ee8760 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f37f400a9d0 tx=0x7f37f400ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97716d42d1e746c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:45.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.432+0000 7f3809ffb640 1 -- 172.21.15.116:0/1261858646 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f37fc0cefc0 con 0x7f380c100e20 2024-09-12T03:10:45.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.432+0000 7f38027fc640 1 -- 172.21.15.116:0/62183838 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f4015070 con 0x55cd99ee6370 2024-09-12T03:10:45.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.432+0000 7f38027fc640 1 -- 172.21.15.116:0/62183838 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37f400cc60 con 0x55cd99ee6370 2024-09-12T03:10:45.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.432+0000 7f38027fc640 1 -- 172.21.15.116:0/62183838 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f40105b0 con 0x55cd99ee6370 2024-09-12T03:10:45.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.433+0000 7f3814951740 1 -- 172.21.15.116:0/62183838 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cd99ee6370 msgr2=0x55cd99ee8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:45.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.433+0000 7f3814951740 1 --2- 172.21.15.116:0/62183838 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cd99ee6370 0x55cd99ee8760 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f37f400a9d0 tx=0x7f37f400ae90 comp rx=0 tx=0).stop 2024-09-12T03:10:45.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.433+0000 7f3810b15640 1 --2- 172.21.15.116:0/1261858646 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f37f80a2c00 0x7f37f80a50c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3804030040 tx=0x7f3804005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:45.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.433+0000 7f3814951740 1 -- 172.21.15.116:0/62183838 shutdown_connections 2024-09-12T03:10:45.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.433+0000 7f3814951740 1 --2- 172.21.15.116:0/62183838 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cd99ee6370 0x55cd99ee8760 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.433+0000 7f3814951740 1 --2- 172.21.15.116:0/62183838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd99edb5d0 0x55cd99edba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.433+0000 7f3814951740 1 --2- 172.21.15.116:0/62183838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd99edcb40 0x55cd99edb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.433+0000 7f3814951740 1 -- 172.21.15.116:0/62183838 >> 172.21.15.116:0/62183838 conn(0x55cd99ea4310 msgr2=0x55cd99ed7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:45.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.433+0000 7f3814951740 1 -- 172.21.15.116:0/62183838 shutdown_connections 2024-09-12T03:10:45.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.433+0000 7f3814951740 1 -- 172.21.15.116:0/62183838 wait complete. 2024-09-12T03:10:45.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.433+0000 7f3814951740 1 -- start start 2024-09-12T03:10:45.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.433+0000 7f3814951740 1 Processor -- start 2024-09-12T03:10:45.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.433+0000 7f3814951740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cd99edb5d0 0x55cd99ee5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:45.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.433+0000 7f3814951740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd99edcb40 0x55cd99ee2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:45.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.433+0000 7f3814951740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd99ee6370 0x55cd99ee3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:45.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.433+0000 7f3814951740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55cd99ee9070 con 0x55cd99edb5d0 2024-09-12T03:10:45.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.433+0000 7f3814951740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55cd99dfc060 con 0x55cd99ee6370 2024-09-12T03:10:45.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.433+0000 7f3814951740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cd99ee91f0 con 0x55cd99edcb40 2024-09-12T03:10:45.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.434+0000 7f3814951740 10 client.0 _ll_register_callbacks cb 0x55cd99b4f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:45.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.434+0000 7f380bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cd99edb5d0 0x55cd99ee5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:45.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.434+0000 7f380bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cd99edb5d0 0x55cd99ee5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55830/0 (socket says 172.21.15.116:55830) 2024-09-12T03:10:45.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.434+0000 7f380bfff640 1 -- 172.21.15.116:0/792076238 learned_addr learned my addr 172.21.15.116:0/792076238 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:45.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.434+0000 7f3810b15640 1 --2- 172.21.15.116:0/792076238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd99ee6370 0x55cd99ee3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:45.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.434+0000 7f3803fff640 1 --2- 172.21.15.116:0/792076238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd99edcb40 0x55cd99ee2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:45.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.434+0000 7f380bfff640 1 -- 172.21.15.116:0/792076238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd99edcb40 msgr2=0x55cd99ee2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:45.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.434+0000 7f380bfff640 1 --2- 172.21.15.116:0/792076238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd99edcb40 0x55cd99ee2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.434+0000 7f380bfff640 1 -- 172.21.15.116:0/792076238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd99ee6370 msgr2=0x55cd99ee3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:45.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.434+0000 7f380bfff640 1 --2- 172.21.15.116:0/792076238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd99ee6370 0x55cd99ee3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.434+0000 7f380bfff640 1 -- 172.21.15.116:0/792076238 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd99bb23c0 con 0x55cd99edb5d0 2024-09-12T03:10:45.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.434+0000 7f3810b15640 1 --2- 172.21.15.116:0/792076238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd99ee6370 0x55cd99ee3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:10:45.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.434+0000 7f380bfff640 1 --2- 172.21.15.116:0/792076238 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cd99edb5d0 0x55cd99ee5ca0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f37fc09b7e0 tx=0x7f37fc09b450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a1bea8d95567ff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:45.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.434+0000 7f37eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:45.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.434+0000 7f37eb7fe640 1 -- 172.21.15.116:0/792076238 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc09b850 con 0x55cd99edb5d0 2024-09-12T03:10:45.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.434+0000 7f37eb7fe640 1 -- 172.21.15.116:0/792076238 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f37fc0860e0 con 0x55cd99edb5d0 2024-09-12T03:10:45.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.434+0000 7f3814951740 1 -- 172.21.15.116:0/792076238 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37f400cc60 con 0x55cd99edb5d0 2024-09-12T03:10:45.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.434+0000 7f37eb7fe640 1 -- 172.21.15.116:0/792076238 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f37fc09fc00 con 0x55cd99edb5d0 2024-09-12T03:10:45.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.434+0000 7f3814951740 10 client.17262 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:45.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.434+0000 7f3814951740 1 -- 172.21.15.116:0/792076238 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cd99ef4a30 con 0x55cd99edb5d0 2024-09-12T03:10:45.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.434+0000 7f37eb7fe640 1 -- 172.21.15.116:0/792076238 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37f400cc60 con 0x55cd99edb5d0 2024-09-12T03:10:45.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.435+0000 7f37eb7fe640 1 -- 172.21.15.116:0/792076238 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f37fc0a4030 con 0x55cd99edb5d0 2024-09-12T03:10:45.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.435+0000 7f3814951740 10 client.17262 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:45.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.435+0000 7f3814951740 10 client.17262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:10:45.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.435+0000 7f3814951740 20 client.17262 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:45.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.435+0000 7f3814951740 1 --2- 172.21.15.116:0/792076238 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55cd99ef5ef0 0x55cd99ef8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:45.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.435+0000 7f3814951740 4 client.17262 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:10:45.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.435+0000 7f3814951740 1 -- 172.21.15.116:0/792076238 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd99e4e860 con 0x55cd99ef5ef0 2024-09-12T03:10:45.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.435+0000 7f3803fff640 1 --2- 172.21.15.116:0/792076238 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55cd99ef5ef0 0x55cd99ef8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:45.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.435+0000 7f37eb7fe640 10 client.17262 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:45.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.435+0000 7f3803fff640 1 --2- 172.21.15.116:0/792076238 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55cd99ef5ef0 0x55cd99ef8320 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6a5935e4649617b server_cookie=76dbbe5b4def9b1d in_seq=0 out_seq=0 2024-09-12T03:10:45.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.436+0000 7f37eb7fe640 1 -- 172.21.15.116:0/792076238 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cd99e4e860 con 0x55cd99ef5ef0 2024-09-12T03:10:45.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.436+0000 7f37eb7fe640 10 client.17262 handle_command_reply: tid=0 2024-09-12T03:10:45.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.436+0000 7f3814951740 1 client.17262 shutdown 2024-09-12T03:10:45.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.436+0000 7f3814951740 1 -- 172.21.15.116:0/792076238 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cd99edb5d0 msgr2=0x55cd99ee5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:45.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.436+0000 7f3814951740 1 --2- 172.21.15.116:0/792076238 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cd99edb5d0 0x55cd99ee5ca0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f37fc09b7e0 tx=0x7f37fc09b450 comp rx=0 tx=0).stop 2024-09-12T03:10:45.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.436+0000 7f3814951740 1 -- 172.21.15.116:0/792076238 shutdown_connections 2024-09-12T03:10:45.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.436+0000 7f3814951740 1 --2- 172.21.15.116:0/792076238 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55cd99ef5ef0 0x55cd99ef8320 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.436+0000 7f3814951740 1 --2- 172.21.15.116:0/792076238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd99ee6370 0x55cd99ee3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.436+0000 7f3814951740 1 --2- 172.21.15.116:0/792076238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd99edcb40 0x55cd99ee2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.436+0000 7f37eb7fe640 0 client.17262 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:45.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.436+0000 7f3814951740 1 --2- 172.21.15.116:0/792076238 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cd99edb5d0 0x55cd99ee5ca0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.436+0000 7f3814951740 1 -- 172.21.15.116:0/792076238 >> 172.21.15.116:0/792076238 conn(0x55cd99ea4310 msgr2=0x55cd99edd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:45.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.436+0000 7f3814951740 1 -- 172.21.15.116:0/792076238 shutdown_connections 2024-09-12T03:10:45.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.436+0000 7f3814951740 1 -- 172.21.15.116:0/792076238 wait complete. 2024-09-12T03:10:45.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.436+0000 7f3814951740 20 client.17262 trim_cache size 0 max 16384 2024-09-12T03:10:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.450+0000 7f3814951740 1 Processor -- start 2024-09-12T03:10:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.450+0000 7f3814951740 1 -- start start 2024-09-12T03:10:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.450+0000 7f3814951740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cd99edb5d0 0x55cd99ed6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.450+0000 7f3814951740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd99edcb40 0x55cd99ede120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.450+0000 7f3814951740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd99ee6370 0x55cd99ede660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.450+0000 7f3814951740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f380c1b2030 con 0x55cd99edb5d0 2024-09-12T03:10:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.450+0000 7f3814951740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f380c1af900 con 0x55cd99edcb40 2024-09-12T03:10:45.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.450+0000 7f3814951740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f380c1ad770 con 0x55cd99ee6370 2024-09-12T03:10:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.450+0000 7f380bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cd99edb5d0 0x55cd99ed6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.450+0000 7f380bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cd99edb5d0 0x55cd99ed6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55832/0 (socket says 172.21.15.116:55832) 2024-09-12T03:10:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.450+0000 7f3810b15640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd99ee6370 0x55cd99ede660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.450+0000 7f380bfff640 1 -- 172.21.15.116:0/3233700597 learned_addr learned my addr 172.21.15.116:0/3233700597 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.450+0000 7f3803fff640 1 --2- 172.21.15.116:0/3233700597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd99edcb40 0x55cd99ede120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.451+0000 7f380bfff640 1 -- 172.21.15.116:0/3233700597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd99ee6370 msgr2=0x55cd99ede660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.451+0000 7f380bfff640 1 --2- 172.21.15.116:0/3233700597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd99ee6370 0x55cd99ede660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.451+0000 7f380bfff640 1 -- 172.21.15.116:0/3233700597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd99edcb40 msgr2=0x55cd99ede120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.451+0000 7f380bfff640 1 --2- 172.21.15.116:0/3233700597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd99edcb40 0x55cd99ede120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.451+0000 7f380bfff640 1 -- 172.21.15.116:0/3233700597 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd99edeba0 con 0x55cd99edb5d0 2024-09-12T03:10:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.451+0000 7f380bfff640 1 --2- 172.21.15.116:0/3233700597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cd99edb5d0 0x55cd99ed6bf0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f37fc009610 tx=0x7f37fc099d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b639a73a2d4c1c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.451+0000 7f3801ffb640 1 -- 172.21.15.116:0/3233700597 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc0a44c0 con 0x55cd99edb5d0 2024-09-12T03:10:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.451+0000 7f3801ffb640 1 -- 172.21.15.116:0/3233700597 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37fc0d10d0 con 0x55cd99edb5d0 2024-09-12T03:10:45.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.451+0000 7f3801ffb640 1 -- 172.21.15.116:0/3233700597 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc09cb80 con 0x55cd99edb5d0 2024-09-12T03:10:45.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.452+0000 7f3814951740 1 -- 172.21.15.116:0/3233700597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cd99edb5d0 msgr2=0x55cd99ed6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:45.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.452+0000 7f3814951740 1 --2- 172.21.15.116:0/3233700597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cd99edb5d0 0x55cd99ed6bf0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f37fc009610 tx=0x7f37fc099d60 comp rx=0 tx=0).stop 2024-09-12T03:10:45.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.452+0000 7f3814951740 1 -- 172.21.15.116:0/3233700597 shutdown_connections 2024-09-12T03:10:45.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.452+0000 7f3814951740 1 --2- 172.21.15.116:0/3233700597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd99ee6370 0x55cd99ede660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.452+0000 7f3814951740 1 --2- 172.21.15.116:0/3233700597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd99edcb40 0x55cd99ede120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.452+0000 7f3814951740 1 --2- 172.21.15.116:0/3233700597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cd99edb5d0 0x55cd99ed6bf0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.452+0000 7f3814951740 1 -- 172.21.15.116:0/3233700597 >> 172.21.15.116:0/3233700597 conn(0x55cd99ea4310 msgr2=0x55cd99ed64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:45.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.452+0000 7f3814951740 1 -- 172.21.15.116:0/3233700597 shutdown_connections 2024-09-12T03:10:45.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.452+0000 7f3814951740 1 -- 172.21.15.116:0/3233700597 wait complete. 2024-09-12T03:10:45.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.452+0000 7f3814951740 1 -- start start 2024-09-12T03:10:45.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.452+0000 7f3814951740 1 Processor -- start 2024-09-12T03:10:45.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.453+0000 7f3814951740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd99edb5d0 0x55cd99eeed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:45.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.453+0000 7f3814951740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd99edcb40 0x55cd99edfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:45.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.453+0000 7f3814951740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cd99ee6370 0x55cd99ee0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:45.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.453+0000 7f3814951740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f380c1addc0 con 0x55cd99ee6370 2024-09-12T03:10:45.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.453+0000 7f3814951740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f380c1af230 con 0x55cd99edb5d0 2024-09-12T03:10:45.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.453+0000 7f3814951740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f380c1ae490 con 0x55cd99edcb40 2024-09-12T03:10:45.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.453+0000 7f3814951740 10 client.0 _ll_register_callbacks cb 0x7f37fc097af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:45.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.453+0000 7f3803fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd99edcb40 0x55cd99edfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:45.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.453+0000 7f3810b15640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cd99ee6370 0x55cd99ee0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:45.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.453+0000 7f3803fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd99edcb40 0x55cd99edfd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.116:59392/0 (socket says 172.21.15.116:59392) 2024-09-12T03:10:45.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.453+0000 7f3803fff640 1 -- 172.21.15.116:0/3629813846 learned_addr learned my addr 172.21.15.116:0/3629813846 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:45.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.453+0000 7f380bfff640 1 --2- 172.21.15.116:0/3629813846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd99edb5d0 0x55cd99eeed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:45.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.453+0000 7f3803fff640 1 -- 172.21.15.116:0/3629813846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd99edb5d0 msgr2=0x55cd99eeed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:45.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.453+0000 7f3803fff640 1 --2- 172.21.15.116:0/3629813846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd99edb5d0 0x55cd99eeed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.453+0000 7f3803fff640 1 -- 172.21.15.116:0/3629813846 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cd99ee6370 msgr2=0x55cd99ee0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:45.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.453+0000 7f3803fff640 1 --2- 172.21.15.116:0/3629813846 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cd99ee6370 0x55cd99ee0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.453+0000 7f3803fff640 1 -- 172.21.15.116:0/3629813846 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd99ee0bb0 con 0x55cd99edcb40 2024-09-12T03:10:45.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.453+0000 7f3810b15640 1 --2- 172.21.15.116:0/3629813846 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cd99ee6370 0x55cd99ee0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:10:45.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.454+0000 7f3803fff640 1 --2- 172.21.15.116:0/3629813846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd99edcb40 0x55cd99edfd90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f37f400aed0 tx=0x7f37f400a4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9a66423cf5a2a07 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:45.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.454+0000 7f37eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T03:10:45.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.454+0000 7f37eb7fe640 1 -- 172.21.15.116:0/3629813846 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f4015040 con 0x55cd99edcb40 2024-09-12T03:10:45.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.454+0000 7f3814951740 1 -- 172.21.15.116:0/3629813846 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37fc0d10d0 con 0x55cd99edcb40 2024-09-12T03:10:45.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.454+0000 7f37eb7fe640 1 -- 172.21.15.116:0/3629813846 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f37f40113f0 con 0x55cd99edcb40 2024-09-12T03:10:45.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.454+0000 7f37eb7fe640 1 -- 172.21.15.116:0/3629813846 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f37f4034070 con 0x55cd99edcb40 2024-09-12T03:10:45.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.454+0000 7f3814951740 10 client.26161 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:45.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.454+0000 7f3814951740 1 -- 172.21.15.116:0/3629813846 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cd99f60c10 con 0x55cd99edcb40 2024-09-12T03:10:45.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.454+0000 7f37eb7fe640 1 -- 172.21.15.116:0/3629813846 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37fc0d10d0 con 0x55cd99edcb40 2024-09-12T03:10:45.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.455+0000 7f37eb7fe640 1 -- 172.21.15.116:0/3629813846 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f37f4023a70 con 0x55cd99edcb40 2024-09-12T03:10:45.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.455+0000 7f3814951740 10 client.26161 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:45.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.455+0000 7f3814951740 10 client.26161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:10:45.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.455+0000 7f3814951740 20 client.26161 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:45.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.455+0000 7f3814951740 1 --2- 172.21.15.116:0/3629813846 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55cd99ef5ef0 0x55cd99f630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:45.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.455+0000 7f3814951740 4 client.26161 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:10:45.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.455+0000 7f3814951740 1 -- 172.21.15.116:0/3629813846 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cd99e48d70 con 0x55cd99ef5ef0 2024-09-12T03:10:45.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.455+0000 7f380bfff640 1 --2- 172.21.15.116:0/3629813846 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55cd99ef5ef0 0x55cd99f630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:45.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.455+0000 7f37eb7fe640 10 client.26161 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:45.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.455+0000 7f380bfff640 1 --2- 172.21.15.116:0/3629813846 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55cd99ef5ef0 0x55cd99f630c0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af17455a0539dace server_cookie=132ce31f83331aed in_seq=0 out_seq=0 2024-09-12T03:10:45.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.456+0000 7f37eb7fe640 1 -- 172.21.15.116:0/3629813846 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cd99e48d70 con 0x55cd99ef5ef0 2024-09-12T03:10:45.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.456+0000 7f37eb7fe640 10 client.26161 handle_command_reply: tid=0 2024-09-12T03:10:45.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.456+0000 7f3814951740 1 client.26161 shutdown 2024-09-12T03:10:45.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.456+0000 7f3814951740 1 -- 172.21.15.116:0/3629813846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd99edcb40 msgr2=0x55cd99edfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:45.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.456+0000 7f3814951740 1 --2- 172.21.15.116:0/3629813846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd99edcb40 0x55cd99edfd90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f37f400aed0 tx=0x7f37f400a4e0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.456+0000 7f3814951740 1 -- 172.21.15.116:0/3629813846 shutdown_connections 2024-09-12T03:10:45.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.456+0000 7f3814951740 1 --2- 172.21.15.116:0/3629813846 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55cd99ef5ef0 0x55cd99f630c0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.456+0000 7f3814951740 1 --2- 172.21.15.116:0/3629813846 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cd99ee6370 0x55cd99ee0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.456+0000 7f3814951740 1 --2- 172.21.15.116:0/3629813846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd99edcb40 0x55cd99edfd90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.456+0000 7f37eb7fe640 0 client.26161 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:45.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.456+0000 7f3814951740 1 --2- 172.21.15.116:0/3629813846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cd99edb5d0 0x55cd99eeed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.456+0000 7f3814951740 1 -- 172.21.15.116:0/3629813846 >> 172.21.15.116:0/3629813846 conn(0x55cd99ea4310 msgr2=0x55cd99ed6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:45.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.456+0000 7f3814951740 1 -- 172.21.15.116:0/3629813846 shutdown_connections 2024-09-12T03:10:45.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.456+0000 7f3814951740 1 -- 172.21.15.116:0/3629813846 wait complete. 2024-09-12T03:10:45.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.456+0000 7f3814951740 20 client.26161 trim_cache size 0 max 16384 2024-09-12T03:10:45.466 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:10:45.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.457+0000 7f3812578640 1 -- 172.21.15.116:0/1261858646 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f37f80a2c00 msgr2=0x7f37f80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:45.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.457+0000 7f3812578640 1 --2- 172.21.15.116:0/1261858646 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f37f80a2c00 0x7f37f80a50c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3804030040 tx=0x7f3804005d80 comp rx=0 tx=0).stop 2024-09-12T03:10:45.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.457+0000 7f3812578640 1 -- 172.21.15.116:0/1261858646 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f380c100e20 msgr2=0x7f380c0fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:45.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.457+0000 7f3812578640 1 --2- 172.21.15.116:0/1261858646 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f380c100e20 0x7f380c0fcc80 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f37fc00d950 tx=0x7f37fc00de10 comp rx=0 tx=0).stop 2024-09-12T03:10:45.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.457+0000 7f3812578640 1 -- 172.21.15.116:0/1261858646 shutdown_connections 2024-09-12T03:10:45.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.457+0000 7f3812578640 1 --2- 172.21.15.116:0/1261858646 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f37f80a2c00 0x7f37f80a50c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.457+0000 7f3812578640 1 --2- 172.21.15.116:0/1261858646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f380c102ef0 0x7f380c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.457+0000 7f3812578640 1 --2- 172.21.15.116:0/1261858646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f380c102570 0x7f380c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.457+0000 7f3812578640 1 --2- 172.21.15.116:0/1261858646 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f380c100e20 0x7f380c0fcc80 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:45.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.457+0000 7f3812578640 1 -- 172.21.15.116:0/1261858646 >> 172.21.15.116:0/1261858646 conn(0x7f380c0fc7d0 msgr2=0x7f380c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:45.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.457+0000 7f3812578640 1 -- 172.21.15.116:0/1261858646 shutdown_connections 2024-09-12T03:10:45.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:45.457+0000 7f3812578640 1 -- 172.21.15.116:0/1261858646 wait complete. 2024-09-12T03:10:45.813 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:10:46.584 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:46 smithi116 ceph-mon[31410]: pgmap v571: 97 pgs: 97 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:10:46.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:46 smithi183 ceph-mon[35535]: pgmap v571: 97 pgs: 97 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:10:46.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:46 smithi177 ceph-mon[35305]: pgmap v571: 97 pgs: 97 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:10:46.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.816+0000 7f1899c2e640 1 -- 172.21.15.116:0/1755041729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1894100a60 msgr2=0x7f1894100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:46.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.816+0000 7f1899c2e640 1 --2- 172.21.15.116:0/1755041729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1894100a60 0x7f1894100ec0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f187c009930 tx=0x7f187c02a420 comp rx=0 tx=0).stop 2024-09-12T03:10:46.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.816+0000 7f1899c2e640 1 -- 172.21.15.116:0/1755041729 shutdown_connections 2024-09-12T03:10:46.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.816+0000 7f1899c2e640 1 --2- 172.21.15.116:0/1755041729 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1894101400 0x7f189410d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:46.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.816+0000 7f1899c2e640 1 --2- 172.21.15.116:0/1755041729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1894100a60 0x7f1894100ec0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:46.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.816+0000 7f1899c2e640 1 --2- 172.21.15.116:0/1755041729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1894106a60 0x7f1894106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:46.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.816+0000 7f1899c2e640 1 -- 172.21.15.116:0/1755041729 >> 172.21.15.116:0/1755041729 conn(0x7f18940fc780 msgr2=0x7f18940feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:46.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.816+0000 7f1899c2e640 1 -- 172.21.15.116:0/1755041729 shutdown_connections 2024-09-12T03:10:46.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.816+0000 7f1899c2e640 1 -- 172.21.15.116:0/1755041729 wait complete. 2024-09-12T03:10:46.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.816+0000 7f1899c2e640 1 Processor -- start 2024-09-12T03:10:46.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.816+0000 7f1899c2e640 1 -- start start 2024-09-12T03:10:46.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.817+0000 7f1899c2e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1894100a60 0x7f18941970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:46.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.817+0000 7f1899c2e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1894101400 0x7f1894197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:46.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.817+0000 7f1899c2e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1894106a60 0x7f1894197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:46.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.817+0000 7f1899c2e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1894110e20 con 0x7f1894100a60 2024-09-12T03:10:46.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.817+0000 7f1899c2e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1894110ca0 con 0x7f1894101400 2024-09-12T03:10:46.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.817+0000 7f1899c2e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1894110fa0 con 0x7f1894106a60 2024-09-12T03:10:46.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.817+0000 7f18937fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1894100a60 0x7f18941970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:46.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.817+0000 7f18937fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1894100a60 0x7f18941970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55848/0 (socket says 172.21.15.116:55848) 2024-09-12T03:10:46.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.817+0000 7f18937fe640 1 -- 172.21.15.116:0/3806829999 learned_addr learned my addr 172.21.15.116:0/3806829999 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:46.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.817+0000 7f188affd640 1 --2- 172.21.15.116:0/3806829999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1894101400 0x7f1894197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:46.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.817+0000 7f1893fff640 1 --2- 172.21.15.116:0/3806829999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1894106a60 0x7f1894197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:46.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.817+0000 7f18937fe640 1 -- 172.21.15.116:0/3806829999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1894106a60 msgr2=0x7f1894197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:46.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.817+0000 7f18937fe640 1 --2- 172.21.15.116:0/3806829999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1894106a60 0x7f1894197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:46.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.817+0000 7f18937fe640 1 -- 172.21.15.116:0/3806829999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1894101400 msgr2=0x7f1894197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:46.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.817+0000 7f18937fe640 1 --2- 172.21.15.116:0/3806829999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1894101400 0x7f1894197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:46.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.817+0000 7f18937fe640 1 -- 172.21.15.116:0/3806829999 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18941a6950 con 0x7f1894100a60 2024-09-12T03:10:46.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.817+0000 7f18937fe640 1 --2- 172.21.15.116:0/3806829999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1894100a60 0x7f18941970e0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f188400c890 tx=0x7f188400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:46.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.818+0000 7f18917fa640 1 -- 172.21.15.116:0/3806829999 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884004220 con 0x7f1894100a60 2024-09-12T03:10:46.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.818+0000 7f1899c2e640 1 -- 172.21.15.116:0/3806829999 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18941a6c40 con 0x7f1894100a60 2024-09-12T03:10:46.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.818+0000 7f18917fa640 1 -- 172.21.15.116:0/3806829999 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18840043c0 con 0x7f1894100a60 2024-09-12T03:10:46.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.818+0000 7f18917fa640 1 -- 172.21.15.116:0/3806829999 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884004a00 con 0x7f1894100a60 2024-09-12T03:10:46.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.818+0000 7f1899c2e640 1 -- 172.21.15.116:0/3806829999 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18941a71a0 con 0x7f1894100a60 2024-09-12T03:10:46.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.819+0000 7f1899c2e640 1 -- 172.21.15.116:0/3806829999 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1894100ec0 con 0x7f1894100a60 2024-09-12T03:10:46.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.820+0000 7f18917fa640 1 -- 172.21.15.116:0/3806829999 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f18840079b0 con 0x7f1894100a60 2024-09-12T03:10:46.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.821+0000 7f18917fa640 1 --2- 172.21.15.116:0/3806829999 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f18800a2b80 0x7f18800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:46.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.821+0000 7f188affd640 1 --2- 172.21.15.116:0/3806829999 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f18800a2b80 0x7f18800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:46.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.821+0000 7f18917fa640 1 -- 172.21.15.116:0/3806829999 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f18840cef90 con 0x7f1894100a60 2024-09-12T03:10:46.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.821+0000 7f188affd640 1 --2- 172.21.15.116:0/3806829999 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f18800a2b80 0x7f18800a5040 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f187c009900 tx=0x7f187c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:46.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:46.824+0000 7f18917fa640 1 -- 172.21.15.116:0/3806829999 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1884014030 con 0x7f1894100a60 2024-09-12T03:10:47.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:47.023+0000 7f1899c2e640 1 -- 172.21.15.116:0/3806829999 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f189402d7c0 con 0x7f1894100a60 2024-09-12T03:10:47.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:47.024+0000 7f18917fa640 1 -- 172.21.15.116:0/3806829999 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f188409b8d0 con 0x7f1894100a60 2024-09-12T03:10:47.026 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:10:47.026 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:10:47.027 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:10:47.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:47.026+0000 7f1899c2e640 1 -- 172.21.15.116:0/3806829999 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f18800a2b80 msgr2=0x7f18800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:47.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:47.026+0000 7f1899c2e640 1 --2- 172.21.15.116:0/3806829999 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f18800a2b80 0x7f18800a5040 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f187c009900 tx=0x7f187c0058d0 comp rx=0 tx=0).stop 2024-09-12T03:10:47.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:47.026+0000 7f1899c2e640 1 -- 172.21.15.116:0/3806829999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1894100a60 msgr2=0x7f18941970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:47.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:47.026+0000 7f1899c2e640 1 --2- 172.21.15.116:0/3806829999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1894100a60 0x7f18941970e0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f188400c890 tx=0x7f188400cd50 comp rx=0 tx=0).stop 2024-09-12T03:10:47.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:47.026+0000 7f1899c2e640 1 -- 172.21.15.116:0/3806829999 shutdown_connections 2024-09-12T03:10:47.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:47.026+0000 7f1899c2e640 1 --2- 172.21.15.116:0/3806829999 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f18800a2b80 0x7f18800a5040 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:47.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:47.026+0000 7f1899c2e640 1 --2- 172.21.15.116:0/3806829999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1894106a60 0x7f1894197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:47.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:47.026+0000 7f1899c2e640 1 --2- 172.21.15.116:0/3806829999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1894101400 0x7f1894197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:47.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:47.026+0000 7f1899c2e640 1 --2- 172.21.15.116:0/3806829999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1894100a60 0x7f18941970e0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:47.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:47.026+0000 7f1899c2e640 1 -- 172.21.15.116:0/3806829999 >> 172.21.15.116:0/3806829999 conn(0x7f18940fc780 msgr2=0x7f18940fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:47.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:47.026+0000 7f1899c2e640 1 -- 172.21.15.116:0/3806829999 shutdown_connections 2024-09-12T03:10:47.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:47.026+0000 7f1899c2e640 1 -- 172.21.15.116:0/3806829999 wait complete. 2024-09-12T03:10:47.393 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:10:47.393 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:10:47.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:47 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3806829999' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:10:47.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:47 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3806829999' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:10:47.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:47 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3806829999' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:10:47.837 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0' 2024-09-12T03:10:47.837 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0 2024-09-12T03:10:47.972 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:10:48.373 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:48 smithi116 ceph-mon[31410]: pgmap v572: 97 pgs: 97 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:10:48.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.456+0000 7fb05f6e7640 1 -- 172.21.15.116:0/453280012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0581027a0 msgr2=0x7fb0581067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:48.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.456+0000 7fb05f6e7640 1 --2- 172.21.15.116:0/453280012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0581027a0 0x7fb0581067f0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fb054009900 tx=0x7fb05402a410 comp rx=0 tx=0).stop 2024-09-12T03:10:48.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.456+0000 7fb05f6e7640 1 -- 172.21.15.116:0/453280012 shutdown_connections 2024-09-12T03:10:48.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.456+0000 7fb05f6e7640 1 --2- 172.21.15.116:0/453280012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb058106f20 0x7fb05810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.456+0000 7fb05f6e7640 1 --2- 172.21.15.116:0/453280012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0581027a0 0x7fb0581067f0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.456+0000 7fb05f6e7640 1 --2- 172.21.15.116:0/453280012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb058101df0 0x7fb0581021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.456+0000 7fb05f6e7640 1 -- 172.21.15.116:0/453280012 >> 172.21.15.116:0/453280012 conn(0x7fb0580fd8f0 msgr2=0x7fb0580ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:48.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.456+0000 7fb05f6e7640 1 -- 172.21.15.116:0/453280012 shutdown_connections 2024-09-12T03:10:48.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.456+0000 7fb05f6e7640 1 -- 172.21.15.116:0/453280012 wait complete. 2024-09-12T03:10:48.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.457+0000 7fb05f6e7640 1 Processor -- start 2024-09-12T03:10:48.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.457+0000 7fb05f6e7640 1 -- start start 2024-09-12T03:10:48.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.457+0000 7fb05f6e7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb058101df0 0x7fb058198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:48.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.457+0000 7fb05f6e7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0581027a0 0x7fb058198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:48.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.457+0000 7fb05f6e7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb058106f20 0x7fb05819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:48.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.457+0000 7fb05f6e7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb058111f80 con 0x7fb058101df0 2024-09-12T03:10:48.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.457+0000 7fb05f6e7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb058111e00 con 0x7fb058106f20 2024-09-12T03:10:48.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.457+0000 7fb05f6e7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb058112100 con 0x7fb0581027a0 2024-09-12T03:10:48.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.457+0000 7fb05d483640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb058101df0 0x7fb058198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:48.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.457+0000 7fb05d483640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb058101df0 0x7fb058198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55858/0 (socket says 172.21.15.116:55858) 2024-09-12T03:10:48.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.457+0000 7fb05d483640 1 -- 172.21.15.116:0/944296973 learned_addr learned my addr 172.21.15.116:0/944296973 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:48.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.457+0000 7fb05cc82640 1 --2- 172.21.15.116:0/944296973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0581027a0 0x7fb058198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:48.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.457+0000 7fb05dc84640 1 --2- 172.21.15.116:0/944296973 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb058106f20 0x7fb05819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:48.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.457+0000 7fb05d483640 1 -- 172.21.15.116:0/944296973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0581027a0 msgr2=0x7fb058198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:48.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.457+0000 7fb05d483640 1 --2- 172.21.15.116:0/944296973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0581027a0 0x7fb058198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.457+0000 7fb05d483640 1 -- 172.21.15.116:0/944296973 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb058106f20 msgr2=0x7fb05819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:48.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.457+0000 7fb05d483640 1 --2- 172.21.15.116:0/944296973 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb058106f20 0x7fb05819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.457+0000 7fb05d483640 1 -- 172.21.15.116:0/944296973 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05819d3d0 con 0x7fb058101df0 2024-09-12T03:10:48.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.458+0000 7fb05d483640 1 --2- 172.21.15.116:0/944296973 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb058101df0 0x7fb058198420 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00e9f0 tx=0x7fb04c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:48.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.458+0000 7fb04a7fc640 1 -- 172.21.15.116:0/944296973 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c00cd80 con 0x7fb058101df0 2024-09-12T03:10:48.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.458+0000 7fb04a7fc640 1 -- 172.21.15.116:0/944296973 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb04c010910 con 0x7fb058101df0 2024-09-12T03:10:48.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.458+0000 7fb05f6e7640 1 -- 172.21.15.116:0/944296973 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb05819d6c0 con 0x7fb058101df0 2024-09-12T03:10:48.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.458+0000 7fb04a7fc640 1 -- 172.21.15.116:0/944296973 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c018990 con 0x7fb058101df0 2024-09-12T03:10:48.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.458+0000 7fb05f6e7640 1 -- 172.21.15.116:0/944296973 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0581a4fa0 con 0x7fb058101df0 2024-09-12T03:10:48.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.460+0000 7fb04a7fc640 1 -- 172.21.15.116:0/944296973 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb04c010ac0 con 0x7fb058101df0 2024-09-12T03:10:48.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.461+0000 7fb061ac0740 1 Processor -- start 2024-09-12T03:10:48.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.461+0000 7fb061ac0740 1 -- start start 2024-09-12T03:10:48.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.461+0000 7fb04a7fc640 1 --2- 172.21.15.116:0/944296973 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb0440a2b80 0x7fb0440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:48.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.461+0000 7fb061ac0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55facb96cb60 0x55facb96cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:48.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.461+0000 7fb061ac0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55facb96d480 0x55facb977800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:48.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.461+0000 7fb04a7fc640 1 -- 172.21.15.116:0/944296973 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fb04c0cf410 con 0x7fb058101df0 2024-09-12T03:10:48.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.461+0000 7fb061ac0740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55facb977d40 0x55facb97a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:48.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.461+0000 7fb061ac0740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55facb8d2280 con 0x55facb977d40 2024-09-12T03:10:48.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.461+0000 7fb061ac0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55facb879b10 con 0x55facb96cb60 2024-09-12T03:10:48.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.461+0000 7fb061ac0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55facb8ce240 con 0x55facb96d480 2024-09-12T03:10:48.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.461+0000 7fb05dc84640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55facb977d40 0x55facb97a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:48.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.461+0000 7fb05cc82640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55facb96d480 0x55facb977800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:48.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.461+0000 7fb05dc84640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55facb977d40 0x55facb97a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55860/0 (socket says 172.21.15.116:55860) 2024-09-12T03:10:48.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.461+0000 7fb05dc84640 1 -- 172.21.15.116:0/1534247909 learned_addr learned my addr 172.21.15.116:0/1534247909 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:48.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.461+0000 7fb05d483640 1 --2- 172.21.15.116:0/1534247909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55facb96cb60 0x55facb96cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:48.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.461+0000 7fb05dc84640 1 -- 172.21.15.116:0/1534247909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55facb96d480 msgr2=0x55facb977800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:48.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.461+0000 7fb05dc84640 1 --2- 172.21.15.116:0/1534247909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55facb96d480 0x55facb977800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.461+0000 7fb05dc84640 1 -- 172.21.15.116:0/1534247909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55facb96cb60 msgr2=0x55facb96cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:48.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.461+0000 7fb05dc84640 1 --2- 172.21.15.116:0/1534247909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55facb96cb60 0x55facb96cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.461+0000 7fb05dc84640 1 -- 172.21.15.116:0/1534247909 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55facb8861f0 con 0x55facb977d40 2024-09-12T03:10:48.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.461+0000 7fb05d483640 1 --2- 172.21.15.116:0/1534247909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55facb96cb60 0x55facb96cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:10:48.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.462+0000 7fb05dc84640 1 --2- 172.21.15.116:0/944296973 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb0440a2b80 0x7fb0440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:48.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.462+0000 7fb05dc84640 1 --2- 172.21.15.116:0/1534247909 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55facb977d40 0x55facb97a130 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fb050012450 tx=0x7fb050012910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b544b01e8c315b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:48.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.462+0000 7fb0437fe640 1 -- 172.21.15.116:0/1534247909 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05000e5b0 con 0x55facb977d40 2024-09-12T03:10:48.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.462+0000 7fb0437fe640 1 -- 172.21.15.116:0/1534247909 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb05001cd50 con 0x55facb977d40 2024-09-12T03:10:48.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.462+0000 7fb0437fe640 1 -- 172.21.15.116:0/1534247909 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05001b380 con 0x55facb977d40 2024-09-12T03:10:48.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.462+0000 7fb05dc84640 1 --2- 172.21.15.116:0/944296973 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb0440a2b80 0x7fb0440a5040 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb050023000 tx=0x7fb050020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:48.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.463+0000 7fb061ac0740 1 -- 172.21.15.116:0/1534247909 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55facb977d40 msgr2=0x55facb97a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:48.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.463+0000 7fb061ac0740 1 --2- 172.21.15.116:0/1534247909 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55facb977d40 0x55facb97a130 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fb050012450 tx=0x7fb050012910 comp rx=0 tx=0).stop 2024-09-12T03:10:48.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.463+0000 7fb061ac0740 1 -- 172.21.15.116:0/1534247909 shutdown_connections 2024-09-12T03:10:48.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.463+0000 7fb061ac0740 1 --2- 172.21.15.116:0/1534247909 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55facb977d40 0x55facb97a130 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.463+0000 7fb061ac0740 1 --2- 172.21.15.116:0/1534247909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55facb96d480 0x55facb977800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.463+0000 7fb061ac0740 1 --2- 172.21.15.116:0/1534247909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55facb96cb60 0x55facb96cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.463+0000 7fb061ac0740 1 -- 172.21.15.116:0/1534247909 >> 172.21.15.116:0/1534247909 conn(0x55facb9690f0 msgr2=0x55facb9694d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:48.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.463+0000 7fb061ac0740 1 -- 172.21.15.116:0/1534247909 shutdown_connections 2024-09-12T03:10:48.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.463+0000 7fb061ac0740 1 -- 172.21.15.116:0/1534247909 wait complete. 2024-09-12T03:10:48.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.463+0000 7fb061ac0740 1 -- start start 2024-09-12T03:10:48.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.463+0000 7fb061ac0740 1 Processor -- start 2024-09-12T03:10:48.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.464+0000 7fb061ac0740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55facb96cb60 0x55facb974bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:48.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.464+0000 7fb061ac0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55facb96d480 0x55facb975130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:48.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.464+0000 7fb061ac0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55facb977d40 0x55facb975670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:48.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.464+0000 7fb061ac0740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55facb97a910 con 0x55facb96cb60 2024-09-12T03:10:48.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.464+0000 7fb061ac0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55facb88cf10 con 0x55facb977d40 2024-09-12T03:10:48.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.464+0000 7fb061ac0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55facb97aa90 con 0x55facb96d480 2024-09-12T03:10:48.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.464+0000 7fb061ac0740 10 client.0 _ll_register_callbacks cb 0x55facb855a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:48.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.464+0000 7fb05d483640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55facb96cb60 0x55facb974bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:48.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.464+0000 7fb05d483640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55facb96cb60 0x55facb974bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55870/0 (socket says 172.21.15.116:55870) 2024-09-12T03:10:48.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.464+0000 7fb05d483640 1 -- 172.21.15.116:0/586093786 learned_addr learned my addr 172.21.15.116:0/586093786 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:48.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.464+0000 7fb05cc82640 1 --2- 172.21.15.116:0/586093786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55facb96d480 0x55facb975130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:48.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.464+0000 7fb05dc84640 1 --2- 172.21.15.116:0/586093786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55facb977d40 0x55facb975670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:48.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.464+0000 7fb05d483640 1 -- 172.21.15.116:0/586093786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55facb96d480 msgr2=0x55facb975130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:48.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.464+0000 7fb05d483640 1 --2- 172.21.15.116:0/586093786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55facb96d480 0x55facb975130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.464+0000 7fb05d483640 1 -- 172.21.15.116:0/586093786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55facb977d40 msgr2=0x55facb975670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:48.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.464+0000 7fb05d483640 1 --2- 172.21.15.116:0/586093786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55facb977d40 0x55facb975670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.464+0000 7fb05d483640 1 -- 172.21.15.116:0/586093786 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55facb6436a0 con 0x55facb96cb60 2024-09-12T03:10:48.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.464+0000 7fb05d483640 1 --2- 172.21.15.116:0/586093786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55facb96cb60 0x55facb974bf0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fb04c09bbd0 tx=0x7fb04c09c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f96006cb393feb42 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:48.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.464+0000 7fb040ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:48.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.464+0000 7fb040ff9640 1 -- 172.21.15.116:0/586093786 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c0a48b0 con 0x55facb96cb60 2024-09-12T03:10:48.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.464+0000 7fb040ff9640 1 -- 172.21.15.116:0/586093786 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fb04c050250 con 0x55facb96cb60 2024-09-12T03:10:48.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.464+0000 7fb061ac0740 1 -- 172.21.15.116:0/586093786 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb05001cd50 con 0x55facb96cb60 2024-09-12T03:10:48.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.465+0000 7fb040ff9640 1 -- 172.21.15.116:0/586093786 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb04c0a4420 con 0x55facb96cb60 2024-09-12T03:10:48.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.465+0000 7fb061ac0740 10 client.17286 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:48.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.465+0000 7fb061ac0740 1 -- 172.21.15.116:0/586093786 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55facb9864e0 con 0x55facb96cb60 2024-09-12T03:10:48.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.465+0000 7fb040ff9640 1 -- 172.21.15.116:0/586093786 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb05001cd50 con 0x55facb96cb60 2024-09-12T03:10:48.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.465+0000 7fb040ff9640 1 -- 172.21.15.116:0/586093786 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb04c09b090 con 0x55facb96cb60 2024-09-12T03:10:48.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.465+0000 7fb061ac0740 10 client.17286 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:48.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.465+0000 7fb061ac0740 10 client.17286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:10:48.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.465+0000 7fb061ac0740 20 client.17286 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:48.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.465+0000 7fb061ac0740 1 --2- 172.21.15.116:0/586093786 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55facb987940 0x55facb989d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:48.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.465+0000 7fb061ac0740 4 client.17286 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:10:48.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.465+0000 7fb061ac0740 1 -- 172.21.15.116:0/586093786 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55facb8df7f0 con 0x55facb987940 2024-09-12T03:10:48.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.465+0000 7fb05cc82640 1 --2- 172.21.15.116:0/586093786 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55facb987940 0x55facb989d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:48.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.466+0000 7fb05cc82640 1 --2- 172.21.15.116:0/586093786 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55facb987940 0x55facb989d70 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=532cf6128cebf4c3 server_cookie=54033486eccfa0cc in_seq=0 out_seq=0 2024-09-12T03:10:48.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.466+0000 7fb040ff9640 10 client.17286 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:48.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.466+0000 7fb040ff9640 1 -- 172.21.15.116:0/586093786 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55facb8df7f0 con 0x55facb987940 2024-09-12T03:10:48.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.466+0000 7fb040ff9640 10 client.17286 handle_command_reply: tid=0 2024-09-12T03:10:48.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.466+0000 7fb061ac0740 1 client.17286 shutdown 2024-09-12T03:10:48.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.466+0000 7fb061ac0740 1 -- 172.21.15.116:0/586093786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55facb96cb60 msgr2=0x55facb974bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:48.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.466+0000 7fb061ac0740 1 --2- 172.21.15.116:0/586093786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55facb96cb60 0x55facb974bf0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fb04c09bbd0 tx=0x7fb04c09c480 comp rx=0 tx=0).stop 2024-09-12T03:10:48.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.466+0000 7fb061ac0740 1 -- 172.21.15.116:0/586093786 shutdown_connections 2024-09-12T03:10:48.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.466+0000 7fb061ac0740 1 --2- 172.21.15.116:0/586093786 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55facb987940 0x55facb989d70 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.466+0000 7fb061ac0740 1 --2- 172.21.15.116:0/586093786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55facb977d40 0x55facb975670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.466+0000 7fb040ff9640 0 client.17286 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:48.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.466+0000 7fb061ac0740 1 --2- 172.21.15.116:0/586093786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55facb96d480 0x55facb975130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.466+0000 7fb061ac0740 1 --2- 172.21.15.116:0/586093786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55facb96cb60 0x55facb974bf0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.466+0000 7fb061ac0740 1 -- 172.21.15.116:0/586093786 >> 172.21.15.116:0/586093786 conn(0x55facb9690f0 msgr2=0x55facb968270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:48.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.467+0000 7fb061ac0740 1 -- 172.21.15.116:0/586093786 shutdown_connections 2024-09-12T03:10:48.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.467+0000 7fb061ac0740 1 -- 172.21.15.116:0/586093786 wait complete. 2024-09-12T03:10:48.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.467+0000 7fb061ac0740 20 client.17286 trim_cache size 0 max 16384 2024-09-12T03:10:48.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.484+0000 7fb061ac0740 1 Processor -- start 2024-09-12T03:10:48.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.484+0000 7fb061ac0740 1 -- start start 2024-09-12T03:10:48.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.484+0000 7fb061ac0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55facb96cb60 0x55facb98f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:48.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.484+0000 7fb061ac0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55facb96d480 0x55facb98fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:48.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.484+0000 7fb061ac0740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55facb977d40 0x55facb967a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:48.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.484+0000 7fb061ac0740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb0581afd20 con 0x55facb977d40 2024-09-12T03:10:48.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.484+0000 7fb061ac0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb0581ad5f0 con 0x55facb96d480 2024-09-12T03:10:48.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.484+0000 7fb061ac0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb0581ab440 con 0x55facb96cb60 2024-09-12T03:10:48.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.484+0000 7fb05dc84640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55facb977d40 0x55facb967a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:48.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.484+0000 7fb05dc84640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55facb977d40 0x55facb967a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55872/0 (socket says 172.21.15.116:55872) 2024-09-12T03:10:48.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.484+0000 7fb05dc84640 1 -- 172.21.15.116:0/1064778687 learned_addr learned my addr 172.21.15.116:0/1064778687 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:48.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.484+0000 7fb05d483640 1 --2- 172.21.15.116:0/1064778687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55facb96cb60 0x55facb98f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:48.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.484+0000 7fb05cc82640 1 --2- 172.21.15.116:0/1064778687 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55facb96d480 0x55facb98fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:48.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.484+0000 7fb05dc84640 1 -- 172.21.15.116:0/1064778687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55facb96cb60 msgr2=0x55facb98f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:48.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.484+0000 7fb05dc84640 1 --2- 172.21.15.116:0/1064778687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55facb96cb60 0x55facb98f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.484+0000 7fb05dc84640 1 -- 172.21.15.116:0/1064778687 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55facb96d480 msgr2=0x55facb98fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:48.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.484+0000 7fb05dc84640 1 --2- 172.21.15.116:0/1064778687 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55facb96d480 0x55facb98fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.484+0000 7fb05dc84640 1 -- 172.21.15.116:0/1064778687 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55facb98a3f0 con 0x55facb977d40 2024-09-12T03:10:48.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.484+0000 7fb05d483640 1 --2- 172.21.15.116:0/1064778687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55facb96cb60 0x55facb98f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:10:48.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.485+0000 7fb05dc84640 1 --2- 172.21.15.116:0/1064778687 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55facb977d40 0x55facb967a50 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fb05000eb00 tx=0x7fb050010f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5efe02c7f0da8dd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:48.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.485+0000 7fb042ffd640 1 -- 172.21.15.116:0/1064778687 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb050035070 con 0x55facb977d40 2024-09-12T03:10:48.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.485+0000 7fb042ffd640 1 -- 172.21.15.116:0/1064778687 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb050003e50 con 0x55facb977d40 2024-09-12T03:10:48.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.485+0000 7fb042ffd640 1 -- 172.21.15.116:0/1064778687 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05000e840 con 0x55facb977d40 2024-09-12T03:10:48.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.485+0000 7fb061ac0740 1 -- 172.21.15.116:0/1064778687 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55facb977d40 msgr2=0x55facb967a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:48.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.485+0000 7fb061ac0740 1 --2- 172.21.15.116:0/1064778687 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55facb977d40 0x55facb967a50 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fb05000eb00 tx=0x7fb050010f60 comp rx=0 tx=0).stop 2024-09-12T03:10:48.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.485+0000 7fb061ac0740 1 -- 172.21.15.116:0/1064778687 shutdown_connections 2024-09-12T03:10:48.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.485+0000 7fb061ac0740 1 --2- 172.21.15.116:0/1064778687 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55facb977d40 0x55facb967a50 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.485+0000 7fb061ac0740 1 --2- 172.21.15.116:0/1064778687 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55facb96d480 0x55facb98fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.485+0000 7fb061ac0740 1 --2- 172.21.15.116:0/1064778687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55facb96cb60 0x55facb98f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.485+0000 7fb061ac0740 1 -- 172.21.15.116:0/1064778687 >> 172.21.15.116:0/1064778687 conn(0x55facb9690f0 msgr2=0x55facb973ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:48.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.486+0000 7fb061ac0740 1 -- 172.21.15.116:0/1064778687 shutdown_connections 2024-09-12T03:10:48.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.486+0000 7fb061ac0740 1 -- 172.21.15.116:0/1064778687 wait complete. 2024-09-12T03:10:48.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.486+0000 7fb061ac0740 1 -- start start 2024-09-12T03:10:48.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.486+0000 7fb061ac0740 1 Processor -- start 2024-09-12T03:10:48.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.486+0000 7fb061ac0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55facb96cb60 0x55facb980530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:48.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.486+0000 7fb061ac0740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55facb96d480 0x55facb97b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:48.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.486+0000 7fb061ac0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55facb977d40 0x55facb97be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:48.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.486+0000 7fb061ac0740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb0581abab0 con 0x55facb96d480 2024-09-12T03:10:48.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.486+0000 7fb061ac0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb0581acf20 con 0x55facb96cb60 2024-09-12T03:10:48.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.486+0000 7fb061ac0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb0581ac180 con 0x55facb977d40 2024-09-12T03:10:48.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.486+0000 7fb061ac0740 10 client.0 _ll_register_callbacks cb 0x7fb04c00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:48.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.486+0000 7fb05cc82640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55facb96d480 0x55facb97b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:48.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.486+0000 7fb05cc82640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55facb96d480 0x55facb97b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55876/0 (socket says 172.21.15.116:55876) 2024-09-12T03:10:48.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.486+0000 7fb05cc82640 1 -- 172.21.15.116:0/121529411 learned_addr learned my addr 172.21.15.116:0/121529411 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:48.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.487+0000 7fb05d483640 1 --2- 172.21.15.116:0/121529411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55facb96cb60 0x55facb980530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:48.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.487+0000 7fb05dc84640 1 --2- 172.21.15.116:0/121529411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55facb977d40 0x55facb97be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:48.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.487+0000 7fb05cc82640 1 -- 172.21.15.116:0/121529411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55facb977d40 msgr2=0x55facb97be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:48.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.487+0000 7fb05cc82640 1 --2- 172.21.15.116:0/121529411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55facb977d40 0x55facb97be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.487+0000 7fb05cc82640 1 -- 172.21.15.116:0/121529411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55facb96cb60 msgr2=0x55facb980530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:48.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.487+0000 7fb05cc82640 1 --2- 172.21.15.116:0/121529411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55facb96cb60 0x55facb980530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.487+0000 7fb05cc82640 1 -- 172.21.15.116:0/121529411 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55facb97c660 con 0x55facb96d480 2024-09-12T03:10:48.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.487+0000 7fb05dc84640 1 --2- 172.21.15.116:0/121529411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55facb977d40 0x55facb97be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:48.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.487+0000 7fb05cc82640 1 --2- 172.21.15.116:0/121529411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55facb96d480 0x55facb97b860 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fb054009900 tx=0x7fb0540318e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595eff1bf202745d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:48.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.487+0000 7fb040ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:48.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.487+0000 7fb040ff9640 1 -- 172.21.15.116:0/121529411 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb054035040 con 0x55facb96d480 2024-09-12T03:10:48.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.487+0000 7fb040ff9640 1 -- 172.21.15.116:0/121529411 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fb054004450 con 0x55facb96d480 2024-09-12T03:10:48.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.487+0000 7fb061ac0740 1 -- 172.21.15.116:0/121529411 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb050003e50 con 0x55facb96d480 2024-09-12T03:10:48.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.487+0000 7fb040ff9640 1 -- 172.21.15.116:0/121529411 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb054057050 con 0x55facb96d480 2024-09-12T03:10:48.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.487+0000 7fb061ac0740 10 client.17292 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:48.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.487+0000 7fb061ac0740 1 -- 172.21.15.116:0/121529411 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55facb9f27e0 con 0x55facb96d480 2024-09-12T03:10:48.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.487+0000 7fb040ff9640 1 -- 172.21.15.116:0/121529411 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb050003e50 con 0x55facb96d480 2024-09-12T03:10:48.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.487+0000 7fb040ff9640 1 -- 172.21.15.116:0/121529411 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb0540035f0 con 0x55facb96d480 2024-09-12T03:10:48.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.488+0000 7fb061ac0740 10 client.17292 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:48.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.488+0000 7fb061ac0740 10 client.17292 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:10:48.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.488+0000 7fb061ac0740 20 client.17292 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:48.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.488+0000 7fb061ac0740 1 --2- 172.21.15.116:0/121529411 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55facb9f2fe0 0x55facb9f5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:48.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.488+0000 7fb061ac0740 4 client.17292 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:10:48.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.488+0000 7fb061ac0740 1 -- 172.21.15.116:0/121529411 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55facb8d9d00 con 0x55facb9f2fe0 2024-09-12T03:10:48.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.488+0000 7fb05d483640 1 --2- 172.21.15.116:0/121529411 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55facb9f2fe0 0x55facb9f5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:48.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.488+0000 7fb040ff9640 10 client.17292 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:48.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.488+0000 7fb05d483640 1 --2- 172.21.15.116:0/121529411 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55facb9f2fe0 0x55facb9f5400 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9453a97eafed5f67 server_cookie=4b6d898d5b0e7f5f in_seq=0 out_seq=0 2024-09-12T03:10:48.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.489+0000 7fb040ff9640 1 -- 172.21.15.116:0/121529411 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55facb8d9d00 con 0x55facb9f2fe0 2024-09-12T03:10:48.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.489+0000 7fb040ff9640 10 client.17292 handle_command_reply: tid=0 2024-09-12T03:10:48.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.489+0000 7fb061ac0740 1 client.17292 shutdown 2024-09-12T03:10:48.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.489+0000 7fb061ac0740 1 -- 172.21.15.116:0/121529411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55facb96d480 msgr2=0x55facb97b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:48.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.489+0000 7fb061ac0740 1 --2- 172.21.15.116:0/121529411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55facb96d480 0x55facb97b860 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fb054009900 tx=0x7fb0540318e0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.489+0000 7fb061ac0740 1 -- 172.21.15.116:0/121529411 shutdown_connections 2024-09-12T03:10:48.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.489+0000 7fb061ac0740 1 --2- 172.21.15.116:0/121529411 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55facb9f2fe0 0x55facb9f5400 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.489+0000 7fb061ac0740 1 --2- 172.21.15.116:0/121529411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55facb977d40 0x55facb97be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.489+0000 7fb061ac0740 1 --2- 172.21.15.116:0/121529411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55facb96d480 0x55facb97b860 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.489+0000 7fb061ac0740 1 --2- 172.21.15.116:0/121529411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55facb96cb60 0x55facb980530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.489+0000 7fb061ac0740 1 -- 172.21.15.116:0/121529411 >> 172.21.15.116:0/121529411 conn(0x55facb9690f0 msgr2=0x55facb967a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:48.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.489+0000 7fb040ff9640 0 client.17292 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:48.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.490+0000 7fb061ac0740 1 -- 172.21.15.116:0/121529411 shutdown_connections 2024-09-12T03:10:48.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.490+0000 7fb061ac0740 1 -- 172.21.15.116:0/121529411 wait complete. 2024-09-12T03:10:48.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.490+0000 7fb061ac0740 20 client.17292 trim_cache size 0 max 16384 2024-09-12T03:10:48.507 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:10:48.507 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:10:48.507 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "a5989cd1-cfee-4639-8c38-1f88fc20e096", 2024-09-12T03:10:48.507 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:10:48.507 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:10:48.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.490+0000 7fb05f6e7640 1 -- 172.21.15.116:0/944296973 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb0440a2b80 msgr2=0x7fb0440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:48.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.490+0000 7fb05f6e7640 1 --2- 172.21.15.116:0/944296973 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb0440a2b80 0x7fb0440a5040 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb050023000 tx=0x7fb050020000 comp rx=0 tx=0).stop 2024-09-12T03:10:48.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.490+0000 7fb05f6e7640 1 -- 172.21.15.116:0/944296973 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb058101df0 msgr2=0x7fb058198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:48.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.490+0000 7fb05f6e7640 1 --2- 172.21.15.116:0/944296973 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb058101df0 0x7fb058198420 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00e9f0 tx=0x7fb04c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.490+0000 7fb05f6e7640 1 -- 172.21.15.116:0/944296973 shutdown_connections 2024-09-12T03:10:48.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.490+0000 7fb05f6e7640 1 --2- 172.21.15.116:0/944296973 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb0440a2b80 0x7fb0440a5040 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.490+0000 7fb05f6e7640 1 --2- 172.21.15.116:0/944296973 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb058106f20 0x7fb05819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.490+0000 7fb05f6e7640 1 --2- 172.21.15.116:0/944296973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0581027a0 0x7fb058198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.490+0000 7fb05f6e7640 1 --2- 172.21.15.116:0/944296973 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb058101df0 0x7fb058198420 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:48.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.490+0000 7fb05f6e7640 1 -- 172.21.15.116:0/944296973 >> 172.21.15.116:0/944296973 conn(0x7fb0580fd8f0 msgr2=0x7fb0580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:48.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.491+0000 7fb05f6e7640 1 -- 172.21.15.116:0/944296973 shutdown_connections 2024-09-12T03:10:48.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:48.491+0000 7fb05f6e7640 1 -- 172.21.15.116:0/944296973 wait complete. 2024-09-12T03:10:48.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:48 smithi183 ceph-mon[35535]: pgmap v572: 97 pgs: 97 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:10:48.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:48 smithi177 ceph-mon[35305]: pgmap v572: 97 pgs: 97 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:10:48.751 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:10:49.282 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:10:49.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:49 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:10:49.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:49 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:10:49.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:49 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:10:49.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:49 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:10:49.403 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:49 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:10:49.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:49 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:10:49.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:49 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:10:49.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:49 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:10:49.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:49 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:10:49.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:49 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:10:49.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:49 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:10:49.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:49 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:10:49.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:49 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:10:49.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:49 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:10:49.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:49 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:10:49.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.789+0000 7fd8fc881640 1 -- 172.21.15.116:0/2774604536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8f8101420 msgr2=0x7fd8f810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:49.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.789+0000 7fd8fc881640 1 --2- 172.21.15.116:0/2774604536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8f8101420 0x7fd8f810d950 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0009cd0 tx=0x7fd8f002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.789+0000 7fd8fc881640 1 -- 172.21.15.116:0/2774604536 shutdown_connections 2024-09-12T03:10:49.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.789+0000 7fd8fc881640 1 --2- 172.21.15.116:0/2774604536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8f8101420 0x7fd8f810d950 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.789+0000 7fd8fc881640 1 --2- 172.21.15.116:0/2774604536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8f8100a80 0x7fd8f8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.789+0000 7fd8fc881640 1 --2- 172.21.15.116:0/2774604536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8f8106a80 0x7fd8f8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.789+0000 7fd8fc881640 1 -- 172.21.15.116:0/2774604536 >> 172.21.15.116:0/2774604536 conn(0x7fd8f80fc7a0 msgr2=0x7fd8f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:49.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.789+0000 7fd8fc881640 1 -- 172.21.15.116:0/2774604536 shutdown_connections 2024-09-12T03:10:49.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.789+0000 7fd8fc881640 1 -- 172.21.15.116:0/2774604536 wait complete. 2024-09-12T03:10:49.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.790+0000 7fd8fc881640 1 Processor -- start 2024-09-12T03:10:49.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.790+0000 7fd8fc881640 1 -- start start 2024-09-12T03:10:49.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.790+0000 7fd8fc881640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8f8100a80 0x7fd8f8199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:49.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.790+0000 7fd8fc881640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8f8101420 0x7fd8f8199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:49.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.790+0000 7fd8fc881640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8f8106a80 0x7fd8f81a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:49.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.790+0000 7fd8fc881640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f8110ec0 con 0x7fd8f8101420 2024-09-12T03:10:49.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.790+0000 7fd8fc881640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f8110d40 con 0x7fd8f8100a80 2024-09-12T03:10:49.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.790+0000 7fd8fc881640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f8111040 con 0x7fd8f8106a80 2024-09-12T03:10:49.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.790+0000 7fd8f5d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8f8101420 0x7fd8f8199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:49.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.790+0000 7fd8f5d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8f8101420 0x7fd8f8199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55894/0 (socket says 172.21.15.116:55894) 2024-09-12T03:10:49.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.790+0000 7fd8f5d9b640 1 -- 172.21.15.116:0/688948843 learned_addr learned my addr 172.21.15.116:0/688948843 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:49.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.790+0000 7fd8f6d9d640 1 --2- 172.21.15.116:0/688948843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8f8106a80 0x7fd8f81a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:49.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.790+0000 7fd8f659c640 1 --2- 172.21.15.116:0/688948843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8f8100a80 0x7fd8f8199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:49.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.790+0000 7fd8f5d9b640 1 -- 172.21.15.116:0/688948843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8f8106a80 msgr2=0x7fd8f81a43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:49.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.790+0000 7fd8f5d9b640 1 --2- 172.21.15.116:0/688948843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8f8106a80 0x7fd8f81a43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.790+0000 7fd8f5d9b640 1 -- 172.21.15.116:0/688948843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8f8100a80 msgr2=0x7fd8f8199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:49.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.790+0000 7fd8f5d9b640 1 --2- 172.21.15.116:0/688948843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8f8100a80 0x7fd8f8199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.790+0000 7fd8f5d9b640 1 -- 172.21.15.116:0/688948843 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f81a48e0 con 0x7fd8f8101420 2024-09-12T03:10:49.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.790+0000 7fd8f6d9d640 1 --2- 172.21.15.116:0/688948843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8f8106a80 0x7fd8f81a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:10:49.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.790+0000 7fd8f659c640 1 --2- 172.21.15.116:0/688948843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8f8100a80 0x7fd8f8199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:49.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.791+0000 7fd8f5d9b640 1 --2- 172.21.15.116:0/688948843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8f8101420 0x7fd8f8199860 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00b4a0 tx=0x7fd8ec00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:49.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.791+0000 7fd8e77fe640 1 -- 172.21.15.116:0/688948843 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec014070 con 0x7fd8f8101420 2024-09-12T03:10:49.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.791+0000 7fd8e77fe640 1 -- 172.21.15.116:0/688948843 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8ec010af0 con 0x7fd8f8101420 2024-09-12T03:10:49.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.791+0000 7fd8fc881640 1 -- 172.21.15.116:0/688948843 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f81a4ab0 con 0x7fd8f8101420 2024-09-12T03:10:49.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.791+0000 7fd8e77fe640 1 -- 172.21.15.116:0/688948843 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec017d10 con 0x7fd8f8101420 2024-09-12T03:10:49.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.791+0000 7fd8fc881640 1 -- 172.21.15.116:0/688948843 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f81a4fc0 con 0x7fd8f8101420 2024-09-12T03:10:49.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.793+0000 7fd8e77fe640 1 -- 172.21.15.116:0/688948843 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd8ec017070 con 0x7fd8f8101420 2024-09-12T03:10:49.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.793+0000 7fd8fec5a740 1 Processor -- start 2024-09-12T03:10:49.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.793+0000 7fd8fec5a740 1 -- start start 2024-09-12T03:10:49.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.794+0000 7fd8fec5a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5579c6913660 0x5579c6913a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:49.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.794+0000 7fd8fec5a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579c6913f80 0x5579c6922810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:49.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.794+0000 7fd8e77fe640 1 --2- 172.21.15.116:0/688948843 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd8e00a2b80 0x7fd8e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:49.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.794+0000 7fd8fec5a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579c6922d50 0x5579c6925140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:49.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.794+0000 7fd8fec5a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5579c68792a0 con 0x5579c6922d50 2024-09-12T03:10:49.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.794+0000 7fd8fec5a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5579c6820b10 con 0x5579c6913f80 2024-09-12T03:10:49.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.794+0000 7fd8fec5a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5579c6875260 con 0x5579c6913660 2024-09-12T03:10:49.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.794+0000 7fd8f6d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579c6922d50 0x5579c6925140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:49.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.794+0000 7fd8f659c640 1 --2- 172.21.15.116:0/688948843 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd8e00a2b80 0x7fd8e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:49.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.794+0000 7fd8f6d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579c6922d50 0x5579c6925140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55898/0 (socket says 172.21.15.116:55898) 2024-09-12T03:10:49.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.794+0000 7fd8f6d9d640 1 -- 172.21.15.116:0/3363085902 learned_addr learned my addr 172.21.15.116:0/3363085902 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:49.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.794+0000 7fd8e77fe640 1 -- 172.21.15.116:0/688948843 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd8ec0ce490 con 0x7fd8f8101420 2024-09-12T03:10:49.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.794+0000 7fd8f5d9b640 1 --2- 172.21.15.116:0/3363085902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579c6913f80 0x5579c6922810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:49.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.794+0000 7fd8f6d9d640 1 --2- 172.21.15.116:0/3363085902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5579c6913660 0x5579c6913a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:49.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.794+0000 7fd8f6d9d640 1 -- 172.21.15.116:0/3363085902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5579c6913660 msgr2=0x5579c6913a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:49.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.794+0000 7fd8f6d9d640 1 --2- 172.21.15.116:0/3363085902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5579c6913660 0x5579c6913a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.794+0000 7fd8f6d9d640 1 -- 172.21.15.116:0/3363085902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579c6913f80 msgr2=0x5579c6922810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:49.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.794+0000 7fd8f6d9d640 1 --2- 172.21.15.116:0/3363085902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579c6913f80 0x5579c6922810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.794+0000 7fd8f6d9d640 1 -- 172.21.15.116:0/3363085902 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579c65eab90 con 0x5579c6922d50 2024-09-12T03:10:49.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.794+0000 7fd8f5d9b640 1 --2- 172.21.15.116:0/3363085902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579c6913f80 0x5579c6922810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:10:49.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.794+0000 7fd8f659c640 1 --2- 172.21.15.116:0/688948843 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd8e00a2b80 0x7fd8e00a5040 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8009b40 tx=0x7fd8e8005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:49.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.795+0000 7fd8f6d9d640 1 --2- 172.21.15.116:0/3363085902 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579c6922d50 0x5579c6925140 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0035000 tx=0x7fd8f0005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd00169badd7b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:49.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.795+0000 7fd8e4ff9640 1 -- 172.21.15.116:0/3363085902 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f00319a0 con 0x5579c6922d50 2024-09-12T03:10:49.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.795+0000 7fd8e4ff9640 1 -- 172.21.15.116:0/3363085902 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8f0036040 con 0x5579c6922d50 2024-09-12T03:10:49.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.795+0000 7fd8e4ff9640 1 -- 172.21.15.116:0/3363085902 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f00045c0 con 0x5579c6922d50 2024-09-12T03:10:49.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.795+0000 7fd8fec5a740 1 -- 172.21.15.116:0/3363085902 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579c6922d50 msgr2=0x5579c6925140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:49.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.795+0000 7fd8fec5a740 1 --2- 172.21.15.116:0/3363085902 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579c6922d50 0x5579c6925140 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0035000 tx=0x7fd8f0005ef0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.795+0000 7fd8fec5a740 1 -- 172.21.15.116:0/3363085902 shutdown_connections 2024-09-12T03:10:49.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.795+0000 7fd8fec5a740 1 --2- 172.21.15.116:0/3363085902 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579c6922d50 0x5579c6925140 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.795+0000 7fd8fec5a740 1 --2- 172.21.15.116:0/3363085902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579c6913f80 0x5579c6922810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.795+0000 7fd8fec5a740 1 --2- 172.21.15.116:0/3363085902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5579c6913660 0x5579c6913a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.795+0000 7fd8fec5a740 1 -- 172.21.15.116:0/3363085902 >> 172.21.15.116:0/3363085902 conn(0x5579c690fcf0 msgr2=0x5579c69100d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:49.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.796+0000 7fd8fec5a740 1 -- 172.21.15.116:0/3363085902 shutdown_connections 2024-09-12T03:10:49.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.796+0000 7fd8fec5a740 1 -- 172.21.15.116:0/3363085902 wait complete. 2024-09-12T03:10:49.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.796+0000 7fd8fec5a740 1 -- start start 2024-09-12T03:10:49.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.796+0000 7fd8fec5a740 1 Processor -- start 2024-09-12T03:10:49.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.796+0000 7fd8fec5a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579c6913660 0x5579c691fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:49.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.796+0000 7fd8fec5a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579c6913f80 0x5579c69201c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:49.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.796+0000 7fd8fec5a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5579c6922d50 0x5579c6920700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:49.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.796+0000 7fd8fec5a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5579c6925920 con 0x5579c6913660 2024-09-12T03:10:49.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.796+0000 7fd8fec5a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5579c68016e0 con 0x5579c6913f80 2024-09-12T03:10:49.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.796+0000 7fd8fec5a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5579c6925aa0 con 0x5579c6922d50 2024-09-12T03:10:49.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.796+0000 7fd8fec5a740 10 client.0 _ll_register_callbacks cb 0x5579c6825a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:49.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.796+0000 7fd8f659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579c6913660 0x5579c691fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:49.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.796+0000 7fd8f659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579c6913660 0x5579c691fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55912/0 (socket says 172.21.15.116:55912) 2024-09-12T03:10:49.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.796+0000 7fd8f659c640 1 -- 172.21.15.116:0/422720804 learned_addr learned my addr 172.21.15.116:0/422720804 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:49.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.797+0000 7fd8f6d9d640 1 --2- 172.21.15.116:0/422720804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5579c6922d50 0x5579c6920700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:49.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.797+0000 7fd8f659c640 1 -- 172.21.15.116:0/422720804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5579c6922d50 msgr2=0x5579c6920700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:49.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.797+0000 7fd8f659c640 1 --2- 172.21.15.116:0/422720804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5579c6922d50 0x5579c6920700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.797+0000 7fd8f5d9b640 1 --2- 172.21.15.116:0/422720804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579c6913f80 0x5579c69201c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:49.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.797+0000 7fd8f659c640 1 -- 172.21.15.116:0/422720804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579c6913f80 msgr2=0x5579c69201c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:49.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.797+0000 7fd8f659c640 1 --2- 172.21.15.116:0/422720804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579c6913f80 0x5579c69201c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.797+0000 7fd8f659c640 1 -- 172.21.15.116:0/422720804 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579c692eea0 con 0x5579c6913660 2024-09-12T03:10:49.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.797+0000 7fd8f6d9d640 1 --2- 172.21.15.116:0/422720804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5579c6922d50 0x5579c6920700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:49.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.797+0000 7fd8f659c640 1 --2- 172.21.15.116:0/422720804 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579c6913660 0x5579c691fc80 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80117a0 tx=0x7fd8e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b61eb274da343cd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:49.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.797+0000 7fd8ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:49.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.797+0000 7fd8ddffb640 1 -- 172.21.15.116:0/422720804 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8022020 con 0x5579c6913660 2024-09-12T03:10:49.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.797+0000 7fd8ddffb640 1 -- 172.21.15.116:0/422720804 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd8e80108d0 con 0x5579c6913660 2024-09-12T03:10:49.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.797+0000 7fd8fec5a740 1 -- 172.21.15.116:0/422720804 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8f0036040 con 0x5579c6913660 2024-09-12T03:10:49.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.797+0000 7fd8ddffb640 1 -- 172.21.15.116:0/422720804 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd8e8010ad0 con 0x5579c6913660 2024-09-12T03:10:49.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.797+0000 7fd8fec5a740 10 client.17304 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:49.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.797+0000 7fd8fec5a740 1 -- 172.21.15.116:0/422720804 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5579c6931890 con 0x5579c6913660 2024-09-12T03:10:49.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.797+0000 7fd8ddffb640 1 -- 172.21.15.116:0/422720804 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8f0036040 con 0x5579c6913660 2024-09-12T03:10:49.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.797+0000 7fd8ddffb640 1 -- 172.21.15.116:0/422720804 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd8e800f040 con 0x5579c6913660 2024-09-12T03:10:49.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.797+0000 7fd8fec5a740 10 client.17304 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:49.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.797+0000 7fd8fec5a740 10 client.17304 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:10:49.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.798+0000 7fd8fec5a740 20 client.17304 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:49.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.798+0000 7fd8fec5a740 1 --2- 172.21.15.116:0/422720804 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5579c6932da0 0x5579c69351f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:49.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.798+0000 7fd8fec5a740 4 client.17304 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:10:49.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.798+0000 7fd8fec5a740 1 -- 172.21.15.116:0/422720804 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579c68869d0 con 0x5579c6932da0 2024-09-12T03:10:49.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.798+0000 7fd8f5d9b640 1 --2- 172.21.15.116:0/422720804 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5579c6932da0 0x5579c69351f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:49.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.798+0000 7fd8ddffb640 10 client.17304 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:49.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.798+0000 7fd8f5d9b640 1 --2- 172.21.15.116:0/422720804 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5579c6932da0 0x5579c69351f0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef02b7347a8c99fa server_cookie=66d8e842398e9ad1 in_seq=0 out_seq=0 2024-09-12T03:10:49.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.799+0000 7fd8ddffb640 1 -- 172.21.15.116:0/422720804 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5579c68869d0 con 0x5579c6932da0 2024-09-12T03:10:49.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.799+0000 7fd8ddffb640 10 client.17304 handle_command_reply: tid=0 2024-09-12T03:10:49.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.799+0000 7fd8fec5a740 1 client.17304 shutdown 2024-09-12T03:10:49.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.799+0000 7fd8fec5a740 1 -- 172.21.15.116:0/422720804 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579c6913660 msgr2=0x5579c691fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:49.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.799+0000 7fd8fec5a740 1 --2- 172.21.15.116:0/422720804 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579c6913660 0x5579c691fc80 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80117a0 tx=0x7fd8e80046c0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.799+0000 7fd8fec5a740 1 -- 172.21.15.116:0/422720804 shutdown_connections 2024-09-12T03:10:49.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.799+0000 7fd8fec5a740 1 --2- 172.21.15.116:0/422720804 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5579c6932da0 0x5579c69351f0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.799+0000 7fd8fec5a740 1 --2- 172.21.15.116:0/422720804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5579c6922d50 0x5579c6920700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.799+0000 7fd8fec5a740 1 --2- 172.21.15.116:0/422720804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579c6913f80 0x5579c69201c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.799+0000 7fd8ddffb640 0 client.17304 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:49.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.799+0000 7fd8fec5a740 1 --2- 172.21.15.116:0/422720804 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579c6913660 0x5579c691fc80 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.799+0000 7fd8fec5a740 1 -- 172.21.15.116:0/422720804 >> 172.21.15.116:0/422720804 conn(0x5579c690fcf0 msgr2=0x5579c6915a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:49.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.799+0000 7fd8fec5a740 1 -- 172.21.15.116:0/422720804 shutdown_connections 2024-09-12T03:10:49.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.799+0000 7fd8fec5a740 1 -- 172.21.15.116:0/422720804 wait complete. 2024-09-12T03:10:49.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.799+0000 7fd8fec5a740 20 client.17304 trim_cache size 0 max 16384 2024-09-12T03:10:49.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.817+0000 7fd8fec5a740 1 Processor -- start 2024-09-12T03:10:49.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.817+0000 7fd8fec5a740 1 -- start start 2024-09-12T03:10:49.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.817+0000 7fd8fec5a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5579c6913660 0x5579c6915940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:49.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.817+0000 7fd8fec5a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579c6913f80 0x5579c693aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:49.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.817+0000 7fd8fec5a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579c6922d50 0x5579c693b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:49.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.818+0000 7fd8fec5a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f81adcb0 con 0x5579c6922d50 2024-09-12T03:10:49.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.818+0000 7fd8fec5a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f81ab580 con 0x5579c6913f80 2024-09-12T03:10:49.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.818+0000 7fd8fec5a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f81a93d0 con 0x5579c6913660 2024-09-12T03:10:49.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.818+0000 7fd8f6d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579c6922d50 0x5579c693b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:49.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.818+0000 7fd8f6d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579c6922d50 0x5579c693b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55926/0 (socket says 172.21.15.116:55926) 2024-09-12T03:10:49.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.818+0000 7fd8f6d9d640 1 -- 172.21.15.116:0/38132796 learned_addr learned my addr 172.21.15.116:0/38132796 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:49.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.818+0000 7fd8f659c640 1 --2- 172.21.15.116:0/38132796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5579c6913660 0x5579c6915940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:49.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.818+0000 7fd8f5d9b640 1 --2- 172.21.15.116:0/38132796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579c6913f80 0x5579c693aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:49.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.818+0000 7fd8f6d9d640 1 -- 172.21.15.116:0/38132796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5579c6913660 msgr2=0x5579c6915940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:49.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.818+0000 7fd8f6d9d640 1 --2- 172.21.15.116:0/38132796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5579c6913660 0x5579c6915940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.818+0000 7fd8f6d9d640 1 -- 172.21.15.116:0/38132796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579c6913f80 msgr2=0x5579c693aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:49.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.818+0000 7fd8f6d9d640 1 --2- 172.21.15.116:0/38132796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579c6913f80 0x5579c693aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.818+0000 7fd8f6d9d640 1 -- 172.21.15.116:0/38132796 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579c691ec30 con 0x5579c6922d50 2024-09-12T03:10:49.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.818+0000 7fd8f6d9d640 1 --2- 172.21.15.116:0/38132796 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579c6922d50 0x5579c693b000 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0031b80 tx=0x7fd8f0031bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2111e2ed364b84ba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:49.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.818+0000 7fd8dffff640 1 -- 172.21.15.116:0/38132796 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0003b70 con 0x5579c6922d50 2024-09-12T03:10:49.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.818+0000 7fd8dffff640 1 -- 172.21.15.116:0/38132796 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8f0036040 con 0x5579c6922d50 2024-09-12T03:10:49.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.818+0000 7fd8dffff640 1 -- 172.21.15.116:0/38132796 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f00305a0 con 0x5579c6922d50 2024-09-12T03:10:49.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.819+0000 7fd8fec5a740 1 -- 172.21.15.116:0/38132796 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579c6922d50 msgr2=0x5579c693b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:49.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.819+0000 7fd8fec5a740 1 --2- 172.21.15.116:0/38132796 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579c6922d50 0x5579c693b000 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0031b80 tx=0x7fd8f0031bb0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.819+0000 7fd8fec5a740 1 -- 172.21.15.116:0/38132796 shutdown_connections 2024-09-12T03:10:49.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.819+0000 7fd8fec5a740 1 --2- 172.21.15.116:0/38132796 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579c6922d50 0x5579c693b000 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.819+0000 7fd8fec5a740 1 --2- 172.21.15.116:0/38132796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579c6913f80 0x5579c693aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.819+0000 7fd8fec5a740 1 --2- 172.21.15.116:0/38132796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5579c6913660 0x5579c6915940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.819+0000 7fd8fec5a740 1 -- 172.21.15.116:0/38132796 >> 172.21.15.116:0/38132796 conn(0x5579c690fcf0 msgr2=0x5579c6915230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:49.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.819+0000 7fd8fec5a740 1 -- 172.21.15.116:0/38132796 shutdown_connections 2024-09-12T03:10:49.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.819+0000 7fd8fec5a740 1 -- 172.21.15.116:0/38132796 wait complete. 2024-09-12T03:10:49.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.820+0000 7fd8fec5a740 1 -- start start 2024-09-12T03:10:49.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.820+0000 7fd8fec5a740 1 Processor -- start 2024-09-12T03:10:49.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.820+0000 7fd8fec5a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579c6913660 0x5579c6927360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:49.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.820+0000 7fd8fec5a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5579c6913f80 0x5579c69278a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:49.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.820+0000 7fd8fec5a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579c6922d50 0x5579c6996ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:49.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.820+0000 7fd8fec5a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f81a9a40 con 0x5579c6913660 2024-09-12T03:10:49.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.820+0000 7fd8fec5a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f81aaeb0 con 0x5579c6922d50 2024-09-12T03:10:49.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.820+0000 7fd8fec5a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f81aa110 con 0x5579c6913f80 2024-09-12T03:10:49.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.820+0000 7fd8fec5a740 10 client.0 _ll_register_callbacks cb 0x7fd8e8011470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:49.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.820+0000 7fd8f659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579c6913660 0x5579c6927360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:49.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.820+0000 7fd8f659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579c6913660 0x5579c6927360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55940/0 (socket says 172.21.15.116:55940) 2024-09-12T03:10:49.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.820+0000 7fd8f659c640 1 -- 172.21.15.116:0/674712190 learned_addr learned my addr 172.21.15.116:0/674712190 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:49.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.820+0000 7fd8f6d9d640 1 --2- 172.21.15.116:0/674712190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579c6922d50 0x5579c6996ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:49.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.820+0000 7fd8f5d9b640 1 --2- 172.21.15.116:0/674712190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5579c6913f80 0x5579c69278a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:49.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.821+0000 7fd8f659c640 1 -- 172.21.15.116:0/674712190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5579c6913f80 msgr2=0x5579c69278a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:49.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.821+0000 7fd8f659c640 1 --2- 172.21.15.116:0/674712190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5579c6913f80 0x5579c69278a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.821+0000 7fd8f659c640 1 -- 172.21.15.116:0/674712190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579c6922d50 msgr2=0x5579c6996ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:49.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.821+0000 7fd8f659c640 1 --2- 172.21.15.116:0/674712190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579c6922d50 0x5579c6996ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.821+0000 7fd8f659c640 1 -- 172.21.15.116:0/674712190 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579c6997110 con 0x5579c6913660 2024-09-12T03:10:49.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.821+0000 7fd8f6d9d640 1 --2- 172.21.15.116:0/674712190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579c6922d50 0x5579c6996ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:10:49.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.821+0000 7fd8f659c640 1 --2- 172.21.15.116:0/674712190 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579c6913660 0x5579c6927360 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800e6c0 tx=0x7fd8e8010ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575e8e17a40cbb28 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:49.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.821+0000 7fd8ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:49.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.821+0000 7fd8ddffb640 1 -- 172.21.15.116:0/674712190 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8036070 con 0x5579c6913660 2024-09-12T03:10:49.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.821+0000 7fd8ddffb640 1 -- 172.21.15.116:0/674712190 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd8e801e950 con 0x5579c6913660 2024-09-12T03:10:49.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.821+0000 7fd8fec5a740 1 -- 172.21.15.116:0/674712190 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8f0036040 con 0x5579c6913660 2024-09-12T03:10:49.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.821+0000 7fd8ddffb640 1 -- 172.21.15.116:0/674712190 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd8e8042050 con 0x5579c6913660 2024-09-12T03:10:49.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.821+0000 7fd8fec5a740 10 client.17310 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:49.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.821+0000 7fd8fec5a740 1 -- 172.21.15.116:0/674712190 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5579c699e440 con 0x5579c6913660 2024-09-12T03:10:49.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.821+0000 7fd8ddffb640 1 -- 172.21.15.116:0/674712190 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8f0036040 con 0x5579c6913660 2024-09-12T03:10:49.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.822+0000 7fd8ddffb640 1 -- 172.21.15.116:0/674712190 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd8e8032420 con 0x5579c6913660 2024-09-12T03:10:49.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.822+0000 7fd8fec5a740 10 client.17310 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:49.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.822+0000 7fd8fec5a740 10 client.17310 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:10:49.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.822+0000 7fd8fec5a740 20 client.17310 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:49.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.822+0000 7fd8fec5a740 1 --2- 172.21.15.116:0/674712190 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5579c699ec30 0x5579c69a1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:49.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.822+0000 7fd8fec5a740 4 client.17310 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}] 2024-09-12T03:10:49.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.822+0000 7fd8fec5a740 1 -- 172.21.15.116:0/674712190 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}) -- 0x5579c6880d20 con 0x5579c699ec30 2024-09-12T03:10:49.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.822+0000 7fd8f5d9b640 1 --2- 172.21.15.116:0/674712190 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5579c699ec30 0x5579c69a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:49.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.822+0000 7fd8f5d9b640 1 --2- 172.21.15.116:0/674712190 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5579c699ec30 0x5579c69a1050 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17c384b16b6b89d6 server_cookie=41071aec6ed5d584 in_seq=0 out_seq=0 2024-09-12T03:10:49.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.822+0000 7fd8ddffb640 10 client.17310 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:49.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.832+0000 7fd8ddffb640 1 -- 172.21.15.116:0/674712190 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5579c6880d20 con 0x5579c699ec30 2024-09-12T03:10:49.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.832+0000 7fd8ddffb640 10 client.17310 handle_command_reply: tid=0 2024-09-12T03:10:49.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.832+0000 7fd8fec5a740 1 client.17310 shutdown 2024-09-12T03:10:49.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.832+0000 7fd8fec5a740 1 -- 172.21.15.116:0/674712190 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579c6913660 msgr2=0x5579c6927360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:49.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.832+0000 7fd8fec5a740 1 --2- 172.21.15.116:0/674712190 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579c6913660 0x5579c6927360 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800e6c0 tx=0x7fd8e8010ea0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.832+0000 7fd8fec5a740 1 -- 172.21.15.116:0/674712190 shutdown_connections 2024-09-12T03:10:49.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.832+0000 7fd8fec5a740 1 --2- 172.21.15.116:0/674712190 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5579c699ec30 0x5579c69a1050 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.832+0000 7fd8ddffb640 0 client.17310 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:49.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.832+0000 7fd8fec5a740 1 --2- 172.21.15.116:0/674712190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5579c6922d50 0x5579c6996ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.832+0000 7fd8fec5a740 1 --2- 172.21.15.116:0/674712190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5579c6913f80 0x5579c69278a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.832+0000 7fd8fec5a740 1 --2- 172.21.15.116:0/674712190 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579c6913660 0x5579c6927360 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.832+0000 7fd8fec5a740 1 -- 172.21.15.116:0/674712190 >> 172.21.15.116:0/674712190 conn(0x5579c690fcf0 msgr2=0x5579c6915980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:49.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.833+0000 7fd8fec5a740 1 -- 172.21.15.116:0/674712190 shutdown_connections 2024-09-12T03:10:49.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.833+0000 7fd8fec5a740 1 -- 172.21.15.116:0/674712190 wait complete. 2024-09-12T03:10:49.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.833+0000 7fd8fec5a740 20 client.17310 trim_cache size 0 max 16384 2024-09-12T03:10:49.835 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:10:49.835 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 25, 2024-09-12T03:10:49.835 INFO:teuthology.orchestra.run.smithi116.stdout: "leader": 24494, 2024-09-12T03:10:49.836 INFO:teuthology.orchestra.run.smithi116.stdout: "set_version": 30, 2024-09-12T03:10:49.836 INFO:teuthology.orchestra.run.smithi116.stdout: "sets": { 2024-09-12T03:10:49.836 INFO:teuthology.orchestra.run.smithi116.stdout: "45ae6e70": { 2024-09-12T03:10:49.836 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 30, 2024-09-12T03:10:49.836 INFO:teuthology.orchestra.run.smithi116.stdout: "age_ref": 0.0, 2024-09-12T03:10:49.836 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:10:49.836 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "QUIESCED", 2024-09-12T03:10:49.836 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:10:49.836 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:10:49.836 INFO:teuthology.orchestra.run.smithi116.stdout: "timeout": 90.0, 2024-09-12T03:10:49.836 INFO:teuthology.orchestra.run.smithi116.stdout: "expiration": 139.4, 2024-09-12T03:10:49.836 INFO:teuthology.orchestra.run.smithi116.stdout: "members": { 2024-09-12T03:10:49.836 INFO:teuthology.orchestra.run.smithi116.stdout: "file:/": { 2024-09-12T03:10:49.837 INFO:teuthology.orchestra.run.smithi116.stdout: "excluded": false, 2024-09-12T03:10:49.837 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:10:49.837 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "QUIESCED", 2024-09-12T03:10:49.837 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:10:49.837 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:10:49.837 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:10:49.837 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:10:49.837 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:10:49.837 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:10:49.837 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:10:49.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.833+0000 7fd8fc881640 1 -- 172.21.15.116:0/688948843 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd8e00a2b80 msgr2=0x7fd8e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:49.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.833+0000 7fd8fc881640 1 --2- 172.21.15.116:0/688948843 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd8e00a2b80 0x7fd8e00a5040 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8009b40 tx=0x7fd8e8005e50 comp rx=0 tx=0).stop 2024-09-12T03:10:49.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.833+0000 7fd8fc881640 1 -- 172.21.15.116:0/688948843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8f8101420 msgr2=0x7fd8f8199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:49.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.833+0000 7fd8fc881640 1 --2- 172.21.15.116:0/688948843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8f8101420 0x7fd8f8199860 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00b4a0 tx=0x7fd8ec00b960 comp rx=0 tx=0).stop 2024-09-12T03:10:49.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.833+0000 7fd8fc881640 1 -- 172.21.15.116:0/688948843 shutdown_connections 2024-09-12T03:10:49.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.833+0000 7fd8fc881640 1 --2- 172.21.15.116:0/688948843 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd8e00a2b80 0x7fd8e00a5040 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.833+0000 7fd8fc881640 1 --2- 172.21.15.116:0/688948843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8f8106a80 0x7fd8f81a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.833+0000 7fd8fc881640 1 --2- 172.21.15.116:0/688948843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8f8101420 0x7fd8f8199860 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.833+0000 7fd8fc881640 1 --2- 172.21.15.116:0/688948843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8f8100a80 0x7fd8f8199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.834+0000 7fd8fc881640 1 -- 172.21.15.116:0/688948843 >> 172.21.15.116:0/688948843 conn(0x7fd8f80fc7a0 msgr2=0x7fd8f80fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:49.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.834+0000 7fd8fc881640 1 -- 172.21.15.116:0/688948843 shutdown_connections 2024-09-12T03:10:49.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:49.834+0000 7fd8fc881640 1 -- 172.21.15.116:0/688948843 wait complete. 2024-09-12T03:10:49.844 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:10:50.545 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:50 smithi116 ceph-mon[31410]: pgmap v573: 97 pgs: 97 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:10:50.575 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '45ae6e70', quiesce took 5.7 seconds. Will release after: 13.7 2024-09-12T03:10:50.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:50 smithi183 ceph-mon[35535]: pgmap v573: 97 pgs: 97 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:10:50.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:50 smithi177 ceph-mon[35305]: pgmap v573: 97 pgs: 97 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:10:51.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.096+0000 7fc6287b7640 1 -- 172.21.15.116:0/402006159 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc620100e30 msgr2=0x7fc620104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:51.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.096+0000 7fc6287b7640 1 --2- 172.21.15.116:0/402006159 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc620100e30 0x7fc620104e80 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fc610009900 tx=0x7fc61002a3f0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.096+0000 7fc6287b7640 1 -- 172.21.15.116:0/402006159 shutdown_connections 2024-09-12T03:10:51.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.096+0000 7fc6287b7640 1 --2- 172.21.15.116:0/402006159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6201055b0 0x7fc62010d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.096+0000 7fc6287b7640 1 --2- 172.21.15.116:0/402006159 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc620100e30 0x7fc620104e80 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.096+0000 7fc6287b7640 1 --2- 172.21.15.116:0/402006159 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc620100480 0x7fc620100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.096+0000 7fc6287b7640 1 -- 172.21.15.116:0/402006159 >> 172.21.15.116:0/402006159 conn(0x7fc620077210 msgr2=0x7fc6200fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:51.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.096+0000 7fc6287b7640 1 -- 172.21.15.116:0/402006159 shutdown_connections 2024-09-12T03:10:51.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.096+0000 7fc6287b7640 1 -- 172.21.15.116:0/402006159 wait complete. 2024-09-12T03:10:51.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.097+0000 7fc6287b7640 1 Processor -- start 2024-09-12T03:10:51.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.097+0000 7fc6287b7640 1 -- start start 2024-09-12T03:10:51.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.097+0000 7fc6287b7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc620100480 0x7fc620198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:51.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.097+0000 7fc6287b7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc620100e30 0x7fc620198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:51.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.097+0000 7fc6287b7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6201055b0 0x7fc62019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:51.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.097+0000 7fc6287b7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201105b0 con 0x7fc620100e30 2024-09-12T03:10:51.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.097+0000 7fc6287b7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc620110430 con 0x7fc6201055b0 2024-09-12T03:10:51.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.097+0000 7fc6287b7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc620110730 con 0x7fc620100480 2024-09-12T03:10:51.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.097+0000 7fc625d52640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc620100e30 0x7fc620198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:51.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.097+0000 7fc625d52640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc620100e30 0x7fc620198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44198/0 (socket says 172.21.15.116:44198) 2024-09-12T03:10:51.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.097+0000 7fc625d52640 1 -- 172.21.15.116:0/1539978237 learned_addr learned my addr 172.21.15.116:0/1539978237 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:51.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.097+0000 7fc626553640 1 --2- 172.21.15.116:0/1539978237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc620100480 0x7fc620198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:51.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.098+0000 7fc625d52640 1 -- 172.21.15.116:0/1539978237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc620100480 msgr2=0x7fc620198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:51.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.098+0000 7fc625d52640 1 --2- 172.21.15.116:0/1539978237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc620100480 0x7fc620198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.098+0000 7fc625d52640 1 -- 172.21.15.116:0/1539978237 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6201055b0 msgr2=0x7fc62019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:51.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.098+0000 7fc625d52640 1 --2- 172.21.15.116:0/1539978237 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6201055b0 0x7fc62019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.098+0000 7fc625d52640 1 -- 172.21.15.116:0/1539978237 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62019d3d0 con 0x7fc620100e30 2024-09-12T03:10:51.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.098+0000 7fc626553640 1 --2- 172.21.15.116:0/1539978237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc620100480 0x7fc620198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:10:51.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.098+0000 7fc625d52640 1 --2- 172.21.15.116:0/1539978237 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc620100e30 0x7fc620198960 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fc610005ab0 tx=0x7fc6100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:51.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.098+0000 7fc6177fe640 1 -- 172.21.15.116:0/1539978237 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61003f030 con 0x7fc620100e30 2024-09-12T03:10:51.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.098+0000 7fc6287b7640 1 -- 172.21.15.116:0/1539978237 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc62019d660 con 0x7fc620100e30 2024-09-12T03:10:51.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.098+0000 7fc6177fe640 1 -- 172.21.15.116:0/1539978237 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc610003ce0 con 0x7fc620100e30 2024-09-12T03:10:51.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.098+0000 7fc6177fe640 1 -- 172.21.15.116:0/1539978237 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610035070 con 0x7fc620100e30 2024-09-12T03:10:51.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.098+0000 7fc6287b7640 1 -- 172.21.15.116:0/1539978237 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6201a4fa0 con 0x7fc620100e30 2024-09-12T03:10:51.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.100+0000 7fc6177fe640 1 -- 172.21.15.116:0/1539978237 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fc610042030 con 0x7fc620100e30 2024-09-12T03:10:51.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.100+0000 7fc62ab90740 1 Processor -- start 2024-09-12T03:10:51.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.100+0000 7fc62ab90740 1 -- start start 2024-09-12T03:10:51.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.101+0000 7fc62ab90740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d6aa014a0 0x560d6aa01880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:51.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.101+0000 7fc62ab90740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d6aa01dc0 0x560d6aa0c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:51.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.101+0000 7fc62ab90740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d6aa0c680 0x560d6aa0ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:51.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.101+0000 7fc62ab90740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560d6a9672e0 con 0x560d6aa014a0 2024-09-12T03:10:51.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.101+0000 7fc62ab90740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560d6a90ece0 con 0x560d6aa01dc0 2024-09-12T03:10:51.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.101+0000 7fc62ab90740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560d6a9632a0 con 0x560d6aa0c680 2024-09-12T03:10:51.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.101+0000 7fc626d54640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d6aa014a0 0x560d6aa01880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:51.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.101+0000 7fc626d54640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d6aa014a0 0x560d6aa01880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44214/0 (socket says 172.21.15.116:44214) 2024-09-12T03:10:51.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.101+0000 7fc626d54640 1 -- 172.21.15.116:0/4015456650 learned_addr learned my addr 172.21.15.116:0/4015456650 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:51.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.101+0000 7fc625d52640 1 --2- 172.21.15.116:0/4015456650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d6aa0c680 0x560d6aa0ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:51.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.101+0000 7fc626553640 1 --2- 172.21.15.116:0/4015456650 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d6aa01dc0 0x560d6aa0c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:51.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.101+0000 7fc6177fe640 1 --2- 172.21.15.116:0/1539978237 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc60c0a2b80 0x7fc60c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:51.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.101+0000 7fc626d54640 1 -- 172.21.15.116:0/4015456650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d6aa0c680 msgr2=0x560d6aa0ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:51.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.101+0000 7fc626d54640 1 --2- 172.21.15.116:0/4015456650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d6aa0c680 0x560d6aa0ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.101+0000 7fc626d54640 1 -- 172.21.15.116:0/4015456650 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d6aa01dc0 msgr2=0x560d6aa0c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:51.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.101+0000 7fc626d54640 1 --2- 172.21.15.116:0/4015456650 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d6aa01dc0 0x560d6aa0c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.101+0000 7fc626d54640 1 -- 172.21.15.116:0/4015456650 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d6a8f3440 con 0x560d6aa014a0 2024-09-12T03:10:51.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.101+0000 7fc626553640 1 --2- 172.21.15.116:0/4015456650 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d6aa01dc0 0x560d6aa0c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:10:51.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.101+0000 7fc626d54640 1 --2- 172.21.15.116:0/1539978237 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc60c0a2b80 0x7fc60c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:51.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.101+0000 7fc6177fe640 1 -- 172.21.15.116:0/1539978237 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fc6100ef9b0 con 0x7fc620100e30 2024-09-12T03:10:51.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.102+0000 7fc626d54640 1 --2- 172.21.15.116:0/4015456650 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d6aa014a0 0x560d6aa01880 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fc618007da0 tx=0x7fc61800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a5addbe1dd2ff1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:51.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.102+0000 7fc614ff9640 1 -- 172.21.15.116:0/4015456650 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc618016070 con 0x560d6aa014a0 2024-09-12T03:10:51.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.102+0000 7fc614ff9640 1 -- 172.21.15.116:0/4015456650 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6180047d0 con 0x560d6aa014a0 2024-09-12T03:10:51.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.102+0000 7fc614ff9640 1 -- 172.21.15.116:0/4015456650 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61800f6a0 con 0x560d6aa014a0 2024-09-12T03:10:51.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.102+0000 7fc62ab90740 1 -- 172.21.15.116:0/4015456650 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d6aa014a0 msgr2=0x560d6aa01880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:51.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.102+0000 7fc62ab90740 1 --2- 172.21.15.116:0/4015456650 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d6aa014a0 0x560d6aa01880 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fc618007da0 tx=0x7fc61800c6a0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.102+0000 7fc626d54640 1 --2- 172.21.15.116:0/1539978237 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc60c0a2b80 0x7fc60c0a5040 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc61801d000 tx=0x7fc61801e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:51.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.103+0000 7fc62ab90740 1 -- 172.21.15.116:0/4015456650 shutdown_connections 2024-09-12T03:10:51.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.103+0000 7fc62ab90740 1 --2- 172.21.15.116:0/4015456650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d6aa0c680 0x560d6aa0ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.103+0000 7fc62ab90740 1 --2- 172.21.15.116:0/4015456650 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d6aa01dc0 0x560d6aa0c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.103+0000 7fc62ab90740 1 --2- 172.21.15.116:0/4015456650 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d6aa014a0 0x560d6aa01880 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.103+0000 7fc62ab90740 1 -- 172.21.15.116:0/4015456650 >> 172.21.15.116:0/4015456650 conn(0x560d6a9fda30 msgr2=0x560d6a9fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:51.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.103+0000 7fc62ab90740 1 -- 172.21.15.116:0/4015456650 shutdown_connections 2024-09-12T03:10:51.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.103+0000 7fc62ab90740 1 -- 172.21.15.116:0/4015456650 wait complete. 2024-09-12T03:10:51.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.103+0000 7fc62ab90740 1 -- start start 2024-09-12T03:10:51.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.103+0000 7fc62ab90740 1 Processor -- start 2024-09-12T03:10:51.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.103+0000 7fc62ab90740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d6aa014a0 0x560d6aa0bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:51.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.103+0000 7fc62ab90740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d6aa01dc0 0x560d6aa09250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:51.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.103+0000 7fc62ab90740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d6aa0c680 0x560d6aa09790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:51.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.103+0000 7fc62ab90740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560d6aa0f250 con 0x560d6aa014a0 2024-09-12T03:10:51.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.103+0000 7fc62ab90740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560d6a922050 con 0x560d6aa01dc0 2024-09-12T03:10:51.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.103+0000 7fc62ab90740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560d6aa0f3d0 con 0x560d6aa0c680 2024-09-12T03:10:51.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.103+0000 7fc62ab90740 10 client.0 _ll_register_callbacks cb 0x560d6a962160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:51.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.103+0000 7fc626553640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d6aa014a0 0x560d6aa0bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:51.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.103+0000 7fc626553640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d6aa014a0 0x560d6aa0bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44220/0 (socket says 172.21.15.116:44220) 2024-09-12T03:10:51.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.103+0000 7fc626553640 1 -- 172.21.15.116:0/2095121221 learned_addr learned my addr 172.21.15.116:0/2095121221 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:51.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.103+0000 7fc625d52640 1 --2- 172.21.15.116:0/2095121221 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d6aa01dc0 0x560d6aa09250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:51.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.104+0000 7fc626d54640 1 --2- 172.21.15.116:0/2095121221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d6aa0c680 0x560d6aa09790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:51.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.104+0000 7fc626553640 1 -- 172.21.15.116:0/2095121221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d6aa0c680 msgr2=0x560d6aa09790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:51.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.104+0000 7fc626553640 1 --2- 172.21.15.116:0/2095121221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d6aa0c680 0x560d6aa09790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.104+0000 7fc626553640 1 -- 172.21.15.116:0/2095121221 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d6aa01dc0 msgr2=0x560d6aa09250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:51.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.104+0000 7fc626553640 1 --2- 172.21.15.116:0/2095121221 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d6aa01dc0 0x560d6aa09250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.104+0000 7fc626553640 1 -- 172.21.15.116:0/2095121221 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d6a6d83c0 con 0x560d6aa014a0 2024-09-12T03:10:51.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.104+0000 7fc626d54640 1 --2- 172.21.15.116:0/2095121221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d6aa0c680 0x560d6aa09790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:10:51.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.104+0000 7fc626553640 1 --2- 172.21.15.116:0/2095121221 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d6aa014a0 0x560d6aa0bfa0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fc61c010840 tx=0x7fc61c010c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab077f6b4661528d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:51.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.104+0000 7fc609ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:51.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.104+0000 7fc609ffb640 1 -- 172.21.15.116:0/2095121221 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c00ece0 con 0x560d6aa014a0 2024-09-12T03:10:51.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.104+0000 7fc609ffb640 1 -- 172.21.15.116:0/2095121221 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fc61c005020 con 0x560d6aa014a0 2024-09-12T03:10:51.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.104+0000 7fc62ab90740 1 -- 172.21.15.116:0/2095121221 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6180047d0 con 0x560d6aa014a0 2024-09-12T03:10:51.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.104+0000 7fc609ffb640 1 -- 172.21.15.116:0/2095121221 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc61c022070 con 0x560d6aa014a0 2024-09-12T03:10:51.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.104+0000 7fc62ab90740 10 client.17322 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:51.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.104+0000 7fc62ab90740 1 -- 172.21.15.116:0/2095121221 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560d6aa1ac30 con 0x560d6aa014a0 2024-09-12T03:10:51.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.104+0000 7fc609ffb640 1 -- 172.21.15.116:0/2095121221 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6180047d0 con 0x560d6aa014a0 2024-09-12T03:10:51.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.105+0000 7fc609ffb640 1 -- 172.21.15.116:0/2095121221 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc61c004d00 con 0x560d6aa014a0 2024-09-12T03:10:51.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.105+0000 7fc62ab90740 10 client.17322 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:51.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.105+0000 7fc62ab90740 10 client.17322 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:10:51.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.105+0000 7fc62ab90740 20 client.17322 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:51.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.105+0000 7fc62ab90740 1 --2- 172.21.15.116:0/2095121221 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560d6aa1c0f0 0x560d6aa1e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:51.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.105+0000 7fc62ab90740 4 client.17322 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:10:51.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.105+0000 7fc62ab90740 1 -- 172.21.15.116:0/2095121221 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d6a974850 con 0x560d6aa1c0f0 2024-09-12T03:10:51.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.105+0000 7fc625d52640 1 --2- 172.21.15.116:0/2095121221 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560d6aa1c0f0 0x560d6aa1e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:51.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.105+0000 7fc625d52640 1 --2- 172.21.15.116:0/2095121221 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560d6aa1c0f0 0x560d6aa1e4e0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73368db9384ca28c server_cookie=5c0ed71a345e2230 in_seq=0 out_seq=0 2024-09-12T03:10:51.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.105+0000 7fc609ffb640 10 client.17322 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:51.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.106+0000 7fc609ffb640 1 -- 172.21.15.116:0/2095121221 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560d6a974850 con 0x560d6aa1c0f0 2024-09-12T03:10:51.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.106+0000 7fc609ffb640 10 client.17322 handle_command_reply: tid=0 2024-09-12T03:10:51.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.106+0000 7fc62ab90740 1 client.17322 shutdown 2024-09-12T03:10:51.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.106+0000 7fc62ab90740 1 -- 172.21.15.116:0/2095121221 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d6aa014a0 msgr2=0x560d6aa0bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:51.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.106+0000 7fc62ab90740 1 --2- 172.21.15.116:0/2095121221 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d6aa014a0 0x560d6aa0bfa0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fc61c010840 tx=0x7fc61c010c40 comp rx=0 tx=0).stop 2024-09-12T03:10:51.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.106+0000 7fc62ab90740 1 -- 172.21.15.116:0/2095121221 shutdown_connections 2024-09-12T03:10:51.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.106+0000 7fc62ab90740 1 --2- 172.21.15.116:0/2095121221 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560d6aa1c0f0 0x560d6aa1e4e0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.106+0000 7fc62ab90740 1 --2- 172.21.15.116:0/2095121221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d6aa0c680 0x560d6aa09790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.106+0000 7fc62ab90740 1 --2- 172.21.15.116:0/2095121221 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d6aa01dc0 0x560d6aa09250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.106+0000 7fc609ffb640 0 client.17322 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:51.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.106+0000 7fc62ab90740 1 --2- 172.21.15.116:0/2095121221 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d6aa014a0 0x560d6aa0bfa0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.106+0000 7fc62ab90740 1 -- 172.21.15.116:0/2095121221 >> 172.21.15.116:0/2095121221 conn(0x560d6a9fda30 msgr2=0x560d6a9fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:51.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.106+0000 7fc62ab90740 1 -- 172.21.15.116:0/2095121221 shutdown_connections 2024-09-12T03:10:51.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.106+0000 7fc62ab90740 1 -- 172.21.15.116:0/2095121221 wait complete. 2024-09-12T03:10:51.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.106+0000 7fc62ab90740 20 client.17322 trim_cache size 0 max 16384 2024-09-12T03:10:51.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.124+0000 7fc62ab90740 1 Processor -- start 2024-09-12T03:10:51.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.124+0000 7fc62ab90740 1 -- start start 2024-09-12T03:10:51.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.124+0000 7fc62ab90740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d6a9fda30 0x560d6a9fca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:51.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.124+0000 7fc62ab90740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d6aa014a0 0x560d6aa23d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:51.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.124+0000 7fc62ab90740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d6aa01dc0 0x560d6aa242a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:51.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.124+0000 7fc62ab90740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201afc70 con 0x560d6aa01dc0 2024-09-12T03:10:51.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.124+0000 7fc62ab90740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201ad540 con 0x560d6aa014a0 2024-09-12T03:10:51.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.124+0000 7fc62ab90740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201ab390 con 0x560d6a9fda30 2024-09-12T03:10:51.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.124+0000 7fc626d54640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d6aa01dc0 0x560d6aa242a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:51.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.124+0000 7fc626d54640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d6aa01dc0 0x560d6aa242a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44224/0 (socket says 172.21.15.116:44224) 2024-09-12T03:10:51.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.126+0000 7fc626d54640 1 -- 172.21.15.116:0/3549536868 learned_addr learned my addr 172.21.15.116:0/3549536868 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:51.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.126+0000 7fc626553640 1 --2- 172.21.15.116:0/3549536868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d6a9fda30 0x560d6a9fca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:51.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.126+0000 7fc625d52640 1 --2- 172.21.15.116:0/3549536868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d6aa014a0 0x560d6aa23d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:51.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.126+0000 7fc626d54640 1 -- 172.21.15.116:0/3549536868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d6a9fda30 msgr2=0x560d6a9fca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:51.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.126+0000 7fc626d54640 1 --2- 172.21.15.116:0/3549536868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d6a9fda30 0x560d6a9fca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.126+0000 7fc626d54640 1 -- 172.21.15.116:0/3549536868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d6aa014a0 msgr2=0x560d6aa23d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:51.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.126+0000 7fc626d54640 1 --2- 172.21.15.116:0/3549536868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d6aa014a0 0x560d6aa23d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.126+0000 7fc626d54640 1 -- 172.21.15.116:0/3549536868 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d6aa1e850 con 0x560d6aa01dc0 2024-09-12T03:10:51.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.126+0000 7fc626d54640 1 --2- 172.21.15.116:0/3549536868 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d6aa01dc0 0x560d6aa242a0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fc61800cdc0 tx=0x7fc618003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c937ec81c8452f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:51.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.126+0000 7fc60bfff640 1 -- 172.21.15.116:0/3549536868 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc618016050 con 0x560d6aa01dc0 2024-09-12T03:10:51.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.126+0000 7fc60bfff640 1 -- 172.21.15.116:0/3549536868 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc61800f640 con 0x560d6aa01dc0 2024-09-12T03:10:51.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.126+0000 7fc60bfff640 1 -- 172.21.15.116:0/3549536868 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc618011760 con 0x560d6aa01dc0 2024-09-12T03:10:51.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.126+0000 7fc62ab90740 1 -- 172.21.15.116:0/3549536868 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d6aa01dc0 msgr2=0x560d6aa242a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:51.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.126+0000 7fc62ab90740 1 --2- 172.21.15.116:0/3549536868 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d6aa01dc0 0x560d6aa242a0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fc61800cdc0 tx=0x7fc618003f60 comp rx=0 tx=0).stop 2024-09-12T03:10:51.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.126+0000 7fc62ab90740 1 -- 172.21.15.116:0/3549536868 shutdown_connections 2024-09-12T03:10:51.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.126+0000 7fc62ab90740 1 --2- 172.21.15.116:0/3549536868 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d6aa01dc0 0x560d6aa242a0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.126+0000 7fc62ab90740 1 --2- 172.21.15.116:0/3549536868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d6aa014a0 0x560d6aa23d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.126+0000 7fc62ab90740 1 --2- 172.21.15.116:0/3549536868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d6a9fda30 0x560d6a9fca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.126+0000 7fc62ab90740 1 -- 172.21.15.116:0/3549536868 >> 172.21.15.116:0/3549536868 conn(0x560d6aa098f0 msgr2=0x560d6a9fc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:51.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.126+0000 7fc62ab90740 1 -- 172.21.15.116:0/3549536868 shutdown_connections 2024-09-12T03:10:51.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.126+0000 7fc62ab90740 1 -- 172.21.15.116:0/3549536868 wait complete. 2024-09-12T03:10:51.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.126+0000 7fc62ab90740 1 -- start start 2024-09-12T03:10:51.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.126+0000 7fc62ab90740 1 Processor -- start 2024-09-12T03:10:51.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.127+0000 7fc62ab90740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d6a9fda30 0x560d6aa14ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:51.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.127+0000 7fc62ab90740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d6aa014a0 0x560d6aa11e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:51.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.127+0000 7fc62ab90740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d6aa01dc0 0x560d6aa123b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:51.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.127+0000 7fc62ab90740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201aba00 con 0x560d6a9fda30 2024-09-12T03:10:51.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.127+0000 7fc62ab90740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201ace70 con 0x560d6aa01dc0 2024-09-12T03:10:51.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.127+0000 7fc62ab90740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201ac0d0 con 0x560d6aa014a0 2024-09-12T03:10:51.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.127+0000 7fc62ab90740 10 client.0 _ll_register_callbacks cb 0x7fc61c00c410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:51.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.127+0000 7fc626553640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d6a9fda30 0x560d6aa14ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:51.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.127+0000 7fc626553640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d6a9fda30 0x560d6aa14ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44232/0 (socket says 172.21.15.116:44232) 2024-09-12T03:10:51.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.127+0000 7fc626553640 1 -- 172.21.15.116:0/2136457749 learned_addr learned my addr 172.21.15.116:0/2136457749 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:51.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.127+0000 7fc625d52640 1 --2- 172.21.15.116:0/2136457749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d6aa014a0 0x560d6aa11e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:51.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.127+0000 7fc626d54640 1 --2- 172.21.15.116:0/2136457749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d6aa01dc0 0x560d6aa123b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:51.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.127+0000 7fc626553640 1 -- 172.21.15.116:0/2136457749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d6aa014a0 msgr2=0x560d6aa11e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:51.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.127+0000 7fc626553640 1 --2- 172.21.15.116:0/2136457749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d6aa014a0 0x560d6aa11e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.127+0000 7fc626553640 1 -- 172.21.15.116:0/2136457749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d6aa01dc0 msgr2=0x560d6aa123b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:51.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.127+0000 7fc626553640 1 --2- 172.21.15.116:0/2136457749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d6aa01dc0 0x560d6aa123b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.127+0000 7fc626553640 1 -- 172.21.15.116:0/2136457749 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d6aa12920 con 0x560d6a9fda30 2024-09-12T03:10:51.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.127+0000 7fc626553640 1 --2- 172.21.15.116:0/2136457749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d6a9fda30 0x560d6aa14ad0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fc61c005cd0 tx=0x7fc61c010fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e6b407f6d39043 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:51.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.127+0000 7fc609ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:51.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.127+0000 7fc609ffb640 1 -- 172.21.15.116:0/2136457749 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c005040 con 0x560d6a9fda30 2024-09-12T03:10:51.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.128+0000 7fc609ffb640 1 -- 172.21.15.116:0/2136457749 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fc61c0051e0 con 0x560d6a9fda30 2024-09-12T03:10:51.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.128+0000 7fc62ab90740 1 -- 172.21.15.116:0/2136457749 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc61800f640 con 0x560d6a9fda30 2024-09-12T03:10:51.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.128+0000 7fc609ffb640 1 -- 172.21.15.116:0/2136457749 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc61c038070 con 0x560d6a9fda30 2024-09-12T03:10:51.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.128+0000 7fc62ab90740 10 client.17328 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:51.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.128+0000 7fc62ab90740 1 -- 172.21.15.116:0/2136457749 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560d6aa86ab0 con 0x560d6a9fda30 2024-09-12T03:10:51.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.128+0000 7fc609ffb640 1 -- 172.21.15.116:0/2136457749 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc61800f640 con 0x560d6a9fda30 2024-09-12T03:10:51.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.128+0000 7fc609ffb640 1 -- 172.21.15.116:0/2136457749 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc61c003710 con 0x560d6a9fda30 2024-09-12T03:10:51.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.128+0000 7fc62ab90740 10 client.17328 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:51.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.128+0000 7fc62ab90740 10 client.17328 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:10:51.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.128+0000 7fc62ab90740 20 client.17328 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:51.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.128+0000 7fc62ab90740 1 --2- 172.21.15.116:0/2136457749 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560d6aa0c680 0x560d6aa89270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:51.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.128+0000 7fc62ab90740 4 client.17328 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:10:51.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.128+0000 7fc62ab90740 1 -- 172.21.15.116:0/2136457749 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560d6a96ed60 con 0x560d6aa0c680 2024-09-12T03:10:51.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.130+0000 7fc625d52640 1 --2- 172.21.15.116:0/2136457749 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560d6aa0c680 0x560d6aa89270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:51.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.130+0000 7fc609ffb640 10 client.17328 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:51.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.130+0000 7fc625d52640 1 --2- 172.21.15.116:0/2136457749 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560d6aa0c680 0x560d6aa89270 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2e300973eee24eb server_cookie=fc0a37fe159f3fbc in_seq=0 out_seq=0 2024-09-12T03:10:51.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.130+0000 7fc609ffb640 1 -- 172.21.15.116:0/2136457749 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560d6a96ed60 con 0x560d6aa0c680 2024-09-12T03:10:51.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.130+0000 7fc609ffb640 10 client.17328 handle_command_reply: tid=0 2024-09-12T03:10:51.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.130+0000 7fc62ab90740 1 client.17328 shutdown 2024-09-12T03:10:51.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.130+0000 7fc62ab90740 1 -- 172.21.15.116:0/2136457749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d6a9fda30 msgr2=0x560d6aa14ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:51.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.130+0000 7fc62ab90740 1 --2- 172.21.15.116:0/2136457749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d6a9fda30 0x560d6aa14ad0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fc61c005cd0 tx=0x7fc61c010fc0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.130+0000 7fc62ab90740 1 -- 172.21.15.116:0/2136457749 shutdown_connections 2024-09-12T03:10:51.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.130+0000 7fc62ab90740 1 --2- 172.21.15.116:0/2136457749 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560d6aa0c680 0x560d6aa89270 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.130+0000 7fc62ab90740 1 --2- 172.21.15.116:0/2136457749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d6aa01dc0 0x560d6aa123b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.130+0000 7fc62ab90740 1 --2- 172.21.15.116:0/2136457749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d6aa014a0 0x560d6aa11e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.130+0000 7fc62ab90740 1 --2- 172.21.15.116:0/2136457749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d6a9fda30 0x560d6aa14ad0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.130+0000 7fc62ab90740 1 -- 172.21.15.116:0/2136457749 >> 172.21.15.116:0/2136457749 conn(0x560d6aa098f0 msgr2=0x560d6a9fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:51.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.130+0000 7fc609ffb640 0 client.17328 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:51.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.130+0000 7fc62ab90740 1 -- 172.21.15.116:0/2136457749 shutdown_connections 2024-09-12T03:10:51.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.130+0000 7fc62ab90740 1 -- 172.21.15.116:0/2136457749 wait complete. 2024-09-12T03:10:51.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.130+0000 7fc62ab90740 20 client.17328 trim_cache size 0 max 16384 2024-09-12T03:10:51.138 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:10:51.138 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:10:51.138 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:10:51.138 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:10:51.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.130+0000 7fc6287b7640 1 -- 172.21.15.116:0/1539978237 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc60c0a2b80 msgr2=0x7fc60c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:51.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.130+0000 7fc6287b7640 1 --2- 172.21.15.116:0/1539978237 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc60c0a2b80 0x7fc60c0a5040 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc61801d000 tx=0x7fc61801e000 comp rx=0 tx=0).stop 2024-09-12T03:10:51.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.130+0000 7fc6287b7640 1 -- 172.21.15.116:0/1539978237 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc620100e30 msgr2=0x7fc620198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:51.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.130+0000 7fc6287b7640 1 --2- 172.21.15.116:0/1539978237 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc620100e30 0x7fc620198960 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fc610005ab0 tx=0x7fc6100042f0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.130+0000 7fc6287b7640 1 -- 172.21.15.116:0/1539978237 shutdown_connections 2024-09-12T03:10:51.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.130+0000 7fc6287b7640 1 --2- 172.21.15.116:0/1539978237 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc60c0a2b80 0x7fc60c0a5040 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.130+0000 7fc6287b7640 1 --2- 172.21.15.116:0/1539978237 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6201055b0 0x7fc62019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.130+0000 7fc6287b7640 1 --2- 172.21.15.116:0/1539978237 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc620100e30 0x7fc620198960 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.130+0000 7fc6287b7640 1 --2- 172.21.15.116:0/1539978237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc620100480 0x7fc620198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.130+0000 7fc6287b7640 1 -- 172.21.15.116:0/1539978237 >> 172.21.15.116:0/1539978237 conn(0x7fc620077210 msgr2=0x7fc620110d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:51.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.130+0000 7fc6287b7640 1 -- 172.21.15.116:0/1539978237 shutdown_connections 2024-09-12T03:10:51.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:51.130+0000 7fc6287b7640 1 -- 172.21.15.116:0/1539978237 wait complete. 2024-09-12T03:10:51.832 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:10:51.832 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:10:52.079 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:10:52.499 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:52 smithi116 ceph-mon[31410]: pgmap v574: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:10:52.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.579+0000 7f0b4b7ce640 1 -- 172.21.15.116:0/219868359 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b44102dd0 msgr2=0x7f0b441031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:52.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.579+0000 7f0b4b7ce640 1 --2- 172.21.15.116:0/219868359 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b44102dd0 0x7f0b441031b0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c009930 tx=0x7f0b3c02a420 comp rx=0 tx=0).stop 2024-09-12T03:10:52.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.579+0000 7f0b4b7ce640 1 -- 172.21.15.116:0/219868359 shutdown_connections 2024-09-12T03:10:52.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.579+0000 7f0b4b7ce640 1 --2- 172.21.15.116:0/219868359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b44108300 0x7f0b4410ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:52.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.579+0000 7f0b4b7ce640 1 --2- 172.21.15.116:0/219868359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b44103780 0x7f0b44107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:52.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.579+0000 7f0b4b7ce640 1 --2- 172.21.15.116:0/219868359 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b44102dd0 0x7f0b441031b0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:52.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.579+0000 7f0b4b7ce640 1 -- 172.21.15.116:0/219868359 >> 172.21.15.116:0/219868359 conn(0x7f0b440fc7d0 msgr2=0x7f0b440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:52.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.580+0000 7f0b4b7ce640 1 -- 172.21.15.116:0/219868359 shutdown_connections 2024-09-12T03:10:52.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.580+0000 7f0b4b7ce640 1 -- 172.21.15.116:0/219868359 wait complete. 2024-09-12T03:10:52.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.580+0000 7f0b4b7ce640 1 Processor -- start 2024-09-12T03:10:52.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.580+0000 7f0b4b7ce640 1 -- start start 2024-09-12T03:10:52.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.580+0000 7f0b4b7ce640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b44102dd0 0x7f0b44100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:52.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.580+0000 7f0b4b7ce640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b44103780 0x7f0b44100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:52.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.580+0000 7f0b4b7ce640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b44108300 0x7f0b441014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:52.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.580+0000 7f0b4b7ce640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0b44112090 con 0x7f0b44103780 2024-09-12T03:10:52.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.580+0000 7f0b4b7ce640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0b44111f10 con 0x7f0b44108300 2024-09-12T03:10:52.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.580+0000 7f0b4b7ce640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0b44112210 con 0x7f0b44102dd0 2024-09-12T03:10:52.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.580+0000 7f0b48d69640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b44103780 0x7f0b44100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:52.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.581+0000 7f0b48d69640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b44103780 0x7f0b44100f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44240/0 (socket says 172.21.15.116:44240) 2024-09-12T03:10:52.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.581+0000 7f0b48d69640 1 -- 172.21.15.116:0/2476569589 learned_addr learned my addr 172.21.15.116:0/2476569589 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:52.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.581+0000 7f0b4956a640 1 --2- 172.21.15.116:0/2476569589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b44102dd0 0x7f0b44100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:52.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.581+0000 7f0b49d6b640 1 --2- 172.21.15.116:0/2476569589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b44108300 0x7f0b441014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:52.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.581+0000 7f0b48d69640 1 -- 172.21.15.116:0/2476569589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b44102dd0 msgr2=0x7f0b44100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:52.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.581+0000 7f0b48d69640 1 --2- 172.21.15.116:0/2476569589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b44102dd0 0x7f0b44100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:52.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.581+0000 7f0b48d69640 1 -- 172.21.15.116:0/2476569589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b44108300 msgr2=0x7f0b441014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:52.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.581+0000 7f0b48d69640 1 --2- 172.21.15.116:0/2476569589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b44108300 0x7f0b441014b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:52.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.581+0000 7f0b48d69640 1 -- 172.21.15.116:0/2476569589 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b441a6a00 con 0x7f0b44103780 2024-09-12T03:10:52.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.581+0000 7f0b48d69640 1 --2- 172.21.15.116:0/2476569589 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b44103780 0x7f0b44100f70 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f0b3800cce0 tx=0x7f0b3800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:52.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.581+0000 7f0b427fc640 1 -- 172.21.15.116:0/2476569589 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3800f450 con 0x7f0b44103780 2024-09-12T03:10:52.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.581+0000 7f0b427fc640 1 -- 172.21.15.116:0/2476569589 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b3800fad0 con 0x7f0b44103780 2024-09-12T03:10:52.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.581+0000 7f0b4b7ce640 1 -- 172.21.15.116:0/2476569589 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b441a6cf0 con 0x7f0b44103780 2024-09-12T03:10:52.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.581+0000 7f0b427fc640 1 -- 172.21.15.116:0/2476569589 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b38018cd0 con 0x7f0b44103780 2024-09-12T03:10:52.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.581+0000 7f0b4b7ce640 1 -- 172.21.15.116:0/2476569589 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b441a71d0 con 0x7f0b44103780 2024-09-12T03:10:52.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.582+0000 7f0b4b7ce640 1 -- 172.21.15.116:0/2476569589 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b4406a7b0 con 0x7f0b44103780 2024-09-12T03:10:52.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.584+0000 7f0b427fc640 1 -- 172.21.15.116:0/2476569589 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f0b3800fc80 con 0x7f0b44103780 2024-09-12T03:10:52.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.584+0000 7f0b427fc640 1 --2- 172.21.15.116:0/2476569589 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0b2c0a2b80 0x7f0b2c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:52.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.585+0000 7f0b4956a640 1 --2- 172.21.15.116:0/2476569589 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0b2c0a2b80 0x7f0b2c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:52.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.585+0000 7f0b427fc640 1 -- 172.21.15.116:0/2476569589 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f0b380cf7c0 con 0x7f0b44103780 2024-09-12T03:10:52.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.585+0000 7f0b4956a640 1 --2- 172.21.15.116:0/2476569589 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0b2c0a2b80 0x7f0b2c0a5040 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c000c00 tx=0x7f0b3c031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:52.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.588+0000 7f0b427fc640 1 -- 172.21.15.116:0/2476569589 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0b38014030 con 0x7f0b44103780 2024-09-12T03:10:52.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:52 smithi183 ceph-mon[35535]: pgmap v574: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:10:52.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:52 smithi177 ceph-mon[35305]: pgmap v574: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:10:52.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.787+0000 7f0b4b7ce640 1 -- 172.21.15.116:0/2476569589 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b441031b0 con 0x7f0b44103780 2024-09-12T03:10:52.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.787+0000 7f0b427fc640 1 -- 172.21.15.116:0/2476569589 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f0b3809c750 con 0x7f0b44103780 2024-09-12T03:10:52.788 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:10:52.788 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:10:52.788 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:10:52.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.789+0000 7f0b4b7ce640 1 -- 172.21.15.116:0/2476569589 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0b2c0a2b80 msgr2=0x7f0b2c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:52.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.789+0000 7f0b4b7ce640 1 --2- 172.21.15.116:0/2476569589 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0b2c0a2b80 0x7f0b2c0a5040 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c000c00 tx=0x7f0b3c031490 comp rx=0 tx=0).stop 2024-09-12T03:10:52.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.789+0000 7f0b4b7ce640 1 -- 172.21.15.116:0/2476569589 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b44103780 msgr2=0x7f0b44100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:52.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.789+0000 7f0b4b7ce640 1 --2- 172.21.15.116:0/2476569589 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b44103780 0x7f0b44100f70 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f0b3800cce0 tx=0x7f0b3800a620 comp rx=0 tx=0).stop 2024-09-12T03:10:52.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.789+0000 7f0b4b7ce640 1 -- 172.21.15.116:0/2476569589 shutdown_connections 2024-09-12T03:10:52.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.789+0000 7f0b4b7ce640 1 --2- 172.21.15.116:0/2476569589 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0b2c0a2b80 0x7f0b2c0a5040 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:52.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.789+0000 7f0b4b7ce640 1 --2- 172.21.15.116:0/2476569589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b44108300 0x7f0b441014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:52.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.789+0000 7f0b4b7ce640 1 --2- 172.21.15.116:0/2476569589 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b44103780 0x7f0b44100f70 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:52.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.789+0000 7f0b4b7ce640 1 --2- 172.21.15.116:0/2476569589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b44102dd0 0x7f0b44100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:52.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.789+0000 7f0b4b7ce640 1 -- 172.21.15.116:0/2476569589 >> 172.21.15.116:0/2476569589 conn(0x7f0b440fc7d0 msgr2=0x7f0b44105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:52.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.790+0000 7f0b4b7ce640 1 -- 172.21.15.116:0/2476569589 shutdown_connections 2024-09-12T03:10:52.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:52.790+0000 7f0b4b7ce640 1 -- 172.21.15.116:0/2476569589 wait complete. 2024-09-12T03:10:53.504 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:53 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2476569589' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:10:53.534 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:10:53.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:53 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2476569589' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:10:53.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:53 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2476569589' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:10:53.778 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:10:54.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.281+0000 7fe8e7131640 1 -- 172.21.15.116:0/2149853121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8e0106f10 msgr2=0x7fe8e010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:54.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.281+0000 7fe8e7131640 1 --2- 172.21.15.116:0/2149853121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8e0106f10 0x7fe8e010ea90 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8009ca0 tx=0x7fe8d802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.281+0000 7fe8e7131640 1 -- 172.21.15.116:0/2149853121 shutdown_connections 2024-09-12T03:10:54.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.281+0000 7fe8e7131640 1 --2- 172.21.15.116:0/2149853121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8e0106f10 0x7fe8e010ea90 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.281+0000 7fe8e7131640 1 --2- 172.21.15.116:0/2149853121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8e0102790 0x7fe8e01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.281+0000 7fe8e7131640 1 --2- 172.21.15.116:0/2149853121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8e0101de0 0x7fe8e01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.281+0000 7fe8e7131640 1 -- 172.21.15.116:0/2149853121 >> 172.21.15.116:0/2149853121 conn(0x7fe8e00fd8c0 msgr2=0x7fe8e00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:54.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.282+0000 7fe8e7131640 1 -- 172.21.15.116:0/2149853121 shutdown_connections 2024-09-12T03:10:54.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.282+0000 7fe8e7131640 1 -- 172.21.15.116:0/2149853121 wait complete. 2024-09-12T03:10:54.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.282+0000 7fe8e7131640 1 Processor -- start 2024-09-12T03:10:54.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.282+0000 7fe8e7131640 1 -- start start 2024-09-12T03:10:54.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.283+0000 7fe8e7131640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8e0101de0 0x7fe8e0198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:54.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.283+0000 7fe8e7131640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8e0102790 0x7fe8e0198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:54.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.283+0000 7fe8e7131640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8e0106f10 0x7fe8e019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:54.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.283+0000 7fe8e7131640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e0111f90 con 0x7fe8e0101de0 2024-09-12T03:10:54.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.283+0000 7fe8e7131640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e0111e10 con 0x7fe8e0102790 2024-09-12T03:10:54.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.283+0000 7fe8e7131640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e0112110 con 0x7fe8e0106f10 2024-09-12T03:10:54.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.283+0000 7fe8e4ecd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8e0101de0 0x7fe8e0198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:54.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.283+0000 7fe8e4ecd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8e0101de0 0x7fe8e0198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44262/0 (socket says 172.21.15.116:44262) 2024-09-12T03:10:54.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.283+0000 7fe8e4ecd640 1 -- 172.21.15.116:0/4028605237 learned_addr learned my addr 172.21.15.116:0/4028605237 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:54.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.283+0000 7fe8e56ce640 1 --2- 172.21.15.116:0/4028605237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8e0106f10 0x7fe8e019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:54.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.283+0000 7fe8d7fff640 1 --2- 172.21.15.116:0/4028605237 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8e0102790 0x7fe8e0198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:54.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.283+0000 7fe8e4ecd640 1 -- 172.21.15.116:0/4028605237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8e0106f10 msgr2=0x7fe8e019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:54.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.283+0000 7fe8e4ecd640 1 --2- 172.21.15.116:0/4028605237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8e0106f10 0x7fe8e019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.283+0000 7fe8e4ecd640 1 -- 172.21.15.116:0/4028605237 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8e0102790 msgr2=0x7fe8e0198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:54.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.283+0000 7fe8e4ecd640 1 --2- 172.21.15.116:0/4028605237 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8e0102790 0x7fe8e0198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.283+0000 7fe8e4ecd640 1 -- 172.21.15.116:0/4028605237 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e019d3c0 con 0x7fe8e0101de0 2024-09-12T03:10:54.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.283+0000 7fe8e56ce640 1 --2- 172.21.15.116:0/4028605237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8e0106f10 0x7fe8e019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:10:54.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.283+0000 7fe8d7fff640 1 --2- 172.21.15.116:0/4028605237 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8e0102790 0x7fe8e0198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:10:54.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.283+0000 7fe8e4ecd640 1 --2- 172.21.15.116:0/4028605237 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8e0101de0 0x7fe8e0198410 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00c920 tx=0x7fe8dc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:54.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.283+0000 7fe8d5ffb640 1 -- 172.21.15.116:0/4028605237 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc007c70 con 0x7fe8e0101de0 2024-09-12T03:10:54.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.283+0000 7fe8d5ffb640 1 -- 172.21.15.116:0/4028605237 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8dc007e10 con 0x7fe8e0101de0 2024-09-12T03:10:54.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.283+0000 7fe8e7131640 1 -- 172.21.15.116:0/4028605237 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e019d6b0 con 0x7fe8e0101de0 2024-09-12T03:10:54.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.283+0000 7fe8d5ffb640 1 -- 172.21.15.116:0/4028605237 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc018890 con 0x7fe8e0101de0 2024-09-12T03:10:54.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.284+0000 7fe8e7131640 1 -- 172.21.15.116:0/4028605237 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e01a4f90 con 0x7fe8e0101de0 2024-09-12T03:10:54.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.286+0000 7fe8d5ffb640 1 -- 172.21.15.116:0/4028605237 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe8dc018a70 con 0x7fe8e0101de0 2024-09-12T03:10:54.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.286+0000 7fe8e950a740 1 Processor -- start 2024-09-12T03:10:54.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.286+0000 7fe8e950a740 1 -- start start 2024-09-12T03:10:54.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.286+0000 7fe8d5ffb640 1 --2- 172.21.15.116:0/4028605237 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe8c80a2b80 0x7fe8c80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:54.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.286+0000 7fe8e950a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ac6c59b40 0x560ac6c58090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:54.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.286+0000 7fe8e950a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560ac6c585d0 0x560ac6c58a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:54.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.286+0000 7fe8e950a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ac6c63370 0x560ac6c65760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:54.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.286+0000 7fe8d5ffb640 1 -- 172.21.15.116:0/4028605237 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe8dc0cf210 con 0x7fe8e0101de0 2024-09-12T03:10:54.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.286+0000 7fe8e56ce640 1 --2- 172.21.15.116:0/4028605237 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe8c80a2b80 0x7fe8c80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:54.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.287+0000 7fe8d7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560ac6c585d0 0x560ac6c58a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:54.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.287+0000 7fe8d7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560ac6c585d0 0x560ac6c58a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44276/0 (socket says 172.21.15.116:44276) 2024-09-12T03:10:54.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.287+0000 7fe8e4ecd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ac6c59b40 0x560ac6c58090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:54.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.287+0000 7fe8d7fff640 1 -- 172.21.15.116:0/1951259567 learned_addr learned my addr 172.21.15.116:0/1951259567 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:54.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.287+0000 7fe8e950a740 1 -- 172.21.15.116:0/1951259567 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560ac6bbe2f0 con 0x560ac6c585d0 2024-09-12T03:10:54.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.287+0000 7fe8e950a740 1 -- 172.21.15.116:0/1951259567 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560ac6b65cf0 con 0x560ac6c59b40 2024-09-12T03:10:54.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.287+0000 7fe8e950a740 1 -- 172.21.15.116:0/1951259567 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560ac6bba2b0 con 0x560ac6c63370 2024-09-12T03:10:54.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.287+0000 7fe8e56ce640 1 --2- 172.21.15.116:0/1951259567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ac6c63370 0x560ac6c65760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:54.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.287+0000 7fe8d7fff640 1 -- 172.21.15.116:0/1951259567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ac6c63370 msgr2=0x560ac6c65760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:54.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.287+0000 7fe8d7fff640 1 --2- 172.21.15.116:0/1951259567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ac6c63370 0x560ac6c65760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.287+0000 7fe8d7fff640 1 -- 172.21.15.116:0/1951259567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ac6c59b40 msgr2=0x560ac6c58090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:54.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.287+0000 7fe8d7fff640 1 --2- 172.21.15.116:0/1951259567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ac6c59b40 0x560ac6c58090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.287+0000 7fe8d7fff640 1 -- 172.21.15.116:0/1951259567 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ac6b4a3a0 con 0x560ac6c585d0 2024-09-12T03:10:54.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.287+0000 7fe8e56ce640 1 --2- 172.21.15.116:0/4028605237 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe8c80a2b80 0x7fe8c80a5040 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8006010 tx=0x7fe8d8031de0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:54.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.287+0000 7fe8d7fff640 1 --2- 172.21.15.116:0/1951259567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560ac6c585d0 0x560ac6c58a10 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0007800 tx=0x7fe8d0007cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2896343218215962 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:54.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.287+0000 7fe8c6ffd640 1 -- 172.21.15.116:0/1951259567 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d000d8d0 con 0x560ac6c585d0 2024-09-12T03:10:54.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.287+0000 7fe8c6ffd640 1 -- 172.21.15.116:0/1951259567 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8d0018070 con 0x560ac6c585d0 2024-09-12T03:10:54.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.287+0000 7fe8c6ffd640 1 -- 172.21.15.116:0/1951259567 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d00123e0 con 0x560ac6c585d0 2024-09-12T03:10:54.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.288+0000 7fe8e950a740 1 -- 172.21.15.116:0/1951259567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560ac6c585d0 msgr2=0x560ac6c58a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:54.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.288+0000 7fe8e950a740 1 --2- 172.21.15.116:0/1951259567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560ac6c585d0 0x560ac6c58a10 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0007800 tx=0x7fe8d0007cc0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.288+0000 7fe8e950a740 1 -- 172.21.15.116:0/1951259567 shutdown_connections 2024-09-12T03:10:54.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.288+0000 7fe8e950a740 1 --2- 172.21.15.116:0/1951259567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ac6c63370 0x560ac6c65760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.288+0000 7fe8e950a740 1 --2- 172.21.15.116:0/1951259567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560ac6c585d0 0x560ac6c58a10 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.288+0000 7fe8e950a740 1 --2- 172.21.15.116:0/1951259567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ac6c59b40 0x560ac6c58090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.288+0000 7fe8e950a740 1 -- 172.21.15.116:0/1951259567 >> 172.21.15.116:0/1951259567 conn(0x560ac6c21310 msgr2=0x560ac6c54c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:54.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.288+0000 7fe8e950a740 1 -- 172.21.15.116:0/1951259567 shutdown_connections 2024-09-12T03:10:54.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.288+0000 7fe8e950a740 1 -- 172.21.15.116:0/1951259567 wait complete. 2024-09-12T03:10:54.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.289+0000 7fe8e950a740 1 -- start start 2024-09-12T03:10:54.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.289+0000 7fe8e950a740 1 Processor -- start 2024-09-12T03:10:54.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.289+0000 7fe8e950a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560ac6c585d0 0x560ac6c62d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:54.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.289+0000 7fe8e950a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ac6c59b40 0x560ac6c5ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:54.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.289+0000 7fe8e950a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ac6c63370 0x560ac6c60530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:54.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.289+0000 7fe8e950a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560ac6c66070 con 0x560ac6c585d0 2024-09-12T03:10:54.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.289+0000 7fe8e950a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560ac6b79060 con 0x560ac6c63370 2024-09-12T03:10:54.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.289+0000 7fe8e950a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560ac6c661f0 con 0x560ac6c59b40 2024-09-12T03:10:54.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.289+0000 7fe8e950a740 10 client.0 _ll_register_callbacks cb 0x560ac68cc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:54.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.289+0000 7fe8e4ecd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560ac6c585d0 0x560ac6c62d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:54.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.289+0000 7fe8e4ecd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560ac6c585d0 0x560ac6c62d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44284/0 (socket says 172.21.15.116:44284) 2024-09-12T03:10:54.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.289+0000 7fe8e4ecd640 1 -- 172.21.15.116:0/591726436 learned_addr learned my addr 172.21.15.116:0/591726436 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:54.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.289+0000 7fe8e4ecd640 1 -- 172.21.15.116:0/591726436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ac6c59b40 msgr2=0x560ac6c5ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:54.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.289+0000 7fe8e4ecd640 1 --2- 172.21.15.116:0/591726436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ac6c59b40 0x560ac6c5ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.289+0000 7fe8e4ecd640 1 -- 172.21.15.116:0/591726436 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ac6c63370 msgr2=0x560ac6c60530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:54.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.289+0000 7fe8e4ecd640 1 --2- 172.21.15.116:0/591726436 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ac6c63370 0x560ac6c60530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.289+0000 7fe8e4ecd640 1 -- 172.21.15.116:0/591726436 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ac692f3c0 con 0x560ac6c585d0 2024-09-12T03:10:54.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.290+0000 7fe8e4ecd640 1 --2- 172.21.15.116:0/591726436 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560ac6c585d0 0x560ac6c62d00 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0ccd40 tx=0x7fe8dc04a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d572333526230b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:54.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.290+0000 7fe8bffff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:54.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.290+0000 7fe8bffff640 1 -- 172.21.15.116:0/591726436 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc09baa0 con 0x560ac6c585d0 2024-09-12T03:10:54.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.290+0000 7fe8bffff640 1 -- 172.21.15.116:0/591726436 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe8dc00fb30 con 0x560ac6c585d0 2024-09-12T03:10:54.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.290+0000 7fe8e950a740 1 -- 172.21.15.116:0/591726436 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8d0018070 con 0x560ac6c585d0 2024-09-12T03:10:54.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.290+0000 7fe8bffff640 1 -- 172.21.15.116:0/591726436 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe8dc09fe50 con 0x560ac6c585d0 2024-09-12T03:10:54.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.290+0000 7fe8e950a740 10 client.17346 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:54.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.290+0000 7fe8e950a740 1 -- 172.21.15.116:0/591726436 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560ac6c71b00 con 0x560ac6c585d0 2024-09-12T03:10:54.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.290+0000 7fe8bffff640 1 -- 172.21.15.116:0/591726436 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8d0018070 con 0x560ac6c585d0 2024-09-12T03:10:54.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.290+0000 7fe8bffff640 1 -- 172.21.15.116:0/591726436 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe8dc09aef0 con 0x560ac6c585d0 2024-09-12T03:10:54.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.290+0000 7fe8e950a740 10 client.17346 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:54.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.290+0000 7fe8e950a740 10 client.17346 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:10:54.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.290+0000 7fe8e950a740 20 client.17346 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:54.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.290+0000 7fe8e950a740 1 --2- 172.21.15.116:0/591726436 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x560ac6c72fb0 0x560ac6c75400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:54.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.290+0000 7fe8e950a740 4 client.17346 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:10:54.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.290+0000 7fe8e950a740 1 -- 172.21.15.116:0/591726436 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ac6bcb860 con 0x560ac6c72fb0 2024-09-12T03:10:54.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.291+0000 7fe8d7fff640 1 --2- 172.21.15.116:0/591726436 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x560ac6c72fb0 0x560ac6c75400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:54.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.291+0000 7fe8d7fff640 1 --2- 172.21.15.116:0/591726436 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x560ac6c72fb0 0x560ac6c75400 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=618467ebf17538d8 server_cookie=d28d5961319be180 in_seq=0 out_seq=0 2024-09-12T03:10:54.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.291+0000 7fe8bffff640 10 client.17346 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:10:54.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.292+0000 7fe8bffff640 1 -- 172.21.15.116:0/591726436 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560ac6bcb860 con 0x560ac6c72fb0 2024-09-12T03:10:54.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.292+0000 7fe8bffff640 10 client.17346 handle_command_reply: tid=0 2024-09-12T03:10:54.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.292+0000 7fe8e950a740 1 client.17346 shutdown 2024-09-12T03:10:54.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.292+0000 7fe8e950a740 1 -- 172.21.15.116:0/591726436 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560ac6c585d0 msgr2=0x560ac6c62d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:54.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.292+0000 7fe8e950a740 1 --2- 172.21.15.116:0/591726436 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560ac6c585d0 0x560ac6c62d00 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0ccd40 tx=0x7fe8dc04a200 comp rx=0 tx=0).stop 2024-09-12T03:10:54.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.292+0000 7fe8e950a740 1 -- 172.21.15.116:0/591726436 shutdown_connections 2024-09-12T03:10:54.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.292+0000 7fe8e950a740 1 --2- 172.21.15.116:0/591726436 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ac6c63370 0x560ac6c60530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.292+0000 7fe8e950a740 1 --2- 172.21.15.116:0/591726436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ac6c59b40 0x560ac6c5ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.292+0000 7fe8e950a740 1 --2- 172.21.15.116:0/591726436 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x560ac6c72fb0 0x560ac6c75400 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.292+0000 7fe8e950a740 1 --2- 172.21.15.116:0/591726436 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560ac6c585d0 0x560ac6c62d00 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.292+0000 7fe8bffff640 0 client.17346 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:10:54.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.292+0000 7fe8e950a740 1 -- 172.21.15.116:0/591726436 >> 172.21.15.116:0/591726436 conn(0x560ac6c21310 msgr2=0x560ac6c5a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:54.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.292+0000 7fe8e950a740 1 -- 172.21.15.116:0/591726436 shutdown_connections 2024-09-12T03:10:54.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.292+0000 7fe8e950a740 1 -- 172.21.15.116:0/591726436 wait complete. 2024-09-12T03:10:54.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.292+0000 7fe8e950a740 20 client.17346 trim_cache size 0 max 16384 2024-09-12T03:10:54.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.305+0000 7fe8e950a740 1 Processor -- start 2024-09-12T03:10:54.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.305+0000 7fe8e950a740 1 -- start start 2024-09-12T03:10:54.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.305+0000 7fe8e950a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560ac6c585d0 0x560ac6c54e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:54.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.305+0000 7fe8e950a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ac6c59b40 0x560ac6c6f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:54.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.305+0000 7fe8e950a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ac6c63370 0x560ac6c7acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:54.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.305+0000 7fe8e950a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e01afc90 con 0x560ac6c585d0 2024-09-12T03:10:54.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.305+0000 7fe8e950a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e01ad560 con 0x560ac6c63370 2024-09-12T03:10:54.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.305+0000 7fe8e950a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e01ab3b0 con 0x560ac6c59b40 2024-09-12T03:10:54.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.305+0000 7fe8e4ecd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560ac6c585d0 0x560ac6c54e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:54.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.305+0000 7fe8e4ecd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560ac6c585d0 0x560ac6c54e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44300/0 (socket says 172.21.15.116:44300) 2024-09-12T03:10:54.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.305+0000 7fe8e4ecd640 1 -- 172.21.15.116:0/3847145606 learned_addr learned my addr 172.21.15.116:0/3847145606 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:54.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.306+0000 7fe8e56ce640 1 --2- 172.21.15.116:0/3847145606 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ac6c63370 0x560ac6c7acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:54.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.306+0000 7fe8d7fff640 1 --2- 172.21.15.116:0/3847145606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ac6c59b40 0x560ac6c6f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:54.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.306+0000 7fe8e4ecd640 1 -- 172.21.15.116:0/3847145606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ac6c59b40 msgr2=0x560ac6c6f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:54.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.306+0000 7fe8e4ecd640 1 --2- 172.21.15.116:0/3847145606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ac6c59b40 0x560ac6c6f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.306+0000 7fe8e4ecd640 1 -- 172.21.15.116:0/3847145606 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ac6c63370 msgr2=0x560ac6c7acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:54.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.306+0000 7fe8e4ecd640 1 --2- 172.21.15.116:0/3847145606 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ac6c63370 0x560ac6c7acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.306+0000 7fe8e4ecd640 1 -- 172.21.15.116:0/3847145606 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ac6c59760 con 0x560ac6c585d0 2024-09-12T03:10:54.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.306+0000 7fe8e4ecd640 1 --2- 172.21.15.116:0/3847145606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560ac6c585d0 0x560ac6c54e10 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc098420 tx=0x7fe8dc099fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b1d0b54135ad8c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:54.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.306+0000 7fe8c67fc640 1 -- 172.21.15.116:0/3847145606 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc0b8070 con 0x560ac6c585d0 2024-09-12T03:10:54.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.306+0000 7fe8c67fc640 1 -- 172.21.15.116:0/3847145606 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8dc0a4710 con 0x560ac6c585d0 2024-09-12T03:10:54.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.306+0000 7fe8c67fc640 1 -- 172.21.15.116:0/3847145606 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc0a44e0 con 0x560ac6c585d0 2024-09-12T03:10:54.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.307+0000 7fe8e950a740 1 -- 172.21.15.116:0/3847145606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560ac6c585d0 msgr2=0x560ac6c54e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:54.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.307+0000 7fe8e950a740 1 --2- 172.21.15.116:0/3847145606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560ac6c585d0 0x560ac6c54e10 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc098420 tx=0x7fe8dc099fb0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.307+0000 7fe8e950a740 1 -- 172.21.15.116:0/3847145606 shutdown_connections 2024-09-12T03:10:54.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.307+0000 7fe8e950a740 1 --2- 172.21.15.116:0/3847145606 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ac6c63370 0x560ac6c7acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.307+0000 7fe8e950a740 1 --2- 172.21.15.116:0/3847145606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ac6c59b40 0x560ac6c6f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.307+0000 7fe8e950a740 1 --2- 172.21.15.116:0/3847145606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560ac6c585d0 0x560ac6c54e10 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.307+0000 7fe8e950a740 1 -- 172.21.15.116:0/3847145606 >> 172.21.15.116:0/3847145606 conn(0x560ac6c21310 msgr2=0x560ac6c54700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:54.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.307+0000 7fe8e950a740 1 -- 172.21.15.116:0/3847145606 shutdown_connections 2024-09-12T03:10:54.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.307+0000 7fe8e950a740 1 -- 172.21.15.116:0/3847145606 wait complete. 2024-09-12T03:10:54.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.307+0000 7fe8e950a740 1 -- start start 2024-09-12T03:10:54.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.307+0000 7fe8e950a740 1 Processor -- start 2024-09-12T03:10:54.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.308+0000 7fe8e950a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560ac6c585d0 0x560ac6c67170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:54.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.308+0000 7fe8e950a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ac6c59b40 0x560ac6c676b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:54.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.308+0000 7fe8e950a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ac6c63370 0x560ac6c67bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:54.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.308+0000 7fe8e950a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e01aba20 con 0x560ac6c585d0 2024-09-12T03:10:54.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.308+0000 7fe8e950a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e01ace90 con 0x560ac6c59b40 2024-09-12T03:10:54.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.308+0000 7fe8e950a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e01ac0f0 con 0x560ac6c63370 2024-09-12T03:10:54.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.308+0000 7fe8e950a740 10 client.0 _ll_register_callbacks cb 0x7fe8dc00c870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:54.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.308+0000 7fe8e4ecd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560ac6c585d0 0x560ac6c67170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:54.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.308+0000 7fe8e4ecd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560ac6c585d0 0x560ac6c67170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44312/0 (socket says 172.21.15.116:44312) 2024-09-12T03:10:54.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.308+0000 7fe8e4ecd640 1 -- 172.21.15.116:0/990907854 learned_addr learned my addr 172.21.15.116:0/990907854 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:54.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.308+0000 7fe8d7fff640 1 --2- 172.21.15.116:0/990907854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ac6c59b40 0x560ac6c676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:54.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.308+0000 7fe8e56ce640 1 --2- 172.21.15.116:0/990907854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ac6c63370 0x560ac6c67bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:54.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.308+0000 7fe8e4ecd640 1 -- 172.21.15.116:0/990907854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ac6c63370 msgr2=0x560ac6c67bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:54.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.308+0000 7fe8e4ecd640 1 --2- 172.21.15.116:0/990907854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ac6c63370 0x560ac6c67bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.308+0000 7fe8e4ecd640 1 -- 172.21.15.116:0/990907854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ac6c59b40 msgr2=0x560ac6c676b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:54.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.308+0000 7fe8e4ecd640 1 --2- 172.21.15.116:0/990907854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ac6c59b40 0x560ac6c676b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.308+0000 7fe8e4ecd640 1 -- 172.21.15.116:0/990907854 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ac6cdcca0 con 0x560ac6c585d0 2024-09-12T03:10:54.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.308+0000 7fe8e4ecd640 1 --2- 172.21.15.116:0/990907854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560ac6c585d0 0x560ac6c67170 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00a670 tx=0x7fe8dc020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec5ec0e4d056ec0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:54.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.308+0000 7fe8bffff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:54.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.309+0000 7fe8bffff640 1 -- 172.21.15.116:0/990907854 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc0b8070 con 0x560ac6c585d0 2024-09-12T03:10:54.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.309+0000 7fe8bffff640 1 -- 172.21.15.116:0/990907854 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe8dc0ba2c0 con 0x560ac6c585d0 2024-09-12T03:10:54.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.309+0000 7fe8e950a740 1 -- 172.21.15.116:0/990907854 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8dc0a4710 con 0x560ac6c585d0 2024-09-12T03:10:54.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.309+0000 7fe8bffff640 1 -- 172.21.15.116:0/990907854 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe8dc065020 con 0x560ac6c585d0 2024-09-12T03:10:54.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.309+0000 7fe8e950a740 10 client.17352 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:54.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.309+0000 7fe8e950a740 1 -- 172.21.15.116:0/990907854 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560ac6cddf20 con 0x560ac6c585d0 2024-09-12T03:10:54.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.309+0000 7fe8bffff640 1 -- 172.21.15.116:0/990907854 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8dc0a4710 con 0x560ac6c585d0 2024-09-12T03:10:54.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.310+0000 7fe8bffff640 1 -- 172.21.15.116:0/990907854 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe8dc0a0fe0 con 0x560ac6c585d0 2024-09-12T03:10:54.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.310+0000 7fe8e950a740 10 client.17352 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:54.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.310+0000 7fe8e950a740 10 client.17352 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:10:54.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.310+0000 7fe8e950a740 20 client.17352 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:54.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.310+0000 7fe8e950a740 1 --2- 172.21.15.116:0/990907854 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x560ac6cde3f0 0x560ac6ce07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:54.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.310+0000 7fe8e950a740 4 client.17352 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:10:54.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.310+0000 7fe8e950a740 1 -- 172.21.15.116:0/990907854 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560ac6bc5d70 con 0x560ac6cde3f0 2024-09-12T03:10:54.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.310+0000 7fe8d7fff640 1 --2- 172.21.15.116:0/990907854 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x560ac6cde3f0 0x560ac6ce07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:54.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.310+0000 7fe8d7fff640 1 --2- 172.21.15.116:0/990907854 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x560ac6cde3f0 0x560ac6ce07e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=971ff07287487f28 server_cookie=9b622f3e28910e7f in_seq=0 out_seq=0 2024-09-12T03:10:54.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.310+0000 7fe8bffff640 10 client.17352 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:10:54.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.311+0000 7fe8bffff640 1 -- 172.21.15.116:0/990907854 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560ac6bc5d70 con 0x560ac6cde3f0 2024-09-12T03:10:54.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.311+0000 7fe8bffff640 10 client.17352 handle_command_reply: tid=0 2024-09-12T03:10:54.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.311+0000 7fe8e950a740 1 client.17352 shutdown 2024-09-12T03:10:54.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.311+0000 7fe8e950a740 1 -- 172.21.15.116:0/990907854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560ac6c585d0 msgr2=0x560ac6c67170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:54.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.311+0000 7fe8e950a740 1 --2- 172.21.15.116:0/990907854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560ac6c585d0 0x560ac6c67170 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00a670 tx=0x7fe8dc020000 comp rx=0 tx=0).stop 2024-09-12T03:10:54.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.311+0000 7fe8e950a740 1 -- 172.21.15.116:0/990907854 shutdown_connections 2024-09-12T03:10:54.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.311+0000 7fe8e950a740 1 --2- 172.21.15.116:0/990907854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ac6c63370 0x560ac6c67bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.311+0000 7fe8e950a740 1 --2- 172.21.15.116:0/990907854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ac6c59b40 0x560ac6c676b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.311+0000 7fe8e950a740 1 --2- 172.21.15.116:0/990907854 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x560ac6cde3f0 0x560ac6ce07e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.311+0000 7fe8bffff640 0 client.17352 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:10:54.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.311+0000 7fe8e950a740 1 --2- 172.21.15.116:0/990907854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560ac6c585d0 0x560ac6c67170 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.311+0000 7fe8e950a740 1 -- 172.21.15.116:0/990907854 >> 172.21.15.116:0/990907854 conn(0x560ac6c21310 msgr2=0x560ac6c5a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:54.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.311+0000 7fe8e950a740 1 -- 172.21.15.116:0/990907854 shutdown_connections 2024-09-12T03:10:54.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.311+0000 7fe8e950a740 1 -- 172.21.15.116:0/990907854 wait complete. 2024-09-12T03:10:54.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.311+0000 7fe8e950a740 20 client.17352 trim_cache size 0 max 16384 2024-09-12T03:10:54.330 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:10:54.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.311+0000 7fe8e7131640 1 -- 172.21.15.116:0/4028605237 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe8c80a2b80 msgr2=0x7fe8c80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:54.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.311+0000 7fe8e7131640 1 --2- 172.21.15.116:0/4028605237 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe8c80a2b80 0x7fe8c80a5040 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8006010 tx=0x7fe8d8031de0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.312+0000 7fe8e7131640 1 -- 172.21.15.116:0/4028605237 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8e0101de0 msgr2=0x7fe8e0198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:54.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.312+0000 7fe8e7131640 1 --2- 172.21.15.116:0/4028605237 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8e0101de0 0x7fe8e0198410 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00c920 tx=0x7fe8dc00cde0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.312+0000 7fe8e7131640 1 -- 172.21.15.116:0/4028605237 shutdown_connections 2024-09-12T03:10:54.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.312+0000 7fe8e7131640 1 --2- 172.21.15.116:0/4028605237 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe8c80a2b80 0x7fe8c80a5040 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.312+0000 7fe8e7131640 1 --2- 172.21.15.116:0/4028605237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8e0106f10 0x7fe8e019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.312+0000 7fe8e7131640 1 --2- 172.21.15.116:0/4028605237 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8e0102790 0x7fe8e0198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.312+0000 7fe8e7131640 1 --2- 172.21.15.116:0/4028605237 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8e0101de0 0x7fe8e0198410 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:54.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.312+0000 7fe8e7131640 1 -- 172.21.15.116:0/4028605237 >> 172.21.15.116:0/4028605237 conn(0x7fe8e00fd8c0 msgr2=0x7fe8e00ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:54.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.312+0000 7fe8e7131640 1 -- 172.21.15.116:0/4028605237 shutdown_connections 2024-09-12T03:10:54.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:54.312+0000 7fe8e7131640 1 -- 172.21.15.116:0/4028605237 wait complete. 2024-09-12T03:10:54.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:54 smithi116 ceph-mon[31410]: pgmap v575: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:10:54.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:54 smithi183 ceph-mon[35535]: pgmap v575: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:10:54.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:54 smithi177 ceph-mon[35305]: pgmap v575: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:10:55.039 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:10:55.288 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:10:55.701 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:55 smithi116 ceph-mon[31410]: pgmap v576: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:10:55.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:55 smithi183 ceph-mon[35535]: pgmap v576: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:10:55.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:55 smithi177 ceph-mon[35305]: pgmap v576: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:10:55.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.782+0000 7fe80f969640 1 -- 172.21.15.116:0/4200560325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe808074620 msgr2=0x7fe808074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:55.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.782+0000 7fe80f969640 1 --2- 172.21.15.116:0/4200560325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe808074620 0x7fe808074a00 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009ca0 tx=0x7fe7f802a950 comp rx=0 tx=0).stop 2024-09-12T03:10:55.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.782+0000 7fe80f969640 1 -- 172.21.15.116:0/4200560325 shutdown_connections 2024-09-12T03:10:55.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.782+0000 7fe80f969640 1 --2- 172.21.15.116:0/4200560325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe80810e3b0 0x7fe808110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.782+0000 7fe80f969640 1 --2- 172.21.15.116:0/4200560325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe808074f40 0x7fe80810de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.782+0000 7fe80f969640 1 --2- 172.21.15.116:0/4200560325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe808074620 0x7fe808074a00 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.782+0000 7fe80f969640 1 -- 172.21.15.116:0/4200560325 >> 172.21.15.116:0/4200560325 conn(0x7fe8080fd8b0 msgr2=0x7fe8080ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:55.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.782+0000 7fe80f969640 1 -- 172.21.15.116:0/4200560325 shutdown_connections 2024-09-12T03:10:55.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.782+0000 7fe80f969640 1 -- 172.21.15.116:0/4200560325 wait complete. 2024-09-12T03:10:55.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.782+0000 7fe80f969640 1 Processor -- start 2024-09-12T03:10:55.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.782+0000 7fe80f969640 1 -- start start 2024-09-12T03:10:55.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.783+0000 7fe80f969640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe808074620 0x7fe80819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:55.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.783+0000 7fe80f969640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe808074f40 0x7fe80819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:55.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.783+0000 7fe80f969640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe80810e3b0 0x7fe80819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:55.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.783+0000 7fe80f969640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe808114160 con 0x7fe808074f40 2024-09-12T03:10:55.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.783+0000 7fe80f969640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe808113fe0 con 0x7fe808074620 2024-09-12T03:10:55.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.783+0000 7fe80f969640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe8081142e0 con 0x7fe80810e3b0 2024-09-12T03:10:55.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.783+0000 7fe80cf04640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe808074f40 0x7fe80819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:55.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.783+0000 7fe80cf04640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe808074f40 0x7fe80819abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44326/0 (socket says 172.21.15.116:44326) 2024-09-12T03:10:55.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.783+0000 7fe80cf04640 1 -- 172.21.15.116:0/2585662750 learned_addr learned my addr 172.21.15.116:0/2585662750 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:55.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.783+0000 7fe80df06640 1 --2- 172.21.15.116:0/2585662750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe80810e3b0 0x7fe80819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:55.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.783+0000 7fe80d705640 1 --2- 172.21.15.116:0/2585662750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe808074620 0x7fe80819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:55.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.783+0000 7fe80cf04640 1 -- 172.21.15.116:0/2585662750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe80810e3b0 msgr2=0x7fe80819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:55.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.783+0000 7fe80cf04640 1 --2- 172.21.15.116:0/2585662750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe80810e3b0 0x7fe80819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.783+0000 7fe80cf04640 1 -- 172.21.15.116:0/2585662750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe808074620 msgr2=0x7fe80819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:55.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.783+0000 7fe80cf04640 1 --2- 172.21.15.116:0/2585662750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe808074620 0x7fe80819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.783+0000 7fe80cf04640 1 -- 172.21.15.116:0/2585662750 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80819f650 con 0x7fe808074f40 2024-09-12T03:10:55.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.783+0000 7fe80d705640 1 --2- 172.21.15.116:0/2585662750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe808074620 0x7fe80819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:10:55.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.783+0000 7fe80cf04640 1 --2- 172.21.15.116:0/2585662750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe808074f40 0x7fe80819abe0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc00d950 tx=0x7fe7fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:55.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.783+0000 7fe8067fc640 1 -- 172.21.15.116:0/2585662750 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc017070 con 0x7fe808074f40 2024-09-12T03:10:55.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.783+0000 7fe8067fc640 1 -- 172.21.15.116:0/2585662750 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7fc009c30 con 0x7fe808074f40 2024-09-12T03:10:55.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.783+0000 7fe80f969640 1 -- 172.21.15.116:0/2585662750 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe80819f940 con 0x7fe808074f40 2024-09-12T03:10:55.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.783+0000 7fe8067fc640 1 -- 172.21.15.116:0/2585662750 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc00f6b0 con 0x7fe808074f40 2024-09-12T03:10:55.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.783+0000 7fe80f969640 1 -- 172.21.15.116:0/2585662750 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8081a7220 con 0x7fe808074f40 2024-09-12T03:10:55.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.786+0000 7fe8067fc640 1 -- 172.21.15.116:0/2585662750 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe7fc0109d0 con 0x7fe808074f40 2024-09-12T03:10:55.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.786+0000 7fe811d42740 1 Processor -- start 2024-09-12T03:10:55.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.786+0000 7fe811d42740 1 -- start start 2024-09-12T03:10:55.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.786+0000 7fe811d42740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e2a317bb40 0x55e2a317a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:55.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.786+0000 7fe811d42740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a317a5d0 0x55e2a317aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:55.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.787+0000 7fe811d42740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e2a3185370 0x55e2a3187760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:55.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.787+0000 7fe811d42740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e2a30e02f0 con 0x55e2a3185370 2024-09-12T03:10:55.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.787+0000 7fe811d42740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e2a3087cf0 con 0x55e2a317bb40 2024-09-12T03:10:55.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.787+0000 7fe811d42740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e2a30dc2b0 con 0x55e2a317a5d0 2024-09-12T03:10:55.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.787+0000 7fe8067fc640 1 --2- 172.21.15.116:0/2585662750 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe7f00a2b80 0x7fe7f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:55.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.787+0000 7fe80cf04640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e2a3185370 0x55e2a3187760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:55.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.787+0000 7fe80cf04640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e2a3185370 0x55e2a3187760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44334/0 (socket says 172.21.15.116:44334) 2024-09-12T03:10:55.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.787+0000 7fe80cf04640 1 -- 172.21.15.116:0/4003357007 learned_addr learned my addr 172.21.15.116:0/4003357007 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:55.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.787+0000 7fe80df06640 1 --2- 172.21.15.116:0/2585662750 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe7f00a2b80 0x7fe7f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:55.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.787+0000 7fe80df06640 1 --2- 172.21.15.116:0/4003357007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e2a317bb40 0x55e2a317a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:55.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.787+0000 7fe80d705640 1 --2- 172.21.15.116:0/4003357007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a317a5d0 0x55e2a317aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:55.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.787+0000 7fe8067fc640 1 -- 172.21.15.116:0/2585662750 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe7fc0ceca0 con 0x7fe808074f40 2024-09-12T03:10:55.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.787+0000 7fe80cf04640 1 -- 172.21.15.116:0/4003357007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a317a5d0 msgr2=0x55e2a317aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:55.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.787+0000 7fe80cf04640 1 --2- 172.21.15.116:0/4003357007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a317a5d0 0x55e2a317aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.787+0000 7fe80cf04640 1 -- 172.21.15.116:0/4003357007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e2a317bb40 msgr2=0x55e2a317a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:55.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.787+0000 7fe80cf04640 1 --2- 172.21.15.116:0/4003357007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e2a317bb40 0x55e2a317a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.787+0000 7fe80cf04640 1 -- 172.21.15.116:0/4003357007 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2a306c3a0 con 0x55e2a3185370 2024-09-12T03:10:55.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.787+0000 7fe80df06640 1 --2- 172.21.15.116:0/2585662750 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe7f00a2b80 0x7fe7f00a5040 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe80000b590 tx=0x7fe80000ba00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:55.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.787+0000 7fe80cf04640 1 --2- 172.21.15.116:0/4003357007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e2a3185370 0x55e2a3187760 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc0a3c70 tx=0x7fe7fc00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4568f13cc66482 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:55.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.788+0000 7fe7ef7fe640 1 -- 172.21.15.116:0/4003357007 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc0bb070 con 0x55e2a3185370 2024-09-12T03:10:55.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.788+0000 7fe7ef7fe640 1 -- 172.21.15.116:0/4003357007 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7fc0a5e20 con 0x55e2a3185370 2024-09-12T03:10:55.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.788+0000 7fe7ef7fe640 1 -- 172.21.15.116:0/4003357007 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc020070 con 0x55e2a3185370 2024-09-12T03:10:55.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.788+0000 7fe811d42740 1 -- 172.21.15.116:0/4003357007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e2a3185370 msgr2=0x55e2a3187760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:55.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.788+0000 7fe811d42740 1 --2- 172.21.15.116:0/4003357007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e2a3185370 0x55e2a3187760 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc0a3c70 tx=0x7fe7fc00bf60 comp rx=0 tx=0).stop 2024-09-12T03:10:55.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.788+0000 7fe811d42740 1 -- 172.21.15.116:0/4003357007 shutdown_connections 2024-09-12T03:10:55.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.788+0000 7fe811d42740 1 --2- 172.21.15.116:0/4003357007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e2a3185370 0x55e2a3187760 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.788+0000 7fe811d42740 1 --2- 172.21.15.116:0/4003357007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a317a5d0 0x55e2a317aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.788+0000 7fe811d42740 1 --2- 172.21.15.116:0/4003357007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e2a317bb40 0x55e2a317a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.788+0000 7fe811d42740 1 -- 172.21.15.116:0/4003357007 >> 172.21.15.116:0/4003357007 conn(0x55e2a3143310 msgr2=0x55e2a3176c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:55.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.788+0000 7fe811d42740 1 -- 172.21.15.116:0/4003357007 shutdown_connections 2024-09-12T03:10:55.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.788+0000 7fe811d42740 1 -- 172.21.15.116:0/4003357007 wait complete. 2024-09-12T03:10:55.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.789+0000 7fe811d42740 1 -- start start 2024-09-12T03:10:55.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.789+0000 7fe811d42740 1 Processor -- start 2024-09-12T03:10:55.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.789+0000 7fe811d42740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a317a5d0 0x55e2a3184d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:55.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.789+0000 7fe811d42740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e2a317bb40 0x55e2a3182020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:55.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.789+0000 7fe811d42740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e2a3185370 0x55e2a3182610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:55.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.789+0000 7fe811d42740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e2a3188010 con 0x55e2a317bb40 2024-09-12T03:10:55.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.789+0000 7fe811d42740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e2a309b060 con 0x55e2a3185370 2024-09-12T03:10:55.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.789+0000 7fe811d42740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e2a3188190 con 0x55e2a317a5d0 2024-09-12T03:10:55.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.789+0000 7fe811d42740 10 client.0 _ll_register_callbacks cb 0x55e2a2dee8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:55.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.789+0000 7fe80cf04640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e2a317bb40 0x55e2a3182020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:55.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.789+0000 7fe80cf04640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e2a317bb40 0x55e2a3182020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44350/0 (socket says 172.21.15.116:44350) 2024-09-12T03:10:55.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.789+0000 7fe80cf04640 1 -- 172.21.15.116:0/147564405 learned_addr learned my addr 172.21.15.116:0/147564405 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:55.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.789+0000 7fe80d705640 1 --2- 172.21.15.116:0/147564405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a317a5d0 0x55e2a3184d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:55.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.789+0000 7fe80df06640 1 --2- 172.21.15.116:0/147564405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e2a3185370 0x55e2a3182610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:55.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.790+0000 7fe80cf04640 1 -- 172.21.15.116:0/147564405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a317a5d0 msgr2=0x55e2a3184d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:55.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.790+0000 7fe80cf04640 1 --2- 172.21.15.116:0/147564405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a317a5d0 0x55e2a3184d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.790+0000 7fe80cf04640 1 -- 172.21.15.116:0/147564405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e2a3185370 msgr2=0x55e2a3182610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:55.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.790+0000 7fe80cf04640 1 --2- 172.21.15.116:0/147564405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e2a3185370 0x55e2a3182610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.790+0000 7fe80cf04640 1 -- 172.21.15.116:0/147564405 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2a2e513c0 con 0x55e2a317bb40 2024-09-12T03:10:55.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.790+0000 7fe80d705640 1 --2- 172.21.15.116:0/147564405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a317a5d0 0x55e2a3184d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:10:55.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.790+0000 7fe80cf04640 1 --2- 172.21.15.116:0/147564405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e2a317bb40 0x55e2a3182020 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc09b4c0 tx=0x7fe7fc010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a0eea2d8a47a29 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:55.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.790+0000 7fe7ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:55.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.790+0000 7fe7ecff9640 1 -- 172.21.15.116:0/147564405 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc020070 con 0x55e2a317bb40 2024-09-12T03:10:55.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.790+0000 7fe7ecff9640 1 -- 172.21.15.116:0/147564405 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe7fc0a5b00 con 0x55e2a317bb40 2024-09-12T03:10:55.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.790+0000 7fe811d42740 1 -- 172.21.15.116:0/147564405 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7fc0a5e20 con 0x55e2a317bb40 2024-09-12T03:10:55.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.790+0000 7fe7ecff9640 1 -- 172.21.15.116:0/147564405 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe7fc0bb040 con 0x55e2a317bb40 2024-09-12T03:10:55.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.790+0000 7fe811d42740 10 client.17364 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:55.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.790+0000 7fe811d42740 1 -- 172.21.15.116:0/147564405 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e2a3188010 con 0x55e2a317bb40 2024-09-12T03:10:55.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.790+0000 7fe7ecff9640 1 -- 172.21.15.116:0/147564405 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7fc09bc30 con 0x55e2a317bb40 2024-09-12T03:10:55.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.791+0000 7fe7ecff9640 1 -- 172.21.15.116:0/147564405 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe7fc0a3d10 con 0x55e2a317bb40 2024-09-12T03:10:55.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.791+0000 7fe811d42740 10 client.17364 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:55.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.791+0000 7fe811d42740 10 client.17364 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:10:55.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.791+0000 7fe811d42740 20 client.17364 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:55.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.791+0000 7fe811d42740 1 --2- 172.21.15.116:0/147564405 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55e2a3194c30 0x55e2a3197020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:55.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.791+0000 7fe811d42740 4 client.17364 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:10:55.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.791+0000 7fe811d42740 1 -- 172.21.15.116:0/147564405 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe7fc0a5e20 con 0x55e2a3194c30 2024-09-12T03:10:55.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.791+0000 7fe80d705640 1 --2- 172.21.15.116:0/147564405 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55e2a3194c30 0x55e2a3197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:55.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.791+0000 7fe7ecff9640 10 client.17364 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:10:55.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.791+0000 7fe80d705640 1 --2- 172.21.15.116:0/147564405 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55e2a3194c30 0x55e2a3197020 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8262262ee9eb2639 server_cookie=25cee4654cae5e46 in_seq=0 out_seq=0 2024-09-12T03:10:55.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.792+0000 7fe7ecff9640 1 -- 172.21.15.116:0/147564405 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fe7fc0a5e20 con 0x55e2a3194c30 2024-09-12T03:10:55.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.792+0000 7fe7ecff9640 10 client.17364 handle_command_reply: tid=0 2024-09-12T03:10:55.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.792+0000 7fe811d42740 1 client.17364 shutdown 2024-09-12T03:10:55.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.792+0000 7fe811d42740 1 -- 172.21.15.116:0/147564405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e2a317bb40 msgr2=0x55e2a3182020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:55.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.792+0000 7fe811d42740 1 --2- 172.21.15.116:0/147564405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e2a317bb40 0x55e2a3182020 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc09b4c0 tx=0x7fe7fc010d70 comp rx=0 tx=0).stop 2024-09-12T03:10:55.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.792+0000 7fe811d42740 1 -- 172.21.15.116:0/147564405 shutdown_connections 2024-09-12T03:10:55.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.792+0000 7fe811d42740 1 --2- 172.21.15.116:0/147564405 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55e2a3194c30 0x55e2a3197020 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.792+0000 7fe7ecff9640 0 client.17364 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:10:55.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.792+0000 7fe811d42740 1 --2- 172.21.15.116:0/147564405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e2a3185370 0x55e2a3182610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.792+0000 7fe811d42740 1 --2- 172.21.15.116:0/147564405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e2a317bb40 0x55e2a3182020 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.792+0000 7fe811d42740 1 --2- 172.21.15.116:0/147564405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a317a5d0 0x55e2a3184d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.792+0000 7fe811d42740 1 -- 172.21.15.116:0/147564405 >> 172.21.15.116:0/147564405 conn(0x55e2a3143310 msgr2=0x55e2a317c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:55.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.792+0000 7fe811d42740 1 -- 172.21.15.116:0/147564405 shutdown_connections 2024-09-12T03:10:55.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.793+0000 7fe811d42740 1 -- 172.21.15.116:0/147564405 wait complete. 2024-09-12T03:10:55.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.793+0000 7fe811d42740 20 client.17364 trim_cache size 0 max 16384 2024-09-12T03:10:55.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.805+0000 7fe811d42740 1 Processor -- start 2024-09-12T03:10:55.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.805+0000 7fe811d42740 1 -- start start 2024-09-12T03:10:55.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.806+0000 7fe811d42740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a317a5d0 0x55e2a31912b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:55.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.806+0000 7fe811d42740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e2a317bb40 0x55e2a31818e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:55.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.806+0000 7fe811d42740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e2a3185370 0x55e2a317d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:55.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.806+0000 7fe811d42740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe8081b2050 con 0x55e2a317bb40 2024-09-12T03:10:55.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.806+0000 7fe811d42740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe8081af920 con 0x55e2a3185370 2024-09-12T03:10:55.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.806+0000 7fe811d42740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe8081ad770 con 0x55e2a317a5d0 2024-09-12T03:10:55.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.806+0000 7fe80cf04640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e2a317bb40 0x55e2a31818e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:55.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.806+0000 7fe80cf04640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e2a317bb40 0x55e2a31818e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44366/0 (socket says 172.21.15.116:44366) 2024-09-12T03:10:55.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.806+0000 7fe80cf04640 1 -- 172.21.15.116:0/2590013168 learned_addr learned my addr 172.21.15.116:0/2590013168 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:55.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.806+0000 7fe80d705640 1 --2- 172.21.15.116:0/2590013168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a317a5d0 0x55e2a31912b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:55.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.806+0000 7fe80cf04640 1 -- 172.21.15.116:0/2590013168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a317a5d0 msgr2=0x55e2a31912b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:55.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.806+0000 7fe80cf04640 1 --2- 172.21.15.116:0/2590013168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a317a5d0 0x55e2a31912b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.806+0000 7fe80cf04640 1 -- 172.21.15.116:0/2590013168 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e2a3185370 msgr2=0x55e2a317d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:55.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.806+0000 7fe80cf04640 1 --2- 172.21.15.116:0/2590013168 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e2a3185370 0x55e2a317d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.806+0000 7fe80cf04640 1 -- 172.21.15.116:0/2590013168 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2a3182980 con 0x55e2a317bb40 2024-09-12T03:10:55.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.806+0000 7fe80cf04640 1 --2- 172.21.15.116:0/2590013168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e2a317bb40 0x55e2a31818e0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc0ccb40 tx=0x7fe7fc099a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934f399d1f646107 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:55.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.806+0000 7fe7eeffd640 1 -- 172.21.15.116:0/2590013168 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc020070 con 0x55e2a317bb40 2024-09-12T03:10:55.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.806+0000 7fe7eeffd640 1 -- 172.21.15.116:0/2590013168 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7fc09bc30 con 0x55e2a317bb40 2024-09-12T03:10:55.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.806+0000 7fe7eeffd640 1 -- 172.21.15.116:0/2590013168 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc0a3d10 con 0x55e2a317bb40 2024-09-12T03:10:55.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.807+0000 7fe811d42740 1 -- 172.21.15.116:0/2590013168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e2a317bb40 msgr2=0x55e2a31818e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:55.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.807+0000 7fe811d42740 1 --2- 172.21.15.116:0/2590013168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e2a317bb40 0x55e2a31818e0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc0ccb40 tx=0x7fe7fc099a40 comp rx=0 tx=0).stop 2024-09-12T03:10:55.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.807+0000 7fe811d42740 1 -- 172.21.15.116:0/2590013168 shutdown_connections 2024-09-12T03:10:55.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.807+0000 7fe811d42740 1 --2- 172.21.15.116:0/2590013168 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e2a3185370 0x55e2a317d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.807+0000 7fe811d42740 1 --2- 172.21.15.116:0/2590013168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e2a317bb40 0x55e2a31818e0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.807+0000 7fe811d42740 1 --2- 172.21.15.116:0/2590013168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a317a5d0 0x55e2a31912b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.807+0000 7fe811d42740 1 -- 172.21.15.116:0/2590013168 >> 172.21.15.116:0/2590013168 conn(0x55e2a3143310 msgr2=0x55e2a3181110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:55.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.807+0000 7fe811d42740 1 -- 172.21.15.116:0/2590013168 shutdown_connections 2024-09-12T03:10:55.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.807+0000 7fe811d42740 1 -- 172.21.15.116:0/2590013168 wait complete. 2024-09-12T03:10:55.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.807+0000 7fe811d42740 1 -- start start 2024-09-12T03:10:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.807+0000 7fe811d42740 1 Processor -- start 2024-09-12T03:10:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.808+0000 7fe811d42740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e2a317a5d0 0x55e2a318db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.808+0000 7fe811d42740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a317bb40 0x55e2a3188e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.808+0000 7fe811d42740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e2a3185370 0x55e2a3189370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.808+0000 7fe811d42740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe8081adde0 con 0x55e2a3185370 2024-09-12T03:10:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.808+0000 7fe811d42740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe8081af250 con 0x55e2a317a5d0 2024-09-12T03:10:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.808+0000 7fe811d42740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe8081ae4b0 con 0x55e2a317bb40 2024-09-12T03:10:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.808+0000 7fe811d42740 10 client.0 _ll_register_callbacks cb 0x7fe8000123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.808+0000 7fe80df06640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e2a3185370 0x55e2a3189370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.808+0000 7fe80df06640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e2a3185370 0x55e2a3189370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44378/0 (socket says 172.21.15.116:44378) 2024-09-12T03:10:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.808+0000 7fe80df06640 1 -- 172.21.15.116:0/1108232381 learned_addr learned my addr 172.21.15.116:0/1108232381 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.808+0000 7fe80cf04640 1 --2- 172.21.15.116:0/1108232381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a317bb40 0x55e2a3188e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.808+0000 7fe80d705640 1 --2- 172.21.15.116:0/1108232381 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e2a317a5d0 0x55e2a318db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:55.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.808+0000 7fe80df06640 1 -- 172.21.15.116:0/1108232381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a317bb40 msgr2=0x55e2a3188e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:55.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.808+0000 7fe80df06640 1 --2- 172.21.15.116:0/1108232381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a317bb40 0x55e2a3188e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.808+0000 7fe80df06640 1 -- 172.21.15.116:0/1108232381 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e2a317a5d0 msgr2=0x55e2a318db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:55.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.808+0000 7fe80df06640 1 --2- 172.21.15.116:0/1108232381 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e2a317a5d0 0x55e2a318db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.808+0000 7fe80df06640 1 -- 172.21.15.116:0/1108232381 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2a3189ba0 con 0x55e2a3185370 2024-09-12T03:10:55.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.808+0000 7fe80cf04640 1 --2- 172.21.15.116:0/1108232381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a317bb40 0x55e2a3188e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:10:55.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.808+0000 7fe80df06640 1 --2- 172.21.15.116:0/1108232381 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e2a3185370 0x55e2a3189370 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fe80000b830 tx=0x7fe800004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562747a301047d16 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:55.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.808+0000 7fe7ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:55.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.808+0000 7fe7ecff9640 1 -- 172.21.15.116:0/1108232381 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80001eb30 con 0x55e2a3185370 2024-09-12T03:10:55.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.809+0000 7fe7ecff9640 1 -- 172.21.15.116:0/1108232381 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe80001ecd0 con 0x55e2a3185370 2024-09-12T03:10:55.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.809+0000 7fe811d42740 1 -- 172.21.15.116:0/1108232381 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7fc09bc30 con 0x55e2a3185370 2024-09-12T03:10:55.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.809+0000 7fe7ecff9640 1 -- 172.21.15.116:0/1108232381 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe800020400 con 0x55e2a3185370 2024-09-12T03:10:55.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.809+0000 7fe811d42740 10 client.17370 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:55.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.809+0000 7fe811d42740 1 -- 172.21.15.116:0/1108232381 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e2a31ff610 con 0x55e2a3185370 2024-09-12T03:10:55.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.809+0000 7fe7ecff9640 1 -- 172.21.15.116:0/1108232381 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7fc09bc30 con 0x55e2a3185370 2024-09-12T03:10:55.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.809+0000 7fe7ecff9640 1 -- 172.21.15.116:0/1108232381 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe80001f070 con 0x55e2a3185370 2024-09-12T03:10:55.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.809+0000 7fe811d42740 10 client.17370 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:55.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.809+0000 7fe811d42740 10 client.17370 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:10:55.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.809+0000 7fe811d42740 20 client.17370 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:55.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.809+0000 7fe811d42740 1 --2- 172.21.15.116:0/1108232381 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55e2a3194c30 0x55e2a3201da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:55.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.809+0000 7fe811d42740 4 client.17370 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:10:55.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.809+0000 7fe811d42740 1 -- 172.21.15.116:0/1108232381 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e2a30ed860 con 0x55e2a3194c30 2024-09-12T03:10:55.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.809+0000 7fe80d705640 1 --2- 172.21.15.116:0/1108232381 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55e2a3194c30 0x55e2a3201da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:55.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.810+0000 7fe80d705640 1 --2- 172.21.15.116:0/1108232381 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55e2a3194c30 0x55e2a3201da0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9ff6e707159a0e4c server_cookie=2f4f8e99e2f8ba58 in_seq=0 out_seq=0 2024-09-12T03:10:55.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.810+0000 7fe7ecff9640 10 client.17370 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:10:55.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.810+0000 7fe7ecff9640 1 -- 172.21.15.116:0/1108232381 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e2a30ed860 con 0x55e2a3194c30 2024-09-12T03:10:55.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.810+0000 7fe7ecff9640 10 client.17370 handle_command_reply: tid=0 2024-09-12T03:10:55.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.810+0000 7fe811d42740 1 client.17370 shutdown 2024-09-12T03:10:55.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.810+0000 7fe811d42740 1 -- 172.21.15.116:0/1108232381 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e2a3185370 msgr2=0x55e2a3189370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:55.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.810+0000 7fe811d42740 1 --2- 172.21.15.116:0/1108232381 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e2a3185370 0x55e2a3189370 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fe80000b830 tx=0x7fe800004730 comp rx=0 tx=0).stop 2024-09-12T03:10:55.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.810+0000 7fe811d42740 1 -- 172.21.15.116:0/1108232381 shutdown_connections 2024-09-12T03:10:55.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.810+0000 7fe811d42740 1 --2- 172.21.15.116:0/1108232381 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55e2a3194c30 0x55e2a3201da0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.810+0000 7fe811d42740 1 --2- 172.21.15.116:0/1108232381 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e2a3185370 0x55e2a3189370 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.810+0000 7fe811d42740 1 --2- 172.21.15.116:0/1108232381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a317bb40 0x55e2a3188e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.810+0000 7fe811d42740 1 --2- 172.21.15.116:0/1108232381 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e2a317a5d0 0x55e2a318db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.810+0000 7fe811d42740 1 -- 172.21.15.116:0/1108232381 >> 172.21.15.116:0/1108232381 conn(0x55e2a3143310 msgr2=0x55e2a31754e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:55.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.810+0000 7fe7ecff9640 0 client.17370 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:10:55.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.810+0000 7fe811d42740 1 -- 172.21.15.116:0/1108232381 shutdown_connections 2024-09-12T03:10:55.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.810+0000 7fe811d42740 1 -- 172.21.15.116:0/1108232381 wait complete. 2024-09-12T03:10:55.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.810+0000 7fe811d42740 20 client.17370 trim_cache size 0 max 16384 2024-09-12T03:10:55.827 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:10:55.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.811+0000 7fe80f969640 1 -- 172.21.15.116:0/2585662750 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe7f00a2b80 msgr2=0x7fe7f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:55.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.811+0000 7fe80f969640 1 --2- 172.21.15.116:0/2585662750 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe7f00a2b80 0x7fe7f00a5040 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe80000b590 tx=0x7fe80000ba00 comp rx=0 tx=0).stop 2024-09-12T03:10:55.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.811+0000 7fe80f969640 1 -- 172.21.15.116:0/2585662750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe808074f40 msgr2=0x7fe80819abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:55.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.811+0000 7fe80f969640 1 --2- 172.21.15.116:0/2585662750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe808074f40 0x7fe80819abe0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc00d950 tx=0x7fe7fc00de10 comp rx=0 tx=0).stop 2024-09-12T03:10:55.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.811+0000 7fe80f969640 1 -- 172.21.15.116:0/2585662750 shutdown_connections 2024-09-12T03:10:55.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.811+0000 7fe80f969640 1 --2- 172.21.15.116:0/2585662750 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe7f00a2b80 0x7fe7f00a5040 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.811+0000 7fe80f969640 1 --2- 172.21.15.116:0/2585662750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe80810e3b0 0x7fe80819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.811+0000 7fe80f969640 1 --2- 172.21.15.116:0/2585662750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe808074f40 0x7fe80819abe0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.811+0000 7fe80f969640 1 --2- 172.21.15.116:0/2585662750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe808074620 0x7fe80819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:55.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.811+0000 7fe80f969640 1 -- 172.21.15.116:0/2585662750 >> 172.21.15.116:0/2585662750 conn(0x7fe8080fd8b0 msgr2=0x7fe8080ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:55.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.811+0000 7fe80f969640 1 -- 172.21.15.116:0/2585662750 shutdown_connections 2024-09-12T03:10:55.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:55.811+0000 7fe80f969640 1 -- 172.21.15.116:0/2585662750 wait complete. 2024-09-12T03:10:56.544 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:10:56.790 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:10:57.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.257+0000 7f1edb98a640 1 -- 172.21.15.116:0/1858326905 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ed4106a80 msgr2=0x7f1ed4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:57.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.257+0000 7f1edb98a640 1 --2- 172.21.15.116:0/1858326905 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ed4106a80 0x7f1ed4106e60 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8009930 tx=0x7f1ec802a420 comp rx=0 tx=0).stop 2024-09-12T03:10:57.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.257+0000 7f1edb98a640 1 -- 172.21.15.116:0/1858326905 shutdown_connections 2024-09-12T03:10:57.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.257+0000 7f1edb98a640 1 --2- 172.21.15.116:0/1858326905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ed4101420 0x7f1ed410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.257+0000 7f1edb98a640 1 --2- 172.21.15.116:0/1858326905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ed4100a80 0x7f1ed4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.257+0000 7f1edb98a640 1 --2- 172.21.15.116:0/1858326905 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ed4106a80 0x7f1ed4106e60 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.257+0000 7f1edb98a640 1 -- 172.21.15.116:0/1858326905 >> 172.21.15.116:0/1858326905 conn(0x7f1ed40fc7a0 msgr2=0x7f1ed40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:57.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.257+0000 7f1edb98a640 1 -- 172.21.15.116:0/1858326905 shutdown_connections 2024-09-12T03:10:57.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.257+0000 7f1edb98a640 1 -- 172.21.15.116:0/1858326905 wait complete. 2024-09-12T03:10:57.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.257+0000 7f1edb98a640 1 Processor -- start 2024-09-12T03:10:57.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.257+0000 7f1edb98a640 1 -- start start 2024-09-12T03:10:57.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.258+0000 7f1edb98a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ed4100a80 0x7f1ed4197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:57.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.258+0000 7f1edb98a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ed4101420 0x7f1ed41975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:57.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.258+0000 7f1edb98a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ed4106a80 0x7f1ed4197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:57.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.258+0000 7f1edb98a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed4110e40 con 0x7f1ed4106a80 2024-09-12T03:10:57.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.258+0000 7f1edb98a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed4110cc0 con 0x7f1ed4101420 2024-09-12T03:10:57.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.258+0000 7f1edb98a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed4110fc0 con 0x7f1ed4100a80 2024-09-12T03:10:57.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.258+0000 7f1ed9f27640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ed4106a80 0x7f1ed4197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:57.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.258+0000 7f1ed9f27640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ed4106a80 0x7f1ed4197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44398/0 (socket says 172.21.15.116:44398) 2024-09-12T03:10:57.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.258+0000 7f1ed9f27640 1 -- 172.21.15.116:0/1409148665 learned_addr learned my addr 172.21.15.116:0/1409148665 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:57.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.258+0000 7f1ed8f25640 1 --2- 172.21.15.116:0/1409148665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ed4101420 0x7f1ed41975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:57.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.258+0000 7f1ed9726640 1 --2- 172.21.15.116:0/1409148665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ed4100a80 0x7f1ed4197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:57.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.258+0000 7f1ed9f27640 1 -- 172.21.15.116:0/1409148665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ed4100a80 msgr2=0x7f1ed4197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:57.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.258+0000 7f1ed9f27640 1 --2- 172.21.15.116:0/1409148665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ed4100a80 0x7f1ed4197080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.258+0000 7f1ed9f27640 1 -- 172.21.15.116:0/1409148665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ed4101420 msgr2=0x7f1ed41975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:57.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.258+0000 7f1ed9f27640 1 --2- 172.21.15.116:0/1409148665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ed4101420 0x7f1ed41975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.258+0000 7f1ed9f27640 1 -- 172.21.15.116:0/1409148665 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed41a68f0 con 0x7f1ed4106a80 2024-09-12T03:10:57.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.258+0000 7f1ed9f27640 1 --2- 172.21.15.116:0/1409148665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ed4106a80 0x7f1ed4197b00 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc00eaa0 tx=0x7f1ecc00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:57.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.258+0000 7f1ec67fc640 1 -- 172.21.15.116:0/1409148665 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc004d20 con 0x7f1ed4106a80 2024-09-12T03:10:57.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.259+0000 7f1edb98a640 1 -- 172.21.15.116:0/1409148665 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ed41a6be0 con 0x7f1ed4106a80 2024-09-12T03:10:57.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.259+0000 7f1ec67fc640 1 -- 172.21.15.116:0/1409148665 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ecc005020 con 0x7f1ed4106a80 2024-09-12T03:10:57.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.259+0000 7f1ec67fc640 1 -- 172.21.15.116:0/1409148665 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc0116d0 con 0x7f1ed4106a80 2024-09-12T03:10:57.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.259+0000 7f1edb98a640 1 -- 172.21.15.116:0/1409148665 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ed41a7120 con 0x7f1ed4106a80 2024-09-12T03:10:57.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.261+0000 7f1ec67fc640 1 -- 172.21.15.116:0/1409148665 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1ecc0051d0 con 0x7f1ed4106a80 2024-09-12T03:10:57.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.261+0000 7f1eddd63740 1 Processor -- start 2024-09-12T03:10:57.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.261+0000 7f1eddd63740 1 -- start start 2024-09-12T03:10:57.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.261+0000 7f1eddd63740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ca07cd0b40 0x55ca07ccf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:57.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.261+0000 7f1eddd63740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca07ccf5d0 0x55ca07ccfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:57.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.261+0000 7f1eddd63740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca07cda370 0x55ca07cdc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:57.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.261+0000 7f1eddd63740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ca07c352f0 con 0x55ca07cd0b40 2024-09-12T03:10:57.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.261+0000 7f1eddd63740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ca07bdccf0 con 0x55ca07cda370 2024-09-12T03:10:57.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.261+0000 7f1eddd63740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ca07c312b0 con 0x55ca07ccf5d0 2024-09-12T03:10:57.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.262+0000 7f1ed8f25640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ca07cd0b40 0x55ca07ccf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:57.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.262+0000 7f1ed8f25640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ca07cd0b40 0x55ca07ccf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44404/0 (socket says 172.21.15.116:44404) 2024-09-12T03:10:57.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.262+0000 7f1ed8f25640 1 -- 172.21.15.116:0/3962802614 learned_addr learned my addr 172.21.15.116:0/3962802614 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:57.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.262+0000 7f1ec67fc640 1 --2- 172.21.15.116:0/1409148665 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1ec00a2b80 0x7f1ec00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:57.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.262+0000 7f1ed8f25640 1 --2- 172.21.15.116:0/3962802614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca07cda370 0x55ca07cdc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:57.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.262+0000 7f1ed9726640 1 --2- 172.21.15.116:0/3962802614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca07ccf5d0 0x55ca07ccfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:57.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.262+0000 7f1ed9f27640 1 --2- 172.21.15.116:0/1409148665 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1ec00a2b80 0x7f1ec00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:57.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.262+0000 7f1ed8f25640 1 -- 172.21.15.116:0/3962802614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca07ccf5d0 msgr2=0x55ca07ccfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:57.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.262+0000 7f1ed8f25640 1 --2- 172.21.15.116:0/3962802614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca07ccf5d0 0x55ca07ccfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.262+0000 7f1ed8f25640 1 -- 172.21.15.116:0/3962802614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca07cda370 msgr2=0x55ca07cdc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:57.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.262+0000 7f1ed8f25640 1 --2- 172.21.15.116:0/3962802614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca07cda370 0x55ca07cdc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.262+0000 7f1ed8f25640 1 -- 172.21.15.116:0/3962802614 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca07bc13a0 con 0x55ca07cd0b40 2024-09-12T03:10:57.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.262+0000 7f1ed9726640 1 --2- 172.21.15.116:0/3962802614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca07ccf5d0 0x55ca07ccfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:57.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.262+0000 7f1ec67fc640 1 -- 172.21.15.116:0/1409148665 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1ecc0cf1d0 con 0x7f1ed4106a80 2024-09-12T03:10:57.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.262+0000 7f1ed8f25640 1 --2- 172.21.15.116:0/3962802614 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ca07cd0b40 0x55ca07ccf090 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f1ed00135e0 tx=0x7f1ed000a800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175547cb62643ce9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:57.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.262+0000 7f1ebf7fe640 1 -- 172.21.15.116:0/3962802614 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0019030 con 0x55ca07cd0b40 2024-09-12T03:10:57.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.262+0000 7f1ebf7fe640 1 -- 172.21.15.116:0/3962802614 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ed00110d0 con 0x55ca07cd0b40 2024-09-12T03:10:57.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.262+0000 7f1ebf7fe640 1 -- 172.21.15.116:0/3962802614 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed000f850 con 0x55ca07cd0b40 2024-09-12T03:10:57.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.262+0000 7f1ed9f27640 1 --2- 172.21.15.116:0/1409148665 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1ec00a2b80 0x7f1ec00a5040 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc097ec0 tx=0x7f1ecc06a9e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:57.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.263+0000 7f1eddd63740 1 -- 172.21.15.116:0/3962802614 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ca07cd0b40 msgr2=0x55ca07ccf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:57.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.263+0000 7f1eddd63740 1 --2- 172.21.15.116:0/3962802614 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ca07cd0b40 0x55ca07ccf090 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f1ed00135e0 tx=0x7f1ed000a800 comp rx=0 tx=0).stop 2024-09-12T03:10:57.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.263+0000 7f1eddd63740 1 -- 172.21.15.116:0/3962802614 shutdown_connections 2024-09-12T03:10:57.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.263+0000 7f1eddd63740 1 --2- 172.21.15.116:0/3962802614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca07cda370 0x55ca07cdc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.263+0000 7f1eddd63740 1 --2- 172.21.15.116:0/3962802614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca07ccf5d0 0x55ca07ccfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.263+0000 7f1eddd63740 1 --2- 172.21.15.116:0/3962802614 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ca07cd0b40 0x55ca07ccf090 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.263+0000 7f1eddd63740 1 -- 172.21.15.116:0/3962802614 >> 172.21.15.116:0/3962802614 conn(0x55ca07c98310 msgr2=0x55ca07ccbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:57.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.263+0000 7f1eddd63740 1 -- 172.21.15.116:0/3962802614 shutdown_connections 2024-09-12T03:10:57.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.263+0000 7f1eddd63740 1 -- 172.21.15.116:0/3962802614 wait complete. 2024-09-12T03:10:57.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.264+0000 7f1eddd63740 1 -- start start 2024-09-12T03:10:57.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.264+0000 7f1eddd63740 1 Processor -- start 2024-09-12T03:10:57.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.264+0000 7f1eddd63740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca07ccf5d0 0x55ca07cd9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:57.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.264+0000 7f1eddd63740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca07cd0b40 0x55ca07cd6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:57.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.264+0000 7f1eddd63740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ca07cda370 0x55ca07cd7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:57.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.264+0000 7f1eddd63740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ca07cdd070 con 0x55ca07cda370 2024-09-12T03:10:57.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.264+0000 7f1eddd63740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ca07bf0060 con 0x55ca07ccf5d0 2024-09-12T03:10:57.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.264+0000 7f1eddd63740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ca07cdd1f0 con 0x55ca07cd0b40 2024-09-12T03:10:57.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.264+0000 7f1eddd63740 10 client.0 _ll_register_callbacks cb 0x55ca079438c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:57.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.264+0000 7f1ed8f25640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ca07cda370 0x55ca07cd7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:57.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.264+0000 7f1ed8f25640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ca07cda370 0x55ca07cd7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44406/0 (socket says 172.21.15.116:44406) 2024-09-12T03:10:57.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.264+0000 7f1ed8f25640 1 -- 172.21.15.116:0/4132550114 learned_addr learned my addr 172.21.15.116:0/4132550114 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:57.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.264+0000 7f1ed8f25640 1 --2- 172.21.15.116:0/4132550114 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca07ccf5d0 0x55ca07cd9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:57.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.264+0000 7f1ed9726640 1 --2- 172.21.15.116:0/4132550114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca07cd0b40 0x55ca07cd6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:57.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.265+0000 7f1ed8f25640 1 -- 172.21.15.116:0/4132550114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca07cd0b40 msgr2=0x55ca07cd6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:57.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.265+0000 7f1ed8f25640 1 --2- 172.21.15.116:0/4132550114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca07cd0b40 0x55ca07cd6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.265+0000 7f1ed8f25640 1 -- 172.21.15.116:0/4132550114 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca07ccf5d0 msgr2=0x55ca07cd9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:57.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.265+0000 7f1ed8f25640 1 --2- 172.21.15.116:0/4132550114 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca07ccf5d0 0x55ca07cd9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.265+0000 7f1ed8f25640 1 -- 172.21.15.116:0/4132550114 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca079a63c0 con 0x55ca07cda370 2024-09-12T03:10:57.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.265+0000 7f1ed8f25640 1 --2- 172.21.15.116:0/4132550114 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ca07cda370 0x55ca07cd7530 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0011f50 tx=0x7f1ed0015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dbe8513d72c766a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:57.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.265+0000 7f1ebcff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:57.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.265+0000 7f1ebcff9640 1 -- 172.21.15.116:0/4132550114 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0019070 con 0x55ca07cda370 2024-09-12T03:10:57.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.265+0000 7f1eddd63740 1 -- 172.21.15.116:0/4132550114 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ed00110d0 con 0x55ca07cda370 2024-09-12T03:10:57.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.265+0000 7f1ebcff9640 1 -- 172.21.15.116:0/4132550114 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1ed000f7e0 con 0x55ca07cda370 2024-09-12T03:10:57.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.265+0000 7f1ebcff9640 1 -- 172.21.15.116:0/4132550114 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1ed0032020 con 0x55ca07cda370 2024-09-12T03:10:57.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.265+0000 7f1eddd63740 10 client.17382 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:57.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.265+0000 7f1eddd63740 1 -- 172.21.15.116:0/4132550114 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ca07ce8a10 con 0x55ca07cda370 2024-09-12T03:10:57.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.265+0000 7f1ebcff9640 1 -- 172.21.15.116:0/4132550114 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ed00110d0 con 0x55ca07cda370 2024-09-12T03:10:57.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.265+0000 7f1ebcff9640 1 -- 172.21.15.116:0/4132550114 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1ed001e8f0 con 0x55ca07cda370 2024-09-12T03:10:57.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.266+0000 7f1eddd63740 10 client.17382 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:57.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.266+0000 7f1eddd63740 10 client.17382 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:10:57.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.266+0000 7f1eddd63740 20 client.17382 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:57.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.266+0000 7f1eddd63740 1 --2- 172.21.15.116:0/4132550114 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ca07ce9dc0 0x55ca07cec1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:57.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.266+0000 7f1eddd63740 4 client.17382 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:10:57.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.266+0000 7f1eddd63740 1 -- 172.21.15.116:0/4132550114 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca07c42860 con 0x55ca07ce9dc0 2024-09-12T03:10:57.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.266+0000 7f1ed9726640 1 --2- 172.21.15.116:0/4132550114 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ca07ce9dc0 0x55ca07cec1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:57.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.266+0000 7f1ebcff9640 10 client.17382 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:57.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.266+0000 7f1ed9726640 1 --2- 172.21.15.116:0/4132550114 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ca07ce9dc0 0x55ca07cec1b0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8b6542c91cadb34 server_cookie=b99805ce70eff6f in_seq=0 out_seq=0 2024-09-12T03:10:57.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.267+0000 7f1ebcff9640 1 -- 172.21.15.116:0/4132550114 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ca07c42860 con 0x55ca07ce9dc0 2024-09-12T03:10:57.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.267+0000 7f1ebcff9640 10 client.17382 handle_command_reply: tid=0 2024-09-12T03:10:57.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.267+0000 7f1eddd63740 1 client.17382 shutdown 2024-09-12T03:10:57.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.267+0000 7f1eddd63740 1 -- 172.21.15.116:0/4132550114 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ca07cda370 msgr2=0x55ca07cd7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:57.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.267+0000 7f1eddd63740 1 --2- 172.21.15.116:0/4132550114 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ca07cda370 0x55ca07cd7530 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0011f50 tx=0x7f1ed0015040 comp rx=0 tx=0).stop 2024-09-12T03:10:57.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.267+0000 7f1eddd63740 1 -- 172.21.15.116:0/4132550114 shutdown_connections 2024-09-12T03:10:57.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.267+0000 7f1eddd63740 1 --2- 172.21.15.116:0/4132550114 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ca07ce9dc0 0x55ca07cec1b0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.267+0000 7f1eddd63740 1 --2- 172.21.15.116:0/4132550114 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ca07cda370 0x55ca07cd7530 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.267+0000 7f1eddd63740 1 --2- 172.21.15.116:0/4132550114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca07cd0b40 0x55ca07cd6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.267+0000 7f1ebcff9640 0 client.17382 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:57.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.267+0000 7f1eddd63740 1 --2- 172.21.15.116:0/4132550114 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca07ccf5d0 0x55ca07cd9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.267+0000 7f1eddd63740 1 -- 172.21.15.116:0/4132550114 >> 172.21.15.116:0/4132550114 conn(0x55ca07c98310 msgr2=0x55ca07cd1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:57.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.267+0000 7f1eddd63740 1 -- 172.21.15.116:0/4132550114 shutdown_connections 2024-09-12T03:10:57.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.267+0000 7f1eddd63740 1 -- 172.21.15.116:0/4132550114 wait complete. 2024-09-12T03:10:57.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.267+0000 7f1eddd63740 20 client.17382 trim_cache size 0 max 16384 2024-09-12T03:10:57.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.280+0000 7f1eddd63740 1 Processor -- start 2024-09-12T03:10:57.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.280+0000 7f1eddd63740 1 -- start start 2024-09-12T03:10:57.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.280+0000 7f1eddd63740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca07ccf5d0 0x55ca07ce6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:57.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.280+0000 7f1eddd63740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca07cd0b40 0x55ca07cd5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:57.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.280+0000 7f1eddd63740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ca07cd2120 0x55ca07cd2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:57.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.280+0000 7f1eddd63740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed41b1fe0 con 0x55ca07cd2120 2024-09-12T03:10:57.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.280+0000 7f1eddd63740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed41af8b0 con 0x55ca07ccf5d0 2024-09-12T03:10:57.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.280+0000 7f1eddd63740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed41ad700 con 0x55ca07cd0b40 2024-09-12T03:10:57.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.281+0000 7f1ed8f25640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ca07cd2120 0x55ca07cd2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:57.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.281+0000 7f1ed8f25640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ca07cd2120 0x55ca07cd2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44414/0 (socket says 172.21.15.116:44414) 2024-09-12T03:10:57.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.281+0000 7f1ed8f25640 1 -- 172.21.15.116:0/3957136705 learned_addr learned my addr 172.21.15.116:0/3957136705 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:57.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.281+0000 7f1ed8f25640 1 --2- 172.21.15.116:0/3957136705 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca07ccf5d0 0x55ca07ce6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:57.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.281+0000 7f1ed9726640 1 --2- 172.21.15.116:0/3957136705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca07cd0b40 0x55ca07cd5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:57.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.281+0000 7f1ed8f25640 1 -- 172.21.15.116:0/3957136705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca07cd0b40 msgr2=0x55ca07cd5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:57.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.281+0000 7f1ed8f25640 1 --2- 172.21.15.116:0/3957136705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca07cd0b40 0x55ca07cd5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.281+0000 7f1ed8f25640 1 -- 172.21.15.116:0/3957136705 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca07ccf5d0 msgr2=0x55ca07ce6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:57.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.281+0000 7f1ed8f25640 1 --2- 172.21.15.116:0/3957136705 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca07ccf5d0 0x55ca07ce6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.281+0000 7f1ed8f25640 1 -- 172.21.15.116:0/3957136705 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca07cd2ac0 con 0x55ca07cd2120 2024-09-12T03:10:57.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.281+0000 7f1ed8f25640 1 --2- 172.21.15.116:0/3957136705 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ca07cd2120 0x55ca07cd2580 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f1ed001ecc0 tx=0x7f1ed000af50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96944e6f6b2eb26 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:57.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.281+0000 7f1ebeffd640 1 -- 172.21.15.116:0/3957136705 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0019070 con 0x55ca07cd2120 2024-09-12T03:10:57.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.281+0000 7f1ebeffd640 1 -- 172.21.15.116:0/3957136705 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ed00110d0 con 0x55ca07cd2120 2024-09-12T03:10:57.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.281+0000 7f1ebeffd640 1 -- 172.21.15.116:0/3957136705 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed002e3a0 con 0x55ca07cd2120 2024-09-12T03:10:57.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.282+0000 7f1eddd63740 1 -- 172.21.15.116:0/3957136705 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ca07cd2120 msgr2=0x55ca07cd2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:57.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.282+0000 7f1eddd63740 1 --2- 172.21.15.116:0/3957136705 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ca07cd2120 0x55ca07cd2580 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f1ed001ecc0 tx=0x7f1ed000af50 comp rx=0 tx=0).stop 2024-09-12T03:10:57.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.282+0000 7f1eddd63740 1 -- 172.21.15.116:0/3957136705 shutdown_connections 2024-09-12T03:10:57.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.282+0000 7f1eddd63740 1 --2- 172.21.15.116:0/3957136705 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ca07cd2120 0x55ca07cd2580 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.282+0000 7f1eddd63740 1 --2- 172.21.15.116:0/3957136705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca07cd0b40 0x55ca07cd5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.282+0000 7f1eddd63740 1 --2- 172.21.15.116:0/3957136705 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca07ccf5d0 0x55ca07ce6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.282+0000 7f1eddd63740 1 -- 172.21.15.116:0/3957136705 >> 172.21.15.116:0/3957136705 conn(0x55ca07c98310 msgr2=0x55ca07cd5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:57.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.282+0000 7f1eddd63740 1 -- 172.21.15.116:0/3957136705 shutdown_connections 2024-09-12T03:10:57.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.282+0000 7f1eddd63740 1 -- 172.21.15.116:0/3957136705 wait complete. 2024-09-12T03:10:57.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.282+0000 7f1eddd63740 1 -- start start 2024-09-12T03:10:57.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.282+0000 7f1eddd63740 1 Processor -- start 2024-09-12T03:10:57.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.283+0000 7f1eddd63740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ca07ccf5d0 0x55ca07ce2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:57.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.283+0000 7f1eddd63740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca07cd0b40 0x55ca07cde070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:57.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.283+0000 7f1eddd63740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca07cd2120 0x55ca07cde800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:57.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.283+0000 7f1eddd63740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed41add70 con 0x55ca07ccf5d0 2024-09-12T03:10:57.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.283+0000 7f1eddd63740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed41af1e0 con 0x55ca07cd2120 2024-09-12T03:10:57.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.283+0000 7f1eddd63740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed41ae440 con 0x55ca07cd0b40 2024-09-12T03:10:57.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.283+0000 7f1eddd63740 10 client.0 _ll_register_callbacks cb 0x7f1ed0013460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:57.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.283+0000 7f1ed8f25640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ca07ccf5d0 0x55ca07ce2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:57.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.283+0000 7f1ed8f25640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ca07ccf5d0 0x55ca07ce2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44428/0 (socket says 172.21.15.116:44428) 2024-09-12T03:10:57.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.283+0000 7f1ed8f25640 1 -- 172.21.15.116:0/1325507059 learned_addr learned my addr 172.21.15.116:0/1325507059 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:57.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.283+0000 7f1ed9726640 1 --2- 172.21.15.116:0/1325507059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca07cd0b40 0x55ca07cde070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:57.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.283+0000 7f1ed8f25640 1 --2- 172.21.15.116:0/1325507059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca07cd2120 0x55ca07cde800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:57.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.283+0000 7f1ed8f25640 1 -- 172.21.15.116:0/1325507059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca07cd0b40 msgr2=0x55ca07cde070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:57.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.283+0000 7f1ed8f25640 1 --2- 172.21.15.116:0/1325507059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca07cd0b40 0x55ca07cde070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.283+0000 7f1ed8f25640 1 -- 172.21.15.116:0/1325507059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca07cd2120 msgr2=0x55ca07cde800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:57.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.283+0000 7f1ed8f25640 1 --2- 172.21.15.116:0/1325507059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca07cd2120 0x55ca07cde800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.283+0000 7f1ed8f25640 1 -- 172.21.15.116:0/1325507059 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca07d538a0 con 0x55ca07ccf5d0 2024-09-12T03:10:57.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.283+0000 7f1ed8f25640 1 --2- 172.21.15.116:0/1325507059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ca07ccf5d0 0x55ca07ce2d40 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0011d10 tx=0x7f1ed0011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f908ea7c8c954f12 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:57.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.283+0000 7f1ebcff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:57.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.284+0000 7f1ebcff9640 1 -- 172.21.15.116:0/1325507059 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0019030 con 0x55ca07ccf5d0 2024-09-12T03:10:57.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.284+0000 7f1eddd63740 1 -- 172.21.15.116:0/1325507059 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ed00110d0 con 0x55ca07ccf5d0 2024-09-12T03:10:57.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.284+0000 7f1ebcff9640 1 -- 172.21.15.116:0/1325507059 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1ed0005040 con 0x55ca07ccf5d0 2024-09-12T03:10:57.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.284+0000 7f1ebcff9640 1 -- 172.21.15.116:0/1325507059 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1ed0005240 con 0x55ca07ccf5d0 2024-09-12T03:10:57.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.284+0000 7f1eddd63740 10 client.17388 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:57.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.284+0000 7f1eddd63740 1 -- 172.21.15.116:0/1325507059 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ca07d54a20 con 0x55ca07ccf5d0 2024-09-12T03:10:57.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.284+0000 7f1ebcff9640 1 -- 172.21.15.116:0/1325507059 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ed00110d0 con 0x55ca07ccf5d0 2024-09-12T03:10:57.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.284+0000 7f1ebcff9640 1 -- 172.21.15.116:0/1325507059 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1ed002ea30 con 0x55ca07ccf5d0 2024-09-12T03:10:57.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.284+0000 7f1eddd63740 10 client.17388 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:57.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.284+0000 7f1eddd63740 10 client.17388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:10:57.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.284+0000 7f1eddd63740 20 client.17388 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:57.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.284+0000 7f1eddd63740 1 --2- 172.21.15.116:0/1325507059 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ca07d55080 0x55ca07d574a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:57.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.284+0000 7f1eddd63740 4 client.17388 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:10:57.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.284+0000 7f1eddd63740 1 -- 172.21.15.116:0/1325507059 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ca07c3cd70 con 0x55ca07d55080 2024-09-12T03:10:57.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.284+0000 7f1ed9726640 1 --2- 172.21.15.116:0/1325507059 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ca07d55080 0x55ca07d574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:57.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.285+0000 7f1ebcff9640 10 client.17388 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:57.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.285+0000 7f1ed9726640 1 --2- 172.21.15.116:0/1325507059 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ca07d55080 0x55ca07d574a0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b644b3c3d7dc0acf server_cookie=92f6e6a2536634c4 in_seq=0 out_seq=0 2024-09-12T03:10:57.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.285+0000 7f1ebcff9640 1 -- 172.21.15.116:0/1325507059 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ca07c3cd70 con 0x55ca07d55080 2024-09-12T03:10:57.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.285+0000 7f1ebcff9640 10 client.17388 handle_command_reply: tid=0 2024-09-12T03:10:57.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.285+0000 7f1eddd63740 1 client.17388 shutdown 2024-09-12T03:10:57.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.285+0000 7f1eddd63740 1 -- 172.21.15.116:0/1325507059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ca07ccf5d0 msgr2=0x55ca07ce2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:57.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.285+0000 7f1eddd63740 1 --2- 172.21.15.116:0/1325507059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ca07ccf5d0 0x55ca07ce2d40 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0011d10 tx=0x7f1ed0011e80 comp rx=0 tx=0).stop 2024-09-12T03:10:57.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.285+0000 7f1eddd63740 1 -- 172.21.15.116:0/1325507059 shutdown_connections 2024-09-12T03:10:57.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.285+0000 7f1eddd63740 1 --2- 172.21.15.116:0/1325507059 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ca07d55080 0x55ca07d574a0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.285+0000 7f1eddd63740 1 --2- 172.21.15.116:0/1325507059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ca07cd2120 0x55ca07cde800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.285+0000 7f1eddd63740 1 --2- 172.21.15.116:0/1325507059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca07cd0b40 0x55ca07cde070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.285+0000 7f1eddd63740 1 --2- 172.21.15.116:0/1325507059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ca07ccf5d0 0x55ca07ce2d40 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.285+0000 7f1ebcff9640 0 client.17388 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:57.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.285+0000 7f1eddd63740 1 -- 172.21.15.116:0/1325507059 >> 172.21.15.116:0/1325507059 conn(0x55ca07c98310 msgr2=0x55ca07cd60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:57.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.285+0000 7f1eddd63740 1 -- 172.21.15.116:0/1325507059 shutdown_connections 2024-09-12T03:10:57.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.285+0000 7f1eddd63740 1 -- 172.21.15.116:0/1325507059 wait complete. 2024-09-12T03:10:57.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.286+0000 7f1eddd63740 20 client.17388 trim_cache size 0 max 16384 2024-09-12T03:10:57.297 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:10:57.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.286+0000 7f1edb98a640 1 -- 172.21.15.116:0/1409148665 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1ec00a2b80 msgr2=0x7f1ec00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:57.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.286+0000 7f1edb98a640 1 --2- 172.21.15.116:0/1409148665 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1ec00a2b80 0x7f1ec00a5040 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc097ec0 tx=0x7f1ecc06a9e0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.286+0000 7f1edb98a640 1 -- 172.21.15.116:0/1409148665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ed4106a80 msgr2=0x7f1ed4197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:57.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.286+0000 7f1edb98a640 1 --2- 172.21.15.116:0/1409148665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ed4106a80 0x7f1ed4197b00 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc00eaa0 tx=0x7f1ecc00eea0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.286+0000 7f1edb98a640 1 -- 172.21.15.116:0/1409148665 shutdown_connections 2024-09-12T03:10:57.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.286+0000 7f1edb98a640 1 --2- 172.21.15.116:0/1409148665 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1ec00a2b80 0x7f1ec00a5040 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.286+0000 7f1edb98a640 1 --2- 172.21.15.116:0/1409148665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ed4106a80 0x7f1ed4197b00 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.286+0000 7f1edb98a640 1 --2- 172.21.15.116:0/1409148665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ed4101420 0x7f1ed41975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.286+0000 7f1edb98a640 1 --2- 172.21.15.116:0/1409148665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ed4100a80 0x7f1ed4197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:57.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.286+0000 7f1edb98a640 1 -- 172.21.15.116:0/1409148665 >> 172.21.15.116:0/1409148665 conn(0x7f1ed40fc7a0 msgr2=0x7f1ed40fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:57.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.286+0000 7f1edb98a640 1 -- 172.21.15.116:0/1409148665 shutdown_connections 2024-09-12T03:10:57.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:57.286+0000 7f1edb98a640 1 -- 172.21.15.116:0/1409148665 wait complete. 2024-09-12T03:10:58.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:10:58 smithi183 ceph-mon[35535]: pgmap v577: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:10:58.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:10:58 smithi177 ceph-mon[35305]: pgmap v577: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:10:58.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:10:58 smithi116 ceph-mon[31410]: pgmap v577: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:10:59.067 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:10:59.067 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:10:59.326 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:10:59.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.835+0000 7f5dbc882640 1 -- 172.21.15.116:0/2605987494 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5db8103660 msgr2=0x7f5db810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:59.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.835+0000 7f5dbc882640 1 --2- 172.21.15.116:0/2605987494 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5db8103660 0x7f5db810bb50 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f5da0009900 tx=0x7f5da002a480 comp rx=0 tx=0).stop 2024-09-12T03:10:59.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.835+0000 7f5dbc882640 1 -- 172.21.15.116:0/2605987494 shutdown_connections 2024-09-12T03:10:59.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.835+0000 7f5dbc882640 1 --2- 172.21.15.116:0/2605987494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5db810c170 0x7f5db810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.835+0000 7f5dbc882640 1 --2- 172.21.15.116:0/2605987494 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5db8103660 0x7f5db810bb50 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.835+0000 7f5dbc882640 1 --2- 172.21.15.116:0/2605987494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5db8102d40 0x7f5db8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.835+0000 7f5dbc882640 1 -- 172.21.15.116:0/2605987494 >> 172.21.15.116:0/2605987494 conn(0x7f5db80fc7a0 msgr2=0x7f5db80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:59.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.835+0000 7f5dbc882640 1 -- 172.21.15.116:0/2605987494 shutdown_connections 2024-09-12T03:10:59.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.835+0000 7f5dbc882640 1 -- 172.21.15.116:0/2605987494 wait complete. 2024-09-12T03:10:59.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.836+0000 7f5dbc882640 1 Processor -- start 2024-09-12T03:10:59.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.836+0000 7f5dbc882640 1 -- start start 2024-09-12T03:10:59.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.836+0000 7f5dbc882640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5db8102d40 0x7f5db81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:59.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.836+0000 7f5dbc882640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5db8103660 0x7f5db8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:59.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.836+0000 7f5dbc882640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5db810c170 0x7f5db819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:59.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.836+0000 7f5dbc882640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5db8111f70 con 0x7f5db8103660 2024-09-12T03:10:59.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.836+0000 7f5dbc882640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5db8111df0 con 0x7f5db8102d40 2024-09-12T03:10:59.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.836+0000 7f5dbc882640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5db81120f0 con 0x7f5db810c170 2024-09-12T03:10:59.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.836+0000 7f5db5d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5db8103660 0x7f5db8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:59.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.836+0000 7f5db5d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5db8103660 0x7f5db8198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44444/0 (socket says 172.21.15.116:44444) 2024-09-12T03:10:59.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.836+0000 7f5db5d9b640 1 -- 172.21.15.116:0/1135775275 learned_addr learned my addr 172.21.15.116:0/1135775275 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:59.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.837+0000 7f5db6d9d640 1 --2- 172.21.15.116:0/1135775275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5db810c170 0x7f5db819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:59.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.837+0000 7f5db659c640 1 --2- 172.21.15.116:0/1135775275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5db8102d40 0x7f5db81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:59.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.837+0000 7f5db5d9b640 1 -- 172.21.15.116:0/1135775275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5db810c170 msgr2=0x7f5db819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:59.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.837+0000 7f5db5d9b640 1 --2- 172.21.15.116:0/1135775275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5db810c170 0x7f5db819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.837+0000 7f5db5d9b640 1 -- 172.21.15.116:0/1135775275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5db8102d40 msgr2=0x7f5db81983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:59.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.837+0000 7f5db5d9b640 1 --2- 172.21.15.116:0/1135775275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5db8102d40 0x7f5db81983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.837+0000 7f5db5d9b640 1 -- 172.21.15.116:0/1135775275 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db819d390 con 0x7f5db8103660 2024-09-12T03:10:59.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.837+0000 7f5db659c640 1 --2- 172.21.15.116:0/1135775275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5db8102d40 0x7f5db81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:59.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.837+0000 7f5db5d9b640 1 --2- 172.21.15.116:0/1135775275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5db8103660 0x7f5db8198920 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f5da0006010 tx=0x7f5da00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:59.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.837+0000 7f5daf7fe640 1 -- 172.21.15.116:0/1135775275 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da003f030 con 0x7f5db8103660 2024-09-12T03:10:59.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.837+0000 7f5daf7fe640 1 -- 172.21.15.116:0/1135775275 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5da0003ce0 con 0x7f5db8103660 2024-09-12T03:10:59.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.837+0000 7f5dbc882640 1 -- 172.21.15.116:0/1135775275 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db819d620 con 0x7f5db8103660 2024-09-12T03:10:59.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.837+0000 7f5daf7fe640 1 -- 172.21.15.116:0/1135775275 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da0035070 con 0x7f5db8103660 2024-09-12T03:10:59.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.837+0000 7f5dbc882640 1 -- 172.21.15.116:0/1135775275 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db81a4f60 con 0x7f5db8103660 2024-09-12T03:10:59.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.839+0000 7f5daf7fe640 1 -- 172.21.15.116:0/1135775275 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f5da0042030 con 0x7f5db8103660 2024-09-12T03:10:59.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.839+0000 7f5dbec5b740 1 Processor -- start 2024-09-12T03:10:59.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.840+0000 7f5dbec5b740 1 -- start start 2024-09-12T03:10:59.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.840+0000 7f5dbec5b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f991eafb60 0x55f991eaff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:59.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.840+0000 7f5dbec5b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f991eb0480 0x55f991eba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:59.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.840+0000 7f5dbec5b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f991ebad40 0x55f991ebd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:59.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.840+0000 7f5dbec5b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55f991e15280 con 0x55f991eb0480 2024-09-12T03:10:59.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.840+0000 7f5dbec5b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f991dbcb10 con 0x55f991ebad40 2024-09-12T03:10:59.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.840+0000 7f5dbec5b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f991e11240 con 0x55f991eafb60 2024-09-12T03:10:59.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.840+0000 7f5daf7fe640 1 --2- 172.21.15.116:0/1135775275 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5da40a2b80 0x7f5da40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:59.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.840+0000 7f5db659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f991eb0480 0x55f991eba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:59.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.840+0000 7f5db659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f991eb0480 0x55f991eba800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44456/0 (socket says 172.21.15.116:44456) 2024-09-12T03:10:59.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.840+0000 7f5db659c640 1 -- 172.21.15.116:0/645902990 learned_addr learned my addr 172.21.15.116:0/645902990 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:59.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.840+0000 7f5db6d9d640 1 --2- 172.21.15.116:0/645902990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f991eafb60 0x55f991eaff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:59.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.840+0000 7f5db6d9d640 1 --2- 172.21.15.116:0/1135775275 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5da40a2b80 0x7f5da40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:59.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.840+0000 7f5daf7fe640 1 -- 172.21.15.116:0/1135775275 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f5da00ef6a0 con 0x7f5db8103660 2024-09-12T03:10:59.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.840+0000 7f5db5d9b640 1 --2- 172.21.15.116:0/645902990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f991ebad40 0x55f991ebd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:59.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.840+0000 7f5db659c640 1 -- 172.21.15.116:0/645902990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f991eafb60 msgr2=0x55f991eaff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:59.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.840+0000 7f5db659c640 1 --2- 172.21.15.116:0/645902990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f991eafb60 0x55f991eaff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.840+0000 7f5db659c640 1 -- 172.21.15.116:0/645902990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f991ebad40 msgr2=0x55f991ebd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:59.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.840+0000 7f5db659c640 1 --2- 172.21.15.116:0/645902990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f991ebad40 0x55f991ebd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.840+0000 7f5db659c640 1 -- 172.21.15.116:0/645902990 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f991dc91f0 con 0x55f991eb0480 2024-09-12T03:10:59.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.840+0000 7f5db6d9d640 1 --2- 172.21.15.116:0/645902990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f991eafb60 0x55f991eaff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:10:59.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.840+0000 7f5db5d9b640 1 --2- 172.21.15.116:0/645902990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f991ebad40 0x55f991ebd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:59.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.841+0000 7f5db659c640 1 --2- 172.21.15.116:0/645902990 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f991eb0480 0x55f991eba800 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f5da8009c50 tx=0x7f5da800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56101b4e9ea26f11 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:59.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.841+0000 7f5db6d9d640 1 --2- 172.21.15.116:0/1135775275 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5da40a2b80 0x7f5da40a5040 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5db000bfd0 tx=0x7f5db0004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:59.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.841+0000 7f5dacff9640 1 -- 172.21.15.116:0/645902990 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da8016070 con 0x55f991eb0480 2024-09-12T03:10:59.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.841+0000 7f5dacff9640 1 -- 172.21.15.116:0/645902990 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5da8005500 con 0x55f991eb0480 2024-09-12T03:10:59.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.841+0000 7f5dacff9640 1 -- 172.21.15.116:0/645902990 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da8011670 con 0x55f991eb0480 2024-09-12T03:10:59.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.842+0000 7f5dbec5b740 1 -- 172.21.15.116:0/645902990 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f991eb0480 msgr2=0x55f991eba800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:59.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.842+0000 7f5dbec5b740 1 --2- 172.21.15.116:0/645902990 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f991eb0480 0x55f991eba800 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f5da8009c50 tx=0x7f5da800e6c0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.842+0000 7f5dbec5b740 1 -- 172.21.15.116:0/645902990 shutdown_connections 2024-09-12T03:10:59.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.842+0000 7f5dbec5b740 1 --2- 172.21.15.116:0/645902990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f991ebad40 0x55f991ebd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.842+0000 7f5dbec5b740 1 --2- 172.21.15.116:0/645902990 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f991eb0480 0x55f991eba800 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.842+0000 7f5dbec5b740 1 --2- 172.21.15.116:0/645902990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f991eafb60 0x55f991eaff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.842+0000 7f5dbec5b740 1 -- 172.21.15.116:0/645902990 >> 172.21.15.116:0/645902990 conn(0x55f991eac0f0 msgr2=0x55f991eac4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:59.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.842+0000 7f5dbec5b740 1 -- 172.21.15.116:0/645902990 shutdown_connections 2024-09-12T03:10:59.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.842+0000 7f5dbec5b740 1 -- 172.21.15.116:0/645902990 wait complete. 2024-09-12T03:10:59.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.842+0000 7f5dbec5b740 1 -- start start 2024-09-12T03:10:59.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.842+0000 7f5dbec5b740 1 Processor -- start 2024-09-12T03:10:59.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.842+0000 7f5dbec5b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f991eafb60 0x55f991eb7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:59.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.842+0000 7f5dbec5b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f991eb0480 0x55f991eb8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:59.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.842+0000 7f5dbec5b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f991ebad40 0x55f991eb8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:59.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.842+0000 7f5dbec5b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55f991ebd910 con 0x55f991ebad40 2024-09-12T03:10:59.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.842+0000 7f5dbec5b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f991dcff10 con 0x55f991eb0480 2024-09-12T03:10:59.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.842+0000 7f5dbec5b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f991ebda90 con 0x55f991eafb60 2024-09-12T03:10:59.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.842+0000 7f5dbec5b740 10 client.0 _ll_register_callbacks cb 0x55f991d98a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:59.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.842+0000 7f5db6d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f991ebad40 0x55f991eb8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:59.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.843+0000 7f5db6d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f991ebad40 0x55f991eb8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44466/0 (socket says 172.21.15.116:44466) 2024-09-12T03:10:59.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.843+0000 7f5db6d9d640 1 -- 172.21.15.116:0/3916268931 learned_addr learned my addr 172.21.15.116:0/3916268931 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:59.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.843+0000 7f5db659c640 1 --2- 172.21.15.116:0/3916268931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f991eafb60 0x55f991eb7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:59.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.843+0000 7f5db5d9b640 1 --2- 172.21.15.116:0/3916268931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f991eb0480 0x55f991eb8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:59.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.843+0000 7f5db6d9d640 1 -- 172.21.15.116:0/3916268931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f991eafb60 msgr2=0x55f991eb7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:59.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.843+0000 7f5db6d9d640 1 --2- 172.21.15.116:0/3916268931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f991eafb60 0x55f991eb7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.843+0000 7f5db6d9d640 1 -- 172.21.15.116:0/3916268931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f991eb0480 msgr2=0x55f991eb8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:59.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.843+0000 7f5db6d9d640 1 --2- 172.21.15.116:0/3916268931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f991eb0480 0x55f991eb8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.843+0000 7f5db6d9d640 1 -- 172.21.15.116:0/3916268931 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f991b866a0 con 0x55f991ebad40 2024-09-12T03:10:59.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.843+0000 7f5db5d9b640 1 --2- 172.21.15.116:0/3916268931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f991eb0480 0x55f991eb8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:59.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.843+0000 7f5db6d9d640 1 --2- 172.21.15.116:0/3916268931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f991ebad40 0x55f991eb8650 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f5db000efc0 tx=0x7f5db0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d84e138c7ab0e79 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:59.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.843+0000 7f5d9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:59.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.843+0000 7f5d9dffb640 1 -- 172.21.15.116:0/3916268931 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0022070 con 0x55f991ebad40 2024-09-12T03:10:59.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.843+0000 7f5d9dffb640 1 -- 172.21.15.116:0/3916268931 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f5db0010d40 con 0x55f991ebad40 2024-09-12T03:10:59.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.843+0000 7f5dbec5b740 1 -- 172.21.15.116:0/3916268931 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5da8005500 con 0x55f991ebad40 2024-09-12T03:10:59.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.843+0000 7f5d9dffb640 1 -- 172.21.15.116:0/3916268931 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5db002e050 con 0x55f991ebad40 2024-09-12T03:10:59.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.843+0000 7f5dbec5b740 10 client.17400 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:59.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.843+0000 7f5dbec5b740 1 -- 172.21.15.116:0/3916268931 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f991ec9430 con 0x55f991ebad40 2024-09-12T03:10:59.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.843+0000 7f5d9dffb640 1 -- 172.21.15.116:0/3916268931 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5da8005500 con 0x55f991ebad40 2024-09-12T03:10:59.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.844+0000 7f5d9dffb640 1 -- 172.21.15.116:0/3916268931 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5db001d040 con 0x55f991ebad40 2024-09-12T03:10:59.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.844+0000 7f5dbec5b740 10 client.17400 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:59.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.844+0000 7f5dbec5b740 10 client.17400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:10:59.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.844+0000 7f5dbec5b740 20 client.17400 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:59.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.844+0000 7f5dbec5b740 1 --2- 172.21.15.116:0/3916268931 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f991eca840 0x55f991eccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:59.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.844+0000 7f5dbec5b740 4 client.17400 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:10:59.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.844+0000 7f5dbec5b740 1 -- 172.21.15.116:0/3916268931 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f991e227f0 con 0x55f991eca840 2024-09-12T03:10:59.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.844+0000 7f5db659c640 1 --2- 172.21.15.116:0/3916268931 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f991eca840 0x55f991eccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:59.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.844+0000 7f5d9dffb640 10 client.17400 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:59.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.844+0000 7f5db659c640 1 --2- 172.21.15.116:0/3916268931 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f991eca840 0x55f991eccce0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=389305b7966d4adf server_cookie=5f71b25ac52511c in_seq=0 out_seq=0 2024-09-12T03:10:59.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.845+0000 7f5d9dffb640 1 -- 172.21.15.116:0/3916268931 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f991e227f0 con 0x55f991eca840 2024-09-12T03:10:59.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.845+0000 7f5d9dffb640 10 client.17400 handle_command_reply: tid=0 2024-09-12T03:10:59.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.845+0000 7f5dbec5b740 1 client.17400 shutdown 2024-09-12T03:10:59.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.845+0000 7f5dbec5b740 1 -- 172.21.15.116:0/3916268931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f991ebad40 msgr2=0x55f991eb8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:59.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.845+0000 7f5dbec5b740 1 --2- 172.21.15.116:0/3916268931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f991ebad40 0x55f991eb8650 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f5db000efc0 tx=0x7f5db0004ab0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.845+0000 7f5dbec5b740 1 -- 172.21.15.116:0/3916268931 shutdown_connections 2024-09-12T03:10:59.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.845+0000 7f5dbec5b740 1 --2- 172.21.15.116:0/3916268931 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f991eca840 0x55f991eccce0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.845+0000 7f5dbec5b740 1 --2- 172.21.15.116:0/3916268931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f991ebad40 0x55f991eb8650 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.845+0000 7f5dbec5b740 1 --2- 172.21.15.116:0/3916268931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f991eb0480 0x55f991eb8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.845+0000 7f5dbec5b740 1 --2- 172.21.15.116:0/3916268931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f991eafb60 0x55f991eb7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.845+0000 7f5d9dffb640 0 client.17400 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:59.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.845+0000 7f5dbec5b740 1 -- 172.21.15.116:0/3916268931 >> 172.21.15.116:0/3916268931 conn(0x55f991eac0f0 msgr2=0x55f991eab270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:59.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.845+0000 7f5dbec5b740 1 -- 172.21.15.116:0/3916268931 shutdown_connections 2024-09-12T03:10:59.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.845+0000 7f5dbec5b740 1 -- 172.21.15.116:0/3916268931 wait complete. 2024-09-12T03:10:59.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.845+0000 7f5dbec5b740 20 client.17400 trim_cache size 0 max 16384 2024-09-12T03:10:59.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.862+0000 7f5dbec5b740 1 Processor -- start 2024-09-12T03:10:59.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.862+0000 7f5dbec5b740 1 -- start start 2024-09-12T03:10:59.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.862+0000 7f5dbec5b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f991eafb60 0x55f991eb1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:59.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.862+0000 7f5dbec5b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f991eb0480 0x55f991ed2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:59.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.862+0000 7f5dbec5b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f991ebad40 0x55f991ed2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:59.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.862+0000 7f5dbec5b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5db81afd70 con 0x55f991ebad40 2024-09-12T03:10:59.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.862+0000 7f5dbec5b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5db81ad640 con 0x55f991eafb60 2024-09-12T03:10:59.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.862+0000 7f5dbec5b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5db81ab490 con 0x55f991eb0480 2024-09-12T03:10:59.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.863+0000 7f5db6d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f991ebad40 0x55f991ed2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:59.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.863+0000 7f5db6d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f991ebad40 0x55f991ed2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44482/0 (socket says 172.21.15.116:44482) 2024-09-12T03:10:59.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.863+0000 7f5db6d9d640 1 -- 172.21.15.116:0/689273963 learned_addr learned my addr 172.21.15.116:0/689273963 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:59.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.863+0000 7f5db659c640 1 --2- 172.21.15.116:0/689273963 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f991eafb60 0x55f991eb1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:59.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.863+0000 7f5db6d9d640 1 -- 172.21.15.116:0/689273963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f991eb0480 msgr2=0x55f991ed2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:59.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.863+0000 7f5db6d9d640 1 --2- 172.21.15.116:0/689273963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f991eb0480 0x55f991ed2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.863+0000 7f5db6d9d640 1 -- 172.21.15.116:0/689273963 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f991eafb60 msgr2=0x55f991eb1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:59.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.863+0000 7f5db6d9d640 1 --2- 172.21.15.116:0/689273963 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f991eafb60 0x55f991eb1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.863+0000 7f5db6d9d640 1 -- 172.21.15.116:0/689273963 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f991eabb90 con 0x55f991ebad40 2024-09-12T03:10:59.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.863+0000 7f5db6d9d640 1 --2- 172.21.15.116:0/689273963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f991ebad40 0x55f991ed2b40 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f5db000ea10 tx=0x7f5db0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7d6b7f6a46c403 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:59.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.863+0000 7f5d9ffff640 1 -- 172.21.15.116:0/689273963 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0022040 con 0x55f991ebad40 2024-09-12T03:10:59.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.863+0000 7f5d9ffff640 1 -- 172.21.15.116:0/689273963 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5db0037070 con 0x55f991ebad40 2024-09-12T03:10:59.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.863+0000 7f5d9ffff640 1 -- 172.21.15.116:0/689273963 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0017040 con 0x55f991ebad40 2024-09-12T03:10:59.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.864+0000 7f5dbec5b740 1 -- 172.21.15.116:0/689273963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f991ebad40 msgr2=0x55f991ed2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:59.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.864+0000 7f5dbec5b740 1 --2- 172.21.15.116:0/689273963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f991ebad40 0x55f991ed2b40 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f5db000ea10 tx=0x7f5db0000f40 comp rx=0 tx=0).stop 2024-09-12T03:10:59.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.864+0000 7f5dbec5b740 1 -- 172.21.15.116:0/689273963 shutdown_connections 2024-09-12T03:10:59.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.864+0000 7f5dbec5b740 1 --2- 172.21.15.116:0/689273963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f991ebad40 0x55f991ed2b40 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.864+0000 7f5dbec5b740 1 --2- 172.21.15.116:0/689273963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f991eb0480 0x55f991ed2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.864+0000 7f5dbec5b740 1 --2- 172.21.15.116:0/689273963 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f991eafb60 0x55f991eb1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.864+0000 7f5dbec5b740 1 -- 172.21.15.116:0/689273963 >> 172.21.15.116:0/689273963 conn(0x55f991eac0f0 msgr2=0x55f991eb15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:59.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.864+0000 7f5dbec5b740 1 -- 172.21.15.116:0/689273963 shutdown_connections 2024-09-12T03:10:59.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.864+0000 7f5dbec5b740 1 -- 172.21.15.116:0/689273963 wait complete. 2024-09-12T03:10:59.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.864+0000 7f5dbec5b740 1 -- start start 2024-09-12T03:10:59.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.864+0000 7f5dbec5b740 1 Processor -- start 2024-09-12T03:10:59.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.865+0000 7f5dbec5b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f991eafb60 0x55f991ebf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:59.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.865+0000 7f5dbec5b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f991eb0480 0x55f991ebf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:59.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.865+0000 7f5dbec5b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f991ebad40 0x55f991f389e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:59.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.865+0000 7f5dbec5b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5db81abb00 con 0x55f991eafb60 2024-09-12T03:10:59.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.865+0000 7f5dbec5b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5db81acf70 con 0x55f991ebad40 2024-09-12T03:10:59.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.865+0000 7f5dbec5b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5db81ac1d0 con 0x55f991eb0480 2024-09-12T03:10:59.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.865+0000 7f5dbec5b740 10 client.0 _ll_register_callbacks cb 0x7f5db000e440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:59.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.865+0000 7f5db659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f991eafb60 0x55f991ebf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:59.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.865+0000 7f5db659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f991eafb60 0x55f991ebf120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44494/0 (socket says 172.21.15.116:44494) 2024-09-12T03:10:59.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.865+0000 7f5db659c640 1 -- 172.21.15.116:0/151168300 learned_addr learned my addr 172.21.15.116:0/151168300 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:10:59.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.865+0000 7f5db5d9b640 1 --2- 172.21.15.116:0/151168300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f991eb0480 0x55f991ebf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:59.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.865+0000 7f5db6d9d640 1 --2- 172.21.15.116:0/151168300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f991ebad40 0x55f991f389e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:59.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.865+0000 7f5db659c640 1 -- 172.21.15.116:0/151168300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f991eb0480 msgr2=0x55f991ebf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:59.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.865+0000 7f5db659c640 1 --2- 172.21.15.116:0/151168300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f991eb0480 0x55f991ebf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.865+0000 7f5db659c640 1 -- 172.21.15.116:0/151168300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f991ebad40 msgr2=0x55f991f389e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:59.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.865+0000 7f5db659c640 1 --2- 172.21.15.116:0/151168300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f991ebad40 0x55f991f389e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.865+0000 7f5db659c640 1 -- 172.21.15.116:0/151168300 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f991f38f20 con 0x55f991eafb60 2024-09-12T03:10:59.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.865+0000 7f5db6d9d640 1 --2- 172.21.15.116:0/151168300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f991ebad40 0x55f991f389e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:59.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.865+0000 7f5db659c640 1 --2- 172.21.15.116:0/151168300 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f991eafb60 0x55f991ebf120 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f5da800ebc0 tx=0x7f5da8009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=271c5fb6d32efb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:59.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.865+0000 7f5d9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:10:59.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.865+0000 7f5d9dffb640 1 -- 172.21.15.116:0/151168300 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da8016040 con 0x55f991eafb60 2024-09-12T03:10:59.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.865+0000 7f5d9dffb640 1 -- 172.21.15.116:0/151168300 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f5da8004880 con 0x55f991eafb60 2024-09-12T03:10:59.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.865+0000 7f5dbec5b740 1 -- 172.21.15.116:0/151168300 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5db0037070 con 0x55f991eafb60 2024-09-12T03:10:59.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.866+0000 7f5d9dffb640 1 -- 172.21.15.116:0/151168300 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5da8038050 con 0x55f991eafb60 2024-09-12T03:10:59.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.866+0000 7f5dbec5b740 10 client.17406 fetch_fsmap learned FSMap version 25 2024-09-12T03:10:59.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.866+0000 7f5dbec5b740 1 -- 172.21.15.116:0/151168300 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f991f3a260 con 0x55f991eafb60 2024-09-12T03:10:59.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.866+0000 7f5d9dffb640 1 -- 172.21.15.116:0/151168300 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5db0037070 con 0x55f991eafb60 2024-09-12T03:10:59.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.866+0000 7f5d9dffb640 1 -- 172.21.15.116:0/151168300 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5da800f040 con 0x55f991eafb60 2024-09-12T03:10:59.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.866+0000 7f5dbec5b740 10 client.17406 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:10:59.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.866+0000 7f5dbec5b740 10 client.17406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:10:59.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.866+0000 7f5dbec5b740 20 client.17406 populate_metadata read hostname 'smithi116' 2024-09-12T03:10:59.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.866+0000 7f5dbec5b740 1 --2- 172.21.15.116:0/151168300 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f991f3a7f0 0x55f991f3cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:59.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.866+0000 7f5dbec5b740 4 client.17406 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:10:59.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.866+0000 7f5dbec5b740 1 -- 172.21.15.116:0/151168300 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f991e1cd00 con 0x55f991f3a7f0 2024-09-12T03:10:59.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.866+0000 7f5db5d9b640 1 --2- 172.21.15.116:0/151168300 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f991f3a7f0 0x55f991f3cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:59.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.867+0000 7f5db5d9b640 1 --2- 172.21.15.116:0/151168300 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f991f3a7f0 0x55f991f3cbe0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b309fa51dc1ef03 server_cookie=c15f6ee33b578d75 in_seq=0 out_seq=0 2024-09-12T03:10:59.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.867+0000 7f5d9dffb640 10 client.17406 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:59.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.868+0000 7f5d9dffb640 1 -- 172.21.15.116:0/151168300 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f991e1cd00 con 0x55f991f3a7f0 2024-09-12T03:10:59.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.868+0000 7f5d9dffb640 10 client.17406 handle_command_reply: tid=0 2024-09-12T03:10:59.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.868+0000 7f5dbec5b740 1 client.17406 shutdown 2024-09-12T03:10:59.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.868+0000 7f5dbec5b740 1 -- 172.21.15.116:0/151168300 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f991eafb60 msgr2=0x55f991ebf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:59.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.868+0000 7f5dbec5b740 1 --2- 172.21.15.116:0/151168300 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f991eafb60 0x55f991ebf120 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f5da800ebc0 tx=0x7f5da8009dd0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.868+0000 7f5dbec5b740 1 -- 172.21.15.116:0/151168300 shutdown_connections 2024-09-12T03:10:59.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.868+0000 7f5dbec5b740 1 --2- 172.21.15.116:0/151168300 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f991f3a7f0 0x55f991f3cbe0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.868+0000 7f5dbec5b740 1 --2- 172.21.15.116:0/151168300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f991ebad40 0x55f991f389e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.868+0000 7f5dbec5b740 1 --2- 172.21.15.116:0/151168300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f991eb0480 0x55f991ebf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.868+0000 7f5dbec5b740 1 --2- 172.21.15.116:0/151168300 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f991eafb60 0x55f991ebf120 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.868+0000 7f5dbec5b740 1 -- 172.21.15.116:0/151168300 >> 172.21.15.116:0/151168300 conn(0x55f991eac0f0 msgr2=0x55f991eab270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:59.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.868+0000 7f5d9dffb640 0 client.17406 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:10:59.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.868+0000 7f5dbec5b740 1 -- 172.21.15.116:0/151168300 shutdown_connections 2024-09-12T03:10:59.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.868+0000 7f5dbec5b740 1 -- 172.21.15.116:0/151168300 wait complete. 2024-09-12T03:10:59.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.868+0000 7f5dbec5b740 20 client.17406 trim_cache size 0 max 16384 2024-09-12T03:10:59.874 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:10:59.874 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:10:59.874 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "33766886-88d2-4a6e-b894-c4169a566de7", 2024-09-12T03:10:59.874 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:10:59.875 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:10:59.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.869+0000 7f5dbc882640 1 -- 172.21.15.116:0/1135775275 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5da40a2b80 msgr2=0x7f5da40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:59.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.869+0000 7f5dbc882640 1 --2- 172.21.15.116:0/1135775275 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5da40a2b80 0x7f5da40a5040 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5db000bfd0 tx=0x7f5db0004a40 comp rx=0 tx=0).stop 2024-09-12T03:10:59.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.869+0000 7f5dbc882640 1 -- 172.21.15.116:0/1135775275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5db8103660 msgr2=0x7f5db8198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:59.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.869+0000 7f5dbc882640 1 --2- 172.21.15.116:0/1135775275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5db8103660 0x7f5db8198920 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f5da0006010 tx=0x7f5da00042f0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.869+0000 7f5dbc882640 1 -- 172.21.15.116:0/1135775275 shutdown_connections 2024-09-12T03:10:59.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.869+0000 7f5dbc882640 1 --2- 172.21.15.116:0/1135775275 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5da40a2b80 0x7f5da40a5040 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.869+0000 7f5dbc882640 1 --2- 172.21.15.116:0/1135775275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5db810c170 0x7f5db819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.869+0000 7f5dbc882640 1 --2- 172.21.15.116:0/1135775275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5db8103660 0x7f5db8198920 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.869+0000 7f5dbc882640 1 --2- 172.21.15.116:0/1135775275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5db8102d40 0x7f5db81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.869+0000 7f5dbc882640 1 -- 172.21.15.116:0/1135775275 >> 172.21.15.116:0/1135775275 conn(0x7f5db80fc7a0 msgr2=0x7f5db80fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:59.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.869+0000 7f5dbc882640 1 -- 172.21.15.116:0/1135775275 shutdown_connections 2024-09-12T03:10:59.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:10:59.869+0000 7f5dbc882640 1 -- 172.21.15.116:0/1135775275 wait complete. 2024-09-12T03:11:00.604 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:00 smithi116 ceph-mon[31410]: pgmap v578: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.0 KiB/s wr, 8 op/s 2024-09-12T03:11:00.605 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:11:00.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:00 smithi183 ceph-mon[35535]: pgmap v578: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.0 KiB/s wr, 8 op/s 2024-09-12T03:11:00.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:00 smithi177 ceph-mon[35305]: pgmap v578: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.0 KiB/s wr, 8 op/s 2024-09-12T03:11:00.853 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:11:01.385 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:01 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:11:01.385 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:01 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:11:01.385 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:01 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:11:01.385 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:01 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:11:01.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.352+0000 7f2faef9e640 1 -- 172.21.15.116:0/2888666158 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fa8101420 msgr2=0x7f2fa810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:01.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.352+0000 7f2faef9e640 1 --2- 172.21.15.116:0/2888666158 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fa8101420 0x7f2fa810d950 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0009cd0 tx=0x7f2fa002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.352+0000 7f2faef9e640 1 -- 172.21.15.116:0/2888666158 shutdown_connections 2024-09-12T03:11:01.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.352+0000 7f2faef9e640 1 --2- 172.21.15.116:0/2888666158 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fa8101420 0x7f2fa810d950 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.352+0000 7f2faef9e640 1 --2- 172.21.15.116:0/2888666158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fa8100a80 0x7f2fa8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.352+0000 7f2faef9e640 1 --2- 172.21.15.116:0/2888666158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2fa8106a80 0x7f2fa8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.352+0000 7f2faef9e640 1 -- 172.21.15.116:0/2888666158 >> 172.21.15.116:0/2888666158 conn(0x7f2fa80fc7a0 msgr2=0x7f2fa80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:01.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.352+0000 7f2faef9e640 1 -- 172.21.15.116:0/2888666158 shutdown_connections 2024-09-12T03:11:01.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.352+0000 7f2faef9e640 1 -- 172.21.15.116:0/2888666158 wait complete. 2024-09-12T03:11:01.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.353+0000 7f2faef9e640 1 Processor -- start 2024-09-12T03:11:01.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.353+0000 7f2faef9e640 1 -- start start 2024-09-12T03:11:01.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.353+0000 7f2faef9e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2fa8100a80 0x7f2fa8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:01.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.353+0000 7f2faef9e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fa8101420 0x7f2fa81988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:01.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.353+0000 7f2faef9e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fa8106a80 0x7f2fa819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:01.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.353+0000 7f2faef9e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa8110ed0 con 0x7f2fa8101420 2024-09-12T03:11:01.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.353+0000 7f2faef9e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa8110d50 con 0x7f2fa8100a80 2024-09-12T03:11:01.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.353+0000 7f2faef9e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa8111050 con 0x7f2fa8106a80 2024-09-12T03:11:01.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.353+0000 7f2fa7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fa8101420 0x7f2fa81988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:01.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.353+0000 7f2fa7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fa8101420 0x7f2fa81988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58422/0 (socket says 172.21.15.116:58422) 2024-09-12T03:11:01.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.353+0000 7f2fa7fff640 1 -- 172.21.15.116:0/3732649250 learned_addr learned my addr 172.21.15.116:0/3732649250 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:01.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.353+0000 7f2facd3a640 1 --2- 172.21.15.116:0/3732649250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2fa8100a80 0x7f2fa8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:01.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.353+0000 7f2fad53b640 1 --2- 172.21.15.116:0/3732649250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fa8106a80 0x7f2fa819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:01.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.353+0000 7f2fa7fff640 1 -- 172.21.15.116:0/3732649250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fa8106a80 msgr2=0x7f2fa819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:01.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.353+0000 7f2fa7fff640 1 --2- 172.21.15.116:0/3732649250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fa8106a80 0x7f2fa819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.353+0000 7f2fa7fff640 1 -- 172.21.15.116:0/3732649250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2fa8100a80 msgr2=0x7f2fa8198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:01.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.353+0000 7f2fa7fff640 1 --2- 172.21.15.116:0/3732649250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2fa8100a80 0x7f2fa8198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.353+0000 7f2fa7fff640 1 -- 172.21.15.116:0/3732649250 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa819d330 con 0x7f2fa8101420 2024-09-12T03:11:01.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.354+0000 7f2fa7fff640 1 --2- 172.21.15.116:0/3732649250 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fa8101420 0x7f2fa81988c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400e9f0 tx=0x7f2f9400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:01.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.354+0000 7f2fa5ffb640 1 -- 172.21.15.116:0/3732649250 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9400cd80 con 0x7f2fa8101420 2024-09-12T03:11:01.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.354+0000 7f2faef9e640 1 -- 172.21.15.116:0/3732649250 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fa819d620 con 0x7f2fa8101420 2024-09-12T03:11:01.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.354+0000 7f2fa5ffb640 1 -- 172.21.15.116:0/3732649250 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f94005500 con 0x7f2fa8101420 2024-09-12T03:11:01.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.354+0000 7f2fa5ffb640 1 -- 172.21.15.116:0/3732649250 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f94010690 con 0x7f2fa8101420 2024-09-12T03:11:01.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.354+0000 7f2faef9e640 1 -- 172.21.15.116:0/3732649250 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fa81a4f00 con 0x7f2fa8101420 2024-09-12T03:11:01.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.356+0000 7f2fa5ffb640 1 -- 172.21.15.116:0/3732649250 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2f940046c0 con 0x7f2fa8101420 2024-09-12T03:11:01.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.356+0000 7f2fb1377740 1 Processor -- start 2024-09-12T03:11:01.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.357+0000 7f2fb1377740 1 -- start start 2024-09-12T03:11:01.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.357+0000 7f2fa5ffb640 1 --2- 172.21.15.116:0/3732649250 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2f980a2b80 0x7f2f980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:01.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.357+0000 7f2fb1377740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56551dbf7360 0x56551dbf7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:01.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.357+0000 7f2fb1377740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56551dbf7c80 0x56551dc06550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:01.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.357+0000 7f2fb1377740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56551dc06a90 0x56551dc08e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:01.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.357+0000 7f2fb1377740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56551db5d2e0 con 0x56551dbf7c80 2024-09-12T03:11:01.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.357+0000 7f2fb1377740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56551db04ce0 con 0x56551dbf7360 2024-09-12T03:11:01.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.357+0000 7f2fb1377740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56551db592a0 con 0x56551dc06a90 2024-09-12T03:11:01.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.357+0000 7f2fad53b640 1 --2- 172.21.15.116:0/3732649250 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2f980a2b80 0x7f2f980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:01.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.357+0000 7f2fa7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56551dbf7c80 0x56551dc06550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:01.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.357+0000 7f2fa5ffb640 1 -- 172.21.15.116:0/3732649250 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f2f940cf450 con 0x7f2fa8101420 2024-09-12T03:11:01.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.357+0000 7f2fa7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56551dbf7c80 0x56551dc06550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58434/0 (socket says 172.21.15.116:58434) 2024-09-12T03:11:01.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.357+0000 7f2fa7fff640 1 -- 172.21.15.116:0/1120988298 learned_addr learned my addr 172.21.15.116:0/1120988298 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:01.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.357+0000 7f2fad53b640 1 --2- 172.21.15.116:0/1120988298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56551dc06a90 0x56551dc08e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:01.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.357+0000 7f2facd3a640 1 --2- 172.21.15.116:0/1120988298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56551dbf7360 0x56551dbf7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:01.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.357+0000 7f2fa7fff640 1 -- 172.21.15.116:0/1120988298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56551dc06a90 msgr2=0x56551dc08e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:01.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.357+0000 7f2fa7fff640 1 --2- 172.21.15.116:0/1120988298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56551dc06a90 0x56551dc08e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.357+0000 7f2fa7fff640 1 -- 172.21.15.116:0/1120988298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56551dbf7360 msgr2=0x56551dbf7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:01.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.357+0000 7f2fa7fff640 1 --2- 172.21.15.116:0/1120988298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56551dbf7360 0x56551dbf7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.357+0000 7f2fa7fff640 1 -- 172.21.15.116:0/1120988298 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56551dae9440 con 0x56551dbf7c80 2024-09-12T03:11:01.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.357+0000 7f2fad53b640 1 --2- 172.21.15.116:0/3732649250 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2f980a2b80 0x7f2f980a5040 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0005e30 tx=0x7f2fa0031770 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:01.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.357+0000 7f2fa7fff640 1 --2- 172.21.15.116:0/1120988298 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56551dbf7c80 0x56551dc06550 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f2f94097fa0 tx=0x7f2f94005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186b117a58e5179c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:01.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.358+0000 7f2f92ffd640 1 -- 172.21.15.116:0/1120988298 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9401c1c0 con 0x56551dbf7c80 2024-09-12T03:11:01.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.358+0000 7f2f92ffd640 1 -- 172.21.15.116:0/1120988298 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f940071c0 con 0x56551dbf7c80 2024-09-12T03:11:01.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.358+0000 7f2f92ffd640 1 -- 172.21.15.116:0/1120988298 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9401dd60 con 0x56551dbf7c80 2024-09-12T03:11:01.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.358+0000 7f2fb1377740 1 -- 172.21.15.116:0/1120988298 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56551dbf7c80 msgr2=0x56551dc06550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:01.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.358+0000 7f2fb1377740 1 --2- 172.21.15.116:0/1120988298 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56551dbf7c80 0x56551dc06550 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f2f94097fa0 tx=0x7f2f94005740 comp rx=0 tx=0).stop 2024-09-12T03:11:01.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.358+0000 7f2fb1377740 1 -- 172.21.15.116:0/1120988298 shutdown_connections 2024-09-12T03:11:01.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.358+0000 7f2fb1377740 1 --2- 172.21.15.116:0/1120988298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56551dc06a90 0x56551dc08e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.358+0000 7f2fb1377740 1 --2- 172.21.15.116:0/1120988298 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56551dbf7c80 0x56551dc06550 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.358+0000 7f2fb1377740 1 --2- 172.21.15.116:0/1120988298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56551dbf7360 0x56551dbf7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.358+0000 7f2fb1377740 1 -- 172.21.15.116:0/1120988298 >> 172.21.15.116:0/1120988298 conn(0x56551dbf3a30 msgr2=0x56551dbf3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:01.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.358+0000 7f2fb1377740 1 -- 172.21.15.116:0/1120988298 shutdown_connections 2024-09-12T03:11:01.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.359+0000 7f2fb1377740 1 -- 172.21.15.116:0/1120988298 wait complete. 2024-09-12T03:11:01.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.359+0000 7f2fb1377740 1 -- start start 2024-09-12T03:11:01.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.359+0000 7f2fb1377740 1 Processor -- start 2024-09-12T03:11:01.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.359+0000 7f2fb1377740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56551dbf7360 0x56551dc06440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:01.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.359+0000 7f2fb1377740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56551dbf7c80 0x56551dc036f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:01.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.359+0000 7f2fb1377740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56551dc06a90 0x56551dc03c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:01.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.359+0000 7f2fb1377740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56551dc09660 con 0x56551dbf7360 2024-09-12T03:11:01.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.359+0000 7f2fb1377740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56551db18050 con 0x56551dbf7c80 2024-09-12T03:11:01.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.359+0000 7f2fb1377740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56551dc097e0 con 0x56551dc06a90 2024-09-12T03:11:01.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.359+0000 7f2fb1377740 10 client.0 _ll_register_callbacks cb 0x56551db58160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:01.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.359+0000 7f2facd3a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56551dbf7360 0x56551dc06440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:01.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.359+0000 7f2facd3a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56551dbf7360 0x56551dc06440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58450/0 (socket says 172.21.15.116:58450) 2024-09-12T03:11:01.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.359+0000 7f2facd3a640 1 -- 172.21.15.116:0/792758891 learned_addr learned my addr 172.21.15.116:0/792758891 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:01.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.360+0000 7f2fa7fff640 1 --2- 172.21.15.116:0/792758891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56551dbf7c80 0x56551dc036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:01.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.360+0000 7f2fad53b640 1 --2- 172.21.15.116:0/792758891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56551dc06a90 0x56551dc03c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:01.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.360+0000 7f2facd3a640 1 -- 172.21.15.116:0/792758891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56551dc06a90 msgr2=0x56551dc03c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:01.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.360+0000 7f2facd3a640 1 --2- 172.21.15.116:0/792758891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56551dc06a90 0x56551dc03c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.360+0000 7f2facd3a640 1 -- 172.21.15.116:0/792758891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56551dbf7c80 msgr2=0x56551dc036f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:01.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.360+0000 7f2facd3a640 1 --2- 172.21.15.116:0/792758891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56551dbf7c80 0x56551dc036f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.360+0000 7f2facd3a640 1 -- 172.21.15.116:0/792758891 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56551d8ce3c0 con 0x56551dbf7360 2024-09-12T03:11:01.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.360+0000 7f2fa7fff640 1 --2- 172.21.15.116:0/792758891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56551dbf7c80 0x56551dc036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:11:01.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.360+0000 7f2facd3a640 1 --2- 172.21.15.116:0/792758891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56551dbf7360 0x56551dc06440 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c0126d0 tx=0x7f2f9c012b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a188ec0bc5c0c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:01.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.360+0000 7f2f87fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:01.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.360+0000 7f2f87fff640 1 -- 172.21.15.116:0/792758891 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c01f060 con 0x56551dbf7360 2024-09-12T03:11:01.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.360+0000 7f2fb1377740 1 -- 172.21.15.116:0/792758891 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f940071c0 con 0x56551dbf7360 2024-09-12T03:11:01.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.360+0000 7f2f87fff640 1 -- 172.21.15.116:0/792758891 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f2f9c010dc0 con 0x56551dbf7360 2024-09-12T03:11:01.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.360+0000 7f2f87fff640 1 -- 172.21.15.116:0/792758891 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2f9c01a430 con 0x56551dbf7360 2024-09-12T03:11:01.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.360+0000 7f2fb1377740 10 client.17418 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:01.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.360+0000 7f2fb1377740 1 -- 172.21.15.116:0/792758891 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56551dc15110 con 0x56551dbf7360 2024-09-12T03:11:01.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.360+0000 7f2f87fff640 1 -- 172.21.15.116:0/792758891 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f940071c0 con 0x56551dbf7360 2024-09-12T03:11:01.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.361+0000 7f2f87fff640 1 -- 172.21.15.116:0/792758891 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2f9c00e570 con 0x56551dbf7360 2024-09-12T03:11:01.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.361+0000 7f2fb1377740 10 client.17418 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:01.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.361+0000 7f2fb1377740 10 client.17418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:11:01.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.361+0000 7f2fb1377740 20 client.17418 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:01.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.361+0000 7f2fb1377740 1 --2- 172.21.15.116:0/792758891 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56551dc165d0 0x56551dc18a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:01.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.361+0000 7f2fb1377740 4 client.17418 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:11:01.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.361+0000 7f2fb1377740 1 -- 172.21.15.116:0/792758891 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56551db6a850 con 0x56551dc165d0 2024-09-12T03:11:01.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.361+0000 7f2fa7fff640 1 --2- 172.21.15.116:0/792758891 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56551dc165d0 0x56551dc18a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:01.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.361+0000 7f2fa7fff640 1 --2- 172.21.15.116:0/792758891 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56551dc165d0 0x56551dc18a00 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53e0cc68c7962dfd server_cookie=2c2bf87e44deff07 in_seq=0 out_seq=0 2024-09-12T03:11:01.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.361+0000 7f2f87fff640 10 client.17418 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:01.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.362+0000 7f2f87fff640 1 -- 172.21.15.116:0/792758891 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56551db6a850 con 0x56551dc165d0 2024-09-12T03:11:01.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.362+0000 7f2f87fff640 10 client.17418 handle_command_reply: tid=0 2024-09-12T03:11:01.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.362+0000 7f2fb1377740 1 client.17418 shutdown 2024-09-12T03:11:01.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.362+0000 7f2fb1377740 1 -- 172.21.15.116:0/792758891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56551dbf7360 msgr2=0x56551dc06440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:01.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.362+0000 7f2fb1377740 1 --2- 172.21.15.116:0/792758891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56551dbf7360 0x56551dc06440 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c0126d0 tx=0x7f2f9c012b90 comp rx=0 tx=0).stop 2024-09-12T03:11:01.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.362+0000 7f2fb1377740 1 -- 172.21.15.116:0/792758891 shutdown_connections 2024-09-12T03:11:01.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.362+0000 7f2fb1377740 1 --2- 172.21.15.116:0/792758891 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56551dc165d0 0x56551dc18a00 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.362+0000 7f2fb1377740 1 --2- 172.21.15.116:0/792758891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56551dc06a90 0x56551dc03c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.362+0000 7f2fb1377740 1 --2- 172.21.15.116:0/792758891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56551dbf7c80 0x56551dc036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.362+0000 7f2fb1377740 1 --2- 172.21.15.116:0/792758891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56551dbf7360 0x56551dc06440 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.362+0000 7f2f87fff640 0 client.17418 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:01.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.362+0000 7f2fb1377740 1 -- 172.21.15.116:0/792758891 >> 172.21.15.116:0/792758891 conn(0x56551dbf3a30 msgr2=0x56551dbf9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:01.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.362+0000 7f2fb1377740 1 -- 172.21.15.116:0/792758891 shutdown_connections 2024-09-12T03:11:01.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.362+0000 7f2fb1377740 1 -- 172.21.15.116:0/792758891 wait complete. 2024-09-12T03:11:01.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.363+0000 7f2fb1377740 20 client.17418 trim_cache size 0 max 16384 2024-09-12T03:11:01.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.379+0000 7f2fb1377740 1 Processor -- start 2024-09-12T03:11:01.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.379+0000 7f2fb1377740 1 -- start start 2024-09-12T03:11:01.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.379+0000 7f2fb1377740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56551dbf7360 0x56551dc12900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:01.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.379+0000 7f2fb1377740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56551dbf7c80 0x56551dc1e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:01.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.379+0000 7f2fb1377740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56551dc06a90 0x56551dc1e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:01.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.379+0000 7f2fb1377740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa81adb90 con 0x56551dbf7360 2024-09-12T03:11:01.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.379+0000 7f2fb1377740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa81ab460 con 0x56551dc06a90 2024-09-12T03:11:01.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.379+0000 7f2fb1377740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa81a92b0 con 0x56551dbf7c80 2024-09-12T03:11:01.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.380+0000 7f2facd3a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56551dbf7360 0x56551dc12900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:01.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.380+0000 7f2facd3a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56551dbf7360 0x56551dc12900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58460/0 (socket says 172.21.15.116:58460) 2024-09-12T03:11:01.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.380+0000 7f2facd3a640 1 -- 172.21.15.116:0/1601639655 learned_addr learned my addr 172.21.15.116:0/1601639655 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:01.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.380+0000 7f2fad53b640 1 --2- 172.21.15.116:0/1601639655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56551dc06a90 0x56551dc1e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:01.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.380+0000 7f2fa7fff640 1 --2- 172.21.15.116:0/1601639655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56551dbf7c80 0x56551dc1e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:01.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.380+0000 7f2facd3a640 1 -- 172.21.15.116:0/1601639655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56551dbf7c80 msgr2=0x56551dc1e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:01.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.380+0000 7f2facd3a640 1 --2- 172.21.15.116:0/1601639655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56551dbf7c80 0x56551dc1e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.380+0000 7f2facd3a640 1 -- 172.21.15.116:0/1601639655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56551dc06a90 msgr2=0x56551dc1e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:01.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.380+0000 7f2facd3a640 1 --2- 172.21.15.116:0/1601639655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56551dc06a90 0x56551dc1e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.380+0000 7f2facd3a640 1 -- 172.21.15.116:0/1601639655 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56551dbf34d0 con 0x56551dbf7360 2024-09-12T03:11:01.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.380+0000 7f2fa7fff640 1 --2- 172.21.15.116:0/1601639655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56551dbf7c80 0x56551dc1e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:01.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.380+0000 7f2facd3a640 1 --2- 172.21.15.116:0/1601639655 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56551dbf7360 0x56551dc12900 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c00ea10 tx=0x7f2f9c00aef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c94d3d0eb6a92fa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:01.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.380+0000 7f2f927fc640 1 -- 172.21.15.116:0/1601639655 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c033070 con 0x56551dbf7360 2024-09-12T03:11:01.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.380+0000 7f2f927fc640 1 -- 172.21.15.116:0/1601639655 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f9c010bc0 con 0x56551dbf7360 2024-09-12T03:11:01.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.380+0000 7f2f927fc640 1 -- 172.21.15.116:0/1601639655 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c02a040 con 0x56551dbf7360 2024-09-12T03:11:01.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.381+0000 7f2fb1377740 1 -- 172.21.15.116:0/1601639655 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56551dbf7360 msgr2=0x56551dc12900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:01.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.381+0000 7f2fb1377740 1 --2- 172.21.15.116:0/1601639655 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56551dbf7360 0x56551dc12900 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c00ea10 tx=0x7f2f9c00aef0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.381+0000 7f2fb1377740 1 -- 172.21.15.116:0/1601639655 shutdown_connections 2024-09-12T03:11:01.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.381+0000 7f2fb1377740 1 --2- 172.21.15.116:0/1601639655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56551dc06a90 0x56551dc1e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.381+0000 7f2fb1377740 1 --2- 172.21.15.116:0/1601639655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56551dbf7c80 0x56551dc1e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.381+0000 7f2fb1377740 1 --2- 172.21.15.116:0/1601639655 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56551dbf7360 0x56551dc12900 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.381+0000 7f2fb1377740 1 -- 172.21.15.116:0/1601639655 >> 172.21.15.116:0/1601639655 conn(0x56551dbf3a30 msgr2=0x56551dbfed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:01.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.381+0000 7f2fb1377740 1 -- 172.21.15.116:0/1601639655 shutdown_connections 2024-09-12T03:11:01.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.381+0000 7f2fb1377740 1 -- 172.21.15.116:0/1601639655 wait complete. 2024-09-12T03:11:01.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.381+0000 7f2fb1377740 1 -- start start 2024-09-12T03:11:01.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.381+0000 7f2fb1377740 1 Processor -- start 2024-09-12T03:11:01.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.381+0000 7f2fb1377740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56551dbf7360 0x56551dc0a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:01.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.381+0000 7f2fb1377740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56551dbf7c80 0x56551dc0ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:01.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.381+0000 7f2fb1377740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56551dc06a90 0x56551dc0b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:01.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.382+0000 7f2fb1377740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa81a9920 con 0x56551dbf7c80 2024-09-12T03:11:01.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.382+0000 7f2fb1377740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa81aad90 con 0x56551dc06a90 2024-09-12T03:11:01.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.382+0000 7f2fb1377740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa81a9ff0 con 0x56551dbf7360 2024-09-12T03:11:01.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.382+0000 7f2fb1377740 10 client.0 _ll_register_callbacks cb 0x7f2f9c00a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:01.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.382+0000 7f2fa7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56551dbf7c80 0x56551dc0ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:01.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.382+0000 7f2fa7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56551dbf7c80 0x56551dc0ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58472/0 (socket says 172.21.15.116:58472) 2024-09-12T03:11:01.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.382+0000 7f2fa7fff640 1 -- 172.21.15.116:0/3706187609 learned_addr learned my addr 172.21.15.116:0/3706187609 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:01.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.382+0000 7f2facd3a640 1 --2- 172.21.15.116:0/3706187609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56551dbf7360 0x56551dc0a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:01.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.382+0000 7f2fad53b640 1 --2- 172.21.15.116:0/3706187609 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56551dc06a90 0x56551dc0b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:01.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.382+0000 7f2fa7fff640 1 -- 172.21.15.116:0/3706187609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56551dbf7360 msgr2=0x56551dc0a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:01.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.382+0000 7f2fa7fff640 1 --2- 172.21.15.116:0/3706187609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56551dbf7360 0x56551dc0a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.382+0000 7f2fa7fff640 1 -- 172.21.15.116:0/3706187609 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56551dc06a90 msgr2=0x56551dc0b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:01.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.382+0000 7f2fa7fff640 1 --2- 172.21.15.116:0/3706187609 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56551dc06a90 0x56551dc0b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.382+0000 7f2fa7fff640 1 -- 172.21.15.116:0/3706187609 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56551dc802f0 con 0x56551dbf7c80 2024-09-12T03:11:01.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.382+0000 7f2fa7fff640 1 --2- 172.21.15.116:0/3706187609 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56551dbf7c80 0x56551dc0ad50 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f2f9401c370 tx=0x7f2f94004ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6927c36dbe813a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:01.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.382+0000 7f2f87fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:01.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.382+0000 7f2f87fff640 1 -- 172.21.15.116:0/3706187609 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f940b8020 con 0x56551dbf7c80 2024-09-12T03:11:01.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.382+0000 7f2f87fff640 1 -- 172.21.15.116:0/3706187609 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f2f940982f0 con 0x56551dbf7c80 2024-09-12T03:11:01.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.382+0000 7f2fb1377740 1 -- 172.21.15.116:0/3706187609 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f9c010bc0 con 0x56551dbf7c80 2024-09-12T03:11:01.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.382+0000 7f2f87fff640 1 -- 172.21.15.116:0/3706187609 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2f9409c380 con 0x56551dbf7c80 2024-09-12T03:11:01.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.382+0000 7f2fb1377740 10 client.17424 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:01.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.382+0000 7f2fb1377740 1 -- 172.21.15.116:0/3706187609 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56551dc815e0 con 0x56551dbf7c80 2024-09-12T03:11:01.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.383+0000 7f2f87fff640 1 -- 172.21.15.116:0/3706187609 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f9c010bc0 con 0x56551dbf7c80 2024-09-12T03:11:01.403 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:11:01.403 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:11:01.403 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": { 2024-09-12T03:11:01.403 INFO:teuthology.orchestra.run.smithi116.stdout: "33766886-88d2-4a6e-b894-c4169a566de7": { 2024-09-12T03:11:01.403 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/", 2024-09-12T03:11:01.404 INFO:teuthology.orchestra.run.smithi116.stdout: "tag": "33766886-88d2-4a6e-b894-c4169a566de7", 2024-09-12T03:11:01.404 INFO:teuthology.orchestra.run.smithi116.stdout: "options": "recursive,force" 2024-09-12T03:11:01.404 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:11:01.404 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:11:01.404 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:11:01.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.383+0000 7f2f87fff640 1 -- 172.21.15.116:0/3706187609 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2f9409bee0 con 0x56551dbf7c80 2024-09-12T03:11:01.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.383+0000 7f2fb1377740 10 client.17424 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:01.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.383+0000 7f2fb1377740 10 client.17424 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:11:01.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.383+0000 7f2fb1377740 20 client.17424 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:01.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.383+0000 7f2fb1377740 1 --2- 172.21.15.116:0/3706187609 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56551dc818d0 0x56551dc83cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:01.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.383+0000 7f2fb1377740 4 client.17424 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:11:01.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.383+0000 7f2fb1377740 1 -- 172.21.15.116:0/3706187609 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56551db64d60 con 0x56551dc818d0 2024-09-12T03:11:01.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.383+0000 7f2facd3a640 1 --2- 172.21.15.116:0/3706187609 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56551dc818d0 0x56551dc83cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:01.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.383+0000 7f2f87fff640 10 client.17424 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:01.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.383+0000 7f2facd3a640 1 --2- 172.21.15.116:0/3706187609 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56551dc818d0 0x56551dc83cc0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efdeae5ec99ccca5 server_cookie=80ce7da665fe93a6 in_seq=0 out_seq=0 2024-09-12T03:11:01.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.384+0000 7f2f87fff640 1 -- 172.21.15.116:0/3706187609 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56551db64d60 con 0x56551dc818d0 2024-09-12T03:11:01.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.384+0000 7f2f87fff640 10 client.17424 handle_command_reply: tid=0 2024-09-12T03:11:01.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.384+0000 7f2fb1377740 1 client.17424 shutdown 2024-09-12T03:11:01.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.384+0000 7f2fb1377740 1 -- 172.21.15.116:0/3706187609 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56551dbf7c80 msgr2=0x56551dc0ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:01.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.384+0000 7f2fb1377740 1 --2- 172.21.15.116:0/3706187609 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56551dbf7c80 0x56551dc0ad50 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f2f9401c370 tx=0x7f2f94004ae0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.384+0000 7f2fb1377740 1 -- 172.21.15.116:0/3706187609 shutdown_connections 2024-09-12T03:11:01.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.384+0000 7f2fb1377740 1 --2- 172.21.15.116:0/3706187609 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56551dc818d0 0x56551dc83cc0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.384+0000 7f2fb1377740 1 --2- 172.21.15.116:0/3706187609 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56551dc06a90 0x56551dc0b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.384+0000 7f2fb1377740 1 --2- 172.21.15.116:0/3706187609 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56551dbf7c80 0x56551dc0ad50 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.384+0000 7f2f87fff640 0 client.17424 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:01.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.384+0000 7f2fb1377740 1 --2- 172.21.15.116:0/3706187609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56551dbf7360 0x56551dc0a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.384+0000 7f2fb1377740 1 -- 172.21.15.116:0/3706187609 >> 172.21.15.116:0/3706187609 conn(0x56551dbf3a30 msgr2=0x56551dbf2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:01.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.384+0000 7f2fb1377740 1 -- 172.21.15.116:0/3706187609 shutdown_connections 2024-09-12T03:11:01.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.384+0000 7f2fb1377740 1 -- 172.21.15.116:0/3706187609 wait complete. 2024-09-12T03:11:01.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.384+0000 7f2fb1377740 20 client.17424 trim_cache size 0 max 16384 2024-09-12T03:11:01.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.384+0000 7f2faef9e640 1 -- 172.21.15.116:0/3732649250 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2f980a2b80 msgr2=0x7f2f980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:01.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.384+0000 7f2faef9e640 1 --2- 172.21.15.116:0/3732649250 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2f980a2b80 0x7f2f980a5040 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0005e30 tx=0x7f2fa0031770 comp rx=0 tx=0).stop 2024-09-12T03:11:01.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.384+0000 7f2faef9e640 1 -- 172.21.15.116:0/3732649250 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fa8101420 msgr2=0x7f2fa81988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:01.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.384+0000 7f2faef9e640 1 --2- 172.21.15.116:0/3732649250 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fa8101420 0x7f2fa81988c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400e9f0 tx=0x7f2f9400edf0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.385+0000 7f2faef9e640 1 -- 172.21.15.116:0/3732649250 shutdown_connections 2024-09-12T03:11:01.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.385+0000 7f2faef9e640 1 --2- 172.21.15.116:0/3732649250 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2f980a2b80 0x7f2f980a5040 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.385+0000 7f2faef9e640 1 --2- 172.21.15.116:0/3732649250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fa8106a80 0x7f2fa819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.385+0000 7f2faef9e640 1 --2- 172.21.15.116:0/3732649250 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fa8101420 0x7f2fa81988c0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.385+0000 7f2faef9e640 1 --2- 172.21.15.116:0/3732649250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2fa8100a80 0x7f2fa8198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.385+0000 7f2faef9e640 1 -- 172.21.15.116:0/3732649250 >> 172.21.15.116:0/3732649250 conn(0x7f2fa80fc7a0 msgr2=0x7f2fa806a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:01.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.385+0000 7f2faef9e640 1 -- 172.21.15.116:0/3732649250 shutdown_connections 2024-09-12T03:11:01.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:01.385+0000 7f2faef9e640 1 -- 172.21.15.116:0/3732649250 wait complete. 2024-09-12T03:11:01.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:01 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:11:01.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:01 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:11:01.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:01 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:11:01.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:01 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:11:01.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:01 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:11:01.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:01 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:11:01.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:01 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:11:01.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:01 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:11:02.189 INFO:tasks.cephfs.filesystem:scrub status for tag:33766886-88d2-4a6e-b894-c4169a566de7 - {'path': '/', 'tag': '33766886-88d2-4a6e-b894-c4169a566de7', 'options': 'recursive,force'} 2024-09-12T03:11:02.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:02 smithi116 ceph-mon[31410]: pgmap v579: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-09-12T03:11:02.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:02 smithi183 ceph-mon[35535]: pgmap v579: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-09-12T03:11:02.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:02 smithi177 ceph-mon[35305]: pgmap v579: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-09-12T03:11:03.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:03 smithi183 ceph-mon[35535]: pgmap v580: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:11:03.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:03 smithi177 ceph-mon[35305]: pgmap v580: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:11:03.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:03 smithi116 ceph-mon[31410]: pgmap v580: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:11:04.276 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:11:04.538 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:11:04.659 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:04 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:11:04.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:04 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:11:04.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:04 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:11:05.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.016+0000 7fd72d921640 1 -- 172.21.15.116:0/615883641 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd728108300 msgr2=0x7fd72810ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:05.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.016+0000 7fd72d921640 1 --2- 172.21.15.116:0/615883641 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd728108300 0x7fd72810ec70 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fd720009930 tx=0x7fd72002a440 comp rx=0 tx=0).stop 2024-09-12T03:11:05.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.016+0000 7fd72d921640 1 -- 172.21.15.116:0/615883641 shutdown_connections 2024-09-12T03:11:05.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.016+0000 7fd72d921640 1 --2- 172.21.15.116:0/615883641 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd728108300 0x7fd72810ec70 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:05.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.016+0000 7fd72d921640 1 --2- 172.21.15.116:0/615883641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd728103780 0x7fd728107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:05.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.016+0000 7fd72d921640 1 --2- 172.21.15.116:0/615883641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd728102dd0 0x7fd7281031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:05.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.016+0000 7fd72d921640 1 -- 172.21.15.116:0/615883641 >> 172.21.15.116:0/615883641 conn(0x7fd7280fc7d0 msgr2=0x7fd7280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:05.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.016+0000 7fd72d921640 1 -- 172.21.15.116:0/615883641 shutdown_connections 2024-09-12T03:11:05.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.016+0000 7fd72d921640 1 -- 172.21.15.116:0/615883641 wait complete. 2024-09-12T03:11:05.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.016+0000 7fd72d921640 1 Processor -- start 2024-09-12T03:11:05.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.017+0000 7fd72d921640 1 -- start start 2024-09-12T03:11:05.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.017+0000 7fd72d921640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd728102dd0 0x7fd7281986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:05.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.017+0000 7fd72d921640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd728103780 0x7fd728198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:05.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.017+0000 7fd72d921640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd728108300 0x7fd72819cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:05.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.017+0000 7fd72d921640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd728112170 con 0x7fd728108300 2024-09-12T03:11:05.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.017+0000 7fd72d921640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd728111ff0 con 0x7fd728103780 2024-09-12T03:11:05.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.017+0000 7fd72d921640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd7281122f0 con 0x7fd728102dd0 2024-09-12T03:11:05.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.017+0000 7fd7277fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd728108300 0x7fd72819cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:05.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.017+0000 7fd7277fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd728108300 0x7fd72819cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58494/0 (socket says 172.21.15.116:58494) 2024-09-12T03:11:05.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.017+0000 7fd7277fe640 1 -- 172.21.15.116:0/937166070 learned_addr learned my addr 172.21.15.116:0/937166070 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:05.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.017+0000 7fd726ffd640 1 --2- 172.21.15.116:0/937166070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd728102dd0 0x7fd7281986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:05.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.017+0000 7fd7267fc640 1 --2- 172.21.15.116:0/937166070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd728103780 0x7fd728198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:05.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.017+0000 7fd7277fe640 1 -- 172.21.15.116:0/937166070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd728102dd0 msgr2=0x7fd7281986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:05.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.017+0000 7fd7277fe640 1 --2- 172.21.15.116:0/937166070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd728102dd0 0x7fd7281986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:05.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.017+0000 7fd7277fe640 1 -- 172.21.15.116:0/937166070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd728103780 msgr2=0x7fd728198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:05.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.017+0000 7fd7277fe640 1 --2- 172.21.15.116:0/937166070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd728103780 0x7fd728198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:05.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.017+0000 7fd7277fe640 1 -- 172.21.15.116:0/937166070 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72819d680 con 0x7fd728108300 2024-09-12T03:11:05.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.017+0000 7fd7267fc640 1 --2- 172.21.15.116:0/937166070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd728103780 0x7fd728198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:11:05.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.017+0000 7fd7277fe640 1 --2- 172.21.15.116:0/937166070 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd728108300 0x7fd72819cfa0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fd720005ab0 tx=0x7fd7200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:05.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.018+0000 7fd72c91f640 1 -- 172.21.15.116:0/937166070 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72003f030 con 0x7fd728108300 2024-09-12T03:11:05.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.018+0000 7fd72c91f640 1 -- 172.21.15.116:0/937166070 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd720003ce0 con 0x7fd728108300 2024-09-12T03:11:05.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.018+0000 7fd72c91f640 1 -- 172.21.15.116:0/937166070 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd720035070 con 0x7fd728108300 2024-09-12T03:11:05.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.018+0000 7fd72d921640 1 -- 172.21.15.116:0/937166070 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd72819d910 con 0x7fd728108300 2024-09-12T03:11:05.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.018+0000 7fd72d921640 1 -- 172.21.15.116:0/937166070 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7281a5250 con 0x7fd728108300 2024-09-12T03:11:05.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.019+0000 7fd72d921640 1 -- 172.21.15.116:0/937166070 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd728103230 con 0x7fd728108300 2024-09-12T03:11:05.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.020+0000 7fd72c91f640 1 -- 172.21.15.116:0/937166070 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd720042030 con 0x7fd728108300 2024-09-12T03:11:05.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.021+0000 7fd72c91f640 1 --2- 172.21.15.116:0/937166070 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd7140a2b80 0x7fd7140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:05.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.021+0000 7fd726ffd640 1 --2- 172.21.15.116:0/937166070 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd7140a2b80 0x7fd7140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:05.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.021+0000 7fd72c91f640 1 -- 172.21.15.116:0/937166070 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd7200ef5b0 con 0x7fd728108300 2024-09-12T03:11:05.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.021+0000 7fd726ffd640 1 --2- 172.21.15.116:0/937166070 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd7140a2b80 0x7fd7140a5040 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd71800a400 tx=0x7fd71800d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:05.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.025+0000 7fd72c91f640 1 -- 172.21.15.116:0/937166070 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd7200bc540 con 0x7fd728108300 2024-09-12T03:11:05.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.224+0000 7fd72d921640 1 -- 172.21.15.116:0/937166070 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd72810a290 con 0x7fd728108300 2024-09-12T03:11:05.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.225+0000 7fd72c91f640 1 -- 172.21.15.116:0/937166070 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7fd7200bbe40 con 0x7fd728108300 2024-09-12T03:11:05.225 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:11:05.225 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:11:05.226 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:11:05.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.227+0000 7fd72d921640 1 -- 172.21.15.116:0/937166070 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd7140a2b80 msgr2=0x7fd7140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:05.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.227+0000 7fd72d921640 1 --2- 172.21.15.116:0/937166070 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd7140a2b80 0x7fd7140a5040 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd71800a400 tx=0x7fd71800d040 comp rx=0 tx=0).stop 2024-09-12T03:11:05.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.227+0000 7fd72d921640 1 -- 172.21.15.116:0/937166070 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd728108300 msgr2=0x7fd72819cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:05.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.227+0000 7fd72d921640 1 --2- 172.21.15.116:0/937166070 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd728108300 0x7fd72819cfa0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fd720005ab0 tx=0x7fd7200042f0 comp rx=0 tx=0).stop 2024-09-12T03:11:05.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.227+0000 7fd72d921640 1 -- 172.21.15.116:0/937166070 shutdown_connections 2024-09-12T03:11:05.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.227+0000 7fd72d921640 1 --2- 172.21.15.116:0/937166070 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd7140a2b80 0x7fd7140a5040 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:05.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.227+0000 7fd72d921640 1 --2- 172.21.15.116:0/937166070 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd728108300 0x7fd72819cfa0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:05.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.227+0000 7fd72d921640 1 --2- 172.21.15.116:0/937166070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd728103780 0x7fd728198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:05.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.227+0000 7fd72d921640 1 --2- 172.21.15.116:0/937166070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd728102dd0 0x7fd7281986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:05.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.227+0000 7fd72d921640 1 -- 172.21.15.116:0/937166070 >> 172.21.15.116:0/937166070 conn(0x7fd7280fc7d0 msgr2=0x7fd728105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:05.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.227+0000 7fd72d921640 1 -- 172.21.15.116:0/937166070 shutdown_connections 2024-09-12T03:11:05.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:05.227+0000 7fd72d921640 1 -- 172.21.15.116:0/937166070 wait complete. 2024-09-12T03:11:05.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:05 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/937166070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:11:05.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:05 smithi116 ceph-mon[31410]: pgmap v581: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:11:05.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:05 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/937166070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:11:05.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:05 smithi183 ceph-mon[35535]: pgmap v581: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:11:05.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:05 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/937166070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:11:05.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:05 smithi177 ceph-mon[35305]: pgmap v581: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:11:06.054 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id 45ae6e70 --release --await' 2024-09-12T03:11:06.054 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id 45ae6e70 --release --await 2024-09-12T03:11:06.300 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:11:06.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.835+0000 7f8deb5ee640 1 -- 172.21.15.116:0/138010358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8de4100e20 msgr2=0x7f8de4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:06.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.835+0000 7f8deb5ee640 1 --2- 172.21.15.116:0/138010358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8de4100e20 0x7f8de4101200 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f8de0009900 tx=0x7f8de002a480 comp rx=0 tx=0).stop 2024-09-12T03:11:06.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.835+0000 7f8deb5ee640 1 -- 172.21.15.116:0/138010358 shutdown_connections 2024-09-12T03:11:06.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.835+0000 7f8deb5ee640 1 --2- 172.21.15.116:0/138010358 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8de4102ef0 0x7f8de410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.835+0000 7f8deb5ee640 1 --2- 172.21.15.116:0/138010358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8de4102570 0x7f8de41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.835+0000 7f8deb5ee640 1 --2- 172.21.15.116:0/138010358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8de4100e20 0x7f8de4101200 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.835+0000 7f8deb5ee640 1 -- 172.21.15.116:0/138010358 >> 172.21.15.116:0/138010358 conn(0x7f8de40fc7d0 msgr2=0x7f8de40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:06.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.835+0000 7f8deb5ee640 1 -- 172.21.15.116:0/138010358 shutdown_connections 2024-09-12T03:11:06.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.835+0000 7f8deb5ee640 1 -- 172.21.15.116:0/138010358 wait complete. 2024-09-12T03:11:06.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.835+0000 7f8deb5ee640 1 Processor -- start 2024-09-12T03:11:06.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.835+0000 7f8deb5ee640 1 -- start start 2024-09-12T03:11:06.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.836+0000 7f8deb5ee640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8de4100e20 0x7f8de4197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:06.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.836+0000 7f8deb5ee640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8de4102570 0x7f8de4197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:06.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.836+0000 7f8deb5ee640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8de4102ef0 0x7f8de419be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:06.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.836+0000 7f8deb5ee640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8de4111020 con 0x7f8de4102ef0 2024-09-12T03:11:06.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.836+0000 7f8deb5ee640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8de4110ea0 con 0x7f8de4102570 2024-09-12T03:11:06.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.836+0000 7f8deb5ee640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8de41111a0 con 0x7f8de4100e20 2024-09-12T03:11:06.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.836+0000 7f8de9b8b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8de4102ef0 0x7f8de419be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:06.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.836+0000 7f8de9b8b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8de4102ef0 0x7f8de419be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58508/0 (socket says 172.21.15.116:58508) 2024-09-12T03:11:06.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.836+0000 7f8de9b8b640 1 -- 172.21.15.116:0/1002421016 learned_addr learned my addr 172.21.15.116:0/1002421016 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:06.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.836+0000 7f8de938a640 1 --2- 172.21.15.116:0/1002421016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8de4100e20 0x7f8de4197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:06.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.836+0000 7f8de8b89640 1 --2- 172.21.15.116:0/1002421016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8de4102570 0x7f8de4197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:06.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.836+0000 7f8de9b8b640 1 -- 172.21.15.116:0/1002421016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8de4100e20 msgr2=0x7f8de4197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:06.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.836+0000 7f8de9b8b640 1 --2- 172.21.15.116:0/1002421016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8de4100e20 0x7f8de4197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.836+0000 7f8de9b8b640 1 -- 172.21.15.116:0/1002421016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8de4102570 msgr2=0x7f8de4197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:06.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.836+0000 7f8de9b8b640 1 --2- 172.21.15.116:0/1002421016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8de4102570 0x7f8de4197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.836+0000 7f8de9b8b640 1 -- 172.21.15.116:0/1002421016 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de419c4f0 con 0x7f8de4102ef0 2024-09-12T03:11:06.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.836+0000 7f8de8b89640 1 --2- 172.21.15.116:0/1002421016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8de4102570 0x7f8de4197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:06.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.836+0000 7f8de9b8b640 1 --2- 172.21.15.116:0/1002421016 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8de4102ef0 0x7f8de419be10 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc00b970 tx=0x7f8ddc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:06.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.836+0000 7f8dda7fc640 1 -- 172.21.15.116:0/1002421016 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc019070 con 0x7f8de4102ef0 2024-09-12T03:11:06.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.836+0000 7f8dda7fc640 1 -- 172.21.15.116:0/1002421016 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ddc00cb40 con 0x7f8de4102ef0 2024-09-12T03:11:06.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.836+0000 7f8dda7fc640 1 -- 172.21.15.116:0/1002421016 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc007650 con 0x7f8de4102ef0 2024-09-12T03:11:06.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.836+0000 7f8deb5ee640 1 -- 172.21.15.116:0/1002421016 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8de419c7e0 con 0x7f8de4102ef0 2024-09-12T03:11:06.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.837+0000 7f8deb5ee640 1 -- 172.21.15.116:0/1002421016 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8de41a40c0 con 0x7f8de4102ef0 2024-09-12T03:11:06.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.839+0000 7f8dda7fc640 1 -- 172.21.15.116:0/1002421016 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8ddc01c070 con 0x7f8de4102ef0 2024-09-12T03:11:06.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.839+0000 7f8ded9c7740 1 Processor -- start 2024-09-12T03:11:06.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.839+0000 7f8ded9c7740 1 -- start start 2024-09-12T03:11:06.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.839+0000 7f8ded9c7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362183270 0x565362183650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:06.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.839+0000 7f8ded9c7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565362183b90 0x56536218df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:06.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.839+0000 7f8ded9c7740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56536218e450 0x565362190840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:06.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.839+0000 7f8ded9c7740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5653620e6620 con 0x56536218e450 2024-09-12T03:11:06.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.840+0000 7f8ded9c7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56536208de50 con 0x565362183b90 2024-09-12T03:11:06.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.840+0000 7f8ded9c7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5653620e25e0 con 0x565362183270 2024-09-12T03:11:06.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.840+0000 7f8dda7fc640 1 --2- 172.21.15.116:0/1002421016 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8dd00a2b80 0x7f8dd00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:06.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.840+0000 7f8de8b89640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56536218e450 0x565362190840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:06.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.840+0000 7f8de8b89640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56536218e450 0x565362190840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58520/0 (socket says 172.21.15.116:58520) 2024-09-12T03:11:06.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.840+0000 7f8de8b89640 1 -- 172.21.15.116:0/3287822526 learned_addr learned my addr 172.21.15.116:0/3287822526 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:06.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.840+0000 7f8de9b8b640 1 --2- 172.21.15.116:0/1002421016 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8dd00a2b80 0x7f8dd00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:06.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.840+0000 7f8dda7fc640 1 -- 172.21.15.116:0/1002421016 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f8ddc0ce600 con 0x7f8de4102ef0 2024-09-12T03:11:06.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.840+0000 7f8de8b89640 1 --2- 172.21.15.116:0/3287822526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362183270 0x565362183650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:06.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.840+0000 7f8de938a640 1 --2- 172.21.15.116:0/3287822526 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565362183b90 0x56536218df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:06.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.840+0000 7f8de8b89640 1 -- 172.21.15.116:0/3287822526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362183270 msgr2=0x565362183650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:06.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.840+0000 7f8de8b89640 1 --2- 172.21.15.116:0/3287822526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362183270 0x565362183650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.840+0000 7f8de8b89640 1 -- 172.21.15.116:0/3287822526 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565362183b90 msgr2=0x56536218df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:06.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.840+0000 7f8de8b89640 1 --2- 172.21.15.116:0/3287822526 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565362183b90 0x56536218df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.840+0000 7f8de8b89640 1 -- 172.21.15.116:0/3287822526 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56536207ed40 con 0x56536218e450 2024-09-12T03:11:06.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.840+0000 7f8de8b89640 1 --2- 172.21.15.116:0/3287822526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56536218e450 0x565362190840 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f8dd400cf10 tx=0x7f8dd400c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d9dfca7882ac7f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:06.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.840+0000 7f8de9b8b640 1 --2- 172.21.15.116:0/1002421016 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8dd00a2b80 0x7f8dd00a5040 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc09ac40 tx=0x7f8ddc090420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:06.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.840+0000 7f8dcf7fe640 1 -- 172.21.15.116:0/3287822526 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd401b070 con 0x56536218e450 2024-09-12T03:11:06.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.841+0000 7f8dcf7fe640 1 -- 172.21.15.116:0/3287822526 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8dd400ee40 con 0x56536218e450 2024-09-12T03:11:06.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.841+0000 7f8dcf7fe640 1 -- 172.21.15.116:0/3287822526 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd400f040 con 0x56536218e450 2024-09-12T03:11:06.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.841+0000 7f8ded9c7740 1 -- 172.21.15.116:0/3287822526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56536218e450 msgr2=0x565362190840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:06.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.841+0000 7f8ded9c7740 1 --2- 172.21.15.116:0/3287822526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56536218e450 0x565362190840 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f8dd400cf10 tx=0x7f8dd400c410 comp rx=0 tx=0).stop 2024-09-12T03:11:06.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.841+0000 7f8ded9c7740 1 -- 172.21.15.116:0/3287822526 shutdown_connections 2024-09-12T03:11:06.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.841+0000 7f8ded9c7740 1 --2- 172.21.15.116:0/3287822526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56536218e450 0x565362190840 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.841+0000 7f8ded9c7740 1 --2- 172.21.15.116:0/3287822526 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565362183b90 0x56536218df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.841+0000 7f8ded9c7740 1 --2- 172.21.15.116:0/3287822526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362183270 0x565362183650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.841+0000 7f8ded9c7740 1 -- 172.21.15.116:0/3287822526 >> 172.21.15.116:0/3287822526 conn(0x56536217f800 msgr2=0x56536217fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:06.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.841+0000 7f8ded9c7740 1 -- 172.21.15.116:0/3287822526 shutdown_connections 2024-09-12T03:11:06.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.841+0000 7f8ded9c7740 1 -- 172.21.15.116:0/3287822526 wait complete. 2024-09-12T03:11:06.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.842+0000 7f8ded9c7740 1 -- start start 2024-09-12T03:11:06.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.842+0000 7f8ded9c7740 1 Processor -- start 2024-09-12T03:11:06.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.842+0000 7f8ded9c7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565362183270 0x56536218b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:06.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.842+0000 7f8ded9c7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362183b90 0x56536218b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:06.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.842+0000 7f8ded9c7740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56536218e450 0x56536218be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:06.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.842+0000 7f8ded9c7740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x565362191020 con 0x56536218e450 2024-09-12T03:11:06.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.842+0000 7f8ded9c7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56536206ef10 con 0x565362183270 2024-09-12T03:11:06.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.842+0000 7f8ded9c7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5653621911a0 con 0x565362183b90 2024-09-12T03:11:06.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.842+0000 7f8ded9c7740 10 client.0 _ll_register_callbacks cb 0x565362092db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:06.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.842+0000 7f8de8b89640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56536218e450 0x56536218be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:06.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.842+0000 7f8de8b89640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56536218e450 0x56536218be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58522/0 (socket says 172.21.15.116:58522) 2024-09-12T03:11:06.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.842+0000 7f8de8b89640 1 -- 172.21.15.116:0/1169704451 learned_addr learned my addr 172.21.15.116:0/1169704451 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:06.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.842+0000 7f8de8b89640 1 -- 172.21.15.116:0/1169704451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362183b90 msgr2=0x56536218b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:06.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.842+0000 7f8de938a640 1 --2- 172.21.15.116:0/1169704451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362183b90 0x56536218b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:06.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.842+0000 7f8de8b89640 1 --2- 172.21.15.116:0/1169704451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362183b90 0x56536218b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.842+0000 7f8de8b89640 1 -- 172.21.15.116:0/1169704451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565362183270 msgr2=0x56536218b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:06.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.842+0000 7f8de8b89640 1 --2- 172.21.15.116:0/1169704451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565362183270 0x56536218b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.842+0000 7f8de8b89640 1 -- 172.21.15.116:0/1169704451 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565361df5580 con 0x56536218e450 2024-09-12T03:11:06.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.843+0000 7f8de938a640 1 --2- 172.21.15.116:0/1169704451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362183b90 0x56536218b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:06.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.843+0000 7f8de8b89640 1 --2- 172.21.15.116:0/1169704451 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56536218e450 0x56536218be40 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4006c30 tx=0x7f8dd4015390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a60f0adbb6763b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:06.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.843+0000 7f8dccff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:06.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.843+0000 7f8dccff9640 1 -- 172.21.15.116:0/1169704451 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd4027060 con 0x56536218e450 2024-09-12T03:11:06.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.843+0000 7f8ded9c7740 1 -- 172.21.15.116:0/1169704451 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5653620f3b90 con 0x56536218e450 2024-09-12T03:11:06.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.843+0000 7f8dccff9640 1 -- 172.21.15.116:0/1169704451 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f8dd40155b0 con 0x56536218e450 2024-09-12T03:11:06.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.843+0000 7f8dccff9640 1 -- 172.21.15.116:0/1169704451 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8dd401b040 con 0x56536218e450 2024-09-12T03:11:06.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.843+0000 7f8ded9c7740 10 client.17442 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:06.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.843+0000 7f8ded9c7740 1 -- 172.21.15.116:0/1169704451 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x565361ebf960 con 0x56536218e450 2024-09-12T03:11:06.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.843+0000 7f8dccff9640 1 -- 172.21.15.116:0/1169704451 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5653620f3b90 con 0x56536218e450 2024-09-12T03:11:06.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.843+0000 7f8dccff9640 1 -- 172.21.15.116:0/1169704451 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8dd4015070 con 0x56536218e450 2024-09-12T03:11:06.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.843+0000 7f8ded9c7740 10 client.17442 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:06.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.843+0000 7f8ded9c7740 10 client.17442 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:11:06.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.843+0000 7f8ded9c7740 20 client.17442 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:06.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.843+0000 7f8ded9c7740 1 --2- 172.21.15.116:0/1169704451 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56536219df90 0x5653621a0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:06.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.843+0000 7f8ded9c7740 4 client.17442 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:11:06.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.843+0000 7f8ded9c7740 1 -- 172.21.15.116:0/1169704451 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653620ee0a0 con 0x56536219df90 2024-09-12T03:11:06.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.844+0000 7f8de938a640 1 --2- 172.21.15.116:0/1169704451 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56536219df90 0x5653621a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:06.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.844+0000 7f8de938a640 1 --2- 172.21.15.116:0/1169704451 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56536219df90 0x5653621a0380 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=704c96d7a758b01 server_cookie=1cc1350aadfe5554 in_seq=0 out_seq=0 2024-09-12T03:11:06.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.844+0000 7f8dccff9640 10 client.17442 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:06.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.845+0000 7f8dccff9640 1 -- 172.21.15.116:0/1169704451 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5653620ee0a0 con 0x56536219df90 2024-09-12T03:11:06.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.845+0000 7f8dccff9640 10 client.17442 handle_command_reply: tid=0 2024-09-12T03:11:06.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.845+0000 7f8ded9c7740 1 client.17442 shutdown 2024-09-12T03:11:06.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.845+0000 7f8ded9c7740 1 -- 172.21.15.116:0/1169704451 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56536218e450 msgr2=0x56536218be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:06.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.845+0000 7f8ded9c7740 1 --2- 172.21.15.116:0/1169704451 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56536218e450 0x56536218be40 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4006c30 tx=0x7f8dd4015390 comp rx=0 tx=0).stop 2024-09-12T03:11:06.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.845+0000 7f8ded9c7740 1 -- 172.21.15.116:0/1169704451 shutdown_connections 2024-09-12T03:11:06.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.845+0000 7f8ded9c7740 1 --2- 172.21.15.116:0/1169704451 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56536219df90 0x5653621a0380 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.845+0000 7f8ded9c7740 1 --2- 172.21.15.116:0/1169704451 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56536218e450 0x56536218be40 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.845+0000 7f8ded9c7740 1 --2- 172.21.15.116:0/1169704451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362183b90 0x56536218b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.845+0000 7f8ded9c7740 1 --2- 172.21.15.116:0/1169704451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565362183270 0x56536218b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.845+0000 7f8ded9c7740 1 -- 172.21.15.116:0/1169704451 >> 172.21.15.116:0/1169704451 conn(0x56536217f800 msgr2=0x56536217c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:06.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.845+0000 7f8dccff9640 0 client.17442 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:06.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.845+0000 7f8ded9c7740 1 -- 172.21.15.116:0/1169704451 shutdown_connections 2024-09-12T03:11:06.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.845+0000 7f8ded9c7740 1 -- 172.21.15.116:0/1169704451 wait complete. 2024-09-12T03:11:06.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.845+0000 7f8ded9c7740 20 client.17442 trim_cache size 0 max 16384 2024-09-12T03:11:06.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.862+0000 7f8ded9c7740 1 Processor -- start 2024-09-12T03:11:06.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.862+0000 7f8ded9c7740 1 -- start start 2024-09-12T03:11:06.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.862+0000 7f8ded9c7740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56536217f800 0x5653621a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:06.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.862+0000 7f8ded9c7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565362183270 0x5653621a5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:06.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.862+0000 7f8ded9c7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362183b90 0x565362188620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:06.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.862+0000 7f8ded9c7740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8de41adcd0 con 0x56536217f800 2024-09-12T03:11:06.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.862+0000 7f8ded9c7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8de41ab5a0 con 0x565362183270 2024-09-12T03:11:06.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.862+0000 7f8ded9c7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8de41a9410 con 0x565362183b90 2024-09-12T03:11:06.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.862+0000 7f8de8b89640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56536217f800 0x5653621a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:06.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.862+0000 7f8de8b89640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56536217f800 0x5653621a5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58528/0 (socket says 172.21.15.116:58528) 2024-09-12T03:11:06.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.862+0000 7f8de8b89640 1 -- 172.21.15.116:0/3801843976 learned_addr learned my addr 172.21.15.116:0/3801843976 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:06.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.862+0000 7f8de938a640 1 --2- 172.21.15.116:0/3801843976 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565362183270 0x5653621a5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:06.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.862+0000 7f8de8b89640 1 -- 172.21.15.116:0/3801843976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362183b90 msgr2=0x565362188620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:06.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.863+0000 7f8de8b89640 1 --2- 172.21.15.116:0/3801843976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362183b90 0x565362188620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.863+0000 7f8de8b89640 1 -- 172.21.15.116:0/3801843976 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565362183270 msgr2=0x5653621a5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:06.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.863+0000 7f8de8b89640 1 --2- 172.21.15.116:0/3801843976 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565362183270 0x5653621a5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.863+0000 7f8de8b89640 1 -- 172.21.15.116:0/3801843976 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653621839c0 con 0x56536217f800 2024-09-12T03:11:06.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.863+0000 7f8de938a640 1 --2- 172.21.15.116:0/3801843976 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565362183270 0x5653621a5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:06.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.863+0000 7f8de8b89640 1 --2- 172.21.15.116:0/3801843976 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56536217f800 0x5653621a5aa0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40154a0 tx=0x7f8dd400c2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df380413304a9d98 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:06.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.863+0000 7f8dceffd640 1 -- 172.21.15.116:0/3801843976 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd4032070 con 0x56536217f800 2024-09-12T03:11:06.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.863+0000 7f8dceffd640 1 -- 172.21.15.116:0/3801843976 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8dd4034070 con 0x56536217f800 2024-09-12T03:11:06.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.863+0000 7f8dceffd640 1 -- 172.21.15.116:0/3801843976 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd400c040 con 0x56536217f800 2024-09-12T03:11:06.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.863+0000 7f8ded9c7740 1 -- 172.21.15.116:0/3801843976 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56536217f800 msgr2=0x5653621a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:06.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.863+0000 7f8ded9c7740 1 --2- 172.21.15.116:0/3801843976 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56536217f800 0x5653621a5aa0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40154a0 tx=0x7f8dd400c2e0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.864+0000 7f8ded9c7740 1 -- 172.21.15.116:0/3801843976 shutdown_connections 2024-09-12T03:11:06.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.864+0000 7f8ded9c7740 1 --2- 172.21.15.116:0/3801843976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362183b90 0x565362188620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.864+0000 7f8ded9c7740 1 --2- 172.21.15.116:0/3801843976 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565362183270 0x5653621a5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.864+0000 7f8ded9c7740 1 --2- 172.21.15.116:0/3801843976 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56536217f800 0x5653621a5aa0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.864+0000 7f8ded9c7740 1 -- 172.21.15.116:0/3801843976 >> 172.21.15.116:0/3801843976 conn(0x565362186db0 msgr2=0x56536217bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:06.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.864+0000 7f8ded9c7740 1 -- 172.21.15.116:0/3801843976 shutdown_connections 2024-09-12T03:11:06.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.864+0000 7f8ded9c7740 1 -- 172.21.15.116:0/3801843976 wait complete. 2024-09-12T03:11:06.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.864+0000 7f8ded9c7740 1 -- start start 2024-09-12T03:11:06.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.864+0000 7f8ded9c7740 1 Processor -- start 2024-09-12T03:11:06.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.864+0000 7f8ded9c7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56536217f800 0x565362192370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:06.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.864+0000 7f8ded9c7740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565362183270 0x5653621928b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:06.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.864+0000 7f8ded9c7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565362183b90 0x565362192df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:06.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.864+0000 7f8ded9c7740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8de41a9a60 con 0x565362183270 2024-09-12T03:11:06.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.864+0000 7f8ded9c7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8de41aaed0 con 0x565362183b90 2024-09-12T03:11:06.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.864+0000 7f8ded9c7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8de41aa130 con 0x56536217f800 2024-09-12T03:11:06.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.864+0000 7f8ded9c7740 10 client.0 _ll_register_callbacks cb 0x7f8dd4016740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:06.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.864+0000 7f8de938a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565362183270 0x5653621928b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:06.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.864+0000 7f8de938a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565362183270 0x5653621928b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58534/0 (socket says 172.21.15.116:58534) 2024-09-12T03:11:06.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.865+0000 7f8de938a640 1 -- 172.21.15.116:0/4004354675 learned_addr learned my addr 172.21.15.116:0/4004354675 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:06.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.865+0000 7f8de8b89640 1 --2- 172.21.15.116:0/4004354675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565362183b90 0x565362192df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:06.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.865+0000 7f8de8b89640 1 --2- 172.21.15.116:0/4004354675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56536217f800 0x565362192370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:06.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.865+0000 7f8de938a640 1 -- 172.21.15.116:0/4004354675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56536217f800 msgr2=0x565362192370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:06.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.865+0000 7f8de938a640 1 --2- 172.21.15.116:0/4004354675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56536217f800 0x565362192370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.865+0000 7f8de938a640 1 -- 172.21.15.116:0/4004354675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565362183b90 msgr2=0x565362192df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:06.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.865+0000 7f8de938a640 1 --2- 172.21.15.116:0/4004354675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565362183b90 0x565362192df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.865+0000 7f8de938a640 1 -- 172.21.15.116:0/4004354675 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565362207c70 con 0x565362183270 2024-09-12T03:11:06.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.865+0000 7f8de8b89640 1 --2- 172.21.15.116:0/4004354675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565362183b90 0x565362192df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:11:06.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.865+0000 7f8de938a640 1 --2- 172.21.15.116:0/4004354675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565362183270 0x5653621928b0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f8de002a9a0 tx=0x7f8de0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a29dbfb3bab4f861 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:06.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.865+0000 7f8dccff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:06.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.865+0000 7f8dccff9640 1 -- 172.21.15.116:0/4004354675 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de0048030 con 0x565362183270 2024-09-12T03:11:06.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.865+0000 7f8dccff9640 1 -- 172.21.15.116:0/4004354675 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f8de00038c0 con 0x565362183270 2024-09-12T03:11:06.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.865+0000 7f8ded9c7740 1 -- 172.21.15.116:0/4004354675 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8dd4034070 con 0x565362183270 2024-09-12T03:11:06.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.865+0000 7f8dccff9640 1 -- 172.21.15.116:0/4004354675 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8de0035040 con 0x565362183270 2024-09-12T03:11:06.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.865+0000 7f8ded9c7740 10 client.17448 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:06.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.865+0000 7f8ded9c7740 1 -- 172.21.15.116:0/4004354675 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x565362208f50 con 0x565362183270 2024-09-12T03:11:06.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.865+0000 7f8dccff9640 1 -- 172.21.15.116:0/4004354675 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8dd4034070 con 0x565362183270 2024-09-12T03:11:06.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.866+0000 7f8dccff9640 1 -- 172.21.15.116:0/4004354675 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8de0044340 con 0x565362183270 2024-09-12T03:11:06.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.866+0000 7f8ded9c7740 10 client.17448 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:06.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.866+0000 7f8ded9c7740 10 client.17448 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:11:06.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.866+0000 7f8ded9c7740 20 client.17448 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:06.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.866+0000 7f8ded9c7740 1 --2- 172.21.15.116:0/4004354675 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56536218e450 0x56536220b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:06.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.866+0000 7f8ded9c7740 4 client.17448 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "45ae6e70", "release": true, "await": true}] 2024-09-12T03:11:06.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.866+0000 7f8ded9c7740 1 -- 172.21.15.116:0/4004354675 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "quiesce db", "set_id": "45ae6e70", "release": true, "await": true}) -- 0x56536206a5a0 con 0x56536218e450 2024-09-12T03:11:06.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.866+0000 7f8de8b89640 1 --2- 172.21.15.116:0/4004354675 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56536218e450 0x56536220b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:06.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.866+0000 7f8de8b89640 1 --2- 172.21.15.116:0/4004354675 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56536218e450 0x56536220b740 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df69c154b283d73a server_cookie=c05c5b905d907d6d in_seq=0 out_seq=0 2024-09-12T03:11:06.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.866+0000 7f8dccff9640 10 client.17448 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:06.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.869+0000 7f8dccff9640 1 -- 172.21.15.116:0/4004354675 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56536206a5a0 con 0x56536218e450 2024-09-12T03:11:06.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.869+0000 7f8dccff9640 10 client.17448 handle_command_reply: tid=0 2024-09-12T03:11:06.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.869+0000 7f8ded9c7740 1 client.17448 shutdown 2024-09-12T03:11:06.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.869+0000 7f8ded9c7740 1 -- 172.21.15.116:0/4004354675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565362183270 msgr2=0x5653621928b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:06.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.869+0000 7f8ded9c7740 1 --2- 172.21.15.116:0/4004354675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565362183270 0x5653621928b0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f8de002a9a0 tx=0x7f8de0005e80 comp rx=0 tx=0).stop 2024-09-12T03:11:06.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.869+0000 7f8ded9c7740 1 -- 172.21.15.116:0/4004354675 shutdown_connections 2024-09-12T03:11:06.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.869+0000 7f8ded9c7740 1 --2- 172.21.15.116:0/4004354675 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56536218e450 0x56536220b740 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.869+0000 7f8ded9c7740 1 --2- 172.21.15.116:0/4004354675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x565362183b90 0x565362192df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.869+0000 7f8ded9c7740 1 --2- 172.21.15.116:0/4004354675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565362183270 0x5653621928b0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.869+0000 7f8dccff9640 0 client.17448 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:06.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.869+0000 7f8ded9c7740 1 --2- 172.21.15.116:0/4004354675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56536217f800 0x565362192370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.869+0000 7f8ded9c7740 1 -- 172.21.15.116:0/4004354675 >> 172.21.15.116:0/4004354675 conn(0x565362186db0 msgr2=0x56536218b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:06.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.869+0000 7f8ded9c7740 1 -- 172.21.15.116:0/4004354675 shutdown_connections 2024-09-12T03:11:06.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.869+0000 7f8ded9c7740 1 -- 172.21.15.116:0/4004354675 wait complete. 2024-09-12T03:11:06.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.869+0000 7f8ded9c7740 20 client.17448 trim_cache size 0 max 16384 2024-09-12T03:11:06.881 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:11:06.882 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 25, 2024-09-12T03:11:06.882 INFO:teuthology.orchestra.run.smithi116.stdout: "leader": 24494, 2024-09-12T03:11:06.882 INFO:teuthology.orchestra.run.smithi116.stdout: "set_version": 32, 2024-09-12T03:11:06.882 INFO:teuthology.orchestra.run.smithi116.stdout: "sets": { 2024-09-12T03:11:06.882 INFO:teuthology.orchestra.run.smithi116.stdout: "45ae6e70": { 2024-09-12T03:11:06.882 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 32, 2024-09-12T03:11:06.882 INFO:teuthology.orchestra.run.smithi116.stdout: "age_ref": 0.0, 2024-09-12T03:11:06.883 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:11:06.883 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RELEASED", 2024-09-12T03:11:06.883 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:11:06.883 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:11:06.883 INFO:teuthology.orchestra.run.smithi116.stdout: "timeout": 90.0, 2024-09-12T03:11:06.883 INFO:teuthology.orchestra.run.smithi116.stdout: "expiration": 139.4, 2024-09-12T03:11:06.883 INFO:teuthology.orchestra.run.smithi116.stdout: "members": { 2024-09-12T03:11:06.883 INFO:teuthology.orchestra.run.smithi116.stdout: "file:/": { 2024-09-12T03:11:06.883 INFO:teuthology.orchestra.run.smithi116.stdout: "excluded": false, 2024-09-12T03:11:06.883 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:11:06.883 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RELEASED", 2024-09-12T03:11:06.883 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:11:06.883 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:11:06.884 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:11:06.884 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:11:06.884 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:11:06.884 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:11:06.884 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:11:06.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.870+0000 7f8deb5ee640 1 -- 172.21.15.116:0/1002421016 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8dd00a2b80 msgr2=0x7f8dd00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:06.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.870+0000 7f8deb5ee640 1 --2- 172.21.15.116:0/1002421016 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8dd00a2b80 0x7f8dd00a5040 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc09ac40 tx=0x7f8ddc090420 comp rx=0 tx=0).stop 2024-09-12T03:11:06.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.870+0000 7f8deb5ee640 1 -- 172.21.15.116:0/1002421016 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8de4102ef0 msgr2=0x7f8de419be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:06.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.870+0000 7f8deb5ee640 1 --2- 172.21.15.116:0/1002421016 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8de4102ef0 0x7f8de419be10 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc00b970 tx=0x7f8ddc00be30 comp rx=0 tx=0).stop 2024-09-12T03:11:06.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.870+0000 7f8deb5ee640 1 -- 172.21.15.116:0/1002421016 shutdown_connections 2024-09-12T03:11:06.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.870+0000 7f8deb5ee640 1 --2- 172.21.15.116:0/1002421016 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8dd00a2b80 0x7f8dd00a5040 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.870+0000 7f8deb5ee640 1 --2- 172.21.15.116:0/1002421016 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8de4102ef0 0x7f8de419be10 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.870+0000 7f8deb5ee640 1 --2- 172.21.15.116:0/1002421016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8de4102570 0x7f8de4197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.870+0000 7f8deb5ee640 1 --2- 172.21.15.116:0/1002421016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8de4100e20 0x7f8de4197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:06.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.870+0000 7f8deb5ee640 1 -- 172.21.15.116:0/1002421016 >> 172.21.15.116:0/1002421016 conn(0x7f8de40fc7d0 msgr2=0x7f8de40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:06.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.870+0000 7f8deb5ee640 1 -- 172.21.15.116:0/1002421016 shutdown_connections 2024-09-12T03:11:06.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:06.870+0000 7f8deb5ee640 1 -- 172.21.15.116:0/1002421016 wait complete. 2024-09-12T03:11:07.606 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '45ae6e70', total seconds elapsed: 22.7 2024-09-12T03:11:07.607 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.4 2024-09-12T03:11:07.607 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:11:07.853 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:11:08.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.372+0000 7f7555506640 1 -- 172.21.15.116:0/980170793 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7550101de0 msgr2=0x7f75501021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:08.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.372+0000 7f7555506640 1 --2- 172.21.15.116:0/980170793 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7550101de0 0x7f75501021c0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f7540009900 tx=0x7f754002a410 comp rx=0 tx=0).stop 2024-09-12T03:11:08.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.372+0000 7f7555506640 1 -- 172.21.15.116:0/980170793 shutdown_connections 2024-09-12T03:11:08.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.372+0000 7f7555506640 1 --2- 172.21.15.116:0/980170793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7550106f10 0x7f755010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:08.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.372+0000 7f7555506640 1 --2- 172.21.15.116:0/980170793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7550102790 0x7f75501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:08.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.372+0000 7f7555506640 1 --2- 172.21.15.116:0/980170793 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7550101de0 0x7f75501021c0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:08.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.372+0000 7f7555506640 1 -- 172.21.15.116:0/980170793 >> 172.21.15.116:0/980170793 conn(0x7f75500fd8c0 msgr2=0x7f75500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:08.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.372+0000 7f7555506640 1 -- 172.21.15.116:0/980170793 shutdown_connections 2024-09-12T03:11:08.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.372+0000 7f7555506640 1 -- 172.21.15.116:0/980170793 wait complete. 2024-09-12T03:11:08.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.373+0000 7f7555506640 1 Processor -- start 2024-09-12T03:11:08.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.373+0000 7f7555506640 1 -- start start 2024-09-12T03:11:08.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.373+0000 7f7555506640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7550101de0 0x7f75500744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:08.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.373+0000 7f7555506640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7550102790 0x7f7550074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:08.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.373+0000 7f7555506640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7550106f10 0x7f7550074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:08.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.373+0000 7f7555506640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7550111f70 con 0x7f7550102790 2024-09-12T03:11:08.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.373+0000 7f7555506640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7550111df0 con 0x7f7550106f10 2024-09-12T03:11:08.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.373+0000 7f7555506640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f75501120f0 con 0x7f7550101de0 2024-09-12T03:11:08.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.373+0000 7f754e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7550102790 0x7f7550074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:08.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.373+0000 7f754e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7550102790 0x7f7550074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58554/0 (socket says 172.21.15.116:58554) 2024-09-12T03:11:08.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.373+0000 7f754e7fc640 1 -- 172.21.15.116:0/3401850874 learned_addr learned my addr 172.21.15.116:0/3401850874 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:08.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.373+0000 7f754f7fe640 1 --2- 172.21.15.116:0/3401850874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7550106f10 0x7f7550074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:08.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.374+0000 7f754effd640 1 --2- 172.21.15.116:0/3401850874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7550101de0 0x7f75500744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:08.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.374+0000 7f754e7fc640 1 -- 172.21.15.116:0/3401850874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7550101de0 msgr2=0x7f75500744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:08.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.374+0000 7f754e7fc640 1 --2- 172.21.15.116:0/3401850874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7550101de0 0x7f75500744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:08.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.374+0000 7f754e7fc640 1 -- 172.21.15.116:0/3401850874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7550106f10 msgr2=0x7f7550074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:08.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.374+0000 7f754e7fc640 1 --2- 172.21.15.116:0/3401850874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7550106f10 0x7f7550074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:08.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.374+0000 7f754e7fc640 1 -- 172.21.15.116:0/3401850874 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75501a6830 con 0x7f7550102790 2024-09-12T03:11:08.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.374+0000 7f754f7fe640 1 --2- 172.21.15.116:0/3401850874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7550106f10 0x7f7550074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:11:08.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.374+0000 7f754e7fc640 1 --2- 172.21.15.116:0/3401850874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7550102790 0x7f7550074a30 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f753800c920 tx=0x7f753800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:08.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.374+0000 7f7547fff640 1 -- 172.21.15.116:0/3401850874 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7538007c70 con 0x7f7550102790 2024-09-12T03:11:08.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.374+0000 7f7547fff640 1 -- 172.21.15.116:0/3401850874 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7538007e10 con 0x7f7550102790 2024-09-12T03:11:08.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.374+0000 7f7555506640 1 -- 172.21.15.116:0/3401850874 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75501a6b20 con 0x7f7550102790 2024-09-12T03:11:08.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.374+0000 7f7547fff640 1 -- 172.21.15.116:0/3401850874 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7538018890 con 0x7f7550102790 2024-09-12T03:11:08.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.374+0000 7f7555506640 1 -- 172.21.15.116:0/3401850874 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75501a7060 con 0x7f7550102790 2024-09-12T03:11:08.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.375+0000 7f7555506640 1 -- 172.21.15.116:0/3401850874 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75501021c0 con 0x7f7550102790 2024-09-12T03:11:08.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.376+0000 7f7547fff640 1 -- 172.21.15.116:0/3401850874 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f7538018a50 con 0x7f7550102790 2024-09-12T03:11:08.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.377+0000 7f7547fff640 1 --2- 172.21.15.116:0/3401850874 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f753c0a2b80 0x7f753c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:08.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.377+0000 7f754effd640 1 --2- 172.21.15.116:0/3401850874 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f753c0a2b80 0x7f753c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:08.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.377+0000 7f7547fff640 1 -- 172.21.15.116:0/3401850874 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f75380cf440 con 0x7f7550102790 2024-09-12T03:11:08.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.377+0000 7f754effd640 1 --2- 172.21.15.116:0/3401850874 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f753c0a2b80 0x7f753c0a5040 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7540005ab0 tx=0x7f7540005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:08.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.380+0000 7f7547fff640 1 -- 172.21.15.116:0/3401850874 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7538014030 con 0x7f7550102790 2024-09-12T03:11:08.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:08 smithi116 ceph-mon[31410]: pgmap v582: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-12T03:11:08.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.582+0000 7f7555506640 1 -- 172.21.15.116:0/3401850874 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7550075a20 con 0x7f7550102790 2024-09-12T03:11:08.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.583+0000 7f7547fff640 1 -- 172.21.15.116:0/3401850874 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f7550075a20 con 0x7f7550102790 2024-09-12T03:11:08.584 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:11:08.585 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:11:08.586 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:11:08.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.585+0000 7f7555506640 1 -- 172.21.15.116:0/3401850874 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f753c0a2b80 msgr2=0x7f753c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:08.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.585+0000 7f7555506640 1 --2- 172.21.15.116:0/3401850874 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f753c0a2b80 0x7f753c0a5040 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7540005ab0 tx=0x7f7540005a20 comp rx=0 tx=0).stop 2024-09-12T03:11:08.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.585+0000 7f7555506640 1 -- 172.21.15.116:0/3401850874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7550102790 msgr2=0x7f7550074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:08.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.585+0000 7f7555506640 1 --2- 172.21.15.116:0/3401850874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7550102790 0x7f7550074a30 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f753800c920 tx=0x7f753800cde0 comp rx=0 tx=0).stop 2024-09-12T03:11:08.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.585+0000 7f7555506640 1 -- 172.21.15.116:0/3401850874 shutdown_connections 2024-09-12T03:11:08.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.585+0000 7f7555506640 1 --2- 172.21.15.116:0/3401850874 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f753c0a2b80 0x7f753c0a5040 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:08.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.585+0000 7f7555506640 1 --2- 172.21.15.116:0/3401850874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7550106f10 0x7f7550074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:08.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.585+0000 7f7555506640 1 --2- 172.21.15.116:0/3401850874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7550102790 0x7f7550074a30 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:08.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.585+0000 7f7555506640 1 --2- 172.21.15.116:0/3401850874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7550101de0 0x7f75500744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:08.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.585+0000 7f7555506640 1 -- 172.21.15.116:0/3401850874 >> 172.21.15.116:0/3401850874 conn(0x7f75500fd8c0 msgr2=0x7f75500fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:08.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.586+0000 7f7555506640 1 -- 172.21.15.116:0/3401850874 shutdown_connections 2024-09-12T03:11:08.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:08.586+0000 7f7555506640 1 -- 172.21.15.116:0/3401850874 wait complete. 2024-09-12T03:11:08.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:08 smithi183 ceph-mon[35535]: pgmap v582: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-12T03:11:08.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:08 smithi177 ceph-mon[35305]: pgmap v582: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-12T03:11:09.393 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0' 2024-09-12T03:11:09.394 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0 2024-09-12T03:11:09.639 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:11:09.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:09 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3401850874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:11:09.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:09 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3401850874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:11:09.760 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:09 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3401850874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:11:10.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.154+0000 7f6b67f06640 1 -- 172.21.15.116:0/2417375276 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b60102d90 msgr2=0x7f6b60103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:10.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.154+0000 7f6b67f06640 1 --2- 172.21.15.116:0/2417375276 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b60102d90 0x7f6b60103170 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009930 tx=0x7f6b5c02a420 comp rx=0 tx=0).stop 2024-09-12T03:11:10.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.154+0000 7f6b67f06640 1 -- 172.21.15.116:0/2417375276 shutdown_connections 2024-09-12T03:11:10.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.154+0000 7f6b67f06640 1 --2- 172.21.15.116:0/2417375276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b60108290 0x7f6b6010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.154+0000 7f6b67f06640 1 --2- 172.21.15.116:0/2417375276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b60103740 0x7f6b60107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.154+0000 7f6b67f06640 1 --2- 172.21.15.116:0/2417375276 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b60102d90 0x7f6b60103170 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.154+0000 7f6b67f06640 1 -- 172.21.15.116:0/2417375276 >> 172.21.15.116:0/2417375276 conn(0x7f6b600fc7d0 msgr2=0x7f6b600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:10.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.154+0000 7f6b67f06640 1 -- 172.21.15.116:0/2417375276 shutdown_connections 2024-09-12T03:11:10.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.154+0000 7f6b67f06640 1 -- 172.21.15.116:0/2417375276 wait complete. 2024-09-12T03:11:10.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.155+0000 7f6b67f06640 1 Processor -- start 2024-09-12T03:11:10.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.155+0000 7f6b67f06640 1 -- start start 2024-09-12T03:11:10.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.155+0000 7f6b67f06640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b60102d90 0x7f6b601985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.155+0000 7f6b67f06640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b60103740 0x7f6b60198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.155+0000 7f6b67f06640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b60108290 0x7f6b6019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.155+0000 7f6b67f06640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6b60112090 con 0x7f6b60103740 2024-09-12T03:11:10.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.155+0000 7f6b67f06640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6b60111f10 con 0x7f6b60102d90 2024-09-12T03:11:10.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.155+0000 7f6b67f06640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6b60112210 con 0x7f6b60108290 2024-09-12T03:11:10.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.155+0000 7f6b654a1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b60103740 0x7f6b60198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.155+0000 7f6b654a1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b60103740 0x7f6b60198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58574/0 (socket says 172.21.15.116:58574) 2024-09-12T03:11:10.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.155+0000 7f6b654a1640 1 -- 172.21.15.116:0/2145443805 learned_addr learned my addr 172.21.15.116:0/2145443805 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:10.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.155+0000 7f6b664a3640 1 --2- 172.21.15.116:0/2145443805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b60108290 0x7f6b6019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.155+0000 7f6b654a1640 1 -- 172.21.15.116:0/2145443805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b60108290 msgr2=0x7f6b6019cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:10.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.155+0000 7f6b654a1640 1 --2- 172.21.15.116:0/2145443805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b60108290 0x7f6b6019cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.155+0000 7f6b654a1640 1 -- 172.21.15.116:0/2145443805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b60102d90 msgr2=0x7f6b601985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:10.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.155+0000 7f6b65ca2640 1 --2- 172.21.15.116:0/2145443805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b60102d90 0x7f6b601985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.155+0000 7f6b654a1640 1 --2- 172.21.15.116:0/2145443805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b60102d90 0x7f6b601985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.155+0000 7f6b654a1640 1 -- 172.21.15.116:0/2145443805 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b6019d5a0 con 0x7f6b60103740 2024-09-12T03:11:10.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.155+0000 7f6b664a3640 1 --2- 172.21.15.116:0/2145443805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b60108290 0x7f6b6019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:11:10.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.156+0000 7f6b654a1640 1 --2- 172.21.15.116:0/2145443805 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b60103740 0x7f6b60198b30 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000c970 tx=0x7f6b5000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:10.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.156+0000 7f6b56ffd640 1 -- 172.21.15.116:0/2145443805 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b50007bb0 con 0x7f6b60103740 2024-09-12T03:11:10.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.156+0000 7f6b56ffd640 1 -- 172.21.15.116:0/2145443805 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b50007d50 con 0x7f6b60103740 2024-09-12T03:11:10.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.156+0000 7f6b67f06640 1 -- 172.21.15.116:0/2145443805 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b6019d890 con 0x7f6b60103740 2024-09-12T03:11:10.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.156+0000 7f6b56ffd640 1 -- 172.21.15.116:0/2145443805 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5000f450 con 0x7f6b60103740 2024-09-12T03:11:10.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.156+0000 7f6b67f06640 1 -- 172.21.15.116:0/2145443805 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b601a5170 con 0x7f6b60103740 2024-09-12T03:11:10.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.158+0000 7f6b56ffd640 1 -- 172.21.15.116:0/2145443805 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6b500046c0 con 0x7f6b60103740 2024-09-12T03:11:10.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.158+0000 7f6b6a2df740 1 Processor -- start 2024-09-12T03:11:10.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.159+0000 7f6b6a2df740 1 -- start start 2024-09-12T03:11:10.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.159+0000 7f6b6a2df740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627f45f5760 0x5627f45f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.159+0000 7f6b6a2df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627f45f6080 0x5627f4600400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.159+0000 7f6b6a2df740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5627f4600940 0x5627f4602d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.159+0000 7f6b6a2df740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5627f455b2a0 con 0x5627f45f5760 2024-09-12T03:11:10.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.159+0000 7f6b6a2df740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5627f4502b10 con 0x5627f4600940 2024-09-12T03:11:10.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.159+0000 7f6b6a2df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5627f4557260 con 0x5627f45f6080 2024-09-12T03:11:10.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.159+0000 7f6b664a3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627f45f5760 0x5627f45f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.159+0000 7f6b56ffd640 1 --2- 172.21.15.116:0/2145443805 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6b4c0a2c00 0x7f6b4c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.159+0000 7f6b664a3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627f45f5760 0x5627f45f5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58584/0 (socket says 172.21.15.116:58584) 2024-09-12T03:11:10.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.159+0000 7f6b664a3640 1 -- 172.21.15.116:0/3345981114 learned_addr learned my addr 172.21.15.116:0/3345981114 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:10.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.159+0000 7f6b65ca2640 1 --2- 172.21.15.116:0/3345981114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627f45f6080 0x5627f4600400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.159+0000 7f6b664a3640 1 --2- 172.21.15.116:0/2145443805 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6b4c0a2c00 0x7f6b4c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.159+0000 7f6b56ffd640 1 -- 172.21.15.116:0/2145443805 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f6b500cf0a0 con 0x7f6b60103740 2024-09-12T03:11:10.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.159+0000 7f6b654a1640 1 --2- 172.21.15.116:0/3345981114 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5627f4600940 0x5627f4602d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.159+0000 7f6b664a3640 1 -- 172.21.15.116:0/3345981114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627f45f6080 msgr2=0x5627f4600400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:10.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.159+0000 7f6b664a3640 1 --2- 172.21.15.116:0/3345981114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627f45f6080 0x5627f4600400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.159+0000 7f6b664a3640 1 -- 172.21.15.116:0/3345981114 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5627f4600940 msgr2=0x5627f4602d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:10.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.159+0000 7f6b664a3640 1 --2- 172.21.15.116:0/3345981114 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5627f4600940 0x5627f4602d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.159+0000 7f6b664a3640 1 -- 172.21.15.116:0/3345981114 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627f42ccb90 con 0x5627f45f5760 2024-09-12T03:11:10.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.159+0000 7f6b654a1640 1 --2- 172.21.15.116:0/3345981114 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5627f4600940 0x5627f4602d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:10.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.160+0000 7f6b664a3640 1 --2- 172.21.15.116:0/3345981114 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627f45f5760 0x5627f45f5b40 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f6b58014a10 tx=0x7f6b58014ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c3561e9d8a239c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:10.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.160+0000 7f6b4bfff640 1 -- 172.21.15.116:0/3345981114 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b58029070 con 0x5627f45f5760 2024-09-12T03:11:10.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.160+0000 7f6b4bfff640 1 -- 172.21.15.116:0/3345981114 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b58014250 con 0x5627f45f5760 2024-09-12T03:11:10.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.160+0000 7f6b4bfff640 1 -- 172.21.15.116:0/3345981114 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b58024b40 con 0x5627f45f5760 2024-09-12T03:11:10.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.160+0000 7f6b664a3640 1 --2- 172.21.15.116:0/2145443805 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6b4c0a2c00 0x7f6b4c0a50c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6b58026000 tx=0x7f6b58022000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:10.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.160+0000 7f6b6a2df740 1 -- 172.21.15.116:0/3345981114 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627f45f5760 msgr2=0x5627f45f5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:10.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.160+0000 7f6b6a2df740 1 --2- 172.21.15.116:0/3345981114 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627f45f5760 0x5627f45f5b40 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f6b58014a10 tx=0x7f6b58014ed0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.161+0000 7f6b6a2df740 1 -- 172.21.15.116:0/3345981114 shutdown_connections 2024-09-12T03:11:10.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.161+0000 7f6b6a2df740 1 --2- 172.21.15.116:0/3345981114 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5627f4600940 0x5627f4602d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.161+0000 7f6b6a2df740 1 --2- 172.21.15.116:0/3345981114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627f45f6080 0x5627f4600400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.161+0000 7f6b6a2df740 1 --2- 172.21.15.116:0/3345981114 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627f45f5760 0x5627f45f5b40 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.161+0000 7f6b6a2df740 1 -- 172.21.15.116:0/3345981114 >> 172.21.15.116:0/3345981114 conn(0x5627f45f1cf0 msgr2=0x5627f45f20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:10.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.161+0000 7f6b6a2df740 1 -- 172.21.15.116:0/3345981114 shutdown_connections 2024-09-12T03:11:10.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.161+0000 7f6b6a2df740 1 -- 172.21.15.116:0/3345981114 wait complete. 2024-09-12T03:11:10.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.161+0000 7f6b6a2df740 1 -- start start 2024-09-12T03:11:10.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.161+0000 7f6b6a2df740 1 Processor -- start 2024-09-12T03:11:10.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.161+0000 7f6b6a2df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627f45f5760 0x5627f45fd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.162+0000 7f6b6a2df740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5627f45f6080 0x5627f45fdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.162+0000 7f6b6a2df740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627f4600940 0x5627f45fe290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.162+0000 7f6b6a2df740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5627f4603510 con 0x5627f4600940 2024-09-12T03:11:10.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.162+0000 7f6b6a2df740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5627f44e36e0 con 0x5627f45f6080 2024-09-12T03:11:10.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.162+0000 7f6b6a2df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5627f4603690 con 0x5627f45f5760 2024-09-12T03:11:10.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.162+0000 7f6b6a2df740 10 client.0 _ll_register_callbacks cb 0x5627f4507a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:10.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.162+0000 7f6b664a3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627f4600940 0x5627f45fe290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.162+0000 7f6b664a3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627f4600940 0x5627f45fe290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58598/0 (socket says 172.21.15.116:58598) 2024-09-12T03:11:10.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.162+0000 7f6b664a3640 1 -- 172.21.15.116:0/1457352916 learned_addr learned my addr 172.21.15.116:0/1457352916 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:10.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.162+0000 7f6b65ca2640 1 --2- 172.21.15.116:0/1457352916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627f45f5760 0x5627f45fd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.162+0000 7f6b654a1640 1 --2- 172.21.15.116:0/1457352916 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5627f45f6080 0x5627f45fdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.162+0000 7f6b664a3640 1 -- 172.21.15.116:0/1457352916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627f45f5760 msgr2=0x5627f45fd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:10.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.162+0000 7f6b664a3640 1 --2- 172.21.15.116:0/1457352916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627f45f5760 0x5627f45fd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.162+0000 7f6b664a3640 1 -- 172.21.15.116:0/1457352916 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5627f45f6080 msgr2=0x5627f45fdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:10.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.162+0000 7f6b664a3640 1 --2- 172.21.15.116:0/1457352916 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5627f45f6080 0x5627f45fdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.162+0000 7f6b664a3640 1 -- 172.21.15.116:0/1457352916 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627f460ca00 con 0x5627f4600940 2024-09-12T03:11:10.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.162+0000 7f6b654a1640 1 --2- 172.21.15.116:0/1457352916 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5627f45f6080 0x5627f45fdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:10.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.162+0000 7f6b65ca2640 1 --2- 172.21.15.116:0/1457352916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627f45f5760 0x5627f45fd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:11:10.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.162+0000 7f6b664a3640 1 --2- 172.21.15.116:0/1457352916 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627f4600940 0x5627f45fe290 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f6b580244f0 tx=0x7f6b58024720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a428df79de2e7b32 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:10.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.162+0000 7f6b497fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:10.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.162+0000 7f6b497fa640 1 -- 172.21.15.116:0/1457352916 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b58029040 con 0x5627f4600940 2024-09-12T03:11:10.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.162+0000 7f6b6a2df740 1 -- 172.21.15.116:0/1457352916 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b58014250 con 0x5627f4600940 2024-09-12T03:11:10.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.162+0000 7f6b497fa640 1 -- 172.21.15.116:0/1457352916 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f6b58012030 con 0x5627f4600940 2024-09-12T03:11:10.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.163+0000 7f6b497fa640 1 -- 172.21.15.116:0/1457352916 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6b58024940 con 0x5627f4600940 2024-09-12T03:11:10.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.163+0000 7f6b6a2df740 10 client.17466 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:10.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.163+0000 7f6b6a2df740 1 -- 172.21.15.116:0/1457352916 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5627f460f2e0 con 0x5627f4600940 2024-09-12T03:11:10.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.163+0000 7f6b497fa640 1 -- 172.21.15.116:0/1457352916 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b58014250 con 0x5627f4600940 2024-09-12T03:11:10.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.163+0000 7f6b497fa640 1 -- 172.21.15.116:0/1457352916 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6b58023030 con 0x5627f4600940 2024-09-12T03:11:10.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.163+0000 7f6b6a2df740 10 client.17466 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:10.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.163+0000 7f6b6a2df740 10 client.17466 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:11:10.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.163+0000 7f6b6a2df740 20 client.17466 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:10.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.163+0000 7f6b6a2df740 1 --2- 172.21.15.116:0/1457352916 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5627f4610740 0x5627f4612b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.163+0000 7f6b6a2df740 4 client.17466 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:11:10.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.163+0000 7f6b6a2df740 1 -- 172.21.15.116:0/1457352916 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627f45689d0 con 0x5627f4610740 2024-09-12T03:11:10.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.163+0000 7f6b65ca2640 1 --2- 172.21.15.116:0/1457352916 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5627f4610740 0x5627f4612b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.164+0000 7f6b497fa640 10 client.17466 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:10.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.164+0000 7f6b65ca2640 1 --2- 172.21.15.116:0/1457352916 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5627f4610740 0x5627f4612b30 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed7e465fc486fbd9 server_cookie=19af9a48c8b6501a in_seq=0 out_seq=0 2024-09-12T03:11:10.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.165+0000 7f6b497fa640 1 -- 172.21.15.116:0/1457352916 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5627f45689d0 con 0x5627f4610740 2024-09-12T03:11:10.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.165+0000 7f6b497fa640 10 client.17466 handle_command_reply: tid=0 2024-09-12T03:11:10.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.165+0000 7f6b6a2df740 1 client.17466 shutdown 2024-09-12T03:11:10.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.165+0000 7f6b6a2df740 1 -- 172.21.15.116:0/1457352916 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627f4600940 msgr2=0x5627f45fe290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:10.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.165+0000 7f6b6a2df740 1 --2- 172.21.15.116:0/1457352916 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627f4600940 0x5627f45fe290 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f6b580244f0 tx=0x7f6b58024720 comp rx=0 tx=0).stop 2024-09-12T03:11:10.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.165+0000 7f6b6a2df740 1 -- 172.21.15.116:0/1457352916 shutdown_connections 2024-09-12T03:11:10.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.165+0000 7f6b6a2df740 1 --2- 172.21.15.116:0/1457352916 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5627f4610740 0x5627f4612b30 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.165+0000 7f6b6a2df740 1 --2- 172.21.15.116:0/1457352916 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627f4600940 0x5627f45fe290 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.165+0000 7f6b6a2df740 1 --2- 172.21.15.116:0/1457352916 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5627f45f6080 0x5627f45fdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.165+0000 7f6b6a2df740 1 --2- 172.21.15.116:0/1457352916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627f45f5760 0x5627f45fd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.165+0000 7f6b497fa640 0 client.17466 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:10.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.165+0000 7f6b6a2df740 1 -- 172.21.15.116:0/1457352916 >> 172.21.15.116:0/1457352916 conn(0x5627f45f1cf0 msgr2=0x5627f45f0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:10.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.165+0000 7f6b6a2df740 1 -- 172.21.15.116:0/1457352916 shutdown_connections 2024-09-12T03:11:10.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.165+0000 7f6b6a2df740 1 -- 172.21.15.116:0/1457352916 wait complete. 2024-09-12T03:11:10.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.165+0000 7f6b6a2df740 20 client.17466 trim_cache size 0 max 16384 2024-09-12T03:11:10.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.182+0000 7f6b6a2df740 1 Processor -- start 2024-09-12T03:11:10.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.182+0000 7f6b6a2df740 1 -- start start 2024-09-12T03:11:10.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.182+0000 7f6b6a2df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627f45f5760 0x5627f45fc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.182+0000 7f6b6a2df740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627f45f6080 0x5627f45f81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.182+0000 7f6b6a2df740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5627f4600940 0x5627f4618320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.182+0000 7f6b6a2df740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6b601adca0 con 0x5627f45f6080 2024-09-12T03:11:10.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.182+0000 7f6b6a2df740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6b601ab570 con 0x5627f4600940 2024-09-12T03:11:10.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.182+0000 7f6b6a2df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6b601a93c0 con 0x5627f45f5760 2024-09-12T03:11:10.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.182+0000 7f6b654a1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627f45f6080 0x5627f45f81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.182+0000 7f6b654a1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627f45f6080 0x5627f45f81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58606/0 (socket says 172.21.15.116:58606) 2024-09-12T03:11:10.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.182+0000 7f6b654a1640 1 -- 172.21.15.116:0/2437051530 learned_addr learned my addr 172.21.15.116:0/2437051530 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:10.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.182+0000 7f6b65ca2640 1 --2- 172.21.15.116:0/2437051530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627f45f5760 0x5627f45fc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.183+0000 7f6b664a3640 1 --2- 172.21.15.116:0/2437051530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5627f4600940 0x5627f4618320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.183+0000 7f6b654a1640 1 -- 172.21.15.116:0/2437051530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627f45f5760 msgr2=0x5627f45fc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:10.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.183+0000 7f6b654a1640 1 --2- 172.21.15.116:0/2437051530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627f45f5760 0x5627f45fc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.183+0000 7f6b654a1640 1 -- 172.21.15.116:0/2437051530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5627f4600940 msgr2=0x5627f4618320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:10.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.183+0000 7f6b654a1640 1 --2- 172.21.15.116:0/2437051530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5627f4600940 0x5627f4618320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.183+0000 7f6b654a1640 1 -- 172.21.15.116:0/2437051530 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627f46130f0 con 0x5627f45f6080 2024-09-12T03:11:10.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.183+0000 7f6b654a1640 1 --2- 172.21.15.116:0/2437051530 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627f45f6080 0x5627f45f81c0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f6b500ccf40 tx=0x7f6b5009b350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5bec16a665879c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:10.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.183+0000 7f6b4b7fe640 1 -- 172.21.15.116:0/2437051530 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b500b4070 con 0x5627f45f6080 2024-09-12T03:11:10.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.183+0000 7f6b4b7fe640 1 -- 172.21.15.116:0/2437051530 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b500afe50 con 0x5627f45f6080 2024-09-12T03:11:10.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.183+0000 7f6b4b7fe640 1 -- 172.21.15.116:0/2437051530 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5001d040 con 0x5627f45f6080 2024-09-12T03:11:10.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.184+0000 7f6b6a2df740 1 -- 172.21.15.116:0/2437051530 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627f45f6080 msgr2=0x5627f45f81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:10.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.184+0000 7f6b6a2df740 1 --2- 172.21.15.116:0/2437051530 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627f45f6080 0x5627f45f81c0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f6b500ccf40 tx=0x7f6b5009b350 comp rx=0 tx=0).stop 2024-09-12T03:11:10.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.184+0000 7f6b6a2df740 1 -- 172.21.15.116:0/2437051530 shutdown_connections 2024-09-12T03:11:10.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.184+0000 7f6b6a2df740 1 --2- 172.21.15.116:0/2437051530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5627f4600940 0x5627f4618320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.184+0000 7f6b6a2df740 1 --2- 172.21.15.116:0/2437051530 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627f45f6080 0x5627f45f81c0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.184+0000 7f6b6a2df740 1 --2- 172.21.15.116:0/2437051530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627f45f5760 0x5627f45fc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.184+0000 7f6b6a2df740 1 -- 172.21.15.116:0/2437051530 >> 172.21.15.116:0/2437051530 conn(0x5627f45f1cf0 msgr2=0x5627f45ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:10.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.184+0000 7f6b6a2df740 1 -- 172.21.15.116:0/2437051530 shutdown_connections 2024-09-12T03:11:10.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.184+0000 7f6b6a2df740 1 -- 172.21.15.116:0/2437051530 wait complete. 2024-09-12T03:11:10.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.184+0000 7f6b6a2df740 1 -- start start 2024-09-12T03:11:10.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.184+0000 7f6b6a2df740 1 Processor -- start 2024-09-12T03:11:10.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.185+0000 7f6b6a2df740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627f45f5760 0x5627f46088e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.185+0000 7f6b6a2df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627f45f6080 0x5627f4608e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.185+0000 7f6b6a2df740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5627f4600940 0x5627f46058b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.185+0000 7f6b6a2df740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6b601a9a30 con 0x5627f45f5760 2024-09-12T03:11:10.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.185+0000 7f6b6a2df740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6b601aaea0 con 0x5627f4600940 2024-09-12T03:11:10.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.185+0000 7f6b6a2df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6b601aa100 con 0x5627f45f6080 2024-09-12T03:11:10.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.185+0000 7f6b6a2df740 10 client.0 _ll_register_callbacks cb 0x7f6b580147e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:10.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.185+0000 7f6b65ca2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627f45f5760 0x5627f46088e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.185+0000 7f6b65ca2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627f45f5760 0x5627f46088e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:58616/0 (socket says 172.21.15.116:58616) 2024-09-12T03:11:10.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.185+0000 7f6b65ca2640 1 -- 172.21.15.116:0/2667843481 learned_addr learned my addr 172.21.15.116:0/2667843481 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:10.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.185+0000 7f6b664a3640 1 --2- 172.21.15.116:0/2667843481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5627f4600940 0x5627f46058b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.185+0000 7f6b654a1640 1 --2- 172.21.15.116:0/2667843481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627f45f6080 0x5627f4608e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.185+0000 7f6b65ca2640 1 -- 172.21.15.116:0/2667843481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627f45f6080 msgr2=0x5627f4608e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:10.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.185+0000 7f6b65ca2640 1 --2- 172.21.15.116:0/2667843481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627f45f6080 0x5627f4608e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.185+0000 7f6b65ca2640 1 -- 172.21.15.116:0/2667843481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5627f4600940 msgr2=0x5627f46058b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:10.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.185+0000 7f6b65ca2640 1 --2- 172.21.15.116:0/2667843481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5627f4600940 0x5627f46058b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.185+0000 7f6b65ca2640 1 -- 172.21.15.116:0/2667843481 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627f45ed260 con 0x5627f45f5760 2024-09-12T03:11:10.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.185+0000 7f6b654a1640 1 --2- 172.21.15.116:0/2667843481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627f45f6080 0x5627f4608e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:10.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.185+0000 7f6b65ca2640 1 --2- 172.21.15.116:0/2667843481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627f45f5760 0x5627f46088e0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c032fd0 tx=0x7f6b5c0041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8478da5c2695de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:10.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.185+0000 7f6b497fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:10.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.185+0000 7f6b497fa640 1 -- 172.21.15.116:0/2667843481 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c035040 con 0x5627f45f5760 2024-09-12T03:11:10.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.185+0000 7f6b497fa640 1 -- 172.21.15.116:0/2667843481 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f6b5c005240 con 0x5627f45f5760 2024-09-12T03:11:10.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.185+0000 7f6b6a2df740 1 -- 172.21.15.116:0/2667843481 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b500afe50 con 0x5627f45f5760 2024-09-12T03:11:10.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.186+0000 7f6b497fa640 1 -- 172.21.15.116:0/2667843481 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6b5c052050 con 0x5627f45f5760 2024-09-12T03:11:10.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.186+0000 7f6b6a2df740 10 client.17472 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:10.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.186+0000 7f6b6a2df740 1 -- 172.21.15.116:0/2667843481 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5627f467b790 con 0x5627f45f5760 2024-09-12T03:11:10.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.186+0000 7f6b497fa640 1 -- 172.21.15.116:0/2667843481 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b500afe50 con 0x5627f45f5760 2024-09-12T03:11:10.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.186+0000 7f6b497fa640 1 -- 172.21.15.116:0/2667843481 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6b5c0035f0 con 0x5627f45f5760 2024-09-12T03:11:10.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.186+0000 7f6b6a2df740 10 client.17472 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:10.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.186+0000 7f6b6a2df740 10 client.17472 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:11:10.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.186+0000 7f6b6a2df740 20 client.17472 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:10.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.186+0000 7f6b6a2df740 1 --2- 172.21.15.116:0/2667843481 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5627f467bf50 0x5627f467e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.186+0000 7f6b6a2df740 4 client.17472 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}] 2024-09-12T03:11:10.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.186+0000 7f6b6a2df740 1 -- 172.21.15.116:0/2667843481 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}) -- 0x5627f4562d20 con 0x5627f467bf50 2024-09-12T03:11:10.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.186+0000 7f6b654a1640 1 --2- 172.21.15.116:0/2667843481 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5627f467bf50 0x5627f467e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.187+0000 7f6b654a1640 1 --2- 172.21.15.116:0/2667843481 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5627f467bf50 0x5627f467e370 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30eb7ec2a5f0e133 server_cookie=2f7639c1b7bb6a6a in_seq=0 out_seq=0 2024-09-12T03:11:10.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.187+0000 7f6b497fa640 10 client.17472 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:10.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.196+0000 7f6b497fa640 1 -- 172.21.15.116:0/2667843481 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5627f4562d20 con 0x5627f467bf50 2024-09-12T03:11:10.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.196+0000 7f6b497fa640 10 client.17472 handle_command_reply: tid=0 2024-09-12T03:11:10.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.196+0000 7f6b6a2df740 1 client.17472 shutdown 2024-09-12T03:11:10.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.196+0000 7f6b6a2df740 1 -- 172.21.15.116:0/2667843481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627f45f5760 msgr2=0x5627f46088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:10.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.196+0000 7f6b6a2df740 1 --2- 172.21.15.116:0/2667843481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627f45f5760 0x5627f46088e0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c032fd0 tx=0x7f6b5c0041a0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.196+0000 7f6b6a2df740 1 -- 172.21.15.116:0/2667843481 shutdown_connections 2024-09-12T03:11:10.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.196+0000 7f6b6a2df740 1 --2- 172.21.15.116:0/2667843481 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5627f467bf50 0x5627f467e370 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.196+0000 7f6b6a2df740 1 --2- 172.21.15.116:0/2667843481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5627f4600940 0x5627f46058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.196+0000 7f6b497fa640 0 client.17472 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:10.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.196+0000 7f6b6a2df740 1 --2- 172.21.15.116:0/2667843481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627f45f6080 0x5627f4608e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.196+0000 7f6b6a2df740 1 --2- 172.21.15.116:0/2667843481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627f45f5760 0x5627f46088e0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.196+0000 7f6b6a2df740 1 -- 172.21.15.116:0/2667843481 >> 172.21.15.116:0/2667843481 conn(0x5627f45f1cf0 msgr2=0x5627f45fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:10.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.196+0000 7f6b6a2df740 1 -- 172.21.15.116:0/2667843481 shutdown_connections 2024-09-12T03:11:10.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.196+0000 7f6b6a2df740 1 -- 172.21.15.116:0/2667843481 wait complete. 2024-09-12T03:11:10.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.196+0000 7f6b6a2df740 20 client.17472 trim_cache size 0 max 16384 2024-09-12T03:11:10.198 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:11:10.198 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 25, 2024-09-12T03:11:10.198 INFO:teuthology.orchestra.run.smithi116.stdout: "leader": 24494, 2024-09-12T03:11:10.198 INFO:teuthology.orchestra.run.smithi116.stdout: "set_version": 34, 2024-09-12T03:11:10.198 INFO:teuthology.orchestra.run.smithi116.stdout: "sets": { 2024-09-12T03:11:10.198 INFO:teuthology.orchestra.run.smithi116.stdout: "d9214b2a": { 2024-09-12T03:11:10.198 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 34, 2024-09-12T03:11:10.199 INFO:teuthology.orchestra.run.smithi116.stdout: "age_ref": 0.0, 2024-09-12T03:11:10.199 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:11:10.199 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "QUIESCED", 2024-09-12T03:11:10.199 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:11:10.199 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:11:10.199 INFO:teuthology.orchestra.run.smithi116.stdout: "timeout": 90.0, 2024-09-12T03:11:10.199 INFO:teuthology.orchestra.run.smithi116.stdout: "expiration": 139.4, 2024-09-12T03:11:10.199 INFO:teuthology.orchestra.run.smithi116.stdout: "members": { 2024-09-12T03:11:10.200 INFO:teuthology.orchestra.run.smithi116.stdout: "file:/": { 2024-09-12T03:11:10.200 INFO:teuthology.orchestra.run.smithi116.stdout: "excluded": false, 2024-09-12T03:11:10.200 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:11:10.200 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "QUIESCED", 2024-09-12T03:11:10.201 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:11:10.201 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:11:10.202 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:11:10.202 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:11:10.202 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:11:10.202 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:11:10.202 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:11:10.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.197+0000 7f6b67f06640 1 -- 172.21.15.116:0/2145443805 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6b4c0a2c00 msgr2=0x7f6b4c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:10.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.197+0000 7f6b67f06640 1 --2- 172.21.15.116:0/2145443805 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6b4c0a2c00 0x7f6b4c0a50c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6b58026000 tx=0x7f6b58022000 comp rx=0 tx=0).stop 2024-09-12T03:11:10.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.197+0000 7f6b67f06640 1 -- 172.21.15.116:0/2145443805 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b60103740 msgr2=0x7f6b60198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:10.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.197+0000 7f6b67f06640 1 --2- 172.21.15.116:0/2145443805 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b60103740 0x7f6b60198b30 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000c970 tx=0x7f6b5000ce30 comp rx=0 tx=0).stop 2024-09-12T03:11:10.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.197+0000 7f6b67f06640 1 -- 172.21.15.116:0/2145443805 shutdown_connections 2024-09-12T03:11:10.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.197+0000 7f6b67f06640 1 --2- 172.21.15.116:0/2145443805 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6b4c0a2c00 0x7f6b4c0a50c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.197+0000 7f6b67f06640 1 --2- 172.21.15.116:0/2145443805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b60108290 0x7f6b6019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.197+0000 7f6b67f06640 1 --2- 172.21.15.116:0/2145443805 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b60103740 0x7f6b60198b30 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.197+0000 7f6b67f06640 1 --2- 172.21.15.116:0/2145443805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b60102d90 0x7f6b601985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.197+0000 7f6b67f06640 1 -- 172.21.15.116:0/2145443805 >> 172.21.15.116:0/2145443805 conn(0x7f6b600fc7d0 msgr2=0x7f6b60106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:10.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.197+0000 7f6b67f06640 1 -- 172.21.15.116:0/2145443805 shutdown_connections 2024-09-12T03:11:10.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:10.197+0000 7f6b67f06640 1 -- 172.21.15.116:0/2145443805 wait complete. 2024-09-12T03:11:10.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:10 smithi183 ceph-mon[35535]: pgmap v583: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:11:10.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:10 smithi177 ceph-mon[35305]: pgmap v583: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:11:10.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:10 smithi116 ceph-mon[31410]: pgmap v583: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:11:10.905 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd9214b2a', quiesce took 3.3 seconds. Will release after: 16.099999999999998 2024-09-12T03:11:11.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:11 smithi183 ceph-mon[35535]: pgmap v584: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.0 KiB/s wr, 9 op/s 2024-09-12T03:11:11.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:11 smithi177 ceph-mon[35305]: pgmap v584: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.0 KiB/s wr, 9 op/s 2024-09-12T03:11:11.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:11 smithi116 ceph-mon[31410]: pgmap v584: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.0 KiB/s wr, 9 op/s 2024-09-12T03:11:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:14 smithi183 ceph-mon[35535]: pgmap v585: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.0 KiB/s wr, 7 op/s 2024-09-12T03:11:14.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:14 smithi177 ceph-mon[35305]: pgmap v585: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.0 KiB/s wr, 7 op/s 2024-09-12T03:11:14.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:14 smithi116 ceph-mon[31410]: pgmap v585: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.0 KiB/s wr, 7 op/s 2024-09-12T03:11:16.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:16 smithi183 ceph-mon[35535]: pgmap v586: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.1 KiB/s wr, 7 op/s 2024-09-12T03:11:16.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:16 smithi177 ceph-mon[35305]: pgmap v586: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.1 KiB/s wr, 7 op/s 2024-09-12T03:11:16.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:16 smithi116 ceph-mon[31410]: pgmap v586: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.1 KiB/s wr, 7 op/s 2024-09-12T03:11:17.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:17 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:17.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:17 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:17.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:17 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:11:17.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:17 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:11:17.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:17 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:17.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:17 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:17.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:17 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:11:17.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:17 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:11:17.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:17 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:17.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:17 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:17.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:17 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:11:17.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:17 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:11:18.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:18 smithi183 ceph-mon[35535]: pgmap v587: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2024-09-12T03:11:18.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:18 smithi177 ceph-mon[35305]: pgmap v587: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2024-09-12T03:11:18.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:18 smithi116 ceph-mon[31410]: pgmap v587: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2024-09-12T03:11:20.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:20 smithi183 ceph-mon[35535]: pgmap v588: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.6 KiB/s wr, 7 op/s 2024-09-12T03:11:20.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:20 smithi177 ceph-mon[35305]: pgmap v588: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.6 KiB/s wr, 7 op/s 2024-09-12T03:11:20.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:20 smithi116 ceph-mon[31410]: pgmap v588: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.6 KiB/s wr, 7 op/s 2024-09-12T03:11:21.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:21 smithi183 ceph-mon[35535]: pgmap v589: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.9 KiB/s wr, 8 op/s 2024-09-12T03:11:21.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:21 smithi177 ceph-mon[35305]: pgmap v589: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.9 KiB/s wr, 8 op/s 2024-09-12T03:11:21.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:21 smithi116 ceph-mon[31410]: pgmap v589: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.9 KiB/s wr, 8 op/s 2024-09-12T03:11:23.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:23 smithi183 ceph-mon[35535]: pgmap v590: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-12T03:11:23.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:23 smithi177 ceph-mon[35305]: pgmap v590: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-12T03:11:24.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:23 smithi116 ceph-mon[31410]: pgmap v590: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-12T03:11:26.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:26 smithi183 ceph-mon[35535]: pgmap v591: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-12T03:11:26.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:26 smithi177 ceph-mon[35305]: pgmap v591: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-12T03:11:26.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:26 smithi116 ceph-mon[31410]: pgmap v591: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-12T03:11:27.006 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:11:27.265 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:11:27.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.744+0000 7fd9ad0e8640 1 -- 172.21.15.116:0/2723668732 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a81027a0 msgr2=0x7fd9a81067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:27.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.744+0000 7fd9ad0e8640 1 --2- 172.21.15.116:0/2723668732 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a81027a0 0x7fd9a81067f0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009900 tx=0x7fd99c02a410 comp rx=0 tx=0).stop 2024-09-12T03:11:27.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.744+0000 7fd9ad0e8640 1 -- 172.21.15.116:0/2723668732 shutdown_connections 2024-09-12T03:11:27.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.744+0000 7fd9ad0e8640 1 --2- 172.21.15.116:0/2723668732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9a8106f20 0x7fd9a810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:27.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.744+0000 7fd9ad0e8640 1 --2- 172.21.15.116:0/2723668732 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a81027a0 0x7fd9a81067f0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:27.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.744+0000 7fd9ad0e8640 1 --2- 172.21.15.116:0/2723668732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9a8101df0 0x7fd9a81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:27.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.744+0000 7fd9ad0e8640 1 -- 172.21.15.116:0/2723668732 >> 172.21.15.116:0/2723668732 conn(0x7fd9a80fd8f0 msgr2=0x7fd9a80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:27.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.744+0000 7fd9ad0e8640 1 -- 172.21.15.116:0/2723668732 shutdown_connections 2024-09-12T03:11:27.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.744+0000 7fd9ad0e8640 1 -- 172.21.15.116:0/2723668732 wait complete. 2024-09-12T03:11:27.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.745+0000 7fd9ad0e8640 1 Processor -- start 2024-09-12T03:11:27.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.745+0000 7fd9ad0e8640 1 -- start start 2024-09-12T03:11:27.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.745+0000 7fd9ad0e8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9a8101df0 0x7fd9a8198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:27.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.745+0000 7fd9ad0e8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a81027a0 0x7fd9a8198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:27.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.745+0000 7fd9ad0e8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9a8106f20 0x7fd9a819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:27.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.745+0000 7fd9ad0e8640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a8111f80 con 0x7fd9a81027a0 2024-09-12T03:11:27.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.745+0000 7fd9ad0e8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a8111e00 con 0x7fd9a8101df0 2024-09-12T03:11:27.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.745+0000 7fd9ad0e8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a8112100 con 0x7fd9a8106f20 2024-09-12T03:11:27.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.745+0000 7fd9a659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a81027a0 0x7fd9a8198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:27.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.745+0000 7fd9a659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a81027a0 0x7fd9a8198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42540/0 (socket says 172.21.15.116:42540) 2024-09-12T03:11:27.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.745+0000 7fd9a659c640 1 -- 172.21.15.116:0/1086971092 learned_addr learned my addr 172.21.15.116:0/1086971092 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:27.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.745+0000 7fd9a6d9d640 1 --2- 172.21.15.116:0/1086971092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9a8101df0 0x7fd9a8198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:27.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.745+0000 7fd9a759e640 1 --2- 172.21.15.116:0/1086971092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9a8106f20 0x7fd9a819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:27.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.746+0000 7fd9a659c640 1 -- 172.21.15.116:0/1086971092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9a8106f20 msgr2=0x7fd9a819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:27.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.746+0000 7fd9a659c640 1 --2- 172.21.15.116:0/1086971092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9a8106f20 0x7fd9a819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:27.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.746+0000 7fd9a659c640 1 -- 172.21.15.116:0/1086971092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9a8101df0 msgr2=0x7fd9a8198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:27.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.746+0000 7fd9a659c640 1 --2- 172.21.15.116:0/1086971092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9a8101df0 0x7fd9a8198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:27.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.746+0000 7fd9a659c640 1 -- 172.21.15.116:0/1086971092 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a819d3f0 con 0x7fd9a81027a0 2024-09-12T03:11:27.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.746+0000 7fd9a659c640 1 --2- 172.21.15.116:0/1086971092 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a81027a0 0x7fd9a8198980 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd99c006010 tx=0x7fd99c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:27.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.746+0000 7fd98ffff640 1 -- 172.21.15.116:0/1086971092 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c03f030 con 0x7fd9a81027a0 2024-09-12T03:11:27.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.746+0000 7fd98ffff640 1 -- 172.21.15.116:0/1086971092 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd99c003ce0 con 0x7fd9a81027a0 2024-09-12T03:11:27.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.746+0000 7fd9ad0e8640 1 -- 172.21.15.116:0/1086971092 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a819d680 con 0x7fd9a81027a0 2024-09-12T03:11:27.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.746+0000 7fd98ffff640 1 -- 172.21.15.116:0/1086971092 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c035070 con 0x7fd9a81027a0 2024-09-12T03:11:27.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.746+0000 7fd9ad0e8640 1 -- 172.21.15.116:0/1086971092 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a81a4fc0 con 0x7fd9a81027a0 2024-09-12T03:11:27.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.747+0000 7fd9ad0e8640 1 -- 172.21.15.116:0/1086971092 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9a81032d0 con 0x7fd9a81027a0 2024-09-12T03:11:27.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.749+0000 7fd98ffff640 1 -- 172.21.15.116:0/1086971092 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd99c042030 con 0x7fd9a81027a0 2024-09-12T03:11:27.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.749+0000 7fd98ffff640 1 --2- 172.21.15.116:0/1086971092 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd9900a2b80 0x7fd9900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:27.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.749+0000 7fd9a6d9d640 1 --2- 172.21.15.116:0/1086971092 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd9900a2b80 0x7fd9900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:27.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.749+0000 7fd98ffff640 1 -- 172.21.15.116:0/1086971092 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd99c0ef7c0 con 0x7fd9a81027a0 2024-09-12T03:11:27.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.750+0000 7fd9a6d9d640 1 --2- 172.21.15.116:0/1086971092 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd9900a2b80 0x7fd9900a5040 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd99800bbe0 tx=0x7fd99800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:27.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.753+0000 7fd98ffff640 1 -- 172.21.15.116:0/1086971092 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd99c0bc1c0 con 0x7fd9a81027a0 2024-09-12T03:11:27.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.951+0000 7fd9ad0e8640 1 -- 172.21.15.116:0/1086971092 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9a81021d0 con 0x7fd9a81027a0 2024-09-12T03:11:27.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.952+0000 7fd98ffff640 1 -- 172.21.15.116:0/1086971092 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7fd9a81021d0 con 0x7fd9a81027a0 2024-09-12T03:11:27.952 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:11:27.952 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:11:27.953 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:11:27.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.954+0000 7fd9ad0e8640 1 -- 172.21.15.116:0/1086971092 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd9900a2b80 msgr2=0x7fd9900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:27.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.954+0000 7fd9ad0e8640 1 --2- 172.21.15.116:0/1086971092 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd9900a2b80 0x7fd9900a5040 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd99800bbe0 tx=0x7fd99800b390 comp rx=0 tx=0).stop 2024-09-12T03:11:27.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.954+0000 7fd9ad0e8640 1 -- 172.21.15.116:0/1086971092 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a81027a0 msgr2=0x7fd9a8198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:27.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.954+0000 7fd9ad0e8640 1 --2- 172.21.15.116:0/1086971092 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a81027a0 0x7fd9a8198980 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd99c006010 tx=0x7fd99c0042f0 comp rx=0 tx=0).stop 2024-09-12T03:11:27.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.954+0000 7fd9ad0e8640 1 -- 172.21.15.116:0/1086971092 shutdown_connections 2024-09-12T03:11:27.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.954+0000 7fd9ad0e8640 1 --2- 172.21.15.116:0/1086971092 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd9900a2b80 0x7fd9900a5040 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:27.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.954+0000 7fd9ad0e8640 1 --2- 172.21.15.116:0/1086971092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9a8106f20 0x7fd9a819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:27.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.954+0000 7fd9ad0e8640 1 --2- 172.21.15.116:0/1086971092 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a81027a0 0x7fd9a8198980 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:27.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.954+0000 7fd9ad0e8640 1 --2- 172.21.15.116:0/1086971092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9a8101df0 0x7fd9a8198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:27.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.954+0000 7fd9ad0e8640 1 -- 172.21.15.116:0/1086971092 >> 172.21.15.116:0/1086971092 conn(0x7fd9a80fd8f0 msgr2=0x7fd9a80fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:27.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.954+0000 7fd9ad0e8640 1 -- 172.21.15.116:0/1086971092 shutdown_connections 2024-09-12T03:11:27.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:27.954+0000 7fd9ad0e8640 1 -- 172.21.15.116:0/1086971092 wait complete. 2024-09-12T03:11:28.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:28 smithi183 ceph-mon[35535]: pgmap v592: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-12T03:11:28.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:28 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1086971092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:11:28.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:28 smithi177 ceph-mon[35305]: pgmap v592: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-12T03:11:28.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:28 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1086971092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:11:28.753 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:28 smithi116 ceph-mon[31410]: pgmap v592: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-12T03:11:28.753 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:28 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1086971092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:11:28.785 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id d9214b2a --release --await' 2024-09-12T03:11:28.786 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id d9214b2a --release --await 2024-09-12T03:11:29.033 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:11:29.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.551+0000 7f67681a2640 1 -- 172.21.15.116:0/293484563 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6760102790 msgr2=0x7f67601067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:29.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.551+0000 7f67681a2640 1 --2- 172.21.15.116:0/293484563 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6760102790 0x7f67601067e0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f675c009900 tx=0x7f675c02a410 comp rx=0 tx=0).stop 2024-09-12T03:11:29.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.551+0000 7f67681a2640 1 -- 172.21.15.116:0/293484563 shutdown_connections 2024-09-12T03:11:29.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.551+0000 7f67681a2640 1 --2- 172.21.15.116:0/293484563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6760106f10 0x7f676010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.551+0000 7f67681a2640 1 --2- 172.21.15.116:0/293484563 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6760102790 0x7f67601067e0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.551+0000 7f67681a2640 1 --2- 172.21.15.116:0/293484563 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6760101de0 0x7f67601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.551+0000 7f67681a2640 1 -- 172.21.15.116:0/293484563 >> 172.21.15.116:0/293484563 conn(0x7f67600fd8c0 msgr2=0x7f67600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:29.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.551+0000 7f67681a2640 1 -- 172.21.15.116:0/293484563 shutdown_connections 2024-09-12T03:11:29.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.551+0000 7f67681a2640 1 -- 172.21.15.116:0/293484563 wait complete. 2024-09-12T03:11:29.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.552+0000 7f67681a2640 1 Processor -- start 2024-09-12T03:11:29.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.552+0000 7f67681a2640 1 -- start start 2024-09-12T03:11:29.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.552+0000 7f67681a2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6760101de0 0x7f6760198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:29.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.552+0000 7f67681a2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6760102790 0x7f6760198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:29.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.552+0000 7f67681a2640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6760106f10 0x7f676019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:29.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.552+0000 7f67681a2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6760112010 con 0x7f6760101de0 2024-09-12T03:11:29.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.552+0000 7f67681a2640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6760111e90 con 0x7f6760106f10 2024-09-12T03:11:29.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.552+0000 7f67681a2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6760112190 con 0x7f6760102790 2024-09-12T03:11:29.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.552+0000 7f6765f3e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6760101de0 0x7f6760198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:29.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.552+0000 7f6765f3e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6760101de0 0x7f6760198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42544/0 (socket says 172.21.15.116:42544) 2024-09-12T03:11:29.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.552+0000 7f6765f3e640 1 -- 172.21.15.116:0/345240241 learned_addr learned my addr 172.21.15.116:0/345240241 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:29.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.553+0000 7f676573d640 1 --2- 172.21.15.116:0/345240241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6760102790 0x7f6760198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:29.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.553+0000 7f6765f3e640 1 -- 172.21.15.116:0/345240241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6760102790 msgr2=0x7f6760198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:29.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.553+0000 7f676673f640 1 --2- 172.21.15.116:0/345240241 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6760106f10 0x7f676019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:29.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.553+0000 7f6765f3e640 1 --2- 172.21.15.116:0/345240241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6760102790 0x7f6760198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.553+0000 7f6765f3e640 1 -- 172.21.15.116:0/345240241 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6760106f10 msgr2=0x7f676019ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:29.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.553+0000 7f6765f3e640 1 --2- 172.21.15.116:0/345240241 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6760106f10 0x7f676019ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.553+0000 7f6765f3e640 1 -- 172.21.15.116:0/345240241 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676019d540 con 0x7f6760101de0 2024-09-12T03:11:29.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.553+0000 7f676673f640 1 --2- 172.21.15.116:0/345240241 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6760106f10 0x7f676019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:29.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.553+0000 7f6765f3e640 1 --2- 172.21.15.116:0/345240241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6760101de0 0x7f6760198590 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f675000b4a0 tx=0x7f675000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:29.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.553+0000 7f6756ffd640 1 -- 172.21.15.116:0/345240241 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750014070 con 0x7f6760101de0 2024-09-12T03:11:29.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.553+0000 7f67681a2640 1 -- 172.21.15.116:0/345240241 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f676019d830 con 0x7f6760101de0 2024-09-12T03:11:29.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.553+0000 7f6756ffd640 1 -- 172.21.15.116:0/345240241 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6750010af0 con 0x7f6760101de0 2024-09-12T03:11:29.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.553+0000 7f6756ffd640 1 -- 172.21.15.116:0/345240241 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750017d10 con 0x7f6760101de0 2024-09-12T03:11:29.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.553+0000 7f67681a2640 1 -- 172.21.15.116:0/345240241 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67601a5110 con 0x7f6760101de0 2024-09-12T03:11:29.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.555+0000 7f6756ffd640 1 -- 172.21.15.116:0/345240241 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6750017070 con 0x7f6760101de0 2024-09-12T03:11:29.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.555+0000 7f676a57b740 1 Processor -- start 2024-09-12T03:11:29.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.556+0000 7f676a57b740 1 -- start start 2024-09-12T03:11:29.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.556+0000 7f676a57b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9fe72b060 0x55c9fe72b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:29.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.556+0000 7f676a57b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9fe72b980 0x55c9fe735d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:29.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.556+0000 7f676a57b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9fe736240 0x55c9fe738630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:29.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.556+0000 7f676a57b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c9fe690620 con 0x55c9fe72b060 2024-09-12T03:11:29.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.556+0000 7f676a57b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c9fe637e50 con 0x55c9fe72b980 2024-09-12T03:11:29.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.556+0000 7f676a57b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c9fe68c5e0 con 0x55c9fe736240 2024-09-12T03:11:29.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.556+0000 7f676673f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9fe72b060 0x55c9fe72b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:29.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.556+0000 7f676673f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9fe72b060 0x55c9fe72b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42558/0 (socket says 172.21.15.116:42558) 2024-09-12T03:11:29.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.556+0000 7f676673f640 1 -- 172.21.15.116:0/2946649409 learned_addr learned my addr 172.21.15.116:0/2946649409 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:29.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.556+0000 7f6756ffd640 1 --2- 172.21.15.116:0/345240241 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f674c0a2b80 0x7f674c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:29.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.556+0000 7f6765f3e640 1 --2- 172.21.15.116:0/2946649409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9fe72b980 0x55c9fe735d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:29.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.556+0000 7f676573d640 1 --2- 172.21.15.116:0/2946649409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9fe736240 0x55c9fe738630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:29.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.556+0000 7f676673f640 1 --2- 172.21.15.116:0/345240241 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f674c0a2b80 0x7f674c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:29.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.556+0000 7f676673f640 1 -- 172.21.15.116:0/2946649409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9fe736240 msgr2=0x55c9fe738630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:29.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.556+0000 7f676673f640 1 --2- 172.21.15.116:0/2946649409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9fe736240 0x55c9fe738630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.556+0000 7f676673f640 1 -- 172.21.15.116:0/2946649409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9fe72b980 msgr2=0x55c9fe735d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:29.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.556+0000 7f676673f640 1 --2- 172.21.15.116:0/2946649409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9fe72b980 0x55c9fe735d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.556+0000 7f6756ffd640 1 -- 172.21.15.116:0/345240241 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f67500ce490 con 0x7f6760101de0 2024-09-12T03:11:29.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.556+0000 7f676673f640 1 -- 172.21.15.116:0/2946649409 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9fe628d40 con 0x55c9fe72b060 2024-09-12T03:11:29.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.557+0000 7f676673f640 1 --2- 172.21.15.116:0/2946649409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9fe72b060 0x55c9fe72b440 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f6758010a30 tx=0x7f6758010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18fdf905afd86906 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:29.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.557+0000 7f676673f640 1 --2- 172.21.15.116:0/345240241 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f674c0a2b80 0x7f674c0a5040 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6758013880 tx=0x7f6758019000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:29.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.557+0000 7f674bfff640 1 -- 172.21.15.116:0/2946649409 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6758020070 con 0x55c9fe72b060 2024-09-12T03:11:29.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.557+0000 7f674bfff640 1 -- 172.21.15.116:0/2946649409 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f675800ede0 con 0x55c9fe72b060 2024-09-12T03:11:29.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.557+0000 7f674bfff640 1 -- 172.21.15.116:0/2946649409 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675800d610 con 0x55c9fe72b060 2024-09-12T03:11:29.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.557+0000 7f676a57b740 1 -- 172.21.15.116:0/2946649409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9fe72b060 msgr2=0x55c9fe72b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:29.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.557+0000 7f676a57b740 1 --2- 172.21.15.116:0/2946649409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9fe72b060 0x55c9fe72b440 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f6758010a30 tx=0x7f6758010ef0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.558+0000 7f676a57b740 1 -- 172.21.15.116:0/2946649409 shutdown_connections 2024-09-12T03:11:29.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.558+0000 7f676a57b740 1 --2- 172.21.15.116:0/2946649409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9fe736240 0x55c9fe738630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.558+0000 7f676a57b740 1 --2- 172.21.15.116:0/2946649409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9fe72b980 0x55c9fe735d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.558+0000 7f676a57b740 1 --2- 172.21.15.116:0/2946649409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9fe72b060 0x55c9fe72b440 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.558+0000 7f676a57b740 1 -- 172.21.15.116:0/2946649409 >> 172.21.15.116:0/2946649409 conn(0x55c9fe7275f0 msgr2=0x55c9fe7279d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:29.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.558+0000 7f676a57b740 1 -- 172.21.15.116:0/2946649409 shutdown_connections 2024-09-12T03:11:29.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.558+0000 7f676a57b740 1 -- 172.21.15.116:0/2946649409 wait complete. 2024-09-12T03:11:29.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.558+0000 7f676a57b740 1 -- start start 2024-09-12T03:11:29.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.558+0000 7f676a57b740 1 Processor -- start 2024-09-12T03:11:29.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.558+0000 7f676a57b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9fe72b060 0x55c9fe733180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:29.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.558+0000 7f676a57b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9fe72b980 0x55c9fe7336c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:29.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.558+0000 7f676a57b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9fe736240 0x55c9fe733c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:29.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.558+0000 7f676a57b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c9fe738e10 con 0x55c9fe736240 2024-09-12T03:11:29.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.558+0000 7f676a57b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c9fe618f10 con 0x55c9fe72b980 2024-09-12T03:11:29.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.558+0000 7f676a57b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c9fe738f90 con 0x55c9fe72b060 2024-09-12T03:11:29.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.559+0000 7f676a57b740 10 client.0 _ll_register_callbacks cb 0x55c9fe63cdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:29.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.559+0000 7f676673f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9fe736240 0x55c9fe733c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:29.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.559+0000 7f676673f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9fe736240 0x55c9fe733c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42562/0 (socket says 172.21.15.116:42562) 2024-09-12T03:11:29.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.559+0000 7f676673f640 1 -- 172.21.15.116:0/1129062007 learned_addr learned my addr 172.21.15.116:0/1129062007 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:29.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.559+0000 7f676573d640 1 --2- 172.21.15.116:0/1129062007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9fe72b980 0x55c9fe7336c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:29.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.559+0000 7f6765f3e640 1 --2- 172.21.15.116:0/1129062007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9fe72b060 0x55c9fe733180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:29.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.559+0000 7f676673f640 1 -- 172.21.15.116:0/1129062007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9fe72b060 msgr2=0x55c9fe733180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:29.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.559+0000 7f676673f640 1 --2- 172.21.15.116:0/1129062007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9fe72b060 0x55c9fe733180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.559+0000 7f676673f640 1 -- 172.21.15.116:0/1129062007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9fe72b980 msgr2=0x55c9fe7336c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:29.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.559+0000 7f676673f640 1 --2- 172.21.15.116:0/1129062007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9fe72b980 0x55c9fe7336c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.559+0000 7f676673f640 1 -- 172.21.15.116:0/1129062007 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9fe39f580 con 0x55c9fe736240 2024-09-12T03:11:29.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.559+0000 7f676673f640 1 --2- 172.21.15.116:0/1129062007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9fe736240 0x55c9fe733c00 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f675801be70 tx=0x7f675801bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de19fb25ff0a7f4b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:29.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.559+0000 7f67497fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:29.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.559+0000 7f67497fa640 1 -- 172.21.15.116:0/1129062007 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6758020040 con 0x55c9fe736240 2024-09-12T03:11:29.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.559+0000 7f67497fa640 1 -- 172.21.15.116:0/1129062007 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f675801b640 con 0x55c9fe736240 2024-09-12T03:11:29.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.559+0000 7f676a57b740 1 -- 172.21.15.116:0/1129062007 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f675800ede0 con 0x55c9fe736240 2024-09-12T03:11:29.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.559+0000 7f67497fa640 1 -- 172.21.15.116:0/1129062007 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6758035070 con 0x55c9fe736240 2024-09-12T03:11:29.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.559+0000 7f676a57b740 10 client.17490 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:29.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.559+0000 7f676a57b740 1 -- 172.21.15.116:0/1129062007 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c9fe469960 con 0x55c9fe736240 2024-09-12T03:11:29.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.560+0000 7f67497fa640 1 -- 172.21.15.116:0/1129062007 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f675800ede0 con 0x55c9fe736240 2024-09-12T03:11:29.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.560+0000 7f67497fa640 1 -- 172.21.15.116:0/1129062007 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6758024cc0 con 0x55c9fe736240 2024-09-12T03:11:29.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.560+0000 7f676a57b740 10 client.17490 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:29.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.560+0000 7f676a57b740 10 client.17490 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:11:29.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.560+0000 7f676a57b740 20 client.17490 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:29.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.560+0000 7f676a57b740 1 --2- 172.21.15.116:0/1129062007 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c9fe745de0 0x55c9fe7481d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:29.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.560+0000 7f676a57b740 4 client.17490 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:11:29.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.560+0000 7f676a57b740 1 -- 172.21.15.116:0/1129062007 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9fe69db90 con 0x55c9fe745de0 2024-09-12T03:11:29.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.560+0000 7f6765f3e640 1 --2- 172.21.15.116:0/1129062007 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c9fe745de0 0x55c9fe7481d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:29.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.560+0000 7f67497fa640 10 client.17490 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:29.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.560+0000 7f6765f3e640 1 --2- 172.21.15.116:0/1129062007 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c9fe745de0 0x55c9fe7481d0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74a8039d326c63ef server_cookie=c7ebea480317ee5f in_seq=0 out_seq=0 2024-09-12T03:11:29.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.561+0000 7f67497fa640 1 -- 172.21.15.116:0/1129062007 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c9fe69db90 con 0x55c9fe745de0 2024-09-12T03:11:29.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.561+0000 7f67497fa640 10 client.17490 handle_command_reply: tid=0 2024-09-12T03:11:29.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.561+0000 7f676a57b740 1 client.17490 shutdown 2024-09-12T03:11:29.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.561+0000 7f676a57b740 1 -- 172.21.15.116:0/1129062007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9fe736240 msgr2=0x55c9fe733c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:29.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.561+0000 7f676a57b740 1 --2- 172.21.15.116:0/1129062007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9fe736240 0x55c9fe733c00 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f675801be70 tx=0x7f675801bcd0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.561+0000 7f676a57b740 1 -- 172.21.15.116:0/1129062007 shutdown_connections 2024-09-12T03:11:29.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.561+0000 7f676a57b740 1 --2- 172.21.15.116:0/1129062007 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c9fe745de0 0x55c9fe7481d0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.561+0000 7f676a57b740 1 --2- 172.21.15.116:0/1129062007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9fe736240 0x55c9fe733c00 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.561+0000 7f67497fa640 0 client.17490 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:29.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.561+0000 7f676a57b740 1 --2- 172.21.15.116:0/1129062007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9fe72b980 0x55c9fe7336c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.561+0000 7f676a57b740 1 --2- 172.21.15.116:0/1129062007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9fe72b060 0x55c9fe733180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.561+0000 7f676a57b740 1 -- 172.21.15.116:0/1129062007 >> 172.21.15.116:0/1129062007 conn(0x55c9fe7275f0 msgr2=0x55c9fe726750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:29.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.561+0000 7f676a57b740 1 -- 172.21.15.116:0/1129062007 shutdown_connections 2024-09-12T03:11:29.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.561+0000 7f676a57b740 1 -- 172.21.15.116:0/1129062007 wait complete. 2024-09-12T03:11:29.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.562+0000 7f676a57b740 20 client.17490 trim_cache size 0 max 16384 2024-09-12T03:11:29.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.578+0000 7f676a57b740 1 Processor -- start 2024-09-12T03:11:29.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.578+0000 7f676a57b740 1 -- start start 2024-09-12T03:11:29.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.578+0000 7f676a57b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9fe72b060 0x55c9fe72d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:29.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.578+0000 7f676a57b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9fe72b980 0x55c9fe74d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:29.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.578+0000 7f676a57b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9fe736240 0x55c9fe74de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:29.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.578+0000 7f676a57b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f67601aff70 con 0x55c9fe72b980 2024-09-12T03:11:29.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.578+0000 7f676a57b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f67601ad840 con 0x55c9fe72b060 2024-09-12T03:11:29.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.578+0000 7f676a57b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f67601ab690 con 0x55c9fe736240 2024-09-12T03:11:29.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.579+0000 7f676573d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9fe72b980 0x55c9fe74d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:29.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.579+0000 7f676573d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9fe72b980 0x55c9fe74d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42572/0 (socket says 172.21.15.116:42572) 2024-09-12T03:11:29.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.579+0000 7f676573d640 1 -- 172.21.15.116:0/4292672397 learned_addr learned my addr 172.21.15.116:0/4292672397 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:29.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.579+0000 7f676673f640 1 --2- 172.21.15.116:0/4292672397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9fe736240 0x55c9fe74de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:29.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.579+0000 7f6765f3e640 1 --2- 172.21.15.116:0/4292672397 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9fe72b060 0x55c9fe72d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:29.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.579+0000 7f676573d640 1 -- 172.21.15.116:0/4292672397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9fe736240 msgr2=0x55c9fe74de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:29.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.579+0000 7f676573d640 1 --2- 172.21.15.116:0/4292672397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9fe736240 0x55c9fe74de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.579+0000 7f676573d640 1 -- 172.21.15.116:0/4292672397 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9fe72b060 msgr2=0x55c9fe72d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:29.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.579+0000 7f676573d640 1 --2- 172.21.15.116:0/4292672397 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9fe72b060 0x55c9fe72d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.579+0000 7f676573d640 1 -- 172.21.15.116:0/4292672397 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9fe728680 con 0x55c9fe72b980 2024-09-12T03:11:29.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.579+0000 7f6765f3e640 1 --2- 172.21.15.116:0/4292672397 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9fe72b060 0x55c9fe72d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:29.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.579+0000 7f676673f640 1 --2- 172.21.15.116:0/4292672397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9fe736240 0x55c9fe74de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:29.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.579+0000 7f676573d640 1 --2- 172.21.15.116:0/4292672397 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9fe72b980 0x55c9fe74d940 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f675c009230 tx=0x7f675c0092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b26ffad8e1d7738 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:29.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.579+0000 7f674b7fe640 1 -- 172.21.15.116:0/4292672397 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c035040 con 0x55c9fe72b980 2024-09-12T03:11:29.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.579+0000 7f674b7fe640 1 -- 172.21.15.116:0/4292672397 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f675c031970 con 0x55c9fe72b980 2024-09-12T03:11:29.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.579+0000 7f674b7fe640 1 -- 172.21.15.116:0/4292672397 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c004920 con 0x55c9fe72b980 2024-09-12T03:11:29.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.580+0000 7f676a57b740 1 -- 172.21.15.116:0/4292672397 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9fe72b980 msgr2=0x55c9fe74d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:29.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.580+0000 7f676a57b740 1 --2- 172.21.15.116:0/4292672397 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9fe72b980 0x55c9fe74d940 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f675c009230 tx=0x7f675c0092d0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.580+0000 7f676a57b740 1 -- 172.21.15.116:0/4292672397 shutdown_connections 2024-09-12T03:11:29.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.580+0000 7f676a57b740 1 --2- 172.21.15.116:0/4292672397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9fe736240 0x55c9fe74de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.580+0000 7f676a57b740 1 --2- 172.21.15.116:0/4292672397 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9fe72b980 0x55c9fe74d940 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.580+0000 7f676a57b740 1 --2- 172.21.15.116:0/4292672397 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9fe72b060 0x55c9fe72d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.580+0000 7f676a57b740 1 -- 172.21.15.116:0/4292672397 >> 172.21.15.116:0/4292672397 conn(0x55c9fe7275f0 msgr2=0x55c9fe72cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:29.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.580+0000 7f676a57b740 1 -- 172.21.15.116:0/4292672397 shutdown_connections 2024-09-12T03:11:29.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.580+0000 7f676a57b740 1 -- 172.21.15.116:0/4292672397 wait complete. 2024-09-12T03:11:29.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.580+0000 7f676a57b740 1 -- start start 2024-09-12T03:11:29.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.580+0000 7f676a57b740 1 Processor -- start 2024-09-12T03:11:29.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.581+0000 7f676a57b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9fe72b060 0x55c9fe73a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:29.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.581+0000 7f676a57b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9fe72b980 0x55c9fe73ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:29.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.581+0000 7f676a57b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9fe736240 0x55c9fe7b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:29.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.581+0000 7f676a57b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f67601abd00 con 0x55c9fe736240 2024-09-12T03:11:29.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.581+0000 7f676a57b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f67601ad170 con 0x55c9fe72b060 2024-09-12T03:11:29.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.581+0000 7f676a57b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f67601ac3d0 con 0x55c9fe72b980 2024-09-12T03:11:29.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.581+0000 7f676a57b740 10 client.0 _ll_register_callbacks cb 0x7f675c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:29.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.581+0000 7f676673f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9fe736240 0x55c9fe7b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:29.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.581+0000 7f676673f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9fe736240 0x55c9fe7b3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42588/0 (socket says 172.21.15.116:42588) 2024-09-12T03:11:29.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.581+0000 7f676573d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9fe72b980 0x55c9fe73ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:29.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.581+0000 7f676673f640 1 -- 172.21.15.116:0/4153002406 learned_addr learned my addr 172.21.15.116:0/4153002406 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:29.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.581+0000 7f6765f3e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9fe72b060 0x55c9fe73a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:29.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.581+0000 7f676673f640 1 -- 172.21.15.116:0/4153002406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9fe72b980 msgr2=0x55c9fe73ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:29.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.581+0000 7f676673f640 1 --2- 172.21.15.116:0/4153002406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9fe72b980 0x55c9fe73ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.581+0000 7f676673f640 1 -- 172.21.15.116:0/4153002406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9fe72b060 msgr2=0x55c9fe73a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:29.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.581+0000 7f676673f640 1 --2- 172.21.15.116:0/4153002406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9fe72b060 0x55c9fe73a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.581+0000 7f676673f640 1 -- 172.21.15.116:0/4153002406 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9fe7b4310 con 0x55c9fe736240 2024-09-12T03:11:29.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.581+0000 7f676673f640 1 --2- 172.21.15.116:0/4153002406 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9fe736240 0x55c9fe7b3d20 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f6758004b90 tx=0x7f6758016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd9983c39cdb9bb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:29.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.581+0000 7f67497fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:29.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.582+0000 7f67497fa640 1 -- 172.21.15.116:0/4153002406 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675803f060 con 0x55c9fe736240 2024-09-12T03:11:29.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.582+0000 7f67497fa640 1 -- 172.21.15.116:0/4153002406 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f6758025a90 con 0x55c9fe736240 2024-09-12T03:11:29.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.582+0000 7f676a57b740 1 -- 172.21.15.116:0/4153002406 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f675c031970 con 0x55c9fe736240 2024-09-12T03:11:29.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.582+0000 7f67497fa640 1 -- 172.21.15.116:0/4153002406 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6758020040 con 0x55c9fe736240 2024-09-12T03:11:29.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.582+0000 7f676a57b740 10 client.17496 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:29.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.582+0000 7f676a57b740 1 -- 172.21.15.116:0/4153002406 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c9fe7b5580 con 0x55c9fe736240 2024-09-12T03:11:29.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.582+0000 7f67497fa640 1 -- 172.21.15.116:0/4153002406 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f675c031970 con 0x55c9fe736240 2024-09-12T03:11:29.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.582+0000 7f67497fa640 1 -- 172.21.15.116:0/4153002406 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f675803a480 con 0x55c9fe736240 2024-09-12T03:11:29.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.582+0000 7f676a57b740 10 client.17496 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:29.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.582+0000 7f676a57b740 10 client.17496 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:11:29.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.582+0000 7f676a57b740 20 client.17496 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:29.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.582+0000 7f676a57b740 1 --2- 172.21.15.116:0/4153002406 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c9fe7b5c40 0x55c9fe7b8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:29.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.582+0000 7f676a57b740 4 client.17496 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d9214b2a", "release": true, "await": true}] 2024-09-12T03:11:29.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.582+0000 7f676a57b740 1 -- 172.21.15.116:0/4153002406 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d9214b2a", "release": true, "await": true}) -- 0x55c9fe722950 con 0x55c9fe7b5c40 2024-09-12T03:11:29.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.582+0000 7f6765f3e640 1 --2- 172.21.15.116:0/4153002406 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c9fe7b5c40 0x55c9fe7b8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:29.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.583+0000 7f6765f3e640 1 --2- 172.21.15.116:0/4153002406 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c9fe7b5c40 0x55c9fe7b8060 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=962145aaec4709f8 server_cookie=ec80283971a4d1d9 in_seq=0 out_seq=0 2024-09-12T03:11:29.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.583+0000 7f67497fa640 10 client.17496 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:29.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.585+0000 7f67497fa640 1 -- 172.21.15.116:0/4153002406 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c9fe722950 con 0x55c9fe7b5c40 2024-09-12T03:11:29.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.585+0000 7f67497fa640 10 client.17496 handle_command_reply: tid=0 2024-09-12T03:11:29.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.585+0000 7f676a57b740 1 client.17496 shutdown 2024-09-12T03:11:29.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.585+0000 7f676a57b740 1 -- 172.21.15.116:0/4153002406 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9fe736240 msgr2=0x55c9fe7b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:29.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.585+0000 7f676a57b740 1 --2- 172.21.15.116:0/4153002406 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9fe736240 0x55c9fe7b3d20 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f6758004b90 tx=0x7f6758016000 comp rx=0 tx=0).stop 2024-09-12T03:11:29.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.585+0000 7f676a57b740 1 -- 172.21.15.116:0/4153002406 shutdown_connections 2024-09-12T03:11:29.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.585+0000 7f676a57b740 1 --2- 172.21.15.116:0/4153002406 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c9fe7b5c40 0x55c9fe7b8060 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.585+0000 7f676a57b740 1 --2- 172.21.15.116:0/4153002406 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9fe736240 0x55c9fe7b3d20 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.585+0000 7f676a57b740 1 --2- 172.21.15.116:0/4153002406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9fe72b980 0x55c9fe73ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.585+0000 7f676a57b740 1 --2- 172.21.15.116:0/4153002406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9fe72b060 0x55c9fe73a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.585+0000 7f676a57b740 1 -- 172.21.15.116:0/4153002406 >> 172.21.15.116:0/4153002406 conn(0x55c9fe7275f0 msgr2=0x55c9fe726750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:29.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.585+0000 7f67497fa640 0 client.17496 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:29.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.585+0000 7f676a57b740 1 -- 172.21.15.116:0/4153002406 shutdown_connections 2024-09-12T03:11:29.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.585+0000 7f676a57b740 1 -- 172.21.15.116:0/4153002406 wait complete. 2024-09-12T03:11:29.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.585+0000 7f676a57b740 20 client.17496 trim_cache size 0 max 16384 2024-09-12T03:11:29.595 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:11:29.595 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 25, 2024-09-12T03:11:29.595 INFO:teuthology.orchestra.run.smithi116.stdout: "leader": 24494, 2024-09-12T03:11:29.595 INFO:teuthology.orchestra.run.smithi116.stdout: "set_version": 36, 2024-09-12T03:11:29.595 INFO:teuthology.orchestra.run.smithi116.stdout: "sets": { 2024-09-12T03:11:29.596 INFO:teuthology.orchestra.run.smithi116.stdout: "d9214b2a": { 2024-09-12T03:11:29.596 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 36, 2024-09-12T03:11:29.596 INFO:teuthology.orchestra.run.smithi116.stdout: "age_ref": 0.0, 2024-09-12T03:11:29.596 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:11:29.596 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RELEASED", 2024-09-12T03:11:29.596 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:11:29.596 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:11:29.596 INFO:teuthology.orchestra.run.smithi116.stdout: "timeout": 90.0, 2024-09-12T03:11:29.596 INFO:teuthology.orchestra.run.smithi116.stdout: "expiration": 139.4, 2024-09-12T03:11:29.596 INFO:teuthology.orchestra.run.smithi116.stdout: "members": { 2024-09-12T03:11:29.596 INFO:teuthology.orchestra.run.smithi116.stdout: "file:/": { 2024-09-12T03:11:29.596 INFO:teuthology.orchestra.run.smithi116.stdout: "excluded": false, 2024-09-12T03:11:29.596 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:11:29.597 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RELEASED", 2024-09-12T03:11:29.597 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:11:29.597 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:11:29.597 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:11:29.597 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:11:29.597 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:11:29.597 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:11:29.597 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:11:29.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.586+0000 7f67681a2640 1 -- 172.21.15.116:0/345240241 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f674c0a2b80 msgr2=0x7f674c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:29.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.586+0000 7f67681a2640 1 --2- 172.21.15.116:0/345240241 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f674c0a2b80 0x7f674c0a5040 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6758013880 tx=0x7f6758019000 comp rx=0 tx=0).stop 2024-09-12T03:11:29.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.586+0000 7f67681a2640 1 -- 172.21.15.116:0/345240241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6760101de0 msgr2=0x7f6760198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:29.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.586+0000 7f67681a2640 1 --2- 172.21.15.116:0/345240241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6760101de0 0x7f6760198590 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f675000b4a0 tx=0x7f675000b960 comp rx=0 tx=0).stop 2024-09-12T03:11:29.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.586+0000 7f67681a2640 1 -- 172.21.15.116:0/345240241 shutdown_connections 2024-09-12T03:11:29.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.586+0000 7f67681a2640 1 --2- 172.21.15.116:0/345240241 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f674c0a2b80 0x7f674c0a5040 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.586+0000 7f67681a2640 1 --2- 172.21.15.116:0/345240241 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6760106f10 0x7f676019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.586+0000 7f67681a2640 1 --2- 172.21.15.116:0/345240241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6760102790 0x7f6760198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.586+0000 7f67681a2640 1 --2- 172.21.15.116:0/345240241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6760101de0 0x7f6760198590 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:29.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.586+0000 7f67681a2640 1 -- 172.21.15.116:0/345240241 >> 172.21.15.116:0/345240241 conn(0x7f67600fd8c0 msgr2=0x7f67600fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:29.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.586+0000 7f67681a2640 1 -- 172.21.15.116:0/345240241 shutdown_connections 2024-09-12T03:11:29.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:29.586+0000 7f67681a2640 1 -- 172.21.15.116:0/345240241 wait complete. 2024-09-12T03:11:30.401 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd9214b2a', total seconds elapsed: 22.8 2024-09-12T03:11:30.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:30 smithi183 ceph-mon[35535]: pgmap v593: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-12T03:11:30.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:30 smithi177 ceph-mon[35305]: pgmap v593: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-12T03:11:30.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:30 smithi116 ceph-mon[31410]: pgmap v593: 97 pgs: 97 active+clean; 4.0 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-12T03:11:31.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:31 smithi183 ceph-mon[35535]: pgmap v594: 97 pgs: 97 active+clean; 4.1 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.6 KiB/s wr, 7 op/s 2024-09-12T03:11:31.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:31 smithi177 ceph-mon[35305]: pgmap v594: 97 pgs: 97 active+clean; 4.1 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.6 KiB/s wr, 7 op/s 2024-09-12T03:11:31.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:31 smithi116 ceph-mon[31410]: pgmap v594: 97 pgs: 97 active+clean; 4.1 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.6 KiB/s wr, 7 op/s 2024-09-12T03:11:32.189 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:11:32.447 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:11:33.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.097+0000 7ff218958640 1 -- 172.21.15.116:0/3294610263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff214103660 msgr2=0x7ff21410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:33.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.097+0000 7ff218958640 1 --2- 172.21.15.116:0/3294610263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff214103660 0x7ff21410bb50 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009900 tx=0x7ff1fc02a480 comp rx=0 tx=0).stop 2024-09-12T03:11:33.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.097+0000 7ff218958640 1 -- 172.21.15.116:0/3294610263 shutdown_connections 2024-09-12T03:11:33.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.097+0000 7ff218958640 1 --2- 172.21.15.116:0/3294610263 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff21410c170 0x7ff21410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.097+0000 7ff218958640 1 --2- 172.21.15.116:0/3294610263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff214103660 0x7ff21410bb50 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.097+0000 7ff218958640 1 --2- 172.21.15.116:0/3294610263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff214102d40 0x7ff214103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.097+0000 7ff218958640 1 -- 172.21.15.116:0/3294610263 >> 172.21.15.116:0/3294610263 conn(0x7ff2140fc7a0 msgr2=0x7ff2140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:33.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.097+0000 7ff218958640 1 -- 172.21.15.116:0/3294610263 shutdown_connections 2024-09-12T03:11:33.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.097+0000 7ff218958640 1 -- 172.21.15.116:0/3294610263 wait complete. 2024-09-12T03:11:33.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.098+0000 7ff218958640 1 Processor -- start 2024-09-12T03:11:33.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.098+0000 7ff218958640 1 -- start start 2024-09-12T03:11:33.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.098+0000 7ff218958640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff214102d40 0x7ff2141984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:33.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.098+0000 7ff218958640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff214103660 0x7ff214198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:33.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.098+0000 7ff218958640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff21410c170 0x7ff21419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:33.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.098+0000 7ff218958640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff214111f70 con 0x7ff214103660 2024-09-12T03:11:33.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.098+0000 7ff218958640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff214111df0 con 0x7ff214102d40 2024-09-12T03:11:33.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.098+0000 7ff218958640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff2141120f0 con 0x7ff21410c170 2024-09-12T03:11:33.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.098+0000 7ff211d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff214103660 0x7ff214198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:33.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.098+0000 7ff211d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff214103660 0x7ff214198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48446/0 (socket says 172.21.15.116:48446) 2024-09-12T03:11:33.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.098+0000 7ff211d9b640 1 -- 172.21.15.116:0/1557193944 learned_addr learned my addr 172.21.15.116:0/1557193944 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:33.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.099+0000 7ff21259c640 1 --2- 172.21.15.116:0/1557193944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff214102d40 0x7ff2141984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:33.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.099+0000 7ff212d9d640 1 --2- 172.21.15.116:0/1557193944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff21410c170 0x7ff21419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:33.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.099+0000 7ff211d9b640 1 -- 172.21.15.116:0/1557193944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff21410c170 msgr2=0x7ff21419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:33.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.099+0000 7ff211d9b640 1 --2- 172.21.15.116:0/1557193944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff21410c170 0x7ff21419cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.099+0000 7ff211d9b640 1 -- 172.21.15.116:0/1557193944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff214102d40 msgr2=0x7ff2141984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:33.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.099+0000 7ff211d9b640 1 --2- 172.21.15.116:0/1557193944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff214102d40 0x7ff2141984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.099+0000 7ff211d9b640 1 -- 172.21.15.116:0/1557193944 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21419d480 con 0x7ff214103660 2024-09-12T03:11:33.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.099+0000 7ff211d9b640 1 --2- 172.21.15.116:0/1557193944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff214103660 0x7ff214198a10 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc006010 tx=0x7ff1fc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:33.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.099+0000 7ff20b7fe640 1 -- 172.21.15.116:0/1557193944 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc03f030 con 0x7ff214103660 2024-09-12T03:11:33.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.099+0000 7ff218958640 1 -- 172.21.15.116:0/1557193944 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff21419d710 con 0x7ff214103660 2024-09-12T03:11:33.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.099+0000 7ff20b7fe640 1 -- 172.21.15.116:0/1557193944 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1fc003ce0 con 0x7ff214103660 2024-09-12T03:11:33.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.099+0000 7ff20b7fe640 1 -- 172.21.15.116:0/1557193944 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc035070 con 0x7ff214103660 2024-09-12T03:11:33.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.099+0000 7ff218958640 1 -- 172.21.15.116:0/1557193944 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2141a5050 con 0x7ff214103660 2024-09-12T03:11:33.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.102+0000 7ff20b7fe640 1 -- 172.21.15.116:0/1557193944 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ff1fc042030 con 0x7ff214103660 2024-09-12T03:11:33.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.102+0000 7ff21ad31740 1 Processor -- start 2024-09-12T03:11:33.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.102+0000 7ff21ad31740 1 -- start start 2024-09-12T03:11:33.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.102+0000 7ff21ad31740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646839bd4a0 0x5646839bd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:33.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.102+0000 7ff21ad31740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5646839bddc0 0x5646839c8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:33.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.102+0000 7ff21ad31740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646839c8680 0x5646839caa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:33.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.102+0000 7ff21ad31740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5646839232e0 con 0x5646839bddc0 2024-09-12T03:11:33.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.102+0000 7ff21ad31740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5646838cace0 con 0x5646839c8680 2024-09-12T03:11:33.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.102+0000 7ff21ad31740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56468391f2a0 con 0x5646839bd4a0 2024-09-12T03:11:33.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.103+0000 7ff21259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5646839bddc0 0x5646839c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:33.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.103+0000 7ff21259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5646839bddc0 0x5646839c8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48450/0 (socket says 172.21.15.116:48450) 2024-09-12T03:11:33.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.103+0000 7ff21259c640 1 -- 172.21.15.116:0/126803282 learned_addr learned my addr 172.21.15.116:0/126803282 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:33.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.103+0000 7ff20b7fe640 1 --2- 172.21.15.116:0/1557193944 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff2000a2b80 0x7ff2000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:33.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.103+0000 7ff211d9b640 1 --2- 172.21.15.116:0/126803282 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646839c8680 0x5646839caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:33.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.103+0000 7ff212d9d640 1 --2- 172.21.15.116:0/126803282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646839bd4a0 0x5646839bd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:33.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.103+0000 7ff212d9d640 1 --2- 172.21.15.116:0/1557193944 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff2000a2b80 0x7ff2000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:33.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.103+0000 7ff21259c640 1 -- 172.21.15.116:0/126803282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646839bd4a0 msgr2=0x5646839bd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:33.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.103+0000 7ff21259c640 1 --2- 172.21.15.116:0/126803282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646839bd4a0 0x5646839bd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.103+0000 7ff21259c640 1 -- 172.21.15.116:0/126803282 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646839c8680 msgr2=0x5646839caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:33.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.103+0000 7ff21259c640 1 --2- 172.21.15.116:0/126803282 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646839c8680 0x5646839caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.103+0000 7ff21259c640 1 -- 172.21.15.116:0/126803282 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646838af440 con 0x5646839bddc0 2024-09-12T03:11:33.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.103+0000 7ff20b7fe640 1 -- 172.21.15.116:0/1557193944 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff1fc0ef5f0 con 0x7ff214103660 2024-09-12T03:11:33.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.103+0000 7ff21259c640 1 --2- 172.21.15.116:0/126803282 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5646839bddc0 0x5646839c8140 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7ff20400ac10 tx=0x7ff20400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=143dcceaf9b9d358 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:33.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.103+0000 7ff208ff9640 1 -- 172.21.15.116:0/126803282 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20401d070 con 0x5646839bddc0 2024-09-12T03:11:33.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.103+0000 7ff208ff9640 1 -- 172.21.15.116:0/126803282 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff204018b20 con 0x5646839bddc0 2024-09-12T03:11:33.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.103+0000 7ff212d9d640 1 --2- 172.21.15.116:0/1557193944 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff2000a2b80 0x7ff2000a5040 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff20c0092f0 tx=0x7ff20c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:33.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.103+0000 7ff208ff9640 1 -- 172.21.15.116:0/126803282 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff204010ba0 con 0x5646839bddc0 2024-09-12T03:11:33.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.104+0000 7ff21ad31740 1 -- 172.21.15.116:0/126803282 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5646839bddc0 msgr2=0x5646839c8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:33.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.104+0000 7ff21ad31740 1 --2- 172.21.15.116:0/126803282 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5646839bddc0 0x5646839c8140 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7ff20400ac10 tx=0x7ff20400f790 comp rx=0 tx=0).stop 2024-09-12T03:11:33.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.104+0000 7ff21ad31740 1 -- 172.21.15.116:0/126803282 shutdown_connections 2024-09-12T03:11:33.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.104+0000 7ff21ad31740 1 --2- 172.21.15.116:0/126803282 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646839c8680 0x5646839caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.104+0000 7ff21ad31740 1 --2- 172.21.15.116:0/126803282 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5646839bddc0 0x5646839c8140 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.104+0000 7ff21ad31740 1 --2- 172.21.15.116:0/126803282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646839bd4a0 0x5646839bd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.104+0000 7ff21ad31740 1 -- 172.21.15.116:0/126803282 >> 172.21.15.116:0/126803282 conn(0x5646839b9a30 msgr2=0x5646839b9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:33.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.104+0000 7ff21ad31740 1 -- 172.21.15.116:0/126803282 shutdown_connections 2024-09-12T03:11:33.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.104+0000 7ff21ad31740 1 -- 172.21.15.116:0/126803282 wait complete. 2024-09-12T03:11:33.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.104+0000 7ff21ad31740 1 -- start start 2024-09-12T03:11:33.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.105+0000 7ff21ad31740 1 Processor -- start 2024-09-12T03:11:33.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.105+0000 7ff21ad31740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646839bd4a0 0x5646839c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:33.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.105+0000 7ff21ad31740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5646839bddc0 0x5646839c5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:33.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.105+0000 7ff21ad31740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646839c8680 0x5646839c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:33.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.105+0000 7ff21ad31740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5646839cb250 con 0x5646839bddc0 2024-09-12T03:11:33.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.105+0000 7ff21ad31740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5646838de050 con 0x5646839bd4a0 2024-09-12T03:11:33.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.105+0000 7ff21ad31740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5646839cb3d0 con 0x5646839c8680 2024-09-12T03:11:33.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.105+0000 7ff21ad31740 10 client.0 _ll_register_callbacks cb 0x56468391e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:33.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.105+0000 7ff211d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5646839bddc0 0x5646839c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:33.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.105+0000 7ff211d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5646839bddc0 0x5646839c5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48466/0 (socket says 172.21.15.116:48466) 2024-09-12T03:11:33.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.105+0000 7ff211d9b640 1 -- 172.21.15.116:0/3116065586 learned_addr learned my addr 172.21.15.116:0/3116065586 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:33.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.105+0000 7ff212d9d640 1 --2- 172.21.15.116:0/3116065586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646839c8680 0x5646839c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:33.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.105+0000 7ff21259c640 1 --2- 172.21.15.116:0/3116065586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646839bd4a0 0x5646839c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:33.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.105+0000 7ff211d9b640 1 -- 172.21.15.116:0/3116065586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646839c8680 msgr2=0x5646839c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:33.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.105+0000 7ff211d9b640 1 --2- 172.21.15.116:0/3116065586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646839c8680 0x5646839c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.105+0000 7ff211d9b640 1 -- 172.21.15.116:0/3116065586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646839bd4a0 msgr2=0x5646839c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:33.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.105+0000 7ff211d9b640 1 --2- 172.21.15.116:0/3116065586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646839bd4a0 0x5646839c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.105+0000 7ff211d9b640 1 -- 172.21.15.116:0/3116065586 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646836943c0 con 0x5646839bddc0 2024-09-12T03:11:33.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.105+0000 7ff211d9b640 1 --2- 172.21.15.116:0/3116065586 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5646839bddc0 0x5646839c5250 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc0c4d70 tx=0x7ff1fc005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0dc320bcd455e13 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:33.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.105+0000 7ff1f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:33.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.106+0000 7ff1f9ffb640 1 -- 172.21.15.116:0/3116065586 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc049040 con 0x5646839bddc0 2024-09-12T03:11:33.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.106+0000 7ff1f9ffb640 1 -- 172.21.15.116:0/3116065586 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff1fc0bc1a0 con 0x5646839bddc0 2024-09-12T03:11:33.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.106+0000 7ff21ad31740 1 -- 172.21.15.116:0/3116065586 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff204018b20 con 0x5646839bddc0 2024-09-12T03:11:33.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.106+0000 7ff1f9ffb640 1 -- 172.21.15.116:0/3116065586 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff1fc0bb8a0 con 0x5646839bddc0 2024-09-12T03:11:33.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.106+0000 7ff21ad31740 10 client.17508 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:33.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.106+0000 7ff21ad31740 1 -- 172.21.15.116:0/3116065586 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5646839d6bd0 con 0x5646839bddc0 2024-09-12T03:11:33.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.106+0000 7ff1f9ffb640 1 -- 172.21.15.116:0/3116065586 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff204018b20 con 0x5646839bddc0 2024-09-12T03:11:33.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.106+0000 7ff1f9ffb640 1 -- 172.21.15.116:0/3116065586 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff1fc03bbb0 con 0x5646839bddc0 2024-09-12T03:11:33.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.106+0000 7ff21ad31740 10 client.17508 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:33.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.106+0000 7ff21ad31740 10 client.17508 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:11:33.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.106+0000 7ff21ad31740 20 client.17508 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:33.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.106+0000 7ff21ad31740 1 --2- 172.21.15.116:0/3116065586 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5646839d7fe0 0x5646839da410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:33.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.106+0000 7ff21ad31740 4 client.17508 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:11:33.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.106+0000 7ff21ad31740 1 -- 172.21.15.116:0/3116065586 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564683930850 con 0x5646839d7fe0 2024-09-12T03:11:33.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.107+0000 7ff21259c640 1 --2- 172.21.15.116:0/3116065586 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5646839d7fe0 0x5646839da410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:33.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.107+0000 7ff1f9ffb640 10 client.17508 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:33.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.107+0000 7ff21259c640 1 --2- 172.21.15.116:0/3116065586 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5646839d7fe0 0x5646839da410 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bc3d4db1c8a769 server_cookie=ab68ee293e368fdd in_seq=0 out_seq=0 2024-09-12T03:11:33.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.108+0000 7ff1f9ffb640 1 -- 172.21.15.116:0/3116065586 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564683930850 con 0x5646839d7fe0 2024-09-12T03:11:33.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.108+0000 7ff1f9ffb640 10 client.17508 handle_command_reply: tid=0 2024-09-12T03:11:33.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.108+0000 7ff21ad31740 1 client.17508 shutdown 2024-09-12T03:11:33.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.108+0000 7ff21ad31740 1 -- 172.21.15.116:0/3116065586 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5646839bddc0 msgr2=0x5646839c5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:33.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.108+0000 7ff21ad31740 1 --2- 172.21.15.116:0/3116065586 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5646839bddc0 0x5646839c5250 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc0c4d70 tx=0x7ff1fc005e50 comp rx=0 tx=0).stop 2024-09-12T03:11:33.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.108+0000 7ff21ad31740 1 -- 172.21.15.116:0/3116065586 shutdown_connections 2024-09-12T03:11:33.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.108+0000 7ff21ad31740 1 --2- 172.21.15.116:0/3116065586 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5646839d7fe0 0x5646839da410 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.108+0000 7ff21ad31740 1 --2- 172.21.15.116:0/3116065586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646839c8680 0x5646839c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.108+0000 7ff21ad31740 1 --2- 172.21.15.116:0/3116065586 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5646839bddc0 0x5646839c5250 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.108+0000 7ff21ad31740 1 --2- 172.21.15.116:0/3116065586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646839bd4a0 0x5646839c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.108+0000 7ff21ad31740 1 -- 172.21.15.116:0/3116065586 >> 172.21.15.116:0/3116065586 conn(0x5646839b9a30 msgr2=0x5646839bf710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:33.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.108+0000 7ff1f9ffb640 0 client.17508 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:33.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.108+0000 7ff21ad31740 1 -- 172.21.15.116:0/3116065586 shutdown_connections 2024-09-12T03:11:33.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.108+0000 7ff21ad31740 1 -- 172.21.15.116:0/3116065586 wait complete. 2024-09-12T03:11:33.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.108+0000 7ff21ad31740 20 client.17508 trim_cache size 0 max 16384 2024-09-12T03:11:33.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.127+0000 7ff21ad31740 1 Processor -- start 2024-09-12T03:11:33.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.127+0000 7ff21ad31740 1 -- start start 2024-09-12T03:11:33.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.127+0000 7ff21ad31740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5646839bd4a0 0x5646839d44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:33.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.127+0000 7ff21ad31740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646839bddc0 0x5646839c3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:33.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.127+0000 7ff21ad31740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646839c8680 0x5646839c0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:33.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.127+0000 7ff21ad31740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff2141afca0 con 0x5646839bd4a0 2024-09-12T03:11:33.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.127+0000 7ff21ad31740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff2141ad570 con 0x5646839c8680 2024-09-12T03:11:33.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.127+0000 7ff21ad31740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff2141ab3c0 con 0x5646839bddc0 2024-09-12T03:11:33.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.127+0000 7ff21259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5646839bd4a0 0x5646839d44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:33.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.127+0000 7ff21259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5646839bd4a0 0x5646839d44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48468/0 (socket says 172.21.15.116:48468) 2024-09-12T03:11:33.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.127+0000 7ff21259c640 1 -- 172.21.15.116:0/2197135948 learned_addr learned my addr 172.21.15.116:0/2197135948 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:33.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.128+0000 7ff211d9b640 1 --2- 172.21.15.116:0/2197135948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646839bddc0 0x5646839c3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:33.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.128+0000 7ff21259c640 1 -- 172.21.15.116:0/2197135948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646839bddc0 msgr2=0x5646839c3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:33.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.128+0000 7ff212d9d640 1 --2- 172.21.15.116:0/2197135948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646839c8680 0x5646839c0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:33.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.128+0000 7ff21259c640 1 --2- 172.21.15.116:0/2197135948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646839bddc0 0x5646839c3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.128+0000 7ff21259c640 1 -- 172.21.15.116:0/2197135948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646839c8680 msgr2=0x5646839c0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:33.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.128+0000 7ff21259c640 1 --2- 172.21.15.116:0/2197135948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646839c8680 0x5646839c0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.128+0000 7ff21259c640 1 -- 172.21.15.116:0/2197135948 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646839c4200 con 0x5646839bd4a0 2024-09-12T03:11:33.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.128+0000 7ff21259c640 1 --2- 172.21.15.116:0/2197135948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5646839bd4a0 0x5646839d44b0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7ff2040058c0 tx=0x7ff20400a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee983b1ede79c98 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:33.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.128+0000 7ff1fbfff640 1 -- 172.21.15.116:0/2197135948 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20401d040 con 0x5646839bd4a0 2024-09-12T03:11:33.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.128+0000 7ff1fbfff640 1 -- 172.21.15.116:0/2197135948 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff20401e040 con 0x5646839bd4a0 2024-09-12T03:11:33.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.128+0000 7ff1fbfff640 1 -- 172.21.15.116:0/2197135948 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff204018ab0 con 0x5646839bd4a0 2024-09-12T03:11:33.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.129+0000 7ff21ad31740 1 -- 172.21.15.116:0/2197135948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5646839bd4a0 msgr2=0x5646839d44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:33.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.129+0000 7ff21ad31740 1 --2- 172.21.15.116:0/2197135948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5646839bd4a0 0x5646839d44b0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7ff2040058c0 tx=0x7ff20400a8e0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.129+0000 7ff21ad31740 1 -- 172.21.15.116:0/2197135948 shutdown_connections 2024-09-12T03:11:33.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.129+0000 7ff21ad31740 1 --2- 172.21.15.116:0/2197135948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646839c8680 0x5646839c0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.129+0000 7ff21ad31740 1 --2- 172.21.15.116:0/2197135948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646839bddc0 0x5646839c3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.129+0000 7ff21ad31740 1 --2- 172.21.15.116:0/2197135948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5646839bd4a0 0x5646839d44b0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.129+0000 7ff21ad31740 1 -- 172.21.15.116:0/2197135948 >> 172.21.15.116:0/2197135948 conn(0x5646839b9a30 msgr2=0x5646839c0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:33.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.129+0000 7ff21ad31740 1 -- 172.21.15.116:0/2197135948 shutdown_connections 2024-09-12T03:11:33.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.129+0000 7ff21ad31740 1 -- 172.21.15.116:0/2197135948 wait complete. 2024-09-12T03:11:33.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.129+0000 7ff21ad31740 1 -- start start 2024-09-12T03:11:33.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.129+0000 7ff21ad31740 1 Processor -- start 2024-09-12T03:11:33.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.129+0000 7ff21ad31740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5646839bd4a0 0x564683a3e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:33.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.130+0000 7ff21ad31740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646839bddc0 0x5646839d0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:33.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.130+0000 7ff21ad31740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646839c8680 0x5646839d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:33.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.130+0000 7ff21ad31740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff2141aba30 con 0x5646839bd4a0 2024-09-12T03:11:33.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.130+0000 7ff21ad31740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff2141acea0 con 0x5646839bddc0 2024-09-12T03:11:33.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.130+0000 7ff21ad31740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff2141ac100 con 0x5646839c8680 2024-09-12T03:11:33.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.130+0000 7ff21ad31740 10 client.0 _ll_register_callbacks cb 0x7ff1fc03b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:33.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.130+0000 7ff21259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5646839bd4a0 0x564683a3e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:33.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.130+0000 7ff21259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5646839bd4a0 0x564683a3e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48478/0 (socket says 172.21.15.116:48478) 2024-09-12T03:11:33.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.130+0000 7ff21259c640 1 -- 172.21.15.116:0/1348649377 learned_addr learned my addr 172.21.15.116:0/1348649377 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:33.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.130+0000 7ff212d9d640 1 --2- 172.21.15.116:0/1348649377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646839c8680 0x5646839d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:33.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.130+0000 7ff211d9b640 1 --2- 172.21.15.116:0/1348649377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646839bddc0 0x5646839d0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:33.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.130+0000 7ff21259c640 1 -- 172.21.15.116:0/1348649377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646839c8680 msgr2=0x5646839d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:33.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.130+0000 7ff21259c640 1 --2- 172.21.15.116:0/1348649377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646839c8680 0x5646839d0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.130+0000 7ff21259c640 1 -- 172.21.15.116:0/1348649377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646839bddc0 msgr2=0x5646839d0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:33.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.130+0000 7ff21259c640 1 --2- 172.21.15.116:0/1348649377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646839bddc0 0x5646839d0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.130+0000 7ff21259c640 1 -- 172.21.15.116:0/1348649377 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646839bdbf0 con 0x5646839bd4a0 2024-09-12T03:11:33.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.130+0000 7ff211d9b640 1 --2- 172.21.15.116:0/1348649377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646839bddc0 0x5646839d0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:33.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.130+0000 7ff212d9d640 1 --2- 172.21.15.116:0/1348649377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646839c8680 0x5646839d0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:11:33.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.130+0000 7ff21259c640 1 --2- 172.21.15.116:0/1348649377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5646839bd4a0 0x564683a3e4b0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7ff20400ab90 tx=0x7ff20400abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb1c1b5d785820a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:33.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.130+0000 7ff1f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:33.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.130+0000 7ff1f9ffb640 1 -- 172.21.15.116:0/1348649377 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20401d040 con 0x5646839bd4a0 2024-09-12T03:11:33.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.130+0000 7ff1f9ffb640 1 -- 172.21.15.116:0/1348649377 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff20402a040 con 0x5646839bd4a0 2024-09-12T03:11:33.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.130+0000 7ff21ad31740 1 -- 172.21.15.116:0/1348649377 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff20401e040 con 0x5646839bd4a0 2024-09-12T03:11:33.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.131+0000 7ff1f9ffb640 1 -- 172.21.15.116:0/1348649377 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff2040056b0 con 0x5646839bd4a0 2024-09-12T03:11:33.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.131+0000 7ff21ad31740 10 client.17514 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:33.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.131+0000 7ff21ad31740 1 -- 172.21.15.116:0/1348649377 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564683a470c0 con 0x5646839bd4a0 2024-09-12T03:11:33.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.131+0000 7ff1f9ffb640 1 -- 172.21.15.116:0/1348649377 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff20401e040 con 0x5646839bd4a0 2024-09-12T03:11:33.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.131+0000 7ff1f9ffb640 1 -- 172.21.15.116:0/1348649377 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff204024430 con 0x5646839bd4a0 2024-09-12T03:11:33.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.131+0000 7ff21ad31740 10 client.17514 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:33.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.131+0000 7ff21ad31740 10 client.17514 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:11:33.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.131+0000 7ff21ad31740 20 client.17514 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:33.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.131+0000 7ff21ad31740 1 --2- 172.21.15.116:0/1348649377 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x564683a47530 0x564683a49920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:33.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.131+0000 7ff21ad31740 4 client.17514 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:11:33.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.131+0000 7ff21ad31740 1 -- 172.21.15.116:0/1348649377 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56468392ad60 con 0x564683a47530 2024-09-12T03:11:33.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.132+0000 7ff211d9b640 1 --2- 172.21.15.116:0/1348649377 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x564683a47530 0x564683a49920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:33.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.132+0000 7ff211d9b640 1 --2- 172.21.15.116:0/1348649377 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x564683a47530 0x564683a49920 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b5c1ebd2c5d5000 server_cookie=afa7ca3ea30132ef in_seq=0 out_seq=0 2024-09-12T03:11:33.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.132+0000 7ff1f9ffb640 10 client.17514 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:33.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.132+0000 7ff1f9ffb640 1 -- 172.21.15.116:0/1348649377 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56468392ad60 con 0x564683a47530 2024-09-12T03:11:33.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.132+0000 7ff1f9ffb640 10 client.17514 handle_command_reply: tid=0 2024-09-12T03:11:33.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.132+0000 7ff21ad31740 1 client.17514 shutdown 2024-09-12T03:11:33.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.132+0000 7ff21ad31740 1 -- 172.21.15.116:0/1348649377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5646839bd4a0 msgr2=0x564683a3e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:33.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.132+0000 7ff21ad31740 1 --2- 172.21.15.116:0/1348649377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5646839bd4a0 0x564683a3e4b0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7ff20400ab90 tx=0x7ff20400abc0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.132+0000 7ff21ad31740 1 -- 172.21.15.116:0/1348649377 shutdown_connections 2024-09-12T03:11:33.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.132+0000 7ff21ad31740 1 --2- 172.21.15.116:0/1348649377 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x564683a47530 0x564683a49920 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.132+0000 7ff21ad31740 1 --2- 172.21.15.116:0/1348649377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646839c8680 0x5646839d0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.132+0000 7ff21ad31740 1 --2- 172.21.15.116:0/1348649377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5646839bddc0 0x5646839d0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.132+0000 7ff21ad31740 1 --2- 172.21.15.116:0/1348649377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5646839bd4a0 0x564683a3e4b0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.132+0000 7ff21ad31740 1 -- 172.21.15.116:0/1348649377 >> 172.21.15.116:0/1348649377 conn(0x5646839b9a30 msgr2=0x5646839beec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:33.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.132+0000 7ff1f9ffb640 0 client.17514 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:33.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.132+0000 7ff21ad31740 1 -- 172.21.15.116:0/1348649377 shutdown_connections 2024-09-12T03:11:33.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.132+0000 7ff21ad31740 1 -- 172.21.15.116:0/1348649377 wait complete. 2024-09-12T03:11:33.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.133+0000 7ff21ad31740 20 client.17514 trim_cache size 0 max 16384 2024-09-12T03:11:33.146 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:11:33.146 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:11:33.146 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:11:33.146 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:11:33.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.133+0000 7ff218958640 1 -- 172.21.15.116:0/1557193944 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff2000a2b80 msgr2=0x7ff2000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:33.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.133+0000 7ff218958640 1 --2- 172.21.15.116:0/1557193944 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff2000a2b80 0x7ff2000a5040 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff20c0092f0 tx=0x7ff20c004a40 comp rx=0 tx=0).stop 2024-09-12T03:11:33.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.133+0000 7ff218958640 1 -- 172.21.15.116:0/1557193944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff214103660 msgr2=0x7ff214198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:33.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.133+0000 7ff218958640 1 --2- 172.21.15.116:0/1557193944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff214103660 0x7ff214198a10 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc006010 tx=0x7ff1fc0042f0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.133+0000 7ff218958640 1 -- 172.21.15.116:0/1557193944 shutdown_connections 2024-09-12T03:11:33.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.133+0000 7ff218958640 1 --2- 172.21.15.116:0/1557193944 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff2000a2b80 0x7ff2000a5040 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.133+0000 7ff218958640 1 --2- 172.21.15.116:0/1557193944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff21410c170 0x7ff21419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.133+0000 7ff218958640 1 --2- 172.21.15.116:0/1557193944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff214103660 0x7ff214198a10 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.133+0000 7ff218958640 1 --2- 172.21.15.116:0/1557193944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff214102d40 0x7ff2141984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:33.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.133+0000 7ff218958640 1 -- 172.21.15.116:0/1557193944 >> 172.21.15.116:0/1557193944 conn(0x7ff2140fc7a0 msgr2=0x7ff2140fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:33.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.133+0000 7ff218958640 1 -- 172.21.15.116:0/1557193944 shutdown_connections 2024-09-12T03:11:33.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:33.133+0000 7ff218958640 1 -- 172.21.15.116:0/1557193944 wait complete. 2024-09-12T03:11:33.959 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:11:33.959 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:11:34.204 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:11:34.592 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:34 smithi116 ceph-mon[31410]: pgmap v595: 97 pgs: 97 active+clean; 4.1 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-12T03:11:34.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.672+0000 7f1ff28c9640 1 -- 172.21.15.116:0/2702891528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fec102dc0 msgr2=0x7f1fec1031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:34.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.672+0000 7f1ff28c9640 1 --2- 172.21.15.116:0/2702891528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fec102dc0 0x7f1fec1031a0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc009900 tx=0x7f1fdc02a480 comp rx=0 tx=0).stop 2024-09-12T03:11:34.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.672+0000 7f1ff28c9640 1 -- 172.21.15.116:0/2702891528 shutdown_connections 2024-09-12T03:11:34.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.672+0000 7f1ff28c9640 1 --2- 172.21.15.116:0/2702891528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fec10c1a0 0x7f1fec10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:34.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.672+0000 7f1ff28c9640 1 --2- 172.21.15.116:0/2702891528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fec1036e0 0x7f1fec10bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:34.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.672+0000 7f1ff28c9640 1 --2- 172.21.15.116:0/2702891528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fec102dc0 0x7f1fec1031a0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:34.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.672+0000 7f1ff28c9640 1 -- 172.21.15.116:0/2702891528 >> 172.21.15.116:0/2702891528 conn(0x7f1fec0fc7a0 msgr2=0x7f1fec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:34.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.672+0000 7f1ff28c9640 1 -- 172.21.15.116:0/2702891528 shutdown_connections 2024-09-12T03:11:34.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.672+0000 7f1ff28c9640 1 -- 172.21.15.116:0/2702891528 wait complete. 2024-09-12T03:11:34.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.673+0000 7f1ff28c9640 1 Processor -- start 2024-09-12T03:11:34.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.673+0000 7f1ff28c9640 1 -- start start 2024-09-12T03:11:34.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.673+0000 7f1ff28c9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fec102dc0 0x7f1fec1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:34.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.673+0000 7f1ff28c9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fec1036e0 0x7f1fec198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:34.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.673+0000 7f1ff28c9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fec10c1a0 0x7f1fec19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:34.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.673+0000 7f1ff28c9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec111fb0 con 0x7f1fec10c1a0 2024-09-12T03:11:34.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.673+0000 7f1ff28c9640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec111e30 con 0x7f1fec1036e0 2024-09-12T03:11:34.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.673+0000 7f1ff28c9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec112130 con 0x7f1fec102dc0 2024-09-12T03:11:34.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.673+0000 7f1ff0e66640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fec10c1a0 0x7f1fec19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:34.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.673+0000 7f1ff0e66640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fec10c1a0 0x7f1fec19ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48504/0 (socket says 172.21.15.116:48504) 2024-09-12T03:11:34.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.673+0000 7f1ff0e66640 1 -- 172.21.15.116:0/3828921509 learned_addr learned my addr 172.21.15.116:0/3828921509 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:34.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.673+0000 7f1febfff640 1 --2- 172.21.15.116:0/3828921509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fec102dc0 0x7f1fec1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:34.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.674+0000 7f1feb7fe640 1 --2- 172.21.15.116:0/3828921509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fec1036e0 0x7f1fec198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:34.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.674+0000 7f1ff0e66640 1 -- 172.21.15.116:0/3828921509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fec102dc0 msgr2=0x7f1fec1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:34.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.674+0000 7f1ff0e66640 1 --2- 172.21.15.116:0/3828921509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fec102dc0 0x7f1fec1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:34.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.674+0000 7f1ff0e66640 1 -- 172.21.15.116:0/3828921509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fec1036e0 msgr2=0x7f1fec198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:34.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.674+0000 7f1ff0e66640 1 --2- 172.21.15.116:0/3828921509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fec1036e0 0x7f1fec198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:34.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.674+0000 7f1ff0e66640 1 -- 172.21.15.116:0/3828921509 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fec19d3a0 con 0x7f1fec10c1a0 2024-09-12T03:11:34.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.674+0000 7f1feb7fe640 1 --2- 172.21.15.116:0/3828921509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fec1036e0 0x7f1fec198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:34.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.674+0000 7f1ff0e66640 1 --2- 172.21.15.116:0/3828921509 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fec10c1a0 0x7f1fec19ccc0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400b970 tx=0x7f1fe400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:34.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.674+0000 7f1fe97fa640 1 -- 172.21.15.116:0/3828921509 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4019070 con 0x7f1fec10c1a0 2024-09-12T03:11:34.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.674+0000 7f1ff28c9640 1 -- 172.21.15.116:0/3828921509 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fec19d690 con 0x7f1fec10c1a0 2024-09-12T03:11:34.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.674+0000 7f1fe97fa640 1 -- 172.21.15.116:0/3828921509 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fe400cb40 con 0x7f1fec10c1a0 2024-09-12T03:11:34.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.674+0000 7f1fe97fa640 1 -- 172.21.15.116:0/3828921509 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4007650 con 0x7f1fec10c1a0 2024-09-12T03:11:34.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.674+0000 7f1ff28c9640 1 -- 172.21.15.116:0/3828921509 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fec1a4f70 con 0x7f1fec10c1a0 2024-09-12T03:11:34.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.675+0000 7f1ff28c9640 1 -- 172.21.15.116:0/3828921509 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fec1031a0 con 0x7f1fec10c1a0 2024-09-12T03:11:34.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.676+0000 7f1fe97fa640 1 -- 172.21.15.116:0/3828921509 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1fe401c070 con 0x7f1fec10c1a0 2024-09-12T03:11:34.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.677+0000 7f1fe97fa640 1 --2- 172.21.15.116:0/3828921509 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1fd80a2b80 0x7f1fd80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:34.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.677+0000 7f1febfff640 1 --2- 172.21.15.116:0/3828921509 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1fd80a2b80 0x7f1fd80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:34.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.677+0000 7f1fe97fa640 1 -- 172.21.15.116:0/3828921509 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1fe40ce600 con 0x7f1fec10c1a0 2024-09-12T03:11:34.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.677+0000 7f1febfff640 1 --2- 172.21.15.116:0/3828921509 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1fd80a2b80 0x7f1fd80a5040 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc000c00 tx=0x7f1fdc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:34.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.681+0000 7f1fe97fa640 1 -- 172.21.15.116:0/3828921509 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1fe409af40 con 0x7f1fec10c1a0 2024-09-12T03:11:34.707 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:34 smithi183 ceph-mon[35535]: pgmap v595: 97 pgs: 97 active+clean; 4.1 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-12T03:11:34.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:34 smithi177 ceph-mon[35305]: pgmap v595: 97 pgs: 97 active+clean; 4.1 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-12T03:11:34.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.881+0000 7f1ff28c9640 1 -- 172.21.15.116:0/3828921509 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1fec106c90 con 0x7f1fec10c1a0 2024-09-12T03:11:34.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.881+0000 7f1fe97fa640 1 -- 172.21.15.116:0/3828921509 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f1fe409f240 con 0x7f1fec10c1a0 2024-09-12T03:11:34.882 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:11:34.883 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:11:34.883 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:11:34.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.884+0000 7f1ff28c9640 1 -- 172.21.15.116:0/3828921509 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1fd80a2b80 msgr2=0x7f1fd80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:34.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.884+0000 7f1ff28c9640 1 --2- 172.21.15.116:0/3828921509 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1fd80a2b80 0x7f1fd80a5040 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc000c00 tx=0x7f1fdc0058d0 comp rx=0 tx=0).stop 2024-09-12T03:11:34.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.884+0000 7f1ff28c9640 1 -- 172.21.15.116:0/3828921509 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fec10c1a0 msgr2=0x7f1fec19ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:34.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.884+0000 7f1ff28c9640 1 --2- 172.21.15.116:0/3828921509 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fec10c1a0 0x7f1fec19ccc0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400b970 tx=0x7f1fe400be30 comp rx=0 tx=0).stop 2024-09-12T03:11:34.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.884+0000 7f1ff28c9640 1 -- 172.21.15.116:0/3828921509 shutdown_connections 2024-09-12T03:11:34.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.884+0000 7f1ff28c9640 1 --2- 172.21.15.116:0/3828921509 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1fd80a2b80 0x7f1fd80a5040 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:34.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.884+0000 7f1ff28c9640 1 --2- 172.21.15.116:0/3828921509 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fec10c1a0 0x7f1fec19ccc0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:34.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.884+0000 7f1ff28c9640 1 --2- 172.21.15.116:0/3828921509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1fec1036e0 0x7f1fec198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:34.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.884+0000 7f1ff28c9640 1 --2- 172.21.15.116:0/3828921509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fec102dc0 0x7f1fec1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:34.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.884+0000 7f1ff28c9640 1 -- 172.21.15.116:0/3828921509 >> 172.21.15.116:0/3828921509 conn(0x7f1fec0fc7a0 msgr2=0x7f1fec0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:34.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.884+0000 7f1ff28c9640 1 -- 172.21.15.116:0/3828921509 shutdown_connections 2024-09-12T03:11:34.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:34.884+0000 7f1ff28c9640 1 -- 172.21.15.116:0/3828921509 wait complete. 2024-09-12T03:11:35.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:35 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3828921509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:11:35.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:35 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3828921509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:11:35.710 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:11:35.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:35 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3828921509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:11:35.957 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:11:36.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.440+0000 7fc924890640 1 -- 172.21.15.116:0/3275993363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc920101de0 msgr2=0x7fc9201021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:36.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.440+0000 7fc924890640 1 --2- 172.21.15.116:0/3275993363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc920101de0 0x7fc9201021c0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fc910009900 tx=0x7fc91002a410 comp rx=0 tx=0).stop 2024-09-12T03:11:36.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.440+0000 7fc924890640 1 -- 172.21.15.116:0/3275993363 shutdown_connections 2024-09-12T03:11:36.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.440+0000 7fc924890640 1 --2- 172.21.15.116:0/3275993363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc920106f10 0x7fc92010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.440+0000 7fc924890640 1 --2- 172.21.15.116:0/3275993363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc920102790 0x7fc9201067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.440+0000 7fc924890640 1 --2- 172.21.15.116:0/3275993363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc920101de0 0x7fc9201021c0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.440+0000 7fc924890640 1 -- 172.21.15.116:0/3275993363 >> 172.21.15.116:0/3275993363 conn(0x7fc9200fd8c0 msgr2=0x7fc9200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:36.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.440+0000 7fc924890640 1 -- 172.21.15.116:0/3275993363 shutdown_connections 2024-09-12T03:11:36.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.440+0000 7fc924890640 1 -- 172.21.15.116:0/3275993363 wait complete. 2024-09-12T03:11:36.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.441+0000 7fc924890640 1 Processor -- start 2024-09-12T03:11:36.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.441+0000 7fc924890640 1 -- start start 2024-09-12T03:11:36.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.441+0000 7fc924890640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc920101de0 0x7fc9201985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:36.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.441+0000 7fc924890640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc920102790 0x7fc920198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:36.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.441+0000 7fc924890640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc920106f10 0x7fc92019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:36.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.441+0000 7fc924890640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc920112010 con 0x7fc920102790 2024-09-12T03:11:36.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.441+0000 7fc924890640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc920111e90 con 0x7fc920106f10 2024-09-12T03:11:36.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.441+0000 7fc924890640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc920112190 con 0x7fc920101de0 2024-09-12T03:11:36.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.441+0000 7fc91dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc920102790 0x7fc920198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:36.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.441+0000 7fc91dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc920102790 0x7fc920198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48516/0 (socket says 172.21.15.116:48516) 2024-09-12T03:11:36.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.441+0000 7fc91dd9b640 1 -- 172.21.15.116:0/626183639 learned_addr learned my addr 172.21.15.116:0/626183639 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:36.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.441+0000 7fc91e59c640 1 --2- 172.21.15.116:0/626183639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc920101de0 0x7fc9201985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:36.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.441+0000 7fc91ed9d640 1 --2- 172.21.15.116:0/626183639 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc920106f10 0x7fc92019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:36.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.442+0000 7fc91dd9b640 1 -- 172.21.15.116:0/626183639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc920101de0 msgr2=0x7fc9201985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:36.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.442+0000 7fc91dd9b640 1 --2- 172.21.15.116:0/626183639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc920101de0 0x7fc9201985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.442+0000 7fc91dd9b640 1 -- 172.21.15.116:0/626183639 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc920106f10 msgr2=0x7fc92019ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:36.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.442+0000 7fc91dd9b640 1 --2- 172.21.15.116:0/626183639 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc920106f10 0x7fc92019ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.442+0000 7fc91dd9b640 1 -- 172.21.15.116:0/626183639 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc92019d560 con 0x7fc920102790 2024-09-12T03:11:36.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.442+0000 7fc91ed9d640 1 --2- 172.21.15.116:0/626183639 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc920106f10 0x7fc92019ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:11:36.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.442+0000 7fc91dd9b640 1 --2- 172.21.15.116:0/626183639 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc920102790 0x7fc920198af0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fc91400b4a0 tx=0x7fc91400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:36.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.442+0000 7fc90f7fe640 1 -- 172.21.15.116:0/626183639 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc914014070 con 0x7fc920102790 2024-09-12T03:11:36.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.442+0000 7fc90f7fe640 1 -- 172.21.15.116:0/626183639 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc914010af0 con 0x7fc920102790 2024-09-12T03:11:36.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.442+0000 7fc924890640 1 -- 172.21.15.116:0/626183639 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc92019d850 con 0x7fc920102790 2024-09-12T03:11:36.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.442+0000 7fc90f7fe640 1 -- 172.21.15.116:0/626183639 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc914017d10 con 0x7fc920102790 2024-09-12T03:11:36.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.442+0000 7fc924890640 1 -- 172.21.15.116:0/626183639 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9201a5130 con 0x7fc920102790 2024-09-12T03:11:36.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.444+0000 7fc90f7fe640 1 -- 172.21.15.116:0/626183639 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fc914017070 con 0x7fc920102790 2024-09-12T03:11:36.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.444+0000 7fc926c69740 1 Processor -- start 2024-09-12T03:11:36.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.445+0000 7fc926c69740 1 -- start start 2024-09-12T03:11:36.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.445+0000 7fc926c69740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564c00225b40 0x564c00224090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:36.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.445+0000 7fc926c69740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c002245d0 0x564c00224a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:36.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.445+0000 7fc926c69740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564c0022f370 0x564c00231760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:36.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.445+0000 7fc926c69740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x564c0018a2f0 con 0x564c0022f370 2024-09-12T03:11:36.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.445+0000 7fc926c69740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x564c00131cf0 con 0x564c00225b40 2024-09-12T03:11:36.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.445+0000 7fc926c69740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564c001862b0 con 0x564c002245d0 2024-09-12T03:11:36.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.445+0000 7fc90f7fe640 1 --2- 172.21.15.116:0/626183639 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc9080a2b80 0x7fc9080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:36.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.445+0000 7fc91dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564c0022f370 0x564c00231760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:36.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.445+0000 7fc91ed9d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564c00225b40 0x564c00224090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:36.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.445+0000 7fc91ed9d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564c00225b40 0x564c00224090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.116:59512/0 (socket says 172.21.15.116:59512) 2024-09-12T03:11:36.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.445+0000 7fc91ed9d640 1 -- 172.21.15.116:0/1915865097 learned_addr learned my addr 172.21.15.116:0/1915865097 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:36.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.445+0000 7fc91e59c640 1 --2- 172.21.15.116:0/1915865097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c002245d0 0x564c00224a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:36.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.445+0000 7fc91ed9d640 1 --2- 172.21.15.116:0/626183639 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc9080a2b80 0x7fc9080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:36.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.445+0000 7fc90f7fe640 1 -- 172.21.15.116:0/626183639 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fc9140ce3e0 con 0x7fc920102790 2024-09-12T03:11:36.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.445+0000 7fc91dd9b640 1 -- 172.21.15.116:0/1915865097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c002245d0 msgr2=0x564c00224a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:36.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.445+0000 7fc91dd9b640 1 --2- 172.21.15.116:0/1915865097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c002245d0 0x564c00224a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.445+0000 7fc91dd9b640 1 -- 172.21.15.116:0/1915865097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564c00225b40 msgr2=0x564c00224090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:36.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.445+0000 7fc91dd9b640 1 --2- 172.21.15.116:0/1915865097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564c00225b40 0x564c00224090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.445+0000 7fc91dd9b640 1 -- 172.21.15.116:0/1915865097 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c001163a0 con 0x564c0022f370 2024-09-12T03:11:36.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.446+0000 7fc91dd9b640 1 --2- 172.21.15.116:0/1915865097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564c0022f370 0x564c00231760 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fc91409a970 tx=0x7fc91406dd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9209e24945d1eb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:36.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.446+0000 7fc91ed9d640 1 --2- 172.21.15.116:0/626183639 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc9080a2b80 0x7fc9080a5040 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc91800ad40 tx=0x7fc918009ad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:36.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.446+0000 7fc90cff9640 1 -- 172.21.15.116:0/1915865097 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91401e070 con 0x564c0022f370 2024-09-12T03:11:36.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.446+0000 7fc90cff9640 1 -- 172.21.15.116:0/1915865097 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc91409b370 con 0x564c0022f370 2024-09-12T03:11:36.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.446+0000 7fc90cff9640 1 -- 172.21.15.116:0/1915865097 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc914099040 con 0x564c0022f370 2024-09-12T03:11:36.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.447+0000 7fc926c69740 1 -- 172.21.15.116:0/1915865097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564c0022f370 msgr2=0x564c00231760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:36.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.447+0000 7fc926c69740 1 --2- 172.21.15.116:0/1915865097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564c0022f370 0x564c00231760 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fc91409a970 tx=0x7fc91406dd30 comp rx=0 tx=0).stop 2024-09-12T03:11:36.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.447+0000 7fc926c69740 1 -- 172.21.15.116:0/1915865097 shutdown_connections 2024-09-12T03:11:36.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.447+0000 7fc926c69740 1 --2- 172.21.15.116:0/1915865097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564c0022f370 0x564c00231760 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.447+0000 7fc926c69740 1 --2- 172.21.15.116:0/1915865097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c002245d0 0x564c00224a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.447+0000 7fc926c69740 1 --2- 172.21.15.116:0/1915865097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564c00225b40 0x564c00224090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.447+0000 7fc926c69740 1 -- 172.21.15.116:0/1915865097 >> 172.21.15.116:0/1915865097 conn(0x564c001ed310 msgr2=0x564c00220c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:36.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.447+0000 7fc926c69740 1 -- 172.21.15.116:0/1915865097 shutdown_connections 2024-09-12T03:11:36.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.447+0000 7fc926c69740 1 -- 172.21.15.116:0/1915865097 wait complete. 2024-09-12T03:11:36.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.447+0000 7fc926c69740 1 -- start start 2024-09-12T03:11:36.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.447+0000 7fc926c69740 1 Processor -- start 2024-09-12T03:11:36.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.447+0000 7fc926c69740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c002245d0 0x564c0022edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:36.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.447+0000 7fc926c69740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564c00225b40 0x564c0022c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:36.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.447+0000 7fc926c69740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564c0022f370 0x564c0022c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:36.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.447+0000 7fc926c69740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x564c00232010 con 0x564c00225b40 2024-09-12T03:11:36.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.447+0000 7fc926c69740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x564c00145060 con 0x564c0022f370 2024-09-12T03:11:36.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.447+0000 7fc926c69740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564c00232190 con 0x564c002245d0 2024-09-12T03:11:36.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.447+0000 7fc926c69740 10 client.0 _ll_register_callbacks cb 0x564bffe988c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:36.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.448+0000 7fc91dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564c00225b40 0x564c0022c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:36.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.448+0000 7fc91dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564c00225b40 0x564c0022c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48544/0 (socket says 172.21.15.116:48544) 2024-09-12T03:11:36.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.448+0000 7fc91dd9b640 1 -- 172.21.15.116:0/3364728395 learned_addr learned my addr 172.21.15.116:0/3364728395 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:36.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.448+0000 7fc91ed9d640 1 --2- 172.21.15.116:0/3364728395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564c0022f370 0x564c0022c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:36.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.448+0000 7fc91e59c640 1 --2- 172.21.15.116:0/3364728395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c002245d0 0x564c0022edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:36.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.448+0000 7fc91dd9b640 1 -- 172.21.15.116:0/3364728395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c002245d0 msgr2=0x564c0022edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:36.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.448+0000 7fc91dd9b640 1 --2- 172.21.15.116:0/3364728395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c002245d0 0x564c0022edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.448+0000 7fc91dd9b640 1 -- 172.21.15.116:0/3364728395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564c0022f370 msgr2=0x564c0022c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:36.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.448+0000 7fc91dd9b640 1 --2- 172.21.15.116:0/3364728395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564c0022f370 0x564c0022c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.448+0000 7fc91dd9b640 1 -- 172.21.15.116:0/3364728395 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bffefb3c0 con 0x564c00225b40 2024-09-12T03:11:36.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.448+0000 7fc91e59c640 1 --2- 172.21.15.116:0/3364728395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c002245d0 0x564c0022edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:11:36.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.448+0000 7fc91dd9b640 1 --2- 172.21.15.116:0/3364728395 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564c00225b40 0x564c0022c080 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fc9140970d0 tx=0x7fc914017730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70253dd4f96999d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:36.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.448+0000 7fc905ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:36.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.448+0000 7fc905ffb640 1 -- 172.21.15.116:0/3364728395 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91401e050 con 0x564c00225b40 2024-09-12T03:11:36.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.448+0000 7fc926c69740 1 -- 172.21.15.116:0/3364728395 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc91409b370 con 0x564c00225b40 2024-09-12T03:11:36.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.448+0000 7fc905ffb640 1 -- 172.21.15.116:0/3364728395 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fc914010ca0 con 0x564c00225b40 2024-09-12T03:11:36.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.448+0000 7fc905ffb640 1 -- 172.21.15.116:0/3364728395 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc9140a38e0 con 0x564c00225b40 2024-09-12T03:11:36.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.448+0000 7fc926c69740 10 client.17532 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:36.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.448+0000 7fc926c69740 1 -- 172.21.15.116:0/3364728395 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564c0023db30 con 0x564c00225b40 2024-09-12T03:11:36.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.448+0000 7fc905ffb640 1 -- 172.21.15.116:0/3364728395 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc91409b370 con 0x564c00225b40 2024-09-12T03:11:36.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.449+0000 7fc905ffb640 1 -- 172.21.15.116:0/3364728395 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc9140a34e0 con 0x564c00225b40 2024-09-12T03:11:36.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.449+0000 7fc926c69740 10 client.17532 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:36.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.449+0000 7fc926c69740 10 client.17532 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:11:36.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.449+0000 7fc926c69740 20 client.17532 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:36.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.449+0000 7fc926c69740 1 --2- 172.21.15.116:0/3364728395 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x564c0023ee80 0x564c00241270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:36.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.449+0000 7fc926c69740 4 client.17532 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:11:36.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.449+0000 7fc926c69740 1 -- 172.21.15.116:0/3364728395 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c00197860 con 0x564c0023ee80 2024-09-12T03:11:36.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.449+0000 7fc91e59c640 1 --2- 172.21.15.116:0/3364728395 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x564c0023ee80 0x564c00241270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:36.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.449+0000 7fc905ffb640 10 client.17532 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:11:36.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.449+0000 7fc91e59c640 1 --2- 172.21.15.116:0/3364728395 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x564c0023ee80 0x564c00241270 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eab3ab57c8a110ac server_cookie=3b9e4363383c25b7 in_seq=0 out_seq=0 2024-09-12T03:11:36.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.450+0000 7fc905ffb640 1 -- 172.21.15.116:0/3364728395 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564c00197860 con 0x564c0023ee80 2024-09-12T03:11:36.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.450+0000 7fc905ffb640 10 client.17532 handle_command_reply: tid=0 2024-09-12T03:11:36.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.450+0000 7fc926c69740 1 client.17532 shutdown 2024-09-12T03:11:36.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.450+0000 7fc926c69740 1 -- 172.21.15.116:0/3364728395 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564c00225b40 msgr2=0x564c0022c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:36.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.450+0000 7fc926c69740 1 --2- 172.21.15.116:0/3364728395 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564c00225b40 0x564c0022c080 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fc9140970d0 tx=0x7fc914017730 comp rx=0 tx=0).stop 2024-09-12T03:11:36.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.450+0000 7fc926c69740 1 -- 172.21.15.116:0/3364728395 shutdown_connections 2024-09-12T03:11:36.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.450+0000 7fc926c69740 1 --2- 172.21.15.116:0/3364728395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564c0022f370 0x564c0022c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.450+0000 7fc926c69740 1 --2- 172.21.15.116:0/3364728395 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x564c0023ee80 0x564c00241270 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.450+0000 7fc926c69740 1 --2- 172.21.15.116:0/3364728395 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564c00225b40 0x564c0022c080 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.450+0000 7fc905ffb640 0 client.17532 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:11:36.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.450+0000 7fc926c69740 1 --2- 172.21.15.116:0/3364728395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c002245d0 0x564c0022edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.450+0000 7fc926c69740 1 -- 172.21.15.116:0/3364728395 >> 172.21.15.116:0/3364728395 conn(0x564c001ed310 msgr2=0x564c00226990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:36.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.450+0000 7fc926c69740 1 -- 172.21.15.116:0/3364728395 shutdown_connections 2024-09-12T03:11:36.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.450+0000 7fc926c69740 1 -- 172.21.15.116:0/3364728395 wait complete. 2024-09-12T03:11:36.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.450+0000 7fc926c69740 20 client.17532 trim_cache size 0 max 16384 2024-09-12T03:11:36.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.463+0000 7fc926c69740 1 Processor -- start 2024-09-12T03:11:36.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.464+0000 7fc926c69740 1 -- start start 2024-09-12T03:11:36.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.464+0000 7fc926c69740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564c002245d0 0x564c00231760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:36.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.464+0000 7fc926c69740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c00225b40 0x564c00227120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:36.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.464+0000 7fc926c69740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564c0022f370 0x564c00227660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:36.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.464+0000 7fc926c69740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc9201adc90 con 0x564c0022f370 2024-09-12T03:11:36.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.464+0000 7fc926c69740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc9201ab560 con 0x564c002245d0 2024-09-12T03:11:36.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.464+0000 7fc926c69740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc9201a93d0 con 0x564c00225b40 2024-09-12T03:11:36.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.464+0000 7fc91ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564c0022f370 0x564c00227660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:36.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.464+0000 7fc91ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564c0022f370 0x564c00227660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48550/0 (socket says 172.21.15.116:48550) 2024-09-12T03:11:36.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.464+0000 7fc91ed9d640 1 -- 172.21.15.116:0/1007427745 learned_addr learned my addr 172.21.15.116:0/1007427745 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:36.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.464+0000 7fc91e59c640 1 --2- 172.21.15.116:0/1007427745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564c002245d0 0x564c00231760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:36.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.464+0000 7fc91dd9b640 1 --2- 172.21.15.116:0/1007427745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c00225b40 0x564c00227120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:36.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.464+0000 7fc91ed9d640 1 -- 172.21.15.116:0/1007427745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c00225b40 msgr2=0x564c00227120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:36.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.464+0000 7fc91ed9d640 1 --2- 172.21.15.116:0/1007427745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c00225b40 0x564c00227120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.464+0000 7fc91ed9d640 1 -- 172.21.15.116:0/1007427745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564c002245d0 msgr2=0x564c00231760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:36.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.464+0000 7fc91ed9d640 1 --2- 172.21.15.116:0/1007427745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564c002245d0 0x564c00231760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.464+0000 7fc91ed9d640 1 -- 172.21.15.116:0/1007427745 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c0022a5d0 con 0x564c0022f370 2024-09-12T03:11:36.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.464+0000 7fc91e59c640 1 --2- 172.21.15.116:0/1007427745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564c002245d0 0x564c00231760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:11:36.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.464+0000 7fc91ed9d640 1 --2- 172.21.15.116:0/1007427745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564c0022f370 0x564c00227660 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fc918009520 tx=0x7fc91800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d67874eda4544 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:36.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.465+0000 7fc907fff640 1 -- 172.21.15.116:0/1007427745 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc918013040 con 0x564c0022f370 2024-09-12T03:11:36.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.465+0000 7fc907fff640 1 -- 172.21.15.116:0/1007427745 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc91802a070 con 0x564c0022f370 2024-09-12T03:11:36.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.465+0000 7fc907fff640 1 -- 172.21.15.116:0/1007427745 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc918025e60 con 0x564c0022f370 2024-09-12T03:11:36.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.465+0000 7fc926c69740 1 -- 172.21.15.116:0/1007427745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564c0022f370 msgr2=0x564c00227660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:36.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.465+0000 7fc926c69740 1 --2- 172.21.15.116:0/1007427745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564c0022f370 0x564c00227660 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fc918009520 tx=0x7fc91800f000 comp rx=0 tx=0).stop 2024-09-12T03:11:36.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.465+0000 7fc926c69740 1 -- 172.21.15.116:0/1007427745 shutdown_connections 2024-09-12T03:11:36.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.465+0000 7fc926c69740 1 --2- 172.21.15.116:0/1007427745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564c0022f370 0x564c00227660 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.465+0000 7fc926c69740 1 --2- 172.21.15.116:0/1007427745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c00225b40 0x564c00227120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.465+0000 7fc926c69740 1 --2- 172.21.15.116:0/1007427745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564c002245d0 0x564c00231760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.465+0000 7fc926c69740 1 -- 172.21.15.116:0/1007427745 >> 172.21.15.116:0/1007427745 conn(0x564c001ed310 msgr2=0x564c0021ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:36.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.465+0000 7fc926c69740 1 -- 172.21.15.116:0/1007427745 shutdown_connections 2024-09-12T03:11:36.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.465+0000 7fc926c69740 1 -- 172.21.15.116:0/1007427745 wait complete. 2024-09-12T03:11:36.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.466+0000 7fc926c69740 1 -- start start 2024-09-12T03:11:36.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.466+0000 7fc926c69740 1 Processor -- start 2024-09-12T03:11:36.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.466+0000 7fc926c69740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c002245d0 0x564c00244760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:36.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.466+0000 7fc926c69740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564c00225b40 0x564c00241ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:36.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.466+0000 7fc926c69740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564c0022f370 0x564c00241ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:36.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.466+0000 7fc926c69740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc9201a9a20 con 0x564c0022f370 2024-09-12T03:11:36.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.466+0000 7fc926c69740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc9201aae90 con 0x564c00225b40 2024-09-12T03:11:36.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.466+0000 7fc926c69740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc9201aa0f0 con 0x564c002245d0 2024-09-12T03:11:36.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.466+0000 7fc926c69740 10 client.0 _ll_register_callbacks cb 0x7fc91800a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:36.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.466+0000 7fc91ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564c0022f370 0x564c00241ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:36.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.466+0000 7fc91ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564c0022f370 0x564c00241ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48560/0 (socket says 172.21.15.116:48560) 2024-09-12T03:11:36.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.466+0000 7fc91ed9d640 1 -- 172.21.15.116:0/580821887 learned_addr learned my addr 172.21.15.116:0/580821887 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:36.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.466+0000 7fc91e59c640 1 --2- 172.21.15.116:0/580821887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c002245d0 0x564c00244760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:36.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.466+0000 7fc91ed9d640 1 -- 172.21.15.116:0/580821887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c002245d0 msgr2=0x564c00244760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:36.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.466+0000 7fc91ed9d640 1 --2- 172.21.15.116:0/580821887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c002245d0 0x564c00244760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.466+0000 7fc91ed9d640 1 -- 172.21.15.116:0/580821887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564c00225b40 msgr2=0x564c00241ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:36.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.466+0000 7fc91ed9d640 1 --2- 172.21.15.116:0/580821887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564c00225b40 0x564c00241ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.466+0000 7fc91ed9d640 1 -- 172.21.15.116:0/580821887 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c00242560 con 0x564c0022f370 2024-09-12T03:11:36.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.467+0000 7fc91ed9d640 1 --2- 172.21.15.116:0/580821887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564c0022f370 0x564c00241ff0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fc918025ac0 tx=0x7fc918019d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02648c2e25047c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:36.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.467+0000 7fc905ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:36.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.467+0000 7fc905ffb640 1 -- 172.21.15.116:0/580821887 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc918034020 con 0x564c0022f370 2024-09-12T03:11:36.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.467+0000 7fc905ffb640 1 -- 172.21.15.116:0/580821887 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fc918025460 con 0x564c0022f370 2024-09-12T03:11:36.477 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:36 smithi116 ceph-mon[31410]: pgmap v596: 97 pgs: 97 active+clean; 4.1 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-12T03:11:36.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.467+0000 7fc926c69740 1 -- 172.21.15.116:0/580821887 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc91802a070 con 0x564c0022f370 2024-09-12T03:11:36.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.468+0000 7fc905ffb640 1 -- 172.21.15.116:0/580821887 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc918025660 con 0x564c0022f370 2024-09-12T03:11:36.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.468+0000 7fc926c69740 10 client.17538 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:36.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.468+0000 7fc926c69740 1 -- 172.21.15.116:0/580821887 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564c002a9520 con 0x564c0022f370 2024-09-12T03:11:36.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.468+0000 7fc905ffb640 1 -- 172.21.15.116:0/580821887 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc91801ae50 con 0x564c0022f370 2024-09-12T03:11:36.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.468+0000 7fc905ffb640 1 -- 172.21.15.116:0/580821887 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc91802e410 con 0x564c0022f370 2024-09-12T03:11:36.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.468+0000 7fc926c69740 10 client.17538 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:36.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.468+0000 7fc926c69740 10 client.17538 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:11:36.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.468+0000 7fc926c69740 20 client.17538 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:36.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.468+0000 7fc926c69740 1 --2- 172.21.15.116:0/580821887 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x564c0023ee80 0x564c002abc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:36.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.468+0000 7fc926c69740 4 client.17538 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:11:36.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.468+0000 7fc926c69740 1 -- 172.21.15.116:0/580821887 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fc91802a070 con 0x564c0023ee80 2024-09-12T03:11:36.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.468+0000 7fc91e59c640 1 --2- 172.21.15.116:0/580821887 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x564c0023ee80 0x564c002abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:36.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.469+0000 7fc905ffb640 10 client.17538 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:11:36.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.469+0000 7fc91e59c640 1 --2- 172.21.15.116:0/580821887 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x564c0023ee80 0x564c002abc20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7a414826becc95ec server_cookie=a852657d51c306fa in_seq=0 out_seq=0 2024-09-12T03:11:36.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.469+0000 7fc905ffb640 1 -- 172.21.15.116:0/580821887 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fc91802a070 con 0x564c0023ee80 2024-09-12T03:11:36.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.469+0000 7fc905ffb640 10 client.17538 handle_command_reply: tid=0 2024-09-12T03:11:36.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.469+0000 7fc926c69740 1 client.17538 shutdown 2024-09-12T03:11:36.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.469+0000 7fc926c69740 1 -- 172.21.15.116:0/580821887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564c0022f370 msgr2=0x564c00241ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:36.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.469+0000 7fc926c69740 1 --2- 172.21.15.116:0/580821887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564c0022f370 0x564c00241ff0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fc918025ac0 tx=0x7fc918019d30 comp rx=0 tx=0).stop 2024-09-12T03:11:36.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.469+0000 7fc926c69740 1 -- 172.21.15.116:0/580821887 shutdown_connections 2024-09-12T03:11:36.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.470+0000 7fc926c69740 1 --2- 172.21.15.116:0/580821887 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x564c0023ee80 0x564c002abc20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.470+0000 7fc905ffb640 0 client.17538 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:11:36.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.470+0000 7fc926c69740 1 --2- 172.21.15.116:0/580821887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564c0022f370 0x564c00241ff0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.470+0000 7fc926c69740 1 --2- 172.21.15.116:0/580821887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564c00225b40 0x564c00241ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.470+0000 7fc926c69740 1 --2- 172.21.15.116:0/580821887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c002245d0 0x564c00244760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.470+0000 7fc926c69740 1 -- 172.21.15.116:0/580821887 >> 172.21.15.116:0/580821887 conn(0x564c001ed310 msgr2=0x564c0023a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:36.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.470+0000 7fc926c69740 1 -- 172.21.15.116:0/580821887 shutdown_connections 2024-09-12T03:11:36.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.470+0000 7fc926c69740 1 -- 172.21.15.116:0/580821887 wait complete. 2024-09-12T03:11:36.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.470+0000 7fc926c69740 20 client.17538 trim_cache size 0 max 16384 2024-09-12T03:11:36.482 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:11:36.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.470+0000 7fc924890640 1 -- 172.21.15.116:0/626183639 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc9080a2b80 msgr2=0x7fc9080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:36.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.470+0000 7fc924890640 1 --2- 172.21.15.116:0/626183639 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc9080a2b80 0x7fc9080a5040 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc91800ad40 tx=0x7fc918009ad0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.471+0000 7fc924890640 1 -- 172.21.15.116:0/626183639 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc920102790 msgr2=0x7fc920198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:36.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.471+0000 7fc924890640 1 --2- 172.21.15.116:0/626183639 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc920102790 0x7fc920198af0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fc91400b4a0 tx=0x7fc91400b960 comp rx=0 tx=0).stop 2024-09-12T03:11:36.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.471+0000 7fc924890640 1 -- 172.21.15.116:0/626183639 shutdown_connections 2024-09-12T03:11:36.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.471+0000 7fc924890640 1 --2- 172.21.15.116:0/626183639 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc9080a2b80 0x7fc9080a5040 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.471+0000 7fc924890640 1 --2- 172.21.15.116:0/626183639 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc920106f10 0x7fc92019ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.471+0000 7fc924890640 1 --2- 172.21.15.116:0/626183639 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc920102790 0x7fc920198af0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.471+0000 7fc924890640 1 --2- 172.21.15.116:0/626183639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc920101de0 0x7fc9201985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:36.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.471+0000 7fc924890640 1 -- 172.21.15.116:0/626183639 >> 172.21.15.116:0/626183639 conn(0x7fc9200fd8c0 msgr2=0x7fc9200fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:36.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.471+0000 7fc924890640 1 -- 172.21.15.116:0/626183639 shutdown_connections 2024-09-12T03:11:36.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:36.471+0000 7fc924890640 1 -- 172.21.15.116:0/626183639 wait complete. 2024-09-12T03:11:36.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:36 smithi183 ceph-mon[35535]: pgmap v596: 97 pgs: 97 active+clean; 4.1 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-12T03:11:36.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:36 smithi177 ceph-mon[35305]: pgmap v596: 97 pgs: 97 active+clean; 4.1 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-12T03:11:37.203 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:11:37.450 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:11:37.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:37 smithi116 ceph-mon[31410]: pgmap v597: 97 pgs: 97 active+clean; 4.1 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.9 KiB/s wr, 9 op/s 2024-09-12T03:11:37.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:37 smithi183 ceph-mon[35535]: pgmap v597: 97 pgs: 97 active+clean; 4.1 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.9 KiB/s wr, 9 op/s 2024-09-12T03:11:37.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:37 smithi177 ceph-mon[35305]: pgmap v597: 97 pgs: 97 active+clean; 4.1 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.9 KiB/s wr, 9 op/s 2024-09-12T03:11:37.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.949+0000 7f2f0037e640 1 -- 172.21.15.116:0/2892829517 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ef8102d40 msgr2=0x7f2ef8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:37.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.949+0000 7f2f0037e640 1 --2- 172.21.15.116:0/2892829517 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ef8102d40 0x7f2ef8103120 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8009900 tx=0x7f2ee802a480 comp rx=0 tx=0).stop 2024-09-12T03:11:37.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.949+0000 7f2f0037e640 1 -- 172.21.15.116:0/2892829517 shutdown_connections 2024-09-12T03:11:37.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.949+0000 7f2f0037e640 1 --2- 172.21.15.116:0/2892829517 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ef810c170 0x7f2ef810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.949+0000 7f2f0037e640 1 --2- 172.21.15.116:0/2892829517 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ef8103660 0x7f2ef810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.949+0000 7f2f0037e640 1 --2- 172.21.15.116:0/2892829517 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ef8102d40 0x7f2ef8103120 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.949+0000 7f2f0037e640 1 -- 172.21.15.116:0/2892829517 >> 172.21.15.116:0/2892829517 conn(0x7f2ef80fc7a0 msgr2=0x7f2ef80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:37.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.949+0000 7f2f0037e640 1 -- 172.21.15.116:0/2892829517 shutdown_connections 2024-09-12T03:11:37.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.949+0000 7f2f0037e640 1 -- 172.21.15.116:0/2892829517 wait complete. 2024-09-12T03:11:37.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.950+0000 7f2f0037e640 1 Processor -- start 2024-09-12T03:11:37.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.950+0000 7f2f0037e640 1 -- start start 2024-09-12T03:11:37.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.950+0000 7f2f0037e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ef8102d40 0x7f2ef81009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:37.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.950+0000 7f2f0037e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ef8103660 0x7f2ef8100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:37.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.950+0000 7f2f0037e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ef810c170 0x7f2ef8101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:37.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.950+0000 7f2f0037e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef8111fb0 con 0x7f2ef810c170 2024-09-12T03:11:37.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.950+0000 7f2f0037e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef8111e30 con 0x7f2ef8102d40 2024-09-12T03:11:37.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.950+0000 7f2f0037e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef8112130 con 0x7f2ef8103660 2024-09-12T03:11:37.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.950+0000 7f2efe91b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ef810c170 0x7f2ef8101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:37.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.950+0000 7f2efe91b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ef810c170 0x7f2ef8101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48586/0 (socket says 172.21.15.116:48586) 2024-09-12T03:11:37.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.950+0000 7f2efe91b640 1 -- 172.21.15.116:0/4158449202 learned_addr learned my addr 172.21.15.116:0/4158449202 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:37.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.950+0000 7f2efd919640 1 --2- 172.21.15.116:0/4158449202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ef8103660 0x7f2ef8100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:37.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.950+0000 7f2efe11a640 1 --2- 172.21.15.116:0/4158449202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ef8102d40 0x7f2ef81009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:37.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.951+0000 7f2efe91b640 1 -- 172.21.15.116:0/4158449202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ef8103660 msgr2=0x7f2ef8100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:37.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.951+0000 7f2efe91b640 1 --2- 172.21.15.116:0/4158449202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ef8103660 0x7f2ef8100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.951+0000 7f2efe91b640 1 -- 172.21.15.116:0/4158449202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ef8102d40 msgr2=0x7f2ef81009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:37.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.951+0000 7f2efe91b640 1 --2- 172.21.15.116:0/4158449202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ef8102d40 0x7f2ef81009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.951+0000 7f2efe91b640 1 -- 172.21.15.116:0/4158449202 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef81a6910 con 0x7f2ef810c170 2024-09-12T03:11:37.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.951+0000 7f2efe11a640 1 --2- 172.21.15.116:0/4158449202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ef8102d40 0x7f2ef81009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:37.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.951+0000 7f2efe91b640 1 --2- 172.21.15.116:0/4158449202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ef810c170 0x7f2ef8101420 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000cce0 tx=0x7f2ef000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:37.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.951+0000 7f2eef7fe640 1 -- 172.21.15.116:0/4158449202 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef000f450 con 0x7f2ef810c170 2024-09-12T03:11:37.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.951+0000 7f2eef7fe640 1 -- 172.21.15.116:0/4158449202 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ef000fad0 con 0x7f2ef810c170 2024-09-12T03:11:37.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.951+0000 7f2f0037e640 1 -- 172.21.15.116:0/4158449202 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef81a6c00 con 0x7f2ef810c170 2024-09-12T03:11:37.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.951+0000 7f2eef7fe640 1 -- 172.21.15.116:0/4158449202 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0018c10 con 0x7f2ef810c170 2024-09-12T03:11:37.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.951+0000 7f2f0037e640 1 -- 172.21.15.116:0/4158449202 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ef81a70f0 con 0x7f2ef810c170 2024-09-12T03:11:37.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.953+0000 7f2eef7fe640 1 -- 172.21.15.116:0/4158449202 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2ef000f5f0 con 0x7f2ef810c170 2024-09-12T03:11:37.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.953+0000 7f2f02757740 1 Processor -- start 2024-09-12T03:11:37.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.954+0000 7f2f02757740 1 -- start start 2024-09-12T03:11:37.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.954+0000 7f2f02757740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b23adb7b40 0x55b23adb6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:37.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.954+0000 7f2f02757740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b23adb65d0 0x55b23adb6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:37.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.954+0000 7f2f02757740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b23adc1370 0x55b23adc3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:37.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.954+0000 7f2f02757740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b23ad1c2f0 con 0x55b23adb7b40 2024-09-12T03:11:37.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.954+0000 7f2f02757740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b23acc3cf0 con 0x55b23adb65d0 2024-09-12T03:11:37.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.954+0000 7f2f02757740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b23ad182b0 con 0x55b23adc1370 2024-09-12T03:11:37.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.954+0000 7f2eef7fe640 1 --2- 172.21.15.116:0/4158449202 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2ee40a2b80 0x7f2ee40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:37.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.954+0000 7f2efd919640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b23adb7b40 0x55b23adb6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:37.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.954+0000 7f2efd919640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b23adb7b40 0x55b23adb6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48598/0 (socket says 172.21.15.116:48598) 2024-09-12T03:11:37.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.954+0000 7f2efd919640 1 -- 172.21.15.116:0/362821793 learned_addr learned my addr 172.21.15.116:0/362821793 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:37.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.954+0000 7f2efe91b640 1 --2- 172.21.15.116:0/4158449202 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2ee40a2b80 0x7f2ee40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:37.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.954+0000 7f2efd919640 1 --2- 172.21.15.116:0/362821793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b23adc1370 0x55b23adc3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:37.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.954+0000 7f2efe11a640 1 --2- 172.21.15.116:0/362821793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b23adb65d0 0x55b23adb6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:37.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.954+0000 7f2eef7fe640 1 -- 172.21.15.116:0/4158449202 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f2ef00cf700 con 0x7f2ef810c170 2024-09-12T03:11:37.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.954+0000 7f2efd919640 1 -- 172.21.15.116:0/362821793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b23adc1370 msgr2=0x55b23adc3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:37.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.954+0000 7f2efd919640 1 --2- 172.21.15.116:0/362821793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b23adc1370 0x55b23adc3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.954+0000 7f2efd919640 1 -- 172.21.15.116:0/362821793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b23adb65d0 msgr2=0x55b23adb6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:37.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.954+0000 7f2efd919640 1 --2- 172.21.15.116:0/362821793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b23adb65d0 0x55b23adb6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.954+0000 7f2efd919640 1 -- 172.21.15.116:0/362821793 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b23aca83a0 con 0x55b23adb7b40 2024-09-12T03:11:37.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.954+0000 7f2efe11a640 1 --2- 172.21.15.116:0/362821793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b23adb65d0 0x55b23adb6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:37.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.955+0000 7f2efe91b640 1 --2- 172.21.15.116:0/4158449202 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2ee40a2b80 0x7f2ee40a5040 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2ef009bf20 tx=0x7f2ef006be40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:37.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.955+0000 7f2efd919640 1 --2- 172.21.15.116:0/362821793 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b23adb7b40 0x55b23adb6090 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4006aa0 tx=0x7f2ef40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1351d5a47623c90 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:37.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.955+0000 7f2eecdea640 1 -- 172.21.15.116:0/362821793 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4017610 con 0x55b23adb7b40 2024-09-12T03:11:37.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.955+0000 7f2eecdea640 1 -- 172.21.15.116:0/362821793 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ef401c070 con 0x55b23adb7b40 2024-09-12T03:11:37.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.955+0000 7f2eecdea640 1 -- 172.21.15.116:0/362821793 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4015970 con 0x55b23adb7b40 2024-09-12T03:11:37.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.955+0000 7f2f02757740 1 -- 172.21.15.116:0/362821793 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b23adb7b40 msgr2=0x55b23adb6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:37.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.955+0000 7f2f02757740 1 --2- 172.21.15.116:0/362821793 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b23adb7b40 0x55b23adb6090 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4006aa0 tx=0x7f2ef40095c0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.955+0000 7f2f02757740 1 -- 172.21.15.116:0/362821793 shutdown_connections 2024-09-12T03:11:37.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.955+0000 7f2f02757740 1 --2- 172.21.15.116:0/362821793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b23adc1370 0x55b23adc3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.956+0000 7f2f02757740 1 --2- 172.21.15.116:0/362821793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b23adb65d0 0x55b23adb6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.956+0000 7f2f02757740 1 --2- 172.21.15.116:0/362821793 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b23adb7b40 0x55b23adb6090 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.956+0000 7f2f02757740 1 -- 172.21.15.116:0/362821793 >> 172.21.15.116:0/362821793 conn(0x55b23ad7f310 msgr2=0x55b23adb2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:37.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.956+0000 7f2f02757740 1 -- 172.21.15.116:0/362821793 shutdown_connections 2024-09-12T03:11:37.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.956+0000 7f2f02757740 1 -- 172.21.15.116:0/362821793 wait complete. 2024-09-12T03:11:37.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.956+0000 7f2f02757740 1 -- start start 2024-09-12T03:11:37.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.956+0000 7f2f02757740 1 Processor -- start 2024-09-12T03:11:37.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.956+0000 7f2f02757740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b23adb65d0 0x55b23adc0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:37.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.956+0000 7f2f02757740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b23adb7b40 0x55b23adbdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:37.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.956+0000 7f2f02757740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b23adc1370 0x55b23adbe480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:37.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.956+0000 7f2f02757740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b23adc4010 con 0x55b23adb65d0 2024-09-12T03:11:37.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.956+0000 7f2f02757740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b23acd7060 con 0x55b23adc1370 2024-09-12T03:11:37.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.956+0000 7f2f02757740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b23adc4190 con 0x55b23adb7b40 2024-09-12T03:11:37.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.956+0000 7f2f02757740 10 client.0 _ll_register_callbacks cb 0x55b23aa2a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:37.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.956+0000 7f2efd919640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b23adb65d0 0x55b23adc0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:37.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.956+0000 7f2efd919640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b23adb65d0 0x55b23adc0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48608/0 (socket says 172.21.15.116:48608) 2024-09-12T03:11:37.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.956+0000 7f2efd919640 1 -- 172.21.15.116:0/2192834296 learned_addr learned my addr 172.21.15.116:0/2192834296 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:37.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.957+0000 7f2efd919640 1 -- 172.21.15.116:0/2192834296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b23adb7b40 msgr2=0x55b23adbdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:37.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.957+0000 7f2efd919640 1 --2- 172.21.15.116:0/2192834296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b23adb7b40 0x55b23adbdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.957+0000 7f2efd919640 1 -- 172.21.15.116:0/2192834296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b23adc1370 msgr2=0x55b23adbe480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:37.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.957+0000 7f2efd919640 1 --2- 172.21.15.116:0/2192834296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b23adc1370 0x55b23adbe480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.957+0000 7f2efd919640 1 -- 172.21.15.116:0/2192834296 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b23aa8d3c0 con 0x55b23adb65d0 2024-09-12T03:11:37.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.957+0000 7f2efd919640 1 --2- 172.21.15.116:0/2192834296 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b23adb65d0 0x55b23adc0bf0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4004810 tx=0x7f2ef4009420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8013de3f307d9c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:37.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.957+0000 7f2ee1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:37.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.957+0000 7f2ee1ffb640 1 -- 172.21.15.116:0/2192834296 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4015e60 con 0x55b23adb65d0 2024-09-12T03:11:37.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.957+0000 7f2ee1ffb640 1 -- 172.21.15.116:0/2192834296 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f2ef4017410 con 0x55b23adb65d0 2024-09-12T03:11:37.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.957+0000 7f2f02757740 1 -- 172.21.15.116:0/2192834296 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ef401c070 con 0x55b23adb65d0 2024-09-12T03:11:37.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.957+0000 7f2ee1ffb640 1 -- 172.21.15.116:0/2192834296 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2ef4017630 con 0x55b23adb65d0 2024-09-12T03:11:37.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.957+0000 7f2f02757740 10 client.17550 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:37.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.957+0000 7f2f02757740 1 -- 172.21.15.116:0/2192834296 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b23adcf9c0 con 0x55b23adb65d0 2024-09-12T03:11:37.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.957+0000 7f2ee1ffb640 1 -- 172.21.15.116:0/2192834296 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ef401c070 con 0x55b23adb65d0 2024-09-12T03:11:37.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.957+0000 7f2ee1ffb640 1 -- 172.21.15.116:0/2192834296 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2ef40299d0 con 0x55b23adb65d0 2024-09-12T03:11:37.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.957+0000 7f2f02757740 10 client.17550 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:37.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.957+0000 7f2f02757740 10 client.17550 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:11:37.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.958+0000 7f2f02757740 20 client.17550 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:37.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.958+0000 7f2f02757740 1 --2- 172.21.15.116:0/2192834296 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55b23add0e80 0x55b23add3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:37.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.958+0000 7f2f02757740 4 client.17550 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:11:37.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.958+0000 7f2f02757740 1 -- 172.21.15.116:0/2192834296 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b23ad29860 con 0x55b23add0e80 2024-09-12T03:11:37.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.958+0000 7f2efe11a640 1 --2- 172.21.15.116:0/2192834296 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55b23add0e80 0x55b23add3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:37.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.958+0000 7f2ee1ffb640 10 client.17550 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:11:37.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.958+0000 7f2efe11a640 1 --2- 172.21.15.116:0/2192834296 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55b23add0e80 0x55b23add3270 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c0a6ec311b20af44 server_cookie=3636dbbf3e4f3ec4 in_seq=0 out_seq=0 2024-09-12T03:11:37.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.959+0000 7f2ee1ffb640 1 -- 172.21.15.116:0/2192834296 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b23ad29860 con 0x55b23add0e80 2024-09-12T03:11:37.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.959+0000 7f2ee1ffb640 10 client.17550 handle_command_reply: tid=0 2024-09-12T03:11:37.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.959+0000 7f2f02757740 1 client.17550 shutdown 2024-09-12T03:11:37.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.959+0000 7f2f02757740 1 -- 172.21.15.116:0/2192834296 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b23adb65d0 msgr2=0x55b23adc0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:37.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.959+0000 7f2f02757740 1 --2- 172.21.15.116:0/2192834296 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b23adb65d0 0x55b23adc0bf0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4004810 tx=0x7f2ef4009420 comp rx=0 tx=0).stop 2024-09-12T03:11:37.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.959+0000 7f2f02757740 1 -- 172.21.15.116:0/2192834296 shutdown_connections 2024-09-12T03:11:37.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.959+0000 7f2f02757740 1 --2- 172.21.15.116:0/2192834296 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55b23add0e80 0x55b23add3270 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.959+0000 7f2f02757740 1 --2- 172.21.15.116:0/2192834296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b23adc1370 0x55b23adbe480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.959+0000 7f2f02757740 1 --2- 172.21.15.116:0/2192834296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b23adb7b40 0x55b23adbdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.959+0000 7f2f02757740 1 --2- 172.21.15.116:0/2192834296 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b23adb65d0 0x55b23adc0bf0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.959+0000 7f2f02757740 1 -- 172.21.15.116:0/2192834296 >> 172.21.15.116:0/2192834296 conn(0x55b23ad7f310 msgr2=0x55b23adb8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:37.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.959+0000 7f2ee1ffb640 0 client.17550 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:11:37.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.959+0000 7f2f02757740 1 -- 172.21.15.116:0/2192834296 shutdown_connections 2024-09-12T03:11:37.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.959+0000 7f2f02757740 1 -- 172.21.15.116:0/2192834296 wait complete. 2024-09-12T03:11:37.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.959+0000 7f2f02757740 20 client.17550 trim_cache size 0 max 16384 2024-09-12T03:11:37.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.972+0000 7f2f02757740 1 Processor -- start 2024-09-12T03:11:37.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.972+0000 7f2f02757740 1 -- start start 2024-09-12T03:11:37.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.972+0000 7f2f02757740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b23adb65d0 0x55b23adb1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:37.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.972+0000 7f2f02757740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b23adb7b40 0x55b23adb9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:37.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.972+0000 7f2f02757740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b23adc1370 0x55b23adb9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:37.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.972+0000 7f2f02757740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef81b1f80 con 0x55b23adb65d0 2024-09-12T03:11:37.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.972+0000 7f2f02757740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef81af850 con 0x55b23adc1370 2024-09-12T03:11:37.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.972+0000 7f2f02757740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef81ad6a0 con 0x55b23adb7b40 2024-09-12T03:11:37.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.972+0000 7f2efd919640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b23adb65d0 0x55b23adb1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:37.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.972+0000 7f2efd919640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b23adb65d0 0x55b23adb1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48610/0 (socket says 172.21.15.116:48610) 2024-09-12T03:11:37.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.972+0000 7f2efd919640 1 -- 172.21.15.116:0/825229100 learned_addr learned my addr 172.21.15.116:0/825229100 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:37.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.972+0000 7f2efe11a640 1 --2- 172.21.15.116:0/825229100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b23adb7b40 0x55b23adb9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:37.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.973+0000 7f2efd919640 1 --2- 172.21.15.116:0/825229100 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b23adc1370 0x55b23adb9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:37.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.973+0000 7f2efd919640 1 -- 172.21.15.116:0/825229100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b23adb7b40 msgr2=0x55b23adb9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:37.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.973+0000 7f2efd919640 1 --2- 172.21.15.116:0/825229100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b23adb7b40 0x55b23adb9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.973+0000 7f2efd919640 1 -- 172.21.15.116:0/825229100 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b23adc1370 msgr2=0x55b23adb9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:37.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.973+0000 7f2efd919640 1 --2- 172.21.15.116:0/825229100 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b23adc1370 0x55b23adb9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.973+0000 7f2efd919640 1 -- 172.21.15.116:0/825229100 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b23adb7300 con 0x55b23adb65d0 2024-09-12T03:11:37.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.973+0000 7f2efd919640 1 --2- 172.21.15.116:0/825229100 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b23adb65d0 0x55b23adb1c10 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4009790 tx=0x7f2ef4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8a2caf94d3430 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:37.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.973+0000 7f2ee3fff640 1 -- 172.21.15.116:0/825229100 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4015c60 con 0x55b23adb65d0 2024-09-12T03:11:37.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.973+0000 7f2ee3fff640 1 -- 172.21.15.116:0/825229100 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ef401c040 con 0x55b23adb65d0 2024-09-12T03:11:37.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.973+0000 7f2ee3fff640 1 -- 172.21.15.116:0/825229100 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4029400 con 0x55b23adb65d0 2024-09-12T03:11:37.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.974+0000 7f2f02757740 1 -- 172.21.15.116:0/825229100 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b23adb65d0 msgr2=0x55b23adb1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:37.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.974+0000 7f2f02757740 1 --2- 172.21.15.116:0/825229100 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b23adb65d0 0x55b23adb1c10 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4009790 tx=0x7f2ef4001e70 comp rx=0 tx=0).stop 2024-09-12T03:11:37.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.974+0000 7f2f02757740 1 -- 172.21.15.116:0/825229100 shutdown_connections 2024-09-12T03:11:37.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.974+0000 7f2f02757740 1 --2- 172.21.15.116:0/825229100 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b23adc1370 0x55b23adb9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.974+0000 7f2f02757740 1 --2- 172.21.15.116:0/825229100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b23adb7b40 0x55b23adb9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.974+0000 7f2f02757740 1 --2- 172.21.15.116:0/825229100 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b23adb65d0 0x55b23adb1c10 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.974+0000 7f2f02757740 1 -- 172.21.15.116:0/825229100 >> 172.21.15.116:0/825229100 conn(0x55b23ad7f310 msgr2=0x55b23adb1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:37.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.974+0000 7f2f02757740 1 -- 172.21.15.116:0/825229100 shutdown_connections 2024-09-12T03:11:37.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.974+0000 7f2f02757740 1 -- 172.21.15.116:0/825229100 wait complete. 2024-09-12T03:11:37.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.974+0000 7f2f02757740 1 -- start start 2024-09-12T03:11:37.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.974+0000 7f2f02757740 1 Processor -- start 2024-09-12T03:11:37.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.974+0000 7f2f02757740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b23adb65d0 0x55b23ae33cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:37.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.974+0000 7f2f02757740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b23adb7b40 0x55b23ae341f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:37.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.974+0000 7f2f02757740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b23adc1370 0x55b23adc49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:37.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.974+0000 7f2f02757740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef81add10 con 0x55b23adb7b40 2024-09-12T03:11:37.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.974+0000 7f2f02757740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef81af180 con 0x55b23adc1370 2024-09-12T03:11:37.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.974+0000 7f2f02757740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef81ae3e0 con 0x55b23adb65d0 2024-09-12T03:11:37.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.974+0000 7f2f02757740 10 client.0 _ll_register_callbacks cb 0x7f2ef4009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:37.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.974+0000 7f2efe11a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b23adb7b40 0x55b23ae341f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:37.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.975+0000 7f2efe11a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b23adb7b40 0x55b23ae341f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48614/0 (socket says 172.21.15.116:48614) 2024-09-12T03:11:37.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.975+0000 7f2efe11a640 1 -- 172.21.15.116:0/980509603 learned_addr learned my addr 172.21.15.116:0/980509603 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:37.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.975+0000 7f2efd919640 1 --2- 172.21.15.116:0/980509603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b23adc1370 0x55b23adc49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:37.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.975+0000 7f2efd919640 1 --2- 172.21.15.116:0/980509603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b23adb65d0 0x55b23ae33cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:37.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.975+0000 7f2efe11a640 1 -- 172.21.15.116:0/980509603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b23adb65d0 msgr2=0x55b23ae33cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:37.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.975+0000 7f2efe11a640 1 --2- 172.21.15.116:0/980509603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b23adb65d0 0x55b23ae33cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.975+0000 7f2efe11a640 1 -- 172.21.15.116:0/980509603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b23adc1370 msgr2=0x55b23adc49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:37.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.975+0000 7f2efe11a640 1 --2- 172.21.15.116:0/980509603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b23adc1370 0x55b23adc49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.975+0000 7f2efe11a640 1 -- 172.21.15.116:0/980509603 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b23adc5210 con 0x55b23adb7b40 2024-09-12T03:11:37.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.975+0000 7f2efd919640 1 --2- 172.21.15.116:0/980509603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b23adc1370 0x55b23adc49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:11:37.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.975+0000 7f2efe11a640 1 --2- 172.21.15.116:0/980509603 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b23adb7b40 0x55b23ae341f0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8009230 tx=0x7f2ee8032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe238d1727b6527 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:37.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.975+0000 7f2ee1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:37.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.975+0000 7f2ee1ffb640 1 -- 172.21.15.116:0/980509603 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8048020 con 0x55b23adb7b40 2024-09-12T03:11:37.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.975+0000 7f2ee1ffb640 1 -- 172.21.15.116:0/980509603 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f2ee8005040 con 0x55b23adb7b40 2024-09-12T03:11:37.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.975+0000 7f2f02757740 1 -- 172.21.15.116:0/980509603 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ef401c040 con 0x55b23adb7b40 2024-09-12T03:11:37.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.975+0000 7f2ee1ffb640 1 -- 172.21.15.116:0/980509603 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2ee8035040 con 0x55b23adb7b40 2024-09-12T03:11:37.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.975+0000 7f2f02757740 10 client.17556 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:37.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.975+0000 7f2f02757740 1 -- 172.21.15.116:0/980509603 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b23ae3b510 con 0x55b23adb7b40 2024-09-12T03:11:37.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.975+0000 7f2ee1ffb640 1 -- 172.21.15.116:0/980509603 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ef401c040 con 0x55b23adb7b40 2024-09-12T03:11:37.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.976+0000 7f2ee1ffb640 1 -- 172.21.15.116:0/980509603 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2ee80035f0 con 0x55b23adb7b40 2024-09-12T03:11:37.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.976+0000 7f2f02757740 10 client.17556 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:37.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.976+0000 7f2f02757740 10 client.17556 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:11:37.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.976+0000 7f2f02757740 20 client.17556 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:37.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.976+0000 7f2f02757740 1 --2- 172.21.15.116:0/980509603 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55b23ae3bc90 0x55b23ae3e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:37.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.976+0000 7f2f02757740 4 client.17556 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:11:37.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.976+0000 7f2f02757740 1 -- 172.21.15.116:0/980509603 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b23ad23d70 con 0x55b23ae3bc90 2024-09-12T03:11:37.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.976+0000 7f2efd919640 1 --2- 172.21.15.116:0/980509603 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55b23ae3bc90 0x55b23ae3e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:37.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.976+0000 7f2efd919640 1 --2- 172.21.15.116:0/980509603 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55b23ae3bc90 0x55b23ae3e0b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9b05b12e39526d80 server_cookie=dace6b86e0b6a83 in_seq=0 out_seq=0 2024-09-12T03:11:37.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.976+0000 7f2ee1ffb640 10 client.17556 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:11:37.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.976+0000 7f2ee1ffb640 1 -- 172.21.15.116:0/980509603 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b23ad23d70 con 0x55b23ae3bc90 2024-09-12T03:11:37.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.976+0000 7f2ee1ffb640 10 client.17556 handle_command_reply: tid=0 2024-09-12T03:11:37.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.976+0000 7f2f02757740 1 client.17556 shutdown 2024-09-12T03:11:37.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.977+0000 7f2f02757740 1 -- 172.21.15.116:0/980509603 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b23adb7b40 msgr2=0x55b23ae341f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:37.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.977+0000 7f2f02757740 1 --2- 172.21.15.116:0/980509603 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b23adb7b40 0x55b23ae341f0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8009230 tx=0x7f2ee8032ec0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.977+0000 7f2f02757740 1 -- 172.21.15.116:0/980509603 shutdown_connections 2024-09-12T03:11:37.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.977+0000 7f2f02757740 1 --2- 172.21.15.116:0/980509603 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55b23ae3bc90 0x55b23ae3e0b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.977+0000 7f2f02757740 1 --2- 172.21.15.116:0/980509603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b23adc1370 0x55b23adc49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.977+0000 7f2f02757740 1 --2- 172.21.15.116:0/980509603 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b23adb7b40 0x55b23ae341f0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.977+0000 7f2f02757740 1 --2- 172.21.15.116:0/980509603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b23adb65d0 0x55b23ae33cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.977+0000 7f2f02757740 1 -- 172.21.15.116:0/980509603 >> 172.21.15.116:0/980509603 conn(0x55b23ad7f310 msgr2=0x55b23adb1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:37.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.977+0000 7f2ee1ffb640 0 client.17556 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:11:37.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.977+0000 7f2f02757740 1 -- 172.21.15.116:0/980509603 shutdown_connections 2024-09-12T03:11:37.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.977+0000 7f2f02757740 1 -- 172.21.15.116:0/980509603 wait complete. 2024-09-12T03:11:37.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.977+0000 7f2f02757740 20 client.17556 trim_cache size 0 max 16384 2024-09-12T03:11:37.991 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:11:37.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.977+0000 7f2f0037e640 1 -- 172.21.15.116:0/4158449202 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2ee40a2b80 msgr2=0x7f2ee40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:37.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.977+0000 7f2f0037e640 1 --2- 172.21.15.116:0/4158449202 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2ee40a2b80 0x7f2ee40a5040 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2ef009bf20 tx=0x7f2ef006be40 comp rx=0 tx=0).stop 2024-09-12T03:11:37.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.977+0000 7f2f0037e640 1 -- 172.21.15.116:0/4158449202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ef810c170 msgr2=0x7f2ef8101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:37.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.977+0000 7f2f0037e640 1 --2- 172.21.15.116:0/4158449202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ef810c170 0x7f2ef8101420 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000cce0 tx=0x7f2ef000a620 comp rx=0 tx=0).stop 2024-09-12T03:11:37.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.977+0000 7f2f0037e640 1 -- 172.21.15.116:0/4158449202 shutdown_connections 2024-09-12T03:11:37.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.977+0000 7f2f0037e640 1 --2- 172.21.15.116:0/4158449202 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2ee40a2b80 0x7f2ee40a5040 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.977+0000 7f2f0037e640 1 --2- 172.21.15.116:0/4158449202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ef810c170 0x7f2ef8101420 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.977+0000 7f2f0037e640 1 --2- 172.21.15.116:0/4158449202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ef8103660 0x7f2ef8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.977+0000 7f2f0037e640 1 --2- 172.21.15.116:0/4158449202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ef8102d40 0x7f2ef81009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:37.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.977+0000 7f2f0037e640 1 -- 172.21.15.116:0/4158449202 >> 172.21.15.116:0/4158449202 conn(0x7f2ef80fc7a0 msgr2=0x7f2ef80fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:37.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.977+0000 7f2f0037e640 1 -- 172.21.15.116:0/4158449202 shutdown_connections 2024-09-12T03:11:37.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:37.978+0000 7f2f0037e640 1 -- 172.21.15.116:0/4158449202 wait complete. 2024-09-12T03:11:38.705 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:11:38.954 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:11:39.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.451+0000 7f0245581640 1 -- 172.21.15.116:0/1895314513 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02401082a0 msgr2=0x7f024010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:39.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.451+0000 7f0245581640 1 --2- 172.21.15.116:0/1895314513 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02401082a0 0x7f024010ec40 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f0238009930 tx=0x7f023802a440 comp rx=0 tx=0).stop 2024-09-12T03:11:39.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.451+0000 7f0245581640 1 -- 172.21.15.116:0/1895314513 shutdown_connections 2024-09-12T03:11:39.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.451+0000 7f0245581640 1 --2- 172.21.15.116:0/1895314513 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02401082a0 0x7f024010ec40 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.451+0000 7f0245581640 1 --2- 172.21.15.116:0/1895314513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02401036e0 0x7f0240107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.451+0000 7f0245581640 1 --2- 172.21.15.116:0/1895314513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0240102d30 0x7f0240103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.451+0000 7f0245581640 1 -- 172.21.15.116:0/1895314513 >> 172.21.15.116:0/1895314513 conn(0x7f02400fc7d0 msgr2=0x7f02400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:39.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.451+0000 7f0245581640 1 -- 172.21.15.116:0/1895314513 shutdown_connections 2024-09-12T03:11:39.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.451+0000 7f0245581640 1 -- 172.21.15.116:0/1895314513 wait complete. 2024-09-12T03:11:39.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.451+0000 7f0245581640 1 Processor -- start 2024-09-12T03:11:39.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.451+0000 7f0245581640 1 -- start start 2024-09-12T03:11:39.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.452+0000 7f0245581640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0240102d30 0x7f0240198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.452+0000 7f0245581640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02401036e0 0x7f0240198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.452+0000 7f0245581640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02401082a0 0x7f024019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.452+0000 7f0245581640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f02401120e0 con 0x7f02401082a0 2024-09-12T03:11:39.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.452+0000 7f0245581640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0240111f60 con 0x7f0240102d30 2024-09-12T03:11:39.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.452+0000 7f0245581640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0240112260 con 0x7f02401036e0 2024-09-12T03:11:39.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.452+0000 7f023f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02401082a0 0x7f024019cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.452+0000 7f023f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02401082a0 0x7f024019cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48632/0 (socket says 172.21.15.116:48632) 2024-09-12T03:11:39.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.452+0000 7f023f7fe640 1 -- 172.21.15.116:0/1980425874 learned_addr learned my addr 172.21.15.116:0/1980425874 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:39.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.452+0000 7f023e7fc640 1 --2- 172.21.15.116:0/1980425874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02401036e0 0x7f0240198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.452+0000 7f023effd640 1 --2- 172.21.15.116:0/1980425874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0240102d30 0x7f0240198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.452+0000 7f023f7fe640 1 -- 172.21.15.116:0/1980425874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02401036e0 msgr2=0x7f0240198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:39.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.452+0000 7f023f7fe640 1 --2- 172.21.15.116:0/1980425874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02401036e0 0x7f0240198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.452+0000 7f023f7fe640 1 -- 172.21.15.116:0/1980425874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0240102d30 msgr2=0x7f0240198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:39.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.452+0000 7f023f7fe640 1 --2- 172.21.15.116:0/1980425874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0240102d30 0x7f0240198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.452+0000 7f023f7fe640 1 -- 172.21.15.116:0/1980425874 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f024019d5d0 con 0x7f02401082a0 2024-09-12T03:11:39.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.452+0000 7f023effd640 1 --2- 172.21.15.116:0/1980425874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0240102d30 0x7f0240198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:39.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.452+0000 7f023f7fe640 1 --2- 172.21.15.116:0/1980425874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02401082a0 0x7f024019cef0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f0238005ab0 tx=0x7f02380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:39.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.452+0000 7f0237fff640 1 -- 172.21.15.116:0/1980425874 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023803f030 con 0x7f02401082a0 2024-09-12T03:11:39.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.452+0000 7f0237fff640 1 -- 172.21.15.116:0/1980425874 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0238003ce0 con 0x7f02401082a0 2024-09-12T03:11:39.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.452+0000 7f0245581640 1 -- 172.21.15.116:0/1980425874 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f024019d860 con 0x7f02401082a0 2024-09-12T03:11:39.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.452+0000 7f0237fff640 1 -- 172.21.15.116:0/1980425874 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0238035070 con 0x7f02401082a0 2024-09-12T03:11:39.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.453+0000 7f0245581640 1 -- 172.21.15.116:0/1980425874 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02401a51a0 con 0x7f02401082a0 2024-09-12T03:11:39.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.455+0000 7f0237fff640 1 -- 172.21.15.116:0/1980425874 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f0238042030 con 0x7f02401082a0 2024-09-12T03:11:39.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.455+0000 7f024795a740 1 Processor -- start 2024-09-12T03:11:39.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.455+0000 7f024795a740 1 -- start start 2024-09-12T03:11:39.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.455+0000 7f024795a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616eefe1b40 0x5616eefe0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.455+0000 7f024795a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5616eefe05d0 0x5616eefe0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.455+0000 7f024795a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616eefef780 0x5616eeff1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.455+0000 7f024795a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5616eef462f0 con 0x5616eefe05d0 2024-09-12T03:11:39.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.455+0000 7f024795a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5616eeeedcf0 con 0x5616eefe1b40 2024-09-12T03:11:39.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.455+0000 7f024795a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5616eef422b0 con 0x5616eefef780 2024-09-12T03:11:39.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.456+0000 7f023effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5616eefe05d0 0x5616eefe0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.456+0000 7f0237fff640 1 --2- 172.21.15.116:0/1980425874 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f022c0a2c00 0x7f022c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.456+0000 7f023effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5616eefe05d0 0x5616eefe0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48646/0 (socket says 172.21.15.116:48646) 2024-09-12T03:11:39.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.456+0000 7f023effd640 1 -- 172.21.15.116:0/197888618 learned_addr learned my addr 172.21.15.116:0/197888618 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:39.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.456+0000 7f023e7fc640 1 --2- 172.21.15.116:0/197888618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616eefe1b40 0x5616eefe0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.456+0000 7f023f7fe640 1 --2- 172.21.15.116:0/1980425874 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f022c0a2c00 0x7f022c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.456+0000 7f023e7fc640 1 --2- 172.21.15.116:0/197888618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616eefef780 0x5616eeff1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.456+0000 7f023effd640 1 -- 172.21.15.116:0/197888618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616eefef780 msgr2=0x5616eeff1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:39.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.456+0000 7f023effd640 1 --2- 172.21.15.116:0/197888618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616eefef780 0x5616eeff1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.456+0000 7f023effd640 1 -- 172.21.15.116:0/197888618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616eefe1b40 msgr2=0x5616eefe0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:39.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.456+0000 7f023effd640 1 --2- 172.21.15.116:0/197888618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616eefe1b40 0x5616eefe0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.456+0000 7f023effd640 1 -- 172.21.15.116:0/197888618 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616eeed23a0 con 0x5616eefe05d0 2024-09-12T03:11:39.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.456+0000 7f023e7fc640 1 --2- 172.21.15.116:0/197888618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616eefe1b40 0x5616eefe0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:11:39.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.456+0000 7f0237fff640 1 -- 172.21.15.116:0/1980425874 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f02380ef7c0 con 0x7f02401082a0 2024-09-12T03:11:39.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.456+0000 7f023effd640 1 --2- 172.21.15.116:0/197888618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5616eefe05d0 0x5616eefe0a10 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f0230009f20 tx=0x7f023000f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca9077447af316a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:39.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.456+0000 7f02357fa640 1 -- 172.21.15.116:0/197888618 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0230003b70 con 0x5616eefe05d0 2024-09-12T03:11:39.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.456+0000 7f02357fa640 1 -- 172.21.15.116:0/197888618 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0230003d10 con 0x5616eefe05d0 2024-09-12T03:11:39.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.456+0000 7f02357fa640 1 -- 172.21.15.116:0/197888618 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0230011a30 con 0x5616eefe05d0 2024-09-12T03:11:39.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.456+0000 7f023f7fe640 1 --2- 172.21.15.116:0/1980425874 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f022c0a2c00 0x7f022c0a50c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f02380bbfe0 tx=0x7f0238009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:39.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.457+0000 7f024795a740 1 -- 172.21.15.116:0/197888618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5616eefe05d0 msgr2=0x5616eefe0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:39.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.457+0000 7f024795a740 1 --2- 172.21.15.116:0/197888618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5616eefe05d0 0x5616eefe0a10 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f0230009f20 tx=0x7f023000f7f0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.457+0000 7f024795a740 1 -- 172.21.15.116:0/197888618 shutdown_connections 2024-09-12T03:11:39.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.457+0000 7f024795a740 1 --2- 172.21.15.116:0/197888618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616eefef780 0x5616eeff1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.457+0000 7f024795a740 1 --2- 172.21.15.116:0/197888618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5616eefe05d0 0x5616eefe0a10 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.457+0000 7f024795a740 1 --2- 172.21.15.116:0/197888618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616eefe1b40 0x5616eefe0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.457+0000 7f024795a740 1 -- 172.21.15.116:0/197888618 >> 172.21.15.116:0/197888618 conn(0x5616eefa9310 msgr2=0x5616eefdcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:39.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.457+0000 7f024795a740 1 -- 172.21.15.116:0/197888618 shutdown_connections 2024-09-12T03:11:39.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.457+0000 7f024795a740 1 -- 172.21.15.116:0/197888618 wait complete. 2024-09-12T03:11:39.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.458+0000 7f024795a740 1 -- start start 2024-09-12T03:11:39.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.458+0000 7f024795a740 1 Processor -- start 2024-09-12T03:11:39.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.458+0000 7f024795a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5616eefe05d0 0x5616eefef190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.458+0000 7f024795a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616eefe1b40 0x5616eefec440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.458+0000 7f024795a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616eefef780 0x5616eefeca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.458+0000 7f024795a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5616eeff2480 con 0x5616eefe05d0 2024-09-12T03:11:39.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.458+0000 7f024795a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5616eef01060 con 0x5616eefef780 2024-09-12T03:11:39.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.458+0000 7f024795a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5616eeff2600 con 0x5616eefe1b40 2024-09-12T03:11:39.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.458+0000 7f024795a740 10 client.0 _ll_register_callbacks cb 0x5616eec548c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:39.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.458+0000 7f023e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5616eefe05d0 0x5616eefef190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.458+0000 7f023e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5616eefe05d0 0x5616eefef190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48648/0 (socket says 172.21.15.116:48648) 2024-09-12T03:11:39.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.458+0000 7f023e7fc640 1 -- 172.21.15.116:0/2495745276 learned_addr learned my addr 172.21.15.116:0/2495745276 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:39.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.458+0000 7f023effd640 1 --2- 172.21.15.116:0/2495745276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616eefe1b40 0x5616eefec440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.458+0000 7f023e7fc640 1 -- 172.21.15.116:0/2495745276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616eefe1b40 msgr2=0x5616eefec440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:39.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.458+0000 7f023e7fc640 1 --2- 172.21.15.116:0/2495745276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616eefe1b40 0x5616eefec440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.458+0000 7f023e7fc640 1 -- 172.21.15.116:0/2495745276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616eefef780 msgr2=0x5616eefeca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:39.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.458+0000 7f023e7fc640 1 --2- 172.21.15.116:0/2495745276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616eefef780 0x5616eefeca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.458+0000 7f023e7fc640 1 -- 172.21.15.116:0/2495745276 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616eecb73c0 con 0x5616eefe05d0 2024-09-12T03:11:39.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.458+0000 7f023effd640 1 --2- 172.21.15.116:0/2495745276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616eefe1b40 0x5616eefec440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:11:39.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.459+0000 7f023e7fc640 1 --2- 172.21.15.116:0/2495745276 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5616eefe05d0 0x5616eefef190 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f0228009f80 tx=0x7f02280157f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dea12168cf00e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:39.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.459+0000 7f02267fc640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:39.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.459+0000 7f02267fc640 1 -- 172.21.15.116:0/2495745276 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022801b070 con 0x5616eefe05d0 2024-09-12T03:11:39.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.459+0000 7f02267fc640 1 -- 172.21.15.116:0/2495745276 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f02280136a0 con 0x5616eefe05d0 2024-09-12T03:11:39.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.459+0000 7f024795a740 1 -- 172.21.15.116:0/2495745276 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0230003d10 con 0x5616eefe05d0 2024-09-12T03:11:39.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.459+0000 7f02267fc640 1 -- 172.21.15.116:0/2495745276 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0228027050 con 0x5616eefe05d0 2024-09-12T03:11:39.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.459+0000 7f024795a740 10 client.17568 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:39.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.459+0000 7f024795a740 1 -- 172.21.15.116:0/2495745276 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5616eeffdfb0 con 0x5616eefe05d0 2024-09-12T03:11:39.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.459+0000 7f02267fc640 1 -- 172.21.15.116:0/2495745276 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0230003d10 con 0x5616eefe05d0 2024-09-12T03:11:39.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.459+0000 7f02267fc640 1 -- 172.21.15.116:0/2495745276 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f02280179c0 con 0x5616eefe05d0 2024-09-12T03:11:39.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.459+0000 7f024795a740 10 client.17568 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:39.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.459+0000 7f024795a740 10 client.17568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:11:39.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.459+0000 7f024795a740 20 client.17568 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:39.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.459+0000 7f024795a740 1 --2- 172.21.15.116:0/2495745276 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5616eefff3c0 0x5616ef0017b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.459+0000 7f024795a740 4 client.17568 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:11:39.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.459+0000 7f024795a740 1 -- 172.21.15.116:0/2495745276 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616eef53860 con 0x5616eefff3c0 2024-09-12T03:11:39.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.460+0000 7f023effd640 1 --2- 172.21.15.116:0/2495745276 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5616eefff3c0 0x5616ef0017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.460+0000 7f02267fc640 10 client.17568 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:39.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.460+0000 7f023effd640 1 --2- 172.21.15.116:0/2495745276 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5616eefff3c0 0x5616ef0017b0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a973878319bfefdf server_cookie=631a1cc582470f7d in_seq=0 out_seq=0 2024-09-12T03:11:39.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.461+0000 7f02267fc640 1 -- 172.21.15.116:0/2495745276 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5616eef53860 con 0x5616eefff3c0 2024-09-12T03:11:39.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.461+0000 7f02267fc640 10 client.17568 handle_command_reply: tid=0 2024-09-12T03:11:39.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.461+0000 7f024795a740 1 client.17568 shutdown 2024-09-12T03:11:39.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.461+0000 7f024795a740 1 -- 172.21.15.116:0/2495745276 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5616eefe05d0 msgr2=0x5616eefef190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:39.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.461+0000 7f024795a740 1 --2- 172.21.15.116:0/2495745276 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5616eefe05d0 0x5616eefef190 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f0228009f80 tx=0x7f02280157f0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.461+0000 7f024795a740 1 -- 172.21.15.116:0/2495745276 shutdown_connections 2024-09-12T03:11:39.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.461+0000 7f024795a740 1 --2- 172.21.15.116:0/2495745276 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5616eefff3c0 0x5616ef0017b0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.461+0000 7f024795a740 1 --2- 172.21.15.116:0/2495745276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616eefef780 0x5616eefeca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.461+0000 7f024795a740 1 --2- 172.21.15.116:0/2495745276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616eefe1b40 0x5616eefec440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.461+0000 7f024795a740 1 --2- 172.21.15.116:0/2495745276 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5616eefe05d0 0x5616eefef190 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.461+0000 7f024795a740 1 -- 172.21.15.116:0/2495745276 >> 172.21.15.116:0/2495745276 conn(0x5616eefa9310 msgr2=0x5616eefe2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:39.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.461+0000 7f02267fc640 0 client.17568 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:39.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.461+0000 7f024795a740 1 -- 172.21.15.116:0/2495745276 shutdown_connections 2024-09-12T03:11:39.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.461+0000 7f024795a740 1 -- 172.21.15.116:0/2495745276 wait complete. 2024-09-12T03:11:39.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.461+0000 7f024795a740 20 client.17568 trim_cache size 0 max 16384 2024-09-12T03:11:39.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.474+0000 7f024795a740 1 Processor -- start 2024-09-12T03:11:39.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.474+0000 7f024795a740 1 -- start start 2024-09-12T03:11:39.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.474+0000 7f024795a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616eefe05d0 0x5616eefdce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.474+0000 7f024795a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5616eefe1b40 0x5616eeffb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.474+0000 7f024795a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616eefef780 0x5616ef007060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.474+0000 7f024795a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f02401adc70 con 0x5616eefe1b40 2024-09-12T03:11:39.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.474+0000 7f024795a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f02401ab540 con 0x5616eefe05d0 2024-09-12T03:11:39.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.474+0000 7f024795a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f02401a93b0 con 0x5616eefef780 2024-09-12T03:11:39.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.475+0000 7f023effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5616eefe1b40 0x5616eeffb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.475+0000 7f023effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5616eefe1b40 0x5616eeffb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48650/0 (socket says 172.21.15.116:48650) 2024-09-12T03:11:39.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.475+0000 7f023effd640 1 -- 172.21.15.116:0/3887455695 learned_addr learned my addr 172.21.15.116:0/3887455695 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:39.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.475+0000 7f023e7fc640 1 --2- 172.21.15.116:0/3887455695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616eefef780 0x5616ef007060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.475+0000 7f023e7fc640 1 --2- 172.21.15.116:0/3887455695 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616eefe05d0 0x5616eefdce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.475+0000 7f023effd640 1 -- 172.21.15.116:0/3887455695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616eefef780 msgr2=0x5616ef007060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:39.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.475+0000 7f023effd640 1 --2- 172.21.15.116:0/3887455695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616eefef780 0x5616ef007060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.475+0000 7f023effd640 1 -- 172.21.15.116:0/3887455695 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616eefe05d0 msgr2=0x5616eefdce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:39.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.475+0000 7f023effd640 1 --2- 172.21.15.116:0/3887455695 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616eefe05d0 0x5616eefdce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.475+0000 7f023effd640 1 -- 172.21.15.116:0/3887455695 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616eefed3e0 con 0x5616eefe1b40 2024-09-12T03:11:39.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.475+0000 7f023effd640 1 --2- 172.21.15.116:0/3887455695 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5616eefe1b40 0x5616eeffb720 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x5616eefddac0 tx=0x7f0230009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ef3028e4c9431b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:39.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.475+0000 7f0234ff9640 1 -- 172.21.15.116:0/3887455695 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0230003b70 con 0x5616eefe1b40 2024-09-12T03:11:39.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.475+0000 7f0234ff9640 1 -- 172.21.15.116:0/3887455695 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0230003d10 con 0x5616eefe1b40 2024-09-12T03:11:39.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.475+0000 7f0234ff9640 1 -- 172.21.15.116:0/3887455695 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0230011860 con 0x5616eefe1b40 2024-09-12T03:11:39.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.476+0000 7f024795a740 1 -- 172.21.15.116:0/3887455695 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5616eefe1b40 msgr2=0x5616eeffb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:39.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.476+0000 7f024795a740 1 --2- 172.21.15.116:0/3887455695 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5616eefe1b40 0x5616eeffb720 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x5616eefddac0 tx=0x7f0230009810 comp rx=0 tx=0).stop 2024-09-12T03:11:39.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.476+0000 7f024795a740 1 -- 172.21.15.116:0/3887455695 shutdown_connections 2024-09-12T03:11:39.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.476+0000 7f024795a740 1 --2- 172.21.15.116:0/3887455695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616eefef780 0x5616ef007060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.476+0000 7f024795a740 1 --2- 172.21.15.116:0/3887455695 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5616eefe1b40 0x5616eeffb720 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.476+0000 7f024795a740 1 --2- 172.21.15.116:0/3887455695 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616eefe05d0 0x5616eefdce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.476+0000 7f024795a740 1 -- 172.21.15.116:0/3887455695 >> 172.21.15.116:0/3887455695 conn(0x5616eefa9310 msgr2=0x5616eefdc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:39.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.476+0000 7f024795a740 1 -- 172.21.15.116:0/3887455695 shutdown_connections 2024-09-12T03:11:39.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.476+0000 7f024795a740 1 -- 172.21.15.116:0/3887455695 wait complete. 2024-09-12T03:11:39.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.476+0000 7f024795a740 1 -- start start 2024-09-12T03:11:39.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.477+0000 7f024795a740 1 Processor -- start 2024-09-12T03:11:39.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.477+0000 7f024795a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616eefe05d0 0x5616eeff7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.477+0000 7f024795a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5616eefe1b40 0x5616eefe8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.477+0000 7f024795a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616eefef780 0x5616eefe94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.477+0000 7f024795a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f02401a9a00 con 0x5616eefe1b40 2024-09-12T03:11:39.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.477+0000 7f024795a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f02401aae70 con 0x5616eefe05d0 2024-09-12T03:11:39.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.477+0000 7f024795a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f02401aa0d0 con 0x5616eefef780 2024-09-12T03:11:39.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.477+0000 7f024795a740 10 client.0 _ll_register_callbacks cb 0x7f0228009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:39.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.477+0000 7f023effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5616eefe1b40 0x5616eefe8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.477+0000 7f023effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5616eefe1b40 0x5616eefe8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48666/0 (socket says 172.21.15.116:48666) 2024-09-12T03:11:39.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.477+0000 7f023effd640 1 -- 172.21.15.116:0/1423950242 learned_addr learned my addr 172.21.15.116:0/1423950242 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:39.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.477+0000 7f023e7fc640 1 --2- 172.21.15.116:0/1423950242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616eefef780 0x5616eefe94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.477+0000 7f023e7fc640 1 --2- 172.21.15.116:0/1423950242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616eefe05d0 0x5616eeff7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.477+0000 7f023effd640 1 -- 172.21.15.116:0/1423950242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616eefef780 msgr2=0x5616eefe94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:39.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.477+0000 7f023effd640 1 --2- 172.21.15.116:0/1423950242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616eefef780 0x5616eefe94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.477+0000 7f023effd640 1 -- 172.21.15.116:0/1423950242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616eefe05d0 msgr2=0x5616eeff7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:39.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.477+0000 7f023effd640 1 --2- 172.21.15.116:0/1423950242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616eefe05d0 0x5616eeff7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.477+0000 7f023effd640 1 -- 172.21.15.116:0/1423950242 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616eefe9b30 con 0x5616eefe1b40 2024-09-12T03:11:39.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.477+0000 7f023e7fc640 1 --2- 172.21.15.116:0/1423950242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616eefef780 0x5616eefe94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:11:39.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.477+0000 7f023effd640 1 --2- 172.21.15.116:0/1423950242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5616eefe1b40 0x5616eefe8f80 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f023000f830 tx=0x7f0230009a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c705bc907729cfe8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:39.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.477+0000 7f02267fc640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:39.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.478+0000 7f02267fc640 1 -- 172.21.15.116:0/1423950242 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0230004a00 con 0x5616eefe1b40 2024-09-12T03:11:39.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.478+0000 7f02267fc640 1 -- 172.21.15.116:0/1423950242 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f0230004ba0 con 0x5616eefe1b40 2024-09-12T03:11:39.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.478+0000 7f024795a740 1 -- 172.21.15.116:0/1423950242 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0230003d10 con 0x5616eefe1b40 2024-09-12T03:11:39.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.478+0000 7f02267fc640 1 -- 172.21.15.116:0/1423950242 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0230004da0 con 0x5616eefe1b40 2024-09-12T03:11:39.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.478+0000 7f024795a740 10 client.17574 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:39.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.478+0000 7f024795a740 1 -- 172.21.15.116:0/1423950242 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5616ef069ee0 con 0x5616eefe1b40 2024-09-12T03:11:39.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.478+0000 7f02267fc640 1 -- 172.21.15.116:0/1423950242 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0230003d10 con 0x5616eefe1b40 2024-09-12T03:11:39.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.478+0000 7f02267fc640 1 -- 172.21.15.116:0/1423950242 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f02300119c0 con 0x5616eefe1b40 2024-09-12T03:11:39.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.478+0000 7f024795a740 10 client.17574 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:39.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.478+0000 7f024795a740 10 client.17574 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:11:39.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.478+0000 7f024795a740 20 client.17574 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:39.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.478+0000 7f024795a740 1 --2- 172.21.15.116:0/1423950242 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5616ef06a3a0 0x5616ef06c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.478+0000 7f024795a740 4 client.17574 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:11:39.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.478+0000 7f024795a740 1 -- 172.21.15.116:0/1423950242 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5616eef4dd70 con 0x5616ef06a3a0 2024-09-12T03:11:39.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.478+0000 7f023e7fc640 1 --2- 172.21.15.116:0/1423950242 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5616ef06a3a0 0x5616ef06c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.479+0000 7f02267fc640 10 client.17574 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:39.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.479+0000 7f023e7fc640 1 --2- 172.21.15.116:0/1423950242 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5616ef06a3a0 0x5616ef06c790 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a16a4c4e0dc8ab4f server_cookie=6a5290437bb1f351 in_seq=0 out_seq=0 2024-09-12T03:11:39.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.479+0000 7f02267fc640 1 -- 172.21.15.116:0/1423950242 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5616eef4dd70 con 0x5616ef06a3a0 2024-09-12T03:11:39.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.479+0000 7f02267fc640 10 client.17574 handle_command_reply: tid=0 2024-09-12T03:11:39.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.479+0000 7f024795a740 1 client.17574 shutdown 2024-09-12T03:11:39.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.479+0000 7f024795a740 1 -- 172.21.15.116:0/1423950242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5616eefe1b40 msgr2=0x5616eefe8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:39.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.479+0000 7f024795a740 1 --2- 172.21.15.116:0/1423950242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5616eefe1b40 0x5616eefe8f80 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f023000f830 tx=0x7f0230009a30 comp rx=0 tx=0).stop 2024-09-12T03:11:39.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.479+0000 7f024795a740 1 -- 172.21.15.116:0/1423950242 shutdown_connections 2024-09-12T03:11:39.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.479+0000 7f024795a740 1 --2- 172.21.15.116:0/1423950242 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5616ef06a3a0 0x5616ef06c790 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.479+0000 7f02267fc640 0 client.17574 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:39.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.479+0000 7f024795a740 1 --2- 172.21.15.116:0/1423950242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616eefef780 0x5616eefe94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.479+0000 7f024795a740 1 --2- 172.21.15.116:0/1423950242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5616eefe1b40 0x5616eefe8f80 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.479+0000 7f024795a740 1 --2- 172.21.15.116:0/1423950242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5616eefe05d0 0x5616eeff7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.479+0000 7f024795a740 1 -- 172.21.15.116:0/1423950242 >> 172.21.15.116:0/1423950242 conn(0x5616eefa9310 msgr2=0x5616eefdb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:39.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.479+0000 7f024795a740 1 -- 172.21.15.116:0/1423950242 shutdown_connections 2024-09-12T03:11:39.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.479+0000 7f024795a740 1 -- 172.21.15.116:0/1423950242 wait complete. 2024-09-12T03:11:39.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.479+0000 7f024795a740 20 client.17574 trim_cache size 0 max 16384 2024-09-12T03:11:39.485 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:11:39.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.480+0000 7f0245581640 1 -- 172.21.15.116:0/1980425874 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f022c0a2c00 msgr2=0x7f022c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:39.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.480+0000 7f0245581640 1 --2- 172.21.15.116:0/1980425874 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f022c0a2c00 0x7f022c0a50c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f02380bbfe0 tx=0x7f0238009260 comp rx=0 tx=0).stop 2024-09-12T03:11:39.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.480+0000 7f0245581640 1 -- 172.21.15.116:0/1980425874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02401082a0 msgr2=0x7f024019cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:39.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.480+0000 7f0245581640 1 --2- 172.21.15.116:0/1980425874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02401082a0 0x7f024019cef0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f0238005ab0 tx=0x7f02380042f0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.480+0000 7f0245581640 1 -- 172.21.15.116:0/1980425874 shutdown_connections 2024-09-12T03:11:39.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.480+0000 7f0245581640 1 --2- 172.21.15.116:0/1980425874 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f022c0a2c00 0x7f022c0a50c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.480+0000 7f0245581640 1 --2- 172.21.15.116:0/1980425874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02401082a0 0x7f024019cef0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.480+0000 7f0245581640 1 --2- 172.21.15.116:0/1980425874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02401036e0 0x7f0240198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.480+0000 7f0245581640 1 --2- 172.21.15.116:0/1980425874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0240102d30 0x7f0240198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.480+0000 7f0245581640 1 -- 172.21.15.116:0/1980425874 >> 172.21.15.116:0/1980425874 conn(0x7f02400fc7d0 msgr2=0x7f0240106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:39.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.480+0000 7f0245581640 1 -- 172.21.15.116:0/1980425874 shutdown_connections 2024-09-12T03:11:39.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:39.480+0000 7f0245581640 1 -- 172.21.15.116:0/1980425874 wait complete. 2024-09-12T03:11:40.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:40 smithi183 ceph-mon[35535]: pgmap v598: 97 pgs: 97 active+clean; 4.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.0 KiB/s wr, 8 op/s 2024-09-12T03:11:40.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:40 smithi177 ceph-mon[35305]: pgmap v598: 97 pgs: 97 active+clean; 4.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.0 KiB/s wr, 8 op/s 2024-09-12T03:11:40.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:40 smithi116 ceph-mon[31410]: pgmap v598: 97 pgs: 97 active+clean; 4.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.0 KiB/s wr, 8 op/s 2024-09-12T03:11:41.390 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:11:41.391 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:11:41.650 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:11:42.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.128+0000 7f7d3aed8640 1 -- 172.21.15.116:0/670879349 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d34102ef0 msgr2=0x7f7d3410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:42.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.128+0000 7f7d3aed8640 1 --2- 172.21.15.116:0/670879349 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d34102ef0 0x7f7d3410da40 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c009ca0 tx=0x7f7d2c02a950 comp rx=0 tx=0).stop 2024-09-12T03:11:42.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.128+0000 7f7d3aed8640 1 -- 172.21.15.116:0/670879349 shutdown_connections 2024-09-12T03:11:42.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.128+0000 7f7d3aed8640 1 --2- 172.21.15.116:0/670879349 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d34102ef0 0x7f7d3410da40 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.128+0000 7f7d3aed8640 1 --2- 172.21.15.116:0/670879349 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d34102570 0x7f7d341029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.128+0000 7f7d3aed8640 1 --2- 172.21.15.116:0/670879349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d34100e20 0x7f7d34101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.128+0000 7f7d3aed8640 1 -- 172.21.15.116:0/670879349 >> 172.21.15.116:0/670879349 conn(0x7f7d340fc7d0 msgr2=0x7f7d340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:42.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.128+0000 7f7d3aed8640 1 -- 172.21.15.116:0/670879349 shutdown_connections 2024-09-12T03:11:42.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.128+0000 7f7d3aed8640 1 -- 172.21.15.116:0/670879349 wait complete. 2024-09-12T03:11:42.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.128+0000 7f7d3aed8640 1 Processor -- start 2024-09-12T03:11:42.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.128+0000 7f7d3aed8640 1 -- start start 2024-09-12T03:11:42.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.128+0000 7f7d3aed8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d34100e20 0x7f7d341992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:42.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.128+0000 7f7d3aed8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d34102570 0x7f7d341997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:42.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.128+0000 7f7d3aed8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d34102ef0 0x7f7d341a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:42.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.128+0000 7f7d3aed8640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7d34110fa0 con 0x7f7d34100e20 2024-09-12T03:11:42.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.128+0000 7f7d3aed8640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7d34110e20 con 0x7f7d34102ef0 2024-09-12T03:11:42.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.129+0000 7f7d3aed8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d34111120 con 0x7f7d34102570 2024-09-12T03:11:42.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.129+0000 7f7d38c74640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d34100e20 0x7f7d341992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:42.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.129+0000 7f7d38c74640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d34100e20 0x7f7d341992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44936/0 (socket says 172.21.15.116:44936) 2024-09-12T03:11:42.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.129+0000 7f7d38c74640 1 -- 172.21.15.116:0/3032453007 learned_addr learned my addr 172.21.15.116:0/3032453007 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:42.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.129+0000 7f7d39475640 1 --2- 172.21.15.116:0/3032453007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d34102ef0 0x7f7d341a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:42.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.129+0000 7f7d38c74640 1 -- 172.21.15.116:0/3032453007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d34102570 msgr2=0x7f7d341997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:42.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.129+0000 7f7d38c74640 1 --2- 172.21.15.116:0/3032453007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d34102570 0x7f7d341997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.129+0000 7f7d38c74640 1 -- 172.21.15.116:0/3032453007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d34102ef0 msgr2=0x7f7d341a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:42.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.129+0000 7f7d38c74640 1 --2- 172.21.15.116:0/3032453007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d34102ef0 0x7f7d341a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.129+0000 7f7d38c74640 1 -- 172.21.15.116:0/3032453007 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d341a4870 con 0x7f7d34100e20 2024-09-12T03:11:42.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.129+0000 7f7d38c74640 1 --2- 172.21.15.116:0/3032453007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d34100e20 0x7f7d341992b0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f7d2400b4a0 tx=0x7f7d2400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:42.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.129+0000 7f7d31ffb640 1 -- 172.21.15.116:0/3032453007 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d24014070 con 0x7f7d34100e20 2024-09-12T03:11:42.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.129+0000 7f7d3aed8640 1 -- 172.21.15.116:0/3032453007 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d341a4a40 con 0x7f7d34100e20 2024-09-12T03:11:42.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.129+0000 7f7d31ffb640 1 -- 172.21.15.116:0/3032453007 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d24010af0 con 0x7f7d34100e20 2024-09-12T03:11:42.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.129+0000 7f7d31ffb640 1 -- 172.21.15.116:0/3032453007 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2401e060 con 0x7f7d34100e20 2024-09-12T03:11:42.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.129+0000 7f7d3aed8640 1 -- 172.21.15.116:0/3032453007 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d341a4f80 con 0x7f7d34100e20 2024-09-12T03:11:42.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.132+0000 7f7d31ffb640 1 -- 172.21.15.116:0/3032453007 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f7d24017070 con 0x7f7d34100e20 2024-09-12T03:11:42.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.132+0000 7f7d3d2b1740 1 Processor -- start 2024-09-12T03:11:42.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.132+0000 7f7d3d2b1740 1 -- start start 2024-09-12T03:11:42.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.133+0000 7f7d3d2b1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b21c639b60 0x55b21c639f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:42.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.133+0000 7f7d3d2b1740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b21c63a480 0x55b21c644800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:42.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.133+0000 7f7d3d2b1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b21c644d40 0x55b21c647130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:42.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.133+0000 7f7d3d2b1740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b21c59f280 con 0x55b21c63a480 2024-09-12T03:11:42.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.133+0000 7f7d3d2b1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b21c546b10 con 0x55b21c644d40 2024-09-12T03:11:42.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.133+0000 7f7d3d2b1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b21c59b240 con 0x55b21c639b60 2024-09-12T03:11:42.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.133+0000 7f7d38c74640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b21c63a480 0x55b21c644800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:42.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.133+0000 7f7d38c74640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b21c63a480 0x55b21c644800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44938/0 (socket says 172.21.15.116:44938) 2024-09-12T03:11:42.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.133+0000 7f7d38c74640 1 -- 172.21.15.116:0/191318596 learned_addr learned my addr 172.21.15.116:0/191318596 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:42.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.133+0000 7f7d31ffb640 1 --2- 172.21.15.116:0/3032453007 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7d1c0a2b80 0x7f7d1c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:42.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.133+0000 7f7d33fff640 1 --2- 172.21.15.116:0/191318596 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b21c644d40 0x55b21c647130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:42.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.133+0000 7f7d39475640 1 --2- 172.21.15.116:0/191318596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b21c639b60 0x55b21c639f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:42.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.133+0000 7f7d39475640 1 --2- 172.21.15.116:0/3032453007 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7d1c0a2b80 0x7f7d1c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:42.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.133+0000 7f7d31ffb640 1 -- 172.21.15.116:0/3032453007 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f7d240ce7c0 con 0x7f7d34100e20 2024-09-12T03:11:42.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.133+0000 7f7d38c74640 1 -- 172.21.15.116:0/191318596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b21c639b60 msgr2=0x55b21c639f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:42.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.133+0000 7f7d38c74640 1 --2- 172.21.15.116:0/191318596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b21c639b60 0x55b21c639f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.133+0000 7f7d38c74640 1 -- 172.21.15.116:0/191318596 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b21c644d40 msgr2=0x55b21c647130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:42.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.133+0000 7f7d38c74640 1 --2- 172.21.15.116:0/191318596 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b21c644d40 0x55b21c647130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.133+0000 7f7d38c74640 1 -- 172.21.15.116:0/191318596 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b21c5531f0 con 0x55b21c63a480 2024-09-12T03:11:42.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.133+0000 7f7d38c74640 1 --2- 172.21.15.116:0/191318596 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b21c63a480 0x55b21c644800 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f7d240974b0 tx=0x7f7d2400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f8521229c840a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:42.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.134+0000 7f7d39475640 1 --2- 172.21.15.116:0/3032453007 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7d1c0a2b80 0x7f7d1c0a5040 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c009a60 tx=0x7f7d2c0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:42.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.134+0000 7f7d1affd640 1 -- 172.21.15.116:0/191318596 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2409ab90 con 0x55b21c63a480 2024-09-12T03:11:42.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.134+0000 7f7d1affd640 1 -- 172.21.15.116:0/191318596 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d2409b750 con 0x55b21c63a480 2024-09-12T03:11:42.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.134+0000 7f7d1affd640 1 -- 172.21.15.116:0/191318596 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2409a9b0 con 0x55b21c63a480 2024-09-12T03:11:42.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.134+0000 7f7d3d2b1740 1 -- 172.21.15.116:0/191318596 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b21c63a480 msgr2=0x55b21c644800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:42.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.134+0000 7f7d3d2b1740 1 --2- 172.21.15.116:0/191318596 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b21c63a480 0x55b21c644800 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f7d240974b0 tx=0x7f7d2400bf90 comp rx=0 tx=0).stop 2024-09-12T03:11:42.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.134+0000 7f7d3d2b1740 1 -- 172.21.15.116:0/191318596 shutdown_connections 2024-09-12T03:11:42.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.134+0000 7f7d3d2b1740 1 --2- 172.21.15.116:0/191318596 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b21c644d40 0x55b21c647130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.134+0000 7f7d3d2b1740 1 --2- 172.21.15.116:0/191318596 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b21c63a480 0x55b21c644800 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.134+0000 7f7d3d2b1740 1 --2- 172.21.15.116:0/191318596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b21c639b60 0x55b21c639f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.134+0000 7f7d3d2b1740 1 -- 172.21.15.116:0/191318596 >> 172.21.15.116:0/191318596 conn(0x55b21c6360f0 msgr2=0x55b21c6364d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:42.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.134+0000 7f7d3d2b1740 1 -- 172.21.15.116:0/191318596 shutdown_connections 2024-09-12T03:11:42.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.134+0000 7f7d3d2b1740 1 -- 172.21.15.116:0/191318596 wait complete. 2024-09-12T03:11:42.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.135+0000 7f7d3d2b1740 1 -- start start 2024-09-12T03:11:42.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.135+0000 7f7d3d2b1740 1 Processor -- start 2024-09-12T03:11:42.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.135+0000 7f7d3d2b1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b21c639b60 0x55b21c641ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:42.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.135+0000 7f7d3d2b1740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b21c63a480 0x55b21c6420e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:42.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.135+0000 7f7d3d2b1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b21c644d40 0x55b21c642620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:42.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.135+0000 7f7d3d2b1740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b21c647910 con 0x55b21c63a480 2024-09-12T03:11:42.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.135+0000 7f7d3d2b1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b21c559f10 con 0x55b21c644d40 2024-09-12T03:11:42.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.135+0000 7f7d3d2b1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b21c647a90 con 0x55b21c639b60 2024-09-12T03:11:42.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.135+0000 7f7d3d2b1740 10 client.0 _ll_register_callbacks cb 0x55b21c522a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:42.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.135+0000 7f7d33fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b21c63a480 0x55b21c6420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:42.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.135+0000 7f7d33fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b21c63a480 0x55b21c6420e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44950/0 (socket says 172.21.15.116:44950) 2024-09-12T03:11:42.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.135+0000 7f7d33fff640 1 -- 172.21.15.116:0/1705116814 learned_addr learned my addr 172.21.15.116:0/1705116814 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:42.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.136+0000 7f7d39475640 1 --2- 172.21.15.116:0/1705116814 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b21c644d40 0x55b21c642620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:42.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.136+0000 7f7d38c74640 1 --2- 172.21.15.116:0/1705116814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b21c639b60 0x55b21c641ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:42.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.136+0000 7f7d33fff640 1 -- 172.21.15.116:0/1705116814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b21c639b60 msgr2=0x55b21c641ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:42.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.136+0000 7f7d33fff640 1 --2- 172.21.15.116:0/1705116814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b21c639b60 0x55b21c641ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.136+0000 7f7d33fff640 1 -- 172.21.15.116:0/1705116814 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b21c644d40 msgr2=0x55b21c642620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:42.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.136+0000 7f7d33fff640 1 --2- 172.21.15.116:0/1705116814 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b21c644d40 0x55b21c642620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.136+0000 7f7d33fff640 1 -- 172.21.15.116:0/1705116814 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b21c3106a0 con 0x55b21c63a480 2024-09-12T03:11:42.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.136+0000 7f7d38c74640 1 --2- 172.21.15.116:0/1705116814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b21c639b60 0x55b21c641ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:42.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.136+0000 7f7d33fff640 1 --2- 172.21.15.116:0/1705116814 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b21c63a480 0x55b21c6420e0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800f6b0 tx=0x7f7d2800fb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f8749df57ffac3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:42.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.136+0000 7f7d13fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:42.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.136+0000 7f7d13fff640 1 -- 172.21.15.116:0/1705116814 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d28004220 con 0x55b21c63a480 2024-09-12T03:11:42.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.136+0000 7f7d13fff640 1 -- 172.21.15.116:0/1705116814 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f7d280043c0 con 0x55b21c63a480 2024-09-12T03:11:42.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.136+0000 7f7d3d2b1740 1 -- 172.21.15.116:0/1705116814 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d2409b750 con 0x55b21c63a480 2024-09-12T03:11:42.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.136+0000 7f7d13fff640 1 -- 172.21.15.116:0/1705116814 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7d2800d920 con 0x55b21c63a480 2024-09-12T03:11:42.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.136+0000 7f7d3d2b1740 10 client.17586 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:42.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.136+0000 7f7d3d2b1740 1 -- 172.21.15.116:0/1705116814 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b21c653530 con 0x55b21c63a480 2024-09-12T03:11:42.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.136+0000 7f7d13fff640 1 -- 172.21.15.116:0/1705116814 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d2409b750 con 0x55b21c63a480 2024-09-12T03:11:42.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.136+0000 7f7d13fff640 1 -- 172.21.15.116:0/1705116814 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7d28024480 con 0x55b21c63a480 2024-09-12T03:11:42.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.136+0000 7f7d3d2b1740 10 client.17586 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:42.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.136+0000 7f7d3d2b1740 10 client.17586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:11:42.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.136+0000 7f7d3d2b1740 20 client.17586 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:42.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.137+0000 7f7d3d2b1740 1 --2- 172.21.15.116:0/1705116814 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b21c654940 0x55b21c656de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:42.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.137+0000 7f7d3d2b1740 4 client.17586 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:11:42.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.137+0000 7f7d3d2b1740 1 -- 172.21.15.116:0/1705116814 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b21c5ac7f0 con 0x55b21c654940 2024-09-12T03:11:42.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.137+0000 7f7d38c74640 1 --2- 172.21.15.116:0/1705116814 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b21c654940 0x55b21c656de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:42.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.137+0000 7f7d38c74640 1 --2- 172.21.15.116:0/1705116814 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b21c654940 0x55b21c656de0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9511f60d481c6cdb server_cookie=4b4bd1260ff23ad1 in_seq=0 out_seq=0 2024-09-12T03:11:42.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.137+0000 7f7d13fff640 10 client.17586 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:42.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.138+0000 7f7d13fff640 1 -- 172.21.15.116:0/1705116814 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b21c5ac7f0 con 0x55b21c654940 2024-09-12T03:11:42.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.138+0000 7f7d13fff640 10 client.17586 handle_command_reply: tid=0 2024-09-12T03:11:42.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.138+0000 7f7d3d2b1740 1 client.17586 shutdown 2024-09-12T03:11:42.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.138+0000 7f7d3d2b1740 1 -- 172.21.15.116:0/1705116814 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b21c63a480 msgr2=0x55b21c6420e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:42.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.138+0000 7f7d3d2b1740 1 --2- 172.21.15.116:0/1705116814 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b21c63a480 0x55b21c6420e0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800f6b0 tx=0x7f7d2800fb70 comp rx=0 tx=0).stop 2024-09-12T03:11:42.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.138+0000 7f7d3d2b1740 1 -- 172.21.15.116:0/1705116814 shutdown_connections 2024-09-12T03:11:42.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.138+0000 7f7d3d2b1740 1 --2- 172.21.15.116:0/1705116814 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b21c654940 0x55b21c656de0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.138+0000 7f7d3d2b1740 1 --2- 172.21.15.116:0/1705116814 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b21c644d40 0x55b21c642620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.138+0000 7f7d3d2b1740 1 --2- 172.21.15.116:0/1705116814 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b21c63a480 0x55b21c6420e0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.138+0000 7f7d13fff640 0 client.17586 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:42.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.138+0000 7f7d3d2b1740 1 --2- 172.21.15.116:0/1705116814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b21c639b60 0x55b21c641ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.138+0000 7f7d3d2b1740 1 -- 172.21.15.116:0/1705116814 >> 172.21.15.116:0/1705116814 conn(0x55b21c6360f0 msgr2=0x55b21c63bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:42.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.138+0000 7f7d3d2b1740 1 -- 172.21.15.116:0/1705116814 shutdown_connections 2024-09-12T03:11:42.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.138+0000 7f7d3d2b1740 1 -- 172.21.15.116:0/1705116814 wait complete. 2024-09-12T03:11:42.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.138+0000 7f7d3d2b1740 20 client.17586 trim_cache size 0 max 16384 2024-09-12T03:11:42.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.155+0000 7f7d3d2b1740 1 Processor -- start 2024-09-12T03:11:42.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.155+0000 7f7d3d2b1740 1 -- start start 2024-09-12T03:11:42.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.155+0000 7f7d3d2b1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b21c639b60 0x55b21c643a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:42.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.155+0000 7f7d3d2b1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b21c63a480 0x55b21c65c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:42.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.155+0000 7f7d3d2b1740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b21c644d40 0x55b21c65cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:42.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.156+0000 7f7d3d2b1740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7d341afdf0 con 0x55b21c644d40 2024-09-12T03:11:42.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.156+0000 7f7d3d2b1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7d341ad6c0 con 0x55b21c63a480 2024-09-12T03:11:42.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.156+0000 7f7d3d2b1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d341ab510 con 0x55b21c639b60 2024-09-12T03:11:42.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.156+0000 7f7d39475640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b21c644d40 0x55b21c65cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:42.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.156+0000 7f7d39475640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b21c644d40 0x55b21c65cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44954/0 (socket says 172.21.15.116:44954) 2024-09-12T03:11:42.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.156+0000 7f7d39475640 1 -- 172.21.15.116:0/2242325199 learned_addr learned my addr 172.21.15.116:0/2242325199 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:42.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.156+0000 7f7d33fff640 1 --2- 172.21.15.116:0/2242325199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b21c63a480 0x55b21c65c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:42.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.156+0000 7f7d38c74640 1 --2- 172.21.15.116:0/2242325199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b21c639b60 0x55b21c643a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:42.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.156+0000 7f7d39475640 1 -- 172.21.15.116:0/2242325199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b21c639b60 msgr2=0x55b21c643a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:42.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.156+0000 7f7d39475640 1 --2- 172.21.15.116:0/2242325199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b21c639b60 0x55b21c643a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.156+0000 7f7d39475640 1 -- 172.21.15.116:0/2242325199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b21c63a480 msgr2=0x55b21c65c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:42.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.156+0000 7f7d39475640 1 --2- 172.21.15.116:0/2242325199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b21c63a480 0x55b21c65c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.156+0000 7f7d39475640 1 -- 172.21.15.116:0/2242325199 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b21c635f20 con 0x55b21c644d40 2024-09-12T03:11:42.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.156+0000 7f7d38c74640 1 --2- 172.21.15.116:0/2242325199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b21c639b60 0x55b21c643a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:42.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.156+0000 7f7d39475640 1 --2- 172.21.15.116:0/2242325199 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b21c644d40 0x55b21c65cbf0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c005ef0 tx=0x7f7d2c042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6aff41178fd06e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:42.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.156+0000 7f7d1a7fc640 1 -- 172.21.15.116:0/2242325199 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c050070 con 0x55b21c644d40 2024-09-12T03:11:42.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.156+0000 7f7d1a7fc640 1 -- 172.21.15.116:0/2242325199 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d2c02f750 con 0x55b21c644d40 2024-09-12T03:11:42.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.156+0000 7f7d1a7fc640 1 -- 172.21.15.116:0/2242325199 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c034040 con 0x55b21c644d40 2024-09-12T03:11:42.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.157+0000 7f7d3d2b1740 1 -- 172.21.15.116:0/2242325199 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b21c644d40 msgr2=0x55b21c65cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:42.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.157+0000 7f7d3d2b1740 1 --2- 172.21.15.116:0/2242325199 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b21c644d40 0x55b21c65cbf0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c005ef0 tx=0x7f7d2c042000 comp rx=0 tx=0).stop 2024-09-12T03:11:42.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.157+0000 7f7d3d2b1740 1 -- 172.21.15.116:0/2242325199 shutdown_connections 2024-09-12T03:11:42.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.157+0000 7f7d3d2b1740 1 --2- 172.21.15.116:0/2242325199 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b21c644d40 0x55b21c65cbf0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.157+0000 7f7d3d2b1740 1 --2- 172.21.15.116:0/2242325199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b21c63a480 0x55b21c65c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.157+0000 7f7d3d2b1740 1 --2- 172.21.15.116:0/2242325199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b21c639b60 0x55b21c643a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.157+0000 7f7d3d2b1740 1 -- 172.21.15.116:0/2242325199 >> 172.21.15.116:0/2242325199 conn(0x55b21c6360f0 msgr2=0x55b21c6432f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:42.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.157+0000 7f7d3d2b1740 1 -- 172.21.15.116:0/2242325199 shutdown_connections 2024-09-12T03:11:42.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.157+0000 7f7d3d2b1740 1 -- 172.21.15.116:0/2242325199 wait complete. 2024-09-12T03:11:42.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.157+0000 7f7d3d2b1740 1 -- start start 2024-09-12T03:11:42.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.157+0000 7f7d3d2b1740 1 Processor -- start 2024-09-12T03:11:42.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.158+0000 7f7d3d2b1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b21c639b60 0x55b21c64cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:42.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.158+0000 7f7d3d2b1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b21c63a480 0x55b21c64d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:42.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.158+0000 7f7d3d2b1740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b21c644d40 0x55b21c647cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:42.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.158+0000 7f7d3d2b1740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7d341abb80 con 0x55b21c644d40 2024-09-12T03:11:42.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.158+0000 7f7d3d2b1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7d341acff0 con 0x55b21c639b60 2024-09-12T03:11:42.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.158+0000 7f7d3d2b1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d341ac250 con 0x55b21c63a480 2024-09-12T03:11:42.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.158+0000 7f7d3d2b1740 10 client.0 _ll_register_callbacks cb 0x7f7d2800a5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:42.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.158+0000 7f7d39475640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b21c644d40 0x55b21c647cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:42.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.158+0000 7f7d39475640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b21c644d40 0x55b21c647cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44968/0 (socket says 172.21.15.116:44968) 2024-09-12T03:11:42.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.158+0000 7f7d39475640 1 -- 172.21.15.116:0/262760025 learned_addr learned my addr 172.21.15.116:0/262760025 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:42.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.158+0000 7f7d38c74640 1 --2- 172.21.15.116:0/262760025 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b21c639b60 0x55b21c64cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:42.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.158+0000 7f7d39475640 1 -- 172.21.15.116:0/262760025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b21c63a480 msgr2=0x55b21c64d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:42.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.158+0000 7f7d39475640 1 --2- 172.21.15.116:0/262760025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b21c63a480 0x55b21c64d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.158+0000 7f7d39475640 1 -- 172.21.15.116:0/262760025 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b21c639b60 msgr2=0x55b21c64cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.158+0000 7f7d39475640 1 --2- 172.21.15.116:0/262760025 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b21c639b60 0x55b21c64cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.158+0000 7f7d39475640 1 -- 172.21.15.116:0/262760025 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b21c64d8d0 con 0x55b21c644d40 2024-09-12T03:11:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.158+0000 7f7d39475640 1 --2- 172.21.15.116:0/262760025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b21c644d40 0x55b21c647cd0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c006010 tx=0x7f7d2c02fe60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb22fd74dfdde67 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.158+0000 7f7d13fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.159+0000 7f7d13fff640 1 -- 172.21.15.116:0/262760025 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c050040 con 0x55b21c644d40 2024-09-12T03:11:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.159+0000 7f7d13fff640 1 -- 172.21.15.116:0/262760025 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f7d2c04c9c0 con 0x55b21c644d40 2024-09-12T03:11:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.159+0000 7f7d3d2b1740 1 -- 172.21.15.116:0/262760025 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d2c02f750 con 0x55b21c644d40 2024-09-12T03:11:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.159+0000 7f7d13fff640 1 -- 172.21.15.116:0/262760025 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7d2c04cbc0 con 0x55b21c644d40 2024-09-12T03:11:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.159+0000 7f7d3d2b1740 10 client.17592 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.159+0000 7f7d3d2b1740 1 -- 172.21.15.116:0/262760025 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b21c6bfbc0 con 0x55b21c644d40 2024-09-12T03:11:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.159+0000 7f7d13fff640 1 -- 172.21.15.116:0/262760025 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d2c02f750 con 0x55b21c644d40 2024-09-12T03:11:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.159+0000 7f7d13fff640 1 -- 172.21.15.116:0/262760025 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7d2c0614e0 con 0x55b21c644d40 2024-09-12T03:11:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.159+0000 7f7d3d2b1740 10 client.17592 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:42.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.159+0000 7f7d3d2b1740 10 client.17592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:11:42.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.159+0000 7f7d3d2b1740 20 client.17592 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:42.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.159+0000 7f7d3d2b1740 1 --2- 172.21.15.116:0/262760025 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b21c6c0230 0x55b21c6c2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:42.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.159+0000 7f7d3d2b1740 4 client.17592 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:11:42.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.159+0000 7f7d3d2b1740 1 -- 172.21.15.116:0/262760025 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b21c5a6d00 con 0x55b21c6c0230 2024-09-12T03:11:42.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.160+0000 7f7d38c74640 1 --2- 172.21.15.116:0/262760025 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b21c6c0230 0x55b21c6c2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:42.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.160+0000 7f7d38c74640 1 --2- 172.21.15.116:0/262760025 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b21c6c0230 0x55b21c6c2620 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=144ca22f952586b4 server_cookie=c04813d2f6fdc811 in_seq=0 out_seq=0 2024-09-12T03:11:42.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.160+0000 7f7d13fff640 10 client.17592 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:42.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.161+0000 7f7d13fff640 1 -- 172.21.15.116:0/262760025 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b21c5a6d00 con 0x55b21c6c0230 2024-09-12T03:11:42.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.161+0000 7f7d13fff640 10 client.17592 handle_command_reply: tid=0 2024-09-12T03:11:42.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.161+0000 7f7d3d2b1740 1 client.17592 shutdown 2024-09-12T03:11:42.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.161+0000 7f7d3d2b1740 1 -- 172.21.15.116:0/262760025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b21c644d40 msgr2=0x55b21c647cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:42.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.161+0000 7f7d3d2b1740 1 --2- 172.21.15.116:0/262760025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b21c644d40 0x55b21c647cd0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c006010 tx=0x7f7d2c02fe60 comp rx=0 tx=0).stop 2024-09-12T03:11:42.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.161+0000 7f7d3d2b1740 1 -- 172.21.15.116:0/262760025 shutdown_connections 2024-09-12T03:11:42.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.161+0000 7f7d3d2b1740 1 --2- 172.21.15.116:0/262760025 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b21c6c0230 0x55b21c6c2620 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.161+0000 7f7d3d2b1740 1 --2- 172.21.15.116:0/262760025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b21c644d40 0x55b21c647cd0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.161+0000 7f7d3d2b1740 1 --2- 172.21.15.116:0/262760025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b21c63a480 0x55b21c64d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.161+0000 7f7d3d2b1740 1 --2- 172.21.15.116:0/262760025 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b21c639b60 0x55b21c64cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.161+0000 7f7d3d2b1740 1 -- 172.21.15.116:0/262760025 >> 172.21.15.116:0/262760025 conn(0x55b21c6360f0 msgr2=0x55b21c635270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:42.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.161+0000 7f7d13fff640 0 client.17592 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:42.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.161+0000 7f7d3d2b1740 1 -- 172.21.15.116:0/262760025 shutdown_connections 2024-09-12T03:11:42.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.161+0000 7f7d3d2b1740 1 -- 172.21.15.116:0/262760025 wait complete. 2024-09-12T03:11:42.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.161+0000 7f7d3d2b1740 20 client.17592 trim_cache size 0 max 16384 2024-09-12T03:11:42.179 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:11:42.179 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:11:42.180 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "ec81d4ff-1e54-4979-b2f7-d118b07c46d7", 2024-09-12T03:11:42.180 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:11:42.180 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:11:42.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.162+0000 7f7d3aed8640 1 -- 172.21.15.116:0/3032453007 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7d1c0a2b80 msgr2=0x7f7d1c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:42.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.162+0000 7f7d3aed8640 1 --2- 172.21.15.116:0/3032453007 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7d1c0a2b80 0x7f7d1c0a5040 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c009a60 tx=0x7f7d2c0095c0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.162+0000 7f7d3aed8640 1 -- 172.21.15.116:0/3032453007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d34100e20 msgr2=0x7f7d341992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:42.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.162+0000 7f7d3aed8640 1 --2- 172.21.15.116:0/3032453007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d34100e20 0x7f7d341992b0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f7d2400b4a0 tx=0x7f7d2400b960 comp rx=0 tx=0).stop 2024-09-12T03:11:42.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.162+0000 7f7d3aed8640 1 -- 172.21.15.116:0/3032453007 shutdown_connections 2024-09-12T03:11:42.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.162+0000 7f7d3aed8640 1 --2- 172.21.15.116:0/3032453007 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7d1c0a2b80 0x7f7d1c0a5040 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.162+0000 7f7d3aed8640 1 --2- 172.21.15.116:0/3032453007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d34102ef0 0x7f7d341a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.162+0000 7f7d3aed8640 1 --2- 172.21.15.116:0/3032453007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d34102570 0x7f7d341997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.162+0000 7f7d3aed8640 1 --2- 172.21.15.116:0/3032453007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d34100e20 0x7f7d341992b0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:42.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.162+0000 7f7d3aed8640 1 -- 172.21.15.116:0/3032453007 >> 172.21.15.116:0/3032453007 conn(0x7f7d340fc7d0 msgr2=0x7f7d34106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:42.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.162+0000 7f7d3aed8640 1 -- 172.21.15.116:0/3032453007 shutdown_connections 2024-09-12T03:11:42.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:42.162+0000 7f7d3aed8640 1 -- 172.21.15.116:0/3032453007 wait complete. 2024-09-12T03:11:42.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:42 smithi183 ceph-mon[35535]: pgmap v599: 97 pgs: 97 active+clean; 4.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-12T03:11:42.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:42 smithi177 ceph-mon[35305]: pgmap v599: 97 pgs: 97 active+clean; 4.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-12T03:11:42.795 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:42 smithi116 ceph-mon[31410]: pgmap v599: 97 pgs: 97 active+clean; 4.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-12T03:11:42.989 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:11:43.236 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:11:43.639 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:43 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:11:43.639 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:43 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:11:43.639 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:43 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:11:43.639 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:43 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:11:43.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:43 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:11:43.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:43 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:11:43.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:43 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:11:43.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:43 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:11:43.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:43 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:11:43.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:43 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:11:43.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:43 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:11:43.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:43 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:11:43.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.719+0000 7fd1f11d9640 1 -- 172.21.15.116:0/3935658082 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1ec102570 msgr2=0x7fd1ec1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:43.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.719+0000 7fd1f11d9640 1 --2- 172.21.15.116:0/3935658082 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1ec102570 0x7fd1ec1029b0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009900 tx=0x7fd1dc02a480 comp rx=0 tx=0).stop 2024-09-12T03:11:43.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.719+0000 7fd1f11d9640 1 -- 172.21.15.116:0/3935658082 shutdown_connections 2024-09-12T03:11:43.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.719+0000 7fd1f11d9640 1 --2- 172.21.15.116:0/3935658082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1ec102ef0 0x7fd1ec10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.719+0000 7fd1f11d9640 1 --2- 172.21.15.116:0/3935658082 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1ec102570 0x7fd1ec1029b0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.719+0000 7fd1f11d9640 1 --2- 172.21.15.116:0/3935658082 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1ec100e20 0x7fd1ec101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.719+0000 7fd1f11d9640 1 -- 172.21.15.116:0/3935658082 >> 172.21.15.116:0/3935658082 conn(0x7fd1ec0fc7d0 msgr2=0x7fd1ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:43.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.719+0000 7fd1f11d9640 1 -- 172.21.15.116:0/3935658082 shutdown_connections 2024-09-12T03:11:43.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.719+0000 7fd1f11d9640 1 -- 172.21.15.116:0/3935658082 wait complete. 2024-09-12T03:11:43.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.720+0000 7fd1f11d9640 1 Processor -- start 2024-09-12T03:11:43.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.721+0000 7fd1f11d9640 1 -- start start 2024-09-12T03:11:43.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.721+0000 7fd1f11d9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1ec100e20 0x7fd1ec1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:43.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.721+0000 7fd1f11d9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1ec102570 0x7fd1ec198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:43.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.721+0000 7fd1f11d9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1ec102ef0 0x7fd1ec19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:43.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.721+0000 7fd1f11d9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec110f50 con 0x7fd1ec102ef0 2024-09-12T03:11:43.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.721+0000 7fd1f11d9640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec110dd0 con 0x7fd1ec100e20 2024-09-12T03:11:43.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.721+0000 7fd1f11d9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec1110d0 con 0x7fd1ec102570 2024-09-12T03:11:43.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.721+0000 7fd1eb59e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1ec102ef0 0x7fd1ec19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:43.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.721+0000 7fd1eb59e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1ec102ef0 0x7fd1ec19cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44988/0 (socket says 172.21.15.116:44988) 2024-09-12T03:11:43.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.721+0000 7fd1eb59e640 1 -- 172.21.15.116:0/3790344443 learned_addr learned my addr 172.21.15.116:0/3790344443 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:43.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.721+0000 7fd1ead9d640 1 --2- 172.21.15.116:0/3790344443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1ec100e20 0x7fd1ec1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:43.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.721+0000 7fd1ea59c640 1 --2- 172.21.15.116:0/3790344443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1ec102570 0x7fd1ec198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:43.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.721+0000 7fd1eb59e640 1 -- 172.21.15.116:0/3790344443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1ec102570 msgr2=0x7fd1ec198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:43.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.721+0000 7fd1eb59e640 1 --2- 172.21.15.116:0/3790344443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1ec102570 0x7fd1ec198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.721+0000 7fd1eb59e640 1 -- 172.21.15.116:0/3790344443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1ec100e20 msgr2=0x7fd1ec1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:43.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.721+0000 7fd1eb59e640 1 --2- 172.21.15.116:0/3790344443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1ec100e20 0x7fd1ec1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.721+0000 7fd1eb59e640 1 -- 172.21.15.116:0/3790344443 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1ec19d4a0 con 0x7fd1ec102ef0 2024-09-12T03:11:43.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.721+0000 7fd1eb59e640 1 --2- 172.21.15.116:0/3790344443 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1ec102ef0 0x7fd1ec19cdc0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fd1e400cad0 tx=0x7fd1e400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:43.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.722+0000 7fd1d3fff640 1 -- 172.21.15.116:0/3790344443 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e400ad60 con 0x7fd1ec102ef0 2024-09-12T03:11:43.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.722+0000 7fd1d3fff640 1 -- 172.21.15.116:0/3790344443 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1e4005020 con 0x7fd1ec102ef0 2024-09-12T03:11:43.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.722+0000 7fd1d3fff640 1 -- 172.21.15.116:0/3790344443 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e400f6b0 con 0x7fd1ec102ef0 2024-09-12T03:11:43.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.722+0000 7fd1f11d9640 1 -- 172.21.15.116:0/3790344443 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1ec19d790 con 0x7fd1ec102ef0 2024-09-12T03:11:43.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.722+0000 7fd1f11d9640 1 -- 172.21.15.116:0/3790344443 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1ec1a5070 con 0x7fd1ec102ef0 2024-09-12T03:11:43.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.724+0000 7fd1d3fff640 1 -- 172.21.15.116:0/3790344443 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd1e4005560 con 0x7fd1ec102ef0 2024-09-12T03:11:43.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.724+0000 7fd1f35b2740 1 Processor -- start 2024-09-12T03:11:43.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.724+0000 7fd1f35b2740 1 -- start start 2024-09-12T03:11:43.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.724+0000 7fd1f35b2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559ac5a3f4a0 0x559ac5a3f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:43.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.724+0000 7fd1f35b2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ac5a3fdc0 0x559ac5a4a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:43.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.725+0000 7fd1f35b2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac5a4a680 0x559ac5a4ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:43.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.725+0000 7fd1f35b2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559ac59a52e0 con 0x559ac5a3f4a0 2024-09-12T03:11:43.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.725+0000 7fd1f35b2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559ac594cce0 con 0x559ac5a4a680 2024-09-12T03:11:43.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.725+0000 7fd1f35b2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559ac59a12a0 con 0x559ac5a3fdc0 2024-09-12T03:11:43.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.725+0000 7fd1ea59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559ac5a3f4a0 0x559ac5a3f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:43.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.725+0000 7fd1ea59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559ac5a3f4a0 0x559ac5a3f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44990/0 (socket says 172.21.15.116:44990) 2024-09-12T03:11:43.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.725+0000 7fd1ea59c640 1 -- 172.21.15.116:0/1744779853 learned_addr learned my addr 172.21.15.116:0/1744779853 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:43.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.725+0000 7fd1d3fff640 1 --2- 172.21.15.116:0/3790344443 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd1d40a2b80 0x7fd1d40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:43.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.725+0000 7fd1ea59c640 1 --2- 172.21.15.116:0/1744779853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac5a4a680 0x559ac5a4ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:43.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.725+0000 7fd1ead9d640 1 --2- 172.21.15.116:0/1744779853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ac5a3fdc0 0x559ac5a4a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:43.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.725+0000 7fd1ea59c640 1 -- 172.21.15.116:0/1744779853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ac5a3fdc0 msgr2=0x559ac5a4a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:43.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.725+0000 7fd1eb59e640 1 --2- 172.21.15.116:0/3790344443 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd1d40a2b80 0x7fd1d40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:43.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.725+0000 7fd1ea59c640 1 --2- 172.21.15.116:0/1744779853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ac5a3fdc0 0x559ac5a4a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.725+0000 7fd1ea59c640 1 -- 172.21.15.116:0/1744779853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac5a4a680 msgr2=0x559ac5a4ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:43.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.725+0000 7fd1ea59c640 1 --2- 172.21.15.116:0/1744779853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac5a4a680 0x559ac5a4ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.725+0000 7fd1ea59c640 1 -- 172.21.15.116:0/1744779853 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ac5931440 con 0x559ac5a3f4a0 2024-09-12T03:11:43.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.725+0000 7fd1d3fff640 1 -- 172.21.15.116:0/3790344443 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd1e40ceff0 con 0x7fd1ec102ef0 2024-09-12T03:11:43.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.725+0000 7fd1ea59c640 1 --2- 172.21.15.116:0/1744779853 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559ac5a3f4a0 0x559ac5a3f880 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc02a9a0 tx=0x7fd1dc005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbbbc63f5abee98 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:43.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.725+0000 7fd1d17fa640 1 -- 172.21.15.116:0/1744779853 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc035070 con 0x559ac5a3f4a0 2024-09-12T03:11:43.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.725+0000 7fd1eb59e640 1 --2- 172.21.15.116:0/3790344443 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd1d40a2b80 0x7fd1d40a5040 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40a3f80 tx=0x7fd1e406e900 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:43.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.725+0000 7fd1d17fa640 1 -- 172.21.15.116:0/1744779853 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1dc003d50 con 0x559ac5a3f4a0 2024-09-12T03:11:43.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.725+0000 7fd1d17fa640 1 -- 172.21.15.116:0/1744779853 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc03b460 con 0x559ac5a3f4a0 2024-09-12T03:11:43.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.726+0000 7fd1f35b2740 1 -- 172.21.15.116:0/1744779853 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559ac5a3f4a0 msgr2=0x559ac5a3f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:43.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.726+0000 7fd1f35b2740 1 --2- 172.21.15.116:0/1744779853 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559ac5a3f4a0 0x559ac5a3f880 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc02a9a0 tx=0x7fd1dc005c00 comp rx=0 tx=0).stop 2024-09-12T03:11:43.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.726+0000 7fd1f35b2740 1 -- 172.21.15.116:0/1744779853 shutdown_connections 2024-09-12T03:11:43.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.726+0000 7fd1f35b2740 1 --2- 172.21.15.116:0/1744779853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac5a4a680 0x559ac5a4ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.726+0000 7fd1f35b2740 1 --2- 172.21.15.116:0/1744779853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ac5a3fdc0 0x559ac5a4a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.726+0000 7fd1f35b2740 1 --2- 172.21.15.116:0/1744779853 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559ac5a3f4a0 0x559ac5a3f880 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.726+0000 7fd1f35b2740 1 -- 172.21.15.116:0/1744779853 >> 172.21.15.116:0/1744779853 conn(0x559ac5a3ba30 msgr2=0x559ac5a3be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:43.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.726+0000 7fd1f35b2740 1 -- 172.21.15.116:0/1744779853 shutdown_connections 2024-09-12T03:11:43.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.726+0000 7fd1f35b2740 1 -- 172.21.15.116:0/1744779853 wait complete. 2024-09-12T03:11:43.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.727+0000 7fd1f35b2740 1 -- start start 2024-09-12T03:11:43.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.727+0000 7fd1f35b2740 1 Processor -- start 2024-09-12T03:11:43.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.727+0000 7fd1f35b2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559ac5a3f4a0 0x559ac5a49fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:43.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.727+0000 7fd1f35b2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ac5a3fdc0 0x559ac5a47280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:43.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.727+0000 7fd1f35b2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac5a4a680 0x559ac5a477e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:43.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.727+0000 7fd1f35b2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559ac5a4d250 con 0x559ac5a3f4a0 2024-09-12T03:11:43.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.727+0000 7fd1f35b2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559ac5960050 con 0x559ac5a4a680 2024-09-12T03:11:43.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.727+0000 7fd1f35b2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559ac5a4d3d0 con 0x559ac5a3fdc0 2024-09-12T03:11:43.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.727+0000 7fd1f35b2740 10 client.0 _ll_register_callbacks cb 0x559ac59a0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:43.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.727+0000 7fd1ea59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559ac5a3f4a0 0x559ac5a49fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:43.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.727+0000 7fd1ea59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559ac5a3f4a0 0x559ac5a49fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44998/0 (socket says 172.21.15.116:44998) 2024-09-12T03:11:43.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.727+0000 7fd1ea59c640 1 -- 172.21.15.116:0/1543292885 learned_addr learned my addr 172.21.15.116:0/1543292885 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:43.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.727+0000 7fd1ead9d640 1 --2- 172.21.15.116:0/1543292885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ac5a3fdc0 0x559ac5a47280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:43.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.727+0000 7fd1ea59c640 1 --2- 172.21.15.116:0/1543292885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac5a4a680 0x559ac5a477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:43.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.727+0000 7fd1ea59c640 1 -- 172.21.15.116:0/1543292885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ac5a3fdc0 msgr2=0x559ac5a47280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:43.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.727+0000 7fd1ea59c640 1 --2- 172.21.15.116:0/1543292885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ac5a3fdc0 0x559ac5a47280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.727+0000 7fd1ea59c640 1 -- 172.21.15.116:0/1543292885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac5a4a680 msgr2=0x559ac5a477e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:43.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.727+0000 7fd1ea59c640 1 --2- 172.21.15.116:0/1543292885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac5a4a680 0x559ac5a477e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.728+0000 7fd1ea59c640 1 -- 172.21.15.116:0/1543292885 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ac57163c0 con 0x559ac5a3f4a0 2024-09-12T03:11:43.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.728+0000 7fd1ea59c640 1 --2- 172.21.15.116:0/1543292885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559ac5a3f4a0 0x559ac5a49fd0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0096c0 tx=0x7fd1dc031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f405758d54bf2fd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:43.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.728+0000 7fd1ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:43.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.728+0000 7fd1ceff5640 1 -- 172.21.15.116:0/1543292885 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc035030 con 0x559ac5a3f4a0 2024-09-12T03:11:43.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.728+0000 7fd1ceff5640 1 -- 172.21.15.116:0/1543292885 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd1dc003a80 con 0x559ac5a3f4a0 2024-09-12T03:11:43.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.728+0000 7fd1f35b2740 1 -- 172.21.15.116:0/1543292885 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1dc003d50 con 0x559ac5a3f4a0 2024-09-12T03:11:43.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.728+0000 7fd1ceff5640 1 -- 172.21.15.116:0/1543292885 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd1dc05b460 con 0x559ac5a3f4a0 2024-09-12T03:11:43.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.728+0000 7fd1f35b2740 10 client.17604 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:43.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.728+0000 7fd1f35b2740 1 -- 172.21.15.116:0/1543292885 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559ac5a4d250 con 0x559ac5a3f4a0 2024-09-12T03:11:43.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.728+0000 7fd1ceff5640 1 -- 172.21.15.116:0/1543292885 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1dc003d50 con 0x559ac5a3f4a0 2024-09-12T03:11:43.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.728+0000 7fd1ceff5640 1 -- 172.21.15.116:0/1543292885 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd1dc0494e0 con 0x559ac5a3f4a0 2024-09-12T03:11:43.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.729+0000 7fd1f35b2740 10 client.17604 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:43.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.729+0000 7fd1f35b2740 10 client.17604 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:11:43.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.729+0000 7fd1f35b2740 20 client.17604 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:43.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.729+0000 7fd1f35b2740 1 --2- 172.21.15.116:0/1543292885 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559ac5a59da0 0x559ac5a5c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:43.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.729+0000 7fd1f35b2740 4 client.17604 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:11:43.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.729+0000 7fd1f35b2740 1 -- 172.21.15.116:0/1543292885 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ac59b2850 con 0x559ac5a59da0 2024-09-12T03:11:43.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.729+0000 7fd1ead9d640 1 --2- 172.21.15.116:0/1543292885 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559ac5a59da0 0x559ac5a5c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:43.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.729+0000 7fd1ead9d640 1 --2- 172.21.15.116:0/1543292885 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559ac5a59da0 0x559ac5a5c190 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9797ee669120148 server_cookie=9b7999b72ad1c091 in_seq=0 out_seq=0 2024-09-12T03:11:43.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.729+0000 7fd1ceff5640 10 client.17604 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:43.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.730+0000 7fd1ceff5640 1 -- 172.21.15.116:0/1543292885 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559ac59b2850 con 0x559ac5a59da0 2024-09-12T03:11:43.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.730+0000 7fd1ceff5640 10 client.17604 handle_command_reply: tid=0 2024-09-12T03:11:43.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.730+0000 7fd1f35b2740 1 client.17604 shutdown 2024-09-12T03:11:43.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.730+0000 7fd1f35b2740 1 -- 172.21.15.116:0/1543292885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559ac5a3f4a0 msgr2=0x559ac5a49fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:43.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.730+0000 7fd1f35b2740 1 --2- 172.21.15.116:0/1543292885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559ac5a3f4a0 0x559ac5a49fd0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0096c0 tx=0x7fd1dc031b10 comp rx=0 tx=0).stop 2024-09-12T03:11:43.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.730+0000 7fd1f35b2740 1 -- 172.21.15.116:0/1543292885 shutdown_connections 2024-09-12T03:11:43.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.730+0000 7fd1f35b2740 1 --2- 172.21.15.116:0/1543292885 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559ac5a59da0 0x559ac5a5c190 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.730+0000 7fd1f35b2740 1 --2- 172.21.15.116:0/1543292885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac5a4a680 0x559ac5a477e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.730+0000 7fd1f35b2740 1 --2- 172.21.15.116:0/1543292885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ac5a3fdc0 0x559ac5a47280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.730+0000 7fd1ceff5640 0 client.17604 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:43.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.730+0000 7fd1f35b2740 1 --2- 172.21.15.116:0/1543292885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559ac5a3f4a0 0x559ac5a49fd0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.730+0000 7fd1f35b2740 1 -- 172.21.15.116:0/1543292885 >> 172.21.15.116:0/1543292885 conn(0x559ac5a3ba30 msgr2=0x559ac5a41760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:43.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.730+0000 7fd1f35b2740 1 -- 172.21.15.116:0/1543292885 shutdown_connections 2024-09-12T03:11:43.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.730+0000 7fd1f35b2740 1 -- 172.21.15.116:0/1543292885 wait complete. 2024-09-12T03:11:43.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.730+0000 7fd1f35b2740 20 client.17604 trim_cache size 0 max 16384 2024-09-12T03:11:43.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.747+0000 7fd1f35b2740 1 Processor -- start 2024-09-12T03:11:43.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.747+0000 7fd1f35b2740 1 -- start start 2024-09-12T03:11:43.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.747+0000 7fd1f35b2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ac5a3f4a0 0x559ac5a61880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:43.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.747+0000 7fd1f35b2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac5a3fdc0 0x559ac5a61dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:43.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.747+0000 7fd1f35b2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559ac5a4a680 0x559ac5a444e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:43.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.747+0000 7fd1f35b2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec1afc20 con 0x559ac5a4a680 2024-09-12T03:11:43.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.747+0000 7fd1f35b2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec1ad4f0 con 0x559ac5a3fdc0 2024-09-12T03:11:43.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.747+0000 7fd1f35b2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec1ab340 con 0x559ac5a3f4a0 2024-09-12T03:11:43.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.747+0000 7fd1ea59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559ac5a4a680 0x559ac5a444e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:43.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.747+0000 7fd1ea59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559ac5a4a680 0x559ac5a444e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45004/0 (socket says 172.21.15.116:45004) 2024-09-12T03:11:43.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.747+0000 7fd1ea59c640 1 -- 172.21.15.116:0/2743727186 learned_addr learned my addr 172.21.15.116:0/2743727186 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:43.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.748+0000 7fd1ea59c640 1 --2- 172.21.15.116:0/2743727186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ac5a3f4a0 0x559ac5a61880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:43.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.748+0000 7fd1ead9d640 1 --2- 172.21.15.116:0/2743727186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac5a3fdc0 0x559ac5a61dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:43.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.748+0000 7fd1ea59c640 1 -- 172.21.15.116:0/2743727186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ac5a3f4a0 msgr2=0x559ac5a61880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:43.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.748+0000 7fd1ea59c640 1 --2- 172.21.15.116:0/2743727186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ac5a3f4a0 0x559ac5a61880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.748+0000 7fd1ea59c640 1 -- 172.21.15.116:0/2743727186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac5a3fdc0 msgr2=0x559ac5a61dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:43.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.748+0000 7fd1ea59c640 1 --2- 172.21.15.116:0/2743727186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac5a3fdc0 0x559ac5a61dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.748+0000 7fd1ea59c640 1 -- 172.21.15.116:0/2743727186 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ac5a3cae0 con 0x559ac5a4a680 2024-09-12T03:11:43.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.748+0000 7fd1ead9d640 1 --2- 172.21.15.116:0/2743727186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac5a3fdc0 0x559ac5a61dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:43.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.748+0000 7fd1ea59c640 1 --2- 172.21.15.116:0/2743727186 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559ac5a4a680 0x559ac5a444e0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc004460 tx=0x7fd1dc040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95434be4154c008e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:43.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.748+0000 7fd1d0ff9640 1 -- 172.21.15.116:0/2743727186 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc035070 con 0x559ac5a4a680 2024-09-12T03:11:43.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.748+0000 7fd1d0ff9640 1 -- 172.21.15.116:0/2743727186 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1dc054070 con 0x559ac5a4a680 2024-09-12T03:11:43.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.748+0000 7fd1d0ff9640 1 -- 172.21.15.116:0/2743727186 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc05b600 con 0x559ac5a4a680 2024-09-12T03:11:43.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.749+0000 7fd1f35b2740 1 -- 172.21.15.116:0/2743727186 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559ac5a4a680 msgr2=0x559ac5a444e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:43.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.749+0000 7fd1f35b2740 1 --2- 172.21.15.116:0/2743727186 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559ac5a4a680 0x559ac5a444e0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc004460 tx=0x7fd1dc040040 comp rx=0 tx=0).stop 2024-09-12T03:11:43.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.749+0000 7fd1f35b2740 1 -- 172.21.15.116:0/2743727186 shutdown_connections 2024-09-12T03:11:43.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.749+0000 7fd1f35b2740 1 --2- 172.21.15.116:0/2743727186 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559ac5a4a680 0x559ac5a444e0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.749+0000 7fd1f35b2740 1 --2- 172.21.15.116:0/2743727186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac5a3fdc0 0x559ac5a61dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.749+0000 7fd1f35b2740 1 --2- 172.21.15.116:0/2743727186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ac5a3f4a0 0x559ac5a61880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.749+0000 7fd1f35b2740 1 -- 172.21.15.116:0/2743727186 >> 172.21.15.116:0/2743727186 conn(0x559ac5a3ba30 msgr2=0x559ac5a3ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:43.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.749+0000 7fd1f35b2740 1 -- 172.21.15.116:0/2743727186 shutdown_connections 2024-09-12T03:11:43.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.749+0000 7fd1f35b2740 1 -- 172.21.15.116:0/2743727186 wait complete. 2024-09-12T03:11:43.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.749+0000 7fd1f35b2740 1 -- start start 2024-09-12T03:11:43.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.749+0000 7fd1f35b2740 1 Processor -- start 2024-09-12T03:11:43.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.749+0000 7fd1f35b2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559ac5a3f4a0 0x559ac5a52c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:43.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.749+0000 7fd1f35b2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac5a3fdc0 0x559ac5a4ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:43.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.749+0000 7fd1f35b2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ac5a4a680 0x559ac5a504d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:43.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.749+0000 7fd1f35b2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec1ab9b0 con 0x559ac5a3f4a0 2024-09-12T03:11:43.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.749+0000 7fd1f35b2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec1ace20 con 0x559ac5a3fdc0 2024-09-12T03:11:43.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.749+0000 7fd1f35b2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec1ac080 con 0x559ac5a4a680 2024-09-12T03:11:43.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.750+0000 7fd1f35b2740 10 client.0 _ll_register_callbacks cb 0x7fd1dc004490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:43.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.750+0000 7fd1ea59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559ac5a3f4a0 0x559ac5a52c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:43.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.750+0000 7fd1ea59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559ac5a3f4a0 0x559ac5a52c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45012/0 (socket says 172.21.15.116:45012) 2024-09-12T03:11:43.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.750+0000 7fd1ea59c640 1 -- 172.21.15.116:0/2657393699 learned_addr learned my addr 172.21.15.116:0/2657393699 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:43.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.750+0000 7fd1ead9d640 1 --2- 172.21.15.116:0/2657393699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac5a3fdc0 0x559ac5a4ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:43.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.750+0000 7fd1ea59c640 1 -- 172.21.15.116:0/2657393699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ac5a4a680 msgr2=0x559ac5a504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:43.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.750+0000 7fd1ea59c640 1 --2- 172.21.15.116:0/2657393699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ac5a4a680 0x559ac5a504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.750+0000 7fd1ea59c640 1 -- 172.21.15.116:0/2657393699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac5a3fdc0 msgr2=0x559ac5a4ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:43.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.750+0000 7fd1ea59c640 1 --2- 172.21.15.116:0/2657393699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac5a3fdc0 0x559ac5a4ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.750+0000 7fd1ea59c640 1 -- 172.21.15.116:0/2657393699 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ac5a50bc0 con 0x559ac5a3f4a0 2024-09-12T03:11:43.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.750+0000 7fd1ea59c640 1 --2- 172.21.15.116:0/2657393699 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559ac5a3f4a0 0x559ac5a52c40 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009900 tx=0x7fd1dc0497e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=babf466b5c562a52 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:43.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.750+0000 7fd1ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:43.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.750+0000 7fd1ceff5640 1 -- 172.21.15.116:0/2657393699 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc06b060 con 0x559ac5a3f4a0 2024-09-12T03:11:43.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.750+0000 7fd1ceff5640 1 -- 172.21.15.116:0/2657393699 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd1dc003a80 con 0x559ac5a3f4a0 2024-09-12T03:11:43.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.750+0000 7fd1f35b2740 1 -- 172.21.15.116:0/2657393699 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1dc054070 con 0x559ac5a3f4a0 2024-09-12T03:11:43.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.750+0000 7fd1ceff5640 1 -- 172.21.15.116:0/2657393699 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd1dc035040 con 0x559ac5a3f4a0 2024-09-12T03:11:43.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.750+0000 7fd1f35b2740 10 client.17610 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:43.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.750+0000 7fd1f35b2740 1 -- 172.21.15.116:0/2657393699 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559ac5ac46a0 con 0x559ac5a3f4a0 2024-09-12T03:11:43.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.751+0000 7fd1ceff5640 1 -- 172.21.15.116:0/2657393699 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1dc054070 con 0x559ac5a3f4a0 2024-09-12T03:11:43.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.751+0000 7fd1ceff5640 1 -- 172.21.15.116:0/2657393699 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd1dc066470 con 0x559ac5a3f4a0 2024-09-12T03:11:43.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.751+0000 7fd1f35b2740 10 client.17610 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:43.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.751+0000 7fd1f35b2740 10 client.17610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:11:43.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.751+0000 7fd1f35b2740 20 client.17610 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:43.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.751+0000 7fd1f35b2740 1 --2- 172.21.15.116:0/2657393699 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559ac5ac4b20 0x559ac5ac6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:43.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.751+0000 7fd1f35b2740 4 client.17610 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:11:43.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.751+0000 7fd1f35b2740 1 -- 172.21.15.116:0/2657393699 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559ac59acd60 con 0x559ac5ac4b20 2024-09-12T03:11:43.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.751+0000 7fd1ead9d640 1 --2- 172.21.15.116:0/2657393699 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559ac5ac4b20 0x559ac5ac6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:43.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.751+0000 7fd1ceff5640 10 client.17610 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:43.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.751+0000 7fd1ead9d640 1 --2- 172.21.15.116:0/2657393699 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559ac5ac4b20 0x559ac5ac6f10 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bd78a9138389be6 server_cookie=f7844095394afbbe in_seq=0 out_seq=0 2024-09-12T03:11:43.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.752+0000 7fd1ceff5640 1 -- 172.21.15.116:0/2657393699 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559ac59acd60 con 0x559ac5ac4b20 2024-09-12T03:11:43.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.752+0000 7fd1ceff5640 10 client.17610 handle_command_reply: tid=0 2024-09-12T03:11:43.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.752+0000 7fd1f35b2740 1 client.17610 shutdown 2024-09-12T03:11:43.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.752+0000 7fd1f35b2740 1 -- 172.21.15.116:0/2657393699 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559ac5a3f4a0 msgr2=0x559ac5a52c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:43.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.752+0000 7fd1f35b2740 1 --2- 172.21.15.116:0/2657393699 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559ac5a3f4a0 0x559ac5a52c40 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009900 tx=0x7fd1dc0497e0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.752+0000 7fd1f35b2740 1 -- 172.21.15.116:0/2657393699 shutdown_connections 2024-09-12T03:11:43.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.752+0000 7fd1f35b2740 1 --2- 172.21.15.116:0/2657393699 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559ac5ac4b20 0x559ac5ac6f10 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.752+0000 7fd1f35b2740 1 --2- 172.21.15.116:0/2657393699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ac5a4a680 0x559ac5a504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.752+0000 7fd1f35b2740 1 --2- 172.21.15.116:0/2657393699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559ac5a3fdc0 0x559ac5a4ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.752+0000 7fd1f35b2740 1 --2- 172.21.15.116:0/2657393699 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559ac5a3f4a0 0x559ac5a52c40 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.752+0000 7fd1ceff5640 0 client.17610 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:43.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.752+0000 7fd1f35b2740 1 -- 172.21.15.116:0/2657393699 >> 172.21.15.116:0/2657393699 conn(0x559ac5a3ba30 msgr2=0x559ac5a3a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:43.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.752+0000 7fd1f35b2740 1 -- 172.21.15.116:0/2657393699 shutdown_connections 2024-09-12T03:11:43.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.752+0000 7fd1f35b2740 1 -- 172.21.15.116:0/2657393699 wait complete. 2024-09-12T03:11:43.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.752+0000 7fd1f35b2740 20 client.17610 trim_cache size 0 max 16384 2024-09-12T03:11:43.766 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:11:43.766 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:11:43.766 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:11:43.766 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:11:43.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.752+0000 7fd1f11d9640 1 -- 172.21.15.116:0/3790344443 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd1d40a2b80 msgr2=0x7fd1d40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:43.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.752+0000 7fd1f11d9640 1 --2- 172.21.15.116:0/3790344443 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd1d40a2b80 0x7fd1d40a5040 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40a3f80 tx=0x7fd1e406e900 comp rx=0 tx=0).stop 2024-09-12T03:11:43.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.752+0000 7fd1f11d9640 1 -- 172.21.15.116:0/3790344443 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1ec102ef0 msgr2=0x7fd1ec19cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:43.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.752+0000 7fd1f11d9640 1 --2- 172.21.15.116:0/3790344443 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1ec102ef0 0x7fd1ec19cdc0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fd1e400cad0 tx=0x7fd1e400ced0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.752+0000 7fd1f11d9640 1 -- 172.21.15.116:0/3790344443 shutdown_connections 2024-09-12T03:11:43.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.752+0000 7fd1f11d9640 1 --2- 172.21.15.116:0/3790344443 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd1d40a2b80 0x7fd1d40a5040 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.752+0000 7fd1f11d9640 1 --2- 172.21.15.116:0/3790344443 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1ec102ef0 0x7fd1ec19cdc0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.752+0000 7fd1f11d9640 1 --2- 172.21.15.116:0/3790344443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1ec102570 0x7fd1ec198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.752+0000 7fd1f11d9640 1 --2- 172.21.15.116:0/3790344443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1ec100e20 0x7fd1ec1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:43.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.752+0000 7fd1f11d9640 1 -- 172.21.15.116:0/3790344443 >> 172.21.15.116:0/3790344443 conn(0x7fd1ec0fc7d0 msgr2=0x7fd1ec06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:43.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.753+0000 7fd1f11d9640 1 -- 172.21.15.116:0/3790344443 shutdown_connections 2024-09-12T03:11:43.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:43.753+0000 7fd1f11d9640 1 -- 172.21.15.116:0/3790344443 wait complete. 2024-09-12T03:11:44.502 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:44 smithi116 ceph-mon[31410]: pgmap v600: 97 pgs: 97 active+clean; 4.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:11:44.502 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:44 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:11:44.532 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:11:44.532 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:11:44.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:44 smithi183 ceph-mon[35535]: pgmap v600: 97 pgs: 97 active+clean; 4.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:11:44.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:44 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:11:44.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:44 smithi177 ceph-mon[35305]: pgmap v600: 97 pgs: 97 active+clean; 4.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:11:44.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:44 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:11:44.777 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:11:45.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.251+0000 7ffae175a640 1 -- 172.21.15.116:0/1003869827 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffadc10c170 msgr2=0x7ffadc10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:45.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.251+0000 7ffae175a640 1 --2- 172.21.15.116:0/1003869827 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffadc10c170 0x7ffadc10e610 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ffad4009ca0 tx=0x7ffad402a950 comp rx=0 tx=0).stop 2024-09-12T03:11:45.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.251+0000 7ffae175a640 1 -- 172.21.15.116:0/1003869827 shutdown_connections 2024-09-12T03:11:45.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.251+0000 7ffae175a640 1 --2- 172.21.15.116:0/1003869827 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffadc10c170 0x7ffadc10e610 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:45.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.251+0000 7ffae175a640 1 --2- 172.21.15.116:0/1003869827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffadc103660 0x7ffadc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:45.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.251+0000 7ffae175a640 1 --2- 172.21.15.116:0/1003869827 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffadc102d40 0x7ffadc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:45.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.251+0000 7ffae175a640 1 -- 172.21.15.116:0/1003869827 >> 172.21.15.116:0/1003869827 conn(0x7ffadc0fc7a0 msgr2=0x7ffadc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:45.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.251+0000 7ffae175a640 1 -- 172.21.15.116:0/1003869827 shutdown_connections 2024-09-12T03:11:45.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.251+0000 7ffae175a640 1 -- 172.21.15.116:0/1003869827 wait complete. 2024-09-12T03:11:45.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.252+0000 7ffae175a640 1 Processor -- start 2024-09-12T03:11:45.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.252+0000 7ffae175a640 1 -- start start 2024-09-12T03:11:45.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.252+0000 7ffae175a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffadc102d40 0x7ffadc198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:45.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.252+0000 7ffae175a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffadc103660 0x7ffadc198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:45.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.252+0000 7ffae175a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffadc10c170 0x7ffadc19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:45.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.252+0000 7ffae175a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ffadc111f70 con 0x7ffadc10c170 2024-09-12T03:11:45.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.252+0000 7ffae175a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ffadc111df0 con 0x7ffadc103660 2024-09-12T03:11:45.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.252+0000 7ffae175a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffadc1120f0 con 0x7ffadc102d40 2024-09-12T03:11:45.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.252+0000 7ffadb7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffadc10c170 0x7ffadc19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:45.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.252+0000 7ffadb7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffadc10c170 0x7ffadc19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45020/0 (socket says 172.21.15.116:45020) 2024-09-12T03:11:45.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.252+0000 7ffadb7fe640 1 -- 172.21.15.116:0/2133484043 learned_addr learned my addr 172.21.15.116:0/2133484043 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:45.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.252+0000 7ffadaffd640 1 --2- 172.21.15.116:0/2133484043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffadc102d40 0x7ffadc198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:45.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.252+0000 7ffada7fc640 1 --2- 172.21.15.116:0/2133484043 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffadc103660 0x7ffadc198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:45.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.253+0000 7ffadb7fe640 1 -- 172.21.15.116:0/2133484043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffadc102d40 msgr2=0x7ffadc198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:45.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.253+0000 7ffadb7fe640 1 --2- 172.21.15.116:0/2133484043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffadc102d40 0x7ffadc198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:45.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.253+0000 7ffadb7fe640 1 -- 172.21.15.116:0/2133484043 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffadc103660 msgr2=0x7ffadc198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:45.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.253+0000 7ffadb7fe640 1 --2- 172.21.15.116:0/2133484043 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffadc103660 0x7ffadc198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:45.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.253+0000 7ffadb7fe640 1 -- 172.21.15.116:0/2133484043 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffadc19d400 con 0x7ffadc10c170 2024-09-12T03:11:45.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.253+0000 7ffadb7fe640 1 --2- 172.21.15.116:0/2133484043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffadc10c170 0x7ffadc19cd20 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7ffad402ae70 tx=0x7ffad402f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:45.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.253+0000 7ffac3fff640 1 -- 172.21.15.116:0/2133484043 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad4034040 con 0x7ffadc10c170 2024-09-12T03:11:45.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.253+0000 7ffac3fff640 1 -- 172.21.15.116:0/2133484043 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffad4030920 con 0x7ffadc10c170 2024-09-12T03:11:45.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.253+0000 7ffae175a640 1 -- 172.21.15.116:0/2133484043 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffadc19d690 con 0x7ffadc10c170 2024-09-12T03:11:45.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.253+0000 7ffac3fff640 1 -- 172.21.15.116:0/2133484043 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad4045480 con 0x7ffadc10c170 2024-09-12T03:11:45.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.253+0000 7ffae175a640 1 -- 172.21.15.116:0/2133484043 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffadc1a4fd0 con 0x7ffadc10c170 2024-09-12T03:11:45.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.254+0000 7ffae175a640 1 -- 172.21.15.116:0/2133484043 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffadc06b070 con 0x7ffadc10c170 2024-09-12T03:11:45.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.255+0000 7ffac3fff640 1 -- 172.21.15.116:0/2133484043 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ffad4037020 con 0x7ffadc10c170 2024-09-12T03:11:45.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.256+0000 7ffac3fff640 1 --2- 172.21.15.116:0/2133484043 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ffac40a2b80 0x7ffac40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:45.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.256+0000 7ffadaffd640 1 --2- 172.21.15.116:0/2133484043 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ffac40a2b80 0x7ffac40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:45.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.256+0000 7ffac3fff640 1 -- 172.21.15.116:0/2133484043 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ffad40efec0 con 0x7ffadc10c170 2024-09-12T03:11:45.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.256+0000 7ffadaffd640 1 --2- 172.21.15.116:0/2133484043 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ffac40a2b80 0x7ffac40a5040 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ffacc00bbe0 tx=0x7ffacc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:45.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.260+0000 7ffac3fff640 1 -- 172.21.15.116:0/2133484043 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffad40bce50 con 0x7ffadc10c170 2024-09-12T03:11:45.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.464+0000 7ffae175a640 1 -- 172.21.15.116:0/2133484043 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffadc1071e0 con 0x7ffadc10c170 2024-09-12T03:11:45.466 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:45 smithi116 ceph-mon[31410]: pgmap v601: 97 pgs: 97 active+clean; 4.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:11:45.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.465+0000 7ffac3fff640 1 -- 172.21.15.116:0/2133484043 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7ffad40bc750 con 0x7ffadc10c170 2024-09-12T03:11:45.466 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:11:45.467 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:11:45.467 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:11:45.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.467+0000 7ffae175a640 1 -- 172.21.15.116:0/2133484043 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ffac40a2b80 msgr2=0x7ffac40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:45.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.467+0000 7ffae175a640 1 --2- 172.21.15.116:0/2133484043 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ffac40a2b80 0x7ffac40a5040 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ffacc00bbe0 tx=0x7ffacc00b390 comp rx=0 tx=0).stop 2024-09-12T03:11:45.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.467+0000 7ffae175a640 1 -- 172.21.15.116:0/2133484043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffadc10c170 msgr2=0x7ffadc19cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:45.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.467+0000 7ffae175a640 1 --2- 172.21.15.116:0/2133484043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffadc10c170 0x7ffadc19cd20 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7ffad402ae70 tx=0x7ffad402f5b0 comp rx=0 tx=0).stop 2024-09-12T03:11:45.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.467+0000 7ffae175a640 1 -- 172.21.15.116:0/2133484043 shutdown_connections 2024-09-12T03:11:45.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.467+0000 7ffae175a640 1 --2- 172.21.15.116:0/2133484043 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ffac40a2b80 0x7ffac40a5040 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:45.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.467+0000 7ffae175a640 1 --2- 172.21.15.116:0/2133484043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffadc10c170 0x7ffadc19cd20 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:45.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.467+0000 7ffae175a640 1 --2- 172.21.15.116:0/2133484043 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffadc103660 0x7ffadc198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:45.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.467+0000 7ffae175a640 1 --2- 172.21.15.116:0/2133484043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffadc102d40 0x7ffadc198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:45.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.467+0000 7ffae175a640 1 -- 172.21.15.116:0/2133484043 >> 172.21.15.116:0/2133484043 conn(0x7ffadc0fc7a0 msgr2=0x7ffadc0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:45.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.467+0000 7ffae175a640 1 -- 172.21.15.116:0/2133484043 shutdown_connections 2024-09-12T03:11:45.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:45.467+0000 7ffae175a640 1 -- 172.21.15.116:0/2133484043 wait complete. 2024-09-12T03:11:45.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:45 smithi183 ceph-mon[35535]: pgmap v601: 97 pgs: 97 active+clean; 4.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:11:45.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:45 smithi177 ceph-mon[35305]: pgmap v601: 97 pgs: 97 active+clean; 4.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:11:46.087 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:11:46.335 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:11:46.616 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:46 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2133484043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:11:46.648 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:46 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2133484043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:11:46.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:46 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2133484043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:11:46.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.802+0000 7f0401f16640 1 -- 172.21.15.116:0/2577005413 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03fc106a80 msgr2=0x7f03fc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:46.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.802+0000 7f0401f16640 1 --2- 172.21.15.116:0/2577005413 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03fc106a80 0x7f03fc106e60 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009cd0 tx=0x7f03ec02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.802+0000 7f0401f16640 1 -- 172.21.15.116:0/2577005413 shutdown_connections 2024-09-12T03:11:46.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.802+0000 7f0401f16640 1 --2- 172.21.15.116:0/2577005413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03fc101420 0x7f03fc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.802+0000 7f0401f16640 1 --2- 172.21.15.116:0/2577005413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03fc100a80 0x7f03fc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.802+0000 7f0401f16640 1 --2- 172.21.15.116:0/2577005413 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03fc106a80 0x7f03fc106e60 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.802+0000 7f0401f16640 1 -- 172.21.15.116:0/2577005413 >> 172.21.15.116:0/2577005413 conn(0x7f03fc0fc7a0 msgr2=0x7f03fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:46.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.802+0000 7f0401f16640 1 -- 172.21.15.116:0/2577005413 shutdown_connections 2024-09-12T03:11:46.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.803+0000 7f0401f16640 1 -- 172.21.15.116:0/2577005413 wait complete. 2024-09-12T03:11:46.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.803+0000 7f0401f16640 1 Processor -- start 2024-09-12T03:11:46.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.803+0000 7f0401f16640 1 -- start start 2024-09-12T03:11:46.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.803+0000 7f0401f16640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03fc100a80 0x7f03fc1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:46.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.803+0000 7f0401f16640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03fc101420 0x7f03fc199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:46.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.803+0000 7f0401f16640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03fc106a80 0x7f03fc199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:46.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.803+0000 7f0401f16640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc110e80 con 0x7f03fc106a80 2024-09-12T03:11:46.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.803+0000 7f0401f16640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc110d00 con 0x7f03fc101420 2024-09-12T03:11:46.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.803+0000 7f0401f16640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc111000 con 0x7f03fc100a80 2024-09-12T03:11:46.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.804+0000 7f03fbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03fc106a80 0x7f03fc199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:46.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.804+0000 7f03fbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03fc106a80 0x7f03fc199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45042/0 (socket says 172.21.15.116:45042) 2024-09-12T03:11:46.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.804+0000 7f03fbfff640 1 -- 172.21.15.116:0/2323393666 learned_addr learned my addr 172.21.15.116:0/2323393666 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:46.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.804+0000 7f03faffd640 1 --2- 172.21.15.116:0/2323393666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03fc101420 0x7f03fc199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:46.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.804+0000 7f03fb7fe640 1 --2- 172.21.15.116:0/2323393666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03fc100a80 0x7f03fc1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:46.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.804+0000 7f03fbfff640 1 -- 172.21.15.116:0/2323393666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03fc100a80 msgr2=0x7f03fc1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:46.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.804+0000 7f03fbfff640 1 --2- 172.21.15.116:0/2323393666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03fc100a80 0x7f03fc1992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.804+0000 7f03fbfff640 1 -- 172.21.15.116:0/2323393666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03fc101420 msgr2=0x7f03fc199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:46.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.804+0000 7f03fbfff640 1 --2- 172.21.15.116:0/2323393666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03fc101420 0x7f03fc199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.804+0000 7f03fbfff640 1 -- 172.21.15.116:0/2323393666 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03fc1a46f0 con 0x7f03fc106a80 2024-09-12T03:11:46.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.804+0000 7f03fbfff640 1 --2- 172.21.15.116:0/2323393666 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03fc106a80 0x7f03fc199db0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f03f400c970 tx=0x7f03f400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:46.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.804+0000 7f03f8ff9640 1 -- 172.21.15.116:0/2323393666 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4007bb0 con 0x7f03fc106a80 2024-09-12T03:11:46.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.804+0000 7f0401f16640 1 -- 172.21.15.116:0/2323393666 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03fc1a49e0 con 0x7f03fc106a80 2024-09-12T03:11:46.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.804+0000 7f03f8ff9640 1 -- 172.21.15.116:0/2323393666 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03f4007d50 con 0x7f03fc106a80 2024-09-12T03:11:46.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.804+0000 7f03f8ff9640 1 -- 172.21.15.116:0/2323393666 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4018770 con 0x7f03fc106a80 2024-09-12T03:11:46.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.804+0000 7f0401f16640 1 -- 172.21.15.116:0/2323393666 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03fc1a4f20 con 0x7f03fc106a80 2024-09-12T03:11:46.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.807+0000 7f03f8ff9640 1 -- 172.21.15.116:0/2323393666 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f03f400f450 con 0x7f03fc106a80 2024-09-12T03:11:46.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.807+0000 7f04042ef740 1 Processor -- start 2024-09-12T03:11:46.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.807+0000 7f04042ef740 1 -- start start 2024-09-12T03:11:46.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.807+0000 7f04042ef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fbb15b40 0x55a7fbb14090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:46.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.807+0000 7f04042ef740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7fbb145d0 0x55a7fbb14a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:46.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.807+0000 7f04042ef740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7fbb23780 0x55a7fbb25b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:46.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.807+0000 7f04042ef740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a7fba7a2f0 con 0x55a7fbb145d0 2024-09-12T03:11:46.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.807+0000 7f04042ef740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a7fba21cf0 con 0x55a7fbb23780 2024-09-12T03:11:46.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.807+0000 7f04042ef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a7fba762b0 con 0x55a7fbb15b40 2024-09-12T03:11:46.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.807+0000 7f03fb7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7fbb145d0 0x55a7fbb14a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:46.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.807+0000 7f03f8ff9640 1 --2- 172.21.15.116:0/2323393666 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f03e40a2b80 0x7f03e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:46.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.807+0000 7f03fb7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7fbb145d0 0x55a7fbb14a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45056/0 (socket says 172.21.15.116:45056) 2024-09-12T03:11:46.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.807+0000 7f03fb7fe640 1 -- 172.21.15.116:0/2541039269 learned_addr learned my addr 172.21.15.116:0/2541039269 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:46.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.808+0000 7f03fbfff640 1 --2- 172.21.15.116:0/2323393666 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f03e40a2b80 0x7f03e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:46.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.808+0000 7f03faffd640 1 --2- 172.21.15.116:0/2541039269 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7fbb23780 0x55a7fbb25b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:46.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.808+0000 7f03faffd640 1 --2- 172.21.15.116:0/2541039269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fbb15b40 0x55a7fbb14090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:46.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.808+0000 7f03f8ff9640 1 -- 172.21.15.116:0/2323393666 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f03f40cef30 con 0x7f03fc106a80 2024-09-12T03:11:46.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.808+0000 7f03fb7fe640 1 -- 172.21.15.116:0/2541039269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fbb15b40 msgr2=0x55a7fbb14090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:46.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.808+0000 7f03fb7fe640 1 --2- 172.21.15.116:0/2541039269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fbb15b40 0x55a7fbb14090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.808+0000 7f03fb7fe640 1 -- 172.21.15.116:0/2541039269 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7fbb23780 msgr2=0x55a7fbb25b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:46.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.808+0000 7f03fb7fe640 1 --2- 172.21.15.116:0/2541039269 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7fbb23780 0x55a7fbb25b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.808+0000 7f03fb7fe640 1 -- 172.21.15.116:0/2541039269 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7fba063a0 con 0x55a7fbb145d0 2024-09-12T03:11:46.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.808+0000 7f03faffd640 1 --2- 172.21.15.116:0/2541039269 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7fbb23780 0x55a7fbb25b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:11:46.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.808+0000 7f03fb7fe640 1 --2- 172.21.15.116:0/2541039269 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7fbb145d0 0x55a7fbb14a10 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009cd0 tx=0x7f03ec031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b20f58a952d12155 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:46.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.808+0000 7f03fbfff640 1 --2- 172.21.15.116:0/2323393666 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f03e40a2b80 0x7f03e40a5040 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f03f400ffd0 tx=0x7f03f4066830 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:46.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.808+0000 7f03e1ffb640 1 -- 172.21.15.116:0/2541039269 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec048070 con 0x55a7fbb145d0 2024-09-12T03:11:46.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.808+0000 7f03e1ffb640 1 -- 172.21.15.116:0/2541039269 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03ec036070 con 0x55a7fbb145d0 2024-09-12T03:11:46.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.808+0000 7f03e1ffb640 1 -- 172.21.15.116:0/2541039269 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec004670 con 0x55a7fbb145d0 2024-09-12T03:11:46.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.809+0000 7f04042ef740 1 -- 172.21.15.116:0/2541039269 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7fbb145d0 msgr2=0x55a7fbb14a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:46.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.809+0000 7f04042ef740 1 --2- 172.21.15.116:0/2541039269 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7fbb145d0 0x55a7fbb14a10 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009cd0 tx=0x7f03ec031700 comp rx=0 tx=0).stop 2024-09-12T03:11:46.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.809+0000 7f04042ef740 1 -- 172.21.15.116:0/2541039269 shutdown_connections 2024-09-12T03:11:46.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.809+0000 7f04042ef740 1 --2- 172.21.15.116:0/2541039269 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7fbb23780 0x55a7fbb25b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.809+0000 7f04042ef740 1 --2- 172.21.15.116:0/2541039269 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7fbb145d0 0x55a7fbb14a10 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.809+0000 7f04042ef740 1 --2- 172.21.15.116:0/2541039269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fbb15b40 0x55a7fbb14090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.809+0000 7f04042ef740 1 -- 172.21.15.116:0/2541039269 >> 172.21.15.116:0/2541039269 conn(0x55a7fbadd310 msgr2=0x55a7fbb10c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:46.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.809+0000 7f04042ef740 1 -- 172.21.15.116:0/2541039269 shutdown_connections 2024-09-12T03:11:46.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.809+0000 7f04042ef740 1 -- 172.21.15.116:0/2541039269 wait complete. 2024-09-12T03:11:46.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.809+0000 7f04042ef740 1 -- start start 2024-09-12T03:11:46.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.809+0000 7f04042ef740 1 Processor -- start 2024-09-12T03:11:46.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.810+0000 7f04042ef740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7fbb145d0 0x55a7fbb23180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:46.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.810+0000 7f04042ef740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7fbb15b40 0x55a7fbb20430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:46.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.810+0000 7f04042ef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fbb23780 0x55a7fbb20a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:46.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.810+0000 7f04042ef740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a7fbb26420 con 0x55a7fbb145d0 2024-09-12T03:11:46.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.810+0000 7f04042ef740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a7fba35060 con 0x55a7fbb15b40 2024-09-12T03:11:46.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.810+0000 7f04042ef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a7fbb265a0 con 0x55a7fbb23780 2024-09-12T03:11:46.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.810+0000 7f04042ef740 10 client.0 _ll_register_callbacks cb 0x55a7fb7888c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:46.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.810+0000 7f03faffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7fbb145d0 0x55a7fbb23180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:46.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.810+0000 7f03faffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7fbb145d0 0x55a7fbb23180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45060/0 (socket says 172.21.15.116:45060) 2024-09-12T03:11:46.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.810+0000 7f03faffd640 1 -- 172.21.15.116:0/935086221 learned_addr learned my addr 172.21.15.116:0/935086221 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:46.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.810+0000 7f03faffd640 1 --2- 172.21.15.116:0/935086221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fbb23780 0x55a7fbb20a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:46.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.810+0000 7f03fb7fe640 1 --2- 172.21.15.116:0/935086221 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7fbb15b40 0x55a7fbb20430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:46.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.810+0000 7f03faffd640 1 -- 172.21.15.116:0/935086221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fbb23780 msgr2=0x55a7fbb20a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:46.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.810+0000 7f03faffd640 1 --2- 172.21.15.116:0/935086221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fbb23780 0x55a7fbb20a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.810+0000 7f03faffd640 1 -- 172.21.15.116:0/935086221 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7fbb15b40 msgr2=0x55a7fbb20430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:46.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.810+0000 7f03faffd640 1 --2- 172.21.15.116:0/935086221 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7fbb15b40 0x55a7fbb20430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.810+0000 7f03faffd640 1 -- 172.21.15.116:0/935086221 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7fb7eb3c0 con 0x55a7fbb145d0 2024-09-12T03:11:46.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.810+0000 7f03faffd640 1 --2- 172.21.15.116:0/935086221 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7fbb145d0 0x55a7fbb23180 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f03f000af40 tx=0x7f03f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dabae41e5cc4fd2b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:46.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.810+0000 7f03daffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:46.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.810+0000 7f03daffd640 1 -- 172.21.15.116:0/935086221 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f000fe90 con 0x55a7fbb145d0 2024-09-12T03:11:46.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.810+0000 7f03daffd640 1 -- 172.21.15.116:0/935086221 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f03f0005020 con 0x55a7fbb145d0 2024-09-12T03:11:46.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.810+0000 7f04042ef740 1 -- 172.21.15.116:0/935086221 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03ec036070 con 0x55a7fbb145d0 2024-09-12T03:11:46.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.811+0000 7f03daffd640 1 -- 172.21.15.116:0/935086221 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f03f0026070 con 0x55a7fbb145d0 2024-09-12T03:11:46.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.811+0000 7f04042ef740 10 client.17628 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:46.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.811+0000 7f04042ef740 1 -- 172.21.15.116:0/935086221 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a7fbb31f50 con 0x55a7fbb145d0 2024-09-12T03:11:46.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.811+0000 7f03daffd640 1 -- 172.21.15.116:0/935086221 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03ec036070 con 0x55a7fbb145d0 2024-09-12T03:11:46.835 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.811+0000 7f03daffd640 1 -- 172.21.15.116:0/935086221 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f03f001b520 con 0x55a7fbb145d0 2024-09-12T03:11:46.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.811+0000 7f04042ef740 10 client.17628 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:46.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.811+0000 7f04042ef740 10 client.17628 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:11:46.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.811+0000 7f04042ef740 20 client.17628 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:46.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.811+0000 7f04042ef740 1 --2- 172.21.15.116:0/935086221 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55a7fbb33410 0x55a7fbb35800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:46.836 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.811+0000 7f04042ef740 4 client.17628 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:11:46.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.811+0000 7f04042ef740 1 -- 172.21.15.116:0/935086221 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7fba87860 con 0x55a7fbb33410 2024-09-12T03:11:46.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.811+0000 7f03fb7fe640 1 --2- 172.21.15.116:0/935086221 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55a7fbb33410 0x55a7fbb35800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:46.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.812+0000 7f03daffd640 10 client.17628 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:11:46.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.812+0000 7f03fb7fe640 1 --2- 172.21.15.116:0/935086221 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55a7fbb33410 0x55a7fbb35800 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f92841f01f64f2bb server_cookie=b8710b8e299a0d13 in_seq=0 out_seq=0 2024-09-12T03:11:46.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.812+0000 7f03daffd640 1 -- 172.21.15.116:0/935086221 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a7fba87860 con 0x55a7fbb33410 2024-09-12T03:11:46.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.812+0000 7f03daffd640 10 client.17628 handle_command_reply: tid=0 2024-09-12T03:11:46.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.812+0000 7f04042ef740 1 client.17628 shutdown 2024-09-12T03:11:46.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.813+0000 7f04042ef740 1 -- 172.21.15.116:0/935086221 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7fbb145d0 msgr2=0x55a7fbb23180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:46.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.813+0000 7f04042ef740 1 --2- 172.21.15.116:0/935086221 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7fbb145d0 0x55a7fbb23180 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f03f000af40 tx=0x7f03f00046c0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.813+0000 7f04042ef740 1 -- 172.21.15.116:0/935086221 shutdown_connections 2024-09-12T03:11:46.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.813+0000 7f04042ef740 1 --2- 172.21.15.116:0/935086221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fbb23780 0x55a7fbb20a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.813+0000 7f04042ef740 1 --2- 172.21.15.116:0/935086221 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7fbb15b40 0x55a7fbb20430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.813+0000 7f04042ef740 1 --2- 172.21.15.116:0/935086221 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55a7fbb33410 0x55a7fbb35800 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.813+0000 7f04042ef740 1 --2- 172.21.15.116:0/935086221 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7fbb145d0 0x55a7fbb23180 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.813+0000 7f03daffd640 0 client.17628 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:11:46.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.813+0000 7f04042ef740 1 -- 172.21.15.116:0/935086221 >> 172.21.15.116:0/935086221 conn(0x55a7fbadd310 msgr2=0x55a7fbb16930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:46.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.813+0000 7f04042ef740 1 -- 172.21.15.116:0/935086221 shutdown_connections 2024-09-12T03:11:46.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.813+0000 7f04042ef740 1 -- 172.21.15.116:0/935086221 wait complete. 2024-09-12T03:11:46.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.813+0000 7f04042ef740 20 client.17628 trim_cache size 0 max 16384 2024-09-12T03:11:46.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.826+0000 7f04042ef740 1 Processor -- start 2024-09-12T03:11:46.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.826+0000 7f04042ef740 1 -- start start 2024-09-12T03:11:46.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.826+0000 7f04042ef740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7fbb145d0 0x55a7fbb10e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:46.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.826+0000 7f04042ef740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7fbb15b40 0x55a7fbb2f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:46.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.826+0000 7f04042ef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fbb23780 0x55a7fbb1fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:46.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.826+0000 7f04042ef740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc1ada10 con 0x55a7fbb145d0 2024-09-12T03:11:46.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.826+0000 7f04042ef740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc1ab2e0 con 0x55a7fbb15b40 2024-09-12T03:11:46.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.826+0000 7f04042ef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc1a9130 con 0x55a7fbb23780 2024-09-12T03:11:46.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.826+0000 7f03faffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7fbb145d0 0x55a7fbb10e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:46.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.826+0000 7f03faffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7fbb145d0 0x55a7fbb10e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45074/0 (socket says 172.21.15.116:45074) 2024-09-12T03:11:46.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.826+0000 7f03faffd640 1 -- 172.21.15.116:0/1166589833 learned_addr learned my addr 172.21.15.116:0/1166589833 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:46.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.826+0000 7f03faffd640 1 -- 172.21.15.116:0/1166589833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fbb23780 msgr2=0x55a7fbb1fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:46.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.826+0000 7f03faffd640 1 --2- 172.21.15.116:0/1166589833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fbb23780 0x55a7fbb1fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.826+0000 7f03faffd640 1 -- 172.21.15.116:0/1166589833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7fbb15b40 msgr2=0x55a7fbb2f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:46.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.826+0000 7f03faffd640 1 --2- 172.21.15.116:0/1166589833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7fbb15b40 0x55a7fbb2f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.826+0000 7f03faffd640 1 -- 172.21.15.116:0/1166589833 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7fbb20d90 con 0x55a7fbb145d0 2024-09-12T03:11:46.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.827+0000 7f03faffd640 1 --2- 172.21.15.116:0/1166589833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7fbb145d0 0x55a7fbb10e10 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f03f0000c00 tx=0x7f03f001bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb20f70095f7785 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:46.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.827+0000 7f03e17fa640 1 -- 172.21.15.116:0/1166589833 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f000fc20 con 0x55a7fbb145d0 2024-09-12T03:11:46.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.827+0000 7f03e17fa640 1 -- 172.21.15.116:0/1166589833 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03f0030030 con 0x55a7fbb145d0 2024-09-12T03:11:46.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.827+0000 7f03e17fa640 1 -- 172.21.15.116:0/1166589833 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f001bb30 con 0x55a7fbb145d0 2024-09-12T03:11:46.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.827+0000 7f04042ef740 1 -- 172.21.15.116:0/1166589833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7fbb145d0 msgr2=0x55a7fbb10e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:46.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.827+0000 7f04042ef740 1 --2- 172.21.15.116:0/1166589833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7fbb145d0 0x55a7fbb10e10 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f03f0000c00 tx=0x7f03f001bfb0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.827+0000 7f04042ef740 1 -- 172.21.15.116:0/1166589833 shutdown_connections 2024-09-12T03:11:46.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.827+0000 7f04042ef740 1 --2- 172.21.15.116:0/1166589833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fbb23780 0x55a7fbb1fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.827+0000 7f04042ef740 1 --2- 172.21.15.116:0/1166589833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7fbb15b40 0x55a7fbb2f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.827+0000 7f04042ef740 1 --2- 172.21.15.116:0/1166589833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7fbb145d0 0x55a7fbb10e10 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.827+0000 7f04042ef740 1 -- 172.21.15.116:0/1166589833 >> 172.21.15.116:0/1166589833 conn(0x55a7fbadd310 msgr2=0x55a7fbb10700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:46.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.827+0000 7f04042ef740 1 -- 172.21.15.116:0/1166589833 shutdown_connections 2024-09-12T03:11:46.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.827+0000 7f04042ef740 1 -- 172.21.15.116:0/1166589833 wait complete. 2024-09-12T03:11:46.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.828+0000 7f04042ef740 1 -- start start 2024-09-12T03:11:46.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.828+0000 7f04042ef740 1 Processor -- start 2024-09-12T03:11:46.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.828+0000 7f04042ef740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7fbb145d0 0x55a7fbb2bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:46.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.828+0000 7f04042ef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fbb15b40 0x55a7fbb2c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:46.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.828+0000 7f04042ef740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7fbb23780 0x55a7fbb1ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:46.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.828+0000 7f04042ef740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc1a97a0 con 0x55a7fbb23780 2024-09-12T03:11:46.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.828+0000 7f04042ef740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc1aac10 con 0x55a7fbb145d0 2024-09-12T03:11:46.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.828+0000 7f04042ef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc1a9e70 con 0x55a7fbb15b40 2024-09-12T03:11:46.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.828+0000 7f04042ef740 10 client.0 _ll_register_callbacks cb 0x7f03f000a970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:46.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.828+0000 7f03faffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7fbb23780 0x55a7fbb1ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:46.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.828+0000 7f03faffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7fbb23780 0x55a7fbb1ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45082/0 (socket says 172.21.15.116:45082) 2024-09-12T03:11:46.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.828+0000 7f03faffd640 1 -- 172.21.15.116:0/1885081956 learned_addr learned my addr 172.21.15.116:0/1885081956 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:46.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.829+0000 7f03fb7fe640 1 --2- 172.21.15.116:0/1885081956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fbb15b40 0x55a7fbb2c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:46.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.829+0000 7f03faffd640 1 --2- 172.21.15.116:0/1885081956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7fbb145d0 0x55a7fbb2bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:46.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.829+0000 7f03faffd640 1 -- 172.21.15.116:0/1885081956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fbb15b40 msgr2=0x55a7fbb2c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:46.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.829+0000 7f03faffd640 1 --2- 172.21.15.116:0/1885081956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fbb15b40 0x55a7fbb2c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.829+0000 7f03faffd640 1 -- 172.21.15.116:0/1885081956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7fbb145d0 msgr2=0x55a7fbb2bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:46.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.829+0000 7f03faffd640 1 --2- 172.21.15.116:0/1885081956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7fbb145d0 0x55a7fbb2bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.829+0000 7f03faffd640 1 -- 172.21.15.116:0/1885081956 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7fbb1d630 con 0x55a7fbb23780 2024-09-12T03:11:46.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.829+0000 7f03fb7fe640 1 --2- 172.21.15.116:0/1885081956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fbb15b40 0x55a7fbb2c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:11:46.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.829+0000 7f03faffd640 1 --2- 172.21.15.116:0/1885081956 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7fbb23780 0x55a7fbb1ce30 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f03f0010010 tx=0x7f03f0019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1a26625737435 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:46.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.829+0000 7f03daffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:46.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.829+0000 7f03daffd640 1 -- 172.21.15.116:0/1885081956 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f003c060 con 0x55a7fbb23780 2024-09-12T03:11:46.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.829+0000 7f03daffd640 1 -- 172.21.15.116:0/1885081956 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f03f001b590 con 0x55a7fbb23780 2024-09-12T03:11:46.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.829+0000 7f04042ef740 1 -- 172.21.15.116:0/1885081956 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03f0030030 con 0x55a7fbb23780 2024-09-12T03:11:46.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.829+0000 7f03daffd640 1 -- 172.21.15.116:0/1885081956 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f03f0005040 con 0x55a7fbb23780 2024-09-12T03:11:46.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.829+0000 7f04042ef740 10 client.17634 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:46.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.829+0000 7f04042ef740 1 -- 172.21.15.116:0/1885081956 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a7fbb9dc30 con 0x55a7fbb23780 2024-09-12T03:11:46.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.829+0000 7f03daffd640 1 -- 172.21.15.116:0/1885081956 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03f0014030 con 0x55a7fbb23780 2024-09-12T03:11:46.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.829+0000 7f03daffd640 1 -- 172.21.15.116:0/1885081956 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f03f0043480 con 0x55a7fbb23780 2024-09-12T03:11:46.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.830+0000 7f04042ef740 10 client.17634 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:46.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.830+0000 7f04042ef740 10 client.17634 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:11:46.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.830+0000 7f04042ef740 20 client.17634 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:46.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.830+0000 7f04042ef740 1 --2- 172.21.15.116:0/1885081956 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55a7fbb9e140 0x55a7fbba0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:46.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.830+0000 7f04042ef740 4 client.17634 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:11:46.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.830+0000 7f04042ef740 1 -- 172.21.15.116:0/1885081956 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f03f0030030 con 0x55a7fbb9e140 2024-09-12T03:11:46.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.830+0000 7f03fb7fe640 1 --2- 172.21.15.116:0/1885081956 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55a7fbb9e140 0x55a7fbba0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:46.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.830+0000 7f03fb7fe640 1 --2- 172.21.15.116:0/1885081956 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55a7fbb9e140 0x55a7fbba0530 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=628eeaff9f9259b2 server_cookie=ccb03085d3c5fb2f in_seq=0 out_seq=0 2024-09-12T03:11:46.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.830+0000 7f03daffd640 10 client.17634 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:11:46.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.830+0000 7f03daffd640 1 -- 172.21.15.116:0/1885081956 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f03f0030030 con 0x55a7fbb9e140 2024-09-12T03:11:46.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.830+0000 7f03daffd640 10 client.17634 handle_command_reply: tid=0 2024-09-12T03:11:46.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.830+0000 7f04042ef740 1 client.17634 shutdown 2024-09-12T03:11:46.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.831+0000 7f04042ef740 1 -- 172.21.15.116:0/1885081956 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7fbb23780 msgr2=0x55a7fbb1ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:46.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.831+0000 7f04042ef740 1 --2- 172.21.15.116:0/1885081956 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7fbb23780 0x55a7fbb1ce30 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f03f0010010 tx=0x7f03f0019040 comp rx=0 tx=0).stop 2024-09-12T03:11:46.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.831+0000 7f04042ef740 1 -- 172.21.15.116:0/1885081956 shutdown_connections 2024-09-12T03:11:46.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.831+0000 7f04042ef740 1 --2- 172.21.15.116:0/1885081956 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55a7fbb9e140 0x55a7fbba0530 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.831+0000 7f04042ef740 1 --2- 172.21.15.116:0/1885081956 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a7fbb23780 0x55a7fbb1ce30 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.831+0000 7f04042ef740 1 --2- 172.21.15.116:0/1885081956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a7fbb15b40 0x55a7fbb2c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.831+0000 7f04042ef740 1 --2- 172.21.15.116:0/1885081956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7fbb145d0 0x55a7fbb2bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.831+0000 7f04042ef740 1 -- 172.21.15.116:0/1885081956 >> 172.21.15.116:0/1885081956 conn(0x55a7fbadd310 msgr2=0x55a7fbb0f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:46.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.831+0000 7f03daffd640 0 client.17634 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:11:46.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.831+0000 7f04042ef740 1 -- 172.21.15.116:0/1885081956 shutdown_connections 2024-09-12T03:11:46.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.831+0000 7f04042ef740 1 -- 172.21.15.116:0/1885081956 wait complete. 2024-09-12T03:11:46.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.831+0000 7f04042ef740 20 client.17634 trim_cache size 0 max 16384 2024-09-12T03:11:46.855 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:11:46.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.831+0000 7f0401f16640 1 -- 172.21.15.116:0/2323393666 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f03e40a2b80 msgr2=0x7f03e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:46.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.831+0000 7f0401f16640 1 --2- 172.21.15.116:0/2323393666 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f03e40a2b80 0x7f03e40a5040 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f03f400ffd0 tx=0x7f03f4066830 comp rx=0 tx=0).stop 2024-09-12T03:11:46.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.831+0000 7f0401f16640 1 -- 172.21.15.116:0/2323393666 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03fc106a80 msgr2=0x7f03fc199db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:46.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.831+0000 7f0401f16640 1 --2- 172.21.15.116:0/2323393666 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03fc106a80 0x7f03fc199db0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f03f400c970 tx=0x7f03f400cd70 comp rx=0 tx=0).stop 2024-09-12T03:11:46.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.831+0000 7f0401f16640 1 -- 172.21.15.116:0/2323393666 shutdown_connections 2024-09-12T03:11:46.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.831+0000 7f0401f16640 1 --2- 172.21.15.116:0/2323393666 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f03e40a2b80 0x7f03e40a5040 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.831+0000 7f0401f16640 1 --2- 172.21.15.116:0/2323393666 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03fc106a80 0x7f03fc199db0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.831+0000 7f0401f16640 1 --2- 172.21.15.116:0/2323393666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03fc101420 0x7f03fc199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.831+0000 7f0401f16640 1 --2- 172.21.15.116:0/2323393666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03fc100a80 0x7f03fc1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:46.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.831+0000 7f0401f16640 1 -- 172.21.15.116:0/2323393666 >> 172.21.15.116:0/2323393666 conn(0x7f03fc0fc7a0 msgr2=0x7f03fc0fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:46.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.832+0000 7f0401f16640 1 -- 172.21.15.116:0/2323393666 shutdown_connections 2024-09-12T03:11:46.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:46.832+0000 7f0401f16640 1 -- 172.21.15.116:0/2323393666 wait complete. 2024-09-12T03:11:47.623 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:47 smithi116 ceph-mon[31410]: pgmap v602: 97 pgs: 97 active+clean; 4.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:11:47.624 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:11:47.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:47 smithi183 ceph-mon[35535]: pgmap v602: 97 pgs: 97 active+clean; 4.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:11:47.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:47 smithi177 ceph-mon[35305]: pgmap v602: 97 pgs: 97 active+clean; 4.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:11:47.872 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:11:48.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.344+0000 7fb4b888e640 1 -- 172.21.15.116:0/2972658800 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4b4101420 msgr2=0x7fb4b410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:48.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.344+0000 7fb4b888e640 1 --2- 172.21.15.116:0/2972658800 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4b4101420 0x7fb4b410d950 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac009cd0 tx=0x7fb4ac02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.345+0000 7fb4b888e640 1 -- 172.21.15.116:0/2972658800 shutdown_connections 2024-09-12T03:11:48.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.345+0000 7fb4b888e640 1 --2- 172.21.15.116:0/2972658800 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4b4101420 0x7fb4b410d950 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.345+0000 7fb4b888e640 1 --2- 172.21.15.116:0/2972658800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4b4100a80 0x7fb4b4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.345+0000 7fb4b888e640 1 --2- 172.21.15.116:0/2972658800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4b4106a80 0x7fb4b4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.345+0000 7fb4b888e640 1 -- 172.21.15.116:0/2972658800 >> 172.21.15.116:0/2972658800 conn(0x7fb4b40fc7a0 msgr2=0x7fb4b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:48.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.345+0000 7fb4b888e640 1 -- 172.21.15.116:0/2972658800 shutdown_connections 2024-09-12T03:11:48.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.345+0000 7fb4b888e640 1 -- 172.21.15.116:0/2972658800 wait complete. 2024-09-12T03:11:48.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.345+0000 7fb4b888e640 1 Processor -- start 2024-09-12T03:11:48.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.345+0000 7fb4b888e640 1 -- start start 2024-09-12T03:11:48.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.346+0000 7fb4b888e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4b4100a80 0x7fb4b419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:48.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.346+0000 7fb4b888e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4b4101420 0x7fb4b419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:48.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.346+0000 7fb4b888e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4b4106a80 0x7fb4b419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:48.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.346+0000 7fb4b888e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b4110ec0 con 0x7fb4b4106a80 2024-09-12T03:11:48.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.346+0000 7fb4b888e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b4110d40 con 0x7fb4b4101420 2024-09-12T03:11:48.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.346+0000 7fb4b888e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b4111040 con 0x7fb4b4100a80 2024-09-12T03:11:48.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.346+0000 7fb4b2d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4b4106a80 0x7fb4b419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:48.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.346+0000 7fb4b2d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4b4106a80 0x7fb4b419ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45102/0 (socket says 172.21.15.116:45102) 2024-09-12T03:11:48.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.346+0000 7fb4b2d9d640 1 -- 172.21.15.116:0/1752924142 learned_addr learned my addr 172.21.15.116:0/1752924142 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:48.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.346+0000 7fb4b259c640 1 --2- 172.21.15.116:0/1752924142 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4b4100a80 0x7fb4b419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:48.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.346+0000 7fb4b1d9b640 1 --2- 172.21.15.116:0/1752924142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4b4101420 0x7fb4b419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:48.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.346+0000 7fb4b2d9d640 1 -- 172.21.15.116:0/1752924142 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4b4100a80 msgr2=0x7fb4b419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:48.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.346+0000 7fb4b2d9d640 1 --2- 172.21.15.116:0/1752924142 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4b4100a80 0x7fb4b419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.346+0000 7fb4b2d9d640 1 -- 172.21.15.116:0/1752924142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4b4101420 msgr2=0x7fb4b419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:48.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.346+0000 7fb4b2d9d640 1 --2- 172.21.15.116:0/1752924142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4b4101420 0x7fb4b419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.346+0000 7fb4b2d9d640 1 -- 172.21.15.116:0/1752924142 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b419f600 con 0x7fb4b4106a80 2024-09-12T03:11:48.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.346+0000 7fb4b1d9b640 1 --2- 172.21.15.116:0/1752924142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4b4101420 0x7fb4b419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:48.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.346+0000 7fb4b2d9d640 1 --2- 172.21.15.116:0/1752924142 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4b4106a80 0x7fb4b419ef20 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac005cc0 tx=0x7fb4ac003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:48.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.346+0000 7fb4a37fe640 1 -- 172.21.15.116:0/1752924142 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac004040 con 0x7fb4b4106a80 2024-09-12T03:11:48.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.346+0000 7fb4b888e640 1 -- 172.21.15.116:0/1752924142 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b419f890 con 0x7fb4b4106a80 2024-09-12T03:11:48.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.346+0000 7fb4a37fe640 1 -- 172.21.15.116:0/1752924142 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4ac0046c0 con 0x7fb4b4106a80 2024-09-12T03:11:48.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.346+0000 7fb4a37fe640 1 -- 172.21.15.116:0/1752924142 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac046480 con 0x7fb4b4106a80 2024-09-12T03:11:48.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.347+0000 7fb4b888e640 1 -- 172.21.15.116:0/1752924142 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b41a71d0 con 0x7fb4b4106a80 2024-09-12T03:11:48.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.349+0000 7fb4a37fe640 1 -- 172.21.15.116:0/1752924142 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb4ac0305a0 con 0x7fb4b4106a80 2024-09-12T03:11:48.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.349+0000 7fb4bac67740 1 Processor -- start 2024-09-12T03:11:48.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.349+0000 7fb4bac67740 1 -- start start 2024-09-12T03:11:48.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.349+0000 7fb4bac67740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555adb7adb40 0x555adb7ac090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:48.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.349+0000 7fb4bac67740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555adb7ac5d0 0x555adb7aca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:48.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.349+0000 7fb4bac67740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555adb7b7370 0x555adb7b9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:48.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.349+0000 7fb4bac67740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555adb7122f0 con 0x555adb7adb40 2024-09-12T03:11:48.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.350+0000 7fb4bac67740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x555adb6b9cf0 con 0x555adb7b7370 2024-09-12T03:11:48.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.350+0000 7fb4bac67740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555adb70e2b0 con 0x555adb7ac5d0 2024-09-12T03:11:48.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.350+0000 7fb4b1d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555adb7adb40 0x555adb7ac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:48.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.350+0000 7fb4b1d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555adb7adb40 0x555adb7ac090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45104/0 (socket says 172.21.15.116:45104) 2024-09-12T03:11:48.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.350+0000 7fb4b1d9b640 1 -- 172.21.15.116:0/3406797398 learned_addr learned my addr 172.21.15.116:0/3406797398 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:48.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.350+0000 7fb4b259c640 1 --2- 172.21.15.116:0/3406797398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555adb7ac5d0 0x555adb7aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:48.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.350+0000 7fb4b1d9b640 1 --2- 172.21.15.116:0/3406797398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555adb7b7370 0x555adb7b9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:48.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.350+0000 7fb4a37fe640 1 --2- 172.21.15.116:0/1752924142 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb49c0a2b80 0x7fb49c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:48.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.350+0000 7fb4b1d9b640 1 -- 172.21.15.116:0/3406797398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555adb7ac5d0 msgr2=0x555adb7aca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:48.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.350+0000 7fb4b1d9b640 1 --2- 172.21.15.116:0/3406797398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555adb7ac5d0 0x555adb7aca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.350+0000 7fb4b1d9b640 1 -- 172.21.15.116:0/3406797398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555adb7b7370 msgr2=0x555adb7b9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:48.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.350+0000 7fb4b1d9b640 1 --2- 172.21.15.116:0/3406797398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555adb7b7370 0x555adb7b9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.350+0000 7fb4b1d9b640 1 -- 172.21.15.116:0/3406797398 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555adb69e3a0 con 0x555adb7adb40 2024-09-12T03:11:48.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.350+0000 7fb4b2d9d640 1 --2- 172.21.15.116:0/1752924142 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb49c0a2b80 0x7fb49c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:48.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.350+0000 7fb4a37fe640 1 -- 172.21.15.116:0/1752924142 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fb4ac02e030 con 0x7fb4b4106a80 2024-09-12T03:11:48.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.350+0000 7fb4b1d9b640 1 --2- 172.21.15.116:0/3406797398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555adb7adb40 0x555adb7ac090 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800af90 tx=0x7fb4a800a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3df060a9e92ead2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:48.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.350+0000 7fb4a0ff9640 1 -- 172.21.15.116:0/3406797398 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a801b070 con 0x555adb7adb40 2024-09-12T03:11:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.350+0000 7fb4a0ff9640 1 -- 172.21.15.116:0/3406797398 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a80168d0 con 0x555adb7adb40 2024-09-12T03:11:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.350+0000 7fb4a0ff9640 1 -- 172.21.15.116:0/3406797398 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a800f040 con 0x555adb7adb40 2024-09-12T03:11:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.351+0000 7fb4bac67740 1 -- 172.21.15.116:0/3406797398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555adb7adb40 msgr2=0x555adb7ac090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.351+0000 7fb4b2d9d640 1 --2- 172.21.15.116:0/1752924142 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb49c0a2b80 0x7fb49c0a5040 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac0bcdd0 tx=0x7fb4ac087040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.351+0000 7fb4bac67740 1 --2- 172.21.15.116:0/3406797398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555adb7adb40 0x555adb7ac090 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800af90 tx=0x7fb4a800a410 comp rx=0 tx=0).stop 2024-09-12T03:11:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.351+0000 7fb4bac67740 1 -- 172.21.15.116:0/3406797398 shutdown_connections 2024-09-12T03:11:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.351+0000 7fb4bac67740 1 --2- 172.21.15.116:0/3406797398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555adb7b7370 0x555adb7b9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.351+0000 7fb4bac67740 1 --2- 172.21.15.116:0/3406797398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555adb7ac5d0 0x555adb7aca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.351+0000 7fb4bac67740 1 --2- 172.21.15.116:0/3406797398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555adb7adb40 0x555adb7ac090 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.351+0000 7fb4bac67740 1 -- 172.21.15.116:0/3406797398 >> 172.21.15.116:0/3406797398 conn(0x555adb775310 msgr2=0x555adb7a8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.351+0000 7fb4bac67740 1 -- 172.21.15.116:0/3406797398 shutdown_connections 2024-09-12T03:11:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.351+0000 7fb4bac67740 1 -- 172.21.15.116:0/3406797398 wait complete. 2024-09-12T03:11:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.352+0000 7fb4bac67740 1 -- start start 2024-09-12T03:11:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.352+0000 7fb4bac67740 1 Processor -- start 2024-09-12T03:11:48.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.352+0000 7fb4bac67740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555adb7ac5d0 0x555adb7b6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:48.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.352+0000 7fb4bac67740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555adb7adb40 0x555adb7b4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:48.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.352+0000 7fb4bac67740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555adb7b7370 0x555adb7b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:48.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.352+0000 7fb4bac67740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555adb7ba010 con 0x555adb7ac5d0 2024-09-12T03:11:48.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.352+0000 7fb4bac67740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x555adb6cd060 con 0x555adb7b7370 2024-09-12T03:11:48.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.352+0000 7fb4bac67740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555adb7ba190 con 0x555adb7adb40 2024-09-12T03:11:48.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.352+0000 7fb4bac67740 10 client.0 _ll_register_callbacks cb 0x555adb4208c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:48.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.352+0000 7fb4b1d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555adb7ac5d0 0x555adb7b6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:48.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.352+0000 7fb4b1d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555adb7ac5d0 0x555adb7b6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45110/0 (socket says 172.21.15.116:45110) 2024-09-12T03:11:48.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.352+0000 7fb4b1d9b640 1 -- 172.21.15.116:0/75932703 learned_addr learned my addr 172.21.15.116:0/75932703 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:48.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.352+0000 7fb4b259c640 1 --2- 172.21.15.116:0/75932703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555adb7adb40 0x555adb7b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:48.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.352+0000 7fb4b1d9b640 1 --2- 172.21.15.116:0/75932703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555adb7b7370 0x555adb7b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:48.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.352+0000 7fb4b1d9b640 1 -- 172.21.15.116:0/75932703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555adb7adb40 msgr2=0x555adb7b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:48.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.352+0000 7fb4b1d9b640 1 --2- 172.21.15.116:0/75932703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555adb7adb40 0x555adb7b4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.352+0000 7fb4b1d9b640 1 -- 172.21.15.116:0/75932703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555adb7b7370 msgr2=0x555adb7b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:48.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.352+0000 7fb4b1d9b640 1 --2- 172.21.15.116:0/75932703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555adb7b7370 0x555adb7b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.352+0000 7fb4b1d9b640 1 -- 172.21.15.116:0/75932703 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555adb4833c0 con 0x555adb7ac5d0 2024-09-12T03:11:48.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.352+0000 7fb4b1d9b640 1 --2- 172.21.15.116:0/75932703 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555adb7ac5d0 0x555adb7b6da0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8006dd0 tx=0x7fb4a8004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2c351659f53ce6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:48.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.353+0000 7fb499ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:48.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.353+0000 7fb499ffb640 1 -- 172.21.15.116:0/75932703 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8028060 con 0x555adb7ac5d0 2024-09-12T03:11:48.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.353+0000 7fb499ffb640 1 -- 172.21.15.116:0/75932703 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fb4a8016c90 con 0x555adb7ac5d0 2024-09-12T03:11:48.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.353+0000 7fb4bac67740 1 -- 172.21.15.116:0/75932703 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4a80168d0 con 0x555adb7ac5d0 2024-09-12T03:11:48.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.353+0000 7fb499ffb640 1 -- 172.21.15.116:0/75932703 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb4a801b050 con 0x555adb7ac5d0 2024-09-12T03:11:48.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.353+0000 7fb4bac67740 10 client.17646 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:48.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.353+0000 7fb4bac67740 1 -- 172.21.15.116:0/75932703 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555adb7c5af0 con 0x555adb7ac5d0 2024-09-12T03:11:48.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.353+0000 7fb499ffb640 1 -- 172.21.15.116:0/75932703 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a80168d0 con 0x555adb7ac5d0 2024-09-12T03:11:48.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.353+0000 7fb499ffb640 1 -- 172.21.15.116:0/75932703 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb4a8015030 con 0x555adb7ac5d0 2024-09-12T03:11:48.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.353+0000 7fb4bac67740 10 client.17646 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:48.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.353+0000 7fb4bac67740 10 client.17646 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:11:48.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.353+0000 7fb4bac67740 20 client.17646 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:48.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.353+0000 7fb4bac67740 1 --2- 172.21.15.116:0/75932703 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x555adb7c6f50 0x555adb7c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:48.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.353+0000 7fb4bac67740 4 client.17646 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:11:48.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.353+0000 7fb4bac67740 1 -- 172.21.15.116:0/75932703 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555adb71f860 con 0x555adb7c6f50 2024-09-12T03:11:48.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.353+0000 7fb4b259c640 1 --2- 172.21.15.116:0/75932703 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x555adb7c6f50 0x555adb7c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:48.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.354+0000 7fb4b259c640 1 --2- 172.21.15.116:0/75932703 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x555adb7c6f50 0x555adb7c9340 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=19c722afda5072f8 server_cookie=df6d9348a80e63c6 in_seq=0 out_seq=0 2024-09-12T03:11:48.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.354+0000 7fb499ffb640 10 client.17646 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:11:48.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.354+0000 7fb499ffb640 1 -- 172.21.15.116:0/75932703 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555adb71f860 con 0x555adb7c6f50 2024-09-12T03:11:48.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.355+0000 7fb499ffb640 10 client.17646 handle_command_reply: tid=0 2024-09-12T03:11:48.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.355+0000 7fb4bac67740 1 client.17646 shutdown 2024-09-12T03:11:48.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.355+0000 7fb4bac67740 1 -- 172.21.15.116:0/75932703 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555adb7ac5d0 msgr2=0x555adb7b6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:48.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.355+0000 7fb4bac67740 1 --2- 172.21.15.116:0/75932703 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555adb7ac5d0 0x555adb7b6da0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8006dd0 tx=0x7fb4a8004660 comp rx=0 tx=0).stop 2024-09-12T03:11:48.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.355+0000 7fb4bac67740 1 -- 172.21.15.116:0/75932703 shutdown_connections 2024-09-12T03:11:48.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.355+0000 7fb4bac67740 1 --2- 172.21.15.116:0/75932703 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x555adb7c6f50 0x555adb7c9340 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.355+0000 7fb4bac67740 1 --2- 172.21.15.116:0/75932703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555adb7b7370 0x555adb7b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.355+0000 7fb4bac67740 1 --2- 172.21.15.116:0/75932703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555adb7adb40 0x555adb7b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.355+0000 7fb499ffb640 0 client.17646 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:11:48.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.355+0000 7fb4bac67740 1 --2- 172.21.15.116:0/75932703 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555adb7ac5d0 0x555adb7b6da0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.355+0000 7fb4bac67740 1 -- 172.21.15.116:0/75932703 >> 172.21.15.116:0/75932703 conn(0x555adb775310 msgr2=0x555adb7ae960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:48.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.355+0000 7fb4bac67740 1 -- 172.21.15.116:0/75932703 shutdown_connections 2024-09-12T03:11:48.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.355+0000 7fb4bac67740 1 -- 172.21.15.116:0/75932703 wait complete. 2024-09-12T03:11:48.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.355+0000 7fb4bac67740 20 client.17646 trim_cache size 0 max 16384 2024-09-12T03:11:48.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.368+0000 7fb4bac67740 1 Processor -- start 2024-09-12T03:11:48.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.368+0000 7fb4bac67740 1 -- start start 2024-09-12T03:11:48.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.368+0000 7fb4bac67740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555adb7ac5d0 0x555adb7b3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:48.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.368+0000 7fb4bac67740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555adb7adb40 0x555adb7ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:48.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.368+0000 7fb4bac67740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555adb7b7370 0x555adb7ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:48.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.368+0000 7fb4bac67740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b41b1e00 con 0x555adb7adb40 2024-09-12T03:11:48.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.368+0000 7fb4bac67740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b41af6d0 con 0x555adb7ac5d0 2024-09-12T03:11:48.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.368+0000 7fb4bac67740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b41ad520 con 0x555adb7b7370 2024-09-12T03:11:48.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.368+0000 7fb4b259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555adb7adb40 0x555adb7ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:48.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.368+0000 7fb4b259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555adb7adb40 0x555adb7ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45122/0 (socket says 172.21.15.116:45122) 2024-09-12T03:11:48.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.368+0000 7fb4b259c640 1 -- 172.21.15.116:0/4159633439 learned_addr learned my addr 172.21.15.116:0/4159633439 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:48.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.368+0000 7fb4b259c640 1 -- 172.21.15.116:0/4159633439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555adb7b7370 msgr2=0x555adb7ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:48.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.368+0000 7fb4b259c640 1 --2- 172.21.15.116:0/4159633439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555adb7b7370 0x555adb7ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.368+0000 7fb4b259c640 1 -- 172.21.15.116:0/4159633439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555adb7ac5d0 msgr2=0x555adb7b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:48.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.368+0000 7fb4b259c640 1 --2- 172.21.15.116:0/4159633439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555adb7ac5d0 0x555adb7b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.368+0000 7fb4b259c640 1 -- 172.21.15.116:0/4159633439 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555adb7b3000 con 0x555adb7adb40 2024-09-12T03:11:48.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.368+0000 7fb4b259c640 1 --2- 172.21.15.116:0/4159633439 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555adb7adb40 0x555adb7ceab0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x555adb7a9ac0 tx=0x7fb4a400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae557dea242e0b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:48.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.368+0000 7fb49bfff640 1 -- 172.21.15.116:0/4159633439 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4005020 con 0x555adb7adb40 2024-09-12T03:11:48.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.368+0000 7fb49bfff640 1 -- 172.21.15.116:0/4159633439 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a40047a0 con 0x555adb7adb40 2024-09-12T03:11:48.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.368+0000 7fb49bfff640 1 -- 172.21.15.116:0/4159633439 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a40116a0 con 0x555adb7adb40 2024-09-12T03:11:48.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.369+0000 7fb4bac67740 1 -- 172.21.15.116:0/4159633439 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555adb7adb40 msgr2=0x555adb7ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:48.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.369+0000 7fb4bac67740 1 --2- 172.21.15.116:0/4159633439 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555adb7adb40 0x555adb7ceab0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x555adb7a9ac0 tx=0x7fb4a400b390 comp rx=0 tx=0).stop 2024-09-12T03:11:48.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.369+0000 7fb4bac67740 1 -- 172.21.15.116:0/4159633439 shutdown_connections 2024-09-12T03:11:48.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.369+0000 7fb4bac67740 1 --2- 172.21.15.116:0/4159633439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555adb7b7370 0x555adb7ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.369+0000 7fb4bac67740 1 --2- 172.21.15.116:0/4159633439 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555adb7adb40 0x555adb7ceab0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.369+0000 7fb4bac67740 1 --2- 172.21.15.116:0/4159633439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555adb7ac5d0 0x555adb7b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.369+0000 7fb4bac67740 1 -- 172.21.15.116:0/4159633439 >> 172.21.15.116:0/4159633439 conn(0x555adb775310 msgr2=0x555adb7ae930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:48.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.369+0000 7fb4bac67740 1 -- 172.21.15.116:0/4159633439 shutdown_connections 2024-09-12T03:11:48.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.369+0000 7fb4bac67740 1 -- 172.21.15.116:0/4159633439 wait complete. 2024-09-12T03:11:48.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.369+0000 7fb4bac67740 1 -- start start 2024-09-12T03:11:48.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.369+0000 7fb4bac67740 1 Processor -- start 2024-09-12T03:11:48.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.370+0000 7fb4bac67740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555adb7ac5d0 0x555adb7bb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:48.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.370+0000 7fb4bac67740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555adb7adb40 0x555adb7bb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:48.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.370+0000 7fb4bac67740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555adb7b7370 0x555adb7bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:48.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.370+0000 7fb4bac67740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b41adb90 con 0x555adb7ac5d0 2024-09-12T03:11:48.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.370+0000 7fb4bac67740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b41af000 con 0x555adb7adb40 2024-09-12T03:11:48.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.370+0000 7fb4bac67740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b41ae260 con 0x555adb7b7370 2024-09-12T03:11:48.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.370+0000 7fb4bac67740 10 client.0 _ll_register_callbacks cb 0x7fb4a8019070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:48.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.370+0000 7fb4b1d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555adb7ac5d0 0x555adb7bb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:48.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.370+0000 7fb4b1d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555adb7ac5d0 0x555adb7bb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45128/0 (socket says 172.21.15.116:45128) 2024-09-12T03:11:48.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.370+0000 7fb4b1d9b640 1 -- 172.21.15.116:0/3047354716 learned_addr learned my addr 172.21.15.116:0/3047354716 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:48.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.370+0000 7fb4b1d9b640 1 --2- 172.21.15.116:0/3047354716 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555adb7b7370 0x555adb7bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:48.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.370+0000 7fb4b1d9b640 1 -- 172.21.15.116:0/3047354716 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555adb7b7370 msgr2=0x555adb7bbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:48.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.370+0000 7fb4b1d9b640 1 --2- 172.21.15.116:0/3047354716 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555adb7b7370 0x555adb7bbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.370+0000 7fb4b1d9b640 1 -- 172.21.15.116:0/3047354716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555adb7adb40 msgr2=0x555adb7bb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:48.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.370+0000 7fb4b1d9b640 1 --2- 172.21.15.116:0/3047354716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555adb7adb40 0x555adb7bb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.370+0000 7fb4b1d9b640 1 -- 172.21.15.116:0/3047354716 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555adb839390 con 0x555adb7ac5d0 2024-09-12T03:11:48.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.370+0000 7fb4b1d9b640 1 --2- 172.21.15.116:0/3047354716 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555adb7ac5d0 0x555adb7bb260 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8016850 tx=0x7fb4a80136b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce4756b22231496 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:48.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.370+0000 7fb499ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:48.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.370+0000 7fb499ffb640 1 -- 172.21.15.116:0/3047354716 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8033070 con 0x555adb7ac5d0 2024-09-12T03:11:48.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.371+0000 7fb499ffb640 1 -- 172.21.15.116:0/3047354716 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fb4a800a570 con 0x555adb7ac5d0 2024-09-12T03:11:48.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.371+0000 7fb4bac67740 1 -- 172.21.15.116:0/3047354716 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4a40047a0 con 0x555adb7ac5d0 2024-09-12T03:11:48.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.371+0000 7fb499ffb640 1 -- 172.21.15.116:0/3047354716 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb4a8028070 con 0x555adb7ac5d0 2024-09-12T03:11:48.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.371+0000 7fb4bac67740 10 client.17652 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:48.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.371+0000 7fb4bac67740 1 -- 172.21.15.116:0/3047354716 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555adb83a650 con 0x555adb7ac5d0 2024-09-12T03:11:48.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.371+0000 7fb499ffb640 1 -- 172.21.15.116:0/3047354716 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a40047a0 con 0x555adb7ac5d0 2024-09-12T03:11:48.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.371+0000 7fb499ffb640 1 -- 172.21.15.116:0/3047354716 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb4a8036020 con 0x555adb7ac5d0 2024-09-12T03:11:48.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.371+0000 7fb4bac67740 10 client.17652 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:48.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.371+0000 7fb4bac67740 10 client.17652 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:11:48.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.371+0000 7fb4bac67740 20 client.17652 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:48.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.371+0000 7fb4bac67740 1 --2- 172.21.15.116:0/3047354716 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x555adb83ad10 0x555adb83d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:48.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.371+0000 7fb4bac67740 4 client.17652 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:11:48.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.371+0000 7fb4bac67740 1 -- 172.21.15.116:0/3047354716 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555adb719d70 con 0x555adb83ad10 2024-09-12T03:11:48.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.371+0000 7fb4b259c640 1 --2- 172.21.15.116:0/3047354716 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x555adb83ad10 0x555adb83d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:48.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.372+0000 7fb499ffb640 10 client.17652 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:11:48.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.372+0000 7fb4b259c640 1 --2- 172.21.15.116:0/3047354716 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x555adb83ad10 0x555adb83d130 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5de188a88c2162c0 server_cookie=386f0ff6a546bf96 in_seq=0 out_seq=0 2024-09-12T03:11:48.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.372+0000 7fb499ffb640 1 -- 172.21.15.116:0/3047354716 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555adb719d70 con 0x555adb83ad10 2024-09-12T03:11:48.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.372+0000 7fb499ffb640 10 client.17652 handle_command_reply: tid=0 2024-09-12T03:11:48.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.372+0000 7fb4bac67740 1 client.17652 shutdown 2024-09-12T03:11:48.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.372+0000 7fb4bac67740 1 -- 172.21.15.116:0/3047354716 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555adb7ac5d0 msgr2=0x555adb7bb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:48.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.372+0000 7fb4bac67740 1 --2- 172.21.15.116:0/3047354716 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555adb7ac5d0 0x555adb7bb260 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8016850 tx=0x7fb4a80136b0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.372+0000 7fb4bac67740 1 -- 172.21.15.116:0/3047354716 shutdown_connections 2024-09-12T03:11:48.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.372+0000 7fb4bac67740 1 --2- 172.21.15.116:0/3047354716 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x555adb83ad10 0x555adb83d130 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.372+0000 7fb4bac67740 1 --2- 172.21.15.116:0/3047354716 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555adb7b7370 0x555adb7bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.372+0000 7fb4bac67740 1 --2- 172.21.15.116:0/3047354716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555adb7adb40 0x555adb7bb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.372+0000 7fb4bac67740 1 --2- 172.21.15.116:0/3047354716 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555adb7ac5d0 0x555adb7bb260 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.372+0000 7fb4bac67740 1 -- 172.21.15.116:0/3047354716 >> 172.21.15.116:0/3047354716 conn(0x555adb775310 msgr2=0x555adb7ae930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:48.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.372+0000 7fb499ffb640 0 client.17652 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:11:48.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.372+0000 7fb4bac67740 1 -- 172.21.15.116:0/3047354716 shutdown_connections 2024-09-12T03:11:48.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.372+0000 7fb4bac67740 1 -- 172.21.15.116:0/3047354716 wait complete. 2024-09-12T03:11:48.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.372+0000 7fb4bac67740 20 client.17652 trim_cache size 0 max 16384 2024-09-12T03:11:48.384 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:11:48.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.373+0000 7fb4b888e640 1 -- 172.21.15.116:0/1752924142 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb49c0a2b80 msgr2=0x7fb49c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:48.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.373+0000 7fb4b888e640 1 --2- 172.21.15.116:0/1752924142 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb49c0a2b80 0x7fb49c0a5040 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac0bcdd0 tx=0x7fb4ac087040 comp rx=0 tx=0).stop 2024-09-12T03:11:48.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.373+0000 7fb4b888e640 1 -- 172.21.15.116:0/1752924142 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4b4106a80 msgr2=0x7fb4b419ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:48.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.373+0000 7fb4b888e640 1 --2- 172.21.15.116:0/1752924142 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4b4106a80 0x7fb4b419ef20 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac005cc0 tx=0x7fb4ac003970 comp rx=0 tx=0).stop 2024-09-12T03:11:48.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.373+0000 7fb4b888e640 1 -- 172.21.15.116:0/1752924142 shutdown_connections 2024-09-12T03:11:48.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.373+0000 7fb4b888e640 1 --2- 172.21.15.116:0/1752924142 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb49c0a2b80 0x7fb49c0a5040 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.373+0000 7fb4b888e640 1 --2- 172.21.15.116:0/1752924142 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4b4106a80 0x7fb4b419ef20 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.373+0000 7fb4b888e640 1 --2- 172.21.15.116:0/1752924142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4b4101420 0x7fb4b419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.373+0000 7fb4b888e640 1 --2- 172.21.15.116:0/1752924142 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4b4100a80 0x7fb4b419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:48.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.373+0000 7fb4b888e640 1 -- 172.21.15.116:0/1752924142 >> 172.21.15.116:0/1752924142 conn(0x7fb4b40fc7a0 msgr2=0x7fb4b40fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:48.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.373+0000 7fb4b888e640 1 -- 172.21.15.116:0/1752924142 shutdown_connections 2024-09-12T03:11:48.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:48.373+0000 7fb4b888e640 1 -- 172.21.15.116:0/1752924142 wait complete. 2024-09-12T03:11:49.048 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:11:49.294 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:11:49.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.783+0000 7f0d548ec640 1 -- 172.21.15.116:0/3494189441 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d50102ef0 msgr2=0x7f0d5010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:49.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.783+0000 7f0d548ec640 1 --2- 172.21.15.116:0/3494189441 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d50102ef0 0x7f0d5010da40 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f0d48009ca0 tx=0x7f0d4802a950 comp rx=0 tx=0).stop 2024-09-12T03:11:49.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.784+0000 7f0d548ec640 1 -- 172.21.15.116:0/3494189441 shutdown_connections 2024-09-12T03:11:49.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.784+0000 7f0d548ec640 1 --2- 172.21.15.116:0/3494189441 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d50102ef0 0x7f0d5010da40 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.784+0000 7f0d548ec640 1 --2- 172.21.15.116:0/3494189441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d50102570 0x7f0d501029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.784+0000 7f0d548ec640 1 --2- 172.21.15.116:0/3494189441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d50100e20 0x7f0d50101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.784+0000 7f0d548ec640 1 -- 172.21.15.116:0/3494189441 >> 172.21.15.116:0/3494189441 conn(0x7f0d500fc7d0 msgr2=0x7f0d500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:49.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.784+0000 7f0d548ec640 1 -- 172.21.15.116:0/3494189441 shutdown_connections 2024-09-12T03:11:49.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.784+0000 7f0d548ec640 1 -- 172.21.15.116:0/3494189441 wait complete. 2024-09-12T03:11:49.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.784+0000 7f0d548ec640 1 Processor -- start 2024-09-12T03:11:49.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.784+0000 7f0d548ec640 1 -- start start 2024-09-12T03:11:49.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.785+0000 7f0d548ec640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d50100e20 0x7f0d501992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:49.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.785+0000 7f0d548ec640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d50102570 0x7f0d501997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:49.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.785+0000 7f0d548ec640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d50102ef0 0x7f0d501a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:49.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.785+0000 7f0d548ec640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0d50110fa0 con 0x7f0d50102570 2024-09-12T03:11:49.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.785+0000 7f0d548ec640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0d50110e20 con 0x7f0d50100e20 2024-09-12T03:11:49.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.785+0000 7f0d548ec640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0d50111120 con 0x7f0d50102ef0 2024-09-12T03:11:49.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.785+0000 7f0d47fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d50102570 0x7f0d501997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:49.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.785+0000 7f0d47fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d50102570 0x7f0d501997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45150/0 (socket says 172.21.15.116:45150) 2024-09-12T03:11:49.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.785+0000 7f0d47fff640 1 -- 172.21.15.116:0/140938281 learned_addr learned my addr 172.21.15.116:0/140938281 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:49.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.785+0000 7f0d4e59c640 1 --2- 172.21.15.116:0/140938281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d50100e20 0x7f0d501992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:49.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.785+0000 7f0d4ed9d640 1 --2- 172.21.15.116:0/140938281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d50102ef0 0x7f0d501a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:49.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.785+0000 7f0d47fff640 1 -- 172.21.15.116:0/140938281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d50102ef0 msgr2=0x7f0d501a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:49.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.785+0000 7f0d47fff640 1 --2- 172.21.15.116:0/140938281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d50102ef0 0x7f0d501a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.785+0000 7f0d47fff640 1 -- 172.21.15.116:0/140938281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d50100e20 msgr2=0x7f0d501992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:49.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.785+0000 7f0d47fff640 1 --2- 172.21.15.116:0/140938281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d50100e20 0x7f0d501992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.785+0000 7f0d47fff640 1 -- 172.21.15.116:0/140938281 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d501a4860 con 0x7f0d50102570 2024-09-12T03:11:49.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.785+0000 7f0d4ed9d640 1 --2- 172.21.15.116:0/140938281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d50102ef0 0x7f0d501a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:11:49.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.785+0000 7f0d47fff640 1 --2- 172.21.15.116:0/140938281 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d50102570 0x7f0d501997e0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800b4a0 tx=0x7f0d3800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:49.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.785+0000 7f0d477fe640 1 -- 172.21.15.116:0/140938281 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38015020 con 0x7f0d50102570 2024-09-12T03:11:49.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.785+0000 7f0d477fe640 1 -- 172.21.15.116:0/140938281 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d38011d10 con 0x7f0d50102570 2024-09-12T03:11:49.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.785+0000 7f0d548ec640 1 -- 172.21.15.116:0/140938281 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d501a4a30 con 0x7f0d50102570 2024-09-12T03:11:49.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.785+0000 7f0d477fe640 1 -- 172.21.15.116:0/140938281 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3800f2e0 con 0x7f0d50102570 2024-09-12T03:11:49.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.786+0000 7f0d548ec640 1 -- 172.21.15.116:0/140938281 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d501a4f40 con 0x7f0d50102570 2024-09-12T03:11:49.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.788+0000 7f0d477fe640 1 -- 172.21.15.116:0/140938281 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f0d38018020 con 0x7f0d50102570 2024-09-12T03:11:49.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.788+0000 7f0d56cc5740 1 Processor -- start 2024-09-12T03:11:49.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.788+0000 7f0d56cc5740 1 -- start start 2024-09-12T03:11:49.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.789+0000 7f0d56cc5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0d1a47b40 0x55a0d1a46090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:49.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.789+0000 7f0d56cc5740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d1a465d0 0x55a0d1a46a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:49.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.789+0000 7f0d56cc5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0d1a55780 0x55a0d1a57b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:49.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.789+0000 7f0d56cc5740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a0d19ac2f0 con 0x55a0d1a465d0 2024-09-12T03:11:49.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.789+0000 7f0d56cc5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a0d1953cf0 con 0x55a0d1a47b40 2024-09-12T03:11:49.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.789+0000 7f0d56cc5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a0d19a82b0 con 0x55a0d1a55780 2024-09-12T03:11:49.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.789+0000 7f0d477fe640 1 --2- 172.21.15.116:0/140938281 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0d3c0a2b80 0x7f0d3c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:49.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.789+0000 7f0d4e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d1a465d0 0x55a0d1a46a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:49.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.789+0000 7f0d4e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d1a465d0 0x55a0d1a46a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45160/0 (socket says 172.21.15.116:45160) 2024-09-12T03:11:49.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.789+0000 7f0d4e59c640 1 -- 172.21.15.116:0/3373103920 learned_addr learned my addr 172.21.15.116:0/3373103920 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:49.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.789+0000 7f0d47fff640 1 --2- 172.21.15.116:0/3373103920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0d1a55780 0x55a0d1a57b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:49.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.789+0000 7f0d4ed9d640 1 --2- 172.21.15.116:0/3373103920 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0d1a47b40 0x55a0d1a46090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:49.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.789+0000 7f0d4ed9d640 1 --2- 172.21.15.116:0/140938281 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0d3c0a2b80 0x7f0d3c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:49.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.789+0000 7f0d477fe640 1 -- 172.21.15.116:0/140938281 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f0d380ce900 con 0x7f0d50102570 2024-09-12T03:11:49.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.789+0000 7f0d4e59c640 1 -- 172.21.15.116:0/3373103920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0d1a55780 msgr2=0x55a0d1a57b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:49.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.789+0000 7f0d4e59c640 1 --2- 172.21.15.116:0/3373103920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0d1a55780 0x55a0d1a57b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.789+0000 7f0d4e59c640 1 -- 172.21.15.116:0/3373103920 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0d1a47b40 msgr2=0x55a0d1a46090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:49.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.789+0000 7f0d4e59c640 1 --2- 172.21.15.116:0/3373103920 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0d1a47b40 0x55a0d1a46090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.789+0000 7f0d4e59c640 1 -- 172.21.15.116:0/3373103920 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0d19383a0 con 0x55a0d1a465d0 2024-09-12T03:11:49.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.789+0000 7f0d4e59c640 1 --2- 172.21.15.116:0/3373103920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d1a465d0 0x55a0d1a46a10 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000ad00 tx=0x7f0d4000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbdb97702bfc2c80 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:49.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.789+0000 7f0d4ed9d640 1 --2- 172.21.15.116:0/140938281 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0d3c0a2b80 0x7f0d3c0a5040 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0d48009a60 tx=0x7f0d480095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:49.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.790+0000 7f0d44ff9640 1 -- 172.21.15.116:0/3373103920 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4001d070 con 0x55a0d1a465d0 2024-09-12T03:11:49.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.790+0000 7f0d44ff9640 1 -- 172.21.15.116:0/3373103920 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d40018b20 con 0x55a0d1a465d0 2024-09-12T03:11:49.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.790+0000 7f0d44ff9640 1 -- 172.21.15.116:0/3373103920 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40010bc0 con 0x55a0d1a465d0 2024-09-12T03:11:49.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.790+0000 7f0d56cc5740 1 -- 172.21.15.116:0/3373103920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d1a465d0 msgr2=0x55a0d1a46a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:49.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.790+0000 7f0d56cc5740 1 --2- 172.21.15.116:0/3373103920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d1a465d0 0x55a0d1a46a10 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000ad00 tx=0x7f0d4000f790 comp rx=0 tx=0).stop 2024-09-12T03:11:49.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.790+0000 7f0d56cc5740 1 -- 172.21.15.116:0/3373103920 shutdown_connections 2024-09-12T03:11:49.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.790+0000 7f0d56cc5740 1 --2- 172.21.15.116:0/3373103920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0d1a55780 0x55a0d1a57b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.790+0000 7f0d56cc5740 1 --2- 172.21.15.116:0/3373103920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d1a465d0 0x55a0d1a46a10 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.790+0000 7f0d56cc5740 1 --2- 172.21.15.116:0/3373103920 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0d1a47b40 0x55a0d1a46090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.790+0000 7f0d56cc5740 1 -- 172.21.15.116:0/3373103920 >> 172.21.15.116:0/3373103920 conn(0x55a0d1a0f310 msgr2=0x55a0d1a42c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:49.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.790+0000 7f0d56cc5740 1 -- 172.21.15.116:0/3373103920 shutdown_connections 2024-09-12T03:11:49.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.790+0000 7f0d56cc5740 1 -- 172.21.15.116:0/3373103920 wait complete. 2024-09-12T03:11:49.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.791+0000 7f0d56cc5740 1 -- start start 2024-09-12T03:11:49.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.791+0000 7f0d56cc5740 1 Processor -- start 2024-09-12T03:11:49.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.791+0000 7f0d56cc5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0d1a465d0 0x55a0d1a55120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:49.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.791+0000 7f0d56cc5740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d1a47b40 0x55a0d1a523d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:49.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.791+0000 7f0d56cc5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0d1a55780 0x55a0d1a52970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:49.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.791+0000 7f0d56cc5740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a0d1a58480 con 0x55a0d1a47b40 2024-09-12T03:11:49.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.791+0000 7f0d56cc5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a0d1967060 con 0x55a0d1a465d0 2024-09-12T03:11:49.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.791+0000 7f0d56cc5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a0d1a58600 con 0x55a0d1a55780 2024-09-12T03:11:49.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.791+0000 7f0d56cc5740 10 client.0 _ll_register_callbacks cb 0x55a0d16ba8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:49.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.791+0000 7f0d47fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d1a47b40 0x55a0d1a523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:49.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.791+0000 7f0d47fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d1a47b40 0x55a0d1a523d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45170/0 (socket says 172.21.15.116:45170) 2024-09-12T03:11:49.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.791+0000 7f0d47fff640 1 -- 172.21.15.116:0/2129491564 learned_addr learned my addr 172.21.15.116:0/2129491564 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:49.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.791+0000 7f0d47fff640 1 -- 172.21.15.116:0/2129491564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0d1a55780 msgr2=0x55a0d1a52970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:11:49.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.791+0000 7f0d47fff640 1 --2- 172.21.15.116:0/2129491564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0d1a55780 0x55a0d1a52970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.791+0000 7f0d47fff640 1 -- 172.21.15.116:0/2129491564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0d1a465d0 msgr2=0x55a0d1a55120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:11:49.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.791+0000 7f0d47fff640 1 --2- 172.21.15.116:0/2129491564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0d1a465d0 0x55a0d1a55120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.791+0000 7f0d47fff640 1 -- 172.21.15.116:0/2129491564 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0d171d3c0 con 0x55a0d1a47b40 2024-09-12T03:11:49.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.792+0000 7f0d47fff640 1 --2- 172.21.15.116:0/2129491564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d1a47b40 0x55a0d1a523d0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f0d380cc480 tx=0x7f0d3806b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c767d18ac43fce90 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:49.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.792+0000 7f0d35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:49.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.792+0000 7f0d35ffb640 1 -- 172.21.15.116:0/2129491564 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3809acd0 con 0x55a0d1a47b40 2024-09-12T03:11:49.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.792+0000 7f0d35ffb640 1 -- 172.21.15.116:0/2129491564 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f0d38097800 con 0x55a0d1a47b40 2024-09-12T03:11:49.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.792+0000 7f0d56cc5740 1 -- 172.21.15.116:0/2129491564 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d40018b20 con 0x55a0d1a47b40 2024-09-12T03:11:49.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.792+0000 7f0d35ffb640 1 -- 172.21.15.116:0/2129491564 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0d3809aaf0 con 0x55a0d1a47b40 2024-09-12T03:11:49.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.792+0000 7f0d56cc5740 10 client.17664 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:49.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.792+0000 7f0d56cc5740 1 -- 172.21.15.116:0/2129491564 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a0d1a63ed0 con 0x55a0d1a47b40 2024-09-12T03:11:49.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.792+0000 7f0d35ffb640 1 -- 172.21.15.116:0/2129491564 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d40018b20 con 0x55a0d1a47b40 2024-09-12T03:11:49.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.792+0000 7f0d35ffb640 1 -- 172.21.15.116:0/2129491564 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0d380bb0b0 con 0x55a0d1a47b40 2024-09-12T03:11:49.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.792+0000 7f0d56cc5740 10 client.17664 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:49.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.792+0000 7f0d56cc5740 10 client.17664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:11:49.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.792+0000 7f0d56cc5740 20 client.17664 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:49.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.792+0000 7f0d56cc5740 1 --2- 172.21.15.116:0/2129491564 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a0d1a65270 0x55a0d1a676c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:49.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.792+0000 7f0d56cc5740 4 client.17664 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:11:49.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.792+0000 7f0d56cc5740 1 -- 172.21.15.116:0/2129491564 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0d19b9860 con 0x55a0d1a65270 2024-09-12T03:11:49.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.793+0000 7f0d4e59c640 1 --2- 172.21.15.116:0/2129491564 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a0d1a65270 0x55a0d1a676c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:49.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.793+0000 7f0d35ffb640 10 client.17664 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:49.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.793+0000 7f0d4e59c640 1 --2- 172.21.15.116:0/2129491564 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a0d1a65270 0x55a0d1a676c0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba8cbf81f2b1ae1f server_cookie=34564528c654b1d6 in_seq=0 out_seq=0 2024-09-12T03:11:49.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.794+0000 7f0d35ffb640 1 -- 172.21.15.116:0/2129491564 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a0d19b9860 con 0x55a0d1a65270 2024-09-12T03:11:49.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.794+0000 7f0d35ffb640 10 client.17664 handle_command_reply: tid=0 2024-09-12T03:11:49.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.794+0000 7f0d56cc5740 1 client.17664 shutdown 2024-09-12T03:11:49.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.794+0000 7f0d56cc5740 1 -- 172.21.15.116:0/2129491564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d1a47b40 msgr2=0x55a0d1a523d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:49.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.794+0000 7f0d56cc5740 1 --2- 172.21.15.116:0/2129491564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d1a47b40 0x55a0d1a523d0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f0d380cc480 tx=0x7f0d3806b040 comp rx=0 tx=0).stop 2024-09-12T03:11:49.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.794+0000 7f0d56cc5740 1 -- 172.21.15.116:0/2129491564 shutdown_connections 2024-09-12T03:11:49.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.794+0000 7f0d56cc5740 1 --2- 172.21.15.116:0/2129491564 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a0d1a65270 0x55a0d1a676c0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.794+0000 7f0d56cc5740 1 --2- 172.21.15.116:0/2129491564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0d1a55780 0x55a0d1a52970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.794+0000 7f0d35ffb640 0 client.17664 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:49.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.794+0000 7f0d56cc5740 1 --2- 172.21.15.116:0/2129491564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d1a47b40 0x55a0d1a523d0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.794+0000 7f0d56cc5740 1 --2- 172.21.15.116:0/2129491564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0d1a465d0 0x55a0d1a55120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.794+0000 7f0d56cc5740 1 -- 172.21.15.116:0/2129491564 >> 172.21.15.116:0/2129491564 conn(0x55a0d1a0f310 msgr2=0x55a0d1a48930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:49.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.794+0000 7f0d56cc5740 1 -- 172.21.15.116:0/2129491564 shutdown_connections 2024-09-12T03:11:49.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.794+0000 7f0d56cc5740 1 -- 172.21.15.116:0/2129491564 wait complete. 2024-09-12T03:11:49.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.794+0000 7f0d56cc5740 20 client.17664 trim_cache size 0 max 16384 2024-09-12T03:11:49.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.807+0000 7f0d56cc5740 1 Processor -- start 2024-09-12T03:11:49.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.807+0000 7f0d56cc5740 1 -- start start 2024-09-12T03:11:49.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.807+0000 7f0d56cc5740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d1a465d0 0x55a0d1a61720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:49.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.807+0000 7f0d56cc5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0d1a47b40 0x55a0d1a47300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:49.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.807+0000 7f0d56cc5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0d1a55780 0x55a0d1a42700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:49.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.807+0000 7f0d56cc5740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501adbc0 con 0x55a0d1a465d0 2024-09-12T03:11:49.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.807+0000 7f0d56cc5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501ab490 con 0x55a0d1a47b40 2024-09-12T03:11:49.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.807+0000 7f0d56cc5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501a9300 con 0x55a0d1a55780 2024-09-12T03:11:49.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.807+0000 7f0d4e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d1a465d0 0x55a0d1a61720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:49.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.807+0000 7f0d4e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d1a465d0 0x55a0d1a61720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45186/0 (socket says 172.21.15.116:45186) 2024-09-12T03:11:49.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.807+0000 7f0d4e59c640 1 -- 172.21.15.116:0/3901150102 learned_addr learned my addr 172.21.15.116:0/3901150102 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:49.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.807+0000 7f0d4ed9d640 1 --2- 172.21.15.116:0/3901150102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0d1a55780 0x55a0d1a42700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:49.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.807+0000 7f0d47fff640 1 --2- 172.21.15.116:0/3901150102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0d1a47b40 0x55a0d1a47300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:49.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.807+0000 7f0d4e59c640 1 -- 172.21.15.116:0/3901150102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0d1a55780 msgr2=0x55a0d1a42700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:49.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.807+0000 7f0d4e59c640 1 --2- 172.21.15.116:0/3901150102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0d1a55780 0x55a0d1a42700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.807+0000 7f0d4e59c640 1 -- 172.21.15.116:0/3901150102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0d1a47b40 msgr2=0x55a0d1a47300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:49.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.807+0000 7f0d4e59c640 1 --2- 172.21.15.116:0/3901150102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0d1a47b40 0x55a0d1a47300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.807+0000 7f0d4e59c640 1 -- 172.21.15.116:0/3901150102 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0d1a50960 con 0x55a0d1a465d0 2024-09-12T03:11:49.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.807+0000 7f0d47fff640 1 --2- 172.21.15.116:0/3901150102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0d1a47b40 0x55a0d1a47300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:49.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.808+0000 7f0d4e59c640 1 --2- 172.21.15.116:0/3901150102 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d1a465d0 0x55a0d1a61720 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000ad00 tx=0x7f0d40010840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe41bfcf244fdaf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:49.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.808+0000 7f0d37fff640 1 -- 172.21.15.116:0/3901150102 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4001d040 con 0x55a0d1a465d0 2024-09-12T03:11:49.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.808+0000 7f0d37fff640 1 -- 172.21.15.116:0/3901150102 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d40029070 con 0x55a0d1a465d0 2024-09-12T03:11:49.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.808+0000 7f0d37fff640 1 -- 172.21.15.116:0/3901150102 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40005650 con 0x55a0d1a465d0 2024-09-12T03:11:49.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.808+0000 7f0d56cc5740 1 -- 172.21.15.116:0/3901150102 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d1a465d0 msgr2=0x55a0d1a61720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:49.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.808+0000 7f0d56cc5740 1 --2- 172.21.15.116:0/3901150102 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d1a465d0 0x55a0d1a61720 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000ad00 tx=0x7f0d40010840 comp rx=0 tx=0).stop 2024-09-12T03:11:49.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.808+0000 7f0d56cc5740 1 -- 172.21.15.116:0/3901150102 shutdown_connections 2024-09-12T03:11:49.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.808+0000 7f0d56cc5740 1 --2- 172.21.15.116:0/3901150102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0d1a55780 0x55a0d1a42700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.808+0000 7f0d56cc5740 1 --2- 172.21.15.116:0/3901150102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0d1a47b40 0x55a0d1a47300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.808+0000 7f0d56cc5740 1 --2- 172.21.15.116:0/3901150102 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d1a465d0 0x55a0d1a61720 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.808+0000 7f0d56cc5740 1 -- 172.21.15.116:0/3901150102 >> 172.21.15.116:0/3901150102 conn(0x55a0d1a0f310 msgr2=0x55a0d1a3da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:49.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.808+0000 7f0d56cc5740 1 -- 172.21.15.116:0/3901150102 shutdown_connections 2024-09-12T03:11:49.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.808+0000 7f0d56cc5740 1 -- 172.21.15.116:0/3901150102 wait complete. 2024-09-12T03:11:49.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.809+0000 7f0d56cc5740 1 -- start start 2024-09-12T03:11:49.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.809+0000 7f0d56cc5740 1 Processor -- start 2024-09-12T03:11:49.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.809+0000 7f0d56cc5740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d1a465d0 0x55a0d1a5e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:49.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.809+0000 7f0d56cc5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0d1a47b40 0x55a0d1a4f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:49.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.809+0000 7f0d56cc5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0d1a55780 0x55a0d1a4fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:49.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.809+0000 7f0d56cc5740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501a9950 con 0x55a0d1a465d0 2024-09-12T03:11:49.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.809+0000 7f0d56cc5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501aadc0 con 0x55a0d1a55780 2024-09-12T03:11:49.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.809+0000 7f0d56cc5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501aa020 con 0x55a0d1a47b40 2024-09-12T03:11:49.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.809+0000 7f0d56cc5740 10 client.0 _ll_register_callbacks cb 0x7f0d3800b3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:49.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.809+0000 7f0d4e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d1a465d0 0x55a0d1a5e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:49.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.809+0000 7f0d4e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d1a465d0 0x55a0d1a5e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45188/0 (socket says 172.21.15.116:45188) 2024-09-12T03:11:49.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.809+0000 7f0d4e59c640 1 -- 172.21.15.116:0/1520534736 learned_addr learned my addr 172.21.15.116:0/1520534736 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:49.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.809+0000 7f0d47fff640 1 --2- 172.21.15.116:0/1520534736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0d1a47b40 0x55a0d1a4f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:49.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.809+0000 7f0d4e59c640 1 -- 172.21.15.116:0/1520534736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0d1a47b40 msgr2=0x55a0d1a4f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:49.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.809+0000 7f0d4e59c640 1 --2- 172.21.15.116:0/1520534736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0d1a47b40 0x55a0d1a4f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.809+0000 7f0d4e59c640 1 -- 172.21.15.116:0/1520534736 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0d1a55780 msgr2=0x55a0d1a4fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:49.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.809+0000 7f0d4e59c640 1 --2- 172.21.15.116:0/1520534736 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0d1a55780 0x55a0d1a4fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.809+0000 7f0d4e59c640 1 -- 172.21.15.116:0/1520534736 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0d1acef60 con 0x55a0d1a465d0 2024-09-12T03:11:49.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.810+0000 7f0d4e59c640 1 --2- 172.21.15.116:0/1520534736 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d1a465d0 0x55a0d1a5e180 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000ac50 tx=0x7f0d40019e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4daa6516a53f7e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:49.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.810+0000 7f0d35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:49.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.810+0000 7f0d35ffb640 1 -- 172.21.15.116:0/1520534736 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4001d040 con 0x55a0d1a465d0 2024-09-12T03:11:49.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.810+0000 7f0d35ffb640 1 -- 172.21.15.116:0/1520534736 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f0d40005390 con 0x55a0d1a465d0 2024-09-12T03:11:49.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.810+0000 7f0d56cc5740 1 -- 172.21.15.116:0/1520534736 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d40029070 con 0x55a0d1a465d0 2024-09-12T03:11:49.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.810+0000 7f0d35ffb640 1 -- 172.21.15.116:0/1520534736 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0d400106a0 con 0x55a0d1a465d0 2024-09-12T03:11:49.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.810+0000 7f0d56cc5740 10 client.17670 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:49.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.810+0000 7f0d56cc5740 1 -- 172.21.15.116:0/1520534736 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a0d1ad0070 con 0x55a0d1a465d0 2024-09-12T03:11:49.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.810+0000 7f0d35ffb640 1 -- 172.21.15.116:0/1520534736 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d40029070 con 0x55a0d1a465d0 2024-09-12T03:11:49.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.810+0000 7f0d35ffb640 1 -- 172.21.15.116:0/1520534736 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0d40038740 con 0x55a0d1a465d0 2024-09-12T03:11:49.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.810+0000 7f0d56cc5740 10 client.17670 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:49.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.810+0000 7f0d56cc5740 10 client.17670 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:11:49.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.810+0000 7f0d56cc5740 20 client.17670 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:49.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.810+0000 7f0d56cc5740 1 --2- 172.21.15.116:0/1520534736 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a0d1a65270 0x55a0d1ad25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:49.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.810+0000 7f0d56cc5740 4 client.17670 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:11:49.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.810+0000 7f0d56cc5740 1 -- 172.21.15.116:0/1520534736 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a0d19b3d70 con 0x55a0d1a65270 2024-09-12T03:11:49.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.811+0000 7f0d47fff640 1 --2- 172.21.15.116:0/1520534736 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a0d1a65270 0x55a0d1ad25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:49.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.811+0000 7f0d35ffb640 10 client.17670 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:49.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.811+0000 7f0d47fff640 1 --2- 172.21.15.116:0/1520534736 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a0d1a65270 0x55a0d1ad25a0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=711c07f90a30753e server_cookie=2c2aa67c17864472 in_seq=0 out_seq=0 2024-09-12T03:11:49.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.811+0000 7f0d35ffb640 1 -- 172.21.15.116:0/1520534736 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a0d19b3d70 con 0x55a0d1a65270 2024-09-12T03:11:49.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.811+0000 7f0d35ffb640 10 client.17670 handle_command_reply: tid=0 2024-09-12T03:11:49.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.811+0000 7f0d56cc5740 1 client.17670 shutdown 2024-09-12T03:11:49.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.811+0000 7f0d56cc5740 1 -- 172.21.15.116:0/1520534736 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d1a465d0 msgr2=0x55a0d1a5e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:49.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.811+0000 7f0d56cc5740 1 --2- 172.21.15.116:0/1520534736 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d1a465d0 0x55a0d1a5e180 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000ac50 tx=0x7f0d40019e80 comp rx=0 tx=0).stop 2024-09-12T03:11:49.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.811+0000 7f0d56cc5740 1 -- 172.21.15.116:0/1520534736 shutdown_connections 2024-09-12T03:11:49.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.811+0000 7f0d56cc5740 1 --2- 172.21.15.116:0/1520534736 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a0d1a65270 0x55a0d1ad25a0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.811+0000 7f0d56cc5740 1 --2- 172.21.15.116:0/1520534736 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0d1a55780 0x55a0d1a4fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.811+0000 7f0d56cc5740 1 --2- 172.21.15.116:0/1520534736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0d1a47b40 0x55a0d1a4f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.811+0000 7f0d35ffb640 0 client.17670 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:49.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.811+0000 7f0d56cc5740 1 --2- 172.21.15.116:0/1520534736 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0d1a465d0 0x55a0d1a5e180 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.811+0000 7f0d56cc5740 1 -- 172.21.15.116:0/1520534736 >> 172.21.15.116:0/1520534736 conn(0x55a0d1a0f310 msgr2=0x55a0d1a426e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:49.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.811+0000 7f0d56cc5740 1 -- 172.21.15.116:0/1520534736 shutdown_connections 2024-09-12T03:11:49.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.811+0000 7f0d56cc5740 1 -- 172.21.15.116:0/1520534736 wait complete. 2024-09-12T03:11:49.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.811+0000 7f0d56cc5740 20 client.17670 trim_cache size 0 max 16384 2024-09-12T03:11:49.821 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:11:49.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.812+0000 7f0d548ec640 1 -- 172.21.15.116:0/140938281 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0d3c0a2b80 msgr2=0x7f0d3c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:49.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.812+0000 7f0d548ec640 1 --2- 172.21.15.116:0/140938281 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0d3c0a2b80 0x7f0d3c0a5040 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0d48009a60 tx=0x7f0d480095c0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.812+0000 7f0d548ec640 1 -- 172.21.15.116:0/140938281 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d50102570 msgr2=0x7f0d501997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:49.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.812+0000 7f0d548ec640 1 --2- 172.21.15.116:0/140938281 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d50102570 0x7f0d501997e0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800b4a0 tx=0x7f0d3800b8a0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.812+0000 7f0d548ec640 1 -- 172.21.15.116:0/140938281 shutdown_connections 2024-09-12T03:11:49.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.812+0000 7f0d548ec640 1 --2- 172.21.15.116:0/140938281 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0d3c0a2b80 0x7f0d3c0a5040 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.812+0000 7f0d548ec640 1 --2- 172.21.15.116:0/140938281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d50102ef0 0x7f0d501a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.812+0000 7f0d548ec640 1 --2- 172.21.15.116:0/140938281 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d50102570 0x7f0d501997e0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.812+0000 7f0d548ec640 1 --2- 172.21.15.116:0/140938281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d50100e20 0x7f0d501992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:49.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.812+0000 7f0d548ec640 1 -- 172.21.15.116:0/140938281 >> 172.21.15.116:0/140938281 conn(0x7f0d500fc7d0 msgr2=0x7f0d50106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:49.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.812+0000 7f0d548ec640 1 -- 172.21.15.116:0/140938281 shutdown_connections 2024-09-12T03:11:49.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:49.812+0000 7f0d548ec640 1 -- 172.21.15.116:0/140938281 wait complete. 2024-09-12T03:11:50.478 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:50 smithi116 ceph-mon[31410]: pgmap v603: 97 pgs: 97 active+clean; 4.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:11:50.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:50 smithi183 ceph-mon[35535]: pgmap v603: 97 pgs: 97 active+clean; 4.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:11:50.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:50 smithi177 ceph-mon[35305]: pgmap v603: 97 pgs: 97 active+clean; 4.2 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:11:51.507 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:11:51.507 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:11:51.765 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:11:52.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.265+0000 7fb625163640 1 -- 172.21.15.116:0/3884195456 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb620101de0 msgr2=0x7fb6201021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:52.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.265+0000 7fb625163640 1 --2- 172.21.15.116:0/3884195456 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb620101de0 0x7fb6201021c0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fb610009900 tx=0x7fb61002a410 comp rx=0 tx=0).stop 2024-09-12T03:11:52.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.265+0000 7fb625163640 1 -- 172.21.15.116:0/3884195456 shutdown_connections 2024-09-12T03:11:52.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.265+0000 7fb625163640 1 --2- 172.21.15.116:0/3884195456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb620106f10 0x7fb62010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.265+0000 7fb625163640 1 --2- 172.21.15.116:0/3884195456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb620102790 0x7fb6201067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.265+0000 7fb625163640 1 --2- 172.21.15.116:0/3884195456 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb620101de0 0x7fb6201021c0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.265+0000 7fb625163640 1 -- 172.21.15.116:0/3884195456 >> 172.21.15.116:0/3884195456 conn(0x7fb6200fd8c0 msgr2=0x7fb6200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:52.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.265+0000 7fb625163640 1 -- 172.21.15.116:0/3884195456 shutdown_connections 2024-09-12T03:11:52.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.265+0000 7fb625163640 1 -- 172.21.15.116:0/3884195456 wait complete. 2024-09-12T03:11:52.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.266+0000 7fb625163640 1 Processor -- start 2024-09-12T03:11:52.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.266+0000 7fb625163640 1 -- start start 2024-09-12T03:11:52.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.266+0000 7fb625163640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb620101de0 0x7fb6201983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:52.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.266+0000 7fb625163640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb620102790 0x7fb620198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:52.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.266+0000 7fb625163640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb620106f10 0x7fb62019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:52.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.266+0000 7fb625163640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb620111f70 con 0x7fb620101de0 2024-09-12T03:11:52.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.266+0000 7fb625163640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb620111df0 con 0x7fb620106f10 2024-09-12T03:11:52.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.266+0000 7fb625163640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb6201120f0 con 0x7fb620102790 2024-09-12T03:11:52.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.266+0000 7fb61ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb620101de0 0x7fb6201983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:52.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.266+0000 7fb61ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb620101de0 0x7fb6201983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48390/0 (socket says 172.21.15.116:48390) 2024-09-12T03:11:52.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.266+0000 7fb61ed9d640 1 -- 172.21.15.116:0/1280944823 learned_addr learned my addr 172.21.15.116:0/1280944823 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:52.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.266+0000 7fb61f59e640 1 --2- 172.21.15.116:0/1280944823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb620106f10 0x7fb62019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:52.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.266+0000 7fb61e59c640 1 --2- 172.21.15.116:0/1280944823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb620102790 0x7fb620198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:52.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.266+0000 7fb61ed9d640 1 -- 172.21.15.116:0/1280944823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb620102790 msgr2=0x7fb620198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:52.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.266+0000 7fb61ed9d640 1 --2- 172.21.15.116:0/1280944823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb620102790 0x7fb620198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.266+0000 7fb61ed9d640 1 -- 172.21.15.116:0/1280944823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb620106f10 msgr2=0x7fb62019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:52.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.266+0000 7fb61ed9d640 1 --2- 172.21.15.116:0/1280944823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb620106f10 0x7fb62019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.266+0000 7fb61ed9d640 1 -- 172.21.15.116:0/1280944823 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb62019d380 con 0x7fb620101de0 2024-09-12T03:11:52.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.267+0000 7fb61e59c640 1 --2- 172.21.15.116:0/1280944823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb620102790 0x7fb620198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:52.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.267+0000 7fb61ed9d640 1 --2- 172.21.15.116:0/1280944823 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb620101de0 0x7fb6201983d0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fb610006010 tx=0x7fb6100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:52.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.267+0000 7fb617fff640 1 -- 172.21.15.116:0/1280944823 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61003f030 con 0x7fb620101de0 2024-09-12T03:11:52.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.267+0000 7fb625163640 1 -- 172.21.15.116:0/1280944823 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb62019d610 con 0x7fb620101de0 2024-09-12T03:11:52.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.267+0000 7fb617fff640 1 -- 172.21.15.116:0/1280944823 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb610003ce0 con 0x7fb620101de0 2024-09-12T03:11:52.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.267+0000 7fb617fff640 1 -- 172.21.15.116:0/1280944823 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb610035070 con 0x7fb620101de0 2024-09-12T03:11:52.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.267+0000 7fb625163640 1 -- 172.21.15.116:0/1280944823 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6201a4f50 con 0x7fb620101de0 2024-09-12T03:11:52.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.269+0000 7fb617fff640 1 -- 172.21.15.116:0/1280944823 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb610042030 con 0x7fb620101de0 2024-09-12T03:11:52.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.270+0000 7fb62753c740 1 Processor -- start 2024-09-12T03:11:52.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.270+0000 7fb62753c740 1 -- start start 2024-09-12T03:11:52.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.270+0000 7fb62753c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624ec4c9b60 0x5624ec4c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:52.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.270+0000 7fb62753c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624ec4ca480 0x5624ec4d4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:52.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.270+0000 7fb62753c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624ec4d4d40 0x5624ec4d7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:52.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.270+0000 7fb62753c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5624ec42f280 con 0x5624ec4d4d40 2024-09-12T03:11:52.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.270+0000 7fb62753c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5624ec3d6b10 con 0x5624ec4ca480 2024-09-12T03:11:52.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.270+0000 7fb62753c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5624ec42b240 con 0x5624ec4c9b60 2024-09-12T03:11:52.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.270+0000 7fb617fff640 1 --2- 172.21.15.116:0/1280944823 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb60c0a2b80 0x7fb60c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:52.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.270+0000 7fb61e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624ec4d4d40 0x5624ec4d7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:52.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.270+0000 7fb61e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624ec4d4d40 0x5624ec4d7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48398/0 (socket says 172.21.15.116:48398) 2024-09-12T03:11:52.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.270+0000 7fb61e59c640 1 -- 172.21.15.116:0/1879639117 learned_addr learned my addr 172.21.15.116:0/1879639117 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:52.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.270+0000 7fb61f59e640 1 --2- 172.21.15.116:0/1280944823 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb60c0a2b80 0x7fb60c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:52.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.270+0000 7fb617fff640 1 -- 172.21.15.116:0/1280944823 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fb6100ef6d0 con 0x7fb620101de0 2024-09-12T03:11:52.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.270+0000 7fb61f59e640 1 --2- 172.21.15.116:0/1879639117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624ec4c9b60 0x5624ec4c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:52.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.271+0000 7fb61ed9d640 1 --2- 172.21.15.116:0/1879639117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624ec4ca480 0x5624ec4d4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:52.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.271+0000 7fb61e59c640 1 -- 172.21.15.116:0/1879639117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624ec4c9b60 msgr2=0x5624ec4c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:52.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.271+0000 7fb61e59c640 1 --2- 172.21.15.116:0/1879639117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624ec4c9b60 0x5624ec4c9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.271+0000 7fb61e59c640 1 -- 172.21.15.116:0/1879639117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624ec4ca480 msgr2=0x5624ec4d4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:52.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.271+0000 7fb61e59c640 1 --2- 172.21.15.116:0/1879639117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624ec4ca480 0x5624ec4d4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.271+0000 7fb61e59c640 1 -- 172.21.15.116:0/1879639117 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624ec3e31f0 con 0x5624ec4d4d40 2024-09-12T03:11:52.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.271+0000 7fb61f59e640 1 --2- 172.21.15.116:0/1879639117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624ec4c9b60 0x5624ec4c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:52.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.271+0000 7fb61ed9d640 1 --2- 172.21.15.116:0/1879639117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624ec4ca480 0x5624ec4d4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:11:52.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.271+0000 7fb61e59c640 1 --2- 172.21.15.116:0/1879639117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624ec4d4d40 0x5624ec4d7130 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fb608009c20 tx=0x7fb60800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d196eab14fbc1349 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:52.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.271+0000 7fb6157fa640 1 -- 172.21.15.116:0/1879639117 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb608016070 con 0x5624ec4d4d40 2024-09-12T03:11:52.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.271+0000 7fb61f59e640 1 --2- 172.21.15.116:0/1280944823 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb60c0a2b80 0x7fb60c0a5040 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb618007310 tx=0x7fb6180072a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:52.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.271+0000 7fb6157fa640 1 -- 172.21.15.116:0/1879639117 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb608011950 con 0x5624ec4d4d40 2024-09-12T03:11:52.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.271+0000 7fb6157fa640 1 -- 172.21.15.116:0/1879639117 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb608010870 con 0x5624ec4d4d40 2024-09-12T03:11:52.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.272+0000 7fb62753c740 1 -- 172.21.15.116:0/1879639117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624ec4d4d40 msgr2=0x5624ec4d7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:52.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.272+0000 7fb62753c740 1 --2- 172.21.15.116:0/1879639117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624ec4d4d40 0x5624ec4d7130 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fb608009c20 tx=0x7fb60800e6c0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.272+0000 7fb62753c740 1 -- 172.21.15.116:0/1879639117 shutdown_connections 2024-09-12T03:11:52.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.272+0000 7fb62753c740 1 --2- 172.21.15.116:0/1879639117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624ec4d4d40 0x5624ec4d7130 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.272+0000 7fb62753c740 1 --2- 172.21.15.116:0/1879639117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624ec4ca480 0x5624ec4d4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.272+0000 7fb62753c740 1 --2- 172.21.15.116:0/1879639117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624ec4c9b60 0x5624ec4c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.272+0000 7fb62753c740 1 -- 172.21.15.116:0/1879639117 >> 172.21.15.116:0/1879639117 conn(0x5624ec4c60f0 msgr2=0x5624ec4c64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:52.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.272+0000 7fb62753c740 1 -- 172.21.15.116:0/1879639117 shutdown_connections 2024-09-12T03:11:52.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.272+0000 7fb62753c740 1 -- 172.21.15.116:0/1879639117 wait complete. 2024-09-12T03:11:52.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.272+0000 7fb62753c740 1 -- start start 2024-09-12T03:11:52.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.272+0000 7fb62753c740 1 Processor -- start 2024-09-12T03:11:52.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.273+0000 7fb62753c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624ec4c9b60 0x5624ec4d4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:52.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.273+0000 7fb62753c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624ec4ca480 0x5624ec4d1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:52.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.273+0000 7fb62753c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624ec4d4d40 0x5624ec4d20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:52.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.273+0000 7fb62753c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5624ec4d7910 con 0x5624ec4d4d40 2024-09-12T03:11:52.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.273+0000 7fb62753c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5624ec3e9f10 con 0x5624ec4ca480 2024-09-12T03:11:52.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.273+0000 7fb62753c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5624ec4d7a90 con 0x5624ec4c9b60 2024-09-12T03:11:52.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.273+0000 7fb62753c740 10 client.0 _ll_register_callbacks cb 0x5624ec3b2a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:52.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.273+0000 7fb61f59e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624ec4d4d40 0x5624ec4d20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:52.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.273+0000 7fb61f59e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624ec4d4d40 0x5624ec4d20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48406/0 (socket says 172.21.15.116:48406) 2024-09-12T03:11:52.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.273+0000 7fb61f59e640 1 -- 172.21.15.116:0/2886984229 learned_addr learned my addr 172.21.15.116:0/2886984229 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:52.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.273+0000 7fb61e59c640 1 --2- 172.21.15.116:0/2886984229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624ec4ca480 0x5624ec4d1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:52.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.273+0000 7fb61ed9d640 1 --2- 172.21.15.116:0/2886984229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624ec4c9b60 0x5624ec4d4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:52.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.273+0000 7fb61f59e640 1 -- 172.21.15.116:0/2886984229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624ec4c9b60 msgr2=0x5624ec4d4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:52.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.273+0000 7fb61f59e640 1 --2- 172.21.15.116:0/2886984229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624ec4c9b60 0x5624ec4d4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.273+0000 7fb61f59e640 1 -- 172.21.15.116:0/2886984229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624ec4ca480 msgr2=0x5624ec4d1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:52.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.273+0000 7fb61f59e640 1 --2- 172.21.15.116:0/2886984229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624ec4ca480 0x5624ec4d1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.273+0000 7fb61f59e640 1 -- 172.21.15.116:0/2886984229 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624ec1a06a0 con 0x5624ec4d4d40 2024-09-12T03:11:52.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.273+0000 7fb61ed9d640 1 --2- 172.21.15.116:0/2886984229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624ec4c9b60 0x5624ec4d4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:52.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.273+0000 7fb61f59e640 1 --2- 172.21.15.116:0/2886984229 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624ec4d4d40 0x5624ec4d20c0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fb618012f20 tx=0x7fb618004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492c37fdc770396e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:52.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.273+0000 7fb6067fc640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:52.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.273+0000 7fb6067fc640 1 -- 172.21.15.116:0/2886984229 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb618022070 con 0x5624ec4d4d40 2024-09-12T03:11:52.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.273+0000 7fb6067fc640 1 -- 172.21.15.116:0/2886984229 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fb61801e820 con 0x5624ec4d4d40 2024-09-12T03:11:52.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.273+0000 7fb62753c740 1 -- 172.21.15.116:0/2886984229 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb608011950 con 0x5624ec4d4d40 2024-09-12T03:11:52.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.273+0000 7fb6067fc640 1 -- 172.21.15.116:0/2886984229 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb61802e050 con 0x5624ec4d4d40 2024-09-12T03:11:52.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.273+0000 7fb62753c740 10 client.17682 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:52.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.273+0000 7fb62753c740 1 -- 172.21.15.116:0/2886984229 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5624ec4e3370 con 0x5624ec4d4d40 2024-09-12T03:11:52.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.274+0000 7fb6067fc640 1 -- 172.21.15.116:0/2886984229 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb608011950 con 0x5624ec4d4d40 2024-09-12T03:11:52.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.274+0000 7fb6067fc640 1 -- 172.21.15.116:0/2886984229 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb61801a070 con 0x5624ec4d4d40 2024-09-12T03:11:52.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.274+0000 7fb62753c740 10 client.17682 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:52.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.274+0000 7fb62753c740 10 client.17682 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:11:52.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.274+0000 7fb62753c740 20 client.17682 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:52.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.274+0000 7fb62753c740 1 --2- 172.21.15.116:0/2886984229 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5624ec4e4780 0x5624ec4e6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:52.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.274+0000 7fb62753c740 4 client.17682 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:11:52.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.274+0000 7fb62753c740 1 -- 172.21.15.116:0/2886984229 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624ec43c7f0 con 0x5624ec4e4780 2024-09-12T03:11:52.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.274+0000 7fb61ed9d640 1 --2- 172.21.15.116:0/2886984229 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5624ec4e4780 0x5624ec4e6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:52.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.275+0000 7fb6067fc640 10 client.17682 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:52.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.275+0000 7fb61ed9d640 1 --2- 172.21.15.116:0/2886984229 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5624ec4e4780 0x5624ec4e6bf0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8af7922cdb5f608 server_cookie=1cbd797043ea0b27 in_seq=0 out_seq=0 2024-09-12T03:11:52.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.275+0000 7fb6067fc640 1 -- 172.21.15.116:0/2886984229 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5624ec43c7f0 con 0x5624ec4e4780 2024-09-12T03:11:52.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.275+0000 7fb6067fc640 10 client.17682 handle_command_reply: tid=0 2024-09-12T03:11:52.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.275+0000 7fb62753c740 1 client.17682 shutdown 2024-09-12T03:11:52.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.275+0000 7fb62753c740 1 -- 172.21.15.116:0/2886984229 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624ec4d4d40 msgr2=0x5624ec4d20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:52.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.275+0000 7fb62753c740 1 --2- 172.21.15.116:0/2886984229 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624ec4d4d40 0x5624ec4d20c0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fb618012f20 tx=0x7fb618004a40 comp rx=0 tx=0).stop 2024-09-12T03:11:52.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.275+0000 7fb62753c740 1 -- 172.21.15.116:0/2886984229 shutdown_connections 2024-09-12T03:11:52.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.275+0000 7fb62753c740 1 --2- 172.21.15.116:0/2886984229 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5624ec4e4780 0x5624ec4e6bf0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.275+0000 7fb62753c740 1 --2- 172.21.15.116:0/2886984229 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624ec4d4d40 0x5624ec4d20c0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.275+0000 7fb62753c740 1 --2- 172.21.15.116:0/2886984229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624ec4ca480 0x5624ec4d1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.275+0000 7fb6067fc640 0 client.17682 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:52.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.275+0000 7fb62753c740 1 --2- 172.21.15.116:0/2886984229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624ec4c9b60 0x5624ec4d4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.275+0000 7fb62753c740 1 -- 172.21.15.116:0/2886984229 >> 172.21.15.116:0/2886984229 conn(0x5624ec4c60f0 msgr2=0x5624ec4c5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:52.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.275+0000 7fb62753c740 1 -- 172.21.15.116:0/2886984229 shutdown_connections 2024-09-12T03:11:52.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.275+0000 7fb62753c740 1 -- 172.21.15.116:0/2886984229 wait complete. 2024-09-12T03:11:52.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.276+0000 7fb62753c740 20 client.17682 trim_cache size 0 max 16384 2024-09-12T03:11:52.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.293+0000 7fb62753c740 1 Processor -- start 2024-09-12T03:11:52.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.293+0000 7fb62753c740 1 -- start start 2024-09-12T03:11:52.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.293+0000 7fb62753c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624ec4c9b60 0x5624ec4e0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:52.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.293+0000 7fb62753c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624ec4ca480 0x5624ec4d3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:52.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.293+0000 7fb62753c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624ec4d4d40 0x5624ec4ec510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:52.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.293+0000 7fb62753c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb6201afd30 con 0x5624ec4d4d40 2024-09-12T03:11:52.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.293+0000 7fb62753c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb6201ad600 con 0x5624ec4c9b60 2024-09-12T03:11:52.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.293+0000 7fb62753c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb6201ab470 con 0x5624ec4ca480 2024-09-12T03:11:52.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.293+0000 7fb61f59e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624ec4d4d40 0x5624ec4ec510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:52.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.293+0000 7fb61f59e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624ec4d4d40 0x5624ec4ec510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48412/0 (socket says 172.21.15.116:48412) 2024-09-12T03:11:52.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.293+0000 7fb61f59e640 1 -- 172.21.15.116:0/4277556913 learned_addr learned my addr 172.21.15.116:0/4277556913 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:52.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.293+0000 7fb61ed9d640 1 --2- 172.21.15.116:0/4277556913 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624ec4c9b60 0x5624ec4e0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:52.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.293+0000 7fb61e59c640 1 --2- 172.21.15.116:0/4277556913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624ec4ca480 0x5624ec4d3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:52.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.293+0000 7fb61f59e640 1 -- 172.21.15.116:0/4277556913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624ec4ca480 msgr2=0x5624ec4d3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:52.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.293+0000 7fb61f59e640 1 --2- 172.21.15.116:0/4277556913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624ec4ca480 0x5624ec4d3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.293+0000 7fb61f59e640 1 -- 172.21.15.116:0/4277556913 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624ec4c9b60 msgr2=0x5624ec4e0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:52.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.293+0000 7fb61f59e640 1 --2- 172.21.15.116:0/4277556913 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624ec4c9b60 0x5624ec4e0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.293+0000 7fb61f59e640 1 -- 172.21.15.116:0/4277556913 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624ec4d0a60 con 0x5624ec4d4d40 2024-09-12T03:11:52.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.294+0000 7fb61f59e640 1 --2- 172.21.15.116:0/4277556913 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624ec4d4d40 0x5624ec4ec510 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fb618011ca0 tx=0x7fb618007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8611af6f4de4d826 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:52.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.294+0000 7fb614ff9640 1 -- 172.21.15.116:0/4277556913 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb618022070 con 0x5624ec4d4d40 2024-09-12T03:11:52.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.294+0000 7fb614ff9640 1 -- 172.21.15.116:0/4277556913 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb61800fc30 con 0x5624ec4d4d40 2024-09-12T03:11:52.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.294+0000 7fb614ff9640 1 -- 172.21.15.116:0/4277556913 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb618016070 con 0x5624ec4d4d40 2024-09-12T03:11:52.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.294+0000 7fb62753c740 1 -- 172.21.15.116:0/4277556913 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624ec4d4d40 msgr2=0x5624ec4ec510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:52.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.294+0000 7fb62753c740 1 --2- 172.21.15.116:0/4277556913 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624ec4d4d40 0x5624ec4ec510 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fb618011ca0 tx=0x7fb618007a10 comp rx=0 tx=0).stop 2024-09-12T03:11:52.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.295+0000 7fb62753c740 1 -- 172.21.15.116:0/4277556913 shutdown_connections 2024-09-12T03:11:52.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.295+0000 7fb62753c740 1 --2- 172.21.15.116:0/4277556913 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624ec4d4d40 0x5624ec4ec510 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.295+0000 7fb62753c740 1 --2- 172.21.15.116:0/4277556913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624ec4ca480 0x5624ec4d3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.295+0000 7fb62753c740 1 --2- 172.21.15.116:0/4277556913 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624ec4c9b60 0x5624ec4e0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.295+0000 7fb62753c740 1 -- 172.21.15.116:0/4277556913 >> 172.21.15.116:0/4277556913 conn(0x5624ec4c60f0 msgr2=0x5624ec4d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:52.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.295+0000 7fb62753c740 1 -- 172.21.15.116:0/4277556913 shutdown_connections 2024-09-12T03:11:52.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.295+0000 7fb62753c740 1 -- 172.21.15.116:0/4277556913 wait complete. 2024-09-12T03:11:52.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.295+0000 7fb62753c740 1 -- start start 2024-09-12T03:11:52.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.295+0000 7fb62753c740 1 Processor -- start 2024-09-12T03:11:52.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.295+0000 7fb62753c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624ec4c9b60 0x5624ec4dd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:52.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.295+0000 7fb62753c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624ec4ca480 0x5624ec4d7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:52.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.295+0000 7fb62753c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624ec4d4d40 0x5624ec4d81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:52.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.295+0000 7fb62753c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb6201abac0 con 0x5624ec4d4d40 2024-09-12T03:11:52.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.295+0000 7fb62753c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb6201acf30 con 0x5624ec4ca480 2024-09-12T03:11:52.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.295+0000 7fb62753c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb6201ac190 con 0x5624ec4c9b60 2024-09-12T03:11:52.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.295+0000 7fb61f59e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624ec4d4d40 0x5624ec4d81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:52.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.295+0000 7fb62753c740 10 client.0 _ll_register_callbacks cb 0x7fb618007c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:52.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.295+0000 7fb61f59e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624ec4d4d40 0x5624ec4d81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48420/0 (socket says 172.21.15.116:48420) 2024-09-12T03:11:52.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.295+0000 7fb61f59e640 1 -- 172.21.15.116:0/322549430 learned_addr learned my addr 172.21.15.116:0/322549430 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:52.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.295+0000 7fb61e59c640 1 --2- 172.21.15.116:0/322549430 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624ec4ca480 0x5624ec4d7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:52.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.296+0000 7fb61ed9d640 1 --2- 172.21.15.116:0/322549430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624ec4c9b60 0x5624ec4dd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:52.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.296+0000 7fb61f59e640 1 -- 172.21.15.116:0/322549430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624ec4c9b60 msgr2=0x5624ec4dd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:52.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.296+0000 7fb61f59e640 1 --2- 172.21.15.116:0/322549430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624ec4c9b60 0x5624ec4dd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.296+0000 7fb61f59e640 1 -- 172.21.15.116:0/322549430 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624ec4ca480 msgr2=0x5624ec4d7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:52.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.296+0000 7fb61f59e640 1 --2- 172.21.15.116:0/322549430 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624ec4ca480 0x5624ec4d7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.296+0000 7fb61f59e640 1 -- 172.21.15.116:0/322549430 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624ec4ca2b0 con 0x5624ec4d4d40 2024-09-12T03:11:52.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.296+0000 7fb61f59e640 1 --2- 172.21.15.116:0/322549430 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624ec4d4d40 0x5624ec4d81e0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fb618007aa0 tx=0x7fb618026ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68276214c2948a81 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:52.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.296+0000 7fb6067fc640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:52.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.296+0000 7fb6067fc640 1 -- 172.21.15.116:0/322549430 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61803f020 con 0x5624ec4d4d40 2024-09-12T03:11:52.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.296+0000 7fb6067fc640 1 -- 172.21.15.116:0/322549430 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fb61801e620 con 0x5624ec4d4d40 2024-09-12T03:11:52.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.296+0000 7fb62753c740 1 -- 172.21.15.116:0/322549430 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb61800fc30 con 0x5624ec4d4d40 2024-09-12T03:11:52.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.296+0000 7fb6067fc640 1 -- 172.21.15.116:0/322549430 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb618022030 con 0x5624ec4d4d40 2024-09-12T03:11:52.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.296+0000 7fb62753c740 10 client.17688 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:52.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.296+0000 7fb62753c740 1 -- 172.21.15.116:0/322549430 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5624ec4d4ad0 con 0x5624ec4d4d40 2024-09-12T03:11:52.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.296+0000 7fb6067fc640 1 -- 172.21.15.116:0/322549430 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb61800fc30 con 0x5624ec4d4d40 2024-09-12T03:11:52.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.297+0000 7fb6067fc640 1 -- 172.21.15.116:0/322549430 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb61801a040 con 0x5624ec4d4d40 2024-09-12T03:11:52.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.297+0000 7fb62753c740 10 client.17688 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:52.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.297+0000 7fb62753c740 10 client.17688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:11:52.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.297+0000 7fb62753c740 20 client.17688 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:52.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.297+0000 7fb62753c740 1 --2- 172.21.15.116:0/322549430 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5624ec5506b0 0x5624ec552aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:52.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.297+0000 7fb62753c740 4 client.17688 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:11:52.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.297+0000 7fb62753c740 1 -- 172.21.15.116:0/322549430 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5624ec436d00 con 0x5624ec5506b0 2024-09-12T03:11:52.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.297+0000 7fb61ed9d640 1 --2- 172.21.15.116:0/322549430 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5624ec5506b0 0x5624ec552aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:52.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.297+0000 7fb61ed9d640 1 --2- 172.21.15.116:0/322549430 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5624ec5506b0 0x5624ec552aa0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c14986daff46630f server_cookie=6fb2358d7aec9e38 in_seq=0 out_seq=0 2024-09-12T03:11:52.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.297+0000 7fb6067fc640 10 client.17688 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:52.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.298+0000 7fb6067fc640 1 -- 172.21.15.116:0/322549430 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5624ec436d00 con 0x5624ec5506b0 2024-09-12T03:11:52.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.298+0000 7fb6067fc640 10 client.17688 handle_command_reply: tid=0 2024-09-12T03:11:52.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.298+0000 7fb62753c740 1 client.17688 shutdown 2024-09-12T03:11:52.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.299+0000 7fb62753c740 1 -- 172.21.15.116:0/322549430 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624ec4d4d40 msgr2=0x5624ec4d81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:52.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.299+0000 7fb62753c740 1 --2- 172.21.15.116:0/322549430 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624ec4d4d40 0x5624ec4d81e0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fb618007aa0 tx=0x7fb618026ea0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.299+0000 7fb62753c740 1 -- 172.21.15.116:0/322549430 shutdown_connections 2024-09-12T03:11:52.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.299+0000 7fb62753c740 1 --2- 172.21.15.116:0/322549430 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5624ec5506b0 0x5624ec552aa0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.299+0000 7fb6067fc640 0 client.17688 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:52.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.299+0000 7fb62753c740 1 --2- 172.21.15.116:0/322549430 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5624ec4d4d40 0x5624ec4d81e0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.299+0000 7fb62753c740 1 --2- 172.21.15.116:0/322549430 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5624ec4ca480 0x5624ec4d7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.299+0000 7fb62753c740 1 --2- 172.21.15.116:0/322549430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5624ec4c9b60 0x5624ec4dd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.299+0000 7fb62753c740 1 -- 172.21.15.116:0/322549430 >> 172.21.15.116:0/322549430 conn(0x5624ec4c60f0 msgr2=0x5624ec4c5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:52.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.299+0000 7fb62753c740 1 -- 172.21.15.116:0/322549430 shutdown_connections 2024-09-12T03:11:52.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.299+0000 7fb62753c740 1 -- 172.21.15.116:0/322549430 wait complete. 2024-09-12T03:11:52.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.299+0000 7fb62753c740 20 client.17688 trim_cache size 0 max 16384 2024-09-12T03:11:52.311 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:11:52.311 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:11:52.311 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "183a0b61-d35b-4717-9cc0-3dab667f3885", 2024-09-12T03:11:52.311 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:11:52.311 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:11:52.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.300+0000 7fb625163640 1 -- 172.21.15.116:0/1280944823 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb60c0a2b80 msgr2=0x7fb60c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:52.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.300+0000 7fb625163640 1 --2- 172.21.15.116:0/1280944823 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb60c0a2b80 0x7fb60c0a5040 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb618007310 tx=0x7fb6180072a0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.300+0000 7fb625163640 1 -- 172.21.15.116:0/1280944823 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb620101de0 msgr2=0x7fb6201983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:52.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.300+0000 7fb625163640 1 --2- 172.21.15.116:0/1280944823 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb620101de0 0x7fb6201983d0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fb610006010 tx=0x7fb6100042f0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.300+0000 7fb625163640 1 -- 172.21.15.116:0/1280944823 shutdown_connections 2024-09-12T03:11:52.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.300+0000 7fb625163640 1 --2- 172.21.15.116:0/1280944823 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb60c0a2b80 0x7fb60c0a5040 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.300+0000 7fb625163640 1 --2- 172.21.15.116:0/1280944823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb620106f10 0x7fb62019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.300+0000 7fb625163640 1 --2- 172.21.15.116:0/1280944823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb620102790 0x7fb620198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.300+0000 7fb625163640 1 --2- 172.21.15.116:0/1280944823 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb620101de0 0x7fb6201983d0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:52.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.300+0000 7fb625163640 1 -- 172.21.15.116:0/1280944823 >> 172.21.15.116:0/1280944823 conn(0x7fb6200fd8c0 msgr2=0x7fb6200febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:52.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.300+0000 7fb625163640 1 -- 172.21.15.116:0/1280944823 shutdown_connections 2024-09-12T03:11:52.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:52.300+0000 7fb625163640 1 -- 172.21.15.116:0/1280944823 wait complete. 2024-09-12T03:11:52.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:52 smithi116 ceph-mon[31410]: pgmap v604: 97 pgs: 97 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:11:52.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:52 smithi183 ceph-mon[35535]: pgmap v604: 97 pgs: 97 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:11:52.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:52 smithi177 ceph-mon[35305]: pgmap v604: 97 pgs: 97 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:11:53.168 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:11:53.418 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:11:53.539 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:53 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:11:53.539 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:53 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:11:53.539 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:53 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:11:53.539 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:53 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:11:53.539 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:53 smithi116 ceph-mon[31410]: pgmap v605: 97 pgs: 97 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:11:53.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:53 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:11:53.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:53 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:11:53.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:53 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:11:53.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:53 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:11:53.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:53 smithi183 ceph-mon[35535]: pgmap v605: 97 pgs: 97 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:11:53.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:53 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:11:53.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:53 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:11:53.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:53 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:11:53.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:53 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:11:53.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:53 smithi177 ceph-mon[35305]: pgmap v605: 97 pgs: 97 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:11:53.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.881+0000 7f09ea18e640 1 -- 172.21.15.116:0/2746902612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09e4102ef0 msgr2=0x7f09e410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:53.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.881+0000 7f09ea18e640 1 --2- 172.21.15.116:0/2746902612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09e4102ef0 0x7f09e410da40 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f09dc009ca0 tx=0x7f09dc02a950 comp rx=0 tx=0).stop 2024-09-12T03:11:53.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.881+0000 7f09ea18e640 1 -- 172.21.15.116:0/2746902612 shutdown_connections 2024-09-12T03:11:53.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.881+0000 7f09ea18e640 1 --2- 172.21.15.116:0/2746902612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09e4102ef0 0x7f09e410da40 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.881+0000 7f09ea18e640 1 --2- 172.21.15.116:0/2746902612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09e4102570 0x7f09e41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.881+0000 7f09ea18e640 1 --2- 172.21.15.116:0/2746902612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09e4100e20 0x7f09e4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.881+0000 7f09ea18e640 1 -- 172.21.15.116:0/2746902612 >> 172.21.15.116:0/2746902612 conn(0x7f09e40fc7d0 msgr2=0x7f09e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:53.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.881+0000 7f09ea18e640 1 -- 172.21.15.116:0/2746902612 shutdown_connections 2024-09-12T03:11:53.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.881+0000 7f09ea18e640 1 -- 172.21.15.116:0/2746902612 wait complete. 2024-09-12T03:11:53.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.881+0000 7f09ea18e640 1 Processor -- start 2024-09-12T03:11:53.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.881+0000 7f09ea18e640 1 -- start start 2024-09-12T03:11:53.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.882+0000 7f09ea18e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09e4100e20 0x7f09e41992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.882+0000 7f09ea18e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09e4102570 0x7f09e41997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.882+0000 7f09ea18e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09e4102ef0 0x7f09e41a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.882+0000 7f09ea18e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f09e4110fa0 con 0x7f09e4100e20 2024-09-12T03:11:53.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.882+0000 7f09ea18e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f09e4110e20 con 0x7f09e4102570 2024-09-12T03:11:53.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.882+0000 7f09ea18e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f09e4111120 con 0x7f09e4102ef0 2024-09-12T03:11:53.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.882+0000 7f09e37fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09e4100e20 0x7f09e41992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.882+0000 7f09e37fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09e4100e20 0x7f09e41992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48428/0 (socket says 172.21.15.116:48428) 2024-09-12T03:11:53.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.882+0000 7f09e37fe640 1 -- 172.21.15.116:0/1861824955 learned_addr learned my addr 172.21.15.116:0/1861824955 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:53.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.882+0000 7f09e3fff640 1 --2- 172.21.15.116:0/1861824955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09e4102ef0 0x7f09e41a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.882+0000 7f09daffd640 1 --2- 172.21.15.116:0/1861824955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09e4102570 0x7f09e41997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.882+0000 7f09e37fe640 1 -- 172.21.15.116:0/1861824955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09e4102ef0 msgr2=0x7f09e41a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:53.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.882+0000 7f09e37fe640 1 --2- 172.21.15.116:0/1861824955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09e4102ef0 0x7f09e41a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.882+0000 7f09e37fe640 1 -- 172.21.15.116:0/1861824955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09e4102570 msgr2=0x7f09e41997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:53.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.882+0000 7f09e37fe640 1 --2- 172.21.15.116:0/1861824955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09e4102570 0x7f09e41997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.882+0000 7f09e37fe640 1 -- 172.21.15.116:0/1861824955 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e41a4860 con 0x7f09e4100e20 2024-09-12T03:11:53.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.882+0000 7f09e3fff640 1 --2- 172.21.15.116:0/1861824955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09e4102ef0 0x7f09e41a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:11:53.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.882+0000 7f09e37fe640 1 --2- 172.21.15.116:0/1861824955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09e4100e20 0x7f09e41992a0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f09d400c920 tx=0x7f09d400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:53.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.882+0000 7f09e17fa640 1 -- 172.21.15.116:0/1861824955 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d4007c70 con 0x7f09e4100e20 2024-09-12T03:11:53.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.882+0000 7f09e17fa640 1 -- 172.21.15.116:0/1861824955 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09d4007e10 con 0x7f09e4100e20 2024-09-12T03:11:53.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.882+0000 7f09ea18e640 1 -- 172.21.15.116:0/1861824955 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e41a4a30 con 0x7f09e4100e20 2024-09-12T03:11:53.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.882+0000 7f09e17fa640 1 -- 172.21.15.116:0/1861824955 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d4018890 con 0x7f09e4100e20 2024-09-12T03:11:53.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.883+0000 7f09ea18e640 1 -- 172.21.15.116:0/1861824955 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e41a4f40 con 0x7f09e4100e20 2024-09-12T03:11:53.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.885+0000 7f09e17fa640 1 -- 172.21.15.116:0/1861824955 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f09d4018a70 con 0x7f09e4100e20 2024-09-12T03:11:53.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.885+0000 7f09ec567740 1 Processor -- start 2024-09-12T03:11:53.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.886+0000 7f09ec567740 1 -- start start 2024-09-12T03:11:53.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.886+0000 7f09ec567740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9eff7c380 0x55c9eff7c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.886+0000 7f09ec567740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9eff7cca0 0x55c9eff8b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.886+0000 7f09ec567740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9eff8ba90 0x55c9eff8de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.886+0000 7f09ec567740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c9efee22e0 con 0x55c9eff8ba90 2024-09-12T03:11:53.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.886+0000 7f09ec567740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c9efe89ce0 con 0x55c9eff7c380 2024-09-12T03:11:53.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.886+0000 7f09ec567740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c9efede2a0 con 0x55c9eff7cca0 2024-09-12T03:11:53.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.886+0000 7f09daffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9eff8ba90 0x55c9eff8de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.886+0000 7f09daffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9eff8ba90 0x55c9eff8de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48432/0 (socket says 172.21.15.116:48432) 2024-09-12T03:11:53.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.886+0000 7f09daffd640 1 -- 172.21.15.116:0/3719138494 learned_addr learned my addr 172.21.15.116:0/3719138494 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:53.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.886+0000 7f09e3fff640 1 --2- 172.21.15.116:0/3719138494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9eff7c380 0x55c9eff7c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.886+0000 7f09e37fe640 1 --2- 172.21.15.116:0/3719138494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9eff7cca0 0x55c9eff8b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.886+0000 7f09daffd640 1 -- 172.21.15.116:0/3719138494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9eff7cca0 msgr2=0x55c9eff8b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:53.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.886+0000 7f09daffd640 1 --2- 172.21.15.116:0/3719138494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9eff7cca0 0x55c9eff8b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.886+0000 7f09daffd640 1 -- 172.21.15.116:0/3719138494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9eff7c380 msgr2=0x55c9eff7c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:53.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.886+0000 7f09daffd640 1 --2- 172.21.15.116:0/3719138494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9eff7c380 0x55c9eff7c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.886+0000 7f09daffd640 1 -- 172.21.15.116:0/3719138494 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9efe6e440 con 0x55c9eff8ba90 2024-09-12T03:11:53.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.886+0000 7f09e17fa640 1 --2- 172.21.15.116:0/1861824955 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f09d00a2b80 0x7f09d00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.886+0000 7f09daffd640 1 --2- 172.21.15.116:0/3719138494 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9eff8ba90 0x55c9eff8de80 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00b7f0 tx=0x7f09cc00bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946d16d2ba38f5c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:53.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.886+0000 7f09e3fff640 1 --2- 172.21.15.116:0/1861824955 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f09d00a2b80 0x7f09d00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.887+0000 7f09e17fa640 1 -- 172.21.15.116:0/1861824955 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f09d40cf210 con 0x7f09e4100e20 2024-09-12T03:11:53.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.887+0000 7f09d9ffb640 1 -- 172.21.15.116:0/3719138494 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc00d950 con 0x55c9eff8ba90 2024-09-12T03:11:53.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.887+0000 7f09d9ffb640 1 -- 172.21.15.116:0/3719138494 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09cc018070 con 0x55c9eff8ba90 2024-09-12T03:11:53.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.887+0000 7f09d9ffb640 1 -- 172.21.15.116:0/3719138494 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc0123e0 con 0x55c9eff8ba90 2024-09-12T03:11:53.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.887+0000 7f09ec567740 1 -- 172.21.15.116:0/3719138494 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9eff8ba90 msgr2=0x55c9eff8de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:53.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.887+0000 7f09e3fff640 1 --2- 172.21.15.116:0/1861824955 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f09d00a2b80 0x7f09d00a5040 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f09dc009430 tx=0x7f09dc009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:53.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.887+0000 7f09ec567740 1 --2- 172.21.15.116:0/3719138494 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9eff8ba90 0x55c9eff8de80 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00b7f0 tx=0x7f09cc00bcb0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.887+0000 7f09ec567740 1 -- 172.21.15.116:0/3719138494 shutdown_connections 2024-09-12T03:11:53.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.887+0000 7f09ec567740 1 --2- 172.21.15.116:0/3719138494 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9eff8ba90 0x55c9eff8de80 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.887+0000 7f09ec567740 1 --2- 172.21.15.116:0/3719138494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9eff7cca0 0x55c9eff8b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.887+0000 7f09ec567740 1 --2- 172.21.15.116:0/3719138494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9eff7c380 0x55c9eff7c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.887+0000 7f09ec567740 1 -- 172.21.15.116:0/3719138494 >> 172.21.15.116:0/3719138494 conn(0x55c9eff78a30 msgr2=0x55c9eff78e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:53.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.887+0000 7f09ec567740 1 -- 172.21.15.116:0/3719138494 shutdown_connections 2024-09-12T03:11:53.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.887+0000 7f09ec567740 1 -- 172.21.15.116:0/3719138494 wait complete. 2024-09-12T03:11:53.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.888+0000 7f09ec567740 1 -- start start 2024-09-12T03:11:53.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.888+0000 7f09ec567740 1 Processor -- start 2024-09-12T03:11:53.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.888+0000 7f09ec567740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9eff7c380 0x55c9eff8b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.888+0000 7f09ec567740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9eff7cca0 0x55c9eff88650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.888+0000 7f09ec567740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9eff8ba90 0x55c9eff88b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.888+0000 7f09ec567740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c9eff8e660 con 0x55c9eff7cca0 2024-09-12T03:11:53.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.888+0000 7f09ec567740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c9efe9d050 con 0x55c9eff8ba90 2024-09-12T03:11:53.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.888+0000 7f09ec567740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c9eff8e7e0 con 0x55c9eff7c380 2024-09-12T03:11:53.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.888+0000 7f09ec567740 10 client.0 _ll_register_callbacks cb 0x55c9efedd160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:53.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.888+0000 7f09daffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9eff7cca0 0x55c9eff88650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.888+0000 7f09daffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9eff7cca0 0x55c9eff88650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48438/0 (socket says 172.21.15.116:48438) 2024-09-12T03:11:53.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.888+0000 7f09daffd640 1 -- 172.21.15.116:0/3726232253 learned_addr learned my addr 172.21.15.116:0/3726232253 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:53.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.888+0000 7f09e3fff640 1 --2- 172.21.15.116:0/3726232253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9eff8ba90 0x55c9eff88b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.888+0000 7f09e37fe640 1 --2- 172.21.15.116:0/3726232253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9eff7c380 0x55c9eff8b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.889+0000 7f09daffd640 1 -- 172.21.15.116:0/3726232253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9eff7c380 msgr2=0x55c9eff8b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:53.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.889+0000 7f09daffd640 1 --2- 172.21.15.116:0/3726232253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9eff7c380 0x55c9eff8b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.889+0000 7f09daffd640 1 -- 172.21.15.116:0/3726232253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9eff8ba90 msgr2=0x55c9eff88b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:53.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.889+0000 7f09daffd640 1 --2- 172.21.15.116:0/3726232253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9eff8ba90 0x55c9eff88b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.889+0000 7f09daffd640 1 -- 172.21.15.116:0/3726232253 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9efc533c0 con 0x55c9eff7cca0 2024-09-12T03:11:53.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.889+0000 7f09e37fe640 1 --2- 172.21.15.116:0/3726232253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9eff7c380 0x55c9eff8b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:11:53.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.889+0000 7f09daffd640 1 --2- 172.21.15.116:0/3726232253 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9eff7cca0 0x55c9eff88650 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f09cc008010 tx=0x7f09cc0136b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a33506094f75c64 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:53.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.889+0000 7f09c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:53.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.889+0000 7f09c2ffd640 1 -- 172.21.15.116:0/3726232253 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc023070 con 0x55c9eff7cca0 2024-09-12T03:11:53.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.889+0000 7f09ec567740 1 -- 172.21.15.116:0/3726232253 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09cc018070 con 0x55c9eff7cca0 2024-09-12T03:11:53.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.889+0000 7f09c2ffd640 1 -- 172.21.15.116:0/3726232253 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f09cc00dd60 con 0x55c9eff7cca0 2024-09-12T03:11:53.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.889+0000 7f09c2ffd640 1 -- 172.21.15.116:0/3726232253 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f09cc02f050 con 0x55c9eff7cca0 2024-09-12T03:11:53.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.889+0000 7f09ec567740 10 client.17700 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:53.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.889+0000 7f09ec567740 1 -- 172.21.15.116:0/3726232253 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c9eff9a050 con 0x55c9eff7cca0 2024-09-12T03:11:53.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.889+0000 7f09c2ffd640 1 -- 172.21.15.116:0/3726232253 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09cc018070 con 0x55c9eff7cca0 2024-09-12T03:11:53.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.889+0000 7f09c2ffd640 1 -- 172.21.15.116:0/3726232253 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f09cc01b380 con 0x55c9eff7cca0 2024-09-12T03:11:53.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.889+0000 7f09ec567740 10 client.17700 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:53.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.889+0000 7f09ec567740 10 client.17700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:11:53.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.889+0000 7f09ec567740 20 client.17700 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:53.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.889+0000 7f09ec567740 1 --2- 172.21.15.116:0/3726232253 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c9eff9b400 0x55c9eff9d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.889+0000 7f09ec567740 4 client.17700 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:11:53.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.889+0000 7f09ec567740 1 -- 172.21.15.116:0/3726232253 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9efeef850 con 0x55c9eff9b400 2024-09-12T03:11:53.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.890+0000 7f09e37fe640 1 --2- 172.21.15.116:0/3726232253 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c9eff9b400 0x55c9eff9d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.890+0000 7f09c2ffd640 10 client.17700 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:53.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.890+0000 7f09e37fe640 1 --2- 172.21.15.116:0/3726232253 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c9eff9b400 0x55c9eff9d7f0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aae508809990f321 server_cookie=886216d59e0c31a7 in_seq=0 out_seq=0 2024-09-12T03:11:53.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.891+0000 7f09c2ffd640 1 -- 172.21.15.116:0/3726232253 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c9efeef850 con 0x55c9eff9b400 2024-09-12T03:11:53.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.891+0000 7f09c2ffd640 10 client.17700 handle_command_reply: tid=0 2024-09-12T03:11:53.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.891+0000 7f09ec567740 1 client.17700 shutdown 2024-09-12T03:11:53.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.891+0000 7f09ec567740 1 -- 172.21.15.116:0/3726232253 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9eff7cca0 msgr2=0x55c9eff88650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:53.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.891+0000 7f09ec567740 1 --2- 172.21.15.116:0/3726232253 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9eff7cca0 0x55c9eff88650 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f09cc008010 tx=0x7f09cc0136b0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.891+0000 7f09ec567740 1 -- 172.21.15.116:0/3726232253 shutdown_connections 2024-09-12T03:11:53.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.891+0000 7f09ec567740 1 --2- 172.21.15.116:0/3726232253 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c9eff9b400 0x55c9eff9d7f0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.891+0000 7f09ec567740 1 --2- 172.21.15.116:0/3726232253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9eff8ba90 0x55c9eff88b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.891+0000 7f09c2ffd640 0 client.17700 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:53.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.891+0000 7f09ec567740 1 --2- 172.21.15.116:0/3726232253 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9eff7cca0 0x55c9eff88650 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.891+0000 7f09ec567740 1 --2- 172.21.15.116:0/3726232253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9eff7c380 0x55c9eff8b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.891+0000 7f09ec567740 1 -- 172.21.15.116:0/3726232253 >> 172.21.15.116:0/3726232253 conn(0x55c9eff78a30 msgr2=0x55c9eff7e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:53.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.891+0000 7f09ec567740 1 -- 172.21.15.116:0/3726232253 shutdown_connections 2024-09-12T03:11:53.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.891+0000 7f09ec567740 1 -- 172.21.15.116:0/3726232253 wait complete. 2024-09-12T03:11:53.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.891+0000 7f09ec567740 20 client.17700 trim_cache size 0 max 16384 2024-09-12T03:11:53.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.908+0000 7f09ec567740 1 Processor -- start 2024-09-12T03:11:53.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.908+0000 7f09ec567740 1 -- start start 2024-09-12T03:11:53.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.908+0000 7f09ec567740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9eff7c380 0x55c9eff97930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.908+0000 7f09ec567740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9eff7cca0 0x55c9effa2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.908+0000 7f09ec567740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9eff8ba90 0x55c9effa3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.908+0000 7f09ec567740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f09e41adb30 con 0x55c9eff7cca0 2024-09-12T03:11:53.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.908+0000 7f09ec567740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f09e41ab400 con 0x55c9eff7c380 2024-09-12T03:11:53.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.908+0000 7f09ec567740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f09e41a9270 con 0x55c9eff8ba90 2024-09-12T03:11:53.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.908+0000 7f09daffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9eff7cca0 0x55c9effa2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.908+0000 7f09daffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9eff7cca0 0x55c9effa2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48448/0 (socket says 172.21.15.116:48448) 2024-09-12T03:11:53.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.908+0000 7f09daffd640 1 -- 172.21.15.116:0/549464305 learned_addr learned my addr 172.21.15.116:0/549464305 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:53.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.909+0000 7f09e37fe640 1 --2- 172.21.15.116:0/549464305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9eff7c380 0x55c9eff97930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.909+0000 7f09e3fff640 1 --2- 172.21.15.116:0/549464305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9eff8ba90 0x55c9effa3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.909+0000 7f09daffd640 1 -- 172.21.15.116:0/549464305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9eff8ba90 msgr2=0x55c9effa3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:53.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.909+0000 7f09daffd640 1 --2- 172.21.15.116:0/549464305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9eff8ba90 0x55c9effa3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.909+0000 7f09daffd640 1 -- 172.21.15.116:0/549464305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9eff7c380 msgr2=0x55c9eff97930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:53.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.909+0000 7f09daffd640 1 --2- 172.21.15.116:0/549464305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9eff7c380 0x55c9eff97930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.909+0000 7f09daffd640 1 -- 172.21.15.116:0/549464305 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9eff8e7e0 con 0x55c9eff7cca0 2024-09-12T03:11:53.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.909+0000 7f09daffd640 1 --2- 172.21.15.116:0/549464305 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9eff7cca0 0x55c9effa2e20 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f09cc01b780 tx=0x7f09cc001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960cfbbd849522bc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:53.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.909+0000 7f09d97fa640 1 -- 172.21.15.116:0/549464305 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc02c070 con 0x55c9eff7cca0 2024-09-12T03:11:53.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.909+0000 7f09d97fa640 1 -- 172.21.15.116:0/549464305 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09cc018040 con 0x55c9eff7cca0 2024-09-12T03:11:53.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.909+0000 7f09d97fa640 1 -- 172.21.15.116:0/549464305 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc023070 con 0x55c9eff7cca0 2024-09-12T03:11:53.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.910+0000 7f09ec567740 1 -- 172.21.15.116:0/549464305 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9eff7cca0 msgr2=0x55c9effa2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:53.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.910+0000 7f09ec567740 1 --2- 172.21.15.116:0/549464305 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9eff7cca0 0x55c9effa2e20 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f09cc01b780 tx=0x7f09cc001e70 comp rx=0 tx=0).stop 2024-09-12T03:11:53.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.910+0000 7f09ec567740 1 -- 172.21.15.116:0/549464305 shutdown_connections 2024-09-12T03:11:53.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.910+0000 7f09ec567740 1 --2- 172.21.15.116:0/549464305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9eff8ba90 0x55c9effa3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.910+0000 7f09ec567740 1 --2- 172.21.15.116:0/549464305 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9eff7cca0 0x55c9effa2e20 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.910+0000 7f09ec567740 1 --2- 172.21.15.116:0/549464305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9eff7c380 0x55c9eff97930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.910+0000 7f09ec567740 1 -- 172.21.15.116:0/549464305 >> 172.21.15.116:0/549464305 conn(0x55c9eff78a30 msgr2=0x55c9eff87740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:53.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.910+0000 7f09ec567740 1 -- 172.21.15.116:0/549464305 shutdown_connections 2024-09-12T03:11:53.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.910+0000 7f09ec567740 1 -- 172.21.15.116:0/549464305 wait complete. 2024-09-12T03:11:53.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.910+0000 7f09ec567740 1 -- start start 2024-09-12T03:11:53.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.910+0000 7f09ec567740 1 Processor -- start 2024-09-12T03:11:53.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.911+0000 7f09ec567740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9eff7c380 0x55c9eff8f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.911+0000 7f09ec567740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9eff7cca0 0x55c9eff8fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.911+0000 7f09ec567740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9eff8ba90 0x55c9eff90210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.911+0000 7f09ec567740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f09e41a98c0 con 0x55c9eff8ba90 2024-09-12T03:11:53.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.911+0000 7f09ec567740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f09e41aad30 con 0x55c9eff7c380 2024-09-12T03:11:53.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.911+0000 7f09ec567740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f09e41a9f90 con 0x55c9eff7cca0 2024-09-12T03:11:53.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.911+0000 7f09ec567740 10 client.0 _ll_register_callbacks cb 0x7f09dc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:53.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.911+0000 7f09e3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9eff8ba90 0x55c9eff90210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.911+0000 7f09e3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9eff8ba90 0x55c9eff90210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48452/0 (socket says 172.21.15.116:48452) 2024-09-12T03:11:53.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.911+0000 7f09e3fff640 1 -- 172.21.15.116:0/1509647428 learned_addr learned my addr 172.21.15.116:0/1509647428 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:53.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.911+0000 7f09e37fe640 1 --2- 172.21.15.116:0/1509647428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9eff7c380 0x55c9eff8f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.911+0000 7f09daffd640 1 --2- 172.21.15.116:0/1509647428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9eff7cca0 0x55c9eff8fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.911+0000 7f09e3fff640 1 -- 172.21.15.116:0/1509647428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9eff7cca0 msgr2=0x55c9eff8fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:53.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.911+0000 7f09e3fff640 1 --2- 172.21.15.116:0/1509647428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9eff7cca0 0x55c9eff8fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.911+0000 7f09e3fff640 1 -- 172.21.15.116:0/1509647428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9eff7c380 msgr2=0x55c9eff8f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:53.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.911+0000 7f09e3fff640 1 --2- 172.21.15.116:0/1509647428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9eff7c380 0x55c9eff8f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.911+0000 7f09e3fff640 1 -- 172.21.15.116:0/1509647428 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9f0004df0 con 0x55c9eff8ba90 2024-09-12T03:11:53.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.911+0000 7f09e3fff640 1 --2- 172.21.15.116:0/1509647428 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9eff8ba90 0x55c9eff90210 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f09dc035000 tx=0x7f09dc02a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2eeeeb8fd3c7135 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:53.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.911+0000 7f09c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:53.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.911+0000 7f09c2ffd640 1 -- 172.21.15.116:0/1509647428 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc034040 con 0x55c9eff8ba90 2024-09-12T03:11:53.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.911+0000 7f09c2ffd640 1 -- 172.21.15.116:0/1509647428 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f09dc041030 con 0x55c9eff8ba90 2024-09-12T03:11:53.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.912+0000 7f09ec567740 1 -- 172.21.15.116:0/1509647428 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09cc018040 con 0x55c9eff8ba90 2024-09-12T03:11:53.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.912+0000 7f09c2ffd640 1 -- 172.21.15.116:0/1509647428 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f09dc004000 con 0x55c9eff8ba90 2024-09-12T03:11:53.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.912+0000 7f09ec567740 10 client.17706 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:53.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.912+0000 7f09ec567740 1 -- 172.21.15.116:0/1509647428 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c9f0006080 con 0x55c9eff8ba90 2024-09-12T03:11:53.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.912+0000 7f09c2ffd640 1 -- 172.21.15.116:0/1509647428 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09cc018040 con 0x55c9eff8ba90 2024-09-12T03:11:53.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.912+0000 7f09c2ffd640 1 -- 172.21.15.116:0/1509647428 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f09dc046980 con 0x55c9eff8ba90 2024-09-12T03:11:53.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.912+0000 7f09ec567740 10 client.17706 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:53.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.912+0000 7f09ec567740 10 client.17706 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:11:53.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.912+0000 7f09ec567740 20 client.17706 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:53.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.912+0000 7f09ec567740 1 --2- 172.21.15.116:0/1509647428 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c9f0006850 0x55c9f0008c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.912+0000 7f09ec567740 4 client.17706 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:11:53.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.912+0000 7f09ec567740 1 -- 172.21.15.116:0/1509647428 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c9efee9d60 con 0x55c9f0006850 2024-09-12T03:11:53.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.912+0000 7f09e37fe640 1 --2- 172.21.15.116:0/1509647428 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c9f0006850 0x55c9f0008c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.913+0000 7f09c2ffd640 10 client.17706 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:53.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.913+0000 7f09e37fe640 1 --2- 172.21.15.116:0/1509647428 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c9f0006850 0x55c9f0008c70 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2b5bd2f885bb8ea server_cookie=5eab6f79179acd54 in_seq=0 out_seq=0 2024-09-12T03:11:53.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.913+0000 7f09c2ffd640 1 -- 172.21.15.116:0/1509647428 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c9efee9d60 con 0x55c9f0006850 2024-09-12T03:11:53.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.913+0000 7f09c2ffd640 10 client.17706 handle_command_reply: tid=0 2024-09-12T03:11:53.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.913+0000 7f09ec567740 1 client.17706 shutdown 2024-09-12T03:11:53.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.913+0000 7f09ec567740 1 -- 172.21.15.116:0/1509647428 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9eff8ba90 msgr2=0x55c9eff90210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:53.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.913+0000 7f09ec567740 1 --2- 172.21.15.116:0/1509647428 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9eff8ba90 0x55c9eff90210 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f09dc035000 tx=0x7f09dc02a950 comp rx=0 tx=0).stop 2024-09-12T03:11:53.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.913+0000 7f09ec567740 1 -- 172.21.15.116:0/1509647428 shutdown_connections 2024-09-12T03:11:53.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.913+0000 7f09ec567740 1 --2- 172.21.15.116:0/1509647428 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c9f0006850 0x55c9f0008c70 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.913+0000 7f09ec567740 1 --2- 172.21.15.116:0/1509647428 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c9eff8ba90 0x55c9eff90210 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.913+0000 7f09ec567740 1 --2- 172.21.15.116:0/1509647428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9eff7cca0 0x55c9eff8fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.913+0000 7f09ec567740 1 --2- 172.21.15.116:0/1509647428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c9eff7c380 0x55c9eff8f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.913+0000 7f09c2ffd640 0 client.17706 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:11:53.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.913+0000 7f09ec567740 1 -- 172.21.15.116:0/1509647428 >> 172.21.15.116:0/1509647428 conn(0x55c9eff78a30 msgr2=0x55c9eff89eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:53.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.913+0000 7f09ec567740 1 -- 172.21.15.116:0/1509647428 shutdown_connections 2024-09-12T03:11:53.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.913+0000 7f09ec567740 1 -- 172.21.15.116:0/1509647428 wait complete. 2024-09-12T03:11:53.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.913+0000 7f09ec567740 20 client.17706 trim_cache size 0 max 16384 2024-09-12T03:11:53.921 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:11:53.922 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:11:53.922 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:11:53.922 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:11:53.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.914+0000 7f09ea18e640 1 -- 172.21.15.116:0/1861824955 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f09d00a2b80 msgr2=0x7f09d00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:53.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.914+0000 7f09ea18e640 1 --2- 172.21.15.116:0/1861824955 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f09d00a2b80 0x7f09d00a5040 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f09dc009430 tx=0x7f09dc009360 comp rx=0 tx=0).stop 2024-09-12T03:11:53.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.914+0000 7f09ea18e640 1 -- 172.21.15.116:0/1861824955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09e4100e20 msgr2=0x7f09e41992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:53.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.914+0000 7f09ea18e640 1 --2- 172.21.15.116:0/1861824955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09e4100e20 0x7f09e41992a0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f09d400c920 tx=0x7f09d400cde0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.914+0000 7f09ea18e640 1 -- 172.21.15.116:0/1861824955 shutdown_connections 2024-09-12T03:11:53.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.914+0000 7f09ea18e640 1 --2- 172.21.15.116:0/1861824955 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f09d00a2b80 0x7f09d00a5040 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.914+0000 7f09ea18e640 1 --2- 172.21.15.116:0/1861824955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09e4102ef0 0x7f09e41a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.914+0000 7f09ea18e640 1 --2- 172.21.15.116:0/1861824955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09e4102570 0x7f09e41997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.914+0000 7f09ea18e640 1 --2- 172.21.15.116:0/1861824955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09e4100e20 0x7f09e41992a0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.914+0000 7f09ea18e640 1 -- 172.21.15.116:0/1861824955 >> 172.21.15.116:0/1861824955 conn(0x7f09e40fc7d0 msgr2=0x7f09e4106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:53.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.914+0000 7f09ea18e640 1 -- 172.21.15.116:0/1861824955 shutdown_connections 2024-09-12T03:11:53.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:53.914+0000 7f09ea18e640 1 -- 172.21.15.116:0/1861824955 wait complete. 2024-09-12T03:11:54.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:54 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:11:54.623 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:11:54.623 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:11:54.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:54 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:11:54.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:54 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:11:54.868 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:11:55.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.345+0000 7f9bab4f4640 1 -- 172.21.15.116:0/1369827885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ba4102790 msgr2=0x7f9ba41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:55.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.345+0000 7f9bab4f4640 1 --2- 172.21.15.116:0/1369827885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ba4102790 0x7f9ba41067e0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f9b98009900 tx=0x7f9b9802a410 comp rx=0 tx=0).stop 2024-09-12T03:11:55.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.345+0000 7f9bab4f4640 1 -- 172.21.15.116:0/1369827885 shutdown_connections 2024-09-12T03:11:55.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.345+0000 7f9bab4f4640 1 --2- 172.21.15.116:0/1369827885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ba4106f10 0x7f9ba410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:55.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.345+0000 7f9bab4f4640 1 --2- 172.21.15.116:0/1369827885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ba4102790 0x7f9ba41067e0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:55.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.345+0000 7f9bab4f4640 1 --2- 172.21.15.116:0/1369827885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ba4101de0 0x7f9ba41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:55.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.345+0000 7f9bab4f4640 1 -- 172.21.15.116:0/1369827885 >> 172.21.15.116:0/1369827885 conn(0x7f9ba40fd8c0 msgr2=0x7f9ba40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:55.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.345+0000 7f9bab4f4640 1 -- 172.21.15.116:0/1369827885 shutdown_connections 2024-09-12T03:11:55.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.345+0000 7f9bab4f4640 1 -- 172.21.15.116:0/1369827885 wait complete. 2024-09-12T03:11:55.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.346+0000 7f9bab4f4640 1 Processor -- start 2024-09-12T03:11:55.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.346+0000 7f9bab4f4640 1 -- start start 2024-09-12T03:11:55.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.346+0000 7f9bab4f4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ba4101de0 0x7f9ba41984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:55.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.346+0000 7f9bab4f4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ba4102790 0x7f9ba41989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:55.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.346+0000 7f9bab4f4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ba4106f10 0x7f9ba419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:55.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.346+0000 7f9bab4f4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba4111fb0 con 0x7f9ba4106f10 2024-09-12T03:11:55.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.346+0000 7f9bab4f4640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba4111e30 con 0x7f9ba4101de0 2024-09-12T03:11:55.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.346+0000 7f9bab4f4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba4112130 con 0x7f9ba4102790 2024-09-12T03:11:55.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.346+0000 7f9ba9a91640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ba4106f10 0x7f9ba419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:55.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.346+0000 7f9ba9a91640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ba4106f10 0x7f9ba419cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48458/0 (socket says 172.21.15.116:48458) 2024-09-12T03:11:55.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.346+0000 7f9ba9a91640 1 -- 172.21.15.116:0/3356223422 learned_addr learned my addr 172.21.15.116:0/3356223422 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:55.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.346+0000 7f9ba9290640 1 --2- 172.21.15.116:0/3356223422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ba4101de0 0x7f9ba41984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:55.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.346+0000 7f9ba8a8f640 1 --2- 172.21.15.116:0/3356223422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ba4102790 0x7f9ba41989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:55.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.347+0000 7f9ba9a91640 1 -- 172.21.15.116:0/3356223422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ba4102790 msgr2=0x7f9ba41989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:55.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.347+0000 7f9ba9a91640 1 --2- 172.21.15.116:0/3356223422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ba4102790 0x7f9ba41989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:55.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.347+0000 7f9ba9a91640 1 -- 172.21.15.116:0/3356223422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ba4101de0 msgr2=0x7f9ba41984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:55.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.347+0000 7f9ba9a91640 1 --2- 172.21.15.116:0/3356223422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ba4101de0 0x7f9ba41984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:55.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.347+0000 7f9ba9a91640 1 -- 172.21.15.116:0/3356223422 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba419d460 con 0x7f9ba4106f10 2024-09-12T03:11:55.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.347+0000 7f9ba9290640 1 --2- 172.21.15.116:0/3356223422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ba4101de0 0x7f9ba41984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:11:55.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.347+0000 7f9ba9a91640 1 --2- 172.21.15.116:0/3356223422 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ba4106f10 0x7f9ba419cd80 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f9b94009fd0 tx=0x7f9b9400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:55.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.347+0000 7f9ba27fc640 1 -- 172.21.15.116:0/3356223422 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b94011830 con 0x7f9ba4106f10 2024-09-12T03:11:55.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.347+0000 7f9ba27fc640 1 -- 172.21.15.116:0/3356223422 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b94016070 con 0x7f9ba4106f10 2024-09-12T03:11:55.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.347+0000 7f9bab4f4640 1 -- 172.21.15.116:0/3356223422 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ba419d750 con 0x7f9ba4106f10 2024-09-12T03:11:55.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.347+0000 7f9ba27fc640 1 -- 172.21.15.116:0/3356223422 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9400f2e0 con 0x7f9ba4106f10 2024-09-12T03:11:55.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.347+0000 7f9bab4f4640 1 -- 172.21.15.116:0/3356223422 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ba41a5030 con 0x7f9ba4106f10 2024-09-12T03:11:55.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.348+0000 7f9bab4f4640 1 -- 172.21.15.116:0/3356223422 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ba41021c0 con 0x7f9ba4106f10 2024-09-12T03:11:55.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.349+0000 7f9ba27fc640 1 -- 172.21.15.116:0/3356223422 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f9b940119d0 con 0x7f9ba4106f10 2024-09-12T03:11:55.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.350+0000 7f9ba27fc640 1 --2- 172.21.15.116:0/3356223422 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9b8c0a2b80 0x7f9b8c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:55.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.350+0000 7f9ba27fc640 1 -- 172.21.15.116:0/3356223422 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f9b940cee30 con 0x7f9ba4106f10 2024-09-12T03:11:55.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.350+0000 7f9ba9290640 1 --2- 172.21.15.116:0/3356223422 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9b8c0a2b80 0x7f9b8c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:55.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.350+0000 7f9ba9290640 1 --2- 172.21.15.116:0/3356223422 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9b8c0a2b80 0x7f9b8c0a5040 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c005fd0 tx=0x7f9b9c009520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:55.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.353+0000 7f9ba27fc640 1 -- 172.21.15.116:0/3356223422 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9b9409bdc0 con 0x7f9ba4106f10 2024-09-12T03:11:55.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:55 smithi116 ceph-mon[31410]: pgmap v606: 97 pgs: 97 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:11:55.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.556+0000 7f9bab4f4640 1 -- 172.21.15.116:0/3356223422 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ba4109bc0 con 0x7f9ba4106f10 2024-09-12T03:11:55.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.557+0000 7f9ba27fc640 1 -- 172.21.15.116:0/3356223422 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f9b9409b6c0 con 0x7f9ba4106f10 2024-09-12T03:11:55.557 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:11:55.557 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:11:55.558 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:11:55.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.559+0000 7f9bab4f4640 1 -- 172.21.15.116:0/3356223422 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9b8c0a2b80 msgr2=0x7f9b8c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:55.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.559+0000 7f9bab4f4640 1 --2- 172.21.15.116:0/3356223422 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9b8c0a2b80 0x7f9b8c0a5040 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c005fd0 tx=0x7f9b9c009520 comp rx=0 tx=0).stop 2024-09-12T03:11:55.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.559+0000 7f9bab4f4640 1 -- 172.21.15.116:0/3356223422 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ba4106f10 msgr2=0x7f9ba419cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:55.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.559+0000 7f9bab4f4640 1 --2- 172.21.15.116:0/3356223422 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ba4106f10 0x7f9ba419cd80 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f9b94009fd0 tx=0x7f9b9400b8e0 comp rx=0 tx=0).stop 2024-09-12T03:11:55.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.559+0000 7f9bab4f4640 1 -- 172.21.15.116:0/3356223422 shutdown_connections 2024-09-12T03:11:55.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.559+0000 7f9bab4f4640 1 --2- 172.21.15.116:0/3356223422 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9b8c0a2b80 0x7f9b8c0a5040 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:55.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.559+0000 7f9bab4f4640 1 --2- 172.21.15.116:0/3356223422 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ba4106f10 0x7f9ba419cd80 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:55.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.559+0000 7f9bab4f4640 1 --2- 172.21.15.116:0/3356223422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ba4102790 0x7f9ba41989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:55.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.559+0000 7f9bab4f4640 1 --2- 172.21.15.116:0/3356223422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ba4101de0 0x7f9ba41984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:55.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.559+0000 7f9bab4f4640 1 -- 172.21.15.116:0/3356223422 >> 172.21.15.116:0/3356223422 conn(0x7f9ba40fd8c0 msgr2=0x7f9ba40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:55.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.559+0000 7f9bab4f4640 1 -- 172.21.15.116:0/3356223422 shutdown_connections 2024-09-12T03:11:55.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:55.559+0000 7f9bab4f4640 1 -- 172.21.15.116:0/3356223422 wait complete. 2024-09-12T03:11:55.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:55 smithi183 ceph-mon[35535]: pgmap v606: 97 pgs: 97 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:11:55.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:55 smithi177 ceph-mon[35305]: pgmap v606: 97 pgs: 97 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:11:56.275 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:11:56.525 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:11:56.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:56 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3356223422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:11:56.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:56 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3356223422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:11:56.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:56 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3356223422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:11:56.791 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.792 INFO:tasks.workunit.client.0.smithi116.stdout:251/600: open fd = 3 2024-09-12T03:11:56.792 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.792 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.792 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.792 INFO:tasks.workunit.client.0.smithi116.stdout:252/600: open fd = 3 2024-09-12T03:11:56.792 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.792 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.792 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.792 INFO:tasks.workunit.client.0.smithi116.stdout:253/600: open fd = 3 2024-09-12T03:11:56.792 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.792 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.793 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.793 INFO:tasks.workunit.client.0.smithi116.stdout:254/600: open fd = 3 2024-09-12T03:11:56.793 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.793 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.793 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.793 INFO:tasks.workunit.client.0.smithi116.stdout:255/600: open fd = 3 2024-09-12T03:11:56.793 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.793 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.793 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.793 INFO:tasks.workunit.client.0.smithi116.stdout:256/600: open fd = 3 2024-09-12T03:11:56.793 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.794 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.794 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.794 INFO:tasks.workunit.client.0.smithi116.stdout:257/600: open fd = 3 2024-09-12T03:11:56.794 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.794 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.794 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.794 INFO:tasks.workunit.client.0.smithi116.stdout:258/600: open fd = 3 2024-09-12T03:11:56.795 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.795 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.795 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.795 INFO:tasks.workunit.client.0.smithi116.stdout:259/600: open fd = 3 2024-09-12T03:11:56.795 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.795 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.795 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.795 INFO:tasks.workunit.client.0.smithi116.stdout:260/600: open fd = 3 2024-09-12T03:11:56.795 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.795 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.795 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.795 INFO:tasks.workunit.client.0.smithi116.stdout:261/600: open fd = 3 2024-09-12T03:11:56.796 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.796 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.796 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.796 INFO:tasks.workunit.client.0.smithi116.stdout:262/600: open fd = 3 2024-09-12T03:11:56.796 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.796 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.796 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.796 INFO:tasks.workunit.client.0.smithi116.stdout:263/600: open fd = 3 2024-09-12T03:11:56.796 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.796 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.796 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.796 INFO:tasks.workunit.client.0.smithi116.stdout:264/600: open fd = 3 2024-09-12T03:11:56.796 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.797 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.797 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.797 INFO:tasks.workunit.client.0.smithi116.stdout:265/600: open fd = 3 2024-09-12T03:11:56.797 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.797 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.797 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.797 INFO:tasks.workunit.client.0.smithi116.stdout:266/600: open fd = 3 2024-09-12T03:11:56.797 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.797 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.797 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.797 INFO:tasks.workunit.client.0.smithi116.stdout:267/600: open fd = 3 2024-09-12T03:11:56.797 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.797 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.797 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.798 INFO:tasks.workunit.client.0.smithi116.stdout:268/600: open fd = 3 2024-09-12T03:11:56.798 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.798 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.798 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.798 INFO:tasks.workunit.client.0.smithi116.stdout:269/600: open fd = 3 2024-09-12T03:11:56.798 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.798 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.798 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.798 INFO:tasks.workunit.client.0.smithi116.stdout:270/600: open fd = 3 2024-09-12T03:11:56.798 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.798 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.798 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.798 INFO:tasks.workunit.client.0.smithi116.stdout:271/600: open fd = 3 2024-09-12T03:11:56.798 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.799 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.799 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.799 INFO:tasks.workunit.client.0.smithi116.stdout:272/600: open fd = 3 2024-09-12T03:11:56.799 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.799 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.799 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.799 INFO:tasks.workunit.client.0.smithi116.stdout:273/600: open fd = 3 2024-09-12T03:11:56.799 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.799 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.799 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.799 INFO:tasks.workunit.client.0.smithi116.stdout:274/600: open fd = 3 2024-09-12T03:11:56.799 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.799 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.799 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.800 INFO:tasks.workunit.client.0.smithi116.stdout:275/600: open fd = 3 2024-09-12T03:11:56.800 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.800 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.800 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.800 INFO:tasks.workunit.client.0.smithi116.stdout:276/600: open fd = 3 2024-09-12T03:11:56.800 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.800 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.800 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.800 INFO:tasks.workunit.client.0.smithi116.stdout:277/600: open fd = 3 2024-09-12T03:11:56.800 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.800 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.800 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.800 INFO:tasks.workunit.client.0.smithi116.stdout:278/600: open fd = 3 2024-09-12T03:11:56.800 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.801 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.801 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.801 INFO:tasks.workunit.client.0.smithi116.stdout:279/600: open fd = 3 2024-09-12T03:11:56.801 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.801 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.801 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.801 INFO:tasks.workunit.client.0.smithi116.stdout:280/600: open fd = 3 2024-09-12T03:11:56.801 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.801 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.801 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.801 INFO:tasks.workunit.client.0.smithi116.stdout:281/600: open fd = 3 2024-09-12T03:11:56.801 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.801 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.801 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.802 INFO:tasks.workunit.client.0.smithi116.stdout:282/600: open fd = 3 2024-09-12T03:11:56.802 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.802 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.802 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.802 INFO:tasks.workunit.client.0.smithi116.stdout:283/600: open fd = 3 2024-09-12T03:11:56.802 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.802 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.802 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.802 INFO:tasks.workunit.client.0.smithi116.stdout:284/600: open fd = 3 2024-09-12T03:11:56.802 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.802 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.802 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.802 INFO:tasks.workunit.client.0.smithi116.stdout:285/600: open fd = 3 2024-09-12T03:11:56.803 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.803 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.803 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.803 INFO:tasks.workunit.client.0.smithi116.stdout:286/600: open fd = 3 2024-09-12T03:11:56.803 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.803 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.803 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.803 INFO:tasks.workunit.client.0.smithi116.stdout:287/600: open fd = 3 2024-09-12T03:11:56.803 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.803 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.803 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.803 INFO:tasks.workunit.client.0.smithi116.stdout:288/600: open fd = 3 2024-09-12T03:11:56.803 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.803 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.803 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.804 INFO:tasks.workunit.client.0.smithi116.stdout:289/600: open fd = 3 2024-09-12T03:11:56.804 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.804 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.804 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.804 INFO:tasks.workunit.client.0.smithi116.stdout:290/600: open fd = 3 2024-09-12T03:11:56.804 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.804 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.804 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.804 INFO:tasks.workunit.client.0.smithi116.stdout:291/600: open fd = 3 2024-09-12T03:11:56.804 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.804 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.804 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.805 INFO:tasks.workunit.client.0.smithi116.stdout:292/600: open fd = 3 2024-09-12T03:11:56.805 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.805 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.805 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.805 INFO:tasks.workunit.client.0.smithi116.stdout:293/600: open fd = 3 2024-09-12T03:11:56.805 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.805 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.805 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.805 INFO:tasks.workunit.client.0.smithi116.stdout:294/600: open fd = 3 2024-09-12T03:11:56.805 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.806 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.806 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.806 INFO:tasks.workunit.client.0.smithi116.stdout:295/600: open fd = 3 2024-09-12T03:11:56.806 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.806 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.806 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.806 INFO:tasks.workunit.client.0.smithi116.stdout:296/600: open fd = 3 2024-09-12T03:11:56.806 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.806 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.806 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.806 INFO:tasks.workunit.client.0.smithi116.stdout:297/600: open fd = 3 2024-09-12T03:11:56.806 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.806 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.807 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.807 INFO:tasks.workunit.client.0.smithi116.stdout:298/600: open fd = 3 2024-09-12T03:11:56.807 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.807 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.807 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.807 INFO:tasks.workunit.client.0.smithi116.stdout:299/600: open fd = 3 2024-09-12T03:11:56.807 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.807 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.807 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.807 INFO:tasks.workunit.client.0.smithi116.stdout:300/600: open fd = 3 2024-09-12T03:11:56.807 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.807 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.807 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.807 INFO:tasks.workunit.client.0.smithi116.stdout:301/600: open fd = 3 2024-09-12T03:11:56.807 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.808 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.808 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.808 INFO:tasks.workunit.client.0.smithi116.stdout:302/600: open fd = 3 2024-09-12T03:11:56.808 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.808 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.808 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.808 INFO:tasks.workunit.client.0.smithi116.stdout:303/600: open fd = 3 2024-09-12T03:11:56.808 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.808 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.808 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.808 INFO:tasks.workunit.client.0.smithi116.stdout:304/600: open fd = 3 2024-09-12T03:11:56.809 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.809 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.809 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.809 INFO:tasks.workunit.client.0.smithi116.stdout:305/600: open fd = 3 2024-09-12T03:11:56.809 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.809 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.809 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.809 INFO:tasks.workunit.client.0.smithi116.stdout:306/600: open fd = 3 2024-09-12T03:11:56.809 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.809 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.809 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.809 INFO:tasks.workunit.client.0.smithi116.stdout:307/600: open fd = 3 2024-09-12T03:11:56.809 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.809 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.810 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.810 INFO:tasks.workunit.client.0.smithi116.stdout:308/600: open fd = 3 2024-09-12T03:11:56.810 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.810 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.810 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.810 INFO:tasks.workunit.client.0.smithi116.stdout:309/600: open fd = 3 2024-09-12T03:11:56.810 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.810 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.810 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.810 INFO:tasks.workunit.client.0.smithi116.stdout:310/600: open fd = 3 2024-09-12T03:11:56.810 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.810 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.810 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.810 INFO:tasks.workunit.client.0.smithi116.stdout:311/600: open fd = 3 2024-09-12T03:11:56.811 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.811 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.811 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:11:56.811 INFO:tasks.workunit.client.0.smithi116.stdout:312/600: open fd = 3 2024-09-12T03:11:56.811 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:56.811 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:11:57.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.039+0000 7f5284877640 1 -- 172.21.15.116:0/4240406929 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5280106f10 msgr2=0x7f528010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:57.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.039+0000 7f5284877640 1 --2- 172.21.15.116:0/4240406929 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5280106f10 0x7f528010ea90 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f5278009ca0 tx=0x7f527802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.039+0000 7f5284877640 1 -- 172.21.15.116:0/4240406929 shutdown_connections 2024-09-12T03:11:57.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.039+0000 7f5284877640 1 --2- 172.21.15.116:0/4240406929 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5280106f10 0x7f528010ea90 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.039+0000 7f5284877640 1 --2- 172.21.15.116:0/4240406929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5280102790 0x7f52801067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.039+0000 7f5284877640 1 --2- 172.21.15.116:0/4240406929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5280101de0 0x7f52801021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.039+0000 7f5284877640 1 -- 172.21.15.116:0/4240406929 >> 172.21.15.116:0/4240406929 conn(0x7f52800fd8c0 msgr2=0x7f52800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:57.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.039+0000 7f5284877640 1 -- 172.21.15.116:0/4240406929 shutdown_connections 2024-09-12T03:11:57.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.039+0000 7f5284877640 1 -- 172.21.15.116:0/4240406929 wait complete. 2024-09-12T03:11:57.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.040+0000 7f5284877640 1 Processor -- start 2024-09-12T03:11:57.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.040+0000 7f5284877640 1 -- start start 2024-09-12T03:11:57.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.040+0000 7f5284877640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5280101de0 0x7f52801985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:57.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.040+0000 7f5284877640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5280102790 0x7f5280198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:57.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.040+0000 7f5284877640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5280106f10 0x7f528019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:57.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.040+0000 7f5284877640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5280112010 con 0x7f5280102790 2024-09-12T03:11:57.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.040+0000 7f5284877640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5280111e90 con 0x7f5280106f10 2024-09-12T03:11:57.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.040+0000 7f5284877640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5280112190 con 0x7f5280101de0 2024-09-12T03:11:57.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.040+0000 7f527dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5280102790 0x7f5280198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:57.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.040+0000 7f527dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5280102790 0x7f5280198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48470/0 (socket says 172.21.15.116:48470) 2024-09-12T03:11:57.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.040+0000 7f527dd9b640 1 -- 172.21.15.116:0/3593907406 learned_addr learned my addr 172.21.15.116:0/3593907406 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:57.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.040+0000 7f527e59c640 1 --2- 172.21.15.116:0/3593907406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5280101de0 0x7f52801985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:57.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.040+0000 7f527ed9d640 1 --2- 172.21.15.116:0/3593907406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5280106f10 0x7f528019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:57.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.040+0000 7f527dd9b640 1 -- 172.21.15.116:0/3593907406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5280101de0 msgr2=0x7f52801985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:57.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.040+0000 7f527dd9b640 1 --2- 172.21.15.116:0/3593907406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5280101de0 0x7f52801985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.040+0000 7f527dd9b640 1 -- 172.21.15.116:0/3593907406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5280106f10 msgr2=0x7f528019cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:57.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.040+0000 7f527dd9b640 1 --2- 172.21.15.116:0/3593907406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5280106f10 0x7f528019cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.040+0000 7f527dd9b640 1 -- 172.21.15.116:0/3593907406 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f528019d5a0 con 0x7f5280102790 2024-09-12T03:11:57.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.040+0000 7f527ed9d640 1 --2- 172.21.15.116:0/3593907406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5280106f10 0x7f528019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:11:57.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.041+0000 7f527dd9b640 1 --2- 172.21.15.116:0/3593907406 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5280102790 0x7f5280198b30 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f527400b4a0 tx=0x7f527400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:57.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.041+0000 7f526f7fe640 1 -- 172.21.15.116:0/3593907406 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5274015020 con 0x7f5280102790 2024-09-12T03:11:57.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.041+0000 7f5284877640 1 -- 172.21.15.116:0/3593907406 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f528019d890 con 0x7f5280102790 2024-09-12T03:11:57.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.041+0000 7f526f7fe640 1 -- 172.21.15.116:0/3593907406 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5274011d10 con 0x7f5280102790 2024-09-12T03:11:57.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.041+0000 7f526f7fe640 1 -- 172.21.15.116:0/3593907406 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527400f2e0 con 0x7f5280102790 2024-09-12T03:11:57.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.041+0000 7f5284877640 1 -- 172.21.15.116:0/3593907406 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52801a5170 con 0x7f5280102790 2024-09-12T03:11:57.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.043+0000 7f526f7fe640 1 -- 172.21.15.116:0/3593907406 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f5274018020 con 0x7f5280102790 2024-09-12T03:11:57.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.043+0000 7f5286c50740 1 Processor -- start 2024-09-12T03:11:57.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.043+0000 7f5286c50740 1 -- start start 2024-09-12T03:11:57.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.044+0000 7f5286c50740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559d34b76b40 0x559d34b75090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:57.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.044+0000 7f5286c50740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d34b755d0 0x559d34b75a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:57.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.044+0000 7f5286c50740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559d34b80370 0x559d34b82760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:57.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.044+0000 7f5286c50740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559d34adb2f0 con 0x559d34b76b40 2024-09-12T03:11:57.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.044+0000 7f5286c50740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559d34a82cf0 con 0x559d34b80370 2024-09-12T03:11:57.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.044+0000 7f5286c50740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559d34ad72b0 con 0x559d34b755d0 2024-09-12T03:11:57.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.044+0000 7f527ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559d34b76b40 0x559d34b75090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:57.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.044+0000 7f527ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559d34b76b40 0x559d34b75090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48472/0 (socket says 172.21.15.116:48472) 2024-09-12T03:11:57.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.044+0000 7f527ed9d640 1 -- 172.21.15.116:0/1743270281 learned_addr learned my addr 172.21.15.116:0/1743270281 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:57.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.044+0000 7f527e59c640 1 --2- 172.21.15.116:0/1743270281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d34b755d0 0x559d34b75a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:57.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.044+0000 7f527dd9b640 1 --2- 172.21.15.116:0/1743270281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559d34b80370 0x559d34b82760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:57.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.044+0000 7f526f7fe640 1 --2- 172.21.15.116:0/3593907406 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f52680a2b80 0x7f52680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:57.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.044+0000 7f527ed9d640 1 -- 172.21.15.116:0/1743270281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d34b755d0 msgr2=0x559d34b75a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:57.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.044+0000 7f527ed9d640 1 --2- 172.21.15.116:0/1743270281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d34b755d0 0x559d34b75a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.044+0000 7f527ed9d640 1 -- 172.21.15.116:0/1743270281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559d34b80370 msgr2=0x559d34b82760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:57.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.044+0000 7f527ed9d640 1 --2- 172.21.15.116:0/1743270281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559d34b80370 0x559d34b82760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.044+0000 7f527ed9d640 1 -- 172.21.15.116:0/1743270281 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d34a673a0 con 0x559d34b76b40 2024-09-12T03:11:57.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.044+0000 7f526f7fe640 1 -- 172.21.15.116:0/3593907406 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f52740ce920 con 0x7f5280102790 2024-09-12T03:11:57.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.044+0000 7f527ed9d640 1 --2- 172.21.15.116:0/3593907406 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f52680a2b80 0x7f52680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:57.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.044+0000 7f527ed9d640 1 --2- 172.21.15.116:0/1743270281 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559d34b76b40 0x559d34b75090 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f5278009940 tx=0x7f5278009970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb8b3d86f7021e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:57.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.045+0000 7f526cff9640 1 -- 172.21.15.116:0/1743270281 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5278003b50 con 0x559d34b76b40 2024-09-12T03:11:57.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.045+0000 7f526cff9640 1 -- 172.21.15.116:0/1743270281 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5278036050 con 0x559d34b76b40 2024-09-12T03:11:57.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.045+0000 7f526cff9640 1 -- 172.21.15.116:0/1743270281 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527802f610 con 0x559d34b76b40 2024-09-12T03:11:57.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.045+0000 7f5286c50740 1 -- 172.21.15.116:0/1743270281 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559d34b76b40 msgr2=0x559d34b75090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:57.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.045+0000 7f527ed9d640 1 --2- 172.21.15.116:0/3593907406 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f52680a2b80 0x7f52680a5040 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5278049000 tx=0x7f527802fca0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:57.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.045+0000 7f5286c50740 1 --2- 172.21.15.116:0/1743270281 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559d34b76b40 0x559d34b75090 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f5278009940 tx=0x7f5278009970 comp rx=0 tx=0).stop 2024-09-12T03:11:57.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.045+0000 7f5286c50740 1 -- 172.21.15.116:0/1743270281 shutdown_connections 2024-09-12T03:11:57.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.045+0000 7f5286c50740 1 --2- 172.21.15.116:0/1743270281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559d34b80370 0x559d34b82760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.045+0000 7f5286c50740 1 --2- 172.21.15.116:0/1743270281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d34b755d0 0x559d34b75a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.045+0000 7f5286c50740 1 --2- 172.21.15.116:0/1743270281 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559d34b76b40 0x559d34b75090 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.045+0000 7f5286c50740 1 -- 172.21.15.116:0/1743270281 >> 172.21.15.116:0/1743270281 conn(0x559d34b3e310 msgr2=0x559d34b71c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:57.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.045+0000 7f5286c50740 1 -- 172.21.15.116:0/1743270281 shutdown_connections 2024-09-12T03:11:57.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.046+0000 7f5286c50740 1 -- 172.21.15.116:0/1743270281 wait complete. 2024-09-12T03:11:57.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.046+0000 7f5286c50740 1 -- start start 2024-09-12T03:11:57.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.046+0000 7f5286c50740 1 Processor -- start 2024-09-12T03:11:57.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.046+0000 7f5286c50740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559d34b755d0 0x559d34b7fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:57.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.046+0000 7f5286c50740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559d34b76b40 0x559d34b7cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:57.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.046+0000 7f5286c50740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d34b80370 0x559d34b7d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:57.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.046+0000 7f5286c50740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559d34b82ef0 con 0x559d34b76b40 2024-09-12T03:11:57.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.046+0000 7f5286c50740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559d34a96060 con 0x559d34b755d0 2024-09-12T03:11:57.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.046+0000 7f5286c50740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559d34b83070 con 0x559d34b80370 2024-09-12T03:11:57.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.046+0000 7f5286c50740 10 client.0 _ll_register_callbacks cb 0x559d347e98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:57.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.046+0000 7f527dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559d34b76b40 0x559d34b7cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:57.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.046+0000 7f527dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559d34b76b40 0x559d34b7cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48480/0 (socket says 172.21.15.116:48480) 2024-09-12T03:11:57.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.046+0000 7f527dd9b640 1 -- 172.21.15.116:0/78109506 learned_addr learned my addr 172.21.15.116:0/78109506 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:57.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.046+0000 7f527ed9d640 1 --2- 172.21.15.116:0/78109506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d34b80370 0x559d34b7d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:57.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.046+0000 7f527e59c640 1 --2- 172.21.15.116:0/78109506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559d34b755d0 0x559d34b7fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:57.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.047+0000 7f527dd9b640 1 -- 172.21.15.116:0/78109506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d34b80370 msgr2=0x559d34b7d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:57.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.047+0000 7f527dd9b640 1 --2- 172.21.15.116:0/78109506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d34b80370 0x559d34b7d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.047+0000 7f527dd9b640 1 -- 172.21.15.116:0/78109506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559d34b755d0 msgr2=0x559d34b7fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:57.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.047+0000 7f527dd9b640 1 --2- 172.21.15.116:0/78109506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559d34b755d0 0x559d34b7fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.047+0000 7f527dd9b640 1 -- 172.21.15.116:0/78109506 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d3484c3c0 con 0x559d34b76b40 2024-09-12T03:11:57.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.047+0000 7f527e59c640 1 --2- 172.21.15.116:0/78109506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559d34b755d0 0x559d34b7fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:11:57.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.047+0000 7f527dd9b640 1 --2- 172.21.15.116:0/78109506 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559d34b76b40 0x559d34b7cfd0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f527400bda0 tx=0x7f527409acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fff5e051cee0a664 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:57.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.047+0000 7f5265ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:57.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.047+0000 7f5265ffb640 1 -- 172.21.15.116:0/78109506 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527409b8b0 con 0x559d34b76b40 2024-09-12T03:11:57.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.047+0000 7f5265ffb640 1 -- 172.21.15.116:0/78109506 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f5274010510 con 0x559d34b76b40 2024-09-12T03:11:57.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.047+0000 7f5286c50740 1 -- 172.21.15.116:0/78109506 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5278036050 con 0x559d34b76b40 2024-09-12T03:11:57.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.047+0000 7f5265ffb640 1 -- 172.21.15.116:0/78109506 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f527409b1b0 con 0x559d34b76b40 2024-09-12T03:11:57.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.047+0000 7f5286c50740 10 client.17724 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:57.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.047+0000 7f5286c50740 1 -- 172.21.15.116:0/78109506 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559d34b8e940 con 0x559d34b76b40 2024-09-12T03:11:57.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.047+0000 7f5265ffb640 1 -- 172.21.15.116:0/78109506 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5278036050 con 0x559d34b76b40 2024-09-12T03:11:57.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.047+0000 7f5265ffb640 1 -- 172.21.15.116:0/78109506 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f52740bb0e0 con 0x559d34b76b40 2024-09-12T03:11:57.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.047+0000 7f5286c50740 10 client.17724 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:57.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.047+0000 7f5286c50740 10 client.17724 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:11:57.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.047+0000 7f5286c50740 20 client.17724 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:57.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.048+0000 7f5286c50740 1 --2- 172.21.15.116:0/78109506 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x559d34b8fe00 0x559d34b92230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:57.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.048+0000 7f5286c50740 4 client.17724 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:11:57.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.048+0000 7f5286c50740 1 -- 172.21.15.116:0/78109506 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d34ae8860 con 0x559d34b8fe00 2024-09-12T03:11:57.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.048+0000 7f527e59c640 1 --2- 172.21.15.116:0/78109506 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x559d34b8fe00 0x559d34b92230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:57.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.048+0000 7f5265ffb640 10 client.17724 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:11:57.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.048+0000 7f527e59c640 1 --2- 172.21.15.116:0/78109506 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x559d34b8fe00 0x559d34b92230 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=60b5a66268a7e9c5 server_cookie=7d41e46cecf1d8f in_seq=0 out_seq=0 2024-09-12T03:11:57.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.049+0000 7f5265ffb640 1 -- 172.21.15.116:0/78109506 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559d34ae8860 con 0x559d34b8fe00 2024-09-12T03:11:57.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.049+0000 7f5265ffb640 10 client.17724 handle_command_reply: tid=0 2024-09-12T03:11:57.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.049+0000 7f5286c50740 1 client.17724 shutdown 2024-09-12T03:11:57.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.049+0000 7f5286c50740 1 -- 172.21.15.116:0/78109506 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559d34b76b40 msgr2=0x559d34b7cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:57.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.049+0000 7f5286c50740 1 --2- 172.21.15.116:0/78109506 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559d34b76b40 0x559d34b7cfd0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f527400bda0 tx=0x7f527409acf0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.049+0000 7f5286c50740 1 -- 172.21.15.116:0/78109506 shutdown_connections 2024-09-12T03:11:57.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.049+0000 7f5286c50740 1 --2- 172.21.15.116:0/78109506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d34b80370 0x559d34b7d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.049+0000 7f5286c50740 1 --2- 172.21.15.116:0/78109506 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x559d34b8fe00 0x559d34b92230 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.049+0000 7f5286c50740 1 --2- 172.21.15.116:0/78109506 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559d34b76b40 0x559d34b7cfd0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.049+0000 7f5286c50740 1 --2- 172.21.15.116:0/78109506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559d34b755d0 0x559d34b7fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.049+0000 7f5265ffb640 0 client.17724 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:11:57.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.049+0000 7f5286c50740 1 -- 172.21.15.116:0/78109506 >> 172.21.15.116:0/78109506 conn(0x559d34b3e310 msgr2=0x559d34b77960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:57.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.049+0000 7f5286c50740 1 -- 172.21.15.116:0/78109506 shutdown_connections 2024-09-12T03:11:57.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.049+0000 7f5286c50740 1 -- 172.21.15.116:0/78109506 wait complete. 2024-09-12T03:11:57.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.049+0000 7f5286c50740 20 client.17724 trim_cache size 0 max 16384 2024-09-12T03:11:57.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.062+0000 7f5286c50740 1 Processor -- start 2024-09-12T03:11:57.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.062+0000 7f5286c50740 1 -- start start 2024-09-12T03:11:57.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.062+0000 7f5286c50740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559d34b755d0 0x559d34b71e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:57.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.062+0000 7f5286c50740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559d34b76b40 0x559d34b8c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:57.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.062+0000 7f5286c50740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d34b80370 0x559d34b886e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:57.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.062+0000 7f5286c50740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f52801afda0 con 0x559d34b76b40 2024-09-12T03:11:57.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.062+0000 7f5286c50740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f52801ad670 con 0x559d34b755d0 2024-09-12T03:11:57.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.062+0000 7f5286c50740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f52801ab4c0 con 0x559d34b80370 2024-09-12T03:11:57.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.062+0000 7f527dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559d34b76b40 0x559d34b8c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:57.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.062+0000 7f527dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559d34b76b40 0x559d34b8c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48494/0 (socket says 172.21.15.116:48494) 2024-09-12T03:11:57.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.062+0000 7f527dd9b640 1 -- 172.21.15.116:0/1866486213 learned_addr learned my addr 172.21.15.116:0/1866486213 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:57.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.063+0000 7f527dd9b640 1 -- 172.21.15.116:0/1866486213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d34b80370 msgr2=0x559d34b886e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:57.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.063+0000 7f527dd9b640 1 --2- 172.21.15.116:0/1866486213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d34b80370 0x559d34b886e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.063+0000 7f527dd9b640 1 -- 172.21.15.116:0/1866486213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559d34b755d0 msgr2=0x559d34b71e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:57.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.063+0000 7f527dd9b640 1 --2- 172.21.15.116:0/1866486213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559d34b755d0 0x559d34b71e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.063+0000 7f527dd9b640 1 -- 172.21.15.116:0/1866486213 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d34b7b540 con 0x559d34b76b40 2024-09-12T03:11:57.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.063+0000 7f527dd9b640 1 --2- 172.21.15.116:0/1866486213 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559d34b76b40 0x559d34b8c190 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f52740108c0 tx=0x7f5274097820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be05ee8df6e0f31 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:57.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.063+0000 7f5267fff640 1 -- 172.21.15.116:0/1866486213 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527409ab10 con 0x559d34b76b40 2024-09-12T03:11:57.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.063+0000 7f5267fff640 1 -- 172.21.15.116:0/1866486213 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f527409b8b0 con 0x559d34b76b40 2024-09-12T03:11:57.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.063+0000 7f5267fff640 1 -- 172.21.15.116:0/1866486213 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527409f560 con 0x559d34b76b40 2024-09-12T03:11:57.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.063+0000 7f5286c50740 1 -- 172.21.15.116:0/1866486213 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559d34b76b40 msgr2=0x559d34b8c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:57.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.063+0000 7f5286c50740 1 --2- 172.21.15.116:0/1866486213 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559d34b76b40 0x559d34b8c190 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f52740108c0 tx=0x7f5274097820 comp rx=0 tx=0).stop 2024-09-12T03:11:57.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.063+0000 7f5286c50740 1 -- 172.21.15.116:0/1866486213 shutdown_connections 2024-09-12T03:11:57.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.064+0000 7f5286c50740 1 --2- 172.21.15.116:0/1866486213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d34b80370 0x559d34b886e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.064+0000 7f5286c50740 1 --2- 172.21.15.116:0/1866486213 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559d34b76b40 0x559d34b8c190 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.064+0000 7f5286c50740 1 --2- 172.21.15.116:0/1866486213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559d34b755d0 0x559d34b71e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.064+0000 7f5286c50740 1 -- 172.21.15.116:0/1866486213 >> 172.21.15.116:0/1866486213 conn(0x559d34b3e310 msgr2=0x559d34b71700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:57.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.064+0000 7f5286c50740 1 -- 172.21.15.116:0/1866486213 shutdown_connections 2024-09-12T03:11:57.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.064+0000 7f5286c50740 1 -- 172.21.15.116:0/1866486213 wait complete. 2024-09-12T03:11:57.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.064+0000 7f5286c50740 1 -- start start 2024-09-12T03:11:57.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.064+0000 7f5286c50740 1 Processor -- start 2024-09-12T03:11:57.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.064+0000 7f5286c50740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559d34b755d0 0x559d34b7a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:57.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.064+0000 7f5286c50740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559d34b76b40 0x559d34b79420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:57.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.064+0000 7f5286c50740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d34b80370 0x559d34b79960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:57.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.064+0000 7f5286c50740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f52801abb30 con 0x559d34b76b40 2024-09-12T03:11:57.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.064+0000 7f5286c50740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f52801acfa0 con 0x559d34b755d0 2024-09-12T03:11:57.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.064+0000 7f5286c50740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f52801ac200 con 0x559d34b80370 2024-09-12T03:11:57.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.064+0000 7f5286c50740 10 client.0 _ll_register_callbacks cb 0x7f527400bf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:57.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.064+0000 7f527dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559d34b76b40 0x559d34b79420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:57.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.064+0000 7f527dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559d34b76b40 0x559d34b79420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48508/0 (socket says 172.21.15.116:48508) 2024-09-12T03:11:57.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.064+0000 7f527dd9b640 1 -- 172.21.15.116:0/1745518696 learned_addr learned my addr 172.21.15.116:0/1745518696 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:57.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.065+0000 7f527ed9d640 1 --2- 172.21.15.116:0/1745518696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d34b80370 0x559d34b79960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:57.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.065+0000 7f527e59c640 1 --2- 172.21.15.116:0/1745518696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559d34b755d0 0x559d34b7a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:57.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.065+0000 7f527dd9b640 1 -- 172.21.15.116:0/1745518696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d34b80370 msgr2=0x559d34b79960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:57.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.065+0000 7f527dd9b640 1 --2- 172.21.15.116:0/1745518696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d34b80370 0x559d34b79960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.065+0000 7f527dd9b640 1 -- 172.21.15.116:0/1745518696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559d34b755d0 msgr2=0x559d34b7a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:57.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.065+0000 7f527dd9b640 1 --2- 172.21.15.116:0/1745518696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559d34b755d0 0x559d34b7a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.065+0000 7f527dd9b640 1 -- 172.21.15.116:0/1745518696 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d34b79f20 con 0x559d34b76b40 2024-09-12T03:11:57.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.065+0000 7f527ed9d640 1 --2- 172.21.15.116:0/1745518696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d34b80370 0x559d34b79960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:57.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.065+0000 7f527e59c640 1 --2- 172.21.15.116:0/1745518696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559d34b755d0 0x559d34b7a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:57.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.065+0000 7f527dd9b640 1 --2- 172.21.15.116:0/1745518696 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559d34b76b40 0x559d34b79420 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f527409aeb0 tx=0x7f527409ae20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59be45e6cec6cc6d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:57.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.065+0000 7f5265ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:57.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.065+0000 7f5265ffb640 1 -- 172.21.15.116:0/1745518696 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52740cec20 con 0x559d34b76b40 2024-09-12T03:11:57.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.065+0000 7f5265ffb640 1 -- 172.21.15.116:0/1745518696 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f527409f0f0 con 0x559d34b76b40 2024-09-12T03:11:57.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.065+0000 7f5286c50740 1 -- 172.21.15.116:0/1745518696 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f527409b8b0 con 0x559d34b76b40 2024-09-12T03:11:57.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.065+0000 7f5265ffb640 1 -- 172.21.15.116:0/1745518696 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f527409f330 con 0x559d34b76b40 2024-09-12T03:11:57.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.065+0000 7f5286c50740 10 client.17730 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:57.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.065+0000 7f5286c50740 1 -- 172.21.15.116:0/1745518696 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559d34bfe8b0 con 0x559d34b76b40 2024-09-12T03:11:57.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.065+0000 7f5265ffb640 1 -- 172.21.15.116:0/1745518696 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f527409b8b0 con 0x559d34b76b40 2024-09-12T03:11:57.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.065+0000 7f5265ffb640 1 -- 172.21.15.116:0/1745518696 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f527409a690 con 0x559d34b76b40 2024-09-12T03:11:57.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.066+0000 7f5286c50740 10 client.17730 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:57.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.066+0000 7f5286c50740 10 client.17730 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:11:57.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.066+0000 7f5286c50740 20 client.17730 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:57.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.066+0000 7f5286c50740 1 --2- 172.21.15.116:0/1745518696 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x559d34bfec90 0x559d34c010b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:57.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.066+0000 7f5286c50740 4 client.17730 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:11:57.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.066+0000 7f5286c50740 1 -- 172.21.15.116:0/1745518696 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559d34ae2d70 con 0x559d34bfec90 2024-09-12T03:11:57.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.066+0000 7f527e59c640 1 --2- 172.21.15.116:0/1745518696 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x559d34bfec90 0x559d34c010b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:57.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.067+0000 7f5265ffb640 10 client.17730 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:11:57.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.067+0000 7f527e59c640 1 --2- 172.21.15.116:0/1745518696 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x559d34bfec90 0x559d34c010b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b0b014432d46e6cb server_cookie=f1d0b0b2b0c23546 in_seq=0 out_seq=0 2024-09-12T03:11:57.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.067+0000 7f5265ffb640 1 -- 172.21.15.116:0/1745518696 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559d34ae2d70 con 0x559d34bfec90 2024-09-12T03:11:57.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.067+0000 7f5265ffb640 10 client.17730 handle_command_reply: tid=0 2024-09-12T03:11:57.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.067+0000 7f5286c50740 1 client.17730 shutdown 2024-09-12T03:11:57.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.067+0000 7f5286c50740 1 -- 172.21.15.116:0/1745518696 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559d34b76b40 msgr2=0x559d34b79420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:57.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.067+0000 7f5286c50740 1 --2- 172.21.15.116:0/1745518696 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559d34b76b40 0x559d34b79420 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f527409aeb0 tx=0x7f527409ae20 comp rx=0 tx=0).stop 2024-09-12T03:11:57.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.067+0000 7f5286c50740 1 -- 172.21.15.116:0/1745518696 shutdown_connections 2024-09-12T03:11:57.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.067+0000 7f5286c50740 1 --2- 172.21.15.116:0/1745518696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d34b80370 0x559d34b79960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.067+0000 7f5286c50740 1 --2- 172.21.15.116:0/1745518696 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x559d34bfec90 0x559d34c010b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.067+0000 7f5265ffb640 0 client.17730 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:11:57.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.067+0000 7f5286c50740 1 --2- 172.21.15.116:0/1745518696 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559d34b76b40 0x559d34b79420 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.067+0000 7f5286c50740 1 --2- 172.21.15.116:0/1745518696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559d34b755d0 0x559d34b7a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.067+0000 7f5286c50740 1 -- 172.21.15.116:0/1745518696 >> 172.21.15.116:0/1745518696 conn(0x559d34b3e310 msgr2=0x559d34b7f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:57.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.067+0000 7f5286c50740 1 -- 172.21.15.116:0/1745518696 shutdown_connections 2024-09-12T03:11:57.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.067+0000 7f5286c50740 1 -- 172.21.15.116:0/1745518696 wait complete. 2024-09-12T03:11:57.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.067+0000 7f5286c50740 20 client.17730 trim_cache size 0 max 16384 2024-09-12T03:11:57.075 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:11:57.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.068+0000 7f5284877640 1 -- 172.21.15.116:0/3593907406 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f52680a2b80 msgr2=0x7f52680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:57.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.068+0000 7f5284877640 1 --2- 172.21.15.116:0/3593907406 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f52680a2b80 0x7f52680a5040 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5278049000 tx=0x7f527802fca0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.068+0000 7f5284877640 1 -- 172.21.15.116:0/3593907406 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5280102790 msgr2=0x7f5280198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:57.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.068+0000 7f5284877640 1 --2- 172.21.15.116:0/3593907406 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5280102790 0x7f5280198b30 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f527400b4a0 tx=0x7f527400b8a0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.068+0000 7f5284877640 1 -- 172.21.15.116:0/3593907406 shutdown_connections 2024-09-12T03:11:57.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.068+0000 7f5284877640 1 --2- 172.21.15.116:0/3593907406 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f52680a2b80 0x7f52680a5040 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.068+0000 7f5284877640 1 --2- 172.21.15.116:0/3593907406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5280106f10 0x7f528019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.068+0000 7f5284877640 1 --2- 172.21.15.116:0/3593907406 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5280102790 0x7f5280198b30 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.068+0000 7f5284877640 1 --2- 172.21.15.116:0/3593907406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5280101de0 0x7f52801985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:57.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.068+0000 7f5284877640 1 -- 172.21.15.116:0/3593907406 >> 172.21.15.116:0/3593907406 conn(0x7f52800fd8c0 msgr2=0x7f52800fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:57.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.068+0000 7f5284877640 1 -- 172.21.15.116:0/3593907406 shutdown_connections 2024-09-12T03:11:57.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:57.068+0000 7f5284877640 1 -- 172.21.15.116:0/3593907406 wait complete. 2024-09-12T03:11:57.683 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:11:57 smithi116 ceph-mon[31410]: pgmap v607: 97 pgs: 97 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:11:57.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:11:57 smithi183 ceph-mon[35535]: pgmap v607: 97 pgs: 97 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:11:57.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:11:57 smithi177 ceph-mon[35305]: pgmap v607: 97 pgs: 97 active+clean; 4.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:11:57.760 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:11:58.006 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:11:58.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.674+0000 7f1516c7b640 1 -- 172.21.15.116:0/2425362017 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1510102790 msgr2=0x7f15101067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:58.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.674+0000 7f1516c7b640 1 --2- 172.21.15.116:0/2425362017 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1510102790 0x7f15101067e0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f14fc009900 tx=0x7f14fc02a410 comp rx=0 tx=0).stop 2024-09-12T03:11:58.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.674+0000 7f1516c7b640 1 -- 172.21.15.116:0/2425362017 shutdown_connections 2024-09-12T03:11:58.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.674+0000 7f1516c7b640 1 --2- 172.21.15.116:0/2425362017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1510106f10 0x7f151010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.674+0000 7f1516c7b640 1 --2- 172.21.15.116:0/2425362017 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1510102790 0x7f15101067e0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.674+0000 7f1516c7b640 1 --2- 172.21.15.116:0/2425362017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1510101de0 0x7f15101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.674+0000 7f1516c7b640 1 -- 172.21.15.116:0/2425362017 >> 172.21.15.116:0/2425362017 conn(0x7f15100fd8c0 msgr2=0x7f15100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:58.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.674+0000 7f1516c7b640 1 -- 172.21.15.116:0/2425362017 shutdown_connections 2024-09-12T03:11:58.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.674+0000 7f1516c7b640 1 -- 172.21.15.116:0/2425362017 wait complete. 2024-09-12T03:11:58.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.675+0000 7f1516c7b640 1 Processor -- start 2024-09-12T03:11:58.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.675+0000 7f1516c7b640 1 -- start start 2024-09-12T03:11:58.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.675+0000 7f1516c7b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1510101de0 0x7f15101983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:58.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.675+0000 7f1516c7b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1510102790 0x7f1510198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:58.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.675+0000 7f1516c7b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1510106f10 0x7f151019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:58.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.675+0000 7f1516c7b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1510111f70 con 0x7f1510101de0 2024-09-12T03:11:58.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.675+0000 7f1516c7b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1510111df0 con 0x7f1510102790 2024-09-12T03:11:58.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.675+0000 7f1516c7b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f15101120f0 con 0x7f1510106f10 2024-09-12T03:11:58.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.675+0000 7f1514a17640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1510101de0 0x7f15101983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:58.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.675+0000 7f1514a17640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1510101de0 0x7f15101983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48514/0 (socket says 172.21.15.116:48514) 2024-09-12T03:11:58.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.675+0000 7f1514a17640 1 -- 172.21.15.116:0/3782252832 learned_addr learned my addr 172.21.15.116:0/3782252832 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:58.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.676+0000 7f1515218640 1 --2- 172.21.15.116:0/3782252832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1510106f10 0x7f151019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:58.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.676+0000 7f150ffff640 1 --2- 172.21.15.116:0/3782252832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1510102790 0x7f1510198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:58.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.676+0000 7f1514a17640 1 -- 172.21.15.116:0/3782252832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1510106f10 msgr2=0x7f151019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:58.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.676+0000 7f1514a17640 1 --2- 172.21.15.116:0/3782252832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1510106f10 0x7f151019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.676+0000 7f1514a17640 1 -- 172.21.15.116:0/3782252832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1510102790 msgr2=0x7f1510198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:58.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.676+0000 7f1514a17640 1 --2- 172.21.15.116:0/3782252832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1510102790 0x7f1510198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.676+0000 7f1514a17640 1 -- 172.21.15.116:0/3782252832 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151019d380 con 0x7f1510101de0 2024-09-12T03:11:58.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.676+0000 7f1515218640 1 --2- 172.21.15.116:0/3782252832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1510106f10 0x7f151019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:11:58.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.676+0000 7f1514a17640 1 --2- 172.21.15.116:0/3782252832 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1510101de0 0x7f15101983d0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f150400d770 tx=0x7f150400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:58.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.676+0000 7f150dffb640 1 -- 172.21.15.116:0/3782252832 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1504017070 con 0x7f1510101de0 2024-09-12T03:11:58.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.676+0000 7f150dffb640 1 -- 172.21.15.116:0/3782252832 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1504005500 con 0x7f1510101de0 2024-09-12T03:11:58.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.676+0000 7f150dffb640 1 -- 172.21.15.116:0/3782252832 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1504004e00 con 0x7f1510101de0 2024-09-12T03:11:58.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.676+0000 7f1516c7b640 1 -- 172.21.15.116:0/3782252832 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f151019d670 con 0x7f1510101de0 2024-09-12T03:11:58.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.676+0000 7f1516c7b640 1 -- 172.21.15.116:0/3782252832 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15101a4f50 con 0x7f1510101de0 2024-09-12T03:11:58.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.679+0000 7f150dffb640 1 -- 172.21.15.116:0/3782252832 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1504004fa0 con 0x7f1510101de0 2024-09-12T03:11:58.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.679+0000 7f1519054740 1 Processor -- start 2024-09-12T03:11:58.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.679+0000 7f1519054740 1 -- start start 2024-09-12T03:11:58.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.679+0000 7f1519054740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96269b40 0x55ea96268090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:58.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.679+0000 7f1519054740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea962685d0 0x55ea96268a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:58.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.679+0000 7f1519054740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea96273370 0x55ea96275760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:58.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.679+0000 7f1519054740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ea961ce2f0 con 0x55ea962685d0 2024-09-12T03:11:58.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.679+0000 7f1519054740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ea96175cf0 con 0x55ea96273370 2024-09-12T03:11:58.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.679+0000 7f1519054740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ea961ca2b0 con 0x55ea96269b40 2024-09-12T03:11:58.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.679+0000 7f150dffb640 1 --2- 172.21.15.116:0/3782252832 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f15000a2c00 0x7f15000a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:58.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.679+0000 7f1514a17640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea962685d0 0x55ea96268a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:58.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.679+0000 7f1515218640 1 --2- 172.21.15.116:0/3782252832 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f15000a2c00 0x7f15000a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:58.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.679+0000 7f1514a17640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea962685d0 0x55ea96268a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48518/0 (socket says 172.21.15.116:48518) 2024-09-12T03:11:58.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.679+0000 7f1514a17640 1 -- 172.21.15.116:0/1911216235 learned_addr learned my addr 172.21.15.116:0/1911216235 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:58.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.680+0000 7f1515218640 1 --2- 172.21.15.116:0/1911216235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96269b40 0x55ea96268090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:58.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.680+0000 7f150ffff640 1 --2- 172.21.15.116:0/1911216235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea96273370 0x55ea96275760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:58.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.680+0000 7f150dffb640 1 -- 172.21.15.116:0/3782252832 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f15040cead0 con 0x7f1510101de0 2024-09-12T03:11:58.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.680+0000 7f1514a17640 1 -- 172.21.15.116:0/1911216235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96269b40 msgr2=0x55ea96268090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:58.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.680+0000 7f1514a17640 1 --2- 172.21.15.116:0/1911216235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96269b40 0x55ea96268090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.680+0000 7f1514a17640 1 -- 172.21.15.116:0/1911216235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea96273370 msgr2=0x55ea96275760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:58.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.680+0000 7f1514a17640 1 --2- 172.21.15.116:0/1911216235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea96273370 0x55ea96275760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.680+0000 7f1514a17640 1 -- 172.21.15.116:0/1911216235 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea9615a3a0 con 0x55ea962685d0 2024-09-12T03:11:58.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.680+0000 7f150ffff640 1 --2- 172.21.15.116:0/1911216235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea96273370 0x55ea96275760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:11:58.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.680+0000 7f1515218640 1 --2- 172.21.15.116:0/3782252832 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f15000a2c00 0x7f15000a50c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1508007fd0 tx=0x7f1508007d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:58.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.680+0000 7f1514a17640 1 --2- 172.21.15.116:0/1911216235 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea962685d0 0x55ea96268a10 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f150400d740 tx=0x7f150406e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e49d59c89a7771b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:58.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.680+0000 7f14faffd640 1 -- 172.21.15.116:0/1911216235 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1504020040 con 0x55ea962685d0 2024-09-12T03:11:58.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.680+0000 7f14faffd640 1 -- 172.21.15.116:0/1911216235 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f150409ba60 con 0x55ea962685d0 2024-09-12T03:11:58.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.680+0000 7f14faffd640 1 -- 172.21.15.116:0/1911216235 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1504099660 con 0x55ea962685d0 2024-09-12T03:11:58.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.681+0000 7f1519054740 1 -- 172.21.15.116:0/1911216235 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea962685d0 msgr2=0x55ea96268a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:58.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.681+0000 7f1519054740 1 --2- 172.21.15.116:0/1911216235 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea962685d0 0x55ea96268a10 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f150400d740 tx=0x7f150406e420 comp rx=0 tx=0).stop 2024-09-12T03:11:58.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.681+0000 7f1519054740 1 -- 172.21.15.116:0/1911216235 shutdown_connections 2024-09-12T03:11:58.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.681+0000 7f1519054740 1 --2- 172.21.15.116:0/1911216235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea96273370 0x55ea96275760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.681+0000 7f1519054740 1 --2- 172.21.15.116:0/1911216235 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea962685d0 0x55ea96268a10 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.681+0000 7f1519054740 1 --2- 172.21.15.116:0/1911216235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96269b40 0x55ea96268090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.681+0000 7f1519054740 1 -- 172.21.15.116:0/1911216235 >> 172.21.15.116:0/1911216235 conn(0x55ea96231310 msgr2=0x55ea96264c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:58.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.681+0000 7f1519054740 1 -- 172.21.15.116:0/1911216235 shutdown_connections 2024-09-12T03:11:58.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.681+0000 7f1519054740 1 -- 172.21.15.116:0/1911216235 wait complete. 2024-09-12T03:11:58.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.681+0000 7f1519054740 1 -- start start 2024-09-12T03:11:58.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.681+0000 7f1519054740 1 Processor -- start 2024-09-12T03:11:58.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.682+0000 7f1519054740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea962685d0 0x55ea96272db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:58.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.682+0000 7f1519054740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96269b40 0x55ea96270060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:58.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.682+0000 7f1519054740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea96273370 0x55ea962707f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:58.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.682+0000 7f1519054740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ea96275fb0 con 0x55ea962685d0 2024-09-12T03:11:58.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.682+0000 7f1519054740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ea96189060 con 0x55ea96273370 2024-09-12T03:11:58.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.682+0000 7f1519054740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ea96276130 con 0x55ea96269b40 2024-09-12T03:11:58.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.682+0000 7f1519054740 10 client.0 _ll_register_callbacks cb 0x55ea95edc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:58.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.682+0000 7f1514a17640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea962685d0 0x55ea96272db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:58.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.682+0000 7f1514a17640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea962685d0 0x55ea96272db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48528/0 (socket says 172.21.15.116:48528) 2024-09-12T03:11:58.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.682+0000 7f1514a17640 1 -- 172.21.15.116:0/1021128734 learned_addr learned my addr 172.21.15.116:0/1021128734 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:58.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.682+0000 7f150ffff640 1 --2- 172.21.15.116:0/1021128734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96269b40 0x55ea96270060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:58.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.682+0000 7f1515218640 1 --2- 172.21.15.116:0/1021128734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea96273370 0x55ea962707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:58.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.682+0000 7f1514a17640 1 -- 172.21.15.116:0/1021128734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96269b40 msgr2=0x55ea96270060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:58.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.682+0000 7f1514a17640 1 --2- 172.21.15.116:0/1021128734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96269b40 0x55ea96270060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.682+0000 7f1514a17640 1 -- 172.21.15.116:0/1021128734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea96273370 msgr2=0x55ea962707f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:58.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.682+0000 7f1514a17640 1 --2- 172.21.15.116:0/1021128734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea96273370 0x55ea962707f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.682+0000 7f1514a17640 1 -- 172.21.15.116:0/1021128734 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea95f3f3c0 con 0x55ea962685d0 2024-09-12T03:11:58.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.682+0000 7f1515218640 1 --2- 172.21.15.116:0/1021128734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea96273370 0x55ea962707f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:58.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.683+0000 7f1514a17640 1 --2- 172.21.15.116:0/1021128734 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea962685d0 0x55ea96272db0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f15040cc650 tx=0x7f15040976a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe9c7fb12eca135 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:58.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.683+0000 7f14effff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:58.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.683+0000 7f14effff640 1 -- 172.21.15.116:0/1021128734 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1504020070 con 0x55ea962685d0 2024-09-12T03:11:58.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.683+0000 7f14effff640 1 -- 172.21.15.116:0/1021128734 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1504009cc0 con 0x55ea962685d0 2024-09-12T03:11:58.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.683+0000 7f1519054740 1 -- 172.21.15.116:0/1021128734 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f150409ba60 con 0x55ea962685d0 2024-09-12T03:11:58.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.683+0000 7f14effff640 1 -- 172.21.15.116:0/1021128734 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f150409f710 con 0x55ea962685d0 2024-09-12T03:11:58.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.683+0000 7f1519054740 10 client.17742 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.683+0000 7f1519054740 1 -- 172.21.15.116:0/1021128734 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ea96281aa0 con 0x55ea962685d0 2024-09-12T03:11:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.683+0000 7f14effff640 1 -- 172.21.15.116:0/1021128734 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f150409ba60 con 0x55ea962685d0 2024-09-12T03:11:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.683+0000 7f14effff640 1 -- 172.21.15.116:0/1021128734 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f150409a7b0 con 0x55ea962685d0 2024-09-12T03:11:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.683+0000 7f1519054740 10 client.17742 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.683+0000 7f1519054740 10 client.17742 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:11:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.683+0000 7f1519054740 20 client.17742 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.683+0000 7f1519054740 1 --2- 172.21.15.116:0/1021128734 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55ea96282df0 0x55ea962851e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.683+0000 7f1519054740 4 client.17742 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:11:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.683+0000 7f1519054740 1 -- 172.21.15.116:0/1021128734 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea961db860 con 0x55ea96282df0 2024-09-12T03:11:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.684+0000 7f150ffff640 1 --2- 172.21.15.116:0/1021128734 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55ea96282df0 0x55ea962851e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:58.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.684+0000 7f150ffff640 1 --2- 172.21.15.116:0/1021128734 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55ea96282df0 0x55ea962851e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=77752ca6fdba200d server_cookie=c9cb70296f3707fa in_seq=0 out_seq=0 2024-09-12T03:11:58.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.684+0000 7f14effff640 10 client.17742 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:11:58.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.685+0000 7f14effff640 1 -- 172.21.15.116:0/1021128734 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ea961db860 con 0x55ea96282df0 2024-09-12T03:11:58.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.685+0000 7f14effff640 10 client.17742 handle_command_reply: tid=0 2024-09-12T03:11:58.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.685+0000 7f1519054740 1 client.17742 shutdown 2024-09-12T03:11:58.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.685+0000 7f1519054740 1 -- 172.21.15.116:0/1021128734 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea962685d0 msgr2=0x55ea96272db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:58.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.685+0000 7f1519054740 1 --2- 172.21.15.116:0/1021128734 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea962685d0 0x55ea96272db0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f15040cc650 tx=0x7f15040976a0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.685+0000 7f1519054740 1 -- 172.21.15.116:0/1021128734 shutdown_connections 2024-09-12T03:11:58.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.685+0000 7f1519054740 1 --2- 172.21.15.116:0/1021128734 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55ea96282df0 0x55ea962851e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.685+0000 7f1519054740 1 --2- 172.21.15.116:0/1021128734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea96273370 0x55ea962707f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.685+0000 7f1519054740 1 --2- 172.21.15.116:0/1021128734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96269b40 0x55ea96270060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.685+0000 7f1519054740 1 --2- 172.21.15.116:0/1021128734 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea962685d0 0x55ea96272db0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.685+0000 7f1519054740 1 -- 172.21.15.116:0/1021128734 >> 172.21.15.116:0/1021128734 conn(0x55ea96231310 msgr2=0x55ea9626a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:58.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.685+0000 7f14effff640 0 client.17742 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:11:58.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.685+0000 7f1519054740 1 -- 172.21.15.116:0/1021128734 shutdown_connections 2024-09-12T03:11:58.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.685+0000 7f1519054740 1 -- 172.21.15.116:0/1021128734 wait complete. 2024-09-12T03:11:58.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.685+0000 7f1519054740 20 client.17742 trim_cache size 0 max 16384 2024-09-12T03:11:58.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.698+0000 7f1519054740 1 Processor -- start 2024-09-12T03:11:58.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.698+0000 7f1519054740 1 -- start start 2024-09-12T03:11:58.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.698+0000 7f1519054740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea962685d0 0x55ea9626b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:58.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.698+0000 7f1519054740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea96269b40 0x55ea9626b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:58.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.698+0000 7f1519054740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96273370 0x55ea96287e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:58.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.698+0000 7f1519054740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f15101afcd0 con 0x55ea96269b40 2024-09-12T03:11:58.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.698+0000 7f1519054740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f15101ad5a0 con 0x55ea962685d0 2024-09-12T03:11:58.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.698+0000 7f1519054740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f15101ab3f0 con 0x55ea96273370 2024-09-12T03:11:58.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.698+0000 7f150ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea96269b40 0x55ea9626b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:58.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.698+0000 7f150ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea96269b40 0x55ea9626b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48542/0 (socket says 172.21.15.116:48542) 2024-09-12T03:11:58.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.698+0000 7f150ffff640 1 -- 172.21.15.116:0/3132668345 learned_addr learned my addr 172.21.15.116:0/3132668345 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:58.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.698+0000 7f1514a17640 1 --2- 172.21.15.116:0/3132668345 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea962685d0 0x55ea9626b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:58.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.698+0000 7f1515218640 1 --2- 172.21.15.116:0/3132668345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96273370 0x55ea96287e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:58.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.699+0000 7f150ffff640 1 -- 172.21.15.116:0/3132668345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96273370 msgr2=0x55ea96287e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:58.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.699+0000 7f150ffff640 1 --2- 172.21.15.116:0/3132668345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96273370 0x55ea96287e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.699+0000 7f150ffff640 1 -- 172.21.15.116:0/3132668345 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea962685d0 msgr2=0x55ea9626b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:58.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.699+0000 7f150ffff640 1 --2- 172.21.15.116:0/3132668345 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea962685d0 0x55ea9626b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.699+0000 7f150ffff640 1 -- 172.21.15.116:0/3132668345 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea96265960 con 0x55ea96269b40 2024-09-12T03:11:58.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.699+0000 7f150ffff640 1 --2- 172.21.15.116:0/3132668345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea96269b40 0x55ea9626b660 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f14fc02a930 tx=0x7f14fc009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8c409df19ef856 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:58.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.699+0000 7f14fa7fc640 1 -- 172.21.15.116:0/3132668345 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc048070 con 0x55ea96269b40 2024-09-12T03:11:58.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.699+0000 7f14fa7fc640 1 -- 172.21.15.116:0/3132668345 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14fc005520 con 0x55ea96269b40 2024-09-12T03:11:58.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.699+0000 7f14fa7fc640 1 -- 172.21.15.116:0/3132668345 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc035030 con 0x55ea96269b40 2024-09-12T03:11:58.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.700+0000 7f1519054740 1 -- 172.21.15.116:0/3132668345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea96269b40 msgr2=0x55ea9626b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:58.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.700+0000 7f1519054740 1 --2- 172.21.15.116:0/3132668345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea96269b40 0x55ea9626b660 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f14fc02a930 tx=0x7f14fc009650 comp rx=0 tx=0).stop 2024-09-12T03:11:58.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.700+0000 7f1519054740 1 -- 172.21.15.116:0/3132668345 shutdown_connections 2024-09-12T03:11:58.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.700+0000 7f1519054740 1 --2- 172.21.15.116:0/3132668345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96273370 0x55ea96287e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.700+0000 7f1519054740 1 --2- 172.21.15.116:0/3132668345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea96269b40 0x55ea9626b660 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.700+0000 7f1519054740 1 --2- 172.21.15.116:0/3132668345 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea962685d0 0x55ea9626b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.700+0000 7f1519054740 1 -- 172.21.15.116:0/3132668345 >> 172.21.15.116:0/3132668345 conn(0x55ea96231310 msgr2=0x55ea9627ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:58.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.700+0000 7f1519054740 1 -- 172.21.15.116:0/3132668345 shutdown_connections 2024-09-12T03:11:58.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.700+0000 7f1519054740 1 -- 172.21.15.116:0/3132668345 wait complete. 2024-09-12T03:11:58.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.700+0000 7f1519054740 1 -- start start 2024-09-12T03:11:58.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.700+0000 7f1519054740 1 Processor -- start 2024-09-12T03:11:58.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.701+0000 7f1519054740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea962685d0 0x55ea96285c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:58.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.701+0000 7f1519054740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea96269b40 0x55ea96286160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:58.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.701+0000 7f1519054740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea96273370 0x55ea962866a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:58.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.701+0000 7f1519054740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f15101aba60 con 0x55ea96273370 2024-09-12T03:11:58.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.701+0000 7f1519054740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f15101aced0 con 0x55ea96269b40 2024-09-12T03:11:58.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.701+0000 7f1519054740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f15101ac130 con 0x55ea962685d0 2024-09-12T03:11:58.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.701+0000 7f1519054740 10 client.0 _ll_register_callbacks cb 0x7f1508013640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:58.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.701+0000 7f1515218640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea96273370 0x55ea962866a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:58.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.701+0000 7f1515218640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea96273370 0x55ea962866a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48552/0 (socket says 172.21.15.116:48552) 2024-09-12T03:11:58.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.701+0000 7f1515218640 1 -- 172.21.15.116:0/1760564390 learned_addr learned my addr 172.21.15.116:0/1760564390 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:11:58.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.701+0000 7f1514a17640 1 --2- 172.21.15.116:0/1760564390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea962685d0 0x55ea96285c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:58.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.701+0000 7f150ffff640 1 --2- 172.21.15.116:0/1760564390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea96269b40 0x55ea96286160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:58.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.701+0000 7f1515218640 1 -- 172.21.15.116:0/1760564390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea962685d0 msgr2=0x55ea96285c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:58.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.701+0000 7f1515218640 1 --2- 172.21.15.116:0/1760564390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea962685d0 0x55ea96285c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.701+0000 7f1515218640 1 -- 172.21.15.116:0/1760564390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea96269b40 msgr2=0x55ea96286160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:58.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.701+0000 7f1515218640 1 --2- 172.21.15.116:0/1760564390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea96269b40 0x55ea96286160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.701+0000 7f1515218640 1 -- 172.21.15.116:0/1760564390 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea962ec7b0 con 0x55ea96273370 2024-09-12T03:11:58.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.701+0000 7f1515218640 1 --2- 172.21.15.116:0/1760564390 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea96273370 0x55ea962866a0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f1508010d50 tx=0x7f15080072a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcae53d6bf4e0fe1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:58.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.701+0000 7f14effff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:11:58.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.702+0000 7f14effff640 1 -- 172.21.15.116:0/1760564390 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15080076b0 con 0x55ea96273370 2024-09-12T03:11:58.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.702+0000 7f14effff640 1 -- 172.21.15.116:0/1760564390 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1508004b20 con 0x55ea96273370 2024-09-12T03:11:58.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.702+0000 7f1519054740 1 -- 172.21.15.116:0/1760564390 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14fc005520 con 0x55ea96273370 2024-09-12T03:11:58.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.702+0000 7f14effff640 1 -- 172.21.15.116:0/1760564390 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1508031070 con 0x55ea96273370 2024-09-12T03:11:58.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.702+0000 7f1519054740 10 client.17748 fetch_fsmap learned FSMap version 25 2024-09-12T03:11:58.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.702+0000 7f1519054740 1 -- 172.21.15.116:0/1760564390 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ea962eda90 con 0x55ea96273370 2024-09-12T03:11:58.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.702+0000 7f14effff640 1 -- 172.21.15.116:0/1760564390 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14fc005520 con 0x55ea96273370 2024-09-12T03:11:58.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.702+0000 7f14effff640 1 -- 172.21.15.116:0/1760564390 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1508005020 con 0x55ea96273370 2024-09-12T03:11:58.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.702+0000 7f1519054740 10 client.17748 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:11:58.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.702+0000 7f1519054740 10 client.17748 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:11:58.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.702+0000 7f1519054740 20 client.17748 populate_metadata read hostname 'smithi116' 2024-09-12T03:11:58.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.702+0000 7f1519054740 1 --2- 172.21.15.116:0/1760564390 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55ea962ee210 0x55ea962f0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:58.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.702+0000 7f1519054740 4 client.17748 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:11:58.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.702+0000 7f1519054740 1 -- 172.21.15.116:0/1760564390 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ea961d5d70 con 0x55ea962ee210 2024-09-12T03:11:58.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.702+0000 7f1514a17640 1 --2- 172.21.15.116:0/1760564390 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55ea962ee210 0x55ea962f0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:58.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.703+0000 7f1514a17640 1 --2- 172.21.15.116:0/1760564390 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55ea962ee210 0x55ea962f0650 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc6a6284672e1cda server_cookie=7688fae4dbfd7707 in_seq=0 out_seq=0 2024-09-12T03:11:58.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.703+0000 7f14effff640 10 client.17748 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:11:58.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.703+0000 7f14effff640 1 -- 172.21.15.116:0/1760564390 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ea961d5d70 con 0x55ea962ee210 2024-09-12T03:11:58.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.703+0000 7f14effff640 10 client.17748 handle_command_reply: tid=0 2024-09-12T03:11:58.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.703+0000 7f1519054740 1 client.17748 shutdown 2024-09-12T03:11:58.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.703+0000 7f1519054740 1 -- 172.21.15.116:0/1760564390 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea96273370 msgr2=0x55ea962866a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:58.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.703+0000 7f1519054740 1 --2- 172.21.15.116:0/1760564390 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea96273370 0x55ea962866a0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f1508010d50 tx=0x7f15080072a0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.703+0000 7f1519054740 1 -- 172.21.15.116:0/1760564390 shutdown_connections 2024-09-12T03:11:58.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.703+0000 7f1519054740 1 --2- 172.21.15.116:0/1760564390 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55ea962ee210 0x55ea962f0650 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.703+0000 7f1519054740 1 --2- 172.21.15.116:0/1760564390 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea96273370 0x55ea962866a0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.703+0000 7f1519054740 1 --2- 172.21.15.116:0/1760564390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea96269b40 0x55ea96286160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.703+0000 7f1519054740 1 --2- 172.21.15.116:0/1760564390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea962685d0 0x55ea96285c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.703+0000 7f1519054740 1 -- 172.21.15.116:0/1760564390 >> 172.21.15.116:0/1760564390 conn(0x55ea96231310 msgr2=0x55ea9627e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:58.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.703+0000 7f14effff640 0 client.17748 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:11:58.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.703+0000 7f1519054740 1 -- 172.21.15.116:0/1760564390 shutdown_connections 2024-09-12T03:11:58.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.703+0000 7f1519054740 1 -- 172.21.15.116:0/1760564390 wait complete. 2024-09-12T03:11:58.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.704+0000 7f1519054740 20 client.17748 trim_cache size 0 max 16384 2024-09-12T03:11:58.711 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:11:58.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.704+0000 7f1516c7b640 1 -- 172.21.15.116:0/3782252832 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f15000a2c00 msgr2=0x7f15000a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:58.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.704+0000 7f1516c7b640 1 --2- 172.21.15.116:0/3782252832 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f15000a2c00 0x7f15000a50c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1508007fd0 tx=0x7f1508007d00 comp rx=0 tx=0).stop 2024-09-12T03:11:58.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.704+0000 7f1516c7b640 1 -- 172.21.15.116:0/3782252832 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1510101de0 msgr2=0x7f15101983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:58.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.704+0000 7f1516c7b640 1 --2- 172.21.15.116:0/3782252832 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1510101de0 0x7f15101983d0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f150400d770 tx=0x7f150400dc30 comp rx=0 tx=0).stop 2024-09-12T03:11:58.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.704+0000 7f1516c7b640 1 -- 172.21.15.116:0/3782252832 shutdown_connections 2024-09-12T03:11:58.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.704+0000 7f1516c7b640 1 --2- 172.21.15.116:0/3782252832 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f15000a2c00 0x7f15000a50c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.704+0000 7f1516c7b640 1 --2- 172.21.15.116:0/3782252832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1510106f10 0x7f151019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.704+0000 7f1516c7b640 1 --2- 172.21.15.116:0/3782252832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1510102790 0x7f1510198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.704+0000 7f1516c7b640 1 --2- 172.21.15.116:0/3782252832 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1510101de0 0x7f15101983d0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:58.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.704+0000 7f1516c7b640 1 -- 172.21.15.116:0/3782252832 >> 172.21.15.116:0/3782252832 conn(0x7f15100fd8c0 msgr2=0x7f15100fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:58.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.704+0000 7f1516c7b640 1 -- 172.21.15.116:0/3782252832 shutdown_connections 2024-09-12T03:11:58.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:11:58.704+0000 7f1516c7b640 1 -- 172.21.15.116:0/3782252832 wait complete. 2024-09-12T03:11:59.511 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:11:59.757 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:12:00.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.267+0000 7f0510819640 1 -- 172.21.15.116:0/338501480 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f050c10c180 msgr2=0x7f050c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:00.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.267+0000 7f0510819640 1 --2- 172.21.15.116:0/338501480 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f050c10c180 0x7f050c10e610 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f0504009ca0 tx=0x7f050402a950 comp rx=0 tx=0).stop 2024-09-12T03:12:00.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.268+0000 7f0510819640 1 -- 172.21.15.116:0/338501480 shutdown_connections 2024-09-12T03:12:00.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.268+0000 7f0510819640 1 --2- 172.21.15.116:0/338501480 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f050c10c180 0x7f050c10e610 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.268+0000 7f0510819640 1 --2- 172.21.15.116:0/338501480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f050c103700 0x7f050c10bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.268+0000 7f0510819640 1 --2- 172.21.15.116:0/338501480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f050c102de0 0x7f050c1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.268+0000 7f0510819640 1 -- 172.21.15.116:0/338501480 >> 172.21.15.116:0/338501480 conn(0x7f050c0fc7a0 msgr2=0x7f050c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:00.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.268+0000 7f0510819640 1 -- 172.21.15.116:0/338501480 shutdown_connections 2024-09-12T03:12:00.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.268+0000 7f0510819640 1 -- 172.21.15.116:0/338501480 wait complete. 2024-09-12T03:12:00.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.268+0000 7f0510819640 1 Processor -- start 2024-09-12T03:12:00.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.268+0000 7f0510819640 1 -- start start 2024-09-12T03:12:00.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.269+0000 7f0510819640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f050c102de0 0x7f050c198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:00.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.269+0000 7f0510819640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f050c103700 0x7f050c198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:00.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.269+0000 7f0510819640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f050c10c180 0x7f050c19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:00.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.269+0000 7f0510819640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f050c112030 con 0x7f050c10c180 2024-09-12T03:12:00.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.269+0000 7f0510819640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f050c111eb0 con 0x7f050c103700 2024-09-12T03:12:00.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.269+0000 7f0510819640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f050c1121b0 con 0x7f050c102de0 2024-09-12T03:12:00.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.269+0000 7f050ad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f050c10c180 0x7f050c19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:00.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.269+0000 7f050ad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f050c10c180 0x7f050c19ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48568/0 (socket says 172.21.15.116:48568) 2024-09-12T03:12:00.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.269+0000 7f050ad9d640 1 -- 172.21.15.116:0/698498404 learned_addr learned my addr 172.21.15.116:0/698498404 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:00.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.269+0000 7f050a59c640 1 --2- 172.21.15.116:0/698498404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f050c102de0 0x7f050c198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:00.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.269+0000 7f0509d9b640 1 --2- 172.21.15.116:0/698498404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f050c103700 0x7f050c198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:00.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.269+0000 7f050ad9d640 1 -- 172.21.15.116:0/698498404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f050c102de0 msgr2=0x7f050c198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:00.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.269+0000 7f050ad9d640 1 --2- 172.21.15.116:0/698498404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f050c102de0 0x7f050c198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.269+0000 7f050ad9d640 1 -- 172.21.15.116:0/698498404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f050c103700 msgr2=0x7f050c198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:00.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.269+0000 7f050ad9d640 1 --2- 172.21.15.116:0/698498404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f050c103700 0x7f050c198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.269+0000 7f050ad9d640 1 -- 172.21.15.116:0/698498404 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f050c19d4f0 con 0x7f050c10c180 2024-09-12T03:12:00.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.269+0000 7f050a59c640 1 --2- 172.21.15.116:0/698498404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f050c102de0 0x7f050c198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:12:00.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.269+0000 7f050ad9d640 1 --2- 172.21.15.116:0/698498404 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f050c10c180 0x7f050c19ce10 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f0504006010 tx=0x7f050402f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:00.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.269+0000 7f04fb7fe640 1 -- 172.21.15.116:0/698498404 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0504034040 con 0x7f050c10c180 2024-09-12T03:12:00.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.269+0000 7f0510819640 1 -- 172.21.15.116:0/698498404 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f050c19d780 con 0x7f050c10c180 2024-09-12T03:12:00.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.269+0000 7f04fb7fe640 1 -- 172.21.15.116:0/698498404 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0504003e00 con 0x7f050c10c180 2024-09-12T03:12:00.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.269+0000 7f04fb7fe640 1 -- 172.21.15.116:0/698498404 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05040456e0 con 0x7f050c10c180 2024-09-12T03:12:00.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.269+0000 7f0510819640 1 -- 172.21.15.116:0/698498404 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f050c1a50c0 con 0x7f050c10c180 2024-09-12T03:12:00.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.272+0000 7f04fb7fe640 1 -- 172.21.15.116:0/698498404 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f0504037020 con 0x7f050c10c180 2024-09-12T03:12:00.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.272+0000 7f0512bf2740 1 Processor -- start 2024-09-12T03:12:00.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.272+0000 7f0512bf2740 1 -- start start 2024-09-12T03:12:00.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.272+0000 7f0512bf2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c4b302b40 0x556c4b301090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:00.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.272+0000 7f0512bf2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c4b3015d0 0x556c4b301a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:00.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.272+0000 7f0512bf2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c4b30c370 0x556c4b30e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:00.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.272+0000 7f0512bf2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556c4b2672f0 con 0x556c4b302b40 2024-09-12T03:12:00.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.272+0000 7f0512bf2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556c4b20ecf0 con 0x556c4b3015d0 2024-09-12T03:12:00.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.272+0000 7f0512bf2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556c4b2632b0 con 0x556c4b30c370 2024-09-12T03:12:00.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.272+0000 7f0509d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c4b302b40 0x556c4b301090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:00.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.273+0000 7f0509d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c4b302b40 0x556c4b301090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48570/0 (socket says 172.21.15.116:48570) 2024-09-12T03:12:00.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.273+0000 7f0509d9b640 1 -- 172.21.15.116:0/2208506012 learned_addr learned my addr 172.21.15.116:0/2208506012 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:00.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.273+0000 7f04fb7fe640 1 --2- 172.21.15.116:0/698498404 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f04f40a2b80 0x7f04f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:00.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.273+0000 7f0509d9b640 1 --2- 172.21.15.116:0/2208506012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c4b30c370 0x556c4b30e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:00.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.273+0000 7f050a59c640 1 --2- 172.21.15.116:0/2208506012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c4b3015d0 0x556c4b301a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:00.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.273+0000 7f0509d9b640 1 -- 172.21.15.116:0/2208506012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c4b30c370 msgr2=0x556c4b30e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:00.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.273+0000 7f0509d9b640 1 --2- 172.21.15.116:0/2208506012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c4b30c370 0x556c4b30e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.273+0000 7f050ad9d640 1 --2- 172.21.15.116:0/698498404 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f04f40a2b80 0x7f04f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:00.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.273+0000 7f0509d9b640 1 -- 172.21.15.116:0/2208506012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c4b3015d0 msgr2=0x556c4b301a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:00.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.273+0000 7f0509d9b640 1 --2- 172.21.15.116:0/2208506012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c4b3015d0 0x556c4b301a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.273+0000 7f0509d9b640 1 -- 172.21.15.116:0/2208506012 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c4b1f33a0 con 0x556c4b302b40 2024-09-12T03:12:00.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.273+0000 7f04fb7fe640 1 -- 172.21.15.116:0/698498404 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f05040f0180 con 0x7f050c10c180 2024-09-12T03:12:00.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.273+0000 7f0509d9b640 1 --2- 172.21.15.116:0/2208506012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c4b302b40 0x556c4b301090 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f0500006e10 tx=0x7f0500009420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ab2423bef8a374 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:00.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.273+0000 7f04f8ff9640 1 -- 172.21.15.116:0/2208506012 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050000fe60 con 0x556c4b302b40 2024-09-12T03:12:00.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.273+0000 7f04f8ff9640 1 -- 172.21.15.116:0/2208506012 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f050001e070 con 0x556c4b302b40 2024-09-12T03:12:00.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.273+0000 7f04f8ff9640 1 -- 172.21.15.116:0/2208506012 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0500014b60 con 0x556c4b302b40 2024-09-12T03:12:00.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.273+0000 7f050ad9d640 1 --2- 172.21.15.116:0/698498404 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f04f40a2b80 0x7f04f40a5040 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f05040bc9a0 tx=0x7f050408c8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:00.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.274+0000 7f0512bf2740 1 -- 172.21.15.116:0/2208506012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c4b302b40 msgr2=0x556c4b301090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.274+0000 7f0512bf2740 1 --2- 172.21.15.116:0/2208506012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c4b302b40 0x556c4b301090 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f0500006e10 tx=0x7f0500009420 comp rx=0 tx=0).stop 2024-09-12T03:12:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.274+0000 7f0512bf2740 1 -- 172.21.15.116:0/2208506012 shutdown_connections 2024-09-12T03:12:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.274+0000 7f0512bf2740 1 --2- 172.21.15.116:0/2208506012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c4b30c370 0x556c4b30e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.274+0000 7f0512bf2740 1 --2- 172.21.15.116:0/2208506012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c4b3015d0 0x556c4b301a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.274+0000 7f0512bf2740 1 --2- 172.21.15.116:0/2208506012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c4b302b40 0x556c4b301090 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.274+0000 7f0512bf2740 1 -- 172.21.15.116:0/2208506012 >> 172.21.15.116:0/2208506012 conn(0x556c4b2ca310 msgr2=0x556c4b2fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.274+0000 7f0512bf2740 1 -- 172.21.15.116:0/2208506012 shutdown_connections 2024-09-12T03:12:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.274+0000 7f0512bf2740 1 -- 172.21.15.116:0/2208506012 wait complete. 2024-09-12T03:12:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.274+0000 7f0512bf2740 1 -- start start 2024-09-12T03:12:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.275+0000 7f0512bf2740 1 Processor -- start 2024-09-12T03:12:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.275+0000 7f0512bf2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c4b3015d0 0x556c4b30bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.275+0000 7f0512bf2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c4b302b40 0x556c4b309090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.275+0000 7f0512bf2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c4b30c370 0x556c4b3097d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.275+0000 7f0512bf2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556c4b30f070 con 0x556c4b30c370 2024-09-12T03:12:00.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.275+0000 7f0512bf2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556c4b222060 con 0x556c4b3015d0 2024-09-12T03:12:00.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.275+0000 7f0512bf2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556c4b30f1f0 con 0x556c4b302b40 2024-09-12T03:12:00.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.275+0000 7f0512bf2740 10 client.0 _ll_register_callbacks cb 0x556c4af758c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:00.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.275+0000 7f0509d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c4b30c370 0x556c4b3097d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:00.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.275+0000 7f0509d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c4b30c370 0x556c4b3097d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48578/0 (socket says 172.21.15.116:48578) 2024-09-12T03:12:00.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.275+0000 7f0509d9b640 1 -- 172.21.15.116:0/977683547 learned_addr learned my addr 172.21.15.116:0/977683547 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:00.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.275+0000 7f050a59c640 1 --2- 172.21.15.116:0/977683547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c4b302b40 0x556c4b309090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:00.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.275+0000 7f0509d9b640 1 --2- 172.21.15.116:0/977683547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c4b3015d0 0x556c4b30bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:00.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.275+0000 7f0509d9b640 1 -- 172.21.15.116:0/977683547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c4b302b40 msgr2=0x556c4b309090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:00.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.275+0000 7f0509d9b640 1 --2- 172.21.15.116:0/977683547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c4b302b40 0x556c4b309090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.275+0000 7f0509d9b640 1 -- 172.21.15.116:0/977683547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c4b3015d0 msgr2=0x556c4b30bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:00.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.275+0000 7f0509d9b640 1 --2- 172.21.15.116:0/977683547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c4b3015d0 0x556c4b30bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.275+0000 7f0509d9b640 1 -- 172.21.15.116:0/977683547 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c4afd83c0 con 0x556c4b30c370 2024-09-12T03:12:00.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.275+0000 7f0509d9b640 1 --2- 172.21.15.116:0/977683547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c4b30c370 0x556c4b3097d0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f0500006cd0 tx=0x7f05000146a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3407faf69fbb676c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:00.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.275+0000 7f04f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:12:00.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.276+0000 7f04f1ffb640 1 -- 172.21.15.116:0/977683547 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050002b030 con 0x556c4b30c370 2024-09-12T03:12:00.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.276+0000 7f04f1ffb640 1 -- 172.21.15.116:0/977683547 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f0500019710 con 0x556c4b30c370 2024-09-12T03:12:00.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.276+0000 7f0512bf2740 1 -- 172.21.15.116:0/977683547 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f050001e070 con 0x556c4b30c370 2024-09-12T03:12:00.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.276+0000 7f04f1ffb640 1 -- 172.21.15.116:0/977683547 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0500019910 con 0x556c4b30c370 2024-09-12T03:12:00.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.276+0000 7f0512bf2740 10 client.17760 fetch_fsmap learned FSMap version 25 2024-09-12T03:12:00.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.276+0000 7f0512bf2740 1 -- 172.21.15.116:0/977683547 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556c4b31aba0 con 0x556c4b30c370 2024-09-12T03:12:00.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.276+0000 7f04f1ffb640 1 -- 172.21.15.116:0/977683547 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f050001e070 con 0x556c4b30c370 2024-09-12T03:12:00.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.276+0000 7f04f1ffb640 1 -- 172.21.15.116:0/977683547 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0500024a80 con 0x556c4b30c370 2024-09-12T03:12:00.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.276+0000 7f0512bf2740 10 client.17760 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:12:00.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.276+0000 7f0512bf2740 10 client.17760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:12:00.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.276+0000 7f0512bf2740 20 client.17760 populate_metadata read hostname 'smithi116' 2024-09-12T03:12:00.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.276+0000 7f0512bf2740 1 --2- 172.21.15.116:0/977683547 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556c4b31bef0 0x556c4b31e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:00.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.276+0000 7f0512bf2740 4 client.17760 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:12:00.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.276+0000 7f0512bf2740 1 -- 172.21.15.116:0/977683547 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c4b274860 con 0x556c4b31bef0 2024-09-12T03:12:00.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.277+0000 7f050a59c640 1 --2- 172.21.15.116:0/977683547 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556c4b31bef0 0x556c4b31e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:00.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.277+0000 7f050a59c640 1 --2- 172.21.15.116:0/977683547 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556c4b31bef0 0x556c4b31e2e0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48278a2f972590d7 server_cookie=f70966353808bb80 in_seq=0 out_seq=0 2024-09-12T03:12:00.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.277+0000 7f04f1ffb640 10 client.17760 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:00.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.278+0000 7f04f1ffb640 1 -- 172.21.15.116:0/977683547 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556c4b274860 con 0x556c4b31bef0 2024-09-12T03:12:00.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.278+0000 7f04f1ffb640 10 client.17760 handle_command_reply: tid=0 2024-09-12T03:12:00.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.278+0000 7f0512bf2740 1 client.17760 shutdown 2024-09-12T03:12:00.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.278+0000 7f0512bf2740 1 -- 172.21.15.116:0/977683547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c4b30c370 msgr2=0x556c4b3097d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:00.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.278+0000 7f0512bf2740 1 --2- 172.21.15.116:0/977683547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c4b30c370 0x556c4b3097d0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f0500006cd0 tx=0x7f05000146a0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.278+0000 7f0512bf2740 1 -- 172.21.15.116:0/977683547 shutdown_connections 2024-09-12T03:12:00.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.278+0000 7f0512bf2740 1 --2- 172.21.15.116:0/977683547 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556c4b31bef0 0x556c4b31e2e0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.278+0000 7f0512bf2740 1 --2- 172.21.15.116:0/977683547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c4b30c370 0x556c4b3097d0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.278+0000 7f0512bf2740 1 --2- 172.21.15.116:0/977683547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c4b302b40 0x556c4b309090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.278+0000 7f04f1ffb640 0 client.17760 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:00.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.278+0000 7f0512bf2740 1 --2- 172.21.15.116:0/977683547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c4b3015d0 0x556c4b30bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.278+0000 7f0512bf2740 1 -- 172.21.15.116:0/977683547 >> 172.21.15.116:0/977683547 conn(0x556c4b2ca310 msgr2=0x556c4b303930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:00.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.278+0000 7f0512bf2740 1 -- 172.21.15.116:0/977683547 shutdown_connections 2024-09-12T03:12:00.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.278+0000 7f0512bf2740 1 -- 172.21.15.116:0/977683547 wait complete. 2024-09-12T03:12:00.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.278+0000 7f0512bf2740 20 client.17760 trim_cache size 0 max 16384 2024-09-12T03:12:00.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.291+0000 7f0512bf2740 1 Processor -- start 2024-09-12T03:12:00.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.291+0000 7f0512bf2740 1 -- start start 2024-09-12T03:12:00.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.291+0000 7f0512bf2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c4b3015d0 0x556c4b3184f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:00.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.291+0000 7f0512bf2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c4b302b40 0x556c4b304120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:00.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.291+0000 7f0512bf2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c4b30c370 0x556c4b304660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:00.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.291+0000 7f0512bf2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f050c1adbc0 con 0x556c4b3015d0 2024-09-12T03:12:00.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.291+0000 7f0512bf2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f050c1ab490 con 0x556c4b30c370 2024-09-12T03:12:00.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.291+0000 7f0512bf2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f050c1a9300 con 0x556c4b302b40 2024-09-12T03:12:00.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.291+0000 7f0509d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c4b3015d0 0x556c4b3184f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:00.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.291+0000 7f0509d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c4b3015d0 0x556c4b3184f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48580/0 (socket says 172.21.15.116:48580) 2024-09-12T03:12:00.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.291+0000 7f0509d9b640 1 -- 172.21.15.116:0/2103813910 learned_addr learned my addr 172.21.15.116:0/2103813910 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:00.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.291+0000 7f0509d9b640 1 -- 172.21.15.116:0/2103813910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c4b302b40 msgr2=0x556c4b304120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:00.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.291+0000 7f050a59c640 1 --2- 172.21.15.116:0/2103813910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c4b302b40 0x556c4b304120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:00.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.291+0000 7f0509d9b640 1 --2- 172.21.15.116:0/2103813910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c4b302b40 0x556c4b304120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.291+0000 7f0509d9b640 1 -- 172.21.15.116:0/2103813910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c4b30c370 msgr2=0x556c4b304660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:00.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.291+0000 7f0509d9b640 1 --2- 172.21.15.116:0/2103813910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c4b30c370 0x556c4b304660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.291+0000 7f0509d9b640 1 -- 172.21.15.116:0/2103813910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c4b2fded0 con 0x556c4b3015d0 2024-09-12T03:12:00.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.291+0000 7f0509d9b640 1 --2- 172.21.15.116:0/2103813910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c4b3015d0 0x556c4b3184f0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f0500023730 tx=0x7f05000090a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31245c8f4fe0e5b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:00.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.292+0000 7f04f3fff640 1 -- 172.21.15.116:0/2103813910 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050002b030 con 0x556c4b3015d0 2024-09-12T03:12:00.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.292+0000 7f04f3fff640 1 -- 172.21.15.116:0/2103813910 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f050001e040 con 0x556c4b3015d0 2024-09-12T03:12:00.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.292+0000 7f04f3fff640 1 -- 172.21.15.116:0/2103813910 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0500023ab0 con 0x556c4b3015d0 2024-09-12T03:12:00.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.292+0000 7f0512bf2740 1 -- 172.21.15.116:0/2103813910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c4b3015d0 msgr2=0x556c4b3184f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:00.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.292+0000 7f0512bf2740 1 --2- 172.21.15.116:0/2103813910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c4b3015d0 0x556c4b3184f0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f0500023730 tx=0x7f05000090a0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.292+0000 7f0512bf2740 1 -- 172.21.15.116:0/2103813910 shutdown_connections 2024-09-12T03:12:00.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.292+0000 7f0512bf2740 1 --2- 172.21.15.116:0/2103813910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c4b30c370 0x556c4b304660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.292+0000 7f0512bf2740 1 --2- 172.21.15.116:0/2103813910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c4b302b40 0x556c4b304120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.292+0000 7f0512bf2740 1 --2- 172.21.15.116:0/2103813910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c4b3015d0 0x556c4b3184f0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.292+0000 7f0512bf2740 1 -- 172.21.15.116:0/2103813910 >> 172.21.15.116:0/2103813910 conn(0x556c4b2ca310 msgr2=0x556c4b30b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:00.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.292+0000 7f0512bf2740 1 -- 172.21.15.116:0/2103813910 shutdown_connections 2024-09-12T03:12:00.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.292+0000 7f0512bf2740 1 -- 172.21.15.116:0/2103813910 wait complete. 2024-09-12T03:12:00.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.293+0000 7f0512bf2740 1 -- start start 2024-09-12T03:12:00.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.293+0000 7f0512bf2740 1 Processor -- start 2024-09-12T03:12:00.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.293+0000 7f0512bf2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c4b3015d0 0x556c4b314990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:00.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.293+0000 7f0512bf2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c4b302b40 0x556c4b30fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:00.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.293+0000 7f0512bf2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c4b30c370 0x556c4b310200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:00.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.293+0000 7f0512bf2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f050c1a9950 con 0x556c4b3015d0 2024-09-12T03:12:00.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.293+0000 7f0512bf2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f050c1aadc0 con 0x556c4b30c370 2024-09-12T03:12:00.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.293+0000 7f0512bf2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f050c1aa020 con 0x556c4b302b40 2024-09-12T03:12:00.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.293+0000 7f0512bf2740 10 client.0 _ll_register_callbacks cb 0x7f0500009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:00.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.293+0000 7f0509d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c4b3015d0 0x556c4b314990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:00.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.293+0000 7f0509d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c4b3015d0 0x556c4b314990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48588/0 (socket says 172.21.15.116:48588) 2024-09-12T03:12:00.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.293+0000 7f0509d9b640 1 -- 172.21.15.116:0/1606080297 learned_addr learned my addr 172.21.15.116:0/1606080297 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:00.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.293+0000 7f050a59c640 1 --2- 172.21.15.116:0/1606080297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c4b302b40 0x556c4b30fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:00.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.293+0000 7f0509d9b640 1 -- 172.21.15.116:0/1606080297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c4b302b40 msgr2=0x556c4b30fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:00.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.293+0000 7f0509d9b640 1 --2- 172.21.15.116:0/1606080297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c4b302b40 0x556c4b30fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.293+0000 7f0509d9b640 1 -- 172.21.15.116:0/1606080297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c4b30c370 msgr2=0x556c4b310200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:00.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.293+0000 7f0509d9b640 1 --2- 172.21.15.116:0/1606080297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c4b30c370 0x556c4b310200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.293+0000 7f0509d9b640 1 -- 172.21.15.116:0/1606080297 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c4b3107c0 con 0x556c4b3015d0 2024-09-12T03:12:00.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.293+0000 7f050a59c640 1 --2- 172.21.15.116:0/1606080297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c4b302b40 0x556c4b30fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:12:00.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.294+0000 7f0509d9b640 1 --2- 172.21.15.116:0/1606080297 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c4b3015d0 0x556c4b314990 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f0500000bc0 tx=0x7f050000fe60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df60f3cfaecd5fbf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:00.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.294+0000 7f04f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:12:00.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.294+0000 7f04f1ffb640 1 -- 172.21.15.116:0/1606080297 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050003f070 con 0x556c4b3015d0 2024-09-12T03:12:00.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.294+0000 7f04f1ffb640 1 -- 172.21.15.116:0/1606080297 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f0500023a50 con 0x556c4b3015d0 2024-09-12T03:12:00.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.294+0000 7f0512bf2740 1 -- 172.21.15.116:0/1606080297 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f050001e040 con 0x556c4b3015d0 2024-09-12T03:12:00.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.294+0000 7f04f1ffb640 1 -- 172.21.15.116:0/1606080297 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f050002b040 con 0x556c4b3015d0 2024-09-12T03:12:00.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.294+0000 7f0512bf2740 10 client.17766 fetch_fsmap learned FSMap version 25 2024-09-12T03:12:00.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.294+0000 7f0512bf2740 1 -- 172.21.15.116:0/1606080297 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556c4b386770 con 0x556c4b3015d0 2024-09-12T03:12:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.294+0000 7f04f1ffb640 1 -- 172.21.15.116:0/1606080297 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f050001e040 con 0x556c4b3015d0 2024-09-12T03:12:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.294+0000 7f04f1ffb640 1 -- 172.21.15.116:0/1606080297 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f050003a070 con 0x556c4b3015d0 2024-09-12T03:12:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.294+0000 7f0512bf2740 10 client.17766 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:12:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.294+0000 7f0512bf2740 10 client.17766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:12:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.294+0000 7f0512bf2740 20 client.17766 populate_metadata read hostname 'smithi116' 2024-09-12T03:12:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.294+0000 7f0512bf2740 1 --2- 172.21.15.116:0/1606080297 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556c4b31bef0 0x556c4b388c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.294+0000 7f0512bf2740 4 client.17766 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:12:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.294+0000 7f0512bf2740 1 -- 172.21.15.116:0/1606080297 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556c4b26ed70 con 0x556c4b31bef0 2024-09-12T03:12:00.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.295+0000 7f050a59c640 1 --2- 172.21.15.116:0/1606080297 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556c4b31bef0 0x556c4b388c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:00.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.295+0000 7f04f1ffb640 10 client.17766 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:00.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.295+0000 7f050a59c640 1 --2- 172.21.15.116:0/1606080297 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556c4b31bef0 0x556c4b388c50 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a451791d30890be server_cookie=6bc2090670b6a628 in_seq=0 out_seq=0 2024-09-12T03:12:00.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.295+0000 7f04f1ffb640 1 -- 172.21.15.116:0/1606080297 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556c4b26ed70 con 0x556c4b31bef0 2024-09-12T03:12:00.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.295+0000 7f04f1ffb640 10 client.17766 handle_command_reply: tid=0 2024-09-12T03:12:00.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.295+0000 7f0512bf2740 1 client.17766 shutdown 2024-09-12T03:12:00.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.295+0000 7f0512bf2740 1 -- 172.21.15.116:0/1606080297 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c4b3015d0 msgr2=0x556c4b314990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:00.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.295+0000 7f0512bf2740 1 --2- 172.21.15.116:0/1606080297 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c4b3015d0 0x556c4b314990 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f0500000bc0 tx=0x7f050000fe60 comp rx=0 tx=0).stop 2024-09-12T03:12:00.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.295+0000 7f0512bf2740 1 -- 172.21.15.116:0/1606080297 shutdown_connections 2024-09-12T03:12:00.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.295+0000 7f0512bf2740 1 --2- 172.21.15.116:0/1606080297 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556c4b31bef0 0x556c4b388c50 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.295+0000 7f0512bf2740 1 --2- 172.21.15.116:0/1606080297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c4b30c370 0x556c4b310200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.295+0000 7f0512bf2740 1 --2- 172.21.15.116:0/1606080297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c4b302b40 0x556c4b30fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.295+0000 7f04f1ffb640 0 client.17766 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:00.346 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:12:00.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.295+0000 7f0512bf2740 1 --2- 172.21.15.116:0/1606080297 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c4b3015d0 0x556c4b314990 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.295+0000 7f0512bf2740 1 -- 172.21.15.116:0/1606080297 >> 172.21.15.116:0/1606080297 conn(0x556c4b2ca310 msgr2=0x556c4b317b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:00.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.295+0000 7f0512bf2740 1 -- 172.21.15.116:0/1606080297 shutdown_connections 2024-09-12T03:12:00.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.295+0000 7f0512bf2740 1 -- 172.21.15.116:0/1606080297 wait complete. 2024-09-12T03:12:00.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.295+0000 7f0512bf2740 20 client.17766 trim_cache size 0 max 16384 2024-09-12T03:12:00.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.296+0000 7f0510819640 1 -- 172.21.15.116:0/698498404 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f04f40a2b80 msgr2=0x7f04f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:00.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.296+0000 7f0510819640 1 --2- 172.21.15.116:0/698498404 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f04f40a2b80 0x7f04f40a5040 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f05040bc9a0 tx=0x7f050408c8c0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.296+0000 7f0510819640 1 -- 172.21.15.116:0/698498404 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f050c10c180 msgr2=0x7f050c19ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:00.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.296+0000 7f0510819640 1 --2- 172.21.15.116:0/698498404 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f050c10c180 0x7f050c19ce10 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f0504006010 tx=0x7f050402f710 comp rx=0 tx=0).stop 2024-09-12T03:12:00.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.296+0000 7f0510819640 1 -- 172.21.15.116:0/698498404 shutdown_connections 2024-09-12T03:12:00.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.296+0000 7f0510819640 1 --2- 172.21.15.116:0/698498404 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f04f40a2b80 0x7f04f40a5040 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.296+0000 7f0510819640 1 --2- 172.21.15.116:0/698498404 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f050c10c180 0x7f050c19ce10 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.296+0000 7f0510819640 1 --2- 172.21.15.116:0/698498404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f050c103700 0x7f050c198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.296+0000 7f0510819640 1 --2- 172.21.15.116:0/698498404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f050c102de0 0x7f050c198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:00.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.296+0000 7f0510819640 1 -- 172.21.15.116:0/698498404 >> 172.21.15.116:0/698498404 conn(0x7f050c0fc7a0 msgr2=0x7f050c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:00.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.296+0000 7f0510819640 1 -- 172.21.15.116:0/698498404 shutdown_connections 2024-09-12T03:12:00.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:00.296+0000 7f0510819640 1 -- 172.21.15.116:0/698498404 wait complete. 2024-09-12T03:12:00.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:00 smithi116 ceph-mon[31410]: pgmap v608: 97 pgs: 97 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:12:00.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:00 smithi183 ceph-mon[35535]: pgmap v608: 97 pgs: 97 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:12:00.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:00 smithi177 ceph-mon[35305]: pgmap v608: 97 pgs: 97 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:12:01.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:01 smithi183 ceph-mon[35535]: pgmap v609: 97 pgs: 97 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-12T03:12:01.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:01 smithi177 ceph-mon[35305]: pgmap v609: 97 pgs: 97 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-12T03:12:01.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:01 smithi116 ceph-mon[31410]: pgmap v609: 97 pgs: 97 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-12T03:12:02.054 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:12:02.055 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:12:02.315 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:12:02.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.768+0000 7ff0aad54640 1 -- 172.21.15.116:0/1098544351 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff0a4101450 msgr2=0x7ff0a4105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:02.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.768+0000 7ff0aad54640 1 --2- 172.21.15.116:0/1098544351 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff0a4101450 0x7ff0a4105920 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7ff090009900 tx=0x7ff09002a470 comp rx=0 tx=0).stop 2024-09-12T03:12:02.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.768+0000 7ff0aad54640 1 -- 172.21.15.116:0/1098544351 shutdown_connections 2024-09-12T03:12:02.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.768+0000 7ff0aad54640 1 --2- 172.21.15.116:0/1098544351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0a4105f40 0x7ff0a41063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.768+0000 7ff0aad54640 1 --2- 172.21.15.116:0/1098544351 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff0a4101450 0x7ff0a4105920 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.768+0000 7ff0aad54640 1 --2- 172.21.15.116:0/1098544351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0a4100b30 0x7ff0a4100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.768+0000 7ff0aad54640 1 -- 172.21.15.116:0/1098544351 >> 172.21.15.116:0/1098544351 conn(0x7ff0a40fc7a0 msgr2=0x7ff0a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:02.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.768+0000 7ff0aad54640 1 -- 172.21.15.116:0/1098544351 shutdown_connections 2024-09-12T03:12:02.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.768+0000 7ff0aad54640 1 -- 172.21.15.116:0/1098544351 wait complete. 2024-09-12T03:12:02.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.769+0000 7ff0aad54640 1 Processor -- start 2024-09-12T03:12:02.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.769+0000 7ff0aad54640 1 -- start start 2024-09-12T03:12:02.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.769+0000 7ff0aad54640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0a4100b30 0x7ff0a41a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:02.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.769+0000 7ff0aad54640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff0a4101450 0x7ff0a41a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:02.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.769+0000 7ff0aad54640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0a4105f40 0x7ff0a419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:02.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.769+0000 7ff0aad54640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a4118560 con 0x7ff0a4101450 2024-09-12T03:12:02.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.769+0000 7ff0aad54640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a41183e0 con 0x7ff0a4100b30 2024-09-12T03:12:02.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.769+0000 7ff0aad54640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a41186e0 con 0x7ff0a4105f40 2024-09-12T03:12:02.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.770+0000 7ff0a3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff0a4101450 0x7ff0a41a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:02.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.770+0000 7ff0a3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff0a4101450 0x7ff0a41a4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42648/0 (socket says 172.21.15.116:42648) 2024-09-12T03:12:02.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.770+0000 7ff0a3fff640 1 -- 172.21.15.116:0/1084183030 learned_addr learned my addr 172.21.15.116:0/1084183030 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:02.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.770+0000 7ff0a8af0640 1 --2- 172.21.15.116:0/1084183030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0a4100b30 0x7ff0a41a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:02.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.770+0000 7ff0a92f1640 1 --2- 172.21.15.116:0/1084183030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0a4105f40 0x7ff0a419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:02.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.770+0000 7ff0a3fff640 1 -- 172.21.15.116:0/1084183030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0a4105f40 msgr2=0x7ff0a419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:02.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.770+0000 7ff0a3fff640 1 --2- 172.21.15.116:0/1084183030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0a4105f40 0x7ff0a419e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.770+0000 7ff0a3fff640 1 -- 172.21.15.116:0/1084183030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0a4100b30 msgr2=0x7ff0a41a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:02.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.770+0000 7ff0a3fff640 1 --2- 172.21.15.116:0/1084183030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0a4100b30 0x7ff0a41a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.770+0000 7ff0a3fff640 1 -- 172.21.15.116:0/1084183030 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a419efe0 con 0x7ff0a4101450 2024-09-12T03:12:02.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.770+0000 7ff0a3fff640 1 --2- 172.21.15.116:0/1084183030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff0a4101450 0x7ff0a41a4c30 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7ff090005ab0 tx=0x7ff0900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:02.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.770+0000 7ff0a1ffb640 1 -- 172.21.15.116:0/1084183030 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09003f030 con 0x7ff0a4101450 2024-09-12T03:12:02.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.770+0000 7ff0a1ffb640 1 -- 172.21.15.116:0/1084183030 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff090003ce0 con 0x7ff0a4101450 2024-09-12T03:12:02.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.770+0000 7ff0aad54640 1 -- 172.21.15.116:0/1084183030 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0a419f270 con 0x7ff0a4101450 2024-09-12T03:12:02.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.770+0000 7ff0a1ffb640 1 -- 172.21.15.116:0/1084183030 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff090035070 con 0x7ff0a4101450 2024-09-12T03:12:02.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.770+0000 7ff0aad54640 1 -- 172.21.15.116:0/1084183030 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0a41ab5c0 con 0x7ff0a4101450 2024-09-12T03:12:02.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.773+0000 7ff0a1ffb640 1 -- 172.21.15.116:0/1084183030 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ff090042030 con 0x7ff0a4101450 2024-09-12T03:12:02.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.773+0000 7ff0ad12d740 1 Processor -- start 2024-09-12T03:12:02.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.773+0000 7ff0ad12d740 1 -- start start 2024-09-12T03:12:02.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.773+0000 7ff0ad12d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e034e07b60 0x55e034e07f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:02.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.773+0000 7ff0ad12d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e034e08480 0x55e034e12800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:02.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.773+0000 7ff0ad12d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e034e12d40 0x55e034e15130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:02.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.773+0000 7ff0ad12d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e034d6d280 con 0x55e034e07b60 2024-09-12T03:12:02.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.773+0000 7ff0ad12d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e034d14b10 con 0x55e034e08480 2024-09-12T03:12:02.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.773+0000 7ff0ad12d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e034d69240 con 0x55e034e12d40 2024-09-12T03:12:02.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.773+0000 7ff0a92f1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e034e07b60 0x55e034e07f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:02.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.773+0000 7ff0a1ffb640 1 --2- 172.21.15.116:0/1084183030 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff0940a2b80 0x7ff0940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:02.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.773+0000 7ff0a92f1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e034e07b60 0x55e034e07f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42652/0 (socket says 172.21.15.116:42652) 2024-09-12T03:12:02.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.773+0000 7ff0a92f1640 1 -- 172.21.15.116:0/686043665 learned_addr learned my addr 172.21.15.116:0/686043665 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:02.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.774+0000 7ff0a92f1640 1 --2- 172.21.15.116:0/1084183030 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff0940a2b80 0x7ff0940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:02.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.774+0000 7ff0a8af0640 1 --2- 172.21.15.116:0/686043665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e034e08480 0x55e034e12800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:02.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.774+0000 7ff0a3fff640 1 --2- 172.21.15.116:0/686043665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e034e12d40 0x55e034e15130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:02.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.774+0000 7ff0a92f1640 1 -- 172.21.15.116:0/686043665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e034e12d40 msgr2=0x55e034e15130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:02.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.774+0000 7ff0a1ffb640 1 -- 172.21.15.116:0/1084183030 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff0900ef870 con 0x7ff0a4101450 2024-09-12T03:12:02.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.774+0000 7ff0a92f1640 1 --2- 172.21.15.116:0/686043665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e034e12d40 0x55e034e15130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.774+0000 7ff0a92f1640 1 -- 172.21.15.116:0/686043665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e034e08480 msgr2=0x55e034e12800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:02.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.774+0000 7ff0a92f1640 1 --2- 172.21.15.116:0/686043665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e034e08480 0x55e034e12800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.774+0000 7ff0a92f1640 1 -- 172.21.15.116:0/686043665 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e034d211f0 con 0x55e034e07b60 2024-09-12T03:12:02.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.774+0000 7ff0a8af0640 1 --2- 172.21.15.116:0/686043665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e034e08480 0x55e034e12800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:12:02.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.774+0000 7ff0a3fff640 1 --2- 172.21.15.116:0/686043665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e034e12d40 0x55e034e15130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:12:02.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.774+0000 7ff0a92f1640 1 --2- 172.21.15.116:0/686043665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e034e07b60 0x55e034e07f40 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7ff09c005fd0 tx=0x7ff09c014dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01604f40ae93da server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:02.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.774+0000 7ff08effd640 1 -- 172.21.15.116:0/686043665 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c021070 con 0x55e034e07b60 2024-09-12T03:12:02.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.774+0000 7ff08effd640 1 -- 172.21.15.116:0/686043665 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff09c014050 con 0x55e034e07b60 2024-09-12T03:12:02.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.774+0000 7ff08effd640 1 -- 172.21.15.116:0/686043665 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c013c80 con 0x55e034e07b60 2024-09-12T03:12:02.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.775+0000 7ff0ad12d740 1 -- 172.21.15.116:0/686043665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e034e07b60 msgr2=0x55e034e07f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:02.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.775+0000 7ff0a92f1640 1 --2- 172.21.15.116:0/1084183030 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff0940a2b80 0x7ff0940a5040 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff09c01e000 tx=0x7ff09c01a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:02.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.775+0000 7ff0ad12d740 1 --2- 172.21.15.116:0/686043665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e034e07b60 0x55e034e07f40 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7ff09c005fd0 tx=0x7ff09c014dc0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.775+0000 7ff0ad12d740 1 -- 172.21.15.116:0/686043665 shutdown_connections 2024-09-12T03:12:02.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.775+0000 7ff0ad12d740 1 --2- 172.21.15.116:0/686043665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e034e12d40 0x55e034e15130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.775+0000 7ff0ad12d740 1 --2- 172.21.15.116:0/686043665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e034e08480 0x55e034e12800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.775+0000 7ff0ad12d740 1 --2- 172.21.15.116:0/686043665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e034e07b60 0x55e034e07f40 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.775+0000 7ff0ad12d740 1 -- 172.21.15.116:0/686043665 >> 172.21.15.116:0/686043665 conn(0x55e034e040f0 msgr2=0x55e034e044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:02.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.775+0000 7ff0ad12d740 1 -- 172.21.15.116:0/686043665 shutdown_connections 2024-09-12T03:12:02.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.775+0000 7ff0ad12d740 1 -- 172.21.15.116:0/686043665 wait complete. 2024-09-12T03:12:02.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.775+0000 7ff0ad12d740 1 -- start start 2024-09-12T03:12:02.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.775+0000 7ff0ad12d740 1 Processor -- start 2024-09-12T03:12:02.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.776+0000 7ff0ad12d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e034e07b60 0x55e034e0fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:02.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.776+0000 7ff0ad12d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e034e08480 0x55e034e10170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:02.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.776+0000 7ff0ad12d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e034e12d40 0x55e034e106b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:02.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.776+0000 7ff0ad12d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e034e15910 con 0x55e034e12d40 2024-09-12T03:12:02.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.776+0000 7ff0ad12d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e034d27f10 con 0x55e034e07b60 2024-09-12T03:12:02.794 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.776+0000 7ff0ad12d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e034e15a90 con 0x55e034e08480 2024-09-12T03:12:02.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.776+0000 7ff0ad12d740 10 client.0 _ll_register_callbacks cb 0x55e034cf0a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:02.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.776+0000 7ff0a92f1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e034e12d40 0x55e034e106b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:02.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.776+0000 7ff0a92f1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e034e12d40 0x55e034e106b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42664/0 (socket says 172.21.15.116:42664) 2024-09-12T03:12:02.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.776+0000 7ff0a92f1640 1 -- 172.21.15.116:0/45045179 learned_addr learned my addr 172.21.15.116:0/45045179 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:02.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.776+0000 7ff0a8af0640 1 --2- 172.21.15.116:0/45045179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e034e07b60 0x55e034e0fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:02.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.776+0000 7ff0a3fff640 1 --2- 172.21.15.116:0/45045179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e034e08480 0x55e034e10170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:02.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.776+0000 7ff0a92f1640 1 -- 172.21.15.116:0/45045179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e034e08480 msgr2=0x55e034e10170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:02.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.776+0000 7ff0a92f1640 1 --2- 172.21.15.116:0/45045179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e034e08480 0x55e034e10170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.776+0000 7ff0a92f1640 1 -- 172.21.15.116:0/45045179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e034e07b60 msgr2=0x55e034e0fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:02.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.776+0000 7ff0a92f1640 1 --2- 172.21.15.116:0/45045179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e034e07b60 0x55e034e0fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.776+0000 7ff0a92f1640 1 -- 172.21.15.116:0/45045179 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e034ade6a0 con 0x55e034e12d40 2024-09-12T03:12:02.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.776+0000 7ff0a8af0640 1 --2- 172.21.15.116:0/45045179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e034e07b60 0x55e034e0fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:12:02.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.776+0000 7ff0a92f1640 1 --2- 172.21.15.116:0/45045179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e034e12d40 0x55e034e106b0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7ff09c011eb0 tx=0x7ff09c0145b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70706fac09a7074c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:02.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.776+0000 7ff083fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:12:02.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.777+0000 7ff083fff640 1 -- 172.21.15.116:0/45045179 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c021050 con 0x55e034e12d40 2024-09-12T03:12:02.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.777+0000 7ff083fff640 1 -- 172.21.15.116:0/45045179 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff09c004cc0 con 0x55e034e12d40 2024-09-12T03:12:02.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.777+0000 7ff0ad12d740 1 -- 172.21.15.116:0/45045179 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff09c014050 con 0x55e034e12d40 2024-09-12T03:12:02.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.777+0000 7ff083fff640 1 -- 172.21.15.116:0/45045179 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff09c013460 con 0x55e034e12d40 2024-09-12T03:12:02.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.777+0000 7ff0ad12d740 10 client.17778 fetch_fsmap learned FSMap version 25 2024-09-12T03:12:02.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.777+0000 7ff0ad12d740 1 -- 172.21.15.116:0/45045179 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e034e21490 con 0x55e034e12d40 2024-09-12T03:12:02.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.777+0000 7ff083fff640 1 -- 172.21.15.116:0/45045179 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff09c014050 con 0x55e034e12d40 2024-09-12T03:12:02.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.777+0000 7ff083fff640 1 -- 172.21.15.116:0/45045179 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff09c005610 con 0x55e034e12d40 2024-09-12T03:12:02.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.777+0000 7ff0ad12d740 10 client.17778 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:12:02.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.777+0000 7ff0ad12d740 10 client.17778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:12:02.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.777+0000 7ff0ad12d740 20 client.17778 populate_metadata read hostname 'smithi116' 2024-09-12T03:12:02.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.777+0000 7ff0ad12d740 1 --2- 172.21.15.116:0/45045179 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e034e228a0 0x55e034e24c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:02.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.777+0000 7ff0ad12d740 4 client.17778 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:12:02.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.777+0000 7ff0ad12d740 1 -- 172.21.15.116:0/45045179 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e034d7a7f0 con 0x55e034e228a0 2024-09-12T03:12:02.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.777+0000 7ff0a8af0640 1 --2- 172.21.15.116:0/45045179 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e034e228a0 0x55e034e24c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:02.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.778+0000 7ff0a8af0640 1 --2- 172.21.15.116:0/45045179 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e034e228a0 0x55e034e24c90 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f700cdeecfb3df4 server_cookie=5c66dc51d87b0b15 in_seq=0 out_seq=0 2024-09-12T03:12:02.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.778+0000 7ff083fff640 10 client.17778 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:02.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.778+0000 7ff083fff640 1 -- 172.21.15.116:0/45045179 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e034d7a7f0 con 0x55e034e228a0 2024-09-12T03:12:02.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.778+0000 7ff083fff640 10 client.17778 handle_command_reply: tid=0 2024-09-12T03:12:02.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.778+0000 7ff0ad12d740 1 client.17778 shutdown 2024-09-12T03:12:02.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.779+0000 7ff0ad12d740 1 -- 172.21.15.116:0/45045179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e034e12d40 msgr2=0x55e034e106b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:02.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.779+0000 7ff0ad12d740 1 --2- 172.21.15.116:0/45045179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e034e12d40 0x55e034e106b0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7ff09c011eb0 tx=0x7ff09c0145b0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.779+0000 7ff0ad12d740 1 -- 172.21.15.116:0/45045179 shutdown_connections 2024-09-12T03:12:02.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.779+0000 7ff0ad12d740 1 --2- 172.21.15.116:0/45045179 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e034e228a0 0x55e034e24c90 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.779+0000 7ff0ad12d740 1 --2- 172.21.15.116:0/45045179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e034e12d40 0x55e034e106b0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.779+0000 7ff0ad12d740 1 --2- 172.21.15.116:0/45045179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e034e08480 0x55e034e10170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.779+0000 7ff0ad12d740 1 --2- 172.21.15.116:0/45045179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e034e07b60 0x55e034e0fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.779+0000 7ff0ad12d740 1 -- 172.21.15.116:0/45045179 >> 172.21.15.116:0/45045179 conn(0x55e034e040f0 msgr2=0x55e034e03270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:02.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.779+0000 7ff083fff640 0 client.17778 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:02.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.779+0000 7ff0ad12d740 1 -- 172.21.15.116:0/45045179 shutdown_connections 2024-09-12T03:12:02.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.779+0000 7ff0ad12d740 1 -- 172.21.15.116:0/45045179 wait complete. 2024-09-12T03:12:02.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.779+0000 7ff0ad12d740 20 client.17778 trim_cache size 0 max 16384 2024-09-12T03:12:02.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.796+0000 7ff0ad12d740 1 Processor -- start 2024-09-12T03:12:02.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.796+0000 7ff0ad12d740 1 -- start start 2024-09-12T03:12:02.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.796+0000 7ff0ad12d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e034e07b60 0x55e034e11a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:02.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.796+0000 7ff0ad12d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e034e08480 0x55e034e2a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:02.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.796+0000 7ff0ad12d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e034e12d40 0x55e034e2aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:02.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.796+0000 7ff0ad12d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a41b6290 con 0x55e034e07b60 2024-09-12T03:12:02.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.796+0000 7ff0ad12d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a41b3b60 con 0x55e034e12d40 2024-09-12T03:12:02.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.796+0000 7ff0ad12d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a41b19b0 con 0x55e034e08480 2024-09-12T03:12:02.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.796+0000 7ff0a8af0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e034e07b60 0x55e034e11a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:02.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.796+0000 7ff0a8af0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e034e07b60 0x55e034e11a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42666/0 (socket says 172.21.15.116:42666) 2024-09-12T03:12:02.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.796+0000 7ff0a8af0640 1 -- 172.21.15.116:0/2654338376 learned_addr learned my addr 172.21.15.116:0/2654338376 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:02.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.797+0000 7ff0a3fff640 1 --2- 172.21.15.116:0/2654338376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e034e08480 0x55e034e2a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:02.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.797+0000 7ff0a92f1640 1 --2- 172.21.15.116:0/2654338376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e034e12d40 0x55e034e2aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:02.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.797+0000 7ff0a8af0640 1 -- 172.21.15.116:0/2654338376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e034e08480 msgr2=0x55e034e2a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:02.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.797+0000 7ff0a8af0640 1 --2- 172.21.15.116:0/2654338376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e034e08480 0x55e034e2a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.797+0000 7ff0a8af0640 1 -- 172.21.15.116:0/2654338376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e034e12d40 msgr2=0x55e034e2aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:02.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.797+0000 7ff0a8af0640 1 --2- 172.21.15.116:0/2654338376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e034e12d40 0x55e034e2aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.797+0000 7ff0a8af0640 1 -- 172.21.15.116:0/2654338376 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e034dfe680 con 0x55e034e07b60 2024-09-12T03:12:02.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.797+0000 7ff0a3fff640 1 --2- 172.21.15.116:0/2654338376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e034e08480 0x55e034e2a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:12:02.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.797+0000 7ff0a8af0640 1 --2- 172.21.15.116:0/2654338376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e034e07b60 0x55e034e11a90 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7ff0a419f9f0 tx=0x7ff09800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53fda66f13a8d78 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:02.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.797+0000 7ff08e7fc640 1 -- 172.21.15.116:0/2654338376 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff098004020 con 0x55e034e07b60 2024-09-12T03:12:02.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.797+0000 7ff08e7fc640 1 -- 172.21.15.116:0/2654338376 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0980041c0 con 0x55e034e07b60 2024-09-12T03:12:02.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.797+0000 7ff08e7fc640 1 -- 172.21.15.116:0/2654338376 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09801c860 con 0x55e034e07b60 2024-09-12T03:12:02.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.798+0000 7ff0ad12d740 1 -- 172.21.15.116:0/2654338376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e034e07b60 msgr2=0x55e034e11a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:02.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.798+0000 7ff0ad12d740 1 --2- 172.21.15.116:0/2654338376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e034e07b60 0x55e034e11a90 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7ff0a419f9f0 tx=0x7ff09800f7f0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.798+0000 7ff0ad12d740 1 -- 172.21.15.116:0/2654338376 shutdown_connections 2024-09-12T03:12:02.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.798+0000 7ff0ad12d740 1 --2- 172.21.15.116:0/2654338376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e034e12d40 0x55e034e2aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.798+0000 7ff0ad12d740 1 --2- 172.21.15.116:0/2654338376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e034e08480 0x55e034e2a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.798+0000 7ff0ad12d740 1 --2- 172.21.15.116:0/2654338376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e034e07b60 0x55e034e11a90 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.798+0000 7ff0ad12d740 1 -- 172.21.15.116:0/2654338376 >> 172.21.15.116:0/2654338376 conn(0x55e034e040f0 msgr2=0x55e034e11380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:02.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.798+0000 7ff0ad12d740 1 -- 172.21.15.116:0/2654338376 shutdown_connections 2024-09-12T03:12:02.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.798+0000 7ff0ad12d740 1 -- 172.21.15.116:0/2654338376 wait complete. 2024-09-12T03:12:02.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.798+0000 7ff0ad12d740 1 -- start start 2024-09-12T03:12:02.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.798+0000 7ff0ad12d740 1 Processor -- start 2024-09-12T03:12:02.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.799+0000 7ff0ad12d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e034e07b60 0x55e034e1b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:02.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.799+0000 7ff0ad12d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e034e08480 0x55e034e0c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:02.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.799+0000 7ff0ad12d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e034e12d40 0x55e034e0cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:02.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.799+0000 7ff0ad12d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a41b2020 con 0x55e034e12d40 2024-09-12T03:12:02.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.799+0000 7ff0ad12d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a41b3490 con 0x55e034e08480 2024-09-12T03:12:02.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.799+0000 7ff0ad12d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a41b26f0 con 0x55e034e07b60 2024-09-12T03:12:02.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.799+0000 7ff0ad12d740 10 client.0 _ll_register_callbacks cb 0x7ff09c01f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:02.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.799+0000 7ff0a92f1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e034e12d40 0x55e034e0cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:02.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.799+0000 7ff0a8af0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e034e07b60 0x55e034e1b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:02.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.799+0000 7ff0a92f1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e034e12d40 0x55e034e0cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42682/0 (socket says 172.21.15.116:42682) 2024-09-12T03:12:02.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.799+0000 7ff0a8af0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e034e07b60 0x55e034e1b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.116:42628/0 (socket says 172.21.15.116:42628) 2024-09-12T03:12:02.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.799+0000 7ff0a92f1640 1 -- 172.21.15.116:0/2167354387 learned_addr learned my addr 172.21.15.116:0/2167354387 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:02.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.799+0000 7ff0a3fff640 1 --2- 172.21.15.116:0/2167354387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e034e08480 0x55e034e0c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:02.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.799+0000 7ff0a92f1640 1 -- 172.21.15.116:0/2167354387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e034e07b60 msgr2=0x55e034e1b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:02.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.799+0000 7ff0a92f1640 1 --2- 172.21.15.116:0/2167354387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e034e07b60 0x55e034e1b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.799+0000 7ff0a92f1640 1 -- 172.21.15.116:0/2167354387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e034e08480 msgr2=0x55e034e0c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:02.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.799+0000 7ff0a92f1640 1 --2- 172.21.15.116:0/2167354387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e034e08480 0x55e034e0c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.799+0000 7ff0a92f1640 1 -- 172.21.15.116:0/2167354387 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e034e0d0f0 con 0x55e034e12d40 2024-09-12T03:12:02.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.800+0000 7ff0a92f1640 1 --2- 172.21.15.116:0/2167354387 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e034e12d40 0x55e034e0cbb0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7ff09c013eb0 tx=0x7ff09c005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a7772f80697de2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:02.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.800+0000 7ff083fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:12:02.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.800+0000 7ff083fff640 1 -- 172.21.15.116:0/2167354387 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c021070 con 0x55e034e12d40 2024-09-12T03:12:02.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.800+0000 7ff083fff640 1 -- 172.21.15.116:0/2167354387 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff09c013800 con 0x55e034e12d40 2024-09-12T03:12:02.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.800+0000 7ff0ad12d740 1 -- 172.21.15.116:0/2167354387 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0980041c0 con 0x55e034e12d40 2024-09-12T03:12:02.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.800+0000 7ff083fff640 1 -- 172.21.15.116:0/2167354387 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff09c048050 con 0x55e034e12d40 2024-09-12T03:12:02.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.800+0000 7ff0ad12d740 10 client.17784 fetch_fsmap learned FSMap version 25 2024-09-12T03:12:02.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.800+0000 7ff0ad12d740 1 -- 172.21.15.116:0/2167354387 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e034e8d210 con 0x55e034e12d40 2024-09-12T03:12:02.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.800+0000 7ff083fff640 1 -- 172.21.15.116:0/2167354387 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0980041c0 con 0x55e034e12d40 2024-09-12T03:12:02.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.800+0000 7ff083fff640 1 -- 172.21.15.116:0/2167354387 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff09c0386a0 con 0x55e034e12d40 2024-09-12T03:12:02.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.800+0000 7ff0ad12d740 10 client.17784 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:12:02.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.800+0000 7ff0ad12d740 10 client.17784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:12:02.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.800+0000 7ff0ad12d740 20 client.17784 populate_metadata read hostname 'smithi116' 2024-09-12T03:12:02.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.800+0000 7ff0ad12d740 1 --2- 172.21.15.116:0/2167354387 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e034e8d900 0x55e034e8fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:02.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.800+0000 7ff0ad12d740 4 client.17784 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:12:02.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.800+0000 7ff0ad12d740 1 -- 172.21.15.116:0/2167354387 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e034d74d00 con 0x55e034e8d900 2024-09-12T03:12:02.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.801+0000 7ff0a8af0640 1 --2- 172.21.15.116:0/2167354387 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e034e8d900 0x55e034e8fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:02.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.801+0000 7ff083fff640 10 client.17784 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:02.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.801+0000 7ff0a8af0640 1 --2- 172.21.15.116:0/2167354387 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e034e8d900 0x55e034e8fd20 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b09292cb8b7f53a2 server_cookie=3516f04815970691 in_seq=0 out_seq=0 2024-09-12T03:12:02.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.802+0000 7ff083fff640 1 -- 172.21.15.116:0/2167354387 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e034d74d00 con 0x55e034e8d900 2024-09-12T03:12:02.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.802+0000 7ff083fff640 10 client.17784 handle_command_reply: tid=0 2024-09-12T03:12:02.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.802+0000 7ff0ad12d740 1 client.17784 shutdown 2024-09-12T03:12:02.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.802+0000 7ff0ad12d740 1 -- 172.21.15.116:0/2167354387 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e034e12d40 msgr2=0x55e034e0cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:02.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.802+0000 7ff0ad12d740 1 --2- 172.21.15.116:0/2167354387 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e034e12d40 0x55e034e0cbb0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7ff09c013eb0 tx=0x7ff09c005c50 comp rx=0 tx=0).stop 2024-09-12T03:12:02.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.802+0000 7ff0ad12d740 1 -- 172.21.15.116:0/2167354387 shutdown_connections 2024-09-12T03:12:02.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.802+0000 7ff0ad12d740 1 --2- 172.21.15.116:0/2167354387 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e034e8d900 0x55e034e8fd20 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.802+0000 7ff083fff640 0 client.17784 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:02.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.802+0000 7ff0ad12d740 1 --2- 172.21.15.116:0/2167354387 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e034e12d40 0x55e034e0cbb0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.802+0000 7ff0ad12d740 1 --2- 172.21.15.116:0/2167354387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e034e08480 0x55e034e0c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.802+0000 7ff0ad12d740 1 --2- 172.21.15.116:0/2167354387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e034e07b60 0x55e034e1b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.802+0000 7ff0ad12d740 1 -- 172.21.15.116:0/2167354387 >> 172.21.15.116:0/2167354387 conn(0x55e034e040f0 msgr2=0x55e034e103f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:02.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.802+0000 7ff0ad12d740 1 -- 172.21.15.116:0/2167354387 shutdown_connections 2024-09-12T03:12:02.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.802+0000 7ff0ad12d740 1 -- 172.21.15.116:0/2167354387 wait complete. 2024-09-12T03:12:02.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.802+0000 7ff0ad12d740 20 client.17784 trim_cache size 0 max 16384 2024-09-12T03:12:02.810 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:12:02.810 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:12:02.810 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "75539e64-3c6c-4d98-9d24-ba56acde3e9f", 2024-09-12T03:12:02.810 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:12:02.810 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:12:02.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.803+0000 7ff0aad54640 1 -- 172.21.15.116:0/1084183030 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff0940a2b80 msgr2=0x7ff0940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:02.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.803+0000 7ff0aad54640 1 --2- 172.21.15.116:0/1084183030 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff0940a2b80 0x7ff0940a5040 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff09c01e000 tx=0x7ff09c01a000 comp rx=0 tx=0).stop 2024-09-12T03:12:02.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.803+0000 7ff0aad54640 1 -- 172.21.15.116:0/1084183030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff0a4101450 msgr2=0x7ff0a41a4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:02.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.803+0000 7ff0aad54640 1 --2- 172.21.15.116:0/1084183030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff0a4101450 0x7ff0a41a4c30 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7ff090005ab0 tx=0x7ff0900042f0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.803+0000 7ff0aad54640 1 -- 172.21.15.116:0/1084183030 shutdown_connections 2024-09-12T03:12:02.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.803+0000 7ff0aad54640 1 --2- 172.21.15.116:0/1084183030 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff0940a2b80 0x7ff0940a5040 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.803+0000 7ff0aad54640 1 --2- 172.21.15.116:0/1084183030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0a4105f40 0x7ff0a419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.803+0000 7ff0aad54640 1 --2- 172.21.15.116:0/1084183030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff0a4101450 0x7ff0a41a4c30 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.803+0000 7ff0aad54640 1 --2- 172.21.15.116:0/1084183030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0a4100b30 0x7ff0a41a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:02.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.803+0000 7ff0aad54640 1 -- 172.21.15.116:0/1084183030 >> 172.21.15.116:0/1084183030 conn(0x7ff0a40fc7a0 msgr2=0x7ff0a41038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:02.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.803+0000 7ff0aad54640 1 -- 172.21.15.116:0/1084183030 shutdown_connections 2024-09-12T03:12:02.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:02.803+0000 7ff0aad54640 1 -- 172.21.15.116:0/1084183030 wait complete. 2024-09-12T03:12:03.510 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:12:03.752 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:12:04.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.250+0000 7fad85a03640 1 -- 172.21.15.116:0/1059701274 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad80100900 msgr2=0x7fad80100d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:04.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.250+0000 7fad85a03640 1 --2- 172.21.15.116:0/1059701274 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad80100900 0x7fad80100d60 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fad68009930 tx=0x7fad6802a420 comp rx=0 tx=0).stop 2024-09-12T03:12:04.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.250+0000 7fad85a03640 1 -- 172.21.15.116:0/1059701274 shutdown_connections 2024-09-12T03:12:04.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.250+0000 7fad85a03640 1 --2- 172.21.15.116:0/1059701274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad801012a0 0x7fad8010d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.250+0000 7fad85a03640 1 --2- 172.21.15.116:0/1059701274 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad80100900 0x7fad80100d60 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.250+0000 7fad85a03640 1 --2- 172.21.15.116:0/1059701274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad80106900 0x7fad80106ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.250+0000 7fad85a03640 1 -- 172.21.15.116:0/1059701274 >> 172.21.15.116:0/1059701274 conn(0x7fad800fc600 msgr2=0x7fad800fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:04.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.250+0000 7fad85a03640 1 -- 172.21.15.116:0/1059701274 shutdown_connections 2024-09-12T03:12:04.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.250+0000 7fad85a03640 1 -- 172.21.15.116:0/1059701274 wait complete. 2024-09-12T03:12:04.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.250+0000 7fad85a03640 1 Processor -- start 2024-09-12T03:12:04.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.251+0000 7fad85a03640 1 -- start start 2024-09-12T03:12:04.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.251+0000 7fad85a03640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad80100900 0x7fad80199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:04.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.251+0000 7fad85a03640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad801012a0 0x7fad801996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:04.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.251+0000 7fad85a03640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad80106900 0x7fad801a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:04.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.251+0000 7fad85a03640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fad80110d00 con 0x7fad801012a0 2024-09-12T03:12:04.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.251+0000 7fad85a03640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fad80110b80 con 0x7fad80106900 2024-09-12T03:12:04.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.251+0000 7fad85a03640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fad80110e80 con 0x7fad80100900 2024-09-12T03:12:04.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.251+0000 7fad7e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad801012a0 0x7fad801996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:04.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.251+0000 7fad7e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad801012a0 0x7fad801996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42696/0 (socket says 172.21.15.116:42696) 2024-09-12T03:12:04.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.251+0000 7fad7e7fc640 1 -- 172.21.15.116:0/2656818151 learned_addr learned my addr 172.21.15.116:0/2656818151 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:04.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.251+0000 7fad7effd640 1 --2- 172.21.15.116:0/2656818151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad80100900 0x7fad80199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:04.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.251+0000 7fad7e7fc640 1 -- 172.21.15.116:0/2656818151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad80100900 msgr2=0x7fad80199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:04.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.251+0000 7fad7e7fc640 1 --2- 172.21.15.116:0/2656818151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad80100900 0x7fad80199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.251+0000 7fad7e7fc640 1 -- 172.21.15.116:0/2656818151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad80106900 msgr2=0x7fad801a4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:04.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.251+0000 7fad7e7fc640 1 --2- 172.21.15.116:0/2656818151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad80106900 0x7fad801a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.251+0000 7fad7e7fc640 1 -- 172.21.15.116:0/2656818151 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad801a4750 con 0x7fad801012a0 2024-09-12T03:12:04.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.251+0000 7fad7e7fc640 1 --2- 172.21.15.116:0/2656818151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad801012a0 0x7fad801996d0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fad68009230 tx=0x7fad68004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:04.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.252+0000 7fad84a01640 1 -- 172.21.15.116:0/2656818151 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6803f030 con 0x7fad801012a0 2024-09-12T03:12:04.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.252+0000 7fad84a01640 1 -- 172.21.15.116:0/2656818151 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad68003ad0 con 0x7fad801012a0 2024-09-12T03:12:04.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.252+0000 7fad84a01640 1 -- 172.21.15.116:0/2656818151 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad68035070 con 0x7fad801012a0 2024-09-12T03:12:04.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.252+0000 7fad85a03640 1 -- 172.21.15.116:0/2656818151 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad801a4920 con 0x7fad801012a0 2024-09-12T03:12:04.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.252+0000 7fad85a03640 1 -- 172.21.15.116:0/2656818151 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad801a4dd0 con 0x7fad801012a0 2024-09-12T03:12:04.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.254+0000 7fad84a01640 1 -- 172.21.15.116:0/2656818151 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fad68042030 con 0x7fad801012a0 2024-09-12T03:12:04.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.254+0000 7fad87ddc740 1 Processor -- start 2024-09-12T03:12:04.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.254+0000 7fad87ddc740 1 -- start start 2024-09-12T03:12:04.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.255+0000 7fad84a01640 1 --2- 172.21.15.116:0/2656818151 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fad6c0a2c00 0x7fad6c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:04.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.255+0000 7fad87ddc740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b53338c590 0x55b53338c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:04.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.255+0000 7fad87ddc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b53338ceb0 0x55b53339b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:04.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.255+0000 7fad87ddc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b53339bc60 0x55b53339e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:04.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.255+0000 7fad87ddc740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b5332f22e0 con 0x55b53338c590 2024-09-12T03:12:04.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.255+0000 7fad87ddc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b533299ce0 con 0x55b53339bc60 2024-09-12T03:12:04.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.255+0000 7fad87ddc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b5332ee2a0 con 0x55b53338ceb0 2024-09-12T03:12:04.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.255+0000 7fad84a01640 1 -- 172.21.15.116:0/2656818151 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fad680ef550 con 0x7fad801012a0 2024-09-12T03:12:04.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.255+0000 7fad7effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b53338c590 0x55b53338c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:04.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.255+0000 7fad7effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b53338c590 0x55b53338c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42706/0 (socket says 172.21.15.116:42706) 2024-09-12T03:12:04.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.255+0000 7fad7effd640 1 -- 172.21.15.116:0/1098775670 learned_addr learned my addr 172.21.15.116:0/1098775670 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:04.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.255+0000 7fad7f7fe640 1 --2- 172.21.15.116:0/2656818151 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fad6c0a2c00 0x7fad6c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:04.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.255+0000 7fad7e7fc640 1 --2- 172.21.15.116:0/1098775670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b53338ceb0 0x55b53339b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:04.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.255+0000 7fad7effd640 1 -- 172.21.15.116:0/1098775670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b53338ceb0 msgr2=0x55b53339b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:04.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.255+0000 7fad7effd640 1 --2- 172.21.15.116:0/1098775670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b53338ceb0 0x55b53339b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.255+0000 7fad7effd640 1 -- 172.21.15.116:0/1098775670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b53339bc60 msgr2=0x55b53339e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:04.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.255+0000 7fad7effd640 1 --2- 172.21.15.116:0/1098775670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b53339bc60 0x55b53339e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.255+0000 7fad7effd640 1 -- 172.21.15.116:0/1098775670 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b53327e440 con 0x55b53338c590 2024-09-12T03:12:04.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.255+0000 7fad7f7fe640 1 --2- 172.21.15.116:0/2656818151 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fad6c0a2c00 0x7fad6c0a50c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fad78005fd0 tx=0x7fad78007640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:04.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.255+0000 7fad7effd640 1 --2- 172.21.15.116:0/1098775670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b53338c590 0x55b53338c970 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fad70009b30 tx=0x7fad7000e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b88d38d7a0799a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:04.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.256+0000 7fad75ffb640 1 -- 172.21.15.116:0/1098775670 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad70016070 con 0x55b53338c590 2024-09-12T03:12:04.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.256+0000 7fad75ffb640 1 -- 172.21.15.116:0/1098775670 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad7000ed20 con 0x55b53338c590 2024-09-12T03:12:04.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.256+0000 7fad75ffb640 1 -- 172.21.15.116:0/1098775670 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad700106f0 con 0x55b53338c590 2024-09-12T03:12:04.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.256+0000 7fad87ddc740 1 -- 172.21.15.116:0/1098775670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b53338c590 msgr2=0x55b53338c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:04.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.256+0000 7fad87ddc740 1 --2- 172.21.15.116:0/1098775670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b53338c590 0x55b53338c970 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fad70009b30 tx=0x7fad7000e2f0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.256+0000 7fad87ddc740 1 -- 172.21.15.116:0/1098775670 shutdown_connections 2024-09-12T03:12:04.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.256+0000 7fad87ddc740 1 --2- 172.21.15.116:0/1098775670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b53339bc60 0x55b53339e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.256+0000 7fad87ddc740 1 --2- 172.21.15.116:0/1098775670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b53338ceb0 0x55b53339b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.256+0000 7fad87ddc740 1 --2- 172.21.15.116:0/1098775670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b53338c590 0x55b53338c970 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.256+0000 7fad87ddc740 1 -- 172.21.15.116:0/1098775670 >> 172.21.15.116:0/1098775670 conn(0x55b533388c00 msgr2=0x55b533388fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:04.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.256+0000 7fad87ddc740 1 -- 172.21.15.116:0/1098775670 shutdown_connections 2024-09-12T03:12:04.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.257+0000 7fad87ddc740 1 -- 172.21.15.116:0/1098775670 wait complete. 2024-09-12T03:12:04.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.257+0000 7fad87ddc740 1 -- start start 2024-09-12T03:12:04.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.257+0000 7fad87ddc740 1 Processor -- start 2024-09-12T03:12:04.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.257+0000 7fad87ddc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b53338c590 0x55b53339b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:04.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.257+0000 7fad87ddc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b53338ceb0 0x55b533398850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:04.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.257+0000 7fad87ddc740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b53339bc60 0x55b533398d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:04.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.257+0000 7fad87ddc740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b53339e7d0 con 0x55b53339bc60 2024-09-12T03:12:04.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.257+0000 7fad87ddc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b5332ad050 con 0x55b53338ceb0 2024-09-12T03:12:04.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.257+0000 7fad87ddc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b53339e950 con 0x55b53338c590 2024-09-12T03:12:04.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.257+0000 7fad87ddc740 10 client.0 _ll_register_callbacks cb 0x55b5332ed160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:04.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.257+0000 7fad7f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b53339bc60 0x55b533398d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:04.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.257+0000 7fad7f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b53339bc60 0x55b533398d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42718/0 (socket says 172.21.15.116:42718) 2024-09-12T03:12:04.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.257+0000 7fad7f7fe640 1 -- 172.21.15.116:0/2995327778 learned_addr learned my addr 172.21.15.116:0/2995327778 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:04.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.257+0000 7fad7effd640 1 --2- 172.21.15.116:0/2995327778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b53338c590 0x55b53339b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:04.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.258+0000 7fad7e7fc640 1 --2- 172.21.15.116:0/2995327778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b53338ceb0 0x55b533398850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:04.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.258+0000 7fad7f7fe640 1 -- 172.21.15.116:0/2995327778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b53338c590 msgr2=0x55b53339b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:04.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.258+0000 7fad7f7fe640 1 --2- 172.21.15.116:0/2995327778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b53338c590 0x55b53339b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.258+0000 7fad7f7fe640 1 -- 172.21.15.116:0/2995327778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b53338ceb0 msgr2=0x55b533398850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:04.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.258+0000 7fad7f7fe640 1 --2- 172.21.15.116:0/2995327778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b53338ceb0 0x55b533398850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.258+0000 7fad7f7fe640 1 -- 172.21.15.116:0/2995327778 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5330633c0 con 0x55b53339bc60 2024-09-12T03:12:04.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.258+0000 7fad7f7fe640 1 --2- 172.21.15.116:0/2995327778 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b53339bc60 0x55b533398d90 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fad78011ac0 tx=0x7fad78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71dd519caabe69c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:04.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.258+0000 7fad5effd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:12:04.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.258+0000 7fad5effd640 1 -- 172.21.15.116:0/2995327778 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad78019050 con 0x55b53339bc60 2024-09-12T03:12:04.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.258+0000 7fad87ddc740 1 -- 172.21.15.116:0/2995327778 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad7000ed20 con 0x55b53339bc60 2024-09-12T03:12:04.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.258+0000 7fad5effd640 1 -- 172.21.15.116:0/2995327778 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fad78010da0 con 0x55b53339bc60 2024-09-12T03:12:04.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.258+0000 7fad5effd640 1 -- 172.21.15.116:0/2995327778 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fad78003970 con 0x55b53339bc60 2024-09-12T03:12:04.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.258+0000 7fad87ddc740 10 client.17796 fetch_fsmap learned FSMap version 25 2024-09-12T03:12:04.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.258+0000 7fad87ddc740 1 -- 172.21.15.116:0/2995327778 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b5333aa250 con 0x55b53339bc60 2024-09-12T03:12:04.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.258+0000 7fad5effd640 1 -- 172.21.15.116:0/2995327778 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad7000ed20 con 0x55b53339bc60 2024-09-12T03:12:04.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.259+0000 7fad5effd640 1 -- 172.21.15.116:0/2995327778 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fad7800f040 con 0x55b53339bc60 2024-09-12T03:12:04.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.259+0000 7fad87ddc740 10 client.17796 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:12:04.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.259+0000 7fad87ddc740 10 client.17796 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:12:04.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.259+0000 7fad87ddc740 20 client.17796 populate_metadata read hostname 'smithi116' 2024-09-12T03:12:04.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.259+0000 7fad87ddc740 1 --2- 172.21.15.116:0/2995327778 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b5333ab710 0x55b5333adb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:04.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.259+0000 7fad87ddc740 4 client.17796 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:12:04.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.259+0000 7fad87ddc740 1 -- 172.21.15.116:0/2995327778 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5332ff850 con 0x55b5333ab710 2024-09-12T03:12:04.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.259+0000 7fad7effd640 1 --2- 172.21.15.116:0/2995327778 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b5333ab710 0x55b5333adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:04.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.259+0000 7fad5effd640 10 client.17796 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:04.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.259+0000 7fad7effd640 1 --2- 172.21.15.116:0/2995327778 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b5333ab710 0x55b5333adb80 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a35d09ebd511e81 server_cookie=b62f1c8ec53e3667 in_seq=0 out_seq=0 2024-09-12T03:12:04.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.260+0000 7fad5effd640 1 -- 172.21.15.116:0/2995327778 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b5332ff850 con 0x55b5333ab710 2024-09-12T03:12:04.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.260+0000 7fad5effd640 10 client.17796 handle_command_reply: tid=0 2024-09-12T03:12:04.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.260+0000 7fad87ddc740 1 client.17796 shutdown 2024-09-12T03:12:04.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.260+0000 7fad87ddc740 1 -- 172.21.15.116:0/2995327778 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b53339bc60 msgr2=0x55b533398d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:04.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.260+0000 7fad87ddc740 1 --2- 172.21.15.116:0/2995327778 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b53339bc60 0x55b533398d90 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fad78011ac0 tx=0x7fad78004a40 comp rx=0 tx=0).stop 2024-09-12T03:12:04.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.260+0000 7fad87ddc740 1 -- 172.21.15.116:0/2995327778 shutdown_connections 2024-09-12T03:12:04.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.260+0000 7fad87ddc740 1 --2- 172.21.15.116:0/2995327778 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b5333ab710 0x55b5333adb80 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.260+0000 7fad87ddc740 1 --2- 172.21.15.116:0/2995327778 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b53339bc60 0x55b533398d90 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.260+0000 7fad87ddc740 1 --2- 172.21.15.116:0/2995327778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b53338ceb0 0x55b533398850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.260+0000 7fad5effd640 0 client.17796 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:04.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.260+0000 7fad87ddc740 1 --2- 172.21.15.116:0/2995327778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b53338c590 0x55b53339b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.260+0000 7fad87ddc740 1 -- 172.21.15.116:0/2995327778 >> 172.21.15.116:0/2995327778 conn(0x55b533388c00 msgr2=0x55b533387d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:04.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.260+0000 7fad87ddc740 1 -- 172.21.15.116:0/2995327778 shutdown_connections 2024-09-12T03:12:04.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.260+0000 7fad87ddc740 1 -- 172.21.15.116:0/2995327778 wait complete. 2024-09-12T03:12:04.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.261+0000 7fad87ddc740 20 client.17796 trim_cache size 0 max 16384 2024-09-12T03:12:04.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.277+0000 7fad87ddc740 1 Processor -- start 2024-09-12T03:12:04.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.277+0000 7fad87ddc740 1 -- start start 2024-09-12T03:12:04.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.277+0000 7fad87ddc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b53338c590 0x55b5333a7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:04.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.277+0000 7fad87ddc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b53338ceb0 0x55b5333b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:04.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.277+0000 7fad87ddc740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b53339bc60 0x55b5333b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:04.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.277+0000 7fad87ddc740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fad801ad9a0 con 0x55b53339bc60 2024-09-12T03:12:04.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.277+0000 7fad87ddc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fad801ab270 con 0x55b53338c590 2024-09-12T03:12:04.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.277+0000 7fad87ddc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fad801a90e0 con 0x55b53338ceb0 2024-09-12T03:12:04.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.278+0000 7fad7e7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b53338ceb0 0x55b5333b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:04.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.278+0000 7fad7e7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b53338ceb0 0x55b5333b34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.116:42700/0 (socket says 172.21.15.116:42700) 2024-09-12T03:12:04.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.278+0000 7fad7e7fc640 1 -- 172.21.15.116:0/947777149 learned_addr learned my addr 172.21.15.116:0/947777149 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:04.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.278+0000 7fad7f7fe640 1 --2- 172.21.15.116:0/947777149 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b53339bc60 0x55b5333b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:04.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.278+0000 7fad7e7fc640 1 -- 172.21.15.116:0/947777149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b53338c590 msgr2=0x55b5333a7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:04.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.278+0000 7fad7e7fc640 1 --2- 172.21.15.116:0/947777149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b53338c590 0x55b5333a7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.278+0000 7fad7e7fc640 1 -- 172.21.15.116:0/947777149 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b53339bc60 msgr2=0x55b5333b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:04.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.278+0000 7fad7e7fc640 1 --2- 172.21.15.116:0/947777149 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b53339bc60 0x55b5333b39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.278+0000 7fad7e7fc640 1 -- 172.21.15.116:0/947777149 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b533396da0 con 0x55b53338ceb0 2024-09-12T03:12:04.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.279+0000 7fad7e7fc640 1 --2- 172.21.15.116:0/947777149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b53338ceb0 0x55b5333b34a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fad680bbd70 tx=0x7fad68005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7934b7a28337010b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:04.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.279+0000 7fad757fa640 1 -- 172.21.15.116:0/947777149 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad68049040 con 0x55b53338ceb0 2024-09-12T03:12:04.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.279+0000 7fad757fa640 1 -- 172.21.15.116:0/947777149 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad680c0190 con 0x55b53338ceb0 2024-09-12T03:12:04.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.279+0000 7fad757fa640 1 -- 172.21.15.116:0/947777149 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad680b8450 con 0x55b53338ceb0 2024-09-12T03:12:04.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.280+0000 7fad87ddc740 1 -- 172.21.15.116:0/947777149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b53338ceb0 msgr2=0x55b5333b34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:04.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.280+0000 7fad87ddc740 1 --2- 172.21.15.116:0/947777149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b53338ceb0 0x55b5333b34a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fad680bbd70 tx=0x7fad68005e50 comp rx=0 tx=0).stop 2024-09-12T03:12:04.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.280+0000 7fad87ddc740 1 -- 172.21.15.116:0/947777149 shutdown_connections 2024-09-12T03:12:04.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.280+0000 7fad87ddc740 1 --2- 172.21.15.116:0/947777149 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b53339bc60 0x55b5333b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.280+0000 7fad87ddc740 1 --2- 172.21.15.116:0/947777149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b53338ceb0 0x55b5333b34a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.280+0000 7fad87ddc740 1 --2- 172.21.15.116:0/947777149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b53338c590 0x55b5333a7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.280+0000 7fad87ddc740 1 -- 172.21.15.116:0/947777149 >> 172.21.15.116:0/947777149 conn(0x55b533388c00 msgr2=0x55b533397120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:04.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.280+0000 7fad87ddc740 1 -- 172.21.15.116:0/947777149 shutdown_connections 2024-09-12T03:12:04.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.280+0000 7fad87ddc740 1 -- 172.21.15.116:0/947777149 wait complete. 2024-09-12T03:12:04.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.280+0000 7fad87ddc740 1 -- start start 2024-09-12T03:12:04.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.280+0000 7fad87ddc740 1 Processor -- start 2024-09-12T03:12:04.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.280+0000 7fad87ddc740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b53338c590 0x55b53339fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:04.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.280+0000 7fad87ddc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b53338ceb0 0x55b5333a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:04.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.280+0000 7fad87ddc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b53339bc60 0x55b5333a0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:04.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.280+0000 7fad87ddc740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fad801a9730 con 0x55b53338c590 2024-09-12T03:12:04.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.280+0000 7fad87ddc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fad801aaba0 con 0x55b53339bc60 2024-09-12T03:12:04.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.280+0000 7fad87ddc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fad801a9e00 con 0x55b53338ceb0 2024-09-12T03:12:04.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.281+0000 7fad87ddc740 10 client.0 _ll_register_callbacks cb 0x7fad7800ef00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:04.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.281+0000 7fad7effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b53338c590 0x55b53339fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:04.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.281+0000 7fad7effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b53338c590 0x55b53339fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42744/0 (socket says 172.21.15.116:42744) 2024-09-12T03:12:04.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.281+0000 7fad7effd640 1 -- 172.21.15.116:0/3806240936 learned_addr learned my addr 172.21.15.116:0/3806240936 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:04.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.281+0000 7fad7f7fe640 1 --2- 172.21.15.116:0/3806240936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b53339bc60 0x55b5333a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:04.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.281+0000 7fad7e7fc640 1 --2- 172.21.15.116:0/3806240936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b53338ceb0 0x55b5333a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:04.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.281+0000 7fad7effd640 1 -- 172.21.15.116:0/3806240936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b53338ceb0 msgr2=0x55b5333a00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:04.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.281+0000 7fad7effd640 1 --2- 172.21.15.116:0/3806240936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b53338ceb0 0x55b5333a00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.281+0000 7fad7effd640 1 -- 172.21.15.116:0/3806240936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b53339bc60 msgr2=0x55b5333a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:04.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.281+0000 7fad7effd640 1 --2- 172.21.15.116:0/3806240936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b53339bc60 0x55b5333a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.281+0000 7fad7effd640 1 -- 172.21.15.116:0/3806240936 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5334156c0 con 0x55b53338c590 2024-09-12T03:12:04.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.281+0000 7fad7f7fe640 1 --2- 172.21.15.116:0/3806240936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b53339bc60 0x55b5333a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:12:04.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.281+0000 7fad7effd640 1 --2- 172.21.15.116:0/3806240936 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b53338c590 0x55b53339fba0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fad7000f010 tx=0x7fad700058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64bb8d77979cba9d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:04.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.281+0000 7fad5effd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:12:04.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.281+0000 7fad5effd640 1 -- 172.21.15.116:0/3806240936 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad70016040 con 0x55b53338c590 2024-09-12T03:12:04.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.281+0000 7fad5effd640 1 -- 172.21.15.116:0/3806240936 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fad70004640 con 0x55b53338c590 2024-09-12T03:12:04.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.281+0000 7fad87ddc740 1 -- 172.21.15.116:0/3806240936 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad680c0190 con 0x55b53338c590 2024-09-12T03:12:04.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.281+0000 7fad5effd640 1 -- 172.21.15.116:0/3806240936 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fad70039050 con 0x55b53338c590 2024-09-12T03:12:04.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.281+0000 7fad87ddc740 10 client.17802 fetch_fsmap learned FSMap version 25 2024-09-12T03:12:04.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.281+0000 7fad87ddc740 1 -- 172.21.15.116:0/3806240936 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b533416990 con 0x55b53338c590 2024-09-12T03:12:04.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.282+0000 7fad5effd640 1 -- 172.21.15.116:0/3806240936 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad680c0190 con 0x55b53338c590 2024-09-12T03:12:04.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.282+0000 7fad5effd640 1 -- 172.21.15.116:0/3806240936 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fad70030070 con 0x55b53338c590 2024-09-12T03:12:04.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.282+0000 7fad87ddc740 10 client.17802 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:12:04.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.282+0000 7fad87ddc740 10 client.17802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:12:04.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.282+0000 7fad87ddc740 20 client.17802 populate_metadata read hostname 'smithi116' 2024-09-12T03:12:04.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.282+0000 7fad87ddc740 1 --2- 172.21.15.116:0/3806240936 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b533416de0 0x55b5334191d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:04.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.282+0000 7fad87ddc740 4 client.17802 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:12:04.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.282+0000 7fad87ddc740 1 -- 172.21.15.116:0/3806240936 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b533399120 con 0x55b533416de0 2024-09-12T03:12:04.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.282+0000 7fad7e7fc640 1 --2- 172.21.15.116:0/3806240936 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b533416de0 0x55b5334191d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:04.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.282+0000 7fad5effd640 10 client.17802 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:04.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.282+0000 7fad7e7fc640 1 --2- 172.21.15.116:0/3806240936 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b533416de0 0x55b5334191d0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb9afd9cea986a31 server_cookie=abdfbf642da03c65 in_seq=0 out_seq=0 2024-09-12T03:12:04.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.283+0000 7fad5effd640 1 -- 172.21.15.116:0/3806240936 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b533399120 con 0x55b533416de0 2024-09-12T03:12:04.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.283+0000 7fad5effd640 10 client.17802 handle_command_reply: tid=0 2024-09-12T03:12:04.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.283+0000 7fad87ddc740 1 client.17802 shutdown 2024-09-12T03:12:04.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.283+0000 7fad87ddc740 1 -- 172.21.15.116:0/3806240936 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b53338c590 msgr2=0x55b53339fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:04.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.283+0000 7fad87ddc740 1 --2- 172.21.15.116:0/3806240936 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b53338c590 0x55b53339fba0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fad7000f010 tx=0x7fad700058e0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.283+0000 7fad87ddc740 1 -- 172.21.15.116:0/3806240936 shutdown_connections 2024-09-12T03:12:04.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.283+0000 7fad87ddc740 1 --2- 172.21.15.116:0/3806240936 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b533416de0 0x55b5334191d0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.283+0000 7fad87ddc740 1 --2- 172.21.15.116:0/3806240936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b53339bc60 0x55b5333a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.283+0000 7fad87ddc740 1 --2- 172.21.15.116:0/3806240936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b53338ceb0 0x55b5333a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.283+0000 7fad87ddc740 1 --2- 172.21.15.116:0/3806240936 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b53338c590 0x55b53339fba0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.283+0000 7fad87ddc740 1 -- 172.21.15.116:0/3806240936 >> 172.21.15.116:0/3806240936 conn(0x55b533388c00 msgr2=0x55b53339a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:04.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.283+0000 7fad5effd640 0 client.17802 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:04.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.283+0000 7fad87ddc740 1 -- 172.21.15.116:0/3806240936 shutdown_connections 2024-09-12T03:12:04.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.283+0000 7fad87ddc740 1 -- 172.21.15.116:0/3806240936 wait complete. 2024-09-12T03:12:04.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.283+0000 7fad87ddc740 20 client.17802 trim_cache size 0 max 16384 2024-09-12T03:12:04.299 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:12:04.299 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:12:04.299 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:12:04.299 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:12:04.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.284+0000 7fad85a03640 1 -- 172.21.15.116:0/2656818151 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fad6c0a2c00 msgr2=0x7fad6c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:04.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.284+0000 7fad85a03640 1 --2- 172.21.15.116:0/2656818151 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fad6c0a2c00 0x7fad6c0a50c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fad78005fd0 tx=0x7fad78007640 comp rx=0 tx=0).stop 2024-09-12T03:12:04.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.284+0000 7fad85a03640 1 -- 172.21.15.116:0/2656818151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad801012a0 msgr2=0x7fad801996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:04.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.284+0000 7fad85a03640 1 --2- 172.21.15.116:0/2656818151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad801012a0 0x7fad801996d0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fad68009230 tx=0x7fad68004270 comp rx=0 tx=0).stop 2024-09-12T03:12:04.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.284+0000 7fad85a03640 1 -- 172.21.15.116:0/2656818151 shutdown_connections 2024-09-12T03:12:04.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.284+0000 7fad85a03640 1 --2- 172.21.15.116:0/2656818151 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fad6c0a2c00 0x7fad6c0a50c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.284+0000 7fad85a03640 1 --2- 172.21.15.116:0/2656818151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad80106900 0x7fad801a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.284+0000 7fad85a03640 1 --2- 172.21.15.116:0/2656818151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad801012a0 0x7fad801996d0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.284+0000 7fad85a03640 1 --2- 172.21.15.116:0/2656818151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad80100900 0x7fad80199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:04.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.284+0000 7fad85a03640 1 -- 172.21.15.116:0/2656818151 >> 172.21.15.116:0/2656818151 conn(0x7fad800fc600 msgr2=0x7fad800fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:04.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.284+0000 7fad85a03640 1 -- 172.21.15.116:0/2656818151 shutdown_connections 2024-09-12T03:12:04.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:04.284+0000 7fad85a03640 1 -- 172.21.15.116:0/2656818151 wait complete. 2024-09-12T03:12:04.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:04 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:12:04.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:04 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:12:04.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:04 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:12:04.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:04 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:12:04.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:04 smithi116 ceph-mon[31410]: pgmap v610: 97 pgs: 97 active+clean; 4.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:12:04.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:04 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:12:04.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:04 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:12:04.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:04 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:12:04.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:04 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:12:04.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:04 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:12:04.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:04 smithi183 ceph-mon[35535]: pgmap v610: 97 pgs: 97 active+clean; 4.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:12:04.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:04 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:12:04.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:04 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:12:04.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:04 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:12:04.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:04 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:12:04.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:04 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:12:04.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:04 smithi177 ceph-mon[35305]: pgmap v610: 97 pgs: 97 active+clean; 4.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:12:04.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:04 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:12:05.080 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:12:05.080 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:12:05.327 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:12:05.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:05 smithi183 ceph-mon[35535]: pgmap v611: 97 pgs: 97 active+clean; 4.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:12:05.737 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:05 smithi116 ceph-mon[31410]: pgmap v611: 97 pgs: 97 active+clean; 4.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:12:05.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:05 smithi177 ceph-mon[35305]: pgmap v611: 97 pgs: 97 active+clean; 4.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:12:05.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.815+0000 7f8f695f6640 1 -- 172.21.15.116:0/1635087769 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f64102ef0 msgr2=0x7f8f6410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:05.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.815+0000 7f8f695f6640 1 --2- 172.21.15.116:0/1635087769 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f64102ef0 0x7f8f6410da40 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009900 tx=0x7f8f5c02a480 comp rx=0 tx=0).stop 2024-09-12T03:12:05.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.815+0000 7f8f695f6640 1 -- 172.21.15.116:0/1635087769 shutdown_connections 2024-09-12T03:12:05.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.815+0000 7f8f695f6640 1 --2- 172.21.15.116:0/1635087769 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f64102ef0 0x7f8f6410da40 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:05.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.815+0000 7f8f695f6640 1 --2- 172.21.15.116:0/1635087769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f64102570 0x7f8f641029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:05.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.815+0000 7f8f695f6640 1 --2- 172.21.15.116:0/1635087769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f64100e20 0x7f8f64101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:05.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.815+0000 7f8f695f6640 1 -- 172.21.15.116:0/1635087769 >> 172.21.15.116:0/1635087769 conn(0x7f8f640fc7d0 msgr2=0x7f8f640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:05.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.815+0000 7f8f695f6640 1 -- 172.21.15.116:0/1635087769 shutdown_connections 2024-09-12T03:12:05.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.815+0000 7f8f695f6640 1 -- 172.21.15.116:0/1635087769 wait complete. 2024-09-12T03:12:05.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.816+0000 7f8f695f6640 1 Processor -- start 2024-09-12T03:12:05.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.816+0000 7f8f695f6640 1 -- start start 2024-09-12T03:12:05.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.816+0000 7f8f695f6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f64100e20 0x7f8f6410e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:05.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.816+0000 7f8f695f6640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f64102570 0x7f8f6410ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:05.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.816+0000 7f8f695f6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f64102ef0 0x7f8f6410ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:05.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.817+0000 7f8f695f6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8f641120a0 con 0x7f8f64102ef0 2024-09-12T03:12:05.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.817+0000 7f8f695f6640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8f64111f20 con 0x7f8f64102570 2024-09-12T03:12:05.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.817+0000 7f8f695f6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8f64112220 con 0x7f8f64100e20 2024-09-12T03:12:05.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.817+0000 7f8f637fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f64102ef0 0x7f8f6410ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:05.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.817+0000 7f8f637fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f64102ef0 0x7f8f6410ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42760/0 (socket says 172.21.15.116:42760) 2024-09-12T03:12:05.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.817+0000 7f8f637fe640 1 -- 172.21.15.116:0/2560699 learned_addr learned my addr 172.21.15.116:0/2560699 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:05.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.817+0000 7f8f627fc640 1 --2- 172.21.15.116:0/2560699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f64102570 0x7f8f6410ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:05.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.817+0000 7f8f62ffd640 1 --2- 172.21.15.116:0/2560699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f64100e20 0x7f8f6410e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:05.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.817+0000 7f8f637fe640 1 -- 172.21.15.116:0/2560699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f64100e20 msgr2=0x7f8f6410e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:05.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.817+0000 7f8f637fe640 1 --2- 172.21.15.116:0/2560699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f64100e20 0x7f8f6410e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:05.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.817+0000 7f8f637fe640 1 -- 172.21.15.116:0/2560699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f64102570 msgr2=0x7f8f6410ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:05.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.817+0000 7f8f637fe640 1 --2- 172.21.15.116:0/2560699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f64102570 0x7f8f6410ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:05.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.817+0000 7f8f637fe640 1 -- 172.21.15.116:0/2560699 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f641a6920 con 0x7f8f64102ef0 2024-09-12T03:12:05.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.817+0000 7f8f637fe640 1 --2- 172.21.15.116:0/2560699 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f64102ef0 0x7f8f6410ef50 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c005ab0 tx=0x7f8f5c003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:05.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.817+0000 7f8f4bfff640 1 -- 172.21.15.116:0/2560699 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c03f070 con 0x7f8f64102ef0 2024-09-12T03:12:05.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.817+0000 7f8f4bfff640 1 -- 172.21.15.116:0/2560699 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f5c031e30 con 0x7f8f64102ef0 2024-09-12T03:12:05.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.817+0000 7f8f695f6640 1 -- 172.21.15.116:0/2560699 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f641a6bb0 con 0x7f8f64102ef0 2024-09-12T03:12:05.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.817+0000 7f8f4bfff640 1 -- 172.21.15.116:0/2560699 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c035030 con 0x7f8f64102ef0 2024-09-12T03:12:05.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.817+0000 7f8f695f6640 1 -- 172.21.15.116:0/2560699 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f641a7090 con 0x7f8f64102ef0 2024-09-12T03:12:05.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.818+0000 7f8f695f6640 1 -- 172.21.15.116:0/2560699 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f64101200 con 0x7f8f64102ef0 2024-09-12T03:12:05.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.820+0000 7f8f4bfff640 1 -- 172.21.15.116:0/2560699 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8f5c042070 con 0x7f8f64102ef0 2024-09-12T03:12:05.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.820+0000 7f8f4bfff640 1 --2- 172.21.15.116:0/2560699 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8f4c0a2b80 0x7f8f4c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:05.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.820+0000 7f8f62ffd640 1 --2- 172.21.15.116:0/2560699 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8f4c0a2b80 0x7f8f4c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:05.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.820+0000 7f8f4bfff640 1 -- 172.21.15.116:0/2560699 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f8f5c0efff0 con 0x7f8f64102ef0 2024-09-12T03:12:05.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.821+0000 7f8f62ffd640 1 --2- 172.21.15.116:0/2560699 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8f4c0a2b80 0x7f8f4c0a5040 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8f54005c40 tx=0x7f8f54008230 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:05.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:05.824+0000 7f8f4bfff640 1 -- 172.21.15.116:0/2560699 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f5c0bcf80 con 0x7f8f64102ef0 2024-09-12T03:12:06.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:06.024+0000 7f8f695f6640 1 -- 172.21.15.116:0/2560699 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f641a72c0 con 0x7f8f64102ef0 2024-09-12T03:12:06.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:06.024+0000 7f8f4bfff640 1 -- 172.21.15.116:0/2560699 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f8f5c003e20 con 0x7f8f64102ef0 2024-09-12T03:12:06.025 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:12:06.025 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:12:06.025 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:12:06.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:06.026+0000 7f8f695f6640 1 -- 172.21.15.116:0/2560699 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8f4c0a2b80 msgr2=0x7f8f4c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:06.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:06.026+0000 7f8f695f6640 1 --2- 172.21.15.116:0/2560699 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8f4c0a2b80 0x7f8f4c0a5040 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8f54005c40 tx=0x7f8f54008230 comp rx=0 tx=0).stop 2024-09-12T03:12:06.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:06.026+0000 7f8f695f6640 1 -- 172.21.15.116:0/2560699 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f64102ef0 msgr2=0x7f8f6410ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:06.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:06.026+0000 7f8f695f6640 1 --2- 172.21.15.116:0/2560699 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f64102ef0 0x7f8f6410ef50 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c005ab0 tx=0x7f8f5c003c30 comp rx=0 tx=0).stop 2024-09-12T03:12:06.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:06.026+0000 7f8f695f6640 1 -- 172.21.15.116:0/2560699 shutdown_connections 2024-09-12T03:12:06.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:06.026+0000 7f8f695f6640 1 --2- 172.21.15.116:0/2560699 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8f4c0a2b80 0x7f8f4c0a5040 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:06.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:06.026+0000 7f8f695f6640 1 --2- 172.21.15.116:0/2560699 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f64102ef0 0x7f8f6410ef50 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:06.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:06.026+0000 7f8f695f6640 1 --2- 172.21.15.116:0/2560699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f64102570 0x7f8f6410ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:06.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:06.026+0000 7f8f695f6640 1 --2- 172.21.15.116:0/2560699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f64100e20 0x7f8f6410e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:06.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:06.027+0000 7f8f695f6640 1 -- 172.21.15.116:0/2560699 >> 172.21.15.116:0/2560699 conn(0x7f8f640fc7d0 msgr2=0x7f8f64106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:06.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:06.027+0000 7f8f695f6640 1 -- 172.21.15.116:0/2560699 shutdown_connections 2024-09-12T03:12:06.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:06.027+0000 7f8f695f6640 1 -- 172.21.15.116:0/2560699 wait complete. 2024-09-12T03:12:06.700 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:06 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2560699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:12:06.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:06 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2560699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:12:06.730 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:12:06.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:06 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2560699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:12:06.976 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:12:07.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.502+0000 7f8912f38640 1 -- 172.21.15.116:0/2722561332 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f890c105590 msgr2=0x7f890c10d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:07.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.502+0000 7f8912f38640 1 --2- 172.21.15.116:0/2722561332 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f890c105590 0x7f890c10d110 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f8904009ca0 tx=0x7f890402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.502+0000 7f8912f38640 1 -- 172.21.15.116:0/2722561332 shutdown_connections 2024-09-12T03:12:07.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.502+0000 7f8912f38640 1 --2- 172.21.15.116:0/2722561332 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f890c105590 0x7f890c10d110 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.502+0000 7f8912f38640 1 --2- 172.21.15.116:0/2722561332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f890c100e10 0x7f890c104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.502+0000 7f8912f38640 1 --2- 172.21.15.116:0/2722561332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f890c100460 0x7f890c100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.502+0000 7f8912f38640 1 -- 172.21.15.116:0/2722561332 >> 172.21.15.116:0/2722561332 conn(0x7f890c0fbee0 msgr2=0x7f890c0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:07.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.502+0000 7f8912f38640 1 -- 172.21.15.116:0/2722561332 shutdown_connections 2024-09-12T03:12:07.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.502+0000 7f8912f38640 1 -- 172.21.15.116:0/2722561332 wait complete. 2024-09-12T03:12:07.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.503+0000 7f8912f38640 1 Processor -- start 2024-09-12T03:12:07.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.503+0000 7f8912f38640 1 -- start start 2024-09-12T03:12:07.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.503+0000 7f8912f38640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f890c100460 0x7f890c1972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.503+0000 7f8912f38640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f890c100e10 0x7f890c197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.503+0000 7f8912f38640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f890c105590 0x7f890c19bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.503+0000 7f8912f38640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f890c110e30 con 0x7f890c105590 2024-09-12T03:12:07.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.503+0000 7f8912f38640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f890c110cb0 con 0x7f890c100460 2024-09-12T03:12:07.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.503+0000 7f8912f38640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f890c110fb0 con 0x7f890c100e10 2024-09-12T03:12:07.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.503+0000 7f89114d5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f890c105590 0x7f890c19bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.503+0000 7f89114d5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f890c105590 0x7f890c19bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42780/0 (socket says 172.21.15.116:42780) 2024-09-12T03:12:07.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.503+0000 7f89114d5640 1 -- 172.21.15.116:0/1448254500 learned_addr learned my addr 172.21.15.116:0/1448254500 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:07.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.503+0000 7f8903fff640 1 --2- 172.21.15.116:0/1448254500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f890c100e10 0x7f890c197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.503+0000 7f8910cd4640 1 --2- 172.21.15.116:0/1448254500 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f890c100460 0x7f890c1972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.504+0000 7f89114d5640 1 -- 172.21.15.116:0/1448254500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f890c100e10 msgr2=0x7f890c197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:07.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.504+0000 7f89114d5640 1 --2- 172.21.15.116:0/1448254500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f890c100e10 0x7f890c197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.504+0000 7f89114d5640 1 -- 172.21.15.116:0/1448254500 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f890c100460 msgr2=0x7f890c1972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:07.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.504+0000 7f89114d5640 1 --2- 172.21.15.116:0/1448254500 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f890c100460 0x7f890c1972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.504+0000 7f89114d5640 1 -- 172.21.15.116:0/1448254500 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890c19c280 con 0x7f890c105590 2024-09-12T03:12:07.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.504+0000 7f89114d5640 1 --2- 172.21.15.116:0/1448254500 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f890c105590 0x7f890c19bba0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f8904031a20 tx=0x7f8904031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:07.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.504+0000 7f8901ffb640 1 -- 172.21.15.116:0/1448254500 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904003c30 con 0x7f890c105590 2024-09-12T03:12:07.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.504+0000 7f8901ffb640 1 -- 172.21.15.116:0/1448254500 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8904003dd0 con 0x7f890c105590 2024-09-12T03:12:07.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.504+0000 7f8912f38640 1 -- 172.21.15.116:0/1448254500 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f890c19c510 con 0x7f890c105590 2024-09-12T03:12:07.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.504+0000 7f8901ffb640 1 -- 172.21.15.116:0/1448254500 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890402fc40 con 0x7f890c105590 2024-09-12T03:12:07.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.504+0000 7f8912f38640 1 -- 172.21.15.116:0/1448254500 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f890c1a3e50 con 0x7f890c105590 2024-09-12T03:12:07.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.507+0000 7f8901ffb640 1 -- 172.21.15.116:0/1448254500 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8904004050 con 0x7f890c105590 2024-09-12T03:12:07.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.507+0000 7f8915311740 1 Processor -- start 2024-09-12T03:12:07.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.507+0000 7f8915311740 1 -- start start 2024-09-12T03:12:07.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.507+0000 7f8915311740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566d449b040 0x5566d449b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.507+0000 7f8915311740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566d449bdc0 0x5566d449c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.507+0000 7f8915311740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566d449c760 0x5566d44a2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.507+0000 7f8915311740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5566d43f72f0 con 0x5566d449b040 2024-09-12T03:12:07.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.507+0000 7f8915311740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5566d439ecf0 con 0x5566d449bdc0 2024-09-12T03:12:07.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.507+0000 7f8915311740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5566d43f32b0 con 0x5566d449c760 2024-09-12T03:12:07.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.508+0000 7f8903fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566d449b040 0x5566d449b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.508+0000 7f8903fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566d449b040 0x5566d449b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42786/0 (socket says 172.21.15.116:42786) 2024-09-12T03:12:07.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.508+0000 7f8903fff640 1 -- 172.21.15.116:0/4260938304 learned_addr learned my addr 172.21.15.116:0/4260938304 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:07.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.508+0000 7f8910cd4640 1 --2- 172.21.15.116:0/4260938304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566d449bdc0 0x5566d449c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.508+0000 7f8903fff640 1 -- 172.21.15.116:0/4260938304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566d449c760 msgr2=0x5566d44a2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:07.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.508+0000 7f8903fff640 1 --2- 172.21.15.116:0/4260938304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566d449c760 0x5566d44a2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.508+0000 7f8903fff640 1 -- 172.21.15.116:0/4260938304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566d449bdc0 msgr2=0x5566d449c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:07.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.508+0000 7f8903fff640 1 --2- 172.21.15.116:0/4260938304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566d449bdc0 0x5566d449c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.508+0000 7f8903fff640 1 -- 172.21.15.116:0/4260938304 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566d43833a0 con 0x5566d449b040 2024-09-12T03:12:07.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.508+0000 7f8901ffb640 1 --2- 172.21.15.116:0/1448254500 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f88f40a2c00 0x7f88f40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.508+0000 7f8903fff640 1 --2- 172.21.15.116:0/4260938304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566d449b040 0x5566d449b420 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f88fc00bd00 tx=0x7f88fc0127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107d3d51da950245 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:07.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.508+0000 7f8901ffb640 1 -- 172.21.15.116:0/1448254500 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f89040f01d0 con 0x7f890c105590 2024-09-12T03:12:07.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.508+0000 7f8903fff640 1 --2- 172.21.15.116:0/1448254500 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f88f40a2c00 0x7f88f40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.508+0000 7f88f2ffd640 1 -- 172.21.15.116:0/4260938304 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc01b470 con 0x5566d449b040 2024-09-12T03:12:07.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.508+0000 7f88f2ffd640 1 -- 172.21.15.116:0/4260938304 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88fc016070 con 0x5566d449b040 2024-09-12T03:12:07.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.509+0000 7f88f2ffd640 1 -- 172.21.15.116:0/4260938304 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc00f3c0 con 0x5566d449b040 2024-09-12T03:12:07.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.509+0000 7f8915311740 1 -- 172.21.15.116:0/4260938304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566d449b040 msgr2=0x5566d449b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:07.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.509+0000 7f8903fff640 1 --2- 172.21.15.116:0/1448254500 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f88f40a2c00 0x7f88f40a50c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f88fc00c010 tx=0x7f88fc00fa90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:07.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.509+0000 7f8915311740 1 --2- 172.21.15.116:0/4260938304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566d449b040 0x5566d449b420 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f88fc00bd00 tx=0x7f88fc0127c0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.509+0000 7f8915311740 1 -- 172.21.15.116:0/4260938304 shutdown_connections 2024-09-12T03:12:07.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.509+0000 7f8915311740 1 --2- 172.21.15.116:0/4260938304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566d449c760 0x5566d44a2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.509+0000 7f8915311740 1 --2- 172.21.15.116:0/4260938304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566d449bdc0 0x5566d449c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.509+0000 7f8915311740 1 --2- 172.21.15.116:0/4260938304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566d449b040 0x5566d449b420 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.509+0000 7f8915311740 1 -- 172.21.15.116:0/4260938304 >> 172.21.15.116:0/4260938304 conn(0x5566d445a310 msgr2=0x5566d448d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:07.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.509+0000 7f8915311740 1 -- 172.21.15.116:0/4260938304 shutdown_connections 2024-09-12T03:12:07.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.509+0000 7f8915311740 1 -- 172.21.15.116:0/4260938304 wait complete. 2024-09-12T03:12:07.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.510+0000 7f8915311740 1 -- start start 2024-09-12T03:12:07.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.510+0000 7f8915311740 1 Processor -- start 2024-09-12T03:12:07.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.510+0000 7f8915311740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566d449b040 0x5566d449a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.510+0000 7f8915311740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566d449bdc0 0x5566d4497860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.510+0000 7f8915311740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566d449c760 0x5566d4497da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.510+0000 7f8915311740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5566d44a34c0 con 0x5566d449b040 2024-09-12T03:12:07.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.510+0000 7f8915311740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5566d43b2060 con 0x5566d449bdc0 2024-09-12T03:12:07.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.510+0000 7f8915311740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5566d44a3640 con 0x5566d449c760 2024-09-12T03:12:07.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.510+0000 7f8915311740 10 client.0 _ll_register_callbacks cb 0x5566d41058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:07.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.510+0000 7f8910cd4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566d449b040 0x5566d449a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.510+0000 7f8910cd4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566d449b040 0x5566d449a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42798/0 (socket says 172.21.15.116:42798) 2024-09-12T03:12:07.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.510+0000 7f8910cd4640 1 -- 172.21.15.116:0/4203470074 learned_addr learned my addr 172.21.15.116:0/4203470074 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:07.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.510+0000 7f8903fff640 1 --2- 172.21.15.116:0/4203470074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566d449bdc0 0x5566d4497860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.510+0000 7f89114d5640 1 --2- 172.21.15.116:0/4203470074 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566d449c760 0x5566d4497da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.511+0000 7f8910cd4640 1 -- 172.21.15.116:0/4203470074 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566d449c760 msgr2=0x5566d4497da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:07.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.511+0000 7f8910cd4640 1 --2- 172.21.15.116:0/4203470074 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566d449c760 0x5566d4497da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.511+0000 7f8910cd4640 1 -- 172.21.15.116:0/4203470074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566d449bdc0 msgr2=0x5566d4497860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:07.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.511+0000 7f8910cd4640 1 --2- 172.21.15.116:0/4203470074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566d449bdc0 0x5566d4497860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.511+0000 7f8910cd4640 1 -- 172.21.15.116:0/4203470074 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566d41683c0 con 0x5566d449b040 2024-09-12T03:12:07.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.511+0000 7f8910cd4640 1 --2- 172.21.15.116:0/4203470074 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566d449b040 0x5566d449a5b0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f89080126b0 tx=0x7f8908012b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167b871d35d97fbe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:07.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.511+0000 7f88ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:12:07.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.511+0000 7f88ebfff640 1 -- 172.21.15.116:0/4203470074 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8908004260 con 0x5566d449b040 2024-09-12T03:12:07.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.511+0000 7f88ebfff640 1 -- 172.21.15.116:0/4203470074 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f8908004400 con 0x5566d449b040 2024-09-12T03:12:07.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.511+0000 7f8915311740 1 -- 172.21.15.116:0/4203470074 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88fc016070 con 0x5566d449b040 2024-09-12T03:12:07.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.511+0000 7f88ebfff640 1 -- 172.21.15.116:0/4203470074 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8908023070 con 0x5566d449b040 2024-09-12T03:12:07.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.511+0000 7f8915311740 10 client.17820 fetch_fsmap learned FSMap version 25 2024-09-12T03:12:07.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.511+0000 7f8915311740 1 -- 172.21.15.116:0/4203470074 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5566d44aee50 con 0x5566d449b040 2024-09-12T03:12:07.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.511+0000 7f88ebfff640 1 -- 172.21.15.116:0/4203470074 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88fc016070 con 0x5566d449b040 2024-09-12T03:12:07.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.511+0000 7f88ebfff640 1 -- 172.21.15.116:0/4203470074 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f890800f440 con 0x5566d449b040 2024-09-12T03:12:07.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.512+0000 7f8915311740 10 client.17820 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:12:07.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.512+0000 7f8915311740 10 client.17820 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:12:07.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.512+0000 7f8915311740 20 client.17820 populate_metadata read hostname 'smithi116' 2024-09-12T03:12:07.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.512+0000 7f8915311740 1 --2- 172.21.15.116:0/4203470074 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5566d44b0310 0x5566d44b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.512+0000 7f8915311740 4 client.17820 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:12:07.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.512+0000 7f8915311740 1 -- 172.21.15.116:0/4203470074 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566d4404860 con 0x5566d44b0310 2024-09-12T03:12:07.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.512+0000 7f8903fff640 1 --2- 172.21.15.116:0/4203470074 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5566d44b0310 0x5566d44b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.512+0000 7f88ebfff640 10 client.17820 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:12:07.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.512+0000 7f8903fff640 1 --2- 172.21.15.116:0/4203470074 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5566d44b0310 0x5566d44b27a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=419b70e981b769e4 server_cookie=3ae0ef689f88af0c in_seq=0 out_seq=0 2024-09-12T03:12:07.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.513+0000 7f88ebfff640 1 -- 172.21.15.116:0/4203470074 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5566d4404860 con 0x5566d44b0310 2024-09-12T03:12:07.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.513+0000 7f88ebfff640 10 client.17820 handle_command_reply: tid=0 2024-09-12T03:12:07.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.513+0000 7f8915311740 1 client.17820 shutdown 2024-09-12T03:12:07.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.513+0000 7f8915311740 1 -- 172.21.15.116:0/4203470074 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566d449b040 msgr2=0x5566d449a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:07.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.513+0000 7f8915311740 1 --2- 172.21.15.116:0/4203470074 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566d449b040 0x5566d449a5b0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f89080126b0 tx=0x7f8908012b70 comp rx=0 tx=0).stop 2024-09-12T03:12:07.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.513+0000 7f8915311740 1 -- 172.21.15.116:0/4203470074 shutdown_connections 2024-09-12T03:12:07.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.513+0000 7f8915311740 1 --2- 172.21.15.116:0/4203470074 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566d449c760 0x5566d4497da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.513+0000 7f8915311740 1 --2- 172.21.15.116:0/4203470074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566d449bdc0 0x5566d4497860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.513+0000 7f8915311740 1 --2- 172.21.15.116:0/4203470074 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5566d44b0310 0x5566d44b27a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.513+0000 7f8915311740 1 --2- 172.21.15.116:0/4203470074 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566d449b040 0x5566d449a5b0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.513+0000 7f88ebfff640 0 client.17820 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:12:07.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.513+0000 7f8915311740 1 -- 172.21.15.116:0/4203470074 >> 172.21.15.116:0/4203470074 conn(0x5566d445a310 msgr2=0x5566d448cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:07.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.514+0000 7f8915311740 1 -- 172.21.15.116:0/4203470074 shutdown_connections 2024-09-12T03:12:07.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.514+0000 7f8915311740 1 -- 172.21.15.116:0/4203470074 wait complete. 2024-09-12T03:12:07.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.514+0000 7f8915311740 20 client.17820 trim_cache size 0 max 16384 2024-09-12T03:12:07.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.526+0000 7f8915311740 1 Processor -- start 2024-09-12T03:12:07.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.526+0000 7f8915311740 1 -- start start 2024-09-12T03:12:07.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.526+0000 7f8915311740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566d449b040 0x5566d4497160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.526+0000 7f8915311740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566d449bdc0 0x5566d4494c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.526+0000 7f8915311740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566d449c760 0x5566d44951c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.526+0000 7f8915311740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1adb00 con 0x5566d449c760 2024-09-12T03:12:07.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.526+0000 7f8915311740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1ab3d0 con 0x5566d449bdc0 2024-09-12T03:12:07.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.526+0000 7f8915311740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1a9240 con 0x5566d449b040 2024-09-12T03:12:07.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.527+0000 7f89114d5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566d449c760 0x5566d44951c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.527+0000 7f89114d5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566d449c760 0x5566d44951c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42810/0 (socket says 172.21.15.116:42810) 2024-09-12T03:12:07.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.527+0000 7f89114d5640 1 -- 172.21.15.116:0/3295713731 learned_addr learned my addr 172.21.15.116:0/3295713731 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:07.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.527+0000 7f8903fff640 1 --2- 172.21.15.116:0/3295713731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566d449bdc0 0x5566d4494c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.527+0000 7f8910cd4640 1 --2- 172.21.15.116:0/3295713731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566d449b040 0x5566d4497160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.527+0000 7f89114d5640 1 -- 172.21.15.116:0/3295713731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566d449b040 msgr2=0x5566d4497160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:07.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.527+0000 7f89114d5640 1 --2- 172.21.15.116:0/3295713731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566d449b040 0x5566d4497160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.527+0000 7f89114d5640 1 -- 172.21.15.116:0/3295713731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566d449bdc0 msgr2=0x5566d4494c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:07.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.527+0000 7f89114d5640 1 --2- 172.21.15.116:0/3295713731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566d449bdc0 0x5566d4494c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.527+0000 7f89114d5640 1 -- 172.21.15.116:0/3295713731 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566d448ef70 con 0x5566d449c760 2024-09-12T03:12:07.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.527+0000 7f8910cd4640 1 --2- 172.21.15.116:0/3295713731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566d449b040 0x5566d4497160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:12:07.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.527+0000 7f89114d5640 1 --2- 172.21.15.116:0/3295713731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566d449c760 0x5566d44951c0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f890402f6c0 tx=0x7f89040095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc536acb5fc673d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:07.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.527+0000 7f88f27fc640 1 -- 172.21.15.116:0/3295713731 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89040da070 con 0x5566d449c760 2024-09-12T03:12:07.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.527+0000 7f88f27fc640 1 -- 172.21.15.116:0/3295713731 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8904036040 con 0x5566d449c760 2024-09-12T03:12:07.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.527+0000 7f88f27fc640 1 -- 172.21.15.116:0/3295713731 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904004730 con 0x5566d449c760 2024-09-12T03:12:07.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.528+0000 7f8915311740 1 -- 172.21.15.116:0/3295713731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566d449c760 msgr2=0x5566d44951c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:07.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.528+0000 7f8915311740 1 --2- 172.21.15.116:0/3295713731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566d449c760 0x5566d44951c0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f890402f6c0 tx=0x7f89040095e0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.528+0000 7f8915311740 1 -- 172.21.15.116:0/3295713731 shutdown_connections 2024-09-12T03:12:07.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.528+0000 7f8915311740 1 --2- 172.21.15.116:0/3295713731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566d449c760 0x5566d44951c0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.528+0000 7f8915311740 1 --2- 172.21.15.116:0/3295713731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566d449bdc0 0x5566d4494c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.528+0000 7f8915311740 1 --2- 172.21.15.116:0/3295713731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566d449b040 0x5566d4497160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.528+0000 7f8915311740 1 -- 172.21.15.116:0/3295713731 >> 172.21.15.116:0/3295713731 conn(0x5566d445a310 msgr2=0x5566d44998e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:07.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.528+0000 7f8915311740 1 -- 172.21.15.116:0/3295713731 shutdown_connections 2024-09-12T03:12:07.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.528+0000 7f8915311740 1 -- 172.21.15.116:0/3295713731 wait complete. 2024-09-12T03:12:07.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.528+0000 7f8915311740 1 -- start start 2024-09-12T03:12:07.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.528+0000 7f8915311740 1 Processor -- start 2024-09-12T03:12:07.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.529+0000 7f8915311740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566d449b040 0x5566d44a8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.529+0000 7f8915311740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566d449bdc0 0x5566d44a8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.529+0000 7f8915311740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566d449c760 0x5566d44a58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.529+0000 7f8915311740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1a9890 con 0x5566d449bdc0 2024-09-12T03:12:07.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.529+0000 7f8915311740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1aad00 con 0x5566d449b040 2024-09-12T03:12:07.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.529+0000 7f8915311740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1a9f60 con 0x5566d449c760 2024-09-12T03:12:07.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.529+0000 7f8915311740 10 client.0 _ll_register_callbacks cb 0x7f89080097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:07.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.529+0000 7f8903fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566d449bdc0 0x5566d44a8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.529+0000 7f8903fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566d449bdc0 0x5566d44a8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42822/0 (socket says 172.21.15.116:42822) 2024-09-12T03:12:07.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.529+0000 7f8903fff640 1 -- 172.21.15.116:0/3285666074 learned_addr learned my addr 172.21.15.116:0/3285666074 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:07.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.529+0000 7f8910cd4640 1 --2- 172.21.15.116:0/3285666074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566d449b040 0x5566d44a8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.529+0000 7f8903fff640 1 -- 172.21.15.116:0/3285666074 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566d449c760 msgr2=0x5566d44a58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:07.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.529+0000 7f8903fff640 1 --2- 172.21.15.116:0/3285666074 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566d449c760 0x5566d44a58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.529+0000 7f8903fff640 1 -- 172.21.15.116:0/3285666074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566d449b040 msgr2=0x5566d44a8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:07.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.529+0000 7f8903fff640 1 --2- 172.21.15.116:0/3285666074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566d449b040 0x5566d44a8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.529+0000 7f8903fff640 1 -- 172.21.15.116:0/3285666074 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566d44a60d0 con 0x5566d449bdc0 2024-09-12T03:12:07.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.529+0000 7f8910cd4640 1 --2- 172.21.15.116:0/3285666074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566d449b040 0x5566d44a8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:12:07.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.529+0000 7f8903fff640 1 --2- 172.21.15.116:0/3285666074 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566d449bdc0 0x5566d44a8ff0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f88fc00bca0 tx=0x7f88fc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e043dbeb8d2a498 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:07.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.529+0000 7f88ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:12:07.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.529+0000 7f88ebfff640 1 -- 172.21.15.116:0/3285666074 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc00b990 con 0x5566d449bdc0 2024-09-12T03:12:07.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.529+0000 7f88ebfff640 1 -- 172.21.15.116:0/3285666074 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f88fc0120d0 con 0x5566d449bdc0 2024-09-12T03:12:07.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.529+0000 7f8915311740 1 -- 172.21.15.116:0/3285666074 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8904036040 con 0x5566d449bdc0 2024-09-12T03:12:07.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.529+0000 7f88ebfff640 1 -- 172.21.15.116:0/3285666074 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f88fc00b690 con 0x5566d449bdc0 2024-09-12T03:12:07.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.529+0000 7f8915311740 10 client.17826 fetch_fsmap learned FSMap version 25 2024-09-12T03:12:07.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.529+0000 7f8915311740 1 -- 172.21.15.116:0/3285666074 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5566d451ba90 con 0x5566d449bdc0 2024-09-12T03:12:07.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.530+0000 7f88ebfff640 1 -- 172.21.15.116:0/3285666074 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8904036040 con 0x5566d449bdc0 2024-09-12T03:12:07.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.530+0000 7f88ebfff640 1 -- 172.21.15.116:0/3285666074 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f88fc005140 con 0x5566d449bdc0 2024-09-12T03:12:07.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.530+0000 7f8915311740 10 client.17826 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:12:07.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.530+0000 7f8915311740 10 client.17826 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:12:07.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.530+0000 7f8915311740 20 client.17826 populate_metadata read hostname 'smithi116' 2024-09-12T03:12:07.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.530+0000 7f8915311740 1 --2- 172.21.15.116:0/3285666074 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5566d451bed0 0x5566d451e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.530+0000 7f8915311740 4 client.17826 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:12:07.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.530+0000 7f8915311740 1 -- 172.21.15.116:0/3285666074 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5566d43fed70 con 0x5566d451bed0 2024-09-12T03:12:07.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.530+0000 7f8910cd4640 1 --2- 172.21.15.116:0/3285666074 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5566d451bed0 0x5566d451e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.531+0000 7f8910cd4640 1 --2- 172.21.15.116:0/3285666074 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5566d451bed0 0x5566d451e2c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=59096a868e3d004e server_cookie=dac9597b4701811c in_seq=0 out_seq=0 2024-09-12T03:12:07.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.531+0000 7f88ebfff640 10 client.17826 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:12:07.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.531+0000 7f88ebfff640 1 -- 172.21.15.116:0/3285666074 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5566d43fed70 con 0x5566d451bed0 2024-09-12T03:12:07.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.531+0000 7f88ebfff640 10 client.17826 handle_command_reply: tid=0 2024-09-12T03:12:07.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.531+0000 7f8915311740 1 client.17826 shutdown 2024-09-12T03:12:07.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.531+0000 7f8915311740 1 -- 172.21.15.116:0/3285666074 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566d449bdc0 msgr2=0x5566d44a8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:07.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.531+0000 7f8915311740 1 --2- 172.21.15.116:0/3285666074 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566d449bdc0 0x5566d44a8ff0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f88fc00bca0 tx=0x7f88fc018000 comp rx=0 tx=0).stop 2024-09-12T03:12:07.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.531+0000 7f8915311740 1 -- 172.21.15.116:0/3285666074 shutdown_connections 2024-09-12T03:12:07.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.531+0000 7f8915311740 1 --2- 172.21.15.116:0/3285666074 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566d449c760 0x5566d44a58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.531+0000 7f8915311740 1 --2- 172.21.15.116:0/3285666074 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5566d451bed0 0x5566d451e2c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.531+0000 7f8915311740 1 --2- 172.21.15.116:0/3285666074 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5566d449bdc0 0x5566d44a8ff0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.531+0000 7f8915311740 1 --2- 172.21.15.116:0/3285666074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566d449b040 0x5566d44a8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.531+0000 7f88ebfff640 0 client.17826 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:12:07.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.531+0000 7f8915311740 1 -- 172.21.15.116:0/3285666074 >> 172.21.15.116:0/3285666074 conn(0x5566d445a310 msgr2=0x5566d44a1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:07.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.531+0000 7f8915311740 1 -- 172.21.15.116:0/3285666074 shutdown_connections 2024-09-12T03:12:07.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.531+0000 7f8915311740 1 -- 172.21.15.116:0/3285666074 wait complete. 2024-09-12T03:12:07.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.531+0000 7f8915311740 20 client.17826 trim_cache size 0 max 16384 2024-09-12T03:12:07.539 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:12:07.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.532+0000 7f8912f38640 1 -- 172.21.15.116:0/1448254500 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f88f40a2c00 msgr2=0x7f88f40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:07.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.532+0000 7f8912f38640 1 --2- 172.21.15.116:0/1448254500 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f88f40a2c00 0x7f88f40a50c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f88fc00c010 tx=0x7f88fc00fa90 comp rx=0 tx=0).stop 2024-09-12T03:12:07.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.532+0000 7f8912f38640 1 -- 172.21.15.116:0/1448254500 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f890c105590 msgr2=0x7f890c19bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:07.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.532+0000 7f8912f38640 1 --2- 172.21.15.116:0/1448254500 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f890c105590 0x7f890c19bba0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f8904031a20 tx=0x7f8904031a50 comp rx=0 tx=0).stop 2024-09-12T03:12:07.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.532+0000 7f8912f38640 1 -- 172.21.15.116:0/1448254500 shutdown_connections 2024-09-12T03:12:07.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.532+0000 7f8912f38640 1 --2- 172.21.15.116:0/1448254500 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f88f40a2c00 0x7f88f40a50c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.532+0000 7f8912f38640 1 --2- 172.21.15.116:0/1448254500 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f890c105590 0x7f890c19bba0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.532+0000 7f8912f38640 1 --2- 172.21.15.116:0/1448254500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f890c100e10 0x7f890c197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.532+0000 7f8912f38640 1 --2- 172.21.15.116:0/1448254500 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f890c100460 0x7f890c1972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.532+0000 7f8912f38640 1 -- 172.21.15.116:0/1448254500 >> 172.21.15.116:0/1448254500 conn(0x7f890c0fbee0 msgr2=0x7f890c0fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:07.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.532+0000 7f8912f38640 1 -- 172.21.15.116:0/1448254500 shutdown_connections 2024-09-12T03:12:07.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:07.532+0000 7f8912f38640 1 -- 172.21.15.116:0/1448254500 wait complete. 2024-09-12T03:12:07.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:07 smithi183 ceph-mon[35535]: pgmap v612: 97 pgs: 97 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:12:07.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:07 smithi177 ceph-mon[35305]: pgmap v612: 97 pgs: 97 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:12:07.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:07 smithi116 ceph-mon[31410]: pgmap v612: 97 pgs: 97 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:12:08.278 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:12:08.531 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:12:09.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.004+0000 7f36f28d6640 1 -- 172.21.15.116:0/3418294247 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36ec101de0 msgr2=0x7f36ec1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:09.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.004+0000 7f36f28d6640 1 --2- 172.21.15.116:0/3418294247 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36ec101de0 0x7f36ec1021c0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f36dc009900 tx=0x7f36dc02a410 comp rx=0 tx=0).stop 2024-09-12T03:12:09.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.004+0000 7f36f28d6640 1 -- 172.21.15.116:0/3418294247 shutdown_connections 2024-09-12T03:12:09.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.004+0000 7f36f28d6640 1 --2- 172.21.15.116:0/3418294247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f36ec106f10 0x7f36ec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.004+0000 7f36f28d6640 1 --2- 172.21.15.116:0/3418294247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36ec102790 0x7f36ec1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.004+0000 7f36f28d6640 1 --2- 172.21.15.116:0/3418294247 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36ec101de0 0x7f36ec1021c0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.004+0000 7f36f28d6640 1 -- 172.21.15.116:0/3418294247 >> 172.21.15.116:0/3418294247 conn(0x7f36ec0fd8c0 msgr2=0x7f36ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:09.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.004+0000 7f36f28d6640 1 -- 172.21.15.116:0/3418294247 shutdown_connections 2024-09-12T03:12:09.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.004+0000 7f36f28d6640 1 -- 172.21.15.116:0/3418294247 wait complete. 2024-09-12T03:12:09.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.005+0000 7f36f28d6640 1 Processor -- start 2024-09-12T03:12:09.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.005+0000 7f36f28d6640 1 -- start start 2024-09-12T03:12:09.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.005+0000 7f36f28d6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36ec101de0 0x7f36ec1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:09.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.005+0000 7f36f28d6640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36ec102790 0x7f36ec198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:09.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.005+0000 7f36f28d6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f36ec106f10 0x7f36ec19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:09.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.005+0000 7f36f28d6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec111fb0 con 0x7f36ec101de0 2024-09-12T03:12:09.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.005+0000 7f36f28d6640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec111e30 con 0x7f36ec102790 2024-09-12T03:12:09.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.005+0000 7f36f28d6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec112130 con 0x7f36ec106f10 2024-09-12T03:12:09.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.005+0000 7f36ebfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36ec101de0 0x7f36ec1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:09.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.006+0000 7f36ebfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36ec101de0 0x7f36ec1985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42836/0 (socket says 172.21.15.116:42836) 2024-09-12T03:12:09.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.006+0000 7f36ebfff640 1 -- 172.21.15.116:0/3805724015 learned_addr learned my addr 172.21.15.116:0/3805724015 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:09.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.006+0000 7f36eb7fe640 1 --2- 172.21.15.116:0/3805724015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36ec102790 0x7f36ec198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:09.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.006+0000 7f36f0e73640 1 --2- 172.21.15.116:0/3805724015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f36ec106f10 0x7f36ec19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:09.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.006+0000 7f36ebfff640 1 -- 172.21.15.116:0/3805724015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f36ec106f10 msgr2=0x7f36ec19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:09.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.006+0000 7f36ebfff640 1 --2- 172.21.15.116:0/3805724015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f36ec106f10 0x7f36ec19ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.006+0000 7f36ebfff640 1 -- 172.21.15.116:0/3805724015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36ec102790 msgr2=0x7f36ec198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:09.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.006+0000 7f36ebfff640 1 --2- 172.21.15.116:0/3805724015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36ec102790 0x7f36ec198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.006+0000 7f36ebfff640 1 -- 172.21.15.116:0/3805724015 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36ec19d560 con 0x7f36ec101de0 2024-09-12T03:12:09.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.006+0000 7f36ebfff640 1 --2- 172.21.15.116:0/3805724015 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36ec101de0 0x7f36ec1985b0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f36dc005ab0 tx=0x7f36dc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:09.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.006+0000 7f36e97fa640 1 -- 172.21.15.116:0/3805724015 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc03f030 con 0x7f36ec101de0 2024-09-12T03:12:09.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.006+0000 7f36e97fa640 1 -- 172.21.15.116:0/3805724015 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36dc003ce0 con 0x7f36ec101de0 2024-09-12T03:12:09.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.006+0000 7f36f28d6640 1 -- 172.21.15.116:0/3805724015 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36ec19d7f0 con 0x7f36ec101de0 2024-09-12T03:12:09.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.006+0000 7f36e97fa640 1 -- 172.21.15.116:0/3805724015 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc035070 con 0x7f36ec101de0 2024-09-12T03:12:09.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.006+0000 7f36f28d6640 1 -- 172.21.15.116:0/3805724015 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36ec1a5130 con 0x7f36ec101de0 2024-09-12T03:12:09.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.009+0000 7f36e97fa640 1 -- 172.21.15.116:0/3805724015 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f36dc042030 con 0x7f36ec101de0 2024-09-12T03:12:09.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.009+0000 7f36f4caf740 1 Processor -- start 2024-09-12T03:12:09.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.009+0000 7f36f4caf740 1 -- start start 2024-09-12T03:12:09.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.009+0000 7f36f4caf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c827559b40 0x55c827558090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:09.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.009+0000 7f36f4caf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c8275585d0 0x55c827558a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:09.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.009+0000 7f36f4caf740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c827563370 0x55c827565760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:09.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.009+0000 7f36f4caf740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c8274be2f0 con 0x55c827563370 2024-09-12T03:12:09.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.009+0000 7f36f4caf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c827465cf0 con 0x55c8275585d0 2024-09-12T03:12:09.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.009+0000 7f36f4caf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c8274ba2b0 con 0x55c827559b40 2024-09-12T03:12:09.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.009+0000 7f36eb7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c827563370 0x55c827565760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:09.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.009+0000 7f36f0e73640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c827559b40 0x55c827558090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:09.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.009+0000 7f36eb7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c827563370 0x55c827565760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42852/0 (socket says 172.21.15.116:42852) 2024-09-12T03:12:09.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.009+0000 7f36eb7fe640 1 -- 172.21.15.116:0/1065169015 learned_addr learned my addr 172.21.15.116:0/1065169015 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:09.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.009+0000 7f36e97fa640 1 --2- 172.21.15.116:0/3805724015 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f36d40a2b80 0x7f36d40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:09.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.010+0000 7f36ebfff640 1 --2- 172.21.15.116:0/1065169015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c8275585d0 0x55c827558a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:09.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.010+0000 7f36eb7fe640 1 -- 172.21.15.116:0/1065169015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c827559b40 msgr2=0x55c827558090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:09.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.010+0000 7f36eb7fe640 1 --2- 172.21.15.116:0/1065169015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c827559b40 0x55c827558090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.010+0000 7f36eb7fe640 1 -- 172.21.15.116:0/1065169015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c8275585d0 msgr2=0x55c827558a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:09.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.010+0000 7f36eb7fe640 1 --2- 172.21.15.116:0/1065169015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c8275585d0 0x55c827558a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.010+0000 7f36eb7fe640 1 -- 172.21.15.116:0/1065169015 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c82744a3a0 con 0x55c827563370 2024-09-12T03:12:09.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.010+0000 7f36ebfff640 1 --2- 172.21.15.116:0/1065169015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c8275585d0 0x55c827558a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:12:09.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.010+0000 7f36f0e73640 1 --2- 172.21.15.116:0/3805724015 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f36d40a2b80 0x7f36d40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:09.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.010+0000 7f36e97fa640 1 -- 172.21.15.116:0/3805724015 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f36dc0ef990 con 0x7f36ec101de0 2024-09-12T03:12:09.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.010+0000 7f36eb7fe640 1 --2- 172.21.15.116:0/1065169015 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c827563370 0x55c827565760 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f36e0009a40 tx=0x7f36e0009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c8d74a903a082b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:09.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.010+0000 7f36d27fc640 1 -- 172.21.15.116:0/1065169015 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e0017070 con 0x55c827563370 2024-09-12T03:12:09.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.010+0000 7f36d27fc640 1 -- 172.21.15.116:0/1065169015 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36e0013470 con 0x55c827563370 2024-09-12T03:12:09.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.010+0000 7f36d27fc640 1 -- 172.21.15.116:0/1065169015 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e00125d0 con 0x55c827563370 2024-09-12T03:12:09.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.011+0000 7f36f4caf740 1 -- 172.21.15.116:0/1065169015 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c827563370 msgr2=0x55c827565760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:09.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.011+0000 7f36f0e73640 1 --2- 172.21.15.116:0/3805724015 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f36d40a2b80 0x7f36d40a5040 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f36e4008ea0 tx=0x7f36e4004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:09.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.011+0000 7f36f4caf740 1 --2- 172.21.15.116:0/1065169015 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c827563370 0x55c827565760 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f36e0009a40 tx=0x7f36e0009f00 comp rx=0 tx=0).stop 2024-09-12T03:12:09.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.011+0000 7f36f4caf740 1 -- 172.21.15.116:0/1065169015 shutdown_connections 2024-09-12T03:12:09.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.011+0000 7f36f4caf740 1 --2- 172.21.15.116:0/1065169015 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c827563370 0x55c827565760 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.011+0000 7f36f4caf740 1 --2- 172.21.15.116:0/1065169015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c8275585d0 0x55c827558a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.011+0000 7f36f4caf740 1 --2- 172.21.15.116:0/1065169015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c827559b40 0x55c827558090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.011+0000 7f36f4caf740 1 -- 172.21.15.116:0/1065169015 >> 172.21.15.116:0/1065169015 conn(0x55c827521310 msgr2=0x55c827554c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:09.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.011+0000 7f36f4caf740 1 -- 172.21.15.116:0/1065169015 shutdown_connections 2024-09-12T03:12:09.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.011+0000 7f36f4caf740 1 -- 172.21.15.116:0/1065169015 wait complete. 2024-09-12T03:12:09.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.011+0000 7f36f4caf740 1 -- start start 2024-09-12T03:12:09.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.011+0000 7f36f4caf740 1 Processor -- start 2024-09-12T03:12:09.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.012+0000 7f36f4caf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8275585d0 0x55c827562d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:09.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.012+0000 7f36f4caf740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c827559b40 0x55c82755ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:09.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.012+0000 7f36f4caf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c827563370 0x55c8275605d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:09.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.012+0000 7f36f4caf740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c82756a480 con 0x55c827559b40 2024-09-12T03:12:09.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.012+0000 7f36f4caf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c827479060 con 0x55c827563370 2024-09-12T03:12:09.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.012+0000 7f36f4caf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c82756a600 con 0x55c8275585d0 2024-09-12T03:12:09.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.012+0000 7f36f4caf740 10 client.0 _ll_register_callbacks cb 0x55c8271cc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:09.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.012+0000 7f36eb7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c827559b40 0x55c82755ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:09.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.012+0000 7f36eb7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c827559b40 0x55c82755ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42862/0 (socket says 172.21.15.116:42862) 2024-09-12T03:12:09.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.012+0000 7f36eb7fe640 1 -- 172.21.15.116:0/2609596275 learned_addr learned my addr 172.21.15.116:0/2609596275 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:09.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.012+0000 7f36ebfff640 1 --2- 172.21.15.116:0/2609596275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8275585d0 0x55c827562d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:09.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.012+0000 7f36eb7fe640 1 -- 172.21.15.116:0/2609596275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8275585d0 msgr2=0x55c827562d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:09.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.012+0000 7f36eb7fe640 1 --2- 172.21.15.116:0/2609596275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8275585d0 0x55c827562d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.012+0000 7f36eb7fe640 1 -- 172.21.15.116:0/2609596275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c827563370 msgr2=0x55c8275605d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:09.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.012+0000 7f36eb7fe640 1 --2- 172.21.15.116:0/2609596275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c827563370 0x55c8275605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.012+0000 7f36eb7fe640 1 -- 172.21.15.116:0/2609596275 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c82722f3c0 con 0x55c827559b40 2024-09-12T03:12:09.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.012+0000 7f36eb7fe640 1 --2- 172.21.15.116:0/2609596275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c827559b40 0x55c82755ffe0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f36e0000c00 tx=0x7f36e0013ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf19c2eacc5ff89 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:09.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.012+0000 7f36cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:12:09.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.012+0000 7f36cb7fe640 1 -- 172.21.15.116:0/2609596275 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e0017050 con 0x55c827559b40 2024-09-12T03:12:09.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.012+0000 7f36cb7fe640 1 -- 172.21.15.116:0/2609596275 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f36e001c450 con 0x55c827559b40 2024-09-12T03:12:09.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.012+0000 7f36f4caf740 1 -- 172.21.15.116:0/2609596275 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36e0013470 con 0x55c827559b40 2024-09-12T03:12:09.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.013+0000 7f36cb7fe640 1 -- 172.21.15.116:0/2609596275 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f36e002d070 con 0x55c827559b40 2024-09-12T03:12:09.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.013+0000 7f36f4caf740 10 client.17838 fetch_fsmap learned FSMap version 25 2024-09-12T03:12:09.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.013+0000 7f36f4caf740 1 -- 172.21.15.116:0/2609596275 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c827575e50 con 0x55c827559b40 2024-09-12T03:12:09.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.013+0000 7f36cb7fe640 1 -- 172.21.15.116:0/2609596275 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36e0013470 con 0x55c827559b40 2024-09-12T03:12:09.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.013+0000 7f36cb7fe640 1 -- 172.21.15.116:0/2609596275 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f36e0026810 con 0x55c827559b40 2024-09-12T03:12:09.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.013+0000 7f36f4caf740 10 client.17838 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:12:09.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.013+0000 7f36f4caf740 10 client.17838 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:12:09.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.013+0000 7f36f4caf740 20 client.17838 populate_metadata read hostname 'smithi116' 2024-09-12T03:12:09.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.013+0000 7f36f4caf740 1 --2- 172.21.15.116:0/2609596275 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c827577250 0x55c827579640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:09.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.013+0000 7f36f4caf740 4 client.17838 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:12:09.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.013+0000 7f36f4caf740 1 -- 172.21.15.116:0/2609596275 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8274cb860 con 0x55c827577250 2024-09-12T03:12:09.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.013+0000 7f36ebfff640 1 --2- 172.21.15.116:0/2609596275 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c827577250 0x55c827579640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:09.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.014+0000 7f36cb7fe640 10 client.17838 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:12:09.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.014+0000 7f36ebfff640 1 --2- 172.21.15.116:0/2609596275 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c827577250 0x55c827579640 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b43629bc4f064c08 server_cookie=87bd8c4d3b984c09 in_seq=0 out_seq=0 2024-09-12T03:12:09.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.014+0000 7f36cb7fe640 1 -- 172.21.15.116:0/2609596275 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c8274cb860 con 0x55c827577250 2024-09-12T03:12:09.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.014+0000 7f36cb7fe640 10 client.17838 handle_command_reply: tid=0 2024-09-12T03:12:09.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.014+0000 7f36f4caf740 1 client.17838 shutdown 2024-09-12T03:12:09.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.014+0000 7f36f4caf740 1 -- 172.21.15.116:0/2609596275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c827559b40 msgr2=0x55c82755ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:09.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.014+0000 7f36f4caf740 1 --2- 172.21.15.116:0/2609596275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c827559b40 0x55c82755ffe0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f36e0000c00 tx=0x7f36e0013ce0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.015+0000 7f36f4caf740 1 -- 172.21.15.116:0/2609596275 shutdown_connections 2024-09-12T03:12:09.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.015+0000 7f36f4caf740 1 --2- 172.21.15.116:0/2609596275 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c827577250 0x55c827579640 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.015+0000 7f36f4caf740 1 --2- 172.21.15.116:0/2609596275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c827563370 0x55c8275605d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.015+0000 7f36f4caf740 1 --2- 172.21.15.116:0/2609596275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c827559b40 0x55c82755ffe0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.015+0000 7f36f4caf740 1 --2- 172.21.15.116:0/2609596275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c8275585d0 0x55c827562d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.015+0000 7f36f4caf740 1 -- 172.21.15.116:0/2609596275 >> 172.21.15.116:0/2609596275 conn(0x55c827521310 msgr2=0x55c82755a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:09.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.015+0000 7f36cb7fe640 0 client.17838 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:12:09.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.015+0000 7f36f4caf740 1 -- 172.21.15.116:0/2609596275 shutdown_connections 2024-09-12T03:12:09.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.015+0000 7f36f4caf740 1 -- 172.21.15.116:0/2609596275 wait complete. 2024-09-12T03:12:09.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.015+0000 7f36f4caf740 20 client.17838 trim_cache size 0 max 16384 2024-09-12T03:12:09.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.027+0000 7f36f4caf740 1 Processor -- start 2024-09-12T03:12:09.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.027+0000 7f36f4caf740 1 -- start start 2024-09-12T03:12:09.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.028+0000 7f36f4caf740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c8275585d0 0x55c827573720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:09.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.028+0000 7f36f4caf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c827559b40 0x55c82757eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:09.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.028+0000 7f36f4caf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c827563370 0x55c82757f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:09.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.028+0000 7f36f4caf740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1adbd0 con 0x55c8275585d0 2024-09-12T03:12:09.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.028+0000 7f36f4caf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1ab4a0 con 0x55c827559b40 2024-09-12T03:12:09.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.028+0000 7f36f4caf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1a9310 con 0x55c827563370 2024-09-12T03:12:09.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.028+0000 7f36ebfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c8275585d0 0x55c827573720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:09.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.028+0000 7f36ebfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c8275585d0 0x55c827573720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42872/0 (socket says 172.21.15.116:42872) 2024-09-12T03:12:09.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.028+0000 7f36ebfff640 1 -- 172.21.15.116:0/1367306283 learned_addr learned my addr 172.21.15.116:0/1367306283 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:09.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.028+0000 7f36ebfff640 1 -- 172.21.15.116:0/1367306283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c827563370 msgr2=0x55c82757f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:09.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.028+0000 7f36f0e73640 1 --2- 172.21.15.116:0/1367306283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c827563370 0x55c82757f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:09.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.028+0000 7f36ebfff640 1 --2- 172.21.15.116:0/1367306283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c827563370 0x55c82757f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.028+0000 7f36eb7fe640 1 --2- 172.21.15.116:0/1367306283 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c827559b40 0x55c82757eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:09.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.028+0000 7f36ebfff640 1 -- 172.21.15.116:0/1367306283 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c827559b40 msgr2=0x55c82757eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:09.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.028+0000 7f36ebfff640 1 --2- 172.21.15.116:0/1367306283 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c827559b40 0x55c82757eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.028+0000 7f36ebfff640 1 -- 172.21.15.116:0/1367306283 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c827560940 con 0x55c8275585d0 2024-09-12T03:12:09.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.028+0000 7f36f0e73640 1 --2- 172.21.15.116:0/1367306283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c827563370 0x55c82757f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:12:09.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.028+0000 7f36eb7fe640 1 --2- 172.21.15.116:0/1367306283 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c827559b40 0x55c82757eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:12:09.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.028+0000 7f36ebfff640 1 --2- 172.21.15.116:0/1367306283 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c8275585d0 0x55c827573720 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f36dc000c00 tx=0x7f36dc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b034d726f961e835 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:09.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.028+0000 7f36d1ffb640 1 -- 172.21.15.116:0/1367306283 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc0dae60 con 0x55c8275585d0 2024-09-12T03:12:09.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.028+0000 7f36d1ffb640 1 -- 172.21.15.116:0/1367306283 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36dc0bc220 con 0x55c8275585d0 2024-09-12T03:12:09.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.028+0000 7f36d1ffb640 1 -- 172.21.15.116:0/1367306283 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc049040 con 0x55c8275585d0 2024-09-12T03:12:09.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.029+0000 7f36f4caf740 1 -- 172.21.15.116:0/1367306283 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c8275585d0 msgr2=0x55c827573720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:09.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.029+0000 7f36f4caf740 1 --2- 172.21.15.116:0/1367306283 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c8275585d0 0x55c827573720 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f36dc000c00 tx=0x7f36dc02a410 comp rx=0 tx=0).stop 2024-09-12T03:12:09.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.029+0000 7f36f4caf740 1 -- 172.21.15.116:0/1367306283 shutdown_connections 2024-09-12T03:12:09.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.029+0000 7f36f4caf740 1 --2- 172.21.15.116:0/1367306283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c827563370 0x55c82757f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.029+0000 7f36f4caf740 1 --2- 172.21.15.116:0/1367306283 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c827559b40 0x55c82757eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.029+0000 7f36f4caf740 1 --2- 172.21.15.116:0/1367306283 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c8275585d0 0x55c827573720 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.029+0000 7f36f4caf740 1 -- 172.21.15.116:0/1367306283 >> 172.21.15.116:0/1367306283 conn(0x55c827521310 msgr2=0x55c82755f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:09.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.029+0000 7f36f4caf740 1 -- 172.21.15.116:0/1367306283 shutdown_connections 2024-09-12T03:12:09.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.029+0000 7f36f4caf740 1 -- 172.21.15.116:0/1367306283 wait complete. 2024-09-12T03:12:09.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.030+0000 7f36f4caf740 1 -- start start 2024-09-12T03:12:09.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.030+0000 7f36f4caf740 1 Processor -- start 2024-09-12T03:12:09.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.030+0000 7f36f4caf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c8275585d0 0x55c82756b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:09.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.030+0000 7f36f4caf740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c827559b40 0x55c82756bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:09.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.030+0000 7f36f4caf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c827563370 0x55c82756c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:09.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.030+0000 7f36f4caf740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1a9960 con 0x55c827559b40 2024-09-12T03:12:09.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.030+0000 7f36f4caf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1aadd0 con 0x55c8275585d0 2024-09-12T03:12:09.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.030+0000 7f36f4caf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1aa030 con 0x55c827563370 2024-09-12T03:12:09.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.030+0000 7f36f4caf740 10 client.0 _ll_register_callbacks cb 0x7f36e4008490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:09.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.030+0000 7f36eb7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c827559b40 0x55c82756bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:09.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.030+0000 7f36eb7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c827559b40 0x55c82756bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42878/0 (socket says 172.21.15.116:42878) 2024-09-12T03:12:09.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.030+0000 7f36eb7fe640 1 -- 172.21.15.116:0/3661589300 learned_addr learned my addr 172.21.15.116:0/3661589300 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:09.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.030+0000 7f36ebfff640 1 --2- 172.21.15.116:0/3661589300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c8275585d0 0x55c82756b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:09.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.030+0000 7f36f0e73640 1 --2- 172.21.15.116:0/3661589300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c827563370 0x55c82756c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:09.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.030+0000 7f36eb7fe640 1 -- 172.21.15.116:0/3661589300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c827563370 msgr2=0x55c82756c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:09.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.031+0000 7f36eb7fe640 1 --2- 172.21.15.116:0/3661589300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c827563370 0x55c82756c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.031+0000 7f36eb7fe640 1 -- 172.21.15.116:0/3661589300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c8275585d0 msgr2=0x55c82756b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:09.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.031+0000 7f36eb7fe640 1 --2- 172.21.15.116:0/3661589300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c8275585d0 0x55c82756b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.031+0000 7f36eb7fe640 1 -- 172.21.15.116:0/3661589300 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8275e0e70 con 0x55c827559b40 2024-09-12T03:12:09.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.031+0000 7f36ebfff640 1 --2- 172.21.15.116:0/3661589300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c8275585d0 0x55c82756b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:12:09.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.031+0000 7f36eb7fe640 1 --2- 172.21.15.116:0/3661589300 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c827559b40 0x55c82756bcf0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f36e00097d0 tx=0x7f36e000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14be9d6609a5086a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:09.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.031+0000 7f36cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:12:09.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.031+0000 7f36cb7fe640 1 -- 172.21.15.116:0/3661589300 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e0037060 con 0x55c827559b40 2024-09-12T03:12:09.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.031+0000 7f36cb7fe640 1 -- 172.21.15.116:0/3661589300 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f36e00263e0 con 0x55c827559b40 2024-09-12T03:12:09.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.031+0000 7f36f4caf740 1 -- 172.21.15.116:0/3661589300 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36dc0bc220 con 0x55c827559b40 2024-09-12T03:12:09.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.031+0000 7f36cb7fe640 1 -- 172.21.15.116:0/3661589300 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f36e0017050 con 0x55c827559b40 2024-09-12T03:12:09.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.031+0000 7f36f4caf740 10 client.17844 fetch_fsmap learned FSMap version 25 2024-09-12T03:12:09.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.031+0000 7f36f4caf740 1 -- 172.21.15.116:0/3661589300 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c8275e20b0 con 0x55c827559b40 2024-09-12T03:12:09.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.031+0000 7f36cb7fe640 1 -- 172.21.15.116:0/3661589300 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36dc0bc220 con 0x55c827559b40 2024-09-12T03:12:09.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.031+0000 7f36cb7fe640 1 -- 172.21.15.116:0/3661589300 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f36e0031c70 con 0x55c827559b40 2024-09-12T03:12:09.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.031+0000 7f36f4caf740 10 client.17844 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:12:09.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.031+0000 7f36f4caf740 10 client.17844 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:12:09.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.031+0000 7f36f4caf740 20 client.17844 populate_metadata read hostname 'smithi116' 2024-09-12T03:12:09.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.031+0000 7f36f4caf740 1 --2- 172.21.15.116:0/3661589300 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c8275e2860 0x55c8275e4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:09.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.031+0000 7f36f4caf740 4 client.17844 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:12:09.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.031+0000 7f36f4caf740 1 -- 172.21.15.116:0/3661589300 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c8274c5d70 con 0x55c8275e2860 2024-09-12T03:12:09.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.032+0000 7f36ebfff640 1 --2- 172.21.15.116:0/3661589300 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c8275e2860 0x55c8275e4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:09.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.032+0000 7f36cb7fe640 10 client.17844 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:12:09.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.032+0000 7f36ebfff640 1 --2- 172.21.15.116:0/3661589300 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c8275e2860 0x55c8275e4c80 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=79c083fab5315cb0 server_cookie=8e0e77c077787c67 in_seq=0 out_seq=0 2024-09-12T03:12:09.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.032+0000 7f36cb7fe640 1 -- 172.21.15.116:0/3661589300 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c8274c5d70 con 0x55c8275e2860 2024-09-12T03:12:09.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.033+0000 7f36cb7fe640 10 client.17844 handle_command_reply: tid=0 2024-09-12T03:12:09.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.033+0000 7f36f4caf740 1 client.17844 shutdown 2024-09-12T03:12:09.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.033+0000 7f36f4caf740 1 -- 172.21.15.116:0/3661589300 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c827559b40 msgr2=0x55c82756bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:09.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.033+0000 7f36f4caf740 1 --2- 172.21.15.116:0/3661589300 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c827559b40 0x55c82756bcf0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f36e00097d0 tx=0x7f36e000ee50 comp rx=0 tx=0).stop 2024-09-12T03:12:09.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.033+0000 7f36f4caf740 1 -- 172.21.15.116:0/3661589300 shutdown_connections 2024-09-12T03:12:09.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.033+0000 7f36f4caf740 1 --2- 172.21.15.116:0/3661589300 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c8275e2860 0x55c8275e4c80 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.033+0000 7f36f4caf740 1 --2- 172.21.15.116:0/3661589300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c827563370 0x55c82756c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.033+0000 7f36f4caf740 1 --2- 172.21.15.116:0/3661589300 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c827559b40 0x55c82756bcf0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.033+0000 7f36cb7fe640 0 client.17844 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:12:09.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.033+0000 7f36f4caf740 1 --2- 172.21.15.116:0/3661589300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c8275585d0 0x55c82756b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.033+0000 7f36f4caf740 1 -- 172.21.15.116:0/3661589300 >> 172.21.15.116:0/3661589300 conn(0x55c827521310 msgr2=0x55c827561840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:09.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.033+0000 7f36f4caf740 1 -- 172.21.15.116:0/3661589300 shutdown_connections 2024-09-12T03:12:09.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.033+0000 7f36f4caf740 1 -- 172.21.15.116:0/3661589300 wait complete. 2024-09-12T03:12:09.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.033+0000 7f36f4caf740 20 client.17844 trim_cache size 0 max 16384 2024-09-12T03:12:09.040 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:12:09.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.033+0000 7f36f28d6640 1 -- 172.21.15.116:0/3805724015 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f36d40a2b80 msgr2=0x7f36d40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:09.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.033+0000 7f36f28d6640 1 --2- 172.21.15.116:0/3805724015 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f36d40a2b80 0x7f36d40a5040 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f36e4008ea0 tx=0x7f36e4004a40 comp rx=0 tx=0).stop 2024-09-12T03:12:09.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.033+0000 7f36f28d6640 1 -- 172.21.15.116:0/3805724015 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36ec101de0 msgr2=0x7f36ec1985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:09.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.033+0000 7f36f28d6640 1 --2- 172.21.15.116:0/3805724015 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36ec101de0 0x7f36ec1985b0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f36dc005ab0 tx=0x7f36dc0042f0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.034+0000 7f36f28d6640 1 -- 172.21.15.116:0/3805724015 shutdown_connections 2024-09-12T03:12:09.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.034+0000 7f36f28d6640 1 --2- 172.21.15.116:0/3805724015 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f36d40a2b80 0x7f36d40a5040 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.034+0000 7f36f28d6640 1 --2- 172.21.15.116:0/3805724015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f36ec106f10 0x7f36ec19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.034+0000 7f36f28d6640 1 --2- 172.21.15.116:0/3805724015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36ec102790 0x7f36ec198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.034+0000 7f36f28d6640 1 --2- 172.21.15.116:0/3805724015 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36ec101de0 0x7f36ec1985b0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:09.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.034+0000 7f36f28d6640 1 -- 172.21.15.116:0/3805724015 >> 172.21.15.116:0/3805724015 conn(0x7f36ec0fd8c0 msgr2=0x7f36ec0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:09.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.034+0000 7f36f28d6640 1 -- 172.21.15.116:0/3805724015 shutdown_connections 2024-09-12T03:12:09.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:09.034+0000 7f36f28d6640 1 -- 172.21.15.116:0/3805724015 wait complete. 2024-09-12T03:12:09.791 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:12:10.039 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:12:10.458 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:10 smithi116 ceph-mon[31410]: pgmap v613: 97 pgs: 97 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:12:10.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.539+0000 7fa32d59a640 1 -- 172.21.15.116:0/3956066583 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa328102d30 msgr2=0x7fa328103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:10.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.539+0000 7fa32d59a640 1 --2- 172.21.15.116:0/3956066583 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa328102d30 0x7fa328103110 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fa318009cd0 tx=0x7fa31802a8e0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.539+0000 7fa32d59a640 1 -- 172.21.15.116:0/3956066583 shutdown_connections 2024-09-12T03:12:10.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.539+0000 7fa32d59a640 1 --2- 172.21.15.116:0/3956066583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3281082a0 0x7fa32810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.539+0000 7fa32d59a640 1 --2- 172.21.15.116:0/3956066583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3281036e0 0x7fa328107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.539+0000 7fa32d59a640 1 --2- 172.21.15.116:0/3956066583 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa328102d30 0x7fa328103110 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.539+0000 7fa32d59a640 1 -- 172.21.15.116:0/3956066583 >> 172.21.15.116:0/3956066583 conn(0x7fa3280fc7d0 msgr2=0x7fa3280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:10.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.539+0000 7fa32d59a640 1 -- 172.21.15.116:0/3956066583 shutdown_connections 2024-09-12T03:12:10.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.539+0000 7fa32d59a640 1 -- 172.21.15.116:0/3956066583 wait complete. 2024-09-12T03:12:10.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.539+0000 7fa32d59a640 1 Processor -- start 2024-09-12T03:12:10.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.539+0000 7fa32d59a640 1 -- start start 2024-09-12T03:12:10.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.540+0000 7fa32d59a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa328102d30 0x7fa3281984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:10.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.540+0000 7fa32d59a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3281036e0 0x7fa3281989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:10.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.540+0000 7fa32d59a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3281082a0 0x7fa32819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:10.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.540+0000 7fa32d59a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa3281120a0 con 0x7fa3281082a0 2024-09-12T03:12:10.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.540+0000 7fa32d59a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa328111f20 con 0x7fa3281036e0 2024-09-12T03:12:10.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.540+0000 7fa32d59a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa328112220 con 0x7fa328102d30 2024-09-12T03:12:10.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.540+0000 7fa3277fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3281082a0 0x7fa32819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:10.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.540+0000 7fa3277fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3281082a0 0x7fa32819cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42898/0 (socket says 172.21.15.116:42898) 2024-09-12T03:12:10.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.540+0000 7fa3277fe640 1 -- 172.21.15.116:0/4258533101 learned_addr learned my addr 172.21.15.116:0/4258533101 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:10.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.540+0000 7fa3267fc640 1 --2- 172.21.15.116:0/4258533101 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3281036e0 0x7fa3281989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:10.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.540+0000 7fa326ffd640 1 --2- 172.21.15.116:0/4258533101 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa328102d30 0x7fa3281984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:10.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.540+0000 7fa3277fe640 1 -- 172.21.15.116:0/4258533101 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa328102d30 msgr2=0x7fa3281984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:10.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.540+0000 7fa3277fe640 1 --2- 172.21.15.116:0/4258533101 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa328102d30 0x7fa3281984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.540+0000 7fa3277fe640 1 -- 172.21.15.116:0/4258533101 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3281036e0 msgr2=0x7fa3281989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:10.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.540+0000 7fa3277fe640 1 --2- 172.21.15.116:0/4258533101 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3281036e0 0x7fa3281989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.540+0000 7fa3277fe640 1 -- 172.21.15.116:0/4258533101 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa32819d450 con 0x7fa3281082a0 2024-09-12T03:12:10.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.540+0000 7fa326ffd640 1 --2- 172.21.15.116:0/4258533101 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa328102d30 0x7fa3281984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:12:10.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.540+0000 7fa3277fe640 1 --2- 172.21.15.116:0/4258533101 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3281082a0 0x7fa32819cd70 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fa32000c970 tx=0x7fa32000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:10.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.540+0000 7fa31ffff640 1 -- 172.21.15.116:0/4258533101 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa320007bb0 con 0x7fa3281082a0 2024-09-12T03:12:10.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.540+0000 7fa31ffff640 1 -- 172.21.15.116:0/4258533101 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa320007d50 con 0x7fa3281082a0 2024-09-12T03:12:10.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.540+0000 7fa32d59a640 1 -- 172.21.15.116:0/4258533101 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa32819d740 con 0x7fa3281082a0 2024-09-12T03:12:10.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.540+0000 7fa31ffff640 1 -- 172.21.15.116:0/4258533101 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa320018800 con 0x7fa3281082a0 2024-09-12T03:12:10.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.540+0000 7fa32d59a640 1 -- 172.21.15.116:0/4258533101 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3281a5020 con 0x7fa3281082a0 2024-09-12T03:12:10.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.543+0000 7fa32f973740 1 Processor -- start 2024-09-12T03:12:10.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.543+0000 7fa31ffff640 1 -- 172.21.15.116:0/4258533101 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa32000fa50 con 0x7fa3281082a0 2024-09-12T03:12:10.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.543+0000 7fa32f973740 1 -- start start 2024-09-12T03:12:10.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.544+0000 7fa32f973740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a86b608b40 0x55a86b607090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:10.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.544+0000 7fa32f973740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86b6075d0 0x55a86b607a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:10.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.544+0000 7fa32f973740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a86b612370 0x55a86b614760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:10.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.544+0000 7fa32f973740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a86b56d2f0 con 0x55a86b612370 2024-09-12T03:12:10.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.544+0000 7fa32f973740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a86b514cf0 con 0x55a86b608b40 2024-09-12T03:12:10.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.544+0000 7fa32f973740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a86b5692b0 con 0x55a86b6075d0 2024-09-12T03:12:10.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.544+0000 7fa3267fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a86b612370 0x55a86b614760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:10.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.544+0000 7fa3267fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a86b612370 0x55a86b614760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42912/0 (socket says 172.21.15.116:42912) 2024-09-12T03:12:10.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.544+0000 7fa3267fc640 1 -- 172.21.15.116:0/2074095754 learned_addr learned my addr 172.21.15.116:0/2074095754 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:10.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.544+0000 7fa326ffd640 1 --2- 172.21.15.116:0/2074095754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86b6075d0 0x55a86b607a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:10.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.544+0000 7fa3267fc640 1 --2- 172.21.15.116:0/2074095754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a86b608b40 0x55a86b607090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:10.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.544+0000 7fa31ffff640 1 --2- 172.21.15.116:0/4258533101 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa3140a2b80 0x7fa3140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:10.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.544+0000 7fa3267fc640 1 -- 172.21.15.116:0/2074095754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86b6075d0 msgr2=0x55a86b607a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:10.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.544+0000 7fa3267fc640 1 --2- 172.21.15.116:0/2074095754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86b6075d0 0x55a86b607a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.544+0000 7fa3267fc640 1 -- 172.21.15.116:0/2074095754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a86b608b40 msgr2=0x55a86b607090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:10.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.544+0000 7fa3267fc640 1 --2- 172.21.15.116:0/2074095754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a86b608b40 0x55a86b607090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.544+0000 7fa3267fc640 1 -- 172.21.15.116:0/2074095754 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a86b4f93a0 con 0x55a86b612370 2024-09-12T03:12:10.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.545+0000 7fa3277fe640 1 --2- 172.21.15.116:0/4258533101 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa3140a2b80 0x7fa3140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:10.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.545+0000 7fa3267fc640 1 --2- 172.21.15.116:0/2074095754 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a86b612370 0x55a86b614760 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fa310005e10 tx=0x7fa31000b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d20b783295d5fe0a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:10.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.545+0000 7fa31ffff640 1 -- 172.21.15.116:0/4258533101 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa3200cf0b0 con 0x7fa3281082a0 2024-09-12T03:12:10.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.545+0000 7fa31d7fa640 1 -- 172.21.15.116:0/2074095754 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310018030 con 0x55a86b612370 2024-09-12T03:12:10.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.545+0000 7fa31d7fa640 1 -- 172.21.15.116:0/2074095754 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa310005500 con 0x55a86b612370 2024-09-12T03:12:10.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.545+0000 7fa31d7fa640 1 -- 172.21.15.116:0/2074095754 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310010720 con 0x55a86b612370 2024-09-12T03:12:10.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.546+0000 7fa32f973740 1 -- 172.21.15.116:0/2074095754 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a86b612370 msgr2=0x55a86b614760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:10.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.546+0000 7fa3277fe640 1 --2- 172.21.15.116:0/4258533101 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa3140a2b80 0x7fa3140a5040 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa32009b8d0 tx=0x7fa320086040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:10.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.546+0000 7fa32f973740 1 --2- 172.21.15.116:0/2074095754 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a86b612370 0x55a86b614760 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fa310005e10 tx=0x7fa31000b620 comp rx=0 tx=0).stop 2024-09-12T03:12:10.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.546+0000 7fa32f973740 1 -- 172.21.15.116:0/2074095754 shutdown_connections 2024-09-12T03:12:10.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.546+0000 7fa32f973740 1 --2- 172.21.15.116:0/2074095754 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a86b612370 0x55a86b614760 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.546+0000 7fa32f973740 1 --2- 172.21.15.116:0/2074095754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86b6075d0 0x55a86b607a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.546+0000 7fa32f973740 1 --2- 172.21.15.116:0/2074095754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a86b608b40 0x55a86b607090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.546+0000 7fa32f973740 1 -- 172.21.15.116:0/2074095754 >> 172.21.15.116:0/2074095754 conn(0x55a86b5d0310 msgr2=0x55a86b603c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:10.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.546+0000 7fa32f973740 1 -- 172.21.15.116:0/2074095754 shutdown_connections 2024-09-12T03:12:10.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.546+0000 7fa32f973740 1 -- 172.21.15.116:0/2074095754 wait complete. 2024-09-12T03:12:10.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.546+0000 7fa32f973740 1 -- start start 2024-09-12T03:12:10.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.546+0000 7fa32f973740 1 Processor -- start 2024-09-12T03:12:10.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.547+0000 7fa32f973740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a86b6075d0 0x55a86b611d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:10.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.547+0000 7fa32f973740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a86b608b40 0x55a86b60f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:10.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.547+0000 7fa32f973740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86b612370 0x55a86b60f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:10.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.547+0000 7fa32f973740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a86b615070 con 0x55a86b608b40 2024-09-12T03:12:10.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.547+0000 7fa32f973740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a86b528060 con 0x55a86b6075d0 2024-09-12T03:12:10.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.547+0000 7fa32f973740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a86b6151f0 con 0x55a86b612370 2024-09-12T03:12:10.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.547+0000 7fa32f973740 10 client.0 _ll_register_callbacks cb 0x55a86b27b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:10.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.547+0000 7fa326ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a86b608b40 0x55a86b60f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:10.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.547+0000 7fa326ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a86b608b40 0x55a86b60f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42916/0 (socket says 172.21.15.116:42916) 2024-09-12T03:12:10.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.547+0000 7fa326ffd640 1 -- 172.21.15.116:0/4050252666 learned_addr learned my addr 172.21.15.116:0/4050252666 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:10.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.547+0000 7fa3267fc640 1 --2- 172.21.15.116:0/4050252666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86b612370 0x55a86b60f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:10.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.547+0000 7fa3267fc640 1 --2- 172.21.15.116:0/4050252666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a86b6075d0 0x55a86b611d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:10.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.547+0000 7fa326ffd640 1 -- 172.21.15.116:0/4050252666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86b612370 msgr2=0x55a86b60f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:10.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.547+0000 7fa326ffd640 1 --2- 172.21.15.116:0/4050252666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86b612370 0x55a86b60f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.547+0000 7fa326ffd640 1 -- 172.21.15.116:0/4050252666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a86b6075d0 msgr2=0x55a86b611d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:10.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.547+0000 7fa326ffd640 1 --2- 172.21.15.116:0/4050252666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a86b6075d0 0x55a86b611d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.547+0000 7fa326ffd640 1 -- 172.21.15.116:0/4050252666 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a86b2de3c0 con 0x55a86b608b40 2024-09-12T03:12:10.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.547+0000 7fa3267fc640 1 --2- 172.21.15.116:0/4050252666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a86b6075d0 0x55a86b611d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:12:10.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.547+0000 7fa326ffd640 1 --2- 172.21.15.116:0/4050252666 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a86b608b40 0x55a86b60f020 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fa318035000 tx=0x7fa318031c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471f7f1b2d26de5b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:10.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.547+0000 7fa30e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:12:10.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.547+0000 7fa30e7fc640 1 -- 172.21.15.116:0/4050252666 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa318049060 con 0x55a86b608b40 2024-09-12T03:12:10.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.547+0000 7fa30e7fc640 1 -- 172.21.15.116:0/4050252666 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa318003b80 con 0x55a86b608b40 2024-09-12T03:12:10.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.547+0000 7fa32f973740 1 -- 172.21.15.116:0/4050252666 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa310005500 con 0x55a86b608b40 2024-09-12T03:12:10.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.548+0000 7fa30e7fc640 1 -- 172.21.15.116:0/4050252666 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa318003d80 con 0x55a86b608b40 2024-09-12T03:12:10.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.548+0000 7fa32f973740 10 client.17856 fetch_fsmap learned FSMap version 25 2024-09-12T03:12:10.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.548+0000 7fa32f973740 1 -- 172.21.15.116:0/4050252666 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a86b620b10 con 0x55a86b608b40 2024-09-12T03:12:10.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.548+0000 7fa30e7fc640 1 -- 172.21.15.116:0/4050252666 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa310005500 con 0x55a86b608b40 2024-09-12T03:12:10.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.548+0000 7fa30e7fc640 1 -- 172.21.15.116:0/4050252666 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa318030070 con 0x55a86b608b40 2024-09-12T03:12:10.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.548+0000 7fa32f973740 10 client.17856 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:12:10.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.548+0000 7fa32f973740 10 client.17856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:12:10.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.548+0000 7fa32f973740 20 client.17856 populate_metadata read hostname 'smithi116' 2024-09-12T03:12:10.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.548+0000 7fa32f973740 1 --2- 172.21.15.116:0/4050252666 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a86b622020 0x55a86b624440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:10.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.548+0000 7fa32f973740 4 client.17856 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:12:10.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.548+0000 7fa32f973740 1 -- 172.21.15.116:0/4050252666 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a86b57a860 con 0x55a86b622020 2024-09-12T03:12:10.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.548+0000 7fa3267fc640 1 --2- 172.21.15.116:0/4050252666 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a86b622020 0x55a86b624440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:10.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.549+0000 7fa3267fc640 1 --2- 172.21.15.116:0/4050252666 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a86b622020 0x55a86b624440 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e9ff0325226ffd1 server_cookie=5a7990afdc2f4647 in_seq=0 out_seq=0 2024-09-12T03:12:10.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.549+0000 7fa30e7fc640 10 client.17856 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:10.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.549+0000 7fa30e7fc640 1 -- 172.21.15.116:0/4050252666 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a86b57a860 con 0x55a86b622020 2024-09-12T03:12:10.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.549+0000 7fa30e7fc640 10 client.17856 handle_command_reply: tid=0 2024-09-12T03:12:10.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.549+0000 7fa32f973740 1 client.17856 shutdown 2024-09-12T03:12:10.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.550+0000 7fa32f973740 1 -- 172.21.15.116:0/4050252666 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a86b608b40 msgr2=0x55a86b60f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:10.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.550+0000 7fa32f973740 1 --2- 172.21.15.116:0/4050252666 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a86b608b40 0x55a86b60f020 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fa318035000 tx=0x7fa318031c30 comp rx=0 tx=0).stop 2024-09-12T03:12:10.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.550+0000 7fa32f973740 1 -- 172.21.15.116:0/4050252666 shutdown_connections 2024-09-12T03:12:10.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.550+0000 7fa32f973740 1 --2- 172.21.15.116:0/4050252666 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a86b622020 0x55a86b624440 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.550+0000 7fa32f973740 1 --2- 172.21.15.116:0/4050252666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86b612370 0x55a86b60f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.550+0000 7fa32f973740 1 --2- 172.21.15.116:0/4050252666 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a86b608b40 0x55a86b60f020 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.550+0000 7fa30e7fc640 0 client.17856 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:10.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.550+0000 7fa32f973740 1 --2- 172.21.15.116:0/4050252666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a86b6075d0 0x55a86b611d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.550+0000 7fa32f973740 1 -- 172.21.15.116:0/4050252666 >> 172.21.15.116:0/4050252666 conn(0x55a86b5d0310 msgr2=0x55a86b609960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:10.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.550+0000 7fa32f973740 1 -- 172.21.15.116:0/4050252666 shutdown_connections 2024-09-12T03:12:10.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.550+0000 7fa32f973740 1 -- 172.21.15.116:0/4050252666 wait complete. 2024-09-12T03:12:10.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.550+0000 7fa32f973740 20 client.17856 trim_cache size 0 max 16384 2024-09-12T03:12:10.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.563+0000 7fa32f973740 1 Processor -- start 2024-09-12T03:12:10.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.563+0000 7fa32f973740 1 -- start start 2024-09-12T03:12:10.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.563+0000 7fa32f973740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86b6075d0 0x55a86b60a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:10.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.563+0000 7fa32f973740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a86b608b40 0x55a86b60a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:10.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.563+0000 7fa32f973740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a86b612370 0x55a86b629d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:10.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.563+0000 7fa32f973740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa3281afc40 con 0x55a86b608b40 2024-09-12T03:12:10.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.563+0000 7fa32f973740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa3281ad510 con 0x55a86b612370 2024-09-12T03:12:10.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.563+0000 7fa32f973740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa3281ab360 con 0x55a86b6075d0 2024-09-12T03:12:10.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.563+0000 7fa326ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a86b608b40 0x55a86b60a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:10.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.563+0000 7fa326ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a86b608b40 0x55a86b60a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42920/0 (socket says 172.21.15.116:42920) 2024-09-12T03:12:10.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.563+0000 7fa326ffd640 1 -- 172.21.15.116:0/1281958485 learned_addr learned my addr 172.21.15.116:0/1281958485 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:10.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.563+0000 7fa326ffd640 1 -- 172.21.15.116:0/1281958485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86b6075d0 msgr2=0x55a86b60a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:10.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.563+0000 7fa326ffd640 1 --2- 172.21.15.116:0/1281958485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86b6075d0 0x55a86b60a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.563+0000 7fa326ffd640 1 -- 172.21.15.116:0/1281958485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a86b612370 msgr2=0x55a86b629d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:10.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.563+0000 7fa326ffd640 1 --2- 172.21.15.116:0/1281958485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a86b612370 0x55a86b629d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.563+0000 7fa326ffd640 1 -- 172.21.15.116:0/1281958485 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a86b604960 con 0x55a86b608b40 2024-09-12T03:12:10.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.563+0000 7fa326ffd640 1 --2- 172.21.15.116:0/1281958485 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a86b608b40 0x55a86b60a660 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fa318002230 tx=0x7fa318009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591ab37e6b930010 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:10.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.564+0000 7fa31cff9640 1 -- 172.21.15.116:0/1281958485 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa318054070 con 0x55a86b608b40 2024-09-12T03:12:10.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.564+0000 7fa31cff9640 1 -- 172.21.15.116:0/1281958485 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa318036040 con 0x55a86b608b40 2024-09-12T03:12:10.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.564+0000 7fa31cff9640 1 -- 172.21.15.116:0/1281958485 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa318049040 con 0x55a86b608b40 2024-09-12T03:12:10.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.564+0000 7fa32f973740 1 -- 172.21.15.116:0/1281958485 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a86b608b40 msgr2=0x55a86b60a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:10.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.564+0000 7fa32f973740 1 --2- 172.21.15.116:0/1281958485 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a86b608b40 0x55a86b60a660 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fa318002230 tx=0x7fa318009560 comp rx=0 tx=0).stop 2024-09-12T03:12:10.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.564+0000 7fa32f973740 1 -- 172.21.15.116:0/1281958485 shutdown_connections 2024-09-12T03:12:10.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.564+0000 7fa32f973740 1 --2- 172.21.15.116:0/1281958485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a86b612370 0x55a86b629d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.564+0000 7fa32f973740 1 --2- 172.21.15.116:0/1281958485 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a86b608b40 0x55a86b60a660 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.564+0000 7fa32f973740 1 --2- 172.21.15.116:0/1281958485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86b6075d0 0x55a86b60a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.564+0000 7fa32f973740 1 -- 172.21.15.116:0/1281958485 >> 172.21.15.116:0/1281958485 conn(0x55a86b5d0310 msgr2=0x55a86b6110a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:10.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.564+0000 7fa32f973740 1 -- 172.21.15.116:0/1281958485 shutdown_connections 2024-09-12T03:12:10.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.564+0000 7fa32f973740 1 -- 172.21.15.116:0/1281958485 wait complete. 2024-09-12T03:12:10.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.565+0000 7fa32f973740 1 -- start start 2024-09-12T03:12:10.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.565+0000 7fa32f973740 1 Processor -- start 2024-09-12T03:12:10.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.565+0000 7fa32f973740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a86b6075d0 0x55a86b61a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:10.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.565+0000 7fa32f973740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a86b608b40 0x55a86b61aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:10.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.565+0000 7fa32f973740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86b612370 0x55a86b617490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:10.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.565+0000 7fa32f973740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa3281ab9d0 con 0x55a86b6075d0 2024-09-12T03:12:10.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.565+0000 7fa32f973740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa3281ace40 con 0x55a86b608b40 2024-09-12T03:12:10.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.565+0000 7fa32f973740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa3281ac0a0 con 0x55a86b612370 2024-09-12T03:12:10.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.565+0000 7fa32f973740 10 client.0 _ll_register_callbacks cb 0x7fa318009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:10.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.565+0000 7fa3267fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a86b6075d0 0x55a86b61a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:10.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.565+0000 7fa3267fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a86b6075d0 0x55a86b61a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42926/0 (socket says 172.21.15.116:42926) 2024-09-12T03:12:10.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.565+0000 7fa3267fc640 1 -- 172.21.15.116:0/3313628363 learned_addr learned my addr 172.21.15.116:0/3313628363 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:10.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.565+0000 7fa326ffd640 1 --2- 172.21.15.116:0/3313628363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a86b608b40 0x55a86b61aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:10.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.565+0000 7fa3267fc640 1 --2- 172.21.15.116:0/3313628363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86b612370 0x55a86b617490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:10.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.565+0000 7fa3267fc640 1 -- 172.21.15.116:0/3313628363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86b612370 msgr2=0x55a86b617490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:10.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.565+0000 7fa3267fc640 1 --2- 172.21.15.116:0/3313628363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86b612370 0x55a86b617490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.565+0000 7fa3267fc640 1 -- 172.21.15.116:0/3313628363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a86b608b40 msgr2=0x55a86b61aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:10.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.565+0000 7fa3267fc640 1 --2- 172.21.15.116:0/3313628363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a86b608b40 0x55a86b61aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.565+0000 7fa3267fc640 1 -- 172.21.15.116:0/3313628363 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a86b6034f0 con 0x55a86b6075d0 2024-09-12T03:12:10.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.565+0000 7fa326ffd640 1 --2- 172.21.15.116:0/3313628363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a86b608b40 0x55a86b61aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:12:10.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.566+0000 7fa3267fc640 1 --2- 172.21.15.116:0/3313628363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a86b6075d0 0x55a86b61a660 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fa3100058c0 tx=0x7fa31000b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f7b486f0fbed3e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:10.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.566+0000 7fa30e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:12:10.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.566+0000 7fa30e7fc640 1 -- 172.21.15.116:0/3313628363 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310018040 con 0x55a86b6075d0 2024-09-12T03:12:10.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.566+0000 7fa30e7fc640 1 -- 172.21.15.116:0/3313628363 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa31002ce80 con 0x55a86b6075d0 2024-09-12T03:12:10.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.566+0000 7fa32f973740 1 -- 172.21.15.116:0/3313628363 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa318036040 con 0x55a86b6075d0 2024-09-12T03:12:10.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.567+0000 7fa30e7fc640 1 -- 172.21.15.116:0/3313628363 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa310004800 con 0x55a86b6075d0 2024-09-12T03:12:10.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.567+0000 7fa32f973740 10 client.17862 fetch_fsmap learned FSMap version 25 2024-09-12T03:12:10.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.567+0000 7fa32f973740 1 -- 172.21.15.116:0/3313628363 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a86b68d690 con 0x55a86b6075d0 2024-09-12T03:12:10.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.567+0000 7fa30e7fc640 1 -- 172.21.15.116:0/3313628363 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3100049a0 con 0x55a86b6075d0 2024-09-12T03:12:10.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.567+0000 7fa30e7fc640 1 -- 172.21.15.116:0/3313628363 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa310010720 con 0x55a86b6075d0 2024-09-12T03:12:10.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.567+0000 7fa32f973740 10 client.17862 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:12:10.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.567+0000 7fa32f973740 10 client.17862 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:12:10.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.567+0000 7fa32f973740 20 client.17862 populate_metadata read hostname 'smithi116' 2024-09-12T03:12:10.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.567+0000 7fa32f973740 1 --2- 172.21.15.116:0/3313628363 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a86b68db50 0x55a86b68ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:10.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.567+0000 7fa32f973740 4 client.17862 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:12:10.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.567+0000 7fa32f973740 1 -- 172.21.15.116:0/3313628363 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa318036040 con 0x55a86b68db50 2024-09-12T03:12:10.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.567+0000 7fa326ffd640 1 --2- 172.21.15.116:0/3313628363 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a86b68db50 0x55a86b68ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:10.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.568+0000 7fa30e7fc640 10 client.17862 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:10.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.568+0000 7fa326ffd640 1 --2- 172.21.15.116:0/3313628363 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a86b68db50 0x55a86b68ff40 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c6be1092b58b11e server_cookie=51de3f5ee4ad4909 in_seq=0 out_seq=0 2024-09-12T03:12:10.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.568+0000 7fa30e7fc640 1 -- 172.21.15.116:0/3313628363 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa318036040 con 0x55a86b68db50 2024-09-12T03:12:10.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.568+0000 7fa30e7fc640 10 client.17862 handle_command_reply: tid=0 2024-09-12T03:12:10.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.568+0000 7fa32f973740 1 client.17862 shutdown 2024-09-12T03:12:10.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.568+0000 7fa32f973740 1 -- 172.21.15.116:0/3313628363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a86b6075d0 msgr2=0x55a86b61a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:10.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.568+0000 7fa32f973740 1 --2- 172.21.15.116:0/3313628363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a86b6075d0 0x55a86b61a660 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fa3100058c0 tx=0x7fa31000b750 comp rx=0 tx=0).stop 2024-09-12T03:12:10.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.568+0000 7fa32f973740 1 -- 172.21.15.116:0/3313628363 shutdown_connections 2024-09-12T03:12:10.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.568+0000 7fa32f973740 1 --2- 172.21.15.116:0/3313628363 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a86b68db50 0x55a86b68ff40 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.568+0000 7fa32f973740 1 --2- 172.21.15.116:0/3313628363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86b612370 0x55a86b617490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.568+0000 7fa32f973740 1 --2- 172.21.15.116:0/3313628363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a86b608b40 0x55a86b61aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.568+0000 7fa30e7fc640 0 client.17862 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:10.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.568+0000 7fa32f973740 1 --2- 172.21.15.116:0/3313628363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a86b6075d0 0x55a86b61a660 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.568+0000 7fa32f973740 1 -- 172.21.15.116:0/3313628363 >> 172.21.15.116:0/3313628363 conn(0x55a86b5d0310 msgr2=0x55a86b610880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:10.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.568+0000 7fa32f973740 1 -- 172.21.15.116:0/3313628363 shutdown_connections 2024-09-12T03:12:10.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.568+0000 7fa32f973740 1 -- 172.21.15.116:0/3313628363 wait complete. 2024-09-12T03:12:10.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.568+0000 7fa32f973740 20 client.17862 trim_cache size 0 max 16384 2024-09-12T03:12:10.585 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:12:10.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.569+0000 7fa32d59a640 1 -- 172.21.15.116:0/4258533101 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa3140a2b80 msgr2=0x7fa3140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:10.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.569+0000 7fa32d59a640 1 --2- 172.21.15.116:0/4258533101 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa3140a2b80 0x7fa3140a5040 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa32009b8d0 tx=0x7fa320086040 comp rx=0 tx=0).stop 2024-09-12T03:12:10.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.569+0000 7fa32d59a640 1 -- 172.21.15.116:0/4258533101 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3281082a0 msgr2=0x7fa32819cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:10.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.569+0000 7fa32d59a640 1 --2- 172.21.15.116:0/4258533101 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3281082a0 0x7fa32819cd70 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fa32000c970 tx=0x7fa32000ce30 comp rx=0 tx=0).stop 2024-09-12T03:12:10.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.569+0000 7fa32d59a640 1 -- 172.21.15.116:0/4258533101 shutdown_connections 2024-09-12T03:12:10.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.569+0000 7fa32d59a640 1 --2- 172.21.15.116:0/4258533101 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa3140a2b80 0x7fa3140a5040 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.569+0000 7fa32d59a640 1 --2- 172.21.15.116:0/4258533101 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3281082a0 0x7fa32819cd70 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.569+0000 7fa32d59a640 1 --2- 172.21.15.116:0/4258533101 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3281036e0 0x7fa3281989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.569+0000 7fa32d59a640 1 --2- 172.21.15.116:0/4258533101 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa328102d30 0x7fa3281984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:10.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.569+0000 7fa32d59a640 1 -- 172.21.15.116:0/4258533101 >> 172.21.15.116:0/4258533101 conn(0x7fa3280fc7d0 msgr2=0x7fa3280fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:10.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.569+0000 7fa32d59a640 1 -- 172.21.15.116:0/4258533101 shutdown_connections 2024-09-12T03:12:10.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:10.569+0000 7fa32d59a640 1 -- 172.21.15.116:0/4258533101 wait complete. 2024-09-12T03:12:10.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:10 smithi183 ceph-mon[35535]: pgmap v613: 97 pgs: 97 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:12:10.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:10 smithi177 ceph-mon[35305]: pgmap v613: 97 pgs: 97 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:12:11.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:11 smithi183 ceph-mon[35535]: pgmap v614: 97 pgs: 97 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T03:12:11.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:11 smithi177 ceph-mon[35305]: pgmap v614: 97 pgs: 97 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T03:12:11.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:11 smithi116 ceph-mon[31410]: pgmap v614: 97 pgs: 97 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T03:12:12.354 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:12:12.354 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:12:12.612 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:12:13.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.101+0000 7f013492a640 1 -- 172.21.15.116:0/2229993903 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f013010c170 msgr2=0x7f013010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:13.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.101+0000 7f013492a640 1 --2- 172.21.15.116:0/2229993903 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f013010c170 0x7f013010e610 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f0128009ca0 tx=0x7f012802a950 comp rx=0 tx=0).stop 2024-09-12T03:12:13.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.101+0000 7f013492a640 1 -- 172.21.15.116:0/2229993903 shutdown_connections 2024-09-12T03:12:13.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.101+0000 7f013492a640 1 --2- 172.21.15.116:0/2229993903 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f013010c170 0x7f013010e610 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.101+0000 7f013492a640 1 --2- 172.21.15.116:0/2229993903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0130103660 0x7f013010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.101+0000 7f013492a640 1 --2- 172.21.15.116:0/2229993903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0130102d40 0x7f0130103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.101+0000 7f013492a640 1 -- 172.21.15.116:0/2229993903 >> 172.21.15.116:0/2229993903 conn(0x7f01300fc7a0 msgr2=0x7f01300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:13.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.101+0000 7f013492a640 1 -- 172.21.15.116:0/2229993903 shutdown_connections 2024-09-12T03:12:13.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.101+0000 7f013492a640 1 -- 172.21.15.116:0/2229993903 wait complete. 2024-09-12T03:12:13.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.102+0000 7f013492a640 1 Processor -- start 2024-09-12T03:12:13.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.102+0000 7f013492a640 1 -- start start 2024-09-12T03:12:13.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.102+0000 7f013492a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0130102d40 0x7f0130198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:13.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.102+0000 7f013492a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0130103660 0x7f0130198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:13.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.102+0000 7f013492a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f013010c170 0x7f013019cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:13.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.102+0000 7f013492a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0130112030 con 0x7f0130103660 2024-09-12T03:12:13.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.102+0000 7f013492a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0130111eb0 con 0x7f0130102d40 2024-09-12T03:12:13.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.102+0000 7f013492a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f01301121b0 con 0x7f013010c170 2024-09-12T03:12:13.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.102+0000 7f012dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0130103660 0x7f0130198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:13.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.102+0000 7f012dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0130103660 0x7f0130198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53884/0 (socket says 172.21.15.116:53884) 2024-09-12T03:12:13.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.102+0000 7f012dd9b640 1 -- 172.21.15.116:0/2323054158 learned_addr learned my addr 172.21.15.116:0/2323054158 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:13.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.102+0000 7f012ed9d640 1 --2- 172.21.15.116:0/2323054158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f013010c170 0x7f013019cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:13.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.103+0000 7f012e59c640 1 --2- 172.21.15.116:0/2323054158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0130102d40 0x7f0130198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:13.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.103+0000 7f012dd9b640 1 -- 172.21.15.116:0/2323054158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f013010c170 msgr2=0x7f013019cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:13.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.103+0000 7f012dd9b640 1 --2- 172.21.15.116:0/2323054158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f013010c170 0x7f013019cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.103+0000 7f012dd9b640 1 -- 172.21.15.116:0/2323054158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0130102d40 msgr2=0x7f0130198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:13.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.103+0000 7f012dd9b640 1 --2- 172.21.15.116:0/2323054158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0130102d40 0x7f0130198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.103+0000 7f012dd9b640 1 -- 172.21.15.116:0/2323054158 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013019d5f0 con 0x7f0130103660 2024-09-12T03:12:13.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.103+0000 7f012dd9b640 1 --2- 172.21.15.116:0/2323054158 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0130103660 0x7f0130198b80 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f012000b3f0 tx=0x7f012000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:13.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.103+0000 7f011f7fe640 1 -- 172.21.15.116:0/2323054158 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0120015020 con 0x7f0130103660 2024-09-12T03:12:13.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.103+0000 7f013492a640 1 -- 172.21.15.116:0/2323054158 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f013019d8e0 con 0x7f0130103660 2024-09-12T03:12:13.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.103+0000 7f011f7fe640 1 -- 172.21.15.116:0/2323054158 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0120011d10 con 0x7f0130103660 2024-09-12T03:12:13.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.103+0000 7f011f7fe640 1 -- 172.21.15.116:0/2323054158 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012000f2e0 con 0x7f0130103660 2024-09-12T03:12:13.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.103+0000 7f013492a640 1 -- 172.21.15.116:0/2323054158 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01301a51c0 con 0x7f0130103660 2024-09-12T03:12:13.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.105+0000 7f011f7fe640 1 -- 172.21.15.116:0/2323054158 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f0120018020 con 0x7f0130103660 2024-09-12T03:12:13.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.105+0000 7f0136d03740 1 Processor -- start 2024-09-12T03:12:13.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.106+0000 7f0136d03740 1 -- start start 2024-09-12T03:12:13.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.106+0000 7f0136d03740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555ab1314a40 0x555ab1314e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:13.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.106+0000 7f0136d03740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ab1315360 0x555ab1323c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:13.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.106+0000 7f0136d03740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ab1324150 0x555ab1326540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:13.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.106+0000 7f0136d03740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555ab127a280 con 0x555ab1314a40 2024-09-12T03:12:13.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.106+0000 7f0136d03740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x555ab1221b10 con 0x555ab1324150 2024-09-12T03:12:13.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.106+0000 7f0136d03740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555ab1276240 con 0x555ab1315360 2024-09-12T03:12:13.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.106+0000 7f011f7fe640 1 --2- 172.21.15.116:0/2323054158 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f01180a2b80 0x7f01180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:13.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.106+0000 7f012ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555ab1314a40 0x555ab1314e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:13.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.106+0000 7f012e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ab1315360 0x555ab1323c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:13.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.106+0000 7f012ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555ab1314a40 0x555ab1314e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53886/0 (socket says 172.21.15.116:53886) 2024-09-12T03:12:13.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.106+0000 7f012e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ab1315360 0x555ab1323c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.116:46906/0 (socket says 172.21.15.116:46906) 2024-09-12T03:12:13.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.106+0000 7f012ed9d640 1 -- 172.21.15.116:0/743645516 learned_addr learned my addr 172.21.15.116:0/743645516 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:13.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.106+0000 7f012ed9d640 1 --2- 172.21.15.116:0/2323054158 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f01180a2b80 0x7f01180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:13.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.106+0000 7f011f7fe640 1 -- 172.21.15.116:0/2323054158 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f01200ce920 con 0x7f0130103660 2024-09-12T03:12:13.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.106+0000 7f012dd9b640 1 --2- 172.21.15.116:0/743645516 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ab1324150 0x555ab1326540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:13.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.106+0000 7f012ed9d640 1 -- 172.21.15.116:0/743645516 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ab1315360 msgr2=0x555ab1323c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:13.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.106+0000 7f012ed9d640 1 --2- 172.21.15.116:0/743645516 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ab1315360 0x555ab1323c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.106+0000 7f012ed9d640 1 -- 172.21.15.116:0/743645516 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ab1324150 msgr2=0x555ab1326540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:13.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.106+0000 7f012ed9d640 1 --2- 172.21.15.116:0/743645516 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ab1324150 0x555ab1326540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.106+0000 7f012ed9d640 1 -- 172.21.15.116:0/743645516 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ab122e1f0 con 0x555ab1314a40 2024-09-12T03:12:13.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.107+0000 7f012ed9d640 1 --2- 172.21.15.116:0/743645516 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555ab1314a40 0x555ab1314e20 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f0128030040 tx=0x7f012802f680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb4a9a5350c8eaa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:13.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.107+0000 7f012ed9d640 1 --2- 172.21.15.116:0/2323054158 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f01180a2b80 0x7f01180a5040 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f012802f6f0 tx=0x7f0128048000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:13.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.107+0000 7f011cff9640 1 -- 172.21.15.116:0/743645516 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012804d070 con 0x555ab1314a40 2024-09-12T03:12:13.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.107+0000 7f011cff9640 1 -- 172.21.15.116:0/743645516 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0128004b40 con 0x555ab1314a40 2024-09-12T03:12:13.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.107+0000 7f011cff9640 1 -- 172.21.15.116:0/743645516 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128034040 con 0x555ab1314a40 2024-09-12T03:12:13.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.108+0000 7f0136d03740 1 -- 172.21.15.116:0/743645516 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555ab1314a40 msgr2=0x555ab1314e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:13.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.108+0000 7f0136d03740 1 --2- 172.21.15.116:0/743645516 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555ab1314a40 0x555ab1314e20 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f0128030040 tx=0x7f012802f680 comp rx=0 tx=0).stop 2024-09-12T03:12:13.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.108+0000 7f0136d03740 1 -- 172.21.15.116:0/743645516 shutdown_connections 2024-09-12T03:12:13.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.108+0000 7f0136d03740 1 --2- 172.21.15.116:0/743645516 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ab1324150 0x555ab1326540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.108+0000 7f0136d03740 1 --2- 172.21.15.116:0/743645516 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ab1315360 0x555ab1323c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.108+0000 7f0136d03740 1 --2- 172.21.15.116:0/743645516 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555ab1314a40 0x555ab1314e20 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.108+0000 7f0136d03740 1 -- 172.21.15.116:0/743645516 >> 172.21.15.116:0/743645516 conn(0x555ab13110f0 msgr2=0x555ab13114d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:13.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.108+0000 7f0136d03740 1 -- 172.21.15.116:0/743645516 shutdown_connections 2024-09-12T03:12:13.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.108+0000 7f0136d03740 1 -- 172.21.15.116:0/743645516 wait complete. 2024-09-12T03:12:13.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.108+0000 7f0136d03740 1 -- start start 2024-09-12T03:12:13.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.108+0000 7f0136d03740 1 Processor -- start 2024-09-12T03:12:13.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.109+0000 7f0136d03740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555ab1314a40 0x555ab1321080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:13.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.109+0000 7f0136d03740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ab1315360 0x555ab13215c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:13.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.109+0000 7f0136d03740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ab1324150 0x555ab1321b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:13.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.109+0000 7f0136d03740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555ab1326d20 con 0x555ab1314a40 2024-09-12T03:12:13.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.109+0000 7f0136d03740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x555ab1234f10 con 0x555ab1324150 2024-09-12T03:12:13.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.109+0000 7f0136d03740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555ab1326ea0 con 0x555ab1315360 2024-09-12T03:12:13.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.109+0000 7f0136d03740 10 client.0 _ll_register_callbacks cb 0x555ab11fda40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:13.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.109+0000 7f012e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555ab1314a40 0x555ab1321080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:13.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.109+0000 7f012e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555ab1314a40 0x555ab1321080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53894/0 (socket says 172.21.15.116:53894) 2024-09-12T03:12:13.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.109+0000 7f012e59c640 1 -- 172.21.15.116:0/820321377 learned_addr learned my addr 172.21.15.116:0/820321377 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:13.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.109+0000 7f012dd9b640 1 --2- 172.21.15.116:0/820321377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ab1315360 0x555ab13215c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:13.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.109+0000 7f012ed9d640 1 --2- 172.21.15.116:0/820321377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ab1324150 0x555ab1321b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:13.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.109+0000 7f012dd9b640 1 -- 172.21.15.116:0/820321377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ab1324150 msgr2=0x555ab1321b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:13.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.109+0000 7f012dd9b640 1 --2- 172.21.15.116:0/820321377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ab1324150 0x555ab1321b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.109+0000 7f012dd9b640 1 -- 172.21.15.116:0/820321377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555ab1314a40 msgr2=0x555ab1321080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:13.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.109+0000 7f012dd9b640 1 --2- 172.21.15.116:0/820321377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555ab1314a40 0x555ab1321080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.109+0000 7f012dd9b640 1 -- 172.21.15.116:0/820321377 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ab0feb6a0 con 0x555ab1315360 2024-09-12T03:12:13.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.110+0000 7f012dd9b640 1 --2- 172.21.15.116:0/820321377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ab1315360 0x555ab13215c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f012009b140 tx=0x7f012009b0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8d1e137d2ed8cf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:13.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.110+0000 7f0115ffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T03:12:13.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.110+0000 7f0115ffb640 1 -- 172.21.15.116:0/820321377 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01200b4060 con 0x555ab1315360 2024-09-12T03:12:13.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.110+0000 7f0136d03740 1 -- 172.21.15.116:0/820321377 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0128004b40 con 0x555ab1315360 2024-09-12T03:12:13.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.110+0000 7f0115ffb640 1 -- 172.21.15.116:0/820321377 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f01200994b0 con 0x555ab1315360 2024-09-12T03:12:13.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.110+0000 7f0115ffb640 1 -- 172.21.15.116:0/820321377 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f012009f560 con 0x555ab1315360 2024-09-12T03:12:13.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.110+0000 7f0136d03740 10 client.26548 fetch_fsmap learned FSMap version 25 2024-09-12T03:12:13.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.110+0000 7f0136d03740 1 -- 172.21.15.116:0/820321377 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555ab13439f0 con 0x555ab1315360 2024-09-12T03:12:13.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.110+0000 7f0115ffb640 1 -- 172.21.15.116:0/820321377 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0128004b40 con 0x555ab1315360 2024-09-12T03:12:13.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.110+0000 7f0115ffb640 1 -- 172.21.15.116:0/820321377 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f01200bb0b0 con 0x555ab1315360 2024-09-12T03:12:13.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.110+0000 7f0136d03740 10 client.26548 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:12:13.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.110+0000 7f0136d03740 10 client.26548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:12:13.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.110+0000 7f0136d03740 20 client.26548 populate_metadata read hostname 'smithi116' 2024-09-12T03:12:13.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.110+0000 7f0136d03740 1 --2- 172.21.15.116:0/820321377 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x555ab1344e00 0x555ab13471f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:13.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.110+0000 7f0136d03740 4 client.26548 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:12:13.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.110+0000 7f0136d03740 1 -- 172.21.15.116:0/820321377 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ab12877f0 con 0x555ab1344e00 2024-09-12T03:12:13.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.111+0000 7f012e59c640 1 --2- 172.21.15.116:0/820321377 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x555ab1344e00 0x555ab13471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:13.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.111+0000 7f012e59c640 1 --2- 172.21.15.116:0/820321377 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x555ab1344e00 0x555ab13471f0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed3a6254f335b10b server_cookie=242c0f0d0070a929 in_seq=0 out_seq=0 2024-09-12T03:12:13.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.111+0000 7f0115ffb640 10 client.26548 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:13.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.112+0000 7f0115ffb640 1 -- 172.21.15.116:0/820321377 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555ab12877f0 con 0x555ab1344e00 2024-09-12T03:12:13.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.112+0000 7f0115ffb640 10 client.26548 handle_command_reply: tid=0 2024-09-12T03:12:13.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.112+0000 7f0136d03740 1 client.26548 shutdown 2024-09-12T03:12:13.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.112+0000 7f0136d03740 1 -- 172.21.15.116:0/820321377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ab1315360 msgr2=0x555ab13215c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:13.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.112+0000 7f0136d03740 1 --2- 172.21.15.116:0/820321377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ab1315360 0x555ab13215c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f012009b140 tx=0x7f012009b0b0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.112+0000 7f0136d03740 1 -- 172.21.15.116:0/820321377 shutdown_connections 2024-09-12T03:12:13.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.112+0000 7f0136d03740 1 --2- 172.21.15.116:0/820321377 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x555ab1344e00 0x555ab13471f0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.112+0000 7f0136d03740 1 --2- 172.21.15.116:0/820321377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ab1324150 0x555ab1321b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.112+0000 7f0136d03740 1 --2- 172.21.15.116:0/820321377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ab1315360 0x555ab13215c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.112+0000 7f0115ffb640 0 client.26548 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:13.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.112+0000 7f0136d03740 1 --2- 172.21.15.116:0/820321377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555ab1314a40 0x555ab1321080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.112+0000 7f0136d03740 1 -- 172.21.15.116:0/820321377 >> 172.21.15.116:0/820321377 conn(0x555ab13110f0 msgr2=0x555ab1316e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:13.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.112+0000 7f0136d03740 1 -- 172.21.15.116:0/820321377 shutdown_connections 2024-09-12T03:12:13.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.112+0000 7f0136d03740 1 -- 172.21.15.116:0/820321377 wait complete. 2024-09-12T03:12:13.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.112+0000 7f0136d03740 20 client.26548 trim_cache size 0 max 16384 2024-09-12T03:12:13.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.129+0000 7f0136d03740 1 Processor -- start 2024-09-12T03:12:13.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.129+0000 7f0136d03740 1 -- start start 2024-09-12T03:12:13.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.129+0000 7f0136d03740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ab1314a40 0x555ab134ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:13.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.129+0000 7f0136d03740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ab1315360 0x555ab134cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:13.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.129+0000 7f0136d03740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555ab1324150 0x555ab131e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:13.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.129+0000 7f0136d03740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f01301a5440 con 0x555ab1324150 2024-09-12T03:12:13.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.129+0000 7f0136d03740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0130071d20 con 0x555ab1314a40 2024-09-12T03:12:13.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.129+0000 7f0136d03740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f013006fb90 con 0x555ab1315360 2024-09-12T03:12:13.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.129+0000 7f012ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555ab1324150 0x555ab131e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:13.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.129+0000 7f012ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555ab1324150 0x555ab131e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53898/0 (socket says 172.21.15.116:53898) 2024-09-12T03:12:13.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.129+0000 7f012ed9d640 1 -- 172.21.15.116:0/1751283868 learned_addr learned my addr 172.21.15.116:0/1751283868 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:13.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.130+0000 7f012dd9b640 1 --2- 172.21.15.116:0/1751283868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ab1315360 0x555ab134cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:13.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.130+0000 7f012ed9d640 1 -- 172.21.15.116:0/1751283868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ab1315360 msgr2=0x555ab134cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:13.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.130+0000 7f012e59c640 1 --2- 172.21.15.116:0/1751283868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ab1314a40 0x555ab134ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:13.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.130+0000 7f012ed9d640 1 --2- 172.21.15.116:0/1751283868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ab1315360 0x555ab134cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.130+0000 7f012ed9d640 1 -- 172.21.15.116:0/1751283868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ab1314a40 msgr2=0x555ab134ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:13.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.130+0000 7f012ed9d640 1 --2- 172.21.15.116:0/1751283868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ab1314a40 0x555ab134ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.130+0000 7f012ed9d640 1 -- 172.21.15.116:0/1751283868 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ab1310b90 con 0x555ab1324150 2024-09-12T03:12:13.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.130+0000 7f012dd9b640 1 --2- 172.21.15.116:0/1751283868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ab1315360 0x555ab134cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:12:13.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.130+0000 7f012ed9d640 1 --2- 172.21.15.116:0/1751283868 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555ab1324150 0x555ab131e2e0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f012802b040 tx=0x7f012802f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83bdd95c3b44778 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:13.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.130+0000 7f0117fff640 1 -- 172.21.15.116:0/1751283868 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128046040 con 0x555ab1324150 2024-09-12T03:12:13.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.130+0000 7f0117fff640 1 -- 172.21.15.116:0/1751283868 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0128056070 con 0x555ab1324150 2024-09-12T03:12:13.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.130+0000 7f0117fff640 1 -- 172.21.15.116:0/1751283868 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012804d040 con 0x555ab1324150 2024-09-12T03:12:13.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.130+0000 7f0136d03740 1 -- 172.21.15.116:0/1751283868 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555ab1324150 msgr2=0x555ab131e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:13.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.130+0000 7f0136d03740 1 --2- 172.21.15.116:0/1751283868 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555ab1324150 0x555ab131e2e0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f012802b040 tx=0x7f012802f8a0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.131+0000 7f0136d03740 1 -- 172.21.15.116:0/1751283868 shutdown_connections 2024-09-12T03:12:13.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.131+0000 7f0136d03740 1 --2- 172.21.15.116:0/1751283868 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555ab1324150 0x555ab131e2e0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.131+0000 7f0136d03740 1 --2- 172.21.15.116:0/1751283868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ab1315360 0x555ab134cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.131+0000 7f0136d03740 1 --2- 172.21.15.116:0/1751283868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ab1314a40 0x555ab134ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.131+0000 7f0136d03740 1 -- 172.21.15.116:0/1751283868 >> 172.21.15.116:0/1751283868 conn(0x555ab13110f0 msgr2=0x555ab1322ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:13.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.131+0000 7f0136d03740 1 -- 172.21.15.116:0/1751283868 shutdown_connections 2024-09-12T03:12:13.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.131+0000 7f0136d03740 1 -- 172.21.15.116:0/1751283868 wait complete. 2024-09-12T03:12:13.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.131+0000 7f0136d03740 1 -- start start 2024-09-12T03:12:13.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.131+0000 7f0136d03740 1 Processor -- start 2024-09-12T03:12:13.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.131+0000 7f0136d03740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ab1314a40 0x555ab13285b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:13.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.131+0000 7f0136d03740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ab1315360 0x555ab1328af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:13.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.131+0000 7f0136d03740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555ab1324150 0x555ab13aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:13.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.131+0000 7f0136d03740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f01300701e0 con 0x555ab1324150 2024-09-12T03:12:13.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.131+0000 7f0136d03740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0130071650 con 0x555ab1315360 2024-09-12T03:12:13.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.131+0000 7f0136d03740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f01300708b0 con 0x555ab1314a40 2024-09-12T03:12:13.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.131+0000 7f0136d03740 10 client.0 _ll_register_callbacks cb 0x7f0120011ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:13.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.131+0000 7f012ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555ab1324150 0x555ab13aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:13.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.131+0000 7f012ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555ab1324150 0x555ab13aea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53910/0 (socket says 172.21.15.116:53910) 2024-09-12T03:12:13.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.131+0000 7f012ed9d640 1 -- 172.21.15.116:0/2245981833 learned_addr learned my addr 172.21.15.116:0/2245981833 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:13.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.131+0000 7f012dd9b640 1 --2- 172.21.15.116:0/2245981833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ab1315360 0x555ab1328af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:13.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.132+0000 7f012e59c640 1 --2- 172.21.15.116:0/2245981833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ab1314a40 0x555ab13285b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:13.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.132+0000 7f012ed9d640 1 -- 172.21.15.116:0/2245981833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ab1314a40 msgr2=0x555ab13285b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:13.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.132+0000 7f012ed9d640 1 --2- 172.21.15.116:0/2245981833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ab1314a40 0x555ab13285b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.132+0000 7f012ed9d640 1 -- 172.21.15.116:0/2245981833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ab1315360 msgr2=0x555ab1328af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:13.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.132+0000 7f012ed9d640 1 --2- 172.21.15.116:0/2245981833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ab1315360 0x555ab1328af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.132+0000 7f012ed9d640 1 -- 172.21.15.116:0/2245981833 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ab13aef90 con 0x555ab1324150 2024-09-12T03:12:13.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.132+0000 7f012ed9d640 1 --2- 172.21.15.116:0/2245981833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555ab1324150 0x555ab13aea50 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f012802fed0 tx=0x7f0128003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8f1c0910baef6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:13.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.132+0000 7f0115ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:12:13.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.132+0000 7f0115ffb640 1 -- 172.21.15.116:0/2245981833 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012804d040 con 0x555ab1324150 2024-09-12T03:12:13.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.132+0000 7f0115ffb640 1 -- 172.21.15.116:0/2245981833 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f0128004240 con 0x555ab1324150 2024-09-12T03:12:13.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.132+0000 7f0136d03740 1 -- 172.21.15.116:0/2245981833 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0128056070 con 0x555ab1324150 2024-09-12T03:12:13.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.132+0000 7f0115ffb640 1 -- 172.21.15.116:0/2245981833 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0128004440 con 0x555ab1324150 2024-09-12T03:12:13.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.132+0000 7f0136d03740 10 client.17880 fetch_fsmap learned FSMap version 25 2024-09-12T03:12:13.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.132+0000 7f0136d03740 1 -- 172.21.15.116:0/2245981833 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555ab13b01c0 con 0x555ab1324150 2024-09-12T03:12:13.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.132+0000 7f0115ffb640 1 -- 172.21.15.116:0/2245981833 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0128056070 con 0x555ab1324150 2024-09-12T03:12:13.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.133+0000 7f0115ffb640 1 -- 172.21.15.116:0/2245981833 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f01280524e0 con 0x555ab1324150 2024-09-12T03:12:13.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.133+0000 7f0136d03740 10 client.17880 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:12:13.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.133+0000 7f0136d03740 10 client.17880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:12:13.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.133+0000 7f0136d03740 20 client.17880 populate_metadata read hostname 'smithi116' 2024-09-12T03:12:13.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.133+0000 7f0136d03740 1 --2- 172.21.15.116:0/2245981833 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x555ab13b0440 0x555ab13b2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:13.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.133+0000 7f0136d03740 4 client.17880 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:12:13.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.133+0000 7f0136d03740 1 -- 172.21.15.116:0/2245981833 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555ab1281d00 con 0x555ab13b0440 2024-09-12T03:12:13.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.133+0000 7f012e59c640 1 --2- 172.21.15.116:0/2245981833 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x555ab13b0440 0x555ab13b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:13.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.133+0000 7f0115ffb640 10 client.17880 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:13.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.133+0000 7f012e59c640 1 --2- 172.21.15.116:0/2245981833 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x555ab13b0440 0x555ab13b2830 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4283e04ab5e63fdc server_cookie=21cd1dfdbba409bf in_seq=0 out_seq=0 2024-09-12T03:12:13.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.134+0000 7f0115ffb640 1 -- 172.21.15.116:0/2245981833 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555ab1281d00 con 0x555ab13b0440 2024-09-12T03:12:13.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.134+0000 7f0115ffb640 10 client.17880 handle_command_reply: tid=0 2024-09-12T03:12:13.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.134+0000 7f0136d03740 1 client.17880 shutdown 2024-09-12T03:12:13.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.134+0000 7f0136d03740 1 -- 172.21.15.116:0/2245981833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555ab1324150 msgr2=0x555ab13aea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:13.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.134+0000 7f0136d03740 1 --2- 172.21.15.116:0/2245981833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555ab1324150 0x555ab13aea50 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f012802fed0 tx=0x7f0128003f90 comp rx=0 tx=0).stop 2024-09-12T03:12:13.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.134+0000 7f0136d03740 1 -- 172.21.15.116:0/2245981833 shutdown_connections 2024-09-12T03:12:13.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.134+0000 7f0136d03740 1 --2- 172.21.15.116:0/2245981833 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x555ab13b0440 0x555ab13b2830 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.134+0000 7f0136d03740 1 --2- 172.21.15.116:0/2245981833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555ab1324150 0x555ab13aea50 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.134+0000 7f0136d03740 1 --2- 172.21.15.116:0/2245981833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ab1315360 0x555ab1328af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.134+0000 7f0115ffb640 0 client.17880 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:13.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.134+0000 7f0136d03740 1 --2- 172.21.15.116:0/2245981833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ab1314a40 0x555ab13285b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.134+0000 7f0136d03740 1 -- 172.21.15.116:0/2245981833 >> 172.21.15.116:0/2245981833 conn(0x555ab13110f0 msgr2=0x555ab130fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:13.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.135+0000 7f0136d03740 1 -- 172.21.15.116:0/2245981833 shutdown_connections 2024-09-12T03:12:13.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.135+0000 7f0136d03740 1 -- 172.21.15.116:0/2245981833 wait complete. 2024-09-12T03:12:13.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.135+0000 7f0136d03740 20 client.17880 trim_cache size 0 max 16384 2024-09-12T03:12:13.140 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:12:13.140 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:12:13.140 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "a640eb4d-dab1-4298-a5b9-355820145027", 2024-09-12T03:12:13.140 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:12:13.140 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:12:13.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.135+0000 7f013492a640 1 -- 172.21.15.116:0/2323054158 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f01180a2b80 msgr2=0x7f01180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:13.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.135+0000 7f013492a640 1 --2- 172.21.15.116:0/2323054158 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f01180a2b80 0x7f01180a5040 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f012802f6f0 tx=0x7f0128048000 comp rx=0 tx=0).stop 2024-09-12T03:12:13.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.135+0000 7f013492a640 1 -- 172.21.15.116:0/2323054158 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0130103660 msgr2=0x7f0130198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:13.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.135+0000 7f013492a640 1 --2- 172.21.15.116:0/2323054158 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0130103660 0x7f0130198b80 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f012000b3f0 tx=0x7f012000b8b0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.135+0000 7f013492a640 1 -- 172.21.15.116:0/2323054158 shutdown_connections 2024-09-12T03:12:13.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.135+0000 7f013492a640 1 --2- 172.21.15.116:0/2323054158 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f01180a2b80 0x7f01180a5040 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.135+0000 7f013492a640 1 --2- 172.21.15.116:0/2323054158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f013010c170 0x7f013019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.135+0000 7f013492a640 1 --2- 172.21.15.116:0/2323054158 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0130103660 0x7f0130198b80 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.135+0000 7f013492a640 1 --2- 172.21.15.116:0/2323054158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0130102d40 0x7f0130198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:13.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.135+0000 7f013492a640 1 -- 172.21.15.116:0/2323054158 >> 172.21.15.116:0/2323054158 conn(0x7f01300fc7a0 msgr2=0x7f01300fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:13.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.135+0000 7f013492a640 1 -- 172.21.15.116:0/2323054158 shutdown_connections 2024-09-12T03:12:13.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:13.135+0000 7f013492a640 1 -- 172.21.15.116:0/2323054158 wait complete. 2024-09-12T03:12:13.913 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:12:14.162 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:12:14.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:14 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:12:14.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:14 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:12:14.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:14 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:12:14.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:14 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:12:14.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:14 smithi116 ceph-mon[31410]: pgmap v615: 97 pgs: 97 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:12:14.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:14 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:12:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:14 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:12:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:14 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:12:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:14 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:12:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:14 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:12:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:14 smithi183 ceph-mon[35535]: pgmap v615: 97 pgs: 97 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:12:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:14 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:12:14.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.683+0000 7f6985619640 1 -- 172.21.15.116:0/323377063 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6980102d50 msgr2=0x7f6980103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:14.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.683+0000 7f6985619640 1 --2- 172.21.15.116:0/323377063 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6980102d50 0x7f6980103130 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f6970009ca0 tx=0x7f697002a950 comp rx=0 tx=0).stop 2024-09-12T03:12:14.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.683+0000 7f6985619640 1 -- 172.21.15.116:0/323377063 shutdown_connections 2024-09-12T03:12:14.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.683+0000 7f6985619640 1 --2- 172.21.15.116:0/323377063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f698010c180 0x7f698010e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.683+0000 7f6985619640 1 --2- 172.21.15.116:0/323377063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6980103670 0x7f698010bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.683+0000 7f6985619640 1 --2- 172.21.15.116:0/323377063 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6980102d50 0x7f6980103130 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.683+0000 7f6985619640 1 -- 172.21.15.116:0/323377063 >> 172.21.15.116:0/323377063 conn(0x7f69800fc7d0 msgr2=0x7f69800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:14.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.683+0000 7f6985619640 1 -- 172.21.15.116:0/323377063 shutdown_connections 2024-09-12T03:12:14.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.683+0000 7f6985619640 1 -- 172.21.15.116:0/323377063 wait complete. 2024-09-12T03:12:14.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.684+0000 7f6985619640 1 Processor -- start 2024-09-12T03:12:14.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.684+0000 7f6985619640 1 -- start start 2024-09-12T03:12:14.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.684+0000 7f6985619640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6980102d50 0x7f69801009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:14.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.684+0000 7f6985619640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6980103670 0x7f6980100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:14.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.684+0000 7f6985619640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f698010c180 0x7f6980101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:14.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.684+0000 7f6985619640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6980111f80 con 0x7f698010c180 2024-09-12T03:12:14.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.684+0000 7f6985619640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6980111e00 con 0x7f6980103670 2024-09-12T03:12:14.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.684+0000 7f6985619640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6980112100 con 0x7f6980102d50 2024-09-12T03:12:14.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.684+0000 7f697f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f698010c180 0x7f6980101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:14.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.684+0000 7f697f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f698010c180 0x7f6980101430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53934/0 (socket says 172.21.15.116:53934) 2024-09-12T03:12:14.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.684+0000 7f697f7fe640 1 -- 172.21.15.116:0/167827270 learned_addr learned my addr 172.21.15.116:0/167827270 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:14.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.684+0000 7f697effd640 1 --2- 172.21.15.116:0/167827270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6980102d50 0x7f69801009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:14.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.684+0000 7f697e7fc640 1 --2- 172.21.15.116:0/167827270 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6980103670 0x7f6980100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:14.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.685+0000 7f697f7fe640 1 -- 172.21.15.116:0/167827270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6980102d50 msgr2=0x7f69801009b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:14.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.685+0000 7f697f7fe640 1 --2- 172.21.15.116:0/167827270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6980102d50 0x7f69801009b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.685+0000 7f697f7fe640 1 -- 172.21.15.116:0/167827270 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6980103670 msgr2=0x7f6980100ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:14.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.685+0000 7f697f7fe640 1 --2- 172.21.15.116:0/167827270 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6980103670 0x7f6980100ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.685+0000 7f697f7fe640 1 -- 172.21.15.116:0/167827270 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69801a68c0 con 0x7f698010c180 2024-09-12T03:12:14.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.685+0000 7f697e7fc640 1 --2- 172.21.15.116:0/167827270 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6980103670 0x7f6980100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:12:14.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.685+0000 7f697f7fe640 1 --2- 172.21.15.116:0/167827270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f698010c180 0x7f6980101430 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f697800c970 tx=0x7f697800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:14.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.685+0000 7f6967fff640 1 -- 172.21.15.116:0/167827270 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6978007bb0 con 0x7f698010c180 2024-09-12T03:12:14.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.685+0000 7f6967fff640 1 -- 172.21.15.116:0/167827270 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6978007d50 con 0x7f698010c180 2024-09-12T03:12:14.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.685+0000 7f6985619640 1 -- 172.21.15.116:0/167827270 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69801a6bb0 con 0x7f698010c180 2024-09-12T03:12:14.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.685+0000 7f6967fff640 1 -- 172.21.15.116:0/167827270 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697800f450 con 0x7f698010c180 2024-09-12T03:12:14.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.685+0000 7f6985619640 1 -- 172.21.15.116:0/167827270 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69801a70f0 con 0x7f698010c180 2024-09-12T03:12:14.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.687+0000 7f6967fff640 1 -- 172.21.15.116:0/167827270 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6978004f20 con 0x7f698010c180 2024-09-12T03:12:14.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.688+0000 7f69879f2740 1 Processor -- start 2024-09-12T03:12:14.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.688+0000 7f69879f2740 1 -- start start 2024-09-12T03:12:14.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.688+0000 7f69879f2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f01b464a0 0x557f01b46880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:14.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.688+0000 7f69879f2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557f01b46dc0 0x557f01b51140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:14.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.688+0000 7f69879f2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f01b51680 0x557f01b53a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:14.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.688+0000 7f69879f2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557f01aac2e0 con 0x557f01b46dc0 2024-09-12T03:12:14.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.688+0000 7f69879f2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557f01a53ce0 con 0x557f01b464a0 2024-09-12T03:12:14.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.688+0000 7f69879f2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557f01aa82a0 con 0x557f01b51680 2024-09-12T03:12:14.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.688+0000 7f697effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557f01b46dc0 0x557f01b51140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:14.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.688+0000 7f697effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557f01b46dc0 0x557f01b51140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53950/0 (socket says 172.21.15.116:53950) 2024-09-12T03:12:14.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.688+0000 7f697effd640 1 -- 172.21.15.116:0/3096480154 learned_addr learned my addr 172.21.15.116:0/3096480154 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:14.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.688+0000 7f6967fff640 1 --2- 172.21.15.116:0/167827270 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f69680a2b80 0x7f69680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:14.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.688+0000 7f697f7fe640 1 --2- 172.21.15.116:0/167827270 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f69680a2b80 0x7f69680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:14.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.688+0000 7f697e7fc640 1 --2- 172.21.15.116:0/3096480154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f01b464a0 0x557f01b46880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:14.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.689+0000 7f697e7fc640 1 --2- 172.21.15.116:0/3096480154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f01b51680 0x557f01b53a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:14.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.689+0000 7f697effd640 1 -- 172.21.15.116:0/3096480154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f01b51680 msgr2=0x557f01b53a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:14.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.689+0000 7f697effd640 1 --2- 172.21.15.116:0/3096480154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f01b51680 0x557f01b53a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.689+0000 7f6967fff640 1 -- 172.21.15.116:0/167827270 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f69780cec80 con 0x7f698010c180 2024-09-12T03:12:14.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.689+0000 7f697effd640 1 -- 172.21.15.116:0/3096480154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f01b464a0 msgr2=0x557f01b46880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:14.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.689+0000 7f697effd640 1 --2- 172.21.15.116:0/3096480154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f01b464a0 0x557f01b46880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.689+0000 7f697effd640 1 -- 172.21.15.116:0/3096480154 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f01a38440 con 0x557f01b46dc0 2024-09-12T03:12:14.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.689+0000 7f697e7fc640 1 --2- 172.21.15.116:0/3096480154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f01b51680 0x557f01b53a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:12:14.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.689+0000 7f697effd640 1 --2- 172.21.15.116:0/3096480154 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557f01b46dc0 0x557f01b51140 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f6970035000 tx=0x7f697002f940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6cafe00b6bdb20 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:14.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.689+0000 7f69657fa640 1 -- 172.21.15.116:0/3096480154 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970048070 con 0x557f01b46dc0 2024-09-12T03:12:14.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.689+0000 7f697f7fe640 1 --2- 172.21.15.116:0/167827270 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f69680a2b80 0x7f69680a5040 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f697809b440 tx=0x7f6978066130 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:14.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.689+0000 7f69657fa640 1 -- 172.21.15.116:0/3096480154 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f697002fba0 con 0x557f01b46dc0 2024-09-12T03:12:14.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.689+0000 7f69657fa640 1 -- 172.21.15.116:0/3096480154 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970034040 con 0x557f01b46dc0 2024-09-12T03:12:14.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.690+0000 7f69879f2740 1 -- 172.21.15.116:0/3096480154 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557f01b46dc0 msgr2=0x557f01b51140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:14.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.690+0000 7f69879f2740 1 --2- 172.21.15.116:0/3096480154 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557f01b46dc0 0x557f01b51140 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f6970035000 tx=0x7f697002f940 comp rx=0 tx=0).stop 2024-09-12T03:12:14.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.690+0000 7f69879f2740 1 -- 172.21.15.116:0/3096480154 shutdown_connections 2024-09-12T03:12:14.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.690+0000 7f69879f2740 1 --2- 172.21.15.116:0/3096480154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f01b51680 0x557f01b53a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.690+0000 7f69879f2740 1 --2- 172.21.15.116:0/3096480154 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557f01b46dc0 0x557f01b51140 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.690+0000 7f69879f2740 1 --2- 172.21.15.116:0/3096480154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f01b464a0 0x557f01b46880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.690+0000 7f69879f2740 1 -- 172.21.15.116:0/3096480154 >> 172.21.15.116:0/3096480154 conn(0x557f01b42a30 msgr2=0x557f01b42e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:14.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.690+0000 7f69879f2740 1 -- 172.21.15.116:0/3096480154 shutdown_connections 2024-09-12T03:12:14.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.690+0000 7f69879f2740 1 -- 172.21.15.116:0/3096480154 wait complete. 2024-09-12T03:12:14.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.690+0000 7f69879f2740 1 -- start start 2024-09-12T03:12:14.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.690+0000 7f69879f2740 1 Processor -- start 2024-09-12T03:12:14.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.690+0000 7f69879f2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557f01b464a0 0x557f01b50f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:14.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.690+0000 7f69879f2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f01b46dc0 0x557f01b4e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:14.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.690+0000 7f69879f2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f01b51680 0x557f01b4e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:14.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.690+0000 7f69879f2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557f01b54250 con 0x557f01b464a0 2024-09-12T03:12:14.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.690+0000 7f69879f2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557f01a67050 con 0x557f01b46dc0 2024-09-12T03:12:14.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.690+0000 7f69879f2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557f01b543d0 con 0x557f01b51680 2024-09-12T03:12:14.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.691+0000 7f69879f2740 10 client.0 _ll_register_callbacks cb 0x557f01aa7160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:14.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.691+0000 7f697e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557f01b464a0 0x557f01b50f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:14.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.691+0000 7f697e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557f01b464a0 0x557f01b50f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53956/0 (socket says 172.21.15.116:53956) 2024-09-12T03:12:14.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.691+0000 7f697e7fc640 1 -- 172.21.15.116:0/184791859 learned_addr learned my addr 172.21.15.116:0/184791859 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:14.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.691+0000 7f697effd640 1 --2- 172.21.15.116:0/184791859 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f01b46dc0 0x557f01b4e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:14.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.691+0000 7f697e7fc640 1 --2- 172.21.15.116:0/184791859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f01b51680 0x557f01b4e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:14.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.691+0000 7f697e7fc640 1 -- 172.21.15.116:0/184791859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f01b51680 msgr2=0x557f01b4e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:14.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.691+0000 7f697e7fc640 1 --2- 172.21.15.116:0/184791859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f01b51680 0x557f01b4e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.691+0000 7f697e7fc640 1 -- 172.21.15.116:0/184791859 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f01b46dc0 msgr2=0x557f01b4e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:14.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.691+0000 7f697e7fc640 1 --2- 172.21.15.116:0/184791859 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f01b46dc0 0x557f01b4e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.691+0000 7f697e7fc640 1 -- 172.21.15.116:0/184791859 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f0181d3c0 con 0x557f01b464a0 2024-09-12T03:12:14.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.691+0000 7f697e7fc640 1 --2- 172.21.15.116:0/184791859 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557f01b464a0 0x557f01b50f40 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f697400be00 tx=0x7f6974004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6d194ff5800035 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:14.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.691+0000 7f6962ff5640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:12:14.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.691+0000 7f6962ff5640 1 -- 172.21.15.116:0/184791859 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974010e40 con 0x557f01b464a0 2024-09-12T03:12:14.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.691+0000 7f6962ff5640 1 -- 172.21.15.116:0/184791859 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f6974004030 con 0x557f01b464a0 2024-09-12T03:12:14.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.691+0000 7f69879f2740 1 -- 172.21.15.116:0/184791859 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f697002fba0 con 0x557f01b464a0 2024-09-12T03:12:14.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.691+0000 7f6962ff5640 1 -- 172.21.15.116:0/184791859 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6974004230 con 0x557f01b464a0 2024-09-12T03:12:14.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.691+0000 7f69879f2740 10 client.17892 fetch_fsmap learned FSMap version 25 2024-09-12T03:12:14.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.691+0000 7f69879f2740 1 -- 172.21.15.116:0/184791859 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557f01b5fba0 con 0x557f01b464a0 2024-09-12T03:12:14.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.692+0000 7f6962ff5640 1 -- 172.21.15.116:0/184791859 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f697002fba0 con 0x557f01b464a0 2024-09-12T03:12:14.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.692+0000 7f6962ff5640 1 -- 172.21.15.116:0/184791859 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f697401e060 con 0x557f01b464a0 2024-09-12T03:12:14.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.692+0000 7f69879f2740 10 client.17892 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:12:14.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.692+0000 7f69879f2740 10 client.17892 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:12:14.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.692+0000 7f69879f2740 20 client.17892 populate_metadata read hostname 'smithi116' 2024-09-12T03:12:14.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.692+0000 7f69879f2740 1 --2- 172.21.15.116:0/184791859 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557f01b61060 0x557f01b63470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:14.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.692+0000 7f69879f2740 4 client.17892 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:12:14.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.692+0000 7f69879f2740 1 -- 172.21.15.116:0/184791859 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f01ab9850 con 0x557f01b61060 2024-09-12T03:12:14.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.692+0000 7f697effd640 1 --2- 172.21.15.116:0/184791859 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557f01b61060 0x557f01b63470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:14.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.693+0000 7f6962ff5640 10 client.17892 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:14.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.693+0000 7f697effd640 1 --2- 172.21.15.116:0/184791859 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557f01b61060 0x557f01b63470 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26008864c1f384d4 server_cookie=317d421de70b2cbc in_seq=0 out_seq=0 2024-09-12T03:12:14.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.693+0000 7f6962ff5640 1 -- 172.21.15.116:0/184791859 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557f01ab9850 con 0x557f01b61060 2024-09-12T03:12:14.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.693+0000 7f6962ff5640 10 client.17892 handle_command_reply: tid=0 2024-09-12T03:12:14.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.693+0000 7f69879f2740 1 client.17892 shutdown 2024-09-12T03:12:14.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.693+0000 7f69879f2740 1 -- 172.21.15.116:0/184791859 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557f01b464a0 msgr2=0x557f01b50f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:14.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.693+0000 7f69879f2740 1 --2- 172.21.15.116:0/184791859 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557f01b464a0 0x557f01b50f40 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f697400be00 tx=0x7f6974004730 comp rx=0 tx=0).stop 2024-09-12T03:12:14.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.693+0000 7f69879f2740 1 -- 172.21.15.116:0/184791859 shutdown_connections 2024-09-12T03:12:14.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.693+0000 7f69879f2740 1 --2- 172.21.15.116:0/184791859 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557f01b61060 0x557f01b63470 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.693+0000 7f69879f2740 1 --2- 172.21.15.116:0/184791859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f01b51680 0x557f01b4e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.693+0000 7f69879f2740 1 --2- 172.21.15.116:0/184791859 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f01b46dc0 0x557f01b4e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.693+0000 7f69879f2740 1 --2- 172.21.15.116:0/184791859 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557f01b464a0 0x557f01b50f40 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.693+0000 7f6962ff5640 0 client.17892 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:14.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.693+0000 7f69879f2740 1 -- 172.21.15.116:0/184791859 >> 172.21.15.116:0/184791859 conn(0x557f01b42a30 msgr2=0x557f01b41b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:14.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.693+0000 7f69879f2740 1 -- 172.21.15.116:0/184791859 shutdown_connections 2024-09-12T03:12:14.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.693+0000 7f69879f2740 1 -- 172.21.15.116:0/184791859 wait complete. 2024-09-12T03:12:14.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.694+0000 7f69879f2740 20 client.17892 trim_cache size 0 max 16384 2024-09-12T03:12:14.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.710+0000 7f69879f2740 1 Processor -- start 2024-09-12T03:12:14.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.710+0000 7f69879f2740 1 -- start start 2024-09-12T03:12:14.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.710+0000 7f69879f2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f01b464a0 0x557f01b5dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:14.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.711+0000 7f69879f2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557f01b46dc0 0x557f01b5d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:14.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.711+0000 7f69879f2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f01b51680 0x557f01b68d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:14.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.711+0000 7f69879f2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f69801b2000 con 0x557f01b46dc0 2024-09-12T03:12:14.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.711+0000 7f69879f2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f69801af8d0 con 0x557f01b51680 2024-09-12T03:12:14.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.711+0000 7f69879f2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f69801ad720 con 0x557f01b464a0 2024-09-12T03:12:14.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.711+0000 7f697effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557f01b46dc0 0x557f01b5d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:14.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.711+0000 7f697effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557f01b46dc0 0x557f01b5d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53958/0 (socket says 172.21.15.116:53958) 2024-09-12T03:12:14.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.711+0000 7f697effd640 1 -- 172.21.15.116:0/3441688151 learned_addr learned my addr 172.21.15.116:0/3441688151 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:14.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.711+0000 7f697effd640 1 -- 172.21.15.116:0/3441688151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f01b464a0 msgr2=0x557f01b5dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:14.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.711+0000 7f697e7fc640 1 --2- 172.21.15.116:0/3441688151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f01b464a0 0x557f01b5dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:14.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.711+0000 7f697effd640 1 --2- 172.21.15.116:0/3441688151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f01b464a0 0x557f01b5dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.711+0000 7f697effd640 1 -- 172.21.15.116:0/3441688151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f01b51680 msgr2=0x557f01b68d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:14.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.711+0000 7f697effd640 1 --2- 172.21.15.116:0/3441688151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f01b51680 0x557f01b68d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.711+0000 7f697effd640 1 -- 172.21.15.116:0/3441688151 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f01b4d1a0 con 0x557f01b46dc0 2024-09-12T03:12:14.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.711+0000 7f697e7fc640 1 --2- 172.21.15.116:0/3441688151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f01b464a0 0x557f01b5dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:12:14.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.711+0000 7f697effd640 1 --2- 172.21.15.116:0/3441688151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557f01b46dc0 0x557f01b5d4e0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f6970006010 tx=0x7f6970003e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794425196cb08a81 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:14.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.711+0000 7f6964ff9640 1 -- 172.21.15.116:0/3441688151 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970048040 con 0x557f01b46dc0 2024-09-12T03:12:14.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.711+0000 7f6964ff9640 1 -- 172.21.15.116:0/3441688151 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6970003a70 con 0x557f01b46dc0 2024-09-12T03:12:14.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.711+0000 7f6964ff9640 1 -- 172.21.15.116:0/3441688151 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697004de80 con 0x557f01b46dc0 2024-09-12T03:12:14.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.712+0000 7f69879f2740 1 -- 172.21.15.116:0/3441688151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557f01b46dc0 msgr2=0x557f01b5d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:14.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.712+0000 7f69879f2740 1 --2- 172.21.15.116:0/3441688151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557f01b46dc0 0x557f01b5d4e0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f6970006010 tx=0x7f6970003e70 comp rx=0 tx=0).stop 2024-09-12T03:12:14.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.712+0000 7f69879f2740 1 -- 172.21.15.116:0/3441688151 shutdown_connections 2024-09-12T03:12:14.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.712+0000 7f69879f2740 1 --2- 172.21.15.116:0/3441688151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f01b51680 0x557f01b68d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.712+0000 7f69879f2740 1 --2- 172.21.15.116:0/3441688151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557f01b46dc0 0x557f01b5d4e0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.712+0000 7f69879f2740 1 --2- 172.21.15.116:0/3441688151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f01b464a0 0x557f01b5dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.712+0000 7f69879f2740 1 -- 172.21.15.116:0/3441688151 >> 172.21.15.116:0/3441688151 conn(0x557f01b42a30 msgr2=0x557f01b497e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:14.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.712+0000 7f69879f2740 1 -- 172.21.15.116:0/3441688151 shutdown_connections 2024-09-12T03:12:14.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.712+0000 7f69879f2740 1 -- 172.21.15.116:0/3441688151 wait complete. 2024-09-12T03:12:14.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.712+0000 7f69879f2740 1 -- start start 2024-09-12T03:12:14.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.712+0000 7f69879f2740 1 Processor -- start 2024-09-12T03:12:14.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.713+0000 7f69879f2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557f01b464a0 0x557f01b553d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:14.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.713+0000 7f69879f2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f01b46dc0 0x557f01b55910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:14.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.713+0000 7f69879f2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f01b51680 0x557f01b55e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:14.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.713+0000 7f69879f2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f69801add90 con 0x557f01b464a0 2024-09-12T03:12:14.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.713+0000 7f69879f2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f69801af200 con 0x557f01b46dc0 2024-09-12T03:12:14.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.713+0000 7f69879f2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f69801ae460 con 0x557f01b51680 2024-09-12T03:12:14.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.713+0000 7f69879f2740 10 client.0 _ll_register_callbacks cb 0x7f698006b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:14.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.713+0000 7f697e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557f01b464a0 0x557f01b553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:14.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.713+0000 7f697e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557f01b464a0 0x557f01b553d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53968/0 (socket says 172.21.15.116:53968) 2024-09-12T03:12:14.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.713+0000 7f697e7fc640 1 -- 172.21.15.116:0/2686412170 learned_addr learned my addr 172.21.15.116:0/2686412170 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:14.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.713+0000 7f697e7fc640 1 --2- 172.21.15.116:0/2686412170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f01b51680 0x557f01b55e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:14.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.713+0000 7f697e7fc640 1 -- 172.21.15.116:0/2686412170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f01b51680 msgr2=0x557f01b55e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:14.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.713+0000 7f697e7fc640 1 --2- 172.21.15.116:0/2686412170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f01b51680 0x557f01b55e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.713+0000 7f697e7fc640 1 -- 172.21.15.116:0/2686412170 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f01b46dc0 msgr2=0x557f01b55910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:14.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.713+0000 7f697e7fc640 1 --2- 172.21.15.116:0/2686412170 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f01b46dc0 0x557f01b55910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.713+0000 7f697e7fc640 1 -- 172.21.15.116:0/2686412170 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f01bcad40 con 0x557f01b464a0 2024-09-12T03:12:14.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.713+0000 7f697e7fc640 1 --2- 172.21.15.116:0/2686412170 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557f01b464a0 0x557f01b553d0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f6974013d30 tx=0x7f697400b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e02b512cf72d9fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:14.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.713+0000 7f6962ff5640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:12:14.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.713+0000 7f6962ff5640 1 -- 172.21.15.116:0/2686412170 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697400e860 con 0x557f01b464a0 2024-09-12T03:12:14.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.713+0000 7f6962ff5640 1 -- 172.21.15.116:0/2686412170 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f697400ea00 con 0x557f01b464a0 2024-09-12T03:12:14.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.713+0000 7f69879f2740 1 -- 172.21.15.116:0/2686412170 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6970003a70 con 0x557f01b464a0 2024-09-12T03:12:14.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.714+0000 7f6962ff5640 1 -- 172.21.15.116:0/2686412170 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f697400b420 con 0x557f01b464a0 2024-09-12T03:12:14.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.714+0000 7f69879f2740 10 client.17898 fetch_fsmap learned FSMap version 25 2024-09-12T03:12:14.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.714+0000 7f69879f2740 1 -- 172.21.15.116:0/2686412170 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557f01bcbf80 con 0x557f01b464a0 2024-09-12T03:12:14.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.714+0000 7f6962ff5640 1 -- 172.21.15.116:0/2686412170 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6970003a70 con 0x557f01b464a0 2024-09-12T03:12:14.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.714+0000 7f6962ff5640 1 -- 172.21.15.116:0/2686412170 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6974011440 con 0x557f01b464a0 2024-09-12T03:12:14.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.714+0000 7f69879f2740 10 client.17898 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:12:14.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.714+0000 7f69879f2740 10 client.17898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:12:14.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.714+0000 7f69879f2740 20 client.17898 populate_metadata read hostname 'smithi116' 2024-09-12T03:12:14.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.714+0000 7f69879f2740 1 --2- 172.21.15.116:0/2686412170 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557f01bcc6b0 0x557f01bceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:14.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.714+0000 7f69879f2740 4 client.17898 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:12:14.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.714+0000 7f69879f2740 1 -- 172.21.15.116:0/2686412170 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557f01ab3d60 con 0x557f01bcc6b0 2024-09-12T03:12:14.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.714+0000 7f697effd640 1 --2- 172.21.15.116:0/2686412170 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557f01bcc6b0 0x557f01bceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:14.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.715+0000 7f697effd640 1 --2- 172.21.15.116:0/2686412170 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557f01bcc6b0 0x557f01bceaf0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f65763b86d4e0ba7 server_cookie=aa2c80bff6c2333f in_seq=0 out_seq=0 2024-09-12T03:12:14.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.715+0000 7f6962ff5640 10 client.17898 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:14.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.715+0000 7f6962ff5640 1 -- 172.21.15.116:0/2686412170 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557f01ab3d60 con 0x557f01bcc6b0 2024-09-12T03:12:14.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.715+0000 7f6962ff5640 10 client.17898 handle_command_reply: tid=0 2024-09-12T03:12:14.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.715+0000 7f69879f2740 1 client.17898 shutdown 2024-09-12T03:12:14.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.715+0000 7f69879f2740 1 -- 172.21.15.116:0/2686412170 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557f01b464a0 msgr2=0x557f01b553d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:14.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.715+0000 7f69879f2740 1 --2- 172.21.15.116:0/2686412170 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557f01b464a0 0x557f01b553d0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f6974013d30 tx=0x7f697400b750 comp rx=0 tx=0).stop 2024-09-12T03:12:14.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.715+0000 7f69879f2740 1 -- 172.21.15.116:0/2686412170 shutdown_connections 2024-09-12T03:12:14.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.715+0000 7f69879f2740 1 --2- 172.21.15.116:0/2686412170 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557f01bcc6b0 0x557f01bceaf0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.715+0000 7f69879f2740 1 --2- 172.21.15.116:0/2686412170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f01b51680 0x557f01b55e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.715+0000 7f69879f2740 1 --2- 172.21.15.116:0/2686412170 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f01b46dc0 0x557f01b55910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.715+0000 7f69879f2740 1 --2- 172.21.15.116:0/2686412170 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557f01b464a0 0x557f01b553d0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.715+0000 7f69879f2740 1 -- 172.21.15.116:0/2686412170 >> 172.21.15.116:0/2686412170 conn(0x557f01b42a30 msgr2=0x557f01b47f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:14.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.715+0000 7f6962ff5640 0 client.17898 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:14.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.715+0000 7f69879f2740 1 -- 172.21.15.116:0/2686412170 shutdown_connections 2024-09-12T03:12:14.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.715+0000 7f69879f2740 1 -- 172.21.15.116:0/2686412170 wait complete. 2024-09-12T03:12:14.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.715+0000 7f69879f2740 20 client.17898 trim_cache size 0 max 16384 2024-09-12T03:12:14.730 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:12:14.730 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:12:14.730 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:12:14.730 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:12:14.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.716+0000 7f6985619640 1 -- 172.21.15.116:0/167827270 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f69680a2b80 msgr2=0x7f69680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:14.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.716+0000 7f6985619640 1 --2- 172.21.15.116:0/167827270 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f69680a2b80 0x7f69680a5040 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f697809b440 tx=0x7f6978066130 comp rx=0 tx=0).stop 2024-09-12T03:12:14.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.716+0000 7f6985619640 1 -- 172.21.15.116:0/167827270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f698010c180 msgr2=0x7f6980101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:14.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.716+0000 7f6985619640 1 --2- 172.21.15.116:0/167827270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f698010c180 0x7f6980101430 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f697800c970 tx=0x7f697800ce30 comp rx=0 tx=0).stop 2024-09-12T03:12:14.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.716+0000 7f6985619640 1 -- 172.21.15.116:0/167827270 shutdown_connections 2024-09-12T03:12:14.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.716+0000 7f6985619640 1 --2- 172.21.15.116:0/167827270 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f69680a2b80 0x7f69680a5040 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.716+0000 7f6985619640 1 --2- 172.21.15.116:0/167827270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f698010c180 0x7f6980101430 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.716+0000 7f6985619640 1 --2- 172.21.15.116:0/167827270 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6980103670 0x7f6980100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.716+0000 7f6985619640 1 --2- 172.21.15.116:0/167827270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6980102d50 0x7f69801009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:14.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.716+0000 7f6985619640 1 -- 172.21.15.116:0/167827270 >> 172.21.15.116:0/167827270 conn(0x7f69800fc7d0 msgr2=0x7f69800fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:14.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.716+0000 7f6985619640 1 -- 172.21.15.116:0/167827270 shutdown_connections 2024-09-12T03:12:14.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:14.716+0000 7f6985619640 1 -- 172.21.15.116:0/167827270 wait complete. 2024-09-12T03:12:14.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:14 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:12:14.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:14 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:12:14.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:14 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:12:14.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:14 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:12:14.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:14 smithi177 ceph-mon[35305]: pgmap v615: 97 pgs: 97 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:12:14.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:14 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:12:15.563 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:12:15.564 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:12:15.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:15 smithi183 ceph-mon[35535]: pgmap v616: 97 pgs: 97 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:12:15.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:15 smithi177 ceph-mon[35305]: pgmap v616: 97 pgs: 97 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:12:15.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:15 smithi116 ceph-mon[31410]: pgmap v616: 97 pgs: 97 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:12:15.813 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:12:16.319 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.298+0000 7f56cb59b640 1 -- 172.21.15.116:0/3553066969 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56c4102d40 msgr2=0x7f56c4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:16.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.298+0000 7f56cb59b640 1 --2- 172.21.15.116:0/3553066969 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56c4102d40 0x7f56c4103120 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f56b8009900 tx=0x7f56b802a480 comp rx=0 tx=0).stop 2024-09-12T03:12:16.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.298+0000 7f56cb59b640 1 -- 172.21.15.116:0/3553066969 shutdown_connections 2024-09-12T03:12:16.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.298+0000 7f56cb59b640 1 --2- 172.21.15.116:0/3553066969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56c410c170 0x7f56c410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:16.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.298+0000 7f56cb59b640 1 --2- 172.21.15.116:0/3553066969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56c4103660 0x7f56c410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:16.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.298+0000 7f56cb59b640 1 --2- 172.21.15.116:0/3553066969 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56c4102d40 0x7f56c4103120 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:16.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.298+0000 7f56cb59b640 1 -- 172.21.15.116:0/3553066969 >> 172.21.15.116:0/3553066969 conn(0x7f56c40fc7a0 msgr2=0x7f56c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:16.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.298+0000 7f56cb59b640 1 -- 172.21.15.116:0/3553066969 shutdown_connections 2024-09-12T03:12:16.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.298+0000 7f56cb59b640 1 -- 172.21.15.116:0/3553066969 wait complete. 2024-09-12T03:12:16.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.299+0000 7f56cb59b640 1 Processor -- start 2024-09-12T03:12:16.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.299+0000 7f56cb59b640 1 -- start start 2024-09-12T03:12:16.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.299+0000 7f56cb59b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56c4102d40 0x7f56c419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:16.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.299+0000 7f56cb59b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56c4103660 0x7f56c419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:16.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.299+0000 7f56cb59b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56c410c170 0x7f56c419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:16.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.299+0000 7f56cb59b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f56c4111fb0 con 0x7f56c4102d40 2024-09-12T03:12:16.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.299+0000 7f56cb59b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f56c4111e30 con 0x7f56c410c170 2024-09-12T03:12:16.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.299+0000 7f56cb59b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f56c4112130 con 0x7f56c4103660 2024-09-12T03:12:16.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.299+0000 7f56c9337640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56c4102d40 0x7f56c419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:16.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.299+0000 7f56c9337640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56c4102d40 0x7f56c419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53984/0 (socket says 172.21.15.116:53984) 2024-09-12T03:12:16.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.299+0000 7f56c9337640 1 -- 172.21.15.116:0/2756274971 learned_addr learned my addr 172.21.15.116:0/2756274971 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:16.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.299+0000 7f56c8b36640 1 --2- 172.21.15.116:0/2756274971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56c4103660 0x7f56c419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:16.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.300+0000 7f56c9337640 1 -- 172.21.15.116:0/2756274971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56c4103660 msgr2=0x7f56c419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:16.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.300+0000 7f56c9b38640 1 --2- 172.21.15.116:0/2756274971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56c410c170 0x7f56c419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:16.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.300+0000 7f56c9337640 1 --2- 172.21.15.116:0/2756274971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56c4103660 0x7f56c419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:16.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.300+0000 7f56c9337640 1 -- 172.21.15.116:0/2756274971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56c410c170 msgr2=0x7f56c419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:16.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.300+0000 7f56c9337640 1 --2- 172.21.15.116:0/2756274971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56c410c170 0x7f56c419eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:16.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.300+0000 7f56c9337640 1 -- 172.21.15.116:0/2756274971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c419f5d0 con 0x7f56c4102d40 2024-09-12T03:12:16.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.300+0000 7f56c8b36640 1 --2- 172.21.15.116:0/2756274971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56c4103660 0x7f56c419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:12:16.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.300+0000 7f56c9337640 1 --2- 172.21.15.116:0/2756274971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56c4102d40 0x7f56c419a620 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f56b8005ab0 tx=0x7f56b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:16.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.300+0000 7f56b67fc640 1 -- 172.21.15.116:0/2756274971 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b803f030 con 0x7f56c4102d40 2024-09-12T03:12:16.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.300+0000 7f56b67fc640 1 -- 172.21.15.116:0/2756274971 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56b8003ce0 con 0x7f56c4102d40 2024-09-12T03:12:16.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.300+0000 7f56cb59b640 1 -- 172.21.15.116:0/2756274971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c419f860 con 0x7f56c4102d40 2024-09-12T03:12:16.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.300+0000 7f56b67fc640 1 -- 172.21.15.116:0/2756274971 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b8035070 con 0x7f56c4102d40 2024-09-12T03:12:16.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.300+0000 7f56cb59b640 1 -- 172.21.15.116:0/2756274971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c41a71a0 con 0x7f56c4102d40 2024-09-12T03:12:16.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.301+0000 7f56cb59b640 1 -- 172.21.15.116:0/2756274971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56c4103120 con 0x7f56c4102d40 2024-09-12T03:12:16.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.302+0000 7f56b67fc640 1 -- 172.21.15.116:0/2756274971 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f56b8042030 con 0x7f56c4102d40 2024-09-12T03:12:16.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.303+0000 7f56b67fc640 1 --2- 172.21.15.116:0/2756274971 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f56b00a2c00 0x7f56b00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:16.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.303+0000 7f56c8b36640 1 --2- 172.21.15.116:0/2756274971 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f56b00a2c00 0x7f56b00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:16.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.303+0000 7f56b67fc640 1 -- 172.21.15.116:0/2756274971 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f56b80efbe0 con 0x7f56c4102d40 2024-09-12T03:12:16.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.303+0000 7f56c8b36640 1 --2- 172.21.15.116:0/2756274971 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f56b00a2c00 0x7f56b00a50c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f56c0006fd0 tx=0x7f56c0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:16.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.306+0000 7f56b67fc640 1 -- 172.21.15.116:0/2756274971 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f56b80bcb70 con 0x7f56c4102d40 2024-09-12T03:12:16.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.508+0000 7f56cb59b640 1 -- 172.21.15.116:0/2756274971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56c419b980 con 0x7f56c4102d40 2024-09-12T03:12:16.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.509+0000 7f56b67fc640 1 -- 172.21.15.116:0/2756274971 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f56b80bc470 con 0x7f56c4102d40 2024-09-12T03:12:16.509 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:12:16.509 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:12:16.510 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:12:16.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.511+0000 7f56cb59b640 1 -- 172.21.15.116:0/2756274971 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f56b00a2c00 msgr2=0x7f56b00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:16.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.511+0000 7f56cb59b640 1 --2- 172.21.15.116:0/2756274971 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f56b00a2c00 0x7f56b00a50c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f56c0006fd0 tx=0x7f56c0008040 comp rx=0 tx=0).stop 2024-09-12T03:12:16.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.511+0000 7f56cb59b640 1 -- 172.21.15.116:0/2756274971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56c4102d40 msgr2=0x7f56c419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:16.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.511+0000 7f56cb59b640 1 --2- 172.21.15.116:0/2756274971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56c4102d40 0x7f56c419a620 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f56b8005ab0 tx=0x7f56b80042f0 comp rx=0 tx=0).stop 2024-09-12T03:12:16.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.511+0000 7f56cb59b640 1 -- 172.21.15.116:0/2756274971 shutdown_connections 2024-09-12T03:12:16.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.511+0000 7f56cb59b640 1 --2- 172.21.15.116:0/2756274971 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f56b00a2c00 0x7f56b00a50c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:16.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.511+0000 7f56cb59b640 1 --2- 172.21.15.116:0/2756274971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56c410c170 0x7f56c419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:16.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.511+0000 7f56cb59b640 1 --2- 172.21.15.116:0/2756274971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56c4103660 0x7f56c419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:16.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.511+0000 7f56cb59b640 1 --2- 172.21.15.116:0/2756274971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56c4102d40 0x7f56c419a620 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:16.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.511+0000 7f56cb59b640 1 -- 172.21.15.116:0/2756274971 >> 172.21.15.116:0/2756274971 conn(0x7f56c40fc7a0 msgr2=0x7f56c410d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:16.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.511+0000 7f56cb59b640 1 -- 172.21.15.116:0/2756274971 shutdown_connections 2024-09-12T03:12:16.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:16.511+0000 7f56cb59b640 1 -- 172.21.15.116:0/2756274971 wait complete. 2024-09-12T03:12:16.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:16 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2756274971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:12:16.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:16 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2756274971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:12:16.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:16 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2756274971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:12:17.339 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:12:17.601 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:12:17.903 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:17 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:12:17.903 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:17 smithi116 ceph-mon[31410]: pgmap v617: 97 pgs: 97 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-12T03:12:17.903 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:17 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:17.903 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:17 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:12:17.904 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:17 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:12:17.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:17 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:12:17.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:17 smithi183 ceph-mon[35535]: pgmap v617: 97 pgs: 97 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-12T03:12:17.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:17 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:17.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:17 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:12:17.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:17 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:12:17.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:17 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:12:17.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:17 smithi177 ceph-mon[35305]: pgmap v617: 97 pgs: 97 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-12T03:12:17.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:17 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:17.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:17 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:12:17.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:17 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:12:18.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.143+0000 7eff7e6d6640 1 -- 172.21.15.116:0/2019441882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff78101de0 msgr2=0x7eff781021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:18.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.143+0000 7eff7e6d6640 1 --2- 172.21.15.116:0/2019441882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff78101de0 0x7eff781021c0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7eff60009900 tx=0x7eff6002a410 comp rx=0 tx=0).stop 2024-09-12T03:12:18.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.143+0000 7eff7e6d6640 1 -- 172.21.15.116:0/2019441882 shutdown_connections 2024-09-12T03:12:18.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.143+0000 7eff7e6d6640 1 --2- 172.21.15.116:0/2019441882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff78106f10 0x7eff7810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.143+0000 7eff7e6d6640 1 --2- 172.21.15.116:0/2019441882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff78102790 0x7eff781067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.143+0000 7eff7e6d6640 1 --2- 172.21.15.116:0/2019441882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff78101de0 0x7eff781021c0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.143+0000 7eff7e6d6640 1 -- 172.21.15.116:0/2019441882 >> 172.21.15.116:0/2019441882 conn(0x7eff780fd8c0 msgr2=0x7eff780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:18.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.144+0000 7eff7e6d6640 1 -- 172.21.15.116:0/2019441882 shutdown_connections 2024-09-12T03:12:18.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.144+0000 7eff7e6d6640 1 -- 172.21.15.116:0/2019441882 wait complete. 2024-09-12T03:12:18.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.144+0000 7eff7e6d6640 1 Processor -- start 2024-09-12T03:12:18.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.144+0000 7eff7e6d6640 1 -- start start 2024-09-12T03:12:18.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.144+0000 7eff7e6d6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff78101de0 0x7eff780744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:18.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.144+0000 7eff7e6d6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff78102790 0x7eff78074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:18.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.145+0000 7eff7e6d6640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff78106f10 0x7eff78074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:18.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.145+0000 7eff7e6d6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7eff78111f30 con 0x7eff78101de0 2024-09-12T03:12:18.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.145+0000 7eff7e6d6640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7eff78111db0 con 0x7eff78106f10 2024-09-12T03:12:18.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.145+0000 7eff7e6d6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff781120b0 con 0x7eff78102790 2024-09-12T03:12:18.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.145+0000 7eff77fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff78101de0 0x7eff780744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:18.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.145+0000 7eff77fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff78101de0 0x7eff780744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53992/0 (socket says 172.21.15.116:53992) 2024-09-12T03:12:18.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.145+0000 7eff77fff640 1 -- 172.21.15.116:0/1365667111 learned_addr learned my addr 172.21.15.116:0/1365667111 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:18.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.145+0000 7eff777fe640 1 --2- 172.21.15.116:0/1365667111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff78102790 0x7eff78074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:18.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.145+0000 7eff7cc73640 1 --2- 172.21.15.116:0/1365667111 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff78106f10 0x7eff78074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:18.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.145+0000 7eff77fff640 1 -- 172.21.15.116:0/1365667111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff78102790 msgr2=0x7eff78074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:18.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.145+0000 7eff77fff640 1 --2- 172.21.15.116:0/1365667111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff78102790 0x7eff78074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.145+0000 7eff77fff640 1 -- 172.21.15.116:0/1365667111 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff78106f10 msgr2=0x7eff78074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:18.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.145+0000 7eff77fff640 1 --2- 172.21.15.116:0/1365667111 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff78106f10 0x7eff78074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.145+0000 7eff77fff640 1 -- 172.21.15.116:0/1365667111 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff781a68f0 con 0x7eff78101de0 2024-09-12T03:12:18.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.145+0000 7eff7cc73640 1 --2- 172.21.15.116:0/1365667111 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff78106f10 0x7eff78074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:12:18.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.145+0000 7eff77fff640 1 --2- 172.21.15.116:0/1365667111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff78101de0 0x7eff780744f0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7eff60006010 tx=0x7eff600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:18.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.145+0000 7eff757fa640 1 -- 172.21.15.116:0/1365667111 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6003f030 con 0x7eff78101de0 2024-09-12T03:12:18.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.145+0000 7eff7e6d6640 1 -- 172.21.15.116:0/1365667111 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff781a6b80 con 0x7eff78101de0 2024-09-12T03:12:18.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.145+0000 7eff757fa640 1 -- 172.21.15.116:0/1365667111 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff60003ce0 con 0x7eff78101de0 2024-09-12T03:12:18.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.145+0000 7eff757fa640 1 -- 172.21.15.116:0/1365667111 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff60035070 con 0x7eff78101de0 2024-09-12T03:12:18.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.145+0000 7eff7e6d6640 1 -- 172.21.15.116:0/1365667111 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff781a7060 con 0x7eff78101de0 2024-09-12T03:12:18.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.147+0000 7eff757fa640 1 -- 172.21.15.116:0/1365667111 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7eff60042030 con 0x7eff78101de0 2024-09-12T03:12:18.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.148+0000 7eff80aaf740 1 Processor -- start 2024-09-12T03:12:18.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.148+0000 7eff80aaf740 1 -- start start 2024-09-12T03:12:18.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.148+0000 7eff80aaf740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582cd293b40 0x5582cd292090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:18.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.148+0000 7eff80aaf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582cd2925d0 0x5582cd292a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:18.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.148+0000 7eff80aaf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582cd29d370 0x5582cd29f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:18.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.148+0000 7eff757fa640 1 --2- 172.21.15.116:0/1365667111 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7eff640a2b80 0x7eff640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:18.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.148+0000 7eff80aaf740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5582cd1f82f0 con 0x5582cd293b40 2024-09-12T03:12:18.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.148+0000 7eff80aaf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5582cd19fcf0 con 0x5582cd29d370 2024-09-12T03:12:18.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.148+0000 7eff80aaf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5582cd1f42b0 con 0x5582cd2925d0 2024-09-12T03:12:18.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.148+0000 7eff7cc73640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582cd293b40 0x5582cd292090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:18.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.148+0000 7eff7cc73640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582cd293b40 0x5582cd292090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54004/0 (socket says 172.21.15.116:54004) 2024-09-12T03:12:18.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.148+0000 7eff7cc73640 1 -- 172.21.15.116:0/3752249006 learned_addr learned my addr 172.21.15.116:0/3752249006 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:18.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.148+0000 7eff7cc73640 1 --2- 172.21.15.116:0/1365667111 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7eff640a2b80 0x7eff640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:18.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.148+0000 7eff757fa640 1 -- 172.21.15.116:0/1365667111 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7eff600ef580 con 0x7eff78101de0 2024-09-12T03:12:18.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.148+0000 7eff77fff640 1 --2- 172.21.15.116:0/3752249006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582cd2925d0 0x5582cd292a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:18.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.149+0000 7eff777fe640 1 --2- 172.21.15.116:0/3752249006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582cd29d370 0x5582cd29f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:18.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.149+0000 7eff7cc73640 1 -- 172.21.15.116:0/3752249006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582cd2925d0 msgr2=0x5582cd292a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:18.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.149+0000 7eff7cc73640 1 --2- 172.21.15.116:0/3752249006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582cd2925d0 0x5582cd292a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.149+0000 7eff7cc73640 1 -- 172.21.15.116:0/3752249006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582cd29d370 msgr2=0x5582cd29f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:18.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.149+0000 7eff7cc73640 1 --2- 172.21.15.116:0/3752249006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582cd29d370 0x5582cd29f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.149+0000 7eff7cc73640 1 -- 172.21.15.116:0/3752249006 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582cd1843a0 con 0x5582cd293b40 2024-09-12T03:12:18.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.149+0000 7eff7cc73640 1 --2- 172.21.15.116:0/3752249006 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582cd293b40 0x5582cd292090 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7eff70013950 tx=0x7eff70013e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f86d3f21b97c56 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:18.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.149+0000 7eff7cc73640 1 --2- 172.21.15.116:0/1365667111 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7eff640a2b80 0x7eff640a5040 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7eff700107a0 tx=0x7eff7001a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:18.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.149+0000 7eff6e7fc640 1 -- 172.21.15.116:0/3752249006 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff70007570 con 0x5582cd293b40 2024-09-12T03:12:18.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.149+0000 7eff6e7fc640 1 -- 172.21.15.116:0/3752249006 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff7000ebe0 con 0x5582cd293b40 2024-09-12T03:12:18.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.149+0000 7eff6e7fc640 1 -- 172.21.15.116:0/3752249006 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7001ccf0 con 0x5582cd293b40 2024-09-12T03:12:18.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.150+0000 7eff80aaf740 1 -- 172.21.15.116:0/3752249006 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582cd293b40 msgr2=0x5582cd292090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:18.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.150+0000 7eff80aaf740 1 --2- 172.21.15.116:0/3752249006 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582cd293b40 0x5582cd292090 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7eff70013950 tx=0x7eff70013e10 comp rx=0 tx=0).stop 2024-09-12T03:12:18.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.150+0000 7eff80aaf740 1 -- 172.21.15.116:0/3752249006 shutdown_connections 2024-09-12T03:12:18.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.150+0000 7eff80aaf740 1 --2- 172.21.15.116:0/3752249006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582cd29d370 0x5582cd29f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.150+0000 7eff80aaf740 1 --2- 172.21.15.116:0/3752249006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582cd2925d0 0x5582cd292a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.150+0000 7eff80aaf740 1 --2- 172.21.15.116:0/3752249006 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582cd293b40 0x5582cd292090 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.150+0000 7eff80aaf740 1 -- 172.21.15.116:0/3752249006 >> 172.21.15.116:0/3752249006 conn(0x5582cd25b310 msgr2=0x5582cd28ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:18.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.150+0000 7eff80aaf740 1 -- 172.21.15.116:0/3752249006 shutdown_connections 2024-09-12T03:12:18.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.150+0000 7eff80aaf740 1 -- 172.21.15.116:0/3752249006 wait complete. 2024-09-12T03:12:18.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.150+0000 7eff80aaf740 1 -- start start 2024-09-12T03:12:18.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.151+0000 7eff80aaf740 1 Processor -- start 2024-09-12T03:12:18.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.151+0000 7eff80aaf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582cd2925d0 0x5582cd29cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:18.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.151+0000 7eff80aaf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582cd293b40 0x5582cd29a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:18.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.151+0000 7eff80aaf740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582cd29d370 0x5582cd29a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:18.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.151+0000 7eff80aaf740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5582cd29ffb0 con 0x5582cd29d370 2024-09-12T03:12:18.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.151+0000 7eff80aaf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5582cd1b3060 con 0x5582cd2925d0 2024-09-12T03:12:18.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.151+0000 7eff80aaf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5582cd2a0130 con 0x5582cd293b40 2024-09-12T03:12:18.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.151+0000 7eff80aaf740 10 client.0 _ll_register_callbacks cb 0x5582ccf068c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:18.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.151+0000 7eff7cc73640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582cd29d370 0x5582cd29a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:18.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.151+0000 7eff7cc73640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582cd29d370 0x5582cd29a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54018/0 (socket says 172.21.15.116:54018) 2024-09-12T03:12:18.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.151+0000 7eff7cc73640 1 -- 172.21.15.116:0/3029833707 learned_addr learned my addr 172.21.15.116:0/3029833707 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:18.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.151+0000 7eff777fe640 1 --2- 172.21.15.116:0/3029833707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582cd293b40 0x5582cd29a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:18.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.151+0000 7eff77fff640 1 --2- 172.21.15.116:0/3029833707 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582cd2925d0 0x5582cd29cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:18.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.151+0000 7eff7cc73640 1 -- 172.21.15.116:0/3029833707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582cd293b40 msgr2=0x5582cd29a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:18.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.151+0000 7eff7cc73640 1 --2- 172.21.15.116:0/3029833707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582cd293b40 0x5582cd29a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.151+0000 7eff7cc73640 1 -- 172.21.15.116:0/3029833707 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582cd2925d0 msgr2=0x5582cd29cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:18.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.151+0000 7eff7cc73640 1 --2- 172.21.15.116:0/3029833707 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582cd2925d0 0x5582cd29cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.151+0000 7eff7cc73640 1 -- 172.21.15.116:0/3029833707 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582ccf693c0 con 0x5582cd29d370 2024-09-12T03:12:18.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.151+0000 7eff7cc73640 1 --2- 172.21.15.116:0/3029833707 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582cd29d370 0x5582cd29a5d0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7eff7001e040 tx=0x7eff7001c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c69010cf1e55a25 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:18.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.151+0000 7eff577fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:12:18.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.152+0000 7eff577fe640 1 -- 172.21.15.116:0/3029833707 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff70029030 con 0x5582cd29d370 2024-09-12T03:12:18.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.152+0000 7eff577fe640 1 -- 172.21.15.116:0/3029833707 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7eff70019050 con 0x5582cd29d370 2024-09-12T03:12:18.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.152+0000 7eff80aaf740 1 -- 172.21.15.116:0/3029833707 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff7000ebe0 con 0x5582cd29d370 2024-09-12T03:12:18.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.152+0000 7eff577fe640 1 -- 172.21.15.116:0/3029833707 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7eff70007570 con 0x5582cd29d370 2024-09-12T03:12:18.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.152+0000 7eff80aaf740 10 client.17916 fetch_fsmap learned FSMap version 25 2024-09-12T03:12:18.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.152+0000 7eff80aaf740 1 -- 172.21.15.116:0/3029833707 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5582cd2aba30 con 0x5582cd29d370 2024-09-12T03:12:18.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.152+0000 7eff577fe640 1 -- 172.21.15.116:0/3029833707 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff7000ebe0 con 0x5582cd29d370 2024-09-12T03:12:18.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.152+0000 7eff577fe640 1 -- 172.21.15.116:0/3029833707 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7eff70025670 con 0x5582cd29d370 2024-09-12T03:12:18.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.152+0000 7eff80aaf740 10 client.17916 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:12:18.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.152+0000 7eff80aaf740 10 client.17916 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:12:18.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.152+0000 7eff80aaf740 20 client.17916 populate_metadata read hostname 'smithi116' 2024-09-12T03:12:18.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.152+0000 7eff80aaf740 1 --2- 172.21.15.116:0/3029833707 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5582cd2ace40 0x5582cd2af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:18.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.152+0000 7eff80aaf740 4 client.17916 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:12:18.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.152+0000 7eff80aaf740 1 -- 172.21.15.116:0/3029833707 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582cd205860 con 0x5582cd2ace40 2024-09-12T03:12:18.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.153+0000 7eff77fff640 1 --2- 172.21.15.116:0/3029833707 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5582cd2ace40 0x5582cd2af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:18.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.153+0000 7eff577fe640 10 client.17916 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:12:18.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.153+0000 7eff77fff640 1 --2- 172.21.15.116:0/3029833707 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5582cd2ace40 0x5582cd2af230 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f35a70ff9ffb341c server_cookie=cfbaa5d08ed24da in_seq=0 out_seq=0 2024-09-12T03:12:18.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.153+0000 7eff577fe640 1 -- 172.21.15.116:0/3029833707 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5582cd205860 con 0x5582cd2ace40 2024-09-12T03:12:18.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.153+0000 7eff577fe640 10 client.17916 handle_command_reply: tid=0 2024-09-12T03:12:18.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.153+0000 7eff80aaf740 1 client.17916 shutdown 2024-09-12T03:12:18.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.154+0000 7eff80aaf740 1 -- 172.21.15.116:0/3029833707 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582cd29d370 msgr2=0x5582cd29a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:18.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.154+0000 7eff80aaf740 1 --2- 172.21.15.116:0/3029833707 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582cd29d370 0x5582cd29a5d0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7eff7001e040 tx=0x7eff7001c650 comp rx=0 tx=0).stop 2024-09-12T03:12:18.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.154+0000 7eff80aaf740 1 -- 172.21.15.116:0/3029833707 shutdown_connections 2024-09-12T03:12:18.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.154+0000 7eff80aaf740 1 --2- 172.21.15.116:0/3029833707 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5582cd2ace40 0x5582cd2af230 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.154+0000 7eff80aaf740 1 --2- 172.21.15.116:0/3029833707 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582cd29d370 0x5582cd29a5d0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.154+0000 7eff80aaf740 1 --2- 172.21.15.116:0/3029833707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582cd293b40 0x5582cd29a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.154+0000 7eff80aaf740 1 --2- 172.21.15.116:0/3029833707 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582cd2925d0 0x5582cd29cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.154+0000 7eff80aaf740 1 -- 172.21.15.116:0/3029833707 >> 172.21.15.116:0/3029833707 conn(0x5582cd25b310 msgr2=0x5582cd294960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:18.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.154+0000 7eff577fe640 0 client.17916 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:12:18.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.154+0000 7eff80aaf740 1 -- 172.21.15.116:0/3029833707 shutdown_connections 2024-09-12T03:12:18.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.154+0000 7eff80aaf740 1 -- 172.21.15.116:0/3029833707 wait complete. 2024-09-12T03:12:18.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.154+0000 7eff80aaf740 20 client.17916 trim_cache size 0 max 16384 2024-09-12T03:12:18.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.167+0000 7eff80aaf740 1 Processor -- start 2024-09-12T03:12:18.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.167+0000 7eff80aaf740 1 -- start start 2024-09-12T03:12:18.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.167+0000 7eff80aaf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582cd2925d0 0x5582cd2a9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:18.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.167+0000 7eff80aaf740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582cd293b40 0x5582cd298fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:18.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.167+0000 7eff80aaf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582cd29d370 0x5582cd2b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:18.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.167+0000 7eff80aaf740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7eff781b1ec0 con 0x5582cd293b40 2024-09-12T03:12:18.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.167+0000 7eff80aaf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7eff781af790 con 0x5582cd29d370 2024-09-12T03:12:18.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.167+0000 7eff80aaf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff781ad600 con 0x5582cd2925d0 2024-09-12T03:12:18.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.167+0000 7eff777fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582cd293b40 0x5582cd298fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:18.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.167+0000 7eff777fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582cd293b40 0x5582cd298fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54020/0 (socket says 172.21.15.116:54020) 2024-09-12T03:12:18.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.167+0000 7eff777fe640 1 -- 172.21.15.116:0/2235588617 learned_addr learned my addr 172.21.15.116:0/2235588617 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:18.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.167+0000 7eff77fff640 1 --2- 172.21.15.116:0/2235588617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582cd2925d0 0x5582cd2a9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:18.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.167+0000 7eff7cc73640 1 --2- 172.21.15.116:0/2235588617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582cd29d370 0x5582cd2b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:18.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.168+0000 7eff777fe640 1 -- 172.21.15.116:0/2235588617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582cd2925d0 msgr2=0x5582cd2a9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:18.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.168+0000 7eff777fe640 1 --2- 172.21.15.116:0/2235588617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582cd2925d0 0x5582cd2a9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.168+0000 7eff777fe640 1 -- 172.21.15.116:0/2235588617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582cd29d370 msgr2=0x5582cd2b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:18.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.168+0000 7eff777fe640 1 --2- 172.21.15.116:0/2235588617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582cd29d370 0x5582cd2b48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.168+0000 7eff777fe640 1 -- 172.21.15.116:0/2235588617 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582cd29fcf0 con 0x5582cd293b40 2024-09-12T03:12:18.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.168+0000 7eff7cc73640 1 --2- 172.21.15.116:0/2235588617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582cd29d370 0x5582cd2b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:12:18.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.168+0000 7eff777fe640 1 --2- 172.21.15.116:0/2235588617 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582cd293b40 0x5582cd298fe0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7eff68004550 tx=0x7eff6800bbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf25c078f2a4436 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:18.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.168+0000 7eff6dffb640 1 -- 172.21.15.116:0/2235588617 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff68003d50 con 0x5582cd293b40 2024-09-12T03:12:18.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.168+0000 7eff6dffb640 1 -- 172.21.15.116:0/2235588617 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff68020070 con 0x5582cd293b40 2024-09-12T03:12:18.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.168+0000 7eff6dffb640 1 -- 172.21.15.116:0/2235588617 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6801b6f0 con 0x5582cd293b40 2024-09-12T03:12:18.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.169+0000 7eff80aaf740 1 -- 172.21.15.116:0/2235588617 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582cd293b40 msgr2=0x5582cd298fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:18.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.169+0000 7eff80aaf740 1 --2- 172.21.15.116:0/2235588617 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582cd293b40 0x5582cd298fe0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7eff68004550 tx=0x7eff6800bbf0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.169+0000 7eff80aaf740 1 -- 172.21.15.116:0/2235588617 shutdown_connections 2024-09-12T03:12:18.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.169+0000 7eff80aaf740 1 --2- 172.21.15.116:0/2235588617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582cd29d370 0x5582cd2b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.169+0000 7eff80aaf740 1 --2- 172.21.15.116:0/2235588617 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582cd293b40 0x5582cd298fe0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.169+0000 7eff80aaf740 1 --2- 172.21.15.116:0/2235588617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582cd2925d0 0x5582cd2a9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.169+0000 7eff80aaf740 1 -- 172.21.15.116:0/2235588617 >> 172.21.15.116:0/2235588617 conn(0x5582cd25b310 msgr2=0x5582cd2988d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:18.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.169+0000 7eff80aaf740 1 -- 172.21.15.116:0/2235588617 shutdown_connections 2024-09-12T03:12:18.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.169+0000 7eff80aaf740 1 -- 172.21.15.116:0/2235588617 wait complete. 2024-09-12T03:12:18.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.169+0000 7eff80aaf740 1 -- start start 2024-09-12T03:12:18.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.169+0000 7eff80aaf740 1 Processor -- start 2024-09-12T03:12:18.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.170+0000 7eff80aaf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582cd2925d0 0x5582cd2a58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:18.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.170+0000 7eff80aaf740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582cd293b40 0x5582cd2a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:18.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.170+0000 7eff80aaf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582cd29d370 0x5582cd2a1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:18.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.170+0000 7eff80aaf740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7eff781adc50 con 0x5582cd293b40 2024-09-12T03:12:18.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.170+0000 7eff80aaf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7eff781af0c0 con 0x5582cd2925d0 2024-09-12T03:12:18.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.170+0000 7eff80aaf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff781ae320 con 0x5582cd29d370 2024-09-12T03:12:18.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.170+0000 7eff80aaf740 10 client.0 _ll_register_callbacks cb 0x7eff6800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:18.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.170+0000 7eff777fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582cd293b40 0x5582cd2a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:18.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.170+0000 7eff777fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582cd293b40 0x5582cd2a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54036/0 (socket says 172.21.15.116:54036) 2024-09-12T03:12:18.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.170+0000 7eff777fe640 1 -- 172.21.15.116:0/1953681633 learned_addr learned my addr 172.21.15.116:0/1953681633 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:18.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.170+0000 7eff7cc73640 1 --2- 172.21.15.116:0/1953681633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582cd29d370 0x5582cd2a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:18.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.170+0000 7eff77fff640 1 --2- 172.21.15.116:0/1953681633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582cd2925d0 0x5582cd2a58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:18.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.170+0000 7eff777fe640 1 -- 172.21.15.116:0/1953681633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582cd29d370 msgr2=0x5582cd2a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:18.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.170+0000 7eff777fe640 1 --2- 172.21.15.116:0/1953681633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582cd29d370 0x5582cd2a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.170+0000 7eff777fe640 1 -- 172.21.15.116:0/1953681633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582cd2925d0 msgr2=0x5582cd2a58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:18.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.170+0000 7eff777fe640 1 --2- 172.21.15.116:0/1953681633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582cd2925d0 0x5582cd2a58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.170+0000 7eff777fe640 1 -- 172.21.15.116:0/1953681633 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582cd28f960 con 0x5582cd293b40 2024-09-12T03:12:18.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.170+0000 7eff77fff640 1 --2- 172.21.15.116:0/1953681633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582cd2925d0 0x5582cd2a58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:12:18.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.170+0000 7eff777fe640 1 --2- 172.21.15.116:0/1953681633 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582cd293b40 0x5582cd2a0be0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7eff68011e30 tx=0x7eff68003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3afb7092b74a834b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:18.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.170+0000 7eff577fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:12:18.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.170+0000 7eff577fe640 1 -- 172.21.15.116:0/1953681633 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff68011640 con 0x5582cd293b40 2024-09-12T03:12:18.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.170+0000 7eff577fe640 1 -- 172.21.15.116:0/1953681633 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7eff680117e0 con 0x5582cd293b40 2024-09-12T03:12:18.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.170+0000 7eff80aaf740 1 -- 172.21.15.116:0/1953681633 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff68020070 con 0x5582cd293b40 2024-09-12T03:12:18.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.170+0000 7eff577fe640 1 -- 172.21.15.116:0/1953681633 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7eff680041d0 con 0x5582cd293b40 2024-09-12T03:12:18.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.170+0000 7eff80aaf740 10 client.17922 fetch_fsmap learned FSMap version 25 2024-09-12T03:12:18.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.170+0000 7eff80aaf740 1 -- 172.21.15.116:0/1953681633 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5582cd317480 con 0x5582cd293b40 2024-09-12T03:12:18.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.171+0000 7eff577fe640 1 -- 172.21.15.116:0/1953681633 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff68020070 con 0x5582cd293b40 2024-09-12T03:12:18.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.171+0000 7eff577fe640 1 -- 172.21.15.116:0/1953681633 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7eff68039420 con 0x5582cd293b40 2024-09-12T03:12:18.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.171+0000 7eff80aaf740 10 client.17922 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:12:18.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.171+0000 7eff80aaf740 10 client.17922 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:12:18.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.171+0000 7eff80aaf740 20 client.17922 populate_metadata read hostname 'smithi116' 2024-09-12T03:12:18.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.171+0000 7eff80aaf740 1 --2- 172.21.15.116:0/1953681633 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5582cd317c20 0x5582cd31a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:18.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.171+0000 7eff80aaf740 4 client.17922 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:12:18.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.171+0000 7eff80aaf740 1 -- 172.21.15.116:0/1953681633 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5582cd1ffd70 con 0x5582cd317c20 2024-09-12T03:12:18.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.171+0000 7eff77fff640 1 --2- 172.21.15.116:0/1953681633 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5582cd317c20 0x5582cd31a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:18.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.172+0000 7eff577fe640 10 client.17922 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:12:18.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.172+0000 7eff77fff640 1 --2- 172.21.15.116:0/1953681633 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5582cd317c20 0x5582cd31a040 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e6286af93691ad3 server_cookie=3a701c5bc2ec05d6 in_seq=0 out_seq=0 2024-09-12T03:12:18.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.172+0000 7eff577fe640 1 -- 172.21.15.116:0/1953681633 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5582cd1ffd70 con 0x5582cd317c20 2024-09-12T03:12:18.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.172+0000 7eff577fe640 10 client.17922 handle_command_reply: tid=0 2024-09-12T03:12:18.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.172+0000 7eff80aaf740 1 client.17922 shutdown 2024-09-12T03:12:18.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.172+0000 7eff80aaf740 1 -- 172.21.15.116:0/1953681633 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582cd293b40 msgr2=0x5582cd2a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:18.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.172+0000 7eff80aaf740 1 --2- 172.21.15.116:0/1953681633 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582cd293b40 0x5582cd2a0be0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7eff68011e30 tx=0x7eff68003e00 comp rx=0 tx=0).stop 2024-09-12T03:12:18.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.172+0000 7eff80aaf740 1 -- 172.21.15.116:0/1953681633 shutdown_connections 2024-09-12T03:12:18.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.172+0000 7eff80aaf740 1 --2- 172.21.15.116:0/1953681633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582cd29d370 0x5582cd2a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.172+0000 7eff80aaf740 1 --2- 172.21.15.116:0/1953681633 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5582cd317c20 0x5582cd31a040 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.172+0000 7eff80aaf740 1 --2- 172.21.15.116:0/1953681633 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5582cd293b40 0x5582cd2a0be0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.172+0000 7eff80aaf740 1 --2- 172.21.15.116:0/1953681633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5582cd2925d0 0x5582cd2a58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.172+0000 7eff577fe640 0 client.17922 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:12:18.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.172+0000 7eff80aaf740 1 -- 172.21.15.116:0/1953681633 >> 172.21.15.116:0/1953681633 conn(0x5582cd25b310 msgr2=0x5582cd2990f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:18.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.172+0000 7eff80aaf740 1 -- 172.21.15.116:0/1953681633 shutdown_connections 2024-09-12T03:12:18.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.172+0000 7eff80aaf740 1 -- 172.21.15.116:0/1953681633 wait complete. 2024-09-12T03:12:18.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.172+0000 7eff80aaf740 20 client.17922 trim_cache size 0 max 16384 2024-09-12T03:12:18.229 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:12:18.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.173+0000 7eff7e6d6640 1 -- 172.21.15.116:0/1365667111 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7eff640a2b80 msgr2=0x7eff640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:18.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.173+0000 7eff7e6d6640 1 --2- 172.21.15.116:0/1365667111 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7eff640a2b80 0x7eff640a5040 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7eff700107a0 tx=0x7eff7001a000 comp rx=0 tx=0).stop 2024-09-12T03:12:18.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.173+0000 7eff7e6d6640 1 -- 172.21.15.116:0/1365667111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff78101de0 msgr2=0x7eff780744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:18.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.173+0000 7eff7e6d6640 1 --2- 172.21.15.116:0/1365667111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff78101de0 0x7eff780744f0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7eff60006010 tx=0x7eff600042f0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.173+0000 7eff7e6d6640 1 -- 172.21.15.116:0/1365667111 shutdown_connections 2024-09-12T03:12:18.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.173+0000 7eff7e6d6640 1 --2- 172.21.15.116:0/1365667111 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7eff640a2b80 0x7eff640a5040 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.173+0000 7eff7e6d6640 1 --2- 172.21.15.116:0/1365667111 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff78106f10 0x7eff78074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.173+0000 7eff7e6d6640 1 --2- 172.21.15.116:0/1365667111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff78102790 0x7eff78074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.173+0000 7eff7e6d6640 1 --2- 172.21.15.116:0/1365667111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff78101de0 0x7eff780744f0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:18.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.173+0000 7eff7e6d6640 1 -- 172.21.15.116:0/1365667111 >> 172.21.15.116:0/1365667111 conn(0x7eff780fd8c0 msgr2=0x7eff780feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:18.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.173+0000 7eff7e6d6640 1 -- 172.21.15.116:0/1365667111 shutdown_connections 2024-09-12T03:12:18.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:18.173+0000 7eff7e6d6640 1 -- 172.21.15.116:0/1365667111 wait complete. 2024-09-12T03:12:19.093 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:12:19.345 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:12:19.662 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:19 smithi116 ceph-mon[31410]: pgmap v618: 97 pgs: 97 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-12T03:12:19.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:19 smithi183 ceph-mon[35535]: pgmap v618: 97 pgs: 97 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-12T03:12:19.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:19 smithi177 ceph-mon[35305]: pgmap v618: 97 pgs: 97 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-12T03:12:19.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.871+0000 7faa542f5640 1 -- 172.21.15.116:0/1681514910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa4c102d30 msgr2=0x7faa4c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:19.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.871+0000 7faa542f5640 1 --2- 172.21.15.116:0/1681514910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa4c102d30 0x7faa4c103110 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7faa3c009cd0 tx=0x7faa3c02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.871+0000 7faa542f5640 1 -- 172.21.15.116:0/1681514910 shutdown_connections 2024-09-12T03:12:19.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.871+0000 7faa542f5640 1 --2- 172.21.15.116:0/1681514910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa4c1082a0 0x7faa4c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.871+0000 7faa542f5640 1 --2- 172.21.15.116:0/1681514910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa4c1036e0 0x7faa4c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.871+0000 7faa542f5640 1 --2- 172.21.15.116:0/1681514910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa4c102d30 0x7faa4c103110 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.871+0000 7faa542f5640 1 -- 172.21.15.116:0/1681514910 >> 172.21.15.116:0/1681514910 conn(0x7faa4c0fc7d0 msgr2=0x7faa4c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:19.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.871+0000 7faa542f5640 1 -- 172.21.15.116:0/1681514910 shutdown_connections 2024-09-12T03:12:19.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.871+0000 7faa542f5640 1 -- 172.21.15.116:0/1681514910 wait complete. 2024-09-12T03:12:19.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.871+0000 7faa542f5640 1 Processor -- start 2024-09-12T03:12:19.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.871+0000 7faa542f5640 1 -- start start 2024-09-12T03:12:19.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.871+0000 7faa542f5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa4c102d30 0x7faa4c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:19.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.872+0000 7faa542f5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa4c1036e0 0x7faa4c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:19.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.872+0000 7faa542f5640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa4c1082a0 0x7faa4c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:19.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.872+0000 7faa542f5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c1120a0 con 0x7faa4c1036e0 2024-09-12T03:12:19.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.872+0000 7faa542f5640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c111f20 con 0x7faa4c1082a0 2024-09-12T03:12:19.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.872+0000 7faa542f5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c112220 con 0x7faa4c102d30 2024-09-12T03:12:19.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.872+0000 7faa51890640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa4c1036e0 0x7faa4c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:19.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.872+0000 7faa51890640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa4c1036e0 0x7faa4c100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54052/0 (socket says 172.21.15.116:54052) 2024-09-12T03:12:19.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.872+0000 7faa51890640 1 -- 172.21.15.116:0/176942781 learned_addr learned my addr 172.21.15.116:0/176942781 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:19.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.872+0000 7faa52091640 1 --2- 172.21.15.116:0/176942781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa4c102d30 0x7faa4c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:19.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.872+0000 7faa52892640 1 --2- 172.21.15.116:0/176942781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa4c1082a0 0x7faa4c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:19.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.872+0000 7faa51890640 1 -- 172.21.15.116:0/176942781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa4c102d30 msgr2=0x7faa4c100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:19.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.872+0000 7faa51890640 1 --2- 172.21.15.116:0/176942781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa4c102d30 0x7faa4c100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.872+0000 7faa51890640 1 -- 172.21.15.116:0/176942781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa4c1082a0 msgr2=0x7faa4c101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:19.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.872+0000 7faa51890640 1 --2- 172.21.15.116:0/176942781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa4c1082a0 0x7faa4c101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.872+0000 7faa51890640 1 -- 172.21.15.116:0/176942781 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa4c1a6aa0 con 0x7faa4c1036e0 2024-09-12T03:12:19.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.872+0000 7faa52091640 1 --2- 172.21.15.116:0/176942781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa4c102d30 0x7faa4c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:12:19.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.872+0000 7faa52892640 1 --2- 172.21.15.116:0/176942781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa4c1082a0 0x7faa4c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:12:19.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.872+0000 7faa51890640 1 --2- 172.21.15.116:0/176942781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa4c1036e0 0x7faa4c100ed0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7faa4000d750 tx=0x7faa4000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:19.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.872+0000 7faa4b7fe640 1 -- 172.21.15.116:0/176942781 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa40017070 con 0x7faa4c1036e0 2024-09-12T03:12:19.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.872+0000 7faa4b7fe640 1 -- 172.21.15.116:0/176942781 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa40005500 con 0x7faa4c1036e0 2024-09-12T03:12:19.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.872+0000 7faa4b7fe640 1 -- 172.21.15.116:0/176942781 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa40004e00 con 0x7faa4c1036e0 2024-09-12T03:12:19.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.872+0000 7faa542f5640 1 -- 172.21.15.116:0/176942781 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa4c1a6d90 con 0x7faa4c1036e0 2024-09-12T03:12:19.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.872+0000 7faa542f5640 1 -- 172.21.15.116:0/176942781 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa4c1a7270 con 0x7faa4c1036e0 2024-09-12T03:12:19.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.874+0000 7faa4b7fe640 1 -- 172.21.15.116:0/176942781 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7faa40004fa0 con 0x7faa4c1036e0 2024-09-12T03:12:19.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.875+0000 7faa566ce740 1 Processor -- start 2024-09-12T03:12:19.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.875+0000 7faa566ce740 1 -- start start 2024-09-12T03:12:19.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.875+0000 7faa4b7fe640 1 --2- 172.21.15.116:0/176942781 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7faa340a2c00 0x7faa340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:19.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.875+0000 7faa566ce740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f2fc31cb40 0x55f2fc31b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:19.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.875+0000 7faa566ce740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f2fc31b5d0 0x55f2fc31ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:19.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.875+0000 7faa566ce740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2fc326370 0x55f2fc328760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:19.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.875+0000 7faa566ce740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55f2fc2812f0 con 0x55f2fc31b5d0 2024-09-12T03:12:19.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.875+0000 7faa566ce740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f2fc228cf0 con 0x55f2fc31cb40 2024-09-12T03:12:19.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.875+0000 7faa566ce740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f2fc27d2b0 con 0x55f2fc326370 2024-09-12T03:12:19.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.875+0000 7faa52892640 1 --2- 172.21.15.116:0/176942781 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7faa340a2c00 0x7faa340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:19.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.875+0000 7faa4b7fe640 1 -- 172.21.15.116:0/176942781 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7faa400cea00 con 0x7faa4c1036e0 2024-09-12T03:12:19.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.876+0000 7faa51890640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f2fc31b5d0 0x55f2fc31ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:19.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.876+0000 7faa52091640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f2fc31cb40 0x55f2fc31b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:19.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.876+0000 7faa51890640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f2fc31b5d0 0x55f2fc31ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54054/0 (socket says 172.21.15.116:54054) 2024-09-12T03:12:19.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.876+0000 7faa51890640 1 -- 172.21.15.116:0/1267758307 learned_addr learned my addr 172.21.15.116:0/1267758307 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:19.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.876+0000 7faa52091640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f2fc31cb40 0x55f2fc31b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.116:50532/0 (socket says 172.21.15.116:50532) 2024-09-12T03:12:19.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.876+0000 7faa52892640 1 --2- 172.21.15.116:0/1267758307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2fc326370 0x55f2fc328760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:19.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.876+0000 7faa51890640 1 -- 172.21.15.116:0/1267758307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2fc326370 msgr2=0x55f2fc328760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:19.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.876+0000 7faa51890640 1 --2- 172.21.15.116:0/1267758307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2fc326370 0x55f2fc328760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.876+0000 7faa51890640 1 -- 172.21.15.116:0/1267758307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f2fc31cb40 msgr2=0x55f2fc31b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:19.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.876+0000 7faa51890640 1 --2- 172.21.15.116:0/1267758307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f2fc31cb40 0x55f2fc31b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.876+0000 7faa51890640 1 -- 172.21.15.116:0/1267758307 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2fc20d3a0 con 0x55f2fc31b5d0 2024-09-12T03:12:19.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.876+0000 7faa52892640 1 --2- 172.21.15.116:0/176942781 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7faa340a2c00 0x7faa340a50c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7faa4400be20 tx=0x7faa44014840 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:19.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.876+0000 7faa51890640 1 --2- 172.21.15.116:0/1267758307 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f2fc31b5d0 0x55f2fc31ba10 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7faa40097c60 tx=0x7faa4000de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54e8996491bc223e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:19.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.876+0000 7faa48dea640 1 -- 172.21.15.116:0/1267758307 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa40020040 con 0x55f2fc31b5d0 2024-09-12T03:12:19.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.876+0000 7faa48dea640 1 -- 172.21.15.116:0/1267758307 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa4009b290 con 0x55f2fc31b5d0 2024-09-12T03:12:19.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.876+0000 7faa48dea640 1 -- 172.21.15.116:0/1267758307 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa40009cc0 con 0x55f2fc31b5d0 2024-09-12T03:12:19.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.877+0000 7faa566ce740 1 -- 172.21.15.116:0/1267758307 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f2fc31b5d0 msgr2=0x55f2fc31ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:19.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.877+0000 7faa566ce740 1 --2- 172.21.15.116:0/1267758307 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f2fc31b5d0 0x55f2fc31ba10 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7faa40097c60 tx=0x7faa4000de60 comp rx=0 tx=0).stop 2024-09-12T03:12:19.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.877+0000 7faa566ce740 1 -- 172.21.15.116:0/1267758307 shutdown_connections 2024-09-12T03:12:19.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.877+0000 7faa566ce740 1 --2- 172.21.15.116:0/1267758307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2fc326370 0x55f2fc328760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.877+0000 7faa566ce740 1 --2- 172.21.15.116:0/1267758307 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f2fc31b5d0 0x55f2fc31ba10 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.877+0000 7faa566ce740 1 --2- 172.21.15.116:0/1267758307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f2fc31cb40 0x55f2fc31b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.877+0000 7faa566ce740 1 -- 172.21.15.116:0/1267758307 >> 172.21.15.116:0/1267758307 conn(0x55f2fc2e4310 msgr2=0x55f2fc317c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:19.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.877+0000 7faa566ce740 1 -- 172.21.15.116:0/1267758307 shutdown_connections 2024-09-12T03:12:19.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.877+0000 7faa566ce740 1 -- 172.21.15.116:0/1267758307 wait complete. 2024-09-12T03:12:19.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.877+0000 7faa566ce740 1 -- start start 2024-09-12T03:12:19.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.878+0000 7faa566ce740 1 Processor -- start 2024-09-12T03:12:19.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.878+0000 7faa566ce740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f2fc31b5d0 0x55f2fc325dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:19.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.878+0000 7faa566ce740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2fc31cb40 0x55f2fc323080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:19.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.878+0000 7faa566ce740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f2fc326370 0x55f2fc323810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:19.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.878+0000 7faa566ce740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55f2fc329010 con 0x55f2fc31b5d0 2024-09-12T03:12:19.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.878+0000 7faa566ce740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f2fc23c060 con 0x55f2fc326370 2024-09-12T03:12:19.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.878+0000 7faa566ce740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f2fc329190 con 0x55f2fc31cb40 2024-09-12T03:12:19.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.878+0000 7faa566ce740 10 client.0 _ll_register_callbacks cb 0x55f2fbf8f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:19.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.878+0000 7faa52091640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f2fc31b5d0 0x55f2fc325dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:19.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.878+0000 7faa52091640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f2fc31b5d0 0x55f2fc325dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54062/0 (socket says 172.21.15.116:54062) 2024-09-12T03:12:19.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.878+0000 7faa52091640 1 -- 172.21.15.116:0/3778172955 learned_addr learned my addr 172.21.15.116:0/3778172955 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:19.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.878+0000 7faa51890640 1 --2- 172.21.15.116:0/3778172955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2fc31cb40 0x55f2fc323080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:19.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.878+0000 7faa52892640 1 --2- 172.21.15.116:0/3778172955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f2fc326370 0x55f2fc323810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:19.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.878+0000 7faa52091640 1 -- 172.21.15.116:0/3778172955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2fc31cb40 msgr2=0x55f2fc323080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:19.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.878+0000 7faa52091640 1 --2- 172.21.15.116:0/3778172955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2fc31cb40 0x55f2fc323080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.878+0000 7faa52091640 1 -- 172.21.15.116:0/3778172955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f2fc326370 msgr2=0x55f2fc323810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:19.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.878+0000 7faa52091640 1 --2- 172.21.15.116:0/3778172955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f2fc326370 0x55f2fc323810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.878+0000 7faa52091640 1 -- 172.21.15.116:0/3778172955 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2fbff23c0 con 0x55f2fc31b5d0 2024-09-12T03:12:19.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.878+0000 7faa52892640 1 --2- 172.21.15.116:0/3778172955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f2fc326370 0x55f2fc323810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:12:19.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.878+0000 7faa52091640 1 --2- 172.21.15.116:0/3778172955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f2fc31b5d0 0x55f2fc325dd0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7faa3c009cd0 tx=0x7faa3c031c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4f3f968874be6b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:19.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.878+0000 7faa31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:12:19.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.879+0000 7faa31ffb640 1 -- 172.21.15.116:0/3778172955 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c049060 con 0x55f2fc31b5d0 2024-09-12T03:12:19.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.879+0000 7faa31ffb640 1 -- 172.21.15.116:0/3778172955 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7faa3c003b80 con 0x55f2fc31b5d0 2024-09-12T03:12:19.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.879+0000 7faa566ce740 1 -- 172.21.15.116:0/3778172955 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa4009b290 con 0x55f2fc31b5d0 2024-09-12T03:12:19.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.879+0000 7faa31ffb640 1 -- 172.21.15.116:0/3778172955 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7faa3c003d80 con 0x55f2fc31b5d0 2024-09-12T03:12:19.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.879+0000 7faa566ce740 10 client.17934 fetch_fsmap learned FSMap version 25 2024-09-12T03:12:19.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.879+0000 7faa566ce740 1 -- 172.21.15.116:0/3778172955 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f2fc334b80 con 0x55f2fc31b5d0 2024-09-12T03:12:19.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.879+0000 7faa31ffb640 1 -- 172.21.15.116:0/3778172955 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa4009b290 con 0x55f2fc31b5d0 2024-09-12T03:12:19.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.879+0000 7faa31ffb640 1 -- 172.21.15.116:0/3778172955 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7faa3c030070 con 0x55f2fc31b5d0 2024-09-12T03:12:19.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.879+0000 7faa566ce740 10 client.17934 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:12:19.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.879+0000 7faa566ce740 10 client.17934 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:12:19.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.879+0000 7faa566ce740 20 client.17934 populate_metadata read hostname 'smithi116' 2024-09-12T03:12:19.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.879+0000 7faa566ce740 1 --2- 172.21.15.116:0/3778172955 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55f2fc336040 0x55f2fc338450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:19.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.879+0000 7faa566ce740 4 client.17934 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:12:19.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.879+0000 7faa566ce740 1 -- 172.21.15.116:0/3778172955 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2fc28e860 con 0x55f2fc336040 2024-09-12T03:12:19.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.880+0000 7faa51890640 1 --2- 172.21.15.116:0/3778172955 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55f2fc336040 0x55f2fc338450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:19.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.880+0000 7faa51890640 1 --2- 172.21.15.116:0/3778172955 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55f2fc336040 0x55f2fc338450 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c9578ea40150866c server_cookie=4328ca5949575436 in_seq=0 out_seq=0 2024-09-12T03:12:19.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.880+0000 7faa31ffb640 10 client.17934 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:12:19.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.881+0000 7faa31ffb640 1 -- 172.21.15.116:0/3778172955 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f2fc28e860 con 0x55f2fc336040 2024-09-12T03:12:19.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.881+0000 7faa31ffb640 10 client.17934 handle_command_reply: tid=0 2024-09-12T03:12:19.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.881+0000 7faa566ce740 1 client.17934 shutdown 2024-09-12T03:12:19.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.881+0000 7faa566ce740 1 -- 172.21.15.116:0/3778172955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f2fc31b5d0 msgr2=0x55f2fc325dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:19.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.881+0000 7faa566ce740 1 --2- 172.21.15.116:0/3778172955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f2fc31b5d0 0x55f2fc325dd0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7faa3c009cd0 tx=0x7faa3c031c30 comp rx=0 tx=0).stop 2024-09-12T03:12:19.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.881+0000 7faa566ce740 1 -- 172.21.15.116:0/3778172955 shutdown_connections 2024-09-12T03:12:19.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.881+0000 7faa566ce740 1 --2- 172.21.15.116:0/3778172955 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55f2fc336040 0x55f2fc338450 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.881+0000 7faa566ce740 1 --2- 172.21.15.116:0/3778172955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f2fc326370 0x55f2fc323810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.881+0000 7faa566ce740 1 --2- 172.21.15.116:0/3778172955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2fc31cb40 0x55f2fc323080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.881+0000 7faa31ffb640 0 client.17934 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:12:19.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.881+0000 7faa566ce740 1 --2- 172.21.15.116:0/3778172955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f2fc31b5d0 0x55f2fc325dd0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.881+0000 7faa566ce740 1 -- 172.21.15.116:0/3778172955 >> 172.21.15.116:0/3778172955 conn(0x55f2fc2e4310 msgr2=0x55f2fc31d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:19.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.881+0000 7faa566ce740 1 -- 172.21.15.116:0/3778172955 shutdown_connections 2024-09-12T03:12:19.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.881+0000 7faa566ce740 1 -- 172.21.15.116:0/3778172955 wait complete. 2024-09-12T03:12:19.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.881+0000 7faa566ce740 20 client.17934 trim_cache size 0 max 16384 2024-09-12T03:12:19.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.894+0000 7faa566ce740 1 Processor -- start 2024-09-12T03:12:19.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.894+0000 7faa566ce740 1 -- start start 2024-09-12T03:12:19.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.894+0000 7faa566ce740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2fc31b5d0 0x55f2fc3319b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:19.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.894+0000 7faa566ce740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f2fc31cb40 0x55f2fc331ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:19.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.894+0000 7faa566ce740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f2fc326370 0x55f2fc31e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:19.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.894+0000 7faa566ce740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c1b1f90 con 0x55f2fc31cb40 2024-09-12T03:12:19.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.894+0000 7faa566ce740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c1af860 con 0x55f2fc326370 2024-09-12T03:12:19.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.894+0000 7faa566ce740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c1ad6b0 con 0x55f2fc31b5d0 2024-09-12T03:12:19.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.894+0000 7faa51890640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f2fc31cb40 0x55f2fc331ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:19.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.894+0000 7faa51890640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f2fc31cb40 0x55f2fc331ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54068/0 (socket says 172.21.15.116:54068) 2024-09-12T03:12:19.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.894+0000 7faa51890640 1 -- 172.21.15.116:0/1512944773 learned_addr learned my addr 172.21.15.116:0/1512944773 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:19.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.894+0000 7faa52091640 1 --2- 172.21.15.116:0/1512944773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2fc31b5d0 0x55f2fc3319b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:19.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.894+0000 7faa51890640 1 -- 172.21.15.116:0/1512944773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2fc31b5d0 msgr2=0x55f2fc3319b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:19.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.894+0000 7faa51890640 1 --2- 172.21.15.116:0/1512944773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2fc31b5d0 0x55f2fc3319b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.894+0000 7faa52892640 1 --2- 172.21.15.116:0/1512944773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f2fc326370 0x55f2fc31e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:19.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.894+0000 7faa51890640 1 -- 172.21.15.116:0/1512944773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f2fc326370 msgr2=0x55f2fc31e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:19.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.894+0000 7faa51890640 1 --2- 172.21.15.116:0/1512944773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f2fc326370 0x55f2fc31e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.894+0000 7faa51890640 1 -- 172.21.15.116:0/1512944773 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2fc324400 con 0x55f2fc31cb40 2024-09-12T03:12:19.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.894+0000 7faa51890640 1 --2- 172.21.15.116:0/1512944773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f2fc31cb40 0x55f2fc331ef0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7faa400cc8a0 tx=0x7faa400bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbef4d7507d2b4e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:19.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.895+0000 7faa33fff640 1 -- 172.21.15.116:0/1512944773 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa40020040 con 0x55f2fc31cb40 2024-09-12T03:12:19.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.895+0000 7faa33fff640 1 -- 172.21.15.116:0/1512944773 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa400a3f00 con 0x55f2fc31cb40 2024-09-12T03:12:19.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.895+0000 7faa33fff640 1 -- 172.21.15.116:0/1512944773 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa40009dc0 con 0x55f2fc31cb40 2024-09-12T03:12:19.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.895+0000 7faa566ce740 1 -- 172.21.15.116:0/1512944773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f2fc31cb40 msgr2=0x55f2fc331ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:19.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.895+0000 7faa566ce740 1 --2- 172.21.15.116:0/1512944773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f2fc31cb40 0x55f2fc331ef0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7faa400cc8a0 tx=0x7faa400bb040 comp rx=0 tx=0).stop 2024-09-12T03:12:19.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.895+0000 7faa566ce740 1 -- 172.21.15.116:0/1512944773 shutdown_connections 2024-09-12T03:12:19.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.895+0000 7faa566ce740 1 --2- 172.21.15.116:0/1512944773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f2fc326370 0x55f2fc31e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.895+0000 7faa566ce740 1 --2- 172.21.15.116:0/1512944773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f2fc31cb40 0x55f2fc331ef0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.895+0000 7faa566ce740 1 --2- 172.21.15.116:0/1512944773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2fc31b5d0 0x55f2fc3319b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.895+0000 7faa566ce740 1 -- 172.21.15.116:0/1512944773 >> 172.21.15.116:0/1512944773 conn(0x55f2fc2e4310 msgr2=0x55f2fc3312a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:19.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.895+0000 7faa566ce740 1 -- 172.21.15.116:0/1512944773 shutdown_connections 2024-09-12T03:12:19.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.895+0000 7faa566ce740 1 -- 172.21.15.116:0/1512944773 wait complete. 2024-09-12T03:12:19.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.896+0000 7faa566ce740 1 -- start start 2024-09-12T03:12:19.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.896+0000 7faa566ce740 1 Processor -- start 2024-09-12T03:12:19.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.896+0000 7faa566ce740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f2fc31b5d0 0x55f2fc320220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:19.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.896+0000 7faa566ce740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f2fc31cb40 0x55f2fc320760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:19.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.896+0000 7faa566ce740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2fc326370 0x55f2fc320ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:19.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.896+0000 7faa566ce740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c1add20 con 0x55f2fc31cb40 2024-09-12T03:12:19.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.896+0000 7faa566ce740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c1af190 con 0x55f2fc31b5d0 2024-09-12T03:12:19.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.896+0000 7faa566ce740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c1ae3f0 con 0x55f2fc326370 2024-09-12T03:12:19.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.896+0000 7faa566ce740 10 client.0 _ll_register_callbacks cb 0x7faa3c009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:19.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.896+0000 7faa51890640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f2fc31cb40 0x55f2fc320760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:19.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.896+0000 7faa51890640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f2fc31cb40 0x55f2fc320760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:54074/0 (socket says 172.21.15.116:54074) 2024-09-12T03:12:19.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.896+0000 7faa51890640 1 -- 172.21.15.116:0/728041644 learned_addr learned my addr 172.21.15.116:0/728041644 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:19.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.896+0000 7faa52892640 1 --2- 172.21.15.116:0/728041644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2fc326370 0x55f2fc320ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:19.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.896+0000 7faa51890640 1 -- 172.21.15.116:0/728041644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2fc326370 msgr2=0x55f2fc320ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:19.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.896+0000 7faa51890640 1 --2- 172.21.15.116:0/728041644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2fc326370 0x55f2fc320ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.896+0000 7faa51890640 1 -- 172.21.15.116:0/728041644 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f2fc31b5d0 msgr2=0x55f2fc320220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:19.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.896+0000 7faa51890640 1 --2- 172.21.15.116:0/728041644 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f2fc31b5d0 0x55f2fc320220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.896+0000 7faa51890640 1 -- 172.21.15.116:0/728041644 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2fc39fe70 con 0x55f2fc31cb40 2024-09-12T03:12:19.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.897+0000 7faa51890640 1 --2- 172.21.15.116:0/728041644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f2fc31cb40 0x55f2fc320760 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7faa400048e0 tx=0x7faa4000b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f28b5ea171712e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:19.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.897+0000 7faa31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:12:19.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.897+0000 7faa31ffb640 1 -- 172.21.15.116:0/728041644 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa400b3020 con 0x55f2fc31cb40 2024-09-12T03:12:19.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.897+0000 7faa31ffb640 1 -- 172.21.15.116:0/728041644 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7faa400b4220 con 0x55f2fc31cb40 2024-09-12T03:12:19.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.897+0000 7faa566ce740 1 -- 172.21.15.116:0/728041644 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa400a3f00 con 0x55f2fc31cb40 2024-09-12T03:12:19.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.897+0000 7faa31ffb640 1 -- 172.21.15.116:0/728041644 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7faa40020050 con 0x55f2fc31cb40 2024-09-12T03:12:19.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.897+0000 7faa566ce740 10 client.17940 fetch_fsmap learned FSMap version 25 2024-09-12T03:12:19.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.897+0000 7faa566ce740 1 -- 172.21.15.116:0/728041644 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f2fc3a1170 con 0x55f2fc31cb40 2024-09-12T03:12:19.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.897+0000 7faa31ffb640 1 -- 172.21.15.116:0/728041644 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa400a3f00 con 0x55f2fc31cb40 2024-09-12T03:12:19.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.897+0000 7faa31ffb640 1 -- 172.21.15.116:0/728041644 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7faa400bb0b0 con 0x55f2fc31cb40 2024-09-12T03:12:19.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.897+0000 7faa566ce740 10 client.17940 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:12:19.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.897+0000 7faa566ce740 10 client.17940 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:12:19.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.897+0000 7faa566ce740 20 client.17940 populate_metadata read hostname 'smithi116' 2024-09-12T03:12:19.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.897+0000 7faa566ce740 1 --2- 172.21.15.116:0/728041644 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55f2fc3a13a0 0x55f2fc3a3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:19.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.897+0000 7faa566ce740 4 client.17940 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:12:19.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.897+0000 7faa566ce740 1 -- 172.21.15.116:0/728041644 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f2fc288d70 con 0x55f2fc3a13a0 2024-09-12T03:12:19.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.898+0000 7faa52091640 1 --2- 172.21.15.116:0/728041644 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55f2fc3a13a0 0x55f2fc3a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:19.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.898+0000 7faa31ffb640 10 client.17940 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:12:19.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.898+0000 7faa52091640 1 --2- 172.21.15.116:0/728041644 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55f2fc3a13a0 0x55f2fc3a3790 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3df792561973535e server_cookie=4f54a1e73d5159be in_seq=0 out_seq=0 2024-09-12T03:12:19.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.898+0000 7faa31ffb640 1 -- 172.21.15.116:0/728041644 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f2fc288d70 con 0x55f2fc3a13a0 2024-09-12T03:12:19.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.898+0000 7faa31ffb640 10 client.17940 handle_command_reply: tid=0 2024-09-12T03:12:19.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.898+0000 7faa566ce740 1 client.17940 shutdown 2024-09-12T03:12:19.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.898+0000 7faa566ce740 1 -- 172.21.15.116:0/728041644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f2fc31cb40 msgr2=0x55f2fc320760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:19.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.898+0000 7faa566ce740 1 --2- 172.21.15.116:0/728041644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f2fc31cb40 0x55f2fc320760 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7faa400048e0 tx=0x7faa4000b400 comp rx=0 tx=0).stop 2024-09-12T03:12:19.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.898+0000 7faa566ce740 1 -- 172.21.15.116:0/728041644 shutdown_connections 2024-09-12T03:12:19.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.898+0000 7faa566ce740 1 --2- 172.21.15.116:0/728041644 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55f2fc3a13a0 0x55f2fc3a3790 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.898+0000 7faa566ce740 1 --2- 172.21.15.116:0/728041644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2fc326370 0x55f2fc320ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.898+0000 7faa566ce740 1 --2- 172.21.15.116:0/728041644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f2fc31cb40 0x55f2fc320760 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.898+0000 7faa566ce740 1 --2- 172.21.15.116:0/728041644 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f2fc31b5d0 0x55f2fc320220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.898+0000 7faa566ce740 1 -- 172.21.15.116:0/728041644 >> 172.21.15.116:0/728041644 conn(0x55f2fc2e4310 msgr2=0x55f2fc331ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:19.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.898+0000 7faa31ffb640 0 client.17940 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:12:19.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.898+0000 7faa566ce740 1 -- 172.21.15.116:0/728041644 shutdown_connections 2024-09-12T03:12:19.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.898+0000 7faa566ce740 1 -- 172.21.15.116:0/728041644 wait complete. 2024-09-12T03:12:19.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.899+0000 7faa566ce740 20 client.17940 trim_cache size 0 max 16384 2024-09-12T03:12:19.912 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:12:19.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.899+0000 7faa542f5640 1 -- 172.21.15.116:0/176942781 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7faa340a2c00 msgr2=0x7faa340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:19.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.899+0000 7faa542f5640 1 --2- 172.21.15.116:0/176942781 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7faa340a2c00 0x7faa340a50c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7faa4400be20 tx=0x7faa44014840 comp rx=0 tx=0).stop 2024-09-12T03:12:19.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.899+0000 7faa542f5640 1 -- 172.21.15.116:0/176942781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa4c1036e0 msgr2=0x7faa4c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:19.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.899+0000 7faa542f5640 1 --2- 172.21.15.116:0/176942781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa4c1036e0 0x7faa4c100ed0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7faa4000d750 tx=0x7faa4000dc10 comp rx=0 tx=0).stop 2024-09-12T03:12:19.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.899+0000 7faa542f5640 1 -- 172.21.15.116:0/176942781 shutdown_connections 2024-09-12T03:12:19.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.899+0000 7faa542f5640 1 --2- 172.21.15.116:0/176942781 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7faa340a2c00 0x7faa340a50c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.899+0000 7faa542f5640 1 --2- 172.21.15.116:0/176942781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa4c1082a0 0x7faa4c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.899+0000 7faa542f5640 1 --2- 172.21.15.116:0/176942781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa4c1036e0 0x7faa4c100ed0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.899+0000 7faa542f5640 1 --2- 172.21.15.116:0/176942781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa4c102d30 0x7faa4c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:19.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.899+0000 7faa542f5640 1 -- 172.21.15.116:0/176942781 >> 172.21.15.116:0/176942781 conn(0x7faa4c0fc7d0 msgr2=0x7faa4c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:19.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.899+0000 7faa542f5640 1 -- 172.21.15.116:0/176942781 shutdown_connections 2024-09-12T03:12:19.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:19.899+0000 7faa542f5640 1 -- 172.21.15.116:0/176942781 wait complete. 2024-09-12T03:12:20.709 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:12:20.955 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:12:21.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.414+0000 7f1afe18f640 1 -- 172.21.15.116:0/3490425424 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1af8106f20 msgr2=0x7f1af810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:21.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.414+0000 7f1afe18f640 1 --2- 172.21.15.116:0/3490425424 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1af8106f20 0x7f1af810eaa0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f1af0009ca0 tx=0x7f1af002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.414+0000 7f1afe18f640 1 -- 172.21.15.116:0/3490425424 shutdown_connections 2024-09-12T03:12:21.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.414+0000 7f1afe18f640 1 --2- 172.21.15.116:0/3490425424 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1af8106f20 0x7f1af810eaa0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.414+0000 7f1afe18f640 1 --2- 172.21.15.116:0/3490425424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1af81027a0 0x7f1af81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.414+0000 7f1afe18f640 1 --2- 172.21.15.116:0/3490425424 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1af8101df0 0x7f1af81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.414+0000 7f1afe18f640 1 -- 172.21.15.116:0/3490425424 >> 172.21.15.116:0/3490425424 conn(0x7f1af80fd8f0 msgr2=0x7f1af80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:21.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.414+0000 7f1afe18f640 1 -- 172.21.15.116:0/3490425424 shutdown_connections 2024-09-12T03:12:21.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.414+0000 7f1afe18f640 1 -- 172.21.15.116:0/3490425424 wait complete. 2024-09-12T03:12:21.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.414+0000 7f1afe18f640 1 Processor -- start 2024-09-12T03:12:21.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.415+0000 7f1afe18f640 1 -- start start 2024-09-12T03:12:21.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.415+0000 7f1afe18f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1af8101df0 0x7f1af8198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:21.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.415+0000 7f1afe18f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1af81027a0 0x7f1af8198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:21.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.415+0000 7f1afe18f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1af8106f20 0x7f1af819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:21.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.415+0000 7f1afe18f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1af8111f80 con 0x7f1af8106f20 2024-09-12T03:12:21.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.415+0000 7f1afe18f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1af8111e00 con 0x7f1af8101df0 2024-09-12T03:12:21.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.415+0000 7f1afe18f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1af8112100 con 0x7f1af81027a0 2024-09-12T03:12:21.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.415+0000 7f1af7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1af8106f20 0x7f1af819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:21.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.415+0000 7f1af7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1af8106f20 0x7f1af819ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40706/0 (socket says 172.21.15.116:40706) 2024-09-12T03:12:21.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.415+0000 7f1af7fff640 1 -- 172.21.15.116:0/116850786 learned_addr learned my addr 172.21.15.116:0/116850786 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:21.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.415+0000 7f1af77fe640 1 --2- 172.21.15.116:0/116850786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1af8101df0 0x7f1af8198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:21.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.415+0000 7f1af6ffd640 1 --2- 172.21.15.116:0/116850786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1af81027a0 0x7f1af8198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:21.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.415+0000 7f1af7fff640 1 -- 172.21.15.116:0/116850786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1af81027a0 msgr2=0x7f1af8198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:21.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.415+0000 7f1af7fff640 1 --2- 172.21.15.116:0/116850786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1af81027a0 0x7f1af8198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.415+0000 7f1af7fff640 1 -- 172.21.15.116:0/116850786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1af8101df0 msgr2=0x7f1af8198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:21.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.415+0000 7f1af7fff640 1 --2- 172.21.15.116:0/116850786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1af8101df0 0x7f1af8198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.415+0000 7f1af7fff640 1 -- 172.21.15.116:0/116850786 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af819d3b0 con 0x7f1af8106f20 2024-09-12T03:12:21.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.415+0000 7f1af7fff640 1 --2- 172.21.15.116:0/116850786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1af8106f20 0x7f1af819ccd0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f1af0031a20 tx=0x7f1af0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:21.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.416+0000 7f1af4ff9640 1 -- 172.21.15.116:0/116850786 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af0003c30 con 0x7f1af8106f20 2024-09-12T03:12:21.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.416+0000 7f1afe18f640 1 -- 172.21.15.116:0/116850786 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af819d640 con 0x7f1af8106f20 2024-09-12T03:12:21.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.416+0000 7f1af4ff9640 1 -- 172.21.15.116:0/116850786 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1af0003dd0 con 0x7f1af8106f20 2024-09-12T03:12:21.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.416+0000 7f1af4ff9640 1 -- 172.21.15.116:0/116850786 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af002fc40 con 0x7f1af8106f20 2024-09-12T03:12:21.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.416+0000 7f1afe18f640 1 -- 172.21.15.116:0/116850786 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af81a4f80 con 0x7f1af8106f20 2024-09-12T03:12:21.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.418+0000 7f1af4ff9640 1 -- 172.21.15.116:0/116850786 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1af0004050 con 0x7f1af8106f20 2024-09-12T03:12:21.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.419+0000 7f1b00568740 1 Processor -- start 2024-09-12T03:12:21.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.419+0000 7f1b00568740 1 -- start start 2024-09-12T03:12:21.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.419+0000 7f1b00568740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd85b40 0x559fedd84090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:21.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.419+0000 7f1b00568740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fedd845d0 0x559fedd84a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:21.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.419+0000 7f1b00568740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fedd8f370 0x559fedd91760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:21.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.419+0000 7f1b00568740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559fedcea2f0 con 0x559fedd8f370 2024-09-12T03:12:21.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.419+0000 7f1b00568740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559fedc91cf0 con 0x559fedd845d0 2024-09-12T03:12:21.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.419+0000 7f1b00568740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559fedce62b0 con 0x559fedd85b40 2024-09-12T03:12:21.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.419+0000 7f1af6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fedd8f370 0x559fedd91760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:21.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.419+0000 7f1af6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fedd8f370 0x559fedd91760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40708/0 (socket says 172.21.15.116:40708) 2024-09-12T03:12:21.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.419+0000 7f1af6ffd640 1 -- 172.21.15.116:0/2239918196 learned_addr learned my addr 172.21.15.116:0/2239918196 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:21.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.419+0000 7f1af6ffd640 1 --2- 172.21.15.116:0/2239918196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd85b40 0x559fedd84090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:21.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.419+0000 7f1af77fe640 1 --2- 172.21.15.116:0/2239918196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fedd845d0 0x559fedd84a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:21.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.419+0000 7f1af4ff9640 1 --2- 172.21.15.116:0/116850786 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1ae00a2b80 0x7f1ae00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:21.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.419+0000 7f1af6ffd640 1 -- 172.21.15.116:0/2239918196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd85b40 msgr2=0x559fedd84090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:21.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.419+0000 7f1af6ffd640 1 --2- 172.21.15.116:0/2239918196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd85b40 0x559fedd84090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.419+0000 7f1af6ffd640 1 -- 172.21.15.116:0/2239918196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fedd845d0 msgr2=0x559fedd84a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:21.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.419+0000 7f1af7fff640 1 --2- 172.21.15.116:0/116850786 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1ae00a2b80 0x7f1ae00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:21.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.419+0000 7f1af6ffd640 1 --2- 172.21.15.116:0/2239918196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fedd845d0 0x559fedd84a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.419+0000 7f1af6ffd640 1 -- 172.21.15.116:0/2239918196 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fedc763a0 con 0x559fedd8f370 2024-09-12T03:12:21.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.420+0000 7f1af4ff9640 1 -- 172.21.15.116:0/116850786 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1af00f01a0 con 0x7f1af8106f20 2024-09-12T03:12:21.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.420+0000 7f1af6ffd640 1 --2- 172.21.15.116:0/2239918196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fedd8f370 0x559fedd91760 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f1aec0058c0 tx=0x7f1aec005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1fde6936065eb7f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:21.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.420+0000 7f1addffb640 1 -- 172.21.15.116:0/2239918196 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec00f7c0 con 0x559fedd8f370 2024-09-12T03:12:21.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.420+0000 7f1af7fff640 1 --2- 172.21.15.116:0/116850786 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1ae00a2b80 0x7f1ae00a5040 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1af00ee040 tx=0x7f1af00b20c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:21.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.420+0000 7f1addffb640 1 -- 172.21.15.116:0/2239918196 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1aec00fe40 con 0x559fedd8f370 2024-09-12T03:12:21.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.420+0000 7f1addffb640 1 -- 172.21.15.116:0/2239918196 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec00ee00 con 0x559fedd8f370 2024-09-12T03:12:21.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.420+0000 7f1b00568740 1 -- 172.21.15.116:0/2239918196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fedd8f370 msgr2=0x559fedd91760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:21.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.420+0000 7f1b00568740 1 --2- 172.21.15.116:0/2239918196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fedd8f370 0x559fedd91760 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f1aec0058c0 tx=0x7f1aec005850 comp rx=0 tx=0).stop 2024-09-12T03:12:21.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.421+0000 7f1b00568740 1 -- 172.21.15.116:0/2239918196 shutdown_connections 2024-09-12T03:12:21.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.421+0000 7f1b00568740 1 --2- 172.21.15.116:0/2239918196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fedd8f370 0x559fedd91760 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.421+0000 7f1b00568740 1 --2- 172.21.15.116:0/2239918196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fedd845d0 0x559fedd84a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.421+0000 7f1b00568740 1 --2- 172.21.15.116:0/2239918196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd85b40 0x559fedd84090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.421+0000 7f1b00568740 1 -- 172.21.15.116:0/2239918196 >> 172.21.15.116:0/2239918196 conn(0x559fedd4d310 msgr2=0x559fedd80c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:21.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.421+0000 7f1b00568740 1 -- 172.21.15.116:0/2239918196 shutdown_connections 2024-09-12T03:12:21.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.421+0000 7f1b00568740 1 -- 172.21.15.116:0/2239918196 wait complete. 2024-09-12T03:12:21.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.421+0000 7f1b00568740 1 -- start start 2024-09-12T03:12:21.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.421+0000 7f1b00568740 1 Processor -- start 2024-09-12T03:12:21.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.421+0000 7f1b00568740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd845d0 0x559fedd8ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:21.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.421+0000 7f1b00568740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fedd85b40 0x559fedd8c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:21.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.421+0000 7f1b00568740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fedd8f370 0x559fedd8c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:21.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.421+0000 7f1b00568740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559fedd92070 con 0x559fedd85b40 2024-09-12T03:12:21.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.421+0000 7f1b00568740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559fedca5060 con 0x559fedd8f370 2024-09-12T03:12:21.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.421+0000 7f1b00568740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559fedd921f0 con 0x559fedd845d0 2024-09-12T03:12:21.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.421+0000 7f1b00568740 10 client.0 _ll_register_callbacks cb 0x559fed9f88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:21.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.422+0000 7f1af77fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fedd85b40 0x559fedd8c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:21.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.422+0000 7f1af77fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fedd85b40 0x559fedd8c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40712/0 (socket says 172.21.15.116:40712) 2024-09-12T03:12:21.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.422+0000 7f1af77fe640 1 -- 172.21.15.116:0/1814661822 learned_addr learned my addr 172.21.15.116:0/1814661822 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:21.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.422+0000 7f1af6ffd640 1 --2- 172.21.15.116:0/1814661822 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fedd8f370 0x559fedd8c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:21.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.422+0000 7f1af6ffd640 1 --2- 172.21.15.116:0/1814661822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd845d0 0x559fedd8ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:21.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.422+0000 7f1af77fe640 1 -- 172.21.15.116:0/1814661822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd845d0 msgr2=0x559fedd8ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:21.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.422+0000 7f1af77fe640 1 --2- 172.21.15.116:0/1814661822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd845d0 0x559fedd8ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.422+0000 7f1af77fe640 1 -- 172.21.15.116:0/1814661822 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fedd8f370 msgr2=0x559fedd8c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:21.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.422+0000 7f1af77fe640 1 --2- 172.21.15.116:0/1814661822 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fedd8f370 0x559fedd8c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.422+0000 7f1af77fe640 1 -- 172.21.15.116:0/1814661822 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559feda5b3c0 con 0x559fedd85b40 2024-09-12T03:12:21.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.422+0000 7f1af77fe640 1 --2- 172.21.15.116:0/1814661822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fedd85b40 0x559fedd8c0b0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f1ae800bad0 tx=0x7f1ae800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85186973f73dcac server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:21.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.422+0000 7f1ad6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:12:21.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.422+0000 7f1ad6ffd640 1 -- 172.21.15.116:0/1814661822 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8005020 con 0x559fedd85b40 2024-09-12T03:12:21.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.422+0000 7f1ad6ffd640 1 -- 172.21.15.116:0/1814661822 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1ae80051c0 con 0x559fedd85b40 2024-09-12T03:12:21.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.422+0000 7f1b00568740 1 -- 172.21.15.116:0/1814661822 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1aec00fe40 con 0x559fedd85b40 2024-09-12T03:12:21.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.422+0000 7f1ad6ffd640 1 -- 172.21.15.116:0/1814661822 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1ae8005400 con 0x559fedd85b40 2024-09-12T03:12:21.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.422+0000 7f1b00568740 10 client.17952 fetch_fsmap learned FSMap version 25 2024-09-12T03:12:21.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.422+0000 7f1b00568740 1 -- 172.21.15.116:0/1814661822 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559fedd9dc20 con 0x559fedd85b40 2024-09-12T03:12:21.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.423+0000 7f1ad6ffd640 1 -- 172.21.15.116:0/1814661822 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1aec00fe40 con 0x559fedd85b40 2024-09-12T03:12:21.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.423+0000 7f1ad6ffd640 1 -- 172.21.15.116:0/1814661822 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1ae801b6d0 con 0x559fedd85b40 2024-09-12T03:12:21.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.423+0000 7f1b00568740 10 client.17952 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:12:21.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.423+0000 7f1b00568740 10 client.17952 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:12:21.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.423+0000 7f1b00568740 20 client.17952 populate_metadata read hostname 'smithi116' 2024-09-12T03:12:21.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.423+0000 7f1b00568740 1 --2- 172.21.15.116:0/1814661822 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559fedd9f130 0x559fedda1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:21.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.423+0000 7f1b00568740 4 client.17952 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:12:21.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.423+0000 7f1b00568740 1 -- 172.21.15.116:0/1814661822 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fedcf7860 con 0x559fedd9f130 2024-09-12T03:12:21.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.423+0000 7f1af6ffd640 1 --2- 172.21.15.116:0/1814661822 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559fedd9f130 0x559fedda1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:21.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.423+0000 7f1af6ffd640 1 --2- 172.21.15.116:0/1814661822 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559fedd9f130 0x559fedda1550 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6066c955f6be9e57 server_cookie=32ee6822b834baa9 in_seq=0 out_seq=0 2024-09-12T03:12:21.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.423+0000 7f1ad6ffd640 10 client.17952 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:21.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.424+0000 7f1ad6ffd640 1 -- 172.21.15.116:0/1814661822 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559fedcf7860 con 0x559fedd9f130 2024-09-12T03:12:21.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.424+0000 7f1ad6ffd640 10 client.17952 handle_command_reply: tid=0 2024-09-12T03:12:21.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.424+0000 7f1b00568740 1 client.17952 shutdown 2024-09-12T03:12:21.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.424+0000 7f1b00568740 1 -- 172.21.15.116:0/1814661822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fedd85b40 msgr2=0x559fedd8c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:21.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.424+0000 7f1b00568740 1 --2- 172.21.15.116:0/1814661822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fedd85b40 0x559fedd8c0b0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f1ae800bad0 tx=0x7f1ae800bed0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.424+0000 7f1b00568740 1 -- 172.21.15.116:0/1814661822 shutdown_connections 2024-09-12T03:12:21.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.424+0000 7f1b00568740 1 --2- 172.21.15.116:0/1814661822 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559fedd9f130 0x559fedda1550 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.424+0000 7f1b00568740 1 --2- 172.21.15.116:0/1814661822 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fedd8f370 0x559fedd8c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.424+0000 7f1ad6ffd640 0 client.17952 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:21.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.424+0000 7f1b00568740 1 --2- 172.21.15.116:0/1814661822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fedd85b40 0x559fedd8c0b0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.424+0000 7f1b00568740 1 --2- 172.21.15.116:0/1814661822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd845d0 0x559fedd8ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.424+0000 7f1b00568740 1 -- 172.21.15.116:0/1814661822 >> 172.21.15.116:0/1814661822 conn(0x559fedd4d310 msgr2=0x559fedd86960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:21.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.424+0000 7f1b00568740 1 -- 172.21.15.116:0/1814661822 shutdown_connections 2024-09-12T03:12:21.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.424+0000 7f1b00568740 1 -- 172.21.15.116:0/1814661822 wait complete. 2024-09-12T03:12:21.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.425+0000 7f1b00568740 20 client.17952 trim_cache size 0 max 16384 2024-09-12T03:12:21.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.438+0000 7f1b00568740 1 Processor -- start 2024-09-12T03:12:21.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.438+0000 7f1b00568740 1 -- start start 2024-09-12T03:12:21.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.438+0000 7f1b00568740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fedd845d0 0x559fedd80d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:21.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.438+0000 7f1b00568740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fedd85b40 0x559fedd8af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:21.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.438+0000 7f1b00568740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd8f370 0x559fedda6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:21.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.438+0000 7f1b00568740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1af81afd40 con 0x559fedd845d0 2024-09-12T03:12:21.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.438+0000 7f1b00568740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1af81ad610 con 0x559fedd85b40 2024-09-12T03:12:21.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.438+0000 7f1b00568740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1af81ab460 con 0x559fedd8f370 2024-09-12T03:12:21.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.438+0000 7f1af6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fedd845d0 0x559fedd80d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:21.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.438+0000 7f1af6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fedd845d0 0x559fedd80d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40718/0 (socket says 172.21.15.116:40718) 2024-09-12T03:12:21.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.438+0000 7f1af6ffd640 1 -- 172.21.15.116:0/1616550541 learned_addr learned my addr 172.21.15.116:0/1616550541 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:21.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.438+0000 7f1af77fe640 1 --2- 172.21.15.116:0/1616550541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fedd85b40 0x559fedd8af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:21.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.438+0000 7f1af6ffd640 1 --2- 172.21.15.116:0/1616550541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd8f370 0x559fedda6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:21.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.438+0000 7f1af6ffd640 1 -- 172.21.15.116:0/1616550541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd8f370 msgr2=0x559fedda6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:21.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.438+0000 7f1af6ffd640 1 --2- 172.21.15.116:0/1616550541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd8f370 0x559fedda6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.438+0000 7f1af6ffd640 1 -- 172.21.15.116:0/1616550541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fedd85b40 msgr2=0x559fedd8af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:21.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.438+0000 7f1af6ffd640 1 --2- 172.21.15.116:0/1616550541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fedd85b40 0x559fedd8af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.438+0000 7f1af6ffd640 1 -- 172.21.15.116:0/1616550541 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fedd80510 con 0x559fedd845d0 2024-09-12T03:12:21.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.438+0000 7f1af6ffd640 1 --2- 172.21.15.116:0/1616550541 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fedd845d0 0x559fedd80d70 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f1aec006010 tx=0x7f1aec009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ede262279bdd09 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:21.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.439+0000 7f1add7fa640 1 -- 172.21.15.116:0/1616550541 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec011100 con 0x559fedd845d0 2024-09-12T03:12:21.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.439+0000 7f1add7fa640 1 -- 172.21.15.116:0/1616550541 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1aec0112a0 con 0x559fedd845d0 2024-09-12T03:12:21.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.439+0000 7f1add7fa640 1 -- 172.21.15.116:0/1616550541 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec00f790 con 0x559fedd845d0 2024-09-12T03:12:21.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.439+0000 7f1b00568740 1 -- 172.21.15.116:0/1616550541 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fedd845d0 msgr2=0x559fedd80d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:21.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.439+0000 7f1b00568740 1 --2- 172.21.15.116:0/1616550541 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fedd845d0 0x559fedd80d70 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f1aec006010 tx=0x7f1aec009c50 comp rx=0 tx=0).stop 2024-09-12T03:12:21.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.439+0000 7f1b00568740 1 -- 172.21.15.116:0/1616550541 shutdown_connections 2024-09-12T03:12:21.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.439+0000 7f1b00568740 1 --2- 172.21.15.116:0/1616550541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd8f370 0x559fedda6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.439+0000 7f1b00568740 1 --2- 172.21.15.116:0/1616550541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fedd85b40 0x559fedd8af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.439+0000 7f1b00568740 1 --2- 172.21.15.116:0/1616550541 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fedd845d0 0x559fedd80d70 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.439+0000 7f1b00568740 1 -- 172.21.15.116:0/1616550541 >> 172.21.15.116:0/1616550541 conn(0x559fedd4d310 msgr2=0x559fedd8a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:21.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.439+0000 7f1b00568740 1 -- 172.21.15.116:0/1616550541 shutdown_connections 2024-09-12T03:12:21.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.439+0000 7f1b00568740 1 -- 172.21.15.116:0/1616550541 wait complete. 2024-09-12T03:12:21.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.439+0000 7f1b00568740 1 -- start start 2024-09-12T03:12:21.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.440+0000 7f1b00568740 1 Processor -- start 2024-09-12T03:12:21.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.440+0000 7f1b00568740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd845d0 0x559fedd97c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:21.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.440+0000 7f1b00568740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fedd85b40 0x559fedd88e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:21.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.440+0000 7f1b00568740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fedd8f370 0x559fedd89390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:21.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.440+0000 7f1b00568740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1af81abad0 con 0x559fedd8f370 2024-09-12T03:12:21.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.440+0000 7f1b00568740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1af81acf40 con 0x559fedd85b40 2024-09-12T03:12:21.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.440+0000 7f1b00568740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1af81ac1a0 con 0x559fedd845d0 2024-09-12T03:12:21.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.440+0000 7f1b00568740 10 client.0 _ll_register_callbacks cb 0x7f1ae80040a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:21.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.440+0000 7f1af6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fedd8f370 0x559fedd89390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:21.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.440+0000 7f1af6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fedd8f370 0x559fedd89390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40724/0 (socket says 172.21.15.116:40724) 2024-09-12T03:12:21.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.440+0000 7f1af6ffd640 1 -- 172.21.15.116:0/3302926972 learned_addr learned my addr 172.21.15.116:0/3302926972 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:21.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.440+0000 7f1af77fe640 1 --2- 172.21.15.116:0/3302926972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fedd85b40 0x559fedd88e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:21.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.440+0000 7f1af6ffd640 1 --2- 172.21.15.116:0/3302926972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd845d0 0x559fedd97c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:21.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.440+0000 7f1af6ffd640 1 -- 172.21.15.116:0/3302926972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd845d0 msgr2=0x559fedd97c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:21.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.440+0000 7f1af6ffd640 1 --2- 172.21.15.116:0/3302926972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd845d0 0x559fedd97c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.440+0000 7f1af6ffd640 1 -- 172.21.15.116:0/3302926972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fedd85b40 msgr2=0x559fedd88e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:21.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.440+0000 7f1af6ffd640 1 --2- 172.21.15.116:0/3302926972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fedd85b40 0x559fedd88e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.440+0000 7f1af6ffd640 1 -- 172.21.15.116:0/3302926972 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fedd89bc0 con 0x559fedd8f370 2024-09-12T03:12:21.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.441+0000 7f1af6ffd640 1 --2- 172.21.15.116:0/3302926972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fedd8f370 0x559fedd89390 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f1aec001eb0 tx=0x7f1aec00e4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42040b0aa6d2dad9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:21.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.441+0000 7f1ad6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:12:21.479 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:12:21.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.441+0000 7f1ad6ffd640 1 -- 172.21.15.116:0/3302926972 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec0050d0 con 0x559fedd8f370 2024-09-12T03:12:21.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.441+0000 7f1ad6ffd640 1 -- 172.21.15.116:0/3302926972 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1aec028040 con 0x559fedd8f370 2024-09-12T03:12:21.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.441+0000 7f1b00568740 1 -- 172.21.15.116:0/3302926972 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1aec0112a0 con 0x559fedd8f370 2024-09-12T03:12:21.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.441+0000 7f1ad6ffd640 1 -- 172.21.15.116:0/3302926972 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1aec005270 con 0x559fedd8f370 2024-09-12T03:12:21.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.441+0000 7f1b00568740 10 client.17958 fetch_fsmap learned FSMap version 25 2024-09-12T03:12:21.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.441+0000 7f1b00568740 1 -- 172.21.15.116:0/3302926972 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559fede09e30 con 0x559fedd8f370 2024-09-12T03:12:21.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.441+0000 7f1ad6ffd640 1 -- 172.21.15.116:0/3302926972 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1aec0112a0 con 0x559fedd8f370 2024-09-12T03:12:21.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.441+0000 7f1ad6ffd640 1 -- 172.21.15.116:0/3302926972 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1aec01d020 con 0x559fedd8f370 2024-09-12T03:12:21.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.441+0000 7f1b00568740 10 client.17958 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:12:21.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.441+0000 7f1b00568740 10 client.17958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:12:21.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.441+0000 7f1b00568740 20 client.17958 populate_metadata read hostname 'smithi116' 2024-09-12T03:12:21.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.441+0000 7f1b00568740 1 --2- 172.21.15.116:0/3302926972 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559fede0a250 0x559fede0c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:21.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.441+0000 7f1b00568740 4 client.17958 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:12:21.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.441+0000 7f1b00568740 1 -- 172.21.15.116:0/3302926972 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559fedcf1d70 con 0x559fede0a250 2024-09-12T03:12:21.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.441+0000 7f1af77fe640 1 --2- 172.21.15.116:0/3302926972 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559fede0a250 0x559fede0c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:21.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.442+0000 7f1af77fe640 1 --2- 172.21.15.116:0/3302926972 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559fede0a250 0x559fede0c640 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=418c378a32f4f5b9 server_cookie=32465a3d88844525 in_seq=0 out_seq=0 2024-09-12T03:12:21.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.442+0000 7f1ad6ffd640 10 client.17958 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:21.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.442+0000 7f1ad6ffd640 1 -- 172.21.15.116:0/3302926972 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559fedcf1d70 con 0x559fede0a250 2024-09-12T03:12:21.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.442+0000 7f1ad6ffd640 10 client.17958 handle_command_reply: tid=0 2024-09-12T03:12:21.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.442+0000 7f1b00568740 1 client.17958 shutdown 2024-09-12T03:12:21.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.442+0000 7f1b00568740 1 -- 172.21.15.116:0/3302926972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fedd8f370 msgr2=0x559fedd89390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:21.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.442+0000 7f1b00568740 1 --2- 172.21.15.116:0/3302926972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fedd8f370 0x559fedd89390 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f1aec001eb0 tx=0x7f1aec00e4f0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.442+0000 7f1b00568740 1 -- 172.21.15.116:0/3302926972 shutdown_connections 2024-09-12T03:12:21.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.442+0000 7f1b00568740 1 --2- 172.21.15.116:0/3302926972 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559fede0a250 0x559fede0c640 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.442+0000 7f1b00568740 1 --2- 172.21.15.116:0/3302926972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fedd8f370 0x559fedd89390 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.442+0000 7f1b00568740 1 --2- 172.21.15.116:0/3302926972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fedd85b40 0x559fedd88e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.442+0000 7f1ad6ffd640 0 client.17958 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:21.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.442+0000 7f1b00568740 1 --2- 172.21.15.116:0/3302926972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd845d0 0x559fedd97c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.442+0000 7f1b00568740 1 -- 172.21.15.116:0/3302926972 >> 172.21.15.116:0/3302926972 conn(0x559fedd4d310 msgr2=0x559fedd8e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:21.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.442+0000 7f1b00568740 1 -- 172.21.15.116:0/3302926972 shutdown_connections 2024-09-12T03:12:21.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.442+0000 7f1b00568740 1 -- 172.21.15.116:0/3302926972 wait complete. 2024-09-12T03:12:21.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.442+0000 7f1b00568740 20 client.17958 trim_cache size 0 max 16384 2024-09-12T03:12:21.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.443+0000 7f1afe18f640 1 -- 172.21.15.116:0/116850786 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1ae00a2b80 msgr2=0x7f1ae00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:21.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.443+0000 7f1afe18f640 1 --2- 172.21.15.116:0/116850786 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1ae00a2b80 0x7f1ae00a5040 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1af00ee040 tx=0x7f1af00b20c0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.443+0000 7f1afe18f640 1 -- 172.21.15.116:0/116850786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1af8106f20 msgr2=0x7f1af819ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:21.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.443+0000 7f1afe18f640 1 --2- 172.21.15.116:0/116850786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1af8106f20 0x7f1af819ccd0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f1af0031a20 tx=0x7f1af0031a50 comp rx=0 tx=0).stop 2024-09-12T03:12:21.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.443+0000 7f1afe18f640 1 -- 172.21.15.116:0/116850786 shutdown_connections 2024-09-12T03:12:21.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.443+0000 7f1afe18f640 1 --2- 172.21.15.116:0/116850786 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1ae00a2b80 0x7f1ae00a5040 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.443+0000 7f1afe18f640 1 --2- 172.21.15.116:0/116850786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1af8106f20 0x7f1af819ccd0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.443+0000 7f1afe18f640 1 --2- 172.21.15.116:0/116850786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1af81027a0 0x7f1af8198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.443+0000 7f1afe18f640 1 --2- 172.21.15.116:0/116850786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1af8101df0 0x7f1af8198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:21.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.443+0000 7f1afe18f640 1 -- 172.21.15.116:0/116850786 >> 172.21.15.116:0/116850786 conn(0x7f1af80fd8f0 msgr2=0x7f1af80fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:21.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.443+0000 7f1afe18f640 1 -- 172.21.15.116:0/116850786 shutdown_connections 2024-09-12T03:12:21.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:21.443+0000 7f1afe18f640 1 -- 172.21.15.116:0/116850786 wait complete. 2024-09-12T03:12:21.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:21 smithi183 ceph-mon[35535]: pgmap v619: 97 pgs: 97 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T03:12:21.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:21 smithi177 ceph-mon[35305]: pgmap v619: 97 pgs: 97 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T03:12:21.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:21 smithi116 ceph-mon[31410]: pgmap v619: 97 pgs: 97 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T03:12:23.174 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:12:23.174 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:12:23.437 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:12:23.857 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:23 smithi116 ceph-mon[31410]: pgmap v620: 97 pgs: 97 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 KiB/s wr, 9 op/s 2024-09-12T03:12:23.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:23 smithi183 ceph-mon[35535]: pgmap v620: 97 pgs: 97 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 KiB/s wr, 9 op/s 2024-09-12T03:12:23.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.941+0000 7fa7738c4640 1 -- 172.21.15.116:0/1199636029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa76c102ef0 msgr2=0x7fa76c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:23.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.941+0000 7fa7738c4640 1 --2- 172.21.15.116:0/1199636029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa76c102ef0 0x7fa76c10da40 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fa764009ca0 tx=0x7fa76402a950 comp rx=0 tx=0).stop 2024-09-12T03:12:23.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.941+0000 7fa7738c4640 1 -- 172.21.15.116:0/1199636029 shutdown_connections 2024-09-12T03:12:23.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.941+0000 7fa7738c4640 1 --2- 172.21.15.116:0/1199636029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa76c102ef0 0x7fa76c10da40 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.941+0000 7fa7738c4640 1 --2- 172.21.15.116:0/1199636029 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa76c102570 0x7fa76c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.941+0000 7fa7738c4640 1 --2- 172.21.15.116:0/1199636029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa76c100e20 0x7fa76c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.941+0000 7fa7738c4640 1 -- 172.21.15.116:0/1199636029 >> 172.21.15.116:0/1199636029 conn(0x7fa76c0fc7d0 msgr2=0x7fa76c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:23.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.941+0000 7fa7738c4640 1 -- 172.21.15.116:0/1199636029 shutdown_connections 2024-09-12T03:12:23.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.941+0000 7fa7738c4640 1 -- 172.21.15.116:0/1199636029 wait complete. 2024-09-12T03:12:23.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.941+0000 7fa7738c4640 1 Processor -- start 2024-09-12T03:12:23.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.941+0000 7fa7738c4640 1 -- start start 2024-09-12T03:12:23.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.942+0000 7fa7738c4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa76c100e20 0x7fa76c198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:23.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.942+0000 7fa7738c4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa76c102570 0x7fa76c198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:23.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.942+0000 7fa7738c4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa76c102ef0 0x7fa76c19cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:23.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.942+0000 7fa7738c4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c111020 con 0x7fa76c102570 2024-09-12T03:12:23.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.942+0000 7fa7738c4640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c110ea0 con 0x7fa76c100e20 2024-09-12T03:12:23.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.942+0000 7fa7738c4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c1111a0 con 0x7fa76c102ef0 2024-09-12T03:12:23.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.942+0000 7fa770e5f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa76c102570 0x7fa76c198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:23.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.942+0000 7fa770e5f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa76c102570 0x7fa76c198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40754/0 (socket says 172.21.15.116:40754) 2024-09-12T03:12:23.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.942+0000 7fa770e5f640 1 -- 172.21.15.116:0/1451018824 learned_addr learned my addr 172.21.15.116:0/1451018824 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:23.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.942+0000 7fa771660640 1 --2- 172.21.15.116:0/1451018824 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa76c100e20 0x7fa76c198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:23.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.942+0000 7fa770e5f640 1 -- 172.21.15.116:0/1451018824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa76c102ef0 msgr2=0x7fa76c19cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:23.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.942+0000 7fa771e61640 1 --2- 172.21.15.116:0/1451018824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa76c102ef0 0x7fa76c19cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:23.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.942+0000 7fa770e5f640 1 --2- 172.21.15.116:0/1451018824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa76c102ef0 0x7fa76c19cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.942+0000 7fa770e5f640 1 -- 172.21.15.116:0/1451018824 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa76c100e20 msgr2=0x7fa76c198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:23.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.942+0000 7fa770e5f640 1 --2- 172.21.15.116:0/1451018824 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa76c100e20 0x7fa76c198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.942+0000 7fa770e5f640 1 -- 172.21.15.116:0/1451018824 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76c19d630 con 0x7fa76c102570 2024-09-12T03:12:23.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.942+0000 7fa771660640 1 --2- 172.21.15.116:0/1451018824 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa76c100e20 0x7fa76c198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:12:23.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.942+0000 7fa770e5f640 1 --2- 172.21.15.116:0/1451018824 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa76c102570 0x7fa76c198bc0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fa76000b4a0 tx=0x7fa76000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:23.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.942+0000 7fa75e7fc640 1 -- 172.21.15.116:0/1451018824 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760014070 con 0x7fa76c102570 2024-09-12T03:12:23.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.942+0000 7fa75e7fc640 1 -- 172.21.15.116:0/1451018824 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa760010af0 con 0x7fa76c102570 2024-09-12T03:12:23.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.942+0000 7fa7738c4640 1 -- 172.21.15.116:0/1451018824 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa76c19d920 con 0x7fa76c102570 2024-09-12T03:12:23.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.942+0000 7fa75e7fc640 1 -- 172.21.15.116:0/1451018824 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760017d10 con 0x7fa76c102570 2024-09-12T03:12:23.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.942+0000 7fa7738c4640 1 -- 172.21.15.116:0/1451018824 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa76c1a5200 con 0x7fa76c102570 2024-09-12T03:12:23.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.945+0000 7fa75e7fc640 1 -- 172.21.15.116:0/1451018824 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa760017070 con 0x7fa76c102570 2024-09-12T03:12:23.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.945+0000 7fa775c9d740 1 Processor -- start 2024-09-12T03:12:23.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.945+0000 7fa775c9d740 1 -- start start 2024-09-12T03:12:23.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.945+0000 7fa775c9d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f3b7ab60 0x5648f3b7af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:23.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.945+0000 7fa775c9d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648f3b7b480 0x5648f3b85800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:23.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.945+0000 7fa775c9d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5648f3b85d40 0x5648f3b88130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:23.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.945+0000 7fa75e7fc640 1 --2- 172.21.15.116:0/1451018824 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa7580a2b80 0x7fa7580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:23.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.945+0000 7fa775c9d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5648f3ae0280 con 0x5648f3b7b480 2024-09-12T03:12:23.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.945+0000 7fa775c9d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5648f3a87b10 con 0x5648f3b85d40 2024-09-12T03:12:23.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.945+0000 7fa775c9d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5648f3adc240 con 0x5648f3b7ab60 2024-09-12T03:12:23.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.946+0000 7fa771660640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648f3b7b480 0x5648f3b85800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:23.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.946+0000 7fa771660640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648f3b7b480 0x5648f3b85800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40762/0 (socket says 172.21.15.116:40762) 2024-09-12T03:12:23.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.946+0000 7fa771e61640 1 --2- 172.21.15.116:0/1451018824 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa7580a2b80 0x7fa7580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:23.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.946+0000 7fa771660640 1 -- 172.21.15.116:0/2237464505 learned_addr learned my addr 172.21.15.116:0/2237464505 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:23.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.946+0000 7fa75e7fc640 1 -- 172.21.15.116:0/1451018824 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa7600ce490 con 0x7fa76c102570 2024-09-12T03:12:23.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.946+0000 7fa770e5f640 1 --2- 172.21.15.116:0/2237464505 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5648f3b85d40 0x5648f3b88130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:23.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.946+0000 7fa771e61640 1 --2- 172.21.15.116:0/2237464505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f3b7ab60 0x5648f3b7af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:23.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.946+0000 7fa771660640 1 -- 172.21.15.116:0/2237464505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f3b7ab60 msgr2=0x5648f3b7af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:23.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.946+0000 7fa771660640 1 --2- 172.21.15.116:0/2237464505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f3b7ab60 0x5648f3b7af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.946+0000 7fa771660640 1 -- 172.21.15.116:0/2237464505 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5648f3b85d40 msgr2=0x5648f3b88130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:23.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.946+0000 7fa771660640 1 --2- 172.21.15.116:0/2237464505 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5648f3b85d40 0x5648f3b88130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.946+0000 7fa771660640 1 -- 172.21.15.116:0/2237464505 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648f3a941f0 con 0x5648f3b7b480 2024-09-12T03:12:23.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.946+0000 7fa771e61640 1 --2- 172.21.15.116:0/2237464505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f3b7ab60 0x5648f3b7af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:12:23.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.946+0000 7fa770e5f640 1 --2- 172.21.15.116:0/2237464505 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5648f3b85d40 0x5648f3b88130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:12:23.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.946+0000 7fa771660640 1 --2- 172.21.15.116:0/2237464505 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648f3b7b480 0x5648f3b85800 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fa768007850 tx=0x7fa768007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9612499cbd22ddc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:23.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.946+0000 7fa771e61640 1 --2- 172.21.15.116:0/1451018824 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa7580a2b80 0x7fa7580a5040 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa764030040 tx=0x7fa764005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:23.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.946+0000 7fa7577fe640 1 -- 172.21.15.116:0/2237464505 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76800d950 con 0x5648f3b7b480 2024-09-12T03:12:23.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.946+0000 7fa7577fe640 1 -- 172.21.15.116:0/2237464505 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa768018070 con 0x5648f3b7b480 2024-09-12T03:12:23.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.946+0000 7fa7577fe640 1 -- 172.21.15.116:0/2237464505 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7680123e0 con 0x5648f3b7b480 2024-09-12T03:12:23.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.947+0000 7fa775c9d740 1 -- 172.21.15.116:0/2237464505 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648f3b7b480 msgr2=0x5648f3b85800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:23.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.947+0000 7fa775c9d740 1 --2- 172.21.15.116:0/2237464505 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648f3b7b480 0x5648f3b85800 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fa768007850 tx=0x7fa768007d10 comp rx=0 tx=0).stop 2024-09-12T03:12:23.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.947+0000 7fa775c9d740 1 -- 172.21.15.116:0/2237464505 shutdown_connections 2024-09-12T03:12:23.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.947+0000 7fa775c9d740 1 --2- 172.21.15.116:0/2237464505 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5648f3b85d40 0x5648f3b88130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.947+0000 7fa775c9d740 1 --2- 172.21.15.116:0/2237464505 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648f3b7b480 0x5648f3b85800 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.947+0000 7fa775c9d740 1 --2- 172.21.15.116:0/2237464505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f3b7ab60 0x5648f3b7af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.947+0000 7fa775c9d740 1 -- 172.21.15.116:0/2237464505 >> 172.21.15.116:0/2237464505 conn(0x5648f3b770f0 msgr2=0x5648f3b774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:23.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.947+0000 7fa775c9d740 1 -- 172.21.15.116:0/2237464505 shutdown_connections 2024-09-12T03:12:23.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.947+0000 7fa775c9d740 1 -- 172.21.15.116:0/2237464505 wait complete. 2024-09-12T03:12:23.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.947+0000 7fa775c9d740 1 -- start start 2024-09-12T03:12:23.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.948+0000 7fa775c9d740 1 Processor -- start 2024-09-12T03:12:23.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.948+0000 7fa775c9d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5648f3b7ab60 0x5648f3b82b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:23.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.948+0000 7fa775c9d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648f3b7b480 0x5648f3b830a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:23.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.948+0000 7fa775c9d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f3b85d40 0x5648f3b835e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:23.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.948+0000 7fa775c9d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5648f3b88910 con 0x5648f3b7b480 2024-09-12T03:12:23.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.948+0000 7fa775c9d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5648f3a9af10 con 0x5648f3b7ab60 2024-09-12T03:12:23.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.948+0000 7fa775c9d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5648f3b88a90 con 0x5648f3b85d40 2024-09-12T03:12:23.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.948+0000 7fa775c9d740 10 client.0 _ll_register_callbacks cb 0x5648f3a63a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:23.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.948+0000 7fa770e5f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648f3b7b480 0x5648f3b830a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:23.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.948+0000 7fa770e5f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648f3b7b480 0x5648f3b830a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40766/0 (socket says 172.21.15.116:40766) 2024-09-12T03:12:23.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.948+0000 7fa770e5f640 1 -- 172.21.15.116:0/4056203603 learned_addr learned my addr 172.21.15.116:0/4056203603 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:23.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.948+0000 7fa771e61640 1 --2- 172.21.15.116:0/4056203603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f3b85d40 0x5648f3b835e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:23.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.948+0000 7fa771660640 1 --2- 172.21.15.116:0/4056203603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5648f3b7ab60 0x5648f3b82b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:23.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.948+0000 7fa770e5f640 1 -- 172.21.15.116:0/4056203603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f3b85d40 msgr2=0x5648f3b835e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:23.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.948+0000 7fa770e5f640 1 --2- 172.21.15.116:0/4056203603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f3b85d40 0x5648f3b835e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.948+0000 7fa770e5f640 1 -- 172.21.15.116:0/4056203603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5648f3b7ab60 msgr2=0x5648f3b82b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:23.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.948+0000 7fa770e5f640 1 --2- 172.21.15.116:0/4056203603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5648f3b7ab60 0x5648f3b82b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.948+0000 7fa770e5f640 1 -- 172.21.15.116:0/4056203603 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648f38516a0 con 0x5648f3b7b480 2024-09-12T03:12:23.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.949+0000 7fa770e5f640 1 --2- 172.21.15.116:0/4056203603 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648f3b7b480 0x5648f3b830a0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fa7600cc010 tx=0x7fa7600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b69343ca246faf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:23.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.949+0000 7fa754ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:12:23.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.949+0000 7fa754ff9640 1 -- 172.21.15.116:0/4056203603 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76001e070 con 0x5648f3b7b480 2024-09-12T03:12:23.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.949+0000 7fa754ff9640 1 -- 172.21.15.116:0/4056203603 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa760017360 con 0x5648f3b7b480 2024-09-12T03:12:23.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.949+0000 7fa775c9d740 1 -- 172.21.15.116:0/4056203603 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa768018070 con 0x5648f3b7b480 2024-09-12T03:12:23.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.949+0000 7fa754ff9640 1 -- 172.21.15.116:0/4056203603 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa7600a3990 con 0x5648f3b7b480 2024-09-12T03:12:23.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.949+0000 7fa775c9d740 10 client.17970 fetch_fsmap learned FSMap version 25 2024-09-12T03:12:23.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.949+0000 7fa775c9d740 1 -- 172.21.15.116:0/4056203603 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5648f3b94430 con 0x5648f3b7b480 2024-09-12T03:12:23.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.949+0000 7fa754ff9640 1 -- 172.21.15.116:0/4056203603 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa768018070 con 0x5648f3b7b480 2024-09-12T03:12:23.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.949+0000 7fa754ff9640 1 -- 172.21.15.116:0/4056203603 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa7600ce110 con 0x5648f3b7b480 2024-09-12T03:12:23.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.949+0000 7fa775c9d740 10 client.17970 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:12:23.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.949+0000 7fa775c9d740 10 client.17970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:12:23.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.949+0000 7fa775c9d740 20 client.17970 populate_metadata read hostname 'smithi116' 2024-09-12T03:12:23.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.949+0000 7fa775c9d740 1 --2- 172.21.15.116:0/4056203603 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5648f3b95890 0x5648f3b97d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:23.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.949+0000 7fa775c9d740 4 client.17970 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:12:23.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.949+0000 7fa775c9d740 1 -- 172.21.15.116:0/4056203603 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648f3aed7f0 con 0x5648f3b95890 2024-09-12T03:12:23.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.949+0000 7fa771660640 1 --2- 172.21.15.116:0/4056203603 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5648f3b95890 0x5648f3b97d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:23.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.950+0000 7fa754ff9640 10 client.17970 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:23.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.950+0000 7fa771660640 1 --2- 172.21.15.116:0/4056203603 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5648f3b95890 0x5648f3b97d20 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0f71097f4eb0497 server_cookie=baeee51cc84ede52 in_seq=0 out_seq=0 2024-09-12T03:12:23.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.951+0000 7fa754ff9640 1 -- 172.21.15.116:0/4056203603 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5648f3aed7f0 con 0x5648f3b95890 2024-09-12T03:12:23.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.951+0000 7fa754ff9640 10 client.17970 handle_command_reply: tid=0 2024-09-12T03:12:23.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.951+0000 7fa775c9d740 1 client.17970 shutdown 2024-09-12T03:12:23.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.951+0000 7fa775c9d740 1 -- 172.21.15.116:0/4056203603 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648f3b7b480 msgr2=0x5648f3b830a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:23.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.951+0000 7fa775c9d740 1 --2- 172.21.15.116:0/4056203603 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648f3b7b480 0x5648f3b830a0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fa7600cc010 tx=0x7fa7600046c0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.951+0000 7fa775c9d740 1 -- 172.21.15.116:0/4056203603 shutdown_connections 2024-09-12T03:12:23.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.951+0000 7fa775c9d740 1 --2- 172.21.15.116:0/4056203603 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5648f3b95890 0x5648f3b97d20 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.951+0000 7fa775c9d740 1 --2- 172.21.15.116:0/4056203603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f3b85d40 0x5648f3b835e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.951+0000 7fa775c9d740 1 --2- 172.21.15.116:0/4056203603 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648f3b7b480 0x5648f3b830a0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.951+0000 7fa775c9d740 1 --2- 172.21.15.116:0/4056203603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5648f3b7ab60 0x5648f3b82b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.951+0000 7fa775c9d740 1 -- 172.21.15.116:0/4056203603 >> 172.21.15.116:0/4056203603 conn(0x5648f3b770f0 msgr2=0x5648f3b7cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:23.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.951+0000 7fa754ff9640 0 client.17970 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:23.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.951+0000 7fa775c9d740 1 -- 172.21.15.116:0/4056203603 shutdown_connections 2024-09-12T03:12:23.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.951+0000 7fa775c9d740 1 -- 172.21.15.116:0/4056203603 wait complete. 2024-09-12T03:12:23.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.951+0000 7fa775c9d740 20 client.17970 trim_cache size 0 max 16384 2024-09-12T03:12:23.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.968+0000 7fa775c9d740 1 Processor -- start 2024-09-12T03:12:23.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.968+0000 7fa775c9d740 1 -- start start 2024-09-12T03:12:23.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.968+0000 7fa775c9d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5648f3b7ab60 0x5648f3b7ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:23.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.968+0000 7fa775c9d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f3b7b480 0x5648f3b9d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:23.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.968+0000 7fa775c9d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648f3b85d40 0x5648f3b9db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:23.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.968+0000 7fa775c9d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c1afdc0 con 0x5648f3b85d40 2024-09-12T03:12:23.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.968+0000 7fa775c9d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c1ad690 con 0x5648f3b7ab60 2024-09-12T03:12:23.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.968+0000 7fa775c9d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c1ab4e0 con 0x5648f3b7b480 2024-09-12T03:12:23.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.968+0000 7fa771e61640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648f3b85d40 0x5648f3b9db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:23.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.969+0000 7fa771e61640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648f3b85d40 0x5648f3b9db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40778/0 (socket says 172.21.15.116:40778) 2024-09-12T03:12:23.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.969+0000 7fa771e61640 1 -- 172.21.15.116:0/2624711487 learned_addr learned my addr 172.21.15.116:0/2624711487 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:23.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.969+0000 7fa770e5f640 1 --2- 172.21.15.116:0/2624711487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f3b7b480 0x5648f3b9d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:23.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.969+0000 7fa771660640 1 --2- 172.21.15.116:0/2624711487 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5648f3b7ab60 0x5648f3b7ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:23.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.969+0000 7fa771e61640 1 -- 172.21.15.116:0/2624711487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f3b7b480 msgr2=0x5648f3b9d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:23.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.969+0000 7fa771e61640 1 --2- 172.21.15.116:0/2624711487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f3b7b480 0x5648f3b9d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.969+0000 7fa771e61640 1 -- 172.21.15.116:0/2624711487 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5648f3b7ab60 msgr2=0x5648f3b7ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:23.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.969+0000 7fa771e61640 1 --2- 172.21.15.116:0/2624711487 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5648f3b7ab60 0x5648f3b7ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.969+0000 7fa771e61640 1 -- 172.21.15.116:0/2624711487 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648f3b76f20 con 0x5648f3b85d40 2024-09-12T03:12:23.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.969+0000 7fa771660640 1 --2- 172.21.15.116:0/2624711487 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5648f3b7ab60 0x5648f3b7ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:12:23.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.969+0000 7fa771e61640 1 --2- 172.21.15.116:0/2624711487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648f3b85d40 0x5648f3b9db00 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fa76402fb70 tx=0x7fa764040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1854d272f03608da server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:23.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.969+0000 7fa756ffd640 1 -- 172.21.15.116:0/2624711487 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa764034040 con 0x5648f3b85d40 2024-09-12T03:12:23.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.969+0000 7fa756ffd640 1 -- 172.21.15.116:0/2624711487 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa764003e50 con 0x5648f3b85d40 2024-09-12T03:12:23.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.969+0000 7fa756ffd640 1 -- 172.21.15.116:0/2624711487 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa764051480 con 0x5648f3b85d40 2024-09-12T03:12:23.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.970+0000 7fa775c9d740 1 -- 172.21.15.116:0/2624711487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648f3b85d40 msgr2=0x5648f3b9db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:23.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.970+0000 7fa775c9d740 1 --2- 172.21.15.116:0/2624711487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648f3b85d40 0x5648f3b9db00 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fa76402fb70 tx=0x7fa764040000 comp rx=0 tx=0).stop 2024-09-12T03:12:23.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.970+0000 7fa775c9d740 1 -- 172.21.15.116:0/2624711487 shutdown_connections 2024-09-12T03:12:23.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.970+0000 7fa775c9d740 1 --2- 172.21.15.116:0/2624711487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648f3b85d40 0x5648f3b9db00 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.970+0000 7fa775c9d740 1 --2- 172.21.15.116:0/2624711487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f3b7b480 0x5648f3b9d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.970+0000 7fa775c9d740 1 --2- 172.21.15.116:0/2624711487 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5648f3b7ab60 0x5648f3b7ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.970+0000 7fa775c9d740 1 -- 172.21.15.116:0/2624711487 >> 172.21.15.116:0/2624711487 conn(0x5648f3b770f0 msgr2=0x5648f3b7c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:23.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.970+0000 7fa775c9d740 1 -- 172.21.15.116:0/2624711487 shutdown_connections 2024-09-12T03:12:23.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.970+0000 7fa775c9d740 1 -- 172.21.15.116:0/2624711487 wait complete. 2024-09-12T03:12:23.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.970+0000 7fa775c9d740 1 -- start start 2024-09-12T03:12:23.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.970+0000 7fa775c9d740 1 Processor -- start 2024-09-12T03:12:23.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.971+0000 7fa775c9d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5648f3b7ab60 0x5648f3b8dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:23.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.971+0000 7fa775c9d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648f3b7b480 0x5648f3b8e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:23.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.971+0000 7fa775c9d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f3b85d40 0x5648f3b88ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:23.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.971+0000 7fa775c9d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c1abb50 con 0x5648f3b7b480 2024-09-12T03:12:23.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.971+0000 7fa775c9d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c1acfc0 con 0x5648f3b7ab60 2024-09-12T03:12:23.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.971+0000 7fa775c9d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa76c1ac220 con 0x5648f3b85d40 2024-09-12T03:12:23.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.971+0000 7fa775c9d740 10 client.0 _ll_register_callbacks cb 0x7fa760096fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:23.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.971+0000 7fa770e5f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648f3b7b480 0x5648f3b8e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:23.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.971+0000 7fa770e5f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648f3b7b480 0x5648f3b8e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40784/0 (socket says 172.21.15.116:40784) 2024-09-12T03:12:23.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.971+0000 7fa770e5f640 1 -- 172.21.15.116:0/1629504559 learned_addr learned my addr 172.21.15.116:0/1629504559 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:23.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.971+0000 7fa771e61640 1 --2- 172.21.15.116:0/1629504559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f3b85d40 0x5648f3b88ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:23.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.971+0000 7fa771660640 1 --2- 172.21.15.116:0/1629504559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5648f3b7ab60 0x5648f3b8dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:23.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.971+0000 7fa770e5f640 1 -- 172.21.15.116:0/1629504559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f3b85d40 msgr2=0x5648f3b88ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:23.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.971+0000 7fa770e5f640 1 --2- 172.21.15.116:0/1629504559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f3b85d40 0x5648f3b88ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.971+0000 7fa770e5f640 1 -- 172.21.15.116:0/1629504559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5648f3b7ab60 msgr2=0x5648f3b8dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:23.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.971+0000 7fa770e5f640 1 --2- 172.21.15.116:0/1629504559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5648f3b7ab60 0x5648f3b8dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.971+0000 7fa770e5f640 1 -- 172.21.15.116:0/1629504559 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648f3b8e8a0 con 0x5648f3b7b480 2024-09-12T03:12:23.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.971+0000 7fa770e5f640 1 --2- 172.21.15.116:0/1629504559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648f3b7b480 0x5648f3b8e230 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fa7600976f0 tx=0x7fa760017f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aeec3635701b8b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:23.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.971+0000 7fa754ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:12:23.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.972+0000 7fa754ff9640 1 -- 172.21.15.116:0/1629504559 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7600b7030 con 0x5648f3b7b480 2024-09-12T03:12:23.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.972+0000 7fa754ff9640 1 -- 172.21.15.116:0/1629504559 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa760010ca0 con 0x5648f3b7b480 2024-09-12T03:12:23.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.972+0000 7fa775c9d740 1 -- 172.21.15.116:0/1629504559 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa764003e50 con 0x5648f3b7b480 2024-09-12T03:12:23.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.972+0000 7fa754ff9640 1 -- 172.21.15.116:0/1629504559 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa760080050 con 0x5648f3b7b480 2024-09-12T03:12:23.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.972+0000 7fa775c9d740 10 client.17976 fetch_fsmap learned FSMap version 25 2024-09-12T03:12:23.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.972+0000 7fa775c9d740 1 -- 172.21.15.116:0/1629504559 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5648f3c04ee0 con 0x5648f3b7b480 2024-09-12T03:12:23.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.972+0000 7fa754ff9640 1 -- 172.21.15.116:0/1629504559 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa764003e50 con 0x5648f3b7b480 2024-09-12T03:12:23.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.972+0000 7fa754ff9640 1 -- 172.21.15.116:0/1629504559 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa760079c90 con 0x5648f3b7b480 2024-09-12T03:12:23.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.972+0000 7fa775c9d740 10 client.17976 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:12:23.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.972+0000 7fa775c9d740 10 client.17976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:12:23.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.972+0000 7fa775c9d740 20 client.17976 populate_metadata read hostname 'smithi116' 2024-09-12T03:12:23.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.972+0000 7fa775c9d740 1 --2- 172.21.15.116:0/1629504559 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5648f3c05690 0x5648f3c07a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:23.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.972+0000 7fa775c9d740 4 client.17976 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:12:23.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.972+0000 7fa775c9d740 1 -- 172.21.15.116:0/1629504559 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5648f3ae7d00 con 0x5648f3c05690 2024-09-12T03:12:23.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.972+0000 7fa771660640 1 --2- 172.21.15.116:0/1629504559 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5648f3c05690 0x5648f3c07a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:23.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.973+0000 7fa754ff9640 10 client.17976 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:23.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.973+0000 7fa771660640 1 --2- 172.21.15.116:0/1629504559 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5648f3c05690 0x5648f3c07a80 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d95e54d49ab0f407 server_cookie=6e48cec922de5a78 in_seq=0 out_seq=0 2024-09-12T03:12:23.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.974+0000 7fa754ff9640 1 -- 172.21.15.116:0/1629504559 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5648f3ae7d00 con 0x5648f3c05690 2024-09-12T03:12:23.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.974+0000 7fa754ff9640 10 client.17976 handle_command_reply: tid=0 2024-09-12T03:12:23.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.974+0000 7fa775c9d740 1 client.17976 shutdown 2024-09-12T03:12:23.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.974+0000 7fa775c9d740 1 -- 172.21.15.116:0/1629504559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648f3b7b480 msgr2=0x5648f3b8e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:23.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.974+0000 7fa775c9d740 1 --2- 172.21.15.116:0/1629504559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648f3b7b480 0x5648f3b8e230 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fa7600976f0 tx=0x7fa760017f20 comp rx=0 tx=0).stop 2024-09-12T03:12:23.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.974+0000 7fa775c9d740 1 -- 172.21.15.116:0/1629504559 shutdown_connections 2024-09-12T03:12:23.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.974+0000 7fa775c9d740 1 --2- 172.21.15.116:0/1629504559 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5648f3c05690 0x5648f3c07a80 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.974+0000 7fa775c9d740 1 --2- 172.21.15.116:0/1629504559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f3b85d40 0x5648f3b88ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.974+0000 7fa775c9d740 1 --2- 172.21.15.116:0/1629504559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5648f3b7b480 0x5648f3b8e230 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.974+0000 7fa775c9d740 1 --2- 172.21.15.116:0/1629504559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5648f3b7ab60 0x5648f3b8dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.974+0000 7fa775c9d740 1 -- 172.21.15.116:0/1629504559 >> 172.21.15.116:0/1629504559 conn(0x5648f3b770f0 msgr2=0x5648f3b7cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:23.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.974+0000 7fa754ff9640 0 client.17976 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:23.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.974+0000 7fa775c9d740 1 -- 172.21.15.116:0/1629504559 shutdown_connections 2024-09-12T03:12:23.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.974+0000 7fa775c9d740 1 -- 172.21.15.116:0/1629504559 wait complete. 2024-09-12T03:12:23.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.974+0000 7fa775c9d740 20 client.17976 trim_cache size 0 max 16384 2024-09-12T03:12:23.979 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:12:23.979 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:12:23.979 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "b59bb574-9dd6-4640-b7dc-aec39274695e", 2024-09-12T03:12:23.979 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:12:23.979 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:12:23.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.975+0000 7fa7738c4640 1 -- 172.21.15.116:0/1451018824 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa7580a2b80 msgr2=0x7fa7580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:23.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.975+0000 7fa7738c4640 1 --2- 172.21.15.116:0/1451018824 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa7580a2b80 0x7fa7580a5040 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa764030040 tx=0x7fa764005d80 comp rx=0 tx=0).stop 2024-09-12T03:12:23.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.975+0000 7fa7738c4640 1 -- 172.21.15.116:0/1451018824 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa76c102570 msgr2=0x7fa76c198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:23.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.975+0000 7fa7738c4640 1 --2- 172.21.15.116:0/1451018824 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa76c102570 0x7fa76c198bc0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fa76000b4a0 tx=0x7fa76000b960 comp rx=0 tx=0).stop 2024-09-12T03:12:23.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.975+0000 7fa7738c4640 1 -- 172.21.15.116:0/1451018824 shutdown_connections 2024-09-12T03:12:23.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.975+0000 7fa7738c4640 1 --2- 172.21.15.116:0/1451018824 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa7580a2b80 0x7fa7580a5040 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.975+0000 7fa7738c4640 1 --2- 172.21.15.116:0/1451018824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa76c102ef0 0x7fa76c19cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.975+0000 7fa7738c4640 1 --2- 172.21.15.116:0/1451018824 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa76c102570 0x7fa76c198bc0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.975+0000 7fa7738c4640 1 --2- 172.21.15.116:0/1451018824 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa76c100e20 0x7fa76c198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:23.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.975+0000 7fa7738c4640 1 -- 172.21.15.116:0/1451018824 >> 172.21.15.116:0/1451018824 conn(0x7fa76c0fc7d0 msgr2=0x7fa76c06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:23.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.975+0000 7fa7738c4640 1 -- 172.21.15.116:0/1451018824 shutdown_connections 2024-09-12T03:12:23.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:23.975+0000 7fa7738c4640 1 -- 172.21.15.116:0/1451018824 wait complete. 2024-09-12T03:12:23.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:23 smithi177 ceph-mon[35305]: pgmap v620: 97 pgs: 97 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 KiB/s wr, 9 op/s 2024-09-12T03:12:24.884 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:12:25.128 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:12:25.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:24 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:12:25.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:24 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:12:25.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:24 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:12:25.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:24 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:12:25.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:24 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:12:25.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:24 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:12:25.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:24 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:12:25.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:24 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:12:25.249 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:24 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:12:25.249 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:24 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:12:25.249 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:24 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:12:25.249 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:24 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:12:25.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.606+0000 7f7d7ebac640 1 -- 172.21.15.116:0/986821270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d78106a80 msgr2=0x7f7d78106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:25.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.606+0000 7f7d7ebac640 1 --2- 172.21.15.116:0/986821270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d78106a80 0x7f7d78106e60 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c009930 tx=0x7f7d6c02a420 comp rx=0 tx=0).stop 2024-09-12T03:12:25.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.606+0000 7f7d7ebac640 1 -- 172.21.15.116:0/986821270 shutdown_connections 2024-09-12T03:12:25.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.606+0000 7f7d7ebac640 1 --2- 172.21.15.116:0/986821270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d78101420 0x7f7d7810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.606+0000 7f7d7ebac640 1 --2- 172.21.15.116:0/986821270 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d78100a80 0x7f7d78100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.606+0000 7f7d7ebac640 1 --2- 172.21.15.116:0/986821270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d78106a80 0x7f7d78106e60 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.606+0000 7f7d7ebac640 1 -- 172.21.15.116:0/986821270 >> 172.21.15.116:0/986821270 conn(0x7f7d780fc7a0 msgr2=0x7f7d780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:25.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.606+0000 7f7d7ebac640 1 -- 172.21.15.116:0/986821270 shutdown_connections 2024-09-12T03:12:25.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.606+0000 7f7d7ebac640 1 -- 172.21.15.116:0/986821270 wait complete. 2024-09-12T03:12:25.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.607+0000 7f7d7ebac640 1 Processor -- start 2024-09-12T03:12:25.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.607+0000 7f7d7ebac640 1 -- start start 2024-09-12T03:12:25.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.607+0000 7f7d7ebac640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d78100a80 0x7f7d781974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:25.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.607+0000 7f7d7ebac640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d78101420 0x7f7d78197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:25.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.607+0000 7f7d7ebac640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d78106a80 0x7f7d7819bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:25.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.607+0000 7f7d7ebac640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7d78110f20 con 0x7f7d78100a80 2024-09-12T03:12:25.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.607+0000 7f7d7ebac640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7d78110da0 con 0x7f7d78101420 2024-09-12T03:12:25.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.607+0000 7f7d7ebac640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d781110a0 con 0x7f7d78106a80 2024-09-12T03:12:25.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.607+0000 7f7d7c948640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d78100a80 0x7f7d781974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:25.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.607+0000 7f7d7c948640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d78100a80 0x7f7d781974f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40802/0 (socket says 172.21.15.116:40802) 2024-09-12T03:12:25.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.607+0000 7f7d7c948640 1 -- 172.21.15.116:0/2482601127 learned_addr learned my addr 172.21.15.116:0/2482601127 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:25.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.607+0000 7f7d77fff640 1 --2- 172.21.15.116:0/2482601127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d78101420 0x7f7d78197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:25.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.608+0000 7f7d7c948640 1 -- 172.21.15.116:0/2482601127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d78106a80 msgr2=0x7f7d7819bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:25.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.608+0000 7f7d7c948640 1 --2- 172.21.15.116:0/2482601127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d78106a80 0x7f7d7819bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.608+0000 7f7d7c948640 1 -- 172.21.15.116:0/2482601127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d78101420 msgr2=0x7f7d78197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:25.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.608+0000 7f7d7c948640 1 --2- 172.21.15.116:0/2482601127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d78101420 0x7f7d78197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.608+0000 7f7d7c948640 1 -- 172.21.15.116:0/2482601127 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7819c4a0 con 0x7f7d78100a80 2024-09-12T03:12:25.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.608+0000 7f7d77fff640 1 --2- 172.21.15.116:0/2482601127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d78101420 0x7f7d78197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:12:25.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.608+0000 7f7d7c948640 1 --2- 172.21.15.116:0/2482601127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d78100a80 0x7f7d781974f0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c009a50 tx=0x7f7d6c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:25.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.608+0000 7f7d75ffb640 1 -- 172.21.15.116:0/2482601127 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c03f030 con 0x7f7d78100a80 2024-09-12T03:12:25.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.608+0000 7f7d75ffb640 1 -- 172.21.15.116:0/2482601127 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d6c003ce0 con 0x7f7d78100a80 2024-09-12T03:12:25.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.608+0000 7f7d7ebac640 1 -- 172.21.15.116:0/2482601127 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d7819c730 con 0x7f7d78100a80 2024-09-12T03:12:25.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.608+0000 7f7d75ffb640 1 -- 172.21.15.116:0/2482601127 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c035070 con 0x7f7d78100a80 2024-09-12T03:12:25.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.608+0000 7f7d7ebac640 1 -- 172.21.15.116:0/2482601127 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d781a4070 con 0x7f7d78100a80 2024-09-12T03:12:25.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.610+0000 7f7d75ffb640 1 -- 172.21.15.116:0/2482601127 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f7d6c042030 con 0x7f7d78100a80 2024-09-12T03:12:25.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.611+0000 7f7d80f85740 1 Processor -- start 2024-09-12T03:12:25.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.611+0000 7f7d80f85740 1 -- start start 2024-09-12T03:12:25.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.611+0000 7f7d75ffb640 1 --2- 172.21.15.116:0/2482601127 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7d680a2b80 0x7f7d680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:25.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.611+0000 7f7d80f85740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55989a22e4a0 0x55989a22e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:25.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.611+0000 7f7d80f85740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55989a22edc0 0x55989a239140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:25.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.611+0000 7f7d75ffb640 1 -- 172.21.15.116:0/2482601127 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f7d6c0ef680 con 0x7f7d78100a80 2024-09-12T03:12:25.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.611+0000 7f7d80f85740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55989a239680 0x55989a23ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:25.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.611+0000 7f7d80f85740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55989a1942e0 con 0x55989a22edc0 2024-09-12T03:12:25.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.611+0000 7f7d80f85740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55989a13bce0 con 0x55989a239680 2024-09-12T03:12:25.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.611+0000 7f7d80f85740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55989a1902a0 con 0x55989a22e4a0 2024-09-12T03:12:25.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.611+0000 7f7d7d149640 1 --2- 172.21.15.116:0/2482601127 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7d680a2b80 0x7f7d680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:25.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.611+0000 7f7d77fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55989a22edc0 0x55989a239140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:25.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.611+0000 7f7d77fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55989a22edc0 0x55989a239140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40816/0 (socket says 172.21.15.116:40816) 2024-09-12T03:12:25.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.611+0000 7f7d77fff640 1 -- 172.21.15.116:0/2851480044 learned_addr learned my addr 172.21.15.116:0/2851480044 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:25.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.611+0000 7f7d7c948640 1 --2- 172.21.15.116:0/2851480044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55989a22e4a0 0x55989a22e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:25.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.611+0000 7f7d7d149640 1 --2- 172.21.15.116:0/2851480044 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55989a239680 0x55989a23ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:25.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.611+0000 7f7d7d149640 1 --2- 172.21.15.116:0/2482601127 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7d680a2b80 0x7f7d680a5040 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000abc0 tx=0x7f7d7000c3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:25.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.611+0000 7f7d77fff640 1 -- 172.21.15.116:0/2851480044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55989a22e4a0 msgr2=0x55989a22e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:25.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.612+0000 7f7d77fff640 1 --2- 172.21.15.116:0/2851480044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55989a22e4a0 0x55989a22e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.612+0000 7f7d77fff640 1 -- 172.21.15.116:0/2851480044 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55989a239680 msgr2=0x55989a23ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:25.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.612+0000 7f7d77fff640 1 --2- 172.21.15.116:0/2851480044 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55989a239680 0x55989a23ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.612+0000 7f7d77fff640 1 -- 172.21.15.116:0/2851480044 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55989a120440 con 0x55989a22edc0 2024-09-12T03:12:25.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.612+0000 7f7d7d149640 1 --2- 172.21.15.116:0/2851480044 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55989a239680 0x55989a23ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:12:25.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.612+0000 7f7d77fff640 1 --2- 172.21.15.116:0/2851480044 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55989a22edc0 0x55989a239140 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400a9d0 tx=0x7f7d6400ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3faeefe0ed55528 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:25.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.612+0000 7f7d62ffd640 1 -- 172.21.15.116:0/2851480044 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d64015070 con 0x55989a22edc0 2024-09-12T03:12:25.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.612+0000 7f7d62ffd640 1 -- 172.21.15.116:0/2851480044 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d6400cc60 con 0x55989a22edc0 2024-09-12T03:12:25.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.612+0000 7f7d62ffd640 1 -- 172.21.15.116:0/2851480044 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d640105b0 con 0x55989a22edc0 2024-09-12T03:12:25.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.613+0000 7f7d80f85740 1 -- 172.21.15.116:0/2851480044 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55989a22edc0 msgr2=0x55989a239140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:25.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.613+0000 7f7d80f85740 1 --2- 172.21.15.116:0/2851480044 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55989a22edc0 0x55989a239140 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400a9d0 tx=0x7f7d6400ae90 comp rx=0 tx=0).stop 2024-09-12T03:12:25.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.613+0000 7f7d80f85740 1 -- 172.21.15.116:0/2851480044 shutdown_connections 2024-09-12T03:12:25.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.613+0000 7f7d80f85740 1 --2- 172.21.15.116:0/2851480044 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55989a239680 0x55989a23ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.613+0000 7f7d80f85740 1 --2- 172.21.15.116:0/2851480044 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55989a22edc0 0x55989a239140 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.613+0000 7f7d80f85740 1 --2- 172.21.15.116:0/2851480044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55989a22e4a0 0x55989a22e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.613+0000 7f7d80f85740 1 -- 172.21.15.116:0/2851480044 >> 172.21.15.116:0/2851480044 conn(0x55989a22aa30 msgr2=0x55989a22ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:25.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.613+0000 7f7d80f85740 1 -- 172.21.15.116:0/2851480044 shutdown_connections 2024-09-12T03:12:25.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.613+0000 7f7d80f85740 1 -- 172.21.15.116:0/2851480044 wait complete. 2024-09-12T03:12:25.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.613+0000 7f7d80f85740 1 -- start start 2024-09-12T03:12:25.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.613+0000 7f7d80f85740 1 Processor -- start 2024-09-12T03:12:25.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.613+0000 7f7d80f85740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55989a22e4a0 0x55989a238fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:25.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.613+0000 7f7d80f85740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55989a22edc0 0x55989a236270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:25.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.613+0000 7f7d80f85740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55989a239680 0x55989a2367b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:25.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.613+0000 7f7d80f85740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55989a23c250 con 0x55989a239680 2024-09-12T03:12:25.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.613+0000 7f7d80f85740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55989a14f050 con 0x55989a22e4a0 2024-09-12T03:12:25.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.613+0000 7f7d80f85740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55989a23c3d0 con 0x55989a22edc0 2024-09-12T03:12:25.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.614+0000 7f7d80f85740 10 client.0 _ll_register_callbacks cb 0x55989a18f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:25.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.614+0000 7f7d7d149640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55989a239680 0x55989a2367b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:25.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.614+0000 7f7d7d149640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55989a239680 0x55989a2367b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40818/0 (socket says 172.21.15.116:40818) 2024-09-12T03:12:25.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.614+0000 7f7d7d149640 1 -- 172.21.15.116:0/97168689 learned_addr learned my addr 172.21.15.116:0/97168689 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:25.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.614+0000 7f7d7c948640 1 --2- 172.21.15.116:0/97168689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55989a22e4a0 0x55989a238fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:25.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.614+0000 7f7d7d149640 1 -- 172.21.15.116:0/97168689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55989a22edc0 msgr2=0x55989a236270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:25.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.614+0000 7f7d77fff640 1 --2- 172.21.15.116:0/97168689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55989a22edc0 0x55989a236270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:25.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.614+0000 7f7d7d149640 1 --2- 172.21.15.116:0/97168689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55989a22edc0 0x55989a236270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.614+0000 7f7d7d149640 1 -- 172.21.15.116:0/97168689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55989a22e4a0 msgr2=0x55989a238fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:25.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.614+0000 7f7d7d149640 1 --2- 172.21.15.116:0/97168689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55989a22e4a0 0x55989a238fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.614+0000 7f7d7d149640 1 -- 172.21.15.116:0/97168689 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559899f053c0 con 0x55989a239680 2024-09-12T03:12:25.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.614+0000 7f7d7c948640 1 --2- 172.21.15.116:0/97168689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55989a22e4a0 0x55989a238fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:12:25.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.614+0000 7f7d7d149640 1 --2- 172.21.15.116:0/97168689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55989a239680 0x55989a2367b0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000cf40 tx=0x7f7d70004bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b53e2b2f364f26 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:25.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.614+0000 7f7d57fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:12:25.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.614+0000 7f7d57fff640 1 -- 172.21.15.116:0/97168689 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7001d860 con 0x55989a239680 2024-09-12T03:12:25.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.614+0000 7f7d57fff640 1 -- 172.21.15.116:0/97168689 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f7d7001da00 con 0x55989a239680 2024-09-12T03:12:25.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.614+0000 7f7d80f85740 1 -- 172.21.15.116:0/97168689 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d6400cc60 con 0x55989a239680 2024-09-12T03:12:25.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.614+0000 7f7d57fff640 1 -- 172.21.15.116:0/97168689 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7d7001dc40 con 0x55989a239680 2024-09-12T03:12:25.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.614+0000 7f7d80f85740 10 client.17988 fetch_fsmap learned FSMap version 25 2024-09-12T03:12:25.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.614+0000 7f7d80f85740 1 -- 172.21.15.116:0/97168689 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55989a247c60 con 0x55989a239680 2024-09-12T03:12:25.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.615+0000 7f7d57fff640 1 -- 172.21.15.116:0/97168689 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d6400cc60 con 0x55989a239680 2024-09-12T03:12:25.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.615+0000 7f7d57fff640 1 -- 172.21.15.116:0/97168689 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7d7001a440 con 0x55989a239680 2024-09-12T03:12:25.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.615+0000 7f7d80f85740 10 client.17988 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:12:25.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.615+0000 7f7d80f85740 10 client.17988 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:12:25.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.615+0000 7f7d80f85740 20 client.17988 populate_metadata read hostname 'smithi116' 2024-09-12T03:12:25.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.615+0000 7f7d80f85740 1 --2- 172.21.15.116:0/97168689 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55989a2490c0 0x55989a24b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:25.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.615+0000 7f7d80f85740 4 client.17988 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:12:25.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.615+0000 7f7d80f85740 1 -- 172.21.15.116:0/97168689 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55989a1a1850 con 0x55989a2490c0 2024-09-12T03:12:25.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.615+0000 7f7d7c948640 1 --2- 172.21.15.116:0/97168689 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55989a2490c0 0x55989a24b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:25.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.615+0000 7f7d57fff640 10 client.17988 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:25.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.615+0000 7f7d7c948640 1 --2- 172.21.15.116:0/97168689 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55989a2490c0 0x55989a24b580 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c83b89d858c4484 server_cookie=2a817fad8dc55f8d in_seq=0 out_seq=0 2024-09-12T03:12:25.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.616+0000 7f7d57fff640 1 -- 172.21.15.116:0/97168689 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55989a1a1850 con 0x55989a2490c0 2024-09-12T03:12:25.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.616+0000 7f7d57fff640 10 client.17988 handle_command_reply: tid=0 2024-09-12T03:12:25.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.616+0000 7f7d80f85740 1 client.17988 shutdown 2024-09-12T03:12:25.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.616+0000 7f7d80f85740 1 -- 172.21.15.116:0/97168689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55989a239680 msgr2=0x55989a2367b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:25.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.616+0000 7f7d80f85740 1 --2- 172.21.15.116:0/97168689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55989a239680 0x55989a2367b0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000cf40 tx=0x7f7d70004bc0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.616+0000 7f7d80f85740 1 -- 172.21.15.116:0/97168689 shutdown_connections 2024-09-12T03:12:25.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.616+0000 7f7d80f85740 1 --2- 172.21.15.116:0/97168689 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55989a2490c0 0x55989a24b580 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.616+0000 7f7d80f85740 1 --2- 172.21.15.116:0/97168689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55989a239680 0x55989a2367b0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.616+0000 7f7d80f85740 1 --2- 172.21.15.116:0/97168689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55989a22edc0 0x55989a236270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.616+0000 7f7d80f85740 1 --2- 172.21.15.116:0/97168689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55989a22e4a0 0x55989a238fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.616+0000 7f7d57fff640 0 client.17988 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:25.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.616+0000 7f7d80f85740 1 -- 172.21.15.116:0/97168689 >> 172.21.15.116:0/97168689 conn(0x55989a22aa30 msgr2=0x55989a230710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:25.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.616+0000 7f7d80f85740 1 -- 172.21.15.116:0/97168689 shutdown_connections 2024-09-12T03:12:25.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.616+0000 7f7d80f85740 1 -- 172.21.15.116:0/97168689 wait complete. 2024-09-12T03:12:25.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.617+0000 7f7d80f85740 20 client.17988 trim_cache size 0 max 16384 2024-09-12T03:12:25.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.633+0000 7f7d80f85740 1 Processor -- start 2024-09-12T03:12:25.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.633+0000 7f7d80f85740 1 -- start start 2024-09-12T03:12:25.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.634+0000 7f7d80f85740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55989a22e4a0 0x55989a2454b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:25.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.634+0000 7f7d80f85740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55989a22edc0 0x55989a231900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:25.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.634+0000 7f7d80f85740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55989a239680 0x55989a2381e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:25.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.634+0000 7f7d80f85740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7d781adc50 con 0x55989a22edc0 2024-09-12T03:12:25.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.634+0000 7f7d80f85740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7d781ab520 con 0x55989a239680 2024-09-12T03:12:25.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.634+0000 7f7d80f85740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d781a9390 con 0x55989a22e4a0 2024-09-12T03:12:25.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.634+0000 7f7d77fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55989a22edc0 0x55989a231900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:25.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.634+0000 7f7d77fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55989a22edc0 0x55989a231900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40822/0 (socket says 172.21.15.116:40822) 2024-09-12T03:12:25.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.634+0000 7f7d77fff640 1 -- 172.21.15.116:0/4253390885 learned_addr learned my addr 172.21.15.116:0/4253390885 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:25.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.634+0000 7f7d7c948640 1 --2- 172.21.15.116:0/4253390885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55989a22e4a0 0x55989a2454b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:25.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.634+0000 7f7d7d149640 1 --2- 172.21.15.116:0/4253390885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55989a239680 0x55989a2381e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:25.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.634+0000 7f7d77fff640 1 -- 172.21.15.116:0/4253390885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55989a22e4a0 msgr2=0x55989a2454b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:25.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.634+0000 7f7d77fff640 1 --2- 172.21.15.116:0/4253390885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55989a22e4a0 0x55989a2454b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.634+0000 7f7d77fff640 1 -- 172.21.15.116:0/4253390885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55989a239680 msgr2=0x55989a2381e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:25.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.634+0000 7f7d77fff640 1 --2- 172.21.15.116:0/4253390885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55989a239680 0x55989a2381e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.634+0000 7f7d77fff640 1 -- 172.21.15.116:0/4253390885 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55989a235220 con 0x55989a22edc0 2024-09-12T03:12:25.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.634+0000 7f7d7d149640 1 --2- 172.21.15.116:0/4253390885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55989a239680 0x55989a2381e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:12:25.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.634+0000 7f7d77fff640 1 --2- 172.21.15.116:0/4253390885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55989a22edc0 0x55989a231900 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f7d64004810 tx=0x7f7d6400a670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c658aaaeb5057ae server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:25.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.634+0000 7f7d627fc640 1 -- 172.21.15.116:0/4253390885 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d64015040 con 0x55989a22edc0 2024-09-12T03:12:25.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.634+0000 7f7d627fc640 1 -- 172.21.15.116:0/4253390885 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d6400cc60 con 0x55989a22edc0 2024-09-12T03:12:25.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.634+0000 7f7d627fc640 1 -- 172.21.15.116:0/4253390885 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d64011670 con 0x55989a22edc0 2024-09-12T03:12:25.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.635+0000 7f7d80f85740 1 -- 172.21.15.116:0/4253390885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55989a22edc0 msgr2=0x55989a231900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:25.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.635+0000 7f7d80f85740 1 --2- 172.21.15.116:0/4253390885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55989a22edc0 0x55989a231900 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f7d64004810 tx=0x7f7d6400a670 comp rx=0 tx=0).stop 2024-09-12T03:12:25.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.635+0000 7f7d80f85740 1 -- 172.21.15.116:0/4253390885 shutdown_connections 2024-09-12T03:12:25.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.635+0000 7f7d80f85740 1 --2- 172.21.15.116:0/4253390885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55989a239680 0x55989a2381e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.635+0000 7f7d80f85740 1 --2- 172.21.15.116:0/4253390885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55989a22edc0 0x55989a231900 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.635+0000 7f7d80f85740 1 --2- 172.21.15.116:0/4253390885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55989a22e4a0 0x55989a2454b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.635+0000 7f7d80f85740 1 -- 172.21.15.116:0/4253390885 >> 172.21.15.116:0/4253390885 conn(0x55989a22aa30 msgr2=0x55989a237ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:25.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.635+0000 7f7d80f85740 1 -- 172.21.15.116:0/4253390885 shutdown_connections 2024-09-12T03:12:25.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.635+0000 7f7d80f85740 1 -- 172.21.15.116:0/4253390885 wait complete. 2024-09-12T03:12:25.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.635+0000 7f7d80f85740 1 -- start start 2024-09-12T03:12:25.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.635+0000 7f7d80f85740 1 Processor -- start 2024-09-12T03:12:25.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.636+0000 7f7d80f85740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55989a22e4a0 0x55989a23dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:25.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.636+0000 7f7d80f85740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55989a22edc0 0x55989a23e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:25.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.636+0000 7f7d80f85740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55989a239680 0x55989a2b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:25.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.636+0000 7f7d80f85740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7d781a99e0 con 0x55989a22edc0 2024-09-12T03:12:25.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.636+0000 7f7d80f85740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7d781aae50 con 0x55989a239680 2024-09-12T03:12:25.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.636+0000 7f7d80f85740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d781aa0b0 con 0x55989a22e4a0 2024-09-12T03:12:25.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.636+0000 7f7d80f85740 10 client.0 _ll_register_callbacks cb 0x7f7d7000cc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:25.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.636+0000 7f7d77fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55989a22edc0 0x55989a23e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:25.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.636+0000 7f7d77fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55989a22edc0 0x55989a23e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40830/0 (socket says 172.21.15.116:40830) 2024-09-12T03:12:25.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.636+0000 7f7d77fff640 1 -- 172.21.15.116:0/31307424 learned_addr learned my addr 172.21.15.116:0/31307424 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:25.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.636+0000 7f7d7c948640 1 --2- 172.21.15.116:0/31307424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55989a22e4a0 0x55989a23dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:25.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.636+0000 7f7d77fff640 1 -- 172.21.15.116:0/31307424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55989a22e4a0 msgr2=0x55989a23dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:25.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.636+0000 7f7d77fff640 1 --2- 172.21.15.116:0/31307424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55989a22e4a0 0x55989a23dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.636+0000 7f7d77fff640 1 -- 172.21.15.116:0/31307424 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55989a239680 msgr2=0x55989a2b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:25.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.636+0000 7f7d77fff640 1 --2- 172.21.15.116:0/31307424 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55989a239680 0x55989a2b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.636+0000 7f7d77fff640 1 -- 172.21.15.116:0/31307424 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55989a2b3360 con 0x55989a22edc0 2024-09-12T03:12:25.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.636+0000 7f7d77fff640 1 --2- 172.21.15.116:0/31307424 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55989a22edc0 0x55989a23e020 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f7d64020000 tx=0x7f7d6401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ac40aefee69338 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:25.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.636+0000 7f7d57fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:12:25.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.637+0000 7f7d57fff640 1 -- 172.21.15.116:0/31307424 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d64032060 con 0x55989a22edc0 2024-09-12T03:12:25.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.637+0000 7f7d57fff640 1 -- 172.21.15.116:0/31307424 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f7d64023650 con 0x55989a22edc0 2024-09-12T03:12:25.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.637+0000 7f7d80f85740 1 -- 172.21.15.116:0/31307424 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d6400cc60 con 0x55989a22edc0 2024-09-12T03:12:25.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.637+0000 7f7d57fff640 1 -- 172.21.15.116:0/31307424 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7d64015040 con 0x55989a22edc0 2024-09-12T03:12:25.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.637+0000 7f7d80f85740 10 client.17994 fetch_fsmap learned FSMap version 25 2024-09-12T03:12:25.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.637+0000 7f7d80f85740 1 -- 172.21.15.116:0/31307424 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55989a2b4640 con 0x55989a22edc0 2024-09-12T03:12:25.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.637+0000 7f7d57fff640 1 -- 172.21.15.116:0/31307424 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d6400cc60 con 0x55989a22edc0 2024-09-12T03:12:25.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.637+0000 7f7d57fff640 1 -- 172.21.15.116:0/31307424 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7d64021070 con 0x55989a22edc0 2024-09-12T03:12:25.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.637+0000 7f7d80f85740 10 client.17994 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:12:25.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.637+0000 7f7d80f85740 10 client.17994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:12:25.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.637+0000 7f7d80f85740 20 client.17994 populate_metadata read hostname 'smithi116' 2024-09-12T03:12:25.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.637+0000 7f7d80f85740 1 --2- 172.21.15.116:0/31307424 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55989a2b49f0 0x55989a2b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:25.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.637+0000 7f7d80f85740 4 client.17994 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:12:25.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.637+0000 7f7d80f85740 1 -- 172.21.15.116:0/31307424 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55989a19bd60 con 0x55989a2b49f0 2024-09-12T03:12:25.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.638+0000 7f7d7c948640 1 --2- 172.21.15.116:0/31307424 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55989a2b49f0 0x55989a2b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:25.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.638+0000 7f7d7c948640 1 --2- 172.21.15.116:0/31307424 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55989a2b49f0 0x55989a2b6e10 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=509043609c2f34a1 server_cookie=1f6a17375e8c7e6b in_seq=0 out_seq=0 2024-09-12T03:12:25.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.638+0000 7f7d57fff640 10 client.17994 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:25.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.638+0000 7f7d57fff640 1 -- 172.21.15.116:0/31307424 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55989a19bd60 con 0x55989a2b49f0 2024-09-12T03:12:25.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.638+0000 7f7d57fff640 10 client.17994 handle_command_reply: tid=0 2024-09-12T03:12:25.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.638+0000 7f7d80f85740 1 client.17994 shutdown 2024-09-12T03:12:25.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.638+0000 7f7d80f85740 1 -- 172.21.15.116:0/31307424 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55989a22edc0 msgr2=0x55989a23e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:25.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.638+0000 7f7d80f85740 1 --2- 172.21.15.116:0/31307424 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55989a22edc0 0x55989a23e020 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f7d64020000 tx=0x7f7d6401acb0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.638+0000 7f7d80f85740 1 -- 172.21.15.116:0/31307424 shutdown_connections 2024-09-12T03:12:25.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.638+0000 7f7d80f85740 1 --2- 172.21.15.116:0/31307424 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55989a2b49f0 0x55989a2b6e10 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.638+0000 7f7d80f85740 1 --2- 172.21.15.116:0/31307424 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55989a239680 0x55989a2b2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.638+0000 7f7d80f85740 1 --2- 172.21.15.116:0/31307424 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55989a22edc0 0x55989a23e020 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.638+0000 7f7d80f85740 1 --2- 172.21.15.116:0/31307424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55989a22e4a0 0x55989a23dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.638+0000 7f7d80f85740 1 -- 172.21.15.116:0/31307424 >> 172.21.15.116:0/31307424 conn(0x55989a22aa30 msgr2=0x55989a237ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:25.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.638+0000 7f7d57fff640 0 client.17994 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:25.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.638+0000 7f7d80f85740 1 -- 172.21.15.116:0/31307424 shutdown_connections 2024-09-12T03:12:25.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.638+0000 7f7d80f85740 1 -- 172.21.15.116:0/31307424 wait complete. 2024-09-12T03:12:25.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.638+0000 7f7d80f85740 20 client.17994 trim_cache size 0 max 16384 2024-09-12T03:12:25.648 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:12:25.648 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:12:25.648 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": { 2024-09-12T03:12:25.648 INFO:teuthology.orchestra.run.smithi116.stdout: "b59bb574-9dd6-4640-b7dc-aec39274695e": { 2024-09-12T03:12:25.648 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/", 2024-09-12T03:12:25.648 INFO:teuthology.orchestra.run.smithi116.stdout: "tag": "b59bb574-9dd6-4640-b7dc-aec39274695e", 2024-09-12T03:12:25.648 INFO:teuthology.orchestra.run.smithi116.stdout: "options": "recursive,force" 2024-09-12T03:12:25.648 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:12:25.648 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:12:25.648 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:12:25.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.639+0000 7f7d7ebac640 1 -- 172.21.15.116:0/2482601127 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7d680a2b80 msgr2=0x7f7d680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:25.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.639+0000 7f7d7ebac640 1 --2- 172.21.15.116:0/2482601127 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7d680a2b80 0x7f7d680a5040 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000abc0 tx=0x7f7d7000c3b0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.639+0000 7f7d7ebac640 1 -- 172.21.15.116:0/2482601127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d78100a80 msgr2=0x7f7d781974f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:25.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.639+0000 7f7d7ebac640 1 --2- 172.21.15.116:0/2482601127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d78100a80 0x7f7d781974f0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c009a50 tx=0x7f7d6c004270 comp rx=0 tx=0).stop 2024-09-12T03:12:25.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.639+0000 7f7d7ebac640 1 -- 172.21.15.116:0/2482601127 shutdown_connections 2024-09-12T03:12:25.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.639+0000 7f7d7ebac640 1 --2- 172.21.15.116:0/2482601127 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7d680a2b80 0x7f7d680a5040 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.639+0000 7f7d7ebac640 1 --2- 172.21.15.116:0/2482601127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d78106a80 0x7f7d7819bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.639+0000 7f7d7ebac640 1 --2- 172.21.15.116:0/2482601127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d78101420 0x7f7d78197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.639+0000 7f7d7ebac640 1 --2- 172.21.15.116:0/2482601127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d78100a80 0x7f7d781974f0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.639+0000 7f7d7ebac640 1 -- 172.21.15.116:0/2482601127 >> 172.21.15.116:0/2482601127 conn(0x7f7d780fc7a0 msgr2=0x7f7d780fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:25.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.639+0000 7f7d7ebac640 1 -- 172.21.15.116:0/2482601127 shutdown_connections 2024-09-12T03:12:25.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:25.639+0000 7f7d7ebac640 1 -- 172.21.15.116:0/2482601127 wait complete. 2024-09-12T03:12:26.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:25 smithi183 ceph-mon[35535]: pgmap v621: 97 pgs: 97 active+clean; 4.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-12T03:12:26.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:25 smithi177 ceph-mon[35305]: pgmap v621: 97 pgs: 97 active+clean; 4.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-12T03:12:26.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:25 smithi116 ceph-mon[31410]: pgmap v621: 97 pgs: 97 active+clean; 4.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-12T03:12:26.450 INFO:tasks.cephfs.filesystem:scrub status for tag:b59bb574-9dd6-4640-b7dc-aec39274695e - {'path': '/', 'tag': 'b59bb574-9dd6-4640-b7dc-aec39274695e', 'options': 'recursive,force'} 2024-09-12T03:12:27.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:27 smithi183 ceph-mon[35535]: pgmap v622: 97 pgs: 97 active+clean; 4.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-09-12T03:12:27.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:27 smithi177 ceph-mon[35305]: pgmap v622: 97 pgs: 97 active+clean; 4.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-09-12T03:12:27.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:27 smithi116 ceph-mon[31410]: pgmap v622: 97 pgs: 97 active+clean; 4.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-09-12T03:12:29.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:28 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:12:29.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:28 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:12:29.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:28 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:12:30.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:29 smithi183 ceph-mon[35535]: pgmap v623: 97 pgs: 97 active+clean; 4.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:12:30.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:29 smithi177 ceph-mon[35305]: pgmap v623: 97 pgs: 97 active+clean; 4.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:12:30.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:29 smithi116 ceph-mon[31410]: pgmap v623: 97 pgs: 97 active+clean; 4.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:12:31.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:31 smithi183 ceph-mon[35535]: pgmap v624: 97 pgs: 97 active+clean; 4.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T03:12:31.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:31 smithi177 ceph-mon[35305]: pgmap v624: 97 pgs: 97 active+clean; 4.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T03:12:31.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:31 smithi116 ceph-mon[31410]: pgmap v624: 97 pgs: 97 active+clean; 4.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-12T03:12:33.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:33 smithi183 ceph-mon[35535]: pgmap v625: 97 pgs: 97 active+clean; 4.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:12:33.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:33 smithi177 ceph-mon[35305]: pgmap v625: 97 pgs: 97 active+clean; 4.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:12:33.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:33 smithi116 ceph-mon[31410]: pgmap v625: 97 pgs: 97 active+clean; 4.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:12:35.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:35 smithi183 ceph-mon[35535]: pgmap v626: 97 pgs: 97 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:12:35.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:35 smithi177 ceph-mon[35305]: pgmap v626: 97 pgs: 97 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:12:35.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:35 smithi116 ceph-mon[31410]: pgmap v626: 97 pgs: 97 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-09-12T03:12:37.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:37 smithi183 ceph-mon[35535]: pgmap v627: 97 pgs: 97 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:12:37.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:37 smithi177 ceph-mon[35305]: pgmap v627: 97 pgs: 97 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:12:37.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:37 smithi116 ceph-mon[31410]: pgmap v627: 97 pgs: 97 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:12:39.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:39 smithi183 ceph-mon[35535]: pgmap v628: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:12:39.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:39 smithi177 ceph-mon[35305]: pgmap v628: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:12:39.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:39 smithi116 ceph-mon[31410]: pgmap v628: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:12:41.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:41 smithi183 ceph-mon[35535]: pgmap v629: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:12:41.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:41 smithi177 ceph-mon[35305]: pgmap v629: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:12:41.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:41 smithi116 ceph-mon[31410]: pgmap v629: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:12:42.459 INFO:tasks.quiescer.fs.[cephfs]:Total duration (45.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T03:12:42.459 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.55 2024-09-12T03:12:42.459 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:12:42.718 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:12:43.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.194+0000 7f6d0b83e640 1 -- 172.21.15.116:0/4221749856 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d04103740 msgr2=0x7f6d04107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:43.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.194+0000 7f6d0b83e640 1 --2- 172.21.15.116:0/4221749856 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d04103740 0x7f6d04107b80 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8009930 tx=0x7f6cf802a420 comp rx=0 tx=0).stop 2024-09-12T03:12:43.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.194+0000 7f6d0b83e640 1 -- 172.21.15.116:0/4221749856 shutdown_connections 2024-09-12T03:12:43.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.194+0000 7f6d0b83e640 1 --2- 172.21.15.116:0/4221749856 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d04108290 0x7f6d0410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:43.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.194+0000 7f6d0b83e640 1 --2- 172.21.15.116:0/4221749856 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d04103740 0x7f6d04107b80 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:43.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.194+0000 7f6d0b83e640 1 --2- 172.21.15.116:0/4221749856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d04102d90 0x7f6d04103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:43.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.194+0000 7f6d0b83e640 1 -- 172.21.15.116:0/4221749856 >> 172.21.15.116:0/4221749856 conn(0x7f6d040fc7d0 msgr2=0x7f6d040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:43.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.194+0000 7f6d0b83e640 1 -- 172.21.15.116:0/4221749856 shutdown_connections 2024-09-12T03:12:43.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.194+0000 7f6d0b83e640 1 -- 172.21.15.116:0/4221749856 wait complete. 2024-09-12T03:12:43.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.194+0000 7f6d0b83e640 1 Processor -- start 2024-09-12T03:12:43.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.194+0000 7f6d0b83e640 1 -- start start 2024-09-12T03:12:43.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.195+0000 7f6d0b83e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d04102d90 0x7f6d04198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:43.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.195+0000 7f6d0b83e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d04103740 0x7f6d04198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:43.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.195+0000 7f6d0b83e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d04108290 0x7f6d0419cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:43.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.195+0000 7f6d0b83e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6d04112090 con 0x7f6d04103740 2024-09-12T03:12:43.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.195+0000 7f6d0b83e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6d04111f10 con 0x7f6d04108290 2024-09-12T03:12:43.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.195+0000 7f6d0b83e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6d04112210 con 0x7f6d04102d90 2024-09-12T03:12:43.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.195+0000 7f6d08dd9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d04103740 0x7f6d04198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:43.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.195+0000 7f6d08dd9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d04103740 0x7f6d04198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53736/0 (socket says 172.21.15.116:53736) 2024-09-12T03:12:43.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.195+0000 7f6d08dd9640 1 -- 172.21.15.116:0/316311130 learned_addr learned my addr 172.21.15.116:0/316311130 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:43.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.195+0000 7f6d095da640 1 --2- 172.21.15.116:0/316311130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d04102d90 0x7f6d04198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:43.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.195+0000 7f6d09ddb640 1 --2- 172.21.15.116:0/316311130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d04108290 0x7f6d0419cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:43.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.195+0000 7f6d08dd9640 1 -- 172.21.15.116:0/316311130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d04102d90 msgr2=0x7f6d04198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:43.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.195+0000 7f6d08dd9640 1 --2- 172.21.15.116:0/316311130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d04102d90 0x7f6d04198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:43.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.195+0000 7f6d08dd9640 1 -- 172.21.15.116:0/316311130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d04108290 msgr2=0x7f6d0419cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:43.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.195+0000 7f6d08dd9640 1 --2- 172.21.15.116:0/316311130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d04108290 0x7f6d0419cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:43.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.195+0000 7f6d08dd9640 1 -- 172.21.15.116:0/316311130 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0419d5c0 con 0x7f6d04103740 2024-09-12T03:12:43.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.195+0000 7f6d08dd9640 1 --2- 172.21.15.116:0/316311130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d04103740 0x7f6d04198b50 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8005940 tx=0x7f6cf80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:43.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.195+0000 7f6d027fc640 1 -- 172.21.15.116:0/316311130 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf803f030 con 0x7f6d04103740 2024-09-12T03:12:43.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.195+0000 7f6d027fc640 1 -- 172.21.15.116:0/316311130 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cf8003ce0 con 0x7f6d04103740 2024-09-12T03:12:43.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.195+0000 7f6d027fc640 1 -- 172.21.15.116:0/316311130 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf8035070 con 0x7f6d04103740 2024-09-12T03:12:43.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.195+0000 7f6d0b83e640 1 -- 172.21.15.116:0/316311130 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d0419d850 con 0x7f6d04103740 2024-09-12T03:12:43.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.195+0000 7f6d0b83e640 1 -- 172.21.15.116:0/316311130 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d041a5190 con 0x7f6d04103740 2024-09-12T03:12:43.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.196+0000 7f6d0b83e640 1 -- 172.21.15.116:0/316311130 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d0406a7f0 con 0x7f6d04103740 2024-09-12T03:12:43.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.197+0000 7f6d027fc640 1 -- 172.21.15.116:0/316311130 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6cf8042030 con 0x7f6d04103740 2024-09-12T03:12:43.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.198+0000 7f6d027fc640 1 --2- 172.21.15.116:0/316311130 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6cec0a2b80 0x7f6cec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:43.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.198+0000 7f6d095da640 1 --2- 172.21.15.116:0/316311130 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6cec0a2b80 0x7f6cec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:43.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.198+0000 7f6d027fc640 1 -- 172.21.15.116:0/316311130 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f6cf80ef580 con 0x7f6d04103740 2024-09-12T03:12:43.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.199+0000 7f6d095da640 1 --2- 172.21.15.116:0/316311130 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6cec0a2b80 0x7f6cec0a5040 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4005fd0 tx=0x7f6cf400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:43.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.202+0000 7f6d027fc640 1 -- 172.21.15.116:0/316311130 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6cf80bc510 con 0x7f6d04103740 2024-09-12T03:12:43.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.405+0000 7f6d0b83e640 1 -- 172.21.15.116:0/316311130 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d0410a2a0 con 0x7f6d04103740 2024-09-12T03:12:43.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.406+0000 7f6d027fc640 1 -- 172.21.15.116:0/316311130 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f6cf80bbe10 con 0x7f6d04103740 2024-09-12T03:12:43.407 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:12:43.407 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:12:43.408 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:12:43.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.408+0000 7f6d0b83e640 1 -- 172.21.15.116:0/316311130 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6cec0a2b80 msgr2=0x7f6cec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:43.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.408+0000 7f6d0b83e640 1 --2- 172.21.15.116:0/316311130 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6cec0a2b80 0x7f6cec0a5040 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4005fd0 tx=0x7f6cf400b390 comp rx=0 tx=0).stop 2024-09-12T03:12:43.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.408+0000 7f6d0b83e640 1 -- 172.21.15.116:0/316311130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d04103740 msgr2=0x7f6d04198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:43.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.408+0000 7f6d0b83e640 1 --2- 172.21.15.116:0/316311130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d04103740 0x7f6d04198b50 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8005940 tx=0x7f6cf80042f0 comp rx=0 tx=0).stop 2024-09-12T03:12:43.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.408+0000 7f6d0b83e640 1 -- 172.21.15.116:0/316311130 shutdown_connections 2024-09-12T03:12:43.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.408+0000 7f6d0b83e640 1 --2- 172.21.15.116:0/316311130 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6cec0a2b80 0x7f6cec0a5040 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:43.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.408+0000 7f6d0b83e640 1 --2- 172.21.15.116:0/316311130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d04108290 0x7f6d0419cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:43.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.408+0000 7f6d0b83e640 1 --2- 172.21.15.116:0/316311130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d04103740 0x7f6d04198b50 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:43.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.408+0000 7f6d0b83e640 1 --2- 172.21.15.116:0/316311130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d04102d90 0x7f6d04198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:43.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.408+0000 7f6d0b83e640 1 -- 172.21.15.116:0/316311130 >> 172.21.15.116:0/316311130 conn(0x7f6d040fc7d0 msgr2=0x7f6d04105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:43.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.408+0000 7f6d0b83e640 1 -- 172.21.15.116:0/316311130 shutdown_connections 2024-09-12T03:12:43.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:43.408+0000 7f6d0b83e640 1 -- 172.21.15.116:0/316311130 wait complete. 2024-09-12T03:12:43.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:43 smithi183 ceph-mon[35535]: pgmap v630: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:12:43.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:43 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/316311130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:12:43.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:43 smithi177 ceph-mon[35305]: pgmap v630: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:12:43.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:43 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/316311130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:12:43.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:43 smithi116 ceph-mon[31410]: pgmap v630: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:12:43.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:43 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/316311130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:12:44.225 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0' 2024-09-12T03:12:44.225 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0 2024-09-12T03:12:44.477 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:12:44.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.973+0000 7fe719809640 1 -- 172.21.15.116:0/1193911882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe714106050 msgr2=0x7fe71410dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:44.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.973+0000 7fe719809640 1 --2- 172.21.15.116:0/1193911882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe714106050 0x7fe71410dbd0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fe70c009900 tx=0x7fe70c02a410 comp rx=0 tx=0).stop 2024-09-12T03:12:44.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.973+0000 7fe719809640 1 -- 172.21.15.116:0/1193911882 shutdown_connections 2024-09-12T03:12:44.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.973+0000 7fe719809640 1 --2- 172.21.15.116:0/1193911882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe714106050 0x7fe71410dbd0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:44.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.973+0000 7fe719809640 1 --2- 172.21.15.116:0/1193911882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe714101df0 0x7fe714105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:44.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.973+0000 7fe719809640 1 --2- 172.21.15.116:0/1193911882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe714101440 0x7fe714101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:44.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.973+0000 7fe719809640 1 -- 172.21.15.116:0/1193911882 >> 172.21.15.116:0/1193911882 conn(0x7fe714077210 msgr2=0x7fe7140fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:44.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.973+0000 7fe719809640 1 -- 172.21.15.116:0/1193911882 shutdown_connections 2024-09-12T03:12:44.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.973+0000 7fe719809640 1 -- 172.21.15.116:0/1193911882 wait complete. 2024-09-12T03:12:44.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.974+0000 7fe719809640 1 Processor -- start 2024-09-12T03:12:44.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.974+0000 7fe719809640 1 -- start start 2024-09-12T03:12:44.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.974+0000 7fe719809640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe714101440 0x7fe7140fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:44.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.974+0000 7fe719809640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe714101df0 0x7fe7140fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:44.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.974+0000 7fe719809640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe714106050 0x7fe7140fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:44.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.974+0000 7fe719809640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe714111050 con 0x7fe714101440 2024-09-12T03:12:44.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.974+0000 7fe719809640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe714110ed0 con 0x7fe714101df0 2024-09-12T03:12:44.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.974+0000 7fe719809640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141111d0 con 0x7fe714106050 2024-09-12T03:12:44.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.974+0000 7fe712ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe714101440 0x7fe7140fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:44.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.974+0000 7fe712ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe714101440 0x7fe7140fc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53756/0 (socket says 172.21.15.116:53756) 2024-09-12T03:12:44.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.974+0000 7fe712ffd640 1 -- 172.21.15.116:0/2156658835 learned_addr learned my addr 172.21.15.116:0/2156658835 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:44.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.975+0000 7fe7127fc640 1 --2- 172.21.15.116:0/2156658835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe714101df0 0x7fe7140fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:44.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.975+0000 7fe7137fe640 1 --2- 172.21.15.116:0/2156658835 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe714106050 0x7fe7140fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:44.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.975+0000 7fe712ffd640 1 -- 172.21.15.116:0/2156658835 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe714106050 msgr2=0x7fe7140fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:44.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.975+0000 7fe712ffd640 1 --2- 172.21.15.116:0/2156658835 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe714106050 0x7fe7140fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:44.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.975+0000 7fe712ffd640 1 -- 172.21.15.116:0/2156658835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe714101df0 msgr2=0x7fe7140fcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:44.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.975+0000 7fe712ffd640 1 --2- 172.21.15.116:0/2156658835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe714101df0 0x7fe7140fcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:44.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.975+0000 7fe712ffd640 1 -- 172.21.15.116:0/2156658835 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7141a6ad0 con 0x7fe714101440 2024-09-12T03:12:44.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.975+0000 7fe712ffd640 1 --2- 172.21.15.116:0/2156658835 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe714101440 0x7fe7140fc690 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fe70400eab0 tx=0x7fe70400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:44.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.975+0000 7fe718807640 1 -- 172.21.15.116:0/2156658835 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70400cd80 con 0x7fe714101440 2024-09-12T03:12:44.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.975+0000 7fe718807640 1 -- 172.21.15.116:0/2156658835 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe704011950 con 0x7fe714101440 2024-09-12T03:12:44.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.975+0000 7fe719809640 1 -- 172.21.15.116:0/2156658835 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7141a6dc0 con 0x7fe714101440 2024-09-12T03:12:44.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.975+0000 7fe718807640 1 -- 172.21.15.116:0/2156658835 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe704010a00 con 0x7fe714101440 2024-09-12T03:12:44.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.975+0000 7fe719809640 1 -- 172.21.15.116:0/2156658835 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7141a7280 con 0x7fe714101440 2024-09-12T03:12:44.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.978+0000 7fe718807640 1 -- 172.21.15.116:0/2156658835 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe704011b00 con 0x7fe714101440 2024-09-12T03:12:44.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.978+0000 7fe71bbe2740 1 Processor -- start 2024-09-12T03:12:44.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.978+0000 7fe71bbe2740 1 -- start start 2024-09-12T03:12:44.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.978+0000 7fe71bbe2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8f78cd850 0x55d8f78cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:44.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.978+0000 7fe71bbe2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8f78ce170 0x55d8f78d84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:44.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.978+0000 7fe71bbe2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8f78d8a30 0x55d8f78dae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:44.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.978+0000 7fe71bbe2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d8f7832160 con 0x55d8f78ce170 2024-09-12T03:12:44.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.978+0000 7fe71bbe2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d8f77d9720 con 0x55d8f78d8a30 2024-09-12T03:12:44.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.978+0000 7fe71bbe2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d8f782e120 con 0x55d8f78cd850 2024-09-12T03:12:44.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.978+0000 7fe712ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8f78ce170 0x55d8f78d84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:44.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.978+0000 7fe712ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8f78ce170 0x55d8f78d84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53768/0 (socket says 172.21.15.116:53768) 2024-09-12T03:12:44.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.978+0000 7fe712ffd640 1 -- 172.21.15.116:0/2782886654 learned_addr learned my addr 172.21.15.116:0/2782886654 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:44.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.978+0000 7fe7137fe640 1 --2- 172.21.15.116:0/2782886654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8f78cd850 0x55d8f78cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:44.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.979+0000 7fe7127fc640 1 --2- 172.21.15.116:0/2782886654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8f78d8a30 0x55d8f78dae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:44.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.979+0000 7fe718807640 1 --2- 172.21.15.116:0/2156658835 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe6fc0a2c00 0x7fe6fc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:44.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.979+0000 7fe712ffd640 1 -- 172.21.15.116:0/2782886654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8f78cd850 msgr2=0x55d8f78cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:44.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.979+0000 7fe712ffd640 1 --2- 172.21.15.116:0/2782886654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8f78cd850 0x55d8f78cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:44.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.979+0000 7fe712ffd640 1 -- 172.21.15.116:0/2782886654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8f78d8a30 msgr2=0x55d8f78dae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:44.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.979+0000 7fe712ffd640 1 --2- 172.21.15.116:0/2782886654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8f78d8a30 0x55d8f78dae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:44.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.979+0000 7fe712ffd640 1 -- 172.21.15.116:0/2782886654 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8f75a3ba0 con 0x55d8f78ce170 2024-09-12T03:12:44.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.979+0000 7fe7127fc640 1 --2- 172.21.15.116:0/2782886654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8f78d8a30 0x55d8f78dae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:12:44.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.979+0000 7fe7137fe640 1 --2- 172.21.15.116:0/2156658835 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe6fc0a2c00 0x7fe6fc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:44.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.979+0000 7fe718807640 1 -- 172.21.15.116:0/2156658835 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe7040cf5e0 con 0x7fe714101440 2024-09-12T03:12:44.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.979+0000 7fe712ffd640 1 --2- 172.21.15.116:0/2782886654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8f78ce170 0x55d8f78d84f0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fe7040c34b0 tx=0x7fe704098240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc36bf0f8b9b8af1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:44.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.979+0000 7fe6f9ffb640 1 -- 172.21.15.116:0/2782886654 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70409bc40 con 0x55d8f78ce170 2024-09-12T03:12:44.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.979+0000 7fe7137fe640 1 --2- 172.21.15.116:0/2156658835 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe6fc0a2c00 0x7fe6fc0a50c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe70c031550 tx=0x7fe70c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:44.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.979+0000 7fe6f9ffb640 1 -- 172.21.15.116:0/2782886654 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe70409c570 con 0x55d8f78ce170 2024-09-12T03:12:44.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.979+0000 7fe6f9ffb640 1 -- 172.21.15.116:0/2782886654 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70409a430 con 0x55d8f78ce170 2024-09-12T03:12:44.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.980+0000 7fe71bbe2740 1 -- 172.21.15.116:0/2782886654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8f78ce170 msgr2=0x55d8f78d84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:44.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.980+0000 7fe71bbe2740 1 --2- 172.21.15.116:0/2782886654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8f78ce170 0x55d8f78d84f0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fe7040c34b0 tx=0x7fe704098240 comp rx=0 tx=0).stop 2024-09-12T03:12:44.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.980+0000 7fe71bbe2740 1 -- 172.21.15.116:0/2782886654 shutdown_connections 2024-09-12T03:12:44.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.980+0000 7fe71bbe2740 1 --2- 172.21.15.116:0/2782886654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8f78d8a30 0x55d8f78dae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:44.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.980+0000 7fe71bbe2740 1 --2- 172.21.15.116:0/2782886654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8f78ce170 0x55d8f78d84f0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:44.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.980+0000 7fe71bbe2740 1 --2- 172.21.15.116:0/2782886654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8f78cd850 0x55d8f78cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:44.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.980+0000 7fe71bbe2740 1 -- 172.21.15.116:0/2782886654 >> 172.21.15.116:0/2782886654 conn(0x55d8f78c9de0 msgr2=0x55d8f78ca1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:44.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.980+0000 7fe71bbe2740 1 -- 172.21.15.116:0/2782886654 shutdown_connections 2024-09-12T03:12:44.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.980+0000 7fe71bbe2740 1 -- 172.21.15.116:0/2782886654 wait complete. 2024-09-12T03:12:44.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.980+0000 7fe71bbe2740 1 -- start start 2024-09-12T03:12:44.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.980+0000 7fe71bbe2740 1 Processor -- start 2024-09-12T03:12:44.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.981+0000 7fe71bbe2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8f78cd850 0x55d8f78d5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:44.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.981+0000 7fe71bbe2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8f78ce170 0x55d8f78d5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:44.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.981+0000 7fe71bbe2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8f78d8a30 0x55d8f78d63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:44.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.981+0000 7fe71bbe2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d8f78db600 con 0x55d8f78ce170 2024-09-12T03:12:44.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.981+0000 7fe71bbe2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d8f77ba900 con 0x55d8f78d8a30 2024-09-12T03:12:44.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.981+0000 7fe71bbe2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d8f78db780 con 0x55d8f78cd850 2024-09-12T03:12:45.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.981+0000 7fe71bbe2740 10 client.0 _ll_register_callbacks cb 0x55d8f77b8780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:45.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.981+0000 7fe7127fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8f78ce170 0x55d8f78d5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:45.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.981+0000 7fe7127fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8f78ce170 0x55d8f78d5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53772/0 (socket says 172.21.15.116:53772) 2024-09-12T03:12:45.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.981+0000 7fe7127fc640 1 -- 172.21.15.116:0/3256163771 learned_addr learned my addr 172.21.15.116:0/3256163771 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:45.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.981+0000 7fe7137fe640 1 --2- 172.21.15.116:0/3256163771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8f78d8a30 0x55d8f78d63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:45.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.981+0000 7fe712ffd640 1 --2- 172.21.15.116:0/3256163771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8f78cd850 0x55d8f78d5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:45.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.981+0000 7fe7127fc640 1 -- 172.21.15.116:0/3256163771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8f78cd850 msgr2=0x55d8f78d5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:45.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.981+0000 7fe7127fc640 1 --2- 172.21.15.116:0/3256163771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8f78cd850 0x55d8f78d5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:45.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.981+0000 7fe7127fc640 1 -- 172.21.15.116:0/3256163771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8f78d8a30 msgr2=0x55d8f78d63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:45.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.981+0000 7fe7127fc640 1 --2- 172.21.15.116:0/3256163771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8f78d8a30 0x55d8f78d63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:45.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.981+0000 7fe7127fc640 1 -- 172.21.15.116:0/3256163771 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8f77fac50 con 0x55d8f78ce170 2024-09-12T03:12:45.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.981+0000 7fe712ffd640 1 --2- 172.21.15.116:0/3256163771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8f78cd850 0x55d8f78d5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:12:45.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.981+0000 7fe7127fc640 1 --2- 172.21.15.116:0/3256163771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8f78ce170 0x55d8f78d5e60 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fe708011660 tx=0x7fe708011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f75380c53926419 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:45.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.981+0000 7fe6f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:12:45.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.982+0000 7fe6f2ffd640 1 -- 172.21.15.116:0/3256163771 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70801f060 con 0x55d8f78ce170 2024-09-12T03:12:45.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.982+0000 7fe6f2ffd640 1 -- 172.21.15.116:0/3256163771 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe70800fb40 con 0x55d8f78ce170 2024-09-12T03:12:45.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.982+0000 7fe71bbe2740 1 -- 172.21.15.116:0/3256163771 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe70409c570 con 0x55d8f78ce170 2024-09-12T03:12:45.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.982+0000 7fe6f2ffd640 1 -- 172.21.15.116:0/3256163771 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe70800fd40 con 0x55d8f78ce170 2024-09-12T03:12:45.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.982+0000 7fe71bbe2740 10 client.18012 fetch_fsmap learned FSMap version 25 2024-09-12T03:12:45.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.982+0000 7fe71bbe2740 1 -- 172.21.15.116:0/3256163771 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d8f78e7280 con 0x55d8f78ce170 2024-09-12T03:12:45.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.982+0000 7fe6f2ffd640 1 -- 172.21.15.116:0/3256163771 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe70409c570 con 0x55d8f78ce170 2024-09-12T03:12:45.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.982+0000 7fe6f2ffd640 1 -- 172.21.15.116:0/3256163771 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe7080254b0 con 0x55d8f78ce170 2024-09-12T03:12:45.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.982+0000 7fe71bbe2740 10 client.18012 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:12:45.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.982+0000 7fe71bbe2740 10 client.18012 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:12:45.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.982+0000 7fe71bbe2740 20 client.18012 populate_metadata read hostname 'smithi116' 2024-09-12T03:12:45.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.982+0000 7fe71bbe2740 1 --2- 172.21.15.116:0/3256163771 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d8f78e8740 0x55d8f78eab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:45.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.982+0000 7fe71bbe2740 4 client.18012 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:12:45.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.982+0000 7fe71bbe2740 1 -- 172.21.15.116:0/3256163771 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8f783f890 con 0x55d8f78e8740 2024-09-12T03:12:45.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.983+0000 7fe712ffd640 1 --2- 172.21.15.116:0/3256163771 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d8f78e8740 0x55d8f78eab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:45.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.983+0000 7fe712ffd640 1 --2- 172.21.15.116:0/3256163771 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d8f78e8740 0x55d8f78eab70 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bf57b9c9338ea4 server_cookie=194bf3fa2dce1183 in_seq=0 out_seq=0 2024-09-12T03:12:45.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.983+0000 7fe6f2ffd640 10 client.18012 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:45.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.984+0000 7fe6f2ffd640 1 -- 172.21.15.116:0/3256163771 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d8f783f890 con 0x55d8f78e8740 2024-09-12T03:12:45.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.984+0000 7fe6f2ffd640 10 client.18012 handle_command_reply: tid=0 2024-09-12T03:12:45.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.984+0000 7fe71bbe2740 1 client.18012 shutdown 2024-09-12T03:12:45.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.984+0000 7fe71bbe2740 1 -- 172.21.15.116:0/3256163771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8f78ce170 msgr2=0x55d8f78d5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:45.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.984+0000 7fe71bbe2740 1 --2- 172.21.15.116:0/3256163771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8f78ce170 0x55d8f78d5e60 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fe708011660 tx=0x7fe708011b20 comp rx=0 tx=0).stop 2024-09-12T03:12:45.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.984+0000 7fe71bbe2740 1 -- 172.21.15.116:0/3256163771 shutdown_connections 2024-09-12T03:12:45.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.984+0000 7fe71bbe2740 1 --2- 172.21.15.116:0/3256163771 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d8f78e8740 0x55d8f78eab70 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:45.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.984+0000 7fe71bbe2740 1 --2- 172.21.15.116:0/3256163771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8f78d8a30 0x55d8f78d63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:45.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.984+0000 7fe71bbe2740 1 --2- 172.21.15.116:0/3256163771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8f78ce170 0x55d8f78d5e60 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:45.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.984+0000 7fe71bbe2740 1 --2- 172.21.15.116:0/3256163771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8f78cd850 0x55d8f78d5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:45.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.984+0000 7fe6f2ffd640 0 client.18012 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:45.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.984+0000 7fe71bbe2740 1 -- 172.21.15.116:0/3256163771 >> 172.21.15.116:0/3256163771 conn(0x55d8f78c9de0 msgr2=0x55d8f78cfb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:45.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.984+0000 7fe71bbe2740 1 -- 172.21.15.116:0/3256163771 shutdown_connections 2024-09-12T03:12:45.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.984+0000 7fe71bbe2740 1 -- 172.21.15.116:0/3256163771 wait complete. 2024-09-12T03:12:45.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:44.984+0000 7fe71bbe2740 20 client.18012 trim_cache size 0 max 16384 2024-09-12T03:12:45.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.001+0000 7fe71bbe2740 1 Processor -- start 2024-09-12T03:12:45.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.001+0000 7fe71bbe2740 1 -- start start 2024-09-12T03:12:45.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.001+0000 7fe71bbe2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8f78cd850 0x55d8f78cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:45.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.001+0000 7fe71bbe2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8f78ce170 0x55d8f78f0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:45.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.001+0000 7fe71bbe2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8f78d8a30 0x55d8f78f09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:45.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.001+0000 7fe71bbe2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141b2000 con 0x55d8f78cd850 2024-09-12T03:12:45.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.001+0000 7fe71bbe2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141af8d0 con 0x55d8f78d8a30 2024-09-12T03:12:45.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.001+0000 7fe71bbe2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141ad720 con 0x55d8f78ce170 2024-09-12T03:12:45.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.001+0000 7fe712ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8f78cd850 0x55d8f78cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:45.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.001+0000 7fe712ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8f78cd850 0x55d8f78cf9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53780/0 (socket says 172.21.15.116:53780) 2024-09-12T03:12:45.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.001+0000 7fe712ffd640 1 -- 172.21.15.116:0/4283351232 learned_addr learned my addr 172.21.15.116:0/4283351232 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:45.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.001+0000 7fe7127fc640 1 --2- 172.21.15.116:0/4283351232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8f78ce170 0x55d8f78f0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:45.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.001+0000 7fe712ffd640 1 -- 172.21.15.116:0/4283351232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8f78ce170 msgr2=0x55d8f78f0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:45.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.001+0000 7fe712ffd640 1 --2- 172.21.15.116:0/4283351232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8f78ce170 0x55d8f78f0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:45.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.001+0000 7fe712ffd640 1 -- 172.21.15.116:0/4283351232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8f78d8a30 msgr2=0x55d8f78f09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:45.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.001+0000 7fe712ffd640 1 --2- 172.21.15.116:0/4283351232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8f78d8a30 0x55d8f78f09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:45.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.001+0000 7fe712ffd640 1 -- 172.21.15.116:0/4283351232 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8f78c9c10 con 0x55d8f78cd850 2024-09-12T03:12:45.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.001+0000 7fe712ffd640 1 --2- 172.21.15.116:0/4283351232 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8f78cd850 0x55d8f78cf9b0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fe70409be00 tx=0x7fe7040d2f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d970991b7909556 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:45.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.002+0000 7fe6f97fa640 1 -- 172.21.15.116:0/4283351232 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70409a620 con 0x55d8f78cd850 2024-09-12T03:12:45.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.002+0000 7fe6f97fa640 1 -- 172.21.15.116:0/4283351232 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe70409c570 con 0x55d8f78cd850 2024-09-12T03:12:45.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.002+0000 7fe6f97fa640 1 -- 172.21.15.116:0/4283351232 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70409b7d0 con 0x55d8f78cd850 2024-09-12T03:12:45.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.002+0000 7fe71bbe2740 1 -- 172.21.15.116:0/4283351232 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8f78cd850 msgr2=0x55d8f78cf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:45.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.002+0000 7fe71bbe2740 1 --2- 172.21.15.116:0/4283351232 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8f78cd850 0x55d8f78cf9b0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fe70409be00 tx=0x7fe7040d2f90 comp rx=0 tx=0).stop 2024-09-12T03:12:45.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.002+0000 7fe71bbe2740 1 -- 172.21.15.116:0/4283351232 shutdown_connections 2024-09-12T03:12:45.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.002+0000 7fe71bbe2740 1 --2- 172.21.15.116:0/4283351232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8f78d8a30 0x55d8f78f09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:45.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.002+0000 7fe71bbe2740 1 --2- 172.21.15.116:0/4283351232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8f78ce170 0x55d8f78f0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:45.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.002+0000 7fe71bbe2740 1 --2- 172.21.15.116:0/4283351232 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8f78cd850 0x55d8f78cf9b0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:45.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.002+0000 7fe71bbe2740 1 -- 172.21.15.116:0/4283351232 >> 172.21.15.116:0/4283351232 conn(0x55d8f78c9de0 msgr2=0x55d8f78cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:45.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.002+0000 7fe71bbe2740 1 -- 172.21.15.116:0/4283351232 shutdown_connections 2024-09-12T03:12:45.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.002+0000 7fe71bbe2740 1 -- 172.21.15.116:0/4283351232 wait complete. 2024-09-12T03:12:45.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.003+0000 7fe71bbe2740 1 -- start start 2024-09-12T03:12:45.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.003+0000 7fe71bbe2740 1 Processor -- start 2024-09-12T03:12:45.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.003+0000 7fe71bbe2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8f78cd850 0x55d8f78e12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:45.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.003+0000 7fe71bbe2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8f78ce170 0x55d8f78dd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:45.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.003+0000 7fe71bbe2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8f78d8a30 0x55d8f78ddd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:45.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.003+0000 7fe71bbe2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141add90 con 0x55d8f78ce170 2024-09-12T03:12:45.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.003+0000 7fe71bbe2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141af200 con 0x55d8f78d8a30 2024-09-12T03:12:45.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.003+0000 7fe71bbe2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe7141ae460 con 0x55d8f78cd850 2024-09-12T03:12:45.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.003+0000 7fe71bbe2740 10 client.0 _ll_register_callbacks cb 0x7fe708005bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:45.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.003+0000 7fe7127fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8f78ce170 0x55d8f78dd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:45.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.003+0000 7fe7127fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8f78ce170 0x55d8f78dd5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:53796/0 (socket says 172.21.15.116:53796) 2024-09-12T03:12:45.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.003+0000 7fe7127fc640 1 -- 172.21.15.116:0/3093697722 learned_addr learned my addr 172.21.15.116:0/3093697722 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:45.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.004+0000 7fe7137fe640 1 --2- 172.21.15.116:0/3093697722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8f78d8a30 0x55d8f78ddd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:45.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.004+0000 7fe712ffd640 1 --2- 172.21.15.116:0/3093697722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8f78cd850 0x55d8f78e12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:45.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.004+0000 7fe7127fc640 1 -- 172.21.15.116:0/3093697722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8f78cd850 msgr2=0x55d8f78e12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:45.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.004+0000 7fe7127fc640 1 --2- 172.21.15.116:0/3093697722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8f78cd850 0x55d8f78e12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:45.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.004+0000 7fe7127fc640 1 -- 172.21.15.116:0/3093697722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8f78d8a30 msgr2=0x55d8f78ddd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:45.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.004+0000 7fe7127fc640 1 --2- 172.21.15.116:0/3093697722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8f78d8a30 0x55d8f78ddd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:45.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.004+0000 7fe7127fc640 1 -- 172.21.15.116:0/3093697722 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8f7953460 con 0x55d8f78ce170 2024-09-12T03:12:45.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.004+0000 7fe712ffd640 1 --2- 172.21.15.116:0/3093697722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8f78cd850 0x55d8f78e12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:12:45.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.004+0000 7fe7137fe640 1 --2- 172.21.15.116:0/3093697722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8f78d8a30 0x55d8f78ddd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:12:45.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.004+0000 7fe7127fc640 1 --2- 172.21.15.116:0/3093697722 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8f78ce170 0x55d8f78dd5e0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fe708000c00 tx=0x7fe708025660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b37f93582b4c789 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:45.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.004+0000 7fe6f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:12:45.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.004+0000 7fe6f2ffd640 1 -- 172.21.15.116:0/3093697722 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708033060 con 0x55d8f78ce170 2024-09-12T03:12:45.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.004+0000 7fe6f2ffd640 1 -- 172.21.15.116:0/3093697722 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe708025d10 con 0x55d8f78ce170 2024-09-12T03:12:45.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.004+0000 7fe71bbe2740 1 -- 172.21.15.116:0/3093697722 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe70409c570 con 0x55d8f78ce170 2024-09-12T03:12:45.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.004+0000 7fe6f2ffd640 1 -- 172.21.15.116:0/3093697722 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe70801f020 con 0x55d8f78ce170 2024-09-12T03:12:45.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.004+0000 7fe71bbe2740 10 client.18018 fetch_fsmap learned FSMap version 25 2024-09-12T03:12:45.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.004+0000 7fe71bbe2740 1 -- 172.21.15.116:0/3093697722 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d8f7954630 con 0x55d8f78ce170 2024-09-12T03:12:45.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.004+0000 7fe6f2ffd640 1 -- 172.21.15.116:0/3093697722 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe70409c570 con 0x55d8f78ce170 2024-09-12T03:12:45.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.004+0000 7fe6f2ffd640 1 -- 172.21.15.116:0/3093697722 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe70802d070 con 0x55d8f78ce170 2024-09-12T03:12:45.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.005+0000 7fe71bbe2740 10 client.18018 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:12:45.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.005+0000 7fe71bbe2740 10 client.18018 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:12:45.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.005+0000 7fe71bbe2740 20 client.18018 populate_metadata read hostname 'smithi116' 2024-09-12T03:12:45.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.005+0000 7fe71bbe2740 1 --2- 172.21.15.116:0/3093697722 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d8f7954e00 0x55d8f7957220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:45.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.005+0000 7fe71bbe2740 4 client.18018 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}] 2024-09-12T03:12:45.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.005+0000 7fe71bbe2740 1 -- 172.21.15.116:0/3093697722 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}) -- 0x55d8f7839be0 con 0x55d8f7954e00 2024-09-12T03:12:45.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.005+0000 7fe712ffd640 1 --2- 172.21.15.116:0/3093697722 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d8f7954e00 0x55d8f7957220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:45.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.005+0000 7fe712ffd640 1 --2- 172.21.15.116:0/3093697722 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d8f7954e00 0x55d8f7957220 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3f6af99212dc932 server_cookie=d4d8b7786c8b1d26 in_seq=0 out_seq=0 2024-09-12T03:12:45.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.005+0000 7fe6f2ffd640 10 client.18018 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:45.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.015+0000 7fe6f2ffd640 1 -- 172.21.15.116:0/3093697722 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d8f7839be0 con 0x55d8f7954e00 2024-09-12T03:12:45.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.015+0000 7fe6f2ffd640 10 client.18018 handle_command_reply: tid=0 2024-09-12T03:12:45.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.015+0000 7fe71bbe2740 1 client.18018 shutdown 2024-09-12T03:12:45.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.015+0000 7fe71bbe2740 1 -- 172.21.15.116:0/3093697722 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8f78ce170 msgr2=0x55d8f78dd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:45.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.015+0000 7fe71bbe2740 1 --2- 172.21.15.116:0/3093697722 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8f78ce170 0x55d8f78dd5e0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fe708000c00 tx=0x7fe708025660 comp rx=0 tx=0).stop 2024-09-12T03:12:45.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.015+0000 7fe71bbe2740 1 -- 172.21.15.116:0/3093697722 shutdown_connections 2024-09-12T03:12:45.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.015+0000 7fe71bbe2740 1 --2- 172.21.15.116:0/3093697722 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d8f7954e00 0x55d8f7957220 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:45.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.015+0000 7fe71bbe2740 1 --2- 172.21.15.116:0/3093697722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8f78d8a30 0x55d8f78ddd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:45.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.015+0000 7fe71bbe2740 1 --2- 172.21.15.116:0/3093697722 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8f78ce170 0x55d8f78dd5e0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:45.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.015+0000 7fe71bbe2740 1 --2- 172.21.15.116:0/3093697722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8f78cd850 0x55d8f78e12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:45.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.015+0000 7fe71bbe2740 1 -- 172.21.15.116:0/3093697722 >> 172.21.15.116:0/3093697722 conn(0x55d8f78c9de0 msgr2=0x55d8f78d6090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:45.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.015+0000 7fe6f2ffd640 0 client.18018 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:45.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.015+0000 7fe71bbe2740 1 -- 172.21.15.116:0/3093697722 shutdown_connections 2024-09-12T03:12:45.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.015+0000 7fe71bbe2740 1 -- 172.21.15.116:0/3093697722 wait complete. 2024-09-12T03:12:45.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.015+0000 7fe71bbe2740 20 client.18018 trim_cache size 0 max 16384 2024-09-12T03:12:45.018 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:12:45.018 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 25, 2024-09-12T03:12:45.018 INFO:teuthology.orchestra.run.smithi116.stdout: "leader": 24494, 2024-09-12T03:12:45.018 INFO:teuthology.orchestra.run.smithi116.stdout: "set_version": 38, 2024-09-12T03:12:45.018 INFO:teuthology.orchestra.run.smithi116.stdout: "sets": { 2024-09-12T03:12:45.018 INFO:teuthology.orchestra.run.smithi116.stdout: "6c5db9e6": { 2024-09-12T03:12:45.019 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 38, 2024-09-12T03:12:45.019 INFO:teuthology.orchestra.run.smithi116.stdout: "age_ref": 0.0, 2024-09-12T03:12:45.019 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:12:45.019 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "QUIESCED", 2024-09-12T03:12:45.019 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:12:45.019 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:12:45.019 INFO:teuthology.orchestra.run.smithi116.stdout: "timeout": 90.0, 2024-09-12T03:12:45.019 INFO:teuthology.orchestra.run.smithi116.stdout: "expiration": 142.6, 2024-09-12T03:12:45.019 INFO:teuthology.orchestra.run.smithi116.stdout: "members": { 2024-09-12T03:12:45.019 INFO:teuthology.orchestra.run.smithi116.stdout: "file:/": { 2024-09-12T03:12:45.019 INFO:teuthology.orchestra.run.smithi116.stdout: "excluded": false, 2024-09-12T03:12:45.019 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:12:45.019 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "QUIESCED", 2024-09-12T03:12:45.020 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:12:45.020 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:12:45.020 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:12:45.020 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:12:45.020 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:12:45.020 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:12:45.020 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:12:45.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.016+0000 7fe719809640 1 -- 172.21.15.116:0/2156658835 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe6fc0a2c00 msgr2=0x7fe6fc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:45.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.016+0000 7fe719809640 1 --2- 172.21.15.116:0/2156658835 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe6fc0a2c00 0x7fe6fc0a50c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe70c031550 tx=0x7fe70c009260 comp rx=0 tx=0).stop 2024-09-12T03:12:45.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.016+0000 7fe719809640 1 -- 172.21.15.116:0/2156658835 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe714101440 msgr2=0x7fe7140fc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:45.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.016+0000 7fe719809640 1 --2- 172.21.15.116:0/2156658835 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe714101440 0x7fe7140fc690 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fe70400eab0 tx=0x7fe70400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:12:45.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.016+0000 7fe719809640 1 -- 172.21.15.116:0/2156658835 shutdown_connections 2024-09-12T03:12:45.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.016+0000 7fe719809640 1 --2- 172.21.15.116:0/2156658835 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe6fc0a2c00 0x7fe6fc0a50c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:45.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.016+0000 7fe719809640 1 --2- 172.21.15.116:0/2156658835 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe714106050 0x7fe7140fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:45.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.016+0000 7fe719809640 1 --2- 172.21.15.116:0/2156658835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe714101df0 0x7fe7140fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:45.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.016+0000 7fe719809640 1 --2- 172.21.15.116:0/2156658835 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe714101440 0x7fe7140fc690 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:45.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.016+0000 7fe719809640 1 -- 172.21.15.116:0/2156658835 >> 172.21.15.116:0/2156658835 conn(0x7fe714077210 msgr2=0x7fe714100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:45.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.016+0000 7fe719809640 1 -- 172.21.15.116:0/2156658835 shutdown_connections 2024-09-12T03:12:45.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:45.016+0000 7fe719809640 1 -- 172.21.15.116:0/2156658835 wait complete. 2024-09-12T03:12:45.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:45 smithi183 ceph-mon[35535]: pgmap v631: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:12:45.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:45 smithi177 ceph-mon[35305]: pgmap v631: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:12:45.781 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:45 smithi116 ceph-mon[31410]: pgmap v631: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:12:45.813 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6c5db9e6', quiesce took 3.4 seconds. Will release after: 19.150000000000002 2024-09-12T03:12:47.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:47 smithi183 ceph-mon[35535]: pgmap v632: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:12:47.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:47 smithi177 ceph-mon[35305]: pgmap v632: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:12:47.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:47 smithi116 ceph-mon[31410]: pgmap v632: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:12:49.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:49 smithi183 ceph-mon[35535]: pgmap v633: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T03:12:49.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:49 smithi177 ceph-mon[35305]: pgmap v633: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T03:12:49.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:49 smithi116 ceph-mon[31410]: pgmap v633: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-12T03:12:51.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:51 smithi183 ceph-mon[35535]: pgmap v634: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-09-12T03:12:51.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:51 smithi177 ceph-mon[35305]: pgmap v634: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-09-12T03:12:51.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:51 smithi116 ceph-mon[31410]: pgmap v634: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-09-12T03:12:53.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:53 smithi183 ceph-mon[35535]: pgmap v635: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-09-12T03:12:53.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:53 smithi177 ceph-mon[35305]: pgmap v635: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-09-12T03:12:53.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:53 smithi116 ceph-mon[31410]: pgmap v635: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 8 op/s 2024-09-12T03:12:55.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:55 smithi183 ceph-mon[35535]: pgmap v636: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-09-12T03:12:55.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:55 smithi177 ceph-mon[35305]: pgmap v636: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-09-12T03:12:55.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:55 smithi116 ceph-mon[31410]: pgmap v636: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-09-12T03:12:56.451 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:12:56.708 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:12:57.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.207+0000 7fce5e9db640 1 -- 172.21.15.116:0/95000578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce58106f10 msgr2=0x7fce5810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:57.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.207+0000 7fce5e9db640 1 --2- 172.21.15.116:0/95000578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce58106f10 0x7fce5810ea90 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fce50009ca0 tx=0x7fce5002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.207+0000 7fce5e9db640 1 -- 172.21.15.116:0/95000578 shutdown_connections 2024-09-12T03:12:57.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.207+0000 7fce5e9db640 1 --2- 172.21.15.116:0/95000578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce58106f10 0x7fce5810ea90 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.207+0000 7fce5e9db640 1 --2- 172.21.15.116:0/95000578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce58102790 0x7fce581067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.207+0000 7fce5e9db640 1 --2- 172.21.15.116:0/95000578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce58101de0 0x7fce581021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.207+0000 7fce5e9db640 1 -- 172.21.15.116:0/95000578 >> 172.21.15.116:0/95000578 conn(0x7fce580fd8c0 msgr2=0x7fce580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:57.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.207+0000 7fce5e9db640 1 -- 172.21.15.116:0/95000578 shutdown_connections 2024-09-12T03:12:57.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.208+0000 7fce5e9db640 1 -- 172.21.15.116:0/95000578 wait complete. 2024-09-12T03:12:57.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.208+0000 7fce5e9db640 1 Processor -- start 2024-09-12T03:12:57.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.208+0000 7fce5e9db640 1 -- start start 2024-09-12T03:12:57.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.208+0000 7fce5e9db640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce58101de0 0x7fce580744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:57.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.208+0000 7fce5e9db640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce58102790 0x7fce58074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:57.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.208+0000 7fce5e9db640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce58106f10 0x7fce58074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:57.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.208+0000 7fce5e9db640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fce58111f70 con 0x7fce58101de0 2024-09-12T03:12:57.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.208+0000 7fce5e9db640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fce58111df0 con 0x7fce58102790 2024-09-12T03:12:57.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.208+0000 7fce5e9db640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fce581120f0 con 0x7fce58106f10 2024-09-12T03:12:57.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.209+0000 7fce57fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce58101de0 0x7fce580744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:57.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.209+0000 7fce5cf78640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce58106f10 0x7fce58074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:57.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.209+0000 7fce57fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce58101de0 0x7fce580744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44096/0 (socket says 172.21.15.116:44096) 2024-09-12T03:12:57.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.209+0000 7fce57fff640 1 -- 172.21.15.116:0/1701561181 learned_addr learned my addr 172.21.15.116:0/1701561181 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:57.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.209+0000 7fce577fe640 1 --2- 172.21.15.116:0/1701561181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce58102790 0x7fce58074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:57.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.209+0000 7fce57fff640 1 -- 172.21.15.116:0/1701561181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce58106f10 msgr2=0x7fce58074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:57.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.209+0000 7fce57fff640 1 --2- 172.21.15.116:0/1701561181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce58106f10 0x7fce58074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.209+0000 7fce57fff640 1 -- 172.21.15.116:0/1701561181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce58102790 msgr2=0x7fce58074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:57.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.209+0000 7fce57fff640 1 --2- 172.21.15.116:0/1701561181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce58102790 0x7fce58074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.209+0000 7fce57fff640 1 -- 172.21.15.116:0/1701561181 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce581a68b0 con 0x7fce58101de0 2024-09-12T03:12:57.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.209+0000 7fce57fff640 1 --2- 172.21.15.116:0/1701561181 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce58101de0 0x7fce580744f0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fce4800c890 tx=0x7fce4800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:57.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.209+0000 7fce557fa640 1 -- 172.21.15.116:0/1701561181 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce48004220 con 0x7fce58101de0 2024-09-12T03:12:57.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.209+0000 7fce557fa640 1 -- 172.21.15.116:0/1701561181 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce480043c0 con 0x7fce58101de0 2024-09-12T03:12:57.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.209+0000 7fce5e9db640 1 -- 172.21.15.116:0/1701561181 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce581a6ba0 con 0x7fce58101de0 2024-09-12T03:12:57.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.209+0000 7fce557fa640 1 -- 172.21.15.116:0/1701561181 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce480049e0 con 0x7fce58101de0 2024-09-12T03:12:57.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.209+0000 7fce5e9db640 1 -- 172.21.15.116:0/1701561181 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce581a70e0 con 0x7fce58101de0 2024-09-12T03:12:57.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.211+0000 7fce557fa640 1 -- 172.21.15.116:0/1701561181 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fce480079b0 con 0x7fce58101de0 2024-09-12T03:12:57.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.212+0000 7fce60db4740 1 Processor -- start 2024-09-12T03:12:57.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.212+0000 7fce60db4740 1 -- start start 2024-09-12T03:12:57.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.212+0000 7fce60db4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0093b04a0 0x55a0093b0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:57.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.212+0000 7fce60db4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0093b0dc0 0x55a0093bb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:57.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.212+0000 7fce60db4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0093bb680 0x55a0093bda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:57.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.212+0000 7fce60db4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a0093162e0 con 0x55a0093b0dc0 2024-09-12T03:12:57.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.212+0000 7fce60db4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a0092bdce0 con 0x55a0093bb680 2024-09-12T03:12:57.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.212+0000 7fce60db4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a0093122a0 con 0x55a0093b04a0 2024-09-12T03:12:57.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.212+0000 7fce57fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0093b0dc0 0x55a0093bb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:57.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.212+0000 7fce57fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0093b0dc0 0x55a0093bb140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44110/0 (socket says 172.21.15.116:44110) 2024-09-12T03:12:57.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.212+0000 7fce57fff640 1 -- 172.21.15.116:0/3968845644 learned_addr learned my addr 172.21.15.116:0/3968845644 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:57.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.212+0000 7fce5cf78640 1 --2- 172.21.15.116:0/3968845644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0093b04a0 0x55a0093b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:57.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.212+0000 7fce557fa640 1 --2- 172.21.15.116:0/1701561181 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fce400a2b80 0x7fce400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:57.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.212+0000 7fce577fe640 1 --2- 172.21.15.116:0/3968845644 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0093bb680 0x55a0093bda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:57.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.212+0000 7fce5cf78640 1 --2- 172.21.15.116:0/1701561181 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fce400a2b80 0x7fce400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:57.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.213+0000 7fce557fa640 1 -- 172.21.15.116:0/1701561181 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fce480cf070 con 0x7fce58101de0 2024-09-12T03:12:57.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.213+0000 7fce5cf78640 1 -- 172.21.15.116:0/3968845644 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0093bb680 msgr2=0x55a0093bda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:57.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.213+0000 7fce5cf78640 1 --2- 172.21.15.116:0/3968845644 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0093bb680 0x55a0093bda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.231 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.213+0000 7fce5cf78640 1 -- 172.21.15.116:0/3968845644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0093b0dc0 msgr2=0x55a0093bb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.213+0000 7fce5cf78640 1 --2- 172.21.15.116:0/3968845644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0093b0dc0 0x55a0093bb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.213+0000 7fce5cf78640 1 -- 172.21.15.116:0/3968845644 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0092a2440 con 0x55a0093b04a0 2024-09-12T03:12:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.213+0000 7fce577fe640 1 --2- 172.21.15.116:0/3968845644 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0093bb680 0x55a0093bda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:12:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.213+0000 7fce57fff640 1 --2- 172.21.15.116:0/3968845644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0093b0dc0 0x55a0093bb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:12:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.213+0000 7fce5cf78640 1 --2- 172.21.15.116:0/3968845644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0093b04a0 0x55a0093b0880 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fce50006010 tx=0x7fce50003a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4d7b0cde5d5f05c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.213+0000 7fce5cf78640 1 --2- 172.21.15.116:0/1701561181 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fce400a2b80 0x7fce400a5040 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fce50048000 tx=0x7fce5004a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.213+0000 7fce3e7fc640 1 -- 172.21.15.116:0/3968845644 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5004f070 con 0x55a0093b04a0 2024-09-12T03:12:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.213+0000 7fce3e7fc640 1 -- 172.21.15.116:0/3968845644 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce50036040 con 0x55a0093b04a0 2024-09-12T03:12:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.213+0000 7fce3e7fc640 1 -- 172.21.15.116:0/3968845644 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5002fc70 con 0x55a0093b04a0 2024-09-12T03:12:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.214+0000 7fce60db4740 1 -- 172.21.15.116:0/3968845644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0093b04a0 msgr2=0x55a0093b0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.214+0000 7fce60db4740 1 --2- 172.21.15.116:0/3968845644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0093b04a0 0x55a0093b0880 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fce50006010 tx=0x7fce50003a70 comp rx=0 tx=0).stop 2024-09-12T03:12:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.214+0000 7fce60db4740 1 -- 172.21.15.116:0/3968845644 shutdown_connections 2024-09-12T03:12:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.214+0000 7fce60db4740 1 --2- 172.21.15.116:0/3968845644 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0093bb680 0x55a0093bda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.214+0000 7fce60db4740 1 --2- 172.21.15.116:0/3968845644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0093b0dc0 0x55a0093bb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.214+0000 7fce60db4740 1 --2- 172.21.15.116:0/3968845644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0093b04a0 0x55a0093b0880 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.214+0000 7fce60db4740 1 -- 172.21.15.116:0/3968845644 >> 172.21.15.116:0/3968845644 conn(0x55a0093aca30 msgr2=0x55a0093ace10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:57.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.214+0000 7fce60db4740 1 -- 172.21.15.116:0/3968845644 shutdown_connections 2024-09-12T03:12:57.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.214+0000 7fce60db4740 1 -- 172.21.15.116:0/3968845644 wait complete. 2024-09-12T03:12:57.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.214+0000 7fce60db4740 1 -- start start 2024-09-12T03:12:57.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.214+0000 7fce60db4740 1 Processor -- start 2024-09-12T03:12:57.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.215+0000 7fce60db4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0093b04a0 0x55a0093bb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:57.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.215+0000 7fce60db4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0093b0dc0 0x55a0093b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:57.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.215+0000 7fce60db4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0093bb680 0x55a0093b8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:57.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.215+0000 7fce60db4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a0093be0d0 con 0x55a0093b0dc0 2024-09-12T03:12:57.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.215+0000 7fce60db4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a0092d1050 con 0x55a0093b04a0 2024-09-12T03:12:57.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.215+0000 7fce60db4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a0093be250 con 0x55a0093bb680 2024-09-12T03:12:57.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.215+0000 7fce60db4740 10 client.0 _ll_register_callbacks cb 0x55a009311160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:57.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.215+0000 7fce577fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0093b0dc0 0x55a0093b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:57.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.215+0000 7fce577fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0093b0dc0 0x55a0093b82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44118/0 (socket says 172.21.15.116:44118) 2024-09-12T03:12:57.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.215+0000 7fce577fe640 1 -- 172.21.15.116:0/3983846657 learned_addr learned my addr 172.21.15.116:0/3983846657 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:57.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.215+0000 7fce5cf78640 1 --2- 172.21.15.116:0/3983846657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0093bb680 0x55a0093b8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:57.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.215+0000 7fce57fff640 1 --2- 172.21.15.116:0/3983846657 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0093b04a0 0x55a0093bb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:57.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.215+0000 7fce577fe640 1 -- 172.21.15.116:0/3983846657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0093bb680 msgr2=0x55a0093b8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:57.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.215+0000 7fce577fe640 1 --2- 172.21.15.116:0/3983846657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0093bb680 0x55a0093b8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.215+0000 7fce577fe640 1 -- 172.21.15.116:0/3983846657 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0093b04a0 msgr2=0x55a0093bb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:57.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.215+0000 7fce577fe640 1 --2- 172.21.15.116:0/3983846657 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0093b04a0 0x55a0093bb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.215+0000 7fce577fe640 1 -- 172.21.15.116:0/3983846657 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0090873c0 con 0x55a0093b0dc0 2024-09-12T03:12:57.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.215+0000 7fce57fff640 1 --2- 172.21.15.116:0/3983846657 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0093b04a0 0x55a0093bb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:12:57.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.215+0000 7fce577fe640 1 --2- 172.21.15.116:0/3983846657 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0093b0dc0 0x55a0093b82c0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00be10 tx=0x7fce4c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d4e8a0a4961755 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:57.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.215+0000 7fce377fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:12:57.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.215+0000 7fce377fe640 1 -- 172.21.15.116:0/3983846657 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4c012650 con 0x55a0093b0dc0 2024-09-12T03:12:57.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.215+0000 7fce377fe640 1 -- 172.21.15.116:0/3983846657 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fce4c0127f0 con 0x55a0093b0dc0 2024-09-12T03:12:57.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.215+0000 7fce60db4740 1 -- 172.21.15.116:0/3983846657 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce50036040 con 0x55a0093b0dc0 2024-09-12T03:12:57.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.216+0000 7fce377fe640 1 -- 172.21.15.116:0/3983846657 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fce4c0129f0 con 0x55a0093b0dc0 2024-09-12T03:12:57.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.216+0000 7fce60db4740 10 client.18030 fetch_fsmap learned FSMap version 25 2024-09-12T03:12:57.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.216+0000 7fce60db4740 1 -- 172.21.15.116:0/3983846657 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a0093c9b20 con 0x55a0093b0dc0 2024-09-12T03:12:57.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.216+0000 7fce377fe640 1 -- 172.21.15.116:0/3983846657 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce50036040 con 0x55a0093b0dc0 2024-09-12T03:12:57.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.216+0000 7fce377fe640 1 -- 172.21.15.116:0/3983846657 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fce4c026420 con 0x55a0093b0dc0 2024-09-12T03:12:57.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.216+0000 7fce60db4740 10 client.18030 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:12:57.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.216+0000 7fce60db4740 10 client.18030 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:12:57.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.216+0000 7fce60db4740 20 client.18030 populate_metadata read hostname 'smithi116' 2024-09-12T03:12:57.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.216+0000 7fce60db4740 1 --2- 172.21.15.116:0/3983846657 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a0093cb080 0x55a0093cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:57.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.216+0000 7fce60db4740 4 client.18030 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:12:57.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.216+0000 7fce60db4740 1 -- 172.21.15.116:0/3983846657 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a009323850 con 0x55a0093cb080 2024-09-12T03:12:57.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.216+0000 7fce57fff640 1 --2- 172.21.15.116:0/3983846657 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a0093cb080 0x55a0093cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:57.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.217+0000 7fce377fe640 10 client.18030 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:57.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.217+0000 7fce57fff640 1 --2- 172.21.15.116:0/3983846657 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a0093cb080 0x55a0093cd4a0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=590d237176f0b540 server_cookie=6286524036ec33ce in_seq=0 out_seq=0 2024-09-12T03:12:57.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.217+0000 7fce377fe640 1 -- 172.21.15.116:0/3983846657 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a009323850 con 0x55a0093cb080 2024-09-12T03:12:57.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.217+0000 7fce377fe640 10 client.18030 handle_command_reply: tid=0 2024-09-12T03:12:57.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.218+0000 7fce60db4740 1 client.18030 shutdown 2024-09-12T03:12:57.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.218+0000 7fce60db4740 1 -- 172.21.15.116:0/3983846657 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0093b0dc0 msgr2=0x55a0093b82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:57.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.218+0000 7fce60db4740 1 --2- 172.21.15.116:0/3983846657 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0093b0dc0 0x55a0093b82c0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00be10 tx=0x7fce4c0117b0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.218+0000 7fce60db4740 1 -- 172.21.15.116:0/3983846657 shutdown_connections 2024-09-12T03:12:57.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.218+0000 7fce60db4740 1 --2- 172.21.15.116:0/3983846657 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a0093cb080 0x55a0093cd4a0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.218+0000 7fce60db4740 1 --2- 172.21.15.116:0/3983846657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0093bb680 0x55a0093b8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.218+0000 7fce60db4740 1 --2- 172.21.15.116:0/3983846657 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0093b0dc0 0x55a0093b82c0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.218+0000 7fce60db4740 1 --2- 172.21.15.116:0/3983846657 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0093b04a0 0x55a0093bb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.218+0000 7fce377fe640 0 client.18030 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:57.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.218+0000 7fce60db4740 1 -- 172.21.15.116:0/3983846657 >> 172.21.15.116:0/3983846657 conn(0x55a0093aca30 msgr2=0x55a0093abb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:57.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.218+0000 7fce60db4740 1 -- 172.21.15.116:0/3983846657 shutdown_connections 2024-09-12T03:12:57.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.218+0000 7fce60db4740 1 -- 172.21.15.116:0/3983846657 wait complete. 2024-09-12T03:12:57.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.218+0000 7fce60db4740 20 client.18030 trim_cache size 0 max 16384 2024-09-12T03:12:57.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.235+0000 7fce60db4740 1 Processor -- start 2024-09-12T03:12:57.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.235+0000 7fce60db4740 1 -- start start 2024-09-12T03:12:57.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.235+0000 7fce60db4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0093b04a0 0x55a0093c7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:57.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.235+0000 7fce60db4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0093b0dc0 0x55a0093d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:57.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.235+0000 7fce60db4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0093bb680 0x55a0093d32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:57.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.235+0000 7fce60db4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fce581b1f90 con 0x55a0093bb680 2024-09-12T03:12:57.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.235+0000 7fce60db4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fce581af860 con 0x55a0093b0dc0 2024-09-12T03:12:57.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.235+0000 7fce60db4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fce581ad6b0 con 0x55a0093b04a0 2024-09-12T03:12:57.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.235+0000 7fce5cf78640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0093bb680 0x55a0093d32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:57.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.235+0000 7fce5cf78640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0093bb680 0x55a0093d32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44122/0 (socket says 172.21.15.116:44122) 2024-09-12T03:12:57.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.235+0000 7fce5cf78640 1 -- 172.21.15.116:0/3326138151 learned_addr learned my addr 172.21.15.116:0/3326138151 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:57.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.235+0000 7fce577fe640 1 --2- 172.21.15.116:0/3326138151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0093b0dc0 0x55a0093d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:57.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.235+0000 7fce57fff640 1 --2- 172.21.15.116:0/3326138151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0093b04a0 0x55a0093c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:57.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.235+0000 7fce5cf78640 1 -- 172.21.15.116:0/3326138151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0093b04a0 msgr2=0x55a0093c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:57.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.235+0000 7fce5cf78640 1 --2- 172.21.15.116:0/3326138151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0093b04a0 0x55a0093c7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.235+0000 7fce5cf78640 1 -- 172.21.15.116:0/3326138151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0093b0dc0 msgr2=0x55a0093d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:57.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.235+0000 7fce5cf78640 1 --2- 172.21.15.116:0/3326138151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0093b0dc0 0x55a0093d2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.235+0000 7fce5cf78640 1 -- 172.21.15.116:0/3326138151 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0093ad6c0 con 0x55a0093bb680 2024-09-12T03:12:57.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.236+0000 7fce5cf78640 1 --2- 172.21.15.116:0/3326138151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0093bb680 0x55a0093d32b0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fce5002f820 tx=0x7fce5002f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c84f132a603e66 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:57.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.236+0000 7fce3dffb640 1 -- 172.21.15.116:0/3326138151 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50058070 con 0x55a0093bb680 2024-09-12T03:12:57.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.236+0000 7fce3dffb640 1 -- 172.21.15.116:0/3326138151 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce5002f310 con 0x55a0093bb680 2024-09-12T03:12:57.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.236+0000 7fce3dffb640 1 -- 172.21.15.116:0/3326138151 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5004f040 con 0x55a0093bb680 2024-09-12T03:12:57.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.237+0000 7fce60db4740 1 -- 172.21.15.116:0/3326138151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0093bb680 msgr2=0x55a0093d32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:57.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.237+0000 7fce60db4740 1 --2- 172.21.15.116:0/3326138151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0093bb680 0x55a0093d32b0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fce5002f820 tx=0x7fce5002f850 comp rx=0 tx=0).stop 2024-09-12T03:12:57.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.237+0000 7fce60db4740 1 -- 172.21.15.116:0/3326138151 shutdown_connections 2024-09-12T03:12:57.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.237+0000 7fce60db4740 1 --2- 172.21.15.116:0/3326138151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0093bb680 0x55a0093d32b0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.237+0000 7fce60db4740 1 --2- 172.21.15.116:0/3326138151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0093b0dc0 0x55a0093d2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.237+0000 7fce60db4740 1 --2- 172.21.15.116:0/3326138151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0093b04a0 0x55a0093c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.237+0000 7fce60db4740 1 -- 172.21.15.116:0/3326138151 >> 172.21.15.116:0/3326138151 conn(0x55a0093aca30 msgr2=0x55a0093ba340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:57.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.237+0000 7fce60db4740 1 -- 172.21.15.116:0/3326138151 shutdown_connections 2024-09-12T03:12:57.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.237+0000 7fce60db4740 1 -- 172.21.15.116:0/3326138151 wait complete. 2024-09-12T03:12:57.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.237+0000 7fce60db4740 1 -- start start 2024-09-12T03:12:57.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.237+0000 7fce60db4740 1 Processor -- start 2024-09-12T03:12:57.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.237+0000 7fce60db4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0093b04a0 0x55a0093bf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:57.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.237+0000 7fce60db4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0093b0dc0 0x55a0093bf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:57.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.237+0000 7fce60db4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0093bb680 0x55a0093bfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:57.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.237+0000 7fce60db4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fce581add20 con 0x55a0093bb680 2024-09-12T03:12:57.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.237+0000 7fce60db4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fce581af190 con 0x55a0093b04a0 2024-09-12T03:12:57.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.237+0000 7fce60db4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fce581ae3f0 con 0x55a0093b0dc0 2024-09-12T03:12:57.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.238+0000 7fce60db4740 10 client.0 _ll_register_callbacks cb 0x7fce4c005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:57.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.238+0000 7fce5cf78640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0093bb680 0x55a0093bfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:57.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.238+0000 7fce5cf78640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0093bb680 0x55a0093bfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44136/0 (socket says 172.21.15.116:44136) 2024-09-12T03:12:57.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.238+0000 7fce5cf78640 1 -- 172.21.15.116:0/2018395262 learned_addr learned my addr 172.21.15.116:0/2018395262 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:57.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.238+0000 7fce577fe640 1 --2- 172.21.15.116:0/2018395262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0093b0dc0 0x55a0093bf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:57.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.238+0000 7fce57fff640 1 --2- 172.21.15.116:0/2018395262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0093b04a0 0x55a0093bf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:57.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.238+0000 7fce5cf78640 1 -- 172.21.15.116:0/2018395262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0093b0dc0 msgr2=0x55a0093bf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:57.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.238+0000 7fce5cf78640 1 --2- 172.21.15.116:0/2018395262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0093b0dc0 0x55a0093bf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.238+0000 7fce5cf78640 1 -- 172.21.15.116:0/2018395262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0093b04a0 msgr2=0x55a0093bf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:57.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.238+0000 7fce5cf78640 1 --2- 172.21.15.116:0/2018395262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0093b04a0 0x55a0093bf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.238+0000 7fce5cf78640 1 -- 172.21.15.116:0/2018395262 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a009434f40 con 0x55a0093bb680 2024-09-12T03:12:57.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.238+0000 7fce57fff640 1 --2- 172.21.15.116:0/2018395262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0093b04a0 0x55a0093bf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:12:57.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.238+0000 7fce5cf78640 1 --2- 172.21.15.116:0/2018395262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0093bb680 0x55a0093bfe90 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fce50031f80 tx=0x7fce5002f760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd55fc983b387b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:57.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.238+0000 7fce377fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:12:57.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.238+0000 7fce377fe640 1 -- 172.21.15.116:0/2018395262 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50064060 con 0x55a0093bb680 2024-09-12T03:12:57.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.238+0000 7fce377fe640 1 -- 172.21.15.116:0/2018395262 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fce500318e0 con 0x55a0093bb680 2024-09-12T03:12:57.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.238+0000 7fce60db4740 1 -- 172.21.15.116:0/2018395262 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce5002f310 con 0x55a0093bb680 2024-09-12T03:12:57.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.238+0000 7fce377fe640 1 -- 172.21.15.116:0/2018395262 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fce50058070 con 0x55a0093bb680 2024-09-12T03:12:57.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.238+0000 7fce60db4740 10 client.18036 fetch_fsmap learned FSMap version 25 2024-09-12T03:12:57.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.238+0000 7fce60db4740 1 -- 172.21.15.116:0/2018395262 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a009436230 con 0x55a0093bb680 2024-09-12T03:12:57.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.239+0000 7fce377fe640 1 -- 172.21.15.116:0/2018395262 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce5002f310 con 0x55a0093bb680 2024-09-12T03:12:57.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.239+0000 7fce377fe640 1 -- 172.21.15.116:0/2018395262 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fce50030d50 con 0x55a0093bb680 2024-09-12T03:12:57.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.239+0000 7fce60db4740 10 client.18036 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:12:57.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.239+0000 7fce60db4740 10 client.18036 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:12:57.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.239+0000 7fce60db4740 20 client.18036 populate_metadata read hostname 'smithi116' 2024-09-12T03:12:57.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.239+0000 7fce60db4740 1 --2- 172.21.15.116:0/2018395262 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a0094365c0 0x55a0094389b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:57.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.239+0000 7fce60db4740 4 client.18036 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:12:57.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.239+0000 7fce60db4740 1 -- 172.21.15.116:0/2018395262 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a00931dd60 con 0x55a0094365c0 2024-09-12T03:12:57.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.239+0000 7fce57fff640 1 --2- 172.21.15.116:0/2018395262 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a0094365c0 0x55a0094389b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:57.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.240+0000 7fce377fe640 10 client.18036 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:57.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.240+0000 7fce57fff640 1 --2- 172.21.15.116:0/2018395262 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a0094365c0 0x55a0094389b0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8390a2ab0b0ed8ed server_cookie=cb1c071f8dc28e6f in_seq=0 out_seq=0 2024-09-12T03:12:57.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.240+0000 7fce377fe640 1 -- 172.21.15.116:0/2018395262 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a00931dd60 con 0x55a0094365c0 2024-09-12T03:12:57.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.240+0000 7fce377fe640 10 client.18036 handle_command_reply: tid=0 2024-09-12T03:12:57.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.240+0000 7fce60db4740 1 client.18036 shutdown 2024-09-12T03:12:57.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.240+0000 7fce60db4740 1 -- 172.21.15.116:0/2018395262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0093bb680 msgr2=0x55a0093bfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:57.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.240+0000 7fce60db4740 1 --2- 172.21.15.116:0/2018395262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0093bb680 0x55a0093bfe90 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fce50031f80 tx=0x7fce5002f760 comp rx=0 tx=0).stop 2024-09-12T03:12:57.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.240+0000 7fce60db4740 1 -- 172.21.15.116:0/2018395262 shutdown_connections 2024-09-12T03:12:57.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.240+0000 7fce60db4740 1 --2- 172.21.15.116:0/2018395262 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a0094365c0 0x55a0094389b0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.240+0000 7fce60db4740 1 --2- 172.21.15.116:0/2018395262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a0093bb680 0x55a0093bfe90 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.240+0000 7fce377fe640 0 client.18036 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:12:57.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.240+0000 7fce60db4740 1 --2- 172.21.15.116:0/2018395262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0093b0dc0 0x55a0093bf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.240+0000 7fce60db4740 1 --2- 172.21.15.116:0/2018395262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a0093b04a0 0x55a0093bf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.240+0000 7fce60db4740 1 -- 172.21.15.116:0/2018395262 >> 172.21.15.116:0/2018395262 conn(0x55a0093aca30 msgr2=0x55a0093b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:57.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.240+0000 7fce60db4740 1 -- 172.21.15.116:0/2018395262 shutdown_connections 2024-09-12T03:12:57.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.240+0000 7fce60db4740 1 -- 172.21.15.116:0/2018395262 wait complete. 2024-09-12T03:12:57.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.240+0000 7fce60db4740 20 client.18036 trim_cache size 0 max 16384 2024-09-12T03:12:57.249 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:12:57.249 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:12:57.250 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:12:57.250 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:12:57.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.241+0000 7fce5e9db640 1 -- 172.21.15.116:0/1701561181 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fce400a2b80 msgr2=0x7fce400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:57.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.241+0000 7fce5e9db640 1 --2- 172.21.15.116:0/1701561181 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fce400a2b80 0x7fce400a5040 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fce50048000 tx=0x7fce5004a000 comp rx=0 tx=0).stop 2024-09-12T03:12:57.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.241+0000 7fce5e9db640 1 -- 172.21.15.116:0/1701561181 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce58101de0 msgr2=0x7fce580744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:57.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.241+0000 7fce5e9db640 1 --2- 172.21.15.116:0/1701561181 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce58101de0 0x7fce580744f0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fce4800c890 tx=0x7fce4800cd50 comp rx=0 tx=0).stop 2024-09-12T03:12:57.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.241+0000 7fce5e9db640 1 -- 172.21.15.116:0/1701561181 shutdown_connections 2024-09-12T03:12:57.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.241+0000 7fce5e9db640 1 --2- 172.21.15.116:0/1701561181 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fce400a2b80 0x7fce400a5040 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.241+0000 7fce5e9db640 1 --2- 172.21.15.116:0/1701561181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce58106f10 0x7fce58074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.241+0000 7fce5e9db640 1 --2- 172.21.15.116:0/1701561181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce58102790 0x7fce58074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.241+0000 7fce5e9db640 1 --2- 172.21.15.116:0/1701561181 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce58101de0 0x7fce580744f0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:57.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.241+0000 7fce5e9db640 1 -- 172.21.15.116:0/1701561181 >> 172.21.15.116:0/1701561181 conn(0x7fce580fd8c0 msgr2=0x7fce580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:57.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.241+0000 7fce5e9db640 1 -- 172.21.15.116:0/1701561181 shutdown_connections 2024-09-12T03:12:57.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:57.241+0000 7fce5e9db640 1 -- 172.21.15.116:0/1701561181 wait complete. 2024-09-12T03:12:57.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:57 smithi183 ceph-mon[35535]: pgmap v637: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-12T03:12:57.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:57 smithi177 ceph-mon[35305]: pgmap v637: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-12T03:12:57.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:57 smithi116 ceph-mon[31410]: pgmap v637: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-12T03:12:58.044 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:12:58.044 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:12:58.289 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:12:58.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.790+0000 7f66a81f0640 1 -- 172.21.15.116:0/2597825562 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66a01082a0 msgr2=0x7f66a010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:58.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.790+0000 7f66a81f0640 1 --2- 172.21.15.116:0/2597825562 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66a01082a0 0x7f66a010ec40 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f6698009930 tx=0x7f669802a420 comp rx=0 tx=0).stop 2024-09-12T03:12:58.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.790+0000 7f66a81f0640 1 -- 172.21.15.116:0/2597825562 shutdown_connections 2024-09-12T03:12:58.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.790+0000 7f66a81f0640 1 --2- 172.21.15.116:0/2597825562 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66a01082a0 0x7f66a010ec40 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:58.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.790+0000 7f66a81f0640 1 --2- 172.21.15.116:0/2597825562 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66a01036e0 0x7f66a0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:58.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.790+0000 7f66a81f0640 1 --2- 172.21.15.116:0/2597825562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66a0102d30 0x7f66a0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:58.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.790+0000 7f66a81f0640 1 -- 172.21.15.116:0/2597825562 >> 172.21.15.116:0/2597825562 conn(0x7f66a00fc7d0 msgr2=0x7f66a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:58.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.790+0000 7f66a81f0640 1 -- 172.21.15.116:0/2597825562 shutdown_connections 2024-09-12T03:12:58.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.790+0000 7f66a81f0640 1 -- 172.21.15.116:0/2597825562 wait complete. 2024-09-12T03:12:58.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.790+0000 7f66a81f0640 1 Processor -- start 2024-09-12T03:12:58.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.791+0000 7f66a81f0640 1 -- start start 2024-09-12T03:12:58.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.791+0000 7f66a81f0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66a0102d30 0x7f66a0198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:58.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.791+0000 7f66a81f0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66a01036e0 0x7f66a0198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:58.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.791+0000 7f66a81f0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66a01082a0 0x7f66a019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:58.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.791+0000 7f66a81f0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f66a01120a0 con 0x7f66a0102d30 2024-09-12T03:12:58.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.791+0000 7f66a81f0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f66a0111f20 con 0x7f66a01036e0 2024-09-12T03:12:58.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.791+0000 7f66a81f0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f66a0112220 con 0x7f66a01082a0 2024-09-12T03:12:58.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.791+0000 7f66a5f8c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66a0102d30 0x7f66a0198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:58.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.791+0000 7f66a5f8c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66a0102d30 0x7f66a0198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44160/0 (socket says 172.21.15.116:44160) 2024-09-12T03:12:58.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.791+0000 7f66a5f8c640 1 -- 172.21.15.116:0/2679783289 learned_addr learned my addr 172.21.15.116:0/2679783289 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:12:58.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.791+0000 7f66a678d640 1 --2- 172.21.15.116:0/2679783289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66a01082a0 0x7f66a019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:58.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.791+0000 7f66a578b640 1 --2- 172.21.15.116:0/2679783289 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66a01036e0 0x7f66a0198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:58.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.791+0000 7f66a5f8c640 1 -- 172.21.15.116:0/2679783289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66a01082a0 msgr2=0x7f66a019ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:58.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.791+0000 7f66a5f8c640 1 --2- 172.21.15.116:0/2679783289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66a01082a0 0x7f66a019ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:58.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.791+0000 7f66a5f8c640 1 -- 172.21.15.116:0/2679783289 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66a01036e0 msgr2=0x7f66a0198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:58.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.791+0000 7f66a5f8c640 1 --2- 172.21.15.116:0/2679783289 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66a01036e0 0x7f66a0198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:58.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.791+0000 7f66a5f8c640 1 -- 172.21.15.116:0/2679783289 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a019d4e0 con 0x7f66a0102d30 2024-09-12T03:12:58.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.791+0000 7f66a578b640 1 --2- 172.21.15.116:0/2679783289 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66a01036e0 0x7f66a0198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:12:58.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.791+0000 7f66a5f8c640 1 --2- 172.21.15.116:0/2679783289 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66a0102d30 0x7f66a0198530 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f669000cce0 tx=0x7f669000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:58.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.792+0000 7f669effd640 1 -- 172.21.15.116:0/2679783289 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690004b20 con 0x7f66a0102d30 2024-09-12T03:12:58.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.792+0000 7f669effd640 1 -- 172.21.15.116:0/2679783289 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6690004cc0 con 0x7f66a0102d30 2024-09-12T03:12:58.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.792+0000 7f66a81f0640 1 -- 172.21.15.116:0/2679783289 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a019d7d0 con 0x7f66a0102d30 2024-09-12T03:12:58.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.792+0000 7f669effd640 1 -- 172.21.15.116:0/2679783289 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669000f7d0 con 0x7f66a0102d30 2024-09-12T03:12:58.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.792+0000 7f66a81f0640 1 -- 172.21.15.116:0/2679783289 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a01a50b0 con 0x7f66a0102d30 2024-09-12T03:12:58.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.793+0000 7f66a81f0640 1 -- 172.21.15.116:0/2679783289 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66a0103110 con 0x7f66a0102d30 2024-09-12T03:12:58.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.794+0000 7f669effd640 1 -- 172.21.15.116:0/2679783289 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f669000f970 con 0x7f66a0102d30 2024-09-12T03:12:58.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.794+0000 7f669effd640 1 --2- 172.21.15.116:0/2679783289 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f66880a2b80 0x7f66880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:58.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.794+0000 7f66a578b640 1 --2- 172.21.15.116:0/2679783289 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f66880a2b80 0x7f66880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:58.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.795+0000 7f669effd640 1 -- 172.21.15.116:0/2679783289 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f66900cf4a0 con 0x7f66a0102d30 2024-09-12T03:12:58.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.795+0000 7f66a578b640 1 --2- 172.21.15.116:0/2679783289 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f66880a2b80 0x7f66880a5040 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6694005d90 tx=0x7f6694005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:58.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.798+0000 7f669effd640 1 -- 172.21.15.116:0/2679783289 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6690014030 con 0x7f66a0102d30 2024-09-12T03:12:58.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.995+0000 7f66a81f0640 1 -- 172.21.15.116:0/2679783289 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66a0199890 con 0x7f66a0102d30 2024-09-12T03:12:58.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.996+0000 7f669effd640 1 -- 172.21.15.116:0/2679783289 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f669009c430 con 0x7f66a0102d30 2024-09-12T03:12:58.996 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:12:58.996 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:12:58.997 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:12:58.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.998+0000 7f66a81f0640 1 -- 172.21.15.116:0/2679783289 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f66880a2b80 msgr2=0x7f66880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:58.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.998+0000 7f66a81f0640 1 --2- 172.21.15.116:0/2679783289 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f66880a2b80 0x7f66880a5040 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6694005d90 tx=0x7f6694005d20 comp rx=0 tx=0).stop 2024-09-12T03:12:58.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.998+0000 7f66a81f0640 1 -- 172.21.15.116:0/2679783289 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66a0102d30 msgr2=0x7f66a0198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:58.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.998+0000 7f66a81f0640 1 --2- 172.21.15.116:0/2679783289 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66a0102d30 0x7f66a0198530 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f669000cce0 tx=0x7f669000a620 comp rx=0 tx=0).stop 2024-09-12T03:12:58.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.998+0000 7f66a81f0640 1 -- 172.21.15.116:0/2679783289 shutdown_connections 2024-09-12T03:12:58.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.998+0000 7f66a81f0640 1 --2- 172.21.15.116:0/2679783289 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f66880a2b80 0x7f66880a5040 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:58.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.998+0000 7f66a81f0640 1 --2- 172.21.15.116:0/2679783289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66a01082a0 0x7f66a019ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:58.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.998+0000 7f66a81f0640 1 --2- 172.21.15.116:0/2679783289 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66a01036e0 0x7f66a0198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:58.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.998+0000 7f66a81f0640 1 --2- 172.21.15.116:0/2679783289 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66a0102d30 0x7f66a0198530 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:58.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.998+0000 7f66a81f0640 1 -- 172.21.15.116:0/2679783289 >> 172.21.15.116:0/2679783289 conn(0x7f66a00fc7d0 msgr2=0x7f66a0105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:58.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.998+0000 7f66a81f0640 1 -- 172.21.15.116:0/2679783289 shutdown_connections 2024-09-12T03:12:58.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:12:58.998+0000 7f66a81f0640 1 -- 172.21.15.116:0/2679783289 wait complete. 2024-09-12T03:12:59.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:12:59 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2679783289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:12:59.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:12:59 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2679783289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:12:59.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:12:59 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2679783289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:12:59.772 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:13:00.016 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:13:00.321 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:00 smithi116 ceph-mon[31410]: pgmap v638: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-09-12T03:13:00.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:00 smithi183 ceph-mon[35535]: pgmap v638: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-09-12T03:13:00.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:00 smithi177 ceph-mon[35305]: pgmap v638: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-09-12T03:13:00.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.535+0000 7f85e02c5640 1 -- 172.21.15.116:0/1783342978 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85d81002f0 msgr2=0x7f85d81006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:00.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.535+0000 7f85e02c5640 1 --2- 172.21.15.116:0/1783342978 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85d81002f0 0x7f85d81006d0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009930 tx=0x7f85d402a420 comp rx=0 tx=0).stop 2024-09-12T03:13:00.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.535+0000 7f85e02c5640 1 -- 172.21.15.116:0/1783342978 shutdown_connections 2024-09-12T03:13:00.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.535+0000 7f85e02c5640 1 --2- 172.21.15.116:0/1783342978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85d8106f50 0x7f85d810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.535+0000 7f85e02c5640 1 --2- 172.21.15.116:0/1783342978 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85d8100ca0 0x7f85d8106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.535+0000 7f85e02c5640 1 --2- 172.21.15.116:0/1783342978 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85d81002f0 0x7f85d81006d0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.535+0000 7f85e02c5640 1 -- 172.21.15.116:0/1783342978 >> 172.21.15.116:0/1783342978 conn(0x7f85d8077210 msgr2=0x7f85d80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:00.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.535+0000 7f85e02c5640 1 -- 172.21.15.116:0/1783342978 shutdown_connections 2024-09-12T03:13:00.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.535+0000 7f85e02c5640 1 -- 172.21.15.116:0/1783342978 wait complete. 2024-09-12T03:13:00.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.536+0000 7f85e02c5640 1 Processor -- start 2024-09-12T03:13:00.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.536+0000 7f85e02c5640 1 -- start start 2024-09-12T03:13:00.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.536+0000 7f85e02c5640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85d81002f0 0x7f85d8198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:00.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.536+0000 7f85e02c5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85d8100ca0 0x7f85d8198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:00.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.536+0000 7f85e02c5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85d8106f50 0x7f85d819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:00.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.536+0000 7f85e02c5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f85d8112020 con 0x7f85d8106f50 2024-09-12T03:13:00.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.536+0000 7f85e02c5640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f85d8111ea0 con 0x7f85d81002f0 2024-09-12T03:13:00.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.536+0000 7f85e02c5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f85d81121a0 con 0x7f85d8100ca0 2024-09-12T03:13:00.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.536+0000 7f85de862640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85d8106f50 0x7f85d819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:00.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.536+0000 7f85de862640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85d8106f50 0x7f85d819ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44184/0 (socket says 172.21.15.116:44184) 2024-09-12T03:13:00.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.536+0000 7f85de862640 1 -- 172.21.15.116:0/4074776224 learned_addr learned my addr 172.21.15.116:0/4074776224 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:00.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.536+0000 7f85dd860640 1 --2- 172.21.15.116:0/4074776224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85d8100ca0 0x7f85d8198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:00.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.536+0000 7f85de061640 1 --2- 172.21.15.116:0/4074776224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85d81002f0 0x7f85d8198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:00.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.536+0000 7f85de862640 1 -- 172.21.15.116:0/4074776224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85d8100ca0 msgr2=0x7f85d8198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:00.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.536+0000 7f85de862640 1 --2- 172.21.15.116:0/4074776224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85d8100ca0 0x7f85d8198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.536+0000 7f85de862640 1 -- 172.21.15.116:0/4074776224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85d81002f0 msgr2=0x7f85d8198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:00.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.536+0000 7f85de862640 1 --2- 172.21.15.116:0/4074776224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85d81002f0 0x7f85d8198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.536+0000 7f85de862640 1 -- 172.21.15.116:0/4074776224 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d819d520 con 0x7f85d8106f50 2024-09-12T03:13:00.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.537+0000 7f85de862640 1 --2- 172.21.15.116:0/4074776224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85d8106f50 0x7f85d819ce40 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f85d000eaa0 tx=0x7f85d000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:00.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.537+0000 7f85cf7fe640 1 -- 172.21.15.116:0/4074776224 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d0004d20 con 0x7f85d8106f50 2024-09-12T03:13:00.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.537+0000 7f85cf7fe640 1 -- 172.21.15.116:0/4074776224 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85d0005020 con 0x7f85d8106f50 2024-09-12T03:13:00.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.537+0000 7f85e02c5640 1 -- 172.21.15.116:0/4074776224 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d819d810 con 0x7f85d8106f50 2024-09-12T03:13:00.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.537+0000 7f85cf7fe640 1 -- 172.21.15.116:0/4074776224 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d00116d0 con 0x7f85d8106f50 2024-09-12T03:13:00.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.537+0000 7f85e02c5640 1 -- 172.21.15.116:0/4074776224 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d81a50f0 con 0x7f85d8106f50 2024-09-12T03:13:00.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.539+0000 7f85cf7fe640 1 -- 172.21.15.116:0/4074776224 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f85d00051d0 con 0x7f85d8106f50 2024-09-12T03:13:00.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.539+0000 7f85e269e740 1 Processor -- start 2024-09-12T03:13:00.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.539+0000 7f85e269e740 1 -- start start 2024-09-12T03:13:00.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.540+0000 7f85e269e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56441c341b40 0x56441c340090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:00.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.540+0000 7f85e269e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56441c3405d0 0x56441c340a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:00.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.540+0000 7f85e269e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56441c34b370 0x56441c34d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:00.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.540+0000 7f85e269e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56441c2a62f0 con 0x56441c34b370 2024-09-12T03:13:00.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.540+0000 7f85e269e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56441c24dcf0 con 0x56441c3405d0 2024-09-12T03:13:00.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.540+0000 7f85e269e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56441c2a22b0 con 0x56441c341b40 2024-09-12T03:13:00.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.540+0000 7f85dd860640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56441c34b370 0x56441c34d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:00.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.540+0000 7f85dd860640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56441c34b370 0x56441c34d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44188/0 (socket says 172.21.15.116:44188) 2024-09-12T03:13:00.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.540+0000 7f85dd860640 1 -- 172.21.15.116:0/248265270 learned_addr learned my addr 172.21.15.116:0/248265270 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:00.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.540+0000 7f85de061640 1 --2- 172.21.15.116:0/248265270 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56441c3405d0 0x56441c340a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:00.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.540+0000 7f85dd860640 1 --2- 172.21.15.116:0/248265270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56441c341b40 0x56441c340090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:00.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.540+0000 7f85cf7fe640 1 --2- 172.21.15.116:0/4074776224 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f85c40a2b80 0x7f85c40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:00.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.540+0000 7f85dd860640 1 -- 172.21.15.116:0/248265270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56441c341b40 msgr2=0x56441c340090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:00.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.540+0000 7f85dd860640 1 --2- 172.21.15.116:0/248265270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56441c341b40 0x56441c340090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.540+0000 7f85dd860640 1 -- 172.21.15.116:0/248265270 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56441c3405d0 msgr2=0x56441c340a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:00.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.540+0000 7f85dd860640 1 --2- 172.21.15.116:0/248265270 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56441c3405d0 0x56441c340a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.540+0000 7f85dd860640 1 -- 172.21.15.116:0/248265270 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56441c2323a0 con 0x56441c34b370 2024-09-12T03:13:00.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.540+0000 7f85de862640 1 --2- 172.21.15.116:0/4074776224 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f85c40a2b80 0x7f85c40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:00.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.540+0000 7f85dd860640 1 --2- 172.21.15.116:0/248265270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56441c34b370 0x56441c34d760 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009f20 tx=0x7f85c8009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b96ead193dcdff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:00.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.540+0000 7f85cf7fe640 1 -- 172.21.15.116:0/4074776224 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f85d00cf2b0 con 0x7f85d8106f50 2024-09-12T03:13:00.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.541+0000 7f85ccdea640 1 -- 172.21.15.116:0/248265270 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8017610 con 0x56441c34b370 2024-09-12T03:13:00.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.541+0000 7f85ccdea640 1 -- 172.21.15.116:0/248265270 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c801c070 con 0x56441c34b370 2024-09-12T03:13:00.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.541+0000 7f85ccdea640 1 -- 172.21.15.116:0/248265270 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8015a40 con 0x56441c34b370 2024-09-12T03:13:00.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.541+0000 7f85e269e740 1 -- 172.21.15.116:0/248265270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56441c34b370 msgr2=0x56441c34d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:00.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.541+0000 7f85e269e740 1 --2- 172.21.15.116:0/248265270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56441c34b370 0x56441c34d760 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009f20 tx=0x7f85c8009560 comp rx=0 tx=0).stop 2024-09-12T03:13:00.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.541+0000 7f85de862640 1 --2- 172.21.15.116:0/4074776224 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f85c40a2b80 0x7f85c40a5040 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f85d009ba70 tx=0x7f85d006eba0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:00.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.541+0000 7f85e269e740 1 -- 172.21.15.116:0/248265270 shutdown_connections 2024-09-12T03:13:00.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.541+0000 7f85e269e740 1 --2- 172.21.15.116:0/248265270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56441c34b370 0x56441c34d760 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.541+0000 7f85e269e740 1 --2- 172.21.15.116:0/248265270 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56441c3405d0 0x56441c340a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.541+0000 7f85e269e740 1 --2- 172.21.15.116:0/248265270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56441c341b40 0x56441c340090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.541+0000 7f85e269e740 1 -- 172.21.15.116:0/248265270 >> 172.21.15.116:0/248265270 conn(0x56441c309310 msgr2=0x56441c33cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:00.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.541+0000 7f85e269e740 1 -- 172.21.15.116:0/248265270 shutdown_connections 2024-09-12T03:13:00.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.542+0000 7f85e269e740 1 -- 172.21.15.116:0/248265270 wait complete. 2024-09-12T03:13:00.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.542+0000 7f85e269e740 1 -- start start 2024-09-12T03:13:00.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.542+0000 7f85e269e740 1 Processor -- start 2024-09-12T03:13:00.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.542+0000 7f85e269e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56441c3405d0 0x56441c34add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:00.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.542+0000 7f85e269e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56441c341b40 0x56441c348080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:00.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.542+0000 7f85e269e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56441c34b370 0x56441c348810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:00.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.542+0000 7f85e269e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56441c34e070 con 0x56441c3405d0 2024-09-12T03:13:00.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.542+0000 7f85e269e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56441c261060 con 0x56441c34b370 2024-09-12T03:13:00.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.542+0000 7f85e269e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56441c34e1f0 con 0x56441c341b40 2024-09-12T03:13:00.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.542+0000 7f85e269e740 10 client.0 _ll_register_callbacks cb 0x56441bfb48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:00.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.542+0000 7f85dd860640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56441c3405d0 0x56441c34add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:00.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.542+0000 7f85dd860640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56441c3405d0 0x56441c34add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44196/0 (socket says 172.21.15.116:44196) 2024-09-12T03:13:00.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.542+0000 7f85dd860640 1 -- 172.21.15.116:0/3757478786 learned_addr learned my addr 172.21.15.116:0/3757478786 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:00.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.542+0000 7f85de061640 1 --2- 172.21.15.116:0/3757478786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56441c341b40 0x56441c348080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:00.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.542+0000 7f85dd860640 1 -- 172.21.15.116:0/3757478786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56441c341b40 msgr2=0x56441c348080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:00.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.542+0000 7f85dd860640 1 --2- 172.21.15.116:0/3757478786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56441c341b40 0x56441c348080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.542+0000 7f85dd860640 1 -- 172.21.15.116:0/3757478786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56441c34b370 msgr2=0x56441c348810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:00.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.542+0000 7f85dd860640 1 --2- 172.21.15.116:0/3757478786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56441c34b370 0x56441c348810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.543+0000 7f85dd860640 1 -- 172.21.15.116:0/3757478786 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56441c0173c0 con 0x56441c3405d0 2024-09-12T03:13:00.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.543+0000 7f85de061640 1 --2- 172.21.15.116:0/3757478786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56441c341b40 0x56441c348080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:13:00.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.543+0000 7f85dd860640 1 --2- 172.21.15.116:0/3757478786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56441c3405d0 0x56441c34add0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009650 tx=0x7f85c8009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b1c5b4bd8eaa56e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:00.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.543+0000 7f85c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:00.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.543+0000 7f85c1ffb640 1 -- 172.21.15.116:0/3757478786 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8005dd0 con 0x56441c3405d0 2024-09-12T03:13:00.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.543+0000 7f85c1ffb640 1 -- 172.21.15.116:0/3757478786 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f85c80157c0 con 0x56441c3405d0 2024-09-12T03:13:00.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.543+0000 7f85e269e740 1 -- 172.21.15.116:0/3757478786 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85c801c070 con 0x56441c3405d0 2024-09-12T03:13:00.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.543+0000 7f85c1ffb640 1 -- 172.21.15.116:0/3757478786 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f85c80159c0 con 0x56441c3405d0 2024-09-12T03:13:00.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.543+0000 7f85e269e740 10 client.18054 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:00.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.543+0000 7f85e269e740 1 -- 172.21.15.116:0/3757478786 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56441c359b90 con 0x56441c3405d0 2024-09-12T03:13:00.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.543+0000 7f85c1ffb640 1 -- 172.21.15.116:0/3757478786 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c801c070 con 0x56441c3405d0 2024-09-12T03:13:00.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.543+0000 7f85c1ffb640 1 -- 172.21.15.116:0/3757478786 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f85c802e450 con 0x56441c3405d0 2024-09-12T03:13:00.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.543+0000 7f85e269e740 10 client.18054 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:00.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.543+0000 7f85e269e740 10 client.18054 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:13:00.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.543+0000 7f85e269e740 20 client.18054 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:00.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.543+0000 7f85e269e740 1 --2- 172.21.15.116:0/3757478786 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x56441c35af40 0x56441c35d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:00.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.543+0000 7f85e269e740 4 client.18054 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:13:00.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.543+0000 7f85e269e740 1 -- 172.21.15.116:0/3757478786 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56441c2b3860 con 0x56441c35af40 2024-09-12T03:13:00.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.544+0000 7f85de061640 1 --2- 172.21.15.116:0/3757478786 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x56441c35af40 0x56441c35d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:00.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.544+0000 7f85de061640 1 --2- 172.21.15.116:0/3757478786 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x56441c35af40 0x56441c35d330 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2aacd4e546e76844 server_cookie=522423fb3cdd740e in_seq=0 out_seq=0 2024-09-12T03:13:00.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.544+0000 7f85c1ffb640 10 client.18054 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:13:00.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.545+0000 7f85c1ffb640 1 -- 172.21.15.116:0/3757478786 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56441c2b3860 con 0x56441c35af40 2024-09-12T03:13:00.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.545+0000 7f85c1ffb640 10 client.18054 handle_command_reply: tid=0 2024-09-12T03:13:00.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.545+0000 7f85e269e740 1 client.18054 shutdown 2024-09-12T03:13:00.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.545+0000 7f85e269e740 1 -- 172.21.15.116:0/3757478786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56441c3405d0 msgr2=0x56441c34add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:00.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.545+0000 7f85e269e740 1 --2- 172.21.15.116:0/3757478786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56441c3405d0 0x56441c34add0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009650 tx=0x7f85c8009110 comp rx=0 tx=0).stop 2024-09-12T03:13:00.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.545+0000 7f85e269e740 1 -- 172.21.15.116:0/3757478786 shutdown_connections 2024-09-12T03:13:00.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.545+0000 7f85e269e740 1 --2- 172.21.15.116:0/3757478786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56441c34b370 0x56441c348810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.545+0000 7f85e269e740 1 --2- 172.21.15.116:0/3757478786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56441c341b40 0x56441c348080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.545+0000 7f85e269e740 1 --2- 172.21.15.116:0/3757478786 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x56441c35af40 0x56441c35d330 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.545+0000 7f85e269e740 1 --2- 172.21.15.116:0/3757478786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56441c3405d0 0x56441c34add0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.545+0000 7f85c1ffb640 0 client.18054 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:13:00.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.545+0000 7f85e269e740 1 -- 172.21.15.116:0/3757478786 >> 172.21.15.116:0/3757478786 conn(0x56441c309310 msgr2=0x56441c342960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:00.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.545+0000 7f85e269e740 1 -- 172.21.15.116:0/3757478786 shutdown_connections 2024-09-12T03:13:00.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.545+0000 7f85e269e740 1 -- 172.21.15.116:0/3757478786 wait complete. 2024-09-12T03:13:00.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.545+0000 7f85e269e740 20 client.18054 trim_cache size 0 max 16384 2024-09-12T03:13:00.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.558+0000 7f85e269e740 1 Processor -- start 2024-09-12T03:13:00.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.558+0000 7f85e269e740 1 -- start start 2024-09-12T03:13:00.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.559+0000 7f85e269e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56441c3405d0 0x56441c347940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:00.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.559+0000 7f85e269e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56441c341b40 0x56441c347060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:00.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.559+0000 7f85e269e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56441c34b370 0x56441c3628d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:00.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.559+0000 7f85e269e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f85d81aff20 con 0x56441c341b40 2024-09-12T03:13:00.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.559+0000 7f85e269e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f85d81ad7f0 con 0x56441c3405d0 2024-09-12T03:13:00.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.559+0000 7f85e269e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f85d81ab660 con 0x56441c34b370 2024-09-12T03:13:00.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.559+0000 7f85de061640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56441c341b40 0x56441c347060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:00.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.559+0000 7f85de061640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56441c341b40 0x56441c347060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44200/0 (socket says 172.21.15.116:44200) 2024-09-12T03:13:00.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.559+0000 7f85de061640 1 -- 172.21.15.116:0/3108042457 learned_addr learned my addr 172.21.15.116:0/3108042457 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:00.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.559+0000 7f85dd860640 1 --2- 172.21.15.116:0/3108042457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56441c3405d0 0x56441c347940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:00.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.559+0000 7f85de061640 1 -- 172.21.15.116:0/3108042457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56441c34b370 msgr2=0x56441c3628d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:00.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.559+0000 7f85de061640 1 --2- 172.21.15.116:0/3108042457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56441c34b370 0x56441c3628d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.559+0000 7f85de061640 1 -- 172.21.15.116:0/3108042457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56441c3405d0 msgr2=0x56441c347940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:00.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.559+0000 7f85de061640 1 --2- 172.21.15.116:0/3108042457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56441c3405d0 0x56441c347940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.559+0000 7f85de061640 1 -- 172.21.15.116:0/3108042457 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56441c340be0 con 0x56441c341b40 2024-09-12T03:13:00.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.559+0000 7f85dd860640 1 --2- 172.21.15.116:0/3108042457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56441c3405d0 0x56441c347940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:13:00.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.559+0000 7f85de061640 1 --2- 172.21.15.116:0/3108042457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56441c341b40 0x56441c347060 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f85d402a940 tx=0x7f85d4005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30e4746109d37af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:00.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.559+0000 7f85c3fff640 1 -- 172.21.15.116:0/3108042457 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4049070 con 0x56441c341b40 2024-09-12T03:13:00.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.559+0000 7f85c3fff640 1 -- 172.21.15.116:0/3108042457 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85d4004460 con 0x56441c341b40 2024-09-12T03:13:00.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.559+0000 7f85c3fff640 1 -- 172.21.15.116:0/3108042457 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4035040 con 0x56441c341b40 2024-09-12T03:13:00.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.560+0000 7f85e269e740 1 -- 172.21.15.116:0/3108042457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56441c341b40 msgr2=0x56441c347060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:00.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.560+0000 7f85e269e740 1 --2- 172.21.15.116:0/3108042457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56441c341b40 0x56441c347060 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f85d402a940 tx=0x7f85d4005d90 comp rx=0 tx=0).stop 2024-09-12T03:13:00.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.560+0000 7f85e269e740 1 -- 172.21.15.116:0/3108042457 shutdown_connections 2024-09-12T03:13:00.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.560+0000 7f85e269e740 1 --2- 172.21.15.116:0/3108042457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56441c34b370 0x56441c3628d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.560+0000 7f85e269e740 1 --2- 172.21.15.116:0/3108042457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56441c341b40 0x56441c347060 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.560+0000 7f85e269e740 1 --2- 172.21.15.116:0/3108042457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56441c3405d0 0x56441c347940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.560+0000 7f85e269e740 1 -- 172.21.15.116:0/3108042457 >> 172.21.15.116:0/3108042457 conn(0x56441c309310 msgr2=0x56441c346950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:00.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.560+0000 7f85e269e740 1 -- 172.21.15.116:0/3108042457 shutdown_connections 2024-09-12T03:13:00.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.560+0000 7f85e269e740 1 -- 172.21.15.116:0/3108042457 wait complete. 2024-09-12T03:13:00.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.560+0000 7f85e269e740 1 -- start start 2024-09-12T03:13:00.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.560+0000 7f85e269e740 1 Processor -- start 2024-09-12T03:13:00.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.561+0000 7f85e269e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56441c3405d0 0x56441c353830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:00.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.561+0000 7f85e269e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56441c341b40 0x56441c353d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:00.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.561+0000 7f85e269e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56441c34b370 0x56441c34ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:00.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.561+0000 7f85e269e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f85d81abcb0 con 0x56441c341b40 2024-09-12T03:13:00.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.561+0000 7f85e269e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f85d81ad120 con 0x56441c3405d0 2024-09-12T03:13:00.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.561+0000 7f85e269e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f85d81ac380 con 0x56441c34b370 2024-09-12T03:13:00.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.561+0000 7f85e269e740 10 client.0 _ll_register_callbacks cb 0x7f85d4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:00.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.561+0000 7f85de061640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56441c341b40 0x56441c353d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:00.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.561+0000 7f85de061640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56441c341b40 0x56441c353d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44216/0 (socket says 172.21.15.116:44216) 2024-09-12T03:13:00.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.561+0000 7f85de061640 1 -- 172.21.15.116:0/203809981 learned_addr learned my addr 172.21.15.116:0/203809981 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:00.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.561+0000 7f85dd860640 1 --2- 172.21.15.116:0/203809981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56441c3405d0 0x56441c353830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:00.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.561+0000 7f85dd860640 1 --2- 172.21.15.116:0/203809981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56441c34b370 0x56441c34ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:00.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.561+0000 7f85de061640 1 -- 172.21.15.116:0/203809981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56441c34b370 msgr2=0x56441c34ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:00.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.561+0000 7f85de061640 1 --2- 172.21.15.116:0/203809981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56441c34b370 0x56441c34ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.561+0000 7f85de061640 1 -- 172.21.15.116:0/203809981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56441c3405d0 msgr2=0x56441c353830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:00.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.561+0000 7f85de061640 1 --2- 172.21.15.116:0/203809981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56441c3405d0 0x56441c353830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.561+0000 7f85de061640 1 -- 172.21.15.116:0/203809981 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56441c34f630 con 0x56441c341b40 2024-09-12T03:13:00.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.561+0000 7f85dd860640 1 --2- 172.21.15.116:0/203809981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56441c34b370 0x56441c34ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:13:00.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.561+0000 7f85de061640 1 --2- 172.21.15.116:0/203809981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56441c341b40 0x56441c353d70 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f85d4047040 tx=0x7f85d402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc75586e062faee4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:00.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.561+0000 7f85c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:00.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.561+0000 7f85c1ffb640 1 -- 172.21.15.116:0/203809981 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4049040 con 0x56441c341b40 2024-09-12T03:13:00.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.562+0000 7f85c1ffb640 1 -- 172.21.15.116:0/203809981 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f85d40035f0 con 0x56441c341b40 2024-09-12T03:13:00.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.562+0000 7f85e269e740 1 -- 172.21.15.116:0/203809981 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85d4004460 con 0x56441c341b40 2024-09-12T03:13:00.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.562+0000 7f85c1ffb640 1 -- 172.21.15.116:0/203809981 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f85d4035040 con 0x56441c341b40 2024-09-12T03:13:00.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.562+0000 7f85e269e740 10 client.18060 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:00.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.562+0000 7f85e269e740 1 -- 172.21.15.116:0/203809981 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56441c3c5480 con 0x56441c341b40 2024-09-12T03:13:00.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.562+0000 7f85c1ffb640 1 -- 172.21.15.116:0/203809981 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85d4004460 con 0x56441c341b40 2024-09-12T03:13:00.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.562+0000 7f85c1ffb640 1 -- 172.21.15.116:0/203809981 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f85d4055030 con 0x56441c341b40 2024-09-12T03:13:00.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.562+0000 7f85e269e740 10 client.18060 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:00.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.562+0000 7f85e269e740 10 client.18060 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:13:00.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.562+0000 7f85e269e740 20 client.18060 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:00.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.562+0000 7f85e269e740 1 --2- 172.21.15.116:0/203809981 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x56441c3c5b70 0x56441c3c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:00.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.562+0000 7f85e269e740 4 client.18060 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:13:00.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.562+0000 7f85e269e740 1 -- 172.21.15.116:0/203809981 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56441c2add70 con 0x56441c3c5b70 2024-09-12T03:13:00.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.562+0000 7f85dd860640 1 --2- 172.21.15.116:0/203809981 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x56441c3c5b70 0x56441c3c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:00.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.563+0000 7f85c1ffb640 10 client.18060 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:13:00.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.563+0000 7f85dd860640 1 --2- 172.21.15.116:0/203809981 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x56441c3c5b70 0x56441c3c7f90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9ba12062f02870cc server_cookie=f15d17eb682a364a in_seq=0 out_seq=0 2024-09-12T03:13:00.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.563+0000 7f85c1ffb640 1 -- 172.21.15.116:0/203809981 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56441c2add70 con 0x56441c3c5b70 2024-09-12T03:13:00.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.563+0000 7f85c1ffb640 10 client.18060 handle_command_reply: tid=0 2024-09-12T03:13:00.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.563+0000 7f85e269e740 1 client.18060 shutdown 2024-09-12T03:13:00.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.563+0000 7f85e269e740 1 -- 172.21.15.116:0/203809981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56441c341b40 msgr2=0x56441c353d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:00.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.563+0000 7f85e269e740 1 --2- 172.21.15.116:0/203809981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56441c341b40 0x56441c353d70 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f85d4047040 tx=0x7f85d402a420 comp rx=0 tx=0).stop 2024-09-12T03:13:00.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.563+0000 7f85e269e740 1 -- 172.21.15.116:0/203809981 shutdown_connections 2024-09-12T03:13:00.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.563+0000 7f85e269e740 1 --2- 172.21.15.116:0/203809981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56441c34b370 0x56441c34ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.563+0000 7f85e269e740 1 --2- 172.21.15.116:0/203809981 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x56441c3c5b70 0x56441c3c7f90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.563+0000 7f85e269e740 1 --2- 172.21.15.116:0/203809981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56441c341b40 0x56441c353d70 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.563+0000 7f85c1ffb640 0 client.18060 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:13:00.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.563+0000 7f85e269e740 1 --2- 172.21.15.116:0/203809981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56441c3405d0 0x56441c353830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.563+0000 7f85e269e740 1 -- 172.21.15.116:0/203809981 >> 172.21.15.116:0/203809981 conn(0x56441c309310 msgr2=0x56441c342930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:00.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.563+0000 7f85e269e740 1 -- 172.21.15.116:0/203809981 shutdown_connections 2024-09-12T03:13:00.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.563+0000 7f85e269e740 1 -- 172.21.15.116:0/203809981 wait complete. 2024-09-12T03:13:00.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.563+0000 7f85e269e740 20 client.18060 trim_cache size 0 max 16384 2024-09-12T03:13:00.576 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:13:00.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.564+0000 7f85e02c5640 1 -- 172.21.15.116:0/4074776224 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f85c40a2b80 msgr2=0x7f85c40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:00.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.564+0000 7f85e02c5640 1 --2- 172.21.15.116:0/4074776224 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f85c40a2b80 0x7f85c40a5040 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f85d009ba70 tx=0x7f85d006eba0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.564+0000 7f85e02c5640 1 -- 172.21.15.116:0/4074776224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85d8106f50 msgr2=0x7f85d819ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:00.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.564+0000 7f85e02c5640 1 --2- 172.21.15.116:0/4074776224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85d8106f50 0x7f85d819ce40 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f85d000eaa0 tx=0x7f85d000eea0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.564+0000 7f85e02c5640 1 -- 172.21.15.116:0/4074776224 shutdown_connections 2024-09-12T03:13:00.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.564+0000 7f85e02c5640 1 --2- 172.21.15.116:0/4074776224 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f85c40a2b80 0x7f85c40a5040 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.564+0000 7f85e02c5640 1 --2- 172.21.15.116:0/4074776224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85d8106f50 0x7f85d819ce40 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.564+0000 7f85e02c5640 1 --2- 172.21.15.116:0/4074776224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85d8100ca0 0x7f85d8198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.564+0000 7f85e02c5640 1 --2- 172.21.15.116:0/4074776224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85d81002f0 0x7f85d8198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:00.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.564+0000 7f85e02c5640 1 -- 172.21.15.116:0/4074776224 >> 172.21.15.116:0/4074776224 conn(0x7f85d8077210 msgr2=0x7f85d81052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:00.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.564+0000 7f85e02c5640 1 -- 172.21.15.116:0/4074776224 shutdown_connections 2024-09-12T03:13:00.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:00.564+0000 7f85e02c5640 1 -- 172.21.15.116:0/4074776224 wait complete. 2024-09-12T03:13:01.415 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:13:01.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:01 smithi116 ceph-mon[31410]: pgmap v639: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-12T03:13:01.662 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:13:01.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:01 smithi183 ceph-mon[35535]: pgmap v639: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-12T03:13:01.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:01 smithi177 ceph-mon[35305]: pgmap v639: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-12T03:13:02.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.172+0000 7fd11b41d640 1 -- 172.21.15.116:0/2094879561 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd114106f20 msgr2=0x7fd11410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:02.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.172+0000 7fd11b41d640 1 --2- 172.21.15.116:0/2094879561 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd114106f20 0x7fd11410eaa0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fd104009ca0 tx=0x7fd10402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.172+0000 7fd11b41d640 1 -- 172.21.15.116:0/2094879561 shutdown_connections 2024-09-12T03:13:02.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.173+0000 7fd11b41d640 1 --2- 172.21.15.116:0/2094879561 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd114106f20 0x7fd11410eaa0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.173+0000 7fd11b41d640 1 --2- 172.21.15.116:0/2094879561 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1141027a0 0x7fd1141067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.173+0000 7fd11b41d640 1 --2- 172.21.15.116:0/2094879561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd114101df0 0x7fd1141021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.173+0000 7fd11b41d640 1 -- 172.21.15.116:0/2094879561 >> 172.21.15.116:0/2094879561 conn(0x7fd1140fd8f0 msgr2=0x7fd1140ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:02.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.173+0000 7fd11b41d640 1 -- 172.21.15.116:0/2094879561 shutdown_connections 2024-09-12T03:13:02.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.173+0000 7fd11b41d640 1 -- 172.21.15.116:0/2094879561 wait complete. 2024-09-12T03:13:02.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.173+0000 7fd11b41d640 1 Processor -- start 2024-09-12T03:13:02.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.173+0000 7fd11b41d640 1 -- start start 2024-09-12T03:13:02.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.173+0000 7fd11b41d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd114101df0 0x7fd11419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:02.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.173+0000 7fd11b41d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1141027a0 0x7fd11419ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:02.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.173+0000 7fd11b41d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd114106f20 0x7fd11419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:02.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.174+0000 7fd11b41d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd114112020 con 0x7fd1141027a0 2024-09-12T03:13:02.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.174+0000 7fd11b41d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd114111ea0 con 0x7fd114106f20 2024-09-12T03:13:02.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.174+0000 7fd11b41d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd1141121a0 con 0x7fd114101df0 2024-09-12T03:13:02.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.174+0000 7fd1189b8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1141027a0 0x7fd11419ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:02.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.174+0000 7fd1189b8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1141027a0 0x7fd11419ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34604/0 (socket says 172.21.15.116:34604) 2024-09-12T03:13:02.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.174+0000 7fd1189b8640 1 -- 172.21.15.116:0/296318641 learned_addr learned my addr 172.21.15.116:0/296318641 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:02.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.174+0000 7fd1199ba640 1 --2- 172.21.15.116:0/296318641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd114106f20 0x7fd11419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:02.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.174+0000 7fd1191b9640 1 --2- 172.21.15.116:0/296318641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd114101df0 0x7fd11419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:02.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.174+0000 7fd1189b8640 1 -- 172.21.15.116:0/296318641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd114101df0 msgr2=0x7fd11419a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:02.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.174+0000 7fd1189b8640 1 --2- 172.21.15.116:0/296318641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd114101df0 0x7fd11419a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.174+0000 7fd1189b8640 1 -- 172.21.15.116:0/296318641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd114106f20 msgr2=0x7fd11419f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:02.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.174+0000 7fd1189b8640 1 --2- 172.21.15.116:0/296318641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd114106f20 0x7fd11419f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.174+0000 7fd1189b8640 1 -- 172.21.15.116:0/296318641 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd11419f6f0 con 0x7fd1141027a0 2024-09-12T03:13:02.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.174+0000 7fd1191b9640 1 --2- 172.21.15.116:0/296318641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd114101df0 0x7fd11419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:13:02.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.174+0000 7fd1189b8640 1 --2- 172.21.15.116:0/296318641 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1141027a0 0x7fd11419ac80 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fd10800b4a0 tx=0x7fd10800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:02.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.174+0000 7fd1127fc640 1 -- 172.21.15.116:0/296318641 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd108014070 con 0x7fd1141027a0 2024-09-12T03:13:02.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.174+0000 7fd11b41d640 1 -- 172.21.15.116:0/296318641 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd11419f9e0 con 0x7fd1141027a0 2024-09-12T03:13:02.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.174+0000 7fd1127fc640 1 -- 172.21.15.116:0/296318641 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd108010af0 con 0x7fd1141027a0 2024-09-12T03:13:02.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.174+0000 7fd1127fc640 1 -- 172.21.15.116:0/296318641 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd108017d10 con 0x7fd1141027a0 2024-09-12T03:13:02.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.174+0000 7fd11b41d640 1 -- 172.21.15.116:0/296318641 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1141a72c0 con 0x7fd1141027a0 2024-09-12T03:13:02.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.176+0000 7fd1127fc640 1 -- 172.21.15.116:0/296318641 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd108017070 con 0x7fd1141027a0 2024-09-12T03:13:02.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.177+0000 7fd11d7f6740 1 Processor -- start 2024-09-12T03:13:02.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.177+0000 7fd11d7f6740 1 -- start start 2024-09-12T03:13:02.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.177+0000 7fd11d7f6740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d08bc61b40 0x55d08bc60090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:02.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.177+0000 7fd11d7f6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d08bc605d0 0x55d08bc60a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:02.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.177+0000 7fd11d7f6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d08bc6b370 0x55d08bc6d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:02.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.177+0000 7fd11d7f6740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d08bbc62f0 con 0x55d08bc61b40 2024-09-12T03:13:02.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.177+0000 7fd11d7f6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d08bb6dcf0 con 0x55d08bc605d0 2024-09-12T03:13:02.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.177+0000 7fd11d7f6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d08bbc22b0 con 0x55d08bc6b370 2024-09-12T03:13:02.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.177+0000 7fd1127fc640 1 --2- 172.21.15.116:0/296318641 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd0fc0a2b80 0x7fd0fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:02.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.177+0000 7fd1199ba640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d08bc61b40 0x55d08bc60090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:02.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.177+0000 7fd1199ba640 1 --2- 172.21.15.116:0/296318641 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd0fc0a2b80 0x7fd0fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:02.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.178+0000 7fd1199ba640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d08bc61b40 0x55d08bc60090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34608/0 (socket says 172.21.15.116:34608) 2024-09-12T03:13:02.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.178+0000 7fd1191b9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d08bc605d0 0x55d08bc60a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:02.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.178+0000 7fd1199ba640 1 -- 172.21.15.116:0/668060337 learned_addr learned my addr 172.21.15.116:0/668060337 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:02.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.178+0000 7fd1127fc640 1 -- 172.21.15.116:0/296318641 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd1080ce3e0 con 0x7fd1141027a0 2024-09-12T03:13:02.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.178+0000 7fd1189b8640 1 --2- 172.21.15.116:0/668060337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d08bc6b370 0x55d08bc6d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:02.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.178+0000 7fd1191b9640 1 -- 172.21.15.116:0/668060337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d08bc6b370 msgr2=0x55d08bc6d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:02.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.178+0000 7fd1191b9640 1 --2- 172.21.15.116:0/668060337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d08bc6b370 0x55d08bc6d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.178+0000 7fd1191b9640 1 -- 172.21.15.116:0/668060337 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d08bc61b40 msgr2=0x55d08bc60090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:02.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.178+0000 7fd1191b9640 1 --2- 172.21.15.116:0/668060337 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d08bc61b40 0x55d08bc60090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.178+0000 7fd1191b9640 1 -- 172.21.15.116:0/668060337 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d08bb523a0 con 0x55d08bc605d0 2024-09-12T03:13:02.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.178+0000 7fd1199ba640 1 --2- 172.21.15.116:0/668060337 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d08bc61b40 0x55d08bc60090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:13:02.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.178+0000 7fd1199ba640 1 --2- 172.21.15.116:0/296318641 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd0fc0a2b80 0x7fd0fc0a5040 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd10402ade0 tx=0x7fd104005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:02.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.178+0000 7fd1191b9640 1 --2- 172.21.15.116:0/668060337 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d08bc605d0 0x55d08bc60a10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00a940 tx=0x7fd10c00ae00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=744987bfbdba3d07 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:02.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.178+0000 7fd0fb7fe640 1 -- 172.21.15.116:0/668060337 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c015070 con 0x55d08bc605d0 2024-09-12T03:13:02.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.178+0000 7fd0fb7fe640 1 -- 172.21.15.116:0/668060337 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd10c00cb40 con 0x55d08bc605d0 2024-09-12T03:13:02.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.178+0000 7fd0fb7fe640 1 -- 172.21.15.116:0/668060337 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c0185a0 con 0x55d08bc605d0 2024-09-12T03:13:02.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.179+0000 7fd11d7f6740 1 -- 172.21.15.116:0/668060337 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d08bc605d0 msgr2=0x55d08bc60a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:02.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.179+0000 7fd11d7f6740 1 --2- 172.21.15.116:0/668060337 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d08bc605d0 0x55d08bc60a10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00a940 tx=0x7fd10c00ae00 comp rx=0 tx=0).stop 2024-09-12T03:13:02.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.179+0000 7fd11d7f6740 1 -- 172.21.15.116:0/668060337 shutdown_connections 2024-09-12T03:13:02.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.179+0000 7fd11d7f6740 1 --2- 172.21.15.116:0/668060337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d08bc6b370 0x55d08bc6d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.179+0000 7fd11d7f6740 1 --2- 172.21.15.116:0/668060337 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d08bc605d0 0x55d08bc60a10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.179+0000 7fd11d7f6740 1 --2- 172.21.15.116:0/668060337 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d08bc61b40 0x55d08bc60090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.179+0000 7fd11d7f6740 1 -- 172.21.15.116:0/668060337 >> 172.21.15.116:0/668060337 conn(0x55d08bc29310 msgr2=0x55d08bc5cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:02.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.179+0000 7fd11d7f6740 1 -- 172.21.15.116:0/668060337 shutdown_connections 2024-09-12T03:13:02.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.179+0000 7fd11d7f6740 1 -- 172.21.15.116:0/668060337 wait complete. 2024-09-12T03:13:02.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.180+0000 7fd11d7f6740 1 -- start start 2024-09-12T03:13:02.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.180+0000 7fd11d7f6740 1 Processor -- start 2024-09-12T03:13:02.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.180+0000 7fd11d7f6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d08bc605d0 0x55d08bc6acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:02.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.180+0000 7fd11d7f6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d08bc61b40 0x55d08bc67fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:02.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.180+0000 7fd11d7f6740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d08bc6b370 0x55d08bc68520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:02.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.180+0000 7fd11d7f6740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d08bc6e070 con 0x55d08bc6b370 2024-09-12T03:13:02.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.180+0000 7fd11d7f6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d08bb81060 con 0x55d08bc61b40 2024-09-12T03:13:02.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.180+0000 7fd11d7f6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d08bc6e1f0 con 0x55d08bc605d0 2024-09-12T03:13:02.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.180+0000 7fd11d7f6740 10 client.0 _ll_register_callbacks cb 0x55d08b8d48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:02.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.180+0000 7fd1199ba640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d08bc6b370 0x55d08bc68520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:02.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.180+0000 7fd1199ba640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d08bc6b370 0x55d08bc68520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34610/0 (socket says 172.21.15.116:34610) 2024-09-12T03:13:02.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.180+0000 7fd1199ba640 1 -- 172.21.15.116:0/3545442986 learned_addr learned my addr 172.21.15.116:0/3545442986 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:02.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.180+0000 7fd1191b9640 1 --2- 172.21.15.116:0/3545442986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d08bc605d0 0x55d08bc6acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:02.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.180+0000 7fd1189b8640 1 --2- 172.21.15.116:0/3545442986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d08bc61b40 0x55d08bc67fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:02.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.181+0000 7fd1199ba640 1 -- 172.21.15.116:0/3545442986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d08bc605d0 msgr2=0x55d08bc6acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:02.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.181+0000 7fd1199ba640 1 --2- 172.21.15.116:0/3545442986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d08bc605d0 0x55d08bc6acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.181+0000 7fd1199ba640 1 -- 172.21.15.116:0/3545442986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d08bc61b40 msgr2=0x55d08bc67fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:02.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.181+0000 7fd1199ba640 1 --2- 172.21.15.116:0/3545442986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d08bc61b40 0x55d08bc67fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.181+0000 7fd1199ba640 1 -- 172.21.15.116:0/3545442986 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d08b9373c0 con 0x55d08bc6b370 2024-09-12T03:13:02.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.181+0000 7fd1199ba640 1 --2- 172.21.15.116:0/3545442986 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d08bc6b370 0x55d08bc68520 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fd10402af90 tx=0x7fd1040093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=493a6f9011fc4156 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:02.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.181+0000 7fd0f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:02.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.181+0000 7fd0f8ff9640 1 -- 172.21.15.116:0/3545442986 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd104041070 con 0x55d08bc6b370 2024-09-12T03:13:02.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.181+0000 7fd11d7f6740 1 -- 172.21.15.116:0/3545442986 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd10c00cb40 con 0x55d08bc6b370 2024-09-12T03:13:02.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.181+0000 7fd0f8ff9640 1 -- 172.21.15.116:0/3545442986 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd104003b70 con 0x55d08bc6b370 2024-09-12T03:13:02.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.181+0000 7fd0f8ff9640 1 -- 172.21.15.116:0/3545442986 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd104044040 con 0x55d08bc6b370 2024-09-12T03:13:02.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.181+0000 7fd11d7f6740 10 client.18072 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:02.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.181+0000 7fd11d7f6740 1 -- 172.21.15.116:0/3545442986 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d08bc79aa0 con 0x55d08bc6b370 2024-09-12T03:13:02.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.181+0000 7fd0f8ff9640 1 -- 172.21.15.116:0/3545442986 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd10c00cb40 con 0x55d08bc6b370 2024-09-12T03:13:02.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.181+0000 7fd0f8ff9640 1 -- 172.21.15.116:0/3545442986 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd1040041e0 con 0x55d08bc6b370 2024-09-12T03:13:02.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.181+0000 7fd11d7f6740 10 client.18072 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:02.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.181+0000 7fd11d7f6740 10 client.18072 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:13:02.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.181+0000 7fd11d7f6740 20 client.18072 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:02.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.181+0000 7fd11d7f6740 1 --2- 172.21.15.116:0/3545442986 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55d08bc7ae40 0x55d08bc7d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:02.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.181+0000 7fd11d7f6740 4 client.18072 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:13:02.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.181+0000 7fd11d7f6740 1 -- 172.21.15.116:0/3545442986 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d08bbd3860 con 0x55d08bc7ae40 2024-09-12T03:13:02.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.182+0000 7fd1191b9640 1 --2- 172.21.15.116:0/3545442986 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55d08bc7ae40 0x55d08bc7d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:02.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.182+0000 7fd1191b9640 1 --2- 172.21.15.116:0/3545442986 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55d08bc7ae40 0x55d08bc7d2b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=51300dbb0eed8ce server_cookie=75061a0c7319ad65 in_seq=0 out_seq=0 2024-09-12T03:13:02.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.182+0000 7fd0f8ff9640 10 client.18072 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:13:02.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.183+0000 7fd0f8ff9640 1 -- 172.21.15.116:0/3545442986 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d08bbd3860 con 0x55d08bc7ae40 2024-09-12T03:13:02.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.183+0000 7fd0f8ff9640 10 client.18072 handle_command_reply: tid=0 2024-09-12T03:13:02.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.183+0000 7fd11d7f6740 1 client.18072 shutdown 2024-09-12T03:13:02.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.183+0000 7fd11d7f6740 1 -- 172.21.15.116:0/3545442986 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d08bc6b370 msgr2=0x55d08bc68520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:02.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.183+0000 7fd11d7f6740 1 --2- 172.21.15.116:0/3545442986 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d08bc6b370 0x55d08bc68520 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fd10402af90 tx=0x7fd1040093f0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.183+0000 7fd11d7f6740 1 -- 172.21.15.116:0/3545442986 shutdown_connections 2024-09-12T03:13:02.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.183+0000 7fd11d7f6740 1 --2- 172.21.15.116:0/3545442986 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55d08bc7ae40 0x55d08bc7d2b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.183+0000 7fd11d7f6740 1 --2- 172.21.15.116:0/3545442986 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d08bc6b370 0x55d08bc68520 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.183+0000 7fd11d7f6740 1 --2- 172.21.15.116:0/3545442986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d08bc61b40 0x55d08bc67fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.183+0000 7fd11d7f6740 1 --2- 172.21.15.116:0/3545442986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d08bc605d0 0x55d08bc6acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.211 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:13:02.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.183+0000 7fd0f8ff9640 0 client.18072 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:13:02.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.183+0000 7fd11d7f6740 1 -- 172.21.15.116:0/3545442986 >> 172.21.15.116:0/3545442986 conn(0x55d08bc29310 msgr2=0x55d08bc62960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:02.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.183+0000 7fd11d7f6740 1 -- 172.21.15.116:0/3545442986 shutdown_connections 2024-09-12T03:13:02.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.183+0000 7fd11d7f6740 1 -- 172.21.15.116:0/3545442986 wait complete. 2024-09-12T03:13:02.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.183+0000 7fd11d7f6740 20 client.18072 trim_cache size 0 max 16384 2024-09-12T03:13:02.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.196+0000 7fd11d7f6740 1 Processor -- start 2024-09-12T03:13:02.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.196+0000 7fd11d7f6740 1 -- start start 2024-09-12T03:13:02.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.196+0000 7fd11d7f6740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d08bc605d0 0x55d08bc69f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:02.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.196+0000 7fd11d7f6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d08bc61b40 0x55d08bc63120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:02.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.196+0000 7fd11d7f6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d08bc6b370 0x55d08bc63660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:02.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.196+0000 7fd11d7f6740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd1141b2130 con 0x55d08bc605d0 2024-09-12T03:13:02.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.196+0000 7fd11d7f6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd1141afa00 con 0x55d08bc6b370 2024-09-12T03:13:02.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.196+0000 7fd11d7f6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd1141ad850 con 0x55d08bc61b40 2024-09-12T03:13:02.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.196+0000 7fd1191b9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d08bc605d0 0x55d08bc69f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:02.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.196+0000 7fd1191b9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d08bc605d0 0x55d08bc69f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34622/0 (socket says 172.21.15.116:34622) 2024-09-12T03:13:02.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.196+0000 7fd1191b9640 1 -- 172.21.15.116:0/1060501981 learned_addr learned my addr 172.21.15.116:0/1060501981 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:02.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.196+0000 7fd1189b8640 1 --2- 172.21.15.116:0/1060501981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d08bc61b40 0x55d08bc63120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:02.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.196+0000 7fd1191b9640 1 -- 172.21.15.116:0/1060501981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d08bc61b40 msgr2=0x55d08bc63120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:02.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.196+0000 7fd1191b9640 1 --2- 172.21.15.116:0/1060501981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d08bc61b40 0x55d08bc63120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.196+0000 7fd1191b9640 1 -- 172.21.15.116:0/1060501981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d08bc6b370 msgr2=0x55d08bc63660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:02.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.196+0000 7fd1191b9640 1 --2- 172.21.15.116:0/1060501981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d08bc6b370 0x55d08bc63660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.196+0000 7fd1191b9640 1 -- 172.21.15.116:0/1060501981 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d08bc6b060 con 0x55d08bc605d0 2024-09-12T03:13:02.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.196+0000 7fd1191b9640 1 --2- 172.21.15.116:0/1060501981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d08bc605d0 0x55d08bc69f10 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00a860 tx=0x7fd10c01c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd8719f0eb5f5a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:02.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.196+0000 7fd0faffd640 1 -- 172.21.15.116:0/1060501981 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c015040 con 0x55d08bc605d0 2024-09-12T03:13:02.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.196+0000 7fd0faffd640 1 -- 172.21.15.116:0/1060501981 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd10c0118d0 con 0x55d08bc605d0 2024-09-12T03:13:02.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.196+0000 7fd0faffd640 1 -- 172.21.15.116:0/1060501981 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10c01a9f0 con 0x55d08bc605d0 2024-09-12T03:13:02.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.197+0000 7fd11d7f6740 1 -- 172.21.15.116:0/1060501981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d08bc605d0 msgr2=0x55d08bc69f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:02.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.197+0000 7fd11d7f6740 1 --2- 172.21.15.116:0/1060501981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d08bc605d0 0x55d08bc69f10 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00a860 tx=0x7fd10c01c040 comp rx=0 tx=0).stop 2024-09-12T03:13:02.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.197+0000 7fd11d7f6740 1 -- 172.21.15.116:0/1060501981 shutdown_connections 2024-09-12T03:13:02.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.197+0000 7fd11d7f6740 1 --2- 172.21.15.116:0/1060501981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d08bc6b370 0x55d08bc63660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.197+0000 7fd11d7f6740 1 --2- 172.21.15.116:0/1060501981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d08bc61b40 0x55d08bc63120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.197+0000 7fd11d7f6740 1 --2- 172.21.15.116:0/1060501981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d08bc605d0 0x55d08bc69f10 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.197+0000 7fd11d7f6740 1 -- 172.21.15.116:0/1060501981 >> 172.21.15.116:0/1060501981 conn(0x55d08bc29310 msgr2=0x55d08bc69800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:02.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.197+0000 7fd11d7f6740 1 -- 172.21.15.116:0/1060501981 shutdown_connections 2024-09-12T03:13:02.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.197+0000 7fd11d7f6740 1 -- 172.21.15.116:0/1060501981 wait complete. 2024-09-12T03:13:02.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.197+0000 7fd11d7f6740 1 -- start start 2024-09-12T03:13:02.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.197+0000 7fd11d7f6740 1 Processor -- start 2024-09-12T03:13:02.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.198+0000 7fd11d7f6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d08bc605d0 0x55d08bc65140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:02.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.198+0000 7fd11d7f6740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d08bc61b40 0x55d08bc65680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:02.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.198+0000 7fd11d7f6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d08bc6b370 0x55d08bc65bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:02.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.198+0000 7fd11d7f6740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd1141adec0 con 0x55d08bc61b40 2024-09-12T03:13:02.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.198+0000 7fd11d7f6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd1141af330 con 0x55d08bc605d0 2024-09-12T03:13:02.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.198+0000 7fd11d7f6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd1141ae590 con 0x55d08bc6b370 2024-09-12T03:13:02.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.198+0000 7fd11d7f6740 10 client.0 _ll_register_callbacks cb 0x7fd104031a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:02.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.198+0000 7fd1189b8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d08bc61b40 0x55d08bc65680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:02.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.198+0000 7fd1189b8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d08bc61b40 0x55d08bc65680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34628/0 (socket says 172.21.15.116:34628) 2024-09-12T03:13:02.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.198+0000 7fd1189b8640 1 -- 172.21.15.116:0/1568344813 learned_addr learned my addr 172.21.15.116:0/1568344813 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:02.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.198+0000 7fd1199ba640 1 --2- 172.21.15.116:0/1568344813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d08bc6b370 0x55d08bc65bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:02.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.198+0000 7fd1189b8640 1 -- 172.21.15.116:0/1568344813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d08bc6b370 msgr2=0x55d08bc65bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:02.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.198+0000 7fd1189b8640 1 --2- 172.21.15.116:0/1568344813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d08bc6b370 0x55d08bc65bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.198+0000 7fd1189b8640 1 -- 172.21.15.116:0/1568344813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d08bc605d0 msgr2=0x55d08bc65140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:02.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.198+0000 7fd1191b9640 1 --2- 172.21.15.116:0/1568344813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d08bc605d0 0x55d08bc65140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:02.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.198+0000 7fd1189b8640 1 --2- 172.21.15.116:0/1568344813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d08bc605d0 0x55d08bc65140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.198+0000 7fd1189b8640 1 -- 172.21.15.116:0/1568344813 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d08bce4cf0 con 0x55d08bc61b40 2024-09-12T03:13:02.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.198+0000 7fd1189b8640 1 --2- 172.21.15.116:0/1568344813 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d08bc61b40 0x55d08bc65680 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fd1080cbf60 tx=0x7fd108017730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382bcc48a6332505 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:02.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.198+0000 7fd0f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:02.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.198+0000 7fd0f8ff9640 1 -- 172.21.15.116:0/1568344813 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1080b8060 con 0x55d08bc61b40 2024-09-12T03:13:02.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.199+0000 7fd0f8ff9640 1 -- 172.21.15.116:0/1568344813 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd108010ca0 con 0x55d08bc61b40 2024-09-12T03:13:02.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.199+0000 7fd11d7f6740 1 -- 172.21.15.116:0/1568344813 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd10c0118d0 con 0x55d08bc61b40 2024-09-12T03:13:02.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.199+0000 7fd0f8ff9640 1 -- 172.21.15.116:0/1568344813 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd10801e040 con 0x55d08bc61b40 2024-09-12T03:13:02.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.199+0000 7fd11d7f6740 10 client.18078 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:02.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.199+0000 7fd11d7f6740 1 -- 172.21.15.116:0/1568344813 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d08bce5fb0 con 0x55d08bc61b40 2024-09-12T03:13:02.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.199+0000 7fd0f8ff9640 1 -- 172.21.15.116:0/1568344813 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd10c0118d0 con 0x55d08bc61b40 2024-09-12T03:13:02.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.199+0000 7fd0f8ff9640 1 -- 172.21.15.116:0/1568344813 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd10809a0c0 con 0x55d08bc61b40 2024-09-12T03:13:02.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.199+0000 7fd11d7f6740 10 client.18078 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:02.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.199+0000 7fd11d7f6740 10 client.18078 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:13:02.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.199+0000 7fd11d7f6740 20 client.18078 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:02.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.199+0000 7fd11d7f6740 1 --2- 172.21.15.116:0/1568344813 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55d08bce6710 0x55d08bce8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:02.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.199+0000 7fd11d7f6740 4 client.18078 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:13:02.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.199+0000 7fd11d7f6740 1 -- 172.21.15.116:0/1568344813 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d08bbcdd70 con 0x55d08bce6710 2024-09-12T03:13:02.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.199+0000 7fd1191b9640 1 --2- 172.21.15.116:0/1568344813 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55d08bce6710 0x55d08bce8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:02.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.200+0000 7fd0f8ff9640 10 client.18078 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:13:02.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.200+0000 7fd1191b9640 1 --2- 172.21.15.116:0/1568344813 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55d08bce6710 0x55d08bce8b30 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8d6806169b31a298 server_cookie=64d3aaff75b1dc06 in_seq=0 out_seq=0 2024-09-12T03:13:02.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.200+0000 7fd0f8ff9640 1 -- 172.21.15.116:0/1568344813 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d08bbcdd70 con 0x55d08bce6710 2024-09-12T03:13:02.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.200+0000 7fd0f8ff9640 10 client.18078 handle_command_reply: tid=0 2024-09-12T03:13:02.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.200+0000 7fd11d7f6740 1 client.18078 shutdown 2024-09-12T03:13:02.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.200+0000 7fd11d7f6740 1 -- 172.21.15.116:0/1568344813 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d08bc61b40 msgr2=0x55d08bc65680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:02.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.200+0000 7fd11d7f6740 1 --2- 172.21.15.116:0/1568344813 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d08bc61b40 0x55d08bc65680 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fd1080cbf60 tx=0x7fd108017730 comp rx=0 tx=0).stop 2024-09-12T03:13:02.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.200+0000 7fd11d7f6740 1 -- 172.21.15.116:0/1568344813 shutdown_connections 2024-09-12T03:13:02.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.200+0000 7fd11d7f6740 1 --2- 172.21.15.116:0/1568344813 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55d08bce6710 0x55d08bce8b30 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.200+0000 7fd11d7f6740 1 --2- 172.21.15.116:0/1568344813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d08bc6b370 0x55d08bc65bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.200+0000 7fd11d7f6740 1 --2- 172.21.15.116:0/1568344813 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d08bc61b40 0x55d08bc65680 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.200+0000 7fd0f8ff9640 0 client.18078 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:13:02.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.200+0000 7fd11d7f6740 1 --2- 172.21.15.116:0/1568344813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d08bc605d0 0x55d08bc65140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.200+0000 7fd11d7f6740 1 -- 172.21.15.116:0/1568344813 >> 172.21.15.116:0/1568344813 conn(0x55d08bc29310 msgr2=0x55d08bc6a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:02.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.200+0000 7fd11d7f6740 1 -- 172.21.15.116:0/1568344813 shutdown_connections 2024-09-12T03:13:02.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.200+0000 7fd11d7f6740 1 -- 172.21.15.116:0/1568344813 wait complete. 2024-09-12T03:13:02.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.200+0000 7fd11d7f6740 20 client.18078 trim_cache size 0 max 16384 2024-09-12T03:13:02.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.201+0000 7fd11b41d640 1 -- 172.21.15.116:0/296318641 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd0fc0a2b80 msgr2=0x7fd0fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:02.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.201+0000 7fd11b41d640 1 --2- 172.21.15.116:0/296318641 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd0fc0a2b80 0x7fd0fc0a5040 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd10402ade0 tx=0x7fd104005ef0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.201+0000 7fd11b41d640 1 -- 172.21.15.116:0/296318641 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1141027a0 msgr2=0x7fd11419ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:02.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.201+0000 7fd11b41d640 1 --2- 172.21.15.116:0/296318641 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1141027a0 0x7fd11419ac80 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fd10800b4a0 tx=0x7fd10800b960 comp rx=0 tx=0).stop 2024-09-12T03:13:02.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.201+0000 7fd11b41d640 1 -- 172.21.15.116:0/296318641 shutdown_connections 2024-09-12T03:13:02.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.201+0000 7fd11b41d640 1 --2- 172.21.15.116:0/296318641 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd0fc0a2b80 0x7fd0fc0a5040 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.201+0000 7fd11b41d640 1 --2- 172.21.15.116:0/296318641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd114106f20 0x7fd11419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.201+0000 7fd11b41d640 1 --2- 172.21.15.116:0/296318641 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1141027a0 0x7fd11419ac80 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.201+0000 7fd11b41d640 1 --2- 172.21.15.116:0/296318641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd114101df0 0x7fd11419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:02.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.201+0000 7fd11b41d640 1 -- 172.21.15.116:0/296318641 >> 172.21.15.116:0/296318641 conn(0x7fd1140fd8f0 msgr2=0x7fd1140fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:02.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.201+0000 7fd11b41d640 1 -- 172.21.15.116:0/296318641 shutdown_connections 2024-09-12T03:13:02.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:02.201+0000 7fd11b41d640 1 -- 172.21.15.116:0/296318641 wait complete. 2024-09-12T03:13:03.092 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:13:03.345 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:13:03.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:03 smithi183 ceph-mon[35535]: pgmap v640: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-12T03:13:03.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:03 smithi177 ceph-mon[35305]: pgmap v640: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-12T03:13:03.777 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:03 smithi116 ceph-mon[31410]: pgmap v640: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-12T03:13:03.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.858+0000 7feaa72e9640 1 -- 172.21.15.116:0/115105472 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feaa0101df0 msgr2=0x7feaa01021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:03.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.858+0000 7feaa72e9640 1 --2- 172.21.15.116:0/115105472 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feaa0101df0 0x7feaa01021d0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fea9c009900 tx=0x7fea9c02a410 comp rx=0 tx=0).stop 2024-09-12T03:13:03.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.858+0000 7feaa72e9640 1 -- 172.21.15.116:0/115105472 shutdown_connections 2024-09-12T03:13:03.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.858+0000 7feaa72e9640 1 --2- 172.21.15.116:0/115105472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feaa0106f20 0x7feaa010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.858+0000 7feaa72e9640 1 --2- 172.21.15.116:0/115105472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feaa01027a0 0x7feaa01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.858+0000 7feaa72e9640 1 --2- 172.21.15.116:0/115105472 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feaa0101df0 0x7feaa01021d0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.858+0000 7feaa72e9640 1 -- 172.21.15.116:0/115105472 >> 172.21.15.116:0/115105472 conn(0x7feaa00fd8b0 msgr2=0x7feaa00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:03.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.858+0000 7feaa72e9640 1 -- 172.21.15.116:0/115105472 shutdown_connections 2024-09-12T03:13:03.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.858+0000 7feaa72e9640 1 -- 172.21.15.116:0/115105472 wait complete. 2024-09-12T03:13:03.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.859+0000 7feaa72e9640 1 Processor -- start 2024-09-12T03:13:03.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.859+0000 7feaa72e9640 1 -- start start 2024-09-12T03:13:03.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.859+0000 7feaa72e9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feaa0101df0 0x7feaa01983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:03.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.859+0000 7feaa72e9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feaa01027a0 0x7feaa0198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:03.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.859+0000 7feaa72e9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feaa0106f20 0x7feaa019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:03.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.859+0000 7feaa72e9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7feaa0111f80 con 0x7feaa0101df0 2024-09-12T03:13:03.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.859+0000 7feaa72e9640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7feaa0111e00 con 0x7feaa0106f20 2024-09-12T03:13:03.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.859+0000 7feaa72e9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feaa0112100 con 0x7feaa01027a0 2024-09-12T03:13:03.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.859+0000 7feaa5085640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feaa0101df0 0x7feaa01983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:03.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.859+0000 7feaa5085640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feaa0101df0 0x7feaa01983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34650/0 (socket says 172.21.15.116:34650) 2024-09-12T03:13:03.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.859+0000 7feaa5085640 1 -- 172.21.15.116:0/1076675403 learned_addr learned my addr 172.21.15.116:0/1076675403 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:03.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.860+0000 7feaa5886640 1 --2- 172.21.15.116:0/1076675403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feaa0106f20 0x7feaa019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:03.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.860+0000 7feaa5085640 1 -- 172.21.15.116:0/1076675403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feaa01027a0 msgr2=0x7feaa0198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:03.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.860+0000 7feaa5085640 1 --2- 172.21.15.116:0/1076675403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feaa01027a0 0x7feaa0198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.860+0000 7feaa5085640 1 -- 172.21.15.116:0/1076675403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feaa0106f20 msgr2=0x7feaa019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:03.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.860+0000 7feaa5085640 1 --2- 172.21.15.116:0/1076675403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feaa0106f20 0x7feaa019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.860+0000 7feaa5085640 1 -- 172.21.15.116:0/1076675403 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa019d370 con 0x7feaa0101df0 2024-09-12T03:13:03.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.860+0000 7feaa5886640 1 --2- 172.21.15.116:0/1076675403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feaa0106f20 0x7feaa019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:13:03.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.860+0000 7feaa5085640 1 --2- 172.21.15.116:0/1076675403 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feaa0101df0 0x7feaa01983c0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fea9c005ab0 tx=0x7fea9c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:03.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.860+0000 7fea967fc640 1 -- 172.21.15.116:0/1076675403 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c03f030 con 0x7feaa0101df0 2024-09-12T03:13:03.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.860+0000 7fea967fc640 1 -- 172.21.15.116:0/1076675403 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea9c003ce0 con 0x7feaa0101df0 2024-09-12T03:13:03.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.860+0000 7feaa72e9640 1 -- 172.21.15.116:0/1076675403 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaa019d600 con 0x7feaa0101df0 2024-09-12T03:13:03.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.860+0000 7fea967fc640 1 -- 172.21.15.116:0/1076675403 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c035070 con 0x7feaa0101df0 2024-09-12T03:13:03.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.860+0000 7feaa72e9640 1 -- 172.21.15.116:0/1076675403 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaa01a4f40 con 0x7feaa0101df0 2024-09-12T03:13:03.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.862+0000 7fea967fc640 1 -- 172.21.15.116:0/1076675403 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fea9c042030 con 0x7feaa0101df0 2024-09-12T03:13:03.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.863+0000 7feaa96c2740 1 Processor -- start 2024-09-12T03:13:03.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.863+0000 7feaa96c2740 1 -- start start 2024-09-12T03:13:03.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.863+0000 7feaa96c2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e5c1ceb40 0x558e5c1cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:03.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.863+0000 7feaa96c2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558e5c1cd5d0 0x558e5c1cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:03.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.863+0000 7feaa96c2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e5c1d8370 0x558e5c1da760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:03.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.863+0000 7feaa96c2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x558e5c1332f0 con 0x558e5c1cd5d0 2024-09-12T03:13:03.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.863+0000 7feaa96c2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558e5c0dacf0 con 0x558e5c1ceb40 2024-09-12T03:13:03.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.863+0000 7feaa96c2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558e5c12f2b0 con 0x558e5c1d8370 2024-09-12T03:13:03.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.863+0000 7feaa5085640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558e5c1cd5d0 0x558e5c1cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:03.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.863+0000 7feaa5085640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558e5c1cd5d0 0x558e5c1cda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34666/0 (socket says 172.21.15.116:34666) 2024-09-12T03:13:03.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.863+0000 7feaa5085640 1 -- 172.21.15.116:0/2705809569 learned_addr learned my addr 172.21.15.116:0/2705809569 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:03.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.863+0000 7fea967fc640 1 --2- 172.21.15.116:0/1076675403 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fea8c0a2b80 0x7fea8c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:03.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.863+0000 7feaa5886640 1 --2- 172.21.15.116:0/2705809569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e5c1ceb40 0x558e5c1cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:03.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.863+0000 7feaa4884640 1 --2- 172.21.15.116:0/2705809569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e5c1d8370 0x558e5c1da760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:03.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.863+0000 7feaa5886640 1 --2- 172.21.15.116:0/1076675403 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fea8c0a2b80 0x7fea8c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:03.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.864+0000 7fea967fc640 1 -- 172.21.15.116:0/1076675403 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fea9c0ef630 con 0x7feaa0101df0 2024-09-12T03:13:03.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.864+0000 7feaa5085640 1 -- 172.21.15.116:0/2705809569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e5c1d8370 msgr2=0x558e5c1da760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:03.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.864+0000 7feaa5085640 1 --2- 172.21.15.116:0/2705809569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e5c1d8370 0x558e5c1da760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.864+0000 7feaa5085640 1 -- 172.21.15.116:0/2705809569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e5c1ceb40 msgr2=0x558e5c1cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:03.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.864+0000 7feaa5085640 1 --2- 172.21.15.116:0/2705809569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e5c1ceb40 0x558e5c1cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.864+0000 7feaa5085640 1 -- 172.21.15.116:0/2705809569 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e5c0bf3a0 con 0x558e5c1cd5d0 2024-09-12T03:13:03.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.864+0000 7feaa5085640 1 --2- 172.21.15.116:0/2705809569 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558e5c1cd5d0 0x558e5c1cda10 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0bbbc0 tx=0x7fea9c0b7b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe466911084966c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:03.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.864+0000 7feaa5886640 1 --2- 172.21.15.116:0/1076675403 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fea8c0a2b80 0x7fea8c0a5040 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fea98005fd0 tx=0x7fea98005c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:03.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.864+0000 7fea8b7fe640 1 -- 172.21.15.116:0/2705809569 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c049070 con 0x558e5c1cd5d0 2024-09-12T03:13:03.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.864+0000 7fea8b7fe640 1 -- 172.21.15.116:0/2705809569 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea9c0bc5c0 con 0x558e5c1cd5d0 2024-09-12T03:13:03.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.864+0000 7fea8b7fe640 1 -- 172.21.15.116:0/2705809569 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c0ed270 con 0x558e5c1cd5d0 2024-09-12T03:13:03.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.865+0000 7feaa96c2740 1 -- 172.21.15.116:0/2705809569 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558e5c1cd5d0 msgr2=0x558e5c1cda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:03.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.865+0000 7feaa96c2740 1 --2- 172.21.15.116:0/2705809569 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558e5c1cd5d0 0x558e5c1cda10 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0bbbc0 tx=0x7fea9c0b7b60 comp rx=0 tx=0).stop 2024-09-12T03:13:03.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.865+0000 7feaa96c2740 1 -- 172.21.15.116:0/2705809569 shutdown_connections 2024-09-12T03:13:03.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.865+0000 7feaa96c2740 1 --2- 172.21.15.116:0/2705809569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e5c1d8370 0x558e5c1da760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.865+0000 7feaa96c2740 1 --2- 172.21.15.116:0/2705809569 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558e5c1cd5d0 0x558e5c1cda10 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.865+0000 7feaa96c2740 1 --2- 172.21.15.116:0/2705809569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e5c1ceb40 0x558e5c1cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.865+0000 7feaa96c2740 1 -- 172.21.15.116:0/2705809569 >> 172.21.15.116:0/2705809569 conn(0x558e5c196310 msgr2=0x558e5c1c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:03.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.865+0000 7feaa96c2740 1 -- 172.21.15.116:0/2705809569 shutdown_connections 2024-09-12T03:13:03.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.865+0000 7feaa96c2740 1 -- 172.21.15.116:0/2705809569 wait complete. 2024-09-12T03:13:03.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.865+0000 7feaa96c2740 1 -- start start 2024-09-12T03:13:03.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.865+0000 7feaa96c2740 1 Processor -- start 2024-09-12T03:13:03.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.866+0000 7feaa96c2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e5c1cd5d0 0x558e5c1d7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:03.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.866+0000 7feaa96c2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e5c1ceb40 0x558e5c1d5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:03.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.866+0000 7feaa96c2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558e5c1d8370 0x558e5c1d5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:03.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.866+0000 7feaa96c2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x558e5c1332f0 con 0x558e5c1d8370 2024-09-12T03:13:03.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.866+0000 7feaa96c2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558e5c0ee060 con 0x558e5c1cd5d0 2024-09-12T03:13:03.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.866+0000 7feaa96c2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558e5c1dae30 con 0x558e5c1ceb40 2024-09-12T03:13:03.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.866+0000 7feaa96c2740 10 client.0 _ll_register_callbacks cb 0x558e5be418c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:03.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.866+0000 7feaa5886640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558e5c1d8370 0x558e5c1d5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:03.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.866+0000 7feaa5886640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558e5c1d8370 0x558e5c1d5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34670/0 (socket says 172.21.15.116:34670) 2024-09-12T03:13:03.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.866+0000 7feaa5886640 1 -- 172.21.15.116:0/103151618 learned_addr learned my addr 172.21.15.116:0/103151618 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:03.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.866+0000 7feaa4884640 1 --2- 172.21.15.116:0/103151618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e5c1ceb40 0x558e5c1d5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:03.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.866+0000 7feaa5085640 1 --2- 172.21.15.116:0/103151618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e5c1cd5d0 0x558e5c1d7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:03.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.866+0000 7feaa5886640 1 -- 172.21.15.116:0/103151618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e5c1ceb40 msgr2=0x558e5c1d5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:03.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.866+0000 7feaa5886640 1 --2- 172.21.15.116:0/103151618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e5c1ceb40 0x558e5c1d5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.866+0000 7feaa5886640 1 -- 172.21.15.116:0/103151618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e5c1cd5d0 msgr2=0x558e5c1d7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:03.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.866+0000 7feaa5886640 1 --2- 172.21.15.116:0/103151618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e5c1cd5d0 0x558e5c1d7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.866+0000 7feaa5886640 1 -- 172.21.15.116:0/103151618 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e5bea43c0 con 0x558e5c1d8370 2024-09-12T03:13:03.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.866+0000 7feaa5085640 1 --2- 172.21.15.116:0/103151618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e5c1cd5d0 0x558e5c1d7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:13:03.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.866+0000 7feaa4884640 1 --2- 172.21.15.116:0/103151618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e5c1ceb40 0x558e5c1d5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:13:03.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.866+0000 7feaa5886640 1 --2- 172.21.15.116:0/103151618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558e5c1d8370 0x558e5c1d5640 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fea9800fb80 tx=0x7fea98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73088fc4a239796 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:03.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.866+0000 7fea88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:03.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.866+0000 7fea88ff9640 1 -- 172.21.15.116:0/103151618 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9800de50 con 0x558e5c1d8370 2024-09-12T03:13:03.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.866+0000 7feaa96c2740 1 -- 172.21.15.116:0/103151618 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea9c0bc5c0 con 0x558e5c1d8370 2024-09-12T03:13:03.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.866+0000 7fea88ff9640 1 -- 172.21.15.116:0/103151618 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fea9801e450 con 0x558e5c1d8370 2024-09-12T03:13:03.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.867+0000 7fea88ff9640 1 -- 172.21.15.116:0/103151618 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fea9802f070 con 0x558e5c1d8370 2024-09-12T03:13:03.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.867+0000 7feaa96c2740 10 client.18090 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:03.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.867+0000 7feaa96c2740 1 -- 172.21.15.116:0/103151618 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558e5c1e67e0 con 0x558e5c1d8370 2024-09-12T03:13:03.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.867+0000 7fea88ff9640 1 -- 172.21.15.116:0/103151618 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea9c0bc5c0 con 0x558e5c1d8370 2024-09-12T03:13:03.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.867+0000 7fea88ff9640 1 -- 172.21.15.116:0/103151618 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fea9801b040 con 0x558e5c1d8370 2024-09-12T03:13:03.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.867+0000 7feaa96c2740 10 client.18090 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:03.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.867+0000 7feaa96c2740 10 client.18090 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:13:03.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.867+0000 7feaa96c2740 20 client.18090 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:03.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.867+0000 7feaa96c2740 1 --2- 172.21.15.116:0/103151618 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x558e5c1e7d40 0x558e5c1ea190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:03.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.867+0000 7feaa96c2740 4 client.18090 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:13:03.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.867+0000 7feaa96c2740 1 -- 172.21.15.116:0/103151618 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e5c140860 con 0x558e5c1e7d40 2024-09-12T03:13:03.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.867+0000 7feaa5085640 1 --2- 172.21.15.116:0/103151618 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x558e5c1e7d40 0x558e5c1ea190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:03.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.868+0000 7fea88ff9640 10 client.18090 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:03.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.868+0000 7feaa5085640 1 --2- 172.21.15.116:0/103151618 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x558e5c1e7d40 0x558e5c1ea190 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2169b50740318e98 server_cookie=e41e8d9e135085cd in_seq=0 out_seq=0 2024-09-12T03:13:03.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.868+0000 7fea88ff9640 1 -- 172.21.15.116:0/103151618 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e5c140860 con 0x558e5c1e7d40 2024-09-12T03:13:03.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.868+0000 7fea88ff9640 10 client.18090 handle_command_reply: tid=0 2024-09-12T03:13:03.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.868+0000 7feaa96c2740 1 client.18090 shutdown 2024-09-12T03:13:03.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.869+0000 7feaa96c2740 1 -- 172.21.15.116:0/103151618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558e5c1d8370 msgr2=0x558e5c1d5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:03.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.869+0000 7feaa96c2740 1 --2- 172.21.15.116:0/103151618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558e5c1d8370 0x558e5c1d5640 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fea9800fb80 tx=0x7fea98004a40 comp rx=0 tx=0).stop 2024-09-12T03:13:03.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.869+0000 7feaa96c2740 1 -- 172.21.15.116:0/103151618 shutdown_connections 2024-09-12T03:13:03.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.869+0000 7feaa96c2740 1 --2- 172.21.15.116:0/103151618 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x558e5c1e7d40 0x558e5c1ea190 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.869+0000 7feaa96c2740 1 --2- 172.21.15.116:0/103151618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558e5c1d8370 0x558e5c1d5640 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.869+0000 7feaa96c2740 1 --2- 172.21.15.116:0/103151618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e5c1ceb40 0x558e5c1d5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.869+0000 7fea88ff9640 0 client.18090 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:03.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.869+0000 7feaa96c2740 1 --2- 172.21.15.116:0/103151618 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e5c1cd5d0 0x558e5c1d7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.869+0000 7feaa96c2740 1 -- 172.21.15.116:0/103151618 >> 172.21.15.116:0/103151618 conn(0x558e5c196310 msgr2=0x558e5c1cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:03.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.869+0000 7feaa96c2740 1 -- 172.21.15.116:0/103151618 shutdown_connections 2024-09-12T03:13:03.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.869+0000 7feaa96c2740 1 -- 172.21.15.116:0/103151618 wait complete. 2024-09-12T03:13:03.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.869+0000 7feaa96c2740 20 client.18090 trim_cache size 0 max 16384 2024-09-12T03:13:03.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.882+0000 7feaa96c2740 1 Processor -- start 2024-09-12T03:13:03.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.882+0000 7feaa96c2740 1 -- start start 2024-09-12T03:13:03.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.882+0000 7feaa96c2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558e5c1cd5d0 0x558e5c1e3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:03.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.882+0000 7feaa96c2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e5c1ceb40 0x558e5c1d0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:03.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.882+0000 7feaa96c2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e5c1d8370 0x558e5c1d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:03.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.882+0000 7feaa96c2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7feaa01afce0 con 0x558e5c1cd5d0 2024-09-12T03:13:03.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.882+0000 7feaa96c2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7feaa01ad5b0 con 0x558e5c1d8370 2024-09-12T03:13:03.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.882+0000 7feaa96c2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feaa01ab420 con 0x558e5c1ceb40 2024-09-12T03:13:03.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.882+0000 7feaa5085640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558e5c1cd5d0 0x558e5c1e3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:03.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.882+0000 7feaa5085640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558e5c1cd5d0 0x558e5c1e3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34676/0 (socket says 172.21.15.116:34676) 2024-09-12T03:13:03.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.882+0000 7feaa5085640 1 -- 172.21.15.116:0/3813506582 learned_addr learned my addr 172.21.15.116:0/3813506582 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:03.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.883+0000 7feaa5886640 1 --2- 172.21.15.116:0/3813506582 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e5c1d8370 0x558e5c1d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:03.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.883+0000 7feaa4884640 1 --2- 172.21.15.116:0/3813506582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e5c1ceb40 0x558e5c1d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:03.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.883+0000 7feaa5085640 1 -- 172.21.15.116:0/3813506582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e5c1ceb40 msgr2=0x558e5c1d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:03.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.883+0000 7feaa5085640 1 --2- 172.21.15.116:0/3813506582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e5c1ceb40 0x558e5c1d0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.883+0000 7feaa5085640 1 -- 172.21.15.116:0/3813506582 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e5c1d8370 msgr2=0x558e5c1d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:03.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.883+0000 7feaa5085640 1 --2- 172.21.15.116:0/3813506582 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e5c1d8370 0x558e5c1d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.883+0000 7feaa5085640 1 -- 172.21.15.116:0/3813506582 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e5c1d3580 con 0x558e5c1cd5d0 2024-09-12T03:13:03.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.883+0000 7feaa5085640 1 --2- 172.21.15.116:0/3813506582 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558e5c1cd5d0 0x558e5c1e3f50 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0ed1b0 tx=0x7fea9c003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843f27e60bae8dae server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:03.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.883+0000 7fea8affd640 1 -- 172.21.15.116:0/3813506582 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c049040 con 0x558e5c1cd5d0 2024-09-12T03:13:03.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.883+0000 7fea8affd640 1 -- 172.21.15.116:0/3813506582 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea9c0bc5c0 con 0x558e5c1cd5d0 2024-09-12T03:13:03.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.883+0000 7fea8affd640 1 -- 172.21.15.116:0/3813506582 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c0ec230 con 0x558e5c1cd5d0 2024-09-12T03:13:03.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.884+0000 7feaa96c2740 1 -- 172.21.15.116:0/3813506582 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558e5c1cd5d0 msgr2=0x558e5c1e3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:03.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.884+0000 7feaa96c2740 1 --2- 172.21.15.116:0/3813506582 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558e5c1cd5d0 0x558e5c1e3f50 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0ed1b0 tx=0x7fea9c003f90 comp rx=0 tx=0).stop 2024-09-12T03:13:03.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.884+0000 7feaa96c2740 1 -- 172.21.15.116:0/3813506582 shutdown_connections 2024-09-12T03:13:03.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.884+0000 7feaa96c2740 1 --2- 172.21.15.116:0/3813506582 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e5c1d8370 0x558e5c1d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.884+0000 7feaa96c2740 1 --2- 172.21.15.116:0/3813506582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e5c1ceb40 0x558e5c1d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.884+0000 7feaa96c2740 1 --2- 172.21.15.116:0/3813506582 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558e5c1cd5d0 0x558e5c1e3f50 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.884+0000 7feaa96c2740 1 -- 172.21.15.116:0/3813506582 >> 172.21.15.116:0/3813506582 conn(0x558e5c196310 msgr2=0x558e5c1d4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:03.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.884+0000 7feaa96c2740 1 -- 172.21.15.116:0/3813506582 shutdown_connections 2024-09-12T03:13:03.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.884+0000 7feaa96c2740 1 -- 172.21.15.116:0/3813506582 wait complete. 2024-09-12T03:13:03.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.884+0000 7feaa96c2740 1 -- start start 2024-09-12T03:13:03.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.884+0000 7feaa96c2740 1 Processor -- start 2024-09-12T03:13:03.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.885+0000 7feaa96c2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e5c1cd5d0 0x558e5c1e07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:03.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.885+0000 7feaa96c2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558e5c1ceb40 0x558e5c1dbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:03.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.885+0000 7feaa96c2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e5c1d8370 0x558e5c1dc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:03.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.885+0000 7feaa96c2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7feaa01aba70 con 0x558e5c1ceb40 2024-09-12T03:13:03.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.885+0000 7feaa96c2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7feaa01acee0 con 0x558e5c1cd5d0 2024-09-12T03:13:03.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.885+0000 7feaa96c2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feaa01ac140 con 0x558e5c1d8370 2024-09-12T03:13:03.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.885+0000 7feaa96c2740 10 client.0 _ll_register_callbacks cb 0x7fea9800f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:03.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.885+0000 7feaa4884640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558e5c1ceb40 0x558e5c1dbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:03.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.885+0000 7feaa4884640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558e5c1ceb40 0x558e5c1dbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34680/0 (socket says 172.21.15.116:34680) 2024-09-12T03:13:03.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.885+0000 7feaa4884640 1 -- 172.21.15.116:0/538825278 learned_addr learned my addr 172.21.15.116:0/538825278 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:03.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.885+0000 7feaa5886640 1 --2- 172.21.15.116:0/538825278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e5c1d8370 0x558e5c1dc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:03.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.885+0000 7feaa5085640 1 --2- 172.21.15.116:0/538825278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e5c1cd5d0 0x558e5c1e07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:03.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.885+0000 7feaa4884640 1 -- 172.21.15.116:0/538825278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e5c1d8370 msgr2=0x558e5c1dc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:03.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.885+0000 7feaa4884640 1 --2- 172.21.15.116:0/538825278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e5c1d8370 0x558e5c1dc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.885+0000 7feaa4884640 1 -- 172.21.15.116:0/538825278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e5c1cd5d0 msgr2=0x558e5c1e07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:03.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.885+0000 7feaa4884640 1 --2- 172.21.15.116:0/538825278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e5c1cd5d0 0x558e5c1e07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.885+0000 7feaa4884640 1 -- 172.21.15.116:0/538825278 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e5c1dc800 con 0x558e5c1ceb40 2024-09-12T03:13:03.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.885+0000 7feaa4884640 1 --2- 172.21.15.116:0/538825278 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558e5c1ceb40 0x558e5c1dbae0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fea90005fd0 tx=0x7fea90011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f86731a89c416524 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:03.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.885+0000 7fea88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:03.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.885+0000 7fea88ff9640 1 -- 172.21.15.116:0/538825278 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90004220 con 0x558e5c1ceb40 2024-09-12T03:13:03.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.885+0000 7fea88ff9640 1 -- 172.21.15.116:0/538825278 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fea900043c0 con 0x558e5c1ceb40 2024-09-12T03:13:03.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.885+0000 7feaa96c2740 1 -- 172.21.15.116:0/538825278 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea9c0bc5c0 con 0x558e5c1ceb40 2024-09-12T03:13:03.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.886+0000 7fea88ff9640 1 -- 172.21.15.116:0/538825278 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fea90020030 con 0x558e5c1ceb40 2024-09-12T03:13:03.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.886+0000 7feaa96c2740 10 client.18096 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:03.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.886+0000 7feaa96c2740 1 -- 172.21.15.116:0/538825278 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558e5c252a80 con 0x558e5c1ceb40 2024-09-12T03:13:03.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.886+0000 7fea88ff9640 1 -- 172.21.15.116:0/538825278 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea9c0bc5c0 con 0x558e5c1ceb40 2024-09-12T03:13:03.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.886+0000 7fea88ff9640 1 -- 172.21.15.116:0/538825278 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fea900047a0 con 0x558e5c1ceb40 2024-09-12T03:13:03.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.886+0000 7feaa96c2740 10 client.18096 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:03.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.886+0000 7feaa96c2740 10 client.18096 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:13:03.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.886+0000 7feaa96c2740 20 client.18096 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:03.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.886+0000 7feaa96c2740 1 --2- 172.21.15.116:0/538825278 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x558e5c253280 0x558e5c2556a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:03.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.886+0000 7feaa96c2740 4 client.18096 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:13:03.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.886+0000 7feaa96c2740 1 -- 172.21.15.116:0/538825278 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558e5c13ad70 con 0x558e5c253280 2024-09-12T03:13:03.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.886+0000 7feaa5085640 1 --2- 172.21.15.116:0/538825278 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x558e5c253280 0x558e5c2556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:03.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.887+0000 7feaa5085640 1 --2- 172.21.15.116:0/538825278 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x558e5c253280 0x558e5c2556a0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=443159baf72fed9a server_cookie=e14eab86e287219f in_seq=0 out_seq=0 2024-09-12T03:13:03.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.887+0000 7fea88ff9640 10 client.18096 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:03.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.887+0000 7fea88ff9640 1 -- 172.21.15.116:0/538825278 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558e5c13ad70 con 0x558e5c253280 2024-09-12T03:13:03.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.887+0000 7fea88ff9640 10 client.18096 handle_command_reply: tid=0 2024-09-12T03:13:03.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.887+0000 7feaa96c2740 1 client.18096 shutdown 2024-09-12T03:13:03.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.887+0000 7feaa96c2740 1 -- 172.21.15.116:0/538825278 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558e5c1ceb40 msgr2=0x558e5c1dbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:03.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.887+0000 7feaa96c2740 1 --2- 172.21.15.116:0/538825278 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558e5c1ceb40 0x558e5c1dbae0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fea90005fd0 tx=0x7fea90011b70 comp rx=0 tx=0).stop 2024-09-12T03:13:03.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.887+0000 7feaa96c2740 1 -- 172.21.15.116:0/538825278 shutdown_connections 2024-09-12T03:13:03.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.887+0000 7feaa96c2740 1 --2- 172.21.15.116:0/538825278 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x558e5c253280 0x558e5c2556a0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.887+0000 7feaa96c2740 1 --2- 172.21.15.116:0/538825278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e5c1d8370 0x558e5c1dc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.887+0000 7feaa96c2740 1 --2- 172.21.15.116:0/538825278 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558e5c1ceb40 0x558e5c1dbae0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.887+0000 7feaa96c2740 1 --2- 172.21.15.116:0/538825278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e5c1cd5d0 0x558e5c1e07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.887+0000 7feaa96c2740 1 -- 172.21.15.116:0/538825278 >> 172.21.15.116:0/538825278 conn(0x558e5c196310 msgr2=0x558e5c1c84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:03.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.887+0000 7fea88ff9640 0 client.18096 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:03.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.887+0000 7feaa96c2740 1 -- 172.21.15.116:0/538825278 shutdown_connections 2024-09-12T03:13:03.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.887+0000 7feaa96c2740 1 -- 172.21.15.116:0/538825278 wait complete. 2024-09-12T03:13:03.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.887+0000 7feaa96c2740 20 client.18096 trim_cache size 0 max 16384 2024-09-12T03:13:03.907 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:13:03.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.888+0000 7feaa72e9640 1 -- 172.21.15.116:0/1076675403 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fea8c0a2b80 msgr2=0x7fea8c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:03.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.888+0000 7feaa72e9640 1 --2- 172.21.15.116:0/1076675403 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fea8c0a2b80 0x7fea8c0a5040 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fea98005fd0 tx=0x7fea98005c80 comp rx=0 tx=0).stop 2024-09-12T03:13:03.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.888+0000 7feaa72e9640 1 -- 172.21.15.116:0/1076675403 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feaa0101df0 msgr2=0x7feaa01983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:03.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.888+0000 7feaa72e9640 1 --2- 172.21.15.116:0/1076675403 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feaa0101df0 0x7feaa01983c0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fea9c005ab0 tx=0x7fea9c0042f0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.888+0000 7feaa72e9640 1 -- 172.21.15.116:0/1076675403 shutdown_connections 2024-09-12T03:13:03.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.888+0000 7feaa72e9640 1 --2- 172.21.15.116:0/1076675403 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fea8c0a2b80 0x7fea8c0a5040 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.888+0000 7feaa72e9640 1 --2- 172.21.15.116:0/1076675403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feaa0106f20 0x7feaa019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.888+0000 7feaa72e9640 1 --2- 172.21.15.116:0/1076675403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feaa01027a0 0x7feaa0198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.908 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.888+0000 7feaa72e9640 1 --2- 172.21.15.116:0/1076675403 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feaa0101df0 0x7feaa01983c0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:03.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.888+0000 7feaa72e9640 1 -- 172.21.15.116:0/1076675403 >> 172.21.15.116:0/1076675403 conn(0x7feaa00fd8b0 msgr2=0x7feaa00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:03.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.888+0000 7feaa72e9640 1 -- 172.21.15.116:0/1076675403 shutdown_connections 2024-09-12T03:13:03.909 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:03.888+0000 7feaa72e9640 1 -- 172.21.15.116:0/1076675403 wait complete. 2024-09-12T03:13:04.964 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:13:05.214 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:13:05.632 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:05 smithi116 ceph-mon[31410]: pgmap v641: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T03:13:05.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:05 smithi183 ceph-mon[35535]: pgmap v641: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T03:13:05.725 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:13:05.725 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:13:05.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.711+0000 7f2f66feb640 1 -- 172.21.15.116:0/3402920344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f60101de0 msgr2=0x7f2f601021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:05.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.711+0000 7f2f66feb640 1 --2- 172.21.15.116:0/3402920344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f60101de0 0x7f2f601021c0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f2f54009900 tx=0x7f2f5402a410 comp rx=0 tx=0).stop 2024-09-12T03:13:05.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.711+0000 7f2f66feb640 1 -- 172.21.15.116:0/3402920344 shutdown_connections 2024-09-12T03:13:05.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.711+0000 7f2f66feb640 1 --2- 172.21.15.116:0/3402920344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f60106f10 0x7f2f6010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:05.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.711+0000 7f2f66feb640 1 --2- 172.21.15.116:0/3402920344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f60102790 0x7f2f601067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:05.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.711+0000 7f2f66feb640 1 --2- 172.21.15.116:0/3402920344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f60101de0 0x7f2f601021c0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:05.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.711+0000 7f2f66feb640 1 -- 172.21.15.116:0/3402920344 >> 172.21.15.116:0/3402920344 conn(0x7f2f600fd8c0 msgr2=0x7f2f600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:05.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.711+0000 7f2f66feb640 1 -- 172.21.15.116:0/3402920344 shutdown_connections 2024-09-12T03:13:05.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.711+0000 7f2f66feb640 1 -- 172.21.15.116:0/3402920344 wait complete. 2024-09-12T03:13:05.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.712+0000 7f2f66feb640 1 Processor -- start 2024-09-12T03:13:05.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.712+0000 7f2f66feb640 1 -- start start 2024-09-12T03:13:05.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.712+0000 7f2f66feb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f60101de0 0x7f2f60198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:05.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.712+0000 7f2f66feb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f60102790 0x7f2f601989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:05.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.712+0000 7f2f66feb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f60106f10 0x7f2f6019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:05.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.712+0000 7f2f66feb640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2f60111f70 con 0x7f2f60101de0 2024-09-12T03:13:05.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.712+0000 7f2f66feb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2f60111df0 con 0x7f2f60106f10 2024-09-12T03:13:05.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.712+0000 7f2f66feb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2f601120f0 con 0x7f2f60102790 2024-09-12T03:13:05.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.712+0000 7f2f64d87640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f60101de0 0x7f2f60198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:05.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.712+0000 7f2f64d87640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f60101de0 0x7f2f60198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34700/0 (socket says 172.21.15.116:34700) 2024-09-12T03:13:05.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.712+0000 7f2f64d87640 1 -- 172.21.15.116:0/3431836655 learned_addr learned my addr 172.21.15.116:0/3431836655 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:05.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.712+0000 7f2f5ffff640 1 --2- 172.21.15.116:0/3431836655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f60102790 0x7f2f601989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:05.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.712+0000 7f2f65588640 1 --2- 172.21.15.116:0/3431836655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f60106f10 0x7f2f6019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:05.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.712+0000 7f2f64d87640 1 -- 172.21.15.116:0/3431836655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f60102790 msgr2=0x7f2f601989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:05.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.712+0000 7f2f64d87640 1 --2- 172.21.15.116:0/3431836655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f60102790 0x7f2f601989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:05.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.712+0000 7f2f64d87640 1 -- 172.21.15.116:0/3431836655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f60106f10 msgr2=0x7f2f6019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:05.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.712+0000 7f2f64d87640 1 --2- 172.21.15.116:0/3431836655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f60106f10 0x7f2f6019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:05.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.712+0000 7f2f64d87640 1 -- 172.21.15.116:0/3431836655 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6019d440 con 0x7f2f60101de0 2024-09-12T03:13:05.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.712+0000 7f2f65588640 1 --2- 172.21.15.116:0/3431836655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f60106f10 0x7f2f6019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:13:05.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.712+0000 7f2f64d87640 1 --2- 172.21.15.116:0/3431836655 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f60101de0 0x7f2f60198490 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f2f54006010 tx=0x7f2f540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:05.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.713+0000 7f2f5dffb640 1 -- 172.21.15.116:0/3431836655 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5403f030 con 0x7f2f60101de0 2024-09-12T03:13:05.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.713+0000 7f2f66feb640 1 -- 172.21.15.116:0/3431836655 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f6019d6d0 con 0x7f2f60101de0 2024-09-12T03:13:05.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.713+0000 7f2f5dffb640 1 -- 172.21.15.116:0/3431836655 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f54003ce0 con 0x7f2f60101de0 2024-09-12T03:13:05.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.713+0000 7f2f5dffb640 1 -- 172.21.15.116:0/3431836655 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f54035070 con 0x7f2f60101de0 2024-09-12T03:13:05.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.713+0000 7f2f66feb640 1 -- 172.21.15.116:0/3431836655 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f601a5010 con 0x7f2f60101de0 2024-09-12T03:13:05.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.714+0000 7f2f66feb640 1 -- 172.21.15.116:0/3431836655 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f601032c0 con 0x7f2f60101de0 2024-09-12T03:13:05.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.715+0000 7f2f5dffb640 1 -- 172.21.15.116:0/3431836655 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2f54042030 con 0x7f2f60101de0 2024-09-12T03:13:05.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.715+0000 7f2f5dffb640 1 --2- 172.21.15.116:0/3431836655 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2f500a2b80 0x7f2f500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:05.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.716+0000 7f2f5ffff640 1 --2- 172.21.15.116:0/3431836655 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2f500a2b80 0x7f2f500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:05.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.716+0000 7f2f5dffb640 1 -- 172.21.15.116:0/3431836655 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f2f540ef7c0 con 0x7f2f60101de0 2024-09-12T03:13:05.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.716+0000 7f2f5ffff640 1 --2- 172.21.15.116:0/3431836655 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2f500a2b80 0x7f2f500a5040 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2f600fdd00 tx=0x7f2f4c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:05.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.719+0000 7f2f5dffb640 1 -- 172.21.15.116:0/3431836655 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2f540bc810 con 0x7f2f60101de0 2024-09-12T03:13:05.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:05 smithi177 ceph-mon[35305]: pgmap v641: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T03:13:05.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.924+0000 7f2f66feb640 1 -- 172.21.15.116:0/3431836655 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f601021c0 con 0x7f2f60101de0 2024-09-12T03:13:05.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.925+0000 7f2f5dffb640 1 -- 172.21.15.116:0/3431836655 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f2f540bc110 con 0x7f2f60101de0 2024-09-12T03:13:05.925 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:13:05.925 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:13:05.926 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:13:05.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.927+0000 7f2f66feb640 1 -- 172.21.15.116:0/3431836655 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2f500a2b80 msgr2=0x7f2f500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:05.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.927+0000 7f2f66feb640 1 --2- 172.21.15.116:0/3431836655 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2f500a2b80 0x7f2f500a5040 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2f600fdd00 tx=0x7f2f4c00b390 comp rx=0 tx=0).stop 2024-09-12T03:13:05.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.927+0000 7f2f66feb640 1 -- 172.21.15.116:0/3431836655 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f60101de0 msgr2=0x7f2f60198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:05.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.927+0000 7f2f66feb640 1 --2- 172.21.15.116:0/3431836655 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f60101de0 0x7f2f60198490 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f2f54006010 tx=0x7f2f540042f0 comp rx=0 tx=0).stop 2024-09-12T03:13:05.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.927+0000 7f2f66feb640 1 -- 172.21.15.116:0/3431836655 shutdown_connections 2024-09-12T03:13:05.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.927+0000 7f2f66feb640 1 --2- 172.21.15.116:0/3431836655 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2f500a2b80 0x7f2f500a5040 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:05.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.927+0000 7f2f66feb640 1 --2- 172.21.15.116:0/3431836655 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f60106f10 0x7f2f6019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:05.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.927+0000 7f2f66feb640 1 --2- 172.21.15.116:0/3431836655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f60102790 0x7f2f601989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:05.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.927+0000 7f2f66feb640 1 --2- 172.21.15.116:0/3431836655 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f60101de0 0x7f2f60198490 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:05.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.927+0000 7f2f66feb640 1 -- 172.21.15.116:0/3431836655 >> 172.21.15.116:0/3431836655 conn(0x7f2f600fd8c0 msgr2=0x7f2f600febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:05.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.927+0000 7f2f66feb640 1 -- 172.21.15.116:0/3431836655 shutdown_connections 2024-09-12T03:13:05.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:05.927+0000 7f2f66feb640 1 -- 172.21.15.116:0/3431836655 wait complete. 2024-09-12T03:13:06.629 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:06 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3431836655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:13:06.635 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:13:06.665 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id 6c5db9e6 --release --await' 2024-09-12T03:13:06.665 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id 6c5db9e6 --release --await 2024-09-12T03:13:06.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:06 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3431836655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:13:06.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:06 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3431836655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:13:07.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.140+0000 7fb011c8e640 1 -- 172.21.15.116:0/1491563951 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb00c101de0 msgr2=0x7fb00c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:07.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.140+0000 7fb011c8e640 1 --2- 172.21.15.116:0/1491563951 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb00c101de0 0x7fb00c1021c0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7faffc009900 tx=0x7faffc02a410 comp rx=0 tx=0).stop 2024-09-12T03:13:07.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.140+0000 7fb011c8e640 1 -- 172.21.15.116:0/1491563951 shutdown_connections 2024-09-12T03:13:07.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.140+0000 7fb011c8e640 1 --2- 172.21.15.116:0/1491563951 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb00c106f10 0x7fb00c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.140+0000 7fb011c8e640 1 --2- 172.21.15.116:0/1491563951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb00c102790 0x7fb00c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.140+0000 7fb011c8e640 1 --2- 172.21.15.116:0/1491563951 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb00c101de0 0x7fb00c1021c0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.140+0000 7fb011c8e640 1 -- 172.21.15.116:0/1491563951 >> 172.21.15.116:0/1491563951 conn(0x7fb00c0fd8c0 msgr2=0x7fb00c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:07.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.140+0000 7fb011c8e640 1 -- 172.21.15.116:0/1491563951 shutdown_connections 2024-09-12T03:13:07.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.140+0000 7fb011c8e640 1 -- 172.21.15.116:0/1491563951 wait complete. 2024-09-12T03:13:07.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.141+0000 7fb011c8e640 1 Processor -- start 2024-09-12T03:13:07.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.141+0000 7fb011c8e640 1 -- start start 2024-09-12T03:13:07.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.141+0000 7fb011c8e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb00c101de0 0x7fb00c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:07.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.141+0000 7fb011c8e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb00c102790 0x7fb00c198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:07.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.141+0000 7fb011c8e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb00c106f10 0x7fb00c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:07.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.141+0000 7fb011c8e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c111f70 con 0x7fb00c102790 2024-09-12T03:13:07.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.141+0000 7fb011c8e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c111df0 con 0x7fb00c101de0 2024-09-12T03:13:07.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.141+0000 7fb011c8e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c1120f0 con 0x7fb00c106f10 2024-09-12T03:13:07.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.142+0000 7fb003fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb00c102790 0x7fb00c198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:07.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.142+0000 7fb003fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb00c102790 0x7fb00c198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34716/0 (socket says 172.21.15.116:34716) 2024-09-12T03:13:07.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.142+0000 7fb003fff640 1 -- 172.21.15.116:0/410979504 learned_addr learned my addr 172.21.15.116:0/410979504 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:07.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.142+0000 7fb00b7fe640 1 --2- 172.21.15.116:0/410979504 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb00c101de0 0x7fb00c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:07.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.142+0000 7fb00bfff640 1 --2- 172.21.15.116:0/410979504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb00c106f10 0x7fb00c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:07.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.142+0000 7fb003fff640 1 -- 172.21.15.116:0/410979504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb00c106f10 msgr2=0x7fb00c19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:07.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.142+0000 7fb003fff640 1 --2- 172.21.15.116:0/410979504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb00c106f10 0x7fb00c19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.142+0000 7fb003fff640 1 -- 172.21.15.116:0/410979504 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb00c101de0 msgr2=0x7fb00c1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:07.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.142+0000 7fb003fff640 1 --2- 172.21.15.116:0/410979504 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb00c101de0 0x7fb00c1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.142+0000 7fb003fff640 1 -- 172.21.15.116:0/410979504 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00c19d480 con 0x7fb00c102790 2024-09-12T03:13:07.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.142+0000 7fb00b7fe640 1 --2- 172.21.15.116:0/410979504 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb00c101de0 0x7fb00c1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:13:07.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.142+0000 7fb00bfff640 1 --2- 172.21.15.116:0/410979504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb00c106f10 0x7fb00c19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:13:07.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.142+0000 7fb003fff640 1 --2- 172.21.15.116:0/410979504 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb00c102790 0x7fb00c198a10 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7faff400d950 tx=0x7faff400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:07.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.142+0000 7fb0097fa640 1 -- 172.21.15.116:0/410979504 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff4017030 con 0x7fb00c102790 2024-09-12T03:13:07.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.142+0000 7fb0097fa640 1 -- 172.21.15.116:0/410979504 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faff4004020 con 0x7fb00c102790 2024-09-12T03:13:07.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.142+0000 7fb0097fa640 1 -- 172.21.15.116:0/410979504 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff40047a0 con 0x7fb00c102790 2024-09-12T03:13:07.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.142+0000 7fb011c8e640 1 -- 172.21.15.116:0/410979504 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb00c19d770 con 0x7fb00c102790 2024-09-12T03:13:07.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.142+0000 7fb011c8e640 1 -- 172.21.15.116:0/410979504 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb00c1a5050 con 0x7fb00c102790 2024-09-12T03:13:07.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.144+0000 7fb0097fa640 1 -- 172.21.15.116:0/410979504 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7faff40041d0 con 0x7fb00c102790 2024-09-12T03:13:07.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.145+0000 7fb014067740 1 Processor -- start 2024-09-12T03:13:07.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.145+0000 7fb014067740 1 -- start start 2024-09-12T03:13:07.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.145+0000 7fb014067740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643cfab4a40 0x5643cfab4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:07.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.145+0000 7fb014067740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643cfab5360 0x5643cfac3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:07.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.145+0000 7fb014067740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643cfac4150 0x5643cfac6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:07.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.145+0000 7fb0097fa640 1 --2- 172.21.15.116:0/410979504 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7faff80a2b80 0x7faff80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:07.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.145+0000 7fb014067740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5643cfa1a280 con 0x5643cfab5360 2024-09-12T03:13:07.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.145+0000 7fb014067740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5643cf9c1b10 con 0x5643cfac4150 2024-09-12T03:13:07.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.145+0000 7fb014067740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5643cfa16240 con 0x5643cfab4a40 2024-09-12T03:13:07.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.145+0000 7fb00b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643cfab5360 0x5643cfac3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:07.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.145+0000 7fb00b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643cfab5360 0x5643cfac3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34726/0 (socket says 172.21.15.116:34726) 2024-09-12T03:13:07.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.145+0000 7fb00b7fe640 1 -- 172.21.15.116:0/1936574115 learned_addr learned my addr 172.21.15.116:0/1936574115 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:07.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.145+0000 7fb003fff640 1 --2- 172.21.15.116:0/410979504 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7faff80a2b80 0x7faff80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:07.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.145+0000 7fb00bfff640 1 --2- 172.21.15.116:0/1936574115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643cfab4a40 0x5643cfab4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:07.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.146+0000 7fb00bfff640 1 --2- 172.21.15.116:0/1936574115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643cfac4150 0x5643cfac6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:07.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.146+0000 7fb0097fa640 1 -- 172.21.15.116:0/410979504 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7faff40ceab0 con 0x7fb00c102790 2024-09-12T03:13:07.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.146+0000 7fb00b7fe640 1 -- 172.21.15.116:0/1936574115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643cfab4a40 msgr2=0x5643cfab4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:07.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.146+0000 7fb00b7fe640 1 --2- 172.21.15.116:0/1936574115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643cfab4a40 0x5643cfab4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.146+0000 7fb00b7fe640 1 -- 172.21.15.116:0/1936574115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643cfac4150 msgr2=0x5643cfac6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:07.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.146+0000 7fb00b7fe640 1 --2- 172.21.15.116:0/1936574115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643cfac4150 0x5643cfac6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.146+0000 7fb00b7fe640 1 -- 172.21.15.116:0/1936574115 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643cf9ce1f0 con 0x5643cfab5360 2024-09-12T03:13:07.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.146+0000 7fb00bfff640 1 --2- 172.21.15.116:0/1936574115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643cfac4150 0x5643cfac6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:13:07.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.146+0000 7fb00b7fe640 1 --2- 172.21.15.116:0/1936574115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643cfab5360 0x5643cfac3c10 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7faffc033000 tx=0x7faffc032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899c4ca74269f5f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:07.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.146+0000 7fb003fff640 1 --2- 172.21.15.116:0/410979504 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7faff80a2b80 0x7faff80a5040 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7faff40a3a80 tx=0x7faff40091d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:07.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.146+0000 7fb001ffb640 1 -- 172.21.15.116:0/1936574115 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faffc03f070 con 0x5643cfab5360 2024-09-12T03:13:07.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.146+0000 7fb001ffb640 1 -- 172.21.15.116:0/1936574115 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faffc0317b0 con 0x5643cfab5360 2024-09-12T03:13:07.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.146+0000 7fb001ffb640 1 -- 172.21.15.116:0/1936574115 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faffc035070 con 0x5643cfab5360 2024-09-12T03:13:07.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.147+0000 7fb014067740 1 -- 172.21.15.116:0/1936574115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643cfab5360 msgr2=0x5643cfac3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:07.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.147+0000 7fb014067740 1 --2- 172.21.15.116:0/1936574115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643cfab5360 0x5643cfac3c10 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7faffc033000 tx=0x7faffc032cb0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.147+0000 7fb014067740 1 -- 172.21.15.116:0/1936574115 shutdown_connections 2024-09-12T03:13:07.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.147+0000 7fb014067740 1 --2- 172.21.15.116:0/1936574115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643cfac4150 0x5643cfac6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.147+0000 7fb014067740 1 --2- 172.21.15.116:0/1936574115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643cfab5360 0x5643cfac3c10 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.147+0000 7fb014067740 1 --2- 172.21.15.116:0/1936574115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643cfab4a40 0x5643cfab4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.147+0000 7fb014067740 1 -- 172.21.15.116:0/1936574115 >> 172.21.15.116:0/1936574115 conn(0x5643cfab10f0 msgr2=0x5643cfab14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:07.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.147+0000 7fb014067740 1 -- 172.21.15.116:0/1936574115 shutdown_connections 2024-09-12T03:13:07.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.147+0000 7fb014067740 1 -- 172.21.15.116:0/1936574115 wait complete. 2024-09-12T03:13:07.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.147+0000 7fb014067740 1 -- start start 2024-09-12T03:13:07.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.147+0000 7fb014067740 1 Processor -- start 2024-09-12T03:13:07.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.148+0000 7fb014067740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643cfab4a40 0x5643cfac10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:07.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.148+0000 7fb014067740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643cfab5360 0x5643cfac15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:07.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.148+0000 7fb014067740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643cfac4150 0x5643cfac1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:07.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.148+0000 7fb014067740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5643cfac6d20 con 0x5643cfab4a40 2024-09-12T03:13:07.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.148+0000 7fb014067740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5643cf9d4f10 con 0x5643cfab5360 2024-09-12T03:13:07.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.148+0000 7fb014067740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5643cfac6ea0 con 0x5643cfac4150 2024-09-12T03:13:07.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.148+0000 7fb014067740 10 client.0 _ll_register_callbacks cb 0x5643cf99da40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:07.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.148+0000 7fb00bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643cfab4a40 0x5643cfac10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:07.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.148+0000 7fb00bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643cfab4a40 0x5643cfac10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34730/0 (socket says 172.21.15.116:34730) 2024-09-12T03:13:07.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.148+0000 7fb00bfff640 1 -- 172.21.15.116:0/2353348045 learned_addr learned my addr 172.21.15.116:0/2353348045 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:07.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.148+0000 7fb00b7fe640 1 --2- 172.21.15.116:0/2353348045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643cfab5360 0x5643cfac15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:07.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.148+0000 7fb00bfff640 1 -- 172.21.15.116:0/2353348045 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643cfac4150 msgr2=0x5643cfac1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:07.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.148+0000 7fb00bfff640 1 --2- 172.21.15.116:0/2353348045 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643cfac4150 0x5643cfac1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.148+0000 7fb00bfff640 1 -- 172.21.15.116:0/2353348045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643cfab5360 msgr2=0x5643cfac15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:07.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.148+0000 7fb00bfff640 1 --2- 172.21.15.116:0/2353348045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643cfab5360 0x5643cfac15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.148+0000 7fb00bfff640 1 -- 172.21.15.116:0/2353348045 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643cf78b6a0 con 0x5643cfab4a40 2024-09-12T03:13:07.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.148+0000 7fb00bfff640 1 --2- 172.21.15.116:0/2353348045 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643cfab4a40 0x5643cfac10b0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fb004009fb0 tx=0x7fb0040117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7818abefd71f46cf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:07.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.148+0000 7fafeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:07.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.148+0000 7fafeaffd640 1 -- 172.21.15.116:0/2353348045 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00401a070 con 0x5643cfab4a40 2024-09-12T03:13:07.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.148+0000 7fafeaffd640 1 -- 172.21.15.116:0/2353348045 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fb004004d20 con 0x5643cfab4a40 2024-09-12T03:13:07.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.148+0000 7fb014067740 1 -- 172.21.15.116:0/2353348045 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faffc0317b0 con 0x5643cfab4a40 2024-09-12T03:13:07.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.149+0000 7fafeaffd640 1 -- 172.21.15.116:0/2353348045 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb004012e60 con 0x5643cfab4a40 2024-09-12T03:13:07.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.149+0000 7fb014067740 10 client.18114 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:07.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.149+0000 7fb014067740 1 -- 172.21.15.116:0/2353348045 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5643cfad29e0 con 0x5643cfab4a40 2024-09-12T03:13:07.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.149+0000 7fafeaffd640 1 -- 172.21.15.116:0/2353348045 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faffc0317b0 con 0x5643cfab4a40 2024-09-12T03:13:07.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.149+0000 7fafeaffd640 1 -- 172.21.15.116:0/2353348045 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb00400f450 con 0x5643cfab4a40 2024-09-12T03:13:07.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.149+0000 7fb014067740 10 client.18114 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:07.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.149+0000 7fb014067740 10 client.18114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:13:07.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.149+0000 7fb014067740 20 client.18114 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:07.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.149+0000 7fb014067740 1 --2- 172.21.15.116:0/2353348045 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5643cfad3df0 0x5643cfad6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:07.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.149+0000 7fb014067740 4 client.18114 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:13:07.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.149+0000 7fb014067740 1 -- 172.21.15.116:0/2353348045 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643cfa277f0 con 0x5643cfad3df0 2024-09-12T03:13:07.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.149+0000 7fb00b7fe640 1 --2- 172.21.15.116:0/2353348045 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5643cfad3df0 0x5643cfad6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:07.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.150+0000 7fb00b7fe640 1 --2- 172.21.15.116:0/2353348045 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5643cfad3df0 0x5643cfad6200 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edbbb4fbc5e245f server_cookie=e243c908177aee8a in_seq=0 out_seq=0 2024-09-12T03:13:07.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.150+0000 7fafeaffd640 10 client.18114 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:07.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.150+0000 7fafeaffd640 1 -- 172.21.15.116:0/2353348045 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5643cfa277f0 con 0x5643cfad3df0 2024-09-12T03:13:07.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.150+0000 7fafeaffd640 10 client.18114 handle_command_reply: tid=0 2024-09-12T03:13:07.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.150+0000 7fb014067740 1 client.18114 shutdown 2024-09-12T03:13:07.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.150+0000 7fb014067740 1 -- 172.21.15.116:0/2353348045 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643cfab4a40 msgr2=0x5643cfac10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:07.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.150+0000 7fb014067740 1 --2- 172.21.15.116:0/2353348045 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643cfab4a40 0x5643cfac10b0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fb004009fb0 tx=0x7fb0040117b0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.150+0000 7fb014067740 1 -- 172.21.15.116:0/2353348045 shutdown_connections 2024-09-12T03:13:07.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.150+0000 7fb014067740 1 --2- 172.21.15.116:0/2353348045 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5643cfad3df0 0x5643cfad6200 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.150+0000 7fb014067740 1 --2- 172.21.15.116:0/2353348045 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643cfac4150 0x5643cfac1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.151+0000 7fafeaffd640 0 client.18114 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:07.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.150+0000 7fb014067740 1 --2- 172.21.15.116:0/2353348045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643cfab5360 0x5643cfac15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.151+0000 7fb014067740 1 --2- 172.21.15.116:0/2353348045 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643cfab4a40 0x5643cfac10b0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.151+0000 7fb014067740 1 -- 172.21.15.116:0/2353348045 >> 172.21.15.116:0/2353348045 conn(0x5643cfab10f0 msgr2=0x5643cfab6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:07.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.151+0000 7fb014067740 1 -- 172.21.15.116:0/2353348045 shutdown_connections 2024-09-12T03:13:07.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.151+0000 7fb014067740 1 -- 172.21.15.116:0/2353348045 wait complete. 2024-09-12T03:13:07.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.151+0000 7fb014067740 20 client.18114 trim_cache size 0 max 16384 2024-09-12T03:13:07.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.168+0000 7fb014067740 1 Processor -- start 2024-09-12T03:13:07.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.168+0000 7fb014067740 1 -- start start 2024-09-12T03:13:07.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.168+0000 7fb014067740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643cfab4a40 0x5643cfadbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:07.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.168+0000 7fb014067740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643cfab5360 0x5643cfadc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:07.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.168+0000 7fb014067740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643cfac4150 0x5643cfac1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:07.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.168+0000 7fb014067740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c1adae0 con 0x5643cfab5360 2024-09-12T03:13:07.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.168+0000 7fb014067740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c1ab3b0 con 0x5643cfac4150 2024-09-12T03:13:07.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.168+0000 7fb014067740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c1a9200 con 0x5643cfab4a40 2024-09-12T03:13:07.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.168+0000 7fb00b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643cfab5360 0x5643cfadc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:07.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.168+0000 7fb00b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643cfab5360 0x5643cfadc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34738/0 (socket says 172.21.15.116:34738) 2024-09-12T03:13:07.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.168+0000 7fb00b7fe640 1 -- 172.21.15.116:0/2578702767 learned_addr learned my addr 172.21.15.116:0/2578702767 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:07.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.168+0000 7fb00b7fe640 1 -- 172.21.15.116:0/2578702767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643cfab4a40 msgr2=0x5643cfadbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:07.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.168+0000 7fb00b7fe640 1 --2- 172.21.15.116:0/2578702767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643cfab4a40 0x5643cfadbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.168+0000 7fb00b7fe640 1 -- 172.21.15.116:0/2578702767 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643cfac4150 msgr2=0x5643cfac1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:07.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.168+0000 7fb00b7fe640 1 --2- 172.21.15.116:0/2578702767 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643cfac4150 0x5643cfac1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.168+0000 7fb00b7fe640 1 -- 172.21.15.116:0/2578702767 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643cfab7560 con 0x5643cfab5360 2024-09-12T03:13:07.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.168+0000 7fb00b7fe640 1 --2- 172.21.15.116:0/2578702767 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643cfab5360 0x5643cfadc010 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7faffc009900 tx=0x7faffc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204d532ae7e96e37 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:07.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.168+0000 7fb0017fa640 1 -- 172.21.15.116:0/2578702767 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faffc035070 con 0x5643cfab5360 2024-09-12T03:13:07.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.168+0000 7fb0017fa640 1 -- 172.21.15.116:0/2578702767 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faffc005610 con 0x5643cfab5360 2024-09-12T03:13:07.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.169+0000 7fb0017fa640 1 -- 172.21.15.116:0/2578702767 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faffc03f040 con 0x5643cfab5360 2024-09-12T03:13:07.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.169+0000 7fb014067740 1 -- 172.21.15.116:0/2578702767 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643cfab5360 msgr2=0x5643cfadc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:07.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.169+0000 7fb014067740 1 --2- 172.21.15.116:0/2578702767 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643cfab5360 0x5643cfadc010 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7faffc009900 tx=0x7faffc000bc0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.169+0000 7fb014067740 1 -- 172.21.15.116:0/2578702767 shutdown_connections 2024-09-12T03:13:07.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.169+0000 7fb014067740 1 --2- 172.21.15.116:0/2578702767 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643cfac4150 0x5643cfac1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.169+0000 7fb014067740 1 --2- 172.21.15.116:0/2578702767 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643cfab5360 0x5643cfadc010 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.169+0000 7fb014067740 1 --2- 172.21.15.116:0/2578702767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643cfab4a40 0x5643cfadbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.169+0000 7fb014067740 1 -- 172.21.15.116:0/2578702767 >> 172.21.15.116:0/2578702767 conn(0x5643cfab10f0 msgr2=0x5643cfac01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:07.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.169+0000 7fb014067740 1 -- 172.21.15.116:0/2578702767 shutdown_connections 2024-09-12T03:13:07.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.169+0000 7fb014067740 1 -- 172.21.15.116:0/2578702767 wait complete. 2024-09-12T03:13:07.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.169+0000 7fb014067740 1 -- start start 2024-09-12T03:13:07.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.170+0000 7fb014067740 1 Processor -- start 2024-09-12T03:13:07.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.170+0000 7fb014067740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643cfab4a40 0x5643cfac80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:07.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.170+0000 7fb014067740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643cfab5360 0x5643cfac85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:07.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.170+0000 7fb014067740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643cfac4150 0x5643cfac8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:07.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.170+0000 7fb014067740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c1a9870 con 0x5643cfab5360 2024-09-12T03:13:07.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.170+0000 7fb014067740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c1aace0 con 0x5643cfac4150 2024-09-12T03:13:07.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.170+0000 7fb014067740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c1a9f40 con 0x5643cfab4a40 2024-09-12T03:13:07.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.170+0000 7fb014067740 10 client.0 _ll_register_callbacks cb 0x7fb004009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:07.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.170+0000 7fb00b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643cfab5360 0x5643cfac85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:07.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.170+0000 7fb00b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643cfab5360 0x5643cfac85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34740/0 (socket says 172.21.15.116:34740) 2024-09-12T03:13:07.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.170+0000 7fb00b7fe640 1 -- 172.21.15.116:0/31296443 learned_addr learned my addr 172.21.15.116:0/31296443 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:07.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.170+0000 7fb00bfff640 1 --2- 172.21.15.116:0/31296443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643cfab4a40 0x5643cfac80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:07.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.170+0000 7fb00bfff640 1 --2- 172.21.15.116:0/31296443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643cfac4150 0x5643cfac8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:07.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.170+0000 7fb00b7fe640 1 -- 172.21.15.116:0/31296443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643cfab4a40 msgr2=0x5643cfac80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:07.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.170+0000 7fb00b7fe640 1 --2- 172.21.15.116:0/31296443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643cfab4a40 0x5643cfac80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.170+0000 7fb00b7fe640 1 -- 172.21.15.116:0/31296443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643cfac4150 msgr2=0x5643cfac8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:07.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.170+0000 7fb00b7fe640 1 --2- 172.21.15.116:0/31296443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643cfac4150 0x5643cfac8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.170+0000 7fb00b7fe640 1 -- 172.21.15.116:0/31296443 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643cfb3dca0 con 0x5643cfab5360 2024-09-12T03:13:07.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.170+0000 7fb00bfff640 1 --2- 172.21.15.116:0/31296443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643cfac4150 0x5643cfac8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:13:07.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.170+0000 7fb00b7fe640 1 --2- 172.21.15.116:0/31296443 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643cfab5360 0x5643cfac85e0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7faffc005bb0 tx=0x7faffc004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7c4d65a189f162 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:07.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.170+0000 7fafeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:07.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.171+0000 7fafeaffd640 1 -- 172.21.15.116:0/31296443 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faffc035070 con 0x5643cfab5360 2024-09-12T03:13:07.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.171+0000 7fafeaffd640 1 -- 172.21.15.116:0/31296443 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7faffc031630 con 0x5643cfab5360 2024-09-12T03:13:07.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.171+0000 7fb014067740 1 -- 172.21.15.116:0/31296443 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faffc005610 con 0x5643cfab5360 2024-09-12T03:13:07.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.171+0000 7fafeaffd640 1 -- 172.21.15.116:0/31296443 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7faffc05d070 con 0x5643cfab5360 2024-09-12T03:13:07.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.171+0000 7fb014067740 10 client.18120 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:07.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.171+0000 7fb014067740 1 -- 172.21.15.116:0/31296443 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5643cfb3efc0 con 0x5643cfab5360 2024-09-12T03:13:07.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.171+0000 7fafeaffd640 1 -- 172.21.15.116:0/31296443 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faffc005610 con 0x5643cfab5360 2024-09-12T03:13:07.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.171+0000 7fafeaffd640 1 -- 172.21.15.116:0/31296443 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7faffc0046c0 con 0x5643cfab5360 2024-09-12T03:13:07.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.171+0000 7fb014067740 10 client.18120 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:07.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.171+0000 7fb014067740 10 client.18120 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:13:07.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.171+0000 7fb014067740 20 client.18120 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:07.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.171+0000 7fb014067740 1 --2- 172.21.15.116:0/31296443 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5643cfb3f260 0x5643cfb41680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:07.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.171+0000 7fb014067740 4 client.18120 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:13:07.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.171+0000 7fb014067740 1 -- 172.21.15.116:0/31296443 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5643cfa21d00 con 0x5643cfb3f260 2024-09-12T03:13:07.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.171+0000 7fb00bfff640 1 --2- 172.21.15.116:0/31296443 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5643cfb3f260 0x5643cfb41680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:07.193 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:13:07.193 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:13:07.193 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "02e618a6-cd26-4ca8-ae78-345f4b44b917", 2024-09-12T03:13:07.193 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:13:07.193 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:13:07.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.172+0000 7fb00bfff640 1 --2- 172.21.15.116:0/31296443 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5643cfb3f260 0x5643cfb41680 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdadaa7732724364 server_cookie=8a2883acab19ae9f in_seq=0 out_seq=0 2024-09-12T03:13:07.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.172+0000 7fafeaffd640 10 client.18120 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:07.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.173+0000 7fafeaffd640 1 -- 172.21.15.116:0/31296443 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5643cfa21d00 con 0x5643cfb3f260 2024-09-12T03:13:07.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.173+0000 7fafeaffd640 10 client.18120 handle_command_reply: tid=0 2024-09-12T03:13:07.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.173+0000 7fb014067740 1 client.18120 shutdown 2024-09-12T03:13:07.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.173+0000 7fb014067740 1 -- 172.21.15.116:0/31296443 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643cfab5360 msgr2=0x5643cfac85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:07.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.173+0000 7fb014067740 1 --2- 172.21.15.116:0/31296443 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643cfab5360 0x5643cfac85e0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7faffc005bb0 tx=0x7faffc004610 comp rx=0 tx=0).stop 2024-09-12T03:13:07.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.173+0000 7fb014067740 1 -- 172.21.15.116:0/31296443 shutdown_connections 2024-09-12T03:13:07.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.173+0000 7fb014067740 1 --2- 172.21.15.116:0/31296443 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5643cfb3f260 0x5643cfb41680 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.173+0000 7fb014067740 1 --2- 172.21.15.116:0/31296443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643cfac4150 0x5643cfac8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.173+0000 7fb014067740 1 --2- 172.21.15.116:0/31296443 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643cfab5360 0x5643cfac85e0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.173+0000 7fafeaffd640 0 client.18120 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:07.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.173+0000 7fb014067740 1 --2- 172.21.15.116:0/31296443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643cfab4a40 0x5643cfac80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.173+0000 7fb014067740 1 -- 172.21.15.116:0/31296443 >> 172.21.15.116:0/31296443 conn(0x5643cfab10f0 msgr2=0x5643cfab6550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:07.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.173+0000 7fb014067740 1 -- 172.21.15.116:0/31296443 shutdown_connections 2024-09-12T03:13:07.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.173+0000 7fb014067740 1 -- 172.21.15.116:0/31296443 wait complete. 2024-09-12T03:13:07.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.173+0000 7fb014067740 20 client.18120 trim_cache size 0 max 16384 2024-09-12T03:13:07.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.174+0000 7fb011c8e640 1 -- 172.21.15.116:0/410979504 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7faff80a2b80 msgr2=0x7faff80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:07.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.174+0000 7fb011c8e640 1 --2- 172.21.15.116:0/410979504 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7faff80a2b80 0x7faff80a5040 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7faff40a3a80 tx=0x7faff40091d0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.174+0000 7fb011c8e640 1 -- 172.21.15.116:0/410979504 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb00c102790 msgr2=0x7fb00c198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:07.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.174+0000 7fb011c8e640 1 --2- 172.21.15.116:0/410979504 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb00c102790 0x7fb00c198a10 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7faff400d950 tx=0x7faff400de10 comp rx=0 tx=0).stop 2024-09-12T03:13:07.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.174+0000 7fb011c8e640 1 -- 172.21.15.116:0/410979504 shutdown_connections 2024-09-12T03:13:07.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.174+0000 7fb011c8e640 1 --2- 172.21.15.116:0/410979504 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7faff80a2b80 0x7faff80a5040 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.174+0000 7fb011c8e640 1 --2- 172.21.15.116:0/410979504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb00c106f10 0x7fb00c19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.174+0000 7fb011c8e640 1 --2- 172.21.15.116:0/410979504 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb00c102790 0x7fb00c198a10 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.174+0000 7fb011c8e640 1 --2- 172.21.15.116:0/410979504 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb00c101de0 0x7fb00c1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:07.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.174+0000 7fb011c8e640 1 -- 172.21.15.116:0/410979504 >> 172.21.15.116:0/410979504 conn(0x7fb00c0fd8c0 msgr2=0x7fb00c0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:07.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.174+0000 7fb011c8e640 1 -- 172.21.15.116:0/410979504 shutdown_connections 2024-09-12T03:13:07.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:07.174+0000 7fb011c8e640 1 -- 172.21.15.116:0/410979504 wait complete. 2024-09-12T03:13:07.544 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:13:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:07 smithi183 ceph-mon[35535]: pgmap v642: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T03:13:07.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:07 smithi177 ceph-mon[35305]: pgmap v642: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T03:13:07.782 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:07 smithi116 ceph-mon[31410]: pgmap v642: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T03:13:08.067 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:13:08.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.579+0000 7f70de304640 1 -- 172.21.15.116:0/962826352 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70d8102d30 msgr2=0x7f70d8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:08.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.579+0000 7f70de304640 1 --2- 172.21.15.116:0/962826352 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70d8102d30 0x7f70d8103110 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f70c8009930 tx=0x7f70c802a420 comp rx=0 tx=0).stop 2024-09-12T03:13:08.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.579+0000 7f70de304640 1 -- 172.21.15.116:0/962826352 shutdown_connections 2024-09-12T03:13:08.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.579+0000 7f70de304640 1 --2- 172.21.15.116:0/962826352 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70d81082a0 0x7f70d810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.579+0000 7f70de304640 1 --2- 172.21.15.116:0/962826352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70d81036e0 0x7f70d8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.579+0000 7f70de304640 1 --2- 172.21.15.116:0/962826352 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70d8102d30 0x7f70d8103110 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.579+0000 7f70de304640 1 -- 172.21.15.116:0/962826352 >> 172.21.15.116:0/962826352 conn(0x7f70d80fc7d0 msgr2=0x7f70d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:08.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.579+0000 7f70de304640 1 -- 172.21.15.116:0/962826352 shutdown_connections 2024-09-12T03:13:08.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.579+0000 7f70de304640 1 -- 172.21.15.116:0/962826352 wait complete. 2024-09-12T03:13:08.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.580+0000 7f70de304640 1 Processor -- start 2024-09-12T03:13:08.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.580+0000 7f70de304640 1 -- start start 2024-09-12T03:13:08.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.580+0000 7f70de304640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70d8102d30 0x7f70d8198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:08.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.580+0000 7f70de304640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70d81036e0 0x7f70d8198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:08.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.580+0000 7f70de304640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70d81082a0 0x7f70d819ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:08.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.580+0000 7f70de304640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f70d81120a0 con 0x7f70d81082a0 2024-09-12T03:13:08.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.580+0000 7f70de304640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f70d8111f20 con 0x7f70d81036e0 2024-09-12T03:13:08.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.580+0000 7f70de304640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f70d8112220 con 0x7f70d8102d30 2024-09-12T03:13:08.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.580+0000 7f70dc8a1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70d81082a0 0x7f70d819ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:08.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.580+0000 7f70dc8a1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70d81082a0 0x7f70d819ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34758/0 (socket says 172.21.15.116:34758) 2024-09-12T03:13:08.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.580+0000 7f70dc8a1640 1 -- 172.21.15.116:0/735199544 learned_addr learned my addr 172.21.15.116:0/735199544 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:08.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.580+0000 7f70cffff640 1 --2- 172.21.15.116:0/735199544 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70d81036e0 0x7f70d8198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:08.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.580+0000 7f70d7fff640 1 --2- 172.21.15.116:0/735199544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70d8102d30 0x7f70d8198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:08.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.580+0000 7f70dc8a1640 1 -- 172.21.15.116:0/735199544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70d8102d30 msgr2=0x7f70d8198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:08.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.580+0000 7f70dc8a1640 1 --2- 172.21.15.116:0/735199544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70d8102d30 0x7f70d8198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.580+0000 7f70dc8a1640 1 -- 172.21.15.116:0/735199544 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70d81036e0 msgr2=0x7f70d8198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:08.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.580+0000 7f70dc8a1640 1 --2- 172.21.15.116:0/735199544 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70d81036e0 0x7f70d8198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.580+0000 7f70dc8a1640 1 -- 172.21.15.116:0/735199544 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d819d530 con 0x7f70d81082a0 2024-09-12T03:13:08.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.581+0000 7f70dc8a1640 1 --2- 172.21.15.116:0/735199544 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70d81082a0 0x7f70d819ce50 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f70d000eaa0 tx=0x7f70d000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:08.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.581+0000 7f70d5ffb640 1 -- 172.21.15.116:0/735199544 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d000cdb0 con 0x7f70d81082a0 2024-09-12T03:13:08.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.581+0000 7f70d5ffb640 1 -- 172.21.15.116:0/735199544 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70d0005020 con 0x7f70d81082a0 2024-09-12T03:13:08.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.581+0000 7f70de304640 1 -- 172.21.15.116:0/735199544 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d819d820 con 0x7f70d81082a0 2024-09-12T03:13:08.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.581+0000 7f70d5ffb640 1 -- 172.21.15.116:0/735199544 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d00116b0 con 0x7f70d81082a0 2024-09-12T03:13:08.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.581+0000 7f70de304640 1 -- 172.21.15.116:0/735199544 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d81a5100 con 0x7f70d81082a0 2024-09-12T03:13:08.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.583+0000 7f70d5ffb640 1 -- 172.21.15.116:0/735199544 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f70d0005560 con 0x7f70d81082a0 2024-09-12T03:13:08.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.583+0000 7f70e06dd740 1 Processor -- start 2024-09-12T03:13:08.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.583+0000 7f70e06dd740 1 -- start start 2024-09-12T03:13:08.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.584+0000 7f70e06dd740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c697d58060 0x55c697d58440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:08.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.584+0000 7f70e06dd740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c697d58980 0x55c697d62d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:08.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.584+0000 7f70e06dd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c697d63240 0x55c697d65630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:08.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.584+0000 7f70e06dd740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c697cbd620 con 0x55c697d58980 2024-09-12T03:13:08.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.584+0000 7f70e06dd740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c697c64e50 con 0x55c697d58060 2024-09-12T03:13:08.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.584+0000 7f70e06dd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c697cb95e0 con 0x55c697d63240 2024-09-12T03:13:08.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.584+0000 7f70d7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c697d58980 0x55c697d62d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:08.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.584+0000 7f70d7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c697d58980 0x55c697d62d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34774/0 (socket says 172.21.15.116:34774) 2024-09-12T03:13:08.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.584+0000 7f70d7fff640 1 -- 172.21.15.116:0/2733247785 learned_addr learned my addr 172.21.15.116:0/2733247785 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:08.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.584+0000 7f70d5ffb640 1 --2- 172.21.15.116:0/735199544 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f70c40a2c00 0x7f70c40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:08.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.584+0000 7f70cffff640 1 --2- 172.21.15.116:0/2733247785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c697d63240 0x55c697d65630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:08.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.584+0000 7f70dc8a1640 1 --2- 172.21.15.116:0/735199544 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f70c40a2c00 0x7f70c40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:08.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.584+0000 7f70cffff640 1 --2- 172.21.15.116:0/2733247785 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c697d58060 0x55c697d58440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:08.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.584+0000 7f70d7fff640 1 -- 172.21.15.116:0/2733247785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c697d63240 msgr2=0x55c697d65630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:08.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.584+0000 7f70d7fff640 1 --2- 172.21.15.116:0/2733247785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c697d63240 0x55c697d65630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.584+0000 7f70d7fff640 1 -- 172.21.15.116:0/2733247785 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c697d58060 msgr2=0x55c697d58440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:08.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.584+0000 7f70d7fff640 1 --2- 172.21.15.116:0/2733247785 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c697d58060 0x55c697d58440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.584+0000 7f70d7fff640 1 -- 172.21.15.116:0/2733247785 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c697c55d40 con 0x55c697d58980 2024-09-12T03:13:08.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.584+0000 7f70cffff640 1 --2- 172.21.15.116:0/2733247785 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c697d58060 0x55c697d58440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:13:08.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.584+0000 7f70d5ffb640 1 -- 172.21.15.116:0/735199544 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f70d00cf0e0 con 0x7f70d81082a0 2024-09-12T03:13:08.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.584+0000 7f70d7fff640 1 --2- 172.21.15.116:0/2733247785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c697d58980 0x55c697d62d00 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f70c802a940 tx=0x7f70c8005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6518d3b1ed82f766 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:08.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.585+0000 7f70dc8a1640 1 --2- 172.21.15.116:0/735199544 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f70c40a2c00 0x7f70c40a50c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f70d00a40b0 tx=0x7f70d0066b40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:08.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.585+0000 7f70ce7fc640 1 -- 172.21.15.116:0/2733247785 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c8040070 con 0x55c697d58980 2024-09-12T03:13:08.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.585+0000 7f70ce7fc640 1 -- 172.21.15.116:0/2733247785 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70c80319f0 con 0x55c697d58980 2024-09-12T03:13:08.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.585+0000 7f70ce7fc640 1 -- 172.21.15.116:0/2733247785 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c8035040 con 0x55c697d58980 2024-09-12T03:13:08.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.585+0000 7f70e06dd740 1 -- 172.21.15.116:0/2733247785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c697d58980 msgr2=0x55c697d62d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:08.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.585+0000 7f70e06dd740 1 --2- 172.21.15.116:0/2733247785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c697d58980 0x55c697d62d00 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f70c802a940 tx=0x7f70c8005c00 comp rx=0 tx=0).stop 2024-09-12T03:13:08.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.585+0000 7f70e06dd740 1 -- 172.21.15.116:0/2733247785 shutdown_connections 2024-09-12T03:13:08.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.585+0000 7f70e06dd740 1 --2- 172.21.15.116:0/2733247785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c697d63240 0x55c697d65630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.585+0000 7f70e06dd740 1 --2- 172.21.15.116:0/2733247785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c697d58980 0x55c697d62d00 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.585+0000 7f70e06dd740 1 --2- 172.21.15.116:0/2733247785 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c697d58060 0x55c697d58440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.585+0000 7f70e06dd740 1 -- 172.21.15.116:0/2733247785 >> 172.21.15.116:0/2733247785 conn(0x55c697d545f0 msgr2=0x55c697d549d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:08.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.585+0000 7f70e06dd740 1 -- 172.21.15.116:0/2733247785 shutdown_connections 2024-09-12T03:13:08.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.585+0000 7f70e06dd740 1 -- 172.21.15.116:0/2733247785 wait complete. 2024-09-12T03:13:08.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.586+0000 7f70e06dd740 1 -- start start 2024-09-12T03:13:08.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.586+0000 7f70e06dd740 1 Processor -- start 2024-09-12T03:13:08.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.586+0000 7f70e06dd740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c697d58060 0x55c697d5ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:08.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.586+0000 7f70e06dd740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c697d58980 0x55c697d60510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:08.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.586+0000 7f70e06dd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c697d63240 0x55c697d60a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:08.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.586+0000 7f70e06dd740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c697d65e10 con 0x55c697d58980 2024-09-12T03:13:08.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.586+0000 7f70e06dd740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c697c45f10 con 0x55c697d58060 2024-09-12T03:13:08.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.586+0000 7f70e06dd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c697d65f90 con 0x55c697d63240 2024-09-12T03:13:08.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.586+0000 7f70e06dd740 10 client.0 _ll_register_callbacks cb 0x55c697c69db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:08.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.586+0000 7f70d7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c697d58980 0x55c697d60510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:08.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.586+0000 7f70d7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c697d58980 0x55c697d60510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34784/0 (socket says 172.21.15.116:34784) 2024-09-12T03:13:08.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.586+0000 7f70d7fff640 1 -- 172.21.15.116:0/3677727376 learned_addr learned my addr 172.21.15.116:0/3677727376 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:08.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.587+0000 7f70cffff640 1 --2- 172.21.15.116:0/3677727376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c697d58060 0x55c697d5ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:08.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.587+0000 7f70cffff640 1 --2- 172.21.15.116:0/3677727376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c697d63240 0x55c697d60a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:08.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.587+0000 7f70d7fff640 1 -- 172.21.15.116:0/3677727376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c697d63240 msgr2=0x55c697d60a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:08.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.587+0000 7f70d7fff640 1 --2- 172.21.15.116:0/3677727376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c697d63240 0x55c697d60a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.587+0000 7f70d7fff640 1 -- 172.21.15.116:0/3677727376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c697d58060 msgr2=0x55c697d5ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:08.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.587+0000 7f70d7fff640 1 --2- 172.21.15.116:0/3677727376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c697d58060 0x55c697d5ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.587+0000 7f70d7fff640 1 -- 172.21.15.116:0/3677727376 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6979cc580 con 0x55c697d58980 2024-09-12T03:13:08.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.587+0000 7f70cffff640 1 --2- 172.21.15.116:0/3677727376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c697d63240 0x55c697d60a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:13:08.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.587+0000 7f70d7fff640 1 --2- 172.21.15.116:0/3677727376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c697d58980 0x55c697d60510 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f70c8006010 tx=0x7f70c802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0af248c125d0e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:08.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.587+0000 7f70b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:08.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.587+0000 7f70b77fe640 1 -- 172.21.15.116:0/3677727376 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c8040030 con 0x55c697d58980 2024-09-12T03:13:08.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.587+0000 7f70b77fe640 1 -- 172.21.15.116:0/3677727376 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f70c8004260 con 0x55c697d58980 2024-09-12T03:13:08.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.587+0000 7f70e06dd740 1 -- 172.21.15.116:0/3677727376 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70c80319f0 con 0x55c697d58980 2024-09-12T03:13:08.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.587+0000 7f70b77fe640 1 -- 172.21.15.116:0/3677727376 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f70c8035040 con 0x55c697d58980 2024-09-12T03:13:08.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.587+0000 7f70e06dd740 10 client.18132 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:08.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.587+0000 7f70e06dd740 1 -- 172.21.15.116:0/3677727376 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c697a96960 con 0x55c697d58980 2024-09-12T03:13:08.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.587+0000 7f70b77fe640 1 -- 172.21.15.116:0/3677727376 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70c80319f0 con 0x55c697d58980 2024-09-12T03:13:08.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.588+0000 7f70b77fe640 1 -- 172.21.15.116:0/3677727376 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f70c8003850 con 0x55c697d58980 2024-09-12T03:13:08.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.588+0000 7f70e06dd740 10 client.18132 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:08.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.588+0000 7f70e06dd740 10 client.18132 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:13:08.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.588+0000 7f70e06dd740 20 client.18132 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:08.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.588+0000 7f70e06dd740 1 --2- 172.21.15.116:0/3677727376 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c697d72a50 0x55c697d74e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:08.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.588+0000 7f70e06dd740 4 client.18132 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:13:08.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.588+0000 7f70e06dd740 1 -- 172.21.15.116:0/3677727376 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c697ccab90 con 0x55c697d72a50 2024-09-12T03:13:08.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.588+0000 7f70cffff640 1 --2- 172.21.15.116:0/3677727376 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c697d72a50 0x55c697d74e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:08.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.588+0000 7f70b77fe640 10 client.18132 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:08.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.588+0000 7f70cffff640 1 --2- 172.21.15.116:0/3677727376 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c697d72a50 0x55c697d74e40 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=604dab571522b72d server_cookie=a006836c97253433 in_seq=0 out_seq=0 2024-09-12T03:13:08.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.589+0000 7f70b77fe640 1 -- 172.21.15.116:0/3677727376 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c697ccab90 con 0x55c697d72a50 2024-09-12T03:13:08.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.589+0000 7f70b77fe640 10 client.18132 handle_command_reply: tid=0 2024-09-12T03:13:08.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.589+0000 7f70e06dd740 1 client.18132 shutdown 2024-09-12T03:13:08.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.589+0000 7f70e06dd740 1 -- 172.21.15.116:0/3677727376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c697d58980 msgr2=0x55c697d60510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:08.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.589+0000 7f70e06dd740 1 --2- 172.21.15.116:0/3677727376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c697d58980 0x55c697d60510 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f70c8006010 tx=0x7f70c802a420 comp rx=0 tx=0).stop 2024-09-12T03:13:08.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.589+0000 7f70e06dd740 1 -- 172.21.15.116:0/3677727376 shutdown_connections 2024-09-12T03:13:08.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.589+0000 7f70e06dd740 1 --2- 172.21.15.116:0/3677727376 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c697d72a50 0x55c697d74e40 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.589+0000 7f70e06dd740 1 --2- 172.21.15.116:0/3677727376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c697d63240 0x55c697d60a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.589+0000 7f70e06dd740 1 --2- 172.21.15.116:0/3677727376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c697d58980 0x55c697d60510 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.589+0000 7f70e06dd740 1 --2- 172.21.15.116:0/3677727376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c697d58060 0x55c697d5ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.589+0000 7f70b77fe640 0 client.18132 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:08.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.589+0000 7f70e06dd740 1 -- 172.21.15.116:0/3677727376 >> 172.21.15.116:0/3677727376 conn(0x55c697d545f0 msgr2=0x55c697d53750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:08.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.589+0000 7f70e06dd740 1 -- 172.21.15.116:0/3677727376 shutdown_connections 2024-09-12T03:13:08.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.589+0000 7f70e06dd740 1 -- 172.21.15.116:0/3677727376 wait complete. 2024-09-12T03:13:08.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.589+0000 7f70e06dd740 20 client.18132 trim_cache size 0 max 16384 2024-09-12T03:13:08.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.609+0000 7f70e06dd740 1 Processor -- start 2024-09-12T03:13:08.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.610+0000 7f70e06dd740 1 -- start start 2024-09-12T03:13:08.610 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:08 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:13:08.610 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:08 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:13:08.610 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:08 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:13:08.610 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:08 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:13:08.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.610+0000 7f70e06dd740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c697d58060 0x55c697d5d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:08.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.610+0000 7f70e06dd740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c697d58980 0x55c697d5d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:08.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.610+0000 7f70e06dd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c697d63240 0x55c697d64630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:08.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.610+0000 7f70e06dd740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f70d81afd70 con 0x55c697d58060 2024-09-12T03:13:08.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.610+0000 7f70e06dd740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f70d81ad640 con 0x55c697d58980 2024-09-12T03:13:08.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.610+0000 7f70e06dd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f70d81ab490 con 0x55c697d63240 2024-09-12T03:13:08.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.610+0000 7f70cffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c697d58060 0x55c697d5d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:08.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.610+0000 7f70cffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c697d58060 0x55c697d5d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34794/0 (socket says 172.21.15.116:34794) 2024-09-12T03:13:08.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.610+0000 7f70cffff640 1 -- 172.21.15.116:0/3433261264 learned_addr learned my addr 172.21.15.116:0/3433261264 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:08.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.610+0000 7f70d7fff640 1 --2- 172.21.15.116:0/3433261264 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c697d58980 0x55c697d5d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:08.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.610+0000 7f70cffff640 1 --2- 172.21.15.116:0/3433261264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c697d63240 0x55c697d64630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:08.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.610+0000 7f70cffff640 1 -- 172.21.15.116:0/3433261264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c697d63240 msgr2=0x55c697d64630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:08.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.610+0000 7f70cffff640 1 --2- 172.21.15.116:0/3433261264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c697d63240 0x55c697d64630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.610+0000 7f70cffff640 1 -- 172.21.15.116:0/3433261264 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c697d58980 msgr2=0x55c697d5d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:08.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.610+0000 7f70cffff640 1 --2- 172.21.15.116:0/3433261264 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c697d58980 0x55c697d5d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.610+0000 7f70cffff640 1 -- 172.21.15.116:0/3433261264 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c697d587b0 con 0x55c697d58060 2024-09-12T03:13:08.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.611+0000 7f70cffff640 1 --2- 172.21.15.116:0/3433261264 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c697d58060 0x55c697d5d250 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f70c000c010 tx=0x7f70c0005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c73aa00a9a2b8f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:08.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.611+0000 7f70cdffb640 1 -- 172.21.15.116:0/3433261264 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c00044d0 con 0x55c697d58060 2024-09-12T03:13:08.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.611+0000 7f70cdffb640 1 -- 172.21.15.116:0/3433261264 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70c0005500 con 0x55c697d58060 2024-09-12T03:13:08.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.611+0000 7f70cdffb640 1 -- 172.21.15.116:0/3433261264 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c001e5f0 con 0x55c697d58060 2024-09-12T03:13:08.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.612+0000 7f70e06dd740 1 -- 172.21.15.116:0/3433261264 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c697d58060 msgr2=0x55c697d5d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:08.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.612+0000 7f70e06dd740 1 --2- 172.21.15.116:0/3433261264 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c697d58060 0x55c697d5d250 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f70c000c010 tx=0x7f70c0005c90 comp rx=0 tx=0).stop 2024-09-12T03:13:08.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.612+0000 7f70e06dd740 1 -- 172.21.15.116:0/3433261264 shutdown_connections 2024-09-12T03:13:08.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.612+0000 7f70e06dd740 1 --2- 172.21.15.116:0/3433261264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c697d63240 0x55c697d64630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.612+0000 7f70e06dd740 1 --2- 172.21.15.116:0/3433261264 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c697d58980 0x55c697d5d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.612+0000 7f70e06dd740 1 --2- 172.21.15.116:0/3433261264 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c697d58060 0x55c697d5d250 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.612+0000 7f70e06dd740 1 -- 172.21.15.116:0/3433261264 >> 172.21.15.116:0/3433261264 conn(0x55c697d545f0 msgr2=0x55c697d5e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:08.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.612+0000 7f70e06dd740 1 -- 172.21.15.116:0/3433261264 shutdown_connections 2024-09-12T03:13:08.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.612+0000 7f70e06dd740 1 -- 172.21.15.116:0/3433261264 wait complete. 2024-09-12T03:13:08.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.612+0000 7f70e06dd740 1 -- start start 2024-09-12T03:13:08.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.613+0000 7f70e06dd740 1 Processor -- start 2024-09-12T03:13:08.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.613+0000 7f70e06dd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c697d58980 0x55c697dd8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:08.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.613+0000 7f70e06dd740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c697d63240 0x55c697dd8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:08.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.613+0000 7f70e06dd740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c697dd6de0 0x55c697dd7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:08.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.613+0000 7f70e06dd740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f70d81abb00 con 0x55c697d63240 2024-09-12T03:13:08.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.613+0000 7f70e06dd740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f70d81acf70 con 0x55c697dd6de0 2024-09-12T03:13:08.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.613+0000 7f70e06dd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f70d81ac1d0 con 0x55c697d58980 2024-09-12T03:13:08.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.613+0000 7f70e06dd740 10 client.0 _ll_register_callbacks cb 0x7f70c8005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:08.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.613+0000 7f70d7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c697d63240 0x55c697dd8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:08.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.613+0000 7f70d7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c697d63240 0x55c697dd8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34796/0 (socket says 172.21.15.116:34796) 2024-09-12T03:13:08.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.613+0000 7f70d7fff640 1 -- 172.21.15.116:0/2308657841 learned_addr learned my addr 172.21.15.116:0/2308657841 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:08.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.613+0000 7f70cffff640 1 --2- 172.21.15.116:0/2308657841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c697d58980 0x55c697dd8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:08.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.614+0000 7f70cffff640 1 --2- 172.21.15.116:0/2308657841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c697dd6de0 0x55c697dd7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:08.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.614+0000 7f70d7fff640 1 -- 172.21.15.116:0/2308657841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c697d58980 msgr2=0x55c697dd8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:08.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.614+0000 7f70d7fff640 1 --2- 172.21.15.116:0/2308657841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c697d58980 0x55c697dd8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.614+0000 7f70d7fff640 1 -- 172.21.15.116:0/2308657841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c697dd6de0 msgr2=0x55c697dd7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:08.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.614+0000 7f70d7fff640 1 --2- 172.21.15.116:0/2308657841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c697dd6de0 0x55c697dd7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.614+0000 7f70d7fff640 1 -- 172.21.15.116:0/2308657841 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c697dd7a60 con 0x55c697d63240 2024-09-12T03:13:08.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.614+0000 7f70cffff640 1 --2- 172.21.15.116:0/2308657841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c697dd6de0 0x55c697dd7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:13:08.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.614+0000 7f70d7fff640 1 --2- 172.21.15.116:0/2308657841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c697d63240 0x55c697dd8dc0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f70c8004720 tx=0x7f70c8003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4c1c5fac87cdfa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:08.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.614+0000 7f70b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:08.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.614+0000 7f70b77fe640 1 -- 172.21.15.116:0/2308657841 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c8040040 con 0x55c697d63240 2024-09-12T03:13:08.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.614+0000 7f70b77fe640 1 -- 172.21.15.116:0/2308657841 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f70c80319f0 con 0x55c697d63240 2024-09-12T03:13:08.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.614+0000 7f70e06dd740 1 -- 172.21.15.116:0/2308657841 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70c0005500 con 0x55c697d63240 2024-09-12T03:13:08.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.614+0000 7f70b77fe640 1 -- 172.21.15.116:0/2308657841 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f70c80035f0 con 0x55c697d63240 2024-09-12T03:13:08.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.614+0000 7f70e06dd740 10 client.18138 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:08.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.614+0000 7f70e06dd740 1 -- 172.21.15.116:0/2308657841 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c697d77fb0 con 0x55c697d63240 2024-09-12T03:13:08.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.615+0000 7f70b77fe640 1 -- 172.21.15.116:0/2308657841 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70c804d070 con 0x55c697d63240 2024-09-12T03:13:08.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.615+0000 7f70b77fe640 1 -- 172.21.15.116:0/2308657841 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f70c8054660 con 0x55c697d63240 2024-09-12T03:13:08.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.615+0000 7f70e06dd740 10 client.18138 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:08.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.615+0000 7f70e06dd740 10 client.18138 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:13:08.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.615+0000 7f70e06dd740 20 client.18138 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:08.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.615+0000 7f70e06dd740 1 --2- 172.21.15.116:0/2308657841 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c697d787b0 0x55c697de57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:08.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.615+0000 7f70e06dd740 4 client.18138 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6c5db9e6", "release": true, "await": true}] 2024-09-12T03:13:08.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.615+0000 7f70e06dd740 1 -- 172.21.15.116:0/2308657841 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6c5db9e6", "release": true, "await": true}) -- 0x7f70c0005500 con 0x55c697d787b0 2024-09-12T03:13:08.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.615+0000 7f70cffff640 1 --2- 172.21.15.116:0/2308657841 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c697d787b0 0x55c697de57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:08.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.616+0000 7f70b77fe640 10 client.18138 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:08.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.616+0000 7f70cffff640 1 --2- 172.21.15.116:0/2308657841 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c697d787b0 0x55c697de57b0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=325263b334d621a8 server_cookie=a42b045477a93383 in_seq=0 out_seq=0 2024-09-12T03:13:08.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.618+0000 7f70b77fe640 1 -- 172.21.15.116:0/2308657841 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f70c0005500 con 0x55c697d787b0 2024-09-12T03:13:08.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.618+0000 7f70b77fe640 10 client.18138 handle_command_reply: tid=0 2024-09-12T03:13:08.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.619+0000 7f70e06dd740 1 client.18138 shutdown 2024-09-12T03:13:08.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.619+0000 7f70e06dd740 1 -- 172.21.15.116:0/2308657841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c697d63240 msgr2=0x55c697dd8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:08.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.619+0000 7f70e06dd740 1 --2- 172.21.15.116:0/2308657841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c697d63240 0x55c697dd8dc0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f70c8004720 tx=0x7f70c8003f00 comp rx=0 tx=0).stop 2024-09-12T03:13:08.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.619+0000 7f70e06dd740 1 -- 172.21.15.116:0/2308657841 shutdown_connections 2024-09-12T03:13:08.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.619+0000 7f70e06dd740 1 --2- 172.21.15.116:0/2308657841 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c697d787b0 0x55c697de57b0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.619+0000 7f70e06dd740 1 --2- 172.21.15.116:0/2308657841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c697dd6de0 0x55c697dd7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.619+0000 7f70b77fe640 0 client.18138 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:08.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.619+0000 7f70e06dd740 1 --2- 172.21.15.116:0/2308657841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c697d63240 0x55c697dd8dc0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.619+0000 7f70e06dd740 1 --2- 172.21.15.116:0/2308657841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c697d58980 0x55c697dd8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.619+0000 7f70e06dd740 1 -- 172.21.15.116:0/2308657841 >> 172.21.15.116:0/2308657841 conn(0x55c697d545f0 msgr2=0x55c697d6e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:08.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.619+0000 7f70e06dd740 1 -- 172.21.15.116:0/2308657841 shutdown_connections 2024-09-12T03:13:08.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.619+0000 7f70e06dd740 1 -- 172.21.15.116:0/2308657841 wait complete. 2024-09-12T03:13:08.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.619+0000 7f70e06dd740 20 client.18138 trim_cache size 0 max 16384 2024-09-12T03:13:08.626 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:13:08.626 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 25, 2024-09-12T03:13:08.626 INFO:teuthology.orchestra.run.smithi116.stdout: "leader": 24494, 2024-09-12T03:13:08.626 INFO:teuthology.orchestra.run.smithi116.stdout: "set_version": 40, 2024-09-12T03:13:08.626 INFO:teuthology.orchestra.run.smithi116.stdout: "sets": { 2024-09-12T03:13:08.627 INFO:teuthology.orchestra.run.smithi116.stdout: "6c5db9e6": { 2024-09-12T03:13:08.627 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 40, 2024-09-12T03:13:08.627 INFO:teuthology.orchestra.run.smithi116.stdout: "age_ref": 0.0, 2024-09-12T03:13:08.627 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:13:08.627 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RELEASED", 2024-09-12T03:13:08.627 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:13:08.627 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:13:08.627 INFO:teuthology.orchestra.run.smithi116.stdout: "timeout": 90.0, 2024-09-12T03:13:08.627 INFO:teuthology.orchestra.run.smithi116.stdout: "expiration": 142.6, 2024-09-12T03:13:08.627 INFO:teuthology.orchestra.run.smithi116.stdout: "members": { 2024-09-12T03:13:08.627 INFO:teuthology.orchestra.run.smithi116.stdout: "file:/": { 2024-09-12T03:13:08.627 INFO:teuthology.orchestra.run.smithi116.stdout: "excluded": false, 2024-09-12T03:13:08.627 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:13:08.628 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RELEASED", 2024-09-12T03:13:08.628 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:13:08.628 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:13:08.628 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:13:08.628 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:13:08.628 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:13:08.628 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:13:08.628 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:13:08.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.620+0000 7f70de304640 1 -- 172.21.15.116:0/735199544 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f70c40a2c00 msgr2=0x7f70c40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:08.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.620+0000 7f70de304640 1 --2- 172.21.15.116:0/735199544 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f70c40a2c00 0x7f70c40a50c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f70d00a40b0 tx=0x7f70d0066b40 comp rx=0 tx=0).stop 2024-09-12T03:13:08.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.620+0000 7f70de304640 1 -- 172.21.15.116:0/735199544 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70d81082a0 msgr2=0x7f70d819ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:08.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.620+0000 7f70de304640 1 --2- 172.21.15.116:0/735199544 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70d81082a0 0x7f70d819ce50 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f70d000eaa0 tx=0x7f70d000ef60 comp rx=0 tx=0).stop 2024-09-12T03:13:08.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.620+0000 7f70de304640 1 -- 172.21.15.116:0/735199544 shutdown_connections 2024-09-12T03:13:08.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.620+0000 7f70de304640 1 --2- 172.21.15.116:0/735199544 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f70c40a2c00 0x7f70c40a50c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.620+0000 7f70de304640 1 --2- 172.21.15.116:0/735199544 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70d81082a0 0x7f70d819ce50 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.620+0000 7f70de304640 1 --2- 172.21.15.116:0/735199544 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70d81036e0 0x7f70d8198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.620+0000 7f70de304640 1 --2- 172.21.15.116:0/735199544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70d8102d30 0x7f70d8198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:08.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.620+0000 7f70de304640 1 -- 172.21.15.116:0/735199544 >> 172.21.15.116:0/735199544 conn(0x7f70d80fc7d0 msgr2=0x7f70d8105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:08.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.620+0000 7f70de304640 1 -- 172.21.15.116:0/735199544 shutdown_connections 2024-09-12T03:13:08.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:08.620+0000 7f70de304640 1 -- 172.21.15.116:0/735199544 wait complete. 2024-09-12T03:13:08.632 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:13:08.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:08 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:13:08.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:08 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:13:08.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:08 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:13:08.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:08 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:13:08.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:08 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:13:08.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:08 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:13:08.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:08 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:13:08.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:08 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:13:09.496 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6c5db9e6', total seconds elapsed: 27.0 2024-09-12T03:13:09.496 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.55 2024-09-12T03:13:09.496 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:13:09.749 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:09 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:13:09.749 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:09 smithi116 ceph-mon[31410]: pgmap v643: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-12T03:13:09.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:09 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:13:09.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:09 smithi183 ceph-mon[35535]: pgmap v643: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-12T03:13:09.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:09 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:13:09.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:09 smithi177 ceph-mon[35305]: pgmap v643: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-12T03:13:10.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.983+0000 7fbf09b43640 1 -- 172.21.15.116:0/3482460639 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf04103660 msgr2=0x7fbf0410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:10.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.983+0000 7fbf09b43640 1 --2- 172.21.15.116:0/3482460639 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf04103660 0x7fbf0410bb50 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fbef8009900 tx=0x7fbef802a480 comp rx=0 tx=0).stop 2024-09-12T03:13:10.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.983+0000 7fbf09b43640 1 -- 172.21.15.116:0/3482460639 shutdown_connections 2024-09-12T03:13:10.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.983+0000 7fbf09b43640 1 --2- 172.21.15.116:0/3482460639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf0410c170 0x7fbf0410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.983+0000 7fbf09b43640 1 --2- 172.21.15.116:0/3482460639 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf04103660 0x7fbf0410bb50 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.983+0000 7fbf09b43640 1 --2- 172.21.15.116:0/3482460639 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf04102d40 0x7fbf04103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.983+0000 7fbf09b43640 1 -- 172.21.15.116:0/3482460639 >> 172.21.15.116:0/3482460639 conn(0x7fbf040fc7a0 msgr2=0x7fbf040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:10.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.983+0000 7fbf09b43640 1 -- 172.21.15.116:0/3482460639 shutdown_connections 2024-09-12T03:13:10.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.983+0000 7fbf09b43640 1 -- 172.21.15.116:0/3482460639 wait complete. 2024-09-12T03:13:10.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.984+0000 7fbf09b43640 1 Processor -- start 2024-09-12T03:13:10.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.984+0000 7fbf09b43640 1 -- start start 2024-09-12T03:13:10.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.985+0000 7fbf09b43640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf04102d40 0x7fbf04198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:10.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.985+0000 7fbf09b43640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf04103660 0x7fbf041989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:10.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.985+0000 7fbf09b43640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf0410c170 0x7fbf0419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:10.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.985+0000 7fbf09b43640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbf04111f70 con 0x7fbf0410c170 2024-09-12T03:13:10.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.985+0000 7fbf09b43640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbf04111df0 con 0x7fbf04103660 2024-09-12T03:13:10.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.985+0000 7fbf09b43640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041120f0 con 0x7fbf04102d40 2024-09-12T03:13:10.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.985+0000 7fbf03fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf0410c170 0x7fbf0419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:10.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.985+0000 7fbf03fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf0410c170 0x7fbf0419cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34824/0 (socket says 172.21.15.116:34824) 2024-09-12T03:13:10.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.985+0000 7fbf03fff640 1 -- 172.21.15.116:0/3975975254 learned_addr learned my addr 172.21.15.116:0/3975975254 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:10.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.985+0000 7fbf02ffd640 1 --2- 172.21.15.116:0/3975975254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf04103660 0x7fbf041989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:10.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.985+0000 7fbf037fe640 1 --2- 172.21.15.116:0/3975975254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf04102d40 0x7fbf04198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:10.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.985+0000 7fbf03fff640 1 -- 172.21.15.116:0/3975975254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf04102d40 msgr2=0x7fbf04198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:10.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.985+0000 7fbf03fff640 1 --2- 172.21.15.116:0/3975975254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf04102d40 0x7fbf04198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.985+0000 7fbf03fff640 1 -- 172.21.15.116:0/3975975254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf04103660 msgr2=0x7fbf041989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:10.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.985+0000 7fbf03fff640 1 --2- 172.21.15.116:0/3975975254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf04103660 0x7fbf041989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.985+0000 7fbf03fff640 1 -- 172.21.15.116:0/3975975254 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0419d420 con 0x7fbf0410c170 2024-09-12T03:13:10.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.985+0000 7fbf037fe640 1 --2- 172.21.15.116:0/3975975254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf04102d40 0x7fbf04198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:13:10.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.985+0000 7fbf03fff640 1 --2- 172.21.15.116:0/3975975254 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf0410c170 0x7fbf0419cd40 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00e970 tx=0x7fbefc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:10.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.985+0000 7fbf00ff9640 1 -- 172.21.15.116:0/3975975254 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc00ccc0 con 0x7fbf0410c170 2024-09-12T03:13:10.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.985+0000 7fbf00ff9640 1 -- 172.21.15.116:0/3975975254 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbefc00ce60 con 0x7fbf0410c170 2024-09-12T03:13:10.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.985+0000 7fbf00ff9640 1 -- 172.21.15.116:0/3975975254 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc010490 con 0x7fbf0410c170 2024-09-12T03:13:10.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.985+0000 7fbf09b43640 1 -- 172.21.15.116:0/3975975254 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf0419d710 con 0x7fbf0410c170 2024-09-12T03:13:10.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.986+0000 7fbf09b43640 1 -- 172.21.15.116:0/3975975254 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf041a4ff0 con 0x7fbf0410c170 2024-09-12T03:13:10.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.988+0000 7fbf00ff9640 1 -- 172.21.15.116:0/3975975254 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fbefc005020 con 0x7fbf0410c170 2024-09-12T03:13:10.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.988+0000 7fbf0bf1c740 1 Processor -- start 2024-09-12T03:13:10.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.988+0000 7fbf0bf1c740 1 -- start start 2024-09-12T03:13:10.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.988+0000 7fbf0bf1c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56431a3ba360 0x56431a3ba740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:10.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.988+0000 7fbf0bf1c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56431a3bac80 0x56431a3c9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:10.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.988+0000 7fbf0bf1c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56431a3c9a90 0x56431a3cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:10.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.988+0000 7fbf0bf1c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56431a3202e0 con 0x56431a3ba360 2024-09-12T03:13:10.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.988+0000 7fbf0bf1c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56431a2c7ce0 con 0x56431a3bac80 2024-09-12T03:13:10.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.988+0000 7fbf0bf1c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56431a31c2a0 con 0x56431a3c9a90 2024-09-12T03:13:10.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.988+0000 7fbf00ff9640 1 --2- 172.21.15.116:0/3975975254 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbeec0a2b80 0x7fbeec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:10.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.988+0000 7fbf02ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56431a3ba360 0x56431a3ba740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:10.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.988+0000 7fbf02ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56431a3ba360 0x56431a3ba740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34834/0 (socket says 172.21.15.116:34834) 2024-09-12T03:13:10.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.989+0000 7fbf02ffd640 1 -- 172.21.15.116:0/4133211783 learned_addr learned my addr 172.21.15.116:0/4133211783 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:10.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.989+0000 7fbf037fe640 1 --2- 172.21.15.116:0/4133211783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56431a3bac80 0x56431a3c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:10.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.989+0000 7fbf03fff640 1 --2- 172.21.15.116:0/3975975254 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbeec0a2b80 0x7fbeec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:10.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.989+0000 7fbf02ffd640 1 --2- 172.21.15.116:0/4133211783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56431a3c9a90 0x56431a3cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:10.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.989+0000 7fbf00ff9640 1 -- 172.21.15.116:0/3975975254 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fbefc0cf4f0 con 0x7fbf0410c170 2024-09-12T03:13:10.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.989+0000 7fbf02ffd640 1 -- 172.21.15.116:0/4133211783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56431a3c9a90 msgr2=0x56431a3cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:10.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.989+0000 7fbf02ffd640 1 --2- 172.21.15.116:0/4133211783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56431a3c9a90 0x56431a3cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.989+0000 7fbf02ffd640 1 -- 172.21.15.116:0/4133211783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56431a3bac80 msgr2=0x56431a3c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:10.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.989+0000 7fbf02ffd640 1 --2- 172.21.15.116:0/4133211783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56431a3bac80 0x56431a3c9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.989+0000 7fbf02ffd640 1 -- 172.21.15.116:0/4133211783 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56431a2ac440 con 0x56431a3ba360 2024-09-12T03:13:10.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.989+0000 7fbf02ffd640 1 --2- 172.21.15.116:0/4133211783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56431a3ba360 0x56431a3ba740 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fbef802a9a0 tx=0x7fbef8005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f954855cd2b0f5c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:10.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.989+0000 7fbf03fff640 1 --2- 172.21.15.116:0/3975975254 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbeec0a2b80 0x7fbeec0a5040 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbefc09ba80 tx=0x7fbefc017fc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:10.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.989+0000 7fbee9ffb640 1 -- 172.21.15.116:0/4133211783 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef8035050 con 0x56431a3ba360 2024-09-12T03:13:10.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.989+0000 7fbee9ffb640 1 -- 172.21.15.116:0/4133211783 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbef8004030 con 0x56431a3ba360 2024-09-12T03:13:10.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.989+0000 7fbee9ffb640 1 -- 172.21.15.116:0/4133211783 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef80055e0 con 0x56431a3ba360 2024-09-12T03:13:10.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.990+0000 7fbf0bf1c740 1 -- 172.21.15.116:0/4133211783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56431a3ba360 msgr2=0x56431a3ba740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:10.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.990+0000 7fbf0bf1c740 1 --2- 172.21.15.116:0/4133211783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56431a3ba360 0x56431a3ba740 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fbef802a9a0 tx=0x7fbef8005b50 comp rx=0 tx=0).stop 2024-09-12T03:13:10.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.990+0000 7fbf0bf1c740 1 -- 172.21.15.116:0/4133211783 shutdown_connections 2024-09-12T03:13:10.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.990+0000 7fbf0bf1c740 1 --2- 172.21.15.116:0/4133211783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56431a3c9a90 0x56431a3cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.990+0000 7fbf0bf1c740 1 --2- 172.21.15.116:0/4133211783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56431a3bac80 0x56431a3c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.990+0000 7fbf0bf1c740 1 --2- 172.21.15.116:0/4133211783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56431a3ba360 0x56431a3ba740 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.990+0000 7fbf0bf1c740 1 -- 172.21.15.116:0/4133211783 >> 172.21.15.116:0/4133211783 conn(0x56431a3b6a30 msgr2=0x56431a3b6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:10.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.990+0000 7fbf0bf1c740 1 -- 172.21.15.116:0/4133211783 shutdown_connections 2024-09-12T03:13:10.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.990+0000 7fbf0bf1c740 1 -- 172.21.15.116:0/4133211783 wait complete. 2024-09-12T03:13:10.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.990+0000 7fbf0bf1c740 1 -- start start 2024-09-12T03:13:10.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.991+0000 7fbf0bf1c740 1 Processor -- start 2024-09-12T03:13:10.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.991+0000 7fbf0bf1c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56431a3ba360 0x56431a3c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:10.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.991+0000 7fbf0bf1c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56431a3bac80 0x56431a3c6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:10.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.991+0000 7fbf0bf1c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56431a3c9a90 0x56431a3c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:10.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.991+0000 7fbf0bf1c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56431a3cc600 con 0x56431a3ba360 2024-09-12T03:13:10.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.991+0000 7fbf0bf1c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56431a2db050 con 0x56431a3bac80 2024-09-12T03:13:10.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.991+0000 7fbf0bf1c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56431a3cc780 con 0x56431a3c9a90 2024-09-12T03:13:10.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.991+0000 7fbf0bf1c740 10 client.0 _ll_register_callbacks cb 0x56431a31b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:10.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.991+0000 7fbf02ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56431a3ba360 0x56431a3c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:10.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.991+0000 7fbf02ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56431a3ba360 0x56431a3c93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34838/0 (socket says 172.21.15.116:34838) 2024-09-12T03:13:10.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.991+0000 7fbf02ffd640 1 -- 172.21.15.116:0/3564700290 learned_addr learned my addr 172.21.15.116:0/3564700290 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:10.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.991+0000 7fbf02ffd640 1 -- 172.21.15.116:0/3564700290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56431a3c9a90 msgr2=0x56431a3c6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:10.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.991+0000 7fbf02ffd640 1 --2- 172.21.15.116:0/3564700290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56431a3c9a90 0x56431a3c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.991+0000 7fbf02ffd640 1 -- 172.21.15.116:0/3564700290 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56431a3bac80 msgr2=0x56431a3c6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:10.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.991+0000 7fbf037fe640 1 --2- 172.21.15.116:0/3564700290 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56431a3bac80 0x56431a3c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:10.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.991+0000 7fbf02ffd640 1 --2- 172.21.15.116:0/3564700290 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56431a3bac80 0x56431a3c6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.991+0000 7fbf02ffd640 1 -- 172.21.15.116:0/3564700290 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56431a0913c0 con 0x56431a3ba360 2024-09-12T03:13:10.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.992+0000 7fbf02ffd640 1 --2- 172.21.15.116:0/3564700290 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56431a3ba360 0x56431a3c93b0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fbef802abc0 tx=0x7fbef80041e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b627b949fb4f356d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:10.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.992+0000 7fbee2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:10.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.992+0000 7fbee2ffd640 1 -- 172.21.15.116:0/3564700290 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef8035040 con 0x56431a3ba360 2024-09-12T03:13:10.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.992+0000 7fbee2ffd640 1 -- 172.21.15.116:0/3564700290 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fbef8031630 con 0x56431a3ba360 2024-09-12T03:13:10.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.992+0000 7fbf0bf1c740 1 -- 172.21.15.116:0/3564700290 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbef8004030 con 0x56431a3ba360 2024-09-12T03:13:10.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.992+0000 7fbee2ffd640 1 -- 172.21.15.116:0/3564700290 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbef8031830 con 0x56431a3ba360 2024-09-12T03:13:10.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.992+0000 7fbf0bf1c740 10 client.18150 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:10.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.992+0000 7fbf0bf1c740 1 -- 172.21.15.116:0/3564700290 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56431a3d7ff0 con 0x56431a3ba360 2024-09-12T03:13:10.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.992+0000 7fbee2ffd640 1 -- 172.21.15.116:0/3564700290 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbef8004030 con 0x56431a3ba360 2024-09-12T03:13:10.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.992+0000 7fbee2ffd640 1 -- 172.21.15.116:0/3564700290 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbef80047e0 con 0x56431a3ba360 2024-09-12T03:13:10.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.992+0000 7fbf0bf1c740 10 client.18150 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:10.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.992+0000 7fbf0bf1c740 10 client.18150 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:13:10.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.992+0000 7fbf0bf1c740 20 client.18150 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:10.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.992+0000 7fbf0bf1c740 1 --2- 172.21.15.116:0/3564700290 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56431a3d9340 0x56431a3db730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:10.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.992+0000 7fbf0bf1c740 4 client.18150 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:13:10.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.992+0000 7fbf0bf1c740 1 -- 172.21.15.116:0/3564700290 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56431a32d850 con 0x56431a3d9340 2024-09-12T03:13:10.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.993+0000 7fbf037fe640 1 --2- 172.21.15.116:0/3564700290 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56431a3d9340 0x56431a3db730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:10.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.993+0000 7fbf037fe640 1 --2- 172.21.15.116:0/3564700290 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56431a3d9340 0x56431a3db730 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=320d118827c05ff3 server_cookie=16b17b77617c2ff9 in_seq=0 out_seq=0 2024-09-12T03:13:10.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.993+0000 7fbee2ffd640 10 client.18150 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:10.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.994+0000 7fbee2ffd640 1 -- 172.21.15.116:0/3564700290 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56431a32d850 con 0x56431a3d9340 2024-09-12T03:13:10.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.994+0000 7fbee2ffd640 10 client.18150 handle_command_reply: tid=0 2024-09-12T03:13:10.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.994+0000 7fbf0bf1c740 1 client.18150 shutdown 2024-09-12T03:13:10.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.994+0000 7fbf0bf1c740 1 -- 172.21.15.116:0/3564700290 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56431a3ba360 msgr2=0x56431a3c93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:10.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.994+0000 7fbf0bf1c740 1 --2- 172.21.15.116:0/3564700290 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56431a3ba360 0x56431a3c93b0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fbef802abc0 tx=0x7fbef80041e0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.994+0000 7fbf0bf1c740 1 -- 172.21.15.116:0/3564700290 shutdown_connections 2024-09-12T03:13:10.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.994+0000 7fbf0bf1c740 1 --2- 172.21.15.116:0/3564700290 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56431a3d9340 0x56431a3db730 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.994+0000 7fbf0bf1c740 1 --2- 172.21.15.116:0/3564700290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56431a3c9a90 0x56431a3c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.994+0000 7fbf0bf1c740 1 --2- 172.21.15.116:0/3564700290 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56431a3bac80 0x56431a3c6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.994+0000 7fbf0bf1c740 1 --2- 172.21.15.116:0/3564700290 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56431a3ba360 0x56431a3c93b0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.994+0000 7fbf0bf1c740 1 -- 172.21.15.116:0/3564700290 >> 172.21.15.116:0/3564700290 conn(0x56431a3b6a30 msgr2=0x56431a3b5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:10.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.994+0000 7fbee2ffd640 0 client.18150 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:10.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.994+0000 7fbf0bf1c740 1 -- 172.21.15.116:0/3564700290 shutdown_connections 2024-09-12T03:13:10.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.994+0000 7fbf0bf1c740 1 -- 172.21.15.116:0/3564700290 wait complete. 2024-09-12T03:13:10.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:09.994+0000 7fbf0bf1c740 20 client.18150 trim_cache size 0 max 16384 2024-09-12T03:13:10.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.012+0000 7fbf0bf1c740 1 Processor -- start 2024-09-12T03:13:10.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.012+0000 7fbf0bf1c740 1 -- start start 2024-09-12T03:13:10.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.012+0000 7fbf0bf1c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56431a3ba360 0x56431a3de370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:10.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.013+0000 7fbf0bf1c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56431a3bac80 0x56431a3de8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:10.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.013+0000 7fbf0bf1c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56431a3c9a90 0x56431a3c0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:10.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.013+0000 7fbf0bf1c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041adb40 con 0x56431a3ba360 2024-09-12T03:13:10.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.013+0000 7fbf0bf1c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041ab410 con 0x56431a3c9a90 2024-09-12T03:13:10.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.013+0000 7fbf0bf1c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041a9260 con 0x56431a3bac80 2024-09-12T03:13:10.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.013+0000 7fbf02ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56431a3ba360 0x56431a3de370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:10.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.013+0000 7fbf02ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56431a3ba360 0x56431a3de370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34850/0 (socket says 172.21.15.116:34850) 2024-09-12T03:13:10.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.013+0000 7fbf02ffd640 1 -- 172.21.15.116:0/3528246358 learned_addr learned my addr 172.21.15.116:0/3528246358 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:10.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.013+0000 7fbf037fe640 1 --2- 172.21.15.116:0/3528246358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56431a3bac80 0x56431a3de8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:10.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.013+0000 7fbf02ffd640 1 --2- 172.21.15.116:0/3528246358 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56431a3c9a90 0x56431a3c0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:10.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.013+0000 7fbf02ffd640 1 -- 172.21.15.116:0/3528246358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56431a3bac80 msgr2=0x56431a3de8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:10.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.013+0000 7fbf02ffd640 1 --2- 172.21.15.116:0/3528246358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56431a3bac80 0x56431a3de8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.013+0000 7fbf02ffd640 1 -- 172.21.15.116:0/3528246358 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56431a3c9a90 msgr2=0x56431a3c0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:10.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.013+0000 7fbf02ffd640 1 --2- 172.21.15.116:0/3528246358 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56431a3c9a90 0x56431a3c0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.013+0000 7fbf02ffd640 1 -- 172.21.15.116:0/3528246358 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56431a3c4bb0 con 0x56431a3ba360 2024-09-12T03:13:10.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.013+0000 7fbf02ffd640 1 --2- 172.21.15.116:0/3528246358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56431a3ba360 0x56431a3de370 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fbef8001eb0 tx=0x7fbef802ab00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b124bc1900e3771 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:10.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.013+0000 7fbee97fa640 1 -- 172.21.15.116:0/3528246358 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef8035040 con 0x56431a3ba360 2024-09-12T03:13:10.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.013+0000 7fbee97fa640 1 -- 172.21.15.116:0/3528246358 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbef804e070 con 0x56431a3ba360 2024-09-12T03:13:10.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.013+0000 7fbee97fa640 1 -- 172.21.15.116:0/3528246358 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef8004920 con 0x56431a3ba360 2024-09-12T03:13:10.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.014+0000 7fbf0bf1c740 1 -- 172.21.15.116:0/3528246358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56431a3ba360 msgr2=0x56431a3de370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:10.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.014+0000 7fbf0bf1c740 1 --2- 172.21.15.116:0/3528246358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56431a3ba360 0x56431a3de370 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fbef8001eb0 tx=0x7fbef802ab00 comp rx=0 tx=0).stop 2024-09-12T03:13:10.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.014+0000 7fbf0bf1c740 1 -- 172.21.15.116:0/3528246358 shutdown_connections 2024-09-12T03:13:10.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.014+0000 7fbf0bf1c740 1 --2- 172.21.15.116:0/3528246358 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56431a3c9a90 0x56431a3c0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.014+0000 7fbf0bf1c740 1 --2- 172.21.15.116:0/3528246358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56431a3bac80 0x56431a3de8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.014+0000 7fbf0bf1c740 1 --2- 172.21.15.116:0/3528246358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56431a3ba360 0x56431a3de370 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.014+0000 7fbf0bf1c740 1 -- 172.21.15.116:0/3528246358 >> 172.21.15.116:0/3528246358 conn(0x56431a3b6a30 msgr2=0x56431a3c1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:10.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.014+0000 7fbf0bf1c740 1 -- 172.21.15.116:0/3528246358 shutdown_connections 2024-09-12T03:13:10.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.014+0000 7fbf0bf1c740 1 -- 172.21.15.116:0/3528246358 wait complete. 2024-09-12T03:13:10.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.015+0000 7fbf0bf1c740 1 -- start start 2024-09-12T03:13:10.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.015+0000 7fbf0bf1c740 1 Processor -- start 2024-09-12T03:13:10.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.015+0000 7fbf0bf1c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56431a3ba360 0x56431a3c1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:10.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.015+0000 7fbf0bf1c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56431a3bac80 0x56431a3c1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:10.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.015+0000 7fbf0bf1c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56431a3c9a90 0x56431a3de010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:10.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.015+0000 7fbf0bf1c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041a98d0 con 0x56431a3ba360 2024-09-12T03:13:10.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.015+0000 7fbf0bf1c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041aad40 con 0x56431a3c9a90 2024-09-12T03:13:10.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.015+0000 7fbf0bf1c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041a9fa0 con 0x56431a3bac80 2024-09-12T03:13:10.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.015+0000 7fbf0bf1c740 10 client.0 _ll_register_callbacks cb 0x7fbef4009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:10.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.015+0000 7fbf02ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56431a3ba360 0x56431a3c1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:10.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.015+0000 7fbf02ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56431a3ba360 0x56431a3c1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34858/0 (socket says 172.21.15.116:34858) 2024-09-12T03:13:10.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.015+0000 7fbf02ffd640 1 -- 172.21.15.116:0/1211157863 learned_addr learned my addr 172.21.15.116:0/1211157863 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:10.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.016+0000 7fbf037fe640 1 --2- 172.21.15.116:0/1211157863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56431a3bac80 0x56431a3c1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:10.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.016+0000 7fbf02ffd640 1 --2- 172.21.15.116:0/1211157863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56431a3c9a90 0x56431a3de010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:10.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.016+0000 7fbf02ffd640 1 -- 172.21.15.116:0/1211157863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56431a3bac80 msgr2=0x56431a3c1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:10.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.016+0000 7fbf02ffd640 1 --2- 172.21.15.116:0/1211157863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56431a3bac80 0x56431a3c1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.016+0000 7fbf02ffd640 1 -- 172.21.15.116:0/1211157863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56431a3c9a90 msgr2=0x56431a3de010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:10.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.016+0000 7fbf02ffd640 1 --2- 172.21.15.116:0/1211157863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56431a3c9a90 0x56431a3de010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.016+0000 7fbf02ffd640 1 -- 172.21.15.116:0/1211157863 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56431a3de7e0 con 0x56431a3ba360 2024-09-12T03:13:10.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.016+0000 7fbf037fe640 1 --2- 172.21.15.116:0/1211157863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56431a3bac80 0x56431a3c1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:13:10.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.016+0000 7fbf02ffd640 1 --2- 172.21.15.116:0/1211157863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56431a3ba360 0x56431a3c1880 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fbef802a9a0 tx=0x7fbef8009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc6fbe89a6dccf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:10.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.016+0000 7fbee2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:10.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.016+0000 7fbee2ffd640 1 -- 172.21.15.116:0/1211157863 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef8035070 con 0x56431a3ba360 2024-09-12T03:13:10.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.016+0000 7fbf0bf1c740 1 -- 172.21.15.116:0/1211157863 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbef804e070 con 0x56431a3ba360 2024-09-12T03:13:10.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.016+0000 7fbee2ffd640 1 -- 172.21.15.116:0/1211157863 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fbef8049550 con 0x56431a3ba360 2024-09-12T03:13:10.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.016+0000 7fbee2ffd640 1 -- 172.21.15.116:0/1211157863 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbef8049750 con 0x56431a3ba360 2024-09-12T03:13:10.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.016+0000 7fbf0bf1c740 10 client.18156 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:10.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.016+0000 7fbf0bf1c740 1 -- 172.21.15.116:0/1211157863 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56431a3bff00 con 0x56431a3ba360 2024-09-12T03:13:10.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.017+0000 7fbee2ffd640 1 -- 172.21.15.116:0/1211157863 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbef804e070 con 0x56431a3ba360 2024-09-12T03:13:10.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.017+0000 7fbee2ffd640 1 -- 172.21.15.116:0/1211157863 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbef803a750 con 0x56431a3ba360 2024-09-12T03:13:10.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.017+0000 7fbf0bf1c740 10 client.18156 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:10.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.017+0000 7fbf0bf1c740 10 client.18156 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:13:10.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.017+0000 7fbf0bf1c740 20 client.18156 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:10.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.017+0000 7fbf0bf1c740 1 --2- 172.21.15.116:0/1211157863 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56431a448460 0x56431a44a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:10.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.017+0000 7fbf0bf1c740 4 client.18156 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:13:10.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.017+0000 7fbf0bf1c740 1 -- 172.21.15.116:0/1211157863 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56431a327d60 con 0x56431a448460 2024-09-12T03:13:10.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.017+0000 7fbf037fe640 1 --2- 172.21.15.116:0/1211157863 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56431a448460 0x56431a44a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:10.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.017+0000 7fbee2ffd640 10 client.18156 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:10.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.017+0000 7fbf037fe640 1 --2- 172.21.15.116:0/1211157863 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56431a448460 0x56431a44a850 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab4fe701e851897e server_cookie=f53f5b4e500f6b2 in_seq=0 out_seq=0 2024-09-12T03:13:10.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.018+0000 7fbee2ffd640 1 -- 172.21.15.116:0/1211157863 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56431a327d60 con 0x56431a448460 2024-09-12T03:13:10.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.018+0000 7fbee2ffd640 10 client.18156 handle_command_reply: tid=0 2024-09-12T03:13:10.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.018+0000 7fbf0bf1c740 1 client.18156 shutdown 2024-09-12T03:13:10.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.018+0000 7fbf0bf1c740 1 -- 172.21.15.116:0/1211157863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56431a3ba360 msgr2=0x56431a3c1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:10.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.018+0000 7fbf0bf1c740 1 --2- 172.21.15.116:0/1211157863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56431a3ba360 0x56431a3c1880 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fbef802a9a0 tx=0x7fbef8009630 comp rx=0 tx=0).stop 2024-09-12T03:13:10.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.018+0000 7fbf0bf1c740 1 -- 172.21.15.116:0/1211157863 shutdown_connections 2024-09-12T03:13:10.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.018+0000 7fbf0bf1c740 1 --2- 172.21.15.116:0/1211157863 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56431a448460 0x56431a44a850 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.018+0000 7fbf0bf1c740 1 --2- 172.21.15.116:0/1211157863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56431a3c9a90 0x56431a3de010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.018+0000 7fbf0bf1c740 1 --2- 172.21.15.116:0/1211157863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56431a3bac80 0x56431a3c1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.018+0000 7fbf0bf1c740 1 --2- 172.21.15.116:0/1211157863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56431a3ba360 0x56431a3c1880 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.018+0000 7fbf0bf1c740 1 -- 172.21.15.116:0/1211157863 >> 172.21.15.116:0/1211157863 conn(0x56431a3b6a30 msgr2=0x56431a3b5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:10.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.018+0000 7fbee2ffd640 0 client.18156 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:10.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.018+0000 7fbf0bf1c740 1 -- 172.21.15.116:0/1211157863 shutdown_connections 2024-09-12T03:13:10.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.018+0000 7fbf0bf1c740 1 -- 172.21.15.116:0/1211157863 wait complete. 2024-09-12T03:13:10.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.018+0000 7fbf0bf1c740 20 client.18156 trim_cache size 0 max 16384 2024-09-12T03:13:10.024 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:13:10.024 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:13:10.024 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:13:10.024 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:13:10.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.019+0000 7fbf09b43640 1 -- 172.21.15.116:0/3975975254 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbeec0a2b80 msgr2=0x7fbeec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:10.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.019+0000 7fbf09b43640 1 --2- 172.21.15.116:0/3975975254 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbeec0a2b80 0x7fbeec0a5040 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbefc09ba80 tx=0x7fbefc017fc0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.019+0000 7fbf09b43640 1 -- 172.21.15.116:0/3975975254 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf0410c170 msgr2=0x7fbf0419cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:10.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.019+0000 7fbf09b43640 1 --2- 172.21.15.116:0/3975975254 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf0410c170 0x7fbf0419cd40 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00e970 tx=0x7fbefc00ed70 comp rx=0 tx=0).stop 2024-09-12T03:13:10.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.019+0000 7fbf09b43640 1 -- 172.21.15.116:0/3975975254 shutdown_connections 2024-09-12T03:13:10.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.019+0000 7fbf09b43640 1 --2- 172.21.15.116:0/3975975254 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbeec0a2b80 0x7fbeec0a5040 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.019+0000 7fbf09b43640 1 --2- 172.21.15.116:0/3975975254 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf0410c170 0x7fbf0419cd40 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.019+0000 7fbf09b43640 1 --2- 172.21.15.116:0/3975975254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf04103660 0x7fbf041989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.019+0000 7fbf09b43640 1 --2- 172.21.15.116:0/3975975254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf04102d40 0x7fbf04198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:10.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.019+0000 7fbf09b43640 1 -- 172.21.15.116:0/3975975254 >> 172.21.15.116:0/3975975254 conn(0x7fbf040fc7a0 msgr2=0x7fbf040fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:10.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.019+0000 7fbf09b43640 1 -- 172.21.15.116:0/3975975254 shutdown_connections 2024-09-12T03:13:10.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:10.019+0000 7fbf09b43640 1 -- 172.21.15.116:0/3975975254 wait complete. 2024-09-12T03:13:10.218 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:13:10.796 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:13:10.796 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:13:11.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.256+0000 7f644d610640 1 -- 172.21.15.116:0/1878084272 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6448102d40 msgr2=0x7f6448103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:11.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.256+0000 7f644d610640 1 --2- 172.21.15.116:0/1878084272 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6448102d40 0x7f6448103120 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f6438009900 tx=0x7f643802a480 comp rx=0 tx=0).stop 2024-09-12T03:13:11.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.256+0000 7f644d610640 1 -- 172.21.15.116:0/1878084272 shutdown_connections 2024-09-12T03:13:11.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.256+0000 7f644d610640 1 --2- 172.21.15.116:0/1878084272 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f644810c170 0x7f644810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:11.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.256+0000 7f644d610640 1 --2- 172.21.15.116:0/1878084272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6448103660 0x7f644810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:11.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.256+0000 7f644d610640 1 --2- 172.21.15.116:0/1878084272 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6448102d40 0x7f6448103120 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:11.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.256+0000 7f644d610640 1 -- 172.21.15.116:0/1878084272 >> 172.21.15.116:0/1878084272 conn(0x7f64480fc7a0 msgr2=0x7f64480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:11.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.256+0000 7f644d610640 1 -- 172.21.15.116:0/1878084272 shutdown_connections 2024-09-12T03:13:11.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.256+0000 7f644d610640 1 -- 172.21.15.116:0/1878084272 wait complete. 2024-09-12T03:13:11.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.256+0000 7f644d610640 1 Processor -- start 2024-09-12T03:13:11.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.257+0000 7f644d610640 1 -- start start 2024-09-12T03:13:11.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.257+0000 7f644d610640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6448102d40 0x7f6448198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:11.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.257+0000 7f644d610640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6448103660 0x7f6448198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:11.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.257+0000 7f644d610640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f644810c170 0x7f644819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:11.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.257+0000 7f644d610640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6448111f70 con 0x7f6448102d40 2024-09-12T03:13:11.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.257+0000 7f644d610640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6448111df0 con 0x7f644810c170 2024-09-12T03:13:11.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.257+0000 7f644d610640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f64481120f0 con 0x7f6448103660 2024-09-12T03:13:11.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.257+0000 7f6446ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6448102d40 0x7f6448198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:11.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.257+0000 7f6446ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6448102d40 0x7f6448198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45984/0 (socket says 172.21.15.116:45984) 2024-09-12T03:13:11.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.257+0000 7f6446ffd640 1 -- 172.21.15.116:0/3484041570 learned_addr learned my addr 172.21.15.116:0/3484041570 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:11.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.257+0000 7f64477fe640 1 --2- 172.21.15.116:0/3484041570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f644810c170 0x7f644819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:11.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.257+0000 7f643ffff640 1 --2- 172.21.15.116:0/3484041570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6448103660 0x7f6448198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:11.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.257+0000 7f6446ffd640 1 -- 172.21.15.116:0/3484041570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6448103660 msgr2=0x7f6448198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:11.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.257+0000 7f6446ffd640 1 --2- 172.21.15.116:0/3484041570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6448103660 0x7f6448198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:11.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.257+0000 7f6446ffd640 1 -- 172.21.15.116:0/3484041570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f644810c170 msgr2=0x7f644819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:11.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.257+0000 7f6446ffd640 1 --2- 172.21.15.116:0/3484041570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f644810c170 0x7f644819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:11.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.257+0000 7f6446ffd640 1 -- 172.21.15.116:0/3484041570 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644819d3f0 con 0x7f6448102d40 2024-09-12T03:13:11.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.257+0000 7f6446ffd640 1 --2- 172.21.15.116:0/3484041570 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6448102d40 0x7f6448198440 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f6438005ab0 tx=0x7f64380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:11.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.258+0000 7f6444ff9640 1 -- 172.21.15.116:0/3484041570 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643803f030 con 0x7f6448102d40 2024-09-12T03:13:11.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.258+0000 7f6444ff9640 1 -- 172.21.15.116:0/3484041570 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6438003ce0 con 0x7f6448102d40 2024-09-12T03:13:11.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.258+0000 7f644d610640 1 -- 172.21.15.116:0/3484041570 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644819d680 con 0x7f6448102d40 2024-09-12T03:13:11.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.258+0000 7f6444ff9640 1 -- 172.21.15.116:0/3484041570 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438035070 con 0x7f6448102d40 2024-09-12T03:13:11.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.258+0000 7f644d610640 1 -- 172.21.15.116:0/3484041570 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64481a4fc0 con 0x7f6448102d40 2024-09-12T03:13:11.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.259+0000 7f644d610640 1 -- 172.21.15.116:0/3484041570 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6448103120 con 0x7f6448102d40 2024-09-12T03:13:11.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.260+0000 7f6444ff9640 1 -- 172.21.15.116:0/3484041570 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6438042030 con 0x7f6448102d40 2024-09-12T03:13:11.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.260+0000 7f6444ff9640 1 --2- 172.21.15.116:0/3484041570 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f64340a2c00 0x7f64340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:11.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.261+0000 7f643ffff640 1 --2- 172.21.15.116:0/3484041570 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f64340a2c00 0x7f64340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:11.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.261+0000 7f6444ff9640 1 -- 172.21.15.116:0/3484041570 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f64380ef9f0 con 0x7f6448102d40 2024-09-12T03:13:11.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.261+0000 7f643ffff640 1 --2- 172.21.15.116:0/3484041570 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f64340a2c00 0x7f64340a50c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f64480fedc0 tx=0x7f643000a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:11.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.264+0000 7f6444ff9640 1 -- 172.21.15.116:0/3484041570 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f64380bc330 con 0x7f6448102d40 2024-09-12T03:13:11.314 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:13:11.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.481+0000 7f644d610640 1 -- 172.21.15.116:0/3484041570 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64481997a0 con 0x7f6448102d40 2024-09-12T03:13:11.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.482+0000 7f6444ff9640 1 -- 172.21.15.116:0/3484041570 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f64380c0630 con 0x7f6448102d40 2024-09-12T03:13:11.482 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:13:11.483 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:13:11.483 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:13:11.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.484+0000 7f644d610640 1 -- 172.21.15.116:0/3484041570 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f64340a2c00 msgr2=0x7f64340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:11.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.484+0000 7f644d610640 1 --2- 172.21.15.116:0/3484041570 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f64340a2c00 0x7f64340a50c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f64480fedc0 tx=0x7f643000a250 comp rx=0 tx=0).stop 2024-09-12T03:13:11.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.484+0000 7f644d610640 1 -- 172.21.15.116:0/3484041570 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6448102d40 msgr2=0x7f6448198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:11.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.484+0000 7f644d610640 1 --2- 172.21.15.116:0/3484041570 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6448102d40 0x7f6448198440 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f6438005ab0 tx=0x7f64380042f0 comp rx=0 tx=0).stop 2024-09-12T03:13:11.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.484+0000 7f644d610640 1 -- 172.21.15.116:0/3484041570 shutdown_connections 2024-09-12T03:13:11.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.484+0000 7f644d610640 1 --2- 172.21.15.116:0/3484041570 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f64340a2c00 0x7f64340a50c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:11.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.484+0000 7f644d610640 1 --2- 172.21.15.116:0/3484041570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f644810c170 0x7f644819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:11.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.484+0000 7f644d610640 1 --2- 172.21.15.116:0/3484041570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6448103660 0x7f6448198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:11.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.484+0000 7f644d610640 1 --2- 172.21.15.116:0/3484041570 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6448102d40 0x7f6448198440 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:11.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.484+0000 7f644d610640 1 -- 172.21.15.116:0/3484041570 >> 172.21.15.116:0/3484041570 conn(0x7f64480fc7a0 msgr2=0x7f64480fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:11.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.484+0000 7f644d610640 1 -- 172.21.15.116:0/3484041570 shutdown_connections 2024-09-12T03:13:11.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:11.484+0000 7f644d610640 1 -- 172.21.15.116:0/3484041570 wait complete. 2024-09-12T03:13:11.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:11 smithi116 ceph-mon[31410]: pgmap v644: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-12T03:13:11.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:11 smithi183 ceph-mon[35535]: pgmap v644: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-12T03:13:11.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:11 smithi177 ceph-mon[35305]: pgmap v644: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-12T03:13:12.204 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0' 2024-09-12T03:13:12.204 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0 2024-09-12T03:13:12.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.419+0000 7ffbc517c640 1 -- 172.21.15.116:0/2396067738 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbc0100e20 msgr2=0x7ffbc0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:12.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.419+0000 7ffbc517c640 1 --2- 172.21.15.116:0/2396067738 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbc0100e20 0x7ffbc0101200 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0009900 tx=0x7ffbb002a480 comp rx=0 tx=0).stop 2024-09-12T03:13:12.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.419+0000 7ffbc517c640 1 -- 172.21.15.116:0/2396067738 shutdown_connections 2024-09-12T03:13:12.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.419+0000 7ffbc517c640 1 --2- 172.21.15.116:0/2396067738 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffbc0102ef0 0x7ffbc010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.419+0000 7ffbc517c640 1 --2- 172.21.15.116:0/2396067738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffbc0102570 0x7ffbc01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.419+0000 7ffbc517c640 1 --2- 172.21.15.116:0/2396067738 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbc0100e20 0x7ffbc0101200 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.419+0000 7ffbc517c640 1 -- 172.21.15.116:0/2396067738 >> 172.21.15.116:0/2396067738 conn(0x7ffbc00fc7d0 msgr2=0x7ffbc00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:12.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.419+0000 7ffbc517c640 1 -- 172.21.15.116:0/2396067738 shutdown_connections 2024-09-12T03:13:12.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.419+0000 7ffbc517c640 1 -- 172.21.15.116:0/2396067738 wait complete. 2024-09-12T03:13:12.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.419+0000 7ffbc517c640 1 Processor -- start 2024-09-12T03:13:12.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.420+0000 7ffbc517c640 1 -- start start 2024-09-12T03:13:12.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.420+0000 7ffbc517c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffbc0100e20 0x7ffbc0199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:12.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.420+0000 7ffbc517c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbc0102570 0x7ffbc01998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:12.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.420+0000 7ffbc517c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffbc0102ef0 0x7ffbc01a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:12.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.420+0000 7ffbc517c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc0110f80 con 0x7ffbc0102570 2024-09-12T03:13:12.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.420+0000 7ffbc517c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc0110e00 con 0x7ffbc0102ef0 2024-09-12T03:13:12.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.420+0000 7ffbc517c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc0111100 con 0x7ffbc0100e20 2024-09-12T03:13:12.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.420+0000 7ffbbe59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbc0102570 0x7ffbc01998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:12.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.420+0000 7ffbbe59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbc0102570 0x7ffbc01998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46004/0 (socket says 172.21.15.116:46004) 2024-09-12T03:13:12.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.420+0000 7ffbbe59c640 1 -- 172.21.15.116:0/2373722337 learned_addr learned my addr 172.21.15.116:0/2373722337 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:12.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.420+0000 7ffbbed9d640 1 --2- 172.21.15.116:0/2373722337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffbc0100e20 0x7ffbc0199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:12.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.420+0000 7ffbbf59e640 1 --2- 172.21.15.116:0/2373722337 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffbc0102ef0 0x7ffbc01a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:12.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.420+0000 7ffbbe59c640 1 -- 172.21.15.116:0/2373722337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffbc0100e20 msgr2=0x7ffbc0199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:12.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.420+0000 7ffbbe59c640 1 --2- 172.21.15.116:0/2373722337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffbc0100e20 0x7ffbc0199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.420+0000 7ffbbe59c640 1 -- 172.21.15.116:0/2373722337 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffbc0102ef0 msgr2=0x7ffbc01a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:12.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.420+0000 7ffbbe59c640 1 --2- 172.21.15.116:0/2373722337 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffbc0102ef0 0x7ffbc01a4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.420+0000 7ffbbe59c640 1 -- 172.21.15.116:0/2373722337 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc01a4950 con 0x7ffbc0102570 2024-09-12T03:13:12.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.420+0000 7ffbbe59c640 1 --2- 172.21.15.116:0/2373722337 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbc0102570 0x7ffbc01998d0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400d770 tx=0x7ffbb400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:12.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.421+0000 7ffba7fff640 1 -- 172.21.15.116:0/2373722337 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4017070 con 0x7ffbc0102570 2024-09-12T03:13:12.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.421+0000 7ffbc517c640 1 -- 172.21.15.116:0/2373722337 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbc01a4b20 con 0x7ffbc0102570 2024-09-12T03:13:12.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.421+0000 7ffba7fff640 1 -- 172.21.15.116:0/2373722337 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbb4005500 con 0x7ffbc0102570 2024-09-12T03:13:12.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.421+0000 7ffba7fff640 1 -- 172.21.15.116:0/2373722337 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4004e00 con 0x7ffbc0102570 2024-09-12T03:13:12.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.421+0000 7ffbc517c640 1 -- 172.21.15.116:0/2373722337 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbc01a5060 con 0x7ffbc0102570 2024-09-12T03:13:12.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.422+0000 7ffbc517c640 1 -- 172.21.15.116:0/2373722337 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbc0101200 con 0x7ffbc0102570 2024-09-12T03:13:12.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.423+0000 7ffba7fff640 1 -- 172.21.15.116:0/2373722337 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ffbb4004fa0 con 0x7ffbc0102570 2024-09-12T03:13:12.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.423+0000 7ffba7fff640 1 --2- 172.21.15.116:0/2373722337 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ffba80a2b80 0x7ffba80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:12.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.423+0000 7ffba7fff640 1 -- 172.21.15.116:0/2373722337 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ffbb40ced90 con 0x7ffbc0102570 2024-09-12T03:13:12.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.423+0000 7ffbbed9d640 1 --2- 172.21.15.116:0/2373722337 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ffba80a2b80 0x7ffba80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:12.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.424+0000 7ffbbed9d640 1 --2- 172.21.15.116:0/2373722337 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ffba80a2b80 0x7ffba80a5040 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0009230 tx=0x7ffbb00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:12.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.427+0000 7ffba7fff640 1 -- 172.21.15.116:0/2373722337 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffbb409b6d0 con 0x7ffbc0102570 2024-09-12T03:13:12.449 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:12 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3484041570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:13:12.541 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:13:12.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.655+0000 7ffbc517c640 1 -- 172.21.15.116:0/2373722337 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffbc0109090 con 0x7ffbc0102570 2024-09-12T03:13:12.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.655+0000 7ffba7fff640 1 -- 172.21.15.116:0/2373722337 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7ffbb409f9d0 con 0x7ffbc0102570 2024-09-12T03:13:12.656 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:13:12.656 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:13:12.656 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:13:12.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.658+0000 7ffbc517c640 1 -- 172.21.15.116:0/2373722337 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ffba80a2b80 msgr2=0x7ffba80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:12.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.658+0000 7ffbc517c640 1 --2- 172.21.15.116:0/2373722337 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ffba80a2b80 0x7ffba80a5040 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0009230 tx=0x7ffbb00058d0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.658+0000 7ffbc517c640 1 -- 172.21.15.116:0/2373722337 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbc0102570 msgr2=0x7ffbc01998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:12.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.658+0000 7ffbc517c640 1 --2- 172.21.15.116:0/2373722337 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbc0102570 0x7ffbc01998d0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400d770 tx=0x7ffbb400dc30 comp rx=0 tx=0).stop 2024-09-12T03:13:12.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.658+0000 7ffbc517c640 1 -- 172.21.15.116:0/2373722337 shutdown_connections 2024-09-12T03:13:12.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.658+0000 7ffbc517c640 1 --2- 172.21.15.116:0/2373722337 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ffba80a2b80 0x7ffba80a5040 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.658+0000 7ffbc517c640 1 --2- 172.21.15.116:0/2373722337 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffbc0102ef0 0x7ffbc01a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.658+0000 7ffbc517c640 1 --2- 172.21.15.116:0/2373722337 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbc0102570 0x7ffbc01998d0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.658+0000 7ffbc517c640 1 --2- 172.21.15.116:0/2373722337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffbc0100e20 0x7ffbc0199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.658+0000 7ffbc517c640 1 -- 172.21.15.116:0/2373722337 >> 172.21.15.116:0/2373722337 conn(0x7ffbc00fc7d0 msgr2=0x7ffbc00fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:12.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.658+0000 7ffbc517c640 1 -- 172.21.15.116:0/2373722337 shutdown_connections 2024-09-12T03:13:12.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:12.658+0000 7ffbc517c640 1 -- 172.21.15.116:0/2373722337 wait complete. 2024-09-12T03:13:12.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:12 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3484041570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:13:12.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:12 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3484041570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:13:13.528 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:13 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2373722337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:13:13.528 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:13 smithi116 ceph-mon[31410]: pgmap v645: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-09-12T03:13:13.557 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:13:13.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:13 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2373722337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:13:13.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:13 smithi183 ceph-mon[35535]: pgmap v645: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-09-12T03:13:13.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:13 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2373722337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:13:13.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:13 smithi177 ceph-mon[35305]: pgmap v645: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-09-12T03:13:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.738+0000 7f8f0b447640 1 -- 172.21.15.116:0/1127466207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f04102570 msgr2=0x7f8f041029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.738+0000 7f8f0b447640 1 --2- 172.21.15.116:0/1127466207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f04102570 0x7f8f041029b0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4009900 tx=0x7f8ef402a480 comp rx=0 tx=0).stop 2024-09-12T03:13:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.738+0000 7f8f0b447640 1 -- 172.21.15.116:0/1127466207 shutdown_connections 2024-09-12T03:13:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.738+0000 7f8f0b447640 1 --2- 172.21.15.116:0/1127466207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f04102ef0 0x7f8f0410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.738+0000 7f8f0b447640 1 --2- 172.21.15.116:0/1127466207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f04102570 0x7f8f041029b0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.738+0000 7f8f0b447640 1 --2- 172.21.15.116:0/1127466207 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f04100e20 0x7f8f04101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.738+0000 7f8f0b447640 1 -- 172.21.15.116:0/1127466207 >> 172.21.15.116:0/1127466207 conn(0x7f8f040fc7d0 msgr2=0x7f8f040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.738+0000 7f8f0b447640 1 -- 172.21.15.116:0/1127466207 shutdown_connections 2024-09-12T03:13:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.738+0000 7f8f0b447640 1 -- 172.21.15.116:0/1127466207 wait complete. 2024-09-12T03:13:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.739+0000 7f8f0b447640 1 Processor -- start 2024-09-12T03:13:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.739+0000 7f8f0b447640 1 -- start start 2024-09-12T03:13:13.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.739+0000 7f8f0b447640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f04100e20 0x7f8f04198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:13.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.739+0000 7f8f0b447640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f04102570 0x7f8f04198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:13.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.739+0000 7f8f0b447640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f04102ef0 0x7f8f0419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:13.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.739+0000 7f8f0b447640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8f04110f80 con 0x7f8f04102570 2024-09-12T03:13:13.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.739+0000 7f8f0b447640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8f04110e00 con 0x7f8f04102ef0 2024-09-12T03:13:13.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.739+0000 7f8f0b447640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8f04111100 con 0x7f8f04100e20 2024-09-12T03:13:13.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.740+0000 7f8f089e2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f04102570 0x7f8f04198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:13.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.740+0000 7f8f089e2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f04102570 0x7f8f04198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46020/0 (socket says 172.21.15.116:46020) 2024-09-12T03:13:13.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.740+0000 7f8f089e2640 1 -- 172.21.15.116:0/2299128416 learned_addr learned my addr 172.21.15.116:0/2299128416 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:13.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.740+0000 7f8f091e3640 1 --2- 172.21.15.116:0/2299128416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f04100e20 0x7f8f04198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:13.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.740+0000 7f8f099e4640 1 --2- 172.21.15.116:0/2299128416 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f04102ef0 0x7f8f0419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:13.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.740+0000 7f8f089e2640 1 -- 172.21.15.116:0/2299128416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f04100e20 msgr2=0x7f8f04198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:13.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.740+0000 7f8f089e2640 1 --2- 172.21.15.116:0/2299128416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f04100e20 0x7f8f04198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.740+0000 7f8f089e2640 1 -- 172.21.15.116:0/2299128416 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f04102ef0 msgr2=0x7f8f0419ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:13.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.740+0000 7f8f089e2640 1 --2- 172.21.15.116:0/2299128416 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f04102ef0 0x7f8f0419ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.740+0000 7f8f089e2640 1 -- 172.21.15.116:0/2299128416 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f0419d4f0 con 0x7f8f04102570 2024-09-12T03:13:13.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.740+0000 7f8f091e3640 1 --2- 172.21.15.116:0/2299128416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f04100e20 0x7f8f04198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:13:13.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.740+0000 7f8f089e2640 1 --2- 172.21.15.116:0/2299128416 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f04102570 0x7f8f04198a80 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4006010 tx=0x7f8ef4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:13.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.740+0000 7f8efa7fc640 1 -- 172.21.15.116:0/2299128416 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef403f070 con 0x7f8f04102570 2024-09-12T03:13:13.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.740+0000 7f8f0b447640 1 -- 172.21.15.116:0/2299128416 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f0419d780 con 0x7f8f04102570 2024-09-12T03:13:13.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.741+0000 7f8efa7fc640 1 -- 172.21.15.116:0/2299128416 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ef4031c30 con 0x7f8f04102570 2024-09-12T03:13:13.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.741+0000 7f8f0b447640 1 -- 172.21.15.116:0/2299128416 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f041a50c0 con 0x7f8f04102570 2024-09-12T03:13:13.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.741+0000 7f8efa7fc640 1 -- 172.21.15.116:0/2299128416 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4049060 con 0x7f8f04102570 2024-09-12T03:13:13.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.743+0000 7f8efa7fc640 1 -- 172.21.15.116:0/2299128416 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8ef4042070 con 0x7f8f04102570 2024-09-12T03:13:13.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.744+0000 7f8f0d820740 1 Processor -- start 2024-09-12T03:13:13.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.744+0000 7f8f0d820740 1 -- start start 2024-09-12T03:13:13.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.744+0000 7f8f0d820740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b89449730 0x559b89449b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:13.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.744+0000 7f8f0d820740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559b8944a050 0x559b89458900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:13.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.744+0000 7f8efa7fc640 1 --2- 172.21.15.116:0/2299128416 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8ef00a2b80 0x7f8ef00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:13.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.744+0000 7f8f0d820740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b89458e40 0x559b8945b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:13.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.744+0000 7f8f0d820740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559b893ae160 con 0x559b8944a050 2024-09-12T03:13:13.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.744+0000 7f8f0d820740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559b89355720 con 0x559b89449730 2024-09-12T03:13:13.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.744+0000 7f8f0d820740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559b893aa120 con 0x559b89458e40 2024-09-12T03:13:13.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.744+0000 7f8f089e2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559b8944a050 0x559b89458900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:13.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.744+0000 7f8f089e2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559b8944a050 0x559b89458900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46028/0 (socket says 172.21.15.116:46028) 2024-09-12T03:13:13.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.744+0000 7f8f089e2640 1 -- 172.21.15.116:0/2385180387 learned_addr learned my addr 172.21.15.116:0/2385180387 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:13.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.744+0000 7f8f099e4640 1 --2- 172.21.15.116:0/2385180387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b89449730 0x559b89449b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:13.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.744+0000 7f8f091e3640 1 --2- 172.21.15.116:0/2299128416 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8ef00a2b80 0x7f8ef00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:13.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.745+0000 7f8efa7fc640 1 -- 172.21.15.116:0/2299128416 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f8ef40ef5d0 con 0x7f8f04102570 2024-09-12T03:13:13.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.745+0000 7f8f099e4640 1 --2- 172.21.15.116:0/2385180387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b89458e40 0x559b8945b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:13.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.745+0000 7f8f099e4640 1 -- 172.21.15.116:0/2385180387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b89458e40 msgr2=0x559b8945b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:13.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.745+0000 7f8f099e4640 1 --2- 172.21.15.116:0/2385180387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b89458e40 0x559b8945b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.745+0000 7f8f099e4640 1 -- 172.21.15.116:0/2385180387 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559b8944a050 msgr2=0x559b89458900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:13.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.745+0000 7f8f099e4640 1 --2- 172.21.15.116:0/2385180387 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559b8944a050 0x559b89458900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.745+0000 7f8f099e4640 1 -- 172.21.15.116:0/2385180387 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b8911fba0 con 0x559b89449730 2024-09-12T03:13:13.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.745+0000 7f8f089e2640 1 --2- 172.21.15.116:0/2385180387 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559b8944a050 0x559b89458900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:13:13.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.745+0000 7f8f099e4640 1 --2- 172.21.15.116:0/2385180387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b89449730 0x559b89449b10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8efc005fd0 tx=0x7f8efc009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd2276188e68618c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:13.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.745+0000 7f8f091e3640 1 --2- 172.21.15.116:0/2299128416 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8ef00a2b80 0x7f8ef00a5040 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8f000058c0 tx=0x7f8f0000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:13.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.745+0000 7f8eef7fe640 1 -- 172.21.15.116:0/2385180387 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc003b70 con 0x559b89449730 2024-09-12T03:13:13.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.745+0000 7f8eef7fe640 1 -- 172.21.15.116:0/2385180387 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8efc010dd0 con 0x559b89449730 2024-09-12T03:13:13.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.745+0000 7f8eef7fe640 1 -- 172.21.15.116:0/2385180387 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc01d3a0 con 0x559b89449730 2024-09-12T03:13:13.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.746+0000 7f8f0d820740 1 -- 172.21.15.116:0/2385180387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b89449730 msgr2=0x559b89449b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:13.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.746+0000 7f8f0d820740 1 --2- 172.21.15.116:0/2385180387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b89449730 0x559b89449b10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8efc005fd0 tx=0x7f8efc009a20 comp rx=0 tx=0).stop 2024-09-12T03:13:13.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.746+0000 7f8f0d820740 1 -- 172.21.15.116:0/2385180387 shutdown_connections 2024-09-12T03:13:13.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.746+0000 7f8f0d820740 1 --2- 172.21.15.116:0/2385180387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b89458e40 0x559b8945b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.746+0000 7f8f0d820740 1 --2- 172.21.15.116:0/2385180387 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559b8944a050 0x559b89458900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.746+0000 7f8f0d820740 1 --2- 172.21.15.116:0/2385180387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b89449730 0x559b89449b10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.746+0000 7f8f0d820740 1 -- 172.21.15.116:0/2385180387 >> 172.21.15.116:0/2385180387 conn(0x559b89445de0 msgr2=0x559b894461c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:13.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.747+0000 7f8f0d820740 1 -- 172.21.15.116:0/2385180387 shutdown_connections 2024-09-12T03:13:13.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.747+0000 7f8f0d820740 1 -- 172.21.15.116:0/2385180387 wait complete. 2024-09-12T03:13:13.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.747+0000 7f8f0d820740 1 -- start start 2024-09-12T03:13:13.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.747+0000 7f8f0d820740 1 Processor -- start 2024-09-12T03:13:13.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.748+0000 7f8f0d820740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559b89449730 0x559b89455dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:13.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.748+0000 7f8f0d820740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b8944a050 0x559b89456300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:13.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.748+0000 7f8f0d820740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b89458e40 0x559b89456840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:13.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.748+0000 7f8f0d820740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559b8945ba60 con 0x559b89449730 2024-09-12T03:13:13.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.748+0000 7f8f0d820740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559b89336900 con 0x559b8944a050 2024-09-12T03:13:13.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.748+0000 7f8f0d820740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559b8945bbe0 con 0x559b89458e40 2024-09-12T03:13:13.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.748+0000 7f8f091e3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559b89449730 0x559b89455dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:13.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.748+0000 7f8f091e3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559b89449730 0x559b89455dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46030/0 (socket says 172.21.15.116:46030) 2024-09-12T03:13:13.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.748+0000 7f8f091e3640 1 -- 172.21.15.116:0/1679993635 learned_addr learned my addr 172.21.15.116:0/1679993635 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:13.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.748+0000 7f8f0d820740 10 client.0 _ll_register_callbacks cb 0x559b89334780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:13.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.748+0000 7f8f099e4640 1 --2- 172.21.15.116:0/1679993635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b89458e40 0x559b89456840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:13.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.748+0000 7f8f089e2640 1 --2- 172.21.15.116:0/1679993635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b8944a050 0x559b89456300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:13.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.749+0000 7f8f091e3640 1 -- 172.21.15.116:0/1679993635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b89458e40 msgr2=0x559b89456840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:13.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.749+0000 7f8f091e3640 1 --2- 172.21.15.116:0/1679993635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b89458e40 0x559b89456840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.749+0000 7f8f091e3640 1 -- 172.21.15.116:0/1679993635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b8944a050 msgr2=0x559b89456300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:13.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.749+0000 7f8f091e3640 1 --2- 172.21.15.116:0/1679993635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b8944a050 0x559b89456300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.749+0000 7f8f091e3640 1 -- 172.21.15.116:0/1679993635 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b89376c50 con 0x559b89449730 2024-09-12T03:13:13.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.749+0000 7f8f099e4640 1 --2- 172.21.15.116:0/1679993635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b89458e40 0x559b89456840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:13:13.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.749+0000 7f8f091e3640 1 --2- 172.21.15.116:0/1679993635 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559b89449730 0x559b89455dc0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000afd0 tx=0x7f8f00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d64982018ee11b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:13.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.749+0000 7f8eecff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:13.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.749+0000 7f8eecff9640 1 -- 172.21.15.116:0/1679993635 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f00016070 con 0x559b89449730 2024-09-12T03:13:13.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.749+0000 7f8eecff9640 1 -- 172.21.15.116:0/1679993635 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f8f0001c420 con 0x559b89449730 2024-09-12T03:13:13.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.749+0000 7f8f0d820740 1 -- 172.21.15.116:0/1679993635 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8efc010dd0 con 0x559b89449730 2024-09-12T03:13:13.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.750+0000 7f8eecff9640 1 -- 172.21.15.116:0/1679993635 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8f00027680 con 0x559b89449730 2024-09-12T03:13:13.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.750+0000 7f8f0d820740 10 client.18180 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:13.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.750+0000 7f8f0d820740 1 -- 172.21.15.116:0/1679993635 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559b89467880 con 0x559b89449730 2024-09-12T03:13:13.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.750+0000 7f8eecff9640 1 -- 172.21.15.116:0/1679993635 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8efc010dd0 con 0x559b89449730 2024-09-12T03:13:13.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.750+0000 7f8eecff9640 1 -- 172.21.15.116:0/1679993635 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8f0001ca90 con 0x559b89449730 2024-09-12T03:13:13.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.750+0000 7f8f0d820740 10 client.18180 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:13.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.750+0000 7f8f0d820740 10 client.18180 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:13:13.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.750+0000 7f8f0d820740 20 client.18180 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:13.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.750+0000 7f8f0d820740 1 --2- 172.21.15.116:0/1679993635 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559b89468d40 0x559b8946b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:13.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.750+0000 7f8f0d820740 4 client.18180 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:13:13.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.750+0000 7f8f0d820740 1 -- 172.21.15.116:0/1679993635 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b893bb890 con 0x559b89468d40 2024-09-12T03:13:13.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.750+0000 7f8f089e2640 1 --2- 172.21.15.116:0/1679993635 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559b89468d40 0x559b8946b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:13.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.751+0000 7f8eecff9640 10 client.18180 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:13.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.751+0000 7f8f089e2640 1 --2- 172.21.15.116:0/1679993635 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559b89468d40 0x559b8946b130 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49ce9b1a0f1a831 server_cookie=532f934e64231c1e in_seq=0 out_seq=0 2024-09-12T03:13:13.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.751+0000 7f8eecff9640 1 -- 172.21.15.116:0/1679993635 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559b893bb890 con 0x559b89468d40 2024-09-12T03:13:13.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.751+0000 7f8eecff9640 10 client.18180 handle_command_reply: tid=0 2024-09-12T03:13:13.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.752+0000 7f8f0d820740 1 client.18180 shutdown 2024-09-12T03:13:13.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.752+0000 7f8f0d820740 1 -- 172.21.15.116:0/1679993635 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559b89449730 msgr2=0x559b89455dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:13.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.752+0000 7f8f0d820740 1 --2- 172.21.15.116:0/1679993635 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559b89449730 0x559b89455dc0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000afd0 tx=0x7f8f00004730 comp rx=0 tx=0).stop 2024-09-12T03:13:13.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.752+0000 7f8f0d820740 1 -- 172.21.15.116:0/1679993635 shutdown_connections 2024-09-12T03:13:13.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.752+0000 7f8f0d820740 1 --2- 172.21.15.116:0/1679993635 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559b89468d40 0x559b8946b130 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.752+0000 7f8f0d820740 1 --2- 172.21.15.116:0/1679993635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b89458e40 0x559b89456840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.752+0000 7f8f0d820740 1 --2- 172.21.15.116:0/1679993635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b8944a050 0x559b89456300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.752+0000 7f8eecff9640 0 client.18180 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:13.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.752+0000 7f8f0d820740 1 --2- 172.21.15.116:0/1679993635 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559b89449730 0x559b89455dc0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.752+0000 7f8f0d820740 1 -- 172.21.15.116:0/1679993635 >> 172.21.15.116:0/1679993635 conn(0x559b89445de0 msgr2=0x559b8944bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:13.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.752+0000 7f8f0d820740 1 -- 172.21.15.116:0/1679993635 shutdown_connections 2024-09-12T03:13:13.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.752+0000 7f8f0d820740 1 -- 172.21.15.116:0/1679993635 wait complete. 2024-09-12T03:13:13.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.752+0000 7f8f0d820740 20 client.18180 trim_cache size 0 max 16384 2024-09-12T03:13:13.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.769+0000 7f8f0d820740 1 Processor -- start 2024-09-12T03:13:13.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.769+0000 7f8f0d820740 1 -- start start 2024-09-12T03:13:13.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.769+0000 7f8f0d820740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559b89449730 0x559b894709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:13.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.769+0000 7f8f0d820740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b8944a050 0x559b89470ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:13.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.769+0000 7f8f0d820740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b89458e40 0x559b89453020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:13.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.769+0000 7f8f0d820740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8f041adaf0 con 0x559b89449730 2024-09-12T03:13:13.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.769+0000 7f8f0d820740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8f041ab3c0 con 0x559b89458e40 2024-09-12T03:13:13.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.770+0000 7f8f0d820740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8f041a9210 con 0x559b8944a050 2024-09-12T03:13:13.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.770+0000 7f8f091e3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559b89449730 0x559b894709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:13.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.770+0000 7f8f091e3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559b89449730 0x559b894709b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46034/0 (socket says 172.21.15.116:46034) 2024-09-12T03:13:13.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.770+0000 7f8f091e3640 1 -- 172.21.15.116:0/748547102 learned_addr learned my addr 172.21.15.116:0/748547102 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:13.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.770+0000 7f8f089e2640 1 --2- 172.21.15.116:0/748547102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b8944a050 0x559b89470ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:13.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.770+0000 7f8f099e4640 1 --2- 172.21.15.116:0/748547102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b89458e40 0x559b89453020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:13.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.770+0000 7f8f091e3640 1 -- 172.21.15.116:0/748547102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b8944a050 msgr2=0x559b89470ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:13.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.770+0000 7f8f091e3640 1 --2- 172.21.15.116:0/748547102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b8944a050 0x559b89470ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.770+0000 7f8f091e3640 1 -- 172.21.15.116:0/748547102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b89458e40 msgr2=0x559b89453020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:13.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.770+0000 7f8f091e3640 1 --2- 172.21.15.116:0/748547102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b89458e40 0x559b89453020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.770+0000 7f8f091e3640 1 -- 172.21.15.116:0/748547102 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b89445c10 con 0x559b89449730 2024-09-12T03:13:13.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.770+0000 7f8f091e3640 1 --2- 172.21.15.116:0/748547102 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559b89449730 0x559b894709b0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f8f0001ceb0 tx=0x7f8f0000ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b47fd7099822e6e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:13.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.771+0000 7f8eeeffd640 1 -- 172.21.15.116:0/748547102 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0000fca0 con 0x559b89449730 2024-09-12T03:13:13.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.771+0000 7f8eeeffd640 1 -- 172.21.15.116:0/748547102 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f00005500 con 0x559b89449730 2024-09-12T03:13:13.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.771+0000 7f8eeeffd640 1 -- 172.21.15.116:0/748547102 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f00004a00 con 0x559b89449730 2024-09-12T03:13:13.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.771+0000 7f8f0d820740 1 -- 172.21.15.116:0/748547102 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559b89449730 msgr2=0x559b894709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:13.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.771+0000 7f8f0d820740 1 --2- 172.21.15.116:0/748547102 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559b89449730 0x559b894709b0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f8f0001ceb0 tx=0x7f8f0000ad60 comp rx=0 tx=0).stop 2024-09-12T03:13:13.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.772+0000 7f8f0d820740 1 -- 172.21.15.116:0/748547102 shutdown_connections 2024-09-12T03:13:13.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.772+0000 7f8f0d820740 1 --2- 172.21.15.116:0/748547102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b89458e40 0x559b89453020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.772+0000 7f8f0d820740 1 --2- 172.21.15.116:0/748547102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b8944a050 0x559b89470ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.772+0000 7f8f0d820740 1 --2- 172.21.15.116:0/748547102 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559b89449730 0x559b894709b0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.772+0000 7f8f0d820740 1 -- 172.21.15.116:0/748547102 >> 172.21.15.116:0/748547102 conn(0x559b89445de0 msgr2=0x559b89444ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:13.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.772+0000 7f8f0d820740 1 -- 172.21.15.116:0/748547102 shutdown_connections 2024-09-12T03:13:13.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.772+0000 7f8f0d820740 1 -- 172.21.15.116:0/748547102 wait complete. 2024-09-12T03:13:13.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.772+0000 7f8f0d820740 1 -- start start 2024-09-12T03:13:13.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.772+0000 7f8f0d820740 1 Processor -- start 2024-09-12T03:13:13.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.773+0000 7f8f0d820740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559b89449730 0x559b8945cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:13.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.773+0000 7f8f0d820740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b8944a050 0x559b8945d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:13.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.773+0000 7f8f0d820740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b89458e40 0x559b8945d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:13.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.773+0000 7f8f0d820740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8f041a9880 con 0x559b89449730 2024-09-12T03:13:13.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.773+0000 7f8f0d820740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8f041aacf0 con 0x559b89458e40 2024-09-12T03:13:13.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.773+0000 7f8f0d820740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8f041a9f50 con 0x559b8944a050 2024-09-12T03:13:13.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.773+0000 7f8f0d820740 10 client.0 _ll_register_callbacks cb 0x7f8f00010ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:13.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.773+0000 7f8f091e3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559b89449730 0x559b8945cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:13.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.773+0000 7f8f091e3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559b89449730 0x559b8945cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46046/0 (socket says 172.21.15.116:46046) 2024-09-12T03:13:13.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.773+0000 7f8f091e3640 1 -- 172.21.15.116:0/2162781330 learned_addr learned my addr 172.21.15.116:0/2162781330 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:13.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.773+0000 7f8f089e2640 1 --2- 172.21.15.116:0/2162781330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b8944a050 0x559b8945d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:13.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.773+0000 7f8f099e4640 1 --2- 172.21.15.116:0/2162781330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b89458e40 0x559b8945d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:13.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.773+0000 7f8f091e3640 1 -- 172.21.15.116:0/2162781330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b8944a050 msgr2=0x559b8945d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:13.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.773+0000 7f8f091e3640 1 --2- 172.21.15.116:0/2162781330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b8944a050 0x559b8945d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.773+0000 7f8f091e3640 1 -- 172.21.15.116:0/2162781330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b89458e40 msgr2=0x559b8945d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:13.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.773+0000 7f8f091e3640 1 --2- 172.21.15.116:0/2162781330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b89458e40 0x559b8945d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.773+0000 7f8f091e3640 1 -- 172.21.15.116:0/2162781330 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b894413f0 con 0x559b89449730 2024-09-12T03:13:13.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.773+0000 7f8f099e4640 1 --2- 172.21.15.116:0/2162781330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b89458e40 0x559b8945d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:13:13.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.774+0000 7f8f091e3640 1 --2- 172.21.15.116:0/2162781330 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559b89449730 0x559b8945cd00 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f8f00027a50 tx=0x7f8f000056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d49ff7ed5233e00 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:13.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.774+0000 7f8eecff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:13.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.774+0000 7f8eecff9640 1 -- 172.21.15.116:0/2162781330 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0003f070 con 0x559b89449730 2024-09-12T03:13:13.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.774+0000 7f8eecff9640 1 -- 172.21.15.116:0/2162781330 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f8f000048f0 con 0x559b89449730 2024-09-12T03:13:13.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.774+0000 7f8f0d820740 1 -- 172.21.15.116:0/2162781330 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f00005500 con 0x559b89449730 2024-09-12T03:13:13.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.774+0000 7f8eecff9640 1 -- 172.21.15.116:0/2162781330 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8f00016040 con 0x559b89449730 2024-09-12T03:13:13.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.774+0000 7f8f0d820740 10 client.18186 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:13.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.774+0000 7f8f0d820740 1 -- 172.21.15.116:0/2162781330 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559b894d3b50 con 0x559b89449730 2024-09-12T03:13:13.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.774+0000 7f8eecff9640 1 -- 172.21.15.116:0/2162781330 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f0001be90 con 0x559b89449730 2024-09-12T03:13:13.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.775+0000 7f8eecff9640 1 -- 172.21.15.116:0/2162781330 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8f00042020 con 0x559b89449730 2024-09-12T03:13:13.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.775+0000 7f8f0d820740 10 client.18186 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:13.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.775+0000 7f8f0d820740 10 client.18186 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:13:13.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.775+0000 7f8f0d820740 20 client.18186 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:13.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.775+0000 7f8f0d820740 1 --2- 172.21.15.116:0/2162781330 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559b894d41c0 0x559b894d65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:13.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.775+0000 7f8f0d820740 4 client.18186 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}] 2024-09-12T03:13:13.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.775+0000 7f8f0d820740 1 -- 172.21.15.116:0/2162781330 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}) -- 0x7f8f00005500 con 0x559b894d41c0 2024-09-12T03:13:13.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.775+0000 7f8f089e2640 1 --2- 172.21.15.116:0/2162781330 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559b894d41c0 0x559b894d65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:13.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.775+0000 7f8eecff9640 10 client.18186 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:13.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.776+0000 7f8f089e2640 1 --2- 172.21.15.116:0/2162781330 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559b894d41c0 0x559b894d65b0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50fd539b8b04171f server_cookie=77b4820f78bd4a03 in_seq=0 out_seq=0 2024-09-12T03:13:13.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.787+0000 7f8eecff9640 1 -- 172.21.15.116:0/2162781330 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f8f00005500 con 0x559b894d41c0 2024-09-12T03:13:13.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.787+0000 7f8eecff9640 10 client.18186 handle_command_reply: tid=0 2024-09-12T03:13:13.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.787+0000 7f8f0d820740 1 client.18186 shutdown 2024-09-12T03:13:13.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.788+0000 7f8f0d820740 1 -- 172.21.15.116:0/2162781330 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559b89449730 msgr2=0x559b8945cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:13.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.788+0000 7f8f0d820740 1 --2- 172.21.15.116:0/2162781330 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559b89449730 0x559b8945cd00 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f8f00027a50 tx=0x7f8f000056b0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.788+0000 7f8f0d820740 1 -- 172.21.15.116:0/2162781330 shutdown_connections 2024-09-12T03:13:13.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.788+0000 7f8f0d820740 1 --2- 172.21.15.116:0/2162781330 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x559b894d41c0 0x559b894d65b0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.788+0000 7f8f0d820740 1 --2- 172.21.15.116:0/2162781330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b89458e40 0x559b8945d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.788+0000 7f8f0d820740 1 --2- 172.21.15.116:0/2162781330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b8944a050 0x559b8945d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.788+0000 7f8f0d820740 1 --2- 172.21.15.116:0/2162781330 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559b89449730 0x559b8945cd00 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.788+0000 7f8f0d820740 1 -- 172.21.15.116:0/2162781330 >> 172.21.15.116:0/2162781330 conn(0x559b89445de0 msgr2=0x559b894446a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:13.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.788+0000 7f8eecff9640 0 client.18186 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:13.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.788+0000 7f8f0d820740 1 -- 172.21.15.116:0/2162781330 shutdown_connections 2024-09-12T03:13:13.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.788+0000 7f8f0d820740 1 -- 172.21.15.116:0/2162781330 wait complete. 2024-09-12T03:13:13.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.788+0000 7f8f0d820740 20 client.18186 trim_cache size 0 max 16384 2024-09-12T03:13:13.790 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:13:13.790 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 25, 2024-09-12T03:13:13.790 INFO:teuthology.orchestra.run.smithi116.stdout: "leader": 24494, 2024-09-12T03:13:13.790 INFO:teuthology.orchestra.run.smithi116.stdout: "set_version": 42, 2024-09-12T03:13:13.790 INFO:teuthology.orchestra.run.smithi116.stdout: "sets": { 2024-09-12T03:13:13.790 INFO:teuthology.orchestra.run.smithi116.stdout: "904ad12d": { 2024-09-12T03:13:13.790 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 42, 2024-09-12T03:13:13.790 INFO:teuthology.orchestra.run.smithi116.stdout: "age_ref": 0.0, 2024-09-12T03:13:13.790 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:13:13.791 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "QUIESCED", 2024-09-12T03:13:13.791 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:13:13.791 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:13:13.791 INFO:teuthology.orchestra.run.smithi116.stdout: "timeout": 90.0, 2024-09-12T03:13:13.791 INFO:teuthology.orchestra.run.smithi116.stdout: "expiration": 142.6, 2024-09-12T03:13:13.791 INFO:teuthology.orchestra.run.smithi116.stdout: "members": { 2024-09-12T03:13:13.791 INFO:teuthology.orchestra.run.smithi116.stdout: "file:/": { 2024-09-12T03:13:13.791 INFO:teuthology.orchestra.run.smithi116.stdout: "excluded": false, 2024-09-12T03:13:13.791 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:13:13.791 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "QUIESCED", 2024-09-12T03:13:13.791 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:13:13.791 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:13:13.791 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:13:13.792 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:13:13.792 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:13:13.792 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:13:13.792 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:13:13.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.789+0000 7f8f0b447640 1 -- 172.21.15.116:0/2299128416 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8ef00a2b80 msgr2=0x7f8ef00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:13.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.789+0000 7f8f0b447640 1 --2- 172.21.15.116:0/2299128416 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8ef00a2b80 0x7f8ef00a5040 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8f000058c0 tx=0x7f8f0000a2d0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.789+0000 7f8f0b447640 1 -- 172.21.15.116:0/2299128416 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f04102570 msgr2=0x7f8f04198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:13.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.789+0000 7f8f0b447640 1 --2- 172.21.15.116:0/2299128416 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f04102570 0x7f8f04198a80 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4006010 tx=0x7f8ef4003c10 comp rx=0 tx=0).stop 2024-09-12T03:13:13.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.789+0000 7f8f0b447640 1 -- 172.21.15.116:0/2299128416 shutdown_connections 2024-09-12T03:13:13.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.789+0000 7f8f0b447640 1 --2- 172.21.15.116:0/2299128416 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8ef00a2b80 0x7f8ef00a5040 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.789+0000 7f8f0b447640 1 --2- 172.21.15.116:0/2299128416 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f04102ef0 0x7f8f0419ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.789+0000 7f8f0b447640 1 --2- 172.21.15.116:0/2299128416 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f04102570 0x7f8f04198a80 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.789+0000 7f8f0b447640 1 --2- 172.21.15.116:0/2299128416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f04100e20 0x7f8f04198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.789+0000 7f8f0b447640 1 -- 172.21.15.116:0/2299128416 >> 172.21.15.116:0/2299128416 conn(0x7f8f040fc7d0 msgr2=0x7f8f0406a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:13.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.789+0000 7f8f0b447640 1 -- 172.21.15.116:0/2299128416 shutdown_connections 2024-09-12T03:13:13.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:13.789+0000 7f8f0b447640 1 -- 172.21.15.116:0/2299128416 wait complete. 2024-09-12T03:13:14.010 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:13:14.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.506+0000 7fc9c0300640 1 -- 172.21.15.116:0/1542760056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9b8102d40 msgr2=0x7fc9b8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:14.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.506+0000 7fc9c0300640 1 --2- 172.21.15.116:0/1542760056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9b8102d40 0x7fc9b8103120 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4009900 tx=0x7fc9b402a480 comp rx=0 tx=0).stop 2024-09-12T03:13:14.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.506+0000 7fc9c0300640 1 -- 172.21.15.116:0/1542760056 shutdown_connections 2024-09-12T03:13:14.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.506+0000 7fc9c0300640 1 --2- 172.21.15.116:0/1542760056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9b810c170 0x7fc9b810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.506+0000 7fc9c0300640 1 --2- 172.21.15.116:0/1542760056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9b8103660 0x7fc9b810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.506+0000 7fc9c0300640 1 --2- 172.21.15.116:0/1542760056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9b8102d40 0x7fc9b8103120 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.506+0000 7fc9c0300640 1 -- 172.21.15.116:0/1542760056 >> 172.21.15.116:0/1542760056 conn(0x7fc9b80fc7a0 msgr2=0x7fc9b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:14.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.507+0000 7fc9c0300640 1 -- 172.21.15.116:0/1542760056 shutdown_connections 2024-09-12T03:13:14.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.507+0000 7fc9c0300640 1 -- 172.21.15.116:0/1542760056 wait complete. 2024-09-12T03:13:14.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.507+0000 7fc9c0300640 1 Processor -- start 2024-09-12T03:13:14.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.507+0000 7fc9c0300640 1 -- start start 2024-09-12T03:13:14.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.507+0000 7fc9c0300640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9b8102d40 0x7fc9b8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:14.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.507+0000 7fc9c0300640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9b8103660 0x7fc9b8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:14.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.507+0000 7fc9c0300640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9b810c170 0x7fc9b819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:14.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.507+0000 7fc9c0300640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b8111f70 con 0x7fc9b8103660 2024-09-12T03:13:14.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.507+0000 7fc9c0300640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b8111df0 con 0x7fc9b810c170 2024-09-12T03:13:14.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.507+0000 7fc9c0300640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b81120f0 con 0x7fc9b8102d40 2024-09-12T03:13:14.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.507+0000 7fc9bd89b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9b8103660 0x7fc9b8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:14.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.507+0000 7fc9bd89b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9b8103660 0x7fc9b8198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46070/0 (socket says 172.21.15.116:46070) 2024-09-12T03:13:14.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.507+0000 7fc9bd89b640 1 -- 172.21.15.116:0/1347447813 learned_addr learned my addr 172.21.15.116:0/1347447813 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:14.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.508+0000 7fc9be89d640 1 --2- 172.21.15.116:0/1347447813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9b810c170 0x7fc9b819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:14.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.508+0000 7fc9bd89b640 1 -- 172.21.15.116:0/1347447813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9b8102d40 msgr2=0x7fc9b8198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:14.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.508+0000 7fc9bd89b640 1 --2- 172.21.15.116:0/1347447813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9b8102d40 0x7fc9b8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.508+0000 7fc9bd89b640 1 -- 172.21.15.116:0/1347447813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9b810c170 msgr2=0x7fc9b819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:14.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.508+0000 7fc9bd89b640 1 --2- 172.21.15.116:0/1347447813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9b810c170 0x7fc9b819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.508+0000 7fc9bd89b640 1 -- 172.21.15.116:0/1347447813 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b819d3e0 con 0x7fc9b8103660 2024-09-12T03:13:14.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.508+0000 7fc9bd89b640 1 --2- 172.21.15.116:0/1347447813 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9b8103660 0x7fc9b8198970 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800d950 tx=0x7fc9a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:14.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.508+0000 7fc9af7fe640 1 -- 172.21.15.116:0/1347447813 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8017030 con 0x7fc9b8103660 2024-09-12T03:13:14.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.508+0000 7fc9af7fe640 1 -- 172.21.15.116:0/1347447813 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9a8004020 con 0x7fc9b8103660 2024-09-12T03:13:14.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.508+0000 7fc9c0300640 1 -- 172.21.15.116:0/1347447813 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9b819d6d0 con 0x7fc9b8103660 2024-09-12T03:13:14.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.508+0000 7fc9af7fe640 1 -- 172.21.15.116:0/1347447813 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a80047a0 con 0x7fc9b8103660 2024-09-12T03:13:14.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.508+0000 7fc9c0300640 1 -- 172.21.15.116:0/1347447813 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9b81a4fb0 con 0x7fc9b8103660 2024-09-12T03:13:14.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.511+0000 7fc9c26d9740 1 Processor -- start 2024-09-12T03:13:14.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.511+0000 7fc9c26d9740 1 -- start start 2024-09-12T03:13:14.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.511+0000 7fc9af7fe640 1 -- 172.21.15.116:0/1347447813 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fc9a80041d0 con 0x7fc9b8103660 2024-09-12T03:13:14.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.511+0000 7fc9c26d9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586693d8b40 0x5586693d7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:14.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.511+0000 7fc9c26d9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5586693d75d0 0x5586693d7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:14.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.511+0000 7fc9c26d9740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586693e2370 0x5586693e4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:14.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.511+0000 7fc9c26d9740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55866933d2f0 con 0x5586693e2370 2024-09-12T03:13:14.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.511+0000 7fc9c26d9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5586692e4cf0 con 0x5586693d8b40 2024-09-12T03:13:14.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.511+0000 7fc9c26d9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5586693392b0 con 0x5586693d75d0 2024-09-12T03:13:14.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.512+0000 7fc9bd89b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586693e2370 0x5586693e4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:14.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.512+0000 7fc9bd89b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586693e2370 0x5586693e4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46078/0 (socket says 172.21.15.116:46078) 2024-09-12T03:13:14.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.512+0000 7fc9bd89b640 1 -- 172.21.15.116:0/104063199 learned_addr learned my addr 172.21.15.116:0/104063199 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:14.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.512+0000 7fc9be89d640 1 --2- 172.21.15.116:0/104063199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586693d8b40 0x5586693d7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:14.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.512+0000 7fc9be09c640 1 --2- 172.21.15.116:0/104063199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5586693d75d0 0x5586693d7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:14.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.512+0000 7fc9bd89b640 1 -- 172.21.15.116:0/104063199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5586693d75d0 msgr2=0x5586693d7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:14.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.512+0000 7fc9bd89b640 1 --2- 172.21.15.116:0/104063199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5586693d75d0 0x5586693d7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.512+0000 7fc9bd89b640 1 -- 172.21.15.116:0/104063199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586693d8b40 msgr2=0x5586693d7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:14.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.512+0000 7fc9bd89b640 1 --2- 172.21.15.116:0/104063199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586693d8b40 0x5586693d7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.512+0000 7fc9bd89b640 1 -- 172.21.15.116:0/104063199 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586692c93a0 con 0x5586693e2370 2024-09-12T03:13:14.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.512+0000 7fc9be09c640 1 --2- 172.21.15.116:0/104063199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5586693d75d0 0x5586693d7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:13:14.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.512+0000 7fc9af7fe640 1 --2- 172.21.15.116:0/1347447813 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc9a40a2b80 0x7fc9a40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:14.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.512+0000 7fc9be89d640 1 --2- 172.21.15.116:0/1347447813 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc9a40a2b80 0x7fc9a40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:14.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.512+0000 7fc9bd89b640 1 --2- 172.21.15.116:0/104063199 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586693e2370 0x5586693e4760 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fc9a80d4a80 tx=0x7fc9a80d4f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460371f68051632b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:14.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.512+0000 7fc9af7fe640 1 -- 172.21.15.116:0/1347447813 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fc9a80cea10 con 0x7fc9b8103660 2024-09-12T03:13:14.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.512+0000 7fc9acdea640 1 -- 172.21.15.116:0/104063199 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a809b070 con 0x5586693e2370 2024-09-12T03:13:14.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.513+0000 7fc9acdea640 1 -- 172.21.15.116:0/104063199 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9a80b6070 con 0x5586693e2370 2024-09-12T03:13:14.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.513+0000 7fc9acdea640 1 -- 172.21.15.116:0/104063199 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a809ac00 con 0x5586693e2370 2024-09-12T03:13:14.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.513+0000 7fc9c26d9740 1 -- 172.21.15.116:0/104063199 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586693e2370 msgr2=0x5586693e4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:14.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.513+0000 7fc9c26d9740 1 --2- 172.21.15.116:0/104063199 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586693e2370 0x5586693e4760 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fc9a80d4a80 tx=0x7fc9a80d4f40 comp rx=0 tx=0).stop 2024-09-12T03:13:14.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.513+0000 7fc9be89d640 1 --2- 172.21.15.116:0/1347447813 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc9a40a2b80 0x7fc9a40a5040 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0005fd0 tx=0x7fc9b0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:14.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.513+0000 7fc9c26d9740 1 -- 172.21.15.116:0/104063199 shutdown_connections 2024-09-12T03:13:14.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.513+0000 7fc9c26d9740 1 --2- 172.21.15.116:0/104063199 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586693e2370 0x5586693e4760 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.513+0000 7fc9c26d9740 1 --2- 172.21.15.116:0/104063199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5586693d75d0 0x5586693d7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.513+0000 7fc9c26d9740 1 --2- 172.21.15.116:0/104063199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586693d8b40 0x5586693d7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.513+0000 7fc9c26d9740 1 -- 172.21.15.116:0/104063199 >> 172.21.15.116:0/104063199 conn(0x5586693a0310 msgr2=0x5586693d3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:14.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.513+0000 7fc9c26d9740 1 -- 172.21.15.116:0/104063199 shutdown_connections 2024-09-12T03:13:14.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.513+0000 7fc9c26d9740 1 -- 172.21.15.116:0/104063199 wait complete. 2024-09-12T03:13:14.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.514+0000 7fc9c26d9740 1 -- start start 2024-09-12T03:13:14.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.514+0000 7fc9c26d9740 1 Processor -- start 2024-09-12T03:13:14.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.514+0000 7fc9c26d9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586693d75d0 0x5586693e1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:14.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.514+0000 7fc9c26d9740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586693d8b40 0x5586693df040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:14.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.514+0000 7fc9c26d9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5586693e2370 0x5586693df650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:14.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.514+0000 7fc9c26d9740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5586693e5070 con 0x5586693d8b40 2024-09-12T03:13:14.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.514+0000 7fc9c26d9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5586692f8060 con 0x5586693d75d0 2024-09-12T03:13:14.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.514+0000 7fc9c26d9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5586693e51f0 con 0x5586693e2370 2024-09-12T03:13:14.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.514+0000 7fc9c26d9740 10 client.0 _ll_register_callbacks cb 0x55866904b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:14.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.514+0000 7fc9bd89b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586693d8b40 0x5586693df040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:14.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.514+0000 7fc9bd89b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586693d8b40 0x5586693df040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46082/0 (socket says 172.21.15.116:46082) 2024-09-12T03:13:14.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.514+0000 7fc9bd89b640 1 -- 172.21.15.116:0/3168416545 learned_addr learned my addr 172.21.15.116:0/3168416545 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:14.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.514+0000 7fc9be09c640 1 --2- 172.21.15.116:0/3168416545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586693d75d0 0x5586693e1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:14.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.514+0000 7fc9bd89b640 1 -- 172.21.15.116:0/3168416545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5586693e2370 msgr2=0x5586693df650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:14.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.514+0000 7fc9be89d640 1 --2- 172.21.15.116:0/3168416545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5586693e2370 0x5586693df650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:14.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.514+0000 7fc9bd89b640 1 --2- 172.21.15.116:0/3168416545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5586693e2370 0x5586693df650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.514+0000 7fc9bd89b640 1 -- 172.21.15.116:0/3168416545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586693d75d0 msgr2=0x5586693e1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:14.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.514+0000 7fc9bd89b640 1 --2- 172.21.15.116:0/3168416545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586693d75d0 0x5586693e1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.514+0000 7fc9bd89b640 1 -- 172.21.15.116:0/3168416545 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586690ae3c0 con 0x5586693d8b40 2024-09-12T03:13:14.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.514+0000 7fc9be09c640 1 --2- 172.21.15.116:0/3168416545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586693d75d0 0x5586693e1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:13:14.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.514+0000 7fc9be89d640 1 --2- 172.21.15.116:0/3168416545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5586693e2370 0x5586693df650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:13:14.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.515+0000 7fc9bd89b640 1 --2- 172.21.15.116:0/3168416545 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586693d8b40 0x5586693df040 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fc9a80cc8b0 tx=0x7fc9a80d4fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343ee941cb8cebf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:14.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.515+0000 7fc9a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:14.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.515+0000 7fc9a1ffb640 1 -- 172.21.15.116:0/3168416545 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a809ade0 con 0x5586693d8b40 2024-09-12T03:13:14.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.515+0000 7fc9a1ffb640 1 -- 172.21.15.116:0/3168416545 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fc9a800e350 con 0x5586693d8b40 2024-09-12T03:13:14.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.515+0000 7fc9c26d9740 1 -- 172.21.15.116:0/3168416545 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9a80b6070 con 0x5586693d8b40 2024-09-12T03:13:14.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.515+0000 7fc9a1ffb640 1 -- 172.21.15.116:0/3168416545 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc9a809ac00 con 0x5586693d8b40 2024-09-12T03:13:14.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.515+0000 7fc9c26d9740 10 client.18198 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:14.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.515+0000 7fc9c26d9740 1 -- 172.21.15.116:0/3168416545 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5586693f0b40 con 0x5586693d8b40 2024-09-12T03:13:14.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.515+0000 7fc9a1ffb640 1 -- 172.21.15.116:0/3168416545 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9a80b6070 con 0x5586693d8b40 2024-09-12T03:13:14.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.515+0000 7fc9a1ffb640 1 -- 172.21.15.116:0/3168416545 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc9a809b500 con 0x5586693d8b40 2024-09-12T03:13:14.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.515+0000 7fc9c26d9740 10 client.18198 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:14.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.515+0000 7fc9c26d9740 10 client.18198 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:13:14.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.515+0000 7fc9c26d9740 20 client.18198 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:14.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.515+0000 7fc9c26d9740 1 --2- 172.21.15.116:0/3168416545 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5586693f1e90 0x5586693f4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:14.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.515+0000 7fc9c26d9740 4 client.18198 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:13:14.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.515+0000 7fc9c26d9740 1 -- 172.21.15.116:0/3168416545 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55866934a860 con 0x5586693f1e90 2024-09-12T03:13:14.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.516+0000 7fc9be09c640 1 --2- 172.21.15.116:0/3168416545 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5586693f1e90 0x5586693f4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:14.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.516+0000 7fc9a1ffb640 10 client.18198 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:13:14.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.516+0000 7fc9be09c640 1 --2- 172.21.15.116:0/3168416545 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5586693f1e90 0x5586693f4280 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c919f857eb9a3035 server_cookie=17a66244fa6210db in_seq=0 out_seq=0 2024-09-12T03:13:14.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.517+0000 7fc9a1ffb640 1 -- 172.21.15.116:0/3168416545 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55866934a860 con 0x5586693f1e90 2024-09-12T03:13:14.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.517+0000 7fc9a1ffb640 10 client.18198 handle_command_reply: tid=0 2024-09-12T03:13:14.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.517+0000 7fc9c26d9740 1 client.18198 shutdown 2024-09-12T03:13:14.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.517+0000 7fc9c26d9740 1 -- 172.21.15.116:0/3168416545 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586693d8b40 msgr2=0x5586693df040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:14.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.517+0000 7fc9c26d9740 1 --2- 172.21.15.116:0/3168416545 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586693d8b40 0x5586693df040 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fc9a80cc8b0 tx=0x7fc9a80d4fb0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.517+0000 7fc9c26d9740 1 -- 172.21.15.116:0/3168416545 shutdown_connections 2024-09-12T03:13:14.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.517+0000 7fc9c26d9740 1 --2- 172.21.15.116:0/3168416545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5586693e2370 0x5586693df650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.517+0000 7fc9c26d9740 1 --2- 172.21.15.116:0/3168416545 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5586693f1e90 0x5586693f4280 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.557 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:13:14.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.517+0000 7fc9c26d9740 1 --2- 172.21.15.116:0/3168416545 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586693d8b40 0x5586693df040 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.517+0000 7fc9a1ffb640 0 client.18198 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:13:14.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.517+0000 7fc9c26d9740 1 --2- 172.21.15.116:0/3168416545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586693d75d0 0x5586693e1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.517+0000 7fc9c26d9740 1 -- 172.21.15.116:0/3168416545 >> 172.21.15.116:0/3168416545 conn(0x5586693a0310 msgr2=0x5586693d9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:14.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.517+0000 7fc9c26d9740 1 -- 172.21.15.116:0/3168416545 shutdown_connections 2024-09-12T03:13:14.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.517+0000 7fc9c26d9740 1 -- 172.21.15.116:0/3168416545 wait complete. 2024-09-12T03:13:14.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.517+0000 7fc9c26d9740 20 client.18198 trim_cache size 0 max 16384 2024-09-12T03:13:14.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.530+0000 7fc9c26d9740 1 Processor -- start 2024-09-12T03:13:14.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.530+0000 7fc9c26d9740 1 -- start start 2024-09-12T03:13:14.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.530+0000 7fc9c26d9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5586693d75d0 0x5586693ee490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:14.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.530+0000 7fc9c26d9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586693d8b40 0x5586693d8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:14.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.530+0000 7fc9c26d9740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586693e2370 0x5586693de900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:14.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.530+0000 7fc9c26d9740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b81afe40 con 0x5586693e2370 2024-09-12T03:13:14.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.530+0000 7fc9c26d9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b81ad710 con 0x5586693d8b40 2024-09-12T03:13:14.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.530+0000 7fc9c26d9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b81ab560 con 0x5586693d75d0 2024-09-12T03:13:14.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.530+0000 7fc9be89d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586693e2370 0x5586693de900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:14.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.530+0000 7fc9be89d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586693e2370 0x5586693de900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46090/0 (socket says 172.21.15.116:46090) 2024-09-12T03:13:14.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.530+0000 7fc9be89d640 1 -- 172.21.15.116:0/141697478 learned_addr learned my addr 172.21.15.116:0/141697478 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:14.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.531+0000 7fc9be09c640 1 --2- 172.21.15.116:0/141697478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5586693d75d0 0x5586693ee490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:14.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.531+0000 7fc9be89d640 1 -- 172.21.15.116:0/141697478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5586693d75d0 msgr2=0x5586693ee490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:14.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.531+0000 7fc9be89d640 1 --2- 172.21.15.116:0/141697478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5586693d75d0 0x5586693ee490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.531+0000 7fc9be89d640 1 -- 172.21.15.116:0/141697478 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586693d8b40 msgr2=0x5586693d8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:14.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.531+0000 7fc9bd89b640 1 --2- 172.21.15.116:0/141697478 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586693d8b40 0x5586693d8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:14.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.531+0000 7fc9be89d640 1 --2- 172.21.15.116:0/141697478 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586693d8b40 0x5586693d8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.531+0000 7fc9be89d640 1 -- 172.21.15.116:0/141697478 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586693d4d80 con 0x5586693e2370 2024-09-12T03:13:14.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.531+0000 7fc9be89d640 1 --2- 172.21.15.116:0/141697478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586693e2370 0x5586693de900 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fc9b001e520 tx=0x7fc9b0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368164a777e98a20 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:14.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.531+0000 7fc9a3fff640 1 -- 172.21.15.116:0/141697478 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b002c070 con 0x5586693e2370 2024-09-12T03:13:14.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.531+0000 7fc9a3fff640 1 -- 172.21.15.116:0/141697478 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9b001e0d0 con 0x5586693e2370 2024-09-12T03:13:14.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.531+0000 7fc9a3fff640 1 -- 172.21.15.116:0/141697478 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b001c5a0 con 0x5586693e2370 2024-09-12T03:13:14.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.532+0000 7fc9c26d9740 1 -- 172.21.15.116:0/141697478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586693e2370 msgr2=0x5586693de900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:14.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.532+0000 7fc9c26d9740 1 --2- 172.21.15.116:0/141697478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586693e2370 0x5586693de900 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fc9b001e520 tx=0x7fc9b0016000 comp rx=0 tx=0).stop 2024-09-12T03:13:14.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.532+0000 7fc9c26d9740 1 -- 172.21.15.116:0/141697478 shutdown_connections 2024-09-12T03:13:14.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.532+0000 7fc9c26d9740 1 --2- 172.21.15.116:0/141697478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586693e2370 0x5586693de900 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.532+0000 7fc9c26d9740 1 --2- 172.21.15.116:0/141697478 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586693d8b40 0x5586693d8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.532+0000 7fc9c26d9740 1 --2- 172.21.15.116:0/141697478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5586693d75d0 0x5586693ee490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.532+0000 7fc9c26d9740 1 -- 172.21.15.116:0/141697478 >> 172.21.15.116:0/141697478 conn(0x5586693a0310 msgr2=0x5586693dd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:14.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.532+0000 7fc9c26d9740 1 -- 172.21.15.116:0/141697478 shutdown_connections 2024-09-12T03:13:14.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.532+0000 7fc9c26d9740 1 -- 172.21.15.116:0/141697478 wait complete. 2024-09-12T03:13:14.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.532+0000 7fc9c26d9740 1 -- start start 2024-09-12T03:13:14.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.532+0000 7fc9c26d9740 1 Processor -- start 2024-09-12T03:13:14.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.532+0000 7fc9c26d9740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586693d75d0 0x5586693f4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:14.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.532+0000 7fc9c26d9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586693d8b40 0x5586693f4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:14.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.532+0000 7fc9c26d9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5586693e2370 0x55866945a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:14.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.532+0000 7fc9c26d9740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b81abbd0 con 0x5586693d75d0 2024-09-12T03:13:14.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.532+0000 7fc9c26d9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b81ad040 con 0x5586693d8b40 2024-09-12T03:13:14.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.532+0000 7fc9c26d9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b81ac2a0 con 0x5586693e2370 2024-09-12T03:13:14.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.532+0000 7fc9c26d9740 10 client.0 _ll_register_callbacks cb 0x7fc9b00099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:14.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.532+0000 7fc9be09c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586693d75d0 0x5586693f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:14.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.533+0000 7fc9be09c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586693d75d0 0x5586693f4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46100/0 (socket says 172.21.15.116:46100) 2024-09-12T03:13:14.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.533+0000 7fc9be09c640 1 -- 172.21.15.116:0/2114802597 learned_addr learned my addr 172.21.15.116:0/2114802597 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:14.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.533+0000 7fc9bd89b640 1 --2- 172.21.15.116:0/2114802597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586693d8b40 0x5586693f4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:14.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.533+0000 7fc9be89d640 1 --2- 172.21.15.116:0/2114802597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5586693e2370 0x55866945a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:14.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.533+0000 7fc9be09c640 1 -- 172.21.15.116:0/2114802597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5586693e2370 msgr2=0x55866945a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:14.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.533+0000 7fc9be09c640 1 --2- 172.21.15.116:0/2114802597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5586693e2370 0x55866945a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.533+0000 7fc9be09c640 1 -- 172.21.15.116:0/2114802597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586693d8b40 msgr2=0x5586693f4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:14.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.533+0000 7fc9be09c640 1 --2- 172.21.15.116:0/2114802597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586693d8b40 0x5586693f4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.533+0000 7fc9be09c640 1 -- 172.21.15.116:0/2114802597 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55866945ad80 con 0x5586693d75d0 2024-09-12T03:13:14.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.533+0000 7fc9be09c640 1 --2- 172.21.15.116:0/2114802597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586693d75d0 0x5586693f4820 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4033000 tx=0x7fc9b4032f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73cfce12d0c4d6d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:14.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.533+0000 7fc9a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:14.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.533+0000 7fc9a1ffb640 1 -- 172.21.15.116:0/2114802597 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b404a060 con 0x5586693d75d0 2024-09-12T03:13:14.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.533+0000 7fc9a1ffb640 1 -- 172.21.15.116:0/2114802597 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fc9b4004480 con 0x5586693d75d0 2024-09-12T03:13:14.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.533+0000 7fc9c26d9740 1 -- 172.21.15.116:0/2114802597 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9b001e0d0 con 0x5586693d75d0 2024-09-12T03:13:14.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.533+0000 7fc9a1ffb640 1 -- 172.21.15.116:0/2114802597 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc9b4035040 con 0x5586693d75d0 2024-09-12T03:13:14.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.533+0000 7fc9c26d9740 10 client.18204 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:14.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.533+0000 7fc9c26d9740 1 -- 172.21.15.116:0/2114802597 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55866945c0a0 con 0x5586693d75d0 2024-09-12T03:13:14.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.533+0000 7fc9a1ffb640 1 -- 172.21.15.116:0/2114802597 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9b001e0d0 con 0x5586693d75d0 2024-09-12T03:13:14.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.534+0000 7fc9a1ffb640 1 -- 172.21.15.116:0/2114802597 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc9b403a040 con 0x5586693d75d0 2024-09-12T03:13:14.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.534+0000 7fc9c26d9740 10 client.18204 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:14.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.534+0000 7fc9c26d9740 10 client.18204 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:13:14.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.534+0000 7fc9c26d9740 20 client.18204 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:14.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.534+0000 7fc9c26d9740 1 --2- 172.21.15.116:0/2114802597 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5586693f1e90 0x55866945e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:14.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.534+0000 7fc9c26d9740 4 client.18204 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:13:14.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.534+0000 7fc9c26d9740 1 -- 172.21.15.116:0/2114802597 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558669344d70 con 0x5586693f1e90 2024-09-12T03:13:14.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.534+0000 7fc9bd89b640 1 --2- 172.21.15.116:0/2114802597 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5586693f1e90 0x55866945e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:14.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.534+0000 7fc9a1ffb640 10 client.18204 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:13:14.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.534+0000 7fc9bd89b640 1 --2- 172.21.15.116:0/2114802597 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5586693f1e90 0x55866945e7e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7acbd99acf48fe02 server_cookie=c4a4a9931a6297b0 in_seq=0 out_seq=0 2024-09-12T03:13:14.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.534+0000 7fc9a1ffb640 1 -- 172.21.15.116:0/2114802597 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558669344d70 con 0x5586693f1e90 2024-09-12T03:13:14.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.534+0000 7fc9a1ffb640 10 client.18204 handle_command_reply: tid=0 2024-09-12T03:13:14.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.534+0000 7fc9c26d9740 1 client.18204 shutdown 2024-09-12T03:13:14.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.535+0000 7fc9c26d9740 1 -- 172.21.15.116:0/2114802597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586693d75d0 msgr2=0x5586693f4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:14.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.535+0000 7fc9c26d9740 1 --2- 172.21.15.116:0/2114802597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586693d75d0 0x5586693f4820 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4033000 tx=0x7fc9b4032f50 comp rx=0 tx=0).stop 2024-09-12T03:13:14.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.535+0000 7fc9c26d9740 1 -- 172.21.15.116:0/2114802597 shutdown_connections 2024-09-12T03:13:14.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.535+0000 7fc9c26d9740 1 --2- 172.21.15.116:0/2114802597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5586693e2370 0x55866945a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.535+0000 7fc9c26d9740 1 --2- 172.21.15.116:0/2114802597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5586693d8b40 0x5586693f4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.535+0000 7fc9c26d9740 1 --2- 172.21.15.116:0/2114802597 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5586693f1e90 0x55866945e7e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.535+0000 7fc9c26d9740 1 --2- 172.21.15.116:0/2114802597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586693d75d0 0x5586693f4820 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.535+0000 7fc9c26d9740 1 -- 172.21.15.116:0/2114802597 >> 172.21.15.116:0/2114802597 conn(0x5586693a0310 msgr2=0x5586693de130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:14.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.535+0000 7fc9a1ffb640 0 client.18204 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:13:14.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.535+0000 7fc9c26d9740 1 -- 172.21.15.116:0/2114802597 shutdown_connections 2024-09-12T03:13:14.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.535+0000 7fc9c26d9740 1 -- 172.21.15.116:0/2114802597 wait complete. 2024-09-12T03:13:14.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.535+0000 7fc9c26d9740 20 client.18204 trim_cache size 0 max 16384 2024-09-12T03:13:14.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.535+0000 7fc9c0300640 1 -- 172.21.15.116:0/1347447813 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc9a40a2b80 msgr2=0x7fc9a40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:14.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.535+0000 7fc9c0300640 1 --2- 172.21.15.116:0/1347447813 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc9a40a2b80 0x7fc9a40a5040 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0005fd0 tx=0x7fc9b0009500 comp rx=0 tx=0).stop 2024-09-12T03:13:14.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.535+0000 7fc9c0300640 1 -- 172.21.15.116:0/1347447813 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9b8103660 msgr2=0x7fc9b8198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:14.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.535+0000 7fc9c0300640 1 --2- 172.21.15.116:0/1347447813 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9b8103660 0x7fc9b8198970 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800d950 tx=0x7fc9a800de10 comp rx=0 tx=0).stop 2024-09-12T03:13:14.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.535+0000 7fc9c0300640 1 -- 172.21.15.116:0/1347447813 shutdown_connections 2024-09-12T03:13:14.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.535+0000 7fc9c0300640 1 --2- 172.21.15.116:0/1347447813 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc9a40a2b80 0x7fc9a40a5040 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.535+0000 7fc9c0300640 1 --2- 172.21.15.116:0/1347447813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9b810c170 0x7fc9b819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.535+0000 7fc9c0300640 1 --2- 172.21.15.116:0/1347447813 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9b8103660 0x7fc9b8198970 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.535+0000 7fc9c0300640 1 --2- 172.21.15.116:0/1347447813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9b8102d40 0x7fc9b8198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:14.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.535+0000 7fc9c0300640 1 -- 172.21.15.116:0/1347447813 >> 172.21.15.116:0/1347447813 conn(0x7fc9b80fc7a0 msgr2=0x7fc9b80fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:14.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.535+0000 7fc9c0300640 1 -- 172.21.15.116:0/1347447813 shutdown_connections 2024-09-12T03:13:14.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:14.535+0000 7fc9c0300640 1 -- 172.21.15.116:0/1347447813 wait complete. 2024-09-12T03:13:15.356 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '904ad12d', quiesce took 5.9 seconds. Will release after: 16.65 2024-09-12T03:13:15.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:15 smithi116 ceph-mon[31410]: pgmap v646: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-09-12T03:13:15.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:15 smithi183 ceph-mon[35535]: pgmap v646: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-09-12T03:13:15.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:15 smithi177 ceph-mon[35305]: pgmap v646: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-09-12T03:13:15.853 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:13:16.105 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:13:16.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.603+0000 7f677f2e1640 1 -- 172.21.15.116:0/2302158375 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6778101420 msgr2=0x7f677810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:16.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.603+0000 7f677f2e1640 1 --2- 172.21.15.116:0/2302158375 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6778101420 0x7f677810d950 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f6768009cd0 tx=0x7f676802a8e0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.603+0000 7f677f2e1640 1 -- 172.21.15.116:0/2302158375 shutdown_connections 2024-09-12T03:13:16.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.603+0000 7f677f2e1640 1 --2- 172.21.15.116:0/2302158375 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6778101420 0x7f677810d950 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.603+0000 7f677f2e1640 1 --2- 172.21.15.116:0/2302158375 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6778100a80 0x7f6778100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.603+0000 7f677f2e1640 1 --2- 172.21.15.116:0/2302158375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6778106a80 0x7f6778106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.603+0000 7f677f2e1640 1 -- 172.21.15.116:0/2302158375 >> 172.21.15.116:0/2302158375 conn(0x7f67780fc7a0 msgr2=0x7f67780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:16.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.603+0000 7f677f2e1640 1 -- 172.21.15.116:0/2302158375 shutdown_connections 2024-09-12T03:13:16.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.603+0000 7f677f2e1640 1 -- 172.21.15.116:0/2302158375 wait complete. 2024-09-12T03:13:16.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.603+0000 7f677f2e1640 1 Processor -- start 2024-09-12T03:13:16.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.603+0000 7f677f2e1640 1 -- start start 2024-09-12T03:13:16.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.604+0000 7f677f2e1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6778100a80 0x7f6778197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:16.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.604+0000 7f677f2e1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6778101420 0x7f6778197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:16.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.604+0000 7f677f2e1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6778106a80 0x7f677819bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:16.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.604+0000 7f677f2e1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6778110e80 con 0x7f6778101420 2024-09-12T03:13:16.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.604+0000 7f677f2e1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6778110d00 con 0x7f6778106a80 2024-09-12T03:13:16.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.604+0000 7f677f2e1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6778111000 con 0x7f6778100a80 2024-09-12T03:13:16.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.604+0000 7f677c87c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6778101420 0x7f6778197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:16.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.604+0000 7f677c87c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6778101420 0x7f6778197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46118/0 (socket says 172.21.15.116:46118) 2024-09-12T03:13:16.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.604+0000 7f677c87c640 1 -- 172.21.15.116:0/2378111833 learned_addr learned my addr 172.21.15.116:0/2378111833 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:16.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.604+0000 7f677d07d640 1 --2- 172.21.15.116:0/2378111833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6778100a80 0x7f6778197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:16.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.604+0000 7f677d87e640 1 --2- 172.21.15.116:0/2378111833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6778106a80 0x7f677819bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:16.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.604+0000 7f677c87c640 1 -- 172.21.15.116:0/2378111833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6778100a80 msgr2=0x7f6778197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:16.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.604+0000 7f677c87c640 1 --2- 172.21.15.116:0/2378111833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6778100a80 0x7f6778197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.604+0000 7f677c87c640 1 -- 172.21.15.116:0/2378111833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6778106a80 msgr2=0x7f677819bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:16.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.604+0000 7f677c87c640 1 --2- 172.21.15.116:0/2378111833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6778106a80 0x7f677819bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.604+0000 7f677c87c640 1 -- 172.21.15.116:0/2378111833 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677819c2b0 con 0x7f6778101420 2024-09-12T03:13:16.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.604+0000 7f677d87e640 1 --2- 172.21.15.116:0/2378111833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6778106a80 0x7f677819bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:13:16.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.604+0000 7f677c87c640 1 --2- 172.21.15.116:0/2378111833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6778101420 0x7f6778197840 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f676c009970 tx=0x7f676c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:16.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.604+0000 7f67767fc640 1 -- 172.21.15.116:0/2378111833 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c00d020 con 0x7f6778101420 2024-09-12T03:13:16.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.604+0000 7f67767fc640 1 -- 172.21.15.116:0/2378111833 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f676c007d40 con 0x7f6778101420 2024-09-12T03:13:16.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.604+0000 7f67767fc640 1 -- 172.21.15.116:0/2378111833 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c00ccc0 con 0x7f6778101420 2024-09-12T03:13:16.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.604+0000 7f677f2e1640 1 -- 172.21.15.116:0/2378111833 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677819c5a0 con 0x7f6778101420 2024-09-12T03:13:16.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.604+0000 7f677f2e1640 1 -- 172.21.15.116:0/2378111833 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f677806a790 con 0x7f6778101420 2024-09-12T03:13:16.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.607+0000 7f67767fc640 1 -- 172.21.15.116:0/2378111833 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f676c00c460 con 0x7f6778101420 2024-09-12T03:13:16.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.607+0000 7f67816ba740 1 Processor -- start 2024-09-12T03:13:16.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.607+0000 7f67816ba740 1 -- start start 2024-09-12T03:13:16.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.607+0000 7f67816ba740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557774bbfb40 0x557774bbe090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:16.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.607+0000 7f67816ba740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557774bbe5d0 0x557774bbea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:16.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.607+0000 7f67816ba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557774bc9370 0x557774bcb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:16.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.607+0000 7f67816ba740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557774b242f0 con 0x557774bbe5d0 2024-09-12T03:13:16.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.607+0000 7f67816ba740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557774acbcf0 con 0x557774bbfb40 2024-09-12T03:13:16.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.607+0000 7f67816ba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557774b202b0 con 0x557774bc9370 2024-09-12T03:13:16.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.608+0000 7f67767fc640 1 --2- 172.21.15.116:0/2378111833 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f67600a2b80 0x7f67600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:16.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.608+0000 7f677d07d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557774bbe5d0 0x557774bbea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:16.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.608+0000 7f677d07d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557774bbe5d0 0x557774bbea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46120/0 (socket says 172.21.15.116:46120) 2024-09-12T03:13:16.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.608+0000 7f677d07d640 1 -- 172.21.15.116:0/2931334907 learned_addr learned my addr 172.21.15.116:0/2931334907 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:16.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.608+0000 7f677d87e640 1 --2- 172.21.15.116:0/2931334907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557774bbfb40 0x557774bbe090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:16.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.608+0000 7f677c87c640 1 --2- 172.21.15.116:0/2931334907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557774bc9370 0x557774bcb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:16.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.608+0000 7f677d87e640 1 --2- 172.21.15.116:0/2378111833 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f67600a2b80 0x7f67600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:16.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.608+0000 7f67767fc640 1 -- 172.21.15.116:0/2378111833 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f676c0d04a0 con 0x7f6778101420 2024-09-12T03:13:16.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.608+0000 7f677d07d640 1 -- 172.21.15.116:0/2931334907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557774bc9370 msgr2=0x557774bcb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:16.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.608+0000 7f677d07d640 1 --2- 172.21.15.116:0/2931334907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557774bc9370 0x557774bcb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.608+0000 7f677d07d640 1 -- 172.21.15.116:0/2931334907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557774bbfb40 msgr2=0x557774bbe090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:16.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.608+0000 7f677d07d640 1 --2- 172.21.15.116:0/2931334907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557774bbfb40 0x557774bbe090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.608+0000 7f677d07d640 1 -- 172.21.15.116:0/2931334907 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557774ab03a0 con 0x557774bbe5d0 2024-09-12T03:13:16.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.608+0000 7f677c87c640 1 --2- 172.21.15.116:0/2931334907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557774bc9370 0x557774bcb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:13:16.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.608+0000 7f677d07d640 1 --2- 172.21.15.116:0/2931334907 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557774bbe5d0 0x557774bbea10 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f677000b710 tx=0x7f6770004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0b6c03a994a925 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:16.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.608+0000 7f677d87e640 1 --2- 172.21.15.116:0/2378111833 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f67600a2b80 0x7f67600a5040 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6768009a60 tx=0x7f67680095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:16.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.608+0000 7f675f7fe640 1 -- 172.21.15.116:0/2931334907 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6770003bd0 con 0x557774bbe5d0 2024-09-12T03:13:16.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.608+0000 7f675f7fe640 1 -- 172.21.15.116:0/2931334907 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6770015070 con 0x557774bbe5d0 2024-09-12T03:13:16.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.608+0000 7f675f7fe640 1 -- 172.21.15.116:0/2931334907 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6770004e00 con 0x557774bbe5d0 2024-09-12T03:13:16.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.609+0000 7f67816ba740 1 -- 172.21.15.116:0/2931334907 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557774bbe5d0 msgr2=0x557774bbea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:16.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.609+0000 7f67816ba740 1 --2- 172.21.15.116:0/2931334907 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557774bbe5d0 0x557774bbea10 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f677000b710 tx=0x7f6770004610 comp rx=0 tx=0).stop 2024-09-12T03:13:16.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.609+0000 7f67816ba740 1 -- 172.21.15.116:0/2931334907 shutdown_connections 2024-09-12T03:13:16.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.609+0000 7f67816ba740 1 --2- 172.21.15.116:0/2931334907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557774bc9370 0x557774bcb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.609+0000 7f67816ba740 1 --2- 172.21.15.116:0/2931334907 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557774bbe5d0 0x557774bbea10 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.609+0000 7f67816ba740 1 --2- 172.21.15.116:0/2931334907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557774bbfb40 0x557774bbe090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.609+0000 7f67816ba740 1 -- 172.21.15.116:0/2931334907 >> 172.21.15.116:0/2931334907 conn(0x557774b87310 msgr2=0x557774bbac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:16.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.609+0000 7f67816ba740 1 -- 172.21.15.116:0/2931334907 shutdown_connections 2024-09-12T03:13:16.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.609+0000 7f67816ba740 1 -- 172.21.15.116:0/2931334907 wait complete. 2024-09-12T03:13:16.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.610+0000 7f67816ba740 1 -- start start 2024-09-12T03:13:16.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.610+0000 7f67816ba740 1 Processor -- start 2024-09-12T03:13:16.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.610+0000 7f67816ba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557774bbe5d0 0x557774bc8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:16.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.610+0000 7f67816ba740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557774bbfb40 0x557774bc5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:16.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.610+0000 7f67816ba740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557774bc9370 0x557774bc65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:16.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.610+0000 7f67816ba740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557774bcc070 con 0x557774bc9370 2024-09-12T03:13:16.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.610+0000 7f67816ba740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557774adf060 con 0x557774bbfb40 2024-09-12T03:13:16.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.610+0000 7f67816ba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557774bcc1f0 con 0x557774bbe5d0 2024-09-12T03:13:16.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.610+0000 7f67816ba740 10 client.0 _ll_register_callbacks cb 0x5577748328c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:16.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.610+0000 7f677d87e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557774bc9370 0x557774bc65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:16.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.610+0000 7f677d87e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557774bc9370 0x557774bc65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46124/0 (socket says 172.21.15.116:46124) 2024-09-12T03:13:16.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.610+0000 7f677d87e640 1 -- 172.21.15.116:0/3442152839 learned_addr learned my addr 172.21.15.116:0/3442152839 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:16.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.610+0000 7f677c87c640 1 --2- 172.21.15.116:0/3442152839 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557774bbfb40 0x557774bc5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:16.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.610+0000 7f677d07d640 1 --2- 172.21.15.116:0/3442152839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557774bbe5d0 0x557774bc8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:16.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.610+0000 7f677d87e640 1 -- 172.21.15.116:0/3442152839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557774bbe5d0 msgr2=0x557774bc8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:16.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.610+0000 7f677d87e640 1 --2- 172.21.15.116:0/3442152839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557774bbe5d0 0x557774bc8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.610+0000 7f677d87e640 1 -- 172.21.15.116:0/3442152839 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557774bbfb40 msgr2=0x557774bc5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:16.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.610+0000 7f677d87e640 1 --2- 172.21.15.116:0/3442152839 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557774bbfb40 0x557774bc5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.611+0000 7f677d87e640 1 -- 172.21.15.116:0/3442152839 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577748953c0 con 0x557774bc9370 2024-09-12T03:13:16.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.611+0000 7f677d87e640 1 --2- 172.21.15.116:0/3442152839 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557774bc9370 0x557774bc65e0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f6768035000 tx=0x7f676802a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1982a4ed3a2a97a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:16.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.611+0000 7f675cff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:16.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.611+0000 7f675cff9640 1 -- 172.21.15.116:0/3442152839 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768051070 con 0x557774bc9370 2024-09-12T03:13:16.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.611+0000 7f675cff9640 1 -- 172.21.15.116:0/3442152839 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f6768004240 con 0x557774bc9370 2024-09-12T03:13:16.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.611+0000 7f67816ba740 1 -- 172.21.15.116:0/3442152839 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6770015070 con 0x557774bc9370 2024-09-12T03:13:16.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.611+0000 7f675cff9640 1 -- 172.21.15.116:0/3442152839 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f676805d050 con 0x557774bc9370 2024-09-12T03:13:16.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.611+0000 7f67816ba740 10 client.18216 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:16.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.611+0000 7f67816ba740 1 -- 172.21.15.116:0/3442152839 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557774bd7aa0 con 0x557774bc9370 2024-09-12T03:13:16.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.611+0000 7f675cff9640 1 -- 172.21.15.116:0/3442152839 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6770015070 con 0x557774bc9370 2024-09-12T03:13:16.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.611+0000 7f675cff9640 1 -- 172.21.15.116:0/3442152839 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6768030070 con 0x557774bc9370 2024-09-12T03:13:16.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.611+0000 7f67816ba740 10 client.18216 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:16.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.611+0000 7f67816ba740 10 client.18216 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:13:16.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.611+0000 7f67816ba740 20 client.18216 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:16.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.611+0000 7f67816ba740 1 --2- 172.21.15.116:0/3442152839 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x557774bd8eb0 0x557774bdb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:16.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.611+0000 7f67816ba740 4 client.18216 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:13:16.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.611+0000 7f67816ba740 1 -- 172.21.15.116:0/3442152839 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557774b31860 con 0x557774bd8eb0 2024-09-12T03:13:16.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.612+0000 7f677d07d640 1 --2- 172.21.15.116:0/3442152839 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x557774bd8eb0 0x557774bdb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:16.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.612+0000 7f675cff9640 10 client.18216 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:13:16.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.612+0000 7f677d07d640 1 --2- 172.21.15.116:0/3442152839 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x557774bd8eb0 0x557774bdb370 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1657cf606480abf7 server_cookie=38eebbb81f126a88 in_seq=0 out_seq=0 2024-09-12T03:13:16.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.613+0000 7f675cff9640 1 -- 172.21.15.116:0/3442152839 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557774b31860 con 0x557774bd8eb0 2024-09-12T03:13:16.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.613+0000 7f675cff9640 10 client.18216 handle_command_reply: tid=0 2024-09-12T03:13:16.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.613+0000 7f67816ba740 1 client.18216 shutdown 2024-09-12T03:13:16.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.613+0000 7f67816ba740 1 -- 172.21.15.116:0/3442152839 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557774bc9370 msgr2=0x557774bc65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:16.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.613+0000 7f67816ba740 1 --2- 172.21.15.116:0/3442152839 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557774bc9370 0x557774bc65e0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f6768035000 tx=0x7f676802a8e0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.613+0000 7f67816ba740 1 -- 172.21.15.116:0/3442152839 shutdown_connections 2024-09-12T03:13:16.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.613+0000 7f67816ba740 1 --2- 172.21.15.116:0/3442152839 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x557774bd8eb0 0x557774bdb370 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.613+0000 7f67816ba740 1 --2- 172.21.15.116:0/3442152839 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557774bc9370 0x557774bc65e0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.613+0000 7f67816ba740 1 --2- 172.21.15.116:0/3442152839 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557774bbfb40 0x557774bc5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.613+0000 7f67816ba740 1 --2- 172.21.15.116:0/3442152839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557774bbe5d0 0x557774bc8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.613+0000 7f675cff9640 0 client.18216 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:13:16.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.613+0000 7f67816ba740 1 -- 172.21.15.116:0/3442152839 >> 172.21.15.116:0/3442152839 conn(0x557774b87310 msgr2=0x557774bc0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:16.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.613+0000 7f67816ba740 1 -- 172.21.15.116:0/3442152839 shutdown_connections 2024-09-12T03:13:16.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.613+0000 7f67816ba740 1 -- 172.21.15.116:0/3442152839 wait complete. 2024-09-12T03:13:16.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.614+0000 7f67816ba740 20 client.18216 trim_cache size 0 max 16384 2024-09-12T03:13:16.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.626+0000 7f67816ba740 1 Processor -- start 2024-09-12T03:13:16.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.626+0000 7f67816ba740 1 -- start start 2024-09-12T03:13:16.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.627+0000 7f67816ba740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557774bbe5d0 0x557774bbae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:16.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.627+0000 7f67816ba740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557774bbfb40 0x557774bd5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:16.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.627+0000 7f67816ba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557774bc9370 0x557774bc1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:16.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.627+0000 7f67816ba740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f67781adb70 con 0x557774bbe5d0 2024-09-12T03:13:16.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.627+0000 7f67816ba740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f67781ab440 con 0x557774bbfb40 2024-09-12T03:13:16.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.627+0000 7f67816ba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f67781a92b0 con 0x557774bc9370 2024-09-12T03:13:16.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.627+0000 7f677d07d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557774bbe5d0 0x557774bbae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:16.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.627+0000 7f677d07d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557774bbe5d0 0x557774bbae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46128/0 (socket says 172.21.15.116:46128) 2024-09-12T03:13:16.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.627+0000 7f677d07d640 1 -- 172.21.15.116:0/2007300311 learned_addr learned my addr 172.21.15.116:0/2007300311 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:16.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.627+0000 7f677d87e640 1 --2- 172.21.15.116:0/2007300311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557774bc9370 0x557774bc1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:16.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.627+0000 7f677d07d640 1 -- 172.21.15.116:0/2007300311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557774bc9370 msgr2=0x557774bc1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:16.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.627+0000 7f677d07d640 1 --2- 172.21.15.116:0/2007300311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557774bc9370 0x557774bc1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.627+0000 7f677c87c640 1 --2- 172.21.15.116:0/2007300311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557774bbfb40 0x557774bd5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:16.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.627+0000 7f677d07d640 1 -- 172.21.15.116:0/2007300311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557774bbfb40 msgr2=0x557774bd5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:16.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.627+0000 7f677d07d640 1 --2- 172.21.15.116:0/2007300311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557774bbfb40 0x557774bd5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.627+0000 7f677d07d640 1 -- 172.21.15.116:0/2007300311 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557774bc6be0 con 0x557774bbe5d0 2024-09-12T03:13:16.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.627+0000 7f677d87e640 1 --2- 172.21.15.116:0/2007300311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557774bc9370 0x557774bc1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:13:16.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.627+0000 7f677c87c640 1 --2- 172.21.15.116:0/2007300311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557774bbfb40 0x557774bd5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:13:16.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.627+0000 7f677d07d640 1 --2- 172.21.15.116:0/2007300311 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557774bbe5d0 0x557774bbae10 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f67700049d0 tx=0x7f6770003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85d056de4ada619 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:16.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.627+0000 7f675effd640 1 -- 172.21.15.116:0/2007300311 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6770011440 con 0x557774bbe5d0 2024-09-12T03:13:16.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.627+0000 7f675effd640 1 -- 172.21.15.116:0/2007300311 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6770015030 con 0x557774bbe5d0 2024-09-12T03:13:16.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.627+0000 7f675effd640 1 -- 172.21.15.116:0/2007300311 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6770019660 con 0x557774bbe5d0 2024-09-12T03:13:16.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.628+0000 7f67816ba740 1 -- 172.21.15.116:0/2007300311 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557774bbe5d0 msgr2=0x557774bbae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:16.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.628+0000 7f67816ba740 1 --2- 172.21.15.116:0/2007300311 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557774bbe5d0 0x557774bbae10 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f67700049d0 tx=0x7f6770003d60 comp rx=0 tx=0).stop 2024-09-12T03:13:16.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.628+0000 7f67816ba740 1 -- 172.21.15.116:0/2007300311 shutdown_connections 2024-09-12T03:13:16.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.628+0000 7f67816ba740 1 --2- 172.21.15.116:0/2007300311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557774bc9370 0x557774bc1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.628+0000 7f67816ba740 1 --2- 172.21.15.116:0/2007300311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557774bbfb40 0x557774bd5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.628+0000 7f67816ba740 1 --2- 172.21.15.116:0/2007300311 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557774bbe5d0 0x557774bbae10 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.628+0000 7f67816ba740 1 -- 172.21.15.116:0/2007300311 >> 172.21.15.116:0/2007300311 conn(0x557774b87310 msgr2=0x557774bba700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:16.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.628+0000 7f67816ba740 1 -- 172.21.15.116:0/2007300311 shutdown_connections 2024-09-12T03:13:16.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.628+0000 7f67816ba740 1 -- 172.21.15.116:0/2007300311 wait complete. 2024-09-12T03:13:16.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.628+0000 7f67816ba740 1 -- start start 2024-09-12T03:13:16.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.628+0000 7f67816ba740 1 Processor -- start 2024-09-12T03:13:16.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.629+0000 7f67816ba740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557774bbe5d0 0x557774bc2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:16.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.629+0000 7f67816ba740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557774bbfb40 0x557774bc34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:16.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.629+0000 7f67816ba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557774bc9370 0x557774bc3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:16.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.629+0000 7f67816ba740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f67781a9900 con 0x557774bbfb40 2024-09-12T03:13:16.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.629+0000 7f67816ba740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f67781aad70 con 0x557774bbe5d0 2024-09-12T03:13:16.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.629+0000 7f67816ba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f67781a9fd0 con 0x557774bc9370 2024-09-12T03:13:16.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.629+0000 7f67816ba740 10 client.0 _ll_register_callbacks cb 0x7f6768005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:16.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.629+0000 7f677c87c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557774bbfb40 0x557774bc34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:16.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.629+0000 7f677c87c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557774bbfb40 0x557774bc34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46136/0 (socket says 172.21.15.116:46136) 2024-09-12T03:13:16.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.629+0000 7f677c87c640 1 -- 172.21.15.116:0/4227384770 learned_addr learned my addr 172.21.15.116:0/4227384770 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:16.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.629+0000 7f677d07d640 1 --2- 172.21.15.116:0/4227384770 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557774bbe5d0 0x557774bc2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:16.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.629+0000 7f677c87c640 1 -- 172.21.15.116:0/4227384770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557774bc9370 msgr2=0x557774bc3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:16.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.629+0000 7f677d87e640 1 --2- 172.21.15.116:0/4227384770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557774bc9370 0x557774bc3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:16.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.629+0000 7f677c87c640 1 --2- 172.21.15.116:0/4227384770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557774bc9370 0x557774bc3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.629+0000 7f677c87c640 1 -- 172.21.15.116:0/4227384770 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557774bbe5d0 msgr2=0x557774bc2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:16.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.629+0000 7f677c87c640 1 --2- 172.21.15.116:0/4227384770 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557774bbe5d0 0x557774bc2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.629+0000 7f677c87c640 1 -- 172.21.15.116:0/4227384770 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557774c42c60 con 0x557774bbfb40 2024-09-12T03:13:16.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.630+0000 7f677c87c640 1 --2- 172.21.15.116:0/4227384770 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557774bbfb40 0x557774bc34e0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f676c0cdfd0 tx=0x7f676c0129c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402ae670805648a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:16.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.630+0000 7f675cff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:16.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.630+0000 7f675cff9640 1 -- 172.21.15.116:0/4227384770 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c09cd30 con 0x557774bbfb40 2024-09-12T03:13:16.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.630+0000 7f675cff9640 1 -- 172.21.15.116:0/4227384770 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f676c013cd0 con 0x557774bbfb40 2024-09-12T03:13:16.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.630+0000 7f67816ba740 1 -- 172.21.15.116:0/4227384770 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6770015030 con 0x557774bbfb40 2024-09-12T03:13:16.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.630+0000 7f675cff9640 1 -- 172.21.15.116:0/4227384770 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f676c013ed0 con 0x557774bbfb40 2024-09-12T03:13:16.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.630+0000 7f67816ba740 10 client.18222 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:16.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.630+0000 7f67816ba740 1 -- 172.21.15.116:0/4227384770 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557774c43e90 con 0x557774bbfb40 2024-09-12T03:13:16.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.630+0000 7f675cff9640 1 -- 172.21.15.116:0/4227384770 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6770015030 con 0x557774bbfb40 2024-09-12T03:13:16.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.630+0000 7f675cff9640 1 -- 172.21.15.116:0/4227384770 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f676c0b3030 con 0x557774bbfb40 2024-09-12T03:13:16.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.630+0000 7f67816ba740 10 client.18222 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:16.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.630+0000 7f67816ba740 10 client.18222 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:13:16.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.630+0000 7f67816ba740 20 client.18222 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:16.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.630+0000 7f67816ba740 1 --2- 172.21.15.116:0/4227384770 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x557774c44650 0x557774c46a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:16.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.630+0000 7f67816ba740 4 client.18222 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:13:16.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.630+0000 7f67816ba740 1 -- 172.21.15.116:0/4227384770 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557774b2bd70 con 0x557774c44650 2024-09-12T03:13:16.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.631+0000 7f677d07d640 1 --2- 172.21.15.116:0/4227384770 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x557774c44650 0x557774c46a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:16.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.631+0000 7f675cff9640 10 client.18222 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:13:16.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.631+0000 7f677d07d640 1 --2- 172.21.15.116:0/4227384770 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x557774c44650 0x557774c46a40 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f64e87a4622031d6 server_cookie=a4435f6b75504d42 in_seq=0 out_seq=0 2024-09-12T03:13:16.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.631+0000 7f675cff9640 1 -- 172.21.15.116:0/4227384770 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557774b2bd70 con 0x557774c44650 2024-09-12T03:13:16.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.631+0000 7f675cff9640 10 client.18222 handle_command_reply: tid=0 2024-09-12T03:13:16.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.631+0000 7f67816ba740 1 client.18222 shutdown 2024-09-12T03:13:16.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.631+0000 7f67816ba740 1 -- 172.21.15.116:0/4227384770 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557774bbfb40 msgr2=0x557774bc34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:16.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.631+0000 7f67816ba740 1 --2- 172.21.15.116:0/4227384770 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557774bbfb40 0x557774bc34e0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f676c0cdfd0 tx=0x7f676c0129c0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.631+0000 7f67816ba740 1 -- 172.21.15.116:0/4227384770 shutdown_connections 2024-09-12T03:13:16.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.631+0000 7f67816ba740 1 --2- 172.21.15.116:0/4227384770 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x557774c44650 0x557774c46a40 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.631+0000 7f67816ba740 1 --2- 172.21.15.116:0/4227384770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557774bc9370 0x557774bc3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.631+0000 7f67816ba740 1 --2- 172.21.15.116:0/4227384770 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557774bbfb40 0x557774bc34e0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.631+0000 7f67816ba740 1 --2- 172.21.15.116:0/4227384770 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557774bbe5d0 0x557774bc2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.631+0000 7f67816ba740 1 -- 172.21.15.116:0/4227384770 >> 172.21.15.116:0/4227384770 conn(0x557774b87310 msgr2=0x557774bc7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:16.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.631+0000 7f675cff9640 0 client.18222 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:13:16.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.632+0000 7f67816ba740 1 -- 172.21.15.116:0/4227384770 shutdown_connections 2024-09-12T03:13:16.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.632+0000 7f67816ba740 1 -- 172.21.15.116:0/4227384770 wait complete. 2024-09-12T03:13:16.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.632+0000 7f67816ba740 20 client.18222 trim_cache size 0 max 16384 2024-09-12T03:13:16.644 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:13:16.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.632+0000 7f677f2e1640 1 -- 172.21.15.116:0/2378111833 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f67600a2b80 msgr2=0x7f67600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:16.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.632+0000 7f677f2e1640 1 --2- 172.21.15.116:0/2378111833 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f67600a2b80 0x7f67600a5040 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6768009a60 tx=0x7f67680095c0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.632+0000 7f677f2e1640 1 -- 172.21.15.116:0/2378111833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6778101420 msgr2=0x7f6778197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:16.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.632+0000 7f677f2e1640 1 --2- 172.21.15.116:0/2378111833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6778101420 0x7f6778197840 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f676c009970 tx=0x7f676c009e30 comp rx=0 tx=0).stop 2024-09-12T03:13:16.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.632+0000 7f677f2e1640 1 -- 172.21.15.116:0/2378111833 shutdown_connections 2024-09-12T03:13:16.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.632+0000 7f677f2e1640 1 --2- 172.21.15.116:0/2378111833 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f67600a2b80 0x7f67600a5040 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.632+0000 7f677f2e1640 1 --2- 172.21.15.116:0/2378111833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6778106a80 0x7f677819bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.632+0000 7f677f2e1640 1 --2- 172.21.15.116:0/2378111833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6778101420 0x7f6778197840 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.632+0000 7f677f2e1640 1 --2- 172.21.15.116:0/2378111833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6778100a80 0x7f6778197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.632+0000 7f677f2e1640 1 -- 172.21.15.116:0/2378111833 >> 172.21.15.116:0/2378111833 conn(0x7f67780fc7a0 msgr2=0x7f67780fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:16.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.632+0000 7f677f2e1640 1 -- 172.21.15.116:0/2378111833 shutdown_connections 2024-09-12T03:13:16.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:16.632+0000 7f677f2e1640 1 -- 172.21.15.116:0/2378111833 wait complete. 2024-09-12T03:13:17.362 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:13:17.611 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:13:17.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:17 smithi183 ceph-mon[35535]: pgmap v647: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-09-12T03:13:17.732 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:17 smithi116 ceph-mon[31410]: pgmap v647: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-09-12T03:13:17.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:17 smithi177 ceph-mon[35305]: pgmap v647: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-09-12T03:13:18.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.103+0000 7fa203474640 1 -- 172.21.15.116:0/665127593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1fc100a80 msgr2=0x7fa1fc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:18.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.103+0000 7fa203474640 1 --2- 172.21.15.116:0/665127593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1fc100a80 0x7fa1fc100ee0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009930 tx=0x7fa1ec02a420 comp rx=0 tx=0).stop 2024-09-12T03:13:18.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.103+0000 7fa203474640 1 -- 172.21.15.116:0/665127593 shutdown_connections 2024-09-12T03:13:18.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.103+0000 7fa203474640 1 --2- 172.21.15.116:0/665127593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1fc101420 0x7fa1fc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.103+0000 7fa203474640 1 --2- 172.21.15.116:0/665127593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1fc100a80 0x7fa1fc100ee0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.103+0000 7fa203474640 1 --2- 172.21.15.116:0/665127593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1fc106a80 0x7fa1fc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.103+0000 7fa203474640 1 -- 172.21.15.116:0/665127593 >> 172.21.15.116:0/665127593 conn(0x7fa1fc0fc7a0 msgr2=0x7fa1fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:18.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.103+0000 7fa203474640 1 -- 172.21.15.116:0/665127593 shutdown_connections 2024-09-12T03:13:18.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.103+0000 7fa203474640 1 -- 172.21.15.116:0/665127593 wait complete. 2024-09-12T03:13:18.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.104+0000 7fa203474640 1 Processor -- start 2024-09-12T03:13:18.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.104+0000 7fa203474640 1 -- start start 2024-09-12T03:13:18.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.104+0000 7fa203474640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1fc100a80 0x7fa1fc198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:18.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.104+0000 7fa203474640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1fc101420 0x7fa1fc198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:18.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.104+0000 7fa203474640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1fc106a80 0x7fa1fc19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:18.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.104+0000 7fa203474640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc111fd0 con 0x7fa1fc100a80 2024-09-12T03:13:18.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.104+0000 7fa203474640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc111e50 con 0x7fa1fc106a80 2024-09-12T03:13:18.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.104+0000 7fa203474640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc112150 con 0x7fa1fc101420 2024-09-12T03:13:18.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.104+0000 7fa201210640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1fc100a80 0x7fa1fc198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:18.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.104+0000 7fa201210640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1fc100a80 0x7fa1fc198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46164/0 (socket says 172.21.15.116:46164) 2024-09-12T03:13:18.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.104+0000 7fa201210640 1 -- 172.21.15.116:0/1880104970 learned_addr learned my addr 172.21.15.116:0/1880104970 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:18.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.104+0000 7fa200a0f640 1 --2- 172.21.15.116:0/1880104970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1fc101420 0x7fa1fc198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:18.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.104+0000 7fa201210640 1 -- 172.21.15.116:0/1880104970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1fc101420 msgr2=0x7fa1fc198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:18.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.104+0000 7fa201210640 1 --2- 172.21.15.116:0/1880104970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1fc101420 0x7fa1fc198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.104+0000 7fa201210640 1 -- 172.21.15.116:0/1880104970 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1fc106a80 msgr2=0x7fa1fc19cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:18.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.104+0000 7fa201210640 1 --2- 172.21.15.116:0/1880104970 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1fc106a80 0x7fa1fc19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.104+0000 7fa201210640 1 -- 172.21.15.116:0/1880104970 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1fc19d4b0 con 0x7fa1fc100a80 2024-09-12T03:13:18.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.104+0000 7fa201210640 1 --2- 172.21.15.116:0/1880104970 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1fc100a80 0x7fa1fc198500 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800b4a0 tx=0x7fa1f800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:18.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.105+0000 7fa1f27fc640 1 -- 172.21.15.116:0/1880104970 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8014070 con 0x7fa1fc100a80 2024-09-12T03:13:18.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.105+0000 7fa1f27fc640 1 -- 172.21.15.116:0/1880104970 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1f8010af0 con 0x7fa1fc100a80 2024-09-12T03:13:18.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.105+0000 7fa203474640 1 -- 172.21.15.116:0/1880104970 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1fc19d7a0 con 0x7fa1fc100a80 2024-09-12T03:13:18.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.105+0000 7fa1f27fc640 1 -- 172.21.15.116:0/1880104970 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8017d10 con 0x7fa1fc100a80 2024-09-12T03:13:18.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.105+0000 7fa203474640 1 -- 172.21.15.116:0/1880104970 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1fc1a5080 con 0x7fa1fc100a80 2024-09-12T03:13:18.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.107+0000 7fa1f27fc640 1 -- 172.21.15.116:0/1880104970 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa1f8017070 con 0x7fa1fc100a80 2024-09-12T03:13:18.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.107+0000 7fa20584d740 1 Processor -- start 2024-09-12T03:13:18.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.107+0000 7fa20584d740 1 -- start start 2024-09-12T03:13:18.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.108+0000 7fa20584d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56262cda6b40 0x56262cda5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:18.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.108+0000 7fa20584d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262cda55d0 0x56262cda5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:18.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.108+0000 7fa20584d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56262cdb4780 0x56262cdb6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:18.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.108+0000 7fa20584d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56262cd0b2f0 con 0x56262cdb4780 2024-09-12T03:13:18.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.108+0000 7fa20584d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56262ccb2cf0 con 0x56262cda55d0 2024-09-12T03:13:18.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.108+0000 7fa20584d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56262cd072b0 con 0x56262cda6b40 2024-09-12T03:13:18.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.108+0000 7fa200a0f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56262cdb4780 0x56262cdb6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:18.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.108+0000 7fa200a0f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56262cdb4780 0x56262cdb6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46170/0 (socket says 172.21.15.116:46170) 2024-09-12T03:13:18.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.108+0000 7fa200a0f640 1 -- 172.21.15.116:0/1426871096 learned_addr learned my addr 172.21.15.116:0/1426871096 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:18.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.108+0000 7fa201a11640 1 --2- 172.21.15.116:0/1426871096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56262cda6b40 0x56262cda5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:18.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.108+0000 7fa201210640 1 --2- 172.21.15.116:0/1426871096 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262cda55d0 0x56262cda5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:18.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.108+0000 7fa200a0f640 1 -- 172.21.15.116:0/1426871096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56262cda6b40 msgr2=0x56262cda5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:18.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.108+0000 7fa200a0f640 1 --2- 172.21.15.116:0/1426871096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56262cda6b40 0x56262cda5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.108+0000 7fa200a0f640 1 -- 172.21.15.116:0/1426871096 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262cda55d0 msgr2=0x56262cda5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:18.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.108+0000 7fa200a0f640 1 --2- 172.21.15.116:0/1426871096 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262cda55d0 0x56262cda5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.108+0000 7fa200a0f640 1 -- 172.21.15.116:0/1426871096 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56262cc973a0 con 0x56262cdb4780 2024-09-12T03:13:18.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.108+0000 7fa1f27fc640 1 --2- 172.21.15.116:0/1880104970 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa1e80a2b80 0x7fa1e80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:18.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.108+0000 7fa201a11640 1 --2- 172.21.15.116:0/1880104970 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa1e80a2b80 0x7fa1e80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:18.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.108+0000 7fa200a0f640 1 --2- 172.21.15.116:0/1426871096 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56262cdb4780 0x56262cdb6b70 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009930 tx=0x7fa1ec032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557f887bf9b64624 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:18.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.108+0000 7fa1f27fc640 1 -- 172.21.15.116:0/1880104970 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa1f80ce500 con 0x7fa1fc100a80 2024-09-12T03:13:18.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.109+0000 7fa1e77fe640 1 -- 172.21.15.116:0/1426871096 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec03f070 con 0x56262cdb4780 2024-09-12T03:13:18.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.109+0000 7fa1e77fe640 1 -- 172.21.15.116:0/1426871096 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1ec0317b0 con 0x56262cdb4780 2024-09-12T03:13:18.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.109+0000 7fa1e77fe640 1 -- 172.21.15.116:0/1426871096 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec035070 con 0x56262cdb4780 2024-09-12T03:13:18.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.109+0000 7fa20584d740 1 -- 172.21.15.116:0/1426871096 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56262cdb4780 msgr2=0x56262cdb6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:18.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.109+0000 7fa201a11640 1 --2- 172.21.15.116:0/1880104970 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa1e80a2b80 0x7fa1e80a5040 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4009910 tx=0x7fa1f40098a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:18.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.109+0000 7fa20584d740 1 --2- 172.21.15.116:0/1426871096 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56262cdb4780 0x56262cdb6b70 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009930 tx=0x7fa1ec032ce0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.109+0000 7fa20584d740 1 -- 172.21.15.116:0/1426871096 shutdown_connections 2024-09-12T03:13:18.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.109+0000 7fa20584d740 1 --2- 172.21.15.116:0/1426871096 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56262cdb4780 0x56262cdb6b70 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.109+0000 7fa20584d740 1 --2- 172.21.15.116:0/1426871096 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262cda55d0 0x56262cda5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.109+0000 7fa20584d740 1 --2- 172.21.15.116:0/1426871096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56262cda6b40 0x56262cda5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.109+0000 7fa20584d740 1 -- 172.21.15.116:0/1426871096 >> 172.21.15.116:0/1426871096 conn(0x56262cd6e310 msgr2=0x56262cda1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:18.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.109+0000 7fa20584d740 1 -- 172.21.15.116:0/1426871096 shutdown_connections 2024-09-12T03:13:18.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.109+0000 7fa20584d740 1 -- 172.21.15.116:0/1426871096 wait complete. 2024-09-12T03:13:18.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.110+0000 7fa20584d740 1 -- start start 2024-09-12T03:13:18.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.110+0000 7fa20584d740 1 Processor -- start 2024-09-12T03:13:18.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.110+0000 7fa20584d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56262cda55d0 0x56262cdb4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:18.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.110+0000 7fa20584d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56262cda6b40 0x56262cdb13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:18.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.110+0000 7fa20584d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262cdb4780 0x56262cdb19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:18.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.110+0000 7fa20584d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56262cdb7420 con 0x56262cda6b40 2024-09-12T03:13:18.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.110+0000 7fa20584d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56262ccc6060 con 0x56262cdb4780 2024-09-12T03:13:18.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.110+0000 7fa20584d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56262cdb75a0 con 0x56262cda55d0 2024-09-12T03:13:18.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.110+0000 7fa20584d740 10 client.0 _ll_register_callbacks cb 0x56262ca198c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:18.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.110+0000 7fa200a0f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56262cda6b40 0x56262cdb13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:18.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.110+0000 7fa200a0f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56262cda6b40 0x56262cdb13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46182/0 (socket says 172.21.15.116:46182) 2024-09-12T03:13:18.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.110+0000 7fa200a0f640 1 -- 172.21.15.116:0/3581588190 learned_addr learned my addr 172.21.15.116:0/3581588190 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:18.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.111+0000 7fa201a11640 1 --2- 172.21.15.116:0/3581588190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262cdb4780 0x56262cdb19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:18.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.111+0000 7fa201210640 1 --2- 172.21.15.116:0/3581588190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56262cda55d0 0x56262cdb4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:18.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.111+0000 7fa200a0f640 1 -- 172.21.15.116:0/3581588190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56262cda55d0 msgr2=0x56262cdb4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:18.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.111+0000 7fa200a0f640 1 --2- 172.21.15.116:0/3581588190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56262cda55d0 0x56262cdb4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.111+0000 7fa200a0f640 1 -- 172.21.15.116:0/3581588190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262cdb4780 msgr2=0x56262cdb19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:18.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.111+0000 7fa200a0f640 1 --2- 172.21.15.116:0/3581588190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262cdb4780 0x56262cdb19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.111+0000 7fa200a0f640 1 -- 172.21.15.116:0/3581588190 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56262ca7c3c0 con 0x56262cda6b40 2024-09-12T03:13:18.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.111+0000 7fa200a0f640 1 --2- 172.21.15.116:0/3581588190 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56262cda6b40 0x56262cdb13f0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec0096c0 tx=0x7fa1ec02aae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38a6863396fdc9e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:18.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.111+0000 7fa1e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:18.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.111+0000 7fa1e4ff9640 1 -- 172.21.15.116:0/3581588190 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec03f030 con 0x56262cda6b40 2024-09-12T03:13:18.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.111+0000 7fa1e4ff9640 1 -- 172.21.15.116:0/3581588190 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa1ec005510 con 0x56262cda6b40 2024-09-12T03:13:18.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.111+0000 7fa20584d740 1 -- 172.21.15.116:0/3581588190 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1ec0317b0 con 0x56262cda6b40 2024-09-12T03:13:18.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.111+0000 7fa1e4ff9640 1 -- 172.21.15.116:0/3581588190 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa1ec035040 con 0x56262cda6b40 2024-09-12T03:13:18.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.111+0000 7fa20584d740 10 client.18234 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:18.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.111+0000 7fa20584d740 1 -- 172.21.15.116:0/3581588190 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56262cdc2e80 con 0x56262cda6b40 2024-09-12T03:13:18.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.111+0000 7fa1e4ff9640 1 -- 172.21.15.116:0/3581588190 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1ec0317b0 con 0x56262cda6b40 2024-09-12T03:13:18.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.111+0000 7fa1e4ff9640 1 -- 172.21.15.116:0/3581588190 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa1ec004960 con 0x56262cda6b40 2024-09-12T03:13:18.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.112+0000 7fa20584d740 10 client.18234 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:18.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.112+0000 7fa20584d740 10 client.18234 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:13:18.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.112+0000 7fa20584d740 20 client.18234 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:18.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.112+0000 7fa20584d740 1 --2- 172.21.15.116:0/3581588190 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56262cdc41d0 0x56262cdc65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:18.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.112+0000 7fa20584d740 4 client.18234 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:13:18.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.112+0000 7fa20584d740 1 -- 172.21.15.116:0/3581588190 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56262cd18860 con 0x56262cdc41d0 2024-09-12T03:13:18.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.112+0000 7fa201210640 1 --2- 172.21.15.116:0/3581588190 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56262cdc41d0 0x56262cdc65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:18.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.112+0000 7fa1e4ff9640 10 client.18234 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:18.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.112+0000 7fa201210640 1 --2- 172.21.15.116:0/3581588190 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56262cdc41d0 0x56262cdc65c0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d50d2cfad3d6dc95 server_cookie=14b78086e3d092ed in_seq=0 out_seq=0 2024-09-12T03:13:18.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.113+0000 7fa1e4ff9640 1 -- 172.21.15.116:0/3581588190 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56262cd18860 con 0x56262cdc41d0 2024-09-12T03:13:18.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.113+0000 7fa1e4ff9640 10 client.18234 handle_command_reply: tid=0 2024-09-12T03:13:18.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.113+0000 7fa20584d740 1 client.18234 shutdown 2024-09-12T03:13:18.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.113+0000 7fa20584d740 1 -- 172.21.15.116:0/3581588190 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56262cda6b40 msgr2=0x56262cdb13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:18.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.113+0000 7fa20584d740 1 --2- 172.21.15.116:0/3581588190 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56262cda6b40 0x56262cdb13f0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec0096c0 tx=0x7fa1ec02aae0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.113+0000 7fa20584d740 1 -- 172.21.15.116:0/3581588190 shutdown_connections 2024-09-12T03:13:18.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.113+0000 7fa20584d740 1 --2- 172.21.15.116:0/3581588190 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56262cdc41d0 0x56262cdc65c0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.113+0000 7fa20584d740 1 --2- 172.21.15.116:0/3581588190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262cdb4780 0x56262cdb19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.113+0000 7fa20584d740 1 --2- 172.21.15.116:0/3581588190 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56262cda6b40 0x56262cdb13f0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.113+0000 7fa1e4ff9640 0 client.18234 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:18.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.113+0000 7fa20584d740 1 --2- 172.21.15.116:0/3581588190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56262cda55d0 0x56262cdb4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.113+0000 7fa20584d740 1 -- 172.21.15.116:0/3581588190 >> 172.21.15.116:0/3581588190 conn(0x56262cd6e310 msgr2=0x56262cda7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:18.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.113+0000 7fa20584d740 1 -- 172.21.15.116:0/3581588190 shutdown_connections 2024-09-12T03:13:18.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.113+0000 7fa20584d740 1 -- 172.21.15.116:0/3581588190 wait complete. 2024-09-12T03:13:18.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.113+0000 7fa20584d740 20 client.18234 trim_cache size 0 max 16384 2024-09-12T03:13:18.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.127+0000 7fa20584d740 1 Processor -- start 2024-09-12T03:13:18.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.127+0000 7fa20584d740 1 -- start start 2024-09-12T03:13:18.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.127+0000 7fa20584d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56262cda55d0 0x56262cdb03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:18.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.127+0000 7fa20584d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56262cda6b40 0x56262cdc95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:18.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.127+0000 7fa20584d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262cdb4780 0x56262cdc9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:18.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.127+0000 7fa20584d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc1adb90 con 0x56262cda55d0 2024-09-12T03:13:18.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.127+0000 7fa20584d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc1ab460 con 0x56262cdb4780 2024-09-12T03:13:18.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.127+0000 7fa20584d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc1a92d0 con 0x56262cda6b40 2024-09-12T03:13:18.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.128+0000 7fa201210640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56262cda55d0 0x56262cdb03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:18.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.128+0000 7fa201210640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56262cda55d0 0x56262cdb03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46198/0 (socket says 172.21.15.116:46198) 2024-09-12T03:13:18.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.128+0000 7fa201210640 1 -- 172.21.15.116:0/4264126486 learned_addr learned my addr 172.21.15.116:0/4264126486 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:18.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.128+0000 7fa201a11640 1 --2- 172.21.15.116:0/4264126486 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262cdb4780 0x56262cdc9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:18.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.128+0000 7fa200a0f640 1 --2- 172.21.15.116:0/4264126486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56262cda6b40 0x56262cdc95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:18.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.128+0000 7fa201210640 1 -- 172.21.15.116:0/4264126486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56262cda6b40 msgr2=0x56262cdc95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:18.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.128+0000 7fa201210640 1 --2- 172.21.15.116:0/4264126486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56262cda6b40 0x56262cdc95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.128+0000 7fa201210640 1 -- 172.21.15.116:0/4264126486 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262cdb4780 msgr2=0x56262cdc9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:18.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.128+0000 7fa201210640 1 --2- 172.21.15.116:0/4264126486 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262cdb4780 0x56262cdc9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.128+0000 7fa201210640 1 -- 172.21.15.116:0/4264126486 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56262cda8120 con 0x56262cda55d0 2024-09-12T03:13:18.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.128+0000 7fa201a11640 1 --2- 172.21.15.116:0/4264126486 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262cdb4780 0x56262cdc9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:13:18.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.128+0000 7fa201210640 1 --2- 172.21.15.116:0/4264126486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56262cda55d0 0x56262cdb03d0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80971f0 tx=0x7fa1f80ce700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7437296dd25e25b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:18.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.128+0000 7fa1e6ffd640 1 -- 172.21.15.116:0/4264126486 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f801e040 con 0x56262cda55d0 2024-09-12T03:13:18.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.128+0000 7fa1e6ffd640 1 -- 172.21.15.116:0/4264126486 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1f809b490 con 0x56262cda55d0 2024-09-12T03:13:18.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.128+0000 7fa1e6ffd640 1 -- 172.21.15.116:0/4264126486 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8017360 con 0x56262cda55d0 2024-09-12T03:13:18.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.129+0000 7fa20584d740 1 -- 172.21.15.116:0/4264126486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56262cda55d0 msgr2=0x56262cdb03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:18.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.129+0000 7fa20584d740 1 --2- 172.21.15.116:0/4264126486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56262cda55d0 0x56262cdb03d0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80971f0 tx=0x7fa1f80ce700 comp rx=0 tx=0).stop 2024-09-12T03:13:18.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.129+0000 7fa20584d740 1 -- 172.21.15.116:0/4264126486 shutdown_connections 2024-09-12T03:13:18.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.129+0000 7fa20584d740 1 --2- 172.21.15.116:0/4264126486 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262cdb4780 0x56262cdc9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.129+0000 7fa20584d740 1 --2- 172.21.15.116:0/4264126486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56262cda6b40 0x56262cdc95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.129+0000 7fa20584d740 1 --2- 172.21.15.116:0/4264126486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56262cda55d0 0x56262cdb03d0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.129+0000 7fa20584d740 1 -- 172.21.15.116:0/4264126486 >> 172.21.15.116:0/4264126486 conn(0x56262cd6e310 msgr2=0x56262cdafcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:18.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.129+0000 7fa20584d740 1 -- 172.21.15.116:0/4264126486 shutdown_connections 2024-09-12T03:13:18.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.129+0000 7fa20584d740 1 -- 172.21.15.116:0/4264126486 wait complete. 2024-09-12T03:13:18.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.130+0000 7fa20584d740 1 -- start start 2024-09-12T03:13:18.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.130+0000 7fa20584d740 1 Processor -- start 2024-09-12T03:13:18.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.130+0000 7fa20584d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262cda55d0 0x56262cdc9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:18.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.130+0000 7fa20584d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56262cda6b40 0x56262cdc7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:18.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.130+0000 7fa20584d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56262cdb4780 0x56262cdc7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:18.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.130+0000 7fa20584d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc1a9920 con 0x56262cdb4780 2024-09-12T03:13:18.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.130+0000 7fa20584d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc1aad90 con 0x56262cda55d0 2024-09-12T03:13:18.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.130+0000 7fa20584d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc1a9ff0 con 0x56262cda6b40 2024-09-12T03:13:18.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.130+0000 7fa20584d740 10 client.0 _ll_register_callbacks cb 0x7fa1f80d1eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:18.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.130+0000 7fa201a11640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56262cdb4780 0x56262cdc7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:18.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.130+0000 7fa201a11640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56262cdb4780 0x56262cdc7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46210/0 (socket says 172.21.15.116:46210) 2024-09-12T03:13:18.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.130+0000 7fa201a11640 1 -- 172.21.15.116:0/669979591 learned_addr learned my addr 172.21.15.116:0/669979591 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:18.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.130+0000 7fa200a0f640 1 --2- 172.21.15.116:0/669979591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56262cda6b40 0x56262cdc7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:18.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.130+0000 7fa201210640 1 --2- 172.21.15.116:0/669979591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262cda55d0 0x56262cdc9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:18.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.131+0000 7fa201a11640 1 -- 172.21.15.116:0/669979591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56262cda6b40 msgr2=0x56262cdc7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:18.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.131+0000 7fa201a11640 1 --2- 172.21.15.116:0/669979591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56262cda6b40 0x56262cdc7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.131+0000 7fa201a11640 1 -- 172.21.15.116:0/669979591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262cda55d0 msgr2=0x56262cdc9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:18.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.131+0000 7fa201a11640 1 --2- 172.21.15.116:0/669979591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262cda55d0 0x56262cdc9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.131+0000 7fa201a11640 1 -- 172.21.15.116:0/669979591 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56262cdc7f30 con 0x56262cdb4780 2024-09-12T03:13:18.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.131+0000 7fa201a11640 1 --2- 172.21.15.116:0/669979591 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56262cdb4780 0x56262cdc7700 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4011af0 tx=0x7fa1f4009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a9119144dcf3eb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:18.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.131+0000 7fa1e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:18.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.131+0000 7fa1e4ff9640 1 -- 172.21.15.116:0/669979591 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f401fcc0 con 0x56262cdb4780 2024-09-12T03:13:18.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.131+0000 7fa1e4ff9640 1 -- 172.21.15.116:0/669979591 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa1f401e420 con 0x56262cdb4780 2024-09-12T03:13:18.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.131+0000 7fa20584d740 1 -- 172.21.15.116:0/669979591 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1f809b490 con 0x56262cdb4780 2024-09-12T03:13:18.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.131+0000 7fa1e4ff9640 1 -- 172.21.15.116:0/669979591 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa1f401fe60 con 0x56262cdb4780 2024-09-12T03:13:18.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.131+0000 7fa20584d740 10 client.18240 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:18.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.131+0000 7fa20584d740 1 -- 172.21.15.116:0/669979591 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56262ce2ec40 con 0x56262cdb4780 2024-09-12T03:13:18.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.131+0000 7fa1e4ff9640 1 -- 172.21.15.116:0/669979591 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1f809b490 con 0x56262cdb4780 2024-09-12T03:13:18.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.132+0000 7fa1e4ff9640 1 -- 172.21.15.116:0/669979591 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa1f4034750 con 0x56262cdb4780 2024-09-12T03:13:18.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.132+0000 7fa20584d740 10 client.18240 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:18.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.132+0000 7fa20584d740 10 client.18240 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:13:18.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.132+0000 7fa20584d740 20 client.18240 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:18.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.132+0000 7fa20584d740 1 --2- 172.21.15.116:0/669979591 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56262ce2f410 0x56262ce31850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:18.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.132+0000 7fa20584d740 4 client.18240 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:13:18.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.132+0000 7fa20584d740 1 -- 172.21.15.116:0/669979591 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56262cd12d70 con 0x56262ce2f410 2024-09-12T03:13:18.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.132+0000 7fa201210640 1 --2- 172.21.15.116:0/669979591 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56262ce2f410 0x56262ce31850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:18.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.132+0000 7fa1e4ff9640 10 client.18240 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:18.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.132+0000 7fa201210640 1 --2- 172.21.15.116:0/669979591 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56262ce2f410 0x56262ce31850 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e958563ba7ac9df server_cookie=95092984c4e773dd in_seq=0 out_seq=0 2024-09-12T03:13:18.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.133+0000 7fa1e4ff9640 1 -- 172.21.15.116:0/669979591 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56262cd12d70 con 0x56262ce2f410 2024-09-12T03:13:18.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.133+0000 7fa1e4ff9640 10 client.18240 handle_command_reply: tid=0 2024-09-12T03:13:18.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.133+0000 7fa20584d740 1 client.18240 shutdown 2024-09-12T03:13:18.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.133+0000 7fa20584d740 1 -- 172.21.15.116:0/669979591 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56262cdb4780 msgr2=0x56262cdc7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:18.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.133+0000 7fa20584d740 1 --2- 172.21.15.116:0/669979591 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56262cdb4780 0x56262cdc7700 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4011af0 tx=0x7fa1f4009620 comp rx=0 tx=0).stop 2024-09-12T03:13:18.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.133+0000 7fa20584d740 1 -- 172.21.15.116:0/669979591 shutdown_connections 2024-09-12T03:13:18.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.133+0000 7fa20584d740 1 --2- 172.21.15.116:0/669979591 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56262ce2f410 0x56262ce31850 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.133+0000 7fa20584d740 1 --2- 172.21.15.116:0/669979591 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56262cdb4780 0x56262cdc7700 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.133+0000 7fa20584d740 1 --2- 172.21.15.116:0/669979591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56262cda6b40 0x56262cdc7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.133+0000 7fa20584d740 1 --2- 172.21.15.116:0/669979591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56262cda55d0 0x56262cdc9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.133+0000 7fa20584d740 1 -- 172.21.15.116:0/669979591 >> 172.21.15.116:0/669979591 conn(0x56262cd6e310 msgr2=0x56262cdb04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:18.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.133+0000 7fa1e4ff9640 0 client.18240 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:18.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.133+0000 7fa20584d740 1 -- 172.21.15.116:0/669979591 shutdown_connections 2024-09-12T03:13:18.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.133+0000 7fa20584d740 1 -- 172.21.15.116:0/669979591 wait complete. 2024-09-12T03:13:18.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.133+0000 7fa20584d740 20 client.18240 trim_cache size 0 max 16384 2024-09-12T03:13:18.139 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:13:18.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.134+0000 7fa203474640 1 -- 172.21.15.116:0/1880104970 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa1e80a2b80 msgr2=0x7fa1e80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:18.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.134+0000 7fa203474640 1 --2- 172.21.15.116:0/1880104970 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa1e80a2b80 0x7fa1e80a5040 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4009910 tx=0x7fa1f40098a0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.134+0000 7fa203474640 1 -- 172.21.15.116:0/1880104970 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1fc100a80 msgr2=0x7fa1fc198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:18.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.134+0000 7fa203474640 1 --2- 172.21.15.116:0/1880104970 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1fc100a80 0x7fa1fc198500 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800b4a0 tx=0x7fa1f800b960 comp rx=0 tx=0).stop 2024-09-12T03:13:18.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.134+0000 7fa203474640 1 -- 172.21.15.116:0/1880104970 shutdown_connections 2024-09-12T03:13:18.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.134+0000 7fa203474640 1 --2- 172.21.15.116:0/1880104970 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa1e80a2b80 0x7fa1e80a5040 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.134+0000 7fa203474640 1 --2- 172.21.15.116:0/1880104970 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa1fc106a80 0x7fa1fc19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.134+0000 7fa203474640 1 --2- 172.21.15.116:0/1880104970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1fc101420 0x7fa1fc198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.134+0000 7fa203474640 1 --2- 172.21.15.116:0/1880104970 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1fc100a80 0x7fa1fc198500 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:18.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.134+0000 7fa203474640 1 -- 172.21.15.116:0/1880104970 >> 172.21.15.116:0/1880104970 conn(0x7fa1fc0fc7a0 msgr2=0x7fa1fc0fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:18.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.134+0000 7fa203474640 1 -- 172.21.15.116:0/1880104970 shutdown_connections 2024-09-12T03:13:18.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:18.134+0000 7fa203474640 1 -- 172.21.15.116:0/1880104970 wait complete. 2024-09-12T03:13:18.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:18 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:18.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:18 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:18.781 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:18 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:19.939 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:13:19.940 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:13:20.203 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:13:20.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:20 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:13:20.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:20 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:13:20.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:20 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:13:20.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:20 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:13:20.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:20 smithi116 ceph-mon[31410]: pgmap v648: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.5 KiB/s wr, 8 op/s 2024-09-12T03:13:20.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:20 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:13:20.303 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:20 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:13:20.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:20 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:13:20.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:20 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:13:20.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:20 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:13:20.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:20 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:13:20.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:20 smithi183 ceph-mon[35535]: pgmap v648: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.5 KiB/s wr, 8 op/s 2024-09-12T03:13:20.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:20 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:13:20.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:20 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:13:20.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:20 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:13:20.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:20 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:13:20.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:20 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:13:20.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:20 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:13:20.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:20 smithi177 ceph-mon[35305]: pgmap v648: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.5 KiB/s wr, 8 op/s 2024-09-12T03:13:20.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:20 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:13:20.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:20 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:13:20.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.726+0000 7fd371571640 1 -- 172.21.15.116:0/1945510478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd36c102570 msgr2=0x7fd36c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:20.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.726+0000 7fd371571640 1 --2- 172.21.15.116:0/1945510478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd36c102570 0x7fd36c1029b0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fd354009900 tx=0x7fd35402a480 comp rx=0 tx=0).stop 2024-09-12T03:13:20.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.726+0000 7fd371571640 1 -- 172.21.15.116:0/1945510478 shutdown_connections 2024-09-12T03:13:20.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.726+0000 7fd371571640 1 --2- 172.21.15.116:0/1945510478 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd36c102ef0 0x7fd36c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.726+0000 7fd371571640 1 --2- 172.21.15.116:0/1945510478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd36c102570 0x7fd36c1029b0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.726+0000 7fd371571640 1 --2- 172.21.15.116:0/1945510478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd36c100e20 0x7fd36c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.726+0000 7fd371571640 1 -- 172.21.15.116:0/1945510478 >> 172.21.15.116:0/1945510478 conn(0x7fd36c0fc7d0 msgr2=0x7fd36c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:20.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.726+0000 7fd371571640 1 -- 172.21.15.116:0/1945510478 shutdown_connections 2024-09-12T03:13:20.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.726+0000 7fd371571640 1 -- 172.21.15.116:0/1945510478 wait complete. 2024-09-12T03:13:20.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.726+0000 7fd371571640 1 Processor -- start 2024-09-12T03:13:20.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.726+0000 7fd371571640 1 -- start start 2024-09-12T03:13:20.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.727+0000 7fd371571640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd36c100e20 0x7fd36c10e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:20.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.727+0000 7fd371571640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd36c102570 0x7fd36c10ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:20.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.727+0000 7fd371571640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd36c102ef0 0x7fd36c10ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:20.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.727+0000 7fd371571640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c1120a0 con 0x7fd36c100e20 2024-09-12T03:13:20.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.727+0000 7fd371571640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c111f20 con 0x7fd36c102ef0 2024-09-12T03:13:20.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.727+0000 7fd371571640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c112220 con 0x7fd36c102570 2024-09-12T03:13:20.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.727+0000 7fd36affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd36c100e20 0x7fd36c10e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:20.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.727+0000 7fd36affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd36c100e20 0x7fd36c10e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46234/0 (socket says 172.21.15.116:46234) 2024-09-12T03:13:20.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.727+0000 7fd36affd640 1 -- 172.21.15.116:0/63074116 learned_addr learned my addr 172.21.15.116:0/63074116 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:20.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.727+0000 7fd36b7fe640 1 --2- 172.21.15.116:0/63074116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd36c102ef0 0x7fd36c10ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:20.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.727+0000 7fd3627fc640 1 --2- 172.21.15.116:0/63074116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd36c102570 0x7fd36c10ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:20.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.727+0000 7fd36affd640 1 -- 172.21.15.116:0/63074116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd36c102570 msgr2=0x7fd36c10ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:20.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.727+0000 7fd36affd640 1 --2- 172.21.15.116:0/63074116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd36c102570 0x7fd36c10ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.727+0000 7fd36affd640 1 -- 172.21.15.116:0/63074116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd36c102ef0 msgr2=0x7fd36c10ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:20.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.727+0000 7fd36affd640 1 --2- 172.21.15.116:0/63074116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd36c102ef0 0x7fd36c10ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.727+0000 7fd36affd640 1 -- 172.21.15.116:0/63074116 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36c1a69e0 con 0x7fd36c100e20 2024-09-12T03:13:20.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.727+0000 7fd3627fc640 1 --2- 172.21.15.116:0/63074116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd36c102570 0x7fd36c10ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:13:20.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.727+0000 7fd36affd640 1 --2- 172.21.15.116:0/63074116 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd36c100e20 0x7fd36c10e4d0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00e9f0 tx=0x7fd35c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:20.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.727+0000 7fd368ff9640 1 -- 172.21.15.116:0/63074116 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c00cd80 con 0x7fd36c100e20 2024-09-12T03:13:20.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.728+0000 7fd371571640 1 -- 172.21.15.116:0/63074116 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd36c1a6cd0 con 0x7fd36c100e20 2024-09-12T03:13:20.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.728+0000 7fd368ff9640 1 -- 172.21.15.116:0/63074116 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd35c005500 con 0x7fd36c100e20 2024-09-12T03:13:20.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.728+0000 7fd371571640 1 -- 172.21.15.116:0/63074116 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd36c1a71c0 con 0x7fd36c100e20 2024-09-12T03:13:20.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.728+0000 7fd368ff9640 1 -- 172.21.15.116:0/63074116 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c010690 con 0x7fd36c100e20 2024-09-12T03:13:20.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.730+0000 7fd368ff9640 1 -- 172.21.15.116:0/63074116 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd35c010830 con 0x7fd36c100e20 2024-09-12T03:13:20.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.730+0000 7fd37394a740 1 Processor -- start 2024-09-12T03:13:20.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.730+0000 7fd37394a740 1 -- start start 2024-09-12T03:13:20.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.730+0000 7fd37394a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563605dc0a80 0x563605dc0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:20.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.730+0000 7fd37394a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563605dc13a0 0x563605dd4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:20.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.730+0000 7fd37394a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563605dd4550 0x563605dd6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:20.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.730+0000 7fd37394a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x563605d26280 con 0x563605dc0a80 2024-09-12T03:13:20.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.730+0000 7fd37394a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x563605ccdb10 con 0x563605dd4550 2024-09-12T03:13:20.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.730+0000 7fd37394a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563605d22240 con 0x563605dc13a0 2024-09-12T03:13:20.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.731+0000 7fd36b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563605dc0a80 0x563605dc0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:20.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.731+0000 7fd36b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563605dc0a80 0x563605dc0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46242/0 (socket says 172.21.15.116:46242) 2024-09-12T03:13:20.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.731+0000 7fd36b7fe640 1 -- 172.21.15.116:0/3395785907 learned_addr learned my addr 172.21.15.116:0/3395785907 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:20.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.731+0000 7fd3627fc640 1 --2- 172.21.15.116:0/3395785907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563605dd4550 0x563605dd6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:20.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.731+0000 7fd36affd640 1 --2- 172.21.15.116:0/3395785907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563605dc13a0 0x563605dd4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:20.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.731+0000 7fd36b7fe640 1 -- 172.21.15.116:0/3395785907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563605dc13a0 msgr2=0x563605dd4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:20.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.731+0000 7fd36b7fe640 1 --2- 172.21.15.116:0/3395785907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563605dc13a0 0x563605dd4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.731+0000 7fd36b7fe640 1 -- 172.21.15.116:0/3395785907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563605dd4550 msgr2=0x563605dd6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:20.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.731+0000 7fd36b7fe640 1 --2- 172.21.15.116:0/3395785907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563605dd4550 0x563605dd6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.731+0000 7fd36b7fe640 1 -- 172.21.15.116:0/3395785907 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563605cda1f0 con 0x563605dc0a80 2024-09-12T03:13:20.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.731+0000 7fd368ff9640 1 --2- 172.21.15.116:0/63074116 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd3580a2c00 0x7fd3580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:20.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.731+0000 7fd36b7fe640 1 --2- 172.21.15.116:0/3395785907 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563605dc0a80 0x563605dc0e60 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fd364009f90 tx=0x7fd364010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539a3fc4993958a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:20.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.731+0000 7fd36affd640 1 --2- 172.21.15.116:0/63074116 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd3580a2c00 0x7fd3580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:20.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.731+0000 7fd368ff9640 1 -- 172.21.15.116:0/63074116 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd35c0cf3a0 con 0x7fd36c100e20 2024-09-12T03:13:20.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.731+0000 7fd3617fa640 1 -- 172.21.15.116:0/3395785907 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd364011700 con 0x563605dc0a80 2024-09-12T03:13:20.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.732+0000 7fd3617fa640 1 -- 172.21.15.116:0/3395785907 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd364011d80 con 0x563605dc0a80 2024-09-12T03:13:20.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.732+0000 7fd3617fa640 1 -- 172.21.15.116:0/3395785907 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd364018380 con 0x563605dc0a80 2024-09-12T03:13:20.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.732+0000 7fd37394a740 1 -- 172.21.15.116:0/3395785907 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563605dc0a80 msgr2=0x563605dc0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:20.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.732+0000 7fd36affd640 1 --2- 172.21.15.116:0/63074116 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd3580a2c00 0x7fd3580a50c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd35c098030 tx=0x7fd35c0d2b10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:20.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.732+0000 7fd37394a740 1 --2- 172.21.15.116:0/3395785907 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563605dc0a80 0x563605dc0e60 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fd364009f90 tx=0x7fd364010860 comp rx=0 tx=0).stop 2024-09-12T03:13:20.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.732+0000 7fd37394a740 1 -- 172.21.15.116:0/3395785907 shutdown_connections 2024-09-12T03:13:20.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.732+0000 7fd37394a740 1 --2- 172.21.15.116:0/3395785907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563605dd4550 0x563605dd6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.732+0000 7fd37394a740 1 --2- 172.21.15.116:0/3395785907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563605dc13a0 0x563605dd4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.732+0000 7fd37394a740 1 --2- 172.21.15.116:0/3395785907 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563605dc0a80 0x563605dc0e60 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.732+0000 7fd37394a740 1 -- 172.21.15.116:0/3395785907 >> 172.21.15.116:0/3395785907 conn(0x563605dbd0f0 msgr2=0x563605dbd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:20.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.732+0000 7fd37394a740 1 -- 172.21.15.116:0/3395785907 shutdown_connections 2024-09-12T03:13:20.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.732+0000 7fd37394a740 1 -- 172.21.15.116:0/3395785907 wait complete. 2024-09-12T03:13:20.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.733+0000 7fd37394a740 1 -- start start 2024-09-12T03:13:20.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.733+0000 7fd37394a740 1 Processor -- start 2024-09-12T03:13:20.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.733+0000 7fd37394a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563605dc0a80 0x563605dd1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:20.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.733+0000 7fd37394a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563605dc13a0 0x563605dd19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:20.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.733+0000 7fd37394a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563605dd4550 0x563605dd1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:20.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.733+0000 7fd37394a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x563605dd70c0 con 0x563605dd4550 2024-09-12T03:13:20.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.733+0000 7fd37394a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x563605ce0f10 con 0x563605dc13a0 2024-09-12T03:13:20.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.733+0000 7fd37394a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563605dd7240 con 0x563605dc0a80 2024-09-12T03:13:20.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.733+0000 7fd37394a740 10 client.0 _ll_register_callbacks cb 0x563605ca9a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:20.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.733+0000 7fd36b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563605dd4550 0x563605dd1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:20.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.733+0000 7fd36b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563605dd4550 0x563605dd1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46258/0 (socket says 172.21.15.116:46258) 2024-09-12T03:13:20.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.733+0000 7fd36b7fe640 1 -- 172.21.15.116:0/262767262 learned_addr learned my addr 172.21.15.116:0/262767262 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:20.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.733+0000 7fd36b7fe640 1 --2- 172.21.15.116:0/262767262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563605dc0a80 0x563605dd1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:20.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.733+0000 7fd3627fc640 1 --2- 172.21.15.116:0/262767262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563605dc13a0 0x563605dd19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:20.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.733+0000 7fd36b7fe640 1 -- 172.21.15.116:0/262767262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563605dc0a80 msgr2=0x563605dd1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:20.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.734+0000 7fd36b7fe640 1 --2- 172.21.15.116:0/262767262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563605dc0a80 0x563605dd1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.734+0000 7fd36b7fe640 1 -- 172.21.15.116:0/262767262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563605dc13a0 msgr2=0x563605dd19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:20.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.734+0000 7fd36b7fe640 1 --2- 172.21.15.116:0/262767262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563605dc13a0 0x563605dd19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.734+0000 7fd36b7fe640 1 -- 172.21.15.116:0/262767262 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563605a976a0 con 0x563605dd4550 2024-09-12T03:13:20.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.734+0000 7fd36b7fe640 1 --2- 172.21.15.116:0/262767262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563605dd4550 0x563605dd1f00 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fd364011fd0 tx=0x7fd364013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=644a1afde93d34e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:20.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.734+0000 7fd3527fc640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:20.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.734+0000 7fd3527fc640 1 -- 172.21.15.116:0/262767262 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd364003970 con 0x563605dd4550 2024-09-12T03:13:20.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.734+0000 7fd3527fc640 1 -- 172.21.15.116:0/262767262 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd364003b10 con 0x563605dd4550 2024-09-12T03:13:20.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.734+0000 7fd37394a740 1 -- 172.21.15.116:0/262767262 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd364011d80 con 0x563605dd4550 2024-09-12T03:13:20.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.734+0000 7fd3527fc640 1 -- 172.21.15.116:0/262767262 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd364003d10 con 0x563605dd4550 2024-09-12T03:13:20.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.734+0000 7fd37394a740 10 client.18252 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:20.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.734+0000 7fd37394a740 1 -- 172.21.15.116:0/262767262 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563605de2dd0 con 0x563605dd4550 2024-09-12T03:13:20.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.734+0000 7fd3527fc640 1 -- 172.21.15.116:0/262767262 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd364011d80 con 0x563605dd4550 2024-09-12T03:13:20.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.734+0000 7fd3527fc640 1 -- 172.21.15.116:0/262767262 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd364004210 con 0x563605dd4550 2024-09-12T03:13:20.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.734+0000 7fd37394a740 10 client.18252 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:20.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.734+0000 7fd37394a740 10 client.18252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:13:20.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.734+0000 7fd37394a740 20 client.18252 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:20.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.734+0000 7fd37394a740 1 --2- 172.21.15.116:0/262767262 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x563605de4180 0x563605de6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:20.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.734+0000 7fd37394a740 4 client.18252 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:13:20.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.734+0000 7fd37394a740 1 -- 172.21.15.116:0/262767262 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563605d337f0 con 0x563605de4180 2024-09-12T03:13:20.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.735+0000 7fd3627fc640 1 --2- 172.21.15.116:0/262767262 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x563605de4180 0x563605de6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:20.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.735+0000 7fd3527fc640 10 client.18252 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:20.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.735+0000 7fd3627fc640 1 --2- 172.21.15.116:0/262767262 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x563605de4180 0x563605de6570 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=582698babb513813 server_cookie=ffdeec06893ba79f in_seq=0 out_seq=0 2024-09-12T03:13:20.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.736+0000 7fd3527fc640 1 -- 172.21.15.116:0/262767262 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563605d337f0 con 0x563605de4180 2024-09-12T03:13:20.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.736+0000 7fd3527fc640 10 client.18252 handle_command_reply: tid=0 2024-09-12T03:13:20.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.736+0000 7fd37394a740 1 client.18252 shutdown 2024-09-12T03:13:20.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.736+0000 7fd37394a740 1 -- 172.21.15.116:0/262767262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563605dd4550 msgr2=0x563605dd1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:20.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.736+0000 7fd37394a740 1 --2- 172.21.15.116:0/262767262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563605dd4550 0x563605dd1f00 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fd364011fd0 tx=0x7fd364013040 comp rx=0 tx=0).stop 2024-09-12T03:13:20.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.736+0000 7fd37394a740 1 -- 172.21.15.116:0/262767262 shutdown_connections 2024-09-12T03:13:20.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.736+0000 7fd37394a740 1 --2- 172.21.15.116:0/262767262 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x563605de4180 0x563605de6570 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.736+0000 7fd37394a740 1 --2- 172.21.15.116:0/262767262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563605dd4550 0x563605dd1f00 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.736+0000 7fd37394a740 1 --2- 172.21.15.116:0/262767262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563605dc13a0 0x563605dd19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.736+0000 7fd37394a740 1 --2- 172.21.15.116:0/262767262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563605dc0a80 0x563605dd1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.736+0000 7fd37394a740 1 -- 172.21.15.116:0/262767262 >> 172.21.15.116:0/262767262 conn(0x563605dbd0f0 msgr2=0x563605dbc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:20.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.736+0000 7fd3527fc640 0 client.18252 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:20.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.736+0000 7fd37394a740 1 -- 172.21.15.116:0/262767262 shutdown_connections 2024-09-12T03:13:20.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.736+0000 7fd37394a740 1 -- 172.21.15.116:0/262767262 wait complete. 2024-09-12T03:13:20.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.736+0000 7fd37394a740 20 client.18252 trim_cache size 0 max 16384 2024-09-12T03:13:20.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.753+0000 7fd37394a740 1 Processor -- start 2024-09-12T03:13:20.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.753+0000 7fd37394a740 1 -- start start 2024-09-12T03:13:20.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.753+0000 7fd37394a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563605dc0a80 0x563605dd32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:20.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.754+0000 7fd37394a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563605dc13a0 0x563605debbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:20.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.754+0000 7fd37394a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563605dd4550 0x563605dec110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:20.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.754+0000 7fd37394a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c010c40 con 0x563605dc13a0 2024-09-12T03:13:20.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.754+0000 7fd37394a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c00e510 con 0x563605dd4550 2024-09-12T03:13:20.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.754+0000 7fd37394a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c00c360 con 0x563605dc0a80 2024-09-12T03:13:20.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.754+0000 7fd3627fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563605dc13a0 0x563605debbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:20.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.754+0000 7fd3627fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563605dc13a0 0x563605debbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46260/0 (socket says 172.21.15.116:46260) 2024-09-12T03:13:20.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.754+0000 7fd3627fc640 1 -- 172.21.15.116:0/332014009 learned_addr learned my addr 172.21.15.116:0/332014009 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:20.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.754+0000 7fd36b7fe640 1 --2- 172.21.15.116:0/332014009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563605dd4550 0x563605dec110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:20.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.754+0000 7fd3627fc640 1 -- 172.21.15.116:0/332014009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563605dc0a80 msgr2=0x563605dd32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:20.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.754+0000 7fd3627fc640 1 --2- 172.21.15.116:0/332014009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563605dc0a80 0x563605dd32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.754+0000 7fd3627fc640 1 -- 172.21.15.116:0/332014009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563605dd4550 msgr2=0x563605dec110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:20.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.754+0000 7fd3627fc640 1 --2- 172.21.15.116:0/332014009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563605dd4550 0x563605dec110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.754+0000 7fd3627fc640 1 -- 172.21.15.116:0/332014009 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563605dd28f0 con 0x563605dc13a0 2024-09-12T03:13:20.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.754+0000 7fd3627fc640 1 --2- 172.21.15.116:0/332014009 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563605dc13a0 0x563605debbd0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fd35402a9a0 tx=0x7fd354031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a477bb3e4f6f8fd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:20.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.754+0000 7fd360ff9640 1 -- 172.21.15.116:0/332014009 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd354035040 con 0x563605dc13a0 2024-09-12T03:13:20.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.754+0000 7fd360ff9640 1 -- 172.21.15.116:0/332014009 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd35404a070 con 0x563605dc13a0 2024-09-12T03:13:20.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.754+0000 7fd360ff9640 1 -- 172.21.15.116:0/332014009 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd354003850 con 0x563605dc13a0 2024-09-12T03:13:20.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.755+0000 7fd37394a740 1 -- 172.21.15.116:0/332014009 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563605dc13a0 msgr2=0x563605debbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:20.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.755+0000 7fd37394a740 1 --2- 172.21.15.116:0/332014009 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563605dc13a0 0x563605debbd0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fd35402a9a0 tx=0x7fd354031400 comp rx=0 tx=0).stop 2024-09-12T03:13:20.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.755+0000 7fd37394a740 1 -- 172.21.15.116:0/332014009 shutdown_connections 2024-09-12T03:13:20.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.755+0000 7fd37394a740 1 --2- 172.21.15.116:0/332014009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563605dd4550 0x563605dec110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.755+0000 7fd37394a740 1 --2- 172.21.15.116:0/332014009 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563605dc13a0 0x563605debbd0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.755+0000 7fd37394a740 1 --2- 172.21.15.116:0/332014009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563605dc0a80 0x563605dd32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.755+0000 7fd37394a740 1 -- 172.21.15.116:0/332014009 >> 172.21.15.116:0/332014009 conn(0x563605dbd0f0 msgr2=0x563605dd2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:20.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.755+0000 7fd37394a740 1 -- 172.21.15.116:0/332014009 shutdown_connections 2024-09-12T03:13:20.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.755+0000 7fd37394a740 1 -- 172.21.15.116:0/332014009 wait complete. 2024-09-12T03:13:20.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.755+0000 7fd37394a740 1 -- start start 2024-09-12T03:13:20.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.756+0000 7fd37394a740 1 Processor -- start 2024-09-12T03:13:20.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.756+0000 7fd37394a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563605dc0a80 0x563605ddcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:20.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.756+0000 7fd37394a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563605dc13a0 0x563605dd7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:20.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.756+0000 7fd37394a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563605dd4550 0x563605dd84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:20.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.756+0000 7fd37394a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c00c9d0 con 0x563605dc0a80 2024-09-12T03:13:20.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.756+0000 7fd37394a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c00de40 con 0x563605dc13a0 2024-09-12T03:13:20.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.756+0000 7fd37394a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c00d0a0 con 0x563605dd4550 2024-09-12T03:13:20.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.756+0000 7fd37394a740 10 client.0 _ll_register_callbacks cb 0x7fd35c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:20.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.756+0000 7fd36b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563605dc0a80 0x563605ddcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:20.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.756+0000 7fd36b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563605dc0a80 0x563605ddcc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46264/0 (socket says 172.21.15.116:46264) 2024-09-12T03:13:20.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.756+0000 7fd36b7fe640 1 -- 172.21.15.116:0/3099381035 learned_addr learned my addr 172.21.15.116:0/3099381035 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:20.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.756+0000 7fd3627fc640 1 --2- 172.21.15.116:0/3099381035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563605dc13a0 0x563605dd7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:20.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.756+0000 7fd36b7fe640 1 --2- 172.21.15.116:0/3099381035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563605dd4550 0x563605dd84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:20.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.756+0000 7fd36b7fe640 1 -- 172.21.15.116:0/3099381035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563605dd4550 msgr2=0x563605dd84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:20.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.756+0000 7fd36b7fe640 1 --2- 172.21.15.116:0/3099381035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563605dd4550 0x563605dd84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.756+0000 7fd36b7fe640 1 -- 172.21.15.116:0/3099381035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563605dc13a0 msgr2=0x563605dd7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:20.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.756+0000 7fd36b7fe640 1 --2- 172.21.15.116:0/3099381035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563605dc13a0 0x563605dd7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.756+0000 7fd36b7fe640 1 -- 172.21.15.116:0/3099381035 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563605dc34e0 con 0x563605dc0a80 2024-09-12T03:13:20.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.756+0000 7fd36b7fe640 1 --2- 172.21.15.116:0/3099381035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563605dc0a80 0x563605ddcc30 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fd364009a80 tx=0x7fd364010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd8accad70e84fb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:20.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.756+0000 7fd3527fc640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:20.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.757+0000 7fd3527fc640 1 -- 172.21.15.116:0/3099381035 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3640193d0 con 0x563605dc0a80 2024-09-12T03:13:20.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.757+0000 7fd3527fc640 1 -- 172.21.15.116:0/3099381035 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd364019570 con 0x563605dc0a80 2024-09-12T03:13:20.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.757+0000 7fd37394a740 1 -- 172.21.15.116:0/3099381035 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd35404a070 con 0x563605dc0a80 2024-09-12T03:13:20.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.757+0000 7fd3527fc640 1 -- 172.21.15.116:0/3099381035 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd36404a050 con 0x563605dc0a80 2024-09-12T03:13:20.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.757+0000 7fd37394a740 10 client.18258 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:20.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.757+0000 7fd37394a740 1 -- 172.21.15.116:0/3099381035 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563605e4e9b0 con 0x563605dc0a80 2024-09-12T03:13:20.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.757+0000 7fd3527fc640 1 -- 172.21.15.116:0/3099381035 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd35404a070 con 0x563605dc0a80 2024-09-12T03:13:20.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.757+0000 7fd3527fc640 1 -- 172.21.15.116:0/3099381035 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd364041070 con 0x563605dc0a80 2024-09-12T03:13:20.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.757+0000 7fd37394a740 10 client.18258 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:20.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.757+0000 7fd37394a740 10 client.18258 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:13:20.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.757+0000 7fd37394a740 20 client.18258 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:20.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.757+0000 7fd37394a740 1 --2- 172.21.15.116:0/3099381035 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x563605e4ef60 0x563605e51350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:20.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.757+0000 7fd37394a740 4 client.18258 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:13:20.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.757+0000 7fd37394a740 1 -- 172.21.15.116:0/3099381035 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563605d2dd00 con 0x563605e4ef60 2024-09-12T03:13:20.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.757+0000 7fd3627fc640 1 --2- 172.21.15.116:0/3099381035 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x563605e4ef60 0x563605e51350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:20.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.758+0000 7fd3527fc640 10 client.18258 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:20.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.758+0000 7fd3627fc640 1 --2- 172.21.15.116:0/3099381035 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x563605e4ef60 0x563605e51350 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94f269dc9a40d390 server_cookie=98d457d9565819ae in_seq=0 out_seq=0 2024-09-12T03:13:20.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.759+0000 7fd3527fc640 1 -- 172.21.15.116:0/3099381035 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563605d2dd00 con 0x563605e4ef60 2024-09-12T03:13:20.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.759+0000 7fd3527fc640 10 client.18258 handle_command_reply: tid=0 2024-09-12T03:13:20.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.759+0000 7fd37394a740 1 client.18258 shutdown 2024-09-12T03:13:20.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.759+0000 7fd37394a740 1 -- 172.21.15.116:0/3099381035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563605dc0a80 msgr2=0x563605ddcc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:20.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.759+0000 7fd37394a740 1 --2- 172.21.15.116:0/3099381035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563605dc0a80 0x563605ddcc30 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fd364009a80 tx=0x7fd364010860 comp rx=0 tx=0).stop 2024-09-12T03:13:20.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.759+0000 7fd37394a740 1 -- 172.21.15.116:0/3099381035 shutdown_connections 2024-09-12T03:13:20.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.759+0000 7fd37394a740 1 --2- 172.21.15.116:0/3099381035 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x563605e4ef60 0x563605e51350 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.759+0000 7fd3527fc640 0 client.18258 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:20.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.759+0000 7fd37394a740 1 --2- 172.21.15.116:0/3099381035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563605dd4550 0x563605dd84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.759+0000 7fd37394a740 1 --2- 172.21.15.116:0/3099381035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563605dc13a0 0x563605dd7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.759+0000 7fd37394a740 1 --2- 172.21.15.116:0/3099381035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563605dc0a80 0x563605ddcc30 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.759+0000 7fd37394a740 1 -- 172.21.15.116:0/3099381035 >> 172.21.15.116:0/3099381035 conn(0x563605dbd0f0 msgr2=0x563605dcf920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:20.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.759+0000 7fd37394a740 1 -- 172.21.15.116:0/3099381035 shutdown_connections 2024-09-12T03:13:20.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.759+0000 7fd37394a740 1 -- 172.21.15.116:0/3099381035 wait complete. 2024-09-12T03:13:20.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.759+0000 7fd37394a740 20 client.18258 trim_cache size 0 max 16384 2024-09-12T03:13:20.769 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:13:20.769 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:13:20.769 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "33fcd4f1-db99-48b2-8fd6-6397b41af3a2", 2024-09-12T03:13:20.769 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:13:20.769 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:13:20.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.760+0000 7fd371571640 1 -- 172.21.15.116:0/63074116 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd3580a2c00 msgr2=0x7fd3580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:20.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.760+0000 7fd371571640 1 --2- 172.21.15.116:0/63074116 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd3580a2c00 0x7fd3580a50c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd35c098030 tx=0x7fd35c0d2b10 comp rx=0 tx=0).stop 2024-09-12T03:13:20.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.760+0000 7fd371571640 1 -- 172.21.15.116:0/63074116 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd36c100e20 msgr2=0x7fd36c10e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:20.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.760+0000 7fd371571640 1 --2- 172.21.15.116:0/63074116 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd36c100e20 0x7fd36c10e4d0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00e9f0 tx=0x7fd35c00edf0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.760+0000 7fd371571640 1 -- 172.21.15.116:0/63074116 shutdown_connections 2024-09-12T03:13:20.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.760+0000 7fd371571640 1 --2- 172.21.15.116:0/63074116 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd3580a2c00 0x7fd3580a50c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.760+0000 7fd371571640 1 --2- 172.21.15.116:0/63074116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd36c102ef0 0x7fd36c10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.760+0000 7fd371571640 1 --2- 172.21.15.116:0/63074116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd36c102570 0x7fd36c10ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.760+0000 7fd371571640 1 --2- 172.21.15.116:0/63074116 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd36c100e20 0x7fd36c10e4d0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:20.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.760+0000 7fd371571640 1 -- 172.21.15.116:0/63074116 >> 172.21.15.116:0/63074116 conn(0x7fd36c0fc7d0 msgr2=0x7fd36c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:20.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.760+0000 7fd371571640 1 -- 172.21.15.116:0/63074116 shutdown_connections 2024-09-12T03:13:20.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:20.760+0000 7fd371571640 1 -- 172.21.15.116:0/63074116 wait complete. 2024-09-12T03:13:21.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:21 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:21.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:21 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:21.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:21 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:13:21.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:21 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:13:21.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:21 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:13:21.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:21 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:13:21.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:21 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:13:21.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:21 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:21.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:21 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:21.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:21 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:13:21.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:21 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:13:21.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:21 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:13:21.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:21 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:13:21.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:21 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:13:21.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:21 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:21.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:21 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:21.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:21 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:13:21.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:21 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:13:21.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:21 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:13:21.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:21 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:13:21.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:21 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:13:21.699 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:13:21.946 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:13:22.357 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:22 smithi116 ceph-mon[31410]: pgmap v649: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.2 KiB/s wr, 9 op/s 2024-09-12T03:13:22.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:22 smithi183 ceph-mon[35535]: pgmap v649: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.2 KiB/s wr, 9 op/s 2024-09-12T03:13:22.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.439+0000 7f6b01910640 1 -- 172.21.15.116:0/4063931038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6afc10c170 msgr2=0x7f6afc10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:22.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.439+0000 7f6b01910640 1 --2- 172.21.15.116:0/4063931038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6afc10c170 0x7f6afc10e610 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f6af4009ca0 tx=0x7f6af402a950 comp rx=0 tx=0).stop 2024-09-12T03:13:22.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.439+0000 7f6b01910640 1 -- 172.21.15.116:0/4063931038 shutdown_connections 2024-09-12T03:13:22.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.439+0000 7f6b01910640 1 --2- 172.21.15.116:0/4063931038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6afc10c170 0x7f6afc10e610 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.439+0000 7f6b01910640 1 --2- 172.21.15.116:0/4063931038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6afc103660 0x7f6afc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.439+0000 7f6b01910640 1 --2- 172.21.15.116:0/4063931038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6afc102d40 0x7f6afc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.439+0000 7f6b01910640 1 -- 172.21.15.116:0/4063931038 >> 172.21.15.116:0/4063931038 conn(0x7f6afc0fc7a0 msgr2=0x7f6afc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:22.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.439+0000 7f6b01910640 1 -- 172.21.15.116:0/4063931038 shutdown_connections 2024-09-12T03:13:22.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.439+0000 7f6b01910640 1 -- 172.21.15.116:0/4063931038 wait complete. 2024-09-12T03:13:22.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.440+0000 7f6b01910640 1 Processor -- start 2024-09-12T03:13:22.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.440+0000 7f6b01910640 1 -- start start 2024-09-12T03:13:22.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.440+0000 7f6b01910640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6afc102d40 0x7f6afc1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:22.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.440+0000 7f6b01910640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6afc103660 0x7f6afc100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:22.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.440+0000 7f6b01910640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6afc10c170 0x7f6afc101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:22.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.440+0000 7f6b01910640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6afc111fd0 con 0x7f6afc103660 2024-09-12T03:13:22.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.440+0000 7f6b01910640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6afc111e50 con 0x7f6afc102d40 2024-09-12T03:13:22.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.440+0000 7f6b01910640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6afc112150 con 0x7f6afc10c170 2024-09-12T03:13:22.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.440+0000 7f6afa7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6afc103660 0x7f6afc100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:22.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.440+0000 7f6afa7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6afc103660 0x7f6afc100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39044/0 (socket says 172.21.15.116:39044) 2024-09-12T03:13:22.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.440+0000 7f6afa7fc640 1 -- 172.21.15.116:0/2485327351 learned_addr learned my addr 172.21.15.116:0/2485327351 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:22.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.440+0000 7f6afb7fe640 1 --2- 172.21.15.116:0/2485327351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6afc10c170 0x7f6afc101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:22.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.440+0000 7f6afaffd640 1 --2- 172.21.15.116:0/2485327351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6afc102d40 0x7f6afc1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:22.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.440+0000 7f6afa7fc640 1 -- 172.21.15.116:0/2485327351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6afc10c170 msgr2=0x7f6afc101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:22.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.440+0000 7f6afa7fc640 1 --2- 172.21.15.116:0/2485327351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6afc10c170 0x7f6afc101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.440+0000 7f6afa7fc640 1 -- 172.21.15.116:0/2485327351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6afc102d40 msgr2=0x7f6afc1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:22.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.440+0000 7f6afa7fc640 1 --2- 172.21.15.116:0/2485327351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6afc102d40 0x7f6afc1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.440+0000 7f6afa7fc640 1 -- 172.21.15.116:0/2485327351 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6afc1a6930 con 0x7f6afc103660 2024-09-12T03:13:22.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.440+0000 7f6afb7fe640 1 --2- 172.21.15.116:0/2485327351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6afc10c170 0x7f6afc101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:13:22.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.441+0000 7f6afa7fc640 1 --2- 172.21.15.116:0/2485327351 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6afc103660 0x7f6afc100ee0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f6af000d750 tx=0x7f6af000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:22.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.441+0000 7f6b0090e640 1 -- 172.21.15.116:0/2485327351 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af0017070 con 0x7f6afc103660 2024-09-12T03:13:22.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.441+0000 7f6b01910640 1 -- 172.21.15.116:0/2485327351 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6afc1a6c20 con 0x7f6afc103660 2024-09-12T03:13:22.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.441+0000 7f6b0090e640 1 -- 172.21.15.116:0/2485327351 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6af0005020 con 0x7f6afc103660 2024-09-12T03:13:22.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.441+0000 7f6b0090e640 1 -- 172.21.15.116:0/2485327351 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af0004d00 con 0x7f6afc103660 2024-09-12T03:13:22.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.441+0000 7f6b01910640 1 -- 172.21.15.116:0/2485327351 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6afc1a7160 con 0x7f6afc103660 2024-09-12T03:13:22.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.443+0000 7f6b0090e640 1 -- 172.21.15.116:0/2485327351 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6af0009a30 con 0x7f6afc103660 2024-09-12T03:13:22.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.444+0000 7f6b03ce9740 1 Processor -- start 2024-09-12T03:13:22.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.444+0000 7f6b03ce9740 1 -- start start 2024-09-12T03:13:22.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.444+0000 7f6b03ce9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea03a324a0 0x55ea03a32880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:22.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.444+0000 7f6b03ce9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea03a32dc0 0x55ea03a3d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:22.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.444+0000 7f6b03ce9740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea03a3d680 0x55ea03a3fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:22.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.444+0000 7f6b03ce9740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ea039982e0 con 0x55ea03a3d680 2024-09-12T03:13:22.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.444+0000 7f6b03ce9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ea0393fce0 con 0x55ea03a32dc0 2024-09-12T03:13:22.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.444+0000 7f6b03ce9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ea039942a0 con 0x55ea03a324a0 2024-09-12T03:13:22.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.444+0000 7f6afa7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea03a3d680 0x55ea03a3fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:22.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.444+0000 7f6b0090e640 1 --2- 172.21.15.116:0/2485327351 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6ae40a2b80 0x7f6ae40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:22.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.444+0000 7f6afa7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea03a3d680 0x55ea03a3fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39050/0 (socket says 172.21.15.116:39050) 2024-09-12T03:13:22.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.444+0000 7f6afa7fc640 1 -- 172.21.15.116:0/1996240626 learned_addr learned my addr 172.21.15.116:0/1996240626 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:22.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.444+0000 7f6afaffd640 1 --2- 172.21.15.116:0/1996240626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea03a32dc0 0x55ea03a3d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:22.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.444+0000 7f6afb7fe640 1 --2- 172.21.15.116:0/1996240626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea03a324a0 0x55ea03a32880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:22.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.444+0000 7f6afb7fe640 1 --2- 172.21.15.116:0/2485327351 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6ae40a2b80 0x7f6ae40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:22.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.444+0000 7f6b0090e640 1 -- 172.21.15.116:0/2485327351 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f6af00ceb30 con 0x7f6afc103660 2024-09-12T03:13:22.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.444+0000 7f6afa7fc640 1 -- 172.21.15.116:0/1996240626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea03a324a0 msgr2=0x55ea03a32880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:22.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.445+0000 7f6afa7fc640 1 --2- 172.21.15.116:0/1996240626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea03a324a0 0x55ea03a32880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.445+0000 7f6afa7fc640 1 -- 172.21.15.116:0/1996240626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea03a32dc0 msgr2=0x55ea03a3d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:22.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.445+0000 7f6afa7fc640 1 --2- 172.21.15.116:0/1996240626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea03a32dc0 0x55ea03a3d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.445+0000 7f6afa7fc640 1 -- 172.21.15.116:0/1996240626 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea03924440 con 0x55ea03a3d680 2024-09-12T03:13:22.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.445+0000 7f6afa7fc640 1 --2- 172.21.15.116:0/1996240626 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea03a3d680 0x55ea03a3fa70 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f6af000b620 tx=0x7f6af0004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287bd78993185712 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:22.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.445+0000 7f6afb7fe640 1 --2- 172.21.15.116:0/2485327351 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6ae40a2b80 0x7f6ae40a5040 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6af4005fc0 tx=0x7f6af40095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:22.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.445+0000 7f6ae1ffb640 1 -- 172.21.15.116:0/1996240626 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af0020070 con 0x55ea03a3d680 2024-09-12T03:13:22.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.445+0000 7f6ae1ffb640 1 -- 172.21.15.116:0/1996240626 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6af009b360 con 0x55ea03a3d680 2024-09-12T03:13:22.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.445+0000 7f6ae1ffb640 1 -- 172.21.15.116:0/1996240626 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af00cc710 con 0x55ea03a3d680 2024-09-12T03:13:22.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.445+0000 7f6b03ce9740 1 -- 172.21.15.116:0/1996240626 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea03a3d680 msgr2=0x55ea03a3fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:22.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.445+0000 7f6b03ce9740 1 --2- 172.21.15.116:0/1996240626 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea03a3d680 0x55ea03a3fa70 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f6af000b620 tx=0x7f6af0004fa0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.446+0000 7f6b03ce9740 1 -- 172.21.15.116:0/1996240626 shutdown_connections 2024-09-12T03:13:22.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.446+0000 7f6b03ce9740 1 --2- 172.21.15.116:0/1996240626 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea03a3d680 0x55ea03a3fa70 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.446+0000 7f6b03ce9740 1 --2- 172.21.15.116:0/1996240626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea03a32dc0 0x55ea03a3d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.446+0000 7f6b03ce9740 1 --2- 172.21.15.116:0/1996240626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea03a324a0 0x55ea03a32880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.446+0000 7f6b03ce9740 1 -- 172.21.15.116:0/1996240626 >> 172.21.15.116:0/1996240626 conn(0x55ea03a2ea30 msgr2=0x55ea03a2ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:22.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.446+0000 7f6b03ce9740 1 -- 172.21.15.116:0/1996240626 shutdown_connections 2024-09-12T03:13:22.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.446+0000 7f6b03ce9740 1 -- 172.21.15.116:0/1996240626 wait complete. 2024-09-12T03:13:22.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.446+0000 7f6b03ce9740 1 -- start start 2024-09-12T03:13:22.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.446+0000 7f6b03ce9740 1 Processor -- start 2024-09-12T03:13:22.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.446+0000 7f6b03ce9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea03a324a0 0x55ea03a3cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:22.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.446+0000 7f6b03ce9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea03a32dc0 0x55ea03a3a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:22.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.446+0000 7f6b03ce9740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea03a3d680 0x55ea03a3a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:22.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.446+0000 7f6b03ce9740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ea03a40250 con 0x55ea03a3d680 2024-09-12T03:13:22.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.446+0000 7f6b03ce9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ea03953050 con 0x55ea03a32dc0 2024-09-12T03:13:22.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.446+0000 7f6b03ce9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ea03a403d0 con 0x55ea03a324a0 2024-09-12T03:13:22.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.446+0000 7f6b03ce9740 10 client.0 _ll_register_callbacks cb 0x55ea03993160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:22.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.447+0000 7f6afb7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea03a3d680 0x55ea03a3a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:22.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.447+0000 7f6afa7fc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea03a32dc0 0x55ea03a3a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:22.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.447+0000 7f6afb7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea03a3d680 0x55ea03a3a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39066/0 (socket says 172.21.15.116:39066) 2024-09-12T03:13:22.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.447+0000 7f6afb7fe640 1 -- 172.21.15.116:0/2262298843 learned_addr learned my addr 172.21.15.116:0/2262298843 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:22.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.447+0000 7f6afaffd640 1 --2- 172.21.15.116:0/2262298843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea03a324a0 0x55ea03a3cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:22.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.447+0000 7f6afb7fe640 1 -- 172.21.15.116:0/2262298843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea03a324a0 msgr2=0x55ea03a3cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:22.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.447+0000 7f6afb7fe640 1 --2- 172.21.15.116:0/2262298843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea03a324a0 0x55ea03a3cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.447+0000 7f6afb7fe640 1 -- 172.21.15.116:0/2262298843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea03a32dc0 msgr2=0x55ea03a3a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:22.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.447+0000 7f6afb7fe640 1 --2- 172.21.15.116:0/2262298843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea03a32dc0 0x55ea03a3a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.447+0000 7f6afb7fe640 1 -- 172.21.15.116:0/2262298843 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea037093c0 con 0x55ea03a3d680 2024-09-12T03:13:22.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.447+0000 7f6afa7fc640 1 --2- 172.21.15.116:0/2262298843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea03a32dc0 0x55ea03a3a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:13:22.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.447+0000 7f6afb7fe640 1 --2- 172.21.15.116:0/2262298843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea03a3d680 0x55ea03a3a730 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f6af402f750 tx=0x7f6af402a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e105d54187d3e263 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:22.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.447+0000 7f6adaffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:22.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.447+0000 7f6adaffd640 1 -- 172.21.15.116:0/2262298843 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af404e030 con 0x55ea03a3d680 2024-09-12T03:13:22.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.447+0000 7f6adaffd640 1 -- 172.21.15.116:0/2262298843 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f6af4004250 con 0x55ea03a3d680 2024-09-12T03:13:22.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.447+0000 7f6b03ce9740 1 -- 172.21.15.116:0/2262298843 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ea039a5850 con 0x55ea03a3d680 2024-09-12T03:13:22.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.447+0000 7f6adaffd640 1 -- 172.21.15.116:0/2262298843 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6af403d070 con 0x55ea03a3d680 2024-09-12T03:13:22.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.447+0000 7f6b03ce9740 10 client.18270 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:22.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.447+0000 7f6b03ce9740 1 -- 172.21.15.116:0/2262298843 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ea03a4bc20 con 0x55ea03a3d680 2024-09-12T03:13:22.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.448+0000 7f6adaffd640 1 -- 172.21.15.116:0/2262298843 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ea039a5850 con 0x55ea03a3d680 2024-09-12T03:13:22.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.448+0000 7f6adaffd640 1 -- 172.21.15.116:0/2262298843 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6af404a980 con 0x55ea03a3d680 2024-09-12T03:13:22.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.448+0000 7f6b03ce9740 10 client.18270 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:22.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.448+0000 7f6b03ce9740 10 client.18270 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:13:22.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.448+0000 7f6b03ce9740 20 client.18270 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:22.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.448+0000 7f6b03ce9740 1 --2- 172.21.15.116:0/2262298843 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ea03a4d030 0x55ea03a4f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:22.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.448+0000 7f6b03ce9740 4 client.18270 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:13:22.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.448+0000 7f6b03ce9740 1 -- 172.21.15.116:0/2262298843 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea0399fd60 con 0x55ea03a4d030 2024-09-12T03:13:22.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.448+0000 7f6afaffd640 1 --2- 172.21.15.116:0/2262298843 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ea03a4d030 0x55ea03a4f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:22.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.448+0000 7f6adaffd640 10 client.18270 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:22.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.448+0000 7f6afaffd640 1 --2- 172.21.15.116:0/2262298843 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ea03a4d030 0x55ea03a4f490 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5c794ab2cc88a52 server_cookie=597c862f16aa38f9 in_seq=0 out_seq=0 2024-09-12T03:13:22.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.449+0000 7f6adaffd640 1 -- 172.21.15.116:0/2262298843 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ea0399fd60 con 0x55ea03a4d030 2024-09-12T03:13:22.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.449+0000 7f6adaffd640 10 client.18270 handle_command_reply: tid=0 2024-09-12T03:13:22.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.449+0000 7f6b03ce9740 1 client.18270 shutdown 2024-09-12T03:13:22.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.449+0000 7f6b03ce9740 1 -- 172.21.15.116:0/2262298843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea03a3d680 msgr2=0x55ea03a3a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:22.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.449+0000 7f6b03ce9740 1 --2- 172.21.15.116:0/2262298843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea03a3d680 0x55ea03a3a730 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f6af402f750 tx=0x7f6af402a950 comp rx=0 tx=0).stop 2024-09-12T03:13:22.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.449+0000 7f6b03ce9740 1 -- 172.21.15.116:0/2262298843 shutdown_connections 2024-09-12T03:13:22.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.449+0000 7f6b03ce9740 1 --2- 172.21.15.116:0/2262298843 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ea03a4d030 0x55ea03a4f490 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.449+0000 7f6b03ce9740 1 --2- 172.21.15.116:0/2262298843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea03a3d680 0x55ea03a3a730 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.449+0000 7f6b03ce9740 1 --2- 172.21.15.116:0/2262298843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea03a32dc0 0x55ea03a3a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.449+0000 7f6b03ce9740 1 --2- 172.21.15.116:0/2262298843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea03a324a0 0x55ea03a3cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.449+0000 7f6adaffd640 0 client.18270 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:22.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.449+0000 7f6b03ce9740 1 -- 172.21.15.116:0/2262298843 >> 172.21.15.116:0/2262298843 conn(0x55ea03a2ea30 msgr2=0x55ea03a34710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:22.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.449+0000 7f6b03ce9740 1 -- 172.21.15.116:0/2262298843 shutdown_connections 2024-09-12T03:13:22.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.449+0000 7f6b03ce9740 1 -- 172.21.15.116:0/2262298843 wait complete. 2024-09-12T03:13:22.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.450+0000 7f6b03ce9740 20 client.18270 trim_cache size 0 max 16384 2024-09-12T03:13:22.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.467+0000 7f6b03ce9740 1 Processor -- start 2024-09-12T03:13:22.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.467+0000 7f6b03ce9740 1 -- start start 2024-09-12T03:13:22.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.467+0000 7f6b03ce9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea03a2ea30 0x55ea03a494e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:22.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.467+0000 7f6b03ce9740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea03a324a0 0x55ea03a3a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:22.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.467+0000 7f6b03ce9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea03a32dc0 0x55ea03a357a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:22.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.467+0000 7f6b03ce9740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6afc1b1f40 con 0x55ea03a324a0 2024-09-12T03:13:22.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.467+0000 7f6b03ce9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6afc1af810 con 0x55ea03a2ea30 2024-09-12T03:13:22.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.467+0000 7f6b03ce9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6afc1ad680 con 0x55ea03a32dc0 2024-09-12T03:13:22.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.467+0000 7f6afa7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea03a324a0 0x55ea03a3a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:22.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.467+0000 7f6afa7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea03a324a0 0x55ea03a3a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39070/0 (socket says 172.21.15.116:39070) 2024-09-12T03:13:22.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.467+0000 7f6afa7fc640 1 -- 172.21.15.116:0/2550255012 learned_addr learned my addr 172.21.15.116:0/2550255012 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:22.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.467+0000 7f6afaffd640 1 --2- 172.21.15.116:0/2550255012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea03a2ea30 0x55ea03a494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:22.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.467+0000 7f6afb7fe640 1 --2- 172.21.15.116:0/2550255012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea03a32dc0 0x55ea03a357a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:22.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.467+0000 7f6afa7fc640 1 -- 172.21.15.116:0/2550255012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea03a32dc0 msgr2=0x55ea03a357a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:22.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.468+0000 7f6afa7fc640 1 --2- 172.21.15.116:0/2550255012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea03a32dc0 0x55ea03a357a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.468+0000 7f6afa7fc640 1 -- 172.21.15.116:0/2550255012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea03a2ea30 msgr2=0x55ea03a494e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:22.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.468+0000 7f6afa7fc640 1 --2- 172.21.15.116:0/2550255012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea03a2ea30 0x55ea03a494e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.468+0000 7f6afa7fc640 1 -- 172.21.15.116:0/2550255012 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea03a38740 con 0x55ea03a324a0 2024-09-12T03:13:22.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.468+0000 7f6afa7fc640 1 --2- 172.21.15.116:0/2550255012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea03a324a0 0x55ea03a3a1b0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f6af0099610 tx=0x7f6af009b130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d61b0e0dbfb4cf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:22.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.468+0000 7f6ae17fa640 1 -- 172.21.15.116:0/2550255012 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af0020040 con 0x55ea03a324a0 2024-09-12T03:13:22.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.468+0000 7f6ae17fa640 1 -- 172.21.15.116:0/2550255012 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6af00a3fd0 con 0x55ea03a324a0 2024-09-12T03:13:22.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.468+0000 7f6ae17fa640 1 -- 172.21.15.116:0/2550255012 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af00b4310 con 0x55ea03a324a0 2024-09-12T03:13:22.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.469+0000 7f6b03ce9740 1 -- 172.21.15.116:0/2550255012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea03a324a0 msgr2=0x55ea03a3a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:22.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.469+0000 7f6b03ce9740 1 --2- 172.21.15.116:0/2550255012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea03a324a0 0x55ea03a3a1b0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f6af0099610 tx=0x7f6af009b130 comp rx=0 tx=0).stop 2024-09-12T03:13:22.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.469+0000 7f6b03ce9740 1 -- 172.21.15.116:0/2550255012 shutdown_connections 2024-09-12T03:13:22.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.469+0000 7f6b03ce9740 1 --2- 172.21.15.116:0/2550255012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea03a32dc0 0x55ea03a357a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.469+0000 7f6b03ce9740 1 --2- 172.21.15.116:0/2550255012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea03a324a0 0x55ea03a3a1b0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.469+0000 7f6b03ce9740 1 --2- 172.21.15.116:0/2550255012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea03a2ea30 0x55ea03a494e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.469+0000 7f6b03ce9740 1 -- 172.21.15.116:0/2550255012 >> 172.21.15.116:0/2550255012 conn(0x55ea03a3a890 msgr2=0x55ea03a35090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:22.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.469+0000 7f6b03ce9740 1 -- 172.21.15.116:0/2550255012 shutdown_connections 2024-09-12T03:13:22.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.469+0000 7f6b03ce9740 1 -- 172.21.15.116:0/2550255012 wait complete. 2024-09-12T03:13:22.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.469+0000 7f6b03ce9740 1 -- start start 2024-09-12T03:13:22.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.469+0000 7f6b03ce9740 1 Processor -- start 2024-09-12T03:13:22.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.470+0000 7f6b03ce9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea03a2ea30 0x55ea03a45f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:22.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.470+0000 7f6b03ce9740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea03a324a0 0x55ea03a37030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:22.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.470+0000 7f6b03ce9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea03a32dc0 0x55ea03a37770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:22.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.470+0000 7f6b03ce9740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6afc1adcd0 con 0x55ea03a324a0 2024-09-12T03:13:22.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.470+0000 7f6b03ce9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6afc1af140 con 0x55ea03a32dc0 2024-09-12T03:13:22.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.470+0000 7f6b03ce9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6afc1ae3a0 con 0x55ea03a2ea30 2024-09-12T03:13:22.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.470+0000 7f6b03ce9740 10 client.0 _ll_register_callbacks cb 0x7f6af402fb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:22.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.470+0000 7f6afa7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea03a324a0 0x55ea03a37030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:22.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.470+0000 7f6afa7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea03a324a0 0x55ea03a37030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39080/0 (socket says 172.21.15.116:39080) 2024-09-12T03:13:22.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.470+0000 7f6afa7fc640 1 -- 172.21.15.116:0/1048243458 learned_addr learned my addr 172.21.15.116:0/1048243458 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:22.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.470+0000 7f6afaffd640 1 --2- 172.21.15.116:0/1048243458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea03a2ea30 0x55ea03a45f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:22.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.470+0000 7f6afb7fe640 1 --2- 172.21.15.116:0/1048243458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea03a32dc0 0x55ea03a37770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:22.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.470+0000 7f6afa7fc640 1 -- 172.21.15.116:0/1048243458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea03a2ea30 msgr2=0x55ea03a45f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:22.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.470+0000 7f6afa7fc640 1 --2- 172.21.15.116:0/1048243458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea03a2ea30 0x55ea03a45f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.470+0000 7f6afa7fc640 1 -- 172.21.15.116:0/1048243458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea03a32dc0 msgr2=0x55ea03a37770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:22.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.470+0000 7f6afa7fc640 1 --2- 172.21.15.116:0/1048243458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea03a32dc0 0x55ea03a37770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.470+0000 7f6afa7fc640 1 -- 172.21.15.116:0/1048243458 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea03ab6d80 con 0x55ea03a324a0 2024-09-12T03:13:22.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.470+0000 7f6afb7fe640 1 --2- 172.21.15.116:0/1048243458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea03a32dc0 0x55ea03a37770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:13:22.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.470+0000 7f6afaffd640 1 --2- 172.21.15.116:0/1048243458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea03a2ea30 0x55ea03a45f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:13:22.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.470+0000 7f6afa7fc640 1 --2- 172.21.15.116:0/1048243458 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea03a324a0 0x55ea03a37030 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f6af0009610 tx=0x7f6af009f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7003a27abd33620d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:22.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.470+0000 7f6adaffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:22.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.470+0000 7f6adaffd640 1 -- 172.21.15.116:0/1048243458 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af0020070 con 0x55ea03a324a0 2024-09-12T03:13:22.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.470+0000 7f6adaffd640 1 -- 172.21.15.116:0/1048243458 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f6af0005560 con 0x55ea03a324a0 2024-09-12T03:13:22.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.470+0000 7f6b03ce9740 1 -- 172.21.15.116:0/1048243458 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6af00a3fd0 con 0x55ea03a324a0 2024-09-12T03:13:22.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.471+0000 7f6adaffd640 1 -- 172.21.15.116:0/1048243458 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6af0009de0 con 0x55ea03a324a0 2024-09-12T03:13:22.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.471+0000 7f6b03ce9740 10 client.18276 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:22.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.471+0000 7f6b03ce9740 1 -- 172.21.15.116:0/1048243458 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ea03ab7ed0 con 0x55ea03a324a0 2024-09-12T03:13:22.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.471+0000 7f6adaffd640 1 -- 172.21.15.116:0/1048243458 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6af00a3fd0 con 0x55ea03a324a0 2024-09-12T03:13:22.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.471+0000 7f6adaffd640 1 -- 172.21.15.116:0/1048243458 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6af009b650 con 0x55ea03a324a0 2024-09-12T03:13:22.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.471+0000 7f6b03ce9740 10 client.18276 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:22.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.471+0000 7f6b03ce9740 10 client.18276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:13:22.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.471+0000 7f6b03ce9740 20 client.18276 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:22.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.471+0000 7f6b03ce9740 1 --2- 172.21.15.116:0/1048243458 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ea03a3d680 0x55ea03aba160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:22.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.471+0000 7f6b03ce9740 4 client.18276 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:13:22.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.471+0000 7f6b03ce9740 1 -- 172.21.15.116:0/1048243458 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ea0391c210 con 0x55ea03a3d680 2024-09-12T03:13:22.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.471+0000 7f6afaffd640 1 --2- 172.21.15.116:0/1048243458 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ea03a3d680 0x55ea03aba160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:22.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.472+0000 7f6afaffd640 1 --2- 172.21.15.116:0/1048243458 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ea03a3d680 0x55ea03aba160 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff6ea2be73465eb6 server_cookie=d837151bafbcdc5e in_seq=0 out_seq=0 2024-09-12T03:13:22.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.472+0000 7f6adaffd640 10 client.18276 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:22.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.472+0000 7f6adaffd640 1 -- 172.21.15.116:0/1048243458 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ea0391c210 con 0x55ea03a3d680 2024-09-12T03:13:22.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.472+0000 7f6adaffd640 10 client.18276 handle_command_reply: tid=0 2024-09-12T03:13:22.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.472+0000 7f6b03ce9740 1 client.18276 shutdown 2024-09-12T03:13:22.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.472+0000 7f6b03ce9740 1 -- 172.21.15.116:0/1048243458 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea03a324a0 msgr2=0x55ea03a37030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:22.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.472+0000 7f6b03ce9740 1 --2- 172.21.15.116:0/1048243458 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea03a324a0 0x55ea03a37030 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f6af0009610 tx=0x7f6af009f710 comp rx=0 tx=0).stop 2024-09-12T03:13:22.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.472+0000 7f6b03ce9740 1 -- 172.21.15.116:0/1048243458 shutdown_connections 2024-09-12T03:13:22.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.472+0000 7f6b03ce9740 1 --2- 172.21.15.116:0/1048243458 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ea03a3d680 0x55ea03aba160 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.472+0000 7f6b03ce9740 1 --2- 172.21.15.116:0/1048243458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea03a32dc0 0x55ea03a37770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.472+0000 7f6b03ce9740 1 --2- 172.21.15.116:0/1048243458 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea03a324a0 0x55ea03a37030 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.472+0000 7f6adaffd640 0 client.18276 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:22.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.472+0000 7f6b03ce9740 1 --2- 172.21.15.116:0/1048243458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea03a2ea30 0x55ea03a45f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.472+0000 7f6b03ce9740 1 -- 172.21.15.116:0/1048243458 >> 172.21.15.116:0/1048243458 conn(0x55ea03a3a890 msgr2=0x55ea03a35890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:22.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.472+0000 7f6b03ce9740 1 -- 172.21.15.116:0/1048243458 shutdown_connections 2024-09-12T03:13:22.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.472+0000 7f6b03ce9740 1 -- 172.21.15.116:0/1048243458 wait complete. 2024-09-12T03:13:22.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.472+0000 7f6b03ce9740 20 client.18276 trim_cache size 0 max 16384 2024-09-12T03:13:22.484 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:13:22.484 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:13:22.484 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": { 2024-09-12T03:13:22.484 INFO:teuthology.orchestra.run.smithi116.stdout: "33fcd4f1-db99-48b2-8fd6-6397b41af3a2": { 2024-09-12T03:13:22.484 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/", 2024-09-12T03:13:22.484 INFO:teuthology.orchestra.run.smithi116.stdout: "tag": "33fcd4f1-db99-48b2-8fd6-6397b41af3a2", 2024-09-12T03:13:22.484 INFO:teuthology.orchestra.run.smithi116.stdout: "options": "recursive,force" 2024-09-12T03:13:22.484 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:13:22.484 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:13:22.484 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:13:22.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.473+0000 7f6b01910640 1 -- 172.21.15.116:0/2485327351 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6ae40a2b80 msgr2=0x7f6ae40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:22.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.473+0000 7f6b01910640 1 --2- 172.21.15.116:0/2485327351 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6ae40a2b80 0x7f6ae40a5040 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6af4005fc0 tx=0x7f6af40095c0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.473+0000 7f6b01910640 1 -- 172.21.15.116:0/2485327351 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6afc103660 msgr2=0x7f6afc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:22.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.473+0000 7f6b01910640 1 --2- 172.21.15.116:0/2485327351 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6afc103660 0x7f6afc100ee0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f6af000d750 tx=0x7f6af000dc10 comp rx=0 tx=0).stop 2024-09-12T03:13:22.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.473+0000 7f6b01910640 1 -- 172.21.15.116:0/2485327351 shutdown_connections 2024-09-12T03:13:22.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.473+0000 7f6b01910640 1 --2- 172.21.15.116:0/2485327351 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6ae40a2b80 0x7f6ae40a5040 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.473+0000 7f6b01910640 1 --2- 172.21.15.116:0/2485327351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6afc10c170 0x7f6afc101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.473+0000 7f6b01910640 1 --2- 172.21.15.116:0/2485327351 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6afc103660 0x7f6afc100ee0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.473+0000 7f6b01910640 1 --2- 172.21.15.116:0/2485327351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6afc102d40 0x7f6afc1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:22.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.473+0000 7f6b01910640 1 -- 172.21.15.116:0/2485327351 >> 172.21.15.116:0/2485327351 conn(0x7f6afc0fc7a0 msgr2=0x7f6afc0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:22.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.473+0000 7f6b01910640 1 -- 172.21.15.116:0/2485327351 shutdown_connections 2024-09-12T03:13:22.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:22.473+0000 7f6b01910640 1 -- 172.21.15.116:0/2485327351 wait complete. 2024-09-12T03:13:22.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:22 smithi177 ceph-mon[35305]: pgmap v649: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.2 KiB/s wr, 9 op/s 2024-09-12T03:13:23.191 INFO:tasks.cephfs.filesystem:scrub status for tag:33fcd4f1-db99-48b2-8fd6-6397b41af3a2 - {'path': '/', 'tag': '33fcd4f1-db99-48b2-8fd6-6397b41af3a2', 'options': 'recursive,force'} 2024-09-12T03:13:23.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:23 smithi183 ceph-mon[35535]: pgmap v650: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.2 KiB/s wr, 7 op/s 2024-09-12T03:13:23.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:23 smithi177 ceph-mon[35305]: pgmap v650: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.2 KiB/s wr, 7 op/s 2024-09-12T03:13:23.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:23 smithi116 ceph-mon[31410]: pgmap v650: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.2 KiB/s wr, 7 op/s 2024-09-12T03:13:24.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:24 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:13:24.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:24 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:13:24.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:24 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:13:25.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:25 smithi183 ceph-mon[35535]: pgmap v651: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.0 KiB/s wr, 9 op/s 2024-09-12T03:13:25.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:25 smithi177 ceph-mon[35305]: pgmap v651: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.0 KiB/s wr, 9 op/s 2024-09-12T03:13:25.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:25 smithi116 ceph-mon[31410]: pgmap v651: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.0 KiB/s wr, 9 op/s 2024-09-12T03:13:27.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:27 smithi183 ceph-mon[35535]: pgmap v652: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:13:27.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:27 smithi177 ceph-mon[35305]: pgmap v652: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:13:27.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:27 smithi116 ceph-mon[31410]: pgmap v652: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:13:29.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:29 smithi183 ceph-mon[35535]: pgmap v653: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:13:29.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:29 smithi177 ceph-mon[35305]: pgmap v653: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:13:29.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:29 smithi116 ceph-mon[31410]: pgmap v653: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:13:31.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:31 smithi183 ceph-mon[35535]: pgmap v654: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-12T03:13:31.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:31 smithi177 ceph-mon[35305]: pgmap v654: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-12T03:13:31.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:31 smithi116 ceph-mon[31410]: pgmap v654: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-12T03:13:32.007 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:13:32.266 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:13:32.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.769+0000 7f3e0798d640 1 -- 172.21.15.116:0/1227712768 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e001027a0 msgr2=0x7f3e001067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:32.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.769+0000 7f3e0798d640 1 --2- 172.21.15.116:0/1227712768 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e001027a0 0x7f3e001067f0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f3df4009900 tx=0x7f3df402a410 comp rx=0 tx=0).stop 2024-09-12T03:13:32.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.769+0000 7f3e0798d640 1 -- 172.21.15.116:0/1227712768 shutdown_connections 2024-09-12T03:13:32.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.769+0000 7f3e0798d640 1 --2- 172.21.15.116:0/1227712768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e00106f20 0x7f3e0010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:32.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.769+0000 7f3e0798d640 1 --2- 172.21.15.116:0/1227712768 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e001027a0 0x7f3e001067f0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:32.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.769+0000 7f3e0798d640 1 --2- 172.21.15.116:0/1227712768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e00101df0 0x7f3e001021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:32.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.769+0000 7f3e0798d640 1 -- 172.21.15.116:0/1227712768 >> 172.21.15.116:0/1227712768 conn(0x7f3e000fd8f0 msgr2=0x7f3e000ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:32.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.770+0000 7f3e0798d640 1 -- 172.21.15.116:0/1227712768 shutdown_connections 2024-09-12T03:13:32.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.770+0000 7f3e0798d640 1 -- 172.21.15.116:0/1227712768 wait complete. 2024-09-12T03:13:32.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.770+0000 7f3e0798d640 1 Processor -- start 2024-09-12T03:13:32.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.770+0000 7f3e0798d640 1 -- start start 2024-09-12T03:13:32.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.770+0000 7f3e0798d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e00101df0 0x7f3e001984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:32.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.770+0000 7f3e0798d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e001027a0 0x7f3e001989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:32.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.770+0000 7f3e0798d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e00106f20 0x7f3e0019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:32.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.770+0000 7f3e0798d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3e00111fa0 con 0x7f3e00101df0 2024-09-12T03:13:32.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.770+0000 7f3e0798d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3e00111e20 con 0x7f3e00106f20 2024-09-12T03:13:32.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.770+0000 7f3e0798d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3e00112120 con 0x7f3e001027a0 2024-09-12T03:13:32.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.771+0000 7f3e05729640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e00101df0 0x7f3e001984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:32.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.771+0000 7f3e05729640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e00101df0 0x7f3e001984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48640/0 (socket says 172.21.15.116:48640) 2024-09-12T03:13:32.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.771+0000 7f3e05729640 1 -- 172.21.15.116:0/4020069545 learned_addr learned my addr 172.21.15.116:0/4020069545 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:32.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.771+0000 7f3e05f2a640 1 --2- 172.21.15.116:0/4020069545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e00106f20 0x7f3e0019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:32.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.771+0000 7f3e04f28640 1 --2- 172.21.15.116:0/4020069545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e001027a0 0x7f3e001989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:32.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.771+0000 7f3e05729640 1 -- 172.21.15.116:0/4020069545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e001027a0 msgr2=0x7f3e001989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:32.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.771+0000 7f3e05729640 1 --2- 172.21.15.116:0/4020069545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e001027a0 0x7f3e001989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:32.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.771+0000 7f3e05729640 1 -- 172.21.15.116:0/4020069545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e00106f20 msgr2=0x7f3e0019cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:32.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.771+0000 7f3e05729640 1 --2- 172.21.15.116:0/4020069545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e00106f20 0x7f3e0019cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:32.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.771+0000 7f3e05729640 1 -- 172.21.15.116:0/4020069545 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e0019d450 con 0x7f3e00101df0 2024-09-12T03:13:32.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.771+0000 7f3e05f2a640 1 --2- 172.21.15.116:0/4020069545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e00106f20 0x7f3e0019cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:13:32.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.771+0000 7f3e05729640 1 --2- 172.21.15.116:0/4020069545 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e00101df0 0x7f3e001984a0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f3df000cce0 tx=0x7f3df000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:32.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.771+0000 7f3dfe7fc640 1 -- 172.21.15.116:0/4020069545 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df0004b20 con 0x7f3e00101df0 2024-09-12T03:13:32.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.771+0000 7f3e0798d640 1 -- 172.21.15.116:0/4020069545 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e0019d740 con 0x7f3e00101df0 2024-09-12T03:13:32.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.771+0000 7f3dfe7fc640 1 -- 172.21.15.116:0/4020069545 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3df0004cc0 con 0x7f3e00101df0 2024-09-12T03:13:32.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.771+0000 7f3dfe7fc640 1 -- 172.21.15.116:0/4020069545 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df000f7d0 con 0x7f3e00101df0 2024-09-12T03:13:32.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.771+0000 7f3e0798d640 1 -- 172.21.15.116:0/4020069545 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e001a5020 con 0x7f3e00101df0 2024-09-12T03:13:32.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.772+0000 7f3e0798d640 1 -- 172.21.15.116:0/4020069545 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e001021d0 con 0x7f3e00101df0 2024-09-12T03:13:32.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.773+0000 7f3dfe7fc640 1 -- 172.21.15.116:0/4020069545 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3df000f970 con 0x7f3e00101df0 2024-09-12T03:13:32.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.774+0000 7f3dfe7fc640 1 --2- 172.21.15.116:0/4020069545 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3de80a2b80 0x7f3de80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:32.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.774+0000 7f3e04f28640 1 --2- 172.21.15.116:0/4020069545 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3de80a2b80 0x7f3de80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:32.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.774+0000 7f3dfe7fc640 1 -- 172.21.15.116:0/4020069545 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f3df00cf550 con 0x7f3e00101df0 2024-09-12T03:13:32.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.774+0000 7f3e04f28640 1 --2- 172.21.15.116:0/4020069545 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3de80a2b80 0x7f3de80a5040 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3e000fdd10 tx=0x7f3df4031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:32.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.777+0000 7f3dfe7fc640 1 -- 172.21.15.116:0/4020069545 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3df0014030 con 0x7f3e00101df0 2024-09-12T03:13:32.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.976+0000 7f3e0798d640 1 -- 172.21.15.116:0/4020069545 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e0010a190 con 0x7f3e00101df0 2024-09-12T03:13:32.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.977+0000 7f3dfe7fc640 1 -- 172.21.15.116:0/4020069545 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f3df009c4e0 con 0x7f3e00101df0 2024-09-12T03:13:32.977 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:13:32.977 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:13:32.978 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:13:32.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.979+0000 7f3e0798d640 1 -- 172.21.15.116:0/4020069545 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3de80a2b80 msgr2=0x7f3de80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:32.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.979+0000 7f3e0798d640 1 --2- 172.21.15.116:0/4020069545 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3de80a2b80 0x7f3de80a5040 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3e000fdd10 tx=0x7f3df4031490 comp rx=0 tx=0).stop 2024-09-12T03:13:32.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.979+0000 7f3e0798d640 1 -- 172.21.15.116:0/4020069545 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e00101df0 msgr2=0x7f3e001984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:32.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.979+0000 7f3e0798d640 1 --2- 172.21.15.116:0/4020069545 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e00101df0 0x7f3e001984a0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f3df000cce0 tx=0x7f3df000a620 comp rx=0 tx=0).stop 2024-09-12T03:13:32.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.979+0000 7f3e0798d640 1 -- 172.21.15.116:0/4020069545 shutdown_connections 2024-09-12T03:13:32.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.979+0000 7f3e0798d640 1 --2- 172.21.15.116:0/4020069545 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3de80a2b80 0x7f3de80a5040 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:32.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.979+0000 7f3e0798d640 1 --2- 172.21.15.116:0/4020069545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e00106f20 0x7f3e0019cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:32.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.979+0000 7f3e0798d640 1 --2- 172.21.15.116:0/4020069545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e001027a0 0x7f3e001989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:32.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.979+0000 7f3e0798d640 1 --2- 172.21.15.116:0/4020069545 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e00101df0 0x7f3e001984a0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:32.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.979+0000 7f3e0798d640 1 -- 172.21.15.116:0/4020069545 >> 172.21.15.116:0/4020069545 conn(0x7f3e000fd8f0 msgr2=0x7f3e000feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:32.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.979+0000 7f3e0798d640 1 -- 172.21.15.116:0/4020069545 shutdown_connections 2024-09-12T03:13:32.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:32.979+0000 7f3e0798d640 1 -- 172.21.15.116:0/4020069545 wait complete. 2024-09-12T03:13:33.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:33 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/4020069545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:13:33.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:33 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/4020069545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:13:33.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:33 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/4020069545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:13:33.945 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id 904ad12d --release --await' 2024-09-12T03:13:33.946 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id 904ad12d --release --await 2024-09-12T03:13:34.198 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:13:34.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:34 smithi116 ceph-mon[31410]: pgmap v655: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-12T03:13:34.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:34 smithi183 ceph-mon[35535]: pgmap v655: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-12T03:13:34.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:34 smithi177 ceph-mon[35305]: pgmap v655: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-12T03:13:34.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.736+0000 7f69dc6e2640 1 -- 172.21.15.116:0/3208717196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69d41002f0 msgr2=0x7f69d41006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:34.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.736+0000 7f69dc6e2640 1 --2- 172.21.15.116:0/3208717196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69d41002f0 0x7f69d41006d0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f69d0009930 tx=0x7f69d002a420 comp rx=0 tx=0).stop 2024-09-12T03:13:34.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.736+0000 7f69dc6e2640 1 -- 172.21.15.116:0/3208717196 shutdown_connections 2024-09-12T03:13:34.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.736+0000 7f69dc6e2640 1 --2- 172.21.15.116:0/3208717196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69d4106f50 0x7f69d410eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.736+0000 7f69dc6e2640 1 --2- 172.21.15.116:0/3208717196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69d4100ca0 0x7f69d4106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.736+0000 7f69dc6e2640 1 --2- 172.21.15.116:0/3208717196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69d41002f0 0x7f69d41006d0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.736+0000 7f69dc6e2640 1 -- 172.21.15.116:0/3208717196 >> 172.21.15.116:0/3208717196 conn(0x7f69d4077210 msgr2=0x7f69d40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:34.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.736+0000 7f69dc6e2640 1 -- 172.21.15.116:0/3208717196 shutdown_connections 2024-09-12T03:13:34.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.736+0000 7f69dc6e2640 1 -- 172.21.15.116:0/3208717196 wait complete. 2024-09-12T03:13:34.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.737+0000 7f69dc6e2640 1 Processor -- start 2024-09-12T03:13:34.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.737+0000 7f69dc6e2640 1 -- start start 2024-09-12T03:13:34.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.737+0000 7f69dc6e2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69d41002f0 0x7f69d4101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:34.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.737+0000 7f69dc6e2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69d4100ca0 0x7f69d41017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:34.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.737+0000 7f69dc6e2640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69d4106f50 0x7f69d4101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:34.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.737+0000 7f69dc6e2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f69d4112000 con 0x7f69d4100ca0 2024-09-12T03:13:34.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.737+0000 7f69dc6e2640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f69d4111e80 con 0x7f69d4106f50 2024-09-12T03:13:34.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.737+0000 7f69dc6e2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f69d4112180 con 0x7f69d41002f0 2024-09-12T03:13:34.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.737+0000 7f69d9c7d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69d4100ca0 0x7f69d41017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:34.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.737+0000 7f69d9c7d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69d4100ca0 0x7f69d41017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48672/0 (socket says 172.21.15.116:48672) 2024-09-12T03:13:34.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.737+0000 7f69d9c7d640 1 -- 172.21.15.116:0/4053813526 learned_addr learned my addr 172.21.15.116:0/4053813526 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:34.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.737+0000 7f69da47e640 1 --2- 172.21.15.116:0/4053813526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69d41002f0 0x7f69d4101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:34.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.737+0000 7f69dac7f640 1 --2- 172.21.15.116:0/4053813526 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69d4106f50 0x7f69d4101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:34.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.738+0000 7f69d9c7d640 1 -- 172.21.15.116:0/4053813526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69d41002f0 msgr2=0x7f69d4101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:34.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.738+0000 7f69d9c7d640 1 --2- 172.21.15.116:0/4053813526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69d41002f0 0x7f69d4101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.738+0000 7f69d9c7d640 1 -- 172.21.15.116:0/4053813526 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69d4106f50 msgr2=0x7f69d4101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:34.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.738+0000 7f69d9c7d640 1 --2- 172.21.15.116:0/4053813526 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69d4106f50 0x7f69d4101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.738+0000 7f69d9c7d640 1 -- 172.21.15.116:0/4053813526 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69d41a69e0 con 0x7f69d4100ca0 2024-09-12T03:13:34.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.738+0000 7f69dac7f640 1 --2- 172.21.15.116:0/4053813526 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69d4106f50 0x7f69d4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:13:34.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.738+0000 7f69d9c7d640 1 --2- 172.21.15.116:0/4053813526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69d4100ca0 0x7f69d41017b0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f69c400c970 tx=0x7f69c400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:34.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.738+0000 7f69cb7fe640 1 -- 172.21.15.116:0/4053813526 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c4014070 con 0x7f69d4100ca0 2024-09-12T03:13:34.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.738+0000 7f69cb7fe640 1 -- 172.21.15.116:0/4053813526 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69c4004440 con 0x7f69d4100ca0 2024-09-12T03:13:34.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.738+0000 7f69dc6e2640 1 -- 172.21.15.116:0/4053813526 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69d41a6cd0 con 0x7f69d4100ca0 2024-09-12T03:13:34.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.738+0000 7f69cb7fe640 1 -- 172.21.15.116:0/4053813526 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c400f450 con 0x7f69d4100ca0 2024-09-12T03:13:34.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.738+0000 7f69dc6e2640 1 -- 172.21.15.116:0/4053813526 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69d41a71c0 con 0x7f69d4100ca0 2024-09-12T03:13:34.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.740+0000 7f69cb7fe640 1 -- 172.21.15.116:0/4053813526 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f69c4017070 con 0x7f69d4100ca0 2024-09-12T03:13:34.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.741+0000 7f69deabb740 1 Processor -- start 2024-09-12T03:13:34.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.741+0000 7f69deabb740 1 -- start start 2024-09-12T03:13:34.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.741+0000 7f69deabb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5691eb060 0x55b5691eb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:34.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.741+0000 7f69deabb740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5691eb980 0x55b5691f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:34.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.741+0000 7f69deabb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5691f6240 0x55b5691f8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:34.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.741+0000 7f69deabb740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b569150620 con 0x55b5691eb980 2024-09-12T03:13:34.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.741+0000 7f69deabb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b5690f7e50 con 0x55b5691f6240 2024-09-12T03:13:34.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.741+0000 7f69deabb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b56914c5e0 con 0x55b5691eb060 2024-09-12T03:13:34.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.741+0000 7f69da47e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5691eb980 0x55b5691f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:34.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.741+0000 7f69cb7fe640 1 --2- 172.21.15.116:0/4053813526 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f69c00a2b80 0x7f69c00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:34.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.741+0000 7f69da47e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5691eb980 0x55b5691f5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48684/0 (socket says 172.21.15.116:48684) 2024-09-12T03:13:34.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.741+0000 7f69da47e640 1 -- 172.21.15.116:0/2213572533 learned_addr learned my addr 172.21.15.116:0/2213572533 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:34.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.741+0000 7f69dac7f640 1 --2- 172.21.15.116:0/2213572533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5691eb060 0x55b5691eb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:34.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.741+0000 7f69dac7f640 1 --2- 172.21.15.116:0/4053813526 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f69c00a2b80 0x7f69c00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:34.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.741+0000 7f69d9c7d640 1 --2- 172.21.15.116:0/2213572533 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5691f6240 0x55b5691f8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:34.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.741+0000 7f69cb7fe640 1 -- 172.21.15.116:0/4053813526 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f69c40cef20 con 0x7f69d4100ca0 2024-09-12T03:13:34.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.741+0000 7f69da47e640 1 -- 172.21.15.116:0/2213572533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5691eb060 msgr2=0x55b5691eb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:34.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.742+0000 7f69da47e640 1 --2- 172.21.15.116:0/2213572533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5691eb060 0x55b5691eb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.742+0000 7f69da47e640 1 -- 172.21.15.116:0/2213572533 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5691f6240 msgr2=0x55b5691f8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:34.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.742+0000 7f69da47e640 1 --2- 172.21.15.116:0/2213572533 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5691f6240 0x55b5691f8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.742+0000 7f69da47e640 1 -- 172.21.15.116:0/2213572533 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5690e8d40 con 0x55b5691eb980 2024-09-12T03:13:34.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.742+0000 7f69d9c7d640 1 --2- 172.21.15.116:0/2213572533 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5691f6240 0x55b5691f8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:13:34.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.742+0000 7f69da47e640 1 --2- 172.21.15.116:0/2213572533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5691eb980 0x55b5691f5d00 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f69d0005ab0 tx=0x7f69d0031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfec53d3093899c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:34.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.742+0000 7f69dac7f640 1 --2- 172.21.15.116:0/4053813526 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f69c00a2b80 0x7f69c00a5040 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f69cc009fd0 tx=0x7f69cc008220 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:34.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.742+0000 7f69c8ff9640 1 -- 172.21.15.116:0/2213572533 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d003f070 con 0x55b5691eb980 2024-09-12T03:13:34.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.742+0000 7f69c8ff9640 1 -- 172.21.15.116:0/2213572533 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69d0031a20 con 0x55b5691eb980 2024-09-12T03:13:34.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.742+0000 7f69c8ff9640 1 -- 172.21.15.116:0/2213572533 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d0035070 con 0x55b5691eb980 2024-09-12T03:13:34.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.743+0000 7f69deabb740 1 -- 172.21.15.116:0/2213572533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5691eb980 msgr2=0x55b5691f5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:34.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.743+0000 7f69deabb740 1 --2- 172.21.15.116:0/2213572533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5691eb980 0x55b5691f5d00 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f69d0005ab0 tx=0x7f69d0031630 comp rx=0 tx=0).stop 2024-09-12T03:13:34.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.743+0000 7f69deabb740 1 -- 172.21.15.116:0/2213572533 shutdown_connections 2024-09-12T03:13:34.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.743+0000 7f69deabb740 1 --2- 172.21.15.116:0/2213572533 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5691f6240 0x55b5691f8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.743+0000 7f69deabb740 1 --2- 172.21.15.116:0/2213572533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5691eb980 0x55b5691f5d00 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.743+0000 7f69deabb740 1 --2- 172.21.15.116:0/2213572533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5691eb060 0x55b5691eb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.743+0000 7f69deabb740 1 -- 172.21.15.116:0/2213572533 >> 172.21.15.116:0/2213572533 conn(0x55b5691e75f0 msgr2=0x55b5691e79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:34.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.743+0000 7f69deabb740 1 -- 172.21.15.116:0/2213572533 shutdown_connections 2024-09-12T03:13:34.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.743+0000 7f69deabb740 1 -- 172.21.15.116:0/2213572533 wait complete. 2024-09-12T03:13:34.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.743+0000 7f69deabb740 1 -- start start 2024-09-12T03:13:34.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.743+0000 7f69deabb740 1 Processor -- start 2024-09-12T03:13:34.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.744+0000 7f69deabb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5691eb060 0x55b5691f3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:34.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.744+0000 7f69deabb740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5691eb980 0x55b5691f35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:34.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.744+0000 7f69deabb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5691f6240 0x55b5691f3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:34.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.744+0000 7f69deabb740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b5691f8e10 con 0x55b5691eb980 2024-09-12T03:13:34.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.744+0000 7f69deabb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b5690d8f10 con 0x55b5691f6240 2024-09-12T03:13:34.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.744+0000 7f69deabb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b5691f8f90 con 0x55b5691eb060 2024-09-12T03:13:34.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.744+0000 7f69deabb740 10 client.0 _ll_register_callbacks cb 0x55b5690fcdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:34.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.744+0000 7f69d9c7d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5691eb980 0x55b5691f35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:34.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.744+0000 7f69d9c7d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5691eb980 0x55b5691f35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48700/0 (socket says 172.21.15.116:48700) 2024-09-12T03:13:34.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.744+0000 7f69d9c7d640 1 -- 172.21.15.116:0/3672848953 learned_addr learned my addr 172.21.15.116:0/3672848953 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:34.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.744+0000 7f69da47e640 1 --2- 172.21.15.116:0/3672848953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5691eb060 0x55b5691f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:34.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.744+0000 7f69dac7f640 1 --2- 172.21.15.116:0/3672848953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5691f6240 0x55b5691f3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:34.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.744+0000 7f69d9c7d640 1 -- 172.21.15.116:0/3672848953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5691eb060 msgr2=0x55b5691f3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:34.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.744+0000 7f69d9c7d640 1 --2- 172.21.15.116:0/3672848953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5691eb060 0x55b5691f3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.744+0000 7f69d9c7d640 1 -- 172.21.15.116:0/3672848953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5691f6240 msgr2=0x55b5691f3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:34.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.744+0000 7f69d9c7d640 1 --2- 172.21.15.116:0/3672848953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5691f6240 0x55b5691f3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.744+0000 7f69d9c7d640 1 -- 172.21.15.116:0/3672848953 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b568e5f580 con 0x55b5691eb980 2024-09-12T03:13:34.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.744+0000 7f69da47e640 1 --2- 172.21.15.116:0/3672848953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5691eb060 0x55b5691f3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:13:34.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.744+0000 7f69d9c7d640 1 --2- 172.21.15.116:0/3672848953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5691eb980 0x55b5691f35d0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f69c40a3ef0 tx=0x7f69c4015450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e730b4c0c0b85a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:34.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.744+0000 7f69bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:34.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.744+0000 7f69bdffb640 1 -- 172.21.15.116:0/3672848953 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c40a4420 con 0x55b5691eb980 2024-09-12T03:13:34.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.744+0000 7f69bdffb640 1 -- 172.21.15.116:0/3672848953 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f69c4097e20 con 0x55b5691eb980 2024-09-12T03:13:34.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.744+0000 7f69deabb740 1 -- 172.21.15.116:0/3672848953 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69d0031a20 con 0x55b5691eb980 2024-09-12T03:13:34.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.744+0000 7f69bdffb640 1 -- 172.21.15.116:0/3672848953 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f69c4015260 con 0x55b5691eb980 2024-09-12T03:13:34.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.744+0000 7f69deabb740 10 client.18294 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:34.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.744+0000 7f69deabb740 1 -- 172.21.15.116:0/3672848953 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b568f29960 con 0x55b5691eb980 2024-09-12T03:13:34.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.745+0000 7f69bdffb640 1 -- 172.21.15.116:0/3672848953 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69d0031a20 con 0x55b5691eb980 2024-09-12T03:13:34.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.745+0000 7f69bdffb640 1 -- 172.21.15.116:0/3672848953 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f69c409ba10 con 0x55b5691eb980 2024-09-12T03:13:34.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.745+0000 7f69deabb740 10 client.18294 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:34.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.745+0000 7f69deabb740 10 client.18294 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:13:34.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.745+0000 7f69deabb740 20 client.18294 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:34.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.745+0000 7f69deabb740 1 --2- 172.21.15.116:0/3672848953 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b569205d10 0x55b569208120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:34.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.745+0000 7f69deabb740 4 client.18294 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:13:34.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.745+0000 7f69deabb740 1 -- 172.21.15.116:0/3672848953 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b56915db90 con 0x55b569205d10 2024-09-12T03:13:34.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.745+0000 7f69da47e640 1 --2- 172.21.15.116:0/3672848953 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b569205d10 0x55b569208120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:34.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.746+0000 7f69da47e640 1 --2- 172.21.15.116:0/3672848953 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b569205d10 0x55b569208120 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21104ac9e4fb5ff5 server_cookie=a6d9a7869ba6f8b7 in_seq=0 out_seq=0 2024-09-12T03:13:34.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.746+0000 7f69bdffb640 10 client.18294 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:34.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.746+0000 7f69bdffb640 1 -- 172.21.15.116:0/3672848953 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b56915db90 con 0x55b569205d10 2024-09-12T03:13:34.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.746+0000 7f69bdffb640 10 client.18294 handle_command_reply: tid=0 2024-09-12T03:13:34.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.746+0000 7f69deabb740 1 client.18294 shutdown 2024-09-12T03:13:34.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.746+0000 7f69deabb740 1 -- 172.21.15.116:0/3672848953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5691eb980 msgr2=0x55b5691f35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:34.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.746+0000 7f69deabb740 1 --2- 172.21.15.116:0/3672848953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5691eb980 0x55b5691f35d0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f69c40a3ef0 tx=0x7f69c4015450 comp rx=0 tx=0).stop 2024-09-12T03:13:34.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.746+0000 7f69deabb740 1 -- 172.21.15.116:0/3672848953 shutdown_connections 2024-09-12T03:13:34.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.746+0000 7f69deabb740 1 --2- 172.21.15.116:0/3672848953 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b569205d10 0x55b569208120 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.746+0000 7f69deabb740 1 --2- 172.21.15.116:0/3672848953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5691f6240 0x55b5691f3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.746+0000 7f69bdffb640 0 client.18294 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:34.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.746+0000 7f69deabb740 1 --2- 172.21.15.116:0/3672848953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5691eb980 0x55b5691f35d0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.746+0000 7f69deabb740 1 --2- 172.21.15.116:0/3672848953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5691eb060 0x55b5691f3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.746+0000 7f69deabb740 1 -- 172.21.15.116:0/3672848953 >> 172.21.15.116:0/3672848953 conn(0x55b5691e75f0 msgr2=0x55b5691ed2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:34.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.746+0000 7f69deabb740 1 -- 172.21.15.116:0/3672848953 shutdown_connections 2024-09-12T03:13:34.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.747+0000 7f69deabb740 1 -- 172.21.15.116:0/3672848953 wait complete. 2024-09-12T03:13:34.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.747+0000 7f69deabb740 20 client.18294 trim_cache size 0 max 16384 2024-09-12T03:13:34.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.763+0000 7f69deabb740 1 Processor -- start 2024-09-12T03:13:34.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.763+0000 7f69deabb740 1 -- start start 2024-09-12T03:13:34.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.763+0000 7f69deabb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5691eb060 0x55b56920d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:34.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.763+0000 7f69deabb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5691eb980 0x55b56920df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:34.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.763+0000 7f69deabb740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5691f6240 0x55b5691f0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:34.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.763+0000 7f69deabb740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f69d41b1f90 con 0x55b5691f6240 2024-09-12T03:13:34.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.763+0000 7f69deabb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f69d41af860 con 0x55b5691eb980 2024-09-12T03:13:34.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.763+0000 7f69deabb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f69d41ad6d0 con 0x55b5691eb060 2024-09-12T03:13:34.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.764+0000 7f69dac7f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5691f6240 0x55b5691f0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:34.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.764+0000 7f69dac7f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5691f6240 0x55b5691f0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48702/0 (socket says 172.21.15.116:48702) 2024-09-12T03:13:34.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.764+0000 7f69dac7f640 1 -- 172.21.15.116:0/367596853 learned_addr learned my addr 172.21.15.116:0/367596853 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:34.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.764+0000 7f69da47e640 1 --2- 172.21.15.116:0/367596853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5691eb060 0x55b56920d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:34.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.764+0000 7f69d9c7d640 1 --2- 172.21.15.116:0/367596853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5691eb980 0x55b56920df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:34.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.764+0000 7f69dac7f640 1 -- 172.21.15.116:0/367596853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5691eb060 msgr2=0x55b56920d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:34.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.764+0000 7f69dac7f640 1 --2- 172.21.15.116:0/367596853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5691eb060 0x55b56920d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.764+0000 7f69dac7f640 1 -- 172.21.15.116:0/367596853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5691eb980 msgr2=0x55b56920df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:34.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.764+0000 7f69dac7f640 1 --2- 172.21.15.116:0/367596853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5691eb980 0x55b56920df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.764+0000 7f69dac7f640 1 -- 172.21.15.116:0/367596853 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5691e7420 con 0x55b5691f6240 2024-09-12T03:13:34.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.764+0000 7f69d9c7d640 1 --2- 172.21.15.116:0/367596853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5691eb980 0x55b56920df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:13:34.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.764+0000 7f69dac7f640 1 --2- 172.21.15.116:0/367596853 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5691f6240 0x55b5691f0310 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f69cc00e8e0 tx=0x7f69cc0096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b679bd350b3a6d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:34.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.764+0000 7f69bffff640 1 -- 172.21.15.116:0/367596853 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc02a070 con 0x55b5691f6240 2024-09-12T03:13:34.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.764+0000 7f69bffff640 1 -- 172.21.15.116:0/367596853 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69cc00ce30 con 0x55b5691f6240 2024-09-12T03:13:34.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.764+0000 7f69bffff640 1 -- 172.21.15.116:0/367596853 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc025990 con 0x55b5691f6240 2024-09-12T03:13:34.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.765+0000 7f69deabb740 1 -- 172.21.15.116:0/367596853 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5691f6240 msgr2=0x55b5691f0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:34.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.765+0000 7f69deabb740 1 --2- 172.21.15.116:0/367596853 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5691f6240 0x55b5691f0310 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f69cc00e8e0 tx=0x7f69cc0096f0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.765+0000 7f69deabb740 1 -- 172.21.15.116:0/367596853 shutdown_connections 2024-09-12T03:13:34.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.765+0000 7f69deabb740 1 --2- 172.21.15.116:0/367596853 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5691f6240 0x55b5691f0310 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.765+0000 7f69deabb740 1 --2- 172.21.15.116:0/367596853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5691eb980 0x55b56920df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.765+0000 7f69deabb740 1 --2- 172.21.15.116:0/367596853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5691eb060 0x55b56920d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.765+0000 7f69deabb740 1 -- 172.21.15.116:0/367596853 >> 172.21.15.116:0/367596853 conn(0x55b5691e75f0 msgr2=0x55b5691f1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:34.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.765+0000 7f69deabb740 1 -- 172.21.15.116:0/367596853 shutdown_connections 2024-09-12T03:13:34.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.765+0000 7f69deabb740 1 -- 172.21.15.116:0/367596853 wait complete. 2024-09-12T03:13:34.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.765+0000 7f69deabb740 1 -- start start 2024-09-12T03:13:34.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.765+0000 7f69deabb740 1 Processor -- start 2024-09-12T03:13:34.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.766+0000 7f69deabb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5691eb060 0x55b5691fe6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:34.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.766+0000 7f69deabb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5691eb980 0x55b5691fb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:34.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.766+0000 7f69deabb740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5691f6240 0x55b5691fb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:34.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.766+0000 7f69deabb740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f69d41add20 con 0x55b5691f6240 2024-09-12T03:13:34.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.766+0000 7f69deabb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f69d41af190 con 0x55b5691eb980 2024-09-12T03:13:34.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.766+0000 7f69deabb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f69d41ae3f0 con 0x55b5691eb060 2024-09-12T03:13:34.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.766+0000 7f69deabb740 10 client.0 _ll_register_callbacks cb 0x7f69c400a560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:34.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.766+0000 7f69dac7f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5691f6240 0x55b5691fb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:34.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.766+0000 7f69dac7f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5691f6240 0x55b5691fb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48714/0 (socket says 172.21.15.116:48714) 2024-09-12T03:13:34.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.766+0000 7f69dac7f640 1 -- 172.21.15.116:0/4231553733 learned_addr learned my addr 172.21.15.116:0/4231553733 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:34.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.766+0000 7f69d9c7d640 1 --2- 172.21.15.116:0/4231553733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5691eb980 0x55b5691fb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:34.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.766+0000 7f69da47e640 1 --2- 172.21.15.116:0/4231553733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5691eb060 0x55b5691fe6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:34.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.766+0000 7f69dac7f640 1 -- 172.21.15.116:0/4231553733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5691eb060 msgr2=0x55b5691fe6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:34.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.766+0000 7f69dac7f640 1 --2- 172.21.15.116:0/4231553733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5691eb060 0x55b5691fe6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.766+0000 7f69dac7f640 1 -- 172.21.15.116:0/4231553733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5691eb980 msgr2=0x55b5691fb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:34.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.766+0000 7f69dac7f640 1 --2- 172.21.15.116:0/4231553733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5691eb980 0x55b5691fb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.766+0000 7f69dac7f640 1 -- 172.21.15.116:0/4231553733 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5691eded0 con 0x55b5691f6240 2024-09-12T03:13:34.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.766+0000 7f69da47e640 1 --2- 172.21.15.116:0/4231553733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5691eb060 0x55b5691fe6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:13:34.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.766+0000 7f69d9c7d640 1 --2- 172.21.15.116:0/4231553733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5691eb980 0x55b5691fb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:13:34.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.766+0000 7f69dac7f640 1 --2- 172.21.15.116:0/4231553733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5691f6240 0x55b5691fb720 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f69cc0159c0 tx=0x7f69cc025830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a544ac79ea40d9a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:34.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.766+0000 7f69bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:34.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.766+0000 7f69bdffb640 1 -- 172.21.15.116:0/4231553733 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69cc033060 con 0x55b5691f6240 2024-09-12T03:13:34.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.766+0000 7f69deabb740 1 -- 172.21.15.116:0/4231553733 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69cc00ce30 con 0x55b5691f6240 2024-09-12T03:13:34.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.766+0000 7f69bdffb640 1 -- 172.21.15.116:0/4231553733 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f69cc00cb60 con 0x55b5691f6240 2024-09-12T03:13:34.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.767+0000 7f69bdffb640 1 -- 172.21.15.116:0/4231553733 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f69cc02a050 con 0x55b5691f6240 2024-09-12T03:13:34.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.767+0000 7f69deabb740 10 client.18300 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:34.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.767+0000 7f69deabb740 1 -- 172.21.15.116:0/4231553733 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b569271650 con 0x55b5691f6240 2024-09-12T03:13:34.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.767+0000 7f69bdffb640 1 -- 172.21.15.116:0/4231553733 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69cc00ce30 con 0x55b5691f6240 2024-09-12T03:13:34.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.767+0000 7f69bdffb640 1 -- 172.21.15.116:0/4231553733 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f69cc02e660 con 0x55b5691f6240 2024-09-12T03:13:34.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.767+0000 7f69deabb740 10 client.18300 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:34.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.767+0000 7f69deabb740 10 client.18300 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:13:34.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.767+0000 7f69deabb740 20 client.18300 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:34.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.767+0000 7f69deabb740 1 --2- 172.21.15.116:0/4231553733 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b569271c40 0x55b569274060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:34.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.767+0000 7f69deabb740 4 client.18300 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "904ad12d", "release": true, "await": true}] 2024-09-12T03:13:34.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.767+0000 7f69deabb740 1 -- 172.21.15.116:0/4231553733 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "quiesce db", "set_id": "904ad12d", "release": true, "await": true}) -- 0x55b5691580a0 con 0x55b569271c40 2024-09-12T03:13:34.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.767+0000 7f69da47e640 1 --2- 172.21.15.116:0/4231553733 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b569271c40 0x55b569274060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:34.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.768+0000 7f69da47e640 1 --2- 172.21.15.116:0/4231553733 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b569271c40 0x55b569274060 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a26734d241df6e7a server_cookie=dd243c857775426a in_seq=0 out_seq=0 2024-09-12T03:13:34.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.768+0000 7f69bdffb640 10 client.18300 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:34.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.770+0000 7f69bdffb640 1 -- 172.21.15.116:0/4231553733 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b5691580a0 con 0x55b569271c40 2024-09-12T03:13:34.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.770+0000 7f69bdffb640 10 client.18300 handle_command_reply: tid=0 2024-09-12T03:13:34.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.770+0000 7f69deabb740 1 client.18300 shutdown 2024-09-12T03:13:34.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.770+0000 7f69deabb740 1 -- 172.21.15.116:0/4231553733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5691f6240 msgr2=0x55b5691fb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:34.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.770+0000 7f69deabb740 1 --2- 172.21.15.116:0/4231553733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5691f6240 0x55b5691fb720 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f69cc0159c0 tx=0x7f69cc025830 comp rx=0 tx=0).stop 2024-09-12T03:13:34.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.770+0000 7f69deabb740 1 -- 172.21.15.116:0/4231553733 shutdown_connections 2024-09-12T03:13:34.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.770+0000 7f69deabb740 1 --2- 172.21.15.116:0/4231553733 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b569271c40 0x55b569274060 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.770+0000 7f69deabb740 1 --2- 172.21.15.116:0/4231553733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5691f6240 0x55b5691fb720 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.770+0000 7f69bdffb640 0 client.18300 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:34.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.770+0000 7f69deabb740 1 --2- 172.21.15.116:0/4231553733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5691eb980 0x55b5691fb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.770+0000 7f69deabb740 1 --2- 172.21.15.116:0/4231553733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5691eb060 0x55b5691fe6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.770+0000 7f69deabb740 1 -- 172.21.15.116:0/4231553733 >> 172.21.15.116:0/4231553733 conn(0x55b5691e75f0 msgr2=0x55b5691f3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:34.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.770+0000 7f69deabb740 1 -- 172.21.15.116:0/4231553733 shutdown_connections 2024-09-12T03:13:34.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.770+0000 7f69deabb740 1 -- 172.21.15.116:0/4231553733 wait complete. 2024-09-12T03:13:34.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.771+0000 7f69deabb740 20 client.18300 trim_cache size 0 max 16384 2024-09-12T03:13:34.775 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:13:34.775 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 25, 2024-09-12T03:13:34.775 INFO:teuthology.orchestra.run.smithi116.stdout: "leader": 24494, 2024-09-12T03:13:34.775 INFO:teuthology.orchestra.run.smithi116.stdout: "set_version": 44, 2024-09-12T03:13:34.775 INFO:teuthology.orchestra.run.smithi116.stdout: "sets": { 2024-09-12T03:13:34.775 INFO:teuthology.orchestra.run.smithi116.stdout: "904ad12d": { 2024-09-12T03:13:34.775 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 44, 2024-09-12T03:13:34.775 INFO:teuthology.orchestra.run.smithi116.stdout: "age_ref": 0.0, 2024-09-12T03:13:34.775 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:13:34.775 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RELEASED", 2024-09-12T03:13:34.776 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:13:34.776 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:13:34.776 INFO:teuthology.orchestra.run.smithi116.stdout: "timeout": 90.0, 2024-09-12T03:13:34.776 INFO:teuthology.orchestra.run.smithi116.stdout: "expiration": 142.6, 2024-09-12T03:13:34.776 INFO:teuthology.orchestra.run.smithi116.stdout: "members": { 2024-09-12T03:13:34.776 INFO:teuthology.orchestra.run.smithi116.stdout: "file:/": { 2024-09-12T03:13:34.776 INFO:teuthology.orchestra.run.smithi116.stdout: "excluded": false, 2024-09-12T03:13:34.776 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:13:34.776 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RELEASED", 2024-09-12T03:13:34.776 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:13:34.776 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:13:34.776 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:13:34.776 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:13:34.777 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:13:34.777 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:13:34.777 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:13:34.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.771+0000 7f69dc6e2640 1 -- 172.21.15.116:0/4053813526 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f69c00a2b80 msgr2=0x7f69c00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:34.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.771+0000 7f69dc6e2640 1 --2- 172.21.15.116:0/4053813526 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f69c00a2b80 0x7f69c00a5040 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f69cc009fd0 tx=0x7f69cc008220 comp rx=0 tx=0).stop 2024-09-12T03:13:34.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.771+0000 7f69dc6e2640 1 -- 172.21.15.116:0/4053813526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69d4100ca0 msgr2=0x7f69d41017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:34.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.771+0000 7f69dc6e2640 1 --2- 172.21.15.116:0/4053813526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69d4100ca0 0x7f69d41017b0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f69c400c970 tx=0x7f69c400ce30 comp rx=0 tx=0).stop 2024-09-12T03:13:34.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.771+0000 7f69dc6e2640 1 -- 172.21.15.116:0/4053813526 shutdown_connections 2024-09-12T03:13:34.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.771+0000 7f69dc6e2640 1 --2- 172.21.15.116:0/4053813526 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f69c00a2b80 0x7f69c00a5040 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.771+0000 7f69dc6e2640 1 --2- 172.21.15.116:0/4053813526 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69d4106f50 0x7f69d4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.771+0000 7f69dc6e2640 1 --2- 172.21.15.116:0/4053813526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69d4100ca0 0x7f69d41017b0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.771+0000 7f69dc6e2640 1 --2- 172.21.15.116:0/4053813526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69d41002f0 0x7f69d4101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:34.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.771+0000 7f69dc6e2640 1 -- 172.21.15.116:0/4053813526 >> 172.21.15.116:0/4053813526 conn(0x7f69d4077210 msgr2=0x7f69d41057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:34.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.771+0000 7f69dc6e2640 1 -- 172.21.15.116:0/4053813526 shutdown_connections 2024-09-12T03:13:34.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:34.771+0000 7f69dc6e2640 1 -- 172.21.15.116:0/4053813526 wait complete. 2024-09-12T03:13:35.514 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:35 smithi116 ceph-mon[31410]: pgmap v656: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:13:35.543 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '904ad12d', total seconds elapsed: 26.0 2024-09-12T03:13:35.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:35 smithi183 ceph-mon[35535]: pgmap v656: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:13:35.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:35 smithi177 ceph-mon[35305]: pgmap v656: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:13:37.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:37 smithi183 ceph-mon[35535]: pgmap v657: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T03:13:37.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:37 smithi177 ceph-mon[35305]: pgmap v657: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T03:13:37.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:37 smithi116 ceph-mon[31410]: pgmap v657: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T03:13:39.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:39 smithi183 ceph-mon[35535]: pgmap v658: 97 pgs: 97 active+clean; 5.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2024-09-12T03:13:39.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:39 smithi177 ceph-mon[35305]: pgmap v658: 97 pgs: 97 active+clean; 5.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2024-09-12T03:13:39.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:39 smithi116 ceph-mon[31410]: pgmap v658: 97 pgs: 97 active+clean; 5.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2024-09-12T03:13:41.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:41 smithi183 ceph-mon[35535]: pgmap v659: 97 pgs: 97 active+clean; 5.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2024-09-12T03:13:41.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:41 smithi177 ceph-mon[35305]: pgmap v659: 97 pgs: 97 active+clean; 5.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2024-09-12T03:13:41.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:41 smithi116 ceph-mon[31410]: pgmap v659: 97 pgs: 97 active+clean; 5.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2024-09-12T03:13:42.836 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.836 INFO:tasks.workunit.client.0.smithi116.stdout:313/600: open fd = 3 2024-09-12T03:13:42.836 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.836 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.837 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.837 INFO:tasks.workunit.client.0.smithi116.stdout:314/600: open fd = 3 2024-09-12T03:13:42.837 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.837 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.837 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.837 INFO:tasks.workunit.client.0.smithi116.stdout:315/600: open fd = 3 2024-09-12T03:13:42.838 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.838 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.838 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.838 INFO:tasks.workunit.client.0.smithi116.stdout:316/600: open fd = 3 2024-09-12T03:13:42.838 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.838 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.839 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.839 INFO:tasks.workunit.client.0.smithi116.stdout:317/600: open fd = 3 2024-09-12T03:13:42.839 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.839 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.839 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.839 INFO:tasks.workunit.client.0.smithi116.stdout:318/600: open fd = 3 2024-09-12T03:13:42.840 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.840 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.840 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.840 INFO:tasks.workunit.client.0.smithi116.stdout:319/600: open fd = 3 2024-09-12T03:13:42.840 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.841 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.841 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.841 INFO:tasks.workunit.client.0.smithi116.stdout:320/600: open fd = 3 2024-09-12T03:13:42.841 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.841 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.841 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.842 INFO:tasks.workunit.client.0.smithi116.stdout:321/600: open fd = 3 2024-09-12T03:13:42.842 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.842 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.842 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.842 INFO:tasks.workunit.client.0.smithi116.stdout:322/600: open fd = 3 2024-09-12T03:13:42.843 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.843 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.843 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.843 INFO:tasks.workunit.client.0.smithi116.stdout:323/600: open fd = 3 2024-09-12T03:13:42.843 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.843 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.844 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.844 INFO:tasks.workunit.client.0.smithi116.stdout:324/600: open fd = 3 2024-09-12T03:13:42.844 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.844 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.844 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.845 INFO:tasks.workunit.client.0.smithi116.stdout:325/600: open fd = 3 2024-09-12T03:13:42.845 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.845 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.845 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.845 INFO:tasks.workunit.client.0.smithi116.stdout:326/600: open fd = 3 2024-09-12T03:13:42.845 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.846 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.846 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.846 INFO:tasks.workunit.client.0.smithi116.stdout:327/600: open fd = 3 2024-09-12T03:13:42.846 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.846 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.846 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.847 INFO:tasks.workunit.client.0.smithi116.stdout:328/600: open fd = 3 2024-09-12T03:13:42.847 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.847 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.847 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.847 INFO:tasks.workunit.client.0.smithi116.stdout:329/600: open fd = 3 2024-09-12T03:13:42.848 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.848 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.848 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.849 INFO:tasks.workunit.client.0.smithi116.stdout:330/600: open fd = 3 2024-09-12T03:13:42.849 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.849 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.849 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.849 INFO:tasks.workunit.client.0.smithi116.stdout:331/600: open fd = 3 2024-09-12T03:13:42.849 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.849 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.850 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.850 INFO:tasks.workunit.client.0.smithi116.stdout:332/600: open fd = 3 2024-09-12T03:13:42.850 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.850 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.850 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.850 INFO:tasks.workunit.client.0.smithi116.stdout:333/600: open fd = 3 2024-09-12T03:13:42.850 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.850 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.851 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.851 INFO:tasks.workunit.client.0.smithi116.stdout:334/600: open fd = 3 2024-09-12T03:13:42.851 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.851 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.851 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.851 INFO:tasks.workunit.client.0.smithi116.stdout:335/600: open fd = 3 2024-09-12T03:13:42.851 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.852 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.852 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.852 INFO:tasks.workunit.client.0.smithi116.stdout:336/600: open fd = 3 2024-09-12T03:13:42.852 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.852 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.853 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.853 INFO:tasks.workunit.client.0.smithi116.stdout:337/600: open fd = 3 2024-09-12T03:13:42.853 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.853 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.853 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.853 INFO:tasks.workunit.client.0.smithi116.stdout:338/600: open fd = 3 2024-09-12T03:13:42.853 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.853 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.854 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.854 INFO:tasks.workunit.client.0.smithi116.stdout:339/600: open fd = 3 2024-09-12T03:13:42.854 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.854 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.854 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.854 INFO:tasks.workunit.client.0.smithi116.stdout:340/600: open fd = 3 2024-09-12T03:13:42.854 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.854 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.855 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.855 INFO:tasks.workunit.client.0.smithi116.stdout:341/600: open fd = 3 2024-09-12T03:13:42.855 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.855 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.855 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.855 INFO:tasks.workunit.client.0.smithi116.stdout:342/600: open fd = 3 2024-09-12T03:13:42.855 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.855 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.855 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.856 INFO:tasks.workunit.client.0.smithi116.stdout:343/600: open fd = 3 2024-09-12T03:13:42.856 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.856 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.856 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.856 INFO:tasks.workunit.client.0.smithi116.stdout:344/600: open fd = 3 2024-09-12T03:13:42.856 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.856 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.856 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.856 INFO:tasks.workunit.client.0.smithi116.stdout:345/600: open fd = 3 2024-09-12T03:13:42.856 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.856 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.856 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.857 INFO:tasks.workunit.client.0.smithi116.stdout:346/600: open fd = 3 2024-09-12T03:13:42.857 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.857 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.857 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.857 INFO:tasks.workunit.client.0.smithi116.stdout:347/600: open fd = 3 2024-09-12T03:13:42.857 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.857 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.857 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.857 INFO:tasks.workunit.client.0.smithi116.stdout:348/600: open fd = 3 2024-09-12T03:13:42.857 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.857 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.857 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.857 INFO:tasks.workunit.client.0.smithi116.stdout:349/600: open fd = 3 2024-09-12T03:13:42.857 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.858 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.858 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.858 INFO:tasks.workunit.client.0.smithi116.stdout:350/600: open fd = 3 2024-09-12T03:13:42.858 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.858 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.858 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.858 INFO:tasks.workunit.client.0.smithi116.stdout:351/600: open fd = 3 2024-09-12T03:13:42.858 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.858 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.858 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.858 INFO:tasks.workunit.client.0.smithi116.stdout:352/600: open fd = 3 2024-09-12T03:13:42.858 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.858 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.859 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.859 INFO:tasks.workunit.client.0.smithi116.stdout:353/600: open fd = 3 2024-09-12T03:13:42.859 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.859 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.859 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.859 INFO:tasks.workunit.client.0.smithi116.stdout:354/600: open fd = 3 2024-09-12T03:13:42.859 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.859 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.859 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.859 INFO:tasks.workunit.client.0.smithi116.stdout:355/600: open fd = 3 2024-09-12T03:13:42.859 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.859 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.859 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.859 INFO:tasks.workunit.client.0.smithi116.stdout:356/600: open fd = 3 2024-09-12T03:13:42.860 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.860 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.860 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.860 INFO:tasks.workunit.client.0.smithi116.stdout:357/600: open fd = 3 2024-09-12T03:13:42.860 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.860 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.860 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.860 INFO:tasks.workunit.client.0.smithi116.stdout:358/600: open fd = 3 2024-09-12T03:13:42.860 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.860 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.860 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.860 INFO:tasks.workunit.client.0.smithi116.stdout:359/600: open fd = 3 2024-09-12T03:13:42.860 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.861 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.861 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.861 INFO:tasks.workunit.client.0.smithi116.stdout:360/600: open fd = 3 2024-09-12T03:13:42.861 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.861 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.861 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.861 INFO:tasks.workunit.client.0.smithi116.stdout:361/600: open fd = 3 2024-09-12T03:13:42.861 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.861 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.861 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.861 INFO:tasks.workunit.client.0.smithi116.stdout:362/600: open fd = 3 2024-09-12T03:13:42.861 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.861 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.861 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.862 INFO:tasks.workunit.client.0.smithi116.stdout:363/600: open fd = 3 2024-09-12T03:13:42.862 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.862 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.862 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.862 INFO:tasks.workunit.client.0.smithi116.stdout:364/600: open fd = 3 2024-09-12T03:13:42.862 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.862 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.862 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.862 INFO:tasks.workunit.client.0.smithi116.stdout:365/600: open fd = 3 2024-09-12T03:13:42.862 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.862 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.862 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.862 INFO:tasks.workunit.client.0.smithi116.stdout:366/600: open fd = 3 2024-09-12T03:13:42.862 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.863 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.863 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.863 INFO:tasks.workunit.client.0.smithi116.stdout:367/600: open fd = 3 2024-09-12T03:13:42.863 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.863 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.863 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.863 INFO:tasks.workunit.client.0.smithi116.stdout:368/600: open fd = 3 2024-09-12T03:13:42.863 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.863 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.863 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.863 INFO:tasks.workunit.client.0.smithi116.stdout:369/600: open fd = 3 2024-09-12T03:13:42.863 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.863 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.863 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.864 INFO:tasks.workunit.client.0.smithi116.stdout:370/600: open fd = 3 2024-09-12T03:13:42.864 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.864 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.864 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.864 INFO:tasks.workunit.client.0.smithi116.stdout:371/600: open fd = 3 2024-09-12T03:13:42.864 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.864 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.864 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.864 INFO:tasks.workunit.client.0.smithi116.stdout:372/600: open fd = 3 2024-09-12T03:13:42.864 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.864 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.864 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.864 INFO:tasks.workunit.client.0.smithi116.stdout:373/600: open fd = 3 2024-09-12T03:13:42.864 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.865 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.865 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:42.865 INFO:tasks.workunit.client.0.smithi116.stdout:374/600: open fd = 3 2024-09-12T03:13:42.865 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.865 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:13:42.865 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:13:43.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:43 smithi183 ceph-mon[35535]: pgmap v660: 97 pgs: 97 active+clean; 5.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.7 KiB/s wr, 7 op/s 2024-09-12T03:13:43.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:43 smithi177 ceph-mon[35305]: pgmap v660: 97 pgs: 97 active+clean; 5.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.7 KiB/s wr, 7 op/s 2024-09-12T03:13:43.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:43 smithi116 ceph-mon[31410]: pgmap v660: 97 pgs: 97 active+clean; 5.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.7 KiB/s wr, 7 op/s 2024-09-12T03:13:45.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:45 smithi183 ceph-mon[35535]: pgmap v661: 97 pgs: 97 active+clean; 5.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:13:45.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:45 smithi177 ceph-mon[35305]: pgmap v661: 97 pgs: 97 active+clean; 5.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:13:45.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:45 smithi116 ceph-mon[31410]: pgmap v661: 97 pgs: 97 active+clean; 5.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:13:47.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:47 smithi183 ceph-mon[35535]: pgmap v662: 97 pgs: 97 active+clean; 5.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:13:47.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:47 smithi177 ceph-mon[35305]: pgmap v662: 97 pgs: 97 active+clean; 5.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:13:47.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:47 smithi116 ceph-mon[31410]: pgmap v662: 97 pgs: 97 active+clean; 5.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:13:49.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:49 smithi183 ceph-mon[35535]: pgmap v663: 97 pgs: 97 active+clean; 5.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-12T03:13:49.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:49 smithi177 ceph-mon[35305]: pgmap v663: 97 pgs: 97 active+clean; 5.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-12T03:13:49.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:49 smithi116 ceph-mon[31410]: pgmap v663: 97 pgs: 97 active+clean; 5.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-12T03:13:51.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:51 smithi183 ceph-mon[35535]: pgmap v664: 97 pgs: 97 active+clean; 5.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T03:13:51.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:51 smithi177 ceph-mon[35305]: pgmap v664: 97 pgs: 97 active+clean; 5.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T03:13:51.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:51 smithi116 ceph-mon[31410]: pgmap v664: 97 pgs: 97 active+clean; 5.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T03:13:53.192 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:13:53.452 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:13:53.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:53 smithi116 ceph-mon[31410]: pgmap v665: 97 pgs: 97 active+clean; 5.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:13:53.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:53 smithi183 ceph-mon[35535]: pgmap v665: 97 pgs: 97 active+clean; 5.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:13:53.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:53 smithi177 ceph-mon[35305]: pgmap v665: 97 pgs: 97 active+clean; 5.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:13:53.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.956+0000 7fde15051640 1 -- 172.21.15.116:0/3328745202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde10102ef0 msgr2=0x7fde1010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:53.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.956+0000 7fde15051640 1 --2- 172.21.15.116:0/3328745202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde10102ef0 0x7fde1010da40 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fde08009ca0 tx=0x7fde0802a950 comp rx=0 tx=0).stop 2024-09-12T03:13:53.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.956+0000 7fde15051640 1 -- 172.21.15.116:0/3328745202 shutdown_connections 2024-09-12T03:13:53.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.956+0000 7fde15051640 1 --2- 172.21.15.116:0/3328745202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde10102ef0 0x7fde1010da40 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.956+0000 7fde15051640 1 --2- 172.21.15.116:0/3328745202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde10102570 0x7fde101029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.956+0000 7fde15051640 1 --2- 172.21.15.116:0/3328745202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde10100e20 0x7fde10101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.956+0000 7fde15051640 1 -- 172.21.15.116:0/3328745202 >> 172.21.15.116:0/3328745202 conn(0x7fde100fc7d0 msgr2=0x7fde100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:53.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.956+0000 7fde15051640 1 -- 172.21.15.116:0/3328745202 shutdown_connections 2024-09-12T03:13:53.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.956+0000 7fde15051640 1 -- 172.21.15.116:0/3328745202 wait complete. 2024-09-12T03:13:53.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.956+0000 7fde15051640 1 Processor -- start 2024-09-12T03:13:53.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.957+0000 7fde15051640 1 -- start start 2024-09-12T03:13:53.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.957+0000 7fde15051640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde10100e20 0x7fde101984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:53.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.957+0000 7fde15051640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde10102570 0x7fde10198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:53.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.957+0000 7fde15051640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde10102ef0 0x7fde1019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:53.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.957+0000 7fde15051640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fde10110f80 con 0x7fde10100e20 2024-09-12T03:13:53.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.957+0000 7fde15051640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fde10110e00 con 0x7fde10102570 2024-09-12T03:13:53.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.957+0000 7fde15051640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fde10111100 con 0x7fde10102ef0 2024-09-12T03:13:53.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.957+0000 7fde0ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde10100e20 0x7fde101984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:53.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.957+0000 7fde0ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde10100e20 0x7fde101984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38608/0 (socket says 172.21.15.116:38608) 2024-09-12T03:13:53.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.957+0000 7fde0ed9d640 1 -- 172.21.15.116:0/1230816750 learned_addr learned my addr 172.21.15.116:0/1230816750 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:53.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.957+0000 7fde0e59c640 1 --2- 172.21.15.116:0/1230816750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde10102570 0x7fde10198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:53.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.957+0000 7fde0f59e640 1 --2- 172.21.15.116:0/1230816750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde10102ef0 0x7fde1019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:53.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.957+0000 7fde0ed9d640 1 -- 172.21.15.116:0/1230816750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde10102ef0 msgr2=0x7fde1019cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:53.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.957+0000 7fde0ed9d640 1 --2- 172.21.15.116:0/1230816750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde10102ef0 0x7fde1019cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.957+0000 7fde0ed9d640 1 -- 172.21.15.116:0/1230816750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde10102570 msgr2=0x7fde10198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:53.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.957+0000 7fde0ed9d640 1 --2- 172.21.15.116:0/1230816750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde10102570 0x7fde10198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.957+0000 7fde0ed9d640 1 -- 172.21.15.116:0/1230816750 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde1019d480 con 0x7fde10100e20 2024-09-12T03:13:53.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.958+0000 7fde0ed9d640 1 --2- 172.21.15.116:0/1230816750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde10100e20 0x7fde101984d0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fde0000c920 tx=0x7fde0000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:53.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.958+0000 7fde07fff640 1 -- 172.21.15.116:0/1230816750 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde00007c70 con 0x7fde10100e20 2024-09-12T03:13:53.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.958+0000 7fde07fff640 1 -- 172.21.15.116:0/1230816750 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde00007e10 con 0x7fde10100e20 2024-09-12T03:13:53.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.958+0000 7fde15051640 1 -- 172.21.15.116:0/1230816750 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde1019d770 con 0x7fde10100e20 2024-09-12T03:13:53.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.958+0000 7fde07fff640 1 -- 172.21.15.116:0/1230816750 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde00018890 con 0x7fde10100e20 2024-09-12T03:13:53.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.958+0000 7fde15051640 1 -- 172.21.15.116:0/1230816750 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde101a5050 con 0x7fde10100e20 2024-09-12T03:13:53.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.960+0000 7fde07fff640 1 -- 172.21.15.116:0/1230816750 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fde00018a70 con 0x7fde10100e20 2024-09-12T03:13:53.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.960+0000 7fde1742a740 1 Processor -- start 2024-09-12T03:13:53.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.960+0000 7fde1742a740 1 -- start start 2024-09-12T03:13:53.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.961+0000 7fde1742a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f185af74a0 0x55f185af7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:53.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.961+0000 7fde1742a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f185af7dc0 0x55f185b02140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:53.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.961+0000 7fde1742a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f185b02680 0x55f185b04a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:53.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.961+0000 7fde1742a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55f185a5d2e0 con 0x55f185af7dc0 2024-09-12T03:13:53.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.961+0000 7fde1742a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f185a04ce0 con 0x55f185af74a0 2024-09-12T03:13:53.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.961+0000 7fde1742a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f185a592a0 con 0x55f185b02680 2024-09-12T03:13:53.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.961+0000 7fde0ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f185af7dc0 0x55f185b02140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:53.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.961+0000 7fde0e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f185b02680 0x55f185b04a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:53.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.961+0000 7fde0ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f185af7dc0 0x55f185b02140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38616/0 (socket says 172.21.15.116:38616) 2024-09-12T03:13:53.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.961+0000 7fde0ed9d640 1 -- 172.21.15.116:0/4073497184 learned_addr learned my addr 172.21.15.116:0/4073497184 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:53.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.961+0000 7fde07fff640 1 --2- 172.21.15.116:0/1230816750 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fddfc0a2b80 0x7fddfc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:53.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.961+0000 7fde0f59e640 1 --2- 172.21.15.116:0/4073497184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f185af74a0 0x55f185af7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:53.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.961+0000 7fde0e59c640 1 -- 172.21.15.116:0/4073497184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f185af74a0 msgr2=0x55f185af7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:53.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.961+0000 7fde0e59c640 1 --2- 172.21.15.116:0/4073497184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f185af74a0 0x55f185af7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.961+0000 7fde0e59c640 1 -- 172.21.15.116:0/4073497184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f185af7dc0 msgr2=0x55f185b02140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:53.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.961+0000 7fde0e59c640 1 --2- 172.21.15.116:0/4073497184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f185af7dc0 0x55f185b02140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.961+0000 7fde0e59c640 1 -- 172.21.15.116:0/4073497184 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1859e9440 con 0x55f185b02680 2024-09-12T03:13:53.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.961+0000 7fde0f59e640 1 --2- 172.21.15.116:0/4073497184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f185af74a0 0x55f185af7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:13:53.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.961+0000 7fde0ed9d640 1 --2- 172.21.15.116:0/4073497184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f185af7dc0 0x55f185b02140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:13:53.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.961+0000 7fde0f59e640 1 --2- 172.21.15.116:0/1230816750 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fddfc0a2b80 0x7fddfc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:53.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.962+0000 7fde07fff640 1 -- 172.21.15.116:0/1230816750 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fde000cf110 con 0x7fde10100e20 2024-09-12T03:13:53.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.962+0000 7fde0e59c640 1 --2- 172.21.15.116:0/4073497184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f185b02680 0x55f185b04a70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009bd0 tx=0x7fddf8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8dc122fccef40cf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:53.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.962+0000 7fde057fa640 1 -- 172.21.15.116:0/4073497184 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf8004050 con 0x55f185b02680 2024-09-12T03:13:53.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.962+0000 7fde0f59e640 1 --2- 172.21.15.116:0/1230816750 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fddfc0a2b80 0x7fddfc0a5040 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fde08005e30 tx=0x7fde0803c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:53.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.962+0000 7fde057fa640 1 -- 172.21.15.116:0/4073497184 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddf8005500 con 0x55f185b02680 2024-09-12T03:13:53.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.962+0000 7fde057fa640 1 -- 172.21.15.116:0/4073497184 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf8004d90 con 0x55f185b02680 2024-09-12T03:13:53.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.963+0000 7fde1742a740 1 -- 172.21.15.116:0/4073497184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f185b02680 msgr2=0x55f185b04a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:53.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.963+0000 7fde1742a740 1 --2- 172.21.15.116:0/4073497184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f185b02680 0x55f185b04a70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009bd0 tx=0x7fddf8011810 comp rx=0 tx=0).stop 2024-09-12T03:13:53.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.963+0000 7fde1742a740 1 -- 172.21.15.116:0/4073497184 shutdown_connections 2024-09-12T03:13:53.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.963+0000 7fde1742a740 1 --2- 172.21.15.116:0/4073497184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f185b02680 0x55f185b04a70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.963+0000 7fde1742a740 1 --2- 172.21.15.116:0/4073497184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f185af7dc0 0x55f185b02140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.963+0000 7fde1742a740 1 --2- 172.21.15.116:0/4073497184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f185af74a0 0x55f185af7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.963+0000 7fde1742a740 1 -- 172.21.15.116:0/4073497184 >> 172.21.15.116:0/4073497184 conn(0x55f185af3a30 msgr2=0x55f185af3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:53.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.963+0000 7fde1742a740 1 -- 172.21.15.116:0/4073497184 shutdown_connections 2024-09-12T03:13:53.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.963+0000 7fde1742a740 1 -- 172.21.15.116:0/4073497184 wait complete. 2024-09-12T03:13:53.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.963+0000 7fde1742a740 1 -- start start 2024-09-12T03:13:53.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.963+0000 7fde1742a740 1 Processor -- start 2024-09-12T03:13:53.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.963+0000 7fde1742a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f185af74a0 0x55f185b01ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:53.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.963+0000 7fde1742a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f185af7dc0 0x55f185aff1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:53.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.963+0000 7fde1742a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f185b02680 0x55f185aff6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:53.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.963+0000 7fde1742a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55f185b05250 con 0x55f185af74a0 2024-09-12T03:13:53.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.963+0000 7fde1742a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f185a18050 con 0x55f185b02680 2024-09-12T03:13:53.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.964+0000 7fde1742a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f185b053d0 con 0x55f185af7dc0 2024-09-12T03:13:53.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.964+0000 7fde1742a740 10 client.0 _ll_register_callbacks cb 0x55f185a58160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:53.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.964+0000 7fde0ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f185af74a0 0x55f185b01ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:53.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.964+0000 7fde0ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f185af74a0 0x55f185b01ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38618/0 (socket says 172.21.15.116:38618) 2024-09-12T03:13:53.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.964+0000 7fde0ed9d640 1 -- 172.21.15.116:0/1809900771 learned_addr learned my addr 172.21.15.116:0/1809900771 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:53.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.964+0000 7fde0f59e640 1 --2- 172.21.15.116:0/1809900771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f185b02680 0x55f185aff6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:53.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.964+0000 7fde0e59c640 1 --2- 172.21.15.116:0/1809900771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f185af7dc0 0x55f185aff1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:53.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.964+0000 7fde0ed9d640 1 -- 172.21.15.116:0/1809900771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f185af7dc0 msgr2=0x55f185aff1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:53.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.964+0000 7fde0ed9d640 1 --2- 172.21.15.116:0/1809900771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f185af7dc0 0x55f185aff1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.964+0000 7fde0ed9d640 1 -- 172.21.15.116:0/1809900771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f185b02680 msgr2=0x55f185aff6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:53.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.964+0000 7fde0ed9d640 1 --2- 172.21.15.116:0/1809900771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f185b02680 0x55f185aff6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.964+0000 7fde0ed9d640 1 -- 172.21.15.116:0/1809900771 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1857ce3c0 con 0x55f185af74a0 2024-09-12T03:13:53.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.964+0000 7fde0e59c640 1 --2- 172.21.15.116:0/1809900771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f185af7dc0 0x55f185aff1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:13:53.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.964+0000 7fde0ed9d640 1 --2- 172.21.15.116:0/1809900771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f185af74a0 0x55f185b01ef0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fde000a4080 tx=0x7fde000ccba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9d374d78c50ab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:53.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.964+0000 7fddf67fc640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:53.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.964+0000 7fddf67fc640 1 -- 172.21.15.116:0/1809900771 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0009b2a0 con 0x55f185af74a0 2024-09-12T03:13:53.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.964+0000 7fddf67fc640 1 -- 172.21.15.116:0/1809900771 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fde0000fb30 con 0x55f185af74a0 2024-09-12T03:13:53.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.964+0000 7fde1742a740 1 -- 172.21.15.116:0/1809900771 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddf8005500 con 0x55f185af74a0 2024-09-12T03:13:53.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.964+0000 7fddf67fc640 1 -- 172.21.15.116:0/1809900771 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fde000afe60 con 0x55f185af74a0 2024-09-12T03:13:53.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.964+0000 7fde1742a740 10 client.18312 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:53.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.964+0000 7fde1742a740 1 -- 172.21.15.116:0/1809900771 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f185b10b60 con 0x55f185af74a0 2024-09-12T03:13:53.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.965+0000 7fddf67fc640 1 -- 172.21.15.116:0/1809900771 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddf8005500 con 0x55f185af74a0 2024-09-12T03:13:53.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.965+0000 7fddf67fc640 1 -- 172.21.15.116:0/1809900771 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fde00097fb0 con 0x55f185af74a0 2024-09-12T03:13:53.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.965+0000 7fde1742a740 10 client.18312 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:53.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.965+0000 7fde1742a740 10 client.18312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:13:53.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.965+0000 7fde1742a740 20 client.18312 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:53.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.965+0000 7fde1742a740 1 --2- 172.21.15.116:0/1809900771 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f185b11f70 0x55f185b143e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:53.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.965+0000 7fde1742a740 4 client.18312 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:13:53.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.965+0000 7fde1742a740 1 -- 172.21.15.116:0/1809900771 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f185a6a850 con 0x55f185b11f70 2024-09-12T03:13:53.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.965+0000 7fde0e59c640 1 --2- 172.21.15.116:0/1809900771 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f185b11f70 0x55f185b143e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:53.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.966+0000 7fde0e59c640 1 --2- 172.21.15.116:0/1809900771 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f185b11f70 0x55f185b143e0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6864807f9b0b952 server_cookie=2a7e613c744abfe0 in_seq=0 out_seq=0 2024-09-12T03:13:53.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.966+0000 7fddf67fc640 10 client.18312 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:53.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.966+0000 7fddf67fc640 1 -- 172.21.15.116:0/1809900771 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f185a6a850 con 0x55f185b11f70 2024-09-12T03:13:53.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.966+0000 7fddf67fc640 10 client.18312 handle_command_reply: tid=0 2024-09-12T03:13:53.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.966+0000 7fde1742a740 1 client.18312 shutdown 2024-09-12T03:13:53.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.966+0000 7fde1742a740 1 -- 172.21.15.116:0/1809900771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f185af74a0 msgr2=0x55f185b01ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:53.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.966+0000 7fde1742a740 1 --2- 172.21.15.116:0/1809900771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f185af74a0 0x55f185b01ef0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fde000a4080 tx=0x7fde000ccba0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.966+0000 7fde1742a740 1 -- 172.21.15.116:0/1809900771 shutdown_connections 2024-09-12T03:13:53.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.966+0000 7fde1742a740 1 --2- 172.21.15.116:0/1809900771 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f185b11f70 0x55f185b143e0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.966+0000 7fde1742a740 1 --2- 172.21.15.116:0/1809900771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f185b02680 0x55f185aff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.966+0000 7fde1742a740 1 --2- 172.21.15.116:0/1809900771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f185af7dc0 0x55f185aff1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.966+0000 7fde1742a740 1 --2- 172.21.15.116:0/1809900771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f185af74a0 0x55f185b01ef0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.966+0000 7fde1742a740 1 -- 172.21.15.116:0/1809900771 >> 172.21.15.116:0/1809900771 conn(0x55f185af3a30 msgr2=0x55f185af9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:53.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.966+0000 7fddf67fc640 0 client.18312 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:53.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.966+0000 7fde1742a740 1 -- 172.21.15.116:0/1809900771 shutdown_connections 2024-09-12T03:13:53.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.966+0000 7fde1742a740 1 -- 172.21.15.116:0/1809900771 wait complete. 2024-09-12T03:13:53.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.967+0000 7fde1742a740 20 client.18312 trim_cache size 0 max 16384 2024-09-12T03:13:53.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.984+0000 7fde1742a740 1 Processor -- start 2024-09-12T03:13:53.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.984+0000 7fde1742a740 1 -- start start 2024-09-12T03:13:53.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.984+0000 7fde1742a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f185af74a0 0x55f185aff390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:53.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.984+0000 7fde1742a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f185af7dc0 0x55f185af2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:53.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.984+0000 7fde1742a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f185b02680 0x55f185af2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:53.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.984+0000 7fde1742a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fde101afca0 con 0x55f185af74a0 2024-09-12T03:13:53.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.984+0000 7fde1742a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fde101ad570 con 0x55f185af7dc0 2024-09-12T03:13:53.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.984+0000 7fde1742a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fde101ab3c0 con 0x55f185b02680 2024-09-12T03:13:53.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.984+0000 7fde0ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f185af74a0 0x55f185aff390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:53.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.984+0000 7fde0ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f185af74a0 0x55f185aff390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38634/0 (socket says 172.21.15.116:38634) 2024-09-12T03:13:53.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.984+0000 7fde0ed9d640 1 -- 172.21.15.116:0/3509030243 learned_addr learned my addr 172.21.15.116:0/3509030243 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:53.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.984+0000 7fde0f59e640 1 --2- 172.21.15.116:0/3509030243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f185b02680 0x55f185af2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:53.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.984+0000 7fde0e59c640 1 --2- 172.21.15.116:0/3509030243 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f185af7dc0 0x55f185af2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:53.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.984+0000 7fde0ed9d640 1 -- 172.21.15.116:0/3509030243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f185b02680 msgr2=0x55f185af2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:53.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.984+0000 7fde0ed9d640 1 --2- 172.21.15.116:0/3509030243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f185b02680 0x55f185af2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.984+0000 7fde0ed9d640 1 -- 172.21.15.116:0/3509030243 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f185af7dc0 msgr2=0x55f185af2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:53.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.984+0000 7fde0ed9d640 1 --2- 172.21.15.116:0/3509030243 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f185af7dc0 0x55f185af2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.984+0000 7fde0ed9d640 1 -- 172.21.15.116:0/3509030243 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f185afe150 con 0x55f185af74a0 2024-09-12T03:13:53.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.984+0000 7fde0f59e640 1 --2- 172.21.15.116:0/3509030243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f185b02680 0x55f185af2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:13:53.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.985+0000 7fde0ed9d640 1 --2- 172.21.15.116:0/3509030243 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f185af74a0 0x55f185aff390 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fde0000a4a0 tx=0x7fde0000ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8a4e071beb844b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:53.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.985+0000 7fde04ff9640 1 -- 172.21.15.116:0/3509030243 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde000b9070 con 0x55f185af74a0 2024-09-12T03:13:53.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.985+0000 7fde04ff9640 1 -- 172.21.15.116:0/3509030243 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde0009c040 con 0x55f185af74a0 2024-09-12T03:13:53.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.985+0000 7fde04ff9640 1 -- 172.21.15.116:0/3509030243 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde00097fb0 con 0x55f185af74a0 2024-09-12T03:13:53.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.985+0000 7fde1742a740 1 -- 172.21.15.116:0/3509030243 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f185af74a0 msgr2=0x55f185aff390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:53.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.985+0000 7fde1742a740 1 --2- 172.21.15.116:0/3509030243 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f185af74a0 0x55f185aff390 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fde0000a4a0 tx=0x7fde0000ff30 comp rx=0 tx=0).stop 2024-09-12T03:13:53.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.986+0000 7fde1742a740 1 -- 172.21.15.116:0/3509030243 shutdown_connections 2024-09-12T03:13:53.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.986+0000 7fde1742a740 1 --2- 172.21.15.116:0/3509030243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f185b02680 0x55f185af2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.986+0000 7fde1742a740 1 --2- 172.21.15.116:0/3509030243 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f185af7dc0 0x55f185af2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.986+0000 7fde1742a740 1 --2- 172.21.15.116:0/3509030243 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f185af74a0 0x55f185aff390 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.986+0000 7fde1742a740 1 -- 172.21.15.116:0/3509030243 >> 172.21.15.116:0/3509030243 conn(0x55f185af3a30 msgr2=0x55f185af2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:53.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.986+0000 7fde1742a740 1 -- 172.21.15.116:0/3509030243 shutdown_connections 2024-09-12T03:13:53.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.986+0000 7fde1742a740 1 -- 172.21.15.116:0/3509030243 wait complete. 2024-09-12T03:13:53.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.986+0000 7fde1742a740 1 -- start start 2024-09-12T03:13:53.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.986+0000 7fde1742a740 1 Processor -- start 2024-09-12T03:13:53.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.986+0000 7fde1742a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f185af74a0 0x55f185b0ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:53.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.986+0000 7fde1742a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f185af7dc0 0x55f185b06040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:53.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.986+0000 7fde1742a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f185b02680 0x55f185b06580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:53.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.986+0000 7fde1742a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fde101aba30 con 0x55f185af7dc0 2024-09-12T03:13:53.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.986+0000 7fde1742a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fde101acea0 con 0x55f185b02680 2024-09-12T03:13:53.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.986+0000 7fde1742a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fde101ac100 con 0x55f185af74a0 2024-09-12T03:13:53.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.986+0000 7fde1742a740 10 client.0 _ll_register_callbacks cb 0x7fde0000c870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:53.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.986+0000 7fde0e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f185af7dc0 0x55f185b06040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:53.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.986+0000 7fde0e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f185af7dc0 0x55f185b06040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38644/0 (socket says 172.21.15.116:38644) 2024-09-12T03:13:53.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.986+0000 7fde0e59c640 1 -- 172.21.15.116:0/2334990707 learned_addr learned my addr 172.21.15.116:0/2334990707 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:53.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.987+0000 7fde0f59e640 1 --2- 172.21.15.116:0/2334990707 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f185b02680 0x55f185b06580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:53.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.987+0000 7fde0e59c640 1 -- 172.21.15.116:0/2334990707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f185af74a0 msgr2=0x55f185b0ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:53.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.987+0000 7fde0e59c640 1 --2- 172.21.15.116:0/2334990707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f185af74a0 0x55f185b0ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.987+0000 7fde0e59c640 1 -- 172.21.15.116:0/2334990707 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f185b02680 msgr2=0x55f185b06580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:53.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.987+0000 7fde0e59c640 1 --2- 172.21.15.116:0/2334990707 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f185b02680 0x55f185b06580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.987+0000 7fde0e59c640 1 -- 172.21.15.116:0/2334990707 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f185b06d60 con 0x55f185af7dc0 2024-09-12T03:13:53.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.987+0000 7fde0f59e640 1 --2- 172.21.15.116:0/2334990707 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f185b02680 0x55f185b06580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:13:53.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.987+0000 7fde0e59c640 1 --2- 172.21.15.116:0/2334990707 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f185af7dc0 0x55f185b06040 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fddf801b040 tx=0x7fddf80044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3632cec92a819da4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:53.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.987+0000 7fddf67fc640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:53.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.987+0000 7fddf67fc640 1 -- 172.21.15.116:0/2334990707 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf8030020 con 0x55f185af7dc0 2024-09-12T03:13:53.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.987+0000 7fddf67fc640 1 -- 172.21.15.116:0/2334990707 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fddf8003a00 con 0x55f185af7dc0 2024-09-12T03:13:53.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.987+0000 7fde1742a740 1 -- 172.21.15.116:0/2334990707 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde0009c040 con 0x55f185af7dc0 2024-09-12T03:13:53.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.987+0000 7fddf67fc640 1 -- 172.21.15.116:0/2334990707 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fddf8005a70 con 0x55f185af7dc0 2024-09-12T03:13:53.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.987+0000 7fde1742a740 10 client.18318 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:53.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.987+0000 7fde1742a740 1 -- 172.21.15.116:0/2334990707 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f185b7cc40 con 0x55f185af7dc0 2024-09-12T03:13:53.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.987+0000 7fddf67fc640 1 -- 172.21.15.116:0/2334990707 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde0009c040 con 0x55f185af7dc0 2024-09-12T03:13:53.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.988+0000 7fddf67fc640 1 -- 172.21.15.116:0/2334990707 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fde000d2ce0 con 0x55f185af7dc0 2024-09-12T03:13:53.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.988+0000 7fde1742a740 10 client.18318 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:53.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.988+0000 7fde1742a740 10 client.18318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:13:53.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.988+0000 7fde1742a740 20 client.18318 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:53.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.988+0000 7fde1742a740 1 --2- 172.21.15.116:0/2334990707 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f185b7d230 0x55f185b7f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:53.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.988+0000 7fde1742a740 4 client.18318 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:13:53.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.988+0000 7fde1742a740 1 -- 172.21.15.116:0/2334990707 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f185affa70 con 0x55f185b7d230 2024-09-12T03:13:53.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.988+0000 7fde0ed9d640 1 --2- 172.21.15.116:0/2334990707 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f185b7d230 0x55f185b7f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:53.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.988+0000 7fddf67fc640 10 client.18318 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:53.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.988+0000 7fde0ed9d640 1 --2- 172.21.15.116:0/2334990707 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f185b7d230 0x55f185b7f650 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=190c054be42fc750 server_cookie=2188021fca67614c in_seq=0 out_seq=0 2024-09-12T03:13:53.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.989+0000 7fddf67fc640 1 -- 172.21.15.116:0/2334990707 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f185affa70 con 0x55f185b7d230 2024-09-12T03:13:53.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.989+0000 7fddf67fc640 10 client.18318 handle_command_reply: tid=0 2024-09-12T03:13:53.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.989+0000 7fde1742a740 1 client.18318 shutdown 2024-09-12T03:13:53.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.989+0000 7fde1742a740 1 -- 172.21.15.116:0/2334990707 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f185af7dc0 msgr2=0x55f185b06040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:53.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.989+0000 7fde1742a740 1 --2- 172.21.15.116:0/2334990707 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f185af7dc0 0x55f185b06040 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fddf801b040 tx=0x7fddf80044f0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.989+0000 7fde1742a740 1 -- 172.21.15.116:0/2334990707 shutdown_connections 2024-09-12T03:13:53.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.989+0000 7fde1742a740 1 --2- 172.21.15.116:0/2334990707 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f185b7d230 0x55f185b7f650 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.989+0000 7fde1742a740 1 --2- 172.21.15.116:0/2334990707 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f185b02680 0x55f185b06580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.989+0000 7fde1742a740 1 --2- 172.21.15.116:0/2334990707 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f185af7dc0 0x55f185b06040 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.989+0000 7fddf67fc640 0 client.18318 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:13:53.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.989+0000 7fde1742a740 1 --2- 172.21.15.116:0/2334990707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f185af74a0 0x55f185b0ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.989+0000 7fde1742a740 1 -- 172.21.15.116:0/2334990707 >> 172.21.15.116:0/2334990707 conn(0x55f185af3a30 msgr2=0x55f185af2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:53.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.989+0000 7fde1742a740 1 -- 172.21.15.116:0/2334990707 shutdown_connections 2024-09-12T03:13:53.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.989+0000 7fde1742a740 1 -- 172.21.15.116:0/2334990707 wait complete. 2024-09-12T03:13:53.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.989+0000 7fde1742a740 20 client.18318 trim_cache size 0 max 16384 2024-09-12T03:13:53.995 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:13:53.995 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:13:53.995 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:13:53.996 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:13:53.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.989+0000 7fde15051640 1 -- 172.21.15.116:0/1230816750 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fddfc0a2b80 msgr2=0x7fddfc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:53.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.989+0000 7fde15051640 1 --2- 172.21.15.116:0/1230816750 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fddfc0a2b80 0x7fddfc0a5040 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fde08005e30 tx=0x7fde0803c040 comp rx=0 tx=0).stop 2024-09-12T03:13:53.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.989+0000 7fde15051640 1 -- 172.21.15.116:0/1230816750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde10100e20 msgr2=0x7fde101984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:53.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.989+0000 7fde15051640 1 --2- 172.21.15.116:0/1230816750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde10100e20 0x7fde101984d0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fde0000c920 tx=0x7fde0000cde0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.990+0000 7fde15051640 1 -- 172.21.15.116:0/1230816750 shutdown_connections 2024-09-12T03:13:53.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.990+0000 7fde15051640 1 --2- 172.21.15.116:0/1230816750 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fddfc0a2b80 0x7fddfc0a5040 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.990+0000 7fde15051640 1 --2- 172.21.15.116:0/1230816750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde10102ef0 0x7fde1019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.990+0000 7fde15051640 1 --2- 172.21.15.116:0/1230816750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde10102570 0x7fde10198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.990+0000 7fde15051640 1 --2- 172.21.15.116:0/1230816750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde10100e20 0x7fde101984d0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:53.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.990+0000 7fde15051640 1 -- 172.21.15.116:0/1230816750 >> 172.21.15.116:0/1230816750 conn(0x7fde100fc7d0 msgr2=0x7fde1006a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:53.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.990+0000 7fde15051640 1 -- 172.21.15.116:0/1230816750 shutdown_connections 2024-09-12T03:13:53.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:53.990+0000 7fde15051640 1 -- 172.21.15.116:0/1230816750 wait complete. 2024-09-12T03:13:54.759 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:13:54.759 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:13:55.005 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:13:55.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.513+0000 7fcc3e994640 1 -- 172.21.15.116:0/1800933018 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc38100a80 msgr2=0x7fcc38100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:55.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.513+0000 7fcc3e994640 1 --2- 172.21.15.116:0/1800933018 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc38100a80 0x7fcc38100ee0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009930 tx=0x7fcc2002a420 comp rx=0 tx=0).stop 2024-09-12T03:13:55.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.513+0000 7fcc3e994640 1 -- 172.21.15.116:0/1800933018 shutdown_connections 2024-09-12T03:13:55.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.513+0000 7fcc3e994640 1 --2- 172.21.15.116:0/1800933018 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc38101420 0x7fcc3810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:55.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.513+0000 7fcc3e994640 1 --2- 172.21.15.116:0/1800933018 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc38100a80 0x7fcc38100ee0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:55.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.513+0000 7fcc3e994640 1 --2- 172.21.15.116:0/1800933018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc38106a80 0x7fcc38106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:55.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.513+0000 7fcc3e994640 1 -- 172.21.15.116:0/1800933018 >> 172.21.15.116:0/1800933018 conn(0x7fcc380fc7a0 msgr2=0x7fcc380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:55.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.513+0000 7fcc3e994640 1 -- 172.21.15.116:0/1800933018 shutdown_connections 2024-09-12T03:13:55.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.513+0000 7fcc3e994640 1 -- 172.21.15.116:0/1800933018 wait complete. 2024-09-12T03:13:55.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.514+0000 7fcc3e994640 1 Processor -- start 2024-09-12T03:13:55.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.514+0000 7fcc3e994640 1 -- start start 2024-09-12T03:13:55.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.514+0000 7fcc3e994640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc38100a80 0x7fcc380fce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:55.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.514+0000 7fcc3e994640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc38101420 0x7fcc380fd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:55.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.514+0000 7fcc3e994640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc38106a80 0x7fcc380fd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:55.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.514+0000 7fcc3e994640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcc38110f20 con 0x7fcc38101420 2024-09-12T03:13:55.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.514+0000 7fcc3e994640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcc38110da0 con 0x7fcc38100a80 2024-09-12T03:13:55.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.514+0000 7fcc3e994640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcc381110a0 con 0x7fcc38106a80 2024-09-12T03:13:55.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.514+0000 7fcc2ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc38101420 0x7fcc380fd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:55.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.514+0000 7fcc2ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc38101420 0x7fcc380fd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38676/0 (socket says 172.21.15.116:38676) 2024-09-12T03:13:55.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.514+0000 7fcc2ffff640 1 -- 172.21.15.116:0/108979660 learned_addr learned my addr 172.21.15.116:0/108979660 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:55.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.514+0000 7fcc37fff640 1 --2- 172.21.15.116:0/108979660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc38100a80 0x7fcc380fce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:55.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.514+0000 7fcc3cf31640 1 --2- 172.21.15.116:0/108979660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc38106a80 0x7fcc380fd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:55.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.514+0000 7fcc2ffff640 1 -- 172.21.15.116:0/108979660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc38106a80 msgr2=0x7fcc380fd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:55.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.514+0000 7fcc2ffff640 1 --2- 172.21.15.116:0/108979660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc38106a80 0x7fcc380fd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:55.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.514+0000 7fcc2ffff640 1 -- 172.21.15.116:0/108979660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc38100a80 msgr2=0x7fcc380fce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:55.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.514+0000 7fcc2ffff640 1 --2- 172.21.15.116:0/108979660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc38100a80 0x7fcc380fce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:55.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.514+0000 7fcc2ffff640 1 -- 172.21.15.116:0/108979660 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc381a6a70 con 0x7fcc38101420 2024-09-12T03:13:55.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.514+0000 7fcc2ffff640 1 --2- 172.21.15.116:0/108979660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc38101420 0x7fcc380fd340 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009a50 tx=0x7fcc20004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:55.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.515+0000 7fcc35ffb640 1 -- 172.21.15.116:0/108979660 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2003f030 con 0x7fcc38101420 2024-09-12T03:13:55.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.515+0000 7fcc3e994640 1 -- 172.21.15.116:0/108979660 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc381a6d00 con 0x7fcc38101420 2024-09-12T03:13:55.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.515+0000 7fcc35ffb640 1 -- 172.21.15.116:0/108979660 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc20003ce0 con 0x7fcc38101420 2024-09-12T03:13:55.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.515+0000 7fcc35ffb640 1 -- 172.21.15.116:0/108979660 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc20035070 con 0x7fcc38101420 2024-09-12T03:13:55.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.515+0000 7fcc3e994640 1 -- 172.21.15.116:0/108979660 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc381a7190 con 0x7fcc38101420 2024-09-12T03:13:55.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.516+0000 7fcc3e994640 1 -- 172.21.15.116:0/108979660 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc38100f60 con 0x7fcc38101420 2024-09-12T03:13:55.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.517+0000 7fcc35ffb640 1 -- 172.21.15.116:0/108979660 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fcc20042030 con 0x7fcc38101420 2024-09-12T03:13:55.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.518+0000 7fcc35ffb640 1 --2- 172.21.15.116:0/108979660 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcc240a2b80 0x7fcc240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:55.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.518+0000 7fcc37fff640 1 --2- 172.21.15.116:0/108979660 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcc240a2b80 0x7fcc240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:55.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.518+0000 7fcc35ffb640 1 -- 172.21.15.116:0/108979660 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fcc200ef640 con 0x7fcc38101420 2024-09-12T03:13:55.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.518+0000 7fcc37fff640 1 --2- 172.21.15.116:0/108979660 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcc240a2b80 0x7fcc240a5040 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcc28009890 tx=0x7fcc28008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:55.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.522+0000 7fcc35ffb640 1 -- 172.21.15.116:0/108979660 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcc200bc5d0 con 0x7fcc38101420 2024-09-12T03:13:55.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:55 smithi183 ceph-mon[35535]: pgmap v666: 97 pgs: 97 active+clean; 5.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:13:55.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.722+0000 7fcc3e994640 1 -- 172.21.15.116:0/108979660 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc3810a020 con 0x7fcc38101420 2024-09-12T03:13:55.722 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:55 smithi116 ceph-mon[31410]: pgmap v666: 97 pgs: 97 active+clean; 5.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:13:55.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.723+0000 7fcc35ffb640 1 -- 172.21.15.116:0/108979660 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7fcc200bbed0 con 0x7fcc38101420 2024-09-12T03:13:55.723 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:13:55.723 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:13:55.723 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:13:55.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.724+0000 7fcc3e994640 1 -- 172.21.15.116:0/108979660 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcc240a2b80 msgr2=0x7fcc240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:55.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.724+0000 7fcc3e994640 1 --2- 172.21.15.116:0/108979660 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcc240a2b80 0x7fcc240a5040 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcc28009890 tx=0x7fcc28008040 comp rx=0 tx=0).stop 2024-09-12T03:13:55.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.724+0000 7fcc3e994640 1 -- 172.21.15.116:0/108979660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc38101420 msgr2=0x7fcc380fd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:55.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.724+0000 7fcc3e994640 1 --2- 172.21.15.116:0/108979660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc38101420 0x7fcc380fd340 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009a50 tx=0x7fcc20004270 comp rx=0 tx=0).stop 2024-09-12T03:13:55.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.725+0000 7fcc3e994640 1 -- 172.21.15.116:0/108979660 shutdown_connections 2024-09-12T03:13:55.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.725+0000 7fcc3e994640 1 --2- 172.21.15.116:0/108979660 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcc240a2b80 0x7fcc240a5040 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:55.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.725+0000 7fcc3e994640 1 --2- 172.21.15.116:0/108979660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc38106a80 0x7fcc380fd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:55.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.725+0000 7fcc3e994640 1 --2- 172.21.15.116:0/108979660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc38101420 0x7fcc380fd340 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:55.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.725+0000 7fcc3e994640 1 --2- 172.21.15.116:0/108979660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc38100a80 0x7fcc380fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:55.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.725+0000 7fcc3e994640 1 -- 172.21.15.116:0/108979660 >> 172.21.15.116:0/108979660 conn(0x7fcc380fc7a0 msgr2=0x7fcc3806aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:55.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.725+0000 7fcc3e994640 1 -- 172.21.15.116:0/108979660 shutdown_connections 2024-09-12T03:13:55.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:55.725+0000 7fcc3e994640 1 -- 172.21.15.116:0/108979660 wait complete. 2024-09-12T03:13:55.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:55 smithi177 ceph-mon[35305]: pgmap v666: 97 pgs: 97 active+clean; 5.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:13:56.578 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:13:56.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:56 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/108979660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:13:56.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:56 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/108979660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:13:56.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:56 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/108979660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:13:56.821 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:13:57.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.299+0000 7f1d2dfe1640 1 -- 172.21.15.116:0/1756627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d2810c180 msgr2=0x7f1d2810e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:57.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.299+0000 7f1d2dfe1640 1 --2- 172.21.15.116:0/1756627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d2810c180 0x7f1d2810e620 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f1d20009ca0 tx=0x7f1d2002a950 comp rx=0 tx=0).stop 2024-09-12T03:13:57.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.299+0000 7f1d2dfe1640 1 -- 172.21.15.116:0/1756627 shutdown_connections 2024-09-12T03:13:57.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.299+0000 7f1d2dfe1640 1 --2- 172.21.15.116:0/1756627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d2810c180 0x7f1d2810e620 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.299+0000 7f1d2dfe1640 1 --2- 172.21.15.116:0/1756627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d28103670 0x7f1d2810bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.299+0000 7f1d2dfe1640 1 --2- 172.21.15.116:0/1756627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d28102d50 0x7f1d28103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.299+0000 7f1d2dfe1640 1 -- 172.21.15.116:0/1756627 >> 172.21.15.116:0/1756627 conn(0x7f1d280fc7d0 msgr2=0x7f1d280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:57.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.299+0000 7f1d2dfe1640 1 -- 172.21.15.116:0/1756627 shutdown_connections 2024-09-12T03:13:57.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.299+0000 7f1d2dfe1640 1 -- 172.21.15.116:0/1756627 wait complete. 2024-09-12T03:13:57.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.300+0000 7f1d2dfe1640 1 Processor -- start 2024-09-12T03:13:57.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.300+0000 7f1d2dfe1640 1 -- start start 2024-09-12T03:13:57.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.300+0000 7f1d2dfe1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d28102d50 0x7f1d28198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:57.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.300+0000 7f1d2dfe1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d28103670 0x7f1d281989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:57.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.300+0000 7f1d2dfe1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d2810c180 0x7f1d2819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:57.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.300+0000 7f1d2dfe1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1d28111fe0 con 0x7f1d28102d50 2024-09-12T03:13:57.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.300+0000 7f1d2dfe1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1d28111e60 con 0x7f1d2810c180 2024-09-12T03:13:57.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.300+0000 7f1d2dfe1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1d28112160 con 0x7f1d28103670 2024-09-12T03:13:57.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.300+0000 7f1d277fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d28102d50 0x7f1d28198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:57.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.300+0000 7f1d277fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d28102d50 0x7f1d28198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38696/0 (socket says 172.21.15.116:38696) 2024-09-12T03:13:57.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.300+0000 7f1d277fe640 1 -- 172.21.15.116:0/4101400850 learned_addr learned my addr 172.21.15.116:0/4101400850 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:57.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.301+0000 7f1d27fff640 1 --2- 172.21.15.116:0/4101400850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d2810c180 0x7f1d2819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:57.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.300+0000 7f1d26ffd640 1 --2- 172.21.15.116:0/4101400850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d28103670 0x7f1d281989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:57.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.301+0000 7f1d277fe640 1 -- 172.21.15.116:0/4101400850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d28103670 msgr2=0x7f1d281989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:57.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.301+0000 7f1d277fe640 1 --2- 172.21.15.116:0/4101400850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d28103670 0x7f1d281989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.301+0000 7f1d277fe640 1 -- 172.21.15.116:0/4101400850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d2810c180 msgr2=0x7f1d2819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:57.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.301+0000 7f1d277fe640 1 --2- 172.21.15.116:0/4101400850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d2810c180 0x7f1d2819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.301+0000 7f1d277fe640 1 -- 172.21.15.116:0/4101400850 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d2819d440 con 0x7f1d28102d50 2024-09-12T03:13:57.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.301+0000 7f1d27fff640 1 --2- 172.21.15.116:0/4101400850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d2810c180 0x7f1d2819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:13:57.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.301+0000 7f1d26ffd640 1 --2- 172.21.15.116:0/4101400850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d28103670 0x7f1d281989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:13:57.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.301+0000 7f1d277fe640 1 --2- 172.21.15.116:0/4101400850 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d28102d50 0x7f1d28198490 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800b4a0 tx=0x7f1d1800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:57.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.301+0000 7f1d24ff9640 1 -- 172.21.15.116:0/4101400850 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d18014070 con 0x7f1d28102d50 2024-09-12T03:13:57.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.301+0000 7f1d24ff9640 1 -- 172.21.15.116:0/4101400850 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d18010af0 con 0x7f1d28102d50 2024-09-12T03:13:57.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.301+0000 7f1d2dfe1640 1 -- 172.21.15.116:0/4101400850 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d2819d730 con 0x7f1d28102d50 2024-09-12T03:13:57.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.301+0000 7f1d24ff9640 1 -- 172.21.15.116:0/4101400850 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d18017d10 con 0x7f1d28102d50 2024-09-12T03:13:57.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.301+0000 7f1d2dfe1640 1 -- 172.21.15.116:0/4101400850 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d281a5010 con 0x7f1d28102d50 2024-09-12T03:13:57.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.303+0000 7f1d24ff9640 1 -- 172.21.15.116:0/4101400850 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1d18017070 con 0x7f1d28102d50 2024-09-12T03:13:57.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.304+0000 7f1d303ba740 1 Processor -- start 2024-09-12T03:13:57.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.304+0000 7f1d303ba740 1 -- start start 2024-09-12T03:13:57.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.304+0000 7f1d303ba740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ba1790bb40 0x55ba1790a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:57.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.304+0000 7f1d24ff9640 1 --2- 172.21.15.116:0/4101400850 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1d100a2b80 0x7f1d100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:57.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.304+0000 7f1d303ba740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba1790a5d0 0x55ba1790aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:57.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.304+0000 7f1d303ba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba17915370 0x55ba17917760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:57.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.304+0000 7f1d303ba740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ba178702f0 con 0x55ba1790bb40 2024-09-12T03:13:57.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.304+0000 7f1d303ba740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ba17817cf0 con 0x55ba1790a5d0 2024-09-12T03:13:57.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.304+0000 7f1d303ba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ba1786c2b0 con 0x55ba17915370 2024-09-12T03:13:57.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.304+0000 7f1d277fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ba1790bb40 0x55ba1790a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:57.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.304+0000 7f1d27fff640 1 --2- 172.21.15.116:0/4101400850 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1d100a2b80 0x7f1d100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:57.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.304+0000 7f1d277fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ba1790bb40 0x55ba1790a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38704/0 (socket says 172.21.15.116:38704) 2024-09-12T03:13:57.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.304+0000 7f1d277fe640 1 -- 172.21.15.116:0/457004411 learned_addr learned my addr 172.21.15.116:0/457004411 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:57.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.304+0000 7f1d26ffd640 1 --2- 172.21.15.116:0/457004411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba1790a5d0 0x55ba1790aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:57.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.304+0000 7f1d24ff9640 1 -- 172.21.15.116:0/4101400850 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1d180ce420 con 0x7f1d28102d50 2024-09-12T03:13:57.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.304+0000 7f1d27fff640 1 --2- 172.21.15.116:0/457004411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba17915370 0x55ba17917760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:57.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.304+0000 7f1d277fe640 1 -- 172.21.15.116:0/457004411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba17915370 msgr2=0x55ba17917760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:57.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.304+0000 7f1d277fe640 1 --2- 172.21.15.116:0/457004411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba17915370 0x55ba17917760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.304+0000 7f1d277fe640 1 -- 172.21.15.116:0/457004411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba1790a5d0 msgr2=0x55ba1790aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:57.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.304+0000 7f1d277fe640 1 --2- 172.21.15.116:0/457004411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba1790a5d0 0x55ba1790aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.304+0000 7f1d277fe640 1 -- 172.21.15.116:0/457004411 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba177fc3a0 con 0x55ba1790bb40 2024-09-12T03:13:57.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.305+0000 7f1d27fff640 1 --2- 172.21.15.116:0/4101400850 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1d100a2b80 0x7f1d100a5040 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1d20005e30 tx=0x7f1d2002fb00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:57.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.305+0000 7f1d277fe640 1 --2- 172.21.15.116:0/457004411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ba1790bb40 0x55ba1790a090 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f1d180a33f0 tx=0x7f1d18017730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6988239f2b99b2f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:57.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.305+0000 7f1d0dffb640 1 -- 172.21.15.116:0/457004411 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1801e040 con 0x55ba1790bb40 2024-09-12T03:13:57.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.305+0000 7f1d0dffb640 1 -- 172.21.15.116:0/457004411 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d1809b3b0 con 0x55ba1790bb40 2024-09-12T03:13:57.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.305+0000 7f1d0dffb640 1 -- 172.21.15.116:0/457004411 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d18088110 con 0x55ba1790bb40 2024-09-12T03:13:57.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.305+0000 7f1d303ba740 1 -- 172.21.15.116:0/457004411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ba1790bb40 msgr2=0x55ba1790a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:57.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.305+0000 7f1d303ba740 1 --2- 172.21.15.116:0/457004411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ba1790bb40 0x55ba1790a090 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f1d180a33f0 tx=0x7f1d18017730 comp rx=0 tx=0).stop 2024-09-12T03:13:57.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.306+0000 7f1d303ba740 1 -- 172.21.15.116:0/457004411 shutdown_connections 2024-09-12T03:13:57.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.306+0000 7f1d303ba740 1 --2- 172.21.15.116:0/457004411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba17915370 0x55ba17917760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.306+0000 7f1d303ba740 1 --2- 172.21.15.116:0/457004411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba1790a5d0 0x55ba1790aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.306+0000 7f1d303ba740 1 --2- 172.21.15.116:0/457004411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ba1790bb40 0x55ba1790a090 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.306+0000 7f1d303ba740 1 -- 172.21.15.116:0/457004411 >> 172.21.15.116:0/457004411 conn(0x55ba178d3310 msgr2=0x55ba17906c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:57.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.306+0000 7f1d303ba740 1 -- 172.21.15.116:0/457004411 shutdown_connections 2024-09-12T03:13:57.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.306+0000 7f1d303ba740 1 -- 172.21.15.116:0/457004411 wait complete. 2024-09-12T03:13:57.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.306+0000 7f1d303ba740 1 -- start start 2024-09-12T03:13:57.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.306+0000 7f1d303ba740 1 Processor -- start 2024-09-12T03:13:57.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.306+0000 7f1d303ba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba1790a5d0 0x55ba17914d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:57.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.306+0000 7f1d303ba740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba1790bb40 0x55ba17912000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:57.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.306+0000 7f1d303ba740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ba17915370 0x55ba179125d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:57.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.306+0000 7f1d303ba740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ba1791c480 con 0x55ba17915370 2024-09-12T03:13:57.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.306+0000 7f1d303ba740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ba1782b060 con 0x55ba1790bb40 2024-09-12T03:13:57.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.306+0000 7f1d303ba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ba1791c600 con 0x55ba1790a5d0 2024-09-12T03:13:57.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.306+0000 7f1d303ba740 10 client.0 _ll_register_callbacks cb 0x55ba1757e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:57.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.307+0000 7f1d27fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ba17915370 0x55ba179125d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:57.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.307+0000 7f1d27fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ba17915370 0x55ba179125d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38714/0 (socket says 172.21.15.116:38714) 2024-09-12T03:13:57.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.307+0000 7f1d27fff640 1 -- 172.21.15.116:0/3874650844 learned_addr learned my addr 172.21.15.116:0/3874650844 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:57.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.307+0000 7f1d26ffd640 1 --2- 172.21.15.116:0/3874650844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba1790bb40 0x55ba17912000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:57.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.307+0000 7f1d277fe640 1 --2- 172.21.15.116:0/3874650844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba1790a5d0 0x55ba17914d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:57.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.307+0000 7f1d27fff640 1 -- 172.21.15.116:0/3874650844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba1790a5d0 msgr2=0x55ba17914d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:57.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.307+0000 7f1d27fff640 1 --2- 172.21.15.116:0/3874650844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba1790a5d0 0x55ba17914d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.307+0000 7f1d27fff640 1 -- 172.21.15.116:0/3874650844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba1790bb40 msgr2=0x55ba17912000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:57.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.307+0000 7f1d27fff640 1 --2- 172.21.15.116:0/3874650844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba1790bb40 0x55ba17912000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.307+0000 7f1d27fff640 1 -- 172.21.15.116:0/3874650844 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba175e13c0 con 0x55ba17915370 2024-09-12T03:13:57.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.307+0000 7f1d27fff640 1 --2- 172.21.15.116:0/3874650844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ba17915370 0x55ba179125d0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f1d2002f6d0 tx=0x7f1d2002a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3796e2b647c8e4d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:57.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.307+0000 7f1d06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:57.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.307+0000 7f1d06ffd640 1 -- 172.21.15.116:0/3874650844 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20050060 con 0x55ba17915370 2024-09-12T03:13:57.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.307+0000 7f1d06ffd640 1 -- 172.21.15.116:0/3874650844 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1d20004230 con 0x55ba17915370 2024-09-12T03:13:57.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.307+0000 7f1d303ba740 1 -- 172.21.15.116:0/3874650844 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d1809b3b0 con 0x55ba17915370 2024-09-12T03:13:57.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.307+0000 7f1d06ffd640 1 -- 172.21.15.116:0/3874650844 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1d20034050 con 0x55ba17915370 2024-09-12T03:13:57.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.307+0000 7f1d303ba740 10 client.18336 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:57.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.307+0000 7f1d303ba740 1 -- 172.21.15.116:0/3874650844 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ba17927f00 con 0x55ba17915370 2024-09-12T03:13:57.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.308+0000 7f1d06ffd640 1 -- 172.21.15.116:0/3874650844 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d1809b3b0 con 0x55ba17915370 2024-09-12T03:13:57.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.308+0000 7f1d06ffd640 1 -- 172.21.15.116:0/3874650844 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1d2003c450 con 0x55ba17915370 2024-09-12T03:13:57.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.308+0000 7f1d303ba740 10 client.18336 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:57.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.308+0000 7f1d303ba740 10 client.18336 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:13:57.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.308+0000 7f1d303ba740 20 client.18336 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:57.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.308+0000 7f1d303ba740 1 --2- 172.21.15.116:0/3874650844 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55ba17929360 0x55ba1792b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:57.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.308+0000 7f1d303ba740 4 client.18336 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:13:57.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.308+0000 7f1d303ba740 1 -- 172.21.15.116:0/3874650844 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba1787d860 con 0x55ba17929360 2024-09-12T03:13:57.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.308+0000 7f1d277fe640 1 --2- 172.21.15.116:0/3874650844 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55ba17929360 0x55ba1792b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:57.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.309+0000 7f1d277fe640 1 --2- 172.21.15.116:0/3874650844 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55ba17929360 0x55ba1792b7e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1b930a33b6b9ecf8 server_cookie=21c38dff54312ea4 in_seq=0 out_seq=0 2024-09-12T03:13:57.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.309+0000 7f1d06ffd640 10 client.18336 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:13:57.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.309+0000 7f1d06ffd640 1 -- 172.21.15.116:0/3874650844 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ba1787d860 con 0x55ba17929360 2024-09-12T03:13:57.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.309+0000 7f1d06ffd640 10 client.18336 handle_command_reply: tid=0 2024-09-12T03:13:57.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.310+0000 7f1d303ba740 1 client.18336 shutdown 2024-09-12T03:13:57.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.310+0000 7f1d303ba740 1 -- 172.21.15.116:0/3874650844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ba17915370 msgr2=0x55ba179125d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:57.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.310+0000 7f1d303ba740 1 --2- 172.21.15.116:0/3874650844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ba17915370 0x55ba179125d0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f1d2002f6d0 tx=0x7f1d2002a950 comp rx=0 tx=0).stop 2024-09-12T03:13:57.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.310+0000 7f1d303ba740 1 -- 172.21.15.116:0/3874650844 shutdown_connections 2024-09-12T03:13:57.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.310+0000 7f1d303ba740 1 --2- 172.21.15.116:0/3874650844 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55ba17929360 0x55ba1792b7e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.310+0000 7f1d303ba740 1 --2- 172.21.15.116:0/3874650844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ba17915370 0x55ba179125d0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.310+0000 7f1d303ba740 1 --2- 172.21.15.116:0/3874650844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba1790bb40 0x55ba17912000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.310+0000 7f1d303ba740 1 --2- 172.21.15.116:0/3874650844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba1790a5d0 0x55ba17914d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.310+0000 7f1d303ba740 1 -- 172.21.15.116:0/3874650844 >> 172.21.15.116:0/3874650844 conn(0x55ba178d3310 msgr2=0x55ba1790c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:57.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.310+0000 7f1d06ffd640 0 client.18336 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:13:57.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.310+0000 7f1d303ba740 1 -- 172.21.15.116:0/3874650844 shutdown_connections 2024-09-12T03:13:57.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.310+0000 7f1d303ba740 1 -- 172.21.15.116:0/3874650844 wait complete. 2024-09-12T03:13:57.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.310+0000 7f1d303ba740 20 client.18336 trim_cache size 0 max 16384 2024-09-12T03:13:57.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.323+0000 7f1d303ba740 1 Processor -- start 2024-09-12T03:13:57.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.323+0000 7f1d303ba740 1 -- start start 2024-09-12T03:13:57.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.323+0000 7f1d303ba740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ba1790a5d0 0x55ba17913f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:57.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.323+0000 7f1d303ba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba1790bb40 0x55ba1790d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:57.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.323+0000 7f1d303ba740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba17915370 0x55ba1790d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:57.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.323+0000 7f1d303ba740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1d281adc80 con 0x55ba1790a5d0 2024-09-12T03:13:57.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.323+0000 7f1d303ba740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1d281ab550 con 0x55ba17915370 2024-09-12T03:13:57.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.323+0000 7f1d303ba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1d281a93a0 con 0x55ba1790bb40 2024-09-12T03:13:57.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.323+0000 7f1d277fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ba1790a5d0 0x55ba17913f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:57.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.323+0000 7f1d277fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ba1790a5d0 0x55ba17913f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38728/0 (socket says 172.21.15.116:38728) 2024-09-12T03:13:57.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.323+0000 7f1d277fe640 1 -- 172.21.15.116:0/1787719904 learned_addr learned my addr 172.21.15.116:0/1787719904 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:57.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.323+0000 7f1d26ffd640 1 --2- 172.21.15.116:0/1787719904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba1790bb40 0x55ba1790d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:57.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.323+0000 7f1d27fff640 1 --2- 172.21.15.116:0/1787719904 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba17915370 0x55ba1790d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:57.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.323+0000 7f1d277fe640 1 -- 172.21.15.116:0/1787719904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba1790bb40 msgr2=0x55ba1790d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:57.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.323+0000 7f1d277fe640 1 --2- 172.21.15.116:0/1787719904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba1790bb40 0x55ba1790d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.323+0000 7f1d277fe640 1 -- 172.21.15.116:0/1787719904 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba17915370 msgr2=0x55ba1790d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:57.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.323+0000 7f1d277fe640 1 --2- 172.21.15.116:0/1787719904 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba17915370 0x55ba1790d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.323+0000 7f1d277fe640 1 -- 172.21.15.116:0/1787719904 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba17914970 con 0x55ba1790a5d0 2024-09-12T03:13:57.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.323+0000 7f1d27fff640 1 --2- 172.21.15.116:0/1787719904 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba17915370 0x55ba1790d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:13:57.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.323+0000 7f1d26ffd640 1 --2- 172.21.15.116:0/1787719904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba1790bb40 0x55ba1790d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:13:57.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.323+0000 7f1d277fe640 1 --2- 172.21.15.116:0/1787719904 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ba1790a5d0 0x55ba17913f70 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f1d180cc2c0 tx=0x7f1d1800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d55242fac35b85 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:57.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.323+0000 7f1d0d7fa640 1 -- 172.21.15.116:0/1787719904 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1801e040 con 0x55ba1790a5d0 2024-09-12T03:13:57.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.323+0000 7f1d0d7fa640 1 -- 172.21.15.116:0/1787719904 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d18098d50 con 0x55ba1790a5d0 2024-09-12T03:13:57.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.323+0000 7f1d0d7fa640 1 -- 172.21.15.116:0/1787719904 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d18017460 con 0x55ba1790a5d0 2024-09-12T03:13:57.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.324+0000 7f1d303ba740 1 -- 172.21.15.116:0/1787719904 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ba1790a5d0 msgr2=0x55ba17913f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:57.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.324+0000 7f1d303ba740 1 --2- 172.21.15.116:0/1787719904 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ba1790a5d0 0x55ba17913f70 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f1d180cc2c0 tx=0x7f1d1800bf90 comp rx=0 tx=0).stop 2024-09-12T03:13:57.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.324+0000 7f1d303ba740 1 -- 172.21.15.116:0/1787719904 shutdown_connections 2024-09-12T03:13:57.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.324+0000 7f1d303ba740 1 --2- 172.21.15.116:0/1787719904 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba17915370 0x55ba1790d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.324+0000 7f1d303ba740 1 --2- 172.21.15.116:0/1787719904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba1790bb40 0x55ba1790d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.324+0000 7f1d303ba740 1 --2- 172.21.15.116:0/1787719904 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ba1790a5d0 0x55ba17913f70 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.324+0000 7f1d303ba740 1 -- 172.21.15.116:0/1787719904 >> 172.21.15.116:0/1787719904 conn(0x55ba178d3310 msgr2=0x55ba17913860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:57.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.324+0000 7f1d303ba740 1 -- 172.21.15.116:0/1787719904 shutdown_connections 2024-09-12T03:13:57.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.324+0000 7f1d303ba740 1 -- 172.21.15.116:0/1787719904 wait complete. 2024-09-12T03:13:57.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.324+0000 7f1d303ba740 1 -- start start 2024-09-12T03:13:57.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.325+0000 7f1d303ba740 1 Processor -- start 2024-09-12T03:13:57.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.325+0000 7f1d303ba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba1790a5d0 0x55ba17921a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:57.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.325+0000 7f1d303ba740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba1790bb40 0x55ba17921fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:57.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.325+0000 7f1d303ba740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ba17915370 0x55ba1790e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:57.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.325+0000 7f1d303ba740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1d281a9a10 con 0x55ba17915370 2024-09-12T03:13:57.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.325+0000 7f1d303ba740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1d281aae80 con 0x55ba1790bb40 2024-09-12T03:13:57.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.325+0000 7f1d303ba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1d281aa0e0 con 0x55ba1790a5d0 2024-09-12T03:13:57.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.325+0000 7f1d303ba740 10 client.0 _ll_register_callbacks cb 0x7f1d20009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:57.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.325+0000 7f1d27fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ba17915370 0x55ba1790e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:57.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.325+0000 7f1d27fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ba17915370 0x55ba1790e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38736/0 (socket says 172.21.15.116:38736) 2024-09-12T03:13:57.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.325+0000 7f1d27fff640 1 -- 172.21.15.116:0/1599306947 learned_addr learned my addr 172.21.15.116:0/1599306947 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:57.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.325+0000 7f1d277fe640 1 --2- 172.21.15.116:0/1599306947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba1790a5d0 0x55ba17921a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:57.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.325+0000 7f1d27fff640 1 -- 172.21.15.116:0/1599306947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba1790a5d0 msgr2=0x55ba17921a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:57.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.325+0000 7f1d27fff640 1 --2- 172.21.15.116:0/1599306947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba1790a5d0 0x55ba17921a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.325+0000 7f1d27fff640 1 -- 172.21.15.116:0/1599306947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba1790bb40 msgr2=0x55ba17921fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:57.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.325+0000 7f1d26ffd640 1 --2- 172.21.15.116:0/1599306947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba1790bb40 0x55ba17921fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:57.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.325+0000 7f1d27fff640 1 --2- 172.21.15.116:0/1599306947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba1790bb40 0x55ba17921fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.325+0000 7f1d27fff640 1 -- 172.21.15.116:0/1599306947 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba1790eec0 con 0x55ba17915370 2024-09-12T03:13:57.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.325+0000 7f1d26ffd640 1 --2- 172.21.15.116:0/1599306947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba1790bb40 0x55ba17921fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:13:57.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.325+0000 7f1d27fff640 1 --2- 172.21.15.116:0/1599306947 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ba17915370 0x55ba1790e830 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f1d2003c8c0 tx=0x7f1d2003c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406013aa45d6f5c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:57.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.326+0000 7f1d06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:57.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.326+0000 7f1d06ffd640 1 -- 172.21.15.116:0/1599306947 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2005b070 con 0x55ba17915370 2024-09-12T03:13:57.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.326+0000 7f1d06ffd640 1 -- 172.21.15.116:0/1599306947 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1d20004240 con 0x55ba17915370 2024-09-12T03:13:57.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.326+0000 7f1d303ba740 1 -- 172.21.15.116:0/1599306947 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ba17877d70 con 0x55ba17915370 2024-09-12T03:13:57.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.326+0000 7f1d06ffd640 1 -- 172.21.15.116:0/1599306947 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1d20050020 con 0x55ba17915370 2024-09-12T03:13:57.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.326+0000 7f1d303ba740 10 client.18342 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:57.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.326+0000 7f1d303ba740 1 -- 172.21.15.116:0/1599306947 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ba179939b0 con 0x55ba17915370 2024-09-12T03:13:57.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.326+0000 7f1d06ffd640 1 -- 172.21.15.116:0/1599306947 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ba17877d70 con 0x55ba17915370 2024-09-12T03:13:57.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.326+0000 7f1d06ffd640 1 -- 172.21.15.116:0/1599306947 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1d2004b9b0 con 0x55ba17915370 2024-09-12T03:13:57.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.326+0000 7f1d303ba740 10 client.18342 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:57.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.326+0000 7f1d303ba740 10 client.18342 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:13:57.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.326+0000 7f1d303ba740 20 client.18342 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:57.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.326+0000 7f1d303ba740 1 --2- 172.21.15.116:0/1599306947 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55ba17994130 0x55ba17996550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:57.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.326+0000 7f1d303ba740 4 client.18342 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:13:57.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.326+0000 7f1d303ba740 1 -- 172.21.15.116:0/1599306947 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ba177f4170 con 0x55ba17994130 2024-09-12T03:13:57.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.327+0000 7f1d277fe640 1 --2- 172.21.15.116:0/1599306947 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55ba17994130 0x55ba17996550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:57.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.327+0000 7f1d277fe640 1 --2- 172.21.15.116:0/1599306947 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55ba17994130 0x55ba17996550 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9bebfd67d4e5e38b server_cookie=5dae7baa5db9657a in_seq=0 out_seq=0 2024-09-12T03:13:57.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.327+0000 7f1d06ffd640 10 client.18342 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:13:57.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.327+0000 7f1d06ffd640 1 -- 172.21.15.116:0/1599306947 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ba177f4170 con 0x55ba17994130 2024-09-12T03:13:57.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.327+0000 7f1d06ffd640 10 client.18342 handle_command_reply: tid=0 2024-09-12T03:13:57.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.327+0000 7f1d303ba740 1 client.18342 shutdown 2024-09-12T03:13:57.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.327+0000 7f1d303ba740 1 -- 172.21.15.116:0/1599306947 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ba17915370 msgr2=0x55ba1790e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:57.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.327+0000 7f1d303ba740 1 --2- 172.21.15.116:0/1599306947 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ba17915370 0x55ba1790e830 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f1d2003c8c0 tx=0x7f1d2003c700 comp rx=0 tx=0).stop 2024-09-12T03:13:57.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.327+0000 7f1d303ba740 1 -- 172.21.15.116:0/1599306947 shutdown_connections 2024-09-12T03:13:57.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.327+0000 7f1d303ba740 1 --2- 172.21.15.116:0/1599306947 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55ba17994130 0x55ba17996550 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.327+0000 7f1d303ba740 1 --2- 172.21.15.116:0/1599306947 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ba17915370 0x55ba1790e830 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.327+0000 7f1d303ba740 1 --2- 172.21.15.116:0/1599306947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba1790bb40 0x55ba17921fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.327+0000 7f1d303ba740 1 --2- 172.21.15.116:0/1599306947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba1790a5d0 0x55ba17921a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.327+0000 7f1d06ffd640 0 client.18342 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:13:57.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.327+0000 7f1d303ba740 1 -- 172.21.15.116:0/1599306947 >> 172.21.15.116:0/1599306947 conn(0x55ba178d3310 msgr2=0x55ba17914080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:57.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.327+0000 7f1d303ba740 1 -- 172.21.15.116:0/1599306947 shutdown_connections 2024-09-12T03:13:57.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.327+0000 7f1d303ba740 1 -- 172.21.15.116:0/1599306947 wait complete. 2024-09-12T03:13:57.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.328+0000 7f1d303ba740 20 client.18342 trim_cache size 0 max 16384 2024-09-12T03:13:57.334 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:13:57.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.328+0000 7f1d2dfe1640 1 -- 172.21.15.116:0/4101400850 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1d100a2b80 msgr2=0x7f1d100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:57.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.328+0000 7f1d2dfe1640 1 --2- 172.21.15.116:0/4101400850 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1d100a2b80 0x7f1d100a5040 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1d20005e30 tx=0x7f1d2002fb00 comp rx=0 tx=0).stop 2024-09-12T03:13:57.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.328+0000 7f1d2dfe1640 1 -- 172.21.15.116:0/4101400850 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d28102d50 msgr2=0x7f1d28198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:57.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.328+0000 7f1d2dfe1640 1 --2- 172.21.15.116:0/4101400850 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d28102d50 0x7f1d28198490 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800b4a0 tx=0x7f1d1800b960 comp rx=0 tx=0).stop 2024-09-12T03:13:57.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.328+0000 7f1d2dfe1640 1 -- 172.21.15.116:0/4101400850 shutdown_connections 2024-09-12T03:13:57.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.328+0000 7f1d2dfe1640 1 --2- 172.21.15.116:0/4101400850 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1d100a2b80 0x7f1d100a5040 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.328+0000 7f1d2dfe1640 1 --2- 172.21.15.116:0/4101400850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d2810c180 0x7f1d2819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.328+0000 7f1d2dfe1640 1 --2- 172.21.15.116:0/4101400850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d28103670 0x7f1d281989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.328+0000 7f1d2dfe1640 1 --2- 172.21.15.116:0/4101400850 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d28102d50 0x7f1d28198490 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:57.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.328+0000 7f1d2dfe1640 1 -- 172.21.15.116:0/4101400850 >> 172.21.15.116:0/4101400850 conn(0x7f1d280fc7d0 msgr2=0x7f1d280fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:57.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.328+0000 7f1d2dfe1640 1 -- 172.21.15.116:0/4101400850 shutdown_connections 2024-09-12T03:13:57.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:57.328+0000 7f1d2dfe1640 1 -- 172.21.15.116:0/4101400850 wait complete. 2024-09-12T03:13:57.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:57 smithi116 ceph-mon[31410]: pgmap v667: 97 pgs: 97 active+clean; 5.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:13:57.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:57 smithi183 ceph-mon[35535]: pgmap v667: 97 pgs: 97 active+clean; 5.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:13:57.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:57 smithi177 ceph-mon[35305]: pgmap v667: 97 pgs: 97 active+clean; 5.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:13:58.078 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:13:58.325 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:13:58.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.833+0000 7f37a6de2640 1 -- 172.21.15.116:0/957387064 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37a01036c0 msgr2=0x7f37a010bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:58.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.833+0000 7f37a6de2640 1 --2- 172.21.15.116:0/957387064 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37a01036c0 0x7f37a010bbb0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f3790009900 tx=0x7f379002a480 comp rx=0 tx=0).stop 2024-09-12T03:13:58.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.834+0000 7f37a6de2640 1 -- 172.21.15.116:0/957387064 shutdown_connections 2024-09-12T03:13:58.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.834+0000 7f37a6de2640 1 --2- 172.21.15.116:0/957387064 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37a010c180 0x7f37a010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.834+0000 7f37a6de2640 1 --2- 172.21.15.116:0/957387064 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37a01036c0 0x7f37a010bbb0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.834+0000 7f37a6de2640 1 --2- 172.21.15.116:0/957387064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37a0102da0 0x7f37a0103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.834+0000 7f37a6de2640 1 -- 172.21.15.116:0/957387064 >> 172.21.15.116:0/957387064 conn(0x7f37a00fc7a0 msgr2=0x7f37a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:58.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.834+0000 7f37a6de2640 1 -- 172.21.15.116:0/957387064 shutdown_connections 2024-09-12T03:13:58.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.834+0000 7f37a6de2640 1 -- 172.21.15.116:0/957387064 wait complete. 2024-09-12T03:13:58.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.834+0000 7f37a6de2640 1 Processor -- start 2024-09-12T03:13:58.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.834+0000 7f37a6de2640 1 -- start start 2024-09-12T03:13:58.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.834+0000 7f37a6de2640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37a0102da0 0x7f37a0100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:58.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.834+0000 7f37a6de2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37a01036c0 0x7f37a0100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:58.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.834+0000 7f37a6de2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37a010c180 0x7f37a0101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:58.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.834+0000 7f37a6de2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f37a0111f60 con 0x7f37a01036c0 2024-09-12T03:13:58.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.834+0000 7f37a6de2640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f37a0111de0 con 0x7f37a0102da0 2024-09-12T03:13:58.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.834+0000 7f37a6de2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f37a01120e0 con 0x7f37a010c180 2024-09-12T03:13:58.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.834+0000 7f3797fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37a01036c0 0x7f37a0100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:58.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.834+0000 7f3797fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37a01036c0 0x7f37a0100f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38764/0 (socket says 172.21.15.116:38764) 2024-09-12T03:13:58.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.834+0000 7f3797fff640 1 -- 172.21.15.116:0/2519394679 learned_addr learned my addr 172.21.15.116:0/2519394679 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:58.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.835+0000 7f37a4b7e640 1 --2- 172.21.15.116:0/2519394679 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37a0102da0 0x7f37a0100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:58.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.835+0000 7f37a537f640 1 --2- 172.21.15.116:0/2519394679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37a010c180 0x7f37a0101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:58.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.835+0000 7f3797fff640 1 -- 172.21.15.116:0/2519394679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37a010c180 msgr2=0x7f37a0101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:58.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.835+0000 7f3797fff640 1 --2- 172.21.15.116:0/2519394679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37a010c180 0x7f37a0101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.835+0000 7f3797fff640 1 -- 172.21.15.116:0/2519394679 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37a0102da0 msgr2=0x7f37a0100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:58.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.835+0000 7f3797fff640 1 --2- 172.21.15.116:0/2519394679 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37a0102da0 0x7f37a0100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.835+0000 7f3797fff640 1 -- 172.21.15.116:0/2519394679 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a01a6920 con 0x7f37a01036c0 2024-09-12T03:13:58.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.835+0000 7f37a537f640 1 --2- 172.21.15.116:0/2519394679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37a010c180 0x7f37a0101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:13:58.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.835+0000 7f3797fff640 1 --2- 172.21.15.116:0/2519394679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37a01036c0 0x7f37a0100f40 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f3790006010 tx=0x7f37900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:58.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.835+0000 7f3795ffb640 1 -- 172.21.15.116:0/2519394679 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379003f030 con 0x7f37a01036c0 2024-09-12T03:13:58.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.835+0000 7f37a6de2640 1 -- 172.21.15.116:0/2519394679 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37a01a6bb0 con 0x7f37a01036c0 2024-09-12T03:13:58.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.835+0000 7f3795ffb640 1 -- 172.21.15.116:0/2519394679 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3790003ce0 con 0x7f37a01036c0 2024-09-12T03:13:58.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.835+0000 7f3795ffb640 1 -- 172.21.15.116:0/2519394679 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3790035070 con 0x7f37a01036c0 2024-09-12T03:13:58.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.835+0000 7f37a6de2640 1 -- 172.21.15.116:0/2519394679 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37a01a7090 con 0x7f37a01036c0 2024-09-12T03:13:58.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.838+0000 7f3795ffb640 1 -- 172.21.15.116:0/2519394679 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3790042030 con 0x7f37a01036c0 2024-09-12T03:13:58.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.838+0000 7f37a91bb740 1 Processor -- start 2024-09-12T03:13:58.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.838+0000 7f37a91bb740 1 -- start start 2024-09-12T03:13:58.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.838+0000 7f37a91bb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a50650eb40 0x55a50650d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:58.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.838+0000 7f37a91bb740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50650d5d0 0x55a50650da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:58.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.838+0000 7f3795ffb640 1 --2- 172.21.15.116:0/2519394679 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f37880a2b80 0x7f37880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:58.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.838+0000 7f37a91bb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a506518370 0x55a50651a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:58.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.838+0000 7f37a91bb740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a5064732f0 con 0x55a50650d5d0 2024-09-12T03:13:58.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.838+0000 7f37a91bb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a50641acf0 con 0x55a506518370 2024-09-12T03:13:58.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.838+0000 7f37a91bb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a50646f2b0 con 0x55a50650eb40 2024-09-12T03:13:58.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.838+0000 7f3797fff640 1 --2- 172.21.15.116:0/2519394679 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f37880a2b80 0x7f37880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:58.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.838+0000 7f37a4b7e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50650d5d0 0x55a50650da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:58.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.838+0000 7f37a4b7e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50650d5d0 0x55a50650da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38768/0 (socket says 172.21.15.116:38768) 2024-09-12T03:13:58.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.839+0000 7f37a4b7e640 1 -- 172.21.15.116:0/2780429912 learned_addr learned my addr 172.21.15.116:0/2780429912 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:58.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.839+0000 7f3795ffb640 1 -- 172.21.15.116:0/2519394679 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f37900ef520 con 0x7f37a01036c0 2024-09-12T03:13:58.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.839+0000 7f37a537f640 1 --2- 172.21.15.116:0/2780429912 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a506518370 0x55a50651a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:58.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.839+0000 7f37a537f640 1 --2- 172.21.15.116:0/2780429912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a50650eb40 0x55a50650d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:58.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.839+0000 7f37a4b7e640 1 -- 172.21.15.116:0/2780429912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a50650eb40 msgr2=0x55a50650d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:58.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.839+0000 7f37a4b7e640 1 --2- 172.21.15.116:0/2780429912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a50650eb40 0x55a50650d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.839+0000 7f37a4b7e640 1 -- 172.21.15.116:0/2780429912 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a506518370 msgr2=0x55a50651a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:58.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.839+0000 7f37a4b7e640 1 --2- 172.21.15.116:0/2780429912 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a506518370 0x55a50651a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.839+0000 7f37a4b7e640 1 -- 172.21.15.116:0/2780429912 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5063ff3a0 con 0x55a50650d5d0 2024-09-12T03:13:58.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.839+0000 7f37a4b7e640 1 --2- 172.21.15.116:0/2780429912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50650d5d0 0x55a50650da10 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f379c00b760 tx=0x7f379c00bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15aebe4fafe64b05 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:58.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.839+0000 7f3797fff640 1 --2- 172.21.15.116:0/2519394679 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f37880a2b80 0x7f37880a5040 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f379004a000 tx=0x7f37900b1350 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:58.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.839+0000 7f3786ffd640 1 -- 172.21.15.116:0/2780429912 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379c003dd0 con 0x55a50650d5d0 2024-09-12T03:13:58.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.839+0000 7f3786ffd640 1 -- 172.21.15.116:0/2780429912 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f379c015070 con 0x55a50650d5d0 2024-09-12T03:13:58.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.839+0000 7f3786ffd640 1 -- 172.21.15.116:0/2780429912 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379c011440 con 0x55a50650d5d0 2024-09-12T03:13:58.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.840+0000 7f37a91bb740 1 -- 172.21.15.116:0/2780429912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50650d5d0 msgr2=0x55a50650da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:58.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.840+0000 7f37a91bb740 1 --2- 172.21.15.116:0/2780429912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50650d5d0 0x55a50650da10 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f379c00b760 tx=0x7f379c00bc20 comp rx=0 tx=0).stop 2024-09-12T03:13:58.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.840+0000 7f37a91bb740 1 -- 172.21.15.116:0/2780429912 shutdown_connections 2024-09-12T03:13:58.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.840+0000 7f37a91bb740 1 --2- 172.21.15.116:0/2780429912 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a506518370 0x55a50651a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.840+0000 7f37a91bb740 1 --2- 172.21.15.116:0/2780429912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50650d5d0 0x55a50650da10 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.840+0000 7f37a91bb740 1 --2- 172.21.15.116:0/2780429912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a50650eb40 0x55a50650d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.840+0000 7f37a91bb740 1 -- 172.21.15.116:0/2780429912 >> 172.21.15.116:0/2780429912 conn(0x55a5064d6310 msgr2=0x55a506509c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:58.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.840+0000 7f37a91bb740 1 -- 172.21.15.116:0/2780429912 shutdown_connections 2024-09-12T03:13:58.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.840+0000 7f37a91bb740 1 -- 172.21.15.116:0/2780429912 wait complete. 2024-09-12T03:13:58.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.840+0000 7f37a91bb740 1 -- start start 2024-09-12T03:13:58.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.840+0000 7f37a91bb740 1 Processor -- start 2024-09-12T03:13:58.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.841+0000 7f37a91bb740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50650d5d0 0x55a506517d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:58.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.841+0000 7f37a91bb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a50650eb40 0x55a506515010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:58.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.841+0000 7f37a91bb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a506518370 0x55a506515600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:58.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.841+0000 7f37a91bb740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a50651b070 con 0x55a50650d5d0 2024-09-12T03:13:58.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.841+0000 7f37a91bb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a50642e060 con 0x55a50650eb40 2024-09-12T03:13:58.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.841+0000 7f37a91bb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a50651b1f0 con 0x55a506518370 2024-09-12T03:13:58.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.841+0000 7f37a91bb740 10 client.0 _ll_register_callbacks cb 0x55a5061818c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:58.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.841+0000 7f37a537f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50650d5d0 0x55a506517d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:58.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.841+0000 7f37a537f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50650d5d0 0x55a506517d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38778/0 (socket says 172.21.15.116:38778) 2024-09-12T03:13:58.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.841+0000 7f37a537f640 1 -- 172.21.15.116:0/2521062408 learned_addr learned my addr 172.21.15.116:0/2521062408 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:58.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.841+0000 7f37a4b7e640 1 --2- 172.21.15.116:0/2521062408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a50650eb40 0x55a506515010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:58.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.841+0000 7f37a537f640 1 --2- 172.21.15.116:0/2521062408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a506518370 0x55a506515600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:58.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.841+0000 7f37a537f640 1 -- 172.21.15.116:0/2521062408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a506518370 msgr2=0x55a506515600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:58.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.841+0000 7f37a537f640 1 --2- 172.21.15.116:0/2521062408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a506518370 0x55a506515600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.841+0000 7f37a537f640 1 -- 172.21.15.116:0/2521062408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a50650eb40 msgr2=0x55a506515010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:58.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.841+0000 7f37a537f640 1 --2- 172.21.15.116:0/2521062408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a50650eb40 0x55a506515010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.841+0000 7f37a537f640 1 -- 172.21.15.116:0/2521062408 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5061e43c0 con 0x55a50650d5d0 2024-09-12T03:13:58.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.841+0000 7f37a537f640 1 --2- 172.21.15.116:0/2521062408 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50650d5d0 0x55a506517d60 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f379800b010 tx=0x7f3798004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0fbe37d4e4f9d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:58.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.841+0000 7f377ffff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:58.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.842+0000 7f377ffff640 1 -- 172.21.15.116:0/2521062408 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3798004b20 con 0x55a50650d5d0 2024-09-12T03:13:58.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.842+0000 7f377ffff640 1 -- 172.21.15.116:0/2521062408 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f3798004cc0 con 0x55a50650d5d0 2024-09-12T03:13:58.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.842+0000 7f37a91bb740 1 -- 172.21.15.116:0/2521062408 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f379c015070 con 0x55a50650d5d0 2024-09-12T03:13:58.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.842+0000 7f377ffff640 1 -- 172.21.15.116:0/2521062408 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3798025050 con 0x55a50650d5d0 2024-09-12T03:13:58.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.842+0000 7f37a91bb740 10 client.18354 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:58.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.842+0000 7f37a91bb740 1 -- 172.21.15.116:0/2521062408 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a506526b70 con 0x55a50650d5d0 2024-09-12T03:13:58.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.842+0000 7f377ffff640 1 -- 172.21.15.116:0/2521062408 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f379c015070 con 0x55a50650d5d0 2024-09-12T03:13:58.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.842+0000 7f377ffff640 1 -- 172.21.15.116:0/2521062408 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3798004420 con 0x55a50650d5d0 2024-09-12T03:13:58.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.842+0000 7f37a91bb740 10 client.18354 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:58.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.842+0000 7f37a91bb740 10 client.18354 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:13:58.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.842+0000 7f37a91bb740 20 client.18354 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:58.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.842+0000 7f37a91bb740 1 --2- 172.21.15.116:0/2521062408 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55a506528030 0x55a50652a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:58.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.842+0000 7f37a91bb740 4 client.18354 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:13:58.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.842+0000 7f37a91bb740 1 -- 172.21.15.116:0/2521062408 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a506480860 con 0x55a506528030 2024-09-12T03:13:58.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.842+0000 7f37a4b7e640 1 --2- 172.21.15.116:0/2521062408 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55a506528030 0x55a50652a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:58.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.843+0000 7f377ffff640 10 client.18354 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:13:58.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.843+0000 7f37a4b7e640 1 --2- 172.21.15.116:0/2521062408 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55a506528030 0x55a50652a480 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7379ec58c533ea5 server_cookie=8d4a4317d7cf0969 in_seq=0 out_seq=0 2024-09-12T03:13:58.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.843+0000 7f377ffff640 1 -- 172.21.15.116:0/2521062408 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a506480860 con 0x55a506528030 2024-09-12T03:13:58.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.843+0000 7f377ffff640 10 client.18354 handle_command_reply: tid=0 2024-09-12T03:13:58.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.844+0000 7f37a91bb740 1 client.18354 shutdown 2024-09-12T03:13:58.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.844+0000 7f37a91bb740 1 -- 172.21.15.116:0/2521062408 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50650d5d0 msgr2=0x55a506517d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:58.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.844+0000 7f37a91bb740 1 --2- 172.21.15.116:0/2521062408 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50650d5d0 0x55a506517d60 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f379800b010 tx=0x7f3798004a40 comp rx=0 tx=0).stop 2024-09-12T03:13:58.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.844+0000 7f37a91bb740 1 -- 172.21.15.116:0/2521062408 shutdown_connections 2024-09-12T03:13:58.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.844+0000 7f37a91bb740 1 --2- 172.21.15.116:0/2521062408 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55a506528030 0x55a50652a480 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.844+0000 7f37a91bb740 1 --2- 172.21.15.116:0/2521062408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a506518370 0x55a506515600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.844+0000 7f37a91bb740 1 --2- 172.21.15.116:0/2521062408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a50650eb40 0x55a506515010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.844+0000 7f37a91bb740 1 --2- 172.21.15.116:0/2521062408 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50650d5d0 0x55a506517d60 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.844+0000 7f377ffff640 0 client.18354 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:13:58.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.844+0000 7f37a91bb740 1 -- 172.21.15.116:0/2521062408 >> 172.21.15.116:0/2521062408 conn(0x55a5064d6310 msgr2=0x55a50650f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:58.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.844+0000 7f37a91bb740 1 -- 172.21.15.116:0/2521062408 shutdown_connections 2024-09-12T03:13:58.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.844+0000 7f37a91bb740 1 -- 172.21.15.116:0/2521062408 wait complete. 2024-09-12T03:13:58.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.844+0000 7f37a91bb740 20 client.18354 trim_cache size 0 max 16384 2024-09-12T03:13:58.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.857+0000 7f37a91bb740 1 Processor -- start 2024-09-12T03:13:58.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.857+0000 7f37a91bb740 1 -- start start 2024-09-12T03:13:58.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.857+0000 7f37a91bb740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50650d5d0 0x55a506508bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:58.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.857+0000 7f37a91bb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a50650eb40 0x55a506510120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:58.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.857+0000 7f37a91bb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a506518370 0x55a506510660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:58.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.857+0000 7f37a91bb740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f37a01b1f60 con 0x55a50650d5d0 2024-09-12T03:13:58.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.857+0000 7f37a91bb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f37a01af830 con 0x55a506518370 2024-09-12T03:13:58.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.857+0000 7f37a91bb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f37a01ad680 con 0x55a50650eb40 2024-09-12T03:13:58.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.857+0000 7f37a537f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50650d5d0 0x55a506508bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:58.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.857+0000 7f37a537f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50650d5d0 0x55a506508bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38780/0 (socket says 172.21.15.116:38780) 2024-09-12T03:13:58.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.857+0000 7f37a537f640 1 -- 172.21.15.116:0/3252520952 learned_addr learned my addr 172.21.15.116:0/3252520952 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:58.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.857+0000 7f37a537f640 1 -- 172.21.15.116:0/3252520952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a50650eb40 msgr2=0x55a506510120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:58.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.857+0000 7f37a537f640 1 --2- 172.21.15.116:0/3252520952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a50650eb40 0x55a506510120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.857+0000 7f37a537f640 1 -- 172.21.15.116:0/3252520952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a506518370 msgr2=0x55a506510660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:58.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.857+0000 7f37a537f640 1 --2- 172.21.15.116:0/3252520952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a506518370 0x55a506510660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.857+0000 7f37a537f640 1 -- 172.21.15.116:0/3252520952 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5065135a0 con 0x55a50650d5d0 2024-09-12T03:13:58.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.857+0000 7f37a537f640 1 --2- 172.21.15.116:0/3252520952 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50650d5d0 0x55a506508bf0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f3798013950 tx=0x7f3798013c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba85935b45073777 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:58.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.857+0000 7f37867fc640 1 -- 172.21.15.116:0/3252520952 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379802e070 con 0x55a50650d5d0 2024-09-12T03:13:58.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.857+0000 7f37867fc640 1 -- 172.21.15.116:0/3252520952 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f379800ecd0 con 0x55a50650d5d0 2024-09-12T03:13:58.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.857+0000 7f37867fc640 1 -- 172.21.15.116:0/3252520952 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379800fc30 con 0x55a50650d5d0 2024-09-12T03:13:58.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.858+0000 7f37a91bb740 1 -- 172.21.15.116:0/3252520952 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50650d5d0 msgr2=0x55a506508bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:58.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.858+0000 7f37a91bb740 1 --2- 172.21.15.116:0/3252520952 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50650d5d0 0x55a506508bf0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f3798013950 tx=0x7f3798013c00 comp rx=0 tx=0).stop 2024-09-12T03:13:58.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.858+0000 7f37a91bb740 1 -- 172.21.15.116:0/3252520952 shutdown_connections 2024-09-12T03:13:58.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.858+0000 7f37a91bb740 1 --2- 172.21.15.116:0/3252520952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a506518370 0x55a506510660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.858+0000 7f37a91bb740 1 --2- 172.21.15.116:0/3252520952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a50650eb40 0x55a506510120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.858+0000 7f37a91bb740 1 --2- 172.21.15.116:0/3252520952 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50650d5d0 0x55a506508bf0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.858+0000 7f37a91bb740 1 -- 172.21.15.116:0/3252520952 >> 172.21.15.116:0/3252520952 conn(0x55a5064d6310 msgr2=0x55a5065084e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:58.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.858+0000 7f37a91bb740 1 -- 172.21.15.116:0/3252520952 shutdown_connections 2024-09-12T03:13:58.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.858+0000 7f37a91bb740 1 -- 172.21.15.116:0/3252520952 wait complete. 2024-09-12T03:13:58.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.858+0000 7f37a91bb740 1 -- start start 2024-09-12T03:13:58.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.858+0000 7f37a91bb740 1 Processor -- start 2024-09-12T03:13:58.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.859+0000 7f37a91bb740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50650d5d0 0x55a506512070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:58.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.859+0000 7f37a91bb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a50650eb40 0x55a5065125b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:58.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.859+0000 7f37a91bb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a506518370 0x55a506512af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:58.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.859+0000 7f37a91bb740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f37a01adcf0 con 0x55a50650d5d0 2024-09-12T03:13:58.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.859+0000 7f37a91bb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f37a01af160 con 0x55a50650eb40 2024-09-12T03:13:58.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.859+0000 7f37a91bb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f37a01ae3c0 con 0x55a506518370 2024-09-12T03:13:58.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.859+0000 7f37a91bb740 10 client.0 _ll_register_callbacks cb 0x7f379800aaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:58.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.859+0000 7f37a537f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50650d5d0 0x55a506512070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:58.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.859+0000 7f37a537f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50650d5d0 0x55a506512070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38788/0 (socket says 172.21.15.116:38788) 2024-09-12T03:13:58.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.859+0000 7f37a537f640 1 -- 172.21.15.116:0/748235433 learned_addr learned my addr 172.21.15.116:0/748235433 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:13:58.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.859+0000 7f37a537f640 1 --2- 172.21.15.116:0/748235433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a506518370 0x55a506512af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:58.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.859+0000 7f37a4b7e640 1 --2- 172.21.15.116:0/748235433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a50650eb40 0x55a5065125b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:58.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.859+0000 7f37a537f640 1 -- 172.21.15.116:0/748235433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a506518370 msgr2=0x55a506512af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:58.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.859+0000 7f37a537f640 1 --2- 172.21.15.116:0/748235433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a506518370 0x55a506512af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.859+0000 7f37a537f640 1 -- 172.21.15.116:0/748235433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a50650eb40 msgr2=0x55a5065125b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:58.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.859+0000 7f37a537f640 1 --2- 172.21.15.116:0/748235433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a50650eb40 0x55a5065125b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.859+0000 7f37a537f640 1 -- 172.21.15.116:0/748235433 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a506591d20 con 0x55a50650d5d0 2024-09-12T03:13:58.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.859+0000 7f37a537f640 1 --2- 172.21.15.116:0/748235433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50650d5d0 0x55a506512070 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f3798005e10 tx=0x7f379800f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6eeaae4a273b018 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:58.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.859+0000 7f377ffff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:13:58.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.859+0000 7f377ffff640 1 -- 172.21.15.116:0/748235433 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379802e040 con 0x55a50650d5d0 2024-09-12T03:13:58.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.859+0000 7f377ffff640 1 -- 172.21.15.116:0/748235433 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f3798005050 con 0x55a50650d5d0 2024-09-12T03:13:58.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.859+0000 7f37a91bb740 1 -- 172.21.15.116:0/748235433 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f379800ecd0 con 0x55a50650d5d0 2024-09-12T03:13:58.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.860+0000 7f377ffff640 1 -- 172.21.15.116:0/748235433 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f379800f810 con 0x55a50650d5d0 2024-09-12T03:13:58.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.860+0000 7f37a91bb740 10 client.18360 fetch_fsmap learned FSMap version 25 2024-09-12T03:13:58.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.860+0000 7f37a91bb740 1 -- 172.21.15.116:0/748235433 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a506593000 con 0x55a50650d5d0 2024-09-12T03:13:58.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.860+0000 7f377ffff640 1 -- 172.21.15.116:0/748235433 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f379800ecd0 con 0x55a50650d5d0 2024-09-12T03:13:58.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.860+0000 7f377ffff640 1 -- 172.21.15.116:0/748235433 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3798005700 con 0x55a50650d5d0 2024-09-12T03:13:58.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.860+0000 7f37a91bb740 10 client.18360 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:13:58.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.860+0000 7f37a91bb740 10 client.18360 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:13:58.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.860+0000 7f37a91bb740 20 client.18360 populate_metadata read hostname 'smithi116' 2024-09-12T03:13:58.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.860+0000 7f37a91bb740 1 --2- 172.21.15.116:0/748235433 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55a506593650 0x55a506595a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:58.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.860+0000 7f37a91bb740 4 client.18360 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:13:58.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.860+0000 7f37a91bb740 1 -- 172.21.15.116:0/748235433 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a50647ad70 con 0x55a506593650 2024-09-12T03:13:58.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.860+0000 7f37a4b7e640 1 --2- 172.21.15.116:0/748235433 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55a506593650 0x55a506595a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:58.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.861+0000 7f377ffff640 10 client.18360 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:13:58.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.861+0000 7f37a4b7e640 1 --2- 172.21.15.116:0/748235433 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55a506593650 0x55a506595a40 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec7c848193d056a5 server_cookie=47389119089b0be1 in_seq=0 out_seq=0 2024-09-12T03:13:58.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.861+0000 7f377ffff640 1 -- 172.21.15.116:0/748235433 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a50647ad70 con 0x55a506593650 2024-09-12T03:13:58.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.861+0000 7f377ffff640 10 client.18360 handle_command_reply: tid=0 2024-09-12T03:13:58.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.861+0000 7f37a91bb740 1 client.18360 shutdown 2024-09-12T03:13:58.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.861+0000 7f37a91bb740 1 -- 172.21.15.116:0/748235433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50650d5d0 msgr2=0x55a506512070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:58.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.861+0000 7f37a91bb740 1 --2- 172.21.15.116:0/748235433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50650d5d0 0x55a506512070 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f3798005e10 tx=0x7f379800f770 comp rx=0 tx=0).stop 2024-09-12T03:13:58.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.861+0000 7f37a91bb740 1 -- 172.21.15.116:0/748235433 shutdown_connections 2024-09-12T03:13:58.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.861+0000 7f37a91bb740 1 --2- 172.21.15.116:0/748235433 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55a506593650 0x55a506595a40 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.861+0000 7f37a91bb740 1 --2- 172.21.15.116:0/748235433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a506518370 0x55a506512af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.861+0000 7f37a91bb740 1 --2- 172.21.15.116:0/748235433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a50650eb40 0x55a5065125b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.861+0000 7f37a91bb740 1 --2- 172.21.15.116:0/748235433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50650d5d0 0x55a506512070 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.861+0000 7f37a91bb740 1 -- 172.21.15.116:0/748235433 >> 172.21.15.116:0/748235433 conn(0x55a5064d6310 msgr2=0x55a506508d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:58.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.861+0000 7f377ffff640 0 client.18360 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:13:58.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.861+0000 7f37a91bb740 1 -- 172.21.15.116:0/748235433 shutdown_connections 2024-09-12T03:13:58.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.861+0000 7f37a91bb740 1 -- 172.21.15.116:0/748235433 wait complete. 2024-09-12T03:13:58.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.861+0000 7f37a91bb740 20 client.18360 trim_cache size 0 max 16384 2024-09-12T03:13:58.879 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:13:58.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.862+0000 7f37a6de2640 1 -- 172.21.15.116:0/2519394679 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f37880a2b80 msgr2=0x7f37880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:58.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.862+0000 7f37a6de2640 1 --2- 172.21.15.116:0/2519394679 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f37880a2b80 0x7f37880a5040 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f379004a000 tx=0x7f37900b1350 comp rx=0 tx=0).stop 2024-09-12T03:13:58.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.862+0000 7f37a6de2640 1 -- 172.21.15.116:0/2519394679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37a01036c0 msgr2=0x7f37a0100f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:58.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.862+0000 7f37a6de2640 1 --2- 172.21.15.116:0/2519394679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37a01036c0 0x7f37a0100f40 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f3790006010 tx=0x7f37900042f0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.862+0000 7f37a6de2640 1 -- 172.21.15.116:0/2519394679 shutdown_connections 2024-09-12T03:13:58.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.862+0000 7f37a6de2640 1 --2- 172.21.15.116:0/2519394679 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f37880a2b80 0x7f37880a5040 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.862+0000 7f37a6de2640 1 --2- 172.21.15.116:0/2519394679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37a010c180 0x7f37a0101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.862+0000 7f37a6de2640 1 --2- 172.21.15.116:0/2519394679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37a01036c0 0x7f37a0100f40 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.862+0000 7f37a6de2640 1 --2- 172.21.15.116:0/2519394679 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37a0102da0 0x7f37a0100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:58.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.862+0000 7f37a6de2640 1 -- 172.21.15.116:0/2519394679 >> 172.21.15.116:0/2519394679 conn(0x7f37a00fc7a0 msgr2=0x7f37a00fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:58.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.862+0000 7f37a6de2640 1 -- 172.21.15.116:0/2519394679 shutdown_connections 2024-09-12T03:13:58.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:13:58.862+0000 7f37a6de2640 1 -- 172.21.15.116:0/2519394679 wait complete. 2024-09-12T03:13:59.541 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:13:59 smithi116 ceph-mon[31410]: pgmap v668: 97 pgs: 97 active+clean; 5.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:13:59.569 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:13:59.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:13:59 smithi183 ceph-mon[35535]: pgmap v668: 97 pgs: 97 active+clean; 5.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:13:59.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:13:59 smithi177 ceph-mon[35305]: pgmap v668: 97 pgs: 97 active+clean; 5.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:13:59.815 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:14:00.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.349+0000 7fe291862640 1 -- 172.21.15.116:0/3653098819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe28c105ba0 msgr2=0x7fe28c105f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:00.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.349+0000 7fe291862640 1 --2- 172.21.15.116:0/3653098819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe28c105ba0 0x7fe28c105f80 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fe27c009900 tx=0x7fe27c02a480 comp rx=0 tx=0).stop 2024-09-12T03:14:00.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.349+0000 7fe291862640 1 -- 172.21.15.116:0/3653098819 shutdown_connections 2024-09-12T03:14:00.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.349+0000 7fe291862640 1 --2- 172.21.15.116:0/3653098819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe28c102110 0x7fe28c10d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.349+0000 7fe291862640 1 --2- 172.21.15.116:0/3653098819 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe28c101790 0x7fe28c101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.349+0000 7fe291862640 1 --2- 172.21.15.116:0/3653098819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe28c105ba0 0x7fe28c105f80 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.349+0000 7fe291862640 1 -- 172.21.15.116:0/3653098819 >> 172.21.15.116:0/3653098819 conn(0x7fe28c077210 msgr2=0x7fe28c077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:00.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.349+0000 7fe291862640 1 -- 172.21.15.116:0/3653098819 shutdown_connections 2024-09-12T03:14:00.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.349+0000 7fe291862640 1 -- 172.21.15.116:0/3653098819 wait complete. 2024-09-12T03:14:00.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.350+0000 7fe291862640 1 Processor -- start 2024-09-12T03:14:00.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.350+0000 7fe291862640 1 -- start start 2024-09-12T03:14:00.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.350+0000 7fe291862640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe28c101790 0x7fe28c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:00.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.350+0000 7fe291862640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe28c102110 0x7fe28c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:00.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.350+0000 7fe291862640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe28c105ba0 0x7fe28c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:00.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.350+0000 7fe291862640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c111f80 con 0x7fe28c105ba0 2024-09-12T03:14:00.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.350+0000 7fe291862640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c111e00 con 0x7fe28c101790 2024-09-12T03:14:00.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.350+0000 7fe291862640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c112100 con 0x7fe28c102110 2024-09-12T03:14:00.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.350+0000 7fe28b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe28c105ba0 0x7fe28c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:00.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.350+0000 7fe28b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe28c105ba0 0x7fe28c19cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38794/0 (socket says 172.21.15.116:38794) 2024-09-12T03:14:00.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.350+0000 7fe28b7fe640 1 -- 172.21.15.116:0/1780547048 learned_addr learned my addr 172.21.15.116:0/1780547048 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:00.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.351+0000 7fe2827fc640 1 --2- 172.21.15.116:0/1780547048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe28c102110 0x7fe28c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:00.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.351+0000 7fe28affd640 1 --2- 172.21.15.116:0/1780547048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe28c101790 0x7fe28c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:00.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.351+0000 7fe28b7fe640 1 -- 172.21.15.116:0/1780547048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe28c102110 msgr2=0x7fe28c1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:00.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.351+0000 7fe28b7fe640 1 --2- 172.21.15.116:0/1780547048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe28c102110 0x7fe28c1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.351+0000 7fe28b7fe640 1 -- 172.21.15.116:0/1780547048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe28c101790 msgr2=0x7fe28c198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:00.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.351+0000 7fe28b7fe640 1 --2- 172.21.15.116:0/1780547048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe28c101790 0x7fe28c198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.351+0000 7fe28b7fe640 1 -- 172.21.15.116:0/1780547048 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe28c19d320 con 0x7fe28c105ba0 2024-09-12T03:14:00.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.351+0000 7fe28b7fe640 1 --2- 172.21.15.116:0/1780547048 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe28c105ba0 0x7fe28c19cc40 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fe284009fd0 tx=0x7fe28400def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:00.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.351+0000 7fe290860640 1 -- 172.21.15.116:0/1780547048 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe284017070 con 0x7fe28c105ba0 2024-09-12T03:14:00.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.351+0000 7fe291862640 1 -- 172.21.15.116:0/1780547048 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe28c19d610 con 0x7fe28c105ba0 2024-09-12T03:14:00.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.351+0000 7fe290860640 1 -- 172.21.15.116:0/1780547048 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe284003e50 con 0x7fe28c105ba0 2024-09-12T03:14:00.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.351+0000 7fe290860640 1 -- 172.21.15.116:0/1780547048 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe284010460 con 0x7fe28c105ba0 2024-09-12T03:14:00.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.351+0000 7fe291862640 1 -- 172.21.15.116:0/1780547048 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe28c1a4ef0 con 0x7fe28c105ba0 2024-09-12T03:14:00.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.353+0000 7fe290860640 1 -- 172.21.15.116:0/1780547048 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe284003970 con 0x7fe28c105ba0 2024-09-12T03:14:00.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.354+0000 7fe293c3b740 1 Processor -- start 2024-09-12T03:14:00.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.354+0000 7fe293c3b740 1 -- start start 2024-09-12T03:14:00.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.354+0000 7fe290860640 1 --2- 172.21.15.116:0/1780547048 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe2780a2b80 0x7fe2780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:00.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.354+0000 7fe293c3b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56281735cb40 0x56281735b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:00.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.354+0000 7fe293c3b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56281735b5d0 0x56281735ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:00.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.354+0000 7fe293c3b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56281736a780 0x56281736cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:00.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.354+0000 7fe293c3b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5628172c12f0 con 0x56281735b5d0 2024-09-12T03:14:00.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.354+0000 7fe293c3b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562817268cf0 con 0x56281735cb40 2024-09-12T03:14:00.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.354+0000 7fe293c3b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5628172bd2b0 con 0x56281736a780 2024-09-12T03:14:00.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.354+0000 7fe290860640 1 -- 172.21.15.116:0/1780547048 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe2840ced70 con 0x7fe28c105ba0 2024-09-12T03:14:00.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.354+0000 7fe2827fc640 1 --2- 172.21.15.116:0/1780547048 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe2780a2b80 0x7fe2780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:00.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.354+0000 7fe2827fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56281735b5d0 0x56281735ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:00.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.354+0000 7fe2827fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56281735b5d0 0x56281735ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38804/0 (socket says 172.21.15.116:38804) 2024-09-12T03:14:00.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.354+0000 7fe28b7fe640 1 --2- 172.21.15.116:0/2242581756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56281736a780 0x56281736cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:00.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.354+0000 7fe2827fc640 1 -- 172.21.15.116:0/2242581756 learned_addr learned my addr 172.21.15.116:0/2242581756 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:00.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.354+0000 7fe28affd640 1 --2- 172.21.15.116:0/2242581756 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56281735cb40 0x56281735b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:00.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.355+0000 7fe28b7fe640 1 -- 172.21.15.116:0/2242581756 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56281735cb40 msgr2=0x56281735b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:00.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.355+0000 7fe28b7fe640 1 --2- 172.21.15.116:0/2242581756 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56281735cb40 0x56281735b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.355+0000 7fe28b7fe640 1 -- 172.21.15.116:0/2242581756 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56281735b5d0 msgr2=0x56281735ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:00.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.355+0000 7fe28b7fe640 1 --2- 172.21.15.116:0/2242581756 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56281735b5d0 0x56281735ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.355+0000 7fe28b7fe640 1 -- 172.21.15.116:0/2242581756 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56281724d3a0 con 0x56281736a780 2024-09-12T03:14:00.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.355+0000 7fe2827fc640 1 --2- 172.21.15.116:0/1780547048 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe2780a2b80 0x7fe2780a5040 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe2740119d0 tx=0x7fe27400ab90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:00.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.355+0000 7fe28b7fe640 1 --2- 172.21.15.116:0/2242581756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56281736a780 0x56281736cb70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe28401cfd0 tx=0x7fe28409b1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4f223d15655a7eb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:00.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.355+0000 7fe281ffb640 1 -- 172.21.15.116:0/2242581756 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe284020050 con 0x56281736a780 2024-09-12T03:14:00.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.355+0000 7fe281ffb640 1 -- 172.21.15.116:0/2242581756 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe28409f950 con 0x56281736a780 2024-09-12T03:14:00.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.355+0000 7fe281ffb640 1 -- 172.21.15.116:0/2242581756 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe284087320 con 0x56281736a780 2024-09-12T03:14:00.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.356+0000 7fe293c3b740 1 -- 172.21.15.116:0/2242581756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56281736a780 msgr2=0x56281736cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:00.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.356+0000 7fe293c3b740 1 --2- 172.21.15.116:0/2242581756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56281736a780 0x56281736cb70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe28401cfd0 tx=0x7fe28409b1a0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.356+0000 7fe293c3b740 1 -- 172.21.15.116:0/2242581756 shutdown_connections 2024-09-12T03:14:00.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.356+0000 7fe293c3b740 1 --2- 172.21.15.116:0/2242581756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56281736a780 0x56281736cb70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.356+0000 7fe293c3b740 1 --2- 172.21.15.116:0/2242581756 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56281735b5d0 0x56281735ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.356+0000 7fe293c3b740 1 --2- 172.21.15.116:0/2242581756 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56281735cb40 0x56281735b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.356+0000 7fe293c3b740 1 -- 172.21.15.116:0/2242581756 >> 172.21.15.116:0/2242581756 conn(0x562817324310 msgr2=0x562817357c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:00.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.356+0000 7fe293c3b740 1 -- 172.21.15.116:0/2242581756 shutdown_connections 2024-09-12T03:14:00.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.356+0000 7fe293c3b740 1 -- 172.21.15.116:0/2242581756 wait complete. 2024-09-12T03:14:00.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.356+0000 7fe293c3b740 1 -- start start 2024-09-12T03:14:00.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.356+0000 7fe293c3b740 1 Processor -- start 2024-09-12T03:14:00.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.357+0000 7fe293c3b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56281735b5d0 0x56281736a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:00.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.357+0000 7fe293c3b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56281735cb40 0x562817367480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:00.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.357+0000 7fe293c3b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56281736a780 0x562817367c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:00.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.357+0000 7fe293c3b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56281736d420 con 0x56281735b5d0 2024-09-12T03:14:00.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.357+0000 7fe293c3b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56281727c060 con 0x56281735cb40 2024-09-12T03:14:00.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.357+0000 7fe293c3b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56281736d5a0 con 0x56281736a780 2024-09-12T03:14:00.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.357+0000 7fe293c3b740 10 client.0 _ll_register_callbacks cb 0x562816fcf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:00.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.357+0000 7fe28affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56281735b5d0 0x56281736a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:00.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.357+0000 7fe28affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56281735b5d0 0x56281736a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38810/0 (socket says 172.21.15.116:38810) 2024-09-12T03:14:00.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.357+0000 7fe28affd640 1 -- 172.21.15.116:0/3964555 learned_addr learned my addr 172.21.15.116:0/3964555 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:00.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.357+0000 7fe28b7fe640 1 --2- 172.21.15.116:0/3964555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56281736a780 0x562817367c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:00.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.357+0000 7fe2827fc640 1 --2- 172.21.15.116:0/3964555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56281735cb40 0x562817367480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:00.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.357+0000 7fe28affd640 1 -- 172.21.15.116:0/3964555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56281736a780 msgr2=0x562817367c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:00.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.357+0000 7fe28affd640 1 --2- 172.21.15.116:0/3964555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56281736a780 0x562817367c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.357+0000 7fe28affd640 1 -- 172.21.15.116:0/3964555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56281735cb40 msgr2=0x562817367480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:00.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.357+0000 7fe28affd640 1 --2- 172.21.15.116:0/3964555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56281735cb40 0x562817367480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.357+0000 7fe28affd640 1 -- 172.21.15.116:0/3964555 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628170323c0 con 0x56281735b5d0 2024-09-12T03:14:00.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.357+0000 7fe28affd640 1 --2- 172.21.15.116:0/3964555 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56281735b5d0 0x56281736a1d0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fe27c02a9a0 tx=0x7fe27c005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeebe06ac409257e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:00.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.357+0000 7fe26affd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:14:00.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.358+0000 7fe26affd640 1 -- 172.21.15.116:0/3964555 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c035040 con 0x56281735b5d0 2024-09-12T03:14:00.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.358+0000 7fe26affd640 1 -- 172.21.15.116:0/3964555 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe27c0048c0 con 0x56281735b5d0 2024-09-12T03:14:00.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.358+0000 7fe293c3b740 1 -- 172.21.15.116:0/3964555 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe28409f950 con 0x56281735b5d0 2024-09-12T03:14:00.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.358+0000 7fe26affd640 1 -- 172.21.15.116:0/3964555 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe27c004ac0 con 0x56281735b5d0 2024-09-12T03:14:00.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.358+0000 7fe293c3b740 10 client.18372 fetch_fsmap learned FSMap version 25 2024-09-12T03:14:00.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.358+0000 7fe293c3b740 1 -- 172.21.15.116:0/3964555 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562817378f30 con 0x56281735b5d0 2024-09-12T03:14:00.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.358+0000 7fe26affd640 1 -- 172.21.15.116:0/3964555 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe28409f950 con 0x56281735b5d0 2024-09-12T03:14:00.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.358+0000 7fe26affd640 1 -- 172.21.15.116:0/3964555 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe27c031540 con 0x56281735b5d0 2024-09-12T03:14:00.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.358+0000 7fe293c3b740 10 client.18372 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:14:00.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.358+0000 7fe293c3b740 10 client.18372 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:14:00.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.358+0000 7fe293c3b740 20 client.18372 populate_metadata read hostname 'smithi116' 2024-09-12T03:14:00.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.358+0000 7fe293c3b740 1 --2- 172.21.15.116:0/3964555 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56281737a3f0 0x56281737c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:00.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.358+0000 7fe293c3b740 4 client.18372 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:14:00.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.358+0000 7fe293c3b740 1 -- 172.21.15.116:0/3964555 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628172ce860 con 0x56281737a3f0 2024-09-12T03:14:00.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.359+0000 7fe2827fc640 1 --2- 172.21.15.116:0/3964555 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56281737a3f0 0x56281737c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:00.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.359+0000 7fe26affd640 10 client.18372 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:00.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.359+0000 7fe2827fc640 1 --2- 172.21.15.116:0/3964555 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56281737a3f0 0x56281737c850 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=479869102fd2dea9 server_cookie=5cac1742d9a76806 in_seq=0 out_seq=0 2024-09-12T03:14:00.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.360+0000 7fe26affd640 1 -- 172.21.15.116:0/3964555 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5628172ce860 con 0x56281737a3f0 2024-09-12T03:14:00.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.360+0000 7fe26affd640 10 client.18372 handle_command_reply: tid=0 2024-09-12T03:14:00.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.360+0000 7fe293c3b740 1 client.18372 shutdown 2024-09-12T03:14:00.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.360+0000 7fe293c3b740 1 -- 172.21.15.116:0/3964555 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56281735b5d0 msgr2=0x56281736a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:00.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.360+0000 7fe293c3b740 1 --2- 172.21.15.116:0/3964555 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56281735b5d0 0x56281736a1d0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fe27c02a9a0 tx=0x7fe27c005250 comp rx=0 tx=0).stop 2024-09-12T03:14:00.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.360+0000 7fe293c3b740 1 -- 172.21.15.116:0/3964555 shutdown_connections 2024-09-12T03:14:00.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.360+0000 7fe293c3b740 1 --2- 172.21.15.116:0/3964555 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56281737a3f0 0x56281737c850 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.360+0000 7fe293c3b740 1 --2- 172.21.15.116:0/3964555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56281736a780 0x562817367c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.360+0000 7fe293c3b740 1 --2- 172.21.15.116:0/3964555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56281735cb40 0x562817367480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.360+0000 7fe293c3b740 1 --2- 172.21.15.116:0/3964555 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56281735b5d0 0x56281736a1d0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.360+0000 7fe293c3b740 1 -- 172.21.15.116:0/3964555 >> 172.21.15.116:0/3964555 conn(0x562817324310 msgr2=0x56281735d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:00.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.360+0000 7fe26affd640 0 client.18372 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:00.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.360+0000 7fe293c3b740 1 -- 172.21.15.116:0/3964555 shutdown_connections 2024-09-12T03:14:00.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.360+0000 7fe293c3b740 1 -- 172.21.15.116:0/3964555 wait complete. 2024-09-12T03:14:00.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.360+0000 7fe293c3b740 20 client.18372 trim_cache size 0 max 16384 2024-09-12T03:14:00.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.373+0000 7fe293c3b740 1 Processor -- start 2024-09-12T03:14:00.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.373+0000 7fe293c3b740 1 -- start start 2024-09-12T03:14:00.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.373+0000 7fe293c3b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56281735b5d0 0x562817357e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:00.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.373+0000 7fe293c3b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56281735cb40 0x562817382120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:00.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.373+0000 7fe293c3b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56281736a780 0x562817382660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:00.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.373+0000 7fe293c3b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c1ada70 con 0x56281736a780 2024-09-12T03:14:00.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.373+0000 7fe293c3b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c1ab340 con 0x56281735b5d0 2024-09-12T03:14:00.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.373+0000 7fe293c3b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c1a9190 con 0x56281735cb40 2024-09-12T03:14:00.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.373+0000 7fe28b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56281736a780 0x562817382660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:00.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.373+0000 7fe28b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56281736a780 0x562817382660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38816/0 (socket says 172.21.15.116:38816) 2024-09-12T03:14:00.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.373+0000 7fe28b7fe640 1 -- 172.21.15.116:0/2223216257 learned_addr learned my addr 172.21.15.116:0/2223216257 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:00.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.373+0000 7fe28affd640 1 --2- 172.21.15.116:0/2223216257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56281735b5d0 0x562817357e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:00.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.373+0000 7fe2827fc640 1 --2- 172.21.15.116:0/2223216257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56281735cb40 0x562817382120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:00.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.373+0000 7fe28b7fe640 1 -- 172.21.15.116:0/2223216257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56281735cb40 msgr2=0x562817382120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:00.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.373+0000 7fe28b7fe640 1 --2- 172.21.15.116:0/2223216257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56281735cb40 0x562817382120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.373+0000 7fe28b7fe640 1 -- 172.21.15.116:0/2223216257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56281735b5d0 msgr2=0x562817357e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:00.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.373+0000 7fe28b7fe640 1 --2- 172.21.15.116:0/2223216257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56281735b5d0 0x562817357e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.373+0000 7fe28b7fe640 1 -- 172.21.15.116:0/2223216257 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56281735c780 con 0x56281736a780 2024-09-12T03:14:00.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.373+0000 7fe2827fc640 1 --2- 172.21.15.116:0/2223216257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56281735cb40 0x562817382120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:14:00.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.374+0000 7fe28b7fe640 1 --2- 172.21.15.116:0/2223216257 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56281736a780 0x562817382660 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fe28409be50 tx=0x7fe284086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f565f2a81ac0ca5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:00.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.374+0000 7fe2817fa640 1 -- 172.21.15.116:0/2223216257 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2840b4070 con 0x56281736a780 2024-09-12T03:14:00.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.374+0000 7fe2817fa640 1 -- 172.21.15.116:0/2223216257 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2840a4210 con 0x56281736a780 2024-09-12T03:14:00.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.374+0000 7fe2817fa640 1 -- 172.21.15.116:0/2223216257 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe284020040 con 0x56281736a780 2024-09-12T03:14:00.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.374+0000 7fe293c3b740 1 -- 172.21.15.116:0/2223216257 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56281736a780 msgr2=0x562817382660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:00.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.374+0000 7fe293c3b740 1 --2- 172.21.15.116:0/2223216257 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56281736a780 0x562817382660 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fe28409be50 tx=0x7fe284086040 comp rx=0 tx=0).stop 2024-09-12T03:14:00.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.374+0000 7fe293c3b740 1 -- 172.21.15.116:0/2223216257 shutdown_connections 2024-09-12T03:14:00.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.374+0000 7fe293c3b740 1 --2- 172.21.15.116:0/2223216257 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56281736a780 0x562817382660 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.374+0000 7fe293c3b740 1 --2- 172.21.15.116:0/2223216257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56281735cb40 0x562817382120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.374+0000 7fe293c3b740 1 --2- 172.21.15.116:0/2223216257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56281735b5d0 0x562817357e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.374+0000 7fe293c3b740 1 -- 172.21.15.116:0/2223216257 >> 172.21.15.116:0/2223216257 conn(0x562817324310 msgr2=0x562817357700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:00.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.374+0000 7fe293c3b740 1 -- 172.21.15.116:0/2223216257 shutdown_connections 2024-09-12T03:14:00.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.375+0000 7fe293c3b740 1 -- 172.21.15.116:0/2223216257 wait complete. 2024-09-12T03:14:00.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.375+0000 7fe293c3b740 1 -- start start 2024-09-12T03:14:00.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.375+0000 7fe293c3b740 1 Processor -- start 2024-09-12T03:14:00.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.375+0000 7fe293c3b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56281735b5d0 0x562817372d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:00.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.375+0000 7fe293c3b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56281735cb40 0x56281736e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:00.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.375+0000 7fe293c3b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56281736a780 0x56281736e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:00.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.375+0000 7fe293c3b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c1a9800 con 0x56281736a780 2024-09-12T03:14:00.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.375+0000 7fe293c3b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c1aac70 con 0x56281735cb40 2024-09-12T03:14:00.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.375+0000 7fe293c3b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c1a9ed0 con 0x56281735b5d0 2024-09-12T03:14:00.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.375+0000 7fe293c3b740 10 client.0 _ll_register_callbacks cb 0x7fe27c004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:00.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.375+0000 7fe28b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56281736a780 0x56281736e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:00.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.375+0000 7fe28b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56281736a780 0x56281736e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:38818/0 (socket says 172.21.15.116:38818) 2024-09-12T03:14:00.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.375+0000 7fe28b7fe640 1 -- 172.21.15.116:0/1912298842 learned_addr learned my addr 172.21.15.116:0/1912298842 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:00.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.376+0000 7fe2827fc640 1 --2- 172.21.15.116:0/1912298842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56281735cb40 0x56281736e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:00.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.376+0000 7fe28affd640 1 --2- 172.21.15.116:0/1912298842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56281735b5d0 0x562817372d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:00.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.376+0000 7fe28b7fe640 1 -- 172.21.15.116:0/1912298842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56281735b5d0 msgr2=0x562817372d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:00.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.376+0000 7fe28b7fe640 1 --2- 172.21.15.116:0/1912298842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56281735b5d0 0x562817372d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.376+0000 7fe28b7fe640 1 -- 172.21.15.116:0/1912298842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56281735cb40 msgr2=0x56281736e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:00.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.376+0000 7fe28b7fe640 1 --2- 172.21.15.116:0/1912298842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56281735cb40 0x56281736e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.376+0000 7fe28b7fe640 1 -- 172.21.15.116:0/1912298842 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56281736eb50 con 0x56281736a780 2024-09-12T03:14:00.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.376+0000 7fe28affd640 1 --2- 172.21.15.116:0/1912298842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56281735b5d0 0x562817372d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:14:00.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.376+0000 7fe2827fc640 1 --2- 172.21.15.116:0/1912298842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56281735cb40 0x56281736e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:00.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.376+0000 7fe28b7fe640 1 --2- 172.21.15.116:0/1912298842 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56281736a780 0x56281736e590 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fe28400b6f0 tx=0x7fe28400bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7663a9e53a2125 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:00.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.376+0000 7fe26affd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:14:00.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.376+0000 7fe26affd640 1 -- 172.21.15.116:0/1912298842 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe284014070 con 0x56281736a780 2024-09-12T03:14:00.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.376+0000 7fe26affd640 1 -- 172.21.15.116:0/1912298842 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe284097c10 con 0x56281736a780 2024-09-12T03:14:00.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.376+0000 7fe293c3b740 1 -- 172.21.15.116:0/1912298842 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2840a4210 con 0x56281736a780 2024-09-12T03:14:00.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.376+0000 7fe26affd640 1 -- 172.21.15.116:0/1912298842 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe2840b4040 con 0x56281736a780 2024-09-12T03:14:00.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.376+0000 7fe293c3b740 10 client.18378 fetch_fsmap learned FSMap version 25 2024-09-12T03:14:00.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.376+0000 7fe293c3b740 1 -- 172.21.15.116:0/1912298842 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5628173e4df0 con 0x56281736a780 2024-09-12T03:14:00.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.376+0000 7fe26affd640 1 -- 172.21.15.116:0/1912298842 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2840a4210 con 0x56281736a780 2024-09-12T03:14:00.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.376+0000 7fe26affd640 1 -- 172.21.15.116:0/1912298842 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe284084b30 con 0x56281736a780 2024-09-12T03:14:00.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.377+0000 7fe293c3b740 10 client.18378 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:14:00.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.377+0000 7fe293c3b740 10 client.18378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:14:00.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.377+0000 7fe293c3b740 20 client.18378 populate_metadata read hostname 'smithi116' 2024-09-12T03:14:00.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.377+0000 7fe293c3b740 1 --2- 172.21.15.116:0/1912298842 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5628173e5020 0x5628173e7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:00.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.377+0000 7fe293c3b740 4 client.18378 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:14:00.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.377+0000 7fe293c3b740 1 -- 172.21.15.116:0/1912298842 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5628172c8d70 con 0x5628173e5020 2024-09-12T03:14:00.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.377+0000 7fe28affd640 1 --2- 172.21.15.116:0/1912298842 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5628173e5020 0x5628173e7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:00.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.377+0000 7fe26affd640 10 client.18378 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:00.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.377+0000 7fe28affd640 1 --2- 172.21.15.116:0/1912298842 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5628173e5020 0x5628173e7410 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc571c01048569ab server_cookie=35db221d33cdaf92 in_seq=0 out_seq=0 2024-09-12T03:14:00.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.377+0000 7fe26affd640 1 -- 172.21.15.116:0/1912298842 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5628172c8d70 con 0x5628173e5020 2024-09-12T03:14:00.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.377+0000 7fe26affd640 10 client.18378 handle_command_reply: tid=0 2024-09-12T03:14:00.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.377+0000 7fe293c3b740 1 client.18378 shutdown 2024-09-12T03:14:00.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.378+0000 7fe293c3b740 1 -- 172.21.15.116:0/1912298842 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56281736a780 msgr2=0x56281736e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:00.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.378+0000 7fe293c3b740 1 --2- 172.21.15.116:0/1912298842 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56281736a780 0x56281736e590 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fe28400b6f0 tx=0x7fe28400bdd0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.378+0000 7fe293c3b740 1 -- 172.21.15.116:0/1912298842 shutdown_connections 2024-09-12T03:14:00.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.378+0000 7fe293c3b740 1 --2- 172.21.15.116:0/1912298842 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5628173e5020 0x5628173e7410 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.378+0000 7fe293c3b740 1 --2- 172.21.15.116:0/1912298842 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56281736a780 0x56281736e590 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.378+0000 7fe293c3b740 1 --2- 172.21.15.116:0/1912298842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56281735cb40 0x56281736e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.378+0000 7fe293c3b740 1 --2- 172.21.15.116:0/1912298842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56281735b5d0 0x562817372d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.378+0000 7fe293c3b740 1 -- 172.21.15.116:0/1912298842 >> 172.21.15.116:0/1912298842 conn(0x562817324310 msgr2=0x5628173564e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:00.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.378+0000 7fe26affd640 0 client.18378 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:00.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.378+0000 7fe293c3b740 1 -- 172.21.15.116:0/1912298842 shutdown_connections 2024-09-12T03:14:00.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.378+0000 7fe293c3b740 1 -- 172.21.15.116:0/1912298842 wait complete. 2024-09-12T03:14:00.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.378+0000 7fe293c3b740 20 client.18378 trim_cache size 0 max 16384 2024-09-12T03:14:00.388 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:14:00.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.378+0000 7fe291862640 1 -- 172.21.15.116:0/1780547048 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe2780a2b80 msgr2=0x7fe2780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:00.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.378+0000 7fe291862640 1 --2- 172.21.15.116:0/1780547048 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe2780a2b80 0x7fe2780a5040 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe2740119d0 tx=0x7fe27400ab90 comp rx=0 tx=0).stop 2024-09-12T03:14:00.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.378+0000 7fe291862640 1 -- 172.21.15.116:0/1780547048 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe28c105ba0 msgr2=0x7fe28c19cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:00.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.378+0000 7fe291862640 1 --2- 172.21.15.116:0/1780547048 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe28c105ba0 0x7fe28c19cc40 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fe284009fd0 tx=0x7fe28400def0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.378+0000 7fe291862640 1 -- 172.21.15.116:0/1780547048 shutdown_connections 2024-09-12T03:14:00.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.378+0000 7fe291862640 1 --2- 172.21.15.116:0/1780547048 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe2780a2b80 0x7fe2780a5040 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.378+0000 7fe291862640 1 --2- 172.21.15.116:0/1780547048 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe28c105ba0 0x7fe28c19cc40 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.378+0000 7fe291862640 1 --2- 172.21.15.116:0/1780547048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe28c102110 0x7fe28c1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.378+0000 7fe291862640 1 --2- 172.21.15.116:0/1780547048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe28c101790 0x7fe28c198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:00.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.378+0000 7fe291862640 1 -- 172.21.15.116:0/1780547048 >> 172.21.15.116:0/1780547048 conn(0x7fe28c077210 msgr2=0x7fe28c0ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:00.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.379+0000 7fe291862640 1 -- 172.21.15.116:0/1780547048 shutdown_connections 2024-09-12T03:14:00.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:00.379+0000 7fe291862640 1 -- 172.21.15.116:0/1780547048 wait complete. 2024-09-12T03:14:01.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:01 smithi183 ceph-mon[35535]: pgmap v669: 97 pgs: 97 active+clean; 5.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:14:01.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:01 smithi177 ceph-mon[35305]: pgmap v669: 97 pgs: 97 active+clean; 5.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:14:01.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:01 smithi116 ceph-mon[31410]: pgmap v669: 97 pgs: 97 active+clean; 5.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:14:02.100 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:14:02.100 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:14:02.359 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:14:02.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.851+0000 7effb9b02640 1 -- 172.21.15.116:0/1916694615 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7effb4106f10 msgr2=0x7effb410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:02.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.851+0000 7effb9b02640 1 --2- 172.21.15.116:0/1916694615 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7effb4106f10 0x7effb410ea90 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7effac009ca0 tx=0x7effac02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.851+0000 7effb9b02640 1 -- 172.21.15.116:0/1916694615 shutdown_connections 2024-09-12T03:14:02.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.851+0000 7effb9b02640 1 --2- 172.21.15.116:0/1916694615 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7effb4106f10 0x7effb410ea90 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.851+0000 7effb9b02640 1 --2- 172.21.15.116:0/1916694615 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effb4102790 0x7effb41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.851+0000 7effb9b02640 1 --2- 172.21.15.116:0/1916694615 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effb4101de0 0x7effb41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.851+0000 7effb9b02640 1 -- 172.21.15.116:0/1916694615 >> 172.21.15.116:0/1916694615 conn(0x7effb40fd8c0 msgr2=0x7effb40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.851+0000 7effb9b02640 1 -- 172.21.15.116:0/1916694615 shutdown_connections 2024-09-12T03:14:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.851+0000 7effb9b02640 1 -- 172.21.15.116:0/1916694615 wait complete. 2024-09-12T03:14:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.852+0000 7effb9b02640 1 Processor -- start 2024-09-12T03:14:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.852+0000 7effb9b02640 1 -- start start 2024-09-12T03:14:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.852+0000 7effb9b02640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effb4101de0 0x7effb41985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.852+0000 7effb9b02640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7effb4102790 0x7effb4198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.852+0000 7effb9b02640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effb4106f10 0x7effb419cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.852+0000 7effb9b02640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7effb4112010 con 0x7effb4102790 2024-09-12T03:14:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.852+0000 7effb9b02640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7effb4111e90 con 0x7effb4106f10 2024-09-12T03:14:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.852+0000 7effb9b02640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7effb4112190 con 0x7effb4101de0 2024-09-12T03:14:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.852+0000 7effb2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7effb4102790 0x7effb4198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.852+0000 7effb2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7effb4102790 0x7effb4198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34230/0 (socket says 172.21.15.116:34230) 2024-09-12T03:14:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.852+0000 7effb2ffd640 1 -- 172.21.15.116:0/3864886146 learned_addr learned my addr 172.21.15.116:0/3864886146 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.852+0000 7effb37fe640 1 --2- 172.21.15.116:0/3864886146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effb4101de0 0x7effb41985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.852+0000 7effb3fff640 1 --2- 172.21.15.116:0/3864886146 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effb4106f10 0x7effb419cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.852+0000 7effb2ffd640 1 -- 172.21.15.116:0/3864886146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effb4101de0 msgr2=0x7effb41985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.852+0000 7effb2ffd640 1 --2- 172.21.15.116:0/3864886146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effb4101de0 0x7effb41985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.852+0000 7effb2ffd640 1 -- 172.21.15.116:0/3864886146 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effb4106f10 msgr2=0x7effb419cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.852+0000 7effb2ffd640 1 --2- 172.21.15.116:0/3864886146 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effb4106f10 0x7effb419cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.852+0000 7effb2ffd640 1 -- 172.21.15.116:0/3864886146 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb419d5a0 con 0x7effb4102790 2024-09-12T03:14:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.852+0000 7effb37fe640 1 --2- 172.21.15.116:0/3864886146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effb4101de0 0x7effb41985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.853+0000 7effb2ffd640 1 --2- 172.21.15.116:0/3864886146 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7effb4102790 0x7effb4198b30 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7effa800b4a0 tx=0x7effa800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.853+0000 7effb0ff9640 1 -- 172.21.15.116:0/3864886146 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa8014070 con 0x7effb4102790 2024-09-12T03:14:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.853+0000 7effb0ff9640 1 -- 172.21.15.116:0/3864886146 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effa8010af0 con 0x7effb4102790 2024-09-12T03:14:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.853+0000 7effb9b02640 1 -- 172.21.15.116:0/3864886146 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effb419d890 con 0x7effb4102790 2024-09-12T03:14:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.853+0000 7effb0ff9640 1 -- 172.21.15.116:0/3864886146 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa8017d10 con 0x7effb4102790 2024-09-12T03:14:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.853+0000 7effb9b02640 1 -- 172.21.15.116:0/3864886146 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effb41a5170 con 0x7effb4102790 2024-09-12T03:14:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.855+0000 7effb0ff9640 1 -- 172.21.15.116:0/3864886146 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7effa8017070 con 0x7effb4102790 2024-09-12T03:14:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.855+0000 7effbbedb740 1 Processor -- start 2024-09-12T03:14:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.855+0000 7effbbedb740 1 -- start start 2024-09-12T03:14:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.856+0000 7effbbedb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c667011b60 0x55c667011f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.856+0000 7effbbedb740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c667012480 0x55c66701c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.856+0000 7effbbedb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c66701cd40 0x55c66701f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.856+0000 7effbbedb740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c666f77280 con 0x55c667012480 2024-09-12T03:14:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.856+0000 7effbbedb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c666f1eb10 con 0x55c667011b60 2024-09-12T03:14:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.856+0000 7effbbedb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c666f73240 con 0x55c66701cd40 2024-09-12T03:14:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.856+0000 7effb37fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c667012480 0x55c66701c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.856+0000 7effb37fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c667012480 0x55c66701c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34240/0 (socket says 172.21.15.116:34240) 2024-09-12T03:14:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.856+0000 7effb37fe640 1 -- 172.21.15.116:0/3256769250 learned_addr learned my addr 172.21.15.116:0/3256769250 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.856+0000 7effb0ff9640 1 --2- 172.21.15.116:0/3864886146 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7eff9c0a2b80 0x7eff9c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.856+0000 7effb3fff640 1 --2- 172.21.15.116:0/3256769250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c667011b60 0x55c667011f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.856+0000 7effb2ffd640 1 --2- 172.21.15.116:0/3256769250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c66701cd40 0x55c66701f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.856+0000 7effb37fe640 1 -- 172.21.15.116:0/3256769250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c66701cd40 msgr2=0x55c66701f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.856+0000 7effb37fe640 1 --2- 172.21.15.116:0/3256769250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c66701cd40 0x55c66701f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.856+0000 7effb37fe640 1 -- 172.21.15.116:0/3256769250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c667011b60 msgr2=0x55c667011f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.856+0000 7effb37fe640 1 --2- 172.21.15.116:0/3256769250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c667011b60 0x55c667011f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.856+0000 7effb37fe640 1 -- 172.21.15.116:0/3256769250 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c666f2b1f0 con 0x55c667012480 2024-09-12T03:14:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.856+0000 7effb3fff640 1 --2- 172.21.15.116:0/3256769250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c667011b60 0x55c667011f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.856+0000 7effb0ff9640 1 -- 172.21.15.116:0/3864886146 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7effa80ce490 con 0x7effb4102790 2024-09-12T03:14:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.856+0000 7effb3fff640 1 --2- 172.21.15.116:0/3864886146 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7eff9c0a2b80 0x7eff9c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.856+0000 7effb37fe640 1 --2- 172.21.15.116:0/3256769250 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c667012480 0x55c66701c800 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7effa4007850 tx=0x7effa4007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff89648d2c424bb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.856+0000 7eff99ffb640 1 -- 172.21.15.116:0/3256769250 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa400d950 con 0x55c667012480 2024-09-12T03:14:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.857+0000 7eff99ffb640 1 -- 172.21.15.116:0/3256769250 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effa4018070 con 0x55c667012480 2024-09-12T03:14:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.857+0000 7eff99ffb640 1 -- 172.21.15.116:0/3256769250 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa40123e0 con 0x55c667012480 2024-09-12T03:14:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.857+0000 7effbbedb740 1 -- 172.21.15.116:0/3256769250 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c667012480 msgr2=0x55c66701c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.857+0000 7effb3fff640 1 --2- 172.21.15.116:0/3864886146 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7eff9c0a2b80 0x7eff9c0a5040 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7effac005cc0 tx=0x7effac03e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.857+0000 7effbbedb740 1 --2- 172.21.15.116:0/3256769250 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c667012480 0x55c66701c800 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7effa4007850 tx=0x7effa4007d10 comp rx=0 tx=0).stop 2024-09-12T03:14:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.857+0000 7effbbedb740 1 -- 172.21.15.116:0/3256769250 shutdown_connections 2024-09-12T03:14:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.857+0000 7effbbedb740 1 --2- 172.21.15.116:0/3256769250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c66701cd40 0x55c66701f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.857+0000 7effbbedb740 1 --2- 172.21.15.116:0/3256769250 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c667012480 0x55c66701c800 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.857+0000 7effbbedb740 1 --2- 172.21.15.116:0/3256769250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c667011b60 0x55c667011f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.857+0000 7effbbedb740 1 -- 172.21.15.116:0/3256769250 >> 172.21.15.116:0/3256769250 conn(0x55c66700e0f0 msgr2=0x55c66700e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.857+0000 7effbbedb740 1 -- 172.21.15.116:0/3256769250 shutdown_connections 2024-09-12T03:14:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.857+0000 7effbbedb740 1 -- 172.21.15.116:0/3256769250 wait complete. 2024-09-12T03:14:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.858+0000 7effbbedb740 1 -- start start 2024-09-12T03:14:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.858+0000 7effbbedb740 1 Processor -- start 2024-09-12T03:14:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.858+0000 7effbbedb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c667011b60 0x55c667019b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:02.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.858+0000 7effbbedb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c667012480 0x55c66701a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:02.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.858+0000 7effbbedb740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c66701cd40 0x55c66701a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:02.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.858+0000 7effbbedb740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c66701f910 con 0x55c66701cd40 2024-09-12T03:14:02.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.858+0000 7effbbedb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c666f31f10 con 0x55c667011b60 2024-09-12T03:14:02.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.858+0000 7effbbedb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c66701fa90 con 0x55c667012480 2024-09-12T03:14:02.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.858+0000 7effbbedb740 10 client.0 _ll_register_callbacks cb 0x55c666efaa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:02.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.858+0000 7effb3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c66701cd40 0x55c66701a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:02.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.858+0000 7effb3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c66701cd40 0x55c66701a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34256/0 (socket says 172.21.15.116:34256) 2024-09-12T03:14:02.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.858+0000 7effb3fff640 1 -- 172.21.15.116:0/320980471 learned_addr learned my addr 172.21.15.116:0/320980471 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:02.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.858+0000 7effb2ffd640 1 --2- 172.21.15.116:0/320980471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c667012480 0x55c66701a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:02.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.859+0000 7effb37fe640 1 --2- 172.21.15.116:0/320980471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c667011b60 0x55c667019b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:02.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.859+0000 7effb3fff640 1 -- 172.21.15.116:0/320980471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c667012480 msgr2=0x55c66701a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:02.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.859+0000 7effb3fff640 1 --2- 172.21.15.116:0/320980471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c667012480 0x55c66701a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.859+0000 7effb3fff640 1 -- 172.21.15.116:0/320980471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c667011b60 msgr2=0x55c667019b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:02.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.859+0000 7effb3fff640 1 --2- 172.21.15.116:0/320980471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c667011b60 0x55c667019b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.859+0000 7effb3fff640 1 -- 172.21.15.116:0/320980471 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c666ce86a0 con 0x55c66701cd40 2024-09-12T03:14:02.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.859+0000 7effb2ffd640 1 --2- 172.21.15.116:0/320980471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c667012480 0x55c66701a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:02.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.859+0000 7effb37fe640 1 --2- 172.21.15.116:0/320980471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c667011b60 0x55c667019b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:14:02.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.859+0000 7effb3fff640 1 --2- 172.21.15.116:0/320980471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c66701cd40 0x55c66701a600 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7effac035000 tx=0x7effac005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f5ee1ed5f0ba20c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:02.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.859+0000 7eff92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:14:02.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.859+0000 7eff92ffd640 1 -- 172.21.15.116:0/320980471 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac050060 con 0x55c66701cd40 2024-09-12T03:14:02.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.859+0000 7eff92ffd640 1 -- 172.21.15.116:0/320980471 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7effac004450 con 0x55c66701cd40 2024-09-12T03:14:02.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.859+0000 7effbbedb740 1 -- 172.21.15.116:0/320980471 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effa4018070 con 0x55c66701cd40 2024-09-12T03:14:02.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.859+0000 7eff92ffd640 1 -- 172.21.15.116:0/320980471 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7effac004650 con 0x55c66701cd40 2024-09-12T03:14:02.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.859+0000 7effbbedb740 10 client.18390 fetch_fsmap learned FSMap version 25 2024-09-12T03:14:02.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.859+0000 7effbbedb740 1 -- 172.21.15.116:0/320980471 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c66702b480 con 0x55c66701cd40 2024-09-12T03:14:02.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.859+0000 7eff92ffd640 1 -- 172.21.15.116:0/320980471 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effa4018070 con 0x55c66701cd40 2024-09-12T03:14:02.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.859+0000 7eff92ffd640 1 -- 172.21.15.116:0/320980471 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7effac030650 con 0x55c66701cd40 2024-09-12T03:14:02.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.859+0000 7effbbedb740 10 client.18390 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:14:02.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.859+0000 7effbbedb740 10 client.18390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:14:02.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.859+0000 7effbbedb740 20 client.18390 populate_metadata read hostname 'smithi116' 2024-09-12T03:14:02.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.859+0000 7effbbedb740 1 --2- 172.21.15.116:0/320980471 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c66702c890 0x55c66702ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:02.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.860+0000 7effbbedb740 4 client.18390 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:14:02.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.860+0000 7effbbedb740 1 -- 172.21.15.116:0/320980471 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c666f847f0 con 0x55c66702c890 2024-09-12T03:14:02.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.860+0000 7effb37fe640 1 --2- 172.21.15.116:0/320980471 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c66702c890 0x55c66702ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:02.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.860+0000 7effb37fe640 1 --2- 172.21.15.116:0/320980471 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c66702c890 0x55c66702ed00 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaa20eb74063a8b5 server_cookie=5e2b8b7380f3d114 in_seq=0 out_seq=0 2024-09-12T03:14:02.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.860+0000 7eff92ffd640 10 client.18390 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:02.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.861+0000 7eff92ffd640 1 -- 172.21.15.116:0/320980471 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c666f847f0 con 0x55c66702c890 2024-09-12T03:14:02.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.861+0000 7eff92ffd640 10 client.18390 handle_command_reply: tid=0 2024-09-12T03:14:02.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.861+0000 7effbbedb740 1 client.18390 shutdown 2024-09-12T03:14:02.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.861+0000 7effbbedb740 1 -- 172.21.15.116:0/320980471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c66701cd40 msgr2=0x55c66701a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:02.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.861+0000 7effbbedb740 1 --2- 172.21.15.116:0/320980471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c66701cd40 0x55c66701a600 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7effac035000 tx=0x7effac005c30 comp rx=0 tx=0).stop 2024-09-12T03:14:02.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.861+0000 7effbbedb740 1 -- 172.21.15.116:0/320980471 shutdown_connections 2024-09-12T03:14:02.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.861+0000 7effbbedb740 1 --2- 172.21.15.116:0/320980471 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c66702c890 0x55c66702ed00 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.861+0000 7effbbedb740 1 --2- 172.21.15.116:0/320980471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c66701cd40 0x55c66701a600 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.861+0000 7effbbedb740 1 --2- 172.21.15.116:0/320980471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c667012480 0x55c66701a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.861+0000 7effbbedb740 1 --2- 172.21.15.116:0/320980471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c667011b60 0x55c667019b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.861+0000 7effbbedb740 1 -- 172.21.15.116:0/320980471 >> 172.21.15.116:0/320980471 conn(0x55c66700e0f0 msgr2=0x55c667013e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:02.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.861+0000 7eff92ffd640 0 client.18390 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:02.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.861+0000 7effbbedb740 1 -- 172.21.15.116:0/320980471 shutdown_connections 2024-09-12T03:14:02.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.861+0000 7effbbedb740 1 -- 172.21.15.116:0/320980471 wait complete. 2024-09-12T03:14:02.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.861+0000 7effbbedb740 20 client.18390 trim_cache size 0 max 16384 2024-09-12T03:14:02.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.878+0000 7effbbedb740 1 Processor -- start 2024-09-12T03:14:02.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.879+0000 7effbbedb740 1 -- start start 2024-09-12T03:14:02.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.879+0000 7effbbedb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c667011b60 0x55c66701b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:02.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.879+0000 7effbbedb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c667012480 0x55c6670345d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:02.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.879+0000 7effbbedb740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c66701cd40 0x55c667034b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:02.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.879+0000 7effbbedb740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7effb41afe60 con 0x55c66701cd40 2024-09-12T03:14:02.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.879+0000 7effbbedb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7effb41ad730 con 0x55c667011b60 2024-09-12T03:14:02.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.879+0000 7effbbedb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7effb41ab580 con 0x55c667012480 2024-09-12T03:14:02.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.879+0000 7effb3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c66701cd40 0x55c667034b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:02.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.879+0000 7effb3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c66701cd40 0x55c667034b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34270/0 (socket says 172.21.15.116:34270) 2024-09-12T03:14:02.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.879+0000 7effb3fff640 1 -- 172.21.15.116:0/3499955989 learned_addr learned my addr 172.21.15.116:0/3499955989 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:02.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.879+0000 7effb37fe640 1 --2- 172.21.15.116:0/3499955989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c667011b60 0x55c66701b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:02.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.879+0000 7effb2ffd640 1 --2- 172.21.15.116:0/3499955989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c667012480 0x55c6670345d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:02.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.879+0000 7effb3fff640 1 -- 172.21.15.116:0/3499955989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c667012480 msgr2=0x55c6670345d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:02.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.879+0000 7effb3fff640 1 --2- 172.21.15.116:0/3499955989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c667012480 0x55c6670345d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.879+0000 7effb3fff640 1 -- 172.21.15.116:0/3499955989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c667011b60 msgr2=0x55c66701b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:02.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.879+0000 7effb3fff640 1 --2- 172.21.15.116:0/3499955989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c667011b60 0x55c66701b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.879+0000 7effb3fff640 1 -- 172.21.15.116:0/3499955989 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c66700df20 con 0x55c66701cd40 2024-09-12T03:14:02.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.879+0000 7effb2ffd640 1 --2- 172.21.15.116:0/3499955989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c667012480 0x55c6670345d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:14:02.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.879+0000 7effb3fff640 1 --2- 172.21.15.116:0/3499955989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c66701cd40 0x55c667034b10 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7effac02ade0 tx=0x7effac009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360a771279f59d94 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:02.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.880+0000 7eff997fa640 1 -- 172.21.15.116:0/3499955989 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac050040 con 0x55c66701cd40 2024-09-12T03:14:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.880+0000 7eff997fa640 1 -- 172.21.15.116:0/3499955989 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effac036040 con 0x55c66701cd40 2024-09-12T03:14:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.880+0000 7eff997fa640 1 -- 172.21.15.116:0/3499955989 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac0305c0 con 0x55c66701cd40 2024-09-12T03:14:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.880+0000 7effbbedb740 1 -- 172.21.15.116:0/3499955989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c66701cd40 msgr2=0x55c667034b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.880+0000 7effbbedb740 1 --2- 172.21.15.116:0/3499955989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c66701cd40 0x55c667034b10 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7effac02ade0 tx=0x7effac009bf0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.880+0000 7effbbedb740 1 -- 172.21.15.116:0/3499955989 shutdown_connections 2024-09-12T03:14:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.880+0000 7effbbedb740 1 --2- 172.21.15.116:0/3499955989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c66701cd40 0x55c667034b10 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.880+0000 7effbbedb740 1 --2- 172.21.15.116:0/3499955989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c667012480 0x55c6670345d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.880+0000 7effbbedb740 1 --2- 172.21.15.116:0/3499955989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c667011b60 0x55c66701b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.880+0000 7effbbedb740 1 -- 172.21.15.116:0/3499955989 >> 172.21.15.116:0/3499955989 conn(0x55c66700e0f0 msgr2=0x55c66701b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.880+0000 7effbbedb740 1 -- 172.21.15.116:0/3499955989 shutdown_connections 2024-09-12T03:14:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.880+0000 7effbbedb740 1 -- 172.21.15.116:0/3499955989 wait complete. 2024-09-12T03:14:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.881+0000 7effbbedb740 1 -- start start 2024-09-12T03:14:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.881+0000 7effbbedb740 1 Processor -- start 2024-09-12T03:14:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.881+0000 7effbbedb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c667011b60 0x55c667020c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.881+0000 7effbbedb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c667012480 0x55c6670211b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.881+0000 7effbbedb740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c66701cd40 0x55c6670216f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.881+0000 7effbbedb740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7effb41abbf0 con 0x55c66701cd40 2024-09-12T03:14:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.881+0000 7effbbedb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7effb41ad060 con 0x55c667012480 2024-09-12T03:14:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.881+0000 7effbbedb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7effb41ac2c0 con 0x55c667011b60 2024-09-12T03:14:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.881+0000 7effbbedb740 10 client.0 _ll_register_callbacks cb 0x7effac009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.881+0000 7effb3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c66701cd40 0x55c6670216f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.881+0000 7effb3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c66701cd40 0x55c6670216f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34274/0 (socket says 172.21.15.116:34274) 2024-09-12T03:14:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.881+0000 7effb3fff640 1 -- 172.21.15.116:0/1485926818 learned_addr learned my addr 172.21.15.116:0/1485926818 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.881+0000 7effb37fe640 1 --2- 172.21.15.116:0/1485926818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c667011b60 0x55c667020c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.882+0000 7effb2ffd640 1 --2- 172.21.15.116:0/1485926818 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c667012480 0x55c6670211b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.882+0000 7effb3fff640 1 -- 172.21.15.116:0/1485926818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c667011b60 msgr2=0x55c667020c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.882+0000 7effb3fff640 1 --2- 172.21.15.116:0/1485926818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c667011b60 0x55c667020c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.882+0000 7effb3fff640 1 -- 172.21.15.116:0/1485926818 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c667012480 msgr2=0x55c6670211b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.882+0000 7effb3fff640 1 --2- 172.21.15.116:0/1485926818 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c667012480 0x55c6670211b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.882+0000 7effb3fff640 1 -- 172.21.15.116:0/1485926818 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c66702f070 con 0x55c66701cd40 2024-09-12T03:14:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.882+0000 7effb3fff640 1 --2- 172.21.15.116:0/1485926818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c66701cd40 0x55c6670216f0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7effac009ca0 tx=0x7effac030fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ddd1237c23faf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.882+0000 7eff92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:14:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.882+0000 7eff92ffd640 1 -- 172.21.15.116:0/1485926818 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac066060 con 0x55c66701cd40 2024-09-12T03:14:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.882+0000 7eff92ffd640 1 -- 172.21.15.116:0/1485926818 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7effac0307d0 con 0x55c66701cd40 2024-09-12T03:14:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.882+0000 7effbbedb740 1 -- 172.21.15.116:0/1485926818 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effac036040 con 0x55c66701cd40 2024-09-12T03:14:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.882+0000 7eff92ffd640 1 -- 172.21.15.116:0/1485926818 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7effac04b030 con 0x55c66701cd40 2024-09-12T03:14:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.882+0000 7effbbedb740 10 client.18396 fetch_fsmap learned FSMap version 25 2024-09-12T03:14:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.882+0000 7effbbedb740 1 -- 172.21.15.116:0/1485926818 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c667097890 con 0x55c66701cd40 2024-09-12T03:14:02.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.882+0000 7eff92ffd640 1 -- 172.21.15.116:0/1485926818 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effac036040 con 0x55c66701cd40 2024-09-12T03:14:02.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.882+0000 7eff92ffd640 1 -- 172.21.15.116:0/1485926818 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7effac057680 con 0x55c66701cd40 2024-09-12T03:14:02.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.882+0000 7effbbedb740 10 client.18396 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:14:02.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.882+0000 7effbbedb740 10 client.18396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:14:02.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.882+0000 7effbbedb740 20 client.18396 populate_metadata read hostname 'smithi116' 2024-09-12T03:14:02.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.883+0000 7effbbedb740 1 --2- 172.21.15.116:0/1485926818 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c667097e40 0x55c66709a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:02.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.883+0000 7effbbedb740 4 client.18396 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:14:02.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.883+0000 7effbbedb740 1 -- 172.21.15.116:0/1485926818 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c666f7ed00 con 0x55c667097e40 2024-09-12T03:14:02.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.883+0000 7effb37fe640 1 --2- 172.21.15.116:0/1485926818 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c667097e40 0x55c66709a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:02.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.883+0000 7effb37fe640 1 --2- 172.21.15.116:0/1485926818 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c667097e40 0x55c66709a230 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f4edf5c891aa0a8 server_cookie=8c7705ea827110e9 in_seq=0 out_seq=0 2024-09-12T03:14:02.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.883+0000 7eff92ffd640 10 client.18396 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:02.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.884+0000 7eff92ffd640 1 -- 172.21.15.116:0/1485926818 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c666f7ed00 con 0x55c667097e40 2024-09-12T03:14:02.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.884+0000 7eff92ffd640 10 client.18396 handle_command_reply: tid=0 2024-09-12T03:14:02.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.884+0000 7effbbedb740 1 client.18396 shutdown 2024-09-12T03:14:02.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.884+0000 7effbbedb740 1 -- 172.21.15.116:0/1485926818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c66701cd40 msgr2=0x55c6670216f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:02.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.884+0000 7effbbedb740 1 --2- 172.21.15.116:0/1485926818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c66701cd40 0x55c6670216f0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7effac009ca0 tx=0x7effac030fa0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.885+0000 7effbbedb740 1 -- 172.21.15.116:0/1485926818 shutdown_connections 2024-09-12T03:14:02.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.885+0000 7effbbedb740 1 --2- 172.21.15.116:0/1485926818 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55c667097e40 0x55c66709a230 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.885+0000 7eff92ffd640 0 client.18396 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:02.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.885+0000 7effbbedb740 1 --2- 172.21.15.116:0/1485926818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c66701cd40 0x55c6670216f0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.885+0000 7effbbedb740 1 --2- 172.21.15.116:0/1485926818 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c667012480 0x55c6670211b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.885+0000 7effbbedb740 1 --2- 172.21.15.116:0/1485926818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c667011b60 0x55c667020c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.885+0000 7effbbedb740 1 -- 172.21.15.116:0/1485926818 >> 172.21.15.116:0/1485926818 conn(0x55c66700e0f0 msgr2=0x55c667013dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:02.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.885+0000 7effbbedb740 1 -- 172.21.15.116:0/1485926818 shutdown_connections 2024-09-12T03:14:02.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.885+0000 7effbbedb740 1 -- 172.21.15.116:0/1485926818 wait complete. 2024-09-12T03:14:02.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.885+0000 7effbbedb740 20 client.18396 trim_cache size 0 max 16384 2024-09-12T03:14:02.892 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:14:02.892 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:14:02.892 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "13b85443-04a3-4593-bb40-7f3ad779e00f", 2024-09-12T03:14:02.892 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:14:02.892 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:14:02.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.885+0000 7effb9b02640 1 -- 172.21.15.116:0/3864886146 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7eff9c0a2b80 msgr2=0x7eff9c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:02.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.885+0000 7effb9b02640 1 --2- 172.21.15.116:0/3864886146 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7eff9c0a2b80 0x7eff9c0a5040 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7effac005cc0 tx=0x7effac03e040 comp rx=0 tx=0).stop 2024-09-12T03:14:02.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.885+0000 7effb9b02640 1 -- 172.21.15.116:0/3864886146 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7effb4102790 msgr2=0x7effb4198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:02.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.885+0000 7effb9b02640 1 --2- 172.21.15.116:0/3864886146 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7effb4102790 0x7effb4198b30 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7effa800b4a0 tx=0x7effa800b960 comp rx=0 tx=0).stop 2024-09-12T03:14:02.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.885+0000 7effb9b02640 1 -- 172.21.15.116:0/3864886146 shutdown_connections 2024-09-12T03:14:02.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.885+0000 7effb9b02640 1 --2- 172.21.15.116:0/3864886146 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7eff9c0a2b80 0x7eff9c0a5040 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.885+0000 7effb9b02640 1 --2- 172.21.15.116:0/3864886146 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effb4106f10 0x7effb419cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.885+0000 7effb9b02640 1 --2- 172.21.15.116:0/3864886146 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7effb4102790 0x7effb4198b30 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.885+0000 7effb9b02640 1 --2- 172.21.15.116:0/3864886146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effb4101de0 0x7effb41985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:02.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.885+0000 7effb9b02640 1 -- 172.21.15.116:0/3864886146 >> 172.21.15.116:0/3864886146 conn(0x7effb40fd8c0 msgr2=0x7effb40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:02.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.886+0000 7effb9b02640 1 -- 172.21.15.116:0/3864886146 shutdown_connections 2024-09-12T03:14:02.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:02.886+0000 7effb9b02640 1 -- 172.21.15.116:0/3864886146 wait complete. 2024-09-12T03:14:03.692 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:03 smithi116 ceph-mon[31410]: pgmap v670: 97 pgs: 97 active+clean; 5.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:14:03.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:03 smithi183 ceph-mon[35535]: pgmap v670: 97 pgs: 97 active+clean; 5.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:14:03.724 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:14:03.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:03 smithi177 ceph-mon[35305]: pgmap v670: 97 pgs: 97 active+clean; 5.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:14:03.971 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:14:04.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.435+0000 7fbce7934640 1 -- 172.21.15.116:0/1289973870 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbce010c180 msgr2=0x7fbce010e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:04.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.435+0000 7fbce7934640 1 --2- 172.21.15.116:0/1289973870 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbce010c180 0x7fbce010e600 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8009ca0 tx=0x7fbcd802a950 comp rx=0 tx=0).stop 2024-09-12T03:14:04.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.435+0000 7fbce7934640 1 -- 172.21.15.116:0/1289973870 shutdown_connections 2024-09-12T03:14:04.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.435+0000 7fbce7934640 1 --2- 172.21.15.116:0/1289973870 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbce010c180 0x7fbce010e600 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.435+0000 7fbce7934640 1 --2- 172.21.15.116:0/1289973870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbce01036c0 0x7fbce010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.435+0000 7fbce7934640 1 --2- 172.21.15.116:0/1289973870 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbce0102da0 0x7fbce0103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.435+0000 7fbce7934640 1 -- 172.21.15.116:0/1289973870 >> 172.21.15.116:0/1289973870 conn(0x7fbce00fc7a0 msgr2=0x7fbce00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:04.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.435+0000 7fbce7934640 1 -- 172.21.15.116:0/1289973870 shutdown_connections 2024-09-12T03:14:04.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.435+0000 7fbce7934640 1 -- 172.21.15.116:0/1289973870 wait complete. 2024-09-12T03:14:04.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.435+0000 7fbce7934640 1 Processor -- start 2024-09-12T03:14:04.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.435+0000 7fbce7934640 1 -- start start 2024-09-12T03:14:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.436+0000 7fbce7934640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbce0102da0 0x7fbce0198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.436+0000 7fbce7934640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbce01036c0 0x7fbce0198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.436+0000 7fbce7934640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbce010c180 0x7fbce019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.436+0000 7fbce7934640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbce0111fa0 con 0x7fbce01036c0 2024-09-12T03:14:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.436+0000 7fbce7934640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbce0111e20 con 0x7fbce0102da0 2024-09-12T03:14:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.436+0000 7fbce7934640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbce0112120 con 0x7fbce010c180 2024-09-12T03:14:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.436+0000 7fbce4ecf640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbce01036c0 0x7fbce0198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.436+0000 7fbce4ecf640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbce01036c0 0x7fbce0198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34280/0 (socket says 172.21.15.116:34280) 2024-09-12T03:14:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.436+0000 7fbce4ecf640 1 -- 172.21.15.116:0/184789681 learned_addr learned my addr 172.21.15.116:0/184789681 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.436+0000 7fbce56d0640 1 --2- 172.21.15.116:0/184789681 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbce0102da0 0x7fbce0198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.436+0000 7fbce5ed1640 1 --2- 172.21.15.116:0/184789681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbce010c180 0x7fbce019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.436+0000 7fbce4ecf640 1 -- 172.21.15.116:0/184789681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbce010c180 msgr2=0x7fbce019cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.436+0000 7fbce4ecf640 1 --2- 172.21.15.116:0/184789681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbce010c180 0x7fbce019cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.436+0000 7fbce4ecf640 1 -- 172.21.15.116:0/184789681 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbce0102da0 msgr2=0x7fbce0198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:04.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.436+0000 7fbce4ecf640 1 --2- 172.21.15.116:0/184789681 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbce0102da0 0x7fbce0198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.436+0000 7fbce4ecf640 1 -- 172.21.15.116:0/184789681 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce019d3f0 con 0x7fbce01036c0 2024-09-12T03:14:04.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.436+0000 7fbce4ecf640 1 --2- 172.21.15.116:0/184789681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbce01036c0 0x7fbce0198980 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400b4a0 tx=0x7fbcd400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:04.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.436+0000 7fbcd27fc640 1 -- 172.21.15.116:0/184789681 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4014070 con 0x7fbce01036c0 2024-09-12T03:14:04.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.436+0000 7fbcd27fc640 1 -- 172.21.15.116:0/184789681 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcd4010af0 con 0x7fbce01036c0 2024-09-12T03:14:04.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.436+0000 7fbce7934640 1 -- 172.21.15.116:0/184789681 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbce019d6e0 con 0x7fbce01036c0 2024-09-12T03:14:04.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.436+0000 7fbcd27fc640 1 -- 172.21.15.116:0/184789681 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4017d10 con 0x7fbce01036c0 2024-09-12T03:14:04.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.436+0000 7fbce7934640 1 -- 172.21.15.116:0/184789681 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbce01a4fc0 con 0x7fbce01036c0 2024-09-12T03:14:04.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.439+0000 7fbcd27fc640 1 -- 172.21.15.116:0/184789681 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fbcd4017070 con 0x7fbce01036c0 2024-09-12T03:14:04.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.439+0000 7fbce9d0d740 1 Processor -- start 2024-09-12T03:14:04.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.439+0000 7fbce9d0d740 1 -- start start 2024-09-12T03:14:04.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.439+0000 7fbce9d0d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5656e0380 0x55b5656e0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:04.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.439+0000 7fbce9d0d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5656e0ca0 0x55b5656ef550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:04.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.439+0000 7fbce9d0d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5656efa90 0x55b5656f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:04.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.439+0000 7fbce9d0d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b5656462e0 con 0x55b5656efa90 2024-09-12T03:14:04.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.439+0000 7fbce9d0d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b5655edce0 con 0x55b5656e0ca0 2024-09-12T03:14:04.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.439+0000 7fbce9d0d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b5656422a0 con 0x55b5656e0380 2024-09-12T03:14:04.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.439+0000 7fbcd27fc640 1 --2- 172.21.15.116:0/184789681 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbccc0a2b80 0x7fbccc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:04.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.439+0000 7fbce4ecf640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5656efa90 0x55b5656f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:04.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.440+0000 7fbce4ecf640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5656efa90 0x55b5656f1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34282/0 (socket says 172.21.15.116:34282) 2024-09-12T03:14:04.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.440+0000 7fbce4ecf640 1 -- 172.21.15.116:0/2129008850 learned_addr learned my addr 172.21.15.116:0/2129008850 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:04.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.440+0000 7fbce5ed1640 1 --2- 172.21.15.116:0/2129008850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5656e0380 0x55b5656e0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:04.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.440+0000 7fbce56d0640 1 --2- 172.21.15.116:0/2129008850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5656e0ca0 0x55b5656ef550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:04.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.440+0000 7fbce5ed1640 1 --2- 172.21.15.116:0/184789681 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbccc0a2b80 0x7fbccc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:04.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.440+0000 7fbcd27fc640 1 -- 172.21.15.116:0/184789681 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fbcd40ce3e0 con 0x7fbce01036c0 2024-09-12T03:14:04.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.440+0000 7fbce4ecf640 1 -- 172.21.15.116:0/2129008850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5656e0380 msgr2=0x55b5656e0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:04.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.440+0000 7fbce4ecf640 1 --2- 172.21.15.116:0/2129008850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5656e0380 0x55b5656e0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.440+0000 7fbce4ecf640 1 -- 172.21.15.116:0/2129008850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5656e0ca0 msgr2=0x55b5656ef550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:04.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.440+0000 7fbce4ecf640 1 --2- 172.21.15.116:0/2129008850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5656e0ca0 0x55b5656ef550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.440+0000 7fbce4ecf640 1 -- 172.21.15.116:0/2129008850 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5655d2440 con 0x55b5656efa90 2024-09-12T03:14:04.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.440+0000 7fbce4ecf640 1 --2- 172.21.15.116:0/2129008850 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5656efa90 0x55b5656f1e80 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fbcd40cbf10 tx=0x7fbcd40047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b888486114715983 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:04.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.440+0000 7fbce5ed1640 1 --2- 172.21.15.116:0/184789681 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbccc0a2b80 0x7fbccc0a5040 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8009a60 tx=0x7fbcd80095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:04.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.440+0000 7fbccb7fe640 1 -- 172.21.15.116:0/2129008850 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd401e070 con 0x55b5656efa90 2024-09-12T03:14:04.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.440+0000 7fbccb7fe640 1 -- 172.21.15.116:0/2129008850 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcd409ac70 con 0x55b5656efa90 2024-09-12T03:14:04.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.440+0000 7fbccb7fe640 1 -- 172.21.15.116:0/2129008850 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4010dc0 con 0x55b5656efa90 2024-09-12T03:14:04.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.441+0000 7fbce9d0d740 1 -- 172.21.15.116:0/2129008850 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5656efa90 msgr2=0x55b5656f1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:04.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.441+0000 7fbce9d0d740 1 --2- 172.21.15.116:0/2129008850 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5656efa90 0x55b5656f1e80 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fbcd40cbf10 tx=0x7fbcd40047d0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.441+0000 7fbce9d0d740 1 -- 172.21.15.116:0/2129008850 shutdown_connections 2024-09-12T03:14:04.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.441+0000 7fbce9d0d740 1 --2- 172.21.15.116:0/2129008850 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5656efa90 0x55b5656f1e80 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.441+0000 7fbce9d0d740 1 --2- 172.21.15.116:0/2129008850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5656e0ca0 0x55b5656ef550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.441+0000 7fbce9d0d740 1 --2- 172.21.15.116:0/2129008850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5656e0380 0x55b5656e0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.441+0000 7fbce9d0d740 1 -- 172.21.15.116:0/2129008850 >> 172.21.15.116:0/2129008850 conn(0x55b5656dca30 msgr2=0x55b5656dce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:04.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.441+0000 7fbce9d0d740 1 -- 172.21.15.116:0/2129008850 shutdown_connections 2024-09-12T03:14:04.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.441+0000 7fbce9d0d740 1 -- 172.21.15.116:0/2129008850 wait complete. 2024-09-12T03:14:04.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.441+0000 7fbce9d0d740 1 -- start start 2024-09-12T03:14:04.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.442+0000 7fbce9d0d740 1 Processor -- start 2024-09-12T03:14:04.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.442+0000 7fbce9d0d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5656e0380 0x55b5656ef400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:04.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.442+0000 7fbce9d0d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5656e0ca0 0x55b5656ec700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:04.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.442+0000 7fbce9d0d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5656efa90 0x55b5656ecd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:04.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.442+0000 7fbce9d0d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b5656f2600 con 0x55b5656e0ca0 2024-09-12T03:14:04.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.442+0000 7fbce9d0d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b565601050 con 0x55b5656e0380 2024-09-12T03:14:04.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.442+0000 7fbce9d0d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b5656f2780 con 0x55b5656efa90 2024-09-12T03:14:04.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.442+0000 7fbce9d0d740 10 client.0 _ll_register_callbacks cb 0x55b565641160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:04.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.442+0000 7fbce4ecf640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5656e0ca0 0x55b5656ec700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:04.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.442+0000 7fbce4ecf640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5656e0ca0 0x55b5656ec700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34292/0 (socket says 172.21.15.116:34292) 2024-09-12T03:14:04.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.442+0000 7fbce4ecf640 1 -- 172.21.15.116:0/3755237285 learned_addr learned my addr 172.21.15.116:0/3755237285 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:04.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.442+0000 7fbce56d0640 1 --2- 172.21.15.116:0/3755237285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5656e0380 0x55b5656ef400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:04.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.442+0000 7fbce5ed1640 1 --2- 172.21.15.116:0/3755237285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5656efa90 0x55b5656ecd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:04.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.442+0000 7fbce4ecf640 1 -- 172.21.15.116:0/3755237285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5656efa90 msgr2=0x55b5656ecd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:04.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.442+0000 7fbce4ecf640 1 --2- 172.21.15.116:0/3755237285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5656efa90 0x55b5656ecd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.442+0000 7fbce4ecf640 1 -- 172.21.15.116:0/3755237285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5656e0380 msgr2=0x55b5656ef400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:04.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.442+0000 7fbce4ecf640 1 --2- 172.21.15.116:0/3755237285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5656e0380 0x55b5656ef400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.442+0000 7fbce4ecf640 1 -- 172.21.15.116:0/3755237285 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5653b73c0 con 0x55b5656e0ca0 2024-09-12T03:14:04.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.442+0000 7fbce56d0640 1 --2- 172.21.15.116:0/3755237285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5656e0380 0x55b5656ef400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:04.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.443+0000 7fbce4ecf640 1 --2- 172.21.15.116:0/3755237285 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5656e0ca0 0x55b5656ec700 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fbcd40cc280 tx=0x7fbcd40a3a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee442833c07f9a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:04.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.443+0000 7fbcc8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:14:04.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.443+0000 7fbcc8ff9640 1 -- 172.21.15.116:0/3755237285 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd401e070 con 0x55b5656e0ca0 2024-09-12T03:14:04.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.443+0000 7fbcc8ff9640 1 -- 172.21.15.116:0/3755237285 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fbcd409a0c0 con 0x55b5656e0ca0 2024-09-12T03:14:04.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.443+0000 7fbce9d0d740 1 -- 172.21.15.116:0/3755237285 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcd409ac70 con 0x55b5656e0ca0 2024-09-12T03:14:04.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.443+0000 7fbcc8ff9640 1 -- 172.21.15.116:0/3755237285 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbcd409a2e0 con 0x55b5656e0ca0 2024-09-12T03:14:04.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.443+0000 7fbce9d0d740 10 client.18408 fetch_fsmap learned FSMap version 25 2024-09-12T03:14:04.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.443+0000 7fbce9d0d740 1 -- 172.21.15.116:0/3755237285 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b5656fe050 con 0x55b5656e0ca0 2024-09-12T03:14:04.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.443+0000 7fbcc8ff9640 1 -- 172.21.15.116:0/3755237285 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcd409ac70 con 0x55b5656e0ca0 2024-09-12T03:14:04.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.443+0000 7fbcc8ff9640 1 -- 172.21.15.116:0/3755237285 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbcd4099b30 con 0x55b5656e0ca0 2024-09-12T03:14:04.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.443+0000 7fbce9d0d740 10 client.18408 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:14:04.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.443+0000 7fbce9d0d740 10 client.18408 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:14:04.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.443+0000 7fbce9d0d740 20 client.18408 populate_metadata read hostname 'smithi116' 2024-09-12T03:14:04.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.443+0000 7fbce9d0d740 1 --2- 172.21.15.116:0/3755237285 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b5656ff3a0 0x55b565701790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:04.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.443+0000 7fbce9d0d740 4 client.18408 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:14:04.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.443+0000 7fbce9d0d740 1 -- 172.21.15.116:0/3755237285 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b565653850 con 0x55b5656ff3a0 2024-09-12T03:14:04.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.443+0000 7fbce56d0640 1 --2- 172.21.15.116:0/3755237285 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b5656ff3a0 0x55b565701790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:04.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.444+0000 7fbcc8ff9640 10 client.18408 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:04.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.444+0000 7fbce56d0640 1 --2- 172.21.15.116:0/3755237285 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b5656ff3a0 0x55b565701790 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a95b03f9875a8d87 server_cookie=b024c304ba6e6eab in_seq=0 out_seq=0 2024-09-12T03:14:04.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.444+0000 7fbcc8ff9640 1 -- 172.21.15.116:0/3755237285 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b565653850 con 0x55b5656ff3a0 2024-09-12T03:14:04.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.444+0000 7fbcc8ff9640 10 client.18408 handle_command_reply: tid=0 2024-09-12T03:14:04.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.444+0000 7fbce9d0d740 1 client.18408 shutdown 2024-09-12T03:14:04.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.445+0000 7fbce9d0d740 1 -- 172.21.15.116:0/3755237285 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5656e0ca0 msgr2=0x55b5656ec700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:04.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.445+0000 7fbce9d0d740 1 --2- 172.21.15.116:0/3755237285 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5656e0ca0 0x55b5656ec700 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fbcd40cc280 tx=0x7fbcd40a3a00 comp rx=0 tx=0).stop 2024-09-12T03:14:04.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.445+0000 7fbce9d0d740 1 -- 172.21.15.116:0/3755237285 shutdown_connections 2024-09-12T03:14:04.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.445+0000 7fbce9d0d740 1 --2- 172.21.15.116:0/3755237285 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b5656ff3a0 0x55b565701790 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.445+0000 7fbce9d0d740 1 --2- 172.21.15.116:0/3755237285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5656efa90 0x55b5656ecd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.445+0000 7fbce9d0d740 1 --2- 172.21.15.116:0/3755237285 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5656e0ca0 0x55b5656ec700 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.445+0000 7fbce9d0d740 1 --2- 172.21.15.116:0/3755237285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5656e0380 0x55b5656ef400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.445+0000 7fbcc8ff9640 0 client.18408 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:04.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.445+0000 7fbce9d0d740 1 -- 172.21.15.116:0/3755237285 >> 172.21.15.116:0/3755237285 conn(0x55b5656dca30 msgr2=0x55b5656e2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:04.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.445+0000 7fbce9d0d740 1 -- 172.21.15.116:0/3755237285 shutdown_connections 2024-09-12T03:14:04.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.445+0000 7fbce9d0d740 1 -- 172.21.15.116:0/3755237285 wait complete. 2024-09-12T03:14:04.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.445+0000 7fbce9d0d740 20 client.18408 trim_cache size 0 max 16384 2024-09-12T03:14:04.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.462+0000 7fbce9d0d740 1 Processor -- start 2024-09-12T03:14:04.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.462+0000 7fbce9d0d740 1 -- start start 2024-09-12T03:14:04.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.462+0000 7fbce9d0d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5656e0380 0x55b5657047b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:04.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.462+0000 7fbce9d0d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5656e0ca0 0x55b565704cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:04.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.462+0000 7fbce9d0d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5656efa90 0x55b5656e9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:04.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.462+0000 7fbce9d0d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbce01adc50 con 0x55b5656efa90 2024-09-12T03:14:04.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.462+0000 7fbce9d0d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbce01ab520 con 0x55b5656e0ca0 2024-09-12T03:14:04.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.462+0000 7fbce9d0d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbce01a9370 con 0x55b5656e0380 2024-09-12T03:14:04.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.462+0000 7fbce5ed1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5656efa90 0x55b5656e9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:04.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.462+0000 7fbce5ed1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5656efa90 0x55b5656e9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34294/0 (socket says 172.21.15.116:34294) 2024-09-12T03:14:04.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.462+0000 7fbce5ed1640 1 -- 172.21.15.116:0/156234397 learned_addr learned my addr 172.21.15.116:0/156234397 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:04.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.462+0000 7fbce56d0640 1 --2- 172.21.15.116:0/156234397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5656e0380 0x55b5657047b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:04.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.462+0000 7fbce4ecf640 1 --2- 172.21.15.116:0/156234397 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5656e0ca0 0x55b565704cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:04.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.462+0000 7fbce5ed1640 1 -- 172.21.15.116:0/156234397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5656e0380 msgr2=0x55b5657047b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:04.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.462+0000 7fbce5ed1640 1 --2- 172.21.15.116:0/156234397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5656e0380 0x55b5657047b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.462+0000 7fbce5ed1640 1 -- 172.21.15.116:0/156234397 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5656e0ca0 msgr2=0x55b565704cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:04.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.462+0000 7fbce5ed1640 1 --2- 172.21.15.116:0/156234397 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5656e0ca0 0x55b565704cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.462+0000 7fbce5ed1640 1 -- 172.21.15.116:0/156234397 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5656ed080 con 0x55b5656efa90 2024-09-12T03:14:04.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.462+0000 7fbce56d0640 1 --2- 172.21.15.116:0/156234397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5656e0380 0x55b5657047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:04.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.463+0000 7fbce5ed1640 1 --2- 172.21.15.116:0/156234397 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5656efa90 0x55b5656e9980 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80098e0 tx=0x7fbcd8040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460289bb0937cf77 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:04.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.463+0000 7fbccaffd640 1 -- 172.21.15.116:0/156234397 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8050070 con 0x55b5656efa90 2024-09-12T03:14:04.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.463+0000 7fbccaffd640 1 -- 172.21.15.116:0/156234397 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcd8004520 con 0x55b5656efa90 2024-09-12T03:14:04.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.463+0000 7fbccaffd640 1 -- 172.21.15.116:0/156234397 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8034040 con 0x55b5656efa90 2024-09-12T03:14:04.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.463+0000 7fbce9d0d740 1 -- 172.21.15.116:0/156234397 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5656efa90 msgr2=0x55b5656e9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:04.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.463+0000 7fbce9d0d740 1 --2- 172.21.15.116:0/156234397 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5656efa90 0x55b5656e9980 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80098e0 tx=0x7fbcd8040000 comp rx=0 tx=0).stop 2024-09-12T03:14:04.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.463+0000 7fbce9d0d740 1 -- 172.21.15.116:0/156234397 shutdown_connections 2024-09-12T03:14:04.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.463+0000 7fbce9d0d740 1 --2- 172.21.15.116:0/156234397 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5656efa90 0x55b5656e9980 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.463+0000 7fbce9d0d740 1 --2- 172.21.15.116:0/156234397 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5656e0ca0 0x55b565704cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.463+0000 7fbce9d0d740 1 --2- 172.21.15.116:0/156234397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5656e0380 0x55b5657047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.463+0000 7fbce9d0d740 1 -- 172.21.15.116:0/156234397 >> 172.21.15.116:0/156234397 conn(0x55b5656dca30 msgr2=0x55b5656e7d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:04.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.463+0000 7fbce9d0d740 1 -- 172.21.15.116:0/156234397 shutdown_connections 2024-09-12T03:14:04.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.464+0000 7fbce9d0d740 1 -- 172.21.15.116:0/156234397 wait complete. 2024-09-12T03:14:04.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.464+0000 7fbce9d0d740 1 -- start start 2024-09-12T03:14:04.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.464+0000 7fbce9d0d740 1 Processor -- start 2024-09-12T03:14:04.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.464+0000 7fbce9d0d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5656e0380 0x55b565702870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:04.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.464+0000 7fbce9d0d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5656e0ca0 0x55b565702db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:04.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.464+0000 7fbce9d0d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5656efa90 0x55b5657032f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:04.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.464+0000 7fbce9d0d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbce01a99e0 con 0x55b5656e0ca0 2024-09-12T03:14:04.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.464+0000 7fbce9d0d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbce01aae50 con 0x55b5656efa90 2024-09-12T03:14:04.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.464+0000 7fbce9d0d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbce01aa0b0 con 0x55b5656e0380 2024-09-12T03:14:04.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.464+0000 7fbce9d0d740 10 client.0 _ll_register_callbacks cb 0x7fbcd8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:04.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.464+0000 7fbce4ecf640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5656e0ca0 0x55b565702db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:04.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.464+0000 7fbce4ecf640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5656e0ca0 0x55b565702db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34308/0 (socket says 172.21.15.116:34308) 2024-09-12T03:14:04.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.464+0000 7fbce4ecf640 1 -- 172.21.15.116:0/1612274298 learned_addr learned my addr 172.21.15.116:0/1612274298 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:04.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.465+0000 7fbce5ed1640 1 --2- 172.21.15.116:0/1612274298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5656efa90 0x55b5657032f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:04.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.465+0000 7fbce56d0640 1 --2- 172.21.15.116:0/1612274298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5656e0380 0x55b565702870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:04.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.465+0000 7fbce4ecf640 1 -- 172.21.15.116:0/1612274298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5656e0380 msgr2=0x55b565702870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:04.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.465+0000 7fbce4ecf640 1 --2- 172.21.15.116:0/1612274298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5656e0380 0x55b565702870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.465+0000 7fbce4ecf640 1 -- 172.21.15.116:0/1612274298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5656efa90 msgr2=0x55b5657032f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:04.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.465+0000 7fbce4ecf640 1 --2- 172.21.15.116:0/1612274298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5656efa90 0x55b5657032f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.465+0000 7fbce4ecf640 1 -- 172.21.15.116:0/1612274298 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b565768f80 con 0x55b5656e0ca0 2024-09-12T03:14:04.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.465+0000 7fbce5ed1640 1 --2- 172.21.15.116:0/1612274298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5656efa90 0x55b5657032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:04.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.465+0000 7fbce56d0640 1 --2- 172.21.15.116:0/1612274298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5656e0380 0x55b565702870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:04.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.465+0000 7fbce4ecf640 1 --2- 172.21.15.116:0/1612274298 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5656e0ca0 0x55b565702db0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400b470 tx=0x7fbcd40a3900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70fd15db2b591e72 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:04.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.465+0000 7fbcc8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:14:04.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.465+0000 7fbcc8ff9640 1 -- 172.21.15.116:0/1612274298 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd401e040 con 0x55b5656e0ca0 2024-09-12T03:14:04.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.465+0000 7fbce9d0d740 1 -- 172.21.15.116:0/1612274298 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcd8004520 con 0x55b5656e0ca0 2024-09-12T03:14:04.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.465+0000 7fbcc8ff9640 1 -- 172.21.15.116:0/1612274298 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fbcd40a0500 con 0x55b5656e0ca0 2024-09-12T03:14:04.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.465+0000 7fbcc8ff9640 1 -- 172.21.15.116:0/1612274298 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbcd4080050 con 0x55b5656e0ca0 2024-09-12T03:14:04.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.465+0000 7fbce9d0d740 10 client.18414 fetch_fsmap learned FSMap version 25 2024-09-12T03:14:04.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.465+0000 7fbce9d0d740 1 -- 172.21.15.116:0/1612274298 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b56576a270 con 0x55b5656e0ca0 2024-09-12T03:14:04.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.465+0000 7fbcc8ff9640 1 -- 172.21.15.116:0/1612274298 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcd8004520 con 0x55b5656e0ca0 2024-09-12T03:14:04.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.466+0000 7fbcc8ff9640 1 -- 172.21.15.116:0/1612274298 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbcd40972e0 con 0x55b5656e0ca0 2024-09-12T03:14:04.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.466+0000 7fbce9d0d740 10 client.18414 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:14:04.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.466+0000 7fbce9d0d740 10 client.18414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:14:04.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.466+0000 7fbce9d0d740 20 client.18414 populate_metadata read hostname 'smithi116' 2024-09-12T03:14:04.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.466+0000 7fbce9d0d740 1 --2- 172.21.15.116:0/1612274298 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b56576a7d0 0x55b56576cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:04.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.466+0000 7fbce9d0d740 4 client.18414 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:14:04.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.466+0000 7fbce9d0d740 1 -- 172.21.15.116:0/1612274298 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b56564dd60 con 0x55b56576a7d0 2024-09-12T03:14:04.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.466+0000 7fbce56d0640 1 --2- 172.21.15.116:0/1612274298 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b56576a7d0 0x55b56576cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:04.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.466+0000 7fbce56d0640 1 --2- 172.21.15.116:0/1612274298 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b56576a7d0 0x55b56576cbf0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f027baebc03b5a4 server_cookie=625edce37a75f169 in_seq=0 out_seq=0 2024-09-12T03:14:04.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.466+0000 7fbcc8ff9640 10 client.18414 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:04.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.466+0000 7fbcc8ff9640 1 -- 172.21.15.116:0/1612274298 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b56564dd60 con 0x55b56576a7d0 2024-09-12T03:14:04.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.466+0000 7fbcc8ff9640 10 client.18414 handle_command_reply: tid=0 2024-09-12T03:14:04.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.466+0000 7fbce9d0d740 1 client.18414 shutdown 2024-09-12T03:14:04.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.466+0000 7fbce9d0d740 1 -- 172.21.15.116:0/1612274298 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5656e0ca0 msgr2=0x55b565702db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:04.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.466+0000 7fbce9d0d740 1 --2- 172.21.15.116:0/1612274298 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5656e0ca0 0x55b565702db0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400b470 tx=0x7fbcd40a3900 comp rx=0 tx=0).stop 2024-09-12T03:14:04.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.467+0000 7fbce9d0d740 1 -- 172.21.15.116:0/1612274298 shutdown_connections 2024-09-12T03:14:04.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.467+0000 7fbce9d0d740 1 --2- 172.21.15.116:0/1612274298 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b56576a7d0 0x55b56576cbf0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.467+0000 7fbce9d0d740 1 --2- 172.21.15.116:0/1612274298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b5656efa90 0x55b5657032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.467+0000 7fbce9d0d740 1 --2- 172.21.15.116:0/1612274298 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b5656e0ca0 0x55b565702db0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.467+0000 7fbce9d0d740 1 --2- 172.21.15.116:0/1612274298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5656e0380 0x55b565702870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.467+0000 7fbce9d0d740 1 -- 172.21.15.116:0/1612274298 >> 172.21.15.116:0/1612274298 conn(0x55b5656dca30 msgr2=0x55b5656edf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:04.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.467+0000 7fbcc8ff9640 0 client.18414 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:04.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.467+0000 7fbce9d0d740 1 -- 172.21.15.116:0/1612274298 shutdown_connections 2024-09-12T03:14:04.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.467+0000 7fbce9d0d740 1 -- 172.21.15.116:0/1612274298 wait complete. 2024-09-12T03:14:04.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.467+0000 7fbce9d0d740 20 client.18414 trim_cache size 0 max 16384 2024-09-12T03:14:04.475 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:14:04.475 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:14:04.475 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:14:04.475 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:14:04.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.467+0000 7fbce7934640 1 -- 172.21.15.116:0/184789681 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbccc0a2b80 msgr2=0x7fbccc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:04.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.467+0000 7fbce7934640 1 --2- 172.21.15.116:0/184789681 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbccc0a2b80 0x7fbccc0a5040 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8009a60 tx=0x7fbcd80095c0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.467+0000 7fbce7934640 1 -- 172.21.15.116:0/184789681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbce01036c0 msgr2=0x7fbce0198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:04.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.467+0000 7fbce7934640 1 --2- 172.21.15.116:0/184789681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbce01036c0 0x7fbce0198980 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400b4a0 tx=0x7fbcd400b960 comp rx=0 tx=0).stop 2024-09-12T03:14:04.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.467+0000 7fbce7934640 1 -- 172.21.15.116:0/184789681 shutdown_connections 2024-09-12T03:14:04.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.467+0000 7fbce7934640 1 --2- 172.21.15.116:0/184789681 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbccc0a2b80 0x7fbccc0a5040 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.467+0000 7fbce7934640 1 --2- 172.21.15.116:0/184789681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbce010c180 0x7fbce019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.467+0000 7fbce7934640 1 --2- 172.21.15.116:0/184789681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbce01036c0 0x7fbce0198980 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.467+0000 7fbce7934640 1 --2- 172.21.15.116:0/184789681 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbce0102da0 0x7fbce0198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:04.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.467+0000 7fbce7934640 1 -- 172.21.15.116:0/184789681 >> 172.21.15.116:0/184789681 conn(0x7fbce00fc7a0 msgr2=0x7fbce00fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:04.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.467+0000 7fbce7934640 1 -- 172.21.15.116:0/184789681 shutdown_connections 2024-09-12T03:14:04.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:04.467+0000 7fbce7934640 1 -- 172.21.15.116:0/184789681 wait complete. 2024-09-12T03:14:04.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:04 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:14:04.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:04 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:14:04.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:04 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:14:04.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:04 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:14:04.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:04 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:14:04.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:04 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:14:04.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:04 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:14:04.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:04 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:14:04.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:04 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:14:04.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:04 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:14:04.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:04 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:14:04.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:04 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:14:04.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:04 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:14:04.799 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:04 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:14:04.800 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:04 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:14:05.359 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:14:05.359 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:14:05.609 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:14:05.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:05 smithi183 ceph-mon[35535]: pgmap v671: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-12T03:14:05.729 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:05 smithi116 ceph-mon[31410]: pgmap v671: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-12T03:14:05.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:05 smithi177 ceph-mon[35305]: pgmap v671: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-12T03:14:06.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.109+0000 7f23a8ab2640 1 -- 172.21.15.116:0/4187287741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23a41009d0 msgr2=0x7f23a4100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:06.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.109+0000 7f23a8ab2640 1 --2- 172.21.15.116:0/4187287741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23a41009d0 0x7f23a4100db0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f2394009900 tx=0x7f239402a410 comp rx=0 tx=0).stop 2024-09-12T03:14:06.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.109+0000 7f23a8ab2640 1 -- 172.21.15.116:0/4187287741 shutdown_connections 2024-09-12T03:14:06.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.109+0000 7f23a8ab2640 1 --2- 172.21.15.116:0/4187287741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23a410e4f0 0x7f23a4110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:06.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.109+0000 7f23a8ab2640 1 --2- 172.21.15.116:0/4187287741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23a4101380 0x7f23a410dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:06.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.109+0000 7f23a8ab2640 1 --2- 172.21.15.116:0/4187287741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23a41009d0 0x7f23a4100db0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:06.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.109+0000 7f23a8ab2640 1 -- 172.21.15.116:0/4187287741 >> 172.21.15.116:0/4187287741 conn(0x7f23a40fc7d0 msgr2=0x7f23a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:06.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.109+0000 7f23a8ab2640 1 -- 172.21.15.116:0/4187287741 shutdown_connections 2024-09-12T03:14:06.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.109+0000 7f23a8ab2640 1 -- 172.21.15.116:0/4187287741 wait complete. 2024-09-12T03:14:06.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.110+0000 7f23a8ab2640 1 Processor -- start 2024-09-12T03:14:06.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.110+0000 7f23a8ab2640 1 -- start start 2024-09-12T03:14:06.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.110+0000 7f23a8ab2640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23a41009d0 0x7f23a41a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:06.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.110+0000 7f23a8ab2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23a4101380 0x7f23a41a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:06.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.110+0000 7f23a8ab2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23a410e4f0 0x7f23a419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:06.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.110+0000 7f23a8ab2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f23a4114340 con 0x7f23a4101380 2024-09-12T03:14:06.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.110+0000 7f23a8ab2640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f23a41141c0 con 0x7f23a41009d0 2024-09-12T03:14:06.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.110+0000 7f23a8ab2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f23a41144c0 con 0x7f23a410e4f0 2024-09-12T03:14:06.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.110+0000 7f23a1d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23a4101380 0x7f23a41a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:06.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.110+0000 7f23a1d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23a4101380 0x7f23a41a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34316/0 (socket says 172.21.15.116:34316) 2024-09-12T03:14:06.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.110+0000 7f23a1d9b640 1 -- 172.21.15.116:0/347834661 learned_addr learned my addr 172.21.15.116:0/347834661 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:06.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.110+0000 7f23a259c640 1 --2- 172.21.15.116:0/347834661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23a41009d0 0x7f23a41a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:06.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.110+0000 7f23a2d9d640 1 --2- 172.21.15.116:0/347834661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23a410e4f0 0x7f23a419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:06.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.110+0000 7f23a1d9b640 1 -- 172.21.15.116:0/347834661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23a410e4f0 msgr2=0x7f23a419a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:06.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.110+0000 7f23a1d9b640 1 --2- 172.21.15.116:0/347834661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23a410e4f0 0x7f23a419a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:06.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.110+0000 7f23a1d9b640 1 -- 172.21.15.116:0/347834661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23a41009d0 msgr2=0x7f23a41a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:06.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.110+0000 7f23a1d9b640 1 --2- 172.21.15.116:0/347834661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23a41009d0 0x7f23a41a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:06.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.110+0000 7f23a1d9b640 1 -- 172.21.15.116:0/347834661 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a419af50 con 0x7f23a4101380 2024-09-12T03:14:06.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.110+0000 7f23a2d9d640 1 --2- 172.21.15.116:0/347834661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23a410e4f0 0x7f23a419a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:06.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.110+0000 7f23a1d9b640 1 --2- 172.21.15.116:0/347834661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23a4101380 0x7f23a41a0b50 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f239800b4a0 tx=0x7f239800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:06.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.111+0000 7f23937fe640 1 -- 172.21.15.116:0/347834661 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2398014070 con 0x7f23a4101380 2024-09-12T03:14:06.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.111+0000 7f23a8ab2640 1 -- 172.21.15.116:0/347834661 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23a419b240 con 0x7f23a4101380 2024-09-12T03:14:06.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.111+0000 7f23937fe640 1 -- 172.21.15.116:0/347834661 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2398010af0 con 0x7f23a4101380 2024-09-12T03:14:06.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.111+0000 7f23937fe640 1 -- 172.21.15.116:0/347834661 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2398017d10 con 0x7f23a4101380 2024-09-12T03:14:06.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.111+0000 7f23a8ab2640 1 -- 172.21.15.116:0/347834661 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23a41a7590 con 0x7f23a4101380 2024-09-12T03:14:06.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.112+0000 7f23a8ab2640 1 -- 172.21.15.116:0/347834661 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23a406a7d0 con 0x7f23a4101380 2024-09-12T03:14:06.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.113+0000 7f23937fe640 1 -- 172.21.15.116:0/347834661 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2398017070 con 0x7f23a4101380 2024-09-12T03:14:06.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.113+0000 7f23937fe640 1 --2- 172.21.15.116:0/347834661 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f238c0a2b80 0x7f238c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:06.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.113+0000 7f23a259c640 1 --2- 172.21.15.116:0/347834661 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f238c0a2b80 0x7f238c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:06.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.113+0000 7f23937fe640 1 -- 172.21.15.116:0/347834661 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f23980ce3e0 con 0x7f23a4101380 2024-09-12T03:14:06.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.114+0000 7f23a259c640 1 --2- 172.21.15.116:0/347834661 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f238c0a2b80 0x7f238c0a5040 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2394000c00 tx=0x7f23940058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:06.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.118+0000 7f23937fe640 1 -- 172.21.15.116:0/347834661 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f239809b370 con 0x7f23a4101380 2024-09-12T03:14:06.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.324+0000 7f23a8ab2640 1 -- 172.21.15.116:0/347834661 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23a419bc40 con 0x7f23a4101380 2024-09-12T03:14:06.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.325+0000 7f23937fe640 1 -- 172.21.15.116:0/347834661 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f23a419bc40 con 0x7f23a4101380 2024-09-12T03:14:06.328 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:14:06.329 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:14:06.329 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:14:06.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.327+0000 7f23a8ab2640 1 -- 172.21.15.116:0/347834661 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f238c0a2b80 msgr2=0x7f238c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:06.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.327+0000 7f23a8ab2640 1 --2- 172.21.15.116:0/347834661 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f238c0a2b80 0x7f238c0a5040 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2394000c00 tx=0x7f23940058d0 comp rx=0 tx=0).stop 2024-09-12T03:14:06.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.327+0000 7f23a8ab2640 1 -- 172.21.15.116:0/347834661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23a4101380 msgr2=0x7f23a41a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:06.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.327+0000 7f23a8ab2640 1 --2- 172.21.15.116:0/347834661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23a4101380 0x7f23a41a0b50 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f239800b4a0 tx=0x7f239800b960 comp rx=0 tx=0).stop 2024-09-12T03:14:06.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.327+0000 7f23a8ab2640 1 -- 172.21.15.116:0/347834661 shutdown_connections 2024-09-12T03:14:06.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.327+0000 7f23a8ab2640 1 --2- 172.21.15.116:0/347834661 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f238c0a2b80 0x7f238c0a5040 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:06.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.327+0000 7f23a8ab2640 1 --2- 172.21.15.116:0/347834661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23a410e4f0 0x7f23a419a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:06.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.327+0000 7f23a8ab2640 1 --2- 172.21.15.116:0/347834661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23a4101380 0x7f23a41a0b50 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:06.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.327+0000 7f23a8ab2640 1 --2- 172.21.15.116:0/347834661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23a41009d0 0x7f23a41a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:06.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.327+0000 7f23a8ab2640 1 -- 172.21.15.116:0/347834661 >> 172.21.15.116:0/347834661 conn(0x7f23a40fc7d0 msgr2=0x7f23a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:06.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.328+0000 7f23a8ab2640 1 -- 172.21.15.116:0/347834661 shutdown_connections 2024-09-12T03:14:06.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:06.328+0000 7f23a8ab2640 1 -- 172.21.15.116:0/347834661 wait complete. 2024-09-12T03:14:06.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:06 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/347834661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:14:06.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:06 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/347834661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:14:06.754 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:06 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/347834661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:14:07.160 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:14:07.409 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:14:07.692 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:07 smithi116 ceph-mon[31410]: pgmap v672: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:14:07.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:07 smithi183 ceph-mon[35535]: pgmap v672: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:14:07.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:07 smithi177 ceph-mon[35305]: pgmap v672: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:14:07.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.910+0000 7f8022b6d640 1 -- 172.21.15.116:0/2370459710 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f801c106ba0 msgr2=0x7f801c106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:07.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.910+0000 7f8022b6d640 1 --2- 172.21.15.116:0/2370459710 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f801c106ba0 0x7f801c106f80 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f8010009900 tx=0x7f801002a410 comp rx=0 tx=0).stop 2024-09-12T03:14:07.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.910+0000 7f8022b6d640 1 -- 172.21.15.116:0/2370459710 shutdown_connections 2024-09-12T03:14:07.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.910+0000 7f8022b6d640 1 --2- 172.21.15.116:0/2370459710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f801c100a70 0x7f801c10ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.910+0000 7f8022b6d640 1 --2- 172.21.15.116:0/2370459710 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f801c1000d0 0x7f801c100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.910+0000 7f8022b6d640 1 --2- 172.21.15.116:0/2370459710 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f801c106ba0 0x7f801c106f80 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.910+0000 7f8022b6d640 1 -- 172.21.15.116:0/2370459710 >> 172.21.15.116:0/2370459710 conn(0x7f801c077210 msgr2=0x7f801c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:07.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.910+0000 7f8022b6d640 1 -- 172.21.15.116:0/2370459710 shutdown_connections 2024-09-12T03:14:07.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.910+0000 7f8022b6d640 1 -- 172.21.15.116:0/2370459710 wait complete. 2024-09-12T03:14:07.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.910+0000 7f8022b6d640 1 Processor -- start 2024-09-12T03:14:07.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.911+0000 7f8022b6d640 1 -- start start 2024-09-12T03:14:07.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.911+0000 7f8022b6d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f801c1000d0 0x7f801c1012b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:07.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.911+0000 7f8022b6d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f801c100a70 0x7f801c1017f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:07.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.911+0000 7f8022b6d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f801c106ba0 0x7f801c101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:07.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.911+0000 7f8022b6d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f801c112090 con 0x7f801c100a70 2024-09-12T03:14:07.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.911+0000 7f8022b6d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f801c111f10 con 0x7f801c106ba0 2024-09-12T03:14:07.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.911+0000 7f8022b6d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f801c112210 con 0x7f801c1000d0 2024-09-12T03:14:07.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.911+0000 7f801bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f801c100a70 0x7f801c1017f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:07.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.911+0000 7f801bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f801c100a70 0x7f801c1017f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34334/0 (socket says 172.21.15.116:34334) 2024-09-12T03:14:07.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.911+0000 7f801bfff640 1 -- 172.21.15.116:0/3416249627 learned_addr learned my addr 172.21.15.116:0/3416249627 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:07.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.911+0000 7f8020909640 1 --2- 172.21.15.116:0/3416249627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f801c1000d0 0x7f801c1012b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:07.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.911+0000 7f802110a640 1 --2- 172.21.15.116:0/3416249627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f801c106ba0 0x7f801c101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:07.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.911+0000 7f801bfff640 1 -- 172.21.15.116:0/3416249627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f801c1000d0 msgr2=0x7f801c1012b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:07.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.911+0000 7f801bfff640 1 --2- 172.21.15.116:0/3416249627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f801c1000d0 0x7f801c1012b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.911+0000 7f801bfff640 1 -- 172.21.15.116:0/3416249627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f801c106ba0 msgr2=0x7f801c101d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:07.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.911+0000 7f801bfff640 1 --2- 172.21.15.116:0/3416249627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f801c106ba0 0x7f801c101d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.911+0000 7f801bfff640 1 -- 172.21.15.116:0/3416249627 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801c1a6970 con 0x7f801c100a70 2024-09-12T03:14:07.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.911+0000 7f801bfff640 1 --2- 172.21.15.116:0/3416249627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f801c100a70 0x7f801c1017f0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f800800d950 tx=0x7f800800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:07.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.912+0000 7f8019ffb640 1 -- 172.21.15.116:0/3416249627 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8008017070 con 0x7f801c100a70 2024-09-12T03:14:07.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.912+0000 7f8022b6d640 1 -- 172.21.15.116:0/3416249627 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f801c1a6c60 con 0x7f801c100a70 2024-09-12T03:14:07.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.912+0000 7f8019ffb640 1 -- 172.21.15.116:0/3416249627 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8008009c30 con 0x7f801c100a70 2024-09-12T03:14:07.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.912+0000 7f8019ffb640 1 -- 172.21.15.116:0/3416249627 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800800f6b0 con 0x7f801c100a70 2024-09-12T03:14:07.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.912+0000 7f8022b6d640 1 -- 172.21.15.116:0/3416249627 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f801c1a71a0 con 0x7f801c100a70 2024-09-12T03:14:07.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.914+0000 7f8019ffb640 1 -- 172.21.15.116:0/3416249627 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f80080109d0 con 0x7f801c100a70 2024-09-12T03:14:07.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.914+0000 7f8024f46740 1 Processor -- start 2024-09-12T03:14:07.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.914+0000 7f8024f46740 1 -- start start 2024-09-12T03:14:07.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.914+0000 7f8019ffb640 1 --2- 172.21.15.116:0/3416249627 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f800c0a2b80 0x7f800c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:07.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.915+0000 7f8019ffb640 1 -- 172.21.15.116:0/3416249627 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f80080cebb0 con 0x7f801c100a70 2024-09-12T03:14:07.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.915+0000 7f802110a640 1 --2- 172.21.15.116:0/3416249627 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f800c0a2b80 0x7f800c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:07.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.915+0000 7f8024f46740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5565e39d0b40 0x5565e39cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:07.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.915+0000 7f8024f46740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565e39cf5d0 0x5565e39cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:07.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.915+0000 7f8024f46740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565e39da370 0x5565e39dc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:07.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.915+0000 7f8024f46740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5565e39352f0 con 0x5565e39d0b40 2024-09-12T03:14:07.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.915+0000 7f8024f46740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5565e38dccf0 con 0x5565e39cf5d0 2024-09-12T03:14:07.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.915+0000 7f8024f46740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5565e39312b0 con 0x5565e39da370 2024-09-12T03:14:07.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.915+0000 7f8020909640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5565e39d0b40 0x5565e39cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:07.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.915+0000 7f8020909640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5565e39d0b40 0x5565e39cf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34336/0 (socket says 172.21.15.116:34336) 2024-09-12T03:14:07.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.915+0000 7f8020909640 1 -- 172.21.15.116:0/1205109777 learned_addr learned my addr 172.21.15.116:0/1205109777 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:07.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.915+0000 7f801bfff640 1 --2- 172.21.15.116:0/1205109777 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565e39cf5d0 0x5565e39cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:07.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.915+0000 7f802110a640 1 --2- 172.21.15.116:0/1205109777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565e39da370 0x5565e39dc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:07.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.915+0000 7f802110a640 1 --2- 172.21.15.116:0/3416249627 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f800c0a2b80 0x7f800c0a5040 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8014005fd0 tx=0x7f8014009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:07.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.915+0000 7f8020909640 1 -- 172.21.15.116:0/1205109777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565e39da370 msgr2=0x5565e39dc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:07.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.915+0000 7f8020909640 1 --2- 172.21.15.116:0/1205109777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565e39da370 0x5565e39dc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.915+0000 7f8020909640 1 -- 172.21.15.116:0/1205109777 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565e39cf5d0 msgr2=0x5565e39cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:07.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.915+0000 7f8020909640 1 --2- 172.21.15.116:0/1205109777 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565e39cf5d0 0x5565e39cfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.915+0000 7f8020909640 1 -- 172.21.15.116:0/1205109777 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565e38c13a0 con 0x5565e39d0b40 2024-09-12T03:14:07.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.915+0000 7f802110a640 1 --2- 172.21.15.116:0/1205109777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565e39da370 0x5565e39dc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:07.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.915+0000 7f801bfff640 1 --2- 172.21.15.116:0/1205109777 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565e39cf5d0 0x5565e39cfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:07.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.915+0000 7f8020909640 1 --2- 172.21.15.116:0/1205109777 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5565e39d0b40 0x5565e39cf090 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f801002a930 tx=0x7f80100040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f1e85689d331d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:07.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.916+0000 7f8006ffd640 1 -- 172.21.15.116:0/1205109777 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8010035030 con 0x5565e39d0b40 2024-09-12T03:14:07.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.916+0000 7f8006ffd640 1 -- 172.21.15.116:0/1205109777 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8010040070 con 0x5565e39d0b40 2024-09-12T03:14:07.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.916+0000 7f8006ffd640 1 -- 172.21.15.116:0/1205109777 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80100319d0 con 0x5565e39d0b40 2024-09-12T03:14:07.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.916+0000 7f8024f46740 1 -- 172.21.15.116:0/1205109777 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5565e39d0b40 msgr2=0x5565e39cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:07.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.916+0000 7f8024f46740 1 --2- 172.21.15.116:0/1205109777 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5565e39d0b40 0x5565e39cf090 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f801002a930 tx=0x7f80100040c0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.916+0000 7f8024f46740 1 -- 172.21.15.116:0/1205109777 shutdown_connections 2024-09-12T03:14:07.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.916+0000 7f8024f46740 1 --2- 172.21.15.116:0/1205109777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565e39da370 0x5565e39dc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.916+0000 7f8024f46740 1 --2- 172.21.15.116:0/1205109777 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565e39cf5d0 0x5565e39cfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.916+0000 7f8024f46740 1 --2- 172.21.15.116:0/1205109777 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5565e39d0b40 0x5565e39cf090 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.916+0000 7f8024f46740 1 -- 172.21.15.116:0/1205109777 >> 172.21.15.116:0/1205109777 conn(0x5565e3998310 msgr2=0x5565e39cbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:07.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.916+0000 7f8024f46740 1 -- 172.21.15.116:0/1205109777 shutdown_connections 2024-09-12T03:14:07.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.916+0000 7f8024f46740 1 -- 172.21.15.116:0/1205109777 wait complete. 2024-09-12T03:14:07.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.917+0000 7f8024f46740 1 -- start start 2024-09-12T03:14:07.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.917+0000 7f8024f46740 1 Processor -- start 2024-09-12T03:14:07.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.917+0000 7f8024f46740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565e39cf5d0 0x5565e39d9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:07.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.917+0000 7f8024f46740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565e39d0b40 0x5565e39d6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:07.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.917+0000 7f8024f46740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5565e39da370 0x5565e39d7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:07.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.917+0000 7f8024f46740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5565e39dd010 con 0x5565e39da370 2024-09-12T03:14:07.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.917+0000 7f8024f46740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5565e38f0060 con 0x5565e39cf5d0 2024-09-12T03:14:07.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.917+0000 7f8024f46740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5565e39dd190 con 0x5565e39d0b40 2024-09-12T03:14:07.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.917+0000 7f8024f46740 10 client.0 _ll_register_callbacks cb 0x5565e36438c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:07.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.918+0000 7f802110a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5565e39da370 0x5565e39d7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:07.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.918+0000 7f802110a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5565e39da370 0x5565e39d7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34348/0 (socket says 172.21.15.116:34348) 2024-09-12T03:14:07.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.918+0000 7f802110a640 1 -- 172.21.15.116:0/1759345309 learned_addr learned my addr 172.21.15.116:0/1759345309 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:07.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.918+0000 7f8020909640 1 --2- 172.21.15.116:0/1759345309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565e39cf5d0 0x5565e39d9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:07.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.918+0000 7f802110a640 1 -- 172.21.15.116:0/1759345309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565e39d0b40 msgr2=0x5565e39d6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:07.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.918+0000 7f801bfff640 1 --2- 172.21.15.116:0/1759345309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565e39d0b40 0x5565e39d6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:07.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.918+0000 7f802110a640 1 --2- 172.21.15.116:0/1759345309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565e39d0b40 0x5565e39d6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.918+0000 7f802110a640 1 -- 172.21.15.116:0/1759345309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565e39cf5d0 msgr2=0x5565e39d9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:07.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.918+0000 7f802110a640 1 --2- 172.21.15.116:0/1759345309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565e39cf5d0 0x5565e39d9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.918+0000 7f802110a640 1 -- 172.21.15.116:0/1759345309 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565e36a63c0 con 0x5565e39da370 2024-09-12T03:14:07.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.918+0000 7f801bfff640 1 --2- 172.21.15.116:0/1759345309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565e39d0b40 0x5565e39d6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:14:07.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.918+0000 7f802110a640 1 --2- 172.21.15.116:0/1759345309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5565e39da370 0x5565e39d7560 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f8014012c70 tx=0x7f8014004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1636c0667313e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:07.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.918+0000 7f7ffbfff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:14:07.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.918+0000 7f7ffbfff640 1 -- 172.21.15.116:0/1759345309 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8014020630 con 0x5565e39da370 2024-09-12T03:14:07.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.918+0000 7f7ffbfff640 1 -- 172.21.15.116:0/1759345309 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f80140207d0 con 0x5565e39da370 2024-09-12T03:14:07.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.918+0000 7f8024f46740 1 -- 172.21.15.116:0/1759345309 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8010040070 con 0x5565e39da370 2024-09-12T03:14:07.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.918+0000 7f7ffbfff640 1 -- 172.21.15.116:0/1759345309 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f80140209d0 con 0x5565e39da370 2024-09-12T03:14:07.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.918+0000 7f8024f46740 10 client.18432 fetch_fsmap learned FSMap version 25 2024-09-12T03:14:07.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.918+0000 7f8024f46740 1 -- 172.21.15.116:0/1759345309 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5565e39e8b10 con 0x5565e39da370 2024-09-12T03:14:07.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.919+0000 7f7ffbfff640 1 -- 172.21.15.116:0/1759345309 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8010040070 con 0x5565e39da370 2024-09-12T03:14:07.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.919+0000 7f7ffbfff640 1 -- 172.21.15.116:0/1759345309 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f801400f040 con 0x5565e39da370 2024-09-12T03:14:07.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.919+0000 7f8024f46740 10 client.18432 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:14:07.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.919+0000 7f8024f46740 10 client.18432 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:14:07.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.919+0000 7f8024f46740 20 client.18432 populate_metadata read hostname 'smithi116' 2024-09-12T03:14:07.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.919+0000 7f8024f46740 1 --2- 172.21.15.116:0/1759345309 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5565e39e9f20 0x5565e39ec370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:07.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.919+0000 7f8024f46740 4 client.18432 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:14:07.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.919+0000 7f8024f46740 1 -- 172.21.15.116:0/1759345309 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565e3942860 con 0x5565e39e9f20 2024-09-12T03:14:07.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.919+0000 7f8020909640 1 --2- 172.21.15.116:0/1759345309 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5565e39e9f20 0x5565e39ec370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:07.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.920+0000 7f7ffbfff640 10 client.18432 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:14:07.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.920+0000 7f8020909640 1 --2- 172.21.15.116:0/1759345309 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5565e39e9f20 0x5565e39ec370 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5f636ed074742236 server_cookie=c31785ea6e56cc3e in_seq=0 out_seq=0 2024-09-12T03:14:07.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.920+0000 7f7ffbfff640 1 -- 172.21.15.116:0/1759345309 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5565e3942860 con 0x5565e39e9f20 2024-09-12T03:14:07.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.920+0000 7f7ffbfff640 10 client.18432 handle_command_reply: tid=0 2024-09-12T03:14:07.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.920+0000 7f8024f46740 1 client.18432 shutdown 2024-09-12T03:14:07.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.920+0000 7f8024f46740 1 -- 172.21.15.116:0/1759345309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5565e39da370 msgr2=0x5565e39d7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:07.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.920+0000 7f8024f46740 1 --2- 172.21.15.116:0/1759345309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5565e39da370 0x5565e39d7560 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f8014012c70 tx=0x7f8014004a40 comp rx=0 tx=0).stop 2024-09-12T03:14:07.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.920+0000 7f8024f46740 1 -- 172.21.15.116:0/1759345309 shutdown_connections 2024-09-12T03:14:07.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.920+0000 7f8024f46740 1 --2- 172.21.15.116:0/1759345309 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5565e39e9f20 0x5565e39ec370 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.920+0000 7f8024f46740 1 --2- 172.21.15.116:0/1759345309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5565e39da370 0x5565e39d7560 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.920+0000 7f8024f46740 1 --2- 172.21.15.116:0/1759345309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565e39d0b40 0x5565e39d6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.920+0000 7f8024f46740 1 --2- 172.21.15.116:0/1759345309 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565e39cf5d0 0x5565e39d9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.920+0000 7f8024f46740 1 -- 172.21.15.116:0/1759345309 >> 172.21.15.116:0/1759345309 conn(0x5565e3998310 msgr2=0x5565e39d1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:07.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.921+0000 7f7ffbfff640 0 client.18432 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:14:07.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.921+0000 7f8024f46740 1 -- 172.21.15.116:0/1759345309 shutdown_connections 2024-09-12T03:14:07.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.921+0000 7f8024f46740 1 -- 172.21.15.116:0/1759345309 wait complete. 2024-09-12T03:14:07.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.921+0000 7f8024f46740 20 client.18432 trim_cache size 0 max 16384 2024-09-12T03:14:07.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.933+0000 7f8024f46740 1 Processor -- start 2024-09-12T03:14:07.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.933+0000 7f8024f46740 1 -- start start 2024-09-12T03:14:07.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.934+0000 7f8024f46740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565e39cf5d0 0x5565e39e62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:07.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.934+0000 7f8024f46740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5565e39d0b40 0x5565e39d2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:07.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.934+0000 7f8024f46740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565e39da370 0x5565e39d2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:07.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.934+0000 7f8024f46740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f801c1b1f40 con 0x5565e39d0b40 2024-09-12T03:14:07.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.934+0000 7f8024f46740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f801c1af810 con 0x5565e39da370 2024-09-12T03:14:07.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.934+0000 7f8024f46740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f801c1ad660 con 0x5565e39cf5d0 2024-09-12T03:14:07.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.934+0000 7f801bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5565e39d0b40 0x5565e39d2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:07.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.934+0000 7f801bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5565e39d0b40 0x5565e39d2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34352/0 (socket says 172.21.15.116:34352) 2024-09-12T03:14:07.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.934+0000 7f801bfff640 1 -- 172.21.15.116:0/3801106274 learned_addr learned my addr 172.21.15.116:0/3801106274 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:07.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.934+0000 7f8020909640 1 --2- 172.21.15.116:0/3801106274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565e39cf5d0 0x5565e39e62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:07.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.934+0000 7f801bfff640 1 -- 172.21.15.116:0/3801106274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565e39cf5d0 msgr2=0x5565e39e62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:07.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.934+0000 7f801bfff640 1 --2- 172.21.15.116:0/3801106274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565e39cf5d0 0x5565e39e62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.934+0000 7f801bfff640 1 -- 172.21.15.116:0/3801106274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565e39da370 msgr2=0x5565e39d2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:07.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.934+0000 7f801bfff640 1 --2- 172.21.15.116:0/3801106274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565e39da370 0x5565e39d2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.934+0000 7f801bfff640 1 -- 172.21.15.116:0/3801106274 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565e39d9930 con 0x5565e39d0b40 2024-09-12T03:14:07.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.934+0000 7f801bfff640 1 --2- 172.21.15.116:0/3801106274 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5565e39d0b40 0x5565e39d2120 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f80080cc6e0 tx=0x7f8008005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa3d5df3bd002df server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:07.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.934+0000 7f80067fc640 1 -- 172.21.15.116:0/3801106274 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8008020040 con 0x5565e39d0b40 2024-09-12T03:14:07.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.934+0000 7f80067fc640 1 -- 172.21.15.116:0/3801106274 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f800809f7f0 con 0x5565e39d0b40 2024-09-12T03:14:07.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.934+0000 7f80067fc640 1 -- 172.21.15.116:0/3801106274 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800809a940 con 0x5565e39d0b40 2024-09-12T03:14:07.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.935+0000 7f8024f46740 1 -- 172.21.15.116:0/3801106274 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5565e39d0b40 msgr2=0x5565e39d2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:07.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.935+0000 7f8024f46740 1 --2- 172.21.15.116:0/3801106274 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5565e39d0b40 0x5565e39d2120 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f80080cc6e0 tx=0x7f8008005880 comp rx=0 tx=0).stop 2024-09-12T03:14:07.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.935+0000 7f8024f46740 1 -- 172.21.15.116:0/3801106274 shutdown_connections 2024-09-12T03:14:07.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.935+0000 7f8024f46740 1 --2- 172.21.15.116:0/3801106274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565e39da370 0x5565e39d2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.935+0000 7f8024f46740 1 --2- 172.21.15.116:0/3801106274 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5565e39d0b40 0x5565e39d2120 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.935+0000 7f8024f46740 1 --2- 172.21.15.116:0/3801106274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565e39cf5d0 0x5565e39e62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.935+0000 7f8024f46740 1 -- 172.21.15.116:0/3801106274 >> 172.21.15.116:0/3801106274 conn(0x5565e3998310 msgr2=0x5565e39d6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:07.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.935+0000 7f8024f46740 1 -- 172.21.15.116:0/3801106274 shutdown_connections 2024-09-12T03:14:07.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.935+0000 7f8024f46740 1 -- 172.21.15.116:0/3801106274 wait complete. 2024-09-12T03:14:07.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.935+0000 7f8024f46740 1 -- start start 2024-09-12T03:14:07.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.935+0000 7f8024f46740 1 Processor -- start 2024-09-12T03:14:07.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.936+0000 7f8024f46740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565e39cf5d0 0x5565e39e2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:07.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.936+0000 7f8024f46740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565e39d0b40 0x5565e39d3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:07.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.936+0000 7f8024f46740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5565e39da370 0x5565e39d4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:07.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.936+0000 7f8024f46740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f801c1adcd0 con 0x5565e39da370 2024-09-12T03:14:07.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.936+0000 7f8024f46740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f801c1af140 con 0x5565e39d0b40 2024-09-12T03:14:07.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.936+0000 7f8024f46740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f801c1ae3a0 con 0x5565e39cf5d0 2024-09-12T03:14:07.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.936+0000 7f8024f46740 10 client.0 _ll_register_callbacks cb 0x7f8014016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:07.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.936+0000 7f802110a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5565e39da370 0x5565e39d4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:07.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.936+0000 7f8020909640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565e39cf5d0 0x5565e39e2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:07.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.936+0000 7f802110a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5565e39da370 0x5565e39d4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34364/0 (socket says 172.21.15.116:34364) 2024-09-12T03:14:07.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.936+0000 7f8020909640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565e39cf5d0 0x5565e39e2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.116:56612/0 (socket says 172.21.15.116:56612) 2024-09-12T03:14:07.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.936+0000 7f802110a640 1 -- 172.21.15.116:0/614760035 learned_addr learned my addr 172.21.15.116:0/614760035 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:07.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.936+0000 7f801bfff640 1 --2- 172.21.15.116:0/614760035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565e39d0b40 0x5565e39d3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:07.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.936+0000 7f802110a640 1 -- 172.21.15.116:0/614760035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565e39cf5d0 msgr2=0x5565e39e2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:07.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.936+0000 7f802110a640 1 --2- 172.21.15.116:0/614760035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565e39cf5d0 0x5565e39e2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.936+0000 7f802110a640 1 -- 172.21.15.116:0/614760035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565e39d0b40 msgr2=0x5565e39d3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:07.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.936+0000 7f802110a640 1 --2- 172.21.15.116:0/614760035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565e39d0b40 0x5565e39d3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.936+0000 7f802110a640 1 -- 172.21.15.116:0/614760035 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565e39d4630 con 0x5565e39da370 2024-09-12T03:14:07.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.936+0000 7f801bfff640 1 --2- 172.21.15.116:0/614760035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565e39d0b40 0x5565e39d3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:07.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.936+0000 7f802110a640 1 --2- 172.21.15.116:0/614760035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5565e39da370 0x5565e39d4070 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f8014005cc0 tx=0x7f80140096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507f3f5f62378946 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:07.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.936+0000 7f7ffbfff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:14:07.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.937+0000 7f7ffbfff640 1 -- 172.21.15.116:0/614760035 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80140127f0 con 0x5565e39da370 2024-09-12T03:14:07.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.937+0000 7f7ffbfff640 1 -- 172.21.15.116:0/614760035 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f8014020d80 con 0x5565e39da370 2024-09-12T03:14:07.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.937+0000 7f8024f46740 1 -- 172.21.15.116:0/614760035 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f800809f7f0 con 0x5565e39da370 2024-09-12T03:14:07.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.937+0000 7f7ffbfff640 1 -- 172.21.15.116:0/614760035 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8014044050 con 0x5565e39da370 2024-09-12T03:14:07.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.937+0000 7f8024f46740 10 client.18438 fetch_fsmap learned FSMap version 25 2024-09-12T03:14:07.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.937+0000 7f8024f46740 1 -- 172.21.15.116:0/614760035 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5565e3a549e0 con 0x5565e39da370 2024-09-12T03:14:07.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.937+0000 7f7ffbfff640 1 -- 172.21.15.116:0/614760035 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f800809f7f0 con 0x5565e39da370 2024-09-12T03:14:07.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.937+0000 7f7ffbfff640 1 -- 172.21.15.116:0/614760035 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8014003b00 con 0x5565e39da370 2024-09-12T03:14:07.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.937+0000 7f8024f46740 10 client.18438 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:14:07.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.937+0000 7f8024f46740 10 client.18438 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:14:07.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.937+0000 7f8024f46740 20 client.18438 populate_metadata read hostname 'smithi116' 2024-09-12T03:14:07.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.937+0000 7f8024f46740 1 --2- 172.21.15.116:0/614760035 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5565e3a55180 0x5565e3a575a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:07.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.937+0000 7f8024f46740 4 client.18438 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:14:07.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.937+0000 7f8024f46740 1 -- 172.21.15.116:0/614760035 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5565e39ec700 con 0x5565e3a55180 2024-09-12T03:14:07.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.937+0000 7f8020909640 1 --2- 172.21.15.116:0/614760035 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5565e3a55180 0x5565e3a575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:07.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.938+0000 7f8020909640 1 --2- 172.21.15.116:0/614760035 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5565e3a55180 0x5565e3a575a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=65dfb1626a705eb7 server_cookie=695f2235a402f84b in_seq=0 out_seq=0 2024-09-12T03:14:07.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.938+0000 7f7ffbfff640 10 client.18438 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:14:07.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.938+0000 7f7ffbfff640 1 -- 172.21.15.116:0/614760035 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5565e39ec700 con 0x5565e3a55180 2024-09-12T03:14:07.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.938+0000 7f7ffbfff640 10 client.18438 handle_command_reply: tid=0 2024-09-12T03:14:07.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.938+0000 7f8024f46740 1 client.18438 shutdown 2024-09-12T03:14:07.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.938+0000 7f8024f46740 1 -- 172.21.15.116:0/614760035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5565e39da370 msgr2=0x5565e39d4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:07.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.938+0000 7f8024f46740 1 --2- 172.21.15.116:0/614760035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5565e39da370 0x5565e39d4070 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f8014005cc0 tx=0x7f80140096e0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.938+0000 7f8024f46740 1 -- 172.21.15.116:0/614760035 shutdown_connections 2024-09-12T03:14:07.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.938+0000 7f8024f46740 1 --2- 172.21.15.116:0/614760035 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5565e3a55180 0x5565e3a575a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.938+0000 7f8024f46740 1 --2- 172.21.15.116:0/614760035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5565e39da370 0x5565e39d4070 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.938+0000 7f8024f46740 1 --2- 172.21.15.116:0/614760035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5565e39d0b40 0x5565e39d3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.938+0000 7f8024f46740 1 --2- 172.21.15.116:0/614760035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565e39cf5d0 0x5565e39e2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.938+0000 7f7ffbfff640 0 client.18438 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:14:07.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.938+0000 7f8024f46740 1 -- 172.21.15.116:0/614760035 >> 172.21.15.116:0/614760035 conn(0x5565e3998310 msgr2=0x5565e39d8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:07.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.938+0000 7f8024f46740 1 -- 172.21.15.116:0/614760035 shutdown_connections 2024-09-12T03:14:07.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.938+0000 7f8024f46740 1 -- 172.21.15.116:0/614760035 wait complete. 2024-09-12T03:14:07.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.938+0000 7f8024f46740 20 client.18438 trim_cache size 0 max 16384 2024-09-12T03:14:07.945 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:14:07.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.939+0000 7f8022b6d640 1 -- 172.21.15.116:0/3416249627 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f800c0a2b80 msgr2=0x7f800c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:07.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.939+0000 7f8022b6d640 1 --2- 172.21.15.116:0/3416249627 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f800c0a2b80 0x7f800c0a5040 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8014005fd0 tx=0x7f8014009600 comp rx=0 tx=0).stop 2024-09-12T03:14:07.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.939+0000 7f8022b6d640 1 -- 172.21.15.116:0/3416249627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f801c100a70 msgr2=0x7f801c1017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:07.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.939+0000 7f8022b6d640 1 --2- 172.21.15.116:0/3416249627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f801c100a70 0x7f801c1017f0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f800800d950 tx=0x7f800800de10 comp rx=0 tx=0).stop 2024-09-12T03:14:07.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.939+0000 7f8022b6d640 1 -- 172.21.15.116:0/3416249627 shutdown_connections 2024-09-12T03:14:07.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.939+0000 7f8022b6d640 1 --2- 172.21.15.116:0/3416249627 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f800c0a2b80 0x7f800c0a5040 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.939+0000 7f8022b6d640 1 --2- 172.21.15.116:0/3416249627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f801c106ba0 0x7f801c101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.939+0000 7f8022b6d640 1 --2- 172.21.15.116:0/3416249627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f801c100a70 0x7f801c1017f0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.939+0000 7f8022b6d640 1 --2- 172.21.15.116:0/3416249627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f801c1000d0 0x7f801c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:07.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.939+0000 7f8022b6d640 1 -- 172.21.15.116:0/3416249627 >> 172.21.15.116:0/3416249627 conn(0x7f801c077210 msgr2=0x7f801c105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:07.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.939+0000 7f8022b6d640 1 -- 172.21.15.116:0/3416249627 shutdown_connections 2024-09-12T03:14:07.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:07.939+0000 7f8022b6d640 1 -- 172.21.15.116:0/3416249627 wait complete. 2024-09-12T03:14:08.858 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:14:09.105 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:14:09.514 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:09 smithi116 ceph-mon[31410]: pgmap v673: 97 pgs: 97 active+clean; 5.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-12T03:14:09.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.595+0000 7f0b86b43640 1 -- 172.21.15.116:0/479584000 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b801008e0 msgr2=0x7f0b80100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:09.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.595+0000 7f0b86b43640 1 --2- 172.21.15.116:0/479584000 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b801008e0 0x7f0b80100cc0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f0b74009930 tx=0x7f0b7402a420 comp rx=0 tx=0).stop 2024-09-12T03:14:09.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.595+0000 7f0b86b43640 1 -- 172.21.15.116:0/479584000 shutdown_connections 2024-09-12T03:14:09.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.595+0000 7f0b86b43640 1 --2- 172.21.15.116:0/479584000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b80108160 0x7f0b8010eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.595+0000 7f0b86b43640 1 --2- 172.21.15.116:0/479584000 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b80101290 0x7f0b80107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.595+0000 7f0b86b43640 1 --2- 172.21.15.116:0/479584000 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b801008e0 0x7f0b80100cc0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.595+0000 7f0b86b43640 1 -- 172.21.15.116:0/479584000 >> 172.21.15.116:0/479584000 conn(0x7f0b80077210 msgr2=0x7f0b80077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:09.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.595+0000 7f0b86b43640 1 -- 172.21.15.116:0/479584000 shutdown_connections 2024-09-12T03:14:09.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.595+0000 7f0b86b43640 1 -- 172.21.15.116:0/479584000 wait complete. 2024-09-12T03:14:09.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.596+0000 7f0b86b43640 1 Processor -- start 2024-09-12T03:14:09.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.596+0000 7f0b86b43640 1 -- start start 2024-09-12T03:14:09.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.596+0000 7f0b86b43640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b801008e0 0x7f0b801984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:09.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.596+0000 7f0b86b43640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b80101290 0x7f0b80198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:09.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.596+0000 7f0b86b43640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b80108160 0x7f0b8019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:09.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.596+0000 7f0b86b43640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0b80111ff0 con 0x7f0b80108160 2024-09-12T03:14:09.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.596+0000 7f0b86b43640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0b80111e70 con 0x7f0b80101290 2024-09-12T03:14:09.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.596+0000 7f0b86b43640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0b80112170 con 0x7f0b801008e0 2024-09-12T03:14:09.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.596+0000 7f0b850e0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b80108160 0x7f0b8019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:09.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.596+0000 7f0b850e0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b80108160 0x7f0b8019cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34382/0 (socket says 172.21.15.116:34382) 2024-09-12T03:14:09.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.596+0000 7f0b850e0640 1 -- 172.21.15.116:0/84041597 learned_addr learned my addr 172.21.15.116:0/84041597 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:09.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.596+0000 7f0b848df640 1 --2- 172.21.15.116:0/84041597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b801008e0 0x7f0b801984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:09.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.596+0000 7f0b7ffff640 1 --2- 172.21.15.116:0/84041597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b80101290 0x7f0b80198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:09.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.596+0000 7f0b850e0640 1 -- 172.21.15.116:0/84041597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b801008e0 msgr2=0x7f0b801984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:09.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.596+0000 7f0b850e0640 1 --2- 172.21.15.116:0/84041597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b801008e0 0x7f0b801984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.596+0000 7f0b850e0640 1 -- 172.21.15.116:0/84041597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b80101290 msgr2=0x7f0b80198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:09.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.596+0000 7f0b850e0640 1 --2- 172.21.15.116:0/84041597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b80101290 0x7f0b80198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.596+0000 7f0b850e0640 1 -- 172.21.15.116:0/84041597 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8019d480 con 0x7f0b80108160 2024-09-12T03:14:09.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.596+0000 7f0b7ffff640 1 --2- 172.21.15.116:0/84041597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b80101290 0x7f0b80198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:09.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.597+0000 7f0b850e0640 1 --2- 172.21.15.116:0/84041597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b80108160 0x7f0b8019cda0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800eaa0 tx=0x7f0b7800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:09.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.597+0000 7f0b7dffb640 1 -- 172.21.15.116:0/84041597 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7800cdb0 con 0x7f0b80108160 2024-09-12T03:14:09.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.597+0000 7f0b7dffb640 1 -- 172.21.15.116:0/84041597 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b78005020 con 0x7f0b80108160 2024-09-12T03:14:09.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.597+0000 7f0b86b43640 1 -- 172.21.15.116:0/84041597 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b8019d770 con 0x7f0b80108160 2024-09-12T03:14:09.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.597+0000 7f0b7dffb640 1 -- 172.21.15.116:0/84041597 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b780116b0 con 0x7f0b80108160 2024-09-12T03:14:09.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.597+0000 7f0b86b43640 1 -- 172.21.15.116:0/84041597 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b801a5050 con 0x7f0b80108160 2024-09-12T03:14:09.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.599+0000 7f0b7dffb640 1 -- 172.21.15.116:0/84041597 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f0b78005560 con 0x7f0b80108160 2024-09-12T03:14:09.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.600+0000 7f0b7dffb640 1 --2- 172.21.15.116:0/84041597 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0b700a2c00 0x7f0b700a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:09.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.600+0000 7f0b88f1c740 1 Processor -- start 2024-09-12T03:14:09.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.600+0000 7f0b88f1c740 1 -- start start 2024-09-12T03:14:09.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.600+0000 7f0b848df640 1 --2- 172.21.15.116:0/84041597 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0b700a2c00 0x7f0b700a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:09.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.600+0000 7f0b7dffb640 1 -- 172.21.15.116:0/84041597 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f0b780cf140 con 0x7f0b80108160 2024-09-12T03:14:09.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.600+0000 7f0b88f1c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a8a70b40 0x5626a8a6f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:09.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.600+0000 7f0b88f1c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626a8a6f5d0 0x5626a8a6fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:09.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.600+0000 7f0b88f1c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626a8a7a370 0x5626a8a7c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:09.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.600+0000 7f0b88f1c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5626a89d52f0 con 0x5626a8a7a370 2024-09-12T03:14:09.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.600+0000 7f0b88f1c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5626a897ccf0 con 0x5626a8a6f5d0 2024-09-12T03:14:09.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.600+0000 7f0b88f1c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5626a89d12b0 con 0x5626a8a70b40 2024-09-12T03:14:09.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.600+0000 7f0b850e0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626a8a7a370 0x5626a8a7c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:09.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.600+0000 7f0b850e0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626a8a7a370 0x5626a8a7c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34386/0 (socket says 172.21.15.116:34386) 2024-09-12T03:14:09.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.600+0000 7f0b850e0640 1 -- 172.21.15.116:0/2580904533 learned_addr learned my addr 172.21.15.116:0/2580904533 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:09.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.600+0000 7f0b848df640 1 --2- 172.21.15.116:0/84041597 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0b700a2c00 0x7f0b700a50c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0b74009900 tx=0x7f0b740058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:09.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.600+0000 7f0b848df640 1 --2- 172.21.15.116:0/2580904533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a8a70b40 0x5626a8a6f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:09.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.600+0000 7f0b7ffff640 1 --2- 172.21.15.116:0/2580904533 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626a8a6f5d0 0x5626a8a6fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:09.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.600+0000 7f0b850e0640 1 -- 172.21.15.116:0/2580904533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a8a70b40 msgr2=0x5626a8a6f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:09.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.600+0000 7f0b850e0640 1 --2- 172.21.15.116:0/2580904533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a8a70b40 0x5626a8a6f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.600+0000 7f0b850e0640 1 -- 172.21.15.116:0/2580904533 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626a8a6f5d0 msgr2=0x5626a8a6fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:09.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.600+0000 7f0b850e0640 1 --2- 172.21.15.116:0/2580904533 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626a8a6f5d0 0x5626a8a6fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.600+0000 7f0b850e0640 1 -- 172.21.15.116:0/2580904533 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626a89613a0 con 0x5626a8a7a370 2024-09-12T03:14:09.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.601+0000 7f0b850e0640 1 --2- 172.21.15.116:0/2580904533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626a8a7a370 0x5626a8a7c760 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f0b7809b960 tx=0x7f0b7809c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a634c8b30df8546b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:09.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.601+0000 7f0b6affd640 1 -- 172.21.15.116:0/2580904533 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7801f040 con 0x5626a8a7a370 2024-09-12T03:14:09.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.601+0000 7f0b6affd640 1 -- 172.21.15.116:0/2580904533 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b78099cd0 con 0x5626a8a7a370 2024-09-12T03:14:09.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.601+0000 7f0b6affd640 1 -- 172.21.15.116:0/2580904533 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7809b040 con 0x5626a8a7a370 2024-09-12T03:14:09.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.601+0000 7f0b88f1c740 1 -- 172.21.15.116:0/2580904533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626a8a7a370 msgr2=0x5626a8a7c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:09.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.601+0000 7f0b88f1c740 1 --2- 172.21.15.116:0/2580904533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626a8a7a370 0x5626a8a7c760 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f0b7809b960 tx=0x7f0b7809c240 comp rx=0 tx=0).stop 2024-09-12T03:14:09.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.601+0000 7f0b88f1c740 1 -- 172.21.15.116:0/2580904533 shutdown_connections 2024-09-12T03:14:09.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.601+0000 7f0b88f1c740 1 --2- 172.21.15.116:0/2580904533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626a8a7a370 0x5626a8a7c760 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.601+0000 7f0b88f1c740 1 --2- 172.21.15.116:0/2580904533 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626a8a6f5d0 0x5626a8a6fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.601+0000 7f0b88f1c740 1 --2- 172.21.15.116:0/2580904533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a8a70b40 0x5626a8a6f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.601+0000 7f0b88f1c740 1 -- 172.21.15.116:0/2580904533 >> 172.21.15.116:0/2580904533 conn(0x5626a8a38310 msgr2=0x5626a8a6bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:09.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.602+0000 7f0b88f1c740 1 -- 172.21.15.116:0/2580904533 shutdown_connections 2024-09-12T03:14:09.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.602+0000 7f0b88f1c740 1 -- 172.21.15.116:0/2580904533 wait complete. 2024-09-12T03:14:09.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.602+0000 7f0b88f1c740 1 -- start start 2024-09-12T03:14:09.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.602+0000 7f0b88f1c740 1 Processor -- start 2024-09-12T03:14:09.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.602+0000 7f0b88f1c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626a8a6f5d0 0x5626a8a79dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:09.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.602+0000 7f0b88f1c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a8a70b40 0x5626a8a77080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:09.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.602+0000 7f0b88f1c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626a8a7a370 0x5626a8a77810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:09.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.602+0000 7f0b88f1c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5626a8a7d070 con 0x5626a8a6f5d0 2024-09-12T03:14:09.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.602+0000 7f0b88f1c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5626a8990060 con 0x5626a8a7a370 2024-09-12T03:14:09.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.602+0000 7f0b88f1c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5626a8a7d1f0 con 0x5626a8a70b40 2024-09-12T03:14:09.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.602+0000 7f0b88f1c740 10 client.0 _ll_register_callbacks cb 0x5626a86e38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:09.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.602+0000 7f0b848df640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626a8a6f5d0 0x5626a8a79dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:09.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.602+0000 7f0b848df640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626a8a6f5d0 0x5626a8a79dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34396/0 (socket says 172.21.15.116:34396) 2024-09-12T03:14:09.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.602+0000 7f0b848df640 1 -- 172.21.15.116:0/3652603668 learned_addr learned my addr 172.21.15.116:0/3652603668 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:09.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.603+0000 7f0b848df640 1 -- 172.21.15.116:0/3652603668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a8a70b40 msgr2=0x5626a8a77080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:09.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.603+0000 7f0b848df640 1 --2- 172.21.15.116:0/3652603668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a8a70b40 0x5626a8a77080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.603+0000 7f0b848df640 1 -- 172.21.15.116:0/3652603668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626a8a7a370 msgr2=0x5626a8a77810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:09.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.603+0000 7f0b850e0640 1 --2- 172.21.15.116:0/3652603668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626a8a7a370 0x5626a8a77810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:09.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.603+0000 7f0b848df640 1 --2- 172.21.15.116:0/3652603668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626a8a7a370 0x5626a8a77810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.603+0000 7f0b848df640 1 -- 172.21.15.116:0/3652603668 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626a87463c0 con 0x5626a8a6f5d0 2024-09-12T03:14:09.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.603+0000 7f0b848df640 1 --2- 172.21.15.116:0/3652603668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626a8a6f5d0 0x5626a8a79dd0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f0b74009930 tx=0x7f0b74009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d836d4900ea28fe5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:09.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.603+0000 7f0b5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:14:09.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.603+0000 7f0b5ffff640 1 -- 172.21.15.116:0/3652603668 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7404e070 con 0x5626a8a6f5d0 2024-09-12T03:14:09.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.603+0000 7f0b5ffff640 1 -- 172.21.15.116:0/3652603668 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f0b74003a00 con 0x5626a8a6f5d0 2024-09-12T03:14:09.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.603+0000 7f0b88f1c740 1 -- 172.21.15.116:0/3652603668 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b78099cd0 con 0x5626a8a6f5d0 2024-09-12T03:14:09.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.603+0000 7f0b5ffff640 1 -- 172.21.15.116:0/3652603668 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0b7405a050 con 0x5626a8a6f5d0 2024-09-12T03:14:09.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.603+0000 7f0b88f1c740 10 client.18450 fetch_fsmap learned FSMap version 25 2024-09-12T03:14:09.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.603+0000 7f0b88f1c740 1 -- 172.21.15.116:0/3652603668 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5626a8a88b90 con 0x5626a8a6f5d0 2024-09-12T03:14:09.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.603+0000 7f0b5ffff640 1 -- 172.21.15.116:0/3652603668 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b78099cd0 con 0x5626a8a6f5d0 2024-09-12T03:14:09.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.604+0000 7f0b5ffff640 1 -- 172.21.15.116:0/3652603668 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0b7404aca0 con 0x5626a8a6f5d0 2024-09-12T03:14:09.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.604+0000 7f0b88f1c740 10 client.18450 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:14:09.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.604+0000 7f0b88f1c740 10 client.18450 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:14:09.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.604+0000 7f0b88f1c740 20 client.18450 populate_metadata read hostname 'smithi116' 2024-09-12T03:14:09.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.604+0000 7f0b88f1c740 1 --2- 172.21.15.116:0/3652603668 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x5626a8a89ff0 0x5626a8a8c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:09.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.604+0000 7f0b88f1c740 4 client.18450 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:14:09.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.604+0000 7f0b88f1c740 1 -- 172.21.15.116:0/3652603668 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626a89e2860 con 0x5626a8a89ff0 2024-09-12T03:14:09.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.604+0000 7f0b7ffff640 1 --2- 172.21.15.116:0/3652603668 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x5626a8a89ff0 0x5626a8a8c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:09.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.604+0000 7f0b7ffff640 1 --2- 172.21.15.116:0/3652603668 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x5626a8a89ff0 0x5626a8a8c440 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b6771945b05ee897 server_cookie=b08ec3be9771d2dc in_seq=0 out_seq=0 2024-09-12T03:14:09.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.604+0000 7f0b5ffff640 10 client.18450 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:14:09.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.605+0000 7f0b5ffff640 1 -- 172.21.15.116:0/3652603668 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5626a89e2860 con 0x5626a8a89ff0 2024-09-12T03:14:09.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.605+0000 7f0b5ffff640 10 client.18450 handle_command_reply: tid=0 2024-09-12T03:14:09.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.605+0000 7f0b88f1c740 1 client.18450 shutdown 2024-09-12T03:14:09.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.605+0000 7f0b88f1c740 1 -- 172.21.15.116:0/3652603668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626a8a6f5d0 msgr2=0x5626a8a79dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:09.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.605+0000 7f0b88f1c740 1 --2- 172.21.15.116:0/3652603668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626a8a6f5d0 0x5626a8a79dd0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f0b74009930 tx=0x7f0b74009260 comp rx=0 tx=0).stop 2024-09-12T03:14:09.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.605+0000 7f0b88f1c740 1 -- 172.21.15.116:0/3652603668 shutdown_connections 2024-09-12T03:14:09.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.605+0000 7f0b88f1c740 1 --2- 172.21.15.116:0/3652603668 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x5626a8a89ff0 0x5626a8a8c440 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.605+0000 7f0b88f1c740 1 --2- 172.21.15.116:0/3652603668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626a8a7a370 0x5626a8a77810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.605+0000 7f0b88f1c740 1 --2- 172.21.15.116:0/3652603668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a8a70b40 0x5626a8a77080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.605+0000 7f0b88f1c740 1 --2- 172.21.15.116:0/3652603668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626a8a6f5d0 0x5626a8a79dd0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.605+0000 7f0b5ffff640 0 client.18450 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:14:09.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.605+0000 7f0b88f1c740 1 -- 172.21.15.116:0/3652603668 >> 172.21.15.116:0/3652603668 conn(0x5626a8a38310 msgr2=0x5626a8a71990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:09.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.606+0000 7f0b88f1c740 1 -- 172.21.15.116:0/3652603668 shutdown_connections 2024-09-12T03:14:09.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.606+0000 7f0b88f1c740 1 -- 172.21.15.116:0/3652603668 wait complete. 2024-09-12T03:14:09.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.606+0000 7f0b88f1c740 20 client.18450 trim_cache size 0 max 16384 2024-09-12T03:14:09.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.618+0000 7f0b88f1c740 1 Processor -- start 2024-09-12T03:14:09.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.618+0000 7f0b88f1c740 1 -- start start 2024-09-12T03:14:09.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.619+0000 7f0b88f1c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a8a6f5d0 0x5626a8a6abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:09.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.619+0000 7f0b88f1c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626a8a70b40 0x5626a8a82860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:09.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.619+0000 7f0b88f1c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626a8a7a370 0x5626a8a82da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:09.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.619+0000 7f0b88f1c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0b801afed0 con 0x5626a8a70b40 2024-09-12T03:14:09.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.619+0000 7f0b88f1c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0b801ad7a0 con 0x5626a8a7a370 2024-09-12T03:14:09.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.619+0000 7f0b88f1c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0b801ab5f0 con 0x5626a8a6f5d0 2024-09-12T03:14:09.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.619+0000 7f0b7ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626a8a70b40 0x5626a8a82860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:09.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.619+0000 7f0b7ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626a8a70b40 0x5626a8a82860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34408/0 (socket says 172.21.15.116:34408) 2024-09-12T03:14:09.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.619+0000 7f0b7ffff640 1 -- 172.21.15.116:0/210972301 learned_addr learned my addr 172.21.15.116:0/210972301 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:09.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.619+0000 7f0b848df640 1 --2- 172.21.15.116:0/210972301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a8a6f5d0 0x5626a8a6abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:09.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.619+0000 7f0b7ffff640 1 -- 172.21.15.116:0/210972301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a8a6f5d0 msgr2=0x5626a8a6abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:09.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.619+0000 7f0b7ffff640 1 --2- 172.21.15.116:0/210972301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a8a6f5d0 0x5626a8a6abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.619+0000 7f0b7ffff640 1 -- 172.21.15.116:0/210972301 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626a8a7a370 msgr2=0x5626a8a82da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:09.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.619+0000 7f0b850e0640 1 --2- 172.21.15.116:0/210972301 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626a8a7a370 0x5626a8a82da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:09.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.619+0000 7f0b7ffff640 1 --2- 172.21.15.116:0/210972301 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626a8a7a370 0x5626a8a82da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.619+0000 7f0b7ffff640 1 -- 172.21.15.116:0/210972301 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626a8a7cad0 con 0x5626a8a70b40 2024-09-12T03:14:09.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.619+0000 7f0b848df640 1 --2- 172.21.15.116:0/210972301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a8a6f5d0 0x5626a8a6abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:14:09.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.619+0000 7f0b7ffff640 1 --2- 172.21.15.116:0/210972301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626a8a70b40 0x5626a8a82860 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00cb40 tx=0x7f0b6c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04498c6ce0564a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:09.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.619+0000 7f0b6a7fc640 1 -- 172.21.15.116:0/210972301 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c004220 con 0x5626a8a70b40 2024-09-12T03:14:09.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.619+0000 7f0b6a7fc640 1 -- 172.21.15.116:0/210972301 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b6c0043c0 con 0x5626a8a70b40 2024-09-12T03:14:09.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.619+0000 7f0b6a7fc640 1 -- 172.21.15.116:0/210972301 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c0035f0 con 0x5626a8a70b40 2024-09-12T03:14:09.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.620+0000 7f0b88f1c740 1 -- 172.21.15.116:0/210972301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626a8a70b40 msgr2=0x5626a8a82860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:09.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.620+0000 7f0b88f1c740 1 --2- 172.21.15.116:0/210972301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626a8a70b40 0x5626a8a82860 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00cb40 tx=0x7f0b6c00c330 comp rx=0 tx=0).stop 2024-09-12T03:14:09.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.620+0000 7f0b88f1c740 1 -- 172.21.15.116:0/210972301 shutdown_connections 2024-09-12T03:14:09.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.620+0000 7f0b88f1c740 1 --2- 172.21.15.116:0/210972301 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626a8a7a370 0x5626a8a82da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.620+0000 7f0b88f1c740 1 --2- 172.21.15.116:0/210972301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626a8a70b40 0x5626a8a82860 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.620+0000 7f0b88f1c740 1 --2- 172.21.15.116:0/210972301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a8a6f5d0 0x5626a8a6abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.620+0000 7f0b88f1c740 1 -- 172.21.15.116:0/210972301 >> 172.21.15.116:0/210972301 conn(0x5626a8a38310 msgr2=0x5626a8a6a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:09.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.620+0000 7f0b88f1c740 1 -- 172.21.15.116:0/210972301 shutdown_connections 2024-09-12T03:14:09.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.620+0000 7f0b88f1c740 1 -- 172.21.15.116:0/210972301 wait complete. 2024-09-12T03:14:09.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.620+0000 7f0b88f1c740 1 -- start start 2024-09-12T03:14:09.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.621+0000 7f0b88f1c740 1 Processor -- start 2024-09-12T03:14:09.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.621+0000 7f0b88f1c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626a8a6f5d0 0x5626a8aecca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:09.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.621+0000 7f0b88f1c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a8a70b40 0x5626a8aed1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:09.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.621+0000 7f0b88f1c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626a8a7a370 0x5626a8aed720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:09.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.621+0000 7f0b88f1c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0b801abc60 con 0x5626a8a7a370 2024-09-12T03:14:09.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.621+0000 7f0b88f1c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0b801ad0d0 con 0x5626a8a6f5d0 2024-09-12T03:14:09.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.621+0000 7f0b88f1c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0b801ac330 con 0x5626a8a70b40 2024-09-12T03:14:09.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.621+0000 7f0b88f1c740 10 client.0 _ll_register_callbacks cb 0x7f0b74040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:09.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.621+0000 7f0b850e0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626a8a7a370 0x5626a8aed720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:09.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.621+0000 7f0b850e0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626a8a7a370 0x5626a8aed720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34414/0 (socket says 172.21.15.116:34414) 2024-09-12T03:14:09.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.621+0000 7f0b850e0640 1 -- 172.21.15.116:0/3684496135 learned_addr learned my addr 172.21.15.116:0/3684496135 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:09.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.621+0000 7f0b7ffff640 1 --2- 172.21.15.116:0/3684496135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a8a70b40 0x5626a8aed1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:09.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.621+0000 7f0b848df640 1 --2- 172.21.15.116:0/3684496135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626a8a6f5d0 0x5626a8aecca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:09.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.621+0000 7f0b850e0640 1 -- 172.21.15.116:0/3684496135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a8a70b40 msgr2=0x5626a8aed1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:09.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.621+0000 7f0b850e0640 1 --2- 172.21.15.116:0/3684496135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a8a70b40 0x5626a8aed1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.621+0000 7f0b850e0640 1 -- 172.21.15.116:0/3684496135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626a8a6f5d0 msgr2=0x5626a8aecca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:09.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.621+0000 7f0b850e0640 1 --2- 172.21.15.116:0/3684496135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626a8a6f5d0 0x5626a8aecca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.621+0000 7f0b850e0640 1 -- 172.21.15.116:0/3684496135 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626a8af7fc0 con 0x5626a8a7a370 2024-09-12T03:14:09.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.622+0000 7f0b850e0640 1 --2- 172.21.15.116:0/3684496135 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626a8a7a370 0x5626a8aed720 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f0b780ccfe0 tx=0x7f0b7809b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f572d4dab6ef52 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:09.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.622+0000 7f0b5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:14:09.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.622+0000 7f0b5ffff640 1 -- 172.21.15.116:0/3684496135 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7801f040 con 0x5626a8a7a370 2024-09-12T03:14:09.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.622+0000 7f0b5ffff640 1 -- 172.21.15.116:0/3684496135 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f0b7807a380 con 0x5626a8a7a370 2024-09-12T03:14:09.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.622+0000 7f0b88f1c740 1 -- 172.21.15.116:0/3684496135 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b6c0043c0 con 0x5626a8a7a370 2024-09-12T03:14:09.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.622+0000 7f0b5ffff640 1 -- 172.21.15.116:0/3684496135 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0b7807a580 con 0x5626a8a7a370 2024-09-12T03:14:09.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.622+0000 7f0b88f1c740 10 client.18456 fetch_fsmap learned FSMap version 25 2024-09-12T03:14:09.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.622+0000 7f0b88f1c740 1 -- 172.21.15.116:0/3684496135 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5626a8af9230 con 0x5626a8a7a370 2024-09-12T03:14:09.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.622+0000 7f0b5ffff640 1 -- 172.21.15.116:0/3684496135 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b6c0043c0 con 0x5626a8a7a370 2024-09-12T03:14:09.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.622+0000 7f0b5ffff640 1 -- 172.21.15.116:0/3684496135 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0b7800f040 con 0x5626a8a7a370 2024-09-12T03:14:09.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.622+0000 7f0b88f1c740 10 client.18456 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:14:09.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.622+0000 7f0b88f1c740 10 client.18456 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:14:09.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.622+0000 7f0b88f1c740 20 client.18456 populate_metadata read hostname 'smithi116' 2024-09-12T03:14:09.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.622+0000 7f0b88f1c740 1 --2- 172.21.15.116:0/3684496135 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x5626a8af9580 0x5626a8afb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:09.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.622+0000 7f0b88f1c740 4 client.18456 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:14:09.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.622+0000 7f0b88f1c740 1 -- 172.21.15.116:0/3684496135 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5626a89dcd70 con 0x5626a8af9580 2024-09-12T03:14:09.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.623+0000 7f0b848df640 1 --2- 172.21.15.116:0/3684496135 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x5626a8af9580 0x5626a8afb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:09.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.623+0000 7f0b848df640 1 --2- 172.21.15.116:0/3684496135 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x5626a8af9580 0x5626a8afb970 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5730b0179e304d53 server_cookie=f9fc8b30fe3e21b9 in_seq=0 out_seq=0 2024-09-12T03:14:09.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.623+0000 7f0b5ffff640 10 client.18456 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:14:09.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.623+0000 7f0b5ffff640 1 -- 172.21.15.116:0/3684496135 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5626a89dcd70 con 0x5626a8af9580 2024-09-12T03:14:09.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.623+0000 7f0b5ffff640 10 client.18456 handle_command_reply: tid=0 2024-09-12T03:14:09.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.623+0000 7f0b88f1c740 1 client.18456 shutdown 2024-09-12T03:14:09.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.623+0000 7f0b88f1c740 1 -- 172.21.15.116:0/3684496135 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626a8a7a370 msgr2=0x5626a8aed720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:09.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.623+0000 7f0b88f1c740 1 --2- 172.21.15.116:0/3684496135 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626a8a7a370 0x5626a8aed720 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f0b780ccfe0 tx=0x7f0b7809b7a0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.623+0000 7f0b88f1c740 1 -- 172.21.15.116:0/3684496135 shutdown_connections 2024-09-12T03:14:09.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.623+0000 7f0b88f1c740 1 --2- 172.21.15.116:0/3684496135 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x5626a8af9580 0x5626a8afb970 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.623+0000 7f0b88f1c740 1 --2- 172.21.15.116:0/3684496135 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5626a8a7a370 0x5626a8aed720 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.623+0000 7f0b88f1c740 1 --2- 172.21.15.116:0/3684496135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5626a8a70b40 0x5626a8aed1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.623+0000 7f0b5ffff640 0 client.18456 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:14:09.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.623+0000 7f0b88f1c740 1 --2- 172.21.15.116:0/3684496135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626a8a6f5d0 0x5626a8aecca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.623+0000 7f0b88f1c740 1 -- 172.21.15.116:0/3684496135 >> 172.21.15.116:0/3684496135 conn(0x5626a8a38310 msgr2=0x5626a8a6ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:09.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.624+0000 7f0b88f1c740 1 -- 172.21.15.116:0/3684496135 shutdown_connections 2024-09-12T03:14:09.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.624+0000 7f0b88f1c740 1 -- 172.21.15.116:0/3684496135 wait complete. 2024-09-12T03:14:09.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.624+0000 7f0b88f1c740 20 client.18456 trim_cache size 0 max 16384 2024-09-12T03:14:09.640 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:14:09.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.624+0000 7f0b86b43640 1 -- 172.21.15.116:0/84041597 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0b700a2c00 msgr2=0x7f0b700a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:09.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.624+0000 7f0b86b43640 1 --2- 172.21.15.116:0/84041597 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0b700a2c00 0x7f0b700a50c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0b74009900 tx=0x7f0b740058d0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.624+0000 7f0b86b43640 1 -- 172.21.15.116:0/84041597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b80108160 msgr2=0x7f0b8019cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:09.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.624+0000 7f0b86b43640 1 --2- 172.21.15.116:0/84041597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b80108160 0x7f0b8019cda0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800eaa0 tx=0x7f0b7800ef60 comp rx=0 tx=0).stop 2024-09-12T03:14:09.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.624+0000 7f0b86b43640 1 -- 172.21.15.116:0/84041597 shutdown_connections 2024-09-12T03:14:09.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.624+0000 7f0b86b43640 1 --2- 172.21.15.116:0/84041597 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0b700a2c00 0x7f0b700a50c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.624+0000 7f0b86b43640 1 --2- 172.21.15.116:0/84041597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b80108160 0x7f0b8019cda0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.624+0000 7f0b86b43640 1 --2- 172.21.15.116:0/84041597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b80101290 0x7f0b80198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.624+0000 7f0b86b43640 1 --2- 172.21.15.116:0/84041597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b801008e0 0x7f0b801984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:09.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.624+0000 7f0b86b43640 1 -- 172.21.15.116:0/84041597 >> 172.21.15.116:0/84041597 conn(0x7f0b80077210 msgr2=0x7f0b800ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:09.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.624+0000 7f0b86b43640 1 -- 172.21.15.116:0/84041597 shutdown_connections 2024-09-12T03:14:09.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:09.624+0000 7f0b86b43640 1 -- 172.21.15.116:0/84041597 wait complete. 2024-09-12T03:14:09.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:09 smithi183 ceph-mon[35535]: pgmap v673: 97 pgs: 97 active+clean; 5.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-12T03:14:09.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:09 smithi177 ceph-mon[35305]: pgmap v673: 97 pgs: 97 active+clean; 5.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-12T03:14:10.331 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:14:10.579 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:14:11.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.055+0000 7f616045a640 1 -- 172.21.15.116:0/2473001385 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6158101df0 msgr2=0x7f61581021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:11.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.055+0000 7f616045a640 1 --2- 172.21.15.116:0/2473001385 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6158101df0 0x7f61581021d0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f6154009900 tx=0x7f615402a410 comp rx=0 tx=0).stop 2024-09-12T03:14:11.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.055+0000 7f616045a640 1 -- 172.21.15.116:0/2473001385 shutdown_connections 2024-09-12T03:14:11.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.055+0000 7f616045a640 1 --2- 172.21.15.116:0/2473001385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6158106f20 0x7f615810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.055+0000 7f616045a640 1 --2- 172.21.15.116:0/2473001385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61581027a0 0x7f61581067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.055+0000 7f616045a640 1 --2- 172.21.15.116:0/2473001385 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6158101df0 0x7f61581021d0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.055+0000 7f616045a640 1 -- 172.21.15.116:0/2473001385 >> 172.21.15.116:0/2473001385 conn(0x7f61580fd8b0 msgr2=0x7f61580ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:11.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.055+0000 7f616045a640 1 -- 172.21.15.116:0/2473001385 shutdown_connections 2024-09-12T03:14:11.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.055+0000 7f616045a640 1 -- 172.21.15.116:0/2473001385 wait complete. 2024-09-12T03:14:11.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.055+0000 7f616045a640 1 Processor -- start 2024-09-12T03:14:11.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.056+0000 7f616045a640 1 -- start start 2024-09-12T03:14:11.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.056+0000 7f616045a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6158101df0 0x7f6158198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:11.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.056+0000 7f616045a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61581027a0 0x7f61581989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:11.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.056+0000 7f616045a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6158106f20 0x7f615819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:11.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.056+0000 7f616045a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6158111f80 con 0x7f6158101df0 2024-09-12T03:14:11.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.056+0000 7f616045a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6158111e00 con 0x7f61581027a0 2024-09-12T03:14:11.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.056+0000 7f616045a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6158112100 con 0x7f6158106f20 2024-09-12T03:14:11.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.056+0000 7f615e1f6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6158101df0 0x7f6158198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:11.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.056+0000 7f615e1f6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6158101df0 0x7f6158198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45998/0 (socket says 172.21.15.116:45998) 2024-09-12T03:14:11.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.056+0000 7f615e1f6640 1 -- 172.21.15.116:0/3748723034 learned_addr learned my addr 172.21.15.116:0/3748723034 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:11.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.056+0000 7f615e9f7640 1 --2- 172.21.15.116:0/3748723034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6158106f20 0x7f615819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:11.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.056+0000 7f615d9f5640 1 --2- 172.21.15.116:0/3748723034 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61581027a0 0x7f61581989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:11.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.056+0000 7f615e1f6640 1 -- 172.21.15.116:0/3748723034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6158106f20 msgr2=0x7f615819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:11.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.056+0000 7f615e1f6640 1 --2- 172.21.15.116:0/3748723034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6158106f20 0x7f615819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.056+0000 7f615e1f6640 1 -- 172.21.15.116:0/3748723034 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61581027a0 msgr2=0x7f61581989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:11.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.056+0000 7f615e1f6640 1 --2- 172.21.15.116:0/3748723034 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61581027a0 0x7f61581989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.056+0000 7f615e1f6640 1 -- 172.21.15.116:0/3748723034 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f615819d410 con 0x7f6158101df0 2024-09-12T03:14:11.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.056+0000 7f615e9f7640 1 --2- 172.21.15.116:0/3748723034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6158106f20 0x7f615819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:11.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.056+0000 7f615e1f6640 1 --2- 172.21.15.116:0/3748723034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6158101df0 0x7f6158198460 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f6154006010 tx=0x7f61540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:11.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.056+0000 7f614f7fe640 1 -- 172.21.15.116:0/3748723034 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615403f030 con 0x7f6158101df0 2024-09-12T03:14:11.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.056+0000 7f614f7fe640 1 -- 172.21.15.116:0/3748723034 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6154003ce0 con 0x7f6158101df0 2024-09-12T03:14:11.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.056+0000 7f616045a640 1 -- 172.21.15.116:0/3748723034 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f615819d6a0 con 0x7f6158101df0 2024-09-12T03:14:11.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.056+0000 7f614f7fe640 1 -- 172.21.15.116:0/3748723034 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6154035070 con 0x7f6158101df0 2024-09-12T03:14:11.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.057+0000 7f616045a640 1 -- 172.21.15.116:0/3748723034 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61581a4fe0 con 0x7f6158101df0 2024-09-12T03:14:11.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.059+0000 7f614f7fe640 1 -- 172.21.15.116:0/3748723034 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6154042030 con 0x7f6158101df0 2024-09-12T03:14:11.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.059+0000 7f6162833740 1 Processor -- start 2024-09-12T03:14:11.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.060+0000 7f6162833740 1 -- start start 2024-09-12T03:14:11.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.060+0000 7f6162833740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5628a2d59f50 0x5628a2d584a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:11.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.060+0000 7f6162833740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628a2d589e0 0x5628a2d58e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:11.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.060+0000 7f6162833740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628a2d63780 0x5628a2d65b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:11.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.060+0000 7f6162833740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5628a2cba2f0 con 0x5628a2d59f50 2024-09-12T03:14:11.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.060+0000 7f6162833740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5628a2c61cf0 con 0x5628a2d63780 2024-09-12T03:14:11.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.060+0000 7f6162833740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5628a2cb62b0 con 0x5628a2d589e0 2024-09-12T03:14:11.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.060+0000 7f615e9f7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5628a2d59f50 0x5628a2d584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:11.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.060+0000 7f615e9f7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5628a2d59f50 0x5628a2d584a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46014/0 (socket says 172.21.15.116:46014) 2024-09-12T03:14:11.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.060+0000 7f615e9f7640 1 -- 172.21.15.116:0/2446379151 learned_addr learned my addr 172.21.15.116:0/2446379151 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:11.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.060+0000 7f615e1f6640 1 --2- 172.21.15.116:0/2446379151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628a2d589e0 0x5628a2d58e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:11.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.060+0000 7f615d9f5640 1 --2- 172.21.15.116:0/2446379151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628a2d63780 0x5628a2d65b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:11.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.060+0000 7f614f7fe640 1 --2- 172.21.15.116:0/3748723034 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f61440a2c00 0x7f61440a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:11.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.060+0000 7f615e1f6640 1 -- 172.21.15.116:0/2446379151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628a2d63780 msgr2=0x5628a2d65b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:11.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.060+0000 7f615e1f6640 1 --2- 172.21.15.116:0/2446379151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628a2d63780 0x5628a2d65b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.060+0000 7f615e1f6640 1 -- 172.21.15.116:0/2446379151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5628a2d59f50 msgr2=0x5628a2d584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:11.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.060+0000 7f615e1f6640 1 --2- 172.21.15.116:0/2446379151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5628a2d59f50 0x5628a2d584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.060+0000 7f615e1f6640 1 -- 172.21.15.116:0/2446379151 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628a2c463a0 con 0x5628a2d589e0 2024-09-12T03:14:11.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.060+0000 7f615e9f7640 1 --2- 172.21.15.116:0/2446379151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5628a2d59f50 0x5628a2d584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:14:11.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.061+0000 7f615e9f7640 1 --2- 172.21.15.116:0/3748723034 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f61440a2c00 0x7f61440a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:11.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.061+0000 7f614f7fe640 1 -- 172.21.15.116:0/3748723034 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f61540ef600 con 0x7f6158101df0 2024-09-12T03:14:11.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.061+0000 7f615e1f6640 1 --2- 172.21.15.116:0/2446379151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628a2d589e0 0x5628a2d58e20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f61540ed100 tx=0x7f61540b1430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c3052d5405c4551 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:11.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.061+0000 7f615e9f7640 1 --2- 172.21.15.116:0/3748723034 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f61440a2c00 0x7f61440a50c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f61500126f0 tx=0x7f6150004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:11.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.061+0000 7f614cdea640 1 -- 172.21.15.116:0/2446379151 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61540bba90 con 0x5628a2d589e0 2024-09-12T03:14:11.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.061+0000 7f614cdea640 1 -- 172.21.15.116:0/2446379151 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61540870f0 con 0x5628a2d589e0 2024-09-12T03:14:11.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.061+0000 7f614cdea640 1 -- 172.21.15.116:0/2446379151 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61540044e0 con 0x5628a2d589e0 2024-09-12T03:14:11.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.062+0000 7f6162833740 1 -- 172.21.15.116:0/2446379151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628a2d589e0 msgr2=0x5628a2d58e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:11.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.062+0000 7f6162833740 1 --2- 172.21.15.116:0/2446379151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628a2d589e0 0x5628a2d58e20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f61540ed100 tx=0x7f61540b1430 comp rx=0 tx=0).stop 2024-09-12T03:14:11.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.062+0000 7f6162833740 1 -- 172.21.15.116:0/2446379151 shutdown_connections 2024-09-12T03:14:11.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.062+0000 7f6162833740 1 --2- 172.21.15.116:0/2446379151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628a2d63780 0x5628a2d65b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.062+0000 7f6162833740 1 --2- 172.21.15.116:0/2446379151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628a2d589e0 0x5628a2d58e20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.062+0000 7f6162833740 1 --2- 172.21.15.116:0/2446379151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5628a2d59f50 0x5628a2d584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.062+0000 7f6162833740 1 -- 172.21.15.116:0/2446379151 >> 172.21.15.116:0/2446379151 conn(0x5628a2d1d310 msgr2=0x5628a2d55070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:11.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.062+0000 7f6162833740 1 -- 172.21.15.116:0/2446379151 shutdown_connections 2024-09-12T03:14:11.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.062+0000 7f6162833740 1 -- 172.21.15.116:0/2446379151 wait complete. 2024-09-12T03:14:11.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.062+0000 7f6162833740 1 -- start start 2024-09-12T03:14:11.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.062+0000 7f6162833740 1 Processor -- start 2024-09-12T03:14:11.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.062+0000 7f6162833740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628a2d589e0 0x5628a2d630f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:11.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.062+0000 7f6162833740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628a2d59f50 0x5628a2d603f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:11.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.062+0000 7f6162833740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5628a2d63780 0x5628a2d60a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:11.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.062+0000 7f6162833740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5628a2d66240 con 0x5628a2d63780 2024-09-12T03:14:11.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.062+0000 7f6162833740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5628a2c75060 con 0x5628a2d59f50 2024-09-12T03:14:11.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.062+0000 7f6162833740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5628a2d663c0 con 0x5628a2d589e0 2024-09-12T03:14:11.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.063+0000 7f6162833740 10 client.0 _ll_register_callbacks cb 0x5628a29c88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:11.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.063+0000 7f615e9f7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5628a2d63780 0x5628a2d60a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:11.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.063+0000 7f615e9f7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5628a2d63780 0x5628a2d60a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46016/0 (socket says 172.21.15.116:46016) 2024-09-12T03:14:11.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.063+0000 7f615e9f7640 1 -- 172.21.15.116:0/2868503175 learned_addr learned my addr 172.21.15.116:0/2868503175 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:11.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.063+0000 7f615e1f6640 1 --2- 172.21.15.116:0/2868503175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628a2d589e0 0x5628a2d630f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:11.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.063+0000 7f615d9f5640 1 --2- 172.21.15.116:0/2868503175 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628a2d59f50 0x5628a2d603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:11.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.063+0000 7f615e9f7640 1 -- 172.21.15.116:0/2868503175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628a2d589e0 msgr2=0x5628a2d630f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:11.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.063+0000 7f615e9f7640 1 --2- 172.21.15.116:0/2868503175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628a2d589e0 0x5628a2d630f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.063+0000 7f615e9f7640 1 -- 172.21.15.116:0/2868503175 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628a2d59f50 msgr2=0x5628a2d603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:11.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.063+0000 7f615e9f7640 1 --2- 172.21.15.116:0/2868503175 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628a2d59f50 0x5628a2d603f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.063+0000 7f615e9f7640 1 -- 172.21.15.116:0/2868503175 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628a2a2b3c0 con 0x5628a2d63780 2024-09-12T03:14:11.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.063+0000 7f615e9f7640 1 --2- 172.21.15.116:0/2868503175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5628a2d63780 0x5628a2d60a00 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f6150012490 tx=0x7f6150004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67842e470fe567f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:11.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.063+0000 7f6141ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:14:11.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.063+0000 7f6141ffb640 1 -- 172.21.15.116:0/2868503175 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615000fe10 con 0x5628a2d63780 2024-09-12T03:14:11.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.063+0000 7f6141ffb640 1 -- 172.21.15.116:0/2868503175 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f615001b690 con 0x5628a2d63780 2024-09-12T03:14:11.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.063+0000 7f6162833740 1 -- 172.21.15.116:0/2868503175 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61540870f0 con 0x5628a2d63780 2024-09-12T03:14:11.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.063+0000 7f6141ffb640 1 -- 172.21.15.116:0/2868503175 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f615001b890 con 0x5628a2d63780 2024-09-12T03:14:11.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.063+0000 7f6162833740 10 client.18468 fetch_fsmap learned FSMap version 25 2024-09-12T03:14:11.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.063+0000 7f6162833740 1 -- 172.21.15.116:0/2868503175 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5628a2d71be0 con 0x5628a2d63780 2024-09-12T03:14:11.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.064+0000 7f6141ffb640 1 -- 172.21.15.116:0/2868503175 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61540870f0 con 0x5628a2d63780 2024-09-12T03:14:11.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.064+0000 7f6141ffb640 1 -- 172.21.15.116:0/2868503175 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6150024070 con 0x5628a2d63780 2024-09-12T03:14:11.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.064+0000 7f6162833740 10 client.18468 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:14:11.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.064+0000 7f6162833740 10 client.18468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:14:11.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.064+0000 7f6162833740 20 client.18468 populate_metadata read hostname 'smithi116' 2024-09-12T03:14:11.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.064+0000 7f6162833740 1 --2- 172.21.15.116:0/2868503175 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5628a2d730f0 0x5628a2d75530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:11.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.064+0000 7f6162833740 4 client.18468 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:14:11.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.064+0000 7f6162833740 1 -- 172.21.15.116:0/2868503175 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628a2cc7860 con 0x5628a2d730f0 2024-09-12T03:14:11.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.064+0000 7f615e1f6640 1 --2- 172.21.15.116:0/2868503175 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5628a2d730f0 0x5628a2d75530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:11.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.064+0000 7f615e1f6640 1 --2- 172.21.15.116:0/2868503175 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5628a2d730f0 0x5628a2d75530 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b204748ebcf1a221 server_cookie=d19215125505f64e in_seq=0 out_seq=0 2024-09-12T03:14:11.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.064+0000 7f6141ffb640 10 client.18468 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:11.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.065+0000 7f6141ffb640 1 -- 172.21.15.116:0/2868503175 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5628a2cc7860 con 0x5628a2d730f0 2024-09-12T03:14:11.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.065+0000 7f6141ffb640 10 client.18468 handle_command_reply: tid=0 2024-09-12T03:14:11.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.065+0000 7f6162833740 1 client.18468 shutdown 2024-09-12T03:14:11.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.065+0000 7f6162833740 1 -- 172.21.15.116:0/2868503175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5628a2d63780 msgr2=0x5628a2d60a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:11.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.065+0000 7f6162833740 1 --2- 172.21.15.116:0/2868503175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5628a2d63780 0x5628a2d60a00 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f6150012490 tx=0x7f6150004ab0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.065+0000 7f6162833740 1 -- 172.21.15.116:0/2868503175 shutdown_connections 2024-09-12T03:14:11.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.065+0000 7f6162833740 1 --2- 172.21.15.116:0/2868503175 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5628a2d730f0 0x5628a2d75530 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.065+0000 7f6162833740 1 --2- 172.21.15.116:0/2868503175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5628a2d63780 0x5628a2d60a00 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.065+0000 7f6162833740 1 --2- 172.21.15.116:0/2868503175 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628a2d59f50 0x5628a2d603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.065+0000 7f6141ffb640 0 client.18468 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:11.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.065+0000 7f6162833740 1 --2- 172.21.15.116:0/2868503175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628a2d589e0 0x5628a2d630f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.065+0000 7f6162833740 1 -- 172.21.15.116:0/2868503175 >> 172.21.15.116:0/2868503175 conn(0x5628a2d1d310 msgr2=0x5628a2d5ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:11.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.065+0000 7f6162833740 1 -- 172.21.15.116:0/2868503175 shutdown_connections 2024-09-12T03:14:11.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.065+0000 7f6162833740 1 -- 172.21.15.116:0/2868503175 wait complete. 2024-09-12T03:14:11.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.065+0000 7f6162833740 20 client.18468 trim_cache size 0 max 16384 2024-09-12T03:14:11.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.078+0000 7f6162833740 1 Processor -- start 2024-09-12T03:14:11.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.078+0000 7f6162833740 1 -- start start 2024-09-12T03:14:11.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.078+0000 7f6162833740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628a2d589e0 0x5628a2d62360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:11.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.078+0000 7f6162833740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5628a2d59f50 0x5628a2d5b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:11.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.078+0000 7f6162833740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628a2d63780 0x5628a2d5ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:11.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.078+0000 7f6162833740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f61581adb90 con 0x5628a2d59f50 2024-09-12T03:14:11.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.078+0000 7f6162833740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f61581ab460 con 0x5628a2d63780 2024-09-12T03:14:11.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.078+0000 7f6162833740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f61581a92b0 con 0x5628a2d589e0 2024-09-12T03:14:11.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.078+0000 7f615d9f5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5628a2d59f50 0x5628a2d5b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:11.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.078+0000 7f615d9f5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5628a2d59f50 0x5628a2d5b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46032/0 (socket says 172.21.15.116:46032) 2024-09-12T03:14:11.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.078+0000 7f615d9f5640 1 -- 172.21.15.116:0/3642318922 learned_addr learned my addr 172.21.15.116:0/3642318922 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:11.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.079+0000 7f615e1f6640 1 --2- 172.21.15.116:0/3642318922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628a2d589e0 0x5628a2d62360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:11.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.079+0000 7f615e9f7640 1 --2- 172.21.15.116:0/3642318922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628a2d63780 0x5628a2d5ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:11.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.079+0000 7f615d9f5640 1 -- 172.21.15.116:0/3642318922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628a2d589e0 msgr2=0x5628a2d62360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:11.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.079+0000 7f615d9f5640 1 --2- 172.21.15.116:0/3642318922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628a2d589e0 0x5628a2d62360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.079+0000 7f615d9f5640 1 -- 172.21.15.116:0/3642318922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628a2d63780 msgr2=0x5628a2d5ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:11.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.079+0000 7f615d9f5640 1 --2- 172.21.15.116:0/3642318922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628a2d63780 0x5628a2d5ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.079+0000 7f615d9f5640 1 -- 172.21.15.116:0/3642318922 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628a2d75bb0 con 0x5628a2d59f50 2024-09-12T03:14:11.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.079+0000 7f615e9f7640 1 --2- 172.21.15.116:0/3642318922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628a2d63780 0x5628a2d5ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:14:11.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.079+0000 7f615d9f5640 1 --2- 172.21.15.116:0/3642318922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5628a2d59f50 0x5628a2d5b530 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f6148009c80 tx=0x7f6148012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3824bf710896e68d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:11.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.079+0000 7f6143fff640 1 -- 172.21.15.116:0/3642318922 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6148021070 con 0x5628a2d59f50 2024-09-12T03:14:11.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.079+0000 7f6143fff640 1 -- 172.21.15.116:0/3642318922 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6148009e30 con 0x5628a2d59f50 2024-09-12T03:14:11.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.079+0000 7f6143fff640 1 -- 172.21.15.116:0/3642318922 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61480116a0 con 0x5628a2d59f50 2024-09-12T03:14:11.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.080+0000 7f6162833740 1 -- 172.21.15.116:0/3642318922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5628a2d59f50 msgr2=0x5628a2d5b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:11.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.080+0000 7f6162833740 1 --2- 172.21.15.116:0/3642318922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5628a2d59f50 0x5628a2d5b530 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f6148009c80 tx=0x7f6148012820 comp rx=0 tx=0).stop 2024-09-12T03:14:11.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.080+0000 7f6162833740 1 -- 172.21.15.116:0/3642318922 shutdown_connections 2024-09-12T03:14:11.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.080+0000 7f6162833740 1 --2- 172.21.15.116:0/3642318922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628a2d63780 0x5628a2d5ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.080+0000 7f6162833740 1 --2- 172.21.15.116:0/3642318922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5628a2d59f50 0x5628a2d5b530 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.080+0000 7f6162833740 1 --2- 172.21.15.116:0/3642318922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628a2d589e0 0x5628a2d62360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.080+0000 7f6162833740 1 -- 172.21.15.116:0/3642318922 >> 172.21.15.116:0/3642318922 conn(0x5628a2d1d310 msgr2=0x5628a2d61c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:11.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.080+0000 7f6162833740 1 -- 172.21.15.116:0/3642318922 shutdown_connections 2024-09-12T03:14:11.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.080+0000 7f6162833740 1 -- 172.21.15.116:0/3642318922 wait complete. 2024-09-12T03:14:11.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.080+0000 7f6162833740 1 -- start start 2024-09-12T03:14:11.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.080+0000 7f6162833740 1 Processor -- start 2024-09-12T03:14:11.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.080+0000 7f6162833740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628a2d589e0 0x5628a2d67470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:11.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.080+0000 7f6162833740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628a2d59f50 0x5628a2d679b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:11.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.080+0000 7f6162833740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5628a2d63780 0x5628a2d67ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:11.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.080+0000 7f6162833740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f61581a9920 con 0x5628a2d63780 2024-09-12T03:14:11.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.080+0000 7f6162833740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f61581aad90 con 0x5628a2d589e0 2024-09-12T03:14:11.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.080+0000 7f6162833740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f61581a9ff0 con 0x5628a2d59f50 2024-09-12T03:14:11.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.081+0000 7f6162833740 10 client.0 _ll_register_callbacks cb 0x7f6150012140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:11.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.081+0000 7f615e9f7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5628a2d63780 0x5628a2d67ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:11.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.081+0000 7f615e9f7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5628a2d63780 0x5628a2d67ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46036/0 (socket says 172.21.15.116:46036) 2024-09-12T03:14:11.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.081+0000 7f615e9f7640 1 -- 172.21.15.116:0/3328671643 learned_addr learned my addr 172.21.15.116:0/3328671643 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:11.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.081+0000 7f615d9f5640 1 --2- 172.21.15.116:0/3328671643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628a2d59f50 0x5628a2d679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:11.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.081+0000 7f615e1f6640 1 --2- 172.21.15.116:0/3328671643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628a2d589e0 0x5628a2d67470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:11.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.081+0000 7f615e9f7640 1 -- 172.21.15.116:0/3328671643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628a2d59f50 msgr2=0x5628a2d679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:11.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.081+0000 7f615e9f7640 1 --2- 172.21.15.116:0/3328671643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628a2d59f50 0x5628a2d679b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.081+0000 7f615e9f7640 1 -- 172.21.15.116:0/3328671643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628a2d589e0 msgr2=0x5628a2d67470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:11.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.081+0000 7f615e9f7640 1 --2- 172.21.15.116:0/3328671643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628a2d589e0 0x5628a2d67470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.081+0000 7f615e9f7640 1 -- 172.21.15.116:0/3328671643 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628a2ddd070 con 0x5628a2d63780 2024-09-12T03:14:11.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.081+0000 7f615e9f7640 1 --2- 172.21.15.116:0/3328671643 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5628a2d63780 0x5628a2d67ef0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f6150026db0 tx=0x7f615000fe10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d299f933ab9712 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:11.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.081+0000 7f6141ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:14:11.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.081+0000 7f6141ffb640 1 -- 172.21.15.116:0/3328671643 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615001bc40 con 0x5628a2d63780 2024-09-12T03:14:11.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.081+0000 7f6141ffb640 1 -- 172.21.15.116:0/3328671643 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f615001bde0 con 0x5628a2d63780 2024-09-12T03:14:11.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.081+0000 7f6162833740 1 -- 172.21.15.116:0/3328671643 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6148009e30 con 0x5628a2d63780 2024-09-12T03:14:11.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.081+0000 7f6141ffb640 1 -- 172.21.15.116:0/3328671643 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6150026440 con 0x5628a2d63780 2024-09-12T03:14:11.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.081+0000 7f6162833740 10 client.18474 fetch_fsmap learned FSMap version 25 2024-09-12T03:14:11.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.081+0000 7f6162833740 1 -- 172.21.15.116:0/3328671643 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5628a2dde2c0 con 0x5628a2d63780 2024-09-12T03:14:11.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.082+0000 7f6141ffb640 1 -- 172.21.15.116:0/3328671643 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6148009e30 con 0x5628a2d63780 2024-09-12T03:14:11.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.082+0000 7f6141ffb640 1 -- 172.21.15.116:0/3328671643 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6150024040 con 0x5628a2d63780 2024-09-12T03:14:11.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.082+0000 7f6162833740 10 client.18474 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:14:11.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.082+0000 7f6162833740 10 client.18474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:14:11.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.082+0000 7f6162833740 20 client.18474 populate_metadata read hostname 'smithi116' 2024-09-12T03:14:11.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.082+0000 7f6162833740 1 --2- 172.21.15.116:0/3328671643 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5628a2ddea60 0x5628a2de0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:11.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.082+0000 7f6162833740 4 client.18474 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:14:11.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.082+0000 7f6162833740 1 -- 172.21.15.116:0/3328671643 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5628a2cc1d70 con 0x5628a2ddea60 2024-09-12T03:14:11.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.082+0000 7f615e1f6640 1 --2- 172.21.15.116:0/3328671643 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5628a2ddea60 0x5628a2de0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:11.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.082+0000 7f6141ffb640 10 client.18474 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:11.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.082+0000 7f615e1f6640 1 --2- 172.21.15.116:0/3328671643 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5628a2ddea60 0x5628a2de0e80 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb07b3f5615468af server_cookie=4d2674957ac78973 in_seq=0 out_seq=0 2024-09-12T03:14:11.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.083+0000 7f6141ffb640 1 -- 172.21.15.116:0/3328671643 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5628a2cc1d70 con 0x5628a2ddea60 2024-09-12T03:14:11.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.083+0000 7f6141ffb640 10 client.18474 handle_command_reply: tid=0 2024-09-12T03:14:11.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.083+0000 7f6162833740 1 client.18474 shutdown 2024-09-12T03:14:11.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.083+0000 7f6162833740 1 -- 172.21.15.116:0/3328671643 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5628a2d63780 msgr2=0x5628a2d67ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:11.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.083+0000 7f6162833740 1 --2- 172.21.15.116:0/3328671643 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5628a2d63780 0x5628a2d67ef0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f6150026db0 tx=0x7f615000fe10 comp rx=0 tx=0).stop 2024-09-12T03:14:11.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.083+0000 7f6162833740 1 -- 172.21.15.116:0/3328671643 shutdown_connections 2024-09-12T03:14:11.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.083+0000 7f6162833740 1 --2- 172.21.15.116:0/3328671643 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5628a2ddea60 0x5628a2de0e80 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.083+0000 7f6162833740 1 --2- 172.21.15.116:0/3328671643 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5628a2d63780 0x5628a2d67ef0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.083+0000 7f6162833740 1 --2- 172.21.15.116:0/3328671643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628a2d59f50 0x5628a2d679b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.083+0000 7f6162833740 1 --2- 172.21.15.116:0/3328671643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628a2d589e0 0x5628a2d67470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.083+0000 7f6162833740 1 -- 172.21.15.116:0/3328671643 >> 172.21.15.116:0/3328671643 conn(0x5628a2d1d310 msgr2=0x5628a2d62470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:11.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.083+0000 7f6141ffb640 0 client.18474 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:11.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.083+0000 7f6162833740 1 -- 172.21.15.116:0/3328671643 shutdown_connections 2024-09-12T03:14:11.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.083+0000 7f6162833740 1 -- 172.21.15.116:0/3328671643 wait complete. 2024-09-12T03:14:11.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.083+0000 7f6162833740 20 client.18474 trim_cache size 0 max 16384 2024-09-12T03:14:11.094 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:14:11.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.083+0000 7f616045a640 1 -- 172.21.15.116:0/3748723034 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f61440a2c00 msgr2=0x7f61440a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:11.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.083+0000 7f616045a640 1 --2- 172.21.15.116:0/3748723034 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f61440a2c00 0x7f61440a50c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f61500126f0 tx=0x7f6150004a40 comp rx=0 tx=0).stop 2024-09-12T03:14:11.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.083+0000 7f616045a640 1 -- 172.21.15.116:0/3748723034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6158101df0 msgr2=0x7f6158198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:11.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.083+0000 7f616045a640 1 --2- 172.21.15.116:0/3748723034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6158101df0 0x7f6158198460 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f6154006010 tx=0x7f61540042f0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.083+0000 7f616045a640 1 -- 172.21.15.116:0/3748723034 shutdown_connections 2024-09-12T03:14:11.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.083+0000 7f616045a640 1 --2- 172.21.15.116:0/3748723034 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f61440a2c00 0x7f61440a50c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.083+0000 7f616045a640 1 --2- 172.21.15.116:0/3748723034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6158106f20 0x7f615819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.083+0000 7f616045a640 1 --2- 172.21.15.116:0/3748723034 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61581027a0 0x7f61581989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.083+0000 7f616045a640 1 --2- 172.21.15.116:0/3748723034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6158101df0 0x7f6158198460 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:11.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.083+0000 7f616045a640 1 -- 172.21.15.116:0/3748723034 >> 172.21.15.116:0/3748723034 conn(0x7f61580fd8b0 msgr2=0x7f61580fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:11.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.083+0000 7f616045a640 1 -- 172.21.15.116:0/3748723034 shutdown_connections 2024-09-12T03:14:11.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:11.084+0000 7f616045a640 1 -- 172.21.15.116:0/3748723034 wait complete. 2024-09-12T03:14:11.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:11 smithi183 ceph-mon[35535]: pgmap v674: 97 pgs: 97 active+clean; 5.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:14:11.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:11 smithi177 ceph-mon[35305]: pgmap v674: 97 pgs: 97 active+clean; 5.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:14:11.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:11 smithi116 ceph-mon[31410]: pgmap v674: 97 pgs: 97 active+clean; 5.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:14:12.881 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:14:12.881 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:14:13.139 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:14:13.531 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:13 smithi116 ceph-mon[31410]: pgmap v675: 97 pgs: 97 active+clean; 5.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:14:13.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.615+0000 7ff0849d4640 1 -- 172.21.15.116:0/2652955948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff080101420 msgr2=0x7ff08010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:13.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.615+0000 7ff0849d4640 1 --2- 172.21.15.116:0/2652955948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff080101420 0x7ff08010d950 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7ff078009cd0 tx=0x7ff07802a8e0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.615+0000 7ff0849d4640 1 -- 172.21.15.116:0/2652955948 shutdown_connections 2024-09-12T03:14:13.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.615+0000 7ff0849d4640 1 --2- 172.21.15.116:0/2652955948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff080101420 0x7ff08010d950 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.615+0000 7ff0849d4640 1 --2- 172.21.15.116:0/2652955948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff080100a80 0x7ff080100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.615+0000 7ff0849d4640 1 --2- 172.21.15.116:0/2652955948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff080106a80 0x7ff080106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.615+0000 7ff0849d4640 1 -- 172.21.15.116:0/2652955948 >> 172.21.15.116:0/2652955948 conn(0x7ff0800fc7a0 msgr2=0x7ff0800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:13.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.615+0000 7ff0849d4640 1 -- 172.21.15.116:0/2652955948 shutdown_connections 2024-09-12T03:14:13.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.616+0000 7ff0849d4640 1 -- 172.21.15.116:0/2652955948 wait complete. 2024-09-12T03:14:13.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.616+0000 7ff0849d4640 1 Processor -- start 2024-09-12T03:14:13.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.616+0000 7ff0849d4640 1 -- start start 2024-09-12T03:14:13.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.616+0000 7ff0849d4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff080100a80 0x7ff080198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:13.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.616+0000 7ff0849d4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff080101420 0x7ff080198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:13.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.616+0000 7ff0849d4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff080106a80 0x7ff08019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:13.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.616+0000 7ff0849d4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff080110e80 con 0x7ff080101420 2024-09-12T03:14:13.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.616+0000 7ff0849d4640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff080110d00 con 0x7ff080100a80 2024-09-12T03:14:13.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.616+0000 7ff0849d4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff080111000 con 0x7ff080106a80 2024-09-12T03:14:13.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.617+0000 7ff077fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff080101420 0x7ff080198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:13.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.617+0000 7ff077fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff080101420 0x7ff080198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46054/0 (socket says 172.21.15.116:46054) 2024-09-12T03:14:13.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.617+0000 7ff077fff640 1 -- 172.21.15.116:0/744230155 learned_addr learned my addr 172.21.15.116:0/744230155 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:13.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.617+0000 7ff07e59c640 1 --2- 172.21.15.116:0/744230155 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff080100a80 0x7ff080198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:13.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.617+0000 7ff07ed9d640 1 --2- 172.21.15.116:0/744230155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff080106a80 0x7ff08019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:13.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.617+0000 7ff077fff640 1 -- 172.21.15.116:0/744230155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff080106a80 msgr2=0x7ff08019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:13.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.617+0000 7ff077fff640 1 --2- 172.21.15.116:0/744230155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff080106a80 0x7ff08019cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.617+0000 7ff077fff640 1 -- 172.21.15.116:0/744230155 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff080100a80 msgr2=0x7ff080198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:13.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.617+0000 7ff077fff640 1 --2- 172.21.15.116:0/744230155 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff080100a80 0x7ff080198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.617+0000 7ff077fff640 1 -- 172.21.15.116:0/744230155 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08019d300 con 0x7ff080101420 2024-09-12T03:14:13.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.617+0000 7ff077fff640 1 --2- 172.21.15.116:0/744230155 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff080101420 0x7ff080198890 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7ff06800c920 tx=0x7ff06800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:13.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.617+0000 7ff0777fe640 1 -- 172.21.15.116:0/744230155 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff068007bb0 con 0x7ff080101420 2024-09-12T03:14:13.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.617+0000 7ff0849d4640 1 -- 172.21.15.116:0/744230155 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff08019d5f0 con 0x7ff080101420 2024-09-12T03:14:13.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.617+0000 7ff0777fe640 1 -- 172.21.15.116:0/744230155 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff068007d50 con 0x7ff080101420 2024-09-12T03:14:13.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.617+0000 7ff0777fe640 1 -- 172.21.15.116:0/744230155 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff068018780 con 0x7ff080101420 2024-09-12T03:14:13.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.617+0000 7ff0849d4640 1 -- 172.21.15.116:0/744230155 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0801a4ed0 con 0x7ff080101420 2024-09-12T03:14:13.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.620+0000 7ff086dad740 1 Processor -- start 2024-09-12T03:14:13.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.620+0000 7ff086dad740 1 -- start start 2024-09-12T03:14:13.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.621+0000 7ff0777fe640 1 -- 172.21.15.116:0/744230155 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ff06800f450 con 0x7ff080101420 2024-09-12T03:14:13.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.621+0000 7ff086dad740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b59a34db60 0x55b59a34df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:13.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.621+0000 7ff086dad740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b59a34e480 0x55b59a358800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:13.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.621+0000 7ff086dad740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b59a358d40 0x55b59a35b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:13.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.621+0000 7ff086dad740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b59a2b3280 con 0x55b59a34db60 2024-09-12T03:14:13.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.621+0000 7ff086dad740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b59a25ab10 con 0x55b59a358d40 2024-09-12T03:14:13.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.621+0000 7ff086dad740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b59a2af240 con 0x55b59a34e480 2024-09-12T03:14:13.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.621+0000 7ff07ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b59a34db60 0x55b59a34df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:13.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.621+0000 7ff07ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b59a34db60 0x55b59a34df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46062/0 (socket says 172.21.15.116:46062) 2024-09-12T03:14:13.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.621+0000 7ff07ed9d640 1 -- 172.21.15.116:0/889765848 learned_addr learned my addr 172.21.15.116:0/889765848 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:13.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.621+0000 7ff07e59c640 1 --2- 172.21.15.116:0/889765848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b59a34e480 0x55b59a358800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:13.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.621+0000 7ff077fff640 1 --2- 172.21.15.116:0/889765848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b59a358d40 0x55b59a35b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:13.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.621+0000 7ff07ed9d640 1 -- 172.21.15.116:0/889765848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b59a34e480 msgr2=0x55b59a358800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:13.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.621+0000 7ff07ed9d640 1 --2- 172.21.15.116:0/889765848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b59a34e480 0x55b59a358800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.621+0000 7ff07ed9d640 1 -- 172.21.15.116:0/889765848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b59a358d40 msgr2=0x55b59a35b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:13.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.621+0000 7ff07ed9d640 1 --2- 172.21.15.116:0/889765848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b59a358d40 0x55b59a35b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.621+0000 7ff07ed9d640 1 -- 172.21.15.116:0/889765848 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b59a2671f0 con 0x55b59a34db60 2024-09-12T03:14:13.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.621+0000 7ff07e59c640 1 --2- 172.21.15.116:0/889765848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b59a34e480 0x55b59a358800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:14:13.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.621+0000 7ff0777fe640 1 --2- 172.21.15.116:0/744230155 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff06c0a2b80 0x7ff06c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:13.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.621+0000 7ff07ed9d640 1 --2- 172.21.15.116:0/889765848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b59a34db60 0x55b59a34df40 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7ff078005cc0 tx=0x7ff078009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4186c0e4e5e11bd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:13.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.622+0000 7ff07e59c640 1 --2- 172.21.15.116:0/744230155 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff06c0a2b80 0x7ff06c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:13.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.622+0000 7ff0777fe640 1 -- 172.21.15.116:0/744230155 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff0680cf170 con 0x7ff080101420 2024-09-12T03:14:13.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.622+0000 7ff074ff9640 1 -- 172.21.15.116:0/889765848 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff078031ac0 con 0x55b59a34db60 2024-09-12T03:14:13.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.622+0000 7ff074ff9640 1 -- 172.21.15.116:0/889765848 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff078036040 con 0x55b59a34db60 2024-09-12T03:14:13.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.622+0000 7ff074ff9640 1 -- 172.21.15.116:0/889765848 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff078004720 con 0x55b59a34db60 2024-09-12T03:14:13.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.622+0000 7ff07e59c640 1 --2- 172.21.15.116:0/744230155 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff06c0a2b80 0x7ff06c0a5040 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x55b59a34af50 tx=0x7ff07000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:13.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.622+0000 7ff086dad740 1 -- 172.21.15.116:0/889765848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b59a34db60 msgr2=0x55b59a34df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:13.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.622+0000 7ff086dad740 1 --2- 172.21.15.116:0/889765848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b59a34db60 0x55b59a34df40 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7ff078005cc0 tx=0x7ff078009a20 comp rx=0 tx=0).stop 2024-09-12T03:14:13.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.622+0000 7ff086dad740 1 -- 172.21.15.116:0/889765848 shutdown_connections 2024-09-12T03:14:13.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.622+0000 7ff086dad740 1 --2- 172.21.15.116:0/889765848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b59a358d40 0x55b59a35b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.622+0000 7ff086dad740 1 --2- 172.21.15.116:0/889765848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b59a34e480 0x55b59a358800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.623+0000 7ff086dad740 1 --2- 172.21.15.116:0/889765848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b59a34db60 0x55b59a34df40 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.623+0000 7ff086dad740 1 -- 172.21.15.116:0/889765848 >> 172.21.15.116:0/889765848 conn(0x55b59a34a0f0 msgr2=0x55b59a34a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:13.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.623+0000 7ff086dad740 1 -- 172.21.15.116:0/889765848 shutdown_connections 2024-09-12T03:14:13.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.623+0000 7ff086dad740 1 -- 172.21.15.116:0/889765848 wait complete. 2024-09-12T03:14:13.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.623+0000 7ff086dad740 1 -- start start 2024-09-12T03:14:13.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.623+0000 7ff086dad740 1 Processor -- start 2024-09-12T03:14:13.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.623+0000 7ff086dad740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b59a34db60 0x55b59a355ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:13.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.623+0000 7ff086dad740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b59a34e480 0x55b59a3561e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:13.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.623+0000 7ff086dad740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b59a358d40 0x55b59a356720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:13.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.623+0000 7ff086dad740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b59a35b910 con 0x55b59a358d40 2024-09-12T03:14:13.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.623+0000 7ff086dad740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b59a26df10 con 0x55b59a34e480 2024-09-12T03:14:13.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.623+0000 7ff086dad740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b59a35ba90 con 0x55b59a34db60 2024-09-12T03:14:13.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.623+0000 7ff086dad740 10 client.0 _ll_register_callbacks cb 0x55b59a236a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:13.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.623+0000 7ff07ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b59a358d40 0x55b59a356720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:13.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.623+0000 7ff07ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b59a358d40 0x55b59a356720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46076/0 (socket says 172.21.15.116:46076) 2024-09-12T03:14:13.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.623+0000 7ff07ed9d640 1 -- 172.21.15.116:0/4175821504 learned_addr learned my addr 172.21.15.116:0/4175821504 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:13.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.624+0000 7ff077fff640 1 --2- 172.21.15.116:0/4175821504 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b59a34e480 0x55b59a3561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:13.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.624+0000 7ff07e59c640 1 --2- 172.21.15.116:0/4175821504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b59a34db60 0x55b59a355ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:13.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.624+0000 7ff07ed9d640 1 -- 172.21.15.116:0/4175821504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b59a34db60 msgr2=0x55b59a355ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:13.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.624+0000 7ff07ed9d640 1 --2- 172.21.15.116:0/4175821504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b59a34db60 0x55b59a355ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.624+0000 7ff07ed9d640 1 -- 172.21.15.116:0/4175821504 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b59a34e480 msgr2=0x55b59a3561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:13.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.624+0000 7ff07ed9d640 1 --2- 172.21.15.116:0/4175821504 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b59a34e480 0x55b59a3561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.624+0000 7ff07ed9d640 1 -- 172.21.15.116:0/4175821504 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b59a0246a0 con 0x55b59a358d40 2024-09-12T03:14:13.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.624+0000 7ff07e59c640 1 --2- 172.21.15.116:0/4175821504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b59a34db60 0x55b59a355ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:14:13.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.624+0000 7ff077fff640 1 --2- 172.21.15.116:0/4175821504 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b59a34e480 0x55b59a3561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:13.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.624+0000 7ff07ed9d640 1 --2- 172.21.15.116:0/4175821504 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b59a358d40 0x55b59a356720 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7ff07802ae00 tx=0x7ff078004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967557b289ce16f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:13.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.624+0000 7ff065ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:14:13.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.624+0000 7ff065ffb640 1 -- 172.21.15.116:0/4175821504 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff078003d80 con 0x55b59a358d40 2024-09-12T03:14:13.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.624+0000 7ff065ffb640 1 -- 172.21.15.116:0/4175821504 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff0780305a0 con 0x55b59a358d40 2024-09-12T03:14:13.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.624+0000 7ff086dad740 1 -- 172.21.15.116:0/4175821504 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff078036040 con 0x55b59a358d40 2024-09-12T03:14:13.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.624+0000 7ff065ffb640 1 -- 172.21.15.116:0/4175821504 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff0780316e0 con 0x55b59a358d40 2024-09-12T03:14:13.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.624+0000 7ff086dad740 10 client.18486 fetch_fsmap learned FSMap version 25 2024-09-12T03:14:13.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.624+0000 7ff086dad740 1 -- 172.21.15.116:0/4175821504 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b59a367590 con 0x55b59a358d40 2024-09-12T03:14:13.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.624+0000 7ff065ffb640 1 -- 172.21.15.116:0/4175821504 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff078036040 con 0x55b59a358d40 2024-09-12T03:14:13.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.625+0000 7ff065ffb640 1 -- 172.21.15.116:0/4175821504 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff078030070 con 0x55b59a358d40 2024-09-12T03:14:13.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.625+0000 7ff086dad740 10 client.18486 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:14:13.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.625+0000 7ff086dad740 10 client.18486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:14:13.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.625+0000 7ff086dad740 20 client.18486 populate_metadata read hostname 'smithi116' 2024-09-12T03:14:13.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.625+0000 7ff086dad740 1 --2- 172.21.15.116:0/4175821504 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b59a3688e0 0x55b59a36acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:13.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.625+0000 7ff086dad740 4 client.18486 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:14:13.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.625+0000 7ff086dad740 1 -- 172.21.15.116:0/4175821504 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b59a2c07f0 con 0x55b59a3688e0 2024-09-12T03:14:13.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.625+0000 7ff07e59c640 1 --2- 172.21.15.116:0/4175821504 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b59a3688e0 0x55b59a36acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:13.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.625+0000 7ff065ffb640 10 client.18486 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:13.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.625+0000 7ff07e59c640 1 --2- 172.21.15.116:0/4175821504 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b59a3688e0 0x55b59a36acd0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30506d241f19ad43 server_cookie=5e86ab03bc71e2b7 in_seq=0 out_seq=0 2024-09-12T03:14:13.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.626+0000 7ff065ffb640 1 -- 172.21.15.116:0/4175821504 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b59a2c07f0 con 0x55b59a3688e0 2024-09-12T03:14:13.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.626+0000 7ff065ffb640 10 client.18486 handle_command_reply: tid=0 2024-09-12T03:14:13.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.626+0000 7ff086dad740 1 client.18486 shutdown 2024-09-12T03:14:13.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.626+0000 7ff086dad740 1 -- 172.21.15.116:0/4175821504 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b59a358d40 msgr2=0x55b59a356720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:13.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.626+0000 7ff086dad740 1 --2- 172.21.15.116:0/4175821504 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b59a358d40 0x55b59a356720 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7ff07802ae00 tx=0x7ff078004750 comp rx=0 tx=0).stop 2024-09-12T03:14:13.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.626+0000 7ff086dad740 1 -- 172.21.15.116:0/4175821504 shutdown_connections 2024-09-12T03:14:13.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.626+0000 7ff086dad740 1 --2- 172.21.15.116:0/4175821504 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b59a3688e0 0x55b59a36acd0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.626+0000 7ff086dad740 1 --2- 172.21.15.116:0/4175821504 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b59a358d40 0x55b59a356720 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.626+0000 7ff086dad740 1 --2- 172.21.15.116:0/4175821504 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b59a34e480 0x55b59a3561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.626+0000 7ff086dad740 1 --2- 172.21.15.116:0/4175821504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b59a34db60 0x55b59a355ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.626+0000 7ff065ffb640 0 client.18486 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:13.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.626+0000 7ff086dad740 1 -- 172.21.15.116:0/4175821504 >> 172.21.15.116:0/4175821504 conn(0x55b59a34a0f0 msgr2=0x55b59a349270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:13.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.626+0000 7ff086dad740 1 -- 172.21.15.116:0/4175821504 shutdown_connections 2024-09-12T03:14:13.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.627+0000 7ff086dad740 1 -- 172.21.15.116:0/4175821504 wait complete. 2024-09-12T03:14:13.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.627+0000 7ff086dad740 20 client.18486 trim_cache size 0 max 16384 2024-09-12T03:14:13.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.644+0000 7ff086dad740 1 Processor -- start 2024-09-12T03:14:13.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.644+0000 7ff086dad740 1 -- start start 2024-09-12T03:14:13.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.644+0000 7ff086dad740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b59a34db60 0x55b59a3505c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:13.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.644+0000 7ff086dad740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b59a34e480 0x55b59a36dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:13.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.644+0000 7ff086dad740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b59a358d40 0x55b59a36e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:13.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.644+0000 7ff086dad740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff0801afc70 con 0x55b59a34e480 2024-09-12T03:14:13.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.644+0000 7ff086dad740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff0801ad540 con 0x55b59a34db60 2024-09-12T03:14:13.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.644+0000 7ff086dad740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0801ab390 con 0x55b59a358d40 2024-09-12T03:14:13.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.644+0000 7ff077fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b59a34e480 0x55b59a36dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:13.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.644+0000 7ff077fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b59a34e480 0x55b59a36dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46092/0 (socket says 172.21.15.116:46092) 2024-09-12T03:14:13.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.644+0000 7ff077fff640 1 -- 172.21.15.116:0/1486170934 learned_addr learned my addr 172.21.15.116:0/1486170934 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:13.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.644+0000 7ff07ed9d640 1 --2- 172.21.15.116:0/1486170934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b59a358d40 0x55b59a36e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:13.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.644+0000 7ff07e59c640 1 --2- 172.21.15.116:0/1486170934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b59a34db60 0x55b59a3505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:13.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.644+0000 7ff077fff640 1 -- 172.21.15.116:0/1486170934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b59a358d40 msgr2=0x55b59a36e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:13.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.644+0000 7ff077fff640 1 --2- 172.21.15.116:0/1486170934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b59a358d40 0x55b59a36e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.644+0000 7ff077fff640 1 -- 172.21.15.116:0/1486170934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b59a34db60 msgr2=0x55b59a3505c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:13.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.644+0000 7ff077fff640 1 --2- 172.21.15.116:0/1486170934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b59a34db60 0x55b59a3505c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.644+0000 7ff077fff640 1 -- 172.21.15.116:0/1486170934 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b59a3541f0 con 0x55b59a34e480 2024-09-12T03:14:13.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.644+0000 7ff07e59c640 1 --2- 172.21.15.116:0/1486170934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b59a34db60 0x55b59a3505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:14:13.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.644+0000 7ff077fff640 1 --2- 172.21.15.116:0/1486170934 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b59a34e480 0x55b59a36dcf0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7ff06800c8f0 tx=0x7ff068084f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1361b1729bd720 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:13.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.645+0000 7ff067fff640 1 -- 172.21.15.116:0/1486170934 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06809b360 con 0x55b59a34e480 2024-09-12T03:14:13.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.645+0000 7ff067fff640 1 -- 172.21.15.116:0/1486170934 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff06801f300 con 0x55b59a34e480 2024-09-12T03:14:13.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.645+0000 7ff067fff640 1 -- 172.21.15.116:0/1486170934 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06800f740 con 0x55b59a34e480 2024-09-12T03:14:13.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.645+0000 7ff086dad740 1 -- 172.21.15.116:0/1486170934 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b59a34e480 msgr2=0x55b59a36dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:13.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.645+0000 7ff086dad740 1 --2- 172.21.15.116:0/1486170934 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b59a34e480 0x55b59a36dcf0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7ff06800c8f0 tx=0x7ff068084f90 comp rx=0 tx=0).stop 2024-09-12T03:14:13.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.645+0000 7ff086dad740 1 -- 172.21.15.116:0/1486170934 shutdown_connections 2024-09-12T03:14:13.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.645+0000 7ff086dad740 1 --2- 172.21.15.116:0/1486170934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b59a358d40 0x55b59a36e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.645+0000 7ff086dad740 1 --2- 172.21.15.116:0/1486170934 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b59a34e480 0x55b59a36dcf0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.645+0000 7ff086dad740 1 --2- 172.21.15.116:0/1486170934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b59a34db60 0x55b59a3505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.645+0000 7ff086dad740 1 -- 172.21.15.116:0/1486170934 >> 172.21.15.116:0/1486170934 conn(0x55b59a34a0f0 msgr2=0x55b59a357c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:13.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.645+0000 7ff086dad740 1 -- 172.21.15.116:0/1486170934 shutdown_connections 2024-09-12T03:14:13.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.645+0000 7ff086dad740 1 -- 172.21.15.116:0/1486170934 wait complete. 2024-09-12T03:14:13.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.646+0000 7ff086dad740 1 -- start start 2024-09-12T03:14:13.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.646+0000 7ff086dad740 1 Processor -- start 2024-09-12T03:14:13.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.646+0000 7ff086dad740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b59a34db60 0x55b59a36c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:13.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.646+0000 7ff086dad740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b59a34e480 0x55b59a36c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:13.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.646+0000 7ff086dad740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b59a358d40 0x55b59a3d6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:13.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.646+0000 7ff086dad740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff0801aba00 con 0x55b59a34e480 2024-09-12T03:14:13.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.646+0000 7ff086dad740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff0801ace70 con 0x55b59a34db60 2024-09-12T03:14:13.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.646+0000 7ff086dad740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0801ac0d0 con 0x55b59a358d40 2024-09-12T03:14:13.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.646+0000 7ff077fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b59a34e480 0x55b59a36c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:13.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.646+0000 7ff086dad740 10 client.0 _ll_register_callbacks cb 0x7ff0680d2b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:13.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.646+0000 7ff077fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b59a34e480 0x55b59a36c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46094/0 (socket says 172.21.15.116:46094) 2024-09-12T03:14:13.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.646+0000 7ff077fff640 1 -- 172.21.15.116:0/2384361660 learned_addr learned my addr 172.21.15.116:0/2384361660 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:13.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.646+0000 7ff07e59c640 1 --2- 172.21.15.116:0/2384361660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b59a34db60 0x55b59a36c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:13.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.646+0000 7ff07ed9d640 1 --2- 172.21.15.116:0/2384361660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b59a358d40 0x55b59a3d6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:13.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.646+0000 7ff077fff640 1 -- 172.21.15.116:0/2384361660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b59a358d40 msgr2=0x55b59a3d6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:13.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.646+0000 7ff077fff640 1 --2- 172.21.15.116:0/2384361660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b59a358d40 0x55b59a3d6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.646+0000 7ff077fff640 1 -- 172.21.15.116:0/2384361660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b59a34db60 msgr2=0x55b59a36c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:13.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.646+0000 7ff077fff640 1 --2- 172.21.15.116:0/2384361660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b59a34db60 0x55b59a36c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.646+0000 7ff077fff640 1 -- 172.21.15.116:0/2384361660 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b59a3d6b70 con 0x55b59a34e480 2024-09-12T03:14:13.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.646+0000 7ff07ed9d640 1 --2- 172.21.15.116:0/2384361660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b59a358d40 0x55b59a3d6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:13.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.647+0000 7ff077fff640 1 --2- 172.21.15.116:0/2384361660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b59a34e480 0x55b59a36c660 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7ff0680cccf0 tx=0x7ff06809b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8d1c74a25bdec6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:13.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.647+0000 7ff065ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:14:13.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.647+0000 7ff065ffb640 1 -- 172.21.15.116:0/2384361660 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06809c100 con 0x55b59a34e480 2024-09-12T03:14:13.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.647+0000 7ff065ffb640 1 -- 172.21.15.116:0/2384361660 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff0680d2e00 con 0x55b59a34e480 2024-09-12T03:14:13.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.647+0000 7ff086dad740 1 -- 172.21.15.116:0/2384361660 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff06801f300 con 0x55b59a34e480 2024-09-12T03:14:13.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.647+0000 7ff065ffb640 1 -- 172.21.15.116:0/2384361660 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff0680ba020 con 0x55b59a34e480 2024-09-12T03:14:13.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.647+0000 7ff086dad740 10 client.18492 fetch_fsmap learned FSMap version 25 2024-09-12T03:14:13.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.647+0000 7ff086dad740 1 -- 172.21.15.116:0/2384361660 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b59a3d7d80 con 0x55b59a34e480 2024-09-12T03:14:13.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.647+0000 7ff065ffb640 1 -- 172.21.15.116:0/2384361660 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff06801f300 con 0x55b59a34e480 2024-09-12T03:14:13.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.647+0000 7ff065ffb640 1 -- 172.21.15.116:0/2384361660 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff0680ba250 con 0x55b59a34e480 2024-09-12T03:14:13.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.647+0000 7ff086dad740 10 client.18492 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:14:13.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.647+0000 7ff086dad740 10 client.18492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:14:13.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.647+0000 7ff086dad740 20 client.18492 populate_metadata read hostname 'smithi116' 2024-09-12T03:14:13.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.647+0000 7ff086dad740 1 --2- 172.21.15.116:0/2384361660 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b59a3d8450 0x55b59a3da870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:13.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.647+0000 7ff086dad740 4 client.18492 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:14:13.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.647+0000 7ff086dad740 1 -- 172.21.15.116:0/2384361660 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b59a2bad00 con 0x55b59a3d8450 2024-09-12T03:14:13.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.647+0000 7ff07e59c640 1 --2- 172.21.15.116:0/2384361660 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b59a3d8450 0x55b59a3da870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:13.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.648+0000 7ff07e59c640 1 --2- 172.21.15.116:0/2384361660 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b59a3d8450 0x55b59a3da870 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6138f51ebfe72249 server_cookie=a3c5a4837a538db1 in_seq=0 out_seq=0 2024-09-12T03:14:13.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.648+0000 7ff065ffb640 10 client.18492 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:13.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.649+0000 7ff065ffb640 1 -- 172.21.15.116:0/2384361660 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b59a2bad00 con 0x55b59a3d8450 2024-09-12T03:14:13.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.649+0000 7ff065ffb640 10 client.18492 handle_command_reply: tid=0 2024-09-12T03:14:13.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.649+0000 7ff086dad740 1 client.18492 shutdown 2024-09-12T03:14:13.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.649+0000 7ff086dad740 1 -- 172.21.15.116:0/2384361660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b59a34e480 msgr2=0x55b59a36c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:13.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.649+0000 7ff086dad740 1 --2- 172.21.15.116:0/2384361660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b59a34e480 0x55b59a36c660 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7ff0680cccf0 tx=0x7ff06809b460 comp rx=0 tx=0).stop 2024-09-12T03:14:13.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.649+0000 7ff086dad740 1 -- 172.21.15.116:0/2384361660 shutdown_connections 2024-09-12T03:14:13.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.649+0000 7ff086dad740 1 --2- 172.21.15.116:0/2384361660 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b59a3d8450 0x55b59a3da870 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.649+0000 7ff086dad740 1 --2- 172.21.15.116:0/2384361660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b59a358d40 0x55b59a3d6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.649+0000 7ff086dad740 1 --2- 172.21.15.116:0/2384361660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b59a34e480 0x55b59a36c660 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.649+0000 7ff086dad740 1 --2- 172.21.15.116:0/2384361660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b59a34db60 0x55b59a36c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.649+0000 7ff086dad740 1 -- 172.21.15.116:0/2384361660 >> 172.21.15.116:0/2384361660 conn(0x55b59a34a0f0 msgr2=0x55b59a348a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:13.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.649+0000 7ff065ffb640 0 client.18492 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:13.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.650+0000 7ff086dad740 1 -- 172.21.15.116:0/2384361660 shutdown_connections 2024-09-12T03:14:13.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.650+0000 7ff086dad740 1 -- 172.21.15.116:0/2384361660 wait complete. 2024-09-12T03:14:13.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.650+0000 7ff086dad740 20 client.18492 trim_cache size 0 max 16384 2024-09-12T03:14:13.655 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:14:13.655 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:14:13.655 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "ebc56093-8ba7-4a23-b2ea-9ea0d555ee56", 2024-09-12T03:14:13.655 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:14:13.656 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:14:13.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.650+0000 7ff0849d4640 1 -- 172.21.15.116:0/744230155 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff06c0a2b80 msgr2=0x7ff06c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:13.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.650+0000 7ff0849d4640 1 --2- 172.21.15.116:0/744230155 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff06c0a2b80 0x7ff06c0a5040 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x55b59a34af50 tx=0x7ff07000a2d0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.650+0000 7ff0849d4640 1 -- 172.21.15.116:0/744230155 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff080101420 msgr2=0x7ff080198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:13.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.650+0000 7ff0849d4640 1 --2- 172.21.15.116:0/744230155 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff080101420 0x7ff080198890 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7ff06800c920 tx=0x7ff06800cd20 comp rx=0 tx=0).stop 2024-09-12T03:14:13.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.650+0000 7ff0849d4640 1 -- 172.21.15.116:0/744230155 shutdown_connections 2024-09-12T03:14:13.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.650+0000 7ff0849d4640 1 --2- 172.21.15.116:0/744230155 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff06c0a2b80 0x7ff06c0a5040 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.650+0000 7ff0849d4640 1 --2- 172.21.15.116:0/744230155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff080106a80 0x7ff08019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.650+0000 7ff0849d4640 1 --2- 172.21.15.116:0/744230155 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff080101420 0x7ff080198890 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.650+0000 7ff0849d4640 1 --2- 172.21.15.116:0/744230155 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff080100a80 0x7ff080198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:13.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.650+0000 7ff0849d4640 1 -- 172.21.15.116:0/744230155 >> 172.21.15.116:0/744230155 conn(0x7ff0800fc7a0 msgr2=0x7ff08006a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:13.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.650+0000 7ff0849d4640 1 -- 172.21.15.116:0/744230155 shutdown_connections 2024-09-12T03:14:13.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:13.650+0000 7ff0849d4640 1 -- 172.21.15.116:0/744230155 wait complete. 2024-09-12T03:14:13.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:13 smithi183 ceph-mon[35535]: pgmap v675: 97 pgs: 97 active+clean; 5.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:14:13.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:13 smithi177 ceph-mon[35305]: pgmap v675: 97 pgs: 97 active+clean; 5.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:14:14.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:14 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:14:14.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:14 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:14:14.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:14 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:14:14.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:14 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:14:14.550 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:14 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:14:14.551 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:14:14.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:14 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:14:14.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:14 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:14:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:14 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:14:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:14 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:14:14.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:14 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:14:14.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:14 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:14:14.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:14 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:14:14.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:14 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:14:14.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:14 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:14:14.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:14 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:14:14.799 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:14:15.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.263+0000 7f04e5649640 1 -- 172.21.15.116:0/120856005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04e0101bc0 msgr2=0x7f04e010d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:15.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.263+0000 7f04e5649640 1 --2- 172.21.15.116:0/120856005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04e0101bc0 0x7f04e010d910 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009ca0 tx=0x7f04d802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.263+0000 7f04e5649640 1 -- 172.21.15.116:0/120856005 shutdown_connections 2024-09-12T03:14:15.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.263+0000 7f04e5649640 1 --2- 172.21.15.116:0/120856005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04e0101bc0 0x7f04e010d910 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.263+0000 7f04e5649640 1 --2- 172.21.15.116:0/120856005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04e0101240 0x7f04e0101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.263+0000 7f04e5649640 1 --2- 172.21.15.116:0/120856005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04e0105a90 0x7f04e0105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.263+0000 7f04e5649640 1 -- 172.21.15.116:0/120856005 >> 172.21.15.116:0/120856005 conn(0x7f04e0077210 msgr2=0x7f04e00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:15.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.263+0000 7f04e5649640 1 -- 172.21.15.116:0/120856005 shutdown_connections 2024-09-12T03:14:15.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.263+0000 7f04e5649640 1 -- 172.21.15.116:0/120856005 wait complete. 2024-09-12T03:14:15.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.264+0000 7f04e5649640 1 Processor -- start 2024-09-12T03:14:15.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.264+0000 7f04e5649640 1 -- start start 2024-09-12T03:14:15.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.264+0000 7f04e5649640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04e0101240 0x7f04e01a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:15.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.264+0000 7f04e5649640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04e0101bc0 0x7f04e01a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:15.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.264+0000 7f04e5649640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04e0105a90 0x7f04e019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:15.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.264+0000 7f04e5649640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f04e0114340 con 0x7f04e0101bc0 2024-09-12T03:14:15.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.264+0000 7f04e5649640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01141c0 con 0x7f04e0105a90 2024-09-12T03:14:15.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.264+0000 7f04e5649640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01144c0 con 0x7f04e0101240 2024-09-12T03:14:15.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.264+0000 7f04de7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04e0101bc0 0x7f04e01a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:15.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.264+0000 7f04de7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04e0101bc0 0x7f04e01a0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46116/0 (socket says 172.21.15.116:46116) 2024-09-12T03:14:15.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.264+0000 7f04de7fc640 1 -- 172.21.15.116:0/1528832024 learned_addr learned my addr 172.21.15.116:0/1528832024 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:15.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.265+0000 7f04df7fe640 1 --2- 172.21.15.116:0/1528832024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04e0105a90 0x7f04e019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:15.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.265+0000 7f04deffd640 1 --2- 172.21.15.116:0/1528832024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04e0101240 0x7f04e01a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:15.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.265+0000 7f04de7fc640 1 -- 172.21.15.116:0/1528832024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04e0101240 msgr2=0x7f04e01a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:15.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.265+0000 7f04de7fc640 1 --2- 172.21.15.116:0/1528832024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04e0101240 0x7f04e01a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.265+0000 7f04de7fc640 1 -- 172.21.15.116:0/1528832024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04e0105a90 msgr2=0x7f04e019a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:15.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.265+0000 7f04de7fc640 1 --2- 172.21.15.116:0/1528832024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04e0105a90 0x7f04e019a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.265+0000 7f04de7fc640 1 -- 172.21.15.116:0/1528832024 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e019af00 con 0x7f04e0101bc0 2024-09-12T03:14:15.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.265+0000 7f04de7fc640 1 --2- 172.21.15.116:0/1528832024 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04e0101bc0 0x7f04e01a0ab0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f04d400b4a0 tx=0x7f04d400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:15.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.265+0000 7f04c7fff640 1 -- 172.21.15.116:0/1528832024 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4015020 con 0x7f04e0101bc0 2024-09-12T03:14:15.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.265+0000 7f04c7fff640 1 -- 172.21.15.116:0/1528832024 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04d4011d10 con 0x7f04e0101bc0 2024-09-12T03:14:15.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.265+0000 7f04c7fff640 1 -- 172.21.15.116:0/1528832024 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d400f2e0 con 0x7f04e0101bc0 2024-09-12T03:14:15.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.265+0000 7f04e5649640 1 -- 172.21.15.116:0/1528832024 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e019b1f0 con 0x7f04e0101bc0 2024-09-12T03:14:15.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.265+0000 7f04e5649640 1 -- 172.21.15.116:0/1528832024 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e01a7560 con 0x7f04e0101bc0 2024-09-12T03:14:15.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.267+0000 7f04c7fff640 1 -- 172.21.15.116:0/1528832024 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f04d4018020 con 0x7f04e0101bc0 2024-09-12T03:14:15.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.268+0000 7f04e7a22740 1 Processor -- start 2024-09-12T03:14:15.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.268+0000 7f04e7a22740 1 -- start start 2024-09-12T03:14:15.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.268+0000 7f04e7a22740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557efd21c4a0 0x557efd21c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:15.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.268+0000 7f04e7a22740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557efd21cdc0 0x557efd227140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:15.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.268+0000 7f04e7a22740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557efd227680 0x557efd229a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:15.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.268+0000 7f04e7a22740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557efd1822e0 con 0x557efd227680 2024-09-12T03:14:15.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.268+0000 7f04e7a22740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557efd129ce0 con 0x557efd21cdc0 2024-09-12T03:14:15.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.268+0000 7f04e7a22740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557efd17e2a0 con 0x557efd21c4a0 2024-09-12T03:14:15.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.268+0000 7f04c7fff640 1 --2- 172.21.15.116:0/1528832024 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f04c80a2b70 0x7f04c80a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:15.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.268+0000 7f04de7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557efd227680 0x557efd229a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:15.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.268+0000 7f04de7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557efd227680 0x557efd229a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46130/0 (socket says 172.21.15.116:46130) 2024-09-12T03:14:15.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.268+0000 7f04de7fc640 1 -- 172.21.15.116:0/3613427998 learned_addr learned my addr 172.21.15.116:0/3613427998 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:15.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.268+0000 7f04df7fe640 1 --2- 172.21.15.116:0/3613427998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557efd21c4a0 0x557efd21c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:15.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.268+0000 7f04df7fe640 1 --2- 172.21.15.116:0/1528832024 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f04c80a2b70 0x7f04c80a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:15.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.268+0000 7f04deffd640 1 --2- 172.21.15.116:0/3613427998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557efd21cdc0 0x557efd227140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:15.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.268+0000 7f04c7fff640 1 -- 172.21.15.116:0/1528832024 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f04d40ce860 con 0x7f04e0101bc0 2024-09-12T03:14:15.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.268+0000 7f04de7fc640 1 -- 172.21.15.116:0/3613427998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557efd21c4a0 msgr2=0x557efd21c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:15.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.268+0000 7f04de7fc640 1 --2- 172.21.15.116:0/3613427998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557efd21c4a0 0x557efd21c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.268+0000 7f04de7fc640 1 -- 172.21.15.116:0/3613427998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557efd21cdc0 msgr2=0x557efd227140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:15.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.268+0000 7f04de7fc640 1 --2- 172.21.15.116:0/3613427998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557efd21cdc0 0x557efd227140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.268+0000 7f04de7fc640 1 -- 172.21.15.116:0/3613427998 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557efd10e440 con 0x557efd227680 2024-09-12T03:14:15.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.268+0000 7f04df7fe640 1 --2- 172.21.15.116:0/3613427998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557efd21c4a0 0x557efd21c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:15.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.269+0000 7f04de7fc640 1 --2- 172.21.15.116:0/3613427998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557efd227680 0x557efd229a70 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f04d40cc2d0 tx=0x7f04d4096d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e15f14a2cd0ed7c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:15.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.269+0000 7f04df7fe640 1 --2- 172.21.15.116:0/1528832024 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f04c80a2b70 0x7f04c80a5030 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f04d80098d0 tx=0x7f04d80095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:15.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.269+0000 7f04c57fa640 1 -- 172.21.15.116:0/3613427998 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d409b7f0 con 0x557efd227680 2024-09-12T03:14:15.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.269+0000 7f04c57fa640 1 -- 172.21.15.116:0/3613427998 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04d40d1e50 con 0x557efd227680 2024-09-12T03:14:15.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.269+0000 7f04c57fa640 1 -- 172.21.15.116:0/3613427998 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4010510 con 0x557efd227680 2024-09-12T03:14:15.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.269+0000 7f04e7a22740 1 -- 172.21.15.116:0/3613427998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557efd227680 msgr2=0x557efd229a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:15.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.269+0000 7f04e7a22740 1 --2- 172.21.15.116:0/3613427998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557efd227680 0x557efd229a70 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f04d40cc2d0 tx=0x7f04d4096d90 comp rx=0 tx=0).stop 2024-09-12T03:14:15.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.270+0000 7f04e7a22740 1 -- 172.21.15.116:0/3613427998 shutdown_connections 2024-09-12T03:14:15.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.270+0000 7f04e7a22740 1 --2- 172.21.15.116:0/3613427998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557efd227680 0x557efd229a70 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.270+0000 7f04e7a22740 1 --2- 172.21.15.116:0/3613427998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557efd21cdc0 0x557efd227140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.270+0000 7f04e7a22740 1 --2- 172.21.15.116:0/3613427998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557efd21c4a0 0x557efd21c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.299 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.270+0000 7f04e7a22740 1 -- 172.21.15.116:0/3613427998 >> 172.21.15.116:0/3613427998 conn(0x557efd218a30 msgr2=0x557efd218e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:15.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.270+0000 7f04e7a22740 1 -- 172.21.15.116:0/3613427998 shutdown_connections 2024-09-12T03:14:15.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.270+0000 7f04e7a22740 1 -- 172.21.15.116:0/3613427998 wait complete. 2024-09-12T03:14:15.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.270+0000 7f04e7a22740 1 -- start start 2024-09-12T03:14:15.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.270+0000 7f04e7a22740 1 Processor -- start 2024-09-12T03:14:15.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.270+0000 7f04e7a22740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557efd21c4a0 0x557efd226f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:15.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.270+0000 7f04e7a22740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557efd21cdc0 0x557efd224240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:15.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.270+0000 7f04e7a22740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557efd227680 0x557efd224780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:15.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.270+0000 7f04e7a22740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557efd22a1f0 con 0x557efd227680 2024-09-12T03:14:15.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.270+0000 7f04e7a22740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557efd13d050 con 0x557efd21cdc0 2024-09-12T03:14:15.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.270+0000 7f04e7a22740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557efd22a370 con 0x557efd21c4a0 2024-09-12T03:14:15.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.270+0000 7f04e7a22740 10 client.0 _ll_register_callbacks cb 0x557efd17d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:15.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.270+0000 7f04df7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557efd227680 0x557efd224780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:15.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.270+0000 7f04df7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557efd227680 0x557efd224780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46132/0 (socket says 172.21.15.116:46132) 2024-09-12T03:14:15.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.270+0000 7f04df7fe640 1 -- 172.21.15.116:0/3462820180 learned_addr learned my addr 172.21.15.116:0/3462820180 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:15.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.271+0000 7f04de7fc640 1 --2- 172.21.15.116:0/3462820180 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557efd21cdc0 0x557efd224240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:15.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.271+0000 7f04deffd640 1 --2- 172.21.15.116:0/3462820180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557efd21c4a0 0x557efd226f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:15.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.271+0000 7f04df7fe640 1 -- 172.21.15.116:0/3462820180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557efd21c4a0 msgr2=0x557efd226f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:15.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.271+0000 7f04df7fe640 1 --2- 172.21.15.116:0/3462820180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557efd21c4a0 0x557efd226f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.271+0000 7f04df7fe640 1 -- 172.21.15.116:0/3462820180 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557efd21cdc0 msgr2=0x557efd224240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:15.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.271+0000 7f04df7fe640 1 --2- 172.21.15.116:0/3462820180 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557efd21cdc0 0x557efd224240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.271+0000 7f04df7fe640 1 -- 172.21.15.116:0/3462820180 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557efcef33c0 con 0x557efd227680 2024-09-12T03:14:15.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.271+0000 7f04deffd640 1 --2- 172.21.15.116:0/3462820180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557efd21c4a0 0x557efd226f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:15.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.271+0000 7f04de7fc640 1 --2- 172.21.15.116:0/3462820180 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557efd21cdc0 0x557efd224240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:15.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.271+0000 7f04df7fe640 1 --2- 172.21.15.116:0/3462820180 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557efd227680 0x557efd224780 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f04d8035000 tx=0x7f04d802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0005c564f5e5a5a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:15.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.271+0000 7f04c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:14:15.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.271+0000 7f04c2ff5640 1 -- 172.21.15.116:0/3462820180 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d8042070 con 0x557efd227680 2024-09-12T03:14:15.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.271+0000 7f04c2ff5640 1 -- 172.21.15.116:0/3462820180 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f04d8031920 con 0x557efd227680 2024-09-12T03:14:15.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.271+0000 7f04e7a22740 1 -- 172.21.15.116:0/3462820180 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04d40d1e50 con 0x557efd227680 2024-09-12T03:14:15.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.271+0000 7f04c2ff5640 1 -- 172.21.15.116:0/3462820180 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f04d8045040 con 0x557efd227680 2024-09-12T03:14:15.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.271+0000 7f04e7a22740 10 client.18504 fetch_fsmap learned FSMap version 25 2024-09-12T03:14:15.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.271+0000 7f04e7a22740 1 -- 172.21.15.116:0/3462820180 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557efd235ba0 con 0x557efd227680 2024-09-12T03:14:15.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.272+0000 7f04c2ff5640 1 -- 172.21.15.116:0/3462820180 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04d40d1e50 con 0x557efd227680 2024-09-12T03:14:15.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.272+0000 7f04c2ff5640 1 -- 172.21.15.116:0/3462820180 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f04d80308c0 con 0x557efd227680 2024-09-12T03:14:15.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.272+0000 7f04e7a22740 10 client.18504 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:14:15.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.272+0000 7f04e7a22740 10 client.18504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:14:15.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.272+0000 7f04e7a22740 20 client.18504 populate_metadata read hostname 'smithi116' 2024-09-12T03:14:15.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.272+0000 7f04e7a22740 1 --2- 172.21.15.116:0/3462820180 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557efd237060 0x557efd239490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:15.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.272+0000 7f04e7a22740 4 client.18504 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:14:15.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.272+0000 7f04e7a22740 1 -- 172.21.15.116:0/3462820180 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557efd18f850 con 0x557efd237060 2024-09-12T03:14:15.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.272+0000 7f04deffd640 1 --2- 172.21.15.116:0/3462820180 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557efd237060 0x557efd239490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:15.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.272+0000 7f04c2ff5640 10 client.18504 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:15.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.272+0000 7f04deffd640 1 --2- 172.21.15.116:0/3462820180 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557efd237060 0x557efd239490 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aa5bee8449beec2 server_cookie=5af3dfa515f1fbf in_seq=0 out_seq=0 2024-09-12T03:14:15.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.273+0000 7f04c2ff5640 1 -- 172.21.15.116:0/3462820180 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557efd18f850 con 0x557efd237060 2024-09-12T03:14:15.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.273+0000 7f04c2ff5640 10 client.18504 handle_command_reply: tid=0 2024-09-12T03:14:15.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.273+0000 7f04e7a22740 1 client.18504 shutdown 2024-09-12T03:14:15.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.273+0000 7f04e7a22740 1 -- 172.21.15.116:0/3462820180 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557efd227680 msgr2=0x557efd224780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:15.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.273+0000 7f04e7a22740 1 --2- 172.21.15.116:0/3462820180 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557efd227680 0x557efd224780 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f04d8035000 tx=0x7f04d802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.273+0000 7f04e7a22740 1 -- 172.21.15.116:0/3462820180 shutdown_connections 2024-09-12T03:14:15.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.273+0000 7f04e7a22740 1 --2- 172.21.15.116:0/3462820180 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557efd237060 0x557efd239490 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.273+0000 7f04e7a22740 1 --2- 172.21.15.116:0/3462820180 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557efd227680 0x557efd224780 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.303 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.273+0000 7f04c2ff5640 0 client.18504 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:15.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.273+0000 7f04e7a22740 1 --2- 172.21.15.116:0/3462820180 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557efd21cdc0 0x557efd224240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.273+0000 7f04e7a22740 1 --2- 172.21.15.116:0/3462820180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557efd21c4a0 0x557efd226f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.273+0000 7f04e7a22740 1 -- 172.21.15.116:0/3462820180 >> 172.21.15.116:0/3462820180 conn(0x557efd218a30 msgr2=0x557efd21e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:15.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.273+0000 7f04e7a22740 1 -- 172.21.15.116:0/3462820180 shutdown_connections 2024-09-12T03:14:15.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.273+0000 7f04e7a22740 1 -- 172.21.15.116:0/3462820180 wait complete. 2024-09-12T03:14:15.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.273+0000 7f04e7a22740 20 client.18504 trim_cache size 0 max 16384 2024-09-12T03:14:15.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.290+0000 7f04e7a22740 1 Processor -- start 2024-09-12T03:14:15.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.290+0000 7f04e7a22740 1 -- start start 2024-09-12T03:14:15.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.290+0000 7f04e7a22740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557efd21c4a0 0x557efd233480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:15.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.290+0000 7f04e7a22740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557efd21cdc0 0x557efd222b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:15.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.290+0000 7f04e7a22740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557efd227680 0x557efd2261b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:15.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.290+0000 7f04e7a22740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01b2080 con 0x557efd21c4a0 2024-09-12T03:14:15.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.290+0000 7f04e7a22740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01af950 con 0x557efd21cdc0 2024-09-12T03:14:15.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.290+0000 7f04e7a22740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01ad7a0 con 0x557efd227680 2024-09-12T03:14:15.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.291+0000 7f04deffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557efd21c4a0 0x557efd233480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:15.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.291+0000 7f04deffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557efd21c4a0 0x557efd233480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46142/0 (socket says 172.21.15.116:46142) 2024-09-12T03:14:15.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.291+0000 7f04deffd640 1 -- 172.21.15.116:0/3319036885 learned_addr learned my addr 172.21.15.116:0/3319036885 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:15.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.291+0000 7f04de7fc640 1 --2- 172.21.15.116:0/3319036885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557efd21cdc0 0x557efd222b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:15.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.291+0000 7f04deffd640 1 -- 172.21.15.116:0/3319036885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557efd227680 msgr2=0x557efd2261b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:15.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.291+0000 7f04df7fe640 1 --2- 172.21.15.116:0/3319036885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557efd227680 0x557efd2261b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:15.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.291+0000 7f04deffd640 1 --2- 172.21.15.116:0/3319036885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557efd227680 0x557efd2261b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.291+0000 7f04deffd640 1 -- 172.21.15.116:0/3319036885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557efd21cdc0 msgr2=0x557efd222b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:15.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.291+0000 7f04deffd640 1 --2- 172.21.15.116:0/3319036885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557efd21cdc0 0x557efd222b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.291+0000 7f04deffd640 1 -- 172.21.15.116:0/3319036885 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557efd2231f0 con 0x557efd21c4a0 2024-09-12T03:14:15.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.291+0000 7f04de7fc640 1 --2- 172.21.15.116:0/3319036885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557efd21cdc0 0x557efd222b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:15.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.291+0000 7f04df7fe640 1 --2- 172.21.15.116:0/3319036885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557efd227680 0x557efd2261b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:14:15.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.291+0000 7f04deffd640 1 --2- 172.21.15.116:0/3319036885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557efd21c4a0 0x557efd233480 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f04d0009bc0 tx=0x7f04d0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a382844510ef54 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:15.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.291+0000 7f04c4ff9640 1 -- 172.21.15.116:0/3319036885 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0026420 con 0x557efd21c4a0 2024-09-12T03:14:15.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.291+0000 7f04c4ff9640 1 -- 172.21.15.116:0/3319036885 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04d0021030 con 0x557efd21c4a0 2024-09-12T03:14:15.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.291+0000 7f04c4ff9640 1 -- 172.21.15.116:0/3319036885 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d001d380 con 0x557efd21c4a0 2024-09-12T03:14:15.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.292+0000 7f04e7a22740 1 -- 172.21.15.116:0/3319036885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557efd21c4a0 msgr2=0x557efd233480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:15.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.292+0000 7f04e7a22740 1 --2- 172.21.15.116:0/3319036885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557efd21c4a0 0x557efd233480 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f04d0009bc0 tx=0x7f04d0008040 comp rx=0 tx=0).stop 2024-09-12T03:14:15.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.292+0000 7f04e7a22740 1 -- 172.21.15.116:0/3319036885 shutdown_connections 2024-09-12T03:14:15.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.292+0000 7f04e7a22740 1 --2- 172.21.15.116:0/3319036885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557efd227680 0x557efd2261b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.292+0000 7f04e7a22740 1 --2- 172.21.15.116:0/3319036885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557efd21cdc0 0x557efd222b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.292+0000 7f04e7a22740 1 --2- 172.21.15.116:0/3319036885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557efd21c4a0 0x557efd233480 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.292+0000 7f04e7a22740 1 -- 172.21.15.116:0/3319036885 >> 172.21.15.116:0/3319036885 conn(0x557efd218a30 msgr2=0x557efd225aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:15.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.292+0000 7f04e7a22740 1 -- 172.21.15.116:0/3319036885 shutdown_connections 2024-09-12T03:14:15.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.292+0000 7f04e7a22740 1 -- 172.21.15.116:0/3319036885 wait complete. 2024-09-12T03:14:15.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.292+0000 7f04e7a22740 1 -- start start 2024-09-12T03:14:15.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.292+0000 7f04e7a22740 1 Processor -- start 2024-09-12T03:14:15.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.293+0000 7f04e7a22740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557efd21c4a0 0x557efd22bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:15.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.293+0000 7f04e7a22740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557efd21cdc0 0x557efd22c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:15.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.293+0000 7f04e7a22740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557efd227680 0x557efd2a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:15.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.293+0000 7f04e7a22740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01ade10 con 0x557efd21c4a0 2024-09-12T03:14:15.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.293+0000 7f04e7a22740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01af280 con 0x557efd21cdc0 2024-09-12T03:14:15.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.293+0000 7f04e7a22740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01ae4e0 con 0x557efd227680 2024-09-12T03:14:15.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.293+0000 7f04e7a22740 10 client.0 _ll_register_callbacks cb 0x7f04d8032ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:15.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.293+0000 7f04deffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557efd21c4a0 0x557efd22bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:15.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.293+0000 7f04deffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557efd21c4a0 0x557efd22bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46158/0 (socket says 172.21.15.116:46158) 2024-09-12T03:14:15.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.293+0000 7f04deffd640 1 -- 172.21.15.116:0/2912876465 learned_addr learned my addr 172.21.15.116:0/2912876465 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:15.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.293+0000 7f04df7fe640 1 --2- 172.21.15.116:0/2912876465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557efd227680 0x557efd2a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:15.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.293+0000 7f04de7fc640 1 --2- 172.21.15.116:0/2912876465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557efd21cdc0 0x557efd22c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:15.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.293+0000 7f04deffd640 1 -- 172.21.15.116:0/2912876465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557efd227680 msgr2=0x557efd2a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:15.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.293+0000 7f04deffd640 1 --2- 172.21.15.116:0/2912876465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557efd227680 0x557efd2a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.293+0000 7f04deffd640 1 -- 172.21.15.116:0/2912876465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557efd21cdc0 msgr2=0x557efd22c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:15.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.293+0000 7f04deffd640 1 --2- 172.21.15.116:0/2912876465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557efd21cdc0 0x557efd22c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.293+0000 7f04deffd640 1 -- 172.21.15.116:0/2912876465 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557efd2a1370 con 0x557efd21c4a0 2024-09-12T03:14:15.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.293+0000 7f04de7fc640 1 --2- 172.21.15.116:0/2912876465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557efd21cdc0 0x557efd22c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:15.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.293+0000 7f04deffd640 1 --2- 172.21.15.116:0/2912876465 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557efd21c4a0 0x557efd22bc00 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f04d0006aa0 tx=0x7f04d001d1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8422d48304d2c9f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:15.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.293+0000 7f04c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:14:15.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.293+0000 7f04c2ff5640 1 -- 172.21.15.116:0/2912876465 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d00258e0 con 0x557efd21c4a0 2024-09-12T03:14:15.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.293+0000 7f04c2ff5640 1 -- 172.21.15.116:0/2912876465 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f04d0025a80 con 0x557efd21c4a0 2024-09-12T03:14:15.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.293+0000 7f04e7a22740 1 -- 172.21.15.116:0/2912876465 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04d0021030 con 0x557efd21c4a0 2024-09-12T03:14:15.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.294+0000 7f04c2ff5640 1 -- 172.21.15.116:0/2912876465 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f04d0025c80 con 0x557efd21c4a0 2024-09-12T03:14:15.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.294+0000 7f04e7a22740 10 client.18510 fetch_fsmap learned FSMap version 25 2024-09-12T03:14:15.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.294+0000 7f04e7a22740 1 -- 172.21.15.116:0/2912876465 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557efd2a25d0 con 0x557efd21c4a0 2024-09-12T03:14:15.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.294+0000 7f04c2ff5640 1 -- 172.21.15.116:0/2912876465 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04d0021030 con 0x557efd21c4a0 2024-09-12T03:14:15.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.294+0000 7f04c2ff5640 1 -- 172.21.15.116:0/2912876465 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f04d001d480 con 0x557efd21c4a0 2024-09-12T03:14:15.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.294+0000 7f04e7a22740 10 client.18510 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:14:15.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.294+0000 7f04e7a22740 10 client.18510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:14:15.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.294+0000 7f04e7a22740 20 client.18510 populate_metadata read hostname 'smithi116' 2024-09-12T03:14:15.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.294+0000 7f04e7a22740 1 --2- 172.21.15.116:0/2912876465 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557efd2a2c30 0x557efd2a5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:15.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.294+0000 7f04e7a22740 4 client.18510 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:14:15.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.294+0000 7f04e7a22740 1 -- 172.21.15.116:0/2912876465 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557efd189d60 con 0x557efd2a2c30 2024-09-12T03:14:15.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.294+0000 7f04de7fc640 1 --2- 172.21.15.116:0/2912876465 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557efd2a2c30 0x557efd2a5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:15.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.295+0000 7f04de7fc640 1 --2- 172.21.15.116:0/2912876465 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557efd2a2c30 0x557efd2a5020 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59dc182999fddcea server_cookie=118cee09b08bfee6 in_seq=0 out_seq=0 2024-09-12T03:14:15.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.295+0000 7f04c2ff5640 10 client.18510 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:15.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.295+0000 7f04c2ff5640 1 -- 172.21.15.116:0/2912876465 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557efd189d60 con 0x557efd2a2c30 2024-09-12T03:14:15.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.295+0000 7f04c2ff5640 10 client.18510 handle_command_reply: tid=0 2024-09-12T03:14:15.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.295+0000 7f04e7a22740 1 client.18510 shutdown 2024-09-12T03:14:15.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.295+0000 7f04e7a22740 1 -- 172.21.15.116:0/2912876465 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557efd21c4a0 msgr2=0x557efd22bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:15.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.295+0000 7f04e7a22740 1 --2- 172.21.15.116:0/2912876465 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557efd21c4a0 0x557efd22bc00 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f04d0006aa0 tx=0x7f04d001d1b0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.295+0000 7f04e7a22740 1 -- 172.21.15.116:0/2912876465 shutdown_connections 2024-09-12T03:14:15.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.295+0000 7f04e7a22740 1 --2- 172.21.15.116:0/2912876465 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557efd2a2c30 0x557efd2a5020 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.295+0000 7f04e7a22740 1 --2- 172.21.15.116:0/2912876465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557efd227680 0x557efd2a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.295+0000 7f04c2ff5640 0 client.18510 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:15.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.295+0000 7f04e7a22740 1 --2- 172.21.15.116:0/2912876465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557efd21cdc0 0x557efd22c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.295+0000 7f04e7a22740 1 --2- 172.21.15.116:0/2912876465 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557efd21c4a0 0x557efd22bc00 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.295+0000 7f04e7a22740 1 -- 172.21.15.116:0/2912876465 >> 172.21.15.116:0/2912876465 conn(0x557efd218a30 msgr2=0x557efd217350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:15.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.295+0000 7f04e7a22740 1 -- 172.21.15.116:0/2912876465 shutdown_connections 2024-09-12T03:14:15.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.295+0000 7f04e7a22740 1 -- 172.21.15.116:0/2912876465 wait complete. 2024-09-12T03:14:15.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.295+0000 7f04e7a22740 20 client.18510 trim_cache size 0 max 16384 2024-09-12T03:14:15.313 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:14:15.313 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:14:15.313 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:14:15.313 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:14:15.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.296+0000 7f04e5649640 1 -- 172.21.15.116:0/1528832024 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f04c80a2b70 msgr2=0x7f04c80a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:15.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.296+0000 7f04e5649640 1 --2- 172.21.15.116:0/1528832024 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f04c80a2b70 0x7f04c80a5030 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f04d80098d0 tx=0x7f04d80095c0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.296+0000 7f04e5649640 1 -- 172.21.15.116:0/1528832024 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04e0101bc0 msgr2=0x7f04e01a0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:15.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.296+0000 7f04e5649640 1 --2- 172.21.15.116:0/1528832024 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04e0101bc0 0x7f04e01a0ab0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f04d400b4a0 tx=0x7f04d400b8a0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.296+0000 7f04e5649640 1 -- 172.21.15.116:0/1528832024 shutdown_connections 2024-09-12T03:14:15.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.296+0000 7f04e5649640 1 --2- 172.21.15.116:0/1528832024 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f04c80a2b70 0x7f04c80a5030 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.296+0000 7f04e5649640 1 --2- 172.21.15.116:0/1528832024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04e0105a90 0x7f04e019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.296+0000 7f04e5649640 1 --2- 172.21.15.116:0/1528832024 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04e0101bc0 0x7f04e01a0ab0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.296+0000 7f04e5649640 1 --2- 172.21.15.116:0/1528832024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04e0101240 0x7f04e01a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:15.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.296+0000 7f04e5649640 1 -- 172.21.15.116:0/1528832024 >> 172.21.15.116:0/1528832024 conn(0x7f04e0077210 msgr2=0x7f04e0103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:15.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.296+0000 7f04e5649640 1 -- 172.21.15.116:0/1528832024 shutdown_connections 2024-09-12T03:14:15.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:15.296+0000 7f04e5649640 1 -- 172.21.15.116:0/1528832024 wait complete. 2024-09-12T03:14:15.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:15 smithi116 ceph-mon[31410]: pgmap v676: 97 pgs: 97 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:14:15.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:15 smithi183 ceph-mon[35535]: pgmap v676: 97 pgs: 97 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:14:15.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:15 smithi177 ceph-mon[35305]: pgmap v676: 97 pgs: 97 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:14:16.107 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:14:16.107 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:14:16.354 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:14:16.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.877+0000 7fd8319a4640 1 -- 172.21.15.116:0/1592422828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd82c106f70 msgr2=0x7fd82c10eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:16.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.877+0000 7fd8319a4640 1 --2- 172.21.15.116:0/1592422828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd82c106f70 0x7fd82c10eab0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fd824009ca0 tx=0x7fd82402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:14:16.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.877+0000 7fd8319a4640 1 -- 172.21.15.116:0/1592422828 shutdown_connections 2024-09-12T03:14:16.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.877+0000 7fd8319a4640 1 --2- 172.21.15.116:0/1592422828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd82c106f70 0x7fd82c10eab0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:16.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.877+0000 7fd8319a4640 1 --2- 172.21.15.116:0/1592422828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd82c100eb0 0x7fd82c106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:16.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.877+0000 7fd8319a4640 1 --2- 172.21.15.116:0/1592422828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd82c100500 0x7fd82c1008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:16.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.877+0000 7fd8319a4640 1 -- 172.21.15.116:0/1592422828 >> 172.21.15.116:0/1592422828 conn(0x7fd82c0fc5c0 msgr2=0x7fd82c0fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:16.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.877+0000 7fd8319a4640 1 -- 172.21.15.116:0/1592422828 shutdown_connections 2024-09-12T03:14:16.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.877+0000 7fd8319a4640 1 -- 172.21.15.116:0/1592422828 wait complete. 2024-09-12T03:14:16.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.878+0000 7fd8319a4640 1 Processor -- start 2024-09-12T03:14:16.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.878+0000 7fd8319a4640 1 -- start start 2024-09-12T03:14:16.901 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.878+0000 7fd8319a4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd82c100500 0x7fd82c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:16.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.878+0000 7fd8319a4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd82c100eb0 0x7fd82c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:16.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.878+0000 7fd8319a4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd82c106f70 0x7fd82c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:16.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.878+0000 7fd8319a4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd82c111f90 con 0x7fd82c106f70 2024-09-12T03:14:16.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.878+0000 7fd8319a4640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd82c111e10 con 0x7fd82c100500 2024-09-12T03:14:16.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.878+0000 7fd8319a4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd82c112110 con 0x7fd82c100eb0 2024-09-12T03:14:16.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.878+0000 7fd82b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd82c106f70 0x7fd82c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:16.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.878+0000 7fd82b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd82c106f70 0x7fd82c19eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46166/0 (socket says 172.21.15.116:46166) 2024-09-12T03:14:16.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.878+0000 7fd82b7fe640 1 -- 172.21.15.116:0/1096194493 learned_addr learned my addr 172.21.15.116:0/1096194493 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:16.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.879+0000 7fd82affd640 1 --2- 172.21.15.116:0/1096194493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd82c100500 0x7fd82c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:16.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.879+0000 7fd82b7fe640 1 -- 172.21.15.116:0/1096194493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd82c100eb0 msgr2=0x7fd82c19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:16.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.879+0000 7fd82b7fe640 1 --2- 172.21.15.116:0/1096194493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd82c100eb0 0x7fd82c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:16.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.879+0000 7fd82b7fe640 1 -- 172.21.15.116:0/1096194493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd82c100500 msgr2=0x7fd82c19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:16.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.879+0000 7fd82b7fe640 1 --2- 172.21.15.116:0/1096194493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd82c100500 0x7fd82c19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:16.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.879+0000 7fd82b7fe640 1 -- 172.21.15.116:0/1096194493 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd82c19f5c0 con 0x7fd82c106f70 2024-09-12T03:14:16.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.879+0000 7fd82b7fe640 1 --2- 172.21.15.116:0/1096194493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd82c106f70 0x7fd82c19eee0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fd824031a60 tx=0x7fd824031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:16.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.879+0000 7fd8309a2640 1 -- 172.21.15.116:0/1096194493 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd824003c30 con 0x7fd82c106f70 2024-09-12T03:14:16.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.879+0000 7fd8319a4640 1 -- 172.21.15.116:0/1096194493 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd82c19f850 con 0x7fd82c106f70 2024-09-12T03:14:16.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.879+0000 7fd8309a2640 1 -- 172.21.15.116:0/1096194493 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd824003dd0 con 0x7fd82c106f70 2024-09-12T03:14:16.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.879+0000 7fd8309a2640 1 -- 172.21.15.116:0/1096194493 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8240047e0 con 0x7fd82c106f70 2024-09-12T03:14:16.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.879+0000 7fd8319a4640 1 -- 172.21.15.116:0/1096194493 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd82c1a7190 con 0x7fd82c106f70 2024-09-12T03:14:16.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.880+0000 7fd8319a4640 1 -- 172.21.15.116:0/1096194493 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd82c1008e0 con 0x7fd82c106f70 2024-09-12T03:14:16.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.881+0000 7fd8309a2640 1 -- 172.21.15.116:0/1096194493 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd82402f790 con 0x7fd82c106f70 2024-09-12T03:14:16.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.882+0000 7fd8309a2640 1 --2- 172.21.15.116:0/1096194493 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd8140a2b80 0x7fd8140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:16.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.882+0000 7fd82affd640 1 --2- 172.21.15.116:0/1096194493 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd8140a2b80 0x7fd8140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:16.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.882+0000 7fd8309a2640 1 -- 172.21.15.116:0/1096194493 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd8240f0640 con 0x7fd82c106f70 2024-09-12T03:14:16.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.882+0000 7fd82affd640 1 --2- 172.21.15.116:0/1096194493 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd8140a2b80 0x7fd8140a5040 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd81c0058c0 tx=0x7fd81c00a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:16.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:16.885+0000 7fd8309a2640 1 -- 172.21.15.116:0/1096194493 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd8240bd5d0 con 0x7fd82c106f70 2024-09-12T03:14:17.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:17.087+0000 7fd8319a4640 1 -- 172.21.15.116:0/1096194493 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd82c1a7490 con 0x7fd82c106f70 2024-09-12T03:14:17.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:17.087+0000 7fd8309a2640 1 -- 172.21.15.116:0/1096194493 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7fd8240bced0 con 0x7fd82c106f70 2024-09-12T03:14:17.088 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:14:17.088 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:14:17.088 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:14:17.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:17.089+0000 7fd8319a4640 1 -- 172.21.15.116:0/1096194493 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd8140a2b80 msgr2=0x7fd8140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:17.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:17.089+0000 7fd8319a4640 1 --2- 172.21.15.116:0/1096194493 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd8140a2b80 0x7fd8140a5040 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd81c0058c0 tx=0x7fd81c00a4c0 comp rx=0 tx=0).stop 2024-09-12T03:14:17.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:17.089+0000 7fd8319a4640 1 -- 172.21.15.116:0/1096194493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd82c106f70 msgr2=0x7fd82c19eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:17.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:17.089+0000 7fd8319a4640 1 --2- 172.21.15.116:0/1096194493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd82c106f70 0x7fd82c19eee0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fd824031a60 tx=0x7fd824031a90 comp rx=0 tx=0).stop 2024-09-12T03:14:17.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:17.089+0000 7fd8319a4640 1 -- 172.21.15.116:0/1096194493 shutdown_connections 2024-09-12T03:14:17.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:17.089+0000 7fd8319a4640 1 --2- 172.21.15.116:0/1096194493 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd8140a2b80 0x7fd8140a5040 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:17.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:17.089+0000 7fd8319a4640 1 --2- 172.21.15.116:0/1096194493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd82c106f70 0x7fd82c19eee0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:17.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:17.089+0000 7fd8319a4640 1 --2- 172.21.15.116:0/1096194493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd82c100eb0 0x7fd82c19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:17.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:17.089+0000 7fd8319a4640 1 --2- 172.21.15.116:0/1096194493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd82c100500 0x7fd82c19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:17.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:17.089+0000 7fd8319a4640 1 -- 172.21.15.116:0/1096194493 >> 172.21.15.116:0/1096194493 conn(0x7fd82c0fc5c0 msgr2=0x7fd82c1057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:17.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:17.089+0000 7fd8319a4640 1 -- 172.21.15.116:0/1096194493 shutdown_connections 2024-09-12T03:14:17.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:17.089+0000 7fd8319a4640 1 -- 172.21.15.116:0/1096194493 wait complete. 2024-09-12T03:14:17.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:17 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1096194493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:14:17.461 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:17 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1096194493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:14:17.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:17 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1096194493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:14:17.818 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:14:18.066 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:14:18.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:18 smithi183 ceph-mon[35535]: pgmap v677: 97 pgs: 97 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:14:18.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:18 smithi177 ceph-mon[35305]: pgmap v677: 97 pgs: 97 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:14:18.500 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:18 smithi116 ceph-mon[31410]: pgmap v677: 97 pgs: 97 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:14:18.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.581+0000 7f92a62f5640 1 -- 172.21.15.116:0/2322521896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92a0100a80 msgr2=0x7f92a0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:18.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.581+0000 7f92a62f5640 1 --2- 172.21.15.116:0/2322521896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92a0100a80 0x7f92a0100ee0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f9294009930 tx=0x7f929402a420 comp rx=0 tx=0).stop 2024-09-12T03:14:18.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.582+0000 7f92a62f5640 1 -- 172.21.15.116:0/2322521896 shutdown_connections 2024-09-12T03:14:18.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.582+0000 7f92a62f5640 1 --2- 172.21.15.116:0/2322521896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92a0101420 0x7f92a010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.582+0000 7f92a62f5640 1 --2- 172.21.15.116:0/2322521896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92a0100a80 0x7f92a0100ee0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.582+0000 7f92a62f5640 1 --2- 172.21.15.116:0/2322521896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92a0106a80 0x7f92a0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.582+0000 7f92a62f5640 1 -- 172.21.15.116:0/2322521896 >> 172.21.15.116:0/2322521896 conn(0x7f92a00fc7a0 msgr2=0x7f92a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:18.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.582+0000 7f92a62f5640 1 -- 172.21.15.116:0/2322521896 shutdown_connections 2024-09-12T03:14:18.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.582+0000 7f92a62f5640 1 -- 172.21.15.116:0/2322521896 wait complete. 2024-09-12T03:14:18.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.582+0000 7f92a62f5640 1 Processor -- start 2024-09-12T03:14:18.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.582+0000 7f92a62f5640 1 -- start start 2024-09-12T03:14:18.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.583+0000 7f92a62f5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92a0100a80 0x7f92a0199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.583+0000 7f92a62f5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92a0101420 0x7f92a0199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.583+0000 7f92a62f5640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92a0106a80 0x7f92a01a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.583+0000 7f92a62f5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f92a0110ec0 con 0x7f92a0101420 2024-09-12T03:14:18.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.583+0000 7f92a62f5640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f92a0110d40 con 0x7f92a0106a80 2024-09-12T03:14:18.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.583+0000 7f92a62f5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f92a0111040 con 0x7f92a0100a80 2024-09-12T03:14:18.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.583+0000 7f929f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92a0101420 0x7f92a0199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.583+0000 7f929f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92a0101420 0x7f92a0199880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46180/0 (socket says 172.21.15.116:46180) 2024-09-12T03:14:18.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.583+0000 7f929f7fe640 1 -- 172.21.15.116:0/2663283218 learned_addr learned my addr 172.21.15.116:0/2663283218 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:18.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.583+0000 7f929ffff640 1 --2- 172.21.15.116:0/2663283218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92a0100a80 0x7f92a0199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.583+0000 7f92a4892640 1 --2- 172.21.15.116:0/2663283218 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92a0106a80 0x7f92a01a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.583+0000 7f929f7fe640 1 -- 172.21.15.116:0/2663283218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92a0100a80 msgr2=0x7f92a0199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:18.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.583+0000 7f929f7fe640 1 --2- 172.21.15.116:0/2663283218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92a0100a80 0x7f92a0199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.583+0000 7f929f7fe640 1 -- 172.21.15.116:0/2663283218 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92a0106a80 msgr2=0x7f92a01a43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:18.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.583+0000 7f929f7fe640 1 --2- 172.21.15.116:0/2663283218 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92a0106a80 0x7f92a01a43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.583+0000 7f929f7fe640 1 -- 172.21.15.116:0/2663283218 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a01a4900 con 0x7f92a0101420 2024-09-12T03:14:18.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.583+0000 7f92a4892640 1 --2- 172.21.15.116:0/2663283218 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92a0106a80 0x7f92a01a43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:14:18.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.583+0000 7f929f7fe640 1 --2- 172.21.15.116:0/2663283218 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92a0101420 0x7f92a0199880 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f9294009230 tx=0x7f9294004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:18.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.583+0000 7f929d7fa640 1 -- 172.21.15.116:0/2663283218 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929403f030 con 0x7f92a0101420 2024-09-12T03:14:18.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.583+0000 7f929d7fa640 1 -- 172.21.15.116:0/2663283218 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9294003ad0 con 0x7f92a0101420 2024-09-12T03:14:18.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.583+0000 7f92a62f5640 1 -- 172.21.15.116:0/2663283218 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a01a4ad0 con 0x7f92a0101420 2024-09-12T03:14:18.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.583+0000 7f929d7fa640 1 -- 172.21.15.116:0/2663283218 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9294035070 con 0x7f92a0101420 2024-09-12T03:14:18.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.583+0000 7f92a62f5640 1 -- 172.21.15.116:0/2663283218 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a01a4f20 con 0x7f92a0101420 2024-09-12T03:14:18.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.586+0000 7f929d7fa640 1 -- 172.21.15.116:0/2663283218 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f9294042030 con 0x7f92a0101420 2024-09-12T03:14:18.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.586+0000 7f92a86ce740 1 Processor -- start 2024-09-12T03:14:18.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.586+0000 7f92a86ce740 1 -- start start 2024-09-12T03:14:18.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.587+0000 7f92a86ce740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563e9368eb40 0x563e9368d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.587+0000 7f92a86ce740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e9368d5d0 0x563e9368da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.587+0000 7f92a86ce740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e93698370 0x563e9369a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.587+0000 7f92a86ce740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x563e935f32f0 con 0x563e9368eb40 2024-09-12T03:14:18.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.587+0000 7f92a86ce740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x563e9359acf0 con 0x563e93698370 2024-09-12T03:14:18.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.587+0000 7f92a86ce740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563e935ef2b0 con 0x563e9368d5d0 2024-09-12T03:14:18.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.587+0000 7f92a4892640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563e9368eb40 0x563e9368d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.587+0000 7f92a4892640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563e9368eb40 0x563e9368d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46192/0 (socket says 172.21.15.116:46192) 2024-09-12T03:14:18.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.587+0000 7f92a4892640 1 -- 172.21.15.116:0/52217465 learned_addr learned my addr 172.21.15.116:0/52217465 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:18.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.587+0000 7f929ffff640 1 --2- 172.21.15.116:0/52217465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e9368d5d0 0x563e9368da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.587+0000 7f929f7fe640 1 --2- 172.21.15.116:0/52217465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e93698370 0x563e9369a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.587+0000 7f92a4892640 1 -- 172.21.15.116:0/52217465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e9368d5d0 msgr2=0x563e9368da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:18.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.587+0000 7f92a4892640 1 --2- 172.21.15.116:0/52217465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e9368d5d0 0x563e9368da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.587+0000 7f92a4892640 1 -- 172.21.15.116:0/52217465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e93698370 msgr2=0x563e9369a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:18.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.587+0000 7f92a4892640 1 --2- 172.21.15.116:0/52217465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e93698370 0x563e9369a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.587+0000 7f92a4892640 1 -- 172.21.15.116:0/52217465 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e9357f3a0 con 0x563e9368eb40 2024-09-12T03:14:18.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.587+0000 7f929d7fa640 1 --2- 172.21.15.116:0/2663283218 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f92880a2b80 0x7f92880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.587+0000 7f929ffff640 1 --2- 172.21.15.116:0/52217465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e9368d5d0 0x563e9368da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:18.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.587+0000 7f929ffff640 1 --2- 172.21.15.116:0/2663283218 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f92880a2b80 0x7f92880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.587+0000 7f929d7fa640 1 -- 172.21.15.116:0/2663283218 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f92940ef500 con 0x7f92a0101420 2024-09-12T03:14:18.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.587+0000 7f92a4892640 1 --2- 172.21.15.116:0/52217465 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563e9368eb40 0x563e9368d090 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f9298007f30 tx=0x7f929800c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa16316fa60cbf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:18.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.587+0000 7f92867fc640 1 -- 172.21.15.116:0/52217465 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9298014070 con 0x563e9368eb40 2024-09-12T03:14:18.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.587+0000 7f92867fc640 1 -- 172.21.15.116:0/52217465 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f929800fd30 con 0x563e9368eb40 2024-09-12T03:14:18.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.588+0000 7f92867fc640 1 -- 172.21.15.116:0/52217465 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929800ed70 con 0x563e9368eb40 2024-09-12T03:14:18.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.588+0000 7f92a86ce740 1 -- 172.21.15.116:0/52217465 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563e9368eb40 msgr2=0x563e9368d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:18.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.588+0000 7f929ffff640 1 --2- 172.21.15.116:0/2663283218 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f92880a2b80 0x7f92880a5040 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x563e9368aac0 tx=0x7f929000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:18.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.588+0000 7f92a86ce740 1 --2- 172.21.15.116:0/52217465 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563e9368eb40 0x563e9368d090 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f9298007f30 tx=0x7f929800c750 comp rx=0 tx=0).stop 2024-09-12T03:14:18.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.588+0000 7f92a86ce740 1 -- 172.21.15.116:0/52217465 shutdown_connections 2024-09-12T03:14:18.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.588+0000 7f92a86ce740 1 --2- 172.21.15.116:0/52217465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e93698370 0x563e9369a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.588+0000 7f92a86ce740 1 --2- 172.21.15.116:0/52217465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e9368d5d0 0x563e9368da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.588+0000 7f92a86ce740 1 --2- 172.21.15.116:0/52217465 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563e9368eb40 0x563e9368d090 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.588+0000 7f92a86ce740 1 -- 172.21.15.116:0/52217465 >> 172.21.15.116:0/52217465 conn(0x563e93656310 msgr2=0x563e93689c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:18.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.588+0000 7f92a86ce740 1 -- 172.21.15.116:0/52217465 shutdown_connections 2024-09-12T03:14:18.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.588+0000 7f92a86ce740 1 -- 172.21.15.116:0/52217465 wait complete. 2024-09-12T03:14:18.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.589+0000 7f92a86ce740 1 -- start start 2024-09-12T03:14:18.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.589+0000 7f92a86ce740 1 Processor -- start 2024-09-12T03:14:18.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.589+0000 7f92a86ce740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563e9368d5d0 0x563e93697d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.589+0000 7f92a86ce740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e9368eb40 0x563e93694fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.589+0000 7f92a86ce740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e93698370 0x563e93695530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.589+0000 7f92a86ce740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x563e9369b070 con 0x563e9368d5d0 2024-09-12T03:14:18.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.589+0000 7f92a86ce740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x563e935ae060 con 0x563e9368eb40 2024-09-12T03:14:18.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.589+0000 7f92a86ce740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563e9369b1f0 con 0x563e93698370 2024-09-12T03:14:18.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.589+0000 7f92a86ce740 10 client.0 _ll_register_callbacks cb 0x563e933018c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:18.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.589+0000 7f929ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563e9368d5d0 0x563e93697d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.589+0000 7f929ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563e9368d5d0 0x563e93697d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46196/0 (socket says 172.21.15.116:46196) 2024-09-12T03:14:18.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.589+0000 7f929ffff640 1 -- 172.21.15.116:0/4196875026 learned_addr learned my addr 172.21.15.116:0/4196875026 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:18.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.589+0000 7f929ffff640 1 -- 172.21.15.116:0/4196875026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e93698370 msgr2=0x563e93695530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:18.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.589+0000 7f929ffff640 1 --2- 172.21.15.116:0/4196875026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e93698370 0x563e93695530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.589+0000 7f929ffff640 1 -- 172.21.15.116:0/4196875026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e9368eb40 msgr2=0x563e93694fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:18.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.589+0000 7f929ffff640 1 --2- 172.21.15.116:0/4196875026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e9368eb40 0x563e93694fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.589+0000 7f929ffff640 1 -- 172.21.15.116:0/4196875026 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e933643c0 con 0x563e9368d5d0 2024-09-12T03:14:18.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.590+0000 7f929ffff640 1 --2- 172.21.15.116:0/4196875026 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563e9368d5d0 0x563e93697d00 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f92900045d0 tx=0x7f92900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a10f28435014ac9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:18.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.590+0000 7f927f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:14:18.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.590+0000 7f927f7fe640 1 -- 172.21.15.116:0/4196875026 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929001b870 con 0x563e9368d5d0 2024-09-12T03:14:18.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.590+0000 7f927f7fe640 1 -- 172.21.15.116:0/4196875026 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f929001ba10 con 0x563e9368d5d0 2024-09-12T03:14:18.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.590+0000 7f92a86ce740 1 -- 172.21.15.116:0/4196875026 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f929800fd30 con 0x563e9368d5d0 2024-09-12T03:14:18.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.590+0000 7f927f7fe640 1 -- 172.21.15.116:0/4196875026 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f929001bc10 con 0x563e9368d5d0 2024-09-12T03:14:18.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.590+0000 7f92a86ce740 10 client.18528 fetch_fsmap learned FSMap version 25 2024-09-12T03:14:18.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.590+0000 7f92a86ce740 1 -- 172.21.15.116:0/4196875026 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563e936a6b50 con 0x563e9368d5d0 2024-09-12T03:14:18.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.590+0000 7f927f7fe640 1 -- 172.21.15.116:0/4196875026 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f929800fd30 con 0x563e9368d5d0 2024-09-12T03:14:18.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.590+0000 7f927f7fe640 1 -- 172.21.15.116:0/4196875026 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9290024070 con 0x563e9368d5d0 2024-09-12T03:14:18.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.590+0000 7f92a86ce740 10 client.18528 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:14:18.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.590+0000 7f92a86ce740 10 client.18528 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:14:18.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.590+0000 7f92a86ce740 20 client.18528 populate_metadata read hostname 'smithi116' 2024-09-12T03:14:18.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.590+0000 7f92a86ce740 1 --2- 172.21.15.116:0/4196875026 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x563e936a7fb0 0x563e936aa450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.590+0000 7f92a86ce740 4 client.18528 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:14:18.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.590+0000 7f92a86ce740 1 -- 172.21.15.116:0/4196875026 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e93600860 con 0x563e936a7fb0 2024-09-12T03:14:18.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.591+0000 7f929f7fe640 1 --2- 172.21.15.116:0/4196875026 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x563e936a7fb0 0x563e936aa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.591+0000 7f927f7fe640 10 client.18528 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:14:18.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.591+0000 7f929f7fe640 1 --2- 172.21.15.116:0/4196875026 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x563e936a7fb0 0x563e936aa450 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e8d2602e740b59e5 server_cookie=c2482f69992408c5 in_seq=0 out_seq=0 2024-09-12T03:14:18.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.592+0000 7f927f7fe640 1 -- 172.21.15.116:0/4196875026 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e93600860 con 0x563e936a7fb0 2024-09-12T03:14:18.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.592+0000 7f927f7fe640 10 client.18528 handle_command_reply: tid=0 2024-09-12T03:14:18.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.592+0000 7f92a86ce740 1 client.18528 shutdown 2024-09-12T03:14:18.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.592+0000 7f92a86ce740 1 -- 172.21.15.116:0/4196875026 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563e9368d5d0 msgr2=0x563e93697d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:18.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.592+0000 7f92a86ce740 1 --2- 172.21.15.116:0/4196875026 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563e9368d5d0 0x563e93697d00 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f92900045d0 tx=0x7f92900046c0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.592+0000 7f92a86ce740 1 -- 172.21.15.116:0/4196875026 shutdown_connections 2024-09-12T03:14:18.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.592+0000 7f92a86ce740 1 --2- 172.21.15.116:0/4196875026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e93698370 0x563e93695530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.592+0000 7f92a86ce740 1 --2- 172.21.15.116:0/4196875026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e9368eb40 0x563e93694fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.592+0000 7f92a86ce740 1 --2- 172.21.15.116:0/4196875026 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x563e936a7fb0 0x563e936aa450 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.592+0000 7f92a86ce740 1 --2- 172.21.15.116:0/4196875026 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563e9368d5d0 0x563e93697d00 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.592+0000 7f92a86ce740 1 -- 172.21.15.116:0/4196875026 >> 172.21.15.116:0/4196875026 conn(0x563e93656310 msgr2=0x563e9368f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:18.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.592+0000 7f927f7fe640 0 client.18528 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:14:18.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.592+0000 7f92a86ce740 1 -- 172.21.15.116:0/4196875026 shutdown_connections 2024-09-12T03:14:18.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.592+0000 7f92a86ce740 1 -- 172.21.15.116:0/4196875026 wait complete. 2024-09-12T03:14:18.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.592+0000 7f92a86ce740 20 client.18528 trim_cache size 0 max 16384 2024-09-12T03:14:18.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.606+0000 7f92a86ce740 1 Processor -- start 2024-09-12T03:14:18.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.606+0000 7f92a86ce740 1 -- start start 2024-09-12T03:14:18.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.606+0000 7f92a86ce740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563e9368d5d0 0x563e936a4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.606+0000 7f92a86ce740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e9368eb40 0x563e93688bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.606+0000 7f92a86ce740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e93698370 0x563e93689130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.606+0000 7f92a86ce740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01afd20 con 0x563e9368d5d0 2024-09-12T03:14:18.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.606+0000 7f92a86ce740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01ad5f0 con 0x563e9368eb40 2024-09-12T03:14:18.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.606+0000 7f92a86ce740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01ab440 con 0x563e93698370 2024-09-12T03:14:18.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.606+0000 7f929ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563e9368d5d0 0x563e936a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.606+0000 7f929ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563e9368d5d0 0x563e936a4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46204/0 (socket says 172.21.15.116:46204) 2024-09-12T03:14:18.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.606+0000 7f929ffff640 1 -- 172.21.15.116:0/452253975 learned_addr learned my addr 172.21.15.116:0/452253975 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:18.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.606+0000 7f92a4892640 1 --2- 172.21.15.116:0/452253975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e93698370 0x563e93689130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.606+0000 7f929f7fe640 1 --2- 172.21.15.116:0/452253975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e9368eb40 0x563e93688bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.606+0000 7f929ffff640 1 -- 172.21.15.116:0/452253975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e93698370 msgr2=0x563e93689130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:18.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.606+0000 7f929ffff640 1 --2- 172.21.15.116:0/452253975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e93698370 0x563e93689130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.606+0000 7f929ffff640 1 -- 172.21.15.116:0/452253975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e9368eb40 msgr2=0x563e93688bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:18.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.606+0000 7f929ffff640 1 --2- 172.21.15.116:0/452253975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e9368eb40 0x563e93688bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.606+0000 7f929ffff640 1 -- 172.21.15.116:0/452253975 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e93690120 con 0x563e9368d5d0 2024-09-12T03:14:18.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.606+0000 7f92a4892640 1 --2- 172.21.15.116:0/452253975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e93698370 0x563e93689130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:18.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.606+0000 7f929ffff640 1 --2- 172.21.15.116:0/452253975 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563e9368d5d0 0x563e936a4310 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f929000b740 tx=0x7f9290012d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a389d806d2baa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:18.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.607+0000 7f9285ffb640 1 -- 172.21.15.116:0/452253975 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929001b670 con 0x563e9368d5d0 2024-09-12T03:14:18.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.607+0000 7f9285ffb640 1 -- 172.21.15.116:0/452253975 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9290035070 con 0x563e9368d5d0 2024-09-12T03:14:18.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.607+0000 7f9285ffb640 1 -- 172.21.15.116:0/452253975 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290004bc0 con 0x563e9368d5d0 2024-09-12T03:14:18.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.607+0000 7f92a86ce740 1 -- 172.21.15.116:0/452253975 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563e9368d5d0 msgr2=0x563e936a4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:18.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.607+0000 7f92a86ce740 1 --2- 172.21.15.116:0/452253975 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563e9368d5d0 0x563e936a4310 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f929000b740 tx=0x7f9290012d30 comp rx=0 tx=0).stop 2024-09-12T03:14:18.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.607+0000 7f92a86ce740 1 -- 172.21.15.116:0/452253975 shutdown_connections 2024-09-12T03:14:18.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.607+0000 7f92a86ce740 1 --2- 172.21.15.116:0/452253975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e93698370 0x563e93689130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.607+0000 7f92a86ce740 1 --2- 172.21.15.116:0/452253975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e9368eb40 0x563e93688bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.607+0000 7f92a86ce740 1 --2- 172.21.15.116:0/452253975 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563e9368d5d0 0x563e936a4310 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.607+0000 7f92a86ce740 1 -- 172.21.15.116:0/452253975 >> 172.21.15.116:0/452253975 conn(0x563e93656310 msgr2=0x563e936884e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:18.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.607+0000 7f92a86ce740 1 -- 172.21.15.116:0/452253975 shutdown_connections 2024-09-12T03:14:18.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.607+0000 7f92a86ce740 1 -- 172.21.15.116:0/452253975 wait complete. 2024-09-12T03:14:18.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.608+0000 7f92a86ce740 1 -- start start 2024-09-12T03:14:18.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.608+0000 7f92a86ce740 1 Processor -- start 2024-09-12T03:14:18.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.608+0000 7f92a86ce740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563e9368d5d0 0x563e9370c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.608+0000 7f92a86ce740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e9368eb40 0x563e9370c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.608+0000 7f92a86ce740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e93698370 0x563e936a0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.608+0000 7f92a86ce740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01abab0 con 0x563e9368d5d0 2024-09-12T03:14:18.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.608+0000 7f92a86ce740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01acf20 con 0x563e9368eb40 2024-09-12T03:14:18.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.608+0000 7f92a86ce740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01ac180 con 0x563e93698370 2024-09-12T03:14:18.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.608+0000 7f92a86ce740 10 client.0 _ll_register_callbacks cb 0x7f9290004500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:18.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.608+0000 7f929ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563e9368d5d0 0x563e9370c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.608+0000 7f929ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563e9368d5d0 0x563e9370c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46212/0 (socket says 172.21.15.116:46212) 2024-09-12T03:14:18.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.608+0000 7f929ffff640 1 -- 172.21.15.116:0/4161407314 learned_addr learned my addr 172.21.15.116:0/4161407314 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:18.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.608+0000 7f929f7fe640 1 --2- 172.21.15.116:0/4161407314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e9368eb40 0x563e9370c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.609+0000 7f929ffff640 1 -- 172.21.15.116:0/4161407314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e93698370 msgr2=0x563e936a0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:18.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.609+0000 7f929ffff640 1 --2- 172.21.15.116:0/4161407314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e93698370 0x563e936a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.609+0000 7f929ffff640 1 -- 172.21.15.116:0/4161407314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e9368eb40 msgr2=0x563e9370c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:18.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.609+0000 7f929ffff640 1 --2- 172.21.15.116:0/4161407314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e9368eb40 0x563e9370c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.609+0000 7f929ffff640 1 -- 172.21.15.116:0/4161407314 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e936a0a40 con 0x563e9368d5d0 2024-09-12T03:14:18.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.609+0000 7f929f7fe640 1 --2- 172.21.15.116:0/4161407314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e9368eb40 0x563e9370c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:18.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.609+0000 7f929ffff640 1 --2- 172.21.15.116:0/4161407314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563e9368d5d0 0x563e9370c3a0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f9290017040 tx=0x7f9290005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356634933c736736 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:18.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.609+0000 7f927f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:14:18.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.609+0000 7f927f7fe640 1 -- 172.21.15.116:0/4161407314 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92900319f0 con 0x563e9368d5d0 2024-09-12T03:14:18.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.609+0000 7f927f7fe640 1 -- 172.21.15.116:0/4161407314 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f9290031b90 con 0x563e9368d5d0 2024-09-12T03:14:18.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.609+0000 7f92a86ce740 1 -- 172.21.15.116:0/4161407314 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9290035070 con 0x563e9368d5d0 2024-09-12T03:14:18.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.609+0000 7f927f7fe640 1 -- 172.21.15.116:0/4161407314 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9290048070 con 0x563e9368d5d0 2024-09-12T03:14:18.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.609+0000 7f92a86ce740 10 client.18534 fetch_fsmap learned FSMap version 25 2024-09-12T03:14:18.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.609+0000 7f92a86ce740 1 -- 172.21.15.116:0/4161407314 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563e93717b20 con 0x563e9368d5d0 2024-09-12T03:14:18.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.609+0000 7f927f7fe640 1 -- 172.21.15.116:0/4161407314 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9290035070 con 0x563e9368d5d0 2024-09-12T03:14:18.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.610+0000 7f927f7fe640 1 -- 172.21.15.116:0/4161407314 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9290024040 con 0x563e9368d5d0 2024-09-12T03:14:18.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.610+0000 7f92a86ce740 10 client.18534 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:14:18.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.610+0000 7f92a86ce740 10 client.18534 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:14:18.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.610+0000 7f92a86ce740 20 client.18534 populate_metadata read hostname 'smithi116' 2024-09-12T03:14:18.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.610+0000 7f92a86ce740 1 --2- 172.21.15.116:0/4161407314 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x563e93718120 0x563e9371a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.610+0000 7f92a86ce740 4 client.18534 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:14:18.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.610+0000 7f92a86ce740 1 -- 172.21.15.116:0/4161407314 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e935fad70 con 0x563e93718120 2024-09-12T03:14:18.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.610+0000 7f929f7fe640 1 --2- 172.21.15.116:0/4161407314 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x563e93718120 0x563e9371a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.610+0000 7f927f7fe640 10 client.18534 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:14:18.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.610+0000 7f929f7fe640 1 --2- 172.21.15.116:0/4161407314 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x563e93718120 0x563e9371a560 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a80e5b61ecec49e7 server_cookie=d9d8dfc5f336712e in_seq=0 out_seq=0 2024-09-12T03:14:18.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.611+0000 7f927f7fe640 1 -- 172.21.15.116:0/4161407314 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e935fad70 con 0x563e93718120 2024-09-12T03:14:18.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.611+0000 7f927f7fe640 10 client.18534 handle_command_reply: tid=0 2024-09-12T03:14:18.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.611+0000 7f92a86ce740 1 client.18534 shutdown 2024-09-12T03:14:18.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.611+0000 7f92a86ce740 1 -- 172.21.15.116:0/4161407314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563e9368d5d0 msgr2=0x563e9370c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:18.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.611+0000 7f92a86ce740 1 --2- 172.21.15.116:0/4161407314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563e9368d5d0 0x563e9370c3a0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f9290017040 tx=0x7f9290005bb0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.611+0000 7f92a86ce740 1 -- 172.21.15.116:0/4161407314 shutdown_connections 2024-09-12T03:14:18.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.611+0000 7f92a86ce740 1 --2- 172.21.15.116:0/4161407314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e93698370 0x563e936a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.611+0000 7f92a86ce740 1 --2- 172.21.15.116:0/4161407314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e9368eb40 0x563e9370c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.611+0000 7f92a86ce740 1 --2- 172.21.15.116:0/4161407314 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x563e93718120 0x563e9371a560 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.611+0000 7f92a86ce740 1 --2- 172.21.15.116:0/4161407314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563e9368d5d0 0x563e9370c3a0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.611+0000 7f92a86ce740 1 -- 172.21.15.116:0/4161407314 >> 172.21.15.116:0/4161407314 conn(0x563e93656310 msgr2=0x563e9368f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:18.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.611+0000 7f927f7fe640 0 client.18534 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:14:18.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.611+0000 7f92a86ce740 1 -- 172.21.15.116:0/4161407314 shutdown_connections 2024-09-12T03:14:18.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.611+0000 7f92a86ce740 1 -- 172.21.15.116:0/4161407314 wait complete. 2024-09-12T03:14:18.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.611+0000 7f92a86ce740 20 client.18534 trim_cache size 0 max 16384 2024-09-12T03:14:18.623 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:14:18.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.612+0000 7f92a62f5640 1 -- 172.21.15.116:0/2663283218 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f92880a2b80 msgr2=0x7f92880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:18.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.612+0000 7f92a62f5640 1 --2- 172.21.15.116:0/2663283218 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f92880a2b80 0x7f92880a5040 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x563e9368aac0 tx=0x7f929000b390 comp rx=0 tx=0).stop 2024-09-12T03:14:18.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.612+0000 7f92a62f5640 1 -- 172.21.15.116:0/2663283218 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92a0101420 msgr2=0x7f92a0199880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:18.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.612+0000 7f92a62f5640 1 --2- 172.21.15.116:0/2663283218 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92a0101420 0x7f92a0199880 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f9294009230 tx=0x7f9294004270 comp rx=0 tx=0).stop 2024-09-12T03:14:18.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.612+0000 7f92a62f5640 1 -- 172.21.15.116:0/2663283218 shutdown_connections 2024-09-12T03:14:18.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.612+0000 7f92a62f5640 1 --2- 172.21.15.116:0/2663283218 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f92880a2b80 0x7f92880a5040 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.612+0000 7f92a62f5640 1 --2- 172.21.15.116:0/2663283218 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92a0106a80 0x7f92a01a43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.612+0000 7f92a62f5640 1 --2- 172.21.15.116:0/2663283218 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92a0101420 0x7f92a0199880 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.612+0000 7f92a62f5640 1 --2- 172.21.15.116:0/2663283218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92a0100a80 0x7f92a0199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.612+0000 7f92a62f5640 1 -- 172.21.15.116:0/2663283218 >> 172.21.15.116:0/2663283218 conn(0x7f92a00fc7a0 msgr2=0x7f92a00fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:18.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.612+0000 7f92a62f5640 1 -- 172.21.15.116:0/2663283218 shutdown_connections 2024-09-12T03:14:18.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:18.612+0000 7f92a62f5640 1 -- 172.21.15.116:0/2663283218 wait complete. 2024-09-12T03:14:19.317 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:14:19.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:19 smithi116 ceph-mon[31410]: pgmap v678: 97 pgs: 97 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:14:19.566 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:14:19.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:19 smithi183 ceph-mon[35535]: pgmap v678: 97 pgs: 97 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:14:19.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:19 smithi177 ceph-mon[35305]: pgmap v678: 97 pgs: 97 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:14:20.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.038+0000 7f5ce7dbc640 1 -- 172.21.15.116:0/3928898696 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ce01082a0 msgr2=0x7f5ce010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:20.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.038+0000 7f5ce7dbc640 1 --2- 172.21.15.116:0/3928898696 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ce01082a0 0x7f5ce010ec40 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0009cd0 tx=0x7f5cd002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.038+0000 7f5ce7dbc640 1 -- 172.21.15.116:0/3928898696 shutdown_connections 2024-09-12T03:14:20.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.038+0000 7f5ce7dbc640 1 --2- 172.21.15.116:0/3928898696 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ce01082a0 0x7f5ce010ec40 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.038+0000 7f5ce7dbc640 1 --2- 172.21.15.116:0/3928898696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ce01036e0 0x7f5ce0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.038+0000 7f5ce7dbc640 1 --2- 172.21.15.116:0/3928898696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ce0102d30 0x7f5ce0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.038+0000 7f5ce7dbc640 1 -- 172.21.15.116:0/3928898696 >> 172.21.15.116:0/3928898696 conn(0x7f5ce00fc7d0 msgr2=0x7f5ce00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:20.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.038+0000 7f5ce7dbc640 1 -- 172.21.15.116:0/3928898696 shutdown_connections 2024-09-12T03:14:20.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.038+0000 7f5ce7dbc640 1 -- 172.21.15.116:0/3928898696 wait complete. 2024-09-12T03:14:20.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.038+0000 7f5ce7dbc640 1 Processor -- start 2024-09-12T03:14:20.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.038+0000 7f5ce7dbc640 1 -- start start 2024-09-12T03:14:20.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.039+0000 7f5ce7dbc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ce0102d30 0x7f5ce0198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:20.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.039+0000 7f5ce7dbc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ce01036e0 0x7f5ce0198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:20.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.039+0000 7f5ce7dbc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ce01082a0 0x7f5ce019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:20.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.039+0000 7f5ce7dbc640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce01120a0 con 0x7f5ce01036e0 2024-09-12T03:14:20.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.039+0000 7f5ce7dbc640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce0111f20 con 0x7f5ce01082a0 2024-09-12T03:14:20.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.039+0000 7f5ce7dbc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce0112220 con 0x7f5ce0102d30 2024-09-12T03:14:20.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.039+0000 7f5ce5357640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ce01036e0 0x7f5ce0198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:20.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.039+0000 7f5ce5357640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ce01036e0 0x7f5ce0198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46230/0 (socket says 172.21.15.116:46230) 2024-09-12T03:14:20.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.039+0000 7f5ce5357640 1 -- 172.21.15.116:0/2137738458 learned_addr learned my addr 172.21.15.116:0/2137738458 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:20.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.039+0000 7f5ce6359640 1 --2- 172.21.15.116:0/2137738458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ce01082a0 0x7f5ce019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:20.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.039+0000 7f5ce5357640 1 -- 172.21.15.116:0/2137738458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ce0102d30 msgr2=0x7f5ce0198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:20.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.039+0000 7f5ce5b58640 1 --2- 172.21.15.116:0/2137738458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ce0102d30 0x7f5ce0198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:20.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.039+0000 7f5ce5357640 1 --2- 172.21.15.116:0/2137738458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ce0102d30 0x7f5ce0198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.039+0000 7f5ce5357640 1 -- 172.21.15.116:0/2137738458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ce01082a0 msgr2=0x7f5ce019ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:20.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.039+0000 7f5ce5357640 1 --2- 172.21.15.116:0/2137738458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ce01082a0 0x7f5ce019ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.039+0000 7f5ce5357640 1 -- 172.21.15.116:0/2137738458 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce019d4e0 con 0x7f5ce01036e0 2024-09-12T03:14:20.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.039+0000 7f5ce6359640 1 --2- 172.21.15.116:0/2137738458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ce01082a0 0x7f5ce019ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:14:20.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.039+0000 7f5ce5b58640 1 --2- 172.21.15.116:0/2137738458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ce0102d30 0x7f5ce0198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:14:20.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.039+0000 7f5ce5357640 1 --2- 172.21.15.116:0/2137738458 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ce01036e0 0x7f5ce0198a70 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f5cd400c970 tx=0x7f5cd400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:20.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.039+0000 7f5cdeffd640 1 -- 172.21.15.116:0/2137738458 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd4007bb0 con 0x7f5ce01036e0 2024-09-12T03:14:20.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.039+0000 7f5ce7dbc640 1 -- 172.21.15.116:0/2137738458 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce019d7d0 con 0x7f5ce01036e0 2024-09-12T03:14:20.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.039+0000 7f5cdeffd640 1 -- 172.21.15.116:0/2137738458 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cd4007d50 con 0x7f5ce01036e0 2024-09-12T03:14:20.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.039+0000 7f5cdeffd640 1 -- 172.21.15.116:0/2137738458 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd400f450 con 0x7f5ce01036e0 2024-09-12T03:14:20.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.039+0000 7f5ce7dbc640 1 -- 172.21.15.116:0/2137738458 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce01a50b0 con 0x7f5ce01036e0 2024-09-12T03:14:20.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.042+0000 7f5cea195740 1 Processor -- start 2024-09-12T03:14:20.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.042+0000 7f5cea195740 1 -- start start 2024-09-12T03:14:20.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.043+0000 7f5cdeffd640 1 -- 172.21.15.116:0/2137738458 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f5cd4004f60 con 0x7f5ce01036e0 2024-09-12T03:14:20.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.043+0000 7f5cea195740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5650ba0ccb40 0x5650ba0cb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:20.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.043+0000 7f5cea195740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650ba0cb5d0 0x5650ba0cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:20.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.043+0000 7f5cea195740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5650ba0d6370 0x5650ba0d8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:20.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.043+0000 7f5cea195740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5650ba0312f0 con 0x5650ba0ccb40 2024-09-12T03:14:20.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.043+0000 7f5cea195740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5650b9fd8cf0 con 0x5650ba0d6370 2024-09-12T03:14:20.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.043+0000 7f5cea195740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5650ba02d2b0 con 0x5650ba0cb5d0 2024-09-12T03:14:20.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.043+0000 7f5ce6359640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5650ba0ccb40 0x5650ba0cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:20.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.043+0000 7f5ce6359640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5650ba0ccb40 0x5650ba0cb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46244/0 (socket says 172.21.15.116:46244) 2024-09-12T03:14:20.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.043+0000 7f5ce6359640 1 -- 172.21.15.116:0/3359253506 learned_addr learned my addr 172.21.15.116:0/3359253506 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:20.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.043+0000 7f5ce5357640 1 --2- 172.21.15.116:0/3359253506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5650ba0d6370 0x5650ba0d8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:20.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.043+0000 7f5ce5b58640 1 --2- 172.21.15.116:0/3359253506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650ba0cb5d0 0x5650ba0cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:20.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.043+0000 7f5ce6359640 1 -- 172.21.15.116:0/3359253506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650ba0cb5d0 msgr2=0x5650ba0cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:20.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.043+0000 7f5ce6359640 1 --2- 172.21.15.116:0/3359253506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650ba0cb5d0 0x5650ba0cba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.043+0000 7f5ce6359640 1 -- 172.21.15.116:0/3359253506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5650ba0d6370 msgr2=0x5650ba0d8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:20.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.043+0000 7f5ce6359640 1 --2- 172.21.15.116:0/3359253506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5650ba0d6370 0x5650ba0d8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.043+0000 7f5ce6359640 1 -- 172.21.15.116:0/3359253506 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650b9fbd3a0 con 0x5650ba0ccb40 2024-09-12T03:14:20.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.043+0000 7f5ce6359640 1 --2- 172.21.15.116:0/3359253506 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5650ba0ccb40 0x5650ba0cb090 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0005e30 tx=0x7f5cd0009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b46a0916e9502be6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:20.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.043+0000 7f5cdeffd640 1 --2- 172.21.15.116:0/2137738458 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5cc80a2b80 0x7f5cc80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:20.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.044+0000 7f5ce5b58640 1 --2- 172.21.15.116:0/2137738458 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5cc80a2b80 0x7f5cc80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:20.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.044+0000 7f5cc7fff640 1 -- 172.21.15.116:0/3359253506 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd00318c0 con 0x5650ba0ccb40 2024-09-12T03:14:20.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.044+0000 7f5cc7fff640 1 -- 172.21.15.116:0/3359253506 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cd0036040 con 0x5650ba0ccb40 2024-09-12T03:14:20.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.044+0000 7f5cdeffd640 1 -- 172.21.15.116:0/2137738458 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f5cd40cedc0 con 0x7f5ce01036e0 2024-09-12T03:14:20.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.044+0000 7f5cc7fff640 1 -- 172.21.15.116:0/3359253506 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0004640 con 0x5650ba0ccb40 2024-09-12T03:14:20.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.044+0000 7f5cea195740 1 -- 172.21.15.116:0/3359253506 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5650ba0ccb40 msgr2=0x5650ba0cb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:20.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.044+0000 7f5ce5b58640 1 --2- 172.21.15.116:0/2137738458 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5cc80a2b80 0x7f5cc80a5040 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x5650ba0c8ac0 tx=0x7f5cd8005e90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:20.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.044+0000 7f5cea195740 1 --2- 172.21.15.116:0/3359253506 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5650ba0ccb40 0x5650ba0cb090 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0005e30 tx=0x7f5cd0009a20 comp rx=0 tx=0).stop 2024-09-12T03:14:20.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.044+0000 7f5cea195740 1 -- 172.21.15.116:0/3359253506 shutdown_connections 2024-09-12T03:14:20.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.044+0000 7f5cea195740 1 --2- 172.21.15.116:0/3359253506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5650ba0d6370 0x5650ba0d8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.044+0000 7f5cea195740 1 --2- 172.21.15.116:0/3359253506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650ba0cb5d0 0x5650ba0cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.044+0000 7f5cea195740 1 --2- 172.21.15.116:0/3359253506 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5650ba0ccb40 0x5650ba0cb090 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.044+0000 7f5cea195740 1 -- 172.21.15.116:0/3359253506 >> 172.21.15.116:0/3359253506 conn(0x5650ba094310 msgr2=0x5650ba0c7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:20.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.044+0000 7f5cea195740 1 -- 172.21.15.116:0/3359253506 shutdown_connections 2024-09-12T03:14:20.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.044+0000 7f5cea195740 1 -- 172.21.15.116:0/3359253506 wait complete. 2024-09-12T03:14:20.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.045+0000 7f5cea195740 1 -- start start 2024-09-12T03:14:20.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.045+0000 7f5cea195740 1 Processor -- start 2024-09-12T03:14:20.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.045+0000 7f5cea195740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5650ba0cb5d0 0x5650ba0d5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:20.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.045+0000 7f5cea195740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650ba0ccb40 0x5650ba0d3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:20.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.045+0000 7f5cea195740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5650ba0d6370 0x5650ba0d3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:20.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.045+0000 7f5cea195740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5650ba0d9010 con 0x5650ba0d6370 2024-09-12T03:14:20.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.045+0000 7f5cea195740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5650b9fec060 con 0x5650ba0cb5d0 2024-09-12T03:14:20.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.045+0000 7f5cea195740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5650ba0d9190 con 0x5650ba0ccb40 2024-09-12T03:14:20.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.045+0000 7f5cea195740 10 client.0 _ll_register_callbacks cb 0x5650b9d3f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:20.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.045+0000 7f5ce6359640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5650ba0d6370 0x5650ba0d3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:20.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.045+0000 7f5ce6359640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5650ba0d6370 0x5650ba0d3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46256/0 (socket says 172.21.15.116:46256) 2024-09-12T03:14:20.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.045+0000 7f5ce6359640 1 -- 172.21.15.116:0/3510036308 learned_addr learned my addr 172.21.15.116:0/3510036308 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:20.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.045+0000 7f5ce5357640 1 --2- 172.21.15.116:0/3510036308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650ba0ccb40 0x5650ba0d3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:20.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.046+0000 7f5ce5b58640 1 --2- 172.21.15.116:0/3510036308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5650ba0cb5d0 0x5650ba0d5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:20.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.046+0000 7f5ce6359640 1 -- 172.21.15.116:0/3510036308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650ba0ccb40 msgr2=0x5650ba0d3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:20.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.046+0000 7f5ce6359640 1 --2- 172.21.15.116:0/3510036308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650ba0ccb40 0x5650ba0d3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.046+0000 7f5ce6359640 1 -- 172.21.15.116:0/3510036308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5650ba0cb5d0 msgr2=0x5650ba0d5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:20.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.046+0000 7f5ce6359640 1 --2- 172.21.15.116:0/3510036308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5650ba0cb5d0 0x5650ba0d5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.046+0000 7f5ce6359640 1 -- 172.21.15.116:0/3510036308 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650b9da23c0 con 0x5650ba0d6370 2024-09-12T03:14:20.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.046+0000 7f5ce6359640 1 --2- 172.21.15.116:0/3510036308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5650ba0d6370 0x5650ba0d3650 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0009800 tx=0x7f5cd00041c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad280dad8c967b75 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:20.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.046+0000 7f5cc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:14:20.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.046+0000 7f5cc57fa640 1 -- 172.21.15.116:0/3510036308 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0003970 con 0x5650ba0d6370 2024-09-12T03:14:20.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.046+0000 7f5cea195740 1 -- 172.21.15.116:0/3510036308 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cd0036040 con 0x5650ba0d6370 2024-09-12T03:14:20.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.046+0000 7f5cc57fa640 1 -- 172.21.15.116:0/3510036308 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f5cd0003b10 con 0x5650ba0d6370 2024-09-12T03:14:20.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.046+0000 7f5cc57fa640 1 -- 172.21.15.116:0/3510036308 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5cd0003d10 con 0x5650ba0d6370 2024-09-12T03:14:20.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.046+0000 7f5cea195740 10 client.18546 fetch_fsmap learned FSMap version 25 2024-09-12T03:14:20.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.046+0000 7f5cea195740 1 -- 172.21.15.116:0/3510036308 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5650ba0e4ae0 con 0x5650ba0d6370 2024-09-12T03:14:20.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.046+0000 7f5cc57fa640 1 -- 172.21.15.116:0/3510036308 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cd0036040 con 0x5650ba0d6370 2024-09-12T03:14:20.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.047+0000 7f5cc57fa640 1 -- 172.21.15.116:0/3510036308 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5cd0030070 con 0x5650ba0d6370 2024-09-12T03:14:20.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.047+0000 7f5cea195740 10 client.18546 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:14:20.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.047+0000 7f5cea195740 10 client.18546 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:14:20.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.047+0000 7f5cea195740 20 client.18546 populate_metadata read hostname 'smithi116' 2024-09-12T03:14:20.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.047+0000 7f5cea195740 1 --2- 172.21.15.116:0/3510036308 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x5650ba0e5e30 0x5650ba0e8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:20.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.047+0000 7f5cea195740 4 client.18546 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:14:20.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.047+0000 7f5cea195740 1 -- 172.21.15.116:0/3510036308 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650ba03e860 con 0x5650ba0e5e30 2024-09-12T03:14:20.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.047+0000 7f5ce5b58640 1 --2- 172.21.15.116:0/3510036308 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x5650ba0e5e30 0x5650ba0e8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:20.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.047+0000 7f5ce5b58640 1 --2- 172.21.15.116:0/3510036308 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x5650ba0e5e30 0x5650ba0e8220 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9ac7151b9d5bd867 server_cookie=5e606a321e37f181 in_seq=0 out_seq=0 2024-09-12T03:14:20.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.047+0000 7f5cc57fa640 10 client.18546 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:14:20.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.048+0000 7f5cc57fa640 1 -- 172.21.15.116:0/3510036308 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5650ba03e860 con 0x5650ba0e5e30 2024-09-12T03:14:20.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.048+0000 7f5cc57fa640 10 client.18546 handle_command_reply: tid=0 2024-09-12T03:14:20.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.048+0000 7f5cea195740 1 client.18546 shutdown 2024-09-12T03:14:20.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.048+0000 7f5cea195740 1 -- 172.21.15.116:0/3510036308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5650ba0d6370 msgr2=0x5650ba0d3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:20.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.048+0000 7f5cea195740 1 --2- 172.21.15.116:0/3510036308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5650ba0d6370 0x5650ba0d3650 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0009800 tx=0x7f5cd00041c0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.048+0000 7f5cea195740 1 -- 172.21.15.116:0/3510036308 shutdown_connections 2024-09-12T03:14:20.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.048+0000 7f5cea195740 1 --2- 172.21.15.116:0/3510036308 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x5650ba0e5e30 0x5650ba0e8220 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.048+0000 7f5cea195740 1 --2- 172.21.15.116:0/3510036308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5650ba0d6370 0x5650ba0d3650 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.048+0000 7f5cea195740 1 --2- 172.21.15.116:0/3510036308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650ba0ccb40 0x5650ba0d3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.048+0000 7f5cc57fa640 0 client.18546 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:14:20.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.048+0000 7f5cea195740 1 --2- 172.21.15.116:0/3510036308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5650ba0cb5d0 0x5650ba0d5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.048+0000 7f5cea195740 1 -- 172.21.15.116:0/3510036308 >> 172.21.15.116:0/3510036308 conn(0x5650ba094310 msgr2=0x5650ba0cd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:20.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.048+0000 7f5cea195740 1 -- 172.21.15.116:0/3510036308 shutdown_connections 2024-09-12T03:14:20.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.048+0000 7f5cea195740 1 -- 172.21.15.116:0/3510036308 wait complete. 2024-09-12T03:14:20.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.048+0000 7f5cea195740 20 client.18546 trim_cache size 0 max 16384 2024-09-12T03:14:20.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.061+0000 7f5cea195740 1 Processor -- start 2024-09-12T03:14:20.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.061+0000 7f5cea195740 1 -- start start 2024-09-12T03:14:20.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.061+0000 7f5cea195740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5650ba0cb5d0 0x5650ba0e22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:20.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.061+0000 7f5cea195740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5650ba0ccb40 0x5650ba0ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:20.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.061+0000 7f5cea195740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650ba0d6370 0x5650ba0ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:20.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.061+0000 7f5cea195740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce01afe50 con 0x5650ba0cb5d0 2024-09-12T03:14:20.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.061+0000 7f5cea195740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce01ad720 con 0x5650ba0ccb40 2024-09-12T03:14:20.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.061+0000 7f5cea195740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce01ab570 con 0x5650ba0d6370 2024-09-12T03:14:20.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.061+0000 7f5ce5b58640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5650ba0cb5d0 0x5650ba0e22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:20.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.062+0000 7f5ce5b58640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5650ba0cb5d0 0x5650ba0e22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46266/0 (socket says 172.21.15.116:46266) 2024-09-12T03:14:20.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.062+0000 7f5ce5b58640 1 -- 172.21.15.116:0/811238384 learned_addr learned my addr 172.21.15.116:0/811238384 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:20.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.062+0000 7f5ce5357640 1 --2- 172.21.15.116:0/811238384 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5650ba0ccb40 0x5650ba0ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:20.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.062+0000 7f5ce5b58640 1 -- 172.21.15.116:0/811238384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650ba0d6370 msgr2=0x5650ba0ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:20.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.062+0000 7f5ce6359640 1 --2- 172.21.15.116:0/811238384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650ba0d6370 0x5650ba0ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:20.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.062+0000 7f5ce5b58640 1 --2- 172.21.15.116:0/811238384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650ba0d6370 0x5650ba0ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.062+0000 7f5ce5b58640 1 -- 172.21.15.116:0/811238384 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5650ba0ccb40 msgr2=0x5650ba0ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:20.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.062+0000 7f5ce5b58640 1 --2- 172.21.15.116:0/811238384 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5650ba0ccb40 0x5650ba0ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.062+0000 7f5ce5b58640 1 -- 172.21.15.116:0/811238384 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650ba0e8590 con 0x5650ba0cb5d0 2024-09-12T03:14:20.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.062+0000 7f5ce5357640 1 --2- 172.21.15.116:0/811238384 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5650ba0ccb40 0x5650ba0ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:14:20.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.062+0000 7f5ce6359640 1 --2- 172.21.15.116:0/811238384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650ba0d6370 0x5650ba0ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:14:20.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.062+0000 7f5ce5b58640 1 --2- 172.21.15.116:0/811238384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5650ba0cb5d0 0x5650ba0e22b0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x5650ba0e87c0 tx=0x7f5cd8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86262a18ce9509ba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:20.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.062+0000 7f5cc77fe640 1 -- 172.21.15.116:0/811238384 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8004020 con 0x5650ba0cb5d0 2024-09-12T03:14:20.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.062+0000 7f5cc77fe640 1 -- 172.21.15.116:0/811238384 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cd80041c0 con 0x5650ba0cb5d0 2024-09-12T03:14:20.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.062+0000 7f5cc77fe640 1 -- 172.21.15.116:0/811238384 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd80116a0 con 0x5650ba0cb5d0 2024-09-12T03:14:20.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.063+0000 7f5cea195740 1 -- 172.21.15.116:0/811238384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5650ba0cb5d0 msgr2=0x5650ba0e22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:20.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.063+0000 7f5cea195740 1 --2- 172.21.15.116:0/811238384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5650ba0cb5d0 0x5650ba0e22b0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x5650ba0e87c0 tx=0x7f5cd8015000 comp rx=0 tx=0).stop 2024-09-12T03:14:20.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.063+0000 7f5cea195740 1 -- 172.21.15.116:0/811238384 shutdown_connections 2024-09-12T03:14:20.083 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:14:20.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.063+0000 7f5cea195740 1 --2- 172.21.15.116:0/811238384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650ba0d6370 0x5650ba0ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.063+0000 7f5cea195740 1 --2- 172.21.15.116:0/811238384 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5650ba0ccb40 0x5650ba0ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.063+0000 7f5cea195740 1 --2- 172.21.15.116:0/811238384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5650ba0cb5d0 0x5650ba0e22b0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.063+0000 7f5cea195740 1 -- 172.21.15.116:0/811238384 >> 172.21.15.116:0/811238384 conn(0x5650ba094310 msgr2=0x5650ba0d50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:20.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.063+0000 7f5cea195740 1 -- 172.21.15.116:0/811238384 shutdown_connections 2024-09-12T03:14:20.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.063+0000 7f5cea195740 1 -- 172.21.15.116:0/811238384 wait complete. 2024-09-12T03:14:20.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.063+0000 7f5cea195740 1 -- start start 2024-09-12T03:14:20.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.063+0000 7f5cea195740 1 Processor -- start 2024-09-12T03:14:20.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.064+0000 7f5cea195740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650ba0cb5d0 0x5650ba0de8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:20.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.064+0000 7f5cea195740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5650ba0ccb40 0x5650ba0cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:20.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.064+0000 7f5cea195740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5650ba0d6370 0x5650ba0cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:20.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.064+0000 7f5cea195740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce01abbe0 con 0x5650ba0d6370 2024-09-12T03:14:20.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.064+0000 7f5cea195740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce01ad050 con 0x5650ba0ccb40 2024-09-12T03:14:20.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.064+0000 7f5cea195740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce01ac2b0 con 0x5650ba0cb5d0 2024-09-12T03:14:20.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.064+0000 7f5cea195740 10 client.0 _ll_register_callbacks cb 0x7f5cd003f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:20.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.064+0000 7f5ce6359640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5650ba0d6370 0x5650ba0cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:20.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.064+0000 7f5ce6359640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5650ba0d6370 0x5650ba0cffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46268/0 (socket says 172.21.15.116:46268) 2024-09-12T03:14:20.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.064+0000 7f5ce6359640 1 -- 172.21.15.116:0/494051891 learned_addr learned my addr 172.21.15.116:0/494051891 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:20.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.064+0000 7f5ce5357640 1 --2- 172.21.15.116:0/494051891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5650ba0ccb40 0x5650ba0cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:20.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.064+0000 7f5ce5b58640 1 --2- 172.21.15.116:0/494051891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650ba0cb5d0 0x5650ba0de8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:20.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.064+0000 7f5ce6359640 1 -- 172.21.15.116:0/494051891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650ba0cb5d0 msgr2=0x5650ba0de8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:20.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.064+0000 7f5ce6359640 1 --2- 172.21.15.116:0/494051891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650ba0cb5d0 0x5650ba0de8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.064+0000 7f5ce6359640 1 -- 172.21.15.116:0/494051891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5650ba0ccb40 msgr2=0x5650ba0cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:20.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.064+0000 7f5ce6359640 1 --2- 172.21.15.116:0/494051891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5650ba0ccb40 0x5650ba0cfa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.064+0000 7f5ce6359640 1 -- 172.21.15.116:0/494051891 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650ba0d0500 con 0x5650ba0d6370 2024-09-12T03:14:20.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.064+0000 7f5ce6359640 1 --2- 172.21.15.116:0/494051891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5650ba0d6370 0x5650ba0cffc0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0004af0 tx=0x7f5cd0004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f52024184dccedc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:20.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.064+0000 7f5cc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:14:20.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.064+0000 7f5cc57fa640 1 -- 172.21.15.116:0/494051891 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0031430 con 0x5650ba0d6370 2024-09-12T03:14:20.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.064+0000 7f5cc57fa640 1 -- 172.21.15.116:0/494051891 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f5cd00305a0 con 0x5650ba0d6370 2024-09-12T03:14:20.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.064+0000 7f5cea195740 1 -- 172.21.15.116:0/494051891 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cd80041c0 con 0x5650ba0d6370 2024-09-12T03:14:20.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.065+0000 7f5cc57fa640 1 -- 172.21.15.116:0/494051891 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5cd00307a0 con 0x5650ba0d6370 2024-09-12T03:14:20.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.065+0000 7f5cea195740 10 client.18552 fetch_fsmap learned FSMap version 25 2024-09-12T03:14:20.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.065+0000 7f5cea195740 1 -- 172.21.15.116:0/494051891 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5650ba150470 con 0x5650ba0d6370 2024-09-12T03:14:20.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.065+0000 7f5cc57fa640 1 -- 172.21.15.116:0/494051891 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cd80041c0 con 0x5650ba0d6370 2024-09-12T03:14:20.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.065+0000 7f5cc57fa640 1 -- 172.21.15.116:0/494051891 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5cd00539f0 con 0x5650ba0d6370 2024-09-12T03:14:20.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.065+0000 7f5cea195740 10 client.18552 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:14:20.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.065+0000 7f5cea195740 10 client.18552 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:14:20.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.065+0000 7f5cea195740 20 client.18552 populate_metadata read hostname 'smithi116' 2024-09-12T03:14:20.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.065+0000 7f5cea195740 1 --2- 172.21.15.116:0/494051891 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x5650ba150bf0 0x5650ba153010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:20.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.065+0000 7f5cea195740 4 client.18552 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:14:20.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.065+0000 7f5cea195740 1 -- 172.21.15.116:0/494051891 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5650ba038d70 con 0x5650ba150bf0 2024-09-12T03:14:20.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.065+0000 7f5ce5b58640 1 --2- 172.21.15.116:0/494051891 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x5650ba150bf0 0x5650ba153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:20.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.066+0000 7f5cc57fa640 10 client.18552 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:14:20.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.066+0000 7f5ce5b58640 1 --2- 172.21.15.116:0/494051891 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x5650ba150bf0 0x5650ba153010 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ee4fc77f36296e4b server_cookie=69ec896ea8b4fa56 in_seq=0 out_seq=0 2024-09-12T03:14:20.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.066+0000 7f5cc57fa640 1 -- 172.21.15.116:0/494051891 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5650ba038d70 con 0x5650ba150bf0 2024-09-12T03:14:20.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.066+0000 7f5cc57fa640 10 client.18552 handle_command_reply: tid=0 2024-09-12T03:14:20.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.066+0000 7f5cea195740 1 client.18552 shutdown 2024-09-12T03:14:20.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.066+0000 7f5cea195740 1 -- 172.21.15.116:0/494051891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5650ba0d6370 msgr2=0x5650ba0cffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:20.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.066+0000 7f5cea195740 1 --2- 172.21.15.116:0/494051891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5650ba0d6370 0x5650ba0cffc0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0004af0 tx=0x7f5cd0004920 comp rx=0 tx=0).stop 2024-09-12T03:14:20.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.066+0000 7f5cea195740 1 -- 172.21.15.116:0/494051891 shutdown_connections 2024-09-12T03:14:20.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.066+0000 7f5cea195740 1 --2- 172.21.15.116:0/494051891 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x5650ba150bf0 0x5650ba153010 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.066+0000 7f5cea195740 1 --2- 172.21.15.116:0/494051891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5650ba0d6370 0x5650ba0cffc0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.066+0000 7f5cea195740 1 --2- 172.21.15.116:0/494051891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5650ba0ccb40 0x5650ba0cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.066+0000 7f5cea195740 1 --2- 172.21.15.116:0/494051891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650ba0cb5d0 0x5650ba0de8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.066+0000 7f5cc57fa640 0 client.18552 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:14:20.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.066+0000 7f5cea195740 1 -- 172.21.15.116:0/494051891 >> 172.21.15.116:0/494051891 conn(0x5650ba094310 msgr2=0x5650ba0c64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:20.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.066+0000 7f5cea195740 1 -- 172.21.15.116:0/494051891 shutdown_connections 2024-09-12T03:14:20.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.066+0000 7f5cea195740 1 -- 172.21.15.116:0/494051891 wait complete. 2024-09-12T03:14:20.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.066+0000 7f5cea195740 20 client.18552 trim_cache size 0 max 16384 2024-09-12T03:14:20.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.067+0000 7f5ce7dbc640 1 -- 172.21.15.116:0/2137738458 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5cc80a2b80 msgr2=0x7f5cc80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:20.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.067+0000 7f5ce7dbc640 1 --2- 172.21.15.116:0/2137738458 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5cc80a2b80 0x7f5cc80a5040 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x5650ba0c8ac0 tx=0x7f5cd8005e90 comp rx=0 tx=0).stop 2024-09-12T03:14:20.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.067+0000 7f5ce7dbc640 1 -- 172.21.15.116:0/2137738458 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ce01036e0 msgr2=0x7f5ce0198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:20.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.067+0000 7f5ce7dbc640 1 --2- 172.21.15.116:0/2137738458 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ce01036e0 0x7f5ce0198a70 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f5cd400c970 tx=0x7f5cd400ce30 comp rx=0 tx=0).stop 2024-09-12T03:14:20.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.067+0000 7f5ce7dbc640 1 -- 172.21.15.116:0/2137738458 shutdown_connections 2024-09-12T03:14:20.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.067+0000 7f5ce7dbc640 1 --2- 172.21.15.116:0/2137738458 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5cc80a2b80 0x7f5cc80a5040 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.067+0000 7f5ce7dbc640 1 --2- 172.21.15.116:0/2137738458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ce01082a0 0x7f5ce019ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.067+0000 7f5ce7dbc640 1 --2- 172.21.15.116:0/2137738458 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ce01036e0 0x7f5ce0198a70 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.067+0000 7f5ce7dbc640 1 --2- 172.21.15.116:0/2137738458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ce0102d30 0x7f5ce0198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:20.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.067+0000 7f5ce7dbc640 1 -- 172.21.15.116:0/2137738458 >> 172.21.15.116:0/2137738458 conn(0x7f5ce00fc7d0 msgr2=0x7f5ce0105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:20.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.067+0000 7f5ce7dbc640 1 -- 172.21.15.116:0/2137738458 shutdown_connections 2024-09-12T03:14:20.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:20.067+0000 7f5ce7dbc640 1 -- 172.21.15.116:0/2137738458 wait complete. 2024-09-12T03:14:20.804 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:14:21.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:20 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:21.060 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:14:21.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:20 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:21.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:20 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:21.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.576+0000 7f9f6116b640 1 -- 172.21.15.116:0/2482188823 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f5c1082a0 msgr2=0x7f9f5c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:21.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.576+0000 7f9f6116b640 1 --2- 172.21.15.116:0/2482188823 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f5c1082a0 0x7f9f5c10ec40 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f9f54009cd0 tx=0x7f9f5402a8e0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.576+0000 7f9f6116b640 1 -- 172.21.15.116:0/2482188823 shutdown_connections 2024-09-12T03:14:21.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.576+0000 7f9f6116b640 1 --2- 172.21.15.116:0/2482188823 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f5c1082a0 0x7f9f5c10ec40 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.576+0000 7f9f6116b640 1 --2- 172.21.15.116:0/2482188823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f5c1036e0 0x7f9f5c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.576+0000 7f9f6116b640 1 --2- 172.21.15.116:0/2482188823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f5c102d30 0x7f9f5c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.576+0000 7f9f6116b640 1 -- 172.21.15.116:0/2482188823 >> 172.21.15.116:0/2482188823 conn(0x7f9f5c0fc7d0 msgr2=0x7f9f5c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:21.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.576+0000 7f9f6116b640 1 -- 172.21.15.116:0/2482188823 shutdown_connections 2024-09-12T03:14:21.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.576+0000 7f9f6116b640 1 -- 172.21.15.116:0/2482188823 wait complete. 2024-09-12T03:14:21.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.577+0000 7f9f6116b640 1 Processor -- start 2024-09-12T03:14:21.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.577+0000 7f9f6116b640 1 -- start start 2024-09-12T03:14:21.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.577+0000 7f9f6116b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f5c102d30 0x7f9f5c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:21.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.577+0000 7f9f6116b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f5c1036e0 0x7f9f5c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:21.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.577+0000 7f9f6116b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f5c1082a0 0x7f9f5c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:21.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.577+0000 7f9f6116b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c1120a0 con 0x7f9f5c102d30 2024-09-12T03:14:21.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.577+0000 7f9f6116b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c111f20 con 0x7f9f5c1082a0 2024-09-12T03:14:21.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.577+0000 7f9f6116b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c112220 con 0x7f9f5c1036e0 2024-09-12T03:14:21.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.577+0000 7f9f5ad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f5c102d30 0x7f9f5c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:21.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.577+0000 7f9f5ad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f5c102d30 0x7f9f5c100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37816/0 (socket says 172.21.15.116:37816) 2024-09-12T03:14:21.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.577+0000 7f9f5ad9d640 1 -- 172.21.15.116:0/2090715611 learned_addr learned my addr 172.21.15.116:0/2090715611 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:21.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.577+0000 7f9f5b59e640 1 --2- 172.21.15.116:0/2090715611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f5c1082a0 0x7f9f5c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:21.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.577+0000 7f9f5a59c640 1 --2- 172.21.15.116:0/2090715611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f5c1036e0 0x7f9f5c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:21.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.578+0000 7f9f5ad9d640 1 -- 172.21.15.116:0/2090715611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f5c1036e0 msgr2=0x7f9f5c100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:21.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.578+0000 7f9f5ad9d640 1 --2- 172.21.15.116:0/2090715611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f5c1036e0 0x7f9f5c100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.578+0000 7f9f5ad9d640 1 -- 172.21.15.116:0/2090715611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f5c1082a0 msgr2=0x7f9f5c101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:21.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.578+0000 7f9f5ad9d640 1 --2- 172.21.15.116:0/2090715611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f5c1082a0 0x7f9f5c101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.578+0000 7f9f5ad9d640 1 -- 172.21.15.116:0/2090715611 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5c1a69b0 con 0x7f9f5c102d30 2024-09-12T03:14:21.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.578+0000 7f9f5ad9d640 1 --2- 172.21.15.116:0/2090715611 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f5c102d30 0x7f9f5c100990 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00d770 tx=0x7f9f4c00db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:21.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.578+0000 7f9f43fff640 1 -- 172.21.15.116:0/2090715611 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c017070 con 0x7f9f5c102d30 2024-09-12T03:14:21.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.578+0000 7f9f6116b640 1 -- 172.21.15.116:0/2090715611 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f5c1a6ca0 con 0x7f9f5c102d30 2024-09-12T03:14:21.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.578+0000 7f9f43fff640 1 -- 172.21.15.116:0/2090715611 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f4c009820 con 0x7f9f5c102d30 2024-09-12T03:14:21.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.578+0000 7f9f43fff640 1 -- 172.21.15.116:0/2090715611 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c004cd0 con 0x7f9f5c102d30 2024-09-12T03:14:21.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.578+0000 7f9f6116b640 1 -- 172.21.15.116:0/2090715611 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f5c1a7190 con 0x7f9f5c102d30 2024-09-12T03:14:21.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.580+0000 7f9f43fff640 1 -- 172.21.15.116:0/2090715611 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f9f4c0099d0 con 0x7f9f5c102d30 2024-09-12T03:14:21.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.580+0000 7f9f63544740 1 Processor -- start 2024-09-12T03:14:21.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.581+0000 7f9f63544740 1 -- start start 2024-09-12T03:14:21.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.581+0000 7f9f63544740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb7a4beb40 0x55cb7a4bd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:21.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.581+0000 7f9f63544740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cb7a4bd5d0 0x55cb7a4bda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:21.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.581+0000 7f9f63544740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb7a4c8370 0x55cb7a4ca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:21.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.581+0000 7f9f63544740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55cb7a4232f0 con 0x55cb7a4bd5d0 2024-09-12T03:14:21.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.581+0000 7f9f63544740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55cb7a3cacf0 con 0x55cb7a4beb40 2024-09-12T03:14:21.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.581+0000 7f9f63544740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cb7a41f2b0 con 0x55cb7a4c8370 2024-09-12T03:14:21.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.581+0000 7f9f43fff640 1 --2- 172.21.15.116:0/2090715611 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9f440a2b80 0x7f9f440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:21.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.581+0000 7f9f5ad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cb7a4bd5d0 0x55cb7a4bda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:21.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.581+0000 7f9f5ad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cb7a4bd5d0 0x55cb7a4bda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37818/0 (socket says 172.21.15.116:37818) 2024-09-12T03:14:21.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.581+0000 7f9f5ad9d640 1 -- 172.21.15.116:0/4278651215 learned_addr learned my addr 172.21.15.116:0/4278651215 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:21.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.581+0000 7f9f5b59e640 1 --2- 172.21.15.116:0/4278651215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb7a4beb40 0x55cb7a4bd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:21.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.581+0000 7f9f5a59c640 1 --2- 172.21.15.116:0/4278651215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb7a4c8370 0x55cb7a4ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:21.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.581+0000 7f9f5b59e640 1 --2- 172.21.15.116:0/2090715611 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9f440a2b80 0x7f9f440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:21.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.581+0000 7f9f43fff640 1 -- 172.21.15.116:0/2090715611 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f9f4c0ce9a0 con 0x7f9f5c102d30 2024-09-12T03:14:21.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.581+0000 7f9f5ad9d640 1 -- 172.21.15.116:0/4278651215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb7a4c8370 msgr2=0x55cb7a4ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:21.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.581+0000 7f9f5ad9d640 1 --2- 172.21.15.116:0/4278651215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb7a4c8370 0x55cb7a4ca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.581+0000 7f9f5ad9d640 1 -- 172.21.15.116:0/4278651215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb7a4beb40 msgr2=0x55cb7a4bd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:21.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.581+0000 7f9f5ad9d640 1 --2- 172.21.15.116:0/4278651215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb7a4beb40 0x55cb7a4bd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.581+0000 7f9f5ad9d640 1 -- 172.21.15.116:0/4278651215 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb7a3af3a0 con 0x55cb7a4bd5d0 2024-09-12T03:14:21.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.581+0000 7f9f5b59e640 1 --2- 172.21.15.116:0/4278651215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb7a4beb40 0x55cb7a4bd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:21.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.582+0000 7f9f5ad9d640 1 --2- 172.21.15.116:0/4278651215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cb7a4bd5d0 0x55cb7a4bda10 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00d740 tx=0x7f9f4c09b130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91320966937cdbf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:21.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.582+0000 7f9f417fa640 1 -- 172.21.15.116:0/4278651215 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c020040 con 0x55cb7a4bd5d0 2024-09-12T03:14:21.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.582+0000 7f9f5b59e640 1 --2- 172.21.15.116:0/2090715611 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9f440a2b80 0x7f9f440a5040 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f9f54004ab0 tx=0x7f9f54004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:21.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.582+0000 7f9f417fa640 1 -- 172.21.15.116:0/4278651215 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f4c0cee50 con 0x55cb7a4bd5d0 2024-09-12T03:14:21.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.582+0000 7f9f417fa640 1 -- 172.21.15.116:0/4278651215 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c0978a0 con 0x55cb7a4bd5d0 2024-09-12T03:14:21.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.582+0000 7f9f63544740 1 -- 172.21.15.116:0/4278651215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cb7a4bd5d0 msgr2=0x55cb7a4bda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:21.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.582+0000 7f9f63544740 1 --2- 172.21.15.116:0/4278651215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cb7a4bd5d0 0x55cb7a4bda10 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00d740 tx=0x7f9f4c09b130 comp rx=0 tx=0).stop 2024-09-12T03:14:21.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.583+0000 7f9f63544740 1 -- 172.21.15.116:0/4278651215 shutdown_connections 2024-09-12T03:14:21.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.583+0000 7f9f63544740 1 --2- 172.21.15.116:0/4278651215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb7a4c8370 0x55cb7a4ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.583+0000 7f9f63544740 1 --2- 172.21.15.116:0/4278651215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cb7a4bd5d0 0x55cb7a4bda10 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.583+0000 7f9f63544740 1 --2- 172.21.15.116:0/4278651215 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb7a4beb40 0x55cb7a4bd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.583+0000 7f9f63544740 1 -- 172.21.15.116:0/4278651215 >> 172.21.15.116:0/4278651215 conn(0x55cb7a486310 msgr2=0x55cb7a4b9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:21.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.583+0000 7f9f63544740 1 -- 172.21.15.116:0/4278651215 shutdown_connections 2024-09-12T03:14:21.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.583+0000 7f9f63544740 1 -- 172.21.15.116:0/4278651215 wait complete. 2024-09-12T03:14:21.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.583+0000 7f9f63544740 1 -- start start 2024-09-12T03:14:21.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.583+0000 7f9f63544740 1 Processor -- start 2024-09-12T03:14:21.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.583+0000 7f9f63544740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cb7a4bd5d0 0x55cb7a4c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:21.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.583+0000 7f9f63544740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb7a4beb40 0x55cb7a4c50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:21.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.583+0000 7f9f63544740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb7a4c8370 0x55cb7a4c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:21.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.583+0000 7f9f63544740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55cb7a4cafb0 con 0x55cb7a4bd5d0 2024-09-12T03:14:21.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.583+0000 7f9f63544740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55cb7a3de060 con 0x55cb7a4c8370 2024-09-12T03:14:21.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.583+0000 7f9f63544740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cb7a4cb130 con 0x55cb7a4beb40 2024-09-12T03:14:21.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.583+0000 7f9f63544740 10 client.0 _ll_register_callbacks cb 0x55cb7a1318c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:21.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.583+0000 7f9f5ad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cb7a4bd5d0 0x55cb7a4c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:21.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.584+0000 7f9f5ad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cb7a4bd5d0 0x55cb7a4c7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37828/0 (socket says 172.21.15.116:37828) 2024-09-12T03:14:21.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.584+0000 7f9f5ad9d640 1 -- 172.21.15.116:0/2010905050 learned_addr learned my addr 172.21.15.116:0/2010905050 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:21.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.584+0000 7f9f5a59c640 1 --2- 172.21.15.116:0/2010905050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb7a4beb40 0x55cb7a4c50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:21.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.584+0000 7f9f5b59e640 1 --2- 172.21.15.116:0/2010905050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb7a4c8370 0x55cb7a4c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:21.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.584+0000 7f9f5ad9d640 1 -- 172.21.15.116:0/2010905050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb7a4beb40 msgr2=0x55cb7a4c50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:21.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.584+0000 7f9f5ad9d640 1 --2- 172.21.15.116:0/2010905050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb7a4beb40 0x55cb7a4c50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.584+0000 7f9f5ad9d640 1 -- 172.21.15.116:0/2010905050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb7a4c8370 msgr2=0x55cb7a4c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:21.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.584+0000 7f9f5ad9d640 1 --2- 172.21.15.116:0/2010905050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb7a4c8370 0x55cb7a4c5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.584+0000 7f9f5ad9d640 1 -- 172.21.15.116:0/2010905050 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb7a1943c0 con 0x55cb7a4bd5d0 2024-09-12T03:14:21.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.584+0000 7f9f5ad9d640 1 --2- 172.21.15.116:0/2010905050 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cb7a4bd5d0 0x55cb7a4c7e20 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00dfd0 tx=0x7f9f4c09b370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db9f79a417708e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:21.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.584+0000 7f9f3eff5640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:14:21.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.584+0000 7f9f3eff5640 1 -- 172.21.15.116:0/2010905050 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c0b3020 con 0x55cb7a4bd5d0 2024-09-12T03:14:21.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.584+0000 7f9f3eff5640 1 -- 172.21.15.116:0/2010905050 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f9f4c0978a0 con 0x55cb7a4bd5d0 2024-09-12T03:14:21.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.584+0000 7f9f63544740 1 -- 172.21.15.116:0/2010905050 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f4c0cee50 con 0x55cb7a4bd5d0 2024-09-12T03:14:21.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.584+0000 7f9f3eff5640 1 -- 172.21.15.116:0/2010905050 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9f4c020050 con 0x55cb7a4bd5d0 2024-09-12T03:14:21.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.584+0000 7f9f63544740 10 client.18564 fetch_fsmap learned FSMap version 25 2024-09-12T03:14:21.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.584+0000 7f9f63544740 1 -- 172.21.15.116:0/2010905050 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cb7a4d6b20 con 0x55cb7a4bd5d0 2024-09-12T03:14:21.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.584+0000 7f9f3eff5640 1 -- 172.21.15.116:0/2010905050 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f4c0cee50 con 0x55cb7a4bd5d0 2024-09-12T03:14:21.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.584+0000 7f9f3eff5640 1 -- 172.21.15.116:0/2010905050 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9f4c09a680 con 0x55cb7a4bd5d0 2024-09-12T03:14:21.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.585+0000 7f9f63544740 10 client.18564 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:14:21.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.585+0000 7f9f63544740 10 client.18564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:14:21.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.585+0000 7f9f63544740 20 client.18564 populate_metadata read hostname 'smithi116' 2024-09-12T03:14:21.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.585+0000 7f9f63544740 1 --2- 172.21.15.116:0/2010905050 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55cb7a4d7e70 0x55cb7a4da260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:21.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.585+0000 7f9f63544740 4 client.18564 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:14:21.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.585+0000 7f9f63544740 1 -- 172.21.15.116:0/2010905050 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb7a430860 con 0x55cb7a4d7e70 2024-09-12T03:14:21.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.585+0000 7f9f5a59c640 1 --2- 172.21.15.116:0/2010905050 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55cb7a4d7e70 0x55cb7a4da260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:21.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.585+0000 7f9f5a59c640 1 --2- 172.21.15.116:0/2010905050 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55cb7a4d7e70 0x55cb7a4da260 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78a24f376d761087 server_cookie=46cd602e8f96e82a in_seq=0 out_seq=0 2024-09-12T03:14:21.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.585+0000 7f9f3eff5640 10 client.18564 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:21.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.586+0000 7f9f3eff5640 1 -- 172.21.15.116:0/2010905050 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cb7a430860 con 0x55cb7a4d7e70 2024-09-12T03:14:21.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.586+0000 7f9f3eff5640 10 client.18564 handle_command_reply: tid=0 2024-09-12T03:14:21.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.586+0000 7f9f63544740 1 client.18564 shutdown 2024-09-12T03:14:21.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.586+0000 7f9f63544740 1 -- 172.21.15.116:0/2010905050 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cb7a4bd5d0 msgr2=0x55cb7a4c7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:21.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.586+0000 7f9f63544740 1 --2- 172.21.15.116:0/2010905050 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cb7a4bd5d0 0x55cb7a4c7e20 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00dfd0 tx=0x7f9f4c09b370 comp rx=0 tx=0).stop 2024-09-12T03:14:21.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.586+0000 7f9f63544740 1 -- 172.21.15.116:0/2010905050 shutdown_connections 2024-09-12T03:14:21.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.586+0000 7f9f63544740 1 --2- 172.21.15.116:0/2010905050 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55cb7a4d7e70 0x55cb7a4da260 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.586+0000 7f9f63544740 1 --2- 172.21.15.116:0/2010905050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb7a4c8370 0x55cb7a4c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.586+0000 7f9f3eff5640 0 client.18564 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:21.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.586+0000 7f9f63544740 1 --2- 172.21.15.116:0/2010905050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb7a4beb40 0x55cb7a4c50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.586+0000 7f9f63544740 1 --2- 172.21.15.116:0/2010905050 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cb7a4bd5d0 0x55cb7a4c7e20 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.586+0000 7f9f63544740 1 -- 172.21.15.116:0/2010905050 >> 172.21.15.116:0/2010905050 conn(0x55cb7a486310 msgr2=0x55cb7a4bf990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:21.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.586+0000 7f9f63544740 1 -- 172.21.15.116:0/2010905050 shutdown_connections 2024-09-12T03:14:21.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.586+0000 7f9f63544740 1 -- 172.21.15.116:0/2010905050 wait complete. 2024-09-12T03:14:21.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.586+0000 7f9f63544740 20 client.18564 trim_cache size 0 max 16384 2024-09-12T03:14:21.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.599+0000 7f9f63544740 1 Processor -- start 2024-09-12T03:14:21.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.599+0000 7f9f63544740 1 -- start start 2024-09-12T03:14:21.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.599+0000 7f9f63544740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cb7a4bd5d0 0x55cb7a4c0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:21.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.599+0000 7f9f63544740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb7a4beb40 0x55cb7a4c0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:21.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.599+0000 7f9f63544740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb7a4c8370 0x55cb7a4dd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:21.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.599+0000 7f9f63544740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c1b1e30 con 0x55cb7a4bd5d0 2024-09-12T03:14:21.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.599+0000 7f9f63544740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c1af700 con 0x55cb7a4beb40 2024-09-12T03:14:21.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.599+0000 7f9f63544740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c1ad550 con 0x55cb7a4c8370 2024-09-12T03:14:21.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.599+0000 7f9f5ad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cb7a4bd5d0 0x55cb7a4c0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:21.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.599+0000 7f9f5ad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cb7a4bd5d0 0x55cb7a4c0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37842/0 (socket says 172.21.15.116:37842) 2024-09-12T03:14:21.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.599+0000 7f9f5ad9d640 1 -- 172.21.15.116:0/4074229302 learned_addr learned my addr 172.21.15.116:0/4074229302 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:21.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.599+0000 7f9f5b59e640 1 --2- 172.21.15.116:0/4074229302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb7a4c8370 0x55cb7a4dd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:21.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.599+0000 7f9f5a59c640 1 --2- 172.21.15.116:0/4074229302 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb7a4beb40 0x55cb7a4c0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:21.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.599+0000 7f9f5ad9d640 1 -- 172.21.15.116:0/4074229302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb7a4c8370 msgr2=0x55cb7a4dd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:21.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.599+0000 7f9f5ad9d640 1 --2- 172.21.15.116:0/4074229302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb7a4c8370 0x55cb7a4dd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.599+0000 7f9f5ad9d640 1 -- 172.21.15.116:0/4074229302 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb7a4beb40 msgr2=0x55cb7a4c0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:21.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.599+0000 7f9f5ad9d640 1 --2- 172.21.15.116:0/4074229302 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb7a4beb40 0x55cb7a4c0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.599+0000 7f9f5ad9d640 1 -- 172.21.15.116:0/4074229302 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb7a4bde70 con 0x55cb7a4bd5d0 2024-09-12T03:14:21.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.599+0000 7f9f5b59e640 1 --2- 172.21.15.116:0/4074229302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb7a4c8370 0x55cb7a4dd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:21.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.599+0000 7f9f5ad9d640 1 --2- 172.21.15.116:0/4074229302 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cb7a4bd5d0 0x55cb7a4c0120 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00d740 tx=0x7f9f4c009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6556a522a1c5fb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:21.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.600+0000 7f9f40ff9640 1 -- 172.21.15.116:0/4074229302 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c0b3070 con 0x55cb7a4bd5d0 2024-09-12T03:14:21.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.600+0000 7f9f40ff9640 1 -- 172.21.15.116:0/4074229302 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f4c09f5e0 con 0x55cb7a4bd5d0 2024-09-12T03:14:21.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.600+0000 7f9f40ff9640 1 -- 172.21.15.116:0/4074229302 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c0978a0 con 0x55cb7a4bd5d0 2024-09-12T03:14:21.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.600+0000 7f9f63544740 1 -- 172.21.15.116:0/4074229302 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cb7a4bd5d0 msgr2=0x55cb7a4c0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:21.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.600+0000 7f9f63544740 1 --2- 172.21.15.116:0/4074229302 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cb7a4bd5d0 0x55cb7a4c0120 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00d740 tx=0x7f9f4c009d60 comp rx=0 tx=0).stop 2024-09-12T03:14:21.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.600+0000 7f9f63544740 1 -- 172.21.15.116:0/4074229302 shutdown_connections 2024-09-12T03:14:21.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.600+0000 7f9f63544740 1 --2- 172.21.15.116:0/4074229302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb7a4c8370 0x55cb7a4dd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.600+0000 7f9f63544740 1 --2- 172.21.15.116:0/4074229302 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb7a4beb40 0x55cb7a4c0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.600+0000 7f9f63544740 1 --2- 172.21.15.116:0/4074229302 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cb7a4bd5d0 0x55cb7a4c0120 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.600+0000 7f9f63544740 1 -- 172.21.15.116:0/4074229302 >> 172.21.15.116:0/4074229302 conn(0x55cb7a486310 msgr2=0x55cb7a4b8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:21.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.600+0000 7f9f63544740 1 -- 172.21.15.116:0/4074229302 shutdown_connections 2024-09-12T03:14:21.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.600+0000 7f9f63544740 1 -- 172.21.15.116:0/4074229302 wait complete. 2024-09-12T03:14:21.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.601+0000 7f9f63544740 1 -- start start 2024-09-12T03:14:21.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.601+0000 7f9f63544740 1 Processor -- start 2024-09-12T03:14:21.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.601+0000 7f9f63544740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb7a4bd5d0 0x55cb7a4dd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:21.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.601+0000 7f9f63544740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cb7a4beb40 0x55cb7a4dac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.601+0000 7f9f63544740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb7a4c8370 0x55cb7a4db180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.601+0000 7f9f63544740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c1adbc0 con 0x55cb7a4beb40 2024-09-12T03:14:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.601+0000 7f9f63544740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c1af030 con 0x55cb7a4c8370 2024-09-12T03:14:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.601+0000 7f9f63544740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c1ae290 con 0x55cb7a4bd5d0 2024-09-12T03:14:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.601+0000 7f9f63544740 10 client.0 _ll_register_callbacks cb 0x7f9f5000a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.601+0000 7f9f5a59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cb7a4beb40 0x55cb7a4dac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.601+0000 7f9f5ad9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb7a4bd5d0 0x55cb7a4dd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.601+0000 7f9f5a59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cb7a4beb40 0x55cb7a4dac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37846/0 (socket says 172.21.15.116:37846) 2024-09-12T03:14:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.601+0000 7f9f5a59c640 1 -- 172.21.15.116:0/2039703332 learned_addr learned my addr 172.21.15.116:0/2039703332 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.601+0000 7f9f5b59e640 1 --2- 172.21.15.116:0/2039703332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb7a4c8370 0x55cb7a4db180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.601+0000 7f9f5ad9d640 1 -- 172.21.15.116:0/2039703332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb7a4c8370 msgr2=0x55cb7a4db180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.601+0000 7f9f5ad9d640 1 --2- 172.21.15.116:0/2039703332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb7a4c8370 0x55cb7a4db180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.601+0000 7f9f5ad9d640 1 -- 172.21.15.116:0/2039703332 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cb7a4beb40 msgr2=0x55cb7a4dac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.601+0000 7f9f5ad9d640 1 --2- 172.21.15.116:0/2039703332 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cb7a4beb40 0x55cb7a4dac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.601+0000 7f9f5ad9d640 1 -- 172.21.15.116:0/2039703332 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb7a4db7f0 con 0x55cb7a4bd5d0 2024-09-12T03:14:21.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.601+0000 7f9f5a59c640 1 --2- 172.21.15.116:0/2039703332 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cb7a4beb40 0x55cb7a4dac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:14:21.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.602+0000 7f9f5ad9d640 1 --2- 172.21.15.116:0/2039703332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb7a4bd5d0 0x55cb7a4dd850 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00d740 tx=0x7f9f4c09ad70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee44c0f18e9f20d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:21.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.602+0000 7f9f3eff5640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T03:14:21.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.602+0000 7f9f3eff5640 1 -- 172.21.15.116:0/2039703332 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c0ad070 con 0x55cb7a4bd5d0 2024-09-12T03:14:21.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.602+0000 7f9f3eff5640 1 -- 172.21.15.116:0/2039703332 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f9f4c0af310 con 0x55cb7a4bd5d0 2024-09-12T03:14:21.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.602+0000 7f9f63544740 1 -- 172.21.15.116:0/2039703332 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f4c09f5e0 con 0x55cb7a4bd5d0 2024-09-12T03:14:21.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.602+0000 7f9f3eff5640 1 -- 172.21.15.116:0/2039703332 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9f4c0b3050 con 0x55cb7a4bd5d0 2024-09-12T03:14:21.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.602+0000 7f9f63544740 10 client.27043 fetch_fsmap learned FSMap version 25 2024-09-12T03:14:21.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.602+0000 7f9f63544740 1 -- 172.21.15.116:0/2039703332 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cb7a4db7f0 con 0x55cb7a4bd5d0 2024-09-12T03:14:21.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.602+0000 7f9f3eff5640 1 -- 172.21.15.116:0/2039703332 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f4c07d020 con 0x55cb7a4bd5d0 2024-09-12T03:14:21.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.602+0000 7f9f3eff5640 1 -- 172.21.15.116:0/2039703332 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9f4c005220 con 0x55cb7a4bd5d0 2024-09-12T03:14:21.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.602+0000 7f9f63544740 10 client.27043 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:14:21.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.602+0000 7f9f63544740 10 client.27043 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:14:21.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.602+0000 7f9f63544740 20 client.27043 populate_metadata read hostname 'smithi116' 2024-09-12T03:14:21.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.602+0000 7f9f63544740 1 --2- 172.21.15.116:0/2039703332 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55cb7a542970 0x55cb7a544d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:21.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.602+0000 7f9f63544740 4 client.27043 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:14:21.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.602+0000 7f9f63544740 1 -- 172.21.15.116:0/2039703332 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cb7a42ad70 con 0x55cb7a542970 2024-09-12T03:14:21.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.602+0000 7f9f5a59c640 1 --2- 172.21.15.116:0/2039703332 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55cb7a542970 0x55cb7a544d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:21.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.603+0000 7f9f5a59c640 1 --2- 172.21.15.116:0/2039703332 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55cb7a542970 0x55cb7a544d60 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6106a013efd5ebe7 server_cookie=e8e240759866837b in_seq=0 out_seq=0 2024-09-12T03:14:21.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.603+0000 7f9f3eff5640 10 client.27043 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:21.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.603+0000 7f9f3eff5640 1 -- 172.21.15.116:0/2039703332 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cb7a42ad70 con 0x55cb7a542970 2024-09-12T03:14:21.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.603+0000 7f9f3eff5640 10 client.27043 handle_command_reply: tid=0 2024-09-12T03:14:21.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.603+0000 7f9f63544740 1 client.27043 shutdown 2024-09-12T03:14:21.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.603+0000 7f9f63544740 1 -- 172.21.15.116:0/2039703332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb7a4bd5d0 msgr2=0x55cb7a4dd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:21.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.603+0000 7f9f63544740 1 --2- 172.21.15.116:0/2039703332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb7a4bd5d0 0x55cb7a4dd850 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00d740 tx=0x7f9f4c09ad70 comp rx=0 tx=0).stop 2024-09-12T03:14:21.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.603+0000 7f9f63544740 1 -- 172.21.15.116:0/2039703332 shutdown_connections 2024-09-12T03:14:21.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.603+0000 7f9f63544740 1 --2- 172.21.15.116:0/2039703332 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55cb7a542970 0x55cb7a544d60 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.603+0000 7f9f63544740 1 --2- 172.21.15.116:0/2039703332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb7a4c8370 0x55cb7a4db180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.603+0000 7f9f63544740 1 --2- 172.21.15.116:0/2039703332 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cb7a4beb40 0x55cb7a4dac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.603+0000 7f9f63544740 1 --2- 172.21.15.116:0/2039703332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cb7a4bd5d0 0x55cb7a4dd850 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.603+0000 7f9f3eff5640 0 client.27043 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:21.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.603+0000 7f9f63544740 1 -- 172.21.15.116:0/2039703332 >> 172.21.15.116:0/2039703332 conn(0x55cb7a486310 msgr2=0x55cb7a4d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:21.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.603+0000 7f9f63544740 1 -- 172.21.15.116:0/2039703332 shutdown_connections 2024-09-12T03:14:21.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.603+0000 7f9f63544740 1 -- 172.21.15.116:0/2039703332 wait complete. 2024-09-12T03:14:21.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.603+0000 7f9f63544740 20 client.27043 trim_cache size 0 max 16384 2024-09-12T03:14:21.643 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:14:21.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.604+0000 7f9f6116b640 1 -- 172.21.15.116:0/2090715611 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9f440a2b80 msgr2=0x7f9f440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:21.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.604+0000 7f9f6116b640 1 --2- 172.21.15.116:0/2090715611 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9f440a2b80 0x7f9f440a5040 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f9f54004ab0 tx=0x7f9f54004a40 comp rx=0 tx=0).stop 2024-09-12T03:14:21.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.604+0000 7f9f6116b640 1 -- 172.21.15.116:0/2090715611 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f5c102d30 msgr2=0x7f9f5c100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:21.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.604+0000 7f9f6116b640 1 --2- 172.21.15.116:0/2090715611 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f5c102d30 0x7f9f5c100990 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00d770 tx=0x7f9f4c00db70 comp rx=0 tx=0).stop 2024-09-12T03:14:21.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.604+0000 7f9f6116b640 1 -- 172.21.15.116:0/2090715611 shutdown_connections 2024-09-12T03:14:21.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.604+0000 7f9f6116b640 1 --2- 172.21.15.116:0/2090715611 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9f440a2b80 0x7f9f440a5040 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.604+0000 7f9f6116b640 1 --2- 172.21.15.116:0/2090715611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f5c1082a0 0x7f9f5c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.604+0000 7f9f6116b640 1 --2- 172.21.15.116:0/2090715611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f5c1036e0 0x7f9f5c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.604+0000 7f9f6116b640 1 --2- 172.21.15.116:0/2090715611 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f5c102d30 0x7f9f5c100990 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:21.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.604+0000 7f9f6116b640 1 -- 172.21.15.116:0/2090715611 >> 172.21.15.116:0/2090715611 conn(0x7f9f5c0fc7d0 msgr2=0x7f9f5c106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:21.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.604+0000 7f9f6116b640 1 -- 172.21.15.116:0/2090715611 shutdown_connections 2024-09-12T03:14:21.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:21.604+0000 7f9f6116b640 1 -- 172.21.15.116:0/2090715611 wait complete. 2024-09-12T03:14:22.296 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:22 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:14:22.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:22 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:14:22.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:22 smithi116 ceph-mon[31410]: pgmap v679: 97 pgs: 97 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:14:22.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:22 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:22.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:22 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:22.297 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:22 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:14:22.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:22 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:14:22.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:22 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:14:22.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:22 smithi183 ceph-mon[35535]: pgmap v679: 97 pgs: 97 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:14:22.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:22 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:22.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:22 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:22.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:22 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:14:22.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:22 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:14:22.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:22 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:14:22.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:22 smithi177 ceph-mon[35305]: pgmap v679: 97 pgs: 97 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:14:22.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:22 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:22.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:22 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:22.496 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:22 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:14:23.325 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:14:23.326 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:14:23.586 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:14:23.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:23 smithi183 ceph-mon[35535]: pgmap v680: 97 pgs: 97 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:14:23.706 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:23 smithi116 ceph-mon[31410]: pgmap v680: 97 pgs: 97 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:14:23.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:23 smithi177 ceph-mon[35305]: pgmap v680: 97 pgs: 97 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:14:24.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.092+0000 7fd2a81be640 1 -- 172.21.15.116:0/799918863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2a0100b30 msgr2=0x7fd2a0100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:24.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.092+0000 7fd2a81be640 1 --2- 172.21.15.116:0/799918863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2a0100b30 0x7fd2a0100f10 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009ca0 tx=0x7fd29c02a940 comp rx=0 tx=0).stop 2024-09-12T03:14:24.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.092+0000 7fd2a81be640 1 -- 172.21.15.116:0/799918863 shutdown_connections 2024-09-12T03:14:24.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.092+0000 7fd2a81be640 1 --2- 172.21.15.116:0/799918863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2a0105f40 0x7fd2a01063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.092+0000 7fd2a81be640 1 --2- 172.21.15.116:0/799918863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2a0101450 0x7fd2a0105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.092+0000 7fd2a81be640 1 --2- 172.21.15.116:0/799918863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2a0100b30 0x7fd2a0100f10 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.092+0000 7fd2a81be640 1 -- 172.21.15.116:0/799918863 >> 172.21.15.116:0/799918863 conn(0x7fd2a00fc7a0 msgr2=0x7fd2a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:24.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.092+0000 7fd2a81be640 1 -- 172.21.15.116:0/799918863 shutdown_connections 2024-09-12T03:14:24.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.092+0000 7fd2a81be640 1 -- 172.21.15.116:0/799918863 wait complete. 2024-09-12T03:14:24.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.093+0000 7fd2a81be640 1 Processor -- start 2024-09-12T03:14:24.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.093+0000 7fd2a81be640 1 -- start start 2024-09-12T03:14:24.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.093+0000 7fd2a81be640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2a0100b30 0x7fd2a01a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:24.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.093+0000 7fd2a81be640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2a0101450 0x7fd2a01a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:24.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.093+0000 7fd2a81be640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2a0105f40 0x7fd2a019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:24.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.093+0000 7fd2a81be640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a0118570 con 0x7fd2a0101450 2024-09-12T03:14:24.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.093+0000 7fd2a81be640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a01183f0 con 0x7fd2a0105f40 2024-09-12T03:14:24.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.093+0000 7fd2a81be640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a01186f0 con 0x7fd2a0100b30 2024-09-12T03:14:24.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.093+0000 7fd2a5759640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2a0101450 0x7fd2a01a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:24.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.093+0000 7fd2a5759640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2a0101450 0x7fd2a01a4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37852/0 (socket says 172.21.15.116:37852) 2024-09-12T03:14:24.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.093+0000 7fd2a5759640 1 -- 172.21.15.116:0/2005650642 learned_addr learned my addr 172.21.15.116:0/2005650642 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:24.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.093+0000 7fd2a5f5a640 1 --2- 172.21.15.116:0/2005650642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2a0100b30 0x7fd2a01a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:24.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.093+0000 7fd2a5759640 1 -- 172.21.15.116:0/2005650642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2a0100b30 msgr2=0x7fd2a01a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:24.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.093+0000 7fd2a5759640 1 --2- 172.21.15.116:0/2005650642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2a0100b30 0x7fd2a01a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.094+0000 7fd2a5759640 1 -- 172.21.15.116:0/2005650642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2a0105f40 msgr2=0x7fd2a019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:24.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.094+0000 7fd2a675b640 1 --2- 172.21.15.116:0/2005650642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2a0105f40 0x7fd2a019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:24.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.094+0000 7fd2a5759640 1 --2- 172.21.15.116:0/2005650642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2a0105f40 0x7fd2a019e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.094+0000 7fd2a5759640 1 -- 172.21.15.116:0/2005650642 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a019ef60 con 0x7fd2a0101450 2024-09-12T03:14:24.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.094+0000 7fd2a675b640 1 --2- 172.21.15.116:0/2005650642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2a0105f40 0x7fd2a019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:14:24.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.094+0000 7fd2a5759640 1 --2- 172.21.15.116:0/2005650642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2a0101450 0x7fd2a01a4b80 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fd29000c970 tx=0x7fd29000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:24.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.094+0000 7fd296ffd640 1 -- 172.21.15.116:0/2005650642 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd290007bb0 con 0x7fd2a0101450 2024-09-12T03:14:24.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.094+0000 7fd296ffd640 1 -- 172.21.15.116:0/2005650642 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd290007d50 con 0x7fd2a0101450 2024-09-12T03:14:24.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.094+0000 7fd2a81be640 1 -- 172.21.15.116:0/2005650642 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a019f250 con 0x7fd2a0101450 2024-09-12T03:14:24.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.094+0000 7fd296ffd640 1 -- 172.21.15.116:0/2005650642 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd290018820 con 0x7fd2a0101450 2024-09-12T03:14:24.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.094+0000 7fd2a81be640 1 -- 172.21.15.116:0/2005650642 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a01ab4a0 con 0x7fd2a0101450 2024-09-12T03:14:24.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.097+0000 7fd296ffd640 1 -- 172.21.15.116:0/2005650642 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd2900189c0 con 0x7fd2a0101450 2024-09-12T03:14:24.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.097+0000 7fd2aa597740 1 Processor -- start 2024-09-12T03:14:24.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.097+0000 7fd2aa597740 1 -- start start 2024-09-12T03:14:24.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.097+0000 7fd2aa597740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c6e991b60 0x557c6e991f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:24.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.097+0000 7fd2aa597740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c6e992480 0x557c6e99c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:24.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.097+0000 7fd2aa597740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557c6e99cd40 0x557c6e99f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:24.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.097+0000 7fd2aa597740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557c6e8f7280 con 0x557c6e99cd40 2024-09-12T03:14:24.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.097+0000 7fd2aa597740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557c6e89eb10 con 0x557c6e992480 2024-09-12T03:14:24.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.097+0000 7fd2aa597740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557c6e8f3240 con 0x557c6e991b60 2024-09-12T03:14:24.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.098+0000 7fd2a5759640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557c6e99cd40 0x557c6e99f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:24.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.098+0000 7fd2a5759640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557c6e99cd40 0x557c6e99f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37866/0 (socket says 172.21.15.116:37866) 2024-09-12T03:14:24.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.098+0000 7fd2a5759640 1 -- 172.21.15.116:0/3172342979 learned_addr learned my addr 172.21.15.116:0/3172342979 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:24.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.098+0000 7fd2a5f5a640 1 --2- 172.21.15.116:0/3172342979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c6e992480 0x557c6e99c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:24.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.098+0000 7fd2a675b640 1 --2- 172.21.15.116:0/3172342979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c6e991b60 0x557c6e991f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:24.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.098+0000 7fd296ffd640 1 --2- 172.21.15.116:0/2005650642 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd28c0a2b80 0x7fd28c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:24.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.098+0000 7fd2a675b640 1 --2- 172.21.15.116:0/2005650642 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd28c0a2b80 0x7fd28c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:24.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.098+0000 7fd2a5759640 1 -- 172.21.15.116:0/3172342979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c6e991b60 msgr2=0x557c6e991f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:24.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.098+0000 7fd2a5759640 1 --2- 172.21.15.116:0/3172342979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c6e991b60 0x557c6e991f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.098+0000 7fd2a5759640 1 -- 172.21.15.116:0/3172342979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c6e992480 msgr2=0x557c6e99c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:24.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.098+0000 7fd2a5759640 1 --2- 172.21.15.116:0/3172342979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c6e992480 0x557c6e99c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.098+0000 7fd296ffd640 1 -- 172.21.15.116:0/2005650642 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd2900cf0a0 con 0x7fd2a0101450 2024-09-12T03:14:24.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.098+0000 7fd2a5759640 1 -- 172.21.15.116:0/3172342979 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c6e8ab1f0 con 0x557c6e99cd40 2024-09-12T03:14:24.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.098+0000 7fd2a5759640 1 --2- 172.21.15.116:0/3172342979 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557c6e99cd40 0x557c6e99f130 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fd2900ccbd0 tx=0x7fd290051040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16da8f1a729d19e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:24.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.098+0000 7fd2a675b640 1 --2- 172.21.15.116:0/2005650642 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd28c0a2b80 0x7fd28c0a5040 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd2980058c0 tx=0x7fd298004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:24.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.098+0000 7fd28bfff640 1 -- 172.21.15.116:0/3172342979 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29009b470 con 0x557c6e99cd40 2024-09-12T03:14:24.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.098+0000 7fd28bfff640 1 -- 172.21.15.116:0/3172342979 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd29009c030 con 0x557c6e99cd40 2024-09-12T03:14:24.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.099+0000 7fd28bfff640 1 -- 172.21.15.116:0/3172342979 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd290018cb0 con 0x557c6e99cd40 2024-09-12T03:14:24.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.099+0000 7fd2aa597740 1 -- 172.21.15.116:0/3172342979 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557c6e99cd40 msgr2=0x557c6e99f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:24.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.099+0000 7fd2aa597740 1 --2- 172.21.15.116:0/3172342979 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557c6e99cd40 0x557c6e99f130 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fd2900ccbd0 tx=0x7fd290051040 comp rx=0 tx=0).stop 2024-09-12T03:14:24.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.099+0000 7fd2aa597740 1 -- 172.21.15.116:0/3172342979 shutdown_connections 2024-09-12T03:14:24.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.099+0000 7fd2aa597740 1 --2- 172.21.15.116:0/3172342979 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557c6e99cd40 0x557c6e99f130 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.099+0000 7fd2aa597740 1 --2- 172.21.15.116:0/3172342979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c6e992480 0x557c6e99c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.099+0000 7fd2aa597740 1 --2- 172.21.15.116:0/3172342979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c6e991b60 0x557c6e991f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.099+0000 7fd2aa597740 1 -- 172.21.15.116:0/3172342979 >> 172.21.15.116:0/3172342979 conn(0x557c6e98e0f0 msgr2=0x557c6e98e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:24.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.099+0000 7fd2aa597740 1 -- 172.21.15.116:0/3172342979 shutdown_connections 2024-09-12T03:14:24.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.099+0000 7fd2aa597740 1 -- 172.21.15.116:0/3172342979 wait complete. 2024-09-12T03:14:24.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.100+0000 7fd2aa597740 1 -- start start 2024-09-12T03:14:24.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.100+0000 7fd2aa597740 1 Processor -- start 2024-09-12T03:14:24.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.100+0000 7fd2aa597740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c6e991b60 0x557c6e999bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:24.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.100+0000 7fd2aa597740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557c6e992480 0x557c6e99a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:24.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.100+0000 7fd2aa597740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c6e99cd40 0x557c6e99a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:24.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.100+0000 7fd2aa597740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557c6e99f8b0 con 0x557c6e992480 2024-09-12T03:14:24.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.100+0000 7fd2aa597740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557c6e8b1f10 con 0x557c6e99cd40 2024-09-12T03:14:24.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.100+0000 7fd2aa597740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557c6e99fa30 con 0x557c6e991b60 2024-09-12T03:14:24.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.100+0000 7fd2aa597740 10 client.0 _ll_register_callbacks cb 0x557c6e87aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:24.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.100+0000 7fd2a5759640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557c6e992480 0x557c6e99a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:24.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.100+0000 7fd2a5759640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557c6e992480 0x557c6e99a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37876/0 (socket says 172.21.15.116:37876) 2024-09-12T03:14:24.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.100+0000 7fd2a5759640 1 -- 172.21.15.116:0/1584897137 learned_addr learned my addr 172.21.15.116:0/1584897137 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:24.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.101+0000 7fd2a675b640 1 --2- 172.21.15.116:0/1584897137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c6e99cd40 0x557c6e99a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:24.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.101+0000 7fd2a5759640 1 -- 172.21.15.116:0/1584897137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c6e991b60 msgr2=0x557c6e999bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:24.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.101+0000 7fd2a5759640 1 --2- 172.21.15.116:0/1584897137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c6e991b60 0x557c6e999bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.101+0000 7fd2a5759640 1 -- 172.21.15.116:0/1584897137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c6e99cd40 msgr2=0x557c6e99a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:24.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.101+0000 7fd2a5759640 1 --2- 172.21.15.116:0/1584897137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c6e99cd40 0x557c6e99a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.101+0000 7fd2a5759640 1 -- 172.21.15.116:0/1584897137 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c6e6686a0 con 0x557c6e992480 2024-09-12T03:14:24.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.101+0000 7fd2a675b640 1 --2- 172.21.15.116:0/1584897137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c6e99cd40 0x557c6e99a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:14:24.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.101+0000 7fd2a5759640 1 --2- 172.21.15.116:0/1584897137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557c6e992480 0x557c6e99a0f0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fd29009b630 tx=0x7fd290099b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76b3b1e44422997 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:24.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.101+0000 7fd2897fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:14:24.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.101+0000 7fd2897fa640 1 -- 172.21.15.116:0/1584897137 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29009b290 con 0x557c6e992480 2024-09-12T03:14:24.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.101+0000 7fd2897fa640 1 -- 172.21.15.116:0/1584897137 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd290086110 con 0x557c6e992480 2024-09-12T03:14:24.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.101+0000 7fd2aa597740 1 -- 172.21.15.116:0/1584897137 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd29009c030 con 0x557c6e992480 2024-09-12T03:14:24.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.101+0000 7fd2897fa640 1 -- 172.21.15.116:0/1584897137 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd2900b9050 con 0x557c6e992480 2024-09-12T03:14:24.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.101+0000 7fd2aa597740 10 client.18582 fetch_fsmap learned FSMap version 25 2024-09-12T03:14:24.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.101+0000 7fd2aa597740 1 -- 172.21.15.116:0/1584897137 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557c6e9ab440 con 0x557c6e992480 2024-09-12T03:14:24.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.101+0000 7fd2897fa640 1 -- 172.21.15.116:0/1584897137 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd29009c030 con 0x557c6e992480 2024-09-12T03:14:24.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.101+0000 7fd2897fa640 1 -- 172.21.15.116:0/1584897137 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd2900a4110 con 0x557c6e992480 2024-09-12T03:14:24.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.101+0000 7fd2aa597740 10 client.18582 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:14:24.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.101+0000 7fd2aa597740 10 client.18582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:14:24.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.101+0000 7fd2aa597740 20 client.18582 populate_metadata read hostname 'smithi116' 2024-09-12T03:14:24.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.101+0000 7fd2aa597740 1 --2- 172.21.15.116:0/1584897137 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557c6e9ac790 0x557c6e9aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:24.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.101+0000 7fd2aa597740 4 client.18582 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:14:24.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.101+0000 7fd2aa597740 1 -- 172.21.15.116:0/1584897137 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c6e9047f0 con 0x557c6e9ac790 2024-09-12T03:14:24.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.102+0000 7fd2a5f5a640 1 --2- 172.21.15.116:0/1584897137 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557c6e9ac790 0x557c6e9aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:24.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.102+0000 7fd2897fa640 10 client.18582 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:24.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.102+0000 7fd2a5f5a640 1 --2- 172.21.15.116:0/1584897137 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557c6e9ac790 0x557c6e9aeb80 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b247430e001036f server_cookie=fe9a522d10ace0f3 in_seq=0 out_seq=0 2024-09-12T03:14:24.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.103+0000 7fd2897fa640 1 -- 172.21.15.116:0/1584897137 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557c6e9047f0 con 0x557c6e9ac790 2024-09-12T03:14:24.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.103+0000 7fd2897fa640 10 client.18582 handle_command_reply: tid=0 2024-09-12T03:14:24.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.103+0000 7fd2aa597740 1 client.18582 shutdown 2024-09-12T03:14:24.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.103+0000 7fd2aa597740 1 -- 172.21.15.116:0/1584897137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557c6e992480 msgr2=0x557c6e99a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:24.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.103+0000 7fd2aa597740 1 --2- 172.21.15.116:0/1584897137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557c6e992480 0x557c6e99a0f0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fd29009b630 tx=0x7fd290099b20 comp rx=0 tx=0).stop 2024-09-12T03:14:24.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.103+0000 7fd2aa597740 1 -- 172.21.15.116:0/1584897137 shutdown_connections 2024-09-12T03:14:24.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.103+0000 7fd2aa597740 1 --2- 172.21.15.116:0/1584897137 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557c6e9ac790 0x557c6e9aeb80 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.103+0000 7fd2aa597740 1 --2- 172.21.15.116:0/1584897137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c6e99cd40 0x557c6e99a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.103+0000 7fd2aa597740 1 --2- 172.21.15.116:0/1584897137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557c6e992480 0x557c6e99a0f0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.103+0000 7fd2897fa640 0 client.18582 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:24.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.103+0000 7fd2aa597740 1 --2- 172.21.15.116:0/1584897137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c6e991b60 0x557c6e999bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.103+0000 7fd2aa597740 1 -- 172.21.15.116:0/1584897137 >> 172.21.15.116:0/1584897137 conn(0x557c6e98e0f0 msgr2=0x557c6e993e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:24.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.103+0000 7fd2aa597740 1 -- 172.21.15.116:0/1584897137 shutdown_connections 2024-09-12T03:14:24.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.103+0000 7fd2aa597740 1 -- 172.21.15.116:0/1584897137 wait complete. 2024-09-12T03:14:24.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.103+0000 7fd2aa597740 20 client.18582 trim_cache size 0 max 16384 2024-09-12T03:14:24.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.120+0000 7fd2aa597740 1 Processor -- start 2024-09-12T03:14:24.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.120+0000 7fd2aa597740 1 -- start start 2024-09-12T03:14:24.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.120+0000 7fd2aa597740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c6e991b60 0x557c6e993cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:24.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.120+0000 7fd2aa597740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c6e992480 0x557c6e9b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:24.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.120+0000 7fd2aa597740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557c6e99cd40 0x557c6e9b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:24.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.120+0000 7fd2aa597740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a01b6350 con 0x557c6e99cd40 2024-09-12T03:14:24.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.120+0000 7fd2aa597740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a01b3c20 con 0x557c6e992480 2024-09-12T03:14:24.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.120+0000 7fd2aa597740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a01b1a70 con 0x557c6e991b60 2024-09-12T03:14:24.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.120+0000 7fd2a675b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557c6e99cd40 0x557c6e9b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:24.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.121+0000 7fd2a675b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557c6e99cd40 0x557c6e9b20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37878/0 (socket says 172.21.15.116:37878) 2024-09-12T03:14:24.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.121+0000 7fd2a675b640 1 -- 172.21.15.116:0/1201803719 learned_addr learned my addr 172.21.15.116:0/1201803719 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:24.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.121+0000 7fd2a5f5a640 1 --2- 172.21.15.116:0/1201803719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c6e991b60 0x557c6e993cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:24.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.121+0000 7fd2a5759640 1 --2- 172.21.15.116:0/1201803719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c6e992480 0x557c6e9b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:24.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.121+0000 7fd2a675b640 1 -- 172.21.15.116:0/1201803719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c6e991b60 msgr2=0x557c6e993cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:24.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.121+0000 7fd2a675b640 1 --2- 172.21.15.116:0/1201803719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c6e991b60 0x557c6e993cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.121+0000 7fd2a675b640 1 -- 172.21.15.116:0/1201803719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c6e992480 msgr2=0x557c6e9b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:24.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.121+0000 7fd2a675b640 1 --2- 172.21.15.116:0/1201803719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c6e992480 0x557c6e9b1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.121+0000 7fd2a675b640 1 -- 172.21.15.116:0/1201803719 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c6e9922b0 con 0x557c6e99cd40 2024-09-12T03:14:24.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.121+0000 7fd2a5759640 1 --2- 172.21.15.116:0/1201803719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c6e992480 0x557c6e9b1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:14:24.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.121+0000 7fd2a675b640 1 --2- 172.21.15.116:0/1201803719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557c6e99cd40 0x557c6e9b20e0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fd298013d20 tx=0x7fd298009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b92f39e0b5cfde server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:24.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.121+0000 7fd28b7fe640 1 -- 172.21.15.116:0/1201803719 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29801f410 con 0x557c6e99cd40 2024-09-12T03:14:24.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.121+0000 7fd28b7fe640 1 -- 172.21.15.116:0/1201803719 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd29801fa90 con 0x557c6e99cd40 2024-09-12T03:14:24.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.121+0000 7fd28b7fe640 1 -- 172.21.15.116:0/1201803719 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298027b00 con 0x557c6e99cd40 2024-09-12T03:14:24.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.122+0000 7fd2aa597740 1 -- 172.21.15.116:0/1201803719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557c6e99cd40 msgr2=0x557c6e9b20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:24.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.122+0000 7fd2aa597740 1 --2- 172.21.15.116:0/1201803719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557c6e99cd40 0x557c6e9b20e0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fd298013d20 tx=0x7fd298009360 comp rx=0 tx=0).stop 2024-09-12T03:14:24.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.122+0000 7fd2aa597740 1 -- 172.21.15.116:0/1201803719 shutdown_connections 2024-09-12T03:14:24.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.122+0000 7fd2aa597740 1 --2- 172.21.15.116:0/1201803719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557c6e99cd40 0x557c6e9b20e0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.122+0000 7fd2aa597740 1 --2- 172.21.15.116:0/1201803719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c6e992480 0x557c6e9b1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.122+0000 7fd2aa597740 1 --2- 172.21.15.116:0/1201803719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c6e991b60 0x557c6e993cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.122+0000 7fd2aa597740 1 -- 172.21.15.116:0/1201803719 >> 172.21.15.116:0/1201803719 conn(0x557c6e98e0f0 msgr2=0x557c6e9935b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:24.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.122+0000 7fd2aa597740 1 -- 172.21.15.116:0/1201803719 shutdown_connections 2024-09-12T03:14:24.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.122+0000 7fd2aa597740 1 -- 172.21.15.116:0/1201803719 wait complete. 2024-09-12T03:14:24.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.122+0000 7fd2aa597740 1 -- start start 2024-09-12T03:14:24.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.122+0000 7fd2aa597740 1 Processor -- start 2024-09-12T03:14:24.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.122+0000 7fd2aa597740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c6e991b60 0x557c6e9afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:24.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.123+0000 7fd2aa597740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c6e992480 0x557c6e9affe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:24.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.123+0000 7fd2aa597740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557c6e99cd40 0x557c6e9b0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:24.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.123+0000 7fd2aa597740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a01b20e0 con 0x557c6e99cd40 2024-09-12T03:14:24.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.123+0000 7fd2aa597740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a01b3550 con 0x557c6e991b60 2024-09-12T03:14:24.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.123+0000 7fd2aa597740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a01b27b0 con 0x557c6e992480 2024-09-12T03:14:24.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.123+0000 7fd2aa597740 10 client.0 _ll_register_callbacks cb 0x7fd29c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:24.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.123+0000 7fd2a675b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557c6e99cd40 0x557c6e9b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:24.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.123+0000 7fd2a675b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557c6e99cd40 0x557c6e9b0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37882/0 (socket says 172.21.15.116:37882) 2024-09-12T03:14:24.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.123+0000 7fd2a675b640 1 -- 172.21.15.116:0/2144317468 learned_addr learned my addr 172.21.15.116:0/2144317468 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:24.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.123+0000 7fd2a5f5a640 1 --2- 172.21.15.116:0/2144317468 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c6e991b60 0x557c6e9afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:24.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.123+0000 7fd2a675b640 1 -- 172.21.15.116:0/2144317468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c6e992480 msgr2=0x557c6e9affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:24.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.123+0000 7fd2a675b640 1 --2- 172.21.15.116:0/2144317468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c6e992480 0x557c6e9affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.123+0000 7fd2a675b640 1 -- 172.21.15.116:0/2144317468 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c6e991b60 msgr2=0x557c6e9afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:24.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.123+0000 7fd2a675b640 1 --2- 172.21.15.116:0/2144317468 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c6e991b60 0x557c6e9afaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.123+0000 7fd2a675b640 1 -- 172.21.15.116:0/2144317468 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c6ea16270 con 0x557c6e99cd40 2024-09-12T03:14:24.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.123+0000 7fd2a675b640 1 --2- 172.21.15.116:0/2144317468 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557c6e99cd40 0x557c6e9b0520 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fd298013f80 tx=0x7fd298027400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1477512c79079302 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:24.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.123+0000 7fd2897fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:14:24.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.123+0000 7fd2897fa640 1 -- 172.21.15.116:0/2144317468 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29801f620 con 0x557c6e99cd40 2024-09-12T03:14:24.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.123+0000 7fd2897fa640 1 -- 172.21.15.116:0/2144317468 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fd29801f7c0 con 0x557c6e99cd40 2024-09-12T03:14:24.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.123+0000 7fd2aa597740 1 -- 172.21.15.116:0/2144317468 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd29801fa90 con 0x557c6e99cd40 2024-09-12T03:14:24.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.123+0000 7fd2897fa640 1 -- 172.21.15.116:0/2144317468 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd298039070 con 0x557c6e99cd40 2024-09-12T03:14:24.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.123+0000 7fd2aa597740 10 client.18588 fetch_fsmap learned FSMap version 25 2024-09-12T03:14:24.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.123+0000 7fd2aa597740 1 -- 172.21.15.116:0/2144317468 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557c6ea174c0 con 0x557c6e99cd40 2024-09-12T03:14:24.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.124+0000 7fd2897fa640 1 -- 172.21.15.116:0/2144317468 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd29801fa90 con 0x557c6e99cd40 2024-09-12T03:14:24.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.124+0000 7fd2897fa640 1 -- 172.21.15.116:0/2144317468 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd29801b070 con 0x557c6e99cd40 2024-09-12T03:14:24.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.124+0000 7fd2aa597740 10 client.18588 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:14:24.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.124+0000 7fd2aa597740 10 client.18588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:14:24.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.124+0000 7fd2aa597740 20 client.18588 populate_metadata read hostname 'smithi116' 2024-09-12T03:14:24.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.124+0000 7fd2aa597740 1 --2- 172.21.15.116:0/2144317468 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557c6ea17b30 0x557c6ea19f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:24.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.124+0000 7fd2aa597740 4 client.18588 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:14:24.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.124+0000 7fd2aa597740 1 -- 172.21.15.116:0/2144317468 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557c6e8fed00 con 0x557c6ea17b30 2024-09-12T03:14:24.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.124+0000 7fd2a5f5a640 1 --2- 172.21.15.116:0/2144317468 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557c6ea17b30 0x557c6ea19f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:24.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.125+0000 7fd2a5f5a640 1 --2- 172.21.15.116:0/2144317468 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557c6ea17b30 0x557c6ea19f50 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71c27351e33a08a3 server_cookie=e2e0074c3881bad2 in_seq=0 out_seq=0 2024-09-12T03:14:24.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.125+0000 7fd2897fa640 10 client.18588 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:24.209 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:14:24.209 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:14:24.209 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "3ab84fa7-814a-4fbc-89e1-1c2d5695ff62", 2024-09-12T03:14:24.209 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:14:24.209 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:14:24.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.126+0000 7fd2897fa640 1 -- 172.21.15.116:0/2144317468 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557c6e8fed00 con 0x557c6ea17b30 2024-09-12T03:14:24.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.126+0000 7fd2897fa640 10 client.18588 handle_command_reply: tid=0 2024-09-12T03:14:24.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.126+0000 7fd2aa597740 1 client.18588 shutdown 2024-09-12T03:14:24.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.126+0000 7fd2aa597740 1 -- 172.21.15.116:0/2144317468 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557c6e99cd40 msgr2=0x557c6e9b0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:24.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.126+0000 7fd2aa597740 1 --2- 172.21.15.116:0/2144317468 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557c6e99cd40 0x557c6e9b0520 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fd298013f80 tx=0x7fd298027400 comp rx=0 tx=0).stop 2024-09-12T03:14:24.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.126+0000 7fd2aa597740 1 -- 172.21.15.116:0/2144317468 shutdown_connections 2024-09-12T03:14:24.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.126+0000 7fd2aa597740 1 --2- 172.21.15.116:0/2144317468 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557c6ea17b30 0x557c6ea19f50 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.126+0000 7fd2aa597740 1 --2- 172.21.15.116:0/2144317468 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557c6e99cd40 0x557c6e9b0520 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.126+0000 7fd2aa597740 1 --2- 172.21.15.116:0/2144317468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c6e992480 0x557c6e9affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.126+0000 7fd2897fa640 0 client.18588 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:24.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.126+0000 7fd2aa597740 1 --2- 172.21.15.116:0/2144317468 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c6e991b60 0x557c6e9afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.126+0000 7fd2aa597740 1 -- 172.21.15.116:0/2144317468 >> 172.21.15.116:0/2144317468 conn(0x557c6e98e0f0 msgr2=0x557c6e993dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:24.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.126+0000 7fd2aa597740 1 -- 172.21.15.116:0/2144317468 shutdown_connections 2024-09-12T03:14:24.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.126+0000 7fd2aa597740 1 -- 172.21.15.116:0/2144317468 wait complete. 2024-09-12T03:14:24.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.126+0000 7fd2aa597740 20 client.18588 trim_cache size 0 max 16384 2024-09-12T03:14:24.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.126+0000 7fd2a81be640 1 -- 172.21.15.116:0/2005650642 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd28c0a2b80 msgr2=0x7fd28c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:24.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.126+0000 7fd2a81be640 1 --2- 172.21.15.116:0/2005650642 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd28c0a2b80 0x7fd28c0a5040 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd2980058c0 tx=0x7fd298004730 comp rx=0 tx=0).stop 2024-09-12T03:14:24.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.126+0000 7fd2a81be640 1 -- 172.21.15.116:0/2005650642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2a0101450 msgr2=0x7fd2a01a4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:24.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.126+0000 7fd2a81be640 1 --2- 172.21.15.116:0/2005650642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2a0101450 0x7fd2a01a4b80 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fd29000c970 tx=0x7fd29000ce30 comp rx=0 tx=0).stop 2024-09-12T03:14:24.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.127+0000 7fd2a81be640 1 -- 172.21.15.116:0/2005650642 shutdown_connections 2024-09-12T03:14:24.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.127+0000 7fd2a81be640 1 --2- 172.21.15.116:0/2005650642 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fd28c0a2b80 0x7fd28c0a5040 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.127+0000 7fd2a81be640 1 --2- 172.21.15.116:0/2005650642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2a0105f40 0x7fd2a019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.127+0000 7fd2a81be640 1 --2- 172.21.15.116:0/2005650642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2a0101450 0x7fd2a01a4b80 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.127+0000 7fd2a81be640 1 --2- 172.21.15.116:0/2005650642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2a0100b30 0x7fd2a01a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:24.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.127+0000 7fd2a81be640 1 -- 172.21.15.116:0/2005650642 >> 172.21.15.116:0/2005650642 conn(0x7fd2a00fc7a0 msgr2=0x7fd2a01038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:24.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.127+0000 7fd2a81be640 1 -- 172.21.15.116:0/2005650642 shutdown_connections 2024-09-12T03:14:24.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:24.127+0000 7fd2a81be640 1 -- 172.21.15.116:0/2005650642 wait complete. 2024-09-12T03:14:25.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:24 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:14:25.039 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:24 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:14:25.040 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:24 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:14:25.040 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:24 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:14:25.075 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:14:25.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:24 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:14:25.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:24 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:14:25.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:24 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:14:25.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:24 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:14:25.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:24 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:14:25.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:24 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:14:25.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:24 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:14:25.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:24 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:14:25.329 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:14:25.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.833+0000 7f3fc3e10640 1 -- 172.21.15.116:0/3860598291 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fbc0ffbe0 msgr2=0x7f3fbc1006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:25.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.833+0000 7f3fc3e10640 1 --2- 172.21.15.116:0/3860598291 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fbc0ffbe0 0x7f3fbc1006d0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8009900 tx=0x7f3fb802a410 comp rx=0 tx=0).stop 2024-09-12T03:14:25.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.833+0000 7f3fc3e10640 1 -- 172.21.15.116:0/3860598291 shutdown_connections 2024-09-12T03:14:25.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.833+0000 7f3fc3e10640 1 --2- 172.21.15.116:0/3860598291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fbc104e50 0x7f3fbc105300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.833+0000 7f3fc3e10640 1 --2- 172.21.15.116:0/3860598291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3fbc100050 0x7f3fbc100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.833+0000 7f3fc3e10640 1 --2- 172.21.15.116:0/3860598291 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fbc0ffbe0 0x7f3fbc1006d0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.833+0000 7f3fc3e10640 1 -- 172.21.15.116:0/3860598291 >> 172.21.15.116:0/3860598291 conn(0x7f3fbc0fba90 msgr2=0x7f3fbc0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:25.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.833+0000 7f3fc3e10640 1 -- 172.21.15.116:0/3860598291 shutdown_connections 2024-09-12T03:14:25.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.833+0000 7f3fc3e10640 1 -- 172.21.15.116:0/3860598291 wait complete. 2024-09-12T03:14:25.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.834+0000 7f3fc3e10640 1 Processor -- start 2024-09-12T03:14:25.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.834+0000 7f3fc3e10640 1 -- start start 2024-09-12T03:14:25.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.834+0000 7f3fc3e10640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fbc0ffbe0 0x7f3fbc1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:25.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.834+0000 7f3fc3e10640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fbc100050 0x7f3fbc1977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:25.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.834+0000 7f3fc3e10640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3fbc104e50 0x7f3fbc19bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:25.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.834+0000 7f3fc3e10640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc110e60 con 0x7f3fbc0ffbe0 2024-09-12T03:14:25.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.834+0000 7f3fc3e10640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc110ce0 con 0x7f3fbc104e50 2024-09-12T03:14:25.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.834+0000 7f3fc3e10640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc110fe0 con 0x7f3fbc100050 2024-09-12T03:14:25.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.834+0000 7f3fc1bac640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fbc0ffbe0 0x7f3fbc1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:25.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.834+0000 7f3fc1bac640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fbc0ffbe0 0x7f3fbc1972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37892/0 (socket says 172.21.15.116:37892) 2024-09-12T03:14:25.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.834+0000 7f3fc1bac640 1 -- 172.21.15.116:0/1056147645 learned_addr learned my addr 172.21.15.116:0/1056147645 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:25.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.835+0000 7f3fc23ad640 1 --2- 172.21.15.116:0/1056147645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3fbc104e50 0x7f3fbc19bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:25.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.835+0000 7f3fc13ab640 1 --2- 172.21.15.116:0/1056147645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fbc100050 0x7f3fbc1977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:25.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.835+0000 7f3fc1bac640 1 -- 172.21.15.116:0/1056147645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fbc100050 msgr2=0x7f3fbc1977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:25.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.835+0000 7f3fc1bac640 1 --2- 172.21.15.116:0/1056147645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fbc100050 0x7f3fbc1977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.835+0000 7f3fc1bac640 1 -- 172.21.15.116:0/1056147645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3fbc104e50 msgr2=0x7f3fbc19bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:25.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.835+0000 7f3fc1bac640 1 --2- 172.21.15.116:0/1056147645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3fbc104e50 0x7f3fbc19bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.835+0000 7f3fc1bac640 1 -- 172.21.15.116:0/1056147645 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fbc19c250 con 0x7f3fbc0ffbe0 2024-09-12T03:14:25.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.835+0000 7f3fc13ab640 1 --2- 172.21.15.116:0/1056147645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fbc100050 0x7f3fbc1977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:14:25.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.835+0000 7f3fc1bac640 1 --2- 172.21.15.116:0/1056147645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fbc0ffbe0 0x7f3fbc1972a0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8006010 tx=0x7f3fb80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:25.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.835+0000 7f3faeffd640 1 -- 172.21.15.116:0/1056147645 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb803f030 con 0x7f3fbc0ffbe0 2024-09-12T03:14:25.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.835+0000 7f3fc3e10640 1 -- 172.21.15.116:0/1056147645 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fbc19c4e0 con 0x7f3fbc0ffbe0 2024-09-12T03:14:25.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.835+0000 7f3faeffd640 1 -- 172.21.15.116:0/1056147645 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fb8003ce0 con 0x7f3fbc0ffbe0 2024-09-12T03:14:25.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.835+0000 7f3fc3e10640 1 -- 172.21.15.116:0/1056147645 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fbc1a3e20 con 0x7f3fbc0ffbe0 2024-09-12T03:14:25.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.835+0000 7f3faeffd640 1 -- 172.21.15.116:0/1056147645 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8049060 con 0x7f3fbc0ffbe0 2024-09-12T03:14:25.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.837+0000 7f3faeffd640 1 -- 172.21.15.116:0/1056147645 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3fb8042030 con 0x7f3fbc0ffbe0 2024-09-12T03:14:25.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.837+0000 7f3fc61e9740 1 Processor -- start 2024-09-12T03:14:25.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.838+0000 7f3fc61e9740 1 -- start start 2024-09-12T03:14:25.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.838+0000 7f3fc61e9740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d292b31310 0x55d292b316f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:25.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.838+0000 7f3fc61e9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d292b31c30 0x55d292b3e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:25.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.838+0000 7f3fc61e9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d292b3e880 0x55d292b40c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:25.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.838+0000 7f3fc61e9740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d292a972e0 con 0x55d292b31310 2024-09-12T03:14:25.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.838+0000 7f3fc61e9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d292a3ece0 con 0x55d292b3e880 2024-09-12T03:14:25.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.838+0000 7f3fc61e9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d292a932a0 con 0x55d292b31c30 2024-09-12T03:14:25.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.838+0000 7f3fc23ad640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d292b31310 0x55d292b316f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:25.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.838+0000 7f3fc23ad640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d292b31310 0x55d292b316f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37904/0 (socket says 172.21.15.116:37904) 2024-09-12T03:14:25.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.838+0000 7f3fc23ad640 1 -- 172.21.15.116:0/960629279 learned_addr learned my addr 172.21.15.116:0/960629279 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:25.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.838+0000 7f3faeffd640 1 --2- 172.21.15.116:0/1056147645 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3fa80a2b80 0x7f3fa80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:25.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.838+0000 7f3fc13ab640 1 --2- 172.21.15.116:0/960629279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d292b3e880 0x55d292b40c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:25.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.838+0000 7f3fc1bac640 1 --2- 172.21.15.116:0/960629279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d292b31c30 0x55d292b3e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:25.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.838+0000 7f3fc23ad640 1 -- 172.21.15.116:0/960629279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d292b31c30 msgr2=0x55d292b3e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:25.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.838+0000 7f3fc23ad640 1 --2- 172.21.15.116:0/960629279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d292b31c30 0x55d292b3e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.838+0000 7f3fc23ad640 1 -- 172.21.15.116:0/960629279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d292b3e880 msgr2=0x55d292b40c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:25.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.838+0000 7f3fc23ad640 1 --2- 172.21.15.116:0/960629279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d292b3e880 0x55d292b40c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.838+0000 7f3fc23ad640 1 -- 172.21.15.116:0/960629279 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d292a23440 con 0x55d292b31310 2024-09-12T03:14:25.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.838+0000 7f3fc1bac640 1 --2- 172.21.15.116:0/960629279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d292b31c30 0x55d292b3e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:14:25.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.838+0000 7f3faeffd640 1 -- 172.21.15.116:0/1056147645 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f3fb80ef820 con 0x7f3fbc0ffbe0 2024-09-12T03:14:25.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.839+0000 7f3fc23ad640 1 --2- 172.21.15.116:0/1056147645 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3fa80a2b80 0x7f3fa80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:25.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.839+0000 7f3fc23ad640 1 --2- 172.21.15.116:0/960629279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d292b31310 0x55d292b316f0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4011590 tx=0x7f3fb4011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb0db060936c743 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:25.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.839+0000 7f3fa7fff640 1 -- 172.21.15.116:0/960629279 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb401f020 con 0x55d292b31310 2024-09-12T03:14:25.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.839+0000 7f3fa7fff640 1 -- 172.21.15.116:0/960629279 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fb401be20 con 0x55d292b31310 2024-09-12T03:14:25.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.839+0000 7f3fa7fff640 1 -- 172.21.15.116:0/960629279 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4017050 con 0x55d292b31310 2024-09-12T03:14:25.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.840+0000 7f3fc61e9740 1 -- 172.21.15.116:0/960629279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d292b31310 msgr2=0x55d292b316f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:25.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.840+0000 7f3fc23ad640 1 --2- 172.21.15.116:0/1056147645 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3fa80a2b80 0x7f3fa80a5040 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4020000 tx=0x7f3fb4021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:25.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.840+0000 7f3fc61e9740 1 --2- 172.21.15.116:0/960629279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d292b31310 0x55d292b316f0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4011590 tx=0x7f3fb4011a50 comp rx=0 tx=0).stop 2024-09-12T03:14:25.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.840+0000 7f3fc61e9740 1 -- 172.21.15.116:0/960629279 shutdown_connections 2024-09-12T03:14:25.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.840+0000 7f3fc61e9740 1 --2- 172.21.15.116:0/960629279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d292b3e880 0x55d292b40c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.840+0000 7f3fc61e9740 1 --2- 172.21.15.116:0/960629279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d292b31c30 0x55d292b3e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.840+0000 7f3fc61e9740 1 --2- 172.21.15.116:0/960629279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d292b31310 0x55d292b316f0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.840+0000 7f3fc61e9740 1 -- 172.21.15.116:0/960629279 >> 172.21.15.116:0/960629279 conn(0x55d292b2da20 msgr2=0x55d292b2de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:25.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.840+0000 7f3fc61e9740 1 -- 172.21.15.116:0/960629279 shutdown_connections 2024-09-12T03:14:25.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.840+0000 7f3fc61e9740 1 -- 172.21.15.116:0/960629279 wait complete. 2024-09-12T03:14:25.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.840+0000 7f3fc61e9740 1 -- start start 2024-09-12T03:14:25.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.840+0000 7f3fc61e9740 1 Processor -- start 2024-09-12T03:14:25.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.841+0000 7f3fc61e9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d292b31310 0x55d292b3e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:25.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.841+0000 7f3fc61e9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d292b31c30 0x55d292b3b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:25.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.841+0000 7f3fc61e9740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d292b3e880 0x55d292b3b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:25.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.841+0000 7f3fc61e9740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d292b414b0 con 0x55d292b3e880 2024-09-12T03:14:25.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.841+0000 7f3fc61e9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d292a52050 con 0x55d292b31310 2024-09-12T03:14:25.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.841+0000 7f3fc61e9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d292b41630 con 0x55d292b31c30 2024-09-12T03:14:25.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.841+0000 7f3fc61e9740 10 client.0 _ll_register_callbacks cb 0x55d292a92160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:25.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.841+0000 7f3fc23ad640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d292b3e880 0x55d292b3b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:25.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.841+0000 7f3fc23ad640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d292b3e880 0x55d292b3b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37920/0 (socket says 172.21.15.116:37920) 2024-09-12T03:14:25.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.841+0000 7f3fc23ad640 1 -- 172.21.15.116:0/767868913 learned_addr learned my addr 172.21.15.116:0/767868913 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:25.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.841+0000 7f3fc1bac640 1 --2- 172.21.15.116:0/767868913 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d292b31310 0x55d292b3e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:25.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.841+0000 7f3fc13ab640 1 --2- 172.21.15.116:0/767868913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d292b31c30 0x55d292b3b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:25.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.841+0000 7f3fc23ad640 1 -- 172.21.15.116:0/767868913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d292b31c30 msgr2=0x55d292b3b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:25.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.841+0000 7f3fc23ad640 1 --2- 172.21.15.116:0/767868913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d292b31c30 0x55d292b3b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.841+0000 7f3fc23ad640 1 -- 172.21.15.116:0/767868913 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d292b31310 msgr2=0x55d292b3e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:25.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.841+0000 7f3fc23ad640 1 --2- 172.21.15.116:0/767868913 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d292b31310 0x55d292b3e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.841+0000 7f3fc23ad640 1 -- 172.21.15.116:0/767868913 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2928083c0 con 0x55d292b3e880 2024-09-12T03:14:25.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.841+0000 7f3fc13ab640 1 --2- 172.21.15.116:0/767868913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d292b31c30 0x55d292b3b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:14:25.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.841+0000 7f3fc23ad640 1 --2- 172.21.15.116:0/767868913 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d292b3e880 0x55d292b3b9b0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4011530 tx=0x7f3fb4013030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9022dad9804ce35 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:25.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.841+0000 7f3fa57fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:14:25.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.841+0000 7f3fa57fa640 1 -- 172.21.15.116:0/767868913 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb401f070 con 0x55d292b3e880 2024-09-12T03:14:25.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.841+0000 7f3fa57fa640 1 -- 172.21.15.116:0/767868913 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f3fb4024420 con 0x55d292b3e880 2024-09-12T03:14:25.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.841+0000 7f3fc61e9740 1 -- 172.21.15.116:0/767868913 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fb401be20 con 0x55d292b3e880 2024-09-12T03:14:25.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.842+0000 7f3fa57fa640 1 -- 172.21.15.116:0/767868913 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3fb4017020 con 0x55d292b3e880 2024-09-12T03:14:25.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.842+0000 7f3fc61e9740 10 client.18600 fetch_fsmap learned FSMap version 25 2024-09-12T03:14:25.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.842+0000 7f3fc61e9740 1 -- 172.21.15.116:0/767868913 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d292b4ce10 con 0x55d292b3e880 2024-09-12T03:14:25.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.842+0000 7f3fa57fa640 1 -- 172.21.15.116:0/767868913 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fb401be20 con 0x55d292b3e880 2024-09-12T03:14:25.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.842+0000 7f3fa57fa640 1 -- 172.21.15.116:0/767868913 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3fb40053a0 con 0x55d292b3e880 2024-09-12T03:14:25.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.842+0000 7f3fc61e9740 10 client.18600 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:14:25.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.842+0000 7f3fc61e9740 10 client.18600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:14:25.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.842+0000 7f3fc61e9740 20 client.18600 populate_metadata read hostname 'smithi116' 2024-09-12T03:14:25.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.842+0000 7f3fc61e9740 1 --2- 172.21.15.116:0/767868913 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d292b4e270 0x55d292b50660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:25.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.842+0000 7f3fc61e9740 4 client.18600 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:14:25.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.842+0000 7f3fc61e9740 1 -- 172.21.15.116:0/767868913 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d292aa4850 con 0x55d292b4e270 2024-09-12T03:14:25.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.842+0000 7f3fc1bac640 1 --2- 172.21.15.116:0/767868913 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d292b4e270 0x55d292b50660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:25.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.843+0000 7f3fc1bac640 1 --2- 172.21.15.116:0/767868913 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d292b4e270 0x55d292b50660 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1e48de360af73a server_cookie=cb810a355b89b533 in_seq=0 out_seq=0 2024-09-12T03:14:25.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.843+0000 7f3fa57fa640 10 client.18600 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:25.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.843+0000 7f3fa57fa640 1 -- 172.21.15.116:0/767868913 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d292aa4850 con 0x55d292b4e270 2024-09-12T03:14:25.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.843+0000 7f3fa57fa640 10 client.18600 handle_command_reply: tid=0 2024-09-12T03:14:25.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.844+0000 7f3fc61e9740 1 client.18600 shutdown 2024-09-12T03:14:25.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.844+0000 7f3fc61e9740 1 -- 172.21.15.116:0/767868913 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d292b3e880 msgr2=0x55d292b3b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:25.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.844+0000 7f3fc61e9740 1 --2- 172.21.15.116:0/767868913 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d292b3e880 0x55d292b3b9b0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4011530 tx=0x7f3fb4013030 comp rx=0 tx=0).stop 2024-09-12T03:14:25.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.844+0000 7f3fc61e9740 1 -- 172.21.15.116:0/767868913 shutdown_connections 2024-09-12T03:14:25.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.844+0000 7f3fc61e9740 1 --2- 172.21.15.116:0/767868913 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d292b4e270 0x55d292b50660 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.844+0000 7f3fc61e9740 1 --2- 172.21.15.116:0/767868913 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d292b3e880 0x55d292b3b9b0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.844+0000 7f3fc61e9740 1 --2- 172.21.15.116:0/767868913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d292b31c30 0x55d292b3b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.844+0000 7f3fa57fa640 0 client.18600 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:25.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.844+0000 7f3fc61e9740 1 --2- 172.21.15.116:0/767868913 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d292b31310 0x55d292b3e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.844+0000 7f3fc61e9740 1 -- 172.21.15.116:0/767868913 >> 172.21.15.116:0/767868913 conn(0x55d292b2da20 msgr2=0x55d292b2cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:25.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.844+0000 7f3fc61e9740 1 -- 172.21.15.116:0/767868913 shutdown_connections 2024-09-12T03:14:25.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.844+0000 7f3fc61e9740 1 -- 172.21.15.116:0/767868913 wait complete. 2024-09-12T03:14:25.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.844+0000 7f3fc61e9740 20 client.18600 trim_cache size 0 max 16384 2024-09-12T03:14:25.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.861+0000 7f3fc61e9740 1 Processor -- start 2024-09-12T03:14:25.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.861+0000 7f3fc61e9740 1 -- start start 2024-09-12T03:14:25.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.861+0000 7f3fc61e9740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d292b31310 0x55d292b4a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:25.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.861+0000 7f3fc61e9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d292b31c30 0x55d292b55d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:25.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.861+0000 7f3fc61e9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d292b3e880 0x55d292b562c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:25.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.861+0000 7f3fc61e9740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc1adb60 con 0x55d292b31310 2024-09-12T03:14:25.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.861+0000 7f3fc61e9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc1ab430 con 0x55d292b31c30 2024-09-12T03:14:25.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.861+0000 7f3fc61e9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc1a9280 con 0x55d292b3e880 2024-09-12T03:14:25.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.861+0000 7f3fc1bac640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d292b31310 0x55d292b4a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:25.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.861+0000 7f3fc1bac640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d292b31310 0x55d292b4a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37932/0 (socket says 172.21.15.116:37932) 2024-09-12T03:14:25.875 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.861+0000 7f3fc1bac640 1 -- 172.21.15.116:0/313581886 learned_addr learned my addr 172.21.15.116:0/313581886 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:25.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.862+0000 7f3fc1bac640 1 -- 172.21.15.116:0/313581886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d292b3e880 msgr2=0x55d292b562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:25.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.862+0000 7f3fc1bac640 1 --2- 172.21.15.116:0/313581886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d292b3e880 0x55d292b562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.862+0000 7f3fc1bac640 1 -- 172.21.15.116:0/313581886 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d292b31c30 msgr2=0x55d292b55d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:25.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.862+0000 7f3fc1bac640 1 --2- 172.21.15.116:0/313581886 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d292b31c30 0x55d292b55d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.862+0000 7f3fc1bac640 1 -- 172.21.15.116:0/313581886 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d292b29000 con 0x55d292b31310 2024-09-12T03:14:25.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.862+0000 7f3fc1bac640 1 --2- 172.21.15.116:0/313581886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d292b31310 0x55d292b4a710 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f3fb80bc100 tx=0x7f3fb808f2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875c9f2c863c1ec1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:25.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.862+0000 7f3fa77fe640 1 -- 172.21.15.116:0/313581886 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8035040 con 0x55d292b31310 2024-09-12T03:14:25.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.862+0000 7f3fa77fe640 1 -- 172.21.15.116:0/313581886 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fb80bc170 con 0x55d292b31310 2024-09-12T03:14:25.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.862+0000 7f3fa77fe640 1 -- 172.21.15.116:0/313581886 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb80dc0b0 con 0x55d292b31310 2024-09-12T03:14:25.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.863+0000 7f3fc61e9740 1 -- 172.21.15.116:0/313581886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d292b31310 msgr2=0x55d292b4a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:25.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.863+0000 7f3fc61e9740 1 --2- 172.21.15.116:0/313581886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d292b31310 0x55d292b4a710 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f3fb80bc100 tx=0x7f3fb808f2f0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.863+0000 7f3fc61e9740 1 -- 172.21.15.116:0/313581886 shutdown_connections 2024-09-12T03:14:25.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.863+0000 7f3fc61e9740 1 --2- 172.21.15.116:0/313581886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d292b3e880 0x55d292b562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.863+0000 7f3fc61e9740 1 --2- 172.21.15.116:0/313581886 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d292b31c30 0x55d292b55d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.863+0000 7f3fc61e9740 1 --2- 172.21.15.116:0/313581886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d292b31310 0x55d292b4a710 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.863+0000 7f3fc61e9740 1 -- 172.21.15.116:0/313581886 >> 172.21.15.116:0/313581886 conn(0x55d292b2da20 msgr2=0x55d292b36b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:25.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.863+0000 7f3fc61e9740 1 -- 172.21.15.116:0/313581886 shutdown_connections 2024-09-12T03:14:25.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.863+0000 7f3fc61e9740 1 -- 172.21.15.116:0/313581886 wait complete. 2024-09-12T03:14:25.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.863+0000 7f3fc61e9740 1 -- start start 2024-09-12T03:14:25.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.863+0000 7f3fc61e9740 1 Processor -- start 2024-09-12T03:14:25.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.864+0000 7f3fc61e9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d292b31310 0x55d292bb1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:25.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.864+0000 7f3fc61e9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d292b31c30 0x55d292bb1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:25.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.864+0000 7f3fc61e9740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d292b3e880 0x55d292b41e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:25.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.864+0000 7f3fc61e9740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc1a98f0 con 0x55d292b3e880 2024-09-12T03:14:25.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.864+0000 7f3fc61e9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc1aad60 con 0x55d292b31c30 2024-09-12T03:14:25.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.864+0000 7f3fc61e9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc1a9fc0 con 0x55d292b31310 2024-09-12T03:14:25.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.864+0000 7f3fc61e9740 10 client.0 _ll_register_callbacks cb 0x7f3fb4011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:25.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.864+0000 7f3fc23ad640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d292b3e880 0x55d292b41e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:25.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.864+0000 7f3fc23ad640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d292b3e880 0x55d292b41e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37944/0 (socket says 172.21.15.116:37944) 2024-09-12T03:14:25.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.864+0000 7f3fc23ad640 1 -- 172.21.15.116:0/1391518852 learned_addr learned my addr 172.21.15.116:0/1391518852 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:25.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.864+0000 7f3fc1bac640 1 --2- 172.21.15.116:0/1391518852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d292b31310 0x55d292bb1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:25.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.864+0000 7f3fc13ab640 1 --2- 172.21.15.116:0/1391518852 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d292b31c30 0x55d292bb1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:25.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.864+0000 7f3fc23ad640 1 -- 172.21.15.116:0/1391518852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d292b31310 msgr2=0x55d292bb1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:25.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.864+0000 7f3fc23ad640 1 --2- 172.21.15.116:0/1391518852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d292b31310 0x55d292bb1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.864+0000 7f3fc23ad640 1 -- 172.21.15.116:0/1391518852 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d292b31c30 msgr2=0x55d292bb1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:25.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.864+0000 7f3fc23ad640 1 --2- 172.21.15.116:0/1391518852 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d292b31c30 0x55d292bb1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.864+0000 7f3fc23ad640 1 -- 172.21.15.116:0/1391518852 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d292b42630 con 0x55d292b3e880 2024-09-12T03:14:25.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.864+0000 7f3fc1bac640 1 --2- 172.21.15.116:0/1391518852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d292b31310 0x55d292bb1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:25.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.864+0000 7f3fc23ad640 1 --2- 172.21.15.116:0/1391518852 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d292b3e880 0x55d292b41e30 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4005770 tx=0x7f3fb401bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4155f18c5d73c015 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:25.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.864+0000 7f3fa57fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:14:25.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.864+0000 7f3fa57fa640 1 -- 172.21.15.116:0/1391518852 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb401f070 con 0x55d292b3e880 2024-09-12T03:14:25.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.865+0000 7f3fa57fa640 1 -- 172.21.15.116:0/1391518852 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f3fb4024630 con 0x55d292b3e880 2024-09-12T03:14:25.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.865+0000 7f3fc61e9740 1 -- 172.21.15.116:0/1391518852 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fb80bc170 con 0x55d292b3e880 2024-09-12T03:14:25.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.865+0000 7f3fa57fa640 1 -- 172.21.15.116:0/1391518852 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3fb4048050 con 0x55d292b3e880 2024-09-12T03:14:25.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.865+0000 7f3fc61e9740 10 client.18606 fetch_fsmap learned FSMap version 25 2024-09-12T03:14:25.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.865+0000 7f3fc61e9740 1 -- 172.21.15.116:0/1391518852 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d292bb8790 con 0x55d292b3e880 2024-09-12T03:14:25.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.865+0000 7f3fa57fa640 1 -- 172.21.15.116:0/1391518852 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fb80bc170 con 0x55d292b3e880 2024-09-12T03:14:25.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.865+0000 7f3fa57fa640 1 -- 172.21.15.116:0/1391518852 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3fb403f070 con 0x55d292b3e880 2024-09-12T03:14:25.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.865+0000 7f3fc61e9740 10 client.18606 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:14:25.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.865+0000 7f3fc61e9740 10 client.18606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:14:25.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.865+0000 7f3fc61e9740 20 client.18606 populate_metadata read hostname 'smithi116' 2024-09-12T03:14:25.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.865+0000 7f3fc61e9740 1 --2- 172.21.15.116:0/1391518852 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d292bb8f10 0x55d292bbb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:25.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.865+0000 7f3fc61e9740 4 client.18606 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:14:25.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.865+0000 7f3fc61e9740 1 -- 172.21.15.116:0/1391518852 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d292a9ed60 con 0x55d292bb8f10 2024-09-12T03:14:25.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.865+0000 7f3fc1bac640 1 --2- 172.21.15.116:0/1391518852 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d292bb8f10 0x55d292bbb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:25.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.866+0000 7f3fa57fa640 10 client.18606 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:25.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.866+0000 7f3fc1bac640 1 --2- 172.21.15.116:0/1391518852 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d292bb8f10 0x55d292bbb330 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a0ee4474720068e server_cookie=8cd74689df555de7 in_seq=0 out_seq=0 2024-09-12T03:14:25.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.866+0000 7f3fa57fa640 1 -- 172.21.15.116:0/1391518852 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d292a9ed60 con 0x55d292bb8f10 2024-09-12T03:14:25.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.866+0000 7f3fa57fa640 10 client.18606 handle_command_reply: tid=0 2024-09-12T03:14:25.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.866+0000 7f3fc61e9740 1 client.18606 shutdown 2024-09-12T03:14:25.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.866+0000 7f3fc61e9740 1 -- 172.21.15.116:0/1391518852 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d292b3e880 msgr2=0x55d292b41e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:25.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.866+0000 7f3fc61e9740 1 --2- 172.21.15.116:0/1391518852 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d292b3e880 0x55d292b41e30 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4005770 tx=0x7f3fb401bc60 comp rx=0 tx=0).stop 2024-09-12T03:14:25.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.866+0000 7f3fc61e9740 1 -- 172.21.15.116:0/1391518852 shutdown_connections 2024-09-12T03:14:25.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.866+0000 7f3fc61e9740 1 --2- 172.21.15.116:0/1391518852 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d292bb8f10 0x55d292bbb330 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.866+0000 7f3fc61e9740 1 --2- 172.21.15.116:0/1391518852 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d292b3e880 0x55d292b41e30 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.866+0000 7f3fc61e9740 1 --2- 172.21.15.116:0/1391518852 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d292b31c30 0x55d292bb1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.866+0000 7f3fc61e9740 1 --2- 172.21.15.116:0/1391518852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d292b31310 0x55d292bb1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.866+0000 7f3fa57fa640 0 client.18606 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:25.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.866+0000 7f3fc61e9740 1 -- 172.21.15.116:0/1391518852 >> 172.21.15.116:0/1391518852 conn(0x55d292b2da20 msgr2=0x55d292b39d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:25.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.866+0000 7f3fc61e9740 1 -- 172.21.15.116:0/1391518852 shutdown_connections 2024-09-12T03:14:25.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.866+0000 7f3fc61e9740 1 -- 172.21.15.116:0/1391518852 wait complete. 2024-09-12T03:14:25.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.866+0000 7f3fc61e9740 20 client.18606 trim_cache size 0 max 16384 2024-09-12T03:14:25.881 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:14:25.881 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:14:25.881 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": { 2024-09-12T03:14:25.882 INFO:teuthology.orchestra.run.smithi116.stdout: "3ab84fa7-814a-4fbc-89e1-1c2d5695ff62": { 2024-09-12T03:14:25.882 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/", 2024-09-12T03:14:25.882 INFO:teuthology.orchestra.run.smithi116.stdout: "tag": "3ab84fa7-814a-4fbc-89e1-1c2d5695ff62", 2024-09-12T03:14:25.882 INFO:teuthology.orchestra.run.smithi116.stdout: "options": "recursive,force" 2024-09-12T03:14:25.882 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:14:25.882 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:14:25.882 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:14:25.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.867+0000 7f3fc3e10640 1 -- 172.21.15.116:0/1056147645 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3fa80a2b80 msgr2=0x7f3fa80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:25.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.867+0000 7f3fc3e10640 1 --2- 172.21.15.116:0/1056147645 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3fa80a2b80 0x7f3fa80a5040 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4020000 tx=0x7f3fb4021000 comp rx=0 tx=0).stop 2024-09-12T03:14:25.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.867+0000 7f3fc3e10640 1 -- 172.21.15.116:0/1056147645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fbc0ffbe0 msgr2=0x7f3fbc1972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:25.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.867+0000 7f3fc3e10640 1 --2- 172.21.15.116:0/1056147645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fbc0ffbe0 0x7f3fbc1972a0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8006010 tx=0x7f3fb80042f0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.867+0000 7f3fc3e10640 1 -- 172.21.15.116:0/1056147645 shutdown_connections 2024-09-12T03:14:25.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.867+0000 7f3fc3e10640 1 --2- 172.21.15.116:0/1056147645 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3fa80a2b80 0x7f3fa80a5040 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.867+0000 7f3fc3e10640 1 --2- 172.21.15.116:0/1056147645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3fbc104e50 0x7f3fbc19bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.867+0000 7f3fc3e10640 1 --2- 172.21.15.116:0/1056147645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fbc100050 0x7f3fbc1977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.867+0000 7f3fc3e10640 1 --2- 172.21.15.116:0/1056147645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fbc0ffbe0 0x7f3fbc1972a0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:25.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.867+0000 7f3fc3e10640 1 -- 172.21.15.116:0/1056147645 >> 172.21.15.116:0/1056147645 conn(0x7f3fbc0fba90 msgr2=0x7f3fbc0fbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:25.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.867+0000 7f3fc3e10640 1 -- 172.21.15.116:0/1056147645 shutdown_connections 2024-09-12T03:14:25.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:25.867+0000 7f3fc3e10640 1 -- 172.21.15.116:0/1056147645 wait complete. 2024-09-12T03:14:26.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:25 smithi116 ceph-mon[31410]: pgmap v681: 97 pgs: 97 active+clean; 5.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-12T03:14:26.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:25 smithi183 ceph-mon[35535]: pgmap v681: 97 pgs: 97 active+clean; 5.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-12T03:14:26.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:25 smithi177 ceph-mon[35305]: pgmap v681: 97 pgs: 97 active+clean; 5.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-12T03:14:26.595 INFO:tasks.cephfs.filesystem:scrub status for tag:3ab84fa7-814a-4fbc-89e1-1c2d5695ff62 - {'path': '/', 'tag': '3ab84fa7-814a-4fbc-89e1-1c2d5695ff62', 'options': 'recursive,force'} 2024-09-12T03:14:27.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:27 smithi183 ceph-mon[35535]: pgmap v682: 97 pgs: 97 active+clean; 5.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:14:27.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:27 smithi177 ceph-mon[35305]: pgmap v682: 97 pgs: 97 active+clean; 5.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:14:27.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:27 smithi116 ceph-mon[31410]: pgmap v682: 97 pgs: 97 active+clean; 5.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:14:29.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:28 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:14:29.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:28 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:14:29.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:28 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:14:30.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:29 smithi183 ceph-mon[35535]: pgmap v683: 97 pgs: 97 active+clean; 5.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:14:30.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:29 smithi177 ceph-mon[35305]: pgmap v683: 97 pgs: 97 active+clean; 5.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:14:30.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:29 smithi116 ceph-mon[31410]: pgmap v683: 97 pgs: 97 active+clean; 5.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:14:31.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:31 smithi183 ceph-mon[35535]: pgmap v684: 97 pgs: 97 active+clean; 5.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:14:31.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:31 smithi177 ceph-mon[35305]: pgmap v684: 97 pgs: 97 active+clean; 5.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:14:31.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:31 smithi116 ceph-mon[31410]: pgmap v684: 97 pgs: 97 active+clean; 5.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:14:33.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:33 smithi183 ceph-mon[35535]: pgmap v685: 97 pgs: 97 active+clean; 5.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:14:33.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:33 smithi177 ceph-mon[35305]: pgmap v685: 97 pgs: 97 active+clean; 5.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:14:33.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:33 smithi116 ceph-mon[31410]: pgmap v685: 97 pgs: 97 active+clean; 5.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:14:35.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:35 smithi183 ceph-mon[35535]: pgmap v686: 97 pgs: 97 active+clean; 5.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:14:35.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:35 smithi177 ceph-mon[35305]: pgmap v686: 97 pgs: 97 active+clean; 5.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:14:35.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:35 smithi116 ceph-mon[31410]: pgmap v686: 97 pgs: 97 active+clean; 5.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:14:37.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:37 smithi183 ceph-mon[35535]: pgmap v687: 97 pgs: 97 active+clean; 5.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:14:37.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:37 smithi177 ceph-mon[35305]: pgmap v687: 97 pgs: 97 active+clean; 5.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:14:37.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:37 smithi116 ceph-mon[31410]: pgmap v687: 97 pgs: 97 active+clean; 5.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:14:39.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:39 smithi183 ceph-mon[35535]: pgmap v688: 97 pgs: 97 active+clean; 5.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-12T03:14:39.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:39 smithi177 ceph-mon[35305]: pgmap v688: 97 pgs: 97 active+clean; 5.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-12T03:14:39.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:39 smithi116 ceph-mon[31410]: pgmap v688: 97 pgs: 97 active+clean; 5.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-12T03:14:41.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:41 smithi183 ceph-mon[35535]: pgmap v689: 97 pgs: 97 active+clean; 5.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:14:41.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:41 smithi177 ceph-mon[35305]: pgmap v689: 97 pgs: 97 active+clean; 5.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:14:41.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:41 smithi116 ceph-mon[31410]: pgmap v689: 97 pgs: 97 active+clean; 5.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:14:43.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:43 smithi183 ceph-mon[35535]: pgmap v690: 97 pgs: 97 active+clean; 5.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:14:43.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:43 smithi177 ceph-mon[35305]: pgmap v690: 97 pgs: 97 active+clean; 5.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:14:43.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:43 smithi116 ceph-mon[31410]: pgmap v690: 97 pgs: 97 active+clean; 5.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:14:45.314 INFO:tasks.workunit.client.0.smithi116.stdout:375/600: open fd = 3 2024-09-12T03:14:45.314 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.314 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.314 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.314 INFO:tasks.workunit.client.0.smithi116.stdout:376/600: open fd = 3 2024-09-12T03:14:45.314 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.314 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.314 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.315 INFO:tasks.workunit.client.0.smithi116.stdout:377/600: open fd = 3 2024-09-12T03:14:45.315 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.315 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.315 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.315 INFO:tasks.workunit.client.0.smithi116.stdout:378/600: open fd = 3 2024-09-12T03:14:45.315 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.315 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.315 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.315 INFO:tasks.workunit.client.0.smithi116.stdout:379/600: open fd = 3 2024-09-12T03:14:45.315 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.315 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.315 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.315 INFO:tasks.workunit.client.0.smithi116.stdout:380/600: open fd = 3 2024-09-12T03:14:45.315 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.316 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.316 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.316 INFO:tasks.workunit.client.0.smithi116.stdout:381/600: open fd = 3 2024-09-12T03:14:45.316 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.316 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.316 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.316 INFO:tasks.workunit.client.0.smithi116.stdout:382/600: open fd = 3 2024-09-12T03:14:45.316 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.316 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.316 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.316 INFO:tasks.workunit.client.0.smithi116.stdout:383/600: open fd = 3 2024-09-12T03:14:45.316 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.316 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.317 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.317 INFO:tasks.workunit.client.0.smithi116.stdout:384/600: open fd = 3 2024-09-12T03:14:45.317 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.317 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.317 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.317 INFO:tasks.workunit.client.0.smithi116.stdout:385/600: open fd = 3 2024-09-12T03:14:45.317 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.317 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.317 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.317 INFO:tasks.workunit.client.0.smithi116.stdout:386/600: open fd = 3 2024-09-12T03:14:45.317 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.317 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.317 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.318 INFO:tasks.workunit.client.0.smithi116.stdout:387/600: open fd = 3 2024-09-12T03:14:45.318 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.318 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.318 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.318 INFO:tasks.workunit.client.0.smithi116.stdout:388/600: open fd = 3 2024-09-12T03:14:45.318 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.318 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.318 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.318 INFO:tasks.workunit.client.0.smithi116.stdout:389/600: open fd = 3 2024-09-12T03:14:45.318 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.318 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.318 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.318 INFO:tasks.workunit.client.0.smithi116.stdout:390/600: open fd = 3 2024-09-12T03:14:45.318 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.319 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.319 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.319 INFO:tasks.workunit.client.0.smithi116.stdout:391/600: open fd = 3 2024-09-12T03:14:45.319 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.319 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.319 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.319 INFO:tasks.workunit.client.0.smithi116.stdout:392/600: open fd = 3 2024-09-12T03:14:45.319 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.319 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.319 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.319 INFO:tasks.workunit.client.0.smithi116.stdout:393/600: open fd = 3 2024-09-12T03:14:45.319 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.319 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.319 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.320 INFO:tasks.workunit.client.0.smithi116.stdout:394/600: open fd = 3 2024-09-12T03:14:45.320 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.320 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.320 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.320 INFO:tasks.workunit.client.0.smithi116.stdout:395/600: open fd = 3 2024-09-12T03:14:45.320 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.320 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.320 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.320 INFO:tasks.workunit.client.0.smithi116.stdout:396/600: open fd = 3 2024-09-12T03:14:45.320 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.320 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.320 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.320 INFO:tasks.workunit.client.0.smithi116.stdout:397/600: open fd = 3 2024-09-12T03:14:45.321 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.321 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.321 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.321 INFO:tasks.workunit.client.0.smithi116.stdout:398/600: open fd = 3 2024-09-12T03:14:45.321 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.321 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.321 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.321 INFO:tasks.workunit.client.0.smithi116.stdout:399/600: open fd = 3 2024-09-12T03:14:45.321 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.321 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.321 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.321 INFO:tasks.workunit.client.0.smithi116.stdout:400/600: open fd = 3 2024-09-12T03:14:45.321 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.321 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.322 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.322 INFO:tasks.workunit.client.0.smithi116.stdout:401/600: open fd = 3 2024-09-12T03:14:45.322 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.322 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.322 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.322 INFO:tasks.workunit.client.0.smithi116.stdout:402/600: open fd = 3 2024-09-12T03:14:45.322 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.322 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.322 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.322 INFO:tasks.workunit.client.0.smithi116.stdout:403/600: open fd = 3 2024-09-12T03:14:45.322 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.322 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.322 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.322 INFO:tasks.workunit.client.0.smithi116.stdout:404/600: open fd = 3 2024-09-12T03:14:45.322 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.323 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.323 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.323 INFO:tasks.workunit.client.0.smithi116.stdout:405/600: open fd = 3 2024-09-12T03:14:45.323 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.323 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.323 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.323 INFO:tasks.workunit.client.0.smithi116.stdout:406/600: open fd = 3 2024-09-12T03:14:45.323 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.323 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.323 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.323 INFO:tasks.workunit.client.0.smithi116.stdout:407/600: open fd = 3 2024-09-12T03:14:45.323 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.323 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.323 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.323 INFO:tasks.workunit.client.0.smithi116.stdout:408/600: open fd = 3 2024-09-12T03:14:45.324 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.324 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.324 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.324 INFO:tasks.workunit.client.0.smithi116.stdout:409/600: open fd = 3 2024-09-12T03:14:45.324 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.324 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.324 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.324 INFO:tasks.workunit.client.0.smithi116.stdout:410/600: open fd = 3 2024-09-12T03:14:45.324 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.325 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.325 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.325 INFO:tasks.workunit.client.0.smithi116.stdout:411/600: open fd = 3 2024-09-12T03:14:45.325 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.325 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.325 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.325 INFO:tasks.workunit.client.0.smithi116.stdout:412/600: open fd = 3 2024-09-12T03:14:45.325 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.325 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.325 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.325 INFO:tasks.workunit.client.0.smithi116.stdout:413/600: open fd = 3 2024-09-12T03:14:45.325 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.325 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.326 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.326 INFO:tasks.workunit.client.0.smithi116.stdout:414/600: open fd = 3 2024-09-12T03:14:45.326 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.326 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.326 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.326 INFO:tasks.workunit.client.0.smithi116.stdout:415/600: open fd = 3 2024-09-12T03:14:45.326 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.326 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.326 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.326 INFO:tasks.workunit.client.0.smithi116.stdout:416/600: open fd = 3 2024-09-12T03:14:45.326 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.326 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.326 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.326 INFO:tasks.workunit.client.0.smithi116.stdout:417/600: open fd = 3 2024-09-12T03:14:45.326 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.327 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.327 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.327 INFO:tasks.workunit.client.0.smithi116.stdout:418/600: open fd = 3 2024-09-12T03:14:45.327 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.327 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.327 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.327 INFO:tasks.workunit.client.0.smithi116.stdout:419/600: open fd = 3 2024-09-12T03:14:45.327 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.327 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.327 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.327 INFO:tasks.workunit.client.0.smithi116.stdout:420/600: open fd = 3 2024-09-12T03:14:45.327 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.327 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.327 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.328 INFO:tasks.workunit.client.0.smithi116.stdout:421/600: open fd = 3 2024-09-12T03:14:45.328 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.328 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.328 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.328 INFO:tasks.workunit.client.0.smithi116.stdout:422/600: open fd = 3 2024-09-12T03:14:45.328 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.328 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.328 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.328 INFO:tasks.workunit.client.0.smithi116.stdout:423/600: open fd = 3 2024-09-12T03:14:45.328 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.328 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.328 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.328 INFO:tasks.workunit.client.0.smithi116.stdout:424/600: open fd = 3 2024-09-12T03:14:45.328 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.329 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.329 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.329 INFO:tasks.workunit.client.0.smithi116.stdout:425/600: open fd = 3 2024-09-12T03:14:45.329 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.329 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.329 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.329 INFO:tasks.workunit.client.0.smithi116.stdout:426/600: open fd = 3 2024-09-12T03:14:45.329 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.329 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.329 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.329 INFO:tasks.workunit.client.0.smithi116.stdout:427/600: open fd = 3 2024-09-12T03:14:45.329 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.329 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.329 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.330 INFO:tasks.workunit.client.0.smithi116.stdout:428/600: open fd = 3 2024-09-12T03:14:45.330 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.330 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.330 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.330 INFO:tasks.workunit.client.0.smithi116.stdout:429/600: open fd = 3 2024-09-12T03:14:45.330 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.330 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.330 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.330 INFO:tasks.workunit.client.0.smithi116.stdout:430/600: open fd = 3 2024-09-12T03:14:45.330 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.331 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.331 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.331 INFO:tasks.workunit.client.0.smithi116.stdout:431/600: open fd = 3 2024-09-12T03:14:45.331 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.331 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.331 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.331 INFO:tasks.workunit.client.0.smithi116.stdout:432/600: open fd = 3 2024-09-12T03:14:45.331 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.331 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.331 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.332 INFO:tasks.workunit.client.0.smithi116.stdout:433/600: open fd = 3 2024-09-12T03:14:45.332 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.332 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.332 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.332 INFO:tasks.workunit.client.0.smithi116.stdout:434/600: open fd = 3 2024-09-12T03:14:45.332 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.332 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.332 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.332 INFO:tasks.workunit.client.0.smithi116.stdout:435/600: open fd = 3 2024-09-12T03:14:45.332 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.332 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.332 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.332 INFO:tasks.workunit.client.0.smithi116.stdout:436/600: open fd = 3 2024-09-12T03:14:45.332 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.333 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:14:45.333 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:14:45.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:45 smithi183 ceph-mon[35535]: pgmap v691: 97 pgs: 97 active+clean; 5.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:14:45.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:45 smithi177 ceph-mon[35305]: pgmap v691: 97 pgs: 97 active+clean; 5.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:14:45.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:45 smithi116 ceph-mon[31410]: pgmap v691: 97 pgs: 97 active+clean; 5.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:14:47.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:47 smithi183 ceph-mon[35535]: pgmap v692: 97 pgs: 97 active+clean; 5.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:14:47.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:47 smithi177 ceph-mon[35305]: pgmap v692: 97 pgs: 97 active+clean; 5.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:14:47.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:47 smithi116 ceph-mon[31410]: pgmap v692: 97 pgs: 97 active+clean; 5.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:14:49.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:49 smithi183 ceph-mon[35535]: pgmap v693: 97 pgs: 97 active+clean; 5.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-12T03:14:49.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:49 smithi177 ceph-mon[35305]: pgmap v693: 97 pgs: 97 active+clean; 5.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-12T03:14:49.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:49 smithi116 ceph-mon[31410]: pgmap v693: 97 pgs: 97 active+clean; 5.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-12T03:14:51.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:51 smithi183 ceph-mon[35535]: pgmap v694: 97 pgs: 97 active+clean; 5.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T03:14:51.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:51 smithi177 ceph-mon[35305]: pgmap v694: 97 pgs: 97 active+clean; 5.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T03:14:51.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:51 smithi116 ceph-mon[31410]: pgmap v694: 97 pgs: 97 active+clean; 5.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T03:14:53.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:53 smithi183 ceph-mon[35535]: pgmap v695: 97 pgs: 97 active+clean; 5.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:14:53.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:53 smithi177 ceph-mon[35305]: pgmap v695: 97 pgs: 97 active+clean; 5.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:14:53.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:53 smithi116 ceph-mon[31410]: pgmap v695: 97 pgs: 97 active+clean; 5.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:14:55.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:55 smithi183 ceph-mon[35535]: pgmap v696: 97 pgs: 97 active+clean; 5.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:14:55.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:55 smithi177 ceph-mon[35305]: pgmap v696: 97 pgs: 97 active+clean; 5.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:14:55.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:55 smithi116 ceph-mon[31410]: pgmap v696: 97 pgs: 97 active+clean; 5.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:14:56.597 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:14:56.855 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:14:57.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.347+0000 7f4611f31640 1 -- 172.21.15.116:0/2423046788 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f460c106a80 msgr2=0x7f460c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:57.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.347+0000 7f4611f31640 1 --2- 172.21.15.116:0/2423046788 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f460c106a80 0x7f460c106e60 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f45fc009930 tx=0x7f45fc02a420 comp rx=0 tx=0).stop 2024-09-12T03:14:57.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.347+0000 7f4611f31640 1 -- 172.21.15.116:0/2423046788 shutdown_connections 2024-09-12T03:14:57.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.347+0000 7f4611f31640 1 --2- 172.21.15.116:0/2423046788 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f460c101420 0x7f460c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.347+0000 7f4611f31640 1 --2- 172.21.15.116:0/2423046788 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f460c100a80 0x7f460c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.347+0000 7f4611f31640 1 --2- 172.21.15.116:0/2423046788 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f460c106a80 0x7f460c106e60 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.347+0000 7f4611f31640 1 -- 172.21.15.116:0/2423046788 >> 172.21.15.116:0/2423046788 conn(0x7f460c0fc7a0 msgr2=0x7f460c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:57.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.347+0000 7f4611f31640 1 -- 172.21.15.116:0/2423046788 shutdown_connections 2024-09-12T03:14:57.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.347+0000 7f4611f31640 1 -- 172.21.15.116:0/2423046788 wait complete. 2024-09-12T03:14:57.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.347+0000 7f4611f31640 1 Processor -- start 2024-09-12T03:14:57.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.347+0000 7f4611f31640 1 -- start start 2024-09-12T03:14:57.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.348+0000 7f4611f31640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f460c100a80 0x7f460c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:57.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.348+0000 7f4611f31640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f460c101420 0x7f460c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:57.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.348+0000 7f4611f31640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f460c106a80 0x7f460c19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:57.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.348+0000 7f4611f31640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f460c110e80 con 0x7f460c106a80 2024-09-12T03:14:57.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.348+0000 7f4611f31640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f460c110d00 con 0x7f460c100a80 2024-09-12T03:14:57.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.348+0000 7f4611f31640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f460c111000 con 0x7f460c101420 2024-09-12T03:14:57.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.348+0000 7f460bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f460c106a80 0x7f460c19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:57.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.348+0000 7f460bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f460c106a80 0x7f460c19cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48612/0 (socket says 172.21.15.116:48612) 2024-09-12T03:14:57.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.348+0000 7f460bfff640 1 -- 172.21.15.116:0/873485627 learned_addr learned my addr 172.21.15.116:0/873485627 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:57.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.348+0000 7f460affd640 1 --2- 172.21.15.116:0/873485627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f460c101420 0x7f460c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:57.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.348+0000 7f460b7fe640 1 --2- 172.21.15.116:0/873485627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f460c100a80 0x7f460c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:57.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.348+0000 7f460bfff640 1 -- 172.21.15.116:0/873485627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f460c101420 msgr2=0x7f460c198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:57.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.348+0000 7f460bfff640 1 --2- 172.21.15.116:0/873485627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f460c101420 0x7f460c198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.348+0000 7f460bfff640 1 -- 172.21.15.116:0/873485627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f460c100a80 msgr2=0x7f460c1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:57.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.348+0000 7f460bfff640 1 --2- 172.21.15.116:0/873485627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f460c100a80 0x7f460c1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.348+0000 7f460bfff640 1 -- 172.21.15.116:0/873485627 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f460c19d370 con 0x7f460c106a80 2024-09-12T03:14:57.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.348+0000 7f460bfff640 1 --2- 172.21.15.116:0/873485627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f460c106a80 0x7f460c19cc90 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f460400eaa0 tx=0x7f460400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:57.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.348+0000 7f4608ff9640 1 -- 172.21.15.116:0/873485627 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4604004d20 con 0x7f460c106a80 2024-09-12T03:14:57.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.348+0000 7f4608ff9640 1 -- 172.21.15.116:0/873485627 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4604005020 con 0x7f460c106a80 2024-09-12T03:14:57.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.348+0000 7f4611f31640 1 -- 172.21.15.116:0/873485627 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f460c19d660 con 0x7f460c106a80 2024-09-12T03:14:57.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.348+0000 7f4608ff9640 1 -- 172.21.15.116:0/873485627 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46040116d0 con 0x7f460c106a80 2024-09-12T03:14:57.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.348+0000 7f4611f31640 1 -- 172.21.15.116:0/873485627 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f460c1a4f40 con 0x7f460c106a80 2024-09-12T03:14:57.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.351+0000 7f4608ff9640 1 -- 172.21.15.116:0/873485627 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f46040051d0 con 0x7f460c106a80 2024-09-12T03:14:57.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.351+0000 7f461430a740 1 Processor -- start 2024-09-12T03:14:57.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.351+0000 7f461430a740 1 -- start start 2024-09-12T03:14:57.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.351+0000 7f461430a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55849ffb9360 0x55849ffb9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:57.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.351+0000 7f461430a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55849ffb9c80 0x55849ffc8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:57.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.351+0000 7f461430a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55849ffc8a90 0x55849ffcae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:57.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.351+0000 7f461430a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55849ff1f2e0 con 0x55849ffb9360 2024-09-12T03:14:57.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.351+0000 7f461430a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55849fec6ce0 con 0x55849ffb9c80 2024-09-12T03:14:57.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.351+0000 7f461430a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55849ff1b2a0 con 0x55849ffc8a90 2024-09-12T03:14:57.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.351+0000 7f460affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55849ffb9360 0x55849ffb9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:57.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.351+0000 7f4608ff9640 1 --2- 172.21.15.116:0/873485627 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f45f40a2b80 0x7f45f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:57.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.351+0000 7f460affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55849ffb9360 0x55849ffb9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48614/0 (socket says 172.21.15.116:48614) 2024-09-12T03:14:57.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.351+0000 7f460affd640 1 -- 172.21.15.116:0/3735687752 learned_addr learned my addr 172.21.15.116:0/3735687752 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:57.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.351+0000 7f460affd640 1 --2- 172.21.15.116:0/3735687752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55849ffc8a90 0x55849ffcae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:57.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.351+0000 7f460bfff640 1 --2- 172.21.15.116:0/873485627 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f45f40a2b80 0x7f45f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:57.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.352+0000 7f460b7fe640 1 --2- 172.21.15.116:0/3735687752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55849ffb9c80 0x55849ffc8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:57.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.352+0000 7f460affd640 1 -- 172.21.15.116:0/3735687752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55849ffc8a90 msgr2=0x55849ffcae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:57.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.352+0000 7f460affd640 1 --2- 172.21.15.116:0/3735687752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55849ffc8a90 0x55849ffcae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.352+0000 7f460affd640 1 -- 172.21.15.116:0/3735687752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55849ffb9c80 msgr2=0x55849ffc8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:57.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.352+0000 7f460affd640 1 --2- 172.21.15.116:0/3735687752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55849ffb9c80 0x55849ffc8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.352+0000 7f460affd640 1 -- 172.21.15.116:0/3735687752 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55849feab440 con 0x55849ffb9360 2024-09-12T03:14:57.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.352+0000 7f460b7fe640 1 --2- 172.21.15.116:0/3735687752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55849ffb9c80 0x55849ffc8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:14:57.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.352+0000 7f4608ff9640 1 -- 172.21.15.116:0/873485627 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f46040cf0f0 con 0x7f460c106a80 2024-09-12T03:14:57.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.352+0000 7f460affd640 1 --2- 172.21.15.116:0/3735687752 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55849ffb9360 0x55849ffb9740 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f4600013560 tx=0x7f460000a820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99746b426e8c71ac server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:57.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.352+0000 7f460bfff640 1 --2- 172.21.15.116:0/873485627 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f45f40a2b80 0x7f45f40a5040 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f460409b680 tx=0x7f460406ea40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:57.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.352+0000 7f45f1ffb640 1 -- 172.21.15.116:0/3735687752 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4600019070 con 0x55849ffb9360 2024-09-12T03:14:57.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.352+0000 7f45f1ffb640 1 -- 172.21.15.116:0/3735687752 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4600011e10 con 0x55849ffb9360 2024-09-12T03:14:57.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.352+0000 7f45f1ffb640 1 -- 172.21.15.116:0/3735687752 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460000fe00 con 0x55849ffb9360 2024-09-12T03:14:57.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.353+0000 7f461430a740 1 -- 172.21.15.116:0/3735687752 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55849ffb9360 msgr2=0x55849ffb9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:57.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.353+0000 7f461430a740 1 --2- 172.21.15.116:0/3735687752 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55849ffb9360 0x55849ffb9740 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f4600013560 tx=0x7f460000a820 comp rx=0 tx=0).stop 2024-09-12T03:14:57.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.353+0000 7f461430a740 1 -- 172.21.15.116:0/3735687752 shutdown_connections 2024-09-12T03:14:57.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.353+0000 7f461430a740 1 --2- 172.21.15.116:0/3735687752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55849ffc8a90 0x55849ffcae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.353+0000 7f461430a740 1 --2- 172.21.15.116:0/3735687752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55849ffb9c80 0x55849ffc8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.353+0000 7f461430a740 1 --2- 172.21.15.116:0/3735687752 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55849ffb9360 0x55849ffb9740 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.353+0000 7f461430a740 1 -- 172.21.15.116:0/3735687752 >> 172.21.15.116:0/3735687752 conn(0x55849ffb5a30 msgr2=0x55849ffb5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:57.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.353+0000 7f461430a740 1 -- 172.21.15.116:0/3735687752 shutdown_connections 2024-09-12T03:14:57.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.353+0000 7f461430a740 1 -- 172.21.15.116:0/3735687752 wait complete. 2024-09-12T03:14:57.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.353+0000 7f461430a740 1 -- start start 2024-09-12T03:14:57.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.353+0000 7f461430a740 1 Processor -- start 2024-09-12T03:14:57.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.354+0000 7f461430a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55849ffb9360 0x55849ffc8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:57.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.354+0000 7f461430a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55849ffb9c80 0x55849ffc5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:57.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.354+0000 7f461430a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55849ffc8a90 0x55849ffc5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:57.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.354+0000 7f461430a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55849ffcb660 con 0x55849ffb9360 2024-09-12T03:14:57.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.354+0000 7f461430a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55849feda050 con 0x55849ffc8a90 2024-09-12T03:14:57.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.354+0000 7f461430a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55849ffcb7e0 con 0x55849ffb9c80 2024-09-12T03:14:57.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.354+0000 7f461430a740 10 client.0 _ll_register_callbacks cb 0x55849ff1a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:57.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.354+0000 7f460affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55849ffb9360 0x55849ffc8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:57.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.354+0000 7f460affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55849ffb9360 0x55849ffc8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48628/0 (socket says 172.21.15.116:48628) 2024-09-12T03:14:57.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.354+0000 7f460affd640 1 -- 172.21.15.116:0/2439368663 learned_addr learned my addr 172.21.15.116:0/2439368663 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:57.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.354+0000 7f460b7fe640 1 --2- 172.21.15.116:0/2439368663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55849ffb9c80 0x55849ffc5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:57.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.354+0000 7f460affd640 1 -- 172.21.15.116:0/2439368663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55849ffb9c80 msgr2=0x55849ffc5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:57.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.354+0000 7f460affd640 1 --2- 172.21.15.116:0/2439368663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55849ffb9c80 0x55849ffc5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.354+0000 7f460affd640 1 -- 172.21.15.116:0/2439368663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55849ffc8a90 msgr2=0x55849ffc5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:57.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.354+0000 7f460affd640 1 --2- 172.21.15.116:0/2439368663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55849ffc8a90 0x55849ffc5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.354+0000 7f460affd640 1 -- 172.21.15.116:0/2439368663 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55849fc903c0 con 0x55849ffb9360 2024-09-12T03:14:57.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.354+0000 7f460affd640 1 --2- 172.21.15.116:0/2439368663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55849ffb9360 0x55849ffc8470 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f460000a740 tx=0x7f460000f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b12b3c6a5689303 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:57.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.354+0000 7f45eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:14:57.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.354+0000 7f45eaffd640 1 -- 172.21.15.116:0/2439368663 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4600019050 con 0x55849ffb9360 2024-09-12T03:14:57.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.354+0000 7f45eaffd640 1 -- 172.21.15.116:0/2439368663 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f4600011930 con 0x55849ffb9360 2024-09-12T03:14:57.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.354+0000 7f461430a740 1 -- 172.21.15.116:0/2439368663 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4600011e10 con 0x55849ffb9360 2024-09-12T03:14:57.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.355+0000 7f45eaffd640 1 -- 172.21.15.116:0/2439368663 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4600011b30 con 0x55849ffb9360 2024-09-12T03:14:57.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.355+0000 7f461430a740 10 client.18618 fetch_fsmap learned FSMap version 25 2024-09-12T03:14:57.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.355+0000 7f461430a740 1 -- 172.21.15.116:0/2439368663 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55849ffd7050 con 0x55849ffb9360 2024-09-12T03:14:57.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.355+0000 7f45eaffd640 1 -- 172.21.15.116:0/2439368663 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4600011e10 con 0x55849ffb9360 2024-09-12T03:14:57.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.355+0000 7f45eaffd640 1 -- 172.21.15.116:0/2439368663 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4600004b70 con 0x55849ffb9360 2024-09-12T03:14:57.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.355+0000 7f461430a740 10 client.18618 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:14:57.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.355+0000 7f461430a740 10 client.18618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:14:57.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.355+0000 7f461430a740 20 client.18618 populate_metadata read hostname 'smithi116' 2024-09-12T03:14:57.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.355+0000 7f461430a740 1 --2- 172.21.15.116:0/2439368663 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55849ffd83a0 0x55849ffda790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:57.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.355+0000 7f461430a740 4 client.18618 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:14:57.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.355+0000 7f461430a740 1 -- 172.21.15.116:0/2439368663 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55849ff2c850 con 0x55849ffd83a0 2024-09-12T03:14:57.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.355+0000 7f460b7fe640 1 --2- 172.21.15.116:0/2439368663 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55849ffd83a0 0x55849ffda790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:57.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.356+0000 7f460b7fe640 1 --2- 172.21.15.116:0/2439368663 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55849ffd83a0 0x55849ffda790 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=701f6ff67f633c02 server_cookie=2f73eed20ab932ad in_seq=0 out_seq=0 2024-09-12T03:14:57.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.356+0000 7f45eaffd640 10 client.18618 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:57.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.356+0000 7f45eaffd640 1 -- 172.21.15.116:0/2439368663 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55849ff2c850 con 0x55849ffd83a0 2024-09-12T03:14:57.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.356+0000 7f45eaffd640 10 client.18618 handle_command_reply: tid=0 2024-09-12T03:14:57.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.356+0000 7f461430a740 1 client.18618 shutdown 2024-09-12T03:14:57.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.357+0000 7f461430a740 1 -- 172.21.15.116:0/2439368663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55849ffb9360 msgr2=0x55849ffc8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:57.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.357+0000 7f461430a740 1 --2- 172.21.15.116:0/2439368663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55849ffb9360 0x55849ffc8470 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f460000a740 tx=0x7f460000f830 comp rx=0 tx=0).stop 2024-09-12T03:14:57.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.357+0000 7f461430a740 1 -- 172.21.15.116:0/2439368663 shutdown_connections 2024-09-12T03:14:57.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.357+0000 7f461430a740 1 --2- 172.21.15.116:0/2439368663 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55849ffd83a0 0x55849ffda790 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.357+0000 7f461430a740 1 --2- 172.21.15.116:0/2439368663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55849ffc8a90 0x55849ffc5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.357+0000 7f461430a740 1 --2- 172.21.15.116:0/2439368663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55849ffb9c80 0x55849ffc5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.357+0000 7f45eaffd640 0 client.18618 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:57.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.357+0000 7f461430a740 1 --2- 172.21.15.116:0/2439368663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55849ffb9360 0x55849ffc8470 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.357+0000 7f461430a740 1 -- 172.21.15.116:0/2439368663 >> 172.21.15.116:0/2439368663 conn(0x55849ffb5a30 msgr2=0x55849ffbb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:57.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.357+0000 7f461430a740 1 -- 172.21.15.116:0/2439368663 shutdown_connections 2024-09-12T03:14:57.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.357+0000 7f461430a740 1 -- 172.21.15.116:0/2439368663 wait complete. 2024-09-12T03:14:57.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.357+0000 7f461430a740 20 client.18618 trim_cache size 0 max 16384 2024-09-12T03:14:57.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.374+0000 7f461430a740 1 Processor -- start 2024-09-12T03:14:57.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.374+0000 7f461430a740 1 -- start start 2024-09-12T03:14:57.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.374+0000 7f461430a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55849ffb9360 0x55849ffb4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:57.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.374+0000 7f461430a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55849ffb9c80 0x55849ffc7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:57.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.374+0000 7f461430a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55849ffc8a90 0x55849ffc7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:57.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.374+0000 7f461430a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f460c1adc00 con 0x55849ffb9360 2024-09-12T03:14:57.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.374+0000 7f461430a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f460c1ab4d0 con 0x55849ffc8a90 2024-09-12T03:14:57.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.374+0000 7f461430a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f460c1a9320 con 0x55849ffb9c80 2024-09-12T03:14:57.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.374+0000 7f460affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55849ffb9360 0x55849ffb4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:57.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.374+0000 7f460affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55849ffb9360 0x55849ffb4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48632/0 (socket says 172.21.15.116:48632) 2024-09-12T03:14:57.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.374+0000 7f460affd640 1 -- 172.21.15.116:0/2725898005 learned_addr learned my addr 172.21.15.116:0/2725898005 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:57.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.374+0000 7f460b7fe640 1 --2- 172.21.15.116:0/2725898005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55849ffb9c80 0x55849ffc7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:57.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.374+0000 7f460affd640 1 -- 172.21.15.116:0/2725898005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55849ffb9c80 msgr2=0x55849ffc7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:57.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.374+0000 7f460affd640 1 --2- 172.21.15.116:0/2725898005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55849ffb9c80 0x55849ffc7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.374+0000 7f460affd640 1 -- 172.21.15.116:0/2725898005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55849ffc8a90 msgr2=0x55849ffc7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:57.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.374+0000 7f460affd640 1 --2- 172.21.15.116:0/2725898005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55849ffc8a90 0x55849ffc7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.374+0000 7f460affd640 1 -- 172.21.15.116:0/2725898005 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55849ffc6060 con 0x55849ffb9360 2024-09-12T03:14:57.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.374+0000 7f460affd640 1 --2- 172.21.15.116:0/2725898005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55849ffb9360 0x55849ffb4e40 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f46000058c0 tx=0x7f460000a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f61baf9b4ef731d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:57.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.374+0000 7f45f17fa640 1 -- 172.21.15.116:0/2725898005 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4600039070 con 0x55849ffb9360 2024-09-12T03:14:57.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.375+0000 7f45f17fa640 1 -- 172.21.15.116:0/2725898005 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4600024070 con 0x55849ffb9360 2024-09-12T03:14:57.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.375+0000 7f45f17fa640 1 -- 172.21.15.116:0/2725898005 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4600019070 con 0x55849ffb9360 2024-09-12T03:14:57.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.375+0000 7f461430a740 1 -- 172.21.15.116:0/2725898005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55849ffb9360 msgr2=0x55849ffb4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:57.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.375+0000 7f461430a740 1 --2- 172.21.15.116:0/2725898005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55849ffb9360 0x55849ffb4e40 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f46000058c0 tx=0x7f460000a6b0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.375+0000 7f461430a740 1 -- 172.21.15.116:0/2725898005 shutdown_connections 2024-09-12T03:14:57.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.375+0000 7f461430a740 1 --2- 172.21.15.116:0/2725898005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55849ffc8a90 0x55849ffc7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.375+0000 7f461430a740 1 --2- 172.21.15.116:0/2725898005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55849ffb9c80 0x55849ffc7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.375+0000 7f461430a740 1 --2- 172.21.15.116:0/2725898005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55849ffb9360 0x55849ffb4e40 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.375+0000 7f461430a740 1 -- 172.21.15.116:0/2725898005 >> 172.21.15.116:0/2725898005 conn(0x55849ffb5a30 msgr2=0x55849ffc6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:57.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.375+0000 7f461430a740 1 -- 172.21.15.116:0/2725898005 shutdown_connections 2024-09-12T03:14:57.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.375+0000 7f461430a740 1 -- 172.21.15.116:0/2725898005 wait complete. 2024-09-12T03:14:57.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.376+0000 7f461430a740 1 -- start start 2024-09-12T03:14:57.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.376+0000 7f461430a740 1 Processor -- start 2024-09-12T03:14:57.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.376+0000 7f461430a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55849ffb9360 0x55849ffdda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:57.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.376+0000 7f461430a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55849ffb9c80 0x55849ffddfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:57.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.376+0000 7f461430a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55849ffc8a90 0x55849ffbeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:57.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.376+0000 7f461430a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f460c1a9990 con 0x55849ffc8a90 2024-09-12T03:14:57.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.376+0000 7f461430a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f460c1aae00 con 0x55849ffb9360 2024-09-12T03:14:57.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.376+0000 7f461430a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f460c1aa060 con 0x55849ffb9c80 2024-09-12T03:14:57.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.376+0000 7f461430a740 10 client.0 _ll_register_callbacks cb 0x7f4600013460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:57.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.376+0000 7f460affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55849ffc8a90 0x55849ffbeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:57.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.376+0000 7f460affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55849ffc8a90 0x55849ffbeaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48648/0 (socket says 172.21.15.116:48648) 2024-09-12T03:14:57.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.376+0000 7f460affd640 1 -- 172.21.15.116:0/154634573 learned_addr learned my addr 172.21.15.116:0/154634573 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:57.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.376+0000 7f460b7fe640 1 --2- 172.21.15.116:0/154634573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55849ffb9c80 0x55849ffddfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:57.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.376+0000 7f460affd640 1 --2- 172.21.15.116:0/154634573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55849ffb9360 0x55849ffdda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:57.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.376+0000 7f460affd640 1 -- 172.21.15.116:0/154634573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55849ffb9c80 msgr2=0x55849ffddfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:57.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.376+0000 7f460affd640 1 --2- 172.21.15.116:0/154634573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55849ffb9c80 0x55849ffddfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.376+0000 7f460affd640 1 -- 172.21.15.116:0/154634573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55849ffb9360 msgr2=0x55849ffdda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:57.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.376+0000 7f460affd640 1 --2- 172.21.15.116:0/154634573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55849ffb9360 0x55849ffdda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.376+0000 7f460affd640 1 -- 172.21.15.116:0/154634573 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55849ffbf320 con 0x55849ffc8a90 2024-09-12T03:14:57.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.377+0000 7f460affd640 1 --2- 172.21.15.116:0/154634573 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55849ffc8a90 0x55849ffbeaf0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f4600029610 tx=0x7f460000ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e004b7e92ddc1b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:57.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.377+0000 7f45eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:14:57.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.377+0000 7f45eaffd640 1 -- 172.21.15.116:0/154634573 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4600039070 con 0x55849ffc8a90 2024-09-12T03:14:57.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.377+0000 7f45eaffd640 1 -- 172.21.15.116:0/154634573 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f4600004fc0 con 0x55849ffc8a90 2024-09-12T03:14:57.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.377+0000 7f461430a740 1 -- 172.21.15.116:0/154634573 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4600024070 con 0x55849ffc8a90 2024-09-12T03:14:57.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.377+0000 7f45eaffd640 1 -- 172.21.15.116:0/154634573 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4600019030 con 0x55849ffc8a90 2024-09-12T03:14:57.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.377+0000 7f461430a740 10 client.18624 fetch_fsmap learned FSMap version 25 2024-09-12T03:14:57.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.377+0000 7f461430a740 1 -- 172.21.15.116:0/154634573 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55849ffdd310 con 0x55849ffc8a90 2024-09-12T03:14:57.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.377+0000 7f45eaffd640 1 -- 172.21.15.116:0/154634573 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4600024070 con 0x55849ffc8a90 2024-09-12T03:14:57.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.377+0000 7f45eaffd640 1 -- 172.21.15.116:0/154634573 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f46000048a0 con 0x55849ffc8a90 2024-09-12T03:14:57.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.377+0000 7f461430a740 10 client.18624 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:14:57.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.377+0000 7f461430a740 10 client.18624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:14:57.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.377+0000 7f461430a740 20 client.18624 populate_metadata read hostname 'smithi116' 2024-09-12T03:14:57.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.377+0000 7f461430a740 1 --2- 172.21.15.116:0/154634573 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5584a00476d0 0x5584a0049af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:57.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.377+0000 7f461430a740 4 client.18624 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:14:57.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.377+0000 7f461430a740 1 -- 172.21.15.116:0/154634573 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55849ff26d60 con 0x5584a00476d0 2024-09-12T03:14:57.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.378+0000 7f460b7fe640 1 --2- 172.21.15.116:0/154634573 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5584a00476d0 0x5584a0049af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:57.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.378+0000 7f45eaffd640 10 client.18624 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:57.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.378+0000 7f460b7fe640 1 --2- 172.21.15.116:0/154634573 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5584a00476d0 0x5584a0049af0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a998ae61a76107d6 server_cookie=1a6b3fcc84a6735d in_seq=0 out_seq=0 2024-09-12T03:14:57.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.378+0000 7f45eaffd640 1 -- 172.21.15.116:0/154634573 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55849ff26d60 con 0x5584a00476d0 2024-09-12T03:14:57.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.378+0000 7f45eaffd640 10 client.18624 handle_command_reply: tid=0 2024-09-12T03:14:57.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.378+0000 7f461430a740 1 client.18624 shutdown 2024-09-12T03:14:57.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.378+0000 7f461430a740 1 -- 172.21.15.116:0/154634573 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55849ffc8a90 msgr2=0x55849ffbeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:57.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.378+0000 7f461430a740 1 --2- 172.21.15.116:0/154634573 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55849ffc8a90 0x55849ffbeaf0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f4600029610 tx=0x7f460000ffd0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.378+0000 7f461430a740 1 -- 172.21.15.116:0/154634573 shutdown_connections 2024-09-12T03:14:57.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.378+0000 7f461430a740 1 --2- 172.21.15.116:0/154634573 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5584a00476d0 0x5584a0049af0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.378+0000 7f461430a740 1 --2- 172.21.15.116:0/154634573 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55849ffc8a90 0x55849ffbeaf0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.378+0000 7f45eaffd640 0 client.18624 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:14:57.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.378+0000 7f461430a740 1 --2- 172.21.15.116:0/154634573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55849ffb9c80 0x55849ffddfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.379+0000 7f461430a740 1 --2- 172.21.15.116:0/154634573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55849ffb9360 0x55849ffdda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.379+0000 7f461430a740 1 -- 172.21.15.116:0/154634573 >> 172.21.15.116:0/154634573 conn(0x55849ffb5a30 msgr2=0x55849ffc3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:57.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.379+0000 7f461430a740 1 -- 172.21.15.116:0/154634573 shutdown_connections 2024-09-12T03:14:57.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.379+0000 7f461430a740 1 -- 172.21.15.116:0/154634573 wait complete. 2024-09-12T03:14:57.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.379+0000 7f461430a740 20 client.18624 trim_cache size 0 max 16384 2024-09-12T03:14:57.387 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:14:57.387 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:14:57.387 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:14:57.387 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:14:57.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.379+0000 7f4611f31640 1 -- 172.21.15.116:0/873485627 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f45f40a2b80 msgr2=0x7f45f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:57.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.379+0000 7f4611f31640 1 --2- 172.21.15.116:0/873485627 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f45f40a2b80 0x7f45f40a5040 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f460409b680 tx=0x7f460406ea40 comp rx=0 tx=0).stop 2024-09-12T03:14:57.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.379+0000 7f4611f31640 1 -- 172.21.15.116:0/873485627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f460c106a80 msgr2=0x7f460c19cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:57.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.379+0000 7f4611f31640 1 --2- 172.21.15.116:0/873485627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f460c106a80 0x7f460c19cc90 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f460400eaa0 tx=0x7f460400eea0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.379+0000 7f4611f31640 1 -- 172.21.15.116:0/873485627 shutdown_connections 2024-09-12T03:14:57.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.379+0000 7f4611f31640 1 --2- 172.21.15.116:0/873485627 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f45f40a2b80 0x7f45f40a5040 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.379+0000 7f4611f31640 1 --2- 172.21.15.116:0/873485627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f460c106a80 0x7f460c19cc90 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.379+0000 7f4611f31640 1 --2- 172.21.15.116:0/873485627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f460c101420 0x7f460c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.379+0000 7f4611f31640 1 --2- 172.21.15.116:0/873485627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f460c100a80 0x7f460c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.379+0000 7f4611f31640 1 -- 172.21.15.116:0/873485627 >> 172.21.15.116:0/873485627 conn(0x7f460c0fc7a0 msgr2=0x7f460c1047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:57.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.379+0000 7f4611f31640 1 -- 172.21.15.116:0/873485627 shutdown_connections 2024-09-12T03:14:57.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:57.379+0000 7f4611f31640 1 -- 172.21.15.116:0/873485627 wait complete. 2024-09-12T03:14:57.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:57 smithi116 ceph-mon[31410]: pgmap v697: 97 pgs: 97 active+clean; 5.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:14:57.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:57 smithi183 ceph-mon[35535]: pgmap v697: 97 pgs: 97 active+clean; 5.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:14:57.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:57 smithi177 ceph-mon[35305]: pgmap v697: 97 pgs: 97 active+clean; 5.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:14:58.211 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:14:58.211 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:14:58.462 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:14:58.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.928+0000 7f4c119aa640 1 -- 172.21.15.116:0/1093776130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c0c102d30 msgr2=0x7f4c0c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:58.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.928+0000 7f4c119aa640 1 --2- 172.21.15.116:0/1093776130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c0c102d30 0x7f4c0c103110 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc009930 tx=0x7f4bfc02a420 comp rx=0 tx=0).stop 2024-09-12T03:14:58.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.929+0000 7f4c119aa640 1 -- 172.21.15.116:0/1093776130 shutdown_connections 2024-09-12T03:14:58.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.929+0000 7f4c119aa640 1 --2- 172.21.15.116:0/1093776130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c0c1082a0 0x7f4c0c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.929+0000 7f4c119aa640 1 --2- 172.21.15.116:0/1093776130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c0c1036e0 0x7f4c0c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.929+0000 7f4c119aa640 1 --2- 172.21.15.116:0/1093776130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c0c102d30 0x7f4c0c103110 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.929+0000 7f4c119aa640 1 -- 172.21.15.116:0/1093776130 >> 172.21.15.116:0/1093776130 conn(0x7f4c0c0fc7d0 msgr2=0x7f4c0c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:58.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.929+0000 7f4c119aa640 1 -- 172.21.15.116:0/1093776130 shutdown_connections 2024-09-12T03:14:58.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.929+0000 7f4c119aa640 1 -- 172.21.15.116:0/1093776130 wait complete. 2024-09-12T03:14:58.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.929+0000 7f4c119aa640 1 Processor -- start 2024-09-12T03:14:58.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.929+0000 7f4c119aa640 1 -- start start 2024-09-12T03:14:58.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.929+0000 7f4c119aa640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c0c102d30 0x7f4c0c1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:58.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.929+0000 7f4c119aa640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c0c1036e0 0x7f4c0c198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:58.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.930+0000 7f4c119aa640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c0c1082a0 0x7f4c0c19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:58.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.930+0000 7f4c119aa640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c1120a0 con 0x7f4c0c1036e0 2024-09-12T03:14:58.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.930+0000 7f4c119aa640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c111f20 con 0x7f4c0c102d30 2024-09-12T03:14:58.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.930+0000 7f4c119aa640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c112220 con 0x7f4c0c1082a0 2024-09-12T03:14:58.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.930+0000 7f4c0a7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c0c1036e0 0x7f4c0c198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:58.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.930+0000 7f4c0a7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c0c1036e0 0x7f4c0c198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48668/0 (socket says 172.21.15.116:48668) 2024-09-12T03:14:58.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.930+0000 7f4c0a7fc640 1 -- 172.21.15.116:0/3424605977 learned_addr learned my addr 172.21.15.116:0/3424605977 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:14:58.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.930+0000 7f4c0b7fe640 1 --2- 172.21.15.116:0/3424605977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c0c1082a0 0x7f4c0c19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:58.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.930+0000 7f4c0affd640 1 --2- 172.21.15.116:0/3424605977 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c0c102d30 0x7f4c0c1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:58.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.930+0000 7f4c0a7fc640 1 -- 172.21.15.116:0/3424605977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c0c1082a0 msgr2=0x7f4c0c19ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:58.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.930+0000 7f4c0a7fc640 1 --2- 172.21.15.116:0/3424605977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c0c1082a0 0x7f4c0c19ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.930+0000 7f4c0a7fc640 1 -- 172.21.15.116:0/3424605977 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c0c102d30 msgr2=0x7f4c0c1985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:58.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.930+0000 7f4c0a7fc640 1 --2- 172.21.15.116:0/3424605977 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c0c102d30 0x7f4c0c1985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.930+0000 7f4c0a7fc640 1 -- 172.21.15.116:0/3424605977 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0c19d590 con 0x7f4c0c1036e0 2024-09-12T03:14:58.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.930+0000 7f4c0affd640 1 --2- 172.21.15.116:0/3424605977 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c0c102d30 0x7f4c0c1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:14:58.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.930+0000 7f4c0a7fc640 1 --2- 172.21.15.116:0/3424605977 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c0c1036e0 0x7f4c0c198b20 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f4bf400d750 tx=0x7f4bf400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:58.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.930+0000 7f4c109a8640 1 -- 172.21.15.116:0/3424605977 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4017070 con 0x7f4c0c1036e0 2024-09-12T03:14:58.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.930+0000 7f4c109a8640 1 -- 172.21.15.116:0/3424605977 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bf4005500 con 0x7f4c0c1036e0 2024-09-12T03:14:58.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.930+0000 7f4c119aa640 1 -- 172.21.15.116:0/3424605977 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c0c19d880 con 0x7f4c0c1036e0 2024-09-12T03:14:58.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.930+0000 7f4c109a8640 1 -- 172.21.15.116:0/3424605977 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4004e00 con 0x7f4c0c1036e0 2024-09-12T03:14:58.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.930+0000 7f4c119aa640 1 -- 172.21.15.116:0/3424605977 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c0c1a5160 con 0x7f4c0c1036e0 2024-09-12T03:14:58.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.931+0000 7f4c119aa640 1 -- 172.21.15.116:0/3424605977 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c0c06a850 con 0x7f4c0c1036e0 2024-09-12T03:14:58.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.932+0000 7f4c109a8640 1 -- 172.21.15.116:0/3424605977 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4bf4004fa0 con 0x7f4c0c1036e0 2024-09-12T03:14:58.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.933+0000 7f4c109a8640 1 --2- 172.21.15.116:0/3424605977 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4bf80a2b80 0x7f4bf80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:58.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.933+0000 7f4c109a8640 1 -- 172.21.15.116:0/3424605977 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f4bf40ce9d0 con 0x7f4c0c1036e0 2024-09-12T03:14:58.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.933+0000 7f4c0affd640 1 --2- 172.21.15.116:0/3424605977 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4bf80a2b80 0x7f4bf80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:58.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.934+0000 7f4c0affd640 1 --2- 172.21.15.116:0/3424605977 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4bf80a2b80 0x7f4bf80a5040 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc000c00 tx=0x7f4bfc032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:58.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:58.937+0000 7f4c109a8640 1 -- 172.21.15.116:0/3424605977 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4bf409b960 con 0x7f4c0c1036e0 2024-09-12T03:14:59.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:59.137+0000 7f4c119aa640 1 -- 172.21.15.116:0/3424605977 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c0c10a2b0 con 0x7f4c0c1036e0 2024-09-12T03:14:59.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:59.138+0000 7f4c109a8640 1 -- 172.21.15.116:0/3424605977 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f4bf409b260 con 0x7f4c0c1036e0 2024-09-12T03:14:59.138 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:14:59.138 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:14:59.139 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:14:59.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:59.140+0000 7f4c119aa640 1 -- 172.21.15.116:0/3424605977 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4bf80a2b80 msgr2=0x7f4bf80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:59.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:59.140+0000 7f4c119aa640 1 --2- 172.21.15.116:0/3424605977 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4bf80a2b80 0x7f4bf80a5040 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc000c00 tx=0x7f4bfc032c70 comp rx=0 tx=0).stop 2024-09-12T03:14:59.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:59.140+0000 7f4c119aa640 1 -- 172.21.15.116:0/3424605977 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c0c1036e0 msgr2=0x7f4c0c198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:59.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:59.140+0000 7f4c119aa640 1 --2- 172.21.15.116:0/3424605977 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c0c1036e0 0x7f4c0c198b20 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f4bf400d750 tx=0x7f4bf400dc10 comp rx=0 tx=0).stop 2024-09-12T03:14:59.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:59.140+0000 7f4c119aa640 1 -- 172.21.15.116:0/3424605977 shutdown_connections 2024-09-12T03:14:59.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:59.140+0000 7f4c119aa640 1 --2- 172.21.15.116:0/3424605977 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4bf80a2b80 0x7f4bf80a5040 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:59.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:59.140+0000 7f4c119aa640 1 --2- 172.21.15.116:0/3424605977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c0c1082a0 0x7f4c0c19ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:59.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:59.140+0000 7f4c119aa640 1 --2- 172.21.15.116:0/3424605977 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c0c1036e0 0x7f4c0c198b20 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:59.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:59.140+0000 7f4c119aa640 1 --2- 172.21.15.116:0/3424605977 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c0c102d30 0x7f4c0c1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:59.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:59.140+0000 7f4c119aa640 1 -- 172.21.15.116:0/3424605977 >> 172.21.15.116:0/3424605977 conn(0x7f4c0c0fc7d0 msgr2=0x7f4c0c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:59.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:59.140+0000 7f4c119aa640 1 -- 172.21.15.116:0/3424605977 shutdown_connections 2024-09-12T03:14:59.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:14:59.140+0000 7f4c119aa640 1 -- 172.21.15.116:0/3424605977 wait complete. 2024-09-12T03:14:59.301 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.2 2024-09-12T03:14:59.301 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:14:59.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:14:59 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3424605977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:14:59.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:14:59 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3424605977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:14:59.511 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:14:59 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3424605977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:14:59.966 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:15:00.082 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:15:00.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:00 smithi183 ceph-mon[35535]: pgmap v698: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:15:00.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:00 smithi177 ceph-mon[35305]: pgmap v698: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:15:00.508 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:00 smithi116 ceph-mon[31410]: pgmap v698: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:15:00.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.587+0000 7f415d1d6640 1 -- 172.21.15.116:0/3858879105 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4158101420 msgr2=0x7f415810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:00.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.587+0000 7f415d1d6640 1 --2- 172.21.15.116:0/3858879105 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4158101420 0x7f415810d950 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f4148009930 tx=0x7f414802a420 comp rx=0 tx=0).stop 2024-09-12T03:15:00.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.587+0000 7f415d1d6640 1 -- 172.21.15.116:0/3858879105 shutdown_connections 2024-09-12T03:15:00.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.587+0000 7f415d1d6640 1 --2- 172.21.15.116:0/3858879105 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4158101420 0x7f415810d950 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:00.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.587+0000 7f415d1d6640 1 --2- 172.21.15.116:0/3858879105 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4158100a80 0x7f4158100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:00.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.587+0000 7f415d1d6640 1 --2- 172.21.15.116:0/3858879105 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4158106a80 0x7f4158106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:00.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.587+0000 7f415d1d6640 1 -- 172.21.15.116:0/3858879105 >> 172.21.15.116:0/3858879105 conn(0x7f41580fc7a0 msgr2=0x7f41580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:00.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.587+0000 7f415d1d6640 1 -- 172.21.15.116:0/3858879105 shutdown_connections 2024-09-12T03:15:00.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.587+0000 7f415d1d6640 1 -- 172.21.15.116:0/3858879105 wait complete. 2024-09-12T03:15:00.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.588+0000 7f415d1d6640 1 Processor -- start 2024-09-12T03:15:00.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.588+0000 7f415d1d6640 1 -- start start 2024-09-12T03:15:00.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.588+0000 7f415d1d6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4158100a80 0x7f415819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:00.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.588+0000 7f415d1d6640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4158101420 0x7f415819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:00.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.588+0000 7f415d1d6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4158106a80 0x7f415819ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:00.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.588+0000 7f415d1d6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4158110e80 con 0x7f4158100a80 2024-09-12T03:15:00.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.588+0000 7f415d1d6640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4158110d00 con 0x7f4158101420 2024-09-12T03:15:00.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.588+0000 7f415d1d6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4158111000 con 0x7f4158106a80 2024-09-12T03:15:00.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.588+0000 7f4156d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4158100a80 0x7f415819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:00.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.588+0000 7f4156d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4158100a80 0x7f415819a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48680/0 (socket says 172.21.15.116:48680) 2024-09-12T03:15:00.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.588+0000 7f4156d9d640 1 -- 172.21.15.116:0/2959109182 learned_addr learned my addr 172.21.15.116:0/2959109182 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:00.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.588+0000 7f415659c640 1 --2- 172.21.15.116:0/2959109182 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4158101420 0x7f415819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:00.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.588+0000 7f415759e640 1 --2- 172.21.15.116:0/2959109182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4158106a80 0x7f415819ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:00.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.589+0000 7f4156d9d640 1 -- 172.21.15.116:0/2959109182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4158106a80 msgr2=0x7f415819ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:00.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.589+0000 7f4156d9d640 1 --2- 172.21.15.116:0/2959109182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4158106a80 0x7f415819ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:00.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.589+0000 7f4156d9d640 1 -- 172.21.15.116:0/2959109182 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4158101420 msgr2=0x7f415819aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:00.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.589+0000 7f4156d9d640 1 --2- 172.21.15.116:0/2959109182 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4158101420 0x7f415819aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:00.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.589+0000 7f4156d9d640 1 -- 172.21.15.116:0/2959109182 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f415819f550 con 0x7f4158100a80 2024-09-12T03:15:00.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.589+0000 7f4156d9d640 1 --2- 172.21.15.116:0/2959109182 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4158100a80 0x7f415819a5a0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f415000eab0 tx=0x7f415000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:00.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.589+0000 7f414ffff640 1 -- 172.21.15.116:0/2959109182 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415000cd80 con 0x7f4158100a80 2024-09-12T03:15:00.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.589+0000 7f414ffff640 1 -- 172.21.15.116:0/2959109182 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4150011950 con 0x7f4158100a80 2024-09-12T03:15:00.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.589+0000 7f415d1d6640 1 -- 172.21.15.116:0/2959109182 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f415819f840 con 0x7f4158100a80 2024-09-12T03:15:00.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.589+0000 7f414ffff640 1 -- 172.21.15.116:0/2959109182 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4150010a00 con 0x7f4158100a80 2024-09-12T03:15:00.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.589+0000 7f415d1d6640 1 -- 172.21.15.116:0/2959109182 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41581a7120 con 0x7f4158100a80 2024-09-12T03:15:00.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.590+0000 7f415d1d6640 1 -- 172.21.15.116:0/2959109182 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4158100ee0 con 0x7f4158100a80 2024-09-12T03:15:00.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.591+0000 7f414ffff640 1 -- 172.21.15.116:0/2959109182 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4150011b00 con 0x7f4158100a80 2024-09-12T03:15:00.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.592+0000 7f414ffff640 1 --2- 172.21.15.116:0/2959109182 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f41440a2b80 0x7f41440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:00.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.592+0000 7f415659c640 1 --2- 172.21.15.116:0/2959109182 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f41440a2b80 0x7f41440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:00.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.592+0000 7f414ffff640 1 -- 172.21.15.116:0/2959109182 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f41500cf4d0 con 0x7f4158100a80 2024-09-12T03:15:00.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.592+0000 7f415659c640 1 --2- 172.21.15.116:0/2959109182 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f41440a2b80 0x7f41440a5040 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f41580febe0 tx=0x7f414000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:00.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.595+0000 7f414ffff640 1 -- 172.21.15.116:0/2959109182 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4150016030 con 0x7f4158100a80 2024-09-12T03:15:00.636 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:15:00.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.803+0000 7f415d1d6640 1 -- 172.21.15.116:0/2959109182 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f415819b900 con 0x7f4158100a80 2024-09-12T03:15:00.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.804+0000 7f414ffff640 1 -- 172.21.15.116:0/2959109182 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f415009c460 con 0x7f4158100a80 2024-09-12T03:15:00.804 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:15:00.804 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:15:00.805 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:15:00.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.806+0000 7f415d1d6640 1 -- 172.21.15.116:0/2959109182 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f41440a2b80 msgr2=0x7f41440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:00.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.806+0000 7f415d1d6640 1 --2- 172.21.15.116:0/2959109182 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f41440a2b80 0x7f41440a5040 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f41580febe0 tx=0x7f414000a2d0 comp rx=0 tx=0).stop 2024-09-12T03:15:00.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.806+0000 7f415d1d6640 1 -- 172.21.15.116:0/2959109182 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4158100a80 msgr2=0x7f415819a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:00.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.806+0000 7f415d1d6640 1 --2- 172.21.15.116:0/2959109182 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4158100a80 0x7f415819a5a0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f415000eab0 tx=0x7f415000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:15:00.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.806+0000 7f415d1d6640 1 -- 172.21.15.116:0/2959109182 shutdown_connections 2024-09-12T03:15:00.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.806+0000 7f415d1d6640 1 --2- 172.21.15.116:0/2959109182 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f41440a2b80 0x7f41440a5040 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:00.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.806+0000 7f415d1d6640 1 --2- 172.21.15.116:0/2959109182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4158106a80 0x7f415819ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:00.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.806+0000 7f415d1d6640 1 --2- 172.21.15.116:0/2959109182 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4158101420 0x7f415819aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:00.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.806+0000 7f415d1d6640 1 --2- 172.21.15.116:0/2959109182 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4158100a80 0x7f415819a5a0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:00.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.806+0000 7f415d1d6640 1 -- 172.21.15.116:0/2959109182 >> 172.21.15.116:0/2959109182 conn(0x7f41580fc7a0 msgr2=0x7f415806a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:00.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.806+0000 7f415d1d6640 1 -- 172.21.15.116:0/2959109182 shutdown_connections 2024-09-12T03:15:00.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:00.806+0000 7f415d1d6640 1 -- 172.21.15.116:0/2959109182 wait complete. 2024-09-12T03:15:01.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.295+0000 7f68b774c640 1 -- 172.21.15.116:0/3268897430 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68b0101ba0 msgr2=0x7f68b010d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:01.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.295+0000 7f68b774c640 1 --2- 172.21.15.116:0/3268897430 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68b0101ba0 0x7f68b010d970 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009ca0 tx=0x7f68a802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.295+0000 7f68b774c640 1 -- 172.21.15.116:0/3268897430 shutdown_connections 2024-09-12T03:15:01.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.295+0000 7f68b774c640 1 --2- 172.21.15.116:0/3268897430 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68b0101ba0 0x7f68b010d970 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.295+0000 7f68b774c640 1 --2- 172.21.15.116:0/3268897430 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68b0101200 0x7f68b0101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.295+0000 7f68b774c640 1 --2- 172.21.15.116:0/3268897430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68b0105ad0 0x7f68b0105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.295+0000 7f68b774c640 1 -- 172.21.15.116:0/3268897430 >> 172.21.15.116:0/3268897430 conn(0x7f68b0077210 msgr2=0x7f68b00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:01.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.295+0000 7f68b774c640 1 -- 172.21.15.116:0/3268897430 shutdown_connections 2024-09-12T03:15:01.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.295+0000 7f68b774c640 1 -- 172.21.15.116:0/3268897430 wait complete. 2024-09-12T03:15:01.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.296+0000 7f68b774c640 1 Processor -- start 2024-09-12T03:15:01.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.296+0000 7f68b774c640 1 -- start start 2024-09-12T03:15:01.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.296+0000 7f68b774c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68b0101200 0x7f68b01991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:01.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.296+0000 7f68b774c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68b0101ba0 0x7f68b0199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:01.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.296+0000 7f68b774c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68b0105ad0 0x7f68b01a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:01.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.296+0000 7f68b774c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f68b0110ef0 con 0x7f68b0105ad0 2024-09-12T03:15:01.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.296+0000 7f68b774c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f68b0110d70 con 0x7f68b0101200 2024-09-12T03:15:01.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.296+0000 7f68b774c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f68b0111070 con 0x7f68b0101ba0 2024-09-12T03:15:01.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.296+0000 7f68b5ce9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68b0105ad0 0x7f68b01a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:01.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.296+0000 7f68b5ce9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68b0105ad0 0x7f68b01a4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47390/0 (socket says 172.21.15.116:47390) 2024-09-12T03:15:01.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.296+0000 7f68b5ce9640 1 -- 172.21.15.116:0/3185977198 learned_addr learned my addr 172.21.15.116:0/3185977198 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:01.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.296+0000 7f68b54e8640 1 --2- 172.21.15.116:0/3185977198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68b0101200 0x7f68b01991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:01.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.296+0000 7f68b4ce7640 1 --2- 172.21.15.116:0/3185977198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68b0101ba0 0x7f68b0199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:01.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.296+0000 7f68b5ce9640 1 -- 172.21.15.116:0/3185977198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68b0101ba0 msgr2=0x7f68b0199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:01.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.296+0000 7f68b5ce9640 1 --2- 172.21.15.116:0/3185977198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68b0101ba0 0x7f68b0199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.296+0000 7f68b5ce9640 1 -- 172.21.15.116:0/3185977198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68b0101200 msgr2=0x7f68b01991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:01.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.296+0000 7f68b5ce9640 1 --2- 172.21.15.116:0/3185977198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68b0101200 0x7f68b01991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.296+0000 7f68b5ce9640 1 -- 172.21.15.116:0/3185977198 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b01a47b0 con 0x7f68b0105ad0 2024-09-12T03:15:01.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.297+0000 7f68b5ce9640 1 --2- 172.21.15.116:0/3185977198 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68b0105ad0 0x7f68b01a4270 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f68a8003f40 tx=0x7f68a8003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:01.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.297+0000 7f68a67fc640 1 -- 172.21.15.116:0/3185977198 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8031790 con 0x7f68b0105ad0 2024-09-12T03:15:01.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.297+0000 7f68a67fc640 1 -- 172.21.15.116:0/3185977198 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68a8031930 con 0x7f68b0105ad0 2024-09-12T03:15:01.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.297+0000 7f68b774c640 1 -- 172.21.15.116:0/3185977198 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b01a4980 con 0x7f68b0105ad0 2024-09-12T03:15:01.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.297+0000 7f68a67fc640 1 -- 172.21.15.116:0/3185977198 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8030bf0 con 0x7f68b0105ad0 2024-09-12T03:15:01.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.297+0000 7f68b774c640 1 -- 172.21.15.116:0/3185977198 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b01a4e30 con 0x7f68b0105ad0 2024-09-12T03:15:01.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.299+0000 7f68a67fc640 1 -- 172.21.15.116:0/3185977198 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f68a8004040 con 0x7f68b0105ad0 2024-09-12T03:15:01.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.299+0000 7f68b9b25740 1 Processor -- start 2024-09-12T03:15:01.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.300+0000 7f68b9b25740 1 -- start start 2024-09-12T03:15:01.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.300+0000 7f68b9b25740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5598e5fa3b40 0x5598e5fa2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:01.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.300+0000 7f68b9b25740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598e5fa25d0 0x5598e5fa2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:01.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.300+0000 7f68b9b25740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598e5fb1780 0x5598e5fb3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:01.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.300+0000 7f68b9b25740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5598e5f082f0 con 0x5598e5fa25d0 2024-09-12T03:15:01.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.300+0000 7f68b9b25740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5598e5eafcf0 con 0x5598e5fa3b40 2024-09-12T03:15:01.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.300+0000 7f68b9b25740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5598e5f042b0 con 0x5598e5fb1780 2024-09-12T03:15:01.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.300+0000 7f68b54e8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598e5fa25d0 0x5598e5fa2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:01.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.300+0000 7f68b54e8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598e5fa25d0 0x5598e5fa2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47406/0 (socket says 172.21.15.116:47406) 2024-09-12T03:15:01.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.300+0000 7f68b54e8640 1 -- 172.21.15.116:0/2537611626 learned_addr learned my addr 172.21.15.116:0/2537611626 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:01.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.300+0000 7f68b4ce7640 1 --2- 172.21.15.116:0/2537611626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598e5fb1780 0x5598e5fb3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:01.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.300+0000 7f68a67fc640 1 --2- 172.21.15.116:0/3185977198 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f689c0a2b80 0x7f689c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:01.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.300+0000 7f68b4ce7640 1 --2- 172.21.15.116:0/2537611626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5598e5fa3b40 0x5598e5fa2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:01.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.300+0000 7f68b54e8640 1 -- 172.21.15.116:0/2537611626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598e5fb1780 msgr2=0x5598e5fb3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:01.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.300+0000 7f68b54e8640 1 --2- 172.21.15.116:0/2537611626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598e5fb1780 0x5598e5fb3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.300+0000 7f68b54e8640 1 -- 172.21.15.116:0/2537611626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5598e5fa3b40 msgr2=0x5598e5fa2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:01.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.300+0000 7f68b54e8640 1 --2- 172.21.15.116:0/2537611626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5598e5fa3b40 0x5598e5fa2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.300+0000 7f68b54e8640 1 -- 172.21.15.116:0/2537611626 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598e5e943a0 con 0x5598e5fa25d0 2024-09-12T03:15:01.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.300+0000 7f68b4ce7640 1 --2- 172.21.15.116:0/2537611626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598e5fb1780 0x5598e5fb3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:15:01.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.300+0000 7f68b5ce9640 1 --2- 172.21.15.116:0/3185977198 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f689c0a2b80 0x7f689c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:01.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.300+0000 7f68b54e8640 1 --2- 172.21.15.116:0/2537611626 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598e5fa25d0 0x5598e5fa2a10 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f68a000bc50 tx=0x7f68a00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36aeeda2d22bd582 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:01.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.300+0000 7f68a67fc640 1 -- 172.21.15.116:0/3185977198 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f68a80f0500 con 0x7f68b0105ad0 2024-09-12T03:15:01.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.301+0000 7f689b7fe640 1 -- 172.21.15.116:0/2537611626 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a001c480 con 0x5598e5fa25d0 2024-09-12T03:15:01.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.301+0000 7f689b7fe640 1 -- 172.21.15.116:0/2537611626 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68a001cb00 con 0x5598e5fa25d0 2024-09-12T03:15:01.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.301+0000 7f689b7fe640 1 -- 172.21.15.116:0/2537611626 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a0011b90 con 0x5598e5fa25d0 2024-09-12T03:15:01.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.301+0000 7f68b9b25740 1 -- 172.21.15.116:0/2537611626 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598e5fa25d0 msgr2=0x5598e5fa2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:01.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.301+0000 7f68b9b25740 1 --2- 172.21.15.116:0/2537611626 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598e5fa25d0 0x5598e5fa2a10 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f68a000bc50 tx=0x7f68a00107a0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.301+0000 7f68b5ce9640 1 --2- 172.21.15.116:0/3185977198 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f689c0a2b80 0x7f689c0a5040 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f68a808b930 tx=0x7f68a80850c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:01.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.301+0000 7f68b9b25740 1 -- 172.21.15.116:0/2537611626 shutdown_connections 2024-09-12T03:15:01.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.301+0000 7f68b9b25740 1 --2- 172.21.15.116:0/2537611626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598e5fb1780 0x5598e5fb3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.301+0000 7f68b9b25740 1 --2- 172.21.15.116:0/2537611626 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598e5fa25d0 0x5598e5fa2a10 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.301+0000 7f68b9b25740 1 --2- 172.21.15.116:0/2537611626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5598e5fa3b40 0x5598e5fa2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.301+0000 7f68b9b25740 1 -- 172.21.15.116:0/2537611626 >> 172.21.15.116:0/2537611626 conn(0x5598e5f6b310 msgr2=0x5598e5f9ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:01.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.302+0000 7f68b9b25740 1 -- 172.21.15.116:0/2537611626 shutdown_connections 2024-09-12T03:15:01.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.302+0000 7f68b9b25740 1 -- 172.21.15.116:0/2537611626 wait complete. 2024-09-12T03:15:01.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.302+0000 7f68b9b25740 1 -- start start 2024-09-12T03:15:01.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.302+0000 7f68b9b25740 1 Processor -- start 2024-09-12T03:15:01.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.302+0000 7f68b9b25740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598e5fa25d0 0x5598e5fb1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:01.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.302+0000 7f68b9b25740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5598e5fa3b40 0x5598e5fae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:01.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.302+0000 7f68b9b25740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598e5fb1780 0x5598e5fae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:01.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.302+0000 7f68b9b25740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5598e5fb4480 con 0x5598e5fa25d0 2024-09-12T03:15:01.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.302+0000 7f68b9b25740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5598e5ec3060 con 0x5598e5fa3b40 2024-09-12T03:15:01.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.302+0000 7f68b9b25740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5598e5fb4600 con 0x5598e5fb1780 2024-09-12T03:15:01.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.302+0000 7f68b9b25740 10 client.0 _ll_register_callbacks cb 0x5598e5c168c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:01.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.302+0000 7f68b4ce7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598e5fa25d0 0x5598e5fb1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:01.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.302+0000 7f68b4ce7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598e5fa25d0 0x5598e5fb1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47416/0 (socket says 172.21.15.116:47416) 2024-09-12T03:15:01.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.302+0000 7f68b4ce7640 1 -- 172.21.15.116:0/1628611911 learned_addr learned my addr 172.21.15.116:0/1628611911 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:01.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.303+0000 7f68b54e8640 1 --2- 172.21.15.116:0/1628611911 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5598e5fa3b40 0x5598e5fae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:01.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.303+0000 7f68b4ce7640 1 -- 172.21.15.116:0/1628611911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598e5fb1780 msgr2=0x5598e5fae9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:01.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.303+0000 7f68b4ce7640 1 --2- 172.21.15.116:0/1628611911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598e5fb1780 0x5598e5fae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.303+0000 7f68b4ce7640 1 -- 172.21.15.116:0/1628611911 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5598e5fa3b40 msgr2=0x5598e5fae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:01.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.303+0000 7f68b4ce7640 1 --2- 172.21.15.116:0/1628611911 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5598e5fa3b40 0x5598e5fae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.303+0000 7f68b4ce7640 1 -- 172.21.15.116:0/1628611911 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598e5c793c0 con 0x5598e5fa25d0 2024-09-12T03:15:01.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.303+0000 7f68b54e8640 1 --2- 172.21.15.116:0/1628611911 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5598e5fa3b40 0x5598e5fae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:15:01.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.303+0000 7f68b4ce7640 1 --2- 172.21.15.116:0/1628611911 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598e5fa25d0 0x5598e5fb1160 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0058c0 tx=0x7f68ac013ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7109051c4ded0f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:01.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.303+0000 7f6898ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:15:01.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.303+0000 7f6898ff9640 1 -- 172.21.15.116:0/1628611911 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac0130d0 con 0x5598e5fa25d0 2024-09-12T03:15:01.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.303+0000 7f6898ff9640 1 -- 172.21.15.116:0/1628611911 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f68ac005020 con 0x5598e5fa25d0 2024-09-12T03:15:01.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.303+0000 7f68b9b25740 1 -- 172.21.15.116:0/1628611911 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68a001cb00 con 0x5598e5fa25d0 2024-09-12T03:15:01.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.303+0000 7f6898ff9640 1 -- 172.21.15.116:0/1628611911 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f68ac022070 con 0x5598e5fa25d0 2024-09-12T03:15:01.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.303+0000 7f68b9b25740 10 client.18648 fetch_fsmap learned FSMap version 25 2024-09-12T03:15:01.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.303+0000 7f68b9b25740 1 -- 172.21.15.116:0/1628611911 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5598e5fbffe0 con 0x5598e5fa25d0 2024-09-12T03:15:01.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.303+0000 7f6898ff9640 1 -- 172.21.15.116:0/1628611911 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68a001cb00 con 0x5598e5fa25d0 2024-09-12T03:15:01.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.304+0000 7f6898ff9640 1 -- 172.21.15.116:0/1628611911 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f68ac011440 con 0x5598e5fa25d0 2024-09-12T03:15:01.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.304+0000 7f68b9b25740 10 client.18648 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:15:01.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.304+0000 7f68b9b25740 10 client.18648 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:15:01.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.304+0000 7f68b9b25740 20 client.18648 populate_metadata read hostname 'smithi116' 2024-09-12T03:15:01.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.304+0000 7f68b9b25740 1 --2- 172.21.15.116:0/1628611911 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5598e5fc14a0 0x5598e5fc38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:01.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.304+0000 7f68b9b25740 4 client.18648 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:15:01.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.304+0000 7f68b9b25740 1 -- 172.21.15.116:0/1628611911 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598e5f15860 con 0x5598e5fc14a0 2024-09-12T03:15:01.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.304+0000 7f68b54e8640 1 --2- 172.21.15.116:0/1628611911 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5598e5fc14a0 0x5598e5fc38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:01.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.304+0000 7f6898ff9640 10 client.18648 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:15:01.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.304+0000 7f68b54e8640 1 --2- 172.21.15.116:0/1628611911 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5598e5fc14a0 0x5598e5fc38d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b7f4a2db5c16991 server_cookie=5dea5926115ddf90 in_seq=0 out_seq=0 2024-09-12T03:15:01.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.305+0000 7f6898ff9640 1 -- 172.21.15.116:0/1628611911 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5598e5f15860 con 0x5598e5fc14a0 2024-09-12T03:15:01.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.305+0000 7f6898ff9640 10 client.18648 handle_command_reply: tid=0 2024-09-12T03:15:01.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.305+0000 7f68b9b25740 1 client.18648 shutdown 2024-09-12T03:15:01.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.305+0000 7f68b9b25740 1 -- 172.21.15.116:0/1628611911 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598e5fa25d0 msgr2=0x5598e5fb1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:01.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.305+0000 7f68b9b25740 1 --2- 172.21.15.116:0/1628611911 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598e5fa25d0 0x5598e5fb1160 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0058c0 tx=0x7f68ac013ee0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.305+0000 7f68b9b25740 1 -- 172.21.15.116:0/1628611911 shutdown_connections 2024-09-12T03:15:01.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.305+0000 7f68b9b25740 1 --2- 172.21.15.116:0/1628611911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598e5fb1780 0x5598e5fae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.305+0000 7f68b9b25740 1 --2- 172.21.15.116:0/1628611911 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5598e5fa3b40 0x5598e5fae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.305+0000 7f68b9b25740 1 --2- 172.21.15.116:0/1628611911 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5598e5fc14a0 0x5598e5fc38d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.305+0000 7f68b9b25740 1 --2- 172.21.15.116:0/1628611911 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598e5fa25d0 0x5598e5fb1160 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.305+0000 7f68b9b25740 1 -- 172.21.15.116:0/1628611911 >> 172.21.15.116:0/1628611911 conn(0x5598e5f6b310 msgr2=0x5598e5fa4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:01.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.305+0000 7f6898ff9640 0 client.18648 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:15:01.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.305+0000 7f68b9b25740 1 -- 172.21.15.116:0/1628611911 shutdown_connections 2024-09-12T03:15:01.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.306+0000 7f68b9b25740 1 -- 172.21.15.116:0/1628611911 wait complete. 2024-09-12T03:15:01.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.306+0000 7f68b9b25740 20 client.18648 trim_cache size 0 max 16384 2024-09-12T03:15:01.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.318+0000 7f68b9b25740 1 Processor -- start 2024-09-12T03:15:01.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.318+0000 7f68b9b25740 1 -- start start 2024-09-12T03:15:01.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.318+0000 7f68b9b25740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5598e5fa25d0 0x5598e5f9ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:01.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.318+0000 7f68b9b25740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598e5fa3b40 0x5598e5fbd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:01.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.319+0000 7f68b9b25740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598e5fb1780 0x5598e5fc91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:01.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.319+0000 7f68b9b25740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f68b01adb20 con 0x5598e5fa3b40 2024-09-12T03:15:01.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.319+0000 7f68b9b25740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f68b01ab3f0 con 0x5598e5fa25d0 2024-09-12T03:15:01.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.319+0000 7f68b9b25740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f68b01a9260 con 0x5598e5fb1780 2024-09-12T03:15:01.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.319+0000 7f68b54e8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598e5fa3b40 0x5598e5fbd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:01.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.319+0000 7f68b54e8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598e5fa3b40 0x5598e5fbd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47432/0 (socket says 172.21.15.116:47432) 2024-09-12T03:15:01.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.319+0000 7f68b54e8640 1 -- 172.21.15.116:0/3023107110 learned_addr learned my addr 172.21.15.116:0/3023107110 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:01.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.319+0000 7f68b4ce7640 1 --2- 172.21.15.116:0/3023107110 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5598e5fa25d0 0x5598e5f9ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:01.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.319+0000 7f68b54e8640 1 -- 172.21.15.116:0/3023107110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598e5fb1780 msgr2=0x5598e5fc91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:01.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.319+0000 7f68b54e8640 1 --2- 172.21.15.116:0/3023107110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598e5fb1780 0x5598e5fc91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.319+0000 7f68b54e8640 1 -- 172.21.15.116:0/3023107110 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5598e5fa25d0 msgr2=0x5598e5f9ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:01.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.319+0000 7f68b54e8640 1 --2- 172.21.15.116:0/3023107110 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5598e5fa25d0 0x5598e5f9ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.319+0000 7f68b54e8640 1 -- 172.21.15.116:0/3023107110 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598e5f9d310 con 0x5598e5fa3b40 2024-09-12T03:15:01.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.319+0000 7f68b54e8640 1 --2- 172.21.15.116:0/3023107110 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598e5fa3b40 0x5598e5fbd750 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f68a0010ca0 tx=0x7f68a000b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba071794e720dc14 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:01.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.319+0000 7f689affd640 1 -- 172.21.15.116:0/3023107110 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a0005040 con 0x5598e5fa3b40 2024-09-12T03:15:01.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.319+0000 7f689affd640 1 -- 172.21.15.116:0/3023107110 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68a00117f0 con 0x5598e5fa3b40 2024-09-12T03:15:01.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.319+0000 7f689affd640 1 -- 172.21.15.116:0/3023107110 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a0004880 con 0x5598e5fa3b40 2024-09-12T03:15:01.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.320+0000 7f68b9b25740 1 -- 172.21.15.116:0/3023107110 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598e5fa3b40 msgr2=0x5598e5fbd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:01.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.320+0000 7f68b9b25740 1 --2- 172.21.15.116:0/3023107110 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598e5fa3b40 0x5598e5fbd750 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f68a0010ca0 tx=0x7f68a000b7e0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.320+0000 7f68b9b25740 1 -- 172.21.15.116:0/3023107110 shutdown_connections 2024-09-12T03:15:01.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.320+0000 7f68b9b25740 1 --2- 172.21.15.116:0/3023107110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598e5fb1780 0x5598e5fc91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.320+0000 7f68b9b25740 1 --2- 172.21.15.116:0/3023107110 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598e5fa3b40 0x5598e5fbd750 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.320+0000 7f68b9b25740 1 --2- 172.21.15.116:0/3023107110 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5598e5fa25d0 0x5598e5f9ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.320+0000 7f68b9b25740 1 -- 172.21.15.116:0/3023107110 >> 172.21.15.116:0/3023107110 conn(0x5598e5f6b310 msgr2=0x5598e5f9e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:01.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.320+0000 7f68b9b25740 1 -- 172.21.15.116:0/3023107110 shutdown_connections 2024-09-12T03:15:01.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.320+0000 7f68b9b25740 1 -- 172.21.15.116:0/3023107110 wait complete. 2024-09-12T03:15:01.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.320+0000 7f68b9b25740 1 -- start start 2024-09-12T03:15:01.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.320+0000 7f68b9b25740 1 Processor -- start 2024-09-12T03:15:01.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.321+0000 7f68b9b25740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5598e5fa25d0 0x5598e5fba0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:01.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.321+0000 7f68b9b25740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598e5fa3b40 0x5598e5fab1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:01.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.321+0000 7f68b9b25740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598e5fb1780 0x5598e5fab740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:01.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.321+0000 7f68b9b25740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f68b01a98b0 con 0x5598e5fa3b40 2024-09-12T03:15:01.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.321+0000 7f68b9b25740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f68b01aad20 con 0x5598e5fa25d0 2024-09-12T03:15:01.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.321+0000 7f68b9b25740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f68b01a9f80 con 0x5598e5fb1780 2024-09-12T03:15:01.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.321+0000 7f68b9b25740 10 client.0 _ll_register_callbacks cb 0x7f68ac013690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:01.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.321+0000 7f68b54e8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598e5fa3b40 0x5598e5fab1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:01.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.321+0000 7f68b54e8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598e5fa3b40 0x5598e5fab1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47442/0 (socket says 172.21.15.116:47442) 2024-09-12T03:15:01.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.321+0000 7f68b54e8640 1 -- 172.21.15.116:0/3419671218 learned_addr learned my addr 172.21.15.116:0/3419671218 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:01.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.321+0000 7f68b4ce7640 1 --2- 172.21.15.116:0/3419671218 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5598e5fa25d0 0x5598e5fba0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:01.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.321+0000 7f68b4ce7640 1 --2- 172.21.15.116:0/3419671218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598e5fb1780 0x5598e5fab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:01.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.321+0000 7f68b54e8640 1 -- 172.21.15.116:0/3419671218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598e5fb1780 msgr2=0x5598e5fab740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:01.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.321+0000 7f68b54e8640 1 --2- 172.21.15.116:0/3419671218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598e5fb1780 0x5598e5fab740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.321+0000 7f68b54e8640 1 -- 172.21.15.116:0/3419671218 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5598e5fa25d0 msgr2=0x5598e5fba0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:01.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.321+0000 7f68b54e8640 1 --2- 172.21.15.116:0/3419671218 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5598e5fa25d0 0x5598e5fba0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.321+0000 7f68b54e8640 1 -- 172.21.15.116:0/3419671218 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598e5fabf70 con 0x5598e5fa3b40 2024-09-12T03:15:01.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.321+0000 7f68b4ce7640 1 --2- 172.21.15.116:0/3419671218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598e5fb1780 0x5598e5fab740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:15:01.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.321+0000 7f68b54e8640 1 --2- 172.21.15.116:0/3419671218 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598e5fa3b40 0x5598e5fab1e0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f68a000bc50 tx=0x7f68a000b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f43cfb2bcf6c37 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:01.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.321+0000 7f6898ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:15:01.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.321+0000 7f6898ff9640 1 -- 172.21.15.116:0/3419671218 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a003a060 con 0x5598e5fa3b40 2024-09-12T03:15:01.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.321+0000 7f6898ff9640 1 -- 172.21.15.116:0/3419671218 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f68a0011db0 con 0x5598e5fa3b40 2024-09-12T03:15:01.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.321+0000 7f68b9b25740 1 -- 172.21.15.116:0/3419671218 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68a00117f0 con 0x5598e5fa3b40 2024-09-12T03:15:01.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.322+0000 7f6898ff9640 1 -- 172.21.15.116:0/3419671218 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f68a001c480 con 0x5598e5fa3b40 2024-09-12T03:15:01.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.322+0000 7f68b9b25740 10 client.18654 fetch_fsmap learned FSMap version 25 2024-09-12T03:15:01.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.322+0000 7f68b9b25740 1 -- 172.21.15.116:0/3419671218 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5598e602c240 con 0x5598e5fa3b40 2024-09-12T03:15:01.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.322+0000 7f6898ff9640 1 -- 172.21.15.116:0/3419671218 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68a00117f0 con 0x5598e5fa3b40 2024-09-12T03:15:01.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.322+0000 7f6898ff9640 1 -- 172.21.15.116:0/3419671218 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f68a0041780 con 0x5598e5fa3b40 2024-09-12T03:15:01.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.322+0000 7f68b9b25740 10 client.18654 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:15:01.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.322+0000 7f68b9b25740 10 client.18654 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:15:01.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.322+0000 7f68b9b25740 20 client.18654 populate_metadata read hostname 'smithi116' 2024-09-12T03:15:01.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.322+0000 7f68b9b25740 1 --2- 172.21.15.116:0/3419671218 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5598e602c740 0x5598e602eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:01.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.322+0000 7f68b9b25740 4 client.18654 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:15:01.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.322+0000 7f68b9b25740 1 -- 172.21.15.116:0/3419671218 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5598e5f0fd70 con 0x5598e602c740 2024-09-12T03:15:01.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.322+0000 7f68b4ce7640 1 --2- 172.21.15.116:0/3419671218 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5598e602c740 0x5598e602eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:01.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.323+0000 7f6898ff9640 10 client.18654 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:15:01.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.323+0000 7f68b4ce7640 1 --2- 172.21.15.116:0/3419671218 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5598e602c740 0x5598e602eb60 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b996492c8ce5b7eb server_cookie=8276d00c82d0da7b in_seq=0 out_seq=0 2024-09-12T03:15:01.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.323+0000 7f6898ff9640 1 -- 172.21.15.116:0/3419671218 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5598e5f0fd70 con 0x5598e602c740 2024-09-12T03:15:01.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.323+0000 7f6898ff9640 10 client.18654 handle_command_reply: tid=0 2024-09-12T03:15:01.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.323+0000 7f68b9b25740 1 client.18654 shutdown 2024-09-12T03:15:01.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.323+0000 7f68b9b25740 1 -- 172.21.15.116:0/3419671218 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598e5fa3b40 msgr2=0x5598e5fab1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:01.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.323+0000 7f68b9b25740 1 --2- 172.21.15.116:0/3419671218 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598e5fa3b40 0x5598e5fab1e0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f68a000bc50 tx=0x7f68a000b940 comp rx=0 tx=0).stop 2024-09-12T03:15:01.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.323+0000 7f68b9b25740 1 -- 172.21.15.116:0/3419671218 shutdown_connections 2024-09-12T03:15:01.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.323+0000 7f68b9b25740 1 --2- 172.21.15.116:0/3419671218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598e5fb1780 0x5598e5fab740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.323+0000 7f68b9b25740 1 --2- 172.21.15.116:0/3419671218 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5598e602c740 0x5598e602eb60 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.323+0000 7f68b9b25740 1 --2- 172.21.15.116:0/3419671218 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598e5fa3b40 0x5598e5fab1e0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.323+0000 7f68b9b25740 1 --2- 172.21.15.116:0/3419671218 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5598e5fa25d0 0x5598e5fba0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.323+0000 7f68b9b25740 1 -- 172.21.15.116:0/3419671218 >> 172.21.15.116:0/3419671218 conn(0x5598e5f6b310 msgr2=0x5598e5f9d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:01.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.323+0000 7f6898ff9640 0 client.18654 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:15:01.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.323+0000 7f68b9b25740 1 -- 172.21.15.116:0/3419671218 shutdown_connections 2024-09-12T03:15:01.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.323+0000 7f68b9b25740 1 -- 172.21.15.116:0/3419671218 wait complete. 2024-09-12T03:15:01.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.323+0000 7f68b9b25740 20 client.18654 trim_cache size 0 max 16384 2024-09-12T03:15:01.331 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:15:01.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.324+0000 7f68b774c640 1 -- 172.21.15.116:0/3185977198 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f689c0a2b80 msgr2=0x7f689c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:01.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.324+0000 7f68b774c640 1 --2- 172.21.15.116:0/3185977198 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f689c0a2b80 0x7f689c0a5040 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f68a808b930 tx=0x7f68a80850c0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.324+0000 7f68b774c640 1 -- 172.21.15.116:0/3185977198 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68b0105ad0 msgr2=0x7f68b01a4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:01.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.324+0000 7f68b774c640 1 --2- 172.21.15.116:0/3185977198 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68b0105ad0 0x7f68b01a4270 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f68a8003f40 tx=0x7f68a8003f70 comp rx=0 tx=0).stop 2024-09-12T03:15:01.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.324+0000 7f68b774c640 1 -- 172.21.15.116:0/3185977198 shutdown_connections 2024-09-12T03:15:01.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.324+0000 7f68b774c640 1 --2- 172.21.15.116:0/3185977198 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f689c0a2b80 0x7f689c0a5040 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.324+0000 7f68b774c640 1 --2- 172.21.15.116:0/3185977198 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68b0105ad0 0x7f68b01a4270 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.324+0000 7f68b774c640 1 --2- 172.21.15.116:0/3185977198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68b0101ba0 0x7f68b0199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.324+0000 7f68b774c640 1 --2- 172.21.15.116:0/3185977198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68b0101200 0x7f68b01991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:01.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.324+0000 7f68b774c640 1 -- 172.21.15.116:0/3185977198 >> 172.21.15.116:0/3185977198 conn(0x7f68b0077210 msgr2=0x7f68b0103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:01.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.324+0000 7f68b774c640 1 -- 172.21.15.116:0/3185977198 shutdown_connections 2024-09-12T03:15:01.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:01.324+0000 7f68b774c640 1 -- 172.21.15.116:0/3185977198 wait complete. 2024-09-12T03:15:01.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:01 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2959109182' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:15:01.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:01 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2959109182' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:15:01.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:01 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2959109182' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:15:02.147 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 147.2 --await-for 45.0' 2024-09-12T03:15:02.147 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 147.2 --await-for 45.0 2024-09-12T03:15:02.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:02 smithi177 ceph-mon[35305]: pgmap v699: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:15:02.524 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:02 smithi116 ceph-mon[31410]: pgmap v699: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:15:02.550 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:15:02.628 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:15:02.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:02 smithi183 ceph-mon[35535]: pgmap v699: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:15:03.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.125+0000 7fefe2a57640 1 -- 172.21.15.116:0/3312590689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefdc106a80 msgr2=0x7fefdc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:03.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.125+0000 7fefe2a57640 1 --2- 172.21.15.116:0/3312590689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefdc106a80 0x7fefdc106e60 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fefcc009930 tx=0x7fefcc02a420 comp rx=0 tx=0).stop 2024-09-12T03:15:03.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.125+0000 7fefe2a57640 1 -- 172.21.15.116:0/3312590689 shutdown_connections 2024-09-12T03:15:03.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.125+0000 7fefe2a57640 1 --2- 172.21.15.116:0/3312590689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefdc101420 0x7fefdc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.125+0000 7fefe2a57640 1 --2- 172.21.15.116:0/3312590689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefdc100a80 0x7fefdc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.125+0000 7fefe2a57640 1 --2- 172.21.15.116:0/3312590689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefdc106a80 0x7fefdc106e60 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.125+0000 7fefe2a57640 1 -- 172.21.15.116:0/3312590689 >> 172.21.15.116:0/3312590689 conn(0x7fefdc0fc7a0 msgr2=0x7fefdc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:03.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.125+0000 7fefe2a57640 1 -- 172.21.15.116:0/3312590689 shutdown_connections 2024-09-12T03:15:03.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.125+0000 7fefe2a57640 1 -- 172.21.15.116:0/3312590689 wait complete. 2024-09-12T03:15:03.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.125+0000 7fefe2a57640 1 Processor -- start 2024-09-12T03:15:03.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.125+0000 7fefe2a57640 1 -- start start 2024-09-12T03:15:03.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.126+0000 7fefe2a57640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefdc100a80 0x7fefdc1974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.126+0000 7fefe2a57640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefdc101420 0x7fefdc197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.126+0000 7fefe2a57640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefdc106a80 0x7fefdc19bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.126+0000 7fefe2a57640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc110f20 con 0x7fefdc101420 2024-09-12T03:15:03.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.126+0000 7fefe2a57640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc110da0 con 0x7fefdc106a80 2024-09-12T03:15:03.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.126+0000 7fefe2a57640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc1110a0 con 0x7fefdc100a80 2024-09-12T03:15:03.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.126+0000 7fefd3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefdc101420 0x7fefdc197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.126+0000 7fefd3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefdc101420 0x7fefdc197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47460/0 (socket says 172.21.15.116:47460) 2024-09-12T03:15:03.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.126+0000 7fefd3fff640 1 -- 172.21.15.116:0/753555171 learned_addr learned my addr 172.21.15.116:0/753555171 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:03.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.126+0000 7fefdbfff640 1 --2- 172.21.15.116:0/753555171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefdc100a80 0x7fefdc1974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.126+0000 7fefe0ff4640 1 --2- 172.21.15.116:0/753555171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefdc106a80 0x7fefdc19bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.126+0000 7fefd3fff640 1 -- 172.21.15.116:0/753555171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefdc100a80 msgr2=0x7fefdc1974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:03.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.126+0000 7fefd3fff640 1 --2- 172.21.15.116:0/753555171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefdc100a80 0x7fefdc1974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.126+0000 7fefd3fff640 1 -- 172.21.15.116:0/753555171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefdc106a80 msgr2=0x7fefdc19bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:03.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.126+0000 7fefd3fff640 1 --2- 172.21.15.116:0/753555171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefdc106a80 0x7fefdc19bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.126+0000 7fefd3fff640 1 -- 172.21.15.116:0/753555171 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefdc19c4a0 con 0x7fefdc101420 2024-09-12T03:15:03.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.126+0000 7fefe0ff4640 1 --2- 172.21.15.116:0/753555171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefdc106a80 0x7fefdc19bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:15:03.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.126+0000 7fefd3fff640 1 --2- 172.21.15.116:0/753555171 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefdc101420 0x7fefdc197a30 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fefc400d990 tx=0x7fefc400de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:03.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.126+0000 7fefd9ffb640 1 -- 172.21.15.116:0/753555171 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc401b070 con 0x7fefdc101420 2024-09-12T03:15:03.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.126+0000 7fefe2a57640 1 -- 172.21.15.116:0/753555171 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefdc19c790 con 0x7fefdc101420 2024-09-12T03:15:03.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.126+0000 7fefd9ffb640 1 -- 172.21.15.116:0/753555171 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefc400edf0 con 0x7fefdc101420 2024-09-12T03:15:03.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.126+0000 7fefd9ffb640 1 -- 172.21.15.116:0/753555171 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc4014870 con 0x7fefdc101420 2024-09-12T03:15:03.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.126+0000 7fefe2a57640 1 -- 172.21.15.116:0/753555171 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefdc06a790 con 0x7fefdc101420 2024-09-12T03:15:03.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.129+0000 7fefd9ffb640 1 -- 172.21.15.116:0/753555171 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fefc401e070 con 0x7fefdc101420 2024-09-12T03:15:03.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.130+0000 7fefe4e30740 1 Processor -- start 2024-09-12T03:15:03.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.130+0000 7fefd9ffb640 1 --2- 172.21.15.116:0/753555171 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fefc80a2b80 0x7fefc80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.130+0000 7fefe4e30740 1 -- start start 2024-09-12T03:15:03.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.130+0000 7fefe0ff4640 1 --2- 172.21.15.116:0/753555171 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fefc80a2b80 0x7fefc80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.130+0000 7fefd9ffb640 1 -- 172.21.15.116:0/753555171 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fefc40d06e0 con 0x7fefdc101420 2024-09-12T03:15:03.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.130+0000 7fefe4e30740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571e063c760 0x5571e063cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.130+0000 7fefe4e30740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5571e063d080 0x5571e0647400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.130+0000 7fefe4e30740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5571e0647940 0x5571e0649d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.130+0000 7fefe4e30740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5571e05a22a0 con 0x5571e0647940 2024-09-12T03:15:03.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.130+0000 7fefe4e30740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5571e0549b10 con 0x5571e063d080 2024-09-12T03:15:03.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.130+0000 7fefe4e30740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5571e059e260 con 0x5571e063c760 2024-09-12T03:15:03.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.131+0000 7fefdbfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571e063c760 0x5571e063cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.131+0000 7fefd3fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5571e063d080 0x5571e0647400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.131+0000 7fefd3fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5571e063d080 0x5571e0647400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.116:51988/0 (socket says 172.21.15.116:51988) 2024-09-12T03:15:03.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.131+0000 7fefd3fff640 1 -- 172.21.15.116:0/3100273051 learned_addr learned my addr 172.21.15.116:0/3100273051 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:03.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.131+0000 7fefe0ff4640 1 --2- 172.21.15.116:0/3100273051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5571e0647940 0x5571e0649d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.131+0000 7fefe0ff4640 1 --2- 172.21.15.116:0/753555171 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fefc80a2b80 0x7fefc80a5040 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009f20 tx=0x7fefd40094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:03.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.131+0000 7fefd3fff640 1 -- 172.21.15.116:0/3100273051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571e063c760 msgr2=0x5571e063cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:03.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.131+0000 7fefd3fff640 1 --2- 172.21.15.116:0/3100273051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571e063c760 0x5571e063cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.131+0000 7fefd3fff640 1 -- 172.21.15.116:0/3100273051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5571e0647940 msgr2=0x5571e0649d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:03.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.131+0000 7fefd3fff640 1 --2- 172.21.15.116:0/3100273051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5571e0647940 0x5571e0649d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.131+0000 7fefd3fff640 1 -- 172.21.15.116:0/3100273051 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571e0313b90 con 0x5571e063d080 2024-09-12T03:15:03.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.131+0000 7fefe0ff4640 1 --2- 172.21.15.116:0/3100273051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5571e0647940 0x5571e0649d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:15:03.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.131+0000 7fefd3fff640 1 --2- 172.21.15.116:0/3100273051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5571e063d080 0x5571e0647400 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fefc409cfb0 tx=0x7fefc40047d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd8735d0f9a25103 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:03.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.131+0000 7fefd27fc640 1 -- 172.21.15.116:0/3100273051 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc40b7070 con 0x5571e063d080 2024-09-12T03:15:03.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.132+0000 7fefd27fc640 1 -- 172.21.15.116:0/3100273051 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefc409b320 con 0x5571e063d080 2024-09-12T03:15:03.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.132+0000 7fefd27fc640 1 -- 172.21.15.116:0/3100273051 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc40a59b0 con 0x5571e063d080 2024-09-12T03:15:03.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.132+0000 7fefe4e30740 1 -- 172.21.15.116:0/3100273051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5571e063d080 msgr2=0x5571e0647400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:03.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.132+0000 7fefe4e30740 1 --2- 172.21.15.116:0/3100273051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5571e063d080 0x5571e0647400 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fefc409cfb0 tx=0x7fefc40047d0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.132+0000 7fefe4e30740 1 -- 172.21.15.116:0/3100273051 shutdown_connections 2024-09-12T03:15:03.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.132+0000 7fefe4e30740 1 --2- 172.21.15.116:0/3100273051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5571e0647940 0x5571e0649d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.132+0000 7fefe4e30740 1 --2- 172.21.15.116:0/3100273051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5571e063d080 0x5571e0647400 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.132+0000 7fefe4e30740 1 --2- 172.21.15.116:0/3100273051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571e063c760 0x5571e063cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.132+0000 7fefe4e30740 1 -- 172.21.15.116:0/3100273051 >> 172.21.15.116:0/3100273051 conn(0x5571e0638cf0 msgr2=0x5571e06390d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:03.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.132+0000 7fefe4e30740 1 -- 172.21.15.116:0/3100273051 shutdown_connections 2024-09-12T03:15:03.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.132+0000 7fefe4e30740 1 -- 172.21.15.116:0/3100273051 wait complete. 2024-09-12T03:15:03.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.133+0000 7fefe4e30740 1 -- start start 2024-09-12T03:15:03.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.133+0000 7fefe4e30740 1 Processor -- start 2024-09-12T03:15:03.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.133+0000 7fefe4e30740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5571e063c760 0x5571e0644850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.133+0000 7fefe4e30740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5571e063d080 0x5571e0644d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.133+0000 7fefe4e30740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571e0647940 0x5571e06452d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.133+0000 7fefe4e30740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5571e064a510 con 0x5571e063c760 2024-09-12T03:15:03.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.133+0000 7fefe4e30740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5571e052a6e0 con 0x5571e063d080 2024-09-12T03:15:03.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.133+0000 7fefe4e30740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5571e064a690 con 0x5571e0647940 2024-09-12T03:15:03.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.133+0000 7fefdbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5571e063c760 0x5571e0644850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.133+0000 7fefe4e30740 10 client.0 _ll_register_callbacks cb 0x5571e054ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:03.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.133+0000 7fefdbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5571e063c760 0x5571e0644850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47476/0 (socket says 172.21.15.116:47476) 2024-09-12T03:15:03.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.133+0000 7fefd3fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5571e063d080 0x5571e0644d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.133+0000 7fefdbfff640 1 -- 172.21.15.116:0/556958156 learned_addr learned my addr 172.21.15.116:0/556958156 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:03.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.134+0000 7fefe0ff4640 1 --2- 172.21.15.116:0/556958156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571e0647940 0x5571e06452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.134+0000 7fefdbfff640 1 -- 172.21.15.116:0/556958156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571e0647940 msgr2=0x5571e06452d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:03.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.134+0000 7fefdbfff640 1 --2- 172.21.15.116:0/556958156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571e0647940 0x5571e06452d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.134+0000 7fefdbfff640 1 -- 172.21.15.116:0/556958156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5571e063d080 msgr2=0x5571e0644d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:03.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.134+0000 7fefdbfff640 1 --2- 172.21.15.116:0/556958156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5571e063d080 0x5571e0644d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.134+0000 7fefdbfff640 1 -- 172.21.15.116:0/556958156 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571e0653a30 con 0x5571e063c760 2024-09-12T03:15:03.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.134+0000 7fefd3fff640 1 --2- 172.21.15.116:0/556958156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5571e063d080 0x5571e0644d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:15:03.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.134+0000 7fefdbfff640 1 --2- 172.21.15.116:0/556958156 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5571e063c760 0x5571e0644850 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fefcc02a940 tx=0x7fefcc031a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b04bf883d629453 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:03.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.134+0000 7fefbb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:15:03.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.134+0000 7fefbb7fe640 1 -- 172.21.15.116:0/556958156 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc035040 con 0x5571e063c760 2024-09-12T03:15:03.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.134+0000 7fefbb7fe640 1 -- 172.21.15.116:0/556958156 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fefcc004380 con 0x5571e063c760 2024-09-12T03:15:03.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.134+0000 7fefe4e30740 1 -- 172.21.15.116:0/556958156 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefc409b320 con 0x5571e063c760 2024-09-12T03:15:03.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.134+0000 7fefbb7fe640 1 -- 172.21.15.116:0/556958156 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fefcc0035f0 con 0x5571e063c760 2024-09-12T03:15:03.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.134+0000 7fefe4e30740 10 client.18666 fetch_fsmap learned FSMap version 25 2024-09-12T03:15:03.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.134+0000 7fefe4e30740 1 -- 172.21.15.116:0/556958156 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5571e06563a0 con 0x5571e063c760 2024-09-12T03:15:03.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.135+0000 7fefbb7fe640 1 -- 172.21.15.116:0/556958156 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefc409b320 con 0x5571e063c760 2024-09-12T03:15:03.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.135+0000 7fefbb7fe640 1 -- 172.21.15.116:0/556958156 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fefcc03d900 con 0x5571e063c760 2024-09-12T03:15:03.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.135+0000 7fefe4e30740 10 client.18666 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:15:03.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.135+0000 7fefe4e30740 10 client.18666 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:15:03.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.135+0000 7fefe4e30740 20 client.18666 populate_metadata read hostname 'smithi116' 2024-09-12T03:15:03.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.135+0000 7fefe4e30740 1 --2- 172.21.15.116:0/556958156 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5571e0657860 0x5571e0659cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.135+0000 7fefe4e30740 4 client.18666 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:15:03.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.136+0000 7fefe4e30740 1 -- 172.21.15.116:0/556958156 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571e05af9d0 con 0x5571e0657860 2024-09-12T03:15:03.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.136+0000 7fefd3fff640 1 --2- 172.21.15.116:0/556958156 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5571e0657860 0x5571e0659cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.136+0000 7fefbb7fe640 10 client.18666 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:03.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.136+0000 7fefd3fff640 1 --2- 172.21.15.116:0/556958156 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5571e0657860 0x5571e0659cf0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f1d14b74002f9a5 server_cookie=4348299ee0296eed in_seq=0 out_seq=0 2024-09-12T03:15:03.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.137+0000 7fefbb7fe640 1 -- 172.21.15.116:0/556958156 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5571e05af9d0 con 0x5571e0657860 2024-09-12T03:15:03.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.137+0000 7fefbb7fe640 10 client.18666 handle_command_reply: tid=0 2024-09-12T03:15:03.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.137+0000 7fefe4e30740 1 client.18666 shutdown 2024-09-12T03:15:03.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.137+0000 7fefe4e30740 1 -- 172.21.15.116:0/556958156 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5571e063c760 msgr2=0x5571e0644850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:03.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.137+0000 7fefe4e30740 1 --2- 172.21.15.116:0/556958156 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5571e063c760 0x5571e0644850 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fefcc02a940 tx=0x7fefcc031a60 comp rx=0 tx=0).stop 2024-09-12T03:15:03.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.137+0000 7fefe4e30740 1 -- 172.21.15.116:0/556958156 shutdown_connections 2024-09-12T03:15:03.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.137+0000 7fefe4e30740 1 --2- 172.21.15.116:0/556958156 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5571e0657860 0x5571e0659cf0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.137+0000 7fefe4e30740 1 --2- 172.21.15.116:0/556958156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571e0647940 0x5571e06452d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.137+0000 7fefe4e30740 1 --2- 172.21.15.116:0/556958156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5571e063d080 0x5571e0644d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.137+0000 7fefe4e30740 1 --2- 172.21.15.116:0/556958156 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5571e063c760 0x5571e0644850 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.137+0000 7fefe4e30740 1 -- 172.21.15.116:0/556958156 >> 172.21.15.116:0/556958156 conn(0x5571e0638cf0 msgr2=0x5571e0637e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:03.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.137+0000 7fefbb7fe640 0 client.18666 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:03.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.137+0000 7fefe4e30740 1 -- 172.21.15.116:0/556958156 shutdown_connections 2024-09-12T03:15:03.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.137+0000 7fefe4e30740 1 -- 172.21.15.116:0/556958156 wait complete. 2024-09-12T03:15:03.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.137+0000 7fefe4e30740 20 client.18666 trim_cache size 0 max 16384 2024-09-12T03:15:03.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.155+0000 7fefe4e30740 1 Processor -- start 2024-09-12T03:15:03.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.155+0000 7fefe4e30740 1 -- start start 2024-09-12T03:15:03.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.155+0000 7fefe4e30740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5571e063c760 0x5571e0644130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.155+0000 7fefe4e30740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571e063d080 0x5571e0644f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.155+0000 7fefe4e30740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5571e0647940 0x5571e0645460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.155+0000 7fefe4e30740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc1adc60 con 0x5571e0647940 2024-09-12T03:15:03.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.155+0000 7fefe4e30740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc1ab530 con 0x5571e063c760 2024-09-12T03:15:03.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.155+0000 7fefe4e30740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc1a93a0 con 0x5571e063d080 2024-09-12T03:15:03.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.155+0000 7fefe0ff4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5571e0647940 0x5571e0645460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.155+0000 7fefe0ff4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5571e0647940 0x5571e0645460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47482/0 (socket says 172.21.15.116:47482) 2024-09-12T03:15:03.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.155+0000 7fefe0ff4640 1 -- 172.21.15.116:0/2915162583 learned_addr learned my addr 172.21.15.116:0/2915162583 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:03.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.155+0000 7fefdbfff640 1 --2- 172.21.15.116:0/2915162583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5571e063c760 0x5571e0644130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.156+0000 7fefd3fff640 1 --2- 172.21.15.116:0/2915162583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571e063d080 0x5571e0644f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.156+0000 7fefe0ff4640 1 -- 172.21.15.116:0/2915162583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571e063d080 msgr2=0x5571e0644f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:03.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.156+0000 7fefe0ff4640 1 --2- 172.21.15.116:0/2915162583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571e063d080 0x5571e0644f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.156+0000 7fefe0ff4640 1 -- 172.21.15.116:0/2915162583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5571e063c760 msgr2=0x5571e0644130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:03.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.156+0000 7fefe0ff4640 1 --2- 172.21.15.116:0/2915162583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5571e063c760 0x5571e0644130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.156+0000 7fefe0ff4640 1 -- 172.21.15.116:0/2915162583 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571e063ceb0 con 0x5571e0647940 2024-09-12T03:15:03.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.156+0000 7fefe0ff4640 1 --2- 172.21.15.116:0/2915162583 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5571e0647940 0x5571e0645460 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fefd40177d0 tx=0x7fefd4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2e6b453c510721 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:03.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.156+0000 7fefd1ffb640 1 -- 172.21.15.116:0/2915162583 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4027070 con 0x5571e0647940 2024-09-12T03:15:03.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.156+0000 7fefd1ffb640 1 -- 172.21.15.116:0/2915162583 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefd4022940 con 0x5571e0647940 2024-09-12T03:15:03.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.156+0000 7fefd1ffb640 1 -- 172.21.15.116:0/2915162583 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4016ae0 con 0x5571e0647940 2024-09-12T03:15:03.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.157+0000 7fefe4e30740 1 -- 172.21.15.116:0/2915162583 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5571e0647940 msgr2=0x5571e0645460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:03.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.157+0000 7fefe4e30740 1 --2- 172.21.15.116:0/2915162583 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5571e0647940 0x5571e0645460 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fefd40177d0 tx=0x7fefd4013000 comp rx=0 tx=0).stop 2024-09-12T03:15:03.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.157+0000 7fefe4e30740 1 -- 172.21.15.116:0/2915162583 shutdown_connections 2024-09-12T03:15:03.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.157+0000 7fefe4e30740 1 --2- 172.21.15.116:0/2915162583 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5571e0647940 0x5571e0645460 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.157+0000 7fefe4e30740 1 --2- 172.21.15.116:0/2915162583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571e063d080 0x5571e0644f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.157+0000 7fefe4e30740 1 --2- 172.21.15.116:0/2915162583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5571e063c760 0x5571e0644130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.157+0000 7fefe4e30740 1 -- 172.21.15.116:0/2915162583 >> 172.21.15.116:0/2915162583 conn(0x5571e0638cf0 msgr2=0x5571e0644810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:03.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.157+0000 7fefe4e30740 1 -- 172.21.15.116:0/2915162583 shutdown_connections 2024-09-12T03:15:03.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.157+0000 7fefe4e30740 1 -- 172.21.15.116:0/2915162583 wait complete. 2024-09-12T03:15:03.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.157+0000 7fefe4e30740 1 -- start start 2024-09-12T03:15:03.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.158+0000 7fefe4e30740 1 Processor -- start 2024-09-12T03:15:03.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.158+0000 7fefe4e30740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5571e063c760 0x5571e064fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.158+0000 7fefe4e30740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5571e063d080 0x5571e06bd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.158+0000 7fefe4e30740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571e0647940 0x5571e06bde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.158+0000 7fefe4e30740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc1a99f0 con 0x5571e063d080 2024-09-12T03:15:03.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.158+0000 7fefe4e30740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc1aae60 con 0x5571e063c760 2024-09-12T03:15:03.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.158+0000 7fefe4e30740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc1aa0c0 con 0x5571e0647940 2024-09-12T03:15:03.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.158+0000 7fefe4e30740 10 client.0 _ll_register_callbacks cb 0x7fefcc0316f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:03.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.158+0000 7fefd3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5571e063d080 0x5571e06bd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.158+0000 7fefd3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5571e063d080 0x5571e06bd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47490/0 (socket says 172.21.15.116:47490) 2024-09-12T03:15:03.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.158+0000 7fefdbfff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5571e063c760 0x5571e064fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.158+0000 7fefd3fff640 1 -- 172.21.15.116:0/1002792719 learned_addr learned my addr 172.21.15.116:0/1002792719 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:03.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.158+0000 7fefe0ff4640 1 --2- 172.21.15.116:0/1002792719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571e0647940 0x5571e06bde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.159+0000 7fefd3fff640 1 -- 172.21.15.116:0/1002792719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571e0647940 msgr2=0x5571e06bde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:03.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.159+0000 7fefd3fff640 1 --2- 172.21.15.116:0/1002792719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571e0647940 0x5571e06bde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.159+0000 7fefd3fff640 1 -- 172.21.15.116:0/1002792719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5571e063c760 msgr2=0x5571e064fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:03.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.159+0000 7fefd3fff640 1 --2- 172.21.15.116:0/1002792719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5571e063c760 0x5571e064fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.159+0000 7fefd3fff640 1 -- 172.21.15.116:0/1002792719 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571e06be3b0 con 0x5571e063d080 2024-09-12T03:15:03.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.159+0000 7fefd3fff640 1 --2- 172.21.15.116:0/1002792719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5571e063d080 0x5571e06bd930 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fefc400e9b0 tx=0x7fefc40a14a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c72aa97e69ccd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:03.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.159+0000 7fefbb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:15:03.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.159+0000 7fefbb7fe640 1 -- 172.21.15.116:0/1002792719 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc40b7070 con 0x5571e063d080 2024-09-12T03:15:03.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.159+0000 7fefbb7fe640 1 -- 172.21.15.116:0/1002792719 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fefc409b320 con 0x5571e063d080 2024-09-12T03:15:03.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.159+0000 7fefe4e30740 1 -- 172.21.15.116:0/1002792719 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefd4022940 con 0x5571e063d080 2024-09-12T03:15:03.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.159+0000 7fefbb7fe640 1 -- 172.21.15.116:0/1002792719 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fefc4016bd0 con 0x5571e063d080 2024-09-12T03:15:03.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.159+0000 7fefe4e30740 10 client.18672 fetch_fsmap learned FSMap version 25 2024-09-12T03:15:03.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.159+0000 7fefe4e30740 1 -- 172.21.15.116:0/1002792719 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5571e06bbca0 con 0x5571e063d080 2024-09-12T03:15:03.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.160+0000 7fefbb7fe640 1 -- 172.21.15.116:0/1002792719 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefd4022940 con 0x5571e063d080 2024-09-12T03:15:03.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.160+0000 7fefbb7fe640 1 -- 172.21.15.116:0/1002792719 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fefc400e040 con 0x5571e063d080 2024-09-12T03:15:03.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.160+0000 7fefe4e30740 10 client.18672 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:15:03.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.160+0000 7fefe4e30740 10 client.18672 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:15:03.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.160+0000 7fefe4e30740 20 client.18672 populate_metadata read hostname 'smithi116' 2024-09-12T03:15:03.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.160+0000 7fefe4e30740 1 --2- 172.21.15.116:0/1002792719 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5571e06cb1e0 0x5571e06cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.160+0000 7fefe4e30740 4 client.18672 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.2, "await_for": 45.0}] 2024-09-12T03:15:03.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.160+0000 7fefe4e30740 1 -- 172.21.15.116:0/1002792719 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.2, "await_for": 45.0}) -- 0x5571e05a9d20 con 0x5571e06cb1e0 2024-09-12T03:15:03.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.161+0000 7fefdbfff640 1 --2- 172.21.15.116:0/1002792719 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5571e06cb1e0 0x5571e06cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.161+0000 7fefbb7fe640 10 client.18672 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:03.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.161+0000 7fefdbfff640 1 --2- 172.21.15.116:0/1002792719 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5571e06cb1e0 0x5571e06cd5d0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5ada7aaa1022274 server_cookie=68b54a48896c461b in_seq=0 out_seq=0 2024-09-12T03:15:03.165 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:15:03.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.170+0000 7fefbb7fe640 1 -- 172.21.15.116:0/1002792719 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5571e05a9d20 con 0x5571e06cb1e0 2024-09-12T03:15:03.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.170+0000 7fefbb7fe640 10 client.18672 handle_command_reply: tid=0 2024-09-12T03:15:03.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.170+0000 7fefe4e30740 1 client.18672 shutdown 2024-09-12T03:15:03.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.170+0000 7fefe4e30740 1 -- 172.21.15.116:0/1002792719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5571e063d080 msgr2=0x5571e06bd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:03.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.170+0000 7fefe4e30740 1 --2- 172.21.15.116:0/1002792719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5571e063d080 0x5571e06bd930 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fefc400e9b0 tx=0x7fefc40a14a0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.170+0000 7fefe4e30740 1 -- 172.21.15.116:0/1002792719 shutdown_connections 2024-09-12T03:15:03.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.170+0000 7fefe4e30740 1 --2- 172.21.15.116:0/1002792719 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5571e06cb1e0 0x5571e06cd5d0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.170+0000 7fefe4e30740 1 --2- 172.21.15.116:0/1002792719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571e0647940 0x5571e06bde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.170+0000 7fefbb7fe640 0 client.18672 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:03.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.170+0000 7fefe4e30740 1 --2- 172.21.15.116:0/1002792719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5571e063d080 0x5571e06bd930 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.170+0000 7fefe4e30740 1 --2- 172.21.15.116:0/1002792719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5571e063c760 0x5571e064fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.170+0000 7fefe4e30740 1 -- 172.21.15.116:0/1002792719 >> 172.21.15.116:0/1002792719 conn(0x5571e0638cf0 msgr2=0x5571e0644fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:03.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.170+0000 7fefe4e30740 1 -- 172.21.15.116:0/1002792719 shutdown_connections 2024-09-12T03:15:03.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.171+0000 7fefe4e30740 1 -- 172.21.15.116:0/1002792719 wait complete. 2024-09-12T03:15:03.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.171+0000 7fefe4e30740 20 client.18672 trim_cache size 0 max 16384 2024-09-12T03:15:03.172 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:15:03.172 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 25, 2024-09-12T03:15:03.172 INFO:teuthology.orchestra.run.smithi116.stdout: "leader": 24494, 2024-09-12T03:15:03.172 INFO:teuthology.orchestra.run.smithi116.stdout: "set_version": 46, 2024-09-12T03:15:03.172 INFO:teuthology.orchestra.run.smithi116.stdout: "sets": { 2024-09-12T03:15:03.172 INFO:teuthology.orchestra.run.smithi116.stdout: "935d5531": { 2024-09-12T03:15:03.172 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 46, 2024-09-12T03:15:03.172 INFO:teuthology.orchestra.run.smithi116.stdout: "age_ref": 0.0, 2024-09-12T03:15:03.173 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:15:03.173 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "QUIESCED", 2024-09-12T03:15:03.173 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:15:03.173 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:15:03.173 INFO:teuthology.orchestra.run.smithi116.stdout: "timeout": 90.0, 2024-09-12T03:15:03.173 INFO:teuthology.orchestra.run.smithi116.stdout: "expiration": 147.2, 2024-09-12T03:15:03.173 INFO:teuthology.orchestra.run.smithi116.stdout: "members": { 2024-09-12T03:15:03.173 INFO:teuthology.orchestra.run.smithi116.stdout: "file:/": { 2024-09-12T03:15:03.173 INFO:teuthology.orchestra.run.smithi116.stdout: "excluded": false, 2024-09-12T03:15:03.173 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:15:03.173 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "QUIESCED", 2024-09-12T03:15:03.173 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:15:03.173 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:15:03.174 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:15:03.174 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:15:03.174 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:15:03.174 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:15:03.174 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:15:03.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.171+0000 7fefe2a57640 1 -- 172.21.15.116:0/753555171 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fefc80a2b80 msgr2=0x7fefc80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:03.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.171+0000 7fefe2a57640 1 --2- 172.21.15.116:0/753555171 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fefc80a2b80 0x7fefc80a5040 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009f20 tx=0x7fefd40094f0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.171+0000 7fefe2a57640 1 -- 172.21.15.116:0/753555171 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefdc101420 msgr2=0x7fefdc197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:03.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.171+0000 7fefe2a57640 1 --2- 172.21.15.116:0/753555171 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefdc101420 0x7fefdc197a30 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fefc400d990 tx=0x7fefc400de50 comp rx=0 tx=0).stop 2024-09-12T03:15:03.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.171+0000 7fefe2a57640 1 -- 172.21.15.116:0/753555171 shutdown_connections 2024-09-12T03:15:03.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.171+0000 7fefe2a57640 1 --2- 172.21.15.116:0/753555171 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fefc80a2b80 0x7fefc80a5040 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.171+0000 7fefe2a57640 1 --2- 172.21.15.116:0/753555171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefdc106a80 0x7fefdc19bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.171+0000 7fefe2a57640 1 --2- 172.21.15.116:0/753555171 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefdc101420 0x7fefdc197a30 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.171+0000 7fefe2a57640 1 --2- 172.21.15.116:0/753555171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefdc100a80 0x7fefdc1974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.171+0000 7fefe2a57640 1 -- 172.21.15.116:0/753555171 >> 172.21.15.116:0/753555171 conn(0x7fefdc0fc7a0 msgr2=0x7fefdc0fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:03.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.171+0000 7fefe2a57640 1 -- 172.21.15.116:0/753555171 shutdown_connections 2024-09-12T03:15:03.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.171+0000 7fefe2a57640 1 -- 172.21.15.116:0/753555171 wait complete. 2024-09-12T03:15:03.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:03 smithi183 ceph-mon[35535]: pgmap v700: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:15:03.707 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:03 smithi116 ceph-mon[31410]: pgmap v700: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:15:03.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:03 smithi177 ceph-mon[35305]: pgmap v700: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:15:03.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.914+0000 7f50f1229640 1 -- 172.21.15.116:0/1694211143 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50ec100e20 msgr2=0x7f50ec101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:03.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.914+0000 7f50f1229640 1 --2- 172.21.15.116:0/1694211143 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50ec100e20 0x7f50ec101200 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009ca0 tx=0x7f50e402a950 comp rx=0 tx=0).stop 2024-09-12T03:15:03.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.914+0000 7f50f1229640 1 -- 172.21.15.116:0/1694211143 shutdown_connections 2024-09-12T03:15:03.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.914+0000 7f50f1229640 1 --2- 172.21.15.116:0/1694211143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50ec102ef0 0x7f50ec10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.914+0000 7f50f1229640 1 --2- 172.21.15.116:0/1694211143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50ec102570 0x7f50ec1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.914+0000 7f50f1229640 1 --2- 172.21.15.116:0/1694211143 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50ec100e20 0x7f50ec101200 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.914+0000 7f50f1229640 1 -- 172.21.15.116:0/1694211143 >> 172.21.15.116:0/1694211143 conn(0x7f50ec0fc7d0 msgr2=0x7f50ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:03.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.914+0000 7f50f1229640 1 -- 172.21.15.116:0/1694211143 shutdown_connections 2024-09-12T03:15:03.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.914+0000 7f50f1229640 1 -- 172.21.15.116:0/1694211143 wait complete. 2024-09-12T03:15:03.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.915+0000 7f50f1229640 1 Processor -- start 2024-09-12T03:15:03.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.915+0000 7f50f1229640 1 -- start start 2024-09-12T03:15:03.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.916+0000 7f50f1229640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50ec100e20 0x7f50ec1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.916+0000 7f50f1229640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50ec102570 0x7f50ec198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.916+0000 7f50f1229640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50ec102ef0 0x7f50ec19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.916+0000 7f50f1229640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec110f50 con 0x7f50ec102ef0 2024-09-12T03:15:03.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.916+0000 7f50f1229640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec110dd0 con 0x7f50ec100e20 2024-09-12T03:15:03.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.916+0000 7f50f1229640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec1110d0 con 0x7f50ec102570 2024-09-12T03:15:03.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.916+0000 7f50eb59e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50ec102ef0 0x7f50ec19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.916+0000 7f50eb59e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50ec102ef0 0x7f50ec19cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47510/0 (socket says 172.21.15.116:47510) 2024-09-12T03:15:03.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.916+0000 7f50eb59e640 1 -- 172.21.15.116:0/84393059 learned_addr learned my addr 172.21.15.116:0/84393059 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:03.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.916+0000 7f50ea59c640 1 --2- 172.21.15.116:0/84393059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50ec102570 0x7f50ec198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.916+0000 7f50ead9d640 1 --2- 172.21.15.116:0/84393059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50ec100e20 0x7f50ec1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.916+0000 7f50eb59e640 1 -- 172.21.15.116:0/84393059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50ec102570 msgr2=0x7f50ec198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:03.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.916+0000 7f50eb59e640 1 --2- 172.21.15.116:0/84393059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50ec102570 0x7f50ec198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.916+0000 7f50eb59e640 1 -- 172.21.15.116:0/84393059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50ec100e20 msgr2=0x7f50ec1984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:03.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.916+0000 7f50eb59e640 1 --2- 172.21.15.116:0/84393059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50ec100e20 0x7f50ec1984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.916+0000 7f50eb59e640 1 -- 172.21.15.116:0/84393059 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50ec19d470 con 0x7f50ec102ef0 2024-09-12T03:15:03.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.916+0000 7f50ead9d640 1 --2- 172.21.15.116:0/84393059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50ec100e20 0x7f50ec1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:15:03.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.916+0000 7f50ea59c640 1 --2- 172.21.15.116:0/84393059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50ec102570 0x7f50ec198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:15:03.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.916+0000 7f50eb59e640 1 --2- 172.21.15.116:0/84393059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50ec102ef0 0x7f50ec19cd90 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f50dc00d750 tx=0x7f50dc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:03.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.916+0000 7f50e3fff640 1 -- 172.21.15.116:0/84393059 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50dc017070 con 0x7f50ec102ef0 2024-09-12T03:15:03.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.916+0000 7f50f1229640 1 -- 172.21.15.116:0/84393059 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50ec19d760 con 0x7f50ec102ef0 2024-09-12T03:15:03.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.916+0000 7f50e3fff640 1 -- 172.21.15.116:0/84393059 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50dc005500 con 0x7f50ec102ef0 2024-09-12T03:15:03.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.916+0000 7f50e3fff640 1 -- 172.21.15.116:0/84393059 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50dc004d90 con 0x7f50ec102ef0 2024-09-12T03:15:03.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.917+0000 7f50f1229640 1 -- 172.21.15.116:0/84393059 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50ec1a5040 con 0x7f50ec102ef0 2024-09-12T03:15:03.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.919+0000 7f50e3fff640 1 -- 172.21.15.116:0/84393059 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f50dc004f30 con 0x7f50ec102ef0 2024-09-12T03:15:03.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.919+0000 7f50f3602740 1 Processor -- start 2024-09-12T03:15:03.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.919+0000 7f50f3602740 1 -- start start 2024-09-12T03:15:03.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.920+0000 7f50f3602740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f61eb82b40 0x55f61eb81090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.920+0000 7f50f3602740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f61eb815d0 0x55f61eb81a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.920+0000 7f50f3602740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f61eb90780 0x55f61eb92b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.920+0000 7f50f3602740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55f61eae72f0 con 0x55f61eb82b40 2024-09-12T03:15:03.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.920+0000 7f50f3602740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f61ea8ecf0 con 0x55f61eb90780 2024-09-12T03:15:03.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.920+0000 7f50f3602740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f61eae32b0 con 0x55f61eb815d0 2024-09-12T03:15:03.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.920+0000 7f50ea59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f61eb82b40 0x55f61eb81090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.920+0000 7f50ea59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f61eb82b40 0x55f61eb81090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47514/0 (socket says 172.21.15.116:47514) 2024-09-12T03:15:03.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.920+0000 7f50ea59c640 1 -- 172.21.15.116:0/47558339 learned_addr learned my addr 172.21.15.116:0/47558339 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:03.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.920+0000 7f50ead9d640 1 --2- 172.21.15.116:0/47558339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f61eb815d0 0x55f61eb81a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.920+0000 7f50ea59c640 1 -- 172.21.15.116:0/47558339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f61eb815d0 msgr2=0x55f61eb81a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:03.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.920+0000 7f50ea59c640 1 --2- 172.21.15.116:0/47558339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f61eb815d0 0x55f61eb81a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.920+0000 7f50ea59c640 1 -- 172.21.15.116:0/47558339 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f61eb90780 msgr2=0x55f61eb92b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:03.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.920+0000 7f50e3fff640 1 --2- 172.21.15.116:0/84393059 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f50d80a2b80 0x7f50d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.920+0000 7f50ea59c640 1 --2- 172.21.15.116:0/47558339 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f61eb90780 0x55f61eb92b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.920+0000 7f50ea59c640 1 -- 172.21.15.116:0/47558339 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f61ea733a0 con 0x55f61eb82b40 2024-09-12T03:15:03.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.920+0000 7f50eb59e640 1 --2- 172.21.15.116:0/84393059 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f50d80a2b80 0x7f50d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.920+0000 7f50ea59c640 1 --2- 172.21.15.116:0/47558339 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f61eb82b40 0x55f61eb81090 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f50d4008cc0 tx=0x7f50d4010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d11c80320a819c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:03.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.921+0000 7f50e3fff640 1 -- 172.21.15.116:0/84393059 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f50dc0cea90 con 0x7f50ec102ef0 2024-09-12T03:15:03.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.921+0000 7f50e17fa640 1 -- 172.21.15.116:0/47558339 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d4016070 con 0x55f61eb82b40 2024-09-12T03:15:03.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.921+0000 7f50e17fa640 1 -- 172.21.15.116:0/47558339 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50d4005700 con 0x55f61eb82b40 2024-09-12T03:15:03.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.921+0000 7f50e17fa640 1 -- 172.21.15.116:0/47558339 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d40127b0 con 0x55f61eb82b40 2024-09-12T03:15:03.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.921+0000 7f50f3602740 1 -- 172.21.15.116:0/47558339 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f61eb82b40 msgr2=0x55f61eb81090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:03.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.921+0000 7f50eb59e640 1 --2- 172.21.15.116:0/84393059 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f50d80a2b80 0x7f50d80a5040 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f50dc097c90 tx=0x7f50dc06e380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:03.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.921+0000 7f50f3602740 1 --2- 172.21.15.116:0/47558339 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f61eb82b40 0x55f61eb81090 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f50d4008cc0 tx=0x7f50d4010740 comp rx=0 tx=0).stop 2024-09-12T03:15:03.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.922+0000 7f50f3602740 1 -- 172.21.15.116:0/47558339 shutdown_connections 2024-09-12T03:15:03.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.922+0000 7f50f3602740 1 --2- 172.21.15.116:0/47558339 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f61eb90780 0x55f61eb92b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.922+0000 7f50f3602740 1 --2- 172.21.15.116:0/47558339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f61eb815d0 0x55f61eb81a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.922+0000 7f50f3602740 1 --2- 172.21.15.116:0/47558339 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f61eb82b40 0x55f61eb81090 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.922+0000 7f50f3602740 1 -- 172.21.15.116:0/47558339 >> 172.21.15.116:0/47558339 conn(0x55f61eb4a310 msgr2=0x55f61eb7dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:03.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.922+0000 7f50f3602740 1 -- 172.21.15.116:0/47558339 shutdown_connections 2024-09-12T03:15:03.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.922+0000 7f50f3602740 1 -- 172.21.15.116:0/47558339 wait complete. 2024-09-12T03:15:03.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.922+0000 7f50f3602740 1 -- start start 2024-09-12T03:15:03.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.922+0000 7f50f3602740 1 Processor -- start 2024-09-12T03:15:03.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.922+0000 7f50f3602740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f61eb815d0 0x55f61eb90140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.922+0000 7f50f3602740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f61eb82b40 0x55f61eb8d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.922+0000 7f50f3602740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f61eb90780 0x55f61eb8d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.922+0000 7f50f3602740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55f61eb93480 con 0x55f61eb82b40 2024-09-12T03:15:03.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.922+0000 7f50f3602740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f61eaa2060 con 0x55f61eb90780 2024-09-12T03:15:03.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.922+0000 7f50f3602740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f61eb93600 con 0x55f61eb815d0 2024-09-12T03:15:03.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.922+0000 7f50f3602740 10 client.0 _ll_register_callbacks cb 0x55f61e7f58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:03.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.922+0000 7f50ead9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f61eb82b40 0x55f61eb8d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.922+0000 7f50ead9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f61eb82b40 0x55f61eb8d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47526/0 (socket says 172.21.15.116:47526) 2024-09-12T03:15:03.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.922+0000 7f50ead9d640 1 -- 172.21.15.116:0/660459225 learned_addr learned my addr 172.21.15.116:0/660459225 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:03.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.922+0000 7f50ea59c640 1 --2- 172.21.15.116:0/660459225 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f61eb90780 0x55f61eb8d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.923+0000 7f50ea59c640 1 --2- 172.21.15.116:0/660459225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f61eb815d0 0x55f61eb90140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.923+0000 7f50ead9d640 1 -- 172.21.15.116:0/660459225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f61eb815d0 msgr2=0x55f61eb90140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:03.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.923+0000 7f50ead9d640 1 --2- 172.21.15.116:0/660459225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f61eb815d0 0x55f61eb90140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.923+0000 7f50ead9d640 1 -- 172.21.15.116:0/660459225 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f61eb90780 msgr2=0x55f61eb8d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:03.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.923+0000 7f50ead9d640 1 --2- 172.21.15.116:0/660459225 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f61eb90780 0x55f61eb8d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.923+0000 7f50ead9d640 1 -- 172.21.15.116:0/660459225 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f61e8583c0 con 0x55f61eb82b40 2024-09-12T03:15:03.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.923+0000 7f50ea59c640 1 --2- 172.21.15.116:0/660459225 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f61eb90780 0x55f61eb8d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:15:03.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.923+0000 7f50ead9d640 1 --2- 172.21.15.116:0/660459225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f61eb82b40 0x55f61eb8d3f0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f50e4006010 tx=0x7f50e402f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238dd70f5c0d7f1a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:03.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.923+0000 7f50d27fc640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:15:03.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.923+0000 7f50d27fc640 1 -- 172.21.15.116:0/660459225 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4034050 con 0x55f61eb82b40 2024-09-12T03:15:03.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.923+0000 7f50d27fc640 1 -- 172.21.15.116:0/660459225 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f50e4003b70 con 0x55f61eb82b40 2024-09-12T03:15:03.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.923+0000 7f50f3602740 1 -- 172.21.15.116:0/660459225 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50d4005700 con 0x55f61eb82b40 2024-09-12T03:15:03.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.923+0000 7f50d27fc640 1 -- 172.21.15.116:0/660459225 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f50e4003d70 con 0x55f61eb82b40 2024-09-12T03:15:03.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.923+0000 7f50f3602740 10 client.18684 fetch_fsmap learned FSMap version 25 2024-09-12T03:15:03.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.923+0000 7f50f3602740 1 -- 172.21.15.116:0/660459225 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f61eb9ef30 con 0x55f61eb82b40 2024-09-12T03:15:03.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.923+0000 7f50d27fc640 1 -- 172.21.15.116:0/660459225 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50d4005700 con 0x55f61eb82b40 2024-09-12T03:15:03.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.923+0000 7f50d27fc640 1 -- 172.21.15.116:0/660459225 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f50e4044670 con 0x55f61eb82b40 2024-09-12T03:15:03.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.924+0000 7f50f3602740 10 client.18684 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:15:03.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.924+0000 7f50f3602740 10 client.18684 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:15:03.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.924+0000 7f50f3602740 20 client.18684 populate_metadata read hostname 'smithi116' 2024-09-12T03:15:03.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.924+0000 7f50f3602740 1 --2- 172.21.15.116:0/660459225 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55f61eba0440 0x55f61eba2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.924+0000 7f50f3602740 4 client.18684 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:15:03.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.924+0000 7f50f3602740 1 -- 172.21.15.116:0/660459225 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f61eaf4860 con 0x55f61eba0440 2024-09-12T03:15:03.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.924+0000 7f50ea59c640 1 --2- 172.21.15.116:0/660459225 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55f61eba0440 0x55f61eba2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.924+0000 7f50d27fc640 10 client.18684 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:15:03.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.924+0000 7f50ea59c640 1 --2- 172.21.15.116:0/660459225 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55f61eba0440 0x55f61eba2860 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bbe349b11e6befba server_cookie=65a8d31146d0666f in_seq=0 out_seq=0 2024-09-12T03:15:03.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.925+0000 7f50d27fc640 1 -- 172.21.15.116:0/660459225 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f61eaf4860 con 0x55f61eba0440 2024-09-12T03:15:03.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.925+0000 7f50d27fc640 10 client.18684 handle_command_reply: tid=0 2024-09-12T03:15:03.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.925+0000 7f50f3602740 1 client.18684 shutdown 2024-09-12T03:15:03.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.925+0000 7f50f3602740 1 -- 172.21.15.116:0/660459225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f61eb82b40 msgr2=0x55f61eb8d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:03.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.925+0000 7f50f3602740 1 --2- 172.21.15.116:0/660459225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f61eb82b40 0x55f61eb8d3f0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f50e4006010 tx=0x7f50e402f3a0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.925+0000 7f50f3602740 1 -- 172.21.15.116:0/660459225 shutdown_connections 2024-09-12T03:15:03.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.925+0000 7f50f3602740 1 --2- 172.21.15.116:0/660459225 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55f61eba0440 0x55f61eba2860 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.925+0000 7f50f3602740 1 --2- 172.21.15.116:0/660459225 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f61eb90780 0x55f61eb8d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.925+0000 7f50f3602740 1 --2- 172.21.15.116:0/660459225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f61eb82b40 0x55f61eb8d3f0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.925+0000 7f50d27fc640 0 client.18684 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:15:03.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.925+0000 7f50f3602740 1 --2- 172.21.15.116:0/660459225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f61eb815d0 0x55f61eb90140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.925+0000 7f50f3602740 1 -- 172.21.15.116:0/660459225 >> 172.21.15.116:0/660459225 conn(0x55f61eb4a310 msgr2=0x55f61eb83960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:03.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.925+0000 7f50f3602740 1 -- 172.21.15.116:0/660459225 shutdown_connections 2024-09-12T03:15:03.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.925+0000 7f50f3602740 1 -- 172.21.15.116:0/660459225 wait complete. 2024-09-12T03:15:03.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.925+0000 7f50f3602740 20 client.18684 trim_cache size 0 max 16384 2024-09-12T03:15:03.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.938+0000 7f50f3602740 1 Processor -- start 2024-09-12T03:15:03.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.938+0000 7f50f3602740 1 -- start start 2024-09-12T03:15:03.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.938+0000 7f50f3602740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f61eb815d0 0x55f61eb9c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.938+0000 7f50f3602740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f61eb82b40 0x55f61eb8c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.938+0000 7f50f3602740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f61eb90780 0x55f61eba8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.938+0000 7f50f3602740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec1adae0 con 0x55f61eb90780 2024-09-12T03:15:03.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.938+0000 7f50f3602740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec1ab3b0 con 0x55f61eb82b40 2024-09-12T03:15:03.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.938+0000 7f50f3602740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec1a9220 con 0x55f61eb815d0 2024-09-12T03:15:03.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.939+0000 7f50ea59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f61eb90780 0x55f61eba8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.939+0000 7f50ea59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f61eb90780 0x55f61eba8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47532/0 (socket says 172.21.15.116:47532) 2024-09-12T03:15:03.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.939+0000 7f50ea59c640 1 -- 172.21.15.116:0/2129436853 learned_addr learned my addr 172.21.15.116:0/2129436853 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:03.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.939+0000 7f50ea59c640 1 --2- 172.21.15.116:0/2129436853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f61eb815d0 0x55f61eb9c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.939+0000 7f50ead9d640 1 --2- 172.21.15.116:0/2129436853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f61eb82b40 0x55f61eb8c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.939+0000 7f50ea59c640 1 -- 172.21.15.116:0/2129436853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f61eb815d0 msgr2=0x55f61eb9c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:03.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.939+0000 7f50ea59c640 1 --2- 172.21.15.116:0/2129436853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f61eb815d0 0x55f61eb9c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.939+0000 7f50ea59c640 1 -- 172.21.15.116:0/2129436853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f61eb82b40 msgr2=0x55f61eb8c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:03.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.939+0000 7f50ea59c640 1 --2- 172.21.15.116:0/2129436853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f61eb82b40 0x55f61eb8c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.939+0000 7f50ea59c640 1 -- 172.21.15.116:0/2129436853 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f61eb904b0 con 0x55f61eb90780 2024-09-12T03:15:03.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.939+0000 7f50ea59c640 1 --2- 172.21.15.116:0/2129436853 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f61eb90780 0x55f61eba8130 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f50d4004e90 tx=0x7f50d4004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53baecb55b2aa973 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:03.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.939+0000 7f50e0ff9640 1 -- 172.21.15.116:0/2129436853 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d402f070 con 0x55f61eb90780 2024-09-12T03:15:03.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.939+0000 7f50e0ff9640 1 -- 172.21.15.116:0/2129436853 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50d40054b0 con 0x55f61eb90780 2024-09-12T03:15:03.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.939+0000 7f50e0ff9640 1 -- 172.21.15.116:0/2129436853 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d4016040 con 0x55f61eb90780 2024-09-12T03:15:03.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.940+0000 7f50f3602740 1 -- 172.21.15.116:0/2129436853 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f61eb90780 msgr2=0x55f61eba8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:03.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.940+0000 7f50f3602740 1 --2- 172.21.15.116:0/2129436853 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f61eb90780 0x55f61eba8130 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f50d4004e90 tx=0x7f50d4004f60 comp rx=0 tx=0).stop 2024-09-12T03:15:03.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.940+0000 7f50f3602740 1 -- 172.21.15.116:0/2129436853 shutdown_connections 2024-09-12T03:15:03.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.940+0000 7f50f3602740 1 --2- 172.21.15.116:0/2129436853 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f61eb90780 0x55f61eba8130 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.940+0000 7f50f3602740 1 --2- 172.21.15.116:0/2129436853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f61eb82b40 0x55f61eb8c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.940+0000 7f50f3602740 1 --2- 172.21.15.116:0/2129436853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f61eb815d0 0x55f61eb9c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.940+0000 7f50f3602740 1 -- 172.21.15.116:0/2129436853 >> 172.21.15.116:0/2129436853 conn(0x55f61eb4a310 msgr2=0x55f61eb8bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:03.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.940+0000 7f50f3602740 1 -- 172.21.15.116:0/2129436853 shutdown_connections 2024-09-12T03:15:03.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.940+0000 7f50f3602740 1 -- 172.21.15.116:0/2129436853 wait complete. 2024-09-12T03:15:03.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.940+0000 7f50f3602740 1 -- start start 2024-09-12T03:15:03.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.940+0000 7f50f3602740 1 Processor -- start 2024-09-12T03:15:03.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.941+0000 7f50f3602740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f61eb815d0 0x55f61eb991a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.941+0000 7f50f3602740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f61eb82b40 0x55f61eb94520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.941+0000 7f50f3602740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f61eb90780 0x55f61eb94cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.941+0000 7f50f3602740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec1a9870 con 0x55f61eb815d0 2024-09-12T03:15:03.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.941+0000 7f50f3602740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec1aace0 con 0x55f61eb82b40 2024-09-12T03:15:03.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.941+0000 7f50f3602740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec1a9f40 con 0x55f61eb90780 2024-09-12T03:15:03.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.941+0000 7f50f3602740 10 client.0 _ll_register_callbacks cb 0x7f50e402ff80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:03.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.941+0000 7f50ea59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f61eb815d0 0x55f61eb991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.941+0000 7f50ea59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f61eb815d0 0x55f61eb991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47548/0 (socket says 172.21.15.116:47548) 2024-09-12T03:15:03.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.941+0000 7f50ea59c640 1 -- 172.21.15.116:0/2299875047 learned_addr learned my addr 172.21.15.116:0/2299875047 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:03.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.941+0000 7f50ead9d640 1 --2- 172.21.15.116:0/2299875047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f61eb82b40 0x55f61eb94520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.941+0000 7f50ea59c640 1 -- 172.21.15.116:0/2299875047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f61eb90780 msgr2=0x55f61eb94cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:03.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.941+0000 7f50ea59c640 1 --2- 172.21.15.116:0/2299875047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f61eb90780 0x55f61eb94cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.941+0000 7f50ea59c640 1 -- 172.21.15.116:0/2299875047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f61eb82b40 msgr2=0x55f61eb94520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:03.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.941+0000 7f50ea59c640 1 --2- 172.21.15.116:0/2299875047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f61eb82b40 0x55f61eb94520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.941+0000 7f50ea59c640 1 -- 172.21.15.116:0/2299875047 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f61ec0a100 con 0x55f61eb815d0 2024-09-12T03:15:03.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.941+0000 7f50ea59c640 1 --2- 172.21.15.116:0/2299875047 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f61eb815d0 0x55f61eb991a0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f50d40128a0 tx=0x7f50d40128d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883f91ffea188b4d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:03.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.941+0000 7f50d27fc640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:15:03.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.941+0000 7f50d27fc640 1 -- 172.21.15.116:0/2299875047 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d4038030 con 0x55f61eb815d0 2024-09-12T03:15:03.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.941+0000 7f50d27fc640 1 -- 172.21.15.116:0/2299875047 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f50d4004910 con 0x55f61eb815d0 2024-09-12T03:15:03.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.941+0000 7f50f3602740 1 -- 172.21.15.116:0/2299875047 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50d40054b0 con 0x55f61eb815d0 2024-09-12T03:15:03.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.942+0000 7f50d27fc640 1 -- 172.21.15.116:0/2299875047 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f50d402f070 con 0x55f61eb815d0 2024-09-12T03:15:03.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.942+0000 7f50f3602740 10 client.18690 fetch_fsmap learned FSMap version 25 2024-09-12T03:15:03.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.942+0000 7f50f3602740 1 -- 172.21.15.116:0/2299875047 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f61ec0b2c0 con 0x55f61eb815d0 2024-09-12T03:15:03.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.942+0000 7f50d27fc640 1 -- 172.21.15.116:0/2299875047 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50d40054b0 con 0x55f61eb815d0 2024-09-12T03:15:03.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.942+0000 7f50d27fc640 1 -- 172.21.15.116:0/2299875047 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f50d402a040 con 0x55f61eb815d0 2024-09-12T03:15:03.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.942+0000 7f50f3602740 10 client.18690 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:15:03.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.942+0000 7f50f3602740 10 client.18690 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:15:03.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.942+0000 7f50f3602740 20 client.18690 populate_metadata read hostname 'smithi116' 2024-09-12T03:15:03.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.942+0000 7f50f3602740 1 --2- 172.21.15.116:0/2299875047 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55f61eba0440 0x55f61ec0d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:03.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.942+0000 7f50f3602740 4 client.18690 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:15:03.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.942+0000 7f50f3602740 1 -- 172.21.15.116:0/2299875047 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f61eaeed70 con 0x55f61eba0440 2024-09-12T03:15:03.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.942+0000 7f50ead9d640 1 --2- 172.21.15.116:0/2299875047 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55f61eba0440 0x55f61ec0d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:03.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.943+0000 7f50ead9d640 1 --2- 172.21.15.116:0/2299875047 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55f61eba0440 0x55f61ec0d7a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e21c92ef7b76c1cd server_cookie=d9ad0474f54e9897 in_seq=0 out_seq=0 2024-09-12T03:15:03.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.943+0000 7f50d27fc640 10 client.18690 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:15:03.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.943+0000 7f50d27fc640 1 -- 172.21.15.116:0/2299875047 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f61eaeed70 con 0x55f61eba0440 2024-09-12T03:15:03.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.943+0000 7f50d27fc640 10 client.18690 handle_command_reply: tid=0 2024-09-12T03:15:03.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.943+0000 7f50f3602740 1 client.18690 shutdown 2024-09-12T03:15:03.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.943+0000 7f50f3602740 1 -- 172.21.15.116:0/2299875047 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f61eb815d0 msgr2=0x55f61eb991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:03.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.943+0000 7f50f3602740 1 --2- 172.21.15.116:0/2299875047 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f61eb815d0 0x55f61eb991a0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f50d40128a0 tx=0x7f50d40128d0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.943+0000 7f50f3602740 1 -- 172.21.15.116:0/2299875047 shutdown_connections 2024-09-12T03:15:03.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.943+0000 7f50f3602740 1 --2- 172.21.15.116:0/2299875047 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55f61eba0440 0x55f61ec0d7a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.943+0000 7f50f3602740 1 --2- 172.21.15.116:0/2299875047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f61eb90780 0x55f61eb94cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.943+0000 7f50f3602740 1 --2- 172.21.15.116:0/2299875047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f61eb82b40 0x55f61eb94520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.943+0000 7f50d27fc640 0 client.18690 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:15:03.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.943+0000 7f50f3602740 1 --2- 172.21.15.116:0/2299875047 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f61eb815d0 0x55f61eb991a0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.943+0000 7f50f3602740 1 -- 172.21.15.116:0/2299875047 >> 172.21.15.116:0/2299875047 conn(0x55f61eb4a310 msgr2=0x55f61eb8c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:03.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.943+0000 7f50f3602740 1 -- 172.21.15.116:0/2299875047 shutdown_connections 2024-09-12T03:15:03.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.943+0000 7f50f3602740 1 -- 172.21.15.116:0/2299875047 wait complete. 2024-09-12T03:15:03.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.943+0000 7f50f3602740 20 client.18690 trim_cache size 0 max 16384 2024-09-12T03:15:03.950 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:15:03.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.944+0000 7f50f1229640 1 -- 172.21.15.116:0/84393059 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f50d80a2b80 msgr2=0x7f50d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:03.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.944+0000 7f50f1229640 1 --2- 172.21.15.116:0/84393059 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f50d80a2b80 0x7f50d80a5040 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f50dc097c90 tx=0x7f50dc06e380 comp rx=0 tx=0).stop 2024-09-12T03:15:03.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.944+0000 7f50f1229640 1 -- 172.21.15.116:0/84393059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50ec102ef0 msgr2=0x7f50ec19cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:03.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.944+0000 7f50f1229640 1 --2- 172.21.15.116:0/84393059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50ec102ef0 0x7f50ec19cd90 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f50dc00d750 tx=0x7f50dc00dc10 comp rx=0 tx=0).stop 2024-09-12T03:15:03.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.944+0000 7f50f1229640 1 -- 172.21.15.116:0/84393059 shutdown_connections 2024-09-12T03:15:03.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.944+0000 7f50f1229640 1 --2- 172.21.15.116:0/84393059 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f50d80a2b80 0x7f50d80a5040 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.944+0000 7f50f1229640 1 --2- 172.21.15.116:0/84393059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50ec102ef0 0x7f50ec19cd90 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.944+0000 7f50f1229640 1 --2- 172.21.15.116:0/84393059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50ec102570 0x7f50ec198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.944+0000 7f50f1229640 1 --2- 172.21.15.116:0/84393059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50ec100e20 0x7f50ec1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:03.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.944+0000 7f50f1229640 1 -- 172.21.15.116:0/84393059 >> 172.21.15.116:0/84393059 conn(0x7f50ec0fc7d0 msgr2=0x7f50ec06a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:03.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.944+0000 7f50f1229640 1 -- 172.21.15.116:0/84393059 shutdown_connections 2024-09-12T03:15:03.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:03.944+0000 7f50f1229640 1 -- 172.21.15.116:0/84393059 wait complete. 2024-09-12T03:15:04.863 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '935d5531', quiesce took 5.6 seconds. Will release after: 21.6 2024-09-12T03:15:05.453 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:15:05.702 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:15:05.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:05 smithi183 ceph-mon[35535]: pgmap v701: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:15:05.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:05 smithi177 ceph-mon[35305]: pgmap v701: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:15:05.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:05 smithi116 ceph-mon[31410]: pgmap v701: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:15:06.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.186+0000 7ff6eb4b9640 1 -- 172.21.15.116:0/3140949917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6e4101410 msgr2=0x7ff6e410d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:06.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.186+0000 7ff6eb4b9640 1 --2- 172.21.15.116:0/3140949917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6e4101410 0x7ff6e410d940 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc009930 tx=0x7ff6dc02a420 comp rx=0 tx=0).stop 2024-09-12T03:15:06.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.186+0000 7ff6eb4b9640 1 -- 172.21.15.116:0/3140949917 shutdown_connections 2024-09-12T03:15:06.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.186+0000 7ff6eb4b9640 1 --2- 172.21.15.116:0/3140949917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6e4101410 0x7ff6e410d940 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.186+0000 7ff6eb4b9640 1 --2- 172.21.15.116:0/3140949917 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6e4100a70 0x7ff6e4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.186+0000 7ff6eb4b9640 1 --2- 172.21.15.116:0/3140949917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6e4106a70 0x7ff6e4106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.186+0000 7ff6eb4b9640 1 -- 172.21.15.116:0/3140949917 >> 172.21.15.116:0/3140949917 conn(0x7ff6e40fc7d0 msgr2=0x7ff6e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:06.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.187+0000 7ff6eb4b9640 1 -- 172.21.15.116:0/3140949917 shutdown_connections 2024-09-12T03:15:06.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.187+0000 7ff6eb4b9640 1 -- 172.21.15.116:0/3140949917 wait complete. 2024-09-12T03:15:06.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.187+0000 7ff6eb4b9640 1 Processor -- start 2024-09-12T03:15:06.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.187+0000 7ff6eb4b9640 1 -- start start 2024-09-12T03:15:06.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.187+0000 7ff6eb4b9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6e4100a70 0x7ff6e4197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:06.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.187+0000 7ff6eb4b9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6e4101410 0x7ff6e41978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:06.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.187+0000 7ff6eb4b9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6e4106a70 0x7ff6e419bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:06.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.187+0000 7ff6eb4b9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e4110e90 con 0x7ff6e4100a70 2024-09-12T03:15:06.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.187+0000 7ff6eb4b9640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e4110d10 con 0x7ff6e4101410 2024-09-12T03:15:06.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.187+0000 7ff6eb4b9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e4111010 con 0x7ff6e4106a70 2024-09-12T03:15:06.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.188+0000 7ff6e9255640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6e4100a70 0x7ff6e4197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:06.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.188+0000 7ff6e9255640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6e4100a70 0x7ff6e4197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47570/0 (socket says 172.21.15.116:47570) 2024-09-12T03:15:06.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.188+0000 7ff6e9255640 1 -- 172.21.15.116:0/3745234293 learned_addr learned my addr 172.21.15.116:0/3745234293 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:06.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.188+0000 7ff6e8a54640 1 --2- 172.21.15.116:0/3745234293 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6e4101410 0x7ff6e41978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:06.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.188+0000 7ff6e9255640 1 -- 172.21.15.116:0/3745234293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6e4106a70 msgr2=0x7ff6e419bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:06.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.188+0000 7ff6e9255640 1 --2- 172.21.15.116:0/3745234293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6e4106a70 0x7ff6e419bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.188+0000 7ff6e9255640 1 -- 172.21.15.116:0/3745234293 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6e4101410 msgr2=0x7ff6e41978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:06.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.188+0000 7ff6e9255640 1 --2- 172.21.15.116:0/3745234293 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6e4101410 0x7ff6e41978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.188+0000 7ff6e9255640 1 -- 172.21.15.116:0/3745234293 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e419c320 con 0x7ff6e4100a70 2024-09-12T03:15:06.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.188+0000 7ff6e9255640 1 --2- 172.21.15.116:0/3745234293 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6e4100a70 0x7ff6e4197370 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7ff6d400ba50 tx=0x7ff6d400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:06.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.188+0000 7ff6e27fc640 1 -- 172.21.15.116:0/3745234293 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4019070 con 0x7ff6e4100a70 2024-09-12T03:15:06.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.188+0000 7ff6e27fc640 1 -- 172.21.15.116:0/3745234293 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6d400cbe0 con 0x7ff6e4100a70 2024-09-12T03:15:06.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.188+0000 7ff6e27fc640 1 -- 172.21.15.116:0/3745234293 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d40076e0 con 0x7ff6e4100a70 2024-09-12T03:15:06.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.188+0000 7ff6eb4b9640 1 -- 172.21.15.116:0/3745234293 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6e419c610 con 0x7ff6e4100a70 2024-09-12T03:15:06.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.188+0000 7ff6eb4b9640 1 -- 172.21.15.116:0/3745234293 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6e41a3ef0 con 0x7ff6e4100a70 2024-09-12T03:15:06.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.191+0000 7ff6e27fc640 1 -- 172.21.15.116:0/3745234293 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ff6d401c070 con 0x7ff6e4100a70 2024-09-12T03:15:06.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.191+0000 7ff6ed892740 1 Processor -- start 2024-09-12T03:15:06.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.191+0000 7ff6ed892740 1 -- start start 2024-09-12T03:15:06.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.191+0000 7ff6ed892740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c68c08d50 0x556c68c072a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:06.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.191+0000 7ff6ed892740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c68c077e0 0x556c68c07c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:06.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.191+0000 7ff6ed892740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c68c12580 0x556c68c14970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:06.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.191+0000 7ff6ed892740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556c68b6b2f0 con 0x556c68c077e0 2024-09-12T03:15:06.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.191+0000 7ff6ed892740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556c68b12cf0 con 0x556c68c12580 2024-09-12T03:15:06.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.191+0000 7ff6ed892740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556c68b672b0 con 0x556c68c08d50 2024-09-12T03:15:06.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.191+0000 7ff6e27fc640 1 --2- 172.21.15.116:0/3745234293 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff6cc0a2c00 0x7ff6cc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:06.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.191+0000 7ff6e9255640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c68c077e0 0x556c68c07c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:06.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.192+0000 7ff6e9255640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c68c077e0 0x556c68c07c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47584/0 (socket says 172.21.15.116:47584) 2024-09-12T03:15:06.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.192+0000 7ff6e9255640 1 -- 172.21.15.116:0/3500994841 learned_addr learned my addr 172.21.15.116:0/3500994841 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:06.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.192+0000 7ff6e9a56640 1 --2- 172.21.15.116:0/3745234293 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff6cc0a2c00 0x7ff6cc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:06.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.192+0000 7ff6e8a54640 1 --2- 172.21.15.116:0/3500994841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c68c12580 0x556c68c14970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:06.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.192+0000 7ff6e9a56640 1 --2- 172.21.15.116:0/3500994841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c68c08d50 0x556c68c072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:06.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.192+0000 7ff6e27fc640 1 -- 172.21.15.116:0/3745234293 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff6d40ce520 con 0x7ff6e4100a70 2024-09-12T03:15:06.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.192+0000 7ff6e9255640 1 -- 172.21.15.116:0/3500994841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c68c08d50 msgr2=0x556c68c072a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:06.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.192+0000 7ff6e9255640 1 --2- 172.21.15.116:0/3500994841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c68c08d50 0x556c68c072a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.192+0000 7ff6e9255640 1 -- 172.21.15.116:0/3500994841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c68c12580 msgr2=0x556c68c14970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:06.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.192+0000 7ff6e9255640 1 --2- 172.21.15.116:0/3500994841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c68c12580 0x556c68c14970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.192+0000 7ff6e9255640 1 -- 172.21.15.116:0/3500994841 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c68af73a0 con 0x556c68c077e0 2024-09-12T03:15:06.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.192+0000 7ff6e9a56640 1 --2- 172.21.15.116:0/3500994841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c68c08d50 0x556c68c072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:15:06.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.192+0000 7ff6e9255640 1 --2- 172.21.15.116:0/3500994841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c68c077e0 0x556c68c07c20 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40cc050 tx=0x7ff6d4014d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2a85bdbabcac95 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:06.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.192+0000 7ff6e9a56640 1 --2- 172.21.15.116:0/3745234293 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff6cc0a2c00 0x7ff6cc0a50c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc031550 tx=0x7ff6dc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:06.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.192+0000 7ff6cb7fe640 1 -- 172.21.15.116:0/3500994841 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d409ae60 con 0x556c68c077e0 2024-09-12T03:15:06.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.192+0000 7ff6cb7fe640 1 -- 172.21.15.116:0/3500994841 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6d409f160 con 0x556c68c077e0 2024-09-12T03:15:06.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.192+0000 7ff6cb7fe640 1 -- 172.21.15.116:0/3500994841 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d40a37f0 con 0x556c68c077e0 2024-09-12T03:15:06.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.193+0000 7ff6ed892740 1 -- 172.21.15.116:0/3500994841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c68c077e0 msgr2=0x556c68c07c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:06.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.193+0000 7ff6ed892740 1 --2- 172.21.15.116:0/3500994841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c68c077e0 0x556c68c07c20 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40cc050 tx=0x7ff6d4014d60 comp rx=0 tx=0).stop 2024-09-12T03:15:06.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.193+0000 7ff6ed892740 1 -- 172.21.15.116:0/3500994841 shutdown_connections 2024-09-12T03:15:06.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.193+0000 7ff6ed892740 1 --2- 172.21.15.116:0/3500994841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c68c12580 0x556c68c14970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.193+0000 7ff6ed892740 1 --2- 172.21.15.116:0/3500994841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c68c077e0 0x556c68c07c20 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.193+0000 7ff6ed892740 1 --2- 172.21.15.116:0/3500994841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c68c08d50 0x556c68c072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.193+0000 7ff6ed892740 1 -- 172.21.15.116:0/3500994841 >> 172.21.15.116:0/3500994841 conn(0x556c68bce310 msgr2=0x556c68c03e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:06.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.193+0000 7ff6ed892740 1 -- 172.21.15.116:0/3500994841 shutdown_connections 2024-09-12T03:15:06.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.193+0000 7ff6ed892740 1 -- 172.21.15.116:0/3500994841 wait complete. 2024-09-12T03:15:06.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.193+0000 7ff6ed892740 1 -- start start 2024-09-12T03:15:06.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.193+0000 7ff6ed892740 1 Processor -- start 2024-09-12T03:15:06.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.194+0000 7ff6ed892740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c68c077e0 0x556c68c11f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:06.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.194+0000 7ff6ed892740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c68c08d50 0x556c68c0f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:06.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.194+0000 7ff6ed892740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c68c12580 0x556c68c0f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:06.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.194+0000 7ff6ed892740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556c68c15280 con 0x556c68c08d50 2024-09-12T03:15:06.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.194+0000 7ff6ed892740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556c68b26060 con 0x556c68c12580 2024-09-12T03:15:06.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.194+0000 7ff6ed892740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556c68c15400 con 0x556c68c077e0 2024-09-12T03:15:06.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.194+0000 7ff6ed892740 10 client.0 _ll_register_callbacks cb 0x556c688798c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:06.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.194+0000 7ff6e8a54640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c68c08d50 0x556c68c0f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:06.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.194+0000 7ff6e8a54640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c68c08d50 0x556c68c0f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47588/0 (socket says 172.21.15.116:47588) 2024-09-12T03:15:06.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.194+0000 7ff6e8a54640 1 -- 172.21.15.116:0/1660835993 learned_addr learned my addr 172.21.15.116:0/1660835993 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:06.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.194+0000 7ff6e9255640 1 --2- 172.21.15.116:0/1660835993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c68c077e0 0x556c68c11f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:06.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.194+0000 7ff6e9a56640 1 --2- 172.21.15.116:0/1660835993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c68c12580 0x556c68c0f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:06.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.194+0000 7ff6e8a54640 1 -- 172.21.15.116:0/1660835993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c68c077e0 msgr2=0x556c68c11f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:06.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.194+0000 7ff6e8a54640 1 --2- 172.21.15.116:0/1660835993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c68c077e0 0x556c68c11f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.194+0000 7ff6e8a54640 1 -- 172.21.15.116:0/1660835993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c68c12580 msgr2=0x556c68c0f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:06.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.194+0000 7ff6e8a54640 1 --2- 172.21.15.116:0/1660835993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c68c12580 0x556c68c0f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.194+0000 7ff6e8a54640 1 -- 172.21.15.116:0/1660835993 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c688dc3c0 con 0x556c68c08d50 2024-09-12T03:15:06.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.194+0000 7ff6e9255640 1 --2- 172.21.15.116:0/1660835993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c68c077e0 0x556c68c11f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:15:06.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.194+0000 7ff6e9a56640 1 --2- 172.21.15.116:0/1660835993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c68c12580 0x556c68c0f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:15:06.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.194+0000 7ff6e8a54640 1 --2- 172.21.15.116:0/1660835993 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c68c08d50 0x556c68c0f240 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7ff6d800bad0 tx=0x7ff6d800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809da5a5d9575e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:06.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.194+0000 7ff6c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:15:06.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.195+0000 7ff6c8ff9640 1 -- 172.21.15.116:0/1660835993 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8003e20 con 0x556c68c08d50 2024-09-12T03:15:06.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.195+0000 7ff6c8ff9640 1 -- 172.21.15.116:0/1660835993 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff6d8011440 con 0x556c68c08d50 2024-09-12T03:15:06.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.195+0000 7ff6ed892740 1 -- 172.21.15.116:0/1660835993 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6d409f160 con 0x556c68c08d50 2024-09-12T03:15:06.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.195+0000 7ff6c8ff9640 1 -- 172.21.15.116:0/1660835993 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff6d8005020 con 0x556c68c08d50 2024-09-12T03:15:06.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.195+0000 7ff6ed892740 10 client.18702 fetch_fsmap learned FSMap version 25 2024-09-12T03:15:06.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.195+0000 7ff6ed892740 1 -- 172.21.15.116:0/1660835993 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556c68c20db0 con 0x556c68c08d50 2024-09-12T03:15:06.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.195+0000 7ff6c8ff9640 1 -- 172.21.15.116:0/1660835993 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6d409f160 con 0x556c68c08d50 2024-09-12T03:15:06.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.195+0000 7ff6c8ff9640 1 -- 172.21.15.116:0/1660835993 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff6d8011640 con 0x556c68c08d50 2024-09-12T03:15:06.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.195+0000 7ff6ed892740 10 client.18702 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:15:06.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.195+0000 7ff6ed892740 10 client.18702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:15:06.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.195+0000 7ff6ed892740 20 client.18702 populate_metadata read hostname 'smithi116' 2024-09-12T03:15:06.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.195+0000 7ff6ed892740 1 --2- 172.21.15.116:0/1660835993 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556c68c22310 0x556c68c24750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:06.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.195+0000 7ff6ed892740 4 client.18702 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:15:06.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.195+0000 7ff6ed892740 1 -- 172.21.15.116:0/1660835993 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c68b78860 con 0x556c68c22310 2024-09-12T03:15:06.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.196+0000 7ff6e9255640 1 --2- 172.21.15.116:0/1660835993 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556c68c22310 0x556c68c24750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:06.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.196+0000 7ff6e9255640 1 --2- 172.21.15.116:0/1660835993 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556c68c22310 0x556c68c24750 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fd925a47309ca06 server_cookie=a04f520d6ded1062 in_seq=0 out_seq=0 2024-09-12T03:15:06.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.196+0000 7ff6c8ff9640 10 client.18702 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:06.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.197+0000 7ff6c8ff9640 1 -- 172.21.15.116:0/1660835993 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556c68b78860 con 0x556c68c22310 2024-09-12T03:15:06.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.197+0000 7ff6c8ff9640 10 client.18702 handle_command_reply: tid=0 2024-09-12T03:15:06.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.197+0000 7ff6ed892740 1 client.18702 shutdown 2024-09-12T03:15:06.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.197+0000 7ff6ed892740 1 -- 172.21.15.116:0/1660835993 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c68c08d50 msgr2=0x556c68c0f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:06.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.197+0000 7ff6ed892740 1 --2- 172.21.15.116:0/1660835993 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c68c08d50 0x556c68c0f240 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7ff6d800bad0 tx=0x7ff6d800bf90 comp rx=0 tx=0).stop 2024-09-12T03:15:06.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.197+0000 7ff6ed892740 1 -- 172.21.15.116:0/1660835993 shutdown_connections 2024-09-12T03:15:06.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.197+0000 7ff6ed892740 1 --2- 172.21.15.116:0/1660835993 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556c68c22310 0x556c68c24750 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.197+0000 7ff6ed892740 1 --2- 172.21.15.116:0/1660835993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c68c12580 0x556c68c0f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.197+0000 7ff6ed892740 1 --2- 172.21.15.116:0/1660835993 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c68c08d50 0x556c68c0f240 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.197+0000 7ff6c8ff9640 0 client.18702 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:06.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.197+0000 7ff6ed892740 1 --2- 172.21.15.116:0/1660835993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c68c077e0 0x556c68c11f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.197+0000 7ff6ed892740 1 -- 172.21.15.116:0/1660835993 >> 172.21.15.116:0/1660835993 conn(0x556c68bce310 msgr2=0x556c68c09ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:06.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.197+0000 7ff6ed892740 1 -- 172.21.15.116:0/1660835993 shutdown_connections 2024-09-12T03:15:06.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.197+0000 7ff6ed892740 1 -- 172.21.15.116:0/1660835993 wait complete. 2024-09-12T03:15:06.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.197+0000 7ff6ed892740 20 client.18702 trim_cache size 0 max 16384 2024-09-12T03:15:06.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.210+0000 7ff6ed892740 1 Processor -- start 2024-09-12T03:15:06.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.210+0000 7ff6ed892740 1 -- start start 2024-09-12T03:15:06.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.210+0000 7ff6ed892740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c68c077e0 0x556c68c1e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:06.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.210+0000 7ff6ed892740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c68c08d50 0x556c68c2a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:06.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.210+0000 7ff6ed892740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c68c12580 0x556c68c2a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:06.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.210+0000 7ff6ed892740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e41adc20 con 0x556c68c08d50 2024-09-12T03:15:06.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.210+0000 7ff6ed892740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e41ab4f0 con 0x556c68c077e0 2024-09-12T03:15:06.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.210+0000 7ff6ed892740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e41a9340 con 0x556c68c12580 2024-09-12T03:15:06.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.210+0000 7ff6e8a54640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c68c08d50 0x556c68c2a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:06.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.210+0000 7ff6e8a54640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c68c08d50 0x556c68c2a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47604/0 (socket says 172.21.15.116:47604) 2024-09-12T03:15:06.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.210+0000 7ff6e8a54640 1 -- 172.21.15.116:0/54542741 learned_addr learned my addr 172.21.15.116:0/54542741 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:06.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.210+0000 7ff6e9255640 1 --2- 172.21.15.116:0/54542741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c68c077e0 0x556c68c1e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:06.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.211+0000 7ff6e9a56640 1 --2- 172.21.15.116:0/54542741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c68c12580 0x556c68c2a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:06.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.211+0000 7ff6e8a54640 1 -- 172.21.15.116:0/54542741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c68c12580 msgr2=0x556c68c2a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:06.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.211+0000 7ff6e8a54640 1 --2- 172.21.15.116:0/54542741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c68c12580 0x556c68c2a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.211+0000 7ff6e8a54640 1 -- 172.21.15.116:0/54542741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c68c077e0 msgr2=0x556c68c1e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:06.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.211+0000 7ff6e8a54640 1 --2- 172.21.15.116:0/54542741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c68c077e0 0x556c68c1e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.211+0000 7ff6e8a54640 1 -- 172.21.15.116:0/54542741 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c68c24dd0 con 0x556c68c08d50 2024-09-12T03:15:06.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.211+0000 7ff6e9a56640 1 --2- 172.21.15.116:0/54542741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c68c12580 0x556c68c2a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:15:06.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.211+0000 7ff6e8a54640 1 --2- 172.21.15.116:0/54542741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c68c08d50 0x556c68c2a020 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8011ab0 tx=0x7ff6d8005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91dd553c6c9afa27 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:06.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.211+0000 7ff6caffd640 1 -- 172.21.15.116:0/54542741 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d800b400 con 0x556c68c08d50 2024-09-12T03:15:06.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.211+0000 7ff6caffd640 1 -- 172.21.15.116:0/54542741 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6d8011d20 con 0x556c68c08d50 2024-09-12T03:15:06.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.211+0000 7ff6caffd640 1 -- 172.21.15.116:0/54542741 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8004820 con 0x556c68c08d50 2024-09-12T03:15:06.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.212+0000 7ff6ed892740 1 -- 172.21.15.116:0/54542741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c68c08d50 msgr2=0x556c68c2a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:06.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.212+0000 7ff6ed892740 1 --2- 172.21.15.116:0/54542741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c68c08d50 0x556c68c2a020 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8011ab0 tx=0x7ff6d8005cd0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.212+0000 7ff6ed892740 1 -- 172.21.15.116:0/54542741 shutdown_connections 2024-09-12T03:15:06.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.212+0000 7ff6ed892740 1 --2- 172.21.15.116:0/54542741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c68c12580 0x556c68c2a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.212+0000 7ff6ed892740 1 --2- 172.21.15.116:0/54542741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c68c08d50 0x556c68c2a020 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.212+0000 7ff6ed892740 1 --2- 172.21.15.116:0/54542741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c68c077e0 0x556c68c1e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.212+0000 7ff6ed892740 1 -- 172.21.15.116:0/54542741 >> 172.21.15.116:0/54542741 conn(0x556c68bce310 msgr2=0x556c68c0a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:06.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.212+0000 7ff6ed892740 1 -- 172.21.15.116:0/54542741 shutdown_connections 2024-09-12T03:15:06.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.212+0000 7ff6ed892740 1 -- 172.21.15.116:0/54542741 wait complete. 2024-09-12T03:15:06.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.212+0000 7ff6ed892740 1 -- start start 2024-09-12T03:15:06.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.212+0000 7ff6ed892740 1 Processor -- start 2024-09-12T03:15:06.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.213+0000 7ff6ed892740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c68c077e0 0x556c68c01b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:06.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.213+0000 7ff6ed892740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c68c08d50 0x556c68c020b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:06.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.213+0000 7ff6ed892740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c68c12580 0x556c68c02630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:06.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.213+0000 7ff6ed892740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e41a99b0 con 0x556c68c077e0 2024-09-12T03:15:06.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.213+0000 7ff6ed892740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e41aae20 con 0x556c68c12580 2024-09-12T03:15:06.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.213+0000 7ff6ed892740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e41aa080 con 0x556c68c08d50 2024-09-12T03:15:06.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.213+0000 7ff6ed892740 10 client.0 _ll_register_callbacks cb 0x7ff6d8004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:06.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.213+0000 7ff6e9255640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c68c077e0 0x556c68c01b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:06.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.213+0000 7ff6e9255640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c68c077e0 0x556c68c01b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47612/0 (socket says 172.21.15.116:47612) 2024-09-12T03:15:06.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.213+0000 7ff6e9255640 1 -- 172.21.15.116:0/1258978072 learned_addr learned my addr 172.21.15.116:0/1258978072 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:06.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.213+0000 7ff6e9a56640 1 --2- 172.21.15.116:0/1258978072 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c68c12580 0x556c68c02630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:06.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.213+0000 7ff6e8a54640 1 --2- 172.21.15.116:0/1258978072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c68c08d50 0x556c68c020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:06.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.213+0000 7ff6e9255640 1 -- 172.21.15.116:0/1258978072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c68c08d50 msgr2=0x556c68c020b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:06.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.213+0000 7ff6e9255640 1 --2- 172.21.15.116:0/1258978072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c68c08d50 0x556c68c020b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.213+0000 7ff6e9255640 1 -- 172.21.15.116:0/1258978072 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c68c12580 msgr2=0x556c68c02630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:06.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.213+0000 7ff6e9255640 1 --2- 172.21.15.116:0/1258978072 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c68c12580 0x556c68c02630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.213+0000 7ff6e9255640 1 -- 172.21.15.116:0/1258978072 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c68c19980 con 0x556c68c077e0 2024-09-12T03:15:06.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.213+0000 7ff6e9255640 1 --2- 172.21.15.116:0/1258978072 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c68c077e0 0x556c68c01b70 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4097140 tx=0x7ff6d40ce900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316e223625583b9c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:06.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.213+0000 7ff6c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:15:06.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.213+0000 7ff6c8ff9640 1 -- 172.21.15.116:0/1258978072 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4098f80 con 0x556c68c077e0 2024-09-12T03:15:06.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.213+0000 7ff6c8ff9640 1 -- 172.21.15.116:0/1258978072 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff6d4099160 con 0x556c68c077e0 2024-09-12T03:15:06.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.213+0000 7ff6ed892740 1 -- 172.21.15.116:0/1258978072 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6d8011d20 con 0x556c68c077e0 2024-09-12T03:15:06.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.214+0000 7ff6c8ff9640 1 -- 172.21.15.116:0/1258978072 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff6d409f160 con 0x556c68c077e0 2024-09-12T03:15:06.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.214+0000 7ff6ed892740 10 client.18708 fetch_fsmap learned FSMap version 25 2024-09-12T03:15:06.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.214+0000 7ff6ed892740 1 -- 172.21.15.116:0/1258978072 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556c68c1ac50 con 0x556c68c077e0 2024-09-12T03:15:06.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.214+0000 7ff6c8ff9640 1 -- 172.21.15.116:0/1258978072 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6d8011d20 con 0x556c68c077e0 2024-09-12T03:15:06.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.214+0000 7ff6c8ff9640 1 -- 172.21.15.116:0/1258978072 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff6d409a2b0 con 0x556c68c077e0 2024-09-12T03:15:06.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.214+0000 7ff6ed892740 10 client.18708 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:15:06.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.214+0000 7ff6ed892740 10 client.18708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:15:06.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.214+0000 7ff6ed892740 20 client.18708 populate_metadata read hostname 'smithi116' 2024-09-12T03:15:06.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.214+0000 7ff6ed892740 1 --2- 172.21.15.116:0/1258978072 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556c68c1b000 0x556c68c95800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:06.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.214+0000 7ff6ed892740 4 client.18708 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:15:06.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.214+0000 7ff6ed892740 1 -- 172.21.15.116:0/1258978072 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556c68b72d70 con 0x556c68c1b000 2024-09-12T03:15:06.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.214+0000 7ff6e8a54640 1 --2- 172.21.15.116:0/1258978072 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556c68c1b000 0x556c68c95800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:06.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.214+0000 7ff6c8ff9640 10 client.18708 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:06.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.214+0000 7ff6e8a54640 1 --2- 172.21.15.116:0/1258978072 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556c68c1b000 0x556c68c95800 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cef35450567ef58a server_cookie=b1a883d3162dd76a in_seq=0 out_seq=0 2024-09-12T03:15:06.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.215+0000 7ff6c8ff9640 1 -- 172.21.15.116:0/1258978072 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556c68b72d70 con 0x556c68c1b000 2024-09-12T03:15:06.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.215+0000 7ff6c8ff9640 10 client.18708 handle_command_reply: tid=0 2024-09-12T03:15:06.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.215+0000 7ff6ed892740 1 client.18708 shutdown 2024-09-12T03:15:06.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.215+0000 7ff6ed892740 1 -- 172.21.15.116:0/1258978072 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c68c077e0 msgr2=0x556c68c01b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:06.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.215+0000 7ff6ed892740 1 --2- 172.21.15.116:0/1258978072 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c68c077e0 0x556c68c01b70 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4097140 tx=0x7ff6d40ce900 comp rx=0 tx=0).stop 2024-09-12T03:15:06.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.215+0000 7ff6ed892740 1 -- 172.21.15.116:0/1258978072 shutdown_connections 2024-09-12T03:15:06.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.215+0000 7ff6ed892740 1 --2- 172.21.15.116:0/1258978072 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556c68c1b000 0x556c68c95800 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.215+0000 7ff6ed892740 1 --2- 172.21.15.116:0/1258978072 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556c68c12580 0x556c68c02630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.215+0000 7ff6ed892740 1 --2- 172.21.15.116:0/1258978072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c68c08d50 0x556c68c020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.215+0000 7ff6c8ff9640 0 client.18708 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:06.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.215+0000 7ff6ed892740 1 --2- 172.21.15.116:0/1258978072 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c68c077e0 0x556c68c01b70 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.215+0000 7ff6ed892740 1 -- 172.21.15.116:0/1258978072 >> 172.21.15.116:0/1258978072 conn(0x556c68bce310 msgr2=0x556c68c0db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:06.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.215+0000 7ff6ed892740 1 -- 172.21.15.116:0/1258978072 shutdown_connections 2024-09-12T03:15:06.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.215+0000 7ff6ed892740 1 -- 172.21.15.116:0/1258978072 wait complete. 2024-09-12T03:15:06.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.215+0000 7ff6ed892740 20 client.18708 trim_cache size 0 max 16384 2024-09-12T03:15:06.226 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:15:06.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.216+0000 7ff6eb4b9640 1 -- 172.21.15.116:0/3745234293 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff6cc0a2c00 msgr2=0x7ff6cc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:06.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.216+0000 7ff6eb4b9640 1 --2- 172.21.15.116:0/3745234293 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff6cc0a2c00 0x7ff6cc0a50c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc031550 tx=0x7ff6dc0058d0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.216+0000 7ff6eb4b9640 1 -- 172.21.15.116:0/3745234293 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6e4100a70 msgr2=0x7ff6e4197370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:06.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.216+0000 7ff6eb4b9640 1 --2- 172.21.15.116:0/3745234293 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6e4100a70 0x7ff6e4197370 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7ff6d400ba50 tx=0x7ff6d400bf10 comp rx=0 tx=0).stop 2024-09-12T03:15:06.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.216+0000 7ff6eb4b9640 1 -- 172.21.15.116:0/3745234293 shutdown_connections 2024-09-12T03:15:06.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.216+0000 7ff6eb4b9640 1 --2- 172.21.15.116:0/3745234293 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff6cc0a2c00 0x7ff6cc0a50c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.216+0000 7ff6eb4b9640 1 --2- 172.21.15.116:0/3745234293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6e4106a70 0x7ff6e419bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.216+0000 7ff6eb4b9640 1 --2- 172.21.15.116:0/3745234293 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6e4101410 0x7ff6e41978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.216+0000 7ff6eb4b9640 1 --2- 172.21.15.116:0/3745234293 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6e4100a70 0x7ff6e4197370 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:06.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.216+0000 7ff6eb4b9640 1 -- 172.21.15.116:0/3745234293 >> 172.21.15.116:0/3745234293 conn(0x7ff6e40fc7d0 msgr2=0x7ff6e40fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:06.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.216+0000 7ff6eb4b9640 1 -- 172.21.15.116:0/3745234293 shutdown_connections 2024-09-12T03:15:06.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:06.216+0000 7ff6eb4b9640 1 -- 172.21.15.116:0/3745234293 wait complete. 2024-09-12T03:15:07.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:07 smithi183 ceph-mon[35535]: pgmap v702: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:15:07.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:07 smithi177 ceph-mon[35305]: pgmap v702: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:15:07.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:07 smithi116 ceph-mon[31410]: pgmap v702: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:15:08.019 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:15:08.020 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:15:08.279 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:15:08.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.747+0000 7fc2c2391640 1 -- 172.21.15.116:0/1813293930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2bc101de0 msgr2=0x7fc2bc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:08.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.747+0000 7fc2c2391640 1 --2- 172.21.15.116:0/1813293930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2bc101de0 0x7fc2bc1021c0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac009900 tx=0x7fc2ac02a410 comp rx=0 tx=0).stop 2024-09-12T03:15:08.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.748+0000 7fc2c2391640 1 -- 172.21.15.116:0/1813293930 shutdown_connections 2024-09-12T03:15:08.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.748+0000 7fc2c2391640 1 --2- 172.21.15.116:0/1813293930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2bc106f10 0x7fc2bc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.748+0000 7fc2c2391640 1 --2- 172.21.15.116:0/1813293930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2bc102790 0x7fc2bc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.748+0000 7fc2c2391640 1 --2- 172.21.15.116:0/1813293930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2bc101de0 0x7fc2bc1021c0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.748+0000 7fc2c2391640 1 -- 172.21.15.116:0/1813293930 >> 172.21.15.116:0/1813293930 conn(0x7fc2bc0fd8c0 msgr2=0x7fc2bc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:08.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.748+0000 7fc2c2391640 1 -- 172.21.15.116:0/1813293930 shutdown_connections 2024-09-12T03:15:08.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.748+0000 7fc2c2391640 1 -- 172.21.15.116:0/1813293930 wait complete. 2024-09-12T03:15:08.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.748+0000 7fc2c2391640 1 Processor -- start 2024-09-12T03:15:08.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.748+0000 7fc2c2391640 1 -- start start 2024-09-12T03:15:08.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.748+0000 7fc2c2391640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2bc101de0 0x7fc2bc198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:08.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.748+0000 7fc2c2391640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2bc102790 0x7fc2bc1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:08.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.748+0000 7fc2c2391640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2bc106f10 0x7fc2bc19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:08.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.748+0000 7fc2c2391640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc111f90 con 0x7fc2bc101de0 2024-09-12T03:15:08.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.748+0000 7fc2c2391640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc111e10 con 0x7fc2bc106f10 2024-09-12T03:15:08.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.748+0000 7fc2c2391640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc112110 con 0x7fc2bc102790 2024-09-12T03:15:08.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.748+0000 7fc2bbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2bc101de0 0x7fc2bc198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:08.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.749+0000 7fc2bbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2bc101de0 0x7fc2bc198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47636/0 (socket says 172.21.15.116:47636) 2024-09-12T03:15:08.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.749+0000 7fc2bbfff640 1 -- 172.21.15.116:0/1360374031 learned_addr learned my addr 172.21.15.116:0/1360374031 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:08.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.749+0000 7fc2b3fff640 1 --2- 172.21.15.116:0/1360374031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2bc102790 0x7fc2bc1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:08.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.749+0000 7fc2c092e640 1 --2- 172.21.15.116:0/1360374031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2bc106f10 0x7fc2bc19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:08.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.749+0000 7fc2bbfff640 1 -- 172.21.15.116:0/1360374031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2bc102790 msgr2=0x7fc2bc1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:08.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.749+0000 7fc2bbfff640 1 --2- 172.21.15.116:0/1360374031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2bc102790 0x7fc2bc1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.749+0000 7fc2bbfff640 1 -- 172.21.15.116:0/1360374031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2bc106f10 msgr2=0x7fc2bc19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:08.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.749+0000 7fc2bbfff640 1 --2- 172.21.15.116:0/1360374031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2bc106f10 0x7fc2bc19cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.749+0000 7fc2bbfff640 1 -- 172.21.15.116:0/1360374031 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2bc19d430 con 0x7fc2bc101de0 2024-09-12T03:15:08.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.749+0000 7fc2bbfff640 1 --2- 172.21.15.116:0/1360374031 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2bc101de0 0x7fc2bc198480 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac005ab0 tx=0x7fc2ac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:08.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.749+0000 7fc2b9ffb640 1 -- 172.21.15.116:0/1360374031 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac03f030 con 0x7fc2bc101de0 2024-09-12T03:15:08.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.749+0000 7fc2c2391640 1 -- 172.21.15.116:0/1360374031 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2bc19d6c0 con 0x7fc2bc101de0 2024-09-12T03:15:08.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.749+0000 7fc2b9ffb640 1 -- 172.21.15.116:0/1360374031 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2ac003ce0 con 0x7fc2bc101de0 2024-09-12T03:15:08.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.749+0000 7fc2b9ffb640 1 -- 172.21.15.116:0/1360374031 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac035070 con 0x7fc2bc101de0 2024-09-12T03:15:08.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.749+0000 7fc2c2391640 1 -- 172.21.15.116:0/1360374031 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2bc1a5000 con 0x7fc2bc101de0 2024-09-12T03:15:08.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.751+0000 7fc2b9ffb640 1 -- 172.21.15.116:0/1360374031 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fc2ac042030 con 0x7fc2bc101de0 2024-09-12T03:15:08.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.752+0000 7fc2c476a740 1 Processor -- start 2024-09-12T03:15:08.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.752+0000 7fc2c476a740 1 -- start start 2024-09-12T03:15:08.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.752+0000 7fc2c476a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fdf2c3a60 0x556fdf2c3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:08.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.752+0000 7fc2c476a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fdf2c4380 0x556fdf2d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:08.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.752+0000 7fc2c476a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fdf2d3150 0x556fdf2d5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:08.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.752+0000 7fc2c476a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556fdf229280 con 0x556fdf2c4380 2024-09-12T03:15:08.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.752+0000 7fc2c476a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556fdf1d0b10 con 0x556fdf2c3a60 2024-09-12T03:15:08.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.752+0000 7fc2c476a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556fdf225240 con 0x556fdf2d3150 2024-09-12T03:15:08.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.752+0000 7fc2b9ffb640 1 --2- 172.21.15.116:0/1360374031 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc2a80a2b80 0x7fc2a80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:08.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.752+0000 7fc2bbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fdf2c4380 0x556fdf2d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:08.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.752+0000 7fc2bbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fdf2c4380 0x556fdf2d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47640/0 (socket says 172.21.15.116:47640) 2024-09-12T03:15:08.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.752+0000 7fc2bbfff640 1 -- 172.21.15.116:0/3880383286 learned_addr learned my addr 172.21.15.116:0/3880383286 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:08.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.752+0000 7fc2c092e640 1 --2- 172.21.15.116:0/1360374031 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc2a80a2b80 0x7fc2a80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:08.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.752+0000 7fc2c092e640 1 --2- 172.21.15.116:0/3880383286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fdf2c3a60 0x556fdf2c3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:08.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.752+0000 7fc2b3fff640 1 --2- 172.21.15.116:0/3880383286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fdf2d3150 0x556fdf2d5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:08.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.752+0000 7fc2b9ffb640 1 -- 172.21.15.116:0/1360374031 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fc2ac0ef850 con 0x7fc2bc101de0 2024-09-12T03:15:08.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.753+0000 7fc2bbfff640 1 -- 172.21.15.116:0/3880383286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fdf2d3150 msgr2=0x556fdf2d5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:08.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.753+0000 7fc2bbfff640 1 --2- 172.21.15.116:0/3880383286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fdf2d3150 0x556fdf2d5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.753+0000 7fc2bbfff640 1 -- 172.21.15.116:0/3880383286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fdf2c3a60 msgr2=0x556fdf2c3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:08.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.753+0000 7fc2bbfff640 1 --2- 172.21.15.116:0/3880383286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fdf2c3a60 0x556fdf2c3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.753+0000 7fc2bbfff640 1 -- 172.21.15.116:0/3880383286 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fdf1dd1f0 con 0x556fdf2c4380 2024-09-12T03:15:08.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.753+0000 7fc2c092e640 1 --2- 172.21.15.116:0/1360374031 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc2a80a2b80 0x7fc2a80a5040 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4009fd0 tx=0x7fc2b4009ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:08.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.753+0000 7fc2bbfff640 1 --2- 172.21.15.116:0/3880383286 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fdf2c4380 0x556fdf2d2c10 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac0ed380 tx=0x7fc2ac0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d22b0cd44805daff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:08.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.753+0000 7fc2b27fc640 1 -- 172.21.15.116:0/3880383286 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac049020 con 0x556fdf2c4380 2024-09-12T03:15:08.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.753+0000 7fc2b27fc640 1 -- 172.21.15.116:0/3880383286 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2ac0c0490 con 0x556fdf2c4380 2024-09-12T03:15:08.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.753+0000 7fc2b27fc640 1 -- 172.21.15.116:0/3880383286 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac03bd70 con 0x556fdf2c4380 2024-09-12T03:15:08.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.754+0000 7fc2c476a740 1 -- 172.21.15.116:0/3880383286 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fdf2c4380 msgr2=0x556fdf2d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:08.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.754+0000 7fc2c476a740 1 --2- 172.21.15.116:0/3880383286 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fdf2c4380 0x556fdf2d2c10 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac0ed380 tx=0x7fc2ac0045e0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.754+0000 7fc2c476a740 1 -- 172.21.15.116:0/3880383286 shutdown_connections 2024-09-12T03:15:08.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.754+0000 7fc2c476a740 1 --2- 172.21.15.116:0/3880383286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fdf2d3150 0x556fdf2d5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.754+0000 7fc2c476a740 1 --2- 172.21.15.116:0/3880383286 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fdf2c4380 0x556fdf2d2c10 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.754+0000 7fc2c476a740 1 --2- 172.21.15.116:0/3880383286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fdf2c3a60 0x556fdf2c3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.754+0000 7fc2c476a740 1 -- 172.21.15.116:0/3880383286 >> 172.21.15.116:0/3880383286 conn(0x556fdf2c00f0 msgr2=0x556fdf2c04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:08.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.754+0000 7fc2c476a740 1 -- 172.21.15.116:0/3880383286 shutdown_connections 2024-09-12T03:15:08.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.754+0000 7fc2c476a740 1 -- 172.21.15.116:0/3880383286 wait complete. 2024-09-12T03:15:08.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.754+0000 7fc2c476a740 1 -- start start 2024-09-12T03:15:08.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.754+0000 7fc2c476a740 1 Processor -- start 2024-09-12T03:15:08.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.755+0000 7fc2c476a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fdf2c3a60 0x556fdf2d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:08.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.755+0000 7fc2c476a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fdf2c4380 0x556fdf2d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:08.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.755+0000 7fc2c476a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fdf2d3150 0x556fdf2d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:08.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.755+0000 7fc2c476a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556fdf2d5b40 con 0x556fdf2c4380 2024-09-12T03:15:08.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.755+0000 7fc2c476a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556fdf1e3f10 con 0x556fdf2c3a60 2024-09-12T03:15:08.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.755+0000 7fc2c476a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556fdf2d5cc0 con 0x556fdf2d3150 2024-09-12T03:15:08.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.755+0000 7fc2c476a740 10 client.0 _ll_register_callbacks cb 0x556fdf1aca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:08.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.755+0000 7fc2b3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fdf2c4380 0x556fdf2d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:08.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.755+0000 7fc2b3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fdf2c4380 0x556fdf2d0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47642/0 (socket says 172.21.15.116:47642) 2024-09-12T03:15:08.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.755+0000 7fc2b3fff640 1 -- 172.21.15.116:0/2224429015 learned_addr learned my addr 172.21.15.116:0/2224429015 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:08.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.755+0000 7fc2bbfff640 1 --2- 172.21.15.116:0/2224429015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fdf2c3a60 0x556fdf2d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:08.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.755+0000 7fc2c092e640 1 --2- 172.21.15.116:0/2224429015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fdf2d3150 0x556fdf2d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:08.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.755+0000 7fc2b3fff640 1 -- 172.21.15.116:0/2224429015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fdf2d3150 msgr2=0x556fdf2d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:08.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.755+0000 7fc2b3fff640 1 --2- 172.21.15.116:0/2224429015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fdf2d3150 0x556fdf2d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.755+0000 7fc2b3fff640 1 -- 172.21.15.116:0/2224429015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fdf2c3a60 msgr2=0x556fdf2d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:08.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.755+0000 7fc2b3fff640 1 --2- 172.21.15.116:0/2224429015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fdf2c3a60 0x556fdf2d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.755+0000 7fc2b3fff640 1 -- 172.21.15.116:0/2224429015 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fdef9a6a0 con 0x556fdf2c4380 2024-09-12T03:15:08.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.755+0000 7fc2c092e640 1 --2- 172.21.15.116:0/2224429015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fdf2d3150 0x556fdf2d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:15:08.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.755+0000 7fc2b3fff640 1 --2- 172.21.15.116:0/2224429015 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fdf2c4380 0x556fdf2d0600 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fc2a40107a0 tx=0x7fc2a4010c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72fad7a99914220 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:08.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.755+0000 7fc29b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:15:08.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.755+0000 7fc29b7fe640 1 -- 172.21.15.116:0/2224429015 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a400fb90 con 0x556fdf2c4380 2024-09-12T03:15:08.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.755+0000 7fc29b7fe640 1 -- 172.21.15.116:0/2224429015 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fc2a400fd30 con 0x556fdf2c4380 2024-09-12T03:15:08.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.755+0000 7fc2c476a740 1 -- 172.21.15.116:0/2224429015 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2ac0c0490 con 0x556fdf2c4380 2024-09-12T03:15:08.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.755+0000 7fc29b7fe640 1 -- 172.21.15.116:0/2224429015 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc2a4004020 con 0x556fdf2c4380 2024-09-12T03:15:08.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.755+0000 7fc2c476a740 10 client.18720 fetch_fsmap learned FSMap version 25 2024-09-12T03:15:08.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.755+0000 7fc2c476a740 1 -- 172.21.15.116:0/2224429015 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556fdf2e1760 con 0x556fdf2c4380 2024-09-12T03:15:08.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.756+0000 7fc29b7fe640 1 -- 172.21.15.116:0/2224429015 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2ac0c0490 con 0x556fdf2c4380 2024-09-12T03:15:08.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.756+0000 7fc29b7fe640 1 -- 172.21.15.116:0/2224429015 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc2a4025480 con 0x556fdf2c4380 2024-09-12T03:15:08.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.756+0000 7fc2c476a740 10 client.18720 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:15:08.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.756+0000 7fc2c476a740 10 client.18720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:15:08.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.756+0000 7fc2c476a740 20 client.18720 populate_metadata read hostname 'smithi116' 2024-09-12T03:15:08.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.756+0000 7fc2c476a740 1 --2- 172.21.15.116:0/2224429015 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556fdf2e2c30 0x556fdf2e5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:08.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.756+0000 7fc2c476a740 4 client.18720 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:15:08.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.756+0000 7fc2c476a740 1 -- 172.21.15.116:0/2224429015 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fdf2367f0 con 0x556fdf2e2c30 2024-09-12T03:15:08.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.756+0000 7fc2bbfff640 1 --2- 172.21.15.116:0/2224429015 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556fdf2e2c30 0x556fdf2e5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:08.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.756+0000 7fc29b7fe640 10 client.18720 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:08.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.756+0000 7fc2bbfff640 1 --2- 172.21.15.116:0/2224429015 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556fdf2e2c30 0x556fdf2e5090 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfb0ecf332688ed4 server_cookie=f55a86edbf3a989b in_seq=0 out_seq=0 2024-09-12T03:15:08.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.757+0000 7fc29b7fe640 1 -- 172.21.15.116:0/2224429015 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556fdf2367f0 con 0x556fdf2e2c30 2024-09-12T03:15:08.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.757+0000 7fc29b7fe640 10 client.18720 handle_command_reply: tid=0 2024-09-12T03:15:08.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.757+0000 7fc2c476a740 1 client.18720 shutdown 2024-09-12T03:15:08.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.757+0000 7fc2c476a740 1 -- 172.21.15.116:0/2224429015 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fdf2c4380 msgr2=0x556fdf2d0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:08.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.757+0000 7fc2c476a740 1 --2- 172.21.15.116:0/2224429015 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fdf2c4380 0x556fdf2d0600 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fc2a40107a0 tx=0x7fc2a4010c60 comp rx=0 tx=0).stop 2024-09-12T03:15:08.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.757+0000 7fc2c476a740 1 -- 172.21.15.116:0/2224429015 shutdown_connections 2024-09-12T03:15:08.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.757+0000 7fc2c476a740 1 --2- 172.21.15.116:0/2224429015 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556fdf2e2c30 0x556fdf2e5090 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.757+0000 7fc2c476a740 1 --2- 172.21.15.116:0/2224429015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fdf2d3150 0x556fdf2d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.757+0000 7fc2c476a740 1 --2- 172.21.15.116:0/2224429015 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fdf2c4380 0x556fdf2d0600 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.757+0000 7fc2c476a740 1 --2- 172.21.15.116:0/2224429015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fdf2c3a60 0x556fdf2d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.757+0000 7fc29b7fe640 0 client.18720 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:08.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.757+0000 7fc2c476a740 1 -- 172.21.15.116:0/2224429015 >> 172.21.15.116:0/2224429015 conn(0x556fdf2c00f0 msgr2=0x556fdf2c5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:08.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.757+0000 7fc2c476a740 1 -- 172.21.15.116:0/2224429015 shutdown_connections 2024-09-12T03:15:08.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.757+0000 7fc2c476a740 1 -- 172.21.15.116:0/2224429015 wait complete. 2024-09-12T03:15:08.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.758+0000 7fc2c476a740 20 client.18720 trim_cache size 0 max 16384 2024-09-12T03:15:08.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.775+0000 7fc2c476a740 1 Processor -- start 2024-09-12T03:15:08.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.775+0000 7fc2c476a740 1 -- start start 2024-09-12T03:15:08.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.775+0000 7fc2c476a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fdf2c3a60 0x556fdf2cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:08.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.775+0000 7fc2c476a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fdf2c4380 0x556fdf2ea960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:08.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.775+0000 7fc2c476a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fdf2d3150 0x556fdf2eaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:08.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.775+0000 7fc2c476a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc1adc00 con 0x556fdf2c3a60 2024-09-12T03:15:08.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.775+0000 7fc2c476a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc1ab4d0 con 0x556fdf2d3150 2024-09-12T03:15:08.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.775+0000 7fc2c476a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc1a9320 con 0x556fdf2c4380 2024-09-12T03:15:08.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.775+0000 7fc2bbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fdf2c3a60 0x556fdf2cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:08.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.775+0000 7fc2bbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fdf2c3a60 0x556fdf2cef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47656/0 (socket says 172.21.15.116:47656) 2024-09-12T03:15:08.777 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.775+0000 7fc2bbfff640 1 -- 172.21.15.116:0/145695503 learned_addr learned my addr 172.21.15.116:0/145695503 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:08.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.775+0000 7fc2c092e640 1 --2- 172.21.15.116:0/145695503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fdf2d3150 0x556fdf2eaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:08.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.775+0000 7fc2bbfff640 1 -- 172.21.15.116:0/145695503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fdf2c4380 msgr2=0x556fdf2ea960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:08.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.775+0000 7fc2b3fff640 1 --2- 172.21.15.116:0/145695503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fdf2c4380 0x556fdf2ea960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:08.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.775+0000 7fc2bbfff640 1 --2- 172.21.15.116:0/145695503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fdf2c4380 0x556fdf2ea960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.775+0000 7fc2bbfff640 1 -- 172.21.15.116:0/145695503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fdf2d3150 msgr2=0x556fdf2eaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:08.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.775+0000 7fc2bbfff640 1 --2- 172.21.15.116:0/145695503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fdf2d3150 0x556fdf2eaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.775+0000 7fc2bbfff640 1 -- 172.21.15.116:0/145695503 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fdf2e5710 con 0x556fdf2c3a60 2024-09-12T03:15:08.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.775+0000 7fc2c092e640 1 --2- 172.21.15.116:0/145695503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fdf2d3150 0x556fdf2eaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:15:08.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.775+0000 7fc2b3fff640 1 --2- 172.21.15.116:0/145695503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fdf2c4380 0x556fdf2ea960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:15:08.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.776+0000 7fc2bbfff640 1 --2- 172.21.15.116:0/145695503 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fdf2c3a60 0x556fdf2cef70 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac0ed3d0 tx=0x7fc2ac009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d26d97daeade3d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:08.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.776+0000 7fc2b1ffb640 1 -- 172.21.15.116:0/145695503 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac049040 con 0x556fdf2c3a60 2024-09-12T03:15:08.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.776+0000 7fc2b1ffb640 1 -- 172.21.15.116:0/145695503 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2ac0ba750 con 0x556fdf2c3a60 2024-09-12T03:15:08.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.776+0000 7fc2b1ffb640 1 -- 172.21.15.116:0/145695503 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac0c0150 con 0x556fdf2c3a60 2024-09-12T03:15:08.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.776+0000 7fc2c476a740 1 -- 172.21.15.116:0/145695503 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fdf2c3a60 msgr2=0x556fdf2cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:08.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.776+0000 7fc2c476a740 1 --2- 172.21.15.116:0/145695503 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fdf2c3a60 0x556fdf2cef70 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac0ed3d0 tx=0x7fc2ac009680 comp rx=0 tx=0).stop 2024-09-12T03:15:08.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.776+0000 7fc2c476a740 1 -- 172.21.15.116:0/145695503 shutdown_connections 2024-09-12T03:15:08.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.776+0000 7fc2c476a740 1 --2- 172.21.15.116:0/145695503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fdf2d3150 0x556fdf2eaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.776+0000 7fc2c476a740 1 --2- 172.21.15.116:0/145695503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fdf2c4380 0x556fdf2ea960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.776+0000 7fc2c476a740 1 --2- 172.21.15.116:0/145695503 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fdf2c3a60 0x556fdf2cef70 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.776+0000 7fc2c476a740 1 -- 172.21.15.116:0/145695503 >> 172.21.15.116:0/145695503 conn(0x556fdf2c00f0 msgr2=0x556fdf2ce860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:08.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.776+0000 7fc2c476a740 1 -- 172.21.15.116:0/145695503 shutdown_connections 2024-09-12T03:15:08.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.776+0000 7fc2c476a740 1 -- 172.21.15.116:0/145695503 wait complete. 2024-09-12T03:15:08.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.777+0000 7fc2c476a740 1 -- start start 2024-09-12T03:15:08.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.777+0000 7fc2c476a740 1 Processor -- start 2024-09-12T03:15:08.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.777+0000 7fc2c476a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fdf2c3a60 0x556fdf2d6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:08.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.777+0000 7fc2c476a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fdf2c4380 0x556fdf2d7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:08.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.777+0000 7fc2c476a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fdf2d3150 0x556fdf2d7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:08.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.777+0000 7fc2c476a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc1a9990 con 0x556fdf2c4380 2024-09-12T03:15:08.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.777+0000 7fc2c476a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc1aae00 con 0x556fdf2d3150 2024-09-12T03:15:08.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.777+0000 7fc2c476a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc1aa060 con 0x556fdf2c3a60 2024-09-12T03:15:08.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.777+0000 7fc2c476a740 10 client.0 _ll_register_callbacks cb 0x7fc2a400a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:08.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.777+0000 7fc2b3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fdf2c4380 0x556fdf2d7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:08.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.777+0000 7fc2b3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fdf2c4380 0x556fdf2d7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47662/0 (socket says 172.21.15.116:47662) 2024-09-12T03:15:08.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.777+0000 7fc2b3fff640 1 -- 172.21.15.116:0/1569976136 learned_addr learned my addr 172.21.15.116:0/1569976136 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:08.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.778+0000 7fc2c092e640 1 --2- 172.21.15.116:0/1569976136 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fdf2d3150 0x556fdf2d7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:08.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.778+0000 7fc2b3fff640 1 -- 172.21.15.116:0/1569976136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fdf2c3a60 msgr2=0x556fdf2d6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:08.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.778+0000 7fc2b3fff640 1 --2- 172.21.15.116:0/1569976136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fdf2c3a60 0x556fdf2d6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.778+0000 7fc2b3fff640 1 -- 172.21.15.116:0/1569976136 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fdf2d3150 msgr2=0x556fdf2d7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:08.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.778+0000 7fc2b3fff640 1 --2- 172.21.15.116:0/1569976136 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fdf2d3150 0x556fdf2d7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.778+0000 7fc2b3fff640 1 -- 172.21.15.116:0/1569976136 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fdf34caa0 con 0x556fdf2c4380 2024-09-12T03:15:08.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.778+0000 7fc2c092e640 1 --2- 172.21.15.116:0/1569976136 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fdf2d3150 0x556fdf2d7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:15:08.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.778+0000 7fc2b3fff640 1 --2- 172.21.15.116:0/1569976136 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fdf2c4380 0x556fdf2d7240 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4001eb0 tx=0x7fc2a400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5d3cf1bf26f5de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:08.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.778+0000 7fc29b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:15:08.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.778+0000 7fc29b7fe640 1 -- 172.21.15.116:0/1569976136 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a402b070 con 0x556fdf2c4380 2024-09-12T03:15:08.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.778+0000 7fc29b7fe640 1 -- 172.21.15.116:0/1569976136 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fc2a40043d0 con 0x556fdf2c4380 2024-09-12T03:15:08.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.778+0000 7fc2c476a740 1 -- 172.21.15.116:0/1569976136 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2ac0ba750 con 0x556fdf2c4380 2024-09-12T03:15:08.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.778+0000 7fc29b7fe640 1 -- 172.21.15.116:0/1569976136 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc2a4038050 con 0x556fdf2c4380 2024-09-12T03:15:08.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.778+0000 7fc2c476a740 10 client.18726 fetch_fsmap learned FSMap version 25 2024-09-12T03:15:08.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.778+0000 7fc2c476a740 1 -- 172.21.15.116:0/1569976136 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556fdf34dd40 con 0x556fdf2c4380 2024-09-12T03:15:08.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.778+0000 7fc29b7fe640 1 -- 172.21.15.116:0/1569976136 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2ac0ba750 con 0x556fdf2c4380 2024-09-12T03:15:08.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.778+0000 7fc29b7fe640 1 -- 172.21.15.116:0/1569976136 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc2a4003920 con 0x556fdf2c4380 2024-09-12T03:15:08.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.778+0000 7fc2c476a740 10 client.18726 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:15:08.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.778+0000 7fc2c476a740 10 client.18726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:15:08.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.779+0000 7fc2c476a740 20 client.18726 populate_metadata read hostname 'smithi116' 2024-09-12T03:15:08.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.779+0000 7fc2c476a740 1 --2- 172.21.15.116:0/1569976136 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556fdf34e510 0x556fdf350930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:08.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.779+0000 7fc2c476a740 4 client.18726 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:15:08.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.779+0000 7fc2c476a740 1 -- 172.21.15.116:0/1569976136 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556fdf230d00 con 0x556fdf34e510 2024-09-12T03:15:08.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.779+0000 7fc2bbfff640 1 --2- 172.21.15.116:0/1569976136 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556fdf34e510 0x556fdf350930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:08.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.779+0000 7fc2bbfff640 1 --2- 172.21.15.116:0/1569976136 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556fdf34e510 0x556fdf350930 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1647751101ca9bf server_cookie=8ddc7659524c5f6e in_seq=0 out_seq=0 2024-09-12T03:15:08.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.779+0000 7fc29b7fe640 10 client.18726 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:08.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.780+0000 7fc29b7fe640 1 -- 172.21.15.116:0/1569976136 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556fdf230d00 con 0x556fdf34e510 2024-09-12T03:15:08.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.780+0000 7fc29b7fe640 10 client.18726 handle_command_reply: tid=0 2024-09-12T03:15:08.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.780+0000 7fc2c476a740 1 client.18726 shutdown 2024-09-12T03:15:08.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.780+0000 7fc2c476a740 1 -- 172.21.15.116:0/1569976136 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fdf2c4380 msgr2=0x556fdf2d7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:08.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.780+0000 7fc2c476a740 1 --2- 172.21.15.116:0/1569976136 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fdf2c4380 0x556fdf2d7240 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4001eb0 tx=0x7fc2a400f7f0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.781+0000 7fc2c476a740 1 -- 172.21.15.116:0/1569976136 shutdown_connections 2024-09-12T03:15:08.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.781+0000 7fc2c476a740 1 --2- 172.21.15.116:0/1569976136 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556fdf34e510 0x556fdf350930 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.781+0000 7fc2c476a740 1 --2- 172.21.15.116:0/1569976136 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556fdf2d3150 0x556fdf2d7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.781+0000 7fc29b7fe640 0 client.18726 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:08.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.781+0000 7fc2c476a740 1 --2- 172.21.15.116:0/1569976136 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556fdf2c4380 0x556fdf2d7240 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.781+0000 7fc2c476a740 1 --2- 172.21.15.116:0/1569976136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556fdf2c3a60 0x556fdf2d6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.781+0000 7fc2c476a740 1 -- 172.21.15.116:0/1569976136 >> 172.21.15.116:0/1569976136 conn(0x556fdf2c00f0 msgr2=0x556fdf2c5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:08.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.781+0000 7fc2c476a740 1 -- 172.21.15.116:0/1569976136 shutdown_connections 2024-09-12T03:15:08.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.781+0000 7fc2c476a740 1 -- 172.21.15.116:0/1569976136 wait complete. 2024-09-12T03:15:08.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.781+0000 7fc2c476a740 20 client.18726 trim_cache size 0 max 16384 2024-09-12T03:15:08.786 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:15:08.787 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:15:08.787 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "4aeeef7e-74fc-4c1d-8637-f2444fc38830", 2024-09-12T03:15:08.787 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:15:08.787 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:15:08.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.782+0000 7fc2c2391640 1 -- 172.21.15.116:0/1360374031 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc2a80a2b80 msgr2=0x7fc2a80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:08.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.782+0000 7fc2c2391640 1 --2- 172.21.15.116:0/1360374031 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc2a80a2b80 0x7fc2a80a5040 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4009fd0 tx=0x7fc2b4009ee0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.782+0000 7fc2c2391640 1 -- 172.21.15.116:0/1360374031 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2bc101de0 msgr2=0x7fc2bc198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:08.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.782+0000 7fc2c2391640 1 --2- 172.21.15.116:0/1360374031 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2bc101de0 0x7fc2bc198480 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac005ab0 tx=0x7fc2ac0042f0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.782+0000 7fc2c2391640 1 -- 172.21.15.116:0/1360374031 shutdown_connections 2024-09-12T03:15:08.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.782+0000 7fc2c2391640 1 --2- 172.21.15.116:0/1360374031 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fc2a80a2b80 0x7fc2a80a5040 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.782+0000 7fc2c2391640 1 --2- 172.21.15.116:0/1360374031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2bc106f10 0x7fc2bc19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.782+0000 7fc2c2391640 1 --2- 172.21.15.116:0/1360374031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2bc102790 0x7fc2bc1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.782+0000 7fc2c2391640 1 --2- 172.21.15.116:0/1360374031 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2bc101de0 0x7fc2bc198480 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:08.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.782+0000 7fc2c2391640 1 -- 172.21.15.116:0/1360374031 >> 172.21.15.116:0/1360374031 conn(0x7fc2bc0fd8c0 msgr2=0x7fc2bc0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:08.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.782+0000 7fc2c2391640 1 -- 172.21.15.116:0/1360374031 shutdown_connections 2024-09-12T03:15:08.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:08.782+0000 7fc2c2391640 1 -- 172.21.15.116:0/1360374031 wait complete. 2024-09-12T03:15:09.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:08 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:15:09.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:08 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:15:09.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:08 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:15:09.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:08 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:15:09.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:08 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:15:09.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:08 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:15:09.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:08 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:15:09.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:08 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:15:09.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:08 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:15:09.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:08 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:15:09.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:08 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:15:09.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:08 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:15:09.656 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:15:09.903 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:15:10.024 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:09 smithi116 ceph-mon[31410]: pgmap v703: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:15:10.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:09 smithi183 ceph-mon[35535]: pgmap v703: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:15:10.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:09 smithi177 ceph-mon[35305]: pgmap v703: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:15:10.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.440+0000 7f8974d13640 1 -- 172.21.15.116:0/4228260684 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8970106a60 msgr2=0x7f8970106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:10.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.440+0000 7f8974d13640 1 --2- 172.21.15.116:0/4228260684 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8970106a60 0x7f8970106e40 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f8960009930 tx=0x7f896002a420 comp rx=0 tx=0).stop 2024-09-12T03:15:10.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.440+0000 7f8974d13640 1 -- 172.21.15.116:0/4228260684 shutdown_connections 2024-09-12T03:15:10.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.440+0000 7f8974d13640 1 --2- 172.21.15.116:0/4228260684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8970101400 0x7f897010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.440+0000 7f8974d13640 1 --2- 172.21.15.116:0/4228260684 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8970100a60 0x7f8970100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.440+0000 7f8974d13640 1 --2- 172.21.15.116:0/4228260684 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8970106a60 0x7f8970106e40 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.440+0000 7f8974d13640 1 -- 172.21.15.116:0/4228260684 >> 172.21.15.116:0/4228260684 conn(0x7f89700fc780 msgr2=0x7f89700feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:10.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.440+0000 7f8974d13640 1 -- 172.21.15.116:0/4228260684 shutdown_connections 2024-09-12T03:15:10.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.440+0000 7f8974d13640 1 -- 172.21.15.116:0/4228260684 wait complete. 2024-09-12T03:15:10.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.441+0000 7f8974d13640 1 Processor -- start 2024-09-12T03:15:10.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.441+0000 7f8974d13640 1 -- start start 2024-09-12T03:15:10.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.441+0000 7f8974d13640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8970100a60 0x7f8970197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:10.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.441+0000 7f8974d13640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8970101400 0x7f89701976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:10.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.441+0000 7f8974d13640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8970106a60 0x7f8970197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:10.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.441+0000 7f8974d13640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8970110e80 con 0x7f8970106a60 2024-09-12T03:15:10.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.441+0000 7f8974d13640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8970110d00 con 0x7f8970101400 2024-09-12T03:15:10.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.441+0000 7f8974d13640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8970111000 con 0x7f8970100a60 2024-09-12T03:15:10.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.441+0000 7f896ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8970106a60 0x7f8970197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:10.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.441+0000 7f896ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8970106a60 0x7f8970197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47684/0 (socket says 172.21.15.116:47684) 2024-09-12T03:15:10.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.441+0000 7f896ed9d640 1 -- 172.21.15.116:0/315149743 learned_addr learned my addr 172.21.15.116:0/315149743 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:10.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.441+0000 7f896e59c640 1 --2- 172.21.15.116:0/315149743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8970100a60 0x7f8970197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:10.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.441+0000 7f896dd9b640 1 --2- 172.21.15.116:0/315149743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8970101400 0x7f89701976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:10.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.441+0000 7f896ed9d640 1 -- 172.21.15.116:0/315149743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8970100a60 msgr2=0x7f8970197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:10.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.441+0000 7f896ed9d640 1 --2- 172.21.15.116:0/315149743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8970100a60 0x7f8970197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.441+0000 7f896ed9d640 1 -- 172.21.15.116:0/315149743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8970101400 msgr2=0x7f89701976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:10.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.441+0000 7f896ed9d640 1 --2- 172.21.15.116:0/315149743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8970101400 0x7f89701976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.441+0000 7f896ed9d640 1 -- 172.21.15.116:0/315149743 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89701a69d0 con 0x7f8970106a60 2024-09-12T03:15:10.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.442+0000 7f896ed9d640 1 --2- 172.21.15.116:0/315149743 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8970106a60 0x7f8970197be0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f896800b970 tx=0x7f896800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:10.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.442+0000 7f895f7fe640 1 -- 172.21.15.116:0/315149743 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8968019070 con 0x7f8970106a60 2024-09-12T03:15:10.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.442+0000 7f895f7fe640 1 -- 172.21.15.116:0/315149743 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f896800cb40 con 0x7f8970106a60 2024-09-12T03:15:10.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.442+0000 7f8974d13640 1 -- 172.21.15.116:0/315149743 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89701a6cc0 con 0x7f8970106a60 2024-09-12T03:15:10.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.442+0000 7f895f7fe640 1 -- 172.21.15.116:0/315149743 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8968007650 con 0x7f8970106a60 2024-09-12T03:15:10.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.442+0000 7f8974d13640 1 -- 172.21.15.116:0/315149743 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89701a7200 con 0x7f8970106a60 2024-09-12T03:15:10.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.444+0000 7f895f7fe640 1 -- 172.21.15.116:0/315149743 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f896801c070 con 0x7f8970106a60 2024-09-12T03:15:10.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.444+0000 7f89770ec740 1 Processor -- start 2024-09-12T03:15:10.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.445+0000 7f89770ec740 1 -- start start 2024-09-12T03:15:10.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.445+0000 7f89770ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620ae07a4a0 0x5620ae07a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:10.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.445+0000 7f89770ec740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5620ae07adc0 0x5620ae085140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:10.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.445+0000 7f89770ec740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5620ae085680 0x5620ae087a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:10.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.445+0000 7f89770ec740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5620adfe02e0 con 0x5620ae085680 2024-09-12T03:15:10.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.445+0000 7f89770ec740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5620adf87ce0 con 0x5620ae07adc0 2024-09-12T03:15:10.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.445+0000 7f89770ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5620adfdc2a0 con 0x5620ae07a4a0 2024-09-12T03:15:10.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.445+0000 7f896dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5620ae085680 0x5620ae087a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:10.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.445+0000 7f896dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5620ae085680 0x5620ae087a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47686/0 (socket says 172.21.15.116:47686) 2024-09-12T03:15:10.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.445+0000 7f896dd9b640 1 -- 172.21.15.116:0/573953567 learned_addr learned my addr 172.21.15.116:0/573953567 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:10.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.445+0000 7f895f7fe640 1 --2- 172.21.15.116:0/315149743 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f89580a2b80 0x7f89580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:10.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.445+0000 7f896dd9b640 1 --2- 172.21.15.116:0/573953567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620ae07a4a0 0x5620ae07a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:10.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.445+0000 7f896ed9d640 1 --2- 172.21.15.116:0/315149743 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f89580a2b80 0x7f89580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:10.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.445+0000 7f896e59c640 1 --2- 172.21.15.116:0/573953567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5620ae07adc0 0x5620ae085140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:10.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.445+0000 7f896dd9b640 1 -- 172.21.15.116:0/573953567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620ae07a4a0 msgr2=0x5620ae07a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:10.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.445+0000 7f896dd9b640 1 --2- 172.21.15.116:0/573953567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620ae07a4a0 0x5620ae07a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.445+0000 7f896dd9b640 1 -- 172.21.15.116:0/573953567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5620ae07adc0 msgr2=0x5620ae085140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:10.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.445+0000 7f896dd9b640 1 --2- 172.21.15.116:0/573953567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5620ae07adc0 0x5620ae085140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.445+0000 7f895f7fe640 1 -- 172.21.15.116:0/315149743 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f89680ce5a0 con 0x7f8970106a60 2024-09-12T03:15:10.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.445+0000 7f896dd9b640 1 -- 172.21.15.116:0/573953567 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620adf6c440 con 0x5620ae085680 2024-09-12T03:15:10.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.446+0000 7f896dd9b640 1 --2- 172.21.15.116:0/573953567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5620ae085680 0x5620ae087a70 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f8964011490 tx=0x7f896400a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4d2907684f1134 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:10.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.446+0000 7f896ed9d640 1 --2- 172.21.15.116:0/315149743 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f89580a2b80 0x7f89580a5040 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f89680cc440 tx=0x7f8968066000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:10.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.446+0000 7f895cff9640 1 -- 172.21.15.116:0/573953567 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8964005220 con 0x5620ae085680 2024-09-12T03:15:10.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.446+0000 7f895cff9640 1 -- 172.21.15.116:0/573953567 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8964004c80 con 0x5620ae085680 2024-09-12T03:15:10.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.446+0000 7f895cff9640 1 -- 172.21.15.116:0/573953567 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896400fcf0 con 0x5620ae085680 2024-09-12T03:15:10.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.447+0000 7f89770ec740 1 -- 172.21.15.116:0/573953567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5620ae085680 msgr2=0x5620ae087a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:10.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.447+0000 7f89770ec740 1 --2- 172.21.15.116:0/573953567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5620ae085680 0x5620ae087a70 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f8964011490 tx=0x7f896400a750 comp rx=0 tx=0).stop 2024-09-12T03:15:10.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.447+0000 7f89770ec740 1 -- 172.21.15.116:0/573953567 shutdown_connections 2024-09-12T03:15:10.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.447+0000 7f89770ec740 1 --2- 172.21.15.116:0/573953567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5620ae085680 0x5620ae087a70 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.447+0000 7f89770ec740 1 --2- 172.21.15.116:0/573953567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5620ae07adc0 0x5620ae085140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.447+0000 7f89770ec740 1 --2- 172.21.15.116:0/573953567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620ae07a4a0 0x5620ae07a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.447+0000 7f89770ec740 1 -- 172.21.15.116:0/573953567 >> 172.21.15.116:0/573953567 conn(0x5620ae076a30 msgr2=0x5620ae076e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:10.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.447+0000 7f89770ec740 1 -- 172.21.15.116:0/573953567 shutdown_connections 2024-09-12T03:15:10.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.447+0000 7f89770ec740 1 -- 172.21.15.116:0/573953567 wait complete. 2024-09-12T03:15:10.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.447+0000 7f89770ec740 1 -- start start 2024-09-12T03:15:10.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.447+0000 7f89770ec740 1 Processor -- start 2024-09-12T03:15:10.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.447+0000 7f89770ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620ae07a4a0 0x5620ae085050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:10.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.447+0000 7f89770ec740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5620ae07adc0 0x5620ae082300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:10.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.447+0000 7f89770ec740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5620ae085680 0x5620ae0828f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:10.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.447+0000 7f89770ec740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5620ae088250 con 0x5620ae085680 2024-09-12T03:15:10.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.447+0000 7f89770ec740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5620adf9b050 con 0x5620ae07adc0 2024-09-12T03:15:10.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.447+0000 7f89770ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5620ae0883d0 con 0x5620ae07a4a0 2024-09-12T03:15:10.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.447+0000 7f89770ec740 10 client.0 _ll_register_callbacks cb 0x5620adfdb160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:10.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.448+0000 7f896dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5620ae085680 0x5620ae0828f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:10.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.448+0000 7f896dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5620ae085680 0x5620ae0828f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47694/0 (socket says 172.21.15.116:47694) 2024-09-12T03:15:10.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.448+0000 7f896dd9b640 1 -- 172.21.15.116:0/2972958452 learned_addr learned my addr 172.21.15.116:0/2972958452 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:10.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.448+0000 7f896dd9b640 1 --2- 172.21.15.116:0/2972958452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620ae07a4a0 0x5620ae085050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:10.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.448+0000 7f896e59c640 1 --2- 172.21.15.116:0/2972958452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5620ae07adc0 0x5620ae082300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:10.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.448+0000 7f896dd9b640 1 -- 172.21.15.116:0/2972958452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620ae07a4a0 msgr2=0x5620ae085050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:10.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.448+0000 7f896dd9b640 1 --2- 172.21.15.116:0/2972958452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620ae07a4a0 0x5620ae085050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.448+0000 7f896dd9b640 1 -- 172.21.15.116:0/2972958452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5620ae07adc0 msgr2=0x5620ae082300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:10.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.448+0000 7f896dd9b640 1 --2- 172.21.15.116:0/2972958452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5620ae07adc0 0x5620ae082300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.448+0000 7f896dd9b640 1 -- 172.21.15.116:0/2972958452 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620add513c0 con 0x5620ae085680 2024-09-12T03:15:10.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.448+0000 7f896dd9b640 1 --2- 172.21.15.116:0/2972958452 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5620ae085680 0x5620ae0828f0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f8964011de0 tx=0x7f896400f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1f5eb82c8a3cd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:10.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.448+0000 7f8955ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:15:10.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.448+0000 7f8955ffb640 1 -- 172.21.15.116:0/2972958452 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8964004ae0 con 0x5620ae085680 2024-09-12T03:15:10.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.448+0000 7f8955ffb640 1 -- 172.21.15.116:0/2972958452 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f8964004220 con 0x5620ae085680 2024-09-12T03:15:10.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.448+0000 7f89770ec740 1 -- 172.21.15.116:0/2972958452 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8964004c80 con 0x5620ae085680 2024-09-12T03:15:10.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.448+0000 7f8955ffb640 1 -- 172.21.15.116:0/2972958452 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8964004420 con 0x5620ae085680 2024-09-12T03:15:10.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.448+0000 7f89770ec740 10 client.18738 fetch_fsmap learned FSMap version 25 2024-09-12T03:15:10.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.448+0000 7f89770ec740 1 -- 172.21.15.116:0/2972958452 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5620ae093c80 con 0x5620ae085680 2024-09-12T03:15:10.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.449+0000 7f8955ffb640 1 -- 172.21.15.116:0/2972958452 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8964004c80 con 0x5620ae085680 2024-09-12T03:15:10.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.449+0000 7f8955ffb640 1 -- 172.21.15.116:0/2972958452 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f896401b030 con 0x5620ae085680 2024-09-12T03:15:10.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.449+0000 7f89770ec740 10 client.18738 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:15:10.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.449+0000 7f89770ec740 10 client.18738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:15:10.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.449+0000 7f89770ec740 20 client.18738 populate_metadata read hostname 'smithi116' 2024-09-12T03:15:10.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.449+0000 7f89770ec740 1 --2- 172.21.15.116:0/2972958452 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5620ae095090 0x5620ae097480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:10.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.449+0000 7f89770ec740 4 client.18738 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:15:10.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.449+0000 7f89770ec740 1 -- 172.21.15.116:0/2972958452 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620adfed850 con 0x5620ae095090 2024-09-12T03:15:10.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.449+0000 7f896e59c640 1 --2- 172.21.15.116:0/2972958452 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5620ae095090 0x5620ae097480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:10.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.449+0000 7f896e59c640 1 --2- 172.21.15.116:0/2972958452 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5620ae095090 0x5620ae097480 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98aaf8411e77af94 server_cookie=4712ba5b20c7bece in_seq=0 out_seq=0 2024-09-12T03:15:10.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.449+0000 7f8955ffb640 10 client.18738 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:10.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.450+0000 7f8955ffb640 1 -- 172.21.15.116:0/2972958452 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5620adfed850 con 0x5620ae095090 2024-09-12T03:15:10.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.450+0000 7f8955ffb640 10 client.18738 handle_command_reply: tid=0 2024-09-12T03:15:10.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.450+0000 7f89770ec740 1 client.18738 shutdown 2024-09-12T03:15:10.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.450+0000 7f89770ec740 1 -- 172.21.15.116:0/2972958452 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5620ae085680 msgr2=0x5620ae0828f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:10.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.450+0000 7f89770ec740 1 --2- 172.21.15.116:0/2972958452 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5620ae085680 0x5620ae0828f0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f8964011de0 tx=0x7f896400f8d0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.450+0000 7f89770ec740 1 -- 172.21.15.116:0/2972958452 shutdown_connections 2024-09-12T03:15:10.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.450+0000 7f89770ec740 1 --2- 172.21.15.116:0/2972958452 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5620ae095090 0x5620ae097480 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.450+0000 7f89770ec740 1 --2- 172.21.15.116:0/2972958452 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5620ae085680 0x5620ae0828f0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.450+0000 7f89770ec740 1 --2- 172.21.15.116:0/2972958452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5620ae07adc0 0x5620ae082300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.450+0000 7f89770ec740 1 --2- 172.21.15.116:0/2972958452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620ae07a4a0 0x5620ae085050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.450+0000 7f89770ec740 1 -- 172.21.15.116:0/2972958452 >> 172.21.15.116:0/2972958452 conn(0x5620ae076a30 msgr2=0x5620ae07c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:10.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.450+0000 7f8955ffb640 0 client.18738 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:10.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.450+0000 7f89770ec740 1 -- 172.21.15.116:0/2972958452 shutdown_connections 2024-09-12T03:15:10.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.450+0000 7f89770ec740 1 -- 172.21.15.116:0/2972958452 wait complete. 2024-09-12T03:15:10.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.451+0000 7f89770ec740 20 client.18738 trim_cache size 0 max 16384 2024-09-12T03:15:10.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.467+0000 7f89770ec740 1 Processor -- start 2024-09-12T03:15:10.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.467+0000 7f89770ec740 1 -- start start 2024-09-12T03:15:10.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.467+0000 7f89770ec740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5620ae07a4a0 0x5620ae0914b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:10.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.467+0000 7f89770ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620ae07adc0 0x5620ae09cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:10.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.467+0000 7f89770ec740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5620ae085680 0x5620ae09d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:10.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.467+0000 7f89770ec740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f89701b20f0 con 0x5620ae085680 2024-09-12T03:15:10.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.467+0000 7f89770ec740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f89701af9c0 con 0x5620ae07a4a0 2024-09-12T03:15:10.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.467+0000 7f89770ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f89701ad810 con 0x5620ae07adc0 2024-09-12T03:15:10.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.468+0000 7f896dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5620ae085680 0x5620ae09d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:10.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.468+0000 7f896dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5620ae085680 0x5620ae09d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47696/0 (socket says 172.21.15.116:47696) 2024-09-12T03:15:10.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.468+0000 7f896dd9b640 1 -- 172.21.15.116:0/1517562218 learned_addr learned my addr 172.21.15.116:0/1517562218 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:10.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.468+0000 7f896e59c640 1 --2- 172.21.15.116:0/1517562218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620ae07adc0 0x5620ae09cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:10.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.468+0000 7f896dd9b640 1 -- 172.21.15.116:0/1517562218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620ae07adc0 msgr2=0x5620ae09cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:10.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.468+0000 7f896dd9b640 1 --2- 172.21.15.116:0/1517562218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620ae07adc0 0x5620ae09cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.468+0000 7f896dd9b640 1 -- 172.21.15.116:0/1517562218 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5620ae07a4a0 msgr2=0x5620ae0914b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:10.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.468+0000 7f896dd9b640 1 --2- 172.21.15.116:0/1517562218 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5620ae07a4a0 0x5620ae0914b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.468+0000 7f896dd9b640 1 -- 172.21.15.116:0/1517562218 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620ae07abf0 con 0x5620ae085680 2024-09-12T03:15:10.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.468+0000 7f896e59c640 1 --2- 172.21.15.116:0/1517562218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620ae07adc0 0x5620ae09cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:15:10.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.468+0000 7f896dd9b640 1 --2- 172.21.15.116:0/1517562218 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5620ae085680 0x5620ae09d0e0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f896401f040 tx=0x7f8964003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ceee1fa9ab33013 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:10.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.468+0000 7f8957fff640 1 -- 172.21.15.116:0/1517562218 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896400a6a0 con 0x5620ae085680 2024-09-12T03:15:10.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.468+0000 7f8957fff640 1 -- 172.21.15.116:0/1517562218 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8964020030 con 0x5620ae085680 2024-09-12T03:15:10.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.468+0000 7f8957fff640 1 -- 172.21.15.116:0/1517562218 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8964005000 con 0x5620ae085680 2024-09-12T03:15:10.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.469+0000 7f89770ec740 1 -- 172.21.15.116:0/1517562218 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5620ae085680 msgr2=0x5620ae09d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:10.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.469+0000 7f89770ec740 1 --2- 172.21.15.116:0/1517562218 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5620ae085680 0x5620ae09d0e0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f896401f040 tx=0x7f8964003e80 comp rx=0 tx=0).stop 2024-09-12T03:15:10.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.469+0000 7f89770ec740 1 -- 172.21.15.116:0/1517562218 shutdown_connections 2024-09-12T03:15:10.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.469+0000 7f89770ec740 1 --2- 172.21.15.116:0/1517562218 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5620ae085680 0x5620ae09d0e0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.469+0000 7f89770ec740 1 --2- 172.21.15.116:0/1517562218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620ae07adc0 0x5620ae09cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.469+0000 7f89770ec740 1 --2- 172.21.15.116:0/1517562218 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5620ae07a4a0 0x5620ae0914b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.469+0000 7f89770ec740 1 -- 172.21.15.116:0/1517562218 >> 172.21.15.116:0/1517562218 conn(0x5620ae076a30 msgr2=0x5620ae0813f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:10.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.469+0000 7f89770ec740 1 -- 172.21.15.116:0/1517562218 shutdown_connections 2024-09-12T03:15:10.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.469+0000 7f89770ec740 1 -- 172.21.15.116:0/1517562218 wait complete. 2024-09-12T03:15:10.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.469+0000 7f89770ec740 1 -- start start 2024-09-12T03:15:10.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.469+0000 7f89770ec740 1 Processor -- start 2024-09-12T03:15:10.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.470+0000 7f89770ec740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5620ae07a4a0 0x5620ae08d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:10.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.470+0000 7f89770ec740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5620ae07adc0 0x5620ae08dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:10.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.470+0000 7f89770ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620ae085680 0x5620ae088f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:10.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.470+0000 7f89770ec740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f89701ade80 con 0x5620ae07a4a0 2024-09-12T03:15:10.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.470+0000 7f89770ec740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f89701af2f0 con 0x5620ae07adc0 2024-09-12T03:15:10.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.470+0000 7f89770ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f89701ae550 con 0x5620ae085680 2024-09-12T03:15:10.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.470+0000 7f89770ec740 10 client.0 _ll_register_callbacks cb 0x7f896400af20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:10.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.470+0000 7f896dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5620ae07a4a0 0x5620ae08d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:10.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.470+0000 7f896dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5620ae07a4a0 0x5620ae08d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47702/0 (socket says 172.21.15.116:47702) 2024-09-12T03:15:10.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.470+0000 7f896dd9b640 1 -- 172.21.15.116:0/2371734076 learned_addr learned my addr 172.21.15.116:0/2371734076 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:10.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.470+0000 7f896dd9b640 1 --2- 172.21.15.116:0/2371734076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620ae085680 0x5620ae088f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:10.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.470+0000 7f896e59c640 1 --2- 172.21.15.116:0/2371734076 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5620ae07adc0 0x5620ae08dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:10.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.470+0000 7f896dd9b640 1 -- 172.21.15.116:0/2371734076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620ae085680 msgr2=0x5620ae088f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:10.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.470+0000 7f896dd9b640 1 --2- 172.21.15.116:0/2371734076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620ae085680 0x5620ae088f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.470+0000 7f896dd9b640 1 -- 172.21.15.116:0/2371734076 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5620ae07adc0 msgr2=0x5620ae08dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:10.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.470+0000 7f896dd9b640 1 --2- 172.21.15.116:0/2371734076 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5620ae07adc0 0x5620ae08dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.470+0000 7f896dd9b640 1 -- 172.21.15.116:0/2371734076 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620ae0853c0 con 0x5620ae07a4a0 2024-09-12T03:15:10.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.470+0000 7f896e59c640 1 --2- 172.21.15.116:0/2371734076 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5620ae07adc0 0x5620ae08dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:15:10.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.470+0000 7f896dd9b640 1 --2- 172.21.15.116:0/2371734076 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5620ae07a4a0 0x5620ae08d9a0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f8964000c00 tx=0x7f8964005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b66f0adf8ecf6d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:10.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.470+0000 7f8955ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:15:10.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.470+0000 7f8955ffb640 1 -- 172.21.15.116:0/2371734076 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896403b040 con 0x5620ae07a4a0 2024-09-12T03:15:10.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.470+0000 7f8955ffb640 1 -- 172.21.15.116:0/2371734076 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f8964003660 con 0x5620ae07a4a0 2024-09-12T03:15:10.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.470+0000 7f89770ec740 1 -- 172.21.15.116:0/2371734076 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8964020030 con 0x5620ae07a4a0 2024-09-12T03:15:10.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.470+0000 7f8955ffb640 1 -- 172.21.15.116:0/2371734076 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8964003860 con 0x5620ae07a4a0 2024-09-12T03:15:10.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.470+0000 7f89770ec740 10 client.18744 fetch_fsmap learned FSMap version 25 2024-09-12T03:15:10.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.470+0000 7f89770ec740 1 -- 172.21.15.116:0/2371734076 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5620ae0ff570 con 0x5620ae07a4a0 2024-09-12T03:15:10.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.471+0000 7f8955ffb640 1 -- 172.21.15.116:0/2371734076 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8964020030 con 0x5620ae07a4a0 2024-09-12T03:15:10.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.471+0000 7f8955ffb640 1 -- 172.21.15.116:0/2371734076 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8964041070 con 0x5620ae07a4a0 2024-09-12T03:15:10.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.471+0000 7f89770ec740 10 client.18744 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:15:10.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.471+0000 7f89770ec740 10 client.18744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:15:10.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.471+0000 7f89770ec740 20 client.18744 populate_metadata read hostname 'smithi116' 2024-09-12T03:15:10.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.471+0000 7f89770ec740 1 --2- 172.21.15.116:0/2371734076 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5620ae0ffba0 0x5620ae101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:10.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.471+0000 7f89770ec740 4 client.18744 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:15:10.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.471+0000 7f89770ec740 1 -- 172.21.15.116:0/2371734076 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5620adfe7d60 con 0x5620ae0ffba0 2024-09-12T03:15:10.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.471+0000 7f896e59c640 1 --2- 172.21.15.116:0/2371734076 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5620ae0ffba0 0x5620ae101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:10.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.471+0000 7f896e59c640 1 --2- 172.21.15.116:0/2371734076 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5620ae0ffba0 0x5620ae101fc0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94691b442f585314 server_cookie=22457d59f73bf078 in_seq=0 out_seq=0 2024-09-12T03:15:10.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.471+0000 7f8955ffb640 10 client.18744 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:10.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.472+0000 7f8955ffb640 1 -- 172.21.15.116:0/2371734076 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5620adfe7d60 con 0x5620ae0ffba0 2024-09-12T03:15:10.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.472+0000 7f8955ffb640 10 client.18744 handle_command_reply: tid=0 2024-09-12T03:15:10.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.472+0000 7f89770ec740 1 client.18744 shutdown 2024-09-12T03:15:10.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.472+0000 7f89770ec740 1 -- 172.21.15.116:0/2371734076 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5620ae07a4a0 msgr2=0x5620ae08d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:10.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.472+0000 7f89770ec740 1 --2- 172.21.15.116:0/2371734076 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5620ae07a4a0 0x5620ae08d9a0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f8964000c00 tx=0x7f8964005fa0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.472+0000 7f89770ec740 1 -- 172.21.15.116:0/2371734076 shutdown_connections 2024-09-12T03:15:10.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.472+0000 7f89770ec740 1 --2- 172.21.15.116:0/2371734076 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5620ae0ffba0 0x5620ae101fc0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.472+0000 7f89770ec740 1 --2- 172.21.15.116:0/2371734076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5620ae085680 0x5620ae088f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.472+0000 7f89770ec740 1 --2- 172.21.15.116:0/2371734076 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5620ae07adc0 0x5620ae08dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.472+0000 7f8955ffb640 0 client.18744 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:10.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.472+0000 7f89770ec740 1 --2- 172.21.15.116:0/2371734076 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5620ae07a4a0 0x5620ae08d9a0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.472+0000 7f89770ec740 1 -- 172.21.15.116:0/2371734076 >> 172.21.15.116:0/2371734076 conn(0x5620ae076a30 msgr2=0x5620ae07bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:10.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.472+0000 7f89770ec740 1 -- 172.21.15.116:0/2371734076 shutdown_connections 2024-09-12T03:15:10.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.472+0000 7f89770ec740 1 -- 172.21.15.116:0/2371734076 wait complete. 2024-09-12T03:15:10.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.472+0000 7f89770ec740 20 client.18744 trim_cache size 0 max 16384 2024-09-12T03:15:10.479 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:15:10.479 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:15:10.479 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": { 2024-09-12T03:15:10.479 INFO:teuthology.orchestra.run.smithi116.stdout: "4aeeef7e-74fc-4c1d-8637-f2444fc38830": { 2024-09-12T03:15:10.479 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/", 2024-09-12T03:15:10.479 INFO:teuthology.orchestra.run.smithi116.stdout: "tag": "4aeeef7e-74fc-4c1d-8637-f2444fc38830", 2024-09-12T03:15:10.479 INFO:teuthology.orchestra.run.smithi116.stdout: "options": "recursive,force" 2024-09-12T03:15:10.479 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:15:10.479 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:15:10.480 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:15:10.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.472+0000 7f8974d13640 1 -- 172.21.15.116:0/315149743 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f89580a2b80 msgr2=0x7f89580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:10.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.472+0000 7f8974d13640 1 --2- 172.21.15.116:0/315149743 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f89580a2b80 0x7f89580a5040 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f89680cc440 tx=0x7f8968066000 comp rx=0 tx=0).stop 2024-09-12T03:15:10.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.473+0000 7f8974d13640 1 -- 172.21.15.116:0/315149743 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8970106a60 msgr2=0x7f8970197be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:10.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.473+0000 7f8974d13640 1 --2- 172.21.15.116:0/315149743 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8970106a60 0x7f8970197be0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f896800b970 tx=0x7f896800be30 comp rx=0 tx=0).stop 2024-09-12T03:15:10.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.473+0000 7f8974d13640 1 -- 172.21.15.116:0/315149743 shutdown_connections 2024-09-12T03:15:10.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.473+0000 7f8974d13640 1 --2- 172.21.15.116:0/315149743 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f89580a2b80 0x7f89580a5040 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.473+0000 7f8974d13640 1 --2- 172.21.15.116:0/315149743 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8970106a60 0x7f8970197be0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.473+0000 7f8974d13640 1 --2- 172.21.15.116:0/315149743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8970101400 0x7f89701976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.473+0000 7f8974d13640 1 --2- 172.21.15.116:0/315149743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8970100a60 0x7f8970197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:10.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.473+0000 7f8974d13640 1 -- 172.21.15.116:0/315149743 >> 172.21.15.116:0/315149743 conn(0x7f89700fc780 msgr2=0x7f89700fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:10.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.473+0000 7f8974d13640 1 -- 172.21.15.116:0/315149743 shutdown_connections 2024-09-12T03:15:10.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:10.473+0000 7f8974d13640 1 -- 172.21.15.116:0/315149743 wait complete. 2024-09-12T03:15:11.168 INFO:tasks.cephfs.filesystem:scrub status for tag:4aeeef7e-74fc-4c1d-8637-f2444fc38830 - {'path': '/', 'tag': '4aeeef7e-74fc-4c1d-8637-f2444fc38830', 'options': 'recursive,force'} 2024-09-12T03:15:11.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:11 smithi183 ceph-mon[35535]: pgmap v704: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.6 KiB/s wr, 9 op/s 2024-09-12T03:15:11.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:11 smithi177 ceph-mon[35305]: pgmap v704: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.6 KiB/s wr, 9 op/s 2024-09-12T03:15:11.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:11 smithi116 ceph-mon[31410]: pgmap v704: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.6 KiB/s wr, 9 op/s 2024-09-12T03:15:13.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:13 smithi183 ceph-mon[35535]: pgmap v705: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.6 KiB/s wr, 7 op/s 2024-09-12T03:15:13.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:13 smithi177 ceph-mon[35305]: pgmap v705: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.6 KiB/s wr, 7 op/s 2024-09-12T03:15:13.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:13 smithi116 ceph-mon[31410]: pgmap v705: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.6 KiB/s wr, 7 op/s 2024-09-12T03:15:14.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:14 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:15:14.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:14 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:15:14.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:14 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:15:15.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:15 smithi183 ceph-mon[35535]: pgmap v706: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.0 KiB/s wr, 8 op/s 2024-09-12T03:15:15.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:15 smithi177 ceph-mon[35305]: pgmap v706: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.0 KiB/s wr, 8 op/s 2024-09-12T03:15:15.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:15 smithi116 ceph-mon[31410]: pgmap v706: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.0 KiB/s wr, 8 op/s 2024-09-12T03:15:17.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:17 smithi183 ceph-mon[35535]: pgmap v707: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-12T03:15:17.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:17 smithi177 ceph-mon[35305]: pgmap v707: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-12T03:15:17.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:17 smithi116 ceph-mon[31410]: pgmap v707: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-12T03:15:19.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:19 smithi183 ceph-mon[35535]: pgmap v708: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-12T03:15:19.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:19 smithi177 ceph-mon[35305]: pgmap v708: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-12T03:15:19.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:19 smithi116 ceph-mon[31410]: pgmap v708: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-12T03:15:21.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:21 smithi183 ceph-mon[35535]: pgmap v709: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-12T03:15:21.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:21 smithi177 ceph-mon[35305]: pgmap v709: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-12T03:15:21.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:21 smithi116 ceph-mon[31410]: pgmap v709: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-12T03:15:22.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:22 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:15:22.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:22 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:15:22.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:22 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:15:22.705 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:22 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:15:22.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:22 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:15:22.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:22 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:22.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:22 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:22.706 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:22 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:15:22.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:22 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:15:22.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:22 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:15:22.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:22 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:15:22.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:22 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:15:22.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:22 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:15:22.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:22 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:22.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:22 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:22.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:22 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:15:22.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:22 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:15:22.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:22 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:15:22.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:22 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:15:22.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:22 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:15:22.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:22 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:15:22.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:22 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:22.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:22 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:22.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:22 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:15:23.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:23 smithi183 ceph-mon[35535]: pgmap v710: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-12T03:15:23.741 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:23 smithi116 ceph-mon[31410]: pgmap v710: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-12T03:15:23.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:23 smithi177 ceph-mon[35305]: pgmap v710: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-12T03:15:25.704 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:25 smithi183 ceph-mon[35535]: pgmap v711: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T03:15:25.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:25 smithi177 ceph-mon[35305]: pgmap v711: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T03:15:25.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:25 smithi116 ceph-mon[31410]: pgmap v711: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T03:15:26.464 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:15:26.723 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:15:27.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.214+0000 7f3a99534640 1 -- 172.21.15.116:0/1684247099 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a94106f10 msgr2=0x7f3a9410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:27.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.214+0000 7f3a99534640 1 --2- 172.21.15.116:0/1684247099 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a94106f10 0x7f3a9410ea90 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c009ca0 tx=0x7f3a8c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:15:27.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.214+0000 7f3a99534640 1 -- 172.21.15.116:0/1684247099 shutdown_connections 2024-09-12T03:15:27.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.214+0000 7f3a99534640 1 --2- 172.21.15.116:0/1684247099 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a94106f10 0x7f3a9410ea90 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:27.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.214+0000 7f3a99534640 1 --2- 172.21.15.116:0/1684247099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a94102790 0x7f3a941067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:27.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.214+0000 7f3a99534640 1 --2- 172.21.15.116:0/1684247099 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a94101de0 0x7f3a941021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:27.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.214+0000 7f3a99534640 1 -- 172.21.15.116:0/1684247099 >> 172.21.15.116:0/1684247099 conn(0x7f3a940fd8c0 msgr2=0x7f3a940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:27.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.214+0000 7f3a99534640 1 -- 172.21.15.116:0/1684247099 shutdown_connections 2024-09-12T03:15:27.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.214+0000 7f3a99534640 1 -- 172.21.15.116:0/1684247099 wait complete. 2024-09-12T03:15:27.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.215+0000 7f3a99534640 1 Processor -- start 2024-09-12T03:15:27.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.215+0000 7f3a99534640 1 -- start start 2024-09-12T03:15:27.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.215+0000 7f3a99534640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a94101de0 0x7f3a94198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:27.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.215+0000 7f3a99534640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a94102790 0x7f3a94198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:27.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.215+0000 7f3a99534640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a94106f10 0x7f3a9419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:27.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.215+0000 7f3a99534640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3a94111f70 con 0x7f3a94101de0 2024-09-12T03:15:27.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.215+0000 7f3a99534640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3a94111df0 con 0x7f3a94102790 2024-09-12T03:15:27.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.215+0000 7f3a99534640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3a941120f0 con 0x7f3a94106f10 2024-09-12T03:15:27.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.215+0000 7f3a92ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a94101de0 0x7f3a94198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:27.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.215+0000 7f3a92ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a94101de0 0x7f3a94198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55974/0 (socket says 172.21.15.116:55974) 2024-09-12T03:15:27.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.215+0000 7f3a92ffd640 1 -- 172.21.15.116:0/3125423157 learned_addr learned my addr 172.21.15.116:0/3125423157 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:27.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.215+0000 7f3a8bfff640 1 --2- 172.21.15.116:0/3125423157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a94102790 0x7f3a94198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:27.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.215+0000 7f3a92ffd640 1 -- 172.21.15.116:0/3125423157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a94106f10 msgr2=0x7f3a9419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:27.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.215+0000 7f3a92ffd640 1 --2- 172.21.15.116:0/3125423157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a94106f10 0x7f3a9419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:27.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.215+0000 7f3a92ffd640 1 -- 172.21.15.116:0/3125423157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a94102790 msgr2=0x7f3a94198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:27.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.215+0000 7f3a92ffd640 1 --2- 172.21.15.116:0/3125423157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a94102790 0x7f3a94198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:27.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.216+0000 7f3a92ffd640 1 -- 172.21.15.116:0/3125423157 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a9419d2e0 con 0x7f3a94101de0 2024-09-12T03:15:27.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.216+0000 7f3a8bfff640 1 --2- 172.21.15.116:0/3125423157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a94102790 0x7f3a94198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:15:27.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.216+0000 7f3a92ffd640 1 --2- 172.21.15.116:0/3125423157 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a94101de0 0x7f3a94198330 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400c920 tx=0x7f3a8400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:27.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.216+0000 7f3a90ff9640 1 -- 172.21.15.116:0/3125423157 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84007c70 con 0x7f3a94101de0 2024-09-12T03:15:27.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.216+0000 7f3a90ff9640 1 -- 172.21.15.116:0/3125423157 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a84007e10 con 0x7f3a94101de0 2024-09-12T03:15:27.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.216+0000 7f3a99534640 1 -- 172.21.15.116:0/3125423157 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a9419d5d0 con 0x7f3a94101de0 2024-09-12T03:15:27.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.216+0000 7f3a90ff9640 1 -- 172.21.15.116:0/3125423157 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84018890 con 0x7f3a94101de0 2024-09-12T03:15:27.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.216+0000 7f3a99534640 1 -- 172.21.15.116:0/3125423157 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a941a4eb0 con 0x7f3a94101de0 2024-09-12T03:15:27.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.217+0000 7f3a99534640 1 -- 172.21.15.116:0/3125423157 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a941021c0 con 0x7f3a94101de0 2024-09-12T03:15:27.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.218+0000 7f3a90ff9640 1 -- 172.21.15.116:0/3125423157 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3a84018a50 con 0x7f3a94101de0 2024-09-12T03:15:27.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.219+0000 7f3a90ff9640 1 --2- 172.21.15.116:0/3125423157 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3a800a2b80 0x7f3a800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:27.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.219+0000 7f3a8bfff640 1 --2- 172.21.15.116:0/3125423157 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3a800a2b80 0x7f3a800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:27.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.219+0000 7f3a90ff9640 1 -- 172.21.15.116:0/3125423157 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f3a840cf440 con 0x7f3a94101de0 2024-09-12T03:15:27.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.219+0000 7f3a8bfff640 1 --2- 172.21.15.116:0/3125423157 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3a800a2b80 0x7f3a800a5040 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c005fd0 tx=0x7f3a7c0094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:27.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.222+0000 7f3a90ff9640 1 -- 172.21.15.116:0/3125423157 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3a84014030 con 0x7f3a94101de0 2024-09-12T03:15:27.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.421+0000 7f3a99534640 1 -- 172.21.15.116:0/3125423157 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a94199690 con 0x7f3a94101de0 2024-09-12T03:15:27.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.421+0000 7f3a90ff9640 1 -- 172.21.15.116:0/3125423157 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f3a8409c460 con 0x7f3a94101de0 2024-09-12T03:15:27.422 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:15:27.422 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:15:27.422 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:15:27.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.423+0000 7f3a99534640 1 -- 172.21.15.116:0/3125423157 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3a800a2b80 msgr2=0x7f3a800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:27.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.423+0000 7f3a99534640 1 --2- 172.21.15.116:0/3125423157 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3a800a2b80 0x7f3a800a5040 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c005fd0 tx=0x7f3a7c0094d0 comp rx=0 tx=0).stop 2024-09-12T03:15:27.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.423+0000 7f3a99534640 1 -- 172.21.15.116:0/3125423157 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a94101de0 msgr2=0x7f3a94198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:27.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.423+0000 7f3a99534640 1 --2- 172.21.15.116:0/3125423157 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a94101de0 0x7f3a94198330 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400c920 tx=0x7f3a8400cde0 comp rx=0 tx=0).stop 2024-09-12T03:15:27.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.424+0000 7f3a99534640 1 -- 172.21.15.116:0/3125423157 shutdown_connections 2024-09-12T03:15:27.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.424+0000 7f3a99534640 1 --2- 172.21.15.116:0/3125423157 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3a800a2b80 0x7f3a800a5040 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:27.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.424+0000 7f3a99534640 1 --2- 172.21.15.116:0/3125423157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a94106f10 0x7f3a9419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:27.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.424+0000 7f3a99534640 1 --2- 172.21.15.116:0/3125423157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a94102790 0x7f3a94198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:27.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.424+0000 7f3a99534640 1 --2- 172.21.15.116:0/3125423157 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a94101de0 0x7f3a94198330 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:27.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.424+0000 7f3a99534640 1 -- 172.21.15.116:0/3125423157 >> 172.21.15.116:0/3125423157 conn(0x7f3a940fd8c0 msgr2=0x7f3a940febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:27.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.424+0000 7f3a99534640 1 -- 172.21.15.116:0/3125423157 shutdown_connections 2024-09-12T03:15:27.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:27.424+0000 7f3a99534640 1 -- 172.21.15.116:0/3125423157 wait complete. 2024-09-12T03:15:27.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:27 smithi183 ceph-mon[35535]: pgmap v712: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-12T03:15:27.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:27 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3125423157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:15:27.708 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:27 smithi116 ceph-mon[31410]: pgmap v712: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-12T03:15:27.708 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:27 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3125423157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:15:27.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:27 smithi177 ceph-mon[35305]: pgmap v712: 97 pgs: 97 active+clean; 6.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-12T03:15:27.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:27 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3125423157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:15:28.267 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id 935d5531 --release --await' 2024-09-12T03:15:28.267 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id 935d5531 --release --await 2024-09-12T03:15:28.525 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:15:29.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.062+0000 7f2a0a2f5640 1 -- 172.21.15.116:0/1966913703 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a04101420 msgr2=0x7f2a0410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:29.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.062+0000 7f2a0a2f5640 1 --2- 172.21.15.116:0/1966913703 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a04101420 0x7f2a0410d950 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009cd0 tx=0x7f29fc02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.062+0000 7f2a0a2f5640 1 -- 172.21.15.116:0/1966913703 shutdown_connections 2024-09-12T03:15:29.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.062+0000 7f2a0a2f5640 1 --2- 172.21.15.116:0/1966913703 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a04101420 0x7f2a0410d950 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.062+0000 7f2a0a2f5640 1 --2- 172.21.15.116:0/1966913703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a04100a80 0x7f2a04100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.062+0000 7f2a0a2f5640 1 --2- 172.21.15.116:0/1966913703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a04106a80 0x7f2a04106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.062+0000 7f2a0a2f5640 1 -- 172.21.15.116:0/1966913703 >> 172.21.15.116:0/1966913703 conn(0x7f2a040fc7a0 msgr2=0x7f2a040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:29.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.062+0000 7f2a0a2f5640 1 -- 172.21.15.116:0/1966913703 shutdown_connections 2024-09-12T03:15:29.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.062+0000 7f2a0a2f5640 1 -- 172.21.15.116:0/1966913703 wait complete. 2024-09-12T03:15:29.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.063+0000 7f2a0a2f5640 1 Processor -- start 2024-09-12T03:15:29.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.063+0000 7f2a0a2f5640 1 -- start start 2024-09-12T03:15:29.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.063+0000 7f2a0a2f5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a04100a80 0x7f2a041992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:29.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.063+0000 7f2a0a2f5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a04101420 0x7f2a041997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:29.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.063+0000 7f2a0a2f5640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a04106a80 0x7f2a041a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:29.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.063+0000 7f2a0a2f5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2a04110e80 con 0x7f2a04100a80 2024-09-12T03:15:29.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.063+0000 7f2a0a2f5640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2a04110d00 con 0x7f2a04106a80 2024-09-12T03:15:29.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.063+0000 7f2a0a2f5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a04111000 con 0x7f2a04101420 2024-09-12T03:15:29.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.063+0000 7f2a03fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a04100a80 0x7f2a041992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:29.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.064+0000 7f2a03fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a04100a80 0x7f2a041992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55994/0 (socket says 172.21.15.116:55994) 2024-09-12T03:15:29.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.064+0000 7f2a03fff640 1 -- 172.21.15.116:0/4189871732 learned_addr learned my addr 172.21.15.116:0/4189871732 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:29.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.064+0000 7f2a08892640 1 --2- 172.21.15.116:0/4189871732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a04106a80 0x7f2a041a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:29.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.064+0000 7f2a03fff640 1 -- 172.21.15.116:0/4189871732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a04101420 msgr2=0x7f2a041997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:29.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.064+0000 7f2a03fff640 1 --2- 172.21.15.116:0/4189871732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a04101420 0x7f2a041997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.064+0000 7f2a03fff640 1 -- 172.21.15.116:0/4189871732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a04106a80 msgr2=0x7f2a041a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:29.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.064+0000 7f2a03fff640 1 --2- 172.21.15.116:0/4189871732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a04106a80 0x7f2a041a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.064+0000 7f2a03fff640 1 -- 172.21.15.116:0/4189871732 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a041a4860 con 0x7f2a04100a80 2024-09-12T03:15:29.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.064+0000 7f2a03fff640 1 --2- 172.21.15.116:0/4189871732 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a04100a80 0x7f2a041992a0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f29f400d770 tx=0x7f29f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:29.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.064+0000 7f2a017fa640 1 -- 172.21.15.116:0/4189871732 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f4017070 con 0x7f2a04100a80 2024-09-12T03:15:29.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.064+0000 7f2a017fa640 1 -- 172.21.15.116:0/4189871732 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29f4005500 con 0x7f2a04100a80 2024-09-12T03:15:29.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.064+0000 7f2a0a2f5640 1 -- 172.21.15.116:0/4189871732 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a041a4a30 con 0x7f2a04100a80 2024-09-12T03:15:29.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.064+0000 7f2a017fa640 1 -- 172.21.15.116:0/4189871732 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f4004e00 con 0x7f2a04100a80 2024-09-12T03:15:29.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.064+0000 7f2a0a2f5640 1 -- 172.21.15.116:0/4189871732 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a041a4ef0 con 0x7f2a04100a80 2024-09-12T03:15:29.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.066+0000 7f2a017fa640 1 -- 172.21.15.116:0/4189871732 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f29f4004fa0 con 0x7f2a04100a80 2024-09-12T03:15:29.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.067+0000 7f2a0c6ce740 1 Processor -- start 2024-09-12T03:15:29.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.067+0000 7f2a0c6ce740 1 -- start start 2024-09-12T03:15:29.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.067+0000 7f2a0c6ce740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562deba1c060 0x562deba1c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:29.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.067+0000 7f2a0c6ce740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562deba1c980 0x562deba26d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:29.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.067+0000 7f2a0c6ce740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562deba27240 0x562deba29630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:29.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.067+0000 7f2a0c6ce740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562deb981620 con 0x562deba1c060 2024-09-12T03:15:29.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.067+0000 7f2a0c6ce740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562deb928e50 con 0x562deba27240 2024-09-12T03:15:29.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.067+0000 7f2a0c6ce740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562deb97d5e0 con 0x562deba1c980 2024-09-12T03:15:29.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.067+0000 7f2a08892640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562deba1c060 0x562deba1c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:29.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.067+0000 7f2a08892640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562deba1c060 0x562deba1c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56010/0 (socket says 172.21.15.116:56010) 2024-09-12T03:15:29.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.067+0000 7f2a08892640 1 -- 172.21.15.116:0/2414880457 learned_addr learned my addr 172.21.15.116:0/2414880457 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:29.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.067+0000 7f2a017fa640 1 --2- 172.21.15.116:0/4189871732 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f29ec0a2b80 0x7f29ec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:29.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.067+0000 7f2a037fe640 1 --2- 172.21.15.116:0/2414880457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562deba27240 0x562deba29630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:29.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.067+0000 7f2a03fff640 1 --2- 172.21.15.116:0/2414880457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562deba1c980 0x562deba26d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:29.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.067+0000 7f2a08892640 1 --2- 172.21.15.116:0/4189871732 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f29ec0a2b80 0x7f29ec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:29.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.067+0000 7f2a08892640 1 -- 172.21.15.116:0/2414880457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562deba1c980 msgr2=0x562deba26d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:29.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.067+0000 7f2a08892640 1 --2- 172.21.15.116:0/2414880457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562deba1c980 0x562deba26d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.067+0000 7f2a08892640 1 -- 172.21.15.116:0/2414880457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562deba27240 msgr2=0x562deba29630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:29.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.067+0000 7f2a08892640 1 --2- 172.21.15.116:0/2414880457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562deba27240 0x562deba29630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.067+0000 7f2a08892640 1 -- 172.21.15.116:0/2414880457 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562deb919d40 con 0x562deba1c060 2024-09-12T03:15:29.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.067+0000 7f2a037fe640 1 --2- 172.21.15.116:0/2414880457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562deba27240 0x562deba29630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:15:29.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.067+0000 7f2a017fa640 1 -- 172.21.15.116:0/4189871732 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f29f40ce9e0 con 0x7f2a04100a80 2024-09-12T03:15:29.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.068+0000 7f2a08892640 1 --2- 172.21.15.116:0/2414880457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562deba1c060 0x562deba1c440 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f29fc004ab0 tx=0x7f29fc031b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bad5c4d166a14b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:29.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.068+0000 7f29ea7fc640 1 -- 172.21.15.116:0/2414880457 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc044040 con 0x562deba1c060 2024-09-12T03:15:29.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.068+0000 7f29ea7fc640 1 -- 172.21.15.116:0/2414880457 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29fc04f070 con 0x562deba1c060 2024-09-12T03:15:29.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.068+0000 7f29ea7fc640 1 -- 172.21.15.116:0/2414880457 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc030b00 con 0x562deba1c060 2024-09-12T03:15:29.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.068+0000 7f2a08892640 1 --2- 172.21.15.116:0/4189871732 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f29ec0a2b80 0x7f29ec0a5040 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f29fc04e000 tx=0x7f29fc04c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:29.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.068+0000 7f2a0c6ce740 1 -- 172.21.15.116:0/2414880457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562deba1c060 msgr2=0x562deba1c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:29.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.068+0000 7f2a0c6ce740 1 --2- 172.21.15.116:0/2414880457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562deba1c060 0x562deba1c440 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f29fc004ab0 tx=0x7f29fc031b00 comp rx=0 tx=0).stop 2024-09-12T03:15:29.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.069+0000 7f2a0c6ce740 1 -- 172.21.15.116:0/2414880457 shutdown_connections 2024-09-12T03:15:29.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.069+0000 7f2a0c6ce740 1 --2- 172.21.15.116:0/2414880457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562deba27240 0x562deba29630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.069+0000 7f2a0c6ce740 1 --2- 172.21.15.116:0/2414880457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562deba1c980 0x562deba26d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.069+0000 7f2a0c6ce740 1 --2- 172.21.15.116:0/2414880457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562deba1c060 0x562deba1c440 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.069+0000 7f2a0c6ce740 1 -- 172.21.15.116:0/2414880457 >> 172.21.15.116:0/2414880457 conn(0x562deba185f0 msgr2=0x562deba189d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:29.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.069+0000 7f2a0c6ce740 1 -- 172.21.15.116:0/2414880457 shutdown_connections 2024-09-12T03:15:29.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.069+0000 7f2a0c6ce740 1 -- 172.21.15.116:0/2414880457 wait complete. 2024-09-12T03:15:29.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.069+0000 7f2a0c6ce740 1 -- start start 2024-09-12T03:15:29.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.069+0000 7f2a0c6ce740 1 Processor -- start 2024-09-12T03:15:29.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.069+0000 7f2a0c6ce740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562deba1c060 0x562deba24150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:29.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.069+0000 7f2a0c6ce740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562deba1c980 0x562deba24690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:29.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.069+0000 7f2a0c6ce740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562deba27240 0x562deba24bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:29.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.069+0000 7f2a0c6ce740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562deba29e10 con 0x562deba27240 2024-09-12T03:15:29.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.069+0000 7f2a0c6ce740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562deb909f10 con 0x562deba1c980 2024-09-12T03:15:29.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.069+0000 7f2a0c6ce740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562deba29f90 con 0x562deba1c060 2024-09-12T03:15:29.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.069+0000 7f2a0c6ce740 10 client.0 _ll_register_callbacks cb 0x562deb92ddb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:29.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.070+0000 7f2a08892640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562deba27240 0x562deba24bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:29.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.070+0000 7f2a08892640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562deba27240 0x562deba24bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56016/0 (socket says 172.21.15.116:56016) 2024-09-12T03:15:29.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.070+0000 7f2a08892640 1 -- 172.21.15.116:0/3140298717 learned_addr learned my addr 172.21.15.116:0/3140298717 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:29.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.070+0000 7f2a037fe640 1 --2- 172.21.15.116:0/3140298717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562deba1c980 0x562deba24690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:29.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.070+0000 7f2a03fff640 1 --2- 172.21.15.116:0/3140298717 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562deba1c060 0x562deba24150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:29.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.070+0000 7f2a08892640 1 -- 172.21.15.116:0/3140298717 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562deba1c060 msgr2=0x562deba24150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:29.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.070+0000 7f2a08892640 1 --2- 172.21.15.116:0/3140298717 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562deba1c060 0x562deba24150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.070+0000 7f2a08892640 1 -- 172.21.15.116:0/3140298717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562deba1c980 msgr2=0x562deba24690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:29.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.070+0000 7f2a08892640 1 --2- 172.21.15.116:0/3140298717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562deba1c980 0x562deba24690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.070+0000 7f2a08892640 1 -- 172.21.15.116:0/3140298717 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562deb690580 con 0x562deba27240 2024-09-12T03:15:29.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.070+0000 7f2a08892640 1 --2- 172.21.15.116:0/3140298717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562deba27240 0x562deba24bd0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009430 tx=0x7f29fc0305a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6065f061877b0bcf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:29.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.070+0000 7f29e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:15:29.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.070+0000 7f29e37fe640 1 -- 172.21.15.116:0/3140298717 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc044070 con 0x562deba27240 2024-09-12T03:15:29.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.070+0000 7f29e37fe640 1 -- 172.21.15.116:0/3140298717 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f29fc004b40 con 0x562deba27240 2024-09-12T03:15:29.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.070+0000 7f2a0c6ce740 1 -- 172.21.15.116:0/3140298717 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29fc04f070 con 0x562deba27240 2024-09-12T03:15:29.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.070+0000 7f29e37fe640 1 -- 172.21.15.116:0/3140298717 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f29fc030ae0 con 0x562deba27240 2024-09-12T03:15:29.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.070+0000 7f2a0c6ce740 10 client.18762 fetch_fsmap learned FSMap version 25 2024-09-12T03:15:29.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.070+0000 7f2a0c6ce740 1 -- 172.21.15.116:0/3140298717 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562deb75a960 con 0x562deba27240 2024-09-12T03:15:29.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.070+0000 7f29e37fe640 1 -- 172.21.15.116:0/3140298717 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29fc04f070 con 0x562deba27240 2024-09-12T03:15:29.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.071+0000 7f29e37fe640 1 -- 172.21.15.116:0/3140298717 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f29fc047030 con 0x562deba27240 2024-09-12T03:15:29.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.071+0000 7f2a0c6ce740 10 client.18762 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:15:29.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.071+0000 7f2a0c6ce740 10 client.18762 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:15:29.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.071+0000 7f2a0c6ce740 20 client.18762 populate_metadata read hostname 'smithi116' 2024-09-12T03:15:29.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.071+0000 7f2a0c6ce740 1 --2- 172.21.15.116:0/3140298717 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562deba36ca0 0x562deba39090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:29.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.071+0000 7f2a0c6ce740 4 client.18762 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:15:29.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.071+0000 7f2a0c6ce740 1 -- 172.21.15.116:0/3140298717 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562deb98eb90 con 0x562deba36ca0 2024-09-12T03:15:29.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.071+0000 7f2a03fff640 1 --2- 172.21.15.116:0/3140298717 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562deba36ca0 0x562deba39090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:29.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.072+0000 7f29e37fe640 10 client.18762 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:29.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.072+0000 7f2a03fff640 1 --2- 172.21.15.116:0/3140298717 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562deba36ca0 0x562deba39090 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2ed0b85a56dfd86 server_cookie=1312b1c79460cf62 in_seq=0 out_seq=0 2024-09-12T03:15:29.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.072+0000 7f29e37fe640 1 -- 172.21.15.116:0/3140298717 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562deb98eb90 con 0x562deba36ca0 2024-09-12T03:15:29.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.072+0000 7f29e37fe640 10 client.18762 handle_command_reply: tid=0 2024-09-12T03:15:29.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.072+0000 7f2a0c6ce740 1 client.18762 shutdown 2024-09-12T03:15:29.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.072+0000 7f2a0c6ce740 1 -- 172.21.15.116:0/3140298717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562deba27240 msgr2=0x562deba24bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:29.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.072+0000 7f2a0c6ce740 1 --2- 172.21.15.116:0/3140298717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562deba27240 0x562deba24bd0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009430 tx=0x7f29fc0305a0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.073+0000 7f2a0c6ce740 1 -- 172.21.15.116:0/3140298717 shutdown_connections 2024-09-12T03:15:29.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.073+0000 7f2a0c6ce740 1 --2- 172.21.15.116:0/3140298717 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562deba36ca0 0x562deba39090 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.073+0000 7f2a0c6ce740 1 --2- 172.21.15.116:0/3140298717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562deba27240 0x562deba24bd0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.073+0000 7f2a0c6ce740 1 --2- 172.21.15.116:0/3140298717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562deba1c980 0x562deba24690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.073+0000 7f2a0c6ce740 1 --2- 172.21.15.116:0/3140298717 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562deba1c060 0x562deba24150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.073+0000 7f29e37fe640 0 client.18762 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:29.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.073+0000 7f2a0c6ce740 1 -- 172.21.15.116:0/3140298717 >> 172.21.15.116:0/3140298717 conn(0x562deba185f0 msgr2=0x562deba1e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:29.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.073+0000 7f2a0c6ce740 1 -- 172.21.15.116:0/3140298717 shutdown_connections 2024-09-12T03:15:29.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.073+0000 7f2a0c6ce740 1 -- 172.21.15.116:0/3140298717 wait complete. 2024-09-12T03:15:29.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.073+0000 7f2a0c6ce740 20 client.18762 trim_cache size 0 max 16384 2024-09-12T03:15:29.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.090+0000 7f2a0c6ce740 1 Processor -- start 2024-09-12T03:15:29.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.090+0000 7f2a0c6ce740 1 -- start start 2024-09-12T03:15:29.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.090+0000 7f2a0c6ce740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562deba1c060 0x562deba1eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:29.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.090+0000 7f2a0c6ce740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562deba1c980 0x562deba1f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:29.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.090+0000 7f2a0c6ce740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562deba27240 0x562deba17640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:29.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.090+0000 7f2a0c6ce740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2a041afd60 con 0x562deba27240 2024-09-12T03:15:29.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.090+0000 7f2a0c6ce740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2a041ad630 con 0x562deba1c060 2024-09-12T03:15:29.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.090+0000 7f2a0c6ce740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a041ab480 con 0x562deba1c980 2024-09-12T03:15:29.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.090+0000 7f2a08892640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562deba27240 0x562deba17640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:29.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.090+0000 7f2a08892640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562deba27240 0x562deba17640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56028/0 (socket says 172.21.15.116:56028) 2024-09-12T03:15:29.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.090+0000 7f2a08892640 1 -- 172.21.15.116:0/1150570039 learned_addr learned my addr 172.21.15.116:0/1150570039 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:29.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.090+0000 7f2a03fff640 1 --2- 172.21.15.116:0/1150570039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562deba1c060 0x562deba1eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:29.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.090+0000 7f2a037fe640 1 --2- 172.21.15.116:0/1150570039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562deba1c980 0x562deba1f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:29.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.090+0000 7f2a08892640 1 -- 172.21.15.116:0/1150570039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562deba1c980 msgr2=0x562deba1f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:29.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.090+0000 7f2a08892640 1 --2- 172.21.15.116:0/1150570039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562deba1c980 0x562deba1f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.090+0000 7f2a08892640 1 -- 172.21.15.116:0/1150570039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562deba1c060 msgr2=0x562deba1eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:29.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.090+0000 7f2a08892640 1 --2- 172.21.15.116:0/1150570039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562deba1c060 0x562deba1eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.090+0000 7f2a08892640 1 -- 172.21.15.116:0/1150570039 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562deba26090 con 0x562deba27240 2024-09-12T03:15:29.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.090+0000 7f2a08892640 1 --2- 172.21.15.116:0/1150570039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562deba27240 0x562deba17640 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f29fc030a90 tx=0x7f29fc02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45cc6c3f487cd567 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:29.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.091+0000 7f29e9ffb640 1 -- 172.21.15.116:0/1150570039 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc044040 con 0x562deba27240 2024-09-12T03:15:29.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.091+0000 7f29e9ffb640 1 -- 172.21.15.116:0/1150570039 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29fc04f030 con 0x562deba27240 2024-09-12T03:15:29.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.091+0000 7f29e9ffb640 1 -- 172.21.15.116:0/1150570039 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc0058e0 con 0x562deba27240 2024-09-12T03:15:29.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.091+0000 7f2a0c6ce740 1 -- 172.21.15.116:0/1150570039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562deba27240 msgr2=0x562deba17640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:29.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.091+0000 7f2a0c6ce740 1 --2- 172.21.15.116:0/1150570039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562deba27240 0x562deba17640 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f29fc030a90 tx=0x7f29fc02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.091+0000 7f2a0c6ce740 1 -- 172.21.15.116:0/1150570039 shutdown_connections 2024-09-12T03:15:29.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.091+0000 7f2a0c6ce740 1 --2- 172.21.15.116:0/1150570039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562deba27240 0x562deba17640 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.091+0000 7f2a0c6ce740 1 --2- 172.21.15.116:0/1150570039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562deba1c980 0x562deba1f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.091+0000 7f2a0c6ce740 1 --2- 172.21.15.116:0/1150570039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562deba1c060 0x562deba1eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.091+0000 7f2a0c6ce740 1 -- 172.21.15.116:0/1150570039 >> 172.21.15.116:0/1150570039 conn(0x562deba185f0 msgr2=0x562deba16f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:29.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.091+0000 7f2a0c6ce740 1 -- 172.21.15.116:0/1150570039 shutdown_connections 2024-09-12T03:15:29.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.091+0000 7f2a0c6ce740 1 -- 172.21.15.116:0/1150570039 wait complete. 2024-09-12T03:15:29.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.092+0000 7f2a0c6ce740 1 -- start start 2024-09-12T03:15:29.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.092+0000 7f2a0c6ce740 1 Processor -- start 2024-09-12T03:15:29.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.092+0000 7f2a0c6ce740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562deba1c060 0x562deba2b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:29.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.092+0000 7f2a0c6ce740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562deba1c980 0x562deba2b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:29.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.092+0000 7f2a0c6ce740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562deba27240 0x562debaa0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:29.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.092+0000 7f2a0c6ce740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2a041abaf0 con 0x562deba1c980 2024-09-12T03:15:29.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.092+0000 7f2a0c6ce740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2a041acf60 con 0x562deba27240 2024-09-12T03:15:29.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.092+0000 7f2a0c6ce740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a041ac1c0 con 0x562deba1c060 2024-09-12T03:15:29.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.092+0000 7f2a0c6ce740 10 client.0 _ll_register_callbacks cb 0x7f29fc031950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:29.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.092+0000 7f2a037fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562deba1c980 0x562deba2b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:29.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.092+0000 7f2a037fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562deba1c980 0x562deba2b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:56038/0 (socket says 172.21.15.116:56038) 2024-09-12T03:15:29.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.092+0000 7f2a037fe640 1 -- 172.21.15.116:0/3206010584 learned_addr learned my addr 172.21.15.116:0/3206010584 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:29.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.092+0000 7f2a08892640 1 --2- 172.21.15.116:0/3206010584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562deba27240 0x562debaa0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:29.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.092+0000 7f2a03fff640 1 --2- 172.21.15.116:0/3206010584 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562deba1c060 0x562deba2b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:29.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.093+0000 7f2a037fe640 1 -- 172.21.15.116:0/3206010584 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562deba1c060 msgr2=0x562deba2b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:29.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.093+0000 7f2a037fe640 1 --2- 172.21.15.116:0/3206010584 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562deba1c060 0x562deba2b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.093+0000 7f2a037fe640 1 -- 172.21.15.116:0/3206010584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562deba27240 msgr2=0x562debaa0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:29.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.093+0000 7f2a037fe640 1 --2- 172.21.15.116:0/3206010584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562deba27240 0x562debaa0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.093+0000 7f2a037fe640 1 -- 172.21.15.116:0/3206010584 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562debaa0b40 con 0x562deba1c980 2024-09-12T03:15:29.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.093+0000 7f2a037fe640 1 --2- 172.21.15.116:0/3206010584 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562deba1c980 0x562deba2b8a0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x562deba19970 tx=0x7f29f8003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=548fb5667b5f4077 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:29.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.093+0000 7f29e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:15:29.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.093+0000 7f29e37fe640 1 -- 172.21.15.116:0/3206010584 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f800bc10 con 0x562deba1c980 2024-09-12T03:15:29.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.093+0000 7f29e37fe640 1 -- 172.21.15.116:0/3206010584 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f29f800bdb0 con 0x562deba1c980 2024-09-12T03:15:29.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.093+0000 7f2a0c6ce740 1 -- 172.21.15.116:0/3206010584 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29fc04f030 con 0x562deba1c980 2024-09-12T03:15:29.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.093+0000 7f29e37fe640 1 -- 172.21.15.116:0/3206010584 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f29f8011440 con 0x562deba1c980 2024-09-12T03:15:29.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.093+0000 7f2a0c6ce740 10 client.18768 fetch_fsmap learned FSMap version 25 2024-09-12T03:15:29.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.093+0000 7f2a0c6ce740 1 -- 172.21.15.116:0/3206010584 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562debaa1d80 con 0x562deba1c980 2024-09-12T03:15:29.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.093+0000 7f29e37fe640 1 -- 172.21.15.116:0/3206010584 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29fc04f030 con 0x562deba1c980 2024-09-12T03:15:29.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.093+0000 7f29e37fe640 1 -- 172.21.15.116:0/3206010584 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f29f80277f0 con 0x562deba1c980 2024-09-12T03:15:29.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.093+0000 7f2a0c6ce740 10 client.18768 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:15:29.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.093+0000 7f2a0c6ce740 10 client.18768 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:15:29.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.093+0000 7f2a0c6ce740 20 client.18768 populate_metadata read hostname 'smithi116' 2024-09-12T03:15:29.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.093+0000 7f2a0c6ce740 1 --2- 172.21.15.116:0/3206010584 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562debaa2550 0x562debaa4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:29.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.094+0000 7f2a0c6ce740 4 client.18768 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "935d5531", "release": true, "await": true}] 2024-09-12T03:15:29.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.094+0000 7f2a0c6ce740 1 -- 172.21.15.116:0/3206010584 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "quiesce db", "set_id": "935d5531", "release": true, "await": true}) -- 0x562deb9890a0 con 0x562debaa2550 2024-09-12T03:15:29.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.094+0000 7f2a03fff640 1 --2- 172.21.15.116:0/3206010584 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562debaa2550 0x562debaa4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:29.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.094+0000 7f2a03fff640 1 --2- 172.21.15.116:0/3206010584 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562debaa2550 0x562debaa4970 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7af2400000c8465b server_cookie=5eac76569e6a21dc in_seq=0 out_seq=0 2024-09-12T03:15:29.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.094+0000 7f29e37fe640 10 client.18768 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:29.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.097+0000 7f29e37fe640 1 -- 172.21.15.116:0/3206010584 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562deb9890a0 con 0x562debaa2550 2024-09-12T03:15:29.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.097+0000 7f29e37fe640 10 client.18768 handle_command_reply: tid=0 2024-09-12T03:15:29.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.097+0000 7f2a0c6ce740 1 client.18768 shutdown 2024-09-12T03:15:29.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.097+0000 7f2a0c6ce740 1 -- 172.21.15.116:0/3206010584 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562deba1c980 msgr2=0x562deba2b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:29.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.097+0000 7f2a0c6ce740 1 --2- 172.21.15.116:0/3206010584 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562deba1c980 0x562deba2b8a0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x562deba19970 tx=0x7f29f8003d00 comp rx=0 tx=0).stop 2024-09-12T03:15:29.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.097+0000 7f2a0c6ce740 1 -- 172.21.15.116:0/3206010584 shutdown_connections 2024-09-12T03:15:29.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.097+0000 7f2a0c6ce740 1 --2- 172.21.15.116:0/3206010584 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562debaa2550 0x562debaa4970 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.097+0000 7f2a0c6ce740 1 --2- 172.21.15.116:0/3206010584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562deba27240 0x562debaa0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.097+0000 7f2a0c6ce740 1 --2- 172.21.15.116:0/3206010584 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562deba1c980 0x562deba2b8a0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.097+0000 7f2a0c6ce740 1 --2- 172.21.15.116:0/3206010584 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562deba1c060 0x562deba2b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.097+0000 7f29e37fe640 0 client.18768 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:29.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.097+0000 7f2a0c6ce740 1 -- 172.21.15.116:0/3206010584 >> 172.21.15.116:0/3206010584 conn(0x562deba185f0 msgr2=0x562deba24910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:29.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.097+0000 7f2a0c6ce740 1 -- 172.21.15.116:0/3206010584 shutdown_connections 2024-09-12T03:15:29.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.097+0000 7f2a0c6ce740 1 -- 172.21.15.116:0/3206010584 wait complete. 2024-09-12T03:15:29.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.097+0000 7f2a0c6ce740 20 client.18768 trim_cache size 0 max 16384 2024-09-12T03:15:29.106 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:15:29.106 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 25, 2024-09-12T03:15:29.106 INFO:teuthology.orchestra.run.smithi116.stdout: "leader": 24494, 2024-09-12T03:15:29.106 INFO:teuthology.orchestra.run.smithi116.stdout: "set_version": 48, 2024-09-12T03:15:29.106 INFO:teuthology.orchestra.run.smithi116.stdout: "sets": { 2024-09-12T03:15:29.106 INFO:teuthology.orchestra.run.smithi116.stdout: "935d5531": { 2024-09-12T03:15:29.106 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 48, 2024-09-12T03:15:29.106 INFO:teuthology.orchestra.run.smithi116.stdout: "age_ref": 0.0, 2024-09-12T03:15:29.106 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:15:29.106 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RELEASED", 2024-09-12T03:15:29.106 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:15:29.107 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:15:29.107 INFO:teuthology.orchestra.run.smithi116.stdout: "timeout": 90.0, 2024-09-12T03:15:29.107 INFO:teuthology.orchestra.run.smithi116.stdout: "expiration": 147.2, 2024-09-12T03:15:29.107 INFO:teuthology.orchestra.run.smithi116.stdout: "members": { 2024-09-12T03:15:29.107 INFO:teuthology.orchestra.run.smithi116.stdout: "file:/": { 2024-09-12T03:15:29.107 INFO:teuthology.orchestra.run.smithi116.stdout: "excluded": false, 2024-09-12T03:15:29.107 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:15:29.107 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RELEASED", 2024-09-12T03:15:29.107 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:15:29.107 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:15:29.107 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:15:29.107 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:15:29.107 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:15:29.107 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:15:29.108 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:15:29.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.097+0000 7f2a0a2f5640 1 -- 172.21.15.116:0/4189871732 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f29ec0a2b80 msgr2=0x7f29ec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:29.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.098+0000 7f2a0a2f5640 1 --2- 172.21.15.116:0/4189871732 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f29ec0a2b80 0x7f29ec0a5040 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f29fc04e000 tx=0x7f29fc04c000 comp rx=0 tx=0).stop 2024-09-12T03:15:29.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.098+0000 7f2a0a2f5640 1 -- 172.21.15.116:0/4189871732 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a04100a80 msgr2=0x7f2a041992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:29.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.098+0000 7f2a0a2f5640 1 --2- 172.21.15.116:0/4189871732 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a04100a80 0x7f2a041992a0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f29f400d770 tx=0x7f29f400dc30 comp rx=0 tx=0).stop 2024-09-12T03:15:29.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.098+0000 7f2a0a2f5640 1 -- 172.21.15.116:0/4189871732 shutdown_connections 2024-09-12T03:15:29.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.098+0000 7f2a0a2f5640 1 --2- 172.21.15.116:0/4189871732 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f29ec0a2b80 0x7f29ec0a5040 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.098+0000 7f2a0a2f5640 1 --2- 172.21.15.116:0/4189871732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a04106a80 0x7f2a041a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.098+0000 7f2a0a2f5640 1 --2- 172.21.15.116:0/4189871732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a04101420 0x7f2a041997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.098+0000 7f2a0a2f5640 1 --2- 172.21.15.116:0/4189871732 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a04100a80 0x7f2a041992a0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:29.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.098+0000 7f2a0a2f5640 1 -- 172.21.15.116:0/4189871732 >> 172.21.15.116:0/4189871732 conn(0x7f2a040fc7a0 msgr2=0x7f2a040fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:29.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.098+0000 7f2a0a2f5640 1 -- 172.21.15.116:0/4189871732 shutdown_connections 2024-09-12T03:15:29.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:29.098+0000 7f2a0a2f5640 1 -- 172.21.15.116:0/4189871732 wait complete. 2024-09-12T03:15:29.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:29 smithi183 ceph-mon[35535]: pgmap v713: 97 pgs: 97 active+clean; 6.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-12T03:15:29.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:29 smithi177 ceph-mon[35305]: pgmap v713: 97 pgs: 97 active+clean; 6.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-12T03:15:29.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:29 smithi116 ceph-mon[31410]: pgmap v713: 97 pgs: 97 active+clean; 6.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-12T03:15:29.823 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '935d5531', total seconds elapsed: 30.5 2024-09-12T03:15:31.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:31 smithi183 ceph-mon[35535]: pgmap v714: 97 pgs: 97 active+clean; 6.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-12T03:15:31.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:31 smithi177 ceph-mon[35305]: pgmap v714: 97 pgs: 97 active+clean; 6.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-12T03:15:31.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:31 smithi116 ceph-mon[31410]: pgmap v714: 97 pgs: 97 active+clean; 6.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-12T03:15:33.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:33 smithi183 ceph-mon[35535]: pgmap v715: 97 pgs: 97 active+clean; 6.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-09-12T03:15:33.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:33 smithi177 ceph-mon[35305]: pgmap v715: 97 pgs: 97 active+clean; 6.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-09-12T03:15:33.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:33 smithi116 ceph-mon[31410]: pgmap v715: 97 pgs: 97 active+clean; 6.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-09-12T03:15:35.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:35 smithi183 ceph-mon[35535]: pgmap v716: 97 pgs: 97 active+clean; 6.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-09-12T03:15:35.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:35 smithi177 ceph-mon[35305]: pgmap v716: 97 pgs: 97 active+clean; 6.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-09-12T03:15:35.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:35 smithi116 ceph-mon[31410]: pgmap v716: 97 pgs: 97 active+clean; 6.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-09-12T03:15:37.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:37 smithi183 ceph-mon[35535]: pgmap v717: 97 pgs: 97 active+clean; 6.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.9 KiB/s wr, 8 op/s 2024-09-12T03:15:37.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:37 smithi177 ceph-mon[35305]: pgmap v717: 97 pgs: 97 active+clean; 6.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.9 KiB/s wr, 8 op/s 2024-09-12T03:15:37.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:37 smithi116 ceph-mon[31410]: pgmap v717: 97 pgs: 97 active+clean; 6.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.9 KiB/s wr, 8 op/s 2024-09-12T03:15:39.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:39 smithi183 ceph-mon[35535]: pgmap v718: 97 pgs: 97 active+clean; 6.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:15:39.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:39 smithi177 ceph-mon[35305]: pgmap v718: 97 pgs: 97 active+clean; 6.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:15:39.804 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:39 smithi116 ceph-mon[31410]: pgmap v718: 97 pgs: 97 active+clean; 6.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:15:41.170 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:15:41.433 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:15:41.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:41 smithi116 ceph-mon[31410]: pgmap v719: 97 pgs: 97 active+clean; 6.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T03:15:41.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:41 smithi183 ceph-mon[35535]: pgmap v719: 97 pgs: 97 active+clean; 6.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T03:15:41.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:41 smithi177 ceph-mon[35305]: pgmap v719: 97 pgs: 97 active+clean; 6.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T03:15:41.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.901+0000 7f4838bbd640 1 -- 172.21.15.116:0/2550173798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4834102ef0 msgr2=0x7f483410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:41.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.901+0000 7f4838bbd640 1 --2- 172.21.15.116:0/2550173798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4834102ef0 0x7f483410da40 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f482c009be0 tx=0x7f482c02a750 comp rx=0 tx=0).stop 2024-09-12T03:15:41.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.901+0000 7f4838bbd640 1 -- 172.21.15.116:0/2550173798 shutdown_connections 2024-09-12T03:15:41.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.901+0000 7f4838bbd640 1 --2- 172.21.15.116:0/2550173798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4834102ef0 0x7f483410da40 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.901+0000 7f4838bbd640 1 --2- 172.21.15.116:0/2550173798 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4834102570 0x7f48341029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.901+0000 7f4838bbd640 1 --2- 172.21.15.116:0/2550173798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4834100e20 0x7f4834101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.901+0000 7f4838bbd640 1 -- 172.21.15.116:0/2550173798 >> 172.21.15.116:0/2550173798 conn(0x7f48340fc7d0 msgr2=0x7f48340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:41.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.901+0000 7f4838bbd640 1 -- 172.21.15.116:0/2550173798 shutdown_connections 2024-09-12T03:15:41.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.901+0000 7f4838bbd640 1 -- 172.21.15.116:0/2550173798 wait complete. 2024-09-12T03:15:41.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.902+0000 7f4838bbd640 1 Processor -- start 2024-09-12T03:15:41.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.902+0000 7f4838bbd640 1 -- start start 2024-09-12T03:15:41.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.902+0000 7f4838bbd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4834100e20 0x7f48341992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:41.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.902+0000 7f4838bbd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4834102570 0x7f4834199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:41.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.902+0000 7f4838bbd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4834102ef0 0x7f48341a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:41.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.902+0000 7f4838bbd640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4834110f30 con 0x7f4834100e20 2024-09-12T03:15:41.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.902+0000 7f4838bbd640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4834110db0 con 0x7f4834102ef0 2024-09-12T03:15:41.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.902+0000 7f4838bbd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f48341110b0 con 0x7f4834102570 2024-09-12T03:15:41.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.902+0000 7f483259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4834100e20 0x7f48341992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:41.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.903+0000 7f483259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4834100e20 0x7f48341992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52608/0 (socket says 172.21.15.116:52608) 2024-09-12T03:15:41.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.903+0000 7f483259c640 1 -- 172.21.15.116:0/1674845986 learned_addr learned my addr 172.21.15.116:0/1674845986 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:41.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.903+0000 7f482bfff640 1 --2- 172.21.15.116:0/1674845986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4834102570 0x7f4834199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:41.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.903+0000 7f4832d9d640 1 --2- 172.21.15.116:0/1674845986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4834102ef0 0x7f48341a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:41.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.903+0000 7f483259c640 1 -- 172.21.15.116:0/1674845986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4834102570 msgr2=0x7f4834199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:41.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.903+0000 7f483259c640 1 --2- 172.21.15.116:0/1674845986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4834102570 0x7f4834199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.903+0000 7f483259c640 1 -- 172.21.15.116:0/1674845986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4834102ef0 msgr2=0x7f48341a4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:41.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.903+0000 7f483259c640 1 --2- 172.21.15.116:0/1674845986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4834102ef0 0x7f48341a4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.903+0000 7f483259c640 1 -- 172.21.15.116:0/1674845986 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48341a48a0 con 0x7f4834100e20 2024-09-12T03:15:41.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.903+0000 7f483259c640 1 --2- 172.21.15.116:0/1674845986 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4834100e20 0x7f48341992e0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f482400e9f0 tx=0x7f482400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:41.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.903+0000 7f482b7fe640 1 -- 172.21.15.116:0/1674845986 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482400ccc0 con 0x7f4834100e20 2024-09-12T03:15:41.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.903+0000 7f4838bbd640 1 -- 172.21.15.116:0/1674845986 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48341a4a70 con 0x7f4834100e20 2024-09-12T03:15:41.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.903+0000 7f482b7fe640 1 -- 172.21.15.116:0/1674845986 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4824005500 con 0x7f4834100e20 2024-09-12T03:15:41.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.903+0000 7f482b7fe640 1 -- 172.21.15.116:0/1674845986 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824010690 con 0x7f4834100e20 2024-09-12T03:15:41.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.903+0000 7f4838bbd640 1 -- 172.21.15.116:0/1674845986 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48341a4f80 con 0x7f4834100e20 2024-09-12T03:15:41.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.905+0000 7f482b7fe640 1 -- 172.21.15.116:0/1674845986 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f48240046c0 con 0x7f4834100e20 2024-09-12T03:15:41.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.906+0000 7f483af96740 1 Processor -- start 2024-09-12T03:15:41.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.906+0000 7f483af96740 1 -- start start 2024-09-12T03:15:41.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.906+0000 7f482b7fe640 1 --2- 172.21.15.116:0/1674845986 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f48200a2b80 0x7f48200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:41.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.906+0000 7f483af96740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a319be4a0 0x560a319be880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:41.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.906+0000 7f483af96740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a319bedc0 0x560a319c9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:41.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.906+0000 7f483af96740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a319c9680 0x560a319cba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:41.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.906+0000 7f483af96740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560a319242e0 con 0x560a319c9680 2024-09-12T03:15:41.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.906+0000 7f483af96740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560a318cbce0 con 0x560a319be4a0 2024-09-12T03:15:41.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.906+0000 7f483af96740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560a319202a0 con 0x560a319bedc0 2024-09-12T03:15:41.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.906+0000 7f482b7fe640 1 -- 172.21.15.116:0/1674845986 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f48240cf420 con 0x7f4834100e20 2024-09-12T03:15:41.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.906+0000 7f4832d9d640 1 --2- 172.21.15.116:0/1674845986 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f48200a2b80 0x7f48200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:41.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.907+0000 7f482bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a319bedc0 0x560a319c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:41.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.907+0000 7f4832d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a319c9680 0x560a319cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:41.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.907+0000 7f4832d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a319c9680 0x560a319cba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52624/0 (socket says 172.21.15.116:52624) 2024-09-12T03:15:41.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.907+0000 7f4832d9d640 1 -- 172.21.15.116:0/3352940209 learned_addr learned my addr 172.21.15.116:0/3352940209 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:41.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.907+0000 7f483259c640 1 --2- 172.21.15.116:0/3352940209 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a319be4a0 0x560a319be880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:41.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.907+0000 7f4832d9d640 1 -- 172.21.15.116:0/3352940209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a319bedc0 msgr2=0x560a319c9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:41.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.907+0000 7f4832d9d640 1 --2- 172.21.15.116:0/3352940209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a319bedc0 0x560a319c9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.907+0000 7f4832d9d640 1 -- 172.21.15.116:0/3352940209 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a319be4a0 msgr2=0x560a319be880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:41.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.907+0000 7f4832d9d640 1 --2- 172.21.15.116:0/3352940209 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a319be4a0 0x560a319be880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.907+0000 7f4832d9d640 1 -- 172.21.15.116:0/3352940209 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a318b0440 con 0x560a319c9680 2024-09-12T03:15:41.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.907+0000 7f482bfff640 1 --2- 172.21.15.116:0/3352940209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a319bedc0 0x560a319c9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:15:41.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.907+0000 7f4832d9d640 1 --2- 172.21.15.116:0/1674845986 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f48200a2b80 0x7f48200a5040 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f482c009be0 tx=0x7f482c0303a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:41.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.907+0000 7f4832d9d640 1 --2- 172.21.15.116:0/3352940209 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a319c9680 0x560a319cba70 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f482c030e40 tx=0x7f482c046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de4283de48cc2d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:41.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.907+0000 7f4828ff9640 1 -- 172.21.15.116:0/3352940209 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c036070 con 0x560a319c9680 2024-09-12T03:15:41.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.907+0000 7f4828ff9640 1 -- 172.21.15.116:0/3352940209 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f482c037040 con 0x560a319c9680 2024-09-12T03:15:41.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.907+0000 7f4828ff9640 1 -- 172.21.15.116:0/3352940209 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c004780 con 0x560a319c9680 2024-09-12T03:15:41.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.908+0000 7f483af96740 1 -- 172.21.15.116:0/3352940209 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a319c9680 msgr2=0x560a319cba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:41.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.908+0000 7f483af96740 1 --2- 172.21.15.116:0/3352940209 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a319c9680 0x560a319cba70 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f482c030e40 tx=0x7f482c046000 comp rx=0 tx=0).stop 2024-09-12T03:15:41.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.908+0000 7f483af96740 1 -- 172.21.15.116:0/3352940209 shutdown_connections 2024-09-12T03:15:41.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.908+0000 7f483af96740 1 --2- 172.21.15.116:0/3352940209 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a319c9680 0x560a319cba70 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.908+0000 7f483af96740 1 --2- 172.21.15.116:0/3352940209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a319bedc0 0x560a319c9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.908+0000 7f483af96740 1 --2- 172.21.15.116:0/3352940209 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a319be4a0 0x560a319be880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.908+0000 7f483af96740 1 -- 172.21.15.116:0/3352940209 >> 172.21.15.116:0/3352940209 conn(0x560a319baa30 msgr2=0x560a319bae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:41.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.908+0000 7f483af96740 1 -- 172.21.15.116:0/3352940209 shutdown_connections 2024-09-12T03:15:41.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.908+0000 7f483af96740 1 -- 172.21.15.116:0/3352940209 wait complete. 2024-09-12T03:15:41.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.908+0000 7f483af96740 1 -- start start 2024-09-12T03:15:41.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.909+0000 7f483af96740 1 Processor -- start 2024-09-12T03:15:41.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.909+0000 7f483af96740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a319be4a0 0x560a319c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:41.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.909+0000 7f483af96740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a319bedc0 0x560a319c6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:41.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.909+0000 7f483af96740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a319c9680 0x560a319c6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:41.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.909+0000 7f483af96740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560a319cc250 con 0x560a319be4a0 2024-09-12T03:15:41.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.909+0000 7f483af96740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560a318df050 con 0x560a319bedc0 2024-09-12T03:15:41.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.909+0000 7f483af96740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560a319cc3d0 con 0x560a319c9680 2024-09-12T03:15:41.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.909+0000 7f483af96740 10 client.0 _ll_register_callbacks cb 0x560a3191f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:41.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.909+0000 7f483259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a319be4a0 0x560a319c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:41.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.909+0000 7f483259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a319be4a0 0x560a319c8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52640/0 (socket says 172.21.15.116:52640) 2024-09-12T03:15:41.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.909+0000 7f483259c640 1 -- 172.21.15.116:0/834286919 learned_addr learned my addr 172.21.15.116:0/834286919 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:41.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.909+0000 7f4832d9d640 1 --2- 172.21.15.116:0/834286919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a319c9680 0x560a319c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:41.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.909+0000 7f482bfff640 1 --2- 172.21.15.116:0/834286919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a319bedc0 0x560a319c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:41.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.909+0000 7f483259c640 1 -- 172.21.15.116:0/834286919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a319c9680 msgr2=0x560a319c6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:41.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.909+0000 7f483259c640 1 --2- 172.21.15.116:0/834286919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a319c9680 0x560a319c6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.909+0000 7f483259c640 1 -- 172.21.15.116:0/834286919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a319bedc0 msgr2=0x560a319c6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:41.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.909+0000 7f483259c640 1 --2- 172.21.15.116:0/834286919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a319bedc0 0x560a319c6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.909+0000 7f483259c640 1 -- 172.21.15.116:0/834286919 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a316953c0 con 0x560a319be4a0 2024-09-12T03:15:41.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.909+0000 7f482bfff640 1 --2- 172.21.15.116:0/834286919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a319bedc0 0x560a319c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:15:41.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.909+0000 7f483259c640 1 --2- 172.21.15.116:0/834286919 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a319be4a0 0x560a319c8f60 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f48240ccf50 tx=0x7f482409b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27533ca37617dd66 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:41.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.909+0000 7f4819ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:15:41.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.910+0000 7f4819ffb640 1 -- 172.21.15.116:0/834286919 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48240a0060 con 0x560a319be4a0 2024-09-12T03:15:41.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.910+0000 7f4819ffb640 1 -- 172.21.15.116:0/834286919 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f4824016d90 con 0x560a319be4a0 2024-09-12T03:15:41.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.910+0000 7f483af96740 1 -- 172.21.15.116:0/834286919 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f482c037040 con 0x560a319be4a0 2024-09-12T03:15:41.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.910+0000 7f4819ffb640 1 -- 172.21.15.116:0/834286919 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f48240a4920 con 0x560a319be4a0 2024-09-12T03:15:41.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.910+0000 7f483af96740 10 client.18780 fetch_fsmap learned FSMap version 25 2024-09-12T03:15:41.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.910+0000 7f483af96740 1 -- 172.21.15.116:0/834286919 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560a319d7ca0 con 0x560a319be4a0 2024-09-12T03:15:41.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.910+0000 7f4819ffb640 1 -- 172.21.15.116:0/834286919 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f482c037040 con 0x560a319be4a0 2024-09-12T03:15:41.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.910+0000 7f4819ffb640 1 -- 172.21.15.116:0/834286919 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f48240a4490 con 0x560a319be4a0 2024-09-12T03:15:41.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.910+0000 7f483af96740 10 client.18780 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:15:41.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.910+0000 7f483af96740 10 client.18780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:15:41.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.910+0000 7f483af96740 20 client.18780 populate_metadata read hostname 'smithi116' 2024-09-12T03:15:41.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.910+0000 7f483af96740 1 --2- 172.21.15.116:0/834286919 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560a319d9100 0x560a319db590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:41.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.910+0000 7f483af96740 4 client.18780 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:15:41.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.910+0000 7f483af96740 1 -- 172.21.15.116:0/834286919 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a31931850 con 0x560a319d9100 2024-09-12T03:15:41.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.910+0000 7f482bfff640 1 --2- 172.21.15.116:0/834286919 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560a319d9100 0x560a319db590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:41.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.911+0000 7f482bfff640 1 --2- 172.21.15.116:0/834286919 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560a319d9100 0x560a319db590 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=668b9f0316bc314c server_cookie=99a71cccee0da369 in_seq=0 out_seq=0 2024-09-12T03:15:41.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.911+0000 7f4819ffb640 10 client.18780 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:41.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.911+0000 7f4819ffb640 1 -- 172.21.15.116:0/834286919 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560a31931850 con 0x560a319d9100 2024-09-12T03:15:41.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.911+0000 7f4819ffb640 10 client.18780 handle_command_reply: tid=0 2024-09-12T03:15:41.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.912+0000 7f483af96740 1 client.18780 shutdown 2024-09-12T03:15:41.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.912+0000 7f483af96740 1 -- 172.21.15.116:0/834286919 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a319be4a0 msgr2=0x560a319c8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:41.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.912+0000 7f483af96740 1 --2- 172.21.15.116:0/834286919 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a319be4a0 0x560a319c8f60 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f48240ccf50 tx=0x7f482409b6d0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.912+0000 7f483af96740 1 -- 172.21.15.116:0/834286919 shutdown_connections 2024-09-12T03:15:41.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.912+0000 7f483af96740 1 --2- 172.21.15.116:0/834286919 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560a319d9100 0x560a319db590 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.912+0000 7f483af96740 1 --2- 172.21.15.116:0/834286919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a319c9680 0x560a319c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.912+0000 7f483af96740 1 --2- 172.21.15.116:0/834286919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a319bedc0 0x560a319c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.912+0000 7f4819ffb640 0 client.18780 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:41.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.912+0000 7f483af96740 1 --2- 172.21.15.116:0/834286919 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a319be4a0 0x560a319c8f60 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.912+0000 7f483af96740 1 -- 172.21.15.116:0/834286919 >> 172.21.15.116:0/834286919 conn(0x560a319baa30 msgr2=0x560a319c0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:41.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.912+0000 7f483af96740 1 -- 172.21.15.116:0/834286919 shutdown_connections 2024-09-12T03:15:41.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.912+0000 7f483af96740 1 -- 172.21.15.116:0/834286919 wait complete. 2024-09-12T03:15:41.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.912+0000 7f483af96740 20 client.18780 trim_cache size 0 max 16384 2024-09-12T03:15:41.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.947+0000 7f483af96740 1 Processor -- start 2024-09-12T03:15:41.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.947+0000 7f483af96740 1 -- start start 2024-09-12T03:15:41.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.947+0000 7f483af96740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a319be4a0 0x560a319d5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:41.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.947+0000 7f483af96740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a319bedc0 0x560a319e0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:41.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.947+0000 7f483af96740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a319c9680 0x560a319e1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:41.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.947+0000 7f483af96740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f48341afde0 con 0x560a319c9680 2024-09-12T03:15:41.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.947+0000 7f483af96740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f48341ad6b0 con 0x560a319be4a0 2024-09-12T03:15:41.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.947+0000 7f483af96740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f48341ab500 con 0x560a319bedc0 2024-09-12T03:15:41.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.948+0000 7f4832d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a319c9680 0x560a319e1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:41.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.948+0000 7f4832d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a319c9680 0x560a319e1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52650/0 (socket says 172.21.15.116:52650) 2024-09-12T03:15:41.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.948+0000 7f4832d9d640 1 -- 172.21.15.116:0/1407937704 learned_addr learned my addr 172.21.15.116:0/1407937704 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:41.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.948+0000 7f483259c640 1 --2- 172.21.15.116:0/1407937704 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a319be4a0 0x560a319d5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:41.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.948+0000 7f482bfff640 1 --2- 172.21.15.116:0/1407937704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a319bedc0 0x560a319e0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:41.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.948+0000 7f4832d9d640 1 -- 172.21.15.116:0/1407937704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a319bedc0 msgr2=0x560a319e0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:41.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.948+0000 7f4832d9d640 1 --2- 172.21.15.116:0/1407937704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a319bedc0 0x560a319e0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.948+0000 7f4832d9d640 1 -- 172.21.15.116:0/1407937704 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a319be4a0 msgr2=0x560a319d5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:41.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.948+0000 7f4832d9d640 1 --2- 172.21.15.116:0/1407937704 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a319be4a0 0x560a319d5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.948+0000 7f4832d9d640 1 -- 172.21.15.116:0/1407937704 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a319bebf0 con 0x560a319c9680 2024-09-12T03:15:41.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.948+0000 7f4832d9d640 1 --2- 172.21.15.116:0/1407937704 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a319c9680 0x560a319e1370 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x560a319bbdb0 tx=0x7f482c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea94e1f3feacef4a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:41.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.948+0000 7f481bfff640 1 -- 172.21.15.116:0/1407937704 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c030c30 con 0x560a319c9680 2024-09-12T03:15:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.948+0000 7f481bfff640 1 -- 172.21.15.116:0/1407937704 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f482c004b40 con 0x560a319c9680 2024-09-12T03:15:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.948+0000 7f481bfff640 1 -- 172.21.15.116:0/1407937704 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c0052a0 con 0x560a319c9680 2024-09-12T03:15:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.949+0000 7f483af96740 1 -- 172.21.15.116:0/1407937704 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a319c9680 msgr2=0x560a319e1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.949+0000 7f483af96740 1 --2- 172.21.15.116:0/1407937704 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a319c9680 0x560a319e1370 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x560a319bbdb0 tx=0x7f482c009500 comp rx=0 tx=0).stop 2024-09-12T03:15:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.949+0000 7f483af96740 1 -- 172.21.15.116:0/1407937704 shutdown_connections 2024-09-12T03:15:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.949+0000 7f483af96740 1 --2- 172.21.15.116:0/1407937704 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a319c9680 0x560a319e1370 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.949+0000 7f483af96740 1 --2- 172.21.15.116:0/1407937704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a319bedc0 0x560a319e0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.949+0000 7f483af96740 1 --2- 172.21.15.116:0/1407937704 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a319be4a0 0x560a319d5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.949+0000 7f483af96740 1 -- 172.21.15.116:0/1407937704 >> 172.21.15.116:0/1407937704 conn(0x560a319baa30 msgr2=0x560a319c18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.949+0000 7f483af96740 1 -- 172.21.15.116:0/1407937704 shutdown_connections 2024-09-12T03:15:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.949+0000 7f483af96740 1 -- 172.21.15.116:0/1407937704 wait complete. 2024-09-12T03:15:41.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.950+0000 7f483af96740 1 -- start start 2024-09-12T03:15:41.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.950+0000 7f483af96740 1 Processor -- start 2024-09-12T03:15:41.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.950+0000 7f483af96740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a319be4a0 0x560a319d1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:41.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.950+0000 7f483af96740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a319bedc0 0x560a319cd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:41.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.950+0000 7f483af96740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a319c9680 0x560a319cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:41.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.950+0000 7f483af96740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f48341abb70 con 0x560a319be4a0 2024-09-12T03:15:41.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.950+0000 7f483af96740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f48341acfe0 con 0x560a319c9680 2024-09-12T03:15:41.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.950+0000 7f483af96740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f48341ac240 con 0x560a319bedc0 2024-09-12T03:15:41.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.950+0000 7f483af96740 10 client.0 _ll_register_callbacks cb 0x7f482400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:41.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.950+0000 7f483259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a319be4a0 0x560a319d1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:41.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.950+0000 7f483259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a319be4a0 0x560a319d1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52658/0 (socket says 172.21.15.116:52658) 2024-09-12T03:15:41.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.950+0000 7f483259c640 1 -- 172.21.15.116:0/985485113 learned_addr learned my addr 172.21.15.116:0/985485113 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:41.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.950+0000 7f482bfff640 1 --2- 172.21.15.116:0/985485113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a319bedc0 0x560a319cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:41.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.950+0000 7f4832d9d640 1 --2- 172.21.15.116:0/985485113 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a319c9680 0x560a319cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:41.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.950+0000 7f483259c640 1 -- 172.21.15.116:0/985485113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a319bedc0 msgr2=0x560a319cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:41.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.950+0000 7f483259c640 1 --2- 172.21.15.116:0/985485113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a319bedc0 0x560a319cd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.950+0000 7f483259c640 1 -- 172.21.15.116:0/985485113 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a319c9680 msgr2=0x560a319cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:41.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.950+0000 7f483259c640 1 --2- 172.21.15.116:0/985485113 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a319c9680 0x560a319cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.950+0000 7f483259c640 1 -- 172.21.15.116:0/985485113 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a319cdc30 con 0x560a319be4a0 2024-09-12T03:15:41.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.950+0000 7f482bfff640 1 --2- 172.21.15.116:0/985485113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a319bedc0 0x560a319cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:15:41.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.951+0000 7f483259c640 1 --2- 172.21.15.116:0/985485113 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a319be4a0 0x560a319d1cd0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f4824098680 tx=0x7f482409bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d22c5e19751f3ff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:41.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.951+0000 7f4819ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:15:41.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.951+0000 7f4819ffb640 1 -- 172.21.15.116:0/985485113 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48240b8070 con 0x560a319be4a0 2024-09-12T03:15:41.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.951+0000 7f4819ffb640 1 -- 172.21.15.116:0/985485113 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f48240a4490 con 0x560a319be4a0 2024-09-12T03:15:41.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.951+0000 7f483af96740 1 -- 172.21.15.116:0/985485113 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f482c004b40 con 0x560a319be4a0 2024-09-12T03:15:41.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.951+0000 7f4819ffb640 1 -- 172.21.15.116:0/985485113 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f48240a4690 con 0x560a319be4a0 2024-09-12T03:15:41.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.951+0000 7f483af96740 10 client.18786 fetch_fsmap learned FSMap version 25 2024-09-12T03:15:41.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.951+0000 7f483af96740 1 -- 172.21.15.116:0/985485113 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560a31a43cf0 con 0x560a319be4a0 2024-09-12T03:15:41.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.951+0000 7f4819ffb640 1 -- 172.21.15.116:0/985485113 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f482c004b40 con 0x560a319be4a0 2024-09-12T03:15:41.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.951+0000 7f4819ffb640 1 -- 172.21.15.116:0/985485113 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f482409ac00 con 0x560a319be4a0 2024-09-12T03:15:41.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.951+0000 7f483af96740 10 client.18786 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:15:41.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.951+0000 7f483af96740 10 client.18786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:15:41.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.951+0000 7f483af96740 20 client.18786 populate_metadata read hostname 'smithi116' 2024-09-12T03:15:41.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.951+0000 7f483af96740 1 --2- 172.21.15.116:0/985485113 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560a31a44410 0x560a31a46850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:41.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.951+0000 7f483af96740 4 client.18786 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:15:41.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.951+0000 7f483af96740 1 -- 172.21.15.116:0/985485113 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560a3192bd60 con 0x560a31a44410 2024-09-12T03:15:41.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.952+0000 7f482bfff640 1 --2- 172.21.15.116:0/985485113 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560a31a44410 0x560a31a46850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:41.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.952+0000 7f482bfff640 1 --2- 172.21.15.116:0/985485113 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560a31a44410 0x560a31a46850 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8f1b8f4a788a750 server_cookie=f43aa2eb1a552595 in_seq=0 out_seq=0 2024-09-12T03:15:41.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.952+0000 7f4819ffb640 10 client.18786 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:41.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.952+0000 7f4819ffb640 1 -- 172.21.15.116:0/985485113 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560a3192bd60 con 0x560a31a44410 2024-09-12T03:15:41.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.952+0000 7f4819ffb640 10 client.18786 handle_command_reply: tid=0 2024-09-12T03:15:41.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.952+0000 7f483af96740 1 client.18786 shutdown 2024-09-12T03:15:41.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.952+0000 7f483af96740 1 -- 172.21.15.116:0/985485113 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a319be4a0 msgr2=0x560a319d1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:41.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.953+0000 7f483af96740 1 --2- 172.21.15.116:0/985485113 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a319be4a0 0x560a319d1cd0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f4824098680 tx=0x7f482409bd70 comp rx=0 tx=0).stop 2024-09-12T03:15:41.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.953+0000 7f483af96740 1 -- 172.21.15.116:0/985485113 shutdown_connections 2024-09-12T03:15:41.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.953+0000 7f483af96740 1 --2- 172.21.15.116:0/985485113 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x560a31a44410 0x560a31a46850 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.953+0000 7f483af96740 1 --2- 172.21.15.116:0/985485113 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a319c9680 0x560a319cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.953+0000 7f483af96740 1 --2- 172.21.15.116:0/985485113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a319bedc0 0x560a319cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.953+0000 7f483af96740 1 --2- 172.21.15.116:0/985485113 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a319be4a0 0x560a319d1cd0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.953+0000 7f483af96740 1 -- 172.21.15.116:0/985485113 >> 172.21.15.116:0/985485113 conn(0x560a319baa30 msgr2=0x560a319c18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:41.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.953+0000 7f4819ffb640 0 client.18786 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:41.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.953+0000 7f483af96740 1 -- 172.21.15.116:0/985485113 shutdown_connections 2024-09-12T03:15:41.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.953+0000 7f483af96740 1 -- 172.21.15.116:0/985485113 wait complete. 2024-09-12T03:15:41.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.953+0000 7f483af96740 20 client.18786 trim_cache size 0 max 16384 2024-09-12T03:15:41.960 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:15:41.960 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:15:41.960 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:15:41.960 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:15:41.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.953+0000 7f4838bbd640 1 -- 172.21.15.116:0/1674845986 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f48200a2b80 msgr2=0x7f48200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:41.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.953+0000 7f4838bbd640 1 --2- 172.21.15.116:0/1674845986 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f48200a2b80 0x7f48200a5040 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f482c009be0 tx=0x7f482c0303a0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.953+0000 7f4838bbd640 1 -- 172.21.15.116:0/1674845986 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4834100e20 msgr2=0x7f48341992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:41.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.953+0000 7f4838bbd640 1 --2- 172.21.15.116:0/1674845986 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4834100e20 0x7f48341992e0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f482400e9f0 tx=0x7f482400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.954+0000 7f4838bbd640 1 -- 172.21.15.116:0/1674845986 shutdown_connections 2024-09-12T03:15:41.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.954+0000 7f4838bbd640 1 --2- 172.21.15.116:0/1674845986 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f48200a2b80 0x7f48200a5040 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.954+0000 7f4838bbd640 1 --2- 172.21.15.116:0/1674845986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4834102ef0 0x7f48341a4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.954+0000 7f4838bbd640 1 --2- 172.21.15.116:0/1674845986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4834102570 0x7f4834199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.954+0000 7f4838bbd640 1 --2- 172.21.15.116:0/1674845986 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4834100e20 0x7f48341992e0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:41.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.954+0000 7f4838bbd640 1 -- 172.21.15.116:0/1674845986 >> 172.21.15.116:0/1674845986 conn(0x7f48340fc7d0 msgr2=0x7f48340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:41.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.954+0000 7f4838bbd640 1 -- 172.21.15.116:0/1674845986 shutdown_connections 2024-09-12T03:15:41.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:41.954+0000 7f4838bbd640 1 -- 172.21.15.116:0/1674845986 wait complete. 2024-09-12T03:15:42.828 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:15:42.828 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:15:43.074 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:15:43.510 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:43 smithi116 ceph-mon[31410]: pgmap v720: 97 pgs: 97 active+clean; 6.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:15:43.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.590+0000 7f00663f4640 1 -- 172.21.15.116:0/2877151941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0060101de0 msgr2=0x7f00601021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:43.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.590+0000 7f00663f4640 1 --2- 172.21.15.116:0/2877151941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0060101de0 0x7f00601021c0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f0050009900 tx=0x7f005002a410 comp rx=0 tx=0).stop 2024-09-12T03:15:43.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.590+0000 7f00663f4640 1 -- 172.21.15.116:0/2877151941 shutdown_connections 2024-09-12T03:15:43.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.590+0000 7f00663f4640 1 --2- 172.21.15.116:0/2877151941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0060106f10 0x7f006010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:43.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.590+0000 7f00663f4640 1 --2- 172.21.15.116:0/2877151941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0060102790 0x7f00601067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:43.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.590+0000 7f00663f4640 1 --2- 172.21.15.116:0/2877151941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0060101de0 0x7f00601021c0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:43.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.590+0000 7f00663f4640 1 -- 172.21.15.116:0/2877151941 >> 172.21.15.116:0/2877151941 conn(0x7f00600fd8c0 msgr2=0x7f00600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:43.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.590+0000 7f00663f4640 1 -- 172.21.15.116:0/2877151941 shutdown_connections 2024-09-12T03:15:43.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.590+0000 7f00663f4640 1 -- 172.21.15.116:0/2877151941 wait complete. 2024-09-12T03:15:43.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.590+0000 7f00663f4640 1 Processor -- start 2024-09-12T03:15:43.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.590+0000 7f00663f4640 1 -- start start 2024-09-12T03:15:43.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.591+0000 7f00663f4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0060101de0 0x7f0060198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:43.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.591+0000 7f00663f4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0060102790 0x7f0060198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:43.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.591+0000 7f00663f4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0060106f10 0x7f006019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:43.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.591+0000 7f00663f4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0060111f70 con 0x7f0060102790 2024-09-12T03:15:43.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.591+0000 7f00663f4640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0060111df0 con 0x7f0060106f10 2024-09-12T03:15:43.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.591+0000 7f00663f4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f00601120f0 con 0x7f0060101de0 2024-09-12T03:15:43.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.591+0000 7f0057fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0060102790 0x7f0060198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:43.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.591+0000 7f0057fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0060102790 0x7f0060198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52680/0 (socket says 172.21.15.116:52680) 2024-09-12T03:15:43.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.591+0000 7f0057fff640 1 -- 172.21.15.116:0/2253333984 learned_addr learned my addr 172.21.15.116:0/2253333984 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:43.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.591+0000 7f005ffff640 1 --2- 172.21.15.116:0/2253333984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0060101de0 0x7f0060198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:43.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.591+0000 7f0057fff640 1 -- 172.21.15.116:0/2253333984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0060101de0 msgr2=0x7f0060198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:43.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.591+0000 7f0057fff640 1 --2- 172.21.15.116:0/2253333984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0060101de0 0x7f0060198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:43.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.591+0000 7f0057fff640 1 -- 172.21.15.116:0/2253333984 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0060106f10 msgr2=0x7f006019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:43.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.591+0000 7f0057fff640 1 --2- 172.21.15.116:0/2253333984 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0060106f10 0x7f006019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:43.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.591+0000 7f0057fff640 1 -- 172.21.15.116:0/2253333984 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006019d3e0 con 0x7f0060102790 2024-09-12T03:15:43.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.591+0000 7f0057fff640 1 --2- 172.21.15.116:0/2253333984 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0060102790 0x7f0060198970 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f004800c890 tx=0x7f004800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:43.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.591+0000 7f005dffb640 1 -- 172.21.15.116:0/2253333984 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0048004220 con 0x7f0060102790 2024-09-12T03:15:43.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.591+0000 7f00663f4640 1 -- 172.21.15.116:0/2253333984 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f006019d6d0 con 0x7f0060102790 2024-09-12T03:15:43.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.591+0000 7f005dffb640 1 -- 172.21.15.116:0/2253333984 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00480043c0 con 0x7f0060102790 2024-09-12T03:15:43.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.591+0000 7f005dffb640 1 -- 172.21.15.116:0/2253333984 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0048004a00 con 0x7f0060102790 2024-09-12T03:15:43.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.591+0000 7f00663f4640 1 -- 172.21.15.116:0/2253333984 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00601a4fb0 con 0x7f0060102790 2024-09-12T03:15:43.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.592+0000 7f00663f4640 1 -- 172.21.15.116:0/2253333984 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00601021c0 con 0x7f0060102790 2024-09-12T03:15:43.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.593+0000 7f005dffb640 1 -- 172.21.15.116:0/2253333984 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f00480079b0 con 0x7f0060102790 2024-09-12T03:15:43.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.594+0000 7f005dffb640 1 --2- 172.21.15.116:0/2253333984 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f004c0a2b80 0x7f004c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:43.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.594+0000 7f005ffff640 1 --2- 172.21.15.116:0/2253333984 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f004c0a2b80 0x7f004c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:43.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.594+0000 7f005dffb640 1 -- 172.21.15.116:0/2253333984 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f00480cf1c0 con 0x7f0060102790 2024-09-12T03:15:43.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.594+0000 7f005ffff640 1 --2- 172.21.15.116:0/2253333984 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f004c0a2b80 0x7f004c0a5040 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f00500096c0 tx=0x7f0050031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:43.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.597+0000 7f005dffb640 1 -- 172.21.15.116:0/2253333984 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0048014030 con 0x7f0060102790 2024-09-12T03:15:43.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:43 smithi183 ceph-mon[35535]: pgmap v720: 97 pgs: 97 active+clean; 6.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:15:43.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:43 smithi177 ceph-mon[35305]: pgmap v720: 97 pgs: 97 active+clean; 6.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:15:43.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.797+0000 7f00663f4640 1 -- 172.21.15.116:0/2253333984 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0060199790 con 0x7f0060102790 2024-09-12T03:15:43.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.798+0000 7f005dffb640 1 -- 172.21.15.116:0/2253333984 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f0060199790 con 0x7f0060102790 2024-09-12T03:15:43.798 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:15:43.799 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:15:43.799 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:15:43.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.800+0000 7f00663f4640 1 -- 172.21.15.116:0/2253333984 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f004c0a2b80 msgr2=0x7f004c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:43.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.800+0000 7f00663f4640 1 --2- 172.21.15.116:0/2253333984 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f004c0a2b80 0x7f004c0a5040 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f00500096c0 tx=0x7f0050031490 comp rx=0 tx=0).stop 2024-09-12T03:15:43.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.800+0000 7f00663f4640 1 -- 172.21.15.116:0/2253333984 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0060102790 msgr2=0x7f0060198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:43.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.800+0000 7f00663f4640 1 --2- 172.21.15.116:0/2253333984 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0060102790 0x7f0060198970 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f004800c890 tx=0x7f004800cd50 comp rx=0 tx=0).stop 2024-09-12T03:15:43.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.800+0000 7f00663f4640 1 -- 172.21.15.116:0/2253333984 shutdown_connections 2024-09-12T03:15:43.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.800+0000 7f00663f4640 1 --2- 172.21.15.116:0/2253333984 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f004c0a2b80 0x7f004c0a5040 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:43.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.800+0000 7f00663f4640 1 --2- 172.21.15.116:0/2253333984 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0060106f10 0x7f006019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:43.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.800+0000 7f00663f4640 1 --2- 172.21.15.116:0/2253333984 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0060102790 0x7f0060198970 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:43.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.800+0000 7f00663f4640 1 --2- 172.21.15.116:0/2253333984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0060101de0 0x7f0060198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:43.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.800+0000 7f00663f4640 1 -- 172.21.15.116:0/2253333984 >> 172.21.15.116:0/2253333984 conn(0x7f00600fd8c0 msgr2=0x7f00600fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:43.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.800+0000 7f00663f4640 1 -- 172.21.15.116:0/2253333984 shutdown_connections 2024-09-12T03:15:43.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:43.800+0000 7f00663f4640 1 -- 172.21.15.116:0/2253333984 wait complete. 2024-09-12T03:15:44.567 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:15:44.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:44 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2253333984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:15:44.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:44 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2253333984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:15:44.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:44 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2253333984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:15:44.812 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:15:45.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.308+0000 7f700ad00640 1 -- 172.21.15.116:0/2800810678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70041027a0 msgr2=0x7f70041067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:45.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.308+0000 7f700ad00640 1 --2- 172.21.15.116:0/2800810678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70041027a0 0x7f70041067f0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f6fec009900 tx=0x7f6fec02a410 comp rx=0 tx=0).stop 2024-09-12T03:15:45.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.308+0000 7f700ad00640 1 -- 172.21.15.116:0/2800810678 shutdown_connections 2024-09-12T03:15:45.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.308+0000 7f700ad00640 1 --2- 172.21.15.116:0/2800810678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7004106f20 0x7f700410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.308+0000 7f700ad00640 1 --2- 172.21.15.116:0/2800810678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70041027a0 0x7f70041067f0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.308+0000 7f700ad00640 1 --2- 172.21.15.116:0/2800810678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7004101df0 0x7f70041021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.308+0000 7f700ad00640 1 -- 172.21.15.116:0/2800810678 >> 172.21.15.116:0/2800810678 conn(0x7f70040fd8f0 msgr2=0x7f70040ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:45.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.308+0000 7f700ad00640 1 -- 172.21.15.116:0/2800810678 shutdown_connections 2024-09-12T03:15:45.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.308+0000 7f700ad00640 1 -- 172.21.15.116:0/2800810678 wait complete. 2024-09-12T03:15:45.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.308+0000 7f700ad00640 1 Processor -- start 2024-09-12T03:15:45.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.308+0000 7f700ad00640 1 -- start start 2024-09-12T03:15:45.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.309+0000 7f700ad00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7004101df0 0x7f7004198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:45.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.309+0000 7f700ad00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70041027a0 0x7f70041989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:45.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.309+0000 7f700ad00640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7004106f20 0x7f700419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:45.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.309+0000 7f700ad00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7004111fa0 con 0x7f7004101df0 2024-09-12T03:15:45.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.309+0000 7f700ad00640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7004111e20 con 0x7f7004106f20 2024-09-12T03:15:45.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.309+0000 7f700ad00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7004112120 con 0x7f70041027a0 2024-09-12T03:15:45.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.309+0000 7f7008a9c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7004101df0 0x7f7004198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:45.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.309+0000 7f7008a9c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7004101df0 0x7f7004198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52690/0 (socket says 172.21.15.116:52690) 2024-09-12T03:15:45.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.309+0000 7f7008a9c640 1 -- 172.21.15.116:0/3056727436 learned_addr learned my addr 172.21.15.116:0/3056727436 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:45.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.309+0000 7f7003fff640 1 --2- 172.21.15.116:0/3056727436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70041027a0 0x7f70041989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:45.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.309+0000 7f700929d640 1 --2- 172.21.15.116:0/3056727436 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7004106f20 0x7f700419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:45.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.309+0000 7f7008a9c640 1 -- 172.21.15.116:0/3056727436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70041027a0 msgr2=0x7f70041989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:45.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.309+0000 7f7008a9c640 1 --2- 172.21.15.116:0/3056727436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70041027a0 0x7f70041989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.309+0000 7f7008a9c640 1 -- 172.21.15.116:0/3056727436 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7004106f20 msgr2=0x7f700419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:45.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.309+0000 7f7008a9c640 1 --2- 172.21.15.116:0/3056727436 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7004106f20 0x7f700419cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.309+0000 7f7008a9c640 1 -- 172.21.15.116:0/3056727436 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f700419d430 con 0x7f7004101df0 2024-09-12T03:15:45.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.309+0000 7f700929d640 1 --2- 172.21.15.116:0/3056727436 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7004106f20 0x7f700419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:15:45.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.309+0000 7f7008a9c640 1 --2- 172.21.15.116:0/3056727436 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7004101df0 0x7f7004198480 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400ba50 tx=0x7f6ff400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:45.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.309+0000 7f7001ffb640 1 -- 172.21.15.116:0/3056727436 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4019070 con 0x7f7004101df0 2024-09-12T03:15:45.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.309+0000 7f700ad00640 1 -- 172.21.15.116:0/3056727436 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f700419d720 con 0x7f7004101df0 2024-09-12T03:15:45.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.309+0000 7f7001ffb640 1 -- 172.21.15.116:0/3056727436 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ff400cbe0 con 0x7f7004101df0 2024-09-12T03:15:45.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.309+0000 7f7001ffb640 1 -- 172.21.15.116:0/3056727436 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff40076e0 con 0x7f7004101df0 2024-09-12T03:15:45.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.310+0000 7f700ad00640 1 -- 172.21.15.116:0/3056727436 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70041a5000 con 0x7f7004101df0 2024-09-12T03:15:45.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.312+0000 7f7001ffb640 1 -- 172.21.15.116:0/3056727436 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6ff401c070 con 0x7f7004101df0 2024-09-12T03:15:45.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.313+0000 7f700d0d9740 1 Processor -- start 2024-09-12T03:15:45.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.313+0000 7f700d0d9740 1 -- start start 2024-09-12T03:15:45.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.313+0000 7f700d0d9740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e21f5beb40 0x55e21f5bd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:45.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.313+0000 7f700d0d9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e21f5bd5d0 0x55e21f5bda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:45.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.313+0000 7f700d0d9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e21f5c8370 0x55e21f5ca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:45.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.313+0000 7f700d0d9740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e21f5232f0 con 0x55e21f5beb40 2024-09-12T03:15:45.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.313+0000 7f700d0d9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e21f4cacf0 con 0x55e21f5bd5d0 2024-09-12T03:15:45.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.313+0000 7f700d0d9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e21f51f2b0 con 0x55e21f5c8370 2024-09-12T03:15:45.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.313+0000 7f7001ffb640 1 --2- 172.21.15.116:0/3056727436 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6ff00a2b80 0x7f6ff00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:45.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.313+0000 7f700929d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e21f5beb40 0x55e21f5bd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:45.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.313+0000 7f700929d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e21f5beb40 0x55e21f5bd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52702/0 (socket says 172.21.15.116:52702) 2024-09-12T03:15:45.340 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.313+0000 7f700929d640 1 -- 172.21.15.116:0/4188717833 learned_addr learned my addr 172.21.15.116:0/4188717833 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:45.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.313+0000 7f700929d640 1 --2- 172.21.15.116:0/3056727436 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6ff00a2b80 0x7f6ff00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:45.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.313+0000 7f7003fff640 1 --2- 172.21.15.116:0/4188717833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e21f5c8370 0x55e21f5ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:45.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.313+0000 7f7001ffb640 1 -- 172.21.15.116:0/3056727436 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f6ff40ce670 con 0x7f7004101df0 2024-09-12T03:15:45.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.313+0000 7f7008a9c640 1 --2- 172.21.15.116:0/4188717833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e21f5bd5d0 0x55e21f5bda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:45.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.313+0000 7f700929d640 1 -- 172.21.15.116:0/4188717833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e21f5c8370 msgr2=0x55e21f5ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:45.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.313+0000 7f700929d640 1 --2- 172.21.15.116:0/4188717833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e21f5c8370 0x55e21f5ca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.313+0000 7f700929d640 1 -- 172.21.15.116:0/4188717833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e21f5bd5d0 msgr2=0x55e21f5bda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:45.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.313+0000 7f700929d640 1 --2- 172.21.15.116:0/4188717833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e21f5bd5d0 0x55e21f5bda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.313+0000 7f700929d640 1 -- 172.21.15.116:0/4188717833 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e21f4af3a0 con 0x55e21f5beb40 2024-09-12T03:15:45.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.314+0000 7f700929d640 1 --2- 172.21.15.116:0/4188717833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e21f5beb40 0x55e21f5bd090 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc012ac0 tx=0x7f6ffc012f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c0ed506534c97df server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:45.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.314+0000 7f700929d640 1 --2- 172.21.15.116:0/3056727436 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6ff00a2b80 0x7f6ff00a5040 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc00f7f0 tx=0x7f6ffc01a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:45.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.314+0000 7f6ffaffd640 1 -- 172.21.15.116:0/4188717833 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc00ab80 con 0x55e21f5beb40 2024-09-12T03:15:45.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.314+0000 7f6ffaffd640 1 -- 172.21.15.116:0/4188717833 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ffc00de50 con 0x55e21f5beb40 2024-09-12T03:15:45.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.314+0000 7f6ffaffd640 1 -- 172.21.15.116:0/4188717833 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc01c410 con 0x55e21f5beb40 2024-09-12T03:15:45.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.314+0000 7f700d0d9740 1 -- 172.21.15.116:0/4188717833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e21f5beb40 msgr2=0x55e21f5bd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:45.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.314+0000 7f700d0d9740 1 --2- 172.21.15.116:0/4188717833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e21f5beb40 0x55e21f5bd090 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc012ac0 tx=0x7f6ffc012f80 comp rx=0 tx=0).stop 2024-09-12T03:15:45.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.315+0000 7f700d0d9740 1 -- 172.21.15.116:0/4188717833 shutdown_connections 2024-09-12T03:15:45.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.315+0000 7f700d0d9740 1 --2- 172.21.15.116:0/4188717833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e21f5c8370 0x55e21f5ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.315+0000 7f700d0d9740 1 --2- 172.21.15.116:0/4188717833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e21f5bd5d0 0x55e21f5bda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.315+0000 7f700d0d9740 1 --2- 172.21.15.116:0/4188717833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e21f5beb40 0x55e21f5bd090 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.315+0000 7f700d0d9740 1 -- 172.21.15.116:0/4188717833 >> 172.21.15.116:0/4188717833 conn(0x55e21f586310 msgr2=0x55e21f5b9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:45.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.315+0000 7f700d0d9740 1 -- 172.21.15.116:0/4188717833 shutdown_connections 2024-09-12T03:15:45.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.315+0000 7f700d0d9740 1 -- 172.21.15.116:0/4188717833 wait complete. 2024-09-12T03:15:45.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.315+0000 7f700d0d9740 1 -- start start 2024-09-12T03:15:45.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.315+0000 7f700d0d9740 1 Processor -- start 2024-09-12T03:15:45.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.315+0000 7f700d0d9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e21f5bd5d0 0x55e21f5c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:45.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.315+0000 7f700d0d9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e21f5beb40 0x55e21f5c5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:45.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.315+0000 7f700d0d9740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e21f5c8370 0x55e21f5c5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:45.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.315+0000 7f700d0d9740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e21f5cb010 con 0x55e21f5c8370 2024-09-12T03:15:45.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.315+0000 7f700d0d9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e21f4de060 con 0x55e21f5bd5d0 2024-09-12T03:15:45.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.315+0000 7f700d0d9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e21f5cb190 con 0x55e21f5beb40 2024-09-12T03:15:45.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.315+0000 7f700d0d9740 10 client.0 _ll_register_callbacks cb 0x55e21f2318c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:45.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.316+0000 7f700929d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e21f5c8370 0x55e21f5c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:45.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.316+0000 7f700929d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e21f5c8370 0x55e21f5c5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52712/0 (socket says 172.21.15.116:52712) 2024-09-12T03:15:45.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.316+0000 7f700929d640 1 -- 172.21.15.116:0/1023035107 learned_addr learned my addr 172.21.15.116:0/1023035107 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:45.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.316+0000 7f7008a9c640 1 --2- 172.21.15.116:0/1023035107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e21f5bd5d0 0x55e21f5c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:45.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.316+0000 7f7003fff640 1 --2- 172.21.15.116:0/1023035107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e21f5beb40 0x55e21f5c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:45.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.316+0000 7f700929d640 1 -- 172.21.15.116:0/1023035107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e21f5beb40 msgr2=0x55e21f5c5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:45.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.316+0000 7f700929d640 1 --2- 172.21.15.116:0/1023035107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e21f5beb40 0x55e21f5c5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.316+0000 7f700929d640 1 -- 172.21.15.116:0/1023035107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e21f5bd5d0 msgr2=0x55e21f5c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:45.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.316+0000 7f700929d640 1 --2- 172.21.15.116:0/1023035107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e21f5bd5d0 0x55e21f5c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.316+0000 7f700929d640 1 -- 172.21.15.116:0/1023035107 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e21f2943c0 con 0x55e21f5c8370 2024-09-12T03:15:45.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.316+0000 7f700929d640 1 --2- 172.21.15.116:0/1023035107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e21f5c8370 0x55e21f5c5800 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc00d920 tx=0x7f6ffc005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d16b72e286584e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:45.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.316+0000 7f6fe3fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:15:45.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.316+0000 7f6fe3fff640 1 -- 172.21.15.116:0/1023035107 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc02a050 con 0x55e21f5c8370 2024-09-12T03:15:45.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.316+0000 7f6fe3fff640 1 -- 172.21.15.116:0/1023035107 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f6ffc024450 con 0x55e21f5c8370 2024-09-12T03:15:45.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.316+0000 7f700d0d9740 1 -- 172.21.15.116:0/1023035107 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ffc00de50 con 0x55e21f5c8370 2024-09-12T03:15:45.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.316+0000 7f6fe3fff640 1 -- 172.21.15.116:0/1023035107 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6ffc024650 con 0x55e21f5c8370 2024-09-12T03:15:45.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.316+0000 7f700d0d9740 10 client.18804 fetch_fsmap learned FSMap version 25 2024-09-12T03:15:45.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.316+0000 7f700d0d9740 1 -- 172.21.15.116:0/1023035107 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e21f5d6b20 con 0x55e21f5c8370 2024-09-12T03:15:45.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.317+0000 7f6fe3fff640 1 -- 172.21.15.116:0/1023035107 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ffc00de50 con 0x55e21f5c8370 2024-09-12T03:15:45.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.317+0000 7f6fe3fff640 1 -- 172.21.15.116:0/1023035107 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6ffc01b070 con 0x55e21f5c8370 2024-09-12T03:15:45.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.317+0000 7f700d0d9740 10 client.18804 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:15:45.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.317+0000 7f700d0d9740 10 client.18804 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:15:45.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.317+0000 7f700d0d9740 20 client.18804 populate_metadata read hostname 'smithi116' 2024-09-12T03:15:45.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.317+0000 7f700d0d9740 1 --2- 172.21.15.116:0/1023035107 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55e21f5d7f30 0x55e21f5da320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:45.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.317+0000 7f700d0d9740 4 client.18804 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:15:45.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.317+0000 7f700d0d9740 1 -- 172.21.15.116:0/1023035107 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e21f530860 con 0x55e21f5d7f30 2024-09-12T03:15:45.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.317+0000 7f7008a9c640 1 --2- 172.21.15.116:0/1023035107 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55e21f5d7f30 0x55e21f5da320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:45.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.318+0000 7f6fe3fff640 10 client.18804 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:15:45.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.318+0000 7f7008a9c640 1 --2- 172.21.15.116:0/1023035107 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55e21f5d7f30 0x55e21f5da320 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cca0dc9af7667831 server_cookie=9214e301ea3605f7 in_seq=0 out_seq=0 2024-09-12T03:15:45.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.318+0000 7f6fe3fff640 1 -- 172.21.15.116:0/1023035107 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e21f530860 con 0x55e21f5d7f30 2024-09-12T03:15:45.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.318+0000 7f6fe3fff640 10 client.18804 handle_command_reply: tid=0 2024-09-12T03:15:45.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.318+0000 7f700d0d9740 1 client.18804 shutdown 2024-09-12T03:15:45.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.318+0000 7f700d0d9740 1 -- 172.21.15.116:0/1023035107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e21f5c8370 msgr2=0x55e21f5c5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:45.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.318+0000 7f700d0d9740 1 --2- 172.21.15.116:0/1023035107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e21f5c8370 0x55e21f5c5800 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc00d920 tx=0x7f6ffc005d60 comp rx=0 tx=0).stop 2024-09-12T03:15:45.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.318+0000 7f700d0d9740 1 -- 172.21.15.116:0/1023035107 shutdown_connections 2024-09-12T03:15:45.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.318+0000 7f700d0d9740 1 --2- 172.21.15.116:0/1023035107 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55e21f5d7f30 0x55e21f5da320 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.318+0000 7f700d0d9740 1 --2- 172.21.15.116:0/1023035107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e21f5c8370 0x55e21f5c5800 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.318+0000 7f6fe3fff640 0 client.18804 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:15:45.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.318+0000 7f700d0d9740 1 --2- 172.21.15.116:0/1023035107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e21f5beb40 0x55e21f5c5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.318+0000 7f700d0d9740 1 --2- 172.21.15.116:0/1023035107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e21f5bd5d0 0x55e21f5c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.318+0000 7f700d0d9740 1 -- 172.21.15.116:0/1023035107 >> 172.21.15.116:0/1023035107 conn(0x55e21f586310 msgr2=0x55e21f5bf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:45.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.319+0000 7f700d0d9740 1 -- 172.21.15.116:0/1023035107 shutdown_connections 2024-09-12T03:15:45.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.319+0000 7f700d0d9740 1 -- 172.21.15.116:0/1023035107 wait complete. 2024-09-12T03:15:45.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.319+0000 7f700d0d9740 20 client.18804 trim_cache size 0 max 16384 2024-09-12T03:15:45.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.331+0000 7f700d0d9740 1 Processor -- start 2024-09-12T03:15:45.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.331+0000 7f700d0d9740 1 -- start start 2024-09-12T03:15:45.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.332+0000 7f700d0d9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e21f5bd5d0 0x55e21f5b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:45.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.332+0000 7f700d0d9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e21f5beb40 0x55e21f5c0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:45.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.332+0000 7f700d0d9740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e21f5c8370 0x55e21f5c0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:45.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.332+0000 7f700d0d9740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f70041afe00 con 0x55e21f5c8370 2024-09-12T03:15:45.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.332+0000 7f700d0d9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f70041ad6d0 con 0x55e21f5bd5d0 2024-09-12T03:15:45.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.332+0000 7f700d0d9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f70041ab520 con 0x55e21f5beb40 2024-09-12T03:15:45.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.332+0000 7f700929d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e21f5c8370 0x55e21f5c0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:45.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.332+0000 7f700929d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e21f5c8370 0x55e21f5c0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52716/0 (socket says 172.21.15.116:52716) 2024-09-12T03:15:45.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.332+0000 7f700929d640 1 -- 172.21.15.116:0/135486427 learned_addr learned my addr 172.21.15.116:0/135486427 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:45.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.332+0000 7f7003fff640 1 --2- 172.21.15.116:0/135486427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e21f5beb40 0x55e21f5c0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:45.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.332+0000 7f700929d640 1 -- 172.21.15.116:0/135486427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e21f5beb40 msgr2=0x55e21f5c0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:45.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.332+0000 7f700929d640 1 --2- 172.21.15.116:0/135486427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e21f5beb40 0x55e21f5c0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.332+0000 7f700929d640 1 -- 172.21.15.116:0/135486427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e21f5bd5d0 msgr2=0x55e21f5b4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:45.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.332+0000 7f700929d640 1 --2- 172.21.15.116:0/135486427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e21f5bd5d0 0x55e21f5b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.332+0000 7f700929d640 1 -- 172.21.15.116:0/135486427 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e21f5c5010 con 0x55e21f5c8370 2024-09-12T03:15:45.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.332+0000 7f7003fff640 1 --2- 172.21.15.116:0/135486427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e21f5beb40 0x55e21f5c0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:15:45.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.332+0000 7f700929d640 1 --2- 172.21.15.116:0/135486427 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e21f5c8370 0x55e21f5c0660 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc00a680 tx=0x7f6ffc00daf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=105ee6774b5e177d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:45.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.332+0000 7f6ffa7fc640 1 -- 172.21.15.116:0/135486427 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc005bc0 con 0x55e21f5c8370 2024-09-12T03:15:45.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.332+0000 7f6ffa7fc640 1 -- 172.21.15.116:0/135486427 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ffc03f020 con 0x55e21f5c8370 2024-09-12T03:15:45.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.332+0000 7f6ffa7fc640 1 -- 172.21.15.116:0/135486427 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc03a960 con 0x55e21f5c8370 2024-09-12T03:15:45.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.333+0000 7f700d0d9740 1 -- 172.21.15.116:0/135486427 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e21f5c8370 msgr2=0x55e21f5c0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:45.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.333+0000 7f700d0d9740 1 --2- 172.21.15.116:0/135486427 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e21f5c8370 0x55e21f5c0660 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc00a680 tx=0x7f6ffc00daf0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.333+0000 7f700d0d9740 1 -- 172.21.15.116:0/135486427 shutdown_connections 2024-09-12T03:15:45.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.333+0000 7f700d0d9740 1 --2- 172.21.15.116:0/135486427 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e21f5c8370 0x55e21f5c0660 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.333+0000 7f700d0d9740 1 --2- 172.21.15.116:0/135486427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e21f5beb40 0x55e21f5c0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.333+0000 7f700d0d9740 1 --2- 172.21.15.116:0/135486427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e21f5bd5d0 0x55e21f5b4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.333+0000 7f700d0d9740 1 -- 172.21.15.116:0/135486427 >> 172.21.15.116:0/135486427 conn(0x55e21f586310 msgr2=0x55e21f5c4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:45.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.333+0000 7f700d0d9740 1 -- 172.21.15.116:0/135486427 shutdown_connections 2024-09-12T03:15:45.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.333+0000 7f700d0d9740 1 -- 172.21.15.116:0/135486427 wait complete. 2024-09-12T03:15:45.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.333+0000 7f700d0d9740 1 -- start start 2024-09-12T03:15:45.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.333+0000 7f700d0d9740 1 Processor -- start 2024-09-12T03:15:45.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.334+0000 7f700d0d9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e21f5bd5d0 0x55e21f63ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:45.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.334+0000 7f700d0d9740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e21f5beb40 0x55e21f63b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:45.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.334+0000 7f700d0d9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e21f5c8370 0x55e21f5cb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:45.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.334+0000 7f700d0d9740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f70041abb90 con 0x55e21f5beb40 2024-09-12T03:15:45.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.334+0000 7f700d0d9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f70041ad000 con 0x55e21f5c8370 2024-09-12T03:15:45.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.334+0000 7f700d0d9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f70041ac260 con 0x55e21f5bd5d0 2024-09-12T03:15:45.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.334+0000 7f700d0d9740 10 client.0 _ll_register_callbacks cb 0x7f6ff4009950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:45.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.334+0000 7f7003fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e21f5beb40 0x55e21f63b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:45.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.334+0000 7f7003fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e21f5beb40 0x55e21f63b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52722/0 (socket says 172.21.15.116:52722) 2024-09-12T03:15:45.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.334+0000 7f7003fff640 1 -- 172.21.15.116:0/1177598954 learned_addr learned my addr 172.21.15.116:0/1177598954 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:45.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.334+0000 7f700929d640 1 --2- 172.21.15.116:0/1177598954 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e21f5c8370 0x55e21f5cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:45.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.334+0000 7f7008a9c640 1 --2- 172.21.15.116:0/1177598954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e21f5bd5d0 0x55e21f63ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:45.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.334+0000 7f7003fff640 1 -- 172.21.15.116:0/1177598954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e21f5bd5d0 msgr2=0x55e21f63ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:45.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.334+0000 7f7003fff640 1 --2- 172.21.15.116:0/1177598954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e21f5bd5d0 0x55e21f63ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.334+0000 7f7003fff640 1 -- 172.21.15.116:0/1177598954 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e21f5c8370 msgr2=0x55e21f5cb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:45.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.334+0000 7f7003fff640 1 --2- 172.21.15.116:0/1177598954 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e21f5c8370 0x55e21f5cb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.334+0000 7f7003fff640 1 -- 172.21.15.116:0/1177598954 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e21f5c8130 con 0x55e21f5beb40 2024-09-12T03:15:45.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.334+0000 7f700929d640 1 --2- 172.21.15.116:0/1177598954 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e21f5c8370 0x55e21f5cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:15:45.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.334+0000 7f7003fff640 1 --2- 172.21.15.116:0/1177598954 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e21f5beb40 0x55e21f63b170 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f6fec02a930 tx=0x7f6fec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bbc5ee1c904b68 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:45.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.334+0000 7f6fe3fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:15:45.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.334+0000 7f6fe3fff640 1 -- 172.21.15.116:0/1177598954 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec035040 con 0x55e21f5beb40 2024-09-12T03:15:45.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.334+0000 7f6fe3fff640 1 -- 172.21.15.116:0/1177598954 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f6fec005030 con 0x55e21f5beb40 2024-09-12T03:15:45.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.334+0000 7f700d0d9740 1 -- 172.21.15.116:0/1177598954 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ffc03f020 con 0x55e21f5beb40 2024-09-12T03:15:45.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.335+0000 7f6fe3fff640 1 -- 172.21.15.116:0/1177598954 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6fec04d070 con 0x55e21f5beb40 2024-09-12T03:15:45.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.335+0000 7f700d0d9740 10 client.18810 fetch_fsmap learned FSMap version 25 2024-09-12T03:15:45.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.335+0000 7f700d0d9740 1 -- 172.21.15.116:0/1177598954 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e21f642190 con 0x55e21f5beb40 2024-09-12T03:15:45.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.335+0000 7f6fe3fff640 1 -- 172.21.15.116:0/1177598954 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ffc03f020 con 0x55e21f5beb40 2024-09-12T03:15:45.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.335+0000 7f6fe3fff640 1 -- 172.21.15.116:0/1177598954 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6fec0035f0 con 0x55e21f5beb40 2024-09-12T03:15:45.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.335+0000 7f700d0d9740 10 client.18810 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:15:45.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.335+0000 7f700d0d9740 10 client.18810 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:15:45.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.335+0000 7f700d0d9740 20 client.18810 populate_metadata read hostname 'smithi116' 2024-09-12T03:15:45.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.335+0000 7f700d0d9740 1 --2- 172.21.15.116:0/1177598954 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55e21f642940 0x55e21f644d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:45.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.335+0000 7f700d0d9740 4 client.18810 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:15:45.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.335+0000 7f700d0d9740 1 -- 172.21.15.116:0/1177598954 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e21f52ad70 con 0x55e21f642940 2024-09-12T03:15:45.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.335+0000 7f7008a9c640 1 --2- 172.21.15.116:0/1177598954 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55e21f642940 0x55e21f644d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:45.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.336+0000 7f7008a9c640 1 --2- 172.21.15.116:0/1177598954 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55e21f642940 0x55e21f644d60 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9994cb630918dcb2 server_cookie=e38beb07dfba8235 in_seq=0 out_seq=0 2024-09-12T03:15:45.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.336+0000 7f6fe3fff640 10 client.18810 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:15:45.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.336+0000 7f6fe3fff640 1 -- 172.21.15.116:0/1177598954 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e21f52ad70 con 0x55e21f642940 2024-09-12T03:15:45.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.336+0000 7f6fe3fff640 10 client.18810 handle_command_reply: tid=0 2024-09-12T03:15:45.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.336+0000 7f700d0d9740 1 client.18810 shutdown 2024-09-12T03:15:45.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.336+0000 7f700d0d9740 1 -- 172.21.15.116:0/1177598954 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e21f5beb40 msgr2=0x55e21f63b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:45.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.336+0000 7f700d0d9740 1 --2- 172.21.15.116:0/1177598954 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e21f5beb40 0x55e21f63b170 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f6fec02a930 tx=0x7f6fec004270 comp rx=0 tx=0).stop 2024-09-12T03:15:45.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.336+0000 7f700d0d9740 1 -- 172.21.15.116:0/1177598954 shutdown_connections 2024-09-12T03:15:45.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.336+0000 7f700d0d9740 1 --2- 172.21.15.116:0/1177598954 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e21f5c8370 0x55e21f5cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.336+0000 7f700d0d9740 1 --2- 172.21.15.116:0/1177598954 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x55e21f642940 0x55e21f644d60 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.336+0000 7f700d0d9740 1 --2- 172.21.15.116:0/1177598954 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e21f5beb40 0x55e21f63b170 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.336+0000 7f700d0d9740 1 --2- 172.21.15.116:0/1177598954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e21f5bd5d0 0x55e21f63ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.336+0000 7f700d0d9740 1 -- 172.21.15.116:0/1177598954 >> 172.21.15.116:0/1177598954 conn(0x55e21f586310 msgr2=0x55e21f5c68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:45.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.336+0000 7f6fe3fff640 0 client.18810 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:15:45.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.336+0000 7f700d0d9740 1 -- 172.21.15.116:0/1177598954 shutdown_connections 2024-09-12T03:15:45.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.336+0000 7f700d0d9740 1 -- 172.21.15.116:0/1177598954 wait complete. 2024-09-12T03:15:45.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.336+0000 7f700d0d9740 20 client.18810 trim_cache size 0 max 16384 2024-09-12T03:15:45.353 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:15:45.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.337+0000 7f700ad00640 1 -- 172.21.15.116:0/3056727436 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6ff00a2b80 msgr2=0x7f6ff00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:45.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.337+0000 7f700ad00640 1 --2- 172.21.15.116:0/3056727436 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6ff00a2b80 0x7f6ff00a5040 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc00f7f0 tx=0x7f6ffc01a000 comp rx=0 tx=0).stop 2024-09-12T03:15:45.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.337+0000 7f700ad00640 1 -- 172.21.15.116:0/3056727436 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7004101df0 msgr2=0x7f7004198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:45.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.337+0000 7f700ad00640 1 --2- 172.21.15.116:0/3056727436 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7004101df0 0x7f7004198480 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400ba50 tx=0x7f6ff400bf10 comp rx=0 tx=0).stop 2024-09-12T03:15:45.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.337+0000 7f700ad00640 1 -- 172.21.15.116:0/3056727436 shutdown_connections 2024-09-12T03:15:45.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.337+0000 7f700ad00640 1 --2- 172.21.15.116:0/3056727436 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f6ff00a2b80 0x7f6ff00a5040 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.337+0000 7f700ad00640 1 --2- 172.21.15.116:0/3056727436 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7004106f20 0x7f700419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.337+0000 7f700ad00640 1 --2- 172.21.15.116:0/3056727436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70041027a0 0x7f70041989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.337+0000 7f700ad00640 1 --2- 172.21.15.116:0/3056727436 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7004101df0 0x7f7004198480 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:45.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.337+0000 7f700ad00640 1 -- 172.21.15.116:0/3056727436 >> 172.21.15.116:0/3056727436 conn(0x7f70040fd8f0 msgr2=0x7f70040fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:45.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.337+0000 7f700ad00640 1 -- 172.21.15.116:0/3056727436 shutdown_connections 2024-09-12T03:15:45.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:45.337+0000 7f700ad00640 1 -- 172.21.15.116:0/3056727436 wait complete. 2024-09-12T03:15:45.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:45 smithi116 ceph-mon[31410]: pgmap v721: 97 pgs: 97 active+clean; 6.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-12T03:15:45.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:45 smithi183 ceph-mon[35535]: pgmap v721: 97 pgs: 97 active+clean; 6.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-12T03:15:45.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:45 smithi177 ceph-mon[35305]: pgmap v721: 97 pgs: 97 active+clean; 6.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-12T03:15:46.178 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:15:46.426 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:15:46.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.960+0000 7f512bcd5640 1 -- 172.21.15.116:0/1834442612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5124100a80 msgr2=0x7f5124100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:46.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.960+0000 7f512bcd5640 1 --2- 172.21.15.116:0/1834442612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5124100a80 0x7f5124100ee0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f5118009930 tx=0x7f511802a420 comp rx=0 tx=0).stop 2024-09-12T03:15:46.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.960+0000 7f512bcd5640 1 -- 172.21.15.116:0/1834442612 shutdown_connections 2024-09-12T03:15:46.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.960+0000 7f512bcd5640 1 --2- 172.21.15.116:0/1834442612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5124101420 0x7f512410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.960+0000 7f512bcd5640 1 --2- 172.21.15.116:0/1834442612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5124100a80 0x7f5124100ee0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.960+0000 7f512bcd5640 1 --2- 172.21.15.116:0/1834442612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5124106a80 0x7f5124106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.960+0000 7f512bcd5640 1 -- 172.21.15.116:0/1834442612 >> 172.21.15.116:0/1834442612 conn(0x7f51240fc7a0 msgr2=0x7f51240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:46.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.960+0000 7f512bcd5640 1 -- 172.21.15.116:0/1834442612 shutdown_connections 2024-09-12T03:15:46.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.960+0000 7f512bcd5640 1 -- 172.21.15.116:0/1834442612 wait complete. 2024-09-12T03:15:46.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.961+0000 7f512bcd5640 1 Processor -- start 2024-09-12T03:15:46.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.961+0000 7f512bcd5640 1 -- start start 2024-09-12T03:15:46.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.961+0000 7f512bcd5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5124100a80 0x7f5124197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:46.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.961+0000 7f512bcd5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5124101420 0x7f51241975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:46.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.961+0000 7f512bcd5640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5124106a80 0x7f5124197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:46.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.961+0000 7f512bcd5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5124110e80 con 0x7f5124101420 2024-09-12T03:15:46.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.961+0000 7f512bcd5640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5124110d00 con 0x7f5124106a80 2024-09-12T03:15:46.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.961+0000 7f512bcd5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5124111000 con 0x7f5124100a80 2024-09-12T03:15:46.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.961+0000 7f5129270640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5124101420 0x7f51241975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:46.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.961+0000 7f5129270640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5124101420 0x7f51241975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52746/0 (socket says 172.21.15.116:52746) 2024-09-12T03:15:46.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.961+0000 7f5129270640 1 -- 172.21.15.116:0/2957831627 learned_addr learned my addr 172.21.15.116:0/2957831627 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:46.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.961+0000 7f5129a71640 1 --2- 172.21.15.116:0/2957831627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5124100a80 0x7f5124197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:46.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.961+0000 7f512a272640 1 --2- 172.21.15.116:0/2957831627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5124106a80 0x7f5124197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:46.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.961+0000 7f5129270640 1 -- 172.21.15.116:0/2957831627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5124100a80 msgr2=0x7f5124197060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:46.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.961+0000 7f5129270640 1 --2- 172.21.15.116:0/2957831627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5124100a80 0x7f5124197060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.961+0000 7f5129270640 1 -- 172.21.15.116:0/2957831627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5124106a80 msgr2=0x7f5124197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:46.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.961+0000 7f5129270640 1 --2- 172.21.15.116:0/2957831627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5124106a80 0x7f5124197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.961+0000 7f5129270640 1 -- 172.21.15.116:0/2957831627 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51241a68d0 con 0x7f5124101420 2024-09-12T03:15:46.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.961+0000 7f5129270640 1 --2- 172.21.15.116:0/2957831627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5124101420 0x7f51241975a0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f5118009230 tx=0x7f5118004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:46.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.962+0000 7f5116ffd640 1 -- 172.21.15.116:0/2957831627 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511803f030 con 0x7f5124101420 2024-09-12T03:15:46.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.962+0000 7f5116ffd640 1 -- 172.21.15.116:0/2957831627 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5118003ad0 con 0x7f5124101420 2024-09-12T03:15:46.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.962+0000 7f5116ffd640 1 -- 172.21.15.116:0/2957831627 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118035070 con 0x7f5124101420 2024-09-12T03:15:46.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.962+0000 7f512bcd5640 1 -- 172.21.15.116:0/2957831627 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51241a6b60 con 0x7f5124101420 2024-09-12T03:15:46.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.962+0000 7f512bcd5640 1 -- 172.21.15.116:0/2957831627 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51241a6fc0 con 0x7f5124101420 2024-09-12T03:15:46.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.964+0000 7f5116ffd640 1 -- 172.21.15.116:0/2957831627 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f5118042030 con 0x7f5124101420 2024-09-12T03:15:46.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.965+0000 7f512e0ae740 1 Processor -- start 2024-09-12T03:15:46.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.965+0000 7f512e0ae740 1 -- start start 2024-09-12T03:15:46.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.965+0000 7f512e0ae740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0282dbb40 0x55d0282da090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:46.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.965+0000 7f512e0ae740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0282da5d0 0x55d0282daa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:46.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.965+0000 7f512e0ae740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0282e5370 0x55d0282e7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:46.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.965+0000 7f512e0ae740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d0282402f0 con 0x55d0282dbb40 2024-09-12T03:15:46.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.965+0000 7f512e0ae740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d0281e7cf0 con 0x55d0282e5370 2024-09-12T03:15:46.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.965+0000 7f512e0ae740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d02823c2b0 con 0x55d0282da5d0 2024-09-12T03:15:46.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.965+0000 7f512a272640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0282dbb40 0x55d0282da090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:46.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.965+0000 7f512a272640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0282dbb40 0x55d0282da090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52760/0 (socket says 172.21.15.116:52760) 2024-09-12T03:15:46.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.965+0000 7f512a272640 1 -- 172.21.15.116:0/2654870527 learned_addr learned my addr 172.21.15.116:0/2654870527 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:46.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.965+0000 7f5129a71640 1 --2- 172.21.15.116:0/2654870527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0282da5d0 0x55d0282daa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:46.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.965+0000 7f5116ffd640 1 --2- 172.21.15.116:0/2957831627 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f51100a2a50 0x7f51100a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:46.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.965+0000 7f5129270640 1 --2- 172.21.15.116:0/2654870527 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0282e5370 0x55d0282e7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:46.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.965+0000 7f512a272640 1 --2- 172.21.15.116:0/2957831627 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f51100a2a50 0x7f51100a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:46.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.966+0000 7f5129a71640 1 -- 172.21.15.116:0/2654870527 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0282e5370 msgr2=0x55d0282e7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:46.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.966+0000 7f5129a71640 1 --2- 172.21.15.116:0/2654870527 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0282e5370 0x55d0282e7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.966+0000 7f5129a71640 1 -- 172.21.15.116:0/2654870527 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0282dbb40 msgr2=0x55d0282da090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:46.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.966+0000 7f5129a71640 1 --2- 172.21.15.116:0/2654870527 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0282dbb40 0x55d0282da090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.966+0000 7f5129a71640 1 -- 172.21.15.116:0/2654870527 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0281cc3a0 con 0x55d0282da5d0 2024-09-12T03:15:46.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.966+0000 7f5116ffd640 1 -- 172.21.15.116:0/2957831627 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f51180ef7d0 con 0x7f5124101420 2024-09-12T03:15:46.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.966+0000 7f5129a71640 1 --2- 172.21.15.116:0/2654870527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0282da5d0 0x55d0282daa10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f51200045b0 tx=0x7f512000bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=174c93686b916f42 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:46.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.966+0000 7f512a272640 1 --2- 172.21.15.116:0/2957831627 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f51100a2a50 0x7f51100a4f10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f511c012850 tx=0x7f511c004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:46.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.966+0000 7f510ffff640 1 -- 172.21.15.116:0/2654870527 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5120003bd0 con 0x55d0282da5d0 2024-09-12T03:15:46.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.966+0000 7f510ffff640 1 -- 172.21.15.116:0/2654870527 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5120015070 con 0x55d0282da5d0 2024-09-12T03:15:46.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.966+0000 7f510ffff640 1 -- 172.21.15.116:0/2654870527 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5120004d70 con 0x55d0282da5d0 2024-09-12T03:15:46.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.967+0000 7f512e0ae740 1 -- 172.21.15.116:0/2654870527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0282da5d0 msgr2=0x55d0282daa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:46.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.967+0000 7f512e0ae740 1 --2- 172.21.15.116:0/2654870527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0282da5d0 0x55d0282daa10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f51200045b0 tx=0x7f512000bb20 comp rx=0 tx=0).stop 2024-09-12T03:15:46.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.967+0000 7f512e0ae740 1 -- 172.21.15.116:0/2654870527 shutdown_connections 2024-09-12T03:15:46.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.967+0000 7f512e0ae740 1 --2- 172.21.15.116:0/2654870527 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0282e5370 0x55d0282e7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.967+0000 7f512e0ae740 1 --2- 172.21.15.116:0/2654870527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0282da5d0 0x55d0282daa10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.967+0000 7f512e0ae740 1 --2- 172.21.15.116:0/2654870527 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0282dbb40 0x55d0282da090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.967+0000 7f512e0ae740 1 -- 172.21.15.116:0/2654870527 >> 172.21.15.116:0/2654870527 conn(0x55d0282a3310 msgr2=0x55d0282d6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:46.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.967+0000 7f512e0ae740 1 -- 172.21.15.116:0/2654870527 shutdown_connections 2024-09-12T03:15:46.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.967+0000 7f512e0ae740 1 -- 172.21.15.116:0/2654870527 wait complete. 2024-09-12T03:15:46.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.967+0000 7f512e0ae740 1 -- start start 2024-09-12T03:15:46.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.967+0000 7f512e0ae740 1 Processor -- start 2024-09-12T03:15:46.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.968+0000 7f512e0ae740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0282da5d0 0x55d0282e4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:46.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.968+0000 7f512e0ae740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0282dbb40 0x55d0282e2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:46.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.968+0000 7f512e0ae740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0282e5370 0x55d0282e2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:46.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.968+0000 7f512e0ae740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d0282e8070 con 0x55d0282e5370 2024-09-12T03:15:46.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.968+0000 7f512e0ae740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d0281fb060 con 0x55d0282dbb40 2024-09-12T03:15:46.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.968+0000 7f512e0ae740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d0282e81f0 con 0x55d0282da5d0 2024-09-12T03:15:46.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.968+0000 7f512e0ae740 10 client.0 _ll_register_callbacks cb 0x55d027f4e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:46.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.968+0000 7f512a272640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0282e5370 0x55d0282e2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:46.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.968+0000 7f512a272640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0282e5370 0x55d0282e2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52764/0 (socket says 172.21.15.116:52764) 2024-09-12T03:15:46.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.968+0000 7f512a272640 1 -- 172.21.15.116:0/3068157178 learned_addr learned my addr 172.21.15.116:0/3068157178 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:46.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.968+0000 7f5129a71640 1 --2- 172.21.15.116:0/3068157178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0282da5d0 0x55d0282e4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:46.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.968+0000 7f5129270640 1 --2- 172.21.15.116:0/3068157178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0282dbb40 0x55d0282e2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:46.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.968+0000 7f512a272640 1 -- 172.21.15.116:0/3068157178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0282da5d0 msgr2=0x55d0282e4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:46.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.968+0000 7f512a272640 1 --2- 172.21.15.116:0/3068157178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0282da5d0 0x55d0282e4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.968+0000 7f512a272640 1 -- 172.21.15.116:0/3068157178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0282dbb40 msgr2=0x55d0282e2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:46.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.968+0000 7f512a272640 1 --2- 172.21.15.116:0/3068157178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0282dbb40 0x55d0282e2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.968+0000 7f512a272640 1 -- 172.21.15.116:0/3068157178 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d027fb13c0 con 0x55d0282e5370 2024-09-12T03:15:46.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.969+0000 7f512a272640 1 --2- 172.21.15.116:0/3068157178 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0282e5370 0x55d0282e2600 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f511c0072f0 tx=0x7f511c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1932dc9dcf98fb2c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:46.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.969+0000 7f510d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:15:46.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.969+0000 7f510d7fa640 1 -- 172.21.15.116:0/3068157178 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c018070 con 0x55d0282e5370 2024-09-12T03:15:46.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.969+0000 7f510d7fa640 1 -- 172.21.15.116:0/3068157178 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f511c01f400 con 0x55d0282e5370 2024-09-12T03:15:46.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.969+0000 7f512e0ae740 1 -- 172.21.15.116:0/3068157178 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5120015070 con 0x55d0282e5370 2024-09-12T03:15:46.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.969+0000 7f510d7fa640 1 -- 172.21.15.116:0/3068157178 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f511c01f600 con 0x55d0282e5370 2024-09-12T03:15:46.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.969+0000 7f512e0ae740 10 client.18822 fetch_fsmap learned FSMap version 25 2024-09-12T03:15:46.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.969+0000 7f512e0ae740 1 -- 172.21.15.116:0/3068157178 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d0282f3b70 con 0x55d0282e5370 2024-09-12T03:15:46.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.969+0000 7f510d7fa640 1 -- 172.21.15.116:0/3068157178 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5120015070 con 0x55d0282e5370 2024-09-12T03:15:46.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.969+0000 7f510d7fa640 1 -- 172.21.15.116:0/3068157178 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f511c027870 con 0x55d0282e5370 2024-09-12T03:15:46.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.969+0000 7f512e0ae740 10 client.18822 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:15:46.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.969+0000 7f512e0ae740 10 client.18822 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:15:46.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.969+0000 7f512e0ae740 20 client.18822 populate_metadata read hostname 'smithi116' 2024-09-12T03:15:46.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.969+0000 7f512e0ae740 1 --2- 172.21.15.116:0/3068157178 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55d0282f5030 0x55d0282f7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:46.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.969+0000 7f512e0ae740 4 client.18822 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:15:46.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.969+0000 7f512e0ae740 1 -- 172.21.15.116:0/3068157178 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d02824d860 con 0x55d0282f5030 2024-09-12T03:15:46.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.970+0000 7f5129a71640 1 --2- 172.21.15.116:0/3068157178 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55d0282f5030 0x55d0282f7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:46.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.970+0000 7f510d7fa640 10 client.18822 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:15:46.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.970+0000 7f5129a71640 1 --2- 172.21.15.116:0/3068157178 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55d0282f5030 0x55d0282f7440 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c861067f92627d62 server_cookie=5a5ae2119ff5f27c in_seq=0 out_seq=0 2024-09-12T03:15:46.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.971+0000 7f510d7fa640 1 -- 172.21.15.116:0/3068157178 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d02824d860 con 0x55d0282f5030 2024-09-12T03:15:46.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.971+0000 7f510d7fa640 10 client.18822 handle_command_reply: tid=0 2024-09-12T03:15:46.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.971+0000 7f512e0ae740 1 client.18822 shutdown 2024-09-12T03:15:46.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.971+0000 7f512e0ae740 1 -- 172.21.15.116:0/3068157178 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0282e5370 msgr2=0x55d0282e2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:46.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.971+0000 7f512e0ae740 1 --2- 172.21.15.116:0/3068157178 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0282e5370 0x55d0282e2600 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f511c0072f0 tx=0x7f511c004a40 comp rx=0 tx=0).stop 2024-09-12T03:15:46.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.971+0000 7f512e0ae740 1 -- 172.21.15.116:0/3068157178 shutdown_connections 2024-09-12T03:15:46.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.971+0000 7f512e0ae740 1 --2- 172.21.15.116:0/3068157178 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55d0282f5030 0x55d0282f7440 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.971+0000 7f512e0ae740 1 --2- 172.21.15.116:0/3068157178 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0282e5370 0x55d0282e2600 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.971+0000 7f512e0ae740 1 --2- 172.21.15.116:0/3068157178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0282dbb40 0x55d0282e2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.971+0000 7f510d7fa640 0 client.18822 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:15:46.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.971+0000 7f512e0ae740 1 --2- 172.21.15.116:0/3068157178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0282da5d0 0x55d0282e4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.971+0000 7f512e0ae740 1 -- 172.21.15.116:0/3068157178 >> 172.21.15.116:0/3068157178 conn(0x55d0282a3310 msgr2=0x55d0282dc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:46.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.972+0000 7f512e0ae740 1 -- 172.21.15.116:0/3068157178 shutdown_connections 2024-09-12T03:15:46.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.972+0000 7f512e0ae740 1 -- 172.21.15.116:0/3068157178 wait complete. 2024-09-12T03:15:46.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.972+0000 7f512e0ae740 20 client.18822 trim_cache size 0 max 16384 2024-09-12T03:15:46.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.984+0000 7f512e0ae740 1 Processor -- start 2024-09-12T03:15:46.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.984+0000 7f512e0ae740 1 -- start start 2024-09-12T03:15:46.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.984+0000 7f512e0ae740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0282da5d0 0x55d0282e3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:46.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.984+0000 7f512e0ae740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0282dbb40 0x55d0282dd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:46.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.984+0000 7f512e0ae740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0282e5370 0x55d0282dd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:46.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.984+0000 7f512e0ae740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f51241b1c00 con 0x55d0282e5370 2024-09-12T03:15:46.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.984+0000 7f512e0ae740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f51241af4d0 con 0x55d0282da5d0 2024-09-12T03:15:46.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.984+0000 7f512e0ae740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f51241ad340 con 0x55d0282dbb40 2024-09-12T03:15:46.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.985+0000 7f512a272640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0282e5370 0x55d0282dd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:46.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.985+0000 7f512a272640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0282e5370 0x55d0282dd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52776/0 (socket says 172.21.15.116:52776) 2024-09-12T03:15:46.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.985+0000 7f512a272640 1 -- 172.21.15.116:0/4209991480 learned_addr learned my addr 172.21.15.116:0/4209991480 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:46.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.985+0000 7f5129a71640 1 --2- 172.21.15.116:0/4209991480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0282da5d0 0x55d0282e3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:46.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.985+0000 7f5129270640 1 --2- 172.21.15.116:0/4209991480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0282dbb40 0x55d0282dd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:46.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.985+0000 7f512a272640 1 -- 172.21.15.116:0/4209991480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0282dbb40 msgr2=0x55d0282dd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:46.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.985+0000 7f512a272640 1 --2- 172.21.15.116:0/4209991480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0282dbb40 0x55d0282dd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.985+0000 7f512a272640 1 -- 172.21.15.116:0/4209991480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0282da5d0 msgr2=0x55d0282e3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:46.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.985+0000 7f512a272640 1 --2- 172.21.15.116:0/4209991480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0282da5d0 0x55d0282e3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.985+0000 7f512a272640 1 -- 172.21.15.116:0/4209991480 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0282e0560 con 0x55d0282e5370 2024-09-12T03:15:46.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.985+0000 7f5129270640 1 --2- 172.21.15.116:0/4209991480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0282dbb40 0x55d0282dd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:15:46.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.985+0000 7f512a272640 1 --2- 172.21.15.116:0/4209991480 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0282e5370 0x55d0282dd660 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f511c027c90 tx=0x7f511c0077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6ce8a65c8dc34 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:46.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.985+0000 7f510f7fe640 1 -- 172.21.15.116:0/4209991480 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c018070 con 0x55d0282e5370 2024-09-12T03:15:46.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.985+0000 7f510f7fe640 1 -- 172.21.15.116:0/4209991480 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f511c00fd50 con 0x55d0282e5370 2024-09-12T03:15:46.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.985+0000 7f510f7fe640 1 -- 172.21.15.116:0/4209991480 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c01fa00 con 0x55d0282e5370 2024-09-12T03:15:46.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.986+0000 7f512e0ae740 1 -- 172.21.15.116:0/4209991480 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0282e5370 msgr2=0x55d0282dd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:46.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.986+0000 7f512e0ae740 1 --2- 172.21.15.116:0/4209991480 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0282e5370 0x55d0282dd660 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f511c027c90 tx=0x7f511c0077a0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.986+0000 7f512e0ae740 1 -- 172.21.15.116:0/4209991480 shutdown_connections 2024-09-12T03:15:46.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.986+0000 7f512e0ae740 1 --2- 172.21.15.116:0/4209991480 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0282e5370 0x55d0282dd660 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.986+0000 7f512e0ae740 1 --2- 172.21.15.116:0/4209991480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0282dbb40 0x55d0282dd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.986+0000 7f512e0ae740 1 --2- 172.21.15.116:0/4209991480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0282da5d0 0x55d0282e3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.986+0000 7f512e0ae740 1 -- 172.21.15.116:0/4209991480 >> 172.21.15.116:0/4209991480 conn(0x55d0282a3310 msgr2=0x55d0282e3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:46.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.986+0000 7f512e0ae740 1 -- 172.21.15.116:0/4209991480 shutdown_connections 2024-09-12T03:15:46.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.986+0000 7f512e0ae740 1 -- 172.21.15.116:0/4209991480 wait complete. 2024-09-12T03:15:46.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.986+0000 7f512e0ae740 1 -- start start 2024-09-12T03:15:46.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.986+0000 7f512e0ae740 1 Processor -- start 2024-09-12T03:15:46.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.987+0000 7f512e0ae740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0282da5d0 0x55d0282ed480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:46.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.987+0000 7f512e0ae740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0282dbb40 0x55d0282ed9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:46.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.987+0000 7f512e0ae740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0282e5370 0x55d0282ea450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:46.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.987+0000 7f512e0ae740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f51241ad990 con 0x55d0282e5370 2024-09-12T03:15:46.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.987+0000 7f512e0ae740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f51241aee00 con 0x55d0282da5d0 2024-09-12T03:15:46.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.987+0000 7f512e0ae740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f51241ae060 con 0x55d0282dbb40 2024-09-12T03:15:46.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.987+0000 7f512e0ae740 10 client.0 _ll_register_callbacks cb 0x7f511c010ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:46.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.987+0000 7f512a272640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0282e5370 0x55d0282ea450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:46.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.987+0000 7f512a272640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0282e5370 0x55d0282ea450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52792/0 (socket says 172.21.15.116:52792) 2024-09-12T03:15:46.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.987+0000 7f512a272640 1 -- 172.21.15.116:0/1275733347 learned_addr learned my addr 172.21.15.116:0/1275733347 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:46.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.987+0000 7f5129a71640 1 --2- 172.21.15.116:0/1275733347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0282da5d0 0x55d0282ed480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:46.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.987+0000 7f5129270640 1 --2- 172.21.15.116:0/1275733347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0282dbb40 0x55d0282ed9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:46.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.987+0000 7f512a272640 1 -- 172.21.15.116:0/1275733347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0282dbb40 msgr2=0x55d0282ed9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:46.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.987+0000 7f512a272640 1 --2- 172.21.15.116:0/1275733347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0282dbb40 0x55d0282ed9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.987+0000 7f512a272640 1 -- 172.21.15.116:0/1275733347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0282da5d0 msgr2=0x55d0282ed480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:46.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.987+0000 7f512a272640 1 --2- 172.21.15.116:0/1275733347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0282da5d0 0x55d0282ed480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.987+0000 7f512a272640 1 -- 172.21.15.116:0/1275733347 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0282eaa40 con 0x55d0282e5370 2024-09-12T03:15:46.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.987+0000 7f512a272640 1 --2- 172.21.15.116:0/1275733347 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0282e5370 0x55d0282ea450 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f511c0108c0 tx=0x7f511c012360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e18c9fbe2aeb89 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:46.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.987+0000 7f510d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:15:46.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.988+0000 7f510d7fa640 1 -- 172.21.15.116:0/1275733347 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c018040 con 0x55d0282e5370 2024-09-12T03:15:46.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.988+0000 7f510d7fa640 1 -- 172.21.15.116:0/1275733347 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f511c027610 con 0x55d0282e5370 2024-09-12T03:15:46.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.988+0000 7f512e0ae740 1 -- 172.21.15.116:0/1275733347 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f511c00fd50 con 0x55d0282e5370 2024-09-12T03:15:46.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.988+0000 7f510d7fa640 1 -- 172.21.15.116:0/1275733347 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f511c00f810 con 0x55d0282e5370 2024-09-12T03:15:46.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.988+0000 7f512e0ae740 10 client.18828 fetch_fsmap learned FSMap version 25 2024-09-12T03:15:46.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.988+0000 7f512e0ae740 1 -- 172.21.15.116:0/1275733347 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d0283605b0 con 0x55d0282e5370 2024-09-12T03:15:46.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.988+0000 7f510d7fa640 1 -- 172.21.15.116:0/1275733347 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f511c00fd50 con 0x55d0282e5370 2024-09-12T03:15:46.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.988+0000 7f510d7fa640 1 -- 172.21.15.116:0/1275733347 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f511c03a020 con 0x55d0282e5370 2024-09-12T03:15:46.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.988+0000 7f512e0ae740 10 client.18828 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:15:46.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.988+0000 7f512e0ae740 10 client.18828 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:15:46.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.988+0000 7f512e0ae740 20 client.18828 populate_metadata read hostname 'smithi116' 2024-09-12T03:15:46.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.988+0000 7f512e0ae740 1 --2- 172.21.15.116:0/1275733347 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55d028360c00 0x55d028362ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:46.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.988+0000 7f512e0ae740 4 client.18828 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:15:46.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.988+0000 7f512e0ae740 1 -- 172.21.15.116:0/1275733347 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d028247d70 con 0x55d028360c00 2024-09-12T03:15:46.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.988+0000 7f5129a71640 1 --2- 172.21.15.116:0/1275733347 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55d028360c00 0x55d028362ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:46.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.989+0000 7f5129a71640 1 --2- 172.21.15.116:0/1275733347 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55d028360c00 0x55d028362ff0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e360eb945293cb4 server_cookie=c780e46fa02a73dc in_seq=0 out_seq=0 2024-09-12T03:15:46.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.989+0000 7f510d7fa640 10 client.18828 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:15:46.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.989+0000 7f510d7fa640 1 -- 172.21.15.116:0/1275733347 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d028247d70 con 0x55d028360c00 2024-09-12T03:15:46.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.989+0000 7f510d7fa640 10 client.18828 handle_command_reply: tid=0 2024-09-12T03:15:46.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.989+0000 7f512e0ae740 1 client.18828 shutdown 2024-09-12T03:15:46.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.989+0000 7f512e0ae740 1 -- 172.21.15.116:0/1275733347 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0282e5370 msgr2=0x55d0282ea450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:46.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.989+0000 7f512e0ae740 1 --2- 172.21.15.116:0/1275733347 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0282e5370 0x55d0282ea450 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f511c0108c0 tx=0x7f511c012360 comp rx=0 tx=0).stop 2024-09-12T03:15:46.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.989+0000 7f512e0ae740 1 -- 172.21.15.116:0/1275733347 shutdown_connections 2024-09-12T03:15:46.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.989+0000 7f512e0ae740 1 --2- 172.21.15.116:0/1275733347 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55d028360c00 0x55d028362ff0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.989+0000 7f512e0ae740 1 --2- 172.21.15.116:0/1275733347 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0282e5370 0x55d0282ea450 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.989+0000 7f512e0ae740 1 --2- 172.21.15.116:0/1275733347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0282dbb40 0x55d0282ed9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.989+0000 7f510d7fa640 0 client.18828 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:15:46.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.989+0000 7f512e0ae740 1 --2- 172.21.15.116:0/1275733347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0282da5d0 0x55d0282ed480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.989+0000 7f512e0ae740 1 -- 172.21.15.116:0/1275733347 >> 172.21.15.116:0/1275733347 conn(0x55d0282a3310 msgr2=0x55d0282e4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:46.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.989+0000 7f512e0ae740 1 -- 172.21.15.116:0/1275733347 shutdown_connections 2024-09-12T03:15:46.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.989+0000 7f512e0ae740 1 -- 172.21.15.116:0/1275733347 wait complete. 2024-09-12T03:15:46.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.990+0000 7f512e0ae740 20 client.18828 trim_cache size 0 max 16384 2024-09-12T03:15:46.997 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:15:46.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.990+0000 7f512bcd5640 1 -- 172.21.15.116:0/2957831627 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f51100a2a50 msgr2=0x7f51100a4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:46.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.990+0000 7f512bcd5640 1 --2- 172.21.15.116:0/2957831627 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f51100a2a50 0x7f51100a4f10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f511c012850 tx=0x7f511c004ab0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.990+0000 7f512bcd5640 1 -- 172.21.15.116:0/2957831627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5124101420 msgr2=0x7f51241975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:46.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.990+0000 7f512bcd5640 1 --2- 172.21.15.116:0/2957831627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5124101420 0x7f51241975a0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f5118009230 tx=0x7f5118004270 comp rx=0 tx=0).stop 2024-09-12T03:15:46.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.990+0000 7f512bcd5640 1 -- 172.21.15.116:0/2957831627 shutdown_connections 2024-09-12T03:15:46.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.990+0000 7f512bcd5640 1 --2- 172.21.15.116:0/2957831627 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f51100a2a50 0x7f51100a4f10 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.990+0000 7f512bcd5640 1 --2- 172.21.15.116:0/2957831627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5124106a80 0x7f5124197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.990+0000 7f512bcd5640 1 --2- 172.21.15.116:0/2957831627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5124101420 0x7f51241975a0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.990+0000 7f512bcd5640 1 --2- 172.21.15.116:0/2957831627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5124100a80 0x7f5124197060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:46.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.990+0000 7f512bcd5640 1 -- 172.21.15.116:0/2957831627 >> 172.21.15.116:0/2957831627 conn(0x7f51240fc7a0 msgr2=0x7f51240fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:46.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.990+0000 7f512bcd5640 1 -- 172.21.15.116:0/2957831627 shutdown_connections 2024-09-12T03:15:46.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:46.990+0000 7f512bcd5640 1 -- 172.21.15.116:0/2957831627 wait complete. 2024-09-12T03:15:47.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:47 smithi183 ceph-mon[35535]: pgmap v722: 97 pgs: 97 active+clean; 6.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:15:47.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:47 smithi177 ceph-mon[35305]: pgmap v722: 97 pgs: 97 active+clean; 6.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:15:47.785 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:47 smithi116 ceph-mon[31410]: pgmap v722: 97 pgs: 97 active+clean; 6.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:15:47.813 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:15:48.058 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:15:48.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.561+0000 7ff603b2f640 1 -- 172.21.15.116:0/1813972843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5fc100480 msgr2=0x7ff5fc100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:48.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.561+0000 7ff603b2f640 1 --2- 172.21.15.116:0/1813972843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5fc100480 0x7ff5fc100860 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009900 tx=0x7ff5f802a410 comp rx=0 tx=0).stop 2024-09-12T03:15:48.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.561+0000 7ff603b2f640 1 -- 172.21.15.116:0/1813972843 shutdown_connections 2024-09-12T03:15:48.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.561+0000 7ff603b2f640 1 --2- 172.21.15.116:0/1813972843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5fc1055b0 0x7ff5fc10d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.561+0000 7ff603b2f640 1 --2- 172.21.15.116:0/1813972843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff5fc100e30 0x7ff5fc104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.561+0000 7ff603b2f640 1 --2- 172.21.15.116:0/1813972843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5fc100480 0x7ff5fc100860 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.561+0000 7ff603b2f640 1 -- 172.21.15.116:0/1813972843 >> 172.21.15.116:0/1813972843 conn(0x7ff5fc077210 msgr2=0x7ff5fc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:48.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.561+0000 7ff603b2f640 1 -- 172.21.15.116:0/1813972843 shutdown_connections 2024-09-12T03:15:48.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.561+0000 7ff603b2f640 1 -- 172.21.15.116:0/1813972843 wait complete. 2024-09-12T03:15:48.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.562+0000 7ff603b2f640 1 Processor -- start 2024-09-12T03:15:48.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.562+0000 7ff603b2f640 1 -- start start 2024-09-12T03:15:48.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.562+0000 7ff603b2f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff5fc100480 0x7ff5fc198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:48.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.562+0000 7ff603b2f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5fc100e30 0x7ff5fc1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:48.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.562+0000 7ff603b2f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5fc1055b0 0x7ff5fc19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:48.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.562+0000 7ff603b2f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc1105b0 con 0x7ff5fc1055b0 2024-09-12T03:15:48.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.562+0000 7ff603b2f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc110430 con 0x7ff5fc100480 2024-09-12T03:15:48.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.562+0000 7ff603b2f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc110730 con 0x7ff5fc100e30 2024-09-12T03:15:48.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.562+0000 7ff6020cc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5fc1055b0 0x7ff5fc19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:48.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.562+0000 7ff6020cc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5fc1055b0 0x7ff5fc19cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52812/0 (socket says 172.21.15.116:52812) 2024-09-12T03:15:48.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.562+0000 7ff6020cc640 1 -- 172.21.15.116:0/2049608841 learned_addr learned my addr 172.21.15.116:0/2049608841 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:48.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.562+0000 7ff6018cb640 1 --2- 172.21.15.116:0/2049608841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff5fc100480 0x7ff5fc198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:48.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.562+0000 7ff6010ca640 1 --2- 172.21.15.116:0/2049608841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5fc100e30 0x7ff5fc1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:48.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.563+0000 7ff6020cc640 1 -- 172.21.15.116:0/2049608841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5fc100e30 msgr2=0x7ff5fc1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:48.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.563+0000 7ff6020cc640 1 --2- 172.21.15.116:0/2049608841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5fc100e30 0x7ff5fc1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.563+0000 7ff6020cc640 1 -- 172.21.15.116:0/2049608841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff5fc100480 msgr2=0x7ff5fc198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:48.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.563+0000 7ff6020cc640 1 --2- 172.21.15.116:0/2049608841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff5fc100480 0x7ff5fc198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.563+0000 7ff6020cc640 1 -- 172.21.15.116:0/2049608841 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5fc19d420 con 0x7ff5fc1055b0 2024-09-12T03:15:48.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.563+0000 7ff6020cc640 1 --2- 172.21.15.116:0/2049608841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5fc1055b0 0x7ff5fc19cd40 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400cad0 tx=0x7ff5f400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:48.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.563+0000 7ff5f2ffd640 1 -- 172.21.15.116:0/2049608841 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f400ae20 con 0x7ff5fc1055b0 2024-09-12T03:15:48.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.563+0000 7ff603b2f640 1 -- 172.21.15.116:0/2049608841 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5fc19d710 con 0x7ff5fc1055b0 2024-09-12T03:15:48.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.563+0000 7ff5f2ffd640 1 -- 172.21.15.116:0/2049608841 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5f4005020 con 0x7ff5fc1055b0 2024-09-12T03:15:48.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.563+0000 7ff5f2ffd640 1 -- 172.21.15.116:0/2049608841 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f400f6f0 con 0x7ff5fc1055b0 2024-09-12T03:15:48.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.563+0000 7ff603b2f640 1 -- 172.21.15.116:0/2049608841 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5fc1a4ff0 con 0x7ff5fc1055b0 2024-09-12T03:15:48.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.565+0000 7ff5f2ffd640 1 -- 172.21.15.116:0/2049608841 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ff5f40051d0 con 0x7ff5fc1055b0 2024-09-12T03:15:48.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.566+0000 7ff605f08740 1 Processor -- start 2024-09-12T03:15:48.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.566+0000 7ff605f08740 1 -- start start 2024-09-12T03:15:48.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.566+0000 7ff605f08740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601dd99eb40 0x5601dd99d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:48.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.566+0000 7ff605f08740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5601dd99d5d0 0x5601dd99da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:48.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.566+0000 7ff605f08740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5601dd9a8370 0x5601dd9aa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:48.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.566+0000 7ff605f08740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5601dd9032f0 con 0x5601dd99d5d0 2024-09-12T03:15:48.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.566+0000 7ff605f08740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5601dd8aacf0 con 0x5601dd9a8370 2024-09-12T03:15:48.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.566+0000 7ff605f08740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5601dd8ff2b0 con 0x5601dd99eb40 2024-09-12T03:15:48.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.566+0000 7ff6018cb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5601dd99d5d0 0x5601dd99da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:48.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.566+0000 7ff6018cb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5601dd99d5d0 0x5601dd99da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52818/0 (socket says 172.21.15.116:52818) 2024-09-12T03:15:48.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.566+0000 7ff6018cb640 1 -- 172.21.15.116:0/3231097820 learned_addr learned my addr 172.21.15.116:0/3231097820 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:48.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.566+0000 7ff5f2ffd640 1 --2- 172.21.15.116:0/2049608841 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff5e80a2b80 0x7ff5e80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:48.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.566+0000 7ff6010ca640 1 --2- 172.21.15.116:0/3231097820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601dd99eb40 0x5601dd99d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:48.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.566+0000 7ff6020cc640 1 --2- 172.21.15.116:0/2049608841 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff5e80a2b80 0x7ff5e80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:48.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.566+0000 7ff6010ca640 1 --2- 172.21.15.116:0/3231097820 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5601dd9a8370 0x5601dd9aa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:48.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.567+0000 7ff6018cb640 1 -- 172.21.15.116:0/3231097820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601dd99eb40 msgr2=0x5601dd99d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:48.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.567+0000 7ff6018cb640 1 --2- 172.21.15.116:0/3231097820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601dd99eb40 0x5601dd99d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.567+0000 7ff6018cb640 1 -- 172.21.15.116:0/3231097820 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5601dd9a8370 msgr2=0x5601dd9aa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:48.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.567+0000 7ff6018cb640 1 --2- 172.21.15.116:0/3231097820 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5601dd9a8370 0x5601dd9aa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.567+0000 7ff6018cb640 1 -- 172.21.15.116:0/3231097820 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601dd88f3a0 con 0x5601dd99d5d0 2024-09-12T03:15:48.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.567+0000 7ff5f2ffd640 1 -- 172.21.15.116:0/2049608841 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff5f40cefb0 con 0x7ff5fc1055b0 2024-09-12T03:15:48.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.567+0000 7ff6018cb640 1 --2- 172.21.15.116:0/3231097820 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5601dd99d5d0 0x5601dd99da10 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009900 tx=0x7ff5f8031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c147b8e7125cfe85 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:48.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.567+0000 7ff6020cc640 1 --2- 172.21.15.116:0/2049608841 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff5e80a2b80 0x7ff5e80a5040 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400a6e0 tx=0x7ff5f406a810 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:48.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.567+0000 7ff5e7fff640 1 -- 172.21.15.116:0/3231097820 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8040070 con 0x5601dd99d5d0 2024-09-12T03:15:48.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.567+0000 7ff5e7fff640 1 -- 172.21.15.116:0/3231097820 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5f8005020 con 0x5601dd99d5d0 2024-09-12T03:15:48.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.567+0000 7ff5e7fff640 1 -- 172.21.15.116:0/3231097820 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8035040 con 0x5601dd99d5d0 2024-09-12T03:15:48.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.568+0000 7ff605f08740 1 -- 172.21.15.116:0/3231097820 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5601dd99d5d0 msgr2=0x5601dd99da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:48.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.568+0000 7ff605f08740 1 --2- 172.21.15.116:0/3231097820 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5601dd99d5d0 0x5601dd99da10 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009900 tx=0x7ff5f8031630 comp rx=0 tx=0).stop 2024-09-12T03:15:48.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.568+0000 7ff605f08740 1 -- 172.21.15.116:0/3231097820 shutdown_connections 2024-09-12T03:15:48.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.568+0000 7ff605f08740 1 --2- 172.21.15.116:0/3231097820 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5601dd9a8370 0x5601dd9aa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.568+0000 7ff605f08740 1 --2- 172.21.15.116:0/3231097820 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5601dd99d5d0 0x5601dd99da10 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.568+0000 7ff605f08740 1 --2- 172.21.15.116:0/3231097820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601dd99eb40 0x5601dd99d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.568+0000 7ff605f08740 1 -- 172.21.15.116:0/3231097820 >> 172.21.15.116:0/3231097820 conn(0x5601dd966310 msgr2=0x5601dd999c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:48.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.568+0000 7ff605f08740 1 -- 172.21.15.116:0/3231097820 shutdown_connections 2024-09-12T03:15:48.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.568+0000 7ff605f08740 1 -- 172.21.15.116:0/3231097820 wait complete. 2024-09-12T03:15:48.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.568+0000 7ff605f08740 1 -- start start 2024-09-12T03:15:48.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.568+0000 7ff605f08740 1 Processor -- start 2024-09-12T03:15:48.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.569+0000 7ff605f08740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601dd99d5d0 0x5601dd9a7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.569+0000 7ff605f08740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5601dd99eb40 0x5601dd9a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.569+0000 7ff605f08740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5601dd9a8370 0x5601dd9a5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.569+0000 7ff605f08740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5601dd9ab010 con 0x5601dd99eb40 2024-09-12T03:15:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.569+0000 7ff605f08740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5601dd8be060 con 0x5601dd9a8370 2024-09-12T03:15:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.569+0000 7ff605f08740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5601dd9ab190 con 0x5601dd99d5d0 2024-09-12T03:15:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.569+0000 7ff605f08740 10 client.0 _ll_register_callbacks cb 0x5601dd6118c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.569+0000 7ff6018cb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5601dd99eb40 0x5601dd9a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.569+0000 7ff6018cb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5601dd99eb40 0x5601dd9a4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52828/0 (socket says 172.21.15.116:52828) 2024-09-12T03:15:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.569+0000 7ff6018cb640 1 -- 172.21.15.116:0/2700472742 learned_addr learned my addr 172.21.15.116:0/2700472742 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.569+0000 7ff6010ca640 1 --2- 172.21.15.116:0/2700472742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601dd99d5d0 0x5601dd9a7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.569+0000 7ff6010ca640 1 --2- 172.21.15.116:0/2700472742 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5601dd9a8370 0x5601dd9a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.569+0000 7ff6018cb640 1 -- 172.21.15.116:0/2700472742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601dd99d5d0 msgr2=0x5601dd9a7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.569+0000 7ff6018cb640 1 --2- 172.21.15.116:0/2700472742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601dd99d5d0 0x5601dd9a7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.569+0000 7ff6018cb640 1 -- 172.21.15.116:0/2700472742 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5601dd9a8370 msgr2=0x5601dd9a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.569+0000 7ff6018cb640 1 --2- 172.21.15.116:0/2700472742 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5601dd9a8370 0x5601dd9a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.569+0000 7ff6018cb640 1 -- 172.21.15.116:0/2700472742 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601dd6743c0 con 0x5601dd99eb40 2024-09-12T03:15:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.569+0000 7ff6010ca640 1 --2- 172.21.15.116:0/2700472742 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5601dd9a8370 0x5601dd9a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:15:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.569+0000 7ff6018cb640 1 --2- 172.21.15.116:0/2700472742 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5601dd99eb40 0x5601dd9a4fd0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8031ba0 tx=0x7ff5f8005480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35b7a8588277670 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.569+0000 7ff5e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:15:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.569+0000 7ff5e57fa640 1 -- 172.21.15.116:0/2700472742 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f804c020 con 0x5601dd99eb40 2024-09-12T03:15:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.569+0000 7ff5e57fa640 1 -- 172.21.15.116:0/2700472742 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff5f8004cb0 con 0x5601dd99eb40 2024-09-12T03:15:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.569+0000 7ff605f08740 1 -- 172.21.15.116:0/2700472742 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5f8005020 con 0x5601dd99eb40 2024-09-12T03:15:48.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.570+0000 7ff5e57fa640 1 -- 172.21.15.116:0/2700472742 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff5f8040070 con 0x5601dd99eb40 2024-09-12T03:15:48.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.570+0000 7ff605f08740 10 client.18840 fetch_fsmap learned FSMap version 25 2024-09-12T03:15:48.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.570+0000 7ff605f08740 1 -- 172.21.15.116:0/2700472742 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5601dd9b6a60 con 0x5601dd99eb40 2024-09-12T03:15:48.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.570+0000 7ff5e57fa640 1 -- 172.21.15.116:0/2700472742 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5f8005020 con 0x5601dd99eb40 2024-09-12T03:15:48.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.570+0000 7ff5e57fa640 1 -- 172.21.15.116:0/2700472742 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff5f8044400 con 0x5601dd99eb40 2024-09-12T03:15:48.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.570+0000 7ff605f08740 10 client.18840 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:15:48.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.570+0000 7ff605f08740 10 client.18840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:15:48.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.570+0000 7ff605f08740 20 client.18840 populate_metadata read hostname 'smithi116' 2024-09-12T03:15:48.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.570+0000 7ff605f08740 1 --2- 172.21.15.116:0/2700472742 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5601dd9b7db0 0x5601dd9ba1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:48.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.570+0000 7ff605f08740 4 client.18840 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:15:48.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.570+0000 7ff605f08740 1 -- 172.21.15.116:0/2700472742 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601dd910860 con 0x5601dd9b7db0 2024-09-12T03:15:48.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.570+0000 7ff6010ca640 1 --2- 172.21.15.116:0/2700472742 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5601dd9b7db0 0x5601dd9ba1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:48.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.571+0000 7ff5e57fa640 10 client.18840 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:48.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.571+0000 7ff6010ca640 1 --2- 172.21.15.116:0/2700472742 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5601dd9b7db0 0x5601dd9ba1a0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bbf463a263afcff server_cookie=4da3a3b1ae127c0b in_seq=0 out_seq=0 2024-09-12T03:15:48.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.571+0000 7ff5e57fa640 1 -- 172.21.15.116:0/2700472742 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5601dd910860 con 0x5601dd9b7db0 2024-09-12T03:15:48.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.571+0000 7ff5e57fa640 10 client.18840 handle_command_reply: tid=0 2024-09-12T03:15:48.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.571+0000 7ff605f08740 1 client.18840 shutdown 2024-09-12T03:15:48.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.572+0000 7ff605f08740 1 -- 172.21.15.116:0/2700472742 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5601dd99eb40 msgr2=0x5601dd9a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:48.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.572+0000 7ff605f08740 1 --2- 172.21.15.116:0/2700472742 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5601dd99eb40 0x5601dd9a4fd0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8031ba0 tx=0x7ff5f8005480 comp rx=0 tx=0).stop 2024-09-12T03:15:48.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.572+0000 7ff605f08740 1 -- 172.21.15.116:0/2700472742 shutdown_connections 2024-09-12T03:15:48.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.572+0000 7ff605f08740 1 --2- 172.21.15.116:0/2700472742 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5601dd9b7db0 0x5601dd9ba1a0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.572+0000 7ff605f08740 1 --2- 172.21.15.116:0/2700472742 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5601dd9a8370 0x5601dd9a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.572+0000 7ff605f08740 1 --2- 172.21.15.116:0/2700472742 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5601dd99eb40 0x5601dd9a4fd0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.572+0000 7ff605f08740 1 --2- 172.21.15.116:0/2700472742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601dd99d5d0 0x5601dd9a7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.572+0000 7ff605f08740 1 -- 172.21.15.116:0/2700472742 >> 172.21.15.116:0/2700472742 conn(0x5601dd966310 msgr2=0x5601dd99f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:48.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.572+0000 7ff5e57fa640 0 client.18840 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:48.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.572+0000 7ff605f08740 1 -- 172.21.15.116:0/2700472742 shutdown_connections 2024-09-12T03:15:48.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.572+0000 7ff605f08740 1 -- 172.21.15.116:0/2700472742 wait complete. 2024-09-12T03:15:48.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.573+0000 7ff605f08740 20 client.18840 trim_cache size 0 max 16384 2024-09-12T03:15:48.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.586+0000 7ff605f08740 1 Processor -- start 2024-09-12T03:15:48.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.586+0000 7ff605f08740 1 -- start start 2024-09-12T03:15:48.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.587+0000 7ff605f08740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5601dd99d5d0 0x5601dd9b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:48.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.587+0000 7ff605f08740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601dd99eb40 0x5601dd9a4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:48.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.587+0000 7ff605f08740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5601dd9a8370 0x5601dd9a9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:48.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.587+0000 7ff605f08740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc1afe10 con 0x5601dd99d5d0 2024-09-12T03:15:48.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.587+0000 7ff605f08740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc1ad6e0 con 0x5601dd9a8370 2024-09-12T03:15:48.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.587+0000 7ff605f08740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc1ab530 con 0x5601dd99eb40 2024-09-12T03:15:48.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.587+0000 7ff6010ca640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5601dd99d5d0 0x5601dd9b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:48.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.587+0000 7ff6010ca640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5601dd99d5d0 0x5601dd9b42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52842/0 (socket says 172.21.15.116:52842) 2024-09-12T03:15:48.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.587+0000 7ff6010ca640 1 -- 172.21.15.116:0/3439766273 learned_addr learned my addr 172.21.15.116:0/3439766273 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:48.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.587+0000 7ff6018cb640 1 --2- 172.21.15.116:0/3439766273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601dd99eb40 0x5601dd9a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:48.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.587+0000 7ff6010ca640 1 -- 172.21.15.116:0/3439766273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601dd99eb40 msgr2=0x5601dd9a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:48.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.587+0000 7ff6010ca640 1 --2- 172.21.15.116:0/3439766273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601dd99eb40 0x5601dd9a4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.587+0000 7ff6010ca640 1 -- 172.21.15.116:0/3439766273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5601dd9a8370 msgr2=0x5601dd9a9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:48.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.587+0000 7ff6010ca640 1 --2- 172.21.15.116:0/3439766273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5601dd9a8370 0x5601dd9a9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.587+0000 7ff6010ca640 1 -- 172.21.15.116:0/3439766273 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601dd9a3520 con 0x5601dd99d5d0 2024-09-12T03:15:48.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.587+0000 7ff6010ca640 1 --2- 172.21.15.116:0/3439766273 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5601dd99d5d0 0x5601dd9b42b0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec00b610 tx=0x7ff5ec011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b793e903c45599 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:48.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.587+0000 7ff5e77fe640 1 -- 172.21.15.116:0/3439766273 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec022070 con 0x5601dd99d5d0 2024-09-12T03:15:48.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.587+0000 7ff5e77fe640 1 -- 172.21.15.116:0/3439766273 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5ec0047a0 con 0x5601dd99d5d0 2024-09-12T03:15:48.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.587+0000 7ff5e77fe640 1 -- 172.21.15.116:0/3439766273 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec01e810 con 0x5601dd99d5d0 2024-09-12T03:15:48.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.588+0000 7ff605f08740 1 -- 172.21.15.116:0/3439766273 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5601dd99d5d0 msgr2=0x5601dd9b42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:48.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.588+0000 7ff605f08740 1 --2- 172.21.15.116:0/3439766273 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5601dd99d5d0 0x5601dd9b42b0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec00b610 tx=0x7ff5ec011810 comp rx=0 tx=0).stop 2024-09-12T03:15:48.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.588+0000 7ff605f08740 1 -- 172.21.15.116:0/3439766273 shutdown_connections 2024-09-12T03:15:48.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.588+0000 7ff605f08740 1 --2- 172.21.15.116:0/3439766273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5601dd9a8370 0x5601dd9a9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.588+0000 7ff605f08740 1 --2- 172.21.15.116:0/3439766273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601dd99eb40 0x5601dd9a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.588+0000 7ff605f08740 1 --2- 172.21.15.116:0/3439766273 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5601dd99d5d0 0x5601dd9b42b0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.588+0000 7ff605f08740 1 -- 172.21.15.116:0/3439766273 >> 172.21.15.116:0/3439766273 conn(0x5601dd966310 msgr2=0x5601dd9a38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:48.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.588+0000 7ff605f08740 1 -- 172.21.15.116:0/3439766273 shutdown_connections 2024-09-12T03:15:48.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.588+0000 7ff605f08740 1 -- 172.21.15.116:0/3439766273 wait complete. 2024-09-12T03:15:48.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.589+0000 7ff605f08740 1 -- start start 2024-09-12T03:15:48.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.589+0000 7ff605f08740 1 Processor -- start 2024-09-12T03:15:48.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.589+0000 7ff605f08740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601dd99d5d0 0x5601dd9b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:48.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.589+0000 7ff605f08740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5601dd99eb40 0x5601dd9a9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:48.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.589+0000 7ff605f08740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5601dd9a8370 0x5601dd9a9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:48.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.589+0000 7ff605f08740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc1abba0 con 0x5601dd99eb40 2024-09-12T03:15:48.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.589+0000 7ff605f08740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc1ad010 con 0x5601dd9a8370 2024-09-12T03:15:48.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.589+0000 7ff605f08740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc1ac270 con 0x5601dd99d5d0 2024-09-12T03:15:48.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.589+0000 7ff605f08740 10 client.0 _ll_register_callbacks cb 0x7ff5f8009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:48.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.589+0000 7ff6018cb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5601dd99eb40 0x5601dd9a9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:48.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.589+0000 7ff6018cb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5601dd99eb40 0x5601dd9a9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52850/0 (socket says 172.21.15.116:52850) 2024-09-12T03:15:48.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.589+0000 7ff6018cb640 1 -- 172.21.15.116:0/4168117683 learned_addr learned my addr 172.21.15.116:0/4168117683 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:48.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.589+0000 7ff6010ca640 1 --2- 172.21.15.116:0/4168117683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601dd99d5d0 0x5601dd9b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:48.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.589+0000 7ff6018cb640 1 -- 172.21.15.116:0/4168117683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601dd99d5d0 msgr2=0x5601dd9b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:48.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.590+0000 7ff6018cb640 1 --2- 172.21.15.116:0/4168117683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601dd99d5d0 0x5601dd9b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.590+0000 7ff6010ca640 1 --2- 172.21.15.116:0/4168117683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5601dd9a8370 0x5601dd9a9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:48.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.590+0000 7ff6018cb640 1 -- 172.21.15.116:0/4168117683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5601dd9a8370 msgr2=0x5601dd9a9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:48.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.590+0000 7ff6018cb640 1 --2- 172.21.15.116:0/4168117683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5601dd9a8370 0x5601dd9a9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.590+0000 7ff6018cb640 1 -- 172.21.15.116:0/4168117683 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601dd9aa000 con 0x5601dd99eb40 2024-09-12T03:15:48.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.590+0000 7ff6010ca640 1 --2- 172.21.15.116:0/4168117683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5601dd9a8370 0x5601dd9a9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:15:48.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.590+0000 7ff6018cb640 1 --2- 172.21.15.116:0/4168117683 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5601dd99eb40 0x5601dd9a9500 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7ff5f802a930 tx=0x7ff5f8044da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cadc4a44ddb22785 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:48.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.590+0000 7ff5e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:15:48.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.590+0000 7ff5e57fa640 1 -- 172.21.15.116:0/4168117683 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8044e60 con 0x5601dd99eb40 2024-09-12T03:15:48.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.590+0000 7ff5e57fa640 1 -- 172.21.15.116:0/4168117683 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff5f8004ab0 con 0x5601dd99eb40 2024-09-12T03:15:48.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.590+0000 7ff605f08740 1 -- 172.21.15.116:0/4168117683 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5ec0047a0 con 0x5601dd99eb40 2024-09-12T03:15:48.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.590+0000 7ff5e57fa640 1 -- 172.21.15.116:0/4168117683 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff5f804c040 con 0x5601dd99eb40 2024-09-12T03:15:48.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.590+0000 7ff605f08740 10 client.18846 fetch_fsmap learned FSMap version 25 2024-09-12T03:15:48.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.590+0000 7ff605f08740 1 -- 172.21.15.116:0/4168117683 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5601dd9af230 con 0x5601dd99eb40 2024-09-12T03:15:48.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.590+0000 7ff5e57fa640 1 -- 172.21.15.116:0/4168117683 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5ec0047a0 con 0x5601dd99eb40 2024-09-12T03:15:48.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.590+0000 7ff5e57fa640 1 -- 172.21.15.116:0/4168117683 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff5f803c830 con 0x5601dd99eb40 2024-09-12T03:15:48.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.590+0000 7ff605f08740 10 client.18846 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:15:48.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.590+0000 7ff605f08740 10 client.18846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:15:48.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.590+0000 7ff605f08740 20 client.18846 populate_metadata read hostname 'smithi116' 2024-09-12T03:15:48.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.590+0000 7ff605f08740 1 --2- 172.21.15.116:0/4168117683 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5601dda22b70 0x5601dda24f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:48.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.590+0000 7ff605f08740 4 client.18846 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:15:48.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.590+0000 7ff605f08740 1 -- 172.21.15.116:0/4168117683 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5601dd90ad70 con 0x5601dda22b70 2024-09-12T03:15:48.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.591+0000 7ff6010ca640 1 --2- 172.21.15.116:0/4168117683 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5601dda22b70 0x5601dda24f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:48.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.591+0000 7ff6010ca640 1 --2- 172.21.15.116:0/4168117683 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5601dda22b70 0x5601dda24f90 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ecbcc4e9076919d server_cookie=11ad00b9a36175e7 in_seq=0 out_seq=0 2024-09-12T03:15:48.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.591+0000 7ff5e57fa640 10 client.18846 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:48.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.591+0000 7ff5e57fa640 1 -- 172.21.15.116:0/4168117683 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5601dd90ad70 con 0x5601dda22b70 2024-09-12T03:15:48.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.591+0000 7ff5e57fa640 10 client.18846 handle_command_reply: tid=0 2024-09-12T03:15:48.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.591+0000 7ff605f08740 1 client.18846 shutdown 2024-09-12T03:15:48.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.591+0000 7ff605f08740 1 -- 172.21.15.116:0/4168117683 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5601dd99eb40 msgr2=0x5601dd9a9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:48.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.591+0000 7ff605f08740 1 --2- 172.21.15.116:0/4168117683 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5601dd99eb40 0x5601dd9a9500 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7ff5f802a930 tx=0x7ff5f8044da0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.591+0000 7ff605f08740 1 -- 172.21.15.116:0/4168117683 shutdown_connections 2024-09-12T03:15:48.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.591+0000 7ff605f08740 1 --2- 172.21.15.116:0/4168117683 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5601dda22b70 0x5601dda24f90 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.591+0000 7ff605f08740 1 --2- 172.21.15.116:0/4168117683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5601dd9a8370 0x5601dd9a9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.591+0000 7ff605f08740 1 --2- 172.21.15.116:0/4168117683 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5601dd99eb40 0x5601dd9a9500 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.591+0000 7ff5e57fa640 0 client.18846 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:48.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.591+0000 7ff605f08740 1 --2- 172.21.15.116:0/4168117683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601dd99d5d0 0x5601dd9b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.591+0000 7ff605f08740 1 -- 172.21.15.116:0/4168117683 >> 172.21.15.116:0/4168117683 conn(0x5601dd966310 msgr2=0x5601dd9a7050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:48.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.592+0000 7ff605f08740 1 -- 172.21.15.116:0/4168117683 shutdown_connections 2024-09-12T03:15:48.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.592+0000 7ff605f08740 1 -- 172.21.15.116:0/4168117683 wait complete. 2024-09-12T03:15:48.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.592+0000 7ff605f08740 20 client.18846 trim_cache size 0 max 16384 2024-09-12T03:15:48.598 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:15:48.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.592+0000 7ff603b2f640 1 -- 172.21.15.116:0/2049608841 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff5e80a2b80 msgr2=0x7ff5e80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:48.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.592+0000 7ff603b2f640 1 --2- 172.21.15.116:0/2049608841 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff5e80a2b80 0x7ff5e80a5040 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400a6e0 tx=0x7ff5f406a810 comp rx=0 tx=0).stop 2024-09-12T03:15:48.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.592+0000 7ff603b2f640 1 -- 172.21.15.116:0/2049608841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5fc1055b0 msgr2=0x7ff5fc19cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:48.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.592+0000 7ff603b2f640 1 --2- 172.21.15.116:0/2049608841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5fc1055b0 0x7ff5fc19cd40 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400cad0 tx=0x7ff5f400cf90 comp rx=0 tx=0).stop 2024-09-12T03:15:48.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.592+0000 7ff603b2f640 1 -- 172.21.15.116:0/2049608841 shutdown_connections 2024-09-12T03:15:48.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.592+0000 7ff603b2f640 1 --2- 172.21.15.116:0/2049608841 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff5e80a2b80 0x7ff5e80a5040 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.592+0000 7ff603b2f640 1 --2- 172.21.15.116:0/2049608841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5fc1055b0 0x7ff5fc19cd40 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.592+0000 7ff603b2f640 1 --2- 172.21.15.116:0/2049608841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5fc100e30 0x7ff5fc1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.592+0000 7ff603b2f640 1 --2- 172.21.15.116:0/2049608841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff5fc100480 0x7ff5fc198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:48.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.592+0000 7ff603b2f640 1 -- 172.21.15.116:0/2049608841 >> 172.21.15.116:0/2049608841 conn(0x7ff5fc077210 msgr2=0x7ff5fc0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:48.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.592+0000 7ff603b2f640 1 -- 172.21.15.116:0/2049608841 shutdown_connections 2024-09-12T03:15:48.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:48.592+0000 7ff603b2f640 1 -- 172.21.15.116:0/2049608841 wait complete. 2024-09-12T03:15:49.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:49 smithi183 ceph-mon[35535]: pgmap v723: 97 pgs: 97 active+clean; 6.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-12T03:15:49.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:49 smithi177 ceph-mon[35305]: pgmap v723: 97 pgs: 97 active+clean; 6.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-12T03:15:49.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:49 smithi116 ceph-mon[31410]: pgmap v723: 97 pgs: 97 active+clean; 6.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-12T03:15:50.442 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:15:50.442 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:15:50.699 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:15:51.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.197+0000 7f9eb250e640 1 -- 172.21.15.116:0/912475231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9eac100e30 msgr2=0x7f9eac104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:51.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.197+0000 7f9eb250e640 1 --2- 172.21.15.116:0/912475231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9eac100e30 0x7f9eac104e80 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009900 tx=0x7f9e9402a410 comp rx=0 tx=0).stop 2024-09-12T03:15:51.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.197+0000 7f9eb250e640 1 -- 172.21.15.116:0/912475231 shutdown_connections 2024-09-12T03:15:51.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.197+0000 7f9eb250e640 1 --2- 172.21.15.116:0/912475231 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9eac1055b0 0x7f9eac10d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.197+0000 7f9eb250e640 1 --2- 172.21.15.116:0/912475231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9eac100e30 0x7f9eac104e80 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.197+0000 7f9eb250e640 1 --2- 172.21.15.116:0/912475231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9eac100480 0x7f9eac100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.197+0000 7f9eb250e640 1 -- 172.21.15.116:0/912475231 >> 172.21.15.116:0/912475231 conn(0x7f9eac077210 msgr2=0x7f9eac0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:51.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.198+0000 7f9eb250e640 1 -- 172.21.15.116:0/912475231 shutdown_connections 2024-09-12T03:15:51.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.198+0000 7f9eb250e640 1 -- 172.21.15.116:0/912475231 wait complete. 2024-09-12T03:15:51.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.198+0000 7f9eb250e640 1 Processor -- start 2024-09-12T03:15:51.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.198+0000 7f9eb250e640 1 -- start start 2024-09-12T03:15:51.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.198+0000 7f9eb250e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9eac100480 0x7f9eac198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.198+0000 7f9eb250e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9eac100e30 0x7f9eac198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.198+0000 7f9eb250e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9eac1055b0 0x7f9eac19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.198+0000 7f9eb250e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac111f90 con 0x7f9eac100480 2024-09-12T03:15:51.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.198+0000 7f9eb250e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac111e10 con 0x7f9eac1055b0 2024-09-12T03:15:51.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.198+0000 7f9eb250e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac112110 con 0x7f9eac100e30 2024-09-12T03:15:51.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.198+0000 7f9eabfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9eac100480 0x7f9eac198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.198+0000 7f9eabfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9eac100480 0x7f9eac198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48616/0 (socket says 172.21.15.116:48616) 2024-09-12T03:15:51.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.199+0000 7f9eabfff640 1 -- 172.21.15.116:0/2917762735 learned_addr learned my addr 172.21.15.116:0/2917762735 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:51.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.199+0000 7f9eb0aab640 1 --2- 172.21.15.116:0/2917762735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9eac1055b0 0x7f9eac19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.199+0000 7f9eabfff640 1 -- 172.21.15.116:0/2917762735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9eac100e30 msgr2=0x7f9eac198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:51.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.199+0000 7f9eabfff640 1 --2- 172.21.15.116:0/2917762735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9eac100e30 0x7f9eac198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.199+0000 7f9eabfff640 1 -- 172.21.15.116:0/2917762735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9eac1055b0 msgr2=0x7f9eac19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:51.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.199+0000 7f9eabfff640 1 --2- 172.21.15.116:0/2917762735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9eac1055b0 0x7f9eac19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.199+0000 7f9eabfff640 1 -- 172.21.15.116:0/2917762735 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eac19d400 con 0x7f9eac100480 2024-09-12T03:15:51.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.199+0000 7f9eabfff640 1 --2- 172.21.15.116:0/2917762735 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9eac100480 0x7f9eac198450 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00d770 tx=0x7f9e9c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:51.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.199+0000 7f9ea9ffb640 1 -- 172.21.15.116:0/2917762735 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c017070 con 0x7f9eac100480 2024-09-12T03:15:51.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.199+0000 7f9ea9ffb640 1 -- 172.21.15.116:0/2917762735 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e9c005500 con 0x7f9eac100480 2024-09-12T03:15:51.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.199+0000 7f9ea9ffb640 1 -- 172.21.15.116:0/2917762735 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c004e00 con 0x7f9eac100480 2024-09-12T03:15:51.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.199+0000 7f9eb250e640 1 -- 172.21.15.116:0/2917762735 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eac19d6f0 con 0x7f9eac100480 2024-09-12T03:15:51.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.199+0000 7f9eb250e640 1 -- 172.21.15.116:0/2917762735 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eac1a4fd0 con 0x7f9eac100480 2024-09-12T03:15:51.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.201+0000 7f9ea9ffb640 1 -- 172.21.15.116:0/2917762735 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f9e9c004fa0 con 0x7f9eac100480 2024-09-12T03:15:51.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.202+0000 7f9eb48e7740 1 Processor -- start 2024-09-12T03:15:51.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.202+0000 7f9eb48e7740 1 -- start start 2024-09-12T03:15:51.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.202+0000 7f9eb48e7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc18d30a40 0x55bc18d30e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.202+0000 7f9eb48e7740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc18d31360 0x55bc18d3fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.202+0000 7f9eb48e7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc18d40150 0x55bc18d42540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.202+0000 7f9eb48e7740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bc18c96280 con 0x55bc18d31360 2024-09-12T03:15:51.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.202+0000 7f9eb48e7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bc18c3db10 con 0x55bc18d30a40 2024-09-12T03:15:51.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.202+0000 7f9eb48e7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bc18c92240 con 0x55bc18d40150 2024-09-12T03:15:51.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.202+0000 7f9eabfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc18d31360 0x55bc18d3fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.202+0000 7f9ea9ffb640 1 --2- 172.21.15.116:0/2917762735 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9e980a2b80 0x7f9e980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.202+0000 7f9eabfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc18d31360 0x55bc18d3fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48630/0 (socket says 172.21.15.116:48630) 2024-09-12T03:15:51.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.202+0000 7f9eabfff640 1 -- 172.21.15.116:0/633359607 learned_addr learned my addr 172.21.15.116:0/633359607 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:51.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.202+0000 7f9eb0aab640 1 --2- 172.21.15.116:0/2917762735 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9e980a2b80 0x7f9e980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.202+0000 7f9eb0aab640 1 --2- 172.21.15.116:0/633359607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc18d30a40 0x55bc18d30e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.202+0000 7f9ea3fff640 1 --2- 172.21.15.116:0/633359607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc18d40150 0x55bc18d42540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.203+0000 7f9ea9ffb640 1 -- 172.21.15.116:0/2917762735 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f9e9c0ceb40 con 0x7f9eac100480 2024-09-12T03:15:51.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.203+0000 7f9eabfff640 1 -- 172.21.15.116:0/633359607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc18d40150 msgr2=0x55bc18d42540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:51.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.203+0000 7f9eabfff640 1 --2- 172.21.15.116:0/633359607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc18d40150 0x55bc18d42540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.203+0000 7f9eabfff640 1 -- 172.21.15.116:0/633359607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc18d30a40 msgr2=0x55bc18d30e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:51.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.203+0000 7f9eabfff640 1 --2- 172.21.15.116:0/633359607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc18d30a40 0x55bc18d30e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.203+0000 7f9eabfff640 1 -- 172.21.15.116:0/633359607 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc18c4a1f0 con 0x55bc18d31360 2024-09-12T03:15:51.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.203+0000 7f9eb0aab640 1 --2- 172.21.15.116:0/633359607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc18d30a40 0x55bc18d30e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:15:51.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.203+0000 7f9eabfff640 1 --2- 172.21.15.116:0/633359607 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc18d31360 0x55bc18d3fc10 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00d740 tx=0x7f9e9c0048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e03330310084c6c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:51.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.203+0000 7f9eb0aab640 1 --2- 172.21.15.116:0/2917762735 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9e980a2b80 0x7f9e980a5040 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4005c40 tx=0x7f9ea40093e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:51.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.203+0000 7f9ea27fc640 1 -- 172.21.15.116:0/633359607 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c020030 con 0x55bc18d31360 2024-09-12T03:15:51.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.203+0000 7f9ea27fc640 1 -- 172.21.15.116:0/633359607 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e9c09f780 con 0x55bc18d31360 2024-09-12T03:15:51.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.203+0000 7f9ea27fc640 1 -- 172.21.15.116:0/633359607 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c009de0 con 0x55bc18d31360 2024-09-12T03:15:51.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.204+0000 7f9eb48e7740 1 -- 172.21.15.116:0/633359607 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc18d31360 msgr2=0x55bc18d3fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:51.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.204+0000 7f9eb48e7740 1 --2- 172.21.15.116:0/633359607 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc18d31360 0x55bc18d3fc10 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00d740 tx=0x7f9e9c0048a0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.204+0000 7f9eb48e7740 1 -- 172.21.15.116:0/633359607 shutdown_connections 2024-09-12T03:15:51.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.204+0000 7f9eb48e7740 1 --2- 172.21.15.116:0/633359607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc18d40150 0x55bc18d42540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.204+0000 7f9eb48e7740 1 --2- 172.21.15.116:0/633359607 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc18d31360 0x55bc18d3fc10 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.204+0000 7f9eb48e7740 1 --2- 172.21.15.116:0/633359607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc18d30a40 0x55bc18d30e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.204+0000 7f9eb48e7740 1 -- 172.21.15.116:0/633359607 >> 172.21.15.116:0/633359607 conn(0x55bc18d2d0f0 msgr2=0x55bc18d2d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:51.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.204+0000 7f9eb48e7740 1 -- 172.21.15.116:0/633359607 shutdown_connections 2024-09-12T03:15:51.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.204+0000 7f9eb48e7740 1 -- 172.21.15.116:0/633359607 wait complete. 2024-09-12T03:15:51.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.204+0000 7f9eb48e7740 1 -- start start 2024-09-12T03:15:51.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.204+0000 7f9eb48e7740 1 Processor -- start 2024-09-12T03:15:51.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.205+0000 7f9eb48e7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc18d30a40 0x55bc18d3d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.205+0000 7f9eb48e7740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc18d31360 0x55bc18d3d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.205+0000 7f9eb48e7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc18d40150 0x55bc18d3db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.205+0000 7f9eb48e7740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bc18d42d20 con 0x55bc18d31360 2024-09-12T03:15:51.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.205+0000 7f9eb48e7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bc18c50f10 con 0x55bc18d30a40 2024-09-12T03:15:51.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.205+0000 7f9eb48e7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bc18d42ea0 con 0x55bc18d40150 2024-09-12T03:15:51.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.205+0000 7f9eb48e7740 10 client.0 _ll_register_callbacks cb 0x55bc18c19a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:51.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.205+0000 7f9ea3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc18d31360 0x55bc18d3d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.205+0000 7f9ea3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc18d31360 0x55bc18d3d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48642/0 (socket says 172.21.15.116:48642) 2024-09-12T03:15:51.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.205+0000 7f9ea3fff640 1 -- 172.21.15.116:0/861879634 learned_addr learned my addr 172.21.15.116:0/861879634 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:51.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.205+0000 7f9eabfff640 1 --2- 172.21.15.116:0/861879634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc18d30a40 0x55bc18d3d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.205+0000 7f9eb0aab640 1 --2- 172.21.15.116:0/861879634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc18d40150 0x55bc18d3db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.205+0000 7f9ea3fff640 1 -- 172.21.15.116:0/861879634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc18d40150 msgr2=0x55bc18d3db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:51.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.205+0000 7f9ea3fff640 1 --2- 172.21.15.116:0/861879634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc18d40150 0x55bc18d3db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.205+0000 7f9ea3fff640 1 -- 172.21.15.116:0/861879634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc18d30a40 msgr2=0x55bc18d3d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:51.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.205+0000 7f9ea3fff640 1 --2- 172.21.15.116:0/861879634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc18d30a40 0x55bc18d3d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.205+0000 7f9ea3fff640 1 -- 172.21.15.116:0/861879634 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc18a076a0 con 0x55bc18d31360 2024-09-12T03:15:51.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.205+0000 7f9eb0aab640 1 --2- 172.21.15.116:0/861879634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc18d40150 0x55bc18d3db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:15:51.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.205+0000 7f9ea3fff640 1 --2- 172.21.15.116:0/861879634 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc18d31360 0x55bc18d3d600 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f9e94033000 tx=0x7f9e940317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4baa40c9fe98dde server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:51.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.205+0000 7f9e8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:15:51.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.205+0000 7f9e8b7fe640 1 -- 172.21.15.116:0/861879634 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e94040070 con 0x55bc18d31360 2024-09-12T03:15:51.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.205+0000 7f9e8b7fe640 1 -- 172.21.15.116:0/861879634 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f9e94004250 con 0x55bc18d31360 2024-09-12T03:15:51.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.205+0000 7f9eb48e7740 1 -- 172.21.15.116:0/861879634 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e9c09f780 con 0x55bc18d31360 2024-09-12T03:15:51.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.205+0000 7f9e8b7fe640 1 -- 172.21.15.116:0/861879634 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9e9404c050 con 0x55bc18d31360 2024-09-12T03:15:51.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.205+0000 7f9eb48e7740 10 client.18858 fetch_fsmap learned FSMap version 25 2024-09-12T03:15:51.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.205+0000 7f9eb48e7740 1 -- 172.21.15.116:0/861879634 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bc18d4e9e0 con 0x55bc18d31360 2024-09-12T03:15:51.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.206+0000 7f9e8b7fe640 1 -- 172.21.15.116:0/861879634 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e9c09f780 con 0x55bc18d31360 2024-09-12T03:15:51.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.206+0000 7f9e8b7fe640 1 -- 172.21.15.116:0/861879634 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9e9403a040 con 0x55bc18d31360 2024-09-12T03:15:51.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.206+0000 7f9eb48e7740 10 client.18858 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:15:51.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.206+0000 7f9eb48e7740 10 client.18858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:15:51.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.206+0000 7f9eb48e7740 20 client.18858 populate_metadata read hostname 'smithi116' 2024-09-12T03:15:51.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.206+0000 7f9eb48e7740 1 --2- 172.21.15.116:0/861879634 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bc18d4fe40 0x55bc18d522a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.206+0000 7f9eb48e7740 4 client.18858 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:15:51.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.206+0000 7f9eb48e7740 1 -- 172.21.15.116:0/861879634 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc18ca37f0 con 0x55bc18d4fe40 2024-09-12T03:15:51.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.207+0000 7f9eabfff640 1 --2- 172.21.15.116:0/861879634 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bc18d4fe40 0x55bc18d522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.207+0000 7f9eabfff640 1 --2- 172.21.15.116:0/861879634 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bc18d4fe40 0x55bc18d522a0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9a0b5ccc57eedcc server_cookie=4a73b368ba41a8ac in_seq=0 out_seq=0 2024-09-12T03:15:51.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.207+0000 7f9e8b7fe640 10 client.18858 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:51.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.208+0000 7f9e8b7fe640 1 -- 172.21.15.116:0/861879634 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc18ca37f0 con 0x55bc18d4fe40 2024-09-12T03:15:51.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.208+0000 7f9e8b7fe640 10 client.18858 handle_command_reply: tid=0 2024-09-12T03:15:51.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.208+0000 7f9eb48e7740 1 client.18858 shutdown 2024-09-12T03:15:51.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.208+0000 7f9eb48e7740 1 -- 172.21.15.116:0/861879634 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc18d31360 msgr2=0x55bc18d3d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:51.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.208+0000 7f9eb48e7740 1 --2- 172.21.15.116:0/861879634 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc18d31360 0x55bc18d3d600 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f9e94033000 tx=0x7f9e940317e0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.208+0000 7f9eb48e7740 1 -- 172.21.15.116:0/861879634 shutdown_connections 2024-09-12T03:15:51.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.208+0000 7f9eb48e7740 1 --2- 172.21.15.116:0/861879634 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bc18d4fe40 0x55bc18d522a0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.208+0000 7f9eb48e7740 1 --2- 172.21.15.116:0/861879634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc18d40150 0x55bc18d3db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.208+0000 7f9eb48e7740 1 --2- 172.21.15.116:0/861879634 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc18d31360 0x55bc18d3d600 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.208+0000 7f9eb48e7740 1 --2- 172.21.15.116:0/861879634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc18d30a40 0x55bc18d3d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.208+0000 7f9e8b7fe640 0 client.18858 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:51.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.208+0000 7f9eb48e7740 1 -- 172.21.15.116:0/861879634 >> 172.21.15.116:0/861879634 conn(0x55bc18d2d0f0 msgr2=0x55bc18d32e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:51.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.208+0000 7f9eb48e7740 1 -- 172.21.15.116:0/861879634 shutdown_connections 2024-09-12T03:15:51.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.208+0000 7f9eb48e7740 1 -- 172.21.15.116:0/861879634 wait complete. 2024-09-12T03:15:51.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.208+0000 7f9eb48e7740 20 client.18858 trim_cache size 0 max 16384 2024-09-12T03:15:51.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.225+0000 7f9eb48e7740 1 Processor -- start 2024-09-12T03:15:51.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.225+0000 7f9eb48e7740 1 -- start start 2024-09-12T03:15:51.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.225+0000 7f9eb48e7740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc18d30a40 0x55bc18d3ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.225+0000 7f9eb48e7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc18d31360 0x55bc18d57b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.225+0000 7f9eb48e7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc18d40150 0x55bc18d580b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.225+0000 7f9eb48e7740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1adbb0 con 0x55bc18d30a40 2024-09-12T03:15:51.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.225+0000 7f9eb48e7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1ab480 con 0x55bc18d31360 2024-09-12T03:15:51.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.225+0000 7f9eb48e7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1a92d0 con 0x55bc18d40150 2024-09-12T03:15:51.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.225+0000 7f9eabfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc18d30a40 0x55bc18d3ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.225+0000 7f9eabfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc18d30a40 0x55bc18d3ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48646/0 (socket says 172.21.15.116:48646) 2024-09-12T03:15:51.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.225+0000 7f9eabfff640 1 -- 172.21.15.116:0/1984608464 learned_addr learned my addr 172.21.15.116:0/1984608464 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:51.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.226+0000 7f9ea3fff640 1 --2- 172.21.15.116:0/1984608464 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc18d31360 0x55bc18d57b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.226+0000 7f9eb0aab640 1 --2- 172.21.15.116:0/1984608464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc18d40150 0x55bc18d580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.226+0000 7f9eabfff640 1 -- 172.21.15.116:0/1984608464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc18d40150 msgr2=0x55bc18d580b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:51.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.226+0000 7f9eabfff640 1 --2- 172.21.15.116:0/1984608464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc18d40150 0x55bc18d580b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.226+0000 7f9eabfff640 1 -- 172.21.15.116:0/1984608464 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc18d31360 msgr2=0x55bc18d57b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:51.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.226+0000 7f9eabfff640 1 --2- 172.21.15.116:0/1984608464 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc18d31360 0x55bc18d57b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.226+0000 7f9eabfff640 1 -- 172.21.15.116:0/1984608464 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc18d2cb90 con 0x55bc18d30a40 2024-09-12T03:15:51.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.226+0000 7f9ea3fff640 1 --2- 172.21.15.116:0/1984608464 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc18d31360 0x55bc18d57b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:15:51.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.226+0000 7f9eb0aab640 1 --2- 172.21.15.116:0/1984608464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc18d40150 0x55bc18d580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:15:51.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.226+0000 7f9eabfff640 1 --2- 172.21.15.116:0/1984608464 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc18d30a40 0x55bc18d3ef20 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c09b0d0 tx=0x7f9e9c097040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e712bd97e2731373 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:51.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.226+0000 7f9ea1ffb640 1 -- 172.21.15.116:0/1984608464 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c020040 con 0x55bc18d30a40 2024-09-12T03:15:51.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.226+0000 7f9ea1ffb640 1 -- 172.21.15.116:0/1984608464 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e9c005290 con 0x55bc18d30a40 2024-09-12T03:15:51.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.226+0000 7f9ea1ffb640 1 -- 172.21.15.116:0/1984608464 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c0a3bb0 con 0x55bc18d30a40 2024-09-12T03:15:51.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.227+0000 7f9eb48e7740 1 -- 172.21.15.116:0/1984608464 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc18d30a40 msgr2=0x55bc18d3ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:51.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.227+0000 7f9eb48e7740 1 --2- 172.21.15.116:0/1984608464 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc18d30a40 0x55bc18d3ef20 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c09b0d0 tx=0x7f9e9c097040 comp rx=0 tx=0).stop 2024-09-12T03:15:51.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.227+0000 7f9eb48e7740 1 -- 172.21.15.116:0/1984608464 shutdown_connections 2024-09-12T03:15:51.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.227+0000 7f9eb48e7740 1 --2- 172.21.15.116:0/1984608464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc18d40150 0x55bc18d580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.227+0000 7f9eb48e7740 1 --2- 172.21.15.116:0/1984608464 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc18d31360 0x55bc18d57b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.227+0000 7f9eb48e7740 1 --2- 172.21.15.116:0/1984608464 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc18d30a40 0x55bc18d3ef20 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.227+0000 7f9eb48e7740 1 -- 172.21.15.116:0/1984608464 >> 172.21.15.116:0/1984608464 conn(0x55bc18d2d0f0 msgr2=0x55bc18d3e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:51.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.227+0000 7f9eb48e7740 1 -- 172.21.15.116:0/1984608464 shutdown_connections 2024-09-12T03:15:51.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.227+0000 7f9eb48e7740 1 -- 172.21.15.116:0/1984608464 wait complete. 2024-09-12T03:15:51.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.227+0000 7f9eb48e7740 1 -- start start 2024-09-12T03:15:51.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.227+0000 7f9eb48e7740 1 Processor -- start 2024-09-12T03:15:51.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.228+0000 7f9eb48e7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc18d30a40 0x55bc18d440d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.228+0000 7f9eb48e7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc18d31360 0x55bc18d44610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.228+0000 7f9eb48e7740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc18d40150 0x55bc18d44b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.228+0000 7f9eb48e7740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1a9940 con 0x55bc18d40150 2024-09-12T03:15:51.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.228+0000 7f9eb48e7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1aadb0 con 0x55bc18d31360 2024-09-12T03:15:51.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.228+0000 7f9eb48e7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1aa010 con 0x55bc18d30a40 2024-09-12T03:15:51.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.228+0000 7f9eb48e7740 10 client.0 _ll_register_callbacks cb 0x7f9e94009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:51.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.228+0000 7f9eb0aab640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc18d40150 0x55bc18d44b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.228+0000 7f9eb0aab640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc18d40150 0x55bc18d44b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48654/0 (socket says 172.21.15.116:48654) 2024-09-12T03:15:51.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.228+0000 7f9eb0aab640 1 -- 172.21.15.116:0/2968716016 learned_addr learned my addr 172.21.15.116:0/2968716016 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:51.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.228+0000 7f9eabfff640 1 --2- 172.21.15.116:0/2968716016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc18d30a40 0x55bc18d440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.228+0000 7f9ea3fff640 1 --2- 172.21.15.116:0/2968716016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc18d31360 0x55bc18d44610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.228+0000 7f9eb0aab640 1 -- 172.21.15.116:0/2968716016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc18d30a40 msgr2=0x55bc18d440d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:51.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.228+0000 7f9eb0aab640 1 --2- 172.21.15.116:0/2968716016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc18d30a40 0x55bc18d440d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.228+0000 7f9eb0aab640 1 -- 172.21.15.116:0/2968716016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc18d31360 msgr2=0x55bc18d44610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:51.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.228+0000 7f9eb0aab640 1 --2- 172.21.15.116:0/2968716016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc18d31360 0x55bc18d44610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.228+0000 7f9eb0aab640 1 -- 172.21.15.116:0/2968716016 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc18d28830 con 0x55bc18d40150 2024-09-12T03:15:51.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.228+0000 7f9ea3fff640 1 --2- 172.21.15.116:0/2968716016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc18d31360 0x55bc18d44610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:15:51.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.228+0000 7f9eb0aab640 1 --2- 172.21.15.116:0/2968716016 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc18d40150 0x55bc18d44b50 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4014580 tx=0x7f9ea4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892ca8a5013786f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:51.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.228+0000 7f9e8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:15:51.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.229+0000 7f9e8b7fe640 1 -- 172.21.15.116:0/2968716016 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4024070 con 0x55bc18d40150 2024-09-12T03:15:51.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.229+0000 7f9e8b7fe640 1 -- 172.21.15.116:0/2968716016 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f9ea4011af0 con 0x55bc18d40150 2024-09-12T03:15:51.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.229+0000 7f9eb48e7740 1 -- 172.21.15.116:0/2968716016 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e9c005290 con 0x55bc18d40150 2024-09-12T03:15:51.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.229+0000 7f9e8b7fe640 1 -- 172.21.15.116:0/2968716016 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9ea4011cf0 con 0x55bc18d40150 2024-09-12T03:15:51.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.229+0000 7f9eb48e7740 10 client.18864 fetch_fsmap learned FSMap version 25 2024-09-12T03:15:51.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.229+0000 7f9eb48e7740 1 -- 172.21.15.116:0/2968716016 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bc18d2e280 con 0x55bc18d40150 2024-09-12T03:15:51.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.229+0000 7f9e8b7fe640 1 -- 172.21.15.116:0/2968716016 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e9c005290 con 0x55bc18d40150 2024-09-12T03:15:51.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.229+0000 7f9e8b7fe640 1 -- 172.21.15.116:0/2968716016 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9ea401d040 con 0x55bc18d40150 2024-09-12T03:15:51.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.229+0000 7f9eb48e7740 10 client.18864 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:15:51.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.229+0000 7f9eb48e7740 10 client.18864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:15:51.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.229+0000 7f9eb48e7740 20 client.18864 populate_metadata read hostname 'smithi116' 2024-09-12T03:15:51.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.229+0000 7f9eb48e7740 1 --2- 172.21.15.116:0/2968716016 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bc18dbb460 0x55bc18dbd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.229+0000 7f9eb48e7740 4 client.18864 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:15:51.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.229+0000 7f9eb48e7740 1 -- 172.21.15.116:0/2968716016 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bc18c9dd00 con 0x55bc18dbb460 2024-09-12T03:15:51.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.230+0000 7f9eabfff640 1 --2- 172.21.15.116:0/2968716016 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bc18dbb460 0x55bc18dbd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.230+0000 7f9e8b7fe640 10 client.18864 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:51.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.230+0000 7f9eabfff640 1 --2- 172.21.15.116:0/2968716016 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bc18dbb460 0x55bc18dbd850 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=773839e6bbe3b44f server_cookie=d0f1fed03b5d08e in_seq=0 out_seq=0 2024-09-12T03:15:51.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.231+0000 7f9e8b7fe640 1 -- 172.21.15.116:0/2968716016 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bc18c9dd00 con 0x55bc18dbb460 2024-09-12T03:15:51.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.231+0000 7f9e8b7fe640 10 client.18864 handle_command_reply: tid=0 2024-09-12T03:15:51.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.231+0000 7f9eb48e7740 1 client.18864 shutdown 2024-09-12T03:15:51.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.231+0000 7f9eb48e7740 1 -- 172.21.15.116:0/2968716016 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc18d40150 msgr2=0x55bc18d44b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:51.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.231+0000 7f9eb48e7740 1 --2- 172.21.15.116:0/2968716016 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc18d40150 0x55bc18d44b50 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4014580 tx=0x7f9ea4004a40 comp rx=0 tx=0).stop 2024-09-12T03:15:51.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.231+0000 7f9eb48e7740 1 -- 172.21.15.116:0/2968716016 shutdown_connections 2024-09-12T03:15:51.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.231+0000 7f9eb48e7740 1 --2- 172.21.15.116:0/2968716016 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bc18dbb460 0x55bc18dbd850 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.231+0000 7f9eb48e7740 1 --2- 172.21.15.116:0/2968716016 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bc18d40150 0x55bc18d44b50 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.231+0000 7f9eb48e7740 1 --2- 172.21.15.116:0/2968716016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bc18d31360 0x55bc18d44610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.231+0000 7f9eb48e7740 1 --2- 172.21.15.116:0/2968716016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc18d30a40 0x55bc18d440d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.231+0000 7f9eb48e7740 1 -- 172.21.15.116:0/2968716016 >> 172.21.15.116:0/2968716016 conn(0x55bc18d2d0f0 msgr2=0x55bc18d32d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:51.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.231+0000 7f9e8b7fe640 0 client.18864 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:51.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.231+0000 7f9eb48e7740 1 -- 172.21.15.116:0/2968716016 shutdown_connections 2024-09-12T03:15:51.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.231+0000 7f9eb48e7740 1 -- 172.21.15.116:0/2968716016 wait complete. 2024-09-12T03:15:51.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.231+0000 7f9eb48e7740 20 client.18864 trim_cache size 0 max 16384 2024-09-12T03:15:51.237 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:15:51.237 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:15:51.237 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "798d81d3-a1c4-45de-8ac9-c1253a43f18f", 2024-09-12T03:15:51.237 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:15:51.237 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:15:51.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.232+0000 7f9eb250e640 1 -- 172.21.15.116:0/2917762735 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9e980a2b80 msgr2=0x7f9e980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:51.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.232+0000 7f9eb250e640 1 --2- 172.21.15.116:0/2917762735 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9e980a2b80 0x7f9e980a5040 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4005c40 tx=0x7f9ea40093e0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.232+0000 7f9eb250e640 1 -- 172.21.15.116:0/2917762735 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9eac100480 msgr2=0x7f9eac198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:51.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.232+0000 7f9eb250e640 1 --2- 172.21.15.116:0/2917762735 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9eac100480 0x7f9eac198450 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00d770 tx=0x7f9e9c00dc30 comp rx=0 tx=0).stop 2024-09-12T03:15:51.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.232+0000 7f9eb250e640 1 -- 172.21.15.116:0/2917762735 shutdown_connections 2024-09-12T03:15:51.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.232+0000 7f9eb250e640 1 --2- 172.21.15.116:0/2917762735 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9e980a2b80 0x7f9e980a5040 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.232+0000 7f9eb250e640 1 --2- 172.21.15.116:0/2917762735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9eac1055b0 0x7f9eac19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.232+0000 7f9eb250e640 1 --2- 172.21.15.116:0/2917762735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9eac100e30 0x7f9eac198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.238 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.232+0000 7f9eb250e640 1 --2- 172.21.15.116:0/2917762735 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9eac100480 0x7f9eac198450 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.232+0000 7f9eb250e640 1 -- 172.21.15.116:0/2917762735 >> 172.21.15.116:0/2917762735 conn(0x7f9eac077210 msgr2=0x7f9eac103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:51.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.232+0000 7f9eb250e640 1 -- 172.21.15.116:0/2917762735 shutdown_connections 2024-09-12T03:15:51.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:51.232+0000 7f9eb250e640 1 -- 172.21.15.116:0/2917762735 wait complete. 2024-09-12T03:15:51.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:51 smithi183 ceph-mon[35535]: pgmap v724: 97 pgs: 97 active+clean; 6.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:15:51.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:51 smithi177 ceph-mon[35305]: pgmap v724: 97 pgs: 97 active+clean; 6.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:15:51.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:51 smithi116 ceph-mon[31410]: pgmap v724: 97 pgs: 97 active+clean; 6.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:15:52.156 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:15:52.406 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:15:52.695 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:52 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:15:52.695 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:52 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:15:52.695 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:52 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:15:52.696 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:52 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:15:52.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:52 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:15:52.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:52 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:15:52.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:52 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:15:52.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:52 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:15:52.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:52 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:15:52.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:52 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:15:52.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:52 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:15:52.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:52 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:15:52.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.960+0000 7f3f88f62640 1 -- 172.21.15.116:0/1170166071 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f8410c170 msgr2=0x7f3f8410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:52.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.960+0000 7f3f88f62640 1 --2- 172.21.15.116:0/1170166071 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f8410c170 0x7f3f8410e610 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c009ca0 tx=0x7f3f7c02a950 comp rx=0 tx=0).stop 2024-09-12T03:15:52.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.961+0000 7f3f88f62640 1 -- 172.21.15.116:0/1170166071 shutdown_connections 2024-09-12T03:15:52.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.961+0000 7f3f88f62640 1 --2- 172.21.15.116:0/1170166071 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f8410c170 0x7f3f8410e610 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:52.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.961+0000 7f3f88f62640 1 --2- 172.21.15.116:0/1170166071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f84103660 0x7f3f8410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:52.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.961+0000 7f3f88f62640 1 --2- 172.21.15.116:0/1170166071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f84102d40 0x7f3f84103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:52.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.961+0000 7f3f88f62640 1 -- 172.21.15.116:0/1170166071 >> 172.21.15.116:0/1170166071 conn(0x7f3f840fc7a0 msgr2=0x7f3f840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:52.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.961+0000 7f3f88f62640 1 -- 172.21.15.116:0/1170166071 shutdown_connections 2024-09-12T03:15:52.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.961+0000 7f3f88f62640 1 -- 172.21.15.116:0/1170166071 wait complete. 2024-09-12T03:15:52.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.961+0000 7f3f88f62640 1 Processor -- start 2024-09-12T03:15:52.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.962+0000 7f3f88f62640 1 -- start start 2024-09-12T03:15:52.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.962+0000 7f3f88f62640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f84102d40 0x7f3f841009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:52.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.962+0000 7f3f88f62640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f84103660 0x7f3f84100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:52.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.962+0000 7f3f88f62640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f8410c170 0x7f3f84101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:52.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.962+0000 7f3f88f62640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3f84112030 con 0x7f3f8410c170 2024-09-12T03:15:52.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.962+0000 7f3f88f62640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3f84111eb0 con 0x7f3f84102d40 2024-09-12T03:15:52.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.962+0000 7f3f88f62640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841121b0 con 0x7f3f84103660 2024-09-12T03:15:52.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.962+0000 7f3f82d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f8410c170 0x7f3f84101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:52.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.962+0000 7f3f81d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f84103660 0x7f3f84100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:52.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.962+0000 7f3f82d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f8410c170 0x7f3f84101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48676/0 (socket says 172.21.15.116:48676) 2024-09-12T03:15:52.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.962+0000 7f3f82d9d640 1 -- 172.21.15.116:0/1119218274 learned_addr learned my addr 172.21.15.116:0/1119218274 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:52.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.962+0000 7f3f8259c640 1 --2- 172.21.15.116:0/1119218274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f84102d40 0x7f3f841009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:52.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.962+0000 7f3f82d9d640 1 -- 172.21.15.116:0/1119218274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f84103660 msgr2=0x7f3f84100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:52.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.962+0000 7f3f82d9d640 1 --2- 172.21.15.116:0/1119218274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f84103660 0x7f3f84100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:52.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.962+0000 7f3f82d9d640 1 -- 172.21.15.116:0/1119218274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f84102d40 msgr2=0x7f3f841009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:52.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.962+0000 7f3f82d9d640 1 --2- 172.21.15.116:0/1119218274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f84102d40 0x7f3f841009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:52.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.962+0000 7f3f82d9d640 1 -- 172.21.15.116:0/1119218274 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f841a69f0 con 0x7f3f8410c170 2024-09-12T03:15:52.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.963+0000 7f3f82d9d640 1 --2- 172.21.15.116:0/1119218274 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f8410c170 0x7f3f84101420 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c006010 tx=0x7f3f7c02f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:52.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.963+0000 7f3f737fe640 1 -- 172.21.15.116:0/1119218274 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c034040 con 0x7f3f8410c170 2024-09-12T03:15:52.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.963+0000 7f3f88f62640 1 -- 172.21.15.116:0/1119218274 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f841a6c80 con 0x7f3f8410c170 2024-09-12T03:15:52.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.963+0000 7f3f737fe640 1 -- 172.21.15.116:0/1119218274 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f7c003e00 con 0x7f3f8410c170 2024-09-12T03:15:52.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.963+0000 7f3f737fe640 1 -- 172.21.15.116:0/1119218274 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c0456e0 con 0x7f3f8410c170 2024-09-12T03:15:52.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.963+0000 7f3f88f62640 1 -- 172.21.15.116:0/1119218274 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f841a71b0 con 0x7f3f8410c170 2024-09-12T03:15:52.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.965+0000 7f3f737fe640 1 -- 172.21.15.116:0/1119218274 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3f7c037020 con 0x7f3f8410c170 2024-09-12T03:15:52.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.965+0000 7f3f8b33b740 1 Processor -- start 2024-09-12T03:15:52.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.966+0000 7f3f8b33b740 1 -- start start 2024-09-12T03:15:52.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.966+0000 7f3f8b33b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643521ca4a0 0x5643521ca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:52.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.966+0000 7f3f8b33b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643521cadc0 0x5643521d5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:52.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.966+0000 7f3f8b33b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643521d5680 0x5643521d7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:52.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.966+0000 7f3f8b33b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5643521302e0 con 0x5643521ca4a0 2024-09-12T03:15:52.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.966+0000 7f3f8b33b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5643520d7ce0 con 0x5643521d5680 2024-09-12T03:15:52.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.966+0000 7f3f8b33b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56435212c2a0 con 0x5643521cadc0 2024-09-12T03:15:52.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.966+0000 7f3f81d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643521ca4a0 0x5643521ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:52.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.966+0000 7f3f81d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643521ca4a0 0x5643521ca880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48690/0 (socket says 172.21.15.116:48690) 2024-09-12T03:15:52.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.966+0000 7f3f81d9b640 1 -- 172.21.15.116:0/4048862918 learned_addr learned my addr 172.21.15.116:0/4048862918 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:52.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.966+0000 7f3f8259c640 1 --2- 172.21.15.116:0/4048862918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643521cadc0 0x5643521d5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:52.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.966+0000 7f3f81d9b640 1 -- 172.21.15.116:0/4048862918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643521cadc0 msgr2=0x5643521d5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:52.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.966+0000 7f3f81d9b640 1 --2- 172.21.15.116:0/4048862918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643521cadc0 0x5643521d5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:52.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.966+0000 7f3f81d9b640 1 -- 172.21.15.116:0/4048862918 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643521d5680 msgr2=0x5643521d7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:52.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.966+0000 7f3f81d9b640 1 --2- 172.21.15.116:0/4048862918 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643521d5680 0x5643521d7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:52.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.966+0000 7f3f81d9b640 1 -- 172.21.15.116:0/4048862918 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643520bc440 con 0x5643521ca4a0 2024-09-12T03:15:52.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.966+0000 7f3f737fe640 1 --2- 172.21.15.116:0/1119218274 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3f6c0a2b80 0x7f3f6c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:52.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.966+0000 7f3f81d9b640 1 --2- 172.21.15.116:0/4048862918 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643521ca4a0 0x5643521ca880 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f3f78006aa0 tx=0x7f3f78009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23536b1ee5ee39d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:52.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.967+0000 7f3f8259c640 1 --2- 172.21.15.116:0/1119218274 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3f6c0a2b80 0x7f3f6c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:52.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.967+0000 7f3f737fe640 1 -- 172.21.15.116:0/1119218274 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f3f7c0f00e0 con 0x7f3f8410c170 2024-09-12T03:15:52.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.967+0000 7f3f70ff9640 1 -- 172.21.15.116:0/4048862918 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78018020 con 0x5643521ca4a0 2024-09-12T03:15:52.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.967+0000 7f3f70ff9640 1 -- 172.21.15.116:0/4048862918 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f78010dc0 con 0x5643521ca4a0 2024-09-12T03:15:52.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.967+0000 7f3f70ff9640 1 -- 172.21.15.116:0/4048862918 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f780137d0 con 0x5643521ca4a0 2024-09-12T03:15:52.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.967+0000 7f3f8b33b740 1 -- 172.21.15.116:0/4048862918 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643521ca4a0 msgr2=0x5643521ca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:52.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.967+0000 7f3f8259c640 1 --2- 172.21.15.116:0/1119218274 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3f6c0a2b80 0x7f3f6c0a5040 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x5643521c7890 tx=0x7f3f74008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:52.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.967+0000 7f3f8b33b740 1 --2- 172.21.15.116:0/4048862918 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643521ca4a0 0x5643521ca880 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f3f78006aa0 tx=0x7f3f78009f50 comp rx=0 tx=0).stop 2024-09-12T03:15:52.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.968+0000 7f3f8b33b740 1 -- 172.21.15.116:0/4048862918 shutdown_connections 2024-09-12T03:15:52.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.968+0000 7f3f8b33b740 1 --2- 172.21.15.116:0/4048862918 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643521d5680 0x5643521d7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:52.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.968+0000 7f3f8b33b740 1 --2- 172.21.15.116:0/4048862918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643521cadc0 0x5643521d5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:52.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.968+0000 7f3f8b33b740 1 --2- 172.21.15.116:0/4048862918 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643521ca4a0 0x5643521ca880 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:52.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.968+0000 7f3f8b33b740 1 -- 172.21.15.116:0/4048862918 >> 172.21.15.116:0/4048862918 conn(0x5643521c6a30 msgr2=0x5643521c6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:52.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.968+0000 7f3f8b33b740 1 -- 172.21.15.116:0/4048862918 shutdown_connections 2024-09-12T03:15:52.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.968+0000 7f3f8b33b740 1 -- 172.21.15.116:0/4048862918 wait complete. 2024-09-12T03:15:52.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.968+0000 7f3f8b33b740 1 -- start start 2024-09-12T03:15:52.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.968+0000 7f3f8b33b740 1 Processor -- start 2024-09-12T03:15:52.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.968+0000 7f3f8b33b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643521ca4a0 0x5643521d4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:52.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.968+0000 7f3f8b33b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643521cadc0 0x5643521d2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:52.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.968+0000 7f3f8b33b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643521d5680 0x5643521d26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:52.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.968+0000 7f3f8b33b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5643521d8250 con 0x5643521d5680 2024-09-12T03:15:52.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.968+0000 7f3f8b33b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5643520eb050 con 0x5643521cadc0 2024-09-12T03:15:52.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.968+0000 7f3f8b33b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5643521d83d0 con 0x5643521ca4a0 2024-09-12T03:15:52.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.968+0000 7f3f8b33b740 10 client.0 _ll_register_callbacks cb 0x56435212b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:53.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.969+0000 7f3f82d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643521d5680 0x5643521d26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:53.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.969+0000 7f3f82d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643521d5680 0x5643521d26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48702/0 (socket says 172.21.15.116:48702) 2024-09-12T03:15:53.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.969+0000 7f3f82d9d640 1 -- 172.21.15.116:0/3032453999 learned_addr learned my addr 172.21.15.116:0/3032453999 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:53.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.969+0000 7f3f81d9b640 1 --2- 172.21.15.116:0/3032453999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643521cadc0 0x5643521d2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:53.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.969+0000 7f3f8259c640 1 --2- 172.21.15.116:0/3032453999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643521ca4a0 0x5643521d4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:53.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.969+0000 7f3f82d9d640 1 -- 172.21.15.116:0/3032453999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643521ca4a0 msgr2=0x5643521d4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:53.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.969+0000 7f3f82d9d640 1 --2- 172.21.15.116:0/3032453999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643521ca4a0 0x5643521d4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:53.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.969+0000 7f3f82d9d640 1 -- 172.21.15.116:0/3032453999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643521cadc0 msgr2=0x5643521d2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:53.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.969+0000 7f3f82d9d640 1 --2- 172.21.15.116:0/3032453999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643521cadc0 0x5643521d2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:53.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.969+0000 7f3f82d9d640 1 -- 172.21.15.116:0/3032453999 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564351ea13c0 con 0x5643521d5680 2024-09-12T03:15:53.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.969+0000 7f3f82d9d640 1 --2- 172.21.15.116:0/3032453999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643521d5680 0x5643521d26d0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0b8dd0 tx=0x7f3f7c0bc7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2249b9d72369639e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:53.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.969+0000 7f3f69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:15:53.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.969+0000 7f3f69ffb640 1 -- 172.21.15.116:0/3032453999 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c0d5060 con 0x5643521d5680 2024-09-12T03:15:53.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.969+0000 7f3f69ffb640 1 -- 172.21.15.116:0/3032453999 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f3f7c0c5150 con 0x5643521d5680 2024-09-12T03:15:53.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.969+0000 7f3f8b33b740 1 -- 172.21.15.116:0/3032453999 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f78010dc0 con 0x5643521d5680 2024-09-12T03:15:53.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.969+0000 7f3f69ffb640 1 -- 172.21.15.116:0/3032453999 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3f7c02c040 con 0x5643521d5680 2024-09-12T03:15:53.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.969+0000 7f3f8b33b740 10 client.18876 fetch_fsmap learned FSMap version 25 2024-09-12T03:15:53.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.969+0000 7f3f8b33b740 1 -- 172.21.15.116:0/3032453999 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5643521e3b70 con 0x5643521d5680 2024-09-12T03:15:53.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.969+0000 7f3f69ffb640 1 -- 172.21.15.116:0/3032453999 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f78010dc0 con 0x5643521d5680 2024-09-12T03:15:53.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.970+0000 7f3f69ffb640 1 -- 172.21.15.116:0/3032453999 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3f7c04d030 con 0x5643521d5680 2024-09-12T03:15:53.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.970+0000 7f3f8b33b740 10 client.18876 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:15:53.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.970+0000 7f3f8b33b740 10 client.18876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:15:53.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.970+0000 7f3f8b33b740 20 client.18876 populate_metadata read hostname 'smithi116' 2024-09-12T03:15:53.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.970+0000 7f3f8b33b740 1 --2- 172.21.15.116:0/3032453999 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5643521e4fb0 0x5643521e73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:53.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.970+0000 7f3f8b33b740 4 client.18876 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:15:53.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.970+0000 7f3f8b33b740 1 -- 172.21.15.116:0/3032453999 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56435213d850 con 0x5643521e4fb0 2024-09-12T03:15:53.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.970+0000 7f3f8259c640 1 --2- 172.21.15.116:0/3032453999 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5643521e4fb0 0x5643521e73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:53.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.970+0000 7f3f69ffb640 10 client.18876 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:53.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.970+0000 7f3f8259c640 1 --2- 172.21.15.116:0/3032453999 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5643521e4fb0 0x5643521e73e0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4fb8bcce942c508 server_cookie=27ea3784641d860b in_seq=0 out_seq=0 2024-09-12T03:15:53.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.971+0000 7f3f69ffb640 1 -- 172.21.15.116:0/3032453999 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56435213d850 con 0x5643521e4fb0 2024-09-12T03:15:53.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.971+0000 7f3f69ffb640 10 client.18876 handle_command_reply: tid=0 2024-09-12T03:15:53.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.971+0000 7f3f8b33b740 1 client.18876 shutdown 2024-09-12T03:15:53.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.971+0000 7f3f8b33b740 1 -- 172.21.15.116:0/3032453999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643521d5680 msgr2=0x5643521d26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:53.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.971+0000 7f3f8b33b740 1 --2- 172.21.15.116:0/3032453999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643521d5680 0x5643521d26d0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0b8dd0 tx=0x7f3f7c0bc7c0 comp rx=0 tx=0).stop 2024-09-12T03:15:53.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.971+0000 7f3f8b33b740 1 -- 172.21.15.116:0/3032453999 shutdown_connections 2024-09-12T03:15:53.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.971+0000 7f3f8b33b740 1 --2- 172.21.15.116:0/3032453999 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5643521e4fb0 0x5643521e73e0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:53.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.971+0000 7f3f8b33b740 1 --2- 172.21.15.116:0/3032453999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643521d5680 0x5643521d26d0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:53.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.971+0000 7f3f8b33b740 1 --2- 172.21.15.116:0/3032453999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643521cadc0 0x5643521d2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:53.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.971+0000 7f3f69ffb640 0 client.18876 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:53.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.971+0000 7f3f8b33b740 1 --2- 172.21.15.116:0/3032453999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643521ca4a0 0x5643521d4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:53.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.971+0000 7f3f8b33b740 1 -- 172.21.15.116:0/3032453999 >> 172.21.15.116:0/3032453999 conn(0x5643521c6a30 msgr2=0x5643521cc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:53.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.971+0000 7f3f8b33b740 1 -- 172.21.15.116:0/3032453999 shutdown_connections 2024-09-12T03:15:53.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.971+0000 7f3f8b33b740 1 -- 172.21.15.116:0/3032453999 wait complete. 2024-09-12T03:15:53.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.971+0000 7f3f8b33b740 20 client.18876 trim_cache size 0 max 16384 2024-09-12T03:15:53.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.988+0000 7f3f8b33b740 1 Processor -- start 2024-09-12T03:15:53.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.988+0000 7f3f8b33b740 1 -- start start 2024-09-12T03:15:53.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.988+0000 7f3f8b33b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643521c6a30 0x5643521d4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:53.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.988+0000 7f3f8b33b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643521ca4a0 0x5643521eccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:53.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.988+0000 7f3f8b33b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643521cadc0 0x5643521ed230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:53.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.988+0000 7f3f8b33b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841b1f00 con 0x5643521ca4a0 2024-09-12T03:15:53.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.988+0000 7f3f8b33b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841af7d0 con 0x5643521cadc0 2024-09-12T03:15:53.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.988+0000 7f3f8b33b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841ad620 con 0x5643521c6a30 2024-09-12T03:15:53.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.988+0000 7f3f81d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643521ca4a0 0x5643521eccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:53.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.988+0000 7f3f81d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643521ca4a0 0x5643521eccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48708/0 (socket says 172.21.15.116:48708) 2024-09-12T03:15:53.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.988+0000 7f3f81d9b640 1 -- 172.21.15.116:0/907054227 learned_addr learned my addr 172.21.15.116:0/907054227 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:53.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.989+0000 7f3f82d9d640 1 --2- 172.21.15.116:0/907054227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643521cadc0 0x5643521ed230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:53.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.989+0000 7f3f81d9b640 1 -- 172.21.15.116:0/907054227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643521c6a30 msgr2=0x5643521d4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:53.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.989+0000 7f3f8259c640 1 --2- 172.21.15.116:0/907054227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643521c6a30 0x5643521d4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:53.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.989+0000 7f3f81d9b640 1 --2- 172.21.15.116:0/907054227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643521c6a30 0x5643521d4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:53.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.989+0000 7f3f81d9b640 1 -- 172.21.15.116:0/907054227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643521cadc0 msgr2=0x5643521ed230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:53.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.989+0000 7f3f81d9b640 1 --2- 172.21.15.116:0/907054227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643521cadc0 0x5643521ed230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:53.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.989+0000 7f3f81d9b640 1 -- 172.21.15.116:0/907054227 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643521e7a60 con 0x5643521ca4a0 2024-09-12T03:15:53.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.989+0000 7f3f82d9d640 1 --2- 172.21.15.116:0/907054227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643521cadc0 0x5643521ed230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:15:53.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.989+0000 7f3f81d9b640 1 --2- 172.21.15.116:0/907054227 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643521ca4a0 0x5643521eccf0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f3f78006a70 tx=0x7f3f78010c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae37a270d4497b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:53.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.989+0000 7f3f6bfff640 1 -- 172.21.15.116:0/907054227 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78018070 con 0x5643521ca4a0 2024-09-12T03:15:53.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.989+0000 7f3f6bfff640 1 -- 172.21.15.116:0/907054227 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f78014b20 con 0x5643521ca4a0 2024-09-12T03:15:53.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.989+0000 7f3f6bfff640 1 -- 172.21.15.116:0/907054227 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7801ebc0 con 0x5643521ca4a0 2024-09-12T03:15:53.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.990+0000 7f3f8b33b740 1 -- 172.21.15.116:0/907054227 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643521ca4a0 msgr2=0x5643521eccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:53.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.990+0000 7f3f8b33b740 1 --2- 172.21.15.116:0/907054227 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643521ca4a0 0x5643521eccf0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f3f78006a70 tx=0x7f3f78010c70 comp rx=0 tx=0).stop 2024-09-12T03:15:53.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.990+0000 7f3f8b33b740 1 -- 172.21.15.116:0/907054227 shutdown_connections 2024-09-12T03:15:53.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.990+0000 7f3f8b33b740 1 --2- 172.21.15.116:0/907054227 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643521cadc0 0x5643521ed230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:53.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.990+0000 7f3f8b33b740 1 --2- 172.21.15.116:0/907054227 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643521ca4a0 0x5643521eccf0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:53.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.990+0000 7f3f8b33b740 1 --2- 172.21.15.116:0/907054227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643521c6a30 0x5643521d4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:53.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.990+0000 7f3f8b33b740 1 -- 172.21.15.116:0/907054227 >> 172.21.15.116:0/907054227 conn(0x5643521d2830 msgr2=0x5643521d39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:53.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.990+0000 7f3f8b33b740 1 -- 172.21.15.116:0/907054227 shutdown_connections 2024-09-12T03:15:53.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.990+0000 7f3f8b33b740 1 -- 172.21.15.116:0/907054227 wait complete. 2024-09-12T03:15:53.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.990+0000 7f3f8b33b740 1 -- start start 2024-09-12T03:15:53.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.990+0000 7f3f8b33b740 1 Processor -- start 2024-09-12T03:15:53.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.990+0000 7f3f8b33b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643521c6a30 0x5643521db790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:53.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.990+0000 7f3f8b33b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643521ca4a0 0x5643521dbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:53.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.991+0000 7f3f8b33b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643521cadc0 0x56435224ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:53.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.991+0000 7f3f8b33b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841adc90 con 0x5643521ca4a0 2024-09-12T03:15:53.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.991+0000 7f3f8b33b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841af100 con 0x5643521c6a30 2024-09-12T03:15:53.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.991+0000 7f3f8b33b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3f841ae360 con 0x5643521cadc0 2024-09-12T03:15:53.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.991+0000 7f3f81d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643521ca4a0 0x5643521dbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:53.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.991+0000 7f3f8b33b740 10 client.0 _ll_register_callbacks cb 0x7f3f7c0b8c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:53.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.991+0000 7f3f81d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643521ca4a0 0x5643521dbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48722/0 (socket says 172.21.15.116:48722) 2024-09-12T03:15:53.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.991+0000 7f3f81d9b640 1 -- 172.21.15.116:0/1899768385 learned_addr learned my addr 172.21.15.116:0/1899768385 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:15:53.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.991+0000 7f3f8259c640 1 --2- 172.21.15.116:0/1899768385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643521c6a30 0x5643521db790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:53.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.991+0000 7f3f82d9d640 1 --2- 172.21.15.116:0/1899768385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643521cadc0 0x56435224ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:53.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.991+0000 7f3f81d9b640 1 -- 172.21.15.116:0/1899768385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643521cadc0 msgr2=0x56435224ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:53.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.991+0000 7f3f81d9b640 1 --2- 172.21.15.116:0/1899768385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643521cadc0 0x56435224ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:53.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.991+0000 7f3f81d9b640 1 -- 172.21.15.116:0/1899768385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643521c6a30 msgr2=0x5643521db790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:53.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.991+0000 7f3f81d9b640 1 --2- 172.21.15.116:0/1899768385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643521c6a30 0x5643521db790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:53.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.991+0000 7f3f81d9b640 1 -- 172.21.15.116:0/1899768385 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643521c76c0 con 0x5643521ca4a0 2024-09-12T03:15:53.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.991+0000 7f3f81d9b640 1 --2- 172.21.15.116:0/1899768385 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643521ca4a0 0x5643521dbcd0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f3f78004810 tx=0x7f3f780098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611a1f686fa0f7e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:53.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.991+0000 7f3f69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:15:53.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.991+0000 7f3f69ffb640 1 -- 172.21.15.116:0/1899768385 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78018040 con 0x5643521ca4a0 2024-09-12T03:15:53.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.991+0000 7f3f69ffb640 1 -- 172.21.15.116:0/1899768385 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f3f78005bc0 con 0x5643521ca4a0 2024-09-12T03:15:53.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.991+0000 7f3f8b33b740 1 -- 172.21.15.116:0/1899768385 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f78014b20 con 0x5643521ca4a0 2024-09-12T03:15:53.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.992+0000 7f3f69ffb640 1 -- 172.21.15.116:0/1899768385 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3f78035020 con 0x5643521ca4a0 2024-09-12T03:15:53.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.992+0000 7f3f8b33b740 10 client.18882 fetch_fsmap learned FSMap version 25 2024-09-12T03:15:53.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.992+0000 7f3f8b33b740 1 -- 172.21.15.116:0/1899768385 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5643521c6860 con 0x5643521ca4a0 2024-09-12T03:15:53.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.992+0000 7f3f69ffb640 1 -- 172.21.15.116:0/1899768385 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f78014b20 con 0x5643521ca4a0 2024-09-12T03:15:53.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.992+0000 7f3f69ffb640 1 -- 172.21.15.116:0/1899768385 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3f780273f0 con 0x5643521ca4a0 2024-09-12T03:15:53.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.992+0000 7f3f8b33b740 10 client.18882 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:15:53.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.992+0000 7f3f8b33b740 10 client.18882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:15:53.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.992+0000 7f3f8b33b740 20 client.18882 populate_metadata read hostname 'smithi116' 2024-09-12T03:15:53.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.992+0000 7f3f8b33b740 1 --2- 172.21.15.116:0/1899768385 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5643521d5680 0x564352252400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:53.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.992+0000 7f3f8b33b740 4 client.18882 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:15:53.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.992+0000 7f3f8b33b740 1 -- 172.21.15.116:0/1899768385 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564352137d60 con 0x5643521d5680 2024-09-12T03:15:53.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.992+0000 7f3f8259c640 1 --2- 172.21.15.116:0/1899768385 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5643521d5680 0x564352252400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:53.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.993+0000 7f3f8259c640 1 --2- 172.21.15.116:0/1899768385 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5643521d5680 0x564352252400 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5297a43ff8e395d1 server_cookie=d16e3288edba1dea in_seq=0 out_seq=0 2024-09-12T03:15:53.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.993+0000 7f3f69ffb640 10 client.18882 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:53.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.993+0000 7f3f69ffb640 1 -- 172.21.15.116:0/1899768385 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564352137d60 con 0x5643521d5680 2024-09-12T03:15:53.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.993+0000 7f3f69ffb640 10 client.18882 handle_command_reply: tid=0 2024-09-12T03:15:53.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.993+0000 7f3f8b33b740 1 client.18882 shutdown 2024-09-12T03:15:53.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.993+0000 7f3f8b33b740 1 -- 172.21.15.116:0/1899768385 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643521ca4a0 msgr2=0x5643521dbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:53.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.993+0000 7f3f8b33b740 1 --2- 172.21.15.116:0/1899768385 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643521ca4a0 0x5643521dbcd0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f3f78004810 tx=0x7f3f780098e0 comp rx=0 tx=0).stop 2024-09-12T03:15:53.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.993+0000 7f3f8b33b740 1 -- 172.21.15.116:0/1899768385 shutdown_connections 2024-09-12T03:15:53.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.993+0000 7f3f8b33b740 1 --2- 172.21.15.116:0/1899768385 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5643521d5680 0x564352252400 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:53.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.993+0000 7f3f8b33b740 1 --2- 172.21.15.116:0/1899768385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643521cadc0 0x56435224ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:53.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.993+0000 7f3f8b33b740 1 --2- 172.21.15.116:0/1899768385 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5643521ca4a0 0x5643521dbcd0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:53.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.993+0000 7f3f8b33b740 1 --2- 172.21.15.116:0/1899768385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643521c6a30 0x5643521db790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:53.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.993+0000 7f3f8b33b740 1 -- 172.21.15.116:0/1899768385 >> 172.21.15.116:0/1899768385 conn(0x5643521d2830 msgr2=0x5643521d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:53.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.993+0000 7f3f69ffb640 0 client.18882 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:15:53.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.993+0000 7f3f8b33b740 1 -- 172.21.15.116:0/1899768385 shutdown_connections 2024-09-12T03:15:53.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.993+0000 7f3f8b33b740 1 -- 172.21.15.116:0/1899768385 wait complete. 2024-09-12T03:15:53.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.993+0000 7f3f8b33b740 20 client.18882 trim_cache size 0 max 16384 2024-09-12T03:15:53.010 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:15:53.010 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:15:53.010 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": { 2024-09-12T03:15:53.011 INFO:teuthology.orchestra.run.smithi116.stdout: "798d81d3-a1c4-45de-8ac9-c1253a43f18f": { 2024-09-12T03:15:53.011 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/", 2024-09-12T03:15:53.011 INFO:teuthology.orchestra.run.smithi116.stdout: "tag": "798d81d3-a1c4-45de-8ac9-c1253a43f18f", 2024-09-12T03:15:53.011 INFO:teuthology.orchestra.run.smithi116.stdout: "options": "recursive,force" 2024-09-12T03:15:53.011 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:15:53.011 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:15:53.011 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:15:53.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.994+0000 7f3f88f62640 1 -- 172.21.15.116:0/1119218274 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3f6c0a2b80 msgr2=0x7f3f6c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:53.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.994+0000 7f3f88f62640 1 --2- 172.21.15.116:0/1119218274 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3f6c0a2b80 0x7f3f6c0a5040 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x5643521c7890 tx=0x7f3f74008040 comp rx=0 tx=0).stop 2024-09-12T03:15:53.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.994+0000 7f3f88f62640 1 -- 172.21.15.116:0/1119218274 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f8410c170 msgr2=0x7f3f84101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:53.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.994+0000 7f3f88f62640 1 --2- 172.21.15.116:0/1119218274 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f8410c170 0x7f3f84101420 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c006010 tx=0x7f3f7c02f710 comp rx=0 tx=0).stop 2024-09-12T03:15:53.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.994+0000 7f3f88f62640 1 -- 172.21.15.116:0/1119218274 shutdown_connections 2024-09-12T03:15:53.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.994+0000 7f3f88f62640 1 --2- 172.21.15.116:0/1119218274 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3f6c0a2b80 0x7f3f6c0a5040 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:53.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.994+0000 7f3f88f62640 1 --2- 172.21.15.116:0/1119218274 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f8410c170 0x7f3f84101420 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:53.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.994+0000 7f3f88f62640 1 --2- 172.21.15.116:0/1119218274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f84103660 0x7f3f84100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:53.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.994+0000 7f3f88f62640 1 --2- 172.21.15.116:0/1119218274 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f84102d40 0x7f3f841009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:53.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.994+0000 7f3f88f62640 1 -- 172.21.15.116:0/1119218274 >> 172.21.15.116:0/1119218274 conn(0x7f3f840fc7a0 msgr2=0x7f3f840fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:53.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.994+0000 7f3f88f62640 1 -- 172.21.15.116:0/1119218274 shutdown_connections 2024-09-12T03:15:53.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:15:52.994+0000 7f3f88f62640 1 -- 172.21.15.116:0/1119218274 wait complete. 2024-09-12T03:15:53.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:53 smithi183 ceph-mon[35535]: pgmap v725: 97 pgs: 97 active+clean; 6.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:15:53.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:53 smithi177 ceph-mon[35305]: pgmap v725: 97 pgs: 97 active+clean; 6.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:15:53.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:53 smithi116 ceph-mon[31410]: pgmap v725: 97 pgs: 97 active+clean; 6.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:15:53.917 INFO:tasks.cephfs.filesystem:scrub status for tag:798d81d3-a1c4-45de-8ac9-c1253a43f18f - {'path': '/', 'tag': '798d81d3-a1c4-45de-8ac9-c1253a43f18f', 'options': 'recursive,force'} 2024-09-12T03:15:54.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:54 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:15:54.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:54 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:15:54.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:54 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:15:55.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:55 smithi177 ceph-mon[35305]: pgmap v726: 97 pgs: 97 active+clean; 6.3 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:15:55.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:55 smithi116 ceph-mon[31410]: pgmap v726: 97 pgs: 97 active+clean; 6.3 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:15:55.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:55 smithi183 ceph-mon[35535]: pgmap v726: 97 pgs: 97 active+clean; 6.3 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:15:57.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:57 smithi183 ceph-mon[35535]: pgmap v727: 97 pgs: 97 active+clean; 6.3 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T03:15:57.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:57 smithi177 ceph-mon[35305]: pgmap v727: 97 pgs: 97 active+clean; 6.3 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T03:15:57.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:57 smithi116 ceph-mon[31410]: pgmap v727: 97 pgs: 97 active+clean; 6.3 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T03:15:59.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:15:59 smithi183 ceph-mon[35535]: pgmap v728: 97 pgs: 97 active+clean; 6.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:15:59.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:15:59 smithi177 ceph-mon[35305]: pgmap v728: 97 pgs: 97 active+clean; 6.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:15:59.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:15:59 smithi116 ceph-mon[31410]: pgmap v728: 97 pgs: 97 active+clean; 6.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:16:01.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:01 smithi183 ceph-mon[35535]: pgmap v729: 97 pgs: 97 active+clean; 6.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:16:01.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:01 smithi177 ceph-mon[35305]: pgmap v729: 97 pgs: 97 active+clean; 6.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:16:01.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:01 smithi116 ceph-mon[31410]: pgmap v729: 97 pgs: 97 active+clean; 6.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:16:03.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:03 smithi183 ceph-mon[35535]: pgmap v730: 97 pgs: 97 active+clean; 6.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:16:03.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:03 smithi177 ceph-mon[35305]: pgmap v730: 97 pgs: 97 active+clean; 6.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:16:03.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:03 smithi116 ceph-mon[31410]: pgmap v730: 97 pgs: 97 active+clean; 6.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:16:05.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:05 smithi183 ceph-mon[35535]: pgmap v731: 97 pgs: 97 active+clean; 6.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:16:05.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:05 smithi177 ceph-mon[35305]: pgmap v731: 97 pgs: 97 active+clean; 6.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:16:05.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:05 smithi116 ceph-mon[31410]: pgmap v731: 97 pgs: 97 active+clean; 6.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:16:07.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:07 smithi183 ceph-mon[35535]: pgmap v732: 97 pgs: 97 active+clean; 6.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:16:07.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:07 smithi177 ceph-mon[35305]: pgmap v732: 97 pgs: 97 active+clean; 6.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:16:07.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:07 smithi116 ceph-mon[31410]: pgmap v732: 97 pgs: 97 active+clean; 6.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:16:09.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:09 smithi183 ceph-mon[35535]: pgmap v733: 97 pgs: 97 active+clean; 6.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-12T03:16:09.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:09 smithi177 ceph-mon[35305]: pgmap v733: 97 pgs: 97 active+clean; 6.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-12T03:16:09.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:09 smithi116 ceph-mon[31410]: pgmap v733: 97 pgs: 97 active+clean; 6.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-12T03:16:11.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:11 smithi183 ceph-mon[35535]: pgmap v734: 97 pgs: 97 active+clean; 6.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:16:11.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:11 smithi177 ceph-mon[35305]: pgmap v734: 97 pgs: 97 active+clean; 6.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:16:11.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:11 smithi116 ceph-mon[31410]: pgmap v734: 97 pgs: 97 active+clean; 6.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:16:13.426 INFO:tasks.workunit.client.0.smithi116.stdout:437/600: open fd = 3 2024-09-12T03:16:13.426 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.426 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.426 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.426 INFO:tasks.workunit.client.0.smithi116.stdout:438/600: open fd = 3 2024-09-12T03:16:13.426 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.426 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.427 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.427 INFO:tasks.workunit.client.0.smithi116.stdout:439/600: open fd = 3 2024-09-12T03:16:13.427 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.427 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.427 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.427 INFO:tasks.workunit.client.0.smithi116.stdout:440/600: open fd = 3 2024-09-12T03:16:13.427 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.427 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.427 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.427 INFO:tasks.workunit.client.0.smithi116.stdout:441/600: open fd = 3 2024-09-12T03:16:13.427 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.427 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.427 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.427 INFO:tasks.workunit.client.0.smithi116.stdout:442/600: open fd = 3 2024-09-12T03:16:13.428 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.428 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.428 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.428 INFO:tasks.workunit.client.0.smithi116.stdout:443/600: open fd = 3 2024-09-12T03:16:13.428 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.428 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.428 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.428 INFO:tasks.workunit.client.0.smithi116.stdout:444/600: open fd = 3 2024-09-12T03:16:13.428 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.428 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.428 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.428 INFO:tasks.workunit.client.0.smithi116.stdout:445/600: open fd = 3 2024-09-12T03:16:13.428 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.428 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.429 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.429 INFO:tasks.workunit.client.0.smithi116.stdout:446/600: open fd = 3 2024-09-12T03:16:13.429 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.429 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.429 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.429 INFO:tasks.workunit.client.0.smithi116.stdout:447/600: open fd = 3 2024-09-12T03:16:13.429 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.429 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.429 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.429 INFO:tasks.workunit.client.0.smithi116.stdout:448/600: open fd = 3 2024-09-12T03:16:13.429 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.429 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.430 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.430 INFO:tasks.workunit.client.0.smithi116.stdout:449/600: open fd = 3 2024-09-12T03:16:13.430 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.430 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.430 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.430 INFO:tasks.workunit.client.0.smithi116.stdout:450/600: open fd = 3 2024-09-12T03:16:13.430 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.430 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.430 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.430 INFO:tasks.workunit.client.0.smithi116.stdout:451/600: open fd = 3 2024-09-12T03:16:13.430 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.430 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.430 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.430 INFO:tasks.workunit.client.0.smithi116.stdout:452/600: open fd = 3 2024-09-12T03:16:13.431 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.431 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.431 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.431 INFO:tasks.workunit.client.0.smithi116.stdout:453/600: open fd = 3 2024-09-12T03:16:13.431 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.431 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.431 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.431 INFO:tasks.workunit.client.0.smithi116.stdout:454/600: open fd = 3 2024-09-12T03:16:13.431 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.431 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.431 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.431 INFO:tasks.workunit.client.0.smithi116.stdout:455/600: open fd = 3 2024-09-12T03:16:13.431 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.431 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.432 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.432 INFO:tasks.workunit.client.0.smithi116.stdout:456/600: open fd = 3 2024-09-12T03:16:13.432 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.432 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.432 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.432 INFO:tasks.workunit.client.0.smithi116.stdout:457/600: open fd = 3 2024-09-12T03:16:13.432 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.432 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.432 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.432 INFO:tasks.workunit.client.0.smithi116.stdout:458/600: open fd = 3 2024-09-12T03:16:13.432 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.432 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.433 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.433 INFO:tasks.workunit.client.0.smithi116.stdout:459/600: open fd = 3 2024-09-12T03:16:13.433 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.433 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.433 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.433 INFO:tasks.workunit.client.0.smithi116.stdout:460/600: open fd = 3 2024-09-12T03:16:13.433 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.433 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.433 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.433 INFO:tasks.workunit.client.0.smithi116.stdout:461/600: open fd = 3 2024-09-12T03:16:13.433 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.433 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.434 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.434 INFO:tasks.workunit.client.0.smithi116.stdout:462/600: open fd = 3 2024-09-12T03:16:13.434 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.434 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.434 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.434 INFO:tasks.workunit.client.0.smithi116.stdout:463/600: open fd = 3 2024-09-12T03:16:13.434 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.434 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.434 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.434 INFO:tasks.workunit.client.0.smithi116.stdout:464/600: open fd = 3 2024-09-12T03:16:13.434 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.434 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.434 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.434 INFO:tasks.workunit.client.0.smithi116.stdout:465/600: open fd = 3 2024-09-12T03:16:13.435 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.435 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.435 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.435 INFO:tasks.workunit.client.0.smithi116.stdout:466/600: open fd = 3 2024-09-12T03:16:13.435 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.435 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.435 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.435 INFO:tasks.workunit.client.0.smithi116.stdout:467/600: open fd = 3 2024-09-12T03:16:13.435 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.435 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.435 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.435 INFO:tasks.workunit.client.0.smithi116.stdout:468/600: open fd = 3 2024-09-12T03:16:13.435 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.435 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.435 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.436 INFO:tasks.workunit.client.0.smithi116.stdout:469/600: open fd = 3 2024-09-12T03:16:13.436 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.436 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.436 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.436 INFO:tasks.workunit.client.0.smithi116.stdout:470/600: open fd = 3 2024-09-12T03:16:13.436 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.436 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.436 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.436 INFO:tasks.workunit.client.0.smithi116.stdout:471/600: open fd = 3 2024-09-12T03:16:13.436 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.436 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.437 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.437 INFO:tasks.workunit.client.0.smithi116.stdout:472/600: open fd = 3 2024-09-12T03:16:13.437 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.437 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.437 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.437 INFO:tasks.workunit.client.0.smithi116.stdout:473/600: open fd = 3 2024-09-12T03:16:13.437 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.437 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.437 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.437 INFO:tasks.workunit.client.0.smithi116.stdout:474/600: open fd = 3 2024-09-12T03:16:13.437 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.437 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.437 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.437 INFO:tasks.workunit.client.0.smithi116.stdout:475/600: open fd = 3 2024-09-12T03:16:13.438 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.438 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.438 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.438 INFO:tasks.workunit.client.0.smithi116.stdout:476/600: open fd = 3 2024-09-12T03:16:13.438 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.438 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.438 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.438 INFO:tasks.workunit.client.0.smithi116.stdout:477/600: open fd = 3 2024-09-12T03:16:13.438 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.438 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.438 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.438 INFO:tasks.workunit.client.0.smithi116.stdout:478/600: open fd = 3 2024-09-12T03:16:13.438 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.438 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.439 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.439 INFO:tasks.workunit.client.0.smithi116.stdout:479/600: open fd = 3 2024-09-12T03:16:13.439 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.439 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.439 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.439 INFO:tasks.workunit.client.0.smithi116.stdout:480/600: open fd = 3 2024-09-12T03:16:13.439 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.439 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.439 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.439 INFO:tasks.workunit.client.0.smithi116.stdout:481/600: open fd = 3 2024-09-12T03:16:13.439 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.439 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.439 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.439 INFO:tasks.workunit.client.0.smithi116.stdout:482/600: open fd = 3 2024-09-12T03:16:13.440 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.440 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.440 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.440 INFO:tasks.workunit.client.0.smithi116.stdout:483/600: open fd = 3 2024-09-12T03:16:13.440 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.440 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.440 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.440 INFO:tasks.workunit.client.0.smithi116.stdout:484/600: open fd = 3 2024-09-12T03:16:13.440 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.440 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.440 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.440 INFO:tasks.workunit.client.0.smithi116.stdout:485/600: open fd = 3 2024-09-12T03:16:13.440 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.441 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.441 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.441 INFO:tasks.workunit.client.0.smithi116.stdout:486/600: open fd = 3 2024-09-12T03:16:13.441 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.441 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.441 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.441 INFO:tasks.workunit.client.0.smithi116.stdout:487/600: open fd = 3 2024-09-12T03:16:13.441 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.441 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.441 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.441 INFO:tasks.workunit.client.0.smithi116.stdout:488/600: open fd = 3 2024-09-12T03:16:13.441 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.441 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.442 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.442 INFO:tasks.workunit.client.0.smithi116.stdout:489/600: open fd = 3 2024-09-12T03:16:13.442 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.442 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.442 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.442 INFO:tasks.workunit.client.0.smithi116.stdout:490/600: open fd = 3 2024-09-12T03:16:13.442 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.442 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.442 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.442 INFO:tasks.workunit.client.0.smithi116.stdout:491/600: open fd = 3 2024-09-12T03:16:13.442 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.442 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.442 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.442 INFO:tasks.workunit.client.0.smithi116.stdout:492/600: open fd = 3 2024-09-12T03:16:13.443 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.443 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.443 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.443 INFO:tasks.workunit.client.0.smithi116.stdout:493/600: open fd = 3 2024-09-12T03:16:13.443 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.443 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.443 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.443 INFO:tasks.workunit.client.0.smithi116.stdout:494/600: open fd = 3 2024-09-12T03:16:13.443 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.443 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.443 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.443 INFO:tasks.workunit.client.0.smithi116.stdout:495/600: open fd = 3 2024-09-12T03:16:13.443 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.444 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.444 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.444 INFO:tasks.workunit.client.0.smithi116.stdout:496/600: open fd = 3 2024-09-12T03:16:13.444 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.444 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.444 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.444 INFO:tasks.workunit.client.0.smithi116.stdout:497/600: open fd = 3 2024-09-12T03:16:13.444 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.444 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.444 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.444 INFO:tasks.workunit.client.0.smithi116.stdout:498/600: open fd = 3 2024-09-12T03:16:13.444 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.444 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:16:13.444 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:16:13.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:13 smithi183 ceph-mon[35535]: pgmap v735: 97 pgs: 97 active+clean; 6.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:16:13.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:13 smithi177 ceph-mon[35305]: pgmap v735: 97 pgs: 97 active+clean; 6.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:16:13.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:13 smithi116 ceph-mon[31410]: pgmap v735: 97 pgs: 97 active+clean; 6.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:16:15.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:15 smithi183 ceph-mon[35535]: pgmap v736: 97 pgs: 97 active+clean; 6.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:16:15.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:15 smithi177 ceph-mon[35305]: pgmap v736: 97 pgs: 97 active+clean; 6.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:16:15.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:15 smithi116 ceph-mon[31410]: pgmap v736: 97 pgs: 97 active+clean; 6.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:16:17.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:17 smithi183 ceph-mon[35535]: pgmap v737: 97 pgs: 97 active+clean; 6.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:16:17.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:17 smithi177 ceph-mon[35305]: pgmap v737: 97 pgs: 97 active+clean; 6.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:16:17.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:17 smithi116 ceph-mon[31410]: pgmap v737: 97 pgs: 97 active+clean; 6.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:16:19.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:19 smithi183 ceph-mon[35535]: pgmap v738: 97 pgs: 97 active+clean; 6.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-12T03:16:19.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:19 smithi177 ceph-mon[35305]: pgmap v738: 97 pgs: 97 active+clean; 6.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-12T03:16:19.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:19 smithi116 ceph-mon[31410]: pgmap v738: 97 pgs: 97 active+clean; 6.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-12T03:16:20.338 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T03:16:20.339 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-12T03:16:20.339 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:16:20.602 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:21.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.146+0000 7f54d2c40640 1 -- 172.21.15.116:0/1763620874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54cc102ef0 msgr2=0x7f54cc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:21.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.146+0000 7f54d2c40640 1 --2- 172.21.15.116:0/1763620874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54cc102ef0 0x7f54cc10da40 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f54c4009ca0 tx=0x7f54c402a950 comp rx=0 tx=0).stop 2024-09-12T03:16:21.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.146+0000 7f54d2c40640 1 -- 172.21.15.116:0/1763620874 shutdown_connections 2024-09-12T03:16:21.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.146+0000 7f54d2c40640 1 --2- 172.21.15.116:0/1763620874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54cc102ef0 0x7f54cc10da40 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:21.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.146+0000 7f54d2c40640 1 --2- 172.21.15.116:0/1763620874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54cc102570 0x7f54cc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:21.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.146+0000 7f54d2c40640 1 --2- 172.21.15.116:0/1763620874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54cc100e20 0x7f54cc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:21.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.146+0000 7f54d2c40640 1 -- 172.21.15.116:0/1763620874 >> 172.21.15.116:0/1763620874 conn(0x7f54cc0fc7d0 msgr2=0x7f54cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:21.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.146+0000 7f54d2c40640 1 -- 172.21.15.116:0/1763620874 shutdown_connections 2024-09-12T03:16:21.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.146+0000 7f54d2c40640 1 -- 172.21.15.116:0/1763620874 wait complete. 2024-09-12T03:16:21.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.146+0000 7f54d2c40640 1 Processor -- start 2024-09-12T03:16:21.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.147+0000 7f54d2c40640 1 -- start start 2024-09-12T03:16:21.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.147+0000 7f54d2c40640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54cc100e20 0x7f54cc1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:21.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.147+0000 7f54d2c40640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54cc102570 0x7f54cc198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:21.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.147+0000 7f54d2c40640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54cc102ef0 0x7f54cc19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:21.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.147+0000 7f54d2c40640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f54cc110f80 con 0x7f54cc100e20 2024-09-12T03:16:21.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.147+0000 7f54d2c40640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f54cc110e00 con 0x7f54cc102570 2024-09-12T03:16:21.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.147+0000 7f54d2c40640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f54cc111100 con 0x7f54cc102ef0 2024-09-12T03:16:21.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.147+0000 7f54d09dc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54cc100e20 0x7f54cc1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:21.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.147+0000 7f54d09dc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54cc100e20 0x7f54cc1983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39690/0 (socket says 172.21.15.116:39690) 2024-09-12T03:16:21.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.147+0000 7f54d09dc640 1 -- 172.21.15.116:0/2518145512 learned_addr learned my addr 172.21.15.116:0/2518145512 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:21.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.147+0000 7f54cbfff640 1 --2- 172.21.15.116:0/2518145512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54cc102570 0x7f54cc198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:21.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.147+0000 7f54d11dd640 1 --2- 172.21.15.116:0/2518145512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54cc102ef0 0x7f54cc19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:21.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.147+0000 7f54d09dc640 1 -- 172.21.15.116:0/2518145512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54cc102ef0 msgr2=0x7f54cc19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:21.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.147+0000 7f54d09dc640 1 --2- 172.21.15.116:0/2518145512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54cc102ef0 0x7f54cc19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:21.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.147+0000 7f54d09dc640 1 -- 172.21.15.116:0/2518145512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54cc102570 msgr2=0x7f54cc198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:21.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.147+0000 7f54d09dc640 1 --2- 172.21.15.116:0/2518145512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54cc102570 0x7f54cc198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:21.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.147+0000 7f54d09dc640 1 -- 172.21.15.116:0/2518145512 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54cc19d380 con 0x7f54cc100e20 2024-09-12T03:16:21.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.147+0000 7f54d09dc640 1 --2- 172.21.15.116:0/2518145512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54cc100e20 0x7f54cc1983d0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f54c000c920 tx=0x7f54c000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:21.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.147+0000 7f54c9ffb640 1 -- 172.21.15.116:0/2518145512 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c0007c70 con 0x7f54cc100e20 2024-09-12T03:16:21.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.148+0000 7f54c9ffb640 1 -- 172.21.15.116:0/2518145512 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54c0007e10 con 0x7f54cc100e20 2024-09-12T03:16:21.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.148+0000 7f54d2c40640 1 -- 172.21.15.116:0/2518145512 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54cc19d670 con 0x7f54cc100e20 2024-09-12T03:16:21.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.148+0000 7f54c9ffb640 1 -- 172.21.15.116:0/2518145512 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c0018890 con 0x7f54cc100e20 2024-09-12T03:16:21.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.148+0000 7f54d2c40640 1 -- 172.21.15.116:0/2518145512 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54cc1a4f50 con 0x7f54cc100e20 2024-09-12T03:16:21.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.149+0000 7f54d2c40640 1 -- 172.21.15.116:0/2518145512 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54cc101200 con 0x7f54cc100e20 2024-09-12T03:16:21.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.150+0000 7f54c9ffb640 1 -- 172.21.15.116:0/2518145512 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f54c0018a70 con 0x7f54cc100e20 2024-09-12T03:16:21.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.150+0000 7f54c9ffb640 1 --2- 172.21.15.116:0/2518145512 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f54bc0a2b80 0x7f54bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:21.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.150+0000 7f54cbfff640 1 --2- 172.21.15.116:0/2518145512 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f54bc0a2b80 0x7f54bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:21.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.150+0000 7f54c9ffb640 1 -- 172.21.15.116:0/2518145512 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f54c00cf160 con 0x7f54cc100e20 2024-09-12T03:16:21.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.151+0000 7f54cbfff640 1 --2- 172.21.15.116:0/2518145512 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f54bc0a2b80 0x7f54bc0a5040 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f54cc1051c0 tx=0x7f54b800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:21.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.155+0000 7f54c9ffb640 1 -- 172.21.15.116:0/2518145512 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f54c0014030 con 0x7f54cc100e20 2024-09-12T03:16:21.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.356+0000 7f54d2c40640 1 -- 172.21.15.116:0/2518145512 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54cc10a0a0 con 0x7f54cc100e20 2024-09-12T03:16:21.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.357+0000 7f54c9ffb640 1 -- 172.21.15.116:0/2518145512 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f54c009c0f0 con 0x7f54cc100e20 2024-09-12T03:16:21.358 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:16:21.358 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:16:21.358 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:16:21.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.360+0000 7f54d2c40640 1 -- 172.21.15.116:0/2518145512 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f54bc0a2b80 msgr2=0x7f54bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:21.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.360+0000 7f54d2c40640 1 --2- 172.21.15.116:0/2518145512 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f54bc0a2b80 0x7f54bc0a5040 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f54cc1051c0 tx=0x7f54b800a2d0 comp rx=0 tx=0).stop 2024-09-12T03:16:21.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.360+0000 7f54d2c40640 1 -- 172.21.15.116:0/2518145512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54cc100e20 msgr2=0x7f54cc1983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:21.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.360+0000 7f54d2c40640 1 --2- 172.21.15.116:0/2518145512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54cc100e20 0x7f54cc1983d0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f54c000c920 tx=0x7f54c000cde0 comp rx=0 tx=0).stop 2024-09-12T03:16:21.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.360+0000 7f54d2c40640 1 -- 172.21.15.116:0/2518145512 shutdown_connections 2024-09-12T03:16:21.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.360+0000 7f54d2c40640 1 --2- 172.21.15.116:0/2518145512 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f54bc0a2b80 0x7f54bc0a5040 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:21.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.360+0000 7f54d2c40640 1 --2- 172.21.15.116:0/2518145512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54cc102ef0 0x7f54cc19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:21.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.360+0000 7f54d2c40640 1 --2- 172.21.15.116:0/2518145512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54cc102570 0x7f54cc198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:21.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.360+0000 7f54d2c40640 1 --2- 172.21.15.116:0/2518145512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54cc100e20 0x7f54cc1983d0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:21.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.360+0000 7f54d2c40640 1 -- 172.21.15.116:0/2518145512 >> 172.21.15.116:0/2518145512 conn(0x7f54cc0fc7d0 msgr2=0x7f54cc06a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:21.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.360+0000 7f54d2c40640 1 -- 172.21.15.116:0/2518145512 shutdown_connections 2024-09-12T03:16:21.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:21.361+0000 7f54d2c40640 1 -- 172.21.15.116:0/2518145512 wait complete. 2024-09-12T03:16:21.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:21 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2518145512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:21.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:21 smithi183 ceph-mon[35535]: pgmap v739: 97 pgs: 97 active+clean; 6.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:16:21.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:21 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2518145512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:21.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:21 smithi177 ceph-mon[35305]: pgmap v739: 97 pgs: 97 active+clean; 6.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:16:21.790 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:21 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2518145512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:21.790 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:21 smithi116 ceph-mon[31410]: pgmap v739: 97 pgs: 97 active+clean; 6.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:16:22.168 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-12T03:16:22.169 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-12T03:16:22.416 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:22.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:22 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:22.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:22 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:22.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.943+0000 7f903484e640 1 -- 172.21.15.116:0/661216644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9030108290 msgr2=0x7f903010ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:22.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.943+0000 7f903484e640 1 --2- 172.21.15.116:0/661216644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9030108290 0x7f903010ec30 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f9028009cd0 tx=0x7f902802a8e0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.943+0000 7f903484e640 1 -- 172.21.15.116:0/661216644 shutdown_connections 2024-09-12T03:16:22.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.943+0000 7f903484e640 1 --2- 172.21.15.116:0/661216644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9030108290 0x7f903010ec30 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.943+0000 7f903484e640 1 --2- 172.21.15.116:0/661216644 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9030103740 0x7f9030107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.943+0000 7f903484e640 1 --2- 172.21.15.116:0/661216644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9030102d90 0x7f9030103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.943+0000 7f903484e640 1 -- 172.21.15.116:0/661216644 >> 172.21.15.116:0/661216644 conn(0x7f90300fc7d0 msgr2=0x7f90300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:22.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.943+0000 7f903484e640 1 -- 172.21.15.116:0/661216644 shutdown_connections 2024-09-12T03:16:22.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.943+0000 7f903484e640 1 -- 172.21.15.116:0/661216644 wait complete. 2024-09-12T03:16:22.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.944+0000 7f903484e640 1 Processor -- start 2024-09-12T03:16:22.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.945+0000 7f903484e640 1 -- start start 2024-09-12T03:16:22.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.945+0000 7f903484e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9030102d90 0x7f9030198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:22.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.945+0000 7f903484e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9030103740 0x7f9030198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:22.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.945+0000 7f903484e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9030108290 0x7f903019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:22.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.945+0000 7f903484e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9030112090 con 0x7f9030108290 2024-09-12T03:16:22.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.945+0000 7f903484e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9030111f10 con 0x7f9030103740 2024-09-12T03:16:22.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.945+0000 7f903484e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9030112210 con 0x7f9030102d90 2024-09-12T03:16:22.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.945+0000 7f902ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9030108290 0x7f903019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:22.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.945+0000 7f902ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9030108290 0x7f903019ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39706/0 (socket says 172.21.15.116:39706) 2024-09-12T03:16:22.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.945+0000 7f902ed9d640 1 -- 172.21.15.116:0/2783320179 learned_addr learned my addr 172.21.15.116:0/2783320179 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:22.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.945+0000 7f9025d9b640 1 --2- 172.21.15.116:0/2783320179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9030103740 0x7f9030198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:22.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.945+0000 7f902e59c640 1 --2- 172.21.15.116:0/2783320179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9030102d90 0x7f9030198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:22.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.945+0000 7f902ed9d640 1 -- 172.21.15.116:0/2783320179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9030102d90 msgr2=0x7f9030198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:22.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.945+0000 7f902ed9d640 1 --2- 172.21.15.116:0/2783320179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9030102d90 0x7f9030198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.945+0000 7f902ed9d640 1 -- 172.21.15.116:0/2783320179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9030103740 msgr2=0x7f9030198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:22.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.945+0000 7f902ed9d640 1 --2- 172.21.15.116:0/2783320179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9030103740 0x7f9030198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.945+0000 7f902ed9d640 1 -- 172.21.15.116:0/2783320179 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f903019d520 con 0x7f9030108290 2024-09-12T03:16:22.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.945+0000 7f9025d9b640 1 --2- 172.21.15.116:0/2783320179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9030103740 0x7f9030198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:22.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.946+0000 7f902ed9d640 1 --2- 172.21.15.116:0/2783320179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9030108290 0x7f903019ce40 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f9028005cc0 tx=0x7f9028032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:22.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.946+0000 7f9027fff640 1 -- 172.21.15.116:0/2783320179 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9028004040 con 0x7f9030108290 2024-09-12T03:16:22.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.946+0000 7f903484e640 1 -- 172.21.15.116:0/2783320179 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f903019d7b0 con 0x7f9030108290 2024-09-12T03:16:22.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.946+0000 7f9027fff640 1 -- 172.21.15.116:0/2783320179 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90280046c0 con 0x7f9030108290 2024-09-12T03:16:22.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.946+0000 7f903484e640 1 -- 172.21.15.116:0/2783320179 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90301a50f0 con 0x7f9030108290 2024-09-12T03:16:22.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.946+0000 7f9027fff640 1 -- 172.21.15.116:0/2783320179 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9028030e00 con 0x7f9030108290 2024-09-12T03:16:22.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.948+0000 7f9027fff640 1 -- 172.21.15.116:0/2783320179 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f90280305a0 con 0x7f9030108290 2024-09-12T03:16:22.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.948+0000 7f9036c27740 1 Processor -- start 2024-09-12T03:16:22.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.948+0000 7f9036c27740 1 -- start start 2024-09-12T03:16:22.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.949+0000 7f9036c27740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ce6680760 0x561ce6680b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:22.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.949+0000 7f9036c27740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ce6681080 0x561ce668b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:22.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.949+0000 7f9036c27740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ce668b940 0x561ce668dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:22.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.949+0000 7f9036c27740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561ce65e62a0 con 0x561ce668b940 2024-09-12T03:16:22.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.949+0000 7f9036c27740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561ce658db10 con 0x561ce6680760 2024-09-12T03:16:22.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.949+0000 7f9036c27740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561ce65e2260 con 0x561ce6681080 2024-09-12T03:16:22.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.949+0000 7f9025d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ce668b940 0x561ce668dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:22.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.949+0000 7f9025d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ce668b940 0x561ce668dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39708/0 (socket says 172.21.15.116:39708) 2024-09-12T03:16:22.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.949+0000 7f9025d9b640 1 -- 172.21.15.116:0/1634841176 learned_addr learned my addr 172.21.15.116:0/1634841176 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:22.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.949+0000 7f9027fff640 1 --2- 172.21.15.116:0/2783320179 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f901c0a2b80 0x7f901c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:22.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.949+0000 7f902e59c640 1 --2- 172.21.15.116:0/1634841176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ce6681080 0x561ce668b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:22.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.949+0000 7f9025d9b640 1 --2- 172.21.15.116:0/1634841176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ce6680760 0x561ce6680b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:22.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.949+0000 7f902ed9d640 1 --2- 172.21.15.116:0/2783320179 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f901c0a2b80 0x7f901c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:22.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.949+0000 7f902e59c640 1 -- 172.21.15.116:0/1634841176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ce6680760 msgr2=0x561ce6680b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:22.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.949+0000 7f902e59c640 1 --2- 172.21.15.116:0/1634841176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ce6680760 0x561ce6680b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.949+0000 7f902e59c640 1 -- 172.21.15.116:0/1634841176 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ce668b940 msgr2=0x561ce668dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:22.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.949+0000 7f902e59c640 1 --2- 172.21.15.116:0/1634841176 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ce668b940 0x561ce668dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.949+0000 7f9027fff640 1 -- 172.21.15.116:0/2783320179 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f902802e030 con 0x7f9030108290 2024-09-12T03:16:22.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.949+0000 7f902e59c640 1 -- 172.21.15.116:0/1634841176 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ce6357b90 con 0x561ce6681080 2024-09-12T03:16:22.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.949+0000 7f902e59c640 1 --2- 172.21.15.116:0/1634841176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ce6681080 0x561ce668b400 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f902000ab40 tx=0x7f902000f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ee6bd9801ba0479 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:22.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.950+0000 7f902ed9d640 1 --2- 172.21.15.116:0/2783320179 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f901c0a2b80 0x7f901c0a5040 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9028006010 tx=0x7f902808bfc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:22.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.950+0000 7f9024d99640 1 -- 172.21.15.116:0/1634841176 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9020004020 con 0x561ce6681080 2024-09-12T03:16:22.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.950+0000 7f9024d99640 1 -- 172.21.15.116:0/1634841176 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90200041c0 con 0x561ce6681080 2024-09-12T03:16:22.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.950+0000 7f9024d99640 1 -- 172.21.15.116:0/1634841176 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9020018770 con 0x561ce6681080 2024-09-12T03:16:22.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.950+0000 7f9036c27740 1 -- 172.21.15.116:0/1634841176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ce6681080 msgr2=0x561ce668b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:22.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.950+0000 7f9036c27740 1 --2- 172.21.15.116:0/1634841176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ce6681080 0x561ce668b400 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f902000ab40 tx=0x7f902000f3c0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.950+0000 7f9036c27740 1 -- 172.21.15.116:0/1634841176 shutdown_connections 2024-09-12T03:16:22.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.950+0000 7f9036c27740 1 --2- 172.21.15.116:0/1634841176 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ce668b940 0x561ce668dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.950+0000 7f9036c27740 1 --2- 172.21.15.116:0/1634841176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ce6681080 0x561ce668b400 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.950+0000 7f9036c27740 1 --2- 172.21.15.116:0/1634841176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ce6680760 0x561ce6680b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.950+0000 7f9036c27740 1 -- 172.21.15.116:0/1634841176 >> 172.21.15.116:0/1634841176 conn(0x561ce667ccf0 msgr2=0x561ce667d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:22.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.950+0000 7f9036c27740 1 -- 172.21.15.116:0/1634841176 shutdown_connections 2024-09-12T03:16:22.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.950+0000 7f9036c27740 1 -- 172.21.15.116:0/1634841176 wait complete. 2024-09-12T03:16:22.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.951+0000 7f9036c27740 1 -- start start 2024-09-12T03:16:22.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.951+0000 7f9036c27740 1 Processor -- start 2024-09-12T03:16:22.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.951+0000 7f9036c27740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ce6680760 0x561ce66887a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:22.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.951+0000 7f9036c27740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ce6681080 0x561ce6688ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:22.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.951+0000 7f9036c27740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ce668b940 0x561ce6689220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:22.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.951+0000 7f9036c27740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561ce668e510 con 0x561ce6681080 2024-09-12T03:16:22.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.951+0000 7f9036c27740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561ce656e6e0 con 0x561ce6680760 2024-09-12T03:16:22.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.951+0000 7f9036c27740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561ce668e690 con 0x561ce668b940 2024-09-12T03:16:22.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.951+0000 7f9036c27740 10 client.0 _ll_register_callbacks cb 0x561ce6592a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:22.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.951+0000 7f902e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ce6681080 0x561ce6688ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:22.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.951+0000 7f9025d9b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ce6680760 0x561ce66887a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:22.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.951+0000 7f902e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ce6681080 0x561ce6688ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39724/0 (socket says 172.21.15.116:39724) 2024-09-12T03:16:22.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.951+0000 7f902e59c640 1 -- 172.21.15.116:0/2710921213 learned_addr learned my addr 172.21.15.116:0/2710921213 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:22.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.951+0000 7f9025d9b640 1 --2- 172.21.15.116:0/2710921213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ce668b940 0x561ce6689220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:22.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.952+0000 7f902e59c640 1 -- 172.21.15.116:0/2710921213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ce668b940 msgr2=0x561ce6689220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:22.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.952+0000 7f902e59c640 1 --2- 172.21.15.116:0/2710921213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ce668b940 0x561ce6689220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.952+0000 7f902e59c640 1 -- 172.21.15.116:0/2710921213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ce6680760 msgr2=0x561ce66887a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:22.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.952+0000 7f902e59c640 1 --2- 172.21.15.116:0/2710921213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ce6680760 0x561ce66887a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.952+0000 7f902e59c640 1 -- 172.21.15.116:0/2710921213 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ce6697980 con 0x561ce6681080 2024-09-12T03:16:22.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.952+0000 7f9025d9b640 1 --2- 172.21.15.116:0/2710921213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ce6680760 0x561ce66887a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:22.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.952+0000 7f902e59c640 1 --2- 172.21.15.116:0/2710921213 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ce6681080 0x561ce6688ce0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f902001a040 tx=0x7f9020003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=782a885d0014c563 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:22.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.952+0000 7f9015ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:16:22.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.952+0000 7f9015ffb640 1 -- 172.21.15.116:0/2710921213 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9020027020 con 0x561ce6681080 2024-09-12T03:16:22.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.952+0000 7f9015ffb640 1 -- 172.21.15.116:0/2710921213 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f902000fe00 con 0x561ce6681080 2024-09-12T03:16:22.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.952+0000 7f9036c27740 1 -- 172.21.15.116:0/2710921213 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90200041c0 con 0x561ce6681080 2024-09-12T03:16:22.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.952+0000 7f9015ffb640 1 -- 172.21.15.116:0/2710921213 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9020005040 con 0x561ce6681080 2024-09-12T03:16:22.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.952+0000 7f9036c27740 10 client.18900 fetch_fsmap learned FSMap version 25 2024-09-12T03:16:22.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.952+0000 7f9036c27740 1 -- 172.21.15.116:0/2710921213 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561ce669a310 con 0x561ce6681080 2024-09-12T03:16:22.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.952+0000 7f9015ffb640 1 -- 172.21.15.116:0/2710921213 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90200041c0 con 0x561ce6681080 2024-09-12T03:16:22.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.953+0000 7f9015ffb640 1 -- 172.21.15.116:0/2710921213 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f90200184b0 con 0x561ce6681080 2024-09-12T03:16:22.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.953+0000 7f9036c27740 10 client.18900 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:16:22.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.953+0000 7f9036c27740 10 client.18900 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:16:22.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.953+0000 7f9036c27740 20 client.18900 populate_metadata read hostname 'smithi116' 2024-09-12T03:16:22.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.953+0000 7f9036c27740 1 --2- 172.21.15.116:0/2710921213 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x561ce669b660 0x561ce669da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:22.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.953+0000 7f9036c27740 4 client.18900 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:16:22.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.953+0000 7f9036c27740 1 -- 172.21.15.116:0/2710921213 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ce65f39d0 con 0x561ce669b660 2024-09-12T03:16:22.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.953+0000 7f9025d9b640 1 --2- 172.21.15.116:0/2710921213 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x561ce669b660 0x561ce669da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:22.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.953+0000 7f9015ffb640 10 client.18900 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:22.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.953+0000 7f9025d9b640 1 --2- 172.21.15.116:0/2710921213 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x561ce669b660 0x561ce669da50 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70796a73a4939b80 server_cookie=92216ea059487d55 in_seq=0 out_seq=0 2024-09-12T03:16:22.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.954+0000 7f9015ffb640 1 -- 172.21.15.116:0/2710921213 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561ce65f39d0 con 0x561ce669b660 2024-09-12T03:16:22.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.954+0000 7f9015ffb640 10 client.18900 handle_command_reply: tid=0 2024-09-12T03:16:22.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.954+0000 7f9036c27740 1 client.18900 shutdown 2024-09-12T03:16:22.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.954+0000 7f9036c27740 1 -- 172.21.15.116:0/2710921213 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ce6681080 msgr2=0x561ce6688ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:22.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.954+0000 7f9036c27740 1 --2- 172.21.15.116:0/2710921213 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ce6681080 0x561ce6688ce0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f902001a040 tx=0x7f9020003c90 comp rx=0 tx=0).stop 2024-09-12T03:16:22.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.954+0000 7f9036c27740 1 -- 172.21.15.116:0/2710921213 shutdown_connections 2024-09-12T03:16:22.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.954+0000 7f9036c27740 1 --2- 172.21.15.116:0/2710921213 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x561ce669b660 0x561ce669da50 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.954+0000 7f9036c27740 1 --2- 172.21.15.116:0/2710921213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ce668b940 0x561ce6689220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.954+0000 7f9036c27740 1 --2- 172.21.15.116:0/2710921213 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ce6681080 0x561ce6688ce0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.954+0000 7f9015ffb640 0 client.18900 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:22.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.954+0000 7f9036c27740 1 --2- 172.21.15.116:0/2710921213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ce6680760 0x561ce66887a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.954+0000 7f9036c27740 1 -- 172.21.15.116:0/2710921213 >> 172.21.15.116:0/2710921213 conn(0x561ce667ccf0 msgr2=0x561ce667be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:22.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.954+0000 7f9036c27740 1 -- 172.21.15.116:0/2710921213 shutdown_connections 2024-09-12T03:16:22.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.954+0000 7f9036c27740 1 -- 172.21.15.116:0/2710921213 wait complete. 2024-09-12T03:16:22.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.954+0000 7f9036c27740 20 client.18900 trim_cache size 0 max 16384 2024-09-12T03:16:22.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.971+0000 7f9036c27740 1 Processor -- start 2024-09-12T03:16:22.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.971+0000 7f9036c27740 1 -- start start 2024-09-12T03:16:22.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.971+0000 7f9036c27740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ce6680760 0x561ce668a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:22.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.971+0000 7f9036c27740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ce6681080 0x561ce6687070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:22.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.971+0000 7f9036c27740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ce668b940 0x561ce66a3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:22.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.971+0000 7f9036c27740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f90301afd50 con 0x561ce6681080 2024-09-12T03:16:22.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.971+0000 7f9036c27740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f90301ad620 con 0x561ce6680760 2024-09-12T03:16:22.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.971+0000 7f9036c27740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f90301ab490 con 0x561ce668b940 2024-09-12T03:16:22.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.971+0000 7f902e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ce6681080 0x561ce6687070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:22.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.971+0000 7f902e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ce6681080 0x561ce6687070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39734/0 (socket says 172.21.15.116:39734) 2024-09-12T03:16:22.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.971+0000 7f902e59c640 1 -- 172.21.15.116:0/2181286735 learned_addr learned my addr 172.21.15.116:0/2181286735 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:22.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.971+0000 7f9025d9b640 1 --2- 172.21.15.116:0/2181286735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ce668b940 0x561ce66a3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:22.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.971+0000 7f9025d9b640 1 --2- 172.21.15.116:0/2181286735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ce6680760 0x561ce668a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:22.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.971+0000 7f902e59c640 1 -- 172.21.15.116:0/2181286735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ce668b940 msgr2=0x561ce66a3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:22.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.971+0000 7f902e59c640 1 --2- 172.21.15.116:0/2181286735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ce668b940 0x561ce66a3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.971+0000 7f902e59c640 1 -- 172.21.15.116:0/2181286735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ce6680760 msgr2=0x561ce668a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:22.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.972+0000 7f902e59c640 1 --2- 172.21.15.116:0/2181286735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ce6680760 0x561ce668a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.972+0000 7f902e59c640 1 -- 172.21.15.116:0/2181286735 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ce6680eb0 con 0x561ce6681080 2024-09-12T03:16:22.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.972+0000 7f9025d9b640 1 --2- 172.21.15.116:0/2181286735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ce6680760 0x561ce668a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:22.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.972+0000 7f902e59c640 1 --2- 172.21.15.116:0/2181286735 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ce6681080 0x561ce6687070 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f90200039f0 tx=0x7f902000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e89acef0771433 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:22.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.972+0000 7f9017fff640 1 -- 172.21.15.116:0/2181286735 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9020027040 con 0x561ce6681080 2024-09-12T03:16:22.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.972+0000 7f9017fff640 1 -- 172.21.15.116:0/2181286735 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f902000a700 con 0x561ce6681080 2024-09-12T03:16:22.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.972+0000 7f9017fff640 1 -- 172.21.15.116:0/2181286735 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9020018440 con 0x561ce6681080 2024-09-12T03:16:22.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.972+0000 7f9036c27740 1 -- 172.21.15.116:0/2181286735 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ce6681080 msgr2=0x561ce6687070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:22.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.972+0000 7f9036c27740 1 --2- 172.21.15.116:0/2181286735 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ce6681080 0x561ce6687070 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f90200039f0 tx=0x7f902000f3c0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.973+0000 7f9036c27740 1 -- 172.21.15.116:0/2181286735 shutdown_connections 2024-09-12T03:16:22.996 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.973+0000 7f9036c27740 1 --2- 172.21.15.116:0/2181286735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ce668b940 0x561ce66a3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.973+0000 7f9036c27740 1 --2- 172.21.15.116:0/2181286735 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ce6681080 0x561ce6687070 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.973+0000 7f9036c27740 1 --2- 172.21.15.116:0/2181286735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ce6680760 0x561ce668a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.973+0000 7f9036c27740 1 -- 172.21.15.116:0/2181286735 >> 172.21.15.116:0/2181286735 conn(0x561ce667ccf0 msgr2=0x561ce667be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:22.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.973+0000 7f9036c27740 1 -- 172.21.15.116:0/2181286735 shutdown_connections 2024-09-12T03:16:22.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.973+0000 7f9036c27740 1 -- 172.21.15.116:0/2181286735 wait complete. 2024-09-12T03:16:22.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.973+0000 7f9036c27740 1 -- start start 2024-09-12T03:16:22.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.973+0000 7f9036c27740 1 Processor -- start 2024-09-12T03:16:22.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.973+0000 7f9036c27740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ce6680760 0x561ce668ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:22.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.973+0000 7f9036c27740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ce6681080 0x561ce6690450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:22.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.973+0000 7f9036c27740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ce668b940 0x561ce66ff060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:22.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.973+0000 7f9036c27740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f90301abae0 con 0x561ce6681080 2024-09-12T03:16:22.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.973+0000 7f9036c27740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f90301acf50 con 0x561ce668b940 2024-09-12T03:16:22.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.973+0000 7f9036c27740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f90301ac1b0 con 0x561ce6680760 2024-09-12T03:16:22.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.973+0000 7f9036c27740 10 client.0 _ll_register_callbacks cb 0x7f902000a9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:22.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.973+0000 7f902e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ce6681080 0x561ce6690450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:22.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.973+0000 7f902e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ce6681080 0x561ce6690450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39746/0 (socket says 172.21.15.116:39746) 2024-09-12T03:16:22.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.973+0000 7f902e59c640 1 -- 172.21.15.116:0/4150250733 learned_addr learned my addr 172.21.15.116:0/4150250733 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:22.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.973+0000 7f9025d9b640 1 --2- 172.21.15.116:0/4150250733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ce6680760 0x561ce668ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:22.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.973+0000 7f9025d9b640 1 --2- 172.21.15.116:0/4150250733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ce668b940 0x561ce66ff060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:22.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.974+0000 7f902e59c640 1 -- 172.21.15.116:0/4150250733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ce6680760 msgr2=0x561ce668ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:22.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.974+0000 7f902e59c640 1 --2- 172.21.15.116:0/4150250733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ce6680760 0x561ce668ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.974+0000 7f902e59c640 1 -- 172.21.15.116:0/4150250733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ce668b940 msgr2=0x561ce66ff060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:22.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.974+0000 7f902e59c640 1 --2- 172.21.15.116:0/4150250733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ce668b940 0x561ce66ff060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.974+0000 7f902e59c640 1 -- 172.21.15.116:0/4150250733 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ce66ff6d0 con 0x561ce6681080 2024-09-12T03:16:22.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.974+0000 7f902e59c640 1 --2- 172.21.15.116:0/4150250733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ce6681080 0x561ce6690450 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f9020018900 tx=0x7f9020003ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c239deb5c1e17a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:22.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.974+0000 7f9015ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:16:22.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.974+0000 7f9015ffb640 1 -- 172.21.15.116:0/4150250733 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9020027040 con 0x561ce6681080 2024-09-12T03:16:22.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.974+0000 7f9015ffb640 1 -- 172.21.15.116:0/4150250733 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f90200051e0 con 0x561ce6681080 2024-09-12T03:16:22.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.974+0000 7f9036c27740 1 -- 172.21.15.116:0/4150250733 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f902000a700 con 0x561ce6681080 2024-09-12T03:16:22.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.974+0000 7f9015ffb640 1 -- 172.21.15.116:0/4150250733 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9020045050 con 0x561ce6681080 2024-09-12T03:16:22.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.974+0000 7f9036c27740 10 client.18906 fetch_fsmap learned FSMap version 25 2024-09-12T03:16:22.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.974+0000 7f9036c27740 1 -- 172.21.15.116:0/4150250733 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561ce6706a30 con 0x561ce6681080 2024-09-12T03:16:22.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.974+0000 7f9015ffb640 1 -- 172.21.15.116:0/4150250733 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f902000a700 con 0x561ce6681080 2024-09-12T03:16:22.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.975+0000 7f9015ffb640 1 -- 172.21.15.116:0/4150250733 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9020035030 con 0x561ce6681080 2024-09-12T03:16:22.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.975+0000 7f9036c27740 10 client.18906 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:16:22.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.975+0000 7f9036c27740 10 client.18906 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:16:22.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.975+0000 7f9036c27740 20 client.18906 populate_metadata read hostname 'smithi116' 2024-09-12T03:16:22.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.975+0000 7f9036c27740 1 --2- 172.21.15.116:0/4150250733 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x561ce669b660 0x561ce67091b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:22.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.975+0000 7f9036c27740 4 client.18906 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-12T03:16:22.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.975+0000 7f9036c27740 1 -- 172.21.15.116:0/4150250733 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x561ce65edd20 con 0x561ce669b660 2024-09-12T03:16:23.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.975+0000 7f9025d9b640 1 --2- 172.21.15.116:0/4150250733 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x561ce669b660 0x561ce67091b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:23.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.975+0000 7f9025d9b640 1 --2- 172.21.15.116:0/4150250733 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x561ce669b660 0x561ce67091b0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d7c6b5d10e9795 server_cookie=e64ff054fb1ab5f3 in_seq=0 out_seq=0 2024-09-12T03:16:23.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.975+0000 7f9015ffb640 10 client.18906 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:23.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.983+0000 7f9015ffb640 1 -- 172.21.15.116:0/4150250733 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561ce65edd20 con 0x561ce669b660 2024-09-12T03:16:23.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.983+0000 7f9015ffb640 10 client.18906 handle_command_reply: tid=0 2024-09-12T03:16:23.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.983+0000 7f9036c27740 1 client.18906 shutdown 2024-09-12T03:16:23.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.983+0000 7f9036c27740 1 -- 172.21.15.116:0/4150250733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ce6681080 msgr2=0x561ce6690450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:23.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.983+0000 7f9036c27740 1 --2- 172.21.15.116:0/4150250733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ce6681080 0x561ce6690450 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f9020018900 tx=0x7f9020003ea0 comp rx=0 tx=0).stop 2024-09-12T03:16:23.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.983+0000 7f9036c27740 1 -- 172.21.15.116:0/4150250733 shutdown_connections 2024-09-12T03:16:23.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.983+0000 7f9036c27740 1 --2- 172.21.15.116:0/4150250733 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x561ce669b660 0x561ce67091b0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:23.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.983+0000 7f9036c27740 1 --2- 172.21.15.116:0/4150250733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561ce668b940 0x561ce66ff060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:23.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.983+0000 7f9015ffb640 0 client.18906 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:23.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.983+0000 7f9036c27740 1 --2- 172.21.15.116:0/4150250733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ce6681080 0x561ce6690450 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:23.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.983+0000 7f9036c27740 1 --2- 172.21.15.116:0/4150250733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ce6680760 0x561ce668ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:23.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.983+0000 7f9036c27740 1 -- 172.21.15.116:0/4150250733 >> 172.21.15.116:0/4150250733 conn(0x561ce667ccf0 msgr2=0x561ce6688f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:23.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.983+0000 7f9036c27740 1 -- 172.21.15.116:0/4150250733 shutdown_connections 2024-09-12T03:16:23.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.983+0000 7f9036c27740 1 -- 172.21.15.116:0/4150250733 wait complete. 2024-09-12T03:16:23.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.983+0000 7f9036c27740 20 client.18906 trim_cache size 0 max 16384 2024-09-12T03:16:23.001 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:16:23.001 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 25, 2024-09-12T03:16:23.001 INFO:teuthology.orchestra.run.smithi116.stdout: "leader": 24494, 2024-09-12T03:16:23.002 INFO:teuthology.orchestra.run.smithi116.stdout: "set_version": 50, 2024-09-12T03:16:23.002 INFO:teuthology.orchestra.run.smithi116.stdout: "sets": { 2024-09-12T03:16:23.002 INFO:teuthology.orchestra.run.smithi116.stdout: "a6a0d1d9": { 2024-09-12T03:16:23.002 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 50, 2024-09-12T03:16:23.002 INFO:teuthology.orchestra.run.smithi116.stdout: "age_ref": 0.0, 2024-09-12T03:16:23.002 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:16:23.002 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "QUIESCED", 2024-09-12T03:16:23.002 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:16:23.002 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:16:23.002 INFO:teuthology.orchestra.run.smithi116.stdout: "timeout": 90.0, 2024-09-12T03:16:23.002 INFO:teuthology.orchestra.run.smithi116.stdout: "expiration": 150.0, 2024-09-12T03:16:23.002 INFO:teuthology.orchestra.run.smithi116.stdout: "members": { 2024-09-12T03:16:23.002 INFO:teuthology.orchestra.run.smithi116.stdout: "file:/": { 2024-09-12T03:16:23.002 INFO:teuthology.orchestra.run.smithi116.stdout: "excluded": false, 2024-09-12T03:16:23.003 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:16:23.003 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "QUIESCED", 2024-09-12T03:16:23.003 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:16:23.003 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:16:23.003 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:16:23.003 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:16:23.003 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:16:23.003 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:16:23.003 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:16:23.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.984+0000 7f903484e640 1 -- 172.21.15.116:0/2783320179 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f901c0a2b80 msgr2=0x7f901c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:23.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.984+0000 7f903484e640 1 --2- 172.21.15.116:0/2783320179 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f901c0a2b80 0x7f901c0a5040 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9028006010 tx=0x7f902808bfc0 comp rx=0 tx=0).stop 2024-09-12T03:16:23.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.984+0000 7f903484e640 1 -- 172.21.15.116:0/2783320179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9030108290 msgr2=0x7f903019ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:23.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.984+0000 7f903484e640 1 --2- 172.21.15.116:0/2783320179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9030108290 0x7f903019ce40 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f9028005cc0 tx=0x7f9028032f40 comp rx=0 tx=0).stop 2024-09-12T03:16:23.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.984+0000 7f903484e640 1 -- 172.21.15.116:0/2783320179 shutdown_connections 2024-09-12T03:16:23.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.984+0000 7f903484e640 1 --2- 172.21.15.116:0/2783320179 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f901c0a2b80 0x7f901c0a5040 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:23.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.984+0000 7f903484e640 1 --2- 172.21.15.116:0/2783320179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9030108290 0x7f903019ce40 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:23.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.984+0000 7f903484e640 1 --2- 172.21.15.116:0/2783320179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9030103740 0x7f9030198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:23.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.984+0000 7f903484e640 1 --2- 172.21.15.116:0/2783320179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9030102d90 0x7f9030198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:23.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.984+0000 7f903484e640 1 -- 172.21.15.116:0/2783320179 >> 172.21.15.116:0/2783320179 conn(0x7f90300fc7d0 msgr2=0x7f9030105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:23.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.984+0000 7f903484e640 1 -- 172.21.15.116:0/2783320179 shutdown_connections 2024-09-12T03:16:23.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:22.984+0000 7f903484e640 1 -- 172.21.15.116:0/2783320179 wait complete. 2024-09-12T03:16:23.005 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:22 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:23.755 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:23 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:23.755 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:23 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:16:23.755 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:23 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:16:23.755 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:23 smithi116 ceph-mon[31410]: pgmap v740: 97 pgs: 97 active+clean; 6.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:16:23.783 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a6a0d1d9', quiesce took 3.4 seconds. Will release after: 26.6 2024-09-12T03:16:23.918 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:16:23.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:23 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:23.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:23 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:16:23.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:23 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:16:23.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:23 smithi183 ceph-mon[35535]: pgmap v740: 97 pgs: 97 active+clean; 6.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:16:23.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:23 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:23.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:23 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:16:23.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:23 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:16:23.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:23 smithi177 ceph-mon[35305]: pgmap v740: 97 pgs: 97 active+clean; 6.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:16:24.165 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:24.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.664+0000 7f2851b1b640 1 -- 172.21.15.116:0/2217478463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f284c102790 msgr2=0x7f284c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:24.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.664+0000 7f2851b1b640 1 --2- 172.21.15.116:0/2217478463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f284c102790 0x7f284c1067e0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f2840009900 tx=0x7f284002a410 comp rx=0 tx=0).stop 2024-09-12T03:16:24.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.664+0000 7f2851b1b640 1 -- 172.21.15.116:0/2217478463 shutdown_connections 2024-09-12T03:16:24.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.664+0000 7f2851b1b640 1 --2- 172.21.15.116:0/2217478463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f284c106f10 0x7f284c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.664+0000 7f2851b1b640 1 --2- 172.21.15.116:0/2217478463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f284c102790 0x7f284c1067e0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.664+0000 7f2851b1b640 1 --2- 172.21.15.116:0/2217478463 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f284c101de0 0x7f284c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.664+0000 7f2851b1b640 1 -- 172.21.15.116:0/2217478463 >> 172.21.15.116:0/2217478463 conn(0x7f284c0fd8c0 msgr2=0x7f284c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:24.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.664+0000 7f2851b1b640 1 -- 172.21.15.116:0/2217478463 shutdown_connections 2024-09-12T03:16:24.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.664+0000 7f2851b1b640 1 -- 172.21.15.116:0/2217478463 wait complete. 2024-09-12T03:16:24.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.665+0000 7f2851b1b640 1 Processor -- start 2024-09-12T03:16:24.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.665+0000 7f2851b1b640 1 -- start start 2024-09-12T03:16:24.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.665+0000 7f2851b1b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f284c101de0 0x7f284c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:24.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.665+0000 7f2851b1b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f284c102790 0x7f284c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:24.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.665+0000 7f2851b1b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f284c106f10 0x7f284c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:24.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.665+0000 7f2851b1b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f284c111f70 con 0x7f284c106f10 2024-09-12T03:16:24.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.665+0000 7f2851b1b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f284c111df0 con 0x7f284c101de0 2024-09-12T03:16:24.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.665+0000 7f2851b1b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1120f0 con 0x7f284c102790 2024-09-12T03:16:24.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.665+0000 7f284bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f284c106f10 0x7f284c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:24.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.665+0000 7f284bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f284c106f10 0x7f284c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39768/0 (socket says 172.21.15.116:39768) 2024-09-12T03:16:24.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.665+0000 7f284bfff640 1 -- 172.21.15.116:0/632974635 learned_addr learned my addr 172.21.15.116:0/632974635 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:24.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.665+0000 7f284affd640 1 --2- 172.21.15.116:0/632974635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f284c102790 0x7f284c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:24.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.665+0000 7f284b7fe640 1 --2- 172.21.15.116:0/632974635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f284c101de0 0x7f284c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:24.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.666+0000 7f284bfff640 1 -- 172.21.15.116:0/632974635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f284c102790 msgr2=0x7f284c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:24.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.666+0000 7f284bfff640 1 --2- 172.21.15.116:0/632974635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f284c102790 0x7f284c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.666+0000 7f284bfff640 1 -- 172.21.15.116:0/632974635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f284c101de0 msgr2=0x7f284c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:24.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.666+0000 7f284bfff640 1 --2- 172.21.15.116:0/632974635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f284c101de0 0x7f284c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.666+0000 7f284bfff640 1 -- 172.21.15.116:0/632974635 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284c1a6870 con 0x7f284c106f10 2024-09-12T03:16:24.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.666+0000 7f284bfff640 1 --2- 172.21.15.116:0/632974635 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f284c106f10 0x7f284c074f70 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f284400cce0 tx=0x7f284400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:24.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.666+0000 7f2848ff9640 1 -- 172.21.15.116:0/632974635 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284400f450 con 0x7f284c106f10 2024-09-12T03:16:24.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.666+0000 7f2848ff9640 1 -- 172.21.15.116:0/632974635 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f284400fad0 con 0x7f284c106f10 2024-09-12T03:16:24.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.666+0000 7f2851b1b640 1 -- 172.21.15.116:0/632974635 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284c1a6b60 con 0x7f284c106f10 2024-09-12T03:16:24.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.666+0000 7f2848ff9640 1 -- 172.21.15.116:0/632974635 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2844018c10 con 0x7f284c106f10 2024-09-12T03:16:24.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.666+0000 7f2851b1b640 1 -- 172.21.15.116:0/632974635 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f284c1a7020 con 0x7f284c106f10 2024-09-12T03:16:24.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.668+0000 7f2848ff9640 1 -- 172.21.15.116:0/632974635 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f284400f5f0 con 0x7f284c106f10 2024-09-12T03:16:24.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.669+0000 7f2853ef4740 1 Processor -- start 2024-09-12T03:16:24.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.669+0000 7f2853ef4740 1 -- start start 2024-09-12T03:16:24.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.669+0000 7f2853ef4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4bdb9b4a0 0x55e4bdb9b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:24.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.669+0000 7f2853ef4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4bdb9bdc0 0x55e4bdba6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:24.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.669+0000 7f2853ef4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4bdba6680 0x55e4bdba8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:24.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.669+0000 7f2848ff9640 1 --2- 172.21.15.116:0/632974635 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f28340a2b80 0x7f28340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:24.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.669+0000 7f2853ef4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e4bdb012e0 con 0x55e4bdb9b4a0 2024-09-12T03:16:24.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.669+0000 7f2853ef4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e4bdaa8ce0 con 0x55e4bdb9bdc0 2024-09-12T03:16:24.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.669+0000 7f2853ef4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e4bdafd2a0 con 0x55e4bdba6680 2024-09-12T03:16:24.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.669+0000 7f284affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4bdb9b4a0 0x55e4bdb9b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:24.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.669+0000 7f284affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4bdb9b4a0 0x55e4bdb9b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39784/0 (socket says 172.21.15.116:39784) 2024-09-12T03:16:24.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.669+0000 7f284affd640 1 -- 172.21.15.116:0/3827171202 learned_addr learned my addr 172.21.15.116:0/3827171202 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:24.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.669+0000 7f2848ff9640 1 -- 172.21.15.116:0/632974635 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f28440cf690 con 0x7f284c106f10 2024-09-12T03:16:24.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.669+0000 7f284bfff640 1 --2- 172.21.15.116:0/632974635 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f28340a2b80 0x7f28340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:24.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.669+0000 7f284affd640 1 --2- 172.21.15.116:0/3827171202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4bdba6680 0x55e4bdba8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:24.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.669+0000 7f284b7fe640 1 --2- 172.21.15.116:0/3827171202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4bdb9bdc0 0x55e4bdba6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:24.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.669+0000 7f284affd640 1 -- 172.21.15.116:0/3827171202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4bdba6680 msgr2=0x55e4bdba8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:24.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.669+0000 7f284affd640 1 --2- 172.21.15.116:0/3827171202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4bdba6680 0x55e4bdba8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.669+0000 7f284affd640 1 -- 172.21.15.116:0/3827171202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4bdb9bdc0 msgr2=0x55e4bdba6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:24.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.669+0000 7f284affd640 1 --2- 172.21.15.116:0/3827171202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4bdb9bdc0 0x55e4bdba6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.669+0000 7f284affd640 1 -- 172.21.15.116:0/3827171202 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4bda8d440 con 0x55e4bdb9b4a0 2024-09-12T03:16:24.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.670+0000 7f284bfff640 1 --2- 172.21.15.116:0/632974635 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f28340a2b80 0x7f28340a5040 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f284400ccb0 tx=0x7f284400af30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:24.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.670+0000 7f284affd640 1 --2- 172.21.15.116:0/3827171202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4bdb9b4a0 0x55e4bdb9b880 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f28400096c0 tx=0x7f2840031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8f1b37a64da14a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:24.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.670+0000 7f2831ffb640 1 -- 172.21.15.116:0/3827171202 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2840044030 con 0x55e4bdb9b4a0 2024-09-12T03:16:24.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.670+0000 7f2831ffb640 1 -- 172.21.15.116:0/3827171202 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2840005020 con 0x55e4bdb9b4a0 2024-09-12T03:16:24.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.670+0000 7f2831ffb640 1 -- 172.21.15.116:0/3827171202 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2840035040 con 0x55e4bdb9b4a0 2024-09-12T03:16:24.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.670+0000 7f2853ef4740 1 -- 172.21.15.116:0/3827171202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4bdb9b4a0 msgr2=0x55e4bdb9b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:24.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.670+0000 7f2853ef4740 1 --2- 172.21.15.116:0/3827171202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4bdb9b4a0 0x55e4bdb9b880 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f28400096c0 tx=0x7f2840031400 comp rx=0 tx=0).stop 2024-09-12T03:16:24.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.671+0000 7f2853ef4740 1 -- 172.21.15.116:0/3827171202 shutdown_connections 2024-09-12T03:16:24.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.671+0000 7f2853ef4740 1 --2- 172.21.15.116:0/3827171202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4bdba6680 0x55e4bdba8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.671+0000 7f2853ef4740 1 --2- 172.21.15.116:0/3827171202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4bdb9bdc0 0x55e4bdba6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.671+0000 7f2853ef4740 1 --2- 172.21.15.116:0/3827171202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4bdb9b4a0 0x55e4bdb9b880 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.671+0000 7f2853ef4740 1 -- 172.21.15.116:0/3827171202 >> 172.21.15.116:0/3827171202 conn(0x55e4bdb97a30 msgr2=0x55e4bdb97e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:24.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.671+0000 7f2853ef4740 1 -- 172.21.15.116:0/3827171202 shutdown_connections 2024-09-12T03:16:24.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.671+0000 7f2853ef4740 1 -- 172.21.15.116:0/3827171202 wait complete. 2024-09-12T03:16:24.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.671+0000 7f2853ef4740 1 -- start start 2024-09-12T03:16:24.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.671+0000 7f2853ef4740 1 Processor -- start 2024-09-12T03:16:24.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.671+0000 7f2853ef4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4bdb9b4a0 0x55e4bdba5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:24.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.671+0000 7f2853ef4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4bdb9bdc0 0x55e4bdba3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:24.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.671+0000 7f2853ef4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4bdba6680 0x55e4bdba3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:24.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.671+0000 7f2853ef4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e4bdba9250 con 0x55e4bdb9bdc0 2024-09-12T03:16:24.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.671+0000 7f2853ef4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e4bdabc050 con 0x55e4bdb9b4a0 2024-09-12T03:16:24.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.671+0000 7f2853ef4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e4bdba93d0 con 0x55e4bdba6680 2024-09-12T03:16:24.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.671+0000 7f2853ef4740 10 client.0 _ll_register_callbacks cb 0x55e4bdafc160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:24.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.671+0000 7f284b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4bdb9bdc0 0x55e4bdba3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:24.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.671+0000 7f284b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4bdb9bdc0 0x55e4bdba3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39796/0 (socket says 172.21.15.116:39796) 2024-09-12T03:16:24.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.671+0000 7f284b7fe640 1 -- 172.21.15.116:0/1865465890 learned_addr learned my addr 172.21.15.116:0/1865465890 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:24.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.672+0000 7f284affd640 1 --2- 172.21.15.116:0/1865465890 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4bdb9b4a0 0x55e4bdba5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:24.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.672+0000 7f284affd640 1 --2- 172.21.15.116:0/1865465890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4bdba6680 0x55e4bdba3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:24.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.672+0000 7f284b7fe640 1 -- 172.21.15.116:0/1865465890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4bdba6680 msgr2=0x55e4bdba3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:24.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.672+0000 7f284b7fe640 1 --2- 172.21.15.116:0/1865465890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4bdba6680 0x55e4bdba3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.672+0000 7f284b7fe640 1 -- 172.21.15.116:0/1865465890 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4bdb9b4a0 msgr2=0x55e4bdba5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:24.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.672+0000 7f284b7fe640 1 --2- 172.21.15.116:0/1865465890 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4bdb9b4a0 0x55e4bdba5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.672+0000 7f284b7fe640 1 -- 172.21.15.116:0/1865465890 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4bd8723c0 con 0x55e4bdb9bdc0 2024-09-12T03:16:24.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.672+0000 7f284affd640 1 --2- 172.21.15.116:0/1865465890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4bdba6680 0x55e4bdba3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:24.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.672+0000 7f284b7fe640 1 --2- 172.21.15.116:0/1865465890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4bdb9bdc0 0x55e4bdba3220 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f283c00bed0 tx=0x7f283c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed03ba016b4cf91 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:24.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.672+0000 7f282affd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:16:24.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.672+0000 7f282affd640 1 -- 172.21.15.116:0/1865465890 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c004220 con 0x55e4bdb9bdc0 2024-09-12T03:16:24.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.672+0000 7f282affd640 1 -- 172.21.15.116:0/1865465890 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f283c0043c0 con 0x55e4bdb9bdc0 2024-09-12T03:16:24.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.672+0000 7f2853ef4740 1 -- 172.21.15.116:0/1865465890 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2840005020 con 0x55e4bdb9bdc0 2024-09-12T03:16:24.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.672+0000 7f282affd640 1 -- 172.21.15.116:0/1865465890 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f283c0035f0 con 0x55e4bdb9bdc0 2024-09-12T03:16:24.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.672+0000 7f2853ef4740 10 client.18918 fetch_fsmap learned FSMap version 25 2024-09-12T03:16:24.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.672+0000 7f2853ef4740 1 -- 172.21.15.116:0/1865465890 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e4bdbb4c20 con 0x55e4bdb9bdc0 2024-09-12T03:16:24.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.672+0000 7f282affd640 1 -- 172.21.15.116:0/1865465890 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2840005020 con 0x55e4bdb9bdc0 2024-09-12T03:16:24.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.673+0000 7f282affd640 1 -- 172.21.15.116:0/1865465890 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f283c01ba50 con 0x55e4bdb9bdc0 2024-09-12T03:16:24.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.673+0000 7f2853ef4740 10 client.18918 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:16:24.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.673+0000 7f2853ef4740 10 client.18918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:16:24.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.673+0000 7f2853ef4740 20 client.18918 populate_metadata read hostname 'smithi116' 2024-09-12T03:16:24.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.673+0000 7f2853ef4740 1 --2- 172.21.15.116:0/1865465890 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e4bdbb60e0 0x55e4bdbb8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:24.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.673+0000 7f2853ef4740 4 client.18918 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:16:24.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.673+0000 7f2853ef4740 1 -- 172.21.15.116:0/1865465890 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4bdb0e850 con 0x55e4bdbb60e0 2024-09-12T03:16:24.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.673+0000 7f284affd640 1 --2- 172.21.15.116:0/1865465890 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e4bdbb60e0 0x55e4bdbb8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:24.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.673+0000 7f284affd640 1 --2- 172.21.15.116:0/1865465890 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e4bdbb60e0 0x55e4bdbb8540 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cd90325e358c0d2 server_cookie=9df77c424e22604d in_seq=0 out_seq=0 2024-09-12T03:16:24.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.673+0000 7f282affd640 10 client.18918 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:24.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.674+0000 7f282affd640 1 -- 172.21.15.116:0/1865465890 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e4bdb0e850 con 0x55e4bdbb60e0 2024-09-12T03:16:24.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.674+0000 7f282affd640 10 client.18918 handle_command_reply: tid=0 2024-09-12T03:16:24.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.674+0000 7f2853ef4740 1 client.18918 shutdown 2024-09-12T03:16:24.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.674+0000 7f2853ef4740 1 -- 172.21.15.116:0/1865465890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4bdb9bdc0 msgr2=0x55e4bdba3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:24.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.674+0000 7f2853ef4740 1 --2- 172.21.15.116:0/1865465890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4bdb9bdc0 0x55e4bdba3220 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f283c00bed0 tx=0x7f283c0117b0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.674+0000 7f2853ef4740 1 -- 172.21.15.116:0/1865465890 shutdown_connections 2024-09-12T03:16:24.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.674+0000 7f2853ef4740 1 --2- 172.21.15.116:0/1865465890 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e4bdbb60e0 0x55e4bdbb8540 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.674+0000 7f2853ef4740 1 --2- 172.21.15.116:0/1865465890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4bdba6680 0x55e4bdba3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.674+0000 7f2853ef4740 1 --2- 172.21.15.116:0/1865465890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4bdb9bdc0 0x55e4bdba3220 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.674+0000 7f2853ef4740 1 --2- 172.21.15.116:0/1865465890 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4bdb9b4a0 0x55e4bdba5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.674+0000 7f2853ef4740 1 -- 172.21.15.116:0/1865465890 >> 172.21.15.116:0/1865465890 conn(0x55e4bdb97a30 msgr2=0x55e4bdb9d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:24.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.674+0000 7f282affd640 0 client.18918 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:24.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.674+0000 7f2853ef4740 1 -- 172.21.15.116:0/1865465890 shutdown_connections 2024-09-12T03:16:24.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.674+0000 7f2853ef4740 1 -- 172.21.15.116:0/1865465890 wait complete. 2024-09-12T03:16:24.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.675+0000 7f2853ef4740 20 client.18918 trim_cache size 0 max 16384 2024-09-12T03:16:24.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.692+0000 7f2853ef4740 1 Processor -- start 2024-09-12T03:16:24.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.692+0000 7f2853ef4740 1 -- start start 2024-09-12T03:16:24.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.692+0000 7f2853ef4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4bdb9b4a0 0x55e4bdbb24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:24.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.692+0000 7f2853ef4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4bdb9bdc0 0x55e4bdbbde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:24.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.692+0000 7f2853ef4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4bdba6680 0x55e4bdbbe350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:24.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.692+0000 7f2853ef4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1b1e40 con 0x55e4bdb9b4a0 2024-09-12T03:16:24.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.692+0000 7f2853ef4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1af710 con 0x55e4bdb9bdc0 2024-09-12T03:16:24.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.692+0000 7f2853ef4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1ad560 con 0x55e4bdba6680 2024-09-12T03:16:24.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.692+0000 7f284affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4bdb9b4a0 0x55e4bdbb24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:24.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.692+0000 7f284affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4bdb9b4a0 0x55e4bdbb24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39806/0 (socket says 172.21.15.116:39806) 2024-09-12T03:16:24.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.692+0000 7f284affd640 1 -- 172.21.15.116:0/3415526457 learned_addr learned my addr 172.21.15.116:0/3415526457 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:24.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.692+0000 7f284b7fe640 1 --2- 172.21.15.116:0/3415526457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4bdb9bdc0 0x55e4bdbbde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:24.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.692+0000 7f284affd640 1 -- 172.21.15.116:0/3415526457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4bdba6680 msgr2=0x55e4bdbbe350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:24.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.692+0000 7f284affd640 1 --2- 172.21.15.116:0/3415526457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4bdba6680 0x55e4bdbbe350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.692+0000 7f284affd640 1 -- 172.21.15.116:0/3415526457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4bdb9bdc0 msgr2=0x55e4bdbbde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:24.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.692+0000 7f284affd640 1 --2- 172.21.15.116:0/3415526457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4bdb9bdc0 0x55e4bdbbde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.692+0000 7f284affd640 1 -- 172.21.15.116:0/3415526457 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4bdb97860 con 0x55e4bdb9b4a0 2024-09-12T03:16:24.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.692+0000 7f284b7fe640 1 --2- 172.21.15.116:0/3415526457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4bdb9bdc0 0x55e4bdbbde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:24.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.693+0000 7f284affd640 1 --2- 172.21.15.116:0/3415526457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4bdb9b4a0 0x55e4bdbb24b0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f28400096c0 tx=0x7f28400314c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45bb4d20009cf960 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:24.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.693+0000 7f28317fa640 1 -- 172.21.15.116:0/3415526457 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2840044040 con 0x55e4bdb9b4a0 2024-09-12T03:16:24.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.693+0000 7f28317fa640 1 -- 172.21.15.116:0/3415526457 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2840045040 con 0x55e4bdb9b4a0 2024-09-12T03:16:24.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.693+0000 7f28317fa640 1 -- 172.21.15.116:0/3415526457 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2840035040 con 0x55e4bdb9b4a0 2024-09-12T03:16:24.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.694+0000 7f2853ef4740 1 -- 172.21.15.116:0/3415526457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4bdb9b4a0 msgr2=0x55e4bdbb24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:24.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.694+0000 7f2853ef4740 1 --2- 172.21.15.116:0/3415526457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4bdb9b4a0 0x55e4bdbb24b0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f28400096c0 tx=0x7f28400314c0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.694+0000 7f2853ef4740 1 -- 172.21.15.116:0/3415526457 shutdown_connections 2024-09-12T03:16:24.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.694+0000 7f2853ef4740 1 --2- 172.21.15.116:0/3415526457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4bdba6680 0x55e4bdbbe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.694+0000 7f2853ef4740 1 --2- 172.21.15.116:0/3415526457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4bdb9bdc0 0x55e4bdbbde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.694+0000 7f2853ef4740 1 --2- 172.21.15.116:0/3415526457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4bdb9b4a0 0x55e4bdbb24b0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.694+0000 7f2853ef4740 1 -- 172.21.15.116:0/3415526457 >> 172.21.15.116:0/3415526457 conn(0x55e4bdb97a30 msgr2=0x55e4bdba2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:24.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.694+0000 7f2853ef4740 1 -- 172.21.15.116:0/3415526457 shutdown_connections 2024-09-12T03:16:24.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.694+0000 7f2853ef4740 1 -- 172.21.15.116:0/3415526457 wait complete. 2024-09-12T03:16:24.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.694+0000 7f2853ef4740 1 -- start start 2024-09-12T03:16:24.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.694+0000 7f2853ef4740 1 Processor -- start 2024-09-12T03:16:24.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.695+0000 7f2853ef4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4bdb9b4a0 0x55e4bdbabb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:24.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.695+0000 7f2853ef4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4bdb9bdc0 0x55e4bdbac0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:24.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.695+0000 7f2853ef4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4bdba6680 0x55e4bdc1ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:24.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.695+0000 7f2853ef4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1adbd0 con 0x55e4bdb9b4a0 2024-09-12T03:16:24.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.695+0000 7f2853ef4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1af040 con 0x55e4bdb9bdc0 2024-09-12T03:16:24.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.695+0000 7f2853ef4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1ae2a0 con 0x55e4bdba6680 2024-09-12T03:16:24.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.695+0000 7f2853ef4740 10 client.0 _ll_register_callbacks cb 0x7f283c00b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:24.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.695+0000 7f284affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4bdb9b4a0 0x55e4bdbabb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:24.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.695+0000 7f284affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4bdb9b4a0 0x55e4bdbabb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39812/0 (socket says 172.21.15.116:39812) 2024-09-12T03:16:24.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.695+0000 7f284affd640 1 -- 172.21.15.116:0/3737328790 learned_addr learned my addr 172.21.15.116:0/3737328790 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:24.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.695+0000 7f284affd640 1 -- 172.21.15.116:0/3737328790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4bdba6680 msgr2=0x55e4bdc1ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:24.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.695+0000 7f284b7fe640 1 --2- 172.21.15.116:0/3737328790 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4bdb9bdc0 0x55e4bdbac0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:24.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.695+0000 7f284affd640 1 --2- 172.21.15.116:0/3737328790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4bdba6680 0x55e4bdc1ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.695+0000 7f284affd640 1 -- 172.21.15.116:0/3737328790 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4bdb9bdc0 msgr2=0x55e4bdbac0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:24.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.695+0000 7f284affd640 1 --2- 172.21.15.116:0/3737328790 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4bdb9bdc0 0x55e4bdbac0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.695+0000 7f284affd640 1 -- 172.21.15.116:0/3737328790 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4bdc1f3d0 con 0x55e4bdb9b4a0 2024-09-12T03:16:24.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.695+0000 7f284affd640 1 --2- 172.21.15.116:0/3737328790 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4bdb9b4a0 0x55e4bdbabb70 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f2840031870 tx=0x7f2840031a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204d5817ea7021b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:24.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.695+0000 7f282affd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:16:24.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.695+0000 7f282affd640 1 -- 172.21.15.116:0/3737328790 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2840057070 con 0x55e4bdb9b4a0 2024-09-12T03:16:24.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.695+0000 7f282affd640 1 -- 172.21.15.116:0/3737328790 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f28400035f0 con 0x55e4bdb9b4a0 2024-09-12T03:16:24.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.695+0000 7f2853ef4740 1 -- 172.21.15.116:0/3737328790 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2840045040 con 0x55e4bdb9b4a0 2024-09-12T03:16:24.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.695+0000 7f282affd640 1 -- 172.21.15.116:0/3737328790 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2840044070 con 0x55e4bdb9b4a0 2024-09-12T03:16:24.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.695+0000 7f2853ef4740 10 client.18924 fetch_fsmap learned FSMap version 25 2024-09-12T03:16:24.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.695+0000 7f2853ef4740 1 -- 172.21.15.116:0/3737328790 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e4bdc206c0 con 0x55e4bdb9b4a0 2024-09-12T03:16:24.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.696+0000 7f282affd640 1 -- 172.21.15.116:0/3737328790 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2840045040 con 0x55e4bdb9b4a0 2024-09-12T03:16:24.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.696+0000 7f282affd640 1 -- 172.21.15.116:0/3737328790 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2840053030 con 0x55e4bdb9b4a0 2024-09-12T03:16:24.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.696+0000 7f2853ef4740 10 client.18924 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:16:24.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.696+0000 7f2853ef4740 10 client.18924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:16:24.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.696+0000 7f2853ef4740 20 client.18924 populate_metadata read hostname 'smithi116' 2024-09-12T03:16:24.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.696+0000 7f2853ef4740 1 --2- 172.21.15.116:0/3737328790 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e4bdc208f0 0x55e4bdc22ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:24.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.696+0000 7f2853ef4740 4 client.18924 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:16:24.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.696+0000 7f2853ef4740 1 -- 172.21.15.116:0/3737328790 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e4bdb08d60 con 0x55e4bdc208f0 2024-09-12T03:16:24.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.696+0000 7f284b7fe640 1 --2- 172.21.15.116:0/3737328790 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e4bdc208f0 0x55e4bdc22ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:24.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.696+0000 7f284b7fe640 1 --2- 172.21.15.116:0/3737328790 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e4bdc208f0 0x55e4bdc22ce0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa091116ea618842 server_cookie=bbada9a99e07d6d8 in_seq=0 out_seq=0 2024-09-12T03:16:24.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.696+0000 7f282affd640 10 client.18924 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:24.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.697+0000 7f282affd640 1 -- 172.21.15.116:0/3737328790 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e4bdb08d60 con 0x55e4bdc208f0 2024-09-12T03:16:24.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.697+0000 7f282affd640 10 client.18924 handle_command_reply: tid=0 2024-09-12T03:16:24.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.697+0000 7f2853ef4740 1 client.18924 shutdown 2024-09-12T03:16:24.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.697+0000 7f2853ef4740 1 -- 172.21.15.116:0/3737328790 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4bdb9b4a0 msgr2=0x55e4bdbabb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:24.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.697+0000 7f2853ef4740 1 --2- 172.21.15.116:0/3737328790 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4bdb9b4a0 0x55e4bdbabb70 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f2840031870 tx=0x7f2840031a10 comp rx=0 tx=0).stop 2024-09-12T03:16:24.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.697+0000 7f2853ef4740 1 -- 172.21.15.116:0/3737328790 shutdown_connections 2024-09-12T03:16:24.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.697+0000 7f2853ef4740 1 --2- 172.21.15.116:0/3737328790 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e4bdc208f0 0x55e4bdc22ce0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.697+0000 7f2853ef4740 1 --2- 172.21.15.116:0/3737328790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4bdba6680 0x55e4bdc1ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.697+0000 7f2853ef4740 1 --2- 172.21.15.116:0/3737328790 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4bdb9bdc0 0x55e4bdbac0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.697+0000 7f282affd640 0 client.18924 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:24.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.697+0000 7f2853ef4740 1 --2- 172.21.15.116:0/3737328790 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4bdb9b4a0 0x55e4bdbabb70 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.697+0000 7f2853ef4740 1 -- 172.21.15.116:0/3737328790 >> 172.21.15.116:0/3737328790 conn(0x55e4bdb97a30 msgr2=0x55e4bdba4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:24.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.697+0000 7f2853ef4740 1 -- 172.21.15.116:0/3737328790 shutdown_connections 2024-09-12T03:16:24.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.697+0000 7f2853ef4740 1 -- 172.21.15.116:0/3737328790 wait complete. 2024-09-12T03:16:24.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.697+0000 7f2853ef4740 20 client.18924 trim_cache size 0 max 16384 2024-09-12T03:16:24.715 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:16:24.715 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:16:24.715 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:16:24.715 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:16:24.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.698+0000 7f2851b1b640 1 -- 172.21.15.116:0/632974635 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f28340a2b80 msgr2=0x7f28340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:24.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.698+0000 7f2851b1b640 1 --2- 172.21.15.116:0/632974635 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f28340a2b80 0x7f28340a5040 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f284400ccb0 tx=0x7f284400af30 comp rx=0 tx=0).stop 2024-09-12T03:16:24.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.698+0000 7f2851b1b640 1 -- 172.21.15.116:0/632974635 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f284c106f10 msgr2=0x7f284c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:24.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.698+0000 7f2851b1b640 1 --2- 172.21.15.116:0/632974635 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f284c106f10 0x7f284c074f70 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f284400cce0 tx=0x7f284400a620 comp rx=0 tx=0).stop 2024-09-12T03:16:24.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.698+0000 7f2851b1b640 1 -- 172.21.15.116:0/632974635 shutdown_connections 2024-09-12T03:16:24.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.698+0000 7f2851b1b640 1 --2- 172.21.15.116:0/632974635 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f28340a2b80 0x7f28340a5040 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.698+0000 7f2851b1b640 1 --2- 172.21.15.116:0/632974635 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f284c106f10 0x7f284c074f70 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.698+0000 7f2851b1b640 1 --2- 172.21.15.116:0/632974635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f284c102790 0x7f284c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.698+0000 7f2851b1b640 1 --2- 172.21.15.116:0/632974635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f284c101de0 0x7f284c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.698+0000 7f2851b1b640 1 -- 172.21.15.116:0/632974635 >> 172.21.15.116:0/632974635 conn(0x7f284c0fd8c0 msgr2=0x7f284c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:24.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.698+0000 7f2851b1b640 1 -- 172.21.15.116:0/632974635 shutdown_connections 2024-09-12T03:16:24.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:24.698+0000 7f2851b1b640 1 -- 172.21.15.116:0/632974635 wait complete. 2024-09-12T03:16:25.510 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:16:25.510 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:16:25.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:25 smithi183 ceph-mon[35535]: pgmap v741: 97 pgs: 97 active+clean; 6.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:16:25.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:25 smithi177 ceph-mon[35305]: pgmap v741: 97 pgs: 97 active+clean; 6.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:16:25.756 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:25.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:25 smithi116 ceph-mon[31410]: pgmap v741: 97 pgs: 97 active+clean; 6.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:16:26.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.280+0000 7feab5ec9640 1 -- 172.21.15.116:0/3607138589 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feab010c170 msgr2=0x7feab010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:26.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.280+0000 7feab5ec9640 1 --2- 172.21.15.116:0/3607138589 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feab010c170 0x7feab010e610 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7feaa8009ca0 tx=0x7feaa802a950 comp rx=0 tx=0).stop 2024-09-12T03:16:26.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.280+0000 7feab5ec9640 1 -- 172.21.15.116:0/3607138589 shutdown_connections 2024-09-12T03:16:26.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.280+0000 7feab5ec9640 1 --2- 172.21.15.116:0/3607138589 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feab010c170 0x7feab010e610 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:26.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.280+0000 7feab5ec9640 1 --2- 172.21.15.116:0/3607138589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feab0103660 0x7feab010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:26.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.280+0000 7feab5ec9640 1 --2- 172.21.15.116:0/3607138589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feab0102d40 0x7feab0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:26.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.280+0000 7feab5ec9640 1 -- 172.21.15.116:0/3607138589 >> 172.21.15.116:0/3607138589 conn(0x7feab00fc7a0 msgr2=0x7feab00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:26.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.280+0000 7feab5ec9640 1 -- 172.21.15.116:0/3607138589 shutdown_connections 2024-09-12T03:16:26.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.280+0000 7feab5ec9640 1 -- 172.21.15.116:0/3607138589 wait complete. 2024-09-12T03:16:26.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.281+0000 7feab5ec9640 1 Processor -- start 2024-09-12T03:16:26.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.281+0000 7feab5ec9640 1 -- start start 2024-09-12T03:16:26.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.281+0000 7feab5ec9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feab0102d40 0x7feab0198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:26.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.281+0000 7feab5ec9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feab0103660 0x7feab0198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:26.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.281+0000 7feab5ec9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feab010c170 0x7feab019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:26.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.281+0000 7feab5ec9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7feab0111fb0 con 0x7feab0103660 2024-09-12T03:16:26.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.281+0000 7feab5ec9640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7feab0111e30 con 0x7feab010c170 2024-09-12T03:16:26.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.281+0000 7feab5ec9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feab0112130 con 0x7feab0102d40 2024-09-12T03:16:26.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.281+0000 7feaa7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feab0103660 0x7feab0198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:26.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.281+0000 7feaa7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feab0103660 0x7feab0198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39828/0 (socket says 172.21.15.116:39828) 2024-09-12T03:16:26.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.281+0000 7feaa7fff640 1 -- 172.21.15.116:0/975484318 learned_addr learned my addr 172.21.15.116:0/975484318 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:26.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.281+0000 7feaaf7fe640 1 --2- 172.21.15.116:0/975484318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feab0102d40 0x7feab0198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:26.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.281+0000 7feaaffff640 1 --2- 172.21.15.116:0/975484318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feab010c170 0x7feab019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:26.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.281+0000 7feaa7fff640 1 -- 172.21.15.116:0/975484318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feab0102d40 msgr2=0x7feab0198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:26.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.281+0000 7feaa7fff640 1 --2- 172.21.15.116:0/975484318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feab0102d40 0x7feab0198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:26.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.281+0000 7feaa7fff640 1 -- 172.21.15.116:0/975484318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feab010c170 msgr2=0x7feab019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:26.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.281+0000 7feaa7fff640 1 --2- 172.21.15.116:0/975484318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feab010c170 0x7feab019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:26.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.281+0000 7feaa7fff640 1 -- 172.21.15.116:0/975484318 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab019d3e0 con 0x7feab0103660 2024-09-12T03:16:26.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.281+0000 7feaaffff640 1 --2- 172.21.15.116:0/975484318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feab010c170 0x7feab019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:26.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.282+0000 7feaa7fff640 1 --2- 172.21.15.116:0/975484318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feab0103660 0x7feab0198970 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fea9800d740 tx=0x7fea9800db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:26.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.282+0000 7feaad7fa640 1 -- 172.21.15.116:0/975484318 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea98017070 con 0x7feab0103660 2024-09-12T03:16:26.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.282+0000 7feaad7fa640 1 -- 172.21.15.116:0/975484318 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea98010460 con 0x7feab0103660 2024-09-12T03:16:26.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.282+0000 7feab5ec9640 1 -- 172.21.15.116:0/975484318 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feab019d6d0 con 0x7feab0103660 2024-09-12T03:16:26.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.282+0000 7feaad7fa640 1 -- 172.21.15.116:0/975484318 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9800f620 con 0x7feab0103660 2024-09-12T03:16:26.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.282+0000 7feab5ec9640 1 -- 172.21.15.116:0/975484318 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feab01a4fb0 con 0x7feab0103660 2024-09-12T03:16:26.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.283+0000 7feab5ec9640 1 -- 172.21.15.116:0/975484318 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feab0103120 con 0x7feab0103660 2024-09-12T03:16:26.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.284+0000 7feaad7fa640 1 -- 172.21.15.116:0/975484318 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fea98010610 con 0x7feab0103660 2024-09-12T03:16:26.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.285+0000 7feaad7fa640 1 --2- 172.21.15.116:0/975484318 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fea9c0a2b80 0x7fea9c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:26.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.285+0000 7feaaf7fe640 1 --2- 172.21.15.116:0/975484318 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fea9c0a2b80 0x7fea9c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:26.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.285+0000 7feaad7fa640 1 -- 172.21.15.116:0/975484318 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fea980ceb50 con 0x7feab0103660 2024-09-12T03:16:26.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.285+0000 7feaaf7fe640 1 --2- 172.21.15.116:0/975484318 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fea9c0a2b80 0x7fea9c0a5040 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7feaa0009990 tx=0x7feaa0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:26.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.288+0000 7feaad7fa640 1 -- 172.21.15.116:0/975484318 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fea9809bae0 con 0x7feab0103660 2024-09-12T03:16:26.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.492+0000 7feab5ec9640 1 -- 172.21.15.116:0/975484318 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feab0107270 con 0x7feab0103660 2024-09-12T03:16:26.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.493+0000 7feaad7fa640 1 -- 172.21.15.116:0/975484318 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7fea9809b3e0 con 0x7feab0103660 2024-09-12T03:16:26.493 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:16:26.493 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:16:26.494 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:16:26.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.495+0000 7feab5ec9640 1 -- 172.21.15.116:0/975484318 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fea9c0a2b80 msgr2=0x7fea9c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:26.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.495+0000 7feab5ec9640 1 --2- 172.21.15.116:0/975484318 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fea9c0a2b80 0x7fea9c0a5040 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7feaa0009990 tx=0x7feaa0009110 comp rx=0 tx=0).stop 2024-09-12T03:16:26.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.495+0000 7feab5ec9640 1 -- 172.21.15.116:0/975484318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feab0103660 msgr2=0x7feab0198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:26.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.495+0000 7feab5ec9640 1 --2- 172.21.15.116:0/975484318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feab0103660 0x7feab0198970 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fea9800d740 tx=0x7fea9800db40 comp rx=0 tx=0).stop 2024-09-12T03:16:26.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.495+0000 7feab5ec9640 1 -- 172.21.15.116:0/975484318 shutdown_connections 2024-09-12T03:16:26.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.495+0000 7feab5ec9640 1 --2- 172.21.15.116:0/975484318 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fea9c0a2b80 0x7fea9c0a5040 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:26.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.495+0000 7feab5ec9640 1 --2- 172.21.15.116:0/975484318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feab010c170 0x7feab019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:26.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.495+0000 7feab5ec9640 1 --2- 172.21.15.116:0/975484318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feab0103660 0x7feab0198970 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:26.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.495+0000 7feab5ec9640 1 --2- 172.21.15.116:0/975484318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feab0102d40 0x7feab0198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:26.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.495+0000 7feab5ec9640 1 -- 172.21.15.116:0/975484318 >> 172.21.15.116:0/975484318 conn(0x7feab00fc7a0 msgr2=0x7feab00fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:26.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.495+0000 7feab5ec9640 1 -- 172.21.15.116:0/975484318 shutdown_connections 2024-09-12T03:16:26.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:26.495+0000 7feab5ec9640 1 -- 172.21.15.116:0/975484318 wait complete. 2024-09-12T03:16:26.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:26 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/975484318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:26.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:26 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/975484318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:26.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:26 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/975484318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:27.151 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:16:27.394 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:27.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:27 smithi116 ceph-mon[31410]: pgmap v742: 97 pgs: 97 active+clean; 6.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:16:27.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.909+0000 7f8cbff5f640 1 -- 172.21.15.116:0/91181385 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cb8101450 msgr2=0x7f8cb8105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:27.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.909+0000 7f8cbff5f640 1 --2- 172.21.15.116:0/91181385 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cb8101450 0x7f8cb8105920 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8009900 tx=0x7f8ca802a470 comp rx=0 tx=0).stop 2024-09-12T03:16:27.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.909+0000 7f8cbff5f640 1 -- 172.21.15.116:0/91181385 shutdown_connections 2024-09-12T03:16:27.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.909+0000 7f8cbff5f640 1 --2- 172.21.15.116:0/91181385 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cb8105f40 0x7f8cb81063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.909+0000 7f8cbff5f640 1 --2- 172.21.15.116:0/91181385 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cb8101450 0x7f8cb8105920 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.909+0000 7f8cbff5f640 1 --2- 172.21.15.116:0/91181385 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cb8100b30 0x7f8cb8100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.909+0000 7f8cbff5f640 1 -- 172.21.15.116:0/91181385 >> 172.21.15.116:0/91181385 conn(0x7f8cb80fc7a0 msgr2=0x7f8cb80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:27.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.910+0000 7f8cbff5f640 1 -- 172.21.15.116:0/91181385 shutdown_connections 2024-09-12T03:16:27.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.910+0000 7f8cbff5f640 1 -- 172.21.15.116:0/91181385 wait complete. 2024-09-12T03:16:27.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.910+0000 7f8cbff5f640 1 Processor -- start 2024-09-12T03:16:27.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.910+0000 7f8cbff5f640 1 -- start start 2024-09-12T03:16:27.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.910+0000 7f8cbff5f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cb8100b30 0x7f8cb81a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:27.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.910+0000 7f8cbff5f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cb8101450 0x7f8cb81a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:27.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.910+0000 7f8cbff5f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cb8105f40 0x7f8cb819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:27.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.910+0000 7f8cbff5f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb8118540 con 0x7f8cb8100b30 2024-09-12T03:16:27.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.910+0000 7f8cbff5f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb81183c0 con 0x7f8cb8105f40 2024-09-12T03:16:27.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.910+0000 7f8cbff5f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb81186c0 con 0x7f8cb8101450 2024-09-12T03:16:27.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.910+0000 7f8cbdcfb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cb8100b30 0x7f8cb81a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:27.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.911+0000 7f8cbdcfb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cb8100b30 0x7f8cb81a4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39846/0 (socket says 172.21.15.116:39846) 2024-09-12T03:16:27.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.911+0000 7f8cbdcfb640 1 -- 172.21.15.116:0/3854647839 learned_addr learned my addr 172.21.15.116:0/3854647839 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:27.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.911+0000 7f8cbe4fc640 1 --2- 172.21.15.116:0/3854647839 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cb8105f40 0x7f8cb819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:27.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.911+0000 7f8cbdcfb640 1 -- 172.21.15.116:0/3854647839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cb8101450 msgr2=0x7f8cb81a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:27.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.911+0000 7f8cbdcfb640 1 --2- 172.21.15.116:0/3854647839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cb8101450 0x7f8cb81a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.911+0000 7f8cbdcfb640 1 -- 172.21.15.116:0/3854647839 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cb8105f40 msgr2=0x7f8cb819e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:27.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.911+0000 7f8cbdcfb640 1 --2- 172.21.15.116:0/3854647839 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cb8105f40 0x7f8cb819e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.911+0000 7f8cbdcfb640 1 -- 172.21.15.116:0/3854647839 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb819efb0 con 0x7f8cb8100b30 2024-09-12T03:16:27.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.911+0000 7f8cbdcfb640 1 --2- 172.21.15.116:0/3854647839 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cb8100b30 0x7f8cb81a4690 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f8cb400c920 tx=0x7f8cb400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:27.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.911+0000 7f8caeffd640 1 -- 172.21.15.116:0/3854647839 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4007c70 con 0x7f8cb8100b30 2024-09-12T03:16:27.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.911+0000 7f8caeffd640 1 -- 172.21.15.116:0/3854647839 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cb4007e10 con 0x7f8cb8100b30 2024-09-12T03:16:27.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.911+0000 7f8cbff5f640 1 -- 172.21.15.116:0/3854647839 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cb819f2a0 con 0x7f8cb8100b30 2024-09-12T03:16:27.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.911+0000 7f8caeffd640 1 -- 172.21.15.116:0/3854647839 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4018890 con 0x7f8cb8100b30 2024-09-12T03:16:27.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.911+0000 7f8cbff5f640 1 -- 172.21.15.116:0/3854647839 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cb81ab4f0 con 0x7f8cb8100b30 2024-09-12T03:16:27.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.913+0000 7f8caeffd640 1 -- 172.21.15.116:0/3854647839 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8cb4018a70 con 0x7f8cb8100b30 2024-09-12T03:16:27.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.914+0000 7f8cc2338740 1 Processor -- start 2024-09-12T03:16:27.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.914+0000 7f8cc2338740 1 -- start start 2024-09-12T03:16:27.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.914+0000 7f8cc2338740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c4eaaeb40 0x559c4eaad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:27.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.914+0000 7f8cc2338740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559c4eaad5d0 0x559c4eaada10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:27.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.914+0000 7f8caeffd640 1 --2- 172.21.15.116:0/3854647839 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8ca40a2b80 0x7f8ca40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:27.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.914+0000 7f8cc2338740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4eab8370 0x559c4eaba760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:27.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.914+0000 7f8cc2338740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559c4ea132f0 con 0x559c4eaad5d0 2024-09-12T03:16:27.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.914+0000 7f8cc2338740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559c4e9bacf0 con 0x559c4eaaeb40 2024-09-12T03:16:27.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.914+0000 7f8cc2338740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559c4ea0f2b0 con 0x559c4eab8370 2024-09-12T03:16:27.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.914+0000 7f8cbdcfb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559c4eaad5d0 0x559c4eaada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:27.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.914+0000 7f8cbd4fa640 1 --2- 172.21.15.116:0/3854647839 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8ca40a2b80 0x7f8ca40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:27.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.914+0000 7f8caeffd640 1 -- 172.21.15.116:0/3854647839 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f8cb40cf270 con 0x7f8cb8100b30 2024-09-12T03:16:27.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.914+0000 7f8cbdcfb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559c4eaad5d0 0x559c4eaada10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39862/0 (socket says 172.21.15.116:39862) 2024-09-12T03:16:27.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.914+0000 7f8cbe4fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4eab8370 0x559c4eaba760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:27.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.914+0000 7f8cbdcfb640 1 -- 172.21.15.116:0/690297650 learned_addr learned my addr 172.21.15.116:0/690297650 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:27.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.915+0000 7f8cbe4fc640 1 --2- 172.21.15.116:0/690297650 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c4eaaeb40 0x559c4eaad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:27.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.915+0000 7f8cbdcfb640 1 -- 172.21.15.116:0/690297650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4eab8370 msgr2=0x559c4eaba760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:27.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.915+0000 7f8cbdcfb640 1 --2- 172.21.15.116:0/690297650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4eab8370 0x559c4eaba760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.915+0000 7f8cbdcfb640 1 -- 172.21.15.116:0/690297650 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c4eaaeb40 msgr2=0x559c4eaad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:27.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.915+0000 7f8cbdcfb640 1 --2- 172.21.15.116:0/690297650 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c4eaaeb40 0x559c4eaad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.915+0000 7f8cbdcfb640 1 -- 172.21.15.116:0/690297650 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c4e99f3a0 con 0x559c4eaad5d0 2024-09-12T03:16:27.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.915+0000 7f8cbe4fc640 1 --2- 172.21.15.116:0/690297650 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c4eaaeb40 0x559c4eaad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:27.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.915+0000 7f8cbdcfb640 1 --2- 172.21.15.116:0/690297650 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559c4eaad5d0 0x559c4eaada10 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f8cb40ccd40 tx=0x7f8cb4090f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5401892511d2b05 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:27.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.915+0000 7f8cbd4fa640 1 --2- 172.21.15.116:0/3854647839 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8ca40a2b80 0x7f8ca40a5040 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8ca802a990 tx=0x7f8ca8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:27.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.915+0000 7f8ca3fff640 1 -- 172.21.15.116:0/690297650 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb409c1a0 con 0x559c4eaad5d0 2024-09-12T03:16:27.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.915+0000 7f8ca3fff640 1 -- 172.21.15.116:0/690297650 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cb400de20 con 0x559c4eaad5d0 2024-09-12T03:16:27.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.915+0000 7f8ca3fff640 1 -- 172.21.15.116:0/690297650 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4014210 con 0x559c4eaad5d0 2024-09-12T03:16:27.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.916+0000 7f8cc2338740 1 -- 172.21.15.116:0/690297650 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559c4eaad5d0 msgr2=0x559c4eaada10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:27.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.916+0000 7f8cc2338740 1 --2- 172.21.15.116:0/690297650 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559c4eaad5d0 0x559c4eaada10 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f8cb40ccd40 tx=0x7f8cb4090f80 comp rx=0 tx=0).stop 2024-09-12T03:16:27.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.916+0000 7f8cc2338740 1 -- 172.21.15.116:0/690297650 shutdown_connections 2024-09-12T03:16:27.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.916+0000 7f8cc2338740 1 --2- 172.21.15.116:0/690297650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4eab8370 0x559c4eaba760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.916+0000 7f8cc2338740 1 --2- 172.21.15.116:0/690297650 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559c4eaad5d0 0x559c4eaada10 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.916+0000 7f8cc2338740 1 --2- 172.21.15.116:0/690297650 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c4eaaeb40 0x559c4eaad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.916+0000 7f8cc2338740 1 -- 172.21.15.116:0/690297650 >> 172.21.15.116:0/690297650 conn(0x559c4ea76310 msgr2=0x559c4eaa9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:27.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.916+0000 7f8cc2338740 1 -- 172.21.15.116:0/690297650 shutdown_connections 2024-09-12T03:16:27.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.916+0000 7f8cc2338740 1 -- 172.21.15.116:0/690297650 wait complete. 2024-09-12T03:16:27.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.916+0000 7f8cc2338740 1 -- start start 2024-09-12T03:16:27.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.916+0000 7f8cc2338740 1 Processor -- start 2024-09-12T03:16:27.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.917+0000 7f8cc2338740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559c4eaad5d0 0x559c4eab7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:27.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.917+0000 7f8cc2338740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c4eaaeb40 0x559c4eab4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:27.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.917+0000 7f8cc2338740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4eab8370 0x559c4eab54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:27.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.917+0000 7f8cc2338740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559c4eabb070 con 0x559c4eaad5d0 2024-09-12T03:16:27.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.917+0000 7f8cc2338740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559c4e9ce060 con 0x559c4eaaeb40 2024-09-12T03:16:27.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.917+0000 7f8cc2338740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559c4eabb1f0 con 0x559c4eab8370 2024-09-12T03:16:27.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.917+0000 7f8cc2338740 10 client.0 _ll_register_callbacks cb 0x559c4e7218c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:27.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.917+0000 7f8cbdcfb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559c4eaad5d0 0x559c4eab7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:27.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.917+0000 7f8cbdcfb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559c4eaad5d0 0x559c4eab7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39868/0 (socket says 172.21.15.116:39868) 2024-09-12T03:16:27.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.917+0000 7f8cbdcfb640 1 -- 172.21.15.116:0/1466024931 learned_addr learned my addr 172.21.15.116:0/1466024931 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:27.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.917+0000 7f8cbe4fc640 1 --2- 172.21.15.116:0/1466024931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4eab8370 0x559c4eab54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:27.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.917+0000 7f8cbd4fa640 1 --2- 172.21.15.116:0/1466024931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c4eaaeb40 0x559c4eab4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:27.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.917+0000 7f8cbdcfb640 1 -- 172.21.15.116:0/1466024931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4eab8370 msgr2=0x559c4eab54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:27.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.917+0000 7f8cbdcfb640 1 --2- 172.21.15.116:0/1466024931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4eab8370 0x559c4eab54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.917+0000 7f8cbdcfb640 1 -- 172.21.15.116:0/1466024931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c4eaaeb40 msgr2=0x559c4eab4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:27.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.917+0000 7f8cbdcfb640 1 --2- 172.21.15.116:0/1466024931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c4eaaeb40 0x559c4eab4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.917+0000 7f8cbdcfb640 1 -- 172.21.15.116:0/1466024931 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c4e7843c0 con 0x559c4eaad5d0 2024-09-12T03:16:27.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.917+0000 7f8cbd4fa640 1 --2- 172.21.15.116:0/1466024931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c4eaaeb40 0x559c4eab4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:27.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.917+0000 7f8cbe4fc640 1 --2- 172.21.15.116:0/1466024931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4eab8370 0x559c4eab54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:27.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.917+0000 7f8cbdcfb640 1 --2- 172.21.15.116:0/1466024931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559c4eaad5d0 0x559c4eab7cd0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4098470 tx=0x7f8cb409c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3570b96921770d77 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:27.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.917+0000 7f8ca17fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:16:27.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.917+0000 7f8ca17fa640 1 -- 172.21.15.116:0/1466024931 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb409baa0 con 0x559c4eaad5d0 2024-09-12T03:16:27.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.917+0000 7f8ca17fa640 1 -- 172.21.15.116:0/1466024931 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f8cb400fb30 con 0x559c4eaad5d0 2024-09-12T03:16:27.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.917+0000 7f8cc2338740 1 -- 172.21.15.116:0/1466024931 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cb400de20 con 0x559c4eaad5d0 2024-09-12T03:16:27.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.918+0000 7f8ca17fa640 1 -- 172.21.15.116:0/1466024931 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8cb409fe50 con 0x559c4eaad5d0 2024-09-12T03:16:27.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.918+0000 7f8cc2338740 10 client.18942 fetch_fsmap learned FSMap version 25 2024-09-12T03:16:27.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.918+0000 7f8cc2338740 1 -- 172.21.15.116:0/1466024931 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559c4eac6ab0 con 0x559c4eaad5d0 2024-09-12T03:16:27.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.918+0000 7f8ca17fa640 1 -- 172.21.15.116:0/1466024931 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cb400de20 con 0x559c4eaad5d0 2024-09-12T03:16:27.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.918+0000 7f8ca17fa640 1 -- 172.21.15.116:0/1466024931 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8cb409af80 con 0x559c4eaad5d0 2024-09-12T03:16:27.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.918+0000 7f8cc2338740 10 client.18942 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:16:27.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.918+0000 7f8cc2338740 10 client.18942 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:16:27.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.918+0000 7f8cc2338740 20 client.18942 populate_metadata read hostname 'smithi116' 2024-09-12T03:16:27.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.918+0000 7f8cc2338740 1 --2- 172.21.15.116:0/1466024931 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x559c4eac7e00 0x559c4eaca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:27.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.918+0000 7f8cc2338740 4 client.18942 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:16:27.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.918+0000 7f8cc2338740 1 -- 172.21.15.116:0/1466024931 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c4ea20860 con 0x559c4eac7e00 2024-09-12T03:16:27.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.919+0000 7f8cbd4fa640 1 --2- 172.21.15.116:0/1466024931 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x559c4eac7e00 0x559c4eaca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:27.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.919+0000 7f8cbd4fa640 1 --2- 172.21.15.116:0/1466024931 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x559c4eac7e00 0x559c4eaca1f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1fc490c744e85268 server_cookie=75c16f9c860919c5 in_seq=0 out_seq=0 2024-09-12T03:16:27.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.919+0000 7f8ca17fa640 10 client.18942 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:16:27.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.920+0000 7f8ca17fa640 1 -- 172.21.15.116:0/1466024931 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559c4ea20860 con 0x559c4eac7e00 2024-09-12T03:16:27.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.920+0000 7f8ca17fa640 10 client.18942 handle_command_reply: tid=0 2024-09-12T03:16:27.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.920+0000 7f8cc2338740 1 client.18942 shutdown 2024-09-12T03:16:27.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.920+0000 7f8cc2338740 1 -- 172.21.15.116:0/1466024931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559c4eaad5d0 msgr2=0x559c4eab7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:27.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.920+0000 7f8cc2338740 1 --2- 172.21.15.116:0/1466024931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559c4eaad5d0 0x559c4eab7cd0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4098470 tx=0x7f8cb409c210 comp rx=0 tx=0).stop 2024-09-12T03:16:27.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.920+0000 7f8cc2338740 1 -- 172.21.15.116:0/1466024931 shutdown_connections 2024-09-12T03:16:27.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.920+0000 7f8cc2338740 1 --2- 172.21.15.116:0/1466024931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4eab8370 0x559c4eab54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.920+0000 7f8cc2338740 1 --2- 172.21.15.116:0/1466024931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c4eaaeb40 0x559c4eab4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.920+0000 7f8cc2338740 1 --2- 172.21.15.116:0/1466024931 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x559c4eac7e00 0x559c4eaca1f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.920+0000 7f8cc2338740 1 --2- 172.21.15.116:0/1466024931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559c4eaad5d0 0x559c4eab7cd0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.920+0000 7f8ca17fa640 0 client.18942 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:16:27.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.920+0000 7f8cc2338740 1 -- 172.21.15.116:0/1466024931 >> 172.21.15.116:0/1466024931 conn(0x559c4ea76310 msgr2=0x559c4eaaf990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:27.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.920+0000 7f8cc2338740 1 -- 172.21.15.116:0/1466024931 shutdown_connections 2024-09-12T03:16:27.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.920+0000 7f8cc2338740 1 -- 172.21.15.116:0/1466024931 wait complete. 2024-09-12T03:16:27.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.920+0000 7f8cc2338740 20 client.18942 trim_cache size 0 max 16384 2024-09-12T03:16:27.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.933+0000 7f8cc2338740 1 Processor -- start 2024-09-12T03:16:27.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.933+0000 7f8cc2338740 1 -- start start 2024-09-12T03:16:27.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.933+0000 7f8cc2338740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4eaad5d0 0x559c4eab6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:27.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.933+0000 7f8cc2338740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c4eaaeb40 0x559c4eab7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:27.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.933+0000 7f8cc2338740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559c4eab8370 0x559c4eab0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:27.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.933+0000 7f8cc2338740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb81b6440 con 0x559c4eab8370 2024-09-12T03:16:27.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.934+0000 7f8cc2338740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb81b3d10 con 0x559c4eaaeb40 2024-09-12T03:16:27.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.934+0000 7f8cc2338740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb81b1b60 con 0x559c4eaad5d0 2024-09-12T03:16:27.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.934+0000 7f8cbe4fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559c4eab8370 0x559c4eab0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:27.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.934+0000 7f8cbe4fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559c4eab8370 0x559c4eab0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39882/0 (socket says 172.21.15.116:39882) 2024-09-12T03:16:27.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.934+0000 7f8cbe4fc640 1 -- 172.21.15.116:0/3390019911 learned_addr learned my addr 172.21.15.116:0/3390019911 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:27.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.934+0000 7f8cbdcfb640 1 --2- 172.21.15.116:0/3390019911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4eaad5d0 0x559c4eab6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:27.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.934+0000 7f8cbe4fc640 1 -- 172.21.15.116:0/3390019911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4eaad5d0 msgr2=0x559c4eab6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:27.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.934+0000 7f8cbe4fc640 1 --2- 172.21.15.116:0/3390019911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4eaad5d0 0x559c4eab6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.934+0000 7f8cbd4fa640 1 --2- 172.21.15.116:0/3390019911 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c4eaaeb40 0x559c4eab7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:27.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.934+0000 7f8cbe4fc640 1 -- 172.21.15.116:0/3390019911 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c4eaaeb40 msgr2=0x559c4eab7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:27.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.934+0000 7f8cbe4fc640 1 --2- 172.21.15.116:0/3390019911 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c4eaaeb40 0x559c4eab7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.934+0000 7f8cbe4fc640 1 -- 172.21.15.116:0/3390019911 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c4eab5850 con 0x559c4eab8370 2024-09-12T03:16:27.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.934+0000 7f8cbe4fc640 1 --2- 172.21.15.116:0/3390019911 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559c4eab8370 0x559c4eab0120 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000a910 tx=0x7f8cb0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fedfbe8b2b324e66 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:27.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.934+0000 7f8ca37fe640 1 -- 172.21.15.116:0/3390019911 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0004d20 con 0x559c4eab8370 2024-09-12T03:16:27.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.934+0000 7f8ca37fe640 1 -- 172.21.15.116:0/3390019911 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cb0018070 con 0x559c4eab8370 2024-09-12T03:16:27.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.934+0000 7f8ca37fe640 1 -- 172.21.15.116:0/3390019911 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0005610 con 0x559c4eab8370 2024-09-12T03:16:27.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.935+0000 7f8cc2338740 1 -- 172.21.15.116:0/3390019911 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559c4eab8370 msgr2=0x559c4eab0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:27.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.935+0000 7f8cc2338740 1 --2- 172.21.15.116:0/3390019911 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559c4eab8370 0x559c4eab0120 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000a910 tx=0x7f8cb0004ab0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.935+0000 7f8cc2338740 1 -- 172.21.15.116:0/3390019911 shutdown_connections 2024-09-12T03:16:27.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.935+0000 7f8cc2338740 1 --2- 172.21.15.116:0/3390019911 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559c4eab8370 0x559c4eab0120 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.935+0000 7f8cc2338740 1 --2- 172.21.15.116:0/3390019911 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c4eaaeb40 0x559c4eab7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.935+0000 7f8cc2338740 1 --2- 172.21.15.116:0/3390019911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4eaad5d0 0x559c4eab6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.935+0000 7f8cc2338740 1 -- 172.21.15.116:0/3390019911 >> 172.21.15.116:0/3390019911 conn(0x559c4ea76310 msgr2=0x559c4eab67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:27.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.935+0000 7f8cc2338740 1 -- 172.21.15.116:0/3390019911 shutdown_connections 2024-09-12T03:16:27.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.935+0000 7f8cc2338740 1 -- 172.21.15.116:0/3390019911 wait complete. 2024-09-12T03:16:27.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.935+0000 7f8cc2338740 1 -- start start 2024-09-12T03:16:27.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.935+0000 7f8cc2338740 1 Processor -- start 2024-09-12T03:16:27.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.936+0000 7f8cc2338740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c4eaad5d0 0x559c4eb29ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:27.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.936+0000 7f8cc2338740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4eaaeb40 0x559c4eb29ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:27.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.936+0000 7f8cc2338740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559c4eab8370 0x559c4eaca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:27.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.936+0000 7f8cc2338740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb81b21d0 con 0x559c4eab8370 2024-09-12T03:16:27.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.936+0000 7f8cc2338740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb81b3640 con 0x559c4eaad5d0 2024-09-12T03:16:27.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.936+0000 7f8cc2338740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb81b28a0 con 0x559c4eaaeb40 2024-09-12T03:16:27.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.936+0000 7f8cc2338740 10 client.0 _ll_register_callbacks cb 0x7f8cb40a47e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:27.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.936+0000 7f8cbe4fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559c4eab8370 0x559c4eaca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:27.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.936+0000 7f8cbe4fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559c4eab8370 0x559c4eaca830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39884/0 (socket says 172.21.15.116:39884) 2024-09-12T03:16:27.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.936+0000 7f8cbe4fc640 1 -- 172.21.15.116:0/1738378591 learned_addr learned my addr 172.21.15.116:0/1738378591 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:27.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.936+0000 7f8cbd4fa640 1 --2- 172.21.15.116:0/1738378591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4eaaeb40 0x559c4eb29ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:27.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.936+0000 7f8cbdcfb640 1 --2- 172.21.15.116:0/1738378591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c4eaad5d0 0x559c4eb29ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:27.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.936+0000 7f8cbe4fc640 1 -- 172.21.15.116:0/1738378591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4eaaeb40 msgr2=0x559c4eb29ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:27.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.936+0000 7f8cbe4fc640 1 --2- 172.21.15.116:0/1738378591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4eaaeb40 0x559c4eb29ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.936+0000 7f8cbe4fc640 1 -- 172.21.15.116:0/1738378591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c4eaad5d0 msgr2=0x559c4eb29ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:27.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.936+0000 7f8cbe4fc640 1 --2- 172.21.15.116:0/1738378591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c4eaad5d0 0x559c4eb29ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.936+0000 7f8cbe4fc640 1 -- 172.21.15.116:0/1738378591 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c4eacb060 con 0x559c4eab8370 2024-09-12T03:16:27.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.936+0000 7f8cbdcfb640 1 --2- 172.21.15.116:0/1738378591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c4eaad5d0 0x559c4eb29ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:27.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.936+0000 7f8cbe4fc640 1 --2- 172.21.15.116:0/1738378591 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559c4eab8370 0x559c4eaca830 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0005230 tx=0x7f8cb0004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b248f312dd7bc08f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:27.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.936+0000 7f8ca17fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:16:27.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.936+0000 7f8ca17fa640 1 -- 172.21.15.116:0/1738378591 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb002b060 con 0x559c4eab8370 2024-09-12T03:16:27.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.937+0000 7f8ca17fa640 1 -- 172.21.15.116:0/1738378591 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f8cb000d7e0 con 0x559c4eab8370 2024-09-12T03:16:27.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.937+0000 7f8cc2338740 1 -- 172.21.15.116:0/1738378591 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cb0018070 con 0x559c4eab8370 2024-09-12T03:16:27.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.937+0000 7f8ca17fa640 1 -- 172.21.15.116:0/1738378591 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8cb000a630 con 0x559c4eab8370 2024-09-12T03:16:27.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.937+0000 7f8cc2338740 10 client.18948 fetch_fsmap learned FSMap version 25 2024-09-12T03:16:27.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.937+0000 7f8cc2338740 1 -- 172.21.15.116:0/1738378591 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559c4eb32270 con 0x559c4eab8370 2024-09-12T03:16:27.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.937+0000 7f8ca17fa640 1 -- 172.21.15.116:0/1738378591 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cb0018070 con 0x559c4eab8370 2024-09-12T03:16:27.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.937+0000 7f8ca17fa640 1 -- 172.21.15.116:0/1738378591 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8cb001c420 con 0x559c4eab8370 2024-09-12T03:16:27.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.937+0000 7f8cc2338740 10 client.18948 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:16:27.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.937+0000 7f8cc2338740 10 client.18948 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:16:27.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.937+0000 7f8cc2338740 20 client.18948 populate_metadata read hostname 'smithi116' 2024-09-12T03:16:27.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.937+0000 7f8cc2338740 1 --2- 172.21.15.116:0/1738378591 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x559c4eb32880 0x559c4eb34c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:27.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.937+0000 7f8cc2338740 4 client.18948 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:16:27.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.937+0000 7f8cc2338740 1 -- 172.21.15.116:0/1738378591 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559c4ea1ad70 con 0x559c4eb32880 2024-09-12T03:16:27.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.937+0000 7f8cbdcfb640 1 --2- 172.21.15.116:0/1738378591 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x559c4eb32880 0x559c4eb34c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:27.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.938+0000 7f8ca17fa640 10 client.18948 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:16:27.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.938+0000 7f8cbdcfb640 1 --2- 172.21.15.116:0/1738378591 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x559c4eb32880 0x559c4eb34c70 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=398b92178e1aff9e server_cookie=2b30d7e87a9b124f in_seq=0 out_seq=0 2024-09-12T03:16:27.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.938+0000 7f8ca17fa640 1 -- 172.21.15.116:0/1738378591 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559c4ea1ad70 con 0x559c4eb32880 2024-09-12T03:16:27.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.938+0000 7f8ca17fa640 10 client.18948 handle_command_reply: tid=0 2024-09-12T03:16:27.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.938+0000 7f8cc2338740 1 client.18948 shutdown 2024-09-12T03:16:27.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.938+0000 7f8cc2338740 1 -- 172.21.15.116:0/1738378591 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559c4eab8370 msgr2=0x559c4eaca830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:27.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.938+0000 7f8cc2338740 1 --2- 172.21.15.116:0/1738378591 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559c4eab8370 0x559c4eaca830 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0005230 tx=0x7f8cb0004d50 comp rx=0 tx=0).stop 2024-09-12T03:16:27.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.938+0000 7f8cc2338740 1 -- 172.21.15.116:0/1738378591 shutdown_connections 2024-09-12T03:16:27.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.938+0000 7f8cc2338740 1 --2- 172.21.15.116:0/1738378591 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x559c4eb32880 0x559c4eb34c70 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.938+0000 7f8cc2338740 1 --2- 172.21.15.116:0/1738378591 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559c4eab8370 0x559c4eaca830 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.938+0000 7f8ca17fa640 0 client.18948 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:16:27.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.938+0000 7f8cc2338740 1 --2- 172.21.15.116:0/1738378591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4eaaeb40 0x559c4eb29ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.938+0000 7f8cc2338740 1 --2- 172.21.15.116:0/1738378591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c4eaad5d0 0x559c4eb29ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.938+0000 7f8cc2338740 1 -- 172.21.15.116:0/1738378591 >> 172.21.15.116:0/1738378591 conn(0x559c4ea76310 msgr2=0x559c4eab7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:27.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.938+0000 7f8cc2338740 1 -- 172.21.15.116:0/1738378591 shutdown_connections 2024-09-12T03:16:27.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.938+0000 7f8cc2338740 1 -- 172.21.15.116:0/1738378591 wait complete. 2024-09-12T03:16:27.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.938+0000 7f8cc2338740 20 client.18948 trim_cache size 0 max 16384 2024-09-12T03:16:27.948 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:16:27.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.939+0000 7f8cbff5f640 1 -- 172.21.15.116:0/3854647839 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8ca40a2b80 msgr2=0x7f8ca40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:27.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.939+0000 7f8cbff5f640 1 --2- 172.21.15.116:0/3854647839 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8ca40a2b80 0x7f8ca40a5040 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8ca802a990 tx=0x7f8ca8000bc0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.939+0000 7f8cbff5f640 1 -- 172.21.15.116:0/3854647839 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cb8100b30 msgr2=0x7f8cb81a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:27.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.939+0000 7f8cbff5f640 1 --2- 172.21.15.116:0/3854647839 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cb8100b30 0x7f8cb81a4690 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f8cb400c920 tx=0x7f8cb400cde0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.939+0000 7f8cbff5f640 1 -- 172.21.15.116:0/3854647839 shutdown_connections 2024-09-12T03:16:27.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.939+0000 7f8cbff5f640 1 --2- 172.21.15.116:0/3854647839 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f8ca40a2b80 0x7f8ca40a5040 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.939+0000 7f8cbff5f640 1 --2- 172.21.15.116:0/3854647839 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cb8105f40 0x7f8cb819e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.939+0000 7f8cbff5f640 1 --2- 172.21.15.116:0/3854647839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cb8101450 0x7f8cb81a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.939+0000 7f8cbff5f640 1 --2- 172.21.15.116:0/3854647839 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cb8100b30 0x7f8cb81a4690 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:27.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.939+0000 7f8cbff5f640 1 -- 172.21.15.116:0/3854647839 >> 172.21.15.116:0/3854647839 conn(0x7f8cb80fc7a0 msgr2=0x7f8cb81038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:27.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.939+0000 7f8cbff5f640 1 -- 172.21.15.116:0/3854647839 shutdown_connections 2024-09-12T03:16:27.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:27.939+0000 7f8cbff5f640 1 -- 172.21.15.116:0/3854647839 wait complete. 2024-09-12T03:16:27.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:27 smithi183 ceph-mon[35535]: pgmap v742: 97 pgs: 97 active+clean; 6.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:16:27.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:27 smithi177 ceph-mon[35305]: pgmap v742: 97 pgs: 97 active+clean; 6.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:16:28.764 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:16:29.010 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:29.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.511+0000 7f9d6e782640 1 -- 172.21.15.116:0/1889394083 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d68101de0 msgr2=0x7f9d681021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:29.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.511+0000 7f9d6e782640 1 --2- 172.21.15.116:0/1889394083 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d68101de0 0x7f9d681021c0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f9d58009900 tx=0x7f9d5802a410 comp rx=0 tx=0).stop 2024-09-12T03:16:29.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.511+0000 7f9d6e782640 1 -- 172.21.15.116:0/1889394083 shutdown_connections 2024-09-12T03:16:29.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.511+0000 7f9d6e782640 1 --2- 172.21.15.116:0/1889394083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d68106f10 0x7f9d6810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.511+0000 7f9d6e782640 1 --2- 172.21.15.116:0/1889394083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d68102790 0x7f9d681067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.511+0000 7f9d6e782640 1 --2- 172.21.15.116:0/1889394083 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d68101de0 0x7f9d681021c0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.511+0000 7f9d6e782640 1 -- 172.21.15.116:0/1889394083 >> 172.21.15.116:0/1889394083 conn(0x7f9d680fd8c0 msgr2=0x7f9d680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:29.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.511+0000 7f9d6e782640 1 -- 172.21.15.116:0/1889394083 shutdown_connections 2024-09-12T03:16:29.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.511+0000 7f9d6e782640 1 -- 172.21.15.116:0/1889394083 wait complete. 2024-09-12T03:16:29.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.511+0000 7f9d6e782640 1 Processor -- start 2024-09-12T03:16:29.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.512+0000 7f9d6e782640 1 -- start start 2024-09-12T03:16:29.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.512+0000 7f9d6e782640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d68101de0 0x7f9d68198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:29.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.512+0000 7f9d6e782640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d68102790 0x7f9d681989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:29.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.512+0000 7f9d6e782640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d68106f10 0x7f9d6819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:29.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.512+0000 7f9d6e782640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9d68111f90 con 0x7f9d68101de0 2024-09-12T03:16:29.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.512+0000 7f9d6e782640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9d68111e10 con 0x7f9d68102790 2024-09-12T03:16:29.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.512+0000 7f9d6e782640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9d68112110 con 0x7f9d68106f10 2024-09-12T03:16:29.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.512+0000 7f9d67fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d68101de0 0x7f9d68198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:29.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.512+0000 7f9d67fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d68101de0 0x7f9d68198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39902/0 (socket says 172.21.15.116:39902) 2024-09-12T03:16:29.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.512+0000 7f9d67fff640 1 -- 172.21.15.116:0/1131607518 learned_addr learned my addr 172.21.15.116:0/1131607518 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:29.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.512+0000 7f9d6cd1f640 1 --2- 172.21.15.116:0/1131607518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d68106f10 0x7f9d6819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:29.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.512+0000 7f9d5ffff640 1 --2- 172.21.15.116:0/1131607518 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d68102790 0x7f9d681989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:29.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.512+0000 7f9d67fff640 1 -- 172.21.15.116:0/1131607518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d68106f10 msgr2=0x7f9d6819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:29.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.512+0000 7f9d67fff640 1 --2- 172.21.15.116:0/1131607518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d68106f10 0x7f9d6819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.512+0000 7f9d67fff640 1 -- 172.21.15.116:0/1131607518 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d68102790 msgr2=0x7f9d681989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:29.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.512+0000 7f9d67fff640 1 --2- 172.21.15.116:0/1131607518 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d68102790 0x7f9d681989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.512+0000 7f9d67fff640 1 -- 172.21.15.116:0/1131607518 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d6819d440 con 0x7f9d68101de0 2024-09-12T03:16:29.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.512+0000 7f9d67fff640 1 --2- 172.21.15.116:0/1131607518 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d68101de0 0x7f9d68198490 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f9d58006010 tx=0x7f9d580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:29.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.513+0000 7f9d65ffb640 1 -- 172.21.15.116:0/1131607518 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5803f030 con 0x7f9d68101de0 2024-09-12T03:16:29.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.513+0000 7f9d65ffb640 1 -- 172.21.15.116:0/1131607518 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d58003ce0 con 0x7f9d68101de0 2024-09-12T03:16:29.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.513+0000 7f9d6e782640 1 -- 172.21.15.116:0/1131607518 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d6819d6d0 con 0x7f9d68101de0 2024-09-12T03:16:29.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.513+0000 7f9d65ffb640 1 -- 172.21.15.116:0/1131607518 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58035070 con 0x7f9d68101de0 2024-09-12T03:16:29.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.513+0000 7f9d6e782640 1 -- 172.21.15.116:0/1131607518 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d681a5010 con 0x7f9d68101de0 2024-09-12T03:16:29.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.515+0000 7f9d65ffb640 1 -- 172.21.15.116:0/1131607518 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f9d58042030 con 0x7f9d68101de0 2024-09-12T03:16:29.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.515+0000 7f9d70b5b740 1 Processor -- start 2024-09-12T03:16:29.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.515+0000 7f9d70b5b740 1 -- start start 2024-09-12T03:16:29.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.515+0000 7f9d70b5b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d07d853b40 0x55d07d852090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:29.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.515+0000 7f9d70b5b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d07d8525d0 0x55d07d852a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:29.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.515+0000 7f9d70b5b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d07d85d370 0x55d07d85f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:29.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.515+0000 7f9d70b5b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d07d7b82f0 con 0x55d07d853b40 2024-09-12T03:16:29.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.516+0000 7f9d70b5b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d07d75fcf0 con 0x55d07d8525d0 2024-09-12T03:16:29.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.516+0000 7f9d70b5b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d07d7b42b0 con 0x55d07d85d370 2024-09-12T03:16:29.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.516+0000 7f9d6cd1f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d07d853b40 0x55d07d852090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:29.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.516+0000 7f9d65ffb640 1 --2- 172.21.15.116:0/1131607518 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9d540a2c00 0x7f9d540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:29.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.516+0000 7f9d6cd1f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d07d853b40 0x55d07d852090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39906/0 (socket says 172.21.15.116:39906) 2024-09-12T03:16:29.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.516+0000 7f9d6cd1f640 1 -- 172.21.15.116:0/188255956 learned_addr learned my addr 172.21.15.116:0/188255956 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:29.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.516+0000 7f9d67fff640 1 --2- 172.21.15.116:0/188255956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d07d8525d0 0x55d07d852a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:29.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.516+0000 7f9d5ffff640 1 --2- 172.21.15.116:0/188255956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d07d85d370 0x55d07d85f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:29.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.516+0000 7f9d65ffb640 1 -- 172.21.15.116:0/1131607518 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f9d580ef5c0 con 0x7f9d68101de0 2024-09-12T03:16:29.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.516+0000 7f9d6cd1f640 1 --2- 172.21.15.116:0/1131607518 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9d540a2c00 0x7f9d540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:29.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.516+0000 7f9d6cd1f640 1 -- 172.21.15.116:0/188255956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d07d85d370 msgr2=0x55d07d85f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:29.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.516+0000 7f9d6cd1f640 1 --2- 172.21.15.116:0/188255956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d07d85d370 0x55d07d85f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.516+0000 7f9d6cd1f640 1 -- 172.21.15.116:0/188255956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d07d8525d0 msgr2=0x55d07d852a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:29.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.516+0000 7f9d6cd1f640 1 --2- 172.21.15.116:0/188255956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d07d8525d0 0x55d07d852a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.516+0000 7f9d6cd1f640 1 -- 172.21.15.116:0/188255956 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d07d7443a0 con 0x55d07d853b40 2024-09-12T03:16:29.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.516+0000 7f9d6cd1f640 1 --2- 172.21.15.116:0/188255956 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d07d853b40 0x55d07d852090 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f9d60005f40 tx=0x7f9d60014f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8507c8bdeffb94ec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:29.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.516+0000 7f9d5e7fc640 1 -- 172.21.15.116:0/188255956 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60012d90 con 0x55d07d853b40 2024-09-12T03:16:29.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.517+0000 7f9d5e7fc640 1 -- 172.21.15.116:0/188255956 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d60023070 con 0x55d07d853b40 2024-09-12T03:16:29.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.517+0000 7f9d5e7fc640 1 -- 172.21.15.116:0/188255956 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60005730 con 0x55d07d853b40 2024-09-12T03:16:29.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.517+0000 7f9d70b5b740 1 -- 172.21.15.116:0/188255956 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d07d853b40 msgr2=0x55d07d852090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:29.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.517+0000 7f9d6cd1f640 1 --2- 172.21.15.116:0/1131607518 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9d540a2c00 0x7f9d540a50c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9d60022000 tx=0x7f9d60020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:29.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.517+0000 7f9d70b5b740 1 --2- 172.21.15.116:0/188255956 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d07d853b40 0x55d07d852090 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f9d60005f40 tx=0x7f9d60014f70 comp rx=0 tx=0).stop 2024-09-12T03:16:29.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.517+0000 7f9d70b5b740 1 -- 172.21.15.116:0/188255956 shutdown_connections 2024-09-12T03:16:29.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.517+0000 7f9d70b5b740 1 --2- 172.21.15.116:0/188255956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d07d85d370 0x55d07d85f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.517+0000 7f9d70b5b740 1 --2- 172.21.15.116:0/188255956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d07d8525d0 0x55d07d852a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.517+0000 7f9d70b5b740 1 --2- 172.21.15.116:0/188255956 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d07d853b40 0x55d07d852090 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.517+0000 7f9d70b5b740 1 -- 172.21.15.116:0/188255956 >> 172.21.15.116:0/188255956 conn(0x55d07d81b310 msgr2=0x55d07d84ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:29.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.517+0000 7f9d70b5b740 1 -- 172.21.15.116:0/188255956 shutdown_connections 2024-09-12T03:16:29.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.517+0000 7f9d70b5b740 1 -- 172.21.15.116:0/188255956 wait complete. 2024-09-12T03:16:29.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.518+0000 7f9d70b5b740 1 -- start start 2024-09-12T03:16:29.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.518+0000 7f9d70b5b740 1 Processor -- start 2024-09-12T03:16:29.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.518+0000 7f9d70b5b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d07d8525d0 0x55d07d85cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:29.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.518+0000 7f9d70b5b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d07d853b40 0x55d07d85a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:29.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.518+0000 7f9d70b5b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d07d85d370 0x55d07d85a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:29.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.518+0000 7f9d70b5b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d07d860070 con 0x55d07d85d370 2024-09-12T03:16:29.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.518+0000 7f9d70b5b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d07d773060 con 0x55d07d853b40 2024-09-12T03:16:29.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.518+0000 7f9d70b5b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d07d8601f0 con 0x55d07d8525d0 2024-09-12T03:16:29.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.518+0000 7f9d70b5b740 10 client.0 _ll_register_callbacks cb 0x55d07d4c68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:29.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.518+0000 7f9d6cd1f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d07d85d370 0x55d07d85a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:29.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.518+0000 7f9d6cd1f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d07d85d370 0x55d07d85a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39918/0 (socket says 172.21.15.116:39918) 2024-09-12T03:16:29.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.518+0000 7f9d6cd1f640 1 -- 172.21.15.116:0/1621096901 learned_addr learned my addr 172.21.15.116:0/1621096901 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:29.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.519+0000 7f9d67fff640 1 --2- 172.21.15.116:0/1621096901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d07d8525d0 0x55d07d85cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:29.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.519+0000 7f9d5ffff640 1 --2- 172.21.15.116:0/1621096901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d07d853b40 0x55d07d85a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:29.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.519+0000 7f9d6cd1f640 1 -- 172.21.15.116:0/1621096901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d07d8525d0 msgr2=0x55d07d85cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:29.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.519+0000 7f9d6cd1f640 1 --2- 172.21.15.116:0/1621096901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d07d8525d0 0x55d07d85cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.519+0000 7f9d6cd1f640 1 -- 172.21.15.116:0/1621096901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d07d853b40 msgr2=0x55d07d85a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:29.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.519+0000 7f9d6cd1f640 1 --2- 172.21.15.116:0/1621096901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d07d853b40 0x55d07d85a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.519+0000 7f9d6cd1f640 1 -- 172.21.15.116:0/1621096901 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d07d5293c0 con 0x55d07d85d370 2024-09-12T03:16:29.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.519+0000 7f9d67fff640 1 --2- 172.21.15.116:0/1621096901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d07d8525d0 0x55d07d85cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:29.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.519+0000 7f9d5ffff640 1 --2- 172.21.15.116:0/1621096901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d07d853b40 0x55d07d85a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:29.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.519+0000 7f9d6cd1f640 1 --2- 172.21.15.116:0/1621096901 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d07d85d370 0x55d07d85a610 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f9d60005090 tx=0x7f9d600147f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed481b427cceae40 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:29.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.519+0000 7f9d477fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:16:29.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.519+0000 7f9d477fe640 1 -- 172.21.15.116:0/1621096901 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60037060 con 0x55d07d85d370 2024-09-12T03:16:29.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.519+0000 7f9d477fe640 1 -- 172.21.15.116:0/1621096901 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f9d600059d0 con 0x55d07d85d370 2024-09-12T03:16:29.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.519+0000 7f9d70b5b740 1 -- 172.21.15.116:0/1621096901 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d60023070 con 0x55d07d85d370 2024-09-12T03:16:29.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.519+0000 7f9d477fe640 1 -- 172.21.15.116:0/1621096901 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9d600144b0 con 0x55d07d85d370 2024-09-12T03:16:29.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.519+0000 7f9d70b5b740 10 client.18960 fetch_fsmap learned FSMap version 25 2024-09-12T03:16:29.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.519+0000 7f9d70b5b740 1 -- 172.21.15.116:0/1621096901 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d07d86bb10 con 0x55d07d85d370 2024-09-12T03:16:29.549 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.519+0000 7f9d477fe640 1 -- 172.21.15.116:0/1621096901 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d60023070 con 0x55d07d85d370 2024-09-12T03:16:29.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.519+0000 7f9d477fe640 1 -- 172.21.15.116:0/1621096901 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9d6000f040 con 0x55d07d85d370 2024-09-12T03:16:29.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.520+0000 7f9d70b5b740 10 client.18960 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:16:29.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.520+0000 7f9d70b5b740 10 client.18960 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:16:29.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.520+0000 7f9d70b5b740 20 client.18960 populate_metadata read hostname 'smithi116' 2024-09-12T03:16:29.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.520+0000 7f9d70b5b740 1 --2- 172.21.15.116:0/1621096901 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55d07d86cf20 0x55d07d86f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:29.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.520+0000 7f9d70b5b740 4 client.18960 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:16:29.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.520+0000 7f9d70b5b740 1 -- 172.21.15.116:0/1621096901 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d07d7c5860 con 0x55d07d86cf20 2024-09-12T03:16:29.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.520+0000 7f9d67fff640 1 --2- 172.21.15.116:0/1621096901 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55d07d86cf20 0x55d07d86f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:29.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.520+0000 7f9d67fff640 1 --2- 172.21.15.116:0/1621096901 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55d07d86cf20 0x55d07d86f310 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=33e0ff10a18fe4c8 server_cookie=7e769e784163424c in_seq=0 out_seq=0 2024-09-12T03:16:29.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.520+0000 7f9d477fe640 10 client.18960 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:16:29.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.521+0000 7f9d477fe640 1 -- 172.21.15.116:0/1621096901 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d07d7c5860 con 0x55d07d86cf20 2024-09-12T03:16:29.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.521+0000 7f9d477fe640 10 client.18960 handle_command_reply: tid=0 2024-09-12T03:16:29.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.521+0000 7f9d70b5b740 1 client.18960 shutdown 2024-09-12T03:16:29.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.521+0000 7f9d70b5b740 1 -- 172.21.15.116:0/1621096901 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d07d85d370 msgr2=0x55d07d85a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:29.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.521+0000 7f9d70b5b740 1 --2- 172.21.15.116:0/1621096901 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d07d85d370 0x55d07d85a610 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f9d60005090 tx=0x7f9d600147f0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.521+0000 7f9d70b5b740 1 -- 172.21.15.116:0/1621096901 shutdown_connections 2024-09-12T03:16:29.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.521+0000 7f9d70b5b740 1 --2- 172.21.15.116:0/1621096901 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55d07d86cf20 0x55d07d86f310 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.521+0000 7f9d70b5b740 1 --2- 172.21.15.116:0/1621096901 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d07d85d370 0x55d07d85a610 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.521+0000 7f9d70b5b740 1 --2- 172.21.15.116:0/1621096901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d07d853b40 0x55d07d85a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.521+0000 7f9d70b5b740 1 --2- 172.21.15.116:0/1621096901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d07d8525d0 0x55d07d85cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.521+0000 7f9d477fe640 0 client.18960 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:16:29.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.521+0000 7f9d70b5b740 1 -- 172.21.15.116:0/1621096901 >> 172.21.15.116:0/1621096901 conn(0x55d07d81b310 msgr2=0x55d07d854960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:29.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.521+0000 7f9d70b5b740 1 -- 172.21.15.116:0/1621096901 shutdown_connections 2024-09-12T03:16:29.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.521+0000 7f9d70b5b740 1 -- 172.21.15.116:0/1621096901 wait complete. 2024-09-12T03:16:29.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.522+0000 7f9d70b5b740 20 client.18960 trim_cache size 0 max 16384 2024-09-12T03:16:29.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.534+0000 7f9d70b5b740 1 Processor -- start 2024-09-12T03:16:29.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.534+0000 7f9d70b5b740 1 -- start start 2024-09-12T03:16:29.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.534+0000 7f9d70b5b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d07d8525d0 0x55d07d85bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:29.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.534+0000 7f9d70b5b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d07d853b40 0x55d07d855120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:29.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.534+0000 7f9d70b5b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d07d85d370 0x55d07d855660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:29.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.534+0000 7f9d70b5b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9d681afdb0 con 0x55d07d85d370 2024-09-12T03:16:29.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.534+0000 7f9d70b5b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9d681ad680 con 0x55d07d8525d0 2024-09-12T03:16:29.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.534+0000 7f9d70b5b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9d681ab4d0 con 0x55d07d853b40 2024-09-12T03:16:29.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.535+0000 7f9d6cd1f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d07d85d370 0x55d07d855660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:29.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.535+0000 7f9d6cd1f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d07d85d370 0x55d07d855660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39924/0 (socket says 172.21.15.116:39924) 2024-09-12T03:16:29.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.535+0000 7f9d6cd1f640 1 -- 172.21.15.116:0/3507746315 learned_addr learned my addr 172.21.15.116:0/3507746315 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:29.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.535+0000 7f9d67fff640 1 --2- 172.21.15.116:0/3507746315 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d07d8525d0 0x55d07d85bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:29.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.535+0000 7f9d5ffff640 1 --2- 172.21.15.116:0/3507746315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d07d853b40 0x55d07d855120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:29.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.535+0000 7f9d6cd1f640 1 -- 172.21.15.116:0/3507746315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d07d853b40 msgr2=0x55d07d855120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:29.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.535+0000 7f9d6cd1f640 1 --2- 172.21.15.116:0/3507746315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d07d853b40 0x55d07d855120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.535+0000 7f9d6cd1f640 1 -- 172.21.15.116:0/3507746315 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d07d8525d0 msgr2=0x55d07d85bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:29.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.535+0000 7f9d6cd1f640 1 --2- 172.21.15.116:0/3507746315 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d07d8525d0 0x55d07d85bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.535+0000 7f9d6cd1f640 1 -- 172.21.15.116:0/3507746315 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d07d84f730 con 0x55d07d85d370 2024-09-12T03:16:29.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.535+0000 7f9d5ffff640 1 --2- 172.21.15.116:0/3507746315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d07d853b40 0x55d07d855120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:29.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.535+0000 7f9d67fff640 1 --2- 172.21.15.116:0/3507746315 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d07d8525d0 0x55d07d85bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:29.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.535+0000 7f9d6cd1f640 1 --2- 172.21.15.116:0/3507746315 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d07d85d370 0x55d07d855660 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f9d60027580 tx=0x7f9d60009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf351b257240daa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:29.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.535+0000 7f9d5dffb640 1 -- 172.21.15.116:0/3507746315 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60042040 con 0x55d07d85d370 2024-09-12T03:16:29.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.535+0000 7f9d5dffb640 1 -- 172.21.15.116:0/3507746315 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d60023070 con 0x55d07d85d370 2024-09-12T03:16:29.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.535+0000 7f9d5dffb640 1 -- 172.21.15.116:0/3507746315 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60037040 con 0x55d07d85d370 2024-09-12T03:16:29.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.536+0000 7f9d70b5b740 1 -- 172.21.15.116:0/3507746315 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d07d85d370 msgr2=0x55d07d855660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:29.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.536+0000 7f9d70b5b740 1 --2- 172.21.15.116:0/3507746315 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d07d85d370 0x55d07d855660 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f9d60027580 tx=0x7f9d60009a70 comp rx=0 tx=0).stop 2024-09-12T03:16:29.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.536+0000 7f9d70b5b740 1 -- 172.21.15.116:0/3507746315 shutdown_connections 2024-09-12T03:16:29.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.536+0000 7f9d70b5b740 1 --2- 172.21.15.116:0/3507746315 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d07d85d370 0x55d07d855660 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.536+0000 7f9d70b5b740 1 --2- 172.21.15.116:0/3507746315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d07d853b40 0x55d07d855120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.536+0000 7f9d70b5b740 1 --2- 172.21.15.116:0/3507746315 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d07d8525d0 0x55d07d85bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.536+0000 7f9d70b5b740 1 -- 172.21.15.116:0/3507746315 >> 172.21.15.116:0/3507746315 conn(0x55d07d81b310 msgr2=0x55d07d85b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:29.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.536+0000 7f9d70b5b740 1 -- 172.21.15.116:0/3507746315 shutdown_connections 2024-09-12T03:16:29.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.536+0000 7f9d70b5b740 1 -- 172.21.15.116:0/3507746315 wait complete. 2024-09-12T03:16:29.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.536+0000 7f9d70b5b740 1 -- start start 2024-09-12T03:16:29.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.536+0000 7f9d70b5b740 1 Processor -- start 2024-09-12T03:16:29.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.536+0000 7f9d70b5b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d07d8525d0 0x55d07d856fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:29.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.536+0000 7f9d70b5b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d07d853b40 0x55d07d857510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:29.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.536+0000 7f9d70b5b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d07d85d370 0x55d07d857a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:29.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.536+0000 7f9d70b5b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9d681abb40 con 0x55d07d8525d0 2024-09-12T03:16:29.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.536+0000 7f9d70b5b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9d681acfb0 con 0x55d07d85d370 2024-09-12T03:16:29.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.536+0000 7f9d70b5b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9d681ac210 con 0x55d07d853b40 2024-09-12T03:16:29.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.537+0000 7f9d70b5b740 10 client.0 _ll_register_callbacks cb 0x7f9d58039f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:29.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.537+0000 7f9d67fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d07d8525d0 0x55d07d856fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:29.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.537+0000 7f9d67fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d07d8525d0 0x55d07d856fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39932/0 (socket says 172.21.15.116:39932) 2024-09-12T03:16:29.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.537+0000 7f9d67fff640 1 -- 172.21.15.116:0/456033122 learned_addr learned my addr 172.21.15.116:0/456033122 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:29.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.537+0000 7f9d6cd1f640 1 --2- 172.21.15.116:0/456033122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d07d85d370 0x55d07d857a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:29.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.537+0000 7f9d5ffff640 1 --2- 172.21.15.116:0/456033122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d07d853b40 0x55d07d857510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:29.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.537+0000 7f9d67fff640 1 -- 172.21.15.116:0/456033122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d07d853b40 msgr2=0x55d07d857510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:29.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.537+0000 7f9d67fff640 1 --2- 172.21.15.116:0/456033122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d07d853b40 0x55d07d857510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.537+0000 7f9d67fff640 1 -- 172.21.15.116:0/456033122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d07d85d370 msgr2=0x55d07d857a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:29.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.537+0000 7f9d67fff640 1 --2- 172.21.15.116:0/456033122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d07d85d370 0x55d07d857a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.537+0000 7f9d67fff640 1 -- 172.21.15.116:0/456033122 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d07d8d6a50 con 0x55d07d8525d0 2024-09-12T03:16:29.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.537+0000 7f9d67fff640 1 --2- 172.21.15.116:0/456033122 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d07d8525d0 0x55d07d856fd0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f9d580ed140 tx=0x7f9d5808af50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c12bafb131c211 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:29.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.537+0000 7f9d477fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:16:29.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.537+0000 7f9d477fe640 1 -- 172.21.15.116:0/456033122 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58049040 con 0x55d07d8525d0 2024-09-12T03:16:29.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.537+0000 7f9d477fe640 1 -- 172.21.15.116:0/456033122 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f9d580b8580 con 0x55d07d8525d0 2024-09-12T03:16:29.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.537+0000 7f9d70b5b740 1 -- 172.21.15.116:0/456033122 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d60023070 con 0x55d07d8525d0 2024-09-12T03:16:29.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.537+0000 7f9d477fe640 1 -- 172.21.15.116:0/456033122 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9d5808a030 con 0x55d07d8525d0 2024-09-12T03:16:29.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.537+0000 7f9d70b5b740 10 client.18966 fetch_fsmap learned FSMap version 25 2024-09-12T03:16:29.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.537+0000 7f9d70b5b740 1 -- 172.21.15.116:0/456033122 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d07d8d7c20 con 0x55d07d8525d0 2024-09-12T03:16:29.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.538+0000 7f9d477fe640 1 -- 172.21.15.116:0/456033122 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d60023070 con 0x55d07d8525d0 2024-09-12T03:16:29.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.538+0000 7f9d477fe640 1 -- 172.21.15.116:0/456033122 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9d580bc170 con 0x55d07d8525d0 2024-09-12T03:16:29.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.538+0000 7f9d70b5b740 10 client.18966 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:16:29.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.538+0000 7f9d70b5b740 10 client.18966 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:16:29.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.538+0000 7f9d70b5b740 20 client.18966 populate_metadata read hostname 'smithi116' 2024-09-12T03:16:29.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.538+0000 7f9d70b5b740 1 --2- 172.21.15.116:0/456033122 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55d07d8d8340 0x55d07d8da730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:29.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.538+0000 7f9d70b5b740 4 client.18966 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:16:29.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.538+0000 7f9d70b5b740 1 -- 172.21.15.116:0/456033122 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d07d7bfd70 con 0x55d07d8d8340 2024-09-12T03:16:29.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.539+0000 7f9d5ffff640 1 --2- 172.21.15.116:0/456033122 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55d07d8d8340 0x55d07d8da730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:29.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.539+0000 7f9d5ffff640 1 --2- 172.21.15.116:0/456033122 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55d07d8d8340 0x55d07d8da730 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c6b781c9d878fd11 server_cookie=ee8269752dad219b in_seq=0 out_seq=0 2024-09-12T03:16:29.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.539+0000 7f9d477fe640 10 client.18966 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:16:29.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.539+0000 7f9d477fe640 1 -- 172.21.15.116:0/456033122 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d07d7bfd70 con 0x55d07d8d8340 2024-09-12T03:16:29.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.539+0000 7f9d477fe640 10 client.18966 handle_command_reply: tid=0 2024-09-12T03:16:29.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.539+0000 7f9d70b5b740 1 client.18966 shutdown 2024-09-12T03:16:29.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.539+0000 7f9d70b5b740 1 -- 172.21.15.116:0/456033122 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d07d8525d0 msgr2=0x55d07d856fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:29.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.539+0000 7f9d70b5b740 1 --2- 172.21.15.116:0/456033122 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d07d8525d0 0x55d07d856fd0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f9d580ed140 tx=0x7f9d5808af50 comp rx=0 tx=0).stop 2024-09-12T03:16:29.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.540+0000 7f9d70b5b740 1 -- 172.21.15.116:0/456033122 shutdown_connections 2024-09-12T03:16:29.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.540+0000 7f9d70b5b740 1 --2- 172.21.15.116:0/456033122 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55d07d8d8340 0x55d07d8da730 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.540+0000 7f9d70b5b740 1 --2- 172.21.15.116:0/456033122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d07d85d370 0x55d07d857a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.540+0000 7f9d70b5b740 1 --2- 172.21.15.116:0/456033122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d07d853b40 0x55d07d857510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.540+0000 7f9d70b5b740 1 --2- 172.21.15.116:0/456033122 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d07d8525d0 0x55d07d856fd0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.540+0000 7f9d70b5b740 1 -- 172.21.15.116:0/456033122 >> 172.21.15.116:0/456033122 conn(0x55d07d81b310 msgr2=0x55d07d85c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:29.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.540+0000 7f9d477fe640 0 client.18966 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:16:29.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.540+0000 7f9d70b5b740 1 -- 172.21.15.116:0/456033122 shutdown_connections 2024-09-12T03:16:29.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.540+0000 7f9d70b5b740 1 -- 172.21.15.116:0/456033122 wait complete. 2024-09-12T03:16:29.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.540+0000 7f9d70b5b740 20 client.18966 trim_cache size 0 max 16384 2024-09-12T03:16:29.559 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:16:29.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.540+0000 7f9d6e782640 1 -- 172.21.15.116:0/1131607518 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9d540a2c00 msgr2=0x7f9d540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:29.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.540+0000 7f9d6e782640 1 --2- 172.21.15.116:0/1131607518 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9d540a2c00 0x7f9d540a50c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9d60022000 tx=0x7f9d60020000 comp rx=0 tx=0).stop 2024-09-12T03:16:29.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.540+0000 7f9d6e782640 1 -- 172.21.15.116:0/1131607518 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d68101de0 msgr2=0x7f9d68198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:29.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.540+0000 7f9d6e782640 1 --2- 172.21.15.116:0/1131607518 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d68101de0 0x7f9d68198490 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f9d58006010 tx=0x7f9d580042f0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.540+0000 7f9d6e782640 1 -- 172.21.15.116:0/1131607518 shutdown_connections 2024-09-12T03:16:29.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.540+0000 7f9d6e782640 1 --2- 172.21.15.116:0/1131607518 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9d540a2c00 0x7f9d540a50c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.540+0000 7f9d6e782640 1 --2- 172.21.15.116:0/1131607518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d68106f10 0x7f9d6819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.540+0000 7f9d6e782640 1 --2- 172.21.15.116:0/1131607518 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d68102790 0x7f9d681989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.540+0000 7f9d6e782640 1 --2- 172.21.15.116:0/1131607518 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d68101de0 0x7f9d68198490 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:29.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.540+0000 7f9d6e782640 1 -- 172.21.15.116:0/1131607518 >> 172.21.15.116:0/1131607518 conn(0x7f9d680fd8c0 msgr2=0x7f9d680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:29.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.540+0000 7f9d6e782640 1 -- 172.21.15.116:0/1131607518 shutdown_connections 2024-09-12T03:16:29.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:29.541+0000 7f9d6e782640 1 -- 172.21.15.116:0/1131607518 wait complete. 2024-09-12T03:16:29.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:29 smithi183 ceph-mon[35535]: pgmap v743: 97 pgs: 97 active+clean; 6.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:16:29.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:29 smithi177 ceph-mon[35305]: pgmap v743: 97 pgs: 97 active+clean; 6.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:16:29.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:29 smithi116 ceph-mon[31410]: pgmap v743: 97 pgs: 97 active+clean; 6.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:16:30.415 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:16:30.659 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:31.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.189+0000 7eff5d290640 1 -- 172.21.15.116:0/344965278 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff581002f0 msgr2=0x7eff581006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:31.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.189+0000 7eff5d290640 1 --2- 172.21.15.116:0/344965278 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff581002f0 0x7eff581006d0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7eff48009930 tx=0x7eff4802a420 comp rx=0 tx=0).stop 2024-09-12T03:16:31.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.189+0000 7eff5d290640 1 -- 172.21.15.116:0/344965278 shutdown_connections 2024-09-12T03:16:31.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.189+0000 7eff5d290640 1 --2- 172.21.15.116:0/344965278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff58106f50 0x7eff5810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.189+0000 7eff5d290640 1 --2- 172.21.15.116:0/344965278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff58100ca0 0x7eff58106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.189+0000 7eff5d290640 1 --2- 172.21.15.116:0/344965278 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff581002f0 0x7eff581006d0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.189+0000 7eff5d290640 1 -- 172.21.15.116:0/344965278 >> 172.21.15.116:0/344965278 conn(0x7eff58077210 msgr2=0x7eff580fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:31.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.189+0000 7eff5d290640 1 -- 172.21.15.116:0/344965278 shutdown_connections 2024-09-12T03:16:31.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.189+0000 7eff5d290640 1 -- 172.21.15.116:0/344965278 wait complete. 2024-09-12T03:16:31.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.190+0000 7eff5d290640 1 Processor -- start 2024-09-12T03:16:31.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.190+0000 7eff5d290640 1 -- start start 2024-09-12T03:16:31.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.190+0000 7eff5d290640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff581002f0 0x7eff58198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:31.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.190+0000 7eff5d290640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff58100ca0 0x7eff581989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:31.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.190+0000 7eff5d290640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff58106f50 0x7eff5819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:31.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.190+0000 7eff5d290640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7eff58111fa0 con 0x7eff58100ca0 2024-09-12T03:16:31.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.190+0000 7eff5d290640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7eff58111e20 con 0x7eff581002f0 2024-09-12T03:16:31.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.190+0000 7eff5d290640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff58112120 con 0x7eff58106f50 2024-09-12T03:16:31.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.190+0000 7eff4e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff58100ca0 0x7eff581989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:31.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.191+0000 7eff4e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff58100ca0 0x7eff581989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47838/0 (socket says 172.21.15.116:47838) 2024-09-12T03:16:31.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.191+0000 7eff4e7fc640 1 -- 172.21.15.116:0/209694235 learned_addr learned my addr 172.21.15.116:0/209694235 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:31.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.191+0000 7eff56ffd640 1 --2- 172.21.15.116:0/209694235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff581002f0 0x7eff58198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:31.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.191+0000 7eff577fe640 1 --2- 172.21.15.116:0/209694235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff58106f50 0x7eff5819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:31.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.191+0000 7eff4e7fc640 1 -- 172.21.15.116:0/209694235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff58106f50 msgr2=0x7eff5819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:31.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.191+0000 7eff4e7fc640 1 --2- 172.21.15.116:0/209694235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff58106f50 0x7eff5819cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.191+0000 7eff4e7fc640 1 -- 172.21.15.116:0/209694235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff581002f0 msgr2=0x7eff58198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:31.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.191+0000 7eff4e7fc640 1 --2- 172.21.15.116:0/209694235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff581002f0 0x7eff58198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.191+0000 7eff4e7fc640 1 -- 172.21.15.116:0/209694235 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5819d430 con 0x7eff58100ca0 2024-09-12T03:16:31.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.191+0000 7eff4e7fc640 1 --2- 172.21.15.116:0/209694235 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff58100ca0 0x7eff581989c0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7eff4000d770 tx=0x7eff4000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:31.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.191+0000 7eff54ff9640 1 -- 172.21.15.116:0/209694235 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff40017070 con 0x7eff58100ca0 2024-09-12T03:16:31.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.191+0000 7eff54ff9640 1 -- 172.21.15.116:0/209694235 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff40009c30 con 0x7eff58100ca0 2024-09-12T03:16:31.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.191+0000 7eff5d290640 1 -- 172.21.15.116:0/209694235 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff5819d720 con 0x7eff58100ca0 2024-09-12T03:16:31.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.191+0000 7eff54ff9640 1 -- 172.21.15.116:0/209694235 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4000f620 con 0x7eff58100ca0 2024-09-12T03:16:31.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.191+0000 7eff5d290640 1 -- 172.21.15.116:0/209694235 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff581a5000 con 0x7eff58100ca0 2024-09-12T03:16:31.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.193+0000 7eff54ff9640 1 -- 172.21.15.116:0/209694235 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7eff4001a070 con 0x7eff58100ca0 2024-09-12T03:16:31.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.194+0000 7eff5f669740 1 Processor -- start 2024-09-12T03:16:31.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.194+0000 7eff5f669740 1 -- start start 2024-09-12T03:16:31.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.194+0000 7eff5f669740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a44f3ab40 0x562a44f39090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:31.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.194+0000 7eff5f669740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562a44f395d0 0x562a44f39a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:31.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.194+0000 7eff5f669740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562a44f44370 0x562a44f46760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:31.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.194+0000 7eff5f669740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562a44e9f2f0 con 0x562a44f44370 2024-09-12T03:16:31.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.194+0000 7eff5f669740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562a44e46cf0 con 0x562a44f395d0 2024-09-12T03:16:31.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.194+0000 7eff5f669740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562a44e9b2b0 con 0x562a44f3ab40 2024-09-12T03:16:31.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.194+0000 7eff4e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562a44f44370 0x562a44f46760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:31.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.194+0000 7eff4e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562a44f44370 0x562a44f46760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47850/0 (socket says 172.21.15.116:47850) 2024-09-12T03:16:31.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.194+0000 7eff4e7fc640 1 -- 172.21.15.116:0/3024359896 learned_addr learned my addr 172.21.15.116:0/3024359896 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:31.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.194+0000 7eff577fe640 1 --2- 172.21.15.116:0/3024359896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a44f3ab40 0x562a44f39090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:31.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.194+0000 7eff56ffd640 1 --2- 172.21.15.116:0/3024359896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562a44f395d0 0x562a44f39a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:31.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.195+0000 7eff54ff9640 1 --2- 172.21.15.116:0/209694235 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7eff440a2b80 0x7eff440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:31.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.195+0000 7eff4e7fc640 1 -- 172.21.15.116:0/3024359896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a44f3ab40 msgr2=0x562a44f39090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:31.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.195+0000 7eff4e7fc640 1 --2- 172.21.15.116:0/3024359896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a44f3ab40 0x562a44f39090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.195+0000 7eff4e7fc640 1 -- 172.21.15.116:0/3024359896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562a44f395d0 msgr2=0x562a44f39a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:31.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.195+0000 7eff4e7fc640 1 --2- 172.21.15.116:0/3024359896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562a44f395d0 0x562a44f39a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.195+0000 7eff4e7fc640 1 -- 172.21.15.116:0/3024359896 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a44e2b3a0 con 0x562a44f44370 2024-09-12T03:16:31.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.195+0000 7eff577fe640 1 --2- 172.21.15.116:0/209694235 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7eff440a2b80 0x7eff440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:31.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.195+0000 7eff54ff9640 1 -- 172.21.15.116:0/209694235 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7eff400cea70 con 0x7eff58100ca0 2024-09-12T03:16:31.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.195+0000 7eff4e7fc640 1 --2- 172.21.15.116:0/3024359896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562a44f44370 0x562a44f46760 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7eff40073960 tx=0x7eff40066030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65084e78749298c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:31.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.195+0000 7eff4d7fa640 1 -- 172.21.15.116:0/3024359896 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff400d8040 con 0x562a44f44370 2024-09-12T03:16:31.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.195+0000 7eff577fe640 1 --2- 172.21.15.116:0/209694235 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7eff440a2b80 0x7eff440a5040 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7eff50005fd0 tx=0x7eff50004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:31.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.195+0000 7eff4d7fa640 1 -- 172.21.15.116:0/3024359896 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff400737a0 con 0x562a44f44370 2024-09-12T03:16:31.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.195+0000 7eff4d7fa640 1 -- 172.21.15.116:0/3024359896 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff400b9f40 con 0x562a44f44370 2024-09-12T03:16:31.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.196+0000 7eff5f669740 1 -- 172.21.15.116:0/3024359896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562a44f44370 msgr2=0x562a44f46760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:31.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.196+0000 7eff5f669740 1 --2- 172.21.15.116:0/3024359896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562a44f44370 0x562a44f46760 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7eff40073960 tx=0x7eff40066030 comp rx=0 tx=0).stop 2024-09-12T03:16:31.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.196+0000 7eff5f669740 1 -- 172.21.15.116:0/3024359896 shutdown_connections 2024-09-12T03:16:31.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.196+0000 7eff5f669740 1 --2- 172.21.15.116:0/3024359896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562a44f44370 0x562a44f46760 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.196+0000 7eff5f669740 1 --2- 172.21.15.116:0/3024359896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562a44f395d0 0x562a44f39a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.196+0000 7eff5f669740 1 --2- 172.21.15.116:0/3024359896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a44f3ab40 0x562a44f39090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.196+0000 7eff5f669740 1 -- 172.21.15.116:0/3024359896 >> 172.21.15.116:0/3024359896 conn(0x562a44f02310 msgr2=0x562a44f35c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:31.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.196+0000 7eff5f669740 1 -- 172.21.15.116:0/3024359896 shutdown_connections 2024-09-12T03:16:31.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.196+0000 7eff5f669740 1 -- 172.21.15.116:0/3024359896 wait complete. 2024-09-12T03:16:31.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.196+0000 7eff5f669740 1 -- start start 2024-09-12T03:16:31.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.197+0000 7eff5f669740 1 Processor -- start 2024-09-12T03:16:31.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.197+0000 7eff5f669740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a44f395d0 0x562a44f43d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:31.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.197+0000 7eff5f669740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562a44f3ab40 0x562a44f40fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:31.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.197+0000 7eff5f669740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562a44f44370 0x562a44f41530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:31.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.197+0000 7eff5f669740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562a44f47010 con 0x562a44f44370 2024-09-12T03:16:31.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.197+0000 7eff5f669740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562a44e5a060 con 0x562a44f3ab40 2024-09-12T03:16:31.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.197+0000 7eff5f669740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562a44f47190 con 0x562a44f395d0 2024-09-12T03:16:31.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.197+0000 7eff5f669740 10 client.0 _ll_register_callbacks cb 0x562a44bad8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:31.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.197+0000 7eff577fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562a44f44370 0x562a44f41530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:31.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.197+0000 7eff577fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562a44f44370 0x562a44f41530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47856/0 (socket says 172.21.15.116:47856) 2024-09-12T03:16:31.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.197+0000 7eff577fe640 1 -- 172.21.15.116:0/824636338 learned_addr learned my addr 172.21.15.116:0/824636338 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:31.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.197+0000 7eff4e7fc640 1 --2- 172.21.15.116:0/824636338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562a44f3ab40 0x562a44f40fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:31.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.197+0000 7eff56ffd640 1 --2- 172.21.15.116:0/824636338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a44f395d0 0x562a44f43d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:31.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.197+0000 7eff577fe640 1 -- 172.21.15.116:0/824636338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a44f395d0 msgr2=0x562a44f43d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:31.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.197+0000 7eff577fe640 1 --2- 172.21.15.116:0/824636338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a44f395d0 0x562a44f43d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.197+0000 7eff577fe640 1 -- 172.21.15.116:0/824636338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562a44f3ab40 msgr2=0x562a44f40fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:31.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.197+0000 7eff577fe640 1 --2- 172.21.15.116:0/824636338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562a44f3ab40 0x562a44f40fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.197+0000 7eff577fe640 1 -- 172.21.15.116:0/824636338 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a44c103c0 con 0x562a44f44370 2024-09-12T03:16:31.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.197+0000 7eff577fe640 1 --2- 172.21.15.116:0/824636338 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562a44f44370 0x562a44f41530 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7eff500126f0 tx=0x7eff50004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9613076f1fabd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:31.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.197+0000 7eff3e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:16:31.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.198+0000 7eff3e7fc640 1 -- 172.21.15.116:0/824636338 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff500107d0 con 0x562a44f44370 2024-09-12T03:16:31.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.198+0000 7eff3e7fc640 1 -- 172.21.15.116:0/824636338 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7eff50010970 con 0x562a44f44370 2024-09-12T03:16:31.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.198+0000 7eff5f669740 1 -- 172.21.15.116:0/824636338 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562a44eac860 con 0x562a44f44370 2024-09-12T03:16:31.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.198+0000 7eff3e7fc640 1 -- 172.21.15.116:0/824636338 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7eff50010b70 con 0x562a44f44370 2024-09-12T03:16:31.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.198+0000 7eff5f669740 10 client.18978 fetch_fsmap learned FSMap version 25 2024-09-12T03:16:31.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.198+0000 7eff5f669740 1 -- 172.21.15.116:0/824636338 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562a44f52af0 con 0x562a44f44370 2024-09-12T03:16:31.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.198+0000 7eff3e7fc640 1 -- 172.21.15.116:0/824636338 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562a44eac860 con 0x562a44f44370 2024-09-12T03:16:31.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.198+0000 7eff3e7fc640 1 -- 172.21.15.116:0/824636338 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7eff5000f040 con 0x562a44f44370 2024-09-12T03:16:31.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.198+0000 7eff5f669740 10 client.18978 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:16:31.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.198+0000 7eff5f669740 10 client.18978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:16:31.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.198+0000 7eff5f669740 20 client.18978 populate_metadata read hostname 'smithi116' 2024-09-12T03:16:31.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.198+0000 7eff5f669740 1 --2- 172.21.15.116:0/824636338 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562a44f53fb0 0x562a44f563f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:31.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.198+0000 7eff5f669740 4 client.18978 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:16:31.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.198+0000 7eff5f669740 1 -- 172.21.15.116:0/824636338 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a44ea6d70 con 0x562a44f53fb0 2024-09-12T03:16:31.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.198+0000 7eff56ffd640 1 --2- 172.21.15.116:0/824636338 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562a44f53fb0 0x562a44f563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:31.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.199+0000 7eff3e7fc640 10 client.18978 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:31.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.199+0000 7eff56ffd640 1 --2- 172.21.15.116:0/824636338 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562a44f53fb0 0x562a44f563f0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70e919e8eba5746a server_cookie=5d674b76db7b1bce in_seq=0 out_seq=0 2024-09-12T03:16:31.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.200+0000 7eff3e7fc640 1 -- 172.21.15.116:0/824636338 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a44ea6d70 con 0x562a44f53fb0 2024-09-12T03:16:31.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.200+0000 7eff3e7fc640 10 client.18978 handle_command_reply: tid=0 2024-09-12T03:16:31.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.200+0000 7eff5f669740 1 client.18978 shutdown 2024-09-12T03:16:31.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.200+0000 7eff5f669740 1 -- 172.21.15.116:0/824636338 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562a44f44370 msgr2=0x562a44f41530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:31.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.200+0000 7eff5f669740 1 --2- 172.21.15.116:0/824636338 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562a44f44370 0x562a44f41530 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7eff500126f0 tx=0x7eff50004ab0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.200+0000 7eff5f669740 1 -- 172.21.15.116:0/824636338 shutdown_connections 2024-09-12T03:16:31.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.200+0000 7eff5f669740 1 --2- 172.21.15.116:0/824636338 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562a44f53fb0 0x562a44f563f0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.200+0000 7eff3e7fc640 0 client.18978 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:31.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.200+0000 7eff5f669740 1 --2- 172.21.15.116:0/824636338 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562a44f44370 0x562a44f41530 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.200+0000 7eff5f669740 1 --2- 172.21.15.116:0/824636338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562a44f3ab40 0x562a44f40fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.200+0000 7eff5f669740 1 --2- 172.21.15.116:0/824636338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a44f395d0 0x562a44f43d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.200+0000 7eff5f669740 1 -- 172.21.15.116:0/824636338 >> 172.21.15.116:0/824636338 conn(0x562a44f02310 msgr2=0x562a44f3b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:31.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.200+0000 7eff5f669740 1 -- 172.21.15.116:0/824636338 shutdown_connections 2024-09-12T03:16:31.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.200+0000 7eff5f669740 1 -- 172.21.15.116:0/824636338 wait complete. 2024-09-12T03:16:31.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.200+0000 7eff5f669740 20 client.18978 trim_cache size 0 max 16384 2024-09-12T03:16:31.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.213+0000 7eff5f669740 1 Processor -- start 2024-09-12T03:16:31.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.213+0000 7eff5f669740 1 -- start start 2024-09-12T03:16:31.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.213+0000 7eff5f669740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562a44f395d0 0x562a44f502b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:31.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.213+0000 7eff5f669740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a44f3ab40 0x562a44f40f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:31.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.213+0000 7eff5f669740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562a44f44370 0x562a44f414b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:31.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.213+0000 7eff5f669740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7eff581afd60 con 0x562a44f44370 2024-09-12T03:16:31.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.213+0000 7eff5f669740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7eff581ad630 con 0x562a44f395d0 2024-09-12T03:16:31.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.213+0000 7eff5f669740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff581ab480 con 0x562a44f3ab40 2024-09-12T03:16:31.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.214+0000 7eff577fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562a44f44370 0x562a44f414b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:31.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.214+0000 7eff577fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562a44f44370 0x562a44f414b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47860/0 (socket says 172.21.15.116:47860) 2024-09-12T03:16:31.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.214+0000 7eff577fe640 1 -- 172.21.15.116:0/2866011039 learned_addr learned my addr 172.21.15.116:0/2866011039 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:31.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.214+0000 7eff56ffd640 1 --2- 172.21.15.116:0/2866011039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562a44f395d0 0x562a44f502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:31.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.214+0000 7eff4e7fc640 1 --2- 172.21.15.116:0/2866011039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a44f3ab40 0x562a44f40f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:31.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.214+0000 7eff577fe640 1 -- 172.21.15.116:0/2866011039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a44f3ab40 msgr2=0x562a44f40f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:31.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.214+0000 7eff577fe640 1 --2- 172.21.15.116:0/2866011039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a44f3ab40 0x562a44f40f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.214+0000 7eff577fe640 1 -- 172.21.15.116:0/2866011039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562a44f395d0 msgr2=0x562a44f502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:31.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.214+0000 7eff577fe640 1 --2- 172.21.15.116:0/2866011039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562a44f395d0 0x562a44f502b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.214+0000 7eff577fe640 1 -- 172.21.15.116:0/2866011039 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a44f44070 con 0x562a44f44370 2024-09-12T03:16:31.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.214+0000 7eff4e7fc640 1 --2- 172.21.15.116:0/2866011039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a44f3ab40 0x562a44f40f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:31.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.214+0000 7eff56ffd640 1 --2- 172.21.15.116:0/2866011039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562a44f395d0 0x562a44f502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:31.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.214+0000 7eff577fe640 1 --2- 172.21.15.116:0/2866011039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562a44f44370 0x562a44f414b0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7eff50004830 tx=0x7eff50005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459b8867b5c3b26a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:31.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.214+0000 7eff4cff9640 1 -- 172.21.15.116:0/2866011039 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50024020 con 0x562a44f44370 2024-09-12T03:16:31.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.214+0000 7eff4cff9640 1 -- 172.21.15.116:0/2866011039 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff50025020 con 0x562a44f44370 2024-09-12T03:16:31.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.214+0000 7eff4cff9640 1 -- 172.21.15.116:0/2866011039 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5002a410 con 0x562a44f44370 2024-09-12T03:16:31.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.215+0000 7eff5f669740 1 -- 172.21.15.116:0/2866011039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562a44f44370 msgr2=0x562a44f414b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:31.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.215+0000 7eff5f669740 1 --2- 172.21.15.116:0/2866011039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562a44f44370 0x562a44f414b0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7eff50004830 tx=0x7eff50005d00 comp rx=0 tx=0).stop 2024-09-12T03:16:31.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.215+0000 7eff5f669740 1 -- 172.21.15.116:0/2866011039 shutdown_connections 2024-09-12T03:16:31.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.215+0000 7eff5f669740 1 --2- 172.21.15.116:0/2866011039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562a44f44370 0x562a44f414b0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.215+0000 7eff5f669740 1 --2- 172.21.15.116:0/2866011039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a44f3ab40 0x562a44f40f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.215+0000 7eff5f669740 1 --2- 172.21.15.116:0/2866011039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562a44f395d0 0x562a44f502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.215+0000 7eff5f669740 1 -- 172.21.15.116:0/2866011039 >> 172.21.15.116:0/2866011039 conn(0x562a44f02310 msgr2=0x562a44f3c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:31.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.215+0000 7eff5f669740 1 -- 172.21.15.116:0/2866011039 shutdown_connections 2024-09-12T03:16:31.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.215+0000 7eff5f669740 1 -- 172.21.15.116:0/2866011039 wait complete. 2024-09-12T03:16:31.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.216+0000 7eff5f669740 1 -- start start 2024-09-12T03:16:31.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.216+0000 7eff5f669740 1 Processor -- start 2024-09-12T03:16:31.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.216+0000 7eff5f669740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a44f395d0 0x562a44f44eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:31.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.216+0000 7eff5f669740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562a44f3ab40 0x562a44f453f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:31.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.216+0000 7eff5f669740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562a44f44370 0x562a44f4b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:31.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.216+0000 7eff5f669740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7eff581abaf0 con 0x562a44f3ab40 2024-09-12T03:16:31.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.216+0000 7eff5f669740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7eff581acf60 con 0x562a44f44370 2024-09-12T03:16:31.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.216+0000 7eff5f669740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff581ac1c0 con 0x562a44f395d0 2024-09-12T03:16:31.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.216+0000 7eff5f669740 10 client.0 _ll_register_callbacks cb 0x7eff50004860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:31.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.216+0000 7eff4e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562a44f3ab40 0x562a44f453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:31.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.216+0000 7eff4e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562a44f3ab40 0x562a44f453f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47862/0 (socket says 172.21.15.116:47862) 2024-09-12T03:16:31.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.216+0000 7eff4e7fc640 1 -- 172.21.15.116:0/2730098066 learned_addr learned my addr 172.21.15.116:0/2730098066 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:31.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.216+0000 7eff56ffd640 1 --2- 172.21.15.116:0/2730098066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a44f395d0 0x562a44f44eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:31.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.216+0000 7eff4e7fc640 1 -- 172.21.15.116:0/2730098066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a44f395d0 msgr2=0x562a44f44eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:31.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.216+0000 7eff4e7fc640 1 --2- 172.21.15.116:0/2730098066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a44f395d0 0x562a44f44eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.216+0000 7eff4e7fc640 1 -- 172.21.15.116:0/2730098066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562a44f44370 msgr2=0x562a44f4b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:31.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.216+0000 7eff577fe640 1 --2- 172.21.15.116:0/2730098066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562a44f44370 0x562a44f4b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:31.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.216+0000 7eff4e7fc640 1 --2- 172.21.15.116:0/2730098066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562a44f44370 0x562a44f4b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.216+0000 7eff4e7fc640 1 -- 172.21.15.116:0/2730098066 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a44f45c80 con 0x562a44f3ab40 2024-09-12T03:16:31.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.216+0000 7eff56ffd640 1 --2- 172.21.15.116:0/2730098066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a44f395d0 0x562a44f44eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:31.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.217+0000 7eff4e7fc640 1 --2- 172.21.15.116:0/2730098066 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562a44f3ab40 0x562a44f453f0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7eff40010fd0 tx=0x7eff40095f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbfb792ea5b734aa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:31.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.217+0000 7eff3e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:16:31.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.217+0000 7eff3e7fc640 1 -- 172.21.15.116:0/2730098066 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff40021070 con 0x562a44f3ab40 2024-09-12T03:16:31.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.217+0000 7eff3e7fc640 1 -- 172.21.15.116:0/2730098066 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7eff400996b0 con 0x562a44f3ab40 2024-09-12T03:16:31.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.217+0000 7eff5f669740 1 -- 172.21.15.116:0/2730098066 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff50025020 con 0x562a44f3ab40 2024-09-12T03:16:31.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.217+0000 7eff3e7fc640 1 -- 172.21.15.116:0/2730098066 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7eff4009aef0 con 0x562a44f3ab40 2024-09-12T03:16:31.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.217+0000 7eff5f669740 10 client.18984 fetch_fsmap learned FSMap version 25 2024-09-12T03:16:31.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.217+0000 7eff5f669740 1 -- 172.21.15.116:0/2730098066 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562a44f4ca90 con 0x562a44f3ab40 2024-09-12T03:16:31.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.217+0000 7eff3e7fc640 1 -- 172.21.15.116:0/2730098066 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff50025020 con 0x562a44f3ab40 2024-09-12T03:16:31.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.217+0000 7eff3e7fc640 1 -- 172.21.15.116:0/2730098066 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7eff40097a20 con 0x562a44f3ab40 2024-09-12T03:16:31.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.217+0000 7eff5f669740 10 client.18984 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:16:31.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.217+0000 7eff5f669740 10 client.18984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:16:31.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.217+0000 7eff5f669740 20 client.18984 populate_metadata read hostname 'smithi116' 2024-09-12T03:16:31.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.217+0000 7eff5f669740 1 --2- 172.21.15.116:0/2730098066 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562a44f53fb0 0x562a44fc0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:31.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.217+0000 7eff5f669740 4 client.18984 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:16:31.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.217+0000 7eff5f669740 1 -- 172.21.15.116:0/2730098066 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562a44e23170 con 0x562a44f53fb0 2024-09-12T03:16:31.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.218+0000 7eff56ffd640 1 --2- 172.21.15.116:0/2730098066 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562a44f53fb0 0x562a44fc0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:31.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.218+0000 7eff56ffd640 1 --2- 172.21.15.116:0/2730098066 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562a44f53fb0 0x562a44fc0cf0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92812667effeb3e2 server_cookie=2d56c5fa0264da12 in_seq=0 out_seq=0 2024-09-12T03:16:31.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.218+0000 7eff3e7fc640 10 client.18984 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:31.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.218+0000 7eff3e7fc640 1 -- 172.21.15.116:0/2730098066 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562a44e23170 con 0x562a44f53fb0 2024-09-12T03:16:31.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.218+0000 7eff3e7fc640 10 client.18984 handle_command_reply: tid=0 2024-09-12T03:16:31.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.218+0000 7eff5f669740 1 client.18984 shutdown 2024-09-12T03:16:31.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.218+0000 7eff5f669740 1 -- 172.21.15.116:0/2730098066 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562a44f3ab40 msgr2=0x562a44f453f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:31.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.218+0000 7eff5f669740 1 --2- 172.21.15.116:0/2730098066 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562a44f3ab40 0x562a44f453f0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7eff40010fd0 tx=0x7eff40095f20 comp rx=0 tx=0).stop 2024-09-12T03:16:31.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.218+0000 7eff5f669740 1 -- 172.21.15.116:0/2730098066 shutdown_connections 2024-09-12T03:16:31.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.218+0000 7eff5f669740 1 --2- 172.21.15.116:0/2730098066 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562a44f53fb0 0x562a44fc0cf0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.218+0000 7eff5f669740 1 --2- 172.21.15.116:0/2730098066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562a44f44370 0x562a44f4b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.218+0000 7eff5f669740 1 --2- 172.21.15.116:0/2730098066 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562a44f3ab40 0x562a44f453f0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.218+0000 7eff5f669740 1 --2- 172.21.15.116:0/2730098066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a44f395d0 0x562a44f44eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.218+0000 7eff5f669740 1 -- 172.21.15.116:0/2730098066 >> 172.21.15.116:0/2730098066 conn(0x562a44f02310 msgr2=0x562a44f344e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:31.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.218+0000 7eff3e7fc640 0 client.18984 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:31.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.218+0000 7eff5f669740 1 -- 172.21.15.116:0/2730098066 shutdown_connections 2024-09-12T03:16:31.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.218+0000 7eff5f669740 1 -- 172.21.15.116:0/2730098066 wait complete. 2024-09-12T03:16:31.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.219+0000 7eff5f669740 20 client.18984 trim_cache size 0 max 16384 2024-09-12T03:16:31.227 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:16:31.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.219+0000 7eff5d290640 1 -- 172.21.15.116:0/209694235 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7eff440a2b80 msgr2=0x7eff440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:31.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.219+0000 7eff5d290640 1 --2- 172.21.15.116:0/209694235 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7eff440a2b80 0x7eff440a5040 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7eff50005fd0 tx=0x7eff50004a40 comp rx=0 tx=0).stop 2024-09-12T03:16:31.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.219+0000 7eff5d290640 1 -- 172.21.15.116:0/209694235 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff58100ca0 msgr2=0x7eff581989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:31.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.219+0000 7eff5d290640 1 --2- 172.21.15.116:0/209694235 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff58100ca0 0x7eff581989c0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7eff4000d770 tx=0x7eff4000dc30 comp rx=0 tx=0).stop 2024-09-12T03:16:31.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.219+0000 7eff5d290640 1 -- 172.21.15.116:0/209694235 shutdown_connections 2024-09-12T03:16:31.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.219+0000 7eff5d290640 1 --2- 172.21.15.116:0/209694235 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7eff440a2b80 0x7eff440a5040 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.219+0000 7eff5d290640 1 --2- 172.21.15.116:0/209694235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff58106f50 0x7eff5819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.219+0000 7eff5d290640 1 --2- 172.21.15.116:0/209694235 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff58100ca0 0x7eff581989c0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.219+0000 7eff5d290640 1 --2- 172.21.15.116:0/209694235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff581002f0 0x7eff58198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.219+0000 7eff5d290640 1 -- 172.21.15.116:0/209694235 >> 172.21.15.116:0/209694235 conn(0x7eff58077210 msgr2=0x7eff58105860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:31.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.219+0000 7eff5d290640 1 -- 172.21.15.116:0/209694235 shutdown_connections 2024-09-12T03:16:31.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:31.219+0000 7eff5d290640 1 -- 172.21.15.116:0/209694235 wait complete. 2024-09-12T03:16:31.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:31 smithi183 ceph-mon[35535]: pgmap v744: 97 pgs: 97 active+clean; 6.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.7 KiB/s wr, 9 op/s 2024-09-12T03:16:31.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:31 smithi177 ceph-mon[35305]: pgmap v744: 97 pgs: 97 active+clean; 6.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.7 KiB/s wr, 9 op/s 2024-09-12T03:16:31.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:31 smithi116 ceph-mon[31410]: pgmap v744: 97 pgs: 97 active+clean; 6.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.7 KiB/s wr, 9 op/s 2024-09-12T03:16:32.931 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:16:32.932 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:16:33.189 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:33.589 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:33 smithi116 ceph-mon[31410]: pgmap v745: 97 pgs: 97 active+clean; 6.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.7 KiB/s wr, 7 op/s 2024-09-12T03:16:33.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:33 smithi183 ceph-mon[35535]: pgmap v745: 97 pgs: 97 active+clean; 6.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.7 KiB/s wr, 7 op/s 2024-09-12T03:16:33.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.671+0000 7f1367872640 1 -- 172.21.15.116:0/2356242373 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1360106f10 msgr2=0x7f136010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:33.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.671+0000 7f1367872640 1 --2- 172.21.15.116:0/2356242373 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1360106f10 0x7f136010ea90 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f1358009ca0 tx=0x7f135802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.671+0000 7f1367872640 1 -- 172.21.15.116:0/2356242373 shutdown_connections 2024-09-12T03:16:33.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.671+0000 7f1367872640 1 --2- 172.21.15.116:0/2356242373 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1360106f10 0x7f136010ea90 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.671+0000 7f1367872640 1 --2- 172.21.15.116:0/2356242373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1360102790 0x7f13601067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.671+0000 7f1367872640 1 --2- 172.21.15.116:0/2356242373 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1360101de0 0x7f13601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.671+0000 7f1367872640 1 -- 172.21.15.116:0/2356242373 >> 172.21.15.116:0/2356242373 conn(0x7f13600fd8c0 msgr2=0x7f13600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:33.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.671+0000 7f1367872640 1 -- 172.21.15.116:0/2356242373 shutdown_connections 2024-09-12T03:16:33.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.671+0000 7f1367872640 1 -- 172.21.15.116:0/2356242373 wait complete. 2024-09-12T03:16:33.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.672+0000 7f1367872640 1 Processor -- start 2024-09-12T03:16:33.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.672+0000 7f1367872640 1 -- start start 2024-09-12T03:16:33.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.672+0000 7f1367872640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1360101de0 0x7f13601983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:33.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.672+0000 7f1367872640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1360102790 0x7f13601988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:33.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.672+0000 7f1367872640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1360106f10 0x7f136019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:33.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.672+0000 7f1367872640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1360111f70 con 0x7f1360101de0 2024-09-12T03:16:33.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.672+0000 7f1367872640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1360111df0 con 0x7f1360106f10 2024-09-12T03:16:33.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.672+0000 7f1367872640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f13601120f0 con 0x7f1360102790 2024-09-12T03:16:33.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.672+0000 7f136560e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1360101de0 0x7f13601983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:33.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.672+0000 7f136560e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1360101de0 0x7f13601983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47872/0 (socket says 172.21.15.116:47872) 2024-09-12T03:16:33.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.672+0000 7f136560e640 1 -- 172.21.15.116:0/4003342115 learned_addr learned my addr 172.21.15.116:0/4003342115 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:33.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.672+0000 7f1364e0d640 1 --2- 172.21.15.116:0/4003342115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1360102790 0x7f13601988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:33.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.672+0000 7f1365e0f640 1 --2- 172.21.15.116:0/4003342115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1360106f10 0x7f136019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:33.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.672+0000 7f136560e640 1 -- 172.21.15.116:0/4003342115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1360102790 msgr2=0x7f13601988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:33.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.672+0000 7f136560e640 1 --2- 172.21.15.116:0/4003342115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1360102790 0x7f13601988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.672+0000 7f136560e640 1 -- 172.21.15.116:0/4003342115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1360106f10 msgr2=0x7f136019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:33.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.672+0000 7f136560e640 1 --2- 172.21.15.116:0/4003342115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1360106f10 0x7f136019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.672+0000 7f136560e640 1 -- 172.21.15.116:0/4003342115 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f136019d360 con 0x7f1360101de0 2024-09-12T03:16:33.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.672+0000 7f1364e0d640 1 --2- 172.21.15.116:0/4003342115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1360102790 0x7f13601988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:33.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.673+0000 7f136560e640 1 --2- 172.21.15.116:0/4003342115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1360101de0 0x7f13601983b0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f135c00d950 tx=0x7f135c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:33.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.673+0000 7f13567fc640 1 -- 172.21.15.116:0/4003342115 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c017030 con 0x7f1360101de0 2024-09-12T03:16:33.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.673+0000 7f1367872640 1 -- 172.21.15.116:0/4003342115 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f136019d650 con 0x7f1360101de0 2024-09-12T03:16:33.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.673+0000 7f13567fc640 1 -- 172.21.15.116:0/4003342115 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f135c004020 con 0x7f1360101de0 2024-09-12T03:16:33.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.673+0000 7f13567fc640 1 -- 172.21.15.116:0/4003342115 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c0047a0 con 0x7f1360101de0 2024-09-12T03:16:33.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.673+0000 7f1367872640 1 -- 172.21.15.116:0/4003342115 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13601a4f30 con 0x7f1360101de0 2024-09-12T03:16:33.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.675+0000 7f13567fc640 1 -- 172.21.15.116:0/4003342115 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f135c0041d0 con 0x7f1360101de0 2024-09-12T03:16:33.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.676+0000 7f1369c4b740 1 Processor -- start 2024-09-12T03:16:33.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.676+0000 7f1369c4b740 1 -- start start 2024-09-12T03:16:33.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.676+0000 7f1369c4b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563fea887a20 0x563fea887e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:33.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.676+0000 7f1369c4b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fea888340 0x563fea896c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:33.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.676+0000 7f1369c4b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563fea897150 0x563fea899540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:33.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.676+0000 7f1369c4b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x563fea7ed280 con 0x563fea887a20 2024-09-12T03:16:33.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.676+0000 7f1369c4b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x563fea794b10 con 0x563fea888340 2024-09-12T03:16:33.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.676+0000 7f1369c4b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563fea7e9240 con 0x563fea897150 2024-09-12T03:16:33.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.676+0000 7f1365e0f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563fea887a20 0x563fea887e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:33.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.676+0000 7f1365e0f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563fea887a20 0x563fea887e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47878/0 (socket says 172.21.15.116:47878) 2024-09-12T03:16:33.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.676+0000 7f1365e0f640 1 -- 172.21.15.116:0/1377455718 learned_addr learned my addr 172.21.15.116:0/1377455718 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:33.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.676+0000 7f1364e0d640 1 --2- 172.21.15.116:0/1377455718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563fea897150 0x563fea899540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:33.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.676+0000 7f13567fc640 1 --2- 172.21.15.116:0/4003342115 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f134c0a2b80 0x7f134c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:33.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.676+0000 7f136560e640 1 --2- 172.21.15.116:0/1377455718 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fea888340 0x563fea896c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:33.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.676+0000 7f1365e0f640 1 -- 172.21.15.116:0/1377455718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563fea897150 msgr2=0x563fea899540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:33.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.676+0000 7f1365e0f640 1 --2- 172.21.15.116:0/1377455718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563fea897150 0x563fea899540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.676+0000 7f1365e0f640 1 -- 172.21.15.116:0/1377455718 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fea888340 msgr2=0x563fea896c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:33.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.676+0000 7f1365e0f640 1 --2- 172.21.15.116:0/1377455718 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fea888340 0x563fea896c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.676+0000 7f1365e0f640 1 -- 172.21.15.116:0/1377455718 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fea7a11f0 con 0x563fea887a20 2024-09-12T03:16:33.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.676+0000 7f1364e0d640 1 --2- 172.21.15.116:0/1377455718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563fea897150 0x563fea899540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.676+0000 7f13567fc640 1 -- 172.21.15.116:0/4003342115 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f135c0ceab0 con 0x7f1360101de0 2024-09-12T03:16:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.677+0000 7f1365e0f640 1 --2- 172.21.15.116:0/4003342115 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f134c0a2b80 0x7f134c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.677+0000 7f1365e0f640 1 --2- 172.21.15.116:0/1377455718 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563fea887a20 0x563fea887e00 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f1358005cc0 tx=0x7f1358009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265bd6f3958b6788 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.677+0000 7f134b7fe640 1 -- 172.21.15.116:0/1377455718 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1358041040 con 0x563fea887a20 2024-09-12T03:16:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.677+0000 7f134b7fe640 1 -- 172.21.15.116:0/1377455718 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1358036070 con 0x563fea887a20 2024-09-12T03:16:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.677+0000 7f134b7fe640 1 -- 172.21.15.116:0/1377455718 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135802f8c0 con 0x563fea887a20 2024-09-12T03:16:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.677+0000 7f1365e0f640 1 --2- 172.21.15.116:0/4003342115 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f134c0a2b80 0x7f134c0a5040 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f135804c000 tx=0x7f135804d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.678+0000 7f1369c4b740 1 -- 172.21.15.116:0/1377455718 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563fea887a20 msgr2=0x563fea887e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.678+0000 7f1369c4b740 1 --2- 172.21.15.116:0/1377455718 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563fea887a20 0x563fea887e00 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f1358005cc0 tx=0x7f1358009880 comp rx=0 tx=0).stop 2024-09-12T03:16:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.678+0000 7f1369c4b740 1 -- 172.21.15.116:0/1377455718 shutdown_connections 2024-09-12T03:16:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.678+0000 7f1369c4b740 1 --2- 172.21.15.116:0/1377455718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563fea897150 0x563fea899540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.678+0000 7f1369c4b740 1 --2- 172.21.15.116:0/1377455718 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fea888340 0x563fea896c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.678+0000 7f1369c4b740 1 --2- 172.21.15.116:0/1377455718 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563fea887a20 0x563fea887e00 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.678+0000 7f1369c4b740 1 -- 172.21.15.116:0/1377455718 >> 172.21.15.116:0/1377455718 conn(0x563fea8840f0 msgr2=0x563fea8844d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:33.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.678+0000 7f1369c4b740 1 -- 172.21.15.116:0/1377455718 shutdown_connections 2024-09-12T03:16:33.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.678+0000 7f1369c4b740 1 -- 172.21.15.116:0/1377455718 wait complete. 2024-09-12T03:16:33.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.678+0000 7f1369c4b740 1 -- start start 2024-09-12T03:16:33.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.678+0000 7f1369c4b740 1 Processor -- start 2024-09-12T03:16:33.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.679+0000 7f1369c4b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563fea887a20 0x563fea8940b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:33.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.679+0000 7f1369c4b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fea888340 0x563fea8945f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:33.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.679+0000 7f1369c4b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563fea897150 0x563fea894b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:33.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.679+0000 7f1369c4b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x563fea899d20 con 0x563fea887a20 2024-09-12T03:16:33.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.679+0000 7f1369c4b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x563fea7a7f10 con 0x563fea888340 2024-09-12T03:16:33.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.679+0000 7f1369c4b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563fea899ea0 con 0x563fea897150 2024-09-12T03:16:33.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.679+0000 7f1369c4b740 10 client.0 _ll_register_callbacks cb 0x563fea770a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:33.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.679+0000 7f136560e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563fea887a20 0x563fea8940b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:33.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.679+0000 7f136560e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563fea887a20 0x563fea8940b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47880/0 (socket says 172.21.15.116:47880) 2024-09-12T03:16:33.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.679+0000 7f136560e640 1 -- 172.21.15.116:0/209530384 learned_addr learned my addr 172.21.15.116:0/209530384 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:33.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.679+0000 7f1364e0d640 1 --2- 172.21.15.116:0/209530384 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fea888340 0x563fea8945f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:33.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.679+0000 7f136560e640 1 -- 172.21.15.116:0/209530384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563fea897150 msgr2=0x563fea894b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:33.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.679+0000 7f1365e0f640 1 --2- 172.21.15.116:0/209530384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563fea897150 0x563fea894b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:33.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.679+0000 7f136560e640 1 --2- 172.21.15.116:0/209530384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563fea897150 0x563fea894b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.679+0000 7f136560e640 1 -- 172.21.15.116:0/209530384 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fea888340 msgr2=0x563fea8945f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:33.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.679+0000 7f136560e640 1 --2- 172.21.15.116:0/209530384 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fea888340 0x563fea8945f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.679+0000 7f136560e640 1 -- 172.21.15.116:0/209530384 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fea55e6a0 con 0x563fea887a20 2024-09-12T03:16:33.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.679+0000 7f1364e0d640 1 --2- 172.21.15.116:0/209530384 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fea888340 0x563fea8945f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:33.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.679+0000 7f1365e0f640 1 --2- 172.21.15.116:0/209530384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563fea897150 0x563fea894b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:33.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.679+0000 7f136560e640 1 --2- 172.21.15.116:0/209530384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563fea887a20 0x563fea8940b0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f135c09b2d0 tx=0x7f135c09aef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e66c22f1f4eb20 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:33.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.679+0000 7f1348ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:16:33.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.679+0000 7f1348ff9640 1 -- 172.21.15.116:0/209530384 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c09b340 con 0x563fea887a20 2024-09-12T03:16:33.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.679+0000 7f1348ff9640 1 -- 172.21.15.116:0/209530384 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f135c0b40b0 con 0x563fea887a20 2024-09-12T03:16:33.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.679+0000 7f1369c4b740 1 -- 172.21.15.116:0/209530384 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1358036070 con 0x563fea887a20 2024-09-12T03:16:33.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.679+0000 7f1348ff9640 1 -- 172.21.15.116:0/209530384 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f135c09f6f0 con 0x563fea887a20 2024-09-12T03:16:33.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.679+0000 7f1369c4b740 10 client.18996 fetch_fsmap learned FSMap version 25 2024-09-12T03:16:33.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.679+0000 7f1369c4b740 1 -- 172.21.15.116:0/209530384 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563fea8a59e0 con 0x563fea887a20 2024-09-12T03:16:33.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.680+0000 7f1348ff9640 1 -- 172.21.15.116:0/209530384 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1358036070 con 0x563fea887a20 2024-09-12T03:16:33.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.680+0000 7f1348ff9640 1 -- 172.21.15.116:0/209530384 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f135c09a790 con 0x563fea887a20 2024-09-12T03:16:33.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.680+0000 7f1369c4b740 10 client.18996 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:16:33.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.680+0000 7f1369c4b740 10 client.18996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:16:33.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.680+0000 7f1369c4b740 20 client.18996 populate_metadata read hostname 'smithi116' 2024-09-12T03:16:33.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.680+0000 7f1369c4b740 1 --2- 172.21.15.116:0/209530384 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x563fea8a6f40 0x563fea8a93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:33.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.680+0000 7f1369c4b740 4 client.18996 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:16:33.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.680+0000 7f1369c4b740 1 -- 172.21.15.116:0/209530384 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fea7fa7f0 con 0x563fea8a6f40 2024-09-12T03:16:33.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.680+0000 7f1364e0d640 1 --2- 172.21.15.116:0/209530384 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x563fea8a6f40 0x563fea8a93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:33.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.681+0000 7f1364e0d640 1 --2- 172.21.15.116:0/209530384 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x563fea8a6f40 0x563fea8a93a0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f615d6004edd1a6 server_cookie=44419985cc86fdb3 in_seq=0 out_seq=0 2024-09-12T03:16:33.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.681+0000 7f1348ff9640 10 client.18996 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:33.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.681+0000 7f1348ff9640 1 -- 172.21.15.116:0/209530384 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563fea7fa7f0 con 0x563fea8a6f40 2024-09-12T03:16:33.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.681+0000 7f1348ff9640 10 client.18996 handle_command_reply: tid=0 2024-09-12T03:16:33.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.681+0000 7f1369c4b740 1 client.18996 shutdown 2024-09-12T03:16:33.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.681+0000 7f1369c4b740 1 -- 172.21.15.116:0/209530384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563fea887a20 msgr2=0x563fea8940b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:33.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.681+0000 7f1369c4b740 1 --2- 172.21.15.116:0/209530384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563fea887a20 0x563fea8940b0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f135c09b2d0 tx=0x7f135c09aef0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.681+0000 7f1369c4b740 1 -- 172.21.15.116:0/209530384 shutdown_connections 2024-09-12T03:16:33.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.681+0000 7f1369c4b740 1 --2- 172.21.15.116:0/209530384 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x563fea8a6f40 0x563fea8a93a0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.681+0000 7f1369c4b740 1 --2- 172.21.15.116:0/209530384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563fea897150 0x563fea894b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.681+0000 7f1369c4b740 1 --2- 172.21.15.116:0/209530384 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fea888340 0x563fea8945f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.681+0000 7f1348ff9640 0 client.18996 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:33.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.681+0000 7f1369c4b740 1 --2- 172.21.15.116:0/209530384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563fea887a20 0x563fea8940b0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.681+0000 7f1369c4b740 1 -- 172.21.15.116:0/209530384 >> 172.21.15.116:0/209530384 conn(0x563fea8840f0 msgr2=0x563fea889e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:33.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.682+0000 7f1369c4b740 1 -- 172.21.15.116:0/209530384 shutdown_connections 2024-09-12T03:16:33.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.682+0000 7f1369c4b740 1 -- 172.21.15.116:0/209530384 wait complete. 2024-09-12T03:16:33.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.682+0000 7f1369c4b740 20 client.18996 trim_cache size 0 max 16384 2024-09-12T03:16:33.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.699+0000 7f1369c4b740 1 Processor -- start 2024-09-12T03:16:33.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.699+0000 7f1369c4b740 1 -- start start 2024-09-12T03:16:33.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.699+0000 7f1369c4b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563fea887a20 0x563fea8aec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:33.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.699+0000 7f1369c4b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fea888340 0x563fea8af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:33.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.699+0000 7f1369c4b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563fea897150 0x563fea891310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:33.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.699+0000 7f1369c4b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f13601ada90 con 0x563fea897150 2024-09-12T03:16:33.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.699+0000 7f1369c4b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f13601ab360 con 0x563fea888340 2024-09-12T03:16:33.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.699+0000 7f1369c4b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f13601a91b0 con 0x563fea887a20 2024-09-12T03:16:33.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.699+0000 7f1365e0f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563fea897150 0x563fea891310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:33.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.699+0000 7f1365e0f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563fea897150 0x563fea891310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47882/0 (socket says 172.21.15.116:47882) 2024-09-12T03:16:33.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.699+0000 7f1365e0f640 1 -- 172.21.15.116:0/1376510306 learned_addr learned my addr 172.21.15.116:0/1376510306 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:33.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.699+0000 7f1365e0f640 1 -- 172.21.15.116:0/1376510306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563fea887a20 msgr2=0x563fea8aec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:33.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.699+0000 7f136560e640 1 --2- 172.21.15.116:0/1376510306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563fea887a20 0x563fea8aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:33.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.699+0000 7f1365e0f640 1 --2- 172.21.15.116:0/1376510306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563fea887a20 0x563fea8aec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.699+0000 7f1365e0f640 1 -- 172.21.15.116:0/1376510306 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fea888340 msgr2=0x563fea8af1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:33.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.699+0000 7f1365e0f640 1 --2- 172.21.15.116:0/1376510306 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fea888340 0x563fea8af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.699+0000 7f1365e0f640 1 -- 172.21.15.116:0/1376510306 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fea883f00 con 0x563fea897150 2024-09-12T03:16:33.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.699+0000 7f136560e640 1 --2- 172.21.15.116:0/1376510306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563fea887a20 0x563fea8aec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:33.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.699+0000 7f1365e0f640 1 --2- 172.21.15.116:0/1376510306 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563fea897150 0x563fea891310 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f1358009b50 tx=0x7f1358005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ced48e688e5c7ec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:33.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.699+0000 7f134affd640 1 -- 172.21.15.116:0/1376510306 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1358041040 con 0x563fea897150 2024-09-12T03:16:33.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.699+0000 7f134affd640 1 -- 172.21.15.116:0/1376510306 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1358059070 con 0x563fea897150 2024-09-12T03:16:33.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.699+0000 7f134affd640 1 -- 172.21.15.116:0/1376510306 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135802f8e0 con 0x563fea897150 2024-09-12T03:16:33.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.700+0000 7f1369c4b740 1 -- 172.21.15.116:0/1376510306 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563fea897150 msgr2=0x563fea891310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:33.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.700+0000 7f1369c4b740 1 --2- 172.21.15.116:0/1376510306 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563fea897150 0x563fea891310 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f1358009b50 tx=0x7f1358005bf0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.700+0000 7f1369c4b740 1 -- 172.21.15.116:0/1376510306 shutdown_connections 2024-09-12T03:16:33.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.700+0000 7f1369c4b740 1 --2- 172.21.15.116:0/1376510306 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563fea897150 0x563fea891310 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.700+0000 7f1369c4b740 1 --2- 172.21.15.116:0/1376510306 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fea888340 0x563fea8af1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.700+0000 7f1369c4b740 1 --2- 172.21.15.116:0/1376510306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563fea887a20 0x563fea8aec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.700+0000 7f1369c4b740 1 -- 172.21.15.116:0/1376510306 >> 172.21.15.116:0/1376510306 conn(0x563fea8840f0 msgr2=0x563fea883270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:33.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.700+0000 7f1369c4b740 1 -- 172.21.15.116:0/1376510306 shutdown_connections 2024-09-12T03:16:33.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.700+0000 7f1369c4b740 1 -- 172.21.15.116:0/1376510306 wait complete. 2024-09-12T03:16:33.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.700+0000 7f1369c4b740 1 -- start start 2024-09-12T03:16:33.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.700+0000 7f1369c4b740 1 Processor -- start 2024-09-12T03:16:33.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.701+0000 7f1369c4b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563fea887a20 0x563fea89f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:33.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.701+0000 7f1369c4b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563fea888340 0x563fea89f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:33.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.701+0000 7f1369c4b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fea897150 0x563fea89b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:33.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.701+0000 7f1369c4b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f13601a9820 con 0x563fea888340 2024-09-12T03:16:33.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.701+0000 7f1369c4b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f13601aac90 con 0x563fea897150 2024-09-12T03:16:33.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.701+0000 7f1369c4b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f13601a9ef0 con 0x563fea887a20 2024-09-12T03:16:33.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.701+0000 7f1369c4b740 10 client.0 _ll_register_callbacks cb 0x7f135c09b1a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:33.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.701+0000 7f1364e0d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563fea888340 0x563fea89f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:33.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.701+0000 7f1364e0d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563fea888340 0x563fea89f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47896/0 (socket says 172.21.15.116:47896) 2024-09-12T03:16:33.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.701+0000 7f1364e0d640 1 -- 172.21.15.116:0/3728636882 learned_addr learned my addr 172.21.15.116:0/3728636882 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:33.719 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:16:33.719 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:16:33.719 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "034041a1-d893-49ae-a4d3-9b8c123f675c", 2024-09-12T03:16:33.719 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:16:33.719 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:16:33.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.701+0000 7f136560e640 1 --2- 172.21.15.116:0/3728636882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563fea887a20 0x563fea89f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:33.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.701+0000 7f1364e0d640 1 -- 172.21.15.116:0/3728636882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563fea887a20 msgr2=0x563fea89f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:33.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.701+0000 7f1364e0d640 1 --2- 172.21.15.116:0/3728636882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563fea887a20 0x563fea89f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.701+0000 7f1364e0d640 1 -- 172.21.15.116:0/3728636882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fea897150 msgr2=0x563fea89b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:33.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.701+0000 7f1364e0d640 1 --2- 172.21.15.116:0/3728636882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fea897150 0x563fea89b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.701+0000 7f1364e0d640 1 -- 172.21.15.116:0/3728636882 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fea89be70 con 0x563fea888340 2024-09-12T03:16:33.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.701+0000 7f1364e0d640 1 --2- 172.21.15.116:0/3728636882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563fea888340 0x563fea89f740 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f1350005d80 tx=0x7f1350004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843b63bd1d36ded0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:33.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.701+0000 7f1348ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:16:33.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.701+0000 7f1348ff9640 1 -- 172.21.15.116:0/3728636882 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350005250 con 0x563fea888340 2024-09-12T03:16:33.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.701+0000 7f1348ff9640 1 -- 172.21.15.116:0/3728636882 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f13500053f0 con 0x563fea888340 2024-09-12T03:16:33.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.701+0000 7f1369c4b740 1 -- 172.21.15.116:0/3728636882 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1358059070 con 0x563fea888340 2024-09-12T03:16:33.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.702+0000 7f1348ff9640 1 -- 172.21.15.116:0/3728636882 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f13500055f0 con 0x563fea888340 2024-09-12T03:16:33.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.702+0000 7f1369c4b740 10 client.19002 fetch_fsmap learned FSMap version 25 2024-09-12T03:16:33.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.702+0000 7f1369c4b740 1 -- 172.21.15.116:0/3728636882 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563fea9126f0 con 0x563fea888340 2024-09-12T03:16:33.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.702+0000 7f1348ff9640 1 -- 172.21.15.116:0/3728636882 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1358059070 con 0x563fea888340 2024-09-12T03:16:33.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.702+0000 7f1348ff9640 1 -- 172.21.15.116:0/3728636882 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1350027870 con 0x563fea888340 2024-09-12T03:16:33.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.702+0000 7f1369c4b740 10 client.19002 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:16:33.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.702+0000 7f1369c4b740 10 client.19002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:16:33.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.702+0000 7f1369c4b740 20 client.19002 populate_metadata read hostname 'smithi116' 2024-09-12T03:16:33.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.702+0000 7f1369c4b740 1 --2- 172.21.15.116:0/3728636882 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x563fea912ea0 0x563fea9152c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:33.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.702+0000 7f1369c4b740 4 client.19002 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:16:33.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.702+0000 7f1369c4b740 1 -- 172.21.15.116:0/3728636882 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563fea7f4d00 con 0x563fea912ea0 2024-09-12T03:16:33.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.702+0000 7f136560e640 1 --2- 172.21.15.116:0/3728636882 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x563fea912ea0 0x563fea9152c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:33.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.703+0000 7f1348ff9640 10 client.19002 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:33.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.703+0000 7f136560e640 1 --2- 172.21.15.116:0/3728636882 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x563fea912ea0 0x563fea9152c0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdd8cf7272d6f23c server_cookie=935cafd286c5ed0c in_seq=0 out_seq=0 2024-09-12T03:16:33.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.704+0000 7f1348ff9640 1 -- 172.21.15.116:0/3728636882 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563fea7f4d00 con 0x563fea912ea0 2024-09-12T03:16:33.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.704+0000 7f1348ff9640 10 client.19002 handle_command_reply: tid=0 2024-09-12T03:16:33.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.704+0000 7f1369c4b740 1 client.19002 shutdown 2024-09-12T03:16:33.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.704+0000 7f1369c4b740 1 -- 172.21.15.116:0/3728636882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563fea888340 msgr2=0x563fea89f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:33.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.704+0000 7f1369c4b740 1 --2- 172.21.15.116:0/3728636882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563fea888340 0x563fea89f740 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f1350005d80 tx=0x7f1350004560 comp rx=0 tx=0).stop 2024-09-12T03:16:33.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.704+0000 7f1369c4b740 1 -- 172.21.15.116:0/3728636882 shutdown_connections 2024-09-12T03:16:33.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.704+0000 7f1369c4b740 1 --2- 172.21.15.116:0/3728636882 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x563fea912ea0 0x563fea9152c0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.704+0000 7f1369c4b740 1 --2- 172.21.15.116:0/3728636882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fea897150 0x563fea89b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.704+0000 7f1369c4b740 1 --2- 172.21.15.116:0/3728636882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563fea888340 0x563fea89f740 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.704+0000 7f1348ff9640 0 client.19002 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:33.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.704+0000 7f1369c4b740 1 --2- 172.21.15.116:0/3728636882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563fea887a20 0x563fea89f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.704+0000 7f1369c4b740 1 -- 172.21.15.116:0/3728636882 >> 172.21.15.116:0/3728636882 conn(0x563fea8840f0 msgr2=0x563fea894070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:33.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.704+0000 7f1369c4b740 1 -- 172.21.15.116:0/3728636882 shutdown_connections 2024-09-12T03:16:33.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.704+0000 7f1369c4b740 1 -- 172.21.15.116:0/3728636882 wait complete. 2024-09-12T03:16:33.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.704+0000 7f1369c4b740 20 client.19002 trim_cache size 0 max 16384 2024-09-12T03:16:33.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.705+0000 7f1367872640 1 -- 172.21.15.116:0/4003342115 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f134c0a2b80 msgr2=0x7f134c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:33.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.705+0000 7f1367872640 1 --2- 172.21.15.116:0/4003342115 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f134c0a2b80 0x7f134c0a5040 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f135804c000 tx=0x7f135804d000 comp rx=0 tx=0).stop 2024-09-12T03:16:33.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.705+0000 7f1367872640 1 -- 172.21.15.116:0/4003342115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1360101de0 msgr2=0x7f13601983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:33.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.705+0000 7f1367872640 1 --2- 172.21.15.116:0/4003342115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1360101de0 0x7f13601983b0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f135c00d950 tx=0x7f135c00de10 comp rx=0 tx=0).stop 2024-09-12T03:16:33.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.705+0000 7f1367872640 1 -- 172.21.15.116:0/4003342115 shutdown_connections 2024-09-12T03:16:33.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.705+0000 7f1367872640 1 --2- 172.21.15.116:0/4003342115 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f134c0a2b80 0x7f134c0a5040 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.705+0000 7f1367872640 1 --2- 172.21.15.116:0/4003342115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1360106f10 0x7f136019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.705+0000 7f1367872640 1 --2- 172.21.15.116:0/4003342115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1360102790 0x7f13601988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.705+0000 7f1367872640 1 --2- 172.21.15.116:0/4003342115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1360101de0 0x7f13601983b0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:33.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.705+0000 7f1367872640 1 -- 172.21.15.116:0/4003342115 >> 172.21.15.116:0/4003342115 conn(0x7f13600fd8c0 msgr2=0x7f13600fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:33.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.705+0000 7f1367872640 1 -- 172.21.15.116:0/4003342115 shutdown_connections 2024-09-12T03:16:33.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:33.705+0000 7f1367872640 1 -- 172.21.15.116:0/4003342115 wait complete. 2024-09-12T03:16:33.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:33 smithi177 ceph-mon[35305]: pgmap v745: 97 pgs: 97 active+clean; 6.7 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.7 KiB/s wr, 7 op/s 2024-09-12T03:16:34.548 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:16:34.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:34 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:16:34.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:34 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:16:34.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:34 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:16:34.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:34 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:16:34.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:34 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:16:34.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:34 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:16:34.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:34 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:16:34.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:34 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:16:34.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:34 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:16:34.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:34 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:16:34.794 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:34.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:34 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:16:34.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:34 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:16:34.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:34 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:16:34.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:34 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:16:34.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:34 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:16:35.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.296+0000 7fe7aed9e640 1 -- 172.21.15.116:0/3495092329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7a8106f50 msgr2=0x7fe7a810eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.296+0000 7fe7aed9e640 1 --2- 172.21.15.116:0/3495092329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7a8106f50 0x7fe7a810eaf0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0009cd0 tx=0x7fe7a002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.296+0000 7fe7aed9e640 1 -- 172.21.15.116:0/3495092329 shutdown_connections 2024-09-12T03:16:35.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.296+0000 7fe7aed9e640 1 --2- 172.21.15.116:0/3495092329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7a8106f50 0x7fe7a810eaf0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.296+0000 7fe7aed9e640 1 --2- 172.21.15.116:0/3495092329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7a8100d00 0x7fe7a8106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.296+0000 7fe7aed9e640 1 --2- 172.21.15.116:0/3495092329 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7a8100350 0x7fe7a8100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.296+0000 7fe7aed9e640 1 -- 172.21.15.116:0/3495092329 >> 172.21.15.116:0/3495092329 conn(0x7fe7a8077210 msgr2=0x7fe7a80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:35.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.296+0000 7fe7aed9e640 1 -- 172.21.15.116:0/3495092329 shutdown_connections 2024-09-12T03:16:35.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.296+0000 7fe7aed9e640 1 -- 172.21.15.116:0/3495092329 wait complete. 2024-09-12T03:16:35.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.296+0000 7fe7aed9e640 1 Processor -- start 2024-09-12T03:16:35.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.296+0000 7fe7aed9e640 1 -- start start 2024-09-12T03:16:35.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.297+0000 7fe7aed9e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7a8100350 0x7fe7a8198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.297+0000 7fe7aed9e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7a8100d00 0x7fe7a8198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.297+0000 7fe7aed9e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7a8106f50 0x7fe7a819ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.297+0000 7fe7aed9e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a8111fa0 con 0x7fe7a8100350 2024-09-12T03:16:35.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.297+0000 7fe7aed9e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a8111e20 con 0x7fe7a8100d00 2024-09-12T03:16:35.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.297+0000 7fe7aed9e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a8112120 con 0x7fe7a8106f50 2024-09-12T03:16:35.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.297+0000 7fe7acb3a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7a8100350 0x7fe7a8198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.297+0000 7fe7acb3a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7a8100350 0x7fe7a8198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47924/0 (socket says 172.21.15.116:47924) 2024-09-12T03:16:35.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.297+0000 7fe7acb3a640 1 -- 172.21.15.116:0/190305950 learned_addr learned my addr 172.21.15.116:0/190305950 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:35.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.297+0000 7fe7a7fff640 1 --2- 172.21.15.116:0/190305950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7a8100d00 0x7fe7a8198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.297+0000 7fe7ad33b640 1 --2- 172.21.15.116:0/190305950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7a8106f50 0x7fe7a819ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.297+0000 7fe7acb3a640 1 -- 172.21.15.116:0/190305950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7a8106f50 msgr2=0x7fe7a819ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.297+0000 7fe7acb3a640 1 --2- 172.21.15.116:0/190305950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7a8106f50 0x7fe7a819ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.297+0000 7fe7acb3a640 1 -- 172.21.15.116:0/190305950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7a8100d00 msgr2=0x7fe7a8198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.297+0000 7fe7acb3a640 1 --2- 172.21.15.116:0/190305950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7a8100d00 0x7fe7a8198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.297+0000 7fe7acb3a640 1 -- 172.21.15.116:0/190305950 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7a819d530 con 0x7fe7a8100350 2024-09-12T03:16:35.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.297+0000 7fe7acb3a640 1 --2- 172.21.15.116:0/190305950 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7a8100350 0x7fe7a8198580 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00c890 tx=0x7fe79c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:35.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.297+0000 7fe7a5ffb640 1 -- 172.21.15.116:0/190305950 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c004220 con 0x7fe7a8100350 2024-09-12T03:16:35.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.298+0000 7fe7a5ffb640 1 -- 172.21.15.116:0/190305950 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe79c0043c0 con 0x7fe7a8100350 2024-09-12T03:16:35.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.298+0000 7fe7aed9e640 1 -- 172.21.15.116:0/190305950 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7a819d820 con 0x7fe7a8100350 2024-09-12T03:16:35.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.298+0000 7fe7a5ffb640 1 -- 172.21.15.116:0/190305950 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c0049e0 con 0x7fe7a8100350 2024-09-12T03:16:35.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.298+0000 7fe7aed9e640 1 -- 172.21.15.116:0/190305950 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7a81a5100 con 0x7fe7a8100350 2024-09-12T03:16:35.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.300+0000 7fe7a5ffb640 1 -- 172.21.15.116:0/190305950 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe79c0079b0 con 0x7fe7a8100350 2024-09-12T03:16:35.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.300+0000 7fe7b1177740 1 Processor -- start 2024-09-12T03:16:35.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.301+0000 7fe7b1177740 1 -- start start 2024-09-12T03:16:35.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.301+0000 7fe7a5ffb640 1 --2- 172.21.15.116:0/190305950 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe7980a2b80 0x7fe7980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.301+0000 7fe7b1177740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ed5ef44a0 0x557ed5ef4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.301+0000 7fe7b1177740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5ef4dc0 0x557ed5eff140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.301+0000 7fe7ad33b640 1 --2- 172.21.15.116:0/190305950 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe7980a2b80 0x7fe7980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.301+0000 7fe7b1177740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ed5eff680 0x557ed5f01a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.301+0000 7fe7b1177740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557ed5e5a2e0 con 0x557ed5ef44a0 2024-09-12T03:16:35.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.301+0000 7fe7b1177740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557ed5e01ce0 con 0x557ed5eff680 2024-09-12T03:16:35.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.301+0000 7fe7b1177740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557ed5e562a0 con 0x557ed5ef4dc0 2024-09-12T03:16:35.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.301+0000 7fe7a5ffb640 1 -- 172.21.15.116:0/190305950 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe79c0cefc0 con 0x7fe7a8100350 2024-09-12T03:16:35.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.301+0000 7fe7acb3a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ed5ef44a0 0x557ed5ef4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.301+0000 7fe7acb3a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ed5ef44a0 0x557ed5ef4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47940/0 (socket says 172.21.15.116:47940) 2024-09-12T03:16:35.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.301+0000 7fe7acb3a640 1 -- 172.21.15.116:0/3777374003 learned_addr learned my addr 172.21.15.116:0/3777374003 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:35.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.301+0000 7fe7a7fff640 1 --2- 172.21.15.116:0/3777374003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5ef4dc0 0x557ed5eff140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.301+0000 7fe7ad33b640 1 --2- 172.21.15.116:0/3777374003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ed5eff680 0x557ed5f01a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.301+0000 7fe7acb3a640 1 -- 172.21.15.116:0/3777374003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5ef4dc0 msgr2=0x557ed5eff140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:35.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.301+0000 7fe7acb3a640 1 --2- 172.21.15.116:0/3777374003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5ef4dc0 0x557ed5eff140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.301+0000 7fe7acb3a640 1 -- 172.21.15.116:0/3777374003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ed5eff680 msgr2=0x557ed5f01a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:35.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.301+0000 7fe7acb3a640 1 --2- 172.21.15.116:0/3777374003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ed5eff680 0x557ed5f01a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.301+0000 7fe7acb3a640 1 -- 172.21.15.116:0/3777374003 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ed5de6440 con 0x557ed5ef44a0 2024-09-12T03:16:35.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.301+0000 7fe7ad33b640 1 --2- 172.21.15.116:0/3777374003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ed5eff680 0x557ed5f01a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:35.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.301+0000 7fe7ad33b640 1 --2- 172.21.15.116:0/190305950 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe7980a2b80 0x7fe7980a5040 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0004ab0 tx=0x7fe7a0009800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:35.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.301+0000 7fe7acb3a640 1 --2- 172.21.15.116:0/3777374003 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ed5ef44a0 0x557ed5ef4880 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fe79c09b600 tx=0x7fe79c09b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da612affd91ade7e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:35.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.302+0000 7fe792ffd640 1 -- 172.21.15.116:0/3777374003 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c01d040 con 0x557ed5ef44a0 2024-09-12T03:16:35.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.302+0000 7fe792ffd640 1 -- 172.21.15.116:0/3777374003 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe79c0a44c0 con 0x557ed5ef44a0 2024-09-12T03:16:35.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.302+0000 7fe792ffd640 1 -- 172.21.15.116:0/3777374003 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c0a4270 con 0x557ed5ef44a0 2024-09-12T03:16:35.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.302+0000 7fe7b1177740 1 -- 172.21.15.116:0/3777374003 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ed5ef44a0 msgr2=0x557ed5ef4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.302+0000 7fe7b1177740 1 --2- 172.21.15.116:0/3777374003 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ed5ef44a0 0x557ed5ef4880 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fe79c09b600 tx=0x7fe79c09b390 comp rx=0 tx=0).stop 2024-09-12T03:16:35.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.302+0000 7fe7b1177740 1 -- 172.21.15.116:0/3777374003 shutdown_connections 2024-09-12T03:16:35.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.302+0000 7fe7b1177740 1 --2- 172.21.15.116:0/3777374003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ed5eff680 0x557ed5f01a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.302+0000 7fe7b1177740 1 --2- 172.21.15.116:0/3777374003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5ef4dc0 0x557ed5eff140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.302+0000 7fe7b1177740 1 --2- 172.21.15.116:0/3777374003 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ed5ef44a0 0x557ed5ef4880 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.302+0000 7fe7b1177740 1 -- 172.21.15.116:0/3777374003 >> 172.21.15.116:0/3777374003 conn(0x557ed5ef0a30 msgr2=0x557ed5ef0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:35.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.302+0000 7fe7b1177740 1 -- 172.21.15.116:0/3777374003 shutdown_connections 2024-09-12T03:16:35.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.302+0000 7fe7b1177740 1 -- 172.21.15.116:0/3777374003 wait complete. 2024-09-12T03:16:35.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.303+0000 7fe7b1177740 1 -- start start 2024-09-12T03:16:35.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.303+0000 7fe7b1177740 1 Processor -- start 2024-09-12T03:16:35.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.303+0000 7fe7b1177740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ed5ef44a0 0x557ed5efefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.303+0000 7fe7b1177740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5ef4dc0 0x557ed5efc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.303+0000 7fe7b1177740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ed5eff680 0x557ed5efc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.303+0000 7fe7b1177740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557ed5f02250 con 0x557ed5ef44a0 2024-09-12T03:16:35.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.303+0000 7fe7b1177740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557ed5e15050 con 0x557ed5eff680 2024-09-12T03:16:35.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.303+0000 7fe7b1177740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557ed5f023d0 con 0x557ed5ef4dc0 2024-09-12T03:16:35.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.303+0000 7fe7b1177740 10 client.0 _ll_register_callbacks cb 0x557ed5e55160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:35.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.303+0000 7fe7acb3a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ed5ef44a0 0x557ed5efefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.303+0000 7fe7acb3a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ed5ef44a0 0x557ed5efefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47946/0 (socket says 172.21.15.116:47946) 2024-09-12T03:16:35.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.303+0000 7fe7acb3a640 1 -- 172.21.15.116:0/68513719 learned_addr learned my addr 172.21.15.116:0/68513719 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:35.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.303+0000 7fe7a7fff640 1 --2- 172.21.15.116:0/68513719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5ef4dc0 0x557ed5efc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.303+0000 7fe7ad33b640 1 --2- 172.21.15.116:0/68513719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ed5eff680 0x557ed5efc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.304+0000 7fe7acb3a640 1 -- 172.21.15.116:0/68513719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5ef4dc0 msgr2=0x557ed5efc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:35.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.304+0000 7fe7acb3a640 1 --2- 172.21.15.116:0/68513719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5ef4dc0 0x557ed5efc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.304+0000 7fe7acb3a640 1 -- 172.21.15.116:0/68513719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ed5eff680 msgr2=0x557ed5efc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:35.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.304+0000 7fe7acb3a640 1 --2- 172.21.15.116:0/68513719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ed5eff680 0x557ed5efc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.304+0000 7fe7acb3a640 1 -- 172.21.15.116:0/68513719 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ed5bcb3c0 con 0x557ed5ef44a0 2024-09-12T03:16:35.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.304+0000 7fe7acb3a640 1 --2- 172.21.15.116:0/68513719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ed5ef44a0 0x557ed5efefd0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fe79c0058c0 tx=0x7fe79c099e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a4e7bada4317aa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:35.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.304+0000 7fe787fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:16:35.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.304+0000 7fe787fff640 1 -- 172.21.15.116:0/68513719 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c0b3020 con 0x557ed5ef44a0 2024-09-12T03:16:35.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.304+0000 7fe787fff640 1 -- 172.21.15.116:0/68513719 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe79c0862c0 con 0x557ed5ef44a0 2024-09-12T03:16:35.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.304+0000 7fe7b1177740 1 -- 172.21.15.116:0/68513719 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe79c0a44c0 con 0x557ed5ef44a0 2024-09-12T03:16:35.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.304+0000 7fe787fff640 1 -- 172.21.15.116:0/68513719 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe79c01d070 con 0x557ed5ef44a0 2024-09-12T03:16:35.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.304+0000 7fe7b1177740 10 client.19014 fetch_fsmap learned FSMap version 25 2024-09-12T03:16:35.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.304+0000 7fe7b1177740 1 -- 172.21.15.116:0/68513719 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557ed5f0db90 con 0x557ed5ef44a0 2024-09-12T03:16:35.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.304+0000 7fe787fff640 1 -- 172.21.15.116:0/68513719 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe79c0a44c0 con 0x557ed5ef44a0 2024-09-12T03:16:35.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.304+0000 7fe787fff640 1 -- 172.21.15.116:0/68513719 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe79c09ad50 con 0x557ed5ef44a0 2024-09-12T03:16:35.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.304+0000 7fe7b1177740 10 client.19014 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:16:35.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.304+0000 7fe7b1177740 10 client.19014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:16:35.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.304+0000 7fe7b1177740 20 client.19014 populate_metadata read hostname 'smithi116' 2024-09-12T03:16:35.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.304+0000 7fe7b1177740 1 --2- 172.21.15.116:0/68513719 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557ed5f0eee0 0x557ed5f112d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.304+0000 7fe7b1177740 4 client.19014 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:16:35.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.304+0000 7fe7b1177740 1 -- 172.21.15.116:0/68513719 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ed5e67850 con 0x557ed5f0eee0 2024-09-12T03:16:35.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.305+0000 7fe7a7fff640 1 --2- 172.21.15.116:0/68513719 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557ed5f0eee0 0x557ed5f112d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.305+0000 7fe787fff640 10 client.19014 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:35.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.305+0000 7fe7a7fff640 1 --2- 172.21.15.116:0/68513719 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557ed5f0eee0 0x557ed5f112d0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90729c47671c22e1 server_cookie=d107cdc26a8f68a0 in_seq=0 out_seq=0 2024-09-12T03:16:35.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.306+0000 7fe787fff640 1 -- 172.21.15.116:0/68513719 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557ed5e67850 con 0x557ed5f0eee0 2024-09-12T03:16:35.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.306+0000 7fe787fff640 10 client.19014 handle_command_reply: tid=0 2024-09-12T03:16:35.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.306+0000 7fe7b1177740 1 client.19014 shutdown 2024-09-12T03:16:35.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.306+0000 7fe7b1177740 1 -- 172.21.15.116:0/68513719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ed5ef44a0 msgr2=0x557ed5efefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.306+0000 7fe7b1177740 1 --2- 172.21.15.116:0/68513719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ed5ef44a0 0x557ed5efefd0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fe79c0058c0 tx=0x7fe79c099e10 comp rx=0 tx=0).stop 2024-09-12T03:16:35.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.306+0000 7fe7b1177740 1 -- 172.21.15.116:0/68513719 shutdown_connections 2024-09-12T03:16:35.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.306+0000 7fe7b1177740 1 --2- 172.21.15.116:0/68513719 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557ed5f0eee0 0x557ed5f112d0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.306+0000 7fe7b1177740 1 --2- 172.21.15.116:0/68513719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ed5eff680 0x557ed5efc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.306+0000 7fe7b1177740 1 --2- 172.21.15.116:0/68513719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5ef4dc0 0x557ed5efc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.306+0000 7fe7b1177740 1 --2- 172.21.15.116:0/68513719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ed5ef44a0 0x557ed5efefd0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.306+0000 7fe787fff640 0 client.19014 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:35.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.306+0000 7fe7b1177740 1 -- 172.21.15.116:0/68513719 >> 172.21.15.116:0/68513719 conn(0x557ed5ef0a30 msgr2=0x557ed5ef6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:35.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.306+0000 7fe7b1177740 1 -- 172.21.15.116:0/68513719 shutdown_connections 2024-09-12T03:16:35.317 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.306+0000 7fe7b1177740 1 -- 172.21.15.116:0/68513719 wait complete. 2024-09-12T03:16:35.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.306+0000 7fe7b1177740 20 client.19014 trim_cache size 0 max 16384 2024-09-12T03:16:35.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.324+0000 7fe7b1177740 1 Processor -- start 2024-09-12T03:16:35.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.324+0000 7fe7b1177740 1 -- start start 2024-09-12T03:16:35.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.324+0000 7fe7b1177740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5ef44a0 0x557ed5ef0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.324+0000 7fe7b1177740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ed5ef4dc0 0x557ed5f14300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.324+0000 7fe7b1177740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ed5eff680 0x557ed5f14840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.324+0000 7fe7b1177740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a81afee0 con 0x557ed5ef4dc0 2024-09-12T03:16:35.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.324+0000 7fe7b1177740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a81ad7b0 con 0x557ed5eff680 2024-09-12T03:16:35.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.324+0000 7fe7b1177740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a81ab5e0 con 0x557ed5ef44a0 2024-09-12T03:16:35.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.324+0000 7fe7a7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ed5ef4dc0 0x557ed5f14300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.324+0000 7fe7a7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ed5ef4dc0 0x557ed5f14300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47956/0 (socket says 172.21.15.116:47956) 2024-09-12T03:16:35.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.324+0000 7fe7a7fff640 1 -- 172.21.15.116:0/4218022201 learned_addr learned my addr 172.21.15.116:0/4218022201 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:35.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.324+0000 7fe7ad33b640 1 --2- 172.21.15.116:0/4218022201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ed5eff680 0x557ed5f14840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.324+0000 7fe7a7fff640 1 -- 172.21.15.116:0/4218022201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5ef44a0 msgr2=0x557ed5ef0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:35.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.324+0000 7fe7a7fff640 1 --2- 172.21.15.116:0/4218022201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5ef44a0 0x557ed5ef0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.324+0000 7fe7a7fff640 1 -- 172.21.15.116:0/4218022201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ed5eff680 msgr2=0x557ed5f14840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:35.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.324+0000 7fe7a7fff640 1 --2- 172.21.15.116:0/4218022201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ed5eff680 0x557ed5f14840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.324+0000 7fe7a7fff640 1 -- 172.21.15.116:0/4218022201 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ed5efa7d0 con 0x557ed5ef4dc0 2024-09-12T03:16:35.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.324+0000 7fe7a7fff640 1 --2- 172.21.15.116:0/4218022201 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ed5ef4dc0 0x557ed5f14300 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fe794005fd0 tx=0x7fe79400bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f004c705533d206d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:35.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.324+0000 7fe7927fc640 1 -- 172.21.15.116:0/4218022201 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794015070 con 0x557ed5ef4dc0 2024-09-12T03:16:35.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.325+0000 7fe7927fc640 1 -- 172.21.15.116:0/4218022201 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe794003e20 con 0x557ed5ef4dc0 2024-09-12T03:16:35.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.325+0000 7fe7927fc640 1 -- 172.21.15.116:0/4218022201 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79401d3b0 con 0x557ed5ef4dc0 2024-09-12T03:16:35.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.325+0000 7fe7b1177740 1 -- 172.21.15.116:0/4218022201 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ed5ef4dc0 msgr2=0x557ed5f14300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.325+0000 7fe7b1177740 1 --2- 172.21.15.116:0/4218022201 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ed5ef4dc0 0x557ed5f14300 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fe794005fd0 tx=0x7fe79400bdd0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.325+0000 7fe7b1177740 1 -- 172.21.15.116:0/4218022201 shutdown_connections 2024-09-12T03:16:35.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.325+0000 7fe7b1177740 1 --2- 172.21.15.116:0/4218022201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ed5eff680 0x557ed5f14840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.325+0000 7fe7b1177740 1 --2- 172.21.15.116:0/4218022201 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ed5ef4dc0 0x557ed5f14300 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.325+0000 7fe7b1177740 1 --2- 172.21.15.116:0/4218022201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5ef44a0 0x557ed5ef0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.325+0000 7fe7b1177740 1 -- 172.21.15.116:0/4218022201 >> 172.21.15.116:0/4218022201 conn(0x557ed5ef0a30 msgr2=0x557ed5efe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:35.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.325+0000 7fe7b1177740 1 -- 172.21.15.116:0/4218022201 shutdown_connections 2024-09-12T03:16:35.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.325+0000 7fe7b1177740 1 -- 172.21.15.116:0/4218022201 wait complete. 2024-09-12T03:16:35.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.326+0000 7fe7b1177740 1 -- start start 2024-09-12T03:16:35.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.326+0000 7fe7b1177740 1 Processor -- start 2024-09-12T03:16:35.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.326+0000 7fe7b1177740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ed5ef44a0 0x557ed5f71bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.326+0000 7fe7b1177740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ed5ef4dc0 0x557ed5f72100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.326+0000 7fe7b1177740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5eff680 0x557ed5f72640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.332 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.326+0000 7fe7b1177740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a81abc70 con 0x557ed5ef4dc0 2024-09-12T03:16:35.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.326+0000 7fe7b1177740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a81ad0e0 con 0x557ed5ef44a0 2024-09-12T03:16:35.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.326+0000 7fe7b1177740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a81ac340 con 0x557ed5eff680 2024-09-12T03:16:35.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.326+0000 7fe7b1177740 10 client.0 _ll_register_callbacks cb 0x7fe79c09b530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:35.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.326+0000 7fe7a7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ed5ef4dc0 0x557ed5f72100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.326+0000 7fe7a7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ed5ef4dc0 0x557ed5f72100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47970/0 (socket says 172.21.15.116:47970) 2024-09-12T03:16:35.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.326+0000 7fe7a7fff640 1 -- 172.21.15.116:0/2030643882 learned_addr learned my addr 172.21.15.116:0/2030643882 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:35.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.326+0000 7fe7acb3a640 1 --2- 172.21.15.116:0/2030643882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ed5ef44a0 0x557ed5f71bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.326+0000 7fe7ad33b640 1 --2- 172.21.15.116:0/2030643882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5eff680 0x557ed5f72640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.326+0000 7fe7a7fff640 1 -- 172.21.15.116:0/2030643882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5eff680 msgr2=0x557ed5f72640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:35.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.326+0000 7fe7a7fff640 1 --2- 172.21.15.116:0/2030643882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5eff680 0x557ed5f72640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.326+0000 7fe7a7fff640 1 -- 172.21.15.116:0/2030643882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ed5ef44a0 msgr2=0x557ed5f71bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:35.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.326+0000 7fe7a7fff640 1 --2- 172.21.15.116:0/2030643882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ed5ef44a0 0x557ed5f71bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.327+0000 7fe7a7fff640 1 -- 172.21.15.116:0/2030643882 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ed5f7ce40 con 0x557ed5ef4dc0 2024-09-12T03:16:35.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.327+0000 7fe7a7fff640 1 --2- 172.21.15.116:0/2030643882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ed5ef4dc0 0x557ed5f72100 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fe794000c00 tx=0x7fe794004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18490ea267c64eb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:35.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.327+0000 7fe787fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:16:35.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.327+0000 7fe787fff640 1 -- 172.21.15.116:0/2030643882 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79402c020 con 0x557ed5ef4dc0 2024-09-12T03:16:35.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.327+0000 7fe787fff640 1 -- 172.21.15.116:0/2030643882 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe7940048c0 con 0x557ed5ef4dc0 2024-09-12T03:16:35.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.327+0000 7fe7b1177740 1 -- 172.21.15.116:0/2030643882 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe794003e20 con 0x557ed5ef4dc0 2024-09-12T03:16:35.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.327+0000 7fe787fff640 1 -- 172.21.15.116:0/2030643882 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe794015040 con 0x557ed5ef4dc0 2024-09-12T03:16:35.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.327+0000 7fe7b1177740 10 client.19020 fetch_fsmap learned FSMap version 25 2024-09-12T03:16:35.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.327+0000 7fe7b1177740 1 -- 172.21.15.116:0/2030643882 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557ed5f7e130 con 0x557ed5ef4dc0 2024-09-12T03:16:35.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.327+0000 7fe787fff640 1 -- 172.21.15.116:0/2030643882 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe794003e20 con 0x557ed5ef4dc0 2024-09-12T03:16:35.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.327+0000 7fe787fff640 1 -- 172.21.15.116:0/2030643882 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe79401d8d0 con 0x557ed5ef4dc0 2024-09-12T03:16:35.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.327+0000 7fe7b1177740 10 client.19020 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:16:35.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.327+0000 7fe7b1177740 10 client.19020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:16:35.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.327+0000 7fe7b1177740 20 client.19020 populate_metadata read hostname 'smithi116' 2024-09-12T03:16:35.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.327+0000 7fe7b1177740 1 --2- 172.21.15.116:0/2030643882 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557ed5f7e7a0 0x557ed5f80bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.327+0000 7fe7b1177740 4 client.19020 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:16:35.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.327+0000 7fe7b1177740 1 -- 172.21.15.116:0/2030643882 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557ed5e61d60 con 0x557ed5f7e7a0 2024-09-12T03:16:35.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.328+0000 7fe7acb3a640 1 --2- 172.21.15.116:0/2030643882 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557ed5f7e7a0 0x557ed5f80bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.328+0000 7fe787fff640 10 client.19020 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:35.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.328+0000 7fe7acb3a640 1 --2- 172.21.15.116:0/2030643882 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557ed5f7e7a0 0x557ed5f80bc0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb3e03db3be9f1bd server_cookie=b56dc5da1dfc864e in_seq=0 out_seq=0 2024-09-12T03:16:35.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.328+0000 7fe787fff640 1 -- 172.21.15.116:0/2030643882 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557ed5e61d60 con 0x557ed5f7e7a0 2024-09-12T03:16:35.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.328+0000 7fe787fff640 10 client.19020 handle_command_reply: tid=0 2024-09-12T03:16:35.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.328+0000 7fe7b1177740 1 client.19020 shutdown 2024-09-12T03:16:35.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.328+0000 7fe7b1177740 1 -- 172.21.15.116:0/2030643882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ed5ef4dc0 msgr2=0x557ed5f72100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.328+0000 7fe7b1177740 1 --2- 172.21.15.116:0/2030643882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ed5ef4dc0 0x557ed5f72100 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fe794000c00 tx=0x7fe794004ce0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.328+0000 7fe7b1177740 1 -- 172.21.15.116:0/2030643882 shutdown_connections 2024-09-12T03:16:35.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.328+0000 7fe7b1177740 1 --2- 172.21.15.116:0/2030643882 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x557ed5f7e7a0 0x557ed5f80bc0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.328+0000 7fe7b1177740 1 --2- 172.21.15.116:0/2030643882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ed5eff680 0x557ed5f72640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.328+0000 7fe7b1177740 1 --2- 172.21.15.116:0/2030643882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ed5ef4dc0 0x557ed5f72100 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.328+0000 7fe7b1177740 1 --2- 172.21.15.116:0/2030643882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557ed5ef44a0 0x557ed5f71bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.328+0000 7fe7b1177740 1 -- 172.21.15.116:0/2030643882 >> 172.21.15.116:0/2030643882 conn(0x557ed5ef0a30 msgr2=0x557ed5efdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:35.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.328+0000 7fe787fff640 0 client.19020 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:35.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.329+0000 7fe7b1177740 1 -- 172.21.15.116:0/2030643882 shutdown_connections 2024-09-12T03:16:35.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.329+0000 7fe7b1177740 1 -- 172.21.15.116:0/2030643882 wait complete. 2024-09-12T03:16:35.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.329+0000 7fe7b1177740 20 client.19020 trim_cache size 0 max 16384 2024-09-12T03:16:35.336 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:16:35.336 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:16:35.336 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:16:35.336 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:16:35.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.329+0000 7fe7aed9e640 1 -- 172.21.15.116:0/190305950 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe7980a2b80 msgr2=0x7fe7980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.329+0000 7fe7aed9e640 1 --2- 172.21.15.116:0/190305950 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe7980a2b80 0x7fe7980a5040 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0004ab0 tx=0x7fe7a0009800 comp rx=0 tx=0).stop 2024-09-12T03:16:35.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.329+0000 7fe7aed9e640 1 -- 172.21.15.116:0/190305950 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7a8100350 msgr2=0x7fe7a8198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.329+0000 7fe7aed9e640 1 --2- 172.21.15.116:0/190305950 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7a8100350 0x7fe7a8198580 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00c890 tx=0x7fe79c00cd50 comp rx=0 tx=0).stop 2024-09-12T03:16:35.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.329+0000 7fe7aed9e640 1 -- 172.21.15.116:0/190305950 shutdown_connections 2024-09-12T03:16:35.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.329+0000 7fe7aed9e640 1 --2- 172.21.15.116:0/190305950 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe7980a2b80 0x7fe7980a5040 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.329+0000 7fe7aed9e640 1 --2- 172.21.15.116:0/190305950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7a8106f50 0x7fe7a819ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.329+0000 7fe7aed9e640 1 --2- 172.21.15.116:0/190305950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7a8100d00 0x7fe7a8198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.329+0000 7fe7aed9e640 1 --2- 172.21.15.116:0/190305950 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7a8100350 0x7fe7a8198580 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.329+0000 7fe7aed9e640 1 -- 172.21.15.116:0/190305950 >> 172.21.15.116:0/190305950 conn(0x7fe7a8077210 msgr2=0x7fe7a80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:35.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.329+0000 7fe7aed9e640 1 -- 172.21.15.116:0/190305950 shutdown_connections 2024-09-12T03:16:35.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:35.329+0000 7fe7aed9e640 1 -- 172.21.15.116:0/190305950 wait complete. 2024-09-12T03:16:35.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:35 smithi116 ceph-mon[31410]: pgmap v746: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2024-09-12T03:16:35.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:35 smithi183 ceph-mon[35535]: pgmap v746: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2024-09-12T03:16:35.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:35 smithi177 ceph-mon[35305]: pgmap v746: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2024-09-12T03:16:36.082 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:16:36.082 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:16:36.328 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:36.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.803+0000 7fa4e4aa1640 1 -- 172.21.15.116:0/1375270958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4e0100a80 msgr2=0x7fa4e0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:36.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.803+0000 7fa4e4aa1640 1 --2- 172.21.15.116:0/1375270958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4e0100a80 0x7fa4e0100ee0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8009930 tx=0x7fa4c802a420 comp rx=0 tx=0).stop 2024-09-12T03:16:36.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.803+0000 7fa4e4aa1640 1 -- 172.21.15.116:0/1375270958 shutdown_connections 2024-09-12T03:16:36.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.803+0000 7fa4e4aa1640 1 --2- 172.21.15.116:0/1375270958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4e0101420 0x7fa4e010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:36.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.803+0000 7fa4e4aa1640 1 --2- 172.21.15.116:0/1375270958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4e0100a80 0x7fa4e0100ee0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:36.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.803+0000 7fa4e4aa1640 1 --2- 172.21.15.116:0/1375270958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4e0106a80 0x7fa4e0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:36.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.803+0000 7fa4e4aa1640 1 -- 172.21.15.116:0/1375270958 >> 172.21.15.116:0/1375270958 conn(0x7fa4e00fc7a0 msgr2=0x7fa4e00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:36.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.803+0000 7fa4e4aa1640 1 -- 172.21.15.116:0/1375270958 shutdown_connections 2024-09-12T03:16:36.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.803+0000 7fa4e4aa1640 1 -- 172.21.15.116:0/1375270958 wait complete. 2024-09-12T03:16:36.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.804+0000 7fa4e4aa1640 1 Processor -- start 2024-09-12T03:16:36.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.804+0000 7fa4e4aa1640 1 -- start start 2024-09-12T03:16:36.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.804+0000 7fa4e4aa1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4e0100a80 0x7fa4e01984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:36.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.804+0000 7fa4e4aa1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4e0101420 0x7fa4e0198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:36.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.804+0000 7fa4e4aa1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4e0106a80 0x7fa4e019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:36.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.804+0000 7fa4e4aa1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e0111fa0 con 0x7fa4e0100a80 2024-09-12T03:16:36.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.804+0000 7fa4e4aa1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e0111e20 con 0x7fa4e0106a80 2024-09-12T03:16:36.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.804+0000 7fa4e4aa1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e0112120 con 0x7fa4e0101420 2024-09-12T03:16:36.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.804+0000 7fa4de59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4e0100a80 0x7fa4e01984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:36.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.804+0000 7fa4de59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4e0100a80 0x7fa4e01984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47978/0 (socket says 172.21.15.116:47978) 2024-09-12T03:16:36.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.804+0000 7fa4de59c640 1 -- 172.21.15.116:0/2176267372 learned_addr learned my addr 172.21.15.116:0/2176267372 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:36.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.804+0000 7fa4ddd9b640 1 --2- 172.21.15.116:0/2176267372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4e0101420 0x7fa4e0198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:36.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.805+0000 7fa4de59c640 1 -- 172.21.15.116:0/2176267372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4e0101420 msgr2=0x7fa4e0198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:36.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.805+0000 7fa4de59c640 1 --2- 172.21.15.116:0/2176267372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4e0101420 0x7fa4e0198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:36.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.805+0000 7fa4de59c640 1 -- 172.21.15.116:0/2176267372 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4e0106a80 msgr2=0x7fa4e019cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:36.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.805+0000 7fa4de59c640 1 --2- 172.21.15.116:0/2176267372 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4e0106a80 0x7fa4e019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:36.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.805+0000 7fa4de59c640 1 -- 172.21.15.116:0/2176267372 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e019d470 con 0x7fa4e0100a80 2024-09-12T03:16:36.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.805+0000 7fa4de59c640 1 --2- 172.21.15.116:0/2176267372 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4e0100a80 0x7fa4e01984c0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fa4d000c970 tx=0x7fa4d000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:36.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.805+0000 7fa4d77fe640 1 -- 172.21.15.116:0/2176267372 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0007bb0 con 0x7fa4e0100a80 2024-09-12T03:16:36.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.805+0000 7fa4d77fe640 1 -- 172.21.15.116:0/2176267372 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4d0007d50 con 0x7fa4e0100a80 2024-09-12T03:16:36.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.805+0000 7fa4e4aa1640 1 -- 172.21.15.116:0/2176267372 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4e019d760 con 0x7fa4e0100a80 2024-09-12T03:16:36.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.805+0000 7fa4d77fe640 1 -- 172.21.15.116:0/2176267372 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d000f450 con 0x7fa4e0100a80 2024-09-12T03:16:36.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.805+0000 7fa4e4aa1640 1 -- 172.21.15.116:0/2176267372 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4e01a5040 con 0x7fa4e0100a80 2024-09-12T03:16:36.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.806+0000 7fa4e4aa1640 1 -- 172.21.15.116:0/2176267372 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4e0100ee0 con 0x7fa4e0100a80 2024-09-12T03:16:36.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.807+0000 7fa4d77fe640 1 -- 172.21.15.116:0/2176267372 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa4d0004fe0 con 0x7fa4e0100a80 2024-09-12T03:16:36.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.808+0000 7fa4d77fe640 1 --2- 172.21.15.116:0/2176267372 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa4cc0a2b80 0x7fa4cc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:36.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.808+0000 7fa4d77fe640 1 -- 172.21.15.116:0/2176267372 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa4d00cf250 con 0x7fa4e0100a80 2024-09-12T03:16:36.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.808+0000 7fa4ddd9b640 1 --2- 172.21.15.116:0/2176267372 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa4cc0a2b80 0x7fa4cc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:36.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.809+0000 7fa4ddd9b640 1 --2- 172.21.15.116:0/2176267372 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa4cc0a2b80 0x7fa4cc0a5040 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8000c00 tx=0x7fa4c8031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:36.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:36.811+0000 7fa4d77fe640 1 -- 172.21.15.116:0/2176267372 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa4d0014030 con 0x7fa4e0100a80 2024-09-12T03:16:37.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:37.011+0000 7fa4e4aa1640 1 -- 172.21.15.116:0/2176267372 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4e0199820 con 0x7fa4e0100a80 2024-09-12T03:16:37.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:37.011+0000 7fa4d77fe640 1 -- 172.21.15.116:0/2176267372 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7fa4d009c1e0 con 0x7fa4e0100a80 2024-09-12T03:16:37.013 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:16:37.013 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:16:37.013 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:16:37.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:37.013+0000 7fa4e4aa1640 1 -- 172.21.15.116:0/2176267372 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa4cc0a2b80 msgr2=0x7fa4cc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:37.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:37.013+0000 7fa4e4aa1640 1 --2- 172.21.15.116:0/2176267372 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa4cc0a2b80 0x7fa4cc0a5040 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8000c00 tx=0x7fa4c8031490 comp rx=0 tx=0).stop 2024-09-12T03:16:37.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:37.013+0000 7fa4e4aa1640 1 -- 172.21.15.116:0/2176267372 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4e0100a80 msgr2=0x7fa4e01984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:37.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:37.013+0000 7fa4e4aa1640 1 --2- 172.21.15.116:0/2176267372 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4e0100a80 0x7fa4e01984c0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fa4d000c970 tx=0x7fa4d000ce30 comp rx=0 tx=0).stop 2024-09-12T03:16:37.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:37.013+0000 7fa4e4aa1640 1 -- 172.21.15.116:0/2176267372 shutdown_connections 2024-09-12T03:16:37.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:37.013+0000 7fa4e4aa1640 1 --2- 172.21.15.116:0/2176267372 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa4cc0a2b80 0x7fa4cc0a5040 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:37.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:37.013+0000 7fa4e4aa1640 1 --2- 172.21.15.116:0/2176267372 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4e0106a80 0x7fa4e019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:37.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:37.013+0000 7fa4e4aa1640 1 --2- 172.21.15.116:0/2176267372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4e0101420 0x7fa4e0198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:37.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:37.013+0000 7fa4e4aa1640 1 --2- 172.21.15.116:0/2176267372 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4e0100a80 0x7fa4e01984c0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:37.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:37.014+0000 7fa4e4aa1640 1 -- 172.21.15.116:0/2176267372 >> 172.21.15.116:0/2176267372 conn(0x7fa4e00fc7a0 msgr2=0x7fa4e00fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:37.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:37.014+0000 7fa4e4aa1640 1 -- 172.21.15.116:0/2176267372 shutdown_connections 2024-09-12T03:16:37.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:37.014+0000 7fa4e4aa1640 1 -- 172.21.15.116:0/2176267372 wait complete. 2024-09-12T03:16:37.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:37 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2176267372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:37.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:37 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2176267372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:37.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:37 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2176267372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:37.899 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:16:38.145 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:38.266 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:38 smithi116 ceph-mon[31410]: pgmap v747: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:16:38.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:38 smithi183 ceph-mon[35535]: pgmap v747: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:16:38.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:38 smithi177 ceph-mon[35305]: pgmap v747: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:16:38.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.699+0000 7f9a5d07e640 1 -- 172.21.15.116:0/2523095123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a5810c170 msgr2=0x7f9a5810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:38.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.699+0000 7f9a5d07e640 1 --2- 172.21.15.116:0/2523095123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a5810c170 0x7f9a5810e610 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009ca0 tx=0x7f9a5002a950 comp rx=0 tx=0).stop 2024-09-12T03:16:38.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.699+0000 7f9a5d07e640 1 -- 172.21.15.116:0/2523095123 shutdown_connections 2024-09-12T03:16:38.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.699+0000 7f9a5d07e640 1 --2- 172.21.15.116:0/2523095123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a5810c170 0x7f9a5810e610 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.699+0000 7f9a5d07e640 1 --2- 172.21.15.116:0/2523095123 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a58103660 0x7f9a5810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.699+0000 7f9a5d07e640 1 --2- 172.21.15.116:0/2523095123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a58102d40 0x7f9a58103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.699+0000 7f9a5d07e640 1 -- 172.21.15.116:0/2523095123 >> 172.21.15.116:0/2523095123 conn(0x7f9a580fc7a0 msgr2=0x7f9a580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:38.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.699+0000 7f9a5d07e640 1 -- 172.21.15.116:0/2523095123 shutdown_connections 2024-09-12T03:16:38.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.699+0000 7f9a5d07e640 1 -- 172.21.15.116:0/2523095123 wait complete. 2024-09-12T03:16:38.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.700+0000 7f9a5d07e640 1 Processor -- start 2024-09-12T03:16:38.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.700+0000 7f9a5d07e640 1 -- start start 2024-09-12T03:16:38.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.700+0000 7f9a5d07e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a58102d40 0x7f9a58198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:38.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.700+0000 7f9a5d07e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a58103660 0x7f9a581988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:38.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.700+0000 7f9a5d07e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a5810c170 0x7f9a5819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:38.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.700+0000 7f9a5d07e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9a58111f70 con 0x7f9a5810c170 2024-09-12T03:16:38.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.700+0000 7f9a5d07e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9a58111df0 con 0x7f9a58102d40 2024-09-12T03:16:38.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.700+0000 7f9a5d07e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581120f0 con 0x7f9a58103660 2024-09-12T03:16:38.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.700+0000 7f9a5759e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a5810c170 0x7f9a5819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:38.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.700+0000 7f9a5759e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a5810c170 0x7f9a5819cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48008/0 (socket says 172.21.15.116:48008) 2024-09-12T03:16:38.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.700+0000 7f9a5759e640 1 -- 172.21.15.116:0/455747307 learned_addr learned my addr 172.21.15.116:0/455747307 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:38.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.700+0000 7f9a4ffff640 1 --2- 172.21.15.116:0/455747307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a58103660 0x7f9a581988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:38.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.700+0000 7f9a56d9d640 1 --2- 172.21.15.116:0/455747307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a58102d40 0x7f9a58198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:38.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.700+0000 7f9a5759e640 1 -- 172.21.15.116:0/455747307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a58103660 msgr2=0x7f9a581988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:38.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.700+0000 7f9a5759e640 1 --2- 172.21.15.116:0/455747307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a58103660 0x7f9a581988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.700+0000 7f9a5759e640 1 -- 172.21.15.116:0/455747307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a58102d40 msgr2=0x7f9a58198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:38.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.700+0000 7f9a5759e640 1 --2- 172.21.15.116:0/455747307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a58102d40 0x7f9a58198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.700+0000 7f9a5759e640 1 -- 172.21.15.116:0/455747307 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a5819d340 con 0x7f9a5810c170 2024-09-12T03:16:38.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.700+0000 7f9a4ffff640 1 --2- 172.21.15.116:0/455747307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a58103660 0x7f9a581988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:38.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.700+0000 7f9a56d9d640 1 --2- 172.21.15.116:0/455747307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a58102d40 0x7f9a58198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:38.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.700+0000 7f9a5759e640 1 --2- 172.21.15.116:0/455747307 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a5810c170 0x7f9a5819cc60 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f9a50006010 tx=0x7f9a5002f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:38.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.701+0000 7f9a54d99640 1 -- 172.21.15.116:0/455747307 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a50034040 con 0x7f9a5810c170 2024-09-12T03:16:38.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.701+0000 7f9a54d99640 1 -- 172.21.15.116:0/455747307 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a50003e00 con 0x7f9a5810c170 2024-09-12T03:16:38.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.701+0000 7f9a5d07e640 1 -- 172.21.15.116:0/455747307 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a5819d5d0 con 0x7f9a5810c170 2024-09-12T03:16:38.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.701+0000 7f9a54d99640 1 -- 172.21.15.116:0/455747307 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a500456e0 con 0x7f9a5810c170 2024-09-12T03:16:38.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.701+0000 7f9a5d07e640 1 -- 172.21.15.116:0/455747307 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a581a4f10 con 0x7f9a5810c170 2024-09-12T03:16:38.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.703+0000 7f9a54d99640 1 -- 172.21.15.116:0/455747307 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f9a50037020 con 0x7f9a5810c170 2024-09-12T03:16:38.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.703+0000 7f9a5f457740 1 Processor -- start 2024-09-12T03:16:38.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.703+0000 7f9a5f457740 1 -- start start 2024-09-12T03:16:38.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.704+0000 7f9a5f457740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647e1ccbb40 0x5647e1cca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:38.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.704+0000 7f9a5f457740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647e1cca5d0 0x5647e1ccaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:38.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.704+0000 7f9a5f457740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647e1cd5370 0x5647e1cd7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:38.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.704+0000 7f9a5f457740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5647e1c302f0 con 0x5647e1ccbb40 2024-09-12T03:16:38.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.704+0000 7f9a5f457740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5647e1bd7cf0 con 0x5647e1cca5d0 2024-09-12T03:16:38.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.704+0000 7f9a5f457740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5647e1c2c2b0 con 0x5647e1cd5370 2024-09-12T03:16:38.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.704+0000 7f9a54d99640 1 --2- 172.21.15.116:0/455747307 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9a440a2c00 0x7f9a440a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:38.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.704+0000 7f9a4ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647e1ccbb40 0x5647e1cca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:38.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.704+0000 7f9a4ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647e1ccbb40 0x5647e1cca090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48010/0 (socket says 172.21.15.116:48010) 2024-09-12T03:16:38.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.704+0000 7f9a4ffff640 1 -- 172.21.15.116:0/829816168 learned_addr learned my addr 172.21.15.116:0/829816168 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:38.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.704+0000 7f9a5759e640 1 --2- 172.21.15.116:0/455747307 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9a440a2c00 0x7f9a440a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:38.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.704+0000 7f9a56d9d640 1 --2- 172.21.15.116:0/829816168 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647e1cca5d0 0x5647e1ccaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:38.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.704+0000 7f9a4ffff640 1 --2- 172.21.15.116:0/829816168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647e1cd5370 0x5647e1cd7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:38.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.704+0000 7f9a54d99640 1 -- 172.21.15.116:0/455747307 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f9a500f01c0 con 0x7f9a5810c170 2024-09-12T03:16:38.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.704+0000 7f9a4ffff640 1 -- 172.21.15.116:0/829816168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647e1cd5370 msgr2=0x5647e1cd7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:38.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.704+0000 7f9a4ffff640 1 --2- 172.21.15.116:0/829816168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647e1cd5370 0x5647e1cd7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.704+0000 7f9a4ffff640 1 -- 172.21.15.116:0/829816168 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647e1cca5d0 msgr2=0x5647e1ccaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:38.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.704+0000 7f9a4ffff640 1 --2- 172.21.15.116:0/829816168 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647e1cca5d0 0x5647e1ccaa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.704+0000 7f9a4ffff640 1 -- 172.21.15.116:0/829816168 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647e1bbc3a0 con 0x5647e1ccbb40 2024-09-12T03:16:38.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.704+0000 7f9a56d9d640 1 --2- 172.21.15.116:0/829816168 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647e1cca5d0 0x5647e1ccaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:38.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.704+0000 7f9a5759e640 1 --2- 172.21.15.116:0/455747307 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9a440a2c00 0x7f9a440a50c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9a500edd40 tx=0x7f9a5008c900 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:38.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.704+0000 7f9a4ffff640 1 --2- 172.21.15.116:0/829816168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647e1ccbb40 0x5647e1cca090 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f9a40008fd0 tx=0x7f9a400085b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ab91676f014c7b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:38.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.704+0000 7f9a4d7fa640 1 -- 172.21.15.116:0/829816168 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a40018030 con 0x5647e1ccbb40 2024-09-12T03:16:38.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.705+0000 7f9a4d7fa640 1 -- 172.21.15.116:0/829816168 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a40005500 con 0x5647e1ccbb40 2024-09-12T03:16:38.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.705+0000 7f9a4d7fa640 1 -- 172.21.15.116:0/829816168 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4000c7b0 con 0x5647e1ccbb40 2024-09-12T03:16:38.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.705+0000 7f9a5f457740 1 -- 172.21.15.116:0/829816168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647e1ccbb40 msgr2=0x5647e1cca090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:38.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.705+0000 7f9a5f457740 1 --2- 172.21.15.116:0/829816168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647e1ccbb40 0x5647e1cca090 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f9a40008fd0 tx=0x7f9a400085b0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.705+0000 7f9a5f457740 1 -- 172.21.15.116:0/829816168 shutdown_connections 2024-09-12T03:16:38.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.705+0000 7f9a5f457740 1 --2- 172.21.15.116:0/829816168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647e1cd5370 0x5647e1cd7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.705+0000 7f9a5f457740 1 --2- 172.21.15.116:0/829816168 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647e1cca5d0 0x5647e1ccaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.705+0000 7f9a5f457740 1 --2- 172.21.15.116:0/829816168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647e1ccbb40 0x5647e1cca090 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.705+0000 7f9a5f457740 1 -- 172.21.15.116:0/829816168 >> 172.21.15.116:0/829816168 conn(0x5647e1c93310 msgr2=0x5647e1cc6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:38.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.705+0000 7f9a5f457740 1 -- 172.21.15.116:0/829816168 shutdown_connections 2024-09-12T03:16:38.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.705+0000 7f9a5f457740 1 -- 172.21.15.116:0/829816168 wait complete. 2024-09-12T03:16:38.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.706+0000 7f9a5f457740 1 -- start start 2024-09-12T03:16:38.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.706+0000 7f9a5f457740 1 Processor -- start 2024-09-12T03:16:38.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.706+0000 7f9a5f457740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647e1cca5d0 0x5647e1cd4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:38.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.706+0000 7f9a5f457740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647e1ccbb40 0x5647e1cd1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:38.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.706+0000 7f9a5f457740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647e1cd5370 0x5647e1cd24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:38.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.706+0000 7f9a5f457740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5647e1cd8070 con 0x5647e1ccbb40 2024-09-12T03:16:38.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.706+0000 7f9a5f457740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5647e1beb060 con 0x5647e1cca5d0 2024-09-12T03:16:38.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.706+0000 7f9a5f457740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5647e1cd81f0 con 0x5647e1cd5370 2024-09-12T03:16:38.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.706+0000 7f9a5f457740 10 client.0 _ll_register_callbacks cb 0x5647e193e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:38.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.706+0000 7f9a56d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647e1ccbb40 0x5647e1cd1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:38.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.706+0000 7f9a56d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647e1ccbb40 0x5647e1cd1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48014/0 (socket says 172.21.15.116:48014) 2024-09-12T03:16:38.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.706+0000 7f9a56d9d640 1 -- 172.21.15.116:0/3640093374 learned_addr learned my addr 172.21.15.116:0/3640093374 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:38.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.706+0000 7f9a4ffff640 1 --2- 172.21.15.116:0/3640093374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647e1cca5d0 0x5647e1cd4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:38.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.706+0000 7f9a56d9d640 1 -- 172.21.15.116:0/3640093374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647e1cd5370 msgr2=0x5647e1cd24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:38.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.706+0000 7f9a56d9d640 1 --2- 172.21.15.116:0/3640093374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647e1cd5370 0x5647e1cd24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.706+0000 7f9a56d9d640 1 -- 172.21.15.116:0/3640093374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647e1cca5d0 msgr2=0x5647e1cd4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:38.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.706+0000 7f9a56d9d640 1 --2- 172.21.15.116:0/3640093374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647e1cca5d0 0x5647e1cd4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.706+0000 7f9a56d9d640 1 -- 172.21.15.116:0/3640093374 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647e19a13c0 con 0x5647e1ccbb40 2024-09-12T03:16:38.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.706+0000 7f9a4ffff640 1 --2- 172.21.15.116:0/3640093374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647e1cca5d0 0x5647e1cd4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:38.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.707+0000 7f9a56d9d640 1 --2- 172.21.15.116:0/3640093374 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647e1ccbb40 0x5647e1cd1f90 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f9a4800f790 tx=0x7f9a4800fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93975eca9f4568f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:38.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.707+0000 7f9a3e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:16:38.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.707+0000 7f9a3e7fc640 1 -- 172.21.15.116:0/3640093374 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4800fea0 con 0x5647e1ccbb40 2024-09-12T03:16:38.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.707+0000 7f9a3e7fc640 1 -- 172.21.15.116:0/3640093374 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f9a4800bd10 con 0x5647e1ccbb40 2024-09-12T03:16:38.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.707+0000 7f9a5f457740 1 -- 172.21.15.116:0/3640093374 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a40005500 con 0x5647e1ccbb40 2024-09-12T03:16:38.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.707+0000 7f9a3e7fc640 1 -- 172.21.15.116:0/3640093374 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9a4801b480 con 0x5647e1ccbb40 2024-09-12T03:16:38.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.707+0000 7f9a5f457740 10 client.19038 fetch_fsmap learned FSMap version 25 2024-09-12T03:16:38.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.707+0000 7f9a5f457740 1 -- 172.21.15.116:0/3640093374 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5647e1ce3a80 con 0x5647e1ccbb40 2024-09-12T03:16:38.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.707+0000 7f9a3e7fc640 1 -- 172.21.15.116:0/3640093374 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a40005500 con 0x5647e1ccbb40 2024-09-12T03:16:38.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.707+0000 7f9a3e7fc640 1 -- 172.21.15.116:0/3640093374 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9a48010950 con 0x5647e1ccbb40 2024-09-12T03:16:38.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.707+0000 7f9a5f457740 10 client.19038 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:16:38.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.707+0000 7f9a5f457740 10 client.19038 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:16:38.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.707+0000 7f9a5f457740 20 client.19038 populate_metadata read hostname 'smithi116' 2024-09-12T03:16:38.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.707+0000 7f9a5f457740 1 --2- 172.21.15.116:0/3640093374 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5647e1ce4f40 0x5647e1ce73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:38.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.707+0000 7f9a5f457740 4 client.19038 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:16:38.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.707+0000 7f9a5f457740 1 -- 172.21.15.116:0/3640093374 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647e1c3d860 con 0x5647e1ce4f40 2024-09-12T03:16:38.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.708+0000 7f9a4ffff640 1 --2- 172.21.15.116:0/3640093374 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5647e1ce4f40 0x5647e1ce73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:38.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.708+0000 7f9a4ffff640 1 --2- 172.21.15.116:0/3640093374 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5647e1ce4f40 0x5647e1ce73c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b304ad6f2440177 server_cookie=76050300adfe2298 in_seq=0 out_seq=0 2024-09-12T03:16:38.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.708+0000 7f9a3e7fc640 10 client.19038 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:16:38.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.709+0000 7f9a3e7fc640 1 -- 172.21.15.116:0/3640093374 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5647e1c3d860 con 0x5647e1ce4f40 2024-09-12T03:16:38.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.709+0000 7f9a3e7fc640 10 client.19038 handle_command_reply: tid=0 2024-09-12T03:16:38.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.709+0000 7f9a5f457740 1 client.19038 shutdown 2024-09-12T03:16:38.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.709+0000 7f9a5f457740 1 -- 172.21.15.116:0/3640093374 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647e1ccbb40 msgr2=0x5647e1cd1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:38.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.709+0000 7f9a5f457740 1 --2- 172.21.15.116:0/3640093374 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647e1ccbb40 0x5647e1cd1f90 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f9a4800f790 tx=0x7f9a4800fc50 comp rx=0 tx=0).stop 2024-09-12T03:16:38.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.709+0000 7f9a5f457740 1 -- 172.21.15.116:0/3640093374 shutdown_connections 2024-09-12T03:16:38.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.709+0000 7f9a5f457740 1 --2- 172.21.15.116:0/3640093374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647e1cd5370 0x5647e1cd24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.709+0000 7f9a5f457740 1 --2- 172.21.15.116:0/3640093374 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5647e1ce4f40 0x5647e1ce73c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.709+0000 7f9a5f457740 1 --2- 172.21.15.116:0/3640093374 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647e1ccbb40 0x5647e1cd1f90 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.709+0000 7f9a5f457740 1 --2- 172.21.15.116:0/3640093374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647e1cca5d0 0x5647e1cd4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.709+0000 7f9a5f457740 1 -- 172.21.15.116:0/3640093374 >> 172.21.15.116:0/3640093374 conn(0x5647e1c93310 msgr2=0x5647e1ccc930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:38.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.709+0000 7f9a3e7fc640 0 client.19038 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:16:38.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.709+0000 7f9a5f457740 1 -- 172.21.15.116:0/3640093374 shutdown_connections 2024-09-12T03:16:38.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.709+0000 7f9a5f457740 1 -- 172.21.15.116:0/3640093374 wait complete. 2024-09-12T03:16:38.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.709+0000 7f9a5f457740 20 client.19038 trim_cache size 0 max 16384 2024-09-12T03:16:38.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.722+0000 7f9a5f457740 1 Processor -- start 2024-09-12T03:16:38.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.722+0000 7f9a5f457740 1 -- start start 2024-09-12T03:16:38.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.722+0000 7f9a5f457740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647e1cca5d0 0x5647e1cd3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:38.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.722+0000 7f9a5f457740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647e1ccbb40 0x5647e1ccd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:38.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.722+0000 7f9a5f457740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647e1cd5370 0x5647e1ccd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:38.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.722+0000 7f9a5f457740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581afc60 con 0x5647e1cca5d0 2024-09-12T03:16:38.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.722+0000 7f9a5f457740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581ad530 con 0x5647e1ccbb40 2024-09-12T03:16:38.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.722+0000 7f9a5f457740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581ab380 con 0x5647e1cd5370 2024-09-12T03:16:38.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.723+0000 7f9a4ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647e1cca5d0 0x5647e1cd3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:38.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.723+0000 7f9a4ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647e1cca5d0 0x5647e1cd3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48026/0 (socket says 172.21.15.116:48026) 2024-09-12T03:16:38.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.723+0000 7f9a4ffff640 1 -- 172.21.15.116:0/408065648 learned_addr learned my addr 172.21.15.116:0/408065648 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:38.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.723+0000 7f9a56d9d640 1 --2- 172.21.15.116:0/408065648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647e1ccbb40 0x5647e1ccd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:38.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.723+0000 7f9a4ffff640 1 -- 172.21.15.116:0/408065648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647e1cd5370 msgr2=0x5647e1ccd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:38.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.723+0000 7f9a4ffff640 1 --2- 172.21.15.116:0/408065648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647e1cd5370 0x5647e1ccd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.723+0000 7f9a4ffff640 1 -- 172.21.15.116:0/408065648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647e1ccbb40 msgr2=0x5647e1ccd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:38.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.723+0000 7f9a4ffff640 1 --2- 172.21.15.116:0/408065648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647e1ccbb40 0x5647e1ccd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.723+0000 7f9a4ffff640 1 -- 172.21.15.116:0/408065648 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647e1cc7960 con 0x5647e1cca5d0 2024-09-12T03:16:38.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.723+0000 7f9a56d9d640 1 --2- 172.21.15.116:0/408065648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647e1ccbb40 0x5647e1ccd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:38.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.723+0000 7f9a4ffff640 1 --2- 172.21.15.116:0/408065648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647e1cca5d0 0x5647e1cd3f00 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f9a40004c50 tx=0x7f9a40004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a311903a1fe41339 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:38.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.723+0000 7f9a4cff9640 1 -- 172.21.15.116:0/408065648 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4002a070 con 0x5647e1cca5d0 2024-09-12T03:16:38.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.723+0000 7f9a4cff9640 1 -- 172.21.15.116:0/408065648 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a40005290 con 0x5647e1cca5d0 2024-09-12T03:16:38.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.723+0000 7f9a4cff9640 1 -- 172.21.15.116:0/408065648 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4000c970 con 0x5647e1cca5d0 2024-09-12T03:16:38.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.724+0000 7f9a5f457740 1 -- 172.21.15.116:0/408065648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647e1cca5d0 msgr2=0x5647e1cd3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:38.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.724+0000 7f9a5f457740 1 --2- 172.21.15.116:0/408065648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647e1cca5d0 0x5647e1cd3f00 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f9a40004c50 tx=0x7f9a40004c80 comp rx=0 tx=0).stop 2024-09-12T03:16:38.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.724+0000 7f9a5f457740 1 -- 172.21.15.116:0/408065648 shutdown_connections 2024-09-12T03:16:38.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.724+0000 7f9a5f457740 1 --2- 172.21.15.116:0/408065648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647e1cd5370 0x5647e1ccd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.724+0000 7f9a5f457740 1 --2- 172.21.15.116:0/408065648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647e1ccbb40 0x5647e1ccd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.724+0000 7f9a5f457740 1 --2- 172.21.15.116:0/408065648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647e1cca5d0 0x5647e1cd3f00 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.724+0000 7f9a5f457740 1 -- 172.21.15.116:0/408065648 >> 172.21.15.116:0/408065648 conn(0x5647e1c93310 msgr2=0x5647e1cd37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:38.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.724+0000 7f9a5f457740 1 -- 172.21.15.116:0/408065648 shutdown_connections 2024-09-12T03:16:38.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.724+0000 7f9a5f457740 1 -- 172.21.15.116:0/408065648 wait complete. 2024-09-12T03:16:38.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.724+0000 7f9a5f457740 1 -- start start 2024-09-12T03:16:38.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.724+0000 7f9a5f457740 1 Processor -- start 2024-09-12T03:16:38.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.725+0000 7f9a5f457740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647e1cca5d0 0x5647e1cddd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:38.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.725+0000 7f9a5f457740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647e1ccbb40 0x5647e1ccede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:38.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.725+0000 7f9a5f457740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647e1cd5370 0x5647e1ccf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:38.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.725+0000 7f9a5f457740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581ab9f0 con 0x5647e1cd5370 2024-09-12T03:16:38.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.725+0000 7f9a5f457740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581ace60 con 0x5647e1ccbb40 2024-09-12T03:16:38.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.725+0000 7f9a5f457740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581ac0c0 con 0x5647e1cca5d0 2024-09-12T03:16:38.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.725+0000 7f9a5f457740 10 client.0 _ll_register_callbacks cb 0x7f9a4800d400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:38.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.725+0000 7f9a4ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647e1cd5370 0x5647e1ccf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:38.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.725+0000 7f9a4ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647e1cd5370 0x5647e1ccf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48036/0 (socket says 172.21.15.116:48036) 2024-09-12T03:16:38.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.725+0000 7f9a4ffff640 1 -- 172.21.15.116:0/3603820512 learned_addr learned my addr 172.21.15.116:0/3603820512 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:38.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.725+0000 7f9a56d9d640 1 --2- 172.21.15.116:0/3603820512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647e1ccbb40 0x5647e1ccede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:38.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.725+0000 7f9a4ffff640 1 --2- 172.21.15.116:0/3603820512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647e1cca5d0 0x5647e1cddd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:38.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.725+0000 7f9a4ffff640 1 -- 172.21.15.116:0/3603820512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647e1cca5d0 msgr2=0x5647e1cddd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:38.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.725+0000 7f9a4ffff640 1 --2- 172.21.15.116:0/3603820512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647e1cca5d0 0x5647e1cddd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.725+0000 7f9a4ffff640 1 -- 172.21.15.116:0/3603820512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647e1ccbb40 msgr2=0x5647e1ccede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:38.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.725+0000 7f9a4ffff640 1 --2- 172.21.15.116:0/3603820512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647e1ccbb40 0x5647e1ccede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.725+0000 7f9a4ffff640 1 -- 172.21.15.116:0/3603820512 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647e1ccfc20 con 0x5647e1cd5370 2024-09-12T03:16:38.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.725+0000 7f9a4ffff640 1 --2- 172.21.15.116:0/3603820512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647e1cd5370 0x5647e1ccf3f0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f9a400084b0 tx=0x7f9a40020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b60f882493c158 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:38.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.725+0000 7f9a3e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:16:38.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.725+0000 7f9a3e7fc640 1 -- 172.21.15.116:0/3603820512 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4002a040 con 0x5647e1cd5370 2024-09-12T03:16:38.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.725+0000 7f9a3e7fc640 1 -- 172.21.15.116:0/3603820512 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f9a40005440 con 0x5647e1cd5370 2024-09-12T03:16:38.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.725+0000 7f9a5f457740 1 -- 172.21.15.116:0/3603820512 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a40005290 con 0x5647e1cd5370 2024-09-12T03:16:38.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.726+0000 7f9a3e7fc640 1 -- 172.21.15.116:0/3603820512 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9a40008d40 con 0x5647e1cd5370 2024-09-12T03:16:38.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.726+0000 7f9a5f457740 10 client.19044 fetch_fsmap learned FSMap version 25 2024-09-12T03:16:38.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.726+0000 7f9a5f457740 1 -- 172.21.15.116:0/3603820512 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5647e1d4fd90 con 0x5647e1cd5370 2024-09-12T03:16:38.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.726+0000 7f9a3e7fc640 1 -- 172.21.15.116:0/3603820512 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a40005290 con 0x5647e1cd5370 2024-09-12T03:16:38.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.726+0000 7f9a3e7fc640 1 -- 172.21.15.116:0/3603820512 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9a40033470 con 0x5647e1cd5370 2024-09-12T03:16:38.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.726+0000 7f9a5f457740 10 client.19044 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:16:38.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.726+0000 7f9a5f457740 10 client.19044 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:16:38.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.726+0000 7f9a5f457740 20 client.19044 populate_metadata read hostname 'smithi116' 2024-09-12T03:16:38.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.726+0000 7f9a5f457740 1 --2- 172.21.15.116:0/3603820512 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5647e1d50160 0x5647e1d52550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:38.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.726+0000 7f9a5f457740 4 client.19044 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:16:38.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.726+0000 7f9a5f457740 1 -- 172.21.15.116:0/3603820512 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5647e1c37d70 con 0x5647e1d50160 2024-09-12T03:16:38.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.726+0000 7f9a56d9d640 1 --2- 172.21.15.116:0/3603820512 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5647e1d50160 0x5647e1d52550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:38.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.727+0000 7f9a3e7fc640 10 client.19044 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:16:38.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.727+0000 7f9a56d9d640 1 --2- 172.21.15.116:0/3603820512 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5647e1d50160 0x5647e1d52550 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=562281078e01819e server_cookie=38f384c29354a570 in_seq=0 out_seq=0 2024-09-12T03:16:38.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.727+0000 7f9a3e7fc640 1 -- 172.21.15.116:0/3603820512 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5647e1c37d70 con 0x5647e1d50160 2024-09-12T03:16:38.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.727+0000 7f9a3e7fc640 10 client.19044 handle_command_reply: tid=0 2024-09-12T03:16:38.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.727+0000 7f9a5f457740 1 client.19044 shutdown 2024-09-12T03:16:38.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.727+0000 7f9a5f457740 1 -- 172.21.15.116:0/3603820512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647e1cd5370 msgr2=0x5647e1ccf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:38.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.727+0000 7f9a5f457740 1 --2- 172.21.15.116:0/3603820512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647e1cd5370 0x5647e1ccf3f0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f9a400084b0 tx=0x7f9a40020040 comp rx=0 tx=0).stop 2024-09-12T03:16:38.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.727+0000 7f9a5f457740 1 -- 172.21.15.116:0/3603820512 shutdown_connections 2024-09-12T03:16:38.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.727+0000 7f9a5f457740 1 --2- 172.21.15.116:0/3603820512 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5647e1d50160 0x5647e1d52550 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.727+0000 7f9a5f457740 1 --2- 172.21.15.116:0/3603820512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647e1cd5370 0x5647e1ccf3f0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.727+0000 7f9a5f457740 1 --2- 172.21.15.116:0/3603820512 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647e1ccbb40 0x5647e1ccede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.727+0000 7f9a5f457740 1 --2- 172.21.15.116:0/3603820512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5647e1cca5d0 0x5647e1cddd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.727+0000 7f9a5f457740 1 -- 172.21.15.116:0/3603820512 >> 172.21.15.116:0/3603820512 conn(0x5647e1c93310 msgr2=0x5647e1cd4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:38.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.727+0000 7f9a3e7fc640 0 client.19044 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:16:38.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.727+0000 7f9a5f457740 1 -- 172.21.15.116:0/3603820512 shutdown_connections 2024-09-12T03:16:38.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.727+0000 7f9a5f457740 1 -- 172.21.15.116:0/3603820512 wait complete. 2024-09-12T03:16:38.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.727+0000 7f9a5f457740 20 client.19044 trim_cache size 0 max 16384 2024-09-12T03:16:38.740 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:16:38.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.728+0000 7f9a5d07e640 1 -- 172.21.15.116:0/455747307 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9a440a2c00 msgr2=0x7f9a440a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:38.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.728+0000 7f9a5d07e640 1 --2- 172.21.15.116:0/455747307 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9a440a2c00 0x7f9a440a50c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9a500edd40 tx=0x7f9a5008c900 comp rx=0 tx=0).stop 2024-09-12T03:16:38.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.728+0000 7f9a5d07e640 1 -- 172.21.15.116:0/455747307 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a5810c170 msgr2=0x7f9a5819cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:38.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.728+0000 7f9a5d07e640 1 --2- 172.21.15.116:0/455747307 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a5810c170 0x7f9a5819cc60 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f9a50006010 tx=0x7f9a5002f710 comp rx=0 tx=0).stop 2024-09-12T03:16:38.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.728+0000 7f9a5d07e640 1 -- 172.21.15.116:0/455747307 shutdown_connections 2024-09-12T03:16:38.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.728+0000 7f9a5d07e640 1 --2- 172.21.15.116:0/455747307 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9a440a2c00 0x7f9a440a50c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.728+0000 7f9a5d07e640 1 --2- 172.21.15.116:0/455747307 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a5810c170 0x7f9a5819cc60 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.728+0000 7f9a5d07e640 1 --2- 172.21.15.116:0/455747307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a58103660 0x7f9a581988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.728+0000 7f9a5d07e640 1 --2- 172.21.15.116:0/455747307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a58102d40 0x7f9a58198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:38.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.728+0000 7f9a5d07e640 1 -- 172.21.15.116:0/455747307 >> 172.21.15.116:0/455747307 conn(0x7f9a580fc7a0 msgr2=0x7f9a580fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:38.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.728+0000 7f9a5d07e640 1 -- 172.21.15.116:0/455747307 shutdown_connections 2024-09-12T03:16:38.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:38.728+0000 7f9a5d07e640 1 -- 172.21.15.116:0/455747307 wait complete. 2024-09-12T03:16:39.643 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:16:39.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:39 smithi183 ceph-mon[35535]: pgmap v748: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-12T03:16:39.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:39 smithi177 ceph-mon[35305]: pgmap v748: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-12T03:16:39.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:39 smithi116 ceph-mon[31410]: pgmap v748: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-12T03:16:39.899 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:40.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.385+0000 7fcf1c4d1640 1 -- 172.21.15.116:0/2611097616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf14101de0 msgr2=0x7fcf141021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.385+0000 7fcf1c4d1640 1 --2- 172.21.15.116:0/2611097616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf14101de0 0x7fcf141021c0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009900 tx=0x7fcf0c02a410 comp rx=0 tx=0).stop 2024-09-12T03:16:40.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.385+0000 7fcf1c4d1640 1 -- 172.21.15.116:0/2611097616 shutdown_connections 2024-09-12T03:16:40.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.385+0000 7fcf1c4d1640 1 --2- 172.21.15.116:0/2611097616 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf14106f10 0x7fcf1410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.385+0000 7fcf1c4d1640 1 --2- 172.21.15.116:0/2611097616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf14102790 0x7fcf141067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.385+0000 7fcf1c4d1640 1 --2- 172.21.15.116:0/2611097616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf14101de0 0x7fcf141021c0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.385+0000 7fcf1c4d1640 1 -- 172.21.15.116:0/2611097616 >> 172.21.15.116:0/2611097616 conn(0x7fcf140fd8c0 msgr2=0x7fcf140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.386+0000 7fcf1c4d1640 1 -- 172.21.15.116:0/2611097616 shutdown_connections 2024-09-12T03:16:40.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.386+0000 7fcf1c4d1640 1 -- 172.21.15.116:0/2611097616 wait complete. 2024-09-12T03:16:40.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.386+0000 7fcf1c4d1640 1 Processor -- start 2024-09-12T03:16:40.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.386+0000 7fcf1c4d1640 1 -- start start 2024-09-12T03:16:40.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.386+0000 7fcf1c4d1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf14101de0 0x7fcf14198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.386+0000 7fcf1c4d1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf14102790 0x7fcf141989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.386+0000 7fcf1c4d1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf14106f10 0x7fcf1419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.386+0000 7fcf1c4d1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcf14111f70 con 0x7fcf14102790 2024-09-12T03:16:40.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.386+0000 7fcf1c4d1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcf14111df0 con 0x7fcf14106f10 2024-09-12T03:16:40.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.386+0000 7fcf1c4d1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf141120f0 con 0x7fcf14101de0 2024-09-12T03:16:40.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.386+0000 7fcf19a6c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf14102790 0x7fcf141989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.386+0000 7fcf19a6c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf14102790 0x7fcf141989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48054/0 (socket says 172.21.15.116:48054) 2024-09-12T03:16:40.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.387+0000 7fcf19a6c640 1 -- 172.21.15.116:0/1807942085 learned_addr learned my addr 172.21.15.116:0/1807942085 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:40.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.387+0000 7fcf1a26d640 1 --2- 172.21.15.116:0/1807942085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf14101de0 0x7fcf14198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.387+0000 7fcf1aa6e640 1 --2- 172.21.15.116:0/1807942085 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf14106f10 0x7fcf1419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.387+0000 7fcf19a6c640 1 -- 172.21.15.116:0/1807942085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf14101de0 msgr2=0x7fcf14198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.387+0000 7fcf19a6c640 1 --2- 172.21.15.116:0/1807942085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf14101de0 0x7fcf14198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.387+0000 7fcf19a6c640 1 -- 172.21.15.116:0/1807942085 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf14106f10 msgr2=0x7fcf1419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.387+0000 7fcf19a6c640 1 --2- 172.21.15.116:0/1807942085 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf14106f10 0x7fcf1419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.387+0000 7fcf19a6c640 1 -- 172.21.15.116:0/1807942085 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1419d420 con 0x7fcf14102790 2024-09-12T03:16:40.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.387+0000 7fcf19a6c640 1 --2- 172.21.15.116:0/1807942085 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf14102790 0x7fcf141989b0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800d770 tx=0x7fcf0800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.387+0000 7fcf137fe640 1 -- 172.21.15.116:0/1807942085 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf08017070 con 0x7fcf14102790 2024-09-12T03:16:40.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.387+0000 7fcf137fe640 1 -- 172.21.15.116:0/1807942085 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf08005500 con 0x7fcf14102790 2024-09-12T03:16:40.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.387+0000 7fcf1c4d1640 1 -- 172.21.15.116:0/1807942085 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf1419d710 con 0x7fcf14102790 2024-09-12T03:16:40.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.387+0000 7fcf137fe640 1 -- 172.21.15.116:0/1807942085 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf08004e00 con 0x7fcf14102790 2024-09-12T03:16:40.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.387+0000 7fcf1c4d1640 1 -- 172.21.15.116:0/1807942085 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf141a4ff0 con 0x7fcf14102790 2024-09-12T03:16:40.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.389+0000 7fcf137fe640 1 -- 172.21.15.116:0/1807942085 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fcf08004fa0 con 0x7fcf14102790 2024-09-12T03:16:40.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.390+0000 7fcf1e8aa740 1 Processor -- start 2024-09-12T03:16:40.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.390+0000 7fcf1e8aa740 1 -- start start 2024-09-12T03:16:40.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.390+0000 7fcf1e8aa740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55835117bb40 0x55835117a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.390+0000 7fcf1e8aa740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55835117a5d0 0x55835117aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.390+0000 7fcf1e8aa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558351185370 0x558351187760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.390+0000 7fcf1e8aa740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5583510e02f0 con 0x55835117a5d0 2024-09-12T03:16:40.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.390+0000 7fcf1e8aa740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558351087cf0 con 0x55835117bb40 2024-09-12T03:16:40.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.390+0000 7fcf1e8aa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5583510dc2b0 con 0x558351185370 2024-09-12T03:16:40.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.390+0000 7fcf1a26d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55835117a5d0 0x55835117aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.390+0000 7fcf1a26d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55835117a5d0 0x55835117aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48060/0 (socket says 172.21.15.116:48060) 2024-09-12T03:16:40.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.390+0000 7fcf1a26d640 1 -- 172.21.15.116:0/4258200595 learned_addr learned my addr 172.21.15.116:0/4258200595 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:40.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.390+0000 7fcf137fe640 1 --2- 172.21.15.116:0/1807942085 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcefc0a2b80 0x7fcefc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.391+0000 7fcf1a26d640 1 -- 172.21.15.116:0/4258200595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558351185370 msgr2=0x558351187760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:40.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.391+0000 7fcf1a26d640 1 --2- 172.21.15.116:0/4258200595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558351185370 0x558351187760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.391+0000 7fcf1a26d640 1 -- 172.21.15.116:0/4258200595 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55835117bb40 msgr2=0x55835117a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:40.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.391+0000 7fcf1aa6e640 1 --2- 172.21.15.116:0/4258200595 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55835117bb40 0x55835117a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.391+0000 7fcf1a26d640 1 --2- 172.21.15.116:0/4258200595 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55835117bb40 0x55835117a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.391+0000 7fcf1a26d640 1 -- 172.21.15.116:0/4258200595 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55835106c3a0 con 0x55835117a5d0 2024-09-12T03:16:40.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.391+0000 7fcf137fe640 1 -- 172.21.15.116:0/1807942085 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fcf080cea40 con 0x7fcf14102790 2024-09-12T03:16:40.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.391+0000 7fcf1aa6e640 1 --2- 172.21.15.116:0/4258200595 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55835117bb40 0x55835117a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:40.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.391+0000 7fcf1aa6e640 1 --2- 172.21.15.116:0/1807942085 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcefc0a2b80 0x7fcefc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.391+0000 7fcf1a26d640 1 --2- 172.21.15.116:0/4258200595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55835117a5d0 0x55835117aa10 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009900 tx=0x7fcf0c031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977cd8ef2d1baaaa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.391+0000 7fcf10ff9640 1 -- 172.21.15.116:0/4258200595 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c040070 con 0x55835117a5d0 2024-09-12T03:16:40.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.391+0000 7fcf10ff9640 1 -- 172.21.15.116:0/4258200595 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf0c005020 con 0x55835117a5d0 2024-09-12T03:16:40.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.391+0000 7fcf10ff9640 1 -- 172.21.15.116:0/4258200595 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c035040 con 0x55835117a5d0 2024-09-12T03:16:40.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.392+0000 7fcf1e8aa740 1 -- 172.21.15.116:0/4258200595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55835117a5d0 msgr2=0x55835117aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.392+0000 7fcf1aa6e640 1 --2- 172.21.15.116:0/1807942085 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcefc0a2b80 0x7fcefc0a5040 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400ae80 tx=0x7fcf04004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.392+0000 7fcf1e8aa740 1 --2- 172.21.15.116:0/4258200595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55835117a5d0 0x55835117aa10 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009900 tx=0x7fcf0c031630 comp rx=0 tx=0).stop 2024-09-12T03:16:40.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.392+0000 7fcf1e8aa740 1 -- 172.21.15.116:0/4258200595 shutdown_connections 2024-09-12T03:16:40.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.392+0000 7fcf1e8aa740 1 --2- 172.21.15.116:0/4258200595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558351185370 0x558351187760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.392+0000 7fcf1e8aa740 1 --2- 172.21.15.116:0/4258200595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55835117a5d0 0x55835117aa10 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.392+0000 7fcf1e8aa740 1 --2- 172.21.15.116:0/4258200595 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55835117bb40 0x55835117a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.392+0000 7fcf1e8aa740 1 -- 172.21.15.116:0/4258200595 >> 172.21.15.116:0/4258200595 conn(0x558351143310 msgr2=0x558351176c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.392+0000 7fcf1e8aa740 1 -- 172.21.15.116:0/4258200595 shutdown_connections 2024-09-12T03:16:40.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.392+0000 7fcf1e8aa740 1 -- 172.21.15.116:0/4258200595 wait complete. 2024-09-12T03:16:40.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.392+0000 7fcf1e8aa740 1 -- start start 2024-09-12T03:16:40.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.392+0000 7fcf1e8aa740 1 Processor -- start 2024-09-12T03:16:40.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.393+0000 7fcf1e8aa740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55835117a5d0 0x558351184de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.393+0000 7fcf1e8aa740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55835117bb40 0x558351182090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.393+0000 7fcf1e8aa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558351185370 0x5583511827d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.393+0000 7fcf1e8aa740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x558351188010 con 0x55835117a5d0 2024-09-12T03:16:40.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.393+0000 7fcf1e8aa740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55835109b060 con 0x55835117bb40 2024-09-12T03:16:40.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.393+0000 7fcf1e8aa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558351188190 con 0x558351185370 2024-09-12T03:16:40.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.393+0000 7fcf1e8aa740 10 client.0 _ll_register_callbacks cb 0x558350dee8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:40.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.393+0000 7fcf1a26d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55835117a5d0 0x558351184de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.393+0000 7fcf1a26d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55835117a5d0 0x558351184de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48062/0 (socket says 172.21.15.116:48062) 2024-09-12T03:16:40.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.393+0000 7fcf1a26d640 1 -- 172.21.15.116:0/1049675685 learned_addr learned my addr 172.21.15.116:0/1049675685 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:40.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.393+0000 7fcf19a6c640 1 --2- 172.21.15.116:0/1049675685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55835117bb40 0x558351182090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.393+0000 7fcf1a26d640 1 -- 172.21.15.116:0/1049675685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558351185370 msgr2=0x5583511827d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:40.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.393+0000 7fcf1aa6e640 1 --2- 172.21.15.116:0/1049675685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558351185370 0x5583511827d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.393+0000 7fcf1a26d640 1 --2- 172.21.15.116:0/1049675685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558351185370 0x5583511827d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.393+0000 7fcf1a26d640 1 -- 172.21.15.116:0/1049675685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55835117bb40 msgr2=0x558351182090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:40.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.393+0000 7fcf1a26d640 1 --2- 172.21.15.116:0/1049675685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55835117bb40 0x558351182090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.393+0000 7fcf1a26d640 1 -- 172.21.15.116:0/1049675685 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558350e513c0 con 0x55835117a5d0 2024-09-12T03:16:40.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.393+0000 7fcf19a6c640 1 --2- 172.21.15.116:0/1049675685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55835117bb40 0x558351182090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:40.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.393+0000 7fcf1a26d640 1 --2- 172.21.15.116:0/1049675685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55835117a5d0 0x558351184de0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c02a450 tx=0x7fcf0c005400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7525d8113a8329f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.393+0000 7fcef9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:16:40.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.393+0000 7fcef9ffb640 1 -- 172.21.15.116:0/1049675685 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c040030 con 0x55835117a5d0 2024-09-12T03:16:40.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.393+0000 7fcef9ffb640 1 -- 172.21.15.116:0/1049675685 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fcf0c004940 con 0x55835117a5d0 2024-09-12T03:16:40.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.393+0000 7fcf1e8aa740 1 -- 172.21.15.116:0/1049675685 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf0c005020 con 0x55835117a5d0 2024-09-12T03:16:40.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.394+0000 7fcef9ffb640 1 -- 172.21.15.116:0/1049675685 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcf0c035070 con 0x55835117a5d0 2024-09-12T03:16:40.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.394+0000 7fcf1e8aa740 10 client.19056 fetch_fsmap learned FSMap version 25 2024-09-12T03:16:40.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.394+0000 7fcf1e8aa740 1 -- 172.21.15.116:0/1049675685 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558351193ab0 con 0x55835117a5d0 2024-09-12T03:16:40.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.394+0000 7fcef9ffb640 1 -- 172.21.15.116:0/1049675685 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf0c005020 con 0x55835117a5d0 2024-09-12T03:16:40.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.394+0000 7fcef9ffb640 1 -- 172.21.15.116:0/1049675685 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcf0c044400 con 0x55835117a5d0 2024-09-12T03:16:40.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.394+0000 7fcf1e8aa740 10 client.19056 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:16:40.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.394+0000 7fcf1e8aa740 10 client.19056 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:16:40.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.394+0000 7fcf1e8aa740 20 client.19056 populate_metadata read hostname 'smithi116' 2024-09-12T03:16:40.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.394+0000 7fcf1e8aa740 1 --2- 172.21.15.116:0/1049675685 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x558351194e00 0x5583511971f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.394+0000 7fcf1e8aa740 4 client.19056 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:16:40.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.394+0000 7fcf1e8aa740 1 -- 172.21.15.116:0/1049675685 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583510ed860 con 0x558351194e00 2024-09-12T03:16:40.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.394+0000 7fcf19a6c640 1 --2- 172.21.15.116:0/1049675685 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x558351194e00 0x5583511971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.395+0000 7fcf19a6c640 1 --2- 172.21.15.116:0/1049675685 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x558351194e00 0x5583511971f0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=307d1dac8a5a8e66 server_cookie=1633467927e2e40c in_seq=0 out_seq=0 2024-09-12T03:16:40.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.395+0000 7fcef9ffb640 10 client.19056 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:16:40.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.395+0000 7fcef9ffb640 1 -- 172.21.15.116:0/1049675685 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5583510ed860 con 0x558351194e00 2024-09-12T03:16:40.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.395+0000 7fcef9ffb640 10 client.19056 handle_command_reply: tid=0 2024-09-12T03:16:40.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.395+0000 7fcf1e8aa740 1 client.19056 shutdown 2024-09-12T03:16:40.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.396+0000 7fcf1e8aa740 1 -- 172.21.15.116:0/1049675685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55835117a5d0 msgr2=0x558351184de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.396+0000 7fcf1e8aa740 1 --2- 172.21.15.116:0/1049675685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55835117a5d0 0x558351184de0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c02a450 tx=0x7fcf0c005400 comp rx=0 tx=0).stop 2024-09-12T03:16:40.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.396+0000 7fcf1e8aa740 1 -- 172.21.15.116:0/1049675685 shutdown_connections 2024-09-12T03:16:40.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.396+0000 7fcf1e8aa740 1 --2- 172.21.15.116:0/1049675685 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x558351194e00 0x5583511971f0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.396+0000 7fcf1e8aa740 1 --2- 172.21.15.116:0/1049675685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558351185370 0x5583511827d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.396+0000 7fcf1e8aa740 1 --2- 172.21.15.116:0/1049675685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55835117bb40 0x558351182090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.396+0000 7fcf1e8aa740 1 --2- 172.21.15.116:0/1049675685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55835117a5d0 0x558351184de0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.396+0000 7fcef9ffb640 0 client.19056 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:16:40.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.396+0000 7fcf1e8aa740 1 -- 172.21.15.116:0/1049675685 >> 172.21.15.116:0/1049675685 conn(0x558351143310 msgr2=0x55835117c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.396+0000 7fcf1e8aa740 1 -- 172.21.15.116:0/1049675685 shutdown_connections 2024-09-12T03:16:40.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.396+0000 7fcf1e8aa740 1 -- 172.21.15.116:0/1049675685 wait complete. 2024-09-12T03:16:40.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.396+0000 7fcf1e8aa740 20 client.19056 trim_cache size 0 max 16384 2024-09-12T03:16:40.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.409+0000 7fcf1e8aa740 1 Processor -- start 2024-09-12T03:16:40.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.409+0000 7fcf1e8aa740 1 -- start start 2024-09-12T03:16:40.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.409+0000 7fcf1e8aa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55835117a5d0 0x558351176e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.409+0000 7fcf1e8aa740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55835117bb40 0x558351191430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.409+0000 7fcf1e8aa740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558351185370 0x55835119a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.409+0000 7fcf1e8aa740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcf141afd10 con 0x55835117bb40 2024-09-12T03:16:40.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.409+0000 7fcf1e8aa740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcf141ad5e0 con 0x558351185370 2024-09-12T03:16:40.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.409+0000 7fcf1e8aa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf141ab430 con 0x55835117a5d0 2024-09-12T03:16:40.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.409+0000 7fcf19a6c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55835117bb40 0x558351191430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.409+0000 7fcf19a6c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55835117bb40 0x558351191430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48068/0 (socket says 172.21.15.116:48068) 2024-09-12T03:16:40.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.409+0000 7fcf19a6c640 1 -- 172.21.15.116:0/4027562532 learned_addr learned my addr 172.21.15.116:0/4027562532 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:40.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.409+0000 7fcf1a26d640 1 --2- 172.21.15.116:0/4027562532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55835117a5d0 0x558351176e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.409+0000 7fcf19a6c640 1 -- 172.21.15.116:0/4027562532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55835117a5d0 msgr2=0x558351176e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:40.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.409+0000 7fcf19a6c640 1 --2- 172.21.15.116:0/4027562532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55835117a5d0 0x558351176e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.409+0000 7fcf19a6c640 1 -- 172.21.15.116:0/4027562532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558351185370 msgr2=0x55835119a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:40.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.409+0000 7fcf19a6c640 1 --2- 172.21.15.116:0/4027562532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558351185370 0x55835119a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.409+0000 7fcf19a6c640 1 -- 172.21.15.116:0/4027562532 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558351176510 con 0x55835117bb40 2024-09-12T03:16:40.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.409+0000 7fcf1a26d640 1 --2- 172.21.15.116:0/4027562532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55835117a5d0 0x558351176e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:40.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.409+0000 7fcf19a6c640 1 --2- 172.21.15.116:0/4027562532 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55835117bb40 0x558351191430 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fcf080cc510 tx=0x7fcf080bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b3e5c16095e87d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.409+0000 7fcefbfff640 1 -- 172.21.15.116:0/4027562532 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf08020040 con 0x55835117bb40 2024-09-12T03:16:40.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.410+0000 7fcefbfff640 1 -- 172.21.15.116:0/4027562532 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf080a3ee0 con 0x55835117bb40 2024-09-12T03:16:40.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.410+0000 7fcefbfff640 1 -- 172.21.15.116:0/4027562532 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf08097990 con 0x55835117bb40 2024-09-12T03:16:40.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.410+0000 7fcf1e8aa740 1 -- 172.21.15.116:0/4027562532 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55835117bb40 msgr2=0x558351191430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.410+0000 7fcf1e8aa740 1 --2- 172.21.15.116:0/4027562532 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55835117bb40 0x558351191430 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fcf080cc510 tx=0x7fcf080bb040 comp rx=0 tx=0).stop 2024-09-12T03:16:40.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.410+0000 7fcf1e8aa740 1 -- 172.21.15.116:0/4027562532 shutdown_connections 2024-09-12T03:16:40.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.410+0000 7fcf1e8aa740 1 --2- 172.21.15.116:0/4027562532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558351185370 0x55835119a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.410+0000 7fcf1e8aa740 1 --2- 172.21.15.116:0/4027562532 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55835117bb40 0x558351191430 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.410+0000 7fcf1e8aa740 1 --2- 172.21.15.116:0/4027562532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55835117a5d0 0x558351176e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.410+0000 7fcf1e8aa740 1 -- 172.21.15.116:0/4027562532 >> 172.21.15.116:0/4027562532 conn(0x558351143310 msgr2=0x558351176700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.410+0000 7fcf1e8aa740 1 -- 172.21.15.116:0/4027562532 shutdown_connections 2024-09-12T03:16:40.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.410+0000 7fcf1e8aa740 1 -- 172.21.15.116:0/4027562532 wait complete. 2024-09-12T03:16:40.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.410+0000 7fcf1e8aa740 1 -- start start 2024-09-12T03:16:40.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.411+0000 7fcf1e8aa740 1 Processor -- start 2024-09-12T03:16:40.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.411+0000 7fcf1e8aa740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55835117a5d0 0x55835119ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.411+0000 7fcf1e8aa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55835117bb40 0x558351197e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.411+0000 7fcf1e8aa740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558351185370 0x558351198590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.411+0000 7fcf1e8aa740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcf141abaa0 con 0x55835117a5d0 2024-09-12T03:16:40.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.411+0000 7fcf1e8aa740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcf141acf10 con 0x558351185370 2024-09-12T03:16:40.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.411+0000 7fcf1e8aa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf141ac170 con 0x55835117bb40 2024-09-12T03:16:40.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.411+0000 7fcf1e8aa740 10 client.0 _ll_register_callbacks cb 0x7fcf0c009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:40.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.411+0000 7fcf1a26d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55835117a5d0 0x55835119ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.411+0000 7fcf1a26d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55835117a5d0 0x55835119ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48084/0 (socket says 172.21.15.116:48084) 2024-09-12T03:16:40.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.411+0000 7fcf1a26d640 1 -- 172.21.15.116:0/2038027980 learned_addr learned my addr 172.21.15.116:0/2038027980 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:40.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.411+0000 7fcf1aa6e640 1 --2- 172.21.15.116:0/2038027980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558351185370 0x558351198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.411+0000 7fcf19a6c640 1 --2- 172.21.15.116:0/2038027980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55835117bb40 0x558351197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.411+0000 7fcf1a26d640 1 -- 172.21.15.116:0/2038027980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55835117bb40 msgr2=0x558351197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:40.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.411+0000 7fcf1a26d640 1 --2- 172.21.15.116:0/2038027980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55835117bb40 0x558351197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.411+0000 7fcf1a26d640 1 -- 172.21.15.116:0/2038027980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558351185370 msgr2=0x558351198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:40.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.411+0000 7fcf1a26d640 1 --2- 172.21.15.116:0/2038027980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558351185370 0x558351198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.411+0000 7fcf1a26d640 1 -- 172.21.15.116:0/2038027980 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583511fe740 con 0x55835117a5d0 2024-09-12T03:16:40.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.411+0000 7fcf1aa6e640 1 --2- 172.21.15.116:0/2038027980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558351185370 0x558351198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:40.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.411+0000 7fcf1a26d640 1 --2- 172.21.15.116:0/2038027980 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55835117a5d0 0x55835119ab00 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c02a930 tx=0x7fcf0c044630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219023def605df8a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:40.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.411+0000 7fcef9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:16:40.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.411+0000 7fcef9ffb640 1 -- 172.21.15.116:0/2038027980 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c05d070 con 0x55835117a5d0 2024-09-12T03:16:40.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.411+0000 7fcef9ffb640 1 -- 172.21.15.116:0/2038027980 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fcf0c0047c0 con 0x55835117a5d0 2024-09-12T03:16:40.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.411+0000 7fcf1e8aa740 1 -- 172.21.15.116:0/2038027980 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf080a3ee0 con 0x55835117a5d0 2024-09-12T03:16:40.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.412+0000 7fcef9ffb640 1 -- 172.21.15.116:0/2038027980 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcf0c040070 con 0x55835117a5d0 2024-09-12T03:16:40.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.412+0000 7fcf1e8aa740 10 client.19062 fetch_fsmap learned FSMap version 25 2024-09-12T03:16:40.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.412+0000 7fcf1e8aa740 1 -- 172.21.15.116:0/2038027980 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5583511ff880 con 0x55835117a5d0 2024-09-12T03:16:40.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.412+0000 7fcef9ffb640 1 -- 172.21.15.116:0/2038027980 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf080a3ee0 con 0x55835117a5d0 2024-09-12T03:16:40.428 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:16:40.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.412+0000 7fcef9ffb640 1 -- 172.21.15.116:0/2038027980 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcf0c062820 con 0x55835117a5d0 2024-09-12T03:16:40.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.412+0000 7fcf1e8aa740 10 client.19062 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:16:40.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.412+0000 7fcf1e8aa740 10 client.19062 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:16:40.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.412+0000 7fcf1e8aa740 20 client.19062 populate_metadata read hostname 'smithi116' 2024-09-12T03:16:40.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.412+0000 7fcf1e8aa740 1 --2- 172.21.15.116:0/2038027980 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x558351200000 0x558351202420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:40.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.412+0000 7fcf1e8aa740 4 client.19062 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:16:40.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.412+0000 7fcf1e8aa740 1 -- 172.21.15.116:0/2038027980 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5583510e7d70 con 0x558351200000 2024-09-12T03:16:40.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.412+0000 7fcf19a6c640 1 --2- 172.21.15.116:0/2038027980 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x558351200000 0x558351202420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:40.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.413+0000 7fcef9ffb640 10 client.19062 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:16:40.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.413+0000 7fcf19a6c640 1 --2- 172.21.15.116:0/2038027980 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x558351200000 0x558351202420 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a8ea17aa9dc9d13d server_cookie=76cd058b411bbeee in_seq=0 out_seq=0 2024-09-12T03:16:40.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.413+0000 7fcef9ffb640 1 -- 172.21.15.116:0/2038027980 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5583510e7d70 con 0x558351200000 2024-09-12T03:16:40.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.413+0000 7fcef9ffb640 10 client.19062 handle_command_reply: tid=0 2024-09-12T03:16:40.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.413+0000 7fcf1e8aa740 1 client.19062 shutdown 2024-09-12T03:16:40.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.413+0000 7fcf1e8aa740 1 -- 172.21.15.116:0/2038027980 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55835117a5d0 msgr2=0x55835119ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.413+0000 7fcf1e8aa740 1 --2- 172.21.15.116:0/2038027980 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55835117a5d0 0x55835119ab00 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c02a930 tx=0x7fcf0c044630 comp rx=0 tx=0).stop 2024-09-12T03:16:40.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.413+0000 7fcf1e8aa740 1 -- 172.21.15.116:0/2038027980 shutdown_connections 2024-09-12T03:16:40.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.413+0000 7fcf1e8aa740 1 --2- 172.21.15.116:0/2038027980 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x558351200000 0x558351202420 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.413+0000 7fcf1e8aa740 1 --2- 172.21.15.116:0/2038027980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558351185370 0x558351198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.413+0000 7fcf1e8aa740 1 --2- 172.21.15.116:0/2038027980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55835117bb40 0x558351197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.413+0000 7fcef9ffb640 0 client.19062 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:16:40.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.413+0000 7fcf1e8aa740 1 --2- 172.21.15.116:0/2038027980 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55835117a5d0 0x55835119ab00 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.413+0000 7fcf1e8aa740 1 -- 172.21.15.116:0/2038027980 >> 172.21.15.116:0/2038027980 conn(0x558351143310 msgr2=0x5583511754e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.413+0000 7fcf1e8aa740 1 -- 172.21.15.116:0/2038027980 shutdown_connections 2024-09-12T03:16:40.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.413+0000 7fcf1e8aa740 1 -- 172.21.15.116:0/2038027980 wait complete. 2024-09-12T03:16:40.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.413+0000 7fcf1e8aa740 20 client.19062 trim_cache size 0 max 16384 2024-09-12T03:16:40.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.414+0000 7fcf1c4d1640 1 -- 172.21.15.116:0/1807942085 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcefc0a2b80 msgr2=0x7fcefc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.414+0000 7fcf1c4d1640 1 --2- 172.21.15.116:0/1807942085 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcefc0a2b80 0x7fcefc0a5040 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400ae80 tx=0x7fcf04004a40 comp rx=0 tx=0).stop 2024-09-12T03:16:40.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.414+0000 7fcf1c4d1640 1 -- 172.21.15.116:0/1807942085 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf14102790 msgr2=0x7fcf141989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:40.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.414+0000 7fcf1c4d1640 1 --2- 172.21.15.116:0/1807942085 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf14102790 0x7fcf141989b0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800d770 tx=0x7fcf0800dc30 comp rx=0 tx=0).stop 2024-09-12T03:16:40.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.414+0000 7fcf1c4d1640 1 -- 172.21.15.116:0/1807942085 shutdown_connections 2024-09-12T03:16:40.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.414+0000 7fcf1c4d1640 1 --2- 172.21.15.116:0/1807942085 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcefc0a2b80 0x7fcefc0a5040 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.414+0000 7fcf1c4d1640 1 --2- 172.21.15.116:0/1807942085 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf14106f10 0x7fcf1419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.414+0000 7fcf1c4d1640 1 --2- 172.21.15.116:0/1807942085 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf14102790 0x7fcf141989b0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.414+0000 7fcf1c4d1640 1 --2- 172.21.15.116:0/1807942085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf14101de0 0x7fcf14198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:40.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.414+0000 7fcf1c4d1640 1 -- 172.21.15.116:0/1807942085 >> 172.21.15.116:0/1807942085 conn(0x7fcf140fd8c0 msgr2=0x7fcf140fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:40.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.414+0000 7fcf1c4d1640 1 -- 172.21.15.116:0/1807942085 shutdown_connections 2024-09-12T03:16:40.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:40.414+0000 7fcf1c4d1640 1 -- 172.21.15.116:0/1807942085 wait complete. 2024-09-12T03:16:41.231 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:16:41.477 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:41.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:41 smithi183 ceph-mon[35535]: pgmap v749: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-12T03:16:41.724 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:41 smithi116 ceph-mon[31410]: pgmap v749: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-12T03:16:41.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:41 smithi177 ceph-mon[35305]: pgmap v749: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-12T03:16:42.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.971+0000 7fcc1cff9640 1 -- 172.21.15.116:0/2539358462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc18101420 msgr2=0x7fcc1810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.971+0000 7fcc1cff9640 1 --2- 172.21.15.116:0/2539358462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc18101420 0x7fcc1810d950 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009cd0 tx=0x7fcc1002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.971+0000 7fcc1cff9640 1 -- 172.21.15.116:0/2539358462 shutdown_connections 2024-09-12T03:16:42.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.971+0000 7fcc1cff9640 1 --2- 172.21.15.116:0/2539358462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc18101420 0x7fcc1810d950 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.971+0000 7fcc1cff9640 1 --2- 172.21.15.116:0/2539358462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc18100a80 0x7fcc18100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.971+0000 7fcc1cff9640 1 --2- 172.21.15.116:0/2539358462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc18106a80 0x7fcc18106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.971+0000 7fcc1cff9640 1 -- 172.21.15.116:0/2539358462 >> 172.21.15.116:0/2539358462 conn(0x7fcc180fc7a0 msgr2=0x7fcc180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:42.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.971+0000 7fcc1cff9640 1 -- 172.21.15.116:0/2539358462 shutdown_connections 2024-09-12T03:16:42.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.971+0000 7fcc1cff9640 1 -- 172.21.15.116:0/2539358462 wait complete. 2024-09-12T03:16:42.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.972+0000 7fcc1cff9640 1 Processor -- start 2024-09-12T03:16:42.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.972+0000 7fcc1cff9640 1 -- start start 2024-09-12T03:16:42.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.972+0000 7fcc1cff9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc18100a80 0x7fcc18197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.972+0000 7fcc1cff9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc18101420 0x7fcc181975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.972+0000 7fcc1cff9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc18106a80 0x7fcc18197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.972+0000 7fcc1cff9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcc18110e80 con 0x7fcc18100a80 2024-09-12T03:16:42.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.972+0000 7fcc1cff9640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcc18110d00 con 0x7fcc18101420 2024-09-12T03:16:42.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.972+0000 7fcc1cff9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcc18111000 con 0x7fcc18106a80 2024-09-12T03:16:42.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.972+0000 7fcc1659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc18100a80 0x7fcc18197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.972+0000 7fcc1659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc18100a80 0x7fcc18197080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41928/0 (socket says 172.21.15.116:41928) 2024-09-12T03:16:42.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.972+0000 7fcc1659c640 1 -- 172.21.15.116:0/3395983567 learned_addr learned my addr 172.21.15.116:0/3395983567 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:42.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.972+0000 7fcc16d9d640 1 --2- 172.21.15.116:0/3395983567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc18106a80 0x7fcc18197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.972+0000 7fcc15d9b640 1 --2- 172.21.15.116:0/3395983567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc18101420 0x7fcc181975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.972+0000 7fcc1659c640 1 -- 172.21.15.116:0/3395983567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc18106a80 msgr2=0x7fcc18197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.972+0000 7fcc1659c640 1 --2- 172.21.15.116:0/3395983567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc18106a80 0x7fcc18197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.972+0000 7fcc1659c640 1 -- 172.21.15.116:0/3395983567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc18101420 msgr2=0x7fcc181975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.972+0000 7fcc1659c640 1 --2- 172.21.15.116:0/3395983567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc18101420 0x7fcc181975c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.972+0000 7fcc1659c640 1 -- 172.21.15.116:0/3395983567 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc181a68f0 con 0x7fcc18100a80 2024-09-12T03:16:42.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.972+0000 7fcc1659c640 1 --2- 172.21.15.116:0/3395983567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc18100a80 0x7fcc18197080 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800d770 tx=0x7fcc0800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:42.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.973+0000 7fcc077fe640 1 -- 172.21.15.116:0/3395983567 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc08017070 con 0x7fcc18100a80 2024-09-12T03:16:42.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.973+0000 7fcc077fe640 1 -- 172.21.15.116:0/3395983567 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc08005500 con 0x7fcc18100a80 2024-09-12T03:16:42.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.973+0000 7fcc1cff9640 1 -- 172.21.15.116:0/3395983567 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc181a6be0 con 0x7fcc18100a80 2024-09-12T03:16:42.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.973+0000 7fcc077fe640 1 -- 172.21.15.116:0/3395983567 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc08004e00 con 0x7fcc18100a80 2024-09-12T03:16:42.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.973+0000 7fcc1cff9640 1 -- 172.21.15.116:0/3395983567 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc181a70d0 con 0x7fcc18100a80 2024-09-12T03:16:42.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.975+0000 7fcc077fe640 1 -- 172.21.15.116:0/3395983567 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fcc08004fa0 con 0x7fcc18100a80 2024-09-12T03:16:42.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.975+0000 7fcc1f3d2740 1 Processor -- start 2024-09-12T03:16:42.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.975+0000 7fcc1f3d2740 1 -- start start 2024-09-12T03:16:42.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.976+0000 7fcc1f3d2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8796deb40 0x55f8796dd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.976+0000 7fcc1f3d2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8796dd5d0 0x55f8796dda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.976+0000 7fcc1f3d2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8796e8370 0x55f8796ea760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.976+0000 7fcc1f3d2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55f8796432f0 con 0x55f8796deb40 2024-09-12T03:16:42.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.976+0000 7fcc1f3d2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f8795eacf0 con 0x55f8796e8370 2024-09-12T03:16:42.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.976+0000 7fcc1f3d2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f87963f2b0 con 0x55f8796dd5d0 2024-09-12T03:16:42.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.976+0000 7fcc077fe640 1 --2- 172.21.15.116:0/3395983567 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcc000a2b80 0x7fcc000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.976+0000 7fcc16d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8796deb40 0x55f8796dd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.976+0000 7fcc15d9b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8796e8370 0x55f8796ea760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.976+0000 7fcc16d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8796deb40 0x55f8796dd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41930/0 (socket says 172.21.15.116:41930) 2024-09-12T03:16:42.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.976+0000 7fcc16d9d640 1 -- 172.21.15.116:0/3893343969 learned_addr learned my addr 172.21.15.116:0/3893343969 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:42.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.976+0000 7fcc077fe640 1 -- 172.21.15.116:0/3395983567 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fcc080cea90 con 0x7fcc18100a80 2024-09-12T03:16:42.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.976+0000 7fcc1659c640 1 --2- 172.21.15.116:0/3893343969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8796dd5d0 0x55f8796dda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.976+0000 7fcc16d9d640 1 --2- 172.21.15.116:0/3395983567 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcc000a2b80 0x7fcc000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.976+0000 7fcc15d9b640 1 -- 172.21.15.116:0/3893343969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8796dd5d0 msgr2=0x55f8796dda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:42.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.976+0000 7fcc15d9b640 1 --2- 172.21.15.116:0/3893343969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8796dd5d0 0x55f8796dda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.976+0000 7fcc15d9b640 1 -- 172.21.15.116:0/3893343969 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8796deb40 msgr2=0x55f8796dd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:42.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.976+0000 7fcc15d9b640 1 --2- 172.21.15.116:0/3893343969 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8796deb40 0x55f8796dd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.976+0000 7fcc15d9b640 1 -- 172.21.15.116:0/3893343969 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8795cf3a0 con 0x55f8796e8370 2024-09-12T03:16:42.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.976+0000 7fcc16d9d640 1 --2- 172.21.15.116:0/3893343969 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8796deb40 0x55f8796dd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:42.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.977+0000 7fcc15d9b640 1 --2- 172.21.15.116:0/3893343969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8796e8370 0x55f8796ea760 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c009a90 tx=0x7fcc0c009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75ac5a8a3628cc7c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:42.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.977+0000 7fcc16d9d640 1 --2- 172.21.15.116:0/3395983567 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcc000a2b80 0x7fcc000a5040 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fcc10005fe0 tx=0x7fcc1002a8e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:42.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.977+0000 7fcc04ff9640 1 -- 172.21.15.116:0/3893343969 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c016070 con 0x55f8796e8370 2024-09-12T03:16:42.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.977+0000 7fcc04ff9640 1 -- 172.21.15.116:0/3893343969 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc0c005500 con 0x55f8796e8370 2024-09-12T03:16:42.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.977+0000 7fcc04ff9640 1 -- 172.21.15.116:0/3893343969 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c011470 con 0x55f8796e8370 2024-09-12T03:16:42.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.978+0000 7fcc1f3d2740 1 -- 172.21.15.116:0/3893343969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8796e8370 msgr2=0x55f8796ea760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.978+0000 7fcc1f3d2740 1 --2- 172.21.15.116:0/3893343969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8796e8370 0x55f8796ea760 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c009a90 tx=0x7fcc0c009f50 comp rx=0 tx=0).stop 2024-09-12T03:16:42.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.978+0000 7fcc1f3d2740 1 -- 172.21.15.116:0/3893343969 shutdown_connections 2024-09-12T03:16:42.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.978+0000 7fcc1f3d2740 1 --2- 172.21.15.116:0/3893343969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8796e8370 0x55f8796ea760 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.978+0000 7fcc1f3d2740 1 --2- 172.21.15.116:0/3893343969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8796dd5d0 0x55f8796dda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.978+0000 7fcc1f3d2740 1 --2- 172.21.15.116:0/3893343969 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8796deb40 0x55f8796dd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.978+0000 7fcc1f3d2740 1 -- 172.21.15.116:0/3893343969 >> 172.21.15.116:0/3893343969 conn(0x55f8796a6310 msgr2=0x55f8796d9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:42.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.978+0000 7fcc1f3d2740 1 -- 172.21.15.116:0/3893343969 shutdown_connections 2024-09-12T03:16:42.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.978+0000 7fcc1f3d2740 1 -- 172.21.15.116:0/3893343969 wait complete. 2024-09-12T03:16:42.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.978+0000 7fcc1f3d2740 1 -- start start 2024-09-12T03:16:42.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.978+0000 7fcc1f3d2740 1 Processor -- start 2024-09-12T03:16:42.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.979+0000 7fcc1f3d2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8796dd5d0 0x55f8796e7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.979+0000 7fcc1f3d2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8796deb40 0x55f8796e4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.979+0000 7fcc1f3d2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8796e8370 0x55f8796e5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.979+0000 7fcc1f3d2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55f8796eb070 con 0x55f8796dd5d0 2024-09-12T03:16:42.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.979+0000 7fcc1f3d2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f8795fe060 con 0x55f8796deb40 2024-09-12T03:16:42.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.979+0000 7fcc1f3d2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f8796eb1f0 con 0x55f8796e8370 2024-09-12T03:16:42.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.979+0000 7fcc1659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8796dd5d0 0x55f8796e7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.979+0000 7fcc1f3d2740 10 client.0 _ll_register_callbacks cb 0x55f8793518c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:42.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.979+0000 7fcc1659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8796dd5d0 0x55f8796e7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41936/0 (socket says 172.21.15.116:41936) 2024-09-12T03:16:42.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.979+0000 7fcc1659c640 1 -- 172.21.15.116:0/3866922080 learned_addr learned my addr 172.21.15.116:0/3866922080 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:42.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.979+0000 7fcc15d9b640 1 --2- 172.21.15.116:0/3866922080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8796deb40 0x55f8796e4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.979+0000 7fcc16d9d640 1 --2- 172.21.15.116:0/3866922080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8796e8370 0x55f8796e5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.979+0000 7fcc1659c640 1 -- 172.21.15.116:0/3866922080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8796e8370 msgr2=0x55f8796e5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:42.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.979+0000 7fcc1659c640 1 --2- 172.21.15.116:0/3866922080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8796e8370 0x55f8796e5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.979+0000 7fcc1659c640 1 -- 172.21.15.116:0/3866922080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8796deb40 msgr2=0x55f8796e4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:42.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.979+0000 7fcc1659c640 1 --2- 172.21.15.116:0/3866922080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8796deb40 0x55f8796e4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.979+0000 7fcc1659c640 1 -- 172.21.15.116:0/3866922080 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8793b43c0 con 0x55f8796dd5d0 2024-09-12T03:16:42.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.979+0000 7fcc15d9b640 1 --2- 172.21.15.116:0/3866922080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8796deb40 0x55f8796e4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:42.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.979+0000 7fcc1659c640 1 --2- 172.21.15.116:0/3866922080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8796dd5d0 0x55f8796e7c70 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fcc080cc5c0 tx=0x7fcc0809adb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf4c46e5f49290c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:42.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.979+0000 7fcbfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:16:42.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.979+0000 7fcbfdffb640 1 -- 172.21.15.116:0/3866922080 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc08020040 con 0x55f8796dd5d0 2024-09-12T03:16:42.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.979+0000 7fcbfdffb640 1 -- 172.21.15.116:0/3866922080 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fcc08009cc0 con 0x55f8796dd5d0 2024-09-12T03:16:42.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.979+0000 7fcc1f3d2740 1 -- 172.21.15.116:0/3866922080 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc0c005500 con 0x55f8796dd5d0 2024-09-12T03:16:42.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.979+0000 7fcbfdffb640 1 -- 172.21.15.116:0/3866922080 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcc0809b320 con 0x55f8796dd5d0 2024-09-12T03:16:42.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.979+0000 7fcc1f3d2740 10 client.19074 fetch_fsmap learned FSMap version 25 2024-09-12T03:16:42.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.979+0000 7fcc1f3d2740 1 -- 172.21.15.116:0/3866922080 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f8796f6a20 con 0x55f8796dd5d0 2024-09-12T03:16:42.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.980+0000 7fcbfdffb640 1 -- 172.21.15.116:0/3866922080 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc0c005500 con 0x55f8796dd5d0 2024-09-12T03:16:42.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.980+0000 7fcbfdffb640 1 -- 172.21.15.116:0/3866922080 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcc0809a770 con 0x55f8796dd5d0 2024-09-12T03:16:42.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.980+0000 7fcc1f3d2740 10 client.19074 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:16:42.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.980+0000 7fcc1f3d2740 10 client.19074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:16:42.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.980+0000 7fcc1f3d2740 20 client.19074 populate_metadata read hostname 'smithi116' 2024-09-12T03:16:42.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.980+0000 7fcc1f3d2740 1 --2- 172.21.15.116:0/3866922080 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f8796f7ee0 0x55f8796fa370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.980+0000 7fcc1f3d2740 4 client.19074 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:16:42.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.980+0000 7fcc1f3d2740 1 -- 172.21.15.116:0/3866922080 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f879650860 con 0x55f8796f7ee0 2024-09-12T03:16:42.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.980+0000 7fcc15d9b640 1 --2- 172.21.15.116:0/3866922080 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f8796f7ee0 0x55f8796fa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.980+0000 7fcc15d9b640 1 --2- 172.21.15.116:0/3866922080 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f8796f7ee0 0x55f8796fa370 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfa49cba21cb8f2e server_cookie=5c68d8545d03f05 in_seq=0 out_seq=0 2024-09-12T03:16:42.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.980+0000 7fcbfdffb640 10 client.19074 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:42.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.981+0000 7fcbfdffb640 1 -- 172.21.15.116:0/3866922080 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f879650860 con 0x55f8796f7ee0 2024-09-12T03:16:42.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.981+0000 7fcbfdffb640 10 client.19074 handle_command_reply: tid=0 2024-09-12T03:16:42.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.981+0000 7fcc1f3d2740 1 client.19074 shutdown 2024-09-12T03:16:42.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.981+0000 7fcc1f3d2740 1 -- 172.21.15.116:0/3866922080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8796dd5d0 msgr2=0x55f8796e7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.981+0000 7fcc1f3d2740 1 --2- 172.21.15.116:0/3866922080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8796dd5d0 0x55f8796e7c70 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fcc080cc5c0 tx=0x7fcc0809adb0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.981+0000 7fcc1f3d2740 1 -- 172.21.15.116:0/3866922080 shutdown_connections 2024-09-12T03:16:42.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.981+0000 7fcc1f3d2740 1 --2- 172.21.15.116:0/3866922080 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f8796f7ee0 0x55f8796fa370 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.981+0000 7fcbfdffb640 0 client.19074 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:42.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.981+0000 7fcc1f3d2740 1 --2- 172.21.15.116:0/3866922080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8796e8370 0x55f8796e5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.981+0000 7fcc1f3d2740 1 --2- 172.21.15.116:0/3866922080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8796deb40 0x55f8796e4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.981+0000 7fcc1f3d2740 1 --2- 172.21.15.116:0/3866922080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8796dd5d0 0x55f8796e7c70 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.981+0000 7fcc1f3d2740 1 -- 172.21.15.116:0/3866922080 >> 172.21.15.116:0/3866922080 conn(0x55f8796a6310 msgr2=0x55f8796df960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:42.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.981+0000 7fcc1f3d2740 1 -- 172.21.15.116:0/3866922080 shutdown_connections 2024-09-12T03:16:42.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.981+0000 7fcc1f3d2740 1 -- 172.21.15.116:0/3866922080 wait complete. 2024-09-12T03:16:42.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.982+0000 7fcc1f3d2740 20 client.19074 trim_cache size 0 max 16384 2024-09-12T03:16:42.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.995+0000 7fcc1f3d2740 1 Processor -- start 2024-09-12T03:16:42.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.995+0000 7fcc1f3d2740 1 -- start start 2024-09-12T03:16:42.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.995+0000 7fcc1f3d2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8796dd5d0 0x55f8796f4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.995+0000 7fcc1f3d2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8796deb40 0x55f8796e0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.995+0000 7fcc1f3d2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8796e8370 0x55f8796ffbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.995+0000 7fcc1f3d2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcc181b1f40 con 0x55f8796dd5d0 2024-09-12T03:16:42.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.995+0000 7fcc1f3d2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcc181af810 con 0x55f8796e8370 2024-09-12T03:16:42.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.995+0000 7fcc1f3d2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcc181ad660 con 0x55f8796deb40 2024-09-12T03:16:42.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.995+0000 7fcc1659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8796dd5d0 0x55f8796f4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.995+0000 7fcc1659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8796dd5d0 0x55f8796f4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41944/0 (socket says 172.21.15.116:41944) 2024-09-12T03:16:42.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.995+0000 7fcc1659c640 1 -- 172.21.15.116:0/1543289712 learned_addr learned my addr 172.21.15.116:0/1543289712 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:42.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.995+0000 7fcc15d9b640 1 --2- 172.21.15.116:0/1543289712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8796deb40 0x55f8796e0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.995+0000 7fcc1659c640 1 -- 172.21.15.116:0/1543289712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8796deb40 msgr2=0x55f8796e0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:42.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.995+0000 7fcc1659c640 1 --2- 172.21.15.116:0/1543289712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8796deb40 0x55f8796e0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.995+0000 7fcc1659c640 1 -- 172.21.15.116:0/1543289712 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8796e8370 msgr2=0x55f8796ffbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:42.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.995+0000 7fcc1659c640 1 --2- 172.21.15.116:0/1543289712 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8796e8370 0x55f8796ffbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.995+0000 7fcc1659c640 1 -- 172.21.15.116:0/1543289712 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8796e3470 con 0x55f8796dd5d0 2024-09-12T03:16:42.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.995+0000 7fcc15d9b640 1 --2- 172.21.15.116:0/1543289712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8796deb40 0x55f8796e0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:42.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.996+0000 7fcc1659c640 1 --2- 172.21.15.116:0/1543289712 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8796dd5d0 0x55f8796f4310 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fcc08009610 tx=0x7fcc08097650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=909f596a58d3b82d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:42.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.996+0000 7fcbfffff640 1 -- 172.21.15.116:0/1543289712 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc08020070 con 0x55f8796dd5d0 2024-09-12T03:16:42.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.996+0000 7fcbfffff640 1 -- 172.21.15.116:0/1543289712 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc0809f6d0 con 0x55f8796dd5d0 2024-09-12T03:16:42.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.996+0000 7fcbfffff640 1 -- 172.21.15.116:0/1543289712 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc08009cc0 con 0x55f8796dd5d0 2024-09-12T03:16:42.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.996+0000 7fcc1f3d2740 1 -- 172.21.15.116:0/1543289712 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8796dd5d0 msgr2=0x55f8796f4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.996+0000 7fcc1f3d2740 1 --2- 172.21.15.116:0/1543289712 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8796dd5d0 0x55f8796f4310 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fcc08009610 tx=0x7fcc08097650 comp rx=0 tx=0).stop 2024-09-12T03:16:42.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.996+0000 7fcc1f3d2740 1 -- 172.21.15.116:0/1543289712 shutdown_connections 2024-09-12T03:16:42.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.996+0000 7fcc1f3d2740 1 --2- 172.21.15.116:0/1543289712 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8796e8370 0x55f8796ffbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.996+0000 7fcc1f3d2740 1 --2- 172.21.15.116:0/1543289712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8796deb40 0x55f8796e0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.996+0000 7fcc1f3d2740 1 --2- 172.21.15.116:0/1543289712 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8796dd5d0 0x55f8796f4310 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.996+0000 7fcc1f3d2740 1 -- 172.21.15.116:0/1543289712 >> 172.21.15.116:0/1543289712 conn(0x55f8796a6310 msgr2=0x55f8796e37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:42.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.996+0000 7fcc1f3d2740 1 -- 172.21.15.116:0/1543289712 shutdown_connections 2024-09-12T03:16:42.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.996+0000 7fcc1f3d2740 1 -- 172.21.15.116:0/1543289712 wait complete. 2024-09-12T03:16:42.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.997+0000 7fcc1f3d2740 1 -- start start 2024-09-12T03:16:42.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.997+0000 7fcc1f3d2740 1 Processor -- start 2024-09-12T03:16:42.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.997+0000 7fcc1f3d2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8796dd5d0 0x55f8796f0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.997+0000 7fcc1f3d2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8796deb40 0x55f8796e1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.997+0000 7fcc1f3d2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8796e8370 0x55f8796e2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.997+0000 7fcc1f3d2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcc181adcd0 con 0x55f8796e8370 2024-09-12T03:16:42.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.997+0000 7fcc1f3d2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcc181af140 con 0x55f8796dd5d0 2024-09-12T03:16:42.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.997+0000 7fcc1f3d2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcc181ae3a0 con 0x55f8796deb40 2024-09-12T03:16:42.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.997+0000 7fcc1f3d2740 10 client.0 _ll_register_callbacks cb 0x7fcc0800d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:42.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.997+0000 7fcc16d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8796e8370 0x55f8796e2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.997+0000 7fcc16d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8796e8370 0x55f8796e2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41946/0 (socket says 172.21.15.116:41946) 2024-09-12T03:16:42.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.997+0000 7fcc16d9d640 1 -- 172.21.15.116:0/2739313262 learned_addr learned my addr 172.21.15.116:0/2739313262 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:42.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.997+0000 7fcc1659c640 1 --2- 172.21.15.116:0/2739313262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8796dd5d0 0x55f8796f0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.997+0000 7fcc15d9b640 1 --2- 172.21.15.116:0/2739313262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8796deb40 0x55f8796e1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.997+0000 7fcc16d9d640 1 -- 172.21.15.116:0/2739313262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8796deb40 msgr2=0x55f8796e1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:42.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.997+0000 7fcc16d9d640 1 --2- 172.21.15.116:0/2739313262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8796deb40 0x55f8796e1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.997+0000 7fcc16d9d640 1 -- 172.21.15.116:0/2739313262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8796dd5d0 msgr2=0x55f8796f0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:42.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.997+0000 7fcc16d9d640 1 --2- 172.21.15.116:0/2739313262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8796dd5d0 0x55f8796f0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.997+0000 7fcc16d9d640 1 -- 172.21.15.116:0/2739313262 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8796e2960 con 0x55f8796e8370 2024-09-12T03:16:42.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.998+0000 7fcc16d9d640 1 --2- 172.21.15.116:0/2739313262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8796e8370 0x55f8796e2150 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fcc10005e30 tx=0x7fcc10009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1322ced05819a2b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:42.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.998+0000 7fcbfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:16:42.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.999+0000 7fcbfdffb640 1 -- 172.21.15.116:0/2739313262 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc100305a0 con 0x55f8796e8370 2024-09-12T03:16:42.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.999+0000 7fcbfdffb640 1 -- 172.21.15.116:0/2739313262 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fcc10030740 con 0x55f8796e8370 2024-09-12T03:16:42.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.999+0000 7fcc1f3d2740 1 -- 172.21.15.116:0/2739313262 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc0809f6d0 con 0x55f8796e8370 2024-09-12T03:16:42.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.999+0000 7fcbfdffb640 1 -- 172.21.15.116:0/2739313262 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcc10004b40 con 0x55f8796e8370 2024-09-12T03:16:42.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.999+0000 7fcc1f3d2740 10 client.19080 fetch_fsmap learned FSMap version 25 2024-09-12T03:16:42.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.999+0000 7fcc1f3d2740 1 -- 172.21.15.116:0/2739313262 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f879762bd0 con 0x55f8796e8370 2024-09-12T03:16:42.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.999+0000 7fcbfdffb640 1 -- 172.21.15.116:0/2739313262 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc0809f6d0 con 0x55f8796e8370 2024-09-12T03:16:42.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.999+0000 7fcbfdffb640 1 -- 172.21.15.116:0/2739313262 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcc10030070 con 0x55f8796e8370 2024-09-12T03:16:42.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.999+0000 7fcc1f3d2740 10 client.19080 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:16:42.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.999+0000 7fcc1f3d2740 10 client.19080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:16:42.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.999+0000 7fcc1f3d2740 20 client.19080 populate_metadata read hostname 'smithi116' 2024-09-12T03:16:42.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.999+0000 7fcc1f3d2740 1 --2- 172.21.15.116:0/2739313262 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f8796f7ee0 0x55f879765360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:42.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.999+0000 7fcc1f3d2740 4 client.19080 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:16:42.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:41.999+0000 7fcc1f3d2740 1 -- 172.21.15.116:0/2739313262 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f87964ad70 con 0x55f8796f7ee0 2024-09-12T03:16:42.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:42.000+0000 7fcc1659c640 1 --2- 172.21.15.116:0/2739313262 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f8796f7ee0 0x55f879765360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:42.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:42.000+0000 7fcc1659c640 1 --2- 172.21.15.116:0/2739313262 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f8796f7ee0 0x55f879765360 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dd371997241152d server_cookie=4b2f5bf5f7c0cad4 in_seq=0 out_seq=0 2024-09-12T03:16:42.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:42.000+0000 7fcbfdffb640 10 client.19080 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:42.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:42.000+0000 7fcbfdffb640 1 -- 172.21.15.116:0/2739313262 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f87964ad70 con 0x55f8796f7ee0 2024-09-12T03:16:42.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:42.000+0000 7fcbfdffb640 10 client.19080 handle_command_reply: tid=0 2024-09-12T03:16:42.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:42.000+0000 7fcc1f3d2740 1 client.19080 shutdown 2024-09-12T03:16:42.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:42.000+0000 7fcc1f3d2740 1 -- 172.21.15.116:0/2739313262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8796e8370 msgr2=0x55f8796e2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:42.000+0000 7fcc1f3d2740 1 --2- 172.21.15.116:0/2739313262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8796e8370 0x55f8796e2150 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fcc10005e30 tx=0x7fcc10009800 comp rx=0 tx=0).stop 2024-09-12T03:16:42.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:42.000+0000 7fcc1f3d2740 1 -- 172.21.15.116:0/2739313262 shutdown_connections 2024-09-12T03:16:42.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:42.000+0000 7fcc1f3d2740 1 --2- 172.21.15.116:0/2739313262 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f8796f7ee0 0x55f879765360 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:42.000+0000 7fcc1f3d2740 1 --2- 172.21.15.116:0/2739313262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8796e8370 0x55f8796e2150 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:42.001+0000 7fcc1f3d2740 1 --2- 172.21.15.116:0/2739313262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f8796deb40 0x55f8796e1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:42.001+0000 7fcbfdffb640 0 client.19080 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:42.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:42.001+0000 7fcc1f3d2740 1 --2- 172.21.15.116:0/2739313262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8796dd5d0 0x55f8796f0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:42.001+0000 7fcc1f3d2740 1 -- 172.21.15.116:0/2739313262 >> 172.21.15.116:0/2739313262 conn(0x55f8796a6310 msgr2=0x55f8796e3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:42.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:42.001+0000 7fcc1f3d2740 1 -- 172.21.15.116:0/2739313262 shutdown_connections 2024-09-12T03:16:42.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:42.001+0000 7fcc1f3d2740 1 -- 172.21.15.116:0/2739313262 wait complete. 2024-09-12T03:16:42.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:42.001+0000 7fcc1f3d2740 20 client.19080 trim_cache size 0 max 16384 2024-09-12T03:16:42.019 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:16:42.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:42.001+0000 7fcc1cff9640 1 -- 172.21.15.116:0/3395983567 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcc000a2b80 msgr2=0x7fcc000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:42.001+0000 7fcc1cff9640 1 --2- 172.21.15.116:0/3395983567 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcc000a2b80 0x7fcc000a5040 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fcc10005fe0 tx=0x7fcc1002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:42.001+0000 7fcc1cff9640 1 -- 172.21.15.116:0/3395983567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc18100a80 msgr2=0x7fcc18197080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:42.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:42.001+0000 7fcc1cff9640 1 --2- 172.21.15.116:0/3395983567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc18100a80 0x7fcc18197080 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800d770 tx=0x7fcc0800dc30 comp rx=0 tx=0).stop 2024-09-12T03:16:42.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:42.001+0000 7fcc1cff9640 1 -- 172.21.15.116:0/3395983567 shutdown_connections 2024-09-12T03:16:42.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:42.001+0000 7fcc1cff9640 1 --2- 172.21.15.116:0/3395983567 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcc000a2b80 0x7fcc000a5040 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:42.001+0000 7fcc1cff9640 1 --2- 172.21.15.116:0/3395983567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc18106a80 0x7fcc18197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:42.001+0000 7fcc1cff9640 1 --2- 172.21.15.116:0/3395983567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc18101420 0x7fcc181975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:42.001+0000 7fcc1cff9640 1 --2- 172.21.15.116:0/3395983567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc18100a80 0x7fcc18197080 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:42.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:42.001+0000 7fcc1cff9640 1 -- 172.21.15.116:0/3395983567 >> 172.21.15.116:0/3395983567 conn(0x7fcc180fc7a0 msgr2=0x7fcc180fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:42.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:42.002+0000 7fcc1cff9640 1 -- 172.21.15.116:0/3395983567 shutdown_connections 2024-09-12T03:16:42.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:42.002+0000 7fcc1cff9640 1 -- 172.21.15.116:0/3395983567 wait complete. 2024-09-12T03:16:43.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:43 smithi116 ceph-mon[31410]: pgmap v750: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-12T03:16:43.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:43 smithi183 ceph-mon[35535]: pgmap v750: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-12T03:16:43.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:43 smithi177 ceph-mon[35305]: pgmap v750: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-12T03:16:43.790 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:16:43.790 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:16:44.048 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:44.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.512+0000 7f9d980ba640 1 -- 172.21.15.116:0/3470330305 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d90101de0 msgr2=0x7f9d901021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:44.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.512+0000 7f9d980ba640 1 --2- 172.21.15.116:0/3470330305 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d90101de0 0x7f9d901021c0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009900 tx=0x7f9d8c02a410 comp rx=0 tx=0).stop 2024-09-12T03:16:44.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.512+0000 7f9d980ba640 1 -- 172.21.15.116:0/3470330305 shutdown_connections 2024-09-12T03:16:44.533 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.512+0000 7f9d980ba640 1 --2- 172.21.15.116:0/3470330305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d90106f10 0x7f9d9010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.512+0000 7f9d980ba640 1 --2- 172.21.15.116:0/3470330305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d90102790 0x7f9d901067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.512+0000 7f9d980ba640 1 --2- 172.21.15.116:0/3470330305 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d90101de0 0x7f9d901021c0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.512+0000 7f9d980ba640 1 -- 172.21.15.116:0/3470330305 >> 172.21.15.116:0/3470330305 conn(0x7f9d900fd8c0 msgr2=0x7f9d900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:44.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.512+0000 7f9d980ba640 1 -- 172.21.15.116:0/3470330305 shutdown_connections 2024-09-12T03:16:44.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.512+0000 7f9d980ba640 1 -- 172.21.15.116:0/3470330305 wait complete. 2024-09-12T03:16:44.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.513+0000 7f9d980ba640 1 Processor -- start 2024-09-12T03:16:44.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.513+0000 7f9d980ba640 1 -- start start 2024-09-12T03:16:44.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.513+0000 7f9d980ba640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d90101de0 0x7f9d90198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.513+0000 7f9d980ba640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d90102790 0x7f9d90198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.534 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.513+0000 7f9d980ba640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d90106f10 0x7f9d9019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.513+0000 7f9d980ba640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9d90111f30 con 0x7f9d90102790 2024-09-12T03:16:44.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.513+0000 7f9d980ba640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9d90111db0 con 0x7f9d90101de0 2024-09-12T03:16:44.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.513+0000 7f9d980ba640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9d901120b0 con 0x7f9d90106f10 2024-09-12T03:16:44.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.513+0000 7f9d95655640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d90102790 0x7f9d90198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.513+0000 7f9d95655640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d90102790 0x7f9d90198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41966/0 (socket says 172.21.15.116:41966) 2024-09-12T03:16:44.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.513+0000 7f9d95655640 1 -- 172.21.15.116:0/2599081519 learned_addr learned my addr 172.21.15.116:0/2599081519 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:44.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.513+0000 7f9d95e56640 1 --2- 172.21.15.116:0/2599081519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d90101de0 0x7f9d90198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.514+0000 7f9d95655640 1 -- 172.21.15.116:0/2599081519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d90106f10 msgr2=0x7f9d9019cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:44.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.514+0000 7f9d95655640 1 --2- 172.21.15.116:0/2599081519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d90106f10 0x7f9d9019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.514+0000 7f9d95655640 1 -- 172.21.15.116:0/2599081519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d90101de0 msgr2=0x7f9d90198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:44.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.514+0000 7f9d95655640 1 --2- 172.21.15.116:0/2599081519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d90101de0 0x7f9d90198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.514+0000 7f9d95655640 1 -- 172.21.15.116:0/2599081519 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d9019d4c0 con 0x7f9d90102790 2024-09-12T03:16:44.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.514+0000 7f9d95e56640 1 --2- 172.21.15.116:0/2599081519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d90101de0 0x7f9d90198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:44.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.514+0000 7f9d95655640 1 --2- 172.21.15.116:0/2599081519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d90102790 0x7f9d90198a50 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400c890 tx=0x7f9d8400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:44.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.514+0000 7f9d82ffd640 1 -- 172.21.15.116:0/2599081519 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84004220 con 0x7f9d90102790 2024-09-12T03:16:44.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.514+0000 7f9d82ffd640 1 -- 172.21.15.116:0/2599081519 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d840043c0 con 0x7f9d90102790 2024-09-12T03:16:44.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.514+0000 7f9d980ba640 1 -- 172.21.15.116:0/2599081519 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d9019d7b0 con 0x7f9d90102790 2024-09-12T03:16:44.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.514+0000 7f9d82ffd640 1 -- 172.21.15.116:0/2599081519 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d840049e0 con 0x7f9d90102790 2024-09-12T03:16:44.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.514+0000 7f9d980ba640 1 -- 172.21.15.116:0/2599081519 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d901a5090 con 0x7f9d90102790 2024-09-12T03:16:44.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.516+0000 7f9d82ffd640 1 -- 172.21.15.116:0/2599081519 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f9d840079b0 con 0x7f9d90102790 2024-09-12T03:16:44.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.517+0000 7f9d82ffd640 1 --2- 172.21.15.116:0/2599081519 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9d7c0a2b80 0x7f9d7c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.517+0000 7f9d9a493740 1 Processor -- start 2024-09-12T03:16:44.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.517+0000 7f9d95e56640 1 --2- 172.21.15.116:0/2599081519 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9d7c0a2b80 0x7f9d7c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.536 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.517+0000 7f9d82ffd640 1 -- 172.21.15.116:0/2599081519 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f9d840cf120 con 0x7f9d90102790 2024-09-12T03:16:44.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.517+0000 7f9d9a493740 1 -- start start 2024-09-12T03:16:44.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.517+0000 7f9d9a493740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a9a668db60 0x55a9a668df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.517+0000 7f9d9a493740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9a668e480 0x55a9a6698800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.517+0000 7f9d9a493740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a9a6698d40 0x55a9a669b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.517+0000 7f9d9a493740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a9a65f3280 con 0x55a9a6698d40 2024-09-12T03:16:44.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.517+0000 7f9d9a493740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a9a659ab10 con 0x55a9a668db60 2024-09-12T03:16:44.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.517+0000 7f9d9a493740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a9a65ef240 con 0x55a9a668e480 2024-09-12T03:16:44.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.517+0000 7f9d95e56640 1 --2- 172.21.15.116:0/2599081519 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9d7c0a2b80 0x7f9d7c0a5040 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c005ab0 tx=0x7f9d8c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:44.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.517+0000 7f9d96657640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a9a6698d40 0x55a9a669b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.517+0000 7f9d96657640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a9a6698d40 0x55a9a669b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41982/0 (socket says 172.21.15.116:41982) 2024-09-12T03:16:44.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.517+0000 7f9d96657640 1 -- 172.21.15.116:0/3924957602 learned_addr learned my addr 172.21.15.116:0/3924957602 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:44.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.518+0000 7f9d95e56640 1 --2- 172.21.15.116:0/3924957602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a9a668db60 0x55a9a668df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.537 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.518+0000 7f9d95655640 1 --2- 172.21.15.116:0/3924957602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9a668e480 0x55a9a6698800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.518+0000 7f9d96657640 1 -- 172.21.15.116:0/3924957602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9a668e480 msgr2=0x55a9a6698800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:44.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.518+0000 7f9d96657640 1 --2- 172.21.15.116:0/3924957602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9a668e480 0x55a9a6698800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.518+0000 7f9d96657640 1 -- 172.21.15.116:0/3924957602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a9a668db60 msgr2=0x55a9a668df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:44.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.518+0000 7f9d96657640 1 --2- 172.21.15.116:0/3924957602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a9a668db60 0x55a9a668df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.518+0000 7f9d96657640 1 -- 172.21.15.116:0/3924957602 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9a65a71f0 con 0x55a9a6698d40 2024-09-12T03:16:44.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.518+0000 7f9d95655640 1 --2- 172.21.15.116:0/3924957602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9a668e480 0x55a9a6698800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:44.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.518+0000 7f9d96657640 1 --2- 172.21.15.116:0/3924957602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a9a6698d40 0x55a9a669b130 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009d10 tx=0x7f9d8800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9175095cefa1aca1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:44.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.518+0000 7f9d7bfff640 1 -- 172.21.15.116:0/3924957602 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d88016070 con 0x55a9a6698d40 2024-09-12T03:16:44.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.518+0000 7f9d7bfff640 1 -- 172.21.15.116:0/3924957602 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d88005020 con 0x55a9a6698d40 2024-09-12T03:16:44.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.518+0000 7f9d7bfff640 1 -- 172.21.15.116:0/3924957602 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d88011690 con 0x55a9a6698d40 2024-09-12T03:16:44.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.519+0000 7f9d9a493740 1 -- 172.21.15.116:0/3924957602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a9a6698d40 msgr2=0x55a9a669b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:44.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.519+0000 7f9d9a493740 1 --2- 172.21.15.116:0/3924957602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a9a6698d40 0x55a9a669b130 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009d10 tx=0x7f9d8800e6c0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.519+0000 7f9d9a493740 1 -- 172.21.15.116:0/3924957602 shutdown_connections 2024-09-12T03:16:44.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.519+0000 7f9d9a493740 1 --2- 172.21.15.116:0/3924957602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a9a6698d40 0x55a9a669b130 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.519+0000 7f9d9a493740 1 --2- 172.21.15.116:0/3924957602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9a668e480 0x55a9a6698800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.519+0000 7f9d9a493740 1 --2- 172.21.15.116:0/3924957602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a9a668db60 0x55a9a668df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.519+0000 7f9d9a493740 1 -- 172.21.15.116:0/3924957602 >> 172.21.15.116:0/3924957602 conn(0x55a9a668a0f0 msgr2=0x55a9a668a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:44.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.519+0000 7f9d9a493740 1 -- 172.21.15.116:0/3924957602 shutdown_connections 2024-09-12T03:16:44.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.519+0000 7f9d9a493740 1 -- 172.21.15.116:0/3924957602 wait complete. 2024-09-12T03:16:44.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.519+0000 7f9d9a493740 1 -- start start 2024-09-12T03:16:44.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.519+0000 7f9d9a493740 1 Processor -- start 2024-09-12T03:16:44.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.520+0000 7f9d9a493740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a9a668db60 0x55a9a6695b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.520+0000 7f9d9a493740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9a668e480 0x55a9a66960a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.520+0000 7f9d9a493740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a9a6698d40 0x55a9a66965e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.520+0000 7f9d9a493740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a9a669b910 con 0x55a9a668db60 2024-09-12T03:16:44.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.520+0000 7f9d9a493740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a9a65adf10 con 0x55a9a6698d40 2024-09-12T03:16:44.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.520+0000 7f9d9a493740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a9a669ba90 con 0x55a9a668e480 2024-09-12T03:16:44.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.520+0000 7f9d9a493740 10 client.0 _ll_register_callbacks cb 0x55a9a6576a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:44.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.520+0000 7f9d95e56640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a9a668db60 0x55a9a6695b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.520+0000 7f9d95e56640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a9a668db60 0x55a9a6695b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41996/0 (socket says 172.21.15.116:41996) 2024-09-12T03:16:44.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.520+0000 7f9d95e56640 1 -- 172.21.15.116:0/2420160735 learned_addr learned my addr 172.21.15.116:0/2420160735 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:44.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.520+0000 7f9d96657640 1 --2- 172.21.15.116:0/2420160735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a9a6698d40 0x55a9a66965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.520+0000 7f9d95655640 1 --2- 172.21.15.116:0/2420160735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9a668e480 0x55a9a66960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.520+0000 7f9d95e56640 1 -- 172.21.15.116:0/2420160735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9a668e480 msgr2=0x55a9a66960a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:44.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.520+0000 7f9d95e56640 1 --2- 172.21.15.116:0/2420160735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9a668e480 0x55a9a66960a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.520+0000 7f9d95e56640 1 -- 172.21.15.116:0/2420160735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a9a6698d40 msgr2=0x55a9a66965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:44.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.520+0000 7f9d95e56640 1 --2- 172.21.15.116:0/2420160735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a9a6698d40 0x55a9a66965e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.520+0000 7f9d95e56640 1 -- 172.21.15.116:0/2420160735 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9a63646a0 con 0x55a9a668db60 2024-09-12T03:16:44.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.520+0000 7f9d95655640 1 --2- 172.21.15.116:0/2420160735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9a668e480 0x55a9a66960a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:44.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.520+0000 7f9d95e56640 1 --2- 172.21.15.116:0/2420160735 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a9a668db60 0x55a9a6695b60 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009900 tx=0x7f9d8c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95cab6c6085d45f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:44.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.520+0000 7f9d797fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:16:44.541 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.521+0000 7f9d797fa640 1 -- 172.21.15.116:0/2420160735 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c035050 con 0x55a9a668db60 2024-09-12T03:16:44.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.521+0000 7f9d797fa640 1 -- 172.21.15.116:0/2420160735 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f9d8c005450 con 0x55a9a668db60 2024-09-12T03:16:44.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.521+0000 7f9d9a493740 1 -- 172.21.15.116:0/2420160735 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d88005020 con 0x55a9a668db60 2024-09-12T03:16:44.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.521+0000 7f9d797fa640 1 -- 172.21.15.116:0/2420160735 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9d8c058070 con 0x55a9a668db60 2024-09-12T03:16:44.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.521+0000 7f9d9a493740 10 client.19092 fetch_fsmap learned FSMap version 25 2024-09-12T03:16:44.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.521+0000 7f9d9a493740 1 -- 172.21.15.116:0/2420160735 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a9a66a7460 con 0x55a9a668db60 2024-09-12T03:16:44.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.521+0000 7f9d797fa640 1 -- 172.21.15.116:0/2420160735 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d88005020 con 0x55a9a668db60 2024-09-12T03:16:44.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.521+0000 7f9d797fa640 1 -- 172.21.15.116:0/2420160735 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9d8c003780 con 0x55a9a668db60 2024-09-12T03:16:44.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.521+0000 7f9d9a493740 10 client.19092 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:16:44.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.521+0000 7f9d9a493740 10 client.19092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:16:44.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.521+0000 7f9d9a493740 20 client.19092 populate_metadata read hostname 'smithi116' 2024-09-12T03:16:44.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.521+0000 7f9d9a493740 1 --2- 172.21.15.116:0/2420160735 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a9a66a8920 0x55a9a66aad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.521+0000 7f9d9a493740 4 client.19092 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:16:44.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.521+0000 7f9d9a493740 1 -- 172.21.15.116:0/2420160735 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9a66007f0 con 0x55a9a66a8920 2024-09-12T03:16:44.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.521+0000 7f9d95655640 1 --2- 172.21.15.116:0/2420160735 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a9a66a8920 0x55a9a66aad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.522+0000 7f9d797fa640 10 client.19092 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:44.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.522+0000 7f9d95655640 1 --2- 172.21.15.116:0/2420160735 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a9a66a8920 0x55a9a66aad50 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d682fdf689d5f947 server_cookie=c406fd68bdb51345 in_seq=0 out_seq=0 2024-09-12T03:16:44.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.523+0000 7f9d797fa640 1 -- 172.21.15.116:0/2420160735 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a9a66007f0 con 0x55a9a66a8920 2024-09-12T03:16:44.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.523+0000 7f9d797fa640 10 client.19092 handle_command_reply: tid=0 2024-09-12T03:16:44.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.523+0000 7f9d9a493740 1 client.19092 shutdown 2024-09-12T03:16:44.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.523+0000 7f9d9a493740 1 -- 172.21.15.116:0/2420160735 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a9a668db60 msgr2=0x55a9a6695b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:44.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.523+0000 7f9d9a493740 1 --2- 172.21.15.116:0/2420160735 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a9a668db60 0x55a9a6695b60 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009900 tx=0x7f9d8c009260 comp rx=0 tx=0).stop 2024-09-12T03:16:44.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.523+0000 7f9d9a493740 1 -- 172.21.15.116:0/2420160735 shutdown_connections 2024-09-12T03:16:44.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.523+0000 7f9d9a493740 1 --2- 172.21.15.116:0/2420160735 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a9a66a8920 0x55a9a66aad50 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.523+0000 7f9d9a493740 1 --2- 172.21.15.116:0/2420160735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a9a6698d40 0x55a9a66965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.523+0000 7f9d9a493740 1 --2- 172.21.15.116:0/2420160735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9a668e480 0x55a9a66960a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.523+0000 7f9d797fa640 0 client.19092 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:44.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.523+0000 7f9d9a493740 1 --2- 172.21.15.116:0/2420160735 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a9a668db60 0x55a9a6695b60 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.523+0000 7f9d9a493740 1 -- 172.21.15.116:0/2420160735 >> 172.21.15.116:0/2420160735 conn(0x55a9a668a0f0 msgr2=0x55a9a6689270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:44.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.523+0000 7f9d9a493740 1 -- 172.21.15.116:0/2420160735 shutdown_connections 2024-09-12T03:16:44.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.523+0000 7f9d9a493740 1 -- 172.21.15.116:0/2420160735 wait complete. 2024-09-12T03:16:44.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.523+0000 7f9d9a493740 20 client.19092 trim_cache size 0 max 16384 2024-09-12T03:16:44.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.540+0000 7f9d9a493740 1 Processor -- start 2024-09-12T03:16:44.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.540+0000 7f9d9a493740 1 -- start start 2024-09-12T03:16:44.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.540+0000 7f9d9a493740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9a668db60 0x55a9a66905c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.540+0000 7f9d9a493740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a9a668e480 0x55a9a66b0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.540+0000 7f9d9a493740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a9a6698d40 0x55a9a66b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.540+0000 7f9d9a493740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9d901afe00 con 0x55a9a668e480 2024-09-12T03:16:44.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.540+0000 7f9d9a493740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9d901ad6d0 con 0x55a9a6698d40 2024-09-12T03:16:44.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.540+0000 7f9d9a493740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9d901ab520 con 0x55a9a668db60 2024-09-12T03:16:44.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.540+0000 7f9d95655640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a9a668e480 0x55a9a66b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.540+0000 7f9d95655640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a9a668e480 0x55a9a66b0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42000/0 (socket says 172.21.15.116:42000) 2024-09-12T03:16:44.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.540+0000 7f9d95655640 1 -- 172.21.15.116:0/3482756525 learned_addr learned my addr 172.21.15.116:0/3482756525 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:44.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.540+0000 7f9d95e56640 1 --2- 172.21.15.116:0/3482756525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9a668db60 0x55a9a66905c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.540+0000 7f9d96657640 1 --2- 172.21.15.116:0/3482756525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a9a6698d40 0x55a9a66b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.540+0000 7f9d95655640 1 -- 172.21.15.116:0/3482756525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9a668db60 msgr2=0x55a9a66905c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:44.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.540+0000 7f9d95655640 1 --2- 172.21.15.116:0/3482756525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9a668db60 0x55a9a66905c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.540+0000 7f9d95655640 1 -- 172.21.15.116:0/3482756525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a9a6698d40 msgr2=0x55a9a66b0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:44.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.540+0000 7f9d95655640 1 --2- 172.21.15.116:0/3482756525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a9a6698d40 0x55a9a66b0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.540+0000 7f9d95655640 1 -- 172.21.15.116:0/3482756525 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9a6689f20 con 0x55a9a668e480 2024-09-12T03:16:44.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.540+0000 7f9d95e56640 1 --2- 172.21.15.116:0/3482756525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9a668db60 0x55a9a66905c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:44.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.541+0000 7f9d95655640 1 --2- 172.21.15.116:0/3482756525 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a9a668e480 0x55a9a66b0620 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f9d8409b940 tx=0x7f9d8401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77bbbcefae293f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:44.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.541+0000 7f9d7b7fe640 1 -- 172.21.15.116:0/3482756525 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d840b9070 con 0x55a9a668e480 2024-09-12T03:16:44.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.541+0000 7f9d7b7fe640 1 -- 172.21.15.116:0/3482756525 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d84099a50 con 0x55a9a668e480 2024-09-12T03:16:44.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.541+0000 7f9d7b7fe640 1 -- 172.21.15.116:0/3482756525 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d840b3070 con 0x55a9a668e480 2024-09-12T03:16:44.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.541+0000 7f9d9a493740 1 -- 172.21.15.116:0/3482756525 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a9a668e480 msgr2=0x55a9a66b0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:44.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.541+0000 7f9d9a493740 1 --2- 172.21.15.116:0/3482756525 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a9a668e480 0x55a9a66b0620 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f9d8409b940 tx=0x7f9d8401a040 comp rx=0 tx=0).stop 2024-09-12T03:16:44.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.541+0000 7f9d9a493740 1 -- 172.21.15.116:0/3482756525 shutdown_connections 2024-09-12T03:16:44.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.541+0000 7f9d9a493740 1 --2- 172.21.15.116:0/3482756525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a9a6698d40 0x55a9a66b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.541+0000 7f9d9a493740 1 --2- 172.21.15.116:0/3482756525 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a9a668e480 0x55a9a66b0620 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.541+0000 7f9d9a493740 1 --2- 172.21.15.116:0/3482756525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9a668db60 0x55a9a66905c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.541+0000 7f9d9a493740 1 -- 172.21.15.116:0/3482756525 >> 172.21.15.116:0/3482756525 conn(0x55a9a668a0f0 msgr2=0x55a9a6697ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:44.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.541+0000 7f9d9a493740 1 -- 172.21.15.116:0/3482756525 shutdown_connections 2024-09-12T03:16:44.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.541+0000 7f9d9a493740 1 -- 172.21.15.116:0/3482756525 wait complete. 2024-09-12T03:16:44.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.542+0000 7f9d9a493740 1 -- start start 2024-09-12T03:16:44.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.542+0000 7f9d9a493740 1 Processor -- start 2024-09-12T03:16:44.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.542+0000 7f9d9a493740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a9a668db60 0x55a9a66a0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.542+0000 7f9d9a493740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9a668e480 0x55a9a66a1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.542+0000 7f9d9a493740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a9a6698d40 0x55a9a669dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.542+0000 7f9d9a493740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9d901abb90 con 0x55a9a6698d40 2024-09-12T03:16:44.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.542+0000 7f9d9a493740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9d901ad000 con 0x55a9a668db60 2024-09-12T03:16:44.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.542+0000 7f9d9a493740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9d901ac260 con 0x55a9a668e480 2024-09-12T03:16:44.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.542+0000 7f9d9a493740 10 client.0 _ll_register_callbacks cb 0x7f9d8c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:44.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.542+0000 7f9d96657640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a9a6698d40 0x55a9a669dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.542+0000 7f9d96657640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a9a6698d40 0x55a9a669dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42008/0 (socket says 172.21.15.116:42008) 2024-09-12T03:16:44.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.542+0000 7f9d96657640 1 -- 172.21.15.116:0/1954729905 learned_addr learned my addr 172.21.15.116:0/1954729905 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:44.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.542+0000 7f9d95e56640 1 --2- 172.21.15.116:0/1954729905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a9a668db60 0x55a9a66a0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.543+0000 7f9d95655640 1 --2- 172.21.15.116:0/1954729905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9a668e480 0x55a9a66a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.543+0000 7f9d96657640 1 -- 172.21.15.116:0/1954729905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9a668e480 msgr2=0x55a9a66a1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:44.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.543+0000 7f9d96657640 1 --2- 172.21.15.116:0/1954729905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9a668e480 0x55a9a66a1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.543+0000 7f9d96657640 1 -- 172.21.15.116:0/1954729905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a9a668db60 msgr2=0x55a9a66a0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:44.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.543+0000 7f9d96657640 1 --2- 172.21.15.116:0/1954729905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a9a668db60 0x55a9a66a0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.543+0000 7f9d96657640 1 -- 172.21.15.116:0/1954729905 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9a669e4c0 con 0x55a9a6698d40 2024-09-12T03:16:44.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.543+0000 7f9d96657640 1 --2- 172.21.15.116:0/1954729905 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a9a6698d40 0x55a9a669dcb0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f9d8800ebc0 tx=0x7f9d88009c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c2a7efe8366b66 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:44.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.543+0000 7f9d797fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:16:44.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.543+0000 7f9d797fa640 1 -- 172.21.15.116:0/1954729905 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d88016040 con 0x55a9a6698d40 2024-09-12T03:16:44.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.543+0000 7f9d797fa640 1 -- 172.21.15.116:0/1954729905 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f9d88004240 con 0x55a9a6698d40 2024-09-12T03:16:44.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.543+0000 7f9d9a493740 1 -- 172.21.15.116:0/1954729905 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d84099a50 con 0x55a9a6698d40 2024-09-12T03:16:44.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.543+0000 7f9d797fa640 1 -- 172.21.15.116:0/1954729905 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9d88003e40 con 0x55a9a6698d40 2024-09-12T03:16:44.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.543+0000 7f9d9a493740 10 client.19098 fetch_fsmap learned FSMap version 25 2024-09-12T03:16:44.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.543+0000 7f9d9a493740 1 -- 172.21.15.116:0/1954729905 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a9a6718550 con 0x55a9a6698d40 2024-09-12T03:16:44.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.543+0000 7f9d797fa640 1 -- 172.21.15.116:0/1954729905 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d84099a50 con 0x55a9a6698d40 2024-09-12T03:16:44.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.544+0000 7f9d797fa640 1 -- 172.21.15.116:0/1954729905 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9d8800f040 con 0x55a9a6698d40 2024-09-12T03:16:44.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.544+0000 7f9d9a493740 10 client.19098 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:16:44.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.544+0000 7f9d9a493740 10 client.19098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:16:44.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.544+0000 7f9d9a493740 20 client.19098 populate_metadata read hostname 'smithi116' 2024-09-12T03:16:44.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.544+0000 7f9d9a493740 1 --2- 172.21.15.116:0/1954729905 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a9a6718aa0 0x55a9a671aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:44.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.544+0000 7f9d9a493740 4 client.19098 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:16:44.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.544+0000 7f9d9a493740 1 -- 172.21.15.116:0/1954729905 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a9a65fad00 con 0x55a9a6718aa0 2024-09-12T03:16:44.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.544+0000 7f9d95e56640 1 --2- 172.21.15.116:0/1954729905 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a9a6718aa0 0x55a9a671aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:44.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.544+0000 7f9d95e56640 1 --2- 172.21.15.116:0/1954729905 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a9a6718aa0 0x55a9a671aec0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=935ef24baf710f08 server_cookie=e9976c918257984d in_seq=0 out_seq=0 2024-09-12T03:16:44.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.544+0000 7f9d797fa640 10 client.19098 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:44.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.545+0000 7f9d797fa640 1 -- 172.21.15.116:0/1954729905 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a9a65fad00 con 0x55a9a6718aa0 2024-09-12T03:16:44.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.545+0000 7f9d797fa640 10 client.19098 handle_command_reply: tid=0 2024-09-12T03:16:44.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.545+0000 7f9d9a493740 1 client.19098 shutdown 2024-09-12T03:16:44.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.546+0000 7f9d9a493740 1 -- 172.21.15.116:0/1954729905 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a9a6698d40 msgr2=0x55a9a669dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:44.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.546+0000 7f9d9a493740 1 --2- 172.21.15.116:0/1954729905 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a9a6698d40 0x55a9a669dcb0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f9d8800ebc0 tx=0x7f9d88009c10 comp rx=0 tx=0).stop 2024-09-12T03:16:44.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.546+0000 7f9d9a493740 1 -- 172.21.15.116:0/1954729905 shutdown_connections 2024-09-12T03:16:44.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.546+0000 7f9d9a493740 1 --2- 172.21.15.116:0/1954729905 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a9a6718aa0 0x55a9a671aec0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.546+0000 7f9d9a493740 1 --2- 172.21.15.116:0/1954729905 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a9a6698d40 0x55a9a669dcb0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.546+0000 7f9d9a493740 1 --2- 172.21.15.116:0/1954729905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9a668e480 0x55a9a66a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.546+0000 7f9d797fa640 0 client.19098 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:44.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.546+0000 7f9d9a493740 1 --2- 172.21.15.116:0/1954729905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a9a668db60 0x55a9a66a0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.546+0000 7f9d9a493740 1 -- 172.21.15.116:0/1954729905 >> 172.21.15.116:0/1954729905 conn(0x55a9a668a0f0 msgr2=0x55a9a6695b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:44.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.546+0000 7f9d9a493740 1 -- 172.21.15.116:0/1954729905 shutdown_connections 2024-09-12T03:16:44.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.546+0000 7f9d9a493740 1 -- 172.21.15.116:0/1954729905 wait complete. 2024-09-12T03:16:44.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.546+0000 7f9d9a493740 20 client.19098 trim_cache size 0 max 16384 2024-09-12T03:16:44.558 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:16:44.559 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:16:44.559 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "e1086b07-af37-4a26-ac95-0907bdd20747", 2024-09-12T03:16:44.559 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:16:44.559 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:16:44.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.546+0000 7f9d980ba640 1 -- 172.21.15.116:0/2599081519 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9d7c0a2b80 msgr2=0x7f9d7c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:44.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.546+0000 7f9d980ba640 1 --2- 172.21.15.116:0/2599081519 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9d7c0a2b80 0x7f9d7c0a5040 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c005ab0 tx=0x7f9d8c005a20 comp rx=0 tx=0).stop 2024-09-12T03:16:44.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.546+0000 7f9d980ba640 1 -- 172.21.15.116:0/2599081519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d90102790 msgr2=0x7f9d90198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:44.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.546+0000 7f9d980ba640 1 --2- 172.21.15.116:0/2599081519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d90102790 0x7f9d90198a50 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400c890 tx=0x7f9d8400cd50 comp rx=0 tx=0).stop 2024-09-12T03:16:44.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.547+0000 7f9d980ba640 1 -- 172.21.15.116:0/2599081519 shutdown_connections 2024-09-12T03:16:44.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.547+0000 7f9d980ba640 1 --2- 172.21.15.116:0/2599081519 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9d7c0a2b80 0x7f9d7c0a5040 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.547+0000 7f9d980ba640 1 --2- 172.21.15.116:0/2599081519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d90106f10 0x7f9d9019cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.547+0000 7f9d980ba640 1 --2- 172.21.15.116:0/2599081519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d90102790 0x7f9d90198a50 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.547+0000 7f9d980ba640 1 --2- 172.21.15.116:0/2599081519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d90101de0 0x7f9d90198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:44.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.547+0000 7f9d980ba640 1 -- 172.21.15.116:0/2599081519 >> 172.21.15.116:0/2599081519 conn(0x7f9d900fd8c0 msgr2=0x7f9d900fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:44.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.547+0000 7f9d980ba640 1 -- 172.21.15.116:0/2599081519 shutdown_connections 2024-09-12T03:16:44.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:44.547+0000 7f9d980ba640 1 -- 172.21.15.116:0/2599081519 wait complete. 2024-09-12T03:16:45.166 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:44 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:16:45.167 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:44 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:16:45.167 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:44 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:16:45.167 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:44 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:16:45.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:44 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:16:45.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:44 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:16:45.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:44 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:16:45.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:44 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:16:45.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:44 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:16:45.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:44 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:16:45.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:44 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:16:45.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:44 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:16:45.374 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:16:45.621 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:46.042 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:45 smithi116 ceph-mon[31410]: pgmap v751: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-12T03:16:46.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.125+0000 7f9b4489b640 1 -- 172.21.15.116:0/656012380 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b40102d90 msgr2=0x7f9b40103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:46.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.125+0000 7f9b4489b640 1 --2- 172.21.15.116:0/656012380 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b40102d90 0x7f9b40103170 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f9b30009cd0 tx=0x7f9b3002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.125+0000 7f9b4489b640 1 -- 172.21.15.116:0/656012380 shutdown_connections 2024-09-12T03:16:46.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.125+0000 7f9b4489b640 1 --2- 172.21.15.116:0/656012380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b40108290 0x7f9b4010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.125+0000 7f9b4489b640 1 --2- 172.21.15.116:0/656012380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b40103740 0x7f9b40107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.125+0000 7f9b4489b640 1 --2- 172.21.15.116:0/656012380 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b40102d90 0x7f9b40103170 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.125+0000 7f9b4489b640 1 -- 172.21.15.116:0/656012380 >> 172.21.15.116:0/656012380 conn(0x7f9b400fc7d0 msgr2=0x7f9b400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:46.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.125+0000 7f9b4489b640 1 -- 172.21.15.116:0/656012380 shutdown_connections 2024-09-12T03:16:46.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.125+0000 7f9b4489b640 1 -- 172.21.15.116:0/656012380 wait complete. 2024-09-12T03:16:46.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.125+0000 7f9b4489b640 1 Processor -- start 2024-09-12T03:16:46.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.126+0000 7f9b4489b640 1 -- start start 2024-09-12T03:16:46.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.126+0000 7f9b4489b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b40102d90 0x7f9b401009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:46.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.126+0000 7f9b4489b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b40103740 0x7f9b40100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:46.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.126+0000 7f9b4489b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b40108290 0x7f9b40101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:46.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.126+0000 7f9b4489b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9b401120d0 con 0x7f9b40102d90 2024-09-12T03:16:46.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.126+0000 7f9b4489b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9b40111f50 con 0x7f9b40103740 2024-09-12T03:16:46.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.126+0000 7f9b4489b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9b40112250 con 0x7f9b40108290 2024-09-12T03:16:46.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.126+0000 7f9b3e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b40102d90 0x7f9b401009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:46.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.126+0000 7f9b3e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b40102d90 0x7f9b401009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42032/0 (socket says 172.21.15.116:42032) 2024-09-12T03:16:46.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.126+0000 7f9b3e59c640 1 -- 172.21.15.116:0/2668505976 learned_addr learned my addr 172.21.15.116:0/2668505976 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:46.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.126+0000 7f9b3dd9b640 1 --2- 172.21.15.116:0/2668505976 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b40103740 0x7f9b40100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:46.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.126+0000 7f9b3ed9d640 1 --2- 172.21.15.116:0/2668505976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b40108290 0x7f9b40101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:46.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.126+0000 7f9b3e59c640 1 -- 172.21.15.116:0/2668505976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b40108290 msgr2=0x7f9b40101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:46.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.126+0000 7f9b3e59c640 1 --2- 172.21.15.116:0/2668505976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b40108290 0x7f9b40101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.126+0000 7f9b3e59c640 1 -- 172.21.15.116:0/2668505976 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b40103740 msgr2=0x7f9b40100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:46.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.126+0000 7f9b3e59c640 1 --2- 172.21.15.116:0/2668505976 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b40103740 0x7f9b40100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.126+0000 7f9b3e59c640 1 -- 172.21.15.116:0/2668505976 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b401a6a30 con 0x7f9b40102d90 2024-09-12T03:16:46.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.126+0000 7f9b3ed9d640 1 --2- 172.21.15.116:0/2668505976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b40108290 0x7f9b40101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:46.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.126+0000 7f9b3e59c640 1 --2- 172.21.15.116:0/2668505976 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b40102d90 0x7f9b401009f0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f9b3002ae00 tx=0x7f9b30003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:46.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.126+0000 7f9b2f7fe640 1 -- 172.21.15.116:0/2668505976 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b30004240 con 0x7f9b40102d90 2024-09-12T03:16:46.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.126+0000 7f9b2f7fe640 1 -- 172.21.15.116:0/2668505976 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b300305a0 con 0x7f9b40102d90 2024-09-12T03:16:46.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.127+0000 7f9b2f7fe640 1 -- 172.21.15.116:0/2668505976 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3003a660 con 0x7f9b40102d90 2024-09-12T03:16:46.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.127+0000 7f9b4489b640 1 -- 172.21.15.116:0/2668505976 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b401a6cc0 con 0x7f9b40102d90 2024-09-12T03:16:46.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.127+0000 7f9b4489b640 1 -- 172.21.15.116:0/2668505976 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b401a71f0 con 0x7f9b40102d90 2024-09-12T03:16:46.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.129+0000 7f9b2f7fe640 1 -- 172.21.15.116:0/2668505976 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f9b30030ac0 con 0x7f9b40102d90 2024-09-12T03:16:46.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.129+0000 7f9b46c74740 1 Processor -- start 2024-09-12T03:16:46.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.129+0000 7f9b46c74740 1 -- start start 2024-09-12T03:16:46.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.129+0000 7f9b46c74740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f802f294a0 0x55f802f29880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:46.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.129+0000 7f9b46c74740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f802f29dc0 0x55f802f34140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:46.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.129+0000 7f9b46c74740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f802f34680 0x55f802f36a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:46.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.129+0000 7f9b46c74740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55f802e8f2e0 con 0x55f802f34680 2024-09-12T03:16:46.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.129+0000 7f9b46c74740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f802e36ce0 con 0x55f802f29dc0 2024-09-12T03:16:46.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.129+0000 7f9b46c74740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f802e8b2a0 con 0x55f802f294a0 2024-09-12T03:16:46.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.130+0000 7f9b3dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f802f34680 0x55f802f36a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:46.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.130+0000 7f9b2f7fe640 1 --2- 172.21.15.116:0/2668505976 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9b280a2c00 0x7f9b280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:46.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.130+0000 7f9b3dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f802f34680 0x55f802f36a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42036/0 (socket says 172.21.15.116:42036) 2024-09-12T03:16:46.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.130+0000 7f9b3dd9b640 1 -- 172.21.15.116:0/1444776314 learned_addr learned my addr 172.21.15.116:0/1444776314 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:46.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.130+0000 7f9b3e59c640 1 --2- 172.21.15.116:0/1444776314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f802f29dc0 0x55f802f34140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:46.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.130+0000 7f9b3ed9d640 1 --2- 172.21.15.116:0/2668505976 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9b280a2c00 0x7f9b280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:46.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.130+0000 7f9b3ed9d640 1 --2- 172.21.15.116:0/1444776314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f802f294a0 0x55f802f29880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:46.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.130+0000 7f9b3dd9b640 1 -- 172.21.15.116:0/1444776314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f802f294a0 msgr2=0x55f802f29880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:46.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.130+0000 7f9b3dd9b640 1 --2- 172.21.15.116:0/1444776314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f802f294a0 0x55f802f29880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.130+0000 7f9b3dd9b640 1 -- 172.21.15.116:0/1444776314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f802f29dc0 msgr2=0x55f802f34140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:46.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.130+0000 7f9b3dd9b640 1 --2- 172.21.15.116:0/1444776314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f802f29dc0 0x55f802f34140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.130+0000 7f9b3dd9b640 1 -- 172.21.15.116:0/1444776314 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f802e1b440 con 0x55f802f34680 2024-09-12T03:16:46.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.130+0000 7f9b3ed9d640 1 --2- 172.21.15.116:0/1444776314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f802f294a0 0x55f802f29880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:46.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.130+0000 7f9b2f7fe640 1 -- 172.21.15.116:0/2668505976 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f9b3002e030 con 0x7f9b40102d90 2024-09-12T03:16:46.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.130+0000 7f9b3e59c640 1 --2- 172.21.15.116:0/1444776314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f802f29dc0 0x55f802f34140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:46.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.130+0000 7f9b3dd9b640 1 --2- 172.21.15.116:0/1444776314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f802f34680 0x55f802f36a70 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400cc00 tx=0x7f9b3400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d87247310d635f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:46.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.130+0000 7f9b3ed9d640 1 --2- 172.21.15.116:0/2668505976 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9b280a2c00 0x7f9b280a50c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9b380098e0 tx=0x7f9b38006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:46.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.130+0000 7f9b2cff9640 1 -- 172.21.15.116:0/1444776314 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b34005220 con 0x55f802f34680 2024-09-12T03:16:46.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.130+0000 7f9b2cff9640 1 -- 172.21.15.116:0/1444776314 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b34004c80 con 0x55f802f34680 2024-09-12T03:16:46.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.130+0000 7f9b2cff9640 1 -- 172.21.15.116:0/1444776314 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b34010840 con 0x55f802f34680 2024-09-12T03:16:46.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.131+0000 7f9b46c74740 1 -- 172.21.15.116:0/1444776314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f802f34680 msgr2=0x55f802f36a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:46.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.131+0000 7f9b46c74740 1 --2- 172.21.15.116:0/1444776314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f802f34680 0x55f802f36a70 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400cc00 tx=0x7f9b3400e720 comp rx=0 tx=0).stop 2024-09-12T03:16:46.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.131+0000 7f9b46c74740 1 -- 172.21.15.116:0/1444776314 shutdown_connections 2024-09-12T03:16:46.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.131+0000 7f9b46c74740 1 --2- 172.21.15.116:0/1444776314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f802f34680 0x55f802f36a70 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.131+0000 7f9b46c74740 1 --2- 172.21.15.116:0/1444776314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f802f29dc0 0x55f802f34140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.131+0000 7f9b46c74740 1 --2- 172.21.15.116:0/1444776314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f802f294a0 0x55f802f29880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.131+0000 7f9b46c74740 1 -- 172.21.15.116:0/1444776314 >> 172.21.15.116:0/1444776314 conn(0x55f802f25a30 msgr2=0x55f802f25e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:46.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.131+0000 7f9b46c74740 1 -- 172.21.15.116:0/1444776314 shutdown_connections 2024-09-12T03:16:46.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.131+0000 7f9b46c74740 1 -- 172.21.15.116:0/1444776314 wait complete. 2024-09-12T03:16:46.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.132+0000 7f9b46c74740 1 -- start start 2024-09-12T03:16:46.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.132+0000 7f9b46c74740 1 Processor -- start 2024-09-12T03:16:46.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.132+0000 7f9b46c74740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f802f294a0 0x55f802f33f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:46.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.132+0000 7f9b46c74740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f802f29dc0 0x55f802f311e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:46.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.132+0000 7f9b46c74740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f802f34680 0x55f802f31720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:46.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.132+0000 7f9b46c74740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55f802f37250 con 0x55f802f29dc0 2024-09-12T03:16:46.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.132+0000 7f9b46c74740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f802e4a050 con 0x55f802f34680 2024-09-12T03:16:46.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.132+0000 7f9b46c74740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f802f373d0 con 0x55f802f294a0 2024-09-12T03:16:46.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.132+0000 7f9b46c74740 10 client.0 _ll_register_callbacks cb 0x55f802e8a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:46.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.132+0000 7f9b3dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f802f29dc0 0x55f802f311e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:46.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.132+0000 7f9b3dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f802f29dc0 0x55f802f311e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42046/0 (socket says 172.21.15.116:42046) 2024-09-12T03:16:46.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.132+0000 7f9b3dd9b640 1 -- 172.21.15.116:0/3567502887 learned_addr learned my addr 172.21.15.116:0/3567502887 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:46.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.132+0000 7f9b3e59c640 1 --2- 172.21.15.116:0/3567502887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f802f294a0 0x55f802f33f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:46.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.132+0000 7f9b3ed9d640 1 --2- 172.21.15.116:0/3567502887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f802f34680 0x55f802f31720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:46.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.132+0000 7f9b3dd9b640 1 -- 172.21.15.116:0/3567502887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f802f294a0 msgr2=0x55f802f33f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:46.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.132+0000 7f9b3dd9b640 1 --2- 172.21.15.116:0/3567502887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f802f294a0 0x55f802f33f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.132+0000 7f9b3dd9b640 1 -- 172.21.15.116:0/3567502887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f802f34680 msgr2=0x55f802f31720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:46.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.132+0000 7f9b3dd9b640 1 --2- 172.21.15.116:0/3567502887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f802f34680 0x55f802f31720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.132+0000 7f9b3dd9b640 1 -- 172.21.15.116:0/3567502887 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f802c003c0 con 0x55f802f29dc0 2024-09-12T03:16:46.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.132+0000 7f9b3ed9d640 1 --2- 172.21.15.116:0/3567502887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f802f34680 0x55f802f31720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:46.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.133+0000 7f9b3dd9b640 1 --2- 172.21.15.116:0/3567502887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f802f29dc0 0x55f802f311e0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400c940 tx=0x7f9b34005430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9e1f7e1bd54036 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:46.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.133+0000 7f9b25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:16:46.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.133+0000 7f9b25ffb640 1 -- 172.21.15.116:0/3567502887 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b340048d0 con 0x55f802f29dc0 2024-09-12T03:16:46.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.133+0000 7f9b25ffb640 1 -- 172.21.15.116:0/3567502887 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f9b34004a70 con 0x55f802f29dc0 2024-09-12T03:16:46.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.133+0000 7f9b46c74740 1 -- 172.21.15.116:0/3567502887 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b34004c80 con 0x55f802f29dc0 2024-09-12T03:16:46.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.133+0000 7f9b25ffb640 1 -- 172.21.15.116:0/3567502887 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9b34004050 con 0x55f802f29dc0 2024-09-12T03:16:46.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.133+0000 7f9b46c74740 10 client.19110 fetch_fsmap learned FSMap version 25 2024-09-12T03:16:46.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.133+0000 7f9b46c74740 1 -- 172.21.15.116:0/3567502887 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f802f42b10 con 0x55f802f29dc0 2024-09-12T03:16:46.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.133+0000 7f9b25ffb640 1 -- 172.21.15.116:0/3567502887 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b34004c80 con 0x55f802f29dc0 2024-09-12T03:16:46.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.133+0000 7f9b25ffb640 1 -- 172.21.15.116:0/3567502887 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9b34010580 con 0x55f802f29dc0 2024-09-12T03:16:46.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.133+0000 7f9b46c74740 10 client.19110 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:16:46.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.133+0000 7f9b46c74740 10 client.19110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:16:46.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.133+0000 7f9b46c74740 20 client.19110 populate_metadata read hostname 'smithi116' 2024-09-12T03:16:46.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.133+0000 7f9b46c74740 1 --2- 172.21.15.116:0/3567502887 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f802f43f70 0x55f802f46360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:46.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.133+0000 7f9b46c74740 4 client.19110 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:16:46.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.133+0000 7f9b46c74740 1 -- 172.21.15.116:0/3567502887 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f802e9c850 con 0x55f802f43f70 2024-09-12T03:16:46.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.134+0000 7f9b3e59c640 1 --2- 172.21.15.116:0/3567502887 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f802f43f70 0x55f802f46360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:46.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.134+0000 7f9b3e59c640 1 --2- 172.21.15.116:0/3567502887 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f802f43f70 0x55f802f46360 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d4eceb1cc8d352b server_cookie=2ec2435ceca0c325 in_seq=0 out_seq=0 2024-09-12T03:16:46.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.134+0000 7f9b25ffb640 10 client.19110 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:46.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.135+0000 7f9b25ffb640 1 -- 172.21.15.116:0/3567502887 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f802e9c850 con 0x55f802f43f70 2024-09-12T03:16:46.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.135+0000 7f9b25ffb640 10 client.19110 handle_command_reply: tid=0 2024-09-12T03:16:46.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.135+0000 7f9b46c74740 1 client.19110 shutdown 2024-09-12T03:16:46.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.135+0000 7f9b46c74740 1 -- 172.21.15.116:0/3567502887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f802f29dc0 msgr2=0x55f802f311e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:46.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.135+0000 7f9b46c74740 1 --2- 172.21.15.116:0/3567502887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f802f29dc0 0x55f802f311e0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400c940 tx=0x7f9b34005430 comp rx=0 tx=0).stop 2024-09-12T03:16:46.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.135+0000 7f9b46c74740 1 -- 172.21.15.116:0/3567502887 shutdown_connections 2024-09-12T03:16:46.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.135+0000 7f9b46c74740 1 --2- 172.21.15.116:0/3567502887 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f802f43f70 0x55f802f46360 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.135+0000 7f9b46c74740 1 --2- 172.21.15.116:0/3567502887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f802f34680 0x55f802f31720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.135+0000 7f9b46c74740 1 --2- 172.21.15.116:0/3567502887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f802f29dc0 0x55f802f311e0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.135+0000 7f9b25ffb640 0 client.19110 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:46.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.135+0000 7f9b46c74740 1 --2- 172.21.15.116:0/3567502887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f802f294a0 0x55f802f33f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.135+0000 7f9b46c74740 1 -- 172.21.15.116:0/3567502887 >> 172.21.15.116:0/3567502887 conn(0x55f802f25a30 msgr2=0x55f802f2b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:46.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.135+0000 7f9b46c74740 1 -- 172.21.15.116:0/3567502887 shutdown_connections 2024-09-12T03:16:46.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.135+0000 7f9b46c74740 1 -- 172.21.15.116:0/3567502887 wait complete. 2024-09-12T03:16:46.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.135+0000 7f9b46c74740 20 client.19110 trim_cache size 0 max 16384 2024-09-12T03:16:46.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.152+0000 7f9b46c74740 1 Processor -- start 2024-09-12T03:16:46.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.152+0000 7f9b46c74740 1 -- start start 2024-09-12T03:16:46.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.152+0000 7f9b46c74740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f802f25a30 0x55f802f404e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:46.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.152+0000 7f9b46c74740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f802f294a0 0x55f802f33150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:46.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.152+0000 7f9b46c74740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f802f29dc0 0x55f802f4bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:46.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.152+0000 7f9b46c74740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9b401b1ed0 con 0x55f802f294a0 2024-09-12T03:16:46.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.152+0000 7f9b46c74740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9b401af7a0 con 0x55f802f29dc0 2024-09-12T03:16:46.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.152+0000 7f9b46c74740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9b401ad5f0 con 0x55f802f25a30 2024-09-12T03:16:46.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.152+0000 7f9b3dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f802f294a0 0x55f802f33150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:46.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.152+0000 7f9b3dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f802f294a0 0x55f802f33150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42054/0 (socket says 172.21.15.116:42054) 2024-09-12T03:16:46.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.152+0000 7f9b3dd9b640 1 -- 172.21.15.116:0/159147785 learned_addr learned my addr 172.21.15.116:0/159147785 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:46.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.153+0000 7f9b3ed9d640 1 --2- 172.21.15.116:0/159147785 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f802f29dc0 0x55f802f4bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:46.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.153+0000 7f9b3e59c640 1 --2- 172.21.15.116:0/159147785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f802f25a30 0x55f802f404e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:46.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.153+0000 7f9b3dd9b640 1 -- 172.21.15.116:0/159147785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f802f25a30 msgr2=0x55f802f404e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:46.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.153+0000 7f9b3dd9b640 1 --2- 172.21.15.116:0/159147785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f802f25a30 0x55f802f404e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.153+0000 7f9b3dd9b640 1 -- 172.21.15.116:0/159147785 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f802f29dc0 msgr2=0x55f802f4bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:46.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.153+0000 7f9b3dd9b640 1 --2- 172.21.15.116:0/159147785 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f802f29dc0 0x55f802f4bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.153+0000 7f9b3dd9b640 1 -- 172.21.15.116:0/159147785 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f802f254d0 con 0x55f802f294a0 2024-09-12T03:16:46.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.153+0000 7f9b3ed9d640 1 --2- 172.21.15.116:0/159147785 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f802f29dc0 0x55f802f4bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:46.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.153+0000 7f9b3e59c640 1 --2- 172.21.15.116:0/159147785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f802f25a30 0x55f802f404e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:46.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.153+0000 7f9b3dd9b640 1 --2- 172.21.15.116:0/159147785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f802f294a0 0x55f802f33150 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f9b34004020 tx=0x7f9b3400c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c944301e9258e341 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:46.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.153+0000 7f9b27fff640 1 -- 172.21.15.116:0/159147785 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b340054a0 con 0x55f802f294a0 2024-09-12T03:16:46.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.153+0000 7f9b27fff640 1 -- 172.21.15.116:0/159147785 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b34035020 con 0x55f802f294a0 2024-09-12T03:16:46.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.153+0000 7f9b27fff640 1 -- 172.21.15.116:0/159147785 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b34010670 con 0x55f802f294a0 2024-09-12T03:16:46.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.154+0000 7f9b46c74740 1 -- 172.21.15.116:0/159147785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f802f294a0 msgr2=0x55f802f33150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:46.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.154+0000 7f9b46c74740 1 --2- 172.21.15.116:0/159147785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f802f294a0 0x55f802f33150 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f9b34004020 tx=0x7f9b3400c4b0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.154+0000 7f9b46c74740 1 -- 172.21.15.116:0/159147785 shutdown_connections 2024-09-12T03:16:46.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.154+0000 7f9b46c74740 1 --2- 172.21.15.116:0/159147785 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f802f29dc0 0x55f802f4bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.154+0000 7f9b46c74740 1 --2- 172.21.15.116:0/159147785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f802f294a0 0x55f802f33150 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.154+0000 7f9b46c74740 1 --2- 172.21.15.116:0/159147785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f802f25a30 0x55f802f404e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.154+0000 7f9b46c74740 1 -- 172.21.15.116:0/159147785 >> 172.21.15.116:0/159147785 conn(0x55f802f31880 msgr2=0x55f802f32a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:46.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.154+0000 7f9b46c74740 1 -- 172.21.15.116:0/159147785 shutdown_connections 2024-09-12T03:16:46.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.154+0000 7f9b46c74740 1 -- 172.21.15.116:0/159147785 wait complete. 2024-09-12T03:16:46.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.154+0000 7f9b46c74740 1 -- start start 2024-09-12T03:16:46.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.154+0000 7f9b46c74740 1 Processor -- start 2024-09-12T03:16:46.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.154+0000 7f9b46c74740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f802f25a30 0x55f802f3ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:46.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.154+0000 7f9b46c74740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f802f294a0 0x55f802f38190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:46.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.154+0000 7f9b46c74740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f802f29dc0 0x55f802f38730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:46.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.154+0000 7f9b46c74740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9b401adc60 con 0x55f802f294a0 2024-09-12T03:16:46.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.154+0000 7f9b46c74740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9b401af0d0 con 0x55f802f29dc0 2024-09-12T03:16:46.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.154+0000 7f9b46c74740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9b401ae330 con 0x55f802f25a30 2024-09-12T03:16:46.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.154+0000 7f9b46c74740 10 client.0 _ll_register_callbacks cb 0x7f9b34014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:46.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.154+0000 7f9b3dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f802f294a0 0x55f802f38190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:46.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.154+0000 7f9b3dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f802f294a0 0x55f802f38190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42056/0 (socket says 172.21.15.116:42056) 2024-09-12T03:16:46.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.155+0000 7f9b3dd9b640 1 -- 172.21.15.116:0/3925426053 learned_addr learned my addr 172.21.15.116:0/3925426053 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:46.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.155+0000 7f9b3ed9d640 1 --2- 172.21.15.116:0/3925426053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f802f29dc0 0x55f802f38730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:46.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.155+0000 7f9b3e59c640 1 --2- 172.21.15.116:0/3925426053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f802f25a30 0x55f802f3ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:46.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.155+0000 7f9b3dd9b640 1 -- 172.21.15.116:0/3925426053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f802f25a30 msgr2=0x55f802f3ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:46.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.155+0000 7f9b3dd9b640 1 --2- 172.21.15.116:0/3925426053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f802f25a30 0x55f802f3ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.155+0000 7f9b3dd9b640 1 -- 172.21.15.116:0/3925426053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f802f29dc0 msgr2=0x55f802f38730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:46.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.155+0000 7f9b3dd9b640 1 --2- 172.21.15.116:0/3925426053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f802f29dc0 0x55f802f38730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.155+0000 7f9b3dd9b640 1 -- 172.21.15.116:0/3925426053 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f802f38f60 con 0x55f802f294a0 2024-09-12T03:16:46.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.155+0000 7f9b3e59c640 1 --2- 172.21.15.116:0/3925426053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f802f25a30 0x55f802f3ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:46.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.155+0000 7f9b3dd9b640 1 --2- 172.21.15.116:0/3925426053 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f802f294a0 0x55f802f38190 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f9b34004020 tx=0x7f9b340106c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da0538dbb1b4818 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:46.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.155+0000 7f9b25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:16:46.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.155+0000 7f9b25ffb640 1 -- 172.21.15.116:0/3925426053 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b340055d0 con 0x55f802f294a0 2024-09-12T03:16:46.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.155+0000 7f9b25ffb640 1 -- 172.21.15.116:0/3925426053 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f9b34036040 con 0x55f802f294a0 2024-09-12T03:16:46.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.155+0000 7f9b46c74740 1 -- 172.21.15.116:0/3925426053 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b34035020 con 0x55f802f294a0 2024-09-12T03:16:46.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.155+0000 7f9b25ffb640 1 -- 172.21.15.116:0/3925426053 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9b340041f0 con 0x55f802f294a0 2024-09-12T03:16:46.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.155+0000 7f9b46c74740 10 client.19116 fetch_fsmap learned FSMap version 25 2024-09-12T03:16:46.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.155+0000 7f9b46c74740 1 -- 172.21.15.116:0/3925426053 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f802faeb60 con 0x55f802f294a0 2024-09-12T03:16:46.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.156+0000 7f9b25ffb640 1 -- 172.21.15.116:0/3925426053 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b34035020 con 0x55f802f294a0 2024-09-12T03:16:46.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.156+0000 7f9b25ffb640 1 -- 172.21.15.116:0/3925426053 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9b34003b10 con 0x55f802f294a0 2024-09-12T03:16:46.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.156+0000 7f9b46c74740 10 client.19116 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:16:46.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.156+0000 7f9b46c74740 10 client.19116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:16:46.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.156+0000 7f9b46c74740 20 client.19116 populate_metadata read hostname 'smithi116' 2024-09-12T03:16:46.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.156+0000 7f9b46c74740 1 --2- 172.21.15.116:0/3925426053 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f802f34680 0x55f802fb11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:46.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.156+0000 7f9b46c74740 4 client.19116 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:16:46.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.156+0000 7f9b46c74740 1 -- 172.21.15.116:0/3925426053 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f802e96d60 con 0x55f802f34680 2024-09-12T03:16:46.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.156+0000 7f9b3e59c640 1 --2- 172.21.15.116:0/3925426053 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f802f34680 0x55f802fb11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:46.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.157+0000 7f9b25ffb640 10 client.19116 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:46.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.157+0000 7f9b3e59c640 1 --2- 172.21.15.116:0/3925426053 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f802f34680 0x55f802fb11e0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d26bcc8cfcd8b016 server_cookie=12b5c4ec6c5046b5 in_seq=0 out_seq=0 2024-09-12T03:16:46.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.157+0000 7f9b25ffb640 1 -- 172.21.15.116:0/3925426053 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f802e96d60 con 0x55f802f34680 2024-09-12T03:16:46.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.157+0000 7f9b25ffb640 10 client.19116 handle_command_reply: tid=0 2024-09-12T03:16:46.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.157+0000 7f9b46c74740 1 client.19116 shutdown 2024-09-12T03:16:46.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.157+0000 7f9b46c74740 1 -- 172.21.15.116:0/3925426053 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f802f294a0 msgr2=0x55f802f38190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:46.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.157+0000 7f9b46c74740 1 --2- 172.21.15.116:0/3925426053 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f802f294a0 0x55f802f38190 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f9b34004020 tx=0x7f9b340106c0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.157+0000 7f9b46c74740 1 -- 172.21.15.116:0/3925426053 shutdown_connections 2024-09-12T03:16:46.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.157+0000 7f9b46c74740 1 --2- 172.21.15.116:0/3925426053 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f802f34680 0x55f802fb11e0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.157+0000 7f9b46c74740 1 --2- 172.21.15.116:0/3925426053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f802f29dc0 0x55f802f38730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.157+0000 7f9b46c74740 1 --2- 172.21.15.116:0/3925426053 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f802f294a0 0x55f802f38190 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.157+0000 7f9b25ffb640 0 client.19116 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:46.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.157+0000 7f9b46c74740 1 --2- 172.21.15.116:0/3925426053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f802f25a30 0x55f802f3ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.157+0000 7f9b46c74740 1 -- 172.21.15.116:0/3925426053 >> 172.21.15.116:0/3925426053 conn(0x55f802f31880 msgr2=0x55f802f33260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:46.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.157+0000 7f9b46c74740 1 -- 172.21.15.116:0/3925426053 shutdown_connections 2024-09-12T03:16:46.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.157+0000 7f9b46c74740 1 -- 172.21.15.116:0/3925426053 wait complete. 2024-09-12T03:16:46.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.157+0000 7f9b46c74740 20 client.19116 trim_cache size 0 max 16384 2024-09-12T03:16:46.164 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:16:46.164 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:16:46.164 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": { 2024-09-12T03:16:46.164 INFO:teuthology.orchestra.run.smithi116.stdout: "e1086b07-af37-4a26-ac95-0907bdd20747": { 2024-09-12T03:16:46.164 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/", 2024-09-12T03:16:46.165 INFO:teuthology.orchestra.run.smithi116.stdout: "tag": "e1086b07-af37-4a26-ac95-0907bdd20747", 2024-09-12T03:16:46.165 INFO:teuthology.orchestra.run.smithi116.stdout: "options": "recursive,force" 2024-09-12T03:16:46.165 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:16:46.165 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:16:46.165 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:16:46.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.158+0000 7f9b4489b640 1 -- 172.21.15.116:0/2668505976 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9b280a2c00 msgr2=0x7f9b280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:46.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.158+0000 7f9b4489b640 1 --2- 172.21.15.116:0/2668505976 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9b280a2c00 0x7f9b280a50c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9b380098e0 tx=0x7f9b38006e10 comp rx=0 tx=0).stop 2024-09-12T03:16:46.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.158+0000 7f9b4489b640 1 -- 172.21.15.116:0/2668505976 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b40102d90 msgr2=0x7f9b401009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:46.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.158+0000 7f9b4489b640 1 --2- 172.21.15.116:0/2668505976 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b40102d90 0x7f9b401009f0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f9b3002ae00 tx=0x7f9b30003ba0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.158+0000 7f9b4489b640 1 -- 172.21.15.116:0/2668505976 shutdown_connections 2024-09-12T03:16:46.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.158+0000 7f9b4489b640 1 --2- 172.21.15.116:0/2668505976 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9b280a2c00 0x7f9b280a50c0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.158+0000 7f9b4489b640 1 --2- 172.21.15.116:0/2668505976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b40108290 0x7f9b40101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.158+0000 7f9b4489b640 1 --2- 172.21.15.116:0/2668505976 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b40103740 0x7f9b40100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.158+0000 7f9b4489b640 1 --2- 172.21.15.116:0/2668505976 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b40102d90 0x7f9b401009f0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.158+0000 7f9b4489b640 1 -- 172.21.15.116:0/2668505976 >> 172.21.15.116:0/2668505976 conn(0x7f9b400fc7d0 msgr2=0x7f9b40106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:46.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.158+0000 7f9b4489b640 1 -- 172.21.15.116:0/2668505976 shutdown_connections 2024-09-12T03:16:46.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:46.158+0000 7f9b4489b640 1 -- 172.21.15.116:0/2668505976 wait complete. 2024-09-12T03:16:46.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:45 smithi183 ceph-mon[35535]: pgmap v751: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-12T03:16:46.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:45 smithi177 ceph-mon[35305]: pgmap v751: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-12T03:16:46.967 INFO:tasks.cephfs.filesystem:scrub status for tag:e1086b07-af37-4a26-ac95-0907bdd20747 - {'path': '/', 'tag': 'e1086b07-af37-4a26-ac95-0907bdd20747', 'options': 'recursive,force'} 2024-09-12T03:16:47.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:47 smithi183 ceph-mon[35535]: pgmap v752: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-12T03:16:47.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:47 smithi177 ceph-mon[35305]: pgmap v752: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-12T03:16:47.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:47 smithi116 ceph-mon[31410]: pgmap v752: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-12T03:16:49.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:48 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:16:49.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:48 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:16:49.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:48 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:16:50.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:49 smithi183 ceph-mon[35535]: pgmap v753: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T03:16:50.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:49 smithi177 ceph-mon[35305]: pgmap v753: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T03:16:50.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:49 smithi116 ceph-mon[31410]: pgmap v753: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-12T03:16:50.384 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:16:50.640 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:51.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.156+0000 7ffb4e342640 1 -- 172.21.15.116:0/3463722879 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb48101df0 msgr2=0x7ffb48105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:51.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.156+0000 7ffb4e342640 1 --2- 172.21.15.116:0/3463722879 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb48101df0 0x7ffb48105920 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7ffb30009900 tx=0x7ffb3002a410 comp rx=0 tx=0).stop 2024-09-12T03:16:51.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.156+0000 7ffb4e342640 1 -- 172.21.15.116:0/3463722879 shutdown_connections 2024-09-12T03:16:51.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.156+0000 7ffb4e342640 1 --2- 172.21.15.116:0/3463722879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb48106050 0x7ffb4810dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:51.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.156+0000 7ffb4e342640 1 --2- 172.21.15.116:0/3463722879 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb48101df0 0x7ffb48105920 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:51.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.156+0000 7ffb4e342640 1 --2- 172.21.15.116:0/3463722879 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb48101440 0x7ffb48101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:51.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.156+0000 7ffb4e342640 1 -- 172.21.15.116:0/3463722879 >> 172.21.15.116:0/3463722879 conn(0x7ffb48077210 msgr2=0x7ffb480fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:51.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.156+0000 7ffb4e342640 1 -- 172.21.15.116:0/3463722879 shutdown_connections 2024-09-12T03:16:51.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.156+0000 7ffb4e342640 1 -- 172.21.15.116:0/3463722879 wait complete. 2024-09-12T03:16:51.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.156+0000 7ffb4e342640 1 Processor -- start 2024-09-12T03:16:51.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.156+0000 7ffb4e342640 1 -- start start 2024-09-12T03:16:51.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.157+0000 7ffb4e342640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb48101440 0x7ffb480fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:51.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.157+0000 7ffb4e342640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb48101df0 0x7ffb480fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:51.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.157+0000 7ffb4e342640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb48106050 0x7ffb480fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:51.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.157+0000 7ffb4e342640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ffb481110d0 con 0x7ffb48101df0 2024-09-12T03:16:51.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.157+0000 7ffb4e342640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ffb48110f50 con 0x7ffb48101440 2024-09-12T03:16:51.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.157+0000 7ffb4e342640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffb48111250 con 0x7ffb48106050 2024-09-12T03:16:51.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.157+0000 7ffb3ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb48101df0 0x7ffb480fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:51.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.157+0000 7ffb3ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb48101df0 0x7ffb480fcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37074/0 (socket says 172.21.15.116:37074) 2024-09-12T03:16:51.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.157+0000 7ffb3ffff640 1 -- 172.21.15.116:0/2058753500 learned_addr learned my addr 172.21.15.116:0/2058753500 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:51.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.157+0000 7ffb4c8df640 1 --2- 172.21.15.116:0/2058753500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb48106050 0x7ffb480fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:51.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.157+0000 7ffb47fff640 1 --2- 172.21.15.116:0/2058753500 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb48101440 0x7ffb480fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:51.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.157+0000 7ffb3ffff640 1 -- 172.21.15.116:0/2058753500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb48106050 msgr2=0x7ffb480fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:51.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.157+0000 7ffb3ffff640 1 --2- 172.21.15.116:0/2058753500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb48106050 0x7ffb480fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:51.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.157+0000 7ffb3ffff640 1 -- 172.21.15.116:0/2058753500 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb48101440 msgr2=0x7ffb480fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:51.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.157+0000 7ffb3ffff640 1 --2- 172.21.15.116:0/2058753500 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb48101440 0x7ffb480fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:51.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.157+0000 7ffb3ffff640 1 -- 172.21.15.116:0/2058753500 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb481a6bb0 con 0x7ffb48101df0 2024-09-12T03:16:51.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.157+0000 7ffb4c8df640 1 --2- 172.21.15.116:0/2058753500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb48106050 0x7ffb480fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:51.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.157+0000 7ffb3ffff640 1 --2- 172.21.15.116:0/2058753500 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb48101df0 0x7ffb480fcbd0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7ffb30006010 tx=0x7ffb300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:51.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.158+0000 7ffb45ffb640 1 -- 172.21.15.116:0/2058753500 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3003f030 con 0x7ffb48101df0 2024-09-12T03:16:51.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.158+0000 7ffb4e342640 1 -- 172.21.15.116:0/2058753500 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb481a6e40 con 0x7ffb48101df0 2024-09-12T03:16:51.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.158+0000 7ffb45ffb640 1 -- 172.21.15.116:0/2058753500 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb30003ce0 con 0x7ffb48101df0 2024-09-12T03:16:51.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.158+0000 7ffb45ffb640 1 -- 172.21.15.116:0/2058753500 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb30035070 con 0x7ffb48101df0 2024-09-12T03:16:51.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.158+0000 7ffb4e342640 1 -- 172.21.15.116:0/2058753500 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb481a72a0 con 0x7ffb48101df0 2024-09-12T03:16:51.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.159+0000 7ffb4e342640 1 -- 172.21.15.116:0/2058753500 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb48101820 con 0x7ffb48101df0 2024-09-12T03:16:51.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.160+0000 7ffb45ffb640 1 -- 172.21.15.116:0/2058753500 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ffb30042030 con 0x7ffb48101df0 2024-09-12T03:16:51.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.160+0000 7ffb45ffb640 1 --2- 172.21.15.116:0/2058753500 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ffb340a2b80 0x7ffb340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:51.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.161+0000 7ffb47fff640 1 --2- 172.21.15.116:0/2058753500 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ffb340a2b80 0x7ffb340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:51.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.161+0000 7ffb45ffb640 1 -- 172.21.15.116:0/2058753500 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ffb300ef860 con 0x7ffb48101df0 2024-09-12T03:16:51.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.161+0000 7ffb47fff640 1 --2- 172.21.15.116:0/2058753500 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ffb340a2b80 0x7ffb340a5040 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ffb38005fd0 tx=0x7ffb3800b590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:51.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.164+0000 7ffb45ffb640 1 -- 172.21.15.116:0/2058753500 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffb300bc8b0 con 0x7ffb48101df0 2024-09-12T03:16:51.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.367+0000 7ffb4e342640 1 -- 172.21.15.116:0/2058753500 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb4806a570 con 0x7ffb48101df0 2024-09-12T03:16:51.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.368+0000 7ffb45ffb640 1 -- 172.21.15.116:0/2058753500 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7ffb300bc1b0 con 0x7ffb48101df0 2024-09-12T03:16:51.368 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:16:51.368 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:16:51.369 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:16:51.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.370+0000 7ffb4e342640 1 -- 172.21.15.116:0/2058753500 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ffb340a2b80 msgr2=0x7ffb340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:51.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.370+0000 7ffb4e342640 1 --2- 172.21.15.116:0/2058753500 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ffb340a2b80 0x7ffb340a5040 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ffb38005fd0 tx=0x7ffb3800b590 comp rx=0 tx=0).stop 2024-09-12T03:16:51.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.370+0000 7ffb4e342640 1 -- 172.21.15.116:0/2058753500 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb48101df0 msgr2=0x7ffb480fcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:51.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.370+0000 7ffb4e342640 1 --2- 172.21.15.116:0/2058753500 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb48101df0 0x7ffb480fcbd0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7ffb30006010 tx=0x7ffb300042f0 comp rx=0 tx=0).stop 2024-09-12T03:16:51.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.370+0000 7ffb4e342640 1 -- 172.21.15.116:0/2058753500 shutdown_connections 2024-09-12T03:16:51.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.370+0000 7ffb4e342640 1 --2- 172.21.15.116:0/2058753500 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ffb340a2b80 0x7ffb340a5040 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:51.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.370+0000 7ffb4e342640 1 --2- 172.21.15.116:0/2058753500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb48106050 0x7ffb480fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:51.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.370+0000 7ffb4e342640 1 --2- 172.21.15.116:0/2058753500 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb48101df0 0x7ffb480fcbd0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:51.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.370+0000 7ffb4e342640 1 --2- 172.21.15.116:0/2058753500 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb48101440 0x7ffb480fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:51.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.370+0000 7ffb4e342640 1 -- 172.21.15.116:0/2058753500 >> 172.21.15.116:0/2058753500 conn(0x7ffb48077210 msgr2=0x7ffb48100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:51.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.370+0000 7ffb4e342640 1 -- 172.21.15.116:0/2058753500 shutdown_connections 2024-09-12T03:16:51.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:51.370+0000 7ffb4e342640 1 -- 172.21.15.116:0/2058753500 wait complete. 2024-09-12T03:16:51.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:51 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2058753500' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:51.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:51 smithi183 ceph-mon[35535]: pgmap v754: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-12T03:16:51.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:51 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2058753500' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:51.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:51 smithi177 ceph-mon[35305]: pgmap v754: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-12T03:16:51.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:51 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2058753500' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:51.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:51 smithi116 ceph-mon[31410]: pgmap v754: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-12T03:16:52.327 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id a6a0d1d9 --release --await' 2024-09-12T03:16:52.327 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id a6a0d1d9 --release --await 2024-09-12T03:16:52.584 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:53.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.090+0000 7fe00771a640 1 -- 172.21.15.116:0/1929935890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe000103740 msgr2=0x7fe000107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:53.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.090+0000 7fe00771a640 1 --2- 172.21.15.116:0/1929935890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe000103740 0x7fe000107b80 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fdff0009930 tx=0x7fdff002a420 comp rx=0 tx=0).stop 2024-09-12T03:16:53.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.090+0000 7fe00771a640 1 -- 172.21.15.116:0/1929935890 shutdown_connections 2024-09-12T03:16:53.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.090+0000 7fe00771a640 1 --2- 172.21.15.116:0/1929935890 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe000108290 0x7fe00010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.090+0000 7fe00771a640 1 --2- 172.21.15.116:0/1929935890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe000103740 0x7fe000107b80 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.090+0000 7fe00771a640 1 --2- 172.21.15.116:0/1929935890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe000102d90 0x7fe000103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.090+0000 7fe00771a640 1 -- 172.21.15.116:0/1929935890 >> 172.21.15.116:0/1929935890 conn(0x7fe0000fc7d0 msgr2=0x7fe0000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:53.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.090+0000 7fe00771a640 1 -- 172.21.15.116:0/1929935890 shutdown_connections 2024-09-12T03:16:53.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.090+0000 7fe00771a640 1 -- 172.21.15.116:0/1929935890 wait complete. 2024-09-12T03:16:53.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.091+0000 7fe00771a640 1 Processor -- start 2024-09-12T03:16:53.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.091+0000 7fe00771a640 1 -- start start 2024-09-12T03:16:53.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.091+0000 7fe00771a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe000102d90 0x7fe000198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:53.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.091+0000 7fe00771a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe000103740 0x7fe000198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:53.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.091+0000 7fe00771a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe000108290 0x7fe00019cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:53.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.091+0000 7fe00771a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe0001120b0 con 0x7fe000108290 2024-09-12T03:16:53.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.091+0000 7fe00771a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe000111f30 con 0x7fe000102d90 2024-09-12T03:16:53.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.091+0000 7fe00771a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe000112230 con 0x7fe000103740 2024-09-12T03:16:53.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.091+0000 7fe005cb7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe000108290 0x7fe00019cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:53.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.091+0000 7fe005cb7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe000108290 0x7fe00019cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37092/0 (socket says 172.21.15.116:37092) 2024-09-12T03:16:53.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.091+0000 7fe005cb7640 1 -- 172.21.15.116:0/3532886292 learned_addr learned my addr 172.21.15.116:0/3532886292 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:53.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.091+0000 7fe005cb7640 1 -- 172.21.15.116:0/3532886292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe000103740 msgr2=0x7fe000198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:53.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.091+0000 7fe005cb7640 1 --2- 172.21.15.116:0/3532886292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe000103740 0x7fe000198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.091+0000 7fe005cb7640 1 -- 172.21.15.116:0/3532886292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe000102d90 msgr2=0x7fe000198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:53.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.091+0000 7fe005cb7640 1 --2- 172.21.15.116:0/3532886292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe000102d90 0x7fe000198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.091+0000 7fe005cb7640 1 -- 172.21.15.116:0/3532886292 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe00019d5e0 con 0x7fe000108290 2024-09-12T03:16:53.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.091+0000 7fe005cb7640 1 --2- 172.21.15.116:0/3532886292 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe000108290 0x7fe00019cf00 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fdff800cad0 tx=0x7fdff800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:53.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.092+0000 7fdff67fc640 1 -- 172.21.15.116:0/3532886292 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff800ae20 con 0x7fe000108290 2024-09-12T03:16:53.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.092+0000 7fdff67fc640 1 -- 172.21.15.116:0/3532886292 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdff8005020 con 0x7fe000108290 2024-09-12T03:16:53.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.092+0000 7fe00771a640 1 -- 172.21.15.116:0/3532886292 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe00019d8d0 con 0x7fe000108290 2024-09-12T03:16:53.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.092+0000 7fdff67fc640 1 -- 172.21.15.116:0/3532886292 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff800f6f0 con 0x7fe000108290 2024-09-12T03:16:53.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.092+0000 7fe00771a640 1 -- 172.21.15.116:0/3532886292 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0001a51b0 con 0x7fe000108290 2024-09-12T03:16:53.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.094+0000 7fdff67fc640 1 -- 172.21.15.116:0/3532886292 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fdff80051d0 con 0x7fe000108290 2024-09-12T03:16:53.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.094+0000 7fe009af3740 1 Processor -- start 2024-09-12T03:16:53.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.094+0000 7fe009af3740 1 -- start start 2024-09-12T03:16:53.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.095+0000 7fe009af3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc0e252060 0x55fc0e252440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:53.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.095+0000 7fe009af3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc0e252980 0x55fc0e25cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:53.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.095+0000 7fe009af3740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc0e25d240 0x55fc0e25f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:53.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.095+0000 7fe009af3740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55fc0e1b7620 con 0x55fc0e25d240 2024-09-12T03:16:53.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.095+0000 7fe009af3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55fc0e15ee50 con 0x55fc0e252980 2024-09-12T03:16:53.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.095+0000 7fe009af3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fc0e1b35e0 con 0x55fc0e252060 2024-09-12T03:16:53.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.095+0000 7fdff67fc640 1 --2- 172.21.15.116:0/3532886292 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fdfec0a2c00 0x7fdfec0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:53.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.095+0000 7fe004cb5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc0e25d240 0x55fc0e25f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:53.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.095+0000 7fe005cb7640 1 --2- 172.21.15.116:0/3532886292 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fdfec0a2c00 0x7fdfec0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:53.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.095+0000 7fe004cb5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc0e25d240 0x55fc0e25f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37100/0 (socket says 172.21.15.116:37100) 2024-09-12T03:16:53.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.095+0000 7fe004cb5640 1 -- 172.21.15.116:0/2652306317 learned_addr learned my addr 172.21.15.116:0/2652306317 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:53.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.095+0000 7fe0054b6640 1 --2- 172.21.15.116:0/2652306317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc0e252980 0x55fc0e25cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:53.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.095+0000 7fe004cb5640 1 --2- 172.21.15.116:0/2652306317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc0e252060 0x55fc0e252440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:53.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.095+0000 7fdff67fc640 1 -- 172.21.15.116:0/3532886292 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fdff80cf0d0 con 0x7fe000108290 2024-09-12T03:16:53.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.095+0000 7fe004cb5640 1 -- 172.21.15.116:0/2652306317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc0e252060 msgr2=0x55fc0e252440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:53.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.095+0000 7fe004cb5640 1 --2- 172.21.15.116:0/2652306317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc0e252060 0x55fc0e252440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.095+0000 7fe004cb5640 1 -- 172.21.15.116:0/2652306317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc0e252980 msgr2=0x55fc0e25cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:53.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.095+0000 7fe004cb5640 1 --2- 172.21.15.116:0/2652306317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc0e252980 0x55fc0e25cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.095+0000 7fe004cb5640 1 -- 172.21.15.116:0/2652306317 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc0e14fd40 con 0x55fc0e25d240 2024-09-12T03:16:53.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.095+0000 7fe005cb7640 1 --2- 172.21.15.116:0/3532886292 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fdfec0a2c00 0x7fdfec0a50c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fdff800caa0 tx=0x7fdff8066530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:53.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.095+0000 7fe004cb5640 1 --2- 172.21.15.116:0/2652306317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc0e25d240 0x55fc0e25f630 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fdff0005ab0 tx=0x7fdff0005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa383a1c9a01762f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:53.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.096+0000 7fdfeb7fe640 1 -- 172.21.15.116:0/2652306317 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff0035070 con 0x55fc0e25d240 2024-09-12T03:16:53.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.096+0000 7fdfeb7fe640 1 -- 172.21.15.116:0/2652306317 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdff0004230 con 0x55fc0e25d240 2024-09-12T03:16:53.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.096+0000 7fdfeb7fe640 1 -- 172.21.15.116:0/2652306317 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff003b400 con 0x55fc0e25d240 2024-09-12T03:16:53.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.096+0000 7fe009af3740 1 -- 172.21.15.116:0/2652306317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc0e25d240 msgr2=0x55fc0e25f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:53.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.096+0000 7fe009af3740 1 --2- 172.21.15.116:0/2652306317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc0e25d240 0x55fc0e25f630 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fdff0005ab0 tx=0x7fdff0005a20 comp rx=0 tx=0).stop 2024-09-12T03:16:53.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.096+0000 7fe009af3740 1 -- 172.21.15.116:0/2652306317 shutdown_connections 2024-09-12T03:16:53.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.096+0000 7fe009af3740 1 --2- 172.21.15.116:0/2652306317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc0e25d240 0x55fc0e25f630 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.096+0000 7fe009af3740 1 --2- 172.21.15.116:0/2652306317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc0e252980 0x55fc0e25cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.096+0000 7fe009af3740 1 --2- 172.21.15.116:0/2652306317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc0e252060 0x55fc0e252440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.096+0000 7fe009af3740 1 -- 172.21.15.116:0/2652306317 >> 172.21.15.116:0/2652306317 conn(0x55fc0e24e5f0 msgr2=0x55fc0e24e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:53.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.096+0000 7fe009af3740 1 -- 172.21.15.116:0/2652306317 shutdown_connections 2024-09-12T03:16:53.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.096+0000 7fe009af3740 1 -- 172.21.15.116:0/2652306317 wait complete. 2024-09-12T03:16:53.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.097+0000 7fe009af3740 1 -- start start 2024-09-12T03:16:53.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.097+0000 7fe009af3740 1 Processor -- start 2024-09-12T03:16:53.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.097+0000 7fe009af3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc0e252060 0x55fc0e25a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:53.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.097+0000 7fe009af3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc0e252980 0x55fc0e25a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:53.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.097+0000 7fe009af3740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc0e25d240 0x55fc0e25abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:53.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.097+0000 7fe009af3740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55fc0e25fe10 con 0x55fc0e25d240 2024-09-12T03:16:53.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.097+0000 7fe009af3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55fc0e13ff10 con 0x55fc0e252980 2024-09-12T03:16:53.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.097+0000 7fe009af3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fc0e25ff90 con 0x55fc0e252060 2024-09-12T03:16:53.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.097+0000 7fe009af3740 10 client.0 _ll_register_callbacks cb 0x55fc0e163db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:53.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.097+0000 7fe004cb5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc0e25d240 0x55fc0e25abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:53.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.097+0000 7fe004cb5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc0e25d240 0x55fc0e25abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37108/0 (socket says 172.21.15.116:37108) 2024-09-12T03:16:53.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.097+0000 7fe004cb5640 1 -- 172.21.15.116:0/2468410194 learned_addr learned my addr 172.21.15.116:0/2468410194 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:53.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.098+0000 7fe0054b6640 1 --2- 172.21.15.116:0/2468410194 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc0e252980 0x55fc0e25a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:53.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.098+0000 7fe004cb5640 1 --2- 172.21.15.116:0/2468410194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc0e252060 0x55fc0e25a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:53.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.098+0000 7fe004cb5640 1 -- 172.21.15.116:0/2468410194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc0e252060 msgr2=0x55fc0e25a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:53.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.098+0000 7fe004cb5640 1 --2- 172.21.15.116:0/2468410194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc0e252060 0x55fc0e25a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.098+0000 7fe004cb5640 1 -- 172.21.15.116:0/2468410194 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc0e252980 msgr2=0x55fc0e25a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:53.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.098+0000 7fe004cb5640 1 --2- 172.21.15.116:0/2468410194 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc0e252980 0x55fc0e25a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.098+0000 7fe004cb5640 1 -- 172.21.15.116:0/2468410194 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc0dec6580 con 0x55fc0e25d240 2024-09-12T03:16:53.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.098+0000 7fe004cb5640 1 --2- 172.21.15.116:0/2468410194 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc0e25d240 0x55fc0e25abb0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fdff0001eb0 tx=0x7fdff00314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77077ef9be26649b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:53.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.098+0000 7fdfe8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:16:53.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.098+0000 7fdfe8ff9640 1 -- 172.21.15.116:0/2468410194 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff0035070 con 0x55fc0e25d240 2024-09-12T03:16:53.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.098+0000 7fdfe8ff9640 1 -- 172.21.15.116:0/2468410194 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fdff00035f0 con 0x55fc0e25d240 2024-09-12T03:16:53.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.098+0000 7fe009af3740 1 -- 172.21.15.116:0/2468410194 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdff0004230 con 0x55fc0e25d240 2024-09-12T03:16:53.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.098+0000 7fdfe8ff9640 1 -- 172.21.15.116:0/2468410194 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdff0004090 con 0x55fc0e25d240 2024-09-12T03:16:53.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.098+0000 7fe009af3740 10 client.19134 fetch_fsmap learned FSMap version 25 2024-09-12T03:16:53.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.098+0000 7fe009af3740 1 -- 172.21.15.116:0/2468410194 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fc0df90960 con 0x55fc0e25d240 2024-09-12T03:16:53.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.098+0000 7fdfe8ff9640 1 -- 172.21.15.116:0/2468410194 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdff0004230 con 0x55fc0e25d240 2024-09-12T03:16:53.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.099+0000 7fdfe8ff9640 1 -- 172.21.15.116:0/2468410194 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdff003a040 con 0x55fc0e25d240 2024-09-12T03:16:53.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.099+0000 7fe009af3740 10 client.19134 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:16:53.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.099+0000 7fe009af3740 10 client.19134 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:16:53.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.099+0000 7fe009af3740 20 client.19134 populate_metadata read hostname 'smithi116' 2024-09-12T03:16:53.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.099+0000 7fe009af3740 1 --2- 172.21.15.116:0/2468410194 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55fc0e26cc40 0x55fc0e26f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:53.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.099+0000 7fe009af3740 4 client.19134 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:16:53.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.099+0000 7fe009af3740 1 -- 172.21.15.116:0/2468410194 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc0e1c4b90 con 0x55fc0e26cc40 2024-09-12T03:16:53.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.099+0000 7fe0054b6640 1 --2- 172.21.15.116:0/2468410194 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55fc0e26cc40 0x55fc0e26f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:53.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.099+0000 7fdfe8ff9640 10 client.19134 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:53.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.099+0000 7fe0054b6640 1 --2- 172.21.15.116:0/2468410194 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55fc0e26cc40 0x55fc0e26f030 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88aeb5026922f1f8 server_cookie=5f68405d6d13dbfd in_seq=0 out_seq=0 2024-09-12T03:16:53.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.100+0000 7fdfe8ff9640 1 -- 172.21.15.116:0/2468410194 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fc0e1c4b90 con 0x55fc0e26cc40 2024-09-12T03:16:53.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.100+0000 7fdfe8ff9640 10 client.19134 handle_command_reply: tid=0 2024-09-12T03:16:53.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.100+0000 7fe009af3740 1 client.19134 shutdown 2024-09-12T03:16:53.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.100+0000 7fe009af3740 1 -- 172.21.15.116:0/2468410194 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc0e25d240 msgr2=0x55fc0e25abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:53.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.100+0000 7fe009af3740 1 --2- 172.21.15.116:0/2468410194 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc0e25d240 0x55fc0e25abb0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fdff0001eb0 tx=0x7fdff00314e0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.100+0000 7fe009af3740 1 -- 172.21.15.116:0/2468410194 shutdown_connections 2024-09-12T03:16:53.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.100+0000 7fe009af3740 1 --2- 172.21.15.116:0/2468410194 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55fc0e26cc40 0x55fc0e26f030 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.100+0000 7fe009af3740 1 --2- 172.21.15.116:0/2468410194 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc0e25d240 0x55fc0e25abb0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.100+0000 7fe009af3740 1 --2- 172.21.15.116:0/2468410194 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc0e252980 0x55fc0e25a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.100+0000 7fdfe8ff9640 0 client.19134 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:53.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.100+0000 7fe009af3740 1 --2- 172.21.15.116:0/2468410194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc0e252060 0x55fc0e25a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.100+0000 7fe009af3740 1 -- 172.21.15.116:0/2468410194 >> 172.21.15.116:0/2468410194 conn(0x55fc0e24e5f0 msgr2=0x55fc0e2542d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:53.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.100+0000 7fe009af3740 1 -- 172.21.15.116:0/2468410194 shutdown_connections 2024-09-12T03:16:53.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.100+0000 7fe009af3740 1 -- 172.21.15.116:0/2468410194 wait complete. 2024-09-12T03:16:53.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.100+0000 7fe009af3740 20 client.19134 trim_cache size 0 max 16384 2024-09-12T03:16:53.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.117+0000 7fe009af3740 1 Processor -- start 2024-09-12T03:16:53.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.117+0000 7fe009af3740 1 -- start start 2024-09-12T03:16:53.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.117+0000 7fe009af3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc0e252060 0x55fc0e259310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:53.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.117+0000 7fe009af3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc0e252980 0x55fc0e24d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:53.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.117+0000 7fe009af3740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc0e25d240 0x55fc0e2573b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:53.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.117+0000 7fe009af3740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe0001afe00 con 0x55fc0e25d240 2024-09-12T03:16:53.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.117+0000 7fe009af3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe0001ad6d0 con 0x55fc0e252980 2024-09-12T03:16:53.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.117+0000 7fe009af3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe0001ab520 con 0x55fc0e252060 2024-09-12T03:16:53.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.117+0000 7fe004cb5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc0e25d240 0x55fc0e2573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:53.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.117+0000 7fe004cb5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc0e25d240 0x55fc0e2573b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37114/0 (socket says 172.21.15.116:37114) 2024-09-12T03:16:53.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.117+0000 7fe004cb5640 1 -- 172.21.15.116:0/3083064601 learned_addr learned my addr 172.21.15.116:0/3083064601 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:53.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.118+0000 7fe004cb5640 1 --2- 172.21.15.116:0/3083064601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc0e252060 0x55fc0e259310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:53.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.118+0000 7fe0054b6640 1 --2- 172.21.15.116:0/3083064601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc0e252980 0x55fc0e24d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:53.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.118+0000 7fe004cb5640 1 -- 172.21.15.116:0/3083064601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc0e252060 msgr2=0x55fc0e259310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:53.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.118+0000 7fe004cb5640 1 --2- 172.21.15.116:0/3083064601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc0e252060 0x55fc0e259310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.118+0000 7fe004cb5640 1 -- 172.21.15.116:0/3083064601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc0e252980 msgr2=0x55fc0e24d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:53.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.118+0000 7fe004cb5640 1 --2- 172.21.15.116:0/3083064601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc0e252980 0x55fc0e24d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.118+0000 7fe004cb5640 1 -- 172.21.15.116:0/3083064601 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc0e255d30 con 0x55fc0e25d240 2024-09-12T03:16:53.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.118+0000 7fe0054b6640 1 --2- 172.21.15.116:0/3083064601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc0e252980 0x55fc0e24d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:53.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.118+0000 7fe004cb5640 1 --2- 172.21.15.116:0/3083064601 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc0e25d240 0x55fc0e2573b0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fdff00057c0 tx=0x7fdff00045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1893650c8750f5f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:53.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.118+0000 7fdfeaffd640 1 -- 172.21.15.116:0/3083064601 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff0035040 con 0x55fc0e25d240 2024-09-12T03:16:53.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.118+0000 7fdfeaffd640 1 -- 172.21.15.116:0/3083064601 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdff00318c0 con 0x55fc0e25d240 2024-09-12T03:16:53.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.118+0000 7fdfeaffd640 1 -- 172.21.15.116:0/3083064601 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff0003a10 con 0x55fc0e25d240 2024-09-12T03:16:53.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.119+0000 7fe009af3740 1 -- 172.21.15.116:0/3083064601 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc0e25d240 msgr2=0x55fc0e2573b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:53.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.119+0000 7fe009af3740 1 --2- 172.21.15.116:0/3083064601 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc0e25d240 0x55fc0e2573b0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fdff00057c0 tx=0x7fdff00045f0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.119+0000 7fe009af3740 1 -- 172.21.15.116:0/3083064601 shutdown_connections 2024-09-12T03:16:53.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.119+0000 7fe009af3740 1 --2- 172.21.15.116:0/3083064601 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc0e25d240 0x55fc0e2573b0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.119+0000 7fe009af3740 1 --2- 172.21.15.116:0/3083064601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc0e252980 0x55fc0e24d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.119+0000 7fe009af3740 1 --2- 172.21.15.116:0/3083064601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc0e252060 0x55fc0e259310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.119+0000 7fe009af3740 1 -- 172.21.15.116:0/3083064601 >> 172.21.15.116:0/3083064601 conn(0x55fc0e24e5f0 msgr2=0x55fc0e24cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:53.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.119+0000 7fe009af3740 1 -- 172.21.15.116:0/3083064601 shutdown_connections 2024-09-12T03:16:53.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.119+0000 7fe009af3740 1 -- 172.21.15.116:0/3083064601 wait complete. 2024-09-12T03:16:53.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.119+0000 7fe009af3740 1 -- start start 2024-09-12T03:16:53.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.119+0000 7fe009af3740 1 Processor -- start 2024-09-12T03:16:53.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.120+0000 7fe009af3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc0e252060 0x55fc0e270570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:53.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.120+0000 7fe009af3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc0e252980 0x55fc0e270ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:53.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.120+0000 7fe009af3740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc0e25d240 0x55fc0e2d6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:53.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.120+0000 7fe009af3740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe0001abb90 con 0x55fc0e25d240 2024-09-12T03:16:53.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.120+0000 7fe009af3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe0001ad000 con 0x55fc0e252980 2024-09-12T03:16:53.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.120+0000 7fe009af3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe0001ac260 con 0x55fc0e252060 2024-09-12T03:16:53.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.120+0000 7fe009af3740 10 client.0 _ll_register_callbacks cb 0x7fdff003e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:53.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.120+0000 7fe004cb5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc0e25d240 0x55fc0e2d6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:53.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.120+0000 7fe004cb5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc0e25d240 0x55fc0e2d6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37122/0 (socket says 172.21.15.116:37122) 2024-09-12T03:16:53.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.120+0000 7fe004cb5640 1 -- 172.21.15.116:0/2500283596 learned_addr learned my addr 172.21.15.116:0/2500283596 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:53.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.120+0000 7fe0054b6640 1 --2- 172.21.15.116:0/2500283596 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc0e252980 0x55fc0e270ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:53.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.120+0000 7fe004cb5640 1 -- 172.21.15.116:0/2500283596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc0e252060 msgr2=0x55fc0e270570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:53.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.120+0000 7fe004cb5640 1 --2- 172.21.15.116:0/2500283596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc0e252060 0x55fc0e270570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.120+0000 7fe004cb5640 1 -- 172.21.15.116:0/2500283596 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc0e252980 msgr2=0x55fc0e270ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:53.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.120+0000 7fe004cb5640 1 --2- 172.21.15.116:0/2500283596 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc0e252980 0x55fc0e270ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.120+0000 7fe004cb5640 1 -- 172.21.15.116:0/2500283596 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc0e2daf80 con 0x55fc0e25d240 2024-09-12T03:16:53.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.120+0000 7fe004cb5640 1 --2- 172.21.15.116:0/2500283596 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc0e25d240 0x55fc0e2d6580 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fdff0003d80 tx=0x7fdff0005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e3932672189921 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:53.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.120+0000 7fdfe8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:16:53.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.120+0000 7fdfe8ff9640 1 -- 172.21.15.116:0/2500283596 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff0035070 con 0x55fc0e25d240 2024-09-12T03:16:53.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.120+0000 7fdfe8ff9640 1 -- 172.21.15.116:0/2500283596 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fdff003ec30 con 0x55fc0e25d240 2024-09-12T03:16:53.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.120+0000 7fe009af3740 1 -- 172.21.15.116:0/2500283596 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fc0e1bf0a0 con 0x55fc0e25d240 2024-09-12T03:16:53.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.121+0000 7fdfe8ff9640 1 -- 172.21.15.116:0/2500283596 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdff0009520 con 0x55fc0e25d240 2024-09-12T03:16:53.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.121+0000 7fe009af3740 10 client.19140 fetch_fsmap learned FSMap version 25 2024-09-12T03:16:53.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.121+0000 7fe009af3740 1 -- 172.21.15.116:0/2500283596 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fc0e2dc1d0 con 0x55fc0e25d240 2024-09-12T03:16:53.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.121+0000 7fdfe8ff9640 1 -- 172.21.15.116:0/2500283596 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55fc0e1bf0a0 con 0x55fc0e25d240 2024-09-12T03:16:53.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.121+0000 7fdfe8ff9640 1 -- 172.21.15.116:0/2500283596 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdff0056410 con 0x55fc0e25d240 2024-09-12T03:16:53.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.121+0000 7fe009af3740 10 client.19140 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:16:53.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.121+0000 7fe009af3740 10 client.19140 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:16:53.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.121+0000 7fe009af3740 20 client.19140 populate_metadata read hostname 'smithi116' 2024-09-12T03:16:53.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.121+0000 7fe009af3740 1 --2- 172.21.15.116:0/2500283596 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55fc0e2dc650 0x55fc0e2dea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:53.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.121+0000 7fe009af3740 4 client.19140 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a6a0d1d9", "release": true, "await": true}] 2024-09-12T03:16:53.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.121+0000 7fe009af3740 1 -- 172.21.15.116:0/2500283596 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a6a0d1d9", "release": true, "await": true}) -- 0x55fc0e13b5a0 con 0x55fc0e2dc650 2024-09-12T03:16:53.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.121+0000 7fe0054b6640 1 --2- 172.21.15.116:0/2500283596 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55fc0e2dc650 0x55fc0e2dea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:53.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.122+0000 7fdfe8ff9640 10 client.19140 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:53.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.122+0000 7fe0054b6640 1 --2- 172.21.15.116:0/2500283596 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55fc0e2dc650 0x55fc0e2dea40 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c48a43439066148 server_cookie=6fa531d1ce42631 in_seq=0 out_seq=0 2024-09-12T03:16:53.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.124+0000 7fdfe8ff9640 1 -- 172.21.15.116:0/2500283596 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55fc0e13b5a0 con 0x55fc0e2dc650 2024-09-12T03:16:53.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.124+0000 7fdfe8ff9640 10 client.19140 handle_command_reply: tid=0 2024-09-12T03:16:53.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.124+0000 7fe009af3740 1 client.19140 shutdown 2024-09-12T03:16:53.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.124+0000 7fe009af3740 1 -- 172.21.15.116:0/2500283596 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc0e25d240 msgr2=0x55fc0e2d6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:53.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.124+0000 7fe009af3740 1 --2- 172.21.15.116:0/2500283596 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc0e25d240 0x55fc0e2d6580 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fdff0003d80 tx=0x7fdff0005c90 comp rx=0 tx=0).stop 2024-09-12T03:16:53.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.124+0000 7fe009af3740 1 -- 172.21.15.116:0/2500283596 shutdown_connections 2024-09-12T03:16:53.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.124+0000 7fe009af3740 1 --2- 172.21.15.116:0/2500283596 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55fc0e2dc650 0x55fc0e2dea40 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.124+0000 7fe009af3740 1 --2- 172.21.15.116:0/2500283596 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fc0e25d240 0x55fc0e2d6580 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.124+0000 7fe009af3740 1 --2- 172.21.15.116:0/2500283596 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fc0e252980 0x55fc0e270ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.124+0000 7fe009af3740 1 --2- 172.21.15.116:0/2500283596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc0e252060 0x55fc0e270570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.124+0000 7fe009af3740 1 -- 172.21.15.116:0/2500283596 >> 172.21.15.116:0/2500283596 conn(0x55fc0e24e5f0 msgr2=0x55fc0e25a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:53.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.124+0000 7fdfe8ff9640 0 client.19140 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:53.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.125+0000 7fe009af3740 1 -- 172.21.15.116:0/2500283596 shutdown_connections 2024-09-12T03:16:53.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.125+0000 7fe009af3740 1 -- 172.21.15.116:0/2500283596 wait complete. 2024-09-12T03:16:53.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.125+0000 7fe009af3740 20 client.19140 trim_cache size 0 max 16384 2024-09-12T03:16:53.137 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:16:53.138 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 25, 2024-09-12T03:16:53.138 INFO:teuthology.orchestra.run.smithi116.stdout: "leader": 24494, 2024-09-12T03:16:53.138 INFO:teuthology.orchestra.run.smithi116.stdout: "set_version": 52, 2024-09-12T03:16:53.138 INFO:teuthology.orchestra.run.smithi116.stdout: "sets": { 2024-09-12T03:16:53.138 INFO:teuthology.orchestra.run.smithi116.stdout: "a6a0d1d9": { 2024-09-12T03:16:53.138 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 52, 2024-09-12T03:16:53.138 INFO:teuthology.orchestra.run.smithi116.stdout: "age_ref": 0.0, 2024-09-12T03:16:53.138 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:16:53.138 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RELEASED", 2024-09-12T03:16:53.138 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:16:53.138 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:16:53.138 INFO:teuthology.orchestra.run.smithi116.stdout: "timeout": 90.0, 2024-09-12T03:16:53.138 INFO:teuthology.orchestra.run.smithi116.stdout: "expiration": 150.0, 2024-09-12T03:16:53.139 INFO:teuthology.orchestra.run.smithi116.stdout: "members": { 2024-09-12T03:16:53.139 INFO:teuthology.orchestra.run.smithi116.stdout: "file:/": { 2024-09-12T03:16:53.139 INFO:teuthology.orchestra.run.smithi116.stdout: "excluded": false, 2024-09-12T03:16:53.139 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:16:53.139 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RELEASED", 2024-09-12T03:16:53.139 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:16:53.139 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:16:53.139 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:16:53.139 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:16:53.139 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:16:53.139 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:16:53.139 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:16:53.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.125+0000 7fe00771a640 1 -- 172.21.15.116:0/3532886292 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fdfec0a2c00 msgr2=0x7fdfec0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:53.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.125+0000 7fe00771a640 1 --2- 172.21.15.116:0/3532886292 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fdfec0a2c00 0x7fdfec0a50c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fdff800caa0 tx=0x7fdff8066530 comp rx=0 tx=0).stop 2024-09-12T03:16:53.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.125+0000 7fe00771a640 1 -- 172.21.15.116:0/3532886292 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe000108290 msgr2=0x7fe00019cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:53.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.125+0000 7fe00771a640 1 --2- 172.21.15.116:0/3532886292 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe000108290 0x7fe00019cf00 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fdff800cad0 tx=0x7fdff800cf90 comp rx=0 tx=0).stop 2024-09-12T03:16:53.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.125+0000 7fe00771a640 1 -- 172.21.15.116:0/3532886292 shutdown_connections 2024-09-12T03:16:53.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.125+0000 7fe00771a640 1 --2- 172.21.15.116:0/3532886292 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fdfec0a2c00 0x7fdfec0a50c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.125+0000 7fe00771a640 1 --2- 172.21.15.116:0/3532886292 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe000108290 0x7fe00019cf00 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.125+0000 7fe00771a640 1 --2- 172.21.15.116:0/3532886292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe000103740 0x7fe000198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.125+0000 7fe00771a640 1 --2- 172.21.15.116:0/3532886292 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe000102d90 0x7fe000198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:53.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.125+0000 7fe00771a640 1 -- 172.21.15.116:0/3532886292 >> 172.21.15.116:0/3532886292 conn(0x7fe0000fc7d0 msgr2=0x7fe000105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:53.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.125+0000 7fe00771a640 1 -- 172.21.15.116:0/3532886292 shutdown_connections 2024-09-12T03:16:53.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:53.125+0000 7fe00771a640 1 -- 172.21.15.116:0/3532886292 wait complete. 2024-09-12T03:16:53.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:53 smithi183 ceph-mon[35535]: pgmap v755: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T03:16:53.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:53 smithi177 ceph-mon[35305]: pgmap v755: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T03:16:53.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:53 smithi116 ceph-mon[31410]: pgmap v755: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T03:16:53.961 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a6a0d1d9', total seconds elapsed: 33.6 2024-09-12T03:16:53.962 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-12T03:16:53.962 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:16:54.208 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:54.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.692+0000 7fa532eb3640 1 -- 172.21.15.116:0/1488016927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa52c100a70 msgr2=0x7fa52c10ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:54.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.692+0000 7fa532eb3640 1 --2- 172.21.15.116:0/1488016927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa52c100a70 0x7fa52c10ea60 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fa524009ca0 tx=0x7fa52402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.693+0000 7fa532eb3640 1 -- 172.21.15.116:0/1488016927 shutdown_connections 2024-09-12T03:16:54.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.693+0000 7fa532eb3640 1 --2- 172.21.15.116:0/1488016927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa52c100a70 0x7fa52c10ea60 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.693+0000 7fa532eb3640 1 --2- 172.21.15.116:0/1488016927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa52c1000d0 0x7fa52c100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.693+0000 7fa532eb3640 1 --2- 172.21.15.116:0/1488016927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa52c106ba0 0x7fa52c106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.693+0000 7fa532eb3640 1 -- 172.21.15.116:0/1488016927 >> 172.21.15.116:0/1488016927 conn(0x7fa52c077210 msgr2=0x7fa52c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:54.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.693+0000 7fa532eb3640 1 -- 172.21.15.116:0/1488016927 shutdown_connections 2024-09-12T03:16:54.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.693+0000 7fa532eb3640 1 -- 172.21.15.116:0/1488016927 wait complete. 2024-09-12T03:16:54.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.693+0000 7fa532eb3640 1 Processor -- start 2024-09-12T03:16:54.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.693+0000 7fa532eb3640 1 -- start start 2024-09-12T03:16:54.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.693+0000 7fa532eb3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa52c1000d0 0x7fa52c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:54.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.693+0000 7fa532eb3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa52c100a70 0x7fa52c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:54.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.693+0000 7fa532eb3640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa52c106ba0 0x7fa52c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:54.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.693+0000 7fa532eb3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c1120b0 con 0x7fa52c100a70 2024-09-12T03:16:54.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.693+0000 7fa532eb3640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c111f30 con 0x7fa52c106ba0 2024-09-12T03:16:54.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.693+0000 7fa532eb3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c112230 con 0x7fa52c1000d0 2024-09-12T03:16:54.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.694+0000 7fa52bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa52c100a70 0x7fa52c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:54.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.694+0000 7fa52bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa52c100a70 0x7fa52c198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37142/0 (socket says 172.21.15.116:37142) 2024-09-12T03:16:54.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.694+0000 7fa52bfff640 1 -- 172.21.15.116:0/99401855 learned_addr learned my addr 172.21.15.116:0/99401855 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:54.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.694+0000 7fa531450640 1 --2- 172.21.15.116:0/99401855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa52c106ba0 0x7fa52c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:54.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.694+0000 7fa52bfff640 1 -- 172.21.15.116:0/99401855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa52c1000d0 msgr2=0x7fa52c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:54.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.694+0000 7fa530c4f640 1 --2- 172.21.15.116:0/99401855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa52c1000d0 0x7fa52c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:54.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.694+0000 7fa52bfff640 1 --2- 172.21.15.116:0/99401855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa52c1000d0 0x7fa52c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.694+0000 7fa52bfff640 1 -- 172.21.15.116:0/99401855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa52c106ba0 msgr2=0x7fa52c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:54.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.694+0000 7fa52bfff640 1 --2- 172.21.15.116:0/99401855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa52c106ba0 0x7fa52c19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.694+0000 7fa52bfff640 1 -- 172.21.15.116:0/99401855 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52c19d3e0 con 0x7fa52c100a70 2024-09-12T03:16:54.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.694+0000 7fa531450640 1 --2- 172.21.15.116:0/99401855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa52c106ba0 0x7fa52c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:54.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.694+0000 7fa530c4f640 1 --2- 172.21.15.116:0/99401855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa52c1000d0 0x7fa52c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:54.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.694+0000 7fa52bfff640 1 --2- 172.21.15.116:0/99401855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa52c100a70 0x7fa52c198970 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fa51800c920 tx=0x7fa51800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:54.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.694+0000 7fa529ffb640 1 -- 172.21.15.116:0/99401855 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa518007c70 con 0x7fa52c100a70 2024-09-12T03:16:54.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.694+0000 7fa529ffb640 1 -- 172.21.15.116:0/99401855 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa518007e10 con 0x7fa52c100a70 2024-09-12T03:16:54.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.694+0000 7fa532eb3640 1 -- 172.21.15.116:0/99401855 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa52c19d6d0 con 0x7fa52c100a70 2024-09-12T03:16:54.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.694+0000 7fa529ffb640 1 -- 172.21.15.116:0/99401855 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa518018890 con 0x7fa52c100a70 2024-09-12T03:16:54.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.694+0000 7fa532eb3640 1 -- 172.21.15.116:0/99401855 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa52c1a4fb0 con 0x7fa52c100a70 2024-09-12T03:16:54.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.695+0000 7fa532eb3640 1 -- 172.21.15.116:0/99401855 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa52c100530 con 0x7fa52c100a70 2024-09-12T03:16:54.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.696+0000 7fa529ffb640 1 -- 172.21.15.116:0/99401855 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa518018a70 con 0x7fa52c100a70 2024-09-12T03:16:54.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.697+0000 7fa529ffb640 1 --2- 172.21.15.116:0/99401855 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa51c0a2b80 0x7fa51c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:54.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.697+0000 7fa530c4f640 1 --2- 172.21.15.116:0/99401855 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa51c0a2b80 0x7fa51c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:54.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.697+0000 7fa529ffb640 1 -- 172.21.15.116:0/99401855 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa5180cf210 con 0x7fa52c100a70 2024-09-12T03:16:54.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.698+0000 7fa530c4f640 1 --2- 172.21.15.116:0/99401855 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa51c0a2b80 0x7fa51c0a5040 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa520006fd0 tx=0x7fa520008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:54.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.701+0000 7fa529ffb640 1 -- 172.21.15.116:0/99401855 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa518014030 con 0x7fa52c100a70 2024-09-12T03:16:54.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.907+0000 7fa532eb3640 1 -- 172.21.15.116:0/99401855 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa52c1a5210 con 0x7fa52c100a70 2024-09-12T03:16:54.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.907+0000 7fa529ffb640 1 -- 172.21.15.116:0/99401855 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7fa51800de20 con 0x7fa52c100a70 2024-09-12T03:16:54.908 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:16:54.908 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:16:54.908 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:16:54.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.910+0000 7fa532eb3640 1 -- 172.21.15.116:0/99401855 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa51c0a2b80 msgr2=0x7fa51c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:54.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.910+0000 7fa532eb3640 1 --2- 172.21.15.116:0/99401855 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa51c0a2b80 0x7fa51c0a5040 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa520006fd0 tx=0x7fa520008040 comp rx=0 tx=0).stop 2024-09-12T03:16:54.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.910+0000 7fa532eb3640 1 -- 172.21.15.116:0/99401855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa52c100a70 msgr2=0x7fa52c198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:54.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.910+0000 7fa532eb3640 1 --2- 172.21.15.116:0/99401855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa52c100a70 0x7fa52c198970 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fa51800c920 tx=0x7fa51800cde0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.910+0000 7fa532eb3640 1 -- 172.21.15.116:0/99401855 shutdown_connections 2024-09-12T03:16:54.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.910+0000 7fa532eb3640 1 --2- 172.21.15.116:0/99401855 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa51c0a2b80 0x7fa51c0a5040 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.910+0000 7fa532eb3640 1 --2- 172.21.15.116:0/99401855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa52c106ba0 0x7fa52c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.910+0000 7fa532eb3640 1 --2- 172.21.15.116:0/99401855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa52c100a70 0x7fa52c198970 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.910+0000 7fa532eb3640 1 --2- 172.21.15.116:0/99401855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa52c1000d0 0x7fa52c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.910+0000 7fa532eb3640 1 -- 172.21.15.116:0/99401855 >> 172.21.15.116:0/99401855 conn(0x7fa52c077210 msgr2=0x7fa52c104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:54.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.910+0000 7fa532eb3640 1 -- 172.21.15.116:0/99401855 shutdown_connections 2024-09-12T03:16:54.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:54.910+0000 7fa532eb3640 1 -- 172.21.15.116:0/99401855 wait complete. 2024-09-12T03:16:55.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:54 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/99401855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:55.266 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:54 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/99401855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:55.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:54 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/99401855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:55.722 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-12T03:16:55.722 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-12T03:16:55.974 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:56.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:55 smithi177 ceph-mon[35305]: pgmap v756: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2024-09-12T03:16:56.254 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:55 smithi116 ceph-mon[31410]: pgmap v756: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2024-09-12T03:16:56.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:55 smithi183 ceph-mon[35535]: pgmap v756: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2024-09-12T03:16:56.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.477+0000 7fabef344640 1 -- 172.21.15.116:0/3936797025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabe8101450 msgr2=0x7fabe8105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:56.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.477+0000 7fabef344640 1 --2- 172.21.15.116:0/3936797025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabe8101450 0x7fabe8105920 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fabd8009900 tx=0x7fabd802a470 comp rx=0 tx=0).stop 2024-09-12T03:16:56.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.477+0000 7fabef344640 1 -- 172.21.15.116:0/3936797025 shutdown_connections 2024-09-12T03:16:56.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.477+0000 7fabef344640 1 --2- 172.21.15.116:0/3936797025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabe8105f40 0x7fabe81063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.477+0000 7fabef344640 1 --2- 172.21.15.116:0/3936797025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabe8101450 0x7fabe8105920 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.477+0000 7fabef344640 1 --2- 172.21.15.116:0/3936797025 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabe8100b30 0x7fabe8100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.477+0000 7fabef344640 1 -- 172.21.15.116:0/3936797025 >> 172.21.15.116:0/3936797025 conn(0x7fabe80fc7a0 msgr2=0x7fabe80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:56.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.477+0000 7fabef344640 1 -- 172.21.15.116:0/3936797025 shutdown_connections 2024-09-12T03:16:56.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.477+0000 7fabef344640 1 -- 172.21.15.116:0/3936797025 wait complete. 2024-09-12T03:16:56.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.478+0000 7fabef344640 1 Processor -- start 2024-09-12T03:16:56.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.478+0000 7fabef344640 1 -- start start 2024-09-12T03:16:56.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.478+0000 7fabef344640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabe8100b30 0x7fabe819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.478+0000 7fabef344640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabe8101450 0x7fabe819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.478+0000 7fabef344640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabe8105f40 0x7fabe81a3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.478+0000 7fabef344640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fabe81185c0 con 0x7fabe8100b30 2024-09-12T03:16:56.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.478+0000 7fabef344640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fabe8118440 con 0x7fabe8105f40 2024-09-12T03:16:56.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.478+0000 7fabef344640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fabe8118740 con 0x7fabe8101450 2024-09-12T03:16:56.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.479+0000 7fabed0e0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabe8100b30 0x7fabe819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.479+0000 7fabed0e0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabe8100b30 0x7fabe819ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37172/0 (socket says 172.21.15.116:37172) 2024-09-12T03:16:56.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.479+0000 7fabed0e0640 1 -- 172.21.15.116:0/3010531822 learned_addr learned my addr 172.21.15.116:0/3010531822 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:56.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.479+0000 7fabed8e1640 1 --2- 172.21.15.116:0/3010531822 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabe8105f40 0x7fabe81a3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.479+0000 7fabec8df640 1 --2- 172.21.15.116:0/3010531822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabe8101450 0x7fabe819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.479+0000 7fabed0e0640 1 -- 172.21.15.116:0/3010531822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabe8101450 msgr2=0x7fabe819f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:56.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.479+0000 7fabed0e0640 1 --2- 172.21.15.116:0/3010531822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabe8101450 0x7fabe819f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.479+0000 7fabed0e0640 1 -- 172.21.15.116:0/3010531822 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabe8105f40 msgr2=0x7fabe81a3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:56.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.479+0000 7fabed0e0640 1 --2- 172.21.15.116:0/3010531822 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabe8105f40 0x7fabe81a3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.479+0000 7fabed0e0640 1 -- 172.21.15.116:0/3010531822 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe81a3c30 con 0x7fabe8100b30 2024-09-12T03:16:56.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.479+0000 7fabec8df640 1 --2- 172.21.15.116:0/3010531822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabe8101450 0x7fabe819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:56.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.479+0000 7fabed0e0640 1 --2- 172.21.15.116:0/3010531822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabe8100b30 0x7fabe819ec00 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fabe400b4a0 tx=0x7fabe400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:56.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.479+0000 7fabde7fc640 1 -- 172.21.15.116:0/3010531822 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe4014070 con 0x7fabe8100b30 2024-09-12T03:16:56.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.479+0000 7fabde7fc640 1 -- 172.21.15.116:0/3010531822 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabe4010af0 con 0x7fabe8100b30 2024-09-12T03:16:56.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.479+0000 7fabef344640 1 -- 172.21.15.116:0/3010531822 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe81a3f20 con 0x7fabe8100b30 2024-09-12T03:16:56.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.479+0000 7fabde7fc640 1 -- 172.21.15.116:0/3010531822 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe4017d10 con 0x7fabe8100b30 2024-09-12T03:16:56.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.479+0000 7fabef344640 1 -- 172.21.15.116:0/3010531822 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe81ab7b0 con 0x7fabe8100b30 2024-09-12T03:16:56.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.482+0000 7fabde7fc640 1 -- 172.21.15.116:0/3010531822 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fabe4017070 con 0x7fabe8100b30 2024-09-12T03:16:56.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.482+0000 7fabf171d740 1 Processor -- start 2024-09-12T03:16:56.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.482+0000 7fabf171d740 1 -- start start 2024-09-12T03:16:56.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.482+0000 7fabde7fc640 1 --2- 172.21.15.116:0/3010531822 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fabd40a2b80 0x7fabd40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.482+0000 7fabf171d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e8153b9760 0x55e8153b9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.482+0000 7fabf171d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8153ba080 0x55e8153c4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.482+0000 7fabf171d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8153c4940 0x55e8153c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.482+0000 7fabf171d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e81531f2a0 con 0x55e8153c4940 2024-09-12T03:16:56.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.482+0000 7fabf171d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e8152c6b10 con 0x55e8153b9760 2024-09-12T03:16:56.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.482+0000 7fabf171d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e81531b260 con 0x55e8153ba080 2024-09-12T03:16:56.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.483+0000 7fabed8e1640 1 --2- 172.21.15.116:0/3010531822 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fabd40a2b80 0x7fabd40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.483+0000 7fabed8e1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8153c4940 0x55e8153c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.483+0000 7fabed0e0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e8153b9760 0x55e8153b9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.483+0000 7fabed8e1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8153c4940 0x55e8153c6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37184/0 (socket says 172.21.15.116:37184) 2024-09-12T03:16:56.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.483+0000 7fabed8e1640 1 -- 172.21.15.116:0/2982045380 learned_addr learned my addr 172.21.15.116:0/2982045380 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:56.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.483+0000 7fabde7fc640 1 -- 172.21.15.116:0/3010531822 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fabe40ce440 con 0x7fabe8100b30 2024-09-12T03:16:56.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.483+0000 7fabed0e0640 1 -- 172.21.15.116:0/2982045380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8153ba080 msgr2=0x55e8153c4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:56.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.483+0000 7fabec8df640 1 --2- 172.21.15.116:0/2982045380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8153ba080 0x55e8153c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.483+0000 7fabed0e0640 1 --2- 172.21.15.116:0/2982045380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8153ba080 0x55e8153c4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.483+0000 7fabed0e0640 1 -- 172.21.15.116:0/2982045380 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8153c4940 msgr2=0x55e8153c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:56.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.483+0000 7fabed0e0640 1 --2- 172.21.15.116:0/2982045380 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8153c4940 0x55e8153c6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.483+0000 7fabed0e0640 1 -- 172.21.15.116:0/2982045380 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e815090b90 con 0x55e8153b9760 2024-09-12T03:16:56.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.483+0000 7fabec8df640 1 --2- 172.21.15.116:0/2982045380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8153ba080 0x55e8153c4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:56.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.483+0000 7fabed8e1640 1 --2- 172.21.15.116:0/2982045380 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8153c4940 0x55e8153c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:56.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.483+0000 7fabed8e1640 1 --2- 172.21.15.116:0/3010531822 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fabd40a2b80 0x7fabd40a5040 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fabe0014c60 tx=0x7fabe000ae60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:56.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.483+0000 7fabed0e0640 1 --2- 172.21.15.116:0/2982045380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e8153b9760 0x55e8153b9b40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fabe4009420 tx=0x7fabe40047d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53c15b5627a2c55e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:56.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.483+0000 7fabd37fe640 1 -- 172.21.15.116:0/2982045380 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe40a82f0 con 0x55e8153b9760 2024-09-12T03:16:56.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.484+0000 7fabd37fe640 1 -- 172.21.15.116:0/2982045380 <== mon.2 v2:172.21.15.177:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabe40a47a0 con 0x55e8153b9760 2024-09-12T03:16:56.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.484+0000 7fabd37fe640 1 -- 172.21.15.116:0/2982045380 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe4017410 con 0x55e8153b9760 2024-09-12T03:16:56.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.484+0000 7fabf171d740 1 -- 172.21.15.116:0/2982045380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e8153b9760 msgr2=0x55e8153b9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:56.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.484+0000 7fabf171d740 1 --2- 172.21.15.116:0/2982045380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e8153b9760 0x55e8153b9b40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fabe4009420 tx=0x7fabe40047d0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.484+0000 7fabf171d740 1 -- 172.21.15.116:0/2982045380 shutdown_connections 2024-09-12T03:16:56.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.484+0000 7fabf171d740 1 --2- 172.21.15.116:0/2982045380 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8153c4940 0x55e8153c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.484+0000 7fabf171d740 1 --2- 172.21.15.116:0/2982045380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8153ba080 0x55e8153c4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.484+0000 7fabf171d740 1 --2- 172.21.15.116:0/2982045380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e8153b9760 0x55e8153b9b40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.484+0000 7fabf171d740 1 -- 172.21.15.116:0/2982045380 >> 172.21.15.116:0/2982045380 conn(0x55e8153b5cf0 msgr2=0x55e8153b60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:56.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.484+0000 7fabf171d740 1 -- 172.21.15.116:0/2982045380 shutdown_connections 2024-09-12T03:16:56.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.484+0000 7fabf171d740 1 -- 172.21.15.116:0/2982045380 wait complete. 2024-09-12T03:16:56.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.485+0000 7fabf171d740 1 -- start start 2024-09-12T03:16:56.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.485+0000 7fabf171d740 1 Processor -- start 2024-09-12T03:16:56.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.485+0000 7fabf171d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8153b9760 0x55e8153c1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.485+0000 7fabf171d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8153ba080 0x55e8153c1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.485+0000 7fabf171d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e8153c4940 0x55e8153c22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.485+0000 7fabf171d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e8153c7510 con 0x55e8153ba080 2024-09-12T03:16:56.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.485+0000 7fabf171d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e8152a76e0 con 0x55e8153c4940 2024-09-12T03:16:56.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.485+0000 7fabf171d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e8153c7690 con 0x55e8153b9760 2024-09-12T03:16:56.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.485+0000 7fabf171d740 10 client.0 _ll_register_callbacks cb 0x55e8152cba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:56.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.485+0000 7fabec8df640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8153ba080 0x55e8153c1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.485+0000 7fabec8df640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8153ba080 0x55e8153c1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37196/0 (socket says 172.21.15.116:37196) 2024-09-12T03:16:56.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.485+0000 7fabec8df640 1 -- 172.21.15.116:0/4055081204 learned_addr learned my addr 172.21.15.116:0/4055081204 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:56.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.485+0000 7fabed8e1640 1 --2- 172.21.15.116:0/4055081204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e8153c4940 0x55e8153c22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.486+0000 7fabed0e0640 1 --2- 172.21.15.116:0/4055081204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8153b9760 0x55e8153c1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.486+0000 7fabec8df640 1 -- 172.21.15.116:0/4055081204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8153b9760 msgr2=0x55e8153c1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:56.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.486+0000 7fabec8df640 1 --2- 172.21.15.116:0/4055081204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8153b9760 0x55e8153c1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.486+0000 7fabec8df640 1 -- 172.21.15.116:0/4055081204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e8153c4940 msgr2=0x55e8153c22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:56.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.486+0000 7fabec8df640 1 --2- 172.21.15.116:0/4055081204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e8153c4940 0x55e8153c22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.486+0000 7fabec8df640 1 -- 172.21.15.116:0/4055081204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8153d0980 con 0x55e8153ba080 2024-09-12T03:16:56.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.486+0000 7fabec8df640 1 --2- 172.21.15.116:0/4055081204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8153ba080 0x55e8153c1d90 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fabd8009230 tx=0x7fabd8032f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=384b1152af913809 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:56.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.486+0000 7fabd0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:16:56.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.486+0000 7fabd0ff9640 1 -- 172.21.15.116:0/4055081204 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd8035030 con 0x55e8153ba080 2024-09-12T03:16:56.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.486+0000 7fabd0ff9640 1 -- 172.21.15.116:0/4055081204 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fabd8004050 con 0x55e8153ba080 2024-09-12T03:16:56.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.486+0000 7fabf171d740 1 -- 172.21.15.116:0/4055081204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabe40a47a0 con 0x55e8153ba080 2024-09-12T03:16:56.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.486+0000 7fabd0ff9640 1 -- 172.21.15.116:0/4055081204 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fabd804a050 con 0x55e8153ba080 2024-09-12T03:16:56.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.486+0000 7fabf171d740 10 client.19158 fetch_fsmap learned FSMap version 25 2024-09-12T03:16:56.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.486+0000 7fabf171d740 1 -- 172.21.15.116:0/4055081204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e8153d3310 con 0x55e8153ba080 2024-09-12T03:16:56.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.486+0000 7fabd0ff9640 1 -- 172.21.15.116:0/4055081204 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabe40a47a0 con 0x55e8153ba080 2024-09-12T03:16:56.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.487+0000 7fabd0ff9640 1 -- 172.21.15.116:0/4055081204 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fabd8004900 con 0x55e8153ba080 2024-09-12T03:16:56.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.487+0000 7fabf171d740 10 client.19158 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:16:56.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.487+0000 7fabf171d740 10 client.19158 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:16:56.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.487+0000 7fabf171d740 20 client.19158 populate_metadata read hostname 'smithi116' 2024-09-12T03:16:56.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.487+0000 7fabf171d740 1 --2- 172.21.15.116:0/4055081204 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e8153d4820 0x55e8153d6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.487+0000 7fabf171d740 4 client.19158 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:16:56.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.487+0000 7fabf171d740 1 -- 172.21.15.116:0/4055081204 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e81532c9d0 con 0x55e8153d4820 2024-09-12T03:16:56.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.487+0000 7fabed0e0640 1 --2- 172.21.15.116:0/4055081204 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e8153d4820 0x55e8153d6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.487+0000 7fabed0e0640 1 --2- 172.21.15.116:0/4055081204 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e8153d4820 0x55e8153d6c60 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7376c1a1a56b5bdd server_cookie=996487a1c87983b7 in_seq=0 out_seq=0 2024-09-12T03:16:56.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.487+0000 7fabd0ff9640 10 client.19158 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:56.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.488+0000 7fabd0ff9640 1 -- 172.21.15.116:0/4055081204 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e81532c9d0 con 0x55e8153d4820 2024-09-12T03:16:56.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.488+0000 7fabd0ff9640 10 client.19158 handle_command_reply: tid=0 2024-09-12T03:16:56.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.488+0000 7fabf171d740 1 client.19158 shutdown 2024-09-12T03:16:56.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.488+0000 7fabf171d740 1 -- 172.21.15.116:0/4055081204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8153ba080 msgr2=0x55e8153c1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:56.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.488+0000 7fabf171d740 1 --2- 172.21.15.116:0/4055081204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8153ba080 0x55e8153c1d90 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fabd8009230 tx=0x7fabd8032f30 comp rx=0 tx=0).stop 2024-09-12T03:16:56.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.488+0000 7fabf171d740 1 -- 172.21.15.116:0/4055081204 shutdown_connections 2024-09-12T03:16:56.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.488+0000 7fabf171d740 1 --2- 172.21.15.116:0/4055081204 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e8153d4820 0x55e8153d6c60 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.488+0000 7fabf171d740 1 --2- 172.21.15.116:0/4055081204 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e8153c4940 0x55e8153c22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.488+0000 7fabf171d740 1 --2- 172.21.15.116:0/4055081204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8153ba080 0x55e8153c1d90 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.488+0000 7fabf171d740 1 --2- 172.21.15.116:0/4055081204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8153b9760 0x55e8153c1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.488+0000 7fabf171d740 1 -- 172.21.15.116:0/4055081204 >> 172.21.15.116:0/4055081204 conn(0x55e8153b5cf0 msgr2=0x55e8153bba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:56.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.488+0000 7fabd0ff9640 0 client.19158 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:56.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.489+0000 7fabf171d740 1 -- 172.21.15.116:0/4055081204 shutdown_connections 2024-09-12T03:16:56.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.489+0000 7fabf171d740 1 -- 172.21.15.116:0/4055081204 wait complete. 2024-09-12T03:16:56.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.489+0000 7fabf171d740 20 client.19158 trim_cache size 0 max 16384 2024-09-12T03:16:56.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.505+0000 7fabf171d740 1 Processor -- start 2024-09-12T03:16:56.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.505+0000 7fabf171d740 1 -- start start 2024-09-12T03:16:56.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.505+0000 7fabf171d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e8153b9760 0x55e8153c37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.505+0000 7fabf171d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8153ba080 0x55e8153c2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.505+0000 7fabf171d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8153c4940 0x55e8153dc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.505+0000 7fabf171d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fabe81b64f0 con 0x55e8153ba080 2024-09-12T03:16:56.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.505+0000 7fabf171d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fabe81b3dc0 con 0x55e8153b9760 2024-09-12T03:16:56.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.505+0000 7fabf171d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fabe81b1c10 con 0x55e8153c4940 2024-09-12T03:16:56.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.506+0000 7fabec8df640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8153ba080 0x55e8153c2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.506+0000 7fabec8df640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8153ba080 0x55e8153c2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37212/0 (socket says 172.21.15.116:37212) 2024-09-12T03:16:56.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.506+0000 7fabec8df640 1 -- 172.21.15.116:0/2880556038 learned_addr learned my addr 172.21.15.116:0/2880556038 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:56.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.506+0000 7fabed0e0640 1 --2- 172.21.15.116:0/2880556038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e8153b9760 0x55e8153c37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.506+0000 7fabed8e1640 1 --2- 172.21.15.116:0/2880556038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8153c4940 0x55e8153dc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.506+0000 7fabec8df640 1 -- 172.21.15.116:0/2880556038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8153c4940 msgr2=0x55e8153dc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:56.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.506+0000 7fabec8df640 1 --2- 172.21.15.116:0/2880556038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8153c4940 0x55e8153dc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.506+0000 7fabec8df640 1 -- 172.21.15.116:0/2880556038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e8153b9760 msgr2=0x55e8153c37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:56.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.506+0000 7fabec8df640 1 --2- 172.21.15.116:0/2880556038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e8153b9760 0x55e8153c37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.506+0000 7fabec8df640 1 -- 172.21.15.116:0/2880556038 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8153c7070 con 0x55e8153ba080 2024-09-12T03:16:56.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.506+0000 7fabec8df640 1 --2- 172.21.15.116:0/2880556038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8153ba080 0x55e8153c2120 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fabd8006010 tx=0x7fabd80037e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7547726f7637e346 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:56.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.506+0000 7fabd2ffd640 1 -- 172.21.15.116:0/2880556038 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd8035070 con 0x55e8153ba080 2024-09-12T03:16:56.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.506+0000 7fabd2ffd640 1 -- 172.21.15.116:0/2880556038 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabd8031e20 con 0x55e8153ba080 2024-09-12T03:16:56.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.506+0000 7fabd2ffd640 1 -- 172.21.15.116:0/2880556038 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd80052a0 con 0x55e8153ba080 2024-09-12T03:16:56.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.507+0000 7fabf171d740 1 -- 172.21.15.116:0/2880556038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8153ba080 msgr2=0x55e8153c2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:56.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.507+0000 7fabf171d740 1 --2- 172.21.15.116:0/2880556038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8153ba080 0x55e8153c2120 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fabd8006010 tx=0x7fabd80037e0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.507+0000 7fabf171d740 1 -- 172.21.15.116:0/2880556038 shutdown_connections 2024-09-12T03:16:56.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.507+0000 7fabf171d740 1 --2- 172.21.15.116:0/2880556038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8153c4940 0x55e8153dc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.507+0000 7fabf171d740 1 --2- 172.21.15.116:0/2880556038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8153ba080 0x55e8153c2120 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.507+0000 7fabf171d740 1 --2- 172.21.15.116:0/2880556038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e8153b9760 0x55e8153c37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.507+0000 7fabf171d740 1 -- 172.21.15.116:0/2880556038 >> 172.21.15.116:0/2880556038 conn(0x55e8153b5cf0 msgr2=0x55e8153c09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:56.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.507+0000 7fabf171d740 1 -- 172.21.15.116:0/2880556038 shutdown_connections 2024-09-12T03:16:56.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.507+0000 7fabf171d740 1 -- 172.21.15.116:0/2880556038 wait complete. 2024-09-12T03:16:56.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.507+0000 7fabf171d740 1 -- start start 2024-09-12T03:16:56.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.508+0000 7fabf171d740 1 Processor -- start 2024-09-12T03:16:56.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.508+0000 7fabf171d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8153b9760 0x55e8153cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.508+0000 7fabf171d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8153ba080 0x55e8153c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.508+0000 7fabf171d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e8153c4940 0x55e8153c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.508+0000 7fabf171d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fabe81b2280 con 0x55e8153ba080 2024-09-12T03:16:56.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.508+0000 7fabf171d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fabe81b36f0 con 0x55e8153c4940 2024-09-12T03:16:56.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.508+0000 7fabf171d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fabe81b2950 con 0x55e8153b9760 2024-09-12T03:16:56.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.508+0000 7fabf171d740 10 client.0 _ll_register_callbacks cb 0x7fabd8005e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:56.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.508+0000 7fabec8df640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8153ba080 0x55e8153c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.508+0000 7fabec8df640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8153ba080 0x55e8153c83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:37228/0 (socket says 172.21.15.116:37228) 2024-09-12T03:16:56.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.508+0000 7fabec8df640 1 -- 172.21.15.116:0/1369633191 learned_addr learned my addr 172.21.15.116:0/1369633191 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:16:56.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.508+0000 7fabed8e1640 1 --2- 172.21.15.116:0/1369633191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e8153c4940 0x55e8153c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.508+0000 7fabed0e0640 1 --2- 172.21.15.116:0/1369633191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8153b9760 0x55e8153cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.509+0000 7fabec8df640 1 -- 172.21.15.116:0/1369633191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8153b9760 msgr2=0x55e8153cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:56.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.509+0000 7fabec8df640 1 --2- 172.21.15.116:0/1369633191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8153b9760 0x55e8153cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.509+0000 7fabec8df640 1 -- 172.21.15.116:0/1369633191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e8153c4940 msgr2=0x55e8153c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:56.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.509+0000 7fabec8df640 1 --2- 172.21.15.116:0/1369633191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e8153c4940 0x55e8153c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.509+0000 7fabec8df640 1 -- 172.21.15.116:0/1369633191 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8153c9130 con 0x55e8153ba080 2024-09-12T03:16:56.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.509+0000 7fabed8e1640 1 --2- 172.21.15.116:0/1369633191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e8153c4940 0x55e8153c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:56.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.509+0000 7fabec8df640 1 --2- 172.21.15.116:0/1369633191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8153ba080 0x55e8153c83c0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fabd8009850 tx=0x7fabd8031bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a065e3a882e2dc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:56.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.509+0000 7fabd0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:16:56.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.509+0000 7fabd0ff9640 1 -- 172.21.15.116:0/1369633191 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd8035040 con 0x55e8153ba080 2024-09-12T03:16:56.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.509+0000 7fabd0ff9640 1 -- 172.21.15.116:0/1369633191 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fabd80048d0 con 0x55e8153ba080 2024-09-12T03:16:56.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.509+0000 7fabf171d740 1 -- 172.21.15.116:0/1369633191 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabd8031e20 con 0x55e8153ba080 2024-09-12T03:16:56.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.509+0000 7fabd0ff9640 1 -- 172.21.15.116:0/1369633191 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fabd80035f0 con 0x55e8153ba080 2024-09-12T03:16:56.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.509+0000 7fabf171d740 10 client.19164 fetch_fsmap learned FSMap version 25 2024-09-12T03:16:56.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.509+0000 7fabf171d740 1 -- 172.21.15.116:0/1369633191 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e81543f540 con 0x55e8153ba080 2024-09-12T03:16:56.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.509+0000 7fabd0ff9640 1 -- 172.21.15.116:0/1369633191 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabd8031e20 con 0x55e8153ba080 2024-09-12T03:16:56.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.510+0000 7fabd0ff9640 1 -- 172.21.15.116:0/1369633191 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fabd803b4f0 con 0x55e8153ba080 2024-09-12T03:16:56.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.510+0000 7fabf171d740 10 client.19164 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:16:56.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.510+0000 7fabf171d740 10 client.19164 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:16:56.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.510+0000 7fabf171d740 20 client.19164 populate_metadata read hostname 'smithi116' 2024-09-12T03:16:56.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.510+0000 7fabf171d740 1 --2- 172.21.15.116:0/1369633191 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e81543fc40 0x55e815442060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.510+0000 7fabf171d740 4 client.19164 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-12T03:16:56.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.510+0000 7fabf171d740 1 -- 172.21.15.116:0/1369633191 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55e815326d20 con 0x55e81543fc40 2024-09-12T03:16:56.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.510+0000 7fabed0e0640 1 --2- 172.21.15.116:0/1369633191 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e81543fc40 0x55e815442060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.510+0000 7fabed0e0640 1 --2- 172.21.15.116:0/1369633191 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e81543fc40 0x55e815442060 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=babcf216aa828c63 server_cookie=406aacfeb55d85bd in_seq=0 out_seq=0 2024-09-12T03:16:56.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.510+0000 7fabd0ff9640 10 client.19164 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:56.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.526+0000 7fabd0ff9640 1 -- 172.21.15.116:0/1369633191 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e815326d20 con 0x55e81543fc40 2024-09-12T03:16:56.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.526+0000 7fabd0ff9640 10 client.19164 handle_command_reply: tid=0 2024-09-12T03:16:56.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.526+0000 7fabf171d740 1 client.19164 shutdown 2024-09-12T03:16:56.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.526+0000 7fabf171d740 1 -- 172.21.15.116:0/1369633191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8153ba080 msgr2=0x55e8153c83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:56.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.526+0000 7fabf171d740 1 --2- 172.21.15.116:0/1369633191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8153ba080 0x55e8153c83c0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fabd8009850 tx=0x7fabd8031bf0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.526+0000 7fabf171d740 1 -- 172.21.15.116:0/1369633191 shutdown_connections 2024-09-12T03:16:56.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.526+0000 7fabf171d740 1 --2- 172.21.15.116:0/1369633191 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e81543fc40 0x55e815442060 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.526+0000 7fabf171d740 1 --2- 172.21.15.116:0/1369633191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e8153c4940 0x55e8153c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.526+0000 7fabf171d740 1 --2- 172.21.15.116:0/1369633191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8153ba080 0x55e8153c83c0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.526+0000 7fabf171d740 1 --2- 172.21.15.116:0/1369633191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8153b9760 0x55e8153cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.526+0000 7fabf171d740 1 -- 172.21.15.116:0/1369633191 >> 172.21.15.116:0/1369633191 conn(0x55e8153b5cf0 msgr2=0x55e8153bb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:56.531 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.526+0000 7fabd0ff9640 0 client.19164 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:16:56.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.526+0000 7fabf171d740 1 -- 172.21.15.116:0/1369633191 shutdown_connections 2024-09-12T03:16:56.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.526+0000 7fabf171d740 1 -- 172.21.15.116:0/1369633191 wait complete. 2024-09-12T03:16:56.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.526+0000 7fabf171d740 20 client.19164 trim_cache size 0 max 16384 2024-09-12T03:16:56.532 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:16:56.533 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 25, 2024-09-12T03:16:56.533 INFO:teuthology.orchestra.run.smithi116.stdout: "leader": 24494, 2024-09-12T03:16:56.533 INFO:teuthology.orchestra.run.smithi116.stdout: "set_version": 54, 2024-09-12T03:16:56.533 INFO:teuthology.orchestra.run.smithi116.stdout: "sets": { 2024-09-12T03:16:56.533 INFO:teuthology.orchestra.run.smithi116.stdout: "1cfce56f": { 2024-09-12T03:16:56.534 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 54, 2024-09-12T03:16:56.534 INFO:teuthology.orchestra.run.smithi116.stdout: "age_ref": 0.0, 2024-09-12T03:16:56.534 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:16:56.534 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "QUIESCED", 2024-09-12T03:16:56.534 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:16:56.534 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:16:56.535 INFO:teuthology.orchestra.run.smithi116.stdout: "timeout": 90.0, 2024-09-12T03:16:56.535 INFO:teuthology.orchestra.run.smithi116.stdout: "expiration": 150.0, 2024-09-12T03:16:56.535 INFO:teuthology.orchestra.run.smithi116.stdout: "members": { 2024-09-12T03:16:56.535 INFO:teuthology.orchestra.run.smithi116.stdout: "file:/": { 2024-09-12T03:16:56.535 INFO:teuthology.orchestra.run.smithi116.stdout: "excluded": false, 2024-09-12T03:16:56.536 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:16:56.536 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "QUIESCED", 2024-09-12T03:16:56.536 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:16:56.536 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:16:56.536 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:16:56.536 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:16:56.537 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:16:56.537 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:16:56.537 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:16:56.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.527+0000 7fabef344640 1 -- 172.21.15.116:0/3010531822 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fabd40a2b80 msgr2=0x7fabd40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:56.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.527+0000 7fabef344640 1 --2- 172.21.15.116:0/3010531822 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fabd40a2b80 0x7fabd40a5040 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fabe0014c60 tx=0x7fabe000ae60 comp rx=0 tx=0).stop 2024-09-12T03:16:56.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.527+0000 7fabef344640 1 -- 172.21.15.116:0/3010531822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabe8100b30 msgr2=0x7fabe819ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:56.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.527+0000 7fabef344640 1 --2- 172.21.15.116:0/3010531822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabe8100b30 0x7fabe819ec00 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fabe400b4a0 tx=0x7fabe400b960 comp rx=0 tx=0).stop 2024-09-12T03:16:56.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.527+0000 7fabef344640 1 -- 172.21.15.116:0/3010531822 shutdown_connections 2024-09-12T03:16:56.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.527+0000 7fabef344640 1 --2- 172.21.15.116:0/3010531822 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fabd40a2b80 0x7fabd40a5040 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.527+0000 7fabef344640 1 --2- 172.21.15.116:0/3010531822 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabe8105f40 0x7fabe81a3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.527+0000 7fabef344640 1 --2- 172.21.15.116:0/3010531822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabe8101450 0x7fabe819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.527+0000 7fabef344640 1 --2- 172.21.15.116:0/3010531822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabe8100b30 0x7fabe819ec00 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.527+0000 7fabef344640 1 -- 172.21.15.116:0/3010531822 >> 172.21.15.116:0/3010531822 conn(0x7fabe80fc7a0 msgr2=0x7fabe810f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:56.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.527+0000 7fabef344640 1 -- 172.21.15.116:0/3010531822 shutdown_connections 2024-09-12T03:16:56.540 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:16:56.527+0000 7fabef344640 1 -- 172.21.15.116:0/3010531822 wait complete. 2024-09-12T03:16:57.341 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1cfce56f', quiesce took 3.4 seconds. Will release after: 26.6 2024-09-12T03:16:57.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:57 smithi183 ceph-mon[35535]: pgmap v757: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-09-12T03:16:57.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:57 smithi177 ceph-mon[35305]: pgmap v757: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-09-12T03:16:57.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:57 smithi116 ceph-mon[31410]: pgmap v757: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-09-12T03:16:59.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:16:59 smithi183 ceph-mon[35535]: pgmap v758: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.5 KiB/s wr, 8 op/s 2024-09-12T03:16:59.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:16:59 smithi177 ceph-mon[35305]: pgmap v758: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.5 KiB/s wr, 8 op/s 2024-09-12T03:16:59.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:16:59 smithi116 ceph-mon[31410]: pgmap v758: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.5 KiB/s wr, 8 op/s 2024-09-12T03:17:01.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:01 smithi183 ceph-mon[35535]: pgmap v759: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-09-12T03:17:01.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:01 smithi177 ceph-mon[35305]: pgmap v759: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-09-12T03:17:01.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:01 smithi116 ceph-mon[31410]: pgmap v759: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-09-12T03:17:03.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:03 smithi183 ceph-mon[35535]: pgmap v760: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.2 KiB/s wr, 7 op/s 2024-09-12T03:17:03.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:03 smithi177 ceph-mon[35305]: pgmap v760: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.2 KiB/s wr, 7 op/s 2024-09-12T03:17:03.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:03 smithi116 ceph-mon[31410]: pgmap v760: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.2 KiB/s wr, 7 op/s 2024-09-12T03:17:05.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:05 smithi183 ceph-mon[35535]: pgmap v761: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-09-12T03:17:05.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:05 smithi177 ceph-mon[35305]: pgmap v761: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-09-12T03:17:05.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:05 smithi116 ceph-mon[31410]: pgmap v761: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-09-12T03:17:07.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:07 smithi183 ceph-mon[35535]: pgmap v762: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-09-12T03:17:07.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:07 smithi177 ceph-mon[35305]: pgmap v762: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-09-12T03:17:07.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:07 smithi116 ceph-mon[31410]: pgmap v762: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-09-12T03:17:09.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:09 smithi183 ceph-mon[35535]: pgmap v763: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-09-12T03:17:09.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:09 smithi177 ceph-mon[35305]: pgmap v763: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-09-12T03:17:09.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:09 smithi116 ceph-mon[31410]: pgmap v763: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-09-12T03:17:11.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:11 smithi183 ceph-mon[35535]: pgmap v764: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6 op/s 2024-09-12T03:17:11.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:11 smithi177 ceph-mon[35305]: pgmap v764: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6 op/s 2024-09-12T03:17:11.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:11 smithi116 ceph-mon[31410]: pgmap v764: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6 op/s 2024-09-12T03:17:13.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:13 smithi183 ceph-mon[35535]: pgmap v765: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:17:13.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:13 smithi177 ceph-mon[35305]: pgmap v765: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:17:13.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:13 smithi116 ceph-mon[31410]: pgmap v765: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:17:15.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:15 smithi183 ceph-mon[35535]: pgmap v766: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-09-12T03:17:15.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:15 smithi177 ceph-mon[35305]: pgmap v766: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-09-12T03:17:15.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:15 smithi116 ceph-mon[31410]: pgmap v766: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-09-12T03:17:16.968 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:17:17.231 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:17.647 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:17 smithi116 ceph-mon[31410]: pgmap v767: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:17:17.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:17 smithi183 ceph-mon[35535]: pgmap v767: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:17:17.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:17 smithi177 ceph-mon[35305]: pgmap v767: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:17:17.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.805+0000 7f1c23d16640 1 -- 172.21.15.116:0/1157829549 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c1c101de0 msgr2=0x7f1c1c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.805+0000 7f1c23d16640 1 --2- 172.21.15.116:0/1157829549 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c1c101de0 0x7f1c1c1021c0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c009900 tx=0x7f1c0c02a410 comp rx=0 tx=0).stop 2024-09-12T03:17:17.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.805+0000 7f1c23d16640 1 -- 172.21.15.116:0/1157829549 shutdown_connections 2024-09-12T03:17:17.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.805+0000 7f1c23d16640 1 --2- 172.21.15.116:0/1157829549 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c1c106f10 0x7f1c1c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.805+0000 7f1c23d16640 1 --2- 172.21.15.116:0/1157829549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c1c102790 0x7f1c1c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.805+0000 7f1c23d16640 1 --2- 172.21.15.116:0/1157829549 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c1c101de0 0x7f1c1c1021c0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.805+0000 7f1c23d16640 1 -- 172.21.15.116:0/1157829549 >> 172.21.15.116:0/1157829549 conn(0x7f1c1c0fd8c0 msgr2=0x7f1c1c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:17.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.805+0000 7f1c23d16640 1 -- 172.21.15.116:0/1157829549 shutdown_connections 2024-09-12T03:17:17.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.805+0000 7f1c23d16640 1 -- 172.21.15.116:0/1157829549 wait complete. 2024-09-12T03:17:17.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.806+0000 7f1c23d16640 1 Processor -- start 2024-09-12T03:17:17.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.806+0000 7f1c23d16640 1 -- start start 2024-09-12T03:17:17.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.806+0000 7f1c23d16640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c1c101de0 0x7f1c1c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.806+0000 7f1c23d16640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c1c102790 0x7f1c1c198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.806+0000 7f1c23d16640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c1c106f10 0x7f1c1c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.806+0000 7f1c23d16640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c111f90 con 0x7f1c1c102790 2024-09-12T03:17:17.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.806+0000 7f1c23d16640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c111e10 con 0x7f1c1c101de0 2024-09-12T03:17:17.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.806+0000 7f1c23d16640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c112110 con 0x7f1c1c106f10 2024-09-12T03:17:17.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.806+0000 7f1c212b1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c1c102790 0x7f1c1c198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.806+0000 7f1c212b1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c1c102790 0x7f1c1c198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46092/0 (socket says 172.21.15.116:46092) 2024-09-12T03:17:17.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.806+0000 7f1c212b1640 1 -- 172.21.15.116:0/1949623883 learned_addr learned my addr 172.21.15.116:0/1949623883 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:17.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.806+0000 7f1c222b3640 1 --2- 172.21.15.116:0/1949623883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c1c106f10 0x7f1c1c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.806+0000 7f1c21ab2640 1 --2- 172.21.15.116:0/1949623883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c1c101de0 0x7f1c1c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.806+0000 7f1c212b1640 1 -- 172.21.15.116:0/1949623883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c1c106f10 msgr2=0x7f1c1c19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.806+0000 7f1c212b1640 1 --2- 172.21.15.116:0/1949623883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c1c106f10 0x7f1c1c19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.806+0000 7f1c212b1640 1 -- 172.21.15.116:0/1949623883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c1c101de0 msgr2=0x7f1c1c1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.806+0000 7f1c212b1640 1 --2- 172.21.15.116:0/1949623883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c1c101de0 0x7f1c1c1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.806+0000 7f1c212b1640 1 -- 172.21.15.116:0/1949623883 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1c19d480 con 0x7f1c1c102790 2024-09-12T03:17:17.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.807+0000 7f1c212b1640 1 --2- 172.21.15.116:0/1949623883 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c1c102790 0x7f1c1c198a10 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f1c1800b4a0 tx=0x7f1c1800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:17.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.807+0000 7f1c12ffd640 1 -- 172.21.15.116:0/1949623883 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18014070 con 0x7f1c1c102790 2024-09-12T03:17:17.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.807+0000 7f1c12ffd640 1 -- 172.21.15.116:0/1949623883 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c18010af0 con 0x7f1c1c102790 2024-09-12T03:17:17.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.807+0000 7f1c23d16640 1 -- 172.21.15.116:0/1949623883 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c1c19d770 con 0x7f1c1c102790 2024-09-12T03:17:17.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.807+0000 7f1c12ffd640 1 -- 172.21.15.116:0/1949623883 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18017d10 con 0x7f1c1c102790 2024-09-12T03:17:17.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.807+0000 7f1c23d16640 1 -- 172.21.15.116:0/1949623883 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c1c1a5050 con 0x7f1c1c102790 2024-09-12T03:17:17.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.809+0000 7f1c12ffd640 1 -- 172.21.15.116:0/1949623883 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1c18017070 con 0x7f1c1c102790 2024-09-12T03:17:17.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.809+0000 7f1c260ef740 1 Processor -- start 2024-09-12T03:17:17.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.810+0000 7f1c260ef740 1 -- start start 2024-09-12T03:17:17.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.810+0000 7f1c260ef740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b57c0d14a0 0x55b57c0d1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.810+0000 7f1c12ffd640 1 --2- 172.21.15.116:0/1949623883 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1c080a2c00 0x7f1c080a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.810+0000 7f1c260ef740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b57c0d1dc0 0x55b57c0dc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.810+0000 7f1c260ef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b57c0dc680 0x55b57c0dea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.810+0000 7f1c260ef740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b57c0372e0 con 0x55b57c0d1dc0 2024-09-12T03:17:17.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.810+0000 7f1c260ef740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b57bfdece0 con 0x55b57c0d14a0 2024-09-12T03:17:17.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.810+0000 7f1c260ef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b57c0332a0 con 0x55b57c0dc680 2024-09-12T03:17:17.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.810+0000 7f1c222b3640 1 --2- 172.21.15.116:0/1949623883 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1c080a2c00 0x7f1c080a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.810+0000 7f1c212b1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b57c0d1dc0 0x55b57c0dc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.810+0000 7f1c12ffd640 1 -- 172.21.15.116:0/1949623883 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1c180ce420 con 0x7f1c1c102790 2024-09-12T03:17:17.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.810+0000 7f1c212b1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b57c0d1dc0 0x55b57c0dc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46106/0 (socket says 172.21.15.116:46106) 2024-09-12T03:17:17.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.810+0000 7f1c212b1640 1 -- 172.21.15.116:0/2732600821 learned_addr learned my addr 172.21.15.116:0/2732600821 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:17.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.810+0000 7f1c222b3640 1 --2- 172.21.15.116:0/2732600821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b57c0dc680 0x55b57c0dea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.810+0000 7f1c21ab2640 1 --2- 172.21.15.116:0/2732600821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b57c0d14a0 0x55b57c0d1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.810+0000 7f1c212b1640 1 -- 172.21.15.116:0/2732600821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b57c0dc680 msgr2=0x55b57c0dea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:17.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.810+0000 7f1c212b1640 1 --2- 172.21.15.116:0/2732600821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b57c0dc680 0x55b57c0dea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.810+0000 7f1c212b1640 1 -- 172.21.15.116:0/2732600821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b57c0d14a0 msgr2=0x55b57c0d1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:17.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.810+0000 7f1c212b1640 1 --2- 172.21.15.116:0/2732600821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b57c0d14a0 0x55b57c0d1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.810+0000 7f1c212b1640 1 -- 172.21.15.116:0/2732600821 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b57bfc3440 con 0x55b57c0d1dc0 2024-09-12T03:17:17.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.811+0000 7f1c222b3640 1 --2- 172.21.15.116:0/1949623883 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1c080a2c00 0x7f1c080a50c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1c14005c40 tx=0x7f1c140129e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:17.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.811+0000 7f1c212b1640 1 --2- 172.21.15.116:0/2732600821 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b57c0d1dc0 0x55b57c0dc140 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f1c180cbf50 tx=0x7f1c18090190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f752183683cc3e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:17.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.811+0000 7f1c07fff640 1 -- 172.21.15.116:0/2732600821 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c180b2520 con 0x55b57c0d1dc0 2024-09-12T03:17:17.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.811+0000 7f1c07fff640 1 -- 172.21.15.116:0/2732600821 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c1809b3b0 con 0x55b57c0d1dc0 2024-09-12T03:17:17.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.811+0000 7f1c07fff640 1 -- 172.21.15.116:0/2732600821 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18013170 con 0x55b57c0d1dc0 2024-09-12T03:17:17.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.811+0000 7f1c260ef740 1 -- 172.21.15.116:0/2732600821 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b57c0d1dc0 msgr2=0x55b57c0dc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.811+0000 7f1c260ef740 1 --2- 172.21.15.116:0/2732600821 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b57c0d1dc0 0x55b57c0dc140 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f1c180cbf50 tx=0x7f1c18090190 comp rx=0 tx=0).stop 2024-09-12T03:17:17.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.812+0000 7f1c260ef740 1 -- 172.21.15.116:0/2732600821 shutdown_connections 2024-09-12T03:17:17.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.812+0000 7f1c260ef740 1 --2- 172.21.15.116:0/2732600821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b57c0dc680 0x55b57c0dea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.812+0000 7f1c260ef740 1 --2- 172.21.15.116:0/2732600821 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b57c0d1dc0 0x55b57c0dc140 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.812+0000 7f1c260ef740 1 --2- 172.21.15.116:0/2732600821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b57c0d14a0 0x55b57c0d1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.812+0000 7f1c260ef740 1 -- 172.21.15.116:0/2732600821 >> 172.21.15.116:0/2732600821 conn(0x55b57c0cda30 msgr2=0x55b57c0cde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:17.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.812+0000 7f1c260ef740 1 -- 172.21.15.116:0/2732600821 shutdown_connections 2024-09-12T03:17:17.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.812+0000 7f1c260ef740 1 -- 172.21.15.116:0/2732600821 wait complete. 2024-09-12T03:17:17.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.812+0000 7f1c260ef740 1 -- start start 2024-09-12T03:17:17.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.812+0000 7f1c260ef740 1 Processor -- start 2024-09-12T03:17:17.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.812+0000 7f1c260ef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b57c0d14a0 0x55b57c0dbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.812+0000 7f1c260ef740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b57c0d1dc0 0x55b57c0d9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.812+0000 7f1c260ef740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b57c0dc680 0x55b57c0d9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.812+0000 7f1c260ef740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b57c0df250 con 0x55b57c0d1dc0 2024-09-12T03:17:17.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.812+0000 7f1c260ef740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b57bff2050 con 0x55b57c0dc680 2024-09-12T03:17:17.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.812+0000 7f1c260ef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b57c0df3d0 con 0x55b57c0d14a0 2024-09-12T03:17:17.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.813+0000 7f1c260ef740 10 client.0 _ll_register_callbacks cb 0x55b57c032160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:17.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.813+0000 7f1c212b1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b57c0d1dc0 0x55b57c0d9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.813+0000 7f1c212b1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b57c0d1dc0 0x55b57c0d9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46112/0 (socket says 172.21.15.116:46112) 2024-09-12T03:17:17.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.813+0000 7f1c212b1640 1 -- 172.21.15.116:0/1010302527 learned_addr learned my addr 172.21.15.116:0/1010302527 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:17.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.813+0000 7f1c222b3640 1 --2- 172.21.15.116:0/1010302527 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b57c0dc680 0x55b57c0d9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.813+0000 7f1c21ab2640 1 --2- 172.21.15.116:0/1010302527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b57c0d14a0 0x55b57c0dbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.813+0000 7f1c212b1640 1 -- 172.21.15.116:0/1010302527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b57c0d14a0 msgr2=0x55b57c0dbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:17.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.813+0000 7f1c212b1640 1 --2- 172.21.15.116:0/1010302527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b57c0d14a0 0x55b57c0dbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.813+0000 7f1c212b1640 1 -- 172.21.15.116:0/1010302527 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b57c0dc680 msgr2=0x55b57c0d9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:17.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.813+0000 7f1c212b1640 1 --2- 172.21.15.116:0/1010302527 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b57c0dc680 0x55b57c0d9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.813+0000 7f1c212b1640 1 -- 172.21.15.116:0/1010302527 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b57bda83c0 con 0x55b57c0d1dc0 2024-09-12T03:17:17.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.813+0000 7f1c21ab2640 1 --2- 172.21.15.116:0/1010302527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b57c0d14a0 0x55b57c0dbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:17.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.813+0000 7f1c212b1640 1 --2- 172.21.15.116:0/1010302527 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b57c0d1dc0 0x55b57c0d9240 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f1c180cbfa0 tx=0x7f1c180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483f5b384dff56e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:17.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.813+0000 7f1c057fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:17.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.813+0000 7f1c057fa640 1 -- 172.21.15.116:0/1010302527 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1801e070 con 0x55b57c0d1dc0 2024-09-12T03:17:17.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.813+0000 7f1c057fa640 1 -- 172.21.15.116:0/1010302527 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1c18098d50 con 0x55b57c0d1dc0 2024-09-12T03:17:17.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.813+0000 7f1c260ef740 1 -- 172.21.15.116:0/1010302527 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c1809b3b0 con 0x55b57c0d1dc0 2024-09-12T03:17:17.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.813+0000 7f1c057fa640 1 -- 172.21.15.116:0/1010302527 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1c180b3eb0 con 0x55b57c0d1dc0 2024-09-12T03:17:17.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.813+0000 7f1c260ef740 10 client.19176 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:17.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.813+0000 7f1c260ef740 1 -- 172.21.15.116:0/1010302527 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b57c0eabd0 con 0x55b57c0d1dc0 2024-09-12T03:17:17.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.814+0000 7f1c057fa640 1 -- 172.21.15.116:0/1010302527 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c1809b3b0 con 0x55b57c0d1dc0 2024-09-12T03:17:17.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.814+0000 7f1c057fa640 1 -- 172.21.15.116:0/1010302527 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1c180500b0 con 0x55b57c0d1dc0 2024-09-12T03:17:17.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.814+0000 7f1c260ef740 10 client.19176 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:17.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.814+0000 7f1c260ef740 10 client.19176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:17:17.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.814+0000 7f1c260ef740 20 client.19176 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:17.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.814+0000 7f1c260ef740 1 --2- 172.21.15.116:0/1010302527 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b57c0ebf20 0x55b57c0ee310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.814+0000 7f1c260ef740 4 client.19176 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:17:17.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.814+0000 7f1c260ef740 1 -- 172.21.15.116:0/1010302527 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b57c044850 con 0x55b57c0ebf20 2024-09-12T03:17:17.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.814+0000 7f1c21ab2640 1 --2- 172.21.15.116:0/1010302527 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b57c0ebf20 0x55b57c0ee310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.814+0000 7f1c21ab2640 1 --2- 172.21.15.116:0/1010302527 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b57c0ebf20 0x55b57c0ee310 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c814d17a7d7e97 server_cookie=994166d4c4892151 in_seq=0 out_seq=0 2024-09-12T03:17:17.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.814+0000 7f1c057fa640 10 client.19176 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:17.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.815+0000 7f1c057fa640 1 -- 172.21.15.116:0/1010302527 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b57c044850 con 0x55b57c0ebf20 2024-09-12T03:17:17.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.815+0000 7f1c057fa640 10 client.19176 handle_command_reply: tid=0 2024-09-12T03:17:17.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.815+0000 7f1c260ef740 1 client.19176 shutdown 2024-09-12T03:17:17.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.815+0000 7f1c260ef740 1 -- 172.21.15.116:0/1010302527 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b57c0d1dc0 msgr2=0x55b57c0d9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.815+0000 7f1c260ef740 1 --2- 172.21.15.116:0/1010302527 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b57c0d1dc0 0x55b57c0d9240 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f1c180cbfa0 tx=0x7f1c180046c0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.815+0000 7f1c260ef740 1 -- 172.21.15.116:0/1010302527 shutdown_connections 2024-09-12T03:17:17.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.815+0000 7f1c260ef740 1 --2- 172.21.15.116:0/1010302527 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b57c0ebf20 0x55b57c0ee310 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.815+0000 7f1c260ef740 1 --2- 172.21.15.116:0/1010302527 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b57c0dc680 0x55b57c0d9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.815+0000 7f1c260ef740 1 --2- 172.21.15.116:0/1010302527 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b57c0d1dc0 0x55b57c0d9240 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.815+0000 7f1c057fa640 0 client.19176 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:17.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.815+0000 7f1c260ef740 1 --2- 172.21.15.116:0/1010302527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b57c0d14a0 0x55b57c0dbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.815+0000 7f1c260ef740 1 -- 172.21.15.116:0/1010302527 >> 172.21.15.116:0/1010302527 conn(0x55b57c0cda30 msgr2=0x55b57c0d3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:17.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.815+0000 7f1c260ef740 1 -- 172.21.15.116:0/1010302527 shutdown_connections 2024-09-12T03:17:17.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.815+0000 7f1c260ef740 1 -- 172.21.15.116:0/1010302527 wait complete. 2024-09-12T03:17:17.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.816+0000 7f1c260ef740 20 client.19176 trim_cache size 0 max 16384 2024-09-12T03:17:17.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.832+0000 7f1c260ef740 1 Processor -- start 2024-09-12T03:17:17.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.833+0000 7f1c260ef740 1 -- start start 2024-09-12T03:17:17.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.833+0000 7f1c260ef740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b57c0d14a0 0x55b57c0daaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.833+0000 7f1c260ef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b57c0d1dc0 0x55b57c0f1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.833+0000 7f1c260ef740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b57c0dc680 0x55b57c0f1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.833+0000 7f1c260ef740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c1afd70 con 0x55b57c0d14a0 2024-09-12T03:17:17.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.833+0000 7f1c260ef740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c1ad640 con 0x55b57c0dc680 2024-09-12T03:17:17.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.833+0000 7f1c260ef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c1ab490 con 0x55b57c0d1dc0 2024-09-12T03:17:17.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.833+0000 7f1c21ab2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b57c0d14a0 0x55b57c0daaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.833+0000 7f1c21ab2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b57c0d14a0 0x55b57c0daaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46116/0 (socket says 172.21.15.116:46116) 2024-09-12T03:17:17.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.833+0000 7f1c21ab2640 1 -- 172.21.15.116:0/1969415632 learned_addr learned my addr 172.21.15.116:0/1969415632 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:17.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.833+0000 7f1c212b1640 1 --2- 172.21.15.116:0/1969415632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b57c0d1dc0 0x55b57c0f1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.833+0000 7f1c222b3640 1 --2- 172.21.15.116:0/1969415632 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b57c0dc680 0x55b57c0f1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.833+0000 7f1c21ab2640 1 -- 172.21.15.116:0/1969415632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b57c0d1dc0 msgr2=0x55b57c0f1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:17.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.833+0000 7f1c21ab2640 1 --2- 172.21.15.116:0/1969415632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b57c0d1dc0 0x55b57c0f1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.833+0000 7f1c21ab2640 1 -- 172.21.15.116:0/1969415632 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b57c0dc680 msgr2=0x55b57c0f1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:17.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.833+0000 7f1c21ab2640 1 --2- 172.21.15.116:0/1969415632 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b57c0dc680 0x55b57c0f1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.833+0000 7f1c21ab2640 1 -- 172.21.15.116:0/1969415632 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b57c0db950 con 0x55b57c0d14a0 2024-09-12T03:17:17.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.833+0000 7f1c21ab2640 1 --2- 172.21.15.116:0/1969415632 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b57c0d14a0 0x55b57c0daaa0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c02a930 tx=0x7f1c0c0314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ab3c98378249fc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:17.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.834+0000 7f1c077fe640 1 -- 172.21.15.116:0/1969415632 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c048070 con 0x55b57c0d14a0 2024-09-12T03:17:17.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.834+0000 7f1c077fe640 1 -- 172.21.15.116:0/1969415632 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c0c005500 con 0x55b57c0d14a0 2024-09-12T03:17:17.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.834+0000 7f1c077fe640 1 -- 172.21.15.116:0/1969415632 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c035040 con 0x55b57c0d14a0 2024-09-12T03:17:17.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.834+0000 7f1c260ef740 1 -- 172.21.15.116:0/1969415632 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b57c0d14a0 msgr2=0x55b57c0daaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.834+0000 7f1c260ef740 1 --2- 172.21.15.116:0/1969415632 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b57c0d14a0 0x55b57c0daaa0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c02a930 tx=0x7f1c0c0314e0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.834+0000 7f1c260ef740 1 -- 172.21.15.116:0/1969415632 shutdown_connections 2024-09-12T03:17:17.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.834+0000 7f1c260ef740 1 --2- 172.21.15.116:0/1969415632 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b57c0dc680 0x55b57c0f1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.834+0000 7f1c260ef740 1 --2- 172.21.15.116:0/1969415632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b57c0d1dc0 0x55b57c0f1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.834+0000 7f1c260ef740 1 --2- 172.21.15.116:0/1969415632 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b57c0d14a0 0x55b57c0daaa0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.834+0000 7f1c260ef740 1 -- 172.21.15.116:0/1969415632 >> 172.21.15.116:0/1969415632 conn(0x55b57c0cda30 msgr2=0x55b57c0ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:17.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.834+0000 7f1c260ef740 1 -- 172.21.15.116:0/1969415632 shutdown_connections 2024-09-12T03:17:17.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.834+0000 7f1c260ef740 1 -- 172.21.15.116:0/1969415632 wait complete. 2024-09-12T03:17:17.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.835+0000 7f1c260ef740 1 -- start start 2024-09-12T03:17:17.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.835+0000 7f1c260ef740 1 Processor -- start 2024-09-12T03:17:17.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.835+0000 7f1c260ef740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b57c0d14a0 0x55b57c0f1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.835+0000 7f1c260ef740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b57c0d1dc0 0x55b57c0f16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.835+0000 7f1c260ef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b57c0dc680 0x55b57c14ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.835+0000 7f1c260ef740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c1abb00 con 0x55b57c0d1dc0 2024-09-12T03:17:17.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.835+0000 7f1c260ef740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c1acf70 con 0x55b57c0d14a0 2024-09-12T03:17:17.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.835+0000 7f1c260ef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c1ac1d0 con 0x55b57c0dc680 2024-09-12T03:17:17.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.835+0000 7f1c260ef740 10 client.0 _ll_register_callbacks cb 0x7f1c18096f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:17.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.835+0000 7f1c212b1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b57c0d1dc0 0x55b57c0f16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.835+0000 7f1c212b1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b57c0d1dc0 0x55b57c0f16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46118/0 (socket says 172.21.15.116:46118) 2024-09-12T03:17:17.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.835+0000 7f1c212b1640 1 -- 172.21.15.116:0/1759251164 learned_addr learned my addr 172.21.15.116:0/1759251164 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:17.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.835+0000 7f1c21ab2640 1 --2- 172.21.15.116:0/1759251164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b57c0d14a0 0x55b57c0f1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.835+0000 7f1c222b3640 1 --2- 172.21.15.116:0/1759251164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b57c0dc680 0x55b57c14ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.835+0000 7f1c212b1640 1 -- 172.21.15.116:0/1759251164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b57c0dc680 msgr2=0x55b57c14ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:17.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.835+0000 7f1c212b1640 1 --2- 172.21.15.116:0/1759251164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b57c0dc680 0x55b57c14ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.835+0000 7f1c212b1640 1 -- 172.21.15.116:0/1759251164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b57c0d14a0 msgr2=0x55b57c0f1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:17.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.835+0000 7f1c212b1640 1 --2- 172.21.15.116:0/1759251164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b57c0d14a0 0x55b57c0f1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.835+0000 7f1c212b1640 1 -- 172.21.15.116:0/1759251164 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b57c14e460 con 0x55b57c0d1dc0 2024-09-12T03:17:17.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.836+0000 7f1c212b1640 1 --2- 172.21.15.116:0/1759251164 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b57c0d1dc0 0x55b57c0f16b0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f1c180cc2c0 tx=0x7f1c180b2380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90799c5c7e2ce7fa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:17.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.836+0000 7f1c057fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:17.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.836+0000 7f1c057fa640 1 -- 172.21.15.116:0/1759251164 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1801e070 con 0x55b57c0d1dc0 2024-09-12T03:17:17.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.836+0000 7f1c057fa640 1 -- 172.21.15.116:0/1759251164 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1c18097320 con 0x55b57c0d1dc0 2024-09-12T03:17:17.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.836+0000 7f1c260ef740 1 -- 172.21.15.116:0/1759251164 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c0c005500 con 0x55b57c0d1dc0 2024-09-12T03:17:17.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.836+0000 7f1c057fa640 1 -- 172.21.15.116:0/1759251164 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1c180b4090 con 0x55b57c0d1dc0 2024-09-12T03:17:17.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.836+0000 7f1c260ef740 10 client.19182 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:17.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.836+0000 7f1c260ef740 1 -- 172.21.15.116:0/1759251164 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b57c156a00 con 0x55b57c0d1dc0 2024-09-12T03:17:17.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.836+0000 7f1c057fa640 1 -- 172.21.15.116:0/1759251164 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c0c005500 con 0x55b57c0d1dc0 2024-09-12T03:17:17.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.836+0000 7f1c057fa640 1 -- 172.21.15.116:0/1759251164 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1c180ae020 con 0x55b57c0d1dc0 2024-09-12T03:17:17.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.836+0000 7f1c260ef740 10 client.19182 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:17.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.836+0000 7f1c260ef740 10 client.19182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:17:17.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.836+0000 7f1c260ef740 20 client.19182 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:17.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.836+0000 7f1c260ef740 1 --2- 172.21.15.116:0/1759251164 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b57c156ed0 0x55b57c1592c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.836+0000 7f1c260ef740 4 client.19182 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:17:17.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.836+0000 7f1c260ef740 1 -- 172.21.15.116:0/1759251164 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b57c0d9b10 con 0x55b57c156ed0 2024-09-12T03:17:17.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.836+0000 7f1c21ab2640 1 --2- 172.21.15.116:0/1759251164 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b57c156ed0 0x55b57c1592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.837+0000 7f1c057fa640 10 client.19182 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:17.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.837+0000 7f1c21ab2640 1 --2- 172.21.15.116:0/1759251164 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b57c156ed0 0x55b57c1592c0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8536c51d88dbde8 server_cookie=c21a1b16776b6df2 in_seq=0 out_seq=0 2024-09-12T03:17:17.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.837+0000 7f1c057fa640 1 -- 172.21.15.116:0/1759251164 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b57c0d9b10 con 0x55b57c156ed0 2024-09-12T03:17:17.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.837+0000 7f1c057fa640 10 client.19182 handle_command_reply: tid=0 2024-09-12T03:17:17.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.837+0000 7f1c260ef740 1 client.19182 shutdown 2024-09-12T03:17:17.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.837+0000 7f1c260ef740 1 -- 172.21.15.116:0/1759251164 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b57c0d1dc0 msgr2=0x55b57c0f16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.837+0000 7f1c260ef740 1 --2- 172.21.15.116:0/1759251164 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b57c0d1dc0 0x55b57c0f16b0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f1c180cc2c0 tx=0x7f1c180b2380 comp rx=0 tx=0).stop 2024-09-12T03:17:17.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.837+0000 7f1c260ef740 1 -- 172.21.15.116:0/1759251164 shutdown_connections 2024-09-12T03:17:17.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.837+0000 7f1c260ef740 1 --2- 172.21.15.116:0/1759251164 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55b57c156ed0 0x55b57c1592c0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.947 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:17:17.947 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:17:17.947 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:17:17.947 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:17:17.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.837+0000 7f1c260ef740 1 --2- 172.21.15.116:0/1759251164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b57c0dc680 0x55b57c14ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.837+0000 7f1c260ef740 1 --2- 172.21.15.116:0/1759251164 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b57c0d1dc0 0x55b57c0f16b0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.837+0000 7f1c057fa640 0 client.19182 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:17.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.837+0000 7f1c260ef740 1 --2- 172.21.15.116:0/1759251164 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b57c0d14a0 0x55b57c0f1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.837+0000 7f1c260ef740 1 -- 172.21.15.116:0/1759251164 >> 172.21.15.116:0/1759251164 conn(0x55b57c0cda30 msgr2=0x55b57c0d7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:17.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.837+0000 7f1c260ef740 1 -- 172.21.15.116:0/1759251164 shutdown_connections 2024-09-12T03:17:17.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.837+0000 7f1c260ef740 1 -- 172.21.15.116:0/1759251164 wait complete. 2024-09-12T03:17:17.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.837+0000 7f1c260ef740 20 client.19182 trim_cache size 0 max 16384 2024-09-12T03:17:17.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.838+0000 7f1c23d16640 1 -- 172.21.15.116:0/1949623883 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1c080a2c00 msgr2=0x7f1c080a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.838+0000 7f1c23d16640 1 --2- 172.21.15.116:0/1949623883 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1c080a2c00 0x7f1c080a50c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1c14005c40 tx=0x7f1c140129e0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.838+0000 7f1c23d16640 1 -- 172.21.15.116:0/1949623883 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c1c102790 msgr2=0x7f1c1c198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.838+0000 7f1c23d16640 1 --2- 172.21.15.116:0/1949623883 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c1c102790 0x7f1c1c198a10 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f1c1800b4a0 tx=0x7f1c1800b960 comp rx=0 tx=0).stop 2024-09-12T03:17:17.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.838+0000 7f1c23d16640 1 -- 172.21.15.116:0/1949623883 shutdown_connections 2024-09-12T03:17:17.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.838+0000 7f1c23d16640 1 --2- 172.21.15.116:0/1949623883 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1c080a2c00 0x7f1c080a50c0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.838+0000 7f1c23d16640 1 --2- 172.21.15.116:0/1949623883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c1c106f10 0x7f1c1c19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.838+0000 7f1c23d16640 1 --2- 172.21.15.116:0/1949623883 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c1c102790 0x7f1c1c198a10 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.838+0000 7f1c23d16640 1 --2- 172.21.15.116:0/1949623883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c1c101de0 0x7f1c1c1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.838+0000 7f1c23d16640 1 -- 172.21.15.116:0/1949623883 >> 172.21.15.116:0/1949623883 conn(0x7f1c1c0fd8c0 msgr2=0x7f1c1c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:17.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.838+0000 7f1c23d16640 1 -- 172.21.15.116:0/1949623883 shutdown_connections 2024-09-12T03:17:17.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:17.838+0000 7f1c23d16640 1 -- 172.21.15.116:0/1949623883 wait complete. 2024-09-12T03:17:18.857 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:17:18.858 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:17:19.103 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:19.508 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:19 smithi116 ceph-mon[31410]: pgmap v768: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:17:19.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.587+0000 7f73f307b640 1 -- 172.21.15.116:0/1628547069 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73ec100a80 msgr2=0x7f73ec100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:19.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.587+0000 7f73f307b640 1 --2- 172.21.15.116:0/1628547069 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73ec100a80 0x7f73ec100ee0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f73e0009930 tx=0x7f73e002a420 comp rx=0 tx=0).stop 2024-09-12T03:17:19.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.587+0000 7f73f307b640 1 -- 172.21.15.116:0/1628547069 shutdown_connections 2024-09-12T03:17:19.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.587+0000 7f73f307b640 1 --2- 172.21.15.116:0/1628547069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73ec101420 0x7f73ec10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.587+0000 7f73f307b640 1 --2- 172.21.15.116:0/1628547069 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73ec100a80 0x7f73ec100ee0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.587+0000 7f73f307b640 1 --2- 172.21.15.116:0/1628547069 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73ec106a80 0x7f73ec106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.587+0000 7f73f307b640 1 -- 172.21.15.116:0/1628547069 >> 172.21.15.116:0/1628547069 conn(0x7f73ec0fc7a0 msgr2=0x7f73ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:19.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.587+0000 7f73f307b640 1 -- 172.21.15.116:0/1628547069 shutdown_connections 2024-09-12T03:17:19.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.587+0000 7f73f307b640 1 -- 172.21.15.116:0/1628547069 wait complete. 2024-09-12T03:17:19.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.588+0000 7f73f307b640 1 Processor -- start 2024-09-12T03:17:19.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.588+0000 7f73f307b640 1 -- start start 2024-09-12T03:17:19.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.588+0000 7f73f307b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73ec100a80 0x7f73ec198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:19.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.588+0000 7f73f307b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73ec101420 0x7f73ec1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:19.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.588+0000 7f73f307b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73ec106a80 0x7f73ec19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:19.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.588+0000 7f73f307b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec110ea0 con 0x7f73ec100a80 2024-09-12T03:17:19.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.588+0000 7f73f307b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec110d20 con 0x7f73ec101420 2024-09-12T03:17:19.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.588+0000 7f73f307b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec111020 con 0x7f73ec106a80 2024-09-12T03:17:19.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.588+0000 7f73f0e17640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73ec100a80 0x7f73ec198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:19.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.588+0000 7f73f0e17640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73ec100a80 0x7f73ec198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46140/0 (socket says 172.21.15.116:46140) 2024-09-12T03:17:19.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.588+0000 7f73f0e17640 1 -- 172.21.15.116:0/1329599378 learned_addr learned my addr 172.21.15.116:0/1329599378 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:19.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.588+0000 7f73f1618640 1 --2- 172.21.15.116:0/1329599378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73ec106a80 0x7f73ec19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:19.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.588+0000 7f73ebfff640 1 --2- 172.21.15.116:0/1329599378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73ec101420 0x7f73ec1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:19.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.588+0000 7f73f0e17640 1 -- 172.21.15.116:0/1329599378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73ec106a80 msgr2=0x7f73ec19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:19.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.588+0000 7f73f0e17640 1 --2- 172.21.15.116:0/1329599378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73ec106a80 0x7f73ec19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.588+0000 7f73f0e17640 1 -- 172.21.15.116:0/1329599378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73ec101420 msgr2=0x7f73ec1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:19.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.588+0000 7f73f0e17640 1 --2- 172.21.15.116:0/1329599378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73ec101420 0x7f73ec1989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.588+0000 7f73f0e17640 1 -- 172.21.15.116:0/1329599378 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ec19d420 con 0x7f73ec100a80 2024-09-12T03:17:19.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.589+0000 7f73f0e17640 1 --2- 172.21.15.116:0/1329599378 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73ec100a80 0x7f73ec198470 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f73dc00c920 tx=0x7f73dc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:19.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.589+0000 7f73e9ffb640 1 -- 172.21.15.116:0/1329599378 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc007c70 con 0x7f73ec100a80 2024-09-12T03:17:19.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.589+0000 7f73e9ffb640 1 -- 172.21.15.116:0/1329599378 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73dc007e10 con 0x7f73ec100a80 2024-09-12T03:17:19.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.589+0000 7f73f307b640 1 -- 172.21.15.116:0/1329599378 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73ec19d710 con 0x7f73ec100a80 2024-09-12T03:17:19.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.589+0000 7f73e9ffb640 1 -- 172.21.15.116:0/1329599378 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc018890 con 0x7f73ec100a80 2024-09-12T03:17:19.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.589+0000 7f73f307b640 1 -- 172.21.15.116:0/1329599378 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73ec1a4ff0 con 0x7f73ec100a80 2024-09-12T03:17:19.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.590+0000 7f73f307b640 1 -- 172.21.15.116:0/1329599378 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73ec100ee0 con 0x7f73ec100a80 2024-09-12T03:17:19.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.591+0000 7f73e9ffb640 1 -- 172.21.15.116:0/1329599378 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f73dc018a70 con 0x7f73ec100a80 2024-09-12T03:17:19.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.591+0000 7f73e9ffb640 1 --2- 172.21.15.116:0/1329599378 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f73d40a2b80 0x7f73d40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:19.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.592+0000 7f73ebfff640 1 --2- 172.21.15.116:0/1329599378 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f73d40a2b80 0x7f73d40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:19.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.592+0000 7f73e9ffb640 1 -- 172.21.15.116:0/1329599378 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f73dc0cf210 con 0x7f73ec100a80 2024-09-12T03:17:19.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.592+0000 7f73ebfff640 1 --2- 172.21.15.116:0/1329599378 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f73d40a2b80 0x7f73d40a5040 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f73e00096c0 tx=0x7f73e0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:19.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.596+0000 7f73e9ffb640 1 -- 172.21.15.116:0/1329599378 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f73dc014030 con 0x7f73ec100a80 2024-09-12T03:17:19.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:19 smithi183 ceph-mon[35535]: pgmap v768: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:17:19.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:19 smithi177 ceph-mon[35305]: pgmap v768: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:17:19.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.795+0000 7f73f307b640 1 -- 172.21.15.116:0/1329599378 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73ec109a90 con 0x7f73ec100a80 2024-09-12T03:17:19.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.796+0000 7f73e9ffb640 1 -- 172.21.15.116:0/1329599378 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f73dc00de20 con 0x7f73ec100a80 2024-09-12T03:17:19.797 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:17:19.797 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:17:19.797 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:17:19.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.798+0000 7f73f307b640 1 -- 172.21.15.116:0/1329599378 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f73d40a2b80 msgr2=0x7f73d40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:19.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.798+0000 7f73f307b640 1 --2- 172.21.15.116:0/1329599378 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f73d40a2b80 0x7f73d40a5040 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f73e00096c0 tx=0x7f73e0000bc0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.798+0000 7f73f307b640 1 -- 172.21.15.116:0/1329599378 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73ec100a80 msgr2=0x7f73ec198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:19.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.798+0000 7f73f307b640 1 --2- 172.21.15.116:0/1329599378 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73ec100a80 0x7f73ec198470 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f73dc00c920 tx=0x7f73dc00cde0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.799+0000 7f73f307b640 1 -- 172.21.15.116:0/1329599378 shutdown_connections 2024-09-12T03:17:19.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.799+0000 7f73f307b640 1 --2- 172.21.15.116:0/1329599378 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f73d40a2b80 0x7f73d40a5040 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.799+0000 7f73f307b640 1 --2- 172.21.15.116:0/1329599378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73ec106a80 0x7f73ec19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.799+0000 7f73f307b640 1 --2- 172.21.15.116:0/1329599378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73ec101420 0x7f73ec1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.799+0000 7f73f307b640 1 --2- 172.21.15.116:0/1329599378 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73ec100a80 0x7f73ec198470 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.799+0000 7f73f307b640 1 -- 172.21.15.116:0/1329599378 >> 172.21.15.116:0/1329599378 conn(0x7f73ec0fc7a0 msgr2=0x7f73ec06a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:19.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.799+0000 7f73f307b640 1 -- 172.21.15.116:0/1329599378 shutdown_connections 2024-09-12T03:17:19.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:19.799+0000 7f73f307b640 1 -- 172.21.15.116:0/1329599378 wait complete. 2024-09-12T03:17:20.630 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:17:20.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:20 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1329599378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:17:20.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:20 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1329599378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:17:20.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:20 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1329599378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:17:20.883 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:21.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.349+0000 7f452bf0d640 1 -- 172.21.15.116:0/1615392813 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4524100a50 msgr2=0x7f45241057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:21.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.349+0000 7f452bf0d640 1 --2- 172.21.15.116:0/1615392813 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4524100a50 0x7f45241057e0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f451c009ca0 tx=0x7f451c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.349+0000 7f452bf0d640 1 -- 172.21.15.116:0/1615392813 shutdown_connections 2024-09-12T03:17:21.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.349+0000 7f452bf0d640 1 --2- 172.21.15.116:0/1615392813 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4524100a50 0x7f45241057e0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.349+0000 7f452bf0d640 1 --2- 172.21.15.116:0/1615392813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4524100090 0x7f4524100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.349+0000 7f452bf0d640 1 --2- 172.21.15.116:0/1615392813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4524106b70 0x7f4524106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.349+0000 7f452bf0d640 1 -- 172.21.15.116:0/1615392813 >> 172.21.15.116:0/1615392813 conn(0x7f4524077210 msgr2=0x7f45240fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:21.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.349+0000 7f452bf0d640 1 -- 172.21.15.116:0/1615392813 shutdown_connections 2024-09-12T03:17:21.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.349+0000 7f452bf0d640 1 -- 172.21.15.116:0/1615392813 wait complete. 2024-09-12T03:17:21.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.350+0000 7f452bf0d640 1 Processor -- start 2024-09-12T03:17:21.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.350+0000 7f452bf0d640 1 -- start start 2024-09-12T03:17:21.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.350+0000 7f452bf0d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4524100090 0x7f45241a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.350+0000 7f452bf0d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4524100a50 0x7f45241a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.350+0000 7f452bf0d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4524106b70 0x7f452419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.350+0000 7f452bf0d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f45241185c0 con 0x7f4524100a50 2024-09-12T03:17:21.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.350+0000 7f452bf0d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4524118440 con 0x7f4524100090 2024-09-12T03:17:21.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.350+0000 7f452bf0d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4524118740 con 0x7f4524106b70 2024-09-12T03:17:21.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.350+0000 7f45294a8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4524100a50 0x7f45241a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.350+0000 7f45294a8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4524100a50 0x7f45241a4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51936/0 (socket says 172.21.15.116:51936) 2024-09-12T03:17:21.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.350+0000 7f45294a8640 1 -- 172.21.15.116:0/3493413698 learned_addr learned my addr 172.21.15.116:0/3493413698 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:21.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.351+0000 7f452a4aa640 1 --2- 172.21.15.116:0/3493413698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4524106b70 0x7f452419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.351+0000 7f4529ca9640 1 --2- 172.21.15.116:0/3493413698 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4524100090 0x7f45241a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.351+0000 7f45294a8640 1 -- 172.21.15.116:0/3493413698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4524106b70 msgr2=0x7f452419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:21.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.351+0000 7f45294a8640 1 --2- 172.21.15.116:0/3493413698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4524106b70 0x7f452419e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.351+0000 7f45294a8640 1 -- 172.21.15.116:0/3493413698 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4524100090 msgr2=0x7f45241a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:21.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.351+0000 7f45294a8640 1 --2- 172.21.15.116:0/3493413698 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4524100090 0x7f45241a4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.351+0000 7f45294a8640 1 -- 172.21.15.116:0/3493413698 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452419ef90 con 0x7f4524100a50 2024-09-12T03:17:21.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.351+0000 7f45294a8640 1 --2- 172.21.15.116:0/3493413698 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4524100a50 0x7f45241a4bb0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f451400c890 tx=0x7f451400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:21.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.351+0000 7f451affd640 1 -- 172.21.15.116:0/3493413698 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4514004220 con 0x7f4524100a50 2024-09-12T03:17:21.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.351+0000 7f451affd640 1 -- 172.21.15.116:0/3493413698 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45140043c0 con 0x7f4524100a50 2024-09-12T03:17:21.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.351+0000 7f451affd640 1 -- 172.21.15.116:0/3493413698 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45140049e0 con 0x7f4524100a50 2024-09-12T03:17:21.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.351+0000 7f452bf0d640 1 -- 172.21.15.116:0/3493413698 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f452419f280 con 0x7f4524100a50 2024-09-12T03:17:21.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.351+0000 7f452bf0d640 1 -- 172.21.15.116:0/3493413698 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45241ab4d0 con 0x7f4524100a50 2024-09-12T03:17:21.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.353+0000 7f451affd640 1 -- 172.21.15.116:0/3493413698 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f45140079b0 con 0x7f4524100a50 2024-09-12T03:17:21.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.354+0000 7f452e2e6740 1 Processor -- start 2024-09-12T03:17:21.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.354+0000 7f452e2e6740 1 -- start start 2024-09-12T03:17:21.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.354+0000 7f451affd640 1 --2- 172.21.15.116:0/3493413698 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f45100a2c00 0x7f45100a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.354+0000 7f452e2e6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bf8c4bb40 0x555bf8c4a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.354+0000 7f452e2e6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555bf8c4a5d0 0x555bf8c4aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.354+0000 7f452e2e6740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555bf8c55370 0x555bf8c57760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.354+0000 7f452e2e6740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555bf8bb02f0 con 0x555bf8c55370 2024-09-12T03:17:21.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.354+0000 7f452e2e6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x555bf8b57cf0 con 0x555bf8c4a5d0 2024-09-12T03:17:21.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.354+0000 7f452e2e6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555bf8bac2b0 con 0x555bf8c4bb40 2024-09-12T03:17:21.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.354+0000 7f451affd640 1 -- 172.21.15.116:0/3493413698 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f45140ceff0 con 0x7f4524100a50 2024-09-12T03:17:21.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.354+0000 7f452a4aa640 1 --2- 172.21.15.116:0/3493413698 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f45100a2c00 0x7f45100a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.355+0000 7f45294a8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555bf8c4a5d0 0x555bf8c4aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.355+0000 7f452a4aa640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555bf8c55370 0x555bf8c57760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.355+0000 7f4529ca9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bf8c4bb40 0x555bf8c4a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.355+0000 7f45294a8640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555bf8c4a5d0 0x555bf8c4aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.116:37420/0 (socket says 172.21.15.116:37420) 2024-09-12T03:17:21.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.355+0000 7f45294a8640 1 -- 172.21.15.116:0/3756416511 learned_addr learned my addr 172.21.15.116:0/3756416511 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:21.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.355+0000 7f4529ca9640 1 -- 172.21.15.116:0/3756416511 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555bf8c4a5d0 msgr2=0x555bf8c4aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:21.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.355+0000 7f452a4aa640 1 --2- 172.21.15.116:0/3493413698 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f45100a2c00 0x7f45100a50c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f451c004af0 tx=0x7f451c030780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:21.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.355+0000 7f4529ca9640 1 --2- 172.21.15.116:0/3756416511 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555bf8c4a5d0 0x555bf8c4aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.355+0000 7f4529ca9640 1 -- 172.21.15.116:0/3756416511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555bf8c55370 msgr2=0x555bf8c57760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:21.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.355+0000 7f4529ca9640 1 --2- 172.21.15.116:0/3756416511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555bf8c55370 0x555bf8c57760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.355+0000 7f4529ca9640 1 -- 172.21.15.116:0/3756416511 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bf8b3c3a0 con 0x555bf8c4bb40 2024-09-12T03:17:21.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.355+0000 7f4529ca9640 1 --2- 172.21.15.116:0/3756416511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bf8c4bb40 0x555bf8c4a090 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f452000bbe0 tx=0x7f452000f360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b1890c6672fca53 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:21.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.355+0000 7f450ffff640 1 -- 172.21.15.116:0/3756416511 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520015070 con 0x555bf8c4bb40 2024-09-12T03:17:21.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.355+0000 7f450ffff640 1 -- 172.21.15.116:0/3756416511 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4520005500 con 0x555bf8c4bb40 2024-09-12T03:17:21.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.355+0000 7f450ffff640 1 -- 172.21.15.116:0/3756416511 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520011440 con 0x555bf8c4bb40 2024-09-12T03:17:21.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.356+0000 7f452e2e6740 1 -- 172.21.15.116:0/3756416511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bf8c4bb40 msgr2=0x555bf8c4a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:21.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.356+0000 7f452e2e6740 1 --2- 172.21.15.116:0/3756416511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bf8c4bb40 0x555bf8c4a090 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f452000bbe0 tx=0x7f452000f360 comp rx=0 tx=0).stop 2024-09-12T03:17:21.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.356+0000 7f452e2e6740 1 -- 172.21.15.116:0/3756416511 shutdown_connections 2024-09-12T03:17:21.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.356+0000 7f452e2e6740 1 --2- 172.21.15.116:0/3756416511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555bf8c55370 0x555bf8c57760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.356+0000 7f452e2e6740 1 --2- 172.21.15.116:0/3756416511 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555bf8c4a5d0 0x555bf8c4aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.356+0000 7f452e2e6740 1 --2- 172.21.15.116:0/3756416511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bf8c4bb40 0x555bf8c4a090 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.356+0000 7f452e2e6740 1 -- 172.21.15.116:0/3756416511 >> 172.21.15.116:0/3756416511 conn(0x555bf8c13310 msgr2=0x555bf8c46c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:21.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.356+0000 7f452e2e6740 1 -- 172.21.15.116:0/3756416511 shutdown_connections 2024-09-12T03:17:21.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.356+0000 7f452e2e6740 1 -- 172.21.15.116:0/3756416511 wait complete. 2024-09-12T03:17:21.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.357+0000 7f452e2e6740 1 -- start start 2024-09-12T03:17:21.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.357+0000 7f452e2e6740 1 Processor -- start 2024-09-12T03:17:21.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.357+0000 7f452e2e6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555bf8c4a5d0 0x555bf8c54cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.357+0000 7f452e2e6740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555bf8c4bb40 0x555bf8c51f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.357+0000 7f452e2e6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bf8c55370 0x555bf8c524e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.357+0000 7f452e2e6740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555bf8c58070 con 0x555bf8c4bb40 2024-09-12T03:17:21.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.357+0000 7f452e2e6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x555bf8b6b060 con 0x555bf8c4a5d0 2024-09-12T03:17:21.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.357+0000 7f452e2e6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555bf8c581f0 con 0x555bf8c55370 2024-09-12T03:17:21.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.357+0000 7f452e2e6740 10 client.0 _ll_register_callbacks cb 0x555bf88be8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:21.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.357+0000 7f45294a8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555bf8c4bb40 0x555bf8c51f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.357+0000 7f45294a8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555bf8c4bb40 0x555bf8c51f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51960/0 (socket says 172.21.15.116:51960) 2024-09-12T03:17:21.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.357+0000 7f45294a8640 1 -- 172.21.15.116:0/2296508062 learned_addr learned my addr 172.21.15.116:0/2296508062 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:21.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.357+0000 7f4529ca9640 1 --2- 172.21.15.116:0/2296508062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555bf8c4a5d0 0x555bf8c54cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.357+0000 7f452a4aa640 1 --2- 172.21.15.116:0/2296508062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bf8c55370 0x555bf8c524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.357+0000 7f45294a8640 1 -- 172.21.15.116:0/2296508062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bf8c55370 msgr2=0x555bf8c524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:21.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.357+0000 7f45294a8640 1 --2- 172.21.15.116:0/2296508062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bf8c55370 0x555bf8c524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.357+0000 7f45294a8640 1 -- 172.21.15.116:0/2296508062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555bf8c4a5d0 msgr2=0x555bf8c54cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:21.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.357+0000 7f45294a8640 1 --2- 172.21.15.116:0/2296508062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555bf8c4a5d0 0x555bf8c54cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.357+0000 7f45294a8640 1 -- 172.21.15.116:0/2296508062 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bf89213c0 con 0x555bf8c4bb40 2024-09-12T03:17:21.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.358+0000 7f452a4aa640 1 --2- 172.21.15.116:0/2296508062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bf8c55370 0x555bf8c524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:21.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.358+0000 7f4529ca9640 1 --2- 172.21.15.116:0/2296508062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555bf8c4a5d0 0x555bf8c54cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:21.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.358+0000 7f45294a8640 1 --2- 172.21.15.116:0/2296508062 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555bf8c4bb40 0x555bf8c51f80 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f451400ff40 tx=0x7f4514097be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9fb4833389e9fd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:21.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.358+0000 7f450d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:21.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.358+0000 7f450d7fa640 1 -- 172.21.15.116:0/2296508062 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451401d040 con 0x555bf8c4bb40 2024-09-12T03:17:21.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.358+0000 7f450d7fa640 1 -- 172.21.15.116:0/2296508062 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f4514007ca0 con 0x555bf8c4bb40 2024-09-12T03:17:21.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.358+0000 7f452e2e6740 1 -- 172.21.15.116:0/2296508062 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4520005500 con 0x555bf8c4bb40 2024-09-12T03:17:21.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.358+0000 7f450d7fa640 1 -- 172.21.15.116:0/2296508062 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f45140b9050 con 0x555bf8c4bb40 2024-09-12T03:17:21.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.358+0000 7f452e2e6740 10 client.19200 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:21.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.358+0000 7f452e2e6740 1 -- 172.21.15.116:0/2296508062 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555bf8c63b10 con 0x555bf8c4bb40 2024-09-12T03:17:21.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.358+0000 7f450d7fa640 1 -- 172.21.15.116:0/2296508062 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4520005500 con 0x555bf8c4bb40 2024-09-12T03:17:21.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.358+0000 7f450d7fa640 1 -- 172.21.15.116:0/2296508062 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f451409bae0 con 0x555bf8c4bb40 2024-09-12T03:17:21.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.358+0000 7f452e2e6740 10 client.19200 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:21.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.358+0000 7f452e2e6740 10 client.19200 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:17:21.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.358+0000 7f452e2e6740 20 client.19200 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:21.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.358+0000 7f452e2e6740 1 --2- 172.21.15.116:0/2296508062 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x555bf8c64f70 0x555bf8c67380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.358+0000 7f452e2e6740 4 client.19200 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:17:21.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.358+0000 7f452e2e6740 1 -- 172.21.15.116:0/2296508062 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bf8bbd860 con 0x555bf8c64f70 2024-09-12T03:17:21.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.359+0000 7f4529ca9640 1 --2- 172.21.15.116:0/2296508062 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x555bf8c64f70 0x555bf8c67380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.359+0000 7f4529ca9640 1 --2- 172.21.15.116:0/2296508062 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x555bf8c64f70 0x555bf8c67380 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6bd2baf70098273e server_cookie=667c059716bae98 in_seq=0 out_seq=0 2024-09-12T03:17:21.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.359+0000 7f450d7fa640 10 client.19200 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:17:21.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.360+0000 7f450d7fa640 1 -- 172.21.15.116:0/2296508062 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555bf8bbd860 con 0x555bf8c64f70 2024-09-12T03:17:21.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.360+0000 7f450d7fa640 10 client.19200 handle_command_reply: tid=0 2024-09-12T03:17:21.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.360+0000 7f452e2e6740 1 client.19200 shutdown 2024-09-12T03:17:21.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.360+0000 7f452e2e6740 1 -- 172.21.15.116:0/2296508062 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555bf8c4bb40 msgr2=0x555bf8c51f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:21.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.360+0000 7f452e2e6740 1 --2- 172.21.15.116:0/2296508062 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555bf8c4bb40 0x555bf8c51f80 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f451400ff40 tx=0x7f4514097be0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.360+0000 7f452e2e6740 1 -- 172.21.15.116:0/2296508062 shutdown_connections 2024-09-12T03:17:21.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.360+0000 7f452e2e6740 1 --2- 172.21.15.116:0/2296508062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bf8c55370 0x555bf8c524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.360+0000 7f452e2e6740 1 --2- 172.21.15.116:0/2296508062 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x555bf8c64f70 0x555bf8c67380 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.360+0000 7f452e2e6740 1 --2- 172.21.15.116:0/2296508062 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555bf8c4bb40 0x555bf8c51f80 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.360+0000 7f452e2e6740 1 --2- 172.21.15.116:0/2296508062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555bf8c4a5d0 0x555bf8c54cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.360+0000 7f450d7fa640 0 client.19200 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:17:21.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.360+0000 7f452e2e6740 1 -- 172.21.15.116:0/2296508062 >> 172.21.15.116:0/2296508062 conn(0x555bf8c13310 msgr2=0x555bf8c4c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:21.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.360+0000 7f452e2e6740 1 -- 172.21.15.116:0/2296508062 shutdown_connections 2024-09-12T03:17:21.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.360+0000 7f452e2e6740 1 -- 172.21.15.116:0/2296508062 wait complete. 2024-09-12T03:17:21.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.360+0000 7f452e2e6740 20 client.19200 trim_cache size 0 max 16384 2024-09-12T03:17:21.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.373+0000 7f452e2e6740 1 Processor -- start 2024-09-12T03:17:21.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.373+0000 7f452e2e6740 1 -- start start 2024-09-12T03:17:21.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.373+0000 7f452e2e6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bf8c4a5d0 0x555bf8c61340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.373+0000 7f452e2e6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555bf8c4bb40 0x555bf8c6cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.373+0000 7f452e2e6740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555bf8c55370 0x555bf8c6d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.373+0000 7f452e2e6740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f45241b62a0 con 0x555bf8c55370 2024-09-12T03:17:21.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.373+0000 7f452e2e6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f45241b3b70 con 0x555bf8c4bb40 2024-09-12T03:17:21.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.373+0000 7f452e2e6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f45241b19c0 con 0x555bf8c4a5d0 2024-09-12T03:17:21.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.373+0000 7f452a4aa640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555bf8c55370 0x555bf8c6d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.373+0000 7f452a4aa640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555bf8c55370 0x555bf8c6d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51970/0 (socket says 172.21.15.116:51970) 2024-09-12T03:17:21.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.373+0000 7f452a4aa640 1 -- 172.21.15.116:0/1764724953 learned_addr learned my addr 172.21.15.116:0/1764724953 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:21.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.373+0000 7f45294a8640 1 --2- 172.21.15.116:0/1764724953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555bf8c4bb40 0x555bf8c6cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.373+0000 7f452a4aa640 1 -- 172.21.15.116:0/1764724953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bf8c4a5d0 msgr2=0x555bf8c61340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:21.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.373+0000 7f452a4aa640 1 --2- 172.21.15.116:0/1764724953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bf8c4a5d0 0x555bf8c61340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.373+0000 7f452a4aa640 1 -- 172.21.15.116:0/1764724953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555bf8c4bb40 msgr2=0x555bf8c6cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:21.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.373+0000 7f452a4aa640 1 --2- 172.21.15.116:0/1764724953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555bf8c4bb40 0x555bf8c6cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.373+0000 7f452a4aa640 1 -- 172.21.15.116:0/1764724953 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bf8c55040 con 0x555bf8c55370 2024-09-12T03:17:21.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.373+0000 7f45294a8640 1 --2- 172.21.15.116:0/1764724953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555bf8c4bb40 0x555bf8c6cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:21.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.374+0000 7f452a4aa640 1 --2- 172.21.15.116:0/1764724953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555bf8c55370 0x555bf8c6d140 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f451c02ade0 tx=0x7f451c04a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bde4da890006cea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:21.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.374+0000 7f450f7fe640 1 -- 172.21.15.116:0/1764724953 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c030e90 con 0x555bf8c55370 2024-09-12T03:17:21.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.374+0000 7f450f7fe640 1 -- 172.21.15.116:0/1764724953 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f451c036040 con 0x555bf8c55370 2024-09-12T03:17:21.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.374+0000 7f450f7fe640 1 -- 172.21.15.116:0/1764724953 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c004690 con 0x555bf8c55370 2024-09-12T03:17:21.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.374+0000 7f452e2e6740 1 -- 172.21.15.116:0/1764724953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555bf8c55370 msgr2=0x555bf8c6d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:21.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.374+0000 7f452e2e6740 1 --2- 172.21.15.116:0/1764724953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555bf8c55370 0x555bf8c6d140 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f451c02ade0 tx=0x7f451c04a000 comp rx=0 tx=0).stop 2024-09-12T03:17:21.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.374+0000 7f452e2e6740 1 -- 172.21.15.116:0/1764724953 shutdown_connections 2024-09-12T03:17:21.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.374+0000 7f452e2e6740 1 --2- 172.21.15.116:0/1764724953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555bf8c55370 0x555bf8c6d140 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.374+0000 7f452e2e6740 1 --2- 172.21.15.116:0/1764724953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555bf8c4bb40 0x555bf8c6cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.374+0000 7f452e2e6740 1 --2- 172.21.15.116:0/1764724953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bf8c4a5d0 0x555bf8c61340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.374+0000 7f452e2e6740 1 -- 172.21.15.116:0/1764724953 >> 172.21.15.116:0/1764724953 conn(0x555bf8c13310 msgr2=0x555bf8c50850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:21.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.374+0000 7f452e2e6740 1 -- 172.21.15.116:0/1764724953 shutdown_connections 2024-09-12T03:17:21.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.374+0000 7f452e2e6740 1 -- 172.21.15.116:0/1764724953 wait complete. 2024-09-12T03:17:21.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.375+0000 7f452e2e6740 1 -- start start 2024-09-12T03:17:21.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.375+0000 7f452e2e6740 1 Processor -- start 2024-09-12T03:17:21.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.375+0000 7f452e2e6740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555bf8c4a5d0 0x555bf8c591b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.375+0000 7f452e2e6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555bf8c4bb40 0x555bf8c596f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.375+0000 7f452e2e6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bf8c55370 0x555bf8c59c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.375+0000 7f452e2e6740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f45241b2030 con 0x555bf8c4a5d0 2024-09-12T03:17:21.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.375+0000 7f452e2e6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f45241b34a0 con 0x555bf8c4bb40 2024-09-12T03:17:21.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.375+0000 7f452e2e6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f45241b2700 con 0x555bf8c55370 2024-09-12T03:17:21.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.375+0000 7f4529ca9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555bf8c4a5d0 0x555bf8c591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.375+0000 7f452e2e6740 10 client.0 _ll_register_callbacks cb 0x7f45140046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:21.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.375+0000 7f4529ca9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555bf8c4a5d0 0x555bf8c591b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51978/0 (socket says 172.21.15.116:51978) 2024-09-12T03:17:21.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.375+0000 7f4529ca9640 1 -- 172.21.15.116:0/254917821 learned_addr learned my addr 172.21.15.116:0/254917821 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:21.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.375+0000 7f4529ca9640 1 -- 172.21.15.116:0/254917821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bf8c55370 msgr2=0x555bf8c59c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:21.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.375+0000 7f4529ca9640 1 --2- 172.21.15.116:0/254917821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bf8c55370 0x555bf8c59c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.375+0000 7f4529ca9640 1 -- 172.21.15.116:0/254917821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555bf8c4bb40 msgr2=0x555bf8c596f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:21.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.375+0000 7f4529ca9640 1 --2- 172.21.15.116:0/254917821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555bf8c4bb40 0x555bf8c596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.375+0000 7f4529ca9640 1 -- 172.21.15.116:0/254917821 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bf8ccebd0 con 0x555bf8c4a5d0 2024-09-12T03:17:21.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.376+0000 7f4529ca9640 1 --2- 172.21.15.116:0/254917821 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555bf8c4a5d0 0x555bf8c591b0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f452000b970 tx=0x7f45200047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34f68f777aec9c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:21.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.376+0000 7f450d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:21.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.376+0000 7f450d7fa640 1 -- 172.21.15.116:0/254917821 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520015050 con 0x555bf8c4a5d0 2024-09-12T03:17:21.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.376+0000 7f450d7fa640 1 -- 172.21.15.116:0/254917821 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f4520025460 con 0x555bf8c4a5d0 2024-09-12T03:17:21.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.376+0000 7f452e2e6740 1 -- 172.21.15.116:0/254917821 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f451c036040 con 0x555bf8c4a5d0 2024-09-12T03:17:21.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.376+0000 7f450d7fa640 1 -- 172.21.15.116:0/254917821 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4520025660 con 0x555bf8c4a5d0 2024-09-12T03:17:21.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.376+0000 7f452e2e6740 10 client.19206 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:21.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.376+0000 7f452e2e6740 1 -- 172.21.15.116:0/254917821 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555bf8ccfd80 con 0x555bf8c4a5d0 2024-09-12T03:17:21.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.376+0000 7f450d7fa640 1 -- 172.21.15.116:0/254917821 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f451c036040 con 0x555bf8c4a5d0 2024-09-12T03:17:21.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.376+0000 7f450d7fa640 1 -- 172.21.15.116:0/254917821 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f45200118b0 con 0x555bf8c4a5d0 2024-09-12T03:17:21.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.376+0000 7f452e2e6740 10 client.19206 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:21.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.376+0000 7f452e2e6740 10 client.19206 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:17:21.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.376+0000 7f452e2e6740 20 client.19206 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:21.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.376+0000 7f452e2e6740 1 --2- 172.21.15.116:0/254917821 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x555bf8cd01c0 0x555bf8cd2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.376+0000 7f452e2e6740 4 client.19206 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:17:21.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.376+0000 7f452e2e6740 1 -- 172.21.15.116:0/254917821 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555bf8bb7d70 con 0x555bf8cd01c0 2024-09-12T03:17:21.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.377+0000 7f45294a8640 1 --2- 172.21.15.116:0/254917821 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x555bf8cd01c0 0x555bf8cd2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.377+0000 7f450d7fa640 10 client.19206 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:17:21.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.377+0000 7f45294a8640 1 --2- 172.21.15.116:0/254917821 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x555bf8cd01c0 0x555bf8cd2640 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d3efae88e99cff2 server_cookie=67d39ec74d6d6d44 in_seq=0 out_seq=0 2024-09-12T03:17:21.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.377+0000 7f450d7fa640 1 -- 172.21.15.116:0/254917821 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555bf8bb7d70 con 0x555bf8cd01c0 2024-09-12T03:17:21.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.377+0000 7f450d7fa640 10 client.19206 handle_command_reply: tid=0 2024-09-12T03:17:21.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.377+0000 7f452e2e6740 1 client.19206 shutdown 2024-09-12T03:17:21.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.377+0000 7f452e2e6740 1 -- 172.21.15.116:0/254917821 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555bf8c4a5d0 msgr2=0x555bf8c591b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:21.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.377+0000 7f452e2e6740 1 --2- 172.21.15.116:0/254917821 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555bf8c4a5d0 0x555bf8c591b0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f452000b970 tx=0x7f45200047e0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.377+0000 7f452e2e6740 1 -- 172.21.15.116:0/254917821 shutdown_connections 2024-09-12T03:17:21.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.377+0000 7f452e2e6740 1 --2- 172.21.15.116:0/254917821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555bf8c55370 0x555bf8c59c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.377+0000 7f452e2e6740 1 --2- 172.21.15.116:0/254917821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555bf8c4bb40 0x555bf8c596f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.377+0000 7f452e2e6740 1 --2- 172.21.15.116:0/254917821 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x555bf8cd01c0 0x555bf8cd2640 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.377+0000 7f452e2e6740 1 --2- 172.21.15.116:0/254917821 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555bf8c4a5d0 0x555bf8c591b0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.377+0000 7f450d7fa640 0 client.19206 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:17:21.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.377+0000 7f452e2e6740 1 -- 172.21.15.116:0/254917821 >> 172.21.15.116:0/254917821 conn(0x555bf8c13310 msgr2=0x555bf8c537e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:21.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.377+0000 7f452e2e6740 1 -- 172.21.15.116:0/254917821 shutdown_connections 2024-09-12T03:17:21.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.378+0000 7f452e2e6740 1 -- 172.21.15.116:0/254917821 wait complete. 2024-09-12T03:17:21.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.378+0000 7f452e2e6740 20 client.19206 trim_cache size 0 max 16384 2024-09-12T03:17:21.395 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:17:21.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.378+0000 7f452bf0d640 1 -- 172.21.15.116:0/3493413698 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f45100a2c00 msgr2=0x7f45100a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:21.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.378+0000 7f452bf0d640 1 --2- 172.21.15.116:0/3493413698 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f45100a2c00 0x7f45100a50c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f451c004af0 tx=0x7f451c030780 comp rx=0 tx=0).stop 2024-09-12T03:17:21.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.378+0000 7f452bf0d640 1 -- 172.21.15.116:0/3493413698 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4524100a50 msgr2=0x7f45241a4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:21.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.378+0000 7f452bf0d640 1 --2- 172.21.15.116:0/3493413698 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4524100a50 0x7f45241a4bb0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f451400c890 tx=0x7f451400cd50 comp rx=0 tx=0).stop 2024-09-12T03:17:21.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.378+0000 7f452bf0d640 1 -- 172.21.15.116:0/3493413698 shutdown_connections 2024-09-12T03:17:21.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.378+0000 7f452bf0d640 1 --2- 172.21.15.116:0/3493413698 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f45100a2c00 0x7f45100a50c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.378+0000 7f452bf0d640 1 --2- 172.21.15.116:0/3493413698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4524106b70 0x7f452419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.378+0000 7f452bf0d640 1 --2- 172.21.15.116:0/3493413698 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4524100a50 0x7f45241a4bb0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.378+0000 7f452bf0d640 1 --2- 172.21.15.116:0/3493413698 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4524100090 0x7f45241a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.378+0000 7f452bf0d640 1 -- 172.21.15.116:0/3493413698 >> 172.21.15.116:0/3493413698 conn(0x7f4524077210 msgr2=0x7f45241037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:21.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.378+0000 7f452bf0d640 1 -- 172.21.15.116:0/3493413698 shutdown_connections 2024-09-12T03:17:21.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:21.378+0000 7f452bf0d640 1 -- 172.21.15.116:0/3493413698 wait complete. 2024-09-12T03:17:21.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:21 smithi116 ceph-mon[31410]: pgmap v769: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:17:21.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:21 smithi177 ceph-mon[35305]: pgmap v769: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:17:21.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:21 smithi183 ceph-mon[35535]: pgmap v769: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:17:22.158 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:17:22.408 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:22.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.889+0000 7faa85a59640 1 -- 172.21.15.116:0/3190176645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa80101df0 msgr2=0x7faa801021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:22.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.889+0000 7faa85a59640 1 --2- 172.21.15.116:0/3190176645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa80101df0 0x7faa801021d0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7faa70009ca0 tx=0x7faa7002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.889+0000 7faa85a59640 1 -- 172.21.15.116:0/3190176645 shutdown_connections 2024-09-12T03:17:22.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.889+0000 7faa85a59640 1 --2- 172.21.15.116:0/3190176645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa80106f20 0x7faa8010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.889+0000 7faa85a59640 1 --2- 172.21.15.116:0/3190176645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa801027a0 0x7faa801067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.889+0000 7faa85a59640 1 --2- 172.21.15.116:0/3190176645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa80101df0 0x7faa801021d0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.889+0000 7faa85a59640 1 -- 172.21.15.116:0/3190176645 >> 172.21.15.116:0/3190176645 conn(0x7faa800fd8b0 msgr2=0x7faa800ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:22.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.890+0000 7faa85a59640 1 -- 172.21.15.116:0/3190176645 shutdown_connections 2024-09-12T03:17:22.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.890+0000 7faa85a59640 1 -- 172.21.15.116:0/3190176645 wait complete. 2024-09-12T03:17:22.910 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.890+0000 7faa85a59640 1 Processor -- start 2024-09-12T03:17:22.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.890+0000 7faa85a59640 1 -- start start 2024-09-12T03:17:22.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.890+0000 7faa85a59640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa80101df0 0x7faa801984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:22.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.890+0000 7faa85a59640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa801027a0 0x7faa80198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:22.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.890+0000 7faa85a59640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa80106f20 0x7faa8019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:22.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.890+0000 7faa85a59640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7faa80111fa0 con 0x7faa80101df0 2024-09-12T03:17:22.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.890+0000 7faa85a59640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7faa80111e20 con 0x7faa80106f20 2024-09-12T03:17:22.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.890+0000 7faa85a59640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faa80112120 con 0x7faa801027a0 2024-09-12T03:17:22.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.891+0000 7faa7effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa80101df0 0x7faa801984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:22.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.891+0000 7faa7effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa80101df0 0x7faa801984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51994/0 (socket says 172.21.15.116:51994) 2024-09-12T03:17:22.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.891+0000 7faa7effd640 1 -- 172.21.15.116:0/2990664319 learned_addr learned my addr 172.21.15.116:0/2990664319 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:22.911 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.891+0000 7faa7e7fc640 1 --2- 172.21.15.116:0/2990664319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa801027a0 0x7faa80198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:22.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.891+0000 7faa7f7fe640 1 --2- 172.21.15.116:0/2990664319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa80106f20 0x7faa8019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:22.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.891+0000 7faa7effd640 1 -- 172.21.15.116:0/2990664319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa801027a0 msgr2=0x7faa80198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:22.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.891+0000 7faa7effd640 1 --2- 172.21.15.116:0/2990664319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa801027a0 0x7faa80198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.891+0000 7faa7effd640 1 -- 172.21.15.116:0/2990664319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa80106f20 msgr2=0x7faa8019cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:22.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.891+0000 7faa7effd640 1 --2- 172.21.15.116:0/2990664319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa80106f20 0x7faa8019cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.891+0000 7faa7effd640 1 -- 172.21.15.116:0/2990664319 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa8019d470 con 0x7faa80101df0 2024-09-12T03:17:22.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.891+0000 7faa7e7fc640 1 --2- 172.21.15.116:0/2990664319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa801027a0 0x7faa80198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:22.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.891+0000 7faa7effd640 1 --2- 172.21.15.116:0/2990664319 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa80101df0 0x7faa801984c0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7faa7002ade0 tx=0x7faa700319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:22.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.891+0000 7faa84a57640 1 -- 172.21.15.116:0/2990664319 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7002fa20 con 0x7faa80101df0 2024-09-12T03:17:22.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.891+0000 7faa85a59640 1 -- 172.21.15.116:0/2990664319 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa8019d700 con 0x7faa80101df0 2024-09-12T03:17:22.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.891+0000 7faa84a57640 1 -- 172.21.15.116:0/2990664319 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa7002fbc0 con 0x7faa80101df0 2024-09-12T03:17:22.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.891+0000 7faa84a57640 1 -- 172.21.15.116:0/2990664319 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa700467b0 con 0x7faa80101df0 2024-09-12T03:17:22.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.891+0000 7faa85a59640 1 -- 172.21.15.116:0/2990664319 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa801a5040 con 0x7faa80101df0 2024-09-12T03:17:22.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.893+0000 7faa84a57640 1 -- 172.21.15.116:0/2990664319 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7faa70046950 con 0x7faa80101df0 2024-09-12T03:17:22.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.894+0000 7faa87e32740 1 Processor -- start 2024-09-12T03:17:22.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.894+0000 7faa87e32740 1 -- start start 2024-09-12T03:17:22.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.894+0000 7faa84a57640 1 --2- 172.21.15.116:0/2990664319 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7faa680a2b80 0x7faa680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:22.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.894+0000 7faa87e32740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a452c0b40 0x560a452bf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:22.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.894+0000 7faa87e32740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a452bf5d0 0x560a452bfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:22.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.894+0000 7faa87e32740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a452ca370 0x560a452cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:22.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.894+0000 7faa87e32740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560a452252f0 con 0x560a452ca370 2024-09-12T03:17:22.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.894+0000 7faa87e32740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560a451cccf0 con 0x560a452c0b40 2024-09-12T03:17:22.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.894+0000 7faa87e32740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560a452212b0 con 0x560a452bf5d0 2024-09-12T03:17:22.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.894+0000 7faa7f7fe640 1 --2- 172.21.15.116:0/2990664319 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7faa680a2b80 0x7faa680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:22.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.894+0000 7faa84a57640 1 -- 172.21.15.116:0/2990664319 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7faa700f0510 con 0x7faa80101df0 2024-09-12T03:17:22.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.894+0000 7faa7f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a452ca370 0x560a452cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:22.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.894+0000 7faa7f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a452ca370 0x560a452cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52010/0 (socket says 172.21.15.116:52010) 2024-09-12T03:17:22.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.894+0000 7faa7e7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a452bf5d0 0x560a452bfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:22.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.894+0000 7faa7f7fe640 1 -- 172.21.15.116:0/3040034108 learned_addr learned my addr 172.21.15.116:0/3040034108 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:22.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.894+0000 7faa7effd640 1 --2- 172.21.15.116:0/3040034108 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a452c0b40 0x560a452bf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:22.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.895+0000 7faa7f7fe640 1 -- 172.21.15.116:0/3040034108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a452bf5d0 msgr2=0x560a452bfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:22.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.895+0000 7faa7f7fe640 1 --2- 172.21.15.116:0/3040034108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a452bf5d0 0x560a452bfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.895+0000 7faa7f7fe640 1 -- 172.21.15.116:0/3040034108 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a452c0b40 msgr2=0x560a452bf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:22.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.895+0000 7faa7f7fe640 1 --2- 172.21.15.116:0/3040034108 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a452c0b40 0x560a452bf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.895+0000 7faa7f7fe640 1 -- 172.21.15.116:0/3040034108 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a451b13a0 con 0x560a452ca370 2024-09-12T03:17:22.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.895+0000 7faa7e7fc640 1 --2- 172.21.15.116:0/3040034108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a452bf5d0 0x560a452bfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:17:22.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.895+0000 7faa7effd640 1 --2- 172.21.15.116:0/3040034108 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a452c0b40 0x560a452bf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:22.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.895+0000 7faa7f7fe640 1 --2- 172.21.15.116:0/2990664319 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7faa680a2b80 0x7faa680a5040 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7faa780156c0 tx=0x7faa78009950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:22.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.895+0000 7faa7f7fe640 1 --2- 172.21.15.116:0/3040034108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a452ca370 0x560a452cc760 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7faa78015fd0 tx=0x7faa7801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d02cc0f7150e55 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:22.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.895+0000 7faa65ffb640 1 -- 172.21.15.116:0/3040034108 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa78027070 con 0x560a452ca370 2024-09-12T03:17:22.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.895+0000 7faa65ffb640 1 -- 172.21.15.116:0/3040034108 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa78017e40 con 0x560a452ca370 2024-09-12T03:17:22.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.895+0000 7faa65ffb640 1 -- 172.21.15.116:0/3040034108 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa78022c50 con 0x560a452ca370 2024-09-12T03:17:22.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.896+0000 7faa87e32740 1 -- 172.21.15.116:0/3040034108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a452ca370 msgr2=0x560a452cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:22.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.896+0000 7faa87e32740 1 --2- 172.21.15.116:0/3040034108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a452ca370 0x560a452cc760 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7faa78015fd0 tx=0x7faa7801a000 comp rx=0 tx=0).stop 2024-09-12T03:17:22.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.896+0000 7faa87e32740 1 -- 172.21.15.116:0/3040034108 shutdown_connections 2024-09-12T03:17:22.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.896+0000 7faa87e32740 1 --2- 172.21.15.116:0/3040034108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a452ca370 0x560a452cc760 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.896+0000 7faa87e32740 1 --2- 172.21.15.116:0/3040034108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a452bf5d0 0x560a452bfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.896+0000 7faa87e32740 1 --2- 172.21.15.116:0/3040034108 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a452c0b40 0x560a452bf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.896+0000 7faa87e32740 1 -- 172.21.15.116:0/3040034108 >> 172.21.15.116:0/3040034108 conn(0x560a45288310 msgr2=0x560a452bbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:22.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.896+0000 7faa87e32740 1 -- 172.21.15.116:0/3040034108 shutdown_connections 2024-09-12T03:17:22.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.896+0000 7faa87e32740 1 -- 172.21.15.116:0/3040034108 wait complete. 2024-09-12T03:17:22.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.896+0000 7faa87e32740 1 -- start start 2024-09-12T03:17:22.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.896+0000 7faa87e32740 1 Processor -- start 2024-09-12T03:17:22.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.896+0000 7faa87e32740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a452bf5d0 0x560a452c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:22.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.896+0000 7faa87e32740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a452c0b40 0x560a452c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:22.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.897+0000 7faa87e32740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a452ca370 0x560a452c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:22.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.897+0000 7faa87e32740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560a452cd010 con 0x560a452bf5d0 2024-09-12T03:17:22.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.897+0000 7faa87e32740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560a451e0060 con 0x560a452c0b40 2024-09-12T03:17:22.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.897+0000 7faa87e32740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560a452cd190 con 0x560a452ca370 2024-09-12T03:17:22.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.897+0000 7faa87e32740 10 client.0 _ll_register_callbacks cb 0x560a44f338c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:22.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.897+0000 7faa7effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a452bf5d0 0x560a452c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:22.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.897+0000 7faa7effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a452bf5d0 0x560a452c9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52024/0 (socket says 172.21.15.116:52024) 2024-09-12T03:17:22.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.897+0000 7faa7effd640 1 -- 172.21.15.116:0/746404969 learned_addr learned my addr 172.21.15.116:0/746404969 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:22.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.897+0000 7faa7e7fc640 1 --2- 172.21.15.116:0/746404969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a452c0b40 0x560a452c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:22.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.897+0000 7faa7effd640 1 -- 172.21.15.116:0/746404969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a452ca370 msgr2=0x560a452c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:22.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.897+0000 7faa7effd640 1 --2- 172.21.15.116:0/746404969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a452ca370 0x560a452c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.897+0000 7faa7effd640 1 -- 172.21.15.116:0/746404969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a452c0b40 msgr2=0x560a452c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:22.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.897+0000 7faa7effd640 1 --2- 172.21.15.116:0/746404969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a452c0b40 0x560a452c70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.897+0000 7faa7effd640 1 -- 172.21.15.116:0/746404969 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a44f963c0 con 0x560a452bf5d0 2024-09-12T03:17:22.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.897+0000 7faa7e7fc640 1 --2- 172.21.15.116:0/746404969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a452c0b40 0x560a452c70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:22.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.897+0000 7faa7effd640 1 --2- 172.21.15.116:0/746404969 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a452bf5d0 0x560a452c9df0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7faa700bcb30 tx=0x7faa700bd530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0c03269f184cf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:22.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.897+0000 7faa5effd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:22.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.897+0000 7faa5effd640 1 -- 172.21.15.116:0/746404969 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa700bd7c0 con 0x560a452bf5d0 2024-09-12T03:17:22.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.897+0000 7faa5effd640 1 -- 172.21.15.116:0/746404969 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7faa70046c40 con 0x560a452bf5d0 2024-09-12T03:17:22.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.897+0000 7faa87e32740 1 -- 172.21.15.116:0/746404969 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa78017e40 con 0x560a452bf5d0 2024-09-12T03:17:22.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.897+0000 7faa5effd640 1 -- 172.21.15.116:0/746404969 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7faa70046e40 con 0x560a452bf5d0 2024-09-12T03:17:22.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.897+0000 7faa87e32740 10 client.19218 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:22.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.897+0000 7faa87e32740 1 -- 172.21.15.116:0/746404969 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560a452cd010 con 0x560a452bf5d0 2024-09-12T03:17:22.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.897+0000 7faa5effd640 1 -- 172.21.15.116:0/746404969 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa78017e40 con 0x560a452bf5d0 2024-09-12T03:17:22.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.898+0000 7faa5effd640 1 -- 172.21.15.116:0/746404969 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7faa7004e030 con 0x560a452bf5d0 2024-09-12T03:17:22.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.898+0000 7faa87e32740 10 client.19218 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:22.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.898+0000 7faa87e32740 10 client.19218 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:17:22.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.898+0000 7faa87e32740 20 client.19218 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:22.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.898+0000 7faa87e32740 1 --2- 172.21.15.116:0/746404969 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x560a452d9f20 0x560a452dc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:22.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.898+0000 7faa87e32740 4 client.19218 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:17:22.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.898+0000 7faa87e32740 1 -- 172.21.15.116:0/746404969 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a45232860 con 0x560a452d9f20 2024-09-12T03:17:22.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.898+0000 7faa7e7fc640 1 --2- 172.21.15.116:0/746404969 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x560a452d9f20 0x560a452dc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:22.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.898+0000 7faa5effd640 10 client.19218 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:17:22.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.898+0000 7faa7e7fc640 1 --2- 172.21.15.116:0/746404969 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x560a452d9f20 0x560a452dc3b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eac9c0d79c03c7c3 server_cookie=85cb5b0fbc6395c3 in_seq=0 out_seq=0 2024-09-12T03:17:22.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.899+0000 7faa5effd640 1 -- 172.21.15.116:0/746404969 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560a45232860 con 0x560a452d9f20 2024-09-12T03:17:22.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.899+0000 7faa5effd640 10 client.19218 handle_command_reply: tid=0 2024-09-12T03:17:22.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.899+0000 7faa87e32740 1 client.19218 shutdown 2024-09-12T03:17:22.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.899+0000 7faa87e32740 1 -- 172.21.15.116:0/746404969 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a452bf5d0 msgr2=0x560a452c9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:22.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.899+0000 7faa87e32740 1 --2- 172.21.15.116:0/746404969 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a452bf5d0 0x560a452c9df0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7faa700bcb30 tx=0x7faa700bd530 comp rx=0 tx=0).stop 2024-09-12T03:17:22.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.899+0000 7faa87e32740 1 -- 172.21.15.116:0/746404969 shutdown_connections 2024-09-12T03:17:22.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.899+0000 7faa87e32740 1 --2- 172.21.15.116:0/746404969 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x560a452d9f20 0x560a452dc3b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.899+0000 7faa87e32740 1 --2- 172.21.15.116:0/746404969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a452ca370 0x560a452c77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.899+0000 7faa87e32740 1 --2- 172.21.15.116:0/746404969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a452c0b40 0x560a452c70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.899+0000 7faa87e32740 1 --2- 172.21.15.116:0/746404969 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a452bf5d0 0x560a452c9df0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.899+0000 7faa87e32740 1 -- 172.21.15.116:0/746404969 >> 172.21.15.116:0/746404969 conn(0x560a45288310 msgr2=0x560a452c1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:22.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.899+0000 7faa5effd640 0 client.19218 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:17:22.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.900+0000 7faa87e32740 1 -- 172.21.15.116:0/746404969 shutdown_connections 2024-09-12T03:17:22.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.900+0000 7faa87e32740 1 -- 172.21.15.116:0/746404969 wait complete. 2024-09-12T03:17:22.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.900+0000 7faa87e32740 20 client.19218 trim_cache size 0 max 16384 2024-09-12T03:17:22.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.913+0000 7faa87e32740 1 Processor -- start 2024-09-12T03:17:22.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.913+0000 7faa87e32740 1 -- start start 2024-09-12T03:17:22.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.913+0000 7faa87e32740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a452bf5d0 0x560a452c2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:22.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.913+0000 7faa87e32740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a452c0b40 0x560a452c2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:22.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.913+0000 7faa87e32740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a452ca370 0x560a452e1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:22.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.913+0000 7faa87e32740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7faa801afbf0 con 0x560a452ca370 2024-09-12T03:17:22.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.913+0000 7faa87e32740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7faa801ad4c0 con 0x560a452bf5d0 2024-09-12T03:17:22.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.913+0000 7faa87e32740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faa801ab310 con 0x560a452c0b40 2024-09-12T03:17:22.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.913+0000 7faa7f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a452ca370 0x560a452e1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:22.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.913+0000 7faa7f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a452ca370 0x560a452e1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52038/0 (socket says 172.21.15.116:52038) 2024-09-12T03:17:22.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.913+0000 7faa7f7fe640 1 -- 172.21.15.116:0/2767614423 learned_addr learned my addr 172.21.15.116:0/2767614423 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:22.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.913+0000 7faa7e7fc640 1 --2- 172.21.15.116:0/2767614423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a452c0b40 0x560a452c2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:22.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.913+0000 7faa7effd640 1 --2- 172.21.15.116:0/2767614423 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a452bf5d0 0x560a452c2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:22.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.913+0000 7faa7f7fe640 1 -- 172.21.15.116:0/2767614423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a452c0b40 msgr2=0x560a452c2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:22.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.913+0000 7faa7f7fe640 1 --2- 172.21.15.116:0/2767614423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a452c0b40 0x560a452c2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.913+0000 7faa7f7fe640 1 -- 172.21.15.116:0/2767614423 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a452bf5d0 msgr2=0x560a452c2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:22.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.913+0000 7faa7f7fe640 1 --2- 172.21.15.116:0/2767614423 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a452bf5d0 0x560a452c2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.913+0000 7faa7f7fe640 1 -- 172.21.15.116:0/2767614423 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a452ba310 con 0x560a452ca370 2024-09-12T03:17:22.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.914+0000 7faa7f7fe640 1 --2- 172.21.15.116:0/2767614423 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a452ca370 0x560a452e1c30 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7faa78025040 tx=0x7faa78006d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb74ff696e8894a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:22.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.914+0000 7faa657fa640 1 -- 172.21.15.116:0/2767614423 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa78027040 con 0x560a452ca370 2024-09-12T03:17:22.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.914+0000 7faa657fa640 1 -- 172.21.15.116:0/2767614423 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa78032070 con 0x560a452ca370 2024-09-12T03:17:22.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.914+0000 7faa657fa640 1 -- 172.21.15.116:0/2767614423 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7802da20 con 0x560a452ca370 2024-09-12T03:17:22.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.915+0000 7faa87e32740 1 -- 172.21.15.116:0/2767614423 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a452ca370 msgr2=0x560a452e1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:22.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.915+0000 7faa87e32740 1 --2- 172.21.15.116:0/2767614423 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a452ca370 0x560a452e1c30 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7faa78025040 tx=0x7faa78006d60 comp rx=0 tx=0).stop 2024-09-12T03:17:22.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.915+0000 7faa87e32740 1 -- 172.21.15.116:0/2767614423 shutdown_connections 2024-09-12T03:17:22.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.915+0000 7faa87e32740 1 --2- 172.21.15.116:0/2767614423 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a452ca370 0x560a452e1c30 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.915+0000 7faa87e32740 1 --2- 172.21.15.116:0/2767614423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a452c0b40 0x560a452c2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.915+0000 7faa87e32740 1 --2- 172.21.15.116:0/2767614423 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a452bf5d0 0x560a452c2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.915+0000 7faa87e32740 1 -- 172.21.15.116:0/2767614423 >> 172.21.15.116:0/2767614423 conn(0x560a45288310 msgr2=0x560a452bad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:22.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.915+0000 7faa87e32740 1 -- 172.21.15.116:0/2767614423 shutdown_connections 2024-09-12T03:17:22.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.915+0000 7faa87e32740 1 -- 172.21.15.116:0/2767614423 wait complete. 2024-09-12T03:17:22.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.915+0000 7faa87e32740 1 -- start start 2024-09-12T03:17:22.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.915+0000 7faa87e32740 1 Processor -- start 2024-09-12T03:17:22.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.916+0000 7faa87e32740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a452bf5d0 0x560a452d2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:22.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.916+0000 7faa87e32740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a452c0b40 0x560a452c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:22.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.916+0000 7faa87e32740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a452ca370 0x560a452c4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:22.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.916+0000 7faa87e32740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7faa801ab980 con 0x560a452bf5d0 2024-09-12T03:17:22.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.916+0000 7faa87e32740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7faa801acdf0 con 0x560a452c0b40 2024-09-12T03:17:22.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.916+0000 7faa87e32740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faa801ac050 con 0x560a452ca370 2024-09-12T03:17:22.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.916+0000 7faa87e32740 10 client.0 _ll_register_callbacks cb 0x7faa70005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:22.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.916+0000 7faa7effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a452bf5d0 0x560a452d2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:22.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.916+0000 7faa7effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a452bf5d0 0x560a452d2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52046/0 (socket says 172.21.15.116:52046) 2024-09-12T03:17:22.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.916+0000 7faa7effd640 1 -- 172.21.15.116:0/1530835501 learned_addr learned my addr 172.21.15.116:0/1530835501 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:22.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.916+0000 7faa7e7fc640 1 --2- 172.21.15.116:0/1530835501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a452c0b40 0x560a452c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:22.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.916+0000 7faa7f7fe640 1 --2- 172.21.15.116:0/1530835501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a452ca370 0x560a452c4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:22.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.916+0000 7faa7effd640 1 -- 172.21.15.116:0/1530835501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a452ca370 msgr2=0x560a452c4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:22.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.916+0000 7faa7effd640 1 --2- 172.21.15.116:0/1530835501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a452ca370 0x560a452c4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.916+0000 7faa7effd640 1 -- 172.21.15.116:0/1530835501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a452c0b40 msgr2=0x560a452c3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:22.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.916+0000 7faa7effd640 1 --2- 172.21.15.116:0/1530835501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a452c0b40 0x560a452c3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.916+0000 7faa7effd640 1 -- 172.21.15.116:0/1530835501 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a452c03a0 con 0x560a452bf5d0 2024-09-12T03:17:22.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.916+0000 7faa7effd640 1 --2- 172.21.15.116:0/1530835501 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a452bf5d0 0x560a452d2c40 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7faa700ee3b0 tx=0x7faa7002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e6cf4c04cd30bb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:22.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.916+0000 7faa5effd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:22.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.916+0000 7faa5effd640 1 -- 172.21.15.116:0/1530835501 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa70087020 con 0x560a452bf5d0 2024-09-12T03:17:22.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.916+0000 7faa5effd640 1 -- 172.21.15.116:0/1530835501 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7faa700bc490 con 0x560a452bf5d0 2024-09-12T03:17:22.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.916+0000 7faa87e32740 1 -- 172.21.15.116:0/1530835501 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa78032070 con 0x560a452bf5d0 2024-09-12T03:17:22.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.916+0000 7faa5effd640 1 -- 172.21.15.116:0/1530835501 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7faa700c5c60 con 0x560a452bf5d0 2024-09-12T03:17:22.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.916+0000 7faa87e32740 10 client.19224 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:22.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.916+0000 7faa87e32740 1 -- 172.21.15.116:0/1530835501 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560a45344b30 con 0x560a452bf5d0 2024-09-12T03:17:22.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.917+0000 7faa5effd640 1 -- 172.21.15.116:0/1530835501 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa78032070 con 0x560a452bf5d0 2024-09-12T03:17:22.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.917+0000 7faa5effd640 1 -- 172.21.15.116:0/1530835501 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7faa7004e050 con 0x560a452bf5d0 2024-09-12T03:17:22.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.917+0000 7faa87e32740 10 client.19224 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:22.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.917+0000 7faa87e32740 10 client.19224 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:17:22.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.917+0000 7faa87e32740 20 client.19224 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:22.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.917+0000 7faa87e32740 1 --2- 172.21.15.116:0/1530835501 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x560a453450f0 0x560a45347510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:22.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.917+0000 7faa87e32740 4 client.19224 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:17:22.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.917+0000 7faa87e32740 1 -- 172.21.15.116:0/1530835501 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560a4522cd70 con 0x560a453450f0 2024-09-12T03:17:22.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.917+0000 7faa7e7fc640 1 --2- 172.21.15.116:0/1530835501 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x560a453450f0 0x560a45347510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:22.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.918+0000 7faa7e7fc640 1 --2- 172.21.15.116:0/1530835501 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x560a453450f0 0x560a45347510 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=81fc3176a03840a1 server_cookie=ea3e78cb2de28e17 in_seq=0 out_seq=0 2024-09-12T03:17:22.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.918+0000 7faa5effd640 10 client.19224 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:17:22.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.918+0000 7faa5effd640 1 -- 172.21.15.116:0/1530835501 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560a4522cd70 con 0x560a453450f0 2024-09-12T03:17:22.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.918+0000 7faa5effd640 10 client.19224 handle_command_reply: tid=0 2024-09-12T03:17:22.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.918+0000 7faa87e32740 1 client.19224 shutdown 2024-09-12T03:17:22.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.918+0000 7faa87e32740 1 -- 172.21.15.116:0/1530835501 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a452bf5d0 msgr2=0x560a452d2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:22.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.918+0000 7faa87e32740 1 --2- 172.21.15.116:0/1530835501 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a452bf5d0 0x560a452d2c40 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7faa700ee3b0 tx=0x7faa7002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.918+0000 7faa87e32740 1 -- 172.21.15.116:0/1530835501 shutdown_connections 2024-09-12T03:17:22.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.918+0000 7faa87e32740 1 --2- 172.21.15.116:0/1530835501 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x560a453450f0 0x560a45347510 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.918+0000 7faa87e32740 1 --2- 172.21.15.116:0/1530835501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a452ca370 0x560a452c4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.918+0000 7faa87e32740 1 --2- 172.21.15.116:0/1530835501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a452c0b40 0x560a452c3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.918+0000 7faa87e32740 1 --2- 172.21.15.116:0/1530835501 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a452bf5d0 0x560a452d2c40 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.918+0000 7faa87e32740 1 -- 172.21.15.116:0/1530835501 >> 172.21.15.116:0/1530835501 conn(0x560a45288310 msgr2=0x560a452c8900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:22.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.918+0000 7faa5effd640 0 client.19224 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:17:22.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.918+0000 7faa87e32740 1 -- 172.21.15.116:0/1530835501 shutdown_connections 2024-09-12T03:17:22.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.918+0000 7faa87e32740 1 -- 172.21.15.116:0/1530835501 wait complete. 2024-09-12T03:17:22.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.918+0000 7faa87e32740 20 client.19224 trim_cache size 0 max 16384 2024-09-12T03:17:22.937 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:17:22.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.919+0000 7faa85a59640 1 -- 172.21.15.116:0/2990664319 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7faa680a2b80 msgr2=0x7faa680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:22.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.919+0000 7faa85a59640 1 --2- 172.21.15.116:0/2990664319 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7faa680a2b80 0x7faa680a5040 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7faa780156c0 tx=0x7faa78009950 comp rx=0 tx=0).stop 2024-09-12T03:17:22.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.919+0000 7faa85a59640 1 -- 172.21.15.116:0/2990664319 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa80101df0 msgr2=0x7faa801984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:22.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.919+0000 7faa85a59640 1 --2- 172.21.15.116:0/2990664319 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa80101df0 0x7faa801984c0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7faa7002ade0 tx=0x7faa700319f0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.919+0000 7faa85a59640 1 -- 172.21.15.116:0/2990664319 shutdown_connections 2024-09-12T03:17:22.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.919+0000 7faa85a59640 1 --2- 172.21.15.116:0/2990664319 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7faa680a2b80 0x7faa680a5040 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.919+0000 7faa85a59640 1 --2- 172.21.15.116:0/2990664319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa80106f20 0x7faa8019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.919+0000 7faa85a59640 1 --2- 172.21.15.116:0/2990664319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa801027a0 0x7faa80198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.919+0000 7faa85a59640 1 --2- 172.21.15.116:0/2990664319 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa80101df0 0x7faa801984c0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:22.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.919+0000 7faa85a59640 1 -- 172.21.15.116:0/2990664319 >> 172.21.15.116:0/2990664319 conn(0x7faa800fd8b0 msgr2=0x7faa800feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:22.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.919+0000 7faa85a59640 1 -- 172.21.15.116:0/2990664319 shutdown_connections 2024-09-12T03:17:22.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:22.919+0000 7faa85a59640 1 -- 172.21.15.116:0/2990664319 wait complete. 2024-09-12T03:17:23.289 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:23 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:23.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:23 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:23.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:23 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:23.809 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:17:23.942 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:17:24.070 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:24.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:24 smithi116 ceph-mon[31410]: pgmap v770: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:17:24.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:24 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:24.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:24 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:24.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:24 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:17:24.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:24 smithi183 ceph-mon[35535]: pgmap v770: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:17:24.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:24 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:24.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:24 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:24.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:24 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:17:24.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:24 smithi177 ceph-mon[35305]: pgmap v770: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:17:24.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:24 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:24.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:24 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:24.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:24 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:17:24.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.586+0000 7fbe276b2640 1 -- 172.21.15.116:0/4157858957 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe20102790 msgr2=0x7fbe201067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:24.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.586+0000 7fbe276b2640 1 --2- 172.21.15.116:0/4157858957 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe20102790 0x7fbe201067e0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fbe10009900 tx=0x7fbe1002a410 comp rx=0 tx=0).stop 2024-09-12T03:17:24.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.586+0000 7fbe276b2640 1 -- 172.21.15.116:0/4157858957 shutdown_connections 2024-09-12T03:17:24.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.586+0000 7fbe276b2640 1 --2- 172.21.15.116:0/4157858957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe20106f10 0x7fbe2010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.586+0000 7fbe276b2640 1 --2- 172.21.15.116:0/4157858957 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe20102790 0x7fbe201067e0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.586+0000 7fbe276b2640 1 --2- 172.21.15.116:0/4157858957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe20101de0 0x7fbe201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.586+0000 7fbe276b2640 1 -- 172.21.15.116:0/4157858957 >> 172.21.15.116:0/4157858957 conn(0x7fbe200fd8c0 msgr2=0x7fbe200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:24.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.586+0000 7fbe276b2640 1 -- 172.21.15.116:0/4157858957 shutdown_connections 2024-09-12T03:17:24.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.586+0000 7fbe276b2640 1 -- 172.21.15.116:0/4157858957 wait complete. 2024-09-12T03:17:24.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.587+0000 7fbe276b2640 1 Processor -- start 2024-09-12T03:17:24.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.587+0000 7fbe276b2640 1 -- start start 2024-09-12T03:17:24.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.587+0000 7fbe276b2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe20101de0 0x7fbe20198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:24.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.587+0000 7fbe276b2640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe20102790 0x7fbe20198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:24.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.587+0000 7fbe276b2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe20106f10 0x7fbe2019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:24.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.587+0000 7fbe276b2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbe20111f70 con 0x7fbe20101de0 2024-09-12T03:17:24.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.587+0000 7fbe276b2640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbe20111df0 con 0x7fbe20102790 2024-09-12T03:17:24.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.587+0000 7fbe276b2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbe201120f0 con 0x7fbe20106f10 2024-09-12T03:17:24.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.587+0000 7fbe2544e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe20101de0 0x7fbe20198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:24.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.587+0000 7fbe2544e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe20101de0 0x7fbe20198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52064/0 (socket says 172.21.15.116:52064) 2024-09-12T03:17:24.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.587+0000 7fbe2544e640 1 -- 172.21.15.116:0/2017054831 learned_addr learned my addr 172.21.15.116:0/2017054831 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:24.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.587+0000 7fbe25c4f640 1 --2- 172.21.15.116:0/2017054831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe20106f10 0x7fbe2019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:24.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.587+0000 7fbe24c4d640 1 --2- 172.21.15.116:0/2017054831 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe20102790 0x7fbe20198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:24.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.587+0000 7fbe2544e640 1 -- 172.21.15.116:0/2017054831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe20106f10 msgr2=0x7fbe2019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:24.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.587+0000 7fbe2544e640 1 --2- 172.21.15.116:0/2017054831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe20106f10 0x7fbe2019cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.587+0000 7fbe2544e640 1 -- 172.21.15.116:0/2017054831 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe20102790 msgr2=0x7fbe20198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:24.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.587+0000 7fbe2544e640 1 --2- 172.21.15.116:0/2017054831 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe20102790 0x7fbe20198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.587+0000 7fbe2544e640 1 -- 172.21.15.116:0/2017054831 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2019d400 con 0x7fbe20101de0 2024-09-12T03:17:24.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.587+0000 7fbe25c4f640 1 --2- 172.21.15.116:0/2017054831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe20106f10 0x7fbe2019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:24.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.588+0000 7fbe2544e640 1 --2- 172.21.15.116:0/2017054831 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe20101de0 0x7fbe20198450 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c00d950 tx=0x7fbe1c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:24.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.588+0000 7fbe167fc640 1 -- 172.21.15.116:0/2017054831 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c017070 con 0x7fbe20101de0 2024-09-12T03:17:24.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.588+0000 7fbe276b2640 1 -- 172.21.15.116:0/2017054831 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe2019d6f0 con 0x7fbe20101de0 2024-09-12T03:17:24.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.588+0000 7fbe167fc640 1 -- 172.21.15.116:0/2017054831 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe1c009c30 con 0x7fbe20101de0 2024-09-12T03:17:24.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.588+0000 7fbe167fc640 1 -- 172.21.15.116:0/2017054831 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c00f6b0 con 0x7fbe20101de0 2024-09-12T03:17:24.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.588+0000 7fbe276b2640 1 -- 172.21.15.116:0/2017054831 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe201a4fd0 con 0x7fbe20101de0 2024-09-12T03:17:24.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.590+0000 7fbe167fc640 1 -- 172.21.15.116:0/2017054831 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fbe1c0109d0 con 0x7fbe20101de0 2024-09-12T03:17:24.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.591+0000 7fbe29a8b740 1 Processor -- start 2024-09-12T03:17:24.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.591+0000 7fbe29a8b740 1 -- start start 2024-09-12T03:17:24.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.591+0000 7fbe29a8b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d00c97b40 0x555d00c96090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:24.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.591+0000 7fbe29a8b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d00c965d0 0x555d00c96a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:24.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.591+0000 7fbe29a8b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d00ca1370 0x555d00ca3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:24.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.591+0000 7fbe29a8b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555d00bfc2f0 con 0x555d00ca1370 2024-09-12T03:17:24.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.591+0000 7fbe29a8b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x555d00ba3cf0 con 0x555d00c965d0 2024-09-12T03:17:24.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.591+0000 7fbe29a8b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555d00bf82b0 con 0x555d00c97b40 2024-09-12T03:17:24.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.591+0000 7fbe167fc640 1 --2- 172.21.15.116:0/2017054831 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbe0c0a2b80 0x7fbe0c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:24.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.591+0000 7fbe24c4d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d00ca1370 0x555d00ca3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:24.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.591+0000 7fbe25c4f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d00c97b40 0x555d00c96090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:24.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.591+0000 7fbe24c4d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d00ca1370 0x555d00ca3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52078/0 (socket says 172.21.15.116:52078) 2024-09-12T03:17:24.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.591+0000 7fbe24c4d640 1 -- 172.21.15.116:0/2456638173 learned_addr learned my addr 172.21.15.116:0/2456638173 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:24.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.591+0000 7fbe25c4f640 1 --2- 172.21.15.116:0/2017054831 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbe0c0a2b80 0x7fbe0c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:24.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.591+0000 7fbe167fc640 1 -- 172.21.15.116:0/2017054831 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fbe1c0cecd0 con 0x7fbe20101de0 2024-09-12T03:17:24.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.591+0000 7fbe2544e640 1 --2- 172.21.15.116:0/2456638173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d00c965d0 0x555d00c96a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:24.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.591+0000 7fbe24c4d640 1 -- 172.21.15.116:0/2456638173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d00c97b40 msgr2=0x555d00c96090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:24.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.591+0000 7fbe24c4d640 1 --2- 172.21.15.116:0/2456638173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d00c97b40 0x555d00c96090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.591+0000 7fbe24c4d640 1 -- 172.21.15.116:0/2456638173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d00c965d0 msgr2=0x555d00c96a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:24.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.591+0000 7fbe24c4d640 1 --2- 172.21.15.116:0/2456638173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d00c965d0 0x555d00c96a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.591+0000 7fbe24c4d640 1 -- 172.21.15.116:0/2456638173 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d00b883a0 con 0x555d00ca1370 2024-09-12T03:17:24.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.592+0000 7fbe25c4f640 1 --2- 172.21.15.116:0/2017054831 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbe0c0a2b80 0x7fbe0c0a5040 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbe18005c40 tx=0x7fbe18009a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:24.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.592+0000 7fbe24c4d640 1 --2- 172.21.15.116:0/2456638173 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d00ca1370 0x555d00ca3760 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fbe1002a930 tx=0x7fbe10005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad09e935973688a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:24.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.592+0000 7fbe0b7fe640 1 -- 172.21.15.116:0/2456638173 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1003f070 con 0x555d00ca1370 2024-09-12T03:17:24.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.592+0000 7fbe0b7fe640 1 -- 172.21.15.116:0/2456638173 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe10004030 con 0x555d00ca1370 2024-09-12T03:17:24.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.592+0000 7fbe0b7fe640 1 -- 172.21.15.116:0/2456638173 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe10035070 con 0x555d00ca1370 2024-09-12T03:17:24.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.593+0000 7fbe29a8b740 1 -- 172.21.15.116:0/2456638173 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d00ca1370 msgr2=0x555d00ca3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:24.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.593+0000 7fbe29a8b740 1 --2- 172.21.15.116:0/2456638173 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d00ca1370 0x555d00ca3760 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fbe1002a930 tx=0x7fbe10005e80 comp rx=0 tx=0).stop 2024-09-12T03:17:24.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.593+0000 7fbe29a8b740 1 -- 172.21.15.116:0/2456638173 shutdown_connections 2024-09-12T03:17:24.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.593+0000 7fbe29a8b740 1 --2- 172.21.15.116:0/2456638173 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d00ca1370 0x555d00ca3760 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.593+0000 7fbe29a8b740 1 --2- 172.21.15.116:0/2456638173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d00c965d0 0x555d00c96a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.593+0000 7fbe29a8b740 1 --2- 172.21.15.116:0/2456638173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d00c97b40 0x555d00c96090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.593+0000 7fbe29a8b740 1 -- 172.21.15.116:0/2456638173 >> 172.21.15.116:0/2456638173 conn(0x555d00c5f310 msgr2=0x555d00c92c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:24.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.593+0000 7fbe29a8b740 1 -- 172.21.15.116:0/2456638173 shutdown_connections 2024-09-12T03:17:24.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.593+0000 7fbe29a8b740 1 -- 172.21.15.116:0/2456638173 wait complete. 2024-09-12T03:17:24.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.593+0000 7fbe29a8b740 1 -- start start 2024-09-12T03:17:24.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.593+0000 7fbe29a8b740 1 Processor -- start 2024-09-12T03:17:24.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.593+0000 7fbe29a8b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d00c965d0 0x555d00ca0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:24.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.593+0000 7fbe29a8b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d00c97b40 0x555d00c9dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:24.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.594+0000 7fbe29a8b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d00ca1370 0x555d00c9e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:24.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.594+0000 7fbe29a8b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555d00ca4010 con 0x555d00c965d0 2024-09-12T03:17:24.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.594+0000 7fbe29a8b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x555d00bb7060 con 0x555d00ca1370 2024-09-12T03:17:24.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.594+0000 7fbe29a8b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555d00ca4190 con 0x555d00c97b40 2024-09-12T03:17:24.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.594+0000 7fbe29a8b740 10 client.0 _ll_register_callbacks cb 0x555d0090a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:24.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.594+0000 7fbe2544e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d00c965d0 0x555d00ca0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:24.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.594+0000 7fbe2544e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d00c965d0 0x555d00ca0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52080/0 (socket says 172.21.15.116:52080) 2024-09-12T03:17:24.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.594+0000 7fbe2544e640 1 -- 172.21.15.116:0/3618411523 learned_addr learned my addr 172.21.15.116:0/3618411523 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:24.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.594+0000 7fbe24c4d640 1 --2- 172.21.15.116:0/3618411523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d00c97b40 0x555d00c9dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:24.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.594+0000 7fbe2544e640 1 -- 172.21.15.116:0/3618411523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d00c97b40 msgr2=0x555d00c9dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:24.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.594+0000 7fbe2544e640 1 --2- 172.21.15.116:0/3618411523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d00c97b40 0x555d00c9dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.594+0000 7fbe2544e640 1 -- 172.21.15.116:0/3618411523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d00ca1370 msgr2=0x555d00c9e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:24.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.594+0000 7fbe25c4f640 1 --2- 172.21.15.116:0/3618411523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d00ca1370 0x555d00c9e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:24.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.594+0000 7fbe2544e640 1 --2- 172.21.15.116:0/3618411523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d00ca1370 0x555d00c9e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.594+0000 7fbe2544e640 1 -- 172.21.15.116:0/3618411523 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d0096d3c0 con 0x555d00c965d0 2024-09-12T03:17:24.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.594+0000 7fbe24c4d640 1 --2- 172.21.15.116:0/3618411523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d00c97b40 0x555d00c9dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:24.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.594+0000 7fbe2544e640 1 --2- 172.21.15.116:0/3618411523 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d00c965d0 0x555d00ca0d20 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c0ccb70 tx=0x7fbe1c09af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61dbadf533865471 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:24.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.594+0000 7fbe08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:24.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.594+0000 7fbe08ff9640 1 -- 172.21.15.116:0/3618411523 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c020040 con 0x555d00c965d0 2024-09-12T03:17:24.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.594+0000 7fbe08ff9640 1 -- 172.21.15.116:0/3618411523 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fbe1c09a9b0 con 0x555d00c965d0 2024-09-12T03:17:24.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.594+0000 7fbe29a8b740 1 -- 172.21.15.116:0/3618411523 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe10004030 con 0x555d00c965d0 2024-09-12T03:17:24.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.594+0000 7fbe08ff9640 1 -- 172.21.15.116:0/3618411523 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbe1c09f910 con 0x555d00c965d0 2024-09-12T03:17:24.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.594+0000 7fbe29a8b740 10 client.19236 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:24.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.594+0000 7fbe29a8b740 1 -- 172.21.15.116:0/3618411523 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555d00cafab0 con 0x555d00c965d0 2024-09-12T03:17:24.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.595+0000 7fbe08ff9640 1 -- 172.21.15.116:0/3618411523 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe10004030 con 0x555d00c965d0 2024-09-12T03:17:24.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.595+0000 7fbe08ff9640 1 -- 172.21.15.116:0/3618411523 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbe1c097340 con 0x555d00c965d0 2024-09-12T03:17:24.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.595+0000 7fbe29a8b740 10 client.19236 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:24.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.595+0000 7fbe29a8b740 10 client.19236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:17:24.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.595+0000 7fbe29a8b740 20 client.19236 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:24.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.595+0000 7fbe29a8b740 1 --2- 172.21.15.116:0/3618411523 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x555d00cb0f70 0x555d00cb3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:24.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.595+0000 7fbe29a8b740 4 client.19236 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:17:24.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.595+0000 7fbe29a8b740 1 -- 172.21.15.116:0/3618411523 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d00c09860 con 0x555d00cb0f70 2024-09-12T03:17:24.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.595+0000 7fbe24c4d640 1 --2- 172.21.15.116:0/3618411523 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x555d00cb0f70 0x555d00cb3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:24.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.596+0000 7fbe24c4d640 1 --2- 172.21.15.116:0/3618411523 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x555d00cb0f70 0x555d00cb3380 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20f3907ea959252a server_cookie=654e39a4501d4590 in_seq=0 out_seq=0 2024-09-12T03:17:24.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.596+0000 7fbe08ff9640 10 client.19236 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:24.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.596+0000 7fbe08ff9640 1 -- 172.21.15.116:0/3618411523 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555d00c09860 con 0x555d00cb0f70 2024-09-12T03:17:24.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.596+0000 7fbe08ff9640 10 client.19236 handle_command_reply: tid=0 2024-09-12T03:17:24.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.596+0000 7fbe29a8b740 1 client.19236 shutdown 2024-09-12T03:17:24.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.597+0000 7fbe29a8b740 1 -- 172.21.15.116:0/3618411523 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d00c965d0 msgr2=0x555d00ca0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:24.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.597+0000 7fbe29a8b740 1 --2- 172.21.15.116:0/3618411523 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d00c965d0 0x555d00ca0d20 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c0ccb70 tx=0x7fbe1c09af80 comp rx=0 tx=0).stop 2024-09-12T03:17:24.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.597+0000 7fbe29a8b740 1 -- 172.21.15.116:0/3618411523 shutdown_connections 2024-09-12T03:17:24.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.597+0000 7fbe29a8b740 1 --2- 172.21.15.116:0/3618411523 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x555d00cb0f70 0x555d00cb3380 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.597+0000 7fbe29a8b740 1 --2- 172.21.15.116:0/3618411523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d00ca1370 0x555d00c9e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.597+0000 7fbe29a8b740 1 --2- 172.21.15.116:0/3618411523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d00c97b40 0x555d00c9dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.597+0000 7fbe29a8b740 1 --2- 172.21.15.116:0/3618411523 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d00c965d0 0x555d00ca0d20 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.597+0000 7fbe29a8b740 1 -- 172.21.15.116:0/3618411523 >> 172.21.15.116:0/3618411523 conn(0x555d00c5f310 msgr2=0x555d00c98960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:24.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.597+0000 7fbe08ff9640 0 client.19236 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:24.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.597+0000 7fbe29a8b740 1 -- 172.21.15.116:0/3618411523 shutdown_connections 2024-09-12T03:17:24.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.597+0000 7fbe29a8b740 1 -- 172.21.15.116:0/3618411523 wait complete. 2024-09-12T03:17:24.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.597+0000 7fbe29a8b740 20 client.19236 trim_cache size 0 max 16384 2024-09-12T03:17:24.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.610+0000 7fbe29a8b740 1 Processor -- start 2024-09-12T03:17:24.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.610+0000 7fbe29a8b740 1 -- start start 2024-09-12T03:17:24.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.610+0000 7fbe29a8b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d00c965d0 0x555d00c92e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:24.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.610+0000 7fbe29a8b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d00c97b40 0x555d00cad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:24.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.610+0000 7fbe29a8b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d00ca1370 0x555d00cb8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:24.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.610+0000 7fbe29a8b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbe201adb00 con 0x555d00ca1370 2024-09-12T03:17:24.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.610+0000 7fbe29a8b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbe201ab3d0 con 0x555d00c965d0 2024-09-12T03:17:24.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.610+0000 7fbe29a8b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbe201a9220 con 0x555d00c97b40 2024-09-12T03:17:24.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.610+0000 7fbe25c4f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d00ca1370 0x555d00cb8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:24.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.610+0000 7fbe25c4f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d00ca1370 0x555d00cb8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52082/0 (socket says 172.21.15.116:52082) 2024-09-12T03:17:24.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.610+0000 7fbe25c4f640 1 -- 172.21.15.116:0/3014680328 learned_addr learned my addr 172.21.15.116:0/3014680328 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:24.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.610+0000 7fbe24c4d640 1 --2- 172.21.15.116:0/3014680328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d00c97b40 0x555d00cad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:24.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.610+0000 7fbe2544e640 1 --2- 172.21.15.116:0/3014680328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d00c965d0 0x555d00c92e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:24.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.610+0000 7fbe25c4f640 1 -- 172.21.15.116:0/3014680328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d00c97b40 msgr2=0x555d00cad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:24.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.610+0000 7fbe25c4f640 1 --2- 172.21.15.116:0/3014680328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d00c97b40 0x555d00cad2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.610+0000 7fbe25c4f640 1 -- 172.21.15.116:0/3014680328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d00c965d0 msgr2=0x555d00c92e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:24.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.610+0000 7fbe25c4f640 1 --2- 172.21.15.116:0/3014680328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d00c965d0 0x555d00c92e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.610+0000 7fbe25c4f640 1 -- 172.21.15.116:0/3014680328 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d00ca0840 con 0x555d00ca1370 2024-09-12T03:17:24.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.610+0000 7fbe2544e640 1 --2- 172.21.15.116:0/3014680328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d00c965d0 0x555d00c92e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:24.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.611+0000 7fbe25c4f640 1 --2- 172.21.15.116:0/3014680328 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d00ca1370 0x555d00cb8c20 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fbe18009e80 tx=0x7fbe18027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b33d11c00fd7c4d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:24.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.611+0000 7fbe0affd640 1 -- 172.21.15.116:0/3014680328 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe18030070 con 0x555d00ca1370 2024-09-12T03:17:24.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.611+0000 7fbe0affd640 1 -- 172.21.15.116:0/3014680328 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe1801ed70 con 0x555d00ca1370 2024-09-12T03:17:24.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.611+0000 7fbe0affd640 1 -- 172.21.15.116:0/3014680328 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe18010430 con 0x555d00ca1370 2024-09-12T03:17:24.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.612+0000 7fbe29a8b740 1 -- 172.21.15.116:0/3014680328 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d00ca1370 msgr2=0x555d00cb8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:24.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.612+0000 7fbe29a8b740 1 --2- 172.21.15.116:0/3014680328 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d00ca1370 0x555d00cb8c20 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fbe18009e80 tx=0x7fbe18027000 comp rx=0 tx=0).stop 2024-09-12T03:17:24.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.612+0000 7fbe29a8b740 1 -- 172.21.15.116:0/3014680328 shutdown_connections 2024-09-12T03:17:24.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.612+0000 7fbe29a8b740 1 --2- 172.21.15.116:0/3014680328 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d00ca1370 0x555d00cb8c20 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.612+0000 7fbe29a8b740 1 --2- 172.21.15.116:0/3014680328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d00c97b40 0x555d00cad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.612+0000 7fbe29a8b740 1 --2- 172.21.15.116:0/3014680328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d00c965d0 0x555d00c92e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.612+0000 7fbe29a8b740 1 -- 172.21.15.116:0/3014680328 >> 172.21.15.116:0/3014680328 conn(0x555d00c5f310 msgr2=0x555d00c92700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:24.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.612+0000 7fbe29a8b740 1 -- 172.21.15.116:0/3014680328 shutdown_connections 2024-09-12T03:17:24.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.612+0000 7fbe29a8b740 1 -- 172.21.15.116:0/3014680328 wait complete. 2024-09-12T03:17:24.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.612+0000 7fbe29a8b740 1 -- start start 2024-09-12T03:17:24.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.612+0000 7fbe29a8b740 1 Processor -- start 2024-09-12T03:17:24.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.613+0000 7fbe29a8b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d00c965d0 0x555d00ca98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:24.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.613+0000 7fbe29a8b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d00c97b40 0x555d00ca4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:24.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.613+0000 7fbe29a8b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d00ca1370 0x555d00ca5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:24.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.613+0000 7fbe29a8b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbe201a9890 con 0x555d00ca1370 2024-09-12T03:17:24.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.613+0000 7fbe29a8b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbe201aad00 con 0x555d00c97b40 2024-09-12T03:17:24.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.613+0000 7fbe29a8b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbe201a9f60 con 0x555d00c965d0 2024-09-12T03:17:24.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.613+0000 7fbe29a8b740 10 client.0 _ll_register_callbacks cb 0x7fbe1c0978f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:24.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.613+0000 7fbe25c4f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d00ca1370 0x555d00ca5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:24.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.613+0000 7fbe2544e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d00c965d0 0x555d00ca98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:24.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.613+0000 7fbe2544e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d00c965d0 0x555d00ca98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.116:41590/0 (socket says 172.21.15.116:41590) 2024-09-12T03:17:24.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.613+0000 7fbe2544e640 1 -- 172.21.15.116:0/2228795150 learned_addr learned my addr 172.21.15.116:0/2228795150 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:24.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.613+0000 7fbe24c4d640 1 --2- 172.21.15.116:0/2228795150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d00c97b40 0x555d00ca4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:24.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.613+0000 7fbe2544e640 1 -- 172.21.15.116:0/2228795150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d00c97b40 msgr2=0x555d00ca4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:24.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.613+0000 7fbe2544e640 1 --2- 172.21.15.116:0/2228795150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d00c97b40 0x555d00ca4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.613+0000 7fbe2544e640 1 -- 172.21.15.116:0/2228795150 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d00ca1370 msgr2=0x555d00ca5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:24.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.613+0000 7fbe2544e640 1 --2- 172.21.15.116:0/2228795150 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d00ca1370 0x555d00ca5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.613+0000 7fbe2544e640 1 -- 172.21.15.116:0/2228795150 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d00ca5650 con 0x555d00c965d0 2024-09-12T03:17:24.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.613+0000 7fbe24c4d640 1 --2- 172.21.15.116:0/2228795150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d00c97b40 0x555d00ca4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:17:24.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.613+0000 7fbe25c4f640 1 --2- 172.21.15.116:0/2228795150 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d00ca1370 0x555d00ca5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:17:24.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.614+0000 7fbe2544e640 1 --2- 172.21.15.116:0/2228795150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d00c965d0 0x555d00ca98a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c09b260 tx=0x7fbe1c0974c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96e882981a0bee46 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:24.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.614+0000 7fbe08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T03:17:24.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.614+0000 7fbe08ff9640 1 -- 172.21.15.116:0/2228795150 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c014070 con 0x555d00c965d0 2024-09-12T03:17:24.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.614+0000 7fbe29a8b740 1 -- 172.21.15.116:0/2228795150 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe1801ed70 con 0x555d00c965d0 2024-09-12T03:17:24.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.614+0000 7fbe08ff9640 1 -- 172.21.15.116:0/2228795150 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fbe1c084af0 con 0x555d00c965d0 2024-09-12T03:17:24.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.614+0000 7fbe08ff9640 1 -- 172.21.15.116:0/2228795150 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbe1c020070 con 0x555d00c965d0 2024-09-12T03:17:24.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.614+0000 7fbe29a8b740 10 client.27457 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:24.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.614+0000 7fbe29a8b740 1 -- 172.21.15.116:0/2228795150 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555d00d1b890 con 0x555d00c965d0 2024-09-12T03:17:24.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.614+0000 7fbe08ff9640 1 -- 172.21.15.116:0/2228795150 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe1801ed70 con 0x555d00c965d0 2024-09-12T03:17:24.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.614+0000 7fbe08ff9640 1 -- 172.21.15.116:0/2228795150 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbe1c0662b0 con 0x555d00c965d0 2024-09-12T03:17:24.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.615+0000 7fbe29a8b740 10 client.27457 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:24.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.615+0000 7fbe29a8b740 10 client.27457 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:17:24.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.615+0000 7fbe29a8b740 20 client.27457 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:24.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.615+0000 7fbe29a8b740 1 --2- 172.21.15.116:0/2228795150 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x555d00d1bdb0 0x555d00d1e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:24.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.615+0000 7fbe29a8b740 4 client.27457 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:17:24.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.615+0000 7fbe29a8b740 1 -- 172.21.15.116:0/2228795150 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555d00c03d70 con 0x555d00d1bdb0 2024-09-12T03:17:24.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.615+0000 7fbe24c4d640 1 --2- 172.21.15.116:0/2228795150 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x555d00d1bdb0 0x555d00d1e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:24.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.615+0000 7fbe08ff9640 10 client.27457 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:24.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.615+0000 7fbe24c4d640 1 --2- 172.21.15.116:0/2228795150 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x555d00d1bdb0 0x555d00d1e1a0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec7f1dd56906a4f0 server_cookie=e9ff90446ab68376 in_seq=0 out_seq=0 2024-09-12T03:17:24.633 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.616+0000 7fbe08ff9640 1 -- 172.21.15.116:0/2228795150 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555d00c03d70 con 0x555d00d1bdb0 2024-09-12T03:17:24.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.616+0000 7fbe08ff9640 10 client.27457 handle_command_reply: tid=0 2024-09-12T03:17:24.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.616+0000 7fbe29a8b740 1 client.27457 shutdown 2024-09-12T03:17:24.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.616+0000 7fbe29a8b740 1 -- 172.21.15.116:0/2228795150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d00c965d0 msgr2=0x555d00ca98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:24.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.616+0000 7fbe29a8b740 1 --2- 172.21.15.116:0/2228795150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d00c965d0 0x555d00ca98a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c09b260 tx=0x7fbe1c0974c0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.616+0000 7fbe29a8b740 1 -- 172.21.15.116:0/2228795150 shutdown_connections 2024-09-12T03:17:24.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.616+0000 7fbe29a8b740 1 --2- 172.21.15.116:0/2228795150 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x555d00d1bdb0 0x555d00d1e1a0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.616+0000 7fbe29a8b740 1 --2- 172.21.15.116:0/2228795150 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d00ca1370 0x555d00ca5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.616+0000 7fbe29a8b740 1 --2- 172.21.15.116:0/2228795150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555d00c97b40 0x555d00ca4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.616+0000 7fbe08ff9640 0 client.27457 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:24.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.616+0000 7fbe29a8b740 1 --2- 172.21.15.116:0/2228795150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d00c965d0 0x555d00ca98a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.616+0000 7fbe29a8b740 1 -- 172.21.15.116:0/2228795150 >> 172.21.15.116:0/2228795150 conn(0x555d00c5f310 msgr2=0x555d00c9f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:24.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.616+0000 7fbe29a8b740 1 -- 172.21.15.116:0/2228795150 shutdown_connections 2024-09-12T03:17:24.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.616+0000 7fbe29a8b740 1 -- 172.21.15.116:0/2228795150 wait complete. 2024-09-12T03:17:24.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.616+0000 7fbe29a8b740 20 client.27457 trim_cache size 0 max 16384 2024-09-12T03:17:24.635 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:17:24.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.617+0000 7fbe276b2640 1 -- 172.21.15.116:0/2017054831 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbe0c0a2b80 msgr2=0x7fbe0c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:24.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.617+0000 7fbe276b2640 1 --2- 172.21.15.116:0/2017054831 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbe0c0a2b80 0x7fbe0c0a5040 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbe18005c40 tx=0x7fbe18009a20 comp rx=0 tx=0).stop 2024-09-12T03:17:24.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.617+0000 7fbe276b2640 1 -- 172.21.15.116:0/2017054831 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe20101de0 msgr2=0x7fbe20198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:24.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.617+0000 7fbe276b2640 1 --2- 172.21.15.116:0/2017054831 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe20101de0 0x7fbe20198450 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c00d950 tx=0x7fbe1c00de10 comp rx=0 tx=0).stop 2024-09-12T03:17:24.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.617+0000 7fbe276b2640 1 -- 172.21.15.116:0/2017054831 shutdown_connections 2024-09-12T03:17:24.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.617+0000 7fbe276b2640 1 --2- 172.21.15.116:0/2017054831 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbe0c0a2b80 0x7fbe0c0a5040 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.617+0000 7fbe276b2640 1 --2- 172.21.15.116:0/2017054831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe20106f10 0x7fbe2019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.617+0000 7fbe276b2640 1 --2- 172.21.15.116:0/2017054831 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe20102790 0x7fbe20198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.617+0000 7fbe276b2640 1 --2- 172.21.15.116:0/2017054831 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe20101de0 0x7fbe20198450 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:24.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.617+0000 7fbe276b2640 1 -- 172.21.15.116:0/2017054831 >> 172.21.15.116:0/2017054831 conn(0x7fbe200fd8c0 msgr2=0x7fbe200fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:24.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.617+0000 7fbe276b2640 1 -- 172.21.15.116:0/2017054831 shutdown_connections 2024-09-12T03:17:24.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:24.617+0000 7fbe276b2640 1 -- 172.21.15.116:0/2017054831 wait complete. 2024-09-12T03:17:24.881 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:25.532 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:25 smithi116 ceph-mon[31410]: pgmap v771: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-09-12T03:17:25.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:25 smithi177 ceph-mon[35305]: pgmap v771: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-09-12T03:17:25.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.900+0000 7f9099381640 1 -- 172.21.15.116:0/2662724497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9094100a80 msgr2=0x7f9094100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:25.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.900+0000 7f9099381640 1 --2- 172.21.15.116:0/2662724497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9094100a80 0x7f9094100ee0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f9088009930 tx=0x7f908802a420 comp rx=0 tx=0).stop 2024-09-12T03:17:25.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.900+0000 7f9099381640 1 -- 172.21.15.116:0/2662724497 shutdown_connections 2024-09-12T03:17:25.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.900+0000 7f9099381640 1 --2- 172.21.15.116:0/2662724497 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9094101420 0x7f909410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.900+0000 7f9099381640 1 --2- 172.21.15.116:0/2662724497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9094100a80 0x7f9094100ee0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.900+0000 7f9099381640 1 --2- 172.21.15.116:0/2662724497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9094106a80 0x7f9094106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.900+0000 7f9099381640 1 -- 172.21.15.116:0/2662724497 >> 172.21.15.116:0/2662724497 conn(0x7f90940fc7a0 msgr2=0x7f90940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:25.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.900+0000 7f9099381640 1 -- 172.21.15.116:0/2662724497 shutdown_connections 2024-09-12T03:17:25.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.900+0000 7f9099381640 1 -- 172.21.15.116:0/2662724497 wait complete. 2024-09-12T03:17:25.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.901+0000 7f9099381640 1 Processor -- start 2024-09-12T03:17:25.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.901+0000 7f9099381640 1 -- start start 2024-09-12T03:17:25.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.901+0000 7f9099381640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9094100a80 0x7f90941992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:25.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.901+0000 7f9099381640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9094101420 0x7f9094199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:25.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.901+0000 7f9099381640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9094106a80 0x7f90941a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:25.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.901+0000 7f9099381640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9094110e80 con 0x7f9094101420 2024-09-12T03:17:25.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.901+0000 7f9099381640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9094110d00 con 0x7f9094100a80 2024-09-12T03:17:25.913 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.901+0000 7f9099381640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9094111000 con 0x7f9094106a80 2024-09-12T03:17:25.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.902+0000 7f90927fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9094101420 0x7f9094199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:25.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.902+0000 7f90927fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9094101420 0x7f9094199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52120/0 (socket says 172.21.15.116:52120) 2024-09-12T03:17:25.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.902+0000 7f90927fc640 1 -- 172.21.15.116:0/393108768 learned_addr learned my addr 172.21.15.116:0/393108768 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:25.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.902+0000 7f9092ffd640 1 --2- 172.21.15.116:0/393108768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9094100a80 0x7f90941992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:25.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.902+0000 7f90927fc640 1 -- 172.21.15.116:0/393108768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9094106a80 msgr2=0x7f90941a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:25.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.902+0000 7f90927fc640 1 --2- 172.21.15.116:0/393108768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9094106a80 0x7f90941a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.902+0000 7f90927fc640 1 -- 172.21.15.116:0/393108768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9094100a80 msgr2=0x7f90941992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:25.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.902+0000 7f90927fc640 1 --2- 172.21.15.116:0/393108768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9094100a80 0x7f90941992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.902+0000 7f90927fc640 1 -- 172.21.15.116:0/393108768 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90941a4890 con 0x7f9094101420 2024-09-12T03:17:25.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.902+0000 7f90927fc640 1 --2- 172.21.15.116:0/393108768 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9094101420 0x7f9094199810 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f9088009230 tx=0x7f9088004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:25.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.902+0000 7f907bfff640 1 -- 172.21.15.116:0/393108768 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908803f030 con 0x7f9094101420 2024-09-12T03:17:25.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.902+0000 7f9099381640 1 -- 172.21.15.116:0/393108768 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90941a4a60 con 0x7f9094101420 2024-09-12T03:17:25.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.902+0000 7f907bfff640 1 -- 172.21.15.116:0/393108768 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9088003ad0 con 0x7f9094101420 2024-09-12T03:17:25.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.902+0000 7f907bfff640 1 -- 172.21.15.116:0/393108768 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088035070 con 0x7f9094101420 2024-09-12T03:17:25.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.902+0000 7f9099381640 1 -- 172.21.15.116:0/393108768 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90941a4f10 con 0x7f9094101420 2024-09-12T03:17:25.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.903+0000 7f9099381640 1 -- 172.21.15.116:0/393108768 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9094100ee0 con 0x7f9094101420 2024-09-12T03:17:25.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.905+0000 7f907bfff640 1 -- 172.21.15.116:0/393108768 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f9088042030 con 0x7f9094101420 2024-09-12T03:17:25.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.905+0000 7f907bfff640 1 --2- 172.21.15.116:0/393108768 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f907c0a2b80 0x7f907c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:25.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.905+0000 7f9092ffd640 1 --2- 172.21.15.116:0/393108768 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f907c0a2b80 0x7f907c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:25.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.905+0000 7f907bfff640 1 -- 172.21.15.116:0/393108768 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f90880ef9f0 con 0x7f9094101420 2024-09-12T03:17:25.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.906+0000 7f9092ffd640 1 --2- 172.21.15.116:0/393108768 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f907c0a2b80 0x7f907c0a5040 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9084005fd0 tx=0x7f908400b510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:25.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:25.908+0000 7f907bfff640 1 -- 172.21.15.116:0/393108768 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f90880bc980 con 0x7f9094101420 2024-09-12T03:17:25.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:25 smithi183 ceph-mon[35535]: pgmap v771: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-09-12T03:17:26.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:26.105+0000 7f9099381640 1 -- 172.21.15.116:0/393108768 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f909402d7c0 con 0x7f9094101420 2024-09-12T03:17:26.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:26.106+0000 7f907bfff640 1 -- 172.21.15.116:0/393108768 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f90880bc280 con 0x7f9094101420 2024-09-12T03:17:26.107 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:17:26.107 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:17:26.107 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:17:26.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:26.108+0000 7f9099381640 1 -- 172.21.15.116:0/393108768 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f907c0a2b80 msgr2=0x7f907c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:26.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:26.108+0000 7f9099381640 1 --2- 172.21.15.116:0/393108768 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f907c0a2b80 0x7f907c0a5040 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9084005fd0 tx=0x7f908400b510 comp rx=0 tx=0).stop 2024-09-12T03:17:26.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:26.108+0000 7f9099381640 1 -- 172.21.15.116:0/393108768 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9094101420 msgr2=0x7f9094199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:26.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:26.108+0000 7f9099381640 1 --2- 172.21.15.116:0/393108768 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9094101420 0x7f9094199810 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f9088009230 tx=0x7f9088004270 comp rx=0 tx=0).stop 2024-09-12T03:17:26.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:26.108+0000 7f9099381640 1 -- 172.21.15.116:0/393108768 shutdown_connections 2024-09-12T03:17:26.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:26.108+0000 7f9099381640 1 --2- 172.21.15.116:0/393108768 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f907c0a2b80 0x7f907c0a5040 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:26.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:26.108+0000 7f9099381640 1 --2- 172.21.15.116:0/393108768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9094106a80 0x7f90941a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:26.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:26.108+0000 7f9099381640 1 --2- 172.21.15.116:0/393108768 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9094101420 0x7f9094199810 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:26.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:26.108+0000 7f9099381640 1 --2- 172.21.15.116:0/393108768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9094100a80 0x7f90941992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:26.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:26.108+0000 7f9099381640 1 -- 172.21.15.116:0/393108768 >> 172.21.15.116:0/393108768 conn(0x7f90940fc7a0 msgr2=0x7f90940fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:26.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:26.109+0000 7f9099381640 1 -- 172.21.15.116:0/393108768 shutdown_connections 2024-09-12T03:17:26.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:26.109+0000 7f9099381640 1 -- 172.21.15.116:0/393108768 wait complete. 2024-09-12T03:17:26.379 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:17:26.379 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:17:26.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:26 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/393108768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:17:26.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:26 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/393108768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:17:26.761 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:26 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/393108768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:17:26.786 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id 1cfce56f --release --await' 2024-09-12T03:17:26.786 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id 1cfce56f --release --await 2024-09-12T03:17:26.860 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:27.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.353+0000 7f32ab1fe640 1 -- 172.21.15.116:0/2655892651 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32a4101420 msgr2=0x7f32a410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.353+0000 7f32ab1fe640 1 --2- 172.21.15.116:0/2655892651 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32a4101420 0x7f32a410d950 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f329c009cd0 tx=0x7f329c02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.353+0000 7f32ab1fe640 1 -- 172.21.15.116:0/2655892651 shutdown_connections 2024-09-12T03:17:27.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.353+0000 7f32ab1fe640 1 --2- 172.21.15.116:0/2655892651 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32a4101420 0x7f32a410d950 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.353+0000 7f32ab1fe640 1 --2- 172.21.15.116:0/2655892651 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32a4100a80 0x7f32a4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.353+0000 7f32ab1fe640 1 --2- 172.21.15.116:0/2655892651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32a4106a80 0x7f32a4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.353+0000 7f32ab1fe640 1 -- 172.21.15.116:0/2655892651 >> 172.21.15.116:0/2655892651 conn(0x7f32a40fc7a0 msgr2=0x7f32a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:27.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.353+0000 7f32ab1fe640 1 -- 172.21.15.116:0/2655892651 shutdown_connections 2024-09-12T03:17:27.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.353+0000 7f32ab1fe640 1 -- 172.21.15.116:0/2655892651 wait complete. 2024-09-12T03:17:27.374 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.354+0000 7f32ab1fe640 1 Processor -- start 2024-09-12T03:17:27.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.354+0000 7f32ab1fe640 1 -- start start 2024-09-12T03:17:27.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.354+0000 7f32ab1fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32a4100a80 0x7f32a4102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.354+0000 7f32ab1fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32a4101420 0x7f32a4102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.354+0000 7f32ab1fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32a4106a80 0x7f32a41030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.354+0000 7f32ab1fe640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f32a4110f20 con 0x7f32a4100a80 2024-09-12T03:17:27.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.354+0000 7f32ab1fe640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f32a4110da0 con 0x7f32a4101420 2024-09-12T03:17:27.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.354+0000 7f32ab1fe640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f32a41110a0 con 0x7f32a4106a80 2024-09-12T03:17:27.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.354+0000 7f32a8f9a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32a4100a80 0x7f32a4102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.354+0000 7f32a8f9a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32a4100a80 0x7f32a4102650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52142/0 (socket says 172.21.15.116:52142) 2024-09-12T03:17:27.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.354+0000 7f32a8f9a640 1 -- 172.21.15.116:0/2543889338 learned_addr learned my addr 172.21.15.116:0/2543889338 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:27.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.354+0000 7f32a979b640 1 --2- 172.21.15.116:0/2543889338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32a4106a80 0x7f32a41030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.354+0000 7f32a3fff640 1 --2- 172.21.15.116:0/2543889338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32a4101420 0x7f32a4102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.375 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.354+0000 7f32a8f9a640 1 -- 172.21.15.116:0/2543889338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32a4106a80 msgr2=0x7f32a41030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.354+0000 7f32a8f9a640 1 --2- 172.21.15.116:0/2543889338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32a4106a80 0x7f32a41030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.354+0000 7f32a8f9a640 1 -- 172.21.15.116:0/2543889338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32a4101420 msgr2=0x7f32a4102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.354+0000 7f32a8f9a640 1 --2- 172.21.15.116:0/2543889338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32a4101420 0x7f32a4102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.354+0000 7f32a8f9a640 1 -- 172.21.15.116:0/2543889338 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a41a6a10 con 0x7f32a4100a80 2024-09-12T03:17:27.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.354+0000 7f32a3fff640 1 --2- 172.21.15.116:0/2543889338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32a4101420 0x7f32a4102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:27.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.354+0000 7f32a8f9a640 1 --2- 172.21.15.116:0/2543889338 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32a4100a80 0x7f32a4102650 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f329800b4a0 tx=0x7f329800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:27.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.355+0000 7f32a1ffb640 1 -- 172.21.15.116:0/2543889338 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298014070 con 0x7f32a4100a80 2024-09-12T03:17:27.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.355+0000 7f32a1ffb640 1 -- 172.21.15.116:0/2543889338 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3298010af0 con 0x7f32a4100a80 2024-09-12T03:17:27.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.355+0000 7f32ab1fe640 1 -- 172.21.15.116:0/2543889338 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a41a6d00 con 0x7f32a4100a80 2024-09-12T03:17:27.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.355+0000 7f32a1ffb640 1 -- 172.21.15.116:0/2543889338 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298017d10 con 0x7f32a4100a80 2024-09-12T03:17:27.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.355+0000 7f32ab1fe640 1 -- 172.21.15.116:0/2543889338 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a41a7190 con 0x7f32a4100a80 2024-09-12T03:17:27.376 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.357+0000 7f32a1ffb640 1 -- 172.21.15.116:0/2543889338 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3298017070 con 0x7f32a4100a80 2024-09-12T03:17:27.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.357+0000 7f32ad5d7740 1 Processor -- start 2024-09-12T03:17:27.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.357+0000 7f32ad5d7740 1 -- start start 2024-09-12T03:17:27.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.358+0000 7f32ad5d7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4db477b60 0x55e4db477f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.358+0000 7f32ad5d7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4db478480 0x55e4db482800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.358+0000 7f32ad5d7740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4db482d40 0x55e4db485130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.358+0000 7f32ad5d7740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e4db3dd280 con 0x55e4db482d40 2024-09-12T03:17:27.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.358+0000 7f32ad5d7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e4db384b10 con 0x55e4db477b60 2024-09-12T03:17:27.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.358+0000 7f32ad5d7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e4db3d9240 con 0x55e4db478480 2024-09-12T03:17:27.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.358+0000 7f32a3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4db482d40 0x55e4db485130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.358+0000 7f32a3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4db482d40 0x55e4db485130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52150/0 (socket says 172.21.15.116:52150) 2024-09-12T03:17:27.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.358+0000 7f32a3fff640 1 -- 172.21.15.116:0/2829679007 learned_addr learned my addr 172.21.15.116:0/2829679007 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:27.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.358+0000 7f32a979b640 1 --2- 172.21.15.116:0/2829679007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4db477b60 0x55e4db477f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.377 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.358+0000 7f32a1ffb640 1 --2- 172.21.15.116:0/2543889338 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f32940a2b80 0x7f32940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.358+0000 7f32a8f9a640 1 --2- 172.21.15.116:0/2829679007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4db478480 0x55e4db482800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.358+0000 7f32a979b640 1 --2- 172.21.15.116:0/2543889338 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f32940a2b80 0x7f32940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.358+0000 7f32a3fff640 1 -- 172.21.15.116:0/2829679007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4db478480 msgr2=0x55e4db482800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:27.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.358+0000 7f32a3fff640 1 --2- 172.21.15.116:0/2829679007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4db478480 0x55e4db482800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.358+0000 7f32a3fff640 1 -- 172.21.15.116:0/2829679007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4db477b60 msgr2=0x55e4db477f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:27.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.358+0000 7f32a3fff640 1 --2- 172.21.15.116:0/2829679007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4db477b60 0x55e4db477f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.358+0000 7f32a1ffb640 1 -- 172.21.15.116:0/2543889338 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f32980ce440 con 0x7f32a4100a80 2024-09-12T03:17:27.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.358+0000 7f32a3fff640 1 -- 172.21.15.116:0/2829679007 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4db3911f0 con 0x55e4db482d40 2024-09-12T03:17:27.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.358+0000 7f32a8f9a640 1 --2- 172.21.15.116:0/2829679007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4db478480 0x55e4db482800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:27.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.358+0000 7f32a3fff640 1 --2- 172.21.15.116:0/2829679007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4db482d40 0x55e4db485130 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f329000a9d0 tx=0x7f329000ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a058fc99e333dc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:27.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.359+0000 7f328effd640 1 -- 172.21.15.116:0/2829679007 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290015070 con 0x55e4db482d40 2024-09-12T03:17:27.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.359+0000 7f32a979b640 1 --2- 172.21.15.116:0/2543889338 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f32940a2b80 0x7f32940a5040 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f329c009a60 tx=0x7f329c02a8e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:27.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.359+0000 7f328effd640 1 -- 172.21.15.116:0/2829679007 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f329000cc60 con 0x55e4db482d40 2024-09-12T03:17:27.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.359+0000 7f328effd640 1 -- 172.21.15.116:0/2829679007 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290018600 con 0x55e4db482d40 2024-09-12T03:17:27.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.359+0000 7f32ad5d7740 1 -- 172.21.15.116:0/2829679007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4db482d40 msgr2=0x55e4db485130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.359+0000 7f32ad5d7740 1 --2- 172.21.15.116:0/2829679007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4db482d40 0x55e4db485130 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f329000a9d0 tx=0x7f329000ae90 comp rx=0 tx=0).stop 2024-09-12T03:17:27.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.359+0000 7f32ad5d7740 1 -- 172.21.15.116:0/2829679007 shutdown_connections 2024-09-12T03:17:27.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.359+0000 7f32ad5d7740 1 --2- 172.21.15.116:0/2829679007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4db482d40 0x55e4db485130 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.359+0000 7f32ad5d7740 1 --2- 172.21.15.116:0/2829679007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4db478480 0x55e4db482800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.359+0000 7f32ad5d7740 1 --2- 172.21.15.116:0/2829679007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4db477b60 0x55e4db477f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.359+0000 7f32ad5d7740 1 -- 172.21.15.116:0/2829679007 >> 172.21.15.116:0/2829679007 conn(0x55e4db4740f0 msgr2=0x55e4db4744d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:27.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.359+0000 7f32ad5d7740 1 -- 172.21.15.116:0/2829679007 shutdown_connections 2024-09-12T03:17:27.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.359+0000 7f32ad5d7740 1 -- 172.21.15.116:0/2829679007 wait complete. 2024-09-12T03:17:27.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.360+0000 7f32ad5d7740 1 -- start start 2024-09-12T03:17:27.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.360+0000 7f32ad5d7740 1 Processor -- start 2024-09-12T03:17:27.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.360+0000 7f32ad5d7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4db477b60 0x55e4db47fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.360+0000 7f32ad5d7740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4db478480 0x55e4db4800f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.360+0000 7f32ad5d7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4db482d40 0x55e4db480630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.360+0000 7f32ad5d7740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e4db485910 con 0x55e4db478480 2024-09-12T03:17:27.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.360+0000 7f32ad5d7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e4db397f10 con 0x55e4db482d40 2024-09-12T03:17:27.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.360+0000 7f32ad5d7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e4db485a90 con 0x55e4db477b60 2024-09-12T03:17:27.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.360+0000 7f32ad5d7740 10 client.0 _ll_register_callbacks cb 0x55e4db360a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:27.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.360+0000 7f32a3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4db478480 0x55e4db4800f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.360+0000 7f32a3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4db478480 0x55e4db4800f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52160/0 (socket says 172.21.15.116:52160) 2024-09-12T03:17:27.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.360+0000 7f32a3fff640 1 -- 172.21.15.116:0/1129449097 learned_addr learned my addr 172.21.15.116:0/1129449097 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:27.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.360+0000 7f32a8f9a640 1 --2- 172.21.15.116:0/1129449097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4db477b60 0x55e4db47fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.361+0000 7f32a3fff640 1 -- 172.21.15.116:0/1129449097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4db477b60 msgr2=0x55e4db47fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:27.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.361+0000 7f32a3fff640 1 --2- 172.21.15.116:0/1129449097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4db477b60 0x55e4db47fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.361+0000 7f32a3fff640 1 -- 172.21.15.116:0/1129449097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4db482d40 msgr2=0x55e4db480630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:27.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.361+0000 7f32a3fff640 1 --2- 172.21.15.116:0/1129449097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4db482d40 0x55e4db480630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.361+0000 7f32a3fff640 1 -- 172.21.15.116:0/1129449097 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4db14e6a0 con 0x55e4db478480 2024-09-12T03:17:27.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.361+0000 7f32a8f9a640 1 --2- 172.21.15.116:0/1129449097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4db477b60 0x55e4db47fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:27.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.361+0000 7f32a3fff640 1 --2- 172.21.15.116:0/1129449097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4db478480 0x55e4db4800f0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f329000a1a0 tx=0x7f3290018590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70068f0dbd515b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:27.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.361+0000 7f3283fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:27.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.361+0000 7f3283fff640 1 -- 172.21.15.116:0/1129449097 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290020070 con 0x55e4db478480 2024-09-12T03:17:27.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.361+0000 7f3283fff640 1 -- 172.21.15.116:0/1129449097 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f3290018990 con 0x55e4db478480 2024-09-12T03:17:27.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.361+0000 7f32ad5d7740 1 -- 172.21.15.116:0/1129449097 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f329000cc60 con 0x55e4db478480 2024-09-12T03:17:27.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.361+0000 7f3283fff640 1 -- 172.21.15.116:0/1129449097 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3290015040 con 0x55e4db478480 2024-09-12T03:17:27.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.361+0000 7f32ad5d7740 10 client.19260 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:27.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.361+0000 7f32ad5d7740 1 -- 172.21.15.116:0/1129449097 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e4db4914d0 con 0x55e4db478480 2024-09-12T03:17:27.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.361+0000 7f3283fff640 1 -- 172.21.15.116:0/1129449097 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f329000cc60 con 0x55e4db478480 2024-09-12T03:17:27.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.361+0000 7f3283fff640 1 -- 172.21.15.116:0/1129449097 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f329001a930 con 0x55e4db478480 2024-09-12T03:17:27.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.361+0000 7f32ad5d7740 10 client.19260 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:27.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.361+0000 7f32ad5d7740 10 client.19260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:17:27.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.361+0000 7f32ad5d7740 20 client.19260 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:27.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.361+0000 7f32ad5d7740 1 --2- 172.21.15.116:0/1129449097 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e4db492930 0x55e4db494d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.361+0000 7f32ad5d7740 4 client.19260 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:17:27.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.361+0000 7f32ad5d7740 1 -- 172.21.15.116:0/1129449097 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4db3ea7f0 con 0x55e4db492930 2024-09-12T03:17:27.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.362+0000 7f32a8f9a640 1 --2- 172.21.15.116:0/1129449097 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e4db492930 0x55e4db494d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.362+0000 7f3283fff640 10 client.19260 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:27.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.362+0000 7f32a8f9a640 1 --2- 172.21.15.116:0/1129449097 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e4db492930 0x55e4db494d20 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dbdad54a886f9bb server_cookie=64487a4c1389ed14 in_seq=0 out_seq=0 2024-09-12T03:17:27.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.363+0000 7f3283fff640 1 -- 172.21.15.116:0/1129449097 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e4db3ea7f0 con 0x55e4db492930 2024-09-12T03:17:27.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.363+0000 7f3283fff640 10 client.19260 handle_command_reply: tid=0 2024-09-12T03:17:27.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.363+0000 7f32ad5d7740 1 client.19260 shutdown 2024-09-12T03:17:27.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.363+0000 7f32ad5d7740 1 -- 172.21.15.116:0/1129449097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4db478480 msgr2=0x55e4db4800f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.363+0000 7f32ad5d7740 1 --2- 172.21.15.116:0/1129449097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4db478480 0x55e4db4800f0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f329000a1a0 tx=0x7f3290018590 comp rx=0 tx=0).stop 2024-09-12T03:17:27.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.363+0000 7f32ad5d7740 1 -- 172.21.15.116:0/1129449097 shutdown_connections 2024-09-12T03:17:27.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.363+0000 7f32ad5d7740 1 --2- 172.21.15.116:0/1129449097 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e4db492930 0x55e4db494d20 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.363+0000 7f32ad5d7740 1 --2- 172.21.15.116:0/1129449097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4db482d40 0x55e4db480630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.363+0000 7f32ad5d7740 1 --2- 172.21.15.116:0/1129449097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4db478480 0x55e4db4800f0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.363+0000 7f32ad5d7740 1 --2- 172.21.15.116:0/1129449097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4db477b60 0x55e4db47fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.363+0000 7f3283fff640 0 client.19260 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:27.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.363+0000 7f32ad5d7740 1 -- 172.21.15.116:0/1129449097 >> 172.21.15.116:0/1129449097 conn(0x55e4db4740f0 msgr2=0x55e4db479dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:27.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.363+0000 7f32ad5d7740 1 -- 172.21.15.116:0/1129449097 shutdown_connections 2024-09-12T03:17:27.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.363+0000 7f32ad5d7740 1 -- 172.21.15.116:0/1129449097 wait complete. 2024-09-12T03:17:27.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.363+0000 7f32ad5d7740 20 client.19260 trim_cache size 0 max 16384 2024-09-12T03:17:27.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.381+0000 7f32ad5d7740 1 Processor -- start 2024-09-12T03:17:27.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.381+0000 7f32ad5d7740 1 -- start start 2024-09-12T03:17:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.381+0000 7f32ad5d7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4db477b60 0x55e4db479cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.381+0000 7f32ad5d7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4db478480 0x55e4db49a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.381+0000 7f32ad5d7740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4db482d40 0x55e4db49a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.381+0000 7f32ad5d7740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f32a41b2030 con 0x55e4db482d40 2024-09-12T03:17:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.381+0000 7f32ad5d7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f32a41af900 con 0x55e4db478480 2024-09-12T03:17:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.382+0000 7f32ad5d7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f32a41ad750 con 0x55e4db477b60 2024-09-12T03:17:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.382+0000 7f32a8f9a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4db477b60 0x55e4db479cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.382+0000 7f32a8f9a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4db477b60 0x55e4db479cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.116:41648/0 (socket says 172.21.15.116:41648) 2024-09-12T03:17:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.382+0000 7f32a8f9a640 1 -- 172.21.15.116:0/613151937 learned_addr learned my addr 172.21.15.116:0/613151937 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.382+0000 7f32a3fff640 1 --2- 172.21.15.116:0/613151937 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4db478480 0x55e4db49a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.382+0000 7f32a979b640 1 --2- 172.21.15.116:0/613151937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4db482d40 0x55e4db49a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.382+0000 7f32a8f9a640 1 -- 172.21.15.116:0/613151937 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4db478480 msgr2=0x55e4db49a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.382+0000 7f32a8f9a640 1 --2- 172.21.15.116:0/613151937 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4db478480 0x55e4db49a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.382+0000 7f32a8f9a640 1 -- 172.21.15.116:0/613151937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4db482d40 msgr2=0x55e4db49a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:27.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.382+0000 7f32a8f9a640 1 --2- 172.21.15.116:0/613151937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4db482d40 0x55e4db49a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.382+0000 7f32a8f9a640 1 -- 172.21.15.116:0/613151937 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4db473f20 con 0x55e4db477b60 2024-09-12T03:17:27.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.382+0000 7f32a8f9a640 1 --2- 172.21.15.116:0/613151937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4db477b60 0x55e4db479cc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f329800be60 tx=0x7f32980047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdd4ebdc744166dc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:27.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.383+0000 7f328e7fc640 1 -- 172.21.15.116:0/613151937 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329801e040 con 0x55e4db477b60 2024-09-12T03:17:27.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.383+0000 7f328e7fc640 1 -- 172.21.15.116:0/613151937 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3298098f70 con 0x55e4db477b60 2024-09-12T03:17:27.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.383+0000 7f328e7fc640 1 -- 172.21.15.116:0/613151937 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329809a0c0 con 0x55e4db477b60 2024-09-12T03:17:27.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.384+0000 7f32ad5d7740 1 -- 172.21.15.116:0/613151937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4db477b60 msgr2=0x55e4db479cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.384+0000 7f32ad5d7740 1 --2- 172.21.15.116:0/613151937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4db477b60 0x55e4db479cc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f329800be60 tx=0x7f32980047d0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.384+0000 7f32ad5d7740 1 -- 172.21.15.116:0/613151937 shutdown_connections 2024-09-12T03:17:27.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.384+0000 7f32ad5d7740 1 --2- 172.21.15.116:0/613151937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4db482d40 0x55e4db49a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.384+0000 7f32ad5d7740 1 --2- 172.21.15.116:0/613151937 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4db478480 0x55e4db49a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.384+0000 7f32ad5d7740 1 --2- 172.21.15.116:0/613151937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4db477b60 0x55e4db479cc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.384+0000 7f32ad5d7740 1 -- 172.21.15.116:0/613151937 >> 172.21.15.116:0/613151937 conn(0x55e4db4740f0 msgr2=0x55e4db4795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:27.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.384+0000 7f32ad5d7740 1 -- 172.21.15.116:0/613151937 shutdown_connections 2024-09-12T03:17:27.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.384+0000 7f32ad5d7740 1 -- 172.21.15.116:0/613151937 wait complete. 2024-09-12T03:17:27.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.384+0000 7f32ad5d7740 1 -- start start 2024-09-12T03:17:27.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.385+0000 7f32ad5d7740 1 Processor -- start 2024-09-12T03:17:27.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.385+0000 7f32ad5d7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4db477b60 0x55e4db48ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.385+0000 7f32ad5d7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4db478480 0x55e4db48b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.385+0000 7f32ad5d7740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4db482d40 0x55e4db487ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.385+0000 7f32ad5d7740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f32a41addc0 con 0x55e4db482d40 2024-09-12T03:17:27.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.385+0000 7f32ad5d7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f32a41af230 con 0x55e4db478480 2024-09-12T03:17:27.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.385+0000 7f32ad5d7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f32a41ae490 con 0x55e4db477b60 2024-09-12T03:17:27.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.385+0000 7f32a8f9a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4db477b60 0x55e4db48ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.385+0000 7f32a8f9a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4db477b60 0x55e4db48ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.116:41656/0 (socket says 172.21.15.116:41656) 2024-09-12T03:17:27.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.385+0000 7f32a8f9a640 1 -- 172.21.15.116:0/1424718702 learned_addr learned my addr 172.21.15.116:0/1424718702 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:27.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.385+0000 7f32ad5d7740 10 client.0 _ll_register_callbacks cb 0x7f329c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:27.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.385+0000 7f32a979b640 1 --2- 172.21.15.116:0/1424718702 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4db482d40 0x55e4db487ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.386+0000 7f32a3fff640 1 --2- 172.21.15.116:0/1424718702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4db478480 0x55e4db48b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.386+0000 7f32a8f9a640 1 -- 172.21.15.116:0/1424718702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4db478480 msgr2=0x55e4db48b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:27.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.386+0000 7f32a8f9a640 1 --2- 172.21.15.116:0/1424718702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4db478480 0x55e4db48b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.386+0000 7f32a8f9a640 1 -- 172.21.15.116:0/1424718702 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4db482d40 msgr2=0x55e4db487ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:27.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.386+0000 7f32a8f9a640 1 --2- 172.21.15.116:0/1424718702 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4db482d40 0x55e4db487ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.386+0000 7f32a8f9a640 1 -- 172.21.15.116:0/1424718702 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4db488220 con 0x55e4db477b60 2024-09-12T03:17:27.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.386+0000 7f32a979b640 1 --2- 172.21.15.116:0/1424718702 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4db482d40 0x55e4db487ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:17:27.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.386+0000 7f32a3fff640 1 --2- 172.21.15.116:0/1424718702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4db478480 0x55e4db48b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:17:27.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.386+0000 7f32a8f9a640 1 --2- 172.21.15.116:0/1424718702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4db477b60 0x55e4db48ad10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3298017690 tx=0x7f32980092f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec142bd21044e4ff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:27.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.386+0000 7f3283fff640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T03:17:27.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.386+0000 7f3283fff640 1 -- 172.21.15.116:0/1424718702 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32980b8060 con 0x55e4db477b60 2024-09-12T03:17:27.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.386+0000 7f3283fff640 1 -- 172.21.15.116:0/1424718702 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f3298010ca0 con 0x55e4db477b60 2024-09-12T03:17:27.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.386+0000 7f32ad5d7740 1 -- 172.21.15.116:0/1424718702 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3298098f70 con 0x55e4db477b60 2024-09-12T03:17:27.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.387+0000 7f3283fff640 1 -- 172.21.15.116:0/1424718702 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f32980b3070 con 0x55e4db477b60 2024-09-12T03:17:27.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.387+0000 7f32ad5d7740 10 client.27472 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:27.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.387+0000 7f32ad5d7740 1 -- 172.21.15.116:0/1424718702 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e4db4fdb20 con 0x55e4db477b60 2024-09-12T03:17:27.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.387+0000 7f3283fff640 1 -- 172.21.15.116:0/1424718702 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3298098f70 con 0x55e4db477b60 2024-09-12T03:17:27.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.387+0000 7f3283fff640 1 -- 172.21.15.116:0/1424718702 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f32980969c0 con 0x55e4db477b60 2024-09-12T03:17:27.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.387+0000 7f32ad5d7740 10 client.27472 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:27.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.387+0000 7f32ad5d7740 10 client.27472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:17:27.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.387+0000 7f32ad5d7740 20 client.27472 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:27.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.387+0000 7f32ad5d7740 1 --2- 172.21.15.116:0/1424718702 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e4db4fe1a0 0x55e4db5005c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:27.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.387+0000 7f32ad5d7740 4 client.27472 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:17:27.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.387+0000 7f32ad5d7740 1 -- 172.21.15.116:0/1424718702 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e4db3e4d00 con 0x55e4db4fe1a0 2024-09-12T03:17:27.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.388+0000 7f32a3fff640 1 --2- 172.21.15.116:0/1424718702 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e4db4fe1a0 0x55e4db5005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:27.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.388+0000 7f3283fff640 10 client.27472 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:27.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.388+0000 7f32a3fff640 1 --2- 172.21.15.116:0/1424718702 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e4db4fe1a0 0x55e4db5005c0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cba15ee1f03674b1 server_cookie=1d802d4f1e3d2765 in_seq=0 out_seq=0 2024-09-12T03:17:27.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.389+0000 7f3283fff640 1 -- 172.21.15.116:0/1424718702 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e4db3e4d00 con 0x55e4db4fe1a0 2024-09-12T03:17:27.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.389+0000 7f3283fff640 10 client.27472 handle_command_reply: tid=0 2024-09-12T03:17:27.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.389+0000 7f32ad5d7740 1 client.27472 shutdown 2024-09-12T03:17:27.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.390+0000 7f32ad5d7740 1 -- 172.21.15.116:0/1424718702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4db477b60 msgr2=0x55e4db48ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.390+0000 7f32ad5d7740 1 --2- 172.21.15.116:0/1424718702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4db477b60 0x55e4db48ad10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3298017690 tx=0x7f32980092f0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.390+0000 7f32ad5d7740 1 -- 172.21.15.116:0/1424718702 shutdown_connections 2024-09-12T03:17:27.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.390+0000 7f32ad5d7740 1 --2- 172.21.15.116:0/1424718702 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55e4db4fe1a0 0x55e4db5005c0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.390+0000 7f32ad5d7740 1 --2- 172.21.15.116:0/1424718702 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e4db482d40 0x55e4db487ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.390+0000 7f3283fff640 0 client.27472 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:27.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.390+0000 7f32ad5d7740 1 --2- 172.21.15.116:0/1424718702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e4db478480 0x55e4db48b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.390+0000 7f32ad5d7740 1 --2- 172.21.15.116:0/1424718702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4db477b60 0x55e4db48ad10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.390+0000 7f32ad5d7740 1 -- 172.21.15.116:0/1424718702 >> 172.21.15.116:0/1424718702 conn(0x55e4db4740f0 msgr2=0x55e4db479dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:27.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.390+0000 7f32ad5d7740 1 -- 172.21.15.116:0/1424718702 shutdown_connections 2024-09-12T03:17:27.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.390+0000 7f32ad5d7740 1 -- 172.21.15.116:0/1424718702 wait complete. 2024-09-12T03:17:27.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.390+0000 7f32ad5d7740 20 client.27472 trim_cache size 0 max 16384 2024-09-12T03:17:27.397 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:17:27.397 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:17:27.397 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "cc7efb26-d85f-45b1-b169-cbebdf16c890", 2024-09-12T03:17:27.397 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:17:27.397 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:17:27.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.391+0000 7f32ab1fe640 1 -- 172.21.15.116:0/2543889338 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f32940a2b80 msgr2=0x7f32940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.391+0000 7f32ab1fe640 1 --2- 172.21.15.116:0/2543889338 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f32940a2b80 0x7f32940a5040 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f329c009a60 tx=0x7f329c02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.391+0000 7f32ab1fe640 1 -- 172.21.15.116:0/2543889338 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32a4100a80 msgr2=0x7f32a4102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.391+0000 7f32ab1fe640 1 --2- 172.21.15.116:0/2543889338 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32a4100a80 0x7f32a4102650 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f329800b4a0 tx=0x7f329800b960 comp rx=0 tx=0).stop 2024-09-12T03:17:27.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.391+0000 7f32ab1fe640 1 -- 172.21.15.116:0/2543889338 shutdown_connections 2024-09-12T03:17:27.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.391+0000 7f32ab1fe640 1 --2- 172.21.15.116:0/2543889338 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f32940a2b80 0x7f32940a5040 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.391+0000 7f32ab1fe640 1 --2- 172.21.15.116:0/2543889338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32a4106a80 0x7f32a41030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.391+0000 7f32ab1fe640 1 --2- 172.21.15.116:0/2543889338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32a4101420 0x7f32a4102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.391+0000 7f32ab1fe640 1 --2- 172.21.15.116:0/2543889338 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f32a4100a80 0x7f32a4102650 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.391+0000 7f32ab1fe640 1 -- 172.21.15.116:0/2543889338 >> 172.21.15.116:0/2543889338 conn(0x7f32a40fc7a0 msgr2=0x7f32a41048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:27.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.391+0000 7f32ab1fe640 1 -- 172.21.15.116:0/2543889338 shutdown_connections 2024-09-12T03:17:27.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:27.392+0000 7f32ab1fe640 1 -- 172.21.15.116:0/2543889338 wait complete. 2024-09-12T03:17:27.738 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:27.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:27 smithi177 ceph-mon[35305]: pgmap v772: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:17:27.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:27 smithi116 ceph-mon[31410]: pgmap v772: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:17:27.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:27 smithi183 ceph-mon[35535]: pgmap v772: 97 pgs: 97 active+clean; 6.7 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-12T03:17:28.223 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:17:28.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:28 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:17:28.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:28 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:17:28.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:28 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:17:28.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:28 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:17:28.784 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:28 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:17:28.784 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:28 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:17:28.784 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:28 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:17:28.785 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:28 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:17:28.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.784+0000 7f4676d99640 1 -- 172.21.15.116:0/124453759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4670106f10 msgr2=0x7f467010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.784+0000 7f4676d99640 1 --2- 172.21.15.116:0/124453759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4670106f10 0x7f467010ea90 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f4668009ca0 tx=0x7f466802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.784+0000 7f4676d99640 1 -- 172.21.15.116:0/124453759 shutdown_connections 2024-09-12T03:17:28.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.784+0000 7f4676d99640 1 --2- 172.21.15.116:0/124453759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4670106f10 0x7f467010ea90 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.784+0000 7f4676d99640 1 --2- 172.21.15.116:0/124453759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4670102790 0x7f46701067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.784+0000 7f4676d99640 1 --2- 172.21.15.116:0/124453759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4670101de0 0x7f46701021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.784+0000 7f4676d99640 1 -- 172.21.15.116:0/124453759 >> 172.21.15.116:0/124453759 conn(0x7f46700fd8c0 msgr2=0x7f46700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:28.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.784+0000 7f4676d99640 1 -- 172.21.15.116:0/124453759 shutdown_connections 2024-09-12T03:17:28.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.784+0000 7f4676d99640 1 -- 172.21.15.116:0/124453759 wait complete. 2024-09-12T03:17:28.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.785+0000 7f4676d99640 1 Processor -- start 2024-09-12T03:17:28.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.785+0000 7f4676d99640 1 -- start start 2024-09-12T03:17:28.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.785+0000 7f4676d99640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4670101de0 0x7f4670198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.785+0000 7f4676d99640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4670102790 0x7f4670198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.785+0000 7f4676d99640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4670106f10 0x7f467019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.785+0000 7f4676d99640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4670111fb0 con 0x7f4670106f10 2024-09-12T03:17:28.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.785+0000 7f4676d99640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4670111e30 con 0x7f4670101de0 2024-09-12T03:17:28.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.785+0000 7f4676d99640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4670112130 con 0x7f4670102790 2024-09-12T03:17:28.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.785+0000 7f4675336640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4670106f10 0x7f467019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.785+0000 7f4675336640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4670106f10 0x7f467019cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52198/0 (socket says 172.21.15.116:52198) 2024-09-12T03:17:28.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.785+0000 7f4675336640 1 -- 172.21.15.116:0/2234064872 learned_addr learned my addr 172.21.15.116:0/2234064872 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:28.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.786+0000 7f4674b35640 1 --2- 172.21.15.116:0/2234064872 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4670101de0 0x7f4670198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.786+0000 7f466ffff640 1 --2- 172.21.15.116:0/2234064872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4670102790 0x7f4670198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.786+0000 7f4675336640 1 -- 172.21.15.116:0/2234064872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4670102790 msgr2=0x7f4670198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.786+0000 7f4675336640 1 --2- 172.21.15.116:0/2234064872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4670102790 0x7f4670198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.786+0000 7f4675336640 1 -- 172.21.15.116:0/2234064872 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4670101de0 msgr2=0x7f4670198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.786+0000 7f4675336640 1 --2- 172.21.15.116:0/2234064872 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4670101de0 0x7f4670198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.786+0000 7f4675336640 1 -- 172.21.15.116:0/2234064872 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f467019d3e0 con 0x7f4670106f10 2024-09-12T03:17:28.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.786+0000 7f4675336640 1 --2- 172.21.15.116:0/2234064872 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4670106f10 0x7f467019cd00 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f4668031a20 tx=0x7f4668031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:28.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.786+0000 7f466dffb640 1 -- 172.21.15.116:0/2234064872 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668003b70 con 0x7f4670106f10 2024-09-12T03:17:28.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.786+0000 7f4676d99640 1 -- 172.21.15.116:0/2234064872 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f467019d670 con 0x7f4670106f10 2024-09-12T03:17:28.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.786+0000 7f466dffb640 1 -- 172.21.15.116:0/2234064872 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4668003d10 con 0x7f4670106f10 2024-09-12T03:17:28.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.786+0000 7f466dffb640 1 -- 172.21.15.116:0/2234064872 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466802fc80 con 0x7f4670106f10 2024-09-12T03:17:28.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.786+0000 7f4676d99640 1 -- 172.21.15.116:0/2234064872 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46701a4fb0 con 0x7f4670106f10 2024-09-12T03:17:28.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.788+0000 7f466dffb640 1 -- 172.21.15.116:0/2234064872 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f46680046a0 con 0x7f4670106f10 2024-09-12T03:17:28.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.789+0000 7f4679172740 1 Processor -- start 2024-09-12T03:17:28.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.789+0000 7f4679172740 1 -- start start 2024-09-12T03:17:28.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.789+0000 7f4679172740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce1dc99060 0x55ce1dc99440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.789+0000 7f4679172740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce1dc99980 0x55ce1dca3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.789+0000 7f4679172740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce1dca4240 0x55ce1dca6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.789+0000 7f4679172740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ce1dbfe620 con 0x55ce1dca4240 2024-09-12T03:17:28.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.789+0000 7f4679172740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ce1dba5e50 con 0x55ce1dc99060 2024-09-12T03:17:28.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.789+0000 7f4679172740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ce1dbfa5e0 con 0x55ce1dc99980 2024-09-12T03:17:28.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.789+0000 7f466dffb640 1 --2- 172.21.15.116:0/2234064872 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f46600a2b80 0x7f46600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.789+0000 7f466ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce1dca4240 0x55ce1dca6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.789+0000 7f466ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce1dca4240 0x55ce1dca6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52202/0 (socket says 172.21.15.116:52202) 2024-09-12T03:17:28.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.789+0000 7f466ffff640 1 -- 172.21.15.116:0/1640806797 learned_addr learned my addr 172.21.15.116:0/1640806797 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:28.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.789+0000 7f4675336640 1 --2- 172.21.15.116:0/2234064872 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f46600a2b80 0x7f46600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.789+0000 7f466ffff640 1 --2- 172.21.15.116:0/1640806797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce1dc99060 0x55ce1dc99440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.789+0000 7f4674b35640 1 --2- 172.21.15.116:0/1640806797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce1dc99980 0x55ce1dca3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.790+0000 7f466dffb640 1 -- 172.21.15.116:0/2234064872 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f46680f01c0 con 0x7f4670106f10 2024-09-12T03:17:28.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.790+0000 7f466ffff640 1 -- 172.21.15.116:0/1640806797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce1dc99980 msgr2=0x55ce1dca3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:28.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.790+0000 7f466ffff640 1 --2- 172.21.15.116:0/1640806797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce1dc99980 0x55ce1dca3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.790+0000 7f466ffff640 1 -- 172.21.15.116:0/1640806797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce1dc99060 msgr2=0x55ce1dc99440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:28.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.790+0000 7f466ffff640 1 --2- 172.21.15.116:0/1640806797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce1dc99060 0x55ce1dc99440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.790+0000 7f466ffff640 1 -- 172.21.15.116:0/1640806797 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce1db96d40 con 0x55ce1dca4240 2024-09-12T03:17:28.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.790+0000 7f466ffff640 1 --2- 172.21.15.116:0/1640806797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce1dca4240 0x55ce1dca6630 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f465c0127a0 tx=0x7f465c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a203ae63374fc4fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:28.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.790+0000 7f4675336640 1 --2- 172.21.15.116:0/2234064872 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f46600a2b80 0x7f46600a5040 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f46680c5190 tx=0x7f466808fba0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:28.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.790+0000 7f465affd640 1 -- 172.21.15.116:0/1640806797 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c012d20 con 0x55ce1dca4240 2024-09-12T03:17:28.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.790+0000 7f465affd640 1 -- 172.21.15.116:0/1640806797 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f465c005020 con 0x55ce1dca4240 2024-09-12T03:17:28.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.790+0000 7f465affd640 1 -- 172.21.15.116:0/1640806797 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c004b20 con 0x55ce1dca4240 2024-09-12T03:17:28.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.791+0000 7f4679172740 1 -- 172.21.15.116:0/1640806797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce1dca4240 msgr2=0x55ce1dca6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.791+0000 7f4679172740 1 --2- 172.21.15.116:0/1640806797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce1dca4240 0x55ce1dca6630 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f465c0127a0 tx=0x7f465c00b7c0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.791+0000 7f4679172740 1 -- 172.21.15.116:0/1640806797 shutdown_connections 2024-09-12T03:17:28.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.791+0000 7f4679172740 1 --2- 172.21.15.116:0/1640806797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce1dca4240 0x55ce1dca6630 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.791+0000 7f4679172740 1 --2- 172.21.15.116:0/1640806797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce1dc99980 0x55ce1dca3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.791+0000 7f4679172740 1 --2- 172.21.15.116:0/1640806797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce1dc99060 0x55ce1dc99440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.791+0000 7f4679172740 1 -- 172.21.15.116:0/1640806797 >> 172.21.15.116:0/1640806797 conn(0x55ce1dc955f0 msgr2=0x55ce1dc959d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:28.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.791+0000 7f4679172740 1 -- 172.21.15.116:0/1640806797 shutdown_connections 2024-09-12T03:17:28.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.791+0000 7f4679172740 1 -- 172.21.15.116:0/1640806797 wait complete. 2024-09-12T03:17:28.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.791+0000 7f4679172740 1 -- start start 2024-09-12T03:17:28.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.791+0000 7f4679172740 1 Processor -- start 2024-09-12T03:17:28.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.792+0000 7f4679172740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce1dc99060 0x55ce1dca1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.792+0000 7f4679172740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce1dc99980 0x55ce1dca1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.792+0000 7f4679172740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce1dca4240 0x55ce1dca1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.792+0000 7f4679172740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ce1dca6e10 con 0x55ce1dc99060 2024-09-12T03:17:28.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.792+0000 7f4679172740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ce1db86f10 con 0x55ce1dc99980 2024-09-12T03:17:28.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.792+0000 7f4679172740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ce1dca6f90 con 0x55ce1dca4240 2024-09-12T03:17:28.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.792+0000 7f4679172740 10 client.0 _ll_register_callbacks cb 0x55ce1dbaadb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:28.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.792+0000 7f466ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce1dc99060 0x55ce1dca1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.792+0000 7f466ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce1dc99060 0x55ce1dca1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52208/0 (socket says 172.21.15.116:52208) 2024-09-12T03:17:28.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.792+0000 7f466ffff640 1 -- 172.21.15.116:0/2949120805 learned_addr learned my addr 172.21.15.116:0/2949120805 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:28.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.792+0000 7f466ffff640 1 --2- 172.21.15.116:0/2949120805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce1dca4240 0x55ce1dca1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.792+0000 7f466ffff640 1 -- 172.21.15.116:0/2949120805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce1dca4240 msgr2=0x55ce1dca1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:28.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.792+0000 7f466ffff640 1 --2- 172.21.15.116:0/2949120805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce1dca4240 0x55ce1dca1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.792+0000 7f4674b35640 1 --2- 172.21.15.116:0/2949120805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce1dc99980 0x55ce1dca1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.792+0000 7f466ffff640 1 -- 172.21.15.116:0/2949120805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce1dc99980 msgr2=0x55ce1dca1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:28.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.792+0000 7f466ffff640 1 --2- 172.21.15.116:0/2949120805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce1dc99980 0x55ce1dca1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.792+0000 7f466ffff640 1 -- 172.21.15.116:0/2949120805 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce1d90d580 con 0x55ce1dc99060 2024-09-12T03:17:28.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.792+0000 7f4674b35640 1 --2- 172.21.15.116:0/2949120805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce1dc99980 0x55ce1dca1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:28.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.792+0000 7f466ffff640 1 --2- 172.21.15.116:0/2949120805 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce1dc99060 0x55ce1dca1120 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f465c015040 tx=0x7f465c005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae80998a3fd857c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:28.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.792+0000 7f464ffff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:28.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.792+0000 7f464ffff640 1 -- 172.21.15.116:0/2949120805 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c0048a0 con 0x55ce1dc99060 2024-09-12T03:17:28.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.792+0000 7f464ffff640 1 -- 172.21.15.116:0/2949120805 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f465c004a40 con 0x55ce1dc99060 2024-09-12T03:17:28.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.793+0000 7f4679172740 1 -- 172.21.15.116:0/2949120805 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f465c005020 con 0x55ce1dc99060 2024-09-12T03:17:28.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.793+0000 7f464ffff640 1 -- 172.21.15.116:0/2949120805 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f465c004c40 con 0x55ce1dc99060 2024-09-12T03:17:28.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.793+0000 7f4679172740 10 client.19278 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:28.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.793+0000 7f4679172740 1 -- 172.21.15.116:0/2949120805 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ce1d9d7960 con 0x55ce1dc99060 2024-09-12T03:17:28.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.793+0000 7f464ffff640 1 -- 172.21.15.116:0/2949120805 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f465c005020 con 0x55ce1dc99060 2024-09-12T03:17:28.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.793+0000 7f464ffff640 1 -- 172.21.15.116:0/2949120805 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f465c033480 con 0x55ce1dc99060 2024-09-12T03:17:28.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.793+0000 7f4679172740 10 client.19278 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:28.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.793+0000 7f4679172740 10 client.19278 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:17:28.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.793+0000 7f4679172740 20 client.19278 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:28.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.793+0000 7f4679172740 1 --2- 172.21.15.116:0/2949120805 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ce1dcb3ca0 0x55ce1dcb6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.793+0000 7f4679172740 4 client.19278 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:17:28.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.793+0000 7f4679172740 1 -- 172.21.15.116:0/2949120805 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce1dc0bb90 con 0x55ce1dcb3ca0 2024-09-12T03:17:28.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.793+0000 7f4674b35640 1 --2- 172.21.15.116:0/2949120805 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ce1dcb3ca0 0x55ce1dcb6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.794+0000 7f4674b35640 1 --2- 172.21.15.116:0/2949120805 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ce1dcb3ca0 0x55ce1dcb6090 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe77d4ec609c6905 server_cookie=5782c00c3413497a in_seq=0 out_seq=0 2024-09-12T03:17:28.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.794+0000 7f464ffff640 10 client.19278 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:28.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.794+0000 7f464ffff640 1 -- 172.21.15.116:0/2949120805 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ce1dc0bb90 con 0x55ce1dcb3ca0 2024-09-12T03:17:28.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.794+0000 7f464ffff640 10 client.19278 handle_command_reply: tid=0 2024-09-12T03:17:28.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.794+0000 7f4679172740 1 client.19278 shutdown 2024-09-12T03:17:28.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.795+0000 7f4679172740 1 -- 172.21.15.116:0/2949120805 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce1dc99060 msgr2=0x55ce1dca1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.795+0000 7f4679172740 1 --2- 172.21.15.116:0/2949120805 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce1dc99060 0x55ce1dca1120 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f465c015040 tx=0x7f465c005c50 comp rx=0 tx=0).stop 2024-09-12T03:17:28.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.795+0000 7f4679172740 1 -- 172.21.15.116:0/2949120805 shutdown_connections 2024-09-12T03:17:28.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.795+0000 7f4679172740 1 --2- 172.21.15.116:0/2949120805 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ce1dcb3ca0 0x55ce1dcb6090 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.795+0000 7f4679172740 1 --2- 172.21.15.116:0/2949120805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce1dca4240 0x55ce1dca1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.795+0000 7f4679172740 1 --2- 172.21.15.116:0/2949120805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce1dc99980 0x55ce1dca1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.795+0000 7f4679172740 1 --2- 172.21.15.116:0/2949120805 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce1dc99060 0x55ce1dca1120 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.795+0000 7f464ffff640 0 client.19278 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:28.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.795+0000 7f4679172740 1 -- 172.21.15.116:0/2949120805 >> 172.21.15.116:0/2949120805 conn(0x55ce1dc955f0 msgr2=0x55ce1dc9b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:28.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.795+0000 7f4679172740 1 -- 172.21.15.116:0/2949120805 shutdown_connections 2024-09-12T03:17:28.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.795+0000 7f4679172740 1 -- 172.21.15.116:0/2949120805 wait complete. 2024-09-12T03:17:28.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.795+0000 7f4679172740 20 client.19278 trim_cache size 0 max 16384 2024-09-12T03:17:28.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.812+0000 7f4679172740 1 Processor -- start 2024-09-12T03:17:28.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.812+0000 7f4679172740 1 -- start start 2024-09-12T03:17:28.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.813+0000 7f4679172740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce1dc99060 0x55ce1dc9c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.813+0000 7f4679172740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce1dc99980 0x55ce1dc94640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.813+0000 7f4679172740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce1dca4240 0x55ce1dcbb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.813+0000 7f4679172740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f46701afd00 con 0x55ce1dc99060 2024-09-12T03:17:28.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.813+0000 7f4679172740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f46701ad5d0 con 0x55ce1dc99980 2024-09-12T03:17:28.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.813+0000 7f4679172740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f46701ab420 con 0x55ce1dca4240 2024-09-12T03:17:28.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.813+0000 7f466ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce1dc99060 0x55ce1dc9c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.813+0000 7f466ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce1dc99060 0x55ce1dc9c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52224/0 (socket says 172.21.15.116:52224) 2024-09-12T03:17:28.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.813+0000 7f466ffff640 1 -- 172.21.15.116:0/3217032593 learned_addr learned my addr 172.21.15.116:0/3217032593 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:28.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.813+0000 7f4674b35640 1 --2- 172.21.15.116:0/3217032593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce1dc99980 0x55ce1dc94640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.813+0000 7f466ffff640 1 -- 172.21.15.116:0/3217032593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce1dca4240 msgr2=0x55ce1dcbb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:28.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.813+0000 7f466ffff640 1 --2- 172.21.15.116:0/3217032593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce1dca4240 0x55ce1dcbb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.813+0000 7f466ffff640 1 -- 172.21.15.116:0/3217032593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce1dc99980 msgr2=0x55ce1dc94640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:28.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.813+0000 7f466ffff640 1 --2- 172.21.15.116:0/3217032593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce1dc99980 0x55ce1dc94640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.813+0000 7f466ffff640 1 -- 172.21.15.116:0/3217032593 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce1dcb6650 con 0x55ce1dc99060 2024-09-12T03:17:28.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.813+0000 7f4674b35640 1 --2- 172.21.15.116:0/3217032593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce1dc99980 0x55ce1dc94640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:28.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.813+0000 7f466ffff640 1 --2- 172.21.15.116:0/3217032593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce1dc99060 0x55ce1dc9c670 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f465c00b620 tx=0x7f465c00bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cca2cb6e7534324 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:28.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.813+0000 7f465a7fc640 1 -- 172.21.15.116:0/3217032593 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c00b7c0 con 0x55ce1dc99060 2024-09-12T03:17:28.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.813+0000 7f465a7fc640 1 -- 172.21.15.116:0/3217032593 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f465c016040 con 0x55ce1dc99060 2024-09-12T03:17:28.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.813+0000 7f465a7fc640 1 -- 172.21.15.116:0/3217032593 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c004de0 con 0x55ce1dc99060 2024-09-12T03:17:28.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.814+0000 7f4679172740 1 -- 172.21.15.116:0/3217032593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce1dc99060 msgr2=0x55ce1dc9c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.814+0000 7f4679172740 1 --2- 172.21.15.116:0/3217032593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce1dc99060 0x55ce1dc9c670 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f465c00b620 tx=0x7f465c00bf20 comp rx=0 tx=0).stop 2024-09-12T03:17:28.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.814+0000 7f4679172740 1 -- 172.21.15.116:0/3217032593 shutdown_connections 2024-09-12T03:17:28.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.814+0000 7f4679172740 1 --2- 172.21.15.116:0/3217032593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce1dca4240 0x55ce1dcbb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.814+0000 7f4679172740 1 --2- 172.21.15.116:0/3217032593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce1dc99980 0x55ce1dc94640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.814+0000 7f4679172740 1 --2- 172.21.15.116:0/3217032593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce1dc99060 0x55ce1dc9c670 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.814+0000 7f4679172740 1 -- 172.21.15.116:0/3217032593 >> 172.21.15.116:0/3217032593 conn(0x55ce1dc955f0 msgr2=0x55ce1dc93f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:28.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.814+0000 7f4679172740 1 -- 172.21.15.116:0/3217032593 shutdown_connections 2024-09-12T03:17:28.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.814+0000 7f4679172740 1 -- 172.21.15.116:0/3217032593 wait complete. 2024-09-12T03:17:28.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.815+0000 7f4679172740 1 -- start start 2024-09-12T03:17:28.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.815+0000 7f4679172740 1 Processor -- start 2024-09-12T03:17:28.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.815+0000 7f4679172740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce1dc99060 0x55ce1dcac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.815+0000 7f4679172740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce1dc99980 0x55ce1dcac900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.815+0000 7f4679172740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce1dca4240 0x55ce1dca91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.815+0000 7f4679172740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f46701aba90 con 0x55ce1dca4240 2024-09-12T03:17:28.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.815+0000 7f4679172740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f46701acf00 con 0x55ce1dc99060 2024-09-12T03:17:28.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.815+0000 7f4679172740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f46701ac160 con 0x55ce1dc99980 2024-09-12T03:17:28.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.815+0000 7f4679172740 10 client.0 _ll_register_callbacks cb 0x7f465c012670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:28.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.816+0000 7f466ffff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce1dc99060 0x55ce1dcac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.816+0000 7f466ffff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce1dc99060 0x55ce1dcac3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.116:37648/0 (socket says 172.21.15.116:37648) 2024-09-12T03:17:28.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.816+0000 7f466ffff640 1 -- 172.21.15.116:0/3557201318 learned_addr learned my addr 172.21.15.116:0/3557201318 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:28.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.816+0000 7f466ffff640 1 --2- 172.21.15.116:0/3557201318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce1dca4240 0x55ce1dca91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.816+0000 7f4674b35640 1 --2- 172.21.15.116:0/3557201318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce1dc99980 0x55ce1dcac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.816+0000 7f466ffff640 1 -- 172.21.15.116:0/3557201318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce1dc99980 msgr2=0x55ce1dcac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:28.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.816+0000 7f466ffff640 1 --2- 172.21.15.116:0/3557201318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce1dc99980 0x55ce1dcac900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.816+0000 7f466ffff640 1 -- 172.21.15.116:0/3557201318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce1dca4240 msgr2=0x55ce1dca91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:28.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.816+0000 7f466ffff640 1 --2- 172.21.15.116:0/3557201318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce1dca4240 0x55ce1dca91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.816+0000 7f466ffff640 1 -- 172.21.15.116:0/3557201318 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce1dca9850 con 0x55ce1dc99060 2024-09-12T03:17:28.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.816+0000 7f466ffff640 1 --2- 172.21.15.116:0/3557201318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce1dc99060 0x55ce1dcac3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f465c005880 tx=0x7f465c012210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9c5630454bd0a9c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:28.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.816+0000 7f464ffff640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-09-12T03:17:28.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.816+0000 7f464ffff640 1 -- 172.21.15.116:0/3557201318 <== mon.2 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c033c40 con 0x55ce1dc99060 2024-09-12T03:17:28.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.817+0000 7f464ffff640 1 -- 172.21.15.116:0/3557201318 <== mon.2 v2:172.21.15.177:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f465c010740 con 0x55ce1dc99060 2024-09-12T03:17:28.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.817+0000 7f4679172740 1 -- 172.21.15.116:0/3557201318 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f465c016040 con 0x55ce1dc99060 2024-09-12T03:17:28.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.817+0000 7f464ffff640 1 -- 172.21.15.116:0/3557201318 <== mon.2 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f465c033640 con 0x55ce1dc99060 2024-09-12T03:17:28.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.817+0000 7f4679172740 10 client.27299 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:28.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.817+0000 7f4679172740 1 -- 172.21.15.116:0/3557201318 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ce1dd1f1f0 con 0x55ce1dc99060 2024-09-12T03:17:28.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.817+0000 7f464ffff640 1 -- 172.21.15.116:0/3557201318 <== mon.2 v2:172.21.15.177:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f465c016040 con 0x55ce1dc99060 2024-09-12T03:17:28.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.817+0000 7f464ffff640 1 -- 172.21.15.116:0/3557201318 <== mon.2 v2:172.21.15.177:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f465c03d500 con 0x55ce1dc99060 2024-09-12T03:17:28.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.818+0000 7f4679172740 10 client.27299 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:28.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.818+0000 7f4679172740 10 client.27299 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:17:28.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.818+0000 7f4679172740 20 client.27299 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:28.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.818+0000 7f4679172740 1 --2- 172.21.15.116:0/3557201318 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ce1dd1f850 0x55ce1dd21c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.818+0000 7f4679172740 4 client.27299 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1cfce56f", "release": true, "await": true}] 2024-09-12T03:17:28.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.818+0000 7f4679172740 1 -- 172.21.15.116:0/3557201318 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1cfce56f", "release": true, "await": true}) -- 0x55ce1dc060a0 con 0x55ce1dd1f850 2024-09-12T03:17:28.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.818+0000 7f4674b35640 1 --2- 172.21.15.116:0/3557201318 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ce1dd1f850 0x55ce1dd21c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.818+0000 7f4674b35640 1 --2- 172.21.15.116:0/3557201318 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ce1dd1f850 0x55ce1dd21c40 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dec92d39584fb92 server_cookie=35bad706240b778 in_seq=0 out_seq=0 2024-09-12T03:17:28.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.818+0000 7f464ffff640 10 client.27299 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:28.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.821+0000 7f464ffff640 1 -- 172.21.15.116:0/3557201318 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ce1dc060a0 con 0x55ce1dd1f850 2024-09-12T03:17:28.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.821+0000 7f464ffff640 10 client.27299 handle_command_reply: tid=0 2024-09-12T03:17:28.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.821+0000 7f4679172740 1 client.27299 shutdown 2024-09-12T03:17:28.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.821+0000 7f4679172740 1 -- 172.21.15.116:0/3557201318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce1dc99060 msgr2=0x55ce1dcac3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.821+0000 7f4679172740 1 --2- 172.21.15.116:0/3557201318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce1dc99060 0x55ce1dcac3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f465c005880 tx=0x7f465c012210 comp rx=0 tx=0).stop 2024-09-12T03:17:28.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.821+0000 7f4679172740 1 -- 172.21.15.116:0/3557201318 shutdown_connections 2024-09-12T03:17:28.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.821+0000 7f4679172740 1 --2- 172.21.15.116:0/3557201318 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ce1dd1f850 0x55ce1dd21c40 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.821+0000 7f4679172740 1 --2- 172.21.15.116:0/3557201318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce1dca4240 0x55ce1dca91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.821+0000 7f4679172740 1 --2- 172.21.15.116:0/3557201318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce1dc99980 0x55ce1dcac900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.821+0000 7f464ffff640 0 client.27299 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:28.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.821+0000 7f4679172740 1 --2- 172.21.15.116:0/3557201318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce1dc99060 0x55ce1dcac3c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.821+0000 7f4679172740 1 -- 172.21.15.116:0/3557201318 >> 172.21.15.116:0/3557201318 conn(0x55ce1dc955f0 msgr2=0x55ce1dc9b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:28.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.821+0000 7f4679172740 1 -- 172.21.15.116:0/3557201318 shutdown_connections 2024-09-12T03:17:28.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.821+0000 7f4679172740 1 -- 172.21.15.116:0/3557201318 wait complete. 2024-09-12T03:17:28.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.821+0000 7f4679172740 20 client.27299 trim_cache size 0 max 16384 2024-09-12T03:17:28.834 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:17:28.834 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 25, 2024-09-12T03:17:28.835 INFO:teuthology.orchestra.run.smithi116.stdout: "leader": 24494, 2024-09-12T03:17:28.835 INFO:teuthology.orchestra.run.smithi116.stdout: "set_version": 56, 2024-09-12T03:17:28.835 INFO:teuthology.orchestra.run.smithi116.stdout: "sets": { 2024-09-12T03:17:28.835 INFO:teuthology.orchestra.run.smithi116.stdout: "1cfce56f": { 2024-09-12T03:17:28.835 INFO:teuthology.orchestra.run.smithi116.stdout: "version": 56, 2024-09-12T03:17:28.835 INFO:teuthology.orchestra.run.smithi116.stdout: "age_ref": 0.0, 2024-09-12T03:17:28.835 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:17:28.835 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RELEASED", 2024-09-12T03:17:28.835 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:17:28.835 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:17:28.835 INFO:teuthology.orchestra.run.smithi116.stdout: "timeout": 90.0, 2024-09-12T03:17:28.835 INFO:teuthology.orchestra.run.smithi116.stdout: "expiration": 150.0, 2024-09-12T03:17:28.835 INFO:teuthology.orchestra.run.smithi116.stdout: "members": { 2024-09-12T03:17:28.836 INFO:teuthology.orchestra.run.smithi116.stdout: "file:/": { 2024-09-12T03:17:28.836 INFO:teuthology.orchestra.run.smithi116.stdout: "excluded": false, 2024-09-12T03:17:28.836 INFO:teuthology.orchestra.run.smithi116.stdout: "state": { 2024-09-12T03:17:28.836 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RELEASED", 2024-09-12T03:17:28.836 INFO:teuthology.orchestra.run.smithi116.stdout: "age": 0.0 2024-09-12T03:17:28.836 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:17:28.836 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:17:28.836 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:17:28.836 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:17:28.836 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:17:28.836 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:17:28.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.822+0000 7f4676d99640 1 -- 172.21.15.116:0/2234064872 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f46600a2b80 msgr2=0x7f46600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.822+0000 7f4676d99640 1 --2- 172.21.15.116:0/2234064872 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f46600a2b80 0x7f46600a5040 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f46680c5190 tx=0x7f466808fba0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.822+0000 7f4676d99640 1 -- 172.21.15.116:0/2234064872 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4670106f10 msgr2=0x7f467019cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.822+0000 7f4676d99640 1 --2- 172.21.15.116:0/2234064872 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4670106f10 0x7f467019cd00 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f4668031a20 tx=0x7f4668031a50 comp rx=0 tx=0).stop 2024-09-12T03:17:28.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.822+0000 7f4676d99640 1 -- 172.21.15.116:0/2234064872 shutdown_connections 2024-09-12T03:17:28.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.822+0000 7f4676d99640 1 --2- 172.21.15.116:0/2234064872 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f46600a2b80 0x7f46600a5040 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.822+0000 7f4676d99640 1 --2- 172.21.15.116:0/2234064872 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4670106f10 0x7f467019cd00 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.822+0000 7f4676d99640 1 --2- 172.21.15.116:0/2234064872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4670102790 0x7f4670198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.822+0000 7f4676d99640 1 --2- 172.21.15.116:0/2234064872 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4670101de0 0x7f4670198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.822+0000 7f4676d99640 1 -- 172.21.15.116:0/2234064872 >> 172.21.15.116:0/2234064872 conn(0x7f46700fd8c0 msgr2=0x7f46700fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:28.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.822+0000 7f4676d99640 1 -- 172.21.15.116:0/2234064872 shutdown_connections 2024-09-12T03:17:28.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:28.822+0000 7f4676d99640 1 -- 172.21.15.116:0/2234064872 wait complete. 2024-09-12T03:17:28.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:28 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:17:28.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:28 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:17:28.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:28 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:17:28.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:28 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:17:29.046 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:29.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:29 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:17:29.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:29 smithi177 ceph-mon[35305]: pgmap v773: 97 pgs: 97 active+clean; 6.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:17:29.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:29 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:17:29.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:29 smithi116 ceph-mon[31410]: pgmap v773: 97 pgs: 97 active+clean; 6.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:17:29.907 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1cfce56f', total seconds elapsed: 35.9 2024-09-12T03:17:30.067 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:29 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:17:30.067 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:29 smithi183 ceph-mon[35535]: pgmap v773: 97 pgs: 97 active+clean; 6.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-12T03:17:30.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.509+0000 7f71740b0640 1 -- 172.21.15.116:0/3020305619 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f716c106a80 msgr2=0x7f716c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:30.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.509+0000 7f71740b0640 1 --2- 172.21.15.116:0/3020305619 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f716c106a80 0x7f716c106e60 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f7168009930 tx=0x7f716802a420 comp rx=0 tx=0).stop 2024-09-12T03:17:30.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.509+0000 7f71740b0640 1 -- 172.21.15.116:0/3020305619 shutdown_connections 2024-09-12T03:17:30.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.509+0000 7f71740b0640 1 --2- 172.21.15.116:0/3020305619 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f716c101420 0x7f716c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.509+0000 7f71740b0640 1 --2- 172.21.15.116:0/3020305619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f716c100a80 0x7f716c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.509+0000 7f71740b0640 1 --2- 172.21.15.116:0/3020305619 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f716c106a80 0x7f716c106e60 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.509+0000 7f71740b0640 1 -- 172.21.15.116:0/3020305619 >> 172.21.15.116:0/3020305619 conn(0x7f716c0fc7a0 msgr2=0x7f716c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:30.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.509+0000 7f71740b0640 1 -- 172.21.15.116:0/3020305619 shutdown_connections 2024-09-12T03:17:30.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.509+0000 7f71740b0640 1 -- 172.21.15.116:0/3020305619 wait complete. 2024-09-12T03:17:30.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.510+0000 7f71740b0640 1 Processor -- start 2024-09-12T03:17:30.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.510+0000 7f71740b0640 1 -- start start 2024-09-12T03:17:30.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.510+0000 7f71740b0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f716c100a80 0x7f716c1973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:30.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.510+0000 7f71740b0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f716c101420 0x7f716c1978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:30.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.510+0000 7f71740b0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f716c106a80 0x7f716c19bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:30.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.510+0000 7f71740b0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f716c110e80 con 0x7f716c100a80 2024-09-12T03:17:30.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.510+0000 7f71740b0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f716c110d00 con 0x7f716c106a80 2024-09-12T03:17:30.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.510+0000 7f71740b0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f716c111000 con 0x7f716c101420 2024-09-12T03:17:30.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.510+0000 7f7171e4c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f716c100a80 0x7f716c1973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:30.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.510+0000 7f7171e4c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f716c100a80 0x7f716c1973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52262/0 (socket says 172.21.15.116:52262) 2024-09-12T03:17:30.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.510+0000 7f7171e4c640 1 -- 172.21.15.116:0/177654924 learned_addr learned my addr 172.21.15.116:0/177654924 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:30.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.510+0000 7f717164b640 1 --2- 172.21.15.116:0/177654924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f716c101420 0x7f716c1978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:30.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.510+0000 7f7171e4c640 1 -- 172.21.15.116:0/177654924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f716c101420 msgr2=0x7f716c1978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:30.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.510+0000 7f7171e4c640 1 --2- 172.21.15.116:0/177654924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f716c101420 0x7f716c1978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.510+0000 7f7171e4c640 1 -- 172.21.15.116:0/177654924 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f716c106a80 msgr2=0x7f716c19bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:30.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.510+0000 7f717264d640 1 --2- 172.21.15.116:0/177654924 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f716c106a80 0x7f716c19bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:30.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.510+0000 7f7171e4c640 1 --2- 172.21.15.116:0/177654924 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f716c106a80 0x7f716c19bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.510+0000 7f7171e4c640 1 -- 172.21.15.116:0/177654924 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f716c19c350 con 0x7f716c100a80 2024-09-12T03:17:30.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.510+0000 7f717164b640 1 --2- 172.21.15.116:0/177654924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f716c101420 0x7f716c1978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:30.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.511+0000 7f7171e4c640 1 --2- 172.21.15.116:0/177654924 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f716c100a80 0x7f716c1973a0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f7168009230 tx=0x7f7168004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:30.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.511+0000 7f7162ffd640 1 -- 172.21.15.116:0/177654924 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716803f030 con 0x7f716c100a80 2024-09-12T03:17:30.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.511+0000 7f71740b0640 1 -- 172.21.15.116:0/177654924 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f716c19c5e0 con 0x7f716c100a80 2024-09-12T03:17:30.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.511+0000 7f7162ffd640 1 -- 172.21.15.116:0/177654924 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7168003ad0 con 0x7f716c100a80 2024-09-12T03:17:30.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.511+0000 7f7162ffd640 1 -- 172.21.15.116:0/177654924 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7168035070 con 0x7f716c100a80 2024-09-12T03:17:30.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.511+0000 7f71740b0640 1 -- 172.21.15.116:0/177654924 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f716c1a3f20 con 0x7f716c100a80 2024-09-12T03:17:30.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.513+0000 7f7162ffd640 1 -- 172.21.15.116:0/177654924 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f7168042030 con 0x7f716c100a80 2024-09-12T03:17:30.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.513+0000 7f7176489740 1 Processor -- start 2024-09-12T03:17:30.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.514+0000 7f7176489740 1 -- start start 2024-09-12T03:17:30.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.514+0000 7f7162ffd640 1 --2- 172.21.15.116:0/177654924 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f71580a2b80 0x7f71580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:30.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.514+0000 7f7176489740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d4e387570 0x562d4e387950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:30.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.514+0000 7f7176489740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d4e387e90 0x562d4e394740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:30.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.514+0000 7f7176489740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d4e394c80 0x562d4e397070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:30.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.514+0000 7f7176489740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562d4e2eb2e0 con 0x562d4e387570 2024-09-12T03:17:30.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.514+0000 7f7176489740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562d4e292ce0 con 0x562d4e394c80 2024-09-12T03:17:30.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.514+0000 7f7176489740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562d4e2e72a0 con 0x562d4e387e90 2024-09-12T03:17:30.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.514+0000 7f7162ffd640 1 -- 172.21.15.116:0/177654924 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f71680ef5d0 con 0x7f716c100a80 2024-09-12T03:17:30.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.514+0000 7f7171e4c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d4e387570 0x562d4e387950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:30.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.514+0000 7f7171e4c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d4e387570 0x562d4e387950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52266/0 (socket says 172.21.15.116:52266) 2024-09-12T03:17:30.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.514+0000 7f7171e4c640 1 -- 172.21.15.116:0/704447122 learned_addr learned my addr 172.21.15.116:0/704447122 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:30.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.514+0000 7f717164b640 1 --2- 172.21.15.116:0/704447122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d4e387e90 0x562d4e394740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:30.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.514+0000 7f7171e4c640 1 -- 172.21.15.116:0/704447122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d4e387e90 msgr2=0x562d4e394740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:30.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.514+0000 7f7171e4c640 1 --2- 172.21.15.116:0/704447122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d4e387e90 0x562d4e394740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.514+0000 7f7171e4c640 1 -- 172.21.15.116:0/704447122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d4e394c80 msgr2=0x562d4e397070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:17:30.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.514+0000 7f7171e4c640 1 --2- 172.21.15.116:0/704447122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d4e394c80 0x562d4e397070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.514+0000 7f7171e4c640 1 -- 172.21.15.116:0/704447122 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d4e277440 con 0x562d4e387570 2024-09-12T03:17:30.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.514+0000 7f717264d640 1 --2- 172.21.15.116:0/177654924 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f71580a2b80 0x7f71580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:30.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.514+0000 7f7171e4c640 1 --2- 172.21.15.116:0/704447122 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d4e387570 0x562d4e387950 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f7168000c00 tx=0x7f716802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e685512bdb1677aa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:30.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.515+0000 7f7157fff640 1 -- 172.21.15.116:0/704447122 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7168049040 con 0x562d4e387570 2024-09-12T03:17:30.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.515+0000 7f7157fff640 1 -- 172.21.15.116:0/704447122 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71680c4ad0 con 0x562d4e387570 2024-09-12T03:17:30.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.515+0000 7f7157fff640 1 -- 172.21.15.116:0/704447122 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7168080740 con 0x562d4e387570 2024-09-12T03:17:30.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.515+0000 7f7176489740 1 -- 172.21.15.116:0/704447122 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d4e387570 msgr2=0x562d4e387950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:30.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.515+0000 7f717264d640 1 --2- 172.21.15.116:0/177654924 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f71580a2b80 0x7f71580a5040 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7164005c40 tx=0x7f7164005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:30.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.515+0000 7f7176489740 1 --2- 172.21.15.116:0/704447122 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d4e387570 0x562d4e387950 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f7168000c00 tx=0x7f716802a420 comp rx=0 tx=0).stop 2024-09-12T03:17:30.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.515+0000 7f7176489740 1 -- 172.21.15.116:0/704447122 shutdown_connections 2024-09-12T03:17:30.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.515+0000 7f7176489740 1 --2- 172.21.15.116:0/704447122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d4e394c80 0x562d4e397070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.515+0000 7f7176489740 1 --2- 172.21.15.116:0/704447122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d4e387e90 0x562d4e394740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.515+0000 7f7176489740 1 --2- 172.21.15.116:0/704447122 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d4e387570 0x562d4e387950 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.515+0000 7f7176489740 1 -- 172.21.15.116:0/704447122 >> 172.21.15.116:0/704447122 conn(0x562d4e383c40 msgr2=0x562d4e384020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:30.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.516+0000 7f7176489740 1 -- 172.21.15.116:0/704447122 shutdown_connections 2024-09-12T03:17:30.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.516+0000 7f7176489740 1 -- 172.21.15.116:0/704447122 wait complete. 2024-09-12T03:17:30.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.516+0000 7f7176489740 1 -- start start 2024-09-12T03:17:30.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.516+0000 7f7176489740 1 Processor -- start 2024-09-12T03:17:30.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.516+0000 7f7176489740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d4e387570 0x562d4e394680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:30.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.516+0000 7f7176489740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d4e387e90 0x562d4e391980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:30.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.516+0000 7f7176489740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d4e394c80 0x562d4e391f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:30.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.516+0000 7f7176489740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562d4e2eb2e0 con 0x562d4e394c80 2024-09-12T03:17:30.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.516+0000 7f7176489740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562d4e2a6050 con 0x562d4e387e90 2024-09-12T03:17:30.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.516+0000 7f7176489740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562d4e3975b0 con 0x562d4e387570 2024-09-12T03:17:30.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.516+0000 7f7176489740 10 client.0 _ll_register_callbacks cb 0x562d4e2e6160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:30.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.516+0000 7f717264d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d4e394c80 0x562d4e391f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:30.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.516+0000 7f717264d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d4e394c80 0x562d4e391f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52268/0 (socket says 172.21.15.116:52268) 2024-09-12T03:17:30.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.516+0000 7f717264d640 1 -- 172.21.15.116:0/3790962971 learned_addr learned my addr 172.21.15.116:0/3790962971 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:30.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.517+0000 7f717164b640 1 --2- 172.21.15.116:0/3790962971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d4e387e90 0x562d4e391980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:30.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.517+0000 7f717264d640 1 -- 172.21.15.116:0/3790962971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d4e387570 msgr2=0x562d4e394680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:30.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.517+0000 7f717264d640 1 --2- 172.21.15.116:0/3790962971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d4e387570 0x562d4e394680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.517+0000 7f717264d640 1 -- 172.21.15.116:0/3790962971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d4e387e90 msgr2=0x562d4e391980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:30.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.517+0000 7f717264d640 1 --2- 172.21.15.116:0/3790962971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d4e387e90 0x562d4e391980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.517+0000 7f717264d640 1 -- 172.21.15.116:0/3790962971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d4e05c3c0 con 0x562d4e394c80 2024-09-12T03:17:30.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.517+0000 7f717164b640 1 --2- 172.21.15.116:0/3790962971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d4e387e90 0x562d4e391980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:30.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.517+0000 7f717264d640 1 --2- 172.21.15.116:0/3790962971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d4e394c80 0x562d4e391f70 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f7164011440 tx=0x7f7164004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9e4a1971af447c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:30.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.517+0000 7f71557fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:30.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.517+0000 7f71557fa640 1 -- 172.21.15.116:0/3790962971 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7164022070 con 0x562d4e394c80 2024-09-12T03:17:30.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.517+0000 7f7176489740 1 -- 172.21.15.116:0/3790962971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71680c4ad0 con 0x562d4e394c80 2024-09-12T03:17:30.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.517+0000 7f71557fa640 1 -- 172.21.15.116:0/3790962971 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f716401e860 con 0x562d4e394c80 2024-09-12T03:17:30.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.517+0000 7f71557fa640 1 -- 172.21.15.116:0/3790962971 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f716402e050 con 0x562d4e394c80 2024-09-12T03:17:30.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.517+0000 7f7176489740 10 client.19296 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:30.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.517+0000 7f7176489740 1 -- 172.21.15.116:0/3790962971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562d4e3a2e70 con 0x562d4e394c80 2024-09-12T03:17:30.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.517+0000 7f71557fa640 1 -- 172.21.15.116:0/3790962971 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71680c4ad0 con 0x562d4e394c80 2024-09-12T03:17:30.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.518+0000 7f71557fa640 1 -- 172.21.15.116:0/3790962971 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7164010c70 con 0x562d4e394c80 2024-09-12T03:17:30.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.518+0000 7f7176489740 10 client.19296 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:30.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.518+0000 7f7176489740 10 client.19296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:17:30.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.518+0000 7f7176489740 20 client.19296 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:30.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.518+0000 7f7176489740 1 --2- 172.21.15.116:0/3790962971 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562d4e3a4390 0x562d4e3a6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:30.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.518+0000 7f7176489740 4 client.19296 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:17:30.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.518+0000 7f7176489740 1 -- 172.21.15.116:0/3790962971 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d4e2f8850 con 0x562d4e3a4390 2024-09-12T03:17:30.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.518+0000 7f7171e4c640 1 --2- 172.21.15.116:0/3790962971 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562d4e3a4390 0x562d4e3a6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:30.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.518+0000 7f7171e4c640 1 --2- 172.21.15.116:0/3790962971 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562d4e3a4390 0x562d4e3a6850 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11c48552b3081539 server_cookie=df32ae04f3b651e1 in_seq=0 out_seq=0 2024-09-12T03:17:30.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.518+0000 7f71557fa640 10 client.19296 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:30.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.519+0000 7f71557fa640 1 -- 172.21.15.116:0/3790962971 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562d4e2f8850 con 0x562d4e3a4390 2024-09-12T03:17:30.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.519+0000 7f71557fa640 10 client.19296 handle_command_reply: tid=0 2024-09-12T03:17:30.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.519+0000 7f7176489740 1 client.19296 shutdown 2024-09-12T03:17:30.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.519+0000 7f7176489740 1 -- 172.21.15.116:0/3790962971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d4e394c80 msgr2=0x562d4e391f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:30.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.519+0000 7f7176489740 1 --2- 172.21.15.116:0/3790962971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d4e394c80 0x562d4e391f70 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f7164011440 tx=0x7f7164004a40 comp rx=0 tx=0).stop 2024-09-12T03:17:30.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.520+0000 7f7176489740 1 -- 172.21.15.116:0/3790962971 shutdown_connections 2024-09-12T03:17:30.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.520+0000 7f7176489740 1 --2- 172.21.15.116:0/3790962971 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562d4e3a4390 0x562d4e3a6850 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.520+0000 7f7176489740 1 --2- 172.21.15.116:0/3790962971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d4e394c80 0x562d4e391f70 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.520+0000 7f7176489740 1 --2- 172.21.15.116:0/3790962971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d4e387e90 0x562d4e391980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.520+0000 7f7176489740 1 --2- 172.21.15.116:0/3790962971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d4e387570 0x562d4e394680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.520+0000 7f71557fa640 0 client.19296 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:30.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.520+0000 7f7176489740 1 -- 172.21.15.116:0/3790962971 >> 172.21.15.116:0/3790962971 conn(0x562d4e383c40 msgr2=0x562d4e3899a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:30.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.520+0000 7f7176489740 1 -- 172.21.15.116:0/3790962971 shutdown_connections 2024-09-12T03:17:30.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.520+0000 7f7176489740 1 -- 172.21.15.116:0/3790962971 wait complete. 2024-09-12T03:17:30.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.520+0000 7f7176489740 20 client.19296 trim_cache size 0 max 16384 2024-09-12T03:17:30.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.537+0000 7f7176489740 1 Processor -- start 2024-09-12T03:17:30.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.537+0000 7f7176489740 1 -- start start 2024-09-12T03:17:30.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.537+0000 7f7176489740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d4e387570 0x562d4e3a0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:30.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.537+0000 7f7176489740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d4e387e90 0x562d4e380220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:30.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.537+0000 7f7176489740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d4e394c80 0x562d4e380760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:30.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.537+0000 7f7176489740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f716c1adc50 con 0x562d4e394c80 2024-09-12T03:17:30.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.537+0000 7f7176489740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f716c1ab520 con 0x562d4e387570 2024-09-12T03:17:30.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.537+0000 7f7176489740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f716c1a9370 con 0x562d4e387e90 2024-09-12T03:17:30.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.538+0000 7f717164b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d4e387e90 0x562d4e380220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:30.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.538+0000 7f717264d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d4e394c80 0x562d4e380760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:30.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.538+0000 7f717164b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d4e387e90 0x562d4e380220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.116:41764/0 (socket says 172.21.15.116:41764) 2024-09-12T03:17:30.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.538+0000 7f717164b640 1 -- 172.21.15.116:0/3860779551 learned_addr learned my addr 172.21.15.116:0/3860779551 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:30.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.538+0000 7f7171e4c640 1 --2- 172.21.15.116:0/3860779551 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d4e387570 0x562d4e3a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:30.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.538+0000 7f717164b640 1 -- 172.21.15.116:0/3860779551 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d4e387570 msgr2=0x562d4e3a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:30.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.538+0000 7f717164b640 1 --2- 172.21.15.116:0/3860779551 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d4e387570 0x562d4e3a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.538+0000 7f717164b640 1 -- 172.21.15.116:0/3860779551 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d4e394c80 msgr2=0x562d4e380760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:30.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.538+0000 7f717164b640 1 --2- 172.21.15.116:0/3860779551 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d4e394c80 0x562d4e380760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.538+0000 7f717164b640 1 -- 172.21.15.116:0/3860779551 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d4e3848d0 con 0x562d4e387e90 2024-09-12T03:17:30.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.538+0000 7f717264d640 1 --2- 172.21.15.116:0/3860779551 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d4e394c80 0x562d4e380760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:17:30.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.538+0000 7f717164b640 1 --2- 172.21.15.116:0/3860779551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d4e387e90 0x562d4e380220 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f715c009d40 tx=0x7f715c0127c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=166ffa06c95033b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:30.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.538+0000 7f71577fe640 1 -- 172.21.15.116:0/3860779551 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c01b640 con 0x562d4e387e90 2024-09-12T03:17:30.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.538+0000 7f71577fe640 1 -- 172.21.15.116:0/3860779551 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f715c009e30 con 0x562d4e387e90 2024-09-12T03:17:30.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.539+0000 7f71577fe640 1 -- 172.21.15.116:0/3860779551 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c010960 con 0x562d4e387e90 2024-09-12T03:17:30.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.539+0000 7f7176489740 1 -- 172.21.15.116:0/3860779551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d4e387e90 msgr2=0x562d4e380220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:30.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.539+0000 7f7176489740 1 --2- 172.21.15.116:0/3860779551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d4e387e90 0x562d4e380220 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f715c009d40 tx=0x7f715c0127c0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.539+0000 7f7176489740 1 -- 172.21.15.116:0/3860779551 shutdown_connections 2024-09-12T03:17:30.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.539+0000 7f7176489740 1 --2- 172.21.15.116:0/3860779551 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d4e394c80 0x562d4e380760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.539+0000 7f7176489740 1 --2- 172.21.15.116:0/3860779551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d4e387e90 0x562d4e380220 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.539+0000 7f7176489740 1 --2- 172.21.15.116:0/3860779551 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d4e387570 0x562d4e3a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.539+0000 7f7176489740 1 -- 172.21.15.116:0/3860779551 >> 172.21.15.116:0/3860779551 conn(0x562d4e383c40 msgr2=0x562d4e3828a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:30.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.539+0000 7f7176489740 1 -- 172.21.15.116:0/3860779551 shutdown_connections 2024-09-12T03:17:30.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.539+0000 7f7176489740 1 -- 172.21.15.116:0/3860779551 wait complete. 2024-09-12T03:17:30.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.540+0000 7f7176489740 1 -- start start 2024-09-12T03:17:30.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.540+0000 7f7176489740 1 Processor -- start 2024-09-12T03:17:30.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.540+0000 7f7176489740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d4e387570 0x562d4e398830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:30.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.540+0000 7f7176489740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d4e387e90 0x562d4e398d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:30.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.540+0000 7f7176489740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d4e394c80 0x562d4e3992b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:30.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.540+0000 7f7176489740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f716c1a99e0 con 0x562d4e387e90 2024-09-12T03:17:30.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.540+0000 7f7176489740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f716c1aae50 con 0x562d4e387570 2024-09-12T03:17:30.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.540+0000 7f7176489740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f716c1aa0b0 con 0x562d4e394c80 2024-09-12T03:17:30.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.540+0000 7f7176489740 10 client.0 _ll_register_callbacks cb 0x7f7164015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:30.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.540+0000 7f717164b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d4e387e90 0x562d4e398d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:30.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.540+0000 7f717164b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d4e387e90 0x562d4e398d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52298/0 (socket says 172.21.15.116:52298) 2024-09-12T03:17:30.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.540+0000 7f717164b640 1 -- 172.21.15.116:0/527619938 learned_addr learned my addr 172.21.15.116:0/527619938 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:30.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.540+0000 7f717264d640 1 --2- 172.21.15.116:0/527619938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d4e394c80 0x562d4e3992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:30.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.540+0000 7f7171e4c640 1 --2- 172.21.15.116:0/527619938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d4e387570 0x562d4e398830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:30.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.540+0000 7f717164b640 1 -- 172.21.15.116:0/527619938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d4e394c80 msgr2=0x562d4e3992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:30.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.540+0000 7f717164b640 1 --2- 172.21.15.116:0/527619938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d4e394c80 0x562d4e3992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.540+0000 7f717164b640 1 -- 172.21.15.116:0/527619938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d4e387570 msgr2=0x562d4e398830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:30.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.540+0000 7f717164b640 1 --2- 172.21.15.116:0/527619938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d4e387570 0x562d4e398830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.541+0000 7f717164b640 1 -- 172.21.15.116:0/527619938 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d4e40e3e0 con 0x562d4e387e90 2024-09-12T03:17:30.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.541+0000 7f7171e4c640 1 --2- 172.21.15.116:0/527619938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d4e387570 0x562d4e398830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:30.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.541+0000 7f717264d640 1 --2- 172.21.15.116:0/527619938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d4e394c80 0x562d4e3992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:30.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.541+0000 7f717164b640 1 --2- 172.21.15.116:0/527619938 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d4e387e90 0x562d4e398d70 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f715c006010 tx=0x7f715c01aee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd538fe6fcf769e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:30.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.541+0000 7f71557fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:30.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.541+0000 7f71557fa640 1 -- 172.21.15.116:0/527619938 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c029020 con 0x562d4e387e90 2024-09-12T03:17:30.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.541+0000 7f71557fa640 1 -- 172.21.15.116:0/527619938 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f715c005250 con 0x562d4e387e90 2024-09-12T03:17:30.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.541+0000 7f7176489740 1 -- 172.21.15.116:0/527619938 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f715c009e30 con 0x562d4e387e90 2024-09-12T03:17:30.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.541+0000 7f71557fa640 1 -- 172.21.15.116:0/527619938 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f715c005450 con 0x562d4e387e90 2024-09-12T03:17:30.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.541+0000 7f7176489740 10 client.19302 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:30.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.541+0000 7f7176489740 1 -- 172.21.15.116:0/527619938 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562d4e40f670 con 0x562d4e387e90 2024-09-12T03:17:30.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.541+0000 7f71557fa640 1 -- 172.21.15.116:0/527619938 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f715c009e30 con 0x562d4e387e90 2024-09-12T03:17:30.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.541+0000 7f71557fa640 1 -- 172.21.15.116:0/527619938 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f715c019070 con 0x562d4e387e90 2024-09-12T03:17:30.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.541+0000 7f7176489740 10 client.19302 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:30.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.541+0000 7f7176489740 10 client.19302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:17:30.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.541+0000 7f7176489740 20 client.19302 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:30.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.541+0000 7f7176489740 1 --2- 172.21.15.116:0/527619938 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562d4e40fc80 0x562d4e412070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:30.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.541+0000 7f7176489740 4 client.19302 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:17:30.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.541+0000 7f7176489740 1 -- 172.21.15.116:0/527619938 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562d4e2f2d60 con 0x562d4e40fc80 2024-09-12T03:17:30.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.542+0000 7f7171e4c640 1 --2- 172.21.15.116:0/527619938 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562d4e40fc80 0x562d4e412070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:30.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.542+0000 7f71557fa640 10 client.19302 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:30.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.542+0000 7f7171e4c640 1 --2- 172.21.15.116:0/527619938 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562d4e40fc80 0x562d4e412070 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2e6c7914ed7e942 server_cookie=50b088dba9113b50 in_seq=0 out_seq=0 2024-09-12T03:17:30.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.542+0000 7f71557fa640 1 -- 172.21.15.116:0/527619938 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562d4e2f2d60 con 0x562d4e40fc80 2024-09-12T03:17:30.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.542+0000 7f71557fa640 10 client.19302 handle_command_reply: tid=0 2024-09-12T03:17:30.083 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:17:30.083 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:17:30.083 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:17:30.084 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:17:30.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.542+0000 7f7176489740 1 client.19302 shutdown 2024-09-12T03:17:30.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.542+0000 7f7176489740 1 -- 172.21.15.116:0/527619938 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d4e387e90 msgr2=0x562d4e398d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:30.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.542+0000 7f7176489740 1 --2- 172.21.15.116:0/527619938 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d4e387e90 0x562d4e398d70 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f715c006010 tx=0x7f715c01aee0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.542+0000 7f7176489740 1 -- 172.21.15.116:0/527619938 shutdown_connections 2024-09-12T03:17:30.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.542+0000 7f7176489740 1 --2- 172.21.15.116:0/527619938 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x562d4e40fc80 0x562d4e412070 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.542+0000 7f7176489740 1 --2- 172.21.15.116:0/527619938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d4e394c80 0x562d4e3992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.542+0000 7f7176489740 1 --2- 172.21.15.116:0/527619938 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d4e387e90 0x562d4e398d70 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.542+0000 7f71557fa640 0 client.19302 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:30.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.542+0000 7f7176489740 1 --2- 172.21.15.116:0/527619938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d4e387570 0x562d4e398830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.542+0000 7f7176489740 1 -- 172.21.15.116:0/527619938 >> 172.21.15.116:0/527619938 conn(0x562d4e383c40 msgr2=0x562d4e393190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:30.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.542+0000 7f7176489740 1 -- 172.21.15.116:0/527619938 shutdown_connections 2024-09-12T03:17:30.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.542+0000 7f7176489740 1 -- 172.21.15.116:0/527619938 wait complete. 2024-09-12T03:17:30.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.543+0000 7f7176489740 20 client.19302 trim_cache size 0 max 16384 2024-09-12T03:17:30.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.543+0000 7f71740b0640 1 -- 172.21.15.116:0/177654924 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f71580a2b80 msgr2=0x7f71580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:30.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.543+0000 7f71740b0640 1 --2- 172.21.15.116:0/177654924 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f71580a2b80 0x7f71580a5040 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7164005c40 tx=0x7f7164005bd0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.543+0000 7f71740b0640 1 -- 172.21.15.116:0/177654924 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f716c100a80 msgr2=0x7f716c1973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:30.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.543+0000 7f71740b0640 1 --2- 172.21.15.116:0/177654924 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f716c100a80 0x7f716c1973a0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f7168009230 tx=0x7f7168004270 comp rx=0 tx=0).stop 2024-09-12T03:17:30.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.543+0000 7f71740b0640 1 -- 172.21.15.116:0/177654924 shutdown_connections 2024-09-12T03:17:30.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.543+0000 7f71740b0640 1 --2- 172.21.15.116:0/177654924 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f71580a2b80 0x7f71580a5040 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.543+0000 7f71740b0640 1 --2- 172.21.15.116:0/177654924 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f716c106a80 0x7f716c19bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.543+0000 7f71740b0640 1 --2- 172.21.15.116:0/177654924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f716c101420 0x7f716c1978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.543+0000 7f71740b0640 1 --2- 172.21.15.116:0/177654924 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f716c100a80 0x7f716c1973a0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.543+0000 7f71740b0640 1 -- 172.21.15.116:0/177654924 >> 172.21.15.116:0/177654924 conn(0x7f716c0fc7a0 msgr2=0x7f716c0fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:30.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.543+0000 7f71740b0640 1 -- 172.21.15.116:0/177654924 shutdown_connections 2024-09-12T03:17:30.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:29.543+0000 7f71740b0640 1 -- 172.21.15.116:0/177654924 wait complete. 2024-09-12T03:17:30.698 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:17:30.699 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:17:30.945 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:31.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.444+0000 7fb6069fb640 1 -- 172.21.15.116:0/188109048 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb600102570 msgr2=0x7fb6001029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:31.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.444+0000 7fb6069fb640 1 --2- 172.21.15.116:0/188109048 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb600102570 0x7fb6001029b0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4009900 tx=0x7fb5f402a480 comp rx=0 tx=0).stop 2024-09-12T03:17:31.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.444+0000 7fb6069fb640 1 -- 172.21.15.116:0/188109048 shutdown_connections 2024-09-12T03:17:31.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.444+0000 7fb6069fb640 1 --2- 172.21.15.116:0/188109048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb600102ef0 0x7fb60010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.444+0000 7fb6069fb640 1 --2- 172.21.15.116:0/188109048 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb600102570 0x7fb6001029b0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.445+0000 7fb6069fb640 1 --2- 172.21.15.116:0/188109048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb600100e20 0x7fb600101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.445+0000 7fb6069fb640 1 -- 172.21.15.116:0/188109048 >> 172.21.15.116:0/188109048 conn(0x7fb6000fc7d0 msgr2=0x7fb6000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:31.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.445+0000 7fb6069fb640 1 -- 172.21.15.116:0/188109048 shutdown_connections 2024-09-12T03:17:31.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.445+0000 7fb6069fb640 1 -- 172.21.15.116:0/188109048 wait complete. 2024-09-12T03:17:31.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.445+0000 7fb6069fb640 1 Processor -- start 2024-09-12T03:17:31.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.445+0000 7fb6069fb640 1 -- start start 2024-09-12T03:17:31.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.445+0000 7fb6069fb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb600100e20 0x7fb600199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:31.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.446+0000 7fb6069fb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb600102570 0x7fb6001997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:31.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.446+0000 7fb6069fb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb600102ef0 0x7fb6001a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:31.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.446+0000 7fb6069fb640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb600110f80 con 0x7fb600102570 2024-09-12T03:17:31.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.446+0000 7fb6069fb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb600110e00 con 0x7fb600100e20 2024-09-12T03:17:31.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.446+0000 7fb6069fb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb600111100 con 0x7fb600102ef0 2024-09-12T03:17:31.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.446+0000 7fb5ff7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb600102570 0x7fb6001997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:31.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.446+0000 7fb5ff7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb600102570 0x7fb6001997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42258/0 (socket says 172.21.15.116:42258) 2024-09-12T03:17:31.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.446+0000 7fb5ff7fe640 1 -- 172.21.15.116:0/667073549 learned_addr learned my addr 172.21.15.116:0/667073549 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:31.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.446+0000 7fb604f98640 1 --2- 172.21.15.116:0/667073549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb600102ef0 0x7fb6001a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:31.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.446+0000 7fb5fffff640 1 --2- 172.21.15.116:0/667073549 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb600100e20 0x7fb600199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:31.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.446+0000 7fb5ff7fe640 1 -- 172.21.15.116:0/667073549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb600102ef0 msgr2=0x7fb6001a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:31.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.446+0000 7fb5ff7fe640 1 --2- 172.21.15.116:0/667073549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb600102ef0 0x7fb6001a4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.446+0000 7fb5ff7fe640 1 -- 172.21.15.116:0/667073549 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb600100e20 msgr2=0x7fb600199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:31.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.446+0000 7fb5ff7fe640 1 --2- 172.21.15.116:0/667073549 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb600100e20 0x7fb600199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.446+0000 7fb5ff7fe640 1 -- 172.21.15.116:0/667073549 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6001a4850 con 0x7fb600102570 2024-09-12T03:17:31.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.446+0000 7fb5fffff640 1 --2- 172.21.15.116:0/667073549 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb600100e20 0x7fb600199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:31.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.446+0000 7fb5ff7fe640 1 --2- 172.21.15.116:0/667073549 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb600102570 0x7fb6001997d0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4006010 tx=0x7fb5f4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:31.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.446+0000 7fb5fd7fa640 1 -- 172.21.15.116:0/667073549 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f403f070 con 0x7fb600102570 2024-09-12T03:17:31.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.446+0000 7fb5fd7fa640 1 -- 172.21.15.116:0/667073549 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5f4031c30 con 0x7fb600102570 2024-09-12T03:17:31.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.446+0000 7fb6069fb640 1 -- 172.21.15.116:0/667073549 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6001a4a20 con 0x7fb600102570 2024-09-12T03:17:31.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.446+0000 7fb5fd7fa640 1 -- 172.21.15.116:0/667073549 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4035030 con 0x7fb600102570 2024-09-12T03:17:31.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.446+0000 7fb6069fb640 1 -- 172.21.15.116:0/667073549 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6001a4e70 con 0x7fb600102570 2024-09-12T03:17:31.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.447+0000 7fb6069fb640 1 -- 172.21.15.116:0/667073549 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb600101200 con 0x7fb600102570 2024-09-12T03:17:31.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.449+0000 7fb5fd7fa640 1 -- 172.21.15.116:0/667073549 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb5f4042070 con 0x7fb600102570 2024-09-12T03:17:31.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.449+0000 7fb5fd7fa640 1 --2- 172.21.15.116:0/667073549 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb5e80a2b80 0x7fb5e80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:31.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.450+0000 7fb5fffff640 1 --2- 172.21.15.116:0/667073549 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb5e80a2b80 0x7fb5e80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:31.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.450+0000 7fb5fd7fa640 1 -- 172.21.15.116:0/667073549 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fb5f40ef870 con 0x7fb600102570 2024-09-12T03:17:31.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.450+0000 7fb5fffff640 1 --2- 172.21.15.116:0/667073549 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb5e80a2b80 0x7fb5e80a5040 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00058c0 tx=0x7fb5f0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:31.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.454+0000 7fb5fd7fa640 1 -- 172.21.15.116:0/667073549 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb5f4031de0 con 0x7fb600102570 2024-09-12T03:17:31.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.657+0000 7fb6069fb640 1 -- 172.21.15.116:0/667073549 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6001a5040 con 0x7fb600102570 2024-09-12T03:17:31.659 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:31 smithi116 ceph-mon[31410]: pgmap v774: 97 pgs: 97 active+clean; 6.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-12T03:17:31.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.658+0000 7fb5fd7fa640 1 -- 172.21.15.116:0/667073549 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7fb5f40bc800 con 0x7fb600102570 2024-09-12T03:17:31.659 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:17:31.660 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:17:31.660 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:17:31.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.660+0000 7fb6069fb640 1 -- 172.21.15.116:0/667073549 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb5e80a2b80 msgr2=0x7fb5e80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:31.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.660+0000 7fb6069fb640 1 --2- 172.21.15.116:0/667073549 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb5e80a2b80 0x7fb5e80a5040 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00058c0 tx=0x7fb5f0005850 comp rx=0 tx=0).stop 2024-09-12T03:17:31.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.660+0000 7fb6069fb640 1 -- 172.21.15.116:0/667073549 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb600102570 msgr2=0x7fb6001997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:31.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.660+0000 7fb6069fb640 1 --2- 172.21.15.116:0/667073549 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb600102570 0x7fb6001997d0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4006010 tx=0x7fb5f4003c10 comp rx=0 tx=0).stop 2024-09-12T03:17:31.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.660+0000 7fb6069fb640 1 -- 172.21.15.116:0/667073549 shutdown_connections 2024-09-12T03:17:31.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.660+0000 7fb6069fb640 1 --2- 172.21.15.116:0/667073549 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fb5e80a2b80 0x7fb5e80a5040 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.660+0000 7fb6069fb640 1 --2- 172.21.15.116:0/667073549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb600102ef0 0x7fb6001a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.660+0000 7fb6069fb640 1 --2- 172.21.15.116:0/667073549 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb600102570 0x7fb6001997d0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.660+0000 7fb6069fb640 1 --2- 172.21.15.116:0/667073549 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb600100e20 0x7fb600199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.660+0000 7fb6069fb640 1 -- 172.21.15.116:0/667073549 >> 172.21.15.116:0/667073549 conn(0x7fb6000fc7d0 msgr2=0x7fb6000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:31.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.660+0000 7fb6069fb640 1 -- 172.21.15.116:0/667073549 shutdown_connections 2024-09-12T03:17:31.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:31.661+0000 7fb6069fb640 1 -- 172.21.15.116:0/667073549 wait complete. 2024-09-12T03:17:31.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:31 smithi177 ceph-mon[35305]: pgmap v774: 97 pgs: 97 active+clean; 6.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-12T03:17:31.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:31 smithi183 ceph-mon[35535]: pgmap v774: 97 pgs: 97 active+clean; 6.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-12T03:17:32.363 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:17:32.610 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:32.731 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:32 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/667073549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:17:32.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:32 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/667073549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:17:32.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:32 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/667073549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:17:33.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.129+0000 7f70ca50b640 1 -- 172.21.15.116:0/2534824179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70c4102790 msgr2=0x7f70c41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.130+0000 7f70ca50b640 1 --2- 172.21.15.116:0/2534824179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70c4102790 0x7f70c41067e0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f70b8009900 tx=0x7f70b802a410 comp rx=0 tx=0).stop 2024-09-12T03:17:33.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.130+0000 7f70ca50b640 1 -- 172.21.15.116:0/2534824179 shutdown_connections 2024-09-12T03:17:33.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.130+0000 7f70ca50b640 1 --2- 172.21.15.116:0/2534824179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70c4106f10 0x7f70c410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.130+0000 7f70ca50b640 1 --2- 172.21.15.116:0/2534824179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70c4102790 0x7f70c41067e0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.130+0000 7f70ca50b640 1 --2- 172.21.15.116:0/2534824179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70c4101de0 0x7f70c41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.130+0000 7f70ca50b640 1 -- 172.21.15.116:0/2534824179 >> 172.21.15.116:0/2534824179 conn(0x7f70c40fd8c0 msgr2=0x7f70c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:33.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.130+0000 7f70ca50b640 1 -- 172.21.15.116:0/2534824179 shutdown_connections 2024-09-12T03:17:33.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.130+0000 7f70ca50b640 1 -- 172.21.15.116:0/2534824179 wait complete. 2024-09-12T03:17:33.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.130+0000 7f70ca50b640 1 Processor -- start 2024-09-12T03:17:33.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.130+0000 7f70ca50b640 1 -- start start 2024-09-12T03:17:33.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.131+0000 7f70ca50b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70c4101de0 0x7f70c40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.131+0000 7f70ca50b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70c4102790 0x7f70c4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.131+0000 7f70ca50b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70c4106f10 0x7f70c4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.131+0000 7f70ca50b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f70c4111f90 con 0x7f70c4102790 2024-09-12T03:17:33.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.131+0000 7f70ca50b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f70c4111e10 con 0x7f70c4106f10 2024-09-12T03:17:33.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.131+0000 7f70ca50b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f70c4112110 con 0x7f70c4101de0 2024-09-12T03:17:33.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.131+0000 7f70c37fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70c4102790 0x7f70c4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.131+0000 7f70c37fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70c4102790 0x7f70c4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42284/0 (socket says 172.21.15.116:42284) 2024-09-12T03:17:33.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.131+0000 7f70c37fe640 1 -- 172.21.15.116:0/4182060439 learned_addr learned my addr 172.21.15.116:0/4182060439 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:33.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.131+0000 7f70c3fff640 1 --2- 172.21.15.116:0/4182060439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70c4101de0 0x7f70c40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.131+0000 7f70c8aa8640 1 --2- 172.21.15.116:0/4182060439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70c4106f10 0x7f70c4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.131+0000 7f70c37fe640 1 -- 172.21.15.116:0/4182060439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70c4101de0 msgr2=0x7f70c40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.131+0000 7f70c37fe640 1 --2- 172.21.15.116:0/4182060439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70c4101de0 0x7f70c40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.131+0000 7f70c37fe640 1 -- 172.21.15.116:0/4182060439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70c4106f10 msgr2=0x7f70c4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.131+0000 7f70c37fe640 1 --2- 172.21.15.116:0/4182060439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70c4106f10 0x7f70c4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.131+0000 7f70c37fe640 1 -- 172.21.15.116:0/4182060439 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c41a69e0 con 0x7f70c4102790 2024-09-12T03:17:33.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.131+0000 7f70c8aa8640 1 --2- 172.21.15.116:0/4182060439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70c4106f10 0x7f70c4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:33.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.131+0000 7f70c37fe640 1 --2- 172.21.15.116:0/4182060439 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70c4102790 0x7f70c4074a30 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f70b8006010 tx=0x7f70b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:33.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.131+0000 7f70c17fa640 1 -- 172.21.15.116:0/4182060439 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b803f030 con 0x7f70c4102790 2024-09-12T03:17:33.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.131+0000 7f70ca50b640 1 -- 172.21.15.116:0/4182060439 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c41a6c70 con 0x7f70c4102790 2024-09-12T03:17:33.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.131+0000 7f70c17fa640 1 -- 172.21.15.116:0/4182060439 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70b8003ce0 con 0x7f70c4102790 2024-09-12T03:17:33.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.131+0000 7f70c17fa640 1 -- 172.21.15.116:0/4182060439 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b8035070 con 0x7f70c4102790 2024-09-12T03:17:33.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.132+0000 7f70ca50b640 1 -- 172.21.15.116:0/4182060439 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c41a7150 con 0x7f70c4102790 2024-09-12T03:17:33.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.134+0000 7f70c17fa640 1 -- 172.21.15.116:0/4182060439 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f70b8042030 con 0x7f70c4102790 2024-09-12T03:17:33.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.134+0000 7f70cc8e4740 1 Processor -- start 2024-09-12T03:17:33.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.134+0000 7f70cc8e4740 1 -- start start 2024-09-12T03:17:33.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.134+0000 7f70cc8e4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56015761ab40 0x560157619090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.134+0000 7f70cc8e4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601576195d0 0x560157619a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.134+0000 7f70cc8e4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560157624370 0x560157626760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.134+0000 7f70cc8e4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56015757f2f0 con 0x560157624370 2024-09-12T03:17:33.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.134+0000 7f70cc8e4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560157526cf0 con 0x56015761ab40 2024-09-12T03:17:33.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.134+0000 7f70cc8e4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56015757b2b0 con 0x5601576195d0 2024-09-12T03:17:33.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.134+0000 7f70c17fa640 1 --2- 172.21.15.116:0/4182060439 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f70ac0a2b80 0x7f70ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.134+0000 7f70c37fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560157624370 0x560157626760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.135+0000 7f70c37fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560157624370 0x560157626760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42294/0 (socket says 172.21.15.116:42294) 2024-09-12T03:17:33.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.135+0000 7f70c3fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601576195d0 0x560157619a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.135+0000 7f70c37fe640 1 -- 172.21.15.116:0/2862689411 learned_addr learned my addr 172.21.15.116:0/2862689411 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:33.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.135+0000 7f70c8aa8640 1 --2- 172.21.15.116:0/2862689411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56015761ab40 0x560157619090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.135+0000 7f70c8aa8640 1 --2- 172.21.15.116:0/4182060439 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f70ac0a2b80 0x7f70ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.135+0000 7f70c17fa640 1 -- 172.21.15.116:0/4182060439 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f70b80ef520 con 0x7f70c4102790 2024-09-12T03:17:33.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.135+0000 7f70c37fe640 1 -- 172.21.15.116:0/2862689411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601576195d0 msgr2=0x560157619a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:33.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.135+0000 7f70c37fe640 1 --2- 172.21.15.116:0/2862689411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601576195d0 0x560157619a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.135+0000 7f70c37fe640 1 -- 172.21.15.116:0/2862689411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56015761ab40 msgr2=0x560157619090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:33.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.135+0000 7f70c37fe640 1 --2- 172.21.15.116:0/2862689411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56015761ab40 0x560157619090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.135+0000 7f70c37fe640 1 -- 172.21.15.116:0/2862689411 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56015750b3a0 con 0x560157624370 2024-09-12T03:17:33.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.135+0000 7f70c37fe640 1 --2- 172.21.15.116:0/2862689411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560157624370 0x560157626760 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f70b804a000 tx=0x7f70b80b68d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0c86010432a44e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:33.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.135+0000 7f70c8aa8640 1 --2- 172.21.15.116:0/4182060439 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f70ac0a2b80 0x7f70ac0a5040 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f70bc014bd0 tx=0x7f70bc0012a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:33.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.135+0000 7f70aa7fc640 1 -- 172.21.15.116:0/2862689411 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b8049070 con 0x560157624370 2024-09-12T03:17:33.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.135+0000 7f70aa7fc640 1 -- 172.21.15.116:0/2862689411 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70b80bc570 con 0x560157624370 2024-09-12T03:17:33.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.135+0000 7f70aa7fc640 1 -- 172.21.15.116:0/2862689411 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b80c0220 con 0x560157624370 2024-09-12T03:17:33.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.136+0000 7f70cc8e4740 1 -- 172.21.15.116:0/2862689411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560157624370 msgr2=0x560157626760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.136+0000 7f70cc8e4740 1 --2- 172.21.15.116:0/2862689411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560157624370 0x560157626760 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f70b804a000 tx=0x7f70b80b68d0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.136+0000 7f70cc8e4740 1 -- 172.21.15.116:0/2862689411 shutdown_connections 2024-09-12T03:17:33.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.136+0000 7f70cc8e4740 1 --2- 172.21.15.116:0/2862689411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560157624370 0x560157626760 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.136+0000 7f70cc8e4740 1 --2- 172.21.15.116:0/2862689411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601576195d0 0x560157619a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.136+0000 7f70cc8e4740 1 --2- 172.21.15.116:0/2862689411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56015761ab40 0x560157619090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.136+0000 7f70cc8e4740 1 -- 172.21.15.116:0/2862689411 >> 172.21.15.116:0/2862689411 conn(0x5601575e2310 msgr2=0x560157615c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:33.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.136+0000 7f70cc8e4740 1 -- 172.21.15.116:0/2862689411 shutdown_connections 2024-09-12T03:17:33.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.136+0000 7f70cc8e4740 1 -- 172.21.15.116:0/2862689411 wait complete. 2024-09-12T03:17:33.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.136+0000 7f70cc8e4740 1 -- start start 2024-09-12T03:17:33.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.137+0000 7f70cc8e4740 1 Processor -- start 2024-09-12T03:17:33.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.137+0000 7f70cc8e4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601576195d0 0x560157623d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.137+0000 7f70cc8e4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56015761ab40 0x560157621020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.137+0000 7f70cc8e4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560157624370 0x560157621610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.137+0000 7f70cc8e4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560157626e30 con 0x56015761ab40 2024-09-12T03:17:33.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.137+0000 7f70cc8e4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56015753a060 con 0x560157624370 2024-09-12T03:17:33.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.137+0000 7f70cc8e4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560157626fb0 con 0x5601576195d0 2024-09-12T03:17:33.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.137+0000 7f70cc8e4740 10 client.0 _ll_register_callbacks cb 0x56015728d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:33.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.137+0000 7f70c37fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56015761ab40 0x560157621020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.137+0000 7f70c37fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56015761ab40 0x560157621020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42304/0 (socket says 172.21.15.116:42304) 2024-09-12T03:17:33.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.137+0000 7f70c37fe640 1 -- 172.21.15.116:0/1499206273 learned_addr learned my addr 172.21.15.116:0/1499206273 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:33.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.137+0000 7f70c3fff640 1 --2- 172.21.15.116:0/1499206273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601576195d0 0x560157623d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.137+0000 7f70c37fe640 1 -- 172.21.15.116:0/1499206273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601576195d0 msgr2=0x560157623d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:33.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.137+0000 7f70c37fe640 1 --2- 172.21.15.116:0/1499206273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601576195d0 0x560157623d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.137+0000 7f70c37fe640 1 -- 172.21.15.116:0/1499206273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560157624370 msgr2=0x560157621610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:33.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.137+0000 7f70c37fe640 1 --2- 172.21.15.116:0/1499206273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560157624370 0x560157621610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.137+0000 7f70c37fe640 1 -- 172.21.15.116:0/1499206273 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601572f03c0 con 0x56015761ab40 2024-09-12T03:17:33.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.137+0000 7f70c37fe640 1 --2- 172.21.15.116:0/1499206273 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56015761ab40 0x560157621020 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f70b80ed3c0 tx=0x7f70b803bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6020eb912210f5c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:33.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.137+0000 7f70a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:33.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.138+0000 7f70a37fe640 1 -- 172.21.15.116:0/1499206273 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b8049070 con 0x56015761ab40 2024-09-12T03:17:33.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.138+0000 7f70a37fe640 1 -- 172.21.15.116:0/1499206273 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f70b80b84e0 con 0x56015761ab40 2024-09-12T03:17:33.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.138+0000 7f70cc8e4740 1 -- 172.21.15.116:0/1499206273 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70b80bc570 con 0x56015761ab40 2024-09-12T03:17:33.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.138+0000 7f70a37fe640 1 -- 172.21.15.116:0/1499206273 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f70b80dd070 con 0x56015761ab40 2024-09-12T03:17:33.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.138+0000 7f70cc8e4740 10 client.19320 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:33.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.138+0000 7f70cc8e4740 1 -- 172.21.15.116:0/1499206273 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560157626e30 con 0x56015761ab40 2024-09-12T03:17:33.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.138+0000 7f70a37fe640 1 -- 172.21.15.116:0/1499206273 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70b80bc570 con 0x56015761ab40 2024-09-12T03:17:33.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.138+0000 7f70a37fe640 1 -- 172.21.15.116:0/1499206273 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f70b80c4650 con 0x56015761ab40 2024-09-12T03:17:33.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.138+0000 7f70cc8e4740 10 client.19320 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:33.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.138+0000 7f70cc8e4740 10 client.19320 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:17:33.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.138+0000 7f70cc8e4740 20 client.19320 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:33.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.138+0000 7f70cc8e4740 1 --2- 172.21.15.116:0/1499206273 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5601576339d0 0x560157635dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.138+0000 7f70cc8e4740 4 client.19320 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:17:33.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.138+0000 7f70cc8e4740 1 -- 172.21.15.116:0/1499206273 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56015758c860 con 0x5601576339d0 2024-09-12T03:17:33.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.139+0000 7f70c3fff640 1 --2- 172.21.15.116:0/1499206273 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5601576339d0 0x560157635dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.139+0000 7f70a37fe640 10 client.19320 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:17:33.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.139+0000 7f70c3fff640 1 --2- 172.21.15.116:0/1499206273 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5601576339d0 0x560157635dc0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5030e42db2e69681 server_cookie=518f2ea3d842c1aa in_seq=0 out_seq=0 2024-09-12T03:17:33.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.140+0000 7f70a37fe640 1 -- 172.21.15.116:0/1499206273 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56015758c860 con 0x5601576339d0 2024-09-12T03:17:33.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.140+0000 7f70a37fe640 10 client.19320 handle_command_reply: tid=0 2024-09-12T03:17:33.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.140+0000 7f70cc8e4740 1 client.19320 shutdown 2024-09-12T03:17:33.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.140+0000 7f70cc8e4740 1 -- 172.21.15.116:0/1499206273 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56015761ab40 msgr2=0x560157621020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.140+0000 7f70cc8e4740 1 --2- 172.21.15.116:0/1499206273 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56015761ab40 0x560157621020 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f70b80ed3c0 tx=0x7f70b803bcb0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.140+0000 7f70cc8e4740 1 -- 172.21.15.116:0/1499206273 shutdown_connections 2024-09-12T03:17:33.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.140+0000 7f70cc8e4740 1 --2- 172.21.15.116:0/1499206273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560157624370 0x560157621610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.140+0000 7f70cc8e4740 1 --2- 172.21.15.116:0/1499206273 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x5601576339d0 0x560157635dc0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.140+0000 7f70cc8e4740 1 --2- 172.21.15.116:0/1499206273 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56015761ab40 0x560157621020 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.140+0000 7f70cc8e4740 1 --2- 172.21.15.116:0/1499206273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601576195d0 0x560157623d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.140+0000 7f70a37fe640 0 client.19320 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:17:33.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.140+0000 7f70cc8e4740 1 -- 172.21.15.116:0/1499206273 >> 172.21.15.116:0/1499206273 conn(0x5601575e2310 msgr2=0x56015761b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:33.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.140+0000 7f70cc8e4740 1 -- 172.21.15.116:0/1499206273 shutdown_connections 2024-09-12T03:17:33.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.140+0000 7f70cc8e4740 1 -- 172.21.15.116:0/1499206273 wait complete. 2024-09-12T03:17:33.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.140+0000 7f70cc8e4740 20 client.19320 trim_cache size 0 max 16384 2024-09-12T03:17:33.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.153+0000 7f70cc8e4740 1 Processor -- start 2024-09-12T03:17:33.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.153+0000 7f70cc8e4740 1 -- start start 2024-09-12T03:17:33.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.153+0000 7f70cc8e4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601576195d0 0x560157620ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.153+0000 7f70cc8e4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56015761ab40 0x560157630090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.153+0000 7f70cc8e4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560157624370 0x56015761a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.153+0000 7f70cc8e4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f70c41b1f50 con 0x560157624370 2024-09-12T03:17:33.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.153+0000 7f70cc8e4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f70c41af820 con 0x56015761ab40 2024-09-12T03:17:33.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.153+0000 7f70cc8e4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f70c41ad670 con 0x5601576195d0 2024-09-12T03:17:33.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.153+0000 7f70c8aa8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560157624370 0x56015761a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.153+0000 7f70c8aa8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560157624370 0x56015761a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42306/0 (socket says 172.21.15.116:42306) 2024-09-12T03:17:33.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.153+0000 7f70c8aa8640 1 -- 172.21.15.116:0/2986992393 learned_addr learned my addr 172.21.15.116:0/2986992393 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:33.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.153+0000 7f70c37fe640 1 --2- 172.21.15.116:0/2986992393 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56015761ab40 0x560157630090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.153+0000 7f70c3fff640 1 --2- 172.21.15.116:0/2986992393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601576195d0 0x560157620ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.153+0000 7f70c8aa8640 1 -- 172.21.15.116:0/2986992393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601576195d0 msgr2=0x560157620ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:33.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.153+0000 7f70c8aa8640 1 --2- 172.21.15.116:0/2986992393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601576195d0 0x560157620ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.153+0000 7f70c8aa8640 1 -- 172.21.15.116:0/2986992393 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56015761ab40 msgr2=0x560157630090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:33.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.153+0000 7f70c8aa8640 1 --2- 172.21.15.116:0/2986992393 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56015761ab40 0x560157630090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.153+0000 7f70c8aa8640 1 -- 172.21.15.116:0/2986992393 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56015761fff0 con 0x560157624370 2024-09-12T03:17:33.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.154+0000 7f70c8aa8640 1 --2- 172.21.15.116:0/2986992393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560157624370 0x56015761a300 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f70bc004a10 tx=0x7f70bc004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c860a0895e34994 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:33.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.154+0000 7f70a9ffb640 1 -- 172.21.15.116:0/2986992393 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc025070 con 0x560157624370 2024-09-12T03:17:33.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.154+0000 7f70a9ffb640 1 -- 172.21.15.116:0/2986992393 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70bc0208f0 con 0x560157624370 2024-09-12T03:17:33.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.154+0000 7f70a9ffb640 1 -- 172.21.15.116:0/2986992393 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc0139e0 con 0x560157624370 2024-09-12T03:17:33.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.154+0000 7f70cc8e4740 1 -- 172.21.15.116:0/2986992393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560157624370 msgr2=0x56015761a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.154+0000 7f70cc8e4740 1 --2- 172.21.15.116:0/2986992393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560157624370 0x56015761a300 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f70bc004a10 tx=0x7f70bc004b50 comp rx=0 tx=0).stop 2024-09-12T03:17:33.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.154+0000 7f70cc8e4740 1 -- 172.21.15.116:0/2986992393 shutdown_connections 2024-09-12T03:17:33.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.154+0000 7f70cc8e4740 1 --2- 172.21.15.116:0/2986992393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560157624370 0x56015761a300 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.154+0000 7f70cc8e4740 1 --2- 172.21.15.116:0/2986992393 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56015761ab40 0x560157630090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.154+0000 7f70cc8e4740 1 --2- 172.21.15.116:0/2986992393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601576195d0 0x560157620ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.154+0000 7f70cc8e4740 1 -- 172.21.15.116:0/2986992393 >> 172.21.15.116:0/2986992393 conn(0x5601575e2310 msgr2=0x5601576208e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:33.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.154+0000 7f70cc8e4740 1 -- 172.21.15.116:0/2986992393 shutdown_connections 2024-09-12T03:17:33.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.154+0000 7f70cc8e4740 1 -- 172.21.15.116:0/2986992393 wait complete. 2024-09-12T03:17:33.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.155+0000 7f70cc8e4740 1 -- start start 2024-09-12T03:17:33.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.155+0000 7f70cc8e4740 1 Processor -- start 2024-09-12T03:17:33.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.155+0000 7f70cc8e4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601576195d0 0x56015762c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.155+0000 7f70cc8e4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56015761ab40 0x56015762c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.155+0000 7f70cc8e4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560157624370 0x5601576291d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.155+0000 7f70cc8e4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f70c41adce0 con 0x56015761ab40 2024-09-12T03:17:33.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.155+0000 7f70cc8e4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f70c41af150 con 0x560157624370 2024-09-12T03:17:33.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.155+0000 7f70cc8e4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f70c41ae3b0 con 0x5601576195d0 2024-09-12T03:17:33.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.155+0000 7f70cc8e4740 10 client.0 _ll_register_callbacks cb 0x7f70bc00ad80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:33.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.155+0000 7f70c37fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56015761ab40 0x56015762c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.155+0000 7f70c37fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56015761ab40 0x56015762c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42314/0 (socket says 172.21.15.116:42314) 2024-09-12T03:17:33.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.155+0000 7f70c37fe640 1 -- 172.21.15.116:0/2703049519 learned_addr learned my addr 172.21.15.116:0/2703049519 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:33.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.156+0000 7f70c3fff640 1 --2- 172.21.15.116:0/2703049519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601576195d0 0x56015762c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.156+0000 7f70c8aa8640 1 --2- 172.21.15.116:0/2703049519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560157624370 0x5601576291d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.156+0000 7f70c37fe640 1 -- 172.21.15.116:0/2703049519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601576195d0 msgr2=0x56015762c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:33.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.156+0000 7f70c37fe640 1 --2- 172.21.15.116:0/2703049519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601576195d0 0x56015762c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.156+0000 7f70c37fe640 1 -- 172.21.15.116:0/2703049519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560157624370 msgr2=0x5601576291d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:33.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.156+0000 7f70c37fe640 1 --2- 172.21.15.116:0/2703049519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560157624370 0x5601576291d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.156+0000 7f70c37fe640 1 -- 172.21.15.116:0/2703049519 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601576240e0 con 0x56015761ab40 2024-09-12T03:17:33.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.156+0000 7f70c8aa8640 1 --2- 172.21.15.116:0/2703049519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560157624370 0x5601576291d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:33.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.156+0000 7f70c37fe640 1 --2- 172.21.15.116:0/2703049519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56015761ab40 0x56015762c740 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f70b80bc4d0 tx=0x7f70b8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed4ff9ae85f9bad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:33.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.156+0000 7f70a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:33.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.156+0000 7f70a37fe640 1 -- 172.21.15.116:0/2703049519 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b8049040 con 0x56015761ab40 2024-09-12T03:17:33.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.156+0000 7f70a37fe640 1 -- 172.21.15.116:0/2703049519 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f70b8031b70 con 0x56015761ab40 2024-09-12T03:17:33.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.156+0000 7f70cc8e4740 1 -- 172.21.15.116:0/2703049519 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70bc0208f0 con 0x56015761ab40 2024-09-12T03:17:33.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.156+0000 7f70a37fe640 1 -- 172.21.15.116:0/2703049519 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f70b80bb7d0 con 0x56015761ab40 2024-09-12T03:17:33.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.156+0000 7f70cc8e4740 10 client.19326 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:33.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.156+0000 7f70cc8e4740 1 -- 172.21.15.116:0/2703049519 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5601576240e0 con 0x56015761ab40 2024-09-12T03:17:33.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.156+0000 7f70a37fe640 1 -- 172.21.15.116:0/2703049519 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70bc0208f0 con 0x56015761ab40 2024-09-12T03:17:33.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.157+0000 7f70a37fe640 1 -- 172.21.15.116:0/2703049519 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f70b80bb2c0 con 0x56015761ab40 2024-09-12T03:17:33.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.157+0000 7f70cc8e4740 10 client.19326 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:33.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.157+0000 7f70cc8e4740 10 client.19326 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:17:33.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.157+0000 7f70cc8e4740 20 client.19326 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:33.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.157+0000 7f70cc8e4740 1 --2- 172.21.15.116:0/2703049519 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x56015769eca0 0x5601576a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.157+0000 7f70cc8e4740 4 client.19326 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:17:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.157+0000 7f70cc8e4740 1 -- 172.21.15.116:0/2703049519 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560157586d70 con 0x56015769eca0 2024-09-12T03:17:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.157+0000 7f70c3fff640 1 --2- 172.21.15.116:0/2703049519 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x56015769eca0 0x5601576a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.157+0000 7f70c3fff640 1 --2- 172.21.15.116:0/2703049519 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x56015769eca0 0x5601576a10c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7d4d9393eaf32d8e server_cookie=ac007e84f8e55f11 in_seq=0 out_seq=0 2024-09-12T03:17:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.157+0000 7f70a37fe640 10 client.19326 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:17:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.158+0000 7f70a37fe640 1 -- 172.21.15.116:0/2703049519 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560157586d70 con 0x56015769eca0 2024-09-12T03:17:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.158+0000 7f70a37fe640 10 client.19326 handle_command_reply: tid=0 2024-09-12T03:17:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.158+0000 7f70cc8e4740 1 client.19326 shutdown 2024-09-12T03:17:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.158+0000 7f70cc8e4740 1 -- 172.21.15.116:0/2703049519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56015761ab40 msgr2=0x56015762c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.158+0000 7f70cc8e4740 1 --2- 172.21.15.116:0/2703049519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56015761ab40 0x56015762c740 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f70b80bc4d0 tx=0x7f70b8003f10 comp rx=0 tx=0).stop 2024-09-12T03:17:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.158+0000 7f70cc8e4740 1 -- 172.21.15.116:0/2703049519 shutdown_connections 2024-09-12T03:17:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.158+0000 7f70cc8e4740 1 --2- 172.21.15.116:0/2703049519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560157624370 0x5601576291d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.158+0000 7f70cc8e4740 1 --2- 172.21.15.116:0/2703049519 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x56015769eca0 0x5601576a10c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.158+0000 7f70cc8e4740 1 --2- 172.21.15.116:0/2703049519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56015761ab40 0x56015762c740 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.158+0000 7f70a37fe640 0 client.19326 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:17:33.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.158+0000 7f70cc8e4740 1 --2- 172.21.15.116:0/2703049519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601576195d0 0x56015762c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.158+0000 7f70cc8e4740 1 -- 172.21.15.116:0/2703049519 >> 172.21.15.116:0/2703049519 conn(0x5601575e2310 msgr2=0x560157622880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:33.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.158+0000 7f70cc8e4740 1 -- 172.21.15.116:0/2703049519 shutdown_connections 2024-09-12T03:17:33.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.158+0000 7f70cc8e4740 1 -- 172.21.15.116:0/2703049519 wait complete. 2024-09-12T03:17:33.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.158+0000 7f70cc8e4740 20 client.19326 trim_cache size 0 max 16384 2024-09-12T03:17:33.175 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:17:33.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.158+0000 7f70ca50b640 1 -- 172.21.15.116:0/4182060439 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f70ac0a2b80 msgr2=0x7f70ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.158+0000 7f70ca50b640 1 --2- 172.21.15.116:0/4182060439 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f70ac0a2b80 0x7f70ac0a5040 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f70bc014bd0 tx=0x7f70bc0012a0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.158+0000 7f70ca50b640 1 -- 172.21.15.116:0/4182060439 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70c4102790 msgr2=0x7f70c4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.158+0000 7f70ca50b640 1 --2- 172.21.15.116:0/4182060439 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70c4102790 0x7f70c4074a30 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f70b8006010 tx=0x7f70b80042f0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.159+0000 7f70ca50b640 1 -- 172.21.15.116:0/4182060439 shutdown_connections 2024-09-12T03:17:33.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.159+0000 7f70ca50b640 1 --2- 172.21.15.116:0/4182060439 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f70ac0a2b80 0x7f70ac0a5040 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.159+0000 7f70ca50b640 1 --2- 172.21.15.116:0/4182060439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f70c4106f10 0x7f70c4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.159+0000 7f70ca50b640 1 --2- 172.21.15.116:0/4182060439 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70c4102790 0x7f70c4074a30 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.159+0000 7f70ca50b640 1 --2- 172.21.15.116:0/4182060439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70c4101de0 0x7f70c40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.159+0000 7f70ca50b640 1 -- 172.21.15.116:0/4182060439 >> 172.21.15.116:0/4182060439 conn(0x7f70c40fd8c0 msgr2=0x7f70c40fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:33.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.159+0000 7f70ca50b640 1 -- 172.21.15.116:0/4182060439 shutdown_connections 2024-09-12T03:17:33.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:33.159+0000 7f70ca50b640 1 -- 172.21.15.116:0/4182060439 wait complete. 2024-09-12T03:17:33.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:33 smithi177 ceph-mon[35305]: pgmap v775: 97 pgs: 97 active+clean; 6.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-12T03:17:33.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:33 smithi116 ceph-mon[31410]: pgmap v775: 97 pgs: 97 active+clean; 6.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-12T03:17:33.857 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:17:33.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:33 smithi183 ceph-mon[35535]: pgmap v775: 97 pgs: 97 active+clean; 6.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-12T03:17:34.103 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:34.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.587+0000 7f40aaa69640 1 -- 172.21.15.116:0/2027625901 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40a4102790 msgr2=0x7f40a41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:34.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.587+0000 7f40aaa69640 1 --2- 172.21.15.116:0/2027625901 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40a4102790 0x7f40a41067e0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f4090009900 tx=0x7f409002a410 comp rx=0 tx=0).stop 2024-09-12T03:17:34.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.587+0000 7f40aaa69640 1 -- 172.21.15.116:0/2027625901 shutdown_connections 2024-09-12T03:17:34.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.587+0000 7f40aaa69640 1 --2- 172.21.15.116:0/2027625901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40a4106f10 0x7f40a410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.587+0000 7f40aaa69640 1 --2- 172.21.15.116:0/2027625901 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40a4102790 0x7f40a41067e0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.587+0000 7f40aaa69640 1 --2- 172.21.15.116:0/2027625901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40a4101de0 0x7f40a41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.587+0000 7f40aaa69640 1 -- 172.21.15.116:0/2027625901 >> 172.21.15.116:0/2027625901 conn(0x7f40a40fd8c0 msgr2=0x7f40a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:34.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.587+0000 7f40aaa69640 1 -- 172.21.15.116:0/2027625901 shutdown_connections 2024-09-12T03:17:34.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.587+0000 7f40aaa69640 1 -- 172.21.15.116:0/2027625901 wait complete. 2024-09-12T03:17:34.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.588+0000 7f40aaa69640 1 Processor -- start 2024-09-12T03:17:34.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.588+0000 7f40aaa69640 1 -- start start 2024-09-12T03:17:34.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.588+0000 7f40aaa69640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40a4101de0 0x7f40a40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.588+0000 7f40aaa69640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40a4102790 0x7f40a4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.588+0000 7f40aaa69640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40a4106f10 0x7f40a4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.588+0000 7f40aaa69640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f40a4111f70 con 0x7f40a4106f10 2024-09-12T03:17:34.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.588+0000 7f40aaa69640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f40a4111df0 con 0x7f40a4101de0 2024-09-12T03:17:34.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.588+0000 7f40aaa69640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f40a41120f0 con 0x7f40a4102790 2024-09-12T03:17:34.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.588+0000 7f40a9006640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40a4106f10 0x7f40a4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.588+0000 7f40a9006640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40a4106f10 0x7f40a4074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42326/0 (socket says 172.21.15.116:42326) 2024-09-12T03:17:34.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.588+0000 7f40a9006640 1 -- 172.21.15.116:0/2038570963 learned_addr learned my addr 172.21.15.116:0/2038570963 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:34.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.588+0000 7f40a8805640 1 --2- 172.21.15.116:0/2038570963 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40a4101de0 0x7f40a40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.588+0000 7f40a3fff640 1 --2- 172.21.15.116:0/2038570963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40a4102790 0x7f40a4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.589+0000 7f40a9006640 1 -- 172.21.15.116:0/2038570963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40a4102790 msgr2=0x7f40a4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:34.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.589+0000 7f40a9006640 1 --2- 172.21.15.116:0/2038570963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40a4102790 0x7f40a4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.589+0000 7f40a9006640 1 -- 172.21.15.116:0/2038570963 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40a4101de0 msgr2=0x7f40a40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:34.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.589+0000 7f40a9006640 1 --2- 172.21.15.116:0/2038570963 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40a4101de0 0x7f40a40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.589+0000 7f40a9006640 1 -- 172.21.15.116:0/2038570963 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40a41a6890 con 0x7f40a4106f10 2024-09-12T03:17:34.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.589+0000 7f40a9006640 1 --2- 172.21.15.116:0/2038570963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40a4106f10 0x7f40a4074f70 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f409c00cce0 tx=0x7f409c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:34.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.589+0000 7f40a1ffb640 1 -- 172.21.15.116:0/2038570963 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c00f450 con 0x7f40a4106f10 2024-09-12T03:17:34.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.589+0000 7f40a1ffb640 1 -- 172.21.15.116:0/2038570963 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f409c00fad0 con 0x7f40a4106f10 2024-09-12T03:17:34.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.589+0000 7f40aaa69640 1 -- 172.21.15.116:0/2038570963 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40a41a6b80 con 0x7f40a4106f10 2024-09-12T03:17:34.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.589+0000 7f40a1ffb640 1 -- 172.21.15.116:0/2038570963 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c018c10 con 0x7f40a4106f10 2024-09-12T03:17:34.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.589+0000 7f40aaa69640 1 -- 172.21.15.116:0/2038570963 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40a41a7070 con 0x7f40a4106f10 2024-09-12T03:17:34.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.591+0000 7f40a1ffb640 1 -- 172.21.15.116:0/2038570963 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f409c00f5f0 con 0x7f40a4106f10 2024-09-12T03:17:34.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.592+0000 7f40ace42740 1 Processor -- start 2024-09-12T03:17:34.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.592+0000 7f40ace42740 1 -- start start 2024-09-12T03:17:34.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.592+0000 7f40ace42740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c133472b40 0x55c133471090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.592+0000 7f40ace42740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1334715d0 0x55c133471a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.592+0000 7f40ace42740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13347c370 0x55c13347e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.592+0000 7f40ace42740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c1333d72f0 con 0x55c133472b40 2024-09-12T03:17:34.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.592+0000 7f40ace42740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c13337ecf0 con 0x55c13347c370 2024-09-12T03:17:34.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.592+0000 7f40ace42740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c1333d32b0 con 0x55c1334715d0 2024-09-12T03:17:34.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.592+0000 7f40a1ffb640 1 --2- 172.21.15.116:0/2038570963 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f40940a2b80 0x7f40940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.592+0000 7f40a3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c133472b40 0x55c133471090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.592+0000 7f40a3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c133472b40 0x55c133471090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42330/0 (socket says 172.21.15.116:42330) 2024-09-12T03:17:34.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.592+0000 7f40a3fff640 1 -- 172.21.15.116:0/4065863196 learned_addr learned my addr 172.21.15.116:0/4065863196 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:34.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.592+0000 7f40a9006640 1 --2- 172.21.15.116:0/2038570963 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f40940a2b80 0x7f40940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.592+0000 7f40a8805640 1 --2- 172.21.15.116:0/4065863196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1334715d0 0x55c133471a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.592+0000 7f40a3fff640 1 --2- 172.21.15.116:0/4065863196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13347c370 0x55c13347e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.592+0000 7f40a3fff640 1 -- 172.21.15.116:0/4065863196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1334715d0 msgr2=0x55c133471a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:34.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.592+0000 7f40a1ffb640 1 -- 172.21.15.116:0/2038570963 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f409c0cf760 con 0x7f40a4106f10 2024-09-12T03:17:34.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.592+0000 7f40a3fff640 1 --2- 172.21.15.116:0/4065863196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1334715d0 0x55c133471a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.592+0000 7f40a3fff640 1 -- 172.21.15.116:0/4065863196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13347c370 msgr2=0x55c13347e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:34.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.592+0000 7f40a3fff640 1 --2- 172.21.15.116:0/4065863196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13347c370 0x55c13347e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.592+0000 7f40a3fff640 1 -- 172.21.15.116:0/4065863196 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1333633a0 con 0x55c133472b40 2024-09-12T03:17:34.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.593+0000 7f40a3fff640 1 --2- 172.21.15.116:0/4065863196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c133472b40 0x55c133471090 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f409002a930 tx=0x7f4090031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a50dcc6745988144 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:34.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.593+0000 7f40a9006640 1 --2- 172.21.15.116:0/2038570963 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f40940a2b80 0x7f40940a5040 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f409c0cd2e0 tx=0x7f409c0c6b80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:34.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.593+0000 7f408effd640 1 -- 172.21.15.116:0/4065863196 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4090035070 con 0x55c133472b40 2024-09-12T03:17:34.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.593+0000 7f408effd640 1 -- 172.21.15.116:0/4065863196 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4090044070 con 0x55c133472b40 2024-09-12T03:17:34.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.593+0000 7f408effd640 1 -- 172.21.15.116:0/4065863196 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40900046c0 con 0x55c133472b40 2024-09-12T03:17:34.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.593+0000 7f40ace42740 1 -- 172.21.15.116:0/4065863196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c133472b40 msgr2=0x55c133471090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:34.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.593+0000 7f40ace42740 1 --2- 172.21.15.116:0/4065863196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c133472b40 0x55c133471090 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f409002a930 tx=0x7f4090031400 comp rx=0 tx=0).stop 2024-09-12T03:17:34.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.593+0000 7f40ace42740 1 -- 172.21.15.116:0/4065863196 shutdown_connections 2024-09-12T03:17:34.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.593+0000 7f40ace42740 1 --2- 172.21.15.116:0/4065863196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13347c370 0x55c13347e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.594+0000 7f40ace42740 1 --2- 172.21.15.116:0/4065863196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1334715d0 0x55c133471a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.594+0000 7f40ace42740 1 --2- 172.21.15.116:0/4065863196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c133472b40 0x55c133471090 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.594+0000 7f40ace42740 1 -- 172.21.15.116:0/4065863196 >> 172.21.15.116:0/4065863196 conn(0x55c13343a310 msgr2=0x55c13346dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:34.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.594+0000 7f40ace42740 1 -- 172.21.15.116:0/4065863196 shutdown_connections 2024-09-12T03:17:34.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.594+0000 7f40ace42740 1 -- 172.21.15.116:0/4065863196 wait complete. 2024-09-12T03:17:34.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.594+0000 7f40ace42740 1 -- start start 2024-09-12T03:17:34.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.594+0000 7f40ace42740 1 Processor -- start 2024-09-12T03:17:34.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.594+0000 7f40ace42740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c1334715d0 0x55c13347bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.594+0000 7f40ace42740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c133472b40 0x55c133478fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.594+0000 7f40ace42740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c13347c370 0x55c133479560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.594+0000 7f40ace42740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c13347f010 con 0x55c133472b40 2024-09-12T03:17:34.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.594+0000 7f40ace42740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c133392060 con 0x55c1334715d0 2024-09-12T03:17:34.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.594+0000 7f40ace42740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c13347f190 con 0x55c13347c370 2024-09-12T03:17:34.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.595+0000 7f40ace42740 10 client.0 _ll_register_callbacks cb 0x55c1330e58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:34.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.595+0000 7f40a8805640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c133472b40 0x55c133478fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.595+0000 7f40a8805640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c133472b40 0x55c133478fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42344/0 (socket says 172.21.15.116:42344) 2024-09-12T03:17:34.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.595+0000 7f40a8805640 1 -- 172.21.15.116:0/2934740887 learned_addr learned my addr 172.21.15.116:0/2934740887 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:34.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.595+0000 7f40a3fff640 1 --2- 172.21.15.116:0/2934740887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c13347c370 0x55c133479560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.595+0000 7f40a8805640 1 -- 172.21.15.116:0/2934740887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c13347c370 msgr2=0x55c133479560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:34.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.595+0000 7f40a8805640 1 --2- 172.21.15.116:0/2934740887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c13347c370 0x55c133479560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.595+0000 7f40a8805640 1 -- 172.21.15.116:0/2934740887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c1334715d0 msgr2=0x55c13347bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:34.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.595+0000 7f40a3fff640 1 --2- 172.21.15.116:0/2934740887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c1334715d0 0x55c13347bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.595+0000 7f40a8805640 1 --2- 172.21.15.116:0/2934740887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c1334715d0 0x55c13347bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.595+0000 7f40a8805640 1 -- 172.21.15.116:0/2934740887 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1331483c0 con 0x55c133472b40 2024-09-12T03:17:34.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.595+0000 7f40a3fff640 1 --2- 172.21.15.116:0/2934740887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c1334715d0 0x55c13347bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:34.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.595+0000 7f40a8805640 1 --2- 172.21.15.116:0/2934740887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c133472b40 0x55c133478fc0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f409800bed0 tx=0x7f40980117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae27203ddcd1cbc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:34.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.595+0000 7f4083fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:34.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.595+0000 7f4083fff640 1 -- 172.21.15.116:0/2934740887 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4098012650 con 0x55c133472b40 2024-09-12T03:17:34.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.595+0000 7f4083fff640 1 -- 172.21.15.116:0/2934740887 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f40980127f0 con 0x55c133472b40 2024-09-12T03:17:34.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.595+0000 7f40ace42740 1 -- 172.21.15.116:0/2934740887 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4090044070 con 0x55c133472b40 2024-09-12T03:17:34.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.595+0000 7f4083fff640 1 -- 172.21.15.116:0/2934740887 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f40980129f0 con 0x55c133472b40 2024-09-12T03:17:34.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.595+0000 7f40ace42740 10 client.19338 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:34.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.595+0000 7f40ace42740 1 -- 172.21.15.116:0/2934740887 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c13348aaf0 con 0x55c133472b40 2024-09-12T03:17:34.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.596+0000 7f4083fff640 1 -- 172.21.15.116:0/2934740887 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4090044070 con 0x55c133472b40 2024-09-12T03:17:34.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.596+0000 7f4083fff640 1 -- 172.21.15.116:0/2934740887 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4098026420 con 0x55c133472b40 2024-09-12T03:17:34.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.596+0000 7f40ace42740 10 client.19338 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:34.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.596+0000 7f40ace42740 10 client.19338 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:17:34.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.596+0000 7f40ace42740 20 client.19338 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:34.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.596+0000 7f40ace42740 1 --2- 172.21.15.116:0/2934740887 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c13348c000 0x55c13348e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.596+0000 7f40ace42740 4 client.19338 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:17:34.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.596+0000 7f40ace42740 1 -- 172.21.15.116:0/2934740887 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1333e4860 con 0x55c13348c000 2024-09-12T03:17:34.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.596+0000 7f40a3fff640 1 --2- 172.21.15.116:0/2934740887 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c13348c000 0x55c13348e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.596+0000 7f4083fff640 10 client.19338 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:17:34.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.596+0000 7f40a3fff640 1 --2- 172.21.15.116:0/2934740887 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c13348c000 0x55c13348e460 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eb3e5c99696d144f server_cookie=462afaf9c6db8731 in_seq=0 out_seq=0 2024-09-12T03:17:34.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.597+0000 7f4083fff640 1 -- 172.21.15.116:0/2934740887 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c1333e4860 con 0x55c13348c000 2024-09-12T03:17:34.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.597+0000 7f4083fff640 10 client.19338 handle_command_reply: tid=0 2024-09-12T03:17:34.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.597+0000 7f40ace42740 1 client.19338 shutdown 2024-09-12T03:17:34.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.598+0000 7f40ace42740 1 -- 172.21.15.116:0/2934740887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c133472b40 msgr2=0x55c133478fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:34.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.598+0000 7f40ace42740 1 --2- 172.21.15.116:0/2934740887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c133472b40 0x55c133478fc0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f409800bed0 tx=0x7f40980117b0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.598+0000 7f40ace42740 1 -- 172.21.15.116:0/2934740887 shutdown_connections 2024-09-12T03:17:34.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.598+0000 7f40ace42740 1 --2- 172.21.15.116:0/2934740887 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c13348c000 0x55c13348e460 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.598+0000 7f40ace42740 1 --2- 172.21.15.116:0/2934740887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c13347c370 0x55c133479560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.598+0000 7f40ace42740 1 --2- 172.21.15.116:0/2934740887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c133472b40 0x55c133478fc0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.598+0000 7f4083fff640 0 client.19338 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:17:34.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.598+0000 7f40ace42740 1 --2- 172.21.15.116:0/2934740887 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c1334715d0 0x55c13347bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.598+0000 7f40ace42740 1 -- 172.21.15.116:0/2934740887 >> 172.21.15.116:0/2934740887 conn(0x55c13343a310 msgr2=0x55c133473930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:34.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.598+0000 7f40ace42740 1 -- 172.21.15.116:0/2934740887 shutdown_connections 2024-09-12T03:17:34.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.598+0000 7f40ace42740 1 -- 172.21.15.116:0/2934740887 wait complete. 2024-09-12T03:17:34.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.598+0000 7f40ace42740 20 client.19338 trim_cache size 0 max 16384 2024-09-12T03:17:34.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.611+0000 7f40ace42740 1 Processor -- start 2024-09-12T03:17:34.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.611+0000 7f40ace42740 1 -- start start 2024-09-12T03:17:34.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.611+0000 7f40ace42740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c1334715d0 0x55c133478880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.611+0000 7f40ace42740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c133472b40 0x55c133474120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.611+0000 7f40ace42740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c13347c370 0x55c133474660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.611+0000 7f40ace42740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f40a41b1dc0 con 0x55c1334715d0 2024-09-12T03:17:34.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.611+0000 7f40ace42740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f40a41af690 con 0x55c133472b40 2024-09-12T03:17:34.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.611+0000 7f40ace42740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f40a41ad500 con 0x55c13347c370 2024-09-12T03:17:34.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.611+0000 7f40a3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c1334715d0 0x55c133478880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.611+0000 7f40a3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c1334715d0 0x55c133478880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42354/0 (socket says 172.21.15.116:42354) 2024-09-12T03:17:34.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.611+0000 7f40a3fff640 1 -- 172.21.15.116:0/2033539607 learned_addr learned my addr 172.21.15.116:0/2033539607 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:34.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.611+0000 7f40a3fff640 1 -- 172.21.15.116:0/2033539607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c13347c370 msgr2=0x55c133474660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:34.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.611+0000 7f40a3fff640 1 --2- 172.21.15.116:0/2033539607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c13347c370 0x55c133474660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.611+0000 7f40a3fff640 1 -- 172.21.15.116:0/2033539607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c133472b40 msgr2=0x55c133474120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:34.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.611+0000 7f40a8805640 1 --2- 172.21.15.116:0/2033539607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c133472b40 0x55c133474120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.611+0000 7f40a3fff640 1 --2- 172.21.15.116:0/2033539607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c133472b40 0x55c133474120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.611+0000 7f40a3fff640 1 -- 172.21.15.116:0/2033539607 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c133477f70 con 0x55c1334715d0 2024-09-12T03:17:34.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.611+0000 7f40a3fff640 1 --2- 172.21.15.116:0/2033539607 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c1334715d0 0x55c133478880 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f4090009900 tx=0x7f40900052a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0f52e57ddd3c8d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:34.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.612+0000 7f408e7fc640 1 -- 172.21.15.116:0/2033539607 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4090059070 con 0x55c1334715d0 2024-09-12T03:17:34.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.612+0000 7f408e7fc640 1 -- 172.21.15.116:0/2033539607 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4090044070 con 0x55c1334715d0 2024-09-12T03:17:34.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.612+0000 7f408e7fc640 1 -- 172.21.15.116:0/2033539607 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4090035040 con 0x55c1334715d0 2024-09-12T03:17:34.617 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.612+0000 7f40ace42740 1 -- 172.21.15.116:0/2033539607 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c1334715d0 msgr2=0x55c133478880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:34.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.612+0000 7f40ace42740 1 --2- 172.21.15.116:0/2033539607 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c1334715d0 0x55c133478880 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f4090009900 tx=0x7f40900052a0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.612+0000 7f40ace42740 1 -- 172.21.15.116:0/2033539607 shutdown_connections 2024-09-12T03:17:34.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.612+0000 7f40ace42740 1 --2- 172.21.15.116:0/2033539607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c13347c370 0x55c133474660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.612+0000 7f40ace42740 1 --2- 172.21.15.116:0/2033539607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c133472b40 0x55c133474120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.612+0000 7f40ace42740 1 --2- 172.21.15.116:0/2033539607 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c1334715d0 0x55c133478880 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.612+0000 7f40ace42740 1 -- 172.21.15.116:0/2033539607 >> 172.21.15.116:0/2033539607 conn(0x55c13343a310 msgr2=0x55c13347b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:34.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.612+0000 7f40ace42740 1 -- 172.21.15.116:0/2033539607 shutdown_connections 2024-09-12T03:17:34.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.612+0000 7f40ace42740 1 -- 172.21.15.116:0/2033539607 wait complete. 2024-09-12T03:17:34.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.612+0000 7f40ace42740 1 -- start start 2024-09-12T03:17:34.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.613+0000 7f40ace42740 1 Processor -- start 2024-09-12T03:17:34.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.613+0000 7f40ace42740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1334715d0 0x55c133484450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.613+0000 7f40ace42740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c133472b40 0x55c133484990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.613+0000 7f40ace42740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13347c370 0x55c133481420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.613+0000 7f40ace42740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f40a41adb50 con 0x55c133472b40 2024-09-12T03:17:34.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.613+0000 7f40ace42740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f40a41aefc0 con 0x55c13347c370 2024-09-12T03:17:34.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.613+0000 7f40ace42740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f40a41ae220 con 0x55c1334715d0 2024-09-12T03:17:34.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.613+0000 7f40ace42740 10 client.0 _ll_register_callbacks cb 0x7f409800b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:34.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.613+0000 7f40a8805640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c133472b40 0x55c133484990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.613+0000 7f40a8805640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c133472b40 0x55c133484990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42356/0 (socket says 172.21.15.116:42356) 2024-09-12T03:17:34.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.613+0000 7f40a8805640 1 -- 172.21.15.116:0/832242317 learned_addr learned my addr 172.21.15.116:0/832242317 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:34.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.613+0000 7f40a3fff640 1 --2- 172.21.15.116:0/832242317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1334715d0 0x55c133484450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.613+0000 7f40a8805640 1 -- 172.21.15.116:0/832242317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1334715d0 msgr2=0x55c133484450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:34.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.613+0000 7f40a8805640 1 --2- 172.21.15.116:0/832242317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1334715d0 0x55c133484450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.613+0000 7f40a8805640 1 -- 172.21.15.116:0/832242317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13347c370 msgr2=0x55c133481420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:34.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.613+0000 7f40a8805640 1 --2- 172.21.15.116:0/832242317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13347c370 0x55c133481420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.613+0000 7f40a8805640 1 -- 172.21.15.116:0/832242317 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c133481960 con 0x55c133472b40 2024-09-12T03:17:34.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.613+0000 7f40a3fff640 1 --2- 172.21.15.116:0/832242317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1334715d0 0x55c133484450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:34.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.614+0000 7f40a8805640 1 --2- 172.21.15.116:0/832242317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c133472b40 0x55c133484990 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x55c13347a600 tx=0x7f4098011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cfa628640cd16d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:34.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.614+0000 7f4083fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:34.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.614+0000 7f4083fff640 1 -- 172.21.15.116:0/832242317 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4098026d10 con 0x55c133472b40 2024-09-12T03:17:34.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.614+0000 7f4083fff640 1 -- 172.21.15.116:0/832242317 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f4098026620 con 0x55c133472b40 2024-09-12T03:17:34.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.614+0000 7f40ace42740 1 -- 172.21.15.116:0/832242317 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4090044070 con 0x55c133472b40 2024-09-12T03:17:34.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.614+0000 7f4083fff640 1 -- 172.21.15.116:0/832242317 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4098037070 con 0x55c133472b40 2024-09-12T03:17:34.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.614+0000 7f40ace42740 10 client.19344 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:34.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.614+0000 7f40ace42740 1 -- 172.21.15.116:0/832242317 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c1334f74d0 con 0x55c133472b40 2024-09-12T03:17:34.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.614+0000 7f4083fff640 1 -- 172.21.15.116:0/832242317 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4090044070 con 0x55c133472b40 2024-09-12T03:17:34.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.614+0000 7f4083fff640 1 -- 172.21.15.116:0/832242317 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4098005420 con 0x55c133472b40 2024-09-12T03:17:34.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.614+0000 7f40ace42740 10 client.19344 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:34.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.614+0000 7f40ace42740 10 client.19344 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:17:34.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.614+0000 7f40ace42740 20 client.19344 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:34.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.614+0000 7f40ace42740 1 --2- 172.21.15.116:0/832242317 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c1334f7c80 0x55c1334fa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:34.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.614+0000 7f40ace42740 4 client.19344 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:17:34.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.614+0000 7f40ace42740 1 -- 172.21.15.116:0/832242317 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c1333ded70 con 0x55c1334f7c80 2024-09-12T03:17:34.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.615+0000 7f40a3fff640 1 --2- 172.21.15.116:0/832242317 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c1334f7c80 0x55c1334fa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:34.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.615+0000 7f4083fff640 10 client.19344 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:17:34.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.615+0000 7f40a3fff640 1 --2- 172.21.15.116:0/832242317 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c1334f7c80 0x55c1334fa0a0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=72e6cd0351a1cb9e server_cookie=89dd259b842dca68 in_seq=0 out_seq=0 2024-09-12T03:17:34.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.615+0000 7f4083fff640 1 -- 172.21.15.116:0/832242317 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c1333ded70 con 0x55c1334f7c80 2024-09-12T03:17:34.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.615+0000 7f4083fff640 10 client.19344 handle_command_reply: tid=0 2024-09-12T03:17:34.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.615+0000 7f40ace42740 1 client.19344 shutdown 2024-09-12T03:17:34.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.615+0000 7f40ace42740 1 -- 172.21.15.116:0/832242317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c133472b40 msgr2=0x55c133484990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:34.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.615+0000 7f40ace42740 1 --2- 172.21.15.116:0/832242317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c133472b40 0x55c133484990 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x55c13347a600 tx=0x7f4098011df0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.615+0000 7f40ace42740 1 -- 172.21.15.116:0/832242317 shutdown_connections 2024-09-12T03:17:34.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.615+0000 7f40ace42740 1 --2- 172.21.15.116:0/832242317 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c1334f7c80 0x55c1334fa0a0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.615+0000 7f4083fff640 0 client.19344 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:17:34.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.615+0000 7f40ace42740 1 --2- 172.21.15.116:0/832242317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c13347c370 0x55c133481420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.615+0000 7f40ace42740 1 --2- 172.21.15.116:0/832242317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c133472b40 0x55c133484990 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.615+0000 7f40ace42740 1 --2- 172.21.15.116:0/832242317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1334715d0 0x55c133484450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.615+0000 7f40ace42740 1 -- 172.21.15.116:0/832242317 >> 172.21.15.116:0/832242317 conn(0x55c13343a310 msgr2=0x55c13346c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:34.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.615+0000 7f40ace42740 1 -- 172.21.15.116:0/832242317 shutdown_connections 2024-09-12T03:17:34.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.615+0000 7f40ace42740 1 -- 172.21.15.116:0/832242317 wait complete. 2024-09-12T03:17:34.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.616+0000 7f40ace42740 20 client.19344 trim_cache size 0 max 16384 2024-09-12T03:17:34.622 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:17:34.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.616+0000 7f40aaa69640 1 -- 172.21.15.116:0/2038570963 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f40940a2b80 msgr2=0x7f40940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:34.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.616+0000 7f40aaa69640 1 --2- 172.21.15.116:0/2038570963 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f40940a2b80 0x7f40940a5040 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f409c0cd2e0 tx=0x7f409c0c6b80 comp rx=0 tx=0).stop 2024-09-12T03:17:34.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.616+0000 7f40aaa69640 1 -- 172.21.15.116:0/2038570963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40a4106f10 msgr2=0x7f40a4074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:34.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.616+0000 7f40aaa69640 1 --2- 172.21.15.116:0/2038570963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40a4106f10 0x7f40a4074f70 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f409c00cce0 tx=0x7f409c00a620 comp rx=0 tx=0).stop 2024-09-12T03:17:34.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.616+0000 7f40aaa69640 1 -- 172.21.15.116:0/2038570963 shutdown_connections 2024-09-12T03:17:34.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.616+0000 7f40aaa69640 1 --2- 172.21.15.116:0/2038570963 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f40940a2b80 0x7f40940a5040 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.616+0000 7f40aaa69640 1 --2- 172.21.15.116:0/2038570963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40a4106f10 0x7f40a4074f70 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.616+0000 7f40aaa69640 1 --2- 172.21.15.116:0/2038570963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40a4102790 0x7f40a4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.616+0000 7f40aaa69640 1 --2- 172.21.15.116:0/2038570963 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40a4101de0 0x7f40a40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:34.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.616+0000 7f40aaa69640 1 -- 172.21.15.116:0/2038570963 >> 172.21.15.116:0/2038570963 conn(0x7f40a40fd8c0 msgr2=0x7f40a40fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:34.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.616+0000 7f40aaa69640 1 -- 172.21.15.116:0/2038570963 shutdown_connections 2024-09-12T03:17:34.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:34.616+0000 7f40aaa69640 1 -- 172.21.15.116:0/2038570963 wait complete. 2024-09-12T03:17:35.282 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:17:35.530 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:35.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:35 smithi116 ceph-mon[31410]: pgmap v776: 97 pgs: 97 active+clean; 6.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-09-12T03:17:35.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:35 smithi177 ceph-mon[35305]: pgmap v776: 97 pgs: 97 active+clean; 6.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-09-12T03:17:35.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:35 smithi183 ceph-mon[35535]: pgmap v776: 97 pgs: 97 active+clean; 6.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-09-12T03:17:36.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.039+0000 7f7f3be71640 1 -- 172.21.15.116:0/1713115234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f34106f10 msgr2=0x7f7f3410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:36.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.039+0000 7f7f3be71640 1 --2- 172.21.15.116:0/1713115234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f34106f10 0x7f7f3410ea90 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009900 tx=0x7f7f3002a410 comp rx=0 tx=0).stop 2024-09-12T03:17:36.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.039+0000 7f7f3be71640 1 -- 172.21.15.116:0/1713115234 shutdown_connections 2024-09-12T03:17:36.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.039+0000 7f7f3be71640 1 --2- 172.21.15.116:0/1713115234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f34106f10 0x7f7f3410ea90 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.039+0000 7f7f3be71640 1 --2- 172.21.15.116:0/1713115234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f34102790 0x7f7f341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.039+0000 7f7f3be71640 1 --2- 172.21.15.116:0/1713115234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f34101de0 0x7f7f341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.039+0000 7f7f3be71640 1 -- 172.21.15.116:0/1713115234 >> 172.21.15.116:0/1713115234 conn(0x7f7f340fd8c0 msgr2=0x7f7f340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:36.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.039+0000 7f7f3be71640 1 -- 172.21.15.116:0/1713115234 shutdown_connections 2024-09-12T03:17:36.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.039+0000 7f7f3be71640 1 -- 172.21.15.116:0/1713115234 wait complete. 2024-09-12T03:17:36.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.039+0000 7f7f3be71640 1 Processor -- start 2024-09-12T03:17:36.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.039+0000 7f7f3be71640 1 -- start start 2024-09-12T03:17:36.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.039+0000 7f7f3be71640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f34101de0 0x7f7f341983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.039+0000 7f7f3be71640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f34102790 0x7f7f34198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.039+0000 7f7f3be71640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f34106f10 0x7f7f3419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.039+0000 7f7f3be71640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7f34111f70 con 0x7f7f34101de0 2024-09-12T03:17:36.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.039+0000 7f7f3be71640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7f34111df0 con 0x7f7f34102790 2024-09-12T03:17:36.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.039+0000 7f7f3be71640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7f341120f0 con 0x7f7f34106f10 2024-09-12T03:17:36.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.040+0000 7f7f39c0d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f34101de0 0x7f7f341983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.040+0000 7f7f39c0d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f34101de0 0x7f7f341983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42380/0 (socket says 172.21.15.116:42380) 2024-09-12T03:17:36.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.040+0000 7f7f39c0d640 1 -- 172.21.15.116:0/20851881 learned_addr learned my addr 172.21.15.116:0/20851881 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:36.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.040+0000 7f7f3940c640 1 --2- 172.21.15.116:0/20851881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f34102790 0x7f7f34198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.040+0000 7f7f3a40e640 1 --2- 172.21.15.116:0/20851881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f34106f10 0x7f7f3419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.040+0000 7f7f39c0d640 1 -- 172.21.15.116:0/20851881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f34106f10 msgr2=0x7f7f3419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:36.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.040+0000 7f7f39c0d640 1 --2- 172.21.15.116:0/20851881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f34106f10 0x7f7f3419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.040+0000 7f7f39c0d640 1 -- 172.21.15.116:0/20851881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f34102790 msgr2=0x7f7f34198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:36.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.040+0000 7f7f39c0d640 1 --2- 172.21.15.116:0/20851881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f34102790 0x7f7f34198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.040+0000 7f7f39c0d640 1 -- 172.21.15.116:0/20851881 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3419d3a0 con 0x7f7f34101de0 2024-09-12T03:17:36.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.040+0000 7f7f39c0d640 1 --2- 172.21.15.116:0/20851881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f34101de0 0x7f7f341983f0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c00ebc0 tx=0x7f7f2c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:36.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.040+0000 7f7f2affd640 1 -- 172.21.15.116:0/20851881 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c010630 con 0x7f7f34101de0 2024-09-12T03:17:36.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.040+0000 7f7f3be71640 1 -- 172.21.15.116:0/20851881 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f3419d690 con 0x7f7f34101de0 2024-09-12T03:17:36.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.040+0000 7f7f2affd640 1 -- 172.21.15.116:0/20851881 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f2c010cb0 con 0x7f7f34101de0 2024-09-12T03:17:36.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.040+0000 7f7f2affd640 1 -- 172.21.15.116:0/20851881 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c018bd0 con 0x7f7f34101de0 2024-09-12T03:17:36.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.040+0000 7f7f3be71640 1 -- 172.21.15.116:0/20851881 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f341a4f70 con 0x7f7f34101de0 2024-09-12T03:17:36.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.042+0000 7f7f2affd640 1 -- 172.21.15.116:0/20851881 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f7f2c0107d0 con 0x7f7f34101de0 2024-09-12T03:17:36.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.043+0000 7f7f3e24a740 1 Processor -- start 2024-09-12T03:17:36.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.043+0000 7f7f3e24a740 1 -- start start 2024-09-12T03:17:36.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.043+0000 7f7f2affd640 1 --2- 172.21.15.116:0/20851881 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7f200a2b80 0x7f7f200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.043+0000 7f7f3e24a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556d09acfb40 0x556d09ace090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.043+0000 7f7f3e24a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d09ace5d0 0x556d09acea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.043+0000 7f7f3e24a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d09ad9370 0x556d09adb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.043+0000 7f7f3e24a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556d09a342f0 con 0x556d09acfb40 2024-09-12T03:17:36.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.043+0000 7f7f3e24a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556d099dbcf0 con 0x556d09ace5d0 2024-09-12T03:17:36.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.043+0000 7f7f3e24a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556d09a302b0 con 0x556d09ad9370 2024-09-12T03:17:36.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.043+0000 7f7f39c0d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556d09acfb40 0x556d09ace090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.043+0000 7f7f3a40e640 1 --2- 172.21.15.116:0/20851881 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7f200a2b80 0x7f7f200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.044+0000 7f7f39c0d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556d09acfb40 0x556d09ace090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42384/0 (socket says 172.21.15.116:42384) 2024-09-12T03:17:36.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.044+0000 7f7f39c0d640 1 -- 172.21.15.116:0/3287389764 learned_addr learned my addr 172.21.15.116:0/3287389764 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:36.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.044+0000 7f7f2affd640 1 -- 172.21.15.116:0/20851881 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f7f2c0cf6e0 con 0x7f7f34101de0 2024-09-12T03:17:36.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.044+0000 7f7f3940c640 1 --2- 172.21.15.116:0/3287389764 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d09ace5d0 0x556d09acea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.044+0000 7f7f3a40e640 1 --2- 172.21.15.116:0/3287389764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d09ad9370 0x556d09adb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.044+0000 7f7f39c0d640 1 -- 172.21.15.116:0/3287389764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d09ad9370 msgr2=0x556d09adb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:36.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.044+0000 7f7f39c0d640 1 --2- 172.21.15.116:0/3287389764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d09ad9370 0x556d09adb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.044+0000 7f7f39c0d640 1 -- 172.21.15.116:0/3287389764 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d09ace5d0 msgr2=0x556d09acea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:36.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.044+0000 7f7f39c0d640 1 --2- 172.21.15.116:0/3287389764 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d09ace5d0 0x556d09acea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.044+0000 7f7f39c0d640 1 -- 172.21.15.116:0/3287389764 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d099c03a0 con 0x556d09acfb40 2024-09-12T03:17:36.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.044+0000 7f7f3a40e640 1 --2- 172.21.15.116:0/20851881 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7f200a2b80 0x7f7f200a5040 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7f300096c0 tx=0x7f7f30032f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:36.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.044+0000 7f7f39c0d640 1 --2- 172.21.15.116:0/3287389764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556d09acfb40 0x556d09ace090 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c01dc10 tx=0x7f7f2c025800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65bb8cdce14d3dd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:36.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.044+0000 7f7f1ffff640 1 -- 172.21.15.116:0/3287389764 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c010e60 con 0x556d09acfb40 2024-09-12T03:17:36.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.044+0000 7f7f1ffff640 1 -- 172.21.15.116:0/3287389764 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f2c01dcb0 con 0x556d09acfb40 2024-09-12T03:17:36.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.044+0000 7f7f1ffff640 1 -- 172.21.15.116:0/3287389764 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c0a0320 con 0x556d09acfb40 2024-09-12T03:17:36.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.045+0000 7f7f3e24a740 1 -- 172.21.15.116:0/3287389764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556d09acfb40 msgr2=0x556d09ace090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:36.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.045+0000 7f7f3e24a740 1 --2- 172.21.15.116:0/3287389764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556d09acfb40 0x556d09ace090 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c01dc10 tx=0x7f7f2c025800 comp rx=0 tx=0).stop 2024-09-12T03:17:36.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.045+0000 7f7f3e24a740 1 -- 172.21.15.116:0/3287389764 shutdown_connections 2024-09-12T03:17:36.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.045+0000 7f7f3e24a740 1 --2- 172.21.15.116:0/3287389764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d09ad9370 0x556d09adb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.045+0000 7f7f3e24a740 1 --2- 172.21.15.116:0/3287389764 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d09ace5d0 0x556d09acea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.045+0000 7f7f3e24a740 1 --2- 172.21.15.116:0/3287389764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556d09acfb40 0x556d09ace090 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.045+0000 7f7f3e24a740 1 -- 172.21.15.116:0/3287389764 >> 172.21.15.116:0/3287389764 conn(0x556d09a97310 msgr2=0x556d09acac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:36.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.045+0000 7f7f3e24a740 1 -- 172.21.15.116:0/3287389764 shutdown_connections 2024-09-12T03:17:36.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.045+0000 7f7f3e24a740 1 -- 172.21.15.116:0/3287389764 wait complete. 2024-09-12T03:17:36.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.045+0000 7f7f3e24a740 1 -- start start 2024-09-12T03:17:36.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.046+0000 7f7f3e24a740 1 Processor -- start 2024-09-12T03:17:36.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.046+0000 7f7f3e24a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556d09ace5d0 0x556d09ad8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.046+0000 7f7f3e24a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d09acfb40 0x556d09ad6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.046+0000 7f7f3e24a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d09ad9370 0x556d09ad6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.046+0000 7f7f3e24a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556d09adc070 con 0x556d09ace5d0 2024-09-12T03:17:36.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.046+0000 7f7f3e24a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556d099ef060 con 0x556d09acfb40 2024-09-12T03:17:36.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.046+0000 7f7f3e24a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556d09adc1f0 con 0x556d09ad9370 2024-09-12T03:17:36.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.046+0000 7f7f3e24a740 10 client.0 _ll_register_callbacks cb 0x556d097428c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:36.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.046+0000 7f7f39c0d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556d09ace5d0 0x556d09ad8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.046+0000 7f7f39c0d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556d09ace5d0 0x556d09ad8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42386/0 (socket says 172.21.15.116:42386) 2024-09-12T03:17:36.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.046+0000 7f7f39c0d640 1 -- 172.21.15.116:0/2688265181 learned_addr learned my addr 172.21.15.116:0/2688265181 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:36.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.046+0000 7f7f3a40e640 1 --2- 172.21.15.116:0/2688265181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d09ad9370 0x556d09ad6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.046+0000 7f7f3940c640 1 --2- 172.21.15.116:0/2688265181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d09acfb40 0x556d09ad6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.046+0000 7f7f39c0d640 1 -- 172.21.15.116:0/2688265181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d09ad9370 msgr2=0x556d09ad6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:36.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.046+0000 7f7f39c0d640 1 --2- 172.21.15.116:0/2688265181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d09ad9370 0x556d09ad6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.046+0000 7f7f39c0d640 1 -- 172.21.15.116:0/2688265181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d09acfb40 msgr2=0x556d09ad6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:36.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.046+0000 7f7f39c0d640 1 --2- 172.21.15.116:0/2688265181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d09acfb40 0x556d09ad6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.046+0000 7f7f39c0d640 1 -- 172.21.15.116:0/2688265181 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d097a53c0 con 0x556d09ace5d0 2024-09-12T03:17:36.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.047+0000 7f7f39c0d640 1 --2- 172.21.15.116:0/2688265181 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556d09ace5d0 0x556d09ad8d60 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c0983d0 tx=0x7f7f2c018670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fc6db05b1bd06f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:36.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.047+0000 7f7f1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:36.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.047+0000 7f7f1d7fa640 1 -- 172.21.15.116:0/2688265181 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c0b6060 con 0x556d09ace5d0 2024-09-12T03:17:36.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.047+0000 7f7f1d7fa640 1 -- 172.21.15.116:0/2688265181 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f7f2c09a270 con 0x556d09ace5d0 2024-09-12T03:17:36.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.047+0000 7f7f3e24a740 1 -- 172.21.15.116:0/2688265181 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556d09a41860 con 0x556d09ace5d0 2024-09-12T03:17:36.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.047+0000 7f7f1d7fa640 1 -- 172.21.15.116:0/2688265181 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7f2c010e60 con 0x556d09ace5d0 2024-09-12T03:17:36.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.047+0000 7f7f3e24a740 10 client.19356 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:36.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.047+0000 7f7f3e24a740 1 -- 172.21.15.116:0/2688265181 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556d09ae7b40 con 0x556d09ace5d0 2024-09-12T03:17:36.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.047+0000 7f7f1d7fa640 1 -- 172.21.15.116:0/2688265181 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556d09a41860 con 0x556d09ace5d0 2024-09-12T03:17:36.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.047+0000 7f7f1d7fa640 1 -- 172.21.15.116:0/2688265181 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7f2c059070 con 0x556d09ace5d0 2024-09-12T03:17:36.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.047+0000 7f7f3e24a740 10 client.19356 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:36.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.047+0000 7f7f3e24a740 10 client.19356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:17:36.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.047+0000 7f7f3e24a740 20 client.19356 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:36.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.047+0000 7f7f3e24a740 1 --2- 172.21.15.116:0/2688265181 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556d09ae8e90 0x556d09aeb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.047+0000 7f7f3e24a740 4 client.19356 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:17:36.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.047+0000 7f7f3e24a740 1 -- 172.21.15.116:0/2688265181 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d09a3bd70 con 0x556d09ae8e90 2024-09-12T03:17:36.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.048+0000 7f7f3940c640 1 --2- 172.21.15.116:0/2688265181 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556d09ae8e90 0x556d09aeb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.048+0000 7f7f1d7fa640 10 client.19356 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:36.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.048+0000 7f7f3940c640 1 --2- 172.21.15.116:0/2688265181 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556d09ae8e90 0x556d09aeb280 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9fb7888b16262b0 server_cookie=24e785c0a45ed44c in_seq=0 out_seq=0 2024-09-12T03:17:36.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.049+0000 7f7f1d7fa640 1 -- 172.21.15.116:0/2688265181 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556d09a3bd70 con 0x556d09ae8e90 2024-09-12T03:17:36.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.049+0000 7f7f1d7fa640 10 client.19356 handle_command_reply: tid=0 2024-09-12T03:17:36.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.049+0000 7f7f3e24a740 1 client.19356 shutdown 2024-09-12T03:17:36.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.049+0000 7f7f3e24a740 1 -- 172.21.15.116:0/2688265181 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556d09ace5d0 msgr2=0x556d09ad8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:36.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.049+0000 7f7f3e24a740 1 --2- 172.21.15.116:0/2688265181 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556d09ace5d0 0x556d09ad8d60 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c0983d0 tx=0x7f7f2c018670 comp rx=0 tx=0).stop 2024-09-12T03:17:36.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.049+0000 7f7f3e24a740 1 -- 172.21.15.116:0/2688265181 shutdown_connections 2024-09-12T03:17:36.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.049+0000 7f7f3e24a740 1 --2- 172.21.15.116:0/2688265181 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556d09ae8e90 0x556d09aeb280 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.049+0000 7f7f3e24a740 1 --2- 172.21.15.116:0/2688265181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d09ad9370 0x556d09ad6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.049+0000 7f7f1d7fa640 0 client.19356 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:36.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.049+0000 7f7f3e24a740 1 --2- 172.21.15.116:0/2688265181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d09acfb40 0x556d09ad6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.049+0000 7f7f3e24a740 1 --2- 172.21.15.116:0/2688265181 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556d09ace5d0 0x556d09ad8d60 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.049+0000 7f7f3e24a740 1 -- 172.21.15.116:0/2688265181 >> 172.21.15.116:0/2688265181 conn(0x556d09a97310 msgr2=0x556d09ad0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:36.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.049+0000 7f7f3e24a740 1 -- 172.21.15.116:0/2688265181 shutdown_connections 2024-09-12T03:17:36.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.049+0000 7f7f3e24a740 1 -- 172.21.15.116:0/2688265181 wait complete. 2024-09-12T03:17:36.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.049+0000 7f7f3e24a740 20 client.19356 trim_cache size 0 max 16384 2024-09-12T03:17:36.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.062+0000 7f7f3e24a740 1 Processor -- start 2024-09-12T03:17:36.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.062+0000 7f7f3e24a740 1 -- start start 2024-09-12T03:17:36.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.063+0000 7f7f3e24a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556d09ace5d0 0x556d09ae5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.063+0000 7f7f3e24a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d09acfb40 0x556d09ad1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.063+0000 7f7f3e24a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d09ad9370 0x556d09ad1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.063+0000 7f7f3e24a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7f341afda0 con 0x556d09ace5d0 2024-09-12T03:17:36.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.063+0000 7f7f3e24a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7f341ad670 con 0x556d09ad9370 2024-09-12T03:17:36.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.063+0000 7f7f3e24a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7f341ab4c0 con 0x556d09acfb40 2024-09-12T03:17:36.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.063+0000 7f7f39c0d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556d09ace5d0 0x556d09ae5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.063+0000 7f7f39c0d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556d09ace5d0 0x556d09ae5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42396/0 (socket says 172.21.15.116:42396) 2024-09-12T03:17:36.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.063+0000 7f7f39c0d640 1 -- 172.21.15.116:0/3605928056 learned_addr learned my addr 172.21.15.116:0/3605928056 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:36.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.063+0000 7f7f3a40e640 1 --2- 172.21.15.116:0/3605928056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d09ad9370 0x556d09ad1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.063+0000 7f7f3940c640 1 --2- 172.21.15.116:0/3605928056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d09acfb40 0x556d09ad1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.063+0000 7f7f39c0d640 1 -- 172.21.15.116:0/3605928056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d09acfb40 msgr2=0x556d09ad1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:36.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.063+0000 7f7f39c0d640 1 --2- 172.21.15.116:0/3605928056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d09acfb40 0x556d09ad1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.063+0000 7f7f39c0d640 1 -- 172.21.15.116:0/3605928056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d09ad9370 msgr2=0x556d09ad1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:36.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.063+0000 7f7f39c0d640 1 --2- 172.21.15.116:0/3605928056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d09ad9370 0x556d09ad1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.063+0000 7f7f39c0d640 1 -- 172.21.15.116:0/3605928056 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d09ad4560 con 0x556d09ace5d0 2024-09-12T03:17:36.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.063+0000 7f7f3a40e640 1 --2- 172.21.15.116:0/3605928056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d09ad9370 0x556d09ad1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:36.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.063+0000 7f7f39c0d640 1 --2- 172.21.15.116:0/3605928056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556d09ace5d0 0x556d09ae5340 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c0b4660 tx=0x7f7f2c00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eead833ec854f42c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:36.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.063+0000 7f7f1f7fe640 1 -- 172.21.15.116:0/3605928056 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c05f070 con 0x556d09ace5d0 2024-09-12T03:17:36.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.063+0000 7f7f1f7fe640 1 -- 172.21.15.116:0/3605928056 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f2c0a4be0 con 0x556d09ace5d0 2024-09-12T03:17:36.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.063+0000 7f7f1f7fe640 1 -- 172.21.15.116:0/3605928056 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c0b6030 con 0x556d09ace5d0 2024-09-12T03:17:36.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.064+0000 7f7f3e24a740 1 -- 172.21.15.116:0/3605928056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556d09ace5d0 msgr2=0x556d09ae5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:36.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.064+0000 7f7f3e24a740 1 --2- 172.21.15.116:0/3605928056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556d09ace5d0 0x556d09ae5340 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c0b4660 tx=0x7f7f2c00ccd0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.064+0000 7f7f3e24a740 1 -- 172.21.15.116:0/3605928056 shutdown_connections 2024-09-12T03:17:36.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.064+0000 7f7f3e24a740 1 --2- 172.21.15.116:0/3605928056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d09ad9370 0x556d09ad1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.064+0000 7f7f3e24a740 1 --2- 172.21.15.116:0/3605928056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d09acfb40 0x556d09ad1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.064+0000 7f7f3e24a740 1 --2- 172.21.15.116:0/3605928056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556d09ace5d0 0x556d09ae5340 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.064+0000 7f7f3e24a740 1 -- 172.21.15.116:0/3605928056 >> 172.21.15.116:0/3605928056 conn(0x556d09a97310 msgr2=0x556d09ad5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:36.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.064+0000 7f7f3e24a740 1 -- 172.21.15.116:0/3605928056 shutdown_connections 2024-09-12T03:17:36.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.064+0000 7f7f3e24a740 1 -- 172.21.15.116:0/3605928056 wait complete. 2024-09-12T03:17:36.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.064+0000 7f7f3e24a740 1 -- start start 2024-09-12T03:17:36.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.065+0000 7f7f3e24a740 1 Processor -- start 2024-09-12T03:17:36.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.065+0000 7f7f3e24a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556d09ace5d0 0x556d09ae17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.065+0000 7f7f3e24a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d09acfb40 0x556d09ae1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.065+0000 7f7f3e24a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d09ad9370 0x556d09ad2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.065+0000 7f7f3e24a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7f341abb30 con 0x556d09ace5d0 2024-09-12T03:17:36.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.065+0000 7f7f3e24a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7f341acfa0 con 0x556d09ad9370 2024-09-12T03:17:36.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.065+0000 7f7f3e24a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7f341ac200 con 0x556d09acfb40 2024-09-12T03:17:36.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.065+0000 7f7f3e24a740 10 client.0 _ll_register_callbacks cb 0x7f7f24009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:36.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.065+0000 7f7f39c0d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556d09ace5d0 0x556d09ae17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.065+0000 7f7f39c0d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556d09ace5d0 0x556d09ae17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42402/0 (socket says 172.21.15.116:42402) 2024-09-12T03:17:36.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.065+0000 7f7f39c0d640 1 -- 172.21.15.116:0/1022856489 learned_addr learned my addr 172.21.15.116:0/1022856489 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:36.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.065+0000 7f7f3940c640 1 --2- 172.21.15.116:0/1022856489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d09acfb40 0x556d09ae1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.065+0000 7f7f3a40e640 1 --2- 172.21.15.116:0/1022856489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d09ad9370 0x556d09ad2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.065+0000 7f7f39c0d640 1 -- 172.21.15.116:0/1022856489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d09acfb40 msgr2=0x556d09ae1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:36.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.065+0000 7f7f39c0d640 1 --2- 172.21.15.116:0/1022856489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d09acfb40 0x556d09ae1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.065+0000 7f7f39c0d640 1 -- 172.21.15.116:0/1022856489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d09ad9370 msgr2=0x556d09ad2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:36.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.065+0000 7f7f39c0d640 1 --2- 172.21.15.116:0/1022856489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d09ad9370 0x556d09ad2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.065+0000 7f7f39c0d640 1 -- 172.21.15.116:0/1022856489 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d09ad3250 con 0x556d09ace5d0 2024-09-12T03:17:36.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.065+0000 7f7f3940c640 1 --2- 172.21.15.116:0/1022856489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d09acfb40 0x556d09ae1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:36.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.065+0000 7f7f3a40e640 1 --2- 172.21.15.116:0/1022856489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d09ad9370 0x556d09ad2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:36.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.066+0000 7f7f39c0d640 1 --2- 172.21.15.116:0/1022856489 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556d09ace5d0 0x556d09ae17d0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c0b43e0 tx=0x7f7f2c00cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f349f4e32bd02eb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:36.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.066+0000 7f7f1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:36.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.066+0000 7f7f1d7fa640 1 -- 172.21.15.116:0/1022856489 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c0b6070 con 0x556d09ace5d0 2024-09-12T03:17:36.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.066+0000 7f7f1d7fa640 1 -- 172.21.15.116:0/1022856489 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f7f2c0b4170 con 0x556d09ace5d0 2024-09-12T03:17:36.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.066+0000 7f7f3e24a740 1 -- 172.21.15.116:0/1022856489 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f2c0a4be0 con 0x556d09ace5d0 2024-09-12T03:17:36.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.066+0000 7f7f1d7fa640 1 -- 172.21.15.116:0/1022856489 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7f2c05f070 con 0x556d09ace5d0 2024-09-12T03:17:36.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.066+0000 7f7f3e24a740 10 client.19362 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:36.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.066+0000 7f7f3e24a740 1 -- 172.21.15.116:0/1022856489 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556d09ad3250 con 0x556d09ace5d0 2024-09-12T03:17:36.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.066+0000 7f7f1d7fa640 1 -- 172.21.15.116:0/1022856489 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f2c0a4be0 con 0x556d09ace5d0 2024-09-12T03:17:36.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.066+0000 7f7f1d7fa640 1 -- 172.21.15.116:0/1022856489 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7f2c059070 con 0x556d09ace5d0 2024-09-12T03:17:36.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.066+0000 7f7f3e24a740 10 client.19362 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:36.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.066+0000 7f7f3e24a740 10 client.19362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:17:36.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.066+0000 7f7f3e24a740 20 client.19362 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:36.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.066+0000 7f7f3e24a740 1 --2- 172.21.15.116:0/1022856489 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556d09b57a90 0x556d09b59e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:36.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.066+0000 7f7f3e24a740 4 client.19362 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:17:36.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.066+0000 7f7f3e24a740 1 -- 172.21.15.116:0/1022856489 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556d099b8170 con 0x556d09b57a90 2024-09-12T03:17:36.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.067+0000 7f7f3940c640 1 --2- 172.21.15.116:0/1022856489 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556d09b57a90 0x556d09b59e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:36.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.067+0000 7f7f1d7fa640 10 client.19362 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:36.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.067+0000 7f7f3940c640 1 --2- 172.21.15.116:0/1022856489 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556d09b57a90 0x556d09b59e80 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd5ffea4fd116dd9 server_cookie=9f537c28c6ad4dd3 in_seq=0 out_seq=0 2024-09-12T03:17:36.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.067+0000 7f7f1d7fa640 1 -- 172.21.15.116:0/1022856489 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556d099b8170 con 0x556d09b57a90 2024-09-12T03:17:36.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.067+0000 7f7f1d7fa640 10 client.19362 handle_command_reply: tid=0 2024-09-12T03:17:36.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.067+0000 7f7f3e24a740 1 client.19362 shutdown 2024-09-12T03:17:36.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.067+0000 7f7f3e24a740 1 -- 172.21.15.116:0/1022856489 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556d09ace5d0 msgr2=0x556d09ae17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:36.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.067+0000 7f7f3e24a740 1 --2- 172.21.15.116:0/1022856489 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556d09ace5d0 0x556d09ae17d0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c0b43e0 tx=0x7f7f2c00cd40 comp rx=0 tx=0).stop 2024-09-12T03:17:36.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.068+0000 7f7f3e24a740 1 -- 172.21.15.116:0/1022856489 shutdown_connections 2024-09-12T03:17:36.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.068+0000 7f7f3e24a740 1 --2- 172.21.15.116:0/1022856489 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x556d09b57a90 0x556d09b59e80 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.068+0000 7f7f3e24a740 1 --2- 172.21.15.116:0/1022856489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d09ad9370 0x556d09ad2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.068+0000 7f7f3e24a740 1 --2- 172.21.15.116:0/1022856489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d09acfb40 0x556d09ae1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.068+0000 7f7f1d7fa640 0 client.19362 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:36.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.068+0000 7f7f3e24a740 1 --2- 172.21.15.116:0/1022856489 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556d09ace5d0 0x556d09ae17d0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.068+0000 7f7f3e24a740 1 -- 172.21.15.116:0/1022856489 >> 172.21.15.116:0/1022856489 conn(0x556d09a97310 msgr2=0x556d09ad7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:36.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.068+0000 7f7f3e24a740 1 -- 172.21.15.116:0/1022856489 shutdown_connections 2024-09-12T03:17:36.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.068+0000 7f7f3e24a740 1 -- 172.21.15.116:0/1022856489 wait complete. 2024-09-12T03:17:36.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.068+0000 7f7f3e24a740 20 client.19362 trim_cache size 0 max 16384 2024-09-12T03:17:36.080 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:17:36.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.068+0000 7f7f3be71640 1 -- 172.21.15.116:0/20851881 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7f200a2b80 msgr2=0x7f7f200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:36.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.068+0000 7f7f3be71640 1 --2- 172.21.15.116:0/20851881 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7f200a2b80 0x7f7f200a5040 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7f300096c0 tx=0x7f7f30032f40 comp rx=0 tx=0).stop 2024-09-12T03:17:36.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.068+0000 7f7f3be71640 1 -- 172.21.15.116:0/20851881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f34101de0 msgr2=0x7f7f341983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:36.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.068+0000 7f7f3be71640 1 --2- 172.21.15.116:0/20851881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f34101de0 0x7f7f341983f0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c00ebc0 tx=0x7f7f2c00c6a0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.068+0000 7f7f3be71640 1 -- 172.21.15.116:0/20851881 shutdown_connections 2024-09-12T03:17:36.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.068+0000 7f7f3be71640 1 --2- 172.21.15.116:0/20851881 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7f200a2b80 0x7f7f200a5040 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.068+0000 7f7f3be71640 1 --2- 172.21.15.116:0/20851881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f34106f10 0x7f7f3419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.068+0000 7f7f3be71640 1 --2- 172.21.15.116:0/20851881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f34102790 0x7f7f34198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.068+0000 7f7f3be71640 1 --2- 172.21.15.116:0/20851881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f34101de0 0x7f7f341983f0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:36.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.068+0000 7f7f3be71640 1 -- 172.21.15.116:0/20851881 >> 172.21.15.116:0/20851881 conn(0x7f7f340fd8c0 msgr2=0x7f7f340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:36.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.069+0000 7f7f3be71640 1 -- 172.21.15.116:0/20851881 shutdown_connections 2024-09-12T03:17:36.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:36.069+0000 7f7f3be71640 1 -- 172.21.15.116:0/20851881 wait complete. 2024-09-12T03:17:37.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:37 smithi183 ceph-mon[35535]: pgmap v777: 97 pgs: 97 active+clean; 6.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-09-12T03:17:37.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:37 smithi177 ceph-mon[35305]: pgmap v777: 97 pgs: 97 active+clean; 6.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-09-12T03:17:37.781 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:17:37.781 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:17:37.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:37 smithi116 ceph-mon[31410]: pgmap v777: 97 pgs: 97 active+clean; 6.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2024-09-12T03:17:38.041 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:38.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.548+0000 7f1e9961b640 1 -- 172.21.15.116:0/1987922669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e94106f20 msgr2=0x7f1e9410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:38.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.548+0000 7f1e9961b640 1 --2- 172.21.15.116:0/1987922669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e94106f20 0x7f1e9410eaa0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c009900 tx=0x7f1e8c02a410 comp rx=0 tx=0).stop 2024-09-12T03:17:38.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.548+0000 7f1e9961b640 1 -- 172.21.15.116:0/1987922669 shutdown_connections 2024-09-12T03:17:38.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.548+0000 7f1e9961b640 1 --2- 172.21.15.116:0/1987922669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e94106f20 0x7f1e9410eaa0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.548+0000 7f1e9961b640 1 --2- 172.21.15.116:0/1987922669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e941027a0 0x7f1e941067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.548+0000 7f1e9961b640 1 --2- 172.21.15.116:0/1987922669 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e94101df0 0x7f1e941021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.548+0000 7f1e9961b640 1 -- 172.21.15.116:0/1987922669 >> 172.21.15.116:0/1987922669 conn(0x7f1e940fd8f0 msgr2=0x7f1e940ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:38.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.548+0000 7f1e9961b640 1 -- 172.21.15.116:0/1987922669 shutdown_connections 2024-09-12T03:17:38.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.548+0000 7f1e9961b640 1 -- 172.21.15.116:0/1987922669 wait complete. 2024-09-12T03:17:38.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.548+0000 7f1e9961b640 1 Processor -- start 2024-09-12T03:17:38.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.549+0000 7f1e9961b640 1 -- start start 2024-09-12T03:17:38.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.549+0000 7f1e9961b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e94101df0 0x7f1e941983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.549+0000 7f1e9961b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e941027a0 0x7f1e94198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.549+0000 7f1e9961b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e94106f20 0x7f1e9419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.549+0000 7f1e9961b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1e94111fa0 con 0x7f1e94101df0 2024-09-12T03:17:38.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.549+0000 7f1e9961b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1e94111e20 con 0x7f1e941027a0 2024-09-12T03:17:38.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.549+0000 7f1e9961b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1e94112120 con 0x7f1e94106f20 2024-09-12T03:17:38.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.549+0000 7f1e92ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e94101df0 0x7f1e941983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.549+0000 7f1e92ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e94101df0 0x7f1e941983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42434/0 (socket says 172.21.15.116:42434) 2024-09-12T03:17:38.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.549+0000 7f1e92ffd640 1 -- 172.21.15.116:0/3214561507 learned_addr learned my addr 172.21.15.116:0/3214561507 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:38.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.549+0000 7f1e937fe640 1 --2- 172.21.15.116:0/3214561507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e94106f20 0x7f1e9419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.549+0000 7f1e927fc640 1 --2- 172.21.15.116:0/3214561507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e941027a0 0x7f1e94198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.549+0000 7f1e92ffd640 1 -- 172.21.15.116:0/3214561507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e94106f20 msgr2=0x7f1e9419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:38.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.549+0000 7f1e92ffd640 1 --2- 172.21.15.116:0/3214561507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e94106f20 0x7f1e9419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.549+0000 7f1e92ffd640 1 -- 172.21.15.116:0/3214561507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e941027a0 msgr2=0x7f1e94198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:38.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.549+0000 7f1e92ffd640 1 --2- 172.21.15.116:0/3214561507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e941027a0 0x7f1e94198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.549+0000 7f1e92ffd640 1 -- 172.21.15.116:0/3214561507 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e9419d390 con 0x7f1e94101df0 2024-09-12T03:17:38.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.549+0000 7f1e92ffd640 1 --2- 172.21.15.116:0/3214561507 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e94101df0 0x7f1e941983e0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f1e8400cce0 tx=0x7f1e8400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:38.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.550+0000 7f1e8bfff640 1 -- 172.21.15.116:0/3214561507 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8400f450 con 0x7f1e94101df0 2024-09-12T03:17:38.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.550+0000 7f1e8bfff640 1 -- 172.21.15.116:0/3214561507 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e8400fad0 con 0x7f1e94101df0 2024-09-12T03:17:38.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.550+0000 7f1e9961b640 1 -- 172.21.15.116:0/3214561507 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e9419d680 con 0x7f1e94101df0 2024-09-12T03:17:38.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.550+0000 7f1e8bfff640 1 -- 172.21.15.116:0/3214561507 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e84018b80 con 0x7f1e94101df0 2024-09-12T03:17:38.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.550+0000 7f1e9961b640 1 -- 172.21.15.116:0/3214561507 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e941a4f60 con 0x7f1e94101df0 2024-09-12T03:17:38.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.552+0000 7f1e8bfff640 1 -- 172.21.15.116:0/3214561507 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1e8400f5f0 con 0x7f1e94101df0 2024-09-12T03:17:38.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.552+0000 7f1e9b9f4740 1 Processor -- start 2024-09-12T03:17:38.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.552+0000 7f1e9b9f4740 1 -- start start 2024-09-12T03:17:38.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.553+0000 7f1e9b9f4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb9c59b60 0x55bbb9c59f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.553+0000 7f1e9b9f4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb9c5a480 0x55bbb9c64800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.553+0000 7f1e9b9f4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb9c64d40 0x55bbb9c67130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.553+0000 7f1e9b9f4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bbb9bbf280 con 0x55bbb9c64d40 2024-09-12T03:17:38.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.553+0000 7f1e9b9f4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bbb9b66b10 con 0x55bbb9c5a480 2024-09-12T03:17:38.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.553+0000 7f1e9b9f4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bbb9bbb240 con 0x55bbb9c59b60 2024-09-12T03:17:38.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.553+0000 7f1e8bfff640 1 --2- 172.21.15.116:0/3214561507 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1e800a2b80 0x7f1e800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.553+0000 7f1e927fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb9c64d40 0x55bbb9c67130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.553+0000 7f1e927fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb9c64d40 0x55bbb9c67130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42440/0 (socket says 172.21.15.116:42440) 2024-09-12T03:17:38.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.553+0000 7f1e927fc640 1 -- 172.21.15.116:0/1695801699 learned_addr learned my addr 172.21.15.116:0/1695801699 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:38.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.553+0000 7f1e8bfff640 1 -- 172.21.15.116:0/3214561507 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1e840cf5a0 con 0x7f1e94101df0 2024-09-12T03:17:38.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.553+0000 7f1e937fe640 1 --2- 172.21.15.116:0/3214561507 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1e800a2b80 0x7f1e800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.553+0000 7f1e92ffd640 1 --2- 172.21.15.116:0/1695801699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb9c5a480 0x55bbb9c64800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.553+0000 7f1e937fe640 1 --2- 172.21.15.116:0/1695801699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb9c59b60 0x55bbb9c59f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.553+0000 7f1e927fc640 1 -- 172.21.15.116:0/1695801699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb9c59b60 msgr2=0x55bbb9c59f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:38.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.553+0000 7f1e927fc640 1 --2- 172.21.15.116:0/1695801699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb9c59b60 0x55bbb9c59f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.553+0000 7f1e927fc640 1 -- 172.21.15.116:0/1695801699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb9c5a480 msgr2=0x55bbb9c64800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:38.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.553+0000 7f1e927fc640 1 --2- 172.21.15.116:0/1695801699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb9c5a480 0x55bbb9c64800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.553+0000 7f1e927fc640 1 -- 172.21.15.116:0/1695801699 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbb9b731f0 con 0x55bbb9c64d40 2024-09-12T03:17:38.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.553+0000 7f1e937fe640 1 --2- 172.21.15.116:0/1695801699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb9c59b60 0x55bbb9c59f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:38.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.554+0000 7f1e927fc640 1 --2- 172.21.15.116:0/1695801699 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb9c64d40 0x55bbb9c67130 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009a40 tx=0x7f1e7c009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04d8a6944411318 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:38.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.554+0000 7f1e937fe640 1 --2- 172.21.15.116:0/3214561507 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1e800a2b80 0x7f1e800a5040 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c031550 tx=0x7f1e8c009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:38.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.554+0000 7f1e897fa640 1 -- 172.21.15.116:0/1695801699 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c017070 con 0x55bbb9c64d40 2024-09-12T03:17:38.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.554+0000 7f1e897fa640 1 -- 172.21.15.116:0/1695801699 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e7c013470 con 0x55bbb9c64d40 2024-09-12T03:17:38.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.554+0000 7f1e897fa640 1 -- 172.21.15.116:0/1695801699 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c012630 con 0x55bbb9c64d40 2024-09-12T03:17:38.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.555+0000 7f1e9b9f4740 1 -- 172.21.15.116:0/1695801699 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb9c64d40 msgr2=0x55bbb9c67130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:38.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.555+0000 7f1e9b9f4740 1 --2- 172.21.15.116:0/1695801699 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb9c64d40 0x55bbb9c67130 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009a40 tx=0x7f1e7c009f00 comp rx=0 tx=0).stop 2024-09-12T03:17:38.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.555+0000 7f1e9b9f4740 1 -- 172.21.15.116:0/1695801699 shutdown_connections 2024-09-12T03:17:38.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.555+0000 7f1e9b9f4740 1 --2- 172.21.15.116:0/1695801699 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb9c64d40 0x55bbb9c67130 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.555+0000 7f1e9b9f4740 1 --2- 172.21.15.116:0/1695801699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb9c5a480 0x55bbb9c64800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.555+0000 7f1e9b9f4740 1 --2- 172.21.15.116:0/1695801699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb9c59b60 0x55bbb9c59f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.555+0000 7f1e9b9f4740 1 -- 172.21.15.116:0/1695801699 >> 172.21.15.116:0/1695801699 conn(0x55bbb9c560f0 msgr2=0x55bbb9c564d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:38.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.555+0000 7f1e9b9f4740 1 -- 172.21.15.116:0/1695801699 shutdown_connections 2024-09-12T03:17:38.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.555+0000 7f1e9b9f4740 1 -- 172.21.15.116:0/1695801699 wait complete. 2024-09-12T03:17:38.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.555+0000 7f1e9b9f4740 1 -- start start 2024-09-12T03:17:38.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.555+0000 7f1e9b9f4740 1 Processor -- start 2024-09-12T03:17:38.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.556+0000 7f1e9b9f4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb9c59b60 0x55bbb9c61b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.556+0000 7f1e9b9f4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb9c5a480 0x55bbb9c620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.556+0000 7f1e9b9f4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb9c64d40 0x55bbb9c62610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.556+0000 7f1e9b9f4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bbb9c67910 con 0x55bbb9c5a480 2024-09-12T03:17:38.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.556+0000 7f1e9b9f4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bbb9b79f10 con 0x55bbb9c59b60 2024-09-12T03:17:38.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.556+0000 7f1e9b9f4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bbb9c67a90 con 0x55bbb9c64d40 2024-09-12T03:17:38.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.556+0000 7f1e9b9f4740 10 client.0 _ll_register_callbacks cb 0x55bbb9b42a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:38.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.556+0000 7f1e927fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb9c5a480 0x55bbb9c620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.556+0000 7f1e927fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb9c5a480 0x55bbb9c620d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42450/0 (socket says 172.21.15.116:42450) 2024-09-12T03:17:38.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.556+0000 7f1e927fc640 1 -- 172.21.15.116:0/3378647288 learned_addr learned my addr 172.21.15.116:0/3378647288 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:38.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.556+0000 7f1e92ffd640 1 --2- 172.21.15.116:0/3378647288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb9c59b60 0x55bbb9c61b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.556+0000 7f1e927fc640 1 -- 172.21.15.116:0/3378647288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb9c64d40 msgr2=0x55bbb9c62610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:38.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.556+0000 7f1e937fe640 1 --2- 172.21.15.116:0/3378647288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb9c64d40 0x55bbb9c62610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.556+0000 7f1e927fc640 1 --2- 172.21.15.116:0/3378647288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb9c64d40 0x55bbb9c62610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.556+0000 7f1e927fc640 1 -- 172.21.15.116:0/3378647288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb9c59b60 msgr2=0x55bbb9c61b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:38.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.556+0000 7f1e927fc640 1 --2- 172.21.15.116:0/3378647288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb9c59b60 0x55bbb9c61b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.556+0000 7f1e927fc640 1 -- 172.21.15.116:0/3378647288 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbb99306a0 con 0x55bbb9c5a480 2024-09-12T03:17:38.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.556+0000 7f1e92ffd640 1 --2- 172.21.15.116:0/3378647288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb9c59b60 0x55bbb9c61b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:38.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.556+0000 7f1e937fe640 1 --2- 172.21.15.116:0/3378647288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb9c64d40 0x55bbb9c62610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:38.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.556+0000 7f1e927fc640 1 --2- 172.21.15.116:0/3378647288 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb9c5a480 0x55bbb9c620d0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c00e660 tx=0x7f1e7c009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5251483d63b47c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:38.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.556+0000 7f1e7a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:38.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.556+0000 7f1e7a7fc640 1 -- 172.21.15.116:0/3378647288 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c017050 con 0x55bbb9c5a480 2024-09-12T03:17:38.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.556+0000 7f1e7a7fc640 1 -- 172.21.15.116:0/3378647288 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1e7c01c450 con 0x55bbb9c5a480 2024-09-12T03:17:38.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.556+0000 7f1e9b9f4740 1 -- 172.21.15.116:0/3378647288 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e7c013470 con 0x55bbb9c5a480 2024-09-12T03:17:38.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.557+0000 7f1e7a7fc640 1 -- 172.21.15.116:0/3378647288 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1e7c02d070 con 0x55bbb9c5a480 2024-09-12T03:17:38.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.557+0000 7f1e9b9f4740 10 client.19374 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:38.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.557+0000 7f1e9b9f4740 1 -- 172.21.15.116:0/3378647288 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bbb9c734a0 con 0x55bbb9c5a480 2024-09-12T03:17:38.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.557+0000 7f1e7a7fc640 1 -- 172.21.15.116:0/3378647288 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e7c013470 con 0x55bbb9c5a480 2024-09-12T03:17:38.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.557+0000 7f1e7a7fc640 1 -- 172.21.15.116:0/3378647288 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1e7c026710 con 0x55bbb9c5a480 2024-09-12T03:17:38.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.557+0000 7f1e9b9f4740 10 client.19374 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:38.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.557+0000 7f1e9b9f4740 10 client.19374 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:17:38.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.557+0000 7f1e9b9f4740 20 client.19374 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:38.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.557+0000 7f1e9b9f4740 1 --2- 172.21.15.116:0/3378647288 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bbb9c74960 0x55bbb9c76d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.557+0000 7f1e9b9f4740 4 client.19374 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:17:38.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.557+0000 7f1e9b9f4740 1 -- 172.21.15.116:0/3378647288 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbb9bcc7f0 con 0x55bbb9c74960 2024-09-12T03:17:38.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.557+0000 7f1e92ffd640 1 --2- 172.21.15.116:0/3378647288 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bbb9c74960 0x55bbb9c76d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.558+0000 7f1e92ffd640 1 --2- 172.21.15.116:0/3378647288 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bbb9c74960 0x55bbb9c76d50 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1130388a24df492c server_cookie=5164e55a2888b41a in_seq=0 out_seq=0 2024-09-12T03:17:38.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.558+0000 7f1e7a7fc640 10 client.19374 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:38.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.558+0000 7f1e7a7fc640 1 -- 172.21.15.116:0/3378647288 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bbb9bcc7f0 con 0x55bbb9c74960 2024-09-12T03:17:38.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.558+0000 7f1e7a7fc640 10 client.19374 handle_command_reply: tid=0 2024-09-12T03:17:38.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.558+0000 7f1e9b9f4740 1 client.19374 shutdown 2024-09-12T03:17:38.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.558+0000 7f1e9b9f4740 1 -- 172.21.15.116:0/3378647288 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb9c5a480 msgr2=0x55bbb9c620d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:38.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.558+0000 7f1e9b9f4740 1 --2- 172.21.15.116:0/3378647288 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb9c5a480 0x55bbb9c620d0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c00e660 tx=0x7f1e7c009740 comp rx=0 tx=0).stop 2024-09-12T03:17:38.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.559+0000 7f1e9b9f4740 1 -- 172.21.15.116:0/3378647288 shutdown_connections 2024-09-12T03:17:38.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.559+0000 7f1e9b9f4740 1 --2- 172.21.15.116:0/3378647288 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bbb9c74960 0x55bbb9c76d50 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.559+0000 7f1e9b9f4740 1 --2- 172.21.15.116:0/3378647288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb9c64d40 0x55bbb9c62610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.559+0000 7f1e7a7fc640 0 client.19374 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:38.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.559+0000 7f1e9b9f4740 1 --2- 172.21.15.116:0/3378647288 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb9c5a480 0x55bbb9c620d0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.559+0000 7f1e9b9f4740 1 --2- 172.21.15.116:0/3378647288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb9c59b60 0x55bbb9c61b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.559+0000 7f1e9b9f4740 1 -- 172.21.15.116:0/3378647288 >> 172.21.15.116:0/3378647288 conn(0x55bbb9c560f0 msgr2=0x55bbb9c55270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:38.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.559+0000 7f1e9b9f4740 1 -- 172.21.15.116:0/3378647288 shutdown_connections 2024-09-12T03:17:38.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.559+0000 7f1e9b9f4740 1 -- 172.21.15.116:0/3378647288 wait complete. 2024-09-12T03:17:38.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.559+0000 7f1e9b9f4740 20 client.19374 trim_cache size 0 max 16384 2024-09-12T03:17:38.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.576+0000 7f1e9b9f4740 1 Processor -- start 2024-09-12T03:17:38.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.576+0000 7f1e9b9f4740 1 -- start start 2024-09-12T03:17:38.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.576+0000 7f1e9b9f4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb9c59b60 0x55bbb9c7c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.576+0000 7f1e9b9f4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb9c5a480 0x55bbb9c7c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.576+0000 7f1e9b9f4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb9c64d40 0x55bbb9c5b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.576+0000 7f1e9b9f4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1e941afd50 con 0x55bbb9c64d40 2024-09-12T03:17:38.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.576+0000 7f1e9b9f4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1e941ad620 con 0x55bbb9c5a480 2024-09-12T03:17:38.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.576+0000 7f1e9b9f4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1e941ab470 con 0x55bbb9c59b60 2024-09-12T03:17:38.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.576+0000 7f1e937fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb9c64d40 0x55bbb9c5b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.576+0000 7f1e937fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb9c64d40 0x55bbb9c5b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42454/0 (socket says 172.21.15.116:42454) 2024-09-12T03:17:38.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.576+0000 7f1e937fe640 1 -- 172.21.15.116:0/260714342 learned_addr learned my addr 172.21.15.116:0/260714342 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:38.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.576+0000 7f1e92ffd640 1 --2- 172.21.15.116:0/260714342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb9c59b60 0x55bbb9c7c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.576+0000 7f1e937fe640 1 -- 172.21.15.116:0/260714342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb9c59b60 msgr2=0x55bbb9c7c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:38.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.576+0000 7f1e937fe640 1 --2- 172.21.15.116:0/260714342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb9c59b60 0x55bbb9c7c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.576+0000 7f1e927fc640 1 --2- 172.21.15.116:0/260714342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb9c5a480 0x55bbb9c7c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.576+0000 7f1e937fe640 1 -- 172.21.15.116:0/260714342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb9c5a480 msgr2=0x55bbb9c7c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:38.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.576+0000 7f1e937fe640 1 --2- 172.21.15.116:0/260714342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb9c5a480 0x55bbb9c7c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.576+0000 7f1e937fe640 1 -- 172.21.15.116:0/260714342 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbb9c5a2b0 con 0x55bbb9c64d40 2024-09-12T03:17:38.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.577+0000 7f1e92ffd640 1 --2- 172.21.15.116:0/260714342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb9c59b60 0x55bbb9c7c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:38.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.577+0000 7f1e937fe640 1 --2- 172.21.15.116:0/260714342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb9c64d40 0x55bbb9c5b5b0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c033000 tx=0x7f1e8c03e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8d464861ce1a6e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:38.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.577+0000 7f1e88ff9640 1 -- 172.21.15.116:0/260714342 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c004240 con 0x55bbb9c64d40 2024-09-12T03:17:38.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.577+0000 7f1e88ff9640 1 -- 172.21.15.116:0/260714342 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e8c0043e0 con 0x55bbb9c64d40 2024-09-12T03:17:38.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.577+0000 7f1e88ff9640 1 -- 172.21.15.116:0/260714342 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c0035f0 con 0x55bbb9c64d40 2024-09-12T03:17:38.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.578+0000 7f1e9b9f4740 1 -- 172.21.15.116:0/260714342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb9c64d40 msgr2=0x55bbb9c5b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:38.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.578+0000 7f1e9b9f4740 1 --2- 172.21.15.116:0/260714342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb9c64d40 0x55bbb9c5b5b0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c033000 tx=0x7f1e8c03e000 comp rx=0 tx=0).stop 2024-09-12T03:17:38.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.578+0000 7f1e9b9f4740 1 -- 172.21.15.116:0/260714342 shutdown_connections 2024-09-12T03:17:38.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.578+0000 7f1e9b9f4740 1 --2- 172.21.15.116:0/260714342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb9c64d40 0x55bbb9c5b5b0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.578+0000 7f1e9b9f4740 1 --2- 172.21.15.116:0/260714342 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb9c5a480 0x55bbb9c7c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.578+0000 7f1e9b9f4740 1 --2- 172.21.15.116:0/260714342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb9c59b60 0x55bbb9c7c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.578+0000 7f1e9b9f4740 1 -- 172.21.15.116:0/260714342 >> 172.21.15.116:0/260714342 conn(0x55bbb9c560f0 msgr2=0x55bbb9c63b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:38.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.578+0000 7f1e9b9f4740 1 -- 172.21.15.116:0/260714342 shutdown_connections 2024-09-12T03:17:38.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.578+0000 7f1e9b9f4740 1 -- 172.21.15.116:0/260714342 wait complete. 2024-09-12T03:17:38.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.578+0000 7f1e9b9f4740 1 -- start start 2024-09-12T03:17:38.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.578+0000 7f1e9b9f4740 1 Processor -- start 2024-09-12T03:17:38.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.578+0000 7f1e9b9f4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb9c59b60 0x55bbb9c6d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.578+0000 7f1e9b9f4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb9c5a480 0x55bbb9c68530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.578+0000 7f1e9b9f4740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb9c64d40 0x55bbb9c68a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.578+0000 7f1e9b9f4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1e941abae0 con 0x55bbb9c59b60 2024-09-12T03:17:38.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.578+0000 7f1e9b9f4740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1e941acf50 con 0x55bbb9c64d40 2024-09-12T03:17:38.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.578+0000 7f1e9b9f4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1e941ac1b0 con 0x55bbb9c5a480 2024-09-12T03:17:38.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.578+0000 7f1e9b9f4740 10 client.0 _ll_register_callbacks cb 0x7f1e84007850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:38.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.578+0000 7f1e92ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb9c59b60 0x55bbb9c6d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.578+0000 7f1e92ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb9c59b60 0x55bbb9c6d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42468/0 (socket says 172.21.15.116:42468) 2024-09-12T03:17:38.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.578+0000 7f1e92ffd640 1 -- 172.21.15.116:0/1357576386 learned_addr learned my addr 172.21.15.116:0/1357576386 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:38.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.579+0000 7f1e927fc640 1 --2- 172.21.15.116:0/1357576386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb9c5a480 0x55bbb9c68530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.579+0000 7f1e937fe640 1 --2- 172.21.15.116:0/1357576386 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb9c64d40 0x55bbb9c68a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.579+0000 7f1e92ffd640 1 -- 172.21.15.116:0/1357576386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb9c5a480 msgr2=0x55bbb9c68530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:38.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.579+0000 7f1e92ffd640 1 --2- 172.21.15.116:0/1357576386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb9c5a480 0x55bbb9c68530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.579+0000 7f1e92ffd640 1 -- 172.21.15.116:0/1357576386 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb9c64d40 msgr2=0x55bbb9c68a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:38.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.579+0000 7f1e92ffd640 1 --2- 172.21.15.116:0/1357576386 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb9c64d40 0x55bbb9c68a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.579+0000 7f1e92ffd640 1 -- 172.21.15.116:0/1357576386 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbb9c69030 con 0x55bbb9c59b60 2024-09-12T03:17:38.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.579+0000 7f1e937fe640 1 --2- 172.21.15.116:0/1357576386 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb9c64d40 0x55bbb9c68a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:38.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.579+0000 7f1e92ffd640 1 --2- 172.21.15.116:0/1357576386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb9c59b60 0x55bbb9c6d200 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f1e8400ab20 tx=0x7f1e84000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad4953249cdbe56 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:38.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.579+0000 7f1e7a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:38.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.579+0000 7f1e7a7fc640 1 -- 172.21.15.116:0/1357576386 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e840a01e0 con 0x55bbb9c59b60 2024-09-12T03:17:38.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.579+0000 7f1e7a7fc640 1 -- 172.21.15.116:0/1357576386 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1e8409a130 con 0x55bbb9c59b60 2024-09-12T03:17:38.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.579+0000 7f1e9b9f4740 1 -- 172.21.15.116:0/1357576386 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e8c0043e0 con 0x55bbb9c59b60 2024-09-12T03:17:38.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.579+0000 7f1e7a7fc640 1 -- 172.21.15.116:0/1357576386 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1e840a4aa0 con 0x55bbb9c59b60 2024-09-12T03:17:38.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.579+0000 7f1e9b9f4740 10 client.19380 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:38.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.579+0000 7f1e9b9f4740 1 -- 172.21.15.116:0/1357576386 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bbb9cdf160 con 0x55bbb9c59b60 2024-09-12T03:17:38.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.579+0000 7f1e7a7fc640 1 -- 172.21.15.116:0/1357576386 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e8c0043e0 con 0x55bbb9c59b60 2024-09-12T03:17:38.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.579+0000 7f1e7a7fc640 1 -- 172.21.15.116:0/1357576386 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1e840a4610 con 0x55bbb9c59b60 2024-09-12T03:17:38.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.580+0000 7f1e9b9f4740 10 client.19380 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:38.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.580+0000 7f1e9b9f4740 10 client.19380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:17:38.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.580+0000 7f1e9b9f4740 20 client.19380 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:38.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.580+0000 7f1e9b9f4740 1 --2- 172.21.15.116:0/1357576386 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bbb9cdf7e0 0x55bbb9ce1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.580+0000 7f1e9b9f4740 4 client.19380 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:17:38.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.580+0000 7f1e9b9f4740 1 -- 172.21.15.116:0/1357576386 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bbb9bc6d00 con 0x55bbb9cdf7e0 2024-09-12T03:17:38.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.580+0000 7f1e927fc640 1 --2- 172.21.15.116:0/1357576386 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bbb9cdf7e0 0x55bbb9ce1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.580+0000 7f1e927fc640 1 --2- 172.21.15.116:0/1357576386 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bbb9cdf7e0 0x55bbb9ce1bd0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eee47309f50719c server_cookie=3ae4b3298dcac634 in_seq=0 out_seq=0 2024-09-12T03:17:38.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.580+0000 7f1e7a7fc640 10 client.19380 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:38.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.581+0000 7f1e7a7fc640 1 -- 172.21.15.116:0/1357576386 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bbb9bc6d00 con 0x55bbb9cdf7e0 2024-09-12T03:17:38.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.581+0000 7f1e7a7fc640 10 client.19380 handle_command_reply: tid=0 2024-09-12T03:17:38.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.581+0000 7f1e9b9f4740 1 client.19380 shutdown 2024-09-12T03:17:38.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.581+0000 7f1e9b9f4740 1 -- 172.21.15.116:0/1357576386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb9c59b60 msgr2=0x55bbb9c6d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:38.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.581+0000 7f1e9b9f4740 1 --2- 172.21.15.116:0/1357576386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb9c59b60 0x55bbb9c6d200 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f1e8400ab20 tx=0x7f1e84000f40 comp rx=0 tx=0).stop 2024-09-12T03:17:38.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.581+0000 7f1e9b9f4740 1 -- 172.21.15.116:0/1357576386 shutdown_connections 2024-09-12T03:17:38.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.581+0000 7f1e9b9f4740 1 --2- 172.21.15.116:0/1357576386 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55bbb9cdf7e0 0x55bbb9ce1bd0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.582+0000 7f1e9b9f4740 1 --2- 172.21.15.116:0/1357576386 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bbb9c64d40 0x55bbb9c68a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.582+0000 7f1e7a7fc640 0 client.19380 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:38.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.582+0000 7f1e9b9f4740 1 --2- 172.21.15.116:0/1357576386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bbb9c5a480 0x55bbb9c68530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.582+0000 7f1e9b9f4740 1 --2- 172.21.15.116:0/1357576386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbb9c59b60 0x55bbb9c6d200 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.582+0000 7f1e9b9f4740 1 -- 172.21.15.116:0/1357576386 >> 172.21.15.116:0/1357576386 conn(0x55bbb9c560f0 msgr2=0x55bbb9c63200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:38.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.582+0000 7f1e9b9f4740 1 -- 172.21.15.116:0/1357576386 shutdown_connections 2024-09-12T03:17:38.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.582+0000 7f1e9b9f4740 1 -- 172.21.15.116:0/1357576386 wait complete. 2024-09-12T03:17:38.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.582+0000 7f1e9b9f4740 20 client.19380 trim_cache size 0 max 16384 2024-09-12T03:17:38.592 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:17:38.592 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:17:38.593 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "75953ccc-19e2-411e-9f38-9481a2e9253a", 2024-09-12T03:17:38.593 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:17:38.593 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:17:38.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.582+0000 7f1e9961b640 1 -- 172.21.15.116:0/3214561507 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1e800a2b80 msgr2=0x7f1e800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:38.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.582+0000 7f1e9961b640 1 --2- 172.21.15.116:0/3214561507 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1e800a2b80 0x7f1e800a5040 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c031550 tx=0x7f1e8c009600 comp rx=0 tx=0).stop 2024-09-12T03:17:38.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.582+0000 7f1e9961b640 1 -- 172.21.15.116:0/3214561507 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e94101df0 msgr2=0x7f1e941983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:38.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.582+0000 7f1e9961b640 1 --2- 172.21.15.116:0/3214561507 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e94101df0 0x7f1e941983e0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f1e8400cce0 tx=0x7f1e8400a620 comp rx=0 tx=0).stop 2024-09-12T03:17:38.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.582+0000 7f1e9961b640 1 -- 172.21.15.116:0/3214561507 shutdown_connections 2024-09-12T03:17:38.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.582+0000 7f1e9961b640 1 --2- 172.21.15.116:0/3214561507 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1e800a2b80 0x7f1e800a5040 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.582+0000 7f1e9961b640 1 --2- 172.21.15.116:0/3214561507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e94106f20 0x7f1e9419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.582+0000 7f1e9961b640 1 --2- 172.21.15.116:0/3214561507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e941027a0 0x7f1e94198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.582+0000 7f1e9961b640 1 --2- 172.21.15.116:0/3214561507 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e94101df0 0x7f1e941983e0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.582+0000 7f1e9961b640 1 -- 172.21.15.116:0/3214561507 >> 172.21.15.116:0/3214561507 conn(0x7f1e940fd8f0 msgr2=0x7f1e940feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:38.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.582+0000 7f1e9961b640 1 -- 172.21.15.116:0/3214561507 shutdown_connections 2024-09-12T03:17:38.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:38.583+0000 7f1e9961b640 1 -- 172.21.15.116:0/3214561507 wait complete. 2024-09-12T03:17:39.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:38 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:17:39.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:38 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:17:39.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:38 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:17:39.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:38 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:17:39.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:38 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:17:39.232 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:38 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:17:39.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:38 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:17:39.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:38 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:17:39.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:38 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:17:39.233 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:38 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:17:39.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:38 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:17:39.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:38 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:17:39.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:38 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:17:39.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:38 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:17:39.245 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:38 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:17:39.322 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:17:39.609 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:40.017 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:39 smithi116 ceph-mon[31410]: pgmap v778: 97 pgs: 97 active+clean; 6.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:17:40.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.097+0000 7fe59e138640 1 -- 172.21.15.116:0/2836497712 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe59810c170 msgr2=0x7fe59810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:40.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.097+0000 7fe59e138640 1 --2- 172.21.15.116:0/2836497712 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe59810c170 0x7fe59810e610 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fe590009900 tx=0x7fe59002a480 comp rx=0 tx=0).stop 2024-09-12T03:17:40.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.097+0000 7fe59e138640 1 -- 172.21.15.116:0/2836497712 shutdown_connections 2024-09-12T03:17:40.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.097+0000 7fe59e138640 1 --2- 172.21.15.116:0/2836497712 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe59810c170 0x7fe59810e610 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.097+0000 7fe59e138640 1 --2- 172.21.15.116:0/2836497712 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe598103660 0x7fe59810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.097+0000 7fe59e138640 1 --2- 172.21.15.116:0/2836497712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe598102d40 0x7fe598103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.097+0000 7fe59e138640 1 -- 172.21.15.116:0/2836497712 >> 172.21.15.116:0/2836497712 conn(0x7fe5980fc7a0 msgr2=0x7fe5980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:40.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.097+0000 7fe59e138640 1 -- 172.21.15.116:0/2836497712 shutdown_connections 2024-09-12T03:17:40.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.097+0000 7fe59e138640 1 -- 172.21.15.116:0/2836497712 wait complete. 2024-09-12T03:17:40.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.098+0000 7fe59e138640 1 Processor -- start 2024-09-12T03:17:40.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.098+0000 7fe59e138640 1 -- start start 2024-09-12T03:17:40.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.098+0000 7fe59e138640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe598102d40 0x7fe598198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.098+0000 7fe59e138640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe598103660 0x7fe5981989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.098+0000 7fe59e138640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe59810c170 0x7fe59819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.098+0000 7fe59e138640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe598111fb0 con 0x7fe598103660 2024-09-12T03:17:40.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.098+0000 7fe59e138640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe598111e30 con 0x7fe59810c170 2024-09-12T03:17:40.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.098+0000 7fe59e138640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe598112130 con 0x7fe598102d40 2024-09-12T03:17:40.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.098+0000 7fe596ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe598103660 0x7fe5981989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.098+0000 7fe596ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe598103660 0x7fe5981989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42482/0 (socket says 172.21.15.116:42482) 2024-09-12T03:17:40.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.098+0000 7fe596ffd640 1 -- 172.21.15.116:0/150375229 learned_addr learned my addr 172.21.15.116:0/150375229 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:40.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.098+0000 7fe5977fe640 1 --2- 172.21.15.116:0/150375229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe598102d40 0x7fe598198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.099+0000 7fe597fff640 1 --2- 172.21.15.116:0/150375229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe59810c170 0x7fe59819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.099+0000 7fe596ffd640 1 -- 172.21.15.116:0/150375229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe598102d40 msgr2=0x7fe598198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:40.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.099+0000 7fe596ffd640 1 --2- 172.21.15.116:0/150375229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe598102d40 0x7fe598198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.099+0000 7fe596ffd640 1 -- 172.21.15.116:0/150375229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe59810c170 msgr2=0x7fe59819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:40.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.099+0000 7fe596ffd640 1 --2- 172.21.15.116:0/150375229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe59810c170 0x7fe59819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.099+0000 7fe596ffd640 1 -- 172.21.15.116:0/150375229 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe59819d420 con 0x7fe598103660 2024-09-12T03:17:40.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.099+0000 7fe597fff640 1 --2- 172.21.15.116:0/150375229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe59810c170 0x7fe59819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:40.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.099+0000 7fe596ffd640 1 --2- 172.21.15.116:0/150375229 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe598103660 0x7fe5981989b0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fe58c00b3f0 tx=0x7fe58c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:40.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.099+0000 7fe594ff9640 1 -- 172.21.15.116:0/150375229 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c015020 con 0x7fe598103660 2024-09-12T03:17:40.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.099+0000 7fe594ff9640 1 -- 172.21.15.116:0/150375229 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe58c011d10 con 0x7fe598103660 2024-09-12T03:17:40.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.099+0000 7fe594ff9640 1 -- 172.21.15.116:0/150375229 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c00f2e0 con 0x7fe598103660 2024-09-12T03:17:40.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.099+0000 7fe59e138640 1 -- 172.21.15.116:0/150375229 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe59819d710 con 0x7fe598103660 2024-09-12T03:17:40.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.099+0000 7fe59e138640 1 -- 172.21.15.116:0/150375229 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5981a4ff0 con 0x7fe598103660 2024-09-12T03:17:40.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.101+0000 7fe594ff9640 1 -- 172.21.15.116:0/150375229 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe58c018020 con 0x7fe598103660 2024-09-12T03:17:40.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.102+0000 7fe5a0511740 1 Processor -- start 2024-09-12T03:17:40.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.102+0000 7fe5a0511740 1 -- start start 2024-09-12T03:17:40.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.102+0000 7fe594ff9640 1 --2- 172.21.15.116:0/150375229 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe5800a2b80 0x7fe5800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.102+0000 7fe5a0511740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae3edf14a0 0x55ae3edf1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.102+0000 7fe5a0511740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ae3edf1dc0 0x55ae3edfc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.102+0000 7fe5a0511740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae3edfc680 0x55ae3edfea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.102+0000 7fe5a0511740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ae3ed572e0 con 0x55ae3edf1dc0 2024-09-12T03:17:40.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.102+0000 7fe5a0511740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ae3ecfece0 con 0x55ae3edf14a0 2024-09-12T03:17:40.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.102+0000 7fe5a0511740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ae3ed532a0 con 0x55ae3edfc680 2024-09-12T03:17:40.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.102+0000 7fe597fff640 1 --2- 172.21.15.116:0/150375229 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe5800a2b80 0x7fe5800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.102+0000 7fe594ff9640 1 -- 172.21.15.116:0/150375229 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe58c0ce840 con 0x7fe598103660 2024-09-12T03:17:40.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.102+0000 7fe596ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ae3edf1dc0 0x55ae3edfc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.102+0000 7fe596ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ae3edf1dc0 0x55ae3edfc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42490/0 (socket says 172.21.15.116:42490) 2024-09-12T03:17:40.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.102+0000 7fe596ffd640 1 -- 172.21.15.116:0/30091325 learned_addr learned my addr 172.21.15.116:0/30091325 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:40.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.102+0000 7fe5977fe640 1 --2- 172.21.15.116:0/30091325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae3edf14a0 0x55ae3edf1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.102+0000 7fe596ffd640 1 -- 172.21.15.116:0/30091325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae3edfc680 msgr2=0x55ae3edfea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:40.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.102+0000 7fe596ffd640 1 --2- 172.21.15.116:0/30091325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae3edfc680 0x55ae3edfea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.102+0000 7fe596ffd640 1 -- 172.21.15.116:0/30091325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae3edf14a0 msgr2=0x55ae3edf1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:40.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.102+0000 7fe596ffd640 1 --2- 172.21.15.116:0/30091325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae3edf14a0 0x55ae3edf1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.102+0000 7fe596ffd640 1 -- 172.21.15.116:0/30091325 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae3ece3440 con 0x55ae3edf1dc0 2024-09-12T03:17:40.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.102+0000 7fe597fff640 1 --2- 172.21.15.116:0/150375229 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe5800a2b80 0x7fe5800a5040 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe590005ab0 tx=0x7fe590031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:40.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.103+0000 7fe596ffd640 1 --2- 172.21.15.116:0/30091325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ae3edf1dc0 0x55ae3edfc140 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fe58c000c00 tx=0x7fe58c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=365b9d0097d6b43 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:40.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.103+0000 7fe57dffb640 1 -- 172.21.15.116:0/30091325 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c010eb0 con 0x55ae3edf1dc0 2024-09-12T03:17:40.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.103+0000 7fe57dffb640 1 -- 172.21.15.116:0/30091325 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe58c09b7d0 con 0x55ae3edf1dc0 2024-09-12T03:17:40.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.103+0000 7fe57dffb640 1 -- 172.21.15.116:0/30091325 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c010510 con 0x55ae3edf1dc0 2024-09-12T03:17:40.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.103+0000 7fe5a0511740 1 -- 172.21.15.116:0/30091325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ae3edf1dc0 msgr2=0x55ae3edfc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:40.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.103+0000 7fe5a0511740 1 --2- 172.21.15.116:0/30091325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ae3edf1dc0 0x55ae3edfc140 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fe58c000c00 tx=0x7fe58c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.103+0000 7fe5a0511740 1 -- 172.21.15.116:0/30091325 shutdown_connections 2024-09-12T03:17:40.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.103+0000 7fe5a0511740 1 --2- 172.21.15.116:0/30091325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae3edfc680 0x55ae3edfea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.103+0000 7fe5a0511740 1 --2- 172.21.15.116:0/30091325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ae3edf1dc0 0x55ae3edfc140 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.103+0000 7fe5a0511740 1 --2- 172.21.15.116:0/30091325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae3edf14a0 0x55ae3edf1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.103+0000 7fe5a0511740 1 -- 172.21.15.116:0/30091325 >> 172.21.15.116:0/30091325 conn(0x55ae3ededa30 msgr2=0x55ae3edede10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:40.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.103+0000 7fe5a0511740 1 -- 172.21.15.116:0/30091325 shutdown_connections 2024-09-12T03:17:40.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.103+0000 7fe5a0511740 1 -- 172.21.15.116:0/30091325 wait complete. 2024-09-12T03:17:40.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.104+0000 7fe5a0511740 1 -- start start 2024-09-12T03:17:40.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.104+0000 7fe5a0511740 1 Processor -- start 2024-09-12T03:17:40.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.104+0000 7fe5a0511740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae3edf14a0 0x55ae3edfc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.104+0000 7fe5a0511740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae3edf1dc0 0x55ae3edf92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.104+0000 7fe5a0511740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ae3edfc680 0x55ae3edf9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.104+0000 7fe5a0511740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ae3edff250 con 0x55ae3edfc680 2024-09-12T03:17:40.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.104+0000 7fe5a0511740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ae3ed12050 con 0x55ae3edf1dc0 2024-09-12T03:17:40.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.104+0000 7fe5a0511740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ae3edff3d0 con 0x55ae3edf14a0 2024-09-12T03:17:40.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.104+0000 7fe5a0511740 10 client.0 _ll_register_callbacks cb 0x55ae3ed52160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:40.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.104+0000 7fe597fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ae3edfc680 0x55ae3edf9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.104+0000 7fe597fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ae3edfc680 0x55ae3edf9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42504/0 (socket says 172.21.15.116:42504) 2024-09-12T03:17:40.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.105+0000 7fe597fff640 1 -- 172.21.15.116:0/3840277418 learned_addr learned my addr 172.21.15.116:0/3840277418 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:40.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.105+0000 7fe596ffd640 1 --2- 172.21.15.116:0/3840277418 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae3edf1dc0 0x55ae3edf92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.105+0000 7fe5977fe640 1 --2- 172.21.15.116:0/3840277418 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae3edf14a0 0x55ae3edfc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.105+0000 7fe597fff640 1 -- 172.21.15.116:0/3840277418 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae3edf14a0 msgr2=0x55ae3edfc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:40.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.105+0000 7fe597fff640 1 --2- 172.21.15.116:0/3840277418 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae3edf14a0 0x55ae3edfc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.105+0000 7fe597fff640 1 -- 172.21.15.116:0/3840277418 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae3edf1dc0 msgr2=0x55ae3edf92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:40.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.105+0000 7fe597fff640 1 --2- 172.21.15.116:0/3840277418 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae3edf1dc0 0x55ae3edf92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.105+0000 7fe597fff640 1 -- 172.21.15.116:0/3840277418 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae3eac83c0 con 0x55ae3edfc680 2024-09-12T03:17:40.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.105+0000 7fe597fff640 1 --2- 172.21.15.116:0/3840277418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ae3edfc680 0x55ae3edf9830 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fe590033000 tx=0x7fe590009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb412626d75ebfb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:40.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.105+0000 7fe576ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:40.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.105+0000 7fe576ffd640 1 -- 172.21.15.116:0/3840277418 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe590035050 con 0x55ae3edfc680 2024-09-12T03:17:40.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.105+0000 7fe576ffd640 1 -- 172.21.15.116:0/3840277418 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe59004a480 con 0x55ae3edfc680 2024-09-12T03:17:40.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.105+0000 7fe5a0511740 1 -- 172.21.15.116:0/3840277418 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe58c09b7d0 con 0x55ae3edfc680 2024-09-12T03:17:40.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.105+0000 7fe576ffd640 1 -- 172.21.15.116:0/3840277418 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe59004a680 con 0x55ae3edfc680 2024-09-12T03:17:40.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.105+0000 7fe5a0511740 10 client.19392 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:40.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.105+0000 7fe5a0511740 1 -- 172.21.15.116:0/3840277418 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ae3ee0ac40 con 0x55ae3edfc680 2024-09-12T03:17:40.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.105+0000 7fe576ffd640 1 -- 172.21.15.116:0/3840277418 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe58c09b7d0 con 0x55ae3edfc680 2024-09-12T03:17:40.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.106+0000 7fe576ffd640 1 -- 172.21.15.116:0/3840277418 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe59003d830 con 0x55ae3edfc680 2024-09-12T03:17:40.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.106+0000 7fe5a0511740 10 client.19392 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:40.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.106+0000 7fe5a0511740 10 client.19392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:17:40.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.106+0000 7fe5a0511740 20 client.19392 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:40.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.106+0000 7fe5a0511740 1 --2- 172.21.15.116:0/3840277418 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ae3ee0c100 0x55ae3ee0e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.106+0000 7fe5a0511740 4 client.19392 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:17:40.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.106+0000 7fe5a0511740 1 -- 172.21.15.116:0/3840277418 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae3ed64850 con 0x55ae3ee0c100 2024-09-12T03:17:40.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.106+0000 7fe5977fe640 1 --2- 172.21.15.116:0/3840277418 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ae3ee0c100 0x55ae3ee0e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.106+0000 7fe576ffd640 10 client.19392 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:40.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.106+0000 7fe5977fe640 1 --2- 172.21.15.116:0/3840277418 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ae3ee0c100 0x55ae3ee0e550 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1efbd335d217338 server_cookie=18b93c3e6f5fc094 in_seq=0 out_seq=0 2024-09-12T03:17:40.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.107+0000 7fe576ffd640 1 -- 172.21.15.116:0/3840277418 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae3ed64850 con 0x55ae3ee0c100 2024-09-12T03:17:40.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.107+0000 7fe576ffd640 10 client.19392 handle_command_reply: tid=0 2024-09-12T03:17:40.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.107+0000 7fe5a0511740 1 client.19392 shutdown 2024-09-12T03:17:40.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.107+0000 7fe5a0511740 1 -- 172.21.15.116:0/3840277418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ae3edfc680 msgr2=0x55ae3edf9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:40.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.107+0000 7fe5a0511740 1 --2- 172.21.15.116:0/3840277418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ae3edfc680 0x55ae3edf9830 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fe590033000 tx=0x7fe590009260 comp rx=0 tx=0).stop 2024-09-12T03:17:40.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.107+0000 7fe5a0511740 1 -- 172.21.15.116:0/3840277418 shutdown_connections 2024-09-12T03:17:40.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.107+0000 7fe5a0511740 1 --2- 172.21.15.116:0/3840277418 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ae3ee0c100 0x55ae3ee0e550 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.107+0000 7fe5a0511740 1 --2- 172.21.15.116:0/3840277418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ae3edfc680 0x55ae3edf9830 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.107+0000 7fe5a0511740 1 --2- 172.21.15.116:0/3840277418 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae3edf1dc0 0x55ae3edf92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.107+0000 7fe5a0511740 1 --2- 172.21.15.116:0/3840277418 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae3edf14a0 0x55ae3edfc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.107+0000 7fe576ffd640 0 client.19392 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:40.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.107+0000 7fe5a0511740 1 -- 172.21.15.116:0/3840277418 >> 172.21.15.116:0/3840277418 conn(0x55ae3ededa30 msgr2=0x55ae3edf3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:40.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.108+0000 7fe5a0511740 1 -- 172.21.15.116:0/3840277418 shutdown_connections 2024-09-12T03:17:40.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.108+0000 7fe5a0511740 1 -- 172.21.15.116:0/3840277418 wait complete. 2024-09-12T03:17:40.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.108+0000 7fe5a0511740 20 client.19392 trim_cache size 0 max 16384 2024-09-12T03:17:40.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.124+0000 7fe5a0511740 1 Processor -- start 2024-09-12T03:17:40.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.124+0000 7fe5a0511740 1 -- start start 2024-09-12T03:17:40.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.125+0000 7fe5a0511740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae3edf14a0 0x55ae3ee13e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.125+0000 7fe5a0511740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae3edf1dc0 0x55ae3ee143b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.125+0000 7fe5a0511740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ae3edfc680 0x55ae3edf6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.125+0000 7fe5a0511740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe5981afe10 con 0x55ae3edfc680 2024-09-12T03:17:40.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.125+0000 7fe5a0511740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe5981ad6e0 con 0x55ae3edf1dc0 2024-09-12T03:17:40.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.125+0000 7fe5a0511740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe5981ab530 con 0x55ae3edf14a0 2024-09-12T03:17:40.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.125+0000 7fe597fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ae3edfc680 0x55ae3edf6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.125+0000 7fe597fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ae3edfc680 0x55ae3edf6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42510/0 (socket says 172.21.15.116:42510) 2024-09-12T03:17:40.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.125+0000 7fe597fff640 1 -- 172.21.15.116:0/1554303592 learned_addr learned my addr 172.21.15.116:0/1554303592 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:40.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.125+0000 7fe5977fe640 1 --2- 172.21.15.116:0/1554303592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae3edf14a0 0x55ae3ee13e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.125+0000 7fe596ffd640 1 --2- 172.21.15.116:0/1554303592 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae3edf1dc0 0x55ae3ee143b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.125+0000 7fe597fff640 1 -- 172.21.15.116:0/1554303592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae3edf14a0 msgr2=0x55ae3ee13e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:40.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.125+0000 7fe597fff640 1 --2- 172.21.15.116:0/1554303592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae3edf14a0 0x55ae3ee13e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.125+0000 7fe597fff640 1 -- 172.21.15.116:0/1554303592 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae3edf1dc0 msgr2=0x55ae3ee143b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:40.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.125+0000 7fe597fff640 1 --2- 172.21.15.116:0/1554303592 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae3edf1dc0 0x55ae3ee143b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.125+0000 7fe597fff640 1 -- 172.21.15.116:0/1554303592 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae3edf8280 con 0x55ae3edfc680 2024-09-12T03:17:40.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.125+0000 7fe596ffd640 1 --2- 172.21.15.116:0/1554303592 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae3edf1dc0 0x55ae3ee143b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:40.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.125+0000 7fe5977fe640 1 --2- 172.21.15.116:0/1554303592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae3edf14a0 0x55ae3ee13e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:40.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.125+0000 7fe597fff640 1 --2- 172.21.15.116:0/1554303592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ae3edfc680 0x55ae3edf6550 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fe59003de10 tx=0x7fe5900043e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf5913efe530dcb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:40.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.125+0000 7fe57d7fa640 1 -- 172.21.15.116:0/1554303592 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe590035040 con 0x55ae3edfc680 2024-09-12T03:17:40.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.125+0000 7fe57d7fa640 1 -- 172.21.15.116:0/1554303592 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe59004f070 con 0x55ae3edfc680 2024-09-12T03:17:40.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.125+0000 7fe57d7fa640 1 -- 172.21.15.116:0/1554303592 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59004a820 con 0x55ae3edfc680 2024-09-12T03:17:40.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.126+0000 7fe5a0511740 1 -- 172.21.15.116:0/1554303592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ae3edfc680 msgr2=0x55ae3edf6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:40.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.126+0000 7fe5a0511740 1 --2- 172.21.15.116:0/1554303592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ae3edfc680 0x55ae3edf6550 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fe59003de10 tx=0x7fe5900043e0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.126+0000 7fe5a0511740 1 -- 172.21.15.116:0/1554303592 shutdown_connections 2024-09-12T03:17:40.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.126+0000 7fe5a0511740 1 --2- 172.21.15.116:0/1554303592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ae3edfc680 0x55ae3edf6550 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.126+0000 7fe5a0511740 1 --2- 172.21.15.116:0/1554303592 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae3edf1dc0 0x55ae3ee143b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.126+0000 7fe5a0511740 1 --2- 172.21.15.116:0/1554303592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae3edf14a0 0x55ae3ee13e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.126+0000 7fe5a0511740 1 -- 172.21.15.116:0/1554303592 >> 172.21.15.116:0/1554303592 conn(0x55ae3ededa30 msgr2=0x55ae3edfb330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:40.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.126+0000 7fe5a0511740 1 -- 172.21.15.116:0/1554303592 shutdown_connections 2024-09-12T03:17:40.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.126+0000 7fe5a0511740 1 -- 172.21.15.116:0/1554303592 wait complete. 2024-09-12T03:17:40.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.126+0000 7fe5a0511740 1 -- start start 2024-09-12T03:17:40.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.127+0000 7fe5a0511740 1 Processor -- start 2024-09-12T03:17:40.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.127+0000 7fe5a0511740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae3edf14a0 0x55ae3ee029e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.127+0000 7fe5a0511740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae3edf1dc0 0x55ae3ee02f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.127+0000 7fe5a0511740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ae3edfc680 0x55ae3ee75e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.127+0000 7fe5a0511740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe5981abba0 con 0x55ae3edfc680 2024-09-12T03:17:40.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.127+0000 7fe5a0511740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe5981ad010 con 0x55ae3edf14a0 2024-09-12T03:17:40.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.127+0000 7fe5a0511740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe5981ac270 con 0x55ae3edf1dc0 2024-09-12T03:17:40.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.127+0000 7fe5a0511740 10 client.0 _ll_register_callbacks cb 0x7fe590041070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:40.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.127+0000 7fe597fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ae3edfc680 0x55ae3ee75e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.127+0000 7fe597fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ae3edfc680 0x55ae3ee75e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42512/0 (socket says 172.21.15.116:42512) 2024-09-12T03:17:40.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.127+0000 7fe597fff640 1 -- 172.21.15.116:0/753573660 learned_addr learned my addr 172.21.15.116:0/753573660 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:40.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.127+0000 7fe5977fe640 1 --2- 172.21.15.116:0/753573660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae3edf14a0 0x55ae3ee029e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.127+0000 7fe596ffd640 1 --2- 172.21.15.116:0/753573660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae3edf1dc0 0x55ae3ee02f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.127+0000 7fe597fff640 1 -- 172.21.15.116:0/753573660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae3edf1dc0 msgr2=0x55ae3ee02f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:40.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.127+0000 7fe597fff640 1 --2- 172.21.15.116:0/753573660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae3edf1dc0 0x55ae3ee02f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.127+0000 7fe597fff640 1 -- 172.21.15.116:0/753573660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae3edf14a0 msgr2=0x55ae3ee029e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:40.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.127+0000 7fe597fff640 1 --2- 172.21.15.116:0/753573660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae3edf14a0 0x55ae3ee029e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.127+0000 7fe597fff640 1 -- 172.21.15.116:0/753573660 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae3ee76390 con 0x55ae3edfc680 2024-09-12T03:17:40.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.127+0000 7fe597fff640 1 --2- 172.21.15.116:0/753573660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ae3edfc680 0x55ae3ee75e50 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fe590031ba0 tx=0x7fe5900092b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125408754c9ee6eb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:40.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.127+0000 7fe576ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:40.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.128+0000 7fe576ffd640 1 -- 172.21.15.116:0/753573660 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe590035040 con 0x55ae3edfc680 2024-09-12T03:17:40.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.128+0000 7fe576ffd640 1 -- 172.21.15.116:0/753573660 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fe59004a480 con 0x55ae3edfc680 2024-09-12T03:17:40.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.128+0000 7fe5a0511740 1 -- 172.21.15.116:0/753573660 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe59004f070 con 0x55ae3edfc680 2024-09-12T03:17:40.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.128+0000 7fe576ffd640 1 -- 172.21.15.116:0/753573660 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe59003de40 con 0x55ae3edfc680 2024-09-12T03:17:40.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.128+0000 7fe5a0511740 10 client.19398 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:40.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.128+0000 7fe5a0511740 1 -- 172.21.15.116:0/753573660 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ae3ee77510 con 0x55ae3edfc680 2024-09-12T03:17:40.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.128+0000 7fe576ffd640 1 -- 172.21.15.116:0/753573660 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe59004f070 con 0x55ae3edfc680 2024-09-12T03:17:40.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.128+0000 7fe576ffd640 1 -- 172.21.15.116:0/753573660 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe5900569f0 con 0x55ae3edfc680 2024-09-12T03:17:40.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.128+0000 7fe5a0511740 10 client.19398 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:40.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.128+0000 7fe5a0511740 10 client.19398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:17:40.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.128+0000 7fe5a0511740 20 client.19398 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:40.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.128+0000 7fe5a0511740 1 --2- 172.21.15.116:0/753573660 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ae3ee77b70 0x55ae3ee79f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.128+0000 7fe5a0511740 4 client.19398 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:17:40.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.128+0000 7fe5a0511740 1 -- 172.21.15.116:0/753573660 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ae3ed5ed60 con 0x55ae3ee77b70 2024-09-12T03:17:40.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.128+0000 7fe5977fe640 1 --2- 172.21.15.116:0/753573660 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ae3ee77b70 0x55ae3ee79f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.129+0000 7fe5977fe640 1 --2- 172.21.15.116:0/753573660 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ae3ee77b70 0x55ae3ee79f60 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5b8a094818d3004 server_cookie=6d35ccca195a48a8 in_seq=0 out_seq=0 2024-09-12T03:17:40.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.129+0000 7fe576ffd640 10 client.19398 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:40.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.129+0000 7fe576ffd640 1 -- 172.21.15.116:0/753573660 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ae3ed5ed60 con 0x55ae3ee77b70 2024-09-12T03:17:40.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.129+0000 7fe576ffd640 10 client.19398 handle_command_reply: tid=0 2024-09-12T03:17:40.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.129+0000 7fe5a0511740 1 client.19398 shutdown 2024-09-12T03:17:40.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.129+0000 7fe5a0511740 1 -- 172.21.15.116:0/753573660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ae3edfc680 msgr2=0x55ae3ee75e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:40.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.129+0000 7fe5a0511740 1 --2- 172.21.15.116:0/753573660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ae3edfc680 0x55ae3ee75e50 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fe590031ba0 tx=0x7fe5900092b0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.129+0000 7fe5a0511740 1 -- 172.21.15.116:0/753573660 shutdown_connections 2024-09-12T03:17:40.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.129+0000 7fe5a0511740 1 --2- 172.21.15.116:0/753573660 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55ae3ee77b70 0x55ae3ee79f60 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.129+0000 7fe5a0511740 1 --2- 172.21.15.116:0/753573660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ae3edfc680 0x55ae3ee75e50 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.129+0000 7fe5a0511740 1 --2- 172.21.15.116:0/753573660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ae3edf1dc0 0x55ae3ee02f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.129+0000 7fe5a0511740 1 --2- 172.21.15.116:0/753573660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ae3edf14a0 0x55ae3ee029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.129+0000 7fe5a0511740 1 -- 172.21.15.116:0/753573660 >> 172.21.15.116:0/753573660 conn(0x55ae3ededa30 msgr2=0x55ae3edec350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:40.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.129+0000 7fe576ffd640 0 client.19398 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:40.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.129+0000 7fe5a0511740 1 -- 172.21.15.116:0/753573660 shutdown_connections 2024-09-12T03:17:40.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.129+0000 7fe5a0511740 1 -- 172.21.15.116:0/753573660 wait complete. 2024-09-12T03:17:40.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.129+0000 7fe5a0511740 20 client.19398 trim_cache size 0 max 16384 2024-09-12T03:17:40.137 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:17:40.137 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:17:40.137 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:17:40.137 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:17:40.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.130+0000 7fe59e138640 1 -- 172.21.15.116:0/150375229 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe5800a2b80 msgr2=0x7fe5800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:40.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.130+0000 7fe59e138640 1 --2- 172.21.15.116:0/150375229 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe5800a2b80 0x7fe5800a5040 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe590005ab0 tx=0x7fe590031510 comp rx=0 tx=0).stop 2024-09-12T03:17:40.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.130+0000 7fe59e138640 1 -- 172.21.15.116:0/150375229 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe598103660 msgr2=0x7fe5981989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:40.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.130+0000 7fe59e138640 1 --2- 172.21.15.116:0/150375229 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe598103660 0x7fe5981989b0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fe58c00b3f0 tx=0x7fe58c00b8b0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.130+0000 7fe59e138640 1 -- 172.21.15.116:0/150375229 shutdown_connections 2024-09-12T03:17:40.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.130+0000 7fe59e138640 1 --2- 172.21.15.116:0/150375229 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fe5800a2b80 0x7fe5800a5040 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.130+0000 7fe59e138640 1 --2- 172.21.15.116:0/150375229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe59810c170 0x7fe59819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.130+0000 7fe59e138640 1 --2- 172.21.15.116:0/150375229 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe598103660 0x7fe5981989b0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.130+0000 7fe59e138640 1 --2- 172.21.15.116:0/150375229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe598102d40 0x7fe598198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.130+0000 7fe59e138640 1 -- 172.21.15.116:0/150375229 >> 172.21.15.116:0/150375229 conn(0x7fe5980fc7a0 msgr2=0x7fe5980fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:40.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.130+0000 7fe59e138640 1 -- 172.21.15.116:0/150375229 shutdown_connections 2024-09-12T03:17:40.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:40.130+0000 7fe59e138640 1 -- 172.21.15.116:0/150375229 wait complete. 2024-09-12T03:17:40.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:39 smithi183 ceph-mon[35535]: pgmap v778: 97 pgs: 97 active+clean; 6.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:17:40.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:39 smithi177 ceph-mon[35305]: pgmap v778: 97 pgs: 97 active+clean; 6.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:17:40.833 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:17:40.833 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:17:41.080 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.563+0000 7fbb0c6fb640 1 -- 172.21.15.116:0/885672344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb04100a80 msgr2=0x7fbb04100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.563+0000 7fbb0c6fb640 1 --2- 172.21.15.116:0/885672344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb04100a80 0x7fbb04100ee0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fbb00009930 tx=0x7fbb0002a420 comp rx=0 tx=0).stop 2024-09-12T03:17:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.563+0000 7fbb0c6fb640 1 -- 172.21.15.116:0/885672344 shutdown_connections 2024-09-12T03:17:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.563+0000 7fbb0c6fb640 1 --2- 172.21.15.116:0/885672344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb04101420 0x7fbb0410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.563+0000 7fbb0c6fb640 1 --2- 172.21.15.116:0/885672344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb04100a80 0x7fbb04100ee0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.563+0000 7fbb0c6fb640 1 --2- 172.21.15.116:0/885672344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb04106a80 0x7fbb04106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.563+0000 7fbb0c6fb640 1 -- 172.21.15.116:0/885672344 >> 172.21.15.116:0/885672344 conn(0x7fbb040fc7a0 msgr2=0x7fbb040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.563+0000 7fbb0c6fb640 1 -- 172.21.15.116:0/885672344 shutdown_connections 2024-09-12T03:17:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.563+0000 7fbb0c6fb640 1 -- 172.21.15.116:0/885672344 wait complete. 2024-09-12T03:17:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.564+0000 7fbb0c6fb640 1 Processor -- start 2024-09-12T03:17:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.564+0000 7fbb0c6fb640 1 -- start start 2024-09-12T03:17:41.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.564+0000 7fbb0c6fb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb04100a80 0x7fbb041a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.564+0000 7fbb0c6fb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb04101420 0x7fbb041a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.564+0000 7fbb0c6fb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb04106a80 0x7fbb0419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.564+0000 7fbb0c6fb640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbb04110f20 con 0x7fbb04100a80 2024-09-12T03:17:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.564+0000 7fbb0c6fb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbb04110da0 con 0x7fbb04101420 2024-09-12T03:17:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.564+0000 7fbb0c6fb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbb041110a0 con 0x7fbb04106a80 2024-09-12T03:17:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.564+0000 7fbb0a497640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb04100a80 0x7fbb041a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.564+0000 7fbb0a497640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb04100a80 0x7fbb041a0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46318/0 (socket says 172.21.15.116:46318) 2024-09-12T03:17:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.564+0000 7fbb0a497640 1 -- 172.21.15.116:0/1650635363 learned_addr learned my addr 172.21.15.116:0/1650635363 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.564+0000 7fbb0ac98640 1 --2- 172.21.15.116:0/1650635363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb04106a80 0x7fbb0419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.564+0000 7fbb09c96640 1 --2- 172.21.15.116:0/1650635363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb04101420 0x7fbb041a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.564+0000 7fbb0a497640 1 -- 172.21.15.116:0/1650635363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb04106a80 msgr2=0x7fbb0419a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.564+0000 7fbb0a497640 1 --2- 172.21.15.116:0/1650635363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb04106a80 0x7fbb0419a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.564+0000 7fbb0a497640 1 -- 172.21.15.116:0/1650635363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb04101420 msgr2=0x7fbb041a0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.564+0000 7fbb0a497640 1 --2- 172.21.15.116:0/1650635363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb04101420 0x7fbb041a0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.564+0000 7fbb0a497640 1 -- 172.21.15.116:0/1650635363 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0419ad10 con 0x7fbb04100a80 2024-09-12T03:17:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.564+0000 7fbb0a497640 1 --2- 172.21.15.116:0/1650635363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb04100a80 0x7fbb041a0420 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400b4a0 tx=0x7fbaf400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.565+0000 7fbafb7fe640 1 -- 172.21.15.116:0/1650635363 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4014070 con 0x7fbb04100a80 2024-09-12T03:17:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.565+0000 7fbafb7fe640 1 -- 172.21.15.116:0/1650635363 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaf4010af0 con 0x7fbb04100a80 2024-09-12T03:17:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.565+0000 7fbb0c6fb640 1 -- 172.21.15.116:0/1650635363 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb0419b000 con 0x7fbb04100a80 2024-09-12T03:17:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.565+0000 7fbafb7fe640 1 -- 172.21.15.116:0/1650635363 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4017d10 con 0x7fbb04100a80 2024-09-12T03:17:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.565+0000 7fbb0c6fb640 1 -- 172.21.15.116:0/1650635363 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb041a72f0 con 0x7fbb04100a80 2024-09-12T03:17:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.566+0000 7fbb0c6fb640 1 -- 172.21.15.116:0/1650635363 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb04100f60 con 0x7fbb04100a80 2024-09-12T03:17:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.567+0000 7fbafb7fe640 1 -- 172.21.15.116:0/1650635363 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fbaf4017070 con 0x7fbb04100a80 2024-09-12T03:17:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.568+0000 7fbafb7fe640 1 --2- 172.21.15.116:0/1650635363 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbaf00a2b80 0x7fbaf00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.568+0000 7fbb09c96640 1 --2- 172.21.15.116:0/1650635363 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbaf00a2b80 0x7fbaf00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:41.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.568+0000 7fbafb7fe640 1 -- 172.21.15.116:0/1650635363 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fbaf40ce8a0 con 0x7fbb04100a80 2024-09-12T03:17:41.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.568+0000 7fbb09c96640 1 --2- 172.21.15.116:0/1650635363 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbaf00a2b80 0x7fbaf00a5040 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fbb00006010 tx=0x7fbb00000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:41.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.571+0000 7fbafb7fe640 1 -- 172.21.15.116:0/1650635363 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbaf409b830 con 0x7fbb04100a80 2024-09-12T03:17:41.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:41 smithi177 ceph-mon[35305]: pgmap v779: 97 pgs: 97 active+clean; 6.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T03:17:41.775 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.775+0000 7fbb0c6fb640 1 -- 172.21.15.116:0/1650635363 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb0419bb10 con 0x7fbb04100a80 2024-09-12T03:17:41.776 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:41 smithi116 ceph-mon[31410]: pgmap v779: 97 pgs: 97 active+clean; 6.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T03:17:41.776 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.776+0000 7fbafb7fe640 1 -- 172.21.15.116:0/1650635363 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7fbaf409b130 con 0x7fbb04100a80 2024-09-12T03:17:41.777 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:17:41.777 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:17:41.777 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:17:41.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.779+0000 7fbb0c6fb640 1 -- 172.21.15.116:0/1650635363 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbaf00a2b80 msgr2=0x7fbaf00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:41.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.779+0000 7fbb0c6fb640 1 --2- 172.21.15.116:0/1650635363 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbaf00a2b80 0x7fbaf00a5040 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fbb00006010 tx=0x7fbb00000bc0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.779+0000 7fbb0c6fb640 1 -- 172.21.15.116:0/1650635363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb04100a80 msgr2=0x7fbb041a0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:41.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.779+0000 7fbb0c6fb640 1 --2- 172.21.15.116:0/1650635363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb04100a80 0x7fbb041a0420 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400b4a0 tx=0x7fbaf400b960 comp rx=0 tx=0).stop 2024-09-12T03:17:41.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.779+0000 7fbb0c6fb640 1 -- 172.21.15.116:0/1650635363 shutdown_connections 2024-09-12T03:17:41.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.779+0000 7fbb0c6fb640 1 --2- 172.21.15.116:0/1650635363 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbaf00a2b80 0x7fbaf00a5040 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.779+0000 7fbb0c6fb640 1 --2- 172.21.15.116:0/1650635363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb04106a80 0x7fbb0419a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.779+0000 7fbb0c6fb640 1 --2- 172.21.15.116:0/1650635363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb04101420 0x7fbb041a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.779+0000 7fbb0c6fb640 1 --2- 172.21.15.116:0/1650635363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb04100a80 0x7fbb041a0420 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.779+0000 7fbb0c6fb640 1 -- 172.21.15.116:0/1650635363 >> 172.21.15.116:0/1650635363 conn(0x7fbb040fc7a0 msgr2=0x7fbb0406a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:41.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.780+0000 7fbb0c6fb640 1 -- 172.21.15.116:0/1650635363 shutdown_connections 2024-09-12T03:17:41.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:41.780+0000 7fbb0c6fb640 1 -- 172.21.15.116:0/1650635363 wait complete. 2024-09-12T03:17:41.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:41 smithi183 ceph-mon[35535]: pgmap v779: 97 pgs: 97 active+clean; 6.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T03:17:42.483 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:17:42.736 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:42.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:42 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1650635363' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:17:42.758 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:42 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1650635363' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:17:42.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:42 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1650635363' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:17:43.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.234+0000 7fea70227640 1 -- 172.21.15.116:0/235958810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea68102d50 msgr2=0x7fea68103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.234+0000 7fea70227640 1 --2- 172.21.15.116:0/235958810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea68102d50 0x7fea68103130 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fea64009900 tx=0x7fea6402a480 comp rx=0 tx=0).stop 2024-09-12T03:17:43.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.234+0000 7fea70227640 1 -- 172.21.15.116:0/235958810 shutdown_connections 2024-09-12T03:17:43.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.234+0000 7fea70227640 1 --2- 172.21.15.116:0/235958810 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea6810c180 0x7fea6810e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.234+0000 7fea70227640 1 --2- 172.21.15.116:0/235958810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea68103670 0x7fea6810bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.234+0000 7fea70227640 1 --2- 172.21.15.116:0/235958810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea68102d50 0x7fea68103130 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.234+0000 7fea70227640 1 -- 172.21.15.116:0/235958810 >> 172.21.15.116:0/235958810 conn(0x7fea680fc7d0 msgr2=0x7fea680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:43.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.234+0000 7fea70227640 1 -- 172.21.15.116:0/235958810 shutdown_connections 2024-09-12T03:17:43.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.234+0000 7fea70227640 1 -- 172.21.15.116:0/235958810 wait complete. 2024-09-12T03:17:43.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.235+0000 7fea70227640 1 Processor -- start 2024-09-12T03:17:43.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.235+0000 7fea70227640 1 -- start start 2024-09-12T03:17:43.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.235+0000 7fea70227640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea68102d50 0x7fea681009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.235+0000 7fea70227640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea68103670 0x7fea68100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.235+0000 7fea70227640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea6810c180 0x7fea68101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.235+0000 7fea70227640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fea68111f80 con 0x7fea6810c180 2024-09-12T03:17:43.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.235+0000 7fea70227640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fea68111e00 con 0x7fea68103670 2024-09-12T03:17:43.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.235+0000 7fea70227640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fea68112100 con 0x7fea68102d50 2024-09-12T03:17:43.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.235+0000 7fea6e7c4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea6810c180 0x7fea68101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.235+0000 7fea6e7c4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea6810c180 0x7fea68101430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46338/0 (socket says 172.21.15.116:46338) 2024-09-12T03:17:43.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.235+0000 7fea6e7c4640 1 -- 172.21.15.116:0/4078503750 learned_addr learned my addr 172.21.15.116:0/4078503750 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:43.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.235+0000 7fea6d7c2640 1 --2- 172.21.15.116:0/4078503750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea68103670 0x7fea68100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.235+0000 7fea6dfc3640 1 --2- 172.21.15.116:0/4078503750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea68102d50 0x7fea681009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.236+0000 7fea6e7c4640 1 -- 172.21.15.116:0/4078503750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea68102d50 msgr2=0x7fea681009b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.236+0000 7fea6e7c4640 1 --2- 172.21.15.116:0/4078503750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea68102d50 0x7fea681009b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.236+0000 7fea6e7c4640 1 -- 172.21.15.116:0/4078503750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea68103670 msgr2=0x7fea68100ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.236+0000 7fea6e7c4640 1 --2- 172.21.15.116:0/4078503750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea68103670 0x7fea68100ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.236+0000 7fea6e7c4640 1 -- 172.21.15.116:0/4078503750 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea681a6850 con 0x7fea6810c180 2024-09-12T03:17:43.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.236+0000 7fea6dfc3640 1 --2- 172.21.15.116:0/4078503750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea68102d50 0x7fea681009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:43.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.236+0000 7fea6e7c4640 1 --2- 172.21.15.116:0/4078503750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea6810c180 0x7fea68101430 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fea6000cce0 tx=0x7fea6000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:43.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.236+0000 7fea5effd640 1 -- 172.21.15.116:0/4078503750 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6000f450 con 0x7fea6810c180 2024-09-12T03:17:43.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.236+0000 7fea70227640 1 -- 172.21.15.116:0/4078503750 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea681a6b40 con 0x7fea6810c180 2024-09-12T03:17:43.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.236+0000 7fea5effd640 1 -- 172.21.15.116:0/4078503750 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea6000fad0 con 0x7fea6810c180 2024-09-12T03:17:43.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.236+0000 7fea5effd640 1 -- 172.21.15.116:0/4078503750 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea60018c10 con 0x7fea6810c180 2024-09-12T03:17:43.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.236+0000 7fea70227640 1 -- 172.21.15.116:0/4078503750 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea681a7030 con 0x7fea6810c180 2024-09-12T03:17:43.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.239+0000 7fea5effd640 1 -- 172.21.15.116:0/4078503750 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fea6000f5f0 con 0x7fea6810c180 2024-09-12T03:17:43.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.239+0000 7fea72600740 1 Processor -- start 2024-09-12T03:17:43.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.239+0000 7fea72600740 1 -- start start 2024-09-12T03:17:43.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.239+0000 7fea72600740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558d2f362b40 0x558d2f361090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.239+0000 7fea72600740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d2f3615d0 0x558d2f361a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.239+0000 7fea72600740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d2f36c370 0x558d2f36e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.239+0000 7fea72600740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x558d2f2c72f0 con 0x558d2f362b40 2024-09-12T03:17:43.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.239+0000 7fea72600740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558d2f26ecf0 con 0x558d2f3615d0 2024-09-12T03:17:43.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.239+0000 7fea72600740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558d2f2c32b0 con 0x558d2f36c370 2024-09-12T03:17:43.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.239+0000 7fea6d7c2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558d2f362b40 0x558d2f361090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.239+0000 7fea6d7c2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558d2f362b40 0x558d2f361090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46340/0 (socket says 172.21.15.116:46340) 2024-09-12T03:17:43.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.239+0000 7fea6d7c2640 1 -- 172.21.15.116:0/3438098202 learned_addr learned my addr 172.21.15.116:0/3438098202 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:43.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.239+0000 7fea5effd640 1 --2- 172.21.15.116:0/4078503750 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fea540a2b80 0x7fea540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.240+0000 7fea6dfc3640 1 --2- 172.21.15.116:0/3438098202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d2f3615d0 0x558d2f361a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.240+0000 7fea6d7c2640 1 --2- 172.21.15.116:0/3438098202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d2f36c370 0x558d2f36e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.240+0000 7fea6e7c4640 1 --2- 172.21.15.116:0/4078503750 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fea540a2b80 0x7fea540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.240+0000 7fea6d7c2640 1 -- 172.21.15.116:0/3438098202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d2f36c370 msgr2=0x558d2f36e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:43.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.240+0000 7fea6d7c2640 1 --2- 172.21.15.116:0/3438098202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d2f36c370 0x558d2f36e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.240+0000 7fea6d7c2640 1 -- 172.21.15.116:0/3438098202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d2f3615d0 msgr2=0x558d2f361a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:43.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.240+0000 7fea6d7c2640 1 --2- 172.21.15.116:0/3438098202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d2f3615d0 0x558d2f361a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.240+0000 7fea5effd640 1 -- 172.21.15.116:0/4078503750 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fea600cf690 con 0x7fea6810c180 2024-09-12T03:17:43.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.240+0000 7fea6d7c2640 1 -- 172.21.15.116:0/3438098202 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d2f2533a0 con 0x558d2f362b40 2024-09-12T03:17:43.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.240+0000 7fea6d7c2640 1 --2- 172.21.15.116:0/3438098202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558d2f362b40 0x558d2f361090 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fea5800afc0 tx=0x7fea5800a7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d53bdf9946c9a79 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:43.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.240+0000 7fea53fff640 1 -- 172.21.15.116:0/3438098202 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea58019030 con 0x558d2f362b40 2024-09-12T03:17:43.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.240+0000 7fea53fff640 1 -- 172.21.15.116:0/3438098202 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea580110d0 con 0x558d2f362b40 2024-09-12T03:17:43.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.240+0000 7fea53fff640 1 -- 172.21.15.116:0/3438098202 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5800f8b0 con 0x558d2f362b40 2024-09-12T03:17:43.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.241+0000 7fea72600740 1 -- 172.21.15.116:0/3438098202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558d2f362b40 msgr2=0x558d2f361090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.241+0000 7fea72600740 1 --2- 172.21.15.116:0/3438098202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558d2f362b40 0x558d2f361090 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fea5800afc0 tx=0x7fea5800a7e0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.241+0000 7fea6e7c4640 1 --2- 172.21.15.116:0/4078503750 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fea540a2b80 0x7fea540a5040 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fea60098380 tx=0x7fea600186b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:43.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.241+0000 7fea72600740 1 -- 172.21.15.116:0/3438098202 shutdown_connections 2024-09-12T03:17:43.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.241+0000 7fea72600740 1 --2- 172.21.15.116:0/3438098202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d2f36c370 0x558d2f36e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.241+0000 7fea72600740 1 --2- 172.21.15.116:0/3438098202 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d2f3615d0 0x558d2f361a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.257 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.241+0000 7fea72600740 1 --2- 172.21.15.116:0/3438098202 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558d2f362b40 0x558d2f361090 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.257 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.241+0000 7fea72600740 1 -- 172.21.15.116:0/3438098202 >> 172.21.15.116:0/3438098202 conn(0x558d2f32a310 msgr2=0x558d2f35dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:43.257 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.241+0000 7fea72600740 1 -- 172.21.15.116:0/3438098202 shutdown_connections 2024-09-12T03:17:43.257 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.241+0000 7fea72600740 1 -- 172.21.15.116:0/3438098202 wait complete. 2024-09-12T03:17:43.257 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.241+0000 7fea72600740 1 -- start start 2024-09-12T03:17:43.257 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.242+0000 7fea72600740 1 Processor -- start 2024-09-12T03:17:43.257 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.242+0000 7fea72600740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d2f3615d0 0x558d2f36bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.257 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.242+0000 7fea72600740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558d2f362b40 0x558d2f368fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.257 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.242+0000 7fea72600740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d2f36c370 0x558d2f369570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.257 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.242+0000 7fea72600740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x558d2f36f070 con 0x558d2f362b40 2024-09-12T03:17:43.257 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.242+0000 7fea72600740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558d2f282060 con 0x558d2f36c370 2024-09-12T03:17:43.257 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.242+0000 7fea72600740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558d2f36f1f0 con 0x558d2f3615d0 2024-09-12T03:17:43.257 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.242+0000 7fea72600740 10 client.0 _ll_register_callbacks cb 0x558d2efd58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:43.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.242+0000 7fea6dfc3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558d2f362b40 0x558d2f368fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.242+0000 7fea6dfc3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558d2f362b40 0x558d2f368fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46348/0 (socket says 172.21.15.116:46348) 2024-09-12T03:17:43.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.242+0000 7fea6dfc3640 1 -- 172.21.15.116:0/748861668 learned_addr learned my addr 172.21.15.116:0/748861668 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:43.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.243+0000 7fea6d7c2640 1 --2- 172.21.15.116:0/748861668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d2f36c370 0x558d2f369570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.243+0000 7fea6d7c2640 1 --2- 172.21.15.116:0/748861668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d2f3615d0 0x558d2f36bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.243+0000 7fea6dfc3640 1 -- 172.21.15.116:0/748861668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d2f3615d0 msgr2=0x558d2f36bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:43.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.243+0000 7fea6dfc3640 1 --2- 172.21.15.116:0/748861668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d2f3615d0 0x558d2f36bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.243+0000 7fea6dfc3640 1 -- 172.21.15.116:0/748861668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d2f36c370 msgr2=0x558d2f369570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:43.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.243+0000 7fea6dfc3640 1 --2- 172.21.15.116:0/748861668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d2f36c370 0x558d2f369570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.243+0000 7fea6dfc3640 1 -- 172.21.15.116:0/748861668 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d2f0383c0 con 0x558d2f362b40 2024-09-12T03:17:43.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.243+0000 7fea6d7c2640 1 --2- 172.21.15.116:0/748861668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d2f36c370 0x558d2f369570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:43.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.243+0000 7fea6dfc3640 1 --2- 172.21.15.116:0/748861668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558d2f362b40 0x558d2f368fd0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fea6402a9a0 tx=0x7fea64004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bce03f7f5956bdc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:43.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.243+0000 7fea517fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:43.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.243+0000 7fea517fa640 1 -- 172.21.15.116:0/748861668 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea64035040 con 0x558d2f362b40 2024-09-12T03:17:43.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.243+0000 7fea517fa640 1 -- 172.21.15.116:0/748861668 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fea64005020 con 0x558d2f362b40 2024-09-12T03:17:43.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.243+0000 7fea72600740 1 -- 172.21.15.116:0/748861668 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea580110d0 con 0x558d2f362b40 2024-09-12T03:17:43.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.243+0000 7fea517fa640 1 -- 172.21.15.116:0/748861668 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fea64005220 con 0x558d2f362b40 2024-09-12T03:17:43.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.243+0000 7fea72600740 10 client.19416 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:43.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.243+0000 7fea72600740 1 -- 172.21.15.116:0/748861668 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558d2f37aac0 con 0x558d2f362b40 2024-09-12T03:17:43.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.243+0000 7fea517fa640 1 -- 172.21.15.116:0/748861668 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea580110d0 con 0x558d2f362b40 2024-09-12T03:17:43.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.244+0000 7fea517fa640 1 -- 172.21.15.116:0/748861668 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fea64003710 con 0x558d2f362b40 2024-09-12T03:17:43.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.244+0000 7fea72600740 10 client.19416 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:43.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.244+0000 7fea72600740 10 client.19416 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:17:43.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.244+0000 7fea72600740 20 client.19416 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:43.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.244+0000 7fea72600740 1 --2- 172.21.15.116:0/748861668 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x558d2f37bf80 0x558d2f37e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.244+0000 7fea72600740 4 client.19416 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:17:43.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.244+0000 7fea72600740 1 -- 172.21.15.116:0/748861668 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d2f2d4860 con 0x558d2f37bf80 2024-09-12T03:17:43.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.244+0000 7fea6d7c2640 1 --2- 172.21.15.116:0/748861668 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x558d2f37bf80 0x558d2f37e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.245+0000 7fea6d7c2640 1 --2- 172.21.15.116:0/748861668 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x558d2f37bf80 0x558d2f37e3a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b0c355d30d6b356 server_cookie=6a9042901fc4a32d in_seq=0 out_seq=0 2024-09-12T03:17:43.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.245+0000 7fea517fa640 10 client.19416 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:17:43.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.245+0000 7fea517fa640 1 -- 172.21.15.116:0/748861668 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558d2f2d4860 con 0x558d2f37bf80 2024-09-12T03:17:43.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.245+0000 7fea517fa640 10 client.19416 handle_command_reply: tid=0 2024-09-12T03:17:43.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.245+0000 7fea72600740 1 client.19416 shutdown 2024-09-12T03:17:43.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.245+0000 7fea72600740 1 -- 172.21.15.116:0/748861668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558d2f362b40 msgr2=0x558d2f368fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.245+0000 7fea72600740 1 --2- 172.21.15.116:0/748861668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558d2f362b40 0x558d2f368fd0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fea6402a9a0 tx=0x7fea64004270 comp rx=0 tx=0).stop 2024-09-12T03:17:43.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.246+0000 7fea72600740 1 -- 172.21.15.116:0/748861668 shutdown_connections 2024-09-12T03:17:43.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.246+0000 7fea72600740 1 --2- 172.21.15.116:0/748861668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d2f36c370 0x558d2f369570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.246+0000 7fea72600740 1 --2- 172.21.15.116:0/748861668 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x558d2f37bf80 0x558d2f37e3a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.246+0000 7fea72600740 1 --2- 172.21.15.116:0/748861668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558d2f362b40 0x558d2f368fd0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.246+0000 7fea72600740 1 --2- 172.21.15.116:0/748861668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d2f3615d0 0x558d2f36bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.246+0000 7fea517fa640 0 client.19416 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:17:43.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.246+0000 7fea72600740 1 -- 172.21.15.116:0/748861668 >> 172.21.15.116:0/748861668 conn(0x558d2f32a310 msgr2=0x558d2f363930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:43.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.246+0000 7fea72600740 1 -- 172.21.15.116:0/748861668 shutdown_connections 2024-09-12T03:17:43.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.246+0000 7fea72600740 1 -- 172.21.15.116:0/748861668 wait complete. 2024-09-12T03:17:43.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.246+0000 7fea72600740 20 client.19416 trim_cache size 0 max 16384 2024-09-12T03:17:43.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.259+0000 7fea72600740 1 Processor -- start 2024-09-12T03:17:43.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.259+0000 7fea72600740 1 -- start start 2024-09-12T03:17:43.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.259+0000 7fea72600740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d2f3615d0 0x558d2f378310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.259+0000 7fea72600740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558d2f362b40 0x558d2f367fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.259+0000 7fea72600740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d2f36c370 0x558d2f383c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.259+0000 7fea72600740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fea681b1e50 con 0x558d2f362b40 2024-09-12T03:17:43.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.259+0000 7fea72600740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fea681af720 con 0x558d2f36c370 2024-09-12T03:17:43.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.259+0000 7fea72600740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fea681ad570 con 0x558d2f3615d0 2024-09-12T03:17:43.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.260+0000 7fea6dfc3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558d2f362b40 0x558d2f367fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.260+0000 7fea6dfc3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558d2f362b40 0x558d2f367fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46356/0 (socket says 172.21.15.116:46356) 2024-09-12T03:17:43.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.260+0000 7fea6dfc3640 1 -- 172.21.15.116:0/3043927855 learned_addr learned my addr 172.21.15.116:0/3043927855 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:43.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.260+0000 7fea6d7c2640 1 --2- 172.21.15.116:0/3043927855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d2f3615d0 0x558d2f378310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.260+0000 7fea6dfc3640 1 -- 172.21.15.116:0/3043927855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d2f3615d0 msgr2=0x558d2f378310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:43.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.260+0000 7fea6dfc3640 1 --2- 172.21.15.116:0/3043927855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d2f3615d0 0x558d2f378310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.260+0000 7fea6dfc3640 1 -- 172.21.15.116:0/3043927855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d2f36c370 msgr2=0x558d2f383c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:43.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.260+0000 7fea6dfc3640 1 --2- 172.21.15.116:0/3043927855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d2f36c370 0x558d2f383c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.260+0000 7fea6dfc3640 1 -- 172.21.15.116:0/3043927855 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d2f35c310 con 0x558d2f362b40 2024-09-12T03:17:43.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.260+0000 7fea6d7c2640 1 --2- 172.21.15.116:0/3043927855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d2f3615d0 0x558d2f378310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:43.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.260+0000 7fea6dfc3640 1 --2- 172.21.15.116:0/3043927855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558d2f362b40 0x558d2f367fb0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fea64031550 tx=0x7fea6402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb6587399fb6d50 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:43.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.260+0000 7fea537fe640 1 -- 172.21.15.116:0/3043927855 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea64035070 con 0x558d2f362b40 2024-09-12T03:17:43.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.260+0000 7fea537fe640 1 -- 172.21.15.116:0/3043927855 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea640042e0 con 0x558d2f362b40 2024-09-12T03:17:43.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.260+0000 7fea537fe640 1 -- 172.21.15.116:0/3043927855 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea64045d30 con 0x558d2f362b40 2024-09-12T03:17:43.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.261+0000 7fea72600740 1 -- 172.21.15.116:0/3043927855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558d2f362b40 msgr2=0x558d2f367fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.261+0000 7fea72600740 1 --2- 172.21.15.116:0/3043927855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558d2f362b40 0x558d2f367fb0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fea64031550 tx=0x7fea6402a480 comp rx=0 tx=0).stop 2024-09-12T03:17:43.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.261+0000 7fea72600740 1 -- 172.21.15.116:0/3043927855 shutdown_connections 2024-09-12T03:17:43.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.261+0000 7fea72600740 1 --2- 172.21.15.116:0/3043927855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d2f36c370 0x558d2f383c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.261+0000 7fea72600740 1 --2- 172.21.15.116:0/3043927855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558d2f362b40 0x558d2f367fb0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.261+0000 7fea72600740 1 --2- 172.21.15.116:0/3043927855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d2f3615d0 0x558d2f378310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.261+0000 7fea72600740 1 -- 172.21.15.116:0/3043927855 >> 172.21.15.116:0/3043927855 conn(0x558d2f32a310 msgr2=0x558d2f3678a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:43.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.261+0000 7fea72600740 1 -- 172.21.15.116:0/3043927855 shutdown_connections 2024-09-12T03:17:43.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.261+0000 7fea72600740 1 -- 172.21.15.116:0/3043927855 wait complete. 2024-09-12T03:17:43.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.261+0000 7fea72600740 1 -- start start 2024-09-12T03:17:43.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.262+0000 7fea72600740 1 Processor -- start 2024-09-12T03:17:43.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.262+0000 7fea72600740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558d2f3615d0 0x558d2f370100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.262+0000 7fea72600740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d2f362b40 0x558d2f370640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.262+0000 7fea72600740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d2f36c370 0x558d2f370b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.262+0000 7fea72600740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fea681adbe0 con 0x558d2f3615d0 2024-09-12T03:17:43.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.262+0000 7fea72600740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fea681af050 con 0x558d2f362b40 2024-09-12T03:17:43.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.262+0000 7fea72600740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fea681ae2b0 con 0x558d2f36c370 2024-09-12T03:17:43.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.262+0000 7fea72600740 10 client.0 _ll_register_callbacks cb 0x7fea64031a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:43.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.262+0000 7fea6d7c2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558d2f3615d0 0x558d2f370100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.262+0000 7fea6d7c2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558d2f3615d0 0x558d2f370100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46366/0 (socket says 172.21.15.116:46366) 2024-09-12T03:17:43.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.262+0000 7fea6d7c2640 1 -- 172.21.15.116:0/1852478740 learned_addr learned my addr 172.21.15.116:0/1852478740 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:43.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.262+0000 7fea6dfc3640 1 --2- 172.21.15.116:0/1852478740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d2f362b40 0x558d2f370640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.262+0000 7fea6d7c2640 1 --2- 172.21.15.116:0/1852478740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d2f36c370 0x558d2f370b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.262+0000 7fea6d7c2640 1 -- 172.21.15.116:0/1852478740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d2f36c370 msgr2=0x558d2f370b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:43.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.262+0000 7fea6d7c2640 1 --2- 172.21.15.116:0/1852478740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d2f36c370 0x558d2f370b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.262+0000 7fea6d7c2640 1 -- 172.21.15.116:0/1852478740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d2f362b40 msgr2=0x558d2f370640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:43.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.262+0000 7fea6d7c2640 1 --2- 172.21.15.116:0/1852478740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d2f362b40 0x558d2f370640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.262+0000 7fea6d7c2640 1 -- 172.21.15.116:0/1852478740 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d2f3e5c40 con 0x558d2f3615d0 2024-09-12T03:17:43.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.263+0000 7fea6d7c2640 1 --2- 172.21.15.116:0/1852478740 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558d2f3615d0 0x558d2f370100 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fea58013550 tx=0x7fea58005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989cb73e4f145ba7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:43.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.263+0000 7fea517fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:43.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.263+0000 7fea517fa640 1 -- 172.21.15.116:0/1852478740 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5802d060 con 0x558d2f3615d0 2024-09-12T03:17:43.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.263+0000 7fea517fa640 1 -- 172.21.15.116:0/1852478740 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fea580110b0 con 0x558d2f3615d0 2024-09-12T03:17:43.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.263+0000 7fea72600740 1 -- 172.21.15.116:0/1852478740 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea640042e0 con 0x558d2f3615d0 2024-09-12T03:17:43.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.263+0000 7fea517fa640 1 -- 172.21.15.116:0/1852478740 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fea58019040 con 0x558d2f3615d0 2024-09-12T03:17:43.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.263+0000 7fea72600740 10 client.19422 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:43.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.263+0000 7fea72600740 1 -- 172.21.15.116:0/1852478740 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558d2f3e6db0 con 0x558d2f3615d0 2024-09-12T03:17:43.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.263+0000 7fea517fa640 1 -- 172.21.15.116:0/1852478740 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea640042e0 con 0x558d2f3615d0 2024-09-12T03:17:43.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.264+0000 7fea517fa640 1 -- 172.21.15.116:0/1852478740 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fea5800fa80 con 0x558d2f3615d0 2024-09-12T03:17:43.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.264+0000 7fea72600740 10 client.19422 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:43.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.264+0000 7fea72600740 10 client.19422 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:17:43.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.264+0000 7fea72600740 20 client.19422 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:43.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.264+0000 7fea72600740 1 --2- 172.21.15.116:0/1852478740 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x558d2f3e7150 0x558d2f3e9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.264+0000 7fea72600740 4 client.19422 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:17:43.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.264+0000 7fea72600740 1 -- 172.21.15.116:0/1852478740 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558d2f2ced70 con 0x558d2f3e7150 2024-09-12T03:17:43.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.264+0000 7fea6dfc3640 1 --2- 172.21.15.116:0/1852478740 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x558d2f3e7150 0x558d2f3e9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.264+0000 7fea517fa640 10 client.19422 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:17:43.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.264+0000 7fea6dfc3640 1 --2- 172.21.15.116:0/1852478740 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x558d2f3e7150 0x558d2f3e9540 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=108527b9e9d1fc5d server_cookie=9841783e0376e3a4 in_seq=0 out_seq=0 2024-09-12T03:17:43.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.264+0000 7fea517fa640 1 -- 172.21.15.116:0/1852478740 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558d2f2ced70 con 0x558d2f3e7150 2024-09-12T03:17:43.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.264+0000 7fea517fa640 10 client.19422 handle_command_reply: tid=0 2024-09-12T03:17:43.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.264+0000 7fea72600740 1 client.19422 shutdown 2024-09-12T03:17:43.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.265+0000 7fea72600740 1 -- 172.21.15.116:0/1852478740 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558d2f3615d0 msgr2=0x558d2f370100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.265+0000 7fea72600740 1 --2- 172.21.15.116:0/1852478740 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558d2f3615d0 0x558d2f370100 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fea58013550 tx=0x7fea58005850 comp rx=0 tx=0).stop 2024-09-12T03:17:43.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.265+0000 7fea72600740 1 -- 172.21.15.116:0/1852478740 shutdown_connections 2024-09-12T03:17:43.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.265+0000 7fea72600740 1 --2- 172.21.15.116:0/1852478740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d2f36c370 0x558d2f370b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.265+0000 7fea72600740 1 --2- 172.21.15.116:0/1852478740 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558d2f362b40 0x558d2f370640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.265+0000 7fea72600740 1 --2- 172.21.15.116:0/1852478740 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x558d2f3e7150 0x558d2f3e9540 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.265+0000 7fea72600740 1 --2- 172.21.15.116:0/1852478740 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558d2f3615d0 0x558d2f370100 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.265+0000 7fea517fa640 0 client.19422 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:17:43.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.265+0000 7fea72600740 1 -- 172.21.15.116:0/1852478740 >> 172.21.15.116:0/1852478740 conn(0x558d2f32a310 msgr2=0x558d2f363110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:43.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.265+0000 7fea72600740 1 -- 172.21.15.116:0/1852478740 shutdown_connections 2024-09-12T03:17:43.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.265+0000 7fea72600740 1 -- 172.21.15.116:0/1852478740 wait complete. 2024-09-12T03:17:43.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.265+0000 7fea72600740 20 client.19422 trim_cache size 0 max 16384 2024-09-12T03:17:43.273 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:17:43.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.265+0000 7fea70227640 1 -- 172.21.15.116:0/4078503750 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fea540a2b80 msgr2=0x7fea540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.265+0000 7fea70227640 1 --2- 172.21.15.116:0/4078503750 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fea540a2b80 0x7fea540a5040 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fea60098380 tx=0x7fea600186b0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.265+0000 7fea70227640 1 -- 172.21.15.116:0/4078503750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea6810c180 msgr2=0x7fea68101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.265+0000 7fea70227640 1 --2- 172.21.15.116:0/4078503750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea6810c180 0x7fea68101430 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fea6000cce0 tx=0x7fea6000a620 comp rx=0 tx=0).stop 2024-09-12T03:17:43.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.266+0000 7fea70227640 1 -- 172.21.15.116:0/4078503750 shutdown_connections 2024-09-12T03:17:43.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.266+0000 7fea70227640 1 --2- 172.21.15.116:0/4078503750 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fea540a2b80 0x7fea540a5040 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.266+0000 7fea70227640 1 --2- 172.21.15.116:0/4078503750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea6810c180 0x7fea68101430 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.266+0000 7fea70227640 1 --2- 172.21.15.116:0/4078503750 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea68103670 0x7fea68100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.266+0000 7fea70227640 1 --2- 172.21.15.116:0/4078503750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea68102d50 0x7fea681009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.266+0000 7fea70227640 1 -- 172.21.15.116:0/4078503750 >> 172.21.15.116:0/4078503750 conn(0x7fea680fc7d0 msgr2=0x7fea680fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:43.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.266+0000 7fea70227640 1 -- 172.21.15.116:0/4078503750 shutdown_connections 2024-09-12T03:17:43.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:43.266+0000 7fea70227640 1 -- 172.21.15.116:0/4078503750 wait complete. 2024-09-12T03:17:43.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:43 smithi177 ceph-mon[35305]: pgmap v780: 97 pgs: 97 active+clean; 6.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:17:43.751 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:43 smithi116 ceph-mon[31410]: pgmap v780: 97 pgs: 97 active+clean; 6.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:17:43.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:43 smithi183 ceph-mon[35535]: pgmap v780: 97 pgs: 97 active+clean; 6.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:17:44.284 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:17:44.535 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:45.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.044+0000 7f5cdbcb7640 1 -- 172.21.15.116:0/1716769963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cd4103760 msgr2=0x7f5cd4107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:45.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.044+0000 7f5cdbcb7640 1 --2- 172.21.15.116:0/1716769963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cd4103760 0x7f5cd4107ba0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4009930 tx=0x7f5cc402a420 comp rx=0 tx=0).stop 2024-09-12T03:17:45.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.044+0000 7f5cdbcb7640 1 -- 172.21.15.116:0/1716769963 shutdown_connections 2024-09-12T03:17:45.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.044+0000 7f5cdbcb7640 1 --2- 172.21.15.116:0/1716769963 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cd41082e0 0x7f5cd410ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.044+0000 7f5cdbcb7640 1 --2- 172.21.15.116:0/1716769963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cd4103760 0x7f5cd4107ba0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.044+0000 7f5cdbcb7640 1 --2- 172.21.15.116:0/1716769963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cd4102db0 0x7f5cd4103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.044+0000 7f5cdbcb7640 1 -- 172.21.15.116:0/1716769963 >> 172.21.15.116:0/1716769963 conn(0x7f5cd40fc7d0 msgr2=0x7f5cd40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:45.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.044+0000 7f5cdbcb7640 1 -- 172.21.15.116:0/1716769963 shutdown_connections 2024-09-12T03:17:45.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.044+0000 7f5cdbcb7640 1 -- 172.21.15.116:0/1716769963 wait complete. 2024-09-12T03:17:45.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.044+0000 7f5cdbcb7640 1 Processor -- start 2024-09-12T03:17:45.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.045+0000 7f5cdbcb7640 1 -- start start 2024-09-12T03:17:45.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.045+0000 7f5cdbcb7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cd4102db0 0x7f5cd4198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:45.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.045+0000 7f5cdbcb7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cd4103760 0x7f5cd4198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:45.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.045+0000 7f5cdbcb7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cd41082e0 0x7f5cd419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:45.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.045+0000 7f5cdbcb7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd4112070 con 0x7f5cd4102db0 2024-09-12T03:17:45.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.045+0000 7f5cdbcb7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd4111ef0 con 0x7f5cd41082e0 2024-09-12T03:17:45.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.045+0000 7f5cdbcb7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd41121f0 con 0x7f5cd4103760 2024-09-12T03:17:45.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.045+0000 7f5cd9a53640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cd4102db0 0x7f5cd4198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:45.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.045+0000 7f5cd9a53640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cd4102db0 0x7f5cd4198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46372/0 (socket says 172.21.15.116:46372) 2024-09-12T03:17:45.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.045+0000 7f5cd9a53640 1 -- 172.21.15.116:0/3207778658 learned_addr learned my addr 172.21.15.116:0/3207778658 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:45.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.045+0000 7f5cd9252640 1 --2- 172.21.15.116:0/3207778658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cd4103760 0x7f5cd4198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:45.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.045+0000 7f5cda254640 1 --2- 172.21.15.116:0/3207778658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cd41082e0 0x7f5cd419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:45.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.045+0000 7f5cd9a53640 1 -- 172.21.15.116:0/3207778658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cd4103760 msgr2=0x7f5cd4198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:45.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.045+0000 7f5cd9a53640 1 --2- 172.21.15.116:0/3207778658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cd4103760 0x7f5cd4198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.045+0000 7f5cd9a53640 1 -- 172.21.15.116:0/3207778658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cd41082e0 msgr2=0x7f5cd419ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:45.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.045+0000 7f5cd9a53640 1 --2- 172.21.15.116:0/3207778658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cd41082e0 0x7f5cd419ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.045+0000 7f5cd9a53640 1 -- 172.21.15.116:0/3207778658 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd419d530 con 0x7f5cd4102db0 2024-09-12T03:17:45.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.045+0000 7f5cd9a53640 1 --2- 172.21.15.116:0/3207778658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cd4102db0 0x7f5cd4198580 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f5cd000d770 tx=0x7f5cd000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:45.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.045+0000 7f5ccaffd640 1 -- 172.21.15.116:0/3207778658 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0017070 con 0x7f5cd4102db0 2024-09-12T03:17:45.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.045+0000 7f5cdbcb7640 1 -- 172.21.15.116:0/3207778658 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cd419d820 con 0x7f5cd4102db0 2024-09-12T03:17:45.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.046+0000 7f5ccaffd640 1 -- 172.21.15.116:0/3207778658 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cd0005500 con 0x7f5cd4102db0 2024-09-12T03:17:45.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.046+0000 7f5ccaffd640 1 -- 172.21.15.116:0/3207778658 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0004e00 con 0x7f5cd4102db0 2024-09-12T03:17:45.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.046+0000 7f5cdbcb7640 1 -- 172.21.15.116:0/3207778658 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cd41a5100 con 0x7f5cd4102db0 2024-09-12T03:17:45.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.048+0000 7f5ccaffd640 1 -- 172.21.15.116:0/3207778658 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f5cd0004fa0 con 0x7f5cd4102db0 2024-09-12T03:17:45.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.048+0000 7f5cde090740 1 Processor -- start 2024-09-12T03:17:45.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.048+0000 7f5cde090740 1 -- start start 2024-09-12T03:17:45.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.048+0000 7f5cde090740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c37d0b40 0x55c2c37cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:45.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.048+0000 7f5cde090740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c37cf5d0 0x55c2c37cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:45.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.048+0000 7f5cde090740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c37da370 0x55c2c37dc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:45.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.048+0000 7f5cde090740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c2c37352f0 con 0x55c2c37cf5d0 2024-09-12T03:17:45.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.048+0000 7f5cde090740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c2c36dccf0 con 0x55c2c37d0b40 2024-09-12T03:17:45.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.048+0000 7f5cde090740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c2c37312b0 con 0x55c2c37da370 2024-09-12T03:17:45.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.048+0000 7f5ccaffd640 1 --2- 172.21.15.116:0/3207778658 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5cc00a2b80 0x7f5cc00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:45.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.049+0000 7f5cd9a53640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c37cf5d0 0x55c2c37cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:45.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.049+0000 7f5cda254640 1 --2- 172.21.15.116:0/3207778658 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5cc00a2b80 0x7f5cc00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:45.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.049+0000 7f5cd9a53640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c37cf5d0 0x55c2c37cfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46384/0 (socket says 172.21.15.116:46384) 2024-09-12T03:17:45.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.049+0000 7f5cd9a53640 1 -- 172.21.15.116:0/1048312193 learned_addr learned my addr 172.21.15.116:0/1048312193 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:45.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.049+0000 7f5cda254640 1 --2- 172.21.15.116:0/1048312193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c37d0b40 0x55c2c37cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:45.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.049+0000 7f5ccaffd640 1 -- 172.21.15.116:0/3207778658 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f5cd00cea40 con 0x7f5cd4102db0 2024-09-12T03:17:45.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.049+0000 7f5cd9252640 1 --2- 172.21.15.116:0/1048312193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c37da370 0x55c2c37dc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:45.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.049+0000 7f5cd9a53640 1 -- 172.21.15.116:0/1048312193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c37da370 msgr2=0x55c2c37dc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:45.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.049+0000 7f5cd9a53640 1 --2- 172.21.15.116:0/1048312193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c37da370 0x55c2c37dc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.049+0000 7f5cd9a53640 1 -- 172.21.15.116:0/1048312193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c37d0b40 msgr2=0x55c2c37cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:45.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.049+0000 7f5cd9a53640 1 --2- 172.21.15.116:0/1048312193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c37d0b40 0x55c2c37cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.049+0000 7f5cd9a53640 1 -- 172.21.15.116:0/1048312193 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2c36c13a0 con 0x55c2c37cf5d0 2024-09-12T03:17:45.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.049+0000 7f5cd9252640 1 --2- 172.21.15.116:0/1048312193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c37da370 0x55c2c37dc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:45.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.049+0000 7f5cda254640 1 --2- 172.21.15.116:0/3207778658 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5cc00a2b80 0x7f5cc00a5040 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc007fd0 tx=0x7f5ccc007c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:45.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.049+0000 7f5cd9a53640 1 --2- 172.21.15.116:0/1048312193 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c37cf5d0 0x55c2c37cfa10 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0009610 tx=0x7f5cd0050a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f7df164ed2d185 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:45.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.049+0000 7f5cbffff640 1 -- 172.21.15.116:0/1048312193 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0020070 con 0x55c2c37cf5d0 2024-09-12T03:17:45.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.049+0000 7f5cbffff640 1 -- 172.21.15.116:0/1048312193 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cd00046c0 con 0x55c2c37cf5d0 2024-09-12T03:17:45.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.049+0000 7f5cbffff640 1 -- 172.21.15.116:0/1048312193 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd00995d0 con 0x55c2c37cf5d0 2024-09-12T03:17:45.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.050+0000 7f5cde090740 1 -- 172.21.15.116:0/1048312193 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c37cf5d0 msgr2=0x55c2c37cfa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:45.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.050+0000 7f5cde090740 1 --2- 172.21.15.116:0/1048312193 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c37cf5d0 0x55c2c37cfa10 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0009610 tx=0x7f5cd0050a90 comp rx=0 tx=0).stop 2024-09-12T03:17:45.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.050+0000 7f5cde090740 1 -- 172.21.15.116:0/1048312193 shutdown_connections 2024-09-12T03:17:45.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.050+0000 7f5cde090740 1 --2- 172.21.15.116:0/1048312193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c37da370 0x55c2c37dc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.050+0000 7f5cde090740 1 --2- 172.21.15.116:0/1048312193 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c37cf5d0 0x55c2c37cfa10 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.050+0000 7f5cde090740 1 --2- 172.21.15.116:0/1048312193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c37d0b40 0x55c2c37cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.050+0000 7f5cde090740 1 -- 172.21.15.116:0/1048312193 >> 172.21.15.116:0/1048312193 conn(0x55c2c3798310 msgr2=0x55c2c37cbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:45.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.050+0000 7f5cde090740 1 -- 172.21.15.116:0/1048312193 shutdown_connections 2024-09-12T03:17:45.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.050+0000 7f5cde090740 1 -- 172.21.15.116:0/1048312193 wait complete. 2024-09-12T03:17:45.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.051+0000 7f5cde090740 1 -- start start 2024-09-12T03:17:45.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.051+0000 7f5cde090740 1 Processor -- start 2024-09-12T03:17:45.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.051+0000 7f5cde090740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c37cf5d0 0x55c2c37d9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:45.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.051+0000 7f5cde090740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c37d0b40 0x55c2c37d6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:45.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.051+0000 7f5cde090740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c37da370 0x55c2c37d7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:45.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.051+0000 7f5cde090740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c2c37dcfb0 con 0x55c2c37da370 2024-09-12T03:17:45.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.051+0000 7f5cde090740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c2c36f0060 con 0x55c2c37d0b40 2024-09-12T03:17:45.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.051+0000 7f5cde090740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c2c37dd130 con 0x55c2c37cf5d0 2024-09-12T03:17:45.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.051+0000 7f5cde090740 10 client.0 _ll_register_callbacks cb 0x55c2c34438c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:45.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.051+0000 7f5cda254640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c37da370 0x55c2c37d7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:45.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.051+0000 7f5cda254640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c37da370 0x55c2c37d7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46386/0 (socket says 172.21.15.116:46386) 2024-09-12T03:17:45.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.051+0000 7f5cda254640 1 -- 172.21.15.116:0/1807185255 learned_addr learned my addr 172.21.15.116:0/1807185255 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:45.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.052+0000 7f5cd9252640 1 --2- 172.21.15.116:0/1807185255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c37d0b40 0x55c2c37d6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:45.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.052+0000 7f5cd9a53640 1 --2- 172.21.15.116:0/1807185255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c37cf5d0 0x55c2c37d9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:45.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.052+0000 7f5cda254640 1 -- 172.21.15.116:0/1807185255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c37cf5d0 msgr2=0x55c2c37d9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:45.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.052+0000 7f5cda254640 1 --2- 172.21.15.116:0/1807185255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c37cf5d0 0x55c2c37d9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.052+0000 7f5cda254640 1 -- 172.21.15.116:0/1807185255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c37d0b40 msgr2=0x55c2c37d6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:45.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.052+0000 7f5cda254640 1 --2- 172.21.15.116:0/1807185255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c37d0b40 0x55c2c37d6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.052+0000 7f5cda254640 1 -- 172.21.15.116:0/1807185255 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2c34a63c0 con 0x55c2c37da370 2024-09-12T03:17:45.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.052+0000 7f5cd9252640 1 --2- 172.21.15.116:0/1807185255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c37d0b40 0x55c2c37d6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:45.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.052+0000 7f5cd9a53640 1 --2- 172.21.15.116:0/1807185255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c37cf5d0 0x55c2c37d9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:45.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.052+0000 7f5cda254640 1 --2- 172.21.15.116:0/1807185255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c37da370 0x55c2c37d7520 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc007f20 tx=0x7f5ccc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9701edb1a332f03f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:45.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.052+0000 7f5cbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:45.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.052+0000 7f5cbd7fa640 1 -- 172.21.15.116:0/1807185255 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc024030 con 0x55c2c37da370 2024-09-12T03:17:45.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.052+0000 7f5cbd7fa640 1 -- 172.21.15.116:0/1807185255 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f5ccc010cc0 con 0x55c2c37da370 2024-09-12T03:17:45.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.052+0000 7f5cde090740 1 -- 172.21.15.116:0/1807185255 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c2c3742860 con 0x55c2c37da370 2024-09-12T03:17:45.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.052+0000 7f5cbd7fa640 1 -- 172.21.15.116:0/1807185255 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5ccc020460 con 0x55c2c37da370 2024-09-12T03:17:45.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.052+0000 7f5cde090740 10 client.19434 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:45.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.052+0000 7f5cde090740 1 -- 172.21.15.116:0/1807185255 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c2c37e89e0 con 0x55c2c37da370 2024-09-12T03:17:45.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.053+0000 7f5cbd7fa640 1 -- 172.21.15.116:0/1807185255 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c2c3742860 con 0x55c2c37da370 2024-09-12T03:17:45.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.053+0000 7f5cbd7fa640 1 -- 172.21.15.116:0/1807185255 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5ccc01f550 con 0x55c2c37da370 2024-09-12T03:17:45.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.053+0000 7f5cde090740 10 client.19434 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:45.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.053+0000 7f5cde090740 10 client.19434 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:17:45.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.053+0000 7f5cde090740 20 client.19434 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:45.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.053+0000 7f5cde090740 1 --2- 172.21.15.116:0/1807185255 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c2c37e9ea0 0x55c2c37ec340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:45.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.053+0000 7f5cde090740 4 client.19434 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:17:45.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.053+0000 7f5cde090740 1 -- 172.21.15.116:0/1807185255 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2c373cd70 con 0x55c2c37e9ea0 2024-09-12T03:17:45.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.053+0000 7f5cd9a53640 1 --2- 172.21.15.116:0/1807185255 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c2c37e9ea0 0x55c2c37ec340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:45.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.053+0000 7f5cd9a53640 1 --2- 172.21.15.116:0/1807185255 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c2c37e9ea0 0x55c2c37ec340 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd26189fcba0612c server_cookie=5e906852bb220897 in_seq=0 out_seq=0 2024-09-12T03:17:45.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.053+0000 7f5cbd7fa640 10 client.19434 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:17:45.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.054+0000 7f5cbd7fa640 1 -- 172.21.15.116:0/1807185255 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c2c373cd70 con 0x55c2c37e9ea0 2024-09-12T03:17:45.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.054+0000 7f5cbd7fa640 10 client.19434 handle_command_reply: tid=0 2024-09-12T03:17:45.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.054+0000 7f5cde090740 1 client.19434 shutdown 2024-09-12T03:17:45.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.054+0000 7f5cde090740 1 -- 172.21.15.116:0/1807185255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c37da370 msgr2=0x55c2c37d7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:45.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.054+0000 7f5cde090740 1 --2- 172.21.15.116:0/1807185255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c37da370 0x55c2c37d7520 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc007f20 tx=0x7f5ccc01a000 comp rx=0 tx=0).stop 2024-09-12T03:17:45.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.054+0000 7f5cde090740 1 -- 172.21.15.116:0/1807185255 shutdown_connections 2024-09-12T03:17:45.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.054+0000 7f5cde090740 1 --2- 172.21.15.116:0/1807185255 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c2c37e9ea0 0x55c2c37ec340 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.054+0000 7f5cde090740 1 --2- 172.21.15.116:0/1807185255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c37da370 0x55c2c37d7520 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.054+0000 7f5cde090740 1 --2- 172.21.15.116:0/1807185255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c37d0b40 0x55c2c37d6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.054+0000 7f5cde090740 1 --2- 172.21.15.116:0/1807185255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c37cf5d0 0x55c2c37d9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.054+0000 7f5cbd7fa640 0 client.19434 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:17:45.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.054+0000 7f5cde090740 1 -- 172.21.15.116:0/1807185255 >> 172.21.15.116:0/1807185255 conn(0x55c2c3798310 msgr2=0x55c2c37d1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:45.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.055+0000 7f5cde090740 1 -- 172.21.15.116:0/1807185255 shutdown_connections 2024-09-12T03:17:45.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.055+0000 7f5cde090740 1 -- 172.21.15.116:0/1807185255 wait complete. 2024-09-12T03:17:45.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.055+0000 7f5cde090740 20 client.19434 trim_cache size 0 max 16384 2024-09-12T03:17:45.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.067+0000 7f5cde090740 1 Processor -- start 2024-09-12T03:17:45.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.067+0000 7f5cde090740 1 -- start start 2024-09-12T03:17:45.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.067+0000 7f5cde090740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c37cf5d0 0x55c2c37e6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:45.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.067+0000 7f5cde090740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c37d0b40 0x55c2c37d2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:45.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.067+0000 7f5cde090740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c37da370 0x55c2c37d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:45.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.068+0000 7f5cde090740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd41afdb0 con 0x55c2c37cf5d0 2024-09-12T03:17:45.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.068+0000 7f5cde090740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd41ad680 con 0x55c2c37da370 2024-09-12T03:17:45.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.068+0000 7f5cde090740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd41ab4d0 con 0x55c2c37d0b40 2024-09-12T03:17:45.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.068+0000 7f5cd9a53640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c37cf5d0 0x55c2c37e6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:45.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.068+0000 7f5cd9a53640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c37cf5d0 0x55c2c37e6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46396/0 (socket says 172.21.15.116:46396) 2024-09-12T03:17:45.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.068+0000 7f5cd9a53640 1 -- 172.21.15.116:0/1573723483 learned_addr learned my addr 172.21.15.116:0/1573723483 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:45.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.068+0000 7f5cda254640 1 --2- 172.21.15.116:0/1573723483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c37da370 0x55c2c37d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:45.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.068+0000 7f5cd9a53640 1 -- 172.21.15.116:0/1573723483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c37d0b40 msgr2=0x55c2c37d2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:45.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.068+0000 7f5cd9a53640 1 --2- 172.21.15.116:0/1573723483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c37d0b40 0x55c2c37d2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.068+0000 7f5cd9a53640 1 -- 172.21.15.116:0/1573723483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c37da370 msgr2=0x55c2c37d26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:45.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.068+0000 7f5cd9a53640 1 --2- 172.21.15.116:0/1573723483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c37da370 0x55c2c37d26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.068+0000 7f5cd9a53640 1 -- 172.21.15.116:0/1573723483 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2c37d54f0 con 0x55c2c37cf5d0 2024-09-12T03:17:45.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.068+0000 7f5cda254640 1 --2- 172.21.15.116:0/1573723483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c37da370 0x55c2c37d26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:45.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.068+0000 7f5cd9a53640 1 --2- 172.21.15.116:0/1573723483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c37cf5d0 0x55c2c37e6250 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0097ca0 tx=0x7f5cd00048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838914b6ab3bb2b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:45.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.068+0000 7f5cbf7fe640 1 -- 172.21.15.116:0/1573723483 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0020040 con 0x55c2c37cf5d0 2024-09-12T03:17:45.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.068+0000 7f5cbf7fe640 1 -- 172.21.15.116:0/1573723483 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cd009b9d0 con 0x55c2c37cf5d0 2024-09-12T03:17:45.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.068+0000 7f5cbf7fe640 1 -- 172.21.15.116:0/1573723483 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd009f170 con 0x55c2c37cf5d0 2024-09-12T03:17:45.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.069+0000 7f5cde090740 1 -- 172.21.15.116:0/1573723483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c37cf5d0 msgr2=0x55c2c37e6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:45.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.069+0000 7f5cde090740 1 --2- 172.21.15.116:0/1573723483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c37cf5d0 0x55c2c37e6250 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0097ca0 tx=0x7f5cd00048a0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.069+0000 7f5cde090740 1 -- 172.21.15.116:0/1573723483 shutdown_connections 2024-09-12T03:17:45.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.069+0000 7f5cde090740 1 --2- 172.21.15.116:0/1573723483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c37da370 0x55c2c37d26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.069+0000 7f5cde090740 1 --2- 172.21.15.116:0/1573723483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c37d0b40 0x55c2c37d2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.069+0000 7f5cde090740 1 --2- 172.21.15.116:0/1573723483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c37cf5d0 0x55c2c37e6250 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.069+0000 7f5cde090740 1 -- 172.21.15.116:0/1573723483 >> 172.21.15.116:0/1573723483 conn(0x55c2c3798310 msgr2=0x55c2c37d6f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:45.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.069+0000 7f5cde090740 1 -- 172.21.15.116:0/1573723483 shutdown_connections 2024-09-12T03:17:45.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.069+0000 7f5cde090740 1 -- 172.21.15.116:0/1573723483 wait complete. 2024-09-12T03:17:45.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.069+0000 7f5cde090740 1 -- start start 2024-09-12T03:17:45.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.069+0000 7f5cde090740 1 Processor -- start 2024-09-12T03:17:45.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.070+0000 7f5cde090740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c37cf5d0 0x55c2c37e2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:45.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.070+0000 7f5cde090740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c37d0b40 0x55c2c37ddfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:45.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.070+0000 7f5cde090740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c37da370 0x55c2c37de730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:45.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.070+0000 7f5cde090740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd41abb40 con 0x55c2c37d0b40 2024-09-12T03:17:45.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.070+0000 7f5cde090740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd41acfb0 con 0x55c2c37cf5d0 2024-09-12T03:17:45.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.070+0000 7f5cde090740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd41ac210 con 0x55c2c37da370 2024-09-12T03:17:45.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.070+0000 7f5cde090740 10 client.0 _ll_register_callbacks cb 0x7f5ccc013310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:45.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.070+0000 7f5cd9252640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c37d0b40 0x55c2c37ddfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:45.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.070+0000 7f5cd9252640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c37d0b40 0x55c2c37ddfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46406/0 (socket says 172.21.15.116:46406) 2024-09-12T03:17:45.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.070+0000 7f5cd9252640 1 -- 172.21.15.116:0/2376075457 learned_addr learned my addr 172.21.15.116:0/2376075457 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:45.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.070+0000 7f5cda254640 1 --2- 172.21.15.116:0/2376075457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c37da370 0x55c2c37de730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:45.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.070+0000 7f5cd9a53640 1 --2- 172.21.15.116:0/2376075457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c37cf5d0 0x55c2c37e2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:45.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.070+0000 7f5cd9252640 1 -- 172.21.15.116:0/2376075457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c37da370 msgr2=0x55c2c37de730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:45.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.070+0000 7f5cd9252640 1 --2- 172.21.15.116:0/2376075457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c37da370 0x55c2c37de730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.070+0000 7f5cd9252640 1 -- 172.21.15.116:0/2376075457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c37cf5d0 msgr2=0x55c2c37e2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:45.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.070+0000 7f5cd9252640 1 --2- 172.21.15.116:0/2376075457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c37cf5d0 0x55c2c37e2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.070+0000 7f5cd9252640 1 -- 172.21.15.116:0/2376075457 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2c3858040 con 0x55c2c37d0b40 2024-09-12T03:17:45.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.070+0000 7f5cd9a53640 1 --2- 172.21.15.116:0/2376075457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c37cf5d0 0x55c2c37e2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:45.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.070+0000 7f5cda254640 1 --2- 172.21.15.116:0/2376075457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c37da370 0x55c2c37de730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:45.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.070+0000 7f5cd9252640 1 --2- 172.21.15.116:0/2376075457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c37d0b40 0x55c2c37ddfa0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4009900 tx=0x7f5cc4004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5422d8fe2af2b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:45.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.070+0000 7f5cbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:45.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.070+0000 7f5cbd7fa640 1 -- 172.21.15.116:0/2376075457 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4035040 con 0x55c2c37d0b40 2024-09-12T03:17:45.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.070+0000 7f5cbd7fa640 1 -- 172.21.15.116:0/2376075457 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f5cc40037f0 con 0x55c2c37d0b40 2024-09-12T03:17:45.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.070+0000 7f5cde090740 1 -- 172.21.15.116:0/2376075457 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cd009b9d0 con 0x55c2c37d0b40 2024-09-12T03:17:45.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.071+0000 7f5cbd7fa640 1 -- 172.21.15.116:0/2376075457 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5cc40039f0 con 0x55c2c37d0b40 2024-09-12T03:17:45.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.071+0000 7f5cde090740 10 client.19440 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:45.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.071+0000 7f5cde090740 1 -- 172.21.15.116:0/2376075457 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c2c3859140 con 0x55c2c37d0b40 2024-09-12T03:17:45.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.071+0000 7f5cbd7fa640 1 -- 172.21.15.116:0/2376075457 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cd009b9d0 con 0x55c2c37d0b40 2024-09-12T03:17:45.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.071+0000 7f5cbd7fa640 1 -- 172.21.15.116:0/2376075457 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5cc403c440 con 0x55c2c37d0b40 2024-09-12T03:17:45.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.071+0000 7f5cde090740 10 client.19440 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:45.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.071+0000 7f5cde090740 10 client.19440 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:17:45.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.071+0000 7f5cde090740 20 client.19440 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:45.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.071+0000 7f5cde090740 1 --2- 172.21.15.116:0/2376075457 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c2c3859940 0x55c2c385bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:45.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.071+0000 7f5cde090740 4 client.19440 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:17:45.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.071+0000 7f5cde090740 1 -- 172.21.15.116:0/2376075457 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c2c36b9170 con 0x55c2c3859940 2024-09-12T03:17:45.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.071+0000 7f5cd9a53640 1 --2- 172.21.15.116:0/2376075457 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c2c3859940 0x55c2c385bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:45.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.072+0000 7f5cd9a53640 1 --2- 172.21.15.116:0/2376075457 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c2c3859940 0x55c2c385bd60 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd7918ca5cb42d4a server_cookie=6ab88c71c9386305 in_seq=0 out_seq=0 2024-09-12T03:17:45.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.072+0000 7f5cbd7fa640 10 client.19440 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:17:45.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.072+0000 7f5cbd7fa640 1 -- 172.21.15.116:0/2376075457 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c2c36b9170 con 0x55c2c3859940 2024-09-12T03:17:45.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.072+0000 7f5cbd7fa640 10 client.19440 handle_command_reply: tid=0 2024-09-12T03:17:45.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.072+0000 7f5cde090740 1 client.19440 shutdown 2024-09-12T03:17:45.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.072+0000 7f5cde090740 1 -- 172.21.15.116:0/2376075457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c37d0b40 msgr2=0x55c2c37ddfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:45.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.072+0000 7f5cde090740 1 --2- 172.21.15.116:0/2376075457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c37d0b40 0x55c2c37ddfa0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4009900 tx=0x7f5cc4004410 comp rx=0 tx=0).stop 2024-09-12T03:17:45.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.072+0000 7f5cde090740 1 -- 172.21.15.116:0/2376075457 shutdown_connections 2024-09-12T03:17:45.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.072+0000 7f5cde090740 1 --2- 172.21.15.116:0/2376075457 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55c2c3859940 0x55c2c385bd60 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.072+0000 7f5cde090740 1 --2- 172.21.15.116:0/2376075457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c37da370 0x55c2c37de730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.072+0000 7f5cde090740 1 --2- 172.21.15.116:0/2376075457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c2c37d0b40 0x55c2c37ddfa0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.072+0000 7f5cbd7fa640 0 client.19440 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:17:45.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.072+0000 7f5cde090740 1 --2- 172.21.15.116:0/2376075457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c2c37cf5d0 0x55c2c37e2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.072+0000 7f5cde090740 1 -- 172.21.15.116:0/2376075457 >> 172.21.15.116:0/2376075457 conn(0x55c2c3798310 msgr2=0x55c2c37d8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:45.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.072+0000 7f5cde090740 1 -- 172.21.15.116:0/2376075457 shutdown_connections 2024-09-12T03:17:45.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.072+0000 7f5cde090740 1 -- 172.21.15.116:0/2376075457 wait complete. 2024-09-12T03:17:45.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.072+0000 7f5cde090740 20 client.19440 trim_cache size 0 max 16384 2024-09-12T03:17:45.085 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:17:45.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.073+0000 7f5cdbcb7640 1 -- 172.21.15.116:0/3207778658 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5cc00a2b80 msgr2=0x7f5cc00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:45.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.073+0000 7f5cdbcb7640 1 --2- 172.21.15.116:0/3207778658 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5cc00a2b80 0x7f5cc00a5040 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc007fd0 tx=0x7f5ccc007c60 comp rx=0 tx=0).stop 2024-09-12T03:17:45.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.073+0000 7f5cdbcb7640 1 -- 172.21.15.116:0/3207778658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cd4102db0 msgr2=0x7f5cd4198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:45.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.073+0000 7f5cdbcb7640 1 --2- 172.21.15.116:0/3207778658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cd4102db0 0x7f5cd4198580 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f5cd000d770 tx=0x7f5cd000dc30 comp rx=0 tx=0).stop 2024-09-12T03:17:45.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.073+0000 7f5cdbcb7640 1 -- 172.21.15.116:0/3207778658 shutdown_connections 2024-09-12T03:17:45.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.073+0000 7f5cdbcb7640 1 --2- 172.21.15.116:0/3207778658 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f5cc00a2b80 0x7f5cc00a5040 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.073+0000 7f5cdbcb7640 1 --2- 172.21.15.116:0/3207778658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cd41082e0 0x7f5cd419ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.073+0000 7f5cdbcb7640 1 --2- 172.21.15.116:0/3207778658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cd4103760 0x7f5cd4198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.073+0000 7f5cdbcb7640 1 --2- 172.21.15.116:0/3207778658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cd4102db0 0x7f5cd4198580 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:45.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.073+0000 7f5cdbcb7640 1 -- 172.21.15.116:0/3207778658 >> 172.21.15.116:0/3207778658 conn(0x7f5cd40fc7d0 msgr2=0x7f5cd4105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:45.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.073+0000 7f5cdbcb7640 1 -- 172.21.15.116:0/3207778658 shutdown_connections 2024-09-12T03:17:45.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:45.073+0000 7f5cdbcb7640 1 -- 172.21.15.116:0/3207778658 wait complete. 2024-09-12T03:17:45.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:45 smithi177 ceph-mon[35305]: pgmap v781: 97 pgs: 97 active+clean; 6.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:17:45.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:45 smithi116 ceph-mon[31410]: pgmap v781: 97 pgs: 97 active+clean; 6.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:17:45.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:45 smithi183 ceph-mon[35535]: pgmap v781: 97 pgs: 97 active+clean; 6.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:17:45.955 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:17:46.200 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:46.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.717+0000 7f7aa2286640 1 -- 172.21.15.116:0/3728964864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a9c106f10 msgr2=0x7f7a9c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:46.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.717+0000 7f7aa2286640 1 --2- 172.21.15.116:0/3728964864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a9c106f10 0x7f7a9c10ea90 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f7a94009ca0 tx=0x7f7a9402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.717+0000 7f7aa2286640 1 -- 172.21.15.116:0/3728964864 shutdown_connections 2024-09-12T03:17:46.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.717+0000 7f7aa2286640 1 --2- 172.21.15.116:0/3728964864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a9c106f10 0x7f7a9c10ea90 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.717+0000 7f7aa2286640 1 --2- 172.21.15.116:0/3728964864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a9c102790 0x7f7a9c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.717+0000 7f7aa2286640 1 --2- 172.21.15.116:0/3728964864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a9c101de0 0x7f7a9c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.717+0000 7f7aa2286640 1 -- 172.21.15.116:0/3728964864 >> 172.21.15.116:0/3728964864 conn(0x7f7a9c0fd8c0 msgr2=0x7f7a9c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:46.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.717+0000 7f7aa2286640 1 -- 172.21.15.116:0/3728964864 shutdown_connections 2024-09-12T03:17:46.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.717+0000 7f7aa2286640 1 -- 172.21.15.116:0/3728964864 wait complete. 2024-09-12T03:17:46.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.717+0000 7f7aa2286640 1 Processor -- start 2024-09-12T03:17:46.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.717+0000 7f7aa2286640 1 -- start start 2024-09-12T03:17:46.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.718+0000 7f7aa2286640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a9c101de0 0x7f7a9c1a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:46.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.718+0000 7f7aa2286640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a9c102790 0x7f7a9c1a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:46.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.718+0000 7f7aa2286640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a9c106f10 0x7f7a9c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:46.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.718+0000 7f7aa2286640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c112010 con 0x7f7a9c102790 2024-09-12T03:17:46.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.718+0000 7f7aa2286640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c111e90 con 0x7f7a9c106f10 2024-09-12T03:17:46.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.718+0000 7f7aa2286640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c112190 con 0x7f7a9c101de0 2024-09-12T03:17:46.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.718+0000 7f7a9b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a9c102790 0x7f7a9c1a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:46.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.718+0000 7f7a9b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a9c102790 0x7f7a9c1a0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46432/0 (socket says 172.21.15.116:46432) 2024-09-12T03:17:46.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.718+0000 7f7a9b7fe640 1 -- 172.21.15.116:0/1303776976 learned_addr learned my addr 172.21.15.116:0/1303776976 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:46.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.718+0000 7f7a9bfff640 1 --2- 172.21.15.116:0/1303776976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a9c101de0 0x7f7a9c1a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:46.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.718+0000 7f7aa0823640 1 --2- 172.21.15.116:0/1303776976 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a9c106f10 0x7f7a9c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:46.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.718+0000 7f7a9b7fe640 1 -- 172.21.15.116:0/1303776976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a9c101de0 msgr2=0x7f7a9c1a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:46.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.718+0000 7f7a9b7fe640 1 --2- 172.21.15.116:0/1303776976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a9c101de0 0x7f7a9c1a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.718+0000 7f7a9b7fe640 1 -- 172.21.15.116:0/1303776976 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a9c106f10 msgr2=0x7f7a9c19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:46.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.718+0000 7f7a9b7fe640 1 --2- 172.21.15.116:0/1303776976 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a9c106f10 0x7f7a9c19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.718+0000 7f7a9b7fe640 1 -- 172.21.15.116:0/1303776976 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a9c19aeb0 con 0x7f7a9c102790 2024-09-12T03:17:46.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.719+0000 7f7a9b7fe640 1 --2- 172.21.15.116:0/1303776976 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a9c102790 0x7f7a9c1a0ab0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000b4a0 tx=0x7f7a9000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:46.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.719+0000 7f7a997fa640 1 -- 172.21.15.116:0/1303776976 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a90015020 con 0x7f7a9c102790 2024-09-12T03:17:46.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.719+0000 7f7a997fa640 1 -- 172.21.15.116:0/1303776976 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a90011d10 con 0x7f7a9c102790 2024-09-12T03:17:46.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.719+0000 7f7aa2286640 1 -- 172.21.15.116:0/1303776976 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a9c19b1a0 con 0x7f7a9c102790 2024-09-12T03:17:46.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.719+0000 7f7a997fa640 1 -- 172.21.15.116:0/1303776976 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9000f2e0 con 0x7f7a9c102790 2024-09-12T03:17:46.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.719+0000 7f7aa2286640 1 -- 172.21.15.116:0/1303776976 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a9c1a74f0 con 0x7f7a9c102790 2024-09-12T03:17:46.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.721+0000 7f7a997fa640 1 -- 172.21.15.116:0/1303776976 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f7a90018020 con 0x7f7a9c102790 2024-09-12T03:17:46.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.721+0000 7f7aa465f740 1 Processor -- start 2024-09-12T03:17:46.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.721+0000 7f7aa465f740 1 -- start start 2024-09-12T03:17:46.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.722+0000 7f7aa465f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0b4e6cb40 0x55d0b4e6b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:46.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.722+0000 7f7aa465f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0b4e6b5d0 0x55d0b4e6ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:46.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.722+0000 7f7aa465f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0b4e76370 0x55d0b4e78760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:46.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.722+0000 7f7aa465f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d0b4dd12f0 con 0x55d0b4e6cb40 2024-09-12T03:17:46.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.722+0000 7f7aa465f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d0b4d78cf0 con 0x55d0b4e76370 2024-09-12T03:17:46.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.722+0000 7f7aa465f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d0b4dcd2b0 con 0x55d0b4e6b5d0 2024-09-12T03:17:46.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.722+0000 7f7aa0823640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0b4e6cb40 0x55d0b4e6b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:46.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.722+0000 7f7aa0823640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0b4e6cb40 0x55d0b4e6b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46444/0 (socket says 172.21.15.116:46444) 2024-09-12T03:17:46.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.722+0000 7f7aa0823640 1 -- 172.21.15.116:0/2490612411 learned_addr learned my addr 172.21.15.116:0/2490612411 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:46.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.722+0000 7f7a997fa640 1 --2- 172.21.15.116:0/1303776976 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7a840a2c00 0x7f7a840a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:46.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.722+0000 7f7a9bfff640 1 --2- 172.21.15.116:0/2490612411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0b4e6b5d0 0x55d0b4e6ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:46.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.722+0000 7f7a9b7fe640 1 --2- 172.21.15.116:0/2490612411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0b4e76370 0x55d0b4e78760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:46.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.722+0000 7f7aa0823640 1 -- 172.21.15.116:0/2490612411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0b4e6b5d0 msgr2=0x55d0b4e6ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:46.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.722+0000 7f7aa0823640 1 --2- 172.21.15.116:0/2490612411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0b4e6b5d0 0x55d0b4e6ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.722+0000 7f7aa0823640 1 -- 172.21.15.116:0/2490612411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0b4e76370 msgr2=0x55d0b4e78760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:46.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.722+0000 7f7aa0823640 1 --2- 172.21.15.116:0/2490612411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0b4e76370 0x55d0b4e78760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.722+0000 7f7aa0823640 1 -- 172.21.15.116:0/2490612411 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0b4d5d3a0 con 0x55d0b4e6cb40 2024-09-12T03:17:46.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.722+0000 7f7a9bfff640 1 --2- 172.21.15.116:0/2490612411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0b4e6b5d0 0x55d0b4e6ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:46.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.722+0000 7f7a9b7fe640 1 --2- 172.21.15.116:0/2490612411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0b4e76370 0x55d0b4e78760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:46.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.722+0000 7f7a997fa640 1 -- 172.21.15.116:0/1303776976 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f7a900ce880 con 0x7f7a9c102790 2024-09-12T03:17:46.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.722+0000 7f7aa0823640 1 --2- 172.21.15.116:0/1303776976 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7a840a2c00 0x7f7a840a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:46.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.722+0000 7f7aa0823640 1 --2- 172.21.15.116:0/2490612411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0b4e6cb40 0x55d0b4e6b090 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f7a94005cc0 tx=0x7f7a94031bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5be82ffa6582de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:46.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.723+0000 7f7a827fc640 1 -- 172.21.15.116:0/2490612411 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a94041040 con 0x55d0b4e6cb40 2024-09-12T03:17:46.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.723+0000 7f7a827fc640 1 -- 172.21.15.116:0/2490612411 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a94036070 con 0x55d0b4e6cb40 2024-09-12T03:17:46.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.723+0000 7f7a827fc640 1 -- 172.21.15.116:0/2490612411 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9402fad0 con 0x55d0b4e6cb40 2024-09-12T03:17:46.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.723+0000 7f7aa465f740 1 -- 172.21.15.116:0/2490612411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0b4e6cb40 msgr2=0x55d0b4e6b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:46.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.723+0000 7f7aa0823640 1 --2- 172.21.15.116:0/1303776976 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7a840a2c00 0x7f7a840a50c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7a9404c000 tx=0x7f7a9404d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:46.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.723+0000 7f7aa465f740 1 --2- 172.21.15.116:0/2490612411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0b4e6cb40 0x55d0b4e6b090 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f7a94005cc0 tx=0x7f7a94031bb0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.723+0000 7f7aa465f740 1 -- 172.21.15.116:0/2490612411 shutdown_connections 2024-09-12T03:17:46.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.723+0000 7f7aa465f740 1 --2- 172.21.15.116:0/2490612411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0b4e76370 0x55d0b4e78760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.723+0000 7f7aa465f740 1 --2- 172.21.15.116:0/2490612411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0b4e6b5d0 0x55d0b4e6ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.723+0000 7f7aa465f740 1 --2- 172.21.15.116:0/2490612411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0b4e6cb40 0x55d0b4e6b090 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.723+0000 7f7aa465f740 1 -- 172.21.15.116:0/2490612411 >> 172.21.15.116:0/2490612411 conn(0x55d0b4e34310 msgr2=0x55d0b4e67c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:46.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.724+0000 7f7aa465f740 1 -- 172.21.15.116:0/2490612411 shutdown_connections 2024-09-12T03:17:46.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.724+0000 7f7aa465f740 1 -- 172.21.15.116:0/2490612411 wait complete. 2024-09-12T03:17:46.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.724+0000 7f7aa465f740 1 -- start start 2024-09-12T03:17:46.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.724+0000 7f7aa465f740 1 Processor -- start 2024-09-12T03:17:46.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.724+0000 7f7aa465f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0b4e6b5d0 0x55d0b4e75d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:46.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.724+0000 7f7aa465f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0b4e6cb40 0x55d0b4e72fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:46.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.724+0000 7f7aa465f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0b4e76370 0x55d0b4e73530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:46.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.724+0000 7f7aa465f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d0b4e79070 con 0x55d0b4e6b5d0 2024-09-12T03:17:46.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.724+0000 7f7aa465f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d0b4d8c060 con 0x55d0b4e76370 2024-09-12T03:17:46.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.724+0000 7f7aa465f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d0b4e791f0 con 0x55d0b4e6cb40 2024-09-12T03:17:46.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.724+0000 7f7aa465f740 10 client.0 _ll_register_callbacks cb 0x55d0b4adf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:46.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.724+0000 7f7a9bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0b4e6b5d0 0x55d0b4e75d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:46.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.724+0000 7f7a9bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0b4e6b5d0 0x55d0b4e75d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46458/0 (socket says 172.21.15.116:46458) 2024-09-12T03:17:46.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.724+0000 7f7a9bfff640 1 -- 172.21.15.116:0/3743002988 learned_addr learned my addr 172.21.15.116:0/3743002988 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:46.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.725+0000 7f7a9b7fe640 1 --2- 172.21.15.116:0/3743002988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0b4e6cb40 0x55d0b4e72fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:46.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.725+0000 7f7aa0823640 1 --2- 172.21.15.116:0/3743002988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0b4e76370 0x55d0b4e73530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:46.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.725+0000 7f7a9bfff640 1 -- 172.21.15.116:0/3743002988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0b4e6cb40 msgr2=0x55d0b4e72fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:46.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.725+0000 7f7a9bfff640 1 --2- 172.21.15.116:0/3743002988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0b4e6cb40 0x55d0b4e72fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.725+0000 7f7a9bfff640 1 -- 172.21.15.116:0/3743002988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0b4e76370 msgr2=0x55d0b4e73530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:46.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.725+0000 7f7a9bfff640 1 --2- 172.21.15.116:0/3743002988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0b4e76370 0x55d0b4e73530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.725+0000 7f7a9bfff640 1 -- 172.21.15.116:0/3743002988 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0b4b423c0 con 0x55d0b4e6b5d0 2024-09-12T03:17:46.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.725+0000 7f7a9bfff640 1 --2- 172.21.15.116:0/3743002988 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0b4e6b5d0 0x55d0b4e75d00 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c009ee0 tx=0x7f7a8c011630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9badbba2e7773a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:46.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.725+0000 7f7a7b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:46.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.725+0000 7f7a7b7fe640 1 -- 172.21.15.116:0/3743002988 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c019070 con 0x55d0b4e6b5d0 2024-09-12T03:17:46.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.725+0000 7f7a7b7fe640 1 -- 172.21.15.116:0/3743002988 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f7a8c015660 con 0x55d0b4e6b5d0 2024-09-12T03:17:46.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.725+0000 7f7aa465f740 1 -- 172.21.15.116:0/3743002988 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a94036070 con 0x55d0b4e6b5d0 2024-09-12T03:17:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.725+0000 7f7a7b7fe640 1 -- 172.21.15.116:0/3743002988 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7a8c025050 con 0x55d0b4e6b5d0 2024-09-12T03:17:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.725+0000 7f7aa465f740 10 client.19452 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.725+0000 7f7aa465f740 1 -- 172.21.15.116:0/3743002988 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d0b4e84ad0 con 0x55d0b4e6b5d0 2024-09-12T03:17:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.725+0000 7f7a7b7fe640 1 -- 172.21.15.116:0/3743002988 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a94036070 con 0x55d0b4e6b5d0 2024-09-12T03:17:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.726+0000 7f7a7b7fe640 1 -- 172.21.15.116:0/3743002988 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7a8c014b60 con 0x55d0b4e6b5d0 2024-09-12T03:17:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.726+0000 7f7aa465f740 10 client.19452 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.726+0000 7f7aa465f740 10 client.19452 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:17:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.726+0000 7f7aa465f740 20 client.19452 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.726+0000 7f7aa465f740 1 --2- 172.21.15.116:0/3743002988 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d0b4e85ee0 0x55d0b4e882f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.726+0000 7f7aa465f740 4 client.19452 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:17:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.726+0000 7f7aa465f740 1 -- 172.21.15.116:0/3743002988 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0b4dde860 con 0x55d0b4e85ee0 2024-09-12T03:17:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.726+0000 7f7a9b7fe640 1 --2- 172.21.15.116:0/3743002988 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d0b4e85ee0 0x55d0b4e882f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.726+0000 7f7a9b7fe640 1 --2- 172.21.15.116:0/3743002988 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d0b4e85ee0 0x55d0b4e882f0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cafdfba0a5313e9a server_cookie=20e88f558492d445 in_seq=0 out_seq=0 2024-09-12T03:17:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.726+0000 7f7a7b7fe640 10 client.19452 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:46.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.727+0000 7f7a7b7fe640 1 -- 172.21.15.116:0/3743002988 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d0b4dde860 con 0x55d0b4e85ee0 2024-09-12T03:17:46.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.727+0000 7f7a7b7fe640 10 client.19452 handle_command_reply: tid=0 2024-09-12T03:17:46.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.727+0000 7f7aa465f740 1 client.19452 shutdown 2024-09-12T03:17:46.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.727+0000 7f7aa465f740 1 -- 172.21.15.116:0/3743002988 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0b4e6b5d0 msgr2=0x55d0b4e75d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:46.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.727+0000 7f7aa465f740 1 --2- 172.21.15.116:0/3743002988 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0b4e6b5d0 0x55d0b4e75d00 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c009ee0 tx=0x7f7a8c011630 comp rx=0 tx=0).stop 2024-09-12T03:17:46.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.727+0000 7f7aa465f740 1 -- 172.21.15.116:0/3743002988 shutdown_connections 2024-09-12T03:17:46.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.727+0000 7f7aa465f740 1 --2- 172.21.15.116:0/3743002988 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d0b4e85ee0 0x55d0b4e882f0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.727+0000 7f7aa465f740 1 --2- 172.21.15.116:0/3743002988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0b4e76370 0x55d0b4e73530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.727+0000 7f7aa465f740 1 --2- 172.21.15.116:0/3743002988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0b4e6cb40 0x55d0b4e72fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.727+0000 7f7aa465f740 1 --2- 172.21.15.116:0/3743002988 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0b4e6b5d0 0x55d0b4e75d00 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.727+0000 7f7a7b7fe640 0 client.19452 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:46.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.727+0000 7f7aa465f740 1 -- 172.21.15.116:0/3743002988 >> 172.21.15.116:0/3743002988 conn(0x55d0b4e34310 msgr2=0x55d0b4e6d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:46.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.727+0000 7f7aa465f740 1 -- 172.21.15.116:0/3743002988 shutdown_connections 2024-09-12T03:17:46.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.727+0000 7f7aa465f740 1 -- 172.21.15.116:0/3743002988 wait complete. 2024-09-12T03:17:46.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.727+0000 7f7aa465f740 20 client.19452 trim_cache size 0 max 16384 2024-09-12T03:17:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.740+0000 7f7aa465f740 1 Processor -- start 2024-09-12T03:17:46.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.741+0000 7f7aa465f740 1 -- start start 2024-09-12T03:17:46.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.741+0000 7f7aa465f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0b4e6b5d0 0x55d0b4e6e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:46.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.741+0000 7f7aa465f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0b4e6cb40 0x55d0b4e6e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:46.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.741+0000 7f7aa465f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0b4e76370 0x55d0b4e8dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:46.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.741+0000 7f7aa465f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c1b20a0 con 0x55d0b4e6b5d0 2024-09-12T03:17:46.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.741+0000 7f7aa465f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c1af970 con 0x55d0b4e6cb40 2024-09-12T03:17:46.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.741+0000 7f7aa465f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c1ad7c0 con 0x55d0b4e76370 2024-09-12T03:17:46.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.741+0000 7f7a9bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0b4e6b5d0 0x55d0b4e6e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:46.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.741+0000 7f7a9bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0b4e6b5d0 0x55d0b4e6e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46468/0 (socket says 172.21.15.116:46468) 2024-09-12T03:17:46.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.741+0000 7f7a9bfff640 1 -- 172.21.15.116:0/3559342963 learned_addr learned my addr 172.21.15.116:0/3559342963 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:46.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.741+0000 7f7a9bfff640 1 -- 172.21.15.116:0/3559342963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0b4e76370 msgr2=0x55d0b4e8dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:46.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.741+0000 7f7aa0823640 1 --2- 172.21.15.116:0/3559342963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0b4e76370 0x55d0b4e8dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:46.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.741+0000 7f7a9b7fe640 1 --2- 172.21.15.116:0/3559342963 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0b4e6cb40 0x55d0b4e6e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:46.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.741+0000 7f7a9bfff640 1 --2- 172.21.15.116:0/3559342963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0b4e76370 0x55d0b4e8dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.741+0000 7f7a9bfff640 1 -- 172.21.15.116:0/3559342963 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0b4e6cb40 msgr2=0x55d0b4e6e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:46.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.741+0000 7f7a9bfff640 1 --2- 172.21.15.116:0/3559342963 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0b4e6cb40 0x55d0b4e6e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.741+0000 7f7a9bfff640 1 -- 172.21.15.116:0/3559342963 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0b4e6eba0 con 0x55d0b4e6b5d0 2024-09-12T03:17:46.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.741+0000 7f7a9bfff640 1 --2- 172.21.15.116:0/3559342963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0b4e6b5d0 0x55d0b4e6e120 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c000c00 tx=0x7f7a8c0046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c724e9c9778254 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:46.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.741+0000 7f7a81ffb640 1 -- 172.21.15.116:0/3559342963 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c02e070 con 0x55d0b4e6b5d0 2024-09-12T03:17:46.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.742+0000 7f7a81ffb640 1 -- 172.21.15.116:0/3559342963 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a8c01d6d0 con 0x55d0b4e6b5d0 2024-09-12T03:17:46.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.742+0000 7f7a81ffb640 1 -- 172.21.15.116:0/3559342963 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c019040 con 0x55d0b4e6b5d0 2024-09-12T03:17:46.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.742+0000 7f7aa465f740 1 -- 172.21.15.116:0/3559342963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0b4e6b5d0 msgr2=0x55d0b4e6e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:46.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.742+0000 7f7aa465f740 1 --2- 172.21.15.116:0/3559342963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0b4e6b5d0 0x55d0b4e6e120 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c000c00 tx=0x7f7a8c0046a0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.742+0000 7f7aa465f740 1 -- 172.21.15.116:0/3559342963 shutdown_connections 2024-09-12T03:17:46.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.742+0000 7f7aa465f740 1 --2- 172.21.15.116:0/3559342963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0b4e76370 0x55d0b4e8dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.742+0000 7f7aa465f740 1 --2- 172.21.15.116:0/3559342963 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0b4e6cb40 0x55d0b4e6e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.742+0000 7f7aa465f740 1 --2- 172.21.15.116:0/3559342963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0b4e6b5d0 0x55d0b4e6e120 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.742+0000 7f7aa465f740 1 -- 172.21.15.116:0/3559342963 >> 172.21.15.116:0/3559342963 conn(0x55d0b4e34310 msgr2=0x55d0b4e66d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:46.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.742+0000 7f7aa465f740 1 -- 172.21.15.116:0/3559342963 shutdown_connections 2024-09-12T03:17:46.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.742+0000 7f7aa465f740 1 -- 172.21.15.116:0/3559342963 wait complete. 2024-09-12T03:17:46.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.743+0000 7f7aa465f740 1 -- start start 2024-09-12T03:17:46.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.743+0000 7f7aa465f740 1 Processor -- start 2024-09-12T03:17:46.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.743+0000 7f7aa465f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0b4e6b5d0 0x55d0b4e7a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:46.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.743+0000 7f7aa465f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0b4e6cb40 0x55d0b4e7abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:46.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.743+0000 7f7aa465f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0b4e76370 0x55d0b4eefb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:46.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.743+0000 7f7aa465f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c1ade30 con 0x55d0b4e6cb40 2024-09-12T03:17:46.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.743+0000 7f7aa465f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c1af2a0 con 0x55d0b4e76370 2024-09-12T03:17:46.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.743+0000 7f7aa465f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c1ae500 con 0x55d0b4e6b5d0 2024-09-12T03:17:46.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.743+0000 7f7aa465f740 10 client.0 _ll_register_callbacks cb 0x7f7a8c009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:46.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.743+0000 7f7a9b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0b4e6cb40 0x55d0b4e7abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:46.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.743+0000 7f7a9b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0b4e6cb40 0x55d0b4e7abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46472/0 (socket says 172.21.15.116:46472) 2024-09-12T03:17:46.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.743+0000 7f7a9b7fe640 1 -- 172.21.15.116:0/104326231 learned_addr learned my addr 172.21.15.116:0/104326231 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:46.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.743+0000 7f7a9bfff640 1 --2- 172.21.15.116:0/104326231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0b4e6b5d0 0x55d0b4e7a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:46.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.744+0000 7f7aa0823640 1 --2- 172.21.15.116:0/104326231 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0b4e76370 0x55d0b4eefb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:46.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.744+0000 7f7a9b7fe640 1 -- 172.21.15.116:0/104326231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0b4e6b5d0 msgr2=0x55d0b4e7a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:46.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.744+0000 7f7a9b7fe640 1 --2- 172.21.15.116:0/104326231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0b4e6b5d0 0x55d0b4e7a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.744+0000 7f7a9b7fe640 1 -- 172.21.15.116:0/104326231 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0b4e76370 msgr2=0x55d0b4eefb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:46.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.744+0000 7f7a9b7fe640 1 --2- 172.21.15.116:0/104326231 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0b4e76370 0x55d0b4eefb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.744+0000 7f7a9b7fe640 1 -- 172.21.15.116:0/104326231 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0b4ef00d0 con 0x55d0b4e6cb40 2024-09-12T03:17:46.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.744+0000 7f7a9b7fe640 1 --2- 172.21.15.116:0/104326231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0b4e6cb40 0x55d0b4e7abc0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f7a90097ae0 tx=0x7f7a9006afc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0af62fcd28d8fd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:46.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.744+0000 7f7a7b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:46.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.744+0000 7f7a7b7fe640 1 -- 172.21.15.116:0/104326231 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9009b810 con 0x55d0b4e6cb40 2024-09-12T03:17:46.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.744+0000 7f7a7b7fe640 1 -- 172.21.15.116:0/104326231 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f7a90010510 con 0x55d0b4e6cb40 2024-09-12T03:17:46.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.744+0000 7f7aa465f740 1 -- 172.21.15.116:0/104326231 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a8c01d6d0 con 0x55d0b4e6cb40 2024-09-12T03:17:46.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.744+0000 7f7a7b7fe640 1 -- 172.21.15.116:0/104326231 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7a9009b110 con 0x55d0b4e6cb40 2024-09-12T03:17:46.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.744+0000 7f7aa465f740 10 client.19458 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:46.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.744+0000 7f7aa465f740 1 -- 172.21.15.116:0/104326231 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d0b4ef1310 con 0x55d0b4e6cb40 2024-09-12T03:17:46.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.744+0000 7f7a7b7fe640 1 -- 172.21.15.116:0/104326231 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a8c01d6d0 con 0x55d0b4e6cb40 2024-09-12T03:17:46.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.744+0000 7f7a7b7fe640 1 -- 172.21.15.116:0/104326231 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7a90097780 con 0x55d0b4e6cb40 2024-09-12T03:17:46.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.745+0000 7f7aa465f740 10 client.19458 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:46.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.745+0000 7f7aa465f740 10 client.19458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:17:46.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.745+0000 7f7aa465f740 20 client.19458 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:46.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.745+0000 7f7aa465f740 1 --2- 172.21.15.116:0/104326231 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d0b4ef1a90 0x55d0b4ef3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:46.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.745+0000 7f7aa465f740 4 client.19458 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:17:46.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.745+0000 7f7aa465f740 1 -- 172.21.15.116:0/104326231 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d0b4dd8d70 con 0x55d0b4ef1a90 2024-09-12T03:17:46.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.745+0000 7f7a9bfff640 1 --2- 172.21.15.116:0/104326231 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d0b4ef1a90 0x55d0b4ef3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:46.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.745+0000 7f7a7b7fe640 10 client.19458 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:46.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.745+0000 7f7a9bfff640 1 --2- 172.21.15.116:0/104326231 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d0b4ef1a90 0x55d0b4ef3eb0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bd16f9556a5788e server_cookie=e522c2b985b4f34b in_seq=0 out_seq=0 2024-09-12T03:17:46.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.745+0000 7f7a7b7fe640 1 -- 172.21.15.116:0/104326231 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d0b4dd8d70 con 0x55d0b4ef1a90 2024-09-12T03:17:46.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.745+0000 7f7a7b7fe640 10 client.19458 handle_command_reply: tid=0 2024-09-12T03:17:46.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.745+0000 7f7aa465f740 1 client.19458 shutdown 2024-09-12T03:17:46.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.746+0000 7f7aa465f740 1 -- 172.21.15.116:0/104326231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0b4e6cb40 msgr2=0x55d0b4e7abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:46.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.746+0000 7f7aa465f740 1 --2- 172.21.15.116:0/104326231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0b4e6cb40 0x55d0b4e7abc0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f7a90097ae0 tx=0x7f7a9006afc0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.746+0000 7f7aa465f740 1 -- 172.21.15.116:0/104326231 shutdown_connections 2024-09-12T03:17:46.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.746+0000 7f7aa465f740 1 --2- 172.21.15.116:0/104326231 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d0b4ef1a90 0x55d0b4ef3eb0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.746+0000 7f7a7b7fe640 0 client.19458 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:46.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.746+0000 7f7aa465f740 1 --2- 172.21.15.116:0/104326231 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0b4e76370 0x55d0b4eefb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.746+0000 7f7aa465f740 1 --2- 172.21.15.116:0/104326231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d0b4e6cb40 0x55d0b4e7abc0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.746+0000 7f7aa465f740 1 --2- 172.21.15.116:0/104326231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0b4e6b5d0 0x55d0b4e7a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.746+0000 7f7aa465f740 1 -- 172.21.15.116:0/104326231 >> 172.21.15.116:0/104326231 conn(0x55d0b4e34310 msgr2=0x55d0b4e74810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:46.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.746+0000 7f7aa465f740 1 -- 172.21.15.116:0/104326231 shutdown_connections 2024-09-12T03:17:46.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.746+0000 7f7aa465f740 1 -- 172.21.15.116:0/104326231 wait complete. 2024-09-12T03:17:46.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.746+0000 7f7aa465f740 20 client.19458 trim_cache size 0 max 16384 2024-09-12T03:17:46.759 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:17:46.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.746+0000 7f7aa2286640 1 -- 172.21.15.116:0/1303776976 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7a840a2c00 msgr2=0x7f7a840a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:46.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.746+0000 7f7aa2286640 1 --2- 172.21.15.116:0/1303776976 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7a840a2c00 0x7f7a840a50c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7a9404c000 tx=0x7f7a9404d000 comp rx=0 tx=0).stop 2024-09-12T03:17:46.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.746+0000 7f7aa2286640 1 -- 172.21.15.116:0/1303776976 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a9c102790 msgr2=0x7f7a9c1a0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:46.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.746+0000 7f7aa2286640 1 --2- 172.21.15.116:0/1303776976 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a9c102790 0x7f7a9c1a0ab0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000b4a0 tx=0x7f7a9000b8a0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.747+0000 7f7aa2286640 1 -- 172.21.15.116:0/1303776976 shutdown_connections 2024-09-12T03:17:46.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.747+0000 7f7aa2286640 1 --2- 172.21.15.116:0/1303776976 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f7a840a2c00 0x7f7a840a50c0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.747+0000 7f7aa2286640 1 --2- 172.21.15.116:0/1303776976 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a9c106f10 0x7f7a9c19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.747+0000 7f7aa2286640 1 --2- 172.21.15.116:0/1303776976 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a9c102790 0x7f7a9c1a0ab0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.747+0000 7f7aa2286640 1 --2- 172.21.15.116:0/1303776976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a9c101de0 0x7f7a9c1a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.747+0000 7f7aa2286640 1 -- 172.21.15.116:0/1303776976 >> 172.21.15.116:0/1303776976 conn(0x7f7a9c0fd8c0 msgr2=0x7f7a9c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:46.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.747+0000 7f7aa2286640 1 -- 172.21.15.116:0/1303776976 shutdown_connections 2024-09-12T03:17:46.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:46.747+0000 7f7aa2286640 1 -- 172.21.15.116:0/1303776976 wait complete. 2024-09-12T03:17:47.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:47 smithi177 ceph-mon[35305]: pgmap v782: 97 pgs: 97 active+clean; 6.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:17:47.755 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:47 smithi183 ceph-mon[35535]: pgmap v782: 97 pgs: 97 active+clean; 6.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:17:47.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:47 smithi116 ceph-mon[31410]: pgmap v782: 97 pgs: 97 active+clean; 6.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:17:48.489 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:17:48.489 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:17:48.748 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:49.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.288+0000 7f3b98a6e640 1 -- 172.21.15.116:0/2993539902 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b94100a80 msgr2=0x7f3b94100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:49.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.288+0000 7f3b98a6e640 1 --2- 172.21.15.116:0/2993539902 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b94100a80 0x7f3b94100ee0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009930 tx=0x7f3b7c02a420 comp rx=0 tx=0).stop 2024-09-12T03:17:49.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.288+0000 7f3b98a6e640 1 -- 172.21.15.116:0/2993539902 shutdown_connections 2024-09-12T03:17:49.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.288+0000 7f3b98a6e640 1 --2- 172.21.15.116:0/2993539902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b94101420 0x7f3b9410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.288+0000 7f3b98a6e640 1 --2- 172.21.15.116:0/2993539902 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b94100a80 0x7f3b94100ee0 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.288+0000 7f3b98a6e640 1 --2- 172.21.15.116:0/2993539902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b94106a80 0x7f3b94106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.288+0000 7f3b98a6e640 1 -- 172.21.15.116:0/2993539902 >> 172.21.15.116:0/2993539902 conn(0x7f3b940fc7a0 msgr2=0x7f3b940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:49.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.288+0000 7f3b98a6e640 1 -- 172.21.15.116:0/2993539902 shutdown_connections 2024-09-12T03:17:49.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.288+0000 7f3b98a6e640 1 -- 172.21.15.116:0/2993539902 wait complete. 2024-09-12T03:17:49.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.289+0000 7f3b98a6e640 1 Processor -- start 2024-09-12T03:17:49.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.289+0000 7f3b98a6e640 1 -- start start 2024-09-12T03:17:49.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.289+0000 7f3b98a6e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b94100a80 0x7f3b941984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.289+0000 7f3b98a6e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b94101420 0x7f3b94198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.289+0000 7f3b98a6e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b94106a80 0x7f3b9419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.289+0000 7f3b98a6e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3b94110ea0 con 0x7f3b94101420 2024-09-12T03:17:49.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.289+0000 7f3b98a6e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3b94110d20 con 0x7f3b94106a80 2024-09-12T03:17:49.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.289+0000 7f3b98a6e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3b94111020 con 0x7f3b94100a80 2024-09-12T03:17:49.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.289+0000 7f3b8bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b94101420 0x7f3b94198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.289+0000 7f3b8bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b94101420 0x7f3b94198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46478/0 (socket says 172.21.15.116:46478) 2024-09-12T03:17:49.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.289+0000 7f3b8bfff640 1 -- 172.21.15.116:0/3384270337 learned_addr learned my addr 172.21.15.116:0/3384270337 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:49.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.289+0000 7f3b9259c640 1 --2- 172.21.15.116:0/3384270337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b94100a80 0x7f3b941984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.289+0000 7f3b8bfff640 1 -- 172.21.15.116:0/3384270337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b94100a80 msgr2=0x7f3b941984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:49.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.289+0000 7f3b92d9d640 1 --2- 172.21.15.116:0/3384270337 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b94106a80 0x7f3b9419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.289+0000 7f3b8bfff640 1 --2- 172.21.15.116:0/3384270337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b94100a80 0x7f3b941984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.289+0000 7f3b8bfff640 1 -- 172.21.15.116:0/3384270337 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b94106a80 msgr2=0x7f3b9419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:49.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.289+0000 7f3b8bfff640 1 --2- 172.21.15.116:0/3384270337 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b94106a80 0x7f3b9419cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.289+0000 7f3b8bfff640 1 -- 172.21.15.116:0/3384270337 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b9419d470 con 0x7f3b94101420 2024-09-12T03:17:49.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.289+0000 7f3b9259c640 1 --2- 172.21.15.116:0/3384270337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b94100a80 0x7f3b941984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:49.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.289+0000 7f3b8bfff640 1 --2- 172.21.15.116:0/3384270337 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b94101420 0x7f3b94198a00 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009a50 tx=0x7f3b7c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:49.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.290+0000 7f3b8b7fe640 1 -- 172.21.15.116:0/3384270337 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c03f030 con 0x7f3b94101420 2024-09-12T03:17:49.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.290+0000 7f3b8b7fe640 1 -- 172.21.15.116:0/3384270337 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b7c003ce0 con 0x7f3b94101420 2024-09-12T03:17:49.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.290+0000 7f3b8b7fe640 1 -- 172.21.15.116:0/3384270337 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c035070 con 0x7f3b94101420 2024-09-12T03:17:49.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.290+0000 7f3b98a6e640 1 -- 172.21.15.116:0/3384270337 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b9419d700 con 0x7f3b94101420 2024-09-12T03:17:49.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.290+0000 7f3b98a6e640 1 -- 172.21.15.116:0/3384270337 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b941a5040 con 0x7f3b94101420 2024-09-12T03:17:49.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.292+0000 7f3b8b7fe640 1 -- 172.21.15.116:0/3384270337 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3b7c042030 con 0x7f3b94101420 2024-09-12T03:17:49.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.292+0000 7f3b9ae47740 1 Processor -- start 2024-09-12T03:17:49.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.292+0000 7f3b9ae47740 1 -- start start 2024-09-12T03:17:49.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.293+0000 7f3b8b7fe640 1 --2- 172.21.15.116:0/3384270337 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3b800a2b80 0x7f3b800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.293+0000 7f3b9ae47740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f57630ea20 0x55f57630ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.293+0000 7f3b9ae47740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f57630f340 0x55f57631dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.293+0000 7f3b9ae47740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f57631e150 0x55f576320540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.293+0000 7f3b9ae47740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55f576274280 con 0x55f57631e150 2024-09-12T03:17:49.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.293+0000 7f3b9ae47740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f57621bb10 con 0x55f57630f340 2024-09-12T03:17:49.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.293+0000 7f3b9ae47740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f576270240 con 0x55f57630ea20 2024-09-12T03:17:49.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.293+0000 7f3b8b7fe640 1 -- 172.21.15.116:0/3384270337 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f3b7c0ef660 con 0x7f3b94101420 2024-09-12T03:17:49.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.293+0000 7f3b92d9d640 1 --2- 172.21.15.116:0/3384270337 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3b800a2b80 0x7f3b800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.293+0000 7f3b92d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f57631e150 0x55f576320540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.293+0000 7f3b9259c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f57630ea20 0x55f57630ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.293+0000 7f3b92d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f57631e150 0x55f576320540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46484/0 (socket says 172.21.15.116:46484) 2024-09-12T03:17:49.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.293+0000 7f3b92d9d640 1 -- 172.21.15.116:0/3632066945 learned_addr learned my addr 172.21.15.116:0/3632066945 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:49.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.293+0000 7f3b9259c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f57630ea20 0x55f57630ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.116:54930/0 (socket says 172.21.15.116:54930) 2024-09-12T03:17:49.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.293+0000 7f3b8bfff640 1 --2- 172.21.15.116:0/3632066945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f57630f340 0x55f57631dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.293+0000 7f3b9259c640 1 -- 172.21.15.116:0/3632066945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f57630f340 msgr2=0x55f57631dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:49.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.293+0000 7f3b9259c640 1 --2- 172.21.15.116:0/3632066945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f57630f340 0x55f57631dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.293+0000 7f3b9259c640 1 -- 172.21.15.116:0/3632066945 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f57631e150 msgr2=0x55f576320540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:49.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.293+0000 7f3b9259c640 1 --2- 172.21.15.116:0/3632066945 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f57631e150 0x55f576320540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.293+0000 7f3b9259c640 1 -- 172.21.15.116:0/3632066945 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5762281f0 con 0x55f57630ea20 2024-09-12T03:17:49.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.293+0000 7f3b92d9d640 1 --2- 172.21.15.116:0/3632066945 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f57631e150 0x55f576320540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:17:49.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.293+0000 7f3b92d9d640 1 --2- 172.21.15.116:0/3384270337 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3b800a2b80 0x7f3b800a5040 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c005fd0 tx=0x7f3b8c00ac80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:49.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.294+0000 7f3b9259c640 1 --2- 172.21.15.116:0/3632066945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f57630ea20 0x55f57630ee00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3b84007760 tx=0x7f3b84007c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=543b8ad0d140706a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:49.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.294+0000 7f3b88ff9640 1 -- 172.21.15.116:0/3632066945 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8400d950 con 0x55f57630ea20 2024-09-12T03:17:49.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.294+0000 7f3b88ff9640 1 -- 172.21.15.116:0/3632066945 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b84018070 con 0x55f57630ea20 2024-09-12T03:17:49.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.294+0000 7f3b88ff9640 1 -- 172.21.15.116:0/3632066945 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b84013db0 con 0x55f57630ea20 2024-09-12T03:17:49.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.295+0000 7f3b9ae47740 1 -- 172.21.15.116:0/3632066945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f57630ea20 msgr2=0x55f57630ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:49.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.295+0000 7f3b9ae47740 1 --2- 172.21.15.116:0/3632066945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f57630ea20 0x55f57630ee00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3b84007760 tx=0x7f3b84007c20 comp rx=0 tx=0).stop 2024-09-12T03:17:49.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.295+0000 7f3b9ae47740 1 -- 172.21.15.116:0/3632066945 shutdown_connections 2024-09-12T03:17:49.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.295+0000 7f3b9ae47740 1 --2- 172.21.15.116:0/3632066945 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f57631e150 0x55f576320540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.295+0000 7f3b9ae47740 1 --2- 172.21.15.116:0/3632066945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f57630f340 0x55f57631dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.295+0000 7f3b9ae47740 1 --2- 172.21.15.116:0/3632066945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f57630ea20 0x55f57630ee00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.295+0000 7f3b9ae47740 1 -- 172.21.15.116:0/3632066945 >> 172.21.15.116:0/3632066945 conn(0x55f57630b0f0 msgr2=0x55f57630b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:49.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.295+0000 7f3b9ae47740 1 -- 172.21.15.116:0/3632066945 shutdown_connections 2024-09-12T03:17:49.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.295+0000 7f3b9ae47740 1 -- 172.21.15.116:0/3632066945 wait complete. 2024-09-12T03:17:49.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.295+0000 7f3b9ae47740 1 -- start start 2024-09-12T03:17:49.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.295+0000 7f3b9ae47740 1 Processor -- start 2024-09-12T03:17:49.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.295+0000 7f3b9ae47740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f57630ea20 0x55f57631b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.295+0000 7f3b9ae47740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f57630f340 0x55f57631b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.295+0000 7f3b9ae47740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f57631e150 0x55f57631bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.295+0000 7f3b9ae47740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55f576320d20 con 0x55f57630f340 2024-09-12T03:17:49.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.295+0000 7f3b9ae47740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f57622ef10 con 0x55f57631e150 2024-09-12T03:17:49.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.295+0000 7f3b9ae47740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f576320ea0 con 0x55f57630ea20 2024-09-12T03:17:49.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.295+0000 7f3b9ae47740 10 client.0 _ll_register_callbacks cb 0x55f5761f7a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:49.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.295+0000 7f3b8bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f57630f340 0x55f57631b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.295+0000 7f3b8bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f57630f340 0x55f57631b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46488/0 (socket says 172.21.15.116:46488) 2024-09-12T03:17:49.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.296+0000 7f3b8bfff640 1 -- 172.21.15.116:0/261068027 learned_addr learned my addr 172.21.15.116:0/261068027 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:49.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.296+0000 7f3b9259c640 1 --2- 172.21.15.116:0/261068027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f57630ea20 0x55f57631b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.296+0000 7f3b92d9d640 1 --2- 172.21.15.116:0/261068027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f57631e150 0x55f57631bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.296+0000 7f3b8bfff640 1 -- 172.21.15.116:0/261068027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f57630ea20 msgr2=0x55f57631b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:49.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.296+0000 7f3b8bfff640 1 --2- 172.21.15.116:0/261068027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f57630ea20 0x55f57631b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.296+0000 7f3b8bfff640 1 -- 172.21.15.116:0/261068027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f57631e150 msgr2=0x55f57631bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:49.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.296+0000 7f3b8bfff640 1 --2- 172.21.15.116:0/261068027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f57631e150 0x55f57631bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.296+0000 7f3b8bfff640 1 -- 172.21.15.116:0/261068027 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f575fe56a0 con 0x55f57630f340 2024-09-12T03:17:49.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.296+0000 7f3b8bfff640 1 --2- 172.21.15.116:0/261068027 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f57630f340 0x55f57631b590 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c0bbbf0 tx=0x7f3b7c0b8190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f837639417fa90a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:49.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.296+0000 7f3b79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:49.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.296+0000 7f3b79ffb640 1 -- 172.21.15.116:0/261068027 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c049040 con 0x55f57630f340 2024-09-12T03:17:49.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.296+0000 7f3b79ffb640 1 -- 172.21.15.116:0/261068027 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f3b7c0b8560 con 0x55f57630f340 2024-09-12T03:17:49.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.296+0000 7f3b9ae47740 1 -- 172.21.15.116:0/261068027 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b84018070 con 0x55f57630f340 2024-09-12T03:17:49.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.296+0000 7f3b79ffb640 1 -- 172.21.15.116:0/261068027 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3b7c0bb850 con 0x55f57630f340 2024-09-12T03:17:49.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.296+0000 7f3b9ae47740 10 client.19470 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:49.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.296+0000 7f3b9ae47740 1 -- 172.21.15.116:0/261068027 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f57632c990 con 0x55f57630f340 2024-09-12T03:17:49.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.296+0000 7f3b79ffb640 1 -- 172.21.15.116:0/261068027 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b84018070 con 0x55f57630f340 2024-09-12T03:17:49.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.297+0000 7f3b79ffb640 1 -- 172.21.15.116:0/261068027 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3b7c03b460 con 0x55f57630f340 2024-09-12T03:17:49.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.297+0000 7f3b9ae47740 10 client.19470 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:49.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.297+0000 7f3b9ae47740 10 client.19470 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:17:49.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.297+0000 7f3b9ae47740 20 client.19470 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:49.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.297+0000 7f3b9ae47740 1 --2- 172.21.15.116:0/261068027 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f57632dee0 0x55f576330300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.297+0000 7f3b9ae47740 4 client.19470 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:17:49.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.297+0000 7f3b9ae47740 1 -- 172.21.15.116:0/261068027 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5762817f0 con 0x55f57632dee0 2024-09-12T03:17:49.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.297+0000 7f3b9259c640 1 --2- 172.21.15.116:0/261068027 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f57632dee0 0x55f576330300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.297+0000 7f3b9259c640 1 --2- 172.21.15.116:0/261068027 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f57632dee0 0x55f576330300 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa1fabf462288ddf server_cookie=d8fb470e4e0264c0 in_seq=0 out_seq=0 2024-09-12T03:17:49.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.297+0000 7f3b79ffb640 10 client.19470 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:49.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.298+0000 7f3b79ffb640 1 -- 172.21.15.116:0/261068027 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f5762817f0 con 0x55f57632dee0 2024-09-12T03:17:49.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.298+0000 7f3b79ffb640 10 client.19470 handle_command_reply: tid=0 2024-09-12T03:17:49.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.298+0000 7f3b9ae47740 1 client.19470 shutdown 2024-09-12T03:17:49.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.298+0000 7f3b9ae47740 1 -- 172.21.15.116:0/261068027 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f57630f340 msgr2=0x55f57631b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:49.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.298+0000 7f3b9ae47740 1 --2- 172.21.15.116:0/261068027 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f57630f340 0x55f57631b590 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c0bbbf0 tx=0x7f3b7c0b8190 comp rx=0 tx=0).stop 2024-09-12T03:17:49.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.298+0000 7f3b9ae47740 1 -- 172.21.15.116:0/261068027 shutdown_connections 2024-09-12T03:17:49.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.298+0000 7f3b9ae47740 1 --2- 172.21.15.116:0/261068027 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f57632dee0 0x55f576330300 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.298+0000 7f3b9ae47740 1 --2- 172.21.15.116:0/261068027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f57631e150 0x55f57631bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.298+0000 7f3b9ae47740 1 --2- 172.21.15.116:0/261068027 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f57630f340 0x55f57631b590 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.298+0000 7f3b9ae47740 1 --2- 172.21.15.116:0/261068027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f57630ea20 0x55f57631b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.298+0000 7f3b9ae47740 1 -- 172.21.15.116:0/261068027 >> 172.21.15.116:0/261068027 conn(0x55f57630b0f0 msgr2=0x55f576310e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:49.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.298+0000 7f3b79ffb640 0 client.19470 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:49.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.298+0000 7f3b9ae47740 1 -- 172.21.15.116:0/261068027 shutdown_connections 2024-09-12T03:17:49.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.298+0000 7f3b9ae47740 1 -- 172.21.15.116:0/261068027 wait complete. 2024-09-12T03:17:49.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.298+0000 7f3b9ae47740 20 client.19470 trim_cache size 0 max 16384 2024-09-12T03:17:49.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.315+0000 7f3b9ae47740 1 Processor -- start 2024-09-12T03:17:49.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.315+0000 7f3b9ae47740 1 -- start start 2024-09-12T03:17:49.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.315+0000 7f3b9ae47740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f57630ea20 0x55f576335bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.315+0000 7f3b9ae47740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f57630f340 0x55f576336100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.315+0000 7f3b9ae47740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f57631e150 0x55f5763182b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.316+0000 7f3b9ae47740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3b941adaf0 con 0x55f57631e150 2024-09-12T03:17:49.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.316+0000 7f3b9ae47740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3b941ab3c0 con 0x55f57630f340 2024-09-12T03:17:49.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.316+0000 7f3b9ae47740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3b941a9230 con 0x55f57630ea20 2024-09-12T03:17:49.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.316+0000 7f3b92d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f57631e150 0x55f5763182b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.316+0000 7f3b92d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f57631e150 0x55f5763182b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46502/0 (socket says 172.21.15.116:46502) 2024-09-12T03:17:49.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.316+0000 7f3b92d9d640 1 -- 172.21.15.116:0/1278743883 learned_addr learned my addr 172.21.15.116:0/1278743883 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:49.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.316+0000 7f3b9259c640 1 --2- 172.21.15.116:0/1278743883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f57630ea20 0x55f576335bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.316+0000 7f3b8bfff640 1 --2- 172.21.15.116:0/1278743883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f57630f340 0x55f576336100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.316+0000 7f3b92d9d640 1 -- 172.21.15.116:0/1278743883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f57630ea20 msgr2=0x55f576335bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:49.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.316+0000 7f3b92d9d640 1 --2- 172.21.15.116:0/1278743883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f57630ea20 0x55f576335bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.316+0000 7f3b92d9d640 1 -- 172.21.15.116:0/1278743883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f57630f340 msgr2=0x55f576336100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:49.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.316+0000 7f3b92d9d640 1 --2- 172.21.15.116:0/1278743883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f57630f340 0x55f576336100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.316+0000 7f3b92d9d640 1 -- 172.21.15.116:0/1278743883 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f576306830 con 0x55f57631e150 2024-09-12T03:17:49.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.316+0000 7f3b92d9d640 1 --2- 172.21.15.116:0/1278743883 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f57631e150 0x55f5763182b0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c00a710 tx=0x7f3b8c00af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a306d4401e98ceea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:49.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.316+0000 7f3b7bfff640 1 -- 172.21.15.116:0/1278743883 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c0117f0 con 0x55f57631e150 2024-09-12T03:17:49.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.316+0000 7f3b7bfff640 1 -- 172.21.15.116:0/1278743883 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b8c020400 con 0x55f57631e150 2024-09-12T03:17:49.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.316+0000 7f3b7bfff640 1 -- 172.21.15.116:0/1278743883 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c0285f0 con 0x55f57631e150 2024-09-12T03:17:49.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.317+0000 7f3b9ae47740 1 -- 172.21.15.116:0/1278743883 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f57631e150 msgr2=0x55f5763182b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:49.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.317+0000 7f3b9ae47740 1 --2- 172.21.15.116:0/1278743883 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f57631e150 0x55f5763182b0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c00a710 tx=0x7f3b8c00af90 comp rx=0 tx=0).stop 2024-09-12T03:17:49.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.317+0000 7f3b9ae47740 1 -- 172.21.15.116:0/1278743883 shutdown_connections 2024-09-12T03:17:49.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.317+0000 7f3b9ae47740 1 --2- 172.21.15.116:0/1278743883 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f57631e150 0x55f5763182b0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.317+0000 7f3b9ae47740 1 --2- 172.21.15.116:0/1278743883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f57630f340 0x55f576336100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.317+0000 7f3b9ae47740 1 --2- 172.21.15.116:0/1278743883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f57630ea20 0x55f576335bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.317+0000 7f3b9ae47740 1 -- 172.21.15.116:0/1278743883 >> 172.21.15.116:0/1278743883 conn(0x55f57630b0f0 msgr2=0x55f57631cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:49.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.317+0000 7f3b9ae47740 1 -- 172.21.15.116:0/1278743883 shutdown_connections 2024-09-12T03:17:49.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.317+0000 7f3b9ae47740 1 -- 172.21.15.116:0/1278743883 wait complete. 2024-09-12T03:17:49.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.317+0000 7f3b9ae47740 1 -- start start 2024-09-12T03:17:49.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.317+0000 7f3b9ae47740 1 Processor -- start 2024-09-12T03:17:49.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.318+0000 7f3b9ae47740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f57630ea20 0x55f5763221a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.318+0000 7f3b9ae47740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f57630f340 0x55f5763226e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.318+0000 7f3b9ae47740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f57631e150 0x55f576322c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.318+0000 7f3b9ae47740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3b941a9880 con 0x55f57631e150 2024-09-12T03:17:49.322 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.318+0000 7f3b9ae47740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3b941aacf0 con 0x55f57630ea20 2024-09-12T03:17:49.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.318+0000 7f3b9ae47740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3b941a9f50 con 0x55f57630f340 2024-09-12T03:17:49.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.318+0000 7f3b9ae47740 10 client.0 _ll_register_callbacks cb 0x7f3b7c0dc0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:49.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.318+0000 7f3b92d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f57631e150 0x55f576322c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.318+0000 7f3b92d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f57631e150 0x55f576322c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46508/0 (socket says 172.21.15.116:46508) 2024-09-12T03:17:49.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.318+0000 7f3b92d9d640 1 -- 172.21.15.116:0/875065946 learned_addr learned my addr 172.21.15.116:0/875065946 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:49.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.318+0000 7f3b8bfff640 1 --2- 172.21.15.116:0/875065946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f57630f340 0x55f5763226e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.318+0000 7f3b9259c640 1 --2- 172.21.15.116:0/875065946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f57630ea20 0x55f5763221a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.318+0000 7f3b92d9d640 1 -- 172.21.15.116:0/875065946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f57630f340 msgr2=0x55f5763226e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:49.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.318+0000 7f3b92d9d640 1 --2- 172.21.15.116:0/875065946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f57630f340 0x55f5763226e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.318+0000 7f3b92d9d640 1 -- 172.21.15.116:0/875065946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f57630ea20 msgr2=0x55f5763221a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:49.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.318+0000 7f3b92d9d640 1 --2- 172.21.15.116:0/875065946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f57630ea20 0x55f5763221a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.318+0000 7f3b92d9d640 1 -- 172.21.15.116:0/875065946 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f576397e30 con 0x55f57631e150 2024-09-12T03:17:49.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.318+0000 7f3b8bfff640 1 --2- 172.21.15.116:0/875065946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f57630f340 0x55f5763226e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:49.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.318+0000 7f3b92d9d640 1 --2- 172.21.15.116:0/875065946 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f57631e150 0x55f576322c20 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c00a2f0 tx=0x7f3b8c004c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb61f67fcb7894fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:49.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.318+0000 7f3b79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:49.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.318+0000 7f3b79ffb640 1 -- 172.21.15.116:0/875065946 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c02e040 con 0x55f57631e150 2024-09-12T03:17:49.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.319+0000 7f3b79ffb640 1 -- 172.21.15.116:0/875065946 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f3b8c0207b0 con 0x55f57631e150 2024-09-12T03:17:49.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.319+0000 7f3b9ae47740 1 -- 172.21.15.116:0/875065946 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b8c020400 con 0x55f57631e150 2024-09-12T03:17:49.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.319+0000 7f3b79ffb640 1 -- 172.21.15.116:0/875065946 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3b8c004e40 con 0x55f57631e150 2024-09-12T03:17:49.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.319+0000 7f3b9ae47740 10 client.19476 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:49.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.319+0000 7f3b9ae47740 1 -- 172.21.15.116:0/875065946 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f576399170 con 0x55f57631e150 2024-09-12T03:17:49.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.319+0000 7f3b79ffb640 1 -- 172.21.15.116:0/875065946 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b8c020400 con 0x55f57631e150 2024-09-12T03:17:49.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.319+0000 7f3b79ffb640 1 -- 172.21.15.116:0/875065946 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3b8c031070 con 0x55f57631e150 2024-09-12T03:17:49.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.319+0000 7f3b9ae47740 10 client.19476 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:49.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.319+0000 7f3b9ae47740 10 client.19476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:17:49.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.319+0000 7f3b9ae47740 20 client.19476 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:49.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.319+0000 7f3b9ae47740 1 --2- 172.21.15.116:0/875065946 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f576399850 0x55f57639bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.319+0000 7f3b9ae47740 4 client.19476 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:17:49.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.319+0000 7f3b9ae47740 1 -- 172.21.15.116:0/875065946 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f57627bd00 con 0x55f576399850 2024-09-12T03:17:49.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.319+0000 7f3b9259c640 1 --2- 172.21.15.116:0/875065946 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f576399850 0x55f57639bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.320+0000 7f3b79ffb640 10 client.19476 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:49.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.320+0000 7f3b9259c640 1 --2- 172.21.15.116:0/875065946 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f576399850 0x55f57639bc70 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af0df2f53d5e5a92 server_cookie=c3a4ab125127025f in_seq=0 out_seq=0 2024-09-12T03:17:49.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.321+0000 7f3b79ffb640 1 -- 172.21.15.116:0/875065946 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f57627bd00 con 0x55f576399850 2024-09-12T03:17:49.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.321+0000 7f3b79ffb640 10 client.19476 handle_command_reply: tid=0 2024-09-12T03:17:49.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.321+0000 7f3b9ae47740 1 client.19476 shutdown 2024-09-12T03:17:49.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.321+0000 7f3b9ae47740 1 -- 172.21.15.116:0/875065946 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f57631e150 msgr2=0x55f576322c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:49.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.321+0000 7f3b9ae47740 1 --2- 172.21.15.116:0/875065946 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f57631e150 0x55f576322c20 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c00a2f0 tx=0x7f3b8c004c50 comp rx=0 tx=0).stop 2024-09-12T03:17:49.326 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.321+0000 7f3b9ae47740 1 -- 172.21.15.116:0/875065946 shutdown_connections 2024-09-12T03:17:49.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.321+0000 7f3b9ae47740 1 --2- 172.21.15.116:0/875065946 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55f576399850 0x55f57639bc70 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.321+0000 7f3b9ae47740 1 --2- 172.21.15.116:0/875065946 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f57631e150 0x55f576322c20 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.321+0000 7f3b9ae47740 1 --2- 172.21.15.116:0/875065946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f57630f340 0x55f5763226e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.321+0000 7f3b9ae47740 1 --2- 172.21.15.116:0/875065946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f57630ea20 0x55f5763221a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.321+0000 7f3b9ae47740 1 -- 172.21.15.116:0/875065946 >> 172.21.15.116:0/875065946 conn(0x55f57630b0f0 msgr2=0x55f576309a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:49.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.321+0000 7f3b79ffb640 0 client.19476 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:49.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.321+0000 7f3b9ae47740 1 -- 172.21.15.116:0/875065946 shutdown_connections 2024-09-12T03:17:49.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.321+0000 7f3b9ae47740 1 -- 172.21.15.116:0/875065946 wait complete. 2024-09-12T03:17:49.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.321+0000 7f3b9ae47740 20 client.19476 trim_cache size 0 max 16384 2024-09-12T03:17:49.327 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:17:49.327 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:17:49.328 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "67b4cd14-d0e0-41e5-9111-a0b36ef7c0e1", 2024-09-12T03:17:49.328 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:17:49.328 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:17:49.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.322+0000 7f3b98a6e640 1 -- 172.21.15.116:0/3384270337 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3b800a2b80 msgr2=0x7f3b800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:49.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.322+0000 7f3b98a6e640 1 --2- 172.21.15.116:0/3384270337 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3b800a2b80 0x7f3b800a5040 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c005fd0 tx=0x7f3b8c00ac80 comp rx=0 tx=0).stop 2024-09-12T03:17:49.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.322+0000 7f3b98a6e640 1 -- 172.21.15.116:0/3384270337 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b94101420 msgr2=0x7f3b94198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:49.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.322+0000 7f3b98a6e640 1 --2- 172.21.15.116:0/3384270337 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b94101420 0x7f3b94198a00 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009a50 tx=0x7f3b7c004270 comp rx=0 tx=0).stop 2024-09-12T03:17:49.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.322+0000 7f3b98a6e640 1 -- 172.21.15.116:0/3384270337 shutdown_connections 2024-09-12T03:17:49.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.322+0000 7f3b98a6e640 1 --2- 172.21.15.116:0/3384270337 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f3b800a2b80 0x7f3b800a5040 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.322+0000 7f3b98a6e640 1 --2- 172.21.15.116:0/3384270337 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b94106a80 0x7f3b9419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.322+0000 7f3b98a6e640 1 --2- 172.21.15.116:0/3384270337 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b94101420 0x7f3b94198a00 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.322+0000 7f3b98a6e640 1 --2- 172.21.15.116:0/3384270337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b94100a80 0x7f3b941984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.322+0000 7f3b98a6e640 1 -- 172.21.15.116:0/3384270337 >> 172.21.15.116:0/3384270337 conn(0x7f3b940fc7a0 msgr2=0x7f3b9406a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:49.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.322+0000 7f3b98a6e640 1 -- 172.21.15.116:0/3384270337 shutdown_connections 2024-09-12T03:17:49.329 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:49.322+0000 7f3b98a6e640 1 -- 172.21.15.116:0/3384270337 wait complete. 2024-09-12T03:17:49.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:49 smithi116 ceph-mon[31410]: pgmap v783: 97 pgs: 97 active+clean; 7.0 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:17:49.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:49 smithi177 ceph-mon[35305]: pgmap v783: 97 pgs: 97 active+clean; 7.0 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:17:49.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:49 smithi183 ceph-mon[35535]: pgmap v783: 97 pgs: 97 active+clean; 7.0 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:17:50.007 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:17:50.251 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:50.674 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:50 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:17:50.674 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:50 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:17:50.674 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:50 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:17:50.675 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:50 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:17:50.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:50 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:17:50.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:50 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:17:50.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:50 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:17:50.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:50 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:17:50.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.756+0000 7f71a442b640 1 -- 172.21.15.116:0/1687045058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f719c102780 msgr2=0x7f719c1067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.756+0000 7f71a442b640 1 --2- 172.21.15.116:0/1687045058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f719c102780 0x7f719c1067d0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f7190009900 tx=0x7f719002a410 comp rx=0 tx=0).stop 2024-09-12T03:17:50.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.756+0000 7f71a442b640 1 -- 172.21.15.116:0/1687045058 shutdown_connections 2024-09-12T03:17:50.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.756+0000 7f71a442b640 1 --2- 172.21.15.116:0/1687045058 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f719c106f00 0x7f719c10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.756+0000 7f71a442b640 1 --2- 172.21.15.116:0/1687045058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f719c102780 0x7f719c1067d0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.756+0000 7f71a442b640 1 --2- 172.21.15.116:0/1687045058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f719c101dd0 0x7f719c1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.756+0000 7f71a442b640 1 -- 172.21.15.116:0/1687045058 >> 172.21.15.116:0/1687045058 conn(0x7f719c0fd8b0 msgr2=0x7f719c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:50.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.756+0000 7f71a442b640 1 -- 172.21.15.116:0/1687045058 shutdown_connections 2024-09-12T03:17:50.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.756+0000 7f71a442b640 1 -- 172.21.15.116:0/1687045058 wait complete. 2024-09-12T03:17:50.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.756+0000 7f71a442b640 1 Processor -- start 2024-09-12T03:17:50.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.757+0000 7f71a442b640 1 -- start start 2024-09-12T03:17:50.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.757+0000 7f71a442b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f719c101dd0 0x7f719c1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.757+0000 7f71a442b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f719c102780 0x7f719c198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.757+0000 7f71a442b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f719c106f00 0x7f719c19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.757+0000 7f71a442b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f719c111f80 con 0x7f719c106f00 2024-09-12T03:17:50.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.757+0000 7f71a442b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f719c111e00 con 0x7f719c102780 2024-09-12T03:17:50.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.757+0000 7f71a442b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f719c112100 con 0x7f719c101dd0 2024-09-12T03:17:50.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.757+0000 7f71a29c8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f719c106f00 0x7f719c19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.757+0000 7f71a29c8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f719c106f00 0x7f719c19cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46526/0 (socket says 172.21.15.116:46526) 2024-09-12T03:17:50.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.757+0000 7f71a29c8640 1 -- 172.21.15.116:0/2306390107 learned_addr learned my addr 172.21.15.116:0/2306390107 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:50.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.757+0000 7f71a19c6640 1 --2- 172.21.15.116:0/2306390107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f719c102780 0x7f719c198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.757+0000 7f71a21c7640 1 --2- 172.21.15.116:0/2306390107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f719c101dd0 0x7f719c1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.757+0000 7f71a29c8640 1 -- 172.21.15.116:0/2306390107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f719c101dd0 msgr2=0x7f719c1984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.757+0000 7f71a29c8640 1 --2- 172.21.15.116:0/2306390107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f719c101dd0 0x7f719c1984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.757+0000 7f71a29c8640 1 -- 172.21.15.116:0/2306390107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f719c102780 msgr2=0x7f719c198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.757+0000 7f71a29c8640 1 --2- 172.21.15.116:0/2306390107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f719c102780 0x7f719c198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.757+0000 7f71a29c8640 1 -- 172.21.15.116:0/2306390107 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f719c19d490 con 0x7f719c106f00 2024-09-12T03:17:50.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.757+0000 7f71a19c6640 1 --2- 172.21.15.116:0/2306390107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f719c102780 0x7f719c198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:50.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.757+0000 7f71a21c7640 1 --2- 172.21.15.116:0/2306390107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f719c101dd0 0x7f719c1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:50.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.757+0000 7f71a29c8640 1 --2- 172.21.15.116:0/2306390107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f719c106f00 0x7f719c19cdb0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f719400eaa0 tx=0x7f719400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:50.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.757+0000 7f718f7fe640 1 -- 172.21.15.116:0/2306390107 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719400cdb0 con 0x7f719c106f00 2024-09-12T03:17:50.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.758+0000 7f718f7fe640 1 -- 172.21.15.116:0/2306390107 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7194005020 con 0x7f719c106f00 2024-09-12T03:17:50.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.758+0000 7f71a442b640 1 -- 172.21.15.116:0/2306390107 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f719c19d780 con 0x7f719c106f00 2024-09-12T03:17:50.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.758+0000 7f718f7fe640 1 -- 172.21.15.116:0/2306390107 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71940116b0 con 0x7f719c106f00 2024-09-12T03:17:50.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.758+0000 7f71a442b640 1 -- 172.21.15.116:0/2306390107 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f719c1a5060 con 0x7f719c106f00 2024-09-12T03:17:50.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.760+0000 7f718f7fe640 1 -- 172.21.15.116:0/2306390107 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f7194005560 con 0x7f719c106f00 2024-09-12T03:17:50.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.760+0000 7f71a6804740 1 Processor -- start 2024-09-12T03:17:50.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.760+0000 7f71a6804740 1 -- start start 2024-09-12T03:17:50.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.760+0000 7f718f7fe640 1 --2- 172.21.15.116:0/2306390107 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f71880a2b80 0x7f71880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.760+0000 7f71a6804740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597a1dd44a0 0x5597a1dd4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.761+0000 7f71a6804740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597a1dd4dc0 0x5597a1ddf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.761+0000 7f71a6804740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a1ddf680 0x5597a1de1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.761+0000 7f71a6804740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5597a1d3a2e0 con 0x5597a1ddf680 2024-09-12T03:17:50.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.761+0000 7f71a6804740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5597a1ce1ce0 con 0x5597a1dd4dc0 2024-09-12T03:17:50.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.761+0000 7f71a6804740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5597a1d362a0 con 0x5597a1dd44a0 2024-09-12T03:17:50.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.761+0000 7f71a29c8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a1ddf680 0x5597a1de1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.761+0000 7f71a19c6640 1 --2- 172.21.15.116:0/2306390107 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f71880a2b80 0x7f71880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.761+0000 7f718f7fe640 1 -- 172.21.15.116:0/2306390107 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f71940cf340 con 0x7f719c106f00 2024-09-12T03:17:50.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.761+0000 7f71a29c8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a1ddf680 0x5597a1de1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46538/0 (socket says 172.21.15.116:46538) 2024-09-12T03:17:50.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.761+0000 7f71a29c8640 1 -- 172.21.15.116:0/4086414626 learned_addr learned my addr 172.21.15.116:0/4086414626 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:50.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.761+0000 7f71a21c7640 1 --2- 172.21.15.116:0/4086414626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597a1dd44a0 0x5597a1dd4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.761+0000 7f71a19c6640 1 --2- 172.21.15.116:0/4086414626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597a1dd4dc0 0x5597a1ddf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.761+0000 7f71a29c8640 1 -- 172.21.15.116:0/4086414626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597a1dd44a0 msgr2=0x5597a1dd4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:50.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.761+0000 7f71a29c8640 1 --2- 172.21.15.116:0/4086414626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597a1dd44a0 0x5597a1dd4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.761+0000 7f71a29c8640 1 -- 172.21.15.116:0/4086414626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597a1dd4dc0 msgr2=0x5597a1ddf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:50.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.761+0000 7f71a29c8640 1 --2- 172.21.15.116:0/4086414626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597a1dd4dc0 0x5597a1ddf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.761+0000 7f71a29c8640 1 -- 172.21.15.116:0/4086414626 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597a1cc6440 con 0x5597a1ddf680 2024-09-12T03:17:50.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.761+0000 7f71a21c7640 1 --2- 172.21.15.116:0/4086414626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597a1dd44a0 0x5597a1dd4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:50.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.761+0000 7f71a19c6640 1 --2- 172.21.15.116:0/2306390107 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f71880a2b80 0x7f71880a5040 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f719c0fdcd0 tx=0x7f7190031710 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:50.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.761+0000 7f71a29c8640 1 --2- 172.21.15.116:0/4086414626 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a1ddf680 0x5597a1de1a70 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f719409b8d0 tx=0x7f719401eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a48d3bfdd8e013 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:50.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.761+0000 7f718cdea640 1 -- 172.21.15.116:0/4086414626 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719409bbd0 con 0x5597a1ddf680 2024-09-12T03:17:50.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.761+0000 7f718cdea640 1 -- 172.21.15.116:0/4086414626 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f719409ff80 con 0x5597a1ddf680 2024-09-12T03:17:50.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.761+0000 7f718cdea640 1 -- 172.21.15.116:0/4086414626 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71940052f0 con 0x5597a1ddf680 2024-09-12T03:17:50.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.762+0000 7f71a6804740 1 -- 172.21.15.116:0/4086414626 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a1ddf680 msgr2=0x5597a1de1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.762+0000 7f71a6804740 1 --2- 172.21.15.116:0/4086414626 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a1ddf680 0x5597a1de1a70 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f719409b8d0 tx=0x7f719401eb40 comp rx=0 tx=0).stop 2024-09-12T03:17:50.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.762+0000 7f71a6804740 1 -- 172.21.15.116:0/4086414626 shutdown_connections 2024-09-12T03:17:50.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.762+0000 7f71a6804740 1 --2- 172.21.15.116:0/4086414626 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a1ddf680 0x5597a1de1a70 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.762+0000 7f71a6804740 1 --2- 172.21.15.116:0/4086414626 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597a1dd4dc0 0x5597a1ddf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.762+0000 7f71a6804740 1 --2- 172.21.15.116:0/4086414626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597a1dd44a0 0x5597a1dd4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.762+0000 7f71a6804740 1 -- 172.21.15.116:0/4086414626 >> 172.21.15.116:0/4086414626 conn(0x5597a1dd0a30 msgr2=0x5597a1dd0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:50.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.762+0000 7f71a6804740 1 -- 172.21.15.116:0/4086414626 shutdown_connections 2024-09-12T03:17:50.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.762+0000 7f71a6804740 1 -- 172.21.15.116:0/4086414626 wait complete. 2024-09-12T03:17:50.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.763+0000 7f71a6804740 1 -- start start 2024-09-12T03:17:50.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.763+0000 7f71a6804740 1 Processor -- start 2024-09-12T03:17:50.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.763+0000 7f71a6804740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597a1dd44a0 0x5597a1ddf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.763+0000 7f71a6804740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597a1dd4dc0 0x5597a1ddc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.763+0000 7f71a6804740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a1ddf680 0x5597a1ddc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.763+0000 7f71a6804740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5597a1de2250 con 0x5597a1ddf680 2024-09-12T03:17:50.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.763+0000 7f71a6804740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5597a1cf5050 con 0x5597a1dd4dc0 2024-09-12T03:17:50.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.763+0000 7f71a6804740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5597a1de23d0 con 0x5597a1dd44a0 2024-09-12T03:17:50.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.763+0000 7f71a6804740 10 client.0 _ll_register_callbacks cb 0x5597a1d35160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:50.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.763+0000 7f71a29c8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a1ddf680 0x5597a1ddc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.763+0000 7f71a29c8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a1ddf680 0x5597a1ddc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46550/0 (socket says 172.21.15.116:46550) 2024-09-12T03:17:50.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.763+0000 7f71a29c8640 1 -- 172.21.15.116:0/1743732999 learned_addr learned my addr 172.21.15.116:0/1743732999 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:50.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.763+0000 7f71a21c7640 1 --2- 172.21.15.116:0/1743732999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597a1dd44a0 0x5597a1ddf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.763+0000 7f71a29c8640 1 -- 172.21.15.116:0/1743732999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597a1dd44a0 msgr2=0x5597a1ddf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:50.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.763+0000 7f71a29c8640 1 --2- 172.21.15.116:0/1743732999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597a1dd44a0 0x5597a1ddf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.763+0000 7f71a29c8640 1 -- 172.21.15.116:0/1743732999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597a1dd4dc0 msgr2=0x5597a1ddc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:50.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.763+0000 7f71a29c8640 1 --2- 172.21.15.116:0/1743732999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597a1dd4dc0 0x5597a1ddc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.763+0000 7f71a29c8640 1 -- 172.21.15.116:0/1743732999 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597a1aab3c0 con 0x5597a1ddf680 2024-09-12T03:17:50.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.763+0000 7f71a29c8640 1 --2- 172.21.15.116:0/1743732999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a1ddf680 0x5597a1ddc8e0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f719400efd0 tx=0x7f7194091120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e679ba8469a57659 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:50.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.763+0000 7f7185ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:50.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.764+0000 7f7185ffb640 1 -- 172.21.15.116:0/1743732999 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7194099c70 con 0x5597a1ddf680 2024-09-12T03:17:50.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.764+0000 7f7185ffb640 1 -- 172.21.15.116:0/1743732999 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f719401ebb0 con 0x5597a1ddf680 2024-09-12T03:17:50.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.764+0000 7f71a6804740 1 -- 172.21.15.116:0/1743732999 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f719409ff80 con 0x5597a1ddf680 2024-09-12T03:17:50.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.764+0000 7f7185ffb640 1 -- 172.21.15.116:0/1743732999 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f71940052d0 con 0x5597a1ddf680 2024-09-12T03:17:50.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.764+0000 7f71a6804740 10 client.19488 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:50.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.764+0000 7f71a6804740 1 -- 172.21.15.116:0/1743732999 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5597a1dedc70 con 0x5597a1ddf680 2024-09-12T03:17:50.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.764+0000 7f7185ffb640 1 -- 172.21.15.116:0/1743732999 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f719409ff80 con 0x5597a1ddf680 2024-09-12T03:17:50.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.764+0000 7f7185ffb640 1 -- 172.21.15.116:0/1743732999 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f719400f040 con 0x5597a1ddf680 2024-09-12T03:17:50.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.764+0000 7f71a6804740 10 client.19488 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:50.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.764+0000 7f71a6804740 10 client.19488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:17:50.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.764+0000 7f71a6804740 20 client.19488 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:50.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.764+0000 7f71a6804740 1 --2- 172.21.15.116:0/1743732999 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5597a1deedf0 0x5597a1df11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.764+0000 7f71a6804740 4 client.19488 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:17:50.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.764+0000 7f71a6804740 1 -- 172.21.15.116:0/1743732999 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597a1d47850 con 0x5597a1deedf0 2024-09-12T03:17:50.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.764+0000 7f71a21c7640 1 --2- 172.21.15.116:0/1743732999 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5597a1deedf0 0x5597a1df11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.765+0000 7f71a21c7640 1 --2- 172.21.15.116:0/1743732999 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5597a1deedf0 0x5597a1df11e0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d62034c26068000 server_cookie=3acdb86d44d817c5 in_seq=0 out_seq=0 2024-09-12T03:17:50.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.765+0000 7f7185ffb640 10 client.19488 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:50.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.765+0000 7f7185ffb640 1 -- 172.21.15.116:0/1743732999 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5597a1d47850 con 0x5597a1deedf0 2024-09-12T03:17:50.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.765+0000 7f7185ffb640 10 client.19488 handle_command_reply: tid=0 2024-09-12T03:17:50.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.765+0000 7f71a6804740 1 client.19488 shutdown 2024-09-12T03:17:50.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.765+0000 7f71a6804740 1 -- 172.21.15.116:0/1743732999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a1ddf680 msgr2=0x5597a1ddc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.765+0000 7f71a6804740 1 --2- 172.21.15.116:0/1743732999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a1ddf680 0x5597a1ddc8e0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f719400efd0 tx=0x7f7194091120 comp rx=0 tx=0).stop 2024-09-12T03:17:50.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.766+0000 7f71a6804740 1 -- 172.21.15.116:0/1743732999 shutdown_connections 2024-09-12T03:17:50.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.766+0000 7f71a6804740 1 --2- 172.21.15.116:0/1743732999 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5597a1deedf0 0x5597a1df11e0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.766+0000 7f71a6804740 1 --2- 172.21.15.116:0/1743732999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a1ddf680 0x5597a1ddc8e0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.766+0000 7f71a6804740 1 --2- 172.21.15.116:0/1743732999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597a1dd4dc0 0x5597a1ddc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.766+0000 7f71a6804740 1 --2- 172.21.15.116:0/1743732999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597a1dd44a0 0x5597a1ddf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.766+0000 7f7185ffb640 0 client.19488 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:50.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.766+0000 7f71a6804740 1 -- 172.21.15.116:0/1743732999 >> 172.21.15.116:0/1743732999 conn(0x5597a1dd0a30 msgr2=0x5597a1dd6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:50.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.766+0000 7f71a6804740 1 -- 172.21.15.116:0/1743732999 shutdown_connections 2024-09-12T03:17:50.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.766+0000 7f71a6804740 1 -- 172.21.15.116:0/1743732999 wait complete. 2024-09-12T03:17:50.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.766+0000 7f71a6804740 20 client.19488 trim_cache size 0 max 16384 2024-09-12T03:17:50.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.782+0000 7f71a6804740 1 Processor -- start 2024-09-12T03:17:50.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.782+0000 7f71a6804740 1 -- start start 2024-09-12T03:17:50.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.782+0000 7f71a6804740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597a1dd44a0 0x5597a1ddabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.782+0000 7f71a6804740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597a1dd4dc0 0x5597a1dde260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.782+0000 7f71a6804740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a1ddf680 0x5597a1dd9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.782+0000 7f71a6804740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f719c1afe30 con 0x5597a1ddf680 2024-09-12T03:17:50.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.782+0000 7f71a6804740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f719c1ad700 con 0x5597a1dd44a0 2024-09-12T03:17:50.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.782+0000 7f71a6804740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f719c1ab570 con 0x5597a1dd4dc0 2024-09-12T03:17:50.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.783+0000 7f71a29c8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a1ddf680 0x5597a1dd9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.783+0000 7f71a29c8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a1ddf680 0x5597a1dd9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46560/0 (socket says 172.21.15.116:46560) 2024-09-12T03:17:50.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.783+0000 7f71a29c8640 1 -- 172.21.15.116:0/2094770078 learned_addr learned my addr 172.21.15.116:0/2094770078 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:50.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.783+0000 7f71a21c7640 1 --2- 172.21.15.116:0/2094770078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597a1dd44a0 0x5597a1ddabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.783+0000 7f71a29c8640 1 -- 172.21.15.116:0/2094770078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597a1dd4dc0 msgr2=0x5597a1dde260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:50.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.783+0000 7f71a29c8640 1 --2- 172.21.15.116:0/2094770078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597a1dd4dc0 0x5597a1dde260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.783+0000 7f71a29c8640 1 -- 172.21.15.116:0/2094770078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597a1dd44a0 msgr2=0x5597a1ddabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:50.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.783+0000 7f71a29c8640 1 --2- 172.21.15.116:0/2094770078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597a1dd44a0 0x5597a1ddabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.783+0000 7f71a29c8640 1 -- 172.21.15.116:0/2094770078 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597a1dda840 con 0x5597a1ddf680 2024-09-12T03:17:50.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.783+0000 7f71a29c8640 1 --2- 172.21.15.116:0/2094770078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a1ddf680 0x5597a1dd9550 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f71940985a0 tx=0x7f71940ccec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3e20014059233a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:50.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.783+0000 7f7187fff640 1 -- 172.21.15.116:0/2094770078 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719401f070 con 0x5597a1ddf680 2024-09-12T03:17:50.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.783+0000 7f7187fff640 1 -- 172.21.15.116:0/2094770078 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f719409ff80 con 0x5597a1ddf680 2024-09-12T03:17:50.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.783+0000 7f7187fff640 1 -- 172.21.15.116:0/2094770078 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719409bbd0 con 0x5597a1ddf680 2024-09-12T03:17:50.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.784+0000 7f71a6804740 1 -- 172.21.15.116:0/2094770078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a1ddf680 msgr2=0x5597a1dd9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.784+0000 7f71a6804740 1 --2- 172.21.15.116:0/2094770078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a1ddf680 0x5597a1dd9550 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f71940985a0 tx=0x7f71940ccec0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.784+0000 7f71a6804740 1 -- 172.21.15.116:0/2094770078 shutdown_connections 2024-09-12T03:17:50.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.784+0000 7f71a6804740 1 --2- 172.21.15.116:0/2094770078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a1ddf680 0x5597a1dd9550 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.784+0000 7f71a6804740 1 --2- 172.21.15.116:0/2094770078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597a1dd4dc0 0x5597a1dde260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.784+0000 7f71a6804740 1 --2- 172.21.15.116:0/2094770078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597a1dd44a0 0x5597a1ddabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.784+0000 7f71a6804740 1 -- 172.21.15.116:0/2094770078 >> 172.21.15.116:0/2094770078 conn(0x5597a1dd0a30 msgr2=0x5597a1dddb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:50.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.784+0000 7f71a6804740 1 -- 172.21.15.116:0/2094770078 shutdown_connections 2024-09-12T03:17:50.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.784+0000 7f71a6804740 1 -- 172.21.15.116:0/2094770078 wait complete. 2024-09-12T03:17:50.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.784+0000 7f71a6804740 1 -- start start 2024-09-12T03:17:50.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.784+0000 7f71a6804740 1 Processor -- start 2024-09-12T03:17:50.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.784+0000 7f71a6804740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a1dd44a0 0x5597a1df2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.784+0000 7f71a6804740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597a1dd4dc0 0x5597a1df2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.784+0000 7f71a6804740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597a1ddf680 0x5597a1e60f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.784+0000 7f71a6804740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f719c1abbc0 con 0x5597a1dd44a0 2024-09-12T03:17:50.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.784+0000 7f71a6804740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f719c1ad030 con 0x5597a1ddf680 2024-09-12T03:17:50.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.784+0000 7f71a6804740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f719c1ac290 con 0x5597a1dd4dc0 2024-09-12T03:17:50.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.784+0000 7f71a6804740 10 client.0 _ll_register_callbacks cb 0x7f719409b9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:50.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.785+0000 7f71a21c7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a1dd44a0 0x5597a1df2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.812 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:17:50.812 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:17:50.812 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": { 2024-09-12T03:17:50.812 INFO:teuthology.orchestra.run.smithi116.stdout: "67b4cd14-d0e0-41e5-9111-a0b36ef7c0e1": { 2024-09-12T03:17:50.813 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/", 2024-09-12T03:17:50.813 INFO:teuthology.orchestra.run.smithi116.stdout: "tag": "67b4cd14-d0e0-41e5-9111-a0b36ef7c0e1", 2024-09-12T03:17:50.813 INFO:teuthology.orchestra.run.smithi116.stdout: "options": "recursive,force" 2024-09-12T03:17:50.813 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:17:50.813 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:17:50.813 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:17:50.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.785+0000 7f71a21c7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a1dd44a0 0x5597a1df2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46568/0 (socket says 172.21.15.116:46568) 2024-09-12T03:17:50.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.785+0000 7f71a21c7640 1 -- 172.21.15.116:0/1871387105 learned_addr learned my addr 172.21.15.116:0/1871387105 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:17:50.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.785+0000 7f71a29c8640 1 --2- 172.21.15.116:0/1871387105 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597a1ddf680 0x5597a1e60f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.785+0000 7f71a19c6640 1 --2- 172.21.15.116:0/1871387105 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597a1dd4dc0 0x5597a1df2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.785+0000 7f71a21c7640 1 -- 172.21.15.116:0/1871387105 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597a1dd4dc0 msgr2=0x5597a1df2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:50.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.785+0000 7f71a21c7640 1 --2- 172.21.15.116:0/1871387105 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597a1dd4dc0 0x5597a1df2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.785+0000 7f71a21c7640 1 -- 172.21.15.116:0/1871387105 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597a1ddf680 msgr2=0x5597a1e60f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:50.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.785+0000 7f71a21c7640 1 --2- 172.21.15.116:0/1871387105 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597a1ddf680 0x5597a1e60f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.785+0000 7f71a21c7640 1 -- 172.21.15.116:0/1871387105 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597a1e61480 con 0x5597a1dd44a0 2024-09-12T03:17:50.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.785+0000 7f71a21c7640 1 --2- 172.21.15.116:0/1871387105 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a1dd44a0 0x5597a1df2350 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f7198004810 tx=0x7f719800ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9acdf3b023779e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:50.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.785+0000 7f7185ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:17:50.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.785+0000 7f7185ffb640 1 -- 172.21.15.116:0/1871387105 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719801f070 con 0x5597a1dd44a0 2024-09-12T03:17:50.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.785+0000 7f7185ffb640 1 -- 172.21.15.116:0/1871387105 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f7198014b90 con 0x5597a1dd44a0 2024-09-12T03:17:50.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.785+0000 7f71a6804740 1 -- 172.21.15.116:0/1871387105 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f719409ff80 con 0x5597a1dd44a0 2024-09-12T03:17:50.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.785+0000 7f7185ffb640 1 -- 172.21.15.116:0/1871387105 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7198014db0 con 0x5597a1dd44a0 2024-09-12T03:17:50.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.785+0000 7f71a6804740 10 client.19494 fetch_fsmap learned FSMap version 25 2024-09-12T03:17:50.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.785+0000 7f71a6804740 1 -- 172.21.15.116:0/1871387105 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5597a1e62650 con 0x5597a1dd44a0 2024-09-12T03:17:50.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.785+0000 7f7185ffb640 1 -- 172.21.15.116:0/1871387105 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f719409ff80 con 0x5597a1dd44a0 2024-09-12T03:17:50.814 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.786+0000 7f7185ffb640 1 -- 172.21.15.116:0/1871387105 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f719802e510 con 0x5597a1dd44a0 2024-09-12T03:17:50.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.786+0000 7f71a6804740 10 client.19494 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:17:50.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.786+0000 7f71a6804740 10 client.19494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:17:50.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.786+0000 7f71a6804740 20 client.19494 populate_metadata read hostname 'smithi116' 2024-09-12T03:17:50.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.786+0000 7f71a6804740 1 --2- 172.21.15.116:0/1871387105 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5597a1e62da0 0x5597a1e651c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.786+0000 7f71a6804740 4 client.19494 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:17:50.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.786+0000 7f71a6804740 1 -- 172.21.15.116:0/1871387105 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5597a1d41d60 con 0x5597a1e62da0 2024-09-12T03:17:50.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.786+0000 7f71a19c6640 1 --2- 172.21.15.116:0/1871387105 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5597a1e62da0 0x5597a1e651c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.786+0000 7f71a19c6640 1 --2- 172.21.15.116:0/1871387105 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5597a1e62da0 0x5597a1e651c0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=def7d40ac5286161 server_cookie=43b7bf811a1daa59 in_seq=0 out_seq=0 2024-09-12T03:17:50.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.786+0000 7f7185ffb640 10 client.19494 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:50.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.787+0000 7f7185ffb640 1 -- 172.21.15.116:0/1871387105 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5597a1d41d60 con 0x5597a1e62da0 2024-09-12T03:17:50.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.787+0000 7f7185ffb640 10 client.19494 handle_command_reply: tid=0 2024-09-12T03:17:50.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.787+0000 7f71a6804740 1 client.19494 shutdown 2024-09-12T03:17:50.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.787+0000 7f71a6804740 1 -- 172.21.15.116:0/1871387105 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a1dd44a0 msgr2=0x5597a1df2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.787+0000 7f71a6804740 1 --2- 172.21.15.116:0/1871387105 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a1dd44a0 0x5597a1df2350 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f7198004810 tx=0x7f719800ce90 comp rx=0 tx=0).stop 2024-09-12T03:17:50.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.787+0000 7f71a6804740 1 -- 172.21.15.116:0/1871387105 shutdown_connections 2024-09-12T03:17:50.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.787+0000 7f71a6804740 1 --2- 172.21.15.116:0/1871387105 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5597a1e62da0 0x5597a1e651c0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.787+0000 7f7185ffb640 0 client.19494 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:17:50.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.787+0000 7f71a6804740 1 --2- 172.21.15.116:0/1871387105 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5597a1ddf680 0x5597a1e60f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.787+0000 7f71a6804740 1 --2- 172.21.15.116:0/1871387105 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597a1dd4dc0 0x5597a1df2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.787+0000 7f71a6804740 1 --2- 172.21.15.116:0/1871387105 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a1dd44a0 0x5597a1df2350 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.787+0000 7f71a6804740 1 -- 172.21.15.116:0/1871387105 >> 172.21.15.116:0/1871387105 conn(0x5597a1dd0a30 msgr2=0x5597a1dde370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:50.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.787+0000 7f71a6804740 1 -- 172.21.15.116:0/1871387105 shutdown_connections 2024-09-12T03:17:50.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.787+0000 7f71a6804740 1 -- 172.21.15.116:0/1871387105 wait complete. 2024-09-12T03:17:50.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.787+0000 7f71a6804740 20 client.19494 trim_cache size 0 max 16384 2024-09-12T03:17:50.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.787+0000 7f71a442b640 1 -- 172.21.15.116:0/2306390107 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f71880a2b80 msgr2=0x7f71880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.787+0000 7f71a442b640 1 --2- 172.21.15.116:0/2306390107 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f71880a2b80 0x7f71880a5040 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f719c0fdcd0 tx=0x7f7190031710 comp rx=0 tx=0).stop 2024-09-12T03:17:50.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.787+0000 7f71a442b640 1 -- 172.21.15.116:0/2306390107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f719c106f00 msgr2=0x7f719c19cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.787+0000 7f71a442b640 1 --2- 172.21.15.116:0/2306390107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f719c106f00 0x7f719c19cdb0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f719400eaa0 tx=0x7f719400ef60 comp rx=0 tx=0).stop 2024-09-12T03:17:50.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.787+0000 7f71a442b640 1 -- 172.21.15.116:0/2306390107 shutdown_connections 2024-09-12T03:17:50.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.787+0000 7f71a442b640 1 --2- 172.21.15.116:0/2306390107 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f71880a2b80 0x7f71880a5040 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.787+0000 7f71a442b640 1 --2- 172.21.15.116:0/2306390107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f719c106f00 0x7f719c19cdb0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.787+0000 7f71a442b640 1 --2- 172.21.15.116:0/2306390107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f719c102780 0x7f719c198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.787+0000 7f71a442b640 1 --2- 172.21.15.116:0/2306390107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f719c101dd0 0x7f719c1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.787+0000 7f71a442b640 1 -- 172.21.15.116:0/2306390107 >> 172.21.15.116:0/2306390107 conn(0x7f719c0fd8b0 msgr2=0x7f719c0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:50.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.788+0000 7f71a442b640 1 -- 172.21.15.116:0/2306390107 shutdown_connections 2024-09-12T03:17:50.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:17:50.788+0000 7f71a442b640 1 -- 172.21.15.116:0/2306390107 wait complete. 2024-09-12T03:17:50.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:50 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:17:50.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:50 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:17:50.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:50 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:17:50.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:50 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:17:51.547 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:51 smithi116 ceph-mon[31410]: pgmap v784: 97 pgs: 97 active+clean; 7.0 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:17:51.578 INFO:tasks.cephfs.filesystem:scrub status for tag:67b4cd14-d0e0-41e5-9111-a0b36ef7c0e1 - {'path': '/', 'tag': '67b4cd14-d0e0-41e5-9111-a0b36ef7c0e1', 'options': 'recursive,force'} 2024-09-12T03:17:51.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:51 smithi177 ceph-mon[35305]: pgmap v784: 97 pgs: 97 active+clean; 7.0 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:17:51.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:51 smithi183 ceph-mon[35535]: pgmap v784: 97 pgs: 97 active+clean; 7.0 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:17:53.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:53 smithi116 ceph-mon[31410]: pgmap v785: 97 pgs: 97 active+clean; 7.0 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:17:53.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:53 smithi177 ceph-mon[35305]: pgmap v785: 97 pgs: 97 active+clean; 7.0 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:17:53.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:53 smithi183 ceph-mon[35535]: pgmap v785: 97 pgs: 97 active+clean; 7.0 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:17:54.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:54 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:17:54.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:54 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:17:54.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:54 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:17:55.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:55 smithi177 ceph-mon[35305]: pgmap v786: 97 pgs: 97 active+clean; 7.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:17:55.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:55 smithi116 ceph-mon[31410]: pgmap v786: 97 pgs: 97 active+clean; 7.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:17:55.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:55 smithi183 ceph-mon[35535]: pgmap v786: 97 pgs: 97 active+clean; 7.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:17:57.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:57 smithi177 ceph-mon[35305]: pgmap v787: 97 pgs: 97 active+clean; 7.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:17:57.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:57 smithi116 ceph-mon[31410]: pgmap v787: 97 pgs: 97 active+clean; 7.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:17:57.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:57 smithi183 ceph-mon[35535]: pgmap v787: 97 pgs: 97 active+clean; 7.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:17:59.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:17:59 smithi177 ceph-mon[35305]: pgmap v788: 97 pgs: 97 active+clean; 7.1 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-12T03:17:59.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:17:59 smithi116 ceph-mon[31410]: pgmap v788: 97 pgs: 97 active+clean; 7.1 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-12T03:17:59.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:17:59 smithi183 ceph-mon[35535]: pgmap v788: 97 pgs: 97 active+clean; 7.1 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-12T03:18:01.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:01 smithi177 ceph-mon[35305]: pgmap v789: 97 pgs: 97 active+clean; 7.1 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:18:01.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:01 smithi116 ceph-mon[31410]: pgmap v789: 97 pgs: 97 active+clean; 7.1 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:18:01.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:01 smithi183 ceph-mon[35535]: pgmap v789: 97 pgs: 97 active+clean; 7.1 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:18:03.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:03 smithi177 ceph-mon[35305]: pgmap v790: 97 pgs: 97 active+clean; 7.1 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:18:03.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:03 smithi116 ceph-mon[31410]: pgmap v790: 97 pgs: 97 active+clean; 7.1 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:18:03.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:03 smithi183 ceph-mon[35535]: pgmap v790: 97 pgs: 97 active+clean; 7.1 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-12T03:18:05.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:05 smithi177 ceph-mon[35305]: pgmap v791: 97 pgs: 97 active+clean; 7.1 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:18:05.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:05 smithi116 ceph-mon[31410]: pgmap v791: 97 pgs: 97 active+clean; 7.1 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:18:05.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:05 smithi183 ceph-mon[35535]: pgmap v791: 97 pgs: 97 active+clean; 7.1 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:18:07.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:07 smithi177 ceph-mon[35305]: pgmap v792: 97 pgs: 97 active+clean; 7.1 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:18:07.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:07 smithi116 ceph-mon[31410]: pgmap v792: 97 pgs: 97 active+clean; 7.1 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:18:07.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:07 smithi183 ceph-mon[35535]: pgmap v792: 97 pgs: 97 active+clean; 7.1 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:18:09.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:09 smithi177 ceph-mon[35305]: pgmap v793: 97 pgs: 97 active+clean; 7.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:18:09.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:09 smithi116 ceph-mon[31410]: pgmap v793: 97 pgs: 97 active+clean; 7.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:18:09.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:09 smithi183 ceph-mon[35535]: pgmap v793: 97 pgs: 97 active+clean; 7.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:18:11.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:11 smithi177 ceph-mon[35305]: pgmap v794: 97 pgs: 97 active+clean; 7.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:18:11.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:11 smithi116 ceph-mon[31410]: pgmap v794: 97 pgs: 97 active+clean; 7.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:18:11.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:11 smithi183 ceph-mon[35535]: pgmap v794: 97 pgs: 97 active+clean; 7.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:18:13.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:13 smithi177 ceph-mon[35305]: pgmap v795: 97 pgs: 97 active+clean; 7.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:18:13.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:13 smithi116 ceph-mon[31410]: pgmap v795: 97 pgs: 97 active+clean; 7.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:18:13.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:13 smithi183 ceph-mon[35535]: pgmap v795: 97 pgs: 97 active+clean; 7.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:18:15.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:15 smithi177 ceph-mon[35305]: pgmap v796: 97 pgs: 97 active+clean; 7.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:18:15.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:15 smithi116 ceph-mon[31410]: pgmap v796: 97 pgs: 97 active+clean; 7.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:18:15.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:15 smithi183 ceph-mon[35535]: pgmap v796: 97 pgs: 97 active+clean; 7.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:18:17.227 INFO:tasks.workunit.client.0.smithi116.stdout:499/600: open fd = 3 2024-09-12T03:18:17.227 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.227 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.227 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.227 INFO:tasks.workunit.client.0.smithi116.stdout:500/600: open fd = 3 2024-09-12T03:18:17.227 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.227 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.227 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.227 INFO:tasks.workunit.client.0.smithi116.stdout:501/600: open fd = 3 2024-09-12T03:18:17.227 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.227 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.227 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.228 INFO:tasks.workunit.client.0.smithi116.stdout:502/600: open fd = 3 2024-09-12T03:18:17.228 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.228 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.228 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.228 INFO:tasks.workunit.client.0.smithi116.stdout:503/600: open fd = 3 2024-09-12T03:18:17.228 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.228 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.228 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.228 INFO:tasks.workunit.client.0.smithi116.stdout:504/600: open fd = 3 2024-09-12T03:18:17.228 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.228 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.228 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.229 INFO:tasks.workunit.client.0.smithi116.stdout:505/600: open fd = 3 2024-09-12T03:18:17.229 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.229 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.229 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.229 INFO:tasks.workunit.client.0.smithi116.stdout:506/600: open fd = 3 2024-09-12T03:18:17.229 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.229 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.229 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.229 INFO:tasks.workunit.client.0.smithi116.stdout:507/600: open fd = 3 2024-09-12T03:18:17.229 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.229 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.229 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.229 INFO:tasks.workunit.client.0.smithi116.stdout:508/600: open fd = 3 2024-09-12T03:18:17.230 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.230 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.230 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.230 INFO:tasks.workunit.client.0.smithi116.stdout:509/600: open fd = 3 2024-09-12T03:18:17.230 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.230 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.230 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.230 INFO:tasks.workunit.client.0.smithi116.stdout:510/600: open fd = 3 2024-09-12T03:18:17.230 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.230 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.230 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.230 INFO:tasks.workunit.client.0.smithi116.stdout:511/600: open fd = 3 2024-09-12T03:18:17.230 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.230 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.231 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.231 INFO:tasks.workunit.client.0.smithi116.stdout:512/600: open fd = 3 2024-09-12T03:18:17.231 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.231 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.231 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.231 INFO:tasks.workunit.client.0.smithi116.stdout:513/600: open fd = 3 2024-09-12T03:18:17.231 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.231 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.231 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.231 INFO:tasks.workunit.client.0.smithi116.stdout:514/600: open fd = 3 2024-09-12T03:18:17.231 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.231 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.231 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.231 INFO:tasks.workunit.client.0.smithi116.stdout:515/600: open fd = 3 2024-09-12T03:18:17.232 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.232 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.232 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.232 INFO:tasks.workunit.client.0.smithi116.stdout:516/600: open fd = 3 2024-09-12T03:18:17.232 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.232 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.232 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.232 INFO:tasks.workunit.client.0.smithi116.stdout:517/600: open fd = 3 2024-09-12T03:18:17.232 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.232 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.232 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.232 INFO:tasks.workunit.client.0.smithi116.stdout:518/600: open fd = 3 2024-09-12T03:18:17.232 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.232 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.233 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.233 INFO:tasks.workunit.client.0.smithi116.stdout:519/600: open fd = 3 2024-09-12T03:18:17.233 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.233 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.233 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.233 INFO:tasks.workunit.client.0.smithi116.stdout:520/600: open fd = 3 2024-09-12T03:18:17.233 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.233 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.234 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.234 INFO:tasks.workunit.client.0.smithi116.stdout:521/600: open fd = 3 2024-09-12T03:18:17.234 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.234 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.234 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.234 INFO:tasks.workunit.client.0.smithi116.stdout:522/600: open fd = 3 2024-09-12T03:18:17.234 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.235 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.235 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.235 INFO:tasks.workunit.client.0.smithi116.stdout:523/600: open fd = 3 2024-09-12T03:18:17.235 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.235 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.235 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.235 INFO:tasks.workunit.client.0.smithi116.stdout:524/600: open fd = 3 2024-09-12T03:18:17.235 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.235 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.235 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.235 INFO:tasks.workunit.client.0.smithi116.stdout:525/600: open fd = 3 2024-09-12T03:18:17.236 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.236 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.236 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.236 INFO:tasks.workunit.client.0.smithi116.stdout:526/600: open fd = 3 2024-09-12T03:18:17.236 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.236 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.236 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.236 INFO:tasks.workunit.client.0.smithi116.stdout:527/600: open fd = 3 2024-09-12T03:18:17.236 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.236 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.236 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.236 INFO:tasks.workunit.client.0.smithi116.stdout:528/600: open fd = 3 2024-09-12T03:18:17.236 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.237 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.237 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.237 INFO:tasks.workunit.client.0.smithi116.stdout:529/600: open fd = 3 2024-09-12T03:18:17.237 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.237 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.237 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.237 INFO:tasks.workunit.client.0.smithi116.stdout:530/600: open fd = 3 2024-09-12T03:18:17.237 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.237 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.237 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.237 INFO:tasks.workunit.client.0.smithi116.stdout:531/600: open fd = 3 2024-09-12T03:18:17.237 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.237 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.237 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.238 INFO:tasks.workunit.client.0.smithi116.stdout:532/600: open fd = 3 2024-09-12T03:18:17.238 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.238 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.238 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.238 INFO:tasks.workunit.client.0.smithi116.stdout:533/600: open fd = 3 2024-09-12T03:18:17.238 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.238 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.238 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.238 INFO:tasks.workunit.client.0.smithi116.stdout:534/600: open fd = 3 2024-09-12T03:18:17.238 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.238 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.238 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.239 INFO:tasks.workunit.client.0.smithi116.stdout:535/600: open fd = 3 2024-09-12T03:18:17.239 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.239 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.239 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.239 INFO:tasks.workunit.client.0.smithi116.stdout:536/600: open fd = 3 2024-09-12T03:18:17.239 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.239 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.239 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.239 INFO:tasks.workunit.client.0.smithi116.stdout:537/600: open fd = 3 2024-09-12T03:18:17.239 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.239 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.239 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.239 INFO:tasks.workunit.client.0.smithi116.stdout:538/600: open fd = 3 2024-09-12T03:18:17.239 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.240 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.240 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.240 INFO:tasks.workunit.client.0.smithi116.stdout:539/600: open fd = 3 2024-09-12T03:18:17.240 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.240 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.240 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.240 INFO:tasks.workunit.client.0.smithi116.stdout:540/600: open fd = 3 2024-09-12T03:18:17.240 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.240 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.240 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.240 INFO:tasks.workunit.client.0.smithi116.stdout:541/600: open fd = 3 2024-09-12T03:18:17.240 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.240 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.240 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.241 INFO:tasks.workunit.client.0.smithi116.stdout:542/600: open fd = 3 2024-09-12T03:18:17.241 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.241 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.241 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.241 INFO:tasks.workunit.client.0.smithi116.stdout:543/600: open fd = 3 2024-09-12T03:18:17.241 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.241 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.241 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.241 INFO:tasks.workunit.client.0.smithi116.stdout:544/600: open fd = 3 2024-09-12T03:18:17.241 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.241 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.242 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.242 INFO:tasks.workunit.client.0.smithi116.stdout:545/600: open fd = 3 2024-09-12T03:18:17.242 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.242 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.242 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.242 INFO:tasks.workunit.client.0.smithi116.stdout:546/600: open fd = 3 2024-09-12T03:18:17.243 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.243 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.243 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.243 INFO:tasks.workunit.client.0.smithi116.stdout:547/600: open fd = 3 2024-09-12T03:18:17.243 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.243 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.243 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.243 INFO:tasks.workunit.client.0.smithi116.stdout:548/600: open fd = 3 2024-09-12T03:18:17.244 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.244 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.244 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.244 INFO:tasks.workunit.client.0.smithi116.stdout:549/600: open fd = 3 2024-09-12T03:18:17.244 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.244 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.244 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.244 INFO:tasks.workunit.client.0.smithi116.stdout:550/600: open fd = 3 2024-09-12T03:18:17.244 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.244 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.244 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.244 INFO:tasks.workunit.client.0.smithi116.stdout:551/600: open fd = 3 2024-09-12T03:18:17.244 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.245 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.245 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.245 INFO:tasks.workunit.client.0.smithi116.stdout:552/600: open fd = 3 2024-09-12T03:18:17.245 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.245 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.245 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.245 INFO:tasks.workunit.client.0.smithi116.stdout:553/600: open fd = 3 2024-09-12T03:18:17.245 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.245 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.245 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.245 INFO:tasks.workunit.client.0.smithi116.stdout:554/600: open fd = 3 2024-09-12T03:18:17.245 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.246 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.246 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.246 INFO:tasks.workunit.client.0.smithi116.stdout:555/600: open fd = 3 2024-09-12T03:18:17.246 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.246 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.246 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.246 INFO:tasks.workunit.client.0.smithi116.stdout:556/600: open fd = 3 2024-09-12T03:18:17.246 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.246 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.246 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.246 INFO:tasks.workunit.client.0.smithi116.stdout:557/600: open fd = 3 2024-09-12T03:18:17.246 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.246 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.246 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.247 INFO:tasks.workunit.client.0.smithi116.stdout:558/600: open fd = 3 2024-09-12T03:18:17.247 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.247 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.247 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.247 INFO:tasks.workunit.client.0.smithi116.stdout:559/600: open fd = 3 2024-09-12T03:18:17.247 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.247 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.247 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.247 INFO:tasks.workunit.client.0.smithi116.stdout:560/600: open fd = 3 2024-09-12T03:18:17.247 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.247 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:17.247 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:17.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:17 smithi177 ceph-mon[35305]: pgmap v797: 97 pgs: 97 active+clean; 7.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:18:17.755 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:17 smithi183 ceph-mon[35535]: pgmap v797: 97 pgs: 97 active+clean; 7.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:18:17.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:17 smithi116 ceph-mon[31410]: pgmap v797: 97 pgs: 97 active+clean; 7.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:18:19.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:19 smithi177 ceph-mon[35305]: pgmap v798: 97 pgs: 97 active+clean; 7.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:18:19.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:19 smithi116 ceph-mon[31410]: pgmap v798: 97 pgs: 97 active+clean; 7.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:18:19.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:19 smithi183 ceph-mon[35535]: pgmap v798: 97 pgs: 97 active+clean; 7.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:18:21.579 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:18:21.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:21 smithi177 ceph-mon[35305]: pgmap v799: 97 pgs: 97 active+clean; 7.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:18:21.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:21 smithi116 ceph-mon[31410]: pgmap v799: 97 pgs: 97 active+clean; 7.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:18:21.842 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:21.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:21 smithi183 ceph-mon[35535]: pgmap v799: 97 pgs: 97 active+clean; 7.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:18:22.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.356+0000 7f0075400640 1 -- 172.21.15.116:0/1130970051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0070100a80 msgr2=0x7f0070100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:22.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.356+0000 7f0075400640 1 --2- 172.21.15.116:0/1130970051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0070100a80 0x7f0070100ee0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f0064009930 tx=0x7f006402a420 comp rx=0 tx=0).stop 2024-09-12T03:18:22.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.356+0000 7f0075400640 1 -- 172.21.15.116:0/1130970051 shutdown_connections 2024-09-12T03:18:22.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.356+0000 7f0075400640 1 --2- 172.21.15.116:0/1130970051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0070101420 0x7f007010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.356+0000 7f0075400640 1 --2- 172.21.15.116:0/1130970051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0070100a80 0x7f0070100ee0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.356+0000 7f0075400640 1 --2- 172.21.15.116:0/1130970051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0070106a80 0x7f0070106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.356+0000 7f0075400640 1 -- 172.21.15.116:0/1130970051 >> 172.21.15.116:0/1130970051 conn(0x7f00700fc7a0 msgr2=0x7f00700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:22.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.356+0000 7f0075400640 1 -- 172.21.15.116:0/1130970051 shutdown_connections 2024-09-12T03:18:22.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.356+0000 7f0075400640 1 -- 172.21.15.116:0/1130970051 wait complete. 2024-09-12T03:18:22.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.357+0000 7f0075400640 1 Processor -- start 2024-09-12T03:18:22.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.357+0000 7f0075400640 1 -- start start 2024-09-12T03:18:22.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.357+0000 7f0075400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0070100a80 0x7f0070199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:22.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.357+0000 7f0075400640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0070101420 0x7f0070199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:22.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.357+0000 7f0075400640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0070106a80 0x7f00701a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:22.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.357+0000 7f0075400640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0070110f20 con 0x7f0070106a80 2024-09-12T03:18:22.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.357+0000 7f0075400640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0070110da0 con 0x7f0070101420 2024-09-12T03:18:22.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.357+0000 7f0075400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f00701110a0 con 0x7f0070100a80 2024-09-12T03:18:22.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.357+0000 7f006f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0070106a80 0x7f00701a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:22.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.357+0000 7f006f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0070106a80 0x7f00701a44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60390/0 (socket says 172.21.15.116:60390) 2024-09-12T03:18:22.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.357+0000 7f006f7fe640 1 -- 172.21.15.116:0/2577653947 learned_addr learned my addr 172.21.15.116:0/2577653947 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:22.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.357+0000 7f006effd640 1 --2- 172.21.15.116:0/2577653947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0070100a80 0x7f0070199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:22.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.358+0000 7f006e7fc640 1 --2- 172.21.15.116:0/2577653947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0070101420 0x7f0070199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:22.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.358+0000 7f006f7fe640 1 -- 172.21.15.116:0/2577653947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0070100a80 msgr2=0x7f0070199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:22.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.358+0000 7f006f7fe640 1 --2- 172.21.15.116:0/2577653947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0070100a80 0x7f0070199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.358+0000 7f006f7fe640 1 -- 172.21.15.116:0/2577653947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0070101420 msgr2=0x7f0070199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:22.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.358+0000 7f006f7fe640 1 --2- 172.21.15.116:0/2577653947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0070101420 0x7f0070199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.358+0000 7f006f7fe640 1 -- 172.21.15.116:0/2577653947 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00701a49f0 con 0x7f0070106a80 2024-09-12T03:18:22.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.358+0000 7f006effd640 1 --2- 172.21.15.116:0/2577653947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0070100a80 0x7f0070199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:22.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.358+0000 7f006e7fc640 1 --2- 172.21.15.116:0/2577653947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0070101420 0x7f0070199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:18:22.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.358+0000 7f006f7fe640 1 --2- 172.21.15.116:0/2577653947 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0070106a80 0x7f00701a44b0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f006800b970 tx=0x7f006800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:22.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.358+0000 7f0057fff640 1 -- 172.21.15.116:0/2577653947 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0068019070 con 0x7f0070106a80 2024-09-12T03:18:22.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.358+0000 7f0075400640 1 -- 172.21.15.116:0/2577653947 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00701a4bc0 con 0x7f0070106a80 2024-09-12T03:18:22.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.358+0000 7f0057fff640 1 -- 172.21.15.116:0/2577653947 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f006800cb40 con 0x7f0070106a80 2024-09-12T03:18:22.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.358+0000 7f0057fff640 1 -- 172.21.15.116:0/2577653947 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0068007610 con 0x7f0070106a80 2024-09-12T03:18:22.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.358+0000 7f0075400640 1 -- 172.21.15.116:0/2577653947 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00701a50d0 con 0x7f0070106a80 2024-09-12T03:18:22.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.361+0000 7f0057fff640 1 -- 172.21.15.116:0/2577653947 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f006801c070 con 0x7f0070106a80 2024-09-12T03:18:22.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.361+0000 7f00777d9740 1 Processor -- start 2024-09-12T03:18:22.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.361+0000 7f00777d9740 1 -- start start 2024-09-12T03:18:22.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.361+0000 7f00777d9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614800b14a0 0x5614800b1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:22.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.361+0000 7f00777d9740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614800b1dc0 0x5614800bc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:22.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.361+0000 7f00777d9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614800bc680 0x5614800bea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:22.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.361+0000 7f00777d9740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5614800172e0 con 0x5614800b1dc0 2024-09-12T03:18:22.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.361+0000 7f00777d9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56147ffbece0 con 0x5614800b14a0 2024-09-12T03:18:22.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.361+0000 7f00777d9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5614800132a0 con 0x5614800bc680 2024-09-12T03:18:22.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.361+0000 7f006effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614800b1dc0 0x5614800bc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:22.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.361+0000 7f006effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614800b1dc0 0x5614800bc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60402/0 (socket says 172.21.15.116:60402) 2024-09-12T03:18:22.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.361+0000 7f006effd640 1 -- 172.21.15.116:0/835403033 learned_addr learned my addr 172.21.15.116:0/835403033 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:22.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.361+0000 7f006e7fc640 1 --2- 172.21.15.116:0/835403033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614800bc680 0x5614800bea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:22.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.361+0000 7f006e7fc640 1 --2- 172.21.15.116:0/835403033 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614800b14a0 0x5614800b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:22.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.361+0000 7f006effd640 1 -- 172.21.15.116:0/835403033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614800bc680 msgr2=0x5614800bea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:22.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.361+0000 7f006effd640 1 --2- 172.21.15.116:0/835403033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614800bc680 0x5614800bea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.361+0000 7f006effd640 1 -- 172.21.15.116:0/835403033 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614800b14a0 msgr2=0x5614800b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:22.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.361+0000 7f006effd640 1 --2- 172.21.15.116:0/835403033 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614800b14a0 0x5614800b1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.361+0000 7f006effd640 1 -- 172.21.15.116:0/835403033 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56147ffa3440 con 0x5614800b1dc0 2024-09-12T03:18:22.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.361+0000 7f0057fff640 1 --2- 172.21.15.116:0/2577653947 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f00580a2b80 0x7f00580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:22.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.362+0000 7f006e7fc640 1 --2- 172.21.15.116:0/2577653947 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f00580a2b80 0x7f00580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:22.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.362+0000 7f006effd640 1 --2- 172.21.15.116:0/835403033 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614800b1dc0 0x5614800bc140 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f0060007850 tx=0x7f0060007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5350c2bde900e0fb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:22.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.362+0000 7f0057fff640 1 -- 172.21.15.116:0/2577653947 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f00680ce630 con 0x7f0070106a80 2024-09-12T03:18:22.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.362+0000 7f00557fa640 1 -- 172.21.15.116:0/835403033 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006000d950 con 0x5614800b1dc0 2024-09-12T03:18:22.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.362+0000 7f00557fa640 1 -- 172.21.15.116:0/835403033 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0060018070 con 0x5614800b1dc0 2024-09-12T03:18:22.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.362+0000 7f00557fa640 1 -- 172.21.15.116:0/835403033 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00600123e0 con 0x5614800b1dc0 2024-09-12T03:18:22.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.363+0000 7f00777d9740 1 -- 172.21.15.116:0/835403033 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614800b1dc0 msgr2=0x5614800bc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:22.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.363+0000 7f006e7fc640 1 --2- 172.21.15.116:0/2577653947 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f00580a2b80 0x7f00580a5040 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0064031550 tx=0x7f0064005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:22.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.363+0000 7f00777d9740 1 --2- 172.21.15.116:0/835403033 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614800b1dc0 0x5614800bc140 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f0060007850 tx=0x7f0060007d10 comp rx=0 tx=0).stop 2024-09-12T03:18:22.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.363+0000 7f00777d9740 1 -- 172.21.15.116:0/835403033 shutdown_connections 2024-09-12T03:18:22.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.363+0000 7f00777d9740 1 --2- 172.21.15.116:0/835403033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614800bc680 0x5614800bea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.363+0000 7f00777d9740 1 --2- 172.21.15.116:0/835403033 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614800b1dc0 0x5614800bc140 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.363+0000 7f00777d9740 1 --2- 172.21.15.116:0/835403033 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614800b14a0 0x5614800b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.363+0000 7f00777d9740 1 -- 172.21.15.116:0/835403033 >> 172.21.15.116:0/835403033 conn(0x5614800ada30 msgr2=0x5614800ade10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:22.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.363+0000 7f00777d9740 1 -- 172.21.15.116:0/835403033 shutdown_connections 2024-09-12T03:18:22.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.363+0000 7f00777d9740 1 -- 172.21.15.116:0/835403033 wait complete. 2024-09-12T03:18:22.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.363+0000 7f00777d9740 1 -- start start 2024-09-12T03:18:22.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.363+0000 7f00777d9740 1 Processor -- start 2024-09-12T03:18:22.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.363+0000 7f00777d9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614800b14a0 0x5614800bbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:22.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.363+0000 7f00777d9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614800b1dc0 0x5614800b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:22.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.363+0000 7f00777d9740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614800bc680 0x5614800b9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:22.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.363+0000 7f00777d9740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5614800bf250 con 0x5614800bc680 2024-09-12T03:18:22.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.363+0000 7f00777d9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56147ffd2050 con 0x5614800b14a0 2024-09-12T03:18:22.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.363+0000 7f00777d9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5614800bf3d0 con 0x5614800b1dc0 2024-09-12T03:18:22.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.363+0000 7f00777d9740 10 client.0 _ll_register_callbacks cb 0x561480012160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:18:22.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.364+0000 7f006f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614800bc680 0x5614800b9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:22.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.364+0000 7f006f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614800bc680 0x5614800b9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60412/0 (socket says 172.21.15.116:60412) 2024-09-12T03:18:22.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.364+0000 7f006f7fe640 1 -- 172.21.15.116:0/2436634527 learned_addr learned my addr 172.21.15.116:0/2436634527 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:22.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.364+0000 7f006effd640 1 --2- 172.21.15.116:0/2436634527 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614800b14a0 0x5614800bbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:22.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.364+0000 7f006e7fc640 1 --2- 172.21.15.116:0/2436634527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614800b1dc0 0x5614800b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:22.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.364+0000 7f006f7fe640 1 -- 172.21.15.116:0/2436634527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614800b1dc0 msgr2=0x5614800b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:22.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.364+0000 7f006f7fe640 1 --2- 172.21.15.116:0/2436634527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614800b1dc0 0x5614800b9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.364+0000 7f006f7fe640 1 -- 172.21.15.116:0/2436634527 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614800b14a0 msgr2=0x5614800bbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:22.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.364+0000 7f006f7fe640 1 --2- 172.21.15.116:0/2436634527 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614800b14a0 0x5614800bbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.364+0000 7f006f7fe640 1 -- 172.21.15.116:0/2436634527 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56147fd883c0 con 0x5614800bc680 2024-09-12T03:18:22.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.364+0000 7f006effd640 1 --2- 172.21.15.116:0/2436634527 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614800b14a0 0x5614800bbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:22.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.364+0000 7f006f7fe640 1 --2- 172.21.15.116:0/2436634527 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614800bc680 0x5614800b9770 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f00680a3600 tx=0x7f006809aec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b28d3503df1b178 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:22.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.364+0000 7f0052ff5640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:18:22.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.364+0000 7f0052ff5640 1 -- 172.21.15.116:0/2436634527 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00680a36a0 con 0x5614800bc680 2024-09-12T03:18:22.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.364+0000 7f0052ff5640 1 -- 172.21.15.116:0/2436634527 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f00680a3840 con 0x5614800bc680 2024-09-12T03:18:22.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.364+0000 7f00777d9740 1 -- 172.21.15.116:0/2436634527 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0060018070 con 0x5614800bc680 2024-09-12T03:18:22.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.364+0000 7f0052ff5640 1 -- 172.21.15.116:0/2436634527 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f006809a310 con 0x5614800bc680 2024-09-12T03:18:22.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.364+0000 7f00777d9740 10 client.19506 fetch_fsmap learned FSMap version 25 2024-09-12T03:18:22.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.364+0000 7f00777d9740 1 -- 172.21.15.116:0/2436634527 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5614800cac50 con 0x5614800bc680 2024-09-12T03:18:22.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.364+0000 7f0052ff5640 1 -- 172.21.15.116:0/2436634527 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0060018070 con 0x5614800bc680 2024-09-12T03:18:22.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.365+0000 7f0052ff5640 1 -- 172.21.15.116:0/2436634527 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f00680ce0f0 con 0x5614800bc680 2024-09-12T03:18:22.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.365+0000 7f00777d9740 10 client.19506 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:18:22.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.365+0000 7f00777d9740 10 client.19506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:18:22.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.365+0000 7f00777d9740 20 client.19506 populate_metadata read hostname 'smithi116' 2024-09-12T03:18:22.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.365+0000 7f00777d9740 1 --2- 172.21.15.116:0/2436634527 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5614800cc110 0x5614800ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:22.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.365+0000 7f00777d9740 4 client.19506 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:18:22.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.365+0000 7f00777d9740 1 -- 172.21.15.116:0/2436634527 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561480024850 con 0x5614800cc110 2024-09-12T03:18:22.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.365+0000 7f006effd640 1 --2- 172.21.15.116:0/2436634527 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5614800cc110 0x5614800ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:22.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.365+0000 7f006effd640 1 --2- 172.21.15.116:0/2436634527 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5614800cc110 0x5614800ce580 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=588234c5b6a5f49f server_cookie=d53d55cc86a5ddd5 in_seq=0 out_seq=0 2024-09-12T03:18:22.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.365+0000 7f0052ff5640 10 client.19506 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:18:22.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.366+0000 7f0052ff5640 1 -- 172.21.15.116:0/2436634527 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561480024850 con 0x5614800cc110 2024-09-12T03:18:22.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.366+0000 7f0052ff5640 10 client.19506 handle_command_reply: tid=0 2024-09-12T03:18:22.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.366+0000 7f00777d9740 1 client.19506 shutdown 2024-09-12T03:18:22.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.366+0000 7f00777d9740 1 -- 172.21.15.116:0/2436634527 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614800bc680 msgr2=0x5614800b9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:22.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.366+0000 7f00777d9740 1 --2- 172.21.15.116:0/2436634527 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614800bc680 0x5614800b9770 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f00680a3600 tx=0x7f006809aec0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.366+0000 7f00777d9740 1 -- 172.21.15.116:0/2436634527 shutdown_connections 2024-09-12T03:18:22.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.366+0000 7f00777d9740 1 --2- 172.21.15.116:0/2436634527 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5614800cc110 0x5614800ce580 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.366+0000 7f00777d9740 1 --2- 172.21.15.116:0/2436634527 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614800bc680 0x5614800b9770 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.366+0000 7f00777d9740 1 --2- 172.21.15.116:0/2436634527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614800b1dc0 0x5614800b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.366+0000 7f00777d9740 1 --2- 172.21.15.116:0/2436634527 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614800b14a0 0x5614800bbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.366+0000 7f0052ff5640 0 client.19506 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:18:22.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.366+0000 7f00777d9740 1 -- 172.21.15.116:0/2436634527 >> 172.21.15.116:0/2436634527 conn(0x5614800ada30 msgr2=0x5614800b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:22.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.366+0000 7f00777d9740 1 -- 172.21.15.116:0/2436634527 shutdown_connections 2024-09-12T03:18:22.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.366+0000 7f00777d9740 1 -- 172.21.15.116:0/2436634527 wait complete. 2024-09-12T03:18:22.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.366+0000 7f00777d9740 20 client.19506 trim_cache size 0 max 16384 2024-09-12T03:18:22.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.383+0000 7f00777d9740 1 Processor -- start 2024-09-12T03:18:22.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.383+0000 7f00777d9740 1 -- start start 2024-09-12T03:18:22.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.383+0000 7f00777d9740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614800b14a0 0x5614800c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:22.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.383+0000 7f00777d9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614800b1dc0 0x5614800aca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:22.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.383+0000 7f00777d9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614800bc680 0x5614800d3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:22.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.383+0000 7f00777d9740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f00701aff60 con 0x5614800b14a0 2024-09-12T03:18:22.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.383+0000 7f00777d9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f00701ad830 con 0x5614800b1dc0 2024-09-12T03:18:22.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.383+0000 7f00777d9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f00701ab680 con 0x5614800bc680 2024-09-12T03:18:22.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.383+0000 7f006effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614800b14a0 0x5614800c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:22.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.383+0000 7f006effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614800b14a0 0x5614800c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60422/0 (socket says 172.21.15.116:60422) 2024-09-12T03:18:22.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.383+0000 7f006effd640 1 -- 172.21.15.116:0/2720653224 learned_addr learned my addr 172.21.15.116:0/2720653224 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:22.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.384+0000 7f006effd640 1 -- 172.21.15.116:0/2720653224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614800bc680 msgr2=0x5614800d3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:22.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.384+0000 7f006effd640 1 --2- 172.21.15.116:0/2720653224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614800bc680 0x5614800d3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.384+0000 7f006effd640 1 -- 172.21.15.116:0/2720653224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614800b1dc0 msgr2=0x5614800aca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:22.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.384+0000 7f006effd640 1 --2- 172.21.15.116:0/2720653224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614800b1dc0 0x5614800aca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.384+0000 7f006effd640 1 -- 172.21.15.116:0/2720653224 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614800ad860 con 0x5614800b14a0 2024-09-12T03:18:22.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.384+0000 7f006effd640 1 --2- 172.21.15.116:0/2720653224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614800b14a0 0x5614800c84e0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f00600077a0 tx=0x7f006000f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=880941bd8f53505c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:22.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.384+0000 7f0054ff9640 1 -- 172.21.15.116:0/2720653224 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006002b020 con 0x5614800b14a0 2024-09-12T03:18:22.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.384+0000 7f0054ff9640 1 -- 172.21.15.116:0/2720653224 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0060018040 con 0x5614800b14a0 2024-09-12T03:18:22.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.384+0000 7f0054ff9640 1 -- 172.21.15.116:0/2720653224 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00600276f0 con 0x5614800b14a0 2024-09-12T03:18:22.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.384+0000 7f00777d9740 1 -- 172.21.15.116:0/2720653224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614800b14a0 msgr2=0x5614800c84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:22.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.384+0000 7f00777d9740 1 --2- 172.21.15.116:0/2720653224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614800b14a0 0x5614800c84e0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f00600077a0 tx=0x7f006000f040 comp rx=0 tx=0).stop 2024-09-12T03:18:22.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.385+0000 7f00777d9740 1 -- 172.21.15.116:0/2720653224 shutdown_connections 2024-09-12T03:18:22.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.385+0000 7f00777d9740 1 --2- 172.21.15.116:0/2720653224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614800bc680 0x5614800d3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.385+0000 7f00777d9740 1 --2- 172.21.15.116:0/2720653224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614800b1dc0 0x5614800aca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.385+0000 7f00777d9740 1 --2- 172.21.15.116:0/2720653224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614800b14a0 0x5614800c84e0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.385+0000 7f00777d9740 1 -- 172.21.15.116:0/2720653224 >> 172.21.15.116:0/2720653224 conn(0x5614800ada30 msgr2=0x5614800ac350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:22.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.385+0000 7f00777d9740 1 -- 172.21.15.116:0/2720653224 shutdown_connections 2024-09-12T03:18:22.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.385+0000 7f00777d9740 1 -- 172.21.15.116:0/2720653224 wait complete. 2024-09-12T03:18:22.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.385+0000 7f00777d9740 1 -- start start 2024-09-12T03:18:22.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.385+0000 7f00777d9740 1 Processor -- start 2024-09-12T03:18:22.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.385+0000 7f00777d9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614800b14a0 0x56148012efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:22.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.385+0000 7f00777d9740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614800b1dc0 0x56148012f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:22.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.385+0000 7f00777d9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614800bc680 0x561480133ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:22.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.385+0000 7f00777d9740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f00701abcf0 con 0x5614800b1dc0 2024-09-12T03:18:22.417 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:18:22.417 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:18:22.417 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:18:22.417 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:18:22.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.385+0000 7f00777d9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f00701ad160 con 0x5614800bc680 2024-09-12T03:18:22.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.385+0000 7f00777d9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f00701ac3c0 con 0x5614800b14a0 2024-09-12T03:18:22.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.385+0000 7f00777d9740 10 client.0 _ll_register_callbacks cb 0x7f0068009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:18:22.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.385+0000 7f006e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614800b1dc0 0x56148012f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:22.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.385+0000 7f006e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614800b1dc0 0x56148012f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60428/0 (socket says 172.21.15.116:60428) 2024-09-12T03:18:22.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.385+0000 7f006e7fc640 1 -- 172.21.15.116:0/741789575 learned_addr learned my addr 172.21.15.116:0/741789575 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:22.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.386+0000 7f006effd640 1 --2- 172.21.15.116:0/741789575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614800b14a0 0x56148012efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:22.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.386+0000 7f006f7fe640 1 --2- 172.21.15.116:0/741789575 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614800bc680 0x561480133ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:22.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.386+0000 7f006e7fc640 1 -- 172.21.15.116:0/741789575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614800b14a0 msgr2=0x56148012efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:22.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.386+0000 7f006e7fc640 1 --2- 172.21.15.116:0/741789575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614800b14a0 0x56148012efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.386+0000 7f006e7fc640 1 -- 172.21.15.116:0/741789575 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614800bc680 msgr2=0x561480133ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:22.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.386+0000 7f006e7fc640 1 --2- 172.21.15.116:0/741789575 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614800bc680 0x561480133ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.386+0000 7f006e7fc640 1 -- 172.21.15.116:0/741789575 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561480134560 con 0x5614800b1dc0 2024-09-12T03:18:22.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.386+0000 7f006effd640 1 --2- 172.21.15.116:0/741789575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614800b14a0 0x56148012efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:18:22.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.386+0000 7f006f7fe640 1 --2- 172.21.15.116:0/741789575 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614800bc680 0x561480133ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:18:22.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.386+0000 7f006e7fc640 1 --2- 172.21.15.116:0/741789575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614800b1dc0 0x56148012f520 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f0064009930 tx=0x7f0064009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d801950f3fd1a82 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:22.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.386+0000 7f0052ff5640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:18:22.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.386+0000 7f0052ff5640 1 -- 172.21.15.116:0/741789575 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0064043070 con 0x5614800b1dc0 2024-09-12T03:18:22.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.386+0000 7f0052ff5640 1 -- 172.21.15.116:0/741789575 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f0064003cc0 con 0x5614800b1dc0 2024-09-12T03:18:22.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.386+0000 7f00777d9740 1 -- 172.21.15.116:0/741789575 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0060018040 con 0x5614800b1dc0 2024-09-12T03:18:22.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.386+0000 7f0052ff5640 1 -- 172.21.15.116:0/741789575 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0064035040 con 0x5614800b1dc0 2024-09-12T03:18:22.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.386+0000 7f00777d9740 10 client.19512 fetch_fsmap learned FSMap version 25 2024-09-12T03:18:22.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.386+0000 7f00777d9740 1 -- 172.21.15.116:0/741789575 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56148013b950 con 0x5614800b1dc0 2024-09-12T03:18:22.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.386+0000 7f0052ff5640 1 -- 172.21.15.116:0/741789575 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0060018040 con 0x5614800b1dc0 2024-09-12T03:18:22.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.386+0000 7f0052ff5640 1 -- 172.21.15.116:0/741789575 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f006403a040 con 0x5614800b1dc0 2024-09-12T03:18:22.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.387+0000 7f00777d9740 10 client.19512 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:18:22.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.387+0000 7f00777d9740 10 client.19512 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:18:22.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.387+0000 7f00777d9740 20 client.19512 populate_metadata read hostname 'smithi116' 2024-09-12T03:18:22.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.387+0000 7f00777d9740 1 --2- 172.21.15.116:0/741789575 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56148013c180 0x56148013e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:22.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.387+0000 7f00777d9740 4 client.19512 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:18:22.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.387+0000 7f00777d9740 1 -- 172.21.15.116:0/741789575 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56148001ed60 con 0x56148013c180 2024-09-12T03:18:22.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.387+0000 7f006effd640 1 --2- 172.21.15.116:0/741789575 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56148013c180 0x56148013e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:22.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.387+0000 7f006effd640 1 --2- 172.21.15.116:0/741789575 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56148013c180 0x56148013e5a0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc78ed92dc8a790 server_cookie=4088fe9641e1a227 in_seq=0 out_seq=0 2024-09-12T03:18:22.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.387+0000 7f0052ff5640 10 client.19512 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:18:22.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.387+0000 7f0052ff5640 1 -- 172.21.15.116:0/741789575 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56148001ed60 con 0x56148013c180 2024-09-12T03:18:22.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.387+0000 7f0052ff5640 10 client.19512 handle_command_reply: tid=0 2024-09-12T03:18:22.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.387+0000 7f00777d9740 1 client.19512 shutdown 2024-09-12T03:18:22.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.387+0000 7f00777d9740 1 -- 172.21.15.116:0/741789575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614800b1dc0 msgr2=0x56148012f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:22.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.387+0000 7f00777d9740 1 --2- 172.21.15.116:0/741789575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614800b1dc0 0x56148012f520 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f0064009930 tx=0x7f0064009260 comp rx=0 tx=0).stop 2024-09-12T03:18:22.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.387+0000 7f00777d9740 1 -- 172.21.15.116:0/741789575 shutdown_connections 2024-09-12T03:18:22.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.387+0000 7f00777d9740 1 --2- 172.21.15.116:0/741789575 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x56148013c180 0x56148013e5a0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.387+0000 7f00777d9740 1 --2- 172.21.15.116:0/741789575 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614800bc680 0x561480133ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.387+0000 7f00777d9740 1 --2- 172.21.15.116:0/741789575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5614800b1dc0 0x56148012f520 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.387+0000 7f0052ff5640 0 client.19512 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:18:22.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.387+0000 7f00777d9740 1 --2- 172.21.15.116:0/741789575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5614800b14a0 0x56148012efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.387+0000 7f00777d9740 1 -- 172.21.15.116:0/741789575 >> 172.21.15.116:0/741789575 conn(0x5614800ada30 msgr2=0x5614800acb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:22.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.388+0000 7f00777d9740 1 -- 172.21.15.116:0/741789575 shutdown_connections 2024-09-12T03:18:22.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.388+0000 7f00777d9740 1 -- 172.21.15.116:0/741789575 wait complete. 2024-09-12T03:18:22.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.388+0000 7f00777d9740 20 client.19512 trim_cache size 0 max 16384 2024-09-12T03:18:22.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.388+0000 7f0075400640 1 -- 172.21.15.116:0/2577653947 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f00580a2b80 msgr2=0x7f00580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:22.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.388+0000 7f0075400640 1 --2- 172.21.15.116:0/2577653947 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f00580a2b80 0x7f00580a5040 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0064031550 tx=0x7f0064005fd0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.388+0000 7f0075400640 1 -- 172.21.15.116:0/2577653947 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0070106a80 msgr2=0x7f00701a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:22.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.388+0000 7f0075400640 1 --2- 172.21.15.116:0/2577653947 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0070106a80 0x7f00701a44b0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f006800b970 tx=0x7f006800be30 comp rx=0 tx=0).stop 2024-09-12T03:18:22.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.388+0000 7f0075400640 1 -- 172.21.15.116:0/2577653947 shutdown_connections 2024-09-12T03:18:22.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.388+0000 7f0075400640 1 --2- 172.21.15.116:0/2577653947 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f00580a2b80 0x7f00580a5040 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.388+0000 7f0075400640 1 --2- 172.21.15.116:0/2577653947 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0070106a80 0x7f00701a44b0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.388+0000 7f0075400640 1 --2- 172.21.15.116:0/2577653947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0070101420 0x7f0070199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.388+0000 7f0075400640 1 --2- 172.21.15.116:0/2577653947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0070100a80 0x7f0070199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.388+0000 7f0075400640 1 -- 172.21.15.116:0/2577653947 >> 172.21.15.116:0/2577653947 conn(0x7f00700fc7a0 msgr2=0x7f00700fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:22.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.388+0000 7f0075400640 1 -- 172.21.15.116:0/2577653947 shutdown_connections 2024-09-12T03:18:22.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:22.388+0000 7f0075400640 1 -- 172.21.15.116:0/2577653947 wait complete. 2024-09-12T03:18:23.198 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:18:23.199 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:18:23.445 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:23.716 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:23 smithi116 ceph-mon[31410]: pgmap v800: 97 pgs: 97 active+clean; 7.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:18:23.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:23 smithi177 ceph-mon[35305]: pgmap v800: 97 pgs: 97 active+clean; 7.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:18:23.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.895+0000 7f4d022c4640 1 -- 172.21.15.116:0/2540454395 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cfc101de0 msgr2=0x7f4cfc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:23.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.895+0000 7f4d022c4640 1 --2- 172.21.15.116:0/2540454395 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cfc101de0 0x7f4cfc1021c0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009900 tx=0x7f4ce402a410 comp rx=0 tx=0).stop 2024-09-12T03:18:23.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.895+0000 7f4d022c4640 1 -- 172.21.15.116:0/2540454395 shutdown_connections 2024-09-12T03:18:23.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.895+0000 7f4d022c4640 1 --2- 172.21.15.116:0/2540454395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cfc106f10 0x7f4cfc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:23.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.895+0000 7f4d022c4640 1 --2- 172.21.15.116:0/2540454395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cfc102790 0x7f4cfc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:23.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.895+0000 7f4d022c4640 1 --2- 172.21.15.116:0/2540454395 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cfc101de0 0x7f4cfc1021c0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:23.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.895+0000 7f4d022c4640 1 -- 172.21.15.116:0/2540454395 >> 172.21.15.116:0/2540454395 conn(0x7f4cfc0fd8c0 msgr2=0x7f4cfc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:23.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.895+0000 7f4d022c4640 1 -- 172.21.15.116:0/2540454395 shutdown_connections 2024-09-12T03:18:23.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.895+0000 7f4d022c4640 1 -- 172.21.15.116:0/2540454395 wait complete. 2024-09-12T03:18:23.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.896+0000 7f4d022c4640 1 Processor -- start 2024-09-12T03:18:23.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.896+0000 7f4d022c4640 1 -- start start 2024-09-12T03:18:23.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.896+0000 7f4d022c4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cfc101de0 0x7f4cfc198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:23.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.896+0000 7f4d022c4640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cfc102790 0x7f4cfc198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:23.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.896+0000 7f4d022c4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cfc106f10 0x7f4cfc19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:23.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.896+0000 7f4d022c4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc111f70 con 0x7f4cfc101de0 2024-09-12T03:18:23.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.896+0000 7f4d022c4640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc111df0 con 0x7f4cfc102790 2024-09-12T03:18:23.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.896+0000 7f4d022c4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4cfc1120f0 con 0x7f4cfc106f10 2024-09-12T03:18:23.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.896+0000 7f4cfbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cfc101de0 0x7f4cfc198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:23.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.896+0000 7f4cfbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cfc101de0 0x7f4cfc198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60438/0 (socket says 172.21.15.116:60438) 2024-09-12T03:18:23.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.896+0000 7f4cfbfff640 1 -- 172.21.15.116:0/2313459870 learned_addr learned my addr 172.21.15.116:0/2313459870 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:23.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.896+0000 7f4cfb7fe640 1 --2- 172.21.15.116:0/2313459870 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cfc102790 0x7f4cfc198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:23.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.896+0000 7f4d00861640 1 --2- 172.21.15.116:0/2313459870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cfc106f10 0x7f4cfc19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:23.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.896+0000 7f4cfbfff640 1 -- 172.21.15.116:0/2313459870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cfc106f10 msgr2=0x7f4cfc19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:23.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.896+0000 7f4cfbfff640 1 --2- 172.21.15.116:0/2313459870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cfc106f10 0x7f4cfc19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:23.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.896+0000 7f4cfbfff640 1 -- 172.21.15.116:0/2313459870 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cfc102790 msgr2=0x7f4cfc198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:23.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.896+0000 7f4cfbfff640 1 --2- 172.21.15.116:0/2313459870 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cfc102790 0x7f4cfc198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:23.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.896+0000 7f4cfbfff640 1 -- 172.21.15.116:0/2313459870 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cfc19d3c0 con 0x7f4cfc101de0 2024-09-12T03:18:23.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.896+0000 7f4d00861640 1 --2- 172.21.15.116:0/2313459870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cfc106f10 0x7f4cfc19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:18:23.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.897+0000 7f4cfbfff640 1 --2- 172.21.15.116:0/2313459870 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cfc101de0 0x7f4cfc198410 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4006010 tx=0x7f4ce40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:23.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.897+0000 7f4cf97fa640 1 -- 172.21.15.116:0/2313459870 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce403f030 con 0x7f4cfc101de0 2024-09-12T03:18:23.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.897+0000 7f4cf97fa640 1 -- 172.21.15.116:0/2313459870 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ce4003ce0 con 0x7f4cfc101de0 2024-09-12T03:18:23.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.897+0000 7f4d022c4640 1 -- 172.21.15.116:0/2313459870 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cfc19d650 con 0x7f4cfc101de0 2024-09-12T03:18:23.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.897+0000 7f4cf97fa640 1 -- 172.21.15.116:0/2313459870 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4035070 con 0x7f4cfc101de0 2024-09-12T03:18:23.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.897+0000 7f4d022c4640 1 -- 172.21.15.116:0/2313459870 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cfc1a4f90 con 0x7f4cfc101de0 2024-09-12T03:18:23.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.898+0000 7f4d022c4640 1 -- 172.21.15.116:0/2313459870 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cfc1021c0 con 0x7f4cfc101de0 2024-09-12T03:18:23.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.899+0000 7f4cf97fa640 1 -- 172.21.15.116:0/2313459870 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4ce4042030 con 0x7f4cfc101de0 2024-09-12T03:18:23.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.900+0000 7f4cf97fa640 1 --2- 172.21.15.116:0/2313459870 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4ce80a2c00 0x7f4ce80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:23.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.900+0000 7f4cfb7fe640 1 --2- 172.21.15.116:0/2313459870 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4ce80a2c00 0x7f4ce80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:23.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.900+0000 7f4cf97fa640 1 -- 172.21.15.116:0/2313459870 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f4ce40ef7f0 con 0x7f4cfc101de0 2024-09-12T03:18:23.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.900+0000 7f4cfb7fe640 1 --2- 172.21.15.116:0/2313459870 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4ce80a2c00 0x7f4ce80a50c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc0fdce0 tx=0x7f4cec00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:23.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:23.903+0000 7f4cf97fa640 1 -- 172.21.15.116:0/2313459870 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4ce40bc840 con 0x7f4cfc101de0 2024-09-12T03:18:23.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:23 smithi183 ceph-mon[35535]: pgmap v800: 97 pgs: 97 active+clean; 7.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:18:24.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:24.102+0000 7f4d022c4640 1 -- 172.21.15.116:0/2313459870 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4cfc199770 con 0x7f4cfc101de0 2024-09-12T03:18:24.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:24.103+0000 7f4cf97fa640 1 -- 172.21.15.116:0/2313459870 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f4cfc199770 con 0x7f4cfc101de0 2024-09-12T03:18:24.103 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:18:24.103 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:18:24.104 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:18:24.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:24.105+0000 7f4d022c4640 1 -- 172.21.15.116:0/2313459870 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4ce80a2c00 msgr2=0x7f4ce80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:24.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:24.105+0000 7f4d022c4640 1 --2- 172.21.15.116:0/2313459870 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4ce80a2c00 0x7f4ce80a50c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc0fdce0 tx=0x7f4cec00b390 comp rx=0 tx=0).stop 2024-09-12T03:18:24.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:24.105+0000 7f4d022c4640 1 -- 172.21.15.116:0/2313459870 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cfc101de0 msgr2=0x7f4cfc198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:24.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:24.105+0000 7f4d022c4640 1 --2- 172.21.15.116:0/2313459870 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cfc101de0 0x7f4cfc198410 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4006010 tx=0x7f4ce40042f0 comp rx=0 tx=0).stop 2024-09-12T03:18:24.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:24.105+0000 7f4d022c4640 1 -- 172.21.15.116:0/2313459870 shutdown_connections 2024-09-12T03:18:24.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:24.105+0000 7f4d022c4640 1 --2- 172.21.15.116:0/2313459870 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f4ce80a2c00 0x7f4ce80a50c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:24.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:24.105+0000 7f4d022c4640 1 --2- 172.21.15.116:0/2313459870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cfc106f10 0x7f4cfc19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:24.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:24.105+0000 7f4d022c4640 1 --2- 172.21.15.116:0/2313459870 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4cfc102790 0x7f4cfc198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:24.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:24.105+0000 7f4d022c4640 1 --2- 172.21.15.116:0/2313459870 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cfc101de0 0x7f4cfc198410 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:24.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:24.105+0000 7f4d022c4640 1 -- 172.21.15.116:0/2313459870 >> 172.21.15.116:0/2313459870 conn(0x7f4cfc0fd8c0 msgr2=0x7f4cfc0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:24.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:24.105+0000 7f4d022c4640 1 -- 172.21.15.116:0/2313459870 shutdown_connections 2024-09-12T03:18:24.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:24.105+0000 7f4d022c4640 1 -- 172.21.15.116:0/2313459870 wait complete. 2024-09-12T03:18:24.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:24 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2313459870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:24.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:24 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:24.763 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:24 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2313459870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:24.763 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:24 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:24.790 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:18:24.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:24 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2313459870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:24.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:24 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:25.033 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:25.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.526+0000 7ff5ffabe640 1 -- 172.21.15.116:0/1359324537 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5f81055b0 msgr2=0x7ff5f810d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:25.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.526+0000 7ff5ffabe640 1 --2- 172.21.15.116:0/1359324537 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5f81055b0 0x7ff5f810d130 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0009ca0 tx=0x7ff5f002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.526+0000 7ff5ffabe640 1 -- 172.21.15.116:0/1359324537 shutdown_connections 2024-09-12T03:18:25.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.526+0000 7ff5ffabe640 1 --2- 172.21.15.116:0/1359324537 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5f81055b0 0x7ff5f810d130 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.526+0000 7ff5ffabe640 1 --2- 172.21.15.116:0/1359324537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5f8100e30 0x7ff5f8104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.526+0000 7ff5ffabe640 1 --2- 172.21.15.116:0/1359324537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff5f8100480 0x7ff5f8100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.526+0000 7ff5ffabe640 1 -- 172.21.15.116:0/1359324537 >> 172.21.15.116:0/1359324537 conn(0x7ff5f8077210 msgr2=0x7ff5f80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:25.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.526+0000 7ff5ffabe640 1 -- 172.21.15.116:0/1359324537 shutdown_connections 2024-09-12T03:18:25.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.526+0000 7ff5ffabe640 1 -- 172.21.15.116:0/1359324537 wait complete. 2024-09-12T03:18:25.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.527+0000 7ff5ffabe640 1 Processor -- start 2024-09-12T03:18:25.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.527+0000 7ff5ffabe640 1 -- start start 2024-09-12T03:18:25.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.527+0000 7ff5ffabe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5f8100480 0x7ff5f8198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:25.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.527+0000 7ff5ffabe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff5f8100e30 0x7ff5f8198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:25.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.527+0000 7ff5ffabe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5f81055b0 0x7ff5f819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:25.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.527+0000 7ff5ffabe640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f810fdb0 con 0x7ff5f81055b0 2024-09-12T03:18:25.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.527+0000 7ff5ffabe640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f810fc30 con 0x7ff5f8100e30 2024-09-12T03:18:25.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.527+0000 7ff5ffabe640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f810ff30 con 0x7ff5f8100480 2024-09-12T03:18:25.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.527+0000 7ff5fe05b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5f81055b0 0x7ff5f819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:25.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.527+0000 7ff5fe05b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5f81055b0 0x7ff5f819cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60464/0 (socket says 172.21.15.116:60464) 2024-09-12T03:18:25.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.527+0000 7ff5fe05b640 1 -- 172.21.15.116:0/995630107 learned_addr learned my addr 172.21.15.116:0/995630107 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:25.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.527+0000 7ff5fd059640 1 --2- 172.21.15.116:0/995630107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff5f8100e30 0x7ff5f8198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:25.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.527+0000 7ff5fd85a640 1 --2- 172.21.15.116:0/995630107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5f8100480 0x7ff5f8198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:25.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.528+0000 7ff5fe05b640 1 -- 172.21.15.116:0/995630107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5f8100480 msgr2=0x7ff5f8198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:25.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.528+0000 7ff5fe05b640 1 --2- 172.21.15.116:0/995630107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5f8100480 0x7ff5f8198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.528+0000 7ff5fe05b640 1 -- 172.21.15.116:0/995630107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff5f8100e30 msgr2=0x7ff5f8198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:25.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.528+0000 7ff5fe05b640 1 --2- 172.21.15.116:0/995630107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff5f8100e30 0x7ff5f8198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.528+0000 7ff5fe05b640 1 -- 172.21.15.116:0/995630107 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f819d400 con 0x7ff5f81055b0 2024-09-12T03:18:25.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.528+0000 7ff5fe05b640 1 --2- 172.21.15.116:0/995630107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5f81055b0 0x7ff5f819cd20 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0031a20 tx=0x7ff5f0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:25.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.528+0000 7ff5eeffd640 1 -- 172.21.15.116:0/995630107 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0003b70 con 0x7ff5f81055b0 2024-09-12T03:18:25.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.528+0000 7ff5eeffd640 1 -- 172.21.15.116:0/995630107 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5f0003d10 con 0x7ff5f81055b0 2024-09-12T03:18:25.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.528+0000 7ff5ffabe640 1 -- 172.21.15.116:0/995630107 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f819d690 con 0x7ff5f81055b0 2024-09-12T03:18:25.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.528+0000 7ff5eeffd640 1 -- 172.21.15.116:0/995630107 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f002fc80 con 0x7ff5f81055b0 2024-09-12T03:18:25.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.528+0000 7ff5ffabe640 1 -- 172.21.15.116:0/995630107 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f81a4fd0 con 0x7ff5f81055b0 2024-09-12T03:18:25.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.530+0000 7ff5eeffd640 1 -- 172.21.15.116:0/995630107 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ff5f00046a0 con 0x7ff5f81055b0 2024-09-12T03:18:25.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.531+0000 7ff601e97740 1 Processor -- start 2024-09-12T03:18:25.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.531+0000 7ff601e97740 1 -- start start 2024-09-12T03:18:25.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.531+0000 7ff601e97740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559985edab40 0x559985ed9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:25.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.531+0000 7ff601e97740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559985ed95d0 0x559985ed9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:25.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.531+0000 7ff601e97740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559985ee4370 0x559985ee6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:25.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.531+0000 7ff601e97740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559985e3f2f0 con 0x559985ee4370 2024-09-12T03:18:25.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.531+0000 7ff601e97740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559985de6cf0 con 0x559985ed95d0 2024-09-12T03:18:25.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.531+0000 7ff601e97740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559985e3b2b0 con 0x559985edab40 2024-09-12T03:18:25.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.531+0000 7ff5eeffd640 1 --2- 172.21.15.116:0/995630107 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff5e40a2c00 0x7ff5e40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:25.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.531+0000 7ff5fd059640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559985ee4370 0x559985ee6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:25.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.531+0000 7ff5fd059640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559985ee4370 0x559985ee6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60472/0 (socket says 172.21.15.116:60472) 2024-09-12T03:18:25.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.531+0000 7ff5fd059640 1 -- 172.21.15.116:0/2933307454 learned_addr learned my addr 172.21.15.116:0/2933307454 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:25.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.531+0000 7ff5fd85a640 1 --2- 172.21.15.116:0/2933307454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559985ed95d0 0x559985ed9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:25.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.531+0000 7ff5fd059640 1 --2- 172.21.15.116:0/2933307454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559985edab40 0x559985ed9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:25.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.531+0000 7ff5fe05b640 1 --2- 172.21.15.116:0/995630107 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff5e40a2c00 0x7ff5e40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:25.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.532+0000 7ff5fd059640 1 -- 172.21.15.116:0/2933307454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559985edab40 msgr2=0x559985ed9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:25.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.532+0000 7ff5eeffd640 1 -- 172.21.15.116:0/995630107 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff5f00f0200 con 0x7ff5f81055b0 2024-09-12T03:18:25.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.532+0000 7ff5fd059640 1 --2- 172.21.15.116:0/2933307454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559985edab40 0x559985ed9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.532+0000 7ff5fd059640 1 -- 172.21.15.116:0/2933307454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559985ed95d0 msgr2=0x559985ed9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:25.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.532+0000 7ff5fd059640 1 --2- 172.21.15.116:0/2933307454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559985ed95d0 0x559985ed9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.532+0000 7ff5fd059640 1 -- 172.21.15.116:0/2933307454 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559985dcb3a0 con 0x559985ee4370 2024-09-12T03:18:25.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.532+0000 7ff5fd85a640 1 --2- 172.21.15.116:0/2933307454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559985ed95d0 0x559985ed9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:18:25.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.532+0000 7ff5fd059640 1 --2- 172.21.15.116:0/2933307454 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559985ee4370 0x559985ee6760 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4006aa0 tx=0x7ff5f4009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a22f2dfb0649a47 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:25.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.532+0000 7ff5fe05b640 1 --2- 172.21.15.116:0/995630107 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff5e40a2c00 0x7ff5e40a50c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff5f00b8f80 tx=0x7ff5f00876f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:25.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.532+0000 7ff5e3fff640 1 -- 172.21.15.116:0/2933307454 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4015760 con 0x559985ee4370 2024-09-12T03:18:25.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.532+0000 7ff5e3fff640 1 -- 172.21.15.116:0/2933307454 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5f401d070 con 0x559985ee4370 2024-09-12T03:18:25.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.532+0000 7ff5e3fff640 1 -- 172.21.15.116:0/2933307454 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4018aa0 con 0x559985ee4370 2024-09-12T03:18:25.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.533+0000 7ff601e97740 1 -- 172.21.15.116:0/2933307454 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559985ee4370 msgr2=0x559985ee6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:25.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.533+0000 7ff601e97740 1 --2- 172.21.15.116:0/2933307454 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559985ee4370 0x559985ee6760 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4006aa0 tx=0x7ff5f4009510 comp rx=0 tx=0).stop 2024-09-12T03:18:25.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.533+0000 7ff601e97740 1 -- 172.21.15.116:0/2933307454 shutdown_connections 2024-09-12T03:18:25.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.533+0000 7ff601e97740 1 --2- 172.21.15.116:0/2933307454 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559985ee4370 0x559985ee6760 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.533+0000 7ff601e97740 1 --2- 172.21.15.116:0/2933307454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559985ed95d0 0x559985ed9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.533+0000 7ff601e97740 1 --2- 172.21.15.116:0/2933307454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559985edab40 0x559985ed9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.533+0000 7ff601e97740 1 -- 172.21.15.116:0/2933307454 >> 172.21.15.116:0/2933307454 conn(0x559985ea2310 msgr2=0x559985ed5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:25.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.533+0000 7ff601e97740 1 -- 172.21.15.116:0/2933307454 shutdown_connections 2024-09-12T03:18:25.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.533+0000 7ff601e97740 1 -- 172.21.15.116:0/2933307454 wait complete. 2024-09-12T03:18:25.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.533+0000 7ff601e97740 1 -- start start 2024-09-12T03:18:25.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.533+0000 7ff601e97740 1 Processor -- start 2024-09-12T03:18:25.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.534+0000 7ff601e97740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559985ed95d0 0x559985ee3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:25.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.534+0000 7ff601e97740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559985edab40 0x559985ee1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:25.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.534+0000 7ff601e97740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559985ee4370 0x559985ee1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:25.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.534+0000 7ff601e97740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559985ee7010 con 0x559985ed95d0 2024-09-12T03:18:25.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.534+0000 7ff601e97740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559985dfa060 con 0x559985ee4370 2024-09-12T03:18:25.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.534+0000 7ff601e97740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559985ee7190 con 0x559985edab40 2024-09-12T03:18:25.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.534+0000 7ff601e97740 10 client.0 _ll_register_callbacks cb 0x559985b4d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:18:25.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.534+0000 7ff5fd059640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559985ed95d0 0x559985ee3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:25.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.534+0000 7ff5fd059640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559985ed95d0 0x559985ee3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60488/0 (socket says 172.21.15.116:60488) 2024-09-12T03:18:25.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.534+0000 7ff5fd059640 1 -- 172.21.15.116:0/3982167669 learned_addr learned my addr 172.21.15.116:0/3982167669 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:25.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.534+0000 7ff5fd059640 1 --2- 172.21.15.116:0/3982167669 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559985ee4370 0x559985ee1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:25.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.534+0000 7ff5fd059640 1 -- 172.21.15.116:0/3982167669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559985edab40 msgr2=0x559985ee1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:25.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.534+0000 7ff5fd85a640 1 --2- 172.21.15.116:0/3982167669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559985edab40 0x559985ee1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:25.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.534+0000 7ff5fd059640 1 --2- 172.21.15.116:0/3982167669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559985edab40 0x559985ee1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.534+0000 7ff5fd059640 1 -- 172.21.15.116:0/3982167669 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559985ee4370 msgr2=0x559985ee1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:25.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.534+0000 7ff5fd059640 1 --2- 172.21.15.116:0/3982167669 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559985ee4370 0x559985ee1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.534+0000 7ff5fd059640 1 -- 172.21.15.116:0/3982167669 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559985bb03c0 con 0x559985ed95d0 2024-09-12T03:18:25.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.534+0000 7ff5fd059640 1 --2- 172.21.15.116:0/3982167669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559985ed95d0 0x559985ee3d90 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7ff5f40094e0 tx=0x7ff5f4018780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02fd1750604c072 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:25.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.534+0000 7ff5e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:18:25.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.534+0000 7ff5e17fa640 1 -- 172.21.15.116:0/3982167669 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4018450 con 0x559985ed95d0 2024-09-12T03:18:25.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.534+0000 7ff5e17fa640 1 -- 172.21.15.116:0/3982167669 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff5f4015730 con 0x559985ed95d0 2024-09-12T03:18:25.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.534+0000 7ff601e97740 1 -- 172.21.15.116:0/3982167669 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5f401d070 con 0x559985ed95d0 2024-09-12T03:18:25.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.535+0000 7ff5e17fa640 1 -- 172.21.15.116:0/3982167669 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff5f4015930 con 0x559985ed95d0 2024-09-12T03:18:25.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.535+0000 7ff601e97740 10 client.19530 fetch_fsmap learned FSMap version 25 2024-09-12T03:18:25.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.535+0000 7ff601e97740 1 -- 172.21.15.116:0/3982167669 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559985ef2ae0 con 0x559985ed95d0 2024-09-12T03:18:25.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.535+0000 7ff5e17fa640 1 -- 172.21.15.116:0/3982167669 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5f401d070 con 0x559985ed95d0 2024-09-12T03:18:25.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.535+0000 7ff5e17fa640 1 -- 172.21.15.116:0/3982167669 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff5f4031ae0 con 0x559985ed95d0 2024-09-12T03:18:25.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.535+0000 7ff601e97740 10 client.19530 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:18:25.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.535+0000 7ff601e97740 10 client.19530 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:18:25.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.535+0000 7ff601e97740 20 client.19530 populate_metadata read hostname 'smithi116' 2024-09-12T03:18:25.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.535+0000 7ff601e97740 1 --2- 172.21.15.116:0/3982167669 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x559985ef3ee0 0x559985ef62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:25.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.535+0000 7ff601e97740 4 client.19530 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:18:25.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.535+0000 7ff601e97740 1 -- 172.21.15.116:0/3982167669 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559985e4c860 con 0x559985ef3ee0 2024-09-12T03:18:25.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.535+0000 7ff5fd85a640 1 --2- 172.21.15.116:0/3982167669 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x559985ef3ee0 0x559985ef62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:25.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.536+0000 7ff5fd85a640 1 --2- 172.21.15.116:0/3982167669 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x559985ef3ee0 0x559985ef62d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=885e1e3794b00a8 server_cookie=862906e7466688ee in_seq=0 out_seq=0 2024-09-12T03:18:25.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.536+0000 7ff5e17fa640 10 client.19530 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:18:25.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.536+0000 7ff5e17fa640 1 -- 172.21.15.116:0/3982167669 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559985e4c860 con 0x559985ef3ee0 2024-09-12T03:18:25.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.536+0000 7ff5e17fa640 10 client.19530 handle_command_reply: tid=0 2024-09-12T03:18:25.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.536+0000 7ff601e97740 1 client.19530 shutdown 2024-09-12T03:18:25.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.536+0000 7ff601e97740 1 -- 172.21.15.116:0/3982167669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559985ed95d0 msgr2=0x559985ee3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:25.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.536+0000 7ff601e97740 1 --2- 172.21.15.116:0/3982167669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559985ed95d0 0x559985ee3d90 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7ff5f40094e0 tx=0x7ff5f4018780 comp rx=0 tx=0).stop 2024-09-12T03:18:25.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.536+0000 7ff601e97740 1 -- 172.21.15.116:0/3982167669 shutdown_connections 2024-09-12T03:18:25.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.536+0000 7ff601e97740 1 --2- 172.21.15.116:0/3982167669 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559985ee4370 0x559985ee1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.536+0000 7ff601e97740 1 --2- 172.21.15.116:0/3982167669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559985edab40 0x559985ee1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.536+0000 7ff601e97740 1 --2- 172.21.15.116:0/3982167669 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x559985ef3ee0 0x559985ef62d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.536+0000 7ff601e97740 1 --2- 172.21.15.116:0/3982167669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559985ed95d0 0x559985ee3d90 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.536+0000 7ff601e97740 1 -- 172.21.15.116:0/3982167669 >> 172.21.15.116:0/3982167669 conn(0x559985ea2310 msgr2=0x559985edb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:25.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.536+0000 7ff5e17fa640 0 client.19530 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:18:25.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.536+0000 7ff601e97740 1 -- 172.21.15.116:0/3982167669 shutdown_connections 2024-09-12T03:18:25.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.536+0000 7ff601e97740 1 -- 172.21.15.116:0/3982167669 wait complete. 2024-09-12T03:18:25.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.537+0000 7ff601e97740 20 client.19530 trim_cache size 0 max 16384 2024-09-12T03:18:25.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.549+0000 7ff601e97740 1 Processor -- start 2024-09-12T03:18:25.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.549+0000 7ff601e97740 1 -- start start 2024-09-12T03:18:25.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.550+0000 7ff601e97740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559985ed95d0 0x559985ef02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:25.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.550+0000 7ff601e97740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559985edab40 0x559985efb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:25.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.550+0000 7ff601e97740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559985ee4370 0x559985efbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:25.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.550+0000 7ff601e97740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f81afd00 con 0x559985edab40 2024-09-12T03:18:25.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.550+0000 7ff601e97740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f81ad5d0 con 0x559985ee4370 2024-09-12T03:18:25.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.550+0000 7ff601e97740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f81ab420 con 0x559985ed95d0 2024-09-12T03:18:25.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.550+0000 7ff5fd85a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559985edab40 0x559985efb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:25.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.550+0000 7ff5fd85a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559985edab40 0x559985efb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60500/0 (socket says 172.21.15.116:60500) 2024-09-12T03:18:25.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.550+0000 7ff5fd85a640 1 -- 172.21.15.116:0/3289903393 learned_addr learned my addr 172.21.15.116:0/3289903393 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:25.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.550+0000 7ff5fd059640 1 --2- 172.21.15.116:0/3289903393 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559985ee4370 0x559985efbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:25.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.550+0000 7ff5fd85a640 1 -- 172.21.15.116:0/3289903393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559985ed95d0 msgr2=0x559985ef02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:25.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.550+0000 7ff5fd85a640 1 --2- 172.21.15.116:0/3289903393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559985ed95d0 0x559985ef02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.550+0000 7ff5fd85a640 1 -- 172.21.15.116:0/3289903393 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559985ee4370 msgr2=0x559985efbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:25.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.550+0000 7ff5fd85a640 1 --2- 172.21.15.116:0/3289903393 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559985ee4370 0x559985efbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.550+0000 7ff5fd85a640 1 -- 172.21.15.116:0/3289903393 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559985ed92d0 con 0x559985edab40 2024-09-12T03:18:25.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.550+0000 7ff5fd85a640 1 --2- 172.21.15.116:0/3289903393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559985edab40 0x559985efb930 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x559985ee08d0 tx=0x7ff5e800dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a82a605ccb91e9ec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:25.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.550+0000 7ff5e37fe640 1 -- 172.21.15.116:0/3289903393 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8020030 con 0x559985edab40 2024-09-12T03:18:25.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.550+0000 7ff5e37fe640 1 -- 172.21.15.116:0/3289903393 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5e8005500 con 0x559985edab40 2024-09-12T03:18:25.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.550+0000 7ff5e37fe640 1 -- 172.21.15.116:0/3289903393 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8004e10 con 0x559985edab40 2024-09-12T03:18:25.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.551+0000 7ff601e97740 1 -- 172.21.15.116:0/3289903393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559985edab40 msgr2=0x559985efb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:25.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.551+0000 7ff601e97740 1 --2- 172.21.15.116:0/3289903393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559985edab40 0x559985efb930 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x559985ee08d0 tx=0x7ff5e800dd60 comp rx=0 tx=0).stop 2024-09-12T03:18:25.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.551+0000 7ff601e97740 1 -- 172.21.15.116:0/3289903393 shutdown_connections 2024-09-12T03:18:25.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.551+0000 7ff601e97740 1 --2- 172.21.15.116:0/3289903393 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559985ee4370 0x559985efbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.551+0000 7ff601e97740 1 --2- 172.21.15.116:0/3289903393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559985edab40 0x559985efb930 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.551+0000 7ff601e97740 1 --2- 172.21.15.116:0/3289903393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559985ed95d0 0x559985ef02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.551+0000 7ff601e97740 1 -- 172.21.15.116:0/3289903393 >> 172.21.15.116:0/3289903393 conn(0x559985ea2310 msgr2=0x559985ee0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:25.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.551+0000 7ff601e97740 1 -- 172.21.15.116:0/3289903393 shutdown_connections 2024-09-12T03:18:25.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.551+0000 7ff601e97740 1 -- 172.21.15.116:0/3289903393 wait complete. 2024-09-12T03:18:25.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.551+0000 7ff601e97740 1 -- start start 2024-09-12T03:18:25.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.552+0000 7ff601e97740 1 Processor -- start 2024-09-12T03:18:25.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.552+0000 7ff601e97740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559985ed95d0 0x559985ede140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:25.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.552+0000 7ff601e97740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559985edab40 0x559985ede680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:25.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.552+0000 7ff601e97740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559985ee4370 0x559985edebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:25.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.552+0000 7ff601e97740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f81aba90 con 0x559985edab40 2024-09-12T03:18:25.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.552+0000 7ff601e97740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f81acf00 con 0x559985ed95d0 2024-09-12T03:18:25.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.552+0000 7ff601e97740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f81ac160 con 0x559985ee4370 2024-09-12T03:18:25.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.552+0000 7ff601e97740 10 client.0 _ll_register_callbacks cb 0x7ff5e800d4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:18:25.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.552+0000 7ff5fd85a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559985edab40 0x559985ede680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:25.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.552+0000 7ff5fd85a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559985edab40 0x559985ede680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60512/0 (socket says 172.21.15.116:60512) 2024-09-12T03:18:25.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.552+0000 7ff5fd85a640 1 -- 172.21.15.116:0/2129191510 learned_addr learned my addr 172.21.15.116:0/2129191510 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:25.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.552+0000 7ff5fd059640 1 --2- 172.21.15.116:0/2129191510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559985ed95d0 0x559985ede140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:25.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.552+0000 7ff5fd059640 1 --2- 172.21.15.116:0/2129191510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559985ee4370 0x559985edebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:25.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.552+0000 7ff5fd85a640 1 -- 172.21.15.116:0/2129191510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559985ee4370 msgr2=0x559985edebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:25.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.552+0000 7ff5fd85a640 1 --2- 172.21.15.116:0/2129191510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559985ee4370 0x559985edebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.552+0000 7ff5fd85a640 1 -- 172.21.15.116:0/2129191510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559985ed95d0 msgr2=0x559985ede140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:25.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.552+0000 7ff5fd85a640 1 --2- 172.21.15.116:0/2129191510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559985ed95d0 0x559985ede140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.552+0000 7ff5fd85a640 1 -- 172.21.15.116:0/2129191510 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559985ee1fe0 con 0x559985edab40 2024-09-12T03:18:25.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.553+0000 7ff5fd85a640 1 --2- 172.21.15.116:0/2129191510 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559985edab40 0x559985ede680 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8000c00 tx=0x7ff5e8004930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac42bedf72fa48fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:25.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.553+0000 7ff5e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:18:25.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.553+0000 7ff5e17fa640 1 -- 172.21.15.116:0/2129191510 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e802a030 con 0x559985edab40 2024-09-12T03:18:25.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.553+0000 7ff5e17fa640 1 -- 172.21.15.116:0/2129191510 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7ff5e8021070 con 0x559985edab40 2024-09-12T03:18:25.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.553+0000 7ff601e97740 1 -- 172.21.15.116:0/2129191510 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5e8005500 con 0x559985edab40 2024-09-12T03:18:25.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.553+0000 7ff5e17fa640 1 -- 172.21.15.116:0/2129191510 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff5e8020030 con 0x559985edab40 2024-09-12T03:18:25.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.553+0000 7ff601e97740 10 client.19536 fetch_fsmap learned FSMap version 25 2024-09-12T03:18:25.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.553+0000 7ff601e97740 1 -- 172.21.15.116:0/2129191510 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559985f62f50 con 0x559985edab40 2024-09-12T03:18:25.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.553+0000 7ff5e17fa640 1 -- 172.21.15.116:0/2129191510 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5e8005500 con 0x559985edab40 2024-09-12T03:18:25.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.553+0000 7ff5e17fa640 1 -- 172.21.15.116:0/2129191510 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff5e800b040 con 0x559985edab40 2024-09-12T03:18:25.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.553+0000 7ff601e97740 10 client.19536 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:18:25.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.553+0000 7ff601e97740 10 client.19536 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:18:25.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.553+0000 7ff601e97740 20 client.19536 populate_metadata read hostname 'smithi116' 2024-09-12T03:18:25.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.553+0000 7ff601e97740 1 --2- 172.21.15.116:0/2129191510 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x559985f636d0 0x559985f65af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:25.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.553+0000 7ff601e97740 4 client.19536 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:18:25.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.553+0000 7ff601e97740 1 -- 172.21.15.116:0/2129191510 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559985e46d70 con 0x559985f636d0 2024-09-12T03:18:25.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.554+0000 7ff5fd059640 1 --2- 172.21.15.116:0/2129191510 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x559985f636d0 0x559985f65af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:25.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.554+0000 7ff5fd059640 1 --2- 172.21.15.116:0/2129191510 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x559985f636d0 0x559985f65af0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=65c3760defeafaad server_cookie=906d33c5211125a2 in_seq=0 out_seq=0 2024-09-12T03:18:25.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.554+0000 7ff5e17fa640 10 client.19536 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:18:25.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.554+0000 7ff5e17fa640 1 -- 172.21.15.116:0/2129191510 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559985e46d70 con 0x559985f636d0 2024-09-12T03:18:25.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.554+0000 7ff5e17fa640 10 client.19536 handle_command_reply: tid=0 2024-09-12T03:18:25.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.554+0000 7ff601e97740 1 client.19536 shutdown 2024-09-12T03:18:25.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.554+0000 7ff601e97740 1 -- 172.21.15.116:0/2129191510 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559985edab40 msgr2=0x559985ede680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:25.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.554+0000 7ff601e97740 1 --2- 172.21.15.116:0/2129191510 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559985edab40 0x559985ede680 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8000c00 tx=0x7ff5e8004930 comp rx=0 tx=0).stop 2024-09-12T03:18:25.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.554+0000 7ff601e97740 1 -- 172.21.15.116:0/2129191510 shutdown_connections 2024-09-12T03:18:25.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.554+0000 7ff601e97740 1 --2- 172.21.15.116:0/2129191510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559985ee4370 0x559985edebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.554+0000 7ff601e97740 1 --2- 172.21.15.116:0/2129191510 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x559985f636d0 0x559985f65af0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.554+0000 7ff601e97740 1 --2- 172.21.15.116:0/2129191510 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559985edab40 0x559985ede680 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.554+0000 7ff601e97740 1 --2- 172.21.15.116:0/2129191510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559985ed95d0 0x559985ede140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.554+0000 7ff5e17fa640 0 client.19536 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:18:25.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.554+0000 7ff601e97740 1 -- 172.21.15.116:0/2129191510 >> 172.21.15.116:0/2129191510 conn(0x559985ea2310 msgr2=0x559985edafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:25.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.555+0000 7ff601e97740 1 -- 172.21.15.116:0/2129191510 shutdown_connections 2024-09-12T03:18:25.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.555+0000 7ff601e97740 1 -- 172.21.15.116:0/2129191510 wait complete. 2024-09-12T03:18:25.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.555+0000 7ff601e97740 20 client.19536 trim_cache size 0 max 16384 2024-09-12T03:18:25.573 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:18:25.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.555+0000 7ff5ffabe640 1 -- 172.21.15.116:0/995630107 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff5e40a2c00 msgr2=0x7ff5e40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:25.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.555+0000 7ff5ffabe640 1 --2- 172.21.15.116:0/995630107 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff5e40a2c00 0x7ff5e40a50c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff5f00b8f80 tx=0x7ff5f00876f0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.555+0000 7ff5ffabe640 1 -- 172.21.15.116:0/995630107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5f81055b0 msgr2=0x7ff5f819cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:25.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.555+0000 7ff5ffabe640 1 --2- 172.21.15.116:0/995630107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5f81055b0 0x7ff5f819cd20 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0031a20 tx=0x7ff5f0031a50 comp rx=0 tx=0).stop 2024-09-12T03:18:25.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.555+0000 7ff5ffabe640 1 -- 172.21.15.116:0/995630107 shutdown_connections 2024-09-12T03:18:25.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.555+0000 7ff5ffabe640 1 --2- 172.21.15.116:0/995630107 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7ff5e40a2c00 0x7ff5e40a50c0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.555+0000 7ff5ffabe640 1 --2- 172.21.15.116:0/995630107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5f81055b0 0x7ff5f819cd20 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.555+0000 7ff5ffabe640 1 --2- 172.21.15.116:0/995630107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff5f8100e30 0x7ff5f8198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.555+0000 7ff5ffabe640 1 --2- 172.21.15.116:0/995630107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5f8100480 0x7ff5f8198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:25.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.555+0000 7ff5ffabe640 1 -- 172.21.15.116:0/995630107 >> 172.21.15.116:0/995630107 conn(0x7ff5f8077210 msgr2=0x7ff5f8103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:25.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.555+0000 7ff5ffabe640 1 -- 172.21.15.116:0/995630107 shutdown_connections 2024-09-12T03:18:25.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:25.555+0000 7ff5ffabe640 1 -- 172.21.15.116:0/995630107 wait complete. 2024-09-12T03:18:25.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:25 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:25.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:25 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:18:25.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:25 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:18:25.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:25 smithi177 ceph-mon[35305]: pgmap v801: 97 pgs: 97 active+clean; 7.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:18:25.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:25 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:25.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:25 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:18:25.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:25 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:18:25.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:25 smithi116 ceph-mon[31410]: pgmap v801: 97 pgs: 97 active+clean; 7.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:18:25.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:25 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:25.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:25 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:18:25.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:25 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:18:25.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:25 smithi183 ceph-mon[35535]: pgmap v801: 97 pgs: 97 active+clean; 7.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:18:26.257 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:18:26.505 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:27.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.990+0000 7fea32f8b640 1 -- 172.21.15.116:0/396973258 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea2c103660 msgr2=0x7fea2c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:27.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.990+0000 7fea32f8b640 1 --2- 172.21.15.116:0/396973258 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea2c103660 0x7fea2c10bb50 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fea18009900 tx=0x7fea1802a480 comp rx=0 tx=0).stop 2024-09-12T03:18:27.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.990+0000 7fea32f8b640 1 -- 172.21.15.116:0/396973258 shutdown_connections 2024-09-12T03:18:27.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.990+0000 7fea32f8b640 1 --2- 172.21.15.116:0/396973258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea2c10c170 0x7fea2c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.990+0000 7fea32f8b640 1 --2- 172.21.15.116:0/396973258 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea2c103660 0x7fea2c10bb50 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.990+0000 7fea32f8b640 1 --2- 172.21.15.116:0/396973258 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea2c102d40 0x7fea2c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.990+0000 7fea32f8b640 1 -- 172.21.15.116:0/396973258 >> 172.21.15.116:0/396973258 conn(0x7fea2c0fc7a0 msgr2=0x7fea2c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:27.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.990+0000 7fea32f8b640 1 -- 172.21.15.116:0/396973258 shutdown_connections 2024-09-12T03:18:27.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.990+0000 7fea32f8b640 1 -- 172.21.15.116:0/396973258 wait complete. 2024-09-12T03:18:27.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.991+0000 7fea32f8b640 1 Processor -- start 2024-09-12T03:18:27.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.991+0000 7fea32f8b640 1 -- start start 2024-09-12T03:18:27.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.991+0000 7fea32f8b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea2c102d40 0x7fea2c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:27.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.991+0000 7fea32f8b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea2c103660 0x7fea2c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:27.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.991+0000 7fea32f8b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea2c10c170 0x7fea2c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:27.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.991+0000 7fea32f8b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c111f70 con 0x7fea2c102d40 2024-09-12T03:18:27.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.991+0000 7fea32f8b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c111df0 con 0x7fea2c10c170 2024-09-12T03:18:27.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.991+0000 7fea32f8b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1120f0 con 0x7fea2c103660 2024-09-12T03:18:27.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.991+0000 7fea30d27640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea2c102d40 0x7fea2c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:27.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.991+0000 7fea30d27640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea2c102d40 0x7fea2c198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60540/0 (socket says 172.21.15.116:60540) 2024-09-12T03:18:27.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.991+0000 7fea30d27640 1 -- 172.21.15.116:0/421471957 learned_addr learned my addr 172.21.15.116:0/421471957 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:27.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.991+0000 7fea31528640 1 --2- 172.21.15.116:0/421471957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea2c10c170 0x7fea2c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:27.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.991+0000 7fea2bfff640 1 --2- 172.21.15.116:0/421471957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea2c103660 0x7fea2c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:27.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.992+0000 7fea30d27640 1 -- 172.21.15.116:0/421471957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea2c103660 msgr2=0x7fea2c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:27.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.992+0000 7fea30d27640 1 --2- 172.21.15.116:0/421471957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea2c103660 0x7fea2c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.992+0000 7fea30d27640 1 -- 172.21.15.116:0/421471957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea2c10c170 msgr2=0x7fea2c19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:27.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.992+0000 7fea30d27640 1 --2- 172.21.15.116:0/421471957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea2c10c170 0x7fea2c19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.992+0000 7fea30d27640 1 -- 172.21.15.116:0/421471957 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea2c19d420 con 0x7fea2c102d40 2024-09-12T03:18:27.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.992+0000 7fea31528640 1 --2- 172.21.15.116:0/421471957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea2c10c170 0x7fea2c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:27.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.992+0000 7fea30d27640 1 --2- 172.21.15.116:0/421471957 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea2c102d40 0x7fea2c198470 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fea2000d950 tx=0x7fea2000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:27.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.992+0000 7fea29ffb640 1 -- 172.21.15.116:0/421471957 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20017070 con 0x7fea2c102d40 2024-09-12T03:18:27.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.992+0000 7fea32f8b640 1 -- 172.21.15.116:0/421471957 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea2c19d710 con 0x7fea2c102d40 2024-09-12T03:18:27.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.992+0000 7fea29ffb640 1 -- 172.21.15.116:0/421471957 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea20009c30 con 0x7fea2c102d40 2024-09-12T03:18:27.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.992+0000 7fea29ffb640 1 -- 172.21.15.116:0/421471957 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2000f6b0 con 0x7fea2c102d40 2024-09-12T03:18:27.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.992+0000 7fea32f8b640 1 -- 172.21.15.116:0/421471957 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea2c1a4ff0 con 0x7fea2c102d40 2024-09-12T03:18:27.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.994+0000 7fea29ffb640 1 -- 172.21.15.116:0/421471957 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fea200109d0 con 0x7fea2c102d40 2024-09-12T03:18:27.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.995+0000 7fea35364740 1 Processor -- start 2024-09-12T03:18:27.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.995+0000 7fea29ffb640 1 --2- 172.21.15.116:0/421471957 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fea1c0a2b80 0x7fea1c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:27.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.995+0000 7fea35364740 1 -- start start 2024-09-12T03:18:27.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.995+0000 7fea31528640 1 --2- 172.21.15.116:0/421471957 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fea1c0a2b80 0x7fea1c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:27.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.995+0000 7fea35364740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d9ab60b40 0x562d9ab5f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:27.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.995+0000 7fea35364740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9ab5f5d0 0x562d9ab5fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:27.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.995+0000 7fea29ffb640 1 -- 172.21.15.116:0/421471957 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fea200cecd0 con 0x7fea2c102d40 2024-09-12T03:18:27.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.995+0000 7fea35364740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d9ab6a370 0x562d9ab6c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:27.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.995+0000 7fea35364740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562d9aac52f0 con 0x562d9ab6a370 2024-09-12T03:18:27.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.995+0000 7fea35364740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562d9aa6ccf0 con 0x562d9ab60b40 2024-09-12T03:18:27.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.995+0000 7fea35364740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562d9aac12b0 con 0x562d9ab5f5d0 2024-09-12T03:18:27.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.995+0000 7fea31528640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d9ab6a370 0x562d9ab6c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:27.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.995+0000 7fea2bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9ab5f5d0 0x562d9ab5fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:27.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.995+0000 7fea2bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9ab5f5d0 0x562d9ab5fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.116:57960/0 (socket says 172.21.15.116:57960) 2024-09-12T03:18:27.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.995+0000 7fea2bfff640 1 -- 172.21.15.116:0/308992132 learned_addr learned my addr 172.21.15.116:0/308992132 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:27.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.995+0000 7fea31528640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d9ab6a370 0x562d9ab6c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60554/0 (socket says 172.21.15.116:60554) 2024-09-12T03:18:27.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.995+0000 7fea30d27640 1 --2- 172.21.15.116:0/308992132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d9ab60b40 0x562d9ab5f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:27.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.995+0000 7fea31528640 1 --2- 172.21.15.116:0/421471957 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fea1c0a2b80 0x7fea1c0a5040 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fea240092f0 tx=0x7fea24014b10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:27.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.995+0000 7fea31528640 1 -- 172.21.15.116:0/308992132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9ab5f5d0 msgr2=0x562d9ab5fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:27.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.995+0000 7fea31528640 1 --2- 172.21.15.116:0/308992132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9ab5f5d0 0x562d9ab5fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.996+0000 7fea31528640 1 -- 172.21.15.116:0/308992132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d9ab60b40 msgr2=0x562d9ab5f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:27.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.996+0000 7fea31528640 1 --2- 172.21.15.116:0/308992132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d9ab60b40 0x562d9ab5f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.996+0000 7fea31528640 1 -- 172.21.15.116:0/308992132 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d9aa513a0 con 0x562d9ab6a370 2024-09-12T03:18:27.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.996+0000 7fea2bfff640 1 --2- 172.21.15.116:0/308992132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9ab5f5d0 0x562d9ab5fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:18:27.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.996+0000 7fea30d27640 1 --2- 172.21.15.116:0/308992132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d9ab60b40 0x562d9ab5f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:18:27.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.996+0000 7fea31528640 1 --2- 172.21.15.116:0/308992132 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d9ab6a370 0x562d9ab6c760 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fea24005fd0 tx=0x7fea2401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f49bc9ae708150 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:27.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.996+0000 7fea16ffd640 1 -- 172.21.15.116:0/308992132 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2400e9e0 con 0x562d9ab6a370 2024-09-12T03:18:27.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.996+0000 7fea16ffd640 1 -- 172.21.15.116:0/308992132 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea2400eb80 con 0x562d9ab6a370 2024-09-12T03:18:27.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.996+0000 7fea16ffd640 1 -- 172.21.15.116:0/308992132 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2401f630 con 0x562d9ab6a370 2024-09-12T03:18:27.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.997+0000 7fea35364740 1 -- 172.21.15.116:0/308992132 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d9ab6a370 msgr2=0x562d9ab6c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:27.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.997+0000 7fea35364740 1 --2- 172.21.15.116:0/308992132 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d9ab6a370 0x562d9ab6c760 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fea24005fd0 tx=0x7fea2401e000 comp rx=0 tx=0).stop 2024-09-12T03:18:27.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.997+0000 7fea35364740 1 -- 172.21.15.116:0/308992132 shutdown_connections 2024-09-12T03:18:27.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.997+0000 7fea35364740 1 --2- 172.21.15.116:0/308992132 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d9ab6a370 0x562d9ab6c760 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.997+0000 7fea35364740 1 --2- 172.21.15.116:0/308992132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9ab5f5d0 0x562d9ab5fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.997+0000 7fea35364740 1 --2- 172.21.15.116:0/308992132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d9ab60b40 0x562d9ab5f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.997+0000 7fea35364740 1 -- 172.21.15.116:0/308992132 >> 172.21.15.116:0/308992132 conn(0x562d9ab28310 msgr2=0x562d9ab5bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:27.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.997+0000 7fea35364740 1 -- 172.21.15.116:0/308992132 shutdown_connections 2024-09-12T03:18:27.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.997+0000 7fea35364740 1 -- 172.21.15.116:0/308992132 wait complete. 2024-09-12T03:18:27.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.997+0000 7fea35364740 1 -- start start 2024-09-12T03:18:27.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.997+0000 7fea35364740 1 Processor -- start 2024-09-12T03:18:27.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.997+0000 7fea35364740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d9ab5f5d0 0x562d9ab69cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:27.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.997+0000 7fea35364740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d9ab60b40 0x562d9ab66f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:27.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.997+0000 7fea35364740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9ab6a370 0x562d9ab674d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:27.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.997+0000 7fea35364740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562d9ab6d010 con 0x562d9ab60b40 2024-09-12T03:18:27.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.997+0000 7fea35364740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562d9aa80060 con 0x562d9ab5f5d0 2024-09-12T03:18:27.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.997+0000 7fea35364740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562d9ab6d190 con 0x562d9ab6a370 2024-09-12T03:18:27.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.998+0000 7fea35364740 10 client.0 _ll_register_callbacks cb 0x562d9a7d38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:18:27.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.998+0000 7fea2bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d9ab60b40 0x562d9ab66f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:27.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.998+0000 7fea2bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d9ab60b40 0x562d9ab66f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60560/0 (socket says 172.21.15.116:60560) 2024-09-12T03:18:27.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.998+0000 7fea2bfff640 1 -- 172.21.15.116:0/1455056812 learned_addr learned my addr 172.21.15.116:0/1455056812 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:27.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.998+0000 7fea30d27640 1 --2- 172.21.15.116:0/1455056812 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d9ab5f5d0 0x562d9ab69cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:27.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.998+0000 7fea31528640 1 --2- 172.21.15.116:0/1455056812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9ab6a370 0x562d9ab674d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:27.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.998+0000 7fea2bfff640 1 -- 172.21.15.116:0/1455056812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9ab6a370 msgr2=0x562d9ab674d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:27.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.998+0000 7fea2bfff640 1 --2- 172.21.15.116:0/1455056812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9ab6a370 0x562d9ab674d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.998+0000 7fea2bfff640 1 -- 172.21.15.116:0/1455056812 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d9ab5f5d0 msgr2=0x562d9ab69cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:27.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.998+0000 7fea2bfff640 1 --2- 172.21.15.116:0/1455056812 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d9ab5f5d0 0x562d9ab69cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.998+0000 7fea2bfff640 1 -- 172.21.15.116:0/1455056812 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d9a8363c0 con 0x562d9ab60b40 2024-09-12T03:18:27.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.998+0000 7fea2bfff640 1 --2- 172.21.15.116:0/1455056812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d9ab60b40 0x562d9ab66f70 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fea18006010 tx=0x7fea180317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d8876667e403ff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:27.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.998+0000 7fea0bfff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:18:27.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.998+0000 7fea0bfff640 1 -- 172.21.15.116:0/1455056812 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18035040 con 0x562d9ab60b40 2024-09-12T03:18:27.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.998+0000 7fea0bfff640 1 -- 172.21.15.116:0/1455056812 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fea18004040 con 0x562d9ab60b40 2024-09-12T03:18:27.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.998+0000 7fea35364740 1 -- 172.21.15.116:0/1455056812 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea2400eb80 con 0x562d9ab60b40 2024-09-12T03:18:27.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.998+0000 7fea0bfff640 1 -- 172.21.15.116:0/1455056812 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fea18004240 con 0x562d9ab60b40 2024-09-12T03:18:27.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.998+0000 7fea35364740 10 client.19548 fetch_fsmap learned FSMap version 25 2024-09-12T03:18:27.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.998+0000 7fea35364740 1 -- 172.21.15.116:0/1455056812 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562d9ab78aa0 con 0x562d9ab60b40 2024-09-12T03:18:27.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.999+0000 7fea0bfff640 1 -- 172.21.15.116:0/1455056812 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea2400eb80 con 0x562d9ab60b40 2024-09-12T03:18:27.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.999+0000 7fea0bfff640 1 -- 172.21.15.116:0/1455056812 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fea18004b10 con 0x562d9ab60b40 2024-09-12T03:18:27.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.999+0000 7fea35364740 10 client.19548 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:18:27.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.999+0000 7fea35364740 10 client.19548 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:18:27.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.999+0000 7fea35364740 20 client.19548 populate_metadata read hostname 'smithi116' 2024-09-12T03:18:27.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.999+0000 7fea35364740 1 --2- 172.21.15.116:0/1455056812 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x562d9ab79f60 0x562d9ab7c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:27.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.999+0000 7fea35364740 4 client.19548 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:18:27.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.999+0000 7fea35364740 1 -- 172.21.15.116:0/1455056812 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d9aad2860 con 0x562d9ab79f60 2024-09-12T03:18:27.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.999+0000 7fea30d27640 1 --2- 172.21.15.116:0/1455056812 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x562d9ab79f60 0x562d9ab7c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:27.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.999+0000 7fea0bfff640 10 client.19548 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:18:27.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:26.999+0000 7fea30d27640 1 --2- 172.21.15.116:0/1455056812 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x562d9ab79f60 0x562d9ab7c3c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=96adba08746c860f server_cookie=c92dbd258f063c30 in_seq=0 out_seq=0 2024-09-12T03:18:27.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.000+0000 7fea0bfff640 1 -- 172.21.15.116:0/1455056812 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562d9aad2860 con 0x562d9ab79f60 2024-09-12T03:18:27.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.000+0000 7fea0bfff640 10 client.19548 handle_command_reply: tid=0 2024-09-12T03:18:27.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.000+0000 7fea35364740 1 client.19548 shutdown 2024-09-12T03:18:27.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.000+0000 7fea35364740 1 -- 172.21.15.116:0/1455056812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d9ab60b40 msgr2=0x562d9ab66f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:27.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.000+0000 7fea35364740 1 --2- 172.21.15.116:0/1455056812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d9ab60b40 0x562d9ab66f70 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fea18006010 tx=0x7fea180317e0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.000+0000 7fea35364740 1 -- 172.21.15.116:0/1455056812 shutdown_connections 2024-09-12T03:18:27.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.000+0000 7fea35364740 1 --2- 172.21.15.116:0/1455056812 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x562d9ab79f60 0x562d9ab7c3c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.000+0000 7fea35364740 1 --2- 172.21.15.116:0/1455056812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9ab6a370 0x562d9ab674d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.000+0000 7fea35364740 1 --2- 172.21.15.116:0/1455056812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d9ab60b40 0x562d9ab66f70 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.000+0000 7fea0bfff640 0 client.19548 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:18:27.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.000+0000 7fea35364740 1 --2- 172.21.15.116:0/1455056812 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d9ab5f5d0 0x562d9ab69cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.000+0000 7fea35364740 1 -- 172.21.15.116:0/1455056812 >> 172.21.15.116:0/1455056812 conn(0x562d9ab28310 msgr2=0x562d9ab61960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:27.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.000+0000 7fea35364740 1 -- 172.21.15.116:0/1455056812 shutdown_connections 2024-09-12T03:18:27.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.000+0000 7fea35364740 1 -- 172.21.15.116:0/1455056812 wait complete. 2024-09-12T03:18:27.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.001+0000 7fea35364740 20 client.19548 trim_cache size 0 max 16384 2024-09-12T03:18:27.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.013+0000 7fea35364740 1 Processor -- start 2024-09-12T03:18:27.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.013+0000 7fea35364740 1 -- start start 2024-09-12T03:18:27.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.013+0000 7fea35364740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9ab5f5d0 0x562d9ab65840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:27.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.013+0000 7fea35364740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d9ab60b40 0x562d9ab72830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:27.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.013+0000 7fea35364740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d9ab6a370 0x562d9ab72d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:27.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.013+0000 7fea35364740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1afdf0 con 0x562d9ab60b40 2024-09-12T03:18:27.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.013+0000 7fea35364740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1ad6c0 con 0x562d9ab6a370 2024-09-12T03:18:27.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.013+0000 7fea35364740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1ab510 con 0x562d9ab5f5d0 2024-09-12T03:18:27.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.014+0000 7fea2bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d9ab60b40 0x562d9ab72830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:27.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.014+0000 7fea2bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d9ab60b40 0x562d9ab72830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60570/0 (socket says 172.21.15.116:60570) 2024-09-12T03:18:27.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.014+0000 7fea2bfff640 1 -- 172.21.15.116:0/3671565598 learned_addr learned my addr 172.21.15.116:0/3671565598 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:27.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.014+0000 7fea30d27640 1 --2- 172.21.15.116:0/3671565598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9ab5f5d0 0x562d9ab65840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:27.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.014+0000 7fea2bfff640 1 -- 172.21.15.116:0/3671565598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9ab5f5d0 msgr2=0x562d9ab65840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:27.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.014+0000 7fea2bfff640 1 --2- 172.21.15.116:0/3671565598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9ab5f5d0 0x562d9ab65840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.014+0000 7fea2bfff640 1 -- 172.21.15.116:0/3671565598 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d9ab6a370 msgr2=0x562d9ab72d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:27.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.014+0000 7fea2bfff640 1 --2- 172.21.15.116:0/3671565598 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d9ab6a370 0x562d9ab72d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.014+0000 7fea2bfff640 1 -- 172.21.15.116:0/3671565598 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d9ab7ca40 con 0x562d9ab60b40 2024-09-12T03:18:27.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.014+0000 7fea2bfff640 1 --2- 172.21.15.116:0/3671565598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d9ab60b40 0x562d9ab72830 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fea18009900 tx=0x7fea18005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a0b35f7c3dce4a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:27.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.014+0000 7fea167fc640 1 -- 172.21.15.116:0/3671565598 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18035030 con 0x562d9ab60b40 2024-09-12T03:18:27.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.014+0000 7fea167fc640 1 -- 172.21.15.116:0/3671565598 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea18003960 con 0x562d9ab60b40 2024-09-12T03:18:27.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.014+0000 7fea167fc640 1 -- 172.21.15.116:0/3671565598 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18004920 con 0x562d9ab60b40 2024-09-12T03:18:27.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.015+0000 7fea35364740 1 -- 172.21.15.116:0/3671565598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d9ab60b40 msgr2=0x562d9ab72830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:27.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.015+0000 7fea35364740 1 --2- 172.21.15.116:0/3671565598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d9ab60b40 0x562d9ab72830 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fea18009900 tx=0x7fea18005b50 comp rx=0 tx=0).stop 2024-09-12T03:18:27.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.015+0000 7fea35364740 1 -- 172.21.15.116:0/3671565598 shutdown_connections 2024-09-12T03:18:27.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.015+0000 7fea35364740 1 --2- 172.21.15.116:0/3671565598 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d9ab6a370 0x562d9ab72d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.015+0000 7fea35364740 1 --2- 172.21.15.116:0/3671565598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d9ab60b40 0x562d9ab72830 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.015+0000 7fea35364740 1 --2- 172.21.15.116:0/3671565598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9ab5f5d0 0x562d9ab65840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.015+0000 7fea35364740 1 -- 172.21.15.116:0/3671565598 >> 172.21.15.116:0/3671565598 conn(0x562d9ab28310 msgr2=0x562d9ab68ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:27.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.015+0000 7fea35364740 1 -- 172.21.15.116:0/3671565598 shutdown_connections 2024-09-12T03:18:27.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.015+0000 7fea35364740 1 -- 172.21.15.116:0/3671565598 wait complete. 2024-09-12T03:18:27.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.015+0000 7fea35364740 1 -- start start 2024-09-12T03:18:27.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.015+0000 7fea35364740 1 Processor -- start 2024-09-12T03:18:27.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.015+0000 7fea35364740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d9ab5f5d0 0x562d9abdce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:27.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.015+0000 7fea35364740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9ab60b40 0x562d9abdd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:27.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.015+0000 7fea35364740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d9ab6a370 0x562d9abe7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:27.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.016+0000 7fea35364740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1abb80 con 0x562d9ab5f5d0 2024-09-12T03:18:27.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.016+0000 7fea35364740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1acff0 con 0x562d9ab6a370 2024-09-12T03:18:27.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.016+0000 7fea35364740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1ac250 con 0x562d9ab60b40 2024-09-12T03:18:27.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.016+0000 7fea35364740 10 client.0 _ll_register_callbacks cb 0x7fea18009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:18:27.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.016+0000 7fea30d27640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d9ab5f5d0 0x562d9abdce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:27.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.016+0000 7fea30d27640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d9ab5f5d0 0x562d9abdce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60580/0 (socket says 172.21.15.116:60580) 2024-09-12T03:18:27.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.016+0000 7fea30d27640 1 -- 172.21.15.116:0/650494308 learned_addr learned my addr 172.21.15.116:0/650494308 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:27.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.016+0000 7fea2bfff640 1 --2- 172.21.15.116:0/650494308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9ab60b40 0x562d9abdd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:27.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.016+0000 7fea31528640 1 --2- 172.21.15.116:0/650494308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d9ab6a370 0x562d9abe7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:27.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.016+0000 7fea30d27640 1 -- 172.21.15.116:0/650494308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9ab60b40 msgr2=0x562d9abdd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:27.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.016+0000 7fea30d27640 1 --2- 172.21.15.116:0/650494308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9ab60b40 0x562d9abdd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.016+0000 7fea30d27640 1 -- 172.21.15.116:0/650494308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d9ab6a370 msgr2=0x562d9abe7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:27.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.016+0000 7fea30d27640 1 --2- 172.21.15.116:0/650494308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d9ab6a370 0x562d9abe7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.016+0000 7fea30d27640 1 -- 172.21.15.116:0/650494308 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d9abe8260 con 0x562d9ab5f5d0 2024-09-12T03:18:27.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.016+0000 7fea31528640 1 --2- 172.21.15.116:0/650494308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d9ab6a370 0x562d9abe7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:18:27.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.016+0000 7fea30d27640 1 --2- 172.21.15.116:0/650494308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d9ab5f5d0 0x562d9abdce00 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fea20097f30 tx=0x7fea2000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756da3bf8c7e148d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:27.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.016+0000 7fea0bfff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:18:27.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.016+0000 7fea0bfff640 1 -- 172.21.15.116:0/650494308 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea200b9060 con 0x562d9ab5f5d0 2024-09-12T03:18:27.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.016+0000 7fea0bfff640 1 -- 172.21.15.116:0/650494308 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fea20097bd0 con 0x562d9ab5f5d0 2024-09-12T03:18:27.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.016+0000 7fea35364740 1 -- 172.21.15.116:0/650494308 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea18003960 con 0x562d9ab5f5d0 2024-09-12T03:18:27.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.016+0000 7fea0bfff640 1 -- 172.21.15.116:0/650494308 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fea200b4070 con 0x562d9ab5f5d0 2024-09-12T03:18:27.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.016+0000 7fea35364740 10 client.19554 fetch_fsmap learned FSMap version 25 2024-09-12T03:18:27.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.016+0000 7fea35364740 1 -- 172.21.15.116:0/650494308 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562d9abe9560 con 0x562d9ab5f5d0 2024-09-12T03:18:27.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.017+0000 7fea0bfff640 1 -- 172.21.15.116:0/650494308 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea18003960 con 0x562d9ab5f5d0 2024-09-12T03:18:27.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.017+0000 7fea0bfff640 1 -- 172.21.15.116:0/650494308 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fea20084af0 con 0x562d9ab5f5d0 2024-09-12T03:18:27.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.017+0000 7fea35364740 10 client.19554 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:18:27.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.017+0000 7fea35364740 10 client.19554 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:18:27.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.017+0000 7fea35364740 20 client.19554 populate_metadata read hostname 'smithi116' 2024-09-12T03:18:27.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.017+0000 7fea35364740 1 --2- 172.21.15.116:0/650494308 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x562d9abe9c30 0x562d9abec050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:27.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.017+0000 7fea35364740 4 client.19554 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:18:27.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.017+0000 7fea35364740 1 -- 172.21.15.116:0/650494308 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562d9aaccd70 con 0x562d9abe9c30 2024-09-12T03:18:27.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.017+0000 7fea2bfff640 1 --2- 172.21.15.116:0/650494308 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x562d9abe9c30 0x562d9abec050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:27.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.017+0000 7fea2bfff640 1 --2- 172.21.15.116:0/650494308 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x562d9abe9c30 0x562d9abec050 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b58456ff30c8abbd server_cookie=844263f234580d29 in_seq=0 out_seq=0 2024-09-12T03:18:27.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.017+0000 7fea0bfff640 10 client.19554 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:18:27.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.018+0000 7fea0bfff640 1 -- 172.21.15.116:0/650494308 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562d9aaccd70 con 0x562d9abe9c30 2024-09-12T03:18:27.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.018+0000 7fea0bfff640 10 client.19554 handle_command_reply: tid=0 2024-09-12T03:18:27.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.018+0000 7fea35364740 1 client.19554 shutdown 2024-09-12T03:18:27.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.018+0000 7fea35364740 1 -- 172.21.15.116:0/650494308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d9ab5f5d0 msgr2=0x562d9abdce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:27.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.018+0000 7fea35364740 1 --2- 172.21.15.116:0/650494308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d9ab5f5d0 0x562d9abdce00 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fea20097f30 tx=0x7fea2000bf60 comp rx=0 tx=0).stop 2024-09-12T03:18:27.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.018+0000 7fea35364740 1 -- 172.21.15.116:0/650494308 shutdown_connections 2024-09-12T03:18:27.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.018+0000 7fea35364740 1 --2- 172.21.15.116:0/650494308 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x562d9abe9c30 0x562d9abec050 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.018+0000 7fea35364740 1 --2- 172.21.15.116:0/650494308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562d9ab6a370 0x562d9abe7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.018+0000 7fea35364740 1 --2- 172.21.15.116:0/650494308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d9ab60b40 0x562d9abdd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.018+0000 7fea35364740 1 --2- 172.21.15.116:0/650494308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562d9ab5f5d0 0x562d9abdce00 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.018+0000 7fea35364740 1 -- 172.21.15.116:0/650494308 >> 172.21.15.116:0/650494308 conn(0x562d9ab28310 msgr2=0x562d9ab687d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:27.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.018+0000 7fea0bfff640 0 client.19554 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:18:27.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.018+0000 7fea35364740 1 -- 172.21.15.116:0/650494308 shutdown_connections 2024-09-12T03:18:27.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.018+0000 7fea35364740 1 -- 172.21.15.116:0/650494308 wait complete. 2024-09-12T03:18:27.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.018+0000 7fea35364740 20 client.19554 trim_cache size 0 max 16384 2024-09-12T03:18:27.035 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:18:27.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.019+0000 7fea32f8b640 1 -- 172.21.15.116:0/421471957 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fea1c0a2b80 msgr2=0x7fea1c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:27.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.019+0000 7fea32f8b640 1 --2- 172.21.15.116:0/421471957 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fea1c0a2b80 0x7fea1c0a5040 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fea240092f0 tx=0x7fea24014b10 comp rx=0 tx=0).stop 2024-09-12T03:18:27.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.019+0000 7fea32f8b640 1 -- 172.21.15.116:0/421471957 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea2c102d40 msgr2=0x7fea2c198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:27.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.019+0000 7fea32f8b640 1 --2- 172.21.15.116:0/421471957 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea2c102d40 0x7fea2c198470 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fea2000d950 tx=0x7fea2000de10 comp rx=0 tx=0).stop 2024-09-12T03:18:27.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.019+0000 7fea32f8b640 1 -- 172.21.15.116:0/421471957 shutdown_connections 2024-09-12T03:18:27.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.019+0000 7fea32f8b640 1 --2- 172.21.15.116:0/421471957 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fea1c0a2b80 0x7fea1c0a5040 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.019+0000 7fea32f8b640 1 --2- 172.21.15.116:0/421471957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea2c10c170 0x7fea2c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.019+0000 7fea32f8b640 1 --2- 172.21.15.116:0/421471957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea2c103660 0x7fea2c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.019+0000 7fea32f8b640 1 --2- 172.21.15.116:0/421471957 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea2c102d40 0x7fea2c198470 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.019+0000 7fea32f8b640 1 -- 172.21.15.116:0/421471957 >> 172.21.15.116:0/421471957 conn(0x7fea2c0fc7a0 msgr2=0x7fea2c0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:27.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.019+0000 7fea32f8b640 1 -- 172.21.15.116:0/421471957 shutdown_connections 2024-09-12T03:18:27.037 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:27.019+0000 7fea32f8b640 1 -- 172.21.15.116:0/421471957 wait complete. 2024-09-12T03:18:27.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:27 smithi177 ceph-mon[35305]: pgmap v802: 97 pgs: 97 active+clean; 7.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:18:27.773 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:27 smithi116 ceph-mon[31410]: pgmap v802: 97 pgs: 97 active+clean; 7.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:18:27.800 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:18:27.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:27 smithi183 ceph-mon[35535]: pgmap v802: 97 pgs: 97 active+clean; 7.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:18:28.042 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:28.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.532+0000 7fc0133b2640 1 -- 172.21.15.116:0/4119946221 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc00c106f10 msgr2=0x7fc00c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:28.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.532+0000 7fc0133b2640 1 --2- 172.21.15.116:0/4119946221 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc00c106f10 0x7fc00c10ea90 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fc004009ca0 tx=0x7fc00402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.532+0000 7fc0133b2640 1 -- 172.21.15.116:0/4119946221 shutdown_connections 2024-09-12T03:18:28.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.532+0000 7fc0133b2640 1 --2- 172.21.15.116:0/4119946221 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc00c106f10 0x7fc00c10ea90 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.532+0000 7fc0133b2640 1 --2- 172.21.15.116:0/4119946221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc00c102790 0x7fc00c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.532+0000 7fc0133b2640 1 --2- 172.21.15.116:0/4119946221 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc00c101de0 0x7fc00c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.532+0000 7fc0133b2640 1 -- 172.21.15.116:0/4119946221 >> 172.21.15.116:0/4119946221 conn(0x7fc00c0fd8c0 msgr2=0x7fc00c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:28.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.532+0000 7fc0133b2640 1 -- 172.21.15.116:0/4119946221 shutdown_connections 2024-09-12T03:18:28.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.532+0000 7fc0133b2640 1 -- 172.21.15.116:0/4119946221 wait complete. 2024-09-12T03:18:28.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.533+0000 7fc0133b2640 1 Processor -- start 2024-09-12T03:18:28.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.533+0000 7fc0133b2640 1 -- start start 2024-09-12T03:18:28.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.533+0000 7fc0133b2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc00c101de0 0x7fc00c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:28.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.533+0000 7fc0133b2640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc00c102790 0x7fc00c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:28.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.533+0000 7fc0133b2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc00c106f10 0x7fc00c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:28.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.533+0000 7fc0133b2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c111f90 con 0x7fc00c101de0 2024-09-12T03:18:28.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.533+0000 7fc0133b2640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c111e10 con 0x7fc00c102790 2024-09-12T03:18:28.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.533+0000 7fc0133b2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c112110 con 0x7fc00c106f10 2024-09-12T03:18:28.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.533+0000 7fc01114e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc00c101de0 0x7fc00c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:28.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.533+0000 7fc01114e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc00c101de0 0x7fc00c198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60590/0 (socket says 172.21.15.116:60590) 2024-09-12T03:18:28.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.533+0000 7fc01114e640 1 -- 172.21.15.116:0/978559589 learned_addr learned my addr 172.21.15.116:0/978559589 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:28.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.533+0000 7fc01094d640 1 --2- 172.21.15.116:0/978559589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc00c102790 0x7fc00c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:28.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.534+0000 7fc01194f640 1 --2- 172.21.15.116:0/978559589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc00c106f10 0x7fc00c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:28.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.534+0000 7fc01114e640 1 -- 172.21.15.116:0/978559589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc00c106f10 msgr2=0x7fc00c19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:28.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.534+0000 7fc01114e640 1 --2- 172.21.15.116:0/978559589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc00c106f10 0x7fc00c19cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.534+0000 7fc01114e640 1 -- 172.21.15.116:0/978559589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc00c102790 msgr2=0x7fc00c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:28.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.534+0000 7fc01114e640 1 --2- 172.21.15.116:0/978559589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc00c102790 0x7fc00c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.534+0000 7fc01114e640 1 -- 172.21.15.116:0/978559589 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00c19d420 con 0x7fc00c101de0 2024-09-12T03:18:28.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.534+0000 7fc01094d640 1 --2- 172.21.15.116:0/978559589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc00c102790 0x7fc00c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:28.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.534+0000 7fc01114e640 1 --2- 172.21.15.116:0/978559589 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc00c101de0 0x7fc00c198470 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fc00800d950 tx=0x7fc00800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:28.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.534+0000 7fc0027fc640 1 -- 172.21.15.116:0/978559589 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc008017030 con 0x7fc00c101de0 2024-09-12T03:18:28.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.534+0000 7fc0133b2640 1 -- 172.21.15.116:0/978559589 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc00c19d710 con 0x7fc00c101de0 2024-09-12T03:18:28.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.534+0000 7fc0027fc640 1 -- 172.21.15.116:0/978559589 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc008004020 con 0x7fc00c101de0 2024-09-12T03:18:28.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.534+0000 7fc0027fc640 1 -- 172.21.15.116:0/978559589 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0080047a0 con 0x7fc00c101de0 2024-09-12T03:18:28.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.534+0000 7fc0133b2640 1 -- 172.21.15.116:0/978559589 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc00c1a4ff0 con 0x7fc00c101de0 2024-09-12T03:18:28.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.537+0000 7fc0027fc640 1 -- 172.21.15.116:0/978559589 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fc0080041d0 con 0x7fc00c101de0 2024-09-12T03:18:28.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.537+0000 7fc01578b740 1 Processor -- start 2024-09-12T03:18:28.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.537+0000 7fc01578b740 1 -- start start 2024-09-12T03:18:28.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.537+0000 7fc01578b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1da548b40 0x55a1da547090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:28.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.537+0000 7fc01578b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1da5475d0 0x55a1da547a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:28.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.537+0000 7fc01578b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1da552370 0x55a1da554760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:28.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.537+0000 7fc01578b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a1da4ad2f0 con 0x55a1da552370 2024-09-12T03:18:28.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.537+0000 7fc01578b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a1da454cf0 con 0x55a1da548b40 2024-09-12T03:18:28.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.537+0000 7fc01578b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a1da4a92b0 con 0x55a1da5475d0 2024-09-12T03:18:28.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.537+0000 7fc01094d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1da552370 0x55a1da554760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:28.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.538+0000 7fc01094d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1da552370 0x55a1da554760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60600/0 (socket says 172.21.15.116:60600) 2024-09-12T03:18:28.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.538+0000 7fc01094d640 1 -- 172.21.15.116:0/261662714 learned_addr learned my addr 172.21.15.116:0/261662714 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:28.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.538+0000 7fc01194f640 1 --2- 172.21.15.116:0/261662714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1da548b40 0x55a1da547090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:28.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.538+0000 7fc01114e640 1 --2- 172.21.15.116:0/261662714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1da5475d0 0x55a1da547a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:28.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.538+0000 7fc0027fc640 1 --2- 172.21.15.116:0/978559589 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbff80a2b80 0x7fbff80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:28.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.538+0000 7fc01094d640 1 -- 172.21.15.116:0/261662714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1da5475d0 msgr2=0x55a1da547a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:28.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.538+0000 7fc01094d640 1 --2- 172.21.15.116:0/261662714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1da5475d0 0x55a1da547a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.538+0000 7fc01094d640 1 -- 172.21.15.116:0/261662714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1da548b40 msgr2=0x55a1da547090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:28.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.538+0000 7fc01094d640 1 --2- 172.21.15.116:0/261662714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1da548b40 0x55a1da547090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.538+0000 7fc01094d640 1 -- 172.21.15.116:0/261662714 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1da4393a0 con 0x55a1da552370 2024-09-12T03:18:28.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.538+0000 7fc01114e640 1 --2- 172.21.15.116:0/261662714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1da5475d0 0x55a1da547a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:18:28.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.538+0000 7fc01194f640 1 --2- 172.21.15.116:0/978559589 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbff80a2b80 0x7fbff80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:28.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.538+0000 7fc0027fc640 1 -- 172.21.15.116:0/978559589 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fc0080ceab0 con 0x7fc00c101de0 2024-09-12T03:18:28.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.538+0000 7fc01094d640 1 --2- 172.21.15.116:0/261662714 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1da552370 0x55a1da554760 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fbffc007800 tx=0x7fbffc007cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4761f3d14215bd3b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:28.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.538+0000 7fbff77fe640 1 -- 172.21.15.116:0/261662714 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc00d950 con 0x55a1da552370 2024-09-12T03:18:28.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.538+0000 7fbff77fe640 1 -- 172.21.15.116:0/261662714 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbffc018070 con 0x55a1da552370 2024-09-12T03:18:28.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.538+0000 7fbff77fe640 1 -- 172.21.15.116:0/261662714 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc0123e0 con 0x55a1da552370 2024-09-12T03:18:28.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.539+0000 7fc01578b740 1 -- 172.21.15.116:0/261662714 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1da552370 msgr2=0x55a1da554760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:28.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.539+0000 7fc01578b740 1 --2- 172.21.15.116:0/261662714 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1da552370 0x55a1da554760 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fbffc007800 tx=0x7fbffc007cc0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.539+0000 7fc01194f640 1 --2- 172.21.15.116:0/978559589 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbff80a2b80 0x7fbff80a5040 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc004006010 tx=0x7fc0040095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:28.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.539+0000 7fc01578b740 1 -- 172.21.15.116:0/261662714 shutdown_connections 2024-09-12T03:18:28.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.539+0000 7fc01578b740 1 --2- 172.21.15.116:0/261662714 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1da552370 0x55a1da554760 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.539+0000 7fc01578b740 1 --2- 172.21.15.116:0/261662714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1da5475d0 0x55a1da547a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.539+0000 7fc01578b740 1 --2- 172.21.15.116:0/261662714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1da548b40 0x55a1da547090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.539+0000 7fc01578b740 1 -- 172.21.15.116:0/261662714 >> 172.21.15.116:0/261662714 conn(0x55a1da510310 msgr2=0x55a1da543c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:28.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.539+0000 7fc01578b740 1 -- 172.21.15.116:0/261662714 shutdown_connections 2024-09-12T03:18:28.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.539+0000 7fc01578b740 1 -- 172.21.15.116:0/261662714 wait complete. 2024-09-12T03:18:28.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.539+0000 7fc01578b740 1 -- start start 2024-09-12T03:18:28.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.539+0000 7fc01578b740 1 Processor -- start 2024-09-12T03:18:28.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.540+0000 7fc01578b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1da5475d0 0x55a1da551cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:28.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.540+0000 7fc01578b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1da548b40 0x55a1da54ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:28.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.540+0000 7fc01578b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1da552370 0x55a1da54f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:28.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.540+0000 7fc01578b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a1da555070 con 0x55a1da5475d0 2024-09-12T03:18:28.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.540+0000 7fc01578b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a1da468060 con 0x55a1da548b40 2024-09-12T03:18:28.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.540+0000 7fc01578b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a1da5551f0 con 0x55a1da552370 2024-09-12T03:18:28.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.540+0000 7fc01578b740 10 client.0 _ll_register_callbacks cb 0x55a1da1bb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:18:28.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.540+0000 7fc01114e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1da5475d0 0x55a1da551cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:28.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.540+0000 7fc01114e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1da5475d0 0x55a1da551cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60616/0 (socket says 172.21.15.116:60616) 2024-09-12T03:18:28.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.540+0000 7fc01114e640 1 -- 172.21.15.116:0/3211621169 learned_addr learned my addr 172.21.15.116:0/3211621169 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:28.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.540+0000 7fc01194f640 1 --2- 172.21.15.116:0/3211621169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1da552370 0x55a1da54f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:28.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.540+0000 7fc01094d640 1 --2- 172.21.15.116:0/3211621169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1da548b40 0x55a1da54ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:28.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.540+0000 7fc01114e640 1 -- 172.21.15.116:0/3211621169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1da552370 msgr2=0x55a1da54f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:28.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.540+0000 7fc01114e640 1 --2- 172.21.15.116:0/3211621169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1da552370 0x55a1da54f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.540+0000 7fc01114e640 1 -- 172.21.15.116:0/3211621169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1da548b40 msgr2=0x55a1da54ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:28.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.540+0000 7fc01114e640 1 --2- 172.21.15.116:0/3211621169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1da548b40 0x55a1da54ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.540+0000 7fc01114e640 1 -- 172.21.15.116:0/3211621169 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1da21e3c0 con 0x55a1da5475d0 2024-09-12T03:18:28.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.540+0000 7fc01114e640 1 --2- 172.21.15.116:0/3211621169 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1da5475d0 0x55a1da551cd0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fc008009fd0 tx=0x7fc0080976a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ae366ff63ac80e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:28.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.540+0000 7fbff4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:18:28.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.541+0000 7fbff4ff9640 1 -- 172.21.15.116:0/3211621169 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0080a3d20 con 0x55a1da5475d0 2024-09-12T03:18:28.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.541+0000 7fbff4ff9640 1 -- 172.21.15.116:0/3211621169 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fc00809a790 con 0x55a1da5475d0 2024-09-12T03:18:28.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.541+0000 7fc01578b740 1 -- 172.21.15.116:0/3211621169 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbffc018070 con 0x55a1da5475d0 2024-09-12T03:18:28.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.541+0000 7fbff4ff9640 1 -- 172.21.15.116:0/3211621169 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc00809ba40 con 0x55a1da5475d0 2024-09-12T03:18:28.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.541+0000 7fc01578b740 10 client.19566 fetch_fsmap learned FSMap version 25 2024-09-12T03:18:28.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.541+0000 7fc01578b740 1 -- 172.21.15.116:0/3211621169 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a1da5609f0 con 0x55a1da5475d0 2024-09-12T03:18:28.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.541+0000 7fbff4ff9640 1 -- 172.21.15.116:0/3211621169 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbffc018070 con 0x55a1da5475d0 2024-09-12T03:18:28.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.541+0000 7fbff4ff9640 1 -- 172.21.15.116:0/3211621169 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc00809a200 con 0x55a1da5475d0 2024-09-12T03:18:28.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.541+0000 7fc01578b740 10 client.19566 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:18:28.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.541+0000 7fc01578b740 10 client.19566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:18:28.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.541+0000 7fc01578b740 20 client.19566 populate_metadata read hostname 'smithi116' 2024-09-12T03:18:28.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.541+0000 7fc01578b740 1 --2- 172.21.15.116:0/3211621169 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a1da561e00 0x55a1da5641f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:28.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.541+0000 7fc01578b740 4 client.19566 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:18:28.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.541+0000 7fc01578b740 1 -- 172.21.15.116:0/3211621169 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1da4ba860 con 0x55a1da561e00 2024-09-12T03:18:28.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.541+0000 7fc01094d640 1 --2- 172.21.15.116:0/3211621169 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a1da561e00 0x55a1da5641f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:28.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.542+0000 7fbff4ff9640 10 client.19566 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:18:28.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.542+0000 7fc01094d640 1 --2- 172.21.15.116:0/3211621169 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a1da561e00 0x55a1da5641f0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f39e5721725f2036 server_cookie=ae9948c6688f5d2a in_seq=0 out_seq=0 2024-09-12T03:18:28.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.542+0000 7fbff4ff9640 1 -- 172.21.15.116:0/3211621169 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a1da4ba860 con 0x55a1da561e00 2024-09-12T03:18:28.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.542+0000 7fbff4ff9640 10 client.19566 handle_command_reply: tid=0 2024-09-12T03:18:28.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.543+0000 7fc01578b740 1 client.19566 shutdown 2024-09-12T03:18:28.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.543+0000 7fc01578b740 1 -- 172.21.15.116:0/3211621169 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1da5475d0 msgr2=0x55a1da551cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:28.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.543+0000 7fc01578b740 1 --2- 172.21.15.116:0/3211621169 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1da5475d0 0x55a1da551cd0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fc008009fd0 tx=0x7fc0080976a0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.543+0000 7fc01578b740 1 -- 172.21.15.116:0/3211621169 shutdown_connections 2024-09-12T03:18:28.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.543+0000 7fc01578b740 1 --2- 172.21.15.116:0/3211621169 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a1da561e00 0x55a1da5641f0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.543+0000 7fc01578b740 1 --2- 172.21.15.116:0/3211621169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1da552370 0x55a1da54f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.543+0000 7fc01578b740 1 --2- 172.21.15.116:0/3211621169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1da548b40 0x55a1da54ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.543+0000 7fc01578b740 1 --2- 172.21.15.116:0/3211621169 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1da5475d0 0x55a1da551cd0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.543+0000 7fc01578b740 1 -- 172.21.15.116:0/3211621169 >> 172.21.15.116:0/3211621169 conn(0x55a1da510310 msgr2=0x55a1da549960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:28.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.543+0000 7fbff4ff9640 0 client.19566 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:18:28.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.543+0000 7fc01578b740 1 -- 172.21.15.116:0/3211621169 shutdown_connections 2024-09-12T03:18:28.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.543+0000 7fc01578b740 1 -- 172.21.15.116:0/3211621169 wait complete. 2024-09-12T03:18:28.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.543+0000 7fc01578b740 20 client.19566 trim_cache size 0 max 16384 2024-09-12T03:18:28.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.556+0000 7fc01578b740 1 Processor -- start 2024-09-12T03:18:28.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.556+0000 7fc01578b740 1 -- start start 2024-09-12T03:18:28.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.557+0000 7fc01578b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1da5475d0 0x55a1da543e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:28.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.557+0000 7fc01578b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1da548b40 0x55a1da55e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:28.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.557+0000 7fc01578b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1da552370 0x55a1da54e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:28.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.557+0000 7fc01578b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1afd80 con 0x55a1da5475d0 2024-09-12T03:18:28.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.557+0000 7fc01578b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1ad650 con 0x55a1da552370 2024-09-12T03:18:28.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.557+0000 7fc01578b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1ab4a0 con 0x55a1da548b40 2024-09-12T03:18:28.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.557+0000 7fc01114e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1da5475d0 0x55a1da543e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:28.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.557+0000 7fc01114e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1da5475d0 0x55a1da543e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60624/0 (socket says 172.21.15.116:60624) 2024-09-12T03:18:28.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.557+0000 7fc01114e640 1 -- 172.21.15.116:0/3069537106 learned_addr learned my addr 172.21.15.116:0/3069537106 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:28.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.557+0000 7fc01194f640 1 --2- 172.21.15.116:0/3069537106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1da552370 0x55a1da54e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:28.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.557+0000 7fc01114e640 1 -- 172.21.15.116:0/3069537106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1da548b40 msgr2=0x55a1da55e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:28.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.557+0000 7fc01114e640 1 --2- 172.21.15.116:0/3069537106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1da548b40 0x55a1da55e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.557+0000 7fc01114e640 1 -- 172.21.15.116:0/3069537106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1da552370 msgr2=0x55a1da54e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:28.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.557+0000 7fc01114e640 1 --2- 172.21.15.116:0/3069537106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1da552370 0x55a1da54e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.557+0000 7fc01114e640 1 -- 172.21.15.116:0/3069537106 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1da54df30 con 0x55a1da5475d0 2024-09-12T03:18:28.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.557+0000 7fc01114e640 1 --2- 172.21.15.116:0/3069537106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1da5475d0 0x55a1da543e10 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fc008009610 tx=0x7fc00801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15e8087e11bb08 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:28.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.557+0000 7fbff6ffd640 1 -- 172.21.15.116:0/3069537106 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0080d2070 con 0x55a1da5475d0 2024-09-12T03:18:28.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.557+0000 7fbff6ffd640 1 -- 172.21.15.116:0/3069537106 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc00809a400 con 0x55a1da5475d0 2024-09-12T03:18:28.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.557+0000 7fbff6ffd640 1 -- 172.21.15.116:0/3069537106 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0080a3d80 con 0x55a1da5475d0 2024-09-12T03:18:28.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.558+0000 7fc01578b740 1 -- 172.21.15.116:0/3069537106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1da5475d0 msgr2=0x55a1da543e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:28.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.558+0000 7fc01578b740 1 --2- 172.21.15.116:0/3069537106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1da5475d0 0x55a1da543e10 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fc008009610 tx=0x7fc00801c000 comp rx=0 tx=0).stop 2024-09-12T03:18:28.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.558+0000 7fc01578b740 1 -- 172.21.15.116:0/3069537106 shutdown_connections 2024-09-12T03:18:28.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.558+0000 7fc01578b740 1 --2- 172.21.15.116:0/3069537106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1da552370 0x55a1da54e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.558+0000 7fc01578b740 1 --2- 172.21.15.116:0/3069537106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1da548b40 0x55a1da55e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.558+0000 7fc01578b740 1 --2- 172.21.15.116:0/3069537106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1da5475d0 0x55a1da543e10 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.558+0000 7fc01578b740 1 -- 172.21.15.116:0/3069537106 >> 172.21.15.116:0/3069537106 conn(0x55a1da510310 msgr2=0x55a1da543700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:28.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.558+0000 7fc01578b740 1 -- 172.21.15.116:0/3069537106 shutdown_connections 2024-09-12T03:18:28.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.558+0000 7fc01578b740 1 -- 172.21.15.116:0/3069537106 wait complete. 2024-09-12T03:18:28.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.558+0000 7fc01578b740 1 -- start start 2024-09-12T03:18:28.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.559+0000 7fc01578b740 1 Processor -- start 2024-09-12T03:18:28.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.559+0000 7fc01578b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1da5475d0 0x55a1da55a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:28.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.559+0000 7fc01578b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1da548b40 0x55a1da55ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:28.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.559+0000 7fc01578b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1da552370 0x55a1da555a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:28.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.559+0000 7fc01578b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1abb10 con 0x55a1da548b40 2024-09-12T03:18:28.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.559+0000 7fc01578b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1acf80 con 0x55a1da5475d0 2024-09-12T03:18:28.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.559+0000 7fc01578b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c1ac1e0 con 0x55a1da552370 2024-09-12T03:18:28.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.559+0000 7fc01578b740 10 client.0 _ll_register_callbacks cb 0x7fc008097bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:18:28.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.559+0000 7fc01094d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1da548b40 0x55a1da55ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:28.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.559+0000 7fc01094d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1da548b40 0x55a1da55ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60634/0 (socket says 172.21.15.116:60634) 2024-09-12T03:18:28.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.559+0000 7fc01094d640 1 -- 172.21.15.116:0/3081870818 learned_addr learned my addr 172.21.15.116:0/3081870818 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:28.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.559+0000 7fc01114e640 1 --2- 172.21.15.116:0/3081870818 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1da5475d0 0x55a1da55a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:28.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.559+0000 7fc01194f640 1 --2- 172.21.15.116:0/3081870818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1da552370 0x55a1da555a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:28.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.559+0000 7fc01094d640 1 -- 172.21.15.116:0/3081870818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1da552370 msgr2=0x55a1da555a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:28.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.559+0000 7fc01094d640 1 --2- 172.21.15.116:0/3081870818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1da552370 0x55a1da555a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.559+0000 7fc01094d640 1 -- 172.21.15.116:0/3081870818 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1da5475d0 msgr2=0x55a1da55a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:28.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.559+0000 7fc01094d640 1 --2- 172.21.15.116:0/3081870818 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1da5475d0 0x55a1da55a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.560+0000 7fc01094d640 1 -- 172.21.15.116:0/3081870818 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1da556240 con 0x55a1da548b40 2024-09-12T03:18:28.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.560+0000 7fc01194f640 1 --2- 172.21.15.116:0/3081870818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1da552370 0x55a1da555a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:28.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.560+0000 7fc01094d640 1 --2- 172.21.15.116:0/3081870818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1da548b40 0x55a1da55ab90 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fbffc013680 tx=0x7fbffc006f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee2cfb53dda068ba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:28.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.560+0000 7fbff4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:18:28.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.560+0000 7fbff4ff9640 1 -- 172.21.15.116:0/3081870818 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc02c060 con 0x55a1da548b40 2024-09-12T03:18:28.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.560+0000 7fbff4ff9640 1 -- 172.21.15.116:0/3081870818 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fbffc0139c0 con 0x55a1da548b40 2024-09-12T03:18:28.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.560+0000 7fc01578b740 1 -- 172.21.15.116:0/3081870818 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc00809a400 con 0x55a1da548b40 2024-09-12T03:18:28.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.560+0000 7fbff4ff9640 1 -- 172.21.15.116:0/3081870818 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbffc00d950 con 0x55a1da548b40 2024-09-12T03:18:28.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.560+0000 7fc01578b740 10 client.19572 fetch_fsmap learned FSMap version 25 2024-09-12T03:18:28.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.560+0000 7fc01578b740 1 -- 172.21.15.116:0/3081870818 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a1da5cc5a0 con 0x55a1da548b40 2024-09-12T03:18:28.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.560+0000 7fbff4ff9640 1 -- 172.21.15.116:0/3081870818 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc00809a400 con 0x55a1da548b40 2024-09-12T03:18:28.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.560+0000 7fbff4ff9640 1 -- 172.21.15.116:0/3081870818 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbffc01d400 con 0x55a1da548b40 2024-09-12T03:18:28.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.560+0000 7fc01578b740 10 client.19572 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:18:28.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.560+0000 7fc01578b740 10 client.19572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:18:28.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.560+0000 7fc01578b740 20 client.19572 populate_metadata read hostname 'smithi116' 2024-09-12T03:18:28.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.560+0000 7fc01578b740 1 --2- 172.21.15.116:0/3081870818 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a1da5cca40 0x55a1da5cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:28.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.560+0000 7fc01578b740 4 client.19572 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:18:28.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.560+0000 7fc01578b740 1 -- 172.21.15.116:0/3081870818 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a1da4b4d70 con 0x55a1da5cca40 2024-09-12T03:18:28.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.561+0000 7fc01114e640 1 --2- 172.21.15.116:0/3081870818 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a1da5cca40 0x55a1da5cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:28.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.561+0000 7fc01114e640 1 --2- 172.21.15.116:0/3081870818 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a1da5cca40 0x55a1da5cee30 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fc86f2a20d4ff2b server_cookie=f985c3241389a67d in_seq=0 out_seq=0 2024-09-12T03:18:28.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.561+0000 7fbff4ff9640 10 client.19572 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:18:28.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.561+0000 7fbff4ff9640 1 -- 172.21.15.116:0/3081870818 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a1da4b4d70 con 0x55a1da5cca40 2024-09-12T03:18:28.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.561+0000 7fbff4ff9640 10 client.19572 handle_command_reply: tid=0 2024-09-12T03:18:28.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.561+0000 7fc01578b740 1 client.19572 shutdown 2024-09-12T03:18:28.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.561+0000 7fc01578b740 1 -- 172.21.15.116:0/3081870818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1da548b40 msgr2=0x55a1da55ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:28.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.561+0000 7fc01578b740 1 --2- 172.21.15.116:0/3081870818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1da548b40 0x55a1da55ab90 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fbffc013680 tx=0x7fbffc006f40 comp rx=0 tx=0).stop 2024-09-12T03:18:28.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.562+0000 7fc01578b740 1 -- 172.21.15.116:0/3081870818 shutdown_connections 2024-09-12T03:18:28.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.562+0000 7fc01578b740 1 --2- 172.21.15.116:0/3081870818 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55a1da5cca40 0x55a1da5cee30 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.562+0000 7fc01578b740 1 --2- 172.21.15.116:0/3081870818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1da552370 0x55a1da555a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.562+0000 7fc01578b740 1 --2- 172.21.15.116:0/3081870818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a1da548b40 0x55a1da55ab90 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.562+0000 7fc01578b740 1 --2- 172.21.15.116:0/3081870818 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a1da5475d0 0x55a1da55a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.562+0000 7fc01578b740 1 -- 172.21.15.116:0/3081870818 >> 172.21.15.116:0/3081870818 conn(0x55a1da510310 msgr2=0x55a1da551000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:28.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.562+0000 7fbff4ff9640 0 client.19572 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:18:28.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.562+0000 7fc01578b740 1 -- 172.21.15.116:0/3081870818 shutdown_connections 2024-09-12T03:18:28.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.562+0000 7fc01578b740 1 -- 172.21.15.116:0/3081870818 wait complete. 2024-09-12T03:18:28.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.562+0000 7fc01578b740 20 client.19572 trim_cache size 0 max 16384 2024-09-12T03:18:28.574 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:18:28.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.562+0000 7fc0133b2640 1 -- 172.21.15.116:0/978559589 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbff80a2b80 msgr2=0x7fbff80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:28.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.562+0000 7fc0133b2640 1 --2- 172.21.15.116:0/978559589 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbff80a2b80 0x7fbff80a5040 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc004006010 tx=0x7fc0040095c0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.562+0000 7fc0133b2640 1 -- 172.21.15.116:0/978559589 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc00c101de0 msgr2=0x7fc00c198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:28.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.562+0000 7fc0133b2640 1 --2- 172.21.15.116:0/978559589 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc00c101de0 0x7fc00c198470 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fc00800d950 tx=0x7fc00800de10 comp rx=0 tx=0).stop 2024-09-12T03:18:28.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.562+0000 7fc0133b2640 1 -- 172.21.15.116:0/978559589 shutdown_connections 2024-09-12T03:18:28.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.562+0000 7fc0133b2640 1 --2- 172.21.15.116:0/978559589 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fbff80a2b80 0x7fbff80a5040 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.562+0000 7fc0133b2640 1 --2- 172.21.15.116:0/978559589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc00c106f10 0x7fc00c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.562+0000 7fc0133b2640 1 --2- 172.21.15.116:0/978559589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc00c102790 0x7fc00c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.562+0000 7fc0133b2640 1 --2- 172.21.15.116:0/978559589 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc00c101de0 0x7fc00c198470 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.562+0000 7fc0133b2640 1 -- 172.21.15.116:0/978559589 >> 172.21.15.116:0/978559589 conn(0x7fc00c0fd8c0 msgr2=0x7fc00c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:28.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.563+0000 7fc0133b2640 1 -- 172.21.15.116:0/978559589 shutdown_connections 2024-09-12T03:18:28.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:28.563+0000 7fc0133b2640 1 -- 172.21.15.116:0/978559589 wait complete. 2024-09-12T03:18:29.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:29 smithi116 ceph-mon[31410]: pgmap v803: 97 pgs: 97 active+clean; 7.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:18:29.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:29 smithi177 ceph-mon[35305]: pgmap v803: 97 pgs: 97 active+clean; 7.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:18:29.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:29 smithi183 ceph-mon[35535]: pgmap v803: 97 pgs: 97 active+clean; 7.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:18:30.269 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:18:30.270 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:18:30.528 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:31.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.059+0000 7f8814f53640 1 -- 172.21.15.116:0/97474580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8810101420 msgr2=0x7f881010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:31.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.059+0000 7f8814f53640 1 --2- 172.21.15.116:0/97474580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8810101420 0x7f881010d950 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f8808009cd0 tx=0x7f880802a8e0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.059+0000 7f8814f53640 1 -- 172.21.15.116:0/97474580 shutdown_connections 2024-09-12T03:18:31.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.059+0000 7f8814f53640 1 --2- 172.21.15.116:0/97474580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8810101420 0x7f881010d950 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.059+0000 7f8814f53640 1 --2- 172.21.15.116:0/97474580 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8810100a80 0x7f8810100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.059+0000 7f8814f53640 1 --2- 172.21.15.116:0/97474580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8810106a80 0x7f8810106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.059+0000 7f8814f53640 1 -- 172.21.15.116:0/97474580 >> 172.21.15.116:0/97474580 conn(0x7f88100fc7a0 msgr2=0x7f88100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:31.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.059+0000 7f8814f53640 1 -- 172.21.15.116:0/97474580 shutdown_connections 2024-09-12T03:18:31.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.059+0000 7f8814f53640 1 -- 172.21.15.116:0/97474580 wait complete. 2024-09-12T03:18:31.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.059+0000 7f8814f53640 1 Processor -- start 2024-09-12T03:18:31.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.059+0000 7f8814f53640 1 -- start start 2024-09-12T03:18:31.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.060+0000 7f8814f53640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8810100a80 0x7f88101983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:31.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.060+0000 7f8814f53640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8810101420 0x7f88101988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:31.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.060+0000 7f8814f53640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8810106a80 0x7f881019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:31.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.060+0000 7f8814f53640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8810111fa0 con 0x7f8810106a80 2024-09-12T03:18:31.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.060+0000 7f8814f53640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8810111e20 con 0x7f8810101420 2024-09-12T03:18:31.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.060+0000 7f8814f53640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8810112120 con 0x7f8810100a80 2024-09-12T03:18:31.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.060+0000 7f880ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8810106a80 0x7f881019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:31.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.060+0000 7f880ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8810106a80 0x7f881019cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41226/0 (socket says 172.21.15.116:41226) 2024-09-12T03:18:31.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.060+0000 7f880ed9d640 1 -- 172.21.15.116:0/1393070357 learned_addr learned my addr 172.21.15.116:0/1393070357 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:31.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.060+0000 7f880e59c640 1 --2- 172.21.15.116:0/1393070357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8810100a80 0x7f88101983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:31.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.060+0000 7f880ed9d640 1 -- 172.21.15.116:0/1393070357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8810100a80 msgr2=0x7f88101983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:31.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.060+0000 7f880ed9d640 1 --2- 172.21.15.116:0/1393070357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8810100a80 0x7f88101983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.060+0000 7f880ed9d640 1 -- 172.21.15.116:0/1393070357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8810101420 msgr2=0x7f88101988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:31.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.060+0000 7f880dd9b640 1 --2- 172.21.15.116:0/1393070357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8810101420 0x7f88101988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:31.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.060+0000 7f880ed9d640 1 --2- 172.21.15.116:0/1393070357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8810101420 0x7f88101988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.060+0000 7f880ed9d640 1 -- 172.21.15.116:0/1393070357 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881019d350 con 0x7f8810106a80 2024-09-12T03:18:31.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.060+0000 7f880ed9d640 1 --2- 172.21.15.116:0/1393070357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8810106a80 0x7f881019cc70 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f8808006010 tx=0x7f8808003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:31.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.060+0000 7f87ff7fe640 1 -- 172.21.15.116:0/1393070357 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8808004450 con 0x7f8810106a80 2024-09-12T03:18:31.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.061+0000 7f8814f53640 1 -- 172.21.15.116:0/1393070357 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f881019d5e0 con 0x7f8810106a80 2024-09-12T03:18:31.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.061+0000 7f87ff7fe640 1 -- 172.21.15.116:0/1393070357 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8808003e10 con 0x7f8810106a80 2024-09-12T03:18:31.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.061+0000 7f87ff7fe640 1 -- 172.21.15.116:0/1393070357 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88080466e0 con 0x7f8810106a80 2024-09-12T03:18:31.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.061+0000 7f8814f53640 1 -- 172.21.15.116:0/1393070357 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88101a4f20 con 0x7f8810106a80 2024-09-12T03:18:31.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.063+0000 7f87ff7fe640 1 -- 172.21.15.116:0/1393070357 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f88080305a0 con 0x7f8810106a80 2024-09-12T03:18:31.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.063+0000 7f881732c740 1 Processor -- start 2024-09-12T03:18:31.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.063+0000 7f881732c740 1 -- start start 2024-09-12T03:18:31.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.064+0000 7f881732c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e522b0b60 0x561e522b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:31.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.064+0000 7f881732c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e522b1480 0x561e522bb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:31.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.064+0000 7f881732c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e522bbd40 0x561e522be130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:31.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.064+0000 7f881732c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561e52216280 con 0x561e522b1480 2024-09-12T03:18:31.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.064+0000 7f881732c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561e521bdb10 con 0x561e522b0b60 2024-09-12T03:18:31.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.064+0000 7f881732c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561e52212240 con 0x561e522bbd40 2024-09-12T03:18:31.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.064+0000 7f880e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e522b1480 0x561e522bb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:31.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.064+0000 7f880e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e522b1480 0x561e522bb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41230/0 (socket says 172.21.15.116:41230) 2024-09-12T03:18:31.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.064+0000 7f880e59c640 1 -- 172.21.15.116:0/2546287621 learned_addr learned my addr 172.21.15.116:0/2546287621 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:31.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.064+0000 7f880dd9b640 1 --2- 172.21.15.116:0/2546287621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e522b0b60 0x561e522b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:31.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.064+0000 7f87ff7fe640 1 --2- 172.21.15.116:0/1393070357 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f87f80a2b80 0x7f87f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:31.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.064+0000 7f880dd9b640 1 --2- 172.21.15.116:0/2546287621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e522bbd40 0x561e522be130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:31.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.064+0000 7f880e59c640 1 -- 172.21.15.116:0/2546287621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e522bbd40 msgr2=0x561e522be130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:31.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.064+0000 7f880e59c640 1 --2- 172.21.15.116:0/2546287621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e522bbd40 0x561e522be130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.064+0000 7f880e59c640 1 -- 172.21.15.116:0/2546287621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e522b0b60 msgr2=0x561e522b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:31.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.064+0000 7f880e59c640 1 --2- 172.21.15.116:0/2546287621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e522b0b60 0x561e522b0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.064+0000 7f880e59c640 1 -- 172.21.15.116:0/2546287621 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e521ca1f0 con 0x561e522b1480 2024-09-12T03:18:31.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.064+0000 7f880dd9b640 1 --2- 172.21.15.116:0/2546287621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e522b0b60 0x561e522b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:31.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.064+0000 7f880ed9d640 1 --2- 172.21.15.116:0/1393070357 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f87f80a2b80 0x7f87f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:31.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.064+0000 7f87ff7fe640 1 -- 172.21.15.116:0/1393070357 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f880802e030 con 0x7f8810106a80 2024-09-12T03:18:31.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.064+0000 7f880e59c640 1 --2- 172.21.15.116:0/2546287621 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e522b1480 0x561e522bb800 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f880000ba30 tx=0x7f880000bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=accbd12dbef77e16 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:31.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.064+0000 7f87fcff9640 1 -- 172.21.15.116:0/2546287621 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800017420 con 0x561e522b1480 2024-09-12T03:18:31.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.064+0000 7f87fcff9640 1 -- 172.21.15.116:0/2546287621 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8800017aa0 con 0x561e522b1480 2024-09-12T03:18:31.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.064+0000 7f87fcff9640 1 -- 172.21.15.116:0/2546287621 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880000fb30 con 0x561e522b1480 2024-09-12T03:18:31.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.065+0000 7f881732c740 1 -- 172.21.15.116:0/2546287621 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e522b1480 msgr2=0x561e522bb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:31.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.065+0000 7f880ed9d640 1 --2- 172.21.15.116:0/1393070357 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f87f80a2b80 0x7f87f80a5040 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f88080bd290 tx=0x7f88080b27e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:31.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.065+0000 7f881732c740 1 --2- 172.21.15.116:0/2546287621 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e522b1480 0x561e522bb800 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f880000ba30 tx=0x7f880000bef0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.065+0000 7f881732c740 1 -- 172.21.15.116:0/2546287621 shutdown_connections 2024-09-12T03:18:31.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.065+0000 7f881732c740 1 --2- 172.21.15.116:0/2546287621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e522bbd40 0x561e522be130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.065+0000 7f881732c740 1 --2- 172.21.15.116:0/2546287621 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e522b1480 0x561e522bb800 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.065+0000 7f881732c740 1 --2- 172.21.15.116:0/2546287621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e522b0b60 0x561e522b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.065+0000 7f881732c740 1 -- 172.21.15.116:0/2546287621 >> 172.21.15.116:0/2546287621 conn(0x561e522ad0f0 msgr2=0x561e522ad4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:31.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.065+0000 7f881732c740 1 -- 172.21.15.116:0/2546287621 shutdown_connections 2024-09-12T03:18:31.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.065+0000 7f881732c740 1 -- 172.21.15.116:0/2546287621 wait complete. 2024-09-12T03:18:31.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.066+0000 7f881732c740 1 -- start start 2024-09-12T03:18:31.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.066+0000 7f881732c740 1 Processor -- start 2024-09-12T03:18:31.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.066+0000 7f881732c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e522b0b60 0x561e522b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:31.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.066+0000 7f881732c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e522b1480 0x561e522b90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:31.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.066+0000 7f881732c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e522bbd40 0x561e522b95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:31.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.066+0000 7f881732c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561e522be910 con 0x561e522bbd40 2024-09-12T03:18:31.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.066+0000 7f881732c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561e521d0f10 con 0x561e522b0b60 2024-09-12T03:18:31.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.066+0000 7f881732c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561e522bea90 con 0x561e522b1480 2024-09-12T03:18:31.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.066+0000 7f881732c740 10 client.0 _ll_register_callbacks cb 0x561e52199a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:18:31.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.066+0000 7f880dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e522bbd40 0x561e522b95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:31.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.066+0000 7f880dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e522bbd40 0x561e522b95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41244/0 (socket says 172.21.15.116:41244) 2024-09-12T03:18:31.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.066+0000 7f880dd9b640 1 -- 172.21.15.116:0/1478446163 learned_addr learned my addr 172.21.15.116:0/1478446163 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:31.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.066+0000 7f880e59c640 1 --2- 172.21.15.116:0/1478446163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e522b1480 0x561e522b90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:31.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.066+0000 7f880dd9b640 1 --2- 172.21.15.116:0/1478446163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e522b0b60 0x561e522b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:31.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.066+0000 7f880dd9b640 1 -- 172.21.15.116:0/1478446163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e522b1480 msgr2=0x561e522b90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:31.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.066+0000 7f880dd9b640 1 --2- 172.21.15.116:0/1478446163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e522b1480 0x561e522b90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.067+0000 7f880dd9b640 1 -- 172.21.15.116:0/1478446163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e522b0b60 msgr2=0x561e522b8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:31.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.067+0000 7f880dd9b640 1 --2- 172.21.15.116:0/1478446163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e522b0b60 0x561e522b8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.067+0000 7f880dd9b640 1 -- 172.21.15.116:0/1478446163 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e51f876a0 con 0x561e522bbd40 2024-09-12T03:18:31.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.067+0000 7f880dd9b640 1 --2- 172.21.15.116:0/1478446163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e522bbd40 0x561e522b95f0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f880400df20 tx=0x7f8804004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85589e70e4b3b257 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:31.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.067+0000 7f87f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:18:31.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.067+0000 7f87f5ffb640 1 -- 172.21.15.116:0/1478446163 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880400fd90 con 0x561e522bbd40 2024-09-12T03:18:31.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.067+0000 7f87f5ffb640 1 -- 172.21.15.116:0/1478446163 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f8804005020 con 0x561e522bbd40 2024-09-12T03:18:31.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.067+0000 7f881732c740 1 -- 172.21.15.116:0/1478446163 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8800017aa0 con 0x561e522bbd40 2024-09-12T03:18:31.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.067+0000 7f87f5ffb640 1 -- 172.21.15.116:0/1478446163 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8804005220 con 0x561e522bbd40 2024-09-12T03:18:31.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.067+0000 7f881732c740 10 client.19584 fetch_fsmap learned FSMap version 25 2024-09-12T03:18:31.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.067+0000 7f881732c740 1 -- 172.21.15.116:0/1478446163 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561e522ca4c0 con 0x561e522bbd40 2024-09-12T03:18:31.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.067+0000 7f87f5ffb640 1 -- 172.21.15.116:0/1478446163 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8800017aa0 con 0x561e522bbd40 2024-09-12T03:18:31.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.067+0000 7f87f5ffb640 1 -- 172.21.15.116:0/1478446163 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f880401c030 con 0x561e522bbd40 2024-09-12T03:18:31.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.067+0000 7f881732c740 10 client.19584 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:18:31.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.067+0000 7f881732c740 10 client.19584 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:18:31.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.067+0000 7f881732c740 20 client.19584 populate_metadata read hostname 'smithi116' 2024-09-12T03:18:31.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.067+0000 7f881732c740 1 --2- 172.21.15.116:0/1478446163 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x561e522cb8d0 0x561e522cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:31.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.067+0000 7f881732c740 4 client.19584 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:18:31.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.067+0000 7f881732c740 1 -- 172.21.15.116:0/1478446163 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e522237f0 con 0x561e522cb8d0 2024-09-12T03:18:31.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.068+0000 7f880e59c640 1 --2- 172.21.15.116:0/1478446163 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x561e522cb8d0 0x561e522cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:31.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.068+0000 7f880e59c640 1 --2- 172.21.15.116:0/1478446163 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x561e522cb8d0 0x561e522cdcc0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9429ef47c9936656 server_cookie=70b730ba52e54146 in_seq=0 out_seq=0 2024-09-12T03:18:31.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.068+0000 7f87f5ffb640 10 client.19584 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:18:31.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.069+0000 7f87f5ffb640 1 -- 172.21.15.116:0/1478446163 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e522237f0 con 0x561e522cb8d0 2024-09-12T03:18:31.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.069+0000 7f87f5ffb640 10 client.19584 handle_command_reply: tid=0 2024-09-12T03:18:31.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.069+0000 7f881732c740 1 client.19584 shutdown 2024-09-12T03:18:31.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.069+0000 7f881732c740 1 -- 172.21.15.116:0/1478446163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e522bbd40 msgr2=0x561e522b95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:31.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.069+0000 7f881732c740 1 --2- 172.21.15.116:0/1478446163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e522bbd40 0x561e522b95f0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f880400df20 tx=0x7f8804004730 comp rx=0 tx=0).stop 2024-09-12T03:18:31.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.069+0000 7f881732c740 1 -- 172.21.15.116:0/1478446163 shutdown_connections 2024-09-12T03:18:31.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.069+0000 7f881732c740 1 --2- 172.21.15.116:0/1478446163 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x561e522cb8d0 0x561e522cdcc0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.069+0000 7f881732c740 1 --2- 172.21.15.116:0/1478446163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e522bbd40 0x561e522b95f0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.069+0000 7f881732c740 1 --2- 172.21.15.116:0/1478446163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e522b1480 0x561e522b90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.069+0000 7f87f5ffb640 0 client.19584 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:18:31.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.069+0000 7f881732c740 1 --2- 172.21.15.116:0/1478446163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e522b0b60 0x561e522b8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.069+0000 7f881732c740 1 -- 172.21.15.116:0/1478446163 >> 172.21.15.116:0/1478446163 conn(0x561e522ad0f0 msgr2=0x561e522ac270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:31.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.069+0000 7f881732c740 1 -- 172.21.15.116:0/1478446163 shutdown_connections 2024-09-12T03:18:31.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.069+0000 7f881732c740 1 -- 172.21.15.116:0/1478446163 wait complete. 2024-09-12T03:18:31.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.069+0000 7f881732c740 20 client.19584 trim_cache size 0 max 16384 2024-09-12T03:18:31.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.086+0000 7f881732c740 1 Processor -- start 2024-09-12T03:18:31.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.086+0000 7f881732c740 1 -- start start 2024-09-12T03:18:31.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.087+0000 7f881732c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e522ad0f0 0x561e522b7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:31.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.087+0000 7f881732c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e522b0b60 0x561e522d34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:31.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.087+0000 7f881732c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e522b1480 0x561e522d3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:31.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.087+0000 7f881732c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f88101afd70 con 0x561e522b1480 2024-09-12T03:18:31.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.087+0000 7f881732c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f88101ad640 con 0x561e522ad0f0 2024-09-12T03:18:31.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.087+0000 7f881732c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f88101ab490 con 0x561e522b0b60 2024-09-12T03:18:31.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.087+0000 7f880dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e522b1480 0x561e522d3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:31.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.087+0000 7f880dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e522b1480 0x561e522d3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41260/0 (socket says 172.21.15.116:41260) 2024-09-12T03:18:31.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.087+0000 7f880dd9b640 1 -- 172.21.15.116:0/1258468999 learned_addr learned my addr 172.21.15.116:0/1258468999 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:31.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.087+0000 7f880dd9b640 1 --2- 172.21.15.116:0/1258468999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e522ad0f0 0x561e522b7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:31.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.087+0000 7f880dd9b640 1 -- 172.21.15.116:0/1258468999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e522b0b60 msgr2=0x561e522d34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:31.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.087+0000 7f880dd9b640 1 --2- 172.21.15.116:0/1258468999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e522b0b60 0x561e522d34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.087+0000 7f880dd9b640 1 -- 172.21.15.116:0/1258468999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e522ad0f0 msgr2=0x561e522b7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:31.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.087+0000 7f880dd9b640 1 --2- 172.21.15.116:0/1258468999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e522ad0f0 0x561e522b7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.087+0000 7f880dd9b640 1 -- 172.21.15.116:0/1258468999 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e522acf20 con 0x561e522b1480 2024-09-12T03:18:31.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.087+0000 7f880dd9b640 1 --2- 172.21.15.116:0/1258468999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e522b1480 0x561e522d3a30 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f8804005be0 tx=0x7f88040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc970815ccd8a8f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:31.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.087+0000 7f87f7fff640 1 -- 172.21.15.116:0/1258468999 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804005ea0 con 0x561e522b1480 2024-09-12T03:18:31.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.087+0000 7f87f7fff640 1 -- 172.21.15.116:0/1258468999 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f880400fb90 con 0x561e522b1480 2024-09-12T03:18:31.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.087+0000 7f87f7fff640 1 -- 172.21.15.116:0/1258468999 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804005620 con 0x561e522b1480 2024-09-12T03:18:31.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.088+0000 7f881732c740 1 -- 172.21.15.116:0/1258468999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e522b1480 msgr2=0x561e522d3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:31.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.088+0000 7f881732c740 1 --2- 172.21.15.116:0/1258468999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e522b1480 0x561e522d3a30 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f8804005be0 tx=0x7f88040046c0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.088+0000 7f881732c740 1 -- 172.21.15.116:0/1258468999 shutdown_connections 2024-09-12T03:18:31.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.088+0000 7f881732c740 1 --2- 172.21.15.116:0/1258468999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e522b1480 0x561e522d3a30 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.088+0000 7f881732c740 1 --2- 172.21.15.116:0/1258468999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e522b0b60 0x561e522d34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.088+0000 7f881732c740 1 --2- 172.21.15.116:0/1258468999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e522ad0f0 0x561e522b7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.088+0000 7f881732c740 1 -- 172.21.15.116:0/1258468999 >> 172.21.15.116:0/1258468999 conn(0x561e522b4560 msgr2=0x561e522b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:31.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.088+0000 7f881732c740 1 -- 172.21.15.116:0/1258468999 shutdown_connections 2024-09-12T03:18:31.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.088+0000 7f881732c740 1 -- 172.21.15.116:0/1258468999 wait complete. 2024-09-12T03:18:31.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.088+0000 7f881732c740 1 -- start start 2024-09-12T03:18:31.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.089+0000 7f881732c740 1 Processor -- start 2024-09-12T03:18:31.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.089+0000 7f881732c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e522ad0f0 0x561e522c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:31.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.089+0000 7f881732c740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e522b0b60 0x561e522c4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:31.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.089+0000 7f881732c740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e522b1480 0x561e522c0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:31.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.089+0000 7f881732c740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f88101abb00 con 0x561e522b1480 2024-09-12T03:18:31.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.089+0000 7f881732c740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f88101acf70 con 0x561e522b0b60 2024-09-12T03:18:31.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.089+0000 7f881732c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f88101ac1d0 con 0x561e522ad0f0 2024-09-12T03:18:31.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.089+0000 7f881732c740 10 client.0 _ll_register_callbacks cb 0x7f880400d510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:18:31.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.089+0000 7f880dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e522b1480 0x561e522c0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:31.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.089+0000 7f880dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e522b1480 0x561e522c0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41270/0 (socket says 172.21.15.116:41270) 2024-09-12T03:18:31.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.089+0000 7f880dd9b640 1 -- 172.21.15.116:0/1420362978 learned_addr learned my addr 172.21.15.116:0/1420362978 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:31.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.089+0000 7f880dd9b640 1 --2- 172.21.15.116:0/1420362978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e522ad0f0 0x561e522c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:31.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.089+0000 7f880e59c640 1 --2- 172.21.15.116:0/1420362978 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e522b0b60 0x561e522c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:31.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.089+0000 7f880dd9b640 1 -- 172.21.15.116:0/1420362978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e522ad0f0 msgr2=0x561e522c3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:31.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.089+0000 7f880dd9b640 1 --2- 172.21.15.116:0/1420362978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e522ad0f0 0x561e522c3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.089+0000 7f880dd9b640 1 -- 172.21.15.116:0/1420362978 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e522b0b60 msgr2=0x561e522c4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:31.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.089+0000 7f880dd9b640 1 --2- 172.21.15.116:0/1420362978 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e522b0b60 0x561e522c4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.089+0000 7f880dd9b640 1 -- 172.21.15.116:0/1420362978 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e522c0a50 con 0x561e522b1480 2024-09-12T03:18:31.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.089+0000 7f880dd9b640 1 --2- 172.21.15.116:0/1420362978 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e522b1480 0x561e522c0220 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f8804014040 tx=0x7f880400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cdcd5bcf93630f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:31.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.089+0000 7f87f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:18:31.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.090+0000 7f87f5ffb640 1 -- 172.21.15.116:0/1420362978 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88040055c0 con 0x561e522b1480 2024-09-12T03:18:31.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.090+0000 7f87f5ffb640 1 -- 172.21.15.116:0/1420362978 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f880401da00 con 0x561e522b1480 2024-09-12T03:18:31.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.090+0000 7f881732c740 1 -- 172.21.15.116:0/1420362978 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f880400fb90 con 0x561e522b1480 2024-09-12T03:18:31.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.090+0000 7f87f5ffb640 1 -- 172.21.15.116:0/1420362978 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f880401dc00 con 0x561e522b1480 2024-09-12T03:18:31.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.090+0000 7f881732c740 10 client.19590 fetch_fsmap learned FSMap version 25 2024-09-12T03:18:31.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.090+0000 7f881732c740 1 -- 172.21.15.116:0/1420362978 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561e52336fd0 con 0x561e522b1480 2024-09-12T03:18:31.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.090+0000 7f87f5ffb640 1 -- 172.21.15.116:0/1420362978 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8804004c10 con 0x561e522b1480 2024-09-12T03:18:31.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.090+0000 7f87f5ffb640 1 -- 172.21.15.116:0/1420362978 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f880401c070 con 0x561e522b1480 2024-09-12T03:18:31.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.090+0000 7f881732c740 10 client.19590 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:18:31.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.090+0000 7f881732c740 10 client.19590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:18:31.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.090+0000 7f881732c740 20 client.19590 populate_metadata read hostname 'smithi116' 2024-09-12T03:18:31.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.090+0000 7f881732c740 1 --2- 172.21.15.116:0/1420362978 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x561e522bbd40 0x561e52339640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:31.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.090+0000 7f881732c740 4 client.19590 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:18:31.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.090+0000 7f881732c740 1 -- 172.21.15.116:0/1420362978 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f880400fb90 con 0x561e522bbd40 2024-09-12T03:18:31.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.090+0000 7f880e59c640 1 --2- 172.21.15.116:0/1420362978 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x561e522bbd40 0x561e52339640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:31.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.091+0000 7f880e59c640 1 --2- 172.21.15.116:0/1420362978 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x561e522bbd40 0x561e52339640 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=533f207ddde13a69 server_cookie=7517f4949b1670c7 in_seq=0 out_seq=0 2024-09-12T03:18:31.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.091+0000 7f87f5ffb640 10 client.19590 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:18:31.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.092+0000 7f87f5ffb640 1 -- 172.21.15.116:0/1420362978 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f880400fb90 con 0x561e522bbd40 2024-09-12T03:18:31.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.092+0000 7f87f5ffb640 10 client.19590 handle_command_reply: tid=0 2024-09-12T03:18:31.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.092+0000 7f881732c740 1 client.19590 shutdown 2024-09-12T03:18:31.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.092+0000 7f881732c740 1 -- 172.21.15.116:0/1420362978 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e522b1480 msgr2=0x561e522c0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:31.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.092+0000 7f881732c740 1 --2- 172.21.15.116:0/1420362978 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e522b1480 0x561e522c0220 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f8804014040 tx=0x7f880400db90 comp rx=0 tx=0).stop 2024-09-12T03:18:31.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.092+0000 7f881732c740 1 -- 172.21.15.116:0/1420362978 shutdown_connections 2024-09-12T03:18:31.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.092+0000 7f881732c740 1 --2- 172.21.15.116:0/1420362978 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x561e522bbd40 0x561e52339640 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.092+0000 7f881732c740 1 --2- 172.21.15.116:0/1420362978 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561e522b1480 0x561e522c0220 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.092+0000 7f881732c740 1 --2- 172.21.15.116:0/1420362978 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e522b0b60 0x561e522c4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.092+0000 7f87f5ffb640 0 client.19590 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:18:31.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.092+0000 7f881732c740 1 --2- 172.21.15.116:0/1420362978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e522ad0f0 0x561e522c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.092+0000 7f881732c740 1 -- 172.21.15.116:0/1420362978 >> 172.21.15.116:0/1420362978 conn(0x561e522b4560 msgr2=0x561e522aba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:31.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.092+0000 7f881732c740 1 -- 172.21.15.116:0/1420362978 shutdown_connections 2024-09-12T03:18:31.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.092+0000 7f881732c740 1 -- 172.21.15.116:0/1420362978 wait complete. 2024-09-12T03:18:31.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.092+0000 7f881732c740 20 client.19590 trim_cache size 0 max 16384 2024-09-12T03:18:31.100 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:18:31.101 INFO:teuthology.orchestra.run.smithi116.stdout: "return_code": 0, 2024-09-12T03:18:31.101 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_tag": "576fe553-92e6-4b35-9c81-58100750824b", 2024-09-12T03:18:31.101 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": "asynchronous" 2024-09-12T03:18:31.101 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:18:31.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.093+0000 7f8814f53640 1 -- 172.21.15.116:0/1393070357 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f87f80a2b80 msgr2=0x7f87f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:31.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.093+0000 7f8814f53640 1 --2- 172.21.15.116:0/1393070357 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f87f80a2b80 0x7f87f80a5040 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f88080bd290 tx=0x7f88080b27e0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.093+0000 7f8814f53640 1 -- 172.21.15.116:0/1393070357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8810106a80 msgr2=0x7f881019cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:31.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.093+0000 7f8814f53640 1 --2- 172.21.15.116:0/1393070357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8810106a80 0x7f881019cc70 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f8808006010 tx=0x7f8808003c20 comp rx=0 tx=0).stop 2024-09-12T03:18:31.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.093+0000 7f8814f53640 1 -- 172.21.15.116:0/1393070357 shutdown_connections 2024-09-12T03:18:31.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.093+0000 7f8814f53640 1 --2- 172.21.15.116:0/1393070357 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f87f80a2b80 0x7f87f80a5040 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.093+0000 7f8814f53640 1 --2- 172.21.15.116:0/1393070357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8810106a80 0x7f881019cc70 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.093+0000 7f8814f53640 1 --2- 172.21.15.116:0/1393070357 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8810101420 0x7f88101988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.093+0000 7f8814f53640 1 --2- 172.21.15.116:0/1393070357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8810100a80 0x7f88101983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:31.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.093+0000 7f8814f53640 1 -- 172.21.15.116:0/1393070357 >> 172.21.15.116:0/1393070357 conn(0x7f88100fc7a0 msgr2=0x7f88100fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:31.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.093+0000 7f8814f53640 1 -- 172.21.15.116:0/1393070357 shutdown_connections 2024-09-12T03:18:31.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:31.093+0000 7f8814f53640 1 -- 172.21.15.116:0/1393070357 wait complete. 2024-09-12T03:18:31.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:31 smithi177 ceph-mon[35305]: pgmap v804: 97 pgs: 97 active+clean; 7.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T03:18:31.757 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:31 smithi116 ceph-mon[31410]: pgmap v804: 97 pgs: 97 active+clean; 7.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T03:18:31.784 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:18:31.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:31 smithi183 ceph-mon[35535]: pgmap v804: 97 pgs: 97 active+clean; 7.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T03:18:32.075 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:32.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.583+0000 7fa3b0eff640 1 -- 172.21.15.116:0/1483047785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3ac106a80 msgr2=0x7fa3ac106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:32.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.583+0000 7fa3b0eff640 1 --2- 172.21.15.116:0/1483047785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3ac106a80 0x7fa3ac106e60 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fa39c009930 tx=0x7fa39c02a420 comp rx=0 tx=0).stop 2024-09-12T03:18:32.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.583+0000 7fa3b0eff640 1 -- 172.21.15.116:0/1483047785 shutdown_connections 2024-09-12T03:18:32.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.583+0000 7fa3b0eff640 1 --2- 172.21.15.116:0/1483047785 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3ac101420 0x7fa3ac10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.583+0000 7fa3b0eff640 1 --2- 172.21.15.116:0/1483047785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3ac100a80 0x7fa3ac100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.583+0000 7fa3b0eff640 1 --2- 172.21.15.116:0/1483047785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3ac106a80 0x7fa3ac106e60 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.583+0000 7fa3b0eff640 1 -- 172.21.15.116:0/1483047785 >> 172.21.15.116:0/1483047785 conn(0x7fa3ac0fc7a0 msgr2=0x7fa3ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:32.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.583+0000 7fa3b0eff640 1 -- 172.21.15.116:0/1483047785 shutdown_connections 2024-09-12T03:18:32.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.583+0000 7fa3b0eff640 1 -- 172.21.15.116:0/1483047785 wait complete. 2024-09-12T03:18:32.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.583+0000 7fa3b0eff640 1 Processor -- start 2024-09-12T03:18:32.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.584+0000 7fa3b0eff640 1 -- start start 2024-09-12T03:18:32.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.584+0000 7fa3b0eff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3ac100a80 0x7fa3ac19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:32.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.584+0000 7fa3b0eff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3ac101420 0x7fa3ac19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:32.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.584+0000 7fa3b0eff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3ac106a80 0x7fa3ac19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:32.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.584+0000 7fa3b0eff640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac110ea0 con 0x7fa3ac106a80 2024-09-12T03:18:32.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.584+0000 7fa3b0eff640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac110d20 con 0x7fa3ac100a80 2024-09-12T03:18:32.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.584+0000 7fa3b0eff640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac111020 con 0x7fa3ac101420 2024-09-12T03:18:32.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.584+0000 7fa3aad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3ac106a80 0x7fa3ac19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:32.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.584+0000 7fa3aad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3ac106a80 0x7fa3ac19ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41276/0 (socket says 172.21.15.116:41276) 2024-09-12T03:18:32.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.584+0000 7fa3aad9d640 1 -- 172.21.15.116:0/271266078 learned_addr learned my addr 172.21.15.116:0/271266078 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:32.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.584+0000 7fa3a9d9b640 1 --2- 172.21.15.116:0/271266078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3ac101420 0x7fa3ac19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:32.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.584+0000 7fa3aa59c640 1 --2- 172.21.15.116:0/271266078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3ac100a80 0x7fa3ac19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:32.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.584+0000 7fa3aad9d640 1 -- 172.21.15.116:0/271266078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3ac101420 msgr2=0x7fa3ac19abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:32.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.584+0000 7fa3aad9d640 1 --2- 172.21.15.116:0/271266078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3ac101420 0x7fa3ac19abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.584+0000 7fa3aad9d640 1 -- 172.21.15.116:0/271266078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3ac100a80 msgr2=0x7fa3ac19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:32.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.584+0000 7fa3aad9d640 1 --2- 172.21.15.116:0/271266078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3ac100a80 0x7fa3ac19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.584+0000 7fa3aad9d640 1 -- 172.21.15.116:0/271266078 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ac19f640 con 0x7fa3ac106a80 2024-09-12T03:18:32.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.584+0000 7fa3aad9d640 1 --2- 172.21.15.116:0/271266078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3ac106a80 0x7fa3ac19ef60 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400cce0 tx=0x7fa3a400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:32.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.585+0000 7fa39b7fe640 1 -- 172.21.15.116:0/271266078 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a400f450 con 0x7fa3ac106a80 2024-09-12T03:18:32.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.585+0000 7fa3b0eff640 1 -- 172.21.15.116:0/271266078 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3ac19f930 con 0x7fa3ac106a80 2024-09-12T03:18:32.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.585+0000 7fa39b7fe640 1 -- 172.21.15.116:0/271266078 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3a400fad0 con 0x7fa3ac106a80 2024-09-12T03:18:32.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.585+0000 7fa39b7fe640 1 -- 172.21.15.116:0/271266078 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4018b60 con 0x7fa3ac106a80 2024-09-12T03:18:32.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.585+0000 7fa3b0eff640 1 -- 172.21.15.116:0/271266078 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3ac1a7210 con 0x7fa3ac106a80 2024-09-12T03:18:32.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.587+0000 7fa39b7fe640 1 -- 172.21.15.116:0/271266078 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa3a4018d00 con 0x7fa3ac106a80 2024-09-12T03:18:32.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.587+0000 7fa3b32d8740 1 Processor -- start 2024-09-12T03:18:32.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.588+0000 7fa3b32d8740 1 -- start start 2024-09-12T03:18:32.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.588+0000 7fa3b32d8740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569ea1d94a0 0x5569ea1d9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:32.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.588+0000 7fa3b32d8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569ea1d9dc0 0x5569ea1e4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:32.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.588+0000 7fa3b32d8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569ea1e4680 0x5569ea1e6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:32.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.588+0000 7fa3b32d8740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5569ea13f2e0 con 0x5569ea1d94a0 2024-09-12T03:18:32.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.588+0000 7fa3b32d8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5569ea0e6ce0 con 0x5569ea1d9dc0 2024-09-12T03:18:32.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.588+0000 7fa3b32d8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5569ea13b2a0 con 0x5569ea1e4680 2024-09-12T03:18:32.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.588+0000 7fa3a9d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569ea1d94a0 0x5569ea1d9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:32.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.588+0000 7fa3a9d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569ea1d94a0 0x5569ea1d9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41290/0 (socket says 172.21.15.116:41290) 2024-09-12T03:18:32.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.588+0000 7fa3a9d9b640 1 -- 172.21.15.116:0/4000879472 learned_addr learned my addr 172.21.15.116:0/4000879472 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:32.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.588+0000 7fa39b7fe640 1 --2- 172.21.15.116:0/271266078 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa3940a2b80 0x7fa3940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:32.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.588+0000 7fa3aa59c640 1 --2- 172.21.15.116:0/4000879472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569ea1d9dc0 0x5569ea1e4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:32.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.588+0000 7fa3a9d9b640 1 --2- 172.21.15.116:0/4000879472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569ea1e4680 0x5569ea1e6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:32.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.588+0000 7fa3a9d9b640 1 -- 172.21.15.116:0/4000879472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569ea1e4680 msgr2=0x5569ea1e6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:32.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.588+0000 7fa3a9d9b640 1 --2- 172.21.15.116:0/4000879472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569ea1e4680 0x5569ea1e6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.588+0000 7fa3a9d9b640 1 -- 172.21.15.116:0/4000879472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569ea1d9dc0 msgr2=0x5569ea1e4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:32.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.588+0000 7fa3a9d9b640 1 --2- 172.21.15.116:0/4000879472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569ea1d9dc0 0x5569ea1e4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.588+0000 7fa3aad9d640 1 --2- 172.21.15.116:0/271266078 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa3940a2b80 0x7fa3940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:32.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.588+0000 7fa3a9d9b640 1 -- 172.21.15.116:0/4000879472 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569ea0cb440 con 0x5569ea1d94a0 2024-09-12T03:18:32.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.588+0000 7fa39b7fe640 1 -- 172.21.15.116:0/271266078 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa3a40cf5f0 con 0x7fa3ac106a80 2024-09-12T03:18:32.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.589+0000 7fa3a9d9b640 1 --2- 172.21.15.116:0/4000879472 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569ea1d94a0 0x5569ea1d9880 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0006aa0 tx=0x7fa3a00095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294c2d746b486ed6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:32.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.589+0000 7fa398ff9640 1 -- 172.21.15.116:0/4000879472 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0017610 con 0x5569ea1d94a0 2024-09-12T03:18:32.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.589+0000 7fa398ff9640 1 -- 172.21.15.116:0/4000879472 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3a001c070 con 0x5569ea1d94a0 2024-09-12T03:18:32.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.589+0000 7fa398ff9640 1 -- 172.21.15.116:0/4000879472 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0015940 con 0x5569ea1d94a0 2024-09-12T03:18:32.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.589+0000 7fa3aad9d640 1 --2- 172.21.15.116:0/271266078 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa3940a2b80 0x7fa3940a5040 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa3a40cd0c0 tx=0x7fa3a401baf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:32.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.589+0000 7fa3b32d8740 1 -- 172.21.15.116:0/4000879472 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569ea1d94a0 msgr2=0x5569ea1d9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:32.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.589+0000 7fa3b32d8740 1 --2- 172.21.15.116:0/4000879472 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569ea1d94a0 0x5569ea1d9880 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0006aa0 tx=0x7fa3a00095d0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.590+0000 7fa3b32d8740 1 -- 172.21.15.116:0/4000879472 shutdown_connections 2024-09-12T03:18:32.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.590+0000 7fa3b32d8740 1 --2- 172.21.15.116:0/4000879472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569ea1e4680 0x5569ea1e6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.590+0000 7fa3b32d8740 1 --2- 172.21.15.116:0/4000879472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569ea1d9dc0 0x5569ea1e4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.590+0000 7fa3b32d8740 1 --2- 172.21.15.116:0/4000879472 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569ea1d94a0 0x5569ea1d9880 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.590+0000 7fa3b32d8740 1 -- 172.21.15.116:0/4000879472 >> 172.21.15.116:0/4000879472 conn(0x5569ea1d5a30 msgr2=0x5569ea1d5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:32.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.590+0000 7fa3b32d8740 1 -- 172.21.15.116:0/4000879472 shutdown_connections 2024-09-12T03:18:32.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.590+0000 7fa3b32d8740 1 -- 172.21.15.116:0/4000879472 wait complete. 2024-09-12T03:18:32.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.590+0000 7fa3b32d8740 1 -- start start 2024-09-12T03:18:32.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.590+0000 7fa3b32d8740 1 Processor -- start 2024-09-12T03:18:32.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.590+0000 7fa3b32d8740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569ea1d94a0 0x5569ea1e3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:32.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.590+0000 7fa3b32d8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569ea1d9dc0 0x5569ea1e11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:32.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.590+0000 7fa3b32d8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569ea1e4680 0x5569ea1e1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:32.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.590+0000 7fa3b32d8740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5569ea1e7250 con 0x5569ea1d94a0 2024-09-12T03:18:32.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.590+0000 7fa3b32d8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5569ea0fa050 con 0x5569ea1d9dc0 2024-09-12T03:18:32.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.590+0000 7fa3b32d8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5569ea1e73d0 con 0x5569ea1e4680 2024-09-12T03:18:32.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.590+0000 7fa3b32d8740 10 client.0 _ll_register_callbacks cb 0x5569ea13a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:18:32.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.590+0000 7fa3a9d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569ea1d94a0 0x5569ea1e3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:32.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.590+0000 7fa3a9d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569ea1d94a0 0x5569ea1e3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41296/0 (socket says 172.21.15.116:41296) 2024-09-12T03:18:32.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.590+0000 7fa3a9d9b640 1 -- 172.21.15.116:0/986149934 learned_addr learned my addr 172.21.15.116:0/986149934 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:32.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.591+0000 7fa3aa59c640 1 --2- 172.21.15.116:0/986149934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569ea1d9dc0 0x5569ea1e11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:32.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.591+0000 7fa3a9d9b640 1 -- 172.21.15.116:0/986149934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569ea1e4680 msgr2=0x5569ea1e1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:32.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.591+0000 7fa3a9d9b640 1 --2- 172.21.15.116:0/986149934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569ea1e4680 0x5569ea1e1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.591+0000 7fa3a9d9b640 1 -- 172.21.15.116:0/986149934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569ea1d9dc0 msgr2=0x5569ea1e11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:32.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.591+0000 7fa3a9d9b640 1 --2- 172.21.15.116:0/986149934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569ea1d9dc0 0x5569ea1e11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.591+0000 7fa3a9d9b640 1 -- 172.21.15.116:0/986149934 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569e9eb03c0 con 0x5569ea1d94a0 2024-09-12T03:18:32.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.591+0000 7fa3aa59c640 1 --2- 172.21.15.116:0/986149934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569ea1d9dc0 0x5569ea1e11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:18:32.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.591+0000 7fa3a9d9b640 1 --2- 172.21.15.116:0/986149934 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569ea1d94a0 0x5569ea1e3f30 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0006a70 tx=0x7fa3a0006ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60572dfa75eff20 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:32.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.591+0000 7fa391ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:18:32.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.591+0000 7fa391ffb640 1 -- 172.21.15.116:0/986149934 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0015d80 con 0x5569ea1d94a0 2024-09-12T03:18:32.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.591+0000 7fa391ffb640 1 -- 172.21.15.116:0/986149934 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa3a0017410 con 0x5569ea1d94a0 2024-09-12T03:18:32.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.591+0000 7fa3b32d8740 1 -- 172.21.15.116:0/986149934 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3a001c070 con 0x5569ea1d94a0 2024-09-12T03:18:32.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.591+0000 7fa391ffb640 1 -- 172.21.15.116:0/986149934 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa3a0017610 con 0x5569ea1d94a0 2024-09-12T03:18:32.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.591+0000 7fa3b32d8740 10 client.19602 fetch_fsmap learned FSMap version 25 2024-09-12T03:18:32.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.591+0000 7fa3b32d8740 1 -- 172.21.15.116:0/986149934 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5569ea1f2b70 con 0x5569ea1d94a0 2024-09-12T03:18:32.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.591+0000 7fa391ffb640 1 -- 172.21.15.116:0/986149934 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3a001c070 con 0x5569ea1d94a0 2024-09-12T03:18:32.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.592+0000 7fa391ffb640 1 -- 172.21.15.116:0/986149934 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa3a0033850 con 0x5569ea1d94a0 2024-09-12T03:18:32.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.592+0000 7fa3b32d8740 10 client.19602 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:18:32.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.592+0000 7fa3b32d8740 10 client.19602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:18:32.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.592+0000 7fa3b32d8740 20 client.19602 populate_metadata read hostname 'smithi116' 2024-09-12T03:18:32.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.592+0000 7fa3b32d8740 1 --2- 172.21.15.116:0/986149934 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5569ea1f4030 0x5569ea1f6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:32.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.592+0000 7fa3b32d8740 4 client.19602 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:18:32.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.592+0000 7fa3b32d8740 1 -- 172.21.15.116:0/986149934 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569ea14c850 con 0x5569ea1f4030 2024-09-12T03:18:32.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.592+0000 7fa3aa59c640 1 --2- 172.21.15.116:0/986149934 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5569ea1f4030 0x5569ea1f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:32.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.592+0000 7fa3aa59c640 1 --2- 172.21.15.116:0/986149934 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5569ea1f4030 0x5569ea1f6420 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6882b8281a076702 server_cookie=aa0ec5ca377e4604 in_seq=0 out_seq=0 2024-09-12T03:18:32.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.592+0000 7fa391ffb640 10 client.19602 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:18:32.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.593+0000 7fa391ffb640 1 -- 172.21.15.116:0/986149934 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5569ea14c850 con 0x5569ea1f4030 2024-09-12T03:18:32.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.593+0000 7fa391ffb640 10 client.19602 handle_command_reply: tid=0 2024-09-12T03:18:32.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.593+0000 7fa3b32d8740 1 client.19602 shutdown 2024-09-12T03:18:32.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.593+0000 7fa3b32d8740 1 -- 172.21.15.116:0/986149934 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569ea1d94a0 msgr2=0x5569ea1e3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:32.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.593+0000 7fa3b32d8740 1 --2- 172.21.15.116:0/986149934 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569ea1d94a0 0x5569ea1e3f30 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0006a70 tx=0x7fa3a0006ad0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.593+0000 7fa3b32d8740 1 -- 172.21.15.116:0/986149934 shutdown_connections 2024-09-12T03:18:32.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.593+0000 7fa3b32d8740 1 --2- 172.21.15.116:0/986149934 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5569ea1f4030 0x5569ea1f6420 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.593+0000 7fa3b32d8740 1 --2- 172.21.15.116:0/986149934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569ea1e4680 0x5569ea1e1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.593+0000 7fa3b32d8740 1 --2- 172.21.15.116:0/986149934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569ea1d9dc0 0x5569ea1e11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.593+0000 7fa3b32d8740 1 --2- 172.21.15.116:0/986149934 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569ea1d94a0 0x5569ea1e3f30 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.593+0000 7fa3b32d8740 1 -- 172.21.15.116:0/986149934 >> 172.21.15.116:0/986149934 conn(0x5569ea1d5a30 msgr2=0x5569ea1db760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:32.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.593+0000 7fa391ffb640 0 client.19602 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:18:32.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.593+0000 7fa3b32d8740 1 -- 172.21.15.116:0/986149934 shutdown_connections 2024-09-12T03:18:32.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.593+0000 7fa3b32d8740 1 -- 172.21.15.116:0/986149934 wait complete. 2024-09-12T03:18:32.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.593+0000 7fa3b32d8740 20 client.19602 trim_cache size 0 max 16384 2024-09-12T03:18:32.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.610+0000 7fa3b32d8740 1 Processor -- start 2024-09-12T03:18:32.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.610+0000 7fa3b32d8740 1 -- start start 2024-09-12T03:18:32.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.610+0000 7fa3b32d8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569ea1d94a0 0x5569ea1e13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:32.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.610+0000 7fa3b32d8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569ea1d9dc0 0x5569ea1d4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:32.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.610+0000 7fa3b32d8740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569ea1e4680 0x5569ea1fb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:32.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.610+0000 7fa3b32d8740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac1b1ff0 con 0x5569ea1e4680 2024-09-12T03:18:32.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.610+0000 7fa3b32d8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac1af8c0 con 0x5569ea1d9dc0 2024-09-12T03:18:32.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.610+0000 7fa3b32d8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac1ad710 con 0x5569ea1d94a0 2024-09-12T03:18:32.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.611+0000 7fa3a9d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569ea1e4680 0x5569ea1fb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:32.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.611+0000 7fa3a9d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569ea1e4680 0x5569ea1fb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41312/0 (socket says 172.21.15.116:41312) 2024-09-12T03:18:32.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.611+0000 7fa3a9d9b640 1 -- 172.21.15.116:0/73457345 learned_addr learned my addr 172.21.15.116:0/73457345 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:32.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.611+0000 7fa3a9d9b640 1 --2- 172.21.15.116:0/73457345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569ea1d94a0 0x5569ea1e13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:32.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.611+0000 7fa3aa59c640 1 --2- 172.21.15.116:0/73457345 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569ea1d9dc0 0x5569ea1d4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:32.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.611+0000 7fa3a9d9b640 1 -- 172.21.15.116:0/73457345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569ea1d94a0 msgr2=0x5569ea1e13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:32.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.611+0000 7fa3a9d9b640 1 --2- 172.21.15.116:0/73457345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569ea1d94a0 0x5569ea1e13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.611+0000 7fa3a9d9b640 1 -- 172.21.15.116:0/73457345 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569ea1d9dc0 msgr2=0x5569ea1d4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:32.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.611+0000 7fa3a9d9b640 1 --2- 172.21.15.116:0/73457345 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569ea1d9dc0 0x5569ea1d4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.611+0000 7fa3a9d9b640 1 -- 172.21.15.116:0/73457345 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569ea1d6ae0 con 0x5569ea1e4680 2024-09-12T03:18:32.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.611+0000 7fa3a9d9b640 1 --2- 172.21.15.116:0/73457345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569ea1e4680 0x5569ea1fb9f0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fa3a00154f0 tx=0x7fa3a0022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5c6fb23de3c4a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:32.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.611+0000 7fa393fff640 1 -- 172.21.15.116:0/73457345 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0038030 con 0x5569ea1e4680 2024-09-12T03:18:32.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.611+0000 7fa393fff640 1 -- 172.21.15.116:0/73457345 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3a001c040 con 0x5569ea1e4680 2024-09-12T03:18:32.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.611+0000 7fa393fff640 1 -- 172.21.15.116:0/73457345 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a00177b0 con 0x5569ea1e4680 2024-09-12T03:18:32.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.612+0000 7fa3b32d8740 1 -- 172.21.15.116:0/73457345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569ea1e4680 msgr2=0x5569ea1fb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:32.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.612+0000 7fa3b32d8740 1 --2- 172.21.15.116:0/73457345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569ea1e4680 0x5569ea1fb9f0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fa3a00154f0 tx=0x7fa3a0022040 comp rx=0 tx=0).stop 2024-09-12T03:18:32.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.612+0000 7fa3b32d8740 1 -- 172.21.15.116:0/73457345 shutdown_connections 2024-09-12T03:18:32.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.612+0000 7fa3b32d8740 1 --2- 172.21.15.116:0/73457345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569ea1e4680 0x5569ea1fb9f0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.612+0000 7fa3b32d8740 1 --2- 172.21.15.116:0/73457345 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569ea1d9dc0 0x5569ea1d4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.612+0000 7fa3b32d8740 1 --2- 172.21.15.116:0/73457345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569ea1d94a0 0x5569ea1e13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.612+0000 7fa3b32d8740 1 -- 172.21.15.116:0/73457345 >> 172.21.15.116:0/73457345 conn(0x5569ea1d5a30 msgr2=0x5569ea1d4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:32.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.612+0000 7fa3b32d8740 1 -- 172.21.15.116:0/73457345 shutdown_connections 2024-09-12T03:18:32.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.612+0000 7fa3b32d8740 1 -- 172.21.15.116:0/73457345 wait complete. 2024-09-12T03:18:32.618 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.612+0000 7fa3b32d8740 1 -- start start 2024-09-12T03:18:32.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.613+0000 7fa3b32d8740 1 Processor -- start 2024-09-12T03:18:32.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.613+0000 7fa3b32d8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569ea1d94a0 0x5569ea1ecbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:32.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.613+0000 7fa3b32d8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569ea1d9dc0 0x5569ea1e9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:32.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.613+0000 7fa3b32d8740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569ea1e4680 0x5569ea1ea4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:32.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.613+0000 7fa3b32d8740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac1add80 con 0x5569ea1e4680 2024-09-12T03:18:32.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.613+0000 7fa3b32d8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac1af1f0 con 0x5569ea1d9dc0 2024-09-12T03:18:32.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.613+0000 7fa3b32d8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac1ae450 con 0x5569ea1d94a0 2024-09-12T03:18:32.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.613+0000 7fa3b32d8740 10 client.0 _ll_register_callbacks cb 0x7fa39c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:18:32.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.613+0000 7fa3a9d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569ea1e4680 0x5569ea1ea4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:32.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.613+0000 7fa3a9d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569ea1e4680 0x5569ea1ea4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41320/0 (socket says 172.21.15.116:41320) 2024-09-12T03:18:32.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.613+0000 7fa3a9d9b640 1 -- 172.21.15.116:0/2320810054 learned_addr learned my addr 172.21.15.116:0/2320810054 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:32.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.613+0000 7fa3a9d9b640 1 --2- 172.21.15.116:0/2320810054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569ea1d94a0 0x5569ea1ecbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:32.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.613+0000 7fa3aa59c640 1 --2- 172.21.15.116:0/2320810054 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569ea1d9dc0 0x5569ea1e9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:32.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.613+0000 7fa3a9d9b640 1 -- 172.21.15.116:0/2320810054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569ea1d94a0 msgr2=0x5569ea1ecbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:32.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.613+0000 7fa3a9d9b640 1 --2- 172.21.15.116:0/2320810054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569ea1d94a0 0x5569ea1ecbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.613+0000 7fa3a9d9b640 1 -- 172.21.15.116:0/2320810054 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569ea1d9dc0 msgr2=0x5569ea1e9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:32.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.613+0000 7fa3a9d9b640 1 --2- 172.21.15.116:0/2320810054 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569ea1d9dc0 0x5569ea1e9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.613+0000 7fa3a9d9b640 1 -- 172.21.15.116:0/2320810054 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569ea1eaae0 con 0x5569ea1e4680 2024-09-12T03:18:32.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.613+0000 7fa3aa59c640 1 --2- 172.21.15.116:0/2320810054 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569ea1d9dc0 0x5569ea1e9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:32.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.613+0000 7fa3a9d9b640 1 --2- 172.21.15.116:0/2320810054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569ea1e4680 0x5569ea1ea4a0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fa3a000b010 tx=0x7fa3a00158d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf44e6fb2c209aa6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:32.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.613+0000 7fa391ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:18:32.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.614+0000 7fa391ffb640 1 -- 172.21.15.116:0/2320810054 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0038070 con 0x5569ea1e4680 2024-09-12T03:18:32.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.614+0000 7fa391ffb640 1 -- 172.21.15.116:0/2320810054 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa3a0005bc0 con 0x5569ea1e4680 2024-09-12T03:18:32.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.614+0000 7fa3b32d8740 1 -- 172.21.15.116:0/2320810054 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3a001c040 con 0x5569ea1e4680 2024-09-12T03:18:32.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.614+0000 7fa391ffb640 1 -- 172.21.15.116:0/2320810054 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa3a00095d0 con 0x5569ea1e4680 2024-09-12T03:18:32.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.614+0000 7fa3b32d8740 10 client.19608 fetch_fsmap learned FSMap version 25 2024-09-12T03:18:32.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.614+0000 7fa3b32d8740 1 -- 172.21.15.116:0/2320810054 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5569ea25e830 con 0x5569ea1e4680 2024-09-12T03:18:32.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.614+0000 7fa391ffb640 1 -- 172.21.15.116:0/2320810054 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3a001c040 con 0x5569ea1e4680 2024-09-12T03:18:32.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.614+0000 7fa391ffb640 1 -- 172.21.15.116:0/2320810054 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa3a004b830 con 0x5569ea1e4680 2024-09-12T03:18:32.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.614+0000 7fa3b32d8740 10 client.19608 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:18:32.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.614+0000 7fa3b32d8740 10 client.19608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:18:32.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.614+0000 7fa3b32d8740 20 client.19608 populate_metadata read hostname 'smithi116' 2024-09-12T03:18:32.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.614+0000 7fa3b32d8740 1 --2- 172.21.15.116:0/2320810054 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5569ea25ee90 0x5569ea261280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:32.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.614+0000 7fa3b32d8740 4 client.19608 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:18:32.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.614+0000 7fa3b32d8740 1 -- 172.21.15.116:0/2320810054 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5569ea1e1ab0 con 0x5569ea25ee90 2024-09-12T03:18:32.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.614+0000 7fa3aa59c640 1 --2- 172.21.15.116:0/2320810054 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5569ea25ee90 0x5569ea261280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:32.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.615+0000 7fa391ffb640 10 client.19608 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:18:32.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.615+0000 7fa3aa59c640 1 --2- 172.21.15.116:0/2320810054 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5569ea25ee90 0x5569ea261280 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfe42c245a8ed5c9 server_cookie=3a2d2ebaeff9f207 in_seq=0 out_seq=0 2024-09-12T03:18:32.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.615+0000 7fa391ffb640 1 -- 172.21.15.116:0/2320810054 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5569ea1e1ab0 con 0x5569ea25ee90 2024-09-12T03:18:32.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.615+0000 7fa391ffb640 10 client.19608 handle_command_reply: tid=0 2024-09-12T03:18:32.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.615+0000 7fa3b32d8740 1 client.19608 shutdown 2024-09-12T03:18:32.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.615+0000 7fa3b32d8740 1 -- 172.21.15.116:0/2320810054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569ea1e4680 msgr2=0x5569ea1ea4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:32.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.615+0000 7fa3b32d8740 1 --2- 172.21.15.116:0/2320810054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569ea1e4680 0x5569ea1ea4a0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fa3a000b010 tx=0x7fa3a00158d0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.615+0000 7fa3b32d8740 1 -- 172.21.15.116:0/2320810054 shutdown_connections 2024-09-12T03:18:32.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.615+0000 7fa3b32d8740 1 --2- 172.21.15.116:0/2320810054 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5569ea25ee90 0x5569ea261280 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.615+0000 7fa3b32d8740 1 --2- 172.21.15.116:0/2320810054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5569ea1e4680 0x5569ea1ea4a0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.615+0000 7fa3b32d8740 1 --2- 172.21.15.116:0/2320810054 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5569ea1d9dc0 0x5569ea1e9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.615+0000 7fa3b32d8740 1 --2- 172.21.15.116:0/2320810054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569ea1d94a0 0x5569ea1ecbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.615+0000 7fa3b32d8740 1 -- 172.21.15.116:0/2320810054 >> 172.21.15.116:0/2320810054 conn(0x5569ea1d5a30 msgr2=0x5569ea1d4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:32.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.615+0000 7fa391ffb640 0 client.19608 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:18:32.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.615+0000 7fa3b32d8740 1 -- 172.21.15.116:0/2320810054 shutdown_connections 2024-09-12T03:18:32.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.615+0000 7fa3b32d8740 1 -- 172.21.15.116:0/2320810054 wait complete. 2024-09-12T03:18:32.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.615+0000 7fa3b32d8740 20 client.19608 trim_cache size 0 max 16384 2024-09-12T03:18:32.623 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:18:32.623 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:18:32.623 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": { 2024-09-12T03:18:32.623 INFO:teuthology.orchestra.run.smithi116.stdout: "576fe553-92e6-4b35-9c81-58100750824b": { 2024-09-12T03:18:32.624 INFO:teuthology.orchestra.run.smithi116.stdout: "path": "/", 2024-09-12T03:18:32.624 INFO:teuthology.orchestra.run.smithi116.stdout: "tag": "576fe553-92e6-4b35-9c81-58100750824b", 2024-09-12T03:18:32.624 INFO:teuthology.orchestra.run.smithi116.stdout: "options": "recursive,force" 2024-09-12T03:18:32.624 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:18:32.624 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:18:32.624 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:18:32.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.616+0000 7fa3b0eff640 1 -- 172.21.15.116:0/271266078 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa3940a2b80 msgr2=0x7fa3940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:32.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.616+0000 7fa3b0eff640 1 --2- 172.21.15.116:0/271266078 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa3940a2b80 0x7fa3940a5040 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa3a40cd0c0 tx=0x7fa3a401baf0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.616+0000 7fa3b0eff640 1 -- 172.21.15.116:0/271266078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3ac106a80 msgr2=0x7fa3ac19ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:32.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.616+0000 7fa3b0eff640 1 --2- 172.21.15.116:0/271266078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3ac106a80 0x7fa3ac19ef60 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400cce0 tx=0x7fa3a400a620 comp rx=0 tx=0).stop 2024-09-12T03:18:32.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.616+0000 7fa3b0eff640 1 -- 172.21.15.116:0/271266078 shutdown_connections 2024-09-12T03:18:32.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.616+0000 7fa3b0eff640 1 --2- 172.21.15.116:0/271266078 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa3940a2b80 0x7fa3940a5040 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.616+0000 7fa3b0eff640 1 --2- 172.21.15.116:0/271266078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3ac106a80 0x7fa3ac19ef60 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.616+0000 7fa3b0eff640 1 --2- 172.21.15.116:0/271266078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3ac101420 0x7fa3ac19abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.616+0000 7fa3b0eff640 1 --2- 172.21.15.116:0/271266078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3ac100a80 0x7fa3ac19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.616+0000 7fa3b0eff640 1 -- 172.21.15.116:0/271266078 >> 172.21.15.116:0/271266078 conn(0x7fa3ac0fc7a0 msgr2=0x7fa3ac06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:32.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.616+0000 7fa3b0eff640 1 -- 172.21.15.116:0/271266078 shutdown_connections 2024-09-12T03:18:32.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:32.616+0000 7fa3b0eff640 1 -- 172.21.15.116:0/271266078 wait complete. 2024-09-12T03:18:32.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:32 smithi177 ceph-mon[35305]: scrub queued for path: / 2024-09-12T03:18:32.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:32 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:18:32.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:32 smithi177 ceph-mon[35305]: scrub summary: active paths [/] 2024-09-12T03:18:32.745 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:32 smithi177 ceph-mon[35305]: scrub summary: idle+waiting paths [/] 2024-09-12T03:18:32.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:32 smithi116 ceph-mon[31410]: scrub queued for path: / 2024-09-12T03:18:32.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:32 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:18:32.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:32 smithi116 ceph-mon[31410]: scrub summary: active paths [/] 2024-09-12T03:18:32.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:32 smithi116 ceph-mon[31410]: scrub summary: idle+waiting paths [/] 2024-09-12T03:18:32.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:32 smithi183 ceph-mon[35535]: scrub queued for path: / 2024-09-12T03:18:32.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:32 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:18:32.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:32 smithi183 ceph-mon[35535]: scrub summary: active paths [/] 2024-09-12T03:18:32.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:32 smithi183 ceph-mon[35535]: scrub summary: idle+waiting paths [/] 2024-09-12T03:18:33.411 INFO:tasks.cephfs.filesystem:scrub status for tag:576fe553-92e6-4b35-9c81-58100750824b - {'path': '/', 'tag': '576fe553-92e6-4b35-9c81-58100750824b', 'options': 'recursive,force'} 2024-09-12T03:18:33.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:33 smithi177 ceph-mon[35305]: pgmap v805: 97 pgs: 97 active+clean; 7.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:18:33.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:33 smithi116 ceph-mon[31410]: pgmap v805: 97 pgs: 97 active+clean; 7.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:18:33.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:33 smithi183 ceph-mon[35535]: pgmap v805: 97 pgs: 97 active+clean; 7.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-12T03:18:34.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:34 smithi177 ceph-mon[35305]: scrub summary: idle 2024-09-12T03:18:34.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:34 smithi116 ceph-mon[31410]: scrub summary: idle 2024-09-12T03:18:34.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:34 smithi183 ceph-mon[35535]: scrub summary: idle 2024-09-12T03:18:35.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:35 smithi177 ceph-mon[35305]: pgmap v806: 97 pgs: 97 active+clean; 7.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:18:35.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:35 smithi116 ceph-mon[31410]: pgmap v806: 97 pgs: 97 active+clean; 7.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:18:35.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:35 smithi183 ceph-mon[35535]: pgmap v806: 97 pgs: 97 active+clean; 7.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:18:38.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:38 smithi177 ceph-mon[35305]: pgmap v807: 97 pgs: 97 active+clean; 7.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:18:38.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:38 smithi116 ceph-mon[31410]: pgmap v807: 97 pgs: 97 active+clean; 7.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:18:38.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:38 smithi183 ceph-mon[35535]: pgmap v807: 97 pgs: 97 active+clean; 7.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:18:40.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:40 smithi177 ceph-mon[35305]: pgmap v808: 97 pgs: 97 active+clean; 7.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:18:40.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:40 smithi116 ceph-mon[31410]: pgmap v808: 97 pgs: 97 active+clean; 7.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:18:40.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:40 smithi183 ceph-mon[35535]: pgmap v808: 97 pgs: 97 active+clean; 7.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:18:42.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:42 smithi177 ceph-mon[35305]: pgmap v809: 97 pgs: 97 active+clean; 7.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:18:42.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:42 smithi116 ceph-mon[31410]: pgmap v809: 97 pgs: 97 active+clean; 7.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:18:42.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:42 smithi183 ceph-mon[35535]: pgmap v809: 97 pgs: 97 active+clean; 7.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:18:44.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:44 smithi177 ceph-mon[35305]: pgmap v810: 97 pgs: 97 active+clean; 7.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:18:44.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:44 smithi116 ceph-mon[31410]: pgmap v810: 97 pgs: 97 active+clean; 7.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:18:44.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:44 smithi183 ceph-mon[35535]: pgmap v810: 97 pgs: 97 active+clean; 7.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:18:46.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:46 smithi177 ceph-mon[35305]: pgmap v811: 97 pgs: 97 active+clean; 7.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:18:46.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:46 smithi116 ceph-mon[31410]: pgmap v811: 97 pgs: 97 active+clean; 7.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:18:46.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:46 smithi183 ceph-mon[35535]: pgmap v811: 97 pgs: 97 active+clean; 7.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T03:18:48.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:48 smithi177 ceph-mon[35305]: pgmap v812: 97 pgs: 97 active+clean; 7.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:18:48.755 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:48 smithi183 ceph-mon[35535]: pgmap v812: 97 pgs: 97 active+clean; 7.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:18:48.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:48 smithi116 ceph-mon[31410]: pgmap v812: 97 pgs: 97 active+clean; 7.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:18:50.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:50 smithi177 ceph-mon[35305]: pgmap v813: 97 pgs: 97 active+clean; 7.7 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-12T03:18:50.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:50 smithi116 ceph-mon[31410]: pgmap v813: 97 pgs: 97 active+clean; 7.7 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-12T03:18:50.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:50 smithi183 ceph-mon[35535]: pgmap v813: 97 pgs: 97 active+clean; 7.7 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-09-12T03:18:52.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:52 smithi177 ceph-mon[35305]: pgmap v814: 97 pgs: 97 active+clean; 7.7 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:18:52.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:52 smithi116 ceph-mon[31410]: pgmap v814: 97 pgs: 97 active+clean; 7.7 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:18:52.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:52 smithi183 ceph-mon[35535]: pgmap v814: 97 pgs: 97 active+clean; 7.7 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:18:54.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:54 smithi177 ceph-mon[35305]: pgmap v815: 97 pgs: 97 active+clean; 7.7 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:18:54.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:54 smithi116 ceph-mon[31410]: pgmap v815: 97 pgs: 97 active+clean; 7.7 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:18:54.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:54 smithi183 ceph-mon[35535]: pgmap v815: 97 pgs: 97 active+clean; 7.7 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T03:18:56.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:56 smithi177 ceph-mon[35305]: pgmap v816: 97 pgs: 97 active+clean; 7.7 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:18:56.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:56 smithi116 ceph-mon[31410]: pgmap v816: 97 pgs: 97 active+clean; 7.7 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:18:56.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:56 smithi183 ceph-mon[35535]: pgmap v816: 97 pgs: 97 active+clean; 7.7 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T03:18:57.534 INFO:tasks.workunit.client.0.smithi116.stdout:561/600: open fd = 3 2024-09-12T03:18:57.534 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.534 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.535 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.535 INFO:tasks.workunit.client.0.smithi116.stdout:562/600: open fd = 3 2024-09-12T03:18:57.535 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.535 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.535 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.535 INFO:tasks.workunit.client.0.smithi116.stdout:563/600: open fd = 3 2024-09-12T03:18:57.535 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.535 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.535 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.536 INFO:tasks.workunit.client.0.smithi116.stdout:564/600: open fd = 3 2024-09-12T03:18:57.536 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.536 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.536 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.536 INFO:tasks.workunit.client.0.smithi116.stdout:565/600: open fd = 3 2024-09-12T03:18:57.536 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.536 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.536 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.536 INFO:tasks.workunit.client.0.smithi116.stdout:566/600: open fd = 3 2024-09-12T03:18:57.536 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.536 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.536 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.537 INFO:tasks.workunit.client.0.smithi116.stdout:567/600: open fd = 3 2024-09-12T03:18:57.537 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.537 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.537 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.537 INFO:tasks.workunit.client.0.smithi116.stdout:568/600: open fd = 3 2024-09-12T03:18:57.537 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.537 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.538 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.538 INFO:tasks.workunit.client.0.smithi116.stdout:569/600: open fd = 3 2024-09-12T03:18:57.538 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.538 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.538 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.538 INFO:tasks.workunit.client.0.smithi116.stdout:570/600: open fd = 3 2024-09-12T03:18:57.538 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.538 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.538 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.539 INFO:tasks.workunit.client.0.smithi116.stdout:571/600: open fd = 3 2024-09-12T03:18:57.539 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.539 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.539 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.539 INFO:tasks.workunit.client.0.smithi116.stdout:572/600: open fd = 3 2024-09-12T03:18:57.539 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.539 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.539 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.539 INFO:tasks.workunit.client.0.smithi116.stdout:573/600: open fd = 3 2024-09-12T03:18:57.539 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.539 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.539 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.540 INFO:tasks.workunit.client.0.smithi116.stdout:574/600: open fd = 3 2024-09-12T03:18:57.540 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.540 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.540 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.540 INFO:tasks.workunit.client.0.smithi116.stdout:575/600: open fd = 3 2024-09-12T03:18:57.540 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.541 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.541 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.541 INFO:tasks.workunit.client.0.smithi116.stdout:576/600: open fd = 3 2024-09-12T03:18:57.541 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.541 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.541 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.541 INFO:tasks.workunit.client.0.smithi116.stdout:577/600: open fd = 3 2024-09-12T03:18:57.541 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.541 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.541 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.541 INFO:tasks.workunit.client.0.smithi116.stdout:578/600: open fd = 3 2024-09-12T03:18:57.541 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.542 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.542 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.542 INFO:tasks.workunit.client.0.smithi116.stdout:579/600: open fd = 3 2024-09-12T03:18:57.542 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.542 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.542 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.542 INFO:tasks.workunit.client.0.smithi116.stdout:580/600: open fd = 3 2024-09-12T03:18:57.542 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.542 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.542 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.542 INFO:tasks.workunit.client.0.smithi116.stdout:581/600: open fd = 3 2024-09-12T03:18:57.542 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.543 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.543 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.543 INFO:tasks.workunit.client.0.smithi116.stdout:582/600: open fd = 3 2024-09-12T03:18:57.543 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.543 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.543 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.543 INFO:tasks.workunit.client.0.smithi116.stdout:583/600: open fd = 3 2024-09-12T03:18:57.543 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.543 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.543 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.543 INFO:tasks.workunit.client.0.smithi116.stdout:584/600: open fd = 3 2024-09-12T03:18:57.543 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.544 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.544 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.544 INFO:tasks.workunit.client.0.smithi116.stdout:585/600: open fd = 3 2024-09-12T03:18:57.544 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.545 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.545 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.545 INFO:tasks.workunit.client.0.smithi116.stdout:586/600: open fd = 3 2024-09-12T03:18:57.545 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.545 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.545 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.545 INFO:tasks.workunit.client.0.smithi116.stdout:587/600: open fd = 3 2024-09-12T03:18:57.545 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.545 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.546 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.546 INFO:tasks.workunit.client.0.smithi116.stdout:588/600: open fd = 3 2024-09-12T03:18:57.546 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.546 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.546 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.546 INFO:tasks.workunit.client.0.smithi116.stdout:589/600: open fd = 3 2024-09-12T03:18:57.546 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.546 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.546 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.546 INFO:tasks.workunit.client.0.smithi116.stdout:590/600: open fd = 3 2024-09-12T03:18:57.546 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.546 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.546 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.546 INFO:tasks.workunit.client.0.smithi116.stdout:591/600: open fd = 3 2024-09-12T03:18:57.547 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.547 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.547 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.547 INFO:tasks.workunit.client.0.smithi116.stdout:592/600: open fd = 3 2024-09-12T03:18:57.547 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.547 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.547 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.547 INFO:tasks.workunit.client.0.smithi116.stdout:593/600: open fd = 3 2024-09-12T03:18:57.547 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.547 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.547 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.547 INFO:tasks.workunit.client.0.smithi116.stdout:594/600: open fd = 3 2024-09-12T03:18:57.547 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.548 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.548 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.548 INFO:tasks.workunit.client.0.smithi116.stdout:595/600: open fd = 3 2024-09-12T03:18:57.548 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.548 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.548 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.548 INFO:tasks.workunit.client.0.smithi116.stdout:596/600: open fd = 3 2024-09-12T03:18:57.548 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.548 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.548 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.548 INFO:tasks.workunit.client.0.smithi116.stdout:597/600: open fd = 3 2024-09-12T03:18:57.549 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.549 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.549 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.549 INFO:tasks.workunit.client.0.smithi116.stdout:598/600: open fd = 3 2024-09-12T03:18:57.549 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.549 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.549 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.549 INFO:tasks.workunit.client.0.smithi116.stdout:599/600: open fd = 3 2024-09-12T03:18:57.549 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.549 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.549 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.549 INFO:tasks.workunit.client.0.smithi116.stdout:600/600: open fd = 3 2024-09-12T03:18:57.549 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.549 INFO:tasks.workunit.client.0.smithi116.stdout:write ret = 32 2024-09-12T03:18:57.550 INFO:tasks.workunit.client.0.smithi116.stdout:pread ret = 64 2024-09-12T03:18:57.550 INFO:tasks.workunit.client.0.smithi116.stderr:+ echo OK 2024-09-12T03:18:57.551 INFO:tasks.workunit.client.0.smithi116.stdout:OK 2024-09-12T03:18:57.551 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-12T03:18:57.551 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-12T03:18:57.597 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-09-12T03:18:57.598 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-12T03:18:57.935 DEBUG:teuthology.parallel:result is None 2024-09-12T03:18:57.936 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-12T03:18:57.967 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-12T03:18:57.967 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-12T03:18:57.978 INFO:tasks.quiescer:joining Quiescers 2024-09-12T03:18:57.978 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-12T03:18:57.979 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:18:58.255 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:58.636 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:58 smithi116 ceph-mon[31410]: pgmap v817: 97 pgs: 97 active+clean; 7.7 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:18:58.744 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:58 smithi177 ceph-mon[35305]: pgmap v817: 97 pgs: 97 active+clean; 7.7 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:18:58.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.886+0000 7f60d7b26640 1 -- 172.21.15.116:0/1225035905 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60d0106f20 msgr2=0x7f60d010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:58.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.886+0000 7f60d7b26640 1 --2- 172.21.15.116:0/1225035905 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60d0106f20 0x7f60d010eaa0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009900 tx=0x7f60c802a410 comp rx=0 tx=0).stop 2024-09-12T03:18:58.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.886+0000 7f60d7b26640 1 -- 172.21.15.116:0/1225035905 shutdown_connections 2024-09-12T03:18:58.902 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.886+0000 7f60d7b26640 1 --2- 172.21.15.116:0/1225035905 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60d0106f20 0x7f60d010eaa0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:58.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.886+0000 7f60d7b26640 1 --2- 172.21.15.116:0/1225035905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60d01027a0 0x7f60d01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:58.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.886+0000 7f60d7b26640 1 --2- 172.21.15.116:0/1225035905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60d0101df0 0x7f60d01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:58.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.886+0000 7f60d7b26640 1 -- 172.21.15.116:0/1225035905 >> 172.21.15.116:0/1225035905 conn(0x7f60d00fd8f0 msgr2=0x7f60d00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:58.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.886+0000 7f60d7b26640 1 -- 172.21.15.116:0/1225035905 shutdown_connections 2024-09-12T03:18:58.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.886+0000 7f60d7b26640 1 -- 172.21.15.116:0/1225035905 wait complete. 2024-09-12T03:18:58.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.886+0000 7f60d7b26640 1 Processor -- start 2024-09-12T03:18:58.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.886+0000 7f60d7b26640 1 -- start start 2024-09-12T03:18:58.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.887+0000 7f60d7b26640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60d0101df0 0x7f60d0198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:58.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.887+0000 7f60d7b26640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60d01027a0 0x7f60d0198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:58.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.887+0000 7f60d7b26640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60d0106f20 0x7f60d019cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:58.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.887+0000 7f60d7b26640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f60d0112020 con 0x7f60d0106f20 2024-09-12T03:18:58.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.887+0000 7f60d7b26640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f60d0111ea0 con 0x7f60d0101df0 2024-09-12T03:18:58.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.887+0000 7f60d7b26640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f60d01121a0 con 0x7f60d01027a0 2024-09-12T03:18:58.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.887+0000 7f60d60c3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60d0106f20 0x7f60d019cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:58.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.887+0000 7f60d60c3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60d0106f20 0x7f60d019cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60266/0 (socket says 172.21.15.116:60266) 2024-09-12T03:18:58.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.887+0000 7f60d60c3640 1 -- 172.21.15.116:0/4123131039 learned_addr learned my addr 172.21.15.116:0/4123131039 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:18:58.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.887+0000 7f60d58c2640 1 --2- 172.21.15.116:0/4123131039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60d0101df0 0x7f60d0198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:58.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.887+0000 7f60d50c1640 1 --2- 172.21.15.116:0/4123131039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60d01027a0 0x7f60d0198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:58.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.887+0000 7f60d60c3640 1 -- 172.21.15.116:0/4123131039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60d01027a0 msgr2=0x7f60d0198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:58.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.887+0000 7f60d60c3640 1 --2- 172.21.15.116:0/4123131039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60d01027a0 0x7f60d0198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:58.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.887+0000 7f60d60c3640 1 -- 172.21.15.116:0/4123131039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60d0101df0 msgr2=0x7f60d0198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:58.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.887+0000 7f60d60c3640 1 --2- 172.21.15.116:0/4123131039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60d0101df0 0x7f60d0198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:58.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.887+0000 7f60d60c3640 1 -- 172.21.15.116:0/4123131039 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d019d5f0 con 0x7f60d0106f20 2024-09-12T03:18:58.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.887+0000 7f60d60c3640 1 --2- 172.21.15.116:0/4123131039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60d0106f20 0x7f60d019cf10 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009230 tx=0x7f60c80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:58.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.888+0000 7f60ceffd640 1 -- 172.21.15.116:0/4123131039 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c803f030 con 0x7f60d0106f20 2024-09-12T03:18:58.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.888+0000 7f60ceffd640 1 -- 172.21.15.116:0/4123131039 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60c8003ce0 con 0x7f60d0106f20 2024-09-12T03:18:58.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.888+0000 7f60d7b26640 1 -- 172.21.15.116:0/4123131039 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d019d880 con 0x7f60d0106f20 2024-09-12T03:18:58.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.888+0000 7f60ceffd640 1 -- 172.21.15.116:0/4123131039 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c8035070 con 0x7f60d0106f20 2024-09-12T03:18:58.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.888+0000 7f60d7b26640 1 -- 172.21.15.116:0/4123131039 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d01a51c0 con 0x7f60d0106f20 2024-09-12T03:18:58.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.889+0000 7f60d7b26640 1 -- 172.21.15.116:0/4123131039 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60d01032d0 con 0x7f60d0106f20 2024-09-12T03:18:58.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.890+0000 7f60ceffd640 1 -- 172.21.15.116:0/4123131039 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f60c8042030 con 0x7f60d0106f20 2024-09-12T03:18:58.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.890+0000 7f60ceffd640 1 --2- 172.21.15.116:0/4123131039 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f60b80a2b80 0x7f60b80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:58.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.890+0000 7f60d58c2640 1 --2- 172.21.15.116:0/4123131039 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f60b80a2b80 0x7f60b80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:58.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.890+0000 7f60ceffd640 1 -- 172.21.15.116:0/4123131039 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f60c80ef820 con 0x7f60d0106f20 2024-09-12T03:18:58.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.891+0000 7f60d58c2640 1 --2- 172.21.15.116:0/4123131039 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f60b80a2b80 0x7f60b80a5040 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f60c000a400 tx=0x7f60c000d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:58.905 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:58.895+0000 7f60ceffd640 1 -- 172.21.15.116:0/4123131039 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f60c80bc7b0 con 0x7f60d0106f20 2024-09-12T03:18:58.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:58 smithi183 ceph-mon[35535]: pgmap v817: 97 pgs: 97 active+clean; 7.7 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:18:59.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:59.096+0000 7f60d7b26640 1 -- 172.21.15.116:0/4123131039 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60d010a100 con 0x7f60d0106f20 2024-09-12T03:18:59.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:59.096+0000 7f60ceffd640 1 -- 172.21.15.116:0/4123131039 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f60c80bc0b0 con 0x7f60d0106f20 2024-09-12T03:18:59.097 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:18:59.097 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:18:59.097 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:18:59.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:59.098+0000 7f60d7b26640 1 -- 172.21.15.116:0/4123131039 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f60b80a2b80 msgr2=0x7f60b80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:59.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:59.098+0000 7f60d7b26640 1 --2- 172.21.15.116:0/4123131039 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f60b80a2b80 0x7f60b80a5040 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f60c000a400 tx=0x7f60c000d040 comp rx=0 tx=0).stop 2024-09-12T03:18:59.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:59.098+0000 7f60d7b26640 1 -- 172.21.15.116:0/4123131039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60d0106f20 msgr2=0x7f60d019cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:59.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:59.098+0000 7f60d7b26640 1 --2- 172.21.15.116:0/4123131039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60d0106f20 0x7f60d019cf10 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009230 tx=0x7f60c80042f0 comp rx=0 tx=0).stop 2024-09-12T03:18:59.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:59.098+0000 7f60d7b26640 1 -- 172.21.15.116:0/4123131039 shutdown_connections 2024-09-12T03:18:59.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:59.098+0000 7f60d7b26640 1 --2- 172.21.15.116:0/4123131039 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f60b80a2b80 0x7f60b80a5040 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:59.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:59.098+0000 7f60d7b26640 1 --2- 172.21.15.116:0/4123131039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60d0106f20 0x7f60d019cf10 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:59.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:59.098+0000 7f60d7b26640 1 --2- 172.21.15.116:0/4123131039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60d01027a0 0x7f60d0198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:59.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:59.098+0000 7f60d7b26640 1 --2- 172.21.15.116:0/4123131039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60d0101df0 0x7f60d0198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:59.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:59.098+0000 7f60d7b26640 1 -- 172.21.15.116:0/4123131039 >> 172.21.15.116:0/4123131039 conn(0x7f60d00fd8f0 msgr2=0x7f60d00fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:59.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:59.098+0000 7f60d7b26640 1 -- 172.21.15.116:0/4123131039 shutdown_connections 2024-09-12T03:18:59.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:18:59.098+0000 7f60d7b26640 1 -- 172.21.15.116:0/4123131039 wait complete. 2024-09-12T03:18:59.603 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:18:59 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/4123131039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:59.923 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --cancel --all' 2024-09-12T03:18:59.924 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --cancel --all 2024-09-12T03:18:59.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:18:59 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/4123131039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:59.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:18:59 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/4123131039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:00.180 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:00.598 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:00 smithi116 ceph-mon[31410]: pgmap v818: 97 pgs: 97 active+clean; 7.8 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-12T03:19:00.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.683+0000 7f2df0ae3640 1 -- 172.21.15.116:0/4020932574 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2dec102ef0 msgr2=0x7f2dec10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:00.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.683+0000 7f2df0ae3640 1 --2- 172.21.15.116:0/4020932574 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2dec102ef0 0x7f2dec10da40 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f2de4009900 tx=0x7f2de402a480 comp rx=0 tx=0).stop 2024-09-12T03:19:00.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.683+0000 7f2df0ae3640 1 -- 172.21.15.116:0/4020932574 shutdown_connections 2024-09-12T03:19:00.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.683+0000 7f2df0ae3640 1 --2- 172.21.15.116:0/4020932574 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2dec102ef0 0x7f2dec10da40 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.683+0000 7f2df0ae3640 1 --2- 172.21.15.116:0/4020932574 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dec102570 0x7f2dec1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.683+0000 7f2df0ae3640 1 --2- 172.21.15.116:0/4020932574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2dec100e20 0x7f2dec101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.683+0000 7f2df0ae3640 1 -- 172.21.15.116:0/4020932574 >> 172.21.15.116:0/4020932574 conn(0x7f2dec0fc7d0 msgr2=0x7f2dec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:00.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.683+0000 7f2df0ae3640 1 -- 172.21.15.116:0/4020932574 shutdown_connections 2024-09-12T03:19:00.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.683+0000 7f2df0ae3640 1 -- 172.21.15.116:0/4020932574 wait complete. 2024-09-12T03:19:00.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.683+0000 7f2df0ae3640 1 Processor -- start 2024-09-12T03:19:00.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.683+0000 7f2df0ae3640 1 -- start start 2024-09-12T03:19:00.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.684+0000 7f2df0ae3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2dec100e20 0x7f2dec198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:00.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.684+0000 7f2df0ae3640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dec102570 0x7f2dec198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:00.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.684+0000 7f2df0ae3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2dec102ef0 0x7f2dec19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:00.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.684+0000 7f2df0ae3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1120a0 con 0x7f2dec100e20 2024-09-12T03:19:00.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.684+0000 7f2df0ae3640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec111f20 con 0x7f2dec102570 2024-09-12T03:19:00.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.684+0000 7f2df0ae3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec112220 con 0x7f2dec102ef0 2024-09-12T03:19:00.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.684+0000 7f2dea59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2dec100e20 0x7f2dec198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:00.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.684+0000 7f2dea59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2dec100e20 0x7f2dec198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60288/0 (socket says 172.21.15.116:60288) 2024-09-12T03:19:00.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.684+0000 7f2dea59c640 1 -- 172.21.15.116:0/538299115 learned_addr learned my addr 172.21.15.116:0/538299115 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:00.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.684+0000 7f2de9d9b640 1 --2- 172.21.15.116:0/538299115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dec102570 0x7f2dec198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:00.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.684+0000 7f2dead9d640 1 --2- 172.21.15.116:0/538299115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2dec102ef0 0x7f2dec19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:00.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.684+0000 7f2dea59c640 1 -- 172.21.15.116:0/538299115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2dec102ef0 msgr2=0x7f2dec19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:00.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.684+0000 7f2dea59c640 1 --2- 172.21.15.116:0/538299115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2dec102ef0 0x7f2dec19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.684+0000 7f2dea59c640 1 -- 172.21.15.116:0/538299115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dec102570 msgr2=0x7f2dec198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:00.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.684+0000 7f2dea59c640 1 --2- 172.21.15.116:0/538299115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dec102570 0x7f2dec198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.684+0000 7f2dea59c640 1 -- 172.21.15.116:0/538299115 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dec19d3f0 con 0x7f2dec100e20 2024-09-12T03:19:00.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.684+0000 7f2de9d9b640 1 --2- 172.21.15.116:0/538299115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dec102570 0x7f2dec198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:00.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.684+0000 7f2dea59c640 1 --2- 172.21.15.116:0/538299115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2dec100e20 0x7f2dec198440 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc00ba50 tx=0x7f2ddc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:00.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.684+0000 7f2ddb7fe640 1 -- 172.21.15.116:0/538299115 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc019070 con 0x7f2dec100e20 2024-09-12T03:19:00.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.684+0000 7f2df0ae3640 1 -- 172.21.15.116:0/538299115 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dec19d6e0 con 0x7f2dec100e20 2024-09-12T03:19:00.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.684+0000 7f2ddb7fe640 1 -- 172.21.15.116:0/538299115 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ddc00cbe0 con 0x7f2dec100e20 2024-09-12T03:19:00.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.685+0000 7f2df0ae3640 1 -- 172.21.15.116:0/538299115 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dec1a4fc0 con 0x7f2dec100e20 2024-09-12T03:19:00.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.685+0000 7f2ddb7fe640 1 -- 172.21.15.116:0/538299115 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc0076a0 con 0x7f2dec100e20 2024-09-12T03:19:00.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.687+0000 7f2ddb7fe640 1 -- 172.21.15.116:0/538299115 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2ddc01c070 con 0x7f2dec100e20 2024-09-12T03:19:00.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.687+0000 7f2df2ebc740 1 Processor -- start 2024-09-12T03:19:00.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.688+0000 7f2df2ebc740 1 -- start start 2024-09-12T03:19:00.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.688+0000 7f2df2ebc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584241c8a00 0x5584241c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:00.709 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.688+0000 7f2df2ebc740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5584241c9320 0x5584241d36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:00.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.688+0000 7f2df2ebc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584241d3be0 0x5584241d5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:00.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.688+0000 7f2df2ebc740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55842412d200 con 0x5584241c9320 2024-09-12T03:19:00.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.688+0000 7f2df2ebc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5584240d3dc0 con 0x5584241c8a00 2024-09-12T03:19:00.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.688+0000 7f2df2ebc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5584241291c0 con 0x5584241d3be0 2024-09-12T03:19:00.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.688+0000 7f2dea59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5584241c9320 0x5584241d36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:00.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.688+0000 7f2ddb7fe640 1 --2- 172.21.15.116:0/538299115 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2dd40a2b80 0x7f2dd40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:00.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.688+0000 7f2dea59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5584241c9320 0x5584241d36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60294/0 (socket says 172.21.15.116:60294) 2024-09-12T03:19:00.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.688+0000 7f2dea59c640 1 -- 172.21.15.116:0/310707509 learned_addr learned my addr 172.21.15.116:0/310707509 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:00.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.688+0000 7f2de9d9b640 1 --2- 172.21.15.116:0/310707509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584241d3be0 0x5584241d5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:00.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.688+0000 7f2dead9d640 1 --2- 172.21.15.116:0/538299115 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2dd40a2b80 0x7f2dd40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:00.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.688+0000 7f2dead9d640 1 --2- 172.21.15.116:0/310707509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584241c8a00 0x5584241c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:00.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.688+0000 7f2ddb7fe640 1 -- 172.21.15.116:0/538299115 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f2ddc0ce720 con 0x7f2dec100e20 2024-09-12T03:19:00.710 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.688+0000 7f2dea59c640 1 -- 172.21.15.116:0/310707509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584241d3be0 msgr2=0x5584241d5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:00.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.688+0000 7f2dea59c640 1 --2- 172.21.15.116:0/310707509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584241d3be0 0x5584241d5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.688+0000 7f2dea59c640 1 -- 172.21.15.116:0/310707509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584241c8a00 msgr2=0x5584241c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:00.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.688+0000 7f2dea59c640 1 --2- 172.21.15.116:0/310707509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584241c8a00 0x5584241c8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.688+0000 7f2dea59c640 1 -- 172.21.15.116:0/310707509 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558423e9e620 con 0x5584241c9320 2024-09-12T03:19:00.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.688+0000 7f2dead9d640 1 --2- 172.21.15.116:0/310707509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584241c8a00 0x5584241c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:00.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.688+0000 7f2de9d9b640 1 --2- 172.21.15.116:0/310707509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584241d3be0 0x5584241d5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:19:00.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.689+0000 7f2dea59c640 1 --2- 172.21.15.116:0/310707509 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5584241c9320 0x5584241d36a0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc0a3690 tx=0x7f2ddc0bb180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5fd75592e5434c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:00.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.689+0000 7f2dead9d640 1 --2- 172.21.15.116:0/538299115 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2dd40a2b80 0x7f2dd40a5040 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2de40314e0 tx=0x7f2de40058b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:00.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.689+0000 7f2dd8ff9640 1 -- 172.21.15.116:0/310707509 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc09af50 con 0x5584241c9320 2024-09-12T03:19:00.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.689+0000 7f2dd8ff9640 1 -- 172.21.15.116:0/310707509 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ddc09f300 con 0x5584241c9320 2024-09-12T03:19:00.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.689+0000 7f2dd8ff9640 1 -- 172.21.15.116:0/310707509 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc09b1b0 con 0x5584241c9320 2024-09-12T03:19:00.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.689+0000 7f2df2ebc740 1 -- 172.21.15.116:0/310707509 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5584241c9320 msgr2=0x5584241d36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:00.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.689+0000 7f2df2ebc740 1 --2- 172.21.15.116:0/310707509 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5584241c9320 0x5584241d36a0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc0a3690 tx=0x7f2ddc0bb180 comp rx=0 tx=0).stop 2024-09-12T03:19:00.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.690+0000 7f2df2ebc740 1 -- 172.21.15.116:0/310707509 shutdown_connections 2024-09-12T03:19:00.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.690+0000 7f2df2ebc740 1 --2- 172.21.15.116:0/310707509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584241d3be0 0x5584241d5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.690+0000 7f2df2ebc740 1 --2- 172.21.15.116:0/310707509 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5584241c9320 0x5584241d36a0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.690+0000 7f2df2ebc740 1 --2- 172.21.15.116:0/310707509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584241c8a00 0x5584241c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.690+0000 7f2df2ebc740 1 -- 172.21.15.116:0/310707509 >> 172.21.15.116:0/310707509 conn(0x5584241c4f90 msgr2=0x5584241c5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:00.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.690+0000 7f2df2ebc740 1 -- 172.21.15.116:0/310707509 shutdown_connections 2024-09-12T03:19:00.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.690+0000 7f2df2ebc740 1 -- 172.21.15.116:0/310707509 wait complete. 2024-09-12T03:19:00.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.690+0000 7f2df2ebc740 1 -- start start 2024-09-12T03:19:00.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.690+0000 7f2df2ebc740 1 Processor -- start 2024-09-12T03:19:00.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.690+0000 7f2df2ebc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584241c8a00 0x5584241d3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:00.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.690+0000 7f2df2ebc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584241c9320 0x5584241d0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:00.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.690+0000 7f2df2ebc740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5584241d3be0 0x5584241d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:00.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.690+0000 7f2df2ebc740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5584241d67b0 con 0x5584241d3be0 2024-09-12T03:19:00.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.690+0000 7f2df2ebc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5584240c7c50 con 0x5584241c8a00 2024-09-12T03:19:00.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.690+0000 7f2df2ebc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5584241d6930 con 0x5584241c9320 2024-09-12T03:19:00.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.690+0000 7f2df2ebc740 10 client.0 _ll_register_callbacks cb 0x5584240d9ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:00.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.690+0000 7f2dead9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5584241d3be0 0x5584241d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:00.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.690+0000 7f2dead9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5584241d3be0 0x5584241d0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60310/0 (socket says 172.21.15.116:60310) 2024-09-12T03:19:00.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.690+0000 7f2dead9d640 1 -- 172.21.15.116:0/1863772453 learned_addr learned my addr 172.21.15.116:0/1863772453 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:00.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.690+0000 7f2dea59c640 1 --2- 172.21.15.116:0/1863772453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584241c8a00 0x5584241d3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:00.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.691+0000 7f2de9d9b640 1 --2- 172.21.15.116:0/1863772453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584241c9320 0x5584241d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:00.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.691+0000 7f2dead9d640 1 -- 172.21.15.116:0/1863772453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584241c9320 msgr2=0x5584241d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:00.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.691+0000 7f2dead9d640 1 --2- 172.21.15.116:0/1863772453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584241c9320 0x5584241d0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.691+0000 7f2dead9d640 1 -- 172.21.15.116:0/1863772453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584241c8a00 msgr2=0x5584241d3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:00.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.691+0000 7f2dead9d640 1 --2- 172.21.15.116:0/1863772453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584241c8a00 0x5584241d3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.691+0000 7f2dead9d640 1 -- 172.21.15.116:0/1863772453 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584241d1600 con 0x5584241d3be0 2024-09-12T03:19:00.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.691+0000 7f2dead9d640 1 --2- 172.21.15.116:0/1863772453 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5584241d3be0 0x5584241d0dd0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f2de4033000 tx=0x7f2de4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=441bcb916935575f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:00.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.691+0000 7f2dd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:19:00.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.691+0000 7f2dd1ffb640 1 -- 172.21.15.116:0/1863772453 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de404f070 con 0x5584241d3be0 2024-09-12T03:19:00.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.691+0000 7f2dd1ffb640 1 -- 172.21.15.116:0/1863772453 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f2de4031cd0 con 0x5584241d3be0 2024-09-12T03:19:00.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.691+0000 7f2df2ebc740 1 -- 172.21.15.116:0/1863772453 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ddc09f300 con 0x5584241d3be0 2024-09-12T03:19:00.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.691+0000 7f2dd1ffb640 1 -- 172.21.15.116:0/1863772453 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2de4035030 con 0x5584241d3be0 2024-09-12T03:19:00.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.691+0000 7f2df2ebc740 10 client.19626 fetch_fsmap learned FSMap version 25 2024-09-12T03:19:00.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.691+0000 7f2df2ebc740 1 -- 172.21.15.116:0/1863772453 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5584241e23a0 con 0x5584241d3be0 2024-09-12T03:19:00.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.691+0000 7f2dd1ffb640 1 -- 172.21.15.116:0/1863772453 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ddc09f300 con 0x5584241d3be0 2024-09-12T03:19:00.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.691+0000 7f2dd1ffb640 1 -- 172.21.15.116:0/1863772453 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2de403c440 con 0x5584241d3be0 2024-09-12T03:19:00.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.692+0000 7f2df2ebc740 10 client.19626 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:19:00.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.692+0000 7f2df2ebc740 10 client.19626 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:19:00.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.692+0000 7f2df2ebc740 20 client.19626 populate_metadata read hostname 'smithi116' 2024-09-12T03:19:00.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.692+0000 7f2df2ebc740 1 --2- 172.21.15.116:0/1863772453 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5584241e3850 0x5584241e5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:00.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.692+0000 7f2df2ebc740 4 client.19626 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:19:00.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.692+0000 7f2df2ebc740 1 -- 172.21.15.116:0/1863772453 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55842413a770 con 0x5584241e3850 2024-09-12T03:19:00.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.692+0000 7f2dea59c640 1 --2- 172.21.15.116:0/1863772453 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5584241e3850 0x5584241e5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:00.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.692+0000 7f2dd1ffb640 10 client.19626 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:19:00.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.692+0000 7f2dea59c640 1 --2- 172.21.15.116:0/1863772453 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5584241e3850 0x5584241e5cb0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c88ed7d311beba4 server_cookie=800c4b8d4e394ca5 in_seq=0 out_seq=0 2024-09-12T03:19:00.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.693+0000 7f2dd1ffb640 1 -- 172.21.15.116:0/1863772453 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55842413a770 con 0x5584241e3850 2024-09-12T03:19:00.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.693+0000 7f2dd1ffb640 10 client.19626 handle_command_reply: tid=0 2024-09-12T03:19:00.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.693+0000 7f2df2ebc740 1 client.19626 shutdown 2024-09-12T03:19:00.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.693+0000 7f2df2ebc740 1 -- 172.21.15.116:0/1863772453 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5584241d3be0 msgr2=0x5584241d0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:00.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.693+0000 7f2df2ebc740 1 --2- 172.21.15.116:0/1863772453 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5584241d3be0 0x5584241d0dd0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f2de4033000 tx=0x7f2de4009260 comp rx=0 tx=0).stop 2024-09-12T03:19:00.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.693+0000 7f2df2ebc740 1 -- 172.21.15.116:0/1863772453 shutdown_connections 2024-09-12T03:19:00.716 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.693+0000 7f2df2ebc740 1 --2- 172.21.15.116:0/1863772453 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x5584241e3850 0x5584241e5cb0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.693+0000 7f2df2ebc740 1 --2- 172.21.15.116:0/1863772453 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5584241d3be0 0x5584241d0dd0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.693+0000 7f2df2ebc740 1 --2- 172.21.15.116:0/1863772453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584241c9320 0x5584241d0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.693+0000 7f2dd1ffb640 0 client.19626 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:19:00.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.693+0000 7f2df2ebc740 1 --2- 172.21.15.116:0/1863772453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584241c8a00 0x5584241d3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.693+0000 7f2df2ebc740 1 -- 172.21.15.116:0/1863772453 >> 172.21.15.116:0/1863772453 conn(0x5584241c4f90 msgr2=0x5584241c41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:00.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.693+0000 7f2df2ebc740 1 -- 172.21.15.116:0/1863772453 shutdown_connections 2024-09-12T03:19:00.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.693+0000 7f2df2ebc740 1 -- 172.21.15.116:0/1863772453 wait complete. 2024-09-12T03:19:00.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.693+0000 7f2df2ebc740 20 client.19626 trim_cache size 0 max 16384 2024-09-12T03:19:00.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.710+0000 7f2df2ebc740 1 Processor -- start 2024-09-12T03:19:00.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.710+0000 7f2df2ebc740 1 -- start start 2024-09-12T03:19:00.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.710+0000 7f2df2ebc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584241c8a00 0x5584241c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:00.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.710+0000 7f2df2ebc740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5584241c9320 0x5584241d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:00.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.710+0000 7f2df2ebc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584241d3be0 0x5584241eb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:00.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.710+0000 7f2df2ebc740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1afdc0 con 0x5584241c9320 2024-09-12T03:19:00.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.710+0000 7f2df2ebc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1ad690 con 0x5584241d3be0 2024-09-12T03:19:00.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.710+0000 7f2df2ebc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1ab4e0 con 0x5584241c8a00 2024-09-12T03:19:00.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.710+0000 7f2de9d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5584241c9320 0x5584241d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:00.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.710+0000 7f2de9d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5584241c9320 0x5584241d27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60326/0 (socket says 172.21.15.116:60326) 2024-09-12T03:19:00.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.710+0000 7f2de9d9b640 1 -- 172.21.15.116:0/2772915073 learned_addr learned my addr 172.21.15.116:0/2772915073 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:00.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.710+0000 7f2dea59c640 1 --2- 172.21.15.116:0/2772915073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584241c8a00 0x5584241c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:00.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.710+0000 7f2dead9d640 1 --2- 172.21.15.116:0/2772915073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584241d3be0 0x5584241eb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:00.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.710+0000 7f2de9d9b640 1 -- 172.21.15.116:0/2772915073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584241c8a00 msgr2=0x5584241c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:00.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.710+0000 7f2de9d9b640 1 --2- 172.21.15.116:0/2772915073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584241c8a00 0x5584241c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.710+0000 7f2de9d9b640 1 -- 172.21.15.116:0/2772915073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584241d3be0 msgr2=0x5584241eb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:00.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.710+0000 7f2de9d9b640 1 --2- 172.21.15.116:0/2772915073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584241d3be0 0x5584241eb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.710+0000 7f2de9d9b640 1 -- 172.21.15.116:0/2772915073 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584241cf7e0 con 0x5584241c9320 2024-09-12T03:19:00.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.711+0000 7f2dead9d640 1 --2- 172.21.15.116:0/2772915073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584241d3be0 0x5584241eb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:00.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.711+0000 7f2dea59c640 1 --2- 172.21.15.116:0/2772915073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584241c8a00 0x5584241c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:00.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.711+0000 7f2de9d9b640 1 --2- 172.21.15.116:0/2772915073 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5584241c9320 0x5584241d27a0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f2dec0fec10 tx=0x7f2de0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37eca50f42df61a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:00.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.711+0000 7f2dd3fff640 1 -- 172.21.15.116:0/2772915073 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de0004050 con 0x5584241c9320 2024-09-12T03:19:00.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.711+0000 7f2dd3fff640 1 -- 172.21.15.116:0/2772915073 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de00041f0 con 0x5584241c9320 2024-09-12T03:19:00.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.711+0000 7f2dd3fff640 1 -- 172.21.15.116:0/2772915073 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de001a710 con 0x5584241c9320 2024-09-12T03:19:00.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.711+0000 7f2df2ebc740 1 -- 172.21.15.116:0/2772915073 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5584241c9320 msgr2=0x5584241d27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:00.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.711+0000 7f2df2ebc740 1 --2- 172.21.15.116:0/2772915073 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5584241c9320 0x5584241d27a0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f2dec0fec10 tx=0x7f2de0011810 comp rx=0 tx=0).stop 2024-09-12T03:19:00.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.712+0000 7f2df2ebc740 1 -- 172.21.15.116:0/2772915073 shutdown_connections 2024-09-12T03:19:00.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.712+0000 7f2df2ebc740 1 --2- 172.21.15.116:0/2772915073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584241d3be0 0x5584241eb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.712+0000 7f2df2ebc740 1 --2- 172.21.15.116:0/2772915073 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5584241c9320 0x5584241d27a0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.712+0000 7f2df2ebc740 1 --2- 172.21.15.116:0/2772915073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584241c8a00 0x5584241c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.712+0000 7f2df2ebc740 1 -- 172.21.15.116:0/2772915073 >> 172.21.15.116:0/2772915073 conn(0x5584241c4f90 msgr2=0x5584241d2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:00.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.712+0000 7f2df2ebc740 1 -- 172.21.15.116:0/2772915073 shutdown_connections 2024-09-12T03:19:00.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.712+0000 7f2df2ebc740 1 -- 172.21.15.116:0/2772915073 wait complete. 2024-09-12T03:19:00.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.712+0000 7f2df2ebc740 1 -- start start 2024-09-12T03:19:00.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.712+0000 7f2df2ebc740 1 Processor -- start 2024-09-12T03:19:00.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.712+0000 7f2df2ebc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584241c8a00 0x5584241dc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:00.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.712+0000 7f2df2ebc740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5584241c9320 0x5584241d7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:00.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.712+0000 7f2df2ebc740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584241d3be0 0x5584241d78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:00.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.712+0000 7f2df2ebc740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1abb50 con 0x5584241c9320 2024-09-12T03:19:00.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.712+0000 7f2df2ebc740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1acfc0 con 0x5584241d3be0 2024-09-12T03:19:00.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.712+0000 7f2df2ebc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1ac220 con 0x5584241c8a00 2024-09-12T03:19:00.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.712+0000 7f2df2ebc740 10 client.0 _ll_register_callbacks cb 0x7f2de4003aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:00.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.712+0000 7f2de9d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5584241c9320 0x5584241d7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:00.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.712+0000 7f2de9d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5584241c9320 0x5584241d7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60342/0 (socket says 172.21.15.116:60342) 2024-09-12T03:19:00.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.712+0000 7f2de9d9b640 1 -- 172.21.15.116:0/3917375507 learned_addr learned my addr 172.21.15.116:0/3917375507 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:00.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.712+0000 7f2dea59c640 1 --2- 172.21.15.116:0/3917375507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584241c8a00 0x5584241dc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:00.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.713+0000 7f2dead9d640 1 --2- 172.21.15.116:0/3917375507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584241d3be0 0x5584241d78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:00.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.713+0000 7f2de9d9b640 1 -- 172.21.15.116:0/3917375507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584241c8a00 msgr2=0x5584241dc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:00.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.713+0000 7f2de9d9b640 1 --2- 172.21.15.116:0/3917375507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584241c8a00 0x5584241dc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.713+0000 7f2de9d9b640 1 -- 172.21.15.116:0/3917375507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584241d3be0 msgr2=0x5584241d78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:00.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.713+0000 7f2de9d9b640 1 --2- 172.21.15.116:0/3917375507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584241d3be0 0x5584241d78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.713+0000 7f2de9d9b640 1 -- 172.21.15.116:0/3917375507 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584241d7e10 con 0x5584241c9320 2024-09-12T03:19:00.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.713+0000 7f2de9d9b640 1 --2- 172.21.15.116:0/3917375507 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5584241c9320 0x5584241d7390 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f2de001b000 tx=0x7f2de00056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee12c4a66c11d1f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:00.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.713+0000 7f2dd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:19:00.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.713+0000 7f2dd1ffb640 1 -- 172.21.15.116:0/3917375507 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de0020040 con 0x5584241c9320 2024-09-12T03:19:00.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.713+0000 7f2dd1ffb640 1 -- 172.21.15.116:0/3917375507 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f2de00043a0 con 0x5584241c9320 2024-09-12T03:19:00.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.713+0000 7f2df2ebc740 1 -- 172.21.15.116:0/3917375507 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2de00041f0 con 0x5584241c9320 2024-09-12T03:19:00.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.713+0000 7f2dd1ffb640 1 -- 172.21.15.116:0/3917375507 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2de00035f0 con 0x5584241c9320 2024-09-12T03:19:00.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.713+0000 7f2df2ebc740 10 client.19632 fetch_fsmap learned FSMap version 25 2024-09-12T03:19:00.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.713+0000 7f2df2ebc740 1 -- 172.21.15.116:0/3917375507 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55842424e1f0 con 0x5584241c9320 2024-09-12T03:19:00.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.713+0000 7f2dd1ffb640 1 -- 172.21.15.116:0/3917375507 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de00041f0 con 0x5584241c9320 2024-09-12T03:19:00.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.714+0000 7f2dd1ffb640 1 -- 172.21.15.116:0/3917375507 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2de0023070 con 0x5584241c9320 2024-09-12T03:19:00.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.714+0000 7f2df2ebc740 10 client.19632 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:19:00.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.714+0000 7f2df2ebc740 10 client.19632 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-12T03:19:00.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.714+0000 7f2df2ebc740 20 client.19632 populate_metadata read hostname 'smithi116' 2024-09-12T03:19:00.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.714+0000 7f2df2ebc740 1 --2- 172.21.15.116:0/3917375507 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55842424e930 0x558424250d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:00.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.714+0000 7f2df2ebc740 4 client.19632 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-12T03:19:00.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.714+0000 7f2df2ebc740 1 -- 172.21.15.116:0/3917375507 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x558424134c80 con 0x55842424e930 2024-09-12T03:19:00.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.714+0000 7f2dea59c640 1 --2- 172.21.15.116:0/3917375507 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55842424e930 0x558424250d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:00.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.714+0000 7f2dea59c640 1 --2- 172.21.15.116:0/3917375507 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55842424e930 0x558424250d50 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2faf9838a3d1ee1 server_cookie=623d9aa46bf24c6b in_seq=0 out_seq=0 2024-09-12T03:19:00.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.714+0000 7f2dd1ffb640 10 client.19632 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:19:00.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.715+0000 7f2dd1ffb640 1 -- 172.21.15.116:0/3917375507 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x558424134c80 con 0x55842424e930 2024-09-12T03:19:00.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.715+0000 7f2dd1ffb640 10 client.19632 handle_command_reply: tid=0 2024-09-12T03:19:00.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.715+0000 7f2df2ebc740 1 client.19632 shutdown 2024-09-12T03:19:00.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.715+0000 7f2df2ebc740 1 -- 172.21.15.116:0/3917375507 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5584241c9320 msgr2=0x5584241d7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:00.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.715+0000 7f2df2ebc740 1 --2- 172.21.15.116:0/3917375507 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5584241c9320 0x5584241d7390 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f2de001b000 tx=0x7f2de00056f0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.715+0000 7f2df2ebc740 1 -- 172.21.15.116:0/3917375507 shutdown_connections 2024-09-12T03:19:00.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.715+0000 7f2df2ebc740 1 --2- 172.21.15.116:0/3917375507 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55842424e930 0x558424250d50 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.715+0000 7f2df2ebc740 1 --2- 172.21.15.116:0/3917375507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584241d3be0 0x5584241d78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.715+0000 7f2df2ebc740 1 --2- 172.21.15.116:0/3917375507 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5584241c9320 0x5584241d7390 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.715+0000 7f2dd1ffb640 0 client.19632 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:19:00.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.715+0000 7f2df2ebc740 1 --2- 172.21.15.116:0/3917375507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584241c8a00 0x5584241dc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.715+0000 7f2df2ebc740 1 -- 172.21.15.116:0/3917375507 >> 172.21.15.116:0/3917375507 conn(0x5584241c4f90 msgr2=0x5584241c41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:00.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.715+0000 7f2df2ebc740 1 -- 172.21.15.116:0/3917375507 shutdown_connections 2024-09-12T03:19:00.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.715+0000 7f2df2ebc740 1 -- 172.21.15.116:0/3917375507 wait complete. 2024-09-12T03:19:00.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.715+0000 7f2df2ebc740 20 client.19632 trim_cache size 0 max 16384 2024-09-12T03:19:00.728 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:19:00.729 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 25, 2024-09-12T03:19:00.729 INFO:teuthology.orchestra.run.smithi116.stdout: "leader": 24494, 2024-09-12T03:19:00.729 INFO:teuthology.orchestra.run.smithi116.stdout: "set_version": 56, 2024-09-12T03:19:00.729 INFO:teuthology.orchestra.run.smithi116.stdout: "sets": {} 2024-09-12T03:19:00.729 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:19:00.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.715+0000 7f2df0ae3640 1 -- 172.21.15.116:0/538299115 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2dd40a2b80 msgr2=0x7f2dd40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:00.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.715+0000 7f2df0ae3640 1 --2- 172.21.15.116:0/538299115 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2dd40a2b80 0x7f2dd40a5040 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2de40314e0 tx=0x7f2de40058b0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.715+0000 7f2df0ae3640 1 -- 172.21.15.116:0/538299115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2dec100e20 msgr2=0x7f2dec198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:00.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.715+0000 7f2df0ae3640 1 --2- 172.21.15.116:0/538299115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2dec100e20 0x7f2dec198440 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc00ba50 tx=0x7f2ddc00bf10 comp rx=0 tx=0).stop 2024-09-12T03:19:00.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.715+0000 7f2df0ae3640 1 -- 172.21.15.116:0/538299115 shutdown_connections 2024-09-12T03:19:00.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.715+0000 7f2df0ae3640 1 --2- 172.21.15.116:0/538299115 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f2dd40a2b80 0x7f2dd40a5040 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.715+0000 7f2df0ae3640 1 --2- 172.21.15.116:0/538299115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2dec102ef0 0x7f2dec19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.715+0000 7f2df0ae3640 1 --2- 172.21.15.116:0/538299115 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dec102570 0x7f2dec198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.715+0000 7f2df0ae3640 1 --2- 172.21.15.116:0/538299115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2dec100e20 0x7f2dec198440 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.715+0000 7f2df0ae3640 1 -- 172.21.15.116:0/538299115 >> 172.21.15.116:0/538299115 conn(0x7f2dec0fc7d0 msgr2=0x7f2dec106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:00.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.716+0000 7f2df0ae3640 1 -- 172.21.15.116:0/538299115 shutdown_connections 2024-09-12T03:19:00.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:00.716+0000 7f2df0ae3640 1 -- 172.21.15.116:0/538299115 wait complete. 2024-09-12T03:19:00.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:00 smithi183 ceph-mon[35535]: pgmap v818: 97 pgs: 97 active+clean; 7.8 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-12T03:19:00.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:00 smithi177 ceph-mon[35305]: pgmap v818: 97 pgs: 97 active+clean; 7.8 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-12T03:19:01.533 INFO:tasks.quiescer:done joining Quiescers 2024-09-12T03:19:01.533 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-12T03:19:01.543 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-12T03:19:02.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:02 smithi116 ceph-mon[31410]: pgmap v819: 97 pgs: 97 active+clean; 7.8 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T03:19:02.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:02 smithi183 ceph-mon[35535]: pgmap v819: 97 pgs: 97 active+clean; 7.8 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T03:19:02.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:02 smithi177 ceph-mon[35305]: pgmap v819: 97 pgs: 97 active+clean; 7.8 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T03:19:03.413 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:19:03.675 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:04.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.174+0000 7f405b625640 1 -- 172.21.15.116:0/2275648160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4054100a60 msgr2=0x7f4054100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:04.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.174+0000 7f405b625640 1 --2- 172.21.15.116:0/2275648160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4054100a60 0x7f4054100ec0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f4044009930 tx=0x7f404402a420 comp rx=0 tx=0).stop 2024-09-12T03:19:04.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.175+0000 7f405b625640 1 -- 172.21.15.116:0/2275648160 shutdown_connections 2024-09-12T03:19:04.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.175+0000 7f405b625640 1 --2- 172.21.15.116:0/2275648160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4054101400 0x7f405410d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.175+0000 7f405b625640 1 --2- 172.21.15.116:0/2275648160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4054100a60 0x7f4054100ec0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.175+0000 7f405b625640 1 --2- 172.21.15.116:0/2275648160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4054106a60 0x7f4054106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.175+0000 7f405b625640 1 -- 172.21.15.116:0/2275648160 >> 172.21.15.116:0/2275648160 conn(0x7f40540fc780 msgr2=0x7f40540feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:04.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.175+0000 7f405b625640 1 -- 172.21.15.116:0/2275648160 shutdown_connections 2024-09-12T03:19:04.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.175+0000 7f405b625640 1 -- 172.21.15.116:0/2275648160 wait complete. 2024-09-12T03:19:04.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.175+0000 7f405b625640 1 Processor -- start 2024-09-12T03:19:04.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.175+0000 7f405b625640 1 -- start start 2024-09-12T03:19:04.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.175+0000 7f405b625640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4054100a60 0x7f4054102630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:04.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.175+0000 7f405b625640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4054101400 0x7f4054102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:04.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.176+0000 7f405b625640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4054106a60 0x7f40541030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:04.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.176+0000 7f405b625640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4054110e60 con 0x7f4054106a60 2024-09-12T03:19:04.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.176+0000 7f405b625640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4054110ce0 con 0x7f4054101400 2024-09-12T03:19:04.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.176+0000 7f405b625640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4054110fe0 con 0x7f4054100a60 2024-09-12T03:19:04.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.176+0000 7f4059bc2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4054106a60 0x7f40541030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:04.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.176+0000 7f4059bc2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4054106a60 0x7f40541030b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32858/0 (socket says 172.21.15.116:32858) 2024-09-12T03:19:04.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.176+0000 7f4059bc2640 1 -- 172.21.15.116:0/1825569990 learned_addr learned my addr 172.21.15.116:0/1825569990 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:04.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.176+0000 7f40593c1640 1 --2- 172.21.15.116:0/1825569990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4054100a60 0x7f4054102630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:04.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.176+0000 7f4058bc0640 1 --2- 172.21.15.116:0/1825569990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4054101400 0x7f4054102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:04.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.176+0000 7f4059bc2640 1 -- 172.21.15.116:0/1825569990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4054100a60 msgr2=0x7f4054102630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:04.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.176+0000 7f4059bc2640 1 --2- 172.21.15.116:0/1825569990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4054100a60 0x7f4054102630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.176+0000 7f4059bc2640 1 -- 172.21.15.116:0/1825569990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4054101400 msgr2=0x7f4054102b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:04.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.176+0000 7f4059bc2640 1 --2- 172.21.15.116:0/1825569990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4054101400 0x7f4054102b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.176+0000 7f4059bc2640 1 -- 172.21.15.116:0/1825569990 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40541a6940 con 0x7f4054106a60 2024-09-12T03:19:04.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.176+0000 7f4059bc2640 1 --2- 172.21.15.116:0/1825569990 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4054106a60 0x7f40541030b0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f404c00eaa0 tx=0x7f404c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:04.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.176+0000 7f404a7fc640 1 -- 172.21.15.116:0/1825569990 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c00cdb0 con 0x7f4054106a60 2024-09-12T03:19:04.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.176+0000 7f404a7fc640 1 -- 172.21.15.116:0/1825569990 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f404c005020 con 0x7f4054106a60 2024-09-12T03:19:04.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.176+0000 7f405b625640 1 -- 172.21.15.116:0/1825569990 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40541a6c30 con 0x7f4054106a60 2024-09-12T03:19:04.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.176+0000 7f404a7fc640 1 -- 172.21.15.116:0/1825569990 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c0116b0 con 0x7f4054106a60 2024-09-12T03:19:04.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.176+0000 7f405b625640 1 -- 172.21.15.116:0/1825569990 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40541a7140 con 0x7f4054106a60 2024-09-12T03:19:04.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.179+0000 7f404a7fc640 1 -- 172.21.15.116:0/1825569990 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f404c005560 con 0x7f4054106a60 2024-09-12T03:19:04.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.179+0000 7f405d9fe740 1 Processor -- start 2024-09-12T03:19:04.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.179+0000 7f405d9fe740 1 -- start start 2024-09-12T03:19:04.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.179+0000 7f405d9fe740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55702cd144a0 0x55702cd14880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:04.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.179+0000 7f405d9fe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55702cd14dc0 0x55702cd1f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:04.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.179+0000 7f405d9fe740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55702cd1f680 0x55702cd21a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:04.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.179+0000 7f405d9fe740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55702cc7a2e0 con 0x55702cd1f680 2024-09-12T03:19:04.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.179+0000 7f405d9fe740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55702cc21ce0 con 0x55702cd144a0 2024-09-12T03:19:04.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.179+0000 7f405d9fe740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55702cc762a0 con 0x55702cd14dc0 2024-09-12T03:19:04.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.179+0000 7f4058bc0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55702cd1f680 0x55702cd21a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:04.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.179+0000 7f4058bc0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55702cd1f680 0x55702cd21a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32864/0 (socket says 172.21.15.116:32864) 2024-09-12T03:19:04.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.179+0000 7f4058bc0640 1 -- 172.21.15.116:0/81390518 learned_addr learned my addr 172.21.15.116:0/81390518 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:04.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.179+0000 7f40593c1640 1 --2- 172.21.15.116:0/81390518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55702cd14dc0 0x55702cd1f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:04.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.179+0000 7f404a7fc640 1 --2- 172.21.15.116:0/1825569990 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f40400a2c00 0x7f40400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:04.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.179+0000 7f4058bc0640 1 --2- 172.21.15.116:0/81390518 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55702cd144a0 0x55702cd14880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:04.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.180+0000 7f4059bc2640 1 --2- 172.21.15.116:0/1825569990 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f40400a2c00 0x7f40400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:04.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.180+0000 7f40593c1640 1 -- 172.21.15.116:0/81390518 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55702cd144a0 msgr2=0x55702cd14880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:04.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.180+0000 7f40593c1640 1 --2- 172.21.15.116:0/81390518 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55702cd144a0 0x55702cd14880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.180+0000 7f404a7fc640 1 -- 172.21.15.116:0/1825569990 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f404c0cf150 con 0x7f4054106a60 2024-09-12T03:19:04.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.180+0000 7f40593c1640 1 -- 172.21.15.116:0/81390518 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55702cd1f680 msgr2=0x55702cd21a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:04.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.180+0000 7f40593c1640 1 --2- 172.21.15.116:0/81390518 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55702cd1f680 0x55702cd21a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.180+0000 7f40593c1640 1 -- 172.21.15.116:0/81390518 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55702cc06440 con 0x55702cd14dc0 2024-09-12T03:19:04.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.180+0000 7f4058bc0640 1 --2- 172.21.15.116:0/81390518 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55702cd1f680 0x55702cd21a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:19:04.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.180+0000 7f40593c1640 1 --2- 172.21.15.116:0/81390518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55702cd14dc0 0x55702cd1f140 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f405000bb70 tx=0x7f405000bf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c281fa16bca90c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:04.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.180+0000 7f4059bc2640 1 --2- 172.21.15.116:0/1825569990 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f40400a2c00 0x7f40400a50c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f404c09ba20 tx=0x7f404c090f70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:04.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.180+0000 7f403f7fe640 1 -- 172.21.15.116:0/81390518 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4050010e50 con 0x55702cd14dc0 2024-09-12T03:19:04.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.180+0000 7f403f7fe640 1 -- 172.21.15.116:0/81390518 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40500035f0 con 0x55702cd14dc0 2024-09-12T03:19:04.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.180+0000 7f403f7fe640 1 -- 172.21.15.116:0/81390518 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40500116a0 con 0x55702cd14dc0 2024-09-12T03:19:04.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.181+0000 7f405d9fe740 1 -- 172.21.15.116:0/81390518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55702cd14dc0 msgr2=0x55702cd1f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:04.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.181+0000 7f405d9fe740 1 --2- 172.21.15.116:0/81390518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55702cd14dc0 0x55702cd1f140 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f405000bb70 tx=0x7f405000bf70 comp rx=0 tx=0).stop 2024-09-12T03:19:04.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.181+0000 7f405d9fe740 1 -- 172.21.15.116:0/81390518 shutdown_connections 2024-09-12T03:19:04.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.181+0000 7f405d9fe740 1 --2- 172.21.15.116:0/81390518 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55702cd1f680 0x55702cd21a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.181+0000 7f405d9fe740 1 --2- 172.21.15.116:0/81390518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55702cd14dc0 0x55702cd1f140 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.181+0000 7f405d9fe740 1 --2- 172.21.15.116:0/81390518 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55702cd144a0 0x55702cd14880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.181+0000 7f405d9fe740 1 -- 172.21.15.116:0/81390518 >> 172.21.15.116:0/81390518 conn(0x55702cd10a30 msgr2=0x55702cd10e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:04.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.181+0000 7f405d9fe740 1 -- 172.21.15.116:0/81390518 shutdown_connections 2024-09-12T03:19:04.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.181+0000 7f405d9fe740 1 -- 172.21.15.116:0/81390518 wait complete. 2024-09-12T03:19:04.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.181+0000 7f405d9fe740 1 -- start start 2024-09-12T03:19:04.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.181+0000 7f405d9fe740 1 Processor -- start 2024-09-12T03:19:04.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.182+0000 7f405d9fe740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55702cd144a0 0x55702cd1ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:04.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.182+0000 7f405d9fe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55702cd14dc0 0x55702cd1c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:04.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.182+0000 7f405d9fe740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55702cd1f680 0x55702cd1c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:04.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.182+0000 7f405d9fe740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55702cd22250 con 0x55702cd1f680 2024-09-12T03:19:04.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.182+0000 7f405d9fe740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55702cc35050 con 0x55702cd144a0 2024-09-12T03:19:04.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.182+0000 7f405d9fe740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55702cd223d0 con 0x55702cd14dc0 2024-09-12T03:19:04.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.182+0000 7f405d9fe740 10 client.0 _ll_register_callbacks cb 0x55702cc75160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:04.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.182+0000 7f4058bc0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55702cd1f680 0x55702cd1c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:04.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.182+0000 7f4058bc0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55702cd1f680 0x55702cd1c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32868/0 (socket says 172.21.15.116:32868) 2024-09-12T03:19:04.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.182+0000 7f4058bc0640 1 -- 172.21.15.116:0/2192489390 learned_addr learned my addr 172.21.15.116:0/2192489390 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:04.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.182+0000 7f40593c1640 1 --2- 172.21.15.116:0/2192489390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55702cd14dc0 0x55702cd1c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:04.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.182+0000 7f4058bc0640 1 --2- 172.21.15.116:0/2192489390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55702cd144a0 0x55702cd1ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:04.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.182+0000 7f40593c1640 1 -- 172.21.15.116:0/2192489390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55702cd144a0 msgr2=0x55702cd1ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:04.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.182+0000 7f40593c1640 1 --2- 172.21.15.116:0/2192489390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55702cd144a0 0x55702cd1ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.182+0000 7f40593c1640 1 -- 172.21.15.116:0/2192489390 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55702cd1f680 msgr2=0x55702cd1c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:04.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.182+0000 7f40593c1640 1 --2- 172.21.15.116:0/2192489390 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55702cd1f680 0x55702cd1c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.182+0000 7f40593c1640 1 -- 172.21.15.116:0/2192489390 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55702c9eb3c0 con 0x55702cd14dc0 2024-09-12T03:19:04.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.182+0000 7f4058bc0640 1 --2- 172.21.15.116:0/2192489390 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55702cd1f680 0x55702cd1c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:19:04.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.183+0000 7f40593c1640 1 --2- 172.21.15.116:0/2192489390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55702cd14dc0 0x55702cd1c210 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4050010860 tx=0x7f4050010e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7caa2325192c2176 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:04.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.183+0000 7f403cff9640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-12T03:19:04.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.183+0000 7f403cff9640 1 -- 172.21.15.116:0/2192489390 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405002a060 con 0x55702cd14dc0 2024-09-12T03:19:04.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.183+0000 7f405d9fe740 1 -- 172.21.15.116:0/2192489390 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40500035f0 con 0x55702cd14dc0 2024-09-12T03:19:04.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.183+0000 7f403cff9640 1 -- 172.21.15.116:0/2192489390 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f40500039b0 con 0x55702cd14dc0 2024-09-12T03:19:04.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.183+0000 7f403cff9640 1 -- 172.21.15.116:0/2192489390 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f405000b740 con 0x55702cd14dc0 2024-09-12T03:19:04.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.183+0000 7f405d9fe740 10 client.27709 fetch_fsmap learned FSMap version 25 2024-09-12T03:19:04.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.183+0000 7f405d9fe740 1 -- 172.21.15.116:0/2192489390 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55702cd2dbf0 con 0x55702cd14dc0 2024-09-12T03:19:04.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.183+0000 7f403cff9640 1 -- 172.21.15.116:0/2192489390 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40500035f0 con 0x55702cd14dc0 2024-09-12T03:19:04.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.183+0000 7f403cff9640 1 -- 172.21.15.116:0/2192489390 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4050004750 con 0x55702cd14dc0 2024-09-12T03:19:04.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.183+0000 7f405d9fe740 10 client.27709 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:19:04.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.183+0000 7f405d9fe740 10 client.27709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:19:04.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.183+0000 7f405d9fe740 20 client.27709 populate_metadata read hostname 'smithi116' 2024-09-12T03:19:04.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.183+0000 7f405d9fe740 1 --2- 172.21.15.116:0/2192489390 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55702cd2f0b0 0x55702cd314a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:04.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.183+0000 7f405d9fe740 4 client.27709 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:19:04.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.183+0000 7f405d9fe740 1 -- 172.21.15.116:0/2192489390 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55702cc87850 con 0x55702cd2f0b0 2024-09-12T03:19:04.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.184+0000 7f4058bc0640 1 --2- 172.21.15.116:0/2192489390 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55702cd2f0b0 0x55702cd314a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:04.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.184+0000 7f403cff9640 10 client.27709 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:19:04.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.184+0000 7f4058bc0640 1 --2- 172.21.15.116:0/2192489390 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55702cd2f0b0 0x55702cd314a0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a94c26f4e81acd6 server_cookie=f160a1d538c2904c in_seq=0 out_seq=0 2024-09-12T03:19:04.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.185+0000 7f403cff9640 1 -- 172.21.15.116:0/2192489390 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55702cc87850 con 0x55702cd2f0b0 2024-09-12T03:19:04.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.185+0000 7f403cff9640 10 client.27709 handle_command_reply: tid=0 2024-09-12T03:19:04.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.185+0000 7f405d9fe740 1 client.27709 shutdown 2024-09-12T03:19:04.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.185+0000 7f405d9fe740 1 -- 172.21.15.116:0/2192489390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55702cd14dc0 msgr2=0x55702cd1c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:04.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.185+0000 7f405d9fe740 1 --2- 172.21.15.116:0/2192489390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55702cd14dc0 0x55702cd1c210 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4050010860 tx=0x7f4050010e80 comp rx=0 tx=0).stop 2024-09-12T03:19:04.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.185+0000 7f405d9fe740 1 -- 172.21.15.116:0/2192489390 shutdown_connections 2024-09-12T03:19:04.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.185+0000 7f405d9fe740 1 --2- 172.21.15.116:0/2192489390 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55702cd2f0b0 0x55702cd314a0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.185+0000 7f405d9fe740 1 --2- 172.21.15.116:0/2192489390 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55702cd1f680 0x55702cd1c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.185+0000 7f405d9fe740 1 --2- 172.21.15.116:0/2192489390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55702cd14dc0 0x55702cd1c210 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.185+0000 7f405d9fe740 1 --2- 172.21.15.116:0/2192489390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55702cd144a0 0x55702cd1ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.185+0000 7f405d9fe740 1 -- 172.21.15.116:0/2192489390 >> 172.21.15.116:0/2192489390 conn(0x55702cd10a30 msgr2=0x55702cd16710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:04.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.185+0000 7f403cff9640 0 client.27709 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:19:04.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.185+0000 7f405d9fe740 1 -- 172.21.15.116:0/2192489390 shutdown_connections 2024-09-12T03:19:04.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.185+0000 7f405d9fe740 1 -- 172.21.15.116:0/2192489390 wait complete. 2024-09-12T03:19:04.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.185+0000 7f405d9fe740 20 client.27709 trim_cache size 0 max 16384 2024-09-12T03:19:04.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.202+0000 7f405d9fe740 1 Processor -- start 2024-09-12T03:19:04.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.202+0000 7f405d9fe740 1 -- start start 2024-09-12T03:19:04.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.202+0000 7f405d9fe740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55702cd144a0 0x55702cd2b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:04.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.202+0000 7f405d9fe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55702cd14dc0 0x55702cd1aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:04.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.202+0000 7f405d9fe740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55702cd1f680 0x55702cd1e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:04.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.202+0000 7f405d9fe740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f40541b1fb0 con 0x55702cd1f680 2024-09-12T03:19:04.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.202+0000 7f405d9fe740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f40541af880 con 0x55702cd144a0 2024-09-12T03:19:04.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.202+0000 7f405d9fe740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f40541ad6d0 con 0x55702cd14dc0 2024-09-12T03:19:04.213 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.202+0000 7f4058bc0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55702cd1f680 0x55702cd1e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:04.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.202+0000 7f4058bc0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55702cd1f680 0x55702cd1e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32870/0 (socket says 172.21.15.116:32870) 2024-09-12T03:19:04.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.203+0000 7f4058bc0640 1 -- 172.21.15.116:0/3020457072 learned_addr learned my addr 172.21.15.116:0/3020457072 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:04.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.203+0000 7f4058bc0640 1 --2- 172.21.15.116:0/3020457072 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55702cd144a0 0x55702cd2b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:04.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.203+0000 7f40593c1640 1 --2- 172.21.15.116:0/3020457072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55702cd14dc0 0x55702cd1aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:04.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.203+0000 7f4058bc0640 1 -- 172.21.15.116:0/3020457072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55702cd14dc0 msgr2=0x55702cd1aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:04.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.203+0000 7f4058bc0640 1 --2- 172.21.15.116:0/3020457072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55702cd14dc0 0x55702cd1aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.203+0000 7f4058bc0640 1 -- 172.21.15.116:0/3020457072 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55702cd144a0 msgr2=0x55702cd2b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:04.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.203+0000 7f4058bc0640 1 --2- 172.21.15.116:0/3020457072 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55702cd144a0 0x55702cd2b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.203+0000 7f4058bc0640 1 -- 172.21.15.116:0/3020457072 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55702cd1b1c0 con 0x55702cd1f680 2024-09-12T03:19:04.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.203+0000 7f4058bc0640 1 --2- 172.21.15.116:0/3020457072 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55702cd1f680 0x55702cd1e180 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f4044009230 tx=0x7f40440058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f658a98a14666eb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:04.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.203+0000 7f403effd640 1 -- 172.21.15.116:0/3020457072 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044047040 con 0x55702cd1f680 2024-09-12T03:19:04.215 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.203+0000 7f403effd640 1 -- 172.21.15.116:0/3020457072 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40440044a0 con 0x55702cd1f680 2024-09-12T03:19:04.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.203+0000 7f403effd640 1 -- 172.21.15.116:0/3020457072 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044035040 con 0x55702cd1f680 2024-09-12T03:19:04.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.204+0000 7f405d9fe740 1 -- 172.21.15.116:0/3020457072 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55702cd1f680 msgr2=0x55702cd1e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:04.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.204+0000 7f405d9fe740 1 --2- 172.21.15.116:0/3020457072 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55702cd1f680 0x55702cd1e180 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f4044009230 tx=0x7f40440058d0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.204+0000 7f405d9fe740 1 -- 172.21.15.116:0/3020457072 shutdown_connections 2024-09-12T03:19:04.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.204+0000 7f405d9fe740 1 --2- 172.21.15.116:0/3020457072 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55702cd1f680 0x55702cd1e180 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.216 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.204+0000 7f405d9fe740 1 --2- 172.21.15.116:0/3020457072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55702cd14dc0 0x55702cd1aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.204+0000 7f405d9fe740 1 --2- 172.21.15.116:0/3020457072 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55702cd144a0 0x55702cd2b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.204+0000 7f405d9fe740 1 -- 172.21.15.116:0/3020457072 >> 172.21.15.116:0/3020457072 conn(0x55702cd10a30 msgr2=0x55702cd1da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:04.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.204+0000 7f405d9fe740 1 -- 172.21.15.116:0/3020457072 shutdown_connections 2024-09-12T03:19:04.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.204+0000 7f405d9fe740 1 -- 172.21.15.116:0/3020457072 wait complete. 2024-09-12T03:19:04.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.204+0000 7f405d9fe740 1 -- start start 2024-09-12T03:19:04.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.204+0000 7f405d9fe740 1 Processor -- start 2024-09-12T03:19:04.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.205+0000 7f405d9fe740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55702cd144a0 0x55702cd23740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:04.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.205+0000 7f405d9fe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55702cd14dc0 0x55702cd23c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:04.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.205+0000 7f405d9fe740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55702cd1f680 0x55702cd241e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:04.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.205+0000 7f405d9fe740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f40541add40 con 0x55702cd144a0 2024-09-12T03:19:04.218 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.205+0000 7f405d9fe740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f40541af1b0 con 0x55702cd1f680 2024-09-12T03:19:04.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.205+0000 7f405d9fe740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f40541ae410 con 0x55702cd14dc0 2024-09-12T03:19:04.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.205+0000 7f405d9fe740 10 client.0 _ll_register_callbacks cb 0x7f405000b9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:04.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.205+0000 7f4058bc0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55702cd144a0 0x55702cd23740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:04.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.205+0000 7f4058bc0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55702cd144a0 0x55702cd23740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32880/0 (socket says 172.21.15.116:32880) 2024-09-12T03:19:04.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.205+0000 7f4058bc0640 1 -- 172.21.15.116:0/4093007263 learned_addr learned my addr 172.21.15.116:0/4093007263 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:04.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.205+0000 7f40593c1640 1 --2- 172.21.15.116:0/4093007263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55702cd14dc0 0x55702cd23c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:04.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.205+0000 7f4058bc0640 1 --2- 172.21.15.116:0/4093007263 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55702cd1f680 0x55702cd241e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:04.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.205+0000 7f4058bc0640 1 -- 172.21.15.116:0/4093007263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55702cd14dc0 msgr2=0x55702cd23c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:04.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.205+0000 7f4058bc0640 1 --2- 172.21.15.116:0/4093007263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55702cd14dc0 0x55702cd23c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.205+0000 7f4058bc0640 1 -- 172.21.15.116:0/4093007263 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55702cd1f680 msgr2=0x55702cd241e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:04.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.205+0000 7f4058bc0640 1 --2- 172.21.15.116:0/4093007263 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55702cd1f680 0x55702cd241e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.205+0000 7f4058bc0640 1 -- 172.21.15.116:0/4093007263 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55702cd14bf0 con 0x55702cd144a0 2024-09-12T03:19:04.220 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.205+0000 7f40593c1640 1 --2- 172.21.15.116:0/4093007263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55702cd14dc0 0x55702cd23c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:04.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.205+0000 7f4058bc0640 1 --2- 172.21.15.116:0/4093007263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55702cd144a0 0x55702cd23740 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f4044004c10 tx=0x7f404402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f64168a6a90327 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:04.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.205+0000 7f403cff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:19:04.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.205+0000 7f403cff9640 1 -- 172.21.15.116:0/4093007263 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044035070 con 0x55702cd144a0 2024-09-12T03:19:04.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.205+0000 7f403cff9640 1 -- 172.21.15.116:0/4093007263 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f4044004970 con 0x55702cd144a0 2024-09-12T03:19:04.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.205+0000 7f405d9fe740 1 -- 172.21.15.116:0/4093007263 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40440044a0 con 0x55702cd144a0 2024-09-12T03:19:04.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.206+0000 7f403cff9640 1 -- 172.21.15.116:0/4093007263 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4044004f90 con 0x55702cd144a0 2024-09-12T03:19:04.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.206+0000 7f405d9fe740 10 client.19650 fetch_fsmap learned FSMap version 25 2024-09-12T03:19:04.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.206+0000 7f405d9fe740 1 -- 172.21.15.116:0/4093007263 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55702cd99e50 con 0x55702cd144a0 2024-09-12T03:19:04.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.206+0000 7f403cff9640 1 -- 172.21.15.116:0/4093007263 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40440044a0 con 0x55702cd144a0 2024-09-12T03:19:04.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.206+0000 7f403cff9640 1 -- 172.21.15.116:0/4093007263 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f404404fa40 con 0x55702cd144a0 2024-09-12T03:19:04.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.206+0000 7f405d9fe740 10 client.19650 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:19:04.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.206+0000 7f405d9fe740 10 client.19650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:19:04.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.206+0000 7f405d9fe740 20 client.19650 populate_metadata read hostname 'smithi116' 2024-09-12T03:19:04.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.206+0000 7f405d9fe740 1 --2- 172.21.15.116:0/4093007263 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55702cd9a4c0 0x55702cd9c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:04.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.206+0000 7f405d9fe740 4 client.19650 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:19:04.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.206+0000 7f405d9fe740 1 -- 172.21.15.116:0/4093007263 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55702cc81d60 con 0x55702cd9a4c0 2024-09-12T03:19:04.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.206+0000 7f40593c1640 1 --2- 172.21.15.116:0/4093007263 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55702cd9a4c0 0x55702cd9c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:04.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.207+0000 7f40593c1640 1 --2- 172.21.15.116:0/4093007263 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55702cd9a4c0 0x55702cd9c8e0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bba12ad735857672 server_cookie=2c30f97bd5d34279 in_seq=0 out_seq=0 2024-09-12T03:19:04.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.207+0000 7f403cff9640 10 client.19650 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:19:04.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.207+0000 7f403cff9640 1 -- 172.21.15.116:0/4093007263 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55702cc81d60 con 0x55702cd9a4c0 2024-09-12T03:19:04.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.207+0000 7f403cff9640 10 client.19650 handle_command_reply: tid=0 2024-09-12T03:19:04.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.207+0000 7f405d9fe740 1 client.19650 shutdown 2024-09-12T03:19:04.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.207+0000 7f405d9fe740 1 -- 172.21.15.116:0/4093007263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55702cd144a0 msgr2=0x55702cd23740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:04.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.207+0000 7f405d9fe740 1 --2- 172.21.15.116:0/4093007263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55702cd144a0 0x55702cd23740 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f4044004c10 tx=0x7f404402a420 comp rx=0 tx=0).stop 2024-09-12T03:19:04.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.207+0000 7f405d9fe740 1 -- 172.21.15.116:0/4093007263 shutdown_connections 2024-09-12T03:19:04.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.207+0000 7f405d9fe740 1 --2- 172.21.15.116:0/4093007263 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55702cd9a4c0 0x55702cd9c8e0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.207+0000 7f405d9fe740 1 --2- 172.21.15.116:0/4093007263 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55702cd1f680 0x55702cd241e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.207+0000 7f405d9fe740 1 --2- 172.21.15.116:0/4093007263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55702cd14dc0 0x55702cd23c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.207+0000 7f403cff9640 0 client.19650 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:19:04.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.207+0000 7f405d9fe740 1 --2- 172.21.15.116:0/4093007263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55702cd144a0 0x55702cd23740 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.207+0000 7f405d9fe740 1 -- 172.21.15.116:0/4093007263 >> 172.21.15.116:0/4093007263 conn(0x55702cd10a30 msgr2=0x55702cd1e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:04.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.207+0000 7f405d9fe740 1 -- 172.21.15.116:0/4093007263 shutdown_connections 2024-09-12T03:19:04.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.207+0000 7f405d9fe740 1 -- 172.21.15.116:0/4093007263 wait complete. 2024-09-12T03:19:04.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.207+0000 7f405d9fe740 20 client.19650 trim_cache size 0 max 16384 2024-09-12T03:19:04.224 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:19:04.224 INFO:teuthology.orchestra.run.smithi116.stdout: "status": "no active scrubs running", 2024-09-12T03:19:04.224 INFO:teuthology.orchestra.run.smithi116.stdout: "scrubs": {} 2024-09-12T03:19:04.224 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:19:04.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.208+0000 7f405b625640 1 -- 172.21.15.116:0/1825569990 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f40400a2c00 msgr2=0x7f40400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:04.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.208+0000 7f405b625640 1 --2- 172.21.15.116:0/1825569990 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f40400a2c00 0x7f40400a50c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f404c09ba20 tx=0x7f404c090f70 comp rx=0 tx=0).stop 2024-09-12T03:19:04.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.208+0000 7f405b625640 1 -- 172.21.15.116:0/1825569990 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4054106a60 msgr2=0x7f40541030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:04.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.208+0000 7f405b625640 1 --2- 172.21.15.116:0/1825569990 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4054106a60 0x7f40541030b0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f404c00eaa0 tx=0x7f404c00ef60 comp rx=0 tx=0).stop 2024-09-12T03:19:04.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.208+0000 7f405b625640 1 -- 172.21.15.116:0/1825569990 shutdown_connections 2024-09-12T03:19:04.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.208+0000 7f405b625640 1 --2- 172.21.15.116:0/1825569990 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f40400a2c00 0x7f40400a50c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.208+0000 7f405b625640 1 --2- 172.21.15.116:0/1825569990 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4054106a60 0x7f40541030b0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.208+0000 7f405b625640 1 --2- 172.21.15.116:0/1825569990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4054101400 0x7f4054102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.208+0000 7f405b625640 1 --2- 172.21.15.116:0/1825569990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4054100a60 0x7f4054102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:04.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.208+0000 7f405b625640 1 -- 172.21.15.116:0/1825569990 >> 172.21.15.116:0/1825569990 conn(0x7f40540fc780 msgr2=0x7f4054104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:04.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.208+0000 7f405b625640 1 -- 172.21.15.116:0/1825569990 shutdown_connections 2024-09-12T03:19:04.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:04.208+0000 7f405b625640 1 -- 172.21.15.116:0/1825569990 wait complete. 2024-09-12T03:19:04.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:04 smithi116 ceph-mon[31410]: pgmap v820: 97 pgs: 97 active+clean; 7.8 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:19:04.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:04 smithi183 ceph-mon[35535]: pgmap v820: 97 pgs: 97 active+clean; 7.8 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:19:04.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:04 smithi177 ceph-mon[35305]: pgmap v820: 97 pgs: 97 active+clean; 7.8 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T03:19:05.039 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:19:05.040 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:19:05.284 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:05.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.780+0000 7f3641ef3640 1 -- 172.21.15.116:0/2285246441 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f363c102ef0 msgr2=0x7f363c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:05.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.780+0000 7f3641ef3640 1 --2- 172.21.15.116:0/2285246441 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f363c102ef0 0x7f363c10da40 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f3634009ca0 tx=0x7f363402a950 comp rx=0 tx=0).stop 2024-09-12T03:19:05.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.780+0000 7f3641ef3640 1 -- 172.21.15.116:0/2285246441 shutdown_connections 2024-09-12T03:19:05.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.780+0000 7f3641ef3640 1 --2- 172.21.15.116:0/2285246441 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f363c102ef0 0x7f363c10da40 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.780+0000 7f3641ef3640 1 --2- 172.21.15.116:0/2285246441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f363c102570 0x7f363c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.780+0000 7f3641ef3640 1 --2- 172.21.15.116:0/2285246441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f363c100e20 0x7f363c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.780+0000 7f3641ef3640 1 -- 172.21.15.116:0/2285246441 >> 172.21.15.116:0/2285246441 conn(0x7f363c0fc7d0 msgr2=0x7f363c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:05.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.780+0000 7f3641ef3640 1 -- 172.21.15.116:0/2285246441 shutdown_connections 2024-09-12T03:19:05.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.780+0000 7f3641ef3640 1 -- 172.21.15.116:0/2285246441 wait complete. 2024-09-12T03:19:05.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.781+0000 7f3641ef3640 1 Processor -- start 2024-09-12T03:19:05.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.781+0000 7f3641ef3640 1 -- start start 2024-09-12T03:19:05.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.781+0000 7f3641ef3640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f363c100e20 0x7f363c197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:05.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.781+0000 7f3641ef3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f363c102570 0x7f363c197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:05.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.781+0000 7f3641ef3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f363c102ef0 0x7f363c19bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:05.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.781+0000 7f3641ef3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f363c110f80 con 0x7f363c102570 2024-09-12T03:19:05.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.781+0000 7f3641ef3640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f363c110e00 con 0x7f363c100e20 2024-09-12T03:19:05.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.781+0000 7f3641ef3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f363c111100 con 0x7f363c102ef0 2024-09-12T03:19:05.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.781+0000 7f3633fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f363c102570 0x7f363c197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:05.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.781+0000 7f3633fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f363c102570 0x7f363c197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32900/0 (socket says 172.21.15.116:32900) 2024-09-12T03:19:05.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.781+0000 7f3633fff640 1 -- 172.21.15.116:0/1746389661 learned_addr learned my addr 172.21.15.116:0/1746389661 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:05.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.781+0000 7f363b7fe640 1 --2- 172.21.15.116:0/1746389661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f363c100e20 0x7f363c197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:05.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.781+0000 7f363bfff640 1 --2- 172.21.15.116:0/1746389661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f363c102ef0 0x7f363c19bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:05.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.782+0000 7f3633fff640 1 -- 172.21.15.116:0/1746389661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f363c102ef0 msgr2=0x7f363c19bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:05.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.782+0000 7f3633fff640 1 --2- 172.21.15.116:0/1746389661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f363c102ef0 0x7f363c19bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.801 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.782+0000 7f3633fff640 1 -- 172.21.15.116:0/1746389661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f363c100e20 msgr2=0x7f363c197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:05.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.782+0000 7f3633fff640 1 --2- 172.21.15.116:0/1746389661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f363c100e20 0x7f363c197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.782+0000 7f3633fff640 1 -- 172.21.15.116:0/1746389661 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f363c19c3b0 con 0x7f363c102570 2024-09-12T03:19:05.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.782+0000 7f3633fff640 1 --2- 172.21.15.116:0/1746389661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f363c102570 0x7f363c197940 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f362400d950 tx=0x7f362400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:05.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.782+0000 7f36397fa640 1 -- 172.21.15.116:0/1746389661 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3624017070 con 0x7f363c102570 2024-09-12T03:19:05.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.782+0000 7f36397fa640 1 -- 172.21.15.116:0/1746389661 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3624009c30 con 0x7f363c102570 2024-09-12T03:19:05.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.782+0000 7f3641ef3640 1 -- 172.21.15.116:0/1746389661 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f363c19c6a0 con 0x7f363c102570 2024-09-12T03:19:05.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.782+0000 7f36397fa640 1 -- 172.21.15.116:0/1746389661 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362400f6b0 con 0x7f363c102570 2024-09-12T03:19:05.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.782+0000 7f3641ef3640 1 -- 172.21.15.116:0/1746389661 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f363c1a3f80 con 0x7f363c102570 2024-09-12T03:19:05.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.783+0000 7f3641ef3640 1 -- 172.21.15.116:0/1746389661 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f363c101200 con 0x7f363c102570 2024-09-12T03:19:05.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.784+0000 7f36397fa640 1 -- 172.21.15.116:0/1746389661 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f36240109d0 con 0x7f363c102570 2024-09-12T03:19:05.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.785+0000 7f36397fa640 1 --2- 172.21.15.116:0/1746389661 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f36280a2c00 0x7f36280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:05.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.785+0000 7f363b7fe640 1 --2- 172.21.15.116:0/1746389661 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f36280a2c00 0x7f36280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:05.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.785+0000 7f36397fa640 1 -- 172.21.15.116:0/1746389661 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f36240ceb50 con 0x7f363c102570 2024-09-12T03:19:05.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.785+0000 7f363b7fe640 1 --2- 172.21.15.116:0/1746389661 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f36280a2c00 0x7f36280a50c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f362c005fd0 tx=0x7f362c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:05.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.789+0000 7f36397fa640 1 -- 172.21.15.116:0/1746389661 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3624009de0 con 0x7f363c102570 2024-09-12T03:19:05.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.987+0000 7f3641ef3640 1 -- 172.21.15.116:0/1746389661 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f363c1a4150 con 0x7f363c102570 2024-09-12T03:19:05.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.988+0000 7f36397fa640 1 -- 172.21.15.116:0/1746389661 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f362409b490 con 0x7f363c102570 2024-09-12T03:19:05.988 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:19:05.988 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:19:05.989 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:19:05.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.991+0000 7f3641ef3640 1 -- 172.21.15.116:0/1746389661 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f36280a2c00 msgr2=0x7f36280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:05.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.991+0000 7f3641ef3640 1 --2- 172.21.15.116:0/1746389661 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f36280a2c00 0x7f36280a50c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f362c005fd0 tx=0x7f362c00b390 comp rx=0 tx=0).stop 2024-09-12T03:19:05.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.991+0000 7f3641ef3640 1 -- 172.21.15.116:0/1746389661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f363c102570 msgr2=0x7f363c197940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:05.991 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.991+0000 7f3641ef3640 1 --2- 172.21.15.116:0/1746389661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f363c102570 0x7f363c197940 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f362400d950 tx=0x7f362400de10 comp rx=0 tx=0).stop 2024-09-12T03:19:05.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.991+0000 7f3641ef3640 1 -- 172.21.15.116:0/1746389661 shutdown_connections 2024-09-12T03:19:05.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.991+0000 7f3641ef3640 1 --2- 172.21.15.116:0/1746389661 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f36280a2c00 0x7f36280a50c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.991+0000 7f3641ef3640 1 --2- 172.21.15.116:0/1746389661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f363c102ef0 0x7f363c19bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.991+0000 7f3641ef3640 1 --2- 172.21.15.116:0/1746389661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f363c102570 0x7f363c197940 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.991+0000 7f3641ef3640 1 --2- 172.21.15.116:0/1746389661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f363c100e20 0x7f363c197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.991+0000 7f3641ef3640 1 -- 172.21.15.116:0/1746389661 >> 172.21.15.116:0/1746389661 conn(0x7f363c0fc7d0 msgr2=0x7f363c106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:05.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.992+0000 7f3641ef3640 1 -- 172.21.15.116:0/1746389661 shutdown_connections 2024-09-12T03:19:05.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:05.992+0000 7f3641ef3640 1 -- 172.21.15.116:0/1746389661 wait complete. 2024-09-12T03:19:06.796 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:06 smithi116 ceph-mon[31410]: pgmap v821: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:19:06.797 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:06 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1746389661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:06.826 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:19:06.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:06 smithi183 ceph-mon[35535]: pgmap v821: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:19:06.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:06 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1746389661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:06.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:06 smithi177 ceph-mon[35305]: pgmap v821: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-12T03:19:06.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:06 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1746389661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:07.072 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:07.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.564+0000 7f6524a50640 1 -- 172.21.15.116:0/2206612143 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6520106a80 msgr2=0x7f6520106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:07.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.564+0000 7f6524a50640 1 --2- 172.21.15.116:0/2206612143 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6520106a80 0x7f6520106e60 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f6510009930 tx=0x7f651002a420 comp rx=0 tx=0).stop 2024-09-12T03:19:07.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.564+0000 7f6524a50640 1 -- 172.21.15.116:0/2206612143 shutdown_connections 2024-09-12T03:19:07.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.564+0000 7f6524a50640 1 --2- 172.21.15.116:0/2206612143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6520101420 0x7f652010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.564+0000 7f6524a50640 1 --2- 172.21.15.116:0/2206612143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6520100a80 0x7f6520100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.564+0000 7f6524a50640 1 --2- 172.21.15.116:0/2206612143 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6520106a80 0x7f6520106e60 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.564+0000 7f6524a50640 1 -- 172.21.15.116:0/2206612143 >> 172.21.15.116:0/2206612143 conn(0x7f65200fc7a0 msgr2=0x7f65200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:07.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.565+0000 7f6524a50640 1 -- 172.21.15.116:0/2206612143 shutdown_connections 2024-09-12T03:19:07.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.565+0000 7f6524a50640 1 -- 172.21.15.116:0/2206612143 wait complete. 2024-09-12T03:19:07.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.565+0000 7f6524a50640 1 Processor -- start 2024-09-12T03:19:07.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.565+0000 7f6524a50640 1 -- start start 2024-09-12T03:19:07.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.566+0000 7f6524a50640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6520100a80 0x7f65201974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.566+0000 7f6524a50640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6520101420 0x7f6520197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.566+0000 7f6524a50640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6520106a80 0x7f652019bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.566+0000 7f6524a50640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6520110f20 con 0x7f6520100a80 2024-09-12T03:19:07.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.566+0000 7f6524a50640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6520110da0 con 0x7f6520106a80 2024-09-12T03:19:07.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.566+0000 7f6524a50640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f65201110a0 con 0x7f6520101420 2024-09-12T03:19:07.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.566+0000 7f651e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6520100a80 0x7f65201974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.566+0000 7f651e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6520100a80 0x7f65201974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32922/0 (socket says 172.21.15.116:32922) 2024-09-12T03:19:07.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.566+0000 7f651e59c640 1 -- 172.21.15.116:0/1631608884 learned_addr learned my addr 172.21.15.116:0/1631608884 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:07.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.566+0000 7f6517fff640 1 --2- 172.21.15.116:0/1631608884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6520101420 0x7f6520197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.566+0000 7f651ed9d640 1 --2- 172.21.15.116:0/1631608884 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6520106a80 0x7f652019bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.566+0000 7f651e59c640 1 -- 172.21.15.116:0/1631608884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6520101420 msgr2=0x7f6520197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:07.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.566+0000 7f651e59c640 1 --2- 172.21.15.116:0/1631608884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6520101420 0x7f6520197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.566+0000 7f651e59c640 1 -- 172.21.15.116:0/1631608884 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6520106a80 msgr2=0x7f652019bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:07.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.566+0000 7f651e59c640 1 --2- 172.21.15.116:0/1631608884 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6520106a80 0x7f652019bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.566+0000 7f651e59c640 1 -- 172.21.15.116:0/1631608884 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652019c490 con 0x7f6520100a80 2024-09-12T03:19:07.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.566+0000 7f6517fff640 1 --2- 172.21.15.116:0/1631608884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6520101420 0x7f6520197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:07.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.566+0000 7f651e59c640 1 --2- 172.21.15.116:0/1631608884 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6520100a80 0x7f65201974e0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f6510009a50 tx=0x7f6510004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:07.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.566+0000 7f65177fe640 1 -- 172.21.15.116:0/1631608884 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651003f030 con 0x7f6520100a80 2024-09-12T03:19:07.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.567+0000 7f6524a50640 1 -- 172.21.15.116:0/1631608884 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f652019c720 con 0x7f6520100a80 2024-09-12T03:19:07.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.567+0000 7f65177fe640 1 -- 172.21.15.116:0/1631608884 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6510003ce0 con 0x7f6520100a80 2024-09-12T03:19:07.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.567+0000 7f65177fe640 1 -- 172.21.15.116:0/1631608884 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6510035070 con 0x7f6520100a80 2024-09-12T03:19:07.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.567+0000 7f6524a50640 1 -- 172.21.15.116:0/1631608884 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65201a4060 con 0x7f6520100a80 2024-09-12T03:19:07.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.569+0000 7f65177fe640 1 -- 172.21.15.116:0/1631608884 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6510042030 con 0x7f6520100a80 2024-09-12T03:19:07.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.569+0000 7f6526e29740 1 Processor -- start 2024-09-12T03:19:07.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.569+0000 7f6526e29740 1 -- start start 2024-09-12T03:19:07.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.569+0000 7f6526e29740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d0e2f1d50 0x560d0e2f02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.569+0000 7f6526e29740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0e2f07e0 0x560d0e2f0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.569+0000 7f6526e29740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d0e2fb580 0x560d0e2fd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.569+0000 7f6526e29740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560d0e2542f0 con 0x560d0e2f1d50 2024-09-12T03:19:07.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.569+0000 7f6526e29740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560d0e1fbcf0 con 0x560d0e2fb580 2024-09-12T03:19:07.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.569+0000 7f6526e29740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560d0e2502b0 con 0x560d0e2f07e0 2024-09-12T03:19:07.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.570+0000 7f651ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d0e2f1d50 0x560d0e2f02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.570+0000 7f651ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d0e2f1d50 0x560d0e2f02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32924/0 (socket says 172.21.15.116:32924) 2024-09-12T03:19:07.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.570+0000 7f651ed9d640 1 -- 172.21.15.116:0/1151408741 learned_addr learned my addr 172.21.15.116:0/1151408741 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:07.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.570+0000 7f65177fe640 1 --2- 172.21.15.116:0/1631608884 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f650c0a2b80 0x7f650c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.570+0000 7f6517fff640 1 --2- 172.21.15.116:0/1151408741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d0e2fb580 0x560d0e2fd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.570+0000 7f651ed9d640 1 --2- 172.21.15.116:0/1631608884 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f650c0a2b80 0x7f650c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.570+0000 7f65177fe640 1 -- 172.21.15.116:0/1631608884 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f65100ef660 con 0x7f6520100a80 2024-09-12T03:19:07.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.570+0000 7f651e59c640 1 --2- 172.21.15.116:0/1151408741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0e2f07e0 0x560d0e2f0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.570+0000 7f651ed9d640 1 -- 172.21.15.116:0/1151408741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0e2f07e0 msgr2=0x560d0e2f0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:07.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.570+0000 7f651ed9d640 1 --2- 172.21.15.116:0/1151408741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0e2f07e0 0x560d0e2f0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.570+0000 7f651ed9d640 1 -- 172.21.15.116:0/1151408741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d0e2fb580 msgr2=0x560d0e2fd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:07.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.570+0000 7f651ed9d640 1 --2- 172.21.15.116:0/1151408741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d0e2fb580 0x560d0e2fd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.570+0000 7f651ed9d640 1 -- 172.21.15.116:0/1151408741 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d0e1e03a0 con 0x560d0e2f1d50 2024-09-12T03:19:07.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.570+0000 7f651e59c640 1 --2- 172.21.15.116:0/1151408741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0e2f07e0 0x560d0e2f0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:07.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.570+0000 7f651ed9d640 1 --2- 172.21.15.116:0/1151408741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d0e2f1d50 0x560d0e2f02a0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f651800cb20 tx=0x7f65180072b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9ac936e5b7b907 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:07.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.570+0000 7f6514ff9640 1 -- 172.21.15.116:0/1151408741 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6518015070 con 0x560d0e2f1d50 2024-09-12T03:19:07.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.570+0000 7f6514ff9640 1 -- 172.21.15.116:0/1151408741 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6518016040 con 0x560d0e2f1d50 2024-09-12T03:19:07.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.570+0000 7f6514ff9640 1 -- 172.21.15.116:0/1151408741 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651801e020 con 0x560d0e2f1d50 2024-09-12T03:19:07.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.571+0000 7f6526e29740 1 -- 172.21.15.116:0/1151408741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d0e2f1d50 msgr2=0x560d0e2f02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:07.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.571+0000 7f651ed9d640 1 --2- 172.21.15.116:0/1631608884 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f650c0a2b80 0x7f650c0a5040 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f651801f000 tx=0x7f651801b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:07.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.571+0000 7f6526e29740 1 --2- 172.21.15.116:0/1151408741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d0e2f1d50 0x560d0e2f02a0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f651800cb20 tx=0x7f65180072b0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.571+0000 7f6526e29740 1 -- 172.21.15.116:0/1151408741 shutdown_connections 2024-09-12T03:19:07.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.571+0000 7f6526e29740 1 --2- 172.21.15.116:0/1151408741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d0e2fb580 0x560d0e2fd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.571+0000 7f6526e29740 1 --2- 172.21.15.116:0/1151408741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0e2f07e0 0x560d0e2f0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.571+0000 7f6526e29740 1 --2- 172.21.15.116:0/1151408741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d0e2f1d50 0x560d0e2f02a0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.571+0000 7f6526e29740 1 -- 172.21.15.116:0/1151408741 >> 172.21.15.116:0/1151408741 conn(0x560d0e2b7310 msgr2=0x560d0e2ece70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:07.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.571+0000 7f6526e29740 1 -- 172.21.15.116:0/1151408741 shutdown_connections 2024-09-12T03:19:07.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.571+0000 7f6526e29740 1 -- 172.21.15.116:0/1151408741 wait complete. 2024-09-12T03:19:07.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.572+0000 7f6526e29740 1 -- start start 2024-09-12T03:19:07.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.572+0000 7f6526e29740 1 Processor -- start 2024-09-12T03:19:07.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.572+0000 7f6526e29740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d0e2f07e0 0x560d0e2f6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.572+0000 7f6526e29740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d0e2f1d50 0x560d0e2f6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.572+0000 7f6526e29740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0e2fb580 0x560d0e2f7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.572+0000 7f6526e29740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560d0e2ea830 con 0x560d0e2f07e0 2024-09-12T03:19:07.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.572+0000 7f6526e29740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560d0e20f060 con 0x560d0e2f1d50 2024-09-12T03:19:07.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.572+0000 7f6526e29740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560d0e2ea9b0 con 0x560d0e2fb580 2024-09-12T03:19:07.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.572+0000 7f6526e29740 10 client.0 _ll_register_callbacks cb 0x560d0df628c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:07.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.572+0000 7f651e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d0e2f07e0 0x560d0e2f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.572+0000 7f651e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d0e2f07e0 0x560d0e2f6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32938/0 (socket says 172.21.15.116:32938) 2024-09-12T03:19:07.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.572+0000 7f651e59c640 1 -- 172.21.15.116:0/2074205597 learned_addr learned my addr 172.21.15.116:0/2074205597 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:07.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.572+0000 7f651ed9d640 1 --2- 172.21.15.116:0/2074205597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0e2fb580 0x560d0e2f7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.572+0000 7f651e59c640 1 -- 172.21.15.116:0/2074205597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0e2fb580 msgr2=0x560d0e2f7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:07.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.572+0000 7f651e59c640 1 --2- 172.21.15.116:0/2074205597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0e2fb580 0x560d0e2f7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.572+0000 7f6517fff640 1 --2- 172.21.15.116:0/2074205597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d0e2f1d50 0x560d0e2f6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.572+0000 7f651e59c640 1 -- 172.21.15.116:0/2074205597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d0e2f1d50 msgr2=0x560d0e2f6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:07.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.572+0000 7f651e59c640 1 --2- 172.21.15.116:0/2074205597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d0e2f1d50 0x560d0e2f6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.572+0000 7f651e59c640 1 -- 172.21.15.116:0/2074205597 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d0dfc53c0 con 0x560d0e2f07e0 2024-09-12T03:19:07.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.572+0000 7f651e59c640 1 --2- 172.21.15.116:0/2074205597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d0e2f07e0 0x560d0e2f6680 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f65100c4630 tx=0x7f65100b8190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f4791d50a88a0f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:07.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.572+0000 7f6505ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:19:07.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.573+0000 7f6505ffb640 1 -- 172.21.15.116:0/2074205597 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6510049040 con 0x560d0e2f07e0 2024-09-12T03:19:07.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.573+0000 7f6505ffb640 1 -- 172.21.15.116:0/2074205597 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f65100b8560 con 0x560d0e2f07e0 2024-09-12T03:19:07.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.573+0000 7f6526e29740 1 -- 172.21.15.116:0/2074205597 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6518016040 con 0x560d0e2f07e0 2024-09-12T03:19:07.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.573+0000 7f6505ffb640 1 -- 172.21.15.116:0/2074205597 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f65100bb850 con 0x560d0e2f07e0 2024-09-12T03:19:07.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.573+0000 7f6526e29740 10 client.19668 fetch_fsmap learned FSMap version 25 2024-09-12T03:19:07.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.573+0000 7f6526e29740 1 -- 172.21.15.116:0/2074205597 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560d0e30be80 con 0x560d0e2f07e0 2024-09-12T03:19:07.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.573+0000 7f6505ffb640 1 -- 172.21.15.116:0/2074205597 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6518016040 con 0x560d0e2f07e0 2024-09-12T03:19:07.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.573+0000 7f6505ffb640 1 -- 172.21.15.116:0/2074205597 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f651003b460 con 0x560d0e2f07e0 2024-09-12T03:19:07.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.573+0000 7f6526e29740 10 client.19668 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:19:07.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.573+0000 7f6526e29740 10 client.19668 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:19:07.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.573+0000 7f6526e29740 20 client.19668 populate_metadata read hostname 'smithi116' 2024-09-12T03:19:07.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.573+0000 7f6526e29740 1 --2- 172.21.15.116:0/2074205597 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x560d0e30d330 0x560d0e30f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.573+0000 7f6526e29740 4 client.19668 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:19:07.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.573+0000 7f6526e29740 1 -- 172.21.15.116:0/2074205597 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d0e261860 con 0x560d0e30d330 2024-09-12T03:19:07.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.574+0000 7f6517fff640 1 --2- 172.21.15.116:0/2074205597 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x560d0e30d330 0x560d0e30f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.574+0000 7f6505ffb640 10 client.19668 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:19:07.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.574+0000 7f6517fff640 1 --2- 172.21.15.116:0/2074205597 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x560d0e30d330 0x560d0e30f7c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6e6d1d77c1a0648e server_cookie=6cd040f0056f4d3c in_seq=0 out_seq=0 2024-09-12T03:19:07.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.575+0000 7f6505ffb640 1 -- 172.21.15.116:0/2074205597 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560d0e261860 con 0x560d0e30d330 2024-09-12T03:19:07.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.575+0000 7f6505ffb640 10 client.19668 handle_command_reply: tid=0 2024-09-12T03:19:07.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.575+0000 7f6526e29740 1 client.19668 shutdown 2024-09-12T03:19:07.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.575+0000 7f6526e29740 1 -- 172.21.15.116:0/2074205597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d0e2f07e0 msgr2=0x560d0e2f6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:07.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.575+0000 7f6526e29740 1 --2- 172.21.15.116:0/2074205597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d0e2f07e0 0x560d0e2f6680 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f65100c4630 tx=0x7f65100b8190 comp rx=0 tx=0).stop 2024-09-12T03:19:07.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.575+0000 7f6526e29740 1 -- 172.21.15.116:0/2074205597 shutdown_connections 2024-09-12T03:19:07.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.575+0000 7f6526e29740 1 --2- 172.21.15.116:0/2074205597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0e2fb580 0x560d0e2f7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.575+0000 7f6526e29740 1 --2- 172.21.15.116:0/2074205597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d0e2f1d50 0x560d0e2f6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.575+0000 7f6526e29740 1 --2- 172.21.15.116:0/2074205597 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x560d0e30d330 0x560d0e30f7c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.575+0000 7f6526e29740 1 --2- 172.21.15.116:0/2074205597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d0e2f07e0 0x560d0e2f6680 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.575+0000 7f6526e29740 1 -- 172.21.15.116:0/2074205597 >> 172.21.15.116:0/2074205597 conn(0x560d0e2b7310 msgr2=0x560d0e2f2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:07.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.575+0000 7f6505ffb640 0 client.19668 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:19:07.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.575+0000 7f6526e29740 1 -- 172.21.15.116:0/2074205597 shutdown_connections 2024-09-12T03:19:07.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.575+0000 7f6526e29740 1 -- 172.21.15.116:0/2074205597 wait complete. 2024-09-12T03:19:07.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.575+0000 7f6526e29740 20 client.19668 trim_cache size 0 max 16384 2024-09-12T03:19:07.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.587+0000 7f6526e29740 1 Processor -- start 2024-09-12T03:19:07.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.588+0000 7f6526e29740 1 -- start start 2024-09-12T03:19:07.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.588+0000 7f6526e29740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d0e2f07e0 0x560d0e2f6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.588+0000 7f6526e29740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d0e2f1d50 0x560d0e2ebc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.588+0000 7f6526e29740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0e2fb580 0x560d0e3150a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.588+0000 7f6526e29740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f65201adc30 con 0x560d0e2f1d50 2024-09-12T03:19:07.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.588+0000 7f6526e29740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f65201ab500 con 0x560d0e2f07e0 2024-09-12T03:19:07.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.588+0000 7f6526e29740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f65201a9350 con 0x560d0e2fb580 2024-09-12T03:19:07.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.588+0000 7f6517fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d0e2f1d50 0x560d0e2ebc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.588+0000 7f6517fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d0e2f1d50 0x560d0e2ebc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32954/0 (socket says 172.21.15.116:32954) 2024-09-12T03:19:07.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.588+0000 7f6517fff640 1 -- 172.21.15.116:0/1807003847 learned_addr learned my addr 172.21.15.116:0/1807003847 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:07.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.588+0000 7f651e59c640 1 --2- 172.21.15.116:0/1807003847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d0e2f07e0 0x560d0e2f6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.588+0000 7f6517fff640 1 -- 172.21.15.116:0/1807003847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0e2fb580 msgr2=0x560d0e3150a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:07.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.588+0000 7f651ed9d640 1 --2- 172.21.15.116:0/1807003847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0e2fb580 0x560d0e3150a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.588+0000 7f6517fff640 1 --2- 172.21.15.116:0/1807003847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0e2fb580 0x560d0e3150a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.588+0000 7f6517fff640 1 -- 172.21.15.116:0/1807003847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d0e2f07e0 msgr2=0x560d0e2f6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:07.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.588+0000 7f6517fff640 1 --2- 172.21.15.116:0/1807003847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d0e2f07e0 0x560d0e2f6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.588+0000 7f6517fff640 1 -- 172.21.15.116:0/1807003847 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d0e30fe10 con 0x560d0e2f1d50 2024-09-12T03:19:07.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.588+0000 7f651ed9d640 1 --2- 172.21.15.116:0/1807003847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0e2fb580 0x560d0e3150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:07.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.588+0000 7f651e59c640 1 --2- 172.21.15.116:0/1807003847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d0e2f07e0 0x560d0e2f6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:07.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.588+0000 7f6517fff640 1 --2- 172.21.15.116:0/1807003847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d0e2f1d50 0x560d0e2ebc10 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x560d0e2f5f10 tx=0x7f6508007fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6570a9b3fcae9e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:07.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.588+0000 7f6507fff640 1 -- 172.21.15.116:0/1807003847 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6508016b00 con 0x560d0e2f1d50 2024-09-12T03:19:07.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.589+0000 7f6507fff640 1 -- 172.21.15.116:0/1807003847 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6508016ca0 con 0x560d0e2f1d50 2024-09-12T03:19:07.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.589+0000 7f6507fff640 1 -- 172.21.15.116:0/1807003847 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65080145f0 con 0x560d0e2f1d50 2024-09-12T03:19:07.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.589+0000 7f6526e29740 1 -- 172.21.15.116:0/1807003847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d0e2f1d50 msgr2=0x560d0e2ebc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:07.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.589+0000 7f6526e29740 1 --2- 172.21.15.116:0/1807003847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d0e2f1d50 0x560d0e2ebc10 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x560d0e2f5f10 tx=0x7f6508007fd0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.589+0000 7f6526e29740 1 -- 172.21.15.116:0/1807003847 shutdown_connections 2024-09-12T03:19:07.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.589+0000 7f6526e29740 1 --2- 172.21.15.116:0/1807003847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0e2fb580 0x560d0e3150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.589+0000 7f6526e29740 1 --2- 172.21.15.116:0/1807003847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d0e2f1d50 0x560d0e2ebc10 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.589+0000 7f6526e29740 1 --2- 172.21.15.116:0/1807003847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d0e2f07e0 0x560d0e2f6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.589+0000 7f6526e29740 1 -- 172.21.15.116:0/1807003847 >> 172.21.15.116:0/1807003847 conn(0x560d0e2b7310 msgr2=0x560d0e2f6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:07.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.589+0000 7f6526e29740 1 -- 172.21.15.116:0/1807003847 shutdown_connections 2024-09-12T03:19:07.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.589+0000 7f6526e29740 1 -- 172.21.15.116:0/1807003847 wait complete. 2024-09-12T03:19:07.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.589+0000 7f6526e29740 1 -- start start 2024-09-12T03:19:07.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.589+0000 7f6526e29740 1 Processor -- start 2024-09-12T03:19:07.608 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:19:07.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.590+0000 7f6526e29740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d0e2f07e0 0x560d0e304870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.590+0000 7f6526e29740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0e2f1d50 0x560d0e304db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.590+0000 7f6526e29740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d0e2fb580 0x560d0e377070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.590+0000 7f6526e29740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f65201a99c0 con 0x560d0e2f07e0 2024-09-12T03:19:07.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.590+0000 7f6526e29740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f65201aae30 con 0x560d0e2fb580 2024-09-12T03:19:07.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.590+0000 7f6526e29740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f65201aa090 con 0x560d0e2f1d50 2024-09-12T03:19:07.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.590+0000 7f6526e29740 10 client.0 _ll_register_callbacks cb 0x7f6510005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:07.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.590+0000 7f651e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d0e2f07e0 0x560d0e304870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.590+0000 7f651e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d0e2f07e0 0x560d0e304870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32956/0 (socket says 172.21.15.116:32956) 2024-09-12T03:19:07.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.590+0000 7f651e59c640 1 -- 172.21.15.116:0/1569514789 learned_addr learned my addr 172.21.15.116:0/1569514789 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:07.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.590+0000 7f6517fff640 1 --2- 172.21.15.116:0/1569514789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0e2f1d50 0x560d0e304db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.590+0000 7f651ed9d640 1 --2- 172.21.15.116:0/1569514789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d0e2fb580 0x560d0e377070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.590+0000 7f651e59c640 1 -- 172.21.15.116:0/1569514789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0e2f1d50 msgr2=0x560d0e304db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:07.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.590+0000 7f651e59c640 1 --2- 172.21.15.116:0/1569514789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0e2f1d50 0x560d0e304db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.590+0000 7f651e59c640 1 -- 172.21.15.116:0/1569514789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d0e2fb580 msgr2=0x560d0e377070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:07.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.590+0000 7f651e59c640 1 --2- 172.21.15.116:0/1569514789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d0e2fb580 0x560d0e377070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.590+0000 7f651e59c640 1 -- 172.21.15.116:0/1569514789 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d0e2f0f90 con 0x560d0e2f07e0 2024-09-12T03:19:07.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.590+0000 7f651e59c640 1 --2- 172.21.15.116:0/1569514789 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d0e2f07e0 0x560d0e304870 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f6510000c00 tx=0x7f65100314b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f75e6a9cd47c96 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:07.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.590+0000 7f6505ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:19:07.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.590+0000 7f6505ffb640 1 -- 172.21.15.116:0/1569514789 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6510086060 con 0x560d0e2f07e0 2024-09-12T03:19:07.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.590+0000 7f6505ffb640 1 -- 172.21.15.116:0/1569514789 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f65100b8560 con 0x560d0e2f07e0 2024-09-12T03:19:07.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.590+0000 7f6526e29740 1 -- 172.21.15.116:0/1569514789 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6508016ca0 con 0x560d0e2f07e0 2024-09-12T03:19:07.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.591+0000 7f6505ffb640 1 -- 172.21.15.116:0/1569514789 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f65100a1050 con 0x560d0e2f07e0 2024-09-12T03:19:07.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.591+0000 7f6526e29740 10 client.19674 fetch_fsmap learned FSMap version 25 2024-09-12T03:19:07.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.591+0000 7f6526e29740 1 -- 172.21.15.116:0/1569514789 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560d0e2edb70 con 0x560d0e2f07e0 2024-09-12T03:19:07.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.591+0000 7f6505ffb640 1 -- 172.21.15.116:0/1569514789 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6508016ca0 con 0x560d0e2f07e0 2024-09-12T03:19:07.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.591+0000 7f6505ffb640 1 -- 172.21.15.116:0/1569514789 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f65100dab30 con 0x560d0e2f07e0 2024-09-12T03:19:07.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.591+0000 7f6526e29740 10 client.19674 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:19:07.610 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.591+0000 7f6526e29740 10 client.19674 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-12T03:19:07.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.591+0000 7f6526e29740 20 client.19674 populate_metadata read hostname 'smithi116' 2024-09-12T03:19:07.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.591+0000 7f6526e29740 1 --2- 172.21.15.116:0/1569514789 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x560d0e378810 0x560d0e37ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.591+0000 7f6526e29740 4 client.19674 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:19:07.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.591+0000 7f6526e29740 1 -- 172.21.15.116:0/1569514789 --> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560d0e25bd70 con 0x560d0e378810 2024-09-12T03:19:07.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.591+0000 7f6517fff640 1 --2- 172.21.15.116:0/1569514789 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x560d0e378810 0x560d0e37ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.592+0000 7f6517fff640 1 --2- 172.21.15.116:0/1569514789 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x560d0e378810 0x560d0e37ac00 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=afd4b65d2cd10f6e server_cookie=97bbd0ed78365456 in_seq=0 out_seq=0 2024-09-12T03:19:07.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.592+0000 7f6505ffb640 10 client.19674 ms_handle_connect on v2:172.21.15.177:6836/1862929607 2024-09-12T03:19:07.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.592+0000 7f6505ffb640 1 -- 172.21.15.116:0/1569514789 <== mds.2 v2:172.21.15.177:6836/1862929607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560d0e25bd70 con 0x560d0e378810 2024-09-12T03:19:07.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.592+0000 7f6505ffb640 10 client.19674 handle_command_reply: tid=0 2024-09-12T03:19:07.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.592+0000 7f6526e29740 1 client.19674 shutdown 2024-09-12T03:19:07.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.592+0000 7f6526e29740 1 -- 172.21.15.116:0/1569514789 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d0e2f07e0 msgr2=0x560d0e304870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:07.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.592+0000 7f6526e29740 1 --2- 172.21.15.116:0/1569514789 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d0e2f07e0 0x560d0e304870 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f6510000c00 tx=0x7f65100314b0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.592+0000 7f6526e29740 1 -- 172.21.15.116:0/1569514789 shutdown_connections 2024-09-12T03:19:07.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.592+0000 7f6526e29740 1 --2- 172.21.15.116:0/1569514789 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560d0e2fb580 0x560d0e377070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.592+0000 7f6526e29740 1 --2- 172.21.15.116:0/1569514789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0e2f1d50 0x560d0e304db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.592+0000 7f6526e29740 1 --2- 172.21.15.116:0/1569514789 >> [v2:172.21.15.177:6836/1862929607,v1:172.21.15.177:6837/1862929607] conn(0x560d0e378810 0x560d0e37ac00 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.592+0000 7f6526e29740 1 --2- 172.21.15.116:0/1569514789 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560d0e2f07e0 0x560d0e304870 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.592+0000 7f6526e29740 1 -- 172.21.15.116:0/1569514789 >> 172.21.15.116:0/1569514789 conn(0x560d0e2b7310 msgr2=0x560d0e2e9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:07.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.592+0000 7f6505ffb640 0 client.19674 ms_handle_reset on v2:172.21.15.177:6836/1862929607 2024-09-12T03:19:07.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.592+0000 7f6526e29740 1 -- 172.21.15.116:0/1569514789 shutdown_connections 2024-09-12T03:19:07.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.593+0000 7f6526e29740 1 -- 172.21.15.116:0/1569514789 wait complete. 2024-09-12T03:19:07.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.593+0000 7f6526e29740 20 client.19674 trim_cache size 0 max 16384 2024-09-12T03:19:07.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.593+0000 7f6524a50640 1 -- 172.21.15.116:0/1631608884 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f650c0a2b80 msgr2=0x7f650c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:07.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.593+0000 7f6524a50640 1 --2- 172.21.15.116:0/1631608884 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f650c0a2b80 0x7f650c0a5040 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f651801f000 tx=0x7f651801b000 comp rx=0 tx=0).stop 2024-09-12T03:19:07.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.593+0000 7f6524a50640 1 -- 172.21.15.116:0/1631608884 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6520100a80 msgr2=0x7f65201974e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:07.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.593+0000 7f6524a50640 1 --2- 172.21.15.116:0/1631608884 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6520100a80 0x7f65201974e0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f6510009a50 tx=0x7f6510004270 comp rx=0 tx=0).stop 2024-09-12T03:19:07.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.593+0000 7f6524a50640 1 -- 172.21.15.116:0/1631608884 shutdown_connections 2024-09-12T03:19:07.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.593+0000 7f6524a50640 1 --2- 172.21.15.116:0/1631608884 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f650c0a2b80 0x7f650c0a5040 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.593+0000 7f6524a50640 1 --2- 172.21.15.116:0/1631608884 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6520106a80 0x7f652019bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.593+0000 7f6524a50640 1 --2- 172.21.15.116:0/1631608884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6520101420 0x7f6520197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.593+0000 7f6524a50640 1 --2- 172.21.15.116:0/1631608884 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6520100a80 0x7f65201974e0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.593+0000 7f6524a50640 1 -- 172.21.15.116:0/1631608884 >> 172.21.15.116:0/1631608884 conn(0x7f65200fc7a0 msgr2=0x7f65200fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:07.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.593+0000 7f6524a50640 1 -- 172.21.15.116:0/1631608884 shutdown_connections 2024-09-12T03:19:07.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:07.593+0000 7f6524a50640 1 -- 172.21.15.116:0/1631608884 wait complete. 2024-09-12T03:19:08.400 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:19:08.647 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:08.768 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:08 smithi116 ceph-mon[31410]: pgmap v822: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-09-12T03:19:08.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:08 smithi183 ceph-mon[35535]: pgmap v822: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-09-12T03:19:08.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:08 smithi177 ceph-mon[35305]: pgmap v822: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-09-12T03:19:09.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.148+0000 7f1c515ca640 1 -- 172.21.15.116:0/3831044937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c4c100a60 msgr2=0x7f1c4c100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:09.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.148+0000 7f1c515ca640 1 --2- 172.21.15.116:0/3831044937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c4c100a60 0x7f1c4c100ec0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f1c40009930 tx=0x7f1c4002a420 comp rx=0 tx=0).stop 2024-09-12T03:19:09.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.148+0000 7f1c515ca640 1 -- 172.21.15.116:0/3831044937 shutdown_connections 2024-09-12T03:19:09.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.148+0000 7f1c515ca640 1 --2- 172.21.15.116:0/3831044937 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c4c101400 0x7f1c4c10d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.148+0000 7f1c515ca640 1 --2- 172.21.15.116:0/3831044937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c4c100a60 0x7f1c4c100ec0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.148+0000 7f1c515ca640 1 --2- 172.21.15.116:0/3831044937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c4c106a60 0x7f1c4c106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.148+0000 7f1c515ca640 1 -- 172.21.15.116:0/3831044937 >> 172.21.15.116:0/3831044937 conn(0x7f1c4c0fc780 msgr2=0x7f1c4c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:09.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.148+0000 7f1c515ca640 1 -- 172.21.15.116:0/3831044937 shutdown_connections 2024-09-12T03:19:09.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.148+0000 7f1c515ca640 1 -- 172.21.15.116:0/3831044937 wait complete. 2024-09-12T03:19:09.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.149+0000 7f1c515ca640 1 Processor -- start 2024-09-12T03:19:09.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.149+0000 7f1c515ca640 1 -- start start 2024-09-12T03:19:09.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.149+0000 7f1c515ca640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c4c100a60 0x7f1c4c19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:09.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.149+0000 7f1c515ca640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c4c101400 0x7f1c4c19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:09.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.149+0000 7f1c515ca640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c4c106a60 0x7f1c4c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:09.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.149+0000 7f1c515ca640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c110e60 con 0x7f1c4c106a60 2024-09-12T03:19:09.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.149+0000 7f1c515ca640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c110ce0 con 0x7f1c4c100a60 2024-09-12T03:19:09.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.149+0000 7f1c515ca640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c110fe0 con 0x7f1c4c101400 2024-09-12T03:19:09.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.149+0000 7f1c4b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c4c106a60 0x7f1c4c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:09.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.149+0000 7f1c4b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c4c106a60 0x7f1c4c19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32984/0 (socket says 172.21.15.116:32984) 2024-09-12T03:19:09.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.149+0000 7f1c4b7fe640 1 -- 172.21.15.116:0/1929024241 learned_addr learned my addr 172.21.15.116:0/1929024241 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:09.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.149+0000 7f1c4a7fc640 1 --2- 172.21.15.116:0/1929024241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c4c101400 0x7f1c4c19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:09.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.149+0000 7f1c4affd640 1 --2- 172.21.15.116:0/1929024241 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c4c100a60 0x7f1c4c19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:09.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.149+0000 7f1c4b7fe640 1 -- 172.21.15.116:0/1929024241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c4c101400 msgr2=0x7f1c4c19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:09.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.150+0000 7f1c4b7fe640 1 --2- 172.21.15.116:0/1929024241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c4c101400 0x7f1c4c19aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.150+0000 7f1c4b7fe640 1 -- 172.21.15.116:0/1929024241 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c4c100a60 msgr2=0x7f1c4c19a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:09.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.150+0000 7f1c4b7fe640 1 --2- 172.21.15.116:0/1929024241 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c4c100a60 0x7f1c4c19a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.150+0000 7f1c4b7fe640 1 -- 172.21.15.116:0/1929024241 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c4c19f610 con 0x7f1c4c106a60 2024-09-12T03:19:09.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.150+0000 7f1c4affd640 1 --2- 172.21.15.116:0/1929024241 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c4c100a60 0x7f1c4c19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:09.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.150+0000 7f1c4b7fe640 1 --2- 172.21.15.116:0/1929024241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c4c106a60 0x7f1c4c19ef30 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400cce0 tx=0x7f1c4400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:09.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.150+0000 7f1c33fff640 1 -- 172.21.15.116:0/1929024241 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4400f450 con 0x7f1c4c106a60 2024-09-12T03:19:09.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.150+0000 7f1c33fff640 1 -- 172.21.15.116:0/1929024241 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c4400fad0 con 0x7f1c4c106a60 2024-09-12T03:19:09.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.150+0000 7f1c515ca640 1 -- 172.21.15.116:0/1929024241 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c4c19f900 con 0x7f1c4c106a60 2024-09-12T03:19:09.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.150+0000 7f1c33fff640 1 -- 172.21.15.116:0/1929024241 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c44018c10 con 0x7f1c4c106a60 2024-09-12T03:19:09.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.150+0000 7f1c515ca640 1 -- 172.21.15.116:0/1929024241 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c4c1a71e0 con 0x7f1c4c106a60 2024-09-12T03:19:09.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.153+0000 7f1c539a3740 1 Processor -- start 2024-09-12T03:19:09.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.153+0000 7f1c33fff640 1 -- 172.21.15.116:0/1929024241 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1c4400f5f0 con 0x7f1c4c106a60 2024-09-12T03:19:09.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.153+0000 7f1c539a3740 1 -- start start 2024-09-12T03:19:09.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.153+0000 7f1c539a3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce48249b40 0x55ce48248090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:09.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.153+0000 7f1c539a3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce482485d0 0x55ce48248a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:09.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.153+0000 7f1c539a3740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce48253370 0x55ce48255760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:09.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.153+0000 7f1c539a3740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ce481ae2f0 con 0x55ce48253370 2024-09-12T03:19:09.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.153+0000 7f1c539a3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ce48155cf0 con 0x55ce48249b40 2024-09-12T03:19:09.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.153+0000 7f1c539a3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ce481aa2b0 con 0x55ce482485d0 2024-09-12T03:19:09.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.153+0000 7f1c4a7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce48253370 0x55ce48255760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:09.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.153+0000 7f1c4a7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce48253370 0x55ce48255760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32998/0 (socket says 172.21.15.116:32998) 2024-09-12T03:19:09.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.153+0000 7f1c4a7fc640 1 -- 172.21.15.116:0/3289369673 learned_addr learned my addr 172.21.15.116:0/3289369673 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:09.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.153+0000 7f1c4affd640 1 --2- 172.21.15.116:0/3289369673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce482485d0 0x55ce48248a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:09.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.153+0000 7f1c4a7fc640 1 --2- 172.21.15.116:0/3289369673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce48249b40 0x55ce48248090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:09.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.154+0000 7f1c33fff640 1 --2- 172.21.15.116:0/1929024241 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1c340a2b80 0x7f1c340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:09.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.154+0000 7f1c4a7fc640 1 -- 172.21.15.116:0/3289369673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce482485d0 msgr2=0x55ce48248a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:09.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.154+0000 7f1c4a7fc640 1 --2- 172.21.15.116:0/3289369673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce482485d0 0x55ce48248a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.154+0000 7f1c4a7fc640 1 -- 172.21.15.116:0/3289369673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce48249b40 msgr2=0x55ce48248090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:09.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.154+0000 7f1c4a7fc640 1 --2- 172.21.15.116:0/3289369673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce48249b40 0x55ce48248090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.154+0000 7f1c4a7fc640 1 -- 172.21.15.116:0/3289369673 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce4813a3a0 con 0x55ce48253370 2024-09-12T03:19:09.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.154+0000 7f1c4a7fc640 1 --2- 172.21.15.116:0/3289369673 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce48253370 0x55ce48255760 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f1c40005940 tx=0x7f1c400058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33968dbaf446d92 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:09.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.154+0000 7f1c4b7fe640 1 --2- 172.21.15.116:0/1929024241 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1c340a2b80 0x7f1c340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:09.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.154+0000 7f1c33fff640 1 -- 172.21.15.116:0/1929024241 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1c440cf7a0 con 0x7f1c4c106a60 2024-09-12T03:19:09.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.154+0000 7f1c317fa640 1 -- 172.21.15.116:0/3289369673 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c40043030 con 0x55ce48253370 2024-09-12T03:19:09.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.154+0000 7f1c317fa640 1 -- 172.21.15.116:0/3289369673 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c40031750 con 0x55ce48253370 2024-09-12T03:19:09.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.154+0000 7f1c317fa640 1 -- 172.21.15.116:0/3289369673 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c40035040 con 0x55ce48253370 2024-09-12T03:19:09.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.155+0000 7f1c539a3740 1 -- 172.21.15.116:0/3289369673 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce48253370 msgr2=0x55ce48255760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:09.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.155+0000 7f1c4b7fe640 1 --2- 172.21.15.116:0/1929024241 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1c340a2b80 0x7f1c340a5040 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1c44098490 tx=0x7f1c44020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:09.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.155+0000 7f1c539a3740 1 --2- 172.21.15.116:0/3289369673 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce48253370 0x55ce48255760 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f1c40005940 tx=0x7f1c400058d0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.155+0000 7f1c539a3740 1 -- 172.21.15.116:0/3289369673 shutdown_connections 2024-09-12T03:19:09.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.155+0000 7f1c539a3740 1 --2- 172.21.15.116:0/3289369673 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce48253370 0x55ce48255760 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.155+0000 7f1c539a3740 1 --2- 172.21.15.116:0/3289369673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce482485d0 0x55ce48248a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.155+0000 7f1c539a3740 1 --2- 172.21.15.116:0/3289369673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce48249b40 0x55ce48248090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.155+0000 7f1c539a3740 1 -- 172.21.15.116:0/3289369673 >> 172.21.15.116:0/3289369673 conn(0x55ce48211310 msgr2=0x55ce48244c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:09.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.155+0000 7f1c539a3740 1 -- 172.21.15.116:0/3289369673 shutdown_connections 2024-09-12T03:19:09.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.155+0000 7f1c539a3740 1 -- 172.21.15.116:0/3289369673 wait complete. 2024-09-12T03:19:09.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.155+0000 7f1c539a3740 1 -- start start 2024-09-12T03:19:09.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.155+0000 7f1c539a3740 1 Processor -- start 2024-09-12T03:19:09.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.156+0000 7f1c539a3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce482485d0 0x55ce48252d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:09.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.156+0000 7f1c539a3740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce48249b40 0x55ce48250020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:09.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.156+0000 7f1c539a3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce48253370 0x55ce48250610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:09.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.156+0000 7f1c539a3740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ce48256010 con 0x55ce48249b40 2024-09-12T03:19:09.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.156+0000 7f1c539a3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ce48169060 con 0x55ce48253370 2024-09-12T03:19:09.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.156+0000 7f1c539a3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ce48256190 con 0x55ce482485d0 2024-09-12T03:19:09.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.156+0000 7f1c539a3740 10 client.0 _ll_register_callbacks cb 0x55ce47ebc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:09.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.156+0000 7f1c4affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce48249b40 0x55ce48250020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:09.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.156+0000 7f1c4affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce48249b40 0x55ce48250020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33000/0 (socket says 172.21.15.116:33000) 2024-09-12T03:19:09.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.156+0000 7f1c4affd640 1 -- 172.21.15.116:0/3309043624 learned_addr learned my addr 172.21.15.116:0/3309043624 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:09.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.156+0000 7f1c4a7fc640 1 --2- 172.21.15.116:0/3309043624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce482485d0 0x55ce48252d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:09.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.156+0000 7f1c4affd640 1 -- 172.21.15.116:0/3309043624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce482485d0 msgr2=0x55ce48252d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:09.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.156+0000 7f1c4a7fc640 1 --2- 172.21.15.116:0/3309043624 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce48253370 0x55ce48250610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:09.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.156+0000 7f1c4affd640 1 --2- 172.21.15.116:0/3309043624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce482485d0 0x55ce48252d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.156+0000 7f1c4affd640 1 -- 172.21.15.116:0/3309043624 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce48253370 msgr2=0x55ce48250610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:09.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.156+0000 7f1c4affd640 1 --2- 172.21.15.116:0/3309043624 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce48253370 0x55ce48250610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.156+0000 7f1c4affd640 1 -- 172.21.15.116:0/3309043624 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce47f1f3c0 con 0x55ce48249b40 2024-09-12T03:19:09.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.156+0000 7f1c4affd640 1 --2- 172.21.15.116:0/3309043624 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce48249b40 0x55ce48250020 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c010750 tx=0x7f1c3c010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f6b5dd6461f531 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:09.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.156+0000 7f1c2eff5640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:19:09.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.156+0000 7f1c2eff5640 1 -- 172.21.15.116:0/3309043624 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c01d060 con 0x55ce48249b40 2024-09-12T03:19:09.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.156+0000 7f1c2eff5640 1 -- 172.21.15.116:0/3309043624 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1c3c00eda0 con 0x55ce48249b40 2024-09-12T03:19:09.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.156+0000 7f1c539a3740 1 -- 172.21.15.116:0/3309043624 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c40031750 con 0x55ce48249b40 2024-09-12T03:19:09.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.156+0000 7f1c2eff5640 1 -- 172.21.15.116:0/3309043624 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1c3c010e60 con 0x55ce48249b40 2024-09-12T03:19:09.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.157+0000 7f1c539a3740 10 client.19686 fetch_fsmap learned FSMap version 25 2024-09-12T03:19:09.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.157+0000 7f1c539a3740 1 -- 172.21.15.116:0/3309043624 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ce48261b10 con 0x55ce48249b40 2024-09-12T03:19:09.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.157+0000 7f1c2eff5640 1 -- 172.21.15.116:0/3309043624 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c40031750 con 0x55ce48249b40 2024-09-12T03:19:09.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.157+0000 7f1c2eff5640 1 -- 172.21.15.116:0/3309043624 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1c3c00d550 con 0x55ce48249b40 2024-09-12T03:19:09.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.157+0000 7f1c539a3740 10 client.19686 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:19:09.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.157+0000 7f1c539a3740 10 client.19686 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:19:09.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.157+0000 7f1c539a3740 20 client.19686 populate_metadata read hostname 'smithi116' 2024-09-12T03:19:09.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.157+0000 7f1c539a3740 1 --2- 172.21.15.116:0/3309043624 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55ce48263020 0x55ce48265440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:09.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.157+0000 7f1c539a3740 4 client.19686 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:19:09.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.157+0000 7f1c539a3740 1 -- 172.21.15.116:0/3309043624 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce481bb860 con 0x55ce48263020 2024-09-12T03:19:09.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.157+0000 7f1c4a7fc640 1 --2- 172.21.15.116:0/3309043624 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55ce48263020 0x55ce48265440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:09.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.158+0000 7f1c2eff5640 10 client.19686 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:19:09.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.158+0000 7f1c4a7fc640 1 --2- 172.21.15.116:0/3309043624 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55ce48263020 0x55ce48265440 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4ec025e616f0fddc server_cookie=1fddc195b02a19b3 in_seq=0 out_seq=0 2024-09-12T03:19:09.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.158+0000 7f1c2eff5640 1 -- 172.21.15.116:0/3309043624 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ce481bb860 con 0x55ce48263020 2024-09-12T03:19:09.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.158+0000 7f1c2eff5640 10 client.19686 handle_command_reply: tid=0 2024-09-12T03:19:09.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.158+0000 7f1c539a3740 1 client.19686 shutdown 2024-09-12T03:19:09.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.158+0000 7f1c539a3740 1 -- 172.21.15.116:0/3309043624 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce48249b40 msgr2=0x55ce48250020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:09.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.158+0000 7f1c539a3740 1 --2- 172.21.15.116:0/3309043624 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce48249b40 0x55ce48250020 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c010750 tx=0x7f1c3c010c10 comp rx=0 tx=0).stop 2024-09-12T03:19:09.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.158+0000 7f1c539a3740 1 -- 172.21.15.116:0/3309043624 shutdown_connections 2024-09-12T03:19:09.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.158+0000 7f1c539a3740 1 --2- 172.21.15.116:0/3309043624 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55ce48263020 0x55ce48265440 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.158+0000 7f1c539a3740 1 --2- 172.21.15.116:0/3309043624 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce48253370 0x55ce48250610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.158+0000 7f1c539a3740 1 --2- 172.21.15.116:0/3309043624 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce48249b40 0x55ce48250020 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.158+0000 7f1c2eff5640 0 client.19686 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:19:09.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.158+0000 7f1c539a3740 1 --2- 172.21.15.116:0/3309043624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce482485d0 0x55ce48252d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.158+0000 7f1c539a3740 1 -- 172.21.15.116:0/3309043624 >> 172.21.15.116:0/3309043624 conn(0x55ce48211310 msgr2=0x55ce4824a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:09.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.158+0000 7f1c539a3740 1 -- 172.21.15.116:0/3309043624 shutdown_connections 2024-09-12T03:19:09.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.159+0000 7f1c539a3740 1 -- 172.21.15.116:0/3309043624 wait complete. 2024-09-12T03:19:09.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.159+0000 7f1c539a3740 20 client.19686 trim_cache size 0 max 16384 2024-09-12T03:19:09.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.171+0000 7f1c539a3740 1 Processor -- start 2024-09-12T03:19:09.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.171+0000 7f1c539a3740 1 -- start start 2024-09-12T03:19:09.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.171+0000 7f1c539a3740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce482485d0 0x55ce4824b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:09.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.171+0000 7f1c539a3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce48249b40 0x55ce4824b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:09.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.171+0000 7f1c539a3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce48253370 0x55ce4826ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:09.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.171+0000 7f1c539a3740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c1b1e90 con 0x55ce482485d0 2024-09-12T03:19:09.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.171+0000 7f1c539a3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c1af760 con 0x55ce48249b40 2024-09-12T03:19:09.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.171+0000 7f1c539a3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c1ad5b0 con 0x55ce48253370 2024-09-12T03:19:09.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.172+0000 7f1c4a7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce482485d0 0x55ce4824b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:09.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.172+0000 7f1c4a7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce482485d0 0x55ce4824b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33014/0 (socket says 172.21.15.116:33014) 2024-09-12T03:19:09.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.172+0000 7f1c4a7fc640 1 -- 172.21.15.116:0/2965037398 learned_addr learned my addr 172.21.15.116:0/2965037398 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:09.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.172+0000 7f1c4affd640 1 --2- 172.21.15.116:0/2965037398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce48249b40 0x55ce4824b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:09.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.172+0000 7f1c4a7fc640 1 -- 172.21.15.116:0/2965037398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce48253370 msgr2=0x55ce4826ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:09.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.172+0000 7f1c4a7fc640 1 --2- 172.21.15.116:0/2965037398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce48253370 0x55ce4826ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.172+0000 7f1c4a7fc640 1 -- 172.21.15.116:0/2965037398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce48249b40 msgr2=0x55ce4824b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:09.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.172+0000 7f1c4a7fc640 1 --2- 172.21.15.116:0/2965037398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce48249b40 0x55ce4824b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.172+0000 7f1c4a7fc640 1 -- 172.21.15.116:0/2965037398 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce48252990 con 0x55ce482485d0 2024-09-12T03:19:09.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.172+0000 7f1c4affd640 1 --2- 172.21.15.116:0/2965037398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce48249b40 0x55ce4824b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:09.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.172+0000 7f1c4a7fc640 1 --2- 172.21.15.116:0/2965037398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce482485d0 0x55ce4824b120 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f1c40009930 tx=0x7f1c40005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb136d2184787d44 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:09.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.172+0000 7f1c30ff9640 1 -- 172.21.15.116:0/2965037398 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c40035040 con 0x55ce482485d0 2024-09-12T03:19:09.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.172+0000 7f1c30ff9640 1 -- 172.21.15.116:0/2965037398 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c400316d0 con 0x55ce482485d0 2024-09-12T03:19:09.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.172+0000 7f1c30ff9640 1 -- 172.21.15.116:0/2965037398 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c400496b0 con 0x55ce482485d0 2024-09-12T03:19:09.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.173+0000 7f1c539a3740 1 -- 172.21.15.116:0/2965037398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce482485d0 msgr2=0x55ce4824b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:09.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.173+0000 7f1c539a3740 1 --2- 172.21.15.116:0/2965037398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce482485d0 0x55ce4824b120 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f1c40009930 tx=0x7f1c40005fd0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.173+0000 7f1c539a3740 1 -- 172.21.15.116:0/2965037398 shutdown_connections 2024-09-12T03:19:09.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.173+0000 7f1c539a3740 1 --2- 172.21.15.116:0/2965037398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce48253370 0x55ce4826ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.173+0000 7f1c539a3740 1 --2- 172.21.15.116:0/2965037398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce48249b40 0x55ce4824b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.173+0000 7f1c539a3740 1 --2- 172.21.15.116:0/2965037398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce482485d0 0x55ce4824b120 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.173+0000 7f1c539a3740 1 -- 172.21.15.116:0/2965037398 >> 172.21.15.116:0/2965037398 conn(0x55ce48211310 msgr2=0x55ce482520a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:09.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.173+0000 7f1c539a3740 1 -- 172.21.15.116:0/2965037398 shutdown_connections 2024-09-12T03:19:09.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.173+0000 7f1c539a3740 1 -- 172.21.15.116:0/2965037398 wait complete. 2024-09-12T03:19:09.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.173+0000 7f1c539a3740 1 -- start start 2024-09-12T03:19:09.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.173+0000 7f1c539a3740 1 Processor -- start 2024-09-12T03:19:09.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.173+0000 7f1c539a3740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce482485d0 0x55ce4825b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:09.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.173+0000 7f1c539a3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce48249b40 0x55ce4825b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:09.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.174+0000 7f1c539a3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce48253370 0x55ce482583f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:09.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.174+0000 7f1c539a3740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c1adc20 con 0x55ce482485d0 2024-09-12T03:19:09.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.174+0000 7f1c539a3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c1af090 con 0x55ce48249b40 2024-09-12T03:19:09.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.174+0000 7f1c539a3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c1ae2f0 con 0x55ce48253370 2024-09-12T03:19:09.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.174+0000 7f1c539a3740 10 client.0 _ll_register_callbacks cb 0x7f1c3c00b5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:09.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.174+0000 7f1c4a7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce482485d0 0x55ce4825b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:09.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.174+0000 7f1c4a7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce482485d0 0x55ce4825b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33024/0 (socket says 172.21.15.116:33024) 2024-09-12T03:19:09.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.174+0000 7f1c4a7fc640 1 -- 172.21.15.116:0/4085546919 learned_addr learned my addr 172.21.15.116:0/4085546919 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:09.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.174+0000 7f1c4affd640 1 --2- 172.21.15.116:0/4085546919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce48249b40 0x55ce4825b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:09.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.174+0000 7f1c4a7fc640 1 -- 172.21.15.116:0/4085546919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce48253370 msgr2=0x55ce482583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:09.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.174+0000 7f1c4a7fc640 1 --2- 172.21.15.116:0/4085546919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce48253370 0x55ce482583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.174+0000 7f1c4a7fc640 1 -- 172.21.15.116:0/4085546919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce48249b40 msgr2=0x55ce4825b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:09.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.174+0000 7f1c4a7fc640 1 --2- 172.21.15.116:0/4085546919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce48249b40 0x55ce4825b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.174+0000 7f1c4a7fc640 1 -- 172.21.15.116:0/4085546919 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce48258930 con 0x55ce482485d0 2024-09-12T03:19:09.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.174+0000 7f1c4affd640 1 --2- 172.21.15.116:0/4085546919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce48249b40 0x55ce4825b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:09.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.174+0000 7f1c4a7fc640 1 --2- 172.21.15.116:0/4085546919 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce482485d0 0x55ce4825b420 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f1c40009a50 tx=0x7f1c40005980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f678d2c514f740 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:09.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.174+0000 7f1c2eff5640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:19:09.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.175+0000 7f1c2eff5640 1 -- 172.21.15.116:0/4085546919 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c40060060 con 0x55ce482485d0 2024-09-12T03:19:09.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.175+0000 7f1c2eff5640 1 -- 172.21.15.116:0/4085546919 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f1c4003d860 con 0x55ce482485d0 2024-09-12T03:19:09.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.175+0000 7f1c539a3740 1 -- 172.21.15.116:0/4085546919 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c400316d0 con 0x55ce482485d0 2024-09-12T03:19:09.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.175+0000 7f1c2eff5640 1 -- 172.21.15.116:0/4085546919 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1c40056070 con 0x55ce482485d0 2024-09-12T03:19:09.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.175+0000 7f1c539a3740 10 client.19692 fetch_fsmap learned FSMap version 25 2024-09-12T03:19:09.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.175+0000 7f1c539a3740 1 -- 172.21.15.116:0/4085546919 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ce482d6b30 con 0x55ce482485d0 2024-09-12T03:19:09.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.175+0000 7f1c2eff5640 1 -- 172.21.15.116:0/4085546919 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c400316d0 con 0x55ce482485d0 2024-09-12T03:19:09.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.175+0000 7f1c2eff5640 1 -- 172.21.15.116:0/4085546919 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1c4003a040 con 0x55ce482485d0 2024-09-12T03:19:09.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.175+0000 7f1c539a3740 10 client.19692 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:19:09.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.175+0000 7f1c539a3740 10 client.19692 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-12T03:19:09.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.175+0000 7f1c539a3740 20 client.19692 populate_metadata read hostname 'smithi116' 2024-09-12T03:19:09.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.175+0000 7f1c539a3740 1 --2- 172.21.15.116:0/4085546919 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55ce482d6d60 0x55ce482d9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:09.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.175+0000 7f1c539a3740 4 client.19692 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:19:09.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.175+0000 7f1c539a3740 1 -- 172.21.15.116:0/4085546919 --> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ce481b5d70 con 0x55ce482d6d60 2024-09-12T03:19:09.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.176+0000 7f1c4affd640 1 --2- 172.21.15.116:0/4085546919 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55ce482d6d60 0x55ce482d9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:09.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.176+0000 7f1c2eff5640 10 client.19692 ms_handle_connect on v2:172.21.15.183:6838/1983839027 2024-09-12T03:19:09.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.176+0000 7f1c4affd640 1 --2- 172.21.15.116:0/4085546919 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55ce482d6d60 0x55ce482d9150 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b4158ce8155838f9 server_cookie=ccc79351d216ee3f in_seq=0 out_seq=0 2024-09-12T03:19:09.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.176+0000 7f1c2eff5640 1 -- 172.21.15.116:0/4085546919 <== mds.1 v2:172.21.15.183:6838/1983839027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ce481b5d70 con 0x55ce482d6d60 2024-09-12T03:19:09.197 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.176+0000 7f1c2eff5640 10 client.19692 handle_command_reply: tid=0 2024-09-12T03:19:09.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.176+0000 7f1c539a3740 1 client.19692 shutdown 2024-09-12T03:19:09.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.176+0000 7f1c539a3740 1 -- 172.21.15.116:0/4085546919 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce482485d0 msgr2=0x55ce4825b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:09.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.176+0000 7f1c539a3740 1 --2- 172.21.15.116:0/4085546919 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce482485d0 0x55ce4825b420 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f1c40009a50 tx=0x7f1c40005980 comp rx=0 tx=0).stop 2024-09-12T03:19:09.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.176+0000 7f1c539a3740 1 -- 172.21.15.116:0/4085546919 shutdown_connections 2024-09-12T03:19:09.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.176+0000 7f1c539a3740 1 --2- 172.21.15.116:0/4085546919 >> [v2:172.21.15.183:6838/1983839027,v1:172.21.15.183:6839/1983839027] conn(0x55ce482d6d60 0x55ce482d9150 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.176+0000 7f1c539a3740 1 --2- 172.21.15.116:0/4085546919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce48253370 0x55ce482583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.176+0000 7f1c539a3740 1 --2- 172.21.15.116:0/4085546919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ce48249b40 0x55ce4825b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.198 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.176+0000 7f1c539a3740 1 --2- 172.21.15.116:0/4085546919 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce482485d0 0x55ce4825b420 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.176+0000 7f1c2eff5640 0 client.19692 ms_handle_reset on v2:172.21.15.183:6838/1983839027 2024-09-12T03:19:09.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.176+0000 7f1c539a3740 1 -- 172.21.15.116:0/4085546919 >> 172.21.15.116:0/4085546919 conn(0x55ce48211310 msgr2=0x55ce482434e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:09.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.176+0000 7f1c539a3740 1 -- 172.21.15.116:0/4085546919 shutdown_connections 2024-09-12T03:19:09.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.176+0000 7f1c539a3740 1 -- 172.21.15.116:0/4085546919 wait complete. 2024-09-12T03:19:09.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.176+0000 7f1c539a3740 20 client.19692 trim_cache size 0 max 16384 2024-09-12T03:19:09.199 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:19:09.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.177+0000 7f1c515ca640 1 -- 172.21.15.116:0/1929024241 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1c340a2b80 msgr2=0x7f1c340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:09.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.177+0000 7f1c515ca640 1 --2- 172.21.15.116:0/1929024241 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1c340a2b80 0x7f1c340a5040 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1c44098490 tx=0x7f1c44020000 comp rx=0 tx=0).stop 2024-09-12T03:19:09.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.177+0000 7f1c515ca640 1 -- 172.21.15.116:0/1929024241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c4c106a60 msgr2=0x7f1c4c19ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:09.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.177+0000 7f1c515ca640 1 --2- 172.21.15.116:0/1929024241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c4c106a60 0x7f1c4c19ef30 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400cce0 tx=0x7f1c4400a620 comp rx=0 tx=0).stop 2024-09-12T03:19:09.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.177+0000 7f1c515ca640 1 -- 172.21.15.116:0/1929024241 shutdown_connections 2024-09-12T03:19:09.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.177+0000 7f1c515ca640 1 --2- 172.21.15.116:0/1929024241 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f1c340a2b80 0x7f1c340a5040 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.177+0000 7f1c515ca640 1 --2- 172.21.15.116:0/1929024241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c4c106a60 0x7f1c4c19ef30 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.177+0000 7f1c515ca640 1 --2- 172.21.15.116:0/1929024241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c4c101400 0x7f1c4c19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.177+0000 7f1c515ca640 1 --2- 172.21.15.116:0/1929024241 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c4c100a60 0x7f1c4c19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:09.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.177+0000 7f1c515ca640 1 -- 172.21.15.116:0/1929024241 >> 172.21.15.116:0/1929024241 conn(0x7f1c4c0fc780 msgr2=0x7f1c4c06a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:09.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.177+0000 7f1c515ca640 1 -- 172.21.15.116:0/1929024241 shutdown_connections 2024-09-12T03:19:09.201 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:09.177+0000 7f1c515ca640 1 -- 172.21.15.116:0/1929024241 wait complete. 2024-09-12T03:19:10.065 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:19:10.309 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:10.724 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:10 smithi116 ceph-mon[31410]: pgmap v823: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-09-12T03:19:10.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.806+0000 7f77fa4b6640 1 -- 172.21.15.116:0/3207153589 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77f4106a80 msgr2=0x7f77f4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:10.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.806+0000 7f77fa4b6640 1 --2- 172.21.15.116:0/3207153589 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77f4106a80 0x7f77f4106e60 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f77e4009930 tx=0x7f77e402a420 comp rx=0 tx=0).stop 2024-09-12T03:19:10.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.806+0000 7f77fa4b6640 1 -- 172.21.15.116:0/3207153589 shutdown_connections 2024-09-12T03:19:10.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.806+0000 7f77fa4b6640 1 --2- 172.21.15.116:0/3207153589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77f4101420 0x7f77f410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.806+0000 7f77fa4b6640 1 --2- 172.21.15.116:0/3207153589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77f4100a80 0x7f77f4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.806+0000 7f77fa4b6640 1 --2- 172.21.15.116:0/3207153589 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77f4106a80 0x7f77f4106e60 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.806+0000 7f77fa4b6640 1 -- 172.21.15.116:0/3207153589 >> 172.21.15.116:0/3207153589 conn(0x7f77f40fc7a0 msgr2=0x7f77f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:10.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.806+0000 7f77fa4b6640 1 -- 172.21.15.116:0/3207153589 shutdown_connections 2024-09-12T03:19:10.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.806+0000 7f77fa4b6640 1 -- 172.21.15.116:0/3207153589 wait complete. 2024-09-12T03:19:10.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.807+0000 7f77fa4b6640 1 Processor -- start 2024-09-12T03:19:10.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.807+0000 7f77fa4b6640 1 -- start start 2024-09-12T03:19:10.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.807+0000 7f77fa4b6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77f4100a80 0x7f77f410e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:10.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.807+0000 7f77fa4b6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77f4101420 0x7f77f410e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:10.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.807+0000 7f77fa4b6640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77f4106a80 0x7f77f410ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:10.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.807+0000 7f77fa4b6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f77f4111fa0 con 0x7f77f4100a80 2024-09-12T03:19:10.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.807+0000 7f77fa4b6640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f77f4111e20 con 0x7f77f4106a80 2024-09-12T03:19:10.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.807+0000 7f77fa4b6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f77f4112120 con 0x7f77f4101420 2024-09-12T03:19:10.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.807+0000 7f77f3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77f4100a80 0x7f77f410e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:10.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.807+0000 7f77f3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77f4100a80 0x7f77f410e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33034/0 (socket says 172.21.15.116:33034) 2024-09-12T03:19:10.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.807+0000 7f77f3fff640 1 -- 172.21.15.116:0/787756410 learned_addr learned my addr 172.21.15.116:0/787756410 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:10.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.807+0000 7f77ebfff640 1 --2- 172.21.15.116:0/787756410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77f4101420 0x7f77f410e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:10.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.807+0000 7f77f8a53640 1 --2- 172.21.15.116:0/787756410 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77f4106a80 0x7f77f410ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:10.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.807+0000 7f77f3fff640 1 -- 172.21.15.116:0/787756410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77f4101420 msgr2=0x7f77f410e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:10.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.807+0000 7f77f3fff640 1 --2- 172.21.15.116:0/787756410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77f4101420 0x7f77f410e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.807+0000 7f77f3fff640 1 -- 172.21.15.116:0/787756410 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77f4106a80 msgr2=0x7f77f410ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:10.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.807+0000 7f77f3fff640 1 --2- 172.21.15.116:0/787756410 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77f4106a80 0x7f77f410ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.807+0000 7f77f3fff640 1 -- 172.21.15.116:0/787756410 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f41a6960 con 0x7f77f4100a80 2024-09-12T03:19:10.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.807+0000 7f77f8a53640 1 --2- 172.21.15.116:0/787756410 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77f4106a80 0x7f77f410ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:10.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.808+0000 7f77f3fff640 1 --2- 172.21.15.116:0/787756410 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77f4100a80 0x7f77f410e3e0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f77e4009230 tx=0x7f77e4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:10.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.808+0000 7f77f1ffb640 1 -- 172.21.15.116:0/787756410 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e403f030 con 0x7f77f4100a80 2024-09-12T03:19:10.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.808+0000 7f77f1ffb640 1 -- 172.21.15.116:0/787756410 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77e4003ad0 con 0x7f77f4100a80 2024-09-12T03:19:10.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.808+0000 7f77f1ffb640 1 -- 172.21.15.116:0/787756410 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e4035070 con 0x7f77f4100a80 2024-09-12T03:19:10.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.808+0000 7f77fa4b6640 1 -- 172.21.15.116:0/787756410 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77f41a6bf0 con 0x7f77f4100a80 2024-09-12T03:19:10.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.808+0000 7f77fa4b6640 1 -- 172.21.15.116:0/787756410 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77f41a70d0 con 0x7f77f4100a80 2024-09-12T03:19:10.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.810+0000 7f77f1ffb640 1 -- 172.21.15.116:0/787756410 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f77e4042030 con 0x7f77f4100a80 2024-09-12T03:19:10.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.810+0000 7f77fc88f740 1 Processor -- start 2024-09-12T03:19:10.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.810+0000 7f77fc88f740 1 -- start start 2024-09-12T03:19:10.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.811+0000 7f77fc88f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4916f5b40 0x55d4916f4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:10.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.811+0000 7f77fc88f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4916f45d0 0x55d4916f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:10.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.811+0000 7f77fc88f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d4916ff370 0x55d491701760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:10.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.811+0000 7f77fc88f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d49165a2f0 con 0x55d4916ff370 2024-09-12T03:19:10.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.811+0000 7f77fc88f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d491601cf0 con 0x55d4916f45d0 2024-09-12T03:19:10.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.811+0000 7f77fc88f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d4916562b0 con 0x55d4916f5b40 2024-09-12T03:19:10.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.811+0000 7f77ebfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d4916ff370 0x55d491701760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:10.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.811+0000 7f77f1ffb640 1 --2- 172.21.15.116:0/787756410 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f77e00a2b80 0x7f77e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:10.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.811+0000 7f77ebfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d4916ff370 0x55d491701760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33050/0 (socket says 172.21.15.116:33050) 2024-09-12T03:19:10.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.811+0000 7f77ebfff640 1 -- 172.21.15.116:0/3028778602 learned_addr learned my addr 172.21.15.116:0/3028778602 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:10.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.811+0000 7f77f3fff640 1 --2- 172.21.15.116:0/3028778602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4916f45d0 0x55d4916f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:10.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.811+0000 7f77f8a53640 1 --2- 172.21.15.116:0/787756410 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f77e00a2b80 0x7f77e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:10.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.811+0000 7f77f1ffb640 1 -- 172.21.15.116:0/787756410 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f77e40ef510 con 0x7f77f4100a80 2024-09-12T03:19:10.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.811+0000 7f77ebfff640 1 -- 172.21.15.116:0/3028778602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4916f5b40 msgr2=0x55d4916f4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:10.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.811+0000 7f77ebfff640 1 --2- 172.21.15.116:0/3028778602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4916f5b40 0x55d4916f4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.811+0000 7f77ebfff640 1 -- 172.21.15.116:0/3028778602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4916f45d0 msgr2=0x55d4916f4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:10.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.811+0000 7f77ebfff640 1 --2- 172.21.15.116:0/3028778602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4916f45d0 0x55d4916f4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.811+0000 7f77ebfff640 1 -- 172.21.15.116:0/3028778602 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4915e63a0 con 0x55d4916ff370 2024-09-12T03:19:10.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.811+0000 7f77ebfff640 1 --2- 172.21.15.116:0/3028778602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d4916ff370 0x55d491701760 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00b740 tx=0x7f77dc00bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6034bdaccc7c632 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:10.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.811+0000 7f77ea7fc640 1 -- 172.21.15.116:0/3028778602 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc003dd0 con 0x55d4916ff370 2024-09-12T03:19:10.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.811+0000 7f77f8a53640 1 --2- 172.21.15.116:0/787756410 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f77e00a2b80 0x7f77e00a5040 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f77ec005fd0 tx=0x7f77ec00a2f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:10.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.812+0000 7f77ea7fc640 1 -- 172.21.15.116:0/3028778602 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77dc015070 con 0x55d4916ff370 2024-09-12T03:19:10.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.812+0000 7f77ea7fc640 1 -- 172.21.15.116:0/3028778602 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc011440 con 0x55d4916ff370 2024-09-12T03:19:10.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.812+0000 7f77fc88f740 1 -- 172.21.15.116:0/3028778602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d4916ff370 msgr2=0x55d491701760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:10.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.812+0000 7f77fc88f740 1 --2- 172.21.15.116:0/3028778602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d4916ff370 0x55d491701760 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00b740 tx=0x7f77dc00bc00 comp rx=0 tx=0).stop 2024-09-12T03:19:10.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.812+0000 7f77fc88f740 1 -- 172.21.15.116:0/3028778602 shutdown_connections 2024-09-12T03:19:10.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.812+0000 7f77fc88f740 1 --2- 172.21.15.116:0/3028778602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d4916ff370 0x55d491701760 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.812+0000 7f77fc88f740 1 --2- 172.21.15.116:0/3028778602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4916f45d0 0x55d4916f4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.812+0000 7f77fc88f740 1 --2- 172.21.15.116:0/3028778602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4916f5b40 0x55d4916f4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.812+0000 7f77fc88f740 1 -- 172.21.15.116:0/3028778602 >> 172.21.15.116:0/3028778602 conn(0x55d4916bd310 msgr2=0x55d4916f0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:10.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.812+0000 7f77fc88f740 1 -- 172.21.15.116:0/3028778602 shutdown_connections 2024-09-12T03:19:10.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.812+0000 7f77fc88f740 1 -- 172.21.15.116:0/3028778602 wait complete. 2024-09-12T03:19:10.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.813+0000 7f77fc88f740 1 -- start start 2024-09-12T03:19:10.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.813+0000 7f77fc88f740 1 Processor -- start 2024-09-12T03:19:10.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.813+0000 7f77fc88f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4916f45d0 0x55d4916fecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:10.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.813+0000 7f77fc88f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d4916f5b40 0x55d4916fbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:10.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.813+0000 7f77fc88f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4916ff370 0x55d4916fc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:10.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.813+0000 7f77fc88f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d491702070 con 0x55d4916f5b40 2024-09-12T03:19:10.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.813+0000 7f77fc88f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d491615060 con 0x55d4916ff370 2024-09-12T03:19:10.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.813+0000 7f77fc88f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d4917021f0 con 0x55d4916f45d0 2024-09-12T03:19:10.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.813+0000 7f77fc88f740 10 client.0 _ll_register_callbacks cb 0x55d4913688c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:10.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.813+0000 7f77ebfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d4916f5b40 0x55d4916fbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:10.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.813+0000 7f77ebfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d4916f5b40 0x55d4916fbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33066/0 (socket says 172.21.15.116:33066) 2024-09-12T03:19:10.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.813+0000 7f77ebfff640 1 -- 172.21.15.116:0/1810667586 learned_addr learned my addr 172.21.15.116:0/1810667586 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:10.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.813+0000 7f77f3fff640 1 --2- 172.21.15.116:0/1810667586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4916f45d0 0x55d4916fecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:10.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.813+0000 7f77ebfff640 1 -- 172.21.15.116:0/1810667586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4916f45d0 msgr2=0x55d4916fecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:10.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.813+0000 7f77f8a53640 1 --2- 172.21.15.116:0/1810667586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4916ff370 0x55d4916fc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:10.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.813+0000 7f77ebfff640 1 --2- 172.21.15.116:0/1810667586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4916f45d0 0x55d4916fecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.813+0000 7f77ebfff640 1 -- 172.21.15.116:0/1810667586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4916ff370 msgr2=0x55d4916fc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:10.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.813+0000 7f77ebfff640 1 --2- 172.21.15.116:0/1810667586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4916ff370 0x55d4916fc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.813+0000 7f77ebfff640 1 -- 172.21.15.116:0/1810667586 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4913cb3c0 con 0x55d4916f5b40 2024-09-12T03:19:10.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.813+0000 7f77f3fff640 1 --2- 172.21.15.116:0/1810667586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4916f45d0 0x55d4916fecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:10.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.814+0000 7f77f8a53640 1 --2- 172.21.15.116:0/1810667586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4916ff370 0x55d4916fc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:10.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.814+0000 7f77ebfff640 1 --2- 172.21.15.116:0/1810667586 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d4916f5b40 0x55d4916fbf80 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00b710 tx=0x7f77dc003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d443a40bf882f9b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:10.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.814+0000 7f77d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:19:10.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.814+0000 7f77d37fe640 1 -- 172.21.15.116:0/1810667586 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc028060 con 0x55d4916f5b40 2024-09-12T03:19:10.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.814+0000 7f77fc88f740 1 -- 172.21.15.116:0/1810667586 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77dc015070 con 0x55d4916f5b40 2024-09-12T03:19:10.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.814+0000 7f77d37fe640 1 -- 172.21.15.116:0/1810667586 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f77dc011850 con 0x55d4916f5b40 2024-09-12T03:19:10.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.814+0000 7f77d37fe640 1 -- 172.21.15.116:0/1810667586 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f77dc00be50 con 0x55d4916f5b40 2024-09-12T03:19:10.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.814+0000 7f77fc88f740 10 client.19704 fetch_fsmap learned FSMap version 25 2024-09-12T03:19:10.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.814+0000 7f77fc88f740 1 -- 172.21.15.116:0/1810667586 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d49170dab0 con 0x55d4916f5b40 2024-09-12T03:19:10.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.814+0000 7f77d37fe640 1 -- 172.21.15.116:0/1810667586 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77dc015070 con 0x55d4916f5b40 2024-09-12T03:19:10.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.814+0000 7f77d37fe640 1 -- 172.21.15.116:0/1810667586 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f77dc022070 con 0x55d4916f5b40 2024-09-12T03:19:10.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.814+0000 7f77fc88f740 10 client.19704 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:19:10.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.814+0000 7f77fc88f740 10 client.19704 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:19:10.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.815+0000 7f77fc88f740 20 client.19704 populate_metadata read hostname 'smithi116' 2024-09-12T03:19:10.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.815+0000 7f77fc88f740 1 --2- 172.21.15.116:0/1810667586 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d49170eeb0 0x55d4917112a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:10.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.815+0000 7f77fc88f740 4 client.19704 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:19:10.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.815+0000 7f77fc88f740 1 -- 172.21.15.116:0/1810667586 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d491667860 con 0x55d49170eeb0 2024-09-12T03:19:10.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.815+0000 7f77f3fff640 1 --2- 172.21.15.116:0/1810667586 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d49170eeb0 0x55d4917112a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:10.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.815+0000 7f77f3fff640 1 --2- 172.21.15.116:0/1810667586 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d49170eeb0 0x55d4917112a0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e50906ebd32a3fa4 server_cookie=76e29bc0d3368afb in_seq=0 out_seq=0 2024-09-12T03:19:10.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.815+0000 7f77d37fe640 10 client.19704 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:19:10.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.816+0000 7f77d37fe640 1 -- 172.21.15.116:0/1810667586 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d491667860 con 0x55d49170eeb0 2024-09-12T03:19:10.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.816+0000 7f77d37fe640 10 client.19704 handle_command_reply: tid=0 2024-09-12T03:19:10.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.816+0000 7f77fc88f740 1 client.19704 shutdown 2024-09-12T03:19:10.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.816+0000 7f77fc88f740 1 -- 172.21.15.116:0/1810667586 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d4916f5b40 msgr2=0x55d4916fbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:10.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.816+0000 7f77fc88f740 1 --2- 172.21.15.116:0/1810667586 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d4916f5b40 0x55d4916fbf80 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00b710 tx=0x7f77dc003e00 comp rx=0 tx=0).stop 2024-09-12T03:19:10.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.816+0000 7f77fc88f740 1 -- 172.21.15.116:0/1810667586 shutdown_connections 2024-09-12T03:19:10.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.816+0000 7f77fc88f740 1 --2- 172.21.15.116:0/1810667586 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d49170eeb0 0x55d4917112a0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.816+0000 7f77fc88f740 1 --2- 172.21.15.116:0/1810667586 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4916ff370 0x55d4916fc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.816+0000 7f77d37fe640 0 client.19704 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:19:10.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.816+0000 7f77fc88f740 1 --2- 172.21.15.116:0/1810667586 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d4916f5b40 0x55d4916fbf80 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.816+0000 7f77fc88f740 1 --2- 172.21.15.116:0/1810667586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4916f45d0 0x55d4916fecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.816+0000 7f77fc88f740 1 -- 172.21.15.116:0/1810667586 >> 172.21.15.116:0/1810667586 conn(0x55d4916bd310 msgr2=0x55d4916f6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:10.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.816+0000 7f77fc88f740 1 -- 172.21.15.116:0/1810667586 shutdown_connections 2024-09-12T03:19:10.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.816+0000 7f77fc88f740 1 -- 172.21.15.116:0/1810667586 wait complete. 2024-09-12T03:19:10.834 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.816+0000 7f77fc88f740 20 client.19704 trim_cache size 0 max 16384 2024-09-12T03:19:10.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.829+0000 7f77fc88f740 1 Processor -- start 2024-09-12T03:19:10.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.829+0000 7f77fc88f740 1 -- start start 2024-09-12T03:19:10.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.829+0000 7f77fc88f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d4916f45d0 0x55d49170b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:10.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.829+0000 7f77fc88f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4916f5b40 0x55d4916f3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:10.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.829+0000 7f77fc88f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4916ff370 0x55d4917168a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:10.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.829+0000 7f77fc88f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f77f41b1f70 con 0x55d4916f45d0 2024-09-12T03:19:10.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.829+0000 7f77fc88f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f77f41af840 con 0x55d4916ff370 2024-09-12T03:19:10.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.829+0000 7f77fc88f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f77f41ad6b0 con 0x55d4916f5b40 2024-09-12T03:19:10.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.829+0000 7f77f3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d4916f45d0 0x55d49170b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:10.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.829+0000 7f77f3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d4916f45d0 0x55d49170b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33074/0 (socket says 172.21.15.116:33074) 2024-09-12T03:19:10.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.829+0000 7f77f3fff640 1 -- 172.21.15.116:0/2569297525 learned_addr learned my addr 172.21.15.116:0/2569297525 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:10.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.830+0000 7f77f3fff640 1 -- 172.21.15.116:0/2569297525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4916f5b40 msgr2=0x55d4916f3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:10.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.830+0000 7f77f3fff640 1 --2- 172.21.15.116:0/2569297525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4916f5b40 0x55d4916f3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.830+0000 7f77f3fff640 1 -- 172.21.15.116:0/2569297525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4916ff370 msgr2=0x55d4917168a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:10.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.830+0000 7f77f3fff640 1 --2- 172.21.15.116:0/2569297525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4916ff370 0x55d4917168a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.830+0000 7f77f3fff640 1 -- 172.21.15.116:0/2569297525 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4916fa4d0 con 0x55d4916f45d0 2024-09-12T03:19:10.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.830+0000 7f77f3fff640 1 --2- 172.21.15.116:0/2569297525 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d4916f45d0 0x55d49170b340 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f77e40b8770 tx=0x7f77e4086ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c2e3afceda4078 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:10.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.830+0000 7f77e9ffb640 1 -- 172.21.15.116:0/2569297525 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e40bf020 con 0x55d4916f45d0 2024-09-12T03:19:10.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.830+0000 7f77e9ffb640 1 -- 172.21.15.116:0/2569297525 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77e40c4a10 con 0x55d4916f45d0 2024-09-12T03:19:10.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.830+0000 7f77e9ffb640 1 -- 172.21.15.116:0/2569297525 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e4049040 con 0x55d4916f45d0 2024-09-12T03:19:10.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.831+0000 7f77fc88f740 1 -- 172.21.15.116:0/2569297525 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d4916f45d0 msgr2=0x55d49170b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:10.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.831+0000 7f77fc88f740 1 --2- 172.21.15.116:0/2569297525 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d4916f45d0 0x55d49170b340 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f77e40b8770 tx=0x7f77e4086ec0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.831+0000 7f77fc88f740 1 -- 172.21.15.116:0/2569297525 shutdown_connections 2024-09-12T03:19:10.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.831+0000 7f77fc88f740 1 --2- 172.21.15.116:0/2569297525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4916ff370 0x55d4917168a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.831+0000 7f77fc88f740 1 --2- 172.21.15.116:0/2569297525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4916f5b40 0x55d4916f3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.831+0000 7f77fc88f740 1 --2- 172.21.15.116:0/2569297525 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d4916f45d0 0x55d49170b340 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.831+0000 7f77fc88f740 1 -- 172.21.15.116:0/2569297525 >> 172.21.15.116:0/2569297525 conn(0x55d4916bd310 msgr2=0x55d4916fb070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:10.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.831+0000 7f77fc88f740 1 -- 172.21.15.116:0/2569297525 shutdown_connections 2024-09-12T03:19:10.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.831+0000 7f77fc88f740 1 -- 172.21.15.116:0/2569297525 wait complete. 2024-09-12T03:19:10.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.831+0000 7f77fc88f740 1 -- start start 2024-09-12T03:19:10.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.831+0000 7f77fc88f740 1 Processor -- start 2024-09-12T03:19:10.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.832+0000 7f77fc88f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d4916f45d0 0x55d491703850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:10.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.832+0000 7f77fc88f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4916f5b40 0x55d491703d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:10.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.832+0000 7f77fc88f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4916ff370 0x55d491778870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:10.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.832+0000 7f77fc88f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f77f41add00 con 0x55d4916f45d0 2024-09-12T03:19:10.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.832+0000 7f77fc88f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f77f41af170 con 0x55d4916ff370 2024-09-12T03:19:10.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.832+0000 7f77fc88f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f77f41ae3d0 con 0x55d4916f5b40 2024-09-12T03:19:10.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.832+0000 7f77fc88f740 10 client.0 _ll_register_callbacks cb 0x7f77dc004530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:10.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.832+0000 7f77f3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d4916f45d0 0x55d491703850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:10.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.832+0000 7f77f3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d4916f45d0 0x55d491703850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33080/0 (socket says 172.21.15.116:33080) 2024-09-12T03:19:10.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.832+0000 7f77f3fff640 1 -- 172.21.15.116:0/1620927363 learned_addr learned my addr 172.21.15.116:0/1620927363 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:10.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.832+0000 7f77f8a53640 1 --2- 172.21.15.116:0/1620927363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4916ff370 0x55d491778870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:10.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.832+0000 7f77ebfff640 1 --2- 172.21.15.116:0/1620927363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4916f5b40 0x55d491703d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:10.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.832+0000 7f77f3fff640 1 -- 172.21.15.116:0/1620927363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4916f5b40 msgr2=0x55d491703d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:10.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.832+0000 7f77f3fff640 1 --2- 172.21.15.116:0/1620927363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4916f5b40 0x55d491703d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.832+0000 7f77f3fff640 1 -- 172.21.15.116:0/1620927363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4916ff370 msgr2=0x55d491778870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:10.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.832+0000 7f77f3fff640 1 --2- 172.21.15.116:0/1620927363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4916ff370 0x55d491778870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.832+0000 7f77f3fff640 1 -- 172.21.15.116:0/1620927363 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d491778db0 con 0x55d4916f45d0 2024-09-12T03:19:10.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.832+0000 7f77ebfff640 1 --2- 172.21.15.116:0/1620927363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4916f5b40 0x55d491703d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:10.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.832+0000 7f77f8a53640 1 --2- 172.21.15.116:0/1620927363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4916ff370 0x55d491778870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:10.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.832+0000 7f77f3fff640 1 --2- 172.21.15.116:0/1620927363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d4916f45d0 0x55d491703850 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f77e4000c00 tx=0x7f77e402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df52be46654c8971 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:10.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.832+0000 7f77d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-09-12T03:19:10.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.832+0000 7f77d37fe640 1 -- 172.21.15.116:0/1620927363 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e40bf040 con 0x55d4916f45d0 2024-09-12T03:19:10.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.832+0000 7f77d37fe640 1 -- 172.21.15.116:0/1620927363 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f77e4003c80 con 0x55d4916f45d0 2024-09-12T03:19:10.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.832+0000 7f77fc88f740 1 -- 172.21.15.116:0/1620927363 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77e40c4a10 con 0x55d4916f45d0 2024-09-12T03:19:10.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.832+0000 7f77d37fe640 1 -- 172.21.15.116:0/1620927363 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f77e4049040 con 0x55d4916f45d0 2024-09-12T03:19:10.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.833+0000 7f77fc88f740 10 client.19710 fetch_fsmap learned FSMap version 25 2024-09-12T03:19:10.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.833+0000 7f77fc88f740 1 -- 172.21.15.116:0/1620927363 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d491779fb0 con 0x55d4916f45d0 2024-09-12T03:19:10.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.833+0000 7f77d37fe640 1 -- 172.21.15.116:0/1620927363 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77e40c4a10 con 0x55d4916f45d0 2024-09-12T03:19:10.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.833+0000 7f77d37fe640 1 -- 172.21.15.116:0/1620927363 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f77e4031bd0 con 0x55d4916f45d0 2024-09-12T03:19:10.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.833+0000 7f77fc88f740 10 client.19710 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:19:10.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.833+0000 7f77fc88f740 10 client.19710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-12T03:19:10.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.833+0000 7f77fc88f740 20 client.19710 populate_metadata read hostname 'smithi116' 2024-09-12T03:19:10.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.833+0000 7f77fc88f740 1 --2- 172.21.15.116:0/1620927363 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d49177a610 0x55d49177ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:10.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.833+0000 7f77fc88f740 4 client.19710 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:19:10.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.833+0000 7f77fc88f740 1 -- 172.21.15.116:0/1620927363 --> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d491661d70 con 0x55d49177a610 2024-09-12T03:19:10.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.833+0000 7f77ebfff640 1 --2- 172.21.15.116:0/1620927363 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d49177a610 0x55d49177ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:10.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.833+0000 7f77d37fe640 10 client.19710 ms_handle_connect on v2:172.21.15.177:6838/2069130361 2024-09-12T03:19:10.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.833+0000 7f77ebfff640 1 --2- 172.21.15.116:0/1620927363 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d49177a610 0x55d49177ca00 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42653f8c00ca4d2c server_cookie=1a2de9b70fde9067 in_seq=0 out_seq=0 2024-09-12T03:19:10.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.834+0000 7f77d37fe640 1 -- 172.21.15.116:0/1620927363 <== mds.0 v2:172.21.15.177:6838/2069130361 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d491661d70 con 0x55d49177a610 2024-09-12T03:19:10.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.834+0000 7f77d37fe640 10 client.19710 handle_command_reply: tid=0 2024-09-12T03:19:10.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.834+0000 7f77fc88f740 1 client.19710 shutdown 2024-09-12T03:19:10.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.834+0000 7f77fc88f740 1 -- 172.21.15.116:0/1620927363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d4916f45d0 msgr2=0x55d491703850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:10.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.834+0000 7f77fc88f740 1 --2- 172.21.15.116:0/1620927363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d4916f45d0 0x55d491703850 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f77e4000c00 tx=0x7f77e402a420 comp rx=0 tx=0).stop 2024-09-12T03:19:10.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.834+0000 7f77fc88f740 1 -- 172.21.15.116:0/1620927363 shutdown_connections 2024-09-12T03:19:10.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.834+0000 7f77fc88f740 1 --2- 172.21.15.116:0/1620927363 >> [v2:172.21.15.177:6838/2069130361,v1:172.21.15.177:6839/2069130361] conn(0x55d49177a610 0x55d49177ca00 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.834+0000 7f77fc88f740 1 --2- 172.21.15.116:0/1620927363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d4916ff370 0x55d491778870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.834+0000 7f77fc88f740 1 --2- 172.21.15.116:0/1620927363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4916f5b40 0x55d491703d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.834+0000 7f77d37fe640 0 client.19710 ms_handle_reset on v2:172.21.15.177:6838/2069130361 2024-09-12T03:19:10.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.834+0000 7f77fc88f740 1 --2- 172.21.15.116:0/1620927363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d4916f45d0 0x55d491703850 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.834+0000 7f77fc88f740 1 -- 172.21.15.116:0/1620927363 >> 172.21.15.116:0/1620927363 conn(0x55d4916bd310 msgr2=0x55d4916fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:10.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.834+0000 7f77fc88f740 1 -- 172.21.15.116:0/1620927363 shutdown_connections 2024-09-12T03:19:10.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.834+0000 7f77fc88f740 1 -- 172.21.15.116:0/1620927363 wait complete. 2024-09-12T03:19:10.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.834+0000 7f77fc88f740 20 client.19710 trim_cache size 0 max 16384 2024-09-12T03:19:10.846 INFO:teuthology.orchestra.run.smithi116.stdout:[] 2024-09-12T03:19:10.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.835+0000 7f77fa4b6640 1 -- 172.21.15.116:0/787756410 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f77e00a2b80 msgr2=0x7f77e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:10.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.835+0000 7f77fa4b6640 1 --2- 172.21.15.116:0/787756410 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f77e00a2b80 0x7f77e00a5040 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f77ec005fd0 tx=0x7f77ec00a2f0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.835+0000 7f77fa4b6640 1 -- 172.21.15.116:0/787756410 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77f4100a80 msgr2=0x7f77f410e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:10.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.835+0000 7f77fa4b6640 1 --2- 172.21.15.116:0/787756410 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77f4100a80 0x7f77f410e3e0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f77e4009230 tx=0x7f77e4004270 comp rx=0 tx=0).stop 2024-09-12T03:19:10.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.835+0000 7f77fa4b6640 1 -- 172.21.15.116:0/787756410 shutdown_connections 2024-09-12T03:19:10.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.835+0000 7f77fa4b6640 1 --2- 172.21.15.116:0/787756410 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f77e00a2b80 0x7f77e00a5040 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.835+0000 7f77fa4b6640 1 --2- 172.21.15.116:0/787756410 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77f4106a80 0x7f77f410ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.835+0000 7f77fa4b6640 1 --2- 172.21.15.116:0/787756410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77f4101420 0x7f77f410e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.835+0000 7f77fa4b6640 1 --2- 172.21.15.116:0/787756410 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77f4100a80 0x7f77f410e3e0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.835+0000 7f77fa4b6640 1 -- 172.21.15.116:0/787756410 >> 172.21.15.116:0/787756410 conn(0x7f77f40fc7a0 msgr2=0x7f77f40fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:10.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.835+0000 7f77fa4b6640 1 -- 172.21.15.116:0/787756410 shutdown_connections 2024-09-12T03:19:10.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:10.835+0000 7f77fa4b6640 1 -- 172.21.15.116:0/787756410 wait complete. 2024-09-12T03:19:10.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:10 smithi183 ceph-mon[35535]: pgmap v823: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-09-12T03:19:10.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:10 smithi177 ceph-mon[35305]: pgmap v823: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.3 KiB/s wr, 9 op/s 2024-09-12T03:19:11.661 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-12T03:19:11.662 INFO:tasks.fwd_scrub:done joining 2024-09-12T03:19:11.662 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-12T03:19:11.675 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:19:11.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.782+0000 7f69294ba640 1 -- 172.21.15.116:0/2281773800 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6924100a30 msgr2=0x7f6924100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:11.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.782+0000 7f69294ba640 1 --2- 172.21.15.116:0/2281773800 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6924100a30 0x7f6924100e30 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f6914009900 tx=0x7f691402a410 comp rx=0 tx=0).stop 2024-09-12T03:19:11.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.782+0000 7f69294ba640 1 -- 172.21.15.116:0/2281773800 shutdown_connections 2024-09-12T03:19:11.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.782+0000 7f69294ba640 1 --2- 172.21.15.116:0/2281773800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69241025e0 0x7f6924108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:11.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.782+0000 7f69294ba640 1 --2- 172.21.15.116:0/2281773800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6924101c20 0x7f69241020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:11.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.782+0000 7f69294ba640 1 --2- 172.21.15.116:0/2281773800 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6924100a30 0x7f6924100e30 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:11.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.782+0000 7f69294ba640 1 -- 172.21.15.116:0/2281773800 >> 172.21.15.116:0/2281773800 conn(0x7f69240fc1b0 msgr2=0x7f69240fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:11.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.782+0000 7f69294ba640 1 -- 172.21.15.116:0/2281773800 shutdown_connections 2024-09-12T03:19:11.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.782+0000 7f69294ba640 1 -- 172.21.15.116:0/2281773800 wait complete. 2024-09-12T03:19:11.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.783+0000 7f69294ba640 1 Processor -- start 2024-09-12T03:19:11.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.783+0000 7f69294ba640 1 -- start start 2024-09-12T03:19:11.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.783+0000 7f69294ba640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6924100a30 0x7f6924198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:11.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.783+0000 7f69294ba640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6924101c20 0x7f6924198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:11.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.783+0000 7f69294ba640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69241025e0 0x7f692419f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:11.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.783+0000 7f69294ba640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f692410d8a0 con 0x7f6924101c20 2024-09-12T03:19:11.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.783+0000 7f69294ba640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f692410d720 con 0x7f6924100a30 2024-09-12T03:19:11.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.783+0000 7f69294ba640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f692410da20 con 0x7f69241025e0 2024-09-12T03:19:11.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.783+0000 7f69227fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6924101c20 0x7f6924198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:11.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.783+0000 7f69227fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6924101c20 0x7f6924198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42294/0 (socket says 172.21.15.116:42294) 2024-09-12T03:19:11.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.783+0000 7f69227fc640 1 -- 172.21.15.116:0/3895480937 learned_addr learned my addr 172.21.15.116:0/3895480937 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:11.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.783+0000 7f69237fe640 1 --2- 172.21.15.116:0/3895480937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69241025e0 0x7f692419f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:11.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.783+0000 7f6922ffd640 1 --2- 172.21.15.116:0/3895480937 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6924100a30 0x7f6924198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:11.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.783+0000 7f69227fc640 1 -- 172.21.15.116:0/3895480937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69241025e0 msgr2=0x7f692419f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:11.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.783+0000 7f69227fc640 1 --2- 172.21.15.116:0/3895480937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69241025e0 0x7f692419f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:11.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.783+0000 7f69227fc640 1 -- 172.21.15.116:0/3895480937 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6924100a30 msgr2=0x7f6924198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:11.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.783+0000 7f69227fc640 1 --2- 172.21.15.116:0/3895480937 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6924100a30 0x7f6924198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:11.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.783+0000 7f69227fc640 1 -- 172.21.15.116:0/3895480937 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f692419fc30 con 0x7f6924101c20 2024-09-12T03:19:11.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.783+0000 7f6922ffd640 1 --2- 172.21.15.116:0/3895480937 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6924100a30 0x7f6924198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:11.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.784+0000 7f69227fc640 1 --2- 172.21.15.116:0/3895480937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6924101c20 0x7f6924198550 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f691800b4a0 tx=0x7f691800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:11.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.784+0000 7f690bfff640 1 -- 172.21.15.116:0/3895480937 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6918014070 con 0x7f6924101c20 2024-09-12T03:19:11.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.784+0000 7f690bfff640 1 -- 172.21.15.116:0/3895480937 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6918010af0 con 0x7f6924101c20 2024-09-12T03:19:11.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.784+0000 7f69294ba640 1 -- 172.21.15.116:0/3895480937 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f692419ff20 con 0x7f6924101c20 2024-09-12T03:19:11.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.784+0000 7f690bfff640 1 -- 172.21.15.116:0/3895480937 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6918017d10 con 0x7f6924101c20 2024-09-12T03:19:11.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.784+0000 7f69294ba640 1 -- 172.21.15.116:0/3895480937 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69241a0460 con 0x7f6924101c20 2024-09-12T03:19:11.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.785+0000 7f69294ba640 1 -- 172.21.15.116:0/3895480937 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6924107f10 con 0x7f6924101c20 2024-09-12T03:19:11.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.787+0000 7f690bfff640 1 -- 172.21.15.116:0/3895480937 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6918017070 con 0x7f6924101c20 2024-09-12T03:19:11.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.787+0000 7f690bfff640 1 --2- 172.21.15.116:0/3895480937 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f690c0a2b90 0x7f690c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:11.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.787+0000 7f6922ffd640 1 --2- 172.21.15.116:0/3895480937 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f690c0a2b90 0x7f690c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:11.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.787+0000 7f690bfff640 1 -- 172.21.15.116:0/3895480937 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f69180ce7d0 con 0x7f6924101c20 2024-09-12T03:19:11.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.788+0000 7f6922ffd640 1 --2- 172.21.15.116:0/3895480937 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f690c0a2b90 0x7f690c0a5050 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6914000c00 tx=0x7f6914031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:11.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.791+0000 7f690bfff640 1 -- 172.21.15.116:0/3895480937 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f691809b760 con 0x7f6924101c20 2024-09-12T03:19:11.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.959+0000 7f69294ba640 1 -- 172.21.15.116:0/3895480937 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f692402d9c0 con 0x7f6924101c20 2024-09-12T03:19:11.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.961+0000 7f690bfff640 1 -- 172.21.15.116:0/3895480937 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22710 (secure 0 0 0) 0x7f691809b060 con 0x7f6924101c20 2024-09-12T03:19:11.961 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:19:11.961 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":80,"fsid":"eb1ac602-70b1-11ef-bcea-c7b262605968","created":"2024-09-12T02:53:07.105686+0000","modified":"2024-09-12T03:02:04.272898+0000","last_up_change":"2024-09-12T03:00:07.202639+0000","last_in_change":"2024-09-12T02:59:43.446336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:56:17.347152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:00:55.191040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:00:57.171142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8cf5a18-dc6f-4bb6-bd74-78adda310118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6803","nonce":862366853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6805","nonce":862366853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6809","nonce":862366853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6807","nonce":862366853}]},"public_addr":"172.21.15.116:6803/862366853","cluster_addr":"172.21.15.116:6805/862366853","heartbeat_back_addr":"172.21.15.116:6809/862366853","heartbeat_front_addr":"172.21.15.116:6807/862366853","state":["exists","up"]},{"osd":1,"uuid":"0affd3cb-9a51-42b0-a2cc-f30784a75cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6801","nonce":2245800547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6803","nonce":2245800547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6807","nonce":2245800547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6805","nonce":2245800547}]},"public_addr":"172.21.15.177:6801/2245800547","cluster_addr":"172.21.15.177:6803/2245800547","heartbeat_back_addr":"172.21.15.177:6807/2245800547","heartbeat_front_addr":"172.21.15.177:6805/2245800547","state":["exists","up"]},{"osd":2,"uuid":"1ce45ea5-84df-4a35-924f-78f92bd4f2cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6801","nonce":2645677759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6803","nonce":2645677759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6807","nonce":2645677759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6805","nonce":2645677759}]},"public_addr":"172.21.15.183:6801/2645677759","cluster_addr":"172.21.15.183:6803/2645677759","heartbeat_back_addr":"172.21.15.183:6807/2645677759","heartbeat_front_addr":"172.21.15.183:6805/2645677759","state":["exists","up"]},{"osd":3,"uuid":"ae5e8178-c998-4648-bc05-4bce822f0cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6811","nonce":686622148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6813","nonce":686622148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6817","nonce":686622148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6815","nonce":686622148}]},"public_addr":"172.21.15.116:6811/686622148","cluster_addr":"172.21.15.116:6813/686622148","heartbeat_back_addr":"172.21.15.116:6817/686622148","heartbeat_front_addr":"172.21.15.116:6815/686622148","state":["exists","up"]},{"osd":4,"uuid":"c81d2e73-0bc2-440d-b5a2-b7428af3ea4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6809","nonce":2382938021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6811","nonce":2382938021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6815","nonce":2382938021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6813","nonce":2382938021}]},"public_addr":"172.21.15.177:6809/2382938021","cluster_addr":"172.21.15.177:6811/2382938021","heartbeat_back_addr":"172.21.15.177:6815/2382938021","heartbeat_front_addr":"172.21.15.177:6813/2382938021","state":["exists","up"]},{"osd":5,"uuid":"c3f168be-9e27-4cf5-b73e-3f47d8990ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6809","nonce":85363432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6811","nonce":85363432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6815","nonce":85363432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6813","nonce":85363432}]},"public_addr":"172.21.15.183:6809/85363432","cluster_addr":"172.21.15.183:6811/85363432","heartbeat_back_addr":"172.21.15.183:6815/85363432","heartbeat_front_addr":"172.21.15.183:6813/85363432","state":["exists","up"]},{"osd":6,"uuid":"beb0a7e9-4909-47e2-9ee7-6951d4745e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6819","nonce":1121291186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6821","nonce":1121291186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6825","nonce":1121291186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6823","nonce":1121291186}]},"public_addr":"172.21.15.116:6819/1121291186","cluster_addr":"172.21.15.116:6821/1121291186","heartbeat_back_addr":"172.21.15.116:6825/1121291186","heartbeat_front_addr":"172.21.15.116:6823/1121291186","state":["exists","up"]},{"osd":7,"uuid":"761f4680-f049-473c-bf44-b19acf8de24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6817","nonce":2177665961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6819","nonce":2177665961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6823","nonce":2177665961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6821","nonce":2177665961}]},"public_addr":"172.21.15.177:6817/2177665961","cluster_addr":"172.21.15.177:6819/2177665961","heartbeat_back_addr":"172.21.15.177:6823/2177665961","heartbeat_front_addr":"172.21.15.177:6821/2177665961","state":["exists","up"]},{"osd":8,"uuid":"2385ea6c-fd5a-4815-9596-77f948629542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6817","nonce":1305725258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6819","nonce":1305725258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6823","nonce":1305725258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6821","nonce":1305725258}]},"public_addr":"172.21.15.183:6817/1305725258","cluster_addr":"172.21.15.183:6819/1305725258","heartbeat_back_addr":"172.21.15.183:6823/1305725258","heartbeat_front_addr":"172.21.15.183:6821/1305725258","state":["exists","up"]},{"osd":9,"uuid":"fd0c0dc4-4990-4f12-aa14-f855504cd995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6827","nonce":3831673668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6829","nonce":3831673668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6833","nonce":3831673668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6831","nonce":3831673668}]},"public_addr":"172.21.15.116:6827/3831673668","cluster_addr":"172.21.15.116:6829/3831673668","heartbeat_back_addr":"172.21.15.116:6833/3831673668","heartbeat_front_addr":"172.21.15.116:6831/3831673668","state":["exists","up"]},{"osd":10,"uuid":"864d1a97-4810-49c8-ad4a-a0c10253833c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6825","nonce":963663634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6827","nonce":963663634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6831","nonce":963663634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6829","nonce":963663634}]},"public_addr":"172.21.15.177:6825/963663634","cluster_addr":"172.21.15.177:6827/963663634","heartbeat_back_addr":"172.21.15.177:6831/963663634","heartbeat_front_addr":"172.21.15.177:6829/963663634","state":["exists","up"]},{"osd":11,"uuid":"d49c0841-4620-4f2f-a6b5-f2a9aa2e3c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6825","nonce":1818319939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6827","nonce":1818319939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6831","nonce":1818319939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6829","nonce":1818319939}]},"public_addr":"172.21.15.183:6825/1818319939","cluster_addr":"172.21.15.183:6827/1818319939","heartbeat_back_addr":"172.21.15.183:6831/1818319939","heartbeat_front_addr":"172.21.15.183:6829/1818319939","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:24.736944+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:50.310980+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:56:14.989533+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:56:41.187047+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:06.981664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:32.872771+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:58.850089+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:23.043121+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:47.839958+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:14.087708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:40.010548+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:05.364820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.116:6801/2966274423":"2024-09-13T02:54:03.026862+0000","172.21.15.116:6800/2966274423":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/686139455":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/1552047390":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/2180363411":"2024-09-13T02:53:39.433769+0000","172.21.15.116:6801/1791219601":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/3907400460":"2024-09-13T02:54:03.026862+0000","172.21.15.116:6800/1791219601":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/1802566286":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/212337802":"2024-09-13T02:53:39.433769+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:19:11.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.963+0000 7f69294ba640 1 -- 172.21.15.116:0/3895480937 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f690c0a2b90 msgr2=0x7f690c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:11.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.963+0000 7f69294ba640 1 --2- 172.21.15.116:0/3895480937 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f690c0a2b90 0x7f690c0a5050 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6914000c00 tx=0x7f6914031400 comp rx=0 tx=0).stop 2024-09-12T03:19:11.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.963+0000 7f69294ba640 1 -- 172.21.15.116:0/3895480937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6924101c20 msgr2=0x7f6924198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:11.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.963+0000 7f69294ba640 1 --2- 172.21.15.116:0/3895480937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6924101c20 0x7f6924198550 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f691800b4a0 tx=0x7f691800b960 comp rx=0 tx=0).stop 2024-09-12T03:19:11.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.963+0000 7f69294ba640 1 -- 172.21.15.116:0/3895480937 shutdown_connections 2024-09-12T03:19:11.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.963+0000 7f69294ba640 1 --2- 172.21.15.116:0/3895480937 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f690c0a2b90 0x7f690c0a5050 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:11.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.963+0000 7f69294ba640 1 --2- 172.21.15.116:0/3895480937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69241025e0 0x7f692419f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:11.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.963+0000 7f69294ba640 1 --2- 172.21.15.116:0/3895480937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6924101c20 0x7f6924198550 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:11.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.963+0000 7f69294ba640 1 --2- 172.21.15.116:0/3895480937 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6924100a30 0x7f6924198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:11.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.963+0000 7f69294ba640 1 -- 172.21.15.116:0/3895480937 >> 172.21.15.116:0/3895480937 conn(0x7f69240fc1b0 msgr2=0x7f69241050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:11.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.963+0000 7f69294ba640 1 -- 172.21.15.116:0/3895480937 shutdown_connections 2024-09-12T03:19:11.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:11.963+0000 7f69294ba640 1 -- 172.21.15.116:0/3895480937 wait complete. 2024-09-12T03:19:11.976 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T02:56:17.347152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-12T03:00:55.191040+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-12T03:00:57.171142+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T03:19:11.977 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T03:19:12.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.080+0000 7fcf42c77640 1 -- 172.21.15.116:0/808261573 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf3c101b10 msgr2=0x7fcf3c101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:12.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.080+0000 7fcf42c77640 1 --2- 172.21.15.116:0/808261573 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf3c101b10 0x7fcf3c101f10 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fcf30009900 tx=0x7fcf3002a410 comp rx=0 tx=0).stop 2024-09-12T03:19:12.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.080+0000 7fcf42c77640 1 -- 172.21.15.116:0/808261573 shutdown_connections 2024-09-12T03:19:12.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.080+0000 7fcf42c77640 1 --2- 172.21.15.116:0/808261573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf3c1036c0 0x7fcf3c109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.080+0000 7fcf42c77640 1 --2- 172.21.15.116:0/808261573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf3c102d00 0x7fcf3c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.080+0000 7fcf42c77640 1 --2- 172.21.15.116:0/808261573 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf3c101b10 0x7fcf3c101f10 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.080+0000 7fcf42c77640 1 -- 172.21.15.116:0/808261573 >> 172.21.15.116:0/808261573 conn(0x7fcf3c0fd2b0 msgr2=0x7fcf3c0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:12.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.080+0000 7fcf42c77640 1 -- 172.21.15.116:0/808261573 shutdown_connections 2024-09-12T03:19:12.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.080+0000 7fcf42c77640 1 -- 172.21.15.116:0/808261573 wait complete. 2024-09-12T03:19:12.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.080+0000 7fcf42c77640 1 Processor -- start 2024-09-12T03:19:12.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.081+0000 7fcf42c77640 1 -- start start 2024-09-12T03:19:12.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.081+0000 7fcf42c77640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf3c101b10 0x7fcf3c197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:12.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.081+0000 7fcf42c77640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf3c102d00 0x7fcf3c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:12.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.081+0000 7fcf42c77640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf3c1036c0 0x7fcf3c19f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:12.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.081+0000 7fcf42c77640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c10d790 con 0x7fcf3c1036c0 2024-09-12T03:19:12.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.081+0000 7fcf42c77640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c10d610 con 0x7fcf3c102d00 2024-09-12T03:19:12.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.081+0000 7fcf42c77640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c10d910 con 0x7fcf3c101b10 2024-09-12T03:19:12.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.081+0000 7fcf41214640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf3c1036c0 0x7fcf3c19f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:12.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.081+0000 7fcf41214640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf3c1036c0 0x7fcf3c19f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42314/0 (socket says 172.21.15.116:42314) 2024-09-12T03:19:12.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.081+0000 7fcf41214640 1 -- 172.21.15.116:0/1564390620 learned_addr learned my addr 172.21.15.116:0/1564390620 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:12.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.081+0000 7fcf40a13640 1 --2- 172.21.15.116:0/1564390620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf3c101b10 0x7fcf3c197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:12.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.081+0000 7fcf3bfff640 1 --2- 172.21.15.116:0/1564390620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf3c102d00 0x7fcf3c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:12.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.081+0000 7fcf41214640 1 -- 172.21.15.116:0/1564390620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf3c101b10 msgr2=0x7fcf3c197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:12.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.081+0000 7fcf41214640 1 --2- 172.21.15.116:0/1564390620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf3c101b10 0x7fcf3c197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.081+0000 7fcf41214640 1 -- 172.21.15.116:0/1564390620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf3c102d00 msgr2=0x7fcf3c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:12.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.081+0000 7fcf41214640 1 --2- 172.21.15.116:0/1564390620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf3c102d00 0x7fcf3c1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.081+0000 7fcf41214640 1 -- 172.21.15.116:0/1564390620 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf3c19fac0 con 0x7fcf3c1036c0 2024-09-12T03:19:12.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.081+0000 7fcf3bfff640 1 --2- 172.21.15.116:0/1564390620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf3c102d00 0x7fcf3c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:12.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.081+0000 7fcf41214640 1 --2- 172.21.15.116:0/1564390620 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf3c1036c0 0x7fcf3c19f450 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fcf3400eb10 tx=0x7fcf3400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:12.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.081+0000 7fcf39ffb640 1 -- 172.21.15.116:0/1564390620 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34004b20 con 0x7fcf3c1036c0 2024-09-12T03:19:12.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.081+0000 7fcf39ffb640 1 -- 172.21.15.116:0/1564390620 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf34004cc0 con 0x7fcf3c1036c0 2024-09-12T03:19:12.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.081+0000 7fcf42c77640 1 -- 172.21.15.116:0/1564390620 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf3c19fdb0 con 0x7fcf3c1036c0 2024-09-12T03:19:12.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.081+0000 7fcf39ffb640 1 -- 172.21.15.116:0/1564390620 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34010760 con 0x7fcf3c1036c0 2024-09-12T03:19:12.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.082+0000 7fcf42c77640 1 -- 172.21.15.116:0/1564390620 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf3c1a0270 con 0x7fcf3c1036c0 2024-09-12T03:19:12.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.083+0000 7fcf42c77640 1 -- 172.21.15.116:0/1564390620 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf3c108fa0 con 0x7fcf3c1036c0 2024-09-12T03:19:12.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.084+0000 7fcf39ffb640 1 -- 172.21.15.116:0/1564390620 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fcf34005020 con 0x7fcf3c1036c0 2024-09-12T03:19:12.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.084+0000 7fcf39ffb640 1 --2- 172.21.15.116:0/1564390620 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcf2c0a2b90 0x7fcf2c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:12.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.085+0000 7fcf40a13640 1 --2- 172.21.15.116:0/1564390620 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcf2c0a2b90 0x7fcf2c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:12.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.085+0000 7fcf39ffb640 1 -- 172.21.15.116:0/1564390620 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fcf340cf800 con 0x7fcf3c1036c0 2024-09-12T03:19:12.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.085+0000 7fcf40a13640 1 --2- 172.21.15.116:0/1564390620 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcf2c0a2b90 0x7fcf2c0a5050 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcf30031610 tx=0x7fcf30000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:12.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.089+0000 7fcf39ffb640 1 -- 172.21.15.116:0/1564390620 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcf3409c790 con 0x7fcf3c1036c0 2024-09-12T03:19:12.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.255+0000 7fcf42c77640 1 -- 172.21.15.116:0/1564390620 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcf3c062e90 con 0x7fcf3c1036c0 2024-09-12T03:19:12.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.255+0000 7fcf39ffb640 1 -- 172.21.15.116:0/1564390620 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) ==== 93+0+10 (secure 0 0 0) 0x7fcf3409c090 con 0x7fcf3c1036c0 2024-09-12T03:19:12.256 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2024-09-12T03:19:12.257 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.257+0000 7fcf42c77640 1 -- 172.21.15.116:0/1564390620 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcf2c0a2b90 msgr2=0x7fcf2c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:12.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.257+0000 7fcf42c77640 1 --2- 172.21.15.116:0/1564390620 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcf2c0a2b90 0x7fcf2c0a5050 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcf30031610 tx=0x7fcf30000bc0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.257+0000 7fcf42c77640 1 -- 172.21.15.116:0/1564390620 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf3c1036c0 msgr2=0x7fcf3c19f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:12.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.257+0000 7fcf42c77640 1 --2- 172.21.15.116:0/1564390620 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf3c1036c0 0x7fcf3c19f450 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fcf3400eb10 tx=0x7fcf3400c2d0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.257+0000 7fcf42c77640 1 -- 172.21.15.116:0/1564390620 shutdown_connections 2024-09-12T03:19:12.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.257+0000 7fcf42c77640 1 --2- 172.21.15.116:0/1564390620 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fcf2c0a2b90 0x7fcf2c0a5050 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.257+0000 7fcf42c77640 1 --2- 172.21.15.116:0/1564390620 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf3c1036c0 0x7fcf3c19f450 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.257+0000 7fcf42c77640 1 --2- 172.21.15.116:0/1564390620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf3c102d00 0x7fcf3c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.257+0000 7fcf42c77640 1 --2- 172.21.15.116:0/1564390620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf3c101b10 0x7fcf3c197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.257+0000 7fcf42c77640 1 -- 172.21.15.116:0/1564390620 >> 172.21.15.116:0/1564390620 conn(0x7fcf3c0fd2b0 msgr2=0x7fcf3c106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:12.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.257+0000 7fcf42c77640 1 -- 172.21.15.116:0/1564390620 shutdown_connections 2024-09-12T03:19:12.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.257+0000 7fcf42c77640 1 -- 172.21.15.116:0/1564390620 wait complete. 2024-09-12T03:19:12.269 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-12T03:19:12.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.374+0000 7f0d57366640 1 -- 172.21.15.116:0/223796273 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d50102cc0 msgr2=0x7f0d50105130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:12.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.374+0000 7f0d57366640 1 --2- 172.21.15.116:0/223796273 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d50102cc0 0x7f0d50105130 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009900 tx=0x7f0d4002a410 comp rx=0 tx=0).stop 2024-09-12T03:19:12.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.374+0000 7f0d57366640 1 -- 172.21.15.116:0/223796273 shutdown_connections 2024-09-12T03:19:12.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.374+0000 7f0d57366640 1 --2- 172.21.15.116:0/223796273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d50105670 0x7f0d50105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.374+0000 7f0d57366640 1 --2- 172.21.15.116:0/223796273 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d50102cc0 0x7f0d50105130 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.374+0000 7f0d57366640 1 --2- 172.21.15.116:0/223796273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d50100370 0x7f0d50102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.374+0000 7f0d57366640 1 -- 172.21.15.116:0/223796273 >> 172.21.15.116:0/223796273 conn(0x7f0d500fc140 msgr2=0x7f0d500fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:12.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.374+0000 7f0d57366640 1 -- 172.21.15.116:0/223796273 shutdown_connections 2024-09-12T03:19:12.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.374+0000 7f0d57366640 1 -- 172.21.15.116:0/223796273 wait complete. 2024-09-12T03:19:12.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.375+0000 7f0d57366640 1 Processor -- start 2024-09-12T03:19:12.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.375+0000 7f0d57366640 1 -- start start 2024-09-12T03:19:12.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.375+0000 7f0d57366640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d50100370 0x7f0d5019e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:12.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.375+0000 7f0d57366640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d50102cc0 0x7f0d5019e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:12.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.376+0000 7f0d57366640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d50105670 0x7f0d501a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:12.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.376+0000 7f0d57366640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0d50113db0 con 0x7f0d50100370 2024-09-12T03:19:12.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.376+0000 7f0d57366640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0d50113c30 con 0x7f0d50105670 2024-09-12T03:19:12.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.376+0000 7f0d57366640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0d50113f30 con 0x7f0d50102cc0 2024-09-12T03:19:12.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.376+0000 7f0d55102640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d50100370 0x7f0d5019e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:12.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.376+0000 7f0d55102640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d50100370 0x7f0d5019e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42334/0 (socket says 172.21.15.116:42334) 2024-09-12T03:19:12.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.376+0000 7f0d55102640 1 -- 172.21.15.116:0/928959942 learned_addr learned my addr 172.21.15.116:0/928959942 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:12.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.376+0000 7f0d55903640 1 --2- 172.21.15.116:0/928959942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d50105670 0x7f0d501a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:12.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.376+0000 7f0d54901640 1 --2- 172.21.15.116:0/928959942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d50102cc0 0x7f0d5019e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:12.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.376+0000 7f0d55102640 1 -- 172.21.15.116:0/928959942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d50102cc0 msgr2=0x7f0d5019e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:12.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.376+0000 7f0d55102640 1 --2- 172.21.15.116:0/928959942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d50102cc0 0x7f0d5019e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.376+0000 7f0d55102640 1 -- 172.21.15.116:0/928959942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d50105670 msgr2=0x7f0d501a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:12.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.376+0000 7f0d55102640 1 --2- 172.21.15.116:0/928959942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d50105670 0x7f0d501a5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.376+0000 7f0d55102640 1 -- 172.21.15.116:0/928959942 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d501a6080 con 0x7f0d50100370 2024-09-12T03:19:12.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.376+0000 7f0d54901640 1 --2- 172.21.15.116:0/928959942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d50102cc0 0x7f0d5019e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:12.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.376+0000 7f0d55903640 1 --2- 172.21.15.116:0/928959942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d50105670 0x7f0d501a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:12.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.376+0000 7f0d55102640 1 --2- 172.21.15.116:0/928959942 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d50100370 0x7f0d5019e460 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00e9f0 tx=0x7f0d4c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:12.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.376+0000 7f0d467fc640 1 -- 172.21.15.116:0/928959942 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c00cd80 con 0x7f0d50100370 2024-09-12T03:19:12.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.376+0000 7f0d467fc640 1 -- 172.21.15.116:0/928959942 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d4c005500 con 0x7f0d50100370 2024-09-12T03:19:12.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.376+0000 7f0d57366640 1 -- 172.21.15.116:0/928959942 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d501a6370 con 0x7f0d50100370 2024-09-12T03:19:12.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.376+0000 7f0d467fc640 1 -- 172.21.15.116:0/928959942 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c010690 con 0x7f0d50100370 2024-09-12T03:19:12.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.376+0000 7f0d57366640 1 -- 172.21.15.116:0/928959942 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d501a6870 con 0x7f0d50100370 2024-09-12T03:19:12.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.377+0000 7f0d57366640 1 -- 172.21.15.116:0/928959942 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d500fdc30 con 0x7f0d50100370 2024-09-12T03:19:12.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.379+0000 7f0d467fc640 1 -- 172.21.15.116:0/928959942 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f0d4c0046c0 con 0x7f0d50100370 2024-09-12T03:19:12.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.380+0000 7f0d467fc640 1 --2- 172.21.15.116:0/928959942 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0d3c0a2b90 0x7f0d3c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:12.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.380+0000 7f0d54901640 1 --2- 172.21.15.116:0/928959942 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0d3c0a2b90 0x7f0d3c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:12.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.380+0000 7f0d467fc640 1 -- 172.21.15.116:0/928959942 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f0d4c0cf7a0 con 0x7f0d50100370 2024-09-12T03:19:12.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.380+0000 7f0d54901640 1 --2- 172.21.15.116:0/928959942 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0d3c0a2b90 0x7f0d3c0a5050 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0d40000c00 tx=0x7f0d40031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:12.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.383+0000 7f0d467fc640 1 -- 172.21.15.116:0/928959942 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0d4c0541b0 con 0x7f0d50100370 2024-09-12T03:19:12.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.557+0000 7f0d57366640 1 -- 172.21.15.116:0/928959942 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0d50062df0 con 0x7f0d50100370 2024-09-12T03:19:12.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.557+0000 7f0d467fc640 1 -- 172.21.15.116:0/928959942 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) ==== 104+0+11 (secure 0 0 0) 0x7f0d4c01f340 con 0x7f0d50100370 2024-09-12T03:19:12.557 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-09-12T03:19:12.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.559+0000 7f0d57366640 1 -- 172.21.15.116:0/928959942 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0d3c0a2b90 msgr2=0x7f0d3c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:12.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.559+0000 7f0d57366640 1 --2- 172.21.15.116:0/928959942 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0d3c0a2b90 0x7f0d3c0a5050 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0d40000c00 tx=0x7f0d40031400 comp rx=0 tx=0).stop 2024-09-12T03:19:12.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.560+0000 7f0d57366640 1 -- 172.21.15.116:0/928959942 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d50100370 msgr2=0x7f0d5019e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:12.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.560+0000 7f0d57366640 1 --2- 172.21.15.116:0/928959942 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d50100370 0x7f0d5019e460 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00e9f0 tx=0x7f0d4c00edf0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.560+0000 7f0d57366640 1 -- 172.21.15.116:0/928959942 shutdown_connections 2024-09-12T03:19:12.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.560+0000 7f0d57366640 1 --2- 172.21.15.116:0/928959942 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f0d3c0a2b90 0x7f0d3c0a5050 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.560+0000 7f0d57366640 1 --2- 172.21.15.116:0/928959942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d50105670 0x7f0d501a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.560+0000 7f0d57366640 1 --2- 172.21.15.116:0/928959942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d50102cc0 0x7f0d5019e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.560+0000 7f0d57366640 1 --2- 172.21.15.116:0/928959942 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d50100370 0x7f0d5019e460 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.560+0000 7f0d57366640 1 -- 172.21.15.116:0/928959942 >> 172.21.15.116:0/928959942 conn(0x7f0d500fc140 msgr2=0x7f0d50100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:12.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.560+0000 7f0d57366640 1 -- 172.21.15.116:0/928959942 shutdown_connections 2024-09-12T03:19:12.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.560+0000 7f0d57366640 1 -- 172.21.15.116:0/928959942 wait complete. 2024-09-12T03:19:12.571 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-12T03:19:12.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.677+0000 7fce7d4c9640 1 -- 172.21.15.116:0/485765181 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce78104ee0 msgr2=0x7fce781072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:12.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.677+0000 7fce7d4c9640 1 --2- 172.21.15.116:0/485765181 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce78104ee0 0x7fce781072d0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009900 tx=0x7fce6c02a410 comp rx=0 tx=0).stop 2024-09-12T03:19:12.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.677+0000 7fce7d4c9640 1 -- 172.21.15.116:0/485765181 shutdown_connections 2024-09-12T03:19:12.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.677+0000 7fce7d4c9640 1 --2- 172.21.15.116:0/485765181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce78107810 0x7fce78109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.677+0000 7fce7d4c9640 1 --2- 172.21.15.116:0/485765181 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce78104ee0 0x7fce781072d0 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.677+0000 7fce7d4c9640 1 --2- 172.21.15.116:0/485765181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce781025b0 0x7fce781049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.677+0000 7fce7d4c9640 1 -- 172.21.15.116:0/485765181 >> 172.21.15.116:0/485765181 conn(0x7fce780fc1b0 msgr2=0x7fce780fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:12.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.677+0000 7fce7d4c9640 1 -- 172.21.15.116:0/485765181 shutdown_connections 2024-09-12T03:19:12.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.677+0000 7fce7d4c9640 1 -- 172.21.15.116:0/485765181 wait complete. 2024-09-12T03:19:12.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.678+0000 7fce7d4c9640 1 Processor -- start 2024-09-12T03:19:12.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.678+0000 7fce7d4c9640 1 -- start start 2024-09-12T03:19:12.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.678+0000 7fce7d4c9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce781025b0 0x7fce78197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:12.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.678+0000 7fce7d4c9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce78104ee0 0x7fce78198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:12.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.678+0000 7fce7d4c9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce78107810 0x7fce7819f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:12.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.678+0000 7fce7d4c9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fce7810d7f0 con 0x7fce781025b0 2024-09-12T03:19:12.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.678+0000 7fce7d4c9640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fce7810d670 con 0x7fce78107810 2024-09-12T03:19:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.678+0000 7fce7d4c9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fce7810d970 con 0x7fce78104ee0 2024-09-12T03:19:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.678+0000 7fce76ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce781025b0 0x7fce78197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.678+0000 7fce76ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce781025b0 0x7fce78197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42362/0 (socket says 172.21.15.116:42362) 2024-09-12T03:19:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.678+0000 7fce76ffd640 1 -- 172.21.15.116:0/3316961184 learned_addr learned my addr 172.21.15.116:0/3316961184 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.678+0000 7fce767fc640 1 --2- 172.21.15.116:0/3316961184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce78104ee0 0x7fce78198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.678+0000 7fce777fe640 1 --2- 172.21.15.116:0/3316961184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce78107810 0x7fce7819f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.679+0000 7fce76ffd640 1 -- 172.21.15.116:0/3316961184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce78104ee0 msgr2=0x7fce78198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.679+0000 7fce76ffd640 1 --2- 172.21.15.116:0/3316961184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce78104ee0 0x7fce78198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.679+0000 7fce76ffd640 1 -- 172.21.15.116:0/3316961184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce78107810 msgr2=0x7fce7819f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.679+0000 7fce76ffd640 1 --2- 172.21.15.116:0/3316961184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce78107810 0x7fce7819f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.679+0000 7fce76ffd640 1 -- 172.21.15.116:0/3316961184 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce7819fb20 con 0x7fce781025b0 2024-09-12T03:19:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.679+0000 7fce76ffd640 1 --2- 172.21.15.116:0/3316961184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce781025b0 0x7fce78197f00 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fce6800d950 tx=0x7fce6800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.679+0000 7fce5ffff640 1 -- 172.21.15.116:0/3316961184 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce68017070 con 0x7fce781025b0 2024-09-12T03:19:12.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.679+0000 7fce5ffff640 1 -- 172.21.15.116:0/3316961184 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce68009c30 con 0x7fce781025b0 2024-09-12T03:19:12.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.679+0000 7fce7d4c9640 1 -- 172.21.15.116:0/3316961184 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce7819fe10 con 0x7fce781025b0 2024-09-12T03:19:12.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.679+0000 7fce5ffff640 1 -- 172.21.15.116:0/3316961184 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6800f6b0 con 0x7fce781025b0 2024-09-12T03:19:12.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.679+0000 7fce7d4c9640 1 -- 172.21.15.116:0/3316961184 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce781a0270 con 0x7fce781025b0 2024-09-12T03:19:12.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.680+0000 7fce7d4c9640 1 -- 172.21.15.116:0/3316961184 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce780fdca0 con 0x7fce781025b0 2024-09-12T03:19:12.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.681+0000 7fce5ffff640 1 -- 172.21.15.116:0/3316961184 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fce680109d0 con 0x7fce781025b0 2024-09-12T03:19:12.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.682+0000 7fce5ffff640 1 --2- 172.21.15.116:0/3316961184 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fce600a2b90 0x7fce600a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:12.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.682+0000 7fce767fc640 1 --2- 172.21.15.116:0/3316961184 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fce600a2b90 0x7fce600a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:12.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.682+0000 7fce5ffff640 1 -- 172.21.15.116:0/3316961184 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fce680cebb0 con 0x7fce781025b0 2024-09-12T03:19:12.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.683+0000 7fce767fc640 1 --2- 172.21.15.116:0/3316961184 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fce600a2b90 0x7fce600a5050 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fce6c005ab0 tx=0x7fce6c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:12.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.687+0000 7fce5ffff640 1 -- 172.21.15.116:0/3316961184 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fce68009de0 con 0x7fce781025b0 2024-09-12T03:19:12.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:12 smithi116 ceph-mon[31410]: pgmap v824: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-12T03:19:12.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:12 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3895480937' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:12.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:12 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1564390620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:19:12.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.861+0000 7fce7d4c9640 1 -- 172.21.15.116:0/3316961184 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fce78062f30 con 0x7fce781025b0 2024-09-12T03:19:12.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.862+0000 7fce5ffff640 1 -- 172.21.15.116:0/3316961184 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) ==== 100+0+11 (secure 0 0 0) 0x7fce6809b4f0 con 0x7fce781025b0 2024-09-12T03:19:12.862 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 64 2024-09-12T03:19:12.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.864+0000 7fce7d4c9640 1 -- 172.21.15.116:0/3316961184 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fce600a2b90 msgr2=0x7fce600a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:12.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.864+0000 7fce7d4c9640 1 --2- 172.21.15.116:0/3316961184 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fce600a2b90 0x7fce600a5050 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fce6c005ab0 tx=0x7fce6c000bc0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.864+0000 7fce7d4c9640 1 -- 172.21.15.116:0/3316961184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce781025b0 msgr2=0x7fce78197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:12.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.864+0000 7fce7d4c9640 1 --2- 172.21.15.116:0/3316961184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce781025b0 0x7fce78197f00 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fce6800d950 tx=0x7fce6800de10 comp rx=0 tx=0).stop 2024-09-12T03:19:12.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.864+0000 7fce7d4c9640 1 -- 172.21.15.116:0/3316961184 shutdown_connections 2024-09-12T03:19:12.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.864+0000 7fce7d4c9640 1 --2- 172.21.15.116:0/3316961184 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fce600a2b90 0x7fce600a5050 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.864+0000 7fce7d4c9640 1 --2- 172.21.15.116:0/3316961184 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce78107810 0x7fce7819f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.864+0000 7fce7d4c9640 1 --2- 172.21.15.116:0/3316961184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce78104ee0 0x7fce78198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.864+0000 7fce7d4c9640 1 --2- 172.21.15.116:0/3316961184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce781025b0 0x7fce78197f00 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.864+0000 7fce7d4c9640 1 -- 172.21.15.116:0/3316961184 >> 172.21.15.116:0/3316961184 conn(0x7fce780fc1b0 msgr2=0x7fce78102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:12.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.864+0000 7fce7d4c9640 1 -- 172.21.15.116:0/3316961184 shutdown_connections 2024-09-12T03:19:12.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.864+0000 7fce7d4c9640 1 -- 172.21.15.116:0/3316961184 wait complete. 2024-09-12T03:19:12.876 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-12T03:19:12.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:12 smithi183 ceph-mon[35535]: pgmap v824: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-12T03:19:12.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:12 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3895480937' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:12.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:12 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1564390620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:19:12.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.977+0000 7f8313ba1640 1 -- 172.21.15.116:0/1536604581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f830c1025c0 msgr2=0x7f830c108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:12.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.977+0000 7f8313ba1640 1 --2- 172.21.15.116:0/1536604581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f830c1025c0 0x7f830c108e50 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f8304009ca0 tx=0x7f830402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.977+0000 7f8313ba1640 1 -- 172.21.15.116:0/1536604581 shutdown_connections 2024-09-12T03:19:12.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.977+0000 7f8313ba1640 1 --2- 172.21.15.116:0/1536604581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f830c1025c0 0x7f830c108e50 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.977+0000 7f8313ba1640 1 --2- 172.21.15.116:0/1536604581 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f830c101c00 0x7f830c102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.977+0000 7f8313ba1640 1 --2- 172.21.15.116:0/1536604581 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f830c100a10 0x7f830c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.977+0000 7f8313ba1640 1 -- 172.21.15.116:0/1536604581 >> 172.21.15.116:0/1536604581 conn(0x7f830c0fc170 msgr2=0x7f830c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:12.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.977+0000 7f8313ba1640 1 -- 172.21.15.116:0/1536604581 shutdown_connections 2024-09-12T03:19:12.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.977+0000 7f8313ba1640 1 -- 172.21.15.116:0/1536604581 wait complete. 2024-09-12T03:19:12.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.977+0000 7f8313ba1640 1 Processor -- start 2024-09-12T03:19:12.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.977+0000 7f8313ba1640 1 -- start start 2024-09-12T03:19:12.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.977+0000 7f8313ba1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f830c100a10 0x7f830c069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:12.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.977+0000 7f8313ba1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f830c101c00 0x7f830c19cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:12.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.977+0000 7f8313ba1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f830c1025c0 0x7f830c19f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:12.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.977+0000 7f8313ba1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f830c10c710 con 0x7f830c101c00 2024-09-12T03:19:12.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.977+0000 7f8313ba1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f830c10c590 con 0x7f830c1025c0 2024-09-12T03:19:12.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.977+0000 7f8313ba1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f830c10c890 con 0x7f830c100a10 2024-09-12T03:19:12.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.978+0000 7f831113c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f830c101c00 0x7f830c19cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:12.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.978+0000 7f831113c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f830c101c00 0x7f830c19cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42378/0 (socket says 172.21.15.116:42378) 2024-09-12T03:19:12.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.978+0000 7f831113c640 1 -- 172.21.15.116:0/3629247043 learned_addr learned my addr 172.21.15.116:0/3629247043 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:12.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.978+0000 7f831213e640 1 --2- 172.21.15.116:0/3629247043 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f830c1025c0 0x7f830c19f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:12.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.978+0000 7f831193d640 1 --2- 172.21.15.116:0/3629247043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f830c100a10 0x7f830c069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:12.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.978+0000 7f831113c640 1 -- 172.21.15.116:0/3629247043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f830c100a10 msgr2=0x7f830c069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:12.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.978+0000 7f831113c640 1 --2- 172.21.15.116:0/3629247043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f830c100a10 0x7f830c069ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.978+0000 7f831113c640 1 -- 172.21.15.116:0/3629247043 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f830c1025c0 msgr2=0x7f830c19f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:12.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.978+0000 7f831113c640 1 --2- 172.21.15.116:0/3629247043 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f830c1025c0 0x7f830c19f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:12.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.978+0000 7f831113c640 1 -- 172.21.15.116:0/3629247043 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830c19fa90 con 0x7f830c101c00 2024-09-12T03:19:12.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.978+0000 7f831193d640 1 --2- 172.21.15.116:0/3629247043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f830c100a10 0x7f830c069ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:12.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.978+0000 7f831113c640 1 --2- 172.21.15.116:0/3629247043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f830c101c00 0x7f830c19cb60 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00c920 tx=0x7f82fc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:12.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.978+0000 7f8302ffd640 1 -- 172.21.15.116:0/3629247043 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc007c70 con 0x7f830c101c00 2024-09-12T03:19:12.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.978+0000 7f8302ffd640 1 -- 172.21.15.116:0/3629247043 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82fc007e10 con 0x7f830c101c00 2024-09-12T03:19:12.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.978+0000 7f8302ffd640 1 -- 172.21.15.116:0/3629247043 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc018890 con 0x7f830c101c00 2024-09-12T03:19:12.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.978+0000 7f8313ba1640 1 -- 172.21.15.116:0/3629247043 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f830c19fd20 con 0x7f830c101c00 2024-09-12T03:19:12.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.978+0000 7f8313ba1640 1 -- 172.21.15.116:0/3629247043 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f830c1a0240 con 0x7f830c101c00 2024-09-12T03:19:12.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.980+0000 7f8313ba1640 1 -- 172.21.15.116:0/3629247043 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f830c107d30 con 0x7f830c101c00 2024-09-12T03:19:12.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.980+0000 7f8302ffd640 1 -- 172.21.15.116:0/3629247043 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f82fc018a70 con 0x7f830c101c00 2024-09-12T03:19:12.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.981+0000 7f8302ffd640 1 --2- 172.21.15.116:0/3629247043 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f82f80a2b90 0x7f82f80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:12.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.981+0000 7f8302ffd640 1 -- 172.21.15.116:0/3629247043 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f82fc0cf160 con 0x7f830c101c00 2024-09-12T03:19:12.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.981+0000 7f831193d640 1 --2- 172.21.15.116:0/3629247043 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f82f80a2b90 0x7f82f80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:12.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.982+0000 7f831193d640 1 --2- 172.21.15.116:0/3629247043 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f82f80a2b90 0x7f82f80a5050 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f830c101a60 tx=0x7f8308008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:12.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:12.985+0000 7f8302ffd640 1 -- 172.21.15.116:0/3629247043 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f82fc014030 con 0x7f830c101c00 2024-09-12T03:19:12.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:12 smithi177 ceph-mon[35305]: pgmap v824: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-12T03:19:12.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:12 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3895480937' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:12.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:12 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1564390620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:19:13.219 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:13.218+0000 7f8313ba1640 1 -- 172.21.15.116:0/3629247043 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f830c062df0 con 0x7f830c101c00 2024-09-12T03:19:13.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:13.230+0000 7f8302ffd640 1 -- 172.21.15.116:0/3629247043 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+740379 (secure 0 0 0) 0x7f82fc09c0f0 con 0x7f830c101c00 2024-09-12T03:19:13.242 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:19:13.242 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:19:13.242 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 20, 2024-09-12T03:19:13.242 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.242 INFO:teuthology.orchestra.run.smithi116.stdout: "active_gid": 14150, 2024-09-12T03:19:13.242 INFO:teuthology.orchestra.run.smithi116.stdout: "active_name": "x", 2024-09-12T03:19:13.242 INFO:teuthology.orchestra.run.smithi116.stdout: "active_addrs": { 2024-09-12T03:19:13.242 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2024-09-12T03:19:13.242 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:13.242 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2024-09-12T03:19:13.243 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6800", 2024-09-12T03:19:13.243 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 2896038008 2024-09-12T03:19:13.243 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.243 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:13.243 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-09-12T03:19:13.243 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6801", 2024-09-12T03:19:13.243 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 2896038008 2024-09-12T03:19:13.243 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.243 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2024-09-12T03:19:13.244 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.244 INFO:teuthology.orchestra.run.smithi116.stdout: "active_addr": "172.21.15.116:6801/2896038008", 2024-09-12T03:19:13.244 INFO:teuthology.orchestra.run.smithi116.stdout: "active_change": "2024-09-12T02:54:03.026986+0000", 2024-09-12T03:19:13.244 INFO:teuthology.orchestra.run.smithi116.stdout: "active_mgr_features": 4540701547738038271, 2024-09-12T03:19:13.244 INFO:teuthology.orchestra.run.smithi116.stdout: "available": true, 2024-09-12T03:19:13.244 INFO:teuthology.orchestra.run.smithi116.stdout: "standbys": [ 2024-09-12T03:19:13.244 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:13.244 INFO:teuthology.orchestra.run.smithi116.stdout: "gid": 24115, 2024-09-12T03:19:13.244 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "z", 2024-09-12T03:19:13.244 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr_features": 4540701547738038271, 2024-09-12T03:19:13.244 INFO:teuthology.orchestra.run.smithi116.stdout: "available_modules": [ 2024-09-12T03:19:13.244 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:13.244 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "alerts", 2024-09-12T03:19:13.245 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:13.245 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:13.245 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:13.245 INFO:teuthology.orchestra.run.smithi116.stdout: "interval": { 2024-09-12T03:19:13.245 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "interval", 2024-09-12T03:19:13.245 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:13.245 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.245 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.245 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "60", 2024-09-12T03:19:13.245 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.246 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.246 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.246 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T03:19:13.246 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.246 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.246 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.246 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.246 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:13.246 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:13.246 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.247 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.247 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.247 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.247 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.247 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.247 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.247 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.247 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.247 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.247 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.247 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.247 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.248 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.248 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.248 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.248 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.248 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.248 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.248 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:13.248 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:13.248 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.249 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.249 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.249 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.249 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.249 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.249 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.249 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.249 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.249 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.250 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.250 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.250 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:13.250 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:13.250 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.250 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.250 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.250 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:13.250 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.251 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.251 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.251 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.251 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.251 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.251 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.251 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.251 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.251 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.251 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.251 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.251 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.252 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.252 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.252 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:13.252 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:13.252 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.252 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.252 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.252 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.252 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.252 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.252 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.252 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.252 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.253 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.253 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.253 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.253 INFO:teuthology.orchestra.run.smithi116.stdout: "smtp_destination": { 2024-09-12T03:19:13.253 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "smtp_destination", 2024-09-12T03:19:13.253 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.253 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.253 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.253 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.253 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.253 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.253 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.253 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Email address to send alerts to", 2024-09-12T03:19:13.253 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.254 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.254 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.254 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.254 INFO:teuthology.orchestra.run.smithi116.stdout: "smtp_from_name": { 2024-09-12T03:19:13.254 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "smtp_from_name", 2024-09-12T03:19:13.254 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.254 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.254 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.254 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "Ceph", 2024-09-12T03:19:13.254 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.254 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.254 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.254 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Email From: name", 2024-09-12T03:19:13.254 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.255 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.255 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.255 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.255 INFO:teuthology.orchestra.run.smithi116.stdout: "smtp_host": { 2024-09-12T03:19:13.255 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "smtp_host", 2024-09-12T03:19:13.255 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.255 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.255 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.255 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.255 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.255 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.255 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.255 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "SMTP server", 2024-09-12T03:19:13.256 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.256 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.256 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.256 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.256 INFO:teuthology.orchestra.run.smithi116.stdout: "smtp_password": { 2024-09-12T03:19:13.256 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "smtp_password", 2024-09-12T03:19:13.256 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.256 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.256 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.256 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.256 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.256 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.256 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.257 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Password to authenticate with", 2024-09-12T03:19:13.257 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.257 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.257 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.257 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.257 INFO:teuthology.orchestra.run.smithi116.stdout: "smtp_port": { 2024-09-12T03:19:13.257 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "smtp_port", 2024-09-12T03:19:13.257 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:13.257 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.257 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.257 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "465", 2024-09-12T03:19:13.257 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.258 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.258 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.258 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "SMTP port", 2024-09-12T03:19:13.258 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.258 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.258 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.258 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.258 INFO:teuthology.orchestra.run.smithi116.stdout: "smtp_sender": { 2024-09-12T03:19:13.258 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "smtp_sender", 2024-09-12T03:19:13.258 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.258 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.258 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.258 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.258 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.259 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.259 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.259 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "SMTP envelope sender", 2024-09-12T03:19:13.259 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.259 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.259 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.259 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.259 INFO:teuthology.orchestra.run.smithi116.stdout: "smtp_ssl": { 2024-09-12T03:19:13.259 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "smtp_ssl", 2024-09-12T03:19:13.259 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.259 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.259 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.260 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:13.260 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.260 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.260 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.260 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T03:19:13.260 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.260 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.260 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.260 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.260 INFO:teuthology.orchestra.run.smithi116.stdout: "smtp_user": { 2024-09-12T03:19:13.260 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "smtp_user", 2024-09-12T03:19:13.261 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.261 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.261 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.261 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.261 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.261 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.261 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.261 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "User to authenticate as", 2024-09-12T03:19:13.261 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.261 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.262 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.262 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.262 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.262 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.262 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:13.262 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "balancer", 2024-09-12T03:19:13.262 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:13.262 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:13.262 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:13.262 INFO:teuthology.orchestra.run.smithi116.stdout: "active": { 2024-09-12T03:19:13.262 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "active", 2024-09-12T03:19:13.263 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.263 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.263 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.263 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:13.263 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.263 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.263 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.263 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T03:19:13.263 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.263 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.263 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.263 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.264 INFO:teuthology.orchestra.run.smithi116.stdout: "begin_time": { 2024-09-12T03:19:13.264 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "begin_time", 2024-09-12T03:19:13.264 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.264 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.264 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.264 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "0000", 2024-09-12T03:19:13.264 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.264 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.264 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.264 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T03:19:13.264 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:19:13.264 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.264 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.265 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.265 INFO:teuthology.orchestra.run.smithi116.stdout: "begin_weekday": { 2024-09-12T03:19:13.265 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "begin_weekday", 2024-09-12T03:19:13.265 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "uint", 2024-09-12T03:19:13.265 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.265 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.265 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "0", 2024-09-12T03:19:13.265 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "0", 2024-09-12T03:19:13.265 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "6", 2024-09-12T03:19:13.265 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.265 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T03:19:13.265 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:19:13.265 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.265 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.266 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.266 INFO:teuthology.orchestra.run.smithi116.stdout: "crush_compat_max_iterations": { 2024-09-12T03:19:13.266 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "crush_compat_max_iterations", 2024-09-12T03:19:13.266 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "uint", 2024-09-12T03:19:13.266 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.266 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.266 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "25", 2024-09-12T03:19:13.266 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "1", 2024-09-12T03:19:13.266 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "250", 2024-09-12T03:19:13.266 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.266 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T03:19:13.266 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.267 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.267 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.267 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.267 INFO:teuthology.orchestra.run.smithi116.stdout: "crush_compat_metrics": { 2024-09-12T03:19:13.267 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "crush_compat_metrics", 2024-09-12T03:19:13.267 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.267 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.267 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.267 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T03:19:13.267 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.267 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.267 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.267 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T03:19:13.267 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T03:19:13.268 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.268 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.268 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.268 INFO:teuthology.orchestra.run.smithi116.stdout: "crush_compat_step": { 2024-09-12T03:19:13.268 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "crush_compat_step", 2024-09-12T03:19:13.268 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "float", 2024-09-12T03:19:13.268 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.268 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.268 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "0.5", 2024-09-12T03:19:13.268 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "0.001", 2024-09-12T03:19:13.268 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "0.999", 2024-09-12T03:19:13.268 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.268 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "aggressiveness of optimization", 2024-09-12T03:19:13.269 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T03:19:13.269 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.269 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.269 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.269 INFO:teuthology.orchestra.run.smithi116.stdout: "end_time": { 2024-09-12T03:19:13.269 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "end_time", 2024-09-12T03:19:13.269 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.269 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.269 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.269 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "2359", 2024-09-12T03:19:13.269 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.269 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.269 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.269 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T03:19:13.270 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:19:13.270 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.270 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.270 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.270 INFO:teuthology.orchestra.run.smithi116.stdout: "end_weekday": { 2024-09-12T03:19:13.270 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "end_weekday", 2024-09-12T03:19:13.270 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "uint", 2024-09-12T03:19:13.270 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.270 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.270 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "0", 2024-09-12T03:19:13.270 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "0", 2024-09-12T03:19:13.270 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "6", 2024-09-12T03:19:13.270 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.270 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T03:19:13.270 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:19:13.271 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.271 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.271 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.271 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:13.271 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:13.271 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.271 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.271 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.271 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.271 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.271 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.272 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.272 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.272 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.272 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.272 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.272 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.272 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.272 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.272 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.272 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.272 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.273 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.273 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.273 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:13.273 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:13.273 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.273 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.273 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.273 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.273 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.273 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.273 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.274 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.274 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.274 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.274 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.274 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.274 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:13.274 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:13.274 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.274 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.274 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.275 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:13.275 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.275 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.275 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.275 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.275 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.275 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.275 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.275 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.275 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.276 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.276 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.276 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.276 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.276 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.276 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.276 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:13.276 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:13.276 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.276 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.276 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.276 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.277 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.277 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.277 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.277 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.277 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.277 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.277 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.277 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.277 INFO:teuthology.orchestra.run.smithi116.stdout: "min_score": { 2024-09-12T03:19:13.277 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "min_score", 2024-09-12T03:19:13.278 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "float", 2024-09-12T03:19:13.278 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.278 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.278 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "0", 2024-09-12T03:19:13.278 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.278 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.278 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.278 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T03:19:13.279 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.279 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.279 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.279 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.279 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": { 2024-09-12T03:19:13.279 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "mode", 2024-09-12T03:19:13.279 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.279 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.279 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.279 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "upmap", 2024-09-12T03:19:13.279 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.280 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.280 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.280 INFO:teuthology.orchestra.run.smithi116.stdout: "crush-compat", 2024-09-12T03:19:13.280 INFO:teuthology.orchestra.run.smithi116.stdout: "none", 2024-09-12T03:19:13.280 INFO:teuthology.orchestra.run.smithi116.stdout: "read", 2024-09-12T03:19:13.280 INFO:teuthology.orchestra.run.smithi116.stdout: "upmap", 2024-09-12T03:19:13.280 INFO:teuthology.orchestra.run.smithi116.stdout: "upmap-read" 2024-09-12T03:19:13.280 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.280 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Balancer mode", 2024-09-12T03:19:13.280 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.281 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.281 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.281 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.281 INFO:teuthology.orchestra.run.smithi116.stdout: "pool_ids": { 2024-09-12T03:19:13.281 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "pool_ids", 2024-09-12T03:19:13.281 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.281 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.281 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.281 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.281 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.282 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.282 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.282 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T03:19:13.282 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.282 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.282 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.282 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.282 INFO:teuthology.orchestra.run.smithi116.stdout: "sleep_interval": { 2024-09-12T03:19:13.282 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "sleep_interval", 2024-09-12T03:19:13.282 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:13.283 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.283 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.283 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "60", 2024-09-12T03:19:13.283 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.283 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.283 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.283 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T03:19:13.283 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.283 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.283 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.283 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.284 INFO:teuthology.orchestra.run.smithi116.stdout: "upmap_max_deviation": { 2024-09-12T03:19:13.284 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "upmap_max_deviation", 2024-09-12T03:19:13.284 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:13.284 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.284 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.284 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "5", 2024-09-12T03:19:13.284 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "1", 2024-09-12T03:19:13.284 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.284 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.284 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T03:19:13.284 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T03:19:13.284 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.284 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.285 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.285 INFO:teuthology.orchestra.run.smithi116.stdout: "upmap_max_optimizations": { 2024-09-12T03:19:13.285 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "upmap_max_optimizations", 2024-09-12T03:19:13.285 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "uint", 2024-09-12T03:19:13.285 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.285 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.285 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "10", 2024-09-12T03:19:13.285 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.285 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.285 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.285 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T03:19:13.285 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.285 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.286 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.286 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.286 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.286 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.286 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:13.286 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "cephadm", 2024-09-12T03:19:13.286 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:13.286 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:13.286 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:13.286 INFO:teuthology.orchestra.run.smithi116.stdout: "agent_down_multiplier": { 2024-09-12T03:19:13.286 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "agent_down_multiplier", 2024-09-12T03:19:13.286 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "float", 2024-09-12T03:19:13.286 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.286 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.287 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "3.0", 2024-09-12T03:19:13.287 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.287 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.287 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.287 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T03:19:13.287 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.287 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.287 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.287 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.287 INFO:teuthology.orchestra.run.smithi116.stdout: "agent_refresh_rate": { 2024-09-12T03:19:13.287 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "agent_refresh_rate", 2024-09-12T03:19:13.288 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:13.288 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.288 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.288 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "20", 2024-09-12T03:19:13.288 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.288 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.288 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.288 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T03:19:13.289 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.289 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.289 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.289 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.289 INFO:teuthology.orchestra.run.smithi116.stdout: "agent_starting_port": { 2024-09-12T03:19:13.289 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "agent_starting_port", 2024-09-12T03:19:13.289 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:13.289 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.289 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.289 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "4721", 2024-09-12T03:19:13.289 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.289 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.290 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.290 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T03:19:13.290 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.290 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.290 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.290 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.290 INFO:teuthology.orchestra.run.smithi116.stdout: "allow_ptrace": { 2024-09-12T03:19:13.290 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "allow_ptrace", 2024-09-12T03:19:13.290 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.290 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.290 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.290 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.290 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.291 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.291 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.291 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T03:19:13.291 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T03:19:13.291 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.291 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.291 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.291 INFO:teuthology.orchestra.run.smithi116.stdout: "autotune_interval": { 2024-09-12T03:19:13.291 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "autotune_interval", 2024-09-12T03:19:13.291 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:13.291 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.291 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.291 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "600", 2024-09-12T03:19:13.292 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.292 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.292 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.292 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T03:19:13.292 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.292 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.292 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.292 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.292 INFO:teuthology.orchestra.run.smithi116.stdout: "autotune_memory_target_ratio": { 2024-09-12T03:19:13.292 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T03:19:13.292 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "float", 2024-09-12T03:19:13.292 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.292 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.293 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "0.7", 2024-09-12T03:19:13.293 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.293 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.293 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.293 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T03:19:13.293 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.293 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.293 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.293 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.293 INFO:teuthology.orchestra.run.smithi116.stdout: "cephadm_log_destination": { 2024-09-12T03:19:13.293 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "cephadm_log_destination", 2024-09-12T03:19:13.293 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.293 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.293 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.294 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.294 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.294 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.294 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.294 INFO:teuthology.orchestra.run.smithi116.stdout: "file", 2024-09-12T03:19:13.294 INFO:teuthology.orchestra.run.smithi116.stdout: "file,syslog", 2024-09-12T03:19:13.294 INFO:teuthology.orchestra.run.smithi116.stdout: "syslog" 2024-09-12T03:19:13.294 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.294 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T03:19:13.294 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.294 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.294 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.294 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.294 INFO:teuthology.orchestra.run.smithi116.stdout: "cgroups_split": { 2024-09-12T03:19:13.295 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "cgroups_split", 2024-09-12T03:19:13.295 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.295 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.295 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.295 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:13.295 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.295 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.295 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.295 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T03:19:13.295 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.295 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.295 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.295 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.296 INFO:teuthology.orchestra.run.smithi116.stdout: "config_checks_enabled": { 2024-09-12T03:19:13.296 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "config_checks_enabled", 2024-09-12T03:19:13.296 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.296 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.296 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.296 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.296 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.296 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.296 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.296 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T03:19:13.296 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.296 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.296 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.296 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.297 INFO:teuthology.orchestra.run.smithi116.stdout: "config_dashboard": { 2024-09-12T03:19:13.297 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "config_dashboard", 2024-09-12T03:19:13.297 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.297 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.297 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.297 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:13.297 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.297 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.298 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.298 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T03:19:13.298 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.298 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.298 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.298 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.298 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_alertmanager": { 2024-09-12T03:19:13.298 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_alertmanager", 2024-09-12T03:19:13.298 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.298 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.298 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.298 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T03:19:13.298 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.298 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.299 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.299 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Prometheus container image", 2024-09-12T03:19:13.299 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.299 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.299 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.299 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.299 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_base": { 2024-09-12T03:19:13.299 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_base", 2024-09-12T03:19:13.299 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.300 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.300 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.300 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T03:19:13.300 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.300 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.300 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.300 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Container image name, without the tag", 2024-09-12T03:19:13.300 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.300 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.300 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.300 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.300 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_elasticsearch": { 2024-09-12T03:19:13.300 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_elasticsearch", 2024-09-12T03:19:13.300 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.301 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.301 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.301 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T03:19:13.301 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.301 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.301 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.301 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "elasticsearch container image", 2024-09-12T03:19:13.301 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.301 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.301 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.301 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.301 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_grafana": { 2024-09-12T03:19:13.301 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_grafana", 2024-09-12T03:19:13.302 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.302 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.302 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.302 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T03:19:13.302 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.302 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.302 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.302 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Prometheus container image", 2024-09-12T03:19:13.302 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.302 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.302 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.302 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.302 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_haproxy": { 2024-09-12T03:19:13.302 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_haproxy", 2024-09-12T03:19:13.303 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.303 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.303 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.303 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T03:19:13.303 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.303 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.303 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.303 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "HAproxy container image", 2024-09-12T03:19:13.303 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.303 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.303 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.303 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.303 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_jaeger_agent": { 2024-09-12T03:19:13.304 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_jaeger_agent", 2024-09-12T03:19:13.304 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.304 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.304 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.304 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T03:19:13.304 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.304 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.304 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.304 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Jaeger agent container image", 2024-09-12T03:19:13.304 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.304 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.304 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.304 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.305 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_jaeger_collector": { 2024-09-12T03:19:13.305 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_jaeger_collector", 2024-09-12T03:19:13.305 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.305 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.305 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.305 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T03:19:13.305 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.305 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.305 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.306 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Jaeger collector container image", 2024-09-12T03:19:13.306 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.306 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.306 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.306 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.306 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_jaeger_query": { 2024-09-12T03:19:13.306 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_jaeger_query", 2024-09-12T03:19:13.306 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.306 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.306 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.306 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T03:19:13.306 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.306 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.306 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.307 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Jaeger query container image", 2024-09-12T03:19:13.307 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.307 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.307 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.307 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.307 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_keepalived": { 2024-09-12T03:19:13.307 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_keepalived", 2024-09-12T03:19:13.307 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.307 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.307 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.307 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T03:19:13.307 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.307 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.308 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.308 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Keepalived container image", 2024-09-12T03:19:13.308 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.308 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.308 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.308 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.308 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_loki": { 2024-09-12T03:19:13.308 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_loki", 2024-09-12T03:19:13.308 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.308 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.308 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.308 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T03:19:13.308 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.308 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.309 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.309 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Loki container image", 2024-09-12T03:19:13.309 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.309 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.309 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.309 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.309 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_node_exporter": { 2024-09-12T03:19:13.309 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_node_exporter", 2024-09-12T03:19:13.309 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.309 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.309 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.309 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T03:19:13.309 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.310 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.310 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.310 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Prometheus container image", 2024-09-12T03:19:13.310 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.310 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.310 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.310 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.310 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_nvmeof": { 2024-09-12T03:19:13.310 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_nvmeof", 2024-09-12T03:19:13.311 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.311 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.311 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.311 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T03:19:13.311 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.311 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.311 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.311 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Nvme-of container image", 2024-09-12T03:19:13.311 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.311 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.311 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.311 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.311 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_prometheus": { 2024-09-12T03:19:13.311 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_prometheus", 2024-09-12T03:19:13.312 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.312 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.312 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.312 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T03:19:13.312 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.312 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.312 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.312 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Prometheus container image", 2024-09-12T03:19:13.312 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.312 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.312 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.313 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.313 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_promtail": { 2024-09-12T03:19:13.313 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_promtail", 2024-09-12T03:19:13.313 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.313 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.313 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.313 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T03:19:13.313 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.313 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.313 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.313 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Promtail container image", 2024-09-12T03:19:13.314 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.314 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.314 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.314 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.314 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_samba": { 2024-09-12T03:19:13.314 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_samba", 2024-09-12T03:19:13.314 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.314 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.314 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.314 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T03:19:13.314 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.314 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.315 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.315 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Samba/SMB container image", 2024-09-12T03:19:13.315 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.315 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.315 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.315 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.315 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_snmp_gateway": { 2024-09-12T03:19:13.315 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_snmp_gateway", 2024-09-12T03:19:13.315 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.315 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.315 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.315 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T03:19:13.316 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.316 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.316 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.316 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "SNMP Gateway container image", 2024-09-12T03:19:13.316 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.316 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.316 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.316 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.316 INFO:teuthology.orchestra.run.smithi116.stdout: "container_init": { 2024-09-12T03:19:13.316 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_init", 2024-09-12T03:19:13.316 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.317 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.317 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.317 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:13.317 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.317 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.317 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.317 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T03:19:13.317 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.317 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.317 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.317 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.318 INFO:teuthology.orchestra.run.smithi116.stdout: "daemon_cache_timeout": { 2024-09-12T03:19:13.318 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "daemon_cache_timeout", 2024-09-12T03:19:13.318 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:13.318 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.318 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.318 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "600", 2024-09-12T03:19:13.318 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.318 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.318 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.318 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T03:19:13.319 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.319 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.319 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.319 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.319 INFO:teuthology.orchestra.run.smithi116.stdout: "default_cephadm_command_timeout": { 2024-09-12T03:19:13.319 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T03:19:13.319 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:13.319 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.319 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.319 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "900", 2024-09-12T03:19:13.319 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.320 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.320 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.320 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T03:19:13.320 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.320 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.320 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.320 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.320 INFO:teuthology.orchestra.run.smithi116.stdout: "default_registry": { 2024-09-12T03:19:13.320 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "default_registry", 2024-09-12T03:19:13.321 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.321 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.321 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.321 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "docker.io", 2024-09-12T03:19:13.321 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.321 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.321 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.321 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T03:19:13.321 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.322 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.322 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.322 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.322 INFO:teuthology.orchestra.run.smithi116.stdout: "device_cache_timeout": { 2024-09-12T03:19:13.322 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "device_cache_timeout", 2024-09-12T03:19:13.322 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:13.322 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.322 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.322 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "1800", 2024-09-12T03:19:13.323 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.323 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.323 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.323 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "seconds to cache device inventory", 2024-09-12T03:19:13.323 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.323 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.323 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.323 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.323 INFO:teuthology.orchestra.run.smithi116.stdout: "device_enhanced_scan": { 2024-09-12T03:19:13.323 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "device_enhanced_scan", 2024-09-12T03:19:13.323 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.323 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.324 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.324 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.324 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.324 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.324 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.324 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T03:19:13.324 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.324 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.324 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.324 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.324 INFO:teuthology.orchestra.run.smithi116.stdout: "facts_cache_timeout": { 2024-09-12T03:19:13.324 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "facts_cache_timeout", 2024-09-12T03:19:13.324 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:13.325 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.325 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.325 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "60", 2024-09-12T03:19:13.325 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.325 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.325 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.325 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "seconds to cache host facts data", 2024-09-12T03:19:13.325 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.325 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.325 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.325 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.325 INFO:teuthology.orchestra.run.smithi116.stdout: "grafana_dashboards_path": { 2024-09-12T03:19:13.325 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "grafana_dashboards_path", 2024-09-12T03:19:13.326 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.326 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.326 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.326 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T03:19:13.326 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.326 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.326 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.326 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T03:19:13.326 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.326 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.326 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.326 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.326 INFO:teuthology.orchestra.run.smithi116.stdout: "host_check_interval": { 2024-09-12T03:19:13.326 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "host_check_interval", 2024-09-12T03:19:13.327 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:13.327 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.327 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.327 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "600", 2024-09-12T03:19:13.327 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.327 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.327 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.327 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "how frequently to perform a host check", 2024-09-12T03:19:13.327 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.327 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.327 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.327 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.327 INFO:teuthology.orchestra.run.smithi116.stdout: "hw_monitoring": { 2024-09-12T03:19:13.328 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "hw_monitoring", 2024-09-12T03:19:13.328 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.328 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.328 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.328 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.328 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.328 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.328 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.328 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T03:19:13.328 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.328 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.328 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.328 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.329 INFO:teuthology.orchestra.run.smithi116.stdout: "inventory_list_all": { 2024-09-12T03:19:13.329 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "inventory_list_all", 2024-09-12T03:19:13.329 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.329 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.329 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.329 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.329 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.329 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.329 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.329 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T03:19:13.329 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.329 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.329 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.330 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.330 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:13.330 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:13.330 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.330 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.330 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.330 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.330 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.330 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.330 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.330 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.330 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.330 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.330 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.331 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.331 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.331 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.331 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.331 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.331 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.331 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.331 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.331 INFO:teuthology.orchestra.run.smithi116.stdout: "log_refresh_metadata": { 2024-09-12T03:19:13.331 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_refresh_metadata", 2024-09-12T03:19:13.331 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.331 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.332 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.332 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.332 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.332 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.332 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.332 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T03:19:13.332 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.332 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.332 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.332 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.332 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:13.332 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:13.332 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.333 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.333 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.333 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:13.333 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.333 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.333 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.333 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T03:19:13.333 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.333 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.333 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.333 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.333 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:13.333 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:13.334 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.334 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.334 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.334 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:13.334 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.334 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.334 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.334 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.334 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.334 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.334 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.334 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.334 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.335 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.335 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.335 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.335 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.335 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.335 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.335 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:13.335 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:13.335 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.335 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.335 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.335 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.335 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.336 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.336 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.336 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.336 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.336 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.336 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.336 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.336 INFO:teuthology.orchestra.run.smithi116.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T03:19:13.336 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T03:19:13.336 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.336 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.336 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.336 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.337 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.337 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.337 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.337 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T03:19:13.337 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.337 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.337 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.337 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.337 INFO:teuthology.orchestra.run.smithi116.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T03:19:13.337 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T03:19:13.337 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.337 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.337 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.338 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "*", 2024-09-12T03:19:13.338 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.338 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.338 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.338 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T03:19:13.338 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.338 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.338 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.338 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.338 INFO:teuthology.orchestra.run.smithi116.stdout: "max_count_per_host": { 2024-09-12T03:19:13.338 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "max_count_per_host", 2024-09-12T03:19:13.338 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:13.338 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.339 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.339 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "10", 2024-09-12T03:19:13.339 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.339 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.339 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.339 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "max number of daemons per service per host", 2024-09-12T03:19:13.339 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.339 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.339 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.339 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.339 INFO:teuthology.orchestra.run.smithi116.stdout: "max_osd_draining_count": { 2024-09-12T03:19:13.339 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "max_osd_draining_count", 2024-09-12T03:19:13.339 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:13.339 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.340 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.340 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "10", 2024-09-12T03:19:13.340 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.340 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.340 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.340 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T03:19:13.340 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.340 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.340 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.340 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.340 INFO:teuthology.orchestra.run.smithi116.stdout: "migration_current": { 2024-09-12T03:19:13.340 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "migration_current", 2024-09-12T03:19:13.340 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:13.341 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.341 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.341 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.341 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.341 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.341 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.341 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "internal - do not modify", 2024-09-12T03:19:13.341 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.341 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.341 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.341 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.341 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": { 2024-09-12T03:19:13.341 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "mode", 2024-09-12T03:19:13.342 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.342 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.342 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.342 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "root", 2024-09-12T03:19:13.342 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.342 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.342 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.342 INFO:teuthology.orchestra.run.smithi116.stdout: "cephadm-package", 2024-09-12T03:19:13.342 INFO:teuthology.orchestra.run.smithi116.stdout: "root" 2024-09-12T03:19:13.342 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.342 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T03:19:13.342 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.342 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.343 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.343 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.343 INFO:teuthology.orchestra.run.smithi116.stdout: "oob_default_addr": { 2024-09-12T03:19:13.343 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "oob_default_addr", 2024-09-12T03:19:13.343 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.343 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.343 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.343 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "169.254.1.1", 2024-09-12T03:19:13.343 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.343 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.343 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.343 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T03:19:13.343 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.344 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.344 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.344 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.344 INFO:teuthology.orchestra.run.smithi116.stdout: "prometheus_alerts_path": { 2024-09-12T03:19:13.344 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "prometheus_alerts_path", 2024-09-12T03:19:13.344 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.344 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.344 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.344 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T03:19:13.345 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.345 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.345 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.345 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T03:19:13.345 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.345 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.345 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.345 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.345 INFO:teuthology.orchestra.run.smithi116.stdout: "registry_insecure": { 2024-09-12T03:19:13.345 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "registry_insecure", 2024-09-12T03:19:13.345 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.345 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.346 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.346 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.346 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.346 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.346 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.346 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T03:19:13.346 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.346 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.346 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.347 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.347 INFO:teuthology.orchestra.run.smithi116.stdout: "registry_password": { 2024-09-12T03:19:13.347 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "registry_password", 2024-09-12T03:19:13.347 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.347 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.347 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.347 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.347 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.347 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.347 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.347 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T03:19:13.347 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.348 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.348 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.348 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.348 INFO:teuthology.orchestra.run.smithi116.stdout: "registry_url": { 2024-09-12T03:19:13.348 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "registry_url", 2024-09-12T03:19:13.348 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.348 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.348 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.348 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.348 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.348 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.349 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.349 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T03:19:13.349 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.349 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.349 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.349 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.349 INFO:teuthology.orchestra.run.smithi116.stdout: "registry_username": { 2024-09-12T03:19:13.349 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "registry_username", 2024-09-12T03:19:13.349 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.349 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.349 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.350 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.350 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.350 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.350 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.350 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T03:19:13.350 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.350 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.350 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.350 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.351 INFO:teuthology.orchestra.run.smithi116.stdout: "secure_monitoring_stack": { 2024-09-12T03:19:13.351 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "secure_monitoring_stack", 2024-09-12T03:19:13.351 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.351 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.351 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.351 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.351 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.351 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.351 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.351 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T03:19:13.351 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.351 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.352 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.352 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.352 INFO:teuthology.orchestra.run.smithi116.stdout: "service_discovery_port": { 2024-09-12T03:19:13.352 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "service_discovery_port", 2024-09-12T03:19:13.352 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:13.352 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.352 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.352 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "8765", 2024-09-12T03:19:13.352 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.353 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.353 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.353 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "cephadm service discovery port", 2024-09-12T03:19:13.353 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.353 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.353 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.353 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.353 INFO:teuthology.orchestra.run.smithi116.stdout: "ssh_config_file": { 2024-09-12T03:19:13.353 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ssh_config_file", 2024-09-12T03:19:13.353 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.354 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.354 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.354 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.354 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.354 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.354 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.354 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T03:19:13.354 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.354 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.354 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.354 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.355 INFO:teuthology.orchestra.run.smithi116.stdout: "use_agent": { 2024-09-12T03:19:13.355 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "use_agent", 2024-09-12T03:19:13.355 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.355 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.355 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.355 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.355 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.355 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.355 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.356 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T03:19:13.356 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.356 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.356 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.356 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.356 INFO:teuthology.orchestra.run.smithi116.stdout: "use_repo_digest": { 2024-09-12T03:19:13.356 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "use_repo_digest", 2024-09-12T03:19:13.356 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.356 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.356 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.356 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:13.356 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.357 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.357 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.357 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T03:19:13.357 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.357 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.357 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.357 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.357 INFO:teuthology.orchestra.run.smithi116.stdout: "warn_on_failed_host_check": { 2024-09-12T03:19:13.357 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "warn_on_failed_host_check", 2024-09-12T03:19:13.357 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.357 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.357 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.357 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:13.358 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.358 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.358 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.358 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T03:19:13.358 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.358 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.358 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.358 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.358 INFO:teuthology.orchestra.run.smithi116.stdout: "warn_on_stray_daemons": { 2024-09-12T03:19:13.358 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "warn_on_stray_daemons", 2024-09-12T03:19:13.358 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.358 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.358 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.359 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:13.359 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.359 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.359 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.359 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T03:19:13.359 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.359 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.359 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.359 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.359 INFO:teuthology.orchestra.run.smithi116.stdout: "warn_on_stray_hosts": { 2024-09-12T03:19:13.359 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "warn_on_stray_hosts", 2024-09-12T03:19:13.359 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.359 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.359 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.360 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:13.360 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.360 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.360 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.360 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T03:19:13.360 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.360 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.360 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.360 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.361 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.361 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.361 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:13.361 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "crash", 2024-09-12T03:19:13.361 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:13.361 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:13.361 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:13.361 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:13.361 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:13.361 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.361 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.361 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.362 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.362 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.362 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.362 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.362 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.362 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.362 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.362 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.362 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.362 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.362 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.362 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.362 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.363 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.363 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.363 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.363 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:13.363 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:13.363 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.363 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.363 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.363 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.363 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.363 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.363 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.363 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.363 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.364 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.364 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.364 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.364 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:13.364 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:13.364 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.364 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.364 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.364 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:13.364 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.364 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.364 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.364 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.365 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.365 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.365 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.365 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.365 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.365 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.365 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.365 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.365 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.365 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.365 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.365 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:13.365 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:13.365 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.366 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.366 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.366 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.366 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.366 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.366 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.366 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.366 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.366 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.366 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.366 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.366 INFO:teuthology.orchestra.run.smithi116.stdout: "retain_interval": { 2024-09-12T03:19:13.366 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "retain_interval", 2024-09-12T03:19:13.367 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:13.367 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.367 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.367 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "31536000", 2024-09-12T03:19:13.367 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.367 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.367 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.367 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T03:19:13.367 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.367 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.367 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.367 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.367 INFO:teuthology.orchestra.run.smithi116.stdout: "warn_recent_interval": { 2024-09-12T03:19:13.367 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "warn_recent_interval", 2024-09-12T03:19:13.368 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:13.368 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.368 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.368 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "1209600", 2024-09-12T03:19:13.368 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.368 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.368 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.368 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T03:19:13.368 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.368 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.368 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.368 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.368 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.368 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.369 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:13.369 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "dashboard", 2024-09-12T03:19:13.369 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:13.369 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:13.369 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:13.369 INFO:teuthology.orchestra.run.smithi116.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T03:19:13.369 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T03:19:13.369 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:13.369 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.369 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.369 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "10", 2024-09-12T03:19:13.369 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.369 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.369 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.370 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.370 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.370 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.370 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.370 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.370 INFO:teuthology.orchestra.run.smithi116.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T03:19:13.370 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T03:19:13.370 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.370 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.370 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.370 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.370 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.370 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.371 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.371 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.371 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.371 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.371 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.371 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.371 INFO:teuthology.orchestra.run.smithi116.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T03:19:13.371 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T03:19:13.371 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.371 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.371 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.371 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:13.371 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.371 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.372 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.372 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.372 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.372 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.372 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.372 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.372 INFO:teuthology.orchestra.run.smithi116.stdout: "AUDIT_API_ENABLED": { 2024-09-12T03:19:13.372 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T03:19:13.372 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.372 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.372 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.372 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.372 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.372 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.373 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.373 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.373 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.373 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.373 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.373 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.373 INFO:teuthology.orchestra.run.smithi116.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T03:19:13.373 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T03:19:13.373 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.373 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.373 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.373 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:13.373 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.373 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.374 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.374 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.374 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.374 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.374 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.374 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.374 INFO:teuthology.orchestra.run.smithi116.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T03:19:13.374 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T03:19:13.374 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.374 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.375 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.375 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:13.375 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.375 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.375 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.375 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.375 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.375 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.375 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.375 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.375 INFO:teuthology.orchestra.run.smithi116.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T03:19:13.375 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T03:19:13.375 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.376 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.376 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.376 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:13.376 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.376 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.376 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.376 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.376 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.376 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.376 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.376 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.376 INFO:teuthology.orchestra.run.smithi116.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T03:19:13.376 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T03:19:13.377 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.377 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.377 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.377 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:13.377 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.377 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.377 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.377 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.377 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.377 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.377 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.377 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.377 INFO:teuthology.orchestra.run.smithi116.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T03:19:13.377 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T03:19:13.378 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.378 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.378 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.378 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:13.378 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.378 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.378 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.378 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.378 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.378 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.378 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.378 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.378 INFO:teuthology.orchestra.run.smithi116.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T03:19:13.379 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T03:19:13.379 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.379 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.379 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.379 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:13.379 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.379 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.379 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.379 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.379 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.379 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.379 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.379 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.379 INFO:teuthology.orchestra.run.smithi116.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T03:19:13.380 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T03:19:13.380 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.380 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.380 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.380 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:13.380 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.380 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.380 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.380 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.380 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.380 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.380 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.380 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.381 INFO:teuthology.orchestra.run.smithi116.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T03:19:13.381 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T03:19:13.381 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.381 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.381 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.381 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:13.381 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.381 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.381 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.381 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.381 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.381 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.381 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.382 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.382 INFO:teuthology.orchestra.run.smithi116.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T03:19:13.382 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T03:19:13.382 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.382 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.382 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.382 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:13.382 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.382 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.382 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.383 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.383 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.383 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.383 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.383 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.383 INFO:teuthology.orchestra.run.smithi116.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T03:19:13.383 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T03:19:13.383 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.383 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.383 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.383 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.383 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.383 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.384 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.384 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.384 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.384 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.384 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.384 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.384 INFO:teuthology.orchestra.run.smithi116.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T03:19:13.384 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T03:19:13.384 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.384 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.384 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.385 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "admin", 2024-09-12T03:19:13.385 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.385 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.385 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.385 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.385 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.385 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.385 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.385 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.385 INFO:teuthology.orchestra.run.smithi116.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T03:19:13.385 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T03:19:13.385 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.386 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.386 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.386 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:13.386 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.386 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.386 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.386 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.386 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.386 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.387 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.387 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.387 INFO:teuthology.orchestra.run.smithi116.stdout: "GRAFANA_API_URL": { 2024-09-12T03:19:13.387 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "GRAFANA_API_URL", 2024-09-12T03:19:13.387 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.387 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.387 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.387 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.387 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.387 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.387 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.387 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.387 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.388 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.388 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.388 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.388 INFO:teuthology.orchestra.run.smithi116.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T03:19:13.388 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T03:19:13.388 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.388 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.388 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.388 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "admin", 2024-09-12T03:19:13.388 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.388 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.389 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.389 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.389 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.389 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.389 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.389 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.389 INFO:teuthology.orchestra.run.smithi116.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T03:19:13.389 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T03:19:13.389 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.389 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.389 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.389 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.390 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.390 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.390 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.390 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.390 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.390 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.390 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.390 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.390 INFO:teuthology.orchestra.run.smithi116.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T03:19:13.390 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T03:19:13.391 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.391 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.391 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.391 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.391 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.391 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.391 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.391 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.391 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.391 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.392 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.392 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.392 INFO:teuthology.orchestra.run.smithi116.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T03:19:13.392 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T03:19:13.392 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.392 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.392 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.392 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:13.392 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.392 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.393 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.393 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.393 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.393 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.393 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.393 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.393 INFO:teuthology.orchestra.run.smithi116.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T03:19:13.393 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T03:19:13.393 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.393 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.394 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.394 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.394 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.394 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.394 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.394 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.394 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.394 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.394 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.395 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.395 INFO:teuthology.orchestra.run.smithi116.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T03:19:13.395 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T03:19:13.395 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.395 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.395 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.395 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.395 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.395 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.395 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.395 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.395 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.396 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.396 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.396 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.396 INFO:teuthology.orchestra.run.smithi116.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T03:19:13.396 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T03:19:13.396 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.396 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.396 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.396 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:13.396 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.396 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.396 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.397 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.397 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.397 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.397 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.397 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.397 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T03:19:13.397 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T03:19:13.397 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.397 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.397 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.397 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.397 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.397 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.398 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.398 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.398 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.398 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.398 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.398 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.398 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T03:19:13.398 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T03:19:13.398 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.398 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.398 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.398 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.398 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.399 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.399 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.399 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.399 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.399 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.399 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.399 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.399 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T03:19:13.399 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T03:19:13.399 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.399 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.399 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.399 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:13.400 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.400 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.400 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.400 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.400 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.400 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.400 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.400 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.400 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T03:19:13.400 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T03:19:13.400 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.400 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.400 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.401 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:13.401 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.401 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.401 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.401 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.401 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.401 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.401 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.401 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.401 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T03:19:13.401 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T03:19:13.401 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.401 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.402 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.402 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.402 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.402 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.402 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.402 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.402 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.402 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.402 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.402 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.402 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T03:19:13.402 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T03:19:13.402 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.403 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.403 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.403 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.403 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.403 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.403 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.403 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.403 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.403 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.403 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.403 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.403 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T03:19:13.403 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T03:19:13.404 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.404 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.404 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.404 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.404 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.404 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.404 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.404 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.404 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.404 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.404 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.404 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.404 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T03:19:13.405 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T03:19:13.405 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.405 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.405 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.405 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:13.405 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.405 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.405 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.405 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.405 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.405 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.406 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.406 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.406 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T03:19:13.406 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T03:19:13.406 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.406 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.406 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.406 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T03:19:13.406 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.406 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.406 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.407 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.407 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.407 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.407 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.407 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.407 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T03:19:13.407 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T03:19:13.407 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:13.407 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.407 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.407 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "10", 2024-09-12T03:19:13.407 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.407 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.408 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.408 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.408 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.408 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.408 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.408 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.408 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T03:19:13.408 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T03:19:13.408 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:13.408 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.408 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.408 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "8", 2024-09-12T03:19:13.408 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.409 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.409 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.409 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.409 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.409 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.409 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.409 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.409 INFO:teuthology.orchestra.run.smithi116.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T03:19:13.409 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T03:19:13.409 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:13.409 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.409 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.409 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "45", 2024-09-12T03:19:13.410 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.410 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.410 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.410 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.410 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.410 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.410 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.410 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.410 INFO:teuthology.orchestra.run.smithi116.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T03:19:13.410 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T03:19:13.410 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.410 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.410 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.411 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.411 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.411 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.411 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.411 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.411 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.411 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.411 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.411 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.411 INFO:teuthology.orchestra.run.smithi116.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T03:19:13.411 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T03:19:13.411 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.411 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.412 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.412 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "admin", 2024-09-12T03:19:13.412 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.412 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.412 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.412 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.412 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.412 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.412 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.412 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.412 INFO:teuthology.orchestra.run.smithi116.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T03:19:13.412 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T03:19:13.412 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.413 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.413 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.413 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.413 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.413 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.413 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.413 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.413 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.413 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.413 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.413 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.413 INFO:teuthology.orchestra.run.smithi116.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T03:19:13.413 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T03:19:13.414 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.414 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.414 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.414 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:13.414 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.414 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.414 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.414 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.414 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.414 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.414 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.414 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.414 INFO:teuthology.orchestra.run.smithi116.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T03:19:13.415 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T03:19:13.415 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:13.415 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.415 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.415 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "0", 2024-09-12T03:19:13.415 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.415 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.415 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.415 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.415 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.415 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.416 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.416 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.416 INFO:teuthology.orchestra.run.smithi116.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T03:19:13.416 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T03:19:13.416 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:13.416 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.416 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.416 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "10", 2024-09-12T03:19:13.416 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.417 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.417 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.417 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.417 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.417 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.417 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.417 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.417 INFO:teuthology.orchestra.run.smithi116.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T03:19:13.417 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T03:19:13.417 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:13.417 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.417 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.417 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "5", 2024-09-12T03:19:13.417 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.418 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.418 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.418 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.418 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.418 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.418 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.418 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.418 INFO:teuthology.orchestra.run.smithi116.stdout: "cross_origin_url": { 2024-09-12T03:19:13.419 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "cross_origin_url", 2024-09-12T03:19:13.419 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.419 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.419 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.419 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.419 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.419 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.419 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.419 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.419 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.419 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.419 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.419 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.420 INFO:teuthology.orchestra.run.smithi116.stdout: "crt_file": { 2024-09-12T03:19:13.420 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "crt_file", 2024-09-12T03:19:13.420 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.420 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.420 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.420 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.420 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.420 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.421 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.421 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.421 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.421 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.421 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.421 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.421 INFO:teuthology.orchestra.run.smithi116.stdout: "debug": { 2024-09-12T03:19:13.421 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "debug", 2024-09-12T03:19:13.421 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.421 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.421 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.422 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.422 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.422 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.422 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.422 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Enable/disable debug options", 2024-09-12T03:19:13.422 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.422 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.422 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.422 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.422 INFO:teuthology.orchestra.run.smithi116.stdout: "jwt_token_ttl": { 2024-09-12T03:19:13.423 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "jwt_token_ttl", 2024-09-12T03:19:13.423 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:13.423 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.423 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.423 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "28800", 2024-09-12T03:19:13.423 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.423 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.423 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.423 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.423 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.423 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.424 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.424 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.424 INFO:teuthology.orchestra.run.smithi116.stdout: "key_file": { 2024-09-12T03:19:13.424 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "key_file", 2024-09-12T03:19:13.424 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.424 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.424 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.424 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.424 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.424 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.424 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.425 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.425 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.425 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.425 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.425 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.425 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:13.425 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:13.425 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.425 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.425 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.426 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.426 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.426 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.426 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.426 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.426 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.426 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.426 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.427 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.427 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.428 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.428 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.428 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.428 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.428 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.429 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.429 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:13.429 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:13.429 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.429 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.429 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.429 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.429 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.429 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.429 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.429 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.429 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.429 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.430 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.430 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.430 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:13.430 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:13.430 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.430 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.430 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.430 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:13.430 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.430 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.430 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.430 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.431 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.431 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.431 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.431 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.431 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.431 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.431 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.431 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.431 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.431 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.431 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.431 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:13.431 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:13.431 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.432 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.432 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.432 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.432 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.432 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.432 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.432 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.432 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.432 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.432 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.432 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.432 INFO:teuthology.orchestra.run.smithi116.stdout: "motd": { 2024-09-12T03:19:13.432 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "motd", 2024-09-12T03:19:13.433 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.433 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.433 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.433 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.433 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.433 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.433 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.433 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "The message of the day", 2024-09-12T03:19:13.433 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.433 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.433 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.433 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.433 INFO:teuthology.orchestra.run.smithi116.stdout: "redirect_resolve_ip_addr": { 2024-09-12T03:19:13.433 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T03:19:13.434 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.434 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.434 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.434 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.434 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.434 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.434 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.434 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.434 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.434 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.434 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.434 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.434 INFO:teuthology.orchestra.run.smithi116.stdout: "server_addr": { 2024-09-12T03:19:13.434 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "server_addr", 2024-09-12T03:19:13.435 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.435 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.435 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.435 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "::", 2024-09-12T03:19:13.435 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.435 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.435 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.435 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.435 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.435 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.435 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.435 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.435 INFO:teuthology.orchestra.run.smithi116.stdout: "server_port": { 2024-09-12T03:19:13.435 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "server_port", 2024-09-12T03:19:13.436 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:13.436 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.436 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.436 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "8080", 2024-09-12T03:19:13.436 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.436 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.436 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.436 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.436 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.436 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.436 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.436 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.436 INFO:teuthology.orchestra.run.smithi116.stdout: "ssl": { 2024-09-12T03:19:13.437 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ssl", 2024-09-12T03:19:13.437 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.437 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.437 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.437 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:13.437 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.437 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.437 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.437 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.437 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.437 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.437 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.437 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.437 INFO:teuthology.orchestra.run.smithi116.stdout: "ssl_server_port": { 2024-09-12T03:19:13.437 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ssl_server_port", 2024-09-12T03:19:13.438 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:13.438 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.438 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.438 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "8443", 2024-09-12T03:19:13.438 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.438 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.438 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.438 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.438 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.439 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.439 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.439 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.439 INFO:teuthology.orchestra.run.smithi116.stdout: "standby_behaviour": { 2024-09-12T03:19:13.439 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "standby_behaviour", 2024-09-12T03:19:13.439 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.439 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.439 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.439 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "redirect", 2024-09-12T03:19:13.439 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.439 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.440 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.440 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.440 INFO:teuthology.orchestra.run.smithi116.stdout: "redirect" 2024-09-12T03:19:13.440 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.440 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.440 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.440 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.440 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.440 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.440 INFO:teuthology.orchestra.run.smithi116.stdout: "standby_error_status_code": { 2024-09-12T03:19:13.440 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "standby_error_status_code", 2024-09-12T03:19:13.440 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:13.440 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.440 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.441 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "500", 2024-09-12T03:19:13.441 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "400", 2024-09-12T03:19:13.441 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "599", 2024-09-12T03:19:13.441 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.441 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.441 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.441 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.441 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.441 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.441 INFO:teuthology.orchestra.run.smithi116.stdout: "url_prefix": { 2024-09-12T03:19:13.441 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "url_prefix", 2024-09-12T03:19:13.441 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.441 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.442 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.442 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.442 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.442 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.442 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.442 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.442 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.442 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.442 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.442 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.442 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.442 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.442 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:13.442 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "devicehealth", 2024-09-12T03:19:13.443 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:13.443 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:13.443 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:13.443 INFO:teuthology.orchestra.run.smithi116.stdout: "enable_monitoring": { 2024-09-12T03:19:13.443 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "enable_monitoring", 2024-09-12T03:19:13.443 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.443 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.443 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.443 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:13.443 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.443 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.443 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.444 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "monitor device health metrics", 2024-09-12T03:19:13.444 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.444 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.444 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.444 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.444 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:13.444 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:13.444 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.444 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.444 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.444 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.444 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.445 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.445 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.445 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.445 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.445 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.445 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.445 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.445 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.445 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.445 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.445 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.445 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.446 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.446 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.446 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:13.446 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:13.446 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.446 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.446 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.446 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.446 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.446 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.446 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.447 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.447 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.447 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.447 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.447 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.447 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:13.447 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:13.447 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.447 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.447 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.447 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:13.447 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.447 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.448 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.448 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.448 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.448 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.448 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.448 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.448 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.448 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.448 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.448 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.448 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.448 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.449 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.449 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:13.449 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:13.449 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.449 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.449 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.449 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.449 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.449 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.449 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.449 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.450 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.450 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.450 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.450 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.450 INFO:teuthology.orchestra.run.smithi116.stdout: "mark_out_threshold": { 2024-09-12T03:19:13.450 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "mark_out_threshold", 2024-09-12T03:19:13.450 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:13.450 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.450 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.450 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "2419200", 2024-09-12T03:19:13.451 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.451 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.451 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.451 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T03:19:13.451 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.451 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.451 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.451 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.451 INFO:teuthology.orchestra.run.smithi116.stdout: "pool_name": { 2024-09-12T03:19:13.451 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "pool_name", 2024-09-12T03:19:13.451 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.451 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.452 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.452 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "device_health_metrics", 2024-09-12T03:19:13.452 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.452 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.452 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.452 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T03:19:13.452 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.452 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.452 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.452 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.452 INFO:teuthology.orchestra.run.smithi116.stdout: "retention_period": { 2024-09-12T03:19:13.452 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "retention_period", 2024-09-12T03:19:13.453 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:13.453 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.453 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.453 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "15552000", 2024-09-12T03:19:13.453 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.453 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.453 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.453 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "how long to retain device health metrics", 2024-09-12T03:19:13.453 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.453 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.453 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.453 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.453 INFO:teuthology.orchestra.run.smithi116.stdout: "scrape_frequency": { 2024-09-12T03:19:13.454 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "scrape_frequency", 2024-09-12T03:19:13.454 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:13.454 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.454 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.454 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "86400", 2024-09-12T03:19:13.454 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.454 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.454 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.454 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T03:19:13.454 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.454 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.454 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.455 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.455 INFO:teuthology.orchestra.run.smithi116.stdout: "self_heal": { 2024-09-12T03:19:13.455 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "self_heal", 2024-09-12T03:19:13.455 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.455 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.455 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.455 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:13.455 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.455 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.455 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.455 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T03:19:13.455 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.456 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.456 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.456 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.456 INFO:teuthology.orchestra.run.smithi116.stdout: "sleep_interval": { 2024-09-12T03:19:13.456 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "sleep_interval", 2024-09-12T03:19:13.456 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:13.456 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.456 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.456 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "600", 2024-09-12T03:19:13.456 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.456 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.456 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.457 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T03:19:13.457 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.457 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.457 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.457 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.457 INFO:teuthology.orchestra.run.smithi116.stdout: "warn_threshold": { 2024-09-12T03:19:13.457 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "warn_threshold", 2024-09-12T03:19:13.457 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:13.457 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.457 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.457 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "7257600", 2024-09-12T03:19:13.457 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.457 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.457 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.458 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T03:19:13.458 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.458 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.458 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.458 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.458 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.458 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.458 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:13.458 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "diskprediction_local", 2024-09-12T03:19:13.458 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:13.459 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:13.459 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:13.459 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:13.459 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:13.459 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.459 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.459 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.459 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.459 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.459 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.459 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.459 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.460 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.460 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.460 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.460 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.460 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.460 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.460 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.460 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.460 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.460 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.461 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.461 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:13.461 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:13.461 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.461 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.461 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.461 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.461 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.461 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.461 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.461 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.462 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.462 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.462 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.462 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.462 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:13.462 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:13.462 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.462 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.462 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.462 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:13.462 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.462 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.463 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.463 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.463 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.463 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.463 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.463 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.463 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.463 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.463 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.463 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.464 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.464 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.464 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.464 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:13.464 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:13.464 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.464 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.465 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.465 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.465 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.465 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.465 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.465 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.465 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.465 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.465 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.466 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.466 INFO:teuthology.orchestra.run.smithi116.stdout: "predict_interval": { 2024-09-12T03:19:13.466 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "predict_interval", 2024-09-12T03:19:13.466 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.466 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.466 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.466 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "86400", 2024-09-12T03:19:13.466 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.466 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.467 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.467 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.467 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.467 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.467 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.467 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.467 INFO:teuthology.orchestra.run.smithi116.stdout: "predictor_model": { 2024-09-12T03:19:13.467 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "predictor_model", 2024-09-12T03:19:13.467 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.467 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.467 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.467 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "prophetstor", 2024-09-12T03:19:13.467 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.467 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.468 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.468 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.468 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.468 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.468 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.468 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.468 INFO:teuthology.orchestra.run.smithi116.stdout: "sleep_interval": { 2024-09-12T03:19:13.468 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "sleep_interval", 2024-09-12T03:19:13.468 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.468 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.468 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.468 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "600", 2024-09-12T03:19:13.468 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.469 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.469 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.469 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.469 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.469 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.469 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.469 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.469 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.469 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.469 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:13.469 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "influx", 2024-09-12T03:19:13.469 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": false, 2024-09-12T03:19:13.469 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "influxdb python module not found", 2024-09-12T03:19:13.469 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:13.470 INFO:teuthology.orchestra.run.smithi116.stdout: "batch_size": { 2024-09-12T03:19:13.470 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "batch_size", 2024-09-12T03:19:13.470 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:13.470 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.470 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.470 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "5000", 2024-09-12T03:19:13.470 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.470 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.470 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.470 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T03:19:13.470 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.470 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.470 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.470 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.471 INFO:teuthology.orchestra.run.smithi116.stdout: "database": { 2024-09-12T03:19:13.471 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "database", 2024-09-12T03:19:13.471 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.471 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.471 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.471 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "ceph", 2024-09-12T03:19:13.471 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.471 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.471 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.471 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T03:19:13.471 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.471 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.472 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.472 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.472 INFO:teuthology.orchestra.run.smithi116.stdout: "hostname": { 2024-09-12T03:19:13.472 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "hostname", 2024-09-12T03:19:13.472 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.472 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.472 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.472 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.472 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.472 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.472 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.472 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "InfluxDB server hostname", 2024-09-12T03:19:13.472 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.472 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.473 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.473 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.473 INFO:teuthology.orchestra.run.smithi116.stdout: "interval": { 2024-09-12T03:19:13.473 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "interval", 2024-09-12T03:19:13.473 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:13.473 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.473 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.473 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "30", 2024-09-12T03:19:13.473 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "5", 2024-09-12T03:19:13.473 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.473 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.473 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T03:19:13.473 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.474 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.474 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.474 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.474 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:13.474 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:13.474 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.474 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.474 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.474 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.474 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.474 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.474 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.474 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.474 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.475 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.475 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.475 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.475 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.475 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.475 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.475 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.475 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.475 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.475 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.475 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:13.475 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:13.475 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.475 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.476 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.476 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.476 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.476 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.476 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.476 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.476 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.476 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.476 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.476 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.476 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:13.476 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:13.476 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.476 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.477 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.477 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:13.477 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.477 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.477 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.477 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.477 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.477 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.477 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.477 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.477 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.477 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.477 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.478 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.478 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.478 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.478 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.478 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:13.478 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:13.478 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.478 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.478 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.478 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.478 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.478 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.478 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.478 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.479 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.479 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.479 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.479 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.479 INFO:teuthology.orchestra.run.smithi116.stdout: "password": { 2024-09-12T03:19:13.479 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "password", 2024-09-12T03:19:13.479 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.479 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.479 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.479 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.479 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.479 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.479 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.479 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "password of InfluxDB server user", 2024-09-12T03:19:13.480 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.480 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.480 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.480 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.480 INFO:teuthology.orchestra.run.smithi116.stdout: "port": { 2024-09-12T03:19:13.480 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "port", 2024-09-12T03:19:13.480 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:13.480 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.480 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.480 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "8086", 2024-09-12T03:19:13.480 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.480 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.480 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.481 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "InfluxDB server port", 2024-09-12T03:19:13.481 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.481 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.481 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.481 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.481 INFO:teuthology.orchestra.run.smithi116.stdout: "ssl": { 2024-09-12T03:19:13.481 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ssl", 2024-09-12T03:19:13.481 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.481 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.481 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.481 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "false", 2024-09-12T03:19:13.481 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.481 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.482 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.482 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:19:13.482 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.482 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.482 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.482 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.482 INFO:teuthology.orchestra.run.smithi116.stdout: "threads": { 2024-09-12T03:19:13.482 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "threads", 2024-09-12T03:19:13.482 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:13.482 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.482 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.482 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "5", 2024-09-12T03:19:13.482 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "1", 2024-09-12T03:19:13.482 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "32", 2024-09-12T03:19:13.483 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.483 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T03:19:13.483 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.483 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.483 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.483 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.483 INFO:teuthology.orchestra.run.smithi116.stdout: "username": { 2024-09-12T03:19:13.483 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "username", 2024-09-12T03:19:13.483 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.483 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.483 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.483 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.483 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.484 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.484 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.484 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "username of InfluxDB server user", 2024-09-12T03:19:13.484 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.484 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.484 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.484 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.484 INFO:teuthology.orchestra.run.smithi116.stdout: "verify_ssl": { 2024-09-12T03:19:13.484 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "verify_ssl", 2024-09-12T03:19:13.484 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.484 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.484 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.485 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "true", 2024-09-12T03:19:13.485 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.485 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.485 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.485 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:19:13.485 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.485 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.485 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.485 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.485 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.486 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.486 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:13.486 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "insights", 2024-09-12T03:19:13.486 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:13.486 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:13.486 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:13.486 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:13.486 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:13.487 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.487 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.487 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.487 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.487 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.487 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.487 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.487 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.487 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.488 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.488 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.488 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.488 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.488 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.488 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.488 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.488 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.488 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.488 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.489 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:13.489 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:13.489 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.489 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.489 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.489 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.489 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.489 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.489 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.489 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.489 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.490 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.490 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.490 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.490 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:13.490 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:13.490 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.490 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.490 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.491 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:13.491 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.491 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.491 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.491 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.491 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.491 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.491 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.491 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.492 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.492 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.492 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.492 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.492 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.492 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.493 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.493 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:13.493 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:13.493 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.493 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.494 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.494 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.494 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.494 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.495 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.495 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.495 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.495 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.495 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.495 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.495 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.496 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.496 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:13.496 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "iostat", 2024-09-12T03:19:13.496 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:13.496 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:13.497 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:13.497 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:13.497 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:13.497 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.497 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.497 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.497 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.498 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.498 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.498 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.498 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.498 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.499 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.499 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.499 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.499 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.499 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.500 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.500 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.500 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.500 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.500 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.501 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:13.501 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:13.501 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.501 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.501 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.502 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.502 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.502 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.502 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.502 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.502 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.503 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.503 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.503 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.503 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:13.503 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:13.503 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.503 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.503 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.503 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:13.503 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.503 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.503 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.503 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.503 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.504 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.504 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.504 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.504 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.504 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.504 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.504 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.504 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.504 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.504 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.505 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:13.505 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:13.505 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.505 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.505 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.505 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.505 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.505 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.506 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.506 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.506 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.507 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.507 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.507 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.507 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.507 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.507 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:13.507 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "k8sevents", 2024-09-12T03:19:13.508 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:13.508 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:13.508 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:13.508 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_event_retention_days": { 2024-09-12T03:19:13.508 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ceph_event_retention_days", 2024-09-12T03:19:13.508 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:13.509 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.509 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.509 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "7", 2024-09-12T03:19:13.509 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.509 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.509 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.509 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T03:19:13.509 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.509 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.510 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.510 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.510 INFO:teuthology.orchestra.run.smithi116.stdout: "config_check_secs": { 2024-09-12T03:19:13.510 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "config_check_secs", 2024-09-12T03:19:13.510 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:13.510 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.510 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.510 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "10", 2024-09-12T03:19:13.510 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "10", 2024-09-12T03:19:13.510 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.511 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.511 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T03:19:13.511 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.511 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.511 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.511 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.511 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:13.511 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:13.511 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.511 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.511 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.512 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.512 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.512 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.512 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.512 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.512 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.512 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.512 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.512 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.512 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.513 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.513 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.513 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.513 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.513 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.513 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.513 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:13.513 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:13.513 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.513 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.514 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.514 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.514 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.514 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.514 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.514 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.515 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.515 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.515 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.515 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.515 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:13.515 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:13.516 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.516 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.516 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.516 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:13.516 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.517 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.517 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.517 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.517 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.517 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.517 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.518 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.518 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.518 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.518 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.518 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.519 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.519 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.519 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.519 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:13.519 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:13.519 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.519 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.519 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.520 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.520 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.520 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.520 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.520 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.520 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.520 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.521 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.521 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.521 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.521 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.521 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:13.521 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "localpool", 2024-09-12T03:19:13.521 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:13.522 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:13.522 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:13.522 INFO:teuthology.orchestra.run.smithi116.stdout: "failure_domain": { 2024-09-12T03:19:13.522 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "failure_domain", 2024-09-12T03:19:13.522 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.522 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.522 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.523 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "host", 2024-09-12T03:19:13.523 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.523 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.523 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.523 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "failure domain for any created local pool", 2024-09-12T03:19:13.524 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T03:19:13.524 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.524 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.524 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.524 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:13.525 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:13.525 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.525 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.525 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.525 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.526 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.526 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.526 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.526 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.526 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.526 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.526 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.527 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.527 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.527 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.527 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.527 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.527 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.527 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.527 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.527 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:13.528 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:13.528 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.528 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.528 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.529 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.529 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.529 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.529 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.529 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.529 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.529 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.529 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.529 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.530 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:13.530 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:13.530 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.530 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.530 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.530 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:13.530 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.530 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.530 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.530 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.530 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.530 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.531 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.531 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.531 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.531 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.531 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.531 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.531 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.531 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.531 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.531 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:13.531 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:13.532 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.532 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.532 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.532 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.532 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.532 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.532 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.532 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.532 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.532 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.532 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.532 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.533 INFO:teuthology.orchestra.run.smithi116.stdout: "min_size": { 2024-09-12T03:19:13.533 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "min_size", 2024-09-12T03:19:13.533 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:13.533 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.533 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.533 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.533 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.533 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.533 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.533 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "default min_size for any created local pool", 2024-09-12T03:19:13.533 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T03:19:13.533 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.534 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.534 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.534 INFO:teuthology.orchestra.run.smithi116.stdout: "num_rep": { 2024-09-12T03:19:13.534 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "num_rep", 2024-09-12T03:19:13.534 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:13.534 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.534 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.534 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "3", 2024-09-12T03:19:13.534 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.535 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.535 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.535 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "default replica count for any created local pool", 2024-09-12T03:19:13.535 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.535 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.535 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.535 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.536 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num": { 2024-09-12T03:19:13.536 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "pg_num", 2024-09-12T03:19:13.536 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:13.536 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.536 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.536 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "128", 2024-09-12T03:19:13.536 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.536 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.536 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.537 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T03:19:13.537 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.537 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.537 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.537 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.537 INFO:teuthology.orchestra.run.smithi116.stdout: "prefix": { 2024-09-12T03:19:13.537 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "prefix", 2024-09-12T03:19:13.537 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.538 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.538 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.538 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.538 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.538 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.538 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.538 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "name prefix for any created local pool", 2024-09-12T03:19:13.538 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.538 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.538 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.539 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.539 INFO:teuthology.orchestra.run.smithi116.stdout: "subtree": { 2024-09-12T03:19:13.539 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "subtree", 2024-09-12T03:19:13.539 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.539 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.539 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.539 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "rack", 2024-09-12T03:19:13.539 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.540 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.540 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.540 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T03:19:13.540 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T03:19:13.540 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.540 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.540 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.540 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.541 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.541 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:13.541 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "mds_autoscaler", 2024-09-12T03:19:13.541 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:13.541 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:13.541 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:13.541 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:13.542 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:13.542 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.542 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.542 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.542 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.542 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.542 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.542 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.542 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.542 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.542 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.542 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.543 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.543 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.543 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.543 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.543 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.543 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.543 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.543 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.543 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:13.543 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:13.543 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.543 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.544 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.544 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.544 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.544 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.544 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.544 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.544 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.544 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.544 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.544 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.544 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:13.545 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:13.545 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.545 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.545 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.545 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:13.545 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.545 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.545 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.545 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.545 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.546 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.546 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.546 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.546 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.546 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.546 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.546 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.546 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.975 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.976 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.976 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:13.976 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:13.976 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.976 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.976 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.976 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.976 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.976 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.976 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.976 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.977 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.977 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.977 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.977 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.977 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.977 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.977 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:13.977 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "mirroring", 2024-09-12T03:19:13.977 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:13.977 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:13.977 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:13.977 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:13.977 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:13.977 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.978 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.978 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.978 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.978 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.978 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.978 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.978 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.978 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.978 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.978 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.978 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.978 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.978 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.978 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.979 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.979 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.979 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.979 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.979 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:13.979 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:13.979 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.979 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.979 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.979 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.979 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.979 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.980 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.980 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.980 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.980 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.980 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.980 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.980 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:13.980 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:13.980 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.980 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.980 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.980 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:13.980 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.981 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.981 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.981 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.981 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.981 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.981 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.981 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.981 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.981 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.981 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.981 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.981 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.981 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.981 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.982 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:13.982 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:13.982 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.982 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.982 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.982 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.982 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.982 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.982 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.982 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.982 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.982 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.983 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.983 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.983 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.983 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.983 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:13.983 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "nfs", 2024-09-12T03:19:13.983 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:13.983 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:13.983 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:13.983 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:13.983 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:13.983 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.983 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.983 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.984 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.984 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.984 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.984 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.984 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.984 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.984 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.984 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.984 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.984 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.984 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.984 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.984 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.984 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.985 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.985 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.985 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:13.985 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:13.985 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.985 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.985 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.985 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.985 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.985 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.985 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.985 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.985 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.985 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.986 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.986 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.986 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:13.986 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:13.986 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.986 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.986 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.986 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:13.986 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.986 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.986 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.986 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.986 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.986 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.987 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.987 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.987 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.987 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.987 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.987 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.987 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.987 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.987 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.987 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:13.987 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:13.987 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.988 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.988 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.988 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.988 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.988 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.988 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.988 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.988 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.988 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.988 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.988 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.989 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.989 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.989 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:13.989 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "orchestrator", 2024-09-12T03:19:13.989 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:13.989 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:13.989 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:13.989 INFO:teuthology.orchestra.run.smithi116.stdout: "fail_fs": { 2024-09-12T03:19:13.989 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "fail_fs", 2024-09-12T03:19:13.989 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.989 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.989 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:13.989 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.990 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.990 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.990 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.990 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T03:19:13.990 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.990 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.990 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.990 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.990 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:13.990 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:13.990 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.990 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.990 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.990 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.991 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.991 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.991 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.991 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.991 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.991 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.991 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.991 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.991 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.991 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.991 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.991 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.992 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.992 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.992 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.992 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:13.992 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:13.992 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.992 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.992 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.992 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.992 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.992 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.992 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.992 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.993 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.993 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.993 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.993 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.993 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:13.993 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:13.993 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.993 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.993 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.993 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:13.993 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.993 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.993 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.994 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.994 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.994 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.994 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.994 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.994 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.994 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.994 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.994 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.994 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.994 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.994 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.995 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:13.995 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:13.995 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:13.995 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.995 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.995 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:13.995 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.995 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.995 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:13.995 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.995 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.995 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.995 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.996 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.996 INFO:teuthology.orchestra.run.smithi116.stdout: "orchestrator": { 2024-09-12T03:19:13.996 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "orchestrator", 2024-09-12T03:19:13.996 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.996 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.996 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.996 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.996 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.996 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.996 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.996 INFO:teuthology.orchestra.run.smithi116.stdout: "cephadm", 2024-09-12T03:19:13.996 INFO:teuthology.orchestra.run.smithi116.stdout: "rook", 2024-09-12T03:19:13.996 INFO:teuthology.orchestra.run.smithi116.stdout: "test_orchestrator" 2024-09-12T03:19:13.997 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.997 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Orchestrator backend", 2024-09-12T03:19:13.997 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.997 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.997 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.997 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.997 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:13.997 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.997 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:13.997 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "osd_perf_query", 2024-09-12T03:19:13.998 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:13.998 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:13.998 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:13.998 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:13.998 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:13.998 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:13.998 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:13.998 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:13.998 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:13.998 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:13.998 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:13.998 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:13.998 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:13.999 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:13.999 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:13.999 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:13.999 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:13.999 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:13.999 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:13.999 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:13.999 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:13.999 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:13.999 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:13.999 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:13.999 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:13.999 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.000 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.000 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.000 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.000 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.000 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.000 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.000 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.000 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.000 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.000 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.000 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.001 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.001 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.001 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.001 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.001 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.001 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.001 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.001 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.001 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.001 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.001 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.001 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.002 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.002 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.002 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.002 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.002 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.002 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.002 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.002 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.002 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.002 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.002 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.002 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.002 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.003 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.003 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.003 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.003 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.003 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.003 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.003 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.003 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.003 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.003 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.003 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.003 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.004 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.004 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.004 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "osd_support", 2024-09-12T03:19:14.004 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.004 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.004 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.004 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.004 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.004 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.004 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.004 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.004 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.005 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.005 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.005 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.005 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.005 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.005 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.005 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.005 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.005 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.005 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.005 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.005 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.005 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.006 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.006 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.006 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.006 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.006 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.006 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.006 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.006 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.006 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.006 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.006 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.006 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.006 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.007 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.007 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.007 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.007 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.007 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.007 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.007 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.007 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.007 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.007 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.007 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.007 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.007 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.007 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.008 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.008 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.008 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.008 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.008 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.008 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.008 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.008 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.008 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.009 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.009 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.009 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.009 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.009 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.009 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.009 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.009 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.009 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.009 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.009 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.009 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.009 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.010 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.010 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.010 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.010 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.010 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.010 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "pg_autoscaler", 2024-09-12T03:19:14.010 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.010 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.010 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.010 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.011 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.011 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.011 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.011 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.011 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.011 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.011 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.011 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.011 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.011 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.011 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.011 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.011 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.012 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.012 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.012 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.012 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.012 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.012 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.012 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.012 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.012 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.012 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.012 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.012 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.013 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.013 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.013 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.013 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.013 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.013 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.013 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.013 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.013 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.013 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.013 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.013 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.014 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.014 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.014 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.014 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.014 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.014 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.014 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.014 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.014 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.014 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.014 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.014 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.014 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.014 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.015 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.015 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.015 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.015 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.015 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.015 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.015 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.015 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.015 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.015 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.015 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.015 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.015 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.016 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.016 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.016 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.016 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.016 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.016 INFO:teuthology.orchestra.run.smithi116.stdout: "sleep_interval": { 2024-09-12T03:19:14.016 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "sleep_interval", 2024-09-12T03:19:14.016 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.016 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.016 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.016 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "60", 2024-09-12T03:19:14.016 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.016 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.017 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.017 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.017 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.017 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.017 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.017 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.017 INFO:teuthology.orchestra.run.smithi116.stdout: "threshold": { 2024-09-12T03:19:14.017 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "threshold", 2024-09-12T03:19:14.017 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "float", 2024-09-12T03:19:14.017 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.017 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.017 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "3.0", 2024-09-12T03:19:14.017 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "1.0", 2024-09-12T03:19:14.017 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.018 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.018 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "scaling threshold", 2024-09-12T03:19:14.018 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T03:19:14.018 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.018 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.018 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.018 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.018 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.018 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.018 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "progress", 2024-09-12T03:19:14.019 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.019 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.019 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.019 INFO:teuthology.orchestra.run.smithi116.stdout: "allow_pg_recovery_event": { 2024-09-12T03:19:14.019 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "allow_pg_recovery_event", 2024-09-12T03:19:14.019 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.019 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.019 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.019 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.019 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.020 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.020 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.020 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T03:19:14.020 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.020 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.020 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.020 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.020 INFO:teuthology.orchestra.run.smithi116.stdout: "enabled": { 2024-09-12T03:19:14.020 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "enabled", 2024-09-12T03:19:14.021 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.021 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.021 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.021 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.021 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.021 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.021 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.021 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.021 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.021 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.022 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.022 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.022 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.022 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.022 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.022 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.022 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.022 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.022 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.022 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.022 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.023 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.023 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.023 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.023 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.023 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.023 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.023 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.023 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.023 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.024 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.024 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.024 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.024 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.024 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.024 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.024 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.024 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.024 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.024 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.024 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.025 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.025 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.025 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.025 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.025 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.025 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.025 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.025 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.025 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.026 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.026 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.026 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.026 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.026 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.026 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.026 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.026 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.026 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.026 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.026 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.027 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.027 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.027 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.027 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.027 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.027 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.027 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.027 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.027 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.027 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.027 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.027 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.027 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.027 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.027 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.028 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.028 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.028 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.028 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.028 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.028 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.028 INFO:teuthology.orchestra.run.smithi116.stdout: "max_completed_events": { 2024-09-12T03:19:14.028 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "max_completed_events", 2024-09-12T03:19:14.028 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.028 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.028 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.028 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "50", 2024-09-12T03:19:14.028 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.029 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.029 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.029 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "number of past completed events to remember", 2024-09-12T03:19:14.029 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.029 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.029 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.029 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.029 INFO:teuthology.orchestra.run.smithi116.stdout: "sleep_interval": { 2024-09-12T03:19:14.029 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "sleep_interval", 2024-09-12T03:19:14.030 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.030 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.030 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.030 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "5", 2024-09-12T03:19:14.030 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.030 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.030 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.030 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "how long the module is going to sleep", 2024-09-12T03:19:14.030 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.030 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.030 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.030 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.030 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.031 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.031 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.031 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "prometheus", 2024-09-12T03:19:14.031 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.031 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.031 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.031 INFO:teuthology.orchestra.run.smithi116.stdout: "cache": { 2024-09-12T03:19:14.031 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "cache", 2024-09-12T03:19:14.031 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.031 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.031 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.031 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.031 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.032 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.032 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.032 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.032 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.032 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.032 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.032 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.032 INFO:teuthology.orchestra.run.smithi116.stdout: "exclude_perf_counters": { 2024-09-12T03:19:14.032 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "exclude_perf_counters", 2024-09-12T03:19:14.032 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.032 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.032 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.033 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.033 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.033 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.033 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.033 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T03:19:14.033 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T03:19:14.033 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.033 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.033 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.033 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.033 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.033 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.033 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.033 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.034 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.034 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.034 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.034 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.034 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.034 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.034 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.034 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.034 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.034 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.034 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.034 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.034 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.034 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.035 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.035 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.035 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.035 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.035 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.035 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.035 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.035 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.035 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.035 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.035 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.035 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.035 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.036 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.036 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.036 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.036 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.036 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.036 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.036 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.036 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.036 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.036 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.036 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.036 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.036 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.037 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.037 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.037 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.037 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.037 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.037 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.037 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.037 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.037 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.037 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.037 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.037 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.037 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.037 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.038 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.038 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.038 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.038 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.038 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.038 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.038 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.038 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.038 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.038 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.038 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.038 INFO:teuthology.orchestra.run.smithi116.stdout: "rbd_stats_pools": { 2024-09-12T03:19:14.038 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rbd_stats_pools", 2024-09-12T03:19:14.038 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.039 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.039 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.039 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.039 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.039 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.039 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.039 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.039 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.039 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.039 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.039 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.039 INFO:teuthology.orchestra.run.smithi116.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T03:19:14.039 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T03:19:14.039 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.040 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.040 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.040 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "300", 2024-09-12T03:19:14.040 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.040 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.040 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.040 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.040 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.040 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.040 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.040 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.040 INFO:teuthology.orchestra.run.smithi116.stdout: "scrape_interval": { 2024-09-12T03:19:14.040 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "scrape_interval", 2024-09-12T03:19:14.040 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "float", 2024-09-12T03:19:14.041 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.042 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.042 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "15.0", 2024-09-12T03:19:14.042 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.042 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.042 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.043 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.043 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.043 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.043 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.044 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.044 INFO:teuthology.orchestra.run.smithi116.stdout: "server_addr": { 2024-09-12T03:19:14.044 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "server_addr", 2024-09-12T03:19:14.044 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.045 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.045 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.045 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "::", 2024-09-12T03:19:14.045 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.045 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.045 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.046 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T03:19:14.046 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.046 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.046 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.046 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.046 INFO:teuthology.orchestra.run.smithi116.stdout: "server_port": { 2024-09-12T03:19:14.047 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "server_port", 2024-09-12T03:19:14.047 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.047 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.047 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.047 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "9283", 2024-09-12T03:19:14.047 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.047 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.048 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.048 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T03:19:14.048 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.048 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.048 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.049 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.049 INFO:teuthology.orchestra.run.smithi116.stdout: "stale_cache_strategy": { 2024-09-12T03:19:14.049 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "stale_cache_strategy", 2024-09-12T03:19:14.049 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.049 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.050 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.050 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "log", 2024-09-12T03:19:14.050 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.050 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.050 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.050 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.051 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.051 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.051 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.051 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.051 INFO:teuthology.orchestra.run.smithi116.stdout: "standby_behaviour": { 2024-09-12T03:19:14.051 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "standby_behaviour", 2024-09-12T03:19:14.052 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.052 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.052 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.052 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "default", 2024-09-12T03:19:14.052 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.053 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.053 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.053 INFO:teuthology.orchestra.run.smithi116.stdout: "default", 2024-09-12T03:19:14.053 INFO:teuthology.orchestra.run.smithi116.stdout: "error" 2024-09-12T03:19:14.054 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.054 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.054 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.054 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.054 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.054 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.055 INFO:teuthology.orchestra.run.smithi116.stdout: "standby_error_status_code": { 2024-09-12T03:19:14.055 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "standby_error_status_code", 2024-09-12T03:19:14.055 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.055 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.055 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.055 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "500", 2024-09-12T03:19:14.056 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "400", 2024-09-12T03:19:14.056 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "599", 2024-09-12T03:19:14.056 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.056 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.056 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.056 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.056 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.057 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.057 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.057 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.057 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.057 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rbd_support", 2024-09-12T03:19:14.057 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.057 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.058 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.058 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.058 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.058 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.058 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.058 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.059 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.059 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.059 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.059 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.060 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.060 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.060 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.060 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.060 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.060 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.061 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.061 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.061 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.061 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.061 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.061 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.062 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.062 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.062 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.062 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.062 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.062 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.063 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.063 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.063 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.063 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.063 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.063 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.064 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.064 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.064 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.064 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.064 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.064 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.065 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.065 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.065 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.065 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.065 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.065 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.066 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.066 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.066 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.066 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.066 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.066 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.067 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.067 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.067 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.067 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.067 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.068 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.068 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.068 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.069 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.069 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.069 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.069 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.069 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.070 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.070 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.070 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.070 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.070 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.071 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.071 INFO:teuthology.orchestra.run.smithi116.stdout: "max_concurrent_snap_create": { 2024-09-12T03:19:14.071 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "max_concurrent_snap_create", 2024-09-12T03:19:14.071 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.071 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.072 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.072 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "10", 2024-09-12T03:19:14.072 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.072 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.072 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.072 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.073 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.073 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.073 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.073 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.073 INFO:teuthology.orchestra.run.smithi116.stdout: "mirror_snapshot_schedule": { 2024-09-12T03:19:14.073 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T03:19:14.074 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.074 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.074 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.074 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.074 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.074 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.075 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.075 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.075 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.075 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.075 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.075 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.076 INFO:teuthology.orchestra.run.smithi116.stdout: "trash_purge_schedule": { 2024-09-12T03:19:14.076 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "trash_purge_schedule", 2024-09-12T03:19:14.076 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.076 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.076 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.077 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.077 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.077 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.077 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.077 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.077 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.078 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.078 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.078 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.078 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.078 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.078 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.079 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "restful", 2024-09-12T03:19:14.079 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.079 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.079 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.079 INFO:teuthology.orchestra.run.smithi116.stdout: "enable_auth": { 2024-09-12T03:19:14.080 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "enable_auth", 2024-09-12T03:19:14.080 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.080 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.080 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.080 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.080 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.081 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.081 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.081 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.081 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.081 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.081 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.082 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.082 INFO:teuthology.orchestra.run.smithi116.stdout: "key_file": { 2024-09-12T03:19:14.082 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "key_file", 2024-09-12T03:19:14.082 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.082 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.082 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.083 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.083 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.083 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.083 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.083 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.083 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.083 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.083 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.084 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.084 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.084 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.084 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.084 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.084 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.084 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.084 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.084 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.084 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.084 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.084 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.084 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.085 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.085 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.085 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.085 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.085 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.085 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.085 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.085 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.085 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.085 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.086 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.086 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.086 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.086 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.086 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.086 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.086 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.086 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.086 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.086 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.086 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.086 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.086 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.087 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.087 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.087 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.087 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.087 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.087 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.087 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.087 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.087 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.087 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.087 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.087 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.087 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.087 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.088 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.088 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.088 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.088 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.088 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.088 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.088 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.088 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.088 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.088 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.088 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.088 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.088 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.089 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.089 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.089 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.089 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.089 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.089 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.089 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.089 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.089 INFO:teuthology.orchestra.run.smithi116.stdout: "server_addr": { 2024-09-12T03:19:14.089 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "server_addr", 2024-09-12T03:19:14.089 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.089 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.089 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.090 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.090 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.090 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.090 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.090 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.090 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.090 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.090 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.090 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.090 INFO:teuthology.orchestra.run.smithi116.stdout: "server_port": { 2024-09-12T03:19:14.090 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "server_port", 2024-09-12T03:19:14.090 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.090 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.091 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.091 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.091 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.091 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.091 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.091 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.091 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.091 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.091 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.091 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.091 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.091 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.091 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.091 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rgw", 2024-09-12T03:19:14.092 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.092 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.092 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.092 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.092 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.092 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.092 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.092 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.092 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.092 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.092 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.092 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.092 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.093 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.093 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.093 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.093 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.093 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.093 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.093 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.093 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.093 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.093 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.093 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.093 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.093 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.093 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.094 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.094 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.094 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.094 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.094 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.094 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.094 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.094 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.094 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.094 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.094 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.094 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.094 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.094 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.095 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.095 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.095 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.095 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.095 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.095 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.095 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.095 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.095 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.095 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.095 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.095 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.095 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.095 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.096 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.096 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.096 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.096 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.096 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.096 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.096 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.096 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.096 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.096 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.096 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.096 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.096 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.096 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.097 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.097 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.097 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.097 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.097 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.097 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.097 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.097 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rook", 2024-09-12T03:19:14.097 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.097 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.097 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.097 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.097 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.098 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.098 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.098 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.098 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.098 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.098 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.098 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.098 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.098 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.098 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.098 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.098 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.098 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.099 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.099 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.099 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.099 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.099 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.099 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.099 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.099 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.099 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.099 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.099 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.099 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.100 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.100 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.100 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.100 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.100 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.100 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.100 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.100 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.100 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.100 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.100 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.100 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.100 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.100 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.101 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.101 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.101 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.101 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.101 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.101 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.101 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.101 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.101 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.101 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.101 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.101 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.101 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.102 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.102 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.102 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.102 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.102 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.102 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.102 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.102 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.102 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.102 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.102 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.102 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.102 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.103 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.103 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.103 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.103 INFO:teuthology.orchestra.run.smithi116.stdout: "storage_class": { 2024-09-12T03:19:14.103 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "storage_class", 2024-09-12T03:19:14.103 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.103 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.103 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.103 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "local", 2024-09-12T03:19:14.103 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.103 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.103 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.103 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T03:19:14.104 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.104 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.104 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.104 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.104 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.104 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.104 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.104 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "selftest", 2024-09-12T03:19:14.104 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.104 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.104 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.104 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.104 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.104 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.105 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.105 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.105 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.105 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.105 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.105 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.105 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.105 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.105 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.105 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.105 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.105 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.105 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.106 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.106 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.106 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.106 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.106 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.106 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.106 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.106 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.106 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.106 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.106 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.106 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.106 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.106 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.107 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.107 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.107 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.107 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.107 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.107 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.107 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.107 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.107 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.107 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.107 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.107 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.107 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.107 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.108 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.108 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.108 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.108 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.108 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.108 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.108 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.108 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.108 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.108 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.108 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.108 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.108 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.109 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.109 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.109 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.109 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.109 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.109 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.109 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.109 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.109 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.109 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.109 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.109 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.109 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.110 INFO:teuthology.orchestra.run.smithi116.stdout: "roption1": { 2024-09-12T03:19:14.110 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "roption1", 2024-09-12T03:19:14.110 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.110 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.110 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.110 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.110 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.110 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.110 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.110 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.110 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.110 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.110 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.110 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.111 INFO:teuthology.orchestra.run.smithi116.stdout: "roption2": { 2024-09-12T03:19:14.111 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "roption2", 2024-09-12T03:19:14.111 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.111 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.111 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.111 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "xyz", 2024-09-12T03:19:14.111 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.111 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.111 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.111 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.111 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.111 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.111 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.111 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.112 INFO:teuthology.orchestra.run.smithi116.stdout: "rwoption1": { 2024-09-12T03:19:14.112 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rwoption1", 2024-09-12T03:19:14.112 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.112 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.112 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.112 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.112 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.112 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.112 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.112 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.112 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.112 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.112 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.113 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.113 INFO:teuthology.orchestra.run.smithi116.stdout: "rwoption2": { 2024-09-12T03:19:14.113 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rwoption2", 2024-09-12T03:19:14.113 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.113 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.113 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.113 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.113 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.113 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.113 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.113 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.113 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.114 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.114 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.114 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.114 INFO:teuthology.orchestra.run.smithi116.stdout: "rwoption3": { 2024-09-12T03:19:14.114 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rwoption3", 2024-09-12T03:19:14.114 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "float", 2024-09-12T03:19:14.114 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.114 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.114 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.114 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.114 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.114 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.114 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.115 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.115 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.115 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.115 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.115 INFO:teuthology.orchestra.run.smithi116.stdout: "rwoption4": { 2024-09-12T03:19:14.115 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rwoption4", 2024-09-12T03:19:14.115 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.115 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.115 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.115 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.115 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.115 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.115 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.115 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.116 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.116 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.116 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.116 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.116 INFO:teuthology.orchestra.run.smithi116.stdout: "rwoption5": { 2024-09-12T03:19:14.116 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rwoption5", 2024-09-12T03:19:14.116 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.116 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.116 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.116 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.116 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.116 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.116 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.117 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.117 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.117 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.117 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.117 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.117 INFO:teuthology.orchestra.run.smithi116.stdout: "rwoption6": { 2024-09-12T03:19:14.117 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rwoption6", 2024-09-12T03:19:14.117 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.117 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.117 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.117 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.117 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.117 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.118 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.118 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.118 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.118 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.118 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.118 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.118 INFO:teuthology.orchestra.run.smithi116.stdout: "rwoption7": { 2024-09-12T03:19:14.118 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rwoption7", 2024-09-12T03:19:14.118 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.118 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.118 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.118 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.118 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "1", 2024-09-12T03:19:14.119 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "42", 2024-09-12T03:19:14.119 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.119 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.119 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.119 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.119 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.119 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.119 INFO:teuthology.orchestra.run.smithi116.stdout: "testkey": { 2024-09-12T03:19:14.119 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "testkey", 2024-09-12T03:19:14.119 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.119 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.119 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.119 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.120 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.120 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.120 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.120 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.120 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.120 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.120 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.120 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.120 INFO:teuthology.orchestra.run.smithi116.stdout: "testlkey": { 2024-09-12T03:19:14.120 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "testlkey", 2024-09-12T03:19:14.120 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.120 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.120 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.121 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.121 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.121 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.121 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.121 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.121 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.121 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.121 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.121 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.121 INFO:teuthology.orchestra.run.smithi116.stdout: "testnewline": { 2024-09-12T03:19:14.121 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "testnewline", 2024-09-12T03:19:14.121 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.121 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.122 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.122 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.122 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.122 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.122 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.122 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.122 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.122 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.122 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.122 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.122 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.122 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.122 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.122 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "snap_schedule", 2024-09-12T03:19:14.123 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.123 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.123 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.123 INFO:teuthology.orchestra.run.smithi116.stdout: "allow_m_granularity": { 2024-09-12T03:19:14.123 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "allow_m_granularity", 2024-09-12T03:19:14.123 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.123 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.123 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.123 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.123 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.123 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.123 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.123 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T03:19:14.124 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.124 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.124 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.124 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.124 INFO:teuthology.orchestra.run.smithi116.stdout: "dump_on_update": { 2024-09-12T03:19:14.124 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "dump_on_update", 2024-09-12T03:19:14.124 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.124 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.124 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.124 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.124 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.124 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.124 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.125 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "dump database to debug log on update", 2024-09-12T03:19:14.125 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.125 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.125 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.125 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.125 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.125 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.125 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.125 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.125 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.125 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.126 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.126 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.126 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.126 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.126 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.126 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.126 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.126 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.126 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.126 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.126 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.126 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.126 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.127 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.127 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.127 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.127 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.127 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.127 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.127 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.127 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.127 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.127 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.127 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.127 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.127 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.127 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.128 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.128 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.128 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.128 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.128 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.128 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.128 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.128 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.128 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.128 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.128 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.128 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.129 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.129 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.129 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.129 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.129 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.129 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.129 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.129 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.129 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.129 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.129 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.129 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.129 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.130 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.130 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.130 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.130 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.130 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.130 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.130 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.130 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.130 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.130 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.130 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.130 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.130 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.130 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.131 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.131 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "stats", 2024-09-12T03:19:14.131 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.131 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.131 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.131 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.131 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.131 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.131 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.131 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.131 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.131 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.131 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.132 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.132 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.132 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.132 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.132 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.132 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.132 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.132 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.132 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.132 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.132 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.132 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.132 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.132 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.133 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.133 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.133 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.133 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.133 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.133 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.133 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.133 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.133 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.133 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.133 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.133 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.133 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.134 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.134 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.134 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.134 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.134 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.134 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.134 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.134 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.134 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.134 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.134 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.134 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.134 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.134 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.135 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.135 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.135 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.135 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.135 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.135 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.135 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.135 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.135 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.135 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.135 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.135 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.135 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.136 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.136 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.136 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.136 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.136 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.136 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.136 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.136 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.136 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.136 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.136 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.136 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "status", 2024-09-12T03:19:14.136 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.136 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.137 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.137 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.137 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.137 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.137 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.137 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.137 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.137 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.137 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.137 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.137 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.137 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.137 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.138 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.138 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.138 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.138 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.138 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.138 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.138 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.138 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.138 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.138 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.138 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.138 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.138 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.138 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.139 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.139 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.139 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.139 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.139 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.139 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.139 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.139 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.139 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.139 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.139 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.140 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.140 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.140 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.140 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.140 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.141 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.141 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.141 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.141 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.141 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.141 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.141 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.141 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.141 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.141 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.141 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.141 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.141 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.142 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.142 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.142 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.142 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.142 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.142 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.142 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.142 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.142 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.142 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.142 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.142 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.142 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.142 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.143 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.143 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.143 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.143 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.143 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "telegraf", 2024-09-12T03:19:14.143 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.143 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.143 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.143 INFO:teuthology.orchestra.run.smithi116.stdout: "address": { 2024-09-12T03:19:14.143 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "address", 2024-09-12T03:19:14.143 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.143 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.143 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.144 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T03:19:14.144 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.144 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.144 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.144 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.144 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.144 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.145 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.145 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.145 INFO:teuthology.orchestra.run.smithi116.stdout: "interval": { 2024-09-12T03:19:14.145 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "interval", 2024-09-12T03:19:14.145 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.145 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.145 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.145 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "15", 2024-09-12T03:19:14.145 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.145 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.145 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.145 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.145 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.145 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.146 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.146 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.146 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.146 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.146 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.146 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.146 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.146 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.146 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.146 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.146 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.146 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.146 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.146 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.147 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.147 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.147 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.147 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.147 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.147 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.147 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.147 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.147 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.147 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.147 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.147 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.148 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.148 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.148 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.148 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.148 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.148 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.148 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.148 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.148 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.148 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.148 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.148 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.148 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.149 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.149 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.149 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.149 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.149 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.149 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.149 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.149 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.149 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.149 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.149 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.149 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.149 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.149 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.150 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.150 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.150 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.150 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.150 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.150 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.150 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.150 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.150 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.150 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.150 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.150 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.150 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.151 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.151 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.151 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.151 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.151 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.151 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.151 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.151 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.151 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.151 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "telemetry", 2024-09-12T03:19:14.151 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.151 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.151 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.151 INFO:teuthology.orchestra.run.smithi116.stdout: "channel_basic": { 2024-09-12T03:19:14.152 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "channel_basic", 2024-09-12T03:19:14.152 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.152 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.152 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.152 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.152 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.152 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.152 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.152 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T03:19:14.152 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.152 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.152 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.152 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.153 INFO:teuthology.orchestra.run.smithi116.stdout: "channel_crash": { 2024-09-12T03:19:14.153 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "channel_crash", 2024-09-12T03:19:14.153 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.153 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.153 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.153 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.153 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.153 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.153 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.153 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T03:19:14.153 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.153 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.153 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.154 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.154 INFO:teuthology.orchestra.run.smithi116.stdout: "channel_device": { 2024-09-12T03:19:14.154 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "channel_device", 2024-09-12T03:19:14.154 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.154 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.154 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.154 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.154 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.154 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.154 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.154 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T03:19:14.154 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.154 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.155 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.155 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.155 INFO:teuthology.orchestra.run.smithi116.stdout: "channel_ident": { 2024-09-12T03:19:14.155 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "channel_ident", 2024-09-12T03:19:14.155 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.155 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.155 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.155 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.155 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.155 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.155 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.155 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T03:19:14.155 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.155 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.156 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.156 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.156 INFO:teuthology.orchestra.run.smithi116.stdout: "channel_perf": { 2024-09-12T03:19:14.156 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "channel_perf", 2024-09-12T03:19:14.156 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.156 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.156 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.156 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.156 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.156 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.156 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.156 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T03:19:14.156 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.157 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.157 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.157 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.157 INFO:teuthology.orchestra.run.smithi116.stdout: "contact": { 2024-09-12T03:19:14.157 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "contact", 2024-09-12T03:19:14.157 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.157 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.157 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.157 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.157 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.157 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.157 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.157 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.157 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.158 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.158 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.158 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.158 INFO:teuthology.orchestra.run.smithi116.stdout: "description": { 2024-09-12T03:19:14.158 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "description", 2024-09-12T03:19:14.158 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.158 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.158 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.158 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.158 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.158 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.158 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.159 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.159 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.159 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.159 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.159 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.159 INFO:teuthology.orchestra.run.smithi116.stdout: "device_url": { 2024-09-12T03:19:14.159 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "device_url", 2024-09-12T03:19:14.159 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.159 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.159 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.159 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T03:19:14.159 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.159 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.160 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.160 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.160 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.160 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.160 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.160 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.160 INFO:teuthology.orchestra.run.smithi116.stdout: "enabled": { 2024-09-12T03:19:14.160 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "enabled", 2024-09-12T03:19:14.160 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.160 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.160 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.160 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.160 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.161 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.161 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.161 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.161 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.161 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.161 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.161 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.161 INFO:teuthology.orchestra.run.smithi116.stdout: "interval": { 2024-09-12T03:19:14.161 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "interval", 2024-09-12T03:19:14.161 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.161 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.161 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.161 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "24", 2024-09-12T03:19:14.162 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "8", 2024-09-12T03:19:14.162 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.162 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.162 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.162 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.162 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.162 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.162 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.162 INFO:teuthology.orchestra.run.smithi116.stdout: "last_opt_revision": { 2024-09-12T03:19:14.162 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "last_opt_revision", 2024-09-12T03:19:14.162 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.162 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.162 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.163 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "1", 2024-09-12T03:19:14.163 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.163 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.163 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.163 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.163 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.163 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.163 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.163 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.163 INFO:teuthology.orchestra.run.smithi116.stdout: "leaderboard": { 2024-09-12T03:19:14.163 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "leaderboard", 2024-09-12T03:19:14.163 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.163 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.164 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.164 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.164 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.164 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.164 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.164 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.164 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.164 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.164 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.164 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.164 INFO:teuthology.orchestra.run.smithi116.stdout: "leaderboard_description": { 2024-09-12T03:19:14.164 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "leaderboard_description", 2024-09-12T03:19:14.164 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.165 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.165 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.165 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.165 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.165 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.165 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.165 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.165 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.165 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.165 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.165 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.165 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.165 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.166 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.166 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.166 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.166 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.166 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.166 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.166 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.166 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.166 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.166 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.166 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.166 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.166 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.166 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.167 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.167 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.167 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.167 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.167 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.167 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.167 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.167 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.167 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.167 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.167 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.167 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.167 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.167 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.168 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.168 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.168 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.168 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.168 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.168 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.168 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.168 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.168 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.169 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.169 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.169 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.169 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.169 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.169 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.169 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.169 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.169 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.169 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.169 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.169 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.169 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.170 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.170 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.170 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.170 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.170 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.170 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.170 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.170 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.170 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.170 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.170 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.170 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.170 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.171 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.171 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.171 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.171 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.171 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.171 INFO:teuthology.orchestra.run.smithi116.stdout: "organization": { 2024-09-12T03:19:14.171 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "organization", 2024-09-12T03:19:14.171 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.171 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.171 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.171 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.171 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.171 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.171 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.172 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.172 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.172 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.172 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.172 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.172 INFO:teuthology.orchestra.run.smithi116.stdout: "proxy": { 2024-09-12T03:19:14.172 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "proxy", 2024-09-12T03:19:14.172 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.172 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.172 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.172 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.172 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.172 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.173 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.173 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.173 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.173 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.173 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.173 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.173 INFO:teuthology.orchestra.run.smithi116.stdout: "url": { 2024-09-12T03:19:14.173 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "url", 2024-09-12T03:19:14.173 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.173 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.173 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.173 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T03:19:14.174 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.174 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.174 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.174 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.174 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.174 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.174 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.174 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.174 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.174 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.174 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.174 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "test_orchestrator", 2024-09-12T03:19:14.174 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.174 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.175 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.175 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.175 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.175 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.175 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.175 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.175 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.175 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.175 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.175 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.175 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.175 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.175 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.176 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.176 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.176 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.176 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.176 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.176 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.176 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.176 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.176 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.176 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.176 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.176 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.176 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.177 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.177 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.177 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.177 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.177 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.177 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.177 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.177 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.177 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.177 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.177 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.177 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.177 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.178 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.178 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.178 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.178 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.178 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.178 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.178 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.178 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.178 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.178 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.178 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.178 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.178 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.178 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.179 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.179 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.179 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.179 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.179 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.179 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.179 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.179 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.179 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.179 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.179 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.179 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.179 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.180 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.180 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.180 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.180 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.180 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.180 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.180 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.180 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.180 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "volumes", 2024-09-12T03:19:14.180 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.180 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.180 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.180 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.181 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.181 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.181 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.181 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.181 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.181 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.181 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.181 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.181 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.181 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.181 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.181 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.181 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.181 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.182 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.182 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.182 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.182 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.182 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.182 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.182 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.182 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.182 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.182 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.182 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.182 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.182 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.183 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.183 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.183 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.183 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.183 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.183 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.183 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.183 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.183 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.183 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.183 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.183 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.183 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.184 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.184 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.184 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.184 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.184 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.184 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.184 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.184 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.184 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.184 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.184 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.184 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.184 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.184 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.185 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.185 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.185 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.185 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.185 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.185 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.185 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.185 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.185 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.185 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.185 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.185 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.185 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.186 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.186 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.186 INFO:teuthology.orchestra.run.smithi116.stdout: "max_concurrent_clones": { 2024-09-12T03:19:14.186 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "max_concurrent_clones", 2024-09-12T03:19:14.186 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.186 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.186 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.186 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "4", 2024-09-12T03:19:14.186 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.186 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.186 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.186 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T03:19:14.186 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.187 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.187 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.187 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.187 INFO:teuthology.orchestra.run.smithi116.stdout: "periodic_async_work": { 2024-09-12T03:19:14.187 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "periodic_async_work", 2024-09-12T03:19:14.187 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.187 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.187 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.187 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.187 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.187 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.187 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.187 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Periodically check for async work", 2024-09-12T03:19:14.187 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.188 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.188 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.188 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.188 INFO:teuthology.orchestra.run.smithi116.stdout: "snapshot_clone_delay": { 2024-09-12T03:19:14.188 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "snapshot_clone_delay", 2024-09-12T03:19:14.188 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.188 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.188 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.188 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "0", 2024-09-12T03:19:14.188 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.188 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.188 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.189 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T03:19:14.189 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.189 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.189 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.189 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.189 INFO:teuthology.orchestra.run.smithi116.stdout: "snapshot_clone_no_wait": { 2024-09-12T03:19:14.189 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T03:19:14.189 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.189 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.189 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.189 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.189 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.189 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.189 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.190 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T03:19:14.190 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.190 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.190 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.190 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.190 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.190 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.190 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.190 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "zabbix", 2024-09-12T03:19:14.190 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.190 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.190 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.190 INFO:teuthology.orchestra.run.smithi116.stdout: "discovery_interval": { 2024-09-12T03:19:14.190 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "discovery_interval", 2024-09-12T03:19:14.191 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "uint", 2024-09-12T03:19:14.191 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.191 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.191 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "100", 2024-09-12T03:19:14.191 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.191 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.191 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.191 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.191 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.191 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.191 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.191 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.191 INFO:teuthology.orchestra.run.smithi116.stdout: "identifier": { 2024-09-12T03:19:14.192 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "identifier", 2024-09-12T03:19:14.192 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.192 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.192 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.192 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.192 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.192 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.192 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.192 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.192 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.192 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.192 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.192 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.192 INFO:teuthology.orchestra.run.smithi116.stdout: "interval": { 2024-09-12T03:19:14.193 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "interval", 2024-09-12T03:19:14.193 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.193 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.193 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.193 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "60", 2024-09-12T03:19:14.193 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.193 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.193 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.193 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.193 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.193 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.193 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.193 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.193 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.194 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.194 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.194 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.194 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.194 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.194 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.194 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.194 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.194 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.194 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.194 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.194 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.194 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.195 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.195 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.195 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.195 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.195 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.195 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.195 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.195 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.195 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.195 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.195 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.195 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.195 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.196 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.196 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.196 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.196 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.196 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.196 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.196 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.196 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.196 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.196 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.196 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.196 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.196 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.196 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.197 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.197 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.197 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.197 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.197 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.197 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.197 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.197 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.197 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.197 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.197 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.197 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.197 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.198 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.198 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.198 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.198 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.198 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.198 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.198 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.198 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.198 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.198 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.198 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.198 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.198 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.199 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.199 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.199 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.199 INFO:teuthology.orchestra.run.smithi116.stdout: "zabbix_host": { 2024-09-12T03:19:14.199 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "zabbix_host", 2024-09-12T03:19:14.199 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.199 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.199 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.199 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.199 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.199 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.199 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.199 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.199 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.200 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.200 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.200 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.200 INFO:teuthology.orchestra.run.smithi116.stdout: "zabbix_port": { 2024-09-12T03:19:14.200 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "zabbix_port", 2024-09-12T03:19:14.200 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.200 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.200 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.200 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "10051", 2024-09-12T03:19:14.200 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.200 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.200 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.200 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.200 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.201 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.201 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.201 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.201 INFO:teuthology.orchestra.run.smithi116.stdout: "zabbix_sender": { 2024-09-12T03:19:14.201 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "zabbix_sender", 2024-09-12T03:19:14.201 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.201 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.201 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.201 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T03:19:14.201 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.201 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.201 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.201 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.202 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.202 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.202 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.202 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.202 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.202 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.202 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2024-09-12T03:19:14.202 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.202 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.202 INFO:teuthology.orchestra.run.smithi116.stdout: "gid": 24122, 2024-09-12T03:19:14.202 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "y", 2024-09-12T03:19:14.202 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr_features": 4540701547738038271, 2024-09-12T03:19:14.202 INFO:teuthology.orchestra.run.smithi116.stdout: "available_modules": [ 2024-09-12T03:19:14.203 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.203 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "alerts", 2024-09-12T03:19:14.203 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.203 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.203 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.203 INFO:teuthology.orchestra.run.smithi116.stdout: "interval": { 2024-09-12T03:19:14.203 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "interval", 2024-09-12T03:19:14.203 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.203 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.203 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.203 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "60", 2024-09-12T03:19:14.203 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.203 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.204 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.204 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T03:19:14.204 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.204 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.204 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.204 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.204 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.204 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.204 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.204 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.204 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.204 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.205 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.205 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.205 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.205 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.205 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.205 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.205 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.205 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.205 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.205 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.205 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.205 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.205 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.205 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.206 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.206 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.206 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.206 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.206 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.206 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.206 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.206 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.206 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.206 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.206 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.206 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.206 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.206 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.207 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.207 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.207 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.207 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.207 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.207 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.207 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.207 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.207 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.207 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.207 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.207 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.207 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.207 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.208 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.208 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.208 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.208 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.208 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.208 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.208 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.208 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.208 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.208 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.208 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.208 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.208 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.209 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.209 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.209 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.209 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.209 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.209 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.209 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.209 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.209 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.209 INFO:teuthology.orchestra.run.smithi116.stdout: "smtp_destination": { 2024-09-12T03:19:14.209 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "smtp_destination", 2024-09-12T03:19:14.209 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.209 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.209 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.210 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.210 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.210 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.210 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.210 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Email address to send alerts to", 2024-09-12T03:19:14.210 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.210 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.210 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.210 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.210 INFO:teuthology.orchestra.run.smithi116.stdout: "smtp_from_name": { 2024-09-12T03:19:14.210 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "smtp_from_name", 2024-09-12T03:19:14.210 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.210 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.211 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.211 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "Ceph", 2024-09-12T03:19:14.211 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.211 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.211 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.211 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Email From: name", 2024-09-12T03:19:14.211 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.211 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.211 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.211 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.211 INFO:teuthology.orchestra.run.smithi116.stdout: "smtp_host": { 2024-09-12T03:19:14.211 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "smtp_host", 2024-09-12T03:19:14.211 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.211 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.212 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.212 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.212 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.212 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.212 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.212 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "SMTP server", 2024-09-12T03:19:14.212 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.212 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.212 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.212 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.212 INFO:teuthology.orchestra.run.smithi116.stdout: "smtp_password": { 2024-09-12T03:19:14.212 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "smtp_password", 2024-09-12T03:19:14.212 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.212 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.213 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.213 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.213 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.213 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.213 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.213 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Password to authenticate with", 2024-09-12T03:19:14.213 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.213 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.213 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.213 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.213 INFO:teuthology.orchestra.run.smithi116.stdout: "smtp_port": { 2024-09-12T03:19:14.213 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "smtp_port", 2024-09-12T03:19:14.213 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.213 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.214 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.214 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "465", 2024-09-12T03:19:14.214 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.214 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.214 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.214 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "SMTP port", 2024-09-12T03:19:14.214 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.214 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.214 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.214 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.214 INFO:teuthology.orchestra.run.smithi116.stdout: "smtp_sender": { 2024-09-12T03:19:14.214 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "smtp_sender", 2024-09-12T03:19:14.214 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.215 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.215 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.215 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.215 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.215 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.215 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.215 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "SMTP envelope sender", 2024-09-12T03:19:14.215 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.215 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.215 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.215 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.215 INFO:teuthology.orchestra.run.smithi116.stdout: "smtp_ssl": { 2024-09-12T03:19:14.215 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "smtp_ssl", 2024-09-12T03:19:14.215 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.216 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.216 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.216 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.216 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.216 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.216 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.216 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T03:19:14.216 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.216 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.216 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.216 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.216 INFO:teuthology.orchestra.run.smithi116.stdout: "smtp_user": { 2024-09-12T03:19:14.216 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "smtp_user", 2024-09-12T03:19:14.217 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.217 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.217 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.217 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.217 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.217 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.217 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.217 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "User to authenticate as", 2024-09-12T03:19:14.217 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.217 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.217 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.217 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.217 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.218 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.218 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.218 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "balancer", 2024-09-12T03:19:14.218 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.218 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.218 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.218 INFO:teuthology.orchestra.run.smithi116.stdout: "active": { 2024-09-12T03:19:14.218 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "active", 2024-09-12T03:19:14.218 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.218 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.218 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.218 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.218 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.218 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.219 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.219 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T03:19:14.219 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.219 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.219 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.219 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.219 INFO:teuthology.orchestra.run.smithi116.stdout: "begin_time": { 2024-09-12T03:19:14.219 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "begin_time", 2024-09-12T03:19:14.219 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.219 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.219 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.219 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "0000", 2024-09-12T03:19:14.219 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.220 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.220 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.220 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T03:19:14.220 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:19:14.220 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.220 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.220 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.220 INFO:teuthology.orchestra.run.smithi116.stdout: "begin_weekday": { 2024-09-12T03:19:14.220 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "begin_weekday", 2024-09-12T03:19:14.220 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "uint", 2024-09-12T03:19:14.220 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.220 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.220 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "0", 2024-09-12T03:19:14.221 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "0", 2024-09-12T03:19:14.221 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "6", 2024-09-12T03:19:14.221 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.221 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T03:19:14.221 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:19:14.221 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.221 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.221 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.221 INFO:teuthology.orchestra.run.smithi116.stdout: "crush_compat_max_iterations": { 2024-09-12T03:19:14.221 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "crush_compat_max_iterations", 2024-09-12T03:19:14.221 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "uint", 2024-09-12T03:19:14.221 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.221 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.222 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "25", 2024-09-12T03:19:14.222 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "1", 2024-09-12T03:19:14.222 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "250", 2024-09-12T03:19:14.222 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.222 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T03:19:14.222 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.222 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.222 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.222 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.222 INFO:teuthology.orchestra.run.smithi116.stdout: "crush_compat_metrics": { 2024-09-12T03:19:14.222 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "crush_compat_metrics", 2024-09-12T03:19:14.222 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.222 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.222 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.223 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T03:19:14.223 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.223 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.223 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.223 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T03:19:14.223 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T03:19:14.223 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.223 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.223 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.223 INFO:teuthology.orchestra.run.smithi116.stdout: "crush_compat_step": { 2024-09-12T03:19:14.223 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "crush_compat_step", 2024-09-12T03:19:14.223 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "float", 2024-09-12T03:19:14.223 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.223 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.224 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "0.5", 2024-09-12T03:19:14.224 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "0.001", 2024-09-12T03:19:14.224 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "0.999", 2024-09-12T03:19:14.224 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.224 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "aggressiveness of optimization", 2024-09-12T03:19:14.224 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T03:19:14.224 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.224 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.224 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.224 INFO:teuthology.orchestra.run.smithi116.stdout: "end_time": { 2024-09-12T03:19:14.224 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "end_time", 2024-09-12T03:19:14.224 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.224 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.225 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.225 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "2359", 2024-09-12T03:19:14.225 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.225 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.225 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.225 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T03:19:14.225 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:19:14.225 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.225 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.225 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.225 INFO:teuthology.orchestra.run.smithi116.stdout: "end_weekday": { 2024-09-12T03:19:14.225 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "end_weekday", 2024-09-12T03:19:14.225 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "uint", 2024-09-12T03:19:14.226 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.226 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.226 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "0", 2024-09-12T03:19:14.226 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "0", 2024-09-12T03:19:14.226 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "6", 2024-09-12T03:19:14.226 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.226 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T03:19:14.226 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:19:14.226 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.226 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.226 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.226 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.226 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.226 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.227 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.227 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.227 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.227 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.227 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.227 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.227 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.227 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.227 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.227 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.227 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.227 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.227 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.228 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.228 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.228 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.228 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.228 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.228 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.228 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.228 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.228 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.228 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.228 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.228 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.228 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.229 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.229 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.229 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.229 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.229 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.229 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.229 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.229 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.229 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.229 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.229 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.229 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.229 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.230 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.230 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.230 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.230 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.230 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.230 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.230 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.230 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.230 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.230 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.230 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.230 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.230 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.230 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.231 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.231 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.231 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.231 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.231 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.231 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.231 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.231 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.231 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.231 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.231 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.231 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.231 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.232 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.232 INFO:teuthology.orchestra.run.smithi116.stdout: "min_score": { 2024-09-12T03:19:14.232 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "min_score", 2024-09-12T03:19:14.232 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "float", 2024-09-12T03:19:14.232 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.232 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.232 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "0", 2024-09-12T03:19:14.232 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.232 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.232 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.232 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T03:19:14.232 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.232 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.233 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.233 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.233 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": { 2024-09-12T03:19:14.233 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "mode", 2024-09-12T03:19:14.233 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.233 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.233 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.233 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "upmap", 2024-09-12T03:19:14.233 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.233 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.233 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.233 INFO:teuthology.orchestra.run.smithi116.stdout: "crush-compat", 2024-09-12T03:19:14.233 INFO:teuthology.orchestra.run.smithi116.stdout: "none", 2024-09-12T03:19:14.233 INFO:teuthology.orchestra.run.smithi116.stdout: "read", 2024-09-12T03:19:14.234 INFO:teuthology.orchestra.run.smithi116.stdout: "upmap", 2024-09-12T03:19:14.234 INFO:teuthology.orchestra.run.smithi116.stdout: "upmap-read" 2024-09-12T03:19:14.234 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.234 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Balancer mode", 2024-09-12T03:19:14.234 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.234 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.234 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.234 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.234 INFO:teuthology.orchestra.run.smithi116.stdout: "pool_ids": { 2024-09-12T03:19:14.234 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "pool_ids", 2024-09-12T03:19:14.234 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.234 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.234 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.235 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.235 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.235 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.235 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.235 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T03:19:14.235 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.235 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.235 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.235 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.235 INFO:teuthology.orchestra.run.smithi116.stdout: "sleep_interval": { 2024-09-12T03:19:14.235 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "sleep_interval", 2024-09-12T03:19:14.235 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.235 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.235 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.236 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "60", 2024-09-12T03:19:14.236 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.236 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.236 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.236 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T03:19:14.236 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.236 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.236 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.236 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.236 INFO:teuthology.orchestra.run.smithi116.stdout: "upmap_max_deviation": { 2024-09-12T03:19:14.236 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "upmap_max_deviation", 2024-09-12T03:19:14.236 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.236 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.237 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.237 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "5", 2024-09-12T03:19:14.237 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "1", 2024-09-12T03:19:14.237 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.237 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.237 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T03:19:14.237 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T03:19:14.237 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.237 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.237 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.237 INFO:teuthology.orchestra.run.smithi116.stdout: "upmap_max_optimizations": { 2024-09-12T03:19:14.237 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "upmap_max_optimizations", 2024-09-12T03:19:14.237 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "uint", 2024-09-12T03:19:14.238 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.238 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.238 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "10", 2024-09-12T03:19:14.238 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.238 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.238 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.238 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T03:19:14.238 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.238 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.238 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.238 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.238 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.238 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.238 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.239 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "cephadm", 2024-09-12T03:19:14.239 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.239 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.239 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.239 INFO:teuthology.orchestra.run.smithi116.stdout: "agent_down_multiplier": { 2024-09-12T03:19:14.239 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "agent_down_multiplier", 2024-09-12T03:19:14.239 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "float", 2024-09-12T03:19:14.239 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.239 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.239 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "3.0", 2024-09-12T03:19:14.239 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.239 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.239 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.240 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T03:19:14.240 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.240 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.240 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.240 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.240 INFO:teuthology.orchestra.run.smithi116.stdout: "agent_refresh_rate": { 2024-09-12T03:19:14.240 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "agent_refresh_rate", 2024-09-12T03:19:14.240 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.240 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.240 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.240 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "20", 2024-09-12T03:19:14.240 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.240 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.241 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.241 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T03:19:14.241 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.241 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.241 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.241 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.241 INFO:teuthology.orchestra.run.smithi116.stdout: "agent_starting_port": { 2024-09-12T03:19:14.241 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "agent_starting_port", 2024-09-12T03:19:14.241 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.241 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.241 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.241 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "4721", 2024-09-12T03:19:14.241 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.241 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.242 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.242 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T03:19:14.242 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.242 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.242 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.242 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.242 INFO:teuthology.orchestra.run.smithi116.stdout: "allow_ptrace": { 2024-09-12T03:19:14.242 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "allow_ptrace", 2024-09-12T03:19:14.242 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.242 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.242 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.242 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.242 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.242 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.243 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.243 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T03:19:14.243 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T03:19:14.243 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.243 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.243 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.243 INFO:teuthology.orchestra.run.smithi116.stdout: "autotune_interval": { 2024-09-12T03:19:14.243 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "autotune_interval", 2024-09-12T03:19:14.243 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.243 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.243 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.243 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "600", 2024-09-12T03:19:14.244 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.244 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.244 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.244 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T03:19:14.244 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.244 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.244 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.244 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.244 INFO:teuthology.orchestra.run.smithi116.stdout: "autotune_memory_target_ratio": { 2024-09-12T03:19:14.244 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T03:19:14.244 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "float", 2024-09-12T03:19:14.244 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.244 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.245 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "0.7", 2024-09-12T03:19:14.245 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.245 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.245 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.245 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T03:19:14.245 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.245 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.245 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.245 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.245 INFO:teuthology.orchestra.run.smithi116.stdout: "cephadm_log_destination": { 2024-09-12T03:19:14.245 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "cephadm_log_destination", 2024-09-12T03:19:14.245 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.245 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.245 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.246 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.246 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.246 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.246 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.246 INFO:teuthology.orchestra.run.smithi116.stdout: "file", 2024-09-12T03:19:14.246 INFO:teuthology.orchestra.run.smithi116.stdout: "file,syslog", 2024-09-12T03:19:14.246 INFO:teuthology.orchestra.run.smithi116.stdout: "syslog" 2024-09-12T03:19:14.246 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.246 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T03:19:14.246 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.246 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.246 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.246 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.247 INFO:teuthology.orchestra.run.smithi116.stdout: "cgroups_split": { 2024-09-12T03:19:14.247 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "cgroups_split", 2024-09-12T03:19:14.247 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.247 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.247 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.247 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.247 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.247 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.247 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.247 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T03:19:14.247 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.247 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.247 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.247 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.248 INFO:teuthology.orchestra.run.smithi116.stdout: "config_checks_enabled": { 2024-09-12T03:19:14.248 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "config_checks_enabled", 2024-09-12T03:19:14.248 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.248 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.248 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.248 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.248 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.248 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.248 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.248 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T03:19:14.248 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.248 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.249 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.249 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.249 INFO:teuthology.orchestra.run.smithi116.stdout: "config_dashboard": { 2024-09-12T03:19:14.249 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "config_dashboard", 2024-09-12T03:19:14.249 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.249 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.249 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.249 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.249 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.249 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.249 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.249 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T03:19:14.249 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.249 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.250 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.250 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.250 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_alertmanager": { 2024-09-12T03:19:14.250 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_alertmanager", 2024-09-12T03:19:14.250 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.250 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.250 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.250 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T03:19:14.250 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.250 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.250 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.250 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Prometheus container image", 2024-09-12T03:19:14.251 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.251 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.251 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.251 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.251 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_base": { 2024-09-12T03:19:14.251 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_base", 2024-09-12T03:19:14.251 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.251 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.251 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.251 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T03:19:14.251 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.251 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.251 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.251 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Container image name, without the tag", 2024-09-12T03:19:14.252 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.252 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.252 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.252 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.252 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_elasticsearch": { 2024-09-12T03:19:14.252 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_elasticsearch", 2024-09-12T03:19:14.252 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.252 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.252 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.252 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T03:19:14.252 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.252 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.252 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.252 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "elasticsearch container image", 2024-09-12T03:19:14.253 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.253 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.253 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.253 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.253 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_grafana": { 2024-09-12T03:19:14.253 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_grafana", 2024-09-12T03:19:14.253 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.253 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.253 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.253 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T03:19:14.253 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.253 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.253 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.253 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Prometheus container image", 2024-09-12T03:19:14.254 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.254 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.254 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.254 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.254 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_haproxy": { 2024-09-12T03:19:14.254 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_haproxy", 2024-09-12T03:19:14.254 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.254 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.254 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.254 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T03:19:14.254 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.254 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.254 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.254 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "HAproxy container image", 2024-09-12T03:19:14.255 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.255 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.255 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.255 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.255 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_jaeger_agent": { 2024-09-12T03:19:14.255 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_jaeger_agent", 2024-09-12T03:19:14.255 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.255 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.255 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.255 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T03:19:14.255 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.255 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.255 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.256 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Jaeger agent container image", 2024-09-12T03:19:14.256 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.256 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.256 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.256 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.256 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_jaeger_collector": { 2024-09-12T03:19:14.256 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_jaeger_collector", 2024-09-12T03:19:14.256 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.256 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.256 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.256 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T03:19:14.256 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.256 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.256 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.257 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Jaeger collector container image", 2024-09-12T03:19:14.257 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.257 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.257 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.257 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.257 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_jaeger_query": { 2024-09-12T03:19:14.257 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_jaeger_query", 2024-09-12T03:19:14.257 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.257 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.257 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.257 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T03:19:14.257 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.257 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.258 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.258 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Jaeger query container image", 2024-09-12T03:19:14.258 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.258 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.258 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.258 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.258 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_keepalived": { 2024-09-12T03:19:14.258 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_keepalived", 2024-09-12T03:19:14.258 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.258 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.258 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.258 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T03:19:14.258 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.258 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.258 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.259 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Keepalived container image", 2024-09-12T03:19:14.259 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.259 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.259 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.259 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.259 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_loki": { 2024-09-12T03:19:14.259 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_loki", 2024-09-12T03:19:14.259 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.259 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.259 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.259 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T03:19:14.259 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.259 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.260 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.260 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Loki container image", 2024-09-12T03:19:14.260 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.260 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.260 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.260 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.260 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_node_exporter": { 2024-09-12T03:19:14.260 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_node_exporter", 2024-09-12T03:19:14.260 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.260 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.260 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.260 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T03:19:14.260 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.260 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.261 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.261 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Prometheus container image", 2024-09-12T03:19:14.261 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.261 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.261 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.261 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.261 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_nvmeof": { 2024-09-12T03:19:14.261 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_nvmeof", 2024-09-12T03:19:14.261 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.261 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.261 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.261 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T03:19:14.261 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.261 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.262 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.262 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Nvme-of container image", 2024-09-12T03:19:14.262 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.262 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.262 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.262 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.262 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_prometheus": { 2024-09-12T03:19:14.262 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_prometheus", 2024-09-12T03:19:14.262 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.262 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.262 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.262 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T03:19:14.262 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.262 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.263 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.263 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Prometheus container image", 2024-09-12T03:19:14.263 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.263 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.263 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.263 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.263 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_promtail": { 2024-09-12T03:19:14.263 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_promtail", 2024-09-12T03:19:14.263 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.263 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.263 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.263 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T03:19:14.263 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.263 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.264 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.264 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Promtail container image", 2024-09-12T03:19:14.264 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.264 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.264 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.264 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.264 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_samba": { 2024-09-12T03:19:14.264 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_samba", 2024-09-12T03:19:14.264 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.264 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.264 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.264 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T03:19:14.264 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.265 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.265 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.265 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Samba/SMB container image", 2024-09-12T03:19:14.265 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.265 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.265 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.265 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.265 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_snmp_gateway": { 2024-09-12T03:19:14.265 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_snmp_gateway", 2024-09-12T03:19:14.265 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.265 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.265 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.265 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T03:19:14.265 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.266 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.266 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.266 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "SNMP Gateway container image", 2024-09-12T03:19:14.266 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.266 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.266 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.266 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.266 INFO:teuthology.orchestra.run.smithi116.stdout: "container_init": { 2024-09-12T03:19:14.266 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_init", 2024-09-12T03:19:14.266 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.266 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.266 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.266 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.266 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.267 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.267 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.267 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T03:19:14.267 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.267 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.267 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.267 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.267 INFO:teuthology.orchestra.run.smithi116.stdout: "daemon_cache_timeout": { 2024-09-12T03:19:14.267 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "daemon_cache_timeout", 2024-09-12T03:19:14.267 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.267 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.267 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.267 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "600", 2024-09-12T03:19:14.268 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.268 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.268 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.268 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T03:19:14.268 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.268 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.268 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.268 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.268 INFO:teuthology.orchestra.run.smithi116.stdout: "default_cephadm_command_timeout": { 2024-09-12T03:19:14.268 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T03:19:14.268 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.268 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.268 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.269 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "900", 2024-09-12T03:19:14.269 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.269 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.269 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.269 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T03:19:14.269 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.269 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.269 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.269 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.269 INFO:teuthology.orchestra.run.smithi116.stdout: "default_registry": { 2024-09-12T03:19:14.269 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "default_registry", 2024-09-12T03:19:14.270 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.270 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.270 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.270 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "docker.io", 2024-09-12T03:19:14.270 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.270 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.270 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.270 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T03:19:14.270 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.270 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.270 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.270 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.270 INFO:teuthology.orchestra.run.smithi116.stdout: "device_cache_timeout": { 2024-09-12T03:19:14.270 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "device_cache_timeout", 2024-09-12T03:19:14.271 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.271 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.271 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.271 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "1800", 2024-09-12T03:19:14.271 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.271 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.271 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.271 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "seconds to cache device inventory", 2024-09-12T03:19:14.271 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.271 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.271 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.271 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.271 INFO:teuthology.orchestra.run.smithi116.stdout: "device_enhanced_scan": { 2024-09-12T03:19:14.271 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "device_enhanced_scan", 2024-09-12T03:19:14.272 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.272 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.272 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.272 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.272 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.272 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.272 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.272 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T03:19:14.272 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.272 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.272 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.272 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.272 INFO:teuthology.orchestra.run.smithi116.stdout: "facts_cache_timeout": { 2024-09-12T03:19:14.273 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "facts_cache_timeout", 2024-09-12T03:19:14.273 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.273 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.273 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.273 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "60", 2024-09-12T03:19:14.273 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.273 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.273 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.273 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "seconds to cache host facts data", 2024-09-12T03:19:14.273 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.273 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.273 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.273 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.274 INFO:teuthology.orchestra.run.smithi116.stdout: "grafana_dashboards_path": { 2024-09-12T03:19:14.274 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "grafana_dashboards_path", 2024-09-12T03:19:14.274 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.274 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.274 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.274 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T03:19:14.274 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.274 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.274 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.274 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T03:19:14.274 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.274 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.274 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.274 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.275 INFO:teuthology.orchestra.run.smithi116.stdout: "host_check_interval": { 2024-09-12T03:19:14.275 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "host_check_interval", 2024-09-12T03:19:14.275 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.275 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.275 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.275 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "600", 2024-09-12T03:19:14.275 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.275 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.275 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.275 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "how frequently to perform a host check", 2024-09-12T03:19:14.275 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.275 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.275 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.275 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.276 INFO:teuthology.orchestra.run.smithi116.stdout: "hw_monitoring": { 2024-09-12T03:19:14.276 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "hw_monitoring", 2024-09-12T03:19:14.276 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.276 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.276 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.276 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.276 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.276 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.276 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.276 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T03:19:14.276 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.276 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.276 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.277 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.277 INFO:teuthology.orchestra.run.smithi116.stdout: "inventory_list_all": { 2024-09-12T03:19:14.277 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "inventory_list_all", 2024-09-12T03:19:14.277 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.277 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.277 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.277 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.277 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.277 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.277 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.277 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T03:19:14.277 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.277 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.277 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.278 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.278 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.278 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.278 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.278 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.278 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.278 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.278 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.278 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.278 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.278 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.278 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.278 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.278 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.279 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.279 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.279 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.279 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.279 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.279 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.279 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.279 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.279 INFO:teuthology.orchestra.run.smithi116.stdout: "log_refresh_metadata": { 2024-09-12T03:19:14.279 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_refresh_metadata", 2024-09-12T03:19:14.279 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.279 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.279 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.279 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.280 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.280 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.280 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.280 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T03:19:14.280 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.280 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.280 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.280 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.280 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.280 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.280 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.280 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.280 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.281 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.281 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.281 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.281 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.281 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T03:19:14.281 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.281 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.281 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.281 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.281 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.281 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.281 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.281 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.282 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.282 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.282 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.282 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.282 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.282 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.282 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.282 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.282 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.282 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.282 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.282 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.282 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.282 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.283 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.283 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.283 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.283 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.283 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.283 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.283 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.283 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.283 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.283 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.283 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.283 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.283 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.283 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.284 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.284 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.284 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.284 INFO:teuthology.orchestra.run.smithi116.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T03:19:14.284 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T03:19:14.284 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.284 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.284 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.284 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.284 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.284 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.284 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.284 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T03:19:14.284 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.285 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.285 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.285 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.285 INFO:teuthology.orchestra.run.smithi116.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T03:19:14.285 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T03:19:14.285 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.285 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.285 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.285 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "*", 2024-09-12T03:19:14.285 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.285 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.285 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.285 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T03:19:14.285 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.286 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.286 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.286 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.286 INFO:teuthology.orchestra.run.smithi116.stdout: "max_count_per_host": { 2024-09-12T03:19:14.286 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "max_count_per_host", 2024-09-12T03:19:14.286 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.286 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.286 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.286 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "10", 2024-09-12T03:19:14.286 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.286 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.286 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.286 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "max number of daemons per service per host", 2024-09-12T03:19:14.286 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.287 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.287 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.287 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.287 INFO:teuthology.orchestra.run.smithi116.stdout: "max_osd_draining_count": { 2024-09-12T03:19:14.287 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "max_osd_draining_count", 2024-09-12T03:19:14.287 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.287 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.287 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.287 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "10", 2024-09-12T03:19:14.287 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.287 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.287 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.287 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T03:19:14.287 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.288 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.288 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.288 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.288 INFO:teuthology.orchestra.run.smithi116.stdout: "migration_current": { 2024-09-12T03:19:14.288 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "migration_current", 2024-09-12T03:19:14.288 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.288 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.288 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.288 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.288 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.288 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.288 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.288 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "internal - do not modify", 2024-09-12T03:19:14.288 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.289 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.289 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.289 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.289 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": { 2024-09-12T03:19:14.289 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "mode", 2024-09-12T03:19:14.289 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.289 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.289 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.289 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "root", 2024-09-12T03:19:14.289 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.289 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.289 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.289 INFO:teuthology.orchestra.run.smithi116.stdout: "cephadm-package", 2024-09-12T03:19:14.290 INFO:teuthology.orchestra.run.smithi116.stdout: "root" 2024-09-12T03:19:14.290 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.290 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T03:19:14.290 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.290 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.290 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.290 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.290 INFO:teuthology.orchestra.run.smithi116.stdout: "oob_default_addr": { 2024-09-12T03:19:14.290 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "oob_default_addr", 2024-09-12T03:19:14.290 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.290 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.290 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.290 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "169.254.1.1", 2024-09-12T03:19:14.290 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.291 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.291 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.291 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T03:19:14.291 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.291 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.291 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.291 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.291 INFO:teuthology.orchestra.run.smithi116.stdout: "prometheus_alerts_path": { 2024-09-12T03:19:14.291 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "prometheus_alerts_path", 2024-09-12T03:19:14.291 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.291 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.291 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.291 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T03:19:14.291 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.292 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.292 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.292 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T03:19:14.292 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.292 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.292 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.292 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.292 INFO:teuthology.orchestra.run.smithi116.stdout: "registry_insecure": { 2024-09-12T03:19:14.292 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "registry_insecure", 2024-09-12T03:19:14.292 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.292 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.292 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.292 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.292 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.293 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.293 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.293 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T03:19:14.293 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.293 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.293 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.293 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.293 INFO:teuthology.orchestra.run.smithi116.stdout: "registry_password": { 2024-09-12T03:19:14.293 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "registry_password", 2024-09-12T03:19:14.293 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.293 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.293 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.293 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.294 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.294 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.294 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.294 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T03:19:14.294 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.294 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.294 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.294 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.294 INFO:teuthology.orchestra.run.smithi116.stdout: "registry_url": { 2024-09-12T03:19:14.294 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "registry_url", 2024-09-12T03:19:14.294 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.294 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.294 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.294 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.295 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.295 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.295 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.295 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T03:19:14.295 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.295 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.295 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.295 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.295 INFO:teuthology.orchestra.run.smithi116.stdout: "registry_username": { 2024-09-12T03:19:14.295 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "registry_username", 2024-09-12T03:19:14.295 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.295 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.295 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.296 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.296 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.296 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.296 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.296 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T03:19:14.296 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.296 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.296 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.296 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.296 INFO:teuthology.orchestra.run.smithi116.stdout: "secure_monitoring_stack": { 2024-09-12T03:19:14.296 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "secure_monitoring_stack", 2024-09-12T03:19:14.296 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.296 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.296 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.297 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.297 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.297 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.297 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.297 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T03:19:14.297 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.297 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.297 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.297 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.297 INFO:teuthology.orchestra.run.smithi116.stdout: "service_discovery_port": { 2024-09-12T03:19:14.297 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "service_discovery_port", 2024-09-12T03:19:14.297 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.297 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.297 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.298 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "8765", 2024-09-12T03:19:14.298 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.298 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.298 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.298 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "cephadm service discovery port", 2024-09-12T03:19:14.298 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.298 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.298 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.298 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.298 INFO:teuthology.orchestra.run.smithi116.stdout: "ssh_config_file": { 2024-09-12T03:19:14.298 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ssh_config_file", 2024-09-12T03:19:14.298 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.298 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.299 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.299 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.299 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.299 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.299 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.299 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T03:19:14.299 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.299 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.299 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.299 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.299 INFO:teuthology.orchestra.run.smithi116.stdout: "use_agent": { 2024-09-12T03:19:14.299 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "use_agent", 2024-09-12T03:19:14.299 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.299 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.300 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.300 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.300 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.300 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.300 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.300 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T03:19:14.300 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.300 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.300 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.300 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.300 INFO:teuthology.orchestra.run.smithi116.stdout: "use_repo_digest": { 2024-09-12T03:19:14.300 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "use_repo_digest", 2024-09-12T03:19:14.300 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.301 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.301 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.301 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.301 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.301 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.301 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.301 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T03:19:14.301 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.301 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.301 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.301 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.301 INFO:teuthology.orchestra.run.smithi116.stdout: "warn_on_failed_host_check": { 2024-09-12T03:19:14.301 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "warn_on_failed_host_check", 2024-09-12T03:19:14.301 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.302 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.302 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.302 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.302 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.302 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.302 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.302 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T03:19:14.302 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.302 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.302 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.302 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.302 INFO:teuthology.orchestra.run.smithi116.stdout: "warn_on_stray_daemons": { 2024-09-12T03:19:14.302 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "warn_on_stray_daemons", 2024-09-12T03:19:14.302 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.303 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.303 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.303 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.303 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.303 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.303 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.303 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T03:19:14.303 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.303 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.303 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.303 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.303 INFO:teuthology.orchestra.run.smithi116.stdout: "warn_on_stray_hosts": { 2024-09-12T03:19:14.303 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "warn_on_stray_hosts", 2024-09-12T03:19:14.303 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.304 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.304 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.304 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.304 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.304 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.304 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.304 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T03:19:14.304 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.304 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.304 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.304 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.304 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.304 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.305 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.305 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "crash", 2024-09-12T03:19:14.305 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.305 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.305 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.305 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.305 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.305 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.305 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.305 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.305 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.305 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.305 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.305 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.306 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.306 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.306 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.306 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.306 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.306 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.306 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.306 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.306 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.306 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.306 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.306 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.306 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.307 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.307 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.307 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.307 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.307 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.307 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.307 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.307 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.307 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.307 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.307 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.307 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.307 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.307 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.308 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.308 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.308 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.308 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.308 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.308 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.308 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.308 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.308 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.308 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.308 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.308 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.308 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.309 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.309 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.309 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.309 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.309 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.309 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.309 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.309 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.309 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.309 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.309 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.309 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.309 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.309 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.310 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.310 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.310 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.310 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.310 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.310 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.310 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.310 INFO:teuthology.orchestra.run.smithi116.stdout: "retain_interval": { 2024-09-12T03:19:14.310 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "retain_interval", 2024-09-12T03:19:14.310 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.310 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.310 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.311 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "31536000", 2024-09-12T03:19:14.311 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.311 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.311 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.311 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T03:19:14.311 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.311 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.311 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.311 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.311 INFO:teuthology.orchestra.run.smithi116.stdout: "warn_recent_interval": { 2024-09-12T03:19:14.311 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "warn_recent_interval", 2024-09-12T03:19:14.311 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.312 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.312 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.312 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "1209600", 2024-09-12T03:19:14.312 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.312 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.312 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.312 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T03:19:14.312 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.312 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.312 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.312 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.312 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.312 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.312 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.313 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "dashboard", 2024-09-12T03:19:14.313 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.313 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.313 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.313 INFO:teuthology.orchestra.run.smithi116.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T03:19:14.313 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T03:19:14.313 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.313 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.313 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.313 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "10", 2024-09-12T03:19:14.313 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.313 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.313 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.314 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.314 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.314 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.314 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.314 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.314 INFO:teuthology.orchestra.run.smithi116.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T03:19:14.314 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T03:19:14.314 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.314 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.314 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.314 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.314 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.314 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.314 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.315 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.315 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.315 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.315 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.315 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.315 INFO:teuthology.orchestra.run.smithi116.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T03:19:14.315 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T03:19:14.315 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.315 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.315 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.315 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.315 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.315 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.316 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.316 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.316 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.316 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.316 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.316 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.316 INFO:teuthology.orchestra.run.smithi116.stdout: "AUDIT_API_ENABLED": { 2024-09-12T03:19:14.316 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T03:19:14.316 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.316 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.316 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.316 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.316 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.316 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.317 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.317 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.317 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.317 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.317 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.317 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.317 INFO:teuthology.orchestra.run.smithi116.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T03:19:14.317 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T03:19:14.317 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.317 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.317 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.317 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.317 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.317 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.318 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.318 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.318 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.318 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.318 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.318 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.318 INFO:teuthology.orchestra.run.smithi116.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T03:19:14.318 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T03:19:14.318 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.318 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.318 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.318 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.318 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.318 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.319 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.319 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.319 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.319 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.319 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.319 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.319 INFO:teuthology.orchestra.run.smithi116.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T03:19:14.319 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T03:19:14.319 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.319 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.319 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.319 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.319 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.319 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.320 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.320 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.320 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.320 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.320 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.320 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.320 INFO:teuthology.orchestra.run.smithi116.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T03:19:14.320 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T03:19:14.320 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.320 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.320 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.320 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.321 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.321 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.321 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.321 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.321 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.321 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.321 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.321 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.321 INFO:teuthology.orchestra.run.smithi116.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T03:19:14.321 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T03:19:14.321 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.321 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.321 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.322 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.322 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.322 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.322 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.322 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.322 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.322 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.322 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.322 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.322 INFO:teuthology.orchestra.run.smithi116.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T03:19:14.322 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T03:19:14.322 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.322 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.322 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.323 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.323 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.323 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.323 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.323 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.323 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.323 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.323 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.323 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.323 INFO:teuthology.orchestra.run.smithi116.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T03:19:14.323 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T03:19:14.323 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.323 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.324 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.324 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.324 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.324 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.324 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.324 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.324 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.324 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.324 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.324 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.324 INFO:teuthology.orchestra.run.smithi116.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T03:19:14.324 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T03:19:14.324 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.325 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.325 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.325 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.325 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.325 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.325 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.325 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.325 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.325 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.325 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.325 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.325 INFO:teuthology.orchestra.run.smithi116.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T03:19:14.325 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T03:19:14.325 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.326 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.326 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.326 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.326 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.326 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.326 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.326 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.326 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.326 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.326 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.326 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.326 INFO:teuthology.orchestra.run.smithi116.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T03:19:14.326 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T03:19:14.327 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.327 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.327 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.327 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.327 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.327 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.327 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.327 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.327 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.327 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.327 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.327 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.327 INFO:teuthology.orchestra.run.smithi116.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T03:19:14.327 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T03:19:14.328 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.328 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.328 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.328 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "admin", 2024-09-12T03:19:14.328 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.328 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.328 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.328 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.328 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.328 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.328 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.328 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.328 INFO:teuthology.orchestra.run.smithi116.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T03:19:14.328 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T03:19:14.329 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.329 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.329 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.329 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.329 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.329 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.329 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.329 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.329 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.329 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.329 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.329 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.329 INFO:teuthology.orchestra.run.smithi116.stdout: "GRAFANA_API_URL": { 2024-09-12T03:19:14.330 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "GRAFANA_API_URL", 2024-09-12T03:19:14.330 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.330 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.330 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.330 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.330 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.330 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.330 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.330 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.330 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.330 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.330 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.330 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.330 INFO:teuthology.orchestra.run.smithi116.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T03:19:14.331 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T03:19:14.331 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.331 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.331 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.331 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "admin", 2024-09-12T03:19:14.331 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.331 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.331 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.331 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.331 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.331 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.331 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.331 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.331 INFO:teuthology.orchestra.run.smithi116.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T03:19:14.332 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T03:19:14.332 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.332 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.332 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.332 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.332 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.332 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.332 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.332 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.332 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.332 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.332 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.332 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.333 INFO:teuthology.orchestra.run.smithi116.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T03:19:14.333 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T03:19:14.333 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.333 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.333 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.333 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.333 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.333 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.333 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.333 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.333 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.333 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.333 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.334 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.334 INFO:teuthology.orchestra.run.smithi116.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T03:19:14.334 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T03:19:14.334 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.334 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.334 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.334 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.334 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.334 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.334 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.334 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.334 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.334 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.334 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.335 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.335 INFO:teuthology.orchestra.run.smithi116.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T03:19:14.335 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T03:19:14.335 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.335 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.335 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.335 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.335 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.335 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.335 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.335 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.335 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.335 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.336 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.336 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.336 INFO:teuthology.orchestra.run.smithi116.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T03:19:14.336 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T03:19:14.336 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.336 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.336 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.336 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.336 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.336 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.336 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.336 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.336 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.337 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.337 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.337 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.337 INFO:teuthology.orchestra.run.smithi116.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T03:19:14.337 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T03:19:14.337 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.337 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.337 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.337 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.337 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.337 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.337 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.337 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.337 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.338 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.338 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.338 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.338 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T03:19:14.338 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T03:19:14.338 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.338 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.338 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.338 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.338 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.338 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.338 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.338 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.339 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.339 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.339 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.339 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.339 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T03:19:14.339 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T03:19:14.339 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.339 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.339 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.339 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.339 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.339 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.339 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.339 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.340 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.340 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.340 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.340 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.340 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T03:19:14.340 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T03:19:14.340 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.340 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.340 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.340 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.340 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.340 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.340 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.341 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.341 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.341 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.341 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.341 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.341 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T03:19:14.341 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T03:19:14.341 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.341 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.341 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.341 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.341 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.341 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.341 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.342 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.342 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.342 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.342 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.342 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.342 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T03:19:14.342 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T03:19:14.342 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.342 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.342 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.342 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.342 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.342 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.343 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.343 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.343 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.343 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.343 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.343 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.343 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T03:19:14.343 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T03:19:14.343 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.343 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.344 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.344 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.344 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.344 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.344 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.345 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.345 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.345 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.345 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.345 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.345 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T03:19:14.345 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T03:19:14.345 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.345 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.345 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.345 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.345 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.345 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.346 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.346 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.346 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.346 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.346 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.346 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.346 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T03:19:14.346 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T03:19:14.346 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.346 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.346 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.346 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.346 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.346 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.347 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.347 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.347 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.347 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.347 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.347 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.347 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T03:19:14.347 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T03:19:14.347 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.347 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.347 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.347 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T03:19:14.347 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.347 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.348 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.348 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.348 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.348 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.348 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.348 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.348 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T03:19:14.348 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T03:19:14.348 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.348 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.348 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.348 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "10", 2024-09-12T03:19:14.349 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.349 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.349 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.349 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.349 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.349 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.349 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.349 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.349 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T03:19:14.349 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T03:19:14.349 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.349 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.349 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.349 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "8", 2024-09-12T03:19:14.350 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.350 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.350 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.350 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.350 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.350 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.350 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.350 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.350 INFO:teuthology.orchestra.run.smithi116.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T03:19:14.350 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T03:19:14.350 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.350 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.350 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.351 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "45", 2024-09-12T03:19:14.351 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.351 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.351 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.351 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.351 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.351 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.351 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.351 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.351 INFO:teuthology.orchestra.run.smithi116.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T03:19:14.351 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T03:19:14.351 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.351 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.351 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.352 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.352 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.352 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.352 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.352 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.352 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.352 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.352 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.352 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.352 INFO:teuthology.orchestra.run.smithi116.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T03:19:14.352 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T03:19:14.352 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.352 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.353 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.353 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "admin", 2024-09-12T03:19:14.353 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.353 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.353 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.353 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.353 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.353 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.353 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.353 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.353 INFO:teuthology.orchestra.run.smithi116.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T03:19:14.353 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T03:19:14.353 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.353 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.354 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.354 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.354 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.354 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.354 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.354 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.354 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.354 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.354 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.354 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.354 INFO:teuthology.orchestra.run.smithi116.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T03:19:14.354 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T03:19:14.354 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.354 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.355 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.355 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.355 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.355 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.355 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.355 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.355 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.355 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.355 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.355 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.355 INFO:teuthology.orchestra.run.smithi116.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T03:19:14.355 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T03:19:14.355 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.355 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.356 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.356 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "0", 2024-09-12T03:19:14.356 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.356 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.356 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.356 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.356 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.356 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.356 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.356 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.356 INFO:teuthology.orchestra.run.smithi116.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T03:19:14.356 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T03:19:14.356 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.357 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.357 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.357 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "10", 2024-09-12T03:19:14.357 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.357 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.357 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.357 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.357 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.357 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.357 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.357 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.357 INFO:teuthology.orchestra.run.smithi116.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T03:19:14.357 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T03:19:14.357 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.358 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.358 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.358 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "5", 2024-09-12T03:19:14.358 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.358 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.358 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.358 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.358 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.358 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.358 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.358 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.358 INFO:teuthology.orchestra.run.smithi116.stdout: "cross_origin_url": { 2024-09-12T03:19:14.358 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "cross_origin_url", 2024-09-12T03:19:14.358 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.359 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.359 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.359 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.359 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.359 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.359 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.359 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.359 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.359 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.359 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.359 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.359 INFO:teuthology.orchestra.run.smithi116.stdout: "crt_file": { 2024-09-12T03:19:14.359 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "crt_file", 2024-09-12T03:19:14.359 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.360 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.360 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.360 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.360 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.360 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.360 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.360 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.360 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.360 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.360 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.360 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.360 INFO:teuthology.orchestra.run.smithi116.stdout: "debug": { 2024-09-12T03:19:14.360 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "debug", 2024-09-12T03:19:14.361 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.361 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.361 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.361 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.361 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.361 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.361 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.361 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Enable/disable debug options", 2024-09-12T03:19:14.361 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.361 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.361 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.361 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.361 INFO:teuthology.orchestra.run.smithi116.stdout: "jwt_token_ttl": { 2024-09-12T03:19:14.362 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "jwt_token_ttl", 2024-09-12T03:19:14.362 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.362 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.362 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.362 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "28800", 2024-09-12T03:19:14.362 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.362 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.362 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.362 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.362 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.362 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.362 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.362 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.363 INFO:teuthology.orchestra.run.smithi116.stdout: "key_file": { 2024-09-12T03:19:14.363 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "key_file", 2024-09-12T03:19:14.363 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.363 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.363 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.363 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.363 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.363 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.363 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.363 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.363 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.363 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.363 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.363 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.364 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.364 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.364 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.364 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.364 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.364 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.364 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.364 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.364 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.364 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.364 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.364 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.364 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.365 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.365 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.365 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.365 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.365 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.365 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.365 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.365 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.365 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.365 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.365 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.365 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.365 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.365 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.366 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.366 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.366 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.366 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.366 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.366 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.366 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.366 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.366 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.366 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.366 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.366 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.366 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.366 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.367 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.367 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.367 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.367 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.367 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.367 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.367 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.367 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.367 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.367 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.367 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.367 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.367 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.367 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.368 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.368 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.368 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.368 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.368 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.368 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.368 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.368 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.368 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.368 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.368 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.368 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.368 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.369 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.369 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.369 INFO:teuthology.orchestra.run.smithi116.stdout: "motd": { 2024-09-12T03:19:14.369 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "motd", 2024-09-12T03:19:14.369 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.369 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.369 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.369 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.369 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.369 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.370 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.370 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "The message of the day", 2024-09-12T03:19:14.370 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.370 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.370 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.370 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.370 INFO:teuthology.orchestra.run.smithi116.stdout: "redirect_resolve_ip_addr": { 2024-09-12T03:19:14.370 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T03:19:14.370 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.370 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.370 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.370 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.370 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.371 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.371 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.371 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.371 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.371 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.371 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.371 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.371 INFO:teuthology.orchestra.run.smithi116.stdout: "server_addr": { 2024-09-12T03:19:14.371 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "server_addr", 2024-09-12T03:19:14.371 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.371 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.371 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.371 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "::", 2024-09-12T03:19:14.371 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.372 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.372 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.372 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.372 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.372 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.372 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.372 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.372 INFO:teuthology.orchestra.run.smithi116.stdout: "server_port": { 2024-09-12T03:19:14.372 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "server_port", 2024-09-12T03:19:14.372 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.372 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.372 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.372 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "8080", 2024-09-12T03:19:14.372 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.373 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.373 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.373 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.373 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.373 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.373 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.373 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.373 INFO:teuthology.orchestra.run.smithi116.stdout: "ssl": { 2024-09-12T03:19:14.373 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ssl", 2024-09-12T03:19:14.373 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.373 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.373 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.373 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.373 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.374 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.374 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.374 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.374 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.374 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.374 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.374 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.374 INFO:teuthology.orchestra.run.smithi116.stdout: "ssl_server_port": { 2024-09-12T03:19:14.374 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ssl_server_port", 2024-09-12T03:19:14.374 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.374 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.374 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.374 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "8443", 2024-09-12T03:19:14.375 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.375 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.375 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.375 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.375 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.375 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.375 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.375 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.375 INFO:teuthology.orchestra.run.smithi116.stdout: "standby_behaviour": { 2024-09-12T03:19:14.375 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "standby_behaviour", 2024-09-12T03:19:14.375 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.375 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.375 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.375 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "redirect", 2024-09-12T03:19:14.376 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.376 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.376 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.376 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.376 INFO:teuthology.orchestra.run.smithi116.stdout: "redirect" 2024-09-12T03:19:14.376 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.376 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.376 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.376 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.376 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.376 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.376 INFO:teuthology.orchestra.run.smithi116.stdout: "standby_error_status_code": { 2024-09-12T03:19:14.376 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "standby_error_status_code", 2024-09-12T03:19:14.377 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.377 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.377 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.377 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "500", 2024-09-12T03:19:14.377 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "400", 2024-09-12T03:19:14.377 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "599", 2024-09-12T03:19:14.377 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.377 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.377 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.377 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.377 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.377 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.377 INFO:teuthology.orchestra.run.smithi116.stdout: "url_prefix": { 2024-09-12T03:19:14.378 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "url_prefix", 2024-09-12T03:19:14.378 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.378 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.378 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.378 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.378 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.378 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.378 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.378 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.378 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.378 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.378 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.378 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.378 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.379 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.379 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.379 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "devicehealth", 2024-09-12T03:19:14.379 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.379 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.379 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.379 INFO:teuthology.orchestra.run.smithi116.stdout: "enable_monitoring": { 2024-09-12T03:19:14.379 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "enable_monitoring", 2024-09-12T03:19:14.379 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.379 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.379 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.379 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.379 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.379 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.380 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.380 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "monitor device health metrics", 2024-09-12T03:19:14.380 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.380 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.380 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.380 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.380 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.380 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.380 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.380 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.380 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.380 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.380 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.381 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.381 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.381 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.381 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.381 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.381 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.381 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.381 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.381 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.381 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.381 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.381 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.381 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.382 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.382 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.382 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.382 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.382 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.382 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.382 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.382 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.382 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.382 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.382 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.382 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.382 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.382 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.383 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.383 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.383 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.383 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.383 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.383 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.383 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.383 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.383 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.383 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.383 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.383 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.383 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.384 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.384 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.384 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.384 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.384 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.384 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.384 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.384 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.384 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.384 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.384 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.384 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.384 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.385 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.385 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.385 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.385 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.385 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.385 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.385 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.385 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.385 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.385 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.385 INFO:teuthology.orchestra.run.smithi116.stdout: "mark_out_threshold": { 2024-09-12T03:19:14.385 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "mark_out_threshold", 2024-09-12T03:19:14.385 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.385 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.386 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.386 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "2419200", 2024-09-12T03:19:14.386 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.386 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.386 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.386 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T03:19:14.386 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.386 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.386 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.386 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.386 INFO:teuthology.orchestra.run.smithi116.stdout: "pool_name": { 2024-09-12T03:19:14.386 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "pool_name", 2024-09-12T03:19:14.386 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.386 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.387 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.387 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "device_health_metrics", 2024-09-12T03:19:14.387 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.387 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.387 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.387 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T03:19:14.387 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.387 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.387 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.387 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.387 INFO:teuthology.orchestra.run.smithi116.stdout: "retention_period": { 2024-09-12T03:19:14.387 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "retention_period", 2024-09-12T03:19:14.387 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.387 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.388 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.388 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "15552000", 2024-09-12T03:19:14.388 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.388 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.388 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.388 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "how long to retain device health metrics", 2024-09-12T03:19:14.388 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.388 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.388 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.388 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.388 INFO:teuthology.orchestra.run.smithi116.stdout: "scrape_frequency": { 2024-09-12T03:19:14.388 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "scrape_frequency", 2024-09-12T03:19:14.388 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.388 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.389 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.389 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "86400", 2024-09-12T03:19:14.389 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.389 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.389 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.389 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T03:19:14.389 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.389 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.389 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.389 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.389 INFO:teuthology.orchestra.run.smithi116.stdout: "self_heal": { 2024-09-12T03:19:14.389 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "self_heal", 2024-09-12T03:19:14.389 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.389 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.390 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.390 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.390 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.390 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.390 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.390 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T03:19:14.390 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.390 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.390 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.390 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.390 INFO:teuthology.orchestra.run.smithi116.stdout: "sleep_interval": { 2024-09-12T03:19:14.390 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "sleep_interval", 2024-09-12T03:19:14.391 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.391 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.391 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.391 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "600", 2024-09-12T03:19:14.391 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.391 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.391 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.391 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T03:19:14.391 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.391 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.391 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.391 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.391 INFO:teuthology.orchestra.run.smithi116.stdout: "warn_threshold": { 2024-09-12T03:19:14.391 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "warn_threshold", 2024-09-12T03:19:14.392 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.392 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.392 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.392 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "7257600", 2024-09-12T03:19:14.392 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.392 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.392 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.392 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T03:19:14.392 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.392 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.392 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.392 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.392 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.393 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.393 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.393 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "diskprediction_local", 2024-09-12T03:19:14.393 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.393 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.393 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.393 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.393 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.393 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.393 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.393 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.393 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.393 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.394 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.394 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.394 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.394 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.394 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.394 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.394 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.394 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.394 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.394 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.394 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.394 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.394 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.394 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.395 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.395 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.395 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.395 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.395 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.395 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.395 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.395 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.395 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.395 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.395 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.395 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.395 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.395 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.396 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.396 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.396 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.396 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.396 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.396 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.396 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.396 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.396 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.396 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.396 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.396 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.396 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.397 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.397 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.397 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.397 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.397 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.397 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.397 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.397 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.397 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.397 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.397 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.397 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.397 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.397 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.398 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.398 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.398 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.398 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.398 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.398 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.398 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.398 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.398 INFO:teuthology.orchestra.run.smithi116.stdout: "predict_interval": { 2024-09-12T03:19:14.398 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "predict_interval", 2024-09-12T03:19:14.398 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.398 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.398 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.398 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "86400", 2024-09-12T03:19:14.399 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.399 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.399 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.399 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.399 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.399 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.399 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.399 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.399 INFO:teuthology.orchestra.run.smithi116.stdout: "predictor_model": { 2024-09-12T03:19:14.399 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "predictor_model", 2024-09-12T03:19:14.399 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.399 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.399 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.399 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "prophetstor", 2024-09-12T03:19:14.400 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.400 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.400 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.400 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.400 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.400 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.400 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.400 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.400 INFO:teuthology.orchestra.run.smithi116.stdout: "sleep_interval": { 2024-09-12T03:19:14.400 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "sleep_interval", 2024-09-12T03:19:14.400 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.400 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.400 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.400 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "600", 2024-09-12T03:19:14.401 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.401 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.401 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.401 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.401 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.401 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.401 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.401 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.401 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.401 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.401 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.401 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "influx", 2024-09-12T03:19:14.401 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": false, 2024-09-12T03:19:14.402 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "influxdb python module not found", 2024-09-12T03:19:14.402 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.402 INFO:teuthology.orchestra.run.smithi116.stdout: "batch_size": { 2024-09-12T03:19:14.402 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "batch_size", 2024-09-12T03:19:14.402 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.402 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.402 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.402 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "5000", 2024-09-12T03:19:14.402 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.402 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.402 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.402 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T03:19:14.402 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.402 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.402 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.403 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.403 INFO:teuthology.orchestra.run.smithi116.stdout: "database": { 2024-09-12T03:19:14.403 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "database", 2024-09-12T03:19:14.403 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.403 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.403 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.403 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "ceph", 2024-09-12T03:19:14.403 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.403 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.403 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.403 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T03:19:14.403 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.404 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.404 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.404 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.404 INFO:teuthology.orchestra.run.smithi116.stdout: "hostname": { 2024-09-12T03:19:14.404 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "hostname", 2024-09-12T03:19:14.404 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.404 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.404 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.404 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.404 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.404 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.404 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.404 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "InfluxDB server hostname", 2024-09-12T03:19:14.404 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.405 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.405 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.405 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.405 INFO:teuthology.orchestra.run.smithi116.stdout: "interval": { 2024-09-12T03:19:14.405 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "interval", 2024-09-12T03:19:14.405 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.405 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.405 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.405 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "30", 2024-09-12T03:19:14.405 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "5", 2024-09-12T03:19:14.405 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.405 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.405 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T03:19:14.405 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.406 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.406 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.406 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.406 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.406 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.406 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.406 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.406 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.406 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.406 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.406 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.406 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.406 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.407 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.407 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.407 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.407 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.407 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.407 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.407 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.407 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.407 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.407 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.407 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.407 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.407 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.407 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.407 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.408 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.408 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.408 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.408 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.408 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.408 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.408 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.408 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.408 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.408 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.408 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.408 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.408 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.409 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.409 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.409 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.409 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.409 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.409 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.409 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.409 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.409 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.409 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.409 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.409 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.409 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.409 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.410 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.410 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.410 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.410 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.410 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.410 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.410 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.410 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.410 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.410 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.410 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.410 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.410 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.411 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.411 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.411 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.411 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.411 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.411 INFO:teuthology.orchestra.run.smithi116.stdout: "password": { 2024-09-12T03:19:14.411 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "password", 2024-09-12T03:19:14.411 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.411 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.411 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.411 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.411 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.411 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.411 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.412 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "password of InfluxDB server user", 2024-09-12T03:19:14.412 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.412 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.412 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.412 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.412 INFO:teuthology.orchestra.run.smithi116.stdout: "port": { 2024-09-12T03:19:14.412 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "port", 2024-09-12T03:19:14.412 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.412 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.412 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.412 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "8086", 2024-09-12T03:19:14.412 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.412 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.413 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.413 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "InfluxDB server port", 2024-09-12T03:19:14.413 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.413 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.413 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.413 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.413 INFO:teuthology.orchestra.run.smithi116.stdout: "ssl": { 2024-09-12T03:19:14.413 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ssl", 2024-09-12T03:19:14.413 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.413 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.413 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.413 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "false", 2024-09-12T03:19:14.413 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.414 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.414 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.414 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:19:14.414 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.414 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.414 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.414 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.414 INFO:teuthology.orchestra.run.smithi116.stdout: "threads": { 2024-09-12T03:19:14.414 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "threads", 2024-09-12T03:19:14.414 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.414 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.414 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.414 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "5", 2024-09-12T03:19:14.415 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "1", 2024-09-12T03:19:14.415 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "32", 2024-09-12T03:19:14.415 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.415 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T03:19:14.415 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.415 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.415 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.415 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.415 INFO:teuthology.orchestra.run.smithi116.stdout: "username": { 2024-09-12T03:19:14.415 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "username", 2024-09-12T03:19:14.415 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.415 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.415 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.415 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.416 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.416 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.416 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.416 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "username of InfluxDB server user", 2024-09-12T03:19:14.416 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.416 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.416 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.416 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.416 INFO:teuthology.orchestra.run.smithi116.stdout: "verify_ssl": { 2024-09-12T03:19:14.416 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "verify_ssl", 2024-09-12T03:19:14.416 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.416 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.416 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.416 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "true", 2024-09-12T03:19:14.417 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.417 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.417 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.417 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:19:14.417 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.417 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.417 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.417 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.417 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.417 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.417 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.417 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "insights", 2024-09-12T03:19:14.418 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.418 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.418 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.418 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.418 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.418 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.418 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.418 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.418 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.418 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.418 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.418 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.418 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.418 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.419 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.419 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.419 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.419 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.419 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.419 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.419 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.419 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.419 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.419 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.419 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.419 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.419 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.419 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.420 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.420 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.420 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.420 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.420 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.420 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.420 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.420 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.420 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.420 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.420 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.420 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.420 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.420 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.421 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.421 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.421 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.421 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.421 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.421 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.421 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.421 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.421 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.421 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.421 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.421 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.421 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.421 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.422 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.422 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.422 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.422 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.422 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.422 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.422 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.422 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.422 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.422 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.422 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.422 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.422 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.423 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.423 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.423 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.423 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.423 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.423 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.423 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.423 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "iostat", 2024-09-12T03:19:14.423 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.423 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.423 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.423 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.423 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.423 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.424 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.424 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.424 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.424 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.424 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.424 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.424 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.424 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.424 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.424 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.424 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.424 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.424 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.425 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.425 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.425 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.425 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.425 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.425 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.425 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.425 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.425 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.425 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.425 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.425 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.425 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.426 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.426 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.426 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.426 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.426 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.426 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.426 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.426 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.426 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.426 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.426 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.426 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.426 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.426 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.427 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.427 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.427 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.427 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.427 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.427 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.427 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.427 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.427 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.427 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.427 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.427 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.427 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.428 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.428 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.428 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.428 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.428 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.428 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.428 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.428 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.428 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.428 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.428 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.428 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.428 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.428 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.429 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.429 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.429 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.429 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "k8sevents", 2024-09-12T03:19:14.429 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.429 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.429 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.429 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_event_retention_days": { 2024-09-12T03:19:14.429 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ceph_event_retention_days", 2024-09-12T03:19:14.429 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.429 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.429 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.429 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "7", 2024-09-12T03:19:14.429 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.430 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.430 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.430 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T03:19:14.430 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.430 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.430 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.430 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.430 INFO:teuthology.orchestra.run.smithi116.stdout: "config_check_secs": { 2024-09-12T03:19:14.430 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "config_check_secs", 2024-09-12T03:19:14.430 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.430 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.430 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.430 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "10", 2024-09-12T03:19:14.430 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "10", 2024-09-12T03:19:14.431 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.431 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.431 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T03:19:14.431 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.431 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.431 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.431 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.431 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.431 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.431 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.431 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.431 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.431 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.432 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.432 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.432 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.432 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.432 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.432 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.432 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.432 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.432 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.432 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.432 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.432 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.432 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.432 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.433 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.433 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.433 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.433 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.433 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.433 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.433 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.433 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.433 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.433 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.433 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.433 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.433 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.434 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.434 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.434 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.434 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.434 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.434 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.434 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.434 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.434 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.434 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.434 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.434 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.434 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.434 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.435 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.435 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.435 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.435 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.435 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.435 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.435 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.435 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.435 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.435 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.435 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.435 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.435 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.435 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.436 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.436 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.436 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.436 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.436 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.436 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.436 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.436 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.436 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.436 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.436 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.436 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.436 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "localpool", 2024-09-12T03:19:14.437 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.437 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.437 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.437 INFO:teuthology.orchestra.run.smithi116.stdout: "failure_domain": { 2024-09-12T03:19:14.437 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "failure_domain", 2024-09-12T03:19:14.437 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.437 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.437 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.437 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "host", 2024-09-12T03:19:14.437 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.437 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.437 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.437 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "failure domain for any created local pool", 2024-09-12T03:19:14.437 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T03:19:14.438 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.438 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.438 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.438 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.438 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.438 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.438 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.438 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.438 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.438 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.438 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.438 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.438 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.438 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.439 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.439 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.439 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.439 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.439 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.439 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.439 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.439 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.439 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.439 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.439 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.439 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.439 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.440 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.440 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.440 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.440 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.440 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.440 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.440 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.440 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.440 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.440 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.440 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.440 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.440 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.440 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.441 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.441 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.441 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.441 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.441 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.441 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.441 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.441 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.441 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.441 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.441 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.441 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.441 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.441 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.442 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.442 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.442 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.442 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.442 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.442 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.442 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.442 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.442 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.442 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.442 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.442 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.442 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.443 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.443 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.443 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.443 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.443 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.443 INFO:teuthology.orchestra.run.smithi116.stdout: "min_size": { 2024-09-12T03:19:14.443 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "min_size", 2024-09-12T03:19:14.443 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.443 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.443 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.443 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.443 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.443 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.443 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.444 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "default min_size for any created local pool", 2024-09-12T03:19:14.444 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T03:19:14.444 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.444 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.444 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.444 INFO:teuthology.orchestra.run.smithi116.stdout: "num_rep": { 2024-09-12T03:19:14.444 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "num_rep", 2024-09-12T03:19:14.444 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.444 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.444 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.444 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "3", 2024-09-12T03:19:14.444 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.444 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.444 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.445 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "default replica count for any created local pool", 2024-09-12T03:19:14.445 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.445 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.445 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.445 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.445 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num": { 2024-09-12T03:19:14.445 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "pg_num", 2024-09-12T03:19:14.445 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.445 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.445 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.445 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "128", 2024-09-12T03:19:14.446 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.446 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.446 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.446 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T03:19:14.446 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.446 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.446 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.446 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.446 INFO:teuthology.orchestra.run.smithi116.stdout: "prefix": { 2024-09-12T03:19:14.446 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "prefix", 2024-09-12T03:19:14.446 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.446 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.446 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.447 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.447 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.447 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.447 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.447 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "name prefix for any created local pool", 2024-09-12T03:19:14.447 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.447 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.447 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.447 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.447 INFO:teuthology.orchestra.run.smithi116.stdout: "subtree": { 2024-09-12T03:19:14.447 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "subtree", 2024-09-12T03:19:14.447 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.447 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.448 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.448 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "rack", 2024-09-12T03:19:14.448 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.448 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.448 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.448 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T03:19:14.448 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T03:19:14.448 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.448 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.448 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.448 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.448 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.448 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.448 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "mds_autoscaler", 2024-09-12T03:19:14.449 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.449 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.449 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.449 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.449 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.449 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.449 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.449 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.449 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.449 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.449 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.449 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.449 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.450 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.450 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.450 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.450 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.450 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.450 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.450 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.450 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.450 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.450 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.450 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.450 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.450 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.450 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.451 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.451 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.451 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.451 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.451 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.451 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.451 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.451 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.451 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.451 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.451 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.451 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.451 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.452 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.452 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.452 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.452 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.452 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.452 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.452 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.452 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.452 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.452 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.452 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.452 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.452 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.452 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.453 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.453 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.453 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.453 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.453 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.453 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.453 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.453 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.453 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.453 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.453 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.453 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.453 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.454 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.454 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.454 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.454 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.454 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.454 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.454 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.454 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.454 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.454 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "mirroring", 2024-09-12T03:19:14.454 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.454 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.454 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.454 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.455 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.455 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.455 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.455 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.455 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.455 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.455 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.455 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.455 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.455 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.455 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.455 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.455 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.455 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.456 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.456 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.456 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.456 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.456 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.456 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.456 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.456 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.456 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.456 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.456 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.456 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.456 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.456 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.457 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.457 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.457 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.457 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.457 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.457 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.457 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.457 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.457 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.457 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.457 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.457 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.457 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.457 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.458 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.458 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.458 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.458 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.458 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.458 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.458 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.458 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.458 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.458 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.458 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.458 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.458 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.458 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.459 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.459 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.459 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.459 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.459 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.459 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.459 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.459 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.459 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.459 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.459 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.459 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.459 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.459 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.460 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.460 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.460 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "nfs", 2024-09-12T03:19:14.460 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.460 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.460 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.460 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.460 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.460 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.460 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.460 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.460 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.461 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.461 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.461 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.461 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.461 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.461 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.461 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.461 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.461 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.461 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.461 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.461 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.461 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.461 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.462 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.462 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.462 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.462 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.462 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.462 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.462 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.462 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.462 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.462 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.462 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.462 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.462 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.462 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.463 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.463 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.463 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.463 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.463 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.463 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.463 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.463 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.463 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.463 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.463 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.463 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.463 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.463 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.464 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.464 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.464 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.464 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.464 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.464 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.464 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.464 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.464 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.464 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.464 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.464 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.464 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.465 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.465 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.465 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.465 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.465 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.465 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.465 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.465 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.465 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.465 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.465 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.465 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.465 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "orchestrator", 2024-09-12T03:19:14.465 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.466 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.466 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.466 INFO:teuthology.orchestra.run.smithi116.stdout: "fail_fs": { 2024-09-12T03:19:14.466 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "fail_fs", 2024-09-12T03:19:14.466 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.466 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.466 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.466 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.466 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.466 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.466 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.466 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T03:19:14.466 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.466 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.467 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.467 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.467 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.467 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.467 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.467 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.467 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.467 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.467 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.467 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.467 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.467 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.467 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.467 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.468 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.468 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.468 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.468 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.468 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.468 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.468 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.468 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.468 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.468 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.468 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.468 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.468 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.468 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.469 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.469 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.469 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.469 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.469 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.469 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.469 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.469 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.469 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.469 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.470 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.470 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.470 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.470 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.470 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.470 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.470 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.470 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.470 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.470 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.470 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.470 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.470 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.471 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.471 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.471 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.471 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.471 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.471 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.471 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.471 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.471 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.471 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.471 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.471 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.471 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.471 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.472 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.472 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.472 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.472 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.472 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.472 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.472 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.472 INFO:teuthology.orchestra.run.smithi116.stdout: "orchestrator": { 2024-09-12T03:19:14.472 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "orchestrator", 2024-09-12T03:19:14.472 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.472 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.472 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.472 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.472 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.473 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.473 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.473 INFO:teuthology.orchestra.run.smithi116.stdout: "cephadm", 2024-09-12T03:19:14.473 INFO:teuthology.orchestra.run.smithi116.stdout: "rook", 2024-09-12T03:19:14.473 INFO:teuthology.orchestra.run.smithi116.stdout: "test_orchestrator" 2024-09-12T03:19:14.473 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.473 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Orchestrator backend", 2024-09-12T03:19:14.473 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.473 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.473 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.473 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.473 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.473 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.473 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.474 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "osd_perf_query", 2024-09-12T03:19:14.474 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.474 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.474 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.474 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.474 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.474 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.474 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.474 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.474 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.474 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.474 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.475 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.475 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.475 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.475 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.475 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.475 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.475 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.475 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.475 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.475 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.475 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.475 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.475 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.476 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.476 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.476 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.476 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.476 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.476 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.476 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.476 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.476 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.476 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.476 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.476 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.476 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.476 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.477 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.477 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.477 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.477 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.477 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.477 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.477 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.477 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.477 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.477 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.477 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.477 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.477 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.478 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.478 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.478 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.478 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.478 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.478 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.478 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.478 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.478 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.478 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.478 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.478 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.478 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.478 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.479 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.479 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.479 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.479 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.479 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.479 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.479 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.479 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.479 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.479 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.479 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.479 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "osd_support", 2024-09-12T03:19:14.479 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.480 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.480 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.480 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.480 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.480 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.480 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.480 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.480 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.480 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.480 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.480 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.480 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.480 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.481 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.481 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.481 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.481 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.481 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.481 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.481 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.481 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.481 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.481 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.481 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.481 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.481 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.482 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.482 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.482 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.482 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.482 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.482 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.482 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.482 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.482 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.482 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.482 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.482 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.482 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.482 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.483 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.483 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.483 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.483 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.483 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.483 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.483 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.483 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.483 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.483 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.483 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.483 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.483 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.483 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.484 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.484 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.484 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.484 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.484 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.484 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.484 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.484 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.484 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.484 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.484 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.484 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.484 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.485 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.485 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.485 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.485 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.485 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.485 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.485 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.485 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.485 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "pg_autoscaler", 2024-09-12T03:19:14.485 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.485 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.485 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.485 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.486 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.486 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.486 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.486 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.486 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.486 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.486 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.486 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.486 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.486 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.486 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.486 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.486 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.486 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.487 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.487 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.487 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.487 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.487 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.487 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.487 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.487 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.487 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.487 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.487 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.487 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.487 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.487 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.488 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.488 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.488 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.488 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.488 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.488 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.488 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.488 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.488 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.488 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.488 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.488 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.488 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.489 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.489 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.489 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.489 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.489 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.489 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.489 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.489 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.489 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.489 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.489 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.489 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.489 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.490 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.490 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.490 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.490 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.490 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.490 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.490 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.490 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.490 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.490 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.490 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.490 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.490 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.491 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.491 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.491 INFO:teuthology.orchestra.run.smithi116.stdout: "sleep_interval": { 2024-09-12T03:19:14.491 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "sleep_interval", 2024-09-12T03:19:14.491 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.491 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.491 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.491 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "60", 2024-09-12T03:19:14.491 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.491 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.491 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.491 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.491 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.491 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.492 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.492 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.492 INFO:teuthology.orchestra.run.smithi116.stdout: "threshold": { 2024-09-12T03:19:14.492 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "threshold", 2024-09-12T03:19:14.492 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "float", 2024-09-12T03:19:14.492 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.492 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.492 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "3.0", 2024-09-12T03:19:14.492 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "1.0", 2024-09-12T03:19:14.492 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.492 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.492 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "scaling threshold", 2024-09-12T03:19:14.492 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T03:19:14.493 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.493 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.493 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.493 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.493 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.493 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.493 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "progress", 2024-09-12T03:19:14.493 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.493 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.493 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.493 INFO:teuthology.orchestra.run.smithi116.stdout: "allow_pg_recovery_event": { 2024-09-12T03:19:14.493 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "allow_pg_recovery_event", 2024-09-12T03:19:14.493 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.494 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.494 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.494 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.494 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.494 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.494 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.494 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T03:19:14.494 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.494 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.494 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.494 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.494 INFO:teuthology.orchestra.run.smithi116.stdout: "enabled": { 2024-09-12T03:19:14.494 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "enabled", 2024-09-12T03:19:14.494 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.495 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.495 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.495 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.495 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.495 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.495 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.495 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.495 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.495 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.495 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.495 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.495 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.495 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.496 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.496 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.496 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.496 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.496 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.496 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.496 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.496 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.496 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.496 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.496 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.496 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.496 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.497 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.497 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.497 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.497 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.497 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.497 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.497 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.497 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.497 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.497 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.497 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.497 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.497 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.497 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.498 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.498 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.498 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.498 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.498 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.498 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.498 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.498 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.498 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.498 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.498 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.498 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.498 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.498 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.499 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.499 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.499 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.499 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.499 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.499 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.499 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.499 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.499 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.499 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.499 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.499 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.499 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.499 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.500 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.500 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.500 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.500 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.500 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.500 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.500 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.500 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.500 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.500 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.500 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.500 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.500 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.501 INFO:teuthology.orchestra.run.smithi116.stdout: "max_completed_events": { 2024-09-12T03:19:14.501 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "max_completed_events", 2024-09-12T03:19:14.501 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.501 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.501 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.501 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "50", 2024-09-12T03:19:14.501 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.501 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.501 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.501 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "number of past completed events to remember", 2024-09-12T03:19:14.501 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.501 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.501 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.501 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.502 INFO:teuthology.orchestra.run.smithi116.stdout: "sleep_interval": { 2024-09-12T03:19:14.502 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "sleep_interval", 2024-09-12T03:19:14.502 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.502 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.502 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.502 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "5", 2024-09-12T03:19:14.502 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.502 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.502 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.502 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "how long the module is going to sleep", 2024-09-12T03:19:14.502 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.502 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.502 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.503 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.503 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.503 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.503 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.503 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "prometheus", 2024-09-12T03:19:14.503 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.503 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.503 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.503 INFO:teuthology.orchestra.run.smithi116.stdout: "cache": { 2024-09-12T03:19:14.503 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "cache", 2024-09-12T03:19:14.503 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.503 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.503 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.504 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.504 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.504 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.504 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.504 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.504 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.504 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.504 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.504 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.504 INFO:teuthology.orchestra.run.smithi116.stdout: "exclude_perf_counters": { 2024-09-12T03:19:14.504 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "exclude_perf_counters", 2024-09-12T03:19:14.504 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.504 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.505 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.505 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.505 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.505 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.505 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.505 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T03:19:14.505 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T03:19:14.505 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.505 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.505 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.505 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.505 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.505 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.506 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.506 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.506 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.506 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.506 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.506 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.506 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.506 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.506 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.506 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.506 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.506 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.506 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.506 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.507 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.507 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.507 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.507 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.507 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.507 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.507 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.507 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.507 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.507 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.507 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.507 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.507 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.508 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.508 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.508 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.508 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.508 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.508 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.508 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.508 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.508 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.508 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.508 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.508 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.508 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.508 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.509 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.509 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.509 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.509 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.509 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.509 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.509 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.509 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.509 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.509 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.509 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.509 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.509 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.510 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.510 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.510 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.510 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.510 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.510 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.510 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.510 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.510 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.510 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.510 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.510 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.510 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.510 INFO:teuthology.orchestra.run.smithi116.stdout: "rbd_stats_pools": { 2024-09-12T03:19:14.511 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rbd_stats_pools", 2024-09-12T03:19:14.511 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.511 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.511 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.511 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.511 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.511 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.511 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.511 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.511 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.511 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.511 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.511 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.511 INFO:teuthology.orchestra.run.smithi116.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T03:19:14.512 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T03:19:14.512 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.512 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.512 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.512 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "300", 2024-09-12T03:19:14.512 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.512 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.512 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.512 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.512 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.512 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.512 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.512 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.513 INFO:teuthology.orchestra.run.smithi116.stdout: "scrape_interval": { 2024-09-12T03:19:14.513 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "scrape_interval", 2024-09-12T03:19:14.513 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "float", 2024-09-12T03:19:14.513 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.513 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.513 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "15.0", 2024-09-12T03:19:14.513 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.513 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.513 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.513 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.513 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.513 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.513 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.513 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.514 INFO:teuthology.orchestra.run.smithi116.stdout: "server_addr": { 2024-09-12T03:19:14.514 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "server_addr", 2024-09-12T03:19:14.514 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.514 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.514 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.514 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "::", 2024-09-12T03:19:14.514 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.514 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.514 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.514 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T03:19:14.514 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.514 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.514 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.514 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.515 INFO:teuthology.orchestra.run.smithi116.stdout: "server_port": { 2024-09-12T03:19:14.515 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "server_port", 2024-09-12T03:19:14.515 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.515 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.515 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.515 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "9283", 2024-09-12T03:19:14.515 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.515 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.515 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.515 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T03:19:14.515 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.515 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.515 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.515 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.516 INFO:teuthology.orchestra.run.smithi116.stdout: "stale_cache_strategy": { 2024-09-12T03:19:14.516 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "stale_cache_strategy", 2024-09-12T03:19:14.516 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.516 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.516 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.516 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "log", 2024-09-12T03:19:14.516 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.516 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.516 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.516 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.516 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.516 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.516 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.516 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.517 INFO:teuthology.orchestra.run.smithi116.stdout: "standby_behaviour": { 2024-09-12T03:19:14.517 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "standby_behaviour", 2024-09-12T03:19:14.517 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.517 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.517 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.517 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "default", 2024-09-12T03:19:14.517 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.517 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.517 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.517 INFO:teuthology.orchestra.run.smithi116.stdout: "default", 2024-09-12T03:19:14.517 INFO:teuthology.orchestra.run.smithi116.stdout: "error" 2024-09-12T03:19:14.517 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.517 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.518 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.518 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.518 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.518 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.518 INFO:teuthology.orchestra.run.smithi116.stdout: "standby_error_status_code": { 2024-09-12T03:19:14.518 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "standby_error_status_code", 2024-09-12T03:19:14.518 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.518 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.518 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.518 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "500", 2024-09-12T03:19:14.518 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "400", 2024-09-12T03:19:14.518 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "599", 2024-09-12T03:19:14.518 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.518 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.519 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.519 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.519 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.519 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.519 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.519 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.519 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.519 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rbd_support", 2024-09-12T03:19:14.519 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.519 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.519 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.519 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.519 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.519 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.520 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.520 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.520 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.520 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.520 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.520 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.520 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.520 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.520 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.520 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.520 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.520 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.520 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.521 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.521 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.521 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.521 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.521 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.521 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.521 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.521 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.521 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.521 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.521 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.521 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.521 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.521 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.522 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.522 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.522 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.522 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.522 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.522 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.522 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.522 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.522 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.522 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.522 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.522 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.522 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.523 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.523 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.523 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.523 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.523 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.523 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.523 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.523 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.523 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.523 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.523 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.523 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.523 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.523 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.524 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.524 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.524 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.524 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.524 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.524 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.524 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.524 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.524 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.524 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.524 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.524 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.524 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.525 INFO:teuthology.orchestra.run.smithi116.stdout: "max_concurrent_snap_create": { 2024-09-12T03:19:14.525 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "max_concurrent_snap_create", 2024-09-12T03:19:14.525 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.525 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.525 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.525 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "10", 2024-09-12T03:19:14.525 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.525 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.525 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.525 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.525 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.525 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.525 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.525 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.526 INFO:teuthology.orchestra.run.smithi116.stdout: "mirror_snapshot_schedule": { 2024-09-12T03:19:14.526 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T03:19:14.526 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.526 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.526 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.526 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.526 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.526 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.526 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.526 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.526 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.526 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.526 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.526 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.527 INFO:teuthology.orchestra.run.smithi116.stdout: "trash_purge_schedule": { 2024-09-12T03:19:14.527 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "trash_purge_schedule", 2024-09-12T03:19:14.527 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.527 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.527 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.527 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.527 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.527 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.527 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.527 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.527 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.527 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.527 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.527 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.528 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.528 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.528 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.528 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "restful", 2024-09-12T03:19:14.528 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.528 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.528 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.528 INFO:teuthology.orchestra.run.smithi116.stdout: "enable_auth": { 2024-09-12T03:19:14.528 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "enable_auth", 2024-09-12T03:19:14.528 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.528 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.528 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.528 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.528 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.529 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.529 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.529 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.529 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.529 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.529 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.529 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.529 INFO:teuthology.orchestra.run.smithi116.stdout: "key_file": { 2024-09-12T03:19:14.529 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "key_file", 2024-09-12T03:19:14.529 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.529 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.529 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.529 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.530 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.530 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.530 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.530 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.530 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.530 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.530 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.530 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.530 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.530 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.530 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.530 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.530 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.530 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.530 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.531 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.531 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.531 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.531 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.531 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.531 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.531 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.531 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.531 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.531 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.531 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.531 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.531 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.532 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.532 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.532 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.532 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.532 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.532 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.532 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.532 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.532 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.532 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.532 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.532 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.532 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.533 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.533 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.533 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.533 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.533 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.533 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.533 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.533 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.533 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.533 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.533 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.533 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.533 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.533 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.534 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.534 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.534 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.534 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.534 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.534 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.534 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.534 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.534 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.534 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.534 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.534 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.534 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.534 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.535 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.535 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.535 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.535 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.535 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.535 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.535 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.535 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.535 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.535 INFO:teuthology.orchestra.run.smithi116.stdout: "server_addr": { 2024-09-12T03:19:14.535 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "server_addr", 2024-09-12T03:19:14.535 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.535 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.535 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.536 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.536 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.536 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.536 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.536 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.536 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.536 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.536 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.536 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.536 INFO:teuthology.orchestra.run.smithi116.stdout: "server_port": { 2024-09-12T03:19:14.536 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "server_port", 2024-09-12T03:19:14.536 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.536 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.537 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.537 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.537 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.537 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.537 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.537 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.537 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.537 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.537 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.537 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.537 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.537 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.537 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.537 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rgw", 2024-09-12T03:19:14.538 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.538 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.538 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.538 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.538 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.538 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.538 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.538 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.538 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.538 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.538 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.538 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.538 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.538 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.539 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.539 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.539 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.539 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.539 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.539 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.539 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.539 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.539 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.539 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.539 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.539 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.539 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.539 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.540 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.540 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.540 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.540 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.540 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.540 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.540 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.540 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.540 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.540 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.540 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.540 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.540 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.540 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.541 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.541 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.541 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.541 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.541 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.541 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.541 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.541 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.541 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.541 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.541 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.541 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.541 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.542 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.542 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.542 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.542 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.542 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.542 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.542 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.542 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.542 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.542 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.542 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.542 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.542 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.542 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.543 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.543 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.543 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.543 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.543 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.543 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.543 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.543 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rook", 2024-09-12T03:19:14.543 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.543 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.543 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.543 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.543 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.543 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.544 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.544 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.544 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.544 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.544 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.544 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.544 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.544 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.544 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.544 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.544 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.544 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.544 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.544 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.545 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.545 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.545 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.545 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.545 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.545 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.545 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.545 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.545 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.545 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.545 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.545 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.545 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.545 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.546 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.546 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.546 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.546 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.546 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.546 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.546 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.546 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.546 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.546 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.546 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.546 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.546 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.546 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.547 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.547 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.547 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.547 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.547 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.547 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.547 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.547 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.547 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.547 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.547 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.547 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.547 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.547 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.548 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.548 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.548 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.548 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.548 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.548 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.548 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.548 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.548 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.548 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.548 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.548 INFO:teuthology.orchestra.run.smithi116.stdout: "storage_class": { 2024-09-12T03:19:14.548 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "storage_class", 2024-09-12T03:19:14.549 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.549 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.552 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.552 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "local", 2024-09-12T03:19:14.553 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.553 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.553 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.553 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T03:19:14.553 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.553 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.553 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.553 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.553 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.553 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.553 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.553 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "selftest", 2024-09-12T03:19:14.553 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.554 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.554 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.554 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.554 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.554 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.554 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.554 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.554 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.554 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.554 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.554 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.554 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.554 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.554 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.554 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.555 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.555 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.555 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.555 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.555 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.555 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.555 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.555 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.555 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.555 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.555 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.555 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.555 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.556 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.556 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.556 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.556 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.556 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.556 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.556 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.556 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.556 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.556 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.556 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.556 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.556 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.556 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.557 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.557 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.557 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.557 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.557 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.557 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.557 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.557 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.557 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.557 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.557 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.557 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.557 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.557 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.558 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.558 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.558 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.558 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.558 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.558 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.558 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.558 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.558 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.558 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.558 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.558 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.558 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.558 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.559 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.559 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.559 INFO:teuthology.orchestra.run.smithi116.stdout: "roption1": { 2024-09-12T03:19:14.559 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "roption1", 2024-09-12T03:19:14.559 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.559 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.559 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.559 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.559 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.559 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.559 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.559 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.559 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.559 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.560 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.560 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.560 INFO:teuthology.orchestra.run.smithi116.stdout: "roption2": { 2024-09-12T03:19:14.560 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "roption2", 2024-09-12T03:19:14.560 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.560 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.560 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.560 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "xyz", 2024-09-12T03:19:14.560 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.560 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.560 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.560 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.560 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.560 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.561 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.561 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.561 INFO:teuthology.orchestra.run.smithi116.stdout: "rwoption1": { 2024-09-12T03:19:14.561 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rwoption1", 2024-09-12T03:19:14.561 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.561 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.561 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.561 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.561 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.561 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.561 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.561 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.561 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.561 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.562 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.562 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.562 INFO:teuthology.orchestra.run.smithi116.stdout: "rwoption2": { 2024-09-12T03:19:14.562 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rwoption2", 2024-09-12T03:19:14.562 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.562 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.562 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.562 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.562 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.562 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.562 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.562 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.562 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.562 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.563 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.563 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.563 INFO:teuthology.orchestra.run.smithi116.stdout: "rwoption3": { 2024-09-12T03:19:14.563 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rwoption3", 2024-09-12T03:19:14.563 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "float", 2024-09-12T03:19:14.563 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.563 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.563 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.563 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.563 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.563 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.563 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.563 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.564 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.564 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.564 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.564 INFO:teuthology.orchestra.run.smithi116.stdout: "rwoption4": { 2024-09-12T03:19:14.564 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rwoption4", 2024-09-12T03:19:14.564 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.564 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.564 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.564 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.564 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.564 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.564 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.564 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.565 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.565 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.565 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.565 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.565 INFO:teuthology.orchestra.run.smithi116.stdout: "rwoption5": { 2024-09-12T03:19:14.565 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rwoption5", 2024-09-12T03:19:14.565 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.565 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.565 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.565 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.565 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.565 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.565 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.565 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.566 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.566 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.566 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.566 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.566 INFO:teuthology.orchestra.run.smithi116.stdout: "rwoption6": { 2024-09-12T03:19:14.566 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rwoption6", 2024-09-12T03:19:14.566 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.566 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.566 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.566 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.566 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.566 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.566 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.566 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.567 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.567 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.567 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.567 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.567 INFO:teuthology.orchestra.run.smithi116.stdout: "rwoption7": { 2024-09-12T03:19:14.567 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rwoption7", 2024-09-12T03:19:14.567 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.567 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.567 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.567 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.567 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "1", 2024-09-12T03:19:14.567 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "42", 2024-09-12T03:19:14.567 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.568 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.568 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.568 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.568 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.568 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.568 INFO:teuthology.orchestra.run.smithi116.stdout: "testkey": { 2024-09-12T03:19:14.568 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "testkey", 2024-09-12T03:19:14.568 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.568 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.568 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.568 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.568 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.568 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.568 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.569 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.569 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.569 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.569 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.569 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.569 INFO:teuthology.orchestra.run.smithi116.stdout: "testlkey": { 2024-09-12T03:19:14.569 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "testlkey", 2024-09-12T03:19:14.569 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.569 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.569 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.569 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.570 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.570 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.570 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.570 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.570 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.570 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.570 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.570 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.570 INFO:teuthology.orchestra.run.smithi116.stdout: "testnewline": { 2024-09-12T03:19:14.570 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "testnewline", 2024-09-12T03:19:14.570 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.570 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.571 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.571 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.571 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.571 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.571 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.571 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.571 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.571 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.571 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.571 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.571 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.571 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.571 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.571 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "snap_schedule", 2024-09-12T03:19:14.572 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.572 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.572 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.572 INFO:teuthology.orchestra.run.smithi116.stdout: "allow_m_granularity": { 2024-09-12T03:19:14.572 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "allow_m_granularity", 2024-09-12T03:19:14.572 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.572 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.572 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.572 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.572 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.572 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.572 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.572 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T03:19:14.573 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.573 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.573 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.573 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.573 INFO:teuthology.orchestra.run.smithi116.stdout: "dump_on_update": { 2024-09-12T03:19:14.573 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "dump_on_update", 2024-09-12T03:19:14.573 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.573 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.573 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.573 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.573 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.573 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.573 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.573 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "dump database to debug log on update", 2024-09-12T03:19:14.574 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.574 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.574 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.574 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.574 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.574 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.574 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.574 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.574 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.574 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.574 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.574 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.574 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.574 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.575 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.575 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.575 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.575 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.575 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.575 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.575 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.575 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.575 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.575 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.575 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.575 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.575 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.575 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.576 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.576 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.576 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.576 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.576 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.576 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.576 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.576 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.576 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.576 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.576 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.576 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.576 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.577 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.577 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.577 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.577 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.577 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.577 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.577 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.577 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.577 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.577 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.577 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.577 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.577 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.578 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.578 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.578 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.578 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.578 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.578 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.578 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.578 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.578 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.578 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.578 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.578 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.578 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.578 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.579 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.579 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.579 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.579 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.579 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.579 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.579 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.579 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.579 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.579 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "stats", 2024-09-12T03:19:14.579 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.579 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.579 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.580 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.580 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.580 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.580 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.580 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.580 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.580 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.580 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.580 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.580 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.580 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.580 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.580 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.581 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.581 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.581 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.581 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.581 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.581 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.581 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.581 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.581 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.581 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.581 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.581 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.581 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.581 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.582 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.582 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.582 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.582 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.582 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.582 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.582 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.582 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.582 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.582 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.582 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.582 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.582 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.583 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.583 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.583 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.583 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.583 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.583 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.583 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.583 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.583 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.583 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.583 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.583 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.583 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.583 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.584 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.584 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.584 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.584 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.584 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.584 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.584 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.584 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.584 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.584 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.584 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.584 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.584 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.585 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.585 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.585 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.585 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.585 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.585 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.585 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "status", 2024-09-12T03:19:14.585 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.585 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.585 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.585 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.585 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.585 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.585 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.586 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.586 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.586 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.586 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.586 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.586 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.586 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.586 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.586 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.586 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.586 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.586 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.586 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.586 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.587 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.587 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.587 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.587 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.587 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.587 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.587 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.587 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.587 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.587 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.587 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.587 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.587 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.588 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.588 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.588 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.588 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.588 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.588 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.588 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.588 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.588 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.588 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.588 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.588 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.588 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.588 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.589 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.589 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.589 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.589 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.589 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.589 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.589 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.589 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.589 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.589 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.589 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.589 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.589 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.590 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.590 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.590 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.590 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.590 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.590 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.590 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.590 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.590 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.590 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.590 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.590 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.590 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.590 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.591 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.591 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "telegraf", 2024-09-12T03:19:14.591 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.591 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.591 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.591 INFO:teuthology.orchestra.run.smithi116.stdout: "address": { 2024-09-12T03:19:14.591 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "address", 2024-09-12T03:19:14.591 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.591 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.591 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.591 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T03:19:14.591 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.591 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.591 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.592 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.592 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.592 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.592 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.592 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.592 INFO:teuthology.orchestra.run.smithi116.stdout: "interval": { 2024-09-12T03:19:14.592 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "interval", 2024-09-12T03:19:14.592 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.592 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.592 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.592 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "15", 2024-09-12T03:19:14.592 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.592 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.593 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.593 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.593 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.593 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.593 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.593 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.593 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.593 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.593 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.593 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.593 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.593 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.593 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.594 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.594 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.594 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.594 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.594 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.594 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.594 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.594 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.594 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.594 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.594 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.594 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.594 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.594 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.595 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.595 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.595 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.595 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.595 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.595 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.595 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.595 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.595 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.595 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.595 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.595 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.595 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.595 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.596 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.596 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.596 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.596 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.596 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.596 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.596 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.596 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.596 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.596 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.596 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.596 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.596 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.596 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.597 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.597 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.597 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.597 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.597 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.597 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.597 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.597 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.597 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.597 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.597 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.597 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.597 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.598 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.598 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.598 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.598 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.598 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.598 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.598 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.598 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.598 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.598 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.598 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.598 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "telemetry", 2024-09-12T03:19:14.598 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.598 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.599 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.599 INFO:teuthology.orchestra.run.smithi116.stdout: "channel_basic": { 2024-09-12T03:19:14.599 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "channel_basic", 2024-09-12T03:19:14.599 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.599 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.599 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.599 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.599 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.599 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.599 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.599 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T03:19:14.599 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.599 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.599 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.600 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.600 INFO:teuthology.orchestra.run.smithi116.stdout: "channel_crash": { 2024-09-12T03:19:14.600 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "channel_crash", 2024-09-12T03:19:14.600 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.600 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.600 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.600 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.600 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.600 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.600 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.600 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T03:19:14.600 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.600 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.601 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.601 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.601 INFO:teuthology.orchestra.run.smithi116.stdout: "channel_device": { 2024-09-12T03:19:14.601 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "channel_device", 2024-09-12T03:19:14.601 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.601 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.601 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.601 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.601 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.601 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.601 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.601 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T03:19:14.601 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.601 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.602 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.602 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.602 INFO:teuthology.orchestra.run.smithi116.stdout: "channel_ident": { 2024-09-12T03:19:14.602 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "channel_ident", 2024-09-12T03:19:14.602 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.602 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.602 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.602 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.602 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.602 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.602 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.602 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T03:19:14.602 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.602 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.603 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.603 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.603 INFO:teuthology.orchestra.run.smithi116.stdout: "channel_perf": { 2024-09-12T03:19:14.603 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "channel_perf", 2024-09-12T03:19:14.603 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.603 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.603 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.603 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.603 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.603 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.603 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.603 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T03:19:14.603 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.603 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.603 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.604 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.604 INFO:teuthology.orchestra.run.smithi116.stdout: "contact": { 2024-09-12T03:19:14.604 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "contact", 2024-09-12T03:19:14.604 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.604 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.604 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.604 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.604 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.604 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.604 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.604 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.604 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.604 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.605 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.605 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.605 INFO:teuthology.orchestra.run.smithi116.stdout: "description": { 2024-09-12T03:19:14.605 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "description", 2024-09-12T03:19:14.605 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.605 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.605 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.605 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.605 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.605 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.605 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.605 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.605 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.605 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.605 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.606 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.606 INFO:teuthology.orchestra.run.smithi116.stdout: "device_url": { 2024-09-12T03:19:14.606 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "device_url", 2024-09-12T03:19:14.606 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.606 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.606 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.606 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T03:19:14.606 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.606 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.606 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.606 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.606 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.606 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.607 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.607 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.607 INFO:teuthology.orchestra.run.smithi116.stdout: "enabled": { 2024-09-12T03:19:14.607 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "enabled", 2024-09-12T03:19:14.607 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.607 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.607 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.607 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.607 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.607 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.607 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.607 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.607 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.607 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.608 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.608 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.608 INFO:teuthology.orchestra.run.smithi116.stdout: "interval": { 2024-09-12T03:19:14.608 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "interval", 2024-09-12T03:19:14.608 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.608 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.608 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.608 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "24", 2024-09-12T03:19:14.608 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "8", 2024-09-12T03:19:14.608 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.608 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.608 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.608 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.608 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.609 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.609 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.609 INFO:teuthology.orchestra.run.smithi116.stdout: "last_opt_revision": { 2024-09-12T03:19:14.609 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "last_opt_revision", 2024-09-12T03:19:14.609 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.609 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.609 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.609 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "1", 2024-09-12T03:19:14.609 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.609 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.609 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.609 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.609 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.610 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.610 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.610 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.610 INFO:teuthology.orchestra.run.smithi116.stdout: "leaderboard": { 2024-09-12T03:19:14.610 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "leaderboard", 2024-09-12T03:19:14.610 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.610 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.610 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.610 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.610 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.610 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.610 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.610 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.611 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.611 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.611 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.611 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.611 INFO:teuthology.orchestra.run.smithi116.stdout: "leaderboard_description": { 2024-09-12T03:19:14.611 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "leaderboard_description", 2024-09-12T03:19:14.611 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.611 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.611 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.611 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.611 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.611 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.611 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.611 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.612 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.612 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.612 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.612 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.612 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.612 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.612 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.612 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.612 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.612 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.612 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.612 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.612 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.612 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.613 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.613 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.613 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.613 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.613 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.613 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.613 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.613 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.613 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.613 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.613 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.613 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.613 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.614 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.614 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.614 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.614 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.614 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.614 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.614 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.614 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.614 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.614 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.614 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.614 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.614 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.614 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.615 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.615 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.615 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.615 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.615 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.615 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.615 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.615 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.615 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.615 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.615 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.615 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.615 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.615 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.616 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.616 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.616 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.616 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.616 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.616 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.616 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.616 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.616 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.616 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.616 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.616 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.616 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.617 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.617 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.617 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.617 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.617 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.617 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.617 INFO:teuthology.orchestra.run.smithi116.stdout: "organization": { 2024-09-12T03:19:14.617 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "organization", 2024-09-12T03:19:14.617 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.617 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.617 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.617 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.617 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.618 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.618 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.618 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.618 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.618 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.618 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.618 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.618 INFO:teuthology.orchestra.run.smithi116.stdout: "proxy": { 2024-09-12T03:19:14.618 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "proxy", 2024-09-12T03:19:14.618 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.618 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.618 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.618 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.618 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.619 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.619 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.619 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.619 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.619 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.619 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.619 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.619 INFO:teuthology.orchestra.run.smithi116.stdout: "url": { 2024-09-12T03:19:14.619 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "url", 2024-09-12T03:19:14.619 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.619 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.619 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.619 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T03:19:14.620 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.620 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.620 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.620 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.620 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.620 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.620 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.620 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.620 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.620 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.620 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.620 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "test_orchestrator", 2024-09-12T03:19:14.620 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.621 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.621 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.621 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.621 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.621 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.621 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.621 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.621 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.621 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.621 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.621 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.621 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.621 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.621 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.622 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.622 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.622 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.622 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.622 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.622 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.622 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.622 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.622 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.622 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.622 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.622 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.622 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.622 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.623 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.623 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.623 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.623 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.623 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.623 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.623 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.623 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.623 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.623 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.623 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.623 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.623 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.623 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.624 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.624 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.624 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.624 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.624 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.624 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.624 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.624 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.624 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.624 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.624 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.624 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.624 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.625 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.625 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.625 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.625 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.625 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.625 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.625 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.625 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.625 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.625 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.625 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.625 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.625 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.626 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.626 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.626 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.626 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.626 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.626 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.626 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.626 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "volumes", 2024-09-12T03:19:14.626 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.626 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.626 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.626 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.626 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.626 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.627 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.627 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.627 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.627 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.627 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.627 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.627 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.627 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.627 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.627 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.627 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.627 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.627 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.627 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.628 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.628 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.628 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.628 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.628 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.628 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.628 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.628 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.628 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.628 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.628 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.628 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.628 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.629 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.629 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.629 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.629 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.629 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.629 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.629 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.629 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.629 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.629 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.629 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.629 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.629 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.629 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.630 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.630 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.630 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.630 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.630 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.630 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.630 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.630 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.630 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.630 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.630 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.630 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.630 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.631 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.631 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.631 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.631 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.631 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.631 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.631 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.631 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.631 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.631 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.631 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.631 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.631 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.631 INFO:teuthology.orchestra.run.smithi116.stdout: "max_concurrent_clones": { 2024-09-12T03:19:14.632 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "max_concurrent_clones", 2024-09-12T03:19:14.632 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.632 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.632 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.632 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "4", 2024-09-12T03:19:14.632 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.632 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.632 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.632 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T03:19:14.632 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.632 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.632 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.632 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.632 INFO:teuthology.orchestra.run.smithi116.stdout: "periodic_async_work": { 2024-09-12T03:19:14.633 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "periodic_async_work", 2024-09-12T03:19:14.633 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.633 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.633 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.633 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.633 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.633 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.633 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.633 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Periodically check for async work", 2024-09-12T03:19:14.633 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.633 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.633 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.633 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.634 INFO:teuthology.orchestra.run.smithi116.stdout: "snapshot_clone_delay": { 2024-09-12T03:19:14.634 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "snapshot_clone_delay", 2024-09-12T03:19:14.634 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.634 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.634 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.634 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "0", 2024-09-12T03:19:14.634 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.634 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.634 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.634 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T03:19:14.634 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.634 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.634 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.635 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.635 INFO:teuthology.orchestra.run.smithi116.stdout: "snapshot_clone_no_wait": { 2024-09-12T03:19:14.635 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T03:19:14.635 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.635 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.635 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.635 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.635 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.635 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.635 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.635 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T03:19:14.635 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.635 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.635 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.636 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.636 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.636 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.636 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.636 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "zabbix", 2024-09-12T03:19:14.636 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.636 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.636 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.636 INFO:teuthology.orchestra.run.smithi116.stdout: "discovery_interval": { 2024-09-12T03:19:14.636 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "discovery_interval", 2024-09-12T03:19:14.636 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "uint", 2024-09-12T03:19:14.636 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.636 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.637 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "100", 2024-09-12T03:19:14.637 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.637 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.637 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.637 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.637 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.637 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.637 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.637 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.637 INFO:teuthology.orchestra.run.smithi116.stdout: "identifier": { 2024-09-12T03:19:14.637 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "identifier", 2024-09-12T03:19:14.637 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.637 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.638 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.638 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.638 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.638 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.638 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.638 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.638 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.638 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.638 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.638 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.638 INFO:teuthology.orchestra.run.smithi116.stdout: "interval": { 2024-09-12T03:19:14.638 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "interval", 2024-09-12T03:19:14.638 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.638 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.639 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.639 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "60", 2024-09-12T03:19:14.639 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.639 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.639 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.639 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.639 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.639 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.639 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.639 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.639 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.639 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.639 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.640 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.640 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.640 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.640 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.640 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.640 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.640 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.640 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.640 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.640 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.640 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.640 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.640 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.640 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.641 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.641 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.641 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.641 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.641 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.641 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.641 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.641 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.641 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.641 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.641 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.641 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.641 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.642 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.642 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.642 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.642 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.642 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.642 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.642 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.642 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.642 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.642 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.642 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.642 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.642 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.642 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.643 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.643 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.643 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.643 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.643 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.643 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.643 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.643 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.643 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.643 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.643 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.643 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.643 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.644 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.644 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.644 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.644 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.644 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.644 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.644 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.644 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.644 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.644 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.644 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.644 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.644 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.645 INFO:teuthology.orchestra.run.smithi116.stdout: "zabbix_host": { 2024-09-12T03:19:14.645 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "zabbix_host", 2024-09-12T03:19:14.645 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.645 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.645 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.645 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.645 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.645 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.645 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.645 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.645 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.645 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.645 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.645 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.646 INFO:teuthology.orchestra.run.smithi116.stdout: "zabbix_port": { 2024-09-12T03:19:14.646 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "zabbix_port", 2024-09-12T03:19:14.646 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.646 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.646 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.646 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "10051", 2024-09-12T03:19:14.646 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.646 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.646 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.646 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.646 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.646 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.646 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.646 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.647 INFO:teuthology.orchestra.run.smithi116.stdout: "zabbix_sender": { 2024-09-12T03:19:14.647 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "zabbix_sender", 2024-09-12T03:19:14.647 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.647 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.647 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.647 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T03:19:14.647 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.647 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.647 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.647 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.647 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.647 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.647 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.647 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.648 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.648 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.648 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2024-09-12T03:19:14.648 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.648 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.648 INFO:teuthology.orchestra.run.smithi116.stdout: "modules": [ 2024-09-12T03:19:14.648 INFO:teuthology.orchestra.run.smithi116.stdout: "cephadm", 2024-09-12T03:19:14.648 INFO:teuthology.orchestra.run.smithi116.stdout: "dashboard", 2024-09-12T03:19:14.648 INFO:teuthology.orchestra.run.smithi116.stdout: "iostat", 2024-09-12T03:19:14.648 INFO:teuthology.orchestra.run.smithi116.stdout: "nfs", 2024-09-12T03:19:14.648 INFO:teuthology.orchestra.run.smithi116.stdout: "restful" 2024-09-12T03:19:14.648 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.648 INFO:teuthology.orchestra.run.smithi116.stdout: "available_modules": [ 2024-09-12T03:19:14.649 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.649 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "alerts", 2024-09-12T03:19:14.649 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.649 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.649 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.649 INFO:teuthology.orchestra.run.smithi116.stdout: "interval": { 2024-09-12T03:19:14.649 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "interval", 2024-09-12T03:19:14.649 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.649 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.649 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.649 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "60", 2024-09-12T03:19:14.649 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.650 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.650 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.650 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T03:19:14.650 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.650 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.650 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.650 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.650 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.650 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.650 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.650 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.650 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.650 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.650 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.651 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.651 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.651 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.651 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.651 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.651 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.651 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.651 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.651 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.651 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.651 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.651 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.651 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.651 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.652 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.652 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.652 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.652 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.652 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.652 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.652 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.652 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.652 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.652 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.652 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.652 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.652 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.653 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.653 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.653 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.653 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.653 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.653 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.653 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.653 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.653 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.653 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.653 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.653 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.653 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.654 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.654 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.654 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.654 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.654 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.654 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.654 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.654 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.654 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.654 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.654 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.654 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.654 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.654 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.655 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.655 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.655 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.655 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.655 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.655 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.655 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.655 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.655 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.655 INFO:teuthology.orchestra.run.smithi116.stdout: "smtp_destination": { 2024-09-12T03:19:14.655 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "smtp_destination", 2024-09-12T03:19:14.655 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.655 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.655 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.656 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.656 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.656 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.656 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.656 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Email address to send alerts to", 2024-09-12T03:19:14.656 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.656 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.656 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.656 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.656 INFO:teuthology.orchestra.run.smithi116.stdout: "smtp_from_name": { 2024-09-12T03:19:14.656 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "smtp_from_name", 2024-09-12T03:19:14.656 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.656 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.657 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.657 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "Ceph", 2024-09-12T03:19:14.657 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.657 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.657 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.657 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Email From: name", 2024-09-12T03:19:14.657 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.657 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.657 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.657 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.657 INFO:teuthology.orchestra.run.smithi116.stdout: "smtp_host": { 2024-09-12T03:19:14.657 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "smtp_host", 2024-09-12T03:19:14.657 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.658 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.658 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.658 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.658 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.658 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.658 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.658 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "SMTP server", 2024-09-12T03:19:14.658 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.658 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.658 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.658 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.658 INFO:teuthology.orchestra.run.smithi116.stdout: "smtp_password": { 2024-09-12T03:19:14.658 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "smtp_password", 2024-09-12T03:19:14.658 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.659 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.659 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.659 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.659 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.659 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.659 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.659 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Password to authenticate with", 2024-09-12T03:19:14.659 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.659 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.659 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.659 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.659 INFO:teuthology.orchestra.run.smithi116.stdout: "smtp_port": { 2024-09-12T03:19:14.659 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "smtp_port", 2024-09-12T03:19:14.660 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.660 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.660 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.660 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "465", 2024-09-12T03:19:14.660 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.660 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.660 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.660 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "SMTP port", 2024-09-12T03:19:14.660 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.660 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.660 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.660 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.660 INFO:teuthology.orchestra.run.smithi116.stdout: "smtp_sender": { 2024-09-12T03:19:14.660 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "smtp_sender", 2024-09-12T03:19:14.661 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.661 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.661 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.661 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.661 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.661 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.661 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.661 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "SMTP envelope sender", 2024-09-12T03:19:14.661 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.661 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.661 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.661 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.661 INFO:teuthology.orchestra.run.smithi116.stdout: "smtp_ssl": { 2024-09-12T03:19:14.662 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "smtp_ssl", 2024-09-12T03:19:14.662 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.662 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.662 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.662 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.662 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.662 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.662 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.662 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T03:19:14.662 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.662 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.662 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.662 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.662 INFO:teuthology.orchestra.run.smithi116.stdout: "smtp_user": { 2024-09-12T03:19:14.663 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "smtp_user", 2024-09-12T03:19:14.663 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.663 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.663 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.663 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.663 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.663 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.663 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.663 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "User to authenticate as", 2024-09-12T03:19:14.663 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.663 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.663 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.663 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.664 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.664 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.664 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.664 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "balancer", 2024-09-12T03:19:14.664 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.664 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.664 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.664 INFO:teuthology.orchestra.run.smithi116.stdout: "active": { 2024-09-12T03:19:14.664 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "active", 2024-09-12T03:19:14.664 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.664 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.664 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.664 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.665 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.665 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.665 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.665 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T03:19:14.665 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.665 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.665 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.665 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.665 INFO:teuthology.orchestra.run.smithi116.stdout: "begin_time": { 2024-09-12T03:19:14.665 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "begin_time", 2024-09-12T03:19:14.665 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.665 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.665 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.665 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "0000", 2024-09-12T03:19:14.666 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.666 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.666 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.666 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T03:19:14.666 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:19:14.666 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.666 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.666 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.666 INFO:teuthology.orchestra.run.smithi116.stdout: "begin_weekday": { 2024-09-12T03:19:14.666 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "begin_weekday", 2024-09-12T03:19:14.666 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "uint", 2024-09-12T03:19:14.666 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.666 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.667 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "0", 2024-09-12T03:19:14.667 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "0", 2024-09-12T03:19:14.667 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "6", 2024-09-12T03:19:14.667 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.667 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T03:19:14.667 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:19:14.667 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.667 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.667 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.667 INFO:teuthology.orchestra.run.smithi116.stdout: "crush_compat_max_iterations": { 2024-09-12T03:19:14.667 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "crush_compat_max_iterations", 2024-09-12T03:19:14.667 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "uint", 2024-09-12T03:19:14.667 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.668 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.668 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "25", 2024-09-12T03:19:14.668 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "1", 2024-09-12T03:19:14.668 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "250", 2024-09-12T03:19:14.668 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.668 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T03:19:14.668 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.668 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.668 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.668 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.668 INFO:teuthology.orchestra.run.smithi116.stdout: "crush_compat_metrics": { 2024-09-12T03:19:14.668 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "crush_compat_metrics", 2024-09-12T03:19:14.668 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.668 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.669 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.669 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T03:19:14.669 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.669 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.669 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.669 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T03:19:14.669 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T03:19:14.669 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.669 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.669 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.669 INFO:teuthology.orchestra.run.smithi116.stdout: "crush_compat_step": { 2024-09-12T03:19:14.669 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "crush_compat_step", 2024-09-12T03:19:14.669 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "float", 2024-09-12T03:19:14.670 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.670 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.670 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "0.5", 2024-09-12T03:19:14.670 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "0.001", 2024-09-12T03:19:14.670 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "0.999", 2024-09-12T03:19:14.670 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.670 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "aggressiveness of optimization", 2024-09-12T03:19:14.670 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T03:19:14.670 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.670 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.671 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.671 INFO:teuthology.orchestra.run.smithi116.stdout: "end_time": { 2024-09-12T03:19:14.671 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "end_time", 2024-09-12T03:19:14.671 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.671 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.671 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.671 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "2359", 2024-09-12T03:19:14.671 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.671 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.671 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.671 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T03:19:14.671 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:19:14.671 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.671 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.672 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.672 INFO:teuthology.orchestra.run.smithi116.stdout: "end_weekday": { 2024-09-12T03:19:14.672 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "end_weekday", 2024-09-12T03:19:14.672 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "uint", 2024-09-12T03:19:14.672 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.672 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.672 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "0", 2024-09-12T03:19:14.672 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "0", 2024-09-12T03:19:14.672 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "6", 2024-09-12T03:19:14.672 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.672 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T03:19:14.672 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:19:14.672 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.673 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.673 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.673 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.673 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.673 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.673 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.673 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.673 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.673 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.673 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.673 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.673 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.673 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.674 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.674 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.674 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.674 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.674 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.674 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.674 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.674 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.674 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.674 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.674 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.674 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.674 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.674 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.675 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.675 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.675 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.675 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.675 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.675 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.675 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.675 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.675 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.675 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.675 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.675 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.675 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.676 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.676 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.676 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.676 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.676 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.676 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.676 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.676 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.676 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.676 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.676 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.676 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.676 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.676 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.677 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.677 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.677 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.677 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.677 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.677 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.677 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.677 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.677 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.677 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.677 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.677 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.677 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.677 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.678 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.678 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.678 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.678 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.678 INFO:teuthology.orchestra.run.smithi116.stdout: "min_score": { 2024-09-12T03:19:14.678 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "min_score", 2024-09-12T03:19:14.678 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "float", 2024-09-12T03:19:14.678 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.678 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.678 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "0", 2024-09-12T03:19:14.678 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.678 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.678 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.678 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T03:19:14.679 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.679 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.679 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.679 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.679 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": { 2024-09-12T03:19:14.679 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "mode", 2024-09-12T03:19:14.679 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.679 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.679 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.679 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "upmap", 2024-09-12T03:19:14.679 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.679 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.679 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.679 INFO:teuthology.orchestra.run.smithi116.stdout: "crush-compat", 2024-09-12T03:19:14.680 INFO:teuthology.orchestra.run.smithi116.stdout: "none", 2024-09-12T03:19:14.680 INFO:teuthology.orchestra.run.smithi116.stdout: "read", 2024-09-12T03:19:14.680 INFO:teuthology.orchestra.run.smithi116.stdout: "upmap", 2024-09-12T03:19:14.680 INFO:teuthology.orchestra.run.smithi116.stdout: "upmap-read" 2024-09-12T03:19:14.680 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.680 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Balancer mode", 2024-09-12T03:19:14.680 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.680 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.680 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.680 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.680 INFO:teuthology.orchestra.run.smithi116.stdout: "pool_ids": { 2024-09-12T03:19:14.680 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "pool_ids", 2024-09-12T03:19:14.680 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.680 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.681 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.681 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.681 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.681 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.681 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.681 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T03:19:14.681 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.681 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.681 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.681 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.681 INFO:teuthology.orchestra.run.smithi116.stdout: "sleep_interval": { 2024-09-12T03:19:14.681 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "sleep_interval", 2024-09-12T03:19:14.681 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.681 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.682 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.682 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "60", 2024-09-12T03:19:14.682 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.682 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.682 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.682 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T03:19:14.682 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.682 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.682 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.682 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.682 INFO:teuthology.orchestra.run.smithi116.stdout: "upmap_max_deviation": { 2024-09-12T03:19:14.682 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "upmap_max_deviation", 2024-09-12T03:19:14.683 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.683 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.683 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.683 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "5", 2024-09-12T03:19:14.683 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "1", 2024-09-12T03:19:14.683 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.683 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.683 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T03:19:14.683 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T03:19:14.683 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.683 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.683 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.683 INFO:teuthology.orchestra.run.smithi116.stdout: "upmap_max_optimizations": { 2024-09-12T03:19:14.683 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "upmap_max_optimizations", 2024-09-12T03:19:14.684 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "uint", 2024-09-12T03:19:14.684 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.684 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.684 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "10", 2024-09-12T03:19:14.684 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.684 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.684 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.684 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T03:19:14.684 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.684 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.684 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.684 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.684 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.685 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.685 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.685 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "cephadm", 2024-09-12T03:19:14.685 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.685 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.685 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.685 INFO:teuthology.orchestra.run.smithi116.stdout: "agent_down_multiplier": { 2024-09-12T03:19:14.685 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "agent_down_multiplier", 2024-09-12T03:19:14.685 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "float", 2024-09-12T03:19:14.685 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.685 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.685 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "3.0", 2024-09-12T03:19:14.685 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.685 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.686 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.686 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T03:19:14.686 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.686 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.686 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.686 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.686 INFO:teuthology.orchestra.run.smithi116.stdout: "agent_refresh_rate": { 2024-09-12T03:19:14.686 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "agent_refresh_rate", 2024-09-12T03:19:14.686 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.686 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.686 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.686 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "20", 2024-09-12T03:19:14.686 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.687 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.687 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.687 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T03:19:14.687 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.687 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.687 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.687 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.687 INFO:teuthology.orchestra.run.smithi116.stdout: "agent_starting_port": { 2024-09-12T03:19:14.687 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "agent_starting_port", 2024-09-12T03:19:14.687 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.687 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.687 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.687 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "4721", 2024-09-12T03:19:14.687 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.688 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.688 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.688 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T03:19:14.688 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.688 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.688 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.688 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.688 INFO:teuthology.orchestra.run.smithi116.stdout: "allow_ptrace": { 2024-09-12T03:19:14.688 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "allow_ptrace", 2024-09-12T03:19:14.688 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.688 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.688 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.688 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.689 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.689 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.689 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.689 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T03:19:14.689 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T03:19:14.689 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.689 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.689 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.689 INFO:teuthology.orchestra.run.smithi116.stdout: "autotune_interval": { 2024-09-12T03:19:14.689 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "autotune_interval", 2024-09-12T03:19:14.689 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.689 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.689 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.690 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "600", 2024-09-12T03:19:14.690 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.690 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.690 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.690 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T03:19:14.690 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.690 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.690 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.690 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.690 INFO:teuthology.orchestra.run.smithi116.stdout: "autotune_memory_target_ratio": { 2024-09-12T03:19:14.690 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T03:19:14.690 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "float", 2024-09-12T03:19:14.690 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.690 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.691 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "0.7", 2024-09-12T03:19:14.691 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.691 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.691 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.691 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T03:19:14.691 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.691 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.691 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.691 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.691 INFO:teuthology.orchestra.run.smithi116.stdout: "cephadm_log_destination": { 2024-09-12T03:19:14.691 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "cephadm_log_destination", 2024-09-12T03:19:14.691 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.691 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.691 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.692 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.692 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.692 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.692 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.692 INFO:teuthology.orchestra.run.smithi116.stdout: "file", 2024-09-12T03:19:14.692 INFO:teuthology.orchestra.run.smithi116.stdout: "file,syslog", 2024-09-12T03:19:14.692 INFO:teuthology.orchestra.run.smithi116.stdout: "syslog" 2024-09-12T03:19:14.692 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.692 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T03:19:14.692 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.692 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.692 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.692 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.692 INFO:teuthology.orchestra.run.smithi116.stdout: "cgroups_split": { 2024-09-12T03:19:14.693 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "cgroups_split", 2024-09-12T03:19:14.693 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.693 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.693 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.693 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.693 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.693 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.693 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.693 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T03:19:14.693 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.693 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.693 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.693 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.693 INFO:teuthology.orchestra.run.smithi116.stdout: "config_checks_enabled": { 2024-09-12T03:19:14.694 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "config_checks_enabled", 2024-09-12T03:19:14.694 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.694 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.694 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.694 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.694 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.694 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.694 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.694 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T03:19:14.694 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.694 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.694 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.694 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.695 INFO:teuthology.orchestra.run.smithi116.stdout: "config_dashboard": { 2024-09-12T03:19:14.695 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "config_dashboard", 2024-09-12T03:19:14.695 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.695 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.695 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.695 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.695 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.695 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.695 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.695 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T03:19:14.695 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.695 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.695 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.695 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.696 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_alertmanager": { 2024-09-12T03:19:14.696 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_alertmanager", 2024-09-12T03:19:14.696 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.696 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.696 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.696 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T03:19:14.696 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.696 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.696 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.696 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Prometheus container image", 2024-09-12T03:19:14.696 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.696 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.696 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.697 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.697 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_base": { 2024-09-12T03:19:14.697 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_base", 2024-09-12T03:19:14.697 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.697 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.697 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.697 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T03:19:14.697 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.697 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.697 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.697 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Container image name, without the tag", 2024-09-12T03:19:14.697 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.697 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.697 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.698 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.698 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_elasticsearch": { 2024-09-12T03:19:14.698 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_elasticsearch", 2024-09-12T03:19:14.698 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.698 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.698 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.698 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T03:19:14.698 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.698 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.698 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.698 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "elasticsearch container image", 2024-09-12T03:19:14.698 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.698 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.698 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.699 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.699 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_grafana": { 2024-09-12T03:19:14.699 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_grafana", 2024-09-12T03:19:14.699 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.699 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.699 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.699 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T03:19:14.699 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.699 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.699 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.699 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Prometheus container image", 2024-09-12T03:19:14.699 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.699 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.700 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.700 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.700 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_haproxy": { 2024-09-12T03:19:14.700 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_haproxy", 2024-09-12T03:19:14.700 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.700 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.700 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.700 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T03:19:14.700 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.700 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.700 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.700 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "HAproxy container image", 2024-09-12T03:19:14.700 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.701 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.701 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.701 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.701 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_jaeger_agent": { 2024-09-12T03:19:14.701 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_jaeger_agent", 2024-09-12T03:19:14.701 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.701 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.701 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.701 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T03:19:14.701 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.701 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.701 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.701 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Jaeger agent container image", 2024-09-12T03:19:14.701 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.702 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.702 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.702 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.702 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_jaeger_collector": { 2024-09-12T03:19:14.702 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_jaeger_collector", 2024-09-12T03:19:14.702 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.702 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.702 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.702 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T03:19:14.702 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.702 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.702 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.702 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Jaeger collector container image", 2024-09-12T03:19:14.703 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.703 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.703 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.703 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.703 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_jaeger_query": { 2024-09-12T03:19:14.703 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_jaeger_query", 2024-09-12T03:19:14.703 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.703 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.703 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.703 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T03:19:14.703 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.703 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.703 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.703 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Jaeger query container image", 2024-09-12T03:19:14.704 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.704 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.704 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.704 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.704 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_keepalived": { 2024-09-12T03:19:14.704 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_keepalived", 2024-09-12T03:19:14.704 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.704 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.704 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.704 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T03:19:14.704 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.704 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.704 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.704 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Keepalived container image", 2024-09-12T03:19:14.705 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.705 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.705 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.705 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.705 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_loki": { 2024-09-12T03:19:14.705 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_loki", 2024-09-12T03:19:14.705 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.705 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.705 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.705 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T03:19:14.705 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.705 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.705 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.706 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Loki container image", 2024-09-12T03:19:14.706 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.706 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.706 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.706 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.706 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_node_exporter": { 2024-09-12T03:19:14.706 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_node_exporter", 2024-09-12T03:19:14.706 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.706 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.706 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.706 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T03:19:14.706 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.706 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.706 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.707 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Prometheus container image", 2024-09-12T03:19:14.707 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.707 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.707 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.707 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.707 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_nvmeof": { 2024-09-12T03:19:14.707 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_nvmeof", 2024-09-12T03:19:14.707 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.707 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.707 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.707 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T03:19:14.707 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.707 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.707 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.708 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Nvme-of container image", 2024-09-12T03:19:14.708 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.708 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.708 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.708 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.708 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_prometheus": { 2024-09-12T03:19:14.708 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_prometheus", 2024-09-12T03:19:14.708 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.708 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.708 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.708 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T03:19:14.708 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.708 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.709 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.709 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Prometheus container image", 2024-09-12T03:19:14.709 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.709 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.709 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.709 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.709 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_promtail": { 2024-09-12T03:19:14.709 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_promtail", 2024-09-12T03:19:14.709 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.709 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.709 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.709 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T03:19:14.709 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.709 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.710 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.710 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Promtail container image", 2024-09-12T03:19:14.710 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.710 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.710 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.710 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.710 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_samba": { 2024-09-12T03:19:14.710 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_samba", 2024-09-12T03:19:14.710 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.710 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.710 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.710 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T03:19:14.710 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.711 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.711 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.711 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Samba/SMB container image", 2024-09-12T03:19:14.711 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.711 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.711 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.711 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.711 INFO:teuthology.orchestra.run.smithi116.stdout: "container_image_snmp_gateway": { 2024-09-12T03:19:14.711 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_image_snmp_gateway", 2024-09-12T03:19:14.711 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.711 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.711 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.711 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T03:19:14.711 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.712 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.712 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.712 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "SNMP Gateway container image", 2024-09-12T03:19:14.712 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.712 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.712 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.712 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.712 INFO:teuthology.orchestra.run.smithi116.stdout: "container_init": { 2024-09-12T03:19:14.712 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "container_init", 2024-09-12T03:19:14.712 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.712 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.712 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.712 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.713 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.713 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.713 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.713 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T03:19:14.713 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.713 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.713 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.713 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.713 INFO:teuthology.orchestra.run.smithi116.stdout: "daemon_cache_timeout": { 2024-09-12T03:19:14.713 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "daemon_cache_timeout", 2024-09-12T03:19:14.713 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.713 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.713 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.714 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "600", 2024-09-12T03:19:14.714 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.714 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.714 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.714 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T03:19:14.714 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.714 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.714 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.714 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.714 INFO:teuthology.orchestra.run.smithi116.stdout: "default_cephadm_command_timeout": { 2024-09-12T03:19:14.714 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T03:19:14.714 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.714 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.714 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.715 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "900", 2024-09-12T03:19:14.715 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.715 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.715 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.715 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T03:19:14.715 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.715 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.715 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.715 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.715 INFO:teuthology.orchestra.run.smithi116.stdout: "default_registry": { 2024-09-12T03:19:14.715 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "default_registry", 2024-09-12T03:19:14.715 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.715 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.715 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.716 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "docker.io", 2024-09-12T03:19:14.716 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.716 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.716 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.716 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T03:19:14.716 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.716 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.716 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.716 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.716 INFO:teuthology.orchestra.run.smithi116.stdout: "device_cache_timeout": { 2024-09-12T03:19:14.716 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "device_cache_timeout", 2024-09-12T03:19:14.716 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.716 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.717 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.717 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "1800", 2024-09-12T03:19:14.717 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.717 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.717 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.717 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "seconds to cache device inventory", 2024-09-12T03:19:14.717 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.717 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.717 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.717 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.717 INFO:teuthology.orchestra.run.smithi116.stdout: "device_enhanced_scan": { 2024-09-12T03:19:14.717 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "device_enhanced_scan", 2024-09-12T03:19:14.717 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.717 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.718 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.718 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.718 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.718 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.718 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.718 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T03:19:14.718 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.718 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.718 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.718 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.718 INFO:teuthology.orchestra.run.smithi116.stdout: "facts_cache_timeout": { 2024-09-12T03:19:14.718 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "facts_cache_timeout", 2024-09-12T03:19:14.718 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.718 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.719 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.719 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "60", 2024-09-12T03:19:14.719 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.719 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.719 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.719 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "seconds to cache host facts data", 2024-09-12T03:19:14.719 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.719 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.719 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.719 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.719 INFO:teuthology.orchestra.run.smithi116.stdout: "grafana_dashboards_path": { 2024-09-12T03:19:14.719 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "grafana_dashboards_path", 2024-09-12T03:19:14.719 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.719 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.720 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.720 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T03:19:14.720 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.720 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.720 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.720 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T03:19:14.720 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.720 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.720 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.720 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.720 INFO:teuthology.orchestra.run.smithi116.stdout: "host_check_interval": { 2024-09-12T03:19:14.720 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "host_check_interval", 2024-09-12T03:19:14.720 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.720 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.721 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.721 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "600", 2024-09-12T03:19:14.721 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.721 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.721 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.721 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "how frequently to perform a host check", 2024-09-12T03:19:14.721 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.721 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.721 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.721 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.721 INFO:teuthology.orchestra.run.smithi116.stdout: "hw_monitoring": { 2024-09-12T03:19:14.721 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "hw_monitoring", 2024-09-12T03:19:14.721 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.722 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.722 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.722 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.722 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.722 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.722 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.722 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T03:19:14.722 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.722 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.722 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.722 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.722 INFO:teuthology.orchestra.run.smithi116.stdout: "inventory_list_all": { 2024-09-12T03:19:14.722 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "inventory_list_all", 2024-09-12T03:19:14.722 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.722 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.723 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.723 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.723 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.723 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.723 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.723 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T03:19:14.723 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.723 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.723 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.723 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.723 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.723 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.723 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.723 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.724 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.724 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.724 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.724 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.724 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.724 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.724 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.724 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.724 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.724 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.724 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.724 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.724 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.725 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.725 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.725 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.725 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.725 INFO:teuthology.orchestra.run.smithi116.stdout: "log_refresh_metadata": { 2024-09-12T03:19:14.725 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_refresh_metadata", 2024-09-12T03:19:14.725 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.725 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.725 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.725 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.725 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.725 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.725 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.725 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T03:19:14.726 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.726 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.726 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.726 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.726 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.726 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.726 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.726 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.726 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.726 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.726 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.726 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.726 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.727 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T03:19:14.727 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.727 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.727 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.727 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.727 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.727 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.727 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.727 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.727 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.727 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.727 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.727 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.728 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.728 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.728 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.728 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.728 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.728 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.728 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.728 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.728 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.728 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.728 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.728 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.728 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.729 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.729 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.729 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.729 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.729 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.729 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.729 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.729 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.729 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.729 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.729 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.729 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.729 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.729 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.730 INFO:teuthology.orchestra.run.smithi116.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T03:19:14.730 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T03:19:14.730 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.730 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.730 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.730 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.730 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.730 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.730 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.730 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T03:19:14.730 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.730 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.730 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.730 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.731 INFO:teuthology.orchestra.run.smithi116.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T03:19:14.731 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T03:19:14.731 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.731 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.731 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.731 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "*", 2024-09-12T03:19:14.731 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.731 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.731 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.731 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T03:19:14.731 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.731 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.731 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.731 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.732 INFO:teuthology.orchestra.run.smithi116.stdout: "max_count_per_host": { 2024-09-12T03:19:14.732 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "max_count_per_host", 2024-09-12T03:19:14.732 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.732 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.732 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.732 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "10", 2024-09-12T03:19:14.732 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.732 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.732 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.732 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "max number of daemons per service per host", 2024-09-12T03:19:14.732 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.732 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.732 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.733 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.733 INFO:teuthology.orchestra.run.smithi116.stdout: "max_osd_draining_count": { 2024-09-12T03:19:14.733 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "max_osd_draining_count", 2024-09-12T03:19:14.733 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.733 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.733 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.733 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "10", 2024-09-12T03:19:14.733 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.733 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.733 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.733 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T03:19:14.733 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.733 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.734 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.734 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.734 INFO:teuthology.orchestra.run.smithi116.stdout: "migration_current": { 2024-09-12T03:19:14.734 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "migration_current", 2024-09-12T03:19:14.734 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.734 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.734 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.734 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.734 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.734 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.734 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.734 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "internal - do not modify", 2024-09-12T03:19:14.734 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.734 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.735 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.735 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.735 INFO:teuthology.orchestra.run.smithi116.stdout: "mode": { 2024-09-12T03:19:14.735 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "mode", 2024-09-12T03:19:14.735 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.735 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.735 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.735 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "root", 2024-09-12T03:19:14.735 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.735 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.735 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.735 INFO:teuthology.orchestra.run.smithi116.stdout: "cephadm-package", 2024-09-12T03:19:14.735 INFO:teuthology.orchestra.run.smithi116.stdout: "root" 2024-09-12T03:19:14.735 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.736 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T03:19:14.736 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.736 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.736 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.736 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.736 INFO:teuthology.orchestra.run.smithi116.stdout: "oob_default_addr": { 2024-09-12T03:19:14.736 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "oob_default_addr", 2024-09-12T03:19:14.736 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.736 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.736 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.736 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "169.254.1.1", 2024-09-12T03:19:14.736 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.736 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.737 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.737 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T03:19:14.737 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.737 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.737 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.737 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.737 INFO:teuthology.orchestra.run.smithi116.stdout: "prometheus_alerts_path": { 2024-09-12T03:19:14.737 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "prometheus_alerts_path", 2024-09-12T03:19:14.737 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.737 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.737 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.737 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T03:19:14.737 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.738 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.738 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.738 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T03:19:14.738 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.738 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.738 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.738 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.738 INFO:teuthology.orchestra.run.smithi116.stdout: "registry_insecure": { 2024-09-12T03:19:14.738 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "registry_insecure", 2024-09-12T03:19:14.738 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.738 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.738 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.738 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.738 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.739 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.739 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.739 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T03:19:14.739 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.739 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.739 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.739 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.739 INFO:teuthology.orchestra.run.smithi116.stdout: "registry_password": { 2024-09-12T03:19:14.739 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "registry_password", 2024-09-12T03:19:14.739 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.739 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.739 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.739 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.739 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.740 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.740 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.740 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T03:19:14.740 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.740 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.740 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.740 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.740 INFO:teuthology.orchestra.run.smithi116.stdout: "registry_url": { 2024-09-12T03:19:14.740 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "registry_url", 2024-09-12T03:19:14.740 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.740 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.740 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.740 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.740 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.741 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.741 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.741 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T03:19:14.741 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.741 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.741 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.741 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.741 INFO:teuthology.orchestra.run.smithi116.stdout: "registry_username": { 2024-09-12T03:19:14.741 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "registry_username", 2024-09-12T03:19:14.741 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.741 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.741 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.741 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.741 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.742 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.742 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.742 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T03:19:14.742 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.742 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.742 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.742 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.742 INFO:teuthology.orchestra.run.smithi116.stdout: "secure_monitoring_stack": { 2024-09-12T03:19:14.742 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "secure_monitoring_stack", 2024-09-12T03:19:14.742 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.742 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.742 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.742 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.743 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.743 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.743 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.743 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T03:19:14.743 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.743 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.743 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.743 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.743 INFO:teuthology.orchestra.run.smithi116.stdout: "service_discovery_port": { 2024-09-12T03:19:14.743 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "service_discovery_port", 2024-09-12T03:19:14.743 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.743 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.743 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.744 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "8765", 2024-09-12T03:19:14.744 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.744 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.744 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.744 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "cephadm service discovery port", 2024-09-12T03:19:14.744 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.744 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.744 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.744 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.744 INFO:teuthology.orchestra.run.smithi116.stdout: "ssh_config_file": { 2024-09-12T03:19:14.744 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ssh_config_file", 2024-09-12T03:19:14.744 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.744 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.744 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.745 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.745 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.745 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.745 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.745 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T03:19:14.745 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.745 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.745 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.745 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.745 INFO:teuthology.orchestra.run.smithi116.stdout: "use_agent": { 2024-09-12T03:19:14.745 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "use_agent", 2024-09-12T03:19:14.745 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.745 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.745 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.746 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.746 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.746 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.746 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.746 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T03:19:14.746 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.746 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.746 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.746 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.746 INFO:teuthology.orchestra.run.smithi116.stdout: "use_repo_digest": { 2024-09-12T03:19:14.746 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "use_repo_digest", 2024-09-12T03:19:14.746 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.746 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.747 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.747 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.747 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.747 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.747 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.747 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T03:19:14.747 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.747 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.747 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.747 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.747 INFO:teuthology.orchestra.run.smithi116.stdout: "warn_on_failed_host_check": { 2024-09-12T03:19:14.747 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "warn_on_failed_host_check", 2024-09-12T03:19:14.747 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.747 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.748 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.748 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.748 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.748 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.748 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.748 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T03:19:14.748 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.748 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.748 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.748 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.748 INFO:teuthology.orchestra.run.smithi116.stdout: "warn_on_stray_daemons": { 2024-09-12T03:19:14.748 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "warn_on_stray_daemons", 2024-09-12T03:19:14.748 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.749 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.749 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.749 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.749 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.749 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.749 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.749 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T03:19:14.749 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.749 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.749 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.749 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.749 INFO:teuthology.orchestra.run.smithi116.stdout: "warn_on_stray_hosts": { 2024-09-12T03:19:14.749 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "warn_on_stray_hosts", 2024-09-12T03:19:14.749 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.750 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.750 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.750 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.750 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.750 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.750 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.750 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T03:19:14.750 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.750 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.750 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.750 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.750 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.750 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.751 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.751 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "crash", 2024-09-12T03:19:14.751 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.751 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.751 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.751 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.751 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.751 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.751 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.751 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.751 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.751 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.751 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.751 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.752 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.752 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.752 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.752 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.752 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.752 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.752 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.752 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.752 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.752 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.752 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.752 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.752 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.752 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.753 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.753 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.753 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.753 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.753 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.753 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.753 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.753 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.753 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.753 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.753 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.753 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.753 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.753 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.754 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.754 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.754 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.754 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.754 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.754 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.754 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.754 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.754 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.754 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.754 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.754 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.754 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.754 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.755 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.755 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.755 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.755 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.755 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.755 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.755 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.755 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.755 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.755 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.755 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.755 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.755 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.756 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.756 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.756 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.756 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.756 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.756 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.756 INFO:teuthology.orchestra.run.smithi116.stdout: "retain_interval": { 2024-09-12T03:19:14.756 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "retain_interval", 2024-09-12T03:19:14.756 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.756 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.756 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.756 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "31536000", 2024-09-12T03:19:14.756 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.756 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.757 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.757 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T03:19:14.757 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.757 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.757 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.757 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.757 INFO:teuthology.orchestra.run.smithi116.stdout: "warn_recent_interval": { 2024-09-12T03:19:14.757 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "warn_recent_interval", 2024-09-12T03:19:14.757 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.757 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.757 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.757 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "1209600", 2024-09-12T03:19:14.757 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.757 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.758 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.758 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T03:19:14.758 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.758 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.758 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.758 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.758 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.758 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.758 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.758 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "dashboard", 2024-09-12T03:19:14.758 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.758 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.758 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.758 INFO:teuthology.orchestra.run.smithi116.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T03:19:14.759 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T03:19:14.759 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.759 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.759 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.759 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "10", 2024-09-12T03:19:14.759 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.759 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.759 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.759 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.759 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.759 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.759 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.759 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.760 INFO:teuthology.orchestra.run.smithi116.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T03:19:14.760 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T03:19:14.760 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.760 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.760 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.760 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.760 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.760 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.760 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.760 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.760 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.760 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.760 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.761 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.761 INFO:teuthology.orchestra.run.smithi116.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T03:19:14.761 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T03:19:14.761 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.761 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.761 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.761 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.761 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.761 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.761 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.761 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.761 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.761 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.761 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.762 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.762 INFO:teuthology.orchestra.run.smithi116.stdout: "AUDIT_API_ENABLED": { 2024-09-12T03:19:14.762 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T03:19:14.762 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.762 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.762 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.762 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.762 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.762 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.762 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.762 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.762 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.762 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.762 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.763 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.763 INFO:teuthology.orchestra.run.smithi116.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T03:19:14.763 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T03:19:14.763 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.763 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.763 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.763 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.763 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.763 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.763 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.763 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.763 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.763 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.764 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.764 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.764 INFO:teuthology.orchestra.run.smithi116.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T03:19:14.764 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T03:19:14.764 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.764 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.764 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.764 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.764 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.764 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.764 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.764 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.764 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.764 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.765 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.765 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.765 INFO:teuthology.orchestra.run.smithi116.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T03:19:14.765 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T03:19:14.765 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.765 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.765 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.765 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.765 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.765 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.765 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.765 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.765 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.766 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.766 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.766 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.766 INFO:teuthology.orchestra.run.smithi116.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T03:19:14.766 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T03:19:14.766 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.766 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.766 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.766 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.766 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.766 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.766 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.766 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.766 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.767 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.767 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.767 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.767 INFO:teuthology.orchestra.run.smithi116.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T03:19:14.767 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T03:19:14.767 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.767 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.767 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.767 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.767 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.767 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.767 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.767 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.768 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.768 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.768 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.768 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.768 INFO:teuthology.orchestra.run.smithi116.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T03:19:14.768 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T03:19:14.768 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.768 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.768 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.768 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.768 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.768 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.768 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.768 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.769 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.769 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.769 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.769 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.769 INFO:teuthology.orchestra.run.smithi116.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T03:19:14.769 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T03:19:14.769 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.769 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.769 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.769 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.769 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.769 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.769 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.769 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.770 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.770 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.770 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.770 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.770 INFO:teuthology.orchestra.run.smithi116.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T03:19:14.770 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T03:19:14.770 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.770 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.770 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.770 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.771 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.771 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.771 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.771 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.771 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.771 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.771 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.771 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.771 INFO:teuthology.orchestra.run.smithi116.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T03:19:14.771 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T03:19:14.771 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.771 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.771 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.771 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.772 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.772 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.772 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.772 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.772 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.772 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.772 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.772 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.772 INFO:teuthology.orchestra.run.smithi116.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T03:19:14.772 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T03:19:14.772 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.772 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.772 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.773 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.773 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.773 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.773 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.773 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.773 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.773 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.773 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.773 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.773 INFO:teuthology.orchestra.run.smithi116.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T03:19:14.773 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T03:19:14.773 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.773 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.773 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.774 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "admin", 2024-09-12T03:19:14.774 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.774 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.774 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.774 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.774 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.774 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.774 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.774 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.774 INFO:teuthology.orchestra.run.smithi116.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T03:19:14.774 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T03:19:14.774 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.774 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.774 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.775 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.775 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.775 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.775 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.775 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.775 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.775 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.775 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.775 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.775 INFO:teuthology.orchestra.run.smithi116.stdout: "GRAFANA_API_URL": { 2024-09-12T03:19:14.775 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "GRAFANA_API_URL", 2024-09-12T03:19:14.775 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.775 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.775 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.776 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.776 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.776 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.776 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.776 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.776 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.776 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.776 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.776 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.776 INFO:teuthology.orchestra.run.smithi116.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T03:19:14.776 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T03:19:14.776 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.776 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.777 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.777 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "admin", 2024-09-12T03:19:14.777 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.777 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.777 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.777 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.777 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.777 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.777 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.777 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.777 INFO:teuthology.orchestra.run.smithi116.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T03:19:14.777 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T03:19:14.777 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.777 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.778 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.778 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.778 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.778 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.778 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.778 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.778 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.778 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.778 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.778 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.778 INFO:teuthology.orchestra.run.smithi116.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T03:19:14.778 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T03:19:14.778 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.778 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.779 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.779 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.779 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.779 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.779 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.779 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.779 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.779 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.779 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.779 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.779 INFO:teuthology.orchestra.run.smithi116.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T03:19:14.779 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T03:19:14.779 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.779 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.780 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.780 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.780 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.780 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.780 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.780 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.780 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.780 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.780 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.780 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.780 INFO:teuthology.orchestra.run.smithi116.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T03:19:14.780 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T03:19:14.780 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.781 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.781 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.781 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.781 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.781 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.781 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.781 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.781 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.781 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.781 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.781 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.781 INFO:teuthology.orchestra.run.smithi116.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T03:19:14.781 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T03:19:14.781 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.782 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.782 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.782 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.782 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.782 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.782 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.782 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.782 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.782 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.782 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.782 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.782 INFO:teuthology.orchestra.run.smithi116.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T03:19:14.782 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T03:19:14.782 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.783 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.783 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.783 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.783 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.783 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.783 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.783 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.783 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.783 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.783 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.783 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.783 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T03:19:14.783 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T03:19:14.783 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.784 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.784 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.784 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.784 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.784 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.784 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.784 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.784 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.784 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.784 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.784 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.784 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T03:19:14.784 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T03:19:14.785 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.785 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.785 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.785 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.785 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.785 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.785 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.785 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.785 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.785 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.785 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.785 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.785 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T03:19:14.785 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T03:19:14.786 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.786 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.786 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.786 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.786 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.786 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.786 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.786 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.786 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.786 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.786 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.786 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.786 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T03:19:14.786 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T03:19:14.787 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.787 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.787 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.787 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.787 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.787 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.787 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.787 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.787 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.787 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.787 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.787 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.787 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T03:19:14.787 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T03:19:14.788 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.788 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.788 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.788 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.788 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.788 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.788 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.788 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.788 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.788 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.788 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.788 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.788 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T03:19:14.788 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T03:19:14.789 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.789 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.789 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.789 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.789 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.789 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.789 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.789 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.789 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.789 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.789 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.789 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.789 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T03:19:14.789 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T03:19:14.790 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.790 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.790 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.790 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.790 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.790 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.790 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.790 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.790 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.790 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.790 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.790 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.790 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T03:19:14.790 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T03:19:14.791 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.791 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.791 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.791 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.791 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.791 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.791 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.791 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.791 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.791 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.791 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.791 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.791 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T03:19:14.792 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T03:19:14.792 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.792 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.792 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.792 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T03:19:14.792 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.792 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.792 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.792 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.792 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.792 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.792 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.792 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.793 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T03:19:14.793 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T03:19:14.793 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.793 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.793 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.793 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "10", 2024-09-12T03:19:14.793 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.793 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.793 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.793 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.793 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.793 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.793 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.794 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.794 INFO:teuthology.orchestra.run.smithi116.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T03:19:14.794 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T03:19:14.794 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.794 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.794 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.794 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "8", 2024-09-12T03:19:14.794 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.794 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.794 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.794 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.794 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.794 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.794 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.795 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.795 INFO:teuthology.orchestra.run.smithi116.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T03:19:14.795 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T03:19:14.795 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.795 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.795 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.795 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "45", 2024-09-12T03:19:14.795 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.795 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.795 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.795 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.795 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.795 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.795 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.796 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.796 INFO:teuthology.orchestra.run.smithi116.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T03:19:14.796 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T03:19:14.796 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.796 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.796 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.796 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.796 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.796 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.796 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.796 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.796 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.797 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.797 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.797 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.797 INFO:teuthology.orchestra.run.smithi116.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T03:19:14.797 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T03:19:14.797 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.797 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.797 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.797 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "admin", 2024-09-12T03:19:14.797 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.797 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.797 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.797 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.797 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.798 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.798 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.798 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.798 INFO:teuthology.orchestra.run.smithi116.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T03:19:14.798 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T03:19:14.798 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.798 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.798 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.798 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.798 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.798 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.798 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.798 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.798 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.799 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.799 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.799 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.799 INFO:teuthology.orchestra.run.smithi116.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T03:19:14.799 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T03:19:14.799 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.799 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.799 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.799 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.799 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.799 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.799 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.799 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.799 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.800 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.800 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.800 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.800 INFO:teuthology.orchestra.run.smithi116.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T03:19:14.800 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T03:19:14.800 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.800 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.800 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.800 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "0", 2024-09-12T03:19:14.800 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.800 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.800 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.800 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.801 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.801 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.801 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.801 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.801 INFO:teuthology.orchestra.run.smithi116.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T03:19:14.801 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T03:19:14.801 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.801 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.801 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.801 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "10", 2024-09-12T03:19:14.801 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.801 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.801 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.801 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.802 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.802 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.802 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.802 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.802 INFO:teuthology.orchestra.run.smithi116.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T03:19:14.802 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T03:19:14.802 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.802 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.802 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.802 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "5", 2024-09-12T03:19:14.802 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.802 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.802 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.802 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.803 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.803 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.803 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.803 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.803 INFO:teuthology.orchestra.run.smithi116.stdout: "cross_origin_url": { 2024-09-12T03:19:14.803 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "cross_origin_url", 2024-09-12T03:19:14.803 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.803 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.803 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.803 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.803 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.803 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.803 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.803 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.803 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.804 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.804 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.804 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.804 INFO:teuthology.orchestra.run.smithi116.stdout: "crt_file": { 2024-09-12T03:19:14.804 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "crt_file", 2024-09-12T03:19:14.804 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.804 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.804 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.804 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.804 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.804 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.804 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.804 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.805 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.805 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.805 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.805 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.805 INFO:teuthology.orchestra.run.smithi116.stdout: "debug": { 2024-09-12T03:19:14.805 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "debug", 2024-09-12T03:19:14.805 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.805 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.805 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.805 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.805 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.805 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.805 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.805 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Enable/disable debug options", 2024-09-12T03:19:14.806 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.806 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.806 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.806 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.806 INFO:teuthology.orchestra.run.smithi116.stdout: "jwt_token_ttl": { 2024-09-12T03:19:14.806 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "jwt_token_ttl", 2024-09-12T03:19:14.806 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.806 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.806 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.806 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "28800", 2024-09-12T03:19:14.806 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.806 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.806 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.806 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.807 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.807 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.807 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.807 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.807 INFO:teuthology.orchestra.run.smithi116.stdout: "key_file": { 2024-09-12T03:19:14.807 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "key_file", 2024-09-12T03:19:14.807 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.807 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.807 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.807 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.807 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.807 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.807 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.807 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.808 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.808 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.808 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.808 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.808 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.808 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.808 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.808 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.808 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.808 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.808 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.808 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.808 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.809 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.809 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.809 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.809 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.809 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.809 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.809 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.809 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.809 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.809 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.809 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.809 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.809 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.809 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.810 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.810 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.810 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.810 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.810 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.810 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.810 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.810 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.810 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.810 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.810 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.810 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.810 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.810 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.811 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.811 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.811 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.811 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.811 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.811 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.811 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.811 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.811 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.811 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.811 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.811 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.811 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.811 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.812 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.812 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.812 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.812 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.812 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.812 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.812 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.812 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.812 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.812 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.812 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.812 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.812 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.813 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.813 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.813 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.813 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.813 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.813 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.813 INFO:teuthology.orchestra.run.smithi116.stdout: "motd": { 2024-09-12T03:19:14.813 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "motd", 2024-09-12T03:19:14.813 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.813 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.813 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.813 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.813 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.813 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.814 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.814 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "The message of the day", 2024-09-12T03:19:14.814 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.814 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.814 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.814 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.814 INFO:teuthology.orchestra.run.smithi116.stdout: "redirect_resolve_ip_addr": { 2024-09-12T03:19:14.814 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T03:19:14.814 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.814 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.814 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.814 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.814 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.815 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.815 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.815 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.815 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.815 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.815 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.815 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.815 INFO:teuthology.orchestra.run.smithi116.stdout: "server_addr": { 2024-09-12T03:19:14.815 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "server_addr", 2024-09-12T03:19:14.815 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.815 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.815 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.815 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "::", 2024-09-12T03:19:14.815 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.816 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.816 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.816 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.816 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.816 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.816 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.816 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.816 INFO:teuthology.orchestra.run.smithi116.stdout: "server_port": { 2024-09-12T03:19:14.816 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "server_port", 2024-09-12T03:19:14.816 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.816 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.816 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.816 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "8080", 2024-09-12T03:19:14.816 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.817 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.817 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.817 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.817 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.817 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.817 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.817 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.817 INFO:teuthology.orchestra.run.smithi116.stdout: "ssl": { 2024-09-12T03:19:14.817 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ssl", 2024-09-12T03:19:14.817 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.817 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.817 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.817 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.818 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.818 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.818 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.818 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.818 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.818 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.818 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.818 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.818 INFO:teuthology.orchestra.run.smithi116.stdout: "ssl_server_port": { 2024-09-12T03:19:14.818 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ssl_server_port", 2024-09-12T03:19:14.818 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.818 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.818 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.818 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "8443", 2024-09-12T03:19:14.819 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.819 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.819 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.819 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.819 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.819 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.819 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.819 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.819 INFO:teuthology.orchestra.run.smithi116.stdout: "standby_behaviour": { 2024-09-12T03:19:14.819 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "standby_behaviour", 2024-09-12T03:19:14.819 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.819 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.819 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.819 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "redirect", 2024-09-12T03:19:14.820 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.820 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.820 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.820 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.820 INFO:teuthology.orchestra.run.smithi116.stdout: "redirect" 2024-09-12T03:19:14.820 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.820 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.820 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.820 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.820 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.820 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.820 INFO:teuthology.orchestra.run.smithi116.stdout: "standby_error_status_code": { 2024-09-12T03:19:14.820 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "standby_error_status_code", 2024-09-12T03:19:14.821 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.821 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.821 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.821 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "500", 2024-09-12T03:19:14.821 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "400", 2024-09-12T03:19:14.821 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "599", 2024-09-12T03:19:14.821 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.821 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.821 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.821 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.821 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.821 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.821 INFO:teuthology.orchestra.run.smithi116.stdout: "url_prefix": { 2024-09-12T03:19:14.821 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "url_prefix", 2024-09-12T03:19:14.822 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.822 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.822 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.822 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.822 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.822 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.822 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.822 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.822 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.822 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.822 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.822 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.822 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.822 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.822 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.823 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "devicehealth", 2024-09-12T03:19:14.823 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.823 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.823 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.823 INFO:teuthology.orchestra.run.smithi116.stdout: "enable_monitoring": { 2024-09-12T03:19:14.823 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "enable_monitoring", 2024-09-12T03:19:14.823 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.823 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.823 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.823 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.823 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.823 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.823 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.823 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "monitor device health metrics", 2024-09-12T03:19:14.824 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.824 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.824 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.824 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.824 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.824 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.824 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.824 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.824 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.824 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.824 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.824 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.824 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.824 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.825 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.825 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.825 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.825 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.825 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.825 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.825 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.825 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.825 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.825 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.825 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.825 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.825 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.826 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.826 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.826 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.826 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.826 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.826 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.826 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.826 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.826 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.826 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.826 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.826 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.826 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.826 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.827 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.827 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.827 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.827 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.827 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.827 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.827 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.827 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.827 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.827 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.827 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.827 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.827 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.827 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.828 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.828 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.828 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.828 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.828 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.828 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.828 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.828 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.828 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.828 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.828 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.828 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.828 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.829 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.829 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.829 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.829 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.829 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.829 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.829 INFO:teuthology.orchestra.run.smithi116.stdout: "mark_out_threshold": { 2024-09-12T03:19:14.829 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "mark_out_threshold", 2024-09-12T03:19:14.829 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.829 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.829 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.829 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "2419200", 2024-09-12T03:19:14.829 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.829 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.830 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.830 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T03:19:14.830 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.830 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.830 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.830 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.830 INFO:teuthology.orchestra.run.smithi116.stdout: "pool_name": { 2024-09-12T03:19:14.830 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "pool_name", 2024-09-12T03:19:14.830 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.830 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.830 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.830 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "device_health_metrics", 2024-09-12T03:19:14.830 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.830 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.831 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.831 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T03:19:14.831 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.831 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.831 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.831 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.831 INFO:teuthology.orchestra.run.smithi116.stdout: "retention_period": { 2024-09-12T03:19:14.831 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "retention_period", 2024-09-12T03:19:14.831 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.831 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.831 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.831 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "15552000", 2024-09-12T03:19:14.831 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.831 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.832 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.832 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "how long to retain device health metrics", 2024-09-12T03:19:14.832 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.832 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.832 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.832 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.832 INFO:teuthology.orchestra.run.smithi116.stdout: "scrape_frequency": { 2024-09-12T03:19:14.832 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "scrape_frequency", 2024-09-12T03:19:14.832 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.832 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.832 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.832 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "86400", 2024-09-12T03:19:14.832 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.832 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.833 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.833 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T03:19:14.833 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.833 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.833 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.833 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.833 INFO:teuthology.orchestra.run.smithi116.stdout: "self_heal": { 2024-09-12T03:19:14.833 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "self_heal", 2024-09-12T03:19:14.833 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.833 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.833 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.833 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.833 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.834 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.834 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.834 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T03:19:14.834 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.834 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.834 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.834 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.834 INFO:teuthology.orchestra.run.smithi116.stdout: "sleep_interval": { 2024-09-12T03:19:14.834 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "sleep_interval", 2024-09-12T03:19:14.834 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.834 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.834 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.834 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "600", 2024-09-12T03:19:14.834 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.835 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.835 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.835 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T03:19:14.835 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.835 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.835 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.835 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.835 INFO:teuthology.orchestra.run.smithi116.stdout: "warn_threshold": { 2024-09-12T03:19:14.835 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "warn_threshold", 2024-09-12T03:19:14.835 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.835 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.835 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.835 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "7257600", 2024-09-12T03:19:14.835 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.836 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.836 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.836 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T03:19:14.836 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.836 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.836 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.836 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.836 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.836 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.836 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.836 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "diskprediction_local", 2024-09-12T03:19:14.836 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.836 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.836 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.837 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.837 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.837 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.837 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.837 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.837 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.837 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.837 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.837 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.837 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.837 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.837 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.837 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.837 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.838 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.838 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.838 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.838 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.838 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.838 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.838 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.838 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.838 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.838 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.838 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.838 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.838 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.838 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.839 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.839 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.839 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.839 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.839 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.839 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.839 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.839 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.839 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.839 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.839 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.839 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.839 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.840 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.840 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.840 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.840 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.840 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.840 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.840 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.840 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.840 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.840 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.840 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.840 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.840 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.840 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.841 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.841 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.841 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.841 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.841 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.841 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.841 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.841 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.841 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.841 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.841 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.841 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.841 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.841 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.842 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.842 INFO:teuthology.orchestra.run.smithi116.stdout: "predict_interval": { 2024-09-12T03:19:14.842 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "predict_interval", 2024-09-12T03:19:14.842 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.842 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.842 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.842 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "86400", 2024-09-12T03:19:14.842 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.842 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.842 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.843 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.843 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.843 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.843 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.843 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.843 INFO:teuthology.orchestra.run.smithi116.stdout: "predictor_model": { 2024-09-12T03:19:14.843 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "predictor_model", 2024-09-12T03:19:14.843 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.843 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.843 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.843 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "prophetstor", 2024-09-12T03:19:14.843 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.843 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.843 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.844 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.844 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.844 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.844 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.844 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.844 INFO:teuthology.orchestra.run.smithi116.stdout: "sleep_interval": { 2024-09-12T03:19:14.844 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "sleep_interval", 2024-09-12T03:19:14.844 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.844 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.844 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.844 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "600", 2024-09-12T03:19:14.844 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.845 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.845 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.845 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.845 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.845 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.845 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.845 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.845 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.845 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.845 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.845 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "influx", 2024-09-12T03:19:14.845 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": false, 2024-09-12T03:19:14.845 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "influxdb python module not found", 2024-09-12T03:19:14.845 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.846 INFO:teuthology.orchestra.run.smithi116.stdout: "batch_size": { 2024-09-12T03:19:14.846 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "batch_size", 2024-09-12T03:19:14.846 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.846 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.846 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.846 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "5000", 2024-09-12T03:19:14.846 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.846 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.846 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.846 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T03:19:14.846 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.846 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.846 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.846 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.847 INFO:teuthology.orchestra.run.smithi116.stdout: "database": { 2024-09-12T03:19:14.847 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "database", 2024-09-12T03:19:14.847 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.847 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.847 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.847 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "ceph", 2024-09-12T03:19:14.847 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.847 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.847 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.847 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T03:19:14.847 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.847 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.847 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.848 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.848 INFO:teuthology.orchestra.run.smithi116.stdout: "hostname": { 2024-09-12T03:19:14.848 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "hostname", 2024-09-12T03:19:14.848 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.848 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.848 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.848 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.848 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.848 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.848 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.848 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "InfluxDB server hostname", 2024-09-12T03:19:14.848 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.848 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.848 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.849 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.849 INFO:teuthology.orchestra.run.smithi116.stdout: "interval": { 2024-09-12T03:19:14.849 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "interval", 2024-09-12T03:19:14.849 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.849 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.849 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.849 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "30", 2024-09-12T03:19:14.849 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "5", 2024-09-12T03:19:14.849 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.849 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.849 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T03:19:14.849 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.849 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.850 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.850 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.850 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.850 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.850 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.850 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.850 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.850 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.850 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.850 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.850 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.850 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.850 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.850 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.851 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.851 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.851 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.851 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.851 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.851 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.851 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.851 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.851 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.851 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.851 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.851 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.851 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.851 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.852 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.852 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.852 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.852 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.852 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.852 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.852 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.852 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.852 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.852 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.852 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.852 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.852 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.853 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.853 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.853 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.853 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.853 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.853 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.853 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.853 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.853 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.853 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.853 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.853 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.853 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.853 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.854 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.854 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.854 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.854 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.854 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.854 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.854 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.854 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.854 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.854 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.854 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.854 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.854 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.854 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.855 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.855 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.855 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.855 INFO:teuthology.orchestra.run.smithi116.stdout: "password": { 2024-09-12T03:19:14.855 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "password", 2024-09-12T03:19:14.855 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.855 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.855 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.855 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.855 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.855 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.855 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.855 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "password of InfluxDB server user", 2024-09-12T03:19:14.855 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.856 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.856 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.856 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.856 INFO:teuthology.orchestra.run.smithi116.stdout: "port": { 2024-09-12T03:19:14.856 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "port", 2024-09-12T03:19:14.856 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.856 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.856 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.856 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "8086", 2024-09-12T03:19:14.856 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.856 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.856 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.856 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "InfluxDB server port", 2024-09-12T03:19:14.856 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.857 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.857 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.857 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.857 INFO:teuthology.orchestra.run.smithi116.stdout: "ssl": { 2024-09-12T03:19:14.857 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ssl", 2024-09-12T03:19:14.857 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.857 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.857 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.857 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "false", 2024-09-12T03:19:14.857 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.857 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.857 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.857 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:19:14.857 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.858 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.858 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.858 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.858 INFO:teuthology.orchestra.run.smithi116.stdout: "threads": { 2024-09-12T03:19:14.858 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "threads", 2024-09-12T03:19:14.858 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.858 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.858 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.858 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "5", 2024-09-12T03:19:14.858 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "1", 2024-09-12T03:19:14.858 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "32", 2024-09-12T03:19:14.858 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.858 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T03:19:14.858 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.859 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.859 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.859 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.859 INFO:teuthology.orchestra.run.smithi116.stdout: "username": { 2024-09-12T03:19:14.859 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "username", 2024-09-12T03:19:14.859 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.859 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.859 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.859 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.859 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.859 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.859 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.859 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "username of InfluxDB server user", 2024-09-12T03:19:14.860 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.860 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.860 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.860 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.860 INFO:teuthology.orchestra.run.smithi116.stdout: "verify_ssl": { 2024-09-12T03:19:14.860 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "verify_ssl", 2024-09-12T03:19:14.860 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.860 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.860 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.860 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "true", 2024-09-12T03:19:14.860 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.860 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.860 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.860 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:19:14.861 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.861 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.861 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.861 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.861 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.861 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.861 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.861 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "insights", 2024-09-12T03:19:14.861 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.861 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.861 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.861 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.861 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.862 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.862 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.862 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.862 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.862 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.862 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.862 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.862 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.862 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.862 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.862 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.862 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.862 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.862 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.863 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.863 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.863 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.863 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.863 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.863 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.863 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.863 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.863 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.863 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.863 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.863 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.863 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.863 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.864 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.864 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.864 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.864 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.864 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.864 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.864 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.864 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.864 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.864 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.864 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.864 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.864 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.865 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.865 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.865 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.865 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.865 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.865 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.865 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.865 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.865 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.865 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.865 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.865 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.865 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.865 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.866 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.866 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.866 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.866 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.866 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.866 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.866 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.866 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.866 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.866 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.866 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.866 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.866 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.866 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.867 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.867 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.867 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "iostat", 2024-09-12T03:19:14.867 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.867 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.867 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.867 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.867 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.867 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.867 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.867 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.867 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.867 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.868 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.868 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.868 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.868 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.868 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.868 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.868 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.868 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.868 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.868 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.868 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.868 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.868 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.869 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.869 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.869 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.869 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.869 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.869 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.869 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.869 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.869 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.869 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.869 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.869 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.869 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.869 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.870 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.870 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.870 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.870 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.870 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.870 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.870 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.870 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.870 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.870 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.870 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.870 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.870 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.870 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.871 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.871 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.871 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.871 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.871 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.871 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.871 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.871 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.871 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.871 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.871 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.871 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.871 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.871 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.872 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.872 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.872 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.872 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.872 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.872 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.872 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.872 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.872 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.872 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.872 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.872 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "k8sevents", 2024-09-12T03:19:14.872 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.873 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.873 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.873 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_event_retention_days": { 2024-09-12T03:19:14.873 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "ceph_event_retention_days", 2024-09-12T03:19:14.873 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.873 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.873 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.873 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "7", 2024-09-12T03:19:14.873 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.873 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.873 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.873 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T03:19:14.873 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.873 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.874 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.874 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.874 INFO:teuthology.orchestra.run.smithi116.stdout: "config_check_secs": { 2024-09-12T03:19:14.874 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "config_check_secs", 2024-09-12T03:19:14.874 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.874 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.874 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.874 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "10", 2024-09-12T03:19:14.874 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "10", 2024-09-12T03:19:14.874 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.874 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.874 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T03:19:14.874 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.874 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.875 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.875 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.875 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.875 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.875 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.875 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.875 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.875 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.875 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.875 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.875 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.875 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.875 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.875 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.876 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.876 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.876 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.876 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.876 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.876 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.876 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.876 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.876 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.876 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.876 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.876 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.876 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.877 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.877 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.877 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.877 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.877 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.877 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.877 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.877 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.877 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.877 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.877 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.877 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.877 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.878 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.878 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.878 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.878 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.878 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.878 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.878 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.878 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.878 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.878 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.878 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.878 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.878 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.878 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.879 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.879 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.879 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.879 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.879 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.879 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.879 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.879 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.879 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.879 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.879 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.879 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.879 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.879 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.880 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.880 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.880 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.880 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.880 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.880 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.880 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.880 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "localpool", 2024-09-12T03:19:14.880 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.880 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.880 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.880 INFO:teuthology.orchestra.run.smithi116.stdout: "failure_domain": { 2024-09-12T03:19:14.880 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "failure_domain", 2024-09-12T03:19:14.880 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.881 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.881 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.881 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "host", 2024-09-12T03:19:14.881 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.881 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.881 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.881 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "failure domain for any created local pool", 2024-09-12T03:19:14.881 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T03:19:14.881 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.881 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.881 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.881 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.881 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.882 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.882 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.882 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.882 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.882 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.882 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.882 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.882 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.882 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.882 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.882 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.882 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.882 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.882 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.883 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.883 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.883 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.883 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.883 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.883 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.883 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.883 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.883 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.883 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.883 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.883 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.883 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.883 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.884 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.884 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.884 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.884 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.884 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.884 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.884 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.884 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.884 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.884 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.884 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.884 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.884 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.885 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.885 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.885 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.885 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.885 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.885 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.885 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.885 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.885 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.885 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.885 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.885 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.885 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.885 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.886 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.886 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.886 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.886 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.886 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.886 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.886 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.886 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.886 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.886 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.886 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.886 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.886 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.886 INFO:teuthology.orchestra.run.smithi116.stdout: "min_size": { 2024-09-12T03:19:14.887 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "min_size", 2024-09-12T03:19:14.887 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.887 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.887 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.887 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.887 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.887 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.887 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.887 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "default min_size for any created local pool", 2024-09-12T03:19:14.887 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T03:19:14.887 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.887 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.887 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.888 INFO:teuthology.orchestra.run.smithi116.stdout: "num_rep": { 2024-09-12T03:19:14.888 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "num_rep", 2024-09-12T03:19:14.888 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.888 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.888 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.888 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "3", 2024-09-12T03:19:14.888 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.888 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.888 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.888 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "default replica count for any created local pool", 2024-09-12T03:19:14.888 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.888 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.888 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.888 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.889 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num": { 2024-09-12T03:19:14.889 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "pg_num", 2024-09-12T03:19:14.889 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.889 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.889 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.889 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "128", 2024-09-12T03:19:14.889 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.889 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.889 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.889 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T03:19:14.889 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.889 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.889 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.889 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.890 INFO:teuthology.orchestra.run.smithi116.stdout: "prefix": { 2024-09-12T03:19:14.890 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "prefix", 2024-09-12T03:19:14.890 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.890 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.890 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.890 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.890 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.890 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.890 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.890 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "name prefix for any created local pool", 2024-09-12T03:19:14.890 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.890 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.890 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.890 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.891 INFO:teuthology.orchestra.run.smithi116.stdout: "subtree": { 2024-09-12T03:19:14.891 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "subtree", 2024-09-12T03:19:14.891 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.891 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.891 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.891 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "rack", 2024-09-12T03:19:14.891 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.891 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.891 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.891 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T03:19:14.891 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T03:19:14.891 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.891 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.891 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.892 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.892 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.892 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.892 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "mds_autoscaler", 2024-09-12T03:19:14.892 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.892 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.892 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.892 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.892 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.892 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.892 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.892 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.892 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.893 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.893 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.893 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.893 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.893 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.893 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.893 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.893 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.893 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.893 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.893 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.893 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.893 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.893 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.894 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.894 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.894 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.894 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.894 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.894 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.894 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.894 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.894 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.894 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.894 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.894 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.894 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.895 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.895 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.895 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.895 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.895 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.895 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.895 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.895 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.895 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.895 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.895 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.895 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.895 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.896 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.896 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.896 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.896 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.896 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.896 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.896 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.896 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.896 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.896 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.896 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.896 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.896 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.897 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.897 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.897 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.897 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.897 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.897 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.897 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.897 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.897 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.897 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.897 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.897 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.897 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.897 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.898 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "mirroring", 2024-09-12T03:19:14.898 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.898 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.898 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.898 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.898 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.898 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.898 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.898 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.898 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.898 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.898 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.898 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.899 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.899 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.899 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.899 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.899 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.899 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.899 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.899 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.899 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.899 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.899 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.899 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.899 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.899 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.900 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.900 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.900 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.900 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.900 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.900 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.900 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.900 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.900 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.900 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.900 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.900 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.900 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.900 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.901 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.901 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.901 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.901 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.901 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.901 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.901 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.901 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.901 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.901 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.901 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.901 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.901 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.902 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.902 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.902 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.902 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.902 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.902 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.902 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.902 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.902 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.902 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.902 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.902 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.902 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.903 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.903 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.903 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.903 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.903 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.903 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.903 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.903 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.903 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.903 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.903 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "nfs", 2024-09-12T03:19:14.903 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.903 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.904 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.904 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.904 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.904 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.904 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.904 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.904 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.904 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.904 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.904 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.904 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.904 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.904 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.905 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.905 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.905 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.905 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.905 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.905 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.905 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.905 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.905 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.905 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.905 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.905 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.905 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.905 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.906 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.906 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.906 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.906 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.906 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.906 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.906 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.906 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.906 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.906 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.906 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.906 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.906 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.906 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.907 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.907 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.907 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.907 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.907 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.907 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.907 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.907 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.907 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.907 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.907 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.907 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.907 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.908 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.908 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.908 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.908 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.908 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.908 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.908 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.908 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.908 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.908 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.908 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.908 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.908 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.908 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.909 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.909 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.909 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.909 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.909 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.909 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.909 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "orchestrator", 2024-09-12T03:19:14.909 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.909 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.909 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.909 INFO:teuthology.orchestra.run.smithi116.stdout: "fail_fs": { 2024-09-12T03:19:14.909 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "fail_fs", 2024-09-12T03:19:14.909 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.910 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.910 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.910 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.910 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.910 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.910 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.910 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T03:19:14.910 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.910 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.910 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.910 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.910 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.910 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.910 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.911 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.911 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.911 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.911 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.911 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.911 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.911 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.911 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.911 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.911 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.911 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.911 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.911 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.912 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.912 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.912 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.912 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.912 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.912 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.912 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.912 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.912 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.912 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.912 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.912 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.912 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.912 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.913 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.913 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.913 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.913 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.913 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.913 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.913 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.913 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.913 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.913 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.913 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.913 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.913 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.913 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.914 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.914 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.914 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.914 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.914 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.914 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.914 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.914 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.914 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.914 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.914 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.914 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.914 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.915 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.915 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.915 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.915 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.915 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.915 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.915 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.915 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.915 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.915 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.915 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.915 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.915 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.915 INFO:teuthology.orchestra.run.smithi116.stdout: "orchestrator": { 2024-09-12T03:19:14.916 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "orchestrator", 2024-09-12T03:19:14.916 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.916 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.916 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.916 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.916 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.916 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.916 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.916 INFO:teuthology.orchestra.run.smithi116.stdout: "cephadm", 2024-09-12T03:19:14.916 INFO:teuthology.orchestra.run.smithi116.stdout: "rook", 2024-09-12T03:19:14.916 INFO:teuthology.orchestra.run.smithi116.stdout: "test_orchestrator" 2024-09-12T03:19:14.916 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.916 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Orchestrator backend", 2024-09-12T03:19:14.916 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.917 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.917 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.917 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.917 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.917 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.917 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.917 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "osd_perf_query", 2024-09-12T03:19:14.917 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.917 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.917 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.917 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.917 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.917 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.917 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.918 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.918 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.918 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.918 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.918 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.918 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.918 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.918 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.918 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.918 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.918 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.918 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.918 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.919 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.919 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.919 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.919 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.919 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.919 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.919 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.919 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.919 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.919 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.919 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.919 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.919 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.919 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.920 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.920 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.920 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.920 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.920 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.920 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.920 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.920 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.920 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.920 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.920 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.920 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.920 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.921 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.921 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.921 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.921 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.921 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.921 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.921 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.921 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.921 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.921 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.921 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.921 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.921 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.921 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.922 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.922 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.922 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.922 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.922 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.922 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.922 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.922 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.922 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.922 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.922 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.922 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.922 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.922 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.923 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.923 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "osd_support", 2024-09-12T03:19:14.923 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.923 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.923 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.923 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.923 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.923 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.923 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.923 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.923 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.923 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.923 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.923 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.924 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.924 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.924 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.924 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.924 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.924 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.924 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.924 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.924 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.924 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.924 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.924 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.924 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.924 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.925 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.925 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.925 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.925 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.925 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.925 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.925 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.925 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.925 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.925 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.925 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.925 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.925 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.925 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.926 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.926 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.926 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.926 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.926 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.926 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.926 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.926 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.926 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.926 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.926 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.926 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.926 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.926 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.927 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.927 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.927 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.927 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.927 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.927 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.927 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.927 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.927 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.927 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.927 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.927 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.927 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.927 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.928 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.928 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.928 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.928 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.928 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.928 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.928 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.928 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.928 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "pg_autoscaler", 2024-09-12T03:19:14.928 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.928 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.928 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.928 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.928 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.929 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.929 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.929 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.929 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.929 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.929 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.929 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.929 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.929 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.929 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.929 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.929 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.929 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.930 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.930 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.930 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.930 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.930 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.930 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.930 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.930 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.930 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.930 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.930 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.930 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.930 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.930 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.931 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.931 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.931 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.931 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.931 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.931 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.931 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.931 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.931 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.931 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.931 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.931 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.931 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.931 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.932 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.932 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.932 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.932 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.932 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.932 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.932 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.932 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.932 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.932 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.932 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.932 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.932 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.933 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.933 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.933 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.933 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.933 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.933 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.933 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.933 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.933 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.933 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.933 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.933 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.933 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.934 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.934 INFO:teuthology.orchestra.run.smithi116.stdout: "sleep_interval": { 2024-09-12T03:19:14.934 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "sleep_interval", 2024-09-12T03:19:14.934 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.934 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.934 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.934 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "60", 2024-09-12T03:19:14.934 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.934 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.934 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.934 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.934 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.934 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.934 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.935 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.935 INFO:teuthology.orchestra.run.smithi116.stdout: "threshold": { 2024-09-12T03:19:14.935 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "threshold", 2024-09-12T03:19:14.935 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "float", 2024-09-12T03:19:14.935 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.935 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.935 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "3.0", 2024-09-12T03:19:14.935 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "1.0", 2024-09-12T03:19:14.935 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.935 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.935 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "scaling threshold", 2024-09-12T03:19:14.935 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T03:19:14.935 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.936 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.936 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.936 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.936 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.936 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.936 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "progress", 2024-09-12T03:19:14.936 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.936 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.936 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.936 INFO:teuthology.orchestra.run.smithi116.stdout: "allow_pg_recovery_event": { 2024-09-12T03:19:14.936 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "allow_pg_recovery_event", 2024-09-12T03:19:14.936 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.936 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.936 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.937 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.937 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.937 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.937 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.937 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T03:19:14.937 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.937 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.937 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.937 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.937 INFO:teuthology.orchestra.run.smithi116.stdout: "enabled": { 2024-09-12T03:19:14.937 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "enabled", 2024-09-12T03:19:14.937 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.937 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.938 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.938 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.938 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.938 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.938 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.938 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.938 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.938 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.938 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.938 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.938 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.938 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.938 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.939 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.939 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.939 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.939 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.939 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.939 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.939 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.939 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.939 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.939 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.939 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.939 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.939 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.939 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.940 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.940 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.940 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.940 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.940 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.940 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.940 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.940 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.940 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.940 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.940 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.940 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.940 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.940 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.941 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.941 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.941 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.941 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.941 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.941 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.941 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.941 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.941 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.941 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.941 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.941 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.941 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.941 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.942 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.942 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.942 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.942 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.942 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.942 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.942 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.942 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.942 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.942 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.942 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.942 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.942 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.943 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.943 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.943 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.943 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.943 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.943 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.943 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.943 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.943 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.943 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.943 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.943 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.943 INFO:teuthology.orchestra.run.smithi116.stdout: "max_completed_events": { 2024-09-12T03:19:14.943 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "max_completed_events", 2024-09-12T03:19:14.944 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.944 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.944 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.944 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "50", 2024-09-12T03:19:14.944 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.944 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.944 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.944 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "number of past completed events to remember", 2024-09-12T03:19:14.944 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.944 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.944 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.944 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.944 INFO:teuthology.orchestra.run.smithi116.stdout: "sleep_interval": { 2024-09-12T03:19:14.945 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "sleep_interval", 2024-09-12T03:19:14.945 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:14.945 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.945 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.945 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "5", 2024-09-12T03:19:14.945 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.945 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.945 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.945 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "how long the module is going to sleep", 2024-09-12T03:19:14.945 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.945 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.945 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.945 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.945 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.946 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.946 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.946 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "prometheus", 2024-09-12T03:19:14.946 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.946 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.946 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.946 INFO:teuthology.orchestra.run.smithi116.stdout: "cache": { 2024-09-12T03:19:14.946 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "cache", 2024-09-12T03:19:14.946 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.946 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.946 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.946 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.946 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.947 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.947 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.947 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.947 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.947 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.947 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.947 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.947 INFO:teuthology.orchestra.run.smithi116.stdout: "exclude_perf_counters": { 2024-09-12T03:19:14.947 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "exclude_perf_counters", 2024-09-12T03:19:14.947 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.947 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.947 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.947 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.948 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.948 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.948 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.948 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T03:19:14.948 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T03:19:14.948 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.948 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.948 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.948 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.948 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.948 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.948 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.948 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.948 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.949 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.949 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.949 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.949 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.949 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.949 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.949 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.949 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.949 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.949 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.949 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.949 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.949 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.950 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.950 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.950 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.950 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.950 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.950 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.950 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.950 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.950 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.950 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.950 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.950 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.950 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.950 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.951 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.951 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.951 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.951 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.951 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.951 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.951 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.951 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.951 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.951 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.951 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.951 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.951 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.952 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.952 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.952 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.952 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.952 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.952 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.952 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.952 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.952 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.952 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.952 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.952 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.952 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.953 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.953 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.953 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.953 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.953 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.953 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.953 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.953 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.953 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.953 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.953 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.953 INFO:teuthology.orchestra.run.smithi116.stdout: "rbd_stats_pools": { 2024-09-12T03:19:14.953 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rbd_stats_pools", 2024-09-12T03:19:14.953 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.954 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.954 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.954 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.954 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.954 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.954 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.954 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.954 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.954 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.954 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.954 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.954 INFO:teuthology.orchestra.run.smithi116.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T03:19:14.954 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T03:19:14.954 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.955 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.955 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.955 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "300", 2024-09-12T03:19:14.955 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.955 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.955 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.955 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.955 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.955 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.955 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.955 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.955 INFO:teuthology.orchestra.run.smithi116.stdout: "scrape_interval": { 2024-09-12T03:19:14.955 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "scrape_interval", 2024-09-12T03:19:14.956 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "float", 2024-09-12T03:19:14.956 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.956 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.956 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "15.0", 2024-09-12T03:19:14.956 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.956 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.956 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.956 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.956 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.956 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.956 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.956 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.956 INFO:teuthology.orchestra.run.smithi116.stdout: "server_addr": { 2024-09-12T03:19:14.956 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "server_addr", 2024-09-12T03:19:14.957 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.957 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.957 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.957 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "::", 2024-09-12T03:19:14.957 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.957 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.957 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.957 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T03:19:14.957 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.957 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.957 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.957 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.957 INFO:teuthology.orchestra.run.smithi116.stdout: "server_port": { 2024-09-12T03:19:14.957 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "server_port", 2024-09-12T03:19:14.958 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.958 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.958 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.958 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "9283", 2024-09-12T03:19:14.958 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.958 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.958 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.958 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T03:19:14.958 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.958 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.958 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.958 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.958 INFO:teuthology.orchestra.run.smithi116.stdout: "stale_cache_strategy": { 2024-09-12T03:19:14.958 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "stale_cache_strategy", 2024-09-12T03:19:14.959 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.959 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.959 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.959 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "log", 2024-09-12T03:19:14.959 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.959 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.959 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.959 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.959 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.959 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.959 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.959 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.959 INFO:teuthology.orchestra.run.smithi116.stdout: "standby_behaviour": { 2024-09-12T03:19:14.959 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "standby_behaviour", 2024-09-12T03:19:14.960 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.960 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.960 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.960 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "default", 2024-09-12T03:19:14.960 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.960 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.960 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.960 INFO:teuthology.orchestra.run.smithi116.stdout: "default", 2024-09-12T03:19:14.960 INFO:teuthology.orchestra.run.smithi116.stdout: "error" 2024-09-12T03:19:14.960 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.960 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.960 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.961 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.961 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.961 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.961 INFO:teuthology.orchestra.run.smithi116.stdout: "standby_error_status_code": { 2024-09-12T03:19:14.961 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "standby_error_status_code", 2024-09-12T03:19:14.961 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.961 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.961 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.961 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "500", 2024-09-12T03:19:14.961 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "400", 2024-09-12T03:19:14.961 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "599", 2024-09-12T03:19:14.961 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.961 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.961 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.962 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.962 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.962 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.962 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.962 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.962 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.962 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rbd_support", 2024-09-12T03:19:14.962 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.962 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.962 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.962 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.962 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.962 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.962 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.963 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.963 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.963 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.963 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.963 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.963 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.963 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.963 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.963 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.963 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.963 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.963 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.963 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.963 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.964 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.964 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.964 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.964 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.964 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.964 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.964 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.964 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.964 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.965 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.965 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.965 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.965 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.965 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.965 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.965 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.965 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.965 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.965 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.965 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.965 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.965 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.966 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.966 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.966 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.966 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.966 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.966 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.966 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.966 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.966 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.966 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.966 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.966 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.966 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.966 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.967 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.967 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.967 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.967 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.967 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.967 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.967 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.967 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.967 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.967 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.967 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.967 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.967 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.968 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.968 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.968 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.968 INFO:teuthology.orchestra.run.smithi116.stdout: "max_concurrent_snap_create": { 2024-09-12T03:19:14.968 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "max_concurrent_snap_create", 2024-09-12T03:19:14.968 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:14.968 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.968 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.968 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "10", 2024-09-12T03:19:14.968 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.968 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.968 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.968 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.969 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.969 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.969 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.969 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.969 INFO:teuthology.orchestra.run.smithi116.stdout: "mirror_snapshot_schedule": { 2024-09-12T03:19:14.969 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T03:19:14.969 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.969 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.969 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.969 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.969 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.969 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.969 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.969 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.970 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.970 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.970 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.970 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.970 INFO:teuthology.orchestra.run.smithi116.stdout: "trash_purge_schedule": { 2024-09-12T03:19:14.970 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "trash_purge_schedule", 2024-09-12T03:19:14.970 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.970 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.970 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.970 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.970 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.970 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.970 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.971 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.971 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.971 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.971 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.971 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.971 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.971 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.971 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.971 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "restful", 2024-09-12T03:19:14.971 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.971 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.971 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.971 INFO:teuthology.orchestra.run.smithi116.stdout: "enable_auth": { 2024-09-12T03:19:14.971 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "enable_auth", 2024-09-12T03:19:14.972 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.972 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.972 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.972 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:14.972 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.972 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.972 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.972 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.972 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.972 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.972 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.972 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.972 INFO:teuthology.orchestra.run.smithi116.stdout: "key_file": { 2024-09-12T03:19:14.973 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "key_file", 2024-09-12T03:19:14.973 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.973 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.973 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.973 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.973 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.973 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.973 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.973 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.973 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.973 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.973 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.973 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.974 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.974 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.974 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.974 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.974 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.974 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.974 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.974 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.974 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.974 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.974 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.974 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.974 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.974 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.975 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.975 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.975 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.975 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.975 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.975 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.975 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.975 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.975 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.975 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.975 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.975 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.975 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.975 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.976 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.976 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.976 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.976 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.976 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.976 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.976 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.976 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.976 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.976 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.976 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.976 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.976 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.976 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.977 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.977 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.977 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.977 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.977 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.977 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.977 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.977 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.977 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.977 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.977 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.977 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.977 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.977 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.978 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.978 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.978 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.978 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.978 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.978 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.978 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.978 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.978 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.978 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.978 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.978 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.978 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.978 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.979 INFO:teuthology.orchestra.run.smithi116.stdout: "server_addr": { 2024-09-12T03:19:14.979 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "server_addr", 2024-09-12T03:19:14.979 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.979 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.979 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.979 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.979 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.979 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.979 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.979 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.979 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.979 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.979 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.979 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.980 INFO:teuthology.orchestra.run.smithi116.stdout: "server_port": { 2024-09-12T03:19:14.980 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "server_port", 2024-09-12T03:19:14.980 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.980 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.980 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.980 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.980 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.980 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.980 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.980 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.980 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.980 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.980 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.980 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.981 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.981 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.981 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.981 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rgw", 2024-09-12T03:19:14.981 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.981 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.981 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.981 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.981 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.981 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.981 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.981 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.981 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.981 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.982 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.982 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.982 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.982 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.982 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.982 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.982 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.982 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.982 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.982 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.982 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.983 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.983 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.983 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.983 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.983 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.983 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.983 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.983 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.983 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.983 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.983 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.983 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.983 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.984 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.984 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.984 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.984 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.984 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.984 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.984 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.984 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.984 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.984 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.984 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.984 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.984 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.985 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.985 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.985 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.985 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.985 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.985 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.985 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.985 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.985 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.985 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.985 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.985 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.985 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.985 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.986 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.986 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.986 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.986 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.986 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.986 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.986 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.986 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.986 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.986 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.986 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.986 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.986 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.986 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.987 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.987 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rook", 2024-09-12T03:19:14.987 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.987 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.987 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.987 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.987 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.987 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.987 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.987 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.987 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.987 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.987 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.987 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.988 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.988 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.988 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.988 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.988 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.988 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.988 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.988 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.988 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.988 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.988 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.988 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.988 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.989 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.989 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.989 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.989 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.989 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.989 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.989 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.989 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.989 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.989 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.989 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.989 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.989 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.989 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.990 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.990 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.990 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.990 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.990 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.990 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.990 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.990 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.990 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.990 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.990 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.990 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.990 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.990 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.991 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.991 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.991 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.991 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.991 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.991 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.991 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.991 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.991 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.991 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.991 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.991 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.991 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.991 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.992 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.992 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.992 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.992 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.992 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.992 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.992 INFO:teuthology.orchestra.run.smithi116.stdout: "storage_class": { 2024-09-12T03:19:14.992 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "storage_class", 2024-09-12T03:19:14.992 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.992 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.992 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.992 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "local", 2024-09-12T03:19:14.992 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.992 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.993 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.993 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T03:19:14.993 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.993 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.993 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.993 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.993 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:14.993 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.993 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:14.993 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "selftest", 2024-09-12T03:19:14.993 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:14.993 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:14.993 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:14.993 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:14.994 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:14.994 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.994 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.994 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.994 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.994 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.994 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.994 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.994 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.994 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.994 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.994 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.994 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.994 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.995 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.995 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.995 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.995 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.995 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.995 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.995 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:14.995 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:14.995 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.995 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.995 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.995 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.995 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.995 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.996 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.996 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.996 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.996 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.996 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.996 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.996 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:14.996 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:14.996 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.996 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.996 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.996 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:14.996 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.996 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.997 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:14.997 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:14.997 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:14.997 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:14.997 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:14.997 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:14.997 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:14.997 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:14.997 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.997 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.997 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.997 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.997 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.997 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:14.998 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:14.998 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:14.998 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.998 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:14.998 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:14.998 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.998 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.998 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.998 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.998 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.998 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.998 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.998 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.998 INFO:teuthology.orchestra.run.smithi116.stdout: "roption1": { 2024-09-12T03:19:14.999 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "roption1", 2024-09-12T03:19:14.999 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:14.999 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:14.999 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:14.999 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:14.999 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:14.999 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:14.999 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:14.999 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:14.999 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:14.999 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:14.999 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:14.999 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:14.999 INFO:teuthology.orchestra.run.smithi116.stdout: "roption2": { 2024-09-12T03:19:15.000 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "roption2", 2024-09-12T03:19:15.000 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.000 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.000 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.000 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "xyz", 2024-09-12T03:19:15.000 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.000 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.000 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.000 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.000 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.000 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.000 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.000 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.001 INFO:teuthology.orchestra.run.smithi116.stdout: "rwoption1": { 2024-09-12T03:19:15.001 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rwoption1", 2024-09-12T03:19:15.001 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.001 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.001 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.001 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:15.001 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.001 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.001 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.001 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.001 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.001 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.001 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.001 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.002 INFO:teuthology.orchestra.run.smithi116.stdout: "rwoption2": { 2024-09-12T03:19:15.002 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rwoption2", 2024-09-12T03:19:15.002 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:15.002 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.002 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.002 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:15.002 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.002 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.002 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.002 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.002 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.002 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.002 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.002 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.003 INFO:teuthology.orchestra.run.smithi116.stdout: "rwoption3": { 2024-09-12T03:19:15.003 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rwoption3", 2024-09-12T03:19:15.003 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "float", 2024-09-12T03:19:15.003 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.003 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.003 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:15.003 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.003 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.003 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.003 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.003 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.003 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.003 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.003 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.004 INFO:teuthology.orchestra.run.smithi116.stdout: "rwoption4": { 2024-09-12T03:19:15.004 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rwoption4", 2024-09-12T03:19:15.004 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.004 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.004 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.004 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:15.004 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.004 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.004 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.004 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.004 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.004 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.005 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.005 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.005 INFO:teuthology.orchestra.run.smithi116.stdout: "rwoption5": { 2024-09-12T03:19:15.005 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rwoption5", 2024-09-12T03:19:15.005 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:15.005 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.005 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.005 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:15.005 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.005 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.005 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.005 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.005 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.005 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.006 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.006 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.006 INFO:teuthology.orchestra.run.smithi116.stdout: "rwoption6": { 2024-09-12T03:19:15.006 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rwoption6", 2024-09-12T03:19:15.006 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:15.006 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.006 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.006 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:15.006 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.006 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.006 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.006 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.006 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.006 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.007 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.007 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.007 INFO:teuthology.orchestra.run.smithi116.stdout: "rwoption7": { 2024-09-12T03:19:15.007 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rwoption7", 2024-09-12T03:19:15.007 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:15.007 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.007 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.007 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:15.007 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "1", 2024-09-12T03:19:15.007 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "42", 2024-09-12T03:19:15.007 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.007 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.007 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.008 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.008 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.008 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.008 INFO:teuthology.orchestra.run.smithi116.stdout: "testkey": { 2024-09-12T03:19:15.008 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "testkey", 2024-09-12T03:19:15.008 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.008 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.008 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.008 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:15.008 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.008 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.008 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.008 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.008 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.009 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.009 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.009 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.009 INFO:teuthology.orchestra.run.smithi116.stdout: "testlkey": { 2024-09-12T03:19:15.009 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "testlkey", 2024-09-12T03:19:15.009 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.009 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.009 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.009 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:15.009 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.009 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.009 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.009 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.009 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.010 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.010 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.010 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.010 INFO:teuthology.orchestra.run.smithi116.stdout: "testnewline": { 2024-09-12T03:19:15.010 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "testnewline", 2024-09-12T03:19:15.010 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.010 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.010 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.010 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:15.010 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.010 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.010 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.010 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.010 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.011 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.011 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.011 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:15.011 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:15.011 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.011 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:15.011 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "snap_schedule", 2024-09-12T03:19:15.011 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:15.011 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:15.011 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:15.011 INFO:teuthology.orchestra.run.smithi116.stdout: "allow_m_granularity": { 2024-09-12T03:19:15.011 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "allow_m_granularity", 2024-09-12T03:19:15.011 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:15.011 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.012 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.012 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:15.012 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.012 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.012 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.012 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T03:19:15.012 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.012 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.012 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.012 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.012 INFO:teuthology.orchestra.run.smithi116.stdout: "dump_on_update": { 2024-09-12T03:19:15.012 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "dump_on_update", 2024-09-12T03:19:15.012 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:15.013 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.013 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.013 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:15.013 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.013 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.013 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.013 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "dump database to debug log on update", 2024-09-12T03:19:15.013 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.013 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.013 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.013 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.013 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:15.013 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:15.013 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.013 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.014 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.014 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:15.014 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.014 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.014 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:15.014 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:15.014 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:15.014 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:15.014 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:15.014 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:15.014 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:15.014 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:15.014 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.014 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.015 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.015 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.015 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.015 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:15.015 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:15.015 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:15.015 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.015 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.015 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:15.015 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.015 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.015 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.015 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.015 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.016 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.016 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.016 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.016 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:15.016 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:15.016 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.016 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.016 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.016 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:15.016 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.016 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.016 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:15.016 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:15.017 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:15.017 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:15.017 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:15.017 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:15.017 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:15.017 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:15.017 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.017 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.017 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.017 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.017 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.017 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:15.017 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:15.017 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:15.018 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.018 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.018 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:15.018 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.018 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.018 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.018 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.018 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.018 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.018 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.018 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:15.018 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:15.018 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.018 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:15.019 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "stats", 2024-09-12T03:19:15.019 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:15.019 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:15.019 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:15.019 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:15.019 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:15.019 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.019 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.019 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.019 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:15.019 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.019 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.019 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:15.019 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:15.020 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:15.020 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:15.020 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:15.020 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:15.020 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:15.020 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:15.020 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.020 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.020 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.020 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.020 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.020 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:15.020 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:15.020 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:15.021 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.021 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.021 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:15.021 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.021 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.021 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.021 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.021 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.021 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.021 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.021 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.021 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:15.021 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:15.022 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.022 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.022 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.022 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:15.022 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.022 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.022 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:15.022 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:15.022 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:15.022 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:15.022 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:15.022 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:15.022 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:15.023 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:15.023 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.023 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.023 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.023 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.023 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.023 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:15.023 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:15.023 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:15.023 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.023 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.023 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:15.023 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.023 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.023 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.024 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.024 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.024 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.024 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.024 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:15.024 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:15.024 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.024 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:15.024 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "status", 2024-09-12T03:19:15.024 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:15.024 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:15.024 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:15.024 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:15.025 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:15.025 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.025 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.025 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.025 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:15.025 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.025 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.025 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:15.025 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:15.025 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:15.025 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:15.025 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:15.025 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:15.025 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:15.026 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:15.026 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.026 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.026 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.026 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.026 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.026 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:15.026 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:15.026 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:15.026 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.026 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.026 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:15.026 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.026 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.027 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.027 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.027 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.027 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.027 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.027 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.027 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:15.027 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:15.027 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.027 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.027 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.027 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:15.027 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.028 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.028 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:15.028 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:15.028 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:15.028 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:15.028 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:15.028 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:15.028 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:15.028 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:15.028 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.028 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.028 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.028 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.028 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.029 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:15.029 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:15.029 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:15.029 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.029 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.029 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:15.029 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.029 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.029 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.029 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.029 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.029 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.029 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.029 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:15.030 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:15.030 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.030 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:15.030 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "telegraf", 2024-09-12T03:19:15.030 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:15.030 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:15.030 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:15.030 INFO:teuthology.orchestra.run.smithi116.stdout: "address": { 2024-09-12T03:19:15.030 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "address", 2024-09-12T03:19:15.030 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.030 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.030 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.030 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T03:19:15.030 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.031 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.031 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.031 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.031 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.031 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.031 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.031 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.031 INFO:teuthology.orchestra.run.smithi116.stdout: "interval": { 2024-09-12T03:19:15.031 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "interval", 2024-09-12T03:19:15.031 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:15.031 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.031 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.031 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "15", 2024-09-12T03:19:15.031 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.032 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.032 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.032 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.032 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.032 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.032 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.032 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.032 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:15.032 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:15.032 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.032 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.032 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.032 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:15.032 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.033 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.033 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:15.033 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:15.033 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:15.033 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:15.033 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:15.033 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:15.033 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:15.033 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:15.033 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.033 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.033 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.033 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.033 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.034 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:15.034 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:15.034 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:15.034 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.034 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.034 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:15.034 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.034 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.034 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.034 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.034 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.034 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.034 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.035 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.035 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:15.035 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:15.035 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.035 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.035 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.035 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:15.035 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.035 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.035 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:15.035 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:15.035 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:15.035 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:15.035 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:15.036 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:15.036 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:15.036 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:15.036 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.036 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.036 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.036 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.036 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.036 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:15.036 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:15.036 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:15.036 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.036 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.036 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:15.037 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.037 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.037 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.037 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.037 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.037 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.037 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.037 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:15.037 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:15.037 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.037 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:15.037 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "telemetry", 2024-09-12T03:19:15.037 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:15.037 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:15.038 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:15.038 INFO:teuthology.orchestra.run.smithi116.stdout: "channel_basic": { 2024-09-12T03:19:15.038 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "channel_basic", 2024-09-12T03:19:15.038 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:15.038 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.038 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.038 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:15.038 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.038 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.038 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.038 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T03:19:15.038 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.038 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.038 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.039 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.039 INFO:teuthology.orchestra.run.smithi116.stdout: "channel_crash": { 2024-09-12T03:19:15.039 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "channel_crash", 2024-09-12T03:19:15.039 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:15.039 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.039 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.039 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:15.039 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.039 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.039 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.039 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T03:19:15.039 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.039 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.039 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.040 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.040 INFO:teuthology.orchestra.run.smithi116.stdout: "channel_device": { 2024-09-12T03:19:15.040 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "channel_device", 2024-09-12T03:19:15.040 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:15.040 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.040 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.040 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:15.040 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.040 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.040 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.040 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T03:19:15.040 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.040 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.041 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.041 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.041 INFO:teuthology.orchestra.run.smithi116.stdout: "channel_ident": { 2024-09-12T03:19:15.041 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "channel_ident", 2024-09-12T03:19:15.041 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:15.041 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.041 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.041 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:15.041 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.041 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.041 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.041 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T03:19:15.041 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.041 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.042 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.042 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.042 INFO:teuthology.orchestra.run.smithi116.stdout: "channel_perf": { 2024-09-12T03:19:15.042 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "channel_perf", 2024-09-12T03:19:15.042 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:15.042 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.042 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.042 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:15.042 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.042 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.042 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.042 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T03:19:15.042 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.042 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.043 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.043 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.043 INFO:teuthology.orchestra.run.smithi116.stdout: "contact": { 2024-09-12T03:19:15.043 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "contact", 2024-09-12T03:19:15.043 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.043 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.043 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.043 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:15.043 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.043 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.043 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.043 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.043 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.044 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.044 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.044 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.044 INFO:teuthology.orchestra.run.smithi116.stdout: "description": { 2024-09-12T03:19:15.044 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "description", 2024-09-12T03:19:15.044 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.044 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.044 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.044 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:15.044 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.044 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.044 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.044 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.044 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.045 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.045 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.045 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.045 INFO:teuthology.orchestra.run.smithi116.stdout: "device_url": { 2024-09-12T03:19:15.045 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "device_url", 2024-09-12T03:19:15.045 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.045 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.045 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.045 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T03:19:15.045 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.045 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.045 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.045 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.046 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.046 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.046 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.046 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.046 INFO:teuthology.orchestra.run.smithi116.stdout: "enabled": { 2024-09-12T03:19:15.046 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "enabled", 2024-09-12T03:19:15.046 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:15.046 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.046 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.046 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:15.046 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.046 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.046 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.046 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.047 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.047 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.047 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.047 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.047 INFO:teuthology.orchestra.run.smithi116.stdout: "interval": { 2024-09-12T03:19:15.047 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "interval", 2024-09-12T03:19:15.047 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:15.047 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.047 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.047 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "24", 2024-09-12T03:19:15.047 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "8", 2024-09-12T03:19:15.047 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.047 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.047 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.048 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.048 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.048 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.048 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.048 INFO:teuthology.orchestra.run.smithi116.stdout: "last_opt_revision": { 2024-09-12T03:19:15.048 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "last_opt_revision", 2024-09-12T03:19:15.048 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:15.048 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.048 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.048 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "1", 2024-09-12T03:19:15.048 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.048 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.048 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.048 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.049 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.049 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.049 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.049 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.049 INFO:teuthology.orchestra.run.smithi116.stdout: "leaderboard": { 2024-09-12T03:19:15.049 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "leaderboard", 2024-09-12T03:19:15.049 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:15.049 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.049 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.049 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:15.049 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.049 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.049 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.049 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.050 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.050 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.050 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.050 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.050 INFO:teuthology.orchestra.run.smithi116.stdout: "leaderboard_description": { 2024-09-12T03:19:15.050 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "leaderboard_description", 2024-09-12T03:19:15.050 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.050 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.050 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.050 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:15.050 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.050 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.050 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.050 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.051 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.051 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.051 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.051 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.051 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:15.051 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:15.051 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.051 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.051 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.051 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:15.051 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.051 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.051 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:15.052 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:15.052 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:15.052 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:15.052 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:15.052 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:15.052 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:15.052 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:15.052 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.052 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.052 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.052 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.052 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.052 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:15.053 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:15.053 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:15.053 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.053 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.053 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:15.053 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.053 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.053 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.053 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.053 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.053 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.053 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.053 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.054 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:15.054 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:15.054 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.054 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.054 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.054 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:15.054 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.054 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.054 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:15.054 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:15.054 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:15.054 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:15.054 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:15.054 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:15.055 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:15.055 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:15.055 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.055 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.055 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.055 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.055 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.055 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:15.055 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:15.055 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:15.055 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.055 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.055 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:15.055 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.056 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.056 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.056 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.056 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.056 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.056 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.056 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.056 INFO:teuthology.orchestra.run.smithi116.stdout: "organization": { 2024-09-12T03:19:15.056 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "organization", 2024-09-12T03:19:15.056 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.056 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.056 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.056 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:15.057 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.057 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.057 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.057 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.057 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.057 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.057 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.057 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.057 INFO:teuthology.orchestra.run.smithi116.stdout: "proxy": { 2024-09-12T03:19:15.057 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "proxy", 2024-09-12T03:19:15.057 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.057 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.057 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.057 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:15.058 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.058 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.058 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.058 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.058 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.058 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.058 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.058 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.058 INFO:teuthology.orchestra.run.smithi116.stdout: "url": { 2024-09-12T03:19:15.058 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "url", 2024-09-12T03:19:15.058 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.058 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.058 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.058 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T03:19:15.059 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.059 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.059 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.059 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.059 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.059 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.059 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.059 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:15.059 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:15.059 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.059 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:15.059 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "test_orchestrator", 2024-09-12T03:19:15.059 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:15.059 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:15.060 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:15.060 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:15.060 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:15.060 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.060 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.060 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.060 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:15.060 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.060 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.060 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:15.060 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:15.060 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:15.060 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:15.061 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:15.061 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:15.061 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:15.061 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:15.061 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.061 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.061 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.061 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.061 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.061 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:15.061 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:15.061 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:15.061 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.061 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.062 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:15.062 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.062 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.062 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.062 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.062 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.062 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.062 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.062 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.062 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:15.062 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:15.062 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.062 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.062 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.062 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:15.063 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.063 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.063 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:15.063 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:15.063 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:15.063 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:15.063 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:15.063 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:15.063 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:15.063 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:15.063 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.063 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.063 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.063 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.064 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.064 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:15.064 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:15.064 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:15.064 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.064 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.064 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:15.064 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.064 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.064 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.064 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.064 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.064 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.064 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.065 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:15.065 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:15.065 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.065 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:15.065 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "volumes", 2024-09-12T03:19:15.065 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:15.065 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:15.065 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:15.065 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:15.065 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:15.065 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.065 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.065 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.065 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:15.066 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.066 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.066 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:15.066 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:15.066 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:15.066 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:15.066 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:15.066 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:15.066 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:15.066 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:15.066 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.066 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.066 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.066 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.067 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.067 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:15.067 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:15.067 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:15.067 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.067 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.067 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:15.067 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.067 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.067 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.067 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.067 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.067 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.067 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.067 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.068 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:15.068 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:15.068 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.068 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.068 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.068 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:15.068 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.068 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.068 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:15.068 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:15.068 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:15.068 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:15.068 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:15.069 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:15.069 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:15.069 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:15.069 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.069 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.069 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.069 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.069 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.069 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:15.069 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:15.069 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:15.069 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.069 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.070 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:15.070 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.070 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.070 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.070 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.070 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.070 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.070 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.070 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.070 INFO:teuthology.orchestra.run.smithi116.stdout: "max_concurrent_clones": { 2024-09-12T03:19:15.070 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "max_concurrent_clones", 2024-09-12T03:19:15.070 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:15.070 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.070 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.071 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "4", 2024-09-12T03:19:15.071 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.071 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.071 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.071 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T03:19:15.071 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.071 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.071 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.071 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.071 INFO:teuthology.orchestra.run.smithi116.stdout: "periodic_async_work": { 2024-09-12T03:19:15.071 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "periodic_async_work", 2024-09-12T03:19:15.071 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:15.071 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.071 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.072 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:15.072 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.072 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.072 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.072 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Periodically check for async work", 2024-09-12T03:19:15.072 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.072 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.072 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.072 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.072 INFO:teuthology.orchestra.run.smithi116.stdout: "snapshot_clone_delay": { 2024-09-12T03:19:15.072 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "snapshot_clone_delay", 2024-09-12T03:19:15.072 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:15.072 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.073 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.073 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "0", 2024-09-12T03:19:15.073 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.073 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.073 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.073 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T03:19:15.073 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.073 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.073 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.073 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.073 INFO:teuthology.orchestra.run.smithi116.stdout: "snapshot_clone_no_wait": { 2024-09-12T03:19:15.073 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T03:19:15.073 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:15.074 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.074 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.074 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "True", 2024-09-12T03:19:15.074 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.074 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.074 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.074 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T03:19:15.074 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.074 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.074 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.074 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:15.074 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:15.074 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.075 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:15.075 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "zabbix", 2024-09-12T03:19:15.075 INFO:teuthology.orchestra.run.smithi116.stdout: "can_run": true, 2024-09-12T03:19:15.075 INFO:teuthology.orchestra.run.smithi116.stdout: "error_string": "", 2024-09-12T03:19:15.075 INFO:teuthology.orchestra.run.smithi116.stdout: "module_options": { 2024-09-12T03:19:15.075 INFO:teuthology.orchestra.run.smithi116.stdout: "discovery_interval": { 2024-09-12T03:19:15.075 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "discovery_interval", 2024-09-12T03:19:15.075 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "uint", 2024-09-12T03:19:15.075 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.075 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.075 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "100", 2024-09-12T03:19:15.075 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.075 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.075 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.076 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.076 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.076 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.076 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.076 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.076 INFO:teuthology.orchestra.run.smithi116.stdout: "identifier": { 2024-09-12T03:19:15.076 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "identifier", 2024-09-12T03:19:15.076 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.076 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.076 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.076 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:15.076 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.076 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.077 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.077 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.077 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.077 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.077 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.077 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.077 INFO:teuthology.orchestra.run.smithi116.stdout: "interval": { 2024-09-12T03:19:15.077 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "interval", 2024-09-12T03:19:15.077 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "secs", 2024-09-12T03:19:15.077 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.077 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.077 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "60", 2024-09-12T03:19:15.077 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.077 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.078 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.078 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.078 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.078 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.078 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.078 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.078 INFO:teuthology.orchestra.run.smithi116.stdout: "log_level": { 2024-09-12T03:19:15.078 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_level", 2024-09-12T03:19:15.078 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.078 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.078 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.078 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:15.078 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.078 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.079 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:15.079 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:15.079 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:15.079 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:15.079 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:15.079 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:15.079 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:15.079 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:15.079 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.079 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.079 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.079 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.079 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.079 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster": { 2024-09-12T03:19:15.079 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster", 2024-09-12T03:19:15.080 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:15.080 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.080 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.080 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:15.080 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.080 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.080 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.080 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.080 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.080 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.080 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.080 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.080 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_cluster_level": { 2024-09-12T03:19:15.081 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_cluster_level", 2024-09-12T03:19:15.081 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.081 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.081 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.081 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "info", 2024-09-12T03:19:15.081 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.081 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.081 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [ 2024-09-12T03:19:15.081 INFO:teuthology.orchestra.run.smithi116.stdout: "", 2024-09-12T03:19:15.081 INFO:teuthology.orchestra.run.smithi116.stdout: "critical", 2024-09-12T03:19:15.081 INFO:teuthology.orchestra.run.smithi116.stdout: "debug", 2024-09-12T03:19:15.081 INFO:teuthology.orchestra.run.smithi116.stdout: "error", 2024-09-12T03:19:15.081 INFO:teuthology.orchestra.run.smithi116.stdout: "info", 2024-09-12T03:19:15.081 INFO:teuthology.orchestra.run.smithi116.stdout: "warning" 2024-09-12T03:19:15.082 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:15.082 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.082 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.082 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.082 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.082 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.082 INFO:teuthology.orchestra.run.smithi116.stdout: "log_to_file": { 2024-09-12T03:19:15.082 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "log_to_file", 2024-09-12T03:19:15.082 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "bool", 2024-09-12T03:19:15.082 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.082 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2024-09-12T03:19:15.082 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "False", 2024-09-12T03:19:15.082 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.082 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.083 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.083 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.083 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.083 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.083 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.083 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.083 INFO:teuthology.orchestra.run.smithi116.stdout: "zabbix_host": { 2024-09-12T03:19:15.083 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "zabbix_host", 2024-09-12T03:19:15.083 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.083 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.083 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.083 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "", 2024-09-12T03:19:15.083 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.083 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.084 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.084 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.084 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.084 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.084 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.084 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.084 INFO:teuthology.orchestra.run.smithi116.stdout: "zabbix_port": { 2024-09-12T03:19:15.084 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "zabbix_port", 2024-09-12T03:19:15.084 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "int", 2024-09-12T03:19:15.084 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.084 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.084 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "10051", 2024-09-12T03:19:15.084 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.084 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.085 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.085 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.085 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.085 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.085 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.085 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.085 INFO:teuthology.orchestra.run.smithi116.stdout: "zabbix_sender": { 2024-09-12T03:19:15.085 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "zabbix_sender", 2024-09-12T03:19:15.085 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "str", 2024-09-12T03:19:15.085 INFO:teuthology.orchestra.run.smithi116.stdout: "level": "advanced", 2024-09-12T03:19:15.085 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 0, 2024-09-12T03:19:15.085 INFO:teuthology.orchestra.run.smithi116.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T03:19:15.085 INFO:teuthology.orchestra.run.smithi116.stdout: "min": "", 2024-09-12T03:19:15.086 INFO:teuthology.orchestra.run.smithi116.stdout: "max": "", 2024-09-12T03:19:15.086 INFO:teuthology.orchestra.run.smithi116.stdout: "enum_allowed": [], 2024-09-12T03:19:15.086 INFO:teuthology.orchestra.run.smithi116.stdout: "desc": "", 2024-09-12T03:19:15.086 INFO:teuthology.orchestra.run.smithi116.stdout: "long_desc": "", 2024-09-12T03:19:15.086 INFO:teuthology.orchestra.run.smithi116.stdout: "tags": [], 2024-09-12T03:19:15.086 INFO:teuthology.orchestra.run.smithi116.stdout: "see_also": [] 2024-09-12T03:19:15.086 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:15.086 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:15.086 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:15.086 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:15.086 INFO:teuthology.orchestra.run.smithi116.stdout: "services": { 2024-09-12T03:19:15.086 INFO:teuthology.orchestra.run.smithi116.stdout: "dashboard": "https://172.21.15.116:8443/" 2024-09-12T03:19:15.086 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.086 INFO:teuthology.orchestra.run.smithi116.stdout: "always_on_modules": { 2024-09-12T03:19:15.087 INFO:teuthology.orchestra.run.smithi116.stdout: "octopus": [ 2024-09-12T03:19:15.087 INFO:teuthology.orchestra.run.smithi116.stdout: "balancer", 2024-09-12T03:19:15.087 INFO:teuthology.orchestra.run.smithi116.stdout: "crash", 2024-09-12T03:19:15.087 INFO:teuthology.orchestra.run.smithi116.stdout: "devicehealth", 2024-09-12T03:19:15.087 INFO:teuthology.orchestra.run.smithi116.stdout: "orchestrator", 2024-09-12T03:19:15.087 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_autoscaler", 2024-09-12T03:19:15.087 INFO:teuthology.orchestra.run.smithi116.stdout: "progress", 2024-09-12T03:19:15.087 INFO:teuthology.orchestra.run.smithi116.stdout: "rbd_support", 2024-09-12T03:19:15.087 INFO:teuthology.orchestra.run.smithi116.stdout: "status", 2024-09-12T03:19:15.087 INFO:teuthology.orchestra.run.smithi116.stdout: "telemetry", 2024-09-12T03:19:15.087 INFO:teuthology.orchestra.run.smithi116.stdout: "volumes" 2024-09-12T03:19:15.087 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:15.087 INFO:teuthology.orchestra.run.smithi116.stdout: "pacific": [ 2024-09-12T03:19:15.087 INFO:teuthology.orchestra.run.smithi116.stdout: "balancer", 2024-09-12T03:19:15.088 INFO:teuthology.orchestra.run.smithi116.stdout: "crash", 2024-09-12T03:19:15.088 INFO:teuthology.orchestra.run.smithi116.stdout: "devicehealth", 2024-09-12T03:19:15.088 INFO:teuthology.orchestra.run.smithi116.stdout: "orchestrator", 2024-09-12T03:19:15.088 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_autoscaler", 2024-09-12T03:19:15.088 INFO:teuthology.orchestra.run.smithi116.stdout: "progress", 2024-09-12T03:19:15.088 INFO:teuthology.orchestra.run.smithi116.stdout: "rbd_support", 2024-09-12T03:19:15.088 INFO:teuthology.orchestra.run.smithi116.stdout: "status", 2024-09-12T03:19:15.088 INFO:teuthology.orchestra.run.smithi116.stdout: "telemetry", 2024-09-12T03:19:15.088 INFO:teuthology.orchestra.run.smithi116.stdout: "volumes" 2024-09-12T03:19:15.088 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:15.088 INFO:teuthology.orchestra.run.smithi116.stdout: "quincy": [ 2024-09-12T03:19:15.088 INFO:teuthology.orchestra.run.smithi116.stdout: "balancer", 2024-09-12T03:19:15.088 INFO:teuthology.orchestra.run.smithi116.stdout: "crash", 2024-09-12T03:19:15.088 INFO:teuthology.orchestra.run.smithi116.stdout: "devicehealth", 2024-09-12T03:19:15.089 INFO:teuthology.orchestra.run.smithi116.stdout: "orchestrator", 2024-09-12T03:19:15.089 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_autoscaler", 2024-09-12T03:19:15.089 INFO:teuthology.orchestra.run.smithi116.stdout: "progress", 2024-09-12T03:19:15.089 INFO:teuthology.orchestra.run.smithi116.stdout: "rbd_support", 2024-09-12T03:19:15.089 INFO:teuthology.orchestra.run.smithi116.stdout: "status", 2024-09-12T03:19:15.089 INFO:teuthology.orchestra.run.smithi116.stdout: "telemetry", 2024-09-12T03:19:15.089 INFO:teuthology.orchestra.run.smithi116.stdout: "volumes" 2024-09-12T03:19:15.089 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:15.089 INFO:teuthology.orchestra.run.smithi116.stdout: "reef": [ 2024-09-12T03:19:15.089 INFO:teuthology.orchestra.run.smithi116.stdout: "balancer", 2024-09-12T03:19:15.089 INFO:teuthology.orchestra.run.smithi116.stdout: "crash", 2024-09-12T03:19:15.089 INFO:teuthology.orchestra.run.smithi116.stdout: "devicehealth", 2024-09-12T03:19:15.089 INFO:teuthology.orchestra.run.smithi116.stdout: "orchestrator", 2024-09-12T03:19:15.089 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_autoscaler", 2024-09-12T03:19:15.090 INFO:teuthology.orchestra.run.smithi116.stdout: "progress", 2024-09-12T03:19:15.090 INFO:teuthology.orchestra.run.smithi116.stdout: "rbd_support", 2024-09-12T03:19:15.090 INFO:teuthology.orchestra.run.smithi116.stdout: "status", 2024-09-12T03:19:15.090 INFO:teuthology.orchestra.run.smithi116.stdout: "telemetry", 2024-09-12T03:19:15.090 INFO:teuthology.orchestra.run.smithi116.stdout: "volumes" 2024-09-12T03:19:15.090 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2024-09-12T03:19:15.090 INFO:teuthology.orchestra.run.smithi116.stdout: "squid": [ 2024-09-12T03:19:15.090 INFO:teuthology.orchestra.run.smithi116.stdout: "balancer", 2024-09-12T03:19:15.090 INFO:teuthology.orchestra.run.smithi116.stdout: "crash", 2024-09-12T03:19:15.090 INFO:teuthology.orchestra.run.smithi116.stdout: "devicehealth", 2024-09-12T03:19:15.090 INFO:teuthology.orchestra.run.smithi116.stdout: "orchestrator", 2024-09-12T03:19:15.090 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_autoscaler", 2024-09-12T03:19:15.090 INFO:teuthology.orchestra.run.smithi116.stdout: "progress", 2024-09-12T03:19:15.090 INFO:teuthology.orchestra.run.smithi116.stdout: "rbd_support", 2024-09-12T03:19:15.091 INFO:teuthology.orchestra.run.smithi116.stdout: "status", 2024-09-12T03:19:15.091 INFO:teuthology.orchestra.run.smithi116.stdout: "telemetry", 2024-09-12T03:19:15.091 INFO:teuthology.orchestra.run.smithi116.stdout: "volumes" 2024-09-12T03:19:15.091 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2024-09-12T03:19:15.091 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.091 INFO:teuthology.orchestra.run.smithi116.stdout: "last_failure_osd_epoch": 3, 2024-09-12T03:19:15.091 INFO:teuthology.orchestra.run.smithi116.stdout: "active_clients": [ 2024-09-12T03:19:15.091 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:15.091 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "devicehealth", 2024-09-12T03:19:15.091 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2024-09-12T03:19:15.091 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:15.091 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2024-09-12T03:19:15.091 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:0", 2024-09-12T03:19:15.091 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 3637480333 2024-09-12T03:19:15.092 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:15.092 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2024-09-12T03:19:15.092 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.092 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:15.092 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "libcephsqlite", 2024-09-12T03:19:15.092 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2024-09-12T03:19:15.092 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:15.092 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2024-09-12T03:19:15.092 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:0", 2024-09-12T03:19:15.092 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 2736300694 2024-09-12T03:19:15.092 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:15.092 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2024-09-12T03:19:15.092 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.092 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:15.093 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "rbd_support", 2024-09-12T03:19:15.093 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2024-09-12T03:19:15.093 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:15.093 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2024-09-12T03:19:15.093 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:0", 2024-09-12T03:19:15.093 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 3973599330 2024-09-12T03:19:15.093 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:15.093 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2024-09-12T03:19:15.093 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.093 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:15.093 INFO:teuthology.orchestra.run.smithi116.stdout: "name": "volumes", 2024-09-12T03:19:15.093 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2024-09-12T03:19:15.093 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-09-12T03:19:15.093 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2024-09-12T03:19:15.094 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:0", 2024-09-12T03:19:15.094 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 1731424106 2024-09-12T03:19:15.094 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:15.094 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2024-09-12T03:19:15.094 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:15.094 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2024-09-12T03:19:15.094 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:19:15.096 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:13 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/928959942' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:19:15.096 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:13 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3316961184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:19:15.096 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:13 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3629247043' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T03:19:15.096 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:14 smithi177 ceph-mon[35305]: pgmap v825: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-12T03:19:15.096 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:13 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/928959942' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:19:15.096 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:13 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3316961184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:19:15.096 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:13 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3629247043' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T03:19:15.096 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:14 smithi183 ceph-mon[35535]: pgmap v825: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-12T03:19:15.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:13.237+0000 7f8313ba1640 1 -- 172.21.15.116:0/3629247043 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f82f80a2b90 msgr2=0x7f82f80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:15.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:13.237+0000 7f8313ba1640 1 --2- 172.21.15.116:0/3629247043 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f82f80a2b90 0x7f82f80a5050 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f830c101a60 tx=0x7f8308008040 comp rx=0 tx=0).stop 2024-09-12T03:19:15.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:13.237+0000 7f8313ba1640 1 -- 172.21.15.116:0/3629247043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f830c101c00 msgr2=0x7f830c19cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:15.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:13.237+0000 7f8313ba1640 1 --2- 172.21.15.116:0/3629247043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f830c101c00 0x7f830c19cb60 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00c920 tx=0x7f82fc00cde0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:13.237+0000 7f8313ba1640 1 -- 172.21.15.116:0/3629247043 shutdown_connections 2024-09-12T03:19:15.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:13.237+0000 7f8313ba1640 1 --2- 172.21.15.116:0/3629247043 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f82f80a2b90 0x7f82f80a5050 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:13.237+0000 7f8313ba1640 1 --2- 172.21.15.116:0/3629247043 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f830c1025c0 0x7f830c19f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:13.237+0000 7f8313ba1640 1 --2- 172.21.15.116:0/3629247043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f830c101c00 0x7f830c19cb60 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:13.237+0000 7f8313ba1640 1 --2- 172.21.15.116:0/3629247043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f830c100a10 0x7f830c069ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:13.237+0000 7f8313ba1640 1 -- 172.21.15.116:0/3629247043 >> 172.21.15.116:0/3629247043 conn(0x7f830c0fc170 msgr2=0x7f830c104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:15.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:13.237+0000 7f8313ba1640 1 -- 172.21.15.116:0/3629247043 shutdown_connections 2024-09-12T03:19:15.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:13.237+0000 7f8313ba1640 1 -- 172.21.15.116:0/3629247043 wait complete. 2024-09-12T03:19:15.104 DEBUG:tasks.check_counter:Getting stats from a 2024-09-12T03:19:15.104 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-12T03:19:15.105 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:13 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/928959942' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:19:15.105 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:13 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3316961184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:19:15.105 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:13 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3629247043' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T03:19:15.105 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:14 smithi116 ceph-mon[31410]: pgmap v825: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-12T03:19:15.362 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:15.883 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:19:15.883 INFO:teuthology.orchestra.run.smithi116.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:19:15.883 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_messages": 0, 2024-09-12T03:19:15.883 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages": 2, 2024-09-12T03:19:15.883 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_bytes": 0, 2024-09-12T03:19:15.883 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_bytes": 0, 2024-09-12T03:19:15.884 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_created_connections": 4, 2024-09-12T03:19:15.884 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_active_connections": 0, 2024-09-12T03:19:15.884 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_total_time": 0.001185700, 2024-09-12T03:19:15.884 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T03:19:15.884 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_recv_time": 0.000213587, 2024-09-12T03:19:15.884 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:19:15.884 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:15.884 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-09-12T03:19:15.884 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-09-12T03:19:15.884 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-09-12T03:19:15.885 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.885 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:15.885 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-09-12T03:19:15.885 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-09-12T03:19:15.885 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-09-12T03:19:15.885 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.885 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:19:15.885 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:19:15.885 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.885 INFO:teuthology.orchestra.run.smithi116.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:19:15.885 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_messages": 2, 2024-09-12T03:19:15.885 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages": 226, 2024-09-12T03:19:15.886 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_bytes": 320, 2024-09-12T03:19:15.886 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_bytes": 197216, 2024-09-12T03:19:15.886 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_created_connections": 4, 2024-09-12T03:19:15.886 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_active_connections": 2, 2024-09-12T03:19:15.886 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_total_time": 0.017742162, 2024-09-12T03:19:15.886 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_send_time": 0.013619685, 2024-09-12T03:19:15.886 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_recv_time": 0.001899104, 2024-09-12T03:19:15.886 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:19:15.886 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:15.886 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 224, 2024-09-12T03:19:15.886 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.013509099, 2024-09-12T03:19:15.886 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000060308 2024-09-12T03:19:15.886 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.887 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:15.887 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-09-12T03:19:15.887 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-09-12T03:19:15.887 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-09-12T03:19:15.887 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.887 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T03:19:15.887 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_encrypted_bytes": 197216 2024-09-12T03:19:15.887 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.887 INFO:teuthology.orchestra.run.smithi116.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:19:15.887 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_messages": 306, 2024-09-12T03:19:15.887 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages": 299, 2024-09-12T03:19:15.887 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_bytes": 780720, 2024-09-12T03:19:15.887 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_bytes": 183408, 2024-09-12T03:19:15.887 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_created_connections": 2, 2024-09-12T03:19:15.888 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_active_connections": 1, 2024-09-12T03:19:15.888 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_total_time": 0.044217805, 2024-09-12T03:19:15.888 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_send_time": 0.021066275, 2024-09-12T03:19:15.888 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_recv_time": 0.027898863, 2024-09-12T03:19:15.888 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:19:15.888 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:15.888 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 299, 2024-09-12T03:19:15.888 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.020661860, 2024-09-12T03:19:15.888 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000069103 2024-09-12T03:19:15.888 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.888 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:15.889 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-09-12T03:19:15.889 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-09-12T03:19:15.889 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-09-12T03:19:15.889 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.889 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_encrypted_bytes": 780720, 2024-09-12T03:19:15.889 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_encrypted_bytes": 183408 2024-09-12T03:19:15.889 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.889 INFO:teuthology.orchestra.run.smithi116.stdout: "cct": { 2024-09-12T03:19:15.889 INFO:teuthology.orchestra.run.smithi116.stdout: "total_workers": 0, 2024-09-12T03:19:15.889 INFO:teuthology.orchestra.run.smithi116.stdout: "unhealthy_workers": 0 2024-09-12T03:19:15.889 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.889 INFO:teuthology.orchestra.run.smithi116.stdout: "mempool": { 2024-09-12T03:19:15.889 INFO:teuthology.orchestra.run.smithi116.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:19:15.890 INFO:teuthology.orchestra.run.smithi116.stdout: "bloom_filter_items": 0, 2024-09-12T03:19:15.890 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:19:15.890 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:19:15.890 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:19:15.890 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:19:15.890 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:19:15.890 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:19:15.890 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:19:15.890 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:19:15.890 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:19:15.890 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:19:15.890 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:19:15.890 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:19:15.890 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:19:15.891 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_extent_items": 0, 2024-09-12T03:19:15.891 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:19:15.891 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_blob_items": 0, 2024-09-12T03:19:15.891 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:19:15.891 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:19:15.891 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:19:15.891 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:19:15.891 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:19:15.891 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:19:15.891 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:19:15.892 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_txc_items": 0, 2024-09-12T03:19:15.892 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:19:15.892 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:19:15.892 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:19:15.892 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_items": 0, 2024-09-12T03:19:15.892 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_bytes": 0, 2024-09-12T03:19:15.892 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_items": 0, 2024-09-12T03:19:15.892 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:19:15.892 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:19:15.892 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:19:15.892 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:19:15.892 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_anon_bytes": 148784, 2024-09-12T03:19:15.893 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_anon_items": 31, 2024-09-12T03:19:15.893 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:19:15.893 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_meta_items": 0, 2024-09-12T03:19:15.893 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_bytes": 0, 2024-09-12T03:19:15.893 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_items": 0, 2024-09-12T03:19:15.893 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:19:15.893 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_mapbl_items": 0, 2024-09-12T03:19:15.893 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:19:15.893 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_pglog_items": 0, 2024-09-12T03:19:15.893 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_bytes": 0, 2024-09-12T03:19:15.893 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_items": 0, 2024-09-12T03:19:15.893 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:19:15.893 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:19:15.893 INFO:teuthology.orchestra.run.smithi116.stdout: "pgmap_bytes": 0, 2024-09-12T03:19:15.894 INFO:teuthology.orchestra.run.smithi116.stdout: "pgmap_items": 0, 2024-09-12T03:19:15.894 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_co_bytes": 928, 2024-09-12T03:19:15.894 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_co_items": 2, 2024-09-12T03:19:15.894 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_1_bytes": 0, 2024-09-12T03:19:15.894 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_1_items": 0, 2024-09-12T03:19:15.894 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_2_bytes": 0, 2024-09-12T03:19:15.894 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_2_items": 0 2024-09-12T03:19:15.894 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:15.894 INFO:teuthology.orchestra.run.smithi116.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:19:15.894 INFO:teuthology.orchestra.run.smithi116.stdout: "val": 0, 2024-09-12T03:19:15.894 INFO:teuthology.orchestra.run.smithi116.stdout: "max": 104857600, 2024-09-12T03:19:15.894 INFO:teuthology.orchestra.run.smithi116.stdout: "get_started": 0, 2024-09-12T03:19:15.894 INFO:teuthology.orchestra.run.smithi116.stdout: "get": 304, 2024-09-12T03:19:15.894 INFO:teuthology.orchestra.run.smithi116.stdout: "get_sum": 735055, 2024-09-12T03:19:15.895 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:15.895 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_success": 304, 2024-09-12T03:19:15.895 INFO:teuthology.orchestra.run.smithi116.stdout: "take": 0, 2024-09-12T03:19:15.895 INFO:teuthology.orchestra.run.smithi116.stdout: "take_sum": 0, 2024-09-12T03:19:15.895 INFO:teuthology.orchestra.run.smithi116.stdout: "put": 304, 2024-09-12T03:19:15.895 INFO:teuthology.orchestra.run.smithi116.stdout: "put_sum": 735055, 2024-09-12T03:19:15.895 INFO:teuthology.orchestra.run.smithi116.stdout: "wait": { 2024-09-12T03:19:15.895 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-09-12T03:19:15.895 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-09-12T03:19:15.895 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-09-12T03:19:15.895 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:15.895 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:15.895 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:19:16.650 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-12T03:19:16.651 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-12T03:19:16.651 DEBUG:tasks.check_counter:Getting stats from d 2024-09-12T03:19:16.651 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-12T03:19:16.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:16 smithi116 ceph-mon[31410]: pgmap v826: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-12T03:19:16.899 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:16.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:16 smithi183 ceph-mon[35535]: pgmap v826: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-12T03:19:16.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:16 smithi177 ceph-mon[35305]: pgmap v826: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-12T03:19:17.427 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:19:17.427 INFO:teuthology.orchestra.run.smithi116.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:19:17.428 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_messages": 307, 2024-09-12T03:19:17.428 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages": 302, 2024-09-12T03:19:17.428 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_bytes": 779056, 2024-09-12T03:19:17.428 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_bytes": 186160, 2024-09-12T03:19:17.428 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_created_connections": 4, 2024-09-12T03:19:17.428 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_active_connections": 0, 2024-09-12T03:19:17.428 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_total_time": 0.042604977, 2024-09-12T03:19:17.428 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_send_time": 0.021324575, 2024-09-12T03:19:17.428 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_recv_time": 0.025258759, 2024-09-12T03:19:17.428 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:19:17.428 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:17.428 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 301, 2024-09-12T03:19:17.429 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.016172086, 2024-09-12T03:19:17.429 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000053727 2024-09-12T03:19:17.429 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:17.429 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:17.429 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-09-12T03:19:17.429 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-09-12T03:19:17.429 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-09-12T03:19:17.429 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:17.429 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_encrypted_bytes": 779056, 2024-09-12T03:19:17.430 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_encrypted_bytes": 186160 2024-09-12T03:19:17.430 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:17.430 INFO:teuthology.orchestra.run.smithi116.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:19:17.430 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_messages": 2, 2024-09-12T03:19:17.430 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages": 231, 2024-09-12T03:19:17.430 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_bytes": 320, 2024-09-12T03:19:17.430 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_bytes": 198096, 2024-09-12T03:19:17.430 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_created_connections": 4, 2024-09-12T03:19:17.430 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_active_connections": 2, 2024-09-12T03:19:17.430 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_total_time": 0.018370721, 2024-09-12T03:19:17.430 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_send_time": 0.014570191, 2024-09-12T03:19:17.431 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_recv_time": 0.001410297, 2024-09-12T03:19:17.431 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:19:17.431 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:17.431 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 229, 2024-09-12T03:19:17.431 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.014208314, 2024-09-12T03:19:17.431 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000062045 2024-09-12T03:19:17.431 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:17.431 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:17.431 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-09-12T03:19:17.431 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-09-12T03:19:17.431 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-09-12T03:19:17.431 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:17.431 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T03:19:17.432 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_encrypted_bytes": 198096 2024-09-12T03:19:17.432 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:17.432 INFO:teuthology.orchestra.run.smithi116.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:19:17.432 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_messages": 4, 2024-09-12T03:19:17.432 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages": 4, 2024-09-12T03:19:17.432 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_bytes": 3024, 2024-09-12T03:19:17.432 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_bytes": 448, 2024-09-12T03:19:17.432 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_created_connections": 2, 2024-09-12T03:19:17.432 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_active_connections": 1, 2024-09-12T03:19:17.432 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_total_time": 0.002223983, 2024-09-12T03:19:17.432 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_send_time": 0.000047636, 2024-09-12T03:19:17.432 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_recv_time": 0.002172140, 2024-09-12T03:19:17.433 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:19:17.433 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:17.433 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 3, 2024-09-12T03:19:17.433 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.003952177, 2024-09-12T03:19:17.433 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.001317392 2024-09-12T03:19:17.433 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:17.433 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:17.433 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-09-12T03:19:17.433 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-09-12T03:19:17.433 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-09-12T03:19:17.433 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:17.433 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T03:19:17.433 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T03:19:17.433 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:17.434 INFO:teuthology.orchestra.run.smithi116.stdout: "cct": { 2024-09-12T03:19:17.434 INFO:teuthology.orchestra.run.smithi116.stdout: "total_workers": 0, 2024-09-12T03:19:17.434 INFO:teuthology.orchestra.run.smithi116.stdout: "unhealthy_workers": 0 2024-09-12T03:19:17.434 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:17.434 INFO:teuthology.orchestra.run.smithi116.stdout: "mempool": { 2024-09-12T03:19:17.434 INFO:teuthology.orchestra.run.smithi116.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:19:17.434 INFO:teuthology.orchestra.run.smithi116.stdout: "bloom_filter_items": 0, 2024-09-12T03:19:17.434 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:19:17.434 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:19:17.434 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:19:17.434 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:19:17.434 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:19:17.434 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:19:17.435 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:19:17.435 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:19:17.435 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:19:17.435 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:19:17.435 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:19:17.435 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:19:17.435 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:19:17.435 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_extent_items": 0, 2024-09-12T03:19:17.435 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:19:17.435 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_blob_items": 0, 2024-09-12T03:19:17.435 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:19:17.435 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:19:17.435 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:19:17.436 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:19:17.436 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:19:17.436 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:19:17.436 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:19:17.436 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_txc_items": 0, 2024-09-12T03:19:17.436 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:19:17.436 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:19:17.436 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:19:17.436 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_items": 0, 2024-09-12T03:19:17.436 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_bytes": 0, 2024-09-12T03:19:17.436 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_items": 0, 2024-09-12T03:19:17.436 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:19:17.436 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:19:17.436 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:19:17.437 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:19:17.437 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_anon_bytes": 148721, 2024-09-12T03:19:17.437 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_anon_items": 30, 2024-09-12T03:19:17.437 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:19:17.437 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_meta_items": 0, 2024-09-12T03:19:17.437 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_bytes": 0, 2024-09-12T03:19:17.437 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_items": 0, 2024-09-12T03:19:17.437 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:19:17.437 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_mapbl_items": 0, 2024-09-12T03:19:17.437 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:19:17.437 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_pglog_items": 0, 2024-09-12T03:19:17.437 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_bytes": 0, 2024-09-12T03:19:17.438 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_items": 0, 2024-09-12T03:19:17.438 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:19:17.438 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:19:17.438 INFO:teuthology.orchestra.run.smithi116.stdout: "pgmap_bytes": 0, 2024-09-12T03:19:17.438 INFO:teuthology.orchestra.run.smithi116.stdout: "pgmap_items": 0, 2024-09-12T03:19:17.438 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_co_bytes": 928, 2024-09-12T03:19:17.438 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_co_items": 2, 2024-09-12T03:19:17.438 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_1_bytes": 0, 2024-09-12T03:19:17.438 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_1_items": 0, 2024-09-12T03:19:17.438 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_2_bytes": 0, 2024-09-12T03:19:17.438 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_2_items": 0 2024-09-12T03:19:17.438 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:17.438 INFO:teuthology.orchestra.run.smithi116.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:19:17.439 INFO:teuthology.orchestra.run.smithi116.stdout: "val": 0, 2024-09-12T03:19:17.439 INFO:teuthology.orchestra.run.smithi116.stdout: "max": 104857600, 2024-09-12T03:19:17.439 INFO:teuthology.orchestra.run.smithi116.stdout: "get_started": 0, 2024-09-12T03:19:17.439 INFO:teuthology.orchestra.run.smithi116.stdout: "get": 309, 2024-09-12T03:19:17.439 INFO:teuthology.orchestra.run.smithi116.stdout: "get_sum": 735705, 2024-09-12T03:19:17.439 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:17.439 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_success": 309, 2024-09-12T03:19:17.439 INFO:teuthology.orchestra.run.smithi116.stdout: "take": 0, 2024-09-12T03:19:17.439 INFO:teuthology.orchestra.run.smithi116.stdout: "take_sum": 0, 2024-09-12T03:19:17.439 INFO:teuthology.orchestra.run.smithi116.stdout: "put": 309, 2024-09-12T03:19:17.439 INFO:teuthology.orchestra.run.smithi116.stdout: "put_sum": 735705, 2024-09-12T03:19:17.439 INFO:teuthology.orchestra.run.smithi116.stdout: "wait": { 2024-09-12T03:19:17.439 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-09-12T03:19:17.440 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-09-12T03:19:17.440 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-09-12T03:19:17.440 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:17.440 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:17.440 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:19:18.279 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-12T03:19:18.279 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-12T03:19:18.280 DEBUG:tasks.check_counter:Getting stats from g 2024-09-12T03:19:18.280 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-12T03:19:18.531 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:18.946 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:18 smithi116 ceph-mon[31410]: pgmap v827: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-12T03:19:18.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:18 smithi183 ceph-mon[35535]: pgmap v827: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-12T03:19:19.004 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:18 smithi177 ceph-mon[35305]: pgmap v827: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-12T03:19:19.068 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:19:19.068 INFO:teuthology.orchestra.run.smithi116.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:19:19.068 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_messages": 308, 2024-09-12T03:19:19.068 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages": 301, 2024-09-12T03:19:19.068 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_bytes": 781264, 2024-09-12T03:19:19.068 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_bytes": 184688, 2024-09-12T03:19:19.068 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_created_connections": 4, 2024-09-12T03:19:19.068 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_active_connections": 0, 2024-09-12T03:19:19.069 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_total_time": 0.046745159, 2024-09-12T03:19:19.069 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_send_time": 0.021676414, 2024-09-12T03:19:19.069 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_recv_time": 0.029308610, 2024-09-12T03:19:19.069 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:19:19.069 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:19.069 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 301, 2024-09-12T03:19:19.069 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.020003118, 2024-09-12T03:19:19.069 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000066455 2024-09-12T03:19:19.069 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:19.069 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:19.069 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-09-12T03:19:19.069 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-09-12T03:19:19.070 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-09-12T03:19:19.070 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:19.070 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_encrypted_bytes": 781264, 2024-09-12T03:19:19.070 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_encrypted_bytes": 184688 2024-09-12T03:19:19.070 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:19.070 INFO:teuthology.orchestra.run.smithi116.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:19:19.070 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_messages": 2, 2024-09-12T03:19:19.070 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages": 228, 2024-09-12T03:19:19.070 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_bytes": 320, 2024-09-12T03:19:19.070 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_bytes": 197568, 2024-09-12T03:19:19.070 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_created_connections": 4, 2024-09-12T03:19:19.070 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_active_connections": 2, 2024-09-12T03:19:19.070 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_total_time": 0.017279343, 2024-09-12T03:19:19.070 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_send_time": 0.013289831, 2024-09-12T03:19:19.071 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_recv_time": 0.001646017, 2024-09-12T03:19:19.071 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:19:19.071 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:19.071 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 226, 2024-09-12T03:19:19.071 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.013700136, 2024-09-12T03:19:19.071 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000060620 2024-09-12T03:19:19.071 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:19.071 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:19.071 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-09-12T03:19:19.071 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-09-12T03:19:19.071 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-09-12T03:19:19.071 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:19.071 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T03:19:19.072 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_encrypted_bytes": 197568 2024-09-12T03:19:19.072 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:19.072 INFO:teuthology.orchestra.run.smithi116.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:19:19.072 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_messages": 0, 2024-09-12T03:19:19.072 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages": 2, 2024-09-12T03:19:19.072 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_bytes": 0, 2024-09-12T03:19:19.072 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_bytes": 0, 2024-09-12T03:19:19.072 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_created_connections": 2, 2024-09-12T03:19:19.072 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_active_connections": 1, 2024-09-12T03:19:19.072 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_total_time": 0.000265210, 2024-09-12T03:19:19.072 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T03:19:19.072 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_recv_time": 0.000205347, 2024-09-12T03:19:19.072 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:19:19.073 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:19.073 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-09-12T03:19:19.073 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-09-12T03:19:19.073 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-09-12T03:19:19.073 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:19.073 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:19.073 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-09-12T03:19:19.073 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-09-12T03:19:19.073 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-09-12T03:19:19.073 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:19.073 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:19:19.073 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:19:19.073 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:19.074 INFO:teuthology.orchestra.run.smithi116.stdout: "cct": { 2024-09-12T03:19:19.074 INFO:teuthology.orchestra.run.smithi116.stdout: "total_workers": 0, 2024-09-12T03:19:19.074 INFO:teuthology.orchestra.run.smithi116.stdout: "unhealthy_workers": 0 2024-09-12T03:19:19.074 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:19.074 INFO:teuthology.orchestra.run.smithi116.stdout: "mempool": { 2024-09-12T03:19:19.074 INFO:teuthology.orchestra.run.smithi116.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:19:19.074 INFO:teuthology.orchestra.run.smithi116.stdout: "bloom_filter_items": 0, 2024-09-12T03:19:19.074 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:19:19.074 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:19:19.074 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:19:19.074 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:19:19.074 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:19:19.074 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:19:19.074 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:19:19.075 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:19:19.075 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:19:19.075 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:19:19.075 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:19:19.075 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:19:19.075 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:19:19.075 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_extent_items": 0, 2024-09-12T03:19:19.075 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:19:19.075 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_blob_items": 0, 2024-09-12T03:19:19.075 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:19:19.075 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:19:19.075 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:19:19.075 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:19:19.076 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:19:19.076 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:19:19.076 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:19:19.076 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_txc_items": 0, 2024-09-12T03:19:19.076 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:19:19.076 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:19:19.076 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:19:19.076 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_items": 0, 2024-09-12T03:19:19.076 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_bytes": 0, 2024-09-12T03:19:19.076 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_items": 0, 2024-09-12T03:19:19.076 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:19:19.076 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:19:19.076 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:19:19.077 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:19:19.077 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_anon_bytes": 148721, 2024-09-12T03:19:19.077 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_anon_items": 30, 2024-09-12T03:19:19.077 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:19:19.077 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_meta_items": 0, 2024-09-12T03:19:19.077 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_bytes": 0, 2024-09-12T03:19:19.077 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_items": 0, 2024-09-12T03:19:19.077 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:19:19.077 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_mapbl_items": 0, 2024-09-12T03:19:19.077 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:19:19.077 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_pglog_items": 0, 2024-09-12T03:19:19.077 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_bytes": 0, 2024-09-12T03:19:19.077 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_items": 0, 2024-09-12T03:19:19.078 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:19:19.078 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:19:19.078 INFO:teuthology.orchestra.run.smithi116.stdout: "pgmap_bytes": 0, 2024-09-12T03:19:19.078 INFO:teuthology.orchestra.run.smithi116.stdout: "pgmap_items": 0, 2024-09-12T03:19:19.078 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_co_bytes": 928, 2024-09-12T03:19:19.078 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_co_items": 2, 2024-09-12T03:19:19.078 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_1_bytes": 0, 2024-09-12T03:19:19.078 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_1_items": 0, 2024-09-12T03:19:19.078 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_2_bytes": 0, 2024-09-12T03:19:19.078 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_2_items": 0 2024-09-12T03:19:19.078 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:19.078 INFO:teuthology.orchestra.run.smithi116.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:19:19.078 INFO:teuthology.orchestra.run.smithi116.stdout: "val": 0, 2024-09-12T03:19:19.078 INFO:teuthology.orchestra.run.smithi116.stdout: "max": 104857600, 2024-09-12T03:19:19.079 INFO:teuthology.orchestra.run.smithi116.stdout: "get_started": 0, 2024-09-12T03:19:19.079 INFO:teuthology.orchestra.run.smithi116.stdout: "get": 306, 2024-09-12T03:19:19.079 INFO:teuthology.orchestra.run.smithi116.stdout: "get_sum": 735315, 2024-09-12T03:19:19.079 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:19.079 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_success": 306, 2024-09-12T03:19:19.079 INFO:teuthology.orchestra.run.smithi116.stdout: "take": 0, 2024-09-12T03:19:19.079 INFO:teuthology.orchestra.run.smithi116.stdout: "take_sum": 0, 2024-09-12T03:19:19.079 INFO:teuthology.orchestra.run.smithi116.stdout: "put": 306, 2024-09-12T03:19:19.079 INFO:teuthology.orchestra.run.smithi116.stdout: "put_sum": 735315, 2024-09-12T03:19:19.079 INFO:teuthology.orchestra.run.smithi116.stdout: "wait": { 2024-09-12T03:19:19.079 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-09-12T03:19:19.079 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-09-12T03:19:19.079 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-09-12T03:19:19.079 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:19.080 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:19.080 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:19:19.861 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-12T03:19:19.861 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-12T03:19:19.861 DEBUG:tasks.check_counter:Getting stats from j 2024-09-12T03:19:19.862 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-12T03:19:20.109 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:20.626 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-12T03:19:20.626 INFO:teuthology.orchestra.run.smithi116.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:19:20.626 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_messages": 4, 2024-09-12T03:19:20.626 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages": 4, 2024-09-12T03:19:20.626 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_bytes": 3024, 2024-09-12T03:19:20.627 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_bytes": 448, 2024-09-12T03:19:20.627 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_created_connections": 4, 2024-09-12T03:19:20.627 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_active_connections": 0, 2024-09-12T03:19:20.627 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_total_time": 0.003393666, 2024-09-12T03:19:20.627 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_send_time": 0.000051001, 2024-09-12T03:19:20.627 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_recv_time": 0.002485355, 2024-09-12T03:19:20.627 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:19:20.627 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:20.627 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 3, 2024-09-12T03:19:20.627 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.004300426, 2024-09-12T03:19:20.628 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.001433475 2024-09-12T03:19:20.628 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:20.628 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:20.628 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-09-12T03:19:20.628 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-09-12T03:19:20.628 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-09-12T03:19:20.628 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:20.628 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T03:19:20.628 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T03:19:20.628 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:20.629 INFO:teuthology.orchestra.run.smithi116.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:19:20.629 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_messages": 2, 2024-09-12T03:19:20.629 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages": 232, 2024-09-12T03:19:20.629 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_bytes": 320, 2024-09-12T03:19:20.629 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_bytes": 198272, 2024-09-12T03:19:20.629 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_created_connections": 4, 2024-09-12T03:19:20.629 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_active_connections": 2, 2024-09-12T03:19:20.629 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_total_time": 0.019159230, 2024-09-12T03:19:20.629 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_send_time": 0.014755333, 2024-09-12T03:19:20.629 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_recv_time": 0.001868480, 2024-09-12T03:19:20.630 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:19:20.630 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:20.630 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 230, 2024-09-12T03:19:20.630 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.014306117, 2024-09-12T03:19:20.630 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000062200 2024-09-12T03:19:20.630 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:20.630 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:20.630 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-09-12T03:19:20.630 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-09-12T03:19:20.630 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-09-12T03:19:20.630 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:20.630 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T03:19:20.631 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_encrypted_bytes": 198272 2024-09-12T03:19:20.631 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:20.631 INFO:teuthology.orchestra.run.smithi116.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:19:20.631 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_messages": 309, 2024-09-12T03:19:20.631 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages": 304, 2024-09-12T03:19:20.631 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_bytes": 779600, 2024-09-12T03:19:20.631 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_bytes": 187440, 2024-09-12T03:19:20.632 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_created_connections": 2, 2024-09-12T03:19:20.632 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_active_connections": 1, 2024-09-12T03:19:20.632 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_total_time": 0.044169979, 2024-09-12T03:19:20.632 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_send_time": 0.022538242, 2024-09-12T03:19:20.632 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_recv_time": 0.026371753, 2024-09-12T03:19:20.632 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:19:20.632 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:20.632 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 303, 2024-09-12T03:19:20.632 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.019426693, 2024-09-12T03:19:20.633 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000064114 2024-09-12T03:19:20.633 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:20.633 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:20.633 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-09-12T03:19:20.633 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-09-12T03:19:20.634 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-09-12T03:19:20.634 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:20.634 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_encrypted_bytes": 779600, 2024-09-12T03:19:20.634 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_encrypted_bytes": 187440 2024-09-12T03:19:20.634 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:20.634 INFO:teuthology.orchestra.run.smithi116.stdout: "cct": { 2024-09-12T03:19:20.634 INFO:teuthology.orchestra.run.smithi116.stdout: "total_workers": 0, 2024-09-12T03:19:20.634 INFO:teuthology.orchestra.run.smithi116.stdout: "unhealthy_workers": 0 2024-09-12T03:19:20.634 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:20.634 INFO:teuthology.orchestra.run.smithi116.stdout: "mempool": { 2024-09-12T03:19:20.634 INFO:teuthology.orchestra.run.smithi116.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:19:20.634 INFO:teuthology.orchestra.run.smithi116.stdout: "bloom_filter_items": 0, 2024-09-12T03:19:20.635 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:19:20.635 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:19:20.635 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:19:20.635 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:19:20.635 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:19:20.635 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:19:20.635 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:19:20.635 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:19:20.635 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:19:20.635 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:19:20.635 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:19:20.636 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:19:20.636 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:19:20.636 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_extent_items": 0, 2024-09-12T03:19:20.636 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:19:20.636 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_blob_items": 0, 2024-09-12T03:19:20.636 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:19:20.636 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:19:20.636 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:19:20.636 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:19:20.636 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:19:20.636 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:19:20.636 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:19:20.637 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_txc_items": 0, 2024-09-12T03:19:20.637 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:19:20.637 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:19:20.637 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:19:20.637 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_items": 0, 2024-09-12T03:19:20.637 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_bytes": 0, 2024-09-12T03:19:20.637 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_items": 0, 2024-09-12T03:19:20.637 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:19:20.637 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:19:20.637 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:19:20.638 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:19:20.638 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_anon_bytes": 148744, 2024-09-12T03:19:20.638 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_anon_items": 30, 2024-09-12T03:19:20.638 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:19:20.638 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_meta_items": 0, 2024-09-12T03:19:20.638 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_bytes": 0, 2024-09-12T03:19:20.638 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_items": 0, 2024-09-12T03:19:20.638 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:19:20.638 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_mapbl_items": 0, 2024-09-12T03:19:20.638 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:19:20.639 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_pglog_items": 0, 2024-09-12T03:19:20.639 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_bytes": 0, 2024-09-12T03:19:20.639 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_items": 0, 2024-09-12T03:19:20.639 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:19:20.639 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:19:20.639 INFO:teuthology.orchestra.run.smithi116.stdout: "pgmap_bytes": 0, 2024-09-12T03:19:20.639 INFO:teuthology.orchestra.run.smithi116.stdout: "pgmap_items": 0, 2024-09-12T03:19:20.639 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_co_bytes": 928, 2024-09-12T03:19:20.639 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_co_items": 2, 2024-09-12T03:19:20.639 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_1_bytes": 0, 2024-09-12T03:19:20.639 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_1_items": 0, 2024-09-12T03:19:20.639 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_2_bytes": 0, 2024-09-12T03:19:20.640 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_2_items": 0 2024-09-12T03:19:20.640 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-12T03:19:20.640 INFO:teuthology.orchestra.run.smithi116.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:19:20.640 INFO:teuthology.orchestra.run.smithi116.stdout: "val": 0, 2024-09-12T03:19:20.640 INFO:teuthology.orchestra.run.smithi116.stdout: "max": 104857600, 2024-09-12T03:19:20.640 INFO:teuthology.orchestra.run.smithi116.stdout: "get_started": 0, 2024-09-12T03:19:20.640 INFO:teuthology.orchestra.run.smithi116.stdout: "get": 311, 2024-09-12T03:19:20.640 INFO:teuthology.orchestra.run.smithi116.stdout: "get_sum": 735965, 2024-09-12T03:19:20.641 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:20.641 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_success": 311, 2024-09-12T03:19:20.641 INFO:teuthology.orchestra.run.smithi116.stdout: "take": 0, 2024-09-12T03:19:20.641 INFO:teuthology.orchestra.run.smithi116.stdout: "take_sum": 0, 2024-09-12T03:19:20.641 INFO:teuthology.orchestra.run.smithi116.stdout: "put": 311, 2024-09-12T03:19:20.641 INFO:teuthology.orchestra.run.smithi116.stdout: "put_sum": 735965, 2024-09-12T03:19:20.641 INFO:teuthology.orchestra.run.smithi116.stdout: "wait": { 2024-09-12T03:19:20.641 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-09-12T03:19:20.641 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-09-12T03:19:20.641 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-09-12T03:19:20.642 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:20.642 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-12T03:19:20.642 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-12T03:19:20.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:20 smithi116 ceph-mon[31410]: pgmap v828: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T03:19:20.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:20 smithi183 ceph-mon[35535]: pgmap v828: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T03:19:20.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:20 smithi177 ceph-mon[35305]: pgmap v828: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T03:19:21.494 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-12T03:19:21.494 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-12T03:19:21.494 DEBUG:tasks.check_counter:Getting stats from b 2024-09-12T03:19:21.494 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-12T03:19:21.752 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:19:22.237 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-09-12T03:19:22.237 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:19:22.237 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 3493, 2024-09-12T03:19:22.237 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 3166, 2024-09-12T03:19:22.237 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 1505310, 2024-09-12T03:19:22.238 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 7912140, 2024-09-12T03:19:22.238 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 299, 2024-09-12T03:19:22.238 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 4, 2024-09-12T03:19:22.238 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.384622596, 2024-09-12T03:19:22.238 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.112909557, 2024-09-12T03:19:22.238 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.204813396, 2024-09-12T03:19:22.238 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.049715597, 2024-09-12T03:19:22.238 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:22.238 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 3168, 2024-09-12T03:19:22.238 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.233954445, 2024-09-12T03:19:22.238 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000073849 2024-09-12T03:19:22.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.239 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:22.239 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 1641, 2024-09-12T03:19:22.239 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000338558, 2024-09-12T03:19:22.239 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000206 2024-09-12T03:19:22.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.239 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 860928, 2024-09-12T03:19:22.239 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 229328 2024-09-12T03:19:22.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.239 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:19:22.239 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 2823, 2024-09-12T03:19:22.239 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 4465, 2024-09-12T03:19:22.239 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 1007310, 2024-09-12T03:19:22.239 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 2179639, 2024-09-12T03:19:22.240 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 28, 2024-09-12T03:19:22.240 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 5, 2024-09-12T03:19:22.240 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.351916271, 2024-09-12T03:19:22.240 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.124074535, 2024-09-12T03:19:22.240 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.157364366, 2024-09-12T03:19:22.240 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.072427710, 2024-09-12T03:19:22.240 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:22.240 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 4464, 2024-09-12T03:19:22.241 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.456211795, 2024-09-12T03:19:22.241 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000102197 2024-09-12T03:19:22.241 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.241 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:22.241 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 5285, 2024-09-12T03:19:22.241 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.001131326, 2024-09-12T03:19:22.241 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000214 2024-09-12T03:19:22.241 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.241 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 57056, 2024-09-12T03:19:22.241 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 518832 2024-09-12T03:19:22.241 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.242 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:19:22.242 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 2708, 2024-09-12T03:19:22.242 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 2740, 2024-09-12T03:19:22.242 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 619881, 2024-09-12T03:19:22.242 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 5014308, 2024-09-12T03:19:22.242 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 11, 2024-09-12T03:19:22.242 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 7, 2024-09-12T03:19:22.242 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.243150318, 2024-09-12T03:19:22.242 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.105479161, 2024-09-12T03:19:22.242 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.093453076, 2024-09-12T03:19:22.242 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.044844390, 2024-09-12T03:19:22.242 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:22.242 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2735, 2024-09-12T03:19:22.243 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.624422220, 2024-09-12T03:19:22.243 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000228307 2024-09-12T03:19:22.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.243 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:22.243 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 1808, 2024-09-12T03:19:22.243 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000411601, 2024-09-12T03:19:22.243 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000227 2024-09-12T03:19:22.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.243 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:19:22.243 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:19:22.243 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.243 INFO:teuthology.orchestra.run.smithi177.stdout: "cct": { 2024-09-12T03:19:22.244 INFO:teuthology.orchestra.run.smithi177.stdout: "total_workers": 1, 2024-09-12T03:19:22.244 INFO:teuthology.orchestra.run.smithi177.stdout: "unhealthy_workers": 0 2024-09-12T03:19:22.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.244 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-MDSRank": { 2024-09-12T03:19:22.244 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2024-09-12T03:19:22.244 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2024-09-12T03:19:22.244 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 5427, 2024-09-12T03:19:22.244 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 1.489005463, 2024-09-12T03:19:22.244 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000274369 2024-09-12T03:19:22.244 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-12T03:19:22.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.244 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-PurgeQueue": { 2024-09-12T03:19:22.245 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2024-09-12T03:19:22.245 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2024-09-12T03:19:22.245 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 17, 2024-09-12T03:19:22.245 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000438947, 2024-09-12T03:19:22.245 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000025820 2024-09-12T03:19:22.245 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-12T03:19:22.245 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.245 INFO:teuthology.orchestra.run.smithi177.stdout: "mds": { 2024-09-12T03:19:22.245 INFO:teuthology.orchestra.run.smithi177.stdout: "request": 664, 2024-09-12T03:19:22.245 INFO:teuthology.orchestra.run.smithi177.stdout: "reply": 664, 2024-09-12T03:19:22.245 INFO:teuthology.orchestra.run.smithi177.stdout: "reply_latency": { 2024-09-12T03:19:22.245 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 664, 2024-09-12T03:19:22.245 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 3.079064656, 2024-09-12T03:19:22.245 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.004637145 2024-09-12T03:19:22.246 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.246 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_reply": 0, 2024-09-12T03:19:22.246 INFO:teuthology.orchestra.run.smithi177.stdout: "forward": 0, 2024-09-12T03:19:22.246 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_complete": 0, 2024-09-12T03:19:22.246 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_keys": 0, 2024-09-12T03:19:22.246 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_commit": 12, 2024-09-12T03:19:22.246 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_split": 0, 2024-09-12T03:19:22.246 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_merge": 0, 2024-09-12T03:19:22.246 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes": 25, 2024-09-12T03:19:22.246 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_top": 0, 2024-09-12T03:19:22.246 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_bottom": 25, 2024-09-12T03:19:22.246 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pin_tail": 0, 2024-09-12T03:19:22.247 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pinned": 24, 2024-09-12T03:19:22.247 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_expired": 0, 2024-09-12T03:19:22.247 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_with_caps": 3, 2024-09-12T03:19:22.247 INFO:teuthology.orchestra.run.smithi177.stdout: "caps": 3, 2024-09-12T03:19:22.247 INFO:teuthology.orchestra.run.smithi177.stdout: "subtrees": 2, 2024-09-12T03:19:22.247 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse": 743, 2024-09-12T03:19:22.247 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_hit": 737, 2024-09-12T03:19:22.247 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_forward": 0, 2024-09-12T03:19:22.247 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_discover": 0, 2024-09-12T03:19:22.247 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:19:22.247 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_remote_ino": 0, 2024-09-12T03:19:22.247 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_lock": 0, 2024-09-12T03:19:22.247 INFO:teuthology.orchestra.run.smithi177.stdout: "load_cent": 0, 2024-09-12T03:19:22.248 INFO:teuthology.orchestra.run.smithi177.stdout: "q": 0, 2024-09-12T03:19:22.248 INFO:teuthology.orchestra.run.smithi177.stdout: "exported": 0, 2024-09-12T03:19:22.248 INFO:teuthology.orchestra.run.smithi177.stdout: "exported_inodes": 0, 2024-09-12T03:19:22.248 INFO:teuthology.orchestra.run.smithi177.stdout: "imported": 0, 2024-09-12T03:19:22.248 INFO:teuthology.orchestra.run.smithi177.stdout: "imported_inodes": 0, 2024-09-12T03:19:22.248 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_dir_fetch": 0, 2024-09-12T03:19:22.248 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_backtrace_fetch": 485, 2024-09-12T03:19:22.248 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_peer_discover": 0, 2024-09-12T03:19:22.248 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rfiles": 4, 2024-09-12T03:19:22.249 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rbytes": 234, 2024-09-12T03:19:22.249 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rsnaps": 0, 2024-09-12T03:19:22.249 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_fetch": 485, 2024-09-12T03:19:22.249 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_set_tag": 0, 2024-09-12T03:19:22.249 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:19:22.249 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:19:22.249 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_inodes": 312, 2024-09-12T03:19:22.249 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_base_inodes": 35, 2024-09-12T03:19:22.249 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dirfrag_rstats": 277, 2024-09-12T03:19:22.249 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_file_inodes": 173, 2024-09-12T03:19:22.249 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:19:22.249 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_revoke": 646, 2024-09-12T03:19:22.249 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_grant": 666, 2024-09-12T03:19:22.250 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_trunc": 599, 2024-09-12T03:19:22.250 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:19:22.250 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:19:22.250 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps": 750, 2024-09-12T03:19:22.250 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps_dirty": 710, 2024-09-12T03:19:22.250 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_cap_release": 2, 2024-09-12T03:19:22.250 INFO:teuthology.orchestra.run.smithi177.stdout: "process_request_cap_release": 626 2024-09-12T03:19:22.250 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.250 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_cache": { 2024-09-12T03:19:22.251 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update": 0, 2024-09-12T03:19:22.251 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update_receipt": 0, 2024-09-12T03:19:22.251 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_try_discover": 0, 2024-09-12T03:19:22.251 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_send_discover": 0, 2024-09-12T03:19:22.251 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_handle_discover": 4, 2024-09-12T03:19:22.251 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays": 1, 2024-09-12T03:19:22.251 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_delayed": 0, 2024-09-12T03:19:22.251 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:19:22.251 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_created": 3, 2024-09-12T03:19:22.251 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_enqueued": 2, 2024-09-12T03:19:22.251 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_reintegrated": 0, 2024-09-12T03:19:22.251 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_migrated": 0, 2024-09-12T03:19:22.252 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_processing": 0, 2024-09-12T03:19:22.252 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:19:22.252 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:19:22.252 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_started": 0, 2024-09-12T03:19:22.252 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_completed": 0, 2024-09-12T03:19:22.252 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_quiesce_path": 14, 2024-09-12T03:19:22.252 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_quiesce_inode": 194, 2024-09-12T03:19:22.252 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_enqueue_scrub": 35, 2024-09-12T03:19:22.252 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_exportdir": 0, 2024-09-12T03:19:22.252 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_flush": 0, 2024-09-12T03:19:22.253 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:19:22.253 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragstats": 0, 2024-09-12T03:19:22.253 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_inodestats": 0 2024-09-12T03:19:22.253 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.253 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_log": { 2024-09-12T03:19:22.253 INFO:teuthology.orchestra.run.smithi177.stdout: "evlrg": 0, 2024-09-12T03:19:22.253 INFO:teuthology.orchestra.run.smithi177.stdout: "evadd": 2283, 2024-09-12T03:19:22.253 INFO:teuthology.orchestra.run.smithi177.stdout: "evex": 0, 2024-09-12T03:19:22.253 INFO:teuthology.orchestra.run.smithi177.stdout: "evtrm": 0, 2024-09-12T03:19:22.253 INFO:teuthology.orchestra.run.smithi177.stdout: "ev": 2283, 2024-09-12T03:19:22.253 INFO:teuthology.orchestra.run.smithi177.stdout: "evexg": 0, 2024-09-12T03:19:22.253 INFO:teuthology.orchestra.run.smithi177.stdout: "evexd": 0, 2024-09-12T03:19:22.253 INFO:teuthology.orchestra.run.smithi177.stdout: "segadd": 4, 2024-09-12T03:19:22.254 INFO:teuthology.orchestra.run.smithi177.stdout: "segex": 0, 2024-09-12T03:19:22.254 INFO:teuthology.orchestra.run.smithi177.stdout: "segtrm": 0, 2024-09-12T03:19:22.254 INFO:teuthology.orchestra.run.smithi177.stdout: "seg": 4, 2024-09-12T03:19:22.254 INFO:teuthology.orchestra.run.smithi177.stdout: "segmjr": 2, 2024-09-12T03:19:22.254 INFO:teuthology.orchestra.run.smithi177.stdout: "segexg": 0, 2024-09-12T03:19:22.254 INFO:teuthology.orchestra.run.smithi177.stdout: "segexd": 0, 2024-09-12T03:19:22.254 INFO:teuthology.orchestra.run.smithi177.stdout: "expos": 4194304, 2024-09-12T03:19:22.254 INFO:teuthology.orchestra.run.smithi177.stdout: "wrpos": 11769917, 2024-09-12T03:19:22.254 INFO:teuthology.orchestra.run.smithi177.stdout: "rdpos": 0, 2024-09-12T03:19:22.254 INFO:teuthology.orchestra.run.smithi177.stdout: "jlat": { 2024-09-12T03:19:22.254 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2268, 2024-09-12T03:19:22.254 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 4.527116068, 2024-09-12T03:19:22.254 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.001996082 2024-09-12T03:19:22.254 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.255 INFO:teuthology.orchestra.run.smithi177.stdout: "replayed": 0 2024-09-12T03:19:22.255 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.255 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_mem": { 2024-09-12T03:19:22.255 INFO:teuthology.orchestra.run.smithi177.stdout: "ino": 24, 2024-09-12T03:19:22.255 INFO:teuthology.orchestra.run.smithi177.stdout: "ino+": 61, 2024-09-12T03:19:22.255 INFO:teuthology.orchestra.run.smithi177.stdout: "ino-": 37, 2024-09-12T03:19:22.255 INFO:teuthology.orchestra.run.smithi177.stdout: "dir": 19, 2024-09-12T03:19:22.255 INFO:teuthology.orchestra.run.smithi177.stdout: "dir+": 20, 2024-09-12T03:19:22.255 INFO:teuthology.orchestra.run.smithi177.stdout: "dir-": 1, 2024-09-12T03:19:22.255 INFO:teuthology.orchestra.run.smithi177.stdout: "dn": 25, 2024-09-12T03:19:22.256 INFO:teuthology.orchestra.run.smithi177.stdout: "dn+": 29, 2024-09-12T03:19:22.256 INFO:teuthology.orchestra.run.smithi177.stdout: "dn-": 4, 2024-09-12T03:19:22.256 INFO:teuthology.orchestra.run.smithi177.stdout: "cap": 3, 2024-09-12T03:19:22.257 INFO:teuthology.orchestra.run.smithi177.stdout: "cap+": 19, 2024-09-12T03:19:22.257 INFO:teuthology.orchestra.run.smithi177.stdout: "cap-": 16, 2024-09-12T03:19:22.257 INFO:teuthology.orchestra.run.smithi177.stdout: "rss": 56760, 2024-09-12T03:19:22.257 INFO:teuthology.orchestra.run.smithi177.stdout: "heap": 231712 2024-09-12T03:19:22.257 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.257 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_server": { 2024-09-12T03:19:22.257 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_client_request": 1879, 2024-09-12T03:19:22.257 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_server_request": 1, 2024-09-12T03:19:22.257 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_request": 664, 2024-09-12T03:19:22.257 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_session": 23, 2024-09-12T03:19:22.258 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_peer_request": 2, 2024-09-12T03:19:22.258 INFO:teuthology.orchestra.run.smithi177.stdout: "req_create_latency": { 2024-09-12T03:19:22.258 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 18, 2024-09-12T03:19:22.258 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.009846115, 2024-09-12T03:19:22.258 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000547006 2024-09-12T03:19:22.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.258 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getattr_latency": { 2024-09-12T03:19:22.258 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 17, 2024-09-12T03:19:22.258 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.003602501, 2024-09-12T03:19:22.258 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000211911 2024-09-12T03:19:22.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.258 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getfilelock_latency": { 2024-09-12T03:19:22.259 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:22.259 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:22.259 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:22.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.259 INFO:teuthology.orchestra.run.smithi177.stdout: "req_link_latency": { 2024-09-12T03:19:22.259 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:22.259 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:22.259 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:22.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.259 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookup_latency": { 2024-09-12T03:19:22.259 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 3, 2024-09-12T03:19:22.259 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.001540360, 2024-09-12T03:19:22.259 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000513453 2024-09-12T03:19:22.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.260 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookuphash_latency": { 2024-09-12T03:19:22.260 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:22.260 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:22.260 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:22.260 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.260 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupino_latency": { 2024-09-12T03:19:22.260 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2, 2024-09-12T03:19:22.260 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000263046, 2024-09-12T03:19:22.261 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000131523 2024-09-12T03:19:22.261 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.261 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupname_latency": { 2024-09-12T03:19:22.261 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:22.261 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:22.262 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:22.262 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.262 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupparent_latency": { 2024-09-12T03:19:22.262 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:22.262 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:22.262 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:22.262 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.262 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupsnap_latency": { 2024-09-12T03:19:22.262 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:22.262 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:22.262 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:22.262 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.262 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lssnap_latency": { 2024-09-12T03:19:22.263 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:22.263 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:22.263 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:22.263 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.263 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mkdir_latency": { 2024-09-12T03:19:22.263 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 8, 2024-09-12T03:19:22.263 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.013148810, 2024-09-12T03:19:22.263 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.001643601 2024-09-12T03:19:22.263 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.263 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mknod_latency": { 2024-09-12T03:19:22.263 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:22.263 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:22.263 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:22.264 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.264 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mksnap_latency": { 2024-09-12T03:19:22.264 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:22.264 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:22.264 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:22.264 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.264 INFO:teuthology.orchestra.run.smithi177.stdout: "req_open_latency": { 2024-09-12T03:19:22.264 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 1, 2024-09-12T03:19:22.264 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000288002, 2024-09-12T03:19:22.264 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000288002 2024-09-12T03:19:22.264 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.264 INFO:teuthology.orchestra.run.smithi177.stdout: "req_readdir_latency": { 2024-09-12T03:19:22.264 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2, 2024-09-12T03:19:22.265 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.003685376, 2024-09-12T03:19:22.265 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.001842688 2024-09-12T03:19:22.265 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.265 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rename_latency": { 2024-09-12T03:19:22.265 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2, 2024-09-12T03:19:22.265 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.002042051, 2024-09-12T03:19:22.265 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.001021025 2024-09-12T03:19:22.265 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.265 INFO:teuthology.orchestra.run.smithi177.stdout: "req_renamesnap_latency": { 2024-09-12T03:19:22.265 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:22.265 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:22.265 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:22.265 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.266 INFO:teuthology.orchestra.run.smithi177.stdout: "req_snapdiff_latency": { 2024-09-12T03:19:22.266 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:22.266 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:22.266 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:22.266 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.266 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmdir_latency": { 2024-09-12T03:19:22.266 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2, 2024-09-12T03:19:22.266 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.006798610, 2024-09-12T03:19:22.266 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.003399305 2024-09-12T03:19:22.266 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.266 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmsnap_latency": { 2024-09-12T03:19:22.266 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:22.266 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:22.267 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:22.267 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.267 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmxattr_latency": { 2024-09-12T03:19:22.267 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:22.267 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:22.267 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:22.267 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.267 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setattr_latency": { 2024-09-12T03:19:22.267 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 600, 2024-09-12T03:19:22.267 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 3.015533410, 2024-09-12T03:19:22.267 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.005025889 2024-09-12T03:19:22.267 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.267 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setdirlayout_latency": { 2024-09-12T03:19:22.268 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:22.268 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:22.268 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:22.268 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.268 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setfilelock_latency": { 2024-09-12T03:19:22.268 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:22.268 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:22.268 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:22.268 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.268 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setlayout_latency": { 2024-09-12T03:19:22.268 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:22.268 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:22.268 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:22.268 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.269 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setxattr_latency": { 2024-09-12T03:19:22.269 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 8, 2024-09-12T03:19:22.269 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.019115403, 2024-09-12T03:19:22.269 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.002389425 2024-09-12T03:19:22.269 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.269 INFO:teuthology.orchestra.run.smithi177.stdout: "req_symlink_latency": { 2024-09-12T03:19:22.269 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:22.269 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:22.269 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:22.269 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.269 INFO:teuthology.orchestra.run.smithi177.stdout: "req_unlink_latency": { 2024-09-12T03:19:22.269 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 1, 2024-09-12T03:19:22.269 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.003200972, 2024-09-12T03:19:22.270 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.003200972 2024-09-12T03:19:22.270 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.270 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:19:22.270 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:19:22.270 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getvxattr_latency": { 2024-09-12T03:19:22.270 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:22.270 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:22.270 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:22.270 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-12T03:19:22.270 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.270 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_sessions": { 2024-09-12T03:19:22.270 INFO:teuthology.orchestra.run.smithi177.stdout: "session_count": 1, 2024-09-12T03:19:22.270 INFO:teuthology.orchestra.run.smithi177.stdout: "session_add": 3, 2024-09-12T03:19:22.271 INFO:teuthology.orchestra.run.smithi177.stdout: "session_remove": 2, 2024-09-12T03:19:22.271 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_open": 1, 2024-09-12T03:19:22.271 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_stale": 0, 2024-09-12T03:19:22.271 INFO:teuthology.orchestra.run.smithi177.stdout: "total_load": 74, 2024-09-12T03:19:22.271 INFO:teuthology.orchestra.run.smithi177.stdout: "average_load": 74, 2024-09-12T03:19:22.271 INFO:teuthology.orchestra.run.smithi177.stdout: "avg_session_uptime": 983, 2024-09-12T03:19:22.271 INFO:teuthology.orchestra.run.smithi177.stdout: "mdthresh_evicted": 0 2024-09-12T03:19:22.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.271 INFO:teuthology.orchestra.run.smithi177.stdout: "mempool": { 2024-09-12T03:19:22.271 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:19:22.271 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_items": 0, 2024-09-12T03:19:22.271 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:19:22.271 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:19:22.272 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:19:22.272 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:19:22.272 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:19:22.272 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:19:22.272 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:19:22.272 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:19:22.272 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:19:22.272 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:19:22.272 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:19:22.272 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:19:22.272 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:19:22.272 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_items": 0, 2024-09-12T03:19:22.273 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:19:22.273 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_items": 0, 2024-09-12T03:19:22.273 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:19:22.273 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:19:22.273 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:19:22.273 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:19:22.273 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:19:22.273 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:19:22.273 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:19:22.273 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_items": 0, 2024-09-12T03:19:22.273 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:19:22.273 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:19:22.273 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:19:22.273 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_items": 0, 2024-09-12T03:19:22.274 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_bytes": 0, 2024-09-12T03:19:22.274 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_items": 0, 2024-09-12T03:19:22.274 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:19:22.274 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:19:22.274 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:19:22.274 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:19:22.274 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_bytes": 294428, 2024-09-12T03:19:22.274 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_items": 98, 2024-09-12T03:19:22.274 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:19:22.274 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_items": 0, 2024-09-12T03:19:22.274 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_bytes": 0, 2024-09-12T03:19:22.274 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_items": 0, 2024-09-12T03:19:22.275 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:19:22.275 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_items": 0, 2024-09-12T03:19:22.275 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:19:22.275 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_items": 0, 2024-09-12T03:19:22.275 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_bytes": 7400, 2024-09-12T03:19:22.275 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_items": 152, 2024-09-12T03:19:22.275 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:19:22.275 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:19:22.275 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_bytes": 0, 2024-09-12T03:19:22.275 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_items": 0, 2024-09-12T03:19:22.275 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_bytes": 99018, 2024-09-12T03:19:22.275 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_items": 712, 2024-09-12T03:19:22.275 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_bytes": 0, 2024-09-12T03:19:22.276 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_items": 0, 2024-09-12T03:19:22.276 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_bytes": 0, 2024-09-12T03:19:22.276 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_items": 0 2024-09-12T03:19:22.276 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.276 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter": { 2024-09-12T03:19:22.276 INFO:teuthology.orchestra.run.smithi177.stdout: "op_active": 0, 2024-09-12T03:19:22.276 INFO:teuthology.orchestra.run.smithi177.stdout: "op_laggy": 0, 2024-09-12T03:19:22.276 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send": 4056, 2024-09-12T03:19:22.276 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send_bytes": 7599456, 2024-09-12T03:19:22.276 INFO:teuthology.orchestra.run.smithi177.stdout: "op_resend": 31, 2024-09-12T03:19:22.276 INFO:teuthology.orchestra.run.smithi177.stdout: "op_reply": 4056, 2024-09-12T03:19:22.276 INFO:teuthology.orchestra.run.smithi177.stdout: "op_latency": { 2024-09-12T03:19:22.276 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 4056, 2024-09-12T03:19:22.276 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 22.344370891, 2024-09-12T03:19:22.277 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.005508967 2024-09-12T03:19:22.277 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.277 INFO:teuthology.orchestra.run.smithi177.stdout: "op_inflight": 0, 2024-09-12T03:19:22.277 INFO:teuthology.orchestra.run.smithi177.stdout: "oplen_avg": { 2024-09-12T03:19:22.277 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 4056, 2024-09-12T03:19:22.277 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 4102 2024-09-12T03:19:22.277 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.277 INFO:teuthology.orchestra.run.smithi177.stdout: "op": 4056, 2024-09-12T03:19:22.277 INFO:teuthology.orchestra.run.smithi177.stdout: "op_r": 556, 2024-09-12T03:19:22.277 INFO:teuthology.orchestra.run.smithi177.stdout: "op_w": 3500, 2024-09-12T03:19:22.277 INFO:teuthology.orchestra.run.smithi177.stdout: "op_rmw": 0, 2024-09-12T03:19:22.277 INFO:teuthology.orchestra.run.smithi177.stdout: "op_pg": 0, 2024-09-12T03:19:22.277 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_stat": 0, 2024-09-12T03:19:22.278 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_create": 11, 2024-09-12T03:19:22.278 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_read": 36, 2024-09-12T03:19:22.278 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_write": 2269, 2024-09-12T03:19:22.278 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writefull": 71, 2024-09-12T03:19:22.278 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writesame": 0, 2024-09-12T03:19:22.278 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_append": 0, 2024-09-12T03:19:22.278 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_zero": 0, 2024-09-12T03:19:22.278 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_truncate": 0, 2024-09-12T03:19:22.278 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_delete": 515, 2024-09-12T03:19:22.278 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_mapext": 0, 2024-09-12T03:19:22.278 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_sparse_read": 0, 2024-09-12T03:19:22.278 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_clonerange": 0, 2024-09-12T03:19:22.278 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_getxattr": 520, 2024-09-12T03:19:22.278 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_setxattr": 22, 2024-09-12T03:19:22.279 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:19:22.279 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_rmxattr": 0, 2024-09-12T03:19:22.279 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:19:22.279 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_call": 0, 2024-09-12T03:19:22.279 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_watch": 0, 2024-09-12T03:19:22.279 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_notify": 0, 2024-09-12T03:19:22.279 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:19:22.279 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls": 0, 2024-09-12T03:19:22.279 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:19:22.279 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_other": 599, 2024-09-12T03:19:22.279 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_active": 0, 2024-09-12T03:19:22.279 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_send": 0, 2024-09-12T03:19:22.279 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_resend": 0, 2024-09-12T03:19:22.280 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_ping": 0, 2024-09-12T03:19:22.280 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_active": 0, 2024-09-12T03:19:22.280 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_send": 0, 2024-09-12T03:19:22.280 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_resend": 0, 2024-09-12T03:19:22.280 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_active": 0, 2024-09-12T03:19:22.280 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_send": 0, 2024-09-12T03:19:22.280 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_resend": 0, 2024-09-12T03:19:22.280 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_active": 0, 2024-09-12T03:19:22.280 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_send": 0, 2024-09-12T03:19:22.280 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_resend": 0, 2024-09-12T03:19:22.280 INFO:teuthology.orchestra.run.smithi177.stdout: "command_active": 0, 2024-09-12T03:19:22.280 INFO:teuthology.orchestra.run.smithi177.stdout: "command_send": 0, 2024-09-12T03:19:22.280 INFO:teuthology.orchestra.run.smithi177.stdout: "command_resend": 0, 2024-09-12T03:19:22.280 INFO:teuthology.orchestra.run.smithi177.stdout: "map_epoch": 80, 2024-09-12T03:19:22.281 INFO:teuthology.orchestra.run.smithi177.stdout: "map_full": 0, 2024-09-12T03:19:22.281 INFO:teuthology.orchestra.run.smithi177.stdout: "map_inc": 2, 2024-09-12T03:19:22.281 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_sessions": 12, 2024-09-12T03:19:22.281 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_open": 12, 2024-09-12T03:19:22.281 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_close": 0, 2024-09-12T03:19:22.281 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_laggy": 0, 2024-09-12T03:19:22.281 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_wr": 50, 2024-09-12T03:19:22.281 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_rd": 0, 2024-09-12T03:19:22.281 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_del": 9 2024-09-12T03:19:22.281 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.281 INFO:teuthology.orchestra.run.smithi177.stdout: "oft": { 2024-09-12T03:19:22.281 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_objs": 1, 2024-09-12T03:19:22.281 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:19:22.282 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_updates": 0, 2024-09-12T03:19:22.282 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_removes": 0 2024-09-12T03:19:22.282 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.282 INFO:teuthology.orchestra.run.smithi177.stdout: "osd-slow-ops": { 2024-09-12T03:19:22.282 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_ops_count": 208 2024-09-12T03:19:22.282 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.282 INFO:teuthology.orchestra.run.smithi177.stdout: "purge_queue": { 2024-09-12T03:19:22.282 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops": 0, 2024-09-12T03:19:22.282 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops_high_water": 2, 2024-09-12T03:19:22.282 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing": 0, 2024-09-12T03:19:22.282 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_high_water": 2, 2024-09-12T03:19:22.282 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executed_ops": 2, 2024-09-12T03:19:22.282 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executed": 2, 2024-09-12T03:19:22.282 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_item_in_journal": 0 2024-09-12T03:19:22.283 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.283 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:19:22.283 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-09-12T03:19:22.283 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2024-09-12T03:19:22.283 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-09-12T03:19:22.283 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 9020, 2024-09-12T03:19:22.283 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 2288272, 2024-09-12T03:19:22.283 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:22.283 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 9020, 2024-09-12T03:19:22.283 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-09-12T03:19:22.283 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-09-12T03:19:22.283 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 9020, 2024-09-12T03:19:22.283 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 2288272, 2024-09-12T03:19:22.283 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-09-12T03:19:22.284 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:22.284 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:22.284 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:22.284 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-12T03:19:22.284 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.284 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_bytes": { 2024-09-12T03:19:22.284 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-09-12T03:19:22.284 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2024-09-12T03:19:22.284 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-09-12T03:19:22.284 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-09-12T03:19:22.284 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-09-12T03:19:22.284 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:22.285 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-09-12T03:19:22.285 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 4056, 2024-09-12T03:19:22.285 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 7599658, 2024-09-12T03:19:22.285 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 4056, 2024-09-12T03:19:22.285 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 7599658, 2024-09-12T03:19:22.285 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-09-12T03:19:22.285 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:22.285 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:22.285 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:22.285 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-12T03:19:22.285 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.285 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_ops": { 2024-09-12T03:19:22.285 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-09-12T03:19:22.286 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 1024, 2024-09-12T03:19:22.286 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-09-12T03:19:22.286 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-09-12T03:19:22.286 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-09-12T03:19:22.286 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:22.286 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-09-12T03:19:22.286 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 4056, 2024-09-12T03:19:22.286 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 4056, 2024-09-12T03:19:22.286 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 4056, 2024-09-12T03:19:22.286 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 4056, 2024-09-12T03:19:22.286 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-09-12T03:19:22.286 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:22.286 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:22.286 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:22.287 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-12T03:19:22.287 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.287 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:19:22.287 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-09-12T03:19:22.287 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2024-09-12T03:19:22.287 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-09-12T03:19:22.287 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 2, 2024-09-12T03:19:22.287 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 202, 2024-09-12T03:19:22.287 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:22.287 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 2, 2024-09-12T03:19:22.287 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-09-12T03:19:22.287 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-09-12T03:19:22.287 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 1, 2024-09-12T03:19:22.287 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 202, 2024-09-12T03:19:22.288 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-09-12T03:19:22.288 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:22.288 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:22.288 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:22.288 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-12T03:19:22.288 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:22.288 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle-0x55e857142960": { 2024-09-12T03:19:22.288 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-09-12T03:19:22.288 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2024-09-12T03:19:22.288 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-09-12T03:19:22.288 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 2283, 2024-09-12T03:19:22.288 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 7575613, 2024-09-12T03:19:22.288 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:22.289 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 2283, 2024-09-12T03:19:22.289 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-09-12T03:19:22.289 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-09-12T03:19:22.289 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 2268, 2024-09-12T03:19:22.289 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 7575613, 2024-09-12T03:19:22.289 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-09-12T03:19:22.289 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:22.289 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:22.289 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:22.289 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-12T03:19:22.289 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-12T03:19:22.289 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-09-12T03:19:22.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:22 smithi183 ceph-mon[35535]: pgmap v829: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-12T03:19:22.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:22 smithi177 ceph-mon[35305]: pgmap v829: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-12T03:19:23.035 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:22 smithi116 ceph-mon[31410]: pgmap v829: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-12T03:19:23.239 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-12T03:19:23.239 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-12T03:19:23.239 DEBUG:tasks.check_counter:Getting stats from e 2024-09-12T03:19:23.239 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-12T03:19:23.481 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:19:23.993 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-09-12T03:19:23.993 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:19:23.993 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 1100, 2024-09-12T03:19:23.993 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 1102, 2024-09-12T03:19:23.993 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 289913, 2024-09-12T03:19:23.993 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 349864, 2024-09-12T03:19:23.993 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 7, 2024-09-12T03:19:23.993 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 2, 2024-09-12T03:19:23.993 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.105801815, 2024-09-12T03:19:23.994 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.044844156, 2024-09-12T03:19:23.994 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.024034621, 2024-09-12T03:19:23.994 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.025589570, 2024-09-12T03:19:23.994 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:23.994 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 1100, 2024-09-12T03:19:23.994 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.046150220, 2024-09-12T03:19:23.994 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000041954 2024-09-12T03:19:23.994 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:23.994 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:23.994 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:23.994 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:23.994 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:23.995 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:23.995 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:19:23.995 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:19:23.995 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:23.995 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:19:23.995 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 1303, 2024-09-12T03:19:23.995 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 1296, 2024-09-12T03:19:23.995 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 1044626, 2024-09-12T03:19:23.995 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 501537, 2024-09-12T03:19:23.995 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 4, 2024-09-12T03:19:23.995 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 3, 2024-09-12T03:19:23.995 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.135356621, 2024-09-12T03:19:23.996 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.049906427, 2024-09-12T03:19:23.996 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.052175782, 2024-09-12T03:19:23.996 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.032399228, 2024-09-12T03:19:23.996 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:23.996 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 1295, 2024-09-12T03:19:23.996 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.054766072, 2024-09-12T03:19:23.996 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000042290 2024-09-12T03:19:23.996 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:23.996 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:23.996 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:23.996 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:23.996 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:23.997 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:23.997 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 802048, 2024-09-12T03:19:23.997 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 187984 2024-09-12T03:19:23.997 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:23.997 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:19:23.997 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 1100, 2024-09-12T03:19:23.997 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 1327, 2024-09-12T03:19:23.997 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 370928, 2024-09-12T03:19:23.997 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 918900, 2024-09-12T03:19:23.997 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 5, 2024-09-12T03:19:23.997 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 3, 2024-09-12T03:19:23.997 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.150368546, 2024-09-12T03:19:23.997 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.050147375, 2024-09-12T03:19:23.998 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.038102272, 2024-09-12T03:19:23.998 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.046080579, 2024-09-12T03:19:23.998 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:23.998 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 1326, 2024-09-12T03:19:23.998 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.089273428, 2024-09-12T03:19:23.998 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000067325 2024-09-12T03:19:23.998 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:23.998 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:23.998 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:23.998 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:23.998 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:23.998 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:23.999 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-12T03:19:23.999 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 571008 2024-09-12T03:19:23.999 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:23.999 INFO:teuthology.orchestra.run.smithi177.stdout: "cct": { 2024-09-12T03:19:23.999 INFO:teuthology.orchestra.run.smithi177.stdout: "total_workers": 1, 2024-09-12T03:19:23.999 INFO:teuthology.orchestra.run.smithi177.stdout: "unhealthy_workers": 0 2024-09-12T03:19:23.999 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:23.999 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-MDSRank": { 2024-09-12T03:19:23.999 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2024-09-12T03:19:23.999 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2024-09-12T03:19:24.000 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 5251, 2024-09-12T03:19:24.000 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.156525553, 2024-09-12T03:19:24.000 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000029808 2024-09-12T03:19:24.000 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-12T03:19:24.000 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.000 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-PurgeQueue": { 2024-09-12T03:19:24.000 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2024-09-12T03:19:24.000 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2024-09-12T03:19:24.000 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.000 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.000 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.000 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-12T03:19:24.000 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.001 INFO:teuthology.orchestra.run.smithi177.stdout: "mds": { 2024-09-12T03:19:24.001 INFO:teuthology.orchestra.run.smithi177.stdout: "request": 0, 2024-09-12T03:19:24.001 INFO:teuthology.orchestra.run.smithi177.stdout: "reply": 0, 2024-09-12T03:19:24.001 INFO:teuthology.orchestra.run.smithi177.stdout: "reply_latency": { 2024-09-12T03:19:24.001 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.001 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.001 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.001 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.002 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_reply": 0, 2024-09-12T03:19:24.002 INFO:teuthology.orchestra.run.smithi177.stdout: "forward": 0, 2024-09-12T03:19:24.002 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_complete": 0, 2024-09-12T03:19:24.002 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_keys": 0, 2024-09-12T03:19:24.002 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_commit": 0, 2024-09-12T03:19:24.002 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_split": 0, 2024-09-12T03:19:24.002 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_merge": 0, 2024-09-12T03:19:24.002 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes": 0, 2024-09-12T03:19:24.002 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_top": 0, 2024-09-12T03:19:24.002 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_bottom": 0, 2024-09-12T03:19:24.002 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pin_tail": 0, 2024-09-12T03:19:24.002 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pinned": 0, 2024-09-12T03:19:24.002 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_expired": 0, 2024-09-12T03:19:24.003 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_with_caps": 0, 2024-09-12T03:19:24.003 INFO:teuthology.orchestra.run.smithi177.stdout: "caps": 0, 2024-09-12T03:19:24.003 INFO:teuthology.orchestra.run.smithi177.stdout: "subtrees": 1, 2024-09-12T03:19:24.003 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse": 0, 2024-09-12T03:19:24.003 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_hit": 0, 2024-09-12T03:19:24.003 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_forward": 0, 2024-09-12T03:19:24.003 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_discover": 0, 2024-09-12T03:19:24.003 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:19:24.003 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_remote_ino": 0, 2024-09-12T03:19:24.003 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_lock": 0, 2024-09-12T03:19:24.004 INFO:teuthology.orchestra.run.smithi177.stdout: "load_cent": 0, 2024-09-12T03:19:24.004 INFO:teuthology.orchestra.run.smithi177.stdout: "q": 0, 2024-09-12T03:19:24.004 INFO:teuthology.orchestra.run.smithi177.stdout: "exported": 0, 2024-09-12T03:19:24.004 INFO:teuthology.orchestra.run.smithi177.stdout: "exported_inodes": 0, 2024-09-12T03:19:24.004 INFO:teuthology.orchestra.run.smithi177.stdout: "imported": 0, 2024-09-12T03:19:24.004 INFO:teuthology.orchestra.run.smithi177.stdout: "imported_inodes": 0, 2024-09-12T03:19:24.004 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_dir_fetch": 0, 2024-09-12T03:19:24.004 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:19:24.004 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_peer_discover": 0, 2024-09-12T03:19:24.004 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rfiles": 0, 2024-09-12T03:19:24.004 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rbytes": 0, 2024-09-12T03:19:24.004 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rsnaps": 0, 2024-09-12T03:19:24.005 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:19:24.005 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_set_tag": 0, 2024-09-12T03:19:24.005 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:19:24.005 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:19:24.005 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:19:24.005 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:19:24.005 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:19:24.005 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_file_inodes": 0, 2024-09-12T03:19:24.005 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:19:24.005 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:19:24.005 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:19:24.005 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:19:24.005 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:19:24.006 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:19:24.006 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps": 0, 2024-09-12T03:19:24.006 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:19:24.006 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_cap_release": 0, 2024-09-12T03:19:24.006 INFO:teuthology.orchestra.run.smithi177.stdout: "process_request_cap_release": 0 2024-09-12T03:19:24.006 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.006 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_cache": { 2024-09-12T03:19:24.006 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update": 0, 2024-09-12T03:19:24.006 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update_receipt": 0, 2024-09-12T03:19:24.006 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_try_discover": 0, 2024-09-12T03:19:24.007 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_send_discover": 0, 2024-09-12T03:19:24.007 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_handle_discover": 0, 2024-09-12T03:19:24.007 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays": 0, 2024-09-12T03:19:24.007 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_delayed": 0, 2024-09-12T03:19:24.007 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:19:24.007 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_created": 0, 2024-09-12T03:19:24.007 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_enqueued": 0, 2024-09-12T03:19:24.007 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_reintegrated": 0, 2024-09-12T03:19:24.007 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_migrated": 0, 2024-09-12T03:19:24.008 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_processing": 0, 2024-09-12T03:19:24.008 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:19:24.008 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:19:24.008 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_started": 0, 2024-09-12T03:19:24.008 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_completed": 0, 2024-09-12T03:19:24.008 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:19:24.008 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:19:24.008 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:19:24.008 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_exportdir": 0, 2024-09-12T03:19:24.009 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_flush": 0, 2024-09-12T03:19:24.009 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:19:24.009 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragstats": 0, 2024-09-12T03:19:24.009 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_inodestats": 0 2024-09-12T03:19:24.009 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.009 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_log": { 2024-09-12T03:19:24.009 INFO:teuthology.orchestra.run.smithi177.stdout: "evlrg": 0, 2024-09-12T03:19:24.009 INFO:teuthology.orchestra.run.smithi177.stdout: "evadd": 0, 2024-09-12T03:19:24.009 INFO:teuthology.orchestra.run.smithi177.stdout: "evex": 0, 2024-09-12T03:19:24.009 INFO:teuthology.orchestra.run.smithi177.stdout: "evtrm": 0, 2024-09-12T03:19:24.009 INFO:teuthology.orchestra.run.smithi177.stdout: "ev": 4, 2024-09-12T03:19:24.009 INFO:teuthology.orchestra.run.smithi177.stdout: "evexg": 0, 2024-09-12T03:19:24.009 INFO:teuthology.orchestra.run.smithi177.stdout: "evexd": 0, 2024-09-12T03:19:24.009 INFO:teuthology.orchestra.run.smithi177.stdout: "segadd": 0, 2024-09-12T03:19:24.010 INFO:teuthology.orchestra.run.smithi177.stdout: "segex": 0, 2024-09-12T03:19:24.010 INFO:teuthology.orchestra.run.smithi177.stdout: "segtrm": 0, 2024-09-12T03:19:24.010 INFO:teuthology.orchestra.run.smithi177.stdout: "seg": 2, 2024-09-12T03:19:24.010 INFO:teuthology.orchestra.run.smithi177.stdout: "segmjr": 2, 2024-09-12T03:19:24.010 INFO:teuthology.orchestra.run.smithi177.stdout: "segexg": 0, 2024-09-12T03:19:24.010 INFO:teuthology.orchestra.run.smithi177.stdout: "segexd": 0, 2024-09-12T03:19:24.010 INFO:teuthology.orchestra.run.smithi177.stdout: "expos": 4194304, 2024-09-12T03:19:24.010 INFO:teuthology.orchestra.run.smithi177.stdout: "wrpos": 4195426, 2024-09-12T03:19:24.010 INFO:teuthology.orchestra.run.smithi177.stdout: "rdpos": 4195285, 2024-09-12T03:19:24.010 INFO:teuthology.orchestra.run.smithi177.stdout: "jlat": { 2024-09-12T03:19:24.010 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.010 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.010 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.011 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.011 INFO:teuthology.orchestra.run.smithi177.stdout: "replayed": 4 2024-09-12T03:19:24.011 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.011 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_mem": { 2024-09-12T03:19:24.011 INFO:teuthology.orchestra.run.smithi177.stdout: "ino": 3, 2024-09-12T03:19:24.011 INFO:teuthology.orchestra.run.smithi177.stdout: "ino+": 3, 2024-09-12T03:19:24.011 INFO:teuthology.orchestra.run.smithi177.stdout: "ino-": 0, 2024-09-12T03:19:24.011 INFO:teuthology.orchestra.run.smithi177.stdout: "dir": 1, 2024-09-12T03:19:24.011 INFO:teuthology.orchestra.run.smithi177.stdout: "dir+": 1, 2024-09-12T03:19:24.011 INFO:teuthology.orchestra.run.smithi177.stdout: "dir-": 0, 2024-09-12T03:19:24.012 INFO:teuthology.orchestra.run.smithi177.stdout: "dn": 0, 2024-09-12T03:19:24.012 INFO:teuthology.orchestra.run.smithi177.stdout: "dn+": 0, 2024-09-12T03:19:24.012 INFO:teuthology.orchestra.run.smithi177.stdout: "dn-": 0, 2024-09-12T03:19:24.012 INFO:teuthology.orchestra.run.smithi177.stdout: "cap": 0, 2024-09-12T03:19:24.012 INFO:teuthology.orchestra.run.smithi177.stdout: "cap+": 0, 2024-09-12T03:19:24.012 INFO:teuthology.orchestra.run.smithi177.stdout: "cap-": 0, 2024-09-12T03:19:24.012 INFO:teuthology.orchestra.run.smithi177.stdout: "rss": 47692, 2024-09-12T03:19:24.012 INFO:teuthology.orchestra.run.smithi177.stdout: "heap": 215328 2024-09-12T03:19:24.012 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.012 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_server": { 2024-09-12T03:19:24.012 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_client_request": 0, 2024-09-12T03:19:24.012 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_server_request": 0, 2024-09-12T03:19:24.012 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_request": 0, 2024-09-12T03:19:24.013 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_session": 0, 2024-09-12T03:19:24.013 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_peer_request": 0, 2024-09-12T03:19:24.013 INFO:teuthology.orchestra.run.smithi177.stdout: "req_create_latency": { 2024-09-12T03:19:24.013 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.013 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.013 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.013 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.013 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getattr_latency": { 2024-09-12T03:19:24.013 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.013 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.013 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.013 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.013 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getfilelock_latency": { 2024-09-12T03:19:24.014 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.014 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.014 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.014 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.014 INFO:teuthology.orchestra.run.smithi177.stdout: "req_link_latency": { 2024-09-12T03:19:24.014 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.014 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.014 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.014 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.015 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookup_latency": { 2024-09-12T03:19:24.015 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.015 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.015 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.015 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookuphash_latency": { 2024-09-12T03:19:24.015 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.015 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.015 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.015 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupino_latency": { 2024-09-12T03:19:24.015 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.015 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.015 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.016 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.016 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupname_latency": { 2024-09-12T03:19:24.016 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.016 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.016 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.016 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.016 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupparent_latency": { 2024-09-12T03:19:24.016 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.016 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.016 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.016 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.017 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupsnap_latency": { 2024-09-12T03:19:24.017 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.017 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.017 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.017 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lssnap_latency": { 2024-09-12T03:19:24.017 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.017 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.017 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.017 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mkdir_latency": { 2024-09-12T03:19:24.017 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.018 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.018 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.019 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mknod_latency": { 2024-09-12T03:19:24.019 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.019 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.019 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.019 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mksnap_latency": { 2024-09-12T03:19:24.019 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.019 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.019 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.019 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.020 INFO:teuthology.orchestra.run.smithi177.stdout: "req_open_latency": { 2024-09-12T03:19:24.020 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.020 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.020 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.020 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.020 INFO:teuthology.orchestra.run.smithi177.stdout: "req_readdir_latency": { 2024-09-12T03:19:24.020 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.020 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.020 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.020 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.020 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rename_latency": { 2024-09-12T03:19:24.021 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.021 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.021 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.021 INFO:teuthology.orchestra.run.smithi177.stdout: "req_renamesnap_latency": { 2024-09-12T03:19:24.021 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.021 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.021 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.021 INFO:teuthology.orchestra.run.smithi177.stdout: "req_snapdiff_latency": { 2024-09-12T03:19:24.021 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.021 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.021 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.022 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmdir_latency": { 2024-09-12T03:19:24.022 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.022 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.022 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.022 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmsnap_latency": { 2024-09-12T03:19:24.022 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.022 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.022 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.022 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmxattr_latency": { 2024-09-12T03:19:24.022 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.023 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.023 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.023 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.023 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setattr_latency": { 2024-09-12T03:19:24.023 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.023 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.023 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.023 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.023 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setdirlayout_latency": { 2024-09-12T03:19:24.023 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.023 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.023 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.024 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.024 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setfilelock_latency": { 2024-09-12T03:19:24.024 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.024 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.024 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.024 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.024 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setlayout_latency": { 2024-09-12T03:19:24.024 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.024 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.024 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.024 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.024 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setxattr_latency": { 2024-09-12T03:19:24.025 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.025 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.025 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.025 INFO:teuthology.orchestra.run.smithi177.stdout: "req_symlink_latency": { 2024-09-12T03:19:24.025 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.025 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.025 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.025 INFO:teuthology.orchestra.run.smithi177.stdout: "req_unlink_latency": { 2024-09-12T03:19:24.025 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.025 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.025 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.025 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.026 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:19:24.026 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:19:24.026 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getvxattr_latency": { 2024-09-12T03:19:24.026 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.026 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.026 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.026 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-12T03:19:24.026 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.026 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_sessions": { 2024-09-12T03:19:24.026 INFO:teuthology.orchestra.run.smithi177.stdout: "session_count": 0, 2024-09-12T03:19:24.026 INFO:teuthology.orchestra.run.smithi177.stdout: "session_add": 1, 2024-09-12T03:19:24.026 INFO:teuthology.orchestra.run.smithi177.stdout: "session_remove": 1, 2024-09-12T03:19:24.026 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_open": 1, 2024-09-12T03:19:24.027 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_stale": 0, 2024-09-12T03:19:24.027 INFO:teuthology.orchestra.run.smithi177.stdout: "total_load": 0, 2024-09-12T03:19:24.027 INFO:teuthology.orchestra.run.smithi177.stdout: "average_load": 0, 2024-09-12T03:19:24.027 INFO:teuthology.orchestra.run.smithi177.stdout: "avg_session_uptime": 1381, 2024-09-12T03:19:24.027 INFO:teuthology.orchestra.run.smithi177.stdout: "mdthresh_evicted": 0 2024-09-12T03:19:24.027 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.027 INFO:teuthology.orchestra.run.smithi177.stdout: "mempool": { 2024-09-12T03:19:24.027 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:19:24.027 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_items": 0, 2024-09-12T03:19:24.027 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:19:24.027 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:19:24.027 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:19:24.027 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:19:24.027 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:19:24.028 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:19:24.028 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:19:24.028 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:19:24.028 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:19:24.028 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:19:24.028 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:19:24.028 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:19:24.028 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:19:24.028 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_items": 0, 2024-09-12T03:19:24.028 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:19:24.028 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_items": 0, 2024-09-12T03:19:24.028 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:19:24.029 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:19:24.029 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:19:24.029 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:19:24.029 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:19:24.029 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:19:24.029 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:19:24.029 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_items": 0, 2024-09-12T03:19:24.029 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:19:24.029 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:19:24.029 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:19:24.030 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_items": 0, 2024-09-12T03:19:24.030 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_bytes": 0, 2024-09-12T03:19:24.030 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_items": 0, 2024-09-12T03:19:24.030 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:19:24.030 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:19:24.030 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:19:24.030 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:19:24.030 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_bytes": 190167, 2024-09-12T03:19:24.030 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_items": 52, 2024-09-12T03:19:24.030 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:19:24.030 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_items": 0, 2024-09-12T03:19:24.030 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_bytes": 0, 2024-09-12T03:19:24.030 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_items": 0, 2024-09-12T03:19:24.030 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:19:24.031 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_items": 0, 2024-09-12T03:19:24.031 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:19:24.031 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_items": 0, 2024-09-12T03:19:24.031 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_bytes": 7400, 2024-09-12T03:19:24.031 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_items": 152, 2024-09-12T03:19:24.031 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:19:24.031 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:19:24.031 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_bytes": 0, 2024-09-12T03:19:24.031 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_items": 0, 2024-09-12T03:19:24.031 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_bytes": 8512, 2024-09-12T03:19:24.031 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_items": 13, 2024-09-12T03:19:24.031 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_bytes": 0, 2024-09-12T03:19:24.032 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_items": 0, 2024-09-12T03:19:24.032 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_bytes": 0, 2024-09-12T03:19:24.032 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_items": 0 2024-09-12T03:19:24.032 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.032 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter": { 2024-09-12T03:19:24.032 INFO:teuthology.orchestra.run.smithi177.stdout: "op_active": 0, 2024-09-12T03:19:24.032 INFO:teuthology.orchestra.run.smithi177.stdout: "op_laggy": 0, 2024-09-12T03:19:24.032 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send": 3180, 2024-09-12T03:19:24.032 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send_bytes": 21, 2024-09-12T03:19:24.032 INFO:teuthology.orchestra.run.smithi177.stdout: "op_resend": 3, 2024-09-12T03:19:24.032 INFO:teuthology.orchestra.run.smithi177.stdout: "op_reply": 3180, 2024-09-12T03:19:24.032 INFO:teuthology.orchestra.run.smithi177.stdout: "op_latency": { 2024-09-12T03:19:24.033 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 3180, 2024-09-12T03:19:24.033 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 1.483957930, 2024-09-12T03:19:24.033 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000466653 2024-09-12T03:19:24.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.033 INFO:teuthology.orchestra.run.smithi177.stdout: "op_inflight": 0, 2024-09-12T03:19:24.033 INFO:teuthology.orchestra.run.smithi177.stdout: "oplen_avg": { 2024-09-12T03:19:24.033 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 3180, 2024-09-12T03:19:24.033 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 3181 2024-09-12T03:19:24.033 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.033 INFO:teuthology.orchestra.run.smithi177.stdout: "op": 3180, 2024-09-12T03:19:24.033 INFO:teuthology.orchestra.run.smithi177.stdout: "op_r": 3180, 2024-09-12T03:19:24.033 INFO:teuthology.orchestra.run.smithi177.stdout: "op_w": 0, 2024-09-12T03:19:24.034 INFO:teuthology.orchestra.run.smithi177.stdout: "op_rmw": 0, 2024-09-12T03:19:24.034 INFO:teuthology.orchestra.run.smithi177.stdout: "op_pg": 0, 2024-09-12T03:19:24.034 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_stat": 2078, 2024-09-12T03:19:24.034 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_create": 0, 2024-09-12T03:19:24.034 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_read": 1100, 2024-09-12T03:19:24.034 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_write": 0, 2024-09-12T03:19:24.034 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writefull": 0, 2024-09-12T03:19:24.034 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writesame": 0, 2024-09-12T03:19:24.034 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_append": 0, 2024-09-12T03:19:24.034 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_zero": 0, 2024-09-12T03:19:24.035 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_truncate": 0, 2024-09-12T03:19:24.035 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_delete": 0, 2024-09-12T03:19:24.035 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_mapext": 0, 2024-09-12T03:19:24.035 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_sparse_read": 0, 2024-09-12T03:19:24.035 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_clonerange": 0, 2024-09-12T03:19:24.035 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_getxattr": 1, 2024-09-12T03:19:24.035 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_setxattr": 0, 2024-09-12T03:19:24.035 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:19:24.035 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_rmxattr": 0, 2024-09-12T03:19:24.035 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:19:24.035 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_call": 0, 2024-09-12T03:19:24.035 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_watch": 0, 2024-09-12T03:19:24.035 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_notify": 0, 2024-09-12T03:19:24.036 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:19:24.036 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls": 0, 2024-09-12T03:19:24.036 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:19:24.036 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_other": 0, 2024-09-12T03:19:24.036 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_active": 0, 2024-09-12T03:19:24.036 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_send": 0, 2024-09-12T03:19:24.036 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_resend": 0, 2024-09-12T03:19:24.036 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_ping": 0, 2024-09-12T03:19:24.036 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_active": 0, 2024-09-12T03:19:24.036 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_send": 0, 2024-09-12T03:19:24.037 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_resend": 0, 2024-09-12T03:19:24.037 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_active": 0, 2024-09-12T03:19:24.037 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_send": 0, 2024-09-12T03:19:24.037 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_resend": 0, 2024-09-12T03:19:24.037 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_active": 0, 2024-09-12T03:19:24.037 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_send": 0, 2024-09-12T03:19:24.037 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_resend": 0, 2024-09-12T03:19:24.037 INFO:teuthology.orchestra.run.smithi177.stdout: "command_active": 0, 2024-09-12T03:19:24.037 INFO:teuthology.orchestra.run.smithi177.stdout: "command_send": 0, 2024-09-12T03:19:24.037 INFO:teuthology.orchestra.run.smithi177.stdout: "command_resend": 0, 2024-09-12T03:19:24.037 INFO:teuthology.orchestra.run.smithi177.stdout: "map_epoch": 80, 2024-09-12T03:19:24.037 INFO:teuthology.orchestra.run.smithi177.stdout: "map_full": 0, 2024-09-12T03:19:24.037 INFO:teuthology.orchestra.run.smithi177.stdout: "map_inc": 2, 2024-09-12T03:19:24.038 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_sessions": 5, 2024-09-12T03:19:24.038 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_open": 6, 2024-09-12T03:19:24.038 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_close": 1, 2024-09-12T03:19:24.038 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_laggy": 0, 2024-09-12T03:19:24.038 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_wr": 0, 2024-09-12T03:19:24.038 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_rd": 2, 2024-09-12T03:19:24.038 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_del": 0 2024-09-12T03:19:24.038 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.038 INFO:teuthology.orchestra.run.smithi177.stdout: "oft": { 2024-09-12T03:19:24.038 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_objs": 0, 2024-09-12T03:19:24.038 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:19:24.038 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_updates": 0, 2024-09-12T03:19:24.039 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_removes": 0 2024-09-12T03:19:24.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.039 INFO:teuthology.orchestra.run.smithi177.stdout: "osd-slow-ops": { 2024-09-12T03:19:24.039 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_ops_count": 0 2024-09-12T03:19:24.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.039 INFO:teuthology.orchestra.run.smithi177.stdout: "purge_queue": { 2024-09-12T03:19:24.039 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops": 0, 2024-09-12T03:19:24.039 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:19:24.039 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing": 0, 2024-09-12T03:19:24.039 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_high_water": 0, 2024-09-12T03:19:24.039 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executed_ops": 0, 2024-09-12T03:19:24.039 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executed": 0, 2024-09-12T03:19:24.039 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_item_in_journal": 0 2024-09-12T03:19:24.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.040 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:19:24.040 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-09-12T03:19:24.040 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2024-09-12T03:19:24.040 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-09-12T03:19:24.040 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 3499, 2024-09-12T03:19:24.040 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 1371199, 2024-09-12T03:19:24.040 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:24.040 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 3499, 2024-09-12T03:19:24.040 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-09-12T03:19:24.040 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-09-12T03:19:24.040 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 3499, 2024-09-12T03:19:24.040 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 1371199, 2024-09-12T03:19:24.040 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-09-12T03:19:24.041 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.041 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.041 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.041 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-12T03:19:24.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.041 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_bytes": { 2024-09-12T03:19:24.041 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-09-12T03:19:24.041 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2024-09-12T03:19:24.041 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-09-12T03:19:24.041 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-09-12T03:19:24.041 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-09-12T03:19:24.041 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:24.041 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-09-12T03:19:24.041 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 3180, 2024-09-12T03:19:24.042 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 1127, 2024-09-12T03:19:24.042 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 3180, 2024-09-12T03:19:24.042 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 1127, 2024-09-12T03:19:24.042 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-09-12T03:19:24.042 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.042 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.042 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.042 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-12T03:19:24.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.042 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_ops": { 2024-09-12T03:19:24.042 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-09-12T03:19:24.042 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 1024, 2024-09-12T03:19:24.042 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-09-12T03:19:24.042 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-09-12T03:19:24.043 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-09-12T03:19:24.043 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:24.043 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-09-12T03:19:24.043 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 3180, 2024-09-12T03:19:24.043 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 3180, 2024-09-12T03:19:24.043 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 3180, 2024-09-12T03:19:24.043 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 3180, 2024-09-12T03:19:24.043 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-09-12T03:19:24.043 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.043 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.043 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.043 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-12T03:19:24.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.043 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:19:24.044 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-09-12T03:19:24.044 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2024-09-12T03:19:24.044 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-09-12T03:19:24.044 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-09-12T03:19:24.044 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-09-12T03:19:24.044 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:24.044 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-09-12T03:19:24.044 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-09-12T03:19:24.044 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-09-12T03:19:24.044 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2024-09-12T03:19:24.044 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2024-09-12T03:19:24.044 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-09-12T03:19:24.044 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.045 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.045 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.045 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-12T03:19:24.045 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:24.045 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle-0x55d33a490b40": { 2024-09-12T03:19:24.045 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-09-12T03:19:24.045 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2024-09-12T03:19:24.045 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-09-12T03:19:24.045 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-09-12T03:19:24.045 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-09-12T03:19:24.045 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:24.045 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-09-12T03:19:24.046 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-09-12T03:19:24.046 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-09-12T03:19:24.046 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2024-09-12T03:19:24.046 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2024-09-12T03:19:24.046 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-09-12T03:19:24.046 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:24.046 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:24.046 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:24.046 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-12T03:19:24.046 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-12T03:19:24.046 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-09-12T03:19:24.893 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-12T03:19:24.893 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-12T03:19:24.893 DEBUG:tasks.check_counter:Getting stats from h 2024-09-12T03:19:24.893 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-12T03:19:24.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:24 smithi183 ceph-mon[35535]: pgmap v830: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-12T03:19:24.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:24 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:24.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:24 smithi177 ceph-mon[35305]: pgmap v830: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-12T03:19:24.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:24 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:25.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:24 smithi116 ceph-mon[31410]: pgmap v830: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-12T03:19:25.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:24 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:25.150 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:19:25.659 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-09-12T03:19:25.659 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:19:25.659 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 1303, 2024-09-12T03:19:25.659 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 1266, 2024-09-12T03:19:25.659 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 1024067, 2024-09-12T03:19:25.659 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 352429, 2024-09-12T03:19:25.659 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 8, 2024-09-12T03:19:25.660 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 2, 2024-09-12T03:19:25.660 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.156399081, 2024-09-12T03:19:25.660 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.072579834, 2024-09-12T03:19:25.660 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.081945836, 2024-09-12T03:19:25.660 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.005612421, 2024-09-12T03:19:25.660 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:25.660 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 1267, 2024-09-12T03:19:25.660 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.073600133, 2024-09-12T03:19:25.660 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000058090 2024-09-12T03:19:25.660 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.660 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:25.660 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 1832, 2024-09-12T03:19:25.660 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000389714, 2024-09-12T03:19:25.661 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000212 2024-09-12T03:19:25.661 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.661 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 811776, 2024-09-12T03:19:25.661 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 192144 2024-09-12T03:19:25.661 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.661 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:19:25.661 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 33, 2024-09-12T03:19:25.661 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 215, 2024-09-12T03:19:25.661 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 6846, 2024-09-12T03:19:25.661 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 609225, 2024-09-12T03:19:25.661 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 25, 2024-09-12T03:19:25.661 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 5, 2024-09-12T03:19:25.662 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.019955561, 2024-09-12T03:19:25.662 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.008603951, 2024-09-12T03:19:25.662 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.008350548, 2024-09-12T03:19:25.662 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.000436333, 2024-09-12T03:19:25.662 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:25.662 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 213, 2024-09-12T03:19:25.662 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.025967817, 2024-09-12T03:19:25.662 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000121914 2024-09-12T03:19:25.662 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.662 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:25.662 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 16, 2024-09-12T03:19:25.662 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000001291, 2024-09-12T03:19:25.662 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000080 2024-09-12T03:19:25.663 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.663 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T03:19:25.663 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 401216 2024-09-12T03:19:25.663 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.663 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:19:25.663 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 88, 2024-09-12T03:19:25.663 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 136, 2024-09-12T03:19:25.663 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 18509, 2024-09-12T03:19:25.663 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 913739, 2024-09-12T03:19:25.663 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 70, 2024-09-12T03:19:25.663 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 5, 2024-09-12T03:19:25.663 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.030552795, 2024-09-12T03:19:25.663 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.004380912, 2024-09-12T03:19:25.663 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.023634829, 2024-09-12T03:19:25.664 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.000434486, 2024-09-12T03:19:25.664 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:25.664 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 133, 2024-09-12T03:19:25.664 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.021027539, 2024-09-12T03:19:25.664 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000158101 2024-09-12T03:19:25.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.664 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:25.664 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 81, 2024-09-12T03:19:25.664 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000006823, 2024-09-12T03:19:25.664 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000084 2024-09-12T03:19:25.664 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.664 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 6560, 2024-09-12T03:19:25.664 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 175760 2024-09-12T03:19:25.665 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.665 INFO:teuthology.orchestra.run.smithi177.stdout: "cct": { 2024-09-12T03:19:25.665 INFO:teuthology.orchestra.run.smithi177.stdout: "total_workers": 1, 2024-09-12T03:19:25.665 INFO:teuthology.orchestra.run.smithi177.stdout: "unhealthy_workers": 0 2024-09-12T03:19:25.665 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.665 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-MDSRank": { 2024-09-12T03:19:25.665 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2024-09-12T03:19:25.665 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2024-09-12T03:19:25.665 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 37, 2024-09-12T03:19:25.665 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.001426364, 2024-09-12T03:19:25.666 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000038550 2024-09-12T03:19:25.666 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-12T03:19:25.666 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.666 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-PurgeQueue": { 2024-09-12T03:19:25.666 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2024-09-12T03:19:25.666 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2024-09-12T03:19:25.666 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2, 2024-09-12T03:19:25.666 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000014749, 2024-09-12T03:19:25.666 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000007374 2024-09-12T03:19:25.666 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-12T03:19:25.666 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.666 INFO:teuthology.orchestra.run.smithi177.stdout: "mds": { 2024-09-12T03:19:25.667 INFO:teuthology.orchestra.run.smithi177.stdout: "request": 4, 2024-09-12T03:19:25.667 INFO:teuthology.orchestra.run.smithi177.stdout: "reply": 3, 2024-09-12T03:19:25.667 INFO:teuthology.orchestra.run.smithi177.stdout: "reply_latency": { 2024-09-12T03:19:25.667 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 3, 2024-09-12T03:19:25.667 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.002704688, 2024-09-12T03:19:25.667 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000901562 2024-09-12T03:19:25.667 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.667 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_reply": 0, 2024-09-12T03:19:25.667 INFO:teuthology.orchestra.run.smithi177.stdout: "forward": 1, 2024-09-12T03:19:25.667 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_complete": 0, 2024-09-12T03:19:25.667 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_keys": 0, 2024-09-12T03:19:25.667 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_commit": 11, 2024-09-12T03:19:25.667 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_split": 0, 2024-09-12T03:19:25.668 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_merge": 0, 2024-09-12T03:19:25.668 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes": 12, 2024-09-12T03:19:25.668 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_top": 0, 2024-09-12T03:19:25.668 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_bottom": 12, 2024-09-12T03:19:25.668 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pin_tail": 0, 2024-09-12T03:19:25.668 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pinned": 11, 2024-09-12T03:19:25.668 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_expired": 0, 2024-09-12T03:19:25.668 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_with_caps": 0, 2024-09-12T03:19:25.668 INFO:teuthology.orchestra.run.smithi177.stdout: "caps": 0, 2024-09-12T03:19:25.668 INFO:teuthology.orchestra.run.smithi177.stdout: "subtrees": 2, 2024-09-12T03:19:25.668 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse": 36, 2024-09-12T03:19:25.668 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_hit": 34, 2024-09-12T03:19:25.668 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_forward": 0, 2024-09-12T03:19:25.668 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_discover": 2, 2024-09-12T03:19:25.669 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:19:25.669 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_remote_ino": 0, 2024-09-12T03:19:25.669 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_lock": 0, 2024-09-12T03:19:25.669 INFO:teuthology.orchestra.run.smithi177.stdout: "load_cent": 0, 2024-09-12T03:19:25.669 INFO:teuthology.orchestra.run.smithi177.stdout: "q": 0, 2024-09-12T03:19:25.669 INFO:teuthology.orchestra.run.smithi177.stdout: "exported": 0, 2024-09-12T03:19:25.669 INFO:teuthology.orchestra.run.smithi177.stdout: "exported_inodes": 0, 2024-09-12T03:19:25.669 INFO:teuthology.orchestra.run.smithi177.stdout: "imported": 0, 2024-09-12T03:19:25.669 INFO:teuthology.orchestra.run.smithi177.stdout: "imported_inodes": 0, 2024-09-12T03:19:25.669 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_dir_fetch": 0, 2024-09-12T03:19:25.669 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:19:25.669 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_peer_discover": 0, 2024-09-12T03:19:25.669 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rfiles": 5, 2024-09-12T03:19:25.670 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rbytes": 298, 2024-09-12T03:19:25.670 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rsnaps": 0, 2024-09-12T03:19:25.670 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:19:25.670 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_set_tag": 0, 2024-09-12T03:19:25.670 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:19:25.670 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:19:25.670 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:19:25.670 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:19:25.670 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:19:25.670 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_file_inodes": 0, 2024-09-12T03:19:25.670 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:19:25.670 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_revoke": 3, 2024-09-12T03:19:25.670 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:19:25.670 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:19:25.671 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:19:25.671 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:19:25.671 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps": 3, 2024-09-12T03:19:25.671 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:19:25.671 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_cap_release": 0, 2024-09-12T03:19:25.671 INFO:teuthology.orchestra.run.smithi177.stdout: "process_request_cap_release": 0 2024-09-12T03:19:25.671 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.671 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_cache": { 2024-09-12T03:19:25.671 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update": 0, 2024-09-12T03:19:25.671 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update_receipt": 0, 2024-09-12T03:19:25.671 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_try_discover": 0, 2024-09-12T03:19:25.671 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_send_discover": 3, 2024-09-12T03:19:25.671 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_handle_discover": 0, 2024-09-12T03:19:25.672 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays": 0, 2024-09-12T03:19:25.672 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_delayed": 0, 2024-09-12T03:19:25.672 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:19:25.672 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_created": 0, 2024-09-12T03:19:25.672 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_enqueued": 0, 2024-09-12T03:19:25.672 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_reintegrated": 0, 2024-09-12T03:19:25.672 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_migrated": 0, 2024-09-12T03:19:25.672 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_processing": 0, 2024-09-12T03:19:25.672 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:19:25.672 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:19:25.672 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_started": 0, 2024-09-12T03:19:25.673 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_completed": 0, 2024-09-12T03:19:25.673 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_quiesce_path": 14, 2024-09-12T03:19:25.673 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_quiesce_inode": 42, 2024-09-12T03:19:25.673 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:19:25.673 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_exportdir": 0, 2024-09-12T03:19:25.673 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_flush": 0, 2024-09-12T03:19:25.673 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:19:25.673 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragstats": 0, 2024-09-12T03:19:25.673 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_inodestats": 0 2024-09-12T03:19:25.673 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.673 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_log": { 2024-09-12T03:19:25.673 INFO:teuthology.orchestra.run.smithi177.stdout: "evlrg": 0, 2024-09-12T03:19:25.673 INFO:teuthology.orchestra.run.smithi177.stdout: "evadd": 4, 2024-09-12T03:19:25.674 INFO:teuthology.orchestra.run.smithi177.stdout: "evex": 0, 2024-09-12T03:19:25.674 INFO:teuthology.orchestra.run.smithi177.stdout: "evtrm": 0, 2024-09-12T03:19:25.674 INFO:teuthology.orchestra.run.smithi177.stdout: "ev": 4, 2024-09-12T03:19:25.674 INFO:teuthology.orchestra.run.smithi177.stdout: "evexg": 0, 2024-09-12T03:19:25.674 INFO:teuthology.orchestra.run.smithi177.stdout: "evexd": 0, 2024-09-12T03:19:25.674 INFO:teuthology.orchestra.run.smithi177.stdout: "segadd": 2, 2024-09-12T03:19:25.675 INFO:teuthology.orchestra.run.smithi177.stdout: "segex": 0, 2024-09-12T03:19:25.675 INFO:teuthology.orchestra.run.smithi177.stdout: "segtrm": 0, 2024-09-12T03:19:25.675 INFO:teuthology.orchestra.run.smithi177.stdout: "seg": 2, 2024-09-12T03:19:25.675 INFO:teuthology.orchestra.run.smithi177.stdout: "segmjr": 2, 2024-09-12T03:19:25.675 INFO:teuthology.orchestra.run.smithi177.stdout: "segexg": 0, 2024-09-12T03:19:25.675 INFO:teuthology.orchestra.run.smithi177.stdout: "segexd": 0, 2024-09-12T03:19:25.675 INFO:teuthology.orchestra.run.smithi177.stdout: "expos": 4194304, 2024-09-12T03:19:25.675 INFO:teuthology.orchestra.run.smithi177.stdout: "wrpos": 4195426, 2024-09-12T03:19:25.675 INFO:teuthology.orchestra.run.smithi177.stdout: "rdpos": 0, 2024-09-12T03:19:25.675 INFO:teuthology.orchestra.run.smithi177.stdout: "jlat": { 2024-09-12T03:19:25.675 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 3, 2024-09-12T03:19:25.675 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.005249341, 2024-09-12T03:19:25.675 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.001749780 2024-09-12T03:19:25.676 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.676 INFO:teuthology.orchestra.run.smithi177.stdout: "replayed": 0 2024-09-12T03:19:25.676 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.676 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_mem": { 2024-09-12T03:19:25.676 INFO:teuthology.orchestra.run.smithi177.stdout: "ino": 15, 2024-09-12T03:19:25.676 INFO:teuthology.orchestra.run.smithi177.stdout: "ino+": 15, 2024-09-12T03:19:25.676 INFO:teuthology.orchestra.run.smithi177.stdout: "ino-": 0, 2024-09-12T03:19:25.676 INFO:teuthology.orchestra.run.smithi177.stdout: "dir": 13, 2024-09-12T03:19:25.676 INFO:teuthology.orchestra.run.smithi177.stdout: "dir+": 14, 2024-09-12T03:19:25.676 INFO:teuthology.orchestra.run.smithi177.stdout: "dir-": 1, 2024-09-12T03:19:25.676 INFO:teuthology.orchestra.run.smithi177.stdout: "dn": 12, 2024-09-12T03:19:25.676 INFO:teuthology.orchestra.run.smithi177.stdout: "dn+": 12, 2024-09-12T03:19:25.676 INFO:teuthology.orchestra.run.smithi177.stdout: "dn-": 0, 2024-09-12T03:19:25.676 INFO:teuthology.orchestra.run.smithi177.stdout: "cap": 0, 2024-09-12T03:19:25.677 INFO:teuthology.orchestra.run.smithi177.stdout: "cap+": 3, 2024-09-12T03:19:25.677 INFO:teuthology.orchestra.run.smithi177.stdout: "cap-": 3, 2024-09-12T03:19:25.677 INFO:teuthology.orchestra.run.smithi177.stdout: "rss": 51132, 2024-09-12T03:19:25.677 INFO:teuthology.orchestra.run.smithi177.stdout: "heap": 223520 2024-09-12T03:19:25.677 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.677 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_server": { 2024-09-12T03:19:25.677 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_client_request": 9, 2024-09-12T03:19:25.677 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_server_request": 0, 2024-09-12T03:19:25.677 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_request": 4, 2024-09-12T03:19:25.677 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_session": 4, 2024-09-12T03:19:25.677 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_peer_request": 1, 2024-09-12T03:19:25.677 INFO:teuthology.orchestra.run.smithi177.stdout: "req_create_latency": { 2024-09-12T03:19:25.677 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.677 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.678 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.678 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.678 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getattr_latency": { 2024-09-12T03:19:25.678 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 1, 2024-09-12T03:19:25.678 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.001680011, 2024-09-12T03:19:25.678 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.001680011 2024-09-12T03:19:25.678 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.678 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getfilelock_latency": { 2024-09-12T03:19:25.678 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.678 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.678 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.678 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.678 INFO:teuthology.orchestra.run.smithi177.stdout: "req_link_latency": { 2024-09-12T03:19:25.678 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.679 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.679 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.679 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.679 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookup_latency": { 2024-09-12T03:19:25.679 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2, 2024-09-12T03:19:25.679 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.001024677, 2024-09-12T03:19:25.679 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000512338 2024-09-12T03:19:25.679 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.679 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookuphash_latency": { 2024-09-12T03:19:25.679 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.679 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.679 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.679 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.680 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupino_latency": { 2024-09-12T03:19:25.680 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.680 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.680 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.680 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.680 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupname_latency": { 2024-09-12T03:19:25.680 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.680 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.680 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.680 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.680 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupparent_latency": { 2024-09-12T03:19:25.680 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.680 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.681 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.681 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.681 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupsnap_latency": { 2024-09-12T03:19:25.681 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.681 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.681 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.681 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.681 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lssnap_latency": { 2024-09-12T03:19:25.681 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.681 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.681 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.681 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.681 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mkdir_latency": { 2024-09-12T03:19:25.682 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.682 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.682 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.682 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.682 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mknod_latency": { 2024-09-12T03:19:25.682 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.682 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.682 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.682 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.682 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mksnap_latency": { 2024-09-12T03:19:25.682 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.682 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.682 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.683 INFO:teuthology.orchestra.run.smithi177.stdout: "req_open_latency": { 2024-09-12T03:19:25.683 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.683 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.683 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.683 INFO:teuthology.orchestra.run.smithi177.stdout: "req_readdir_latency": { 2024-09-12T03:19:25.683 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.683 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.683 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.683 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.683 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rename_latency": { 2024-09-12T03:19:25.683 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.683 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.684 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.684 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.684 INFO:teuthology.orchestra.run.smithi177.stdout: "req_renamesnap_latency": { 2024-09-12T03:19:25.684 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.684 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.684 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.684 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.684 INFO:teuthology.orchestra.run.smithi177.stdout: "req_snapdiff_latency": { 2024-09-12T03:19:25.684 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.684 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.684 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.684 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.684 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmdir_latency": { 2024-09-12T03:19:25.685 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.685 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.685 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.685 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmsnap_latency": { 2024-09-12T03:19:25.685 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.685 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.685 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.685 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmxattr_latency": { 2024-09-12T03:19:25.685 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.685 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.685 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.685 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.686 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setattr_latency": { 2024-09-12T03:19:25.686 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.686 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.686 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.686 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.686 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setdirlayout_latency": { 2024-09-12T03:19:25.686 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.686 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.686 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.686 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.686 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setfilelock_latency": { 2024-09-12T03:19:25.686 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.687 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.687 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.687 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setlayout_latency": { 2024-09-12T03:19:25.687 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.687 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.687 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.687 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setxattr_latency": { 2024-09-12T03:19:25.687 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.687 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.687 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.687 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.687 INFO:teuthology.orchestra.run.smithi177.stdout: "req_symlink_latency": { 2024-09-12T03:19:25.688 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.688 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.688 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.688 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.688 INFO:teuthology.orchestra.run.smithi177.stdout: "req_unlink_latency": { 2024-09-12T03:19:25.688 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.688 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.688 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.688 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.688 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:19:25.688 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:19:25.688 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getvxattr_latency": { 2024-09-12T03:19:25.688 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.689 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.689 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.689 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-12T03:19:25.689 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.689 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_sessions": { 2024-09-12T03:19:25.689 INFO:teuthology.orchestra.run.smithi177.stdout: "session_count": 0, 2024-09-12T03:19:25.689 INFO:teuthology.orchestra.run.smithi177.stdout: "session_add": 1, 2024-09-12T03:19:25.689 INFO:teuthology.orchestra.run.smithi177.stdout: "session_remove": 1, 2024-09-12T03:19:25.689 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_open": 0, 2024-09-12T03:19:25.689 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_stale": 0, 2024-09-12T03:19:25.689 INFO:teuthology.orchestra.run.smithi177.stdout: "total_load": 2, 2024-09-12T03:19:25.689 INFO:teuthology.orchestra.run.smithi177.stdout: "average_load": 2, 2024-09-12T03:19:25.690 INFO:teuthology.orchestra.run.smithi177.stdout: "avg_session_uptime": 64, 2024-09-12T03:19:25.690 INFO:teuthology.orchestra.run.smithi177.stdout: "mdthresh_evicted": 0 2024-09-12T03:19:25.690 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.690 INFO:teuthology.orchestra.run.smithi177.stdout: "mempool": { 2024-09-12T03:19:25.690 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:19:25.690 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_items": 0, 2024-09-12T03:19:25.690 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:19:25.690 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:19:25.690 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:19:25.690 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:19:25.690 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:19:25.690 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:19:25.690 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:19:25.690 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:19:25.691 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:19:25.691 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:19:25.691 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:19:25.691 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:19:25.691 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:19:25.691 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_items": 0, 2024-09-12T03:19:25.691 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:19:25.691 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_items": 0, 2024-09-12T03:19:25.691 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:19:25.691 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:19:25.691 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:19:25.691 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:19:25.691 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:19:25.692 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:19:25.692 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:19:25.692 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_items": 0, 2024-09-12T03:19:25.692 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:19:25.692 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:19:25.692 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:19:25.692 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_items": 0, 2024-09-12T03:19:25.692 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_bytes": 0, 2024-09-12T03:19:25.692 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_items": 0, 2024-09-12T03:19:25.692 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:19:25.692 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:19:25.692 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:19:25.692 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:19:25.692 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_bytes": 238631, 2024-09-12T03:19:25.693 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_items": 73, 2024-09-12T03:19:25.693 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:19:25.693 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_items": 0, 2024-09-12T03:19:25.693 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_bytes": 0, 2024-09-12T03:19:25.693 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_items": 0, 2024-09-12T03:19:25.693 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:19:25.693 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_items": 0, 2024-09-12T03:19:25.693 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:19:25.693 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_items": 0, 2024-09-12T03:19:25.693 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_bytes": 7400, 2024-09-12T03:19:25.693 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_items": 152, 2024-09-12T03:19:25.693 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:19:25.694 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:19:25.694 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_bytes": 0, 2024-09-12T03:19:25.694 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_items": 0, 2024-09-12T03:19:25.694 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_bytes": 59360, 2024-09-12T03:19:25.694 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_items": 237, 2024-09-12T03:19:25.694 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_bytes": 0, 2024-09-12T03:19:25.694 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_items": 0, 2024-09-12T03:19:25.694 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_bytes": 0, 2024-09-12T03:19:25.694 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_items": 0 2024-09-12T03:19:25.694 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.694 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter": { 2024-09-12T03:19:25.694 INFO:teuthology.orchestra.run.smithi177.stdout: "op_active": 0, 2024-09-12T03:19:25.694 INFO:teuthology.orchestra.run.smithi177.stdout: "op_laggy": 0, 2024-09-12T03:19:25.694 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send": 41, 2024-09-12T03:19:25.695 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send_bytes": 11649, 2024-09-12T03:19:25.695 INFO:teuthology.orchestra.run.smithi177.stdout: "op_resend": 33, 2024-09-12T03:19:25.695 INFO:teuthology.orchestra.run.smithi177.stdout: "op_reply": 41, 2024-09-12T03:19:25.695 INFO:teuthology.orchestra.run.smithi177.stdout: "op_latency": { 2024-09-12T03:19:25.695 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 41, 2024-09-12T03:19:25.695 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.194994482, 2024-09-12T03:19:25.695 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.004755962 2024-09-12T03:19:25.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.695 INFO:teuthology.orchestra.run.smithi177.stdout: "op_inflight": 0, 2024-09-12T03:19:25.695 INFO:teuthology.orchestra.run.smithi177.stdout: "oplen_avg": { 2024-09-12T03:19:25.695 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 41, 2024-09-12T03:19:25.695 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 65 2024-09-12T03:19:25.695 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.695 INFO:teuthology.orchestra.run.smithi177.stdout: "op": 41, 2024-09-12T03:19:25.696 INFO:teuthology.orchestra.run.smithi177.stdout: "op_r": 0, 2024-09-12T03:19:25.696 INFO:teuthology.orchestra.run.smithi177.stdout: "op_w": 41, 2024-09-12T03:19:25.696 INFO:teuthology.orchestra.run.smithi177.stdout: "op_rmw": 0, 2024-09-12T03:19:25.696 INFO:teuthology.orchestra.run.smithi177.stdout: "op_pg": 0, 2024-09-12T03:19:25.696 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_stat": 0, 2024-09-12T03:19:25.696 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_create": 10, 2024-09-12T03:19:25.696 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_read": 0, 2024-09-12T03:19:25.696 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_write": 3, 2024-09-12T03:19:25.696 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writefull": 7, 2024-09-12T03:19:25.696 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writesame": 0, 2024-09-12T03:19:25.696 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_append": 0, 2024-09-12T03:19:25.696 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_zero": 0, 2024-09-12T03:19:25.696 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_truncate": 0, 2024-09-12T03:19:25.697 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_delete": 6, 2024-09-12T03:19:25.697 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_mapext": 0, 2024-09-12T03:19:25.697 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_sparse_read": 0, 2024-09-12T03:19:25.697 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_clonerange": 0, 2024-09-12T03:19:25.697 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_getxattr": 0, 2024-09-12T03:19:25.697 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_setxattr": 20, 2024-09-12T03:19:25.697 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:19:25.697 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_rmxattr": 0, 2024-09-12T03:19:25.697 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:19:25.697 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_call": 0, 2024-09-12T03:19:25.697 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_watch": 0, 2024-09-12T03:19:25.697 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_notify": 0, 2024-09-12T03:19:25.697 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:19:25.697 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls": 0, 2024-09-12T03:19:25.698 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:19:25.698 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_other": 0, 2024-09-12T03:19:25.698 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_active": 0, 2024-09-12T03:19:25.698 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_send": 0, 2024-09-12T03:19:25.698 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_resend": 0, 2024-09-12T03:19:25.698 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_ping": 0, 2024-09-12T03:19:25.698 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_active": 0, 2024-09-12T03:19:25.698 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_send": 0, 2024-09-12T03:19:25.698 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_resend": 0, 2024-09-12T03:19:25.698 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_active": 0, 2024-09-12T03:19:25.698 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_send": 0, 2024-09-12T03:19:25.698 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_resend": 0, 2024-09-12T03:19:25.698 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_active": 0, 2024-09-12T03:19:25.698 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_send": 0, 2024-09-12T03:19:25.699 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_resend": 0, 2024-09-12T03:19:25.699 INFO:teuthology.orchestra.run.smithi177.stdout: "command_active": 0, 2024-09-12T03:19:25.699 INFO:teuthology.orchestra.run.smithi177.stdout: "command_send": 0, 2024-09-12T03:19:25.699 INFO:teuthology.orchestra.run.smithi177.stdout: "command_resend": 0, 2024-09-12T03:19:25.699 INFO:teuthology.orchestra.run.smithi177.stdout: "map_epoch": 80, 2024-09-12T03:19:25.699 INFO:teuthology.orchestra.run.smithi177.stdout: "map_full": 0, 2024-09-12T03:19:25.699 INFO:teuthology.orchestra.run.smithi177.stdout: "map_inc": 2, 2024-09-12T03:19:25.699 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_sessions": 9, 2024-09-12T03:19:25.699 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_open": 14, 2024-09-12T03:19:25.699 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_close": 5, 2024-09-12T03:19:25.699 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_laggy": 0, 2024-09-12T03:19:25.699 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_wr": 17, 2024-09-12T03:19:25.699 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_rd": 0, 2024-09-12T03:19:25.699 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_del": 2 2024-09-12T03:19:25.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.700 INFO:teuthology.orchestra.run.smithi177.stdout: "oft": { 2024-09-12T03:19:25.700 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_objs": 1, 2024-09-12T03:19:25.700 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:19:25.700 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_updates": 0, 2024-09-12T03:19:25.700 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_removes": 0 2024-09-12T03:19:25.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.700 INFO:teuthology.orchestra.run.smithi177.stdout: "osd-slow-ops": { 2024-09-12T03:19:25.700 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_ops_count": 56 2024-09-12T03:19:25.700 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.700 INFO:teuthology.orchestra.run.smithi177.stdout: "purge_queue": { 2024-09-12T03:19:25.700 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops": 0, 2024-09-12T03:19:25.700 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:19:25.701 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing": 0, 2024-09-12T03:19:25.701 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_high_water": 0, 2024-09-12T03:19:25.701 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executed_ops": 0, 2024-09-12T03:19:25.701 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executed": 0, 2024-09-12T03:19:25.701 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_item_in_journal": 0 2024-09-12T03:19:25.701 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.701 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:19:25.701 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-09-12T03:19:25.701 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2024-09-12T03:19:25.701 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-09-12T03:19:25.701 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 1420, 2024-09-12T03:19:25.701 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 900335, 2024-09-12T03:19:25.701 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:25.701 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 1420, 2024-09-12T03:19:25.702 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-09-12T03:19:25.702 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-09-12T03:19:25.702 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 1420, 2024-09-12T03:19:25.702 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 900335, 2024-09-12T03:19:25.702 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-09-12T03:19:25.702 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.702 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.702 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.702 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-12T03:19:25.702 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.702 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_bytes": { 2024-09-12T03:19:25.702 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-09-12T03:19:25.702 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2024-09-12T03:19:25.702 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-09-12T03:19:25.703 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-09-12T03:19:25.703 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-09-12T03:19:25.703 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:25.703 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-09-12T03:19:25.703 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 41, 2024-09-12T03:19:25.703 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 11649, 2024-09-12T03:19:25.703 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 41, 2024-09-12T03:19:25.703 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 11649, 2024-09-12T03:19:25.703 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-09-12T03:19:25.703 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.703 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.703 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.703 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-12T03:19:25.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.704 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_ops": { 2024-09-12T03:19:25.704 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-09-12T03:19:25.704 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 1024, 2024-09-12T03:19:25.704 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-09-12T03:19:25.704 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-09-12T03:19:25.704 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-09-12T03:19:25.704 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:25.704 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-09-12T03:19:25.704 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 41, 2024-09-12T03:19:25.704 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 41, 2024-09-12T03:19:25.704 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 41, 2024-09-12T03:19:25.705 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 41, 2024-09-12T03:19:25.705 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-09-12T03:19:25.705 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.705 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.705 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.705 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-12T03:19:25.705 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.705 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:19:25.705 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-09-12T03:19:25.705 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2024-09-12T03:19:25.705 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-09-12T03:19:25.705 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-09-12T03:19:25.705 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-09-12T03:19:25.705 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:25.706 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-09-12T03:19:25.706 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-09-12T03:19:25.706 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-09-12T03:19:25.706 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2024-09-12T03:19:25.706 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2024-09-12T03:19:25.706 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-09-12T03:19:25.706 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.706 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.706 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.706 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-12T03:19:25.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:25.707 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle-0x55b0ab324960": { 2024-09-12T03:19:25.707 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-09-12T03:19:25.707 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2024-09-12T03:19:25.707 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-09-12T03:19:25.707 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 4, 2024-09-12T03:19:25.707 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 1122, 2024-09-12T03:19:25.707 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:25.707 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 4, 2024-09-12T03:19:25.707 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-09-12T03:19:25.707 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-09-12T03:19:25.707 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 3, 2024-09-12T03:19:25.708 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 1122, 2024-09-12T03:19:25.708 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-09-12T03:19:25.708 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:25.708 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:25.708 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:25.708 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-12T03:19:25.708 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-12T03:19:25.708 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-09-12T03:19:25.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:25 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:19:25.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:25 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:19:25.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:25 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:19:25.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:25 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:19:25.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:25 smithi183 ceph-mon[35535]: Adjusting osd_memory_target on smithi116 to 176.0M 2024-09-12T03:19:25.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:25 smithi183 ceph-mon[35535]: Unable to set osd_memory_target on smithi116 to 184577433: error parsing value: Value '184577433' is below minimum 939524096 2024-09-12T03:19:25.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:25 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:25.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:25 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:19:25.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:25 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:19:25.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:25 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:19:25.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:25 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:19:25.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:25 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:19:25.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:25 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:19:25.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:25 smithi177 ceph-mon[35305]: Adjusting osd_memory_target on smithi116 to 176.0M 2024-09-12T03:19:25.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:25 smithi177 ceph-mon[35305]: Unable to set osd_memory_target on smithi116 to 184577433: error parsing value: Value '184577433' is below minimum 939524096 2024-09-12T03:19:25.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:25 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:25.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:25 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:19:25.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:25 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:19:26.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:25 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:19:26.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:25 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:19:26.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:25 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:19:26.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:25 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:19:26.052 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:25 smithi116 ceph-mon[31410]: Adjusting osd_memory_target on smithi116 to 176.0M 2024-09-12T03:19:26.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:25 smithi116 ceph-mon[31410]: Unable to set osd_memory_target on smithi116 to 184577433: error parsing value: Value '184577433' is below minimum 939524096 2024-09-12T03:19:26.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:25 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:26.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:25 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:19:26.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:25 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' 2024-09-12T03:19:26.486 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-12T03:19:26.486 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-12T03:19:26.486 DEBUG:tasks.check_counter:Getting stats from k 2024-09-12T03:19:26.487 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-12T03:19:26.729 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:19:27.297 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:27 smithi177 ceph-mon[35305]: pgmap v831: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-12T03:19:27.367 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-09-12T03:19:27.368 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:19:27.368 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 0, 2024-09-12T03:19:27.368 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 2, 2024-09-12T03:19:27.368 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 0, 2024-09-12T03:19:27.368 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 0, 2024-09-12T03:19:27.368 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 4, 2024-09-12T03:19:27.368 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 0, 2024-09-12T03:19:27.368 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.001091080, 2024-09-12T03:19:27.368 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T03:19:27.369 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.000288620, 2024-09-12T03:19:27.369 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:19:27.369 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:27.369 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:27.369 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:27.369 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:27.369 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:27.369 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:27.369 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:27.369 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:27.369 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:27.370 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:27.370 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:19:27.370 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:19:27.370 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:27.370 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:19:27.370 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 6, 2024-09-12T03:19:27.370 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 236, 2024-09-12T03:19:27.370 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 3344, 2024-09-12T03:19:27.370 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 199072, 2024-09-12T03:19:27.370 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 4, 2024-09-12T03:19:27.370 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 2, 2024-09-12T03:19:27.370 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.016727525, 2024-09-12T03:19:27.371 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.010716470, 2024-09-12T03:19:27.371 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.003220559, 2024-09-12T03:19:27.371 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:19:27.371 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:27.371 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 235, 2024-09-12T03:19:27.371 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.019370337, 2024-09-12T03:19:27.371 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000082426 2024-09-12T03:19:27.371 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:27.371 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:27.371 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:27.371 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:27.372 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:27.372 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:27.372 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-12T03:19:27.372 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 199072 2024-09-12T03:19:27.372 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:27.372 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:19:27.372 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 311, 2024-09-12T03:19:27.372 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 306, 2024-09-12T03:19:27.372 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 780144, 2024-09-12T03:19:27.372 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 188720, 2024-09-12T03:19:27.372 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 2, 2024-09-12T03:19:27.373 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 1, 2024-09-12T03:19:27.373 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.047798969, 2024-09-12T03:19:27.373 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.022895198, 2024-09-12T03:19:27.373 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.030864247, 2024-09-12T03:19:27.373 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:19:27.373 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:27.373 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 305, 2024-09-12T03:19:27.373 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.018927966, 2024-09-12T03:19:27.373 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000062058 2024-09-12T03:19:27.373 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:27.373 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:27.373 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:27.374 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:27.374 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:27.374 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:27.374 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 780144, 2024-09-12T03:19:27.374 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 188720 2024-09-12T03:19:27.374 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:27.374 INFO:teuthology.orchestra.run.smithi177.stdout: "cct": { 2024-09-12T03:19:27.374 INFO:teuthology.orchestra.run.smithi177.stdout: "total_workers": 0, 2024-09-12T03:19:27.374 INFO:teuthology.orchestra.run.smithi177.stdout: "unhealthy_workers": 0 2024-09-12T03:19:27.374 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:27.374 INFO:teuthology.orchestra.run.smithi177.stdout: "mempool": { 2024-09-12T03:19:27.374 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:19:27.375 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_items": 0, 2024-09-12T03:19:27.375 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:19:27.375 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:19:27.375 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:19:27.375 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:19:27.375 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:19:27.375 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:19:27.375 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:19:27.375 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:19:27.376 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:19:27.376 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:19:27.376 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:19:27.376 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:19:27.376 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:19:27.376 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_items": 0, 2024-09-12T03:19:27.376 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:19:27.376 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_items": 0, 2024-09-12T03:19:27.376 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:19:27.377 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:19:27.377 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:19:27.377 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:19:27.377 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:19:27.377 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:19:27.377 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:19:27.377 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_items": 0, 2024-09-12T03:19:27.377 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:19:27.377 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:19:27.377 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:19:27.378 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_items": 0, 2024-09-12T03:19:27.378 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_bytes": 0, 2024-09-12T03:19:27.378 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_items": 0, 2024-09-12T03:19:27.378 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:19:27.378 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:19:27.378 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:19:27.378 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:19:27.378 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_bytes": 148704, 2024-09-12T03:19:27.378 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_items": 29, 2024-09-12T03:19:27.378 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:19:27.378 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_items": 0, 2024-09-12T03:19:27.379 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_bytes": 0, 2024-09-12T03:19:27.379 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_items": 0, 2024-09-12T03:19:27.379 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:19:27.379 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_items": 0, 2024-09-12T03:19:27.379 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:19:27.379 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_items": 0, 2024-09-12T03:19:27.379 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_bytes": 0, 2024-09-12T03:19:27.379 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_items": 0, 2024-09-12T03:19:27.379 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:19:27.379 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:19:27.379 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_bytes": 0, 2024-09-12T03:19:27.379 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_items": 0, 2024-09-12T03:19:27.380 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_bytes": 928, 2024-09-12T03:19:27.380 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_items": 2, 2024-09-12T03:19:27.380 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_bytes": 0, 2024-09-12T03:19:27.380 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_items": 0, 2024-09-12T03:19:27.381 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_bytes": 0, 2024-09-12T03:19:27.381 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_items": 0 2024-09-12T03:19:27.381 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-09-12T03:19:27.381 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:19:27.381 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-09-12T03:19:27.381 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2024-09-12T03:19:27.381 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-09-12T03:19:27.381 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 313, 2024-09-12T03:19:27.381 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 736225, 2024-09-12T03:19:27.381 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:27.381 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 313, 2024-09-12T03:19:27.381 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-09-12T03:19:27.382 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-09-12T03:19:27.382 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 313, 2024-09-12T03:19:27.382 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 736225, 2024-09-12T03:19:27.382 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-09-12T03:19:27.382 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-09-12T03:19:27.383 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-09-12T03:19:27.383 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-09-12T03:19:27.383 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-12T03:19:27.383 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-09-12T03:19:27.383 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-09-12T03:19:27.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:27 smithi183 ceph-mon[35535]: pgmap v831: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-12T03:19:27.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:27 smithi116 ceph-mon[31410]: pgmap v831: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-12T03:19:28.122 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-12T03:19:28.122 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-12T03:19:28.122 DEBUG:tasks.check_counter:Getting stats from c 2024-09-12T03:19:28.122 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-12T03:19:28.379 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:19:28.858 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-12T03:19:28.858 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:19:28.858 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 2469, 2024-09-12T03:19:28.858 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 2460, 2024-09-12T03:19:28.858 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 5638090, 2024-09-12T03:19:28.858 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 871342, 2024-09-12T03:19:28.859 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 5, 2024-09-12T03:19:28.859 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 1, 2024-09-12T03:19:28.859 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.264038469, 2024-09-12T03:19:28.859 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.086731219, 2024-09-12T03:19:28.859 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.095315941, 2024-09-12T03:19:28.859 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.068069425, 2024-09-12T03:19:28.859 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:28.859 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2460, 2024-09-12T03:19:28.859 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.113967706, 2024-09-12T03:19:28.859 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000046328 2024-09-12T03:19:28.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.859 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:28.859 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.860 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.860 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.860 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 805232, 2024-09-12T03:19:28.860 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 188592 2024-09-12T03:19:28.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.860 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:19:28.860 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 423, 2024-09-12T03:19:28.860 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 652, 2024-09-12T03:19:28.860 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 104644, 2024-09-12T03:19:28.860 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 618086, 2024-09-12T03:19:28.860 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 6, 2024-09-12T03:19:28.861 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 4, 2024-09-12T03:19:28.861 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.056961833, 2024-09-12T03:19:28.861 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.025734876, 2024-09-12T03:19:28.865 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.010707696, 2024-09-12T03:19:28.865 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.014199129, 2024-09-12T03:19:28.865 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:28.866 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 650, 2024-09-12T03:19:28.866 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.034128795, 2024-09-12T03:19:28.866 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000052505 2024-09-12T03:19:28.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.866 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:28.866 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.866 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.866 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.866 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T03:19:28.866 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 483824 2024-09-12T03:19:28.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.866 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:19:28.867 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1385, 2024-09-12T03:19:28.867 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1387, 2024-09-12T03:19:28.867 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 3729053, 2024-09-12T03:19:28.867 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 440437, 2024-09-12T03:19:28.867 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 4, 2024-09-12T03:19:28.867 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 3, 2024-09-12T03:19:28.867 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.127652785, 2024-09-12T03:19:28.867 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.048365174, 2024-09-12T03:19:28.867 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.030770538, 2024-09-12T03:19:28.867 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.036596173, 2024-09-12T03:19:28.867 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:28.867 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1385, 2024-09-12T03:19:28.867 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.051552258, 2024-09-12T03:19:28.868 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000037221 2024-09-12T03:19:28.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.868 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:28.868 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.868 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.868 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.868 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:19:28.868 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:19:28.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.868 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-09-12T03:19:28.868 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-09-12T03:19:28.869 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-09-12T03:19:28.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.869 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-09-12T03:19:28.869 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-12T03:19:28.869 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-12T03:19:28.869 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 6684, 2024-09-12T03:19:28.869 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.176562870, 2024-09-12T03:19:28.869 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000026415 2024-09-12T03:19:28.869 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T03:19:28.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.869 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-09-12T03:19:28.869 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-12T03:19:28.869 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-12T03:19:28.870 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.870 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.870 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.870 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T03:19:28.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.870 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-09-12T03:19:28.870 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 0, 2024-09-12T03:19:28.870 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 0, 2024-09-12T03:19:28.870 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-09-12T03:19:28.870 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.870 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.870 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.870 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-09-12T03:19:28.871 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 0, 2024-09-12T03:19:28.871 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-09-12T03:19:28.871 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-09-12T03:19:28.871 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 0, 2024-09-12T03:19:28.871 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-09-12T03:19:28.871 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-09-12T03:19:28.871 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 18, 2024-09-12T03:19:28.871 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 0, 2024-09-12T03:19:28.871 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 18, 2024-09-12T03:19:28.871 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-09-12T03:19:28.871 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 18, 2024-09-12T03:19:28.871 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 0, 2024-09-12T03:19:28.871 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-09-12T03:19:28.872 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-09-12T03:19:28.872 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 2, 2024-09-12T03:19:28.872 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 0, 2024-09-12T03:19:28.872 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 0, 2024-09-12T03:19:28.872 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2024-09-12T03:19:28.872 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 0, 2024-09-12T03:19:28.872 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:19:28.872 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-09-12T03:19:28.872 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-09-12T03:19:28.872 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-09-12T03:19:28.872 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-09-12T03:19:28.872 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2024-09-12T03:19:28.872 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-09-12T03:19:28.873 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-09-12T03:19:28.873 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-09-12T03:19:28.873 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-09-12T03:19:28.873 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:19:28.873 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-09-12T03:19:28.873 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 4, 2024-09-12T03:19:28.873 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 234, 2024-09-12T03:19:28.873 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 0, 2024-09-12T03:19:28.873 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:19:28.873 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-09-12T03:19:28.873 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:19:28.873 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:19:28.873 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:19:28.874 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:19:28.874 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:19:28.874 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-09-12T03:19:28.874 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:19:28.874 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:19:28.874 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:19:28.874 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:19:28.874 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:19:28.874 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:19:28.874 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 0, 2024-09-12T03:19:28.874 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:19:28.874 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 0, 2024-09-12T03:19:28.874 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 0 2024-09-12T03:19:28.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.875 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-09-12T03:19:28.875 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-09-12T03:19:28.875 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-09-12T03:19:28.875 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-09-12T03:19:28.875 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 0, 2024-09-12T03:19:28.875 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 0, 2024-09-12T03:19:28.875 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-09-12T03:19:28.875 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-09-12T03:19:28.875 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:19:28.875 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 0, 2024-09-12T03:19:28.875 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 0, 2024-09-12T03:19:28.875 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-09-12T03:19:28.875 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-09-12T03:19:28.875 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-09-12T03:19:28.876 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:19:28.876 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:19:28.876 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-09-12T03:19:28.876 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-09-12T03:19:28.876 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:19:28.876 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:19:28.876 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:19:28.876 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2024-09-12T03:19:28.876 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-09-12T03:19:28.876 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:19:28.876 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-09-12T03:19:28.876 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-09-12T03:19:28.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.877 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-09-12T03:19:28.877 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-09-12T03:19:28.877 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 0, 2024-09-12T03:19:28.877 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-09-12T03:19:28.877 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-09-12T03:19:28.877 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 2283, 2024-09-12T03:19:28.877 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-09-12T03:19:28.877 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-09-12T03:19:28.877 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 0, 2024-09-12T03:19:28.877 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-09-12T03:19:28.877 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-09-12T03:19:28.877 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 4, 2024-09-12T03:19:28.877 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2024-09-12T03:19:28.877 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-09-12T03:19:28.878 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-09-12T03:19:28.878 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2024-09-12T03:19:28.878 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 11769917, 2024-09-12T03:19:28.878 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 11767420, 2024-09-12T03:19:28.878 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-09-12T03:19:28.878 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.878 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.878 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.878 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 2283 2024-09-12T03:19:28.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.878 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-09-12T03:19:28.879 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 15, 2024-09-12T03:19:28.879 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 17, 2024-09-12T03:19:28.879 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 2, 2024-09-12T03:19:28.879 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 10, 2024-09-12T03:19:28.879 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 11, 2024-09-12T03:19:28.879 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 1, 2024-09-12T03:19:28.879 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 18, 2024-09-12T03:19:28.879 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 19, 2024-09-12T03:19:28.879 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 1, 2024-09-12T03:19:28.879 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-09-12T03:19:28.879 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 0, 2024-09-12T03:19:28.879 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 0, 2024-09-12T03:19:28.879 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 48928, 2024-09-12T03:19:28.880 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 215328 2024-09-12T03:19:28.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.880 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-09-12T03:19:28.880 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 0, 2024-09-12T03:19:28.880 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 0, 2024-09-12T03:19:28.880 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 0, 2024-09-12T03:19:28.880 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 0, 2024-09-12T03:19:28.880 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 0, 2024-09-12T03:19:28.880 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-09-12T03:19:28.880 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.880 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.880 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.880 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-09-12T03:19:28.881 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.881 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.881 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.881 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-09-12T03:19:28.881 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.881 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.881 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.881 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-09-12T03:19:28.881 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.881 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.882 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.882 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-09-12T03:19:28.882 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.882 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.882 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.882 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-09-12T03:19:28.882 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.882 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.882 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.882 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-09-12T03:19:28.883 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.883 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.883 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.883 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-09-12T03:19:28.883 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.883 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.883 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.883 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-09-12T03:19:28.883 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.883 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.883 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.884 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-09-12T03:19:28.884 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.884 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.884 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.884 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-09-12T03:19:28.884 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.884 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.884 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.884 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-09-12T03:19:28.885 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.885 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.885 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.885 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-09-12T03:19:28.885 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.886 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.886 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.886 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-09-12T03:19:28.886 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.886 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.886 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.886 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-09-12T03:19:28.887 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.887 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.887 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.887 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-09-12T03:19:28.887 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.887 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.887 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.887 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-09-12T03:19:28.887 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.887 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.887 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.888 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-09-12T03:19:28.888 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.888 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.888 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.888 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-09-12T03:19:28.888 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.888 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.888 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.888 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-09-12T03:19:28.888 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.889 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.889 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.889 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-09-12T03:19:28.889 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.889 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.889 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.889 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-09-12T03:19:28.889 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.889 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.889 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.890 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-09-12T03:19:28.890 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.890 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.890 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.890 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-09-12T03:19:28.890 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.890 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.890 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.890 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-09-12T03:19:28.890 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.890 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.890 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.891 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-09-12T03:19:28.891 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.891 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.891 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.891 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-09-12T03:19:28.891 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.891 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.891 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.891 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-09-12T03:19:28.891 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.891 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.892 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.892 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-09-12T03:19:28.892 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.892 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.892 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.892 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:19:28.892 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:19:28.892 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-09-12T03:19:28.892 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.892 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.892 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.892 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T03:19:28.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.893 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-09-12T03:19:28.893 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 1, 2024-09-12T03:19:28.893 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 3, 2024-09-12T03:19:28.893 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 2, 2024-09-12T03:19:28.893 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 2, 2024-09-12T03:19:28.893 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-09-12T03:19:28.893 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 0, 2024-09-12T03:19:28.893 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 0, 2024-09-12T03:19:28.893 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 3606, 2024-09-12T03:19:28.893 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-09-12T03:19:28.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.893 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-09-12T03:19:28.893 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:19:28.894 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-09-12T03:19:28.894 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:19:28.894 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:19:28.894 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:19:28.894 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:19:28.894 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:19:28.894 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:19:28.894 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:19:28.894 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:19:28.894 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:19:28.894 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:19:28.894 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:19:28.894 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:19:28.895 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:19:28.895 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-09-12T03:19:28.895 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:19:28.895 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-09-12T03:19:28.895 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:19:28.895 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:19:28.895 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:19:28.895 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:19:28.895 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:19:28.895 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:19:28.895 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:19:28.895 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-09-12T03:19:28.895 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:19:28.895 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:19:28.896 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:19:28.896 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-09-12T03:19:28.896 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-09-12T03:19:28.896 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-09-12T03:19:28.896 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:19:28.896 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:19:28.896 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:19:28.896 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:19:28.896 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 190204, 2024-09-12T03:19:28.896 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 53, 2024-09-12T03:19:28.896 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:19:28.896 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-09-12T03:19:28.896 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-09-12T03:19:28.896 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-09-12T03:19:28.897 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:19:28.897 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-09-12T03:19:28.897 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:19:28.897 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-09-12T03:19:28.897 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 7400, 2024-09-12T03:19:28.897 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 152, 2024-09-12T03:19:28.897 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:19:28.897 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:19:28.897 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-09-12T03:19:28.897 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-09-12T03:19:28.897 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 59298, 2024-09-12T03:19:28.897 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 354, 2024-09-12T03:19:28.897 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-09-12T03:19:28.897 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-09-12T03:19:28.898 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-09-12T03:19:28.898 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-09-12T03:19:28.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.898 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-09-12T03:19:28.898 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-09-12T03:19:28.898 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-09-12T03:19:28.898 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 3954, 2024-09-12T03:19:28.898 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 26, 2024-09-12T03:19:28.898 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 4, 2024-09-12T03:19:28.898 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 3954, 2024-09-12T03:19:28.898 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-09-12T03:19:28.898 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3954, 2024-09-12T03:19:28.898 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1.757951401, 2024-09-12T03:19:28.898 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000444600 2024-09-12T03:19:28.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.899 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-09-12T03:19:28.899 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-09-12T03:19:28.899 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3954, 2024-09-12T03:19:28.899 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 3955 2024-09-12T03:19:28.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.899 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 3954, 2024-09-12T03:19:28.899 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 3954, 2024-09-12T03:19:28.899 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 0, 2024-09-12T03:19:28.899 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-09-12T03:19:28.899 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-09-12T03:19:28.899 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 2189, 2024-09-12T03:19:28.899 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 0, 2024-09-12T03:19:28.899 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 1762, 2024-09-12T03:19:28.900 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 0, 2024-09-12T03:19:28.900 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 0, 2024-09-12T03:19:28.900 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-09-12T03:19:28.900 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-09-12T03:19:28.900 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-09-12T03:19:28.900 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-09-12T03:19:28.900 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 0, 2024-09-12T03:19:28.900 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-09-12T03:19:28.900 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-09-12T03:19:28.900 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-09-12T03:19:28.900 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 2, 2024-09-12T03:19:28.900 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 0, 2024-09-12T03:19:28.900 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:19:28.900 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-09-12T03:19:28.901 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:19:28.901 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-09-12T03:19:28.901 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-09-12T03:19:28.901 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-09-12T03:19:28.901 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:19:28.901 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-09-12T03:19:28.901 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:19:28.901 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-09-12T03:19:28.901 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-09-12T03:19:28.901 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-09-12T03:19:28.901 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-09-12T03:19:28.901 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-09-12T03:19:28.901 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-09-12T03:19:28.901 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-09-12T03:19:28.902 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-09-12T03:19:28.902 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-09-12T03:19:28.902 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-09-12T03:19:28.902 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-09-12T03:19:28.902 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-09-12T03:19:28.902 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-09-12T03:19:28.902 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-09-12T03:19:28.902 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-09-12T03:19:28.902 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-09-12T03:19:28.902 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-09-12T03:19:28.902 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 80, 2024-09-12T03:19:28.902 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-09-12T03:19:28.902 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 2, 2024-09-12T03:19:28.902 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 5, 2024-09-12T03:19:28.903 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 6, 2024-09-12T03:19:28.903 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 1, 2024-09-12T03:19:28.903 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-09-12T03:19:28.903 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 0, 2024-09-12T03:19:28.903 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 2, 2024-09-12T03:19:28.903 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 0 2024-09-12T03:19:28.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.903 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-09-12T03:19:28.903 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 0, 2024-09-12T03:19:28.903 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:19:28.903 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-09-12T03:19:28.903 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-09-12T03:19:28.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.904 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-09-12T03:19:28.904 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2024-09-12T03:19:28.904 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.904 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-09-12T03:19:28.904 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-09-12T03:19:28.904 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:19:28.904 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-09-12T03:19:28.904 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 0, 2024-09-12T03:19:28.904 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 0, 2024-09-12T03:19:28.904 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 0, 2024-09-12T03:19:28.904 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-09-12T03:19:28.904 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.904 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:19:28.904 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-12T03:19:28.905 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-12T03:19:28.905 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-12T03:19:28.905 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 4273, 2024-09-12T03:19:28.905 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 9067856, 2024-09-12T03:19:28.905 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:28.905 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 4273, 2024-09-12T03:19:28.905 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-12T03:19:28.905 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-12T03:19:28.905 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 4273, 2024-09-12T03:19:28.905 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 9067856, 2024-09-12T03:19:28.905 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-12T03:19:28.905 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.905 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.906 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.906 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T03:19:28.906 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.906 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-09-12T03:19:28.906 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-12T03:19:28.906 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-12T03:19:28.906 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-12T03:19:28.906 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-12T03:19:28.906 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-12T03:19:28.906 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:28.906 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-12T03:19:28.906 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 3954, 2024-09-12T03:19:28.906 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 7575623, 2024-09-12T03:19:28.907 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3954, 2024-09-12T03:19:28.907 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 7575623, 2024-09-12T03:19:28.907 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-12T03:19:28.907 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.907 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.907 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.907 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T03:19:28.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.907 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-09-12T03:19:28.907 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-12T03:19:28.907 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-09-12T03:19:28.907 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-12T03:19:28.907 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-12T03:19:28.908 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-12T03:19:28.908 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:28.908 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-12T03:19:28.908 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 3954, 2024-09-12T03:19:28.908 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 3954, 2024-09-12T03:19:28.908 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3954, 2024-09-12T03:19:28.908 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 3954, 2024-09-12T03:19:28.908 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-12T03:19:28.908 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.908 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.909 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.909 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T03:19:28.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.909 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:19:28.909 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-12T03:19:28.909 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-12T03:19:28.909 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-12T03:19:28.909 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-12T03:19:28.909 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-12T03:19:28.909 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:28.909 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-12T03:19:28.909 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-12T03:19:28.910 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-12T03:19:28.910 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-09-12T03:19:28.910 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-09-12T03:19:28.910 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-12T03:19:28.910 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.910 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.910 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.910 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T03:19:28.910 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:28.910 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x559623c5cb40": { 2024-09-12T03:19:28.910 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-12T03:19:28.910 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-12T03:19:28.910 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-12T03:19:28.910 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-12T03:19:28.911 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-12T03:19:28.911 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:28.911 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-12T03:19:28.911 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-12T03:19:28.911 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-12T03:19:28.911 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-09-12T03:19:28.911 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-09-12T03:19:28.911 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-12T03:19:28.911 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:28.912 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:28.912 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:28.912 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T03:19:28.912 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T03:19:28.912 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-12T03:19:29.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:29 smithi183 ceph-mon[35535]: pgmap v832: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-12T03:19:29.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:29 smithi177 ceph-mon[35305]: pgmap v832: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-12T03:19:29.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:29 smithi116 ceph-mon[31410]: pgmap v832: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-12T03:19:29.899 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-12T03:19:29.899 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-12T03:19:29.899 DEBUG:tasks.check_counter:Getting stats from f 2024-09-12T03:19:29.899 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-12T03:19:30.144 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:19:30.653 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-12T03:19:30.653 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:19:30.653 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1419, 2024-09-12T03:19:30.653 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1412, 2024-09-12T03:19:30.654 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1172256, 2024-09-12T03:19:30.654 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 538742, 2024-09-12T03:19:30.654 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 5, 2024-09-12T03:19:30.654 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 1, 2024-09-12T03:19:30.654 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.187532074, 2024-09-12T03:19:30.654 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.064402766, 2024-09-12T03:19:30.654 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.070083702, 2024-09-12T03:19:30.654 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.045704898, 2024-09-12T03:19:30.654 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:30.654 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1411, 2024-09-12T03:19:30.654 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.089726612, 2024-09-12T03:19:30.654 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000063590 2024-09-12T03:19:30.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.655 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:30.655 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.655 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.655 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.655 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 802320, 2024-09-12T03:19:30.655 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 188624 2024-09-12T03:19:30.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.655 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:19:30.655 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 8, 2024-09-12T03:19:30.655 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 236, 2024-09-12T03:19:30.656 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 3900, 2024-09-12T03:19:30.656 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 573498, 2024-09-12T03:19:30.656 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 5, 2024-09-12T03:19:30.656 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 3, 2024-09-12T03:19:30.656 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.015136540, 2024-09-12T03:19:30.656 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.009427300, 2024-09-12T03:19:30.656 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.003466113, 2024-09-12T03:19:30.656 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000057730, 2024-09-12T03:19:30.656 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:30.656 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 235, 2024-09-12T03:19:30.656 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.018402550, 2024-09-12T03:19:30.656 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000078308 2024-09-12T03:19:30.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.657 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:30.657 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.657 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.657 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.657 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-12T03:19:30.657 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 572800 2024-09-12T03:19:30.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.657 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:19:30.657 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1105, 2024-09-12T03:19:30.657 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1107, 2024-09-12T03:19:30.658 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 290679, 2024-09-12T03:19:30.658 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 351397, 2024-09-12T03:19:30.658 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 5, 2024-09-12T03:19:30.658 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 3, 2024-09-12T03:19:30.658 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.101988169, 2024-09-12T03:19:30.658 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.041860808, 2024-09-12T03:19:30.658 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.024571020, 2024-09-12T03:19:30.658 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.025172784, 2024-09-12T03:19:30.658 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:30.658 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1105, 2024-09-12T03:19:30.658 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.047588893, 2024-09-12T03:19:30.658 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000043066 2024-09-12T03:19:30.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.659 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:30.659 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.659 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.659 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.659 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:19:30.659 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:19:30.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.659 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-09-12T03:19:30.659 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-09-12T03:19:30.659 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-09-12T03:19:30.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.660 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-09-12T03:19:30.660 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-12T03:19:30.660 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-12T03:19:30.660 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 4409, 2024-09-12T03:19:30.660 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.130462610, 2024-09-12T03:19:30.660 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000029590 2024-09-12T03:19:30.660 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T03:19:30.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.660 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-09-12T03:19:30.661 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-12T03:19:30.661 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-12T03:19:30.661 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.661 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.661 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.661 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T03:19:30.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.661 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-09-12T03:19:30.661 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 0, 2024-09-12T03:19:30.661 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 0, 2024-09-12T03:19:30.661 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-09-12T03:19:30.661 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.661 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.662 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.662 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-09-12T03:19:30.662 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 0, 2024-09-12T03:19:30.662 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-09-12T03:19:30.662 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-09-12T03:19:30.662 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 0, 2024-09-12T03:19:30.662 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-09-12T03:19:30.662 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-09-12T03:19:30.662 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 0, 2024-09-12T03:19:30.663 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 0, 2024-09-12T03:19:30.663 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 0, 2024-09-12T03:19:30.663 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-09-12T03:19:30.663 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 0, 2024-09-12T03:19:30.663 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 0, 2024-09-12T03:19:30.663 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-09-12T03:19:30.663 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-09-12T03:19:30.663 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 1, 2024-09-12T03:19:30.663 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 0, 2024-09-12T03:19:30.663 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 0, 2024-09-12T03:19:30.663 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2024-09-12T03:19:30.663 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 0, 2024-09-12T03:19:30.664 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:19:30.664 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-09-12T03:19:30.664 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-09-12T03:19:30.664 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-09-12T03:19:30.664 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-09-12T03:19:30.664 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2024-09-12T03:19:30.664 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-09-12T03:19:30.664 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-09-12T03:19:30.664 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-09-12T03:19:30.665 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-09-12T03:19:30.665 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:19:30.665 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-09-12T03:19:30.665 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 0, 2024-09-12T03:19:30.665 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 0, 2024-09-12T03:19:30.665 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 0, 2024-09-12T03:19:30.665 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:19:30.665 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-09-12T03:19:30.665 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:19:30.665 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:19:30.665 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:19:30.666 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:19:30.666 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:19:30.666 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-09-12T03:19:30.666 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:19:30.666 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:19:30.666 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:19:30.666 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:19:30.666 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:19:30.666 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:19:30.666 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 0, 2024-09-12T03:19:30.666 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:19:30.666 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 0, 2024-09-12T03:19:30.666 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 0 2024-09-12T03:19:30.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.667 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-09-12T03:19:30.667 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-09-12T03:19:30.667 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-09-12T03:19:30.667 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-09-12T03:19:30.667 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 0, 2024-09-12T03:19:30.667 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 0, 2024-09-12T03:19:30.667 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-09-12T03:19:30.667 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-09-12T03:19:30.667 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:19:30.667 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 0, 2024-09-12T03:19:30.667 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 0, 2024-09-12T03:19:30.667 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-09-12T03:19:30.667 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-09-12T03:19:30.668 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-09-12T03:19:30.668 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:19:30.668 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:19:30.668 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-09-12T03:19:30.668 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-09-12T03:19:30.668 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:19:30.668 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:19:30.668 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:19:30.668 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2024-09-12T03:19:30.668 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-09-12T03:19:30.668 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:19:30.668 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-09-12T03:19:30.668 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-09-12T03:19:30.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.669 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-09-12T03:19:30.669 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-09-12T03:19:30.669 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 0, 2024-09-12T03:19:30.669 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-09-12T03:19:30.669 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-09-12T03:19:30.669 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 2, 2024-09-12T03:19:30.669 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-09-12T03:19:30.669 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-09-12T03:19:30.669 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 0, 2024-09-12T03:19:30.669 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-09-12T03:19:30.669 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-09-12T03:19:30.669 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 2, 2024-09-12T03:19:30.669 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2024-09-12T03:19:30.669 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-09-12T03:19:30.670 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-09-12T03:19:30.670 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2024-09-12T03:19:30.670 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 4194892, 2024-09-12T03:19:30.670 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 4194352, 2024-09-12T03:19:30.670 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-09-12T03:19:30.670 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.670 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.670 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.670 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 2 2024-09-12T03:19:30.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.670 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-09-12T03:19:30.670 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 3, 2024-09-12T03:19:30.671 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 3, 2024-09-12T03:19:30.671 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 0, 2024-09-12T03:19:30.671 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 1, 2024-09-12T03:19:30.671 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 1, 2024-09-12T03:19:30.671 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 0, 2024-09-12T03:19:30.671 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 0, 2024-09-12T03:19:30.671 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 0, 2024-09-12T03:19:30.671 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 0, 2024-09-12T03:19:30.671 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-09-12T03:19:30.671 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 0, 2024-09-12T03:19:30.671 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 0, 2024-09-12T03:19:30.671 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 48352, 2024-09-12T03:19:30.671 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 215328 2024-09-12T03:19:30.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.672 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-09-12T03:19:30.672 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 0, 2024-09-12T03:19:30.672 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 0, 2024-09-12T03:19:30.672 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 0, 2024-09-12T03:19:30.672 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 0, 2024-09-12T03:19:30.672 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 0, 2024-09-12T03:19:30.672 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-09-12T03:19:30.672 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.672 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.672 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.672 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-09-12T03:19:30.672 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.672 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.673 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.673 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-09-12T03:19:30.673 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.673 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.673 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.673 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-09-12T03:19:30.673 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.673 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.673 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.673 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-09-12T03:19:30.673 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.674 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.674 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.674 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-09-12T03:19:30.674 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.674 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.674 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.674 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-09-12T03:19:30.674 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.675 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.675 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.675 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-09-12T03:19:30.675 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.675 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.675 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.675 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-09-12T03:19:30.675 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.675 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.675 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.675 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-09-12T03:19:30.676 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.676 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.676 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.676 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-09-12T03:19:30.676 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.676 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.676 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.676 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-09-12T03:19:30.676 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.677 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.677 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.677 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-09-12T03:19:30.677 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.677 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.678 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.678 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-09-12T03:19:30.678 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.678 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.678 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.678 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-09-12T03:19:30.678 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.678 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.678 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.678 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-09-12T03:19:30.679 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.679 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.679 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.679 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-09-12T03:19:30.679 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.679 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.679 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.679 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-09-12T03:19:30.679 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.679 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.679 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.680 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-09-12T03:19:30.680 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.680 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.680 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.680 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-09-12T03:19:30.680 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.680 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.680 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.680 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-09-12T03:19:30.680 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.680 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.680 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.681 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-09-12T03:19:30.681 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.681 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.681 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.681 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-09-12T03:19:30.681 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.681 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.681 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.681 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-09-12T03:19:30.681 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.682 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.682 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.682 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-09-12T03:19:30.682 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.682 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.682 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.682 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-09-12T03:19:30.682 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.682 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.682 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.682 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-09-12T03:19:30.683 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.683 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.683 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.683 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-09-12T03:19:30.683 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.683 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.683 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.683 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-09-12T03:19:30.683 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.683 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.683 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.684 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:19:30.684 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:19:30.684 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-09-12T03:19:30.684 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.684 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.684 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.684 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T03:19:30.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.684 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-09-12T03:19:30.684 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 0, 2024-09-12T03:19:30.684 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 0, 2024-09-12T03:19:30.684 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 0, 2024-09-12T03:19:30.685 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 0, 2024-09-12T03:19:30.685 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-09-12T03:19:30.685 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 0, 2024-09-12T03:19:30.685 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 0, 2024-09-12T03:19:30.685 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 0, 2024-09-12T03:19:30.685 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-09-12T03:19:30.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.685 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-09-12T03:19:30.685 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:19:30.685 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-09-12T03:19:30.685 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:19:30.685 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:19:30.685 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:19:30.685 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:19:30.686 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:19:30.686 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:19:30.686 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:19:30.686 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:19:30.686 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:19:30.686 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:19:30.686 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:19:30.686 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:19:30.686 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:19:30.686 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-09-12T03:19:30.686 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:19:30.686 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-09-12T03:19:30.686 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:19:30.686 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:19:30.687 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:19:30.687 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:19:30.687 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:19:30.687 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:19:30.687 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:19:30.687 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-09-12T03:19:30.687 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:19:30.687 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:19:30.687 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:19:30.687 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-09-12T03:19:30.687 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-09-12T03:19:30.687 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-09-12T03:19:30.687 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:19:30.687 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:19:30.688 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:19:30.688 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:19:30.688 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 182087, 2024-09-12T03:19:30.688 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 48, 2024-09-12T03:19:30.688 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:19:30.688 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-09-12T03:19:30.688 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-09-12T03:19:30.688 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-09-12T03:19:30.688 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:19:30.688 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-09-12T03:19:30.688 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:19:30.688 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-09-12T03:19:30.688 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 7400, 2024-09-12T03:19:30.688 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 152, 2024-09-12T03:19:30.689 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:19:30.689 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:19:30.689 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-09-12T03:19:30.689 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-09-12T03:19:30.689 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 8512, 2024-09-12T03:19:30.689 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 13, 2024-09-12T03:19:30.689 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-09-12T03:19:30.689 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-09-12T03:19:30.689 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-09-12T03:19:30.689 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-09-12T03:19:30.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.689 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-09-12T03:19:30.689 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-09-12T03:19:30.690 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-09-12T03:19:30.690 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 2208, 2024-09-12T03:19:30.690 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 21, 2024-09-12T03:19:30.690 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 3, 2024-09-12T03:19:30.690 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 2208, 2024-09-12T03:19:30.690 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-09-12T03:19:30.690 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2208, 2024-09-12T03:19:30.690 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1.140968455, 2024-09-12T03:19:30.690 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000516742 2024-09-12T03:19:30.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.690 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-09-12T03:19:30.690 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-09-12T03:19:30.690 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2208, 2024-09-12T03:19:30.690 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 2209 2024-09-12T03:19:30.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.691 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 2208, 2024-09-12T03:19:30.691 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 2208, 2024-09-12T03:19:30.691 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 0, 2024-09-12T03:19:30.691 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-09-12T03:19:30.691 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-09-12T03:19:30.691 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 1101, 2024-09-12T03:19:30.691 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 0, 2024-09-12T03:19:30.691 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 1105, 2024-09-12T03:19:30.691 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 0, 2024-09-12T03:19:30.691 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 0, 2024-09-12T03:19:30.691 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-09-12T03:19:30.691 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-09-12T03:19:30.691 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-09-12T03:19:30.691 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-09-12T03:19:30.692 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 0, 2024-09-12T03:19:30.692 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-09-12T03:19:30.692 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-09-12T03:19:30.692 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-09-12T03:19:30.692 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 1, 2024-09-12T03:19:30.692 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 0, 2024-09-12T03:19:30.692 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:19:30.692 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-09-12T03:19:30.692 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:19:30.692 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-09-12T03:19:30.692 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-09-12T03:19:30.692 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-09-12T03:19:30.692 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:19:30.693 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-09-12T03:19:30.693 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:19:30.693 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-09-12T03:19:30.693 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-09-12T03:19:30.693 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-09-12T03:19:30.693 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-09-12T03:19:30.693 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-09-12T03:19:30.693 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-09-12T03:19:30.693 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-09-12T03:19:30.693 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-09-12T03:19:30.693 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-09-12T03:19:30.693 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-09-12T03:19:30.693 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-09-12T03:19:30.693 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-09-12T03:19:30.694 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-09-12T03:19:30.694 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-09-12T03:19:30.694 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-09-12T03:19:30.694 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-09-12T03:19:30.694 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-09-12T03:19:30.694 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 80, 2024-09-12T03:19:30.694 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-09-12T03:19:30.694 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 2, 2024-09-12T03:19:30.694 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 4, 2024-09-12T03:19:30.694 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 5, 2024-09-12T03:19:30.694 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 1, 2024-09-12T03:19:30.694 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-09-12T03:19:30.694 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 0, 2024-09-12T03:19:30.694 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 2, 2024-09-12T03:19:30.695 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 0 2024-09-12T03:19:30.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.695 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-09-12T03:19:30.695 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 0, 2024-09-12T03:19:30.695 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:19:30.695 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-09-12T03:19:30.695 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-09-12T03:19:30.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.695 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-09-12T03:19:30.695 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2024-09-12T03:19:30.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.695 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-09-12T03:19:30.695 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-09-12T03:19:30.695 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:19:30.696 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-09-12T03:19:30.696 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 0, 2024-09-12T03:19:30.696 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 0, 2024-09-12T03:19:30.696 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 0, 2024-09-12T03:19:30.696 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-09-12T03:19:30.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.696 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:19:30.696 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-12T03:19:30.696 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-12T03:19:30.696 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-12T03:19:30.696 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 2528, 2024-09-12T03:19:30.696 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 1219905, 2024-09-12T03:19:30.696 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:30.697 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 2528, 2024-09-12T03:19:30.697 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-12T03:19:30.697 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-12T03:19:30.697 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 2528, 2024-09-12T03:19:30.697 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 1219905, 2024-09-12T03:19:30.697 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-12T03:19:30.697 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.697 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.697 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.697 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T03:19:30.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.698 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-09-12T03:19:30.698 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-12T03:19:30.698 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-12T03:19:30.698 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-12T03:19:30.698 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-12T03:19:30.698 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-12T03:19:30.698 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:30.698 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-12T03:19:30.698 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 2208, 2024-09-12T03:19:30.698 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 593, 2024-09-12T03:19:30.698 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 2208, 2024-09-12T03:19:30.698 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 593, 2024-09-12T03:19:30.699 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-12T03:19:30.699 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.699 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.699 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.699 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T03:19:30.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.699 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-09-12T03:19:30.699 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-12T03:19:30.699 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-09-12T03:19:30.699 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-12T03:19:30.699 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-12T03:19:30.699 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-12T03:19:30.699 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:30.699 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-12T03:19:30.700 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 2208, 2024-09-12T03:19:30.700 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 2208, 2024-09-12T03:19:30.700 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 2208, 2024-09-12T03:19:30.700 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 2208, 2024-09-12T03:19:30.700 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-12T03:19:30.700 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.700 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.700 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.700 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T03:19:30.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.700 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:19:30.700 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-12T03:19:30.700 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-12T03:19:30.700 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-12T03:19:30.701 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-12T03:19:30.701 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-12T03:19:30.701 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:30.701 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-12T03:19:30.701 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-12T03:19:30.701 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-12T03:19:30.701 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-09-12T03:19:30.701 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-09-12T03:19:30.701 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-12T03:19:30.701 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.701 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.701 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.701 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T03:19:30.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:30.702 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x555e2bc4cb40": { 2024-09-12T03:19:30.702 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-12T03:19:30.702 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-12T03:19:30.702 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-12T03:19:30.702 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-12T03:19:30.702 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-12T03:19:30.702 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:30.702 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-12T03:19:30.702 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-12T03:19:30.702 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-12T03:19:30.702 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-09-12T03:19:30.702 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-09-12T03:19:30.702 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-12T03:19:30.702 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:30.703 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:30.703 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:30.703 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T03:19:30.703 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T03:19:30.703 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-12T03:19:31.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:31 smithi183 ceph-mon[35535]: pgmap v833: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-12T03:19:31.450 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-12T03:19:31.450 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-12T03:19:31.450 DEBUG:tasks.check_counter:Getting stats from i 2024-09-12T03:19:31.451 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-12T03:19:31.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:31 smithi177 ceph-mon[35305]: pgmap v833: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-12T03:19:31.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:31 smithi116 ceph-mon[31410]: pgmap v833: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-12T03:19:31.695 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:19:32.207 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-12T03:19:32.207 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:19:32.207 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 83, 2024-09-12T03:19:32.207 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 83, 2024-09-12T03:19:32.207 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 16499, 2024-09-12T03:19:32.208 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 765508, 2024-09-12T03:19:32.208 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 72, 2024-09-12T03:19:32.208 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 2, 2024-09-12T03:19:32.208 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.032040291, 2024-09-12T03:19:32.208 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.001801242, 2024-09-12T03:19:32.208 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.023064413, 2024-09-12T03:19:32.208 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000312355, 2024-09-12T03:19:32.208 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:32.208 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 82, 2024-09-12T03:19:32.208 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.021443457, 2024-09-12T03:19:32.208 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000261505 2024-09-12T03:19:32.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.208 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:32.209 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 62, 2024-09-12T03:19:32.209 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000004557, 2024-09-12T03:19:32.209 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000073 2024-09-12T03:19:32.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.209 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T03:19:32.209 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T03:19:32.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.209 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:19:32.209 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 890, 2024-09-12T03:19:32.209 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1127, 2024-09-12T03:19:32.209 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 923100, 2024-09-12T03:19:32.209 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 476452, 2024-09-12T03:19:32.209 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 22, 2024-09-12T03:19:32.210 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 8, 2024-09-12T03:19:32.210 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.127337482, 2024-09-12T03:19:32.210 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.053424746, 2024-09-12T03:19:32.210 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.070274362, 2024-09-12T03:19:32.210 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.006728281, 2024-09-12T03:19:32.210 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:32.210 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1126, 2024-09-12T03:19:32.210 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.068862962, 2024-09-12T03:19:32.210 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000061157 2024-09-12T03:19:32.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.210 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:32.210 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1286, 2024-09-12T03:19:32.210 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000332827, 2024-09-12T03:19:32.211 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000258 2024-09-12T03:19:32.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.211 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 815216, 2024-09-12T03:19:32.211 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 191984 2024-09-12T03:19:32.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.211 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:19:32.211 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 10, 2024-09-12T03:19:32.211 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 239, 2024-09-12T03:19:32.211 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 2547, 2024-09-12T03:19:32.211 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 582851, 2024-09-12T03:19:32.211 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 9, 2024-09-12T03:19:32.211 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 4, 2024-09-12T03:19:32.211 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.015723624, 2024-09-12T03:19:32.211 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.009973218, 2024-09-12T03:19:32.212 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.002914772, 2024-09-12T03:19:32.212 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000196168, 2024-09-12T03:19:32.212 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:32.212 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 237, 2024-09-12T03:19:32.212 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.020811435, 2024-09-12T03:19:32.212 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000087811 2024-09-12T03:19:32.212 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.212 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:32.212 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.212 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.212 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.212 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.212 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T03:19:32.213 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 573792 2024-09-12T03:19:32.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.213 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-09-12T03:19:32.213 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-09-12T03:19:32.213 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-09-12T03:19:32.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.213 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-09-12T03:19:32.213 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-12T03:19:32.213 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-12T03:19:32.213 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 37, 2024-09-12T03:19:32.213 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001271340, 2024-09-12T03:19:32.213 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000034360 2024-09-12T03:19:32.213 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T03:19:32.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.214 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-09-12T03:19:32.214 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-12T03:19:32.214 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-12T03:19:32.214 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2, 2024-09-12T03:19:32.214 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000012777, 2024-09-12T03:19:32.214 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000006388 2024-09-12T03:19:32.214 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T03:19:32.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.214 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-09-12T03:19:32.214 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 0, 2024-09-12T03:19:32.214 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 0, 2024-09-12T03:19:32.214 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-09-12T03:19:32.214 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.215 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.215 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.215 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-09-12T03:19:32.215 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 0, 2024-09-12T03:19:32.215 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-09-12T03:19:32.215 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-09-12T03:19:32.215 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 11, 2024-09-12T03:19:32.215 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-09-12T03:19:32.215 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-09-12T03:19:32.215 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 10, 2024-09-12T03:19:32.215 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 0, 2024-09-12T03:19:32.215 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 10, 2024-09-12T03:19:32.215 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-09-12T03:19:32.216 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 10, 2024-09-12T03:19:32.216 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 0, 2024-09-12T03:19:32.216 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-09-12T03:19:32.216 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-09-12T03:19:32.216 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 1, 2024-09-12T03:19:32.216 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 28, 2024-09-12T03:19:32.216 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 28, 2024-09-12T03:19:32.216 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2024-09-12T03:19:32.216 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 0, 2024-09-12T03:19:32.216 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:19:32.216 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-09-12T03:19:32.216 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-09-12T03:19:32.216 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-09-12T03:19:32.217 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-09-12T03:19:32.217 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2024-09-12T03:19:32.217 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-09-12T03:19:32.217 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-09-12T03:19:32.217 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-09-12T03:19:32.217 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-09-12T03:19:32.217 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:19:32.217 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-09-12T03:19:32.217 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 5, 2024-09-12T03:19:32.217 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 298, 2024-09-12T03:19:32.217 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 0, 2024-09-12T03:19:32.217 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:19:32.217 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-09-12T03:19:32.217 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:19:32.218 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:19:32.218 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:19:32.218 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:19:32.218 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:19:32.218 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-09-12T03:19:32.218 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:19:32.218 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:19:32.218 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:19:32.218 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:19:32.218 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:19:32.218 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:19:32.218 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 0, 2024-09-12T03:19:32.218 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:19:32.218 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 0, 2024-09-12T03:19:32.219 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 0 2024-09-12T03:19:32.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.219 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-09-12T03:19:32.219 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-09-12T03:19:32.219 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-09-12T03:19:32.219 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-09-12T03:19:32.219 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 1, 2024-09-12T03:19:32.219 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 0, 2024-09-12T03:19:32.219 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-09-12T03:19:32.219 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-09-12T03:19:32.219 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:19:32.219 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 0, 2024-09-12T03:19:32.219 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 0, 2024-09-12T03:19:32.219 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-09-12T03:19:32.220 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-09-12T03:19:32.220 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-09-12T03:19:32.220 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:19:32.220 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:19:32.220 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-09-12T03:19:32.220 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-09-12T03:19:32.220 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 14, 2024-09-12T03:19:32.220 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 14, 2024-09-12T03:19:32.220 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:19:32.220 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2024-09-12T03:19:32.220 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-09-12T03:19:32.220 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:19:32.220 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-09-12T03:19:32.221 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-09-12T03:19:32.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.221 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-09-12T03:19:32.221 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-09-12T03:19:32.221 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 2, 2024-09-12T03:19:32.221 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-09-12T03:19:32.221 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-09-12T03:19:32.221 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 2, 2024-09-12T03:19:32.221 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-09-12T03:19:32.221 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-09-12T03:19:32.221 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 2, 2024-09-12T03:19:32.221 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-09-12T03:19:32.221 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-09-12T03:19:32.221 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 2, 2024-09-12T03:19:32.222 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2024-09-12T03:19:32.222 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-09-12T03:19:32.222 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-09-12T03:19:32.222 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2024-09-12T03:19:32.222 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 4194892, 2024-09-12T03:19:32.222 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 0, 2024-09-12T03:19:32.222 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-09-12T03:19:32.222 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1, 2024-09-12T03:19:32.222 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001176262, 2024-09-12T03:19:32.222 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001176262 2024-09-12T03:19:32.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.222 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 0 2024-09-12T03:19:32.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.222 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-09-12T03:19:32.223 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 13, 2024-09-12T03:19:32.223 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 13, 2024-09-12T03:19:32.223 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 0, 2024-09-12T03:19:32.223 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 11, 2024-09-12T03:19:32.223 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 12, 2024-09-12T03:19:32.223 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 1, 2024-09-12T03:19:32.223 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 10, 2024-09-12T03:19:32.223 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 10, 2024-09-12T03:19:32.223 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 0, 2024-09-12T03:19:32.223 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-09-12T03:19:32.223 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 0, 2024-09-12T03:19:32.223 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 0, 2024-09-12T03:19:32.223 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 50752, 2024-09-12T03:19:32.223 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 223520 2024-09-12T03:19:32.224 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.224 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-09-12T03:19:32.224 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 0, 2024-09-12T03:19:32.224 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 0, 2024-09-12T03:19:32.224 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 0, 2024-09-12T03:19:32.224 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 0, 2024-09-12T03:19:32.224 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 0, 2024-09-12T03:19:32.224 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-09-12T03:19:32.224 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.224 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.224 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.224 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.224 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-09-12T03:19:32.225 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.225 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.225 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.225 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-09-12T03:19:32.225 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.225 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.225 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.225 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-09-12T03:19:32.225 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.225 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.225 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.226 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-09-12T03:19:32.226 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.226 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.226 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.226 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-09-12T03:19:32.226 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.226 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.226 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.226 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-09-12T03:19:32.226 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.226 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.226 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.227 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-09-12T03:19:32.227 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.227 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.227 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.227 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-09-12T03:19:32.227 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.227 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.227 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.227 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-09-12T03:19:32.227 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.228 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.228 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.228 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-09-12T03:19:32.228 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.228 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.228 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.228 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-09-12T03:19:32.228 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.229 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.229 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.229 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.230 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-09-12T03:19:32.230 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.230 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.230 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.230 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.230 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-09-12T03:19:32.230 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.230 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.230 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.230 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.230 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-09-12T03:19:32.230 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.230 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.231 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.231 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-09-12T03:19:32.231 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.231 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.231 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.231 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-09-12T03:19:32.231 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.231 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.231 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.231 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-09-12T03:19:32.232 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.232 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.232 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.232 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-09-12T03:19:32.232 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.232 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.232 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.232 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-09-12T03:19:32.232 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.232 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.232 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.233 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-09-12T03:19:32.233 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.233 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.233 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.233 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-09-12T03:19:32.233 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.233 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.233 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.233 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-09-12T03:19:32.234 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.234 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.234 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.234 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.234 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-09-12T03:19:32.234 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.234 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.234 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.234 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.234 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-09-12T03:19:32.234 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.235 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.235 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.235 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-09-12T03:19:32.235 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.235 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.235 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.235 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-09-12T03:19:32.235 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.235 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.235 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.236 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-09-12T03:19:32.236 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.236 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.236 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.236 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-09-12T03:19:32.236 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.236 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.236 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.236 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:19:32.236 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:19:32.237 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-09-12T03:19:32.237 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.237 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.237 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.237 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T03:19:32.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.237 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-09-12T03:19:32.237 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 0, 2024-09-12T03:19:32.237 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 0, 2024-09-12T03:19:32.237 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 0, 2024-09-12T03:19:32.237 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 0, 2024-09-12T03:19:32.237 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-09-12T03:19:32.237 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 0, 2024-09-12T03:19:32.238 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 0, 2024-09-12T03:19:32.238 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 0, 2024-09-12T03:19:32.238 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-09-12T03:19:32.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.238 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-09-12T03:19:32.238 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:19:32.238 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-09-12T03:19:32.238 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:19:32.238 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:19:32.238 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:19:32.238 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:19:32.238 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:19:32.238 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:19:32.238 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:19:32.239 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:19:32.239 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:19:32.239 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:19:32.239 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:19:32.239 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:19:32.239 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:19:32.239 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-09-12T03:19:32.239 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:19:32.239 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-09-12T03:19:32.239 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:19:32.239 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:19:32.239 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:19:32.239 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:19:32.240 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:19:32.240 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:19:32.240 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:19:32.240 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-09-12T03:19:32.240 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:19:32.240 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:19:32.240 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:19:32.240 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-09-12T03:19:32.240 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-09-12T03:19:32.240 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-09-12T03:19:32.240 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:19:32.240 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:19:32.240 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:19:32.240 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:19:32.241 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 239111, 2024-09-12T03:19:32.241 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 81, 2024-09-12T03:19:32.241 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:19:32.241 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-09-12T03:19:32.241 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-09-12T03:19:32.241 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-09-12T03:19:32.241 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:19:32.241 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-09-12T03:19:32.241 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:19:32.241 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-09-12T03:19:32.241 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 7400, 2024-09-12T03:19:32.241 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 152, 2024-09-12T03:19:32.241 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:19:32.242 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:19:32.242 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-09-12T03:19:32.242 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-09-12T03:19:32.242 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 50872, 2024-09-12T03:19:32.242 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 198, 2024-09-12T03:19:32.242 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-09-12T03:19:32.242 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-09-12T03:19:32.242 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-09-12T03:19:32.242 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-09-12T03:19:32.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.242 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-09-12T03:19:32.242 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-09-12T03:19:32.242 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-09-12T03:19:32.242 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 35, 2024-09-12T03:19:32.243 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 10710, 2024-09-12T03:19:32.243 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 28, 2024-09-12T03:19:32.243 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 35, 2024-09-12T03:19:32.243 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-09-12T03:19:32.243 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 35, 2024-09-12T03:19:32.243 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.197994530, 2024-09-12T03:19:32.243 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.005656986 2024-09-12T03:19:32.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.243 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-09-12T03:19:32.243 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-09-12T03:19:32.243 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 35, 2024-09-12T03:19:32.243 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 57 2024-09-12T03:19:32.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.243 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 35, 2024-09-12T03:19:32.244 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 0, 2024-09-12T03:19:32.244 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 35, 2024-09-12T03:19:32.244 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-09-12T03:19:32.244 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-09-12T03:19:32.244 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 0, 2024-09-12T03:19:32.244 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 10, 2024-09-12T03:19:32.244 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 0, 2024-09-12T03:19:32.244 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 1, 2024-09-12T03:19:32.244 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 5, 2024-09-12T03:19:32.244 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-09-12T03:19:32.244 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-09-12T03:19:32.244 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-09-12T03:19:32.244 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-09-12T03:19:32.245 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 6, 2024-09-12T03:19:32.245 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-09-12T03:19:32.245 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-09-12T03:19:32.245 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-09-12T03:19:32.245 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 0, 2024-09-12T03:19:32.245 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 20, 2024-09-12T03:19:32.245 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:19:32.245 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-09-12T03:19:32.245 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:19:32.245 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-09-12T03:19:32.245 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-09-12T03:19:32.245 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-09-12T03:19:32.246 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:19:32.246 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-09-12T03:19:32.246 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:19:32.246 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-09-12T03:19:32.246 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-09-12T03:19:32.246 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-09-12T03:19:32.246 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-09-12T03:19:32.246 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-09-12T03:19:32.246 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-09-12T03:19:32.246 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-09-12T03:19:32.246 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-09-12T03:19:32.246 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-09-12T03:19:32.246 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-09-12T03:19:32.246 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-09-12T03:19:32.247 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-09-12T03:19:32.247 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-09-12T03:19:32.247 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-09-12T03:19:32.247 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-09-12T03:19:32.247 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-09-12T03:19:32.247 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-09-12T03:19:32.247 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 80, 2024-09-12T03:19:32.247 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-09-12T03:19:32.247 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 2, 2024-09-12T03:19:32.248 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 9, 2024-09-12T03:19:32.248 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 15, 2024-09-12T03:19:32.248 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 6, 2024-09-12T03:19:32.248 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-09-12T03:19:32.248 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 14, 2024-09-12T03:19:32.248 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 0, 2024-09-12T03:19:32.248 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 1 2024-09-12T03:19:32.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.248 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-09-12T03:19:32.248 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 1, 2024-09-12T03:19:32.248 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:19:32.248 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-09-12T03:19:32.248 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-09-12T03:19:32.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.249 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-09-12T03:19:32.249 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 28 2024-09-12T03:19:32.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.249 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-09-12T03:19:32.249 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-09-12T03:19:32.249 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:19:32.249 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-09-12T03:19:32.249 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 0, 2024-09-12T03:19:32.249 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 0, 2024-09-12T03:19:32.249 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 0, 2024-09-12T03:19:32.249 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-09-12T03:19:32.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.250 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:19:32.250 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-12T03:19:32.250 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-12T03:19:32.250 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-12T03:19:32.250 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 979, 2024-09-12T03:19:32.250 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 833342, 2024-09-12T03:19:32.250 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:32.250 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 979, 2024-09-12T03:19:32.250 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-12T03:19:32.250 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-12T03:19:32.250 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 979, 2024-09-12T03:19:32.250 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 833342, 2024-09-12T03:19:32.250 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-12T03:19:32.250 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.251 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.251 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.251 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T03:19:32.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.251 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-09-12T03:19:32.251 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-12T03:19:32.251 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-12T03:19:32.251 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-12T03:19:32.251 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-12T03:19:32.251 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-12T03:19:32.251 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:32.251 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-12T03:19:32.251 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 35, 2024-09-12T03:19:32.251 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 10710, 2024-09-12T03:19:32.252 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 35, 2024-09-12T03:19:32.252 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 10710, 2024-09-12T03:19:32.252 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-12T03:19:32.252 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.252 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.252 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.252 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T03:19:32.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.252 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-09-12T03:19:32.252 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-12T03:19:32.252 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-09-12T03:19:32.252 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-12T03:19:32.252 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-12T03:19:32.253 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-12T03:19:32.253 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:32.253 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-12T03:19:32.253 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 35, 2024-09-12T03:19:32.253 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 35, 2024-09-12T03:19:32.253 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 35, 2024-09-12T03:19:32.253 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 35, 2024-09-12T03:19:32.253 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-12T03:19:32.253 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.253 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.253 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.253 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T03:19:32.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.254 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:19:32.254 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-12T03:19:32.254 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-12T03:19:32.254 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-12T03:19:32.254 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-12T03:19:32.254 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-12T03:19:32.254 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:32.254 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-12T03:19:32.254 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-12T03:19:32.254 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-12T03:19:32.254 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-09-12T03:19:32.254 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-09-12T03:19:32.254 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-12T03:19:32.254 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.255 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.255 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.255 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T03:19:32.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:32.255 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x55f783f0e960": { 2024-09-12T03:19:32.255 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-12T03:19:32.255 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-12T03:19:32.255 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-12T03:19:32.255 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 2, 2024-09-12T03:19:32.255 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 588, 2024-09-12T03:19:32.255 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:32.255 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 2, 2024-09-12T03:19:32.255 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-12T03:19:32.255 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-12T03:19:32.256 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 1, 2024-09-12T03:19:32.256 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 588, 2024-09-12T03:19:32.256 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-12T03:19:32.256 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:32.256 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:32.256 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:32.256 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T03:19:32.256 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T03:19:32.256 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-12T03:19:32.997 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-12T03:19:32.997 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-12T03:19:32.997 DEBUG:tasks.check_counter:Getting stats from l 2024-09-12T03:19:32.997 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-12T03:19:33.246 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:19:33.418 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:33 smithi183 ceph-mon[35535]: pgmap v834: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-12T03:19:33.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:33 smithi177 ceph-mon[35305]: pgmap v834: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-12T03:19:33.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:33 smithi116 ceph-mon[31410]: pgmap v834: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-12T03:19:33.788 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-12T03:19:33.789 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:19:33.789 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 0, 2024-09-12T03:19:33.789 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 2, 2024-09-12T03:19:33.789 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 0, 2024-09-12T03:19:33.789 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 0, 2024-09-12T03:19:33.789 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 4, 2024-09-12T03:19:33.789 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 0, 2024-09-12T03:19:33.789 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.000998962, 2024-09-12T03:19:33.789 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T03:19:33.789 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.000149331, 2024-09-12T03:19:33.789 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:19:33.789 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:33.790 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:33.790 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:33.790 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:33.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:33.790 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:33.790 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:33.790 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:33.790 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:33.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:33.790 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:19:33.790 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:19:33.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:33.791 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:19:33.791 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 6, 2024-09-12T03:19:33.791 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 238, 2024-09-12T03:19:33.791 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 3344, 2024-09-12T03:19:33.791 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 199424, 2024-09-12T03:19:33.791 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 4, 2024-09-12T03:19:33.791 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 2, 2024-09-12T03:19:33.791 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.017590449, 2024-09-12T03:19:33.791 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.011601680, 2024-09-12T03:19:33.791 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.003039397, 2024-09-12T03:19:33.791 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:19:33.791 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:33.791 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 237, 2024-09-12T03:19:33.792 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.021257997, 2024-09-12T03:19:33.792 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000089696 2024-09-12T03:19:33.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:33.792 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:33.792 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:33.792 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:33.792 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:33.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:33.792 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-12T03:19:33.792 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 199424 2024-09-12T03:19:33.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:33.792 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:19:33.792 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 314, 2024-09-12T03:19:33.793 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 309, 2024-09-12T03:19:33.793 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 780960, 2024-09-12T03:19:33.793 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 190640, 2024-09-12T03:19:33.793 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 2, 2024-09-12T03:19:33.793 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 1, 2024-09-12T03:19:33.793 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.048843547, 2024-09-12T03:19:33.793 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.022736434, 2024-09-12T03:19:33.793 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.033408256, 2024-09-12T03:19:33.793 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:19:33.793 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:19:33.793 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 308, 2024-09-12T03:19:33.793 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.019414187, 2024-09-12T03:19:33.793 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000063033 2024-09-12T03:19:33.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:33.794 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:19:33.794 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:33.794 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:33.794 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:33.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:33.794 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 780960, 2024-09-12T03:19:33.794 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 190640 2024-09-12T03:19:33.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:33.794 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-09-12T03:19:33.794 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 0, 2024-09-12T03:19:33.794 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-09-12T03:19:33.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:33.794 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-09-12T03:19:33.795 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:19:33.795 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-09-12T03:19:33.795 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:19:33.795 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:19:33.795 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:19:33.795 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:19:33.795 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:19:33.795 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:19:33.795 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:19:33.795 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:19:33.795 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:19:33.795 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:19:33.795 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:19:33.796 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:19:33.796 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:19:33.796 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-09-12T03:19:33.796 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:19:33.796 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-09-12T03:19:33.796 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:19:33.796 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:19:33.796 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:19:33.796 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:19:33.796 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:19:33.796 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:19:33.796 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:19:33.796 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-09-12T03:19:33.797 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:19:33.797 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:19:33.797 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:19:33.797 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-09-12T03:19:33.797 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-09-12T03:19:33.797 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-09-12T03:19:33.797 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:19:33.797 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:19:33.797 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:19:33.797 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:19:33.797 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 148704, 2024-09-12T03:19:33.797 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 29, 2024-09-12T03:19:33.797 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:19:33.798 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-09-12T03:19:33.798 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-09-12T03:19:33.798 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-09-12T03:19:33.798 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:19:33.798 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-09-12T03:19:33.798 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:19:33.798 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-09-12T03:19:33.798 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 0, 2024-09-12T03:19:33.798 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 0, 2024-09-12T03:19:33.798 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:19:33.798 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:19:33.798 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-09-12T03:19:33.798 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-09-12T03:19:33.798 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 928, 2024-09-12T03:19:33.799 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 2, 2024-09-12T03:19:33.799 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-09-12T03:19:33.799 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-09-12T03:19:33.799 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-09-12T03:19:33.799 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-09-12T03:19:33.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-12T03:19:33.799 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:19:33.799 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-12T03:19:33.799 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-12T03:19:33.799 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-12T03:19:33.799 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 316, 2024-09-12T03:19:33.799 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 736615, 2024-09-12T03:19:33.799 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-12T03:19:33.799 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 316, 2024-09-12T03:19:33.800 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-12T03:19:33.800 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-12T03:19:33.800 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 316, 2024-09-12T03:19:33.800 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 736615, 2024-09-12T03:19:33.800 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-12T03:19:33.800 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-12T03:19:33.800 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-12T03:19:33.800 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-12T03:19:33.800 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T03:19:33.800 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-12T03:19:33.800 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-12T03:19:34.495 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-09-12T03:19:34.495 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-09-12T03:19:34.495 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-12T03:19:34.505 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-12T03:19:34.506 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-12T03:19:34.506 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:19:34.527 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-12T03:19:34.527 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:19:34.588 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-12T03:19:34.883 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:35.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:35 smithi116 ceph-mon[31410]: pgmap v835: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-12T03:19:35.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:35 smithi183 ceph-mon[35535]: pgmap v835: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-12T03:19:35.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.390+0000 7fa005ac1640 1 -- 172.21.15.116:0/2781754054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa000100a80 msgr2=0x7fa000100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:35.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.390+0000 7fa005ac1640 1 --2- 172.21.15.116:0/2781754054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa000100a80 0x7fa000100ee0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8009930 tx=0x7f9fe802a420 comp rx=0 tx=0).stop 2024-09-12T03:19:35.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.390+0000 7fa005ac1640 1 -- 172.21.15.116:0/2781754054 shutdown_connections 2024-09-12T03:19:35.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.390+0000 7fa005ac1640 1 --2- 172.21.15.116:0/2781754054 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa000101420 0x7fa00010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.390+0000 7fa005ac1640 1 --2- 172.21.15.116:0/2781754054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa000100a80 0x7fa000100ee0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.390+0000 7fa005ac1640 1 --2- 172.21.15.116:0/2781754054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa000106a80 0x7fa000106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.390+0000 7fa005ac1640 1 -- 172.21.15.116:0/2781754054 >> 172.21.15.116:0/2781754054 conn(0x7fa0000fc7a0 msgr2=0x7fa0000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:35.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.390+0000 7fa005ac1640 1 -- 172.21.15.116:0/2781754054 shutdown_connections 2024-09-12T03:19:35.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.390+0000 7fa005ac1640 1 -- 172.21.15.116:0/2781754054 wait complete. 2024-09-12T03:19:35.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.390+0000 7fa005ac1640 1 Processor -- start 2024-09-12T03:19:35.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.390+0000 7fa005ac1640 1 -- start start 2024-09-12T03:19:35.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.391+0000 7fa005ac1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa000100a80 0x7fa00010e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:35.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.391+0000 7fa005ac1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa000101420 0x7fa00010e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:35.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.391+0000 7fa005ac1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa000106a80 0x7fa00010ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:35.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.391+0000 7fa005ac1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa000111fa0 con 0x7fa000100a80 2024-09-12T03:19:35.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.391+0000 7fa005ac1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa000111e20 con 0x7fa000106a80 2024-09-12T03:19:35.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.391+0000 7fa005ac1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa000112120 con 0x7fa000101420 2024-09-12T03:19:35.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.391+0000 7f9fff7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa000100a80 0x7fa00010e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:35.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.391+0000 7f9fff7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa000100a80 0x7fa00010e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55018/0 (socket says 172.21.15.116:55018) 2024-09-12T03:19:35.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.391+0000 7f9fff7fe640 1 -- 172.21.15.116:0/4092953569 learned_addr learned my addr 172.21.15.116:0/4092953569 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:35.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.391+0000 7f9ff7fff640 1 --2- 172.21.15.116:0/4092953569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa000101420 0x7fa00010e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:35.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.391+0000 7f9ffffff640 1 --2- 172.21.15.116:0/4092953569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa000106a80 0x7fa00010ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:35.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.391+0000 7f9fff7fe640 1 -- 172.21.15.116:0/4092953569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa000101420 msgr2=0x7fa00010e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:35.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.391+0000 7f9fff7fe640 1 --2- 172.21.15.116:0/4092953569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa000101420 0x7fa00010e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.391+0000 7f9fff7fe640 1 -- 172.21.15.116:0/4092953569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa000106a80 msgr2=0x7fa00010ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:35.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.391+0000 7f9fff7fe640 1 --2- 172.21.15.116:0/4092953569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa000106a80 0x7fa00010ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.391+0000 7f9fff7fe640 1 -- 172.21.15.116:0/4092953569 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0001a6940 con 0x7fa000100a80 2024-09-12T03:19:35.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.391+0000 7f9fff7fe640 1 --2- 172.21.15.116:0/4092953569 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa000100a80 0x7fa00010e3e0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f9ff000d770 tx=0x7f9ff000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:35.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.391+0000 7f9ffd7fa640 1 -- 172.21.15.116:0/4092953569 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0017070 con 0x7fa000100a80 2024-09-12T03:19:35.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.391+0000 7fa005ac1640 1 -- 172.21.15.116:0/4092953569 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0001a6c30 con 0x7fa000100a80 2024-09-12T03:19:35.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.391+0000 7f9ffd7fa640 1 -- 172.21.15.116:0/4092953569 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ff0005500 con 0x7fa000100a80 2024-09-12T03:19:35.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.391+0000 7f9ffd7fa640 1 -- 172.21.15.116:0/4092953569 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0004e00 con 0x7fa000100a80 2024-09-12T03:19:35.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.391+0000 7fa005ac1640 1 -- 172.21.15.116:0/4092953569 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0001a7140 con 0x7fa000100a80 2024-09-12T03:19:35.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.392+0000 7fa005ac1640 1 -- 172.21.15.116:0/4092953569 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa000100ee0 con 0x7fa000100a80 2024-09-12T03:19:35.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.394+0000 7f9ffd7fa640 1 -- 172.21.15.116:0/4092953569 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f9ff0004fa0 con 0x7fa000100a80 2024-09-12T03:19:35.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.394+0000 7f9ffd7fa640 1 --2- 172.21.15.116:0/4092953569 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9fec0a2b80 0x7f9fec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:35.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.394+0000 7f9ff7fff640 1 --2- 172.21.15.116:0/4092953569 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9fec0a2b80 0x7f9fec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:35.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.394+0000 7f9ffd7fa640 1 -- 172.21.15.116:0/4092953569 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f9ff00cee50 con 0x7fa000100a80 2024-09-12T03:19:35.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.395+0000 7f9ff7fff640 1 --2- 172.21.15.116:0/4092953569 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9fec0a2b80 0x7f9fec0a5040 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8000c00 tx=0x7f9fe8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:35.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.397+0000 7f9ffd7fa640 1 -- 172.21.15.116:0/4092953569 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9ff009bde0 con 0x7fa000100a80 2024-09-12T03:19:35.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:35 smithi177 ceph-mon[35305]: pgmap v835: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-12T03:19:35.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.575+0000 7fa005ac1640 1 -- 172.21.15.116:0/4092953569 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa000108ff0 con 0x7fa000100a80 2024-09-12T03:19:35.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.575+0000 7f9ffd7fa640 1 -- 172.21.15.116:0/4092953569 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v80) ==== 81+0+600 (secure 0 0 0) 0x7f9ff009b6e0 con 0x7fa000100a80 2024-09-12T03:19:35.576 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116:6801/2966274423 2024-09-13T02:54:03.026862+0000 2024-09-12T03:19:35.576 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116:6800/2966274423 2024-09-13T02:54:03.026862+0000 2024-09-12T03:19:35.576 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116:0/686139455 2024-09-13T02:54:03.026862+0000 2024-09-12T03:19:35.576 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116:0/1552047390 2024-09-13T02:54:03.026862+0000 2024-09-12T03:19:35.576 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116:0/2180363411 2024-09-13T02:53:39.433769+0000 2024-09-12T03:19:35.576 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116:6801/1791219601 2024-09-13T02:53:39.433769+0000 2024-09-12T03:19:35.576 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116:0/3907400460 2024-09-13T02:54:03.026862+0000 2024-09-12T03:19:35.576 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116:6800/1791219601 2024-09-13T02:53:39.433769+0000 2024-09-12T03:19:35.577 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116:0/1802566286 2024-09-13T02:53:39.433769+0000 2024-09-12T03:19:35.577 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116:0/212337802 2024-09-13T02:53:39.433769+0000 2024-09-12T03:19:35.577 INFO:teuthology.orchestra.run.smithi116.stderr:listed 10 entries 2024-09-12T03:19:35.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.577+0000 7fa005ac1640 1 -- 172.21.15.116:0/4092953569 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9fec0a2b80 msgr2=0x7f9fec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:35.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.577+0000 7fa005ac1640 1 --2- 172.21.15.116:0/4092953569 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9fec0a2b80 0x7f9fec0a5040 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8000c00 tx=0x7f9fe8005a20 comp rx=0 tx=0).stop 2024-09-12T03:19:35.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.577+0000 7fa005ac1640 1 -- 172.21.15.116:0/4092953569 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa000100a80 msgr2=0x7fa00010e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:35.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.577+0000 7fa005ac1640 1 --2- 172.21.15.116:0/4092953569 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa000100a80 0x7fa00010e3e0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f9ff000d770 tx=0x7f9ff000dc30 comp rx=0 tx=0).stop 2024-09-12T03:19:35.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.577+0000 7fa005ac1640 1 -- 172.21.15.116:0/4092953569 shutdown_connections 2024-09-12T03:19:35.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.577+0000 7fa005ac1640 1 --2- 172.21.15.116:0/4092953569 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f9fec0a2b80 0x7f9fec0a5040 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.577+0000 7fa005ac1640 1 --2- 172.21.15.116:0/4092953569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa000106a80 0x7fa00010ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.577+0000 7fa005ac1640 1 --2- 172.21.15.116:0/4092953569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa000101420 0x7fa00010e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.578 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.577+0000 7fa005ac1640 1 --2- 172.21.15.116:0/4092953569 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa000100a80 0x7fa00010e3e0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.577+0000 7fa005ac1640 1 -- 172.21.15.116:0/4092953569 >> 172.21.15.116:0/4092953569 conn(0x7fa0000fc7a0 msgr2=0x7fa0000fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:35.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.578+0000 7fa005ac1640 1 -- 172.21.15.116:0/4092953569 shutdown_connections 2024-09-12T03:19:35.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:35.578+0000 7fa005ac1640 1 -- 172.21.15.116:0/4092953569 wait complete. 2024-09-12T03:19:36.387 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-12T03:19:36.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:36.388 DEBUG:teuthology.orchestra.run.smithi116:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-12T03:19:36.411 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:36 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/4092953569' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:19:36.433 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-09-12T03:19:36.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:36.433 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T03:19:36.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:36 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/4092953569' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:19:36.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:36.491 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-09-12T03:19:36.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:36 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/4092953569' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:19:36.547 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T03:19:36.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:36.548 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:19:36.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:36.614 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link delete ceph-brx 2024-09-12T03:19:36.731 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-12T03:19:36.741 INFO:tasks.cephadm:Teardown begin 2024-09-12T03:19:36.742 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T03:19:36.742 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:19:36.994 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:37.409 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:37 smithi116 ceph-mon[31410]: pgmap v836: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s 2024-09-12T03:19:37.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:37 smithi183 ceph-mon[35535]: pgmap v836: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s 2024-09-12T03:19:37.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:37 smithi177 ceph-mon[35305]: pgmap v836: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s 2024-09-12T03:19:37.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.490+0000 7f35e3a8c640 1 -- 172.21.15.116:0/965004363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35dc102ef0 msgr2=0x7f35dc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:37.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.490+0000 7f35e3a8c640 1 --2- 172.21.15.116:0/965004363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35dc102ef0 0x7f35dc10da40 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f35d4009900 tx=0x7f35d402a480 comp rx=0 tx=0).stop 2024-09-12T03:19:37.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.490+0000 7f35e3a8c640 1 -- 172.21.15.116:0/965004363 shutdown_connections 2024-09-12T03:19:37.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.490+0000 7f35e3a8c640 1 --2- 172.21.15.116:0/965004363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35dc102ef0 0x7f35dc10da40 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.490+0000 7f35e3a8c640 1 --2- 172.21.15.116:0/965004363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35dc102570 0x7f35dc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.490+0000 7f35e3a8c640 1 --2- 172.21.15.116:0/965004363 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35dc100e20 0x7f35dc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.490+0000 7f35e3a8c640 1 -- 172.21.15.116:0/965004363 >> 172.21.15.116:0/965004363 conn(0x7f35dc0fc7d0 msgr2=0x7f35dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:37.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.490+0000 7f35e3a8c640 1 -- 172.21.15.116:0/965004363 shutdown_connections 2024-09-12T03:19:37.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.490+0000 7f35e3a8c640 1 -- 172.21.15.116:0/965004363 wait complete. 2024-09-12T03:19:37.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.490+0000 7f35e3a8c640 1 Processor -- start 2024-09-12T03:19:37.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.490+0000 7f35e3a8c640 1 -- start start 2024-09-12T03:19:37.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.491+0000 7f35e3a8c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35dc100e20 0x7f35dc10e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.491+0000 7f35e3a8c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35dc102570 0x7f35dc10ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.491+0000 7f35e3a8c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35dc102ef0 0x7f35dc10ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.491+0000 7f35e3a8c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc1120a0 con 0x7f35dc100e20 2024-09-12T03:19:37.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.491+0000 7f35e3a8c640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc111f20 con 0x7f35dc102570 2024-09-12T03:19:37.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.491+0000 7f35e3a8c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc112220 con 0x7f35dc102ef0 2024-09-12T03:19:37.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.491+0000 7f35e1828640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35dc100e20 0x7f35dc10e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.491+0000 7f35e1828640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35dc100e20 0x7f35dc10e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55040/0 (socket says 172.21.15.116:55040) 2024-09-12T03:19:37.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.491+0000 7f35e1828640 1 -- 172.21.15.116:0/98254448 learned_addr learned my addr 172.21.15.116:0/98254448 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:37.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.491+0000 7f35e2029640 1 --2- 172.21.15.116:0/98254448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35dc102ef0 0x7f35dc10ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.491+0000 7f35e1027640 1 --2- 172.21.15.116:0/98254448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35dc102570 0x7f35dc10ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.491+0000 7f35e1828640 1 -- 172.21.15.116:0/98254448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35dc102ef0 msgr2=0x7f35dc10ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:37.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.491+0000 7f35e1828640 1 --2- 172.21.15.116:0/98254448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35dc102ef0 0x7f35dc10ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.491+0000 7f35e1828640 1 -- 172.21.15.116:0/98254448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35dc102570 msgr2=0x7f35dc10ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:37.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.491+0000 7f35e1828640 1 --2- 172.21.15.116:0/98254448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35dc102570 0x7f35dc10ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.491+0000 7f35e1828640 1 -- 172.21.15.116:0/98254448 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35dc1a6920 con 0x7f35dc100e20 2024-09-12T03:19:37.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.491+0000 7f35e1027640 1 --2- 172.21.15.116:0/98254448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35dc102570 0x7f35dc10ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:37.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.491+0000 7f35e1828640 1 --2- 172.21.15.116:0/98254448 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35dc100e20 0x7f35dc10e4d0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00ba50 tx=0x7f35cc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:37.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.491+0000 7f35daffd640 1 -- 172.21.15.116:0/98254448 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc019070 con 0x7f35dc100e20 2024-09-12T03:19:37.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.491+0000 7f35daffd640 1 -- 172.21.15.116:0/98254448 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35cc00cbe0 con 0x7f35dc100e20 2024-09-12T03:19:37.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.491+0000 7f35daffd640 1 -- 172.21.15.116:0/98254448 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc0076e0 con 0x7f35dc100e20 2024-09-12T03:19:37.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.491+0000 7f35e3a8c640 1 -- 172.21.15.116:0/98254448 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35dc1a6c10 con 0x7f35dc100e20 2024-09-12T03:19:37.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.492+0000 7f35e3a8c640 1 -- 172.21.15.116:0/98254448 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35dc1a7100 con 0x7f35dc100e20 2024-09-12T03:19:37.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.492+0000 7f35e3a8c640 1 -- 172.21.15.116:0/98254448 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35dc101200 con 0x7f35dc100e20 2024-09-12T03:19:37.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.494+0000 7f35daffd640 1 -- 172.21.15.116:0/98254448 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f35cc01c070 con 0x7f35dc100e20 2024-09-12T03:19:37.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.494+0000 7f35daffd640 1 --2- 172.21.15.116:0/98254448 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f35c40a2b80 0x7f35c40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.495+0000 7f35e1027640 1 --2- 172.21.15.116:0/98254448 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f35c40a2b80 0x7f35c40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.495+0000 7f35daffd640 1 -- 172.21.15.116:0/98254448 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f35cc0ceb20 con 0x7f35dc100e20 2024-09-12T03:19:37.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.495+0000 7f35e1027640 1 --2- 172.21.15.116:0/98254448 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f35c40a2b80 0x7f35c40a5040 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f35dc10ff60 tx=0x7f35d000a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:37.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.498+0000 7f35daffd640 1 -- 172.21.15.116:0/98254448 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f35cc09b400 con 0x7f35dc100e20 2024-09-12T03:19:37.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.700+0000 7f35e3a8c640 1 -- 172.21.15.116:0/98254448 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35dc10fa00 con 0x7f35dc100e20 2024-09-12T03:19:37.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.701+0000 7f35daffd640 1 -- 172.21.15.116:0/98254448 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f35dc10fa00 con 0x7f35dc100e20 2024-09-12T03:19:37.701 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:19:37.701 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:19:37.702 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:19:37.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.703+0000 7f35e3a8c640 1 -- 172.21.15.116:0/98254448 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f35c40a2b80 msgr2=0x7f35c40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:37.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.703+0000 7f35e3a8c640 1 --2- 172.21.15.116:0/98254448 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f35c40a2b80 0x7f35c40a5040 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f35dc10ff60 tx=0x7f35d000a5c0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.703+0000 7f35e3a8c640 1 -- 172.21.15.116:0/98254448 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35dc100e20 msgr2=0x7f35dc10e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:37.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.703+0000 7f35e3a8c640 1 --2- 172.21.15.116:0/98254448 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35dc100e20 0x7f35dc10e4d0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00ba50 tx=0x7f35cc00bf10 comp rx=0 tx=0).stop 2024-09-12T03:19:37.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.703+0000 7f35e3a8c640 1 -- 172.21.15.116:0/98254448 shutdown_connections 2024-09-12T03:19:37.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.703+0000 7f35e3a8c640 1 --2- 172.21.15.116:0/98254448 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f35c40a2b80 0x7f35c40a5040 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.703+0000 7f35e3a8c640 1 --2- 172.21.15.116:0/98254448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35dc102ef0 0x7f35dc10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.703+0000 7f35e3a8c640 1 --2- 172.21.15.116:0/98254448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35dc102570 0x7f35dc10ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.703+0000 7f35e3a8c640 1 --2- 172.21.15.116:0/98254448 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35dc100e20 0x7f35dc10e4d0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.703+0000 7f35e3a8c640 1 -- 172.21.15.116:0/98254448 >> 172.21.15.116:0/98254448 conn(0x7f35dc0fc7d0 msgr2=0x7f35dc1067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:37.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.703+0000 7f35e3a8c640 1 -- 172.21.15.116:0/98254448 shutdown_connections 2024-09-12T03:19:37.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:37.703+0000 7f35e3a8c640 1 -- 172.21.15.116:0/98254448 wait complete. 2024-09-12T03:19:38.383 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:38 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/98254448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:38.413 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:19:38.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:38 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/98254448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:38.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:38 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/98254448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:38.660 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:39.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.166+0000 7f4669a6f640 1 -- 172.21.15.116:0/1243318376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4664102ef0 msgr2=0x7f466410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:39.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.166+0000 7f4669a6f640 1 --2- 172.21.15.116:0/1243318376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4664102ef0 0x7f466410da40 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f465c009ca0 tx=0x7f465c02a950 comp rx=0 tx=0).stop 2024-09-12T03:19:39.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.166+0000 7f4669a6f640 1 -- 172.21.15.116:0/1243318376 shutdown_connections 2024-09-12T03:19:39.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.166+0000 7f4669a6f640 1 --2- 172.21.15.116:0/1243318376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4664102ef0 0x7f466410da40 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.166+0000 7f4669a6f640 1 --2- 172.21.15.116:0/1243318376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4664102570 0x7f46641029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.166+0000 7f4669a6f640 1 --2- 172.21.15.116:0/1243318376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4664100e20 0x7f4664101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.166+0000 7f4669a6f640 1 -- 172.21.15.116:0/1243318376 >> 172.21.15.116:0/1243318376 conn(0x7f46640fc7d0 msgr2=0x7f46640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:39.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.166+0000 7f4669a6f640 1 -- 172.21.15.116:0/1243318376 shutdown_connections 2024-09-12T03:19:39.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.166+0000 7f4669a6f640 1 -- 172.21.15.116:0/1243318376 wait complete. 2024-09-12T03:19:39.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.166+0000 7f4669a6f640 1 Processor -- start 2024-09-12T03:19:39.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.167+0000 7f4669a6f640 1 -- start start 2024-09-12T03:19:39.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.167+0000 7f4669a6f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4664100e20 0x7f4664198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:39.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.167+0000 7f4669a6f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4664102570 0x7f4664198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:39.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.167+0000 7f4669a6f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4664102ef0 0x7f466419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:39.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.167+0000 7f4669a6f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4664110f80 con 0x7f4664100e20 2024-09-12T03:19:39.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.167+0000 7f4669a6f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4664110e00 con 0x7f4664102570 2024-09-12T03:19:39.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.167+0000 7f4669a6f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4664111100 con 0x7f4664102ef0 2024-09-12T03:19:39.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.167+0000 7f46637fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4664100e20 0x7f4664198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:39.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.167+0000 7f46637fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4664100e20 0x7f4664198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55062/0 (socket says 172.21.15.116:55062) 2024-09-12T03:19:39.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.167+0000 7f46637fe640 1 -- 172.21.15.116:0/3815706765 learned_addr learned my addr 172.21.15.116:0/3815706765 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:39.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.167+0000 7f4663fff640 1 --2- 172.21.15.116:0/3815706765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4664102ef0 0x7f466419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:39.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.167+0000 7f4662ffd640 1 --2- 172.21.15.116:0/3815706765 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4664102570 0x7f4664198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:39.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.167+0000 7f46637fe640 1 -- 172.21.15.116:0/3815706765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4664102ef0 msgr2=0x7f466419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:39.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.167+0000 7f46637fe640 1 --2- 172.21.15.116:0/3815706765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4664102ef0 0x7f466419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.167+0000 7f46637fe640 1 -- 172.21.15.116:0/3815706765 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4664102570 msgr2=0x7f4664198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:39.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.167+0000 7f46637fe640 1 --2- 172.21.15.116:0/3815706765 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4664102570 0x7f4664198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.167+0000 7f46637fe640 1 -- 172.21.15.116:0/3815706765 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466419d4b0 con 0x7f4664100e20 2024-09-12T03:19:39.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.167+0000 7f4662ffd640 1 --2- 172.21.15.116:0/3815706765 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4664102570 0x7f4664198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:39.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.167+0000 7f46637fe640 1 --2- 172.21.15.116:0/3815706765 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4664100e20 0x7f4664198500 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f465400d770 tx=0x7f465400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:39.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.168+0000 7f4660ff9640 1 -- 172.21.15.116:0/3815706765 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4654017070 con 0x7f4664100e20 2024-09-12T03:19:39.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.168+0000 7f4669a6f640 1 -- 172.21.15.116:0/3815706765 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f466419d7a0 con 0x7f4664100e20 2024-09-12T03:19:39.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.168+0000 7f4660ff9640 1 -- 172.21.15.116:0/3815706765 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4654005500 con 0x7f4664100e20 2024-09-12T03:19:39.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.168+0000 7f4660ff9640 1 -- 172.21.15.116:0/3815706765 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4654004e00 con 0x7f4664100e20 2024-09-12T03:19:39.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.168+0000 7f4669a6f640 1 -- 172.21.15.116:0/3815706765 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46641a5080 con 0x7f4664100e20 2024-09-12T03:19:39.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.169+0000 7f4669a6f640 1 -- 172.21.15.116:0/3815706765 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4664101200 con 0x7f4664100e20 2024-09-12T03:19:39.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.170+0000 7f4660ff9640 1 -- 172.21.15.116:0/3815706765 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4654004fa0 con 0x7f4664100e20 2024-09-12T03:19:39.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.171+0000 7f4660ff9640 1 --2- 172.21.15.116:0/3815706765 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f464c0a2b80 0x7f464c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:39.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.171+0000 7f4662ffd640 1 --2- 172.21.15.116:0/3815706765 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f464c0a2b80 0x7f464c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:39.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.171+0000 7f4660ff9640 1 -- 172.21.15.116:0/3815706765 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f46540ceec0 con 0x7f4664100e20 2024-09-12T03:19:39.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.171+0000 7f4662ffd640 1 --2- 172.21.15.116:0/3815706765 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f464c0a2b80 0x7f464c0a5040 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4658005fd0 tx=0x7f46580094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:39.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.174+0000 7f4660ff9640 1 -- 172.21.15.116:0/3815706765 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f465409be50 con 0x7f4664100e20 2024-09-12T03:19:39.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.377+0000 7f4669a6f640 1 -- 172.21.15.116:0/3815706765 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f466410a0a0 con 0x7f4664100e20 2024-09-12T03:19:39.726 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:39 smithi116 ceph-mon[31410]: pgmap v837: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-12T03:19:39.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.377+0000 7f4660ff9640 1 -- 172.21.15.116:0/3815706765 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11938 (secure 0 0 0) 0x7f465409b750 con 0x7f4664100e20 2024-09-12T03:19:39.727 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:19:39.727 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"btime":"2024-09-12T03:02:57:223057+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3909394246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3909394246},{"type":"v1","addr":"172.21.15.116:6835","nonce":3909394246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/3839272388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":3839272388},{"type":"v1","addr":"172.21.15.116:6837","nonce":3839272388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1740752718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1740752718},{"type":"v1","addr":"172.21.15.116:6839","nonce":1740752718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6841/2495398640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6840","nonce":2495398640},{"type":"v1","addr":"172.21.15.116:6841","nonce":2495398640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3132380205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3132380205},{"type":"v1","addr":"172.21.15.177:6833","nonce":3132380205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1068409597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1068409597},{"type":"v1","addr":"172.21.15.183:6833","nonce":1068409597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:00:59.195360+0000","modified":"2024-09-12T03:02:56.334134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24475,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/2224643300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":2224643300},{"type":"v1","addr":"172.21.15.177:6835","nonce":2224643300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4028826399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4028826399},{"type":"v1","addr":"172.21.15.183:6835","nonce":4028826399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1288373243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1288373243},{"type":"v1","addr":"172.21.15.183:6837","nonce":1288373243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6837/1862929607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1862929607},{"type":"v1","addr":"172.21.15.177:6837","nonce":1862929607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.183:6839/1983839027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1983839027},{"type":"v1","addr":"172.21.15.183:6839","nonce":1983839027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.177:6839/2069130361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":2069130361},{"type":"v1","addr":"172.21.15.177:6839","nonce":2069130361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24475,24467]},"id":1}]} 2024-09-12T03:19:39.728 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 25 2024-09-12T03:19:39.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.380+0000 7f4669a6f640 1 -- 172.21.15.116:0/3815706765 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f464c0a2b80 msgr2=0x7f464c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:39.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.380+0000 7f4669a6f640 1 --2- 172.21.15.116:0/3815706765 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f464c0a2b80 0x7f464c0a5040 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4658005fd0 tx=0x7f46580094d0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.380+0000 7f4669a6f640 1 -- 172.21.15.116:0/3815706765 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4664100e20 msgr2=0x7f4664198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:39.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.380+0000 7f4669a6f640 1 --2- 172.21.15.116:0/3815706765 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4664100e20 0x7f4664198500 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f465400d770 tx=0x7f465400dc30 comp rx=0 tx=0).stop 2024-09-12T03:19:39.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.380+0000 7f4669a6f640 1 -- 172.21.15.116:0/3815706765 shutdown_connections 2024-09-12T03:19:39.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.380+0000 7f4669a6f640 1 --2- 172.21.15.116:0/3815706765 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f464c0a2b80 0x7f464c0a5040 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.380+0000 7f4669a6f640 1 --2- 172.21.15.116:0/3815706765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4664102ef0 0x7f466419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.380+0000 7f4669a6f640 1 --2- 172.21.15.116:0/3815706765 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4664102570 0x7f4664198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.380+0000 7f4669a6f640 1 --2- 172.21.15.116:0/3815706765 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4664100e20 0x7f4664198500 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.380+0000 7f4669a6f640 1 -- 172.21.15.116:0/3815706765 >> 172.21.15.116:0/3815706765 conn(0x7f46640fc7d0 msgr2=0x7f466406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:39.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.380+0000 7f4669a6f640 1 -- 172.21.15.116:0/3815706765 shutdown_connections 2024-09-12T03:19:39.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:39.380+0000 7f4669a6f640 1 -- 172.21.15.116:0/3815706765 wait complete. 2024-09-12T03:19:39.729 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:39 smithi177 ceph-mon[35305]: pgmap v837: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-12T03:19:39.730 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:39 smithi183 ceph-mon[35535]: pgmap v837: 97 pgs: 97 active+clean; 7.8 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-12T03:19:40.078 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:19:40.325 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:40.446 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:40 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3815706765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:40.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:40 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3815706765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:40.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:40 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3815706765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:40.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.823+0000 7fa6a623b640 1 -- 172.21.15.116:0/1462912108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6a0100a80 msgr2=0x7fa6a0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:40.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.823+0000 7fa6a623b640 1 --2- 172.21.15.116:0/1462912108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6a0100a80 0x7fa6a0100ee0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fa688009930 tx=0x7fa68802a420 comp rx=0 tx=0).stop 2024-09-12T03:19:40.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.823+0000 7fa6a623b640 1 -- 172.21.15.116:0/1462912108 shutdown_connections 2024-09-12T03:19:40.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.823+0000 7fa6a623b640 1 --2- 172.21.15.116:0/1462912108 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6a0101420 0x7fa6a010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.823+0000 7fa6a623b640 1 --2- 172.21.15.116:0/1462912108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6a0100a80 0x7fa6a0100ee0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.823+0000 7fa6a623b640 1 --2- 172.21.15.116:0/1462912108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6a0106a80 0x7fa6a0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.823+0000 7fa6a623b640 1 -- 172.21.15.116:0/1462912108 >> 172.21.15.116:0/1462912108 conn(0x7fa6a00fc7a0 msgr2=0x7fa6a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:40.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.823+0000 7fa6a623b640 1 -- 172.21.15.116:0/1462912108 shutdown_connections 2024-09-12T03:19:40.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.823+0000 7fa6a623b640 1 -- 172.21.15.116:0/1462912108 wait complete. 2024-09-12T03:19:40.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.824+0000 7fa6a623b640 1 Processor -- start 2024-09-12T03:19:40.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.824+0000 7fa6a623b640 1 -- start start 2024-09-12T03:19:40.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.824+0000 7fa6a623b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6a0100a80 0x7fa6a0198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:40.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.824+0000 7fa6a623b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6a0101420 0x7fa6a01989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:40.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.824+0000 7fa6a623b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6a0106a80 0x7fa6a019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:40.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.824+0000 7fa6a623b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a0110ec0 con 0x7fa6a0106a80 2024-09-12T03:19:40.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.824+0000 7fa6a623b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a0110d40 con 0x7fa6a0101420 2024-09-12T03:19:40.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.824+0000 7fa6a623b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a0111040 con 0x7fa6a0100a80 2024-09-12T03:19:40.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.824+0000 7fa69ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6a0106a80 0x7fa6a019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:40.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.824+0000 7fa69ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6a0106a80 0x7fa6a019cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:55078/0 (socket says 172.21.15.116:55078) 2024-09-12T03:19:40.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.824+0000 7fa69ffff640 1 -- 172.21.15.116:0/2474549758 learned_addr learned my addr 172.21.15.116:0/2474549758 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:40.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.824+0000 7fa69f7fe640 1 --2- 172.21.15.116:0/2474549758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6a0100a80 0x7fa6a0198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:40.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.824+0000 7fa697fff640 1 --2- 172.21.15.116:0/2474549758 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6a0101420 0x7fa6a01989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:40.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.825+0000 7fa69ffff640 1 -- 172.21.15.116:0/2474549758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6a0100a80 msgr2=0x7fa6a0198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:40.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.825+0000 7fa69ffff640 1 --2- 172.21.15.116:0/2474549758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6a0100a80 0x7fa6a0198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.825+0000 7fa69ffff640 1 -- 172.21.15.116:0/2474549758 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6a0101420 msgr2=0x7fa6a01989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:40.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.825+0000 7fa69ffff640 1 --2- 172.21.15.116:0/2474549758 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6a0101420 0x7fa6a01989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.825+0000 7fa69ffff640 1 -- 172.21.15.116:0/2474549758 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a019d440 con 0x7fa6a0106a80 2024-09-12T03:19:40.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.825+0000 7fa697fff640 1 --2- 172.21.15.116:0/2474549758 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6a0101420 0x7fa6a01989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:40.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.825+0000 7fa69ffff640 1 --2- 172.21.15.116:0/2474549758 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6a0106a80 0x7fa6a019cd60 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fa69800b970 tx=0x7fa69800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:40.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.825+0000 7fa69d7fa640 1 -- 172.21.15.116:0/2474549758 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa698019070 con 0x7fa6a0106a80 2024-09-12T03:19:40.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.825+0000 7fa6a623b640 1 -- 172.21.15.116:0/2474549758 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a019d730 con 0x7fa6a0106a80 2024-09-12T03:19:40.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.825+0000 7fa69d7fa640 1 -- 172.21.15.116:0/2474549758 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa69800cb40 con 0x7fa6a0106a80 2024-09-12T03:19:40.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.825+0000 7fa69d7fa640 1 -- 172.21.15.116:0/2474549758 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa698007650 con 0x7fa6a0106a80 2024-09-12T03:19:40.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.825+0000 7fa6a623b640 1 -- 172.21.15.116:0/2474549758 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a01a5010 con 0x7fa6a0106a80 2024-09-12T03:19:40.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.826+0000 7fa6a623b640 1 -- 172.21.15.116:0/2474549758 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6a0100ee0 con 0x7fa6a0106a80 2024-09-12T03:19:40.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.827+0000 7fa69d7fa640 1 -- 172.21.15.116:0/2474549758 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa69801c070 con 0x7fa6a0106a80 2024-09-12T03:19:40.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.828+0000 7fa69d7fa640 1 --2- 172.21.15.116:0/2474549758 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa68c0a2b80 0x7fa68c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:40.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.828+0000 7fa69d7fa640 1 -- 172.21.15.116:0/2474549758 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7fa6980ce600 con 0x7fa6a0106a80 2024-09-12T03:19:40.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.828+0000 7fa69f7fe640 1 --2- 172.21.15.116:0/2474549758 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa68c0a2b80 0x7fa68c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:40.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.828+0000 7fa69f7fe640 1 --2- 172.21.15.116:0/2474549758 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa68c0a2b80 0x7fa68c0a5040 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa6900058c0 tx=0x7fa69000a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:40.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:40.832+0000 7fa69d7fa640 1 -- 172.21.15.116:0/2474549758 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa69809af40 con 0x7fa6a0106a80 2024-09-12T03:19:41.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:41.000+0000 7fa6a623b640 1 -- 172.21.15.116:0/2474549758 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6a010a080 con 0x7fa6a0106a80 2024-09-12T03:19:41.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:41.002+0000 7fa69d7fa640 1 -- 172.21.15.116:0/2474549758 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22710 (secure 0 0 0) 0x7fa69809f240 con 0x7fa6a0106a80 2024-09-12T03:19:41.002 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-12T03:19:41.002 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":80,"fsid":"eb1ac602-70b1-11ef-bcea-c7b262605968","created":"2024-09-12T02:53:07.105686+0000","modified":"2024-09-12T03:02:04.272898+0000","last_up_change":"2024-09-12T03:00:07.202639+0000","last_in_change":"2024-09-12T02:59:43.446336+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:56:17.347152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:00:55.191040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:00:57.171142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8cf5a18-dc6f-4bb6-bd74-78adda310118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6803","nonce":862366853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6805","nonce":862366853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6809","nonce":862366853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":862366853},{"type":"v1","addr":"172.21.15.116:6807","nonce":862366853}]},"public_addr":"172.21.15.116:6803/862366853","cluster_addr":"172.21.15.116:6805/862366853","heartbeat_back_addr":"172.21.15.116:6809/862366853","heartbeat_front_addr":"172.21.15.116:6807/862366853","state":["exists","up"]},{"osd":1,"uuid":"0affd3cb-9a51-42b0-a2cc-f30784a75cf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6801","nonce":2245800547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6803","nonce":2245800547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6807","nonce":2245800547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2245800547},{"type":"v1","addr":"172.21.15.177:6805","nonce":2245800547}]},"public_addr":"172.21.15.177:6801/2245800547","cluster_addr":"172.21.15.177:6803/2245800547","heartbeat_back_addr":"172.21.15.177:6807/2245800547","heartbeat_front_addr":"172.21.15.177:6805/2245800547","state":["exists","up"]},{"osd":2,"uuid":"1ce45ea5-84df-4a35-924f-78f92bd4f2cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6801","nonce":2645677759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6803","nonce":2645677759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6807","nonce":2645677759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2645677759},{"type":"v1","addr":"172.21.15.183:6805","nonce":2645677759}]},"public_addr":"172.21.15.183:6801/2645677759","cluster_addr":"172.21.15.183:6803/2645677759","heartbeat_back_addr":"172.21.15.183:6807/2645677759","heartbeat_front_addr":"172.21.15.183:6805/2645677759","state":["exists","up"]},{"osd":3,"uuid":"ae5e8178-c998-4648-bc05-4bce822f0cd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6811","nonce":686622148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6813","nonce":686622148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6817","nonce":686622148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":686622148},{"type":"v1","addr":"172.21.15.116:6815","nonce":686622148}]},"public_addr":"172.21.15.116:6811/686622148","cluster_addr":"172.21.15.116:6813/686622148","heartbeat_back_addr":"172.21.15.116:6817/686622148","heartbeat_front_addr":"172.21.15.116:6815/686622148","state":["exists","up"]},{"osd":4,"uuid":"c81d2e73-0bc2-440d-b5a2-b7428af3ea4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6809","nonce":2382938021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6811","nonce":2382938021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6815","nonce":2382938021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":2382938021},{"type":"v1","addr":"172.21.15.177:6813","nonce":2382938021}]},"public_addr":"172.21.15.177:6809/2382938021","cluster_addr":"172.21.15.177:6811/2382938021","heartbeat_back_addr":"172.21.15.177:6815/2382938021","heartbeat_front_addr":"172.21.15.177:6813/2382938021","state":["exists","up"]},{"osd":5,"uuid":"c3f168be-9e27-4cf5-b73e-3f47d8990ed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6809","nonce":85363432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6811","nonce":85363432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6815","nonce":85363432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":85363432},{"type":"v1","addr":"172.21.15.183:6813","nonce":85363432}]},"public_addr":"172.21.15.183:6809/85363432","cluster_addr":"172.21.15.183:6811/85363432","heartbeat_back_addr":"172.21.15.183:6815/85363432","heartbeat_front_addr":"172.21.15.183:6813/85363432","state":["exists","up"]},{"osd":6,"uuid":"beb0a7e9-4909-47e2-9ee7-6951d4745e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6819","nonce":1121291186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6821","nonce":1121291186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6825","nonce":1121291186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":1121291186},{"type":"v1","addr":"172.21.15.116:6823","nonce":1121291186}]},"public_addr":"172.21.15.116:6819/1121291186","cluster_addr":"172.21.15.116:6821/1121291186","heartbeat_back_addr":"172.21.15.116:6825/1121291186","heartbeat_front_addr":"172.21.15.116:6823/1121291186","state":["exists","up"]},{"osd":7,"uuid":"761f4680-f049-473c-bf44-b19acf8de24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6817","nonce":2177665961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6819","nonce":2177665961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6823","nonce":2177665961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":2177665961},{"type":"v1","addr":"172.21.15.177:6821","nonce":2177665961}]},"public_addr":"172.21.15.177:6817/2177665961","cluster_addr":"172.21.15.177:6819/2177665961","heartbeat_back_addr":"172.21.15.177:6823/2177665961","heartbeat_front_addr":"172.21.15.177:6821/2177665961","state":["exists","up"]},{"osd":8,"uuid":"2385ea6c-fd5a-4815-9596-77f948629542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6817","nonce":1305725258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6819","nonce":1305725258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6823","nonce":1305725258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1305725258},{"type":"v1","addr":"172.21.15.183:6821","nonce":1305725258}]},"public_addr":"172.21.15.183:6817/1305725258","cluster_addr":"172.21.15.183:6819/1305725258","heartbeat_back_addr":"172.21.15.183:6823/1305725258","heartbeat_front_addr":"172.21.15.183:6821/1305725258","state":["exists","up"]},{"osd":9,"uuid":"fd0c0dc4-4990-4f12-aa14-f855504cd995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6827","nonce":3831673668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6829","nonce":3831673668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6833","nonce":3831673668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3831673668},{"type":"v1","addr":"172.21.15.116:6831","nonce":3831673668}]},"public_addr":"172.21.15.116:6827/3831673668","cluster_addr":"172.21.15.116:6829/3831673668","heartbeat_back_addr":"172.21.15.116:6833/3831673668","heartbeat_front_addr":"172.21.15.116:6831/3831673668","state":["exists","up"]},{"osd":10,"uuid":"864d1a97-4810-49c8-ad4a-a0c10253833c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6825","nonce":963663634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6827","nonce":963663634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6831","nonce":963663634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":963663634},{"type":"v1","addr":"172.21.15.177:6829","nonce":963663634}]},"public_addr":"172.21.15.177:6825/963663634","cluster_addr":"172.21.15.177:6827/963663634","heartbeat_back_addr":"172.21.15.177:6831/963663634","heartbeat_front_addr":"172.21.15.177:6829/963663634","state":["exists","up"]},{"osd":11,"uuid":"d49c0841-4620-4f2f-a6b5-f2a9aa2e3c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6825","nonce":1818319939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6827","nonce":1818319939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6831","nonce":1818319939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1818319939},{"type":"v1","addr":"172.21.15.183:6829","nonce":1818319939}]},"public_addr":"172.21.15.183:6825/1818319939","cluster_addr":"172.21.15.183:6827/1818319939","heartbeat_back_addr":"172.21.15.183:6831/1818319939","heartbeat_front_addr":"172.21.15.183:6829/1818319939","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:24.736944+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:50.310980+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:56:14.989533+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:56:41.187047+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:06.981664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:32.872771+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:58.850089+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:23.043121+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:47.839958+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:14.087708+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:40.010548+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:05.364820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.116:6801/2966274423":"2024-09-13T02:54:03.026862+0000","172.21.15.116:6800/2966274423":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/686139455":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/1552047390":"2024-09-13T02:54:03.026862+0000","172.21.15.116:0/2180363411":"2024-09-13T02:53:39.433769+0000","172.21.15.116:6801/1791219601":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/3907400460":"2024-09-13T02:54:03.026862+0000","172.21.15.116:6800/1791219601":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/1802566286":"2024-09-13T02:53:39.433769+0000","172.21.15.116:0/212337802":"2024-09-13T02:53:39.433769+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:19:41.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:41.004+0000 7fa6a623b640 1 -- 172.21.15.116:0/2474549758 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa68c0a2b80 msgr2=0x7fa68c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:41.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:41.004+0000 7fa6a623b640 1 --2- 172.21.15.116:0/2474549758 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa68c0a2b80 0x7fa68c0a5040 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa6900058c0 tx=0x7fa69000a250 comp rx=0 tx=0).stop 2024-09-12T03:19:41.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:41.004+0000 7fa6a623b640 1 -- 172.21.15.116:0/2474549758 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6a0106a80 msgr2=0x7fa6a019cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:41.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:41.004+0000 7fa6a623b640 1 --2- 172.21.15.116:0/2474549758 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6a0106a80 0x7fa6a019cd60 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fa69800b970 tx=0x7fa69800be30 comp rx=0 tx=0).stop 2024-09-12T03:19:41.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:41.004+0000 7fa6a623b640 1 -- 172.21.15.116:0/2474549758 shutdown_connections 2024-09-12T03:19:41.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:41.004+0000 7fa6a623b640 1 --2- 172.21.15.116:0/2474549758 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7fa68c0a2b80 0x7fa68c0a5040 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:41.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:41.004+0000 7fa6a623b640 1 --2- 172.21.15.116:0/2474549758 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6a0106a80 0x7fa6a019cd60 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:41.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:41.004+0000 7fa6a623b640 1 --2- 172.21.15.116:0/2474549758 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6a0101420 0x7fa6a01989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:41.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:41.004+0000 7fa6a623b640 1 --2- 172.21.15.116:0/2474549758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6a0100a80 0x7fa6a0198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:41.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:41.004+0000 7fa6a623b640 1 -- 172.21.15.116:0/2474549758 >> 172.21.15.116:0/2474549758 conn(0x7fa6a00fc7a0 msgr2=0x7fa6a006aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:41.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:41.004+0000 7fa6a623b640 1 -- 172.21.15.116:0/2474549758 shutdown_connections 2024-09-12T03:19:41.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:41.004+0000 7fa6a623b640 1 -- 172.21.15.116:0/2474549758 wait complete. 2024-09-12T03:19:41.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:41 smithi116 ceph-mon[31410]: pgmap v838: 97 pgs: 97 active+clean; 7.8 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-12T03:19:41.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:41 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2474549758' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:41.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:41 smithi183 ceph-mon[35535]: pgmap v838: 97 pgs: 97 active+clean; 7.8 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-12T03:19:41.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:41 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2474549758' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:41.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:41 smithi177 ceph-mon[35305]: pgmap v838: 97 pgs: 97 active+clean; 7.8 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-12T03:19:41.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:41 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2474549758' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:41.733 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-12T03:19:41.983 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:42.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.483+0000 7f95cef5a640 1 -- 172.21.15.116:0/297308305 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95c8100a80 msgr2=0x7f95c8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.483+0000 7f95cef5a640 1 --2- 172.21.15.116:0/297308305 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95c8100a80 0x7f95c8100ee0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f95b4009930 tx=0x7f95b402a420 comp rx=0 tx=0).stop 2024-09-12T03:19:42.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.483+0000 7f95cef5a640 1 -- 172.21.15.116:0/297308305 shutdown_connections 2024-09-12T03:19:42.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.483+0000 7f95cef5a640 1 --2- 172.21.15.116:0/297308305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95c8101420 0x7f95c810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.483+0000 7f95cef5a640 1 --2- 172.21.15.116:0/297308305 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95c8100a80 0x7f95c8100ee0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.483+0000 7f95cef5a640 1 --2- 172.21.15.116:0/297308305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95c8106a80 0x7f95c8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.483+0000 7f95cef5a640 1 -- 172.21.15.116:0/297308305 >> 172.21.15.116:0/297308305 conn(0x7f95c80fc7a0 msgr2=0x7f95c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:42.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.483+0000 7f95cef5a640 1 -- 172.21.15.116:0/297308305 shutdown_connections 2024-09-12T03:19:42.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.483+0000 7f95cef5a640 1 -- 172.21.15.116:0/297308305 wait complete. 2024-09-12T03:19:42.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.483+0000 7f95cef5a640 1 Processor -- start 2024-09-12T03:19:42.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.483+0000 7f95cef5a640 1 -- start start 2024-09-12T03:19:42.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.484+0000 7f95cef5a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95c8100a80 0x7f95c81985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.484+0000 7f95cef5a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95c8101420 0x7f95c8198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.484+0000 7f95cef5a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95c8106a80 0x7f95c819cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.484+0000 7f95cef5a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f95c8110f20 con 0x7f95c8100a80 2024-09-12T03:19:42.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.484+0000 7f95cef5a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f95c8110da0 con 0x7f95c8101420 2024-09-12T03:19:42.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.484+0000 7f95cef5a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f95c81110a0 con 0x7f95c8106a80 2024-09-12T03:19:42.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.484+0000 7f95cccf6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95c8100a80 0x7f95c81985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.484+0000 7f95cccf6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95c8100a80 0x7f95c81985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:60998/0 (socket says 172.21.15.116:60998) 2024-09-12T03:19:42.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.484+0000 7f95cccf6640 1 -- 172.21.15.116:0/1793533011 learned_addr learned my addr 172.21.15.116:0/1793533011 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:42.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.484+0000 7f95cd4f7640 1 --2- 172.21.15.116:0/1793533011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95c8106a80 0x7f95c819cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.484+0000 7f95cccf6640 1 -- 172.21.15.116:0/1793533011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95c8106a80 msgr2=0x7f95c819cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.484+0000 7f95c7fff640 1 --2- 172.21.15.116:0/1793533011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95c8101420 0x7f95c8198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.484+0000 7f95cccf6640 1 --2- 172.21.15.116:0/1793533011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95c8106a80 0x7f95c819cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.484+0000 7f95cccf6640 1 -- 172.21.15.116:0/1793533011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95c8101420 msgr2=0x7f95c8198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.484+0000 7f95cccf6640 1 --2- 172.21.15.116:0/1793533011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95c8101420 0x7f95c8198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.484+0000 7f95cccf6640 1 -- 172.21.15.116:0/1793533011 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c819d5a0 con 0x7f95c8100a80 2024-09-12T03:19:42.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.484+0000 7f95cd4f7640 1 --2- 172.21.15.116:0/1793533011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95c8106a80 0x7f95c819cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:42.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.484+0000 7f95cccf6640 1 --2- 172.21.15.116:0/1793533011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95c8100a80 0x7f95c81985f0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00b4a0 tx=0x7f95bc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:42.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.484+0000 7f95c5ffb640 1 -- 172.21.15.116:0/1793533011 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc014070 con 0x7f95c8100a80 2024-09-12T03:19:42.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.484+0000 7f95cef5a640 1 -- 172.21.15.116:0/1793533011 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95c819d890 con 0x7f95c8100a80 2024-09-12T03:19:42.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.484+0000 7f95c5ffb640 1 -- 172.21.15.116:0/1793533011 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95bc010af0 con 0x7f95c8100a80 2024-09-12T03:19:42.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.484+0000 7f95c5ffb640 1 -- 172.21.15.116:0/1793533011 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc017d10 con 0x7f95c8100a80 2024-09-12T03:19:42.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.485+0000 7f95cef5a640 1 -- 172.21.15.116:0/1793533011 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95c81a5170 con 0x7f95c8100a80 2024-09-12T03:19:42.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.486+0000 7f95cef5a640 1 -- 172.21.15.116:0/1793533011 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95c8100f60 con 0x7f95c8100a80 2024-09-12T03:19:42.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.487+0000 7f95c5ffb640 1 -- 172.21.15.116:0/1793533011 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f95bc017070 con 0x7f95c8100a80 2024-09-12T03:19:42.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.487+0000 7f95c5ffb640 1 --2- 172.21.15.116:0/1793533011 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f95b80a2b80 0x7f95b80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.487+0000 7f95c7fff640 1 --2- 172.21.15.116:0/1793533011 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f95b80a2b80 0x7f95b80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.487+0000 7f95c5ffb640 1 -- 172.21.15.116:0/1793533011 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8182+0+0 (secure 0 0 0) 0x7f95bc0ce500 con 0x7f95c8100a80 2024-09-12T03:19:42.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.488+0000 7f95c7fff640 1 --2- 172.21.15.116:0/1793533011 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f95b80a2b80 0x7f95b80a5040 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f95b40096c0 tx=0x7f95b4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:42.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.491+0000 7f95c5ffb640 1 -- 172.21.15.116:0/1793533011 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f95bc09b490 con 0x7f95c8100a80 2024-09-12T03:19:42.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.695+0000 7f95cef5a640 1 -- 172.21.15.116:0/1793533011 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f95c8199950 con 0x7f95c8100a80 2024-09-12T03:19:42.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.697+0000 7f95c5ffb640 1 -- 172.21.15.116:0/1793533011 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) ==== 161+0+0 (secure 0 0 0) 0x7f95bc09ad90 con 0x7f95c8100a80 2024-09-12T03:19:42.697 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T03:19:42.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.700+0000 7f95cef5a640 1 -- 172.21.15.116:0/1793533011 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f95b80a2b80 msgr2=0x7f95b80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.700+0000 7f95cef5a640 1 --2- 172.21.15.116:0/1793533011 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f95b80a2b80 0x7f95b80a5040 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f95b40096c0 tx=0x7f95b4000bc0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.700+0000 7f95cef5a640 1 -- 172.21.15.116:0/1793533011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95c8100a80 msgr2=0x7f95c81985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.700+0000 7f95cef5a640 1 --2- 172.21.15.116:0/1793533011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95c8100a80 0x7f95c81985f0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00b4a0 tx=0x7f95bc00b960 comp rx=0 tx=0).stop 2024-09-12T03:19:42.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.700+0000 7f95cef5a640 1 -- 172.21.15.116:0/1793533011 shutdown_connections 2024-09-12T03:19:42.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.700+0000 7f95cef5a640 1 --2- 172.21.15.116:0/1793533011 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f95b80a2b80 0x7f95b80a5040 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.700+0000 7f95cef5a640 1 --2- 172.21.15.116:0/1793533011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95c8106a80 0x7f95c819cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.700+0000 7f95cef5a640 1 --2- 172.21.15.116:0/1793533011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95c8101420 0x7f95c8198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.700+0000 7f95cef5a640 1 --2- 172.21.15.116:0/1793533011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95c8100a80 0x7f95c81985f0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.700+0000 7f95cef5a640 1 -- 172.21.15.116:0/1793533011 >> 172.21.15.116:0/1793533011 conn(0x7f95c80fc7a0 msgr2=0x7f95c81048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:42.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.701+0000 7f95cef5a640 1 -- 172.21.15.116:0/1793533011 shutdown_connections 2024-09-12T03:19:42.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:42.701+0000 7f95cef5a640 1 -- 172.21.15.116:0/1793533011 wait complete. 2024-09-12T03:19:42.994 INFO:journalctl@ceph.mds.b.smithi177.stdout:Sep 12 03:19:42 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-b[65339]: ignoring --setuser ceph since I am not root 2024-09-12T03:19:42.994 INFO:journalctl@ceph.mds.b.smithi177.stdout:Sep 12 03:19:42 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-b[65339]: ignoring --setgroup ceph since I am not root 2024-09-12T03:19:42.995 INFO:journalctl@ceph.mds.b.smithi177.stdout:Sep 12 03:19:42 smithi177 ceph-mds[65343]: -- 172.21.15.177:0/3391374561 <== mon.0 v2:172.21.15.116:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55afe308d860 con 0x55afe306b400 2024-09-12T03:19:42.995 INFO:journalctl@ceph.mds.b.smithi177.stdout:Sep 12 03:19:42 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-b[65339]: starting mds.b at 2024-09-12T03:19:42.995 INFO:journalctl@ceph.mds.e.smithi177.stdout:Sep 12 03:19:42 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-e[64726]: ignoring --setuser ceph since I am not root 2024-09-12T03:19:42.995 INFO:journalctl@ceph.mds.e.smithi177.stdout:Sep 12 03:19:42 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-e[64726]: ignoring --setgroup ceph since I am not root 2024-09-12T03:19:42.995 INFO:journalctl@ceph.mds.e.smithi177.stdout:Sep 12 03:19:42 smithi177 ceph-mds[64730]: -- 172.21.15.177:0/558072455 <== mon.2 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5571ee387860 con 0x5571ee364800 2024-09-12T03:19:42.995 INFO:journalctl@ceph.mds.e.smithi177.stdout:Sep 12 03:19:42 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-e[64726]: starting mds.e at 2024-09-12T03:19:42.995 INFO:journalctl@ceph.mds.h.smithi177.stdout:Sep 12 03:19:42 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-h[65028]: ignoring --setuser ceph since I am not root 2024-09-12T03:19:42.995 INFO:journalctl@ceph.mds.h.smithi177.stdout:Sep 12 03:19:42 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-h[65028]: ignoring --setgroup ceph since I am not root 2024-09-12T03:19:42.996 INFO:journalctl@ceph.mds.h.smithi177.stdout:Sep 12 03:19:42 smithi177 ceph-mds[65032]: -- 172.21.15.177:0/3261184660 <== mon.0 v2:172.21.15.116:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55abe5721c20 con 0x55abe56ff000 2024-09-12T03:19:42.996 INFO:journalctl@ceph.mds.h.smithi177.stdout:Sep 12 03:19:42 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-h[65028]: starting mds.h at 2024-09-12T03:19:43.015 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:42 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mon-a[31406]: 2024-09-12T03:19:42.696+0000 7ff9a1f65640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:19:43.163 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 03:19:42 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-c[65771]: ignoring --setuser ceph since I am not root 2024-09-12T03:19:43.163 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 03:19:42 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-c[65771]: ignoring --setgroup ceph since I am not root 2024-09-12T03:19:43.163 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 03:19:42 smithi183 ceph-mds[65775]: -- 172.21.15.183:0/3253912096 <== mon.2 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558482743860 con 0x558482721000 2024-09-12T03:19:43.163 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 03:19:42 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-c[65771]: starting mds.c at 2024-09-12T03:19:43.163 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 03:19:42 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-f[65466]: ignoring --setuser ceph since I am not root 2024-09-12T03:19:43.164 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 03:19:42 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-f[65466]: ignoring --setgroup ceph since I am not root 2024-09-12T03:19:43.164 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 03:19:42 smithi183 ceph-mds[65470]: -- 172.21.15.183:0/3196760208 <== mon.0 v2:172.21.15.116:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dc184c3860 con 0x55dc184a1400 2024-09-12T03:19:43.164 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 03:19:42 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-f[65466]: starting mds.f at 2024-09-12T03:19:43.164 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 03:19:42 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-i[66074]: ignoring --setuser ceph since I am not root 2024-09-12T03:19:43.164 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 03:19:42 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-i[66074]: ignoring --setgroup ceph since I am not root 2024-09-12T03:19:43.164 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 03:19:42 smithi183 ceph-mds[66078]: -- 172.21.15.183:0/3487970268 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563e3663d860 con 0x563e3661b000 2024-09-12T03:19:43.164 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 03:19:42 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-i[66074]: starting mds.i at 2024-09-12T03:19:43.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:43 smithi116 ceph-mon[31410]: pgmap v839: 97 pgs: 97 active+clean; 7.8 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T03:19:43.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:43 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1793533011' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T03:19:43.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:43 smithi116 ceph-mon[31410]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T03:19:43.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:43 smithi116 ceph-mon[31410]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:19:43.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:43 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/1793533011' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T03:19:43.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:43 smithi116 ceph-mon[31410]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T03:19:43.302 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:43 smithi116 ceph-mon[31410]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-12T03:19:43.402 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T03:19:43.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:43 smithi183 ceph-mon[35535]: pgmap v839: 97 pgs: 97 active+clean; 7.8 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T03:19:43.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:43 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1793533011' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T03:19:43.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:43 smithi183 ceph-mon[35535]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T03:19:43.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:43 smithi183 ceph-mon[35535]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:19:43.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:43 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/1793533011' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T03:19:43.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:43 smithi183 ceph-mon[35535]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T03:19:43.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:43 smithi183 ceph-mon[35535]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-12T03:19:43.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:43 smithi177 ceph-mon[35305]: pgmap v839: 97 pgs: 97 active+clean; 7.8 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T03:19:43.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:43 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1793533011' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T03:19:43.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:43 smithi177 ceph-mon[35305]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T03:19:43.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:43 smithi177 ceph-mon[35305]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:19:43.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:43 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/1793533011' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T03:19:43.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:43 smithi177 ceph-mon[35305]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T03:19:43.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:43 smithi177 ceph-mon[35305]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-12T03:19:43.650 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:44.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.178+0000 7f41ec248640 1 -- 172.21.15.116:0/1152127380 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f41e4102570 msgr2=0x7f41e41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:44.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.178+0000 7f41ec248640 1 --2- 172.21.15.116:0/1152127380 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f41e4102570 0x7f41e41029b0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f41d8009900 tx=0x7f41d802a480 comp rx=0 tx=0).stop 2024-09-12T03:19:44.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.178+0000 7f41ec248640 1 -- 172.21.15.116:0/1152127380 shutdown_connections 2024-09-12T03:19:44.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.178+0000 7f41ec248640 1 --2- 172.21.15.116:0/1152127380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41e4102ef0 0x7f41e410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:44.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.178+0000 7f41ec248640 1 --2- 172.21.15.116:0/1152127380 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f41e4102570 0x7f41e41029b0 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:44.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.178+0000 7f41ec248640 1 --2- 172.21.15.116:0/1152127380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f41e4100e20 0x7f41e4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:44.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.178+0000 7f41ec248640 1 -- 172.21.15.116:0/1152127380 >> 172.21.15.116:0/1152127380 conn(0x7f41e40fc7d0 msgr2=0x7f41e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:44.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.178+0000 7f41ec248640 1 -- 172.21.15.116:0/1152127380 shutdown_connections 2024-09-12T03:19:44.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.178+0000 7f41ec248640 1 -- 172.21.15.116:0/1152127380 wait complete. 2024-09-12T03:19:44.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.179+0000 7f41ec248640 1 Processor -- start 2024-09-12T03:19:44.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.179+0000 7f41ec248640 1 -- start start 2024-09-12T03:19:44.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.179+0000 7f41ec248640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41e4100e20 0x7f41e4198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:44.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.179+0000 7f41ec248640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f41e4102570 0x7f41e4198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:44.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.179+0000 7f41ec248640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f41e4102ef0 0x7f41e419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:44.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.179+0000 7f41ec248640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f41e4110f50 con 0x7f41e4102570 2024-09-12T03:19:44.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.179+0000 7f41ec248640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f41e4110dd0 con 0x7f41e4102ef0 2024-09-12T03:19:44.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.179+0000 7f41ec248640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f41e41110d0 con 0x7f41e4100e20 2024-09-12T03:19:44.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.179+0000 7f41e97e3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f41e4102570 0x7f41e4198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:44.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.179+0000 7f41e97e3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f41e4102570 0x7f41e4198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32788/0 (socket says 172.21.15.116:32788) 2024-09-12T03:19:44.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.179+0000 7f41e97e3640 1 -- 172.21.15.116:0/803527366 learned_addr learned my addr 172.21.15.116:0/803527366 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:44.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.179+0000 7f41ea7e5640 1 --2- 172.21.15.116:0/803527366 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f41e4102ef0 0x7f41e419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:44.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.179+0000 7f41e9fe4640 1 --2- 172.21.15.116:0/803527366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41e4100e20 0x7f41e4198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:44.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.179+0000 7f41e97e3640 1 -- 172.21.15.116:0/803527366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41e4100e20 msgr2=0x7f41e4198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:44.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.179+0000 7f41e97e3640 1 --2- 172.21.15.116:0/803527366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41e4100e20 0x7f41e4198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:44.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.179+0000 7f41e97e3640 1 -- 172.21.15.116:0/803527366 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f41e4102ef0 msgr2=0x7f41e419ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:44.205 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.179+0000 7f41e97e3640 1 --2- 172.21.15.116:0/803527366 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f41e4102ef0 0x7f41e419ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:44.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.179+0000 7f41e97e3640 1 -- 172.21.15.116:0/803527366 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e419d500 con 0x7f41e4102570 2024-09-12T03:19:44.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.179+0000 7f41e97e3640 1 --2- 172.21.15.116:0/803527366 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f41e4102570 0x7f41e4198a90 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f41d8006010 tx=0x7f41d8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:44.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.180+0000 7f41e2ffd640 1 -- 172.21.15.116:0/803527366 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d803f070 con 0x7f41e4102570 2024-09-12T03:19:44.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.180+0000 7f41ec248640 1 -- 172.21.15.116:0/803527366 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41e419d790 con 0x7f41e4102570 2024-09-12T03:19:44.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.180+0000 7f41e2ffd640 1 -- 172.21.15.116:0/803527366 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41d8031c30 con 0x7f41e4102570 2024-09-12T03:19:44.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.180+0000 7f41e2ffd640 1 -- 172.21.15.116:0/803527366 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d8035030 con 0x7f41e4102570 2024-09-12T03:19:44.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.180+0000 7f41ec248640 1 -- 172.21.15.116:0/803527366 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41e41a50d0 con 0x7f41e4102570 2024-09-12T03:19:44.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.181+0000 7f41ec248640 1 -- 172.21.15.116:0/803527366 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41e4101200 con 0x7f41e4102570 2024-09-12T03:19:44.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.182+0000 7f41e2ffd640 1 -- 172.21.15.116:0/803527366 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f41d8042070 con 0x7f41e4102570 2024-09-12T03:19:44.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.183+0000 7f41e2ffd640 1 --2- 172.21.15.116:0/803527366 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f41cc0a2b80 0x7f41cc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:44.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.183+0000 7f41e9fe4640 1 --2- 172.21.15.116:0/803527366 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f41cc0a2b80 0x7f41cc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:44.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.183+0000 7f41e2ffd640 1 -- 172.21.15.116:0/803527366 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8440+0+0 (secure 0 0 0) 0x7f41d80efd90 con 0x7f41e4102570 2024-09-12T03:19:44.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.183+0000 7f41e9fe4640 1 --2- 172.21.15.116:0/803527366 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f41cc0a2b80 0x7f41cc0a5040 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f41dc009b60 tx=0x7f41dc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:44.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.186+0000 7f41e2ffd640 1 -- 172.21.15.116:0/803527366 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f41d80bcc20 con 0x7f41e4102570 2024-09-12T03:19:44.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:44.387+0000 7f41ec248640 1 -- 172.21.15.116:0/803527366 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f41e410a0a0 con 0x7f41e4102570 2024-09-12T03:19:44.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:44 smithi183 ceph-mon[35535]: pgmap v841: 97 pgs: 97 active+clean; 7.8 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2024-09-12T03:19:44.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:44 smithi183 ceph-mon[35535]: mds.? [v2:172.21.15.177:6834/3462591824,v1:172.21.15.177:6837/3462591824] up:boot 2024-09-12T03:19:44.953 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:44 smithi183 ceph-mon[35535]: mds.? [v2:172.21.15.177:6836/1200888138,v1:172.21.15.177:6839/1200888138] up:boot 2024-09-12T03:19:44.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:44 smithi183 ceph-mon[35535]: mds.? [v2:172.21.15.177:6835/1480767339,v1:172.21.15.177:6838/1480767339] up:boot 2024-09-12T03:19:44.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:44 smithi183 ceph-mon[35535]: mds.? [v2:172.21.15.183:6834/3509939389,v1:172.21.15.183:6835/3509939389] up:boot 2024-09-12T03:19:44.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:44 smithi183 ceph-mon[35535]: mds.? [v2:172.21.15.183:6836/3849598580,v1:172.21.15.183:6838/3849598580] up:boot 2024-09-12T03:19:44.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:44 smithi183 ceph-mon[35535]: mds.? [v2:172.21.15.183:6837/3061409579,v1:172.21.15.183:6839/3061409579] up:boot 2024-09-12T03:19:44.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:44 smithi183 ceph-mon[35535]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-12T03:19:44.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:44 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:19:44.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:44 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:19:44.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:44 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:19:44.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:44 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:19:44.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:44 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:19:44.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:44 smithi183 ceph-mon[35535]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:19:44.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:44 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/803527366' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:19:44.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:44 smithi177 ceph-mon[35305]: pgmap v841: 97 pgs: 97 active+clean; 7.8 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2024-09-12T03:19:44.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:44 smithi177 ceph-mon[35305]: mds.? [v2:172.21.15.177:6834/3462591824,v1:172.21.15.177:6837/3462591824] up:boot 2024-09-12T03:19:44.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:44 smithi177 ceph-mon[35305]: mds.? [v2:172.21.15.177:6836/1200888138,v1:172.21.15.177:6839/1200888138] up:boot 2024-09-12T03:19:44.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:44 smithi177 ceph-mon[35305]: mds.? [v2:172.21.15.177:6835/1480767339,v1:172.21.15.177:6838/1480767339] up:boot 2024-09-12T03:19:44.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:44 smithi177 ceph-mon[35305]: mds.? [v2:172.21.15.183:6834/3509939389,v1:172.21.15.183:6835/3509939389] up:boot 2024-09-12T03:19:44.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:44 smithi177 ceph-mon[35305]: mds.? [v2:172.21.15.183:6836/3849598580,v1:172.21.15.183:6838/3849598580] up:boot 2024-09-12T03:19:44.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:44 smithi177 ceph-mon[35305]: mds.? [v2:172.21.15.183:6837/3061409579,v1:172.21.15.183:6839/3061409579] up:boot 2024-09-12T03:19:44.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:44 smithi177 ceph-mon[35305]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-12T03:19:44.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:44 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:19:44.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:44 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:19:44.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:44 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:19:44.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:44 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:19:44.996 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:44 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:19:44.996 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:44 smithi177 ceph-mon[35305]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:19:44.996 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:44 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/803527366' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:19:45.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:44 smithi116 ceph-mon[31410]: pgmap v841: 97 pgs: 97 active+clean; 7.8 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2024-09-12T03:19:45.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:44 smithi116 ceph-mon[31410]: mds.? [v2:172.21.15.177:6834/3462591824,v1:172.21.15.177:6837/3462591824] up:boot 2024-09-12T03:19:45.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:44 smithi116 ceph-mon[31410]: mds.? [v2:172.21.15.177:6836/1200888138,v1:172.21.15.177:6839/1200888138] up:boot 2024-09-12T03:19:45.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:44 smithi116 ceph-mon[31410]: mds.? [v2:172.21.15.177:6835/1480767339,v1:172.21.15.177:6838/1480767339] up:boot 2024-09-12T03:19:45.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:44 smithi116 ceph-mon[31410]: mds.? [v2:172.21.15.183:6834/3509939389,v1:172.21.15.183:6835/3509939389] up:boot 2024-09-12T03:19:45.053 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:44 smithi116 ceph-mon[31410]: mds.? [v2:172.21.15.183:6836/3849598580,v1:172.21.15.183:6838/3849598580] up:boot 2024-09-12T03:19:45.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:44 smithi116 ceph-mon[31410]: mds.? [v2:172.21.15.183:6837/3061409579,v1:172.21.15.183:6839/3061409579] up:boot 2024-09-12T03:19:45.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:44 smithi116 ceph-mon[31410]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-12T03:19:45.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:44 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:19:45.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:44 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:19:45.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:44 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:19:45.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:44 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:19:45.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:44 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:19:45.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:44 smithi116 ceph-mon[31410]: from='mgr.14150 172.21.15.116:0/2233841128' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:19:45.054 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:44 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/803527366' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:19:45.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:45.180+0000 7f41e2ffd640 1 -- 172.21.15.116:0/803527366 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7f41d80bc520 con 0x7f41e4102570 2024-09-12T03:19:45.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:45.183+0000 7f41ec248640 1 -- 172.21.15.116:0/803527366 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f41cc0a2b80 msgr2=0x7f41cc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:45.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:45.183+0000 7f41ec248640 1 --2- 172.21.15.116:0/803527366 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f41cc0a2b80 0x7f41cc0a5040 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f41dc009b60 tx=0x7f41dc005f90 comp rx=0 tx=0).stop 2024-09-12T03:19:45.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:45.183+0000 7f41ec248640 1 -- 172.21.15.116:0/803527366 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f41e4102570 msgr2=0x7f41e4198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:45.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:45.183+0000 7f41ec248640 1 --2- 172.21.15.116:0/803527366 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f41e4102570 0x7f41e4198a90 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f41d8006010 tx=0x7f41d8003c10 comp rx=0 tx=0).stop 2024-09-12T03:19:45.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:45.183+0000 7f41ec248640 1 -- 172.21.15.116:0/803527366 shutdown_connections 2024-09-12T03:19:45.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:45.183+0000 7f41ec248640 1 --2- 172.21.15.116:0/803527366 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f41cc0a2b80 0x7f41cc0a5040 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:45.183+0000 7f41ec248640 1 --2- 172.21.15.116:0/803527366 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f41e4102ef0 0x7f41e419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:45.183+0000 7f41ec248640 1 --2- 172.21.15.116:0/803527366 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f41e4102570 0x7f41e4198a90 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:45.183+0000 7f41ec248640 1 --2- 172.21.15.116:0/803527366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41e4100e20 0x7f41e4198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:45.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:45.183+0000 7f41ec248640 1 -- 172.21.15.116:0/803527366 >> 172.21.15.116:0/803527366 conn(0x7f41e40fc7d0 msgr2=0x7f41e406a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:45.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:45.183+0000 7f41ec248640 1 -- 172.21.15.116:0/803527366 shutdown_connections 2024-09-12T03:19:45.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:45.183+0000 7f41ec248640 1 -- 172.21.15.116:0/803527366 wait complete. 2024-09-12T03:19:45.802 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:45 smithi116 ceph-mon[31410]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T03:19:45.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:45 smithi116 ceph-mon[31410]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T03:19:45.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:45 smithi116 ceph-mon[31410]: Cluster is now healthy 2024-09-12T03:19:45.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:45 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/803527366' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:19:45.803 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:45 smithi116 ceph-mon[31410]: fsmap 12 up:standby 2024-09-12T03:19:45.893 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T03:19:45.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:45 smithi177 ceph-mon[35305]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T03:19:45.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:45 smithi177 ceph-mon[35305]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T03:19:45.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:45 smithi177 ceph-mon[35305]: Cluster is now healthy 2024-09-12T03:19:45.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:45 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/803527366' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:19:45.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:45 smithi177 ceph-mon[35305]: fsmap 12 up:standby 2024-09-12T03:19:46.144 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:46.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:45 smithi183 ceph-mon[35535]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T03:19:46.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:45 smithi183 ceph-mon[35535]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T03:19:46.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:45 smithi183 ceph-mon[35535]: Cluster is now healthy 2024-09-12T03:19:46.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:45 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/803527366' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:19:46.203 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:45 smithi183 ceph-mon[35535]: fsmap 12 up:standby 2024-09-12T03:19:46.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.628+0000 7f98bdaed640 1 -- 172.21.15.116:0/322200081 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98b8102d40 msgr2=0x7f98b8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.628+0000 7f98bdaed640 1 --2- 172.21.15.116:0/322200081 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98b8102d40 0x7f98b8103120 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f98a8009ca0 tx=0x7f98a802a980 comp rx=0 tx=0).stop 2024-09-12T03:19:46.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.628+0000 7f98bdaed640 1 -- 172.21.15.116:0/322200081 shutdown_connections 2024-09-12T03:19:46.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.628+0000 7f98bdaed640 1 --2- 172.21.15.116:0/322200081 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98b810c170 0x7f98b810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.628+0000 7f98bdaed640 1 --2- 172.21.15.116:0/322200081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98b8103660 0x7f98b810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.628+0000 7f98bdaed640 1 --2- 172.21.15.116:0/322200081 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98b8102d40 0x7f98b8103120 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.628+0000 7f98bdaed640 1 -- 172.21.15.116:0/322200081 >> 172.21.15.116:0/322200081 conn(0x7f98b80fc7a0 msgr2=0x7f98b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:46.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.628+0000 7f98bdaed640 1 -- 172.21.15.116:0/322200081 shutdown_connections 2024-09-12T03:19:46.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.628+0000 7f98bdaed640 1 -- 172.21.15.116:0/322200081 wait complete. 2024-09-12T03:19:46.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.628+0000 7f98bdaed640 1 Processor -- start 2024-09-12T03:19:46.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.629+0000 7f98bdaed640 1 -- start start 2024-09-12T03:19:46.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.629+0000 7f98bdaed640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98b8102d40 0x7f98b81009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.629+0000 7f98bdaed640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98b8103660 0x7f98b8100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.629+0000 7f98bdaed640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98b810c170 0x7f98b8101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.629+0000 7f98bdaed640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f98b8112070 con 0x7f98b8102d40 2024-09-12T03:19:46.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.629+0000 7f98bdaed640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f98b8111ef0 con 0x7f98b810c170 2024-09-12T03:19:46.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.629+0000 7f98bdaed640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f98b81121f0 con 0x7f98b8103660 2024-09-12T03:19:46.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.629+0000 7f98b77fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98b8102d40 0x7f98b81009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.629+0000 7f98b77fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98b8102d40 0x7f98b81009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32812/0 (socket says 172.21.15.116:32812) 2024-09-12T03:19:46.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.629+0000 7f98b77fe640 1 -- 172.21.15.116:0/2656961674 learned_addr learned my addr 172.21.15.116:0/2656961674 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:46.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.629+0000 7f98b77fe640 1 -- 172.21.15.116:0/2656961674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98b8103660 msgr2=0x7f98b8100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.629+0000 7f98b77fe640 1 --2- 172.21.15.116:0/2656961674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98b8103660 0x7f98b8100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.629+0000 7f98b77fe640 1 -- 172.21.15.116:0/2656961674 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98b810c170 msgr2=0x7f98b8101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:46.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.629+0000 7f98b7fff640 1 --2- 172.21.15.116:0/2656961674 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98b810c170 0x7f98b8101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.629+0000 7f98b77fe640 1 --2- 172.21.15.116:0/2656961674 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98b810c170 0x7f98b8101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:46.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.629+0000 7f98b77fe640 1 -- 172.21.15.116:0/2656961674 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b81a6a20 con 0x7f98b8102d40 2024-09-12T03:19:46.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.629+0000 7f98b7fff640 1 --2- 172.21.15.116:0/2656961674 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98b810c170 0x7f98b8101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:46.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.630+0000 7f98b77fe640 1 --2- 172.21.15.116:0/2656961674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98b8102d40 0x7f98b81009a0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f98a8000f80 tx=0x7f98a80308b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:46.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.630+0000 7f98b4ff9640 1 -- 172.21.15.116:0/2656961674 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8034050 con 0x7f98b8102d40 2024-09-12T03:19:46.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.630+0000 7f98b4ff9640 1 -- 172.21.15.116:0/2656961674 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98a8003e20 con 0x7f98b8102d40 2024-09-12T03:19:46.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.630+0000 7f98b4ff9640 1 -- 172.21.15.116:0/2656961674 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a802e7c0 con 0x7f98b8102d40 2024-09-12T03:19:46.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.630+0000 7f98bdaed640 1 -- 172.21.15.116:0/2656961674 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98b81a6cb0 con 0x7f98b8102d40 2024-09-12T03:19:46.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.630+0000 7f98bdaed640 1 -- 172.21.15.116:0/2656961674 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98b81a7190 con 0x7f98b8102d40 2024-09-12T03:19:46.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.631+0000 7f98bdaed640 1 -- 172.21.15.116:0/2656961674 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98b8103120 con 0x7f98b8102d40 2024-09-12T03:19:46.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.632+0000 7f98b4ff9640 1 -- 172.21.15.116:0/2656961674 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f98a8037020 con 0x7f98b8102d40 2024-09-12T03:19:46.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.633+0000 7f98b4ff9640 1 --2- 172.21.15.116:0/2656961674 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f98a40a2c00 0x7f98a40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:46.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.633+0000 7f98b6ffd640 1 --2- 172.21.15.116:0/2656961674 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f98a40a2c00 0x7f98a40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:46.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.633+0000 7f98b4ff9640 1 -- 172.21.15.116:0/2656961674 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8440+0+0 (secure 0 0 0) 0x7f98a80f02c0 con 0x7f98b8102d40 2024-09-12T03:19:46.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.633+0000 7f98b6ffd640 1 --2- 172.21.15.116:0/2656961674 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f98a40a2c00 0x7f98a40a50c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f98a0006fd0 tx=0x7f98a0006ce0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:46.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.636+0000 7f98b4ff9640 1 -- 172.21.15.116:0/2656961674 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f98a80bd150 con 0x7f98b8102d40 2024-09-12T03:19:46.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:46.806+0000 7f98bdaed640 1 -- 172.21.15.116:0/2656961674 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f98b8101ed0 con 0x7f98b8102d40 2024-09-12T03:19:47.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:47.183+0000 7f98b4ff9640 1 -- 172.21.15.116:0/2656961674 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v82) ==== 181+0+0 (secure 0 0 0) 0x7f98a80098b0 con 0x7f98b8102d40 2024-09-12T03:19:47.183 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' removed 2024-09-12T03:19:47.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:47.186+0000 7f98bdaed640 1 -- 172.21.15.116:0/2656961674 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f98a40a2c00 msgr2=0x7f98a40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:47.186+0000 7f98bdaed640 1 --2- 172.21.15.116:0/2656961674 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f98a40a2c00 0x7f98a40a50c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f98a0006fd0 tx=0x7f98a0006ce0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:47.186+0000 7f98bdaed640 1 -- 172.21.15.116:0/2656961674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98b8102d40 msgr2=0x7f98b81009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:47.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:47.186+0000 7f98bdaed640 1 --2- 172.21.15.116:0/2656961674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98b8102d40 0x7f98b81009a0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f98a8000f80 tx=0x7f98a80308b0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:47.187+0000 7f98bdaed640 1 -- 172.21.15.116:0/2656961674 shutdown_connections 2024-09-12T03:19:47.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:47.187+0000 7f98bdaed640 1 --2- 172.21.15.116:0/2656961674 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f98a40a2c00 0x7f98a40a50c0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:47.187+0000 7f98bdaed640 1 --2- 172.21.15.116:0/2656961674 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98b810c170 0x7f98b8101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:47.187+0000 7f98bdaed640 1 --2- 172.21.15.116:0/2656961674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98b8103660 0x7f98b8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:47.187+0000 7f98bdaed640 1 --2- 172.21.15.116:0/2656961674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98b8102d40 0x7f98b81009a0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:47.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:47.187+0000 7f98bdaed640 1 -- 172.21.15.116:0/2656961674 >> 172.21.15.116:0/2656961674 conn(0x7f98b80fc7a0 msgr2=0x7f98b80fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:47.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:47.187+0000 7f98bdaed640 1 -- 172.21.15.116:0/2656961674 shutdown_connections 2024-09-12T03:19:47.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:47.187+0000 7f98bdaed640 1 -- 172.21.15.116:0/2656961674 wait complete. 2024-09-12T03:19:47.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:47 smithi183 ceph-mon[35535]: pgmap v842: 97 pgs: 97 active+clean; 7.8 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 5 op/s 2024-09-12T03:19:47.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:47 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2656961674' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:19:47.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:47 smithi177 ceph-mon[35305]: pgmap v842: 97 pgs: 97 active+clean; 7.8 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 5 op/s 2024-09-12T03:19:47.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:47 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2656961674' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:19:47.507 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:47 smithi116 ceph-mon[31410]: pgmap v842: 97 pgs: 97 active+clean; 7.8 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 5 op/s 2024-09-12T03:19:47.507 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:47 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2656961674' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:19:47.830 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T03:19:48.077 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:48.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:48 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/2656961674' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:19:48.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:48 smithi183 ceph-mon[35535]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T03:19:48.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:48 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/2656961674' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:19:48.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:48 smithi177 ceph-mon[35305]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T03:19:48.503 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:48 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/2656961674' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:19:48.503 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:48 smithi116 ceph-mon[31410]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T03:19:48.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.586+0000 7f05d7beb640 1 -- 172.21.15.116:0/641021900 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05d0102790 msgr2=0x7f05d01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:48.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.586+0000 7f05d7beb640 1 --2- 172.21.15.116:0/641021900 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05d0102790 0x7f05d01067e0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f05cc009900 tx=0x7f05cc02a410 comp rx=0 tx=0).stop 2024-09-12T03:19:48.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.586+0000 7f05d7beb640 1 -- 172.21.15.116:0/641021900 shutdown_connections 2024-09-12T03:19:48.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.586+0000 7f05d7beb640 1 --2- 172.21.15.116:0/641021900 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05d0106f10 0x7f05d010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:48.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.586+0000 7f05d7beb640 1 --2- 172.21.15.116:0/641021900 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05d0102790 0x7f05d01067e0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:48.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.586+0000 7f05d7beb640 1 --2- 172.21.15.116:0/641021900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05d0101de0 0x7f05d01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:48.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.586+0000 7f05d7beb640 1 -- 172.21.15.116:0/641021900 >> 172.21.15.116:0/641021900 conn(0x7f05d00fd8c0 msgr2=0x7f05d00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:48.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.586+0000 7f05d7beb640 1 -- 172.21.15.116:0/641021900 shutdown_connections 2024-09-12T03:19:48.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.587+0000 7f05d7beb640 1 -- 172.21.15.116:0/641021900 wait complete. 2024-09-12T03:19:48.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.588+0000 7f05d7beb640 1 Processor -- start 2024-09-12T03:19:48.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.588+0000 7f05d7beb640 1 -- start start 2024-09-12T03:19:48.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.588+0000 7f05d7beb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05d0101de0 0x7f05d0198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:48.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.588+0000 7f05d7beb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05d0102790 0x7f05d0198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:48.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.588+0000 7f05d7beb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05d0106f10 0x7f05d019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:48.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.588+0000 7f05d7beb640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f05d0111f80 con 0x7f05d0101de0 2024-09-12T03:19:48.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.588+0000 7f05d7beb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f05d0111e00 con 0x7f05d0102790 2024-09-12T03:19:48.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.588+0000 7f05d7beb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f05d0112100 con 0x7f05d0106f10 2024-09-12T03:19:48.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.588+0000 7f05d5987640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05d0101de0 0x7f05d0198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:48.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.588+0000 7f05d5987640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05d0101de0 0x7f05d0198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:32830/0 (socket says 172.21.15.116:32830) 2024-09-12T03:19:48.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.588+0000 7f05d5987640 1 -- 172.21.15.116:0/3184985560 learned_addr learned my addr 172.21.15.116:0/3184985560 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-12T03:19:48.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.588+0000 7f05d6188640 1 --2- 172.21.15.116:0/3184985560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05d0106f10 0x7f05d019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:48.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.588+0000 7f05d5186640 1 --2- 172.21.15.116:0/3184985560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05d0102790 0x7f05d0198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:48.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.589+0000 7f05d5987640 1 -- 172.21.15.116:0/3184985560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05d0106f10 msgr2=0x7f05d019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:48.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.589+0000 7f05d5987640 1 --2- 172.21.15.116:0/3184985560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05d0106f10 0x7f05d019ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:48.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.589+0000 7f05d5987640 1 -- 172.21.15.116:0/3184985560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05d0102790 msgr2=0x7f05d0198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:48.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.589+0000 7f05d5987640 1 --2- 172.21.15.116:0/3184985560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05d0102790 0x7f05d0198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:48.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.589+0000 7f05d5987640 1 -- 172.21.15.116:0/3184985560 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d019d3d0 con 0x7f05d0101de0 2024-09-12T03:19:48.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.589+0000 7f05d5987640 1 --2- 172.21.15.116:0/3184985560 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05d0101de0 0x7f05d0198420 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f05c400d950 tx=0x7f05c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:48.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.589+0000 7f05c2ffd640 1 -- 172.21.15.116:0/3184985560 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c4017030 con 0x7f05d0101de0 2024-09-12T03:19:48.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.589+0000 7f05d7beb640 1 -- 172.21.15.116:0/3184985560 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05d019d6c0 con 0x7f05d0101de0 2024-09-12T03:19:48.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.589+0000 7f05c2ffd640 1 -- 172.21.15.116:0/3184985560 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05c4004020 con 0x7f05d0101de0 2024-09-12T03:19:48.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.589+0000 7f05c2ffd640 1 -- 172.21.15.116:0/3184985560 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c40047a0 con 0x7f05d0101de0 2024-09-12T03:19:48.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.589+0000 7f05d7beb640 1 -- 172.21.15.116:0/3184985560 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05d01a4fa0 con 0x7f05d0101de0 2024-09-12T03:19:48.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.590+0000 7f05d7beb640 1 -- 172.21.15.116:0/3184985560 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05d01021c0 con 0x7f05d0101de0 2024-09-12T03:19:48.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.591+0000 7f05c2ffd640 1 -- 172.21.15.116:0/3184985560 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f05c40041d0 con 0x7f05d0101de0 2024-09-12T03:19:48.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.592+0000 7f05c2ffd640 1 --2- 172.21.15.116:0/3184985560 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f05bc0a2b80 0x7f05bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:48.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.592+0000 7f05d5186640 1 --2- 172.21.15.116:0/3184985560 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f05bc0a2b80 0x7f05bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:48.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.592+0000 7f05c2ffd640 1 -- 172.21.15.116:0/3184985560 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7944+0+0 (secure 0 0 0) 0x7f05c40ce810 con 0x7f05d0101de0 2024-09-12T03:19:48.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.592+0000 7f05d5186640 1 --2- 172.21.15.116:0/3184985560 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f05bc0a2b80 0x7f05bc0a5040 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f05cc000c00 tx=0x7f05cc005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:48.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.596+0000 7f05c2ffd640 1 -- 172.21.15.116:0/3184985560 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f05c409b8e0 con 0x7f05d0101de0 2024-09-12T03:19:48.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:48.765+0000 7f05d7beb640 1 -- 172.21.15.116:0/3184985560 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f05d010a150 con 0x7f05d0101de0 2024-09-12T03:19:49.196 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:49.195+0000 7f05c2ffd640 1 -- 172.21.15.116:0/3184985560 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v83) ==== 169+0+0 (secure 0 0 0) 0x7f05c40b9e20 con 0x7f05d0101de0 2024-09-12T03:19:49.196 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' removed 2024-09-12T03:19:49.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:49.199+0000 7f05d7beb640 1 -- 172.21.15.116:0/3184985560 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f05bc0a2b80 msgr2=0x7f05bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:49.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:49.199+0000 7f05d7beb640 1 --2- 172.21.15.116:0/3184985560 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f05bc0a2b80 0x7f05bc0a5040 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f05cc000c00 tx=0x7f05cc005b50 comp rx=0 tx=0).stop 2024-09-12T03:19:49.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:49.199+0000 7f05d7beb640 1 -- 172.21.15.116:0/3184985560 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05d0101de0 msgr2=0x7f05d0198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:49.199 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:49.199+0000 7f05d7beb640 1 --2- 172.21.15.116:0/3184985560 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05d0101de0 0x7f05d0198420 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f05c400d950 tx=0x7f05c400de10 comp rx=0 tx=0).stop 2024-09-12T03:19:49.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:49.199+0000 7f05d7beb640 1 -- 172.21.15.116:0/3184985560 shutdown_connections 2024-09-12T03:19:49.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:49.199+0000 7f05d7beb640 1 --2- 172.21.15.116:0/3184985560 >> [v2:172.21.15.116:6800/2896038008,v1:172.21.15.116:6801/2896038008] conn(0x7f05bc0a2b80 0x7f05bc0a5040 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:49.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:49.199+0000 7f05d7beb640 1 --2- 172.21.15.116:0/3184985560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05d0106f10 0x7f05d019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:49.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:49.199+0000 7f05d7beb640 1 --2- 172.21.15.116:0/3184985560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05d0102790 0x7f05d0198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:49.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:49.199+0000 7f05d7beb640 1 --2- 172.21.15.116:0/3184985560 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05d0101de0 0x7f05d0198420 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:49.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:49.199+0000 7f05d7beb640 1 -- 172.21.15.116:0/3184985560 >> 172.21.15.116:0/3184985560 conn(0x7f05d00fd8c0 msgr2=0x7f05d00fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:49.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:49.199+0000 7f05d7beb640 1 -- 172.21.15.116:0/3184985560 shutdown_connections 2024-09-12T03:19:49.200 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-12T03:19:49.199+0000 7f05d7beb640 1 -- 172.21.15.116:0/3184985560 wait complete. 2024-09-12T03:19:49.452 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:49 smithi183 ceph-mon[35535]: pgmap v844: 65 pgs: 65 active+clean; 577 KiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:19:49.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:49 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3184985560' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:19:49.494 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:49 smithi177 ceph-mon[35305]: pgmap v844: 65 pgs: 65 active+clean; 577 KiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:19:49.495 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:49 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3184985560' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:19:49.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:49 smithi116 ceph-mon[31410]: pgmap v844: 65 pgs: 65 active+clean; 577 KiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:19:49.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:49 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3184985560' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:19:49.913 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:19:49.940 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:19:49.969 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:19:49.999 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-12T03:19:50.000 DEBUG:teuthology.orchestra.run.smithi116:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T03:19:50.015 DEBUG:teuthology.orchestra.run.smithi177:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T03:19:50.031 DEBUG:teuthology.orchestra.run.smithi183:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T03:19:50.056 INFO:tasks.cephadm:Stopping all daemons... 2024-09-12T03:19:50.056 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-12T03:19:50.056 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mon.a 2024-09-12T03:19:50.205 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:50 smithi116 systemd[1]: Stopping Ceph mon.a for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:19:50.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:50 smithi177 ceph-mon[35305]: from='client.? 172.21.15.116:0/3184985560' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:19:50.244 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:50 smithi177 ceph-mon[35305]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T03:19:50.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:50 smithi183 ceph-mon[35535]: from='client.? 172.21.15.116:0/3184985560' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:19:50.453 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:50 smithi183 ceph-mon[35535]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T03:19:50.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:50 smithi116 ceph-mon[31410]: from='client.? 172.21.15.116:0/3184985560' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:19:50.552 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:50 smithi116 ceph-mon[31410]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T03:19:50.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:50 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mon-a[31406]: 2024-09-12T03:19:50.337+0000 7ff9a7770640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:19:50.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:50 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mon-a[31406]: 2024-09-12T03:19:50.337+0000 7ff9a7770640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-12T03:19:50.553 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:50 smithi116 podman[128983]: 2024-09-12 03:19:50.358966343 +0000 UTC m=+0.114911690 container died 3479f0eb634584799d95b5090fc6e44659aea28a4a22f04fc967e4c3ba35010a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-12T03:19:51.030 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:50 smithi116 podman[128983]: 2024-09-12 03:19:50.729416333 +0000 UTC m=+0.485361664 container cleanup 3479f0eb634584799d95b5090fc6e44659aea28a4a22f04fc967e4c3ba35010a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-12T03:19:51.030 INFO:journalctl@ceph.mon.a.smithi116.stdout:Sep 12 03:19:50 smithi116 bash[128983]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mon-a 2024-09-12T03:19:51.206 DEBUG:teuthology.orchestra.run.smithi116:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mon.a.service' 2024-09-12T03:19:51.250 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:19:51.250 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-12T03:19:51.250 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-12T03:19:51.251 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mon.b 2024-09-12T03:19:51.577 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:51 smithi177 systemd[1]: Stopping Ceph mon.b for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:19:51.994 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:51 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mon-b[35301]: 2024-09-12T03:19:51.577+0000 7fc029997640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:19:51.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:51 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mon-b[35301]: 2024-09-12T03:19:51.577+0000 7fc029997640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-12T03:19:51.995 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:51 smithi177 podman[70273]: 2024-09-12 03:19:51.761048835 +0000 UTC m=+0.313075946 container died 0b4bfaf4a88e1114e9bae344286a70d7e317ba37741fa287e339718b46bc8590 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mon-b, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-12T03:19:52.437 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:52 smithi177 podman[70273]: 2024-09-12 03:19:52.16116865 +0000 UTC m=+0.713195760 container cleanup 0b4bfaf4a88e1114e9bae344286a70d7e317ba37741fa287e339718b46bc8590 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mon-b, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-12T03:19:52.437 INFO:journalctl@ceph.mon.b.smithi177.stdout:Sep 12 03:19:52 smithi177 bash[70273]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mon-b 2024-09-12T03:19:52.589 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mon.b.service' 2024-09-12T03:19:52.636 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:19:52.637 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-12T03:19:52.637 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-12T03:19:52.637 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mon.c 2024-09-12T03:19:52.952 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:52 smithi183 systemd[1]: Stopping Ceph mon.c for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:19:53.362 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:52 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mon-c[35531]: 2024-09-12T03:19:52.969+0000 7f9efda3e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:19:53.362 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:52 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mon-c[35531]: 2024-09-12T03:19:52.969+0000 7f9efda3e640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-12T03:19:53.362 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:53 smithi183 podman[71733]: 2024-09-12 03:19:53.072555431 +0000 UTC m=+0.237912914 container died 05ceb6fdd0c14208962f51656693be07d7af73ae03f3c33494b5f138fd346600 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mon-c, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-12T03:19:53.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:53 smithi183 podman[71733]: 2024-09-12 03:19:53.462644022 +0000 UTC m=+0.628001506 container cleanup 05ceb6fdd0c14208962f51656693be07d7af73ae03f3c33494b5f138fd346600 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mon-c, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-12T03:19:53.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 12 03:19:53 smithi183 bash[71733]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mon-c 2024-09-12T03:19:53.951 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mon.c.service' 2024-09-12T03:19:54.000 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:19:54.000 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-12T03:19:54.000 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-12T03:19:54.000 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mgr.x 2024-09-12T03:19:54.302 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 03:19:54 smithi116 systemd[1]: Stopping Ceph mgr.x for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:19:54.634 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 03:19:54 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T03:19:54.316+0000 7fc380ed6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:19:54.634 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 03:19:54 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x[31636]: 2024-09-12T03:19:54.316+0000 7fc380ed6640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-12T03:19:54.634 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 03:19:54 smithi116 podman[129101]: 2024-09-12 03:19:54.348188841 +0000 UTC m=+0.147555537 container died 3b0b103b7373583925494cda22ad0e7c824339f5c2a54aadcfba3a8cdd3b52ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-12T03:19:55.009 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 03:19:54 smithi116 podman[129101]: 2024-09-12 03:19:54.725502641 +0000 UTC m=+0.524869333 container cleanup 3b0b103b7373583925494cda22ad0e7c824339f5c2a54aadcfba3a8cdd3b52ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-12T03:19:55.009 INFO:journalctl@ceph.mgr.x.smithi116.stdout:Sep 12 03:19:54 smithi116 bash[129101]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-x 2024-09-12T03:19:55.153 DEBUG:teuthology.orchestra.run.smithi116:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-12T03:19:55.189 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:19:55.189 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-12T03:19:55.189 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-12T03:19:55.189 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mgr.y 2024-09-12T03:19:55.494 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 03:19:55 smithi177 systemd[1]: Stopping Ceph mgr.y for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:19:55.833 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 03:19:55 smithi177 podman[70394]: 2024-09-12 03:19:55.53903911 +0000 UTC m=+0.149748878 container died e90c4be67b8a61d1077c97f1607c9f22b3c6d3b6322bbe65600c84dba3c8c1d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-12T03:19:56.167 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 03:19:55 smithi177 podman[70394]: 2024-09-12 03:19:55.958459523 +0000 UTC m=+0.569169289 container cleanup e90c4be67b8a61d1077c97f1607c9f22b3c6d3b6322bbe65600c84dba3c8c1d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-12T03:19:56.167 INFO:journalctl@ceph.mgr.y.smithi177.stdout:Sep 12 03:19:55 smithi177 bash[70394]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-y 2024-09-12T03:19:56.382 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-12T03:19:56.416 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:19:56.417 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-12T03:19:56.417 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-12T03:19:56.417 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mgr.z 2024-09-12T03:19:56.703 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 03:19:56 smithi183 systemd[1]: Stopping Ceph mgr.z for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:19:57.010 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 03:19:56 smithi183 podman[71860]: 2024-09-12 03:19:56.738778022 +0000 UTC m=+0.117498144 container died 74021d43cead5c866c9a20949dc0bdb48bb033987342011401155c05b1ffd400 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:19:57.394 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 03:19:57 smithi183 podman[71860]: 2024-09-12 03:19:57.10148264 +0000 UTC m=+0.480202761 container cleanup 74021d43cead5c866c9a20949dc0bdb48bb033987342011401155c05b1ffd400 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3) 2024-09-12T03:19:57.394 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 12 03:19:57 smithi183 bash[71860]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mgr-z 2024-09-12T03:19:57.521 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-12T03:19:57.556 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:19:57.557 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-12T03:19:57.557 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-12T03:19:57.557 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.0 2024-09-12T03:19:57.863 INFO:journalctl@ceph.osd.0.smithi116.stdout:Sep 12 03:19:57 smithi116 systemd[1]: Stopping Ceph osd.0 for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:19:58.303 INFO:journalctl@ceph.osd.0.smithi116.stdout:Sep 12 03:19:57 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-0[45396]: 2024-09-12T03:19:57.862+0000 7f02b34eb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:19:58.303 INFO:journalctl@ceph.osd.0.smithi116.stdout:Sep 12 03:19:57 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-0[45396]: 2024-09-12T03:19:57.862+0000 7f02b34eb640 -1 osd.0 83 *** Got signal Terminated *** 2024-09-12T03:19:58.303 INFO:journalctl@ceph.osd.0.smithi116.stdout:Sep 12 03:19:57 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-0[45396]: 2024-09-12T03:19:57.862+0000 7f02b34eb640 -1 osd.0 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:20:03.689 INFO:journalctl@ceph.osd.0.smithi116.stdout:Sep 12 03:20:03 smithi116 podman[129225]: 2024-09-12 03:20:03.374821426 +0000 UTC m=+5.629154034 container died 69274d5ee1fdd1e31914e49577905816a25ab0fa65c1dfd53431c024aebc3636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:20:04.052 INFO:journalctl@ceph.osd.0.smithi116.stdout:Sep 12 03:20:03 smithi116 podman[129225]: 2024-09-12 03:20:03.888689363 +0000 UTC m=+6.143022006 container cleanup 69274d5ee1fdd1e31914e49577905816a25ab0fa65c1dfd53431c024aebc3636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:20:04.053 INFO:journalctl@ceph.osd.0.smithi116.stdout:Sep 12 03:20:03 smithi116 bash[129225]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-0 2024-09-12T03:20:04.541 INFO:journalctl@ceph.osd.0.smithi116.stdout:Sep 12 03:20:04 smithi116 podman[129242]: 2024-09-12 03:20:04.13198651 +0000 UTC m=+0.752210728 container remove 69274d5ee1fdd1e31914e49577905816a25ab0fa65c1dfd53431c024aebc3636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-12T03:20:04.802 INFO:journalctl@ceph.osd.0.smithi116.stdout:Sep 12 03:20:04 smithi116 podman[129331]: 2024-09-12 03:20:04.44504487 +0000 UTC m=+0.015997267 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:20:04.802 INFO:journalctl@ceph.osd.0.smithi116.stdout:Sep 12 03:20:04 smithi116 podman[129331]: 2024-09-12 03:20:04.584775751 +0000 UTC m=+0.155728135 container create 2a17933692a455022d22336f71ce763cac9e6e395cf29a9731ce097f3c721fb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-12T03:20:05.302 INFO:journalctl@ceph.osd.0.smithi116.stdout:Sep 12 03:20:04 smithi116 podman[129331]: 2024-09-12 03:20:04.843777934 +0000 UTC m=+0.414730318 container init 2a17933692a455022d22336f71ce763cac9e6e395cf29a9731ce097f3c721fb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-12T03:20:05.302 INFO:journalctl@ceph.osd.0.smithi116.stdout:Sep 12 03:20:04 smithi116 podman[129331]: 2024-09-12 03:20:04.84732313 +0000 UTC m=+0.418275525 container start 2a17933692a455022d22336f71ce763cac9e6e395cf29a9731ce097f3c721fb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-12T03:20:05.302 INFO:journalctl@ceph.osd.0.smithi116.stdout:Sep 12 03:20:04 smithi116 podman[129331]: 2024-09-12 03:20:04.952381084 +0000 UTC m=+0.523333482 container attach 2a17933692a455022d22336f71ce763cac9e6e395cf29a9731ce097f3c721fb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-12T03:20:05.302 INFO:journalctl@ceph.osd.0.smithi116.stdout:Sep 12 03:20:05 smithi116 podman[129331]: 2024-09-12 03:20:05.039087717 +0000 UTC m=+0.610040116 container died 2a17933692a455022d22336f71ce763cac9e6e395cf29a9731ce097f3c721fb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-12T03:20:05.724 DEBUG:teuthology.orchestra.run.smithi116:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.0.service' 2024-09-12T03:20:05.764 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:20:05.765 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-12T03:20:05.766 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-12T03:20:05.766 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.1 2024-09-12T03:20:06.087 INFO:journalctl@ceph.osd.1.smithi177.stdout:Sep 12 03:20:05 smithi177 systemd[1]: Stopping Ceph osd.1 for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:20:06.494 INFO:journalctl@ceph.osd.1.smithi177.stdout:Sep 12 03:20:06 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-1[39769]: 2024-09-12T03:20:06.086+0000 7ff5be374640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:20:06.494 INFO:journalctl@ceph.osd.1.smithi177.stdout:Sep 12 03:20:06 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-1[39769]: 2024-09-12T03:20:06.086+0000 7ff5be374640 -1 osd.1 83 *** Got signal Terminated *** 2024-09-12T03:20:06.494 INFO:journalctl@ceph.osd.1.smithi177.stdout:Sep 12 03:20:06 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-1[39769]: 2024-09-12T03:20:06.086+0000 7ff5be374640 -1 osd.1 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:20:11.862 INFO:journalctl@ceph.osd.1.smithi177.stdout:Sep 12 03:20:11 smithi177 podman[70524]: 2024-09-12 03:20:11.554560202 +0000 UTC m=+5.592003311 container died 9f6b17aca9fd600c9ae85703aa0ab7cb2b4ad195023f4bfb10c3dd1948e67241 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-1, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:20:12.221 INFO:journalctl@ceph.osd.1.smithi177.stdout:Sep 12 03:20:11 smithi177 podman[70524]: 2024-09-12 03:20:11.961953897 +0000 UTC m=+5.999396993 container cleanup 9f6b17aca9fd600c9ae85703aa0ab7cb2b4ad195023f4bfb10c3dd1948e67241 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-12T03:20:12.221 INFO:journalctl@ceph.osd.1.smithi177.stdout:Sep 12 03:20:11 smithi177 bash[70524]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-1 2024-09-12T03:20:12.494 INFO:journalctl@ceph.osd.1.smithi177.stdout:Sep 12 03:20:12 smithi177 podman[70540]: 2024-09-12 03:20:12.222040129 +0000 UTC m=+0.667688579 container remove 9f6b17aca9fd600c9ae85703aa0ab7cb2b4ad195023f4bfb10c3dd1948e67241 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-1, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:20:12.891 INFO:journalctl@ceph.osd.1.smithi177.stdout:Sep 12 03:20:12 smithi177 podman[70631]: 2024-09-12 03:20:12.535334994 +0000 UTC m=+0.015597378 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:20:12.892 INFO:journalctl@ceph.osd.1.smithi177.stdout:Sep 12 03:20:12 smithi177 podman[70631]: 2024-09-12 03:20:12.647840802 +0000 UTC m=+0.128103170 container create b431a4dd86a6adac616004ae3dbe61115f56eb463081822566888e8f8d9b6494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-12T03:20:13.245 INFO:journalctl@ceph.osd.1.smithi177.stdout:Sep 12 03:20:12 smithi177 podman[70631]: 2024-09-12 03:20:12.892397679 +0000 UTC m=+0.372660055 container init b431a4dd86a6adac616004ae3dbe61115f56eb463081822566888e8f8d9b6494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-12T03:20:13.245 INFO:journalctl@ceph.osd.1.smithi177.stdout:Sep 12 03:20:12 smithi177 podman[70631]: 2024-09-12 03:20:12.895950387 +0000 UTC m=+0.376212756 container start b431a4dd86a6adac616004ae3dbe61115f56eb463081822566888e8f8d9b6494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3) 2024-09-12T03:20:13.245 INFO:journalctl@ceph.osd.1.smithi177.stdout:Sep 12 03:20:13 smithi177 podman[70631]: 2024-09-12 03:20:13.00070518 +0000 UTC m=+0.480967566 container attach b431a4dd86a6adac616004ae3dbe61115f56eb463081822566888e8f8d9b6494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:20:13.246 INFO:journalctl@ceph.osd.1.smithi177.stdout:Sep 12 03:20:13 smithi177 podman[70631]: 2024-09-12 03:20:13.091712573 +0000 UTC m=+0.571974954 container died b431a4dd86a6adac616004ae3dbe61115f56eb463081822566888e8f8d9b6494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-12T03:20:13.773 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.1.service' 2024-09-12T03:20:13.813 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:20:13.813 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-12T03:20:13.813 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-12T03:20:13.813 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.2 2024-09-12T03:20:14.121 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 03:20:13 smithi183 systemd[1]: Stopping Ceph osd.2 for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:20:14.454 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 03:20:14 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-2[40517]: 2024-09-12T03:20:14.121+0000 7f566981c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:20:14.454 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 03:20:14 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-2[40517]: 2024-09-12T03:20:14.121+0000 7f566981c640 -1 osd.2 83 *** Got signal Terminated *** 2024-09-12T03:20:14.454 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 03:20:14 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-2[40517]: 2024-09-12T03:20:14.121+0000 7f566981c640 -1 osd.2 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:20:19.860 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 03:20:19 smithi183 podman[71988]: 2024-09-12 03:20:19.604675123 +0000 UTC m=+5.592521815 container died 58496194ce45d3d45974ffb03cc881a9b1dcea196b6d2ac95d415a3915c5d224 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2) 2024-09-12T03:20:20.203 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 03:20:19 smithi183 podman[71988]: 2024-09-12 03:20:19.988289291 +0000 UTC m=+5.976135982 container cleanup 58496194ce45d3d45974ffb03cc881a9b1dcea196b6d2ac95d415a3915c5d224 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-12T03:20:20.203 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 03:20:19 smithi183 bash[71988]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-2 2024-09-12T03:20:20.595 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 03:20:20 smithi183 podman[72004]: 2024-09-12 03:20:20.247944547 +0000 UTC m=+0.639603631 container remove 58496194ce45d3d45974ffb03cc881a9b1dcea196b6d2ac95d415a3915c5d224 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-12T03:20:20.884 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 03:20:20 smithi183 podman[72093]: 2024-09-12 03:20:20.4983114 +0000 UTC m=+0.015798328 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:20:20.884 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 03:20:20 smithi183 podman[72093]: 2024-09-12 03:20:20.632582266 +0000 UTC m=+0.150069223 container create a2c96c1027121b772b58d31dd8e26529324b5f3ce5f88bda74a51e95d48da20f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-2-deactivate, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:20:21.203 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 03:20:20 smithi183 podman[72093]: 2024-09-12 03:20:20.88480778 +0000 UTC m=+0.402294700 container init a2c96c1027121b772b58d31dd8e26529324b5f3ce5f88bda74a51e95d48da20f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:20:21.203 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 03:20:20 smithi183 podman[72093]: 2024-09-12 03:20:20.888415731 +0000 UTC m=+0.405902648 container start a2c96c1027121b772b58d31dd8e26529324b5f3ce5f88bda74a51e95d48da20f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:20:21.203 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 03:20:20 smithi183 podman[72093]: 2024-09-12 03:20:20.984871315 +0000 UTC m=+0.502358237 container attach a2c96c1027121b772b58d31dd8e26529324b5f3ce5f88bda74a51e95d48da20f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-12T03:20:21.203 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 12 03:20:21 smithi183 podman[72093]: 2024-09-12 03:20:21.069590512 +0000 UTC m=+0.587077426 container died a2c96c1027121b772b58d31dd8e26529324b5f3ce5f88bda74a51e95d48da20f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-2-deactivate, RELEASE=squid-b53b6d3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-12T03:20:21.665 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.2.service' 2024-09-12T03:20:21.705 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:20:21.705 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-12T03:20:21.705 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-12T03:20:21.705 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.3 2024-09-12T03:20:22.016 INFO:journalctl@ceph.osd.3.smithi116.stdout:Sep 12 03:20:21 smithi116 systemd[1]: Stopping Ceph osd.3 for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:20:22.302 INFO:journalctl@ceph.osd.3.smithi116.stdout:Sep 12 03:20:22 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-3[52293]: 2024-09-12T03:20:22.016+0000 7fe483c76640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:20:22.302 INFO:journalctl@ceph.osd.3.smithi116.stdout:Sep 12 03:20:22 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-3[52293]: 2024-09-12T03:20:22.016+0000 7fe483c76640 -1 osd.3 83 *** Got signal Terminated *** 2024-09-12T03:20:22.302 INFO:journalctl@ceph.osd.3.smithi116.stdout:Sep 12 03:20:22 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-3[52293]: 2024-09-12T03:20:22.016+0000 7fe483c76640 -1 osd.3 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:20:24.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:23 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:23.875+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:24.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:24 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:24.212+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:58.029532+0000 front 2024-09-12T03:19:58.029595+0000 (oldest deadline 2024-09-12T03:20:23.929695+0000) 2024-09-12T03:20:24.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:24 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:24.376+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:25.202 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:24 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:24.904+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:25.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:25 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:25.257+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:58.029532+0000 front 2024-09-12T03:19:58.029595+0000 (oldest deadline 2024-09-12T03:20:23.929695+0000) 2024-09-12T03:20:25.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:25 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:25.374+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:25.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:25 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:25.371+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:25.703 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:25 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:25.471+0000 7fef046e2640 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:59.341114+0000 front 2024-09-12T03:19:59.341286+0000 (oldest deadline 2024-09-12T03:20:24.641054+0000) 2024-09-12T03:20:26.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:25 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:25.887+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:26.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:26 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:26.353+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:26.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:26 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:26.234+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:58.029532+0000 front 2024-09-12T03:19:58.029595+0000 (oldest deadline 2024-09-12T03:20:23.929695+0000) 2024-09-12T03:20:26.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:26 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:26.416+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:26.703 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:26 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:26.498+0000 7fef046e2640 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:59.341114+0000 front 2024-09-12T03:19:59.341286+0000 (oldest deadline 2024-09-12T03:20:24.641054+0000) 2024-09-12T03:20:26.994 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:26 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:26.648+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:27.202 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:26 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:26.857+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:27.250 INFO:journalctl@ceph.osd.3.smithi116.stdout:Sep 12 03:20:26 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-3[52293]: 2024-09-12T03:20:26.923+0000 7fe48028c640 -1 osd.3 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.392139+0000 front 2024-09-12T03:20:02.392169+0000 (oldest deadline 2024-09-12T03:20:26.492280+0000) 2024-09-12T03:20:27.554 INFO:journalctl@ceph.osd.3.smithi116.stdout:Sep 12 03:20:27 smithi116 podman[129434]: 2024-09-12 03:20:27.442934839 +0000 UTC m=+5.541118774 container died 9c2d49b0330eac978a12278e5b480deae5b4660c03748cf2d949b3542f70ac38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-3, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:20:27.554 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:27 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:27.250+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:58.029532+0000 front 2024-09-12T03:19:58.029595+0000 (oldest deadline 2024-09-12T03:20:23.929695+0000) 2024-09-12T03:20:27.555 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:27 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:27.333+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:27.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:27 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:27.383+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:27.703 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:27 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:27.494+0000 7fef046e2640 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:59.341114+0000 front 2024-09-12T03:19:59.341286+0000 (oldest deadline 2024-09-12T03:20:24.641054+0000) 2024-09-12T03:20:27.994 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:27 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:27.671+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:28.052 INFO:journalctl@ceph.osd.3.smithi116.stdout:Sep 12 03:20:27 smithi116 podman[129434]: 2024-09-12 03:20:27.950212407 +0000 UTC m=+6.048396341 container cleanup 9c2d49b0330eac978a12278e5b480deae5b4660c03748cf2d949b3542f70ac38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:20:28.052 INFO:journalctl@ceph.osd.3.smithi116.stdout:Sep 12 03:20:27 smithi116 bash[129434]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-3 2024-09-12T03:20:28.202 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:27 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:27.867+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:28.521 INFO:journalctl@ceph.osd.3.smithi116.stdout:Sep 12 03:20:28 smithi116 podman[129461]: 2024-09-12 03:20:28.2100342 +0000 UTC m=+0.756499430 container remove 9c2d49b0330eac978a12278e5b480deae5b4660c03748cf2d949b3542f70ac38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-12T03:20:28.521 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:28 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:28.200+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:58.029532+0000 front 2024-09-12T03:19:58.029595+0000 (oldest deadline 2024-09-12T03:20:23.929695+0000) 2024-09-12T03:20:28.521 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:28 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:28.369+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:28.702 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:28 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:28.337+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:28.703 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:28 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:28.464+0000 7fef046e2640 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:59.341114+0000 front 2024-09-12T03:19:59.341286+0000 (oldest deadline 2024-09-12T03:20:24.641054+0000) 2024-09-12T03:20:28.796 INFO:journalctl@ceph.osd.3.smithi116.stdout:Sep 12 03:20:28 smithi116 podman[129551]: 2024-09-12 03:20:28.424589343 +0000 UTC m=+0.015956937 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:20:28.797 INFO:journalctl@ceph.osd.3.smithi116.stdout:Sep 12 03:20:28 smithi116 podman[129551]: 2024-09-12 03:20:28.53980223 +0000 UTC m=+0.131169822 container create 051a3a3ea7ebe9685f345b2604036fe3bee34237f7c91e692ea746a308cdc606 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-12T03:20:28.994 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:28 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:28.686+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:29.052 INFO:journalctl@ceph.osd.3.smithi116.stdout:Sep 12 03:20:28 smithi116 podman[129551]: 2024-09-12 03:20:28.797134212 +0000 UTC m=+0.388501805 container init 051a3a3ea7ebe9685f345b2604036fe3bee34237f7c91e692ea746a308cdc606 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-12T03:20:29.052 INFO:journalctl@ceph.osd.3.smithi116.stdout:Sep 12 03:20:28 smithi116 podman[129551]: 2024-09-12 03:20:28.80052018 +0000 UTC m=+0.391887782 container start 051a3a3ea7ebe9685f345b2604036fe3bee34237f7c91e692ea746a308cdc606 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-b53b6d3, GIT_CLEAN=True) 2024-09-12T03:20:29.052 INFO:journalctl@ceph.osd.3.smithi116.stdout:Sep 12 03:20:28 smithi116 podman[129551]: 2024-09-12 03:20:28.872102187 +0000 UTC m=+0.463469787 container attach 051a3a3ea7ebe9685f345b2604036fe3bee34237f7c91e692ea746a308cdc606 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3) 2024-09-12T03:20:29.053 INFO:journalctl@ceph.osd.3.smithi116.stdout:Sep 12 03:20:28 smithi116 podman[129551]: 2024-09-12 03:20:28.985383174 +0000 UTC m=+0.576750769 container died 051a3a3ea7ebe9685f345b2604036fe3bee34237f7c91e692ea746a308cdc606 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-12T03:20:29.202 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:28 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:28.835+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:29.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:29 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:29.196+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:58.029532+0000 front 2024-09-12T03:19:58.029595+0000 (oldest deadline 2024-09-12T03:20:23.929695+0000) 2024-09-12T03:20:29.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:29 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:29.412+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:29.702 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:29 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:29.311+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:29.703 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:29 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:29.512+0000 7fef046e2640 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:59.341114+0000 front 2024-09-12T03:19:59.341286+0000 (oldest deadline 2024-09-12T03:20:24.641054+0000) 2024-09-12T03:20:29.729 DEBUG:teuthology.orchestra.run.smithi116:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.3.service' 2024-09-12T03:20:29.770 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:20:29.770 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-12T03:20:29.770 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-12T03:20:29.770 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.4 2024-09-12T03:20:29.994 INFO:journalctl@ceph.osd.4.smithi177.stdout:Sep 12 03:20:29 smithi177 systemd[1]: Stopping Ceph osd.4 for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:20:29.994 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:29 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:29.706+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:30.202 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:29 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:29.839+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:30.494 INFO:journalctl@ceph.osd.4.smithi177.stdout:Sep 12 03:20:30 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-4[46254]: 2024-09-12T03:20:30.073+0000 7fdb5dbbf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:20:30.494 INFO:journalctl@ceph.osd.4.smithi177.stdout:Sep 12 03:20:30 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-4[46254]: 2024-09-12T03:20:30.073+0000 7fdb5dbbf640 -1 osd.4 83 *** Got signal Terminated *** 2024-09-12T03:20:30.495 INFO:journalctl@ceph.osd.4.smithi177.stdout:Sep 12 03:20:30 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-4[46254]: 2024-09-12T03:20:30.073+0000 7fdb5dbbf640 -1 osd.4 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:20:30.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:30 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:30.216+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:58.029532+0000 front 2024-09-12T03:19:58.029595+0000 (oldest deadline 2024-09-12T03:20:23.929695+0000) 2024-09-12T03:20:30.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:30 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:30.376+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:30.703 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:30 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:30.554+0000 7fef046e2640 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:59.341114+0000 front 2024-09-12T03:19:59.341286+0000 (oldest deadline 2024-09-12T03:20:24.641054+0000) 2024-09-12T03:20:30.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:30 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:30.352+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:30.994 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:30 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:30.693+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:31.202 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:30 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:30.888+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:31.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:31 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:31.236+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:58.029532+0000 front 2024-09-12T03:19:58.029595+0000 (oldest deadline 2024-09-12T03:20:23.929695+0000) 2024-09-12T03:20:31.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:31 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:31.335+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:31.702 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:31 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:31.596+0000 7fef046e2640 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:59.341114+0000 front 2024-09-12T03:19:59.341286+0000 (oldest deadline 2024-09-12T03:20:24.641054+0000) 2024-09-12T03:20:31.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:31 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:31.363+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:31.994 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:31 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:31.671+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:32.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:31 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:31.930+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:32.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:32 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:32.237+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:58.029532+0000 front 2024-09-12T03:19:58.029595+0000 (oldest deadline 2024-09-12T03:20:23.929695+0000) 2024-09-12T03:20:32.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:32 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:32.319+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:32.702 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:32 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:32.561+0000 7fef046e2640 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:59.341114+0000 front 2024-09-12T03:19:59.341286+0000 (oldest deadline 2024-09-12T03:20:24.641054+0000) 2024-09-12T03:20:32.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:32 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:32.341+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:32.994 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:32 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:32.674+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:32.994 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:32 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:32.674+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:20:32.995 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:32 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:32.551+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.471419+0000 front 2024-09-12T03:20:10.471560+0000 (oldest deadline 2024-09-12T03:20:31.571589+0000) 2024-09-12T03:20:33.202 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:32 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:32.887+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:33.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:33 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:33.210+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:58.029532+0000 front 2024-09-12T03:19:58.029595+0000 (oldest deadline 2024-09-12T03:20:23.929695+0000) 2024-09-12T03:20:33.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:33 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:33.365+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:33.704 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:33 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:33.528+0000 7fef046e2640 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:59.341114+0000 front 2024-09-12T03:19:59.341286+0000 (oldest deadline 2024-09-12T03:20:24.641054+0000) 2024-09-12T03:20:33.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:33 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:33.297+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:33.996 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:33 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:33.515+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.471419+0000 front 2024-09-12T03:20:10.471560+0000 (oldest deadline 2024-09-12T03:20:31.571589+0000) 2024-09-12T03:20:33.996 INFO:journalctl@ceph.osd.4.smithi177.stdout:Sep 12 03:20:33 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-4[46254]: 2024-09-12T03:20:33.676+0000 7fdb561cd640 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.666338+0000 front 2024-09-12T03:20:10.666365+0000 (oldest deadline 2024-09-12T03:20:33.566475+0000) 2024-09-12T03:20:33.997 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:33 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:33.710+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:33.997 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:33 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:33.710+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:20:34.202 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:33 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:33.893+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:34.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:33 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:33.893+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:20:34.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:34 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:34.377+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:34.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:34 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:34.246+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:58.029532+0000 front 2024-09-12T03:19:58.029595+0000 (oldest deadline 2024-09-12T03:20:23.929695+0000) 2024-09-12T03:20:34.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:34 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:34.246+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:09.830752+0000 front 2024-09-12T03:20:09.830800+0000 (oldest deadline 2024-09-12T03:20:33.930756+0000) 2024-09-12T03:20:34.703 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:34 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:34.539+0000 7fef046e2640 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:59.341114+0000 front 2024-09-12T03:19:59.341286+0000 (oldest deadline 2024-09-12T03:20:24.641054+0000) 2024-09-12T03:20:34.703 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:34 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:34.539+0000 7fef046e2640 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.442783+0000 front 2024-09-12T03:20:10.442772+0000 (oldest deadline 2024-09-12T03:20:33.942734+0000) 2024-09-12T03:20:34.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:34 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:34.298+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:34.994 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:34 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:34.526+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.471419+0000 front 2024-09-12T03:20:10.471560+0000 (oldest deadline 2024-09-12T03:20:31.571589+0000) 2024-09-12T03:20:34.995 INFO:journalctl@ceph.osd.4.smithi177.stdout:Sep 12 03:20:34 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-4[46254]: 2024-09-12T03:20:34.636+0000 7fdb561cd640 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.666338+0000 front 2024-09-12T03:20:10.666365+0000 (oldest deadline 2024-09-12T03:20:33.566475+0000) 2024-09-12T03:20:34.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:34 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:34.740+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:34.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:34 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:34.740+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:20:35.202 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:34 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:34.894+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:35.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:34 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:34.894+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:20:35.550 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:35 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:35.299+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:35.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:35 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:35.381+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:35.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:35 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:35.253+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:58.029532+0000 front 2024-09-12T03:19:58.029595+0000 (oldest deadline 2024-09-12T03:20:23.929695+0000) 2024-09-12T03:20:35.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:35 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:35.253+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:09.830752+0000 front 2024-09-12T03:20:09.830800+0000 (oldest deadline 2024-09-12T03:20:33.930756+0000) 2024-09-12T03:20:35.744 INFO:journalctl@ceph.osd.4.smithi177.stdout:Sep 12 03:20:35 smithi177 podman[70745]: 2024-09-12 03:20:35.612061537 +0000 UTC m=+5.642410268 container died 2fa119bca937a4f10034556b1d22a989bc33c8849f1fbf27dcc231aa7d5d70fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:20:35.745 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:35 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:35.488+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.471419+0000 front 2024-09-12T03:20:10.471560+0000 (oldest deadline 2024-09-12T03:20:31.571589+0000) 2024-09-12T03:20:35.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:35 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:35.708+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:35.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:35 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:35.708+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:20:35.861 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:35 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:35.551+0000 7fef046e2640 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:59.341114+0000 front 2024-09-12T03:19:59.341286+0000 (oldest deadline 2024-09-12T03:20:24.641054+0000) 2024-09-12T03:20:35.861 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:35 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:35.551+0000 7fef046e2640 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.442783+0000 front 2024-09-12T03:20:10.442772+0000 (oldest deadline 2024-09-12T03:20:33.942734+0000) 2024-09-12T03:20:36.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:35 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:35.861+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:36.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:35 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:35.861+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:20:36.244 INFO:journalctl@ceph.osd.4.smithi177.stdout:Sep 12 03:20:36 smithi177 podman[70745]: 2024-09-12 03:20:36.023735595 +0000 UTC m=+6.054084317 container cleanup 2fa119bca937a4f10034556b1d22a989bc33c8849f1fbf27dcc231aa7d5d70fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-12T03:20:36.244 INFO:journalctl@ceph.osd.4.smithi177.stdout:Sep 12 03:20:36 smithi177 bash[70745]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-4 2024-09-12T03:20:36.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:36 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:36.205+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:58.029532+0000 front 2024-09-12T03:19:58.029595+0000 (oldest deadline 2024-09-12T03:20:23.929695+0000) 2024-09-12T03:20:36.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:36 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:36.205+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:09.830752+0000 front 2024-09-12T03:20:09.830800+0000 (oldest deadline 2024-09-12T03:20:33.930756+0000) 2024-09-12T03:20:36.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:36 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:36.357+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:36.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:36 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:36.357+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:20:36.646 INFO:journalctl@ceph.osd.4.smithi177.stdout:Sep 12 03:20:36 smithi177 podman[70760]: 2024-09-12 03:20:36.2836721 +0000 UTC m=+0.660419605 container remove 2fa119bca937a4f10034556b1d22a989bc33c8849f1fbf27dcc231aa7d5d70fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-4, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-12T03:20:36.647 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:36 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:36.480+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.471419+0000 front 2024-09-12T03:20:10.471560+0000 (oldest deadline 2024-09-12T03:20:31.571589+0000) 2024-09-12T03:20:36.702 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:36 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:36.518+0000 7fef046e2640 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:59.341114+0000 front 2024-09-12T03:19:59.341286+0000 (oldest deadline 2024-09-12T03:20:24.641054+0000) 2024-09-12T03:20:36.703 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:36 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:36.518+0000 7fef046e2640 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.442783+0000 front 2024-09-12T03:20:10.442772+0000 (oldest deadline 2024-09-12T03:20:33.942734+0000) 2024-09-12T03:20:36.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:36 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:36.280+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:36.903 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:36 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:36.743+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:36.903 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:36 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:36.743+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:20:36.903 INFO:journalctl@ceph.osd.4.smithi177.stdout:Sep 12 03:20:36 smithi177 podman[70852]: 2024-09-12 03:20:36.549598567 +0000 UTC m=+0.015863785 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:20:36.903 INFO:journalctl@ceph.osd.4.smithi177.stdout:Sep 12 03:20:36 smithi177 podman[70852]: 2024-09-12 03:20:36.66780444 +0000 UTC m=+0.134069661 container create 246c4c55866cfa0ef39e88937a97923e80d8f9bee975b613bfb4e487fd9bdacf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-4-deactivate, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:20:36.903 INFO:journalctl@ceph.osd.4.smithi177.stdout:Sep 12 03:20:36 smithi177 podman[70852]: 2024-09-12 03:20:36.903524092 +0000 UTC m=+0.369789310 container init 246c4c55866cfa0ef39e88937a97923e80d8f9bee975b613bfb4e487fd9bdacf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-12T03:20:37.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:36 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:36.884+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:37.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:36 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:36.884+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:20:37.244 INFO:journalctl@ceph.osd.4.smithi177.stdout:Sep 12 03:20:36 smithi177 podman[70852]: 2024-09-12 03:20:36.907390567 +0000 UTC m=+0.373655794 container start 246c4c55866cfa0ef39e88937a97923e80d8f9bee975b613bfb4e487fd9bdacf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True) 2024-09-12T03:20:37.245 INFO:journalctl@ceph.osd.4.smithi177.stdout:Sep 12 03:20:37 smithi177 podman[70852]: 2024-09-12 03:20:37.012015058 +0000 UTC m=+0.478280290 container attach 246c4c55866cfa0ef39e88937a97923e80d8f9bee975b613bfb4e487fd9bdacf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-12T03:20:37.245 INFO:journalctl@ceph.osd.4.smithi177.stdout:Sep 12 03:20:37 smithi177 podman[70852]: 2024-09-12 03:20:37.099415394 +0000 UTC m=+0.565680625 container died 246c4c55866cfa0ef39e88937a97923e80d8f9bee975b613bfb4e487fd9bdacf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-12T03:20:37.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:37 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:37.249+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:58.029532+0000 front 2024-09-12T03:19:58.029595+0000 (oldest deadline 2024-09-12T03:20:23.929695+0000) 2024-09-12T03:20:37.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:37 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:37.249+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:09.830752+0000 front 2024-09-12T03:20:09.830800+0000 (oldest deadline 2024-09-12T03:20:33.930756+0000) 2024-09-12T03:20:37.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:37 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:37.337+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:37.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:37 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:37.337+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:20:37.702 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:37 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:37.483+0000 7fef046e2640 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:59.341114+0000 front 2024-09-12T03:19:59.341286+0000 (oldest deadline 2024-09-12T03:20:24.641054+0000) 2024-09-12T03:20:37.703 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:37 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:37.483+0000 7fef046e2640 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.442783+0000 front 2024-09-12T03:20:10.442772+0000 (oldest deadline 2024-09-12T03:20:33.942734+0000) 2024-09-12T03:20:37.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:37 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:37.266+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:37.739 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:37 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:37.470+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.471419+0000 front 2024-09-12T03:20:10.471560+0000 (oldest deadline 2024-09-12T03:20:31.571589+0000) 2024-09-12T03:20:37.763 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.4.service' 2024-09-12T03:20:37.803 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:20:37.803 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-12T03:20:37.803 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-12T03:20:37.803 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.5 2024-09-12T03:20:37.994 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:37 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:37.757+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:37.994 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:37 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:37.757+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:20:38.126 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:37 smithi183 systemd[1]: Stopping Ceph osd.5 for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:20:38.127 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:37 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:37.897+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:38.127 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:37 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:37.897+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:20:38.451 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:38 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:38.242+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:38.451 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:38 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:38.125+0000 7fef0c0d4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:20:38.451 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:38 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:38.125+0000 7fef0c0d4640 -1 osd.5 83 *** Got signal Terminated *** 2024-09-12T03:20:38.451 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:38 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:38.125+0000 7fef0c0d4640 -1 osd.5 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:20:38.451 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:38 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:38.451+0000 7fef046e2640 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:59.341114+0000 front 2024-09-12T03:19:59.341286+0000 (oldest deadline 2024-09-12T03:20:24.641054+0000) 2024-09-12T03:20:38.451 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:38 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:38.451+0000 7fef046e2640 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.442783+0000 front 2024-09-12T03:20:10.442772+0000 (oldest deadline 2024-09-12T03:20:33.942734+0000) 2024-09-12T03:20:38.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:38 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:38.209+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:58.029532+0000 front 2024-09-12T03:19:58.029595+0000 (oldest deadline 2024-09-12T03:20:23.929695+0000) 2024-09-12T03:20:38.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:38 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:38.209+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:09.830752+0000 front 2024-09-12T03:20:09.830800+0000 (oldest deadline 2024-09-12T03:20:33.930756+0000) 2024-09-12T03:20:38.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:38 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:38.311+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:38.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:38 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:38.311+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:20:38.730 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:38 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:38.447+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.471419+0000 front 2024-09-12T03:20:10.471560+0000 (oldest deadline 2024-09-12T03:20:31.571589+0000) 2024-09-12T03:20:38.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:38 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:38.729+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:38.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:38 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:38.729+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:20:39.202 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:38 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:38.889+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:39.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:38 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:38.889+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:20:39.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:39 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:39.201+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:58.029532+0000 front 2024-09-12T03:19:58.029595+0000 (oldest deadline 2024-09-12T03:20:23.929695+0000) 2024-09-12T03:20:39.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:39 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:39.201+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:09.830752+0000 front 2024-09-12T03:20:09.830800+0000 (oldest deadline 2024-09-12T03:20:33.930756+0000) 2024-09-12T03:20:39.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:39 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:39.289+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:39.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:39 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:39.289+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:20:39.701 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:39 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:39.417+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.471419+0000 front 2024-09-12T03:20:10.471560+0000 (oldest deadline 2024-09-12T03:20:31.571589+0000) 2024-09-12T03:20:39.702 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:39 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:39.497+0000 7fef046e2640 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:59.341114+0000 front 2024-09-12T03:19:59.341286+0000 (oldest deadline 2024-09-12T03:20:24.641054+0000) 2024-09-12T03:20:39.703 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:39 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:39.497+0000 7fef046e2640 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.442783+0000 front 2024-09-12T03:20:10.442772+0000 (oldest deadline 2024-09-12T03:20:33.942734+0000) 2024-09-12T03:20:39.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:39 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:39.259+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:39.994 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:39 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:39.702+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:39.994 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:39 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:39.702+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:20:40.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:39 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:39.895+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:40.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:39 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:39.895+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:20:40.536 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:40 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:40.246+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:40.536 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:40 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:40.246+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:20:40.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:40 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:40.182+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:58.029532+0000 front 2024-09-12T03:19:58.029595+0000 (oldest deadline 2024-09-12T03:20:23.929695+0000) 2024-09-12T03:20:40.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:40 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:40.182+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:09.830752+0000 front 2024-09-12T03:20:09.830800+0000 (oldest deadline 2024-09-12T03:20:33.930756+0000) 2024-09-12T03:20:40.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:40 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:40.307+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:40.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:40 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:40.307+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:20:40.744 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:40 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:40.459+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.471419+0000 front 2024-09-12T03:20:10.471560+0000 (oldest deadline 2024-09-12T03:20:31.571589+0000) 2024-09-12T03:20:40.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:40 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:40.653+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:40.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:40 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:40.653+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:20:40.895 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:40 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:40.535+0000 7fef046e2640 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:59.341114+0000 front 2024-09-12T03:19:59.341286+0000 (oldest deadline 2024-09-12T03:20:24.641054+0000) 2024-09-12T03:20:40.896 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:40 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:40.535+0000 7fef046e2640 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.442783+0000 front 2024-09-12T03:20:10.442772+0000 (oldest deadline 2024-09-12T03:20:33.942734+0000) 2024-09-12T03:20:40.896 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:40 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:40.535+0000 7fef046e2640 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.443624+0000 front 2024-09-12T03:20:17.443651+0000 (oldest deadline 2024-09-12T03:20:40.343749+0000) 2024-09-12T03:20:41.202 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:40 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:40.895+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:41.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:40 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:40.895+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:20:41.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:41 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:41.262+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:41.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:41 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:41.262+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:20:41.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:41 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:41.223+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:58.029532+0000 front 2024-09-12T03:19:58.029595+0000 (oldest deadline 2024-09-12T03:20:23.929695+0000) 2024-09-12T03:20:41.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:41 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:41.223+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:09.830752+0000 front 2024-09-12T03:20:09.830800+0000 (oldest deadline 2024-09-12T03:20:33.930756+0000) 2024-09-12T03:20:41.553 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:41 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:41.223+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.432029+0000 front 2024-09-12T03:20:17.431990+0000 (oldest deadline 2024-09-12T03:20:40.331843+0000) 2024-09-12T03:20:41.556 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:41 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:41.209+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:41.556 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:41 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:41.209+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:20:41.744 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:41 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:41.472+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.471419+0000 front 2024-09-12T03:20:10.471560+0000 (oldest deadline 2024-09-12T03:20:31.571589+0000) 2024-09-12T03:20:41.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:41 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:41.612+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:41.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:41 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:41.612+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:20:41.881 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:41 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:41.556+0000 7fef046e2640 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:59.341114+0000 front 2024-09-12T03:19:59.341286+0000 (oldest deadline 2024-09-12T03:20:24.641054+0000) 2024-09-12T03:20:41.881 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:41 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:41.556+0000 7fef046e2640 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.442783+0000 front 2024-09-12T03:20:10.442772+0000 (oldest deadline 2024-09-12T03:20:33.942734+0000) 2024-09-12T03:20:41.882 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:41 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:41.556+0000 7fef046e2640 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.443624+0000 front 2024-09-12T03:20:17.443651+0000 (oldest deadline 2024-09-12T03:20:40.343749+0000) 2024-09-12T03:20:42.202 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:41 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:41.882+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:42.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:41 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:41.882+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:20:42.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:42 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:42.273+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:58.029532+0000 front 2024-09-12T03:19:58.029595+0000 (oldest deadline 2024-09-12T03:20:23.929695+0000) 2024-09-12T03:20:42.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:42 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:42.273+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:09.830752+0000 front 2024-09-12T03:20:09.830800+0000 (oldest deadline 2024-09-12T03:20:33.930756+0000) 2024-09-12T03:20:42.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:42 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:42.273+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.432029+0000 front 2024-09-12T03:20:17.431990+0000 (oldest deadline 2024-09-12T03:20:40.331843+0000) 2024-09-12T03:20:42.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:42 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:42.242+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:42.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:42 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:42.242+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:20:42.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:42 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:42.242+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:20:42.562 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:42 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:42.209+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:42.562 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:42 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:42.209+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:20:42.884 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:42 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:42.561+0000 7fef046e2640 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:59.341114+0000 front 2024-09-12T03:19:59.341286+0000 (oldest deadline 2024-09-12T03:20:24.641054+0000) 2024-09-12T03:20:42.885 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:42 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:42.561+0000 7fef046e2640 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.442783+0000 front 2024-09-12T03:20:10.442772+0000 (oldest deadline 2024-09-12T03:20:33.942734+0000) 2024-09-12T03:20:42.885 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:42 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5[47420]: 2024-09-12T03:20:42.561+0000 7fef046e2640 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.443624+0000 front 2024-09-12T03:20:17.443651+0000 (oldest deadline 2024-09-12T03:20:40.343749+0000) 2024-09-12T03:20:42.994 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:42 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:42.510+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.471419+0000 front 2024-09-12T03:20:10.471560+0000 (oldest deadline 2024-09-12T03:20:31.571589+0000) 2024-09-12T03:20:42.994 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:42 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:42.642+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:42.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:42 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:42.642+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:20:42.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:42 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:42.642+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:20:43.202 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:42 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:42.885+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:43.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:42 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:42.885+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:20:43.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:43 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:43.255+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:58.029532+0000 front 2024-09-12T03:19:58.029595+0000 (oldest deadline 2024-09-12T03:20:23.929695+0000) 2024-09-12T03:20:43.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:43 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:43.255+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:09.830752+0000 front 2024-09-12T03:20:09.830800+0000 (oldest deadline 2024-09-12T03:20:33.930756+0000) 2024-09-12T03:20:43.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:43 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:43.255+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.432029+0000 front 2024-09-12T03:20:17.431990+0000 (oldest deadline 2024-09-12T03:20:40.331843+0000) 2024-09-12T03:20:43.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:43 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:43.214+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:43.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:43 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:43.214+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:20:43.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:43 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:43.214+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:20:43.703 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:43 smithi183 podman[72207]: 2024-09-12 03:20:43.463547235 +0000 UTC m=+5.467133751 container died 2d9d2fc285e1056e0db4b489c67d115cf641348ce52651da9352b0f47dc0c75f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-12T03:20:43.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:43 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:43.259+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:43.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:43 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:43.259+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:20:43.994 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:43 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:43.515+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.471419+0000 front 2024-09-12T03:20:10.471560+0000 (oldest deadline 2024-09-12T03:20:31.571589+0000) 2024-09-12T03:20:43.995 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:43 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:43.515+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:19.074057+0000 front 2024-09-12T03:20:19.073877+0000 (oldest deadline 2024-09-12T03:20:42.573956+0000) 2024-09-12T03:20:43.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:43 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:43.608+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:43.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:43 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:43.608+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:20:43.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:43 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:43.608+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:20:44.110 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:43 smithi183 podman[72207]: 2024-09-12 03:20:43.851532084 +0000 UTC m=+5.855118598 container cleanup 2d9d2fc285e1056e0db4b489c67d115cf641348ce52651da9352b0f47dc0c75f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:20:44.111 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:43 smithi183 bash[72207]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5 2024-09-12T03:20:44.111 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:43 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:43.909+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:44.111 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:43 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:43.909+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:20:44.453 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:44 smithi183 podman[72222]: 2024-09-12 03:20:44.111333447 +0000 UTC m=+0.646475071 container remove 2d9d2fc285e1056e0db4b489c67d115cf641348ce52651da9352b0f47dc0c75f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True) 2024-09-12T03:20:44.453 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:44 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:44.284+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:44.453 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:44 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:44.284+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:20:44.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:44 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:44.257+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:58.029532+0000 front 2024-09-12T03:19:58.029595+0000 (oldest deadline 2024-09-12T03:20:23.929695+0000) 2024-09-12T03:20:44.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:44 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:44.257+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:09.830752+0000 front 2024-09-12T03:20:09.830800+0000 (oldest deadline 2024-09-12T03:20:33.930756+0000) 2024-09-12T03:20:44.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:44 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:44.257+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.432029+0000 front 2024-09-12T03:20:17.431990+0000 (oldest deadline 2024-09-12T03:20:40.331843+0000) 2024-09-12T03:20:44.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:44 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:44.206+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:44.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:44 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:44.206+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:20:44.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:44 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:44.206+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:20:44.730 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:44 smithi183 podman[72311]: 2024-09-12 03:20:44.36337732 +0000 UTC m=+0.015786876 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:20:44.731 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:44 smithi183 podman[72311]: 2024-09-12 03:20:44.487230409 +0000 UTC m=+0.139639949 container create 0f24d3a7d843a517590459b5603a42f111fac02304b964ff445d7a209967538a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-09-12T03:20:44.731 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:44 smithi183 podman[72311]: 2024-09-12 03:20:44.731492156 +0000 UTC m=+0.383901702 container init 0f24d3a7d843a517590459b5603a42f111fac02304b964ff445d7a209967538a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:20:44.995 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:44 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:44.551+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.471419+0000 front 2024-09-12T03:20:10.471560+0000 (oldest deadline 2024-09-12T03:20:31.571589+0000) 2024-09-12T03:20:44.995 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:44 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:44.551+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:19.074057+0000 front 2024-09-12T03:20:19.073877+0000 (oldest deadline 2024-09-12T03:20:42.573956+0000) 2024-09-12T03:20:44.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:44 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:44.589+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:44.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:44 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:44.589+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:20:44.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:44 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:44.589+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:20:45.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:44 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:44.914+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:45.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:44 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:44.914+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:20:45.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:44 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:44.914+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:18.820516+0000 front 2024-09-12T03:20:18.820577+0000 (oldest deadline 2024-09-12T03:20:44.120637+0000) 2024-09-12T03:20:45.203 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:44 smithi183 podman[72311]: 2024-09-12 03:20:44.73481234 +0000 UTC m=+0.387221877 container start 0f24d3a7d843a517590459b5603a42f111fac02304b964ff445d7a209967538a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-12T03:20:45.203 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:44 smithi183 podman[72311]: 2024-09-12 03:20:44.823223085 +0000 UTC m=+0.475632622 container attach 0f24d3a7d843a517590459b5603a42f111fac02304b964ff445d7a209967538a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:20:45.203 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 12 03:20:44 smithi183 podman[72311]: 2024-09-12 03:20:44.91813319 +0000 UTC m=+0.570542734 container died 0f24d3a7d843a517590459b5603a42f111fac02304b964ff445d7a209967538a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-12T03:20:45.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:45 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:45.298+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:58.029532+0000 front 2024-09-12T03:19:58.029595+0000 (oldest deadline 2024-09-12T03:20:23.929695+0000) 2024-09-12T03:20:45.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:45 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:45.298+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:09.830752+0000 front 2024-09-12T03:20:09.830800+0000 (oldest deadline 2024-09-12T03:20:33.930756+0000) 2024-09-12T03:20:45.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:45 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:45.298+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.432029+0000 front 2024-09-12T03:20:17.431990+0000 (oldest deadline 2024-09-12T03:20:40.331843+0000) 2024-09-12T03:20:45.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:45 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:45.157+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:45.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:45 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:45.157+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:20:45.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:45 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:45.157+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:20:45.618 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:45 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:45.239+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:45.618 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:45 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:45.239+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:20:45.645 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.5.service' 2024-09-12T03:20:45.687 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:20:45.688 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-12T03:20:45.688 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-12T03:20:45.688 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.6 2024-09-12T03:20:45.987 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:45 smithi116 systemd[1]: Stopping Ceph osd.6 for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:20:45.994 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:45 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:45.563+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.471419+0000 front 2024-09-12T03:20:10.471560+0000 (oldest deadline 2024-09-12T03:20:31.571589+0000) 2024-09-12T03:20:45.994 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:45 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:45.563+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:19.074057+0000 front 2024-09-12T03:20:19.073877+0000 (oldest deadline 2024-09-12T03:20:42.573956+0000) 2024-09-12T03:20:45.994 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:45 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:45.581+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:45.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:45 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:45.581+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:20:45.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:45 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:45.581+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:20:46.202 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:45 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:45.947+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:46.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:45 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:45.947+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:20:46.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:45 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:45.947+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:18.820516+0000 front 2024-09-12T03:20:18.820577+0000 (oldest deadline 2024-09-12T03:20:44.120637+0000) 2024-09-12T03:20:46.296 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:46 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:46.179+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:46.297 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:46 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:46.179+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:20:46.297 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:46 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:46.179+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:20:46.297 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:45 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:45.986+0000 7fc39588b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:20:46.297 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:45 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:45.986+0000 7fc39588b640 -1 osd.6 83 *** Got signal Terminated *** 2024-09-12T03:20:46.297 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:45 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:45.986+0000 7fc39588b640 -1 osd.6 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:20:46.297 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:46 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:46.296+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:58.029532+0000 front 2024-09-12T03:19:58.029595+0000 (oldest deadline 2024-09-12T03:20:23.929695+0000) 2024-09-12T03:20:46.297 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:46 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:46.296+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:09.830752+0000 front 2024-09-12T03:20:09.830800+0000 (oldest deadline 2024-09-12T03:20:33.930756+0000) 2024-09-12T03:20:46.297 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:46 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:46.296+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.432029+0000 front 2024-09-12T03:20:17.431990+0000 (oldest deadline 2024-09-12T03:20:40.331843+0000) 2024-09-12T03:20:46.702 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:46 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:46.271+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:46.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:46 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:46.271+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:20:46.994 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:46 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:46.542+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.471419+0000 front 2024-09-12T03:20:10.471560+0000 (oldest deadline 2024-09-12T03:20:31.571589+0000) 2024-09-12T03:20:46.994 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:46 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:46.542+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:19.074057+0000 front 2024-09-12T03:20:19.073877+0000 (oldest deadline 2024-09-12T03:20:42.573956+0000) 2024-09-12T03:20:46.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:46 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:46.588+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:46.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:46 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:46.588+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:20:46.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:46 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:46.588+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:20:47.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:46 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:46.917+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:47.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:46 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:46.917+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:20:47.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:46 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:46.917+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:18.820516+0000 front 2024-09-12T03:20:18.820577+0000 (oldest deadline 2024-09-12T03:20:44.120637+0000) 2024-09-12T03:20:47.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:47 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:47.321+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:58.029532+0000 front 2024-09-12T03:19:58.029595+0000 (oldest deadline 2024-09-12T03:20:23.929695+0000) 2024-09-12T03:20:47.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:47 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:47.321+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:09.830752+0000 front 2024-09-12T03:20:09.830800+0000 (oldest deadline 2024-09-12T03:20:33.930756+0000) 2024-09-12T03:20:47.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:47 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:47.321+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.432029+0000 front 2024-09-12T03:20:17.431990+0000 (oldest deadline 2024-09-12T03:20:40.331843+0000) 2024-09-12T03:20:47.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:47 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:47.214+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:47.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:47 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:47.214+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:20:47.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:47 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:47.214+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:20:47.702 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:47 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:47.246+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:47.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:47 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:47.246+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:20:47.994 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:47 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:47.559+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.471419+0000 front 2024-09-12T03:20:10.471560+0000 (oldest deadline 2024-09-12T03:20:31.571589+0000) 2024-09-12T03:20:47.994 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:47 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:47.559+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:19.074057+0000 front 2024-09-12T03:20:19.073877+0000 (oldest deadline 2024-09-12T03:20:42.573956+0000) 2024-09-12T03:20:47.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:47 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:47.557+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:47.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:47 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:47.557+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:20:47.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:47 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:47.557+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:20:47.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:47 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:47.557+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:20:48.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:47 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:47.876+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:48.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:47 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:47.876+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:20:48.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:47 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:47.876+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:18.820516+0000 front 2024-09-12T03:20:18.820577+0000 (oldest deadline 2024-09-12T03:20:44.120637+0000) 2024-09-12T03:20:48.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:48 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:48.345+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:58.029532+0000 front 2024-09-12T03:19:58.029595+0000 (oldest deadline 2024-09-12T03:20:23.929695+0000) 2024-09-12T03:20:48.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:48 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:48.345+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:09.830752+0000 front 2024-09-12T03:20:09.830800+0000 (oldest deadline 2024-09-12T03:20:33.930756+0000) 2024-09-12T03:20:48.553 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:48 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:48.345+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.432029+0000 front 2024-09-12T03:20:17.431990+0000 (oldest deadline 2024-09-12T03:20:40.331843+0000) 2024-09-12T03:20:48.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:48 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:48.234+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:48.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:48 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:48.234+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:20:48.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:48 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:48.234+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:20:48.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:48 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:48.234+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:22.047371+0000 front 2024-09-12T03:20:22.047302+0000 (oldest deadline 2024-09-12T03:20:47.847767+0000) 2024-09-12T03:20:48.702 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:48 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:48.265+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:48.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:48 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:48.265+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:20:48.994 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:48 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:48.593+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.471419+0000 front 2024-09-12T03:20:10.471560+0000 (oldest deadline 2024-09-12T03:20:31.571589+0000) 2024-09-12T03:20:48.994 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:48 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:48.593+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:19.074057+0000 front 2024-09-12T03:20:19.073877+0000 (oldest deadline 2024-09-12T03:20:42.573956+0000) 2024-09-12T03:20:48.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:48 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:48.519+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:48.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:48 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:48.519+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:20:48.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:48 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:48.519+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:20:48.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:48 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:48.519+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:20:49.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:48 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:48.858+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:49.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:48 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:48.858+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:20:49.204 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:48 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:48.858+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:18.820516+0000 front 2024-09-12T03:20:18.820577+0000 (oldest deadline 2024-09-12T03:20:44.120637+0000) 2024-09-12T03:20:49.468 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:49 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:49.254+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:49.468 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:49 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:49.254+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:20:49.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:49 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:49.393+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:58.029532+0000 front 2024-09-12T03:19:58.029595+0000 (oldest deadline 2024-09-12T03:20:23.929695+0000) 2024-09-12T03:20:49.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:49 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:49.393+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:09.830752+0000 front 2024-09-12T03:20:09.830800+0000 (oldest deadline 2024-09-12T03:20:33.930756+0000) 2024-09-12T03:20:49.553 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:49 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:49.393+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.432029+0000 front 2024-09-12T03:20:17.431990+0000 (oldest deadline 2024-09-12T03:20:40.331843+0000) 2024-09-12T03:20:49.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:49 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:49.268+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:49.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:49 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:49.268+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:20:49.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:49 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:49.268+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:20:49.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:49 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:49.268+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:22.047371+0000 front 2024-09-12T03:20:22.047302+0000 (oldest deadline 2024-09-12T03:20:47.847767+0000) 2024-09-12T03:20:49.994 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:49 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:49.610+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.471419+0000 front 2024-09-12T03:20:10.471560+0000 (oldest deadline 2024-09-12T03:20:31.571589+0000) 2024-09-12T03:20:49.994 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:49 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:49.610+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:19.074057+0000 front 2024-09-12T03:20:19.073877+0000 (oldest deadline 2024-09-12T03:20:42.573956+0000) 2024-09-12T03:20:49.994 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:49 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:49.610+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:24.275198+0000 front 2024-09-12T03:20:24.275279+0000 (oldest deadline 2024-09-12T03:20:48.975033+0000) 2024-09-12T03:20:49.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:49 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:49.498+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:49.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:49 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:49.498+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:20:49.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:49 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:49.498+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:20:49.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:49 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:49.498+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:20:50.202 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:49 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:49.866+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:50.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:49 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:49.866+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:20:50.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:49 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:49.866+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:18.820516+0000 front 2024-09-12T03:20:18.820577+0000 (oldest deadline 2024-09-12T03:20:44.120637+0000) 2024-09-12T03:20:50.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:50 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:50.258+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:50.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:50 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:50.258+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:20:50.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:50 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:50.258+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:20:50.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:50 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:50.258+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:22.047371+0000 front 2024-09-12T03:20:22.047302+0000 (oldest deadline 2024-09-12T03:20:47.847767+0000) 2024-09-12T03:20:50.553 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:50 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:50.437+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:19:58.029532+0000 front 2024-09-12T03:19:58.029595+0000 (oldest deadline 2024-09-12T03:20:23.929695+0000) 2024-09-12T03:20:50.553 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:50 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:50.437+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:09.830752+0000 front 2024-09-12T03:20:09.830800+0000 (oldest deadline 2024-09-12T03:20:33.930756+0000) 2024-09-12T03:20:50.553 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:50 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:50.437+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.432029+0000 front 2024-09-12T03:20:17.431990+0000 (oldest deadline 2024-09-12T03:20:40.331843+0000) 2024-09-12T03:20:50.553 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:50 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6[59068]: 2024-09-12T03:20:50.437+0000 7fc38de99640 -1 osd.6 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:24.333242+0000 front 2024-09-12T03:20:24.333259+0000 (oldest deadline 2024-09-12T03:20:50.233421+0000) 2024-09-12T03:20:50.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:50 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:50.277+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:50.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:50 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:50.277+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:20:50.744 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:50 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:50.646+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.471419+0000 front 2024-09-12T03:20:10.471560+0000 (oldest deadline 2024-09-12T03:20:31.571589+0000) 2024-09-12T03:20:50.745 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:50 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:50.646+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:19.074057+0000 front 2024-09-12T03:20:19.073877+0000 (oldest deadline 2024-09-12T03:20:42.573956+0000) 2024-09-12T03:20:50.745 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:50 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:50.646+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:24.275198+0000 front 2024-09-12T03:20:24.275279+0000 (oldest deadline 2024-09-12T03:20:48.975033+0000) 2024-09-12T03:20:50.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:50 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:50.480+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:50.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:50 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:50.480+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:20:50.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:50 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:50.480+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:20:50.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:50 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:50.480+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:20:51.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:50 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:50.861+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:51.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:50 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:50.861+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:20:51.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:50 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:50.861+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:18.820516+0000 front 2024-09-12T03:20:18.820577+0000 (oldest deadline 2024-09-12T03:20:44.120637+0000) 2024-09-12T03:20:51.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:50 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:50.861+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.221507+0000 front 2024-09-12T03:20:25.221564+0000 (oldest deadline 2024-09-12T03:20:50.521570+0000) 2024-09-12T03:20:51.552 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:51 smithi116 podman[129654]: 2024-09-12 03:20:51.297994996 +0000 UTC m=+5.414152038 container died d7a5e87090af8dc6b7f1ba22fad357254718243aeecd7015f433a9312cb84f22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-12T03:20:51.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:51 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:51.277+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:51.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:51 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:51.277+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:20:51.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:51 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:51.278+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:20:51.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:51 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:51.278+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:22.047371+0000 front 2024-09-12T03:20:22.047302+0000 (oldest deadline 2024-09-12T03:20:47.847767+0000) 2024-09-12T03:20:51.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:51 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:51.280+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:51.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:51 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:51.280+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:20:51.994 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:51 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:51.627+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.471419+0000 front 2024-09-12T03:20:10.471560+0000 (oldest deadline 2024-09-12T03:20:31.571589+0000) 2024-09-12T03:20:51.995 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:51 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:51.627+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:19.074057+0000 front 2024-09-12T03:20:19.073877+0000 (oldest deadline 2024-09-12T03:20:42.573956+0000) 2024-09-12T03:20:51.995 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:51 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:51.627+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:24.275198+0000 front 2024-09-12T03:20:24.275279+0000 (oldest deadline 2024-09-12T03:20:48.975033+0000) 2024-09-12T03:20:51.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:51 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:51.521+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:51.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:51 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:51.521+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:20:51.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:51 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:51.521+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:20:51.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:51 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:51.521+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:20:52.052 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:51 smithi116 podman[129654]: 2024-09-12 03:20:51.803402749 +0000 UTC m=+5.919559780 container cleanup d7a5e87090af8dc6b7f1ba22fad357254718243aeecd7015f433a9312cb84f22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True) 2024-09-12T03:20:52.052 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:51 smithi116 bash[129654]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6 2024-09-12T03:20:52.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:51 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:51.894+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:52.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:51 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:51.894+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:20:52.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:51 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:51.894+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:18.820516+0000 front 2024-09-12T03:20:18.820577+0000 (oldest deadline 2024-09-12T03:20:44.120637+0000) 2024-09-12T03:20:52.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:51 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:51.894+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.221507+0000 front 2024-09-12T03:20:25.221564+0000 (oldest deadline 2024-09-12T03:20:50.521570+0000) 2024-09-12T03:20:52.398 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:52 smithi116 podman[129671]: 2024-09-12 03:20:52.088561769 +0000 UTC m=+0.779756279 container remove d7a5e87090af8dc6b7f1ba22fad357254718243aeecd7015f433a9312cb84f22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:20:52.398 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:52 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:52.232+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:52.398 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:52 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:52.232+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:20:52.399 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:52 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:52.232+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:20:52.399 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:52 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:52.232+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:22.047371+0000 front 2024-09-12T03:20:22.047302+0000 (oldest deadline 2024-09-12T03:20:47.847767+0000) 2024-09-12T03:20:52.666 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:52 smithi116 podman[129759]: 2024-09-12 03:20:52.3013487 +0000 UTC m=+0.016195877 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:20:52.666 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:52 smithi116 podman[129759]: 2024-09-12 03:20:52.439379321 +0000 UTC m=+0.154226536 container create 25b8427d3c1651ae4b94ce0170d0cf8e55a1e5c51ceb162392cfddee08f2ca28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-12T03:20:52.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:52 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:52.290+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:52.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:52 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:52.290+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:20:52.994 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:52 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:52.666+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.471419+0000 front 2024-09-12T03:20:10.471560+0000 (oldest deadline 2024-09-12T03:20:31.571589+0000) 2024-09-12T03:20:52.994 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:52 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:52.666+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:19.074057+0000 front 2024-09-12T03:20:19.073877+0000 (oldest deadline 2024-09-12T03:20:42.573956+0000) 2024-09-12T03:20:52.995 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:52 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:52.666+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:24.275198+0000 front 2024-09-12T03:20:24.275279+0000 (oldest deadline 2024-09-12T03:20:48.975033+0000) 2024-09-12T03:20:52.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:52 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:52.504+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:52.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:52 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:52.504+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:20:52.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:52 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:52.504+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:20:52.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:52 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:52.504+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:20:53.052 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:52 smithi116 podman[129759]: 2024-09-12 03:20:52.666750465 +0000 UTC m=+0.381597637 container init 25b8427d3c1651ae4b94ce0170d0cf8e55a1e5c51ceb162392cfddee08f2ca28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6-deactivate, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:20:53.052 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:52 smithi116 podman[129759]: 2024-09-12 03:20:52.670429965 +0000 UTC m=+0.385277140 container start 25b8427d3c1651ae4b94ce0170d0cf8e55a1e5c51ceb162392cfddee08f2ca28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6-deactivate, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-12T03:20:53.052 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:52 smithi116 podman[129759]: 2024-09-12 03:20:52.775307238 +0000 UTC m=+0.490154416 container attach 25b8427d3c1651ae4b94ce0170d0cf8e55a1e5c51ceb162392cfddee08f2ca28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS) 2024-09-12T03:20:53.053 INFO:journalctl@ceph.osd.6.smithi116.stdout:Sep 12 03:20:52 smithi116 podman[129759]: 2024-09-12 03:20:52.860133793 +0000 UTC m=+0.574980971 container died 25b8427d3c1651ae4b94ce0170d0cf8e55a1e5c51ceb162392cfddee08f2ca28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-6-deactivate, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True) 2024-09-12T03:20:53.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:52 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:52.849+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:53.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:52 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:52.849+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:20:53.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:52 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:52.849+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:18.820516+0000 front 2024-09-12T03:20:18.820577+0000 (oldest deadline 2024-09-12T03:20:44.120637+0000) 2024-09-12T03:20:53.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:52 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:52.849+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.221507+0000 front 2024-09-12T03:20:25.221564+0000 (oldest deadline 2024-09-12T03:20:50.521570+0000) 2024-09-12T03:20:53.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:53 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:53.268+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:53.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:53 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:53.269+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:20:53.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:53 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:53.269+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:20:53.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:53 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:53.269+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:22.047371+0000 front 2024-09-12T03:20:22.047302+0000 (oldest deadline 2024-09-12T03:20:47.847767+0000) 2024-09-12T03:20:53.678 DEBUG:teuthology.orchestra.run.smithi116:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.6.service' 2024-09-12T03:20:53.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:53 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:53.338+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:53.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:53 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:53.338+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:20:53.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:53 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:53.338+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:20:53.720 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:20:53.720 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-12T03:20:53.721 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-12T03:20:53.721 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.7 2024-09-12T03:20:53.994 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:53 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:53.646+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.471419+0000 front 2024-09-12T03:20:10.471560+0000 (oldest deadline 2024-09-12T03:20:31.571589+0000) 2024-09-12T03:20:53.995 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:53 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:53.646+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:19.074057+0000 front 2024-09-12T03:20:19.073877+0000 (oldest deadline 2024-09-12T03:20:42.573956+0000) 2024-09-12T03:20:53.995 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:53 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:53.646+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:24.275198+0000 front 2024-09-12T03:20:24.275279+0000 (oldest deadline 2024-09-12T03:20:48.975033+0000) 2024-09-12T03:20:53.995 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:53 smithi177 systemd[1]: Stopping Ceph osd.7 for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:20:53.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:53 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:53.518+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:53.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:53 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:53.518+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:20:53.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:53 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:53.518+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:20:53.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:53 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:53.518+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:20:54.202 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:53 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:53.870+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:54.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:53 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:53.870+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:20:54.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:53 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:53.870+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:18.820516+0000 front 2024-09-12T03:20:18.820577+0000 (oldest deadline 2024-09-12T03:20:44.120637+0000) 2024-09-12T03:20:54.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:53 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:53.870+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.221507+0000 front 2024-09-12T03:20:25.221564+0000 (oldest deadline 2024-09-12T03:20:50.521570+0000) 2024-09-12T03:20:54.494 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:54 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:54.025+0000 7f606d22d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:20:54.494 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:54 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:54.025+0000 7f606d22d640 -1 osd.7 83 *** Got signal Terminated *** 2024-09-12T03:20:54.494 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:54 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:54.025+0000 7f606d22d640 -1 osd.7 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:20:54.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:54 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:54.237+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:54.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:54 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:54.237+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:20:54.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:54 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:54.237+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:20:54.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:54 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:54.237+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:22.047371+0000 front 2024-09-12T03:20:22.047302+0000 (oldest deadline 2024-09-12T03:20:47.847767+0000) 2024-09-12T03:20:54.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:54 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:54.355+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:54.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:54 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:54.355+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:20:54.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:54 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:54.355+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:20:54.748 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:54 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:54.622+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.471419+0000 front 2024-09-12T03:20:10.471560+0000 (oldest deadline 2024-09-12T03:20:31.571589+0000) 2024-09-12T03:20:54.748 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:54 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:54.622+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:19.074057+0000 front 2024-09-12T03:20:19.073877+0000 (oldest deadline 2024-09-12T03:20:42.573956+0000) 2024-09-12T03:20:54.749 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:54 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:54.622+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:24.275198+0000 front 2024-09-12T03:20:24.275279+0000 (oldest deadline 2024-09-12T03:20:48.975033+0000) 2024-09-12T03:20:54.749 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:54 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:54.504+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:54.749 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:54 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:54.504+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:20:54.749 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:54 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:54.504+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:20:54.749 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:54 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:54.504+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:20:55.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:54 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:54.892+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:55.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:54 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:54.892+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:20:55.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:54 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:54.892+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:18.820516+0000 front 2024-09-12T03:20:18.820577+0000 (oldest deadline 2024-09-12T03:20:44.120637+0000) 2024-09-12T03:20:55.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:54 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:54.892+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.221507+0000 front 2024-09-12T03:20:25.221564+0000 (oldest deadline 2024-09-12T03:20:50.521570+0000) 2024-09-12T03:20:55.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:55 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:55.264+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:55.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:55 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:55.264+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:20:55.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:55 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:55.264+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:20:55.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:55 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:55.264+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:22.047371+0000 front 2024-09-12T03:20:22.047302+0000 (oldest deadline 2024-09-12T03:20:47.847767+0000) 2024-09-12T03:20:55.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:55 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:55.379+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:55.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:55 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:55.379+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:20:55.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:55 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:55.379+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:20:55.745 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:55 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:55.659+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.471419+0000 front 2024-09-12T03:20:10.471560+0000 (oldest deadline 2024-09-12T03:20:31.571589+0000) 2024-09-12T03:20:55.745 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:55 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:55.659+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:19.074057+0000 front 2024-09-12T03:20:19.073877+0000 (oldest deadline 2024-09-12T03:20:42.573956+0000) 2024-09-12T03:20:55.745 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:55 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:55.659+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:24.275198+0000 front 2024-09-12T03:20:24.275279+0000 (oldest deadline 2024-09-12T03:20:48.975033+0000) 2024-09-12T03:20:55.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:55 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:55.493+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:55.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:55 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:55.493+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:20:55.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:55 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:55.493+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:20:55.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:55 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:55.493+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:20:56.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:55 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:55.927+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:56.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:55 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:55.927+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:20:56.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:55 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:55.927+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:18.820516+0000 front 2024-09-12T03:20:18.820577+0000 (oldest deadline 2024-09-12T03:20:44.120637+0000) 2024-09-12T03:20:56.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:55 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:55.927+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.221507+0000 front 2024-09-12T03:20:25.221564+0000 (oldest deadline 2024-09-12T03:20:50.521570+0000) 2024-09-12T03:20:56.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:56 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:56.235+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:56.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:56 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:56.235+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:20:56.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:56 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:56.235+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:20:56.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:56 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:56.235+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:22.047371+0000 front 2024-09-12T03:20:22.047302+0000 (oldest deadline 2024-09-12T03:20:47.847767+0000) 2024-09-12T03:20:56.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:56 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:56.342+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:56.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:56 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:56.342+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:20:56.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:56 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:56.342+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:20:56.994 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:56 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:56.499+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:56.994 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:56 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:56.499+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:20:56.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:56 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:56.499+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:20:56.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:56 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:56.499+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:20:56.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:56 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:56.499+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.429190+0000 front 2024-09-12T03:20:30.429210+0000 (oldest deadline 2024-09-12T03:20:56.329320+0000) 2024-09-12T03:20:56.995 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:56 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:56.656+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.471419+0000 front 2024-09-12T03:20:10.471560+0000 (oldest deadline 2024-09-12T03:20:31.571589+0000) 2024-09-12T03:20:56.995 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:56 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:56.656+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:19.074057+0000 front 2024-09-12T03:20:19.073877+0000 (oldest deadline 2024-09-12T03:20:42.573956+0000) 2024-09-12T03:20:56.995 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:56 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:56.656+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:24.275198+0000 front 2024-09-12T03:20:24.275279+0000 (oldest deadline 2024-09-12T03:20:48.975033+0000) 2024-09-12T03:20:56.995 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:56 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:56.656+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:34.276117+0000 front 2024-09-12T03:20:34.276045+0000 (oldest deadline 2024-09-12T03:20:56.576137+0000) 2024-09-12T03:20:57.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:56 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:56.940+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:57.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:56 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:56.940+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:20:57.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:56 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:56.940+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:18.820516+0000 front 2024-09-12T03:20:18.820577+0000 (oldest deadline 2024-09-12T03:20:44.120637+0000) 2024-09-12T03:20:57.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:56 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:56.940+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.221507+0000 front 2024-09-12T03:20:25.221564+0000 (oldest deadline 2024-09-12T03:20:50.521570+0000) 2024-09-12T03:20:57.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:57 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:57.248+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:57.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:57 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:57.248+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:20:57.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:57 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:57.248+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:20:57.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:57 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:57.248+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:22.047371+0000 front 2024-09-12T03:20:22.047302+0000 (oldest deadline 2024-09-12T03:20:47.847767+0000) 2024-09-12T03:20:57.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:57 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:57.248+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.748800+0000 front 2024-09-12T03:20:30.748906+0000 (oldest deadline 2024-09-12T03:20:56.648741+0000) 2024-09-12T03:20:57.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:57 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:57.322+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:57.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:57 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:57.322+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:20:57.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:57 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:57.322+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:20:57.994 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:57 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:57.689+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.471419+0000 front 2024-09-12T03:20:10.471560+0000 (oldest deadline 2024-09-12T03:20:31.571589+0000) 2024-09-12T03:20:57.994 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:57 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:57.689+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:19.074057+0000 front 2024-09-12T03:20:19.073877+0000 (oldest deadline 2024-09-12T03:20:42.573956+0000) 2024-09-12T03:20:57.995 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:57 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:57.689+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:24.275198+0000 front 2024-09-12T03:20:24.275279+0000 (oldest deadline 2024-09-12T03:20:48.975033+0000) 2024-09-12T03:20:57.995 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:57 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:57.689+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:34.276117+0000 front 2024-09-12T03:20:34.276045+0000 (oldest deadline 2024-09-12T03:20:56.576137+0000) 2024-09-12T03:20:57.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:57 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:57.497+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:57.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:57 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:57.497+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:20:57.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:57 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:57.497+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:20:57.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:57 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:57.497+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:20:57.995 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:57 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:57.497+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.429190+0000 front 2024-09-12T03:20:30.429210+0000 (oldest deadline 2024-09-12T03:20:56.329320+0000) 2024-09-12T03:20:58.277 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:57 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:57.977+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:58.277 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:57 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:57.977+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:20:58.277 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:57 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:57.977+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:18.820516+0000 front 2024-09-12T03:20:18.820577+0000 (oldest deadline 2024-09-12T03:20:44.120637+0000) 2024-09-12T03:20:58.278 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:57 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:57.977+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.221507+0000 front 2024-09-12T03:20:25.221564+0000 (oldest deadline 2024-09-12T03:20:50.521570+0000) 2024-09-12T03:20:58.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:58 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:58.218+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:58.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:58 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:58.218+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:20:58.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:58 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:58.218+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:20:58.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:58 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:58.218+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:22.047371+0000 front 2024-09-12T03:20:22.047302+0000 (oldest deadline 2024-09-12T03:20:47.847767+0000) 2024-09-12T03:20:58.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:58 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:58.218+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.748800+0000 front 2024-09-12T03:20:30.748906+0000 (oldest deadline 2024-09-12T03:20:56.648741+0000) 2024-09-12T03:20:58.702 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:58 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:58.276+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:58.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:58 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:58.276+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:20:58.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:58 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:58.276+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:20:58.744 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:58 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:58.684+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.471419+0000 front 2024-09-12T03:20:10.471560+0000 (oldest deadline 2024-09-12T03:20:31.571589+0000) 2024-09-12T03:20:58.745 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:58 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:58.684+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:19.074057+0000 front 2024-09-12T03:20:19.073877+0000 (oldest deadline 2024-09-12T03:20:42.573956+0000) 2024-09-12T03:20:58.745 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:58 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:58.684+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:24.275198+0000 front 2024-09-12T03:20:24.275279+0000 (oldest deadline 2024-09-12T03:20:48.975033+0000) 2024-09-12T03:20:58.745 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:58 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7[52774]: 2024-09-12T03:20:58.684+0000 7f6069843640 -1 osd.7 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:34.276117+0000 front 2024-09-12T03:20:34.276045+0000 (oldest deadline 2024-09-12T03:20:56.576137+0000) 2024-09-12T03:20:58.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:58 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:58.478+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:58.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:58 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:58.478+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:20:58.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:58 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:58.478+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:20:58.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:58 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:58.478+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:20:58.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:58 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:58.478+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.429190+0000 front 2024-09-12T03:20:30.429210+0000 (oldest deadline 2024-09-12T03:20:56.329320+0000) 2024-09-12T03:20:59.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:58 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:58.931+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:20:59.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:58 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:58.931+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:20:59.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:58 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:58.931+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:18.820516+0000 front 2024-09-12T03:20:18.820577+0000 (oldest deadline 2024-09-12T03:20:44.120637+0000) 2024-09-12T03:20:59.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:58 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:58.931+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.221507+0000 front 2024-09-12T03:20:25.221564+0000 (oldest deadline 2024-09-12T03:20:50.521570+0000) 2024-09-12T03:20:59.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:59 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:59.250+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:20:59.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:59 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:59.250+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:20:59.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:59 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:59.250+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:20:59.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:59 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:59.250+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:22.047371+0000 front 2024-09-12T03:20:22.047302+0000 (oldest deadline 2024-09-12T03:20:47.847767+0000) 2024-09-12T03:20:59.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:20:59 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:20:59.250+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.748800+0000 front 2024-09-12T03:20:30.748906+0000 (oldest deadline 2024-09-12T03:20:56.648741+0000) 2024-09-12T03:20:59.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:59 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:59.319+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:20:59.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:59 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:59.319+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:20:59.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:59 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:59.319+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:20:59.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:20:59 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:20:59.319+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:20:59.745 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:59 smithi177 podman[70952]: 2024-09-12 03:20:59.454114089 +0000 UTC m=+5.541922300 container died ce0e2a06101e3c0491ed6f09fcc70d1beb331554a48a337ffe8204a785c82e8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:20:59.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:59 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:59.502+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:20:59.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:59 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:59.502+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:20:59.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:59 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:59.502+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:20:59.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:59 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:59.502+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:20:59.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:20:59 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:20:59.502+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.429190+0000 front 2024-09-12T03:20:30.429210+0000 (oldest deadline 2024-09-12T03:20:56.329320+0000) 2024-09-12T03:21:00.118 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:59 smithi177 podman[70952]: 2024-09-12 03:20:59.869316184 +0000 UTC m=+5.957124393 container cleanup ce0e2a06101e3c0491ed6f09fcc70d1beb331554a48a337ffe8204a785c82e8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-12T03:21:00.118 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:20:59 smithi177 bash[70952]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7 2024-09-12T03:21:00.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:59 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:59.932+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:21:00.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:59 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:59.932+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:21:00.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:59 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:59.932+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:18.820516+0000 front 2024-09-12T03:20:18.820577+0000 (oldest deadline 2024-09-12T03:20:44.120637+0000) 2024-09-12T03:21:00.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:20:59 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:20:59.932+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.221507+0000 front 2024-09-12T03:20:25.221564+0000 (oldest deadline 2024-09-12T03:20:50.521570+0000) 2024-09-12T03:21:00.457 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:21:00 smithi177 podman[70979]: 2024-09-12 03:21:00.118335657 +0000 UTC m=+0.661180317 container remove ce0e2a06101e3c0491ed6f09fcc70d1beb331554a48a337ffe8204a785c82e8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-12T03:21:00.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:00 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:00.261+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:21:00.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:00 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:00.261+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:21:00.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:00 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:00.261+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:21:00.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:00 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:00.261+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:22.047371+0000 front 2024-09-12T03:20:22.047302+0000 (oldest deadline 2024-09-12T03:20:47.847767+0000) 2024-09-12T03:21:00.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:00 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:00.261+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.748800+0000 front 2024-09-12T03:20:30.748906+0000 (oldest deadline 2024-09-12T03:20:56.648741+0000) 2024-09-12T03:21:00.702 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:00 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:00.331+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:00.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:00 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:00.331+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:00.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:00 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:00.331+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:00.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:00 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:00.331+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:00.744 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:21:00 smithi177 podman[71067]: 2024-09-12 03:21:00.438850826 +0000 UTC m=+0.015993135 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:21:00.744 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:21:00 smithi177 podman[71067]: 2024-09-12 03:21:00.569204199 +0000 UTC m=+0.146346497 container create eb39ad09bb940e04028ec4df1b9ee85fb4b1534fb323ca2bcf573bfe0362d41a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-12T03:21:00.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:00 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:00.457+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:21:00.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:00 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:00.457+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:21:00.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:00 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:00.457+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:21:00.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:00 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:00.457+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:21:00.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:00 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:00.457+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.429190+0000 front 2024-09-12T03:20:30.429210+0000 (oldest deadline 2024-09-12T03:20:56.329320+0000) 2024-09-12T03:21:01.021 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:21:00 smithi177 podman[71067]: 2024-09-12 03:21:00.822217943 +0000 UTC m=+0.399360238 container init eb39ad09bb940e04028ec4df1b9ee85fb4b1534fb323ca2bcf573bfe0362d41a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-12T03:21:01.021 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:21:00 smithi177 podman[71067]: 2024-09-12 03:21:00.825706445 +0000 UTC m=+0.402848745 container start eb39ad09bb940e04028ec4df1b9ee85fb4b1534fb323ca2bcf573bfe0362d41a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:21:01.021 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:21:00 smithi177 podman[71067]: 2024-09-12 03:21:00.913789605 +0000 UTC m=+0.490931909 container attach eb39ad09bb940e04028ec4df1b9ee85fb4b1534fb323ca2bcf573bfe0362d41a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-12T03:21:01.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:00 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:00.924+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:21:01.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:00 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:00.924+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:21:01.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:00 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:00.924+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:18.820516+0000 front 2024-09-12T03:20:18.820577+0000 (oldest deadline 2024-09-12T03:20:44.120637+0000) 2024-09-12T03:21:01.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:00 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:00.924+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.221507+0000 front 2024-09-12T03:20:25.221564+0000 (oldest deadline 2024-09-12T03:20:50.521570+0000) 2024-09-12T03:21:01.366 INFO:journalctl@ceph.osd.7.smithi177.stdout:Sep 12 03:21:01 smithi177 podman[71067]: 2024-09-12 03:21:01.022056759 +0000 UTC m=+0.599199060 container died eb39ad09bb940e04028ec4df1b9ee85fb4b1534fb323ca2bcf573bfe0362d41a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-12T03:21:01.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:01 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:01.237+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:21:01.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:01 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:01.237+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:21:01.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:01 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:01.237+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:21:01.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:01 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:01.237+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:22.047371+0000 front 2024-09-12T03:20:22.047302+0000 (oldest deadline 2024-09-12T03:20:47.847767+0000) 2024-09-12T03:21:01.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:01 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:01.237+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.748800+0000 front 2024-09-12T03:20:30.748906+0000 (oldest deadline 2024-09-12T03:20:56.648741+0000) 2024-09-12T03:21:01.675 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:01 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:01.435+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:21:01.675 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:01 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:01.435+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:21:01.675 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:01 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:01.435+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:21:01.675 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:01 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:01.435+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:21:01.675 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:01 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:01.435+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.429190+0000 front 2024-09-12T03:20:30.429210+0000 (oldest deadline 2024-09-12T03:20:56.329320+0000) 2024-09-12T03:21:01.694 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.7.service' 2024-09-12T03:21:01.702 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:01 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:01.373+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:01.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:01 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:01.373+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:01.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:01 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:01.373+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:01.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:01 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:01.373+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:01.734 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:21:01.734 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-12T03:21:01.735 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-12T03:21:01.735 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.8 2024-09-12T03:21:02.063 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:01 smithi183 systemd[1]: Stopping Ceph osd.8 for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:21:02.063 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:01 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:01.879+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:21:02.063 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:01 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:01.879+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:21:02.063 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:01 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:01.879+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:18.820516+0000 front 2024-09-12T03:20:18.820577+0000 (oldest deadline 2024-09-12T03:20:44.120637+0000) 2024-09-12T03:21:02.063 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:01 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:01.879+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.221507+0000 front 2024-09-12T03:20:25.221564+0000 (oldest deadline 2024-09-12T03:20:50.521570+0000) 2024-09-12T03:21:02.063 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:02 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:02.062+0000 7f9006683640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:21:02.063 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:02 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:02.062+0000 7f9006683640 -1 osd.8 83 *** Got signal Terminated *** 2024-09-12T03:21:02.063 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:02 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:02.062+0000 7f9006683640 -1 osd.8 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:21:02.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:02 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:02.220+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:21:02.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:02 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:02.220+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:21:02.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:02 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:02.220+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:21:02.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:02 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:02.220+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:22.047371+0000 front 2024-09-12T03:20:22.047302+0000 (oldest deadline 2024-09-12T03:20:47.847767+0000) 2024-09-12T03:21:02.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:02 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:02.220+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.748800+0000 front 2024-09-12T03:20:30.748906+0000 (oldest deadline 2024-09-12T03:20:56.648741+0000) 2024-09-12T03:21:02.702 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:02 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:02.337+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:02.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:02 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:02.337+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:02.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:02 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:02.337+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:02.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:02 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:02.337+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:02.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:02 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:02.398+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:21:02.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:02 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:02.398+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:21:02.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:02 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:02.398+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:21:02.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:02 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:02.398+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:21:02.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:02 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:02.398+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.429190+0000 front 2024-09-12T03:20:30.429210+0000 (oldest deadline 2024-09-12T03:20:56.329320+0000) 2024-09-12T03:21:03.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:02 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:02.888+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:21:03.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:02 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:02.888+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:21:03.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:02 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:02.888+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:18.820516+0000 front 2024-09-12T03:20:18.820577+0000 (oldest deadline 2024-09-12T03:20:44.120637+0000) 2024-09-12T03:21:03.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:02 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:02.888+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.221507+0000 front 2024-09-12T03:20:25.221564+0000 (oldest deadline 2024-09-12T03:20:50.521570+0000) 2024-09-12T03:21:03.554 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:03 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:03.238+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:21:03.611 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:03 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:03.238+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:21:03.612 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:03 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:03.238+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:21:03.612 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:03 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:03.238+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:22.047371+0000 front 2024-09-12T03:20:22.047302+0000 (oldest deadline 2024-09-12T03:20:47.847767+0000) 2024-09-12T03:21:03.612 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:03 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:03.238+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.748800+0000 front 2024-09-12T03:20:30.748906+0000 (oldest deadline 2024-09-12T03:20:56.648741+0000) 2024-09-12T03:21:03.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:03 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:03.326+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:03.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:03 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:03.326+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:03.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:03 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:03.326+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:03.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:03 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:03.326+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:03.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:03 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:03.432+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:21:03.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:03 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:03.432+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:21:03.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:03 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:03.432+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:21:03.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:03 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:03.432+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:21:03.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:03 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:03.432+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.429190+0000 front 2024-09-12T03:20:30.429210+0000 (oldest deadline 2024-09-12T03:20:56.329320+0000) 2024-09-12T03:21:04.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:03 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:03.903+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:21:04.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:03 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:03.903+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:21:04.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:03 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:03.903+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:18.820516+0000 front 2024-09-12T03:20:18.820577+0000 (oldest deadline 2024-09-12T03:20:44.120637+0000) 2024-09-12T03:21:04.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:03 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:03.903+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.221507+0000 front 2024-09-12T03:20:25.221564+0000 (oldest deadline 2024-09-12T03:20:50.521570+0000) 2024-09-12T03:21:04.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:04 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:04.203+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:21:04.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:04 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:04.203+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:21:04.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:04 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:04.203+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:21:04.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:04 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:04.203+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:22.047371+0000 front 2024-09-12T03:20:22.047302+0000 (oldest deadline 2024-09-12T03:20:47.847767+0000) 2024-09-12T03:21:04.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:04 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:04.203+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.748800+0000 front 2024-09-12T03:20:30.748906+0000 (oldest deadline 2024-09-12T03:20:56.648741+0000) 2024-09-12T03:21:04.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:04 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:04.330+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:04.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:04 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:04.330+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:04.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:04 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:04.330+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:04.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:04 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:04.330+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:04.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:04 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:04.422+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:21:04.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:04 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:04.422+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:21:04.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:04 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:04.422+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:21:04.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:04 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:04.422+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:21:04.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:04 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:04.422+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.429190+0000 front 2024-09-12T03:20:30.429210+0000 (oldest deadline 2024-09-12T03:20:56.329320+0000) 2024-09-12T03:21:04.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:04 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:04.422+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:41.030723+0000 front 2024-09-12T03:20:41.030608+0000 (oldest deadline 2024-09-12T03:21:03.930458+0000) 2024-09-12T03:21:05.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:04 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:04.866+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:21:05.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:04 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:04.866+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:21:05.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:04 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:04.866+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:18.820516+0000 front 2024-09-12T03:20:18.820577+0000 (oldest deadline 2024-09-12T03:20:44.120637+0000) 2024-09-12T03:21:05.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:04 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:04.866+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.221507+0000 front 2024-09-12T03:20:25.221564+0000 (oldest deadline 2024-09-12T03:20:50.521570+0000) 2024-09-12T03:21:05.494 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:05 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:05.456+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:21:05.495 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:05 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:05.456+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:21:05.495 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:05 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:05.456+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:21:05.495 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:05 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:05.456+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:21:05.495 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:05 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:05.456+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.429190+0000 front 2024-09-12T03:20:30.429210+0000 (oldest deadline 2024-09-12T03:20:56.329320+0000) 2024-09-12T03:21:05.495 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:05 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:05.456+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:41.030723+0000 front 2024-09-12T03:20:41.030608+0000 (oldest deadline 2024-09-12T03:21:03.930458+0000) 2024-09-12T03:21:05.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:05 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:05.233+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:21:05.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:05 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:05.233+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:21:05.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:05 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:05.233+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:21:05.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:05 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:05.233+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:22.047371+0000 front 2024-09-12T03:20:22.047302+0000 (oldest deadline 2024-09-12T03:20:47.847767+0000) 2024-09-12T03:21:05.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:05 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:05.233+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.748800+0000 front 2024-09-12T03:20:30.748906+0000 (oldest deadline 2024-09-12T03:20:56.648741+0000) 2024-09-12T03:21:05.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:05 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:05.308+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:05.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:05 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:05.308+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:05.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:05 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:05.308+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:05.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:05 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:05.308+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:05.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:05 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:05.308+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:39.204667+0000 front 2024-09-12T03:20:39.204818+0000 (oldest deadline 2024-09-12T03:21:05.104795+0000) 2024-09-12T03:21:06.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:05 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:05.852+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:21:06.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:05 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:05.852+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:21:06.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:05 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:05.852+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:18.820516+0000 front 2024-09-12T03:20:18.820577+0000 (oldest deadline 2024-09-12T03:20:44.120637+0000) 2024-09-12T03:21:06.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:05 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:05.852+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.221507+0000 front 2024-09-12T03:20:25.221564+0000 (oldest deadline 2024-09-12T03:20:50.521570+0000) 2024-09-12T03:21:06.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:05 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:05.852+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:42.624230+0000 front 2024-09-12T03:20:42.624258+0000 (oldest deadline 2024-09-12T03:21:04.924312+0000) 2024-09-12T03:21:06.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:06 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:06.187+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:21:06.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:06 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:06.187+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:21:06.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:06 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:06.187+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:21:06.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:06 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:06.187+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:22.047371+0000 front 2024-09-12T03:20:22.047302+0000 (oldest deadline 2024-09-12T03:20:47.847767+0000) 2024-09-12T03:21:06.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:06 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:06.187+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.748800+0000 front 2024-09-12T03:20:30.748906+0000 (oldest deadline 2024-09-12T03:20:56.648741+0000) 2024-09-12T03:21:06.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:06 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:06.345+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:06.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:06 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:06.345+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:06.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:06 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:06.345+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:06.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:06 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:06.345+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:06.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:06 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:06.345+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:39.204667+0000 front 2024-09-12T03:20:39.204818+0000 (oldest deadline 2024-09-12T03:21:05.104795+0000) 2024-09-12T03:21:06.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:06 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:06.430+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:21:06.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:06 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:06.430+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:21:06.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:06 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:06.430+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:21:06.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:06 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:06.430+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:21:06.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:06 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:06.430+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.429190+0000 front 2024-09-12T03:20:30.429210+0000 (oldest deadline 2024-09-12T03:20:56.329320+0000) 2024-09-12T03:21:06.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:06 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:06.430+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:41.030723+0000 front 2024-09-12T03:20:41.030608+0000 (oldest deadline 2024-09-12T03:21:03.930458+0000) 2024-09-12T03:21:07.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:06 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:06.810+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.617425+0000 front 2024-09-12T03:20:02.617370+0000 (oldest deadline 2024-09-12T03:20:23.617883+0000) 2024-09-12T03:21:07.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:06 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:06.810+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:07.718884+0000 front 2024-09-12T03:20:07.719088+0000 (oldest deadline 2024-09-12T03:20:33.618939+0000) 2024-09-12T03:21:07.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:06 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:06.810+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:18.820516+0000 front 2024-09-12T03:20:18.820577+0000 (oldest deadline 2024-09-12T03:20:44.120637+0000) 2024-09-12T03:21:07.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:06 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:06.810+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.221507+0000 front 2024-09-12T03:20:25.221564+0000 (oldest deadline 2024-09-12T03:20:50.521570+0000) 2024-09-12T03:21:07.203 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:06 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8[54370]: 2024-09-12T03:21:06.810+0000 7f8ffec91640 -1 osd.8 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:42.624230+0000 front 2024-09-12T03:20:42.624258+0000 (oldest deadline 2024-09-12T03:21:04.924312+0000) 2024-09-12T03:21:07.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:07 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:07.192+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:21:07.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:07 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:07.192+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:21:07.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:07 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:07.192+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:21:07.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:07 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:07.192+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:22.047371+0000 front 2024-09-12T03:20:22.047302+0000 (oldest deadline 2024-09-12T03:20:47.847767+0000) 2024-09-12T03:21:07.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:07 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:07.192+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.748800+0000 front 2024-09-12T03:20:30.748906+0000 (oldest deadline 2024-09-12T03:20:56.648741+0000) 2024-09-12T03:21:07.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:07 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:07.192+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:40.749773+0000 front 2024-09-12T03:20:40.749755+0000 (oldest deadline 2024-09-12T03:21:06.649713+0000) 2024-09-12T03:21:07.703 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:07 smithi183 podman[72426]: 2024-09-12 03:21:07.422194607 +0000 UTC m=+5.485342542 container died faf5be3c6bc103d296190ce59cb44d98761897a3809762c8150ccacbe0abcdb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:21:07.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:07 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:07.384+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:07.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:07 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:07.384+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:07.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:07 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:07.384+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:07.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:07 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:07.384+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:07.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:07 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:07.384+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:39.204667+0000 front 2024-09-12T03:20:39.204818+0000 (oldest deadline 2024-09-12T03:21:05.104795+0000) 2024-09-12T03:21:07.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:07 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:07.396+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:21:07.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:07 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:07.396+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:21:07.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:07 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:07.396+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:21:07.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:07 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:07.396+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:21:07.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:07 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:07.396+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.429190+0000 front 2024-09-12T03:20:30.429210+0000 (oldest deadline 2024-09-12T03:20:56.329320+0000) 2024-09-12T03:21:07.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:07 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:07.396+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:41.030723+0000 front 2024-09-12T03:20:41.030608+0000 (oldest deadline 2024-09-12T03:21:03.930458+0000) 2024-09-12T03:21:08.077 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:07 smithi183 podman[72426]: 2024-09-12 03:21:07.804467996 +0000 UTC m=+5.867615909 container cleanup faf5be3c6bc103d296190ce59cb44d98761897a3809762c8150ccacbe0abcdb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-12T03:21:08.077 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:07 smithi183 bash[72426]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8 2024-09-12T03:21:08.421 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:08 smithi183 podman[72444]: 2024-09-12 03:21:08.072766791 +0000 UTC m=+0.653285500 container remove faf5be3c6bc103d296190ce59cb44d98761897a3809762c8150ccacbe0abcdb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:21:08.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:08 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:08.165+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:21:08.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:08 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:08.165+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:21:08.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:08 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:08.165+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:21:08.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:08 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:08.165+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:22.047371+0000 front 2024-09-12T03:20:22.047302+0000 (oldest deadline 2024-09-12T03:20:47.847767+0000) 2024-09-12T03:21:08.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:08 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:08.165+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.748800+0000 front 2024-09-12T03:20:30.748906+0000 (oldest deadline 2024-09-12T03:20:56.648741+0000) 2024-09-12T03:21:08.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:08 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:08.165+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:40.749773+0000 front 2024-09-12T03:20:40.749755+0000 (oldest deadline 2024-09-12T03:21:06.649713+0000) 2024-09-12T03:21:08.675 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:08 smithi183 podman[72534]: 2024-09-12 03:21:08.431862665 +0000 UTC m=+0.111294927 container create b43a9cc90b18541cdb019e4eaf48a697509e804f3a38aea2661ce7aac4850451 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-12T03:21:08.675 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:08 smithi183 podman[72534]: 2024-09-12 03:21:08.336327082 +0000 UTC m=+0.015759354 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:21:08.676 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:08 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:08.420+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:08.676 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:08 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:08.420+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:08.676 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:08 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:08.420+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:08.676 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:08 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:08.420+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:08.676 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:08 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:08.420+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:39.204667+0000 front 2024-09-12T03:20:39.204818+0000 (oldest deadline 2024-09-12T03:21:05.104795+0000) 2024-09-12T03:21:08.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:08 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:08.386+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:21:08.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:08 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:08.386+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:21:08.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:08 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:08.386+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:21:08.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:08 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:08.386+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:21:08.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:08 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:08.386+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.429190+0000 front 2024-09-12T03:20:30.429210+0000 (oldest deadline 2024-09-12T03:20:56.329320+0000) 2024-09-12T03:21:08.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:08 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:08.386+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:41.030723+0000 front 2024-09-12T03:20:41.030608+0000 (oldest deadline 2024-09-12T03:21:03.930458+0000) 2024-09-12T03:21:08.953 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:08 smithi183 podman[72534]: 2024-09-12 03:21:08.675939512 +0000 UTC m=+0.355371780 container init b43a9cc90b18541cdb019e4eaf48a697509e804f3a38aea2661ce7aac4850451 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-12T03:21:08.953 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:08 smithi183 podman[72534]: 2024-09-12 03:21:08.679483785 +0000 UTC m=+0.358916048 container start b43a9cc90b18541cdb019e4eaf48a697509e804f3a38aea2661ce7aac4850451 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-12T03:21:08.953 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:08 smithi183 podman[72534]: 2024-09-12 03:21:08.776044856 +0000 UTC m=+0.455477142 container attach b43a9cc90b18541cdb019e4eaf48a697509e804f3a38aea2661ce7aac4850451 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-12T03:21:08.953 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 12 03:21:08 smithi183 podman[72534]: 2024-09-12 03:21:08.869498654 +0000 UTC m=+0.548930917 container died b43a9cc90b18541cdb019e4eaf48a697509e804f3a38aea2661ce7aac4850451 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-12T03:21:09.521 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:09 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:09.424+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:09.521 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:09 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:09.424+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:09.521 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:09 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:09.424+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:09.521 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:09 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:09.424+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:09.521 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:09 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:09.424+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:39.204667+0000 front 2024-09-12T03:20:39.204818+0000 (oldest deadline 2024-09-12T03:21:05.104795+0000) 2024-09-12T03:21:09.550 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.8.service' 2024-09-12T03:21:09.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:09 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:09.205+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:21:09.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:09 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:09.205+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:21:09.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:09 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:09.205+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:21:09.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:09 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:09.205+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:22.047371+0000 front 2024-09-12T03:20:22.047302+0000 (oldest deadline 2024-09-12T03:20:47.847767+0000) 2024-09-12T03:21:09.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:09 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:09.205+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.748800+0000 front 2024-09-12T03:20:30.748906+0000 (oldest deadline 2024-09-12T03:20:56.648741+0000) 2024-09-12T03:21:09.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:09 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:09.205+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:40.749773+0000 front 2024-09-12T03:20:40.749755+0000 (oldest deadline 2024-09-12T03:21:06.649713+0000) 2024-09-12T03:21:09.590 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:21:09.590 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-12T03:21:09.590 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-12T03:21:09.590 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.9 2024-09-12T03:21:09.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:09 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:09.349+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:21:09.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:09 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:09.349+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:21:09.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:09 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:09.349+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:21:09.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:09 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:09.349+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:21:09.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:09 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:09.349+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.429190+0000 front 2024-09-12T03:20:30.429210+0000 (oldest deadline 2024-09-12T03:20:56.329320+0000) 2024-09-12T03:21:09.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:09 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:09.349+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:41.030723+0000 front 2024-09-12T03:20:41.030608+0000 (oldest deadline 2024-09-12T03:21:03.930458+0000) 2024-09-12T03:21:09.898 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:09 smithi116 systemd[1]: Stopping Ceph osd.9 for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:21:09.898 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:09 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:09.898+0000 7f6eff0b5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:21:09.898 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:09 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:09.898+0000 7f6eff0b5640 -1 osd.9 83 *** Got signal Terminated *** 2024-09-12T03:21:09.898 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:09 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:09.898+0000 7f6eff0b5640 -1 osd.9 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:21:10.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:10 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:10.214+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:21:10.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:10 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:10.214+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:21:10.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:10 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:10.214+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:21:10.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:10 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:10.214+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:22.047371+0000 front 2024-09-12T03:20:22.047302+0000 (oldest deadline 2024-09-12T03:20:47.847767+0000) 2024-09-12T03:21:10.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:10 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:10.214+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.748800+0000 front 2024-09-12T03:20:30.748906+0000 (oldest deadline 2024-09-12T03:20:56.648741+0000) 2024-09-12T03:21:10.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:10 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:10.214+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:40.749773+0000 front 2024-09-12T03:20:40.749755+0000 (oldest deadline 2024-09-12T03:21:06.649713+0000) 2024-09-12T03:21:10.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:10 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:10.415+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:10.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:10 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:10.415+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:10.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:10 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:10.415+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:10.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:10 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:10.415+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:10.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:10 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:10.415+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:39.204667+0000 front 2024-09-12T03:20:39.204818+0000 (oldest deadline 2024-09-12T03:21:05.104795+0000) 2024-09-12T03:21:10.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:10 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:10.310+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:21:10.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:10 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:10.310+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:21:10.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:10 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:10.310+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:21:10.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:10 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:10.310+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:21:10.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:10 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:10.310+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.429190+0000 front 2024-09-12T03:20:30.429210+0000 (oldest deadline 2024-09-12T03:20:56.329320+0000) 2024-09-12T03:21:10.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:10 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:10.310+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:41.030723+0000 front 2024-09-12T03:20:41.030608+0000 (oldest deadline 2024-09-12T03:21:03.930458+0000) 2024-09-12T03:21:11.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:11 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:11.262+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:21:11.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:11 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:11.262+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:21:11.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:11 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:11.262+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:21:11.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:11 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:11.262+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:22.047371+0000 front 2024-09-12T03:20:22.047302+0000 (oldest deadline 2024-09-12T03:20:47.847767+0000) 2024-09-12T03:21:11.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:11 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:11.262+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.748800+0000 front 2024-09-12T03:20:30.748906+0000 (oldest deadline 2024-09-12T03:20:56.648741+0000) 2024-09-12T03:21:11.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:11 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:11.262+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:40.749773+0000 front 2024-09-12T03:20:40.749755+0000 (oldest deadline 2024-09-12T03:21:06.649713+0000) 2024-09-12T03:21:11.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:11 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:11.416+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:11.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:11 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:11.416+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:11.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:11 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:11.416+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:11.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:11 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:11.416+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:11.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:11 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:11.416+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:39.204667+0000 front 2024-09-12T03:20:39.204818+0000 (oldest deadline 2024-09-12T03:21:05.104795+0000) 2024-09-12T03:21:11.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:11 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:11.330+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:21:11.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:11 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:11.330+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:21:11.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:11 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:11.330+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:21:11.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:11 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:11.330+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:21:11.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:11 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:11.330+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.429190+0000 front 2024-09-12T03:20:30.429210+0000 (oldest deadline 2024-09-12T03:20:56.329320+0000) 2024-09-12T03:21:11.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:11 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:11.330+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:41.030723+0000 front 2024-09-12T03:20:41.030608+0000 (oldest deadline 2024-09-12T03:21:03.930458+0000) 2024-09-12T03:21:12.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:12 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:12.239+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:21:12.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:12 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:12.239+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:21:12.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:12 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:12.239+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:21:12.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:12 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:12.239+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:22.047371+0000 front 2024-09-12T03:20:22.047302+0000 (oldest deadline 2024-09-12T03:20:47.847767+0000) 2024-09-12T03:21:12.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:12 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:12.239+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.748800+0000 front 2024-09-12T03:20:30.748906+0000 (oldest deadline 2024-09-12T03:20:56.648741+0000) 2024-09-12T03:21:12.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:12 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:12.239+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:40.749773+0000 front 2024-09-12T03:20:40.749755+0000 (oldest deadline 2024-09-12T03:21:06.649713+0000) 2024-09-12T03:21:12.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:12 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:12.382+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:12.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:12 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:12.382+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:12.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:12 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:12.382+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:12.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:12 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:12.382+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:12.705 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:12 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:12.382+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:39.204667+0000 front 2024-09-12T03:20:39.204818+0000 (oldest deadline 2024-09-12T03:21:05.104795+0000) 2024-09-12T03:21:12.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:12 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:12.316+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:21:12.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:12 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:12.316+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:21:12.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:12 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:12.316+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:21:12.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:12 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:12.316+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:21:12.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:12 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:12.316+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.429190+0000 front 2024-09-12T03:20:30.429210+0000 (oldest deadline 2024-09-12T03:20:56.329320+0000) 2024-09-12T03:21:12.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:12 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:12.316+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:41.030723+0000 front 2024-09-12T03:20:41.030608+0000 (oldest deadline 2024-09-12T03:21:03.930458+0000) 2024-09-12T03:21:13.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:13 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:13.205+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:21:13.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:13 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:13.205+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:21:13.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:13 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:13.205+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:21:13.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:13 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:13.205+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:22.047371+0000 front 2024-09-12T03:20:22.047302+0000 (oldest deadline 2024-09-12T03:20:47.847767+0000) 2024-09-12T03:21:13.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:13 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:13.205+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.748800+0000 front 2024-09-12T03:20:30.748906+0000 (oldest deadline 2024-09-12T03:20:56.648741+0000) 2024-09-12T03:21:13.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:13 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:13.205+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:40.749773+0000 front 2024-09-12T03:20:40.749755+0000 (oldest deadline 2024-09-12T03:21:06.649713+0000) 2024-09-12T03:21:13.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:13 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:13.389+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:13.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:13 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:13.389+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:13.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:13 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:13.389+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:13.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:13 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:13.389+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:13.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:13 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:13.389+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:39.204667+0000 front 2024-09-12T03:20:39.204818+0000 (oldest deadline 2024-09-12T03:21:05.104795+0000) 2024-09-12T03:21:13.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:13 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:13.294+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:21:13.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:13 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:13.294+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:21:13.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:13 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:13.294+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:21:13.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:13 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:13.294+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:21:13.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:13 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:13.294+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.429190+0000 front 2024-09-12T03:20:30.429210+0000 (oldest deadline 2024-09-12T03:20:56.329320+0000) 2024-09-12T03:21:13.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:13 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:13.294+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:41.030723+0000 front 2024-09-12T03:20:41.030608+0000 (oldest deadline 2024-09-12T03:21:03.930458+0000) 2024-09-12T03:21:13.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:13 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:13.294+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6822 osd.6 since back 2024-09-12T03:20:47.931928+0000 front 2024-09-12T03:20:47.931886+0000 (oldest deadline 2024-09-12T03:21:13.231933+0000) 2024-09-12T03:21:14.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:14 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:14.171+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:02.144858+0000 front 2024-09-12T03:20:02.144885+0000 (oldest deadline 2024-09-12T03:20:24.445016+0000) 2024-09-12T03:21:14.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:14 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:14.171+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.345911+0000 front 2024-09-12T03:20:10.346096+0000 (oldest deadline 2024-09-12T03:20:36.146415+0000) 2024-09-12T03:21:14.552 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:14 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:14.171+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.146962+0000 front 2024-09-12T03:20:16.147047+0000 (oldest deadline 2024-09-12T03:20:42.046995+0000) 2024-09-12T03:21:14.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:14 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:14.171+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:22.047371+0000 front 2024-09-12T03:20:22.047302+0000 (oldest deadline 2024-09-12T03:20:47.847767+0000) 2024-09-12T03:21:14.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:14 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:14.171+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.748800+0000 front 2024-09-12T03:20:30.748906+0000 (oldest deadline 2024-09-12T03:20:56.648741+0000) 2024-09-12T03:21:14.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:14 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:14.171+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:40.749773+0000 front 2024-09-12T03:20:40.749755+0000 (oldest deadline 2024-09-12T03:21:06.649713+0000) 2024-09-12T03:21:14.553 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:14 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9[65721]: 2024-09-12T03:21:14.171+0000 7f6efb6cb640 -1 osd.9 83 heartbeat_check: no reply from 172.21.15.116:6822 osd.6 since back 2024-09-12T03:20:50.051145+0000 front 2024-09-12T03:20:50.051186+0000 (oldest deadline 2024-09-12T03:21:13.551208+0000) 2024-09-12T03:21:14.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:14 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:14.346+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:14.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:14 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:14.346+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:14.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:14 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:14.346+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:14.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:14 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:14.346+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:14.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:14 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:14.346+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:39.204667+0000 front 2024-09-12T03:20:39.204818+0000 (oldest deadline 2024-09-12T03:21:05.104795+0000) 2024-09-12T03:21:14.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:14 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:14.346+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6822 osd.6 since back 2024-09-12T03:20:49.205798+0000 front 2024-09-12T03:20:49.205810+0000 (oldest deadline 2024-09-12T03:21:13.905583+0000) 2024-09-12T03:21:14.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:14 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:14.276+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:21:14.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:14 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:14.276+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:21:14.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:14 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:14.276+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:21:14.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:14 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:14.276+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:21:14.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:14 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:14.276+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.429190+0000 front 2024-09-12T03:20:30.429210+0000 (oldest deadline 2024-09-12T03:20:56.329320+0000) 2024-09-12T03:21:14.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:14 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:14.276+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:41.030723+0000 front 2024-09-12T03:20:41.030608+0000 (oldest deadline 2024-09-12T03:21:03.930458+0000) 2024-09-12T03:21:14.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:14 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:14.276+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6822 osd.6 since back 2024-09-12T03:20:47.931928+0000 front 2024-09-12T03:20:47.931886+0000 (oldest deadline 2024-09-12T03:21:13.231933+0000) 2024-09-12T03:21:15.656 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:15 smithi116 podman[129873]: 2024-09-12 03:21:15.399804566 +0000 UTC m=+5.614120938 container died d20ea0fc4a88f8c01fbbffc6efcba13775b73760c119d5e87511762737bcb844 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-12T03:21:15.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:15 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:15.317+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:15.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:15 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:15.317+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:15.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:15 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:15.317+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:15.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:15 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:15.317+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:15.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:15 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:15.317+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:39.204667+0000 front 2024-09-12T03:20:39.204818+0000 (oldest deadline 2024-09-12T03:21:05.104795+0000) 2024-09-12T03:21:15.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:15 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:15.317+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6822 osd.6 since back 2024-09-12T03:20:49.205798+0000 front 2024-09-12T03:20:49.205810+0000 (oldest deadline 2024-09-12T03:21:13.905583+0000) 2024-09-12T03:21:15.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:15 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:15.304+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:21:15.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:15 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:15.304+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:21:15.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:15 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:15.304+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:21:15.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:15 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:15.304+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:21:15.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:15 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:15.304+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.429190+0000 front 2024-09-12T03:20:30.429210+0000 (oldest deadline 2024-09-12T03:20:56.329320+0000) 2024-09-12T03:21:15.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:15 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:15.304+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:41.030723+0000 front 2024-09-12T03:20:41.030608+0000 (oldest deadline 2024-09-12T03:21:03.930458+0000) 2024-09-12T03:21:15.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:15 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:15.304+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6822 osd.6 since back 2024-09-12T03:20:47.931928+0000 front 2024-09-12T03:20:47.931886+0000 (oldest deadline 2024-09-12T03:21:13.231933+0000) 2024-09-12T03:21:16.052 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:15 smithi116 podman[129873]: 2024-09-12 03:21:15.731088246 +0000 UTC m=+5.945404620 container cleanup d20ea0fc4a88f8c01fbbffc6efcba13775b73760c119d5e87511762737bcb844 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-12T03:21:16.052 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:15 smithi116 bash[129873]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9 2024-09-12T03:21:16.052 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:15 smithi116 podman[129891]: 2024-09-12 03:21:15.898736475 +0000 UTC m=+0.493888582 container remove d20ea0fc4a88f8c01fbbffc6efcba13775b73760c119d5e87511762737bcb844 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-12T03:21:16.601 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:16 smithi116 podman[129979]: 2024-09-12 03:21:16.241641055 +0000 UTC m=+0.015444289 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:21:16.601 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:16 smithi116 podman[129979]: 2024-09-12 03:21:16.34098412 +0000 UTC m=+0.114787342 container create 010ed3a0c37f3bf4eaaed6a55df92dc6e28b4c358e25a8fcb23e863957b8104d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-12T03:21:16.702 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:16 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:16.294+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:16.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:16 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:16.294+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:16.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:16 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:16.294+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:16.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:16 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:16.294+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:16.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:16 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:16.294+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:39.204667+0000 front 2024-09-12T03:20:39.204818+0000 (oldest deadline 2024-09-12T03:21:05.104795+0000) 2024-09-12T03:21:16.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:16 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:16.294+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6822 osd.6 since back 2024-09-12T03:20:49.205798+0000 front 2024-09-12T03:20:49.205810+0000 (oldest deadline 2024-09-12T03:21:13.905583+0000) 2024-09-12T03:21:16.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:16 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:16.338+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:21:16.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:16 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:16.338+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:21:16.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:16 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:16.338+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:21:16.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:16 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:16.338+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:21:16.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:16 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:16.338+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.429190+0000 front 2024-09-12T03:20:30.429210+0000 (oldest deadline 2024-09-12T03:20:56.329320+0000) 2024-09-12T03:21:16.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:16 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:16.338+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:41.030723+0000 front 2024-09-12T03:20:41.030608+0000 (oldest deadline 2024-09-12T03:21:03.930458+0000) 2024-09-12T03:21:16.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:16 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:16.338+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6822 osd.6 since back 2024-09-12T03:20:47.931928+0000 front 2024-09-12T03:20:47.931886+0000 (oldest deadline 2024-09-12T03:21:13.231933+0000) 2024-09-12T03:21:17.052 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:16 smithi116 podman[129979]: 2024-09-12 03:21:16.602011508 +0000 UTC m=+0.375814740 container init 010ed3a0c37f3bf4eaaed6a55df92dc6e28b4c358e25a8fcb23e863957b8104d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-12T03:21:17.053 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:16 smithi116 podman[129979]: 2024-09-12 03:21:16.605323778 +0000 UTC m=+0.379127002 container start 010ed3a0c37f3bf4eaaed6a55df92dc6e28b4c358e25a8fcb23e863957b8104d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True) 2024-09-12T03:21:17.053 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:16 smithi116 podman[129979]: 2024-09-12 03:21:16.693517987 +0000 UTC m=+0.467321218 container attach 010ed3a0c37f3bf4eaaed6a55df92dc6e28b4c358e25a8fcb23e863957b8104d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9-deactivate, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:21:17.053 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:16 smithi116 podman[129979]: 2024-09-12 03:21:16.798389181 +0000 UTC m=+0.572192413 container died 010ed3a0c37f3bf4eaaed6a55df92dc6e28b4c358e25a8fcb23e863957b8104d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:21:17.359 INFO:journalctl@ceph.osd.9.smithi116.stdout:Sep 12 03:21:17 smithi116 podman[130003]: 2024-09-12 03:21:17.338492597 +0000 UTC m=+0.534174617 container remove 010ed3a0c37f3bf4eaaed6a55df92dc6e28b4c358e25a8fcb23e863957b8104d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-12T03:21:17.364 DEBUG:teuthology.orchestra.run.smithi116:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.9.service' 2024-09-12T03:21:17.404 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:21:17.405 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-12T03:21:17.405 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-12T03:21:17.405 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.10 2024-09-12T03:21:17.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:17 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:17.253+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:17.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:17 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:17.253+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:17.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:17 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:17.253+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:17.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:17 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:17.253+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:17.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:17 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:17.253+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:39.204667+0000 front 2024-09-12T03:20:39.204818+0000 (oldest deadline 2024-09-12T03:21:05.104795+0000) 2024-09-12T03:21:17.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:17 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:17.253+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6822 osd.6 since back 2024-09-12T03:20:49.205798+0000 front 2024-09-12T03:20:49.205810+0000 (oldest deadline 2024-09-12T03:21:13.905583+0000) 2024-09-12T03:21:17.735 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:17 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:17.348+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:21:17.735 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:17 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:17.348+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:21:17.735 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:17 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:17.348+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:21:17.735 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:17 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:17.348+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:21:17.735 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:17 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:17.348+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.429190+0000 front 2024-09-12T03:20:30.429210+0000 (oldest deadline 2024-09-12T03:20:56.329320+0000) 2024-09-12T03:21:17.735 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:17 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:17.348+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:41.030723+0000 front 2024-09-12T03:20:41.030608+0000 (oldest deadline 2024-09-12T03:21:03.930458+0000) 2024-09-12T03:21:17.735 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:17 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:17.348+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6822 osd.6 since back 2024-09-12T03:20:47.931928+0000 front 2024-09-12T03:20:47.931886+0000 (oldest deadline 2024-09-12T03:21:13.231933+0000) 2024-09-12T03:21:17.735 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:17 smithi177 systemd[1]: Stopping Ceph osd.10 for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:21:17.735 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:17 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:17.734+0000 7fe5aa63f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:21:17.736 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:17 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:17.734+0000 7fe5aa63f640 -1 osd.10 83 *** Got signal Terminated *** 2024-09-12T03:21:17.736 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:17 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:17.734+0000 7fe5aa63f640 -1 osd.10 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:21:18.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:18 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:18.287+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:18.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:18 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:18.287+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:18.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:18 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:18.287+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:18.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:18 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:18.287+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:18.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:18 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:18.287+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:39.204667+0000 front 2024-09-12T03:20:39.204818+0000 (oldest deadline 2024-09-12T03:21:05.104795+0000) 2024-09-12T03:21:18.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:18 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:18.287+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6822 osd.6 since back 2024-09-12T03:20:49.205798+0000 front 2024-09-12T03:20:49.205810+0000 (oldest deadline 2024-09-12T03:21:13.905583+0000) 2024-09-12T03:21:18.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:18 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:18.368+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:21:18.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:18 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:18.368+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:21:18.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:18 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:18.368+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:21:18.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:18 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:18.368+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:21:18.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:18 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:18.368+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.429190+0000 front 2024-09-12T03:20:30.429210+0000 (oldest deadline 2024-09-12T03:20:56.329320+0000) 2024-09-12T03:21:18.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:18 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:18.368+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:41.030723+0000 front 2024-09-12T03:20:41.030608+0000 (oldest deadline 2024-09-12T03:21:03.930458+0000) 2024-09-12T03:21:18.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:18 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:18.368+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6822 osd.6 since back 2024-09-12T03:20:47.931928+0000 front 2024-09-12T03:20:47.931886+0000 (oldest deadline 2024-09-12T03:21:13.231933+0000) 2024-09-12T03:21:19.604 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:19 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:19.318+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:19.604 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:19 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:19.318+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:19.605 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:19 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:19.318+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:19.605 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:19 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:19.318+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:19.605 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:19 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:19.318+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:39.204667+0000 front 2024-09-12T03:20:39.204818+0000 (oldest deadline 2024-09-12T03:21:05.104795+0000) 2024-09-12T03:21:19.605 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:19 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:19.318+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6822 osd.6 since back 2024-09-12T03:20:49.205798+0000 front 2024-09-12T03:20:49.205810+0000 (oldest deadline 2024-09-12T03:21:13.905583+0000) 2024-09-12T03:21:19.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:19 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:19.376+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:21:19.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:19 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:19.376+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:21:19.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:19 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:19.376+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:21:19.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:19 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:19.376+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:21:19.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:19 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:19.376+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.429190+0000 front 2024-09-12T03:20:30.429210+0000 (oldest deadline 2024-09-12T03:20:56.329320+0000) 2024-09-12T03:21:19.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:19 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:19.376+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:41.030723+0000 front 2024-09-12T03:20:41.030608+0000 (oldest deadline 2024-09-12T03:21:03.930458+0000) 2024-09-12T03:21:19.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:19 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:19.376+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6822 osd.6 since back 2024-09-12T03:20:47.931928+0000 front 2024-09-12T03:20:47.931886+0000 (oldest deadline 2024-09-12T03:21:13.231933+0000) 2024-09-12T03:21:20.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:20 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:20.303+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:20.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:20 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:20.303+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:20.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:20 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:20.303+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:20.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:20 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:20.303+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:20.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:20 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:20.303+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:39.204667+0000 front 2024-09-12T03:20:39.204818+0000 (oldest deadline 2024-09-12T03:21:05.104795+0000) 2024-09-12T03:21:20.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:20 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:20.303+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6822 osd.6 since back 2024-09-12T03:20:49.205798+0000 front 2024-09-12T03:20:49.205810+0000 (oldest deadline 2024-09-12T03:21:13.905583+0000) 2024-09-12T03:21:20.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:20 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:20.303+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6820 osd.7 since back 2024-09-12T03:20:59.007533+0000 front 2024-09-12T03:20:59.007457+0000 (oldest deadline 2024-09-12T03:21:19.507346+0000) 2024-09-12T03:21:20.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:20 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:20.383+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:21:20.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:20 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:20.383+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:21:20.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:20 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:20.383+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:21:20.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:20 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:20.383+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:21:20.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:20 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:20.383+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.429190+0000 front 2024-09-12T03:20:30.429210+0000 (oldest deadline 2024-09-12T03:20:56.329320+0000) 2024-09-12T03:21:20.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:20 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:20.383+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:41.030723+0000 front 2024-09-12T03:20:41.030608+0000 (oldest deadline 2024-09-12T03:21:03.930458+0000) 2024-09-12T03:21:20.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:20 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:20.383+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6822 osd.6 since back 2024-09-12T03:20:47.931928+0000 front 2024-09-12T03:20:47.931886+0000 (oldest deadline 2024-09-12T03:21:13.231933+0000) 2024-09-12T03:21:21.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:21 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:21.321+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:21.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:21 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:21.321+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:21.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:21 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:21.321+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:21.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:21 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:21.321+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:21.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:21 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:21.321+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:39.204667+0000 front 2024-09-12T03:20:39.204818+0000 (oldest deadline 2024-09-12T03:21:05.104795+0000) 2024-09-12T03:21:21.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:21 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:21.321+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6822 osd.6 since back 2024-09-12T03:20:49.205798+0000 front 2024-09-12T03:20:49.205810+0000 (oldest deadline 2024-09-12T03:21:13.905583+0000) 2024-09-12T03:21:21.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:21 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:21.321+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6820 osd.7 since back 2024-09-12T03:20:59.007533+0000 front 2024-09-12T03:20:59.007457+0000 (oldest deadline 2024-09-12T03:21:19.507346+0000) 2024-09-12T03:21:21.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:21 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:21.398+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:21:21.746 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:21 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:21.398+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:21:21.746 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:21 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:21.398+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:21:21.746 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:21 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:21.398+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:21:21.746 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:21 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:21.398+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.429190+0000 front 2024-09-12T03:20:30.429210+0000 (oldest deadline 2024-09-12T03:20:56.329320+0000) 2024-09-12T03:21:21.746 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:21 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:21.398+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:41.030723+0000 front 2024-09-12T03:20:41.030608+0000 (oldest deadline 2024-09-12T03:21:03.930458+0000) 2024-09-12T03:21:21.746 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:21 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:21.398+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6822 osd.6 since back 2024-09-12T03:20:47.931928+0000 front 2024-09-12T03:20:47.931886+0000 (oldest deadline 2024-09-12T03:21:13.231933+0000) 2024-09-12T03:21:22.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:22 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:22.365+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:22.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:22 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:22.365+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:22.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:22 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:22.365+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:22.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:22 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:22.365+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:22.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:22 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:22.365+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:39.204667+0000 front 2024-09-12T03:20:39.204818+0000 (oldest deadline 2024-09-12T03:21:05.104795+0000) 2024-09-12T03:21:22.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:22 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:22.365+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6822 osd.6 since back 2024-09-12T03:20:49.205798+0000 front 2024-09-12T03:20:49.205810+0000 (oldest deadline 2024-09-12T03:21:13.905583+0000) 2024-09-12T03:21:22.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:22 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:22.365+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6820 osd.7 since back 2024-09-12T03:20:59.007533+0000 front 2024-09-12T03:20:59.007457+0000 (oldest deadline 2024-09-12T03:21:19.507346+0000) 2024-09-12T03:21:22.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:22 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:22.406+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.324951+0000 front 2024-09-12T03:20:01.324932+0000 (oldest deadline 2024-09-12T03:20:26.624749+0000) 2024-09-12T03:21:22.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:22 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:22.406+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6804 osd.1 since back 2024-09-12T03:20:10.626177+0000 front 2024-09-12T03:20:10.626199+0000 (oldest deadline 2024-09-12T03:20:32.326396+0000) 2024-09-12T03:21:22.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:22 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:22.406+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:16.427333+0000 front 2024-09-12T03:20:16.427276+0000 (oldest deadline 2024-09-12T03:20:41.727456+0000) 2024-09-12T03:21:22.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:22 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:22.406+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:25.828584+0000 front 2024-09-12T03:20:25.828747+0000 (oldest deadline 2024-09-12T03:20:47.528577+0000) 2024-09-12T03:21:22.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:22 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:22.406+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:30.429190+0000 front 2024-09-12T03:20:30.429210+0000 (oldest deadline 2024-09-12T03:20:56.329320+0000) 2024-09-12T03:21:22.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:22 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:22.406+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:41.030723+0000 front 2024-09-12T03:20:41.030608+0000 (oldest deadline 2024-09-12T03:21:03.930458+0000) 2024-09-12T03:21:22.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:22 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10[59714]: 2024-09-12T03:21:22.406+0000 7fe5a2c4d640 -1 osd.10 83 heartbeat_check: no reply from 172.21.15.116:6822 osd.6 since back 2024-09-12T03:20:47.931928+0000 front 2024-09-12T03:20:47.931886+0000 (oldest deadline 2024-09-12T03:21:13.231933+0000) 2024-09-12T03:21:23.452 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:23 smithi177 podman[71168]: 2024-09-12 03:21:23.179025204 +0000 UTC m=+5.572858606 container died 12214f9b0b40cd430d7ed6ead8d0c85a15f54a58fc087e353a90868634e4e24d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-12T03:21:23.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:23 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:23.399+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:23.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:23 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:23.399+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:23.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:23 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:23.399+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:23.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:23 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:23.399+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:23.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:23 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:23.399+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:39.204667+0000 front 2024-09-12T03:20:39.204818+0000 (oldest deadline 2024-09-12T03:21:05.104795+0000) 2024-09-12T03:21:23.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:23 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:23.399+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6822 osd.6 since back 2024-09-12T03:20:49.205798+0000 front 2024-09-12T03:20:49.205810+0000 (oldest deadline 2024-09-12T03:21:13.905583+0000) 2024-09-12T03:21:23.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:23 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:23.399+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6820 osd.7 since back 2024-09-12T03:20:59.007533+0000 front 2024-09-12T03:20:59.007457+0000 (oldest deadline 2024-09-12T03:21:19.507346+0000) 2024-09-12T03:21:23.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:23 smithi177 podman[71168]: 2024-09-12 03:21:23.551436544 +0000 UTC m=+5.945269919 container cleanup 12214f9b0b40cd430d7ed6ead8d0c85a15f54a58fc087e353a90868634e4e24d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-12T03:21:23.745 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:23 smithi177 bash[71168]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10 2024-09-12T03:21:24.159 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:23 smithi177 podman[71182]: 2024-09-12 03:21:23.844455222 +0000 UTC m=+0.666918042 container remove 12214f9b0b40cd430d7ed6ead8d0c85a15f54a58fc087e353a90868634e4e24d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:21:24.472 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:24 smithi177 podman[71270]: 2024-09-12 03:21:24.0623685 +0000 UTC m=+0.015660639 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:21:24.472 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:24 smithi177 podman[71270]: 2024-09-12 03:21:24.170015812 +0000 UTC m=+0.123307938 container create 48a63c8cdaf697f16531159f74e487a073ff3381f964b104a6ad4055aac5a8ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-12T03:21:24.472 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:24 smithi177 podman[71270]: 2024-09-12 03:21:24.389157886 +0000 UTC m=+0.342450013 container init 48a63c8cdaf697f16531159f74e487a073ff3381f964b104a6ad4055aac5a8ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-09-12T03:21:24.472 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:24 smithi177 podman[71270]: 2024-09-12 03:21:24.392986957 +0000 UTC m=+0.346279091 container start 48a63c8cdaf697f16531159f74e487a073ff3381f964b104a6ad4055aac5a8ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3) 2024-09-12T03:21:24.472 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:24 smithi177 podman[71270]: 2024-09-12 03:21:24.47242649 +0000 UTC m=+0.425718622 container attach 48a63c8cdaf697f16531159f74e487a073ff3381f964b104a6ad4055aac5a8ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-12T03:21:24.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:24 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:24.403+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:24.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:24 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:24.403+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:24.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:24 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:24.403+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:24.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:24 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:24.403+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:24.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:24 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:24.403+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:39.204667+0000 front 2024-09-12T03:20:39.204818+0000 (oldest deadline 2024-09-12T03:21:05.104795+0000) 2024-09-12T03:21:24.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:24 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:24.403+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6822 osd.6 since back 2024-09-12T03:20:49.205798+0000 front 2024-09-12T03:20:49.205810+0000 (oldest deadline 2024-09-12T03:21:13.905583+0000) 2024-09-12T03:21:24.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:24 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:24.403+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6820 osd.7 since back 2024-09-12T03:20:59.007533+0000 front 2024-09-12T03:20:59.007457+0000 (oldest deadline 2024-09-12T03:21:19.507346+0000) 2024-09-12T03:21:24.744 INFO:journalctl@ceph.osd.10.smithi177.stdout:Sep 12 03:21:24 smithi177 podman[71270]: 2024-09-12 03:21:24.578347418 +0000 UTC m=+0.531639545 container died 48a63c8cdaf697f16531159f74e487a073ff3381f964b104a6ad4055aac5a8ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-10-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-12T03:21:25.194 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.10.service' 2024-09-12T03:21:25.235 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:21:25.236 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-12T03:21:25.236 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-12T03:21:25.236 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.11 2024-09-12T03:21:25.549 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:25 smithi183 systemd[1]: Stopping Ceph osd.11 for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:21:25.549 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:25 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:25.449+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:25.549 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:25 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:25.449+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:25.549 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:25 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:25.449+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:25.549 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:25 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:25.449+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:25.549 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:25 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:25.449+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:39.204667+0000 front 2024-09-12T03:20:39.204818+0000 (oldest deadline 2024-09-12T03:21:05.104795+0000) 2024-09-12T03:21:25.549 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:25 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:25.449+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6822 osd.6 since back 2024-09-12T03:20:49.205798+0000 front 2024-09-12T03:20:49.205810+0000 (oldest deadline 2024-09-12T03:21:13.905583+0000) 2024-09-12T03:21:25.549 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:25 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:25.449+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6820 osd.7 since back 2024-09-12T03:20:59.007533+0000 front 2024-09-12T03:20:59.007457+0000 (oldest deadline 2024-09-12T03:21:19.507346+0000) 2024-09-12T03:21:25.549 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:25 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:25.549+0000 7f861e83b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:21:25.549 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:25 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:25.549+0000 7f861e83b640 -1 osd.11 83 *** Got signal Terminated *** 2024-09-12T03:21:25.549 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:25 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:25.549+0000 7f861e83b640 -1 osd.11 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:21:26.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:26 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:26.401+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:26.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:26 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:26.401+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:26.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:26 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:26.401+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:26.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:26 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:26.401+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:26.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:26 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:26.401+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:39.204667+0000 front 2024-09-12T03:20:39.204818+0000 (oldest deadline 2024-09-12T03:21:05.104795+0000) 2024-09-12T03:21:26.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:26 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:26.401+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6822 osd.6 since back 2024-09-12T03:20:49.205798+0000 front 2024-09-12T03:20:49.205810+0000 (oldest deadline 2024-09-12T03:21:13.905583+0000) 2024-09-12T03:21:26.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:26 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:26.401+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6820 osd.7 since back 2024-09-12T03:20:59.007533+0000 front 2024-09-12T03:20:59.007457+0000 (oldest deadline 2024-09-12T03:21:19.507346+0000) 2024-09-12T03:21:27.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:27 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:27.364+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:27.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:27 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:27.364+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:27.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:27 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:27.364+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:27.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:27 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:27.364+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:27.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:27 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:27.364+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:39.204667+0000 front 2024-09-12T03:20:39.204818+0000 (oldest deadline 2024-09-12T03:21:05.104795+0000) 2024-09-12T03:21:27.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:27 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:27.364+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6822 osd.6 since back 2024-09-12T03:20:49.205798+0000 front 2024-09-12T03:20:49.205810+0000 (oldest deadline 2024-09-12T03:21:13.905583+0000) 2024-09-12T03:21:27.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:27 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:27.364+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6820 osd.7 since back 2024-09-12T03:20:59.007533+0000 front 2024-09-12T03:20:59.007457+0000 (oldest deadline 2024-09-12T03:21:19.507346+0000) 2024-09-12T03:21:28.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:28 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:28.364+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:28.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:28 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:28.364+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:28.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:28 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:28.364+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:28.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:28 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:28.364+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:28.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:28 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:28.364+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:39.204667+0000 front 2024-09-12T03:20:39.204818+0000 (oldest deadline 2024-09-12T03:21:05.104795+0000) 2024-09-12T03:21:28.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:28 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:28.364+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6822 osd.6 since back 2024-09-12T03:20:49.205798+0000 front 2024-09-12T03:20:49.205810+0000 (oldest deadline 2024-09-12T03:21:13.905583+0000) 2024-09-12T03:21:28.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:28 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:28.364+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6820 osd.7 since back 2024-09-12T03:20:59.007533+0000 front 2024-09-12T03:20:59.007457+0000 (oldest deadline 2024-09-12T03:21:19.507346+0000) 2024-09-12T03:21:28.704 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:28 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:28.364+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6820 osd.8 since back 2024-09-12T03:21:06.508717+0000 front 2024-09-12T03:21:06.508779+0000 (oldest deadline 2024-09-12T03:21:27.608822+0000) 2024-09-12T03:21:29.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:29 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:29.365+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:29.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:29 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:29.365+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:29.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:29 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:29.365+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:29.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:29 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:29.365+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:29.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:29 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:29.365+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:39.204667+0000 front 2024-09-12T03:20:39.204818+0000 (oldest deadline 2024-09-12T03:21:05.104795+0000) 2024-09-12T03:21:29.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:29 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:29.365+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6822 osd.6 since back 2024-09-12T03:20:49.205798+0000 front 2024-09-12T03:20:49.205810+0000 (oldest deadline 2024-09-12T03:21:13.905583+0000) 2024-09-12T03:21:29.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:29 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:29.365+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6820 osd.7 since back 2024-09-12T03:20:59.007533+0000 front 2024-09-12T03:20:59.007457+0000 (oldest deadline 2024-09-12T03:21:19.507346+0000) 2024-09-12T03:21:29.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:29 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:29.365+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6820 osd.8 since back 2024-09-12T03:21:06.508717+0000 front 2024-09-12T03:21:06.508779+0000 (oldest deadline 2024-09-12T03:21:27.608822+0000) 2024-09-12T03:21:30.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:30 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:30.414+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6806 osd.0 since back 2024-09-12T03:20:01.398929+0000 front 2024-09-12T03:20:01.398987+0000 (oldest deadline 2024-09-12T03:20:23.698874+0000) 2024-09-12T03:21:30.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:30 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:30.414+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-12T03:20:17.001849+0000 front 2024-09-12T03:20:17.001870+0000 (oldest deadline 2024-09-12T03:20:39.901950+0000) 2024-09-12T03:21:30.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:30 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:30.414+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6814 osd.3 since back 2024-09-12T03:20:26.903577+0000 front 2024-09-12T03:20:26.903626+0000 (oldest deadline 2024-09-12T03:20:52.803496+0000) 2024-09-12T03:21:30.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:30 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:30.414+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6812 osd.4 since back 2024-09-12T03:20:33.904355+0000 front 2024-09-12T03:20:33.904332+0000 (oldest deadline 2024-09-12T03:20:59.204338+0000) 2024-09-12T03:21:30.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:30 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:30.414+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-12T03:20:39.204667+0000 front 2024-09-12T03:20:39.204818+0000 (oldest deadline 2024-09-12T03:21:05.104795+0000) 2024-09-12T03:21:30.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:30 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:30.414+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.116:6822 osd.6 since back 2024-09-12T03:20:49.205798+0000 front 2024-09-12T03:20:49.205810+0000 (oldest deadline 2024-09-12T03:21:13.905583+0000) 2024-09-12T03:21:30.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:30 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:30.414+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.177:6820 osd.7 since back 2024-09-12T03:20:59.007533+0000 front 2024-09-12T03:20:59.007457+0000 (oldest deadline 2024-09-12T03:21:19.507346+0000) 2024-09-12T03:21:30.703 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:30 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11[60931]: 2024-09-12T03:21:30.414+0000 7f861ae51640 -1 osd.11 83 heartbeat_check: no reply from 172.21.15.183:6820 osd.8 since back 2024-09-12T03:21:06.508717+0000 front 2024-09-12T03:21:06.508779+0000 (oldest deadline 2024-09-12T03:21:27.608822+0000) 2024-09-12T03:21:31.341 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:31 smithi183 podman[72654]: 2024-09-12 03:21:31.074753665 +0000 UTC m=+5.640226009 container died a2c40c7e535b7d93096361c78c863c26ff90d810b076bc47fd3867d328aec6ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-12T03:21:31.690 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:31 smithi183 podman[72654]: 2024-09-12 03:21:31.43171168 +0000 UTC m=+5.997184022 container cleanup a2c40c7e535b7d93096361c78c863c26ff90d810b076bc47fd3867d328aec6ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0) 2024-09-12T03:21:31.690 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:31 smithi183 bash[72654]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11 2024-09-12T03:21:31.953 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:31 smithi183 podman[72669]: 2024-09-12 03:21:31.690943392 +0000 UTC m=+0.615512897 container remove a2c40c7e535b7d93096361c78c863c26ff90d810b076bc47fd3867d328aec6ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-12T03:21:32.302 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:32 smithi183 podman[72758]: 2024-09-12 03:21:31.902576197 +0000 UTC m=+0.015606712 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:21:32.302 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:32 smithi183 podman[72758]: 2024-09-12 03:21:32.000167149 +0000 UTC m=+0.113197675 container create b1c7dc85a43675d57ab601cf903fa23487dd7688bbc3d855e17c3d0e47162f3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3) 2024-09-12T03:21:32.302 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:32 smithi183 podman[72758]: 2024-09-12 03:21:32.219542731 +0000 UTC m=+0.332573236 container init b1c7dc85a43675d57ab601cf903fa23487dd7688bbc3d855e17c3d0e47162f3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:21:32.302 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:32 smithi183 podman[72758]: 2024-09-12 03:21:32.222911813 +0000 UTC m=+0.335942320 container start b1c7dc85a43675d57ab601cf903fa23487dd7688bbc3d855e17c3d0e47162f3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11-deactivate, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-12T03:21:32.302 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:32 smithi183 podman[72758]: 2024-09-12 03:21:32.302740095 +0000 UTC m=+0.415770600 container attach b1c7dc85a43675d57ab601cf903fa23487dd7688bbc3d855e17c3d0e47162f3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-12T03:21:32.696 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 12 03:21:32 smithi183 podman[72758]: 2024-09-12 03:21:32.411104598 +0000 UTC m=+0.524135104 container died b1c7dc85a43675d57ab601cf903fa23487dd7688bbc3d855e17c3d0e47162f3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:21:32.973 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@osd.11.service' 2024-09-12T03:21:33.011 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:21:33.011 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-12T03:21:33.011 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-12T03:21:33.011 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.a 2024-09-12T03:21:33.302 INFO:journalctl@ceph.mds.a.smithi116.stdout:Sep 12 03:21:33 smithi116 systemd[1]: Stopping Ceph mds.a for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:21:33.802 INFO:journalctl@ceph.mds.a.smithi116.stdout:Sep 12 03:21:33 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-a[72987]: 2024-09-12T03:21:33.306+0000 7f7238124640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:21:33.802 INFO:journalctl@ceph.mds.a.smithi116.stdout:Sep 12 03:21:33 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-a[72987]: 2024-09-12T03:21:33.306+0000 7f7238124640 -1 mds.a *** got signal Terminated *** 2024-09-12T03:21:37.631 INFO:journalctl@ceph.mds.a.smithi116.stdout:Sep 12 03:21:37 smithi116 podman[130094]: 2024-09-12 03:21:37.321279265 +0000 UTC m=+4.110187611 container died d519324b9a1bf92d6198c7de34b57305b1bec4a4bb05a77133dc404713c3c26a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-a, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-12T03:21:38.052 INFO:journalctl@ceph.mds.a.smithi116.stdout:Sep 12 03:21:37 smithi116 podman[130094]: 2024-09-12 03:21:37.814275787 +0000 UTC m=+4.603184132 container cleanup d519324b9a1bf92d6198c7de34b57305b1bec4a4bb05a77133dc404713c3c26a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-12T03:21:38.052 INFO:journalctl@ceph.mds.a.smithi116.stdout:Sep 12 03:21:37 smithi116 bash[130094]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-a 2024-09-12T03:21:38.192 DEBUG:teuthology.orchestra.run.smithi116:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.a.service' 2024-09-12T03:21:38.226 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:21:38.226 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-12T03:21:38.227 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-12T03:21:38.227 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.d 2024-09-12T03:21:38.802 INFO:journalctl@ceph.mds.d.smithi116.stdout:Sep 12 03:21:38 smithi116 systemd[1]: Stopping Ceph mds.d for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:21:38.803 INFO:journalctl@ceph.mds.d.smithi116.stdout:Sep 12 03:21:38 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-d[71802]: 2024-09-12T03:21:38.553+0000 7fd10cc30640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:21:38.803 INFO:journalctl@ceph.mds.d.smithi116.stdout:Sep 12 03:21:38 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-d[71802]: 2024-09-12T03:21:38.553+0000 7fd10cc30640 -1 mds.d *** got signal Terminated *** 2024-09-12T03:21:42.994 INFO:journalctl@ceph.mds.d.smithi116.stdout:Sep 12 03:21:42 smithi116 podman[130215]: 2024-09-12 03:21:42.56151636 +0000 UTC m=+4.106779189 container died f7f6d74490c1f932c47b7764c3eaa66e0982854a787add1eec1b263893224000 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-12T03:21:43.302 INFO:journalctl@ceph.mds.d.smithi116.stdout:Sep 12 03:21:42 smithi116 podman[130215]: 2024-09-12 03:21:42.994972264 +0000 UTC m=+4.540235093 container cleanup f7f6d74490c1f932c47b7764c3eaa66e0982854a787add1eec1b263893224000 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2) 2024-09-12T03:21:43.302 INFO:journalctl@ceph.mds.d.smithi116.stdout:Sep 12 03:21:42 smithi116 bash[130215]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-d 2024-09-12T03:21:43.302 INFO:journalctl@ceph.mds.d.smithi116.stdout:Sep 12 03:21:43 smithi116 podman[130227]: 2024-09-12 03:21:43.179624444 +0000 UTC m=+0.609333950 container remove f7f6d74490c1f932c47b7764c3eaa66e0982854a787add1eec1b263893224000 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-d, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-12T03:21:43.419 DEBUG:teuthology.orchestra.run.smithi116:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.d.service' 2024-09-12T03:21:43.452 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:21:43.452 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-12T03:21:43.452 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-12T03:21:43.452 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.g 2024-09-12T03:21:43.773 INFO:journalctl@ceph.mds.g.smithi116.stdout:Sep 12 03:21:43 smithi116 systemd[1]: Stopping Ceph mds.g for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:21:44.052 INFO:journalctl@ceph.mds.g.smithi116.stdout:Sep 12 03:21:43 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-g[72602]: 2024-09-12T03:21:43.773+0000 7fe96fbde640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:21:44.052 INFO:journalctl@ceph.mds.g.smithi116.stdout:Sep 12 03:21:43 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-g[72602]: 2024-09-12T03:21:43.773+0000 7fe96fbde640 -1 mds.g *** got signal Terminated *** 2024-09-12T03:21:46.856 INFO:journalctl@ceph.mds.g.smithi116.stdout:Sep 12 03:21:46 smithi116 podman[130331]: 2024-09-12 03:21:46.570989349 +0000 UTC m=+2.900455098 container died 199e293b829746f3c5a951a22905b7a8efc358d109b08e206a5e0db09e14f5b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-g, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:21:47.290 INFO:journalctl@ceph.mds.g.smithi116.stdout:Sep 12 03:21:47 smithi116 podman[130331]: 2024-09-12 03:21:47.023323364 +0000 UTC m=+3.352789154 container cleanup 199e293b829746f3c5a951a22905b7a8efc358d109b08e206a5e0db09e14f5b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-12T03:21:47.290 INFO:journalctl@ceph.mds.g.smithi116.stdout:Sep 12 03:21:47 smithi116 bash[130331]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-g 2024-09-12T03:21:47.445 DEBUG:teuthology.orchestra.run.smithi116:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.g.service' 2024-09-12T03:21:47.477 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:21:47.478 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-12T03:21:47.478 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-12T03:21:47.478 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.j 2024-09-12T03:21:48.052 INFO:journalctl@ceph.mds.j.smithi116.stdout:Sep 12 03:21:47 smithi116 systemd[1]: Stopping Ceph mds.j for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:21:48.052 INFO:journalctl@ceph.mds.j.smithi116.stdout:Sep 12 03:21:47 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-j[71406]: 2024-09-12T03:21:47.793+0000 7f00fad2a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:21:48.052 INFO:journalctl@ceph.mds.j.smithi116.stdout:Sep 12 03:21:47 smithi116 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-j[71406]: 2024-09-12T03:21:47.793+0000 7f00fad2a640 -1 mds.j *** got signal Terminated *** 2024-09-12T03:21:51.451 INFO:journalctl@ceph.mds.j.smithi116.stdout:Sep 12 03:21:51 smithi116 podman[130451]: 2024-09-12 03:21:51.070820063 +0000 UTC m=+3.366507159 container died 5b19ae306a97636689f4aae1bd9988fc9d4a71f505e5577e3d847e149bc7211b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-j, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2) 2024-09-12T03:21:51.452 INFO:journalctl@ceph.mds.j.smithi116.stdout:Sep 12 03:21:51 smithi116 podman[130451]: 2024-09-12 03:21:51.452220968 +0000 UTC m=+3.747908095 container cleanup 5b19ae306a97636689f4aae1bd9988fc9d4a71f505e5577e3d847e149bc7211b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-j, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-12T03:21:51.753 INFO:journalctl@ceph.mds.j.smithi116.stdout:Sep 12 03:21:51 smithi116 bash[130451]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-j 2024-09-12T03:21:51.924 DEBUG:teuthology.orchestra.run.smithi116:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.j.service' 2024-09-12T03:21:51.955 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:21:51.955 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-12T03:21:51.955 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-12T03:21:51.955 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.b 2024-09-12T03:21:52.244 INFO:journalctl@ceph.mds.b.smithi177.stdout:Sep 12 03:21:52 smithi177 systemd[1]: Stopping Ceph mds.b for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:21:52.744 INFO:journalctl@ceph.mds.b.smithi177.stdout:Sep 12 03:21:52 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-b[65339]: 2024-09-12T03:21:52.258+0000 7fed7d2a3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:21:52.744 INFO:journalctl@ceph.mds.b.smithi177.stdout:Sep 12 03:21:52 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-b[65339]: 2024-09-12T03:21:52.258+0000 7fed7d2a3640 -1 mds.b *** got signal Terminated *** 2024-09-12T03:21:55.201 INFO:journalctl@ceph.mds.b.smithi177.stdout:Sep 12 03:21:54 smithi177 podman[71385]: 2024-09-12 03:21:54.767439485 +0000 UTC m=+2.620603077 container died 14a73268da50e4c4e77ed332f29012e459761332e3d5a1472e38dc9acac2eb24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:21:55.483 INFO:journalctl@ceph.mds.b.smithi177.stdout:Sep 12 03:21:55 smithi177 podman[71385]: 2024-09-12 03:21:55.2918312 +0000 UTC m=+3.144994824 container cleanup 14a73268da50e4c4e77ed332f29012e459761332e3d5a1472e38dc9acac2eb24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-12T03:21:55.484 INFO:journalctl@ceph.mds.b.smithi177.stdout:Sep 12 03:21:55 smithi177 bash[71385]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-b 2024-09-12T03:21:55.716 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.b.service' 2024-09-12T03:21:55.738 INFO:journalctl@ceph.mds.b.smithi177.stdout:Sep 12 03:21:55 smithi177 podman[71398]: 2024-09-12 03:21:55.484439852 +0000 UTC m=+0.718380893 container remove 14a73268da50e4c4e77ed332f29012e459761332e3d5a1472e38dc9acac2eb24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, ceph=True) 2024-09-12T03:21:55.738 INFO:journalctl@ceph.mds.b.smithi177.stdout:Sep 12 03:21:55 smithi177 systemd[1]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.b.service: Deactivated successfully. 2024-09-12T03:21:55.738 INFO:journalctl@ceph.mds.b.smithi177.stdout:Sep 12 03:21:55 smithi177 systemd[1]: Stopped Ceph mds.b for eb1ac602-70b1-11ef-bcea-c7b262605968. 2024-09-12T03:21:55.738 INFO:journalctl@ceph.mds.b.smithi177.stdout:Sep 12 03:21:55 smithi177 systemd[1]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.b.service: Consumed 8.726s CPU time. 2024-09-12T03:21:55.752 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:21:55.752 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-12T03:21:55.752 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-12T03:21:55.752 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.e 2024-09-12T03:21:55.994 INFO:journalctl@ceph.mds.e.smithi177.stdout:Sep 12 03:21:55 smithi177 systemd[1]: Stopping Ceph mds.e for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:21:56.495 INFO:journalctl@ceph.mds.e.smithi177.stdout:Sep 12 03:21:56 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-e[64726]: 2024-09-12T03:21:56.077+0000 7f0392def640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:21:56.495 INFO:journalctl@ceph.mds.e.smithi177.stdout:Sep 12 03:21:56 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-e[64726]: 2024-09-12T03:21:56.077+0000 7f0392def640 -1 mds.e *** got signal Terminated *** 2024-09-12T03:21:59.040 INFO:journalctl@ceph.mds.e.smithi177.stdout:Sep 12 03:21:58 smithi177 podman[71506]: 2024-09-12 03:21:58.761421309 +0000 UTC m=+2.792322843 container died 735c3254bcdd08ecc6ffacf4b1af20e25214483c31229bbd04aebb4bd8c5a430 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-e, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:21:59.420 INFO:journalctl@ceph.mds.e.smithi177.stdout:Sep 12 03:21:59 smithi177 podman[71506]: 2024-09-12 03:21:59.18694177 +0000 UTC m=+3.217843305 container cleanup 735c3254bcdd08ecc6ffacf4b1af20e25214483c31229bbd04aebb4bd8c5a430 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731) 2024-09-12T03:21:59.420 INFO:journalctl@ceph.mds.e.smithi177.stdout:Sep 12 03:21:59 smithi177 bash[71506]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-e 2024-09-12T03:21:59.551 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.e.service' 2024-09-12T03:21:59.586 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:21:59.587 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-12T03:21:59.587 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-12T03:21:59.587 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.h 2024-09-12T03:21:59.994 INFO:journalctl@ceph.mds.h.smithi177.stdout:Sep 12 03:21:59 smithi177 systemd[1]: Stopping Ceph mds.h for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:21:59.995 INFO:journalctl@ceph.mds.h.smithi177.stdout:Sep 12 03:21:59 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-h[65028]: 2024-09-12T03:21:59.915+0000 7fc25f202640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:21:59.995 INFO:journalctl@ceph.mds.h.smithi177.stdout:Sep 12 03:21:59 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-h[65028]: 2024-09-12T03:21:59.915+0000 7fc25f202640 -1 mds.h *** got signal Terminated *** 2024-09-12T03:22:03.115 INFO:journalctl@ceph.mds.h.smithi177.stdout:Sep 12 03:22:02 smithi177 podman[71639]: 2024-09-12 03:22:02.763158954 +0000 UTC m=+2.955659610 container died 0aabcdea56ce5275b17ac77b009dcb5884a1fea5be9c05803f960a0411750cd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-h, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:22:03.494 INFO:journalctl@ceph.mds.h.smithi177.stdout:Sep 12 03:22:03 smithi177 podman[71639]: 2024-09-12 03:22:03.214933114 +0000 UTC m=+3.407433769 container cleanup 0aabcdea56ce5275b17ac77b009dcb5884a1fea5be9c05803f960a0411750cd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-h, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-12T03:22:03.495 INFO:journalctl@ceph.mds.h.smithi177.stdout:Sep 12 03:22:03 smithi177 bash[71639]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-h 2024-09-12T03:22:03.689 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.h.service' 2024-09-12T03:22:03.723 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:22:03.723 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-12T03:22:03.723 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-12T03:22:03.724 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.k 2024-09-12T03:22:04.076 INFO:journalctl@ceph.mds.k.smithi177.stdout:Sep 12 03:22:03 smithi177 systemd[1]: Stopping Ceph mds.k for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:22:04.494 INFO:journalctl@ceph.mds.k.smithi177.stdout:Sep 12 03:22:04 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-k[64414]: 2024-09-12T03:22:04.076+0000 7fc00accd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:22:04.494 INFO:journalctl@ceph.mds.k.smithi177.stdout:Sep 12 03:22:04 smithi177 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-k[64414]: 2024-09-12T03:22:04.076+0000 7fc00accd640 -1 mds.k *** got signal Terminated *** 2024-09-12T03:22:08.160 INFO:journalctl@ceph.mds.k.smithi177.stdout:Sep 12 03:22:07 smithi177 podman[71760]: 2024-09-12 03:22:07.797692921 +0000 UTC m=+3.848866898 container died 2d7de1207a651987336e01a598bf3f177206ec3a2e8010970c7ec41430160ad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-k, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-12T03:22:08.494 INFO:journalctl@ceph.mds.k.smithi177.stdout:Sep 12 03:22:08 smithi177 podman[71760]: 2024-09-12 03:22:08.242961468 +0000 UTC m=+4.294135447 container cleanup 2d7de1207a651987336e01a598bf3f177206ec3a2e8010970c7ec41430160ad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-k, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-12T03:22:08.494 INFO:journalctl@ceph.mds.k.smithi177.stdout:Sep 12 03:22:08 smithi177 bash[71760]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-k 2024-09-12T03:22:08.665 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.k.service' 2024-09-12T03:22:08.697 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:22:08.697 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-12T03:22:08.698 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-12T03:22:08.698 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.c 2024-09-12T03:22:08.987 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 03:22:08 smithi183 systemd[1]: Stopping Ceph mds.c for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:22:09.453 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 03:22:08 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-c[65771]: 2024-09-12T03:22:08.986+0000 7efe3d780640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:22:09.453 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 03:22:08 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-c[65771]: 2024-09-12T03:22:08.986+0000 7efe3d780640 -1 mds.c *** got signal Terminated *** 2024-09-12T03:22:11.062 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 03:22:10 smithi183 podman[72859]: 2024-09-12 03:22:10.779770562 +0000 UTC m=+1.889254472 container died 7fbbfbd37f266079448dfa64480626c73ce7935d437e3a9e10357d79f69ef43a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-c, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-12T03:22:11.370 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 03:22:11 smithi183 podman[72859]: 2024-09-12 03:22:11.162118358 +0000 UTC m=+2.271602278 container cleanup 7fbbfbd37f266079448dfa64480626c73ce7935d437e3a9e10357d79f69ef43a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-c, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3) 2024-09-12T03:22:11.371 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 12 03:22:11 smithi183 bash[72859]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-c 2024-09-12T03:22:11.505 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.c.service' 2024-09-12T03:22:11.538 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:22:11.538 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-12T03:22:11.539 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-12T03:22:11.539 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.f 2024-09-12T03:22:11.702 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 03:22:11 smithi183 systemd[1]: Stopping Ceph mds.f for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:22:12.203 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 03:22:11 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-f[65466]: 2024-09-12T03:22:11.857+0000 7f9a1e2b2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:22:12.203 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 03:22:11 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-f[65466]: 2024-09-12T03:22:11.857+0000 7f9a1e2b2640 -1 mds.f *** got signal Terminated *** 2024-09-12T03:22:15.041 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 03:22:14 smithi183 podman[72979]: 2024-09-12 03:22:14.769275435 +0000 UTC m=+3.011825156 container died 29b50457e05dbcb2df5fa716d0710cf11c0f8f51904de08a825cb0b5ffe2c604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-f, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-12T03:22:15.374 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 03:22:15 smithi183 podman[72979]: 2024-09-12 03:22:15.115796807 +0000 UTC m=+3.358346524 container cleanup 29b50457e05dbcb2df5fa716d0710cf11c0f8f51904de08a825cb0b5ffe2c604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-12T03:22:15.374 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 12 03:22:15 smithi183 bash[72979]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-f 2024-09-12T03:22:15.546 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.f.service' 2024-09-12T03:22:15.581 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:22:15.581 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-12T03:22:15.581 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-12T03:22:15.581 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.i 2024-09-12T03:22:15.894 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 03:22:15 smithi183 systemd[1]: Stopping Ceph mds.i for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:22:16.202 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 03:22:15 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-i[66074]: 2024-09-12T03:22:15.893+0000 7f0b31e5e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:22:16.203 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 03:22:15 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-i[66074]: 2024-09-12T03:22:15.893+0000 7f0b31e5e640 -1 mds.i *** got signal Terminated *** 2024-09-12T03:22:19.027 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 03:22:18 smithi183 podman[73100]: 2024-09-12 03:22:18.769760923 +0000 UTC m=+2.966804135 container died a6a2f21bf45e4df9f25767643bbb3b53a9182e7c1d3c1d1513ecdfcdd3a0e627 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-i, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:22:19.295 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 03:22:19 smithi183 podman[73100]: 2024-09-12 03:22:19.111128435 +0000 UTC m=+3.308171646 container cleanup a6a2f21bf45e4df9f25767643bbb3b53a9182e7c1d3c1d1513ecdfcdd3a0e627 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3) 2024-09-12T03:22:19.295 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 03:22:19 smithi183 bash[73100]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-i 2024-09-12T03:22:19.529 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.i.service' 2024-09-12T03:22:19.550 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 03:22:19 smithi183 podman[73114]: 2024-09-12 03:22:19.295648795 +0000 UTC m=+0.518941265 container remove a6a2f21bf45e4df9f25767643bbb3b53a9182e7c1d3c1d1513ecdfcdd3a0e627 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-i, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:22:19.550 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 03:22:19 smithi183 systemd[1]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.i.service: Deactivated successfully. 2024-09-12T03:22:19.550 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 03:22:19 smithi183 systemd[1]: Stopped Ceph mds.i for eb1ac602-70b1-11ef-bcea-c7b262605968. 2024-09-12T03:22:19.550 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 12 03:22:19 smithi183 systemd[1]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.i.service: Consumed 2.294s CPU time. 2024-09-12T03:22:19.561 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:22:19.561 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-12T03:22:19.561 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-12T03:22:19.561 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.l 2024-09-12T03:22:19.881 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 03:22:19 smithi183 systemd[1]: Stopping Ceph mds.l for eb1ac602-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:22:20.202 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 03:22:19 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-l[65167]: 2024-09-12T03:22:19.880+0000 7f3c0b687640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:22:20.203 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 03:22:19 smithi183 ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-l[65167]: 2024-09-12T03:22:19.880+0000 7f3c0b687640 -1 mds.l *** got signal Terminated *** 2024-09-12T03:22:21.423 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 03:22:21 smithi183 podman[73223]: 2024-09-12 03:22:21.172010662 +0000 UTC m=+1.394951061 container died 7d1a29e0bc14897f7cd1e6bb082e0c98a954f1782577e602bb1e8619166a4ebb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-l, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, ceph=True) 2024-09-12T03:22:21.703 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 03:22:21 smithi183 podman[73223]: 2024-09-12 03:22:21.514932265 +0000 UTC m=+1.737872675 container cleanup 7d1a29e0bc14897f7cd1e6bb082e0c98a954f1782577e602bb1e8619166a4ebb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-l, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-12T03:22:21.703 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 12 03:22:21 smithi183 bash[73223]: ceph-eb1ac602-70b1-11ef-bcea-c7b262605968-mds-l 2024-09-12T03:22:21.933 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb1ac602-70b1-11ef-bcea-c7b262605968@mds.l.service' 2024-09-12T03:22:21.964 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:22:21.964 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-12T03:22:21.965 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T03:22:22.166 INFO:teuthology.orchestra.run.smithi116.stdout:Deleting cluster with fsid: eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T03:22:24.740 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T03:22:24.941 INFO:teuthology.orchestra.run.smithi177.stdout:Deleting cluster with fsid: eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T03:22:27.448 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T03:22:27.648 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T03:22:30.020 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:22:30.054 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:22:30.087 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:22:30.114 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-12T03:22:30.117 DEBUG:teuthology.misc:Transferring archived files from smithi116:/var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901644/remote/smithi116/crash 2024-09-12T03:22:30.117 DEBUG:teuthology.orchestra.run.smithi116:> sudo tar c -f - -C /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/crash -- . 2024-09-12T03:22:30.377 INFO:teuthology.orchestra.run.smithi116.stderr:tar: /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T03:22:30.378 INFO:teuthology.orchestra.run.smithi116.stderr:tar: Error is not recoverable: exiting now 2024-09-12T03:22:30.380 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901644/remote/smithi177/crash 2024-09-12T03:22:30.381 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/crash -- . 2024-09-12T03:22:30.450 INFO:teuthology.orchestra.run.smithi177.stderr:tar: /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T03:22:30.450 INFO:teuthology.orchestra.run.smithi177.stderr:tar: Error is not recoverable: exiting now 2024-09-12T03:22:30.452 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901644/remote/smithi183/crash 2024-09-12T03:22:30.453 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/crash -- . 2024-09-12T03:22:30.497 INFO:teuthology.orchestra.run.smithi183.stderr:tar: /var/lib/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T03:22:30.497 INFO:teuthology.orchestra.run.smithi183.stderr:tar: Error is not recoverable: exiting now 2024-09-12T03:22:30.498 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-12T03:22:30.498 DEBUG:teuthology.orchestra.run.smithi116:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-12T03:22:30.541 INFO:tasks.cephadm:Compressing logs... 2024-09-12T03:22:30.541 DEBUG:teuthology.orchestra.run.smithi116:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T03:22:30.585 DEBUG:teuthology.orchestra.run.smithi177:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T03:22:30.587 DEBUG:teuthology.orchestra.run.smithi183:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T03:22:30.609 INFO:teuthology.orchestra.run.smithi116.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-12T03:22:30.609 INFO:teuthology.orchestra.run.smithi116.stderr:: No such file or directory 2024-09-12T03:22:30.609 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-12T03:22:30.611 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T03:22:30.611 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.0.log: /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T03:22:30.611 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-12T03:22:30.611 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T03:22:30.611 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-12T03:22:30.612 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-12T03:22:30.613 INFO:teuthology.orchestra.run.smithi183.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T03:22:30.613 INFO:teuthology.orchestra.run.smithi183.stderr:: No such file or directory 2024-09-12T03:22:30.613 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T03:22:30.613 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-12T03:22:30.613 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-volume.log: /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-12T03:22:30.613 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T03:22:30.613 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-12T03:22:30.616 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mon.c.log: /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-12T03:22:30.616 INFO:teuthology.orchestra.run.smithi183.stderr: 83.1% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T03:22:30.616 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-12T03:22:30.616 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph.log 2024-09-12T03:22:30.616 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-12T03:22:30.616 INFO:teuthology.orchestra.run.smithi183.stderr: 90.5% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T03:22:30.616 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-12T03:22:30.618 INFO:teuthology.orchestra.run.smithi177.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-12T03:22:30.618 INFO:teuthology.orchestra.run.smithi177.stderr:: No such file or directory 2024-09-12T03:22:30.619 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-12T03:22:30.619 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-12T03:22:30.619 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T03:22:30.619 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-12T03:22:30.619 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-12T03:22:30.619 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-12T03:22:30.619 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T03:22:30.619 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T03:22:30.620 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.l.log: 88.0% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T03:22:30.620 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-12T03:22:30.621 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mon.a.log: 82.4% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T03:22:30.621 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.g.log: 90.3% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T03:22:30.621 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-12T03:22:30.621 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph.log 2024-09-12T03:22:30.621 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-12T03:22:30.621 INFO:teuthology.orchestra.run.smithi116.stderr: 94.7% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-12T03:22:30.621 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-12T03:22:30.621 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.d.log: 94.7% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-12T03:22:30.622 INFO:teuthology.orchestra.run.smithi116.stderr: 87.9% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T03:22:30.622 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-12T03:22:30.622 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-12T03:22:30.623 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.j.log: 94.7% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-12T03:22:30.623 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.8.log: 94.7% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-12T03:22:30.624 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-12T03:22:30.626 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph.audit.log: 90.5% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T03:22:30.626 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-12T03:22:30.626 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph.cephadm.log: 83.1% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T03:22:30.626 INFO:teuthology.orchestra.run.smithi177.stderr: 94.5% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-12T03:22:30.626 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph.log 2024-09-12T03:22:30.626 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-12T03:22:30.628 INFO:teuthology.orchestra.run.smithi116.stderr: 91.1% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T03:22:30.628 INFO:teuthology.orchestra.run.smithi116.stderr: 94.8% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-12T03:22:30.628 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T03:22:30.629 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph.log: /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mgr.y.log: 88.0% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T03:22:30.629 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-12T03:22:30.629 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mgr.z.log: 90.9% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T03:22:30.629 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-12T03:22:30.630 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.h.log: 91.0% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T03:22:30.631 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.c.log: 93.2% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-12T03:22:30.631 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T03:22:30.631 INFO:teuthology.orchestra.run.smithi177.stderr: 93.1% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-12T03:22:30.632 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-12T03:22:30.633 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T03:22:30.640 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T03:22:30.653 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T03:22:30.654 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mon.b.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T03:22:30.689 INFO:teuthology.orchestra.run.smithi183.stderr: 93.1% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-12T03:22:30.706 INFO:teuthology.orchestra.run.smithi183.stderr: 94.6% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-12T03:22:30.719 INFO:teuthology.orchestra.run.smithi177.stderr: 94.7% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-12T03:22:30.719 INFO:teuthology.orchestra.run.smithi177.stderr: 93.0% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-12T03:22:30.920 INFO:teuthology.orchestra.run.smithi183.stderr: 94.2% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-12T03:22:30.957 INFO:teuthology.orchestra.run.smithi116.stderr: 90.1% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-12T03:22:31.013 INFO:teuthology.orchestra.run.smithi183.stderr: 91.7% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-12T03:22:31.214 INFO:teuthology.orchestra.run.smithi177.stderr: 91.7% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-12T03:22:31.323 INFO:teuthology.orchestra.run.smithi183.stderr: 92.9% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-12T03:22:31.495 INFO:teuthology.orchestra.run.smithi116.stderr: 92.4% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-12T03:22:31.561 INFO:teuthology.orchestra.run.smithi177.stderr: 92.2% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-12T03:22:31.662 INFO:teuthology.orchestra.run.smithi116.stderr: 90.3% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-12T03:22:31.684 INFO:teuthology.orchestra.run.smithi116.stderr: 92.4% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-12T03:22:31.734 INFO:teuthology.orchestra.run.smithi183.stderr: 93.1% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-12T03:22:31.771 INFO:teuthology.orchestra.run.smithi116.stderr: 92.9% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-12T03:22:31.799 INFO:teuthology.orchestra.run.smithi116.stderr: 93.3% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-12T03:22:31.801 INFO:teuthology.orchestra.run.smithi116.stderr: 2024-09-12T03:22:31.801 INFO:teuthology.orchestra.run.smithi116.stderr:real 0m1.202s 2024-09-12T03:22:31.801 INFO:teuthology.orchestra.run.smithi116.stderr:user 0m5.523s 2024-09-12T03:22:31.801 INFO:teuthology.orchestra.run.smithi116.stderr:sys 0m0.231s 2024-09-12T03:22:32.268 INFO:teuthology.orchestra.run.smithi177.stderr: 92.2% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-12T03:22:32.411 INFO:teuthology.orchestra.run.smithi183.stderr: 92.4% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-12T03:22:32.525 INFO:teuthology.orchestra.run.smithi177.stderr: 93.0% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-12T03:22:32.548 INFO:teuthology.orchestra.run.smithi177.stderr: 92.3% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-12T03:22:32.656 INFO:teuthology.orchestra.run.smithi177.stderr: 92.1% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-12T03:22:32.657 INFO:teuthology.orchestra.run.smithi177.stderr: 2024-09-12T03:22:32.658 INFO:teuthology.orchestra.run.smithi177.stderr:real 0m2.055s 2024-09-12T03:22:32.658 INFO:teuthology.orchestra.run.smithi177.stderr:user 0m8.952s 2024-09-12T03:22:32.658 INFO:teuthology.orchestra.run.smithi177.stderr:sys 0m0.343s 2024-09-12T03:22:32.880 INFO:teuthology.orchestra.run.smithi183.stderr: 91.8% -- replaced with /var/log/ceph/eb1ac602-70b1-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-12T03:22:32.882 INFO:teuthology.orchestra.run.smithi183.stderr: 2024-09-12T03:22:32.882 INFO:teuthology.orchestra.run.smithi183.stderr:real 0m2.279s 2024-09-12T03:22:32.882 INFO:teuthology.orchestra.run.smithi183.stderr:user 0m6.537s 2024-09-12T03:22:32.882 INFO:teuthology.orchestra.run.smithi183.stderr:sys 0m0.243s 2024-09-12T03:22:32.882 INFO:tasks.cephadm:Archiving logs... 2024-09-12T03:22:32.882 DEBUG:teuthology.misc:Transferring archived files from smithi116:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901644/remote/smithi116/log 2024-09-12T03:22:32.883 DEBUG:teuthology.orchestra.run.smithi116:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T03:22:33.788 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901644/remote/smithi177/log 2024-09-12T03:22:33.789 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T03:22:35.162 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901644/remote/smithi183/log 2024-09-12T03:22:35.163 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T03:22:36.169 INFO:tasks.cephadm:Removing cluster... 2024-09-12T03:22:36.169 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 --force 2024-09-12T03:22:36.371 INFO:teuthology.orchestra.run.smithi116.stdout:Deleting cluster with fsid: eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T03:22:36.628 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 --force 2024-09-12T03:22:36.816 INFO:teuthology.orchestra.run.smithi177.stdout:Deleting cluster with fsid: eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T03:22:37.080 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid eb1ac602-70b1-11ef-bcea-c7b262605968 --force 2024-09-12T03:22:37.279 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: eb1ac602-70b1-11ef-bcea-c7b262605968 2024-09-12T03:22:37.540 INFO:tasks.cephadm:Removing cephadm ... 2024-09-12T03:22:37.540 DEBUG:teuthology.orchestra.run.smithi116:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T03:22:37.559 DEBUG:teuthology.orchestra.run.smithi177:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T03:22:37.576 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T03:22:37.592 INFO:tasks.cephadm:Teardown complete 2024-09-12T03:22:37.592 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-12T03:22:37.603 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-12T03:22:37.603 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T03:22:37.606 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T03:22:37.618 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T03:22:37.758 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-12T03:22:37.770 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-12T03:22:37.771 DEBUG:teuthology.orchestra.run.smithi116:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:22:37.774 DEBUG:teuthology.orchestra.run.smithi177:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:22:37.777 DEBUG:teuthology.orchestra.run.smithi183:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:22:37.788 INFO:teuthology.orchestra.run.smithi116.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:22:37.789 INFO:teuthology.orchestra.run.smithi177.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:22:37.792 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:22:37.793 INFO:teuthology.orchestra.run.smithi116.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:22:37.793 INFO:teuthology.orchestra.run.smithi116.stdout:=============================================================================== 2024-09-12T03:22:37.794 INFO:teuthology.orchestra.run.smithi116.stdout:^* hv01.front.sepia.ceph.com 3 6 377 57 -3034us[-3026us] +/- 43ms 2024-09-12T03:22:37.794 INFO:teuthology.orchestra.run.smithi116.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 56 +1758us[+1758us] +/- 58ms 2024-09-12T03:22:37.794 INFO:teuthology.orchestra.run.smithi116.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 53 +2603us[+2603us] +/- 68ms 2024-09-12T03:22:37.794 INFO:teuthology.orchestra.run.smithi116.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:22:37.795 INFO:teuthology.orchestra.run.smithi177.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:22:37.795 INFO:teuthology.orchestra.run.smithi177.stdout:=============================================================================== 2024-09-12T03:22:37.795 INFO:teuthology.orchestra.run.smithi177.stdout:^* hv01.front.sepia.ceph.com 3 6 377 56 -3027us[-3030us] +/- 43ms 2024-09-12T03:22:37.795 INFO:teuthology.orchestra.run.smithi177.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 55 +1772us[+1772us] +/- 58ms 2024-09-12T03:22:37.795 INFO:teuthology.orchestra.run.smithi177.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 56 +2620us[+2620us] +/- 68ms 2024-09-12T03:22:37.795 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:22:37.796 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:22:37.796 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-09-12T03:22:37.796 INFO:teuthology.orchestra.run.smithi183.stdout:^* hv01.front.sepia.ceph.com 3 6 377 54 -3028us[-3034us] +/- 43ms 2024-09-12T03:22:37.796 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 50 +1769us[+1769us] +/- 58ms 2024-09-12T03:22:37.796 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 57 +2615us[+2609us] +/- 68ms 2024-09-12T03:22:37.797 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:22:37.797 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-12T03:22:37.811 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-12T03:22:37.811 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-12T03:22:37.827 DEBUG:teuthology.orchestra.run.smithi116:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T03:22:37.927 DEBUG:teuthology.orchestra.run.smithi177:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T03:22:37.994 DEBUG:teuthology.orchestra.run.smithi183:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T03:22:38.062 DEBUG:teuthology.orchestra.run.smithi116:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:22:38.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:22:38.094 DEBUG:teuthology.orchestra.run.smithi177:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:22:38.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:22:38.123 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:22:38.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:22:38.152 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-12T03:22:38.170 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-12T03:22:38.188 INFO:teuthology.task.internal:Duration was 2249.286710 seconds 2024-09-12T03:22:38.188 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-12T03:22:38.208 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-12T03:22:38.208 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T03:22:38.212 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T03:22:38.215 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T03:22:38.250 INFO:teuthology.orchestra.run.smithi116.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:22:38.250 INFO:teuthology.orchestra.run.smithi177.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:22:38.254 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:22:38.608 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-12T03:22:38.609 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi116.front.sepia.ceph.com 2024-09-12T03:22:38.609 DEBUG:teuthology.orchestra.run.smithi116:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T03:22:38.671 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi177.front.sepia.ceph.com 2024-09-12T03:22:38.671 DEBUG:teuthology.orchestra.run.smithi177:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T03:22:38.699 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi183.front.sepia.ceph.com 2024-09-12T03:22:38.699 DEBUG:teuthology.orchestra.run.smithi183:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T03:22:38.720 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-12T03:22:38.720 DEBUG:teuthology.orchestra.run.smithi116:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T03:22:38.723 DEBUG:teuthology.orchestra.run.smithi177:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T03:22:38.741 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T03:22:38.786 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-12T03:22:38.786 DEBUG:teuthology.orchestra.run.smithi116:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T03:22:38.789 DEBUG:teuthology.orchestra.run.smithi177:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T03:22:38.807 DEBUG:teuthology.orchestra.run.smithi183:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T03:22:40.615 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-12T03:22:40.627 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-12T03:22:40.627 DEBUG:teuthology.orchestra.run.smithi116:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T03:22:40.652 DEBUG:teuthology.orchestra.run.smithi177:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T03:22:40.684 DEBUG:teuthology.orchestra.run.smithi183:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T03:22:40.714 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-12T03:22:40.725 DEBUG:teuthology.orchestra.run.smithi116:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:22:40.727 DEBUG:teuthology.orchestra.run.smithi177:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:22:40.730 DEBUG:teuthology.orchestra.run.smithi183:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:22:40.748 INFO:teuthology.orchestra.run.smithi116.stdout:kernel.core_pattern = core 2024-09-12T03:22:40.755 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern = core 2024-09-12T03:22:40.782 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = core 2024-09-12T03:22:40.827 DEBUG:teuthology.orchestra.run.smithi116:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:22:40.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:22:40.842 DEBUG:teuthology.orchestra.run.smithi177:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:22:40.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:22:40.859 DEBUG:teuthology.orchestra.run.smithi183:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:22:40.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:22:40.884 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-12T03:22:40.895 INFO:teuthology.task.internal:Transferring archived files... 2024-09-12T03:22:40.896 DEBUG:teuthology.misc:Transferring archived files from smithi116:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901644/remote/smithi116 2024-09-12T03:22:40.897 DEBUG:teuthology.orchestra.run.smithi116:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T03:22:40.978 DEBUG:teuthology.misc:Transferring archived files from smithi177:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901644/remote/smithi177 2024-09-12T03:22:40.978 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T03:22:41.019 DEBUG:teuthology.misc:Transferring archived files from smithi183:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901644/remote/smithi183 2024-09-12T03:22:41.019 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T03:22:41.059 INFO:teuthology.task.internal:Removing archive directory... 2024-09-12T03:22:41.059 DEBUG:teuthology.orchestra.run.smithi116:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T03:22:41.061 DEBUG:teuthology.orchestra.run.smithi177:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T03:22:41.064 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T03:22:41.111 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-12T03:22:41.124 INFO:teuthology.task.internal:Not uploading archives. 2024-09-12T03:22:41.124 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-12T03:22:41.134 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-12T03:22:41.135 DEBUG:teuthology.orchestra.run.smithi116:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T03:22:41.137 DEBUG:teuthology.orchestra.run.smithi177:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T03:22:41.139 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T03:22:41.149 INFO:teuthology.orchestra.run.smithi116.stdout: 266063 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 03:22 /home/ubuntu/cephtest 2024-09-12T03:22:41.153 INFO:teuthology.orchestra.run.smithi177.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 03:22 /home/ubuntu/cephtest 2024-09-12T03:22:41.169 INFO:teuthology.orchestra.run.smithi183.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 03:22 /home/ubuntu/cephtest 2024-09-12T03:22:41.171 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-12T03:22:41.183 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-12T03:22:41.213 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} duration: 2249.2867097854614 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-12T03:22:41.214 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T03:22:41.313 INFO:teuthology.run:pass